WorksPhysics and Astronomy
Quantum natural gradient without monotonicity
Abstract
Natural gradient (NG) is an information-geometric optimization method that plays a crucial role, especially in the estimation of parameters for machine learning models like neural networks. To apply NG to quantum systems, the quantum natural gradient (QNG) was introduced and utilized for noisy intermediate-scale devices. Additionally, a mathematically equivalent approach to QNG, known as the stochastic reconfiguration method, has been implemented to enhance the performance of quantum Monte Carlo methods. It is worth noting that these methods are based on the symmetric logarithmic derivative (SLD) metric, which is one of the monotone metrics. So far, monotonicity has been believed to be a guiding principle to construct a geometry in physics. In this paper, we propose generalized QNG by removing the condition of monotonicity. Initially, we demonstrate that monotonicity is a crucial condition for conventional QNG to be optimal. Subsequently, we provide analytical and numerical evidence showing that non-monotone QNG outperforms conventional QNG based on the SLD metric in terms of convergence speed.
Introduction
Continuous optimization is prevalent in scientific and industrial problem-solving, with a plethora of methods proposed [1]. Natural gradient (NG) stands out as a novel continuous optimization method rooted in information geometry (IG) [2, 3, 4]. The field of machine learning has gained significant attention recently, with neural networks (NNs) holding paramount importance [5, 6, 7]. NG is frequently employed in the optimization of NNs, demonstrating numerical effectiveness [8].
In the realm of physics, the era of quantum computing has dawned, prompting investigations into the hidden potential of noisy intermediate-scale quantum (NISQ) devices [9]. For NISQ devices, the parameter estimation of quantum circuits is imperative [10, 11, 12]. In addition to quantum circuit learning, estimating parameters of quantum states is essential in various contexts, including the variational Monte Carlo method [13], and tensor networks [14]. Quantum natural gradient (QNG), introduced as the quantum counterpart of NG, has recently garnered attention for its promising performance in applications [15, 16]. A mathematically equivalent method to QNG, known in condensed matter physics as the stochastic reconfiguration (SR) method [17, 18, 19, 20, 21]. Furthermore, quantum metrics find applications in classical machine learning contexts [22, 23, 24, 25]. It is noteworthy that both conventional QNG and the SR method are grounded in the symmetric logarithmic derivative (SLD) metric, which is one of the monotone quantum Fisher metrics and a fundamental concept in quantum information geometry (QIG).
In this paper, we present a generalization of QNG based on QIG. Monotonicity implies that information is not gained via completely positive and trace-preserving (CPTP) maps; so it has been believed to be a guiding principle for constructing a geometry in physics. However, from the viewpoint of optimization, it has not been clear whether it is necessary. Initially, we establish that monotonicity is a prerequisite for the SLD metric to optimize the speed of QNG. Subsequently, we theoretically demonstrate that QNG utilizing non-monotone quantum Fisher metrics exhibits faster performance than its SLD-based counterpart. Furthermore, in practical numerical simulations for both classical and quantum cases, the diagonal approximation of the Fisher metric is often employed to mitigate the computational costs associated with inverting the Fisher metric. We show that even under this approximation, the aforementioned findings remain valid. Lastly, we provide numerical simulations to illustrate that non-monotone quantum Fisher metrics outperform the SLD metric in terms of speed.
Natural gradient
In the derivation of NG, the Fisher metric and Kullback-Leibler (KL) divergence play an essential role. Then, we first explain the Fisher metric. Assume that we have , which is a probability distribution parameterized by . We define tangent vectors as , where for . Then, the Fisher metric between and is defined as
where the e- and the m-representations of at are given, respectively, by
Next, we turn our attention to classical rescaled Rényi divergence [26]. When we explain NG only, KL divergence is sufficient; but, the purpose of this paper is to discuss QNG. Then, we explain classical rescaled Rényi divergence, which is a generalized version of KL divergence. Classical rescaled Rényi divergence is given by
Note that “rescaled” means the additional factor in Eq. (4). Furthermore the rescaled Rényi divergence, Eq. (4), in the limit $\alpha \to 1$ corresponds to the KL divergence [26]`:
Notably, Eqs. (1) and (4) satisfy the following relationship:
where are the tangent vectors acting on . The key observation is that the right-hand side of Eq. (6) do not depend on . As we will see later, the quantum counterpart of Eq. (4) yields different metric contrarily to the classical limit, and this fact will play an important role in QNG.
We here review NG, which is one of the optimization methods for continuous variables motivated by IG [4]. Let us consider the minimization problem of . Then, the dynamics of NG is given by
where is a positive number. The key point of Eq. (6) is the constraint in Eq. (6b) since this term does not exist in conventional optimization methods such as the steepest decent method and Newton’s method. Note that in this paper, we focus on Eq. (4), but we do not need to limit ourselves to Eq. (3). First, we transform Eq. (6b) by using the lowest-order approximation:
where
and is the matrix representation of the Fisher metric induced by . To solve Eq. (9), we use the method of Lagrange’s multiplier [27]. Then the solution of Eq. (9) is rewritten as
Note that we have added a negative sign to decrease the value of by this update. In a practical usage of NG, we may use the following update rule instead of Eq. (10):
where is a positive number.
Quantum natural gradient
To discuss QNG, we explain their quantum counterparts of the Fisher metric, the rescaled Rényi divergence, respectively. Assume that we have , which is a quantum state parameterized by , and its spectral decomposition is given by . We first introduce the quantum extension of the Fisher metric, Eq. (1) [28, 29, 30, 31]. To this end, we define the following linear operator . And we also introduce the nonlinear transformation of :
Using Eq. (12), the quantum counterparts of Eq. (2) at are defined as follows:
where is an operator-monotone function such that
Note that in Eq. (11b) is called the Petz function. Then the quantum Fisher metric is given by
The key point of the quantum Fisher metric, Eq. (17), is its dependence on the operator-monotone function . Equation (12a) implies the classical limit of the quantum Fisher metric goes to the classical Fisher metric without depending on the choice of and Eq. (12b) guarantee for the quantum Fisher metric to be real.
We then consider a quantum extension of Eq. (4). Quantum rescaled sandwiched Rényi divergence is defined as follows [32]:
Note that “sandwiched” means is sandwiched by . In Ref. [2], quantum -divergence, in which is not sandwiched by , is investigated. Rescaled sandwiched Rényi divergence, Eq. (18), becomes quantum KL divergence [33] in the limit : . Introducing the Petz function given by
Eq. (17) and Eq. (18) satisfy the following relation:
where , .
As the quantum counterpart of Eq. (6), let us consider the minimization problem of :
In Eq. (17b), appears in the constraint instead of . By using the lowest-order approximation, Eq. (17b) can be transformed into
where is the quantum Fisher metric induced by ; that is, it can be rewritten as because of Eq. (20). To solve Eq. (17), we employ almost the same technique to that used to derive NG. Then, we reach the same equation in the classical case, Eqs. (10) and (11), but is computed from Eq. (20). Note that we do not need to limit ourselves to Eq. (18) though in this paper we focus on Eq. (18).
Monotonicity
Characterizing metrics and divergences, monotonicity is an essential concept [28, 34, 35]. For example, the monotonicity of quantum KL divergence is considered to be quite natural since it implies that information is not gained via CPTP maps.
We review some important properties of operator functions [34, 35, 36, 37]. We first state the definition of the monotonicity of a function: that is, is called monotone if and only if
where . Metric is called a monotone metric if and only if it satisfies the following relation for arbitrary CPTP map : where is the pushforward associated with . Note that Eq. (19) is operator-monotone if and only if and Eq. (20) is a monotone metric [32].
We define the following partial order on :
For monotone functions, the following theorem holds [28]:
Theorem 1. and are the maximum and the minimum elements with respect to Eq. (25) under the condition of monotonicity, Eq. (24), , and . That is, we have the following relation for any such that the condition of monotonicity, Eq. (24), , and :
where and are the Petz functions for the SLD and rRLD metrics defined, respectively, as , .
We review some important properties of Eq. (19). For simplicity, we substitute into Eq. (19):
Two important theorems on Eq. (27) are shown in Ref. [32]. The first one is as follows [32]:
Theorem 2. Eq. (27) monotonically increases when increased.
Thm. 2 implies that , Eq. (19), is monotonically decreasing with increasing, but is singular. Thus, , Eq. (19), monotonically increases toward .
The second one is as follows [32]:
Theorem 3. Eq. (27) is operator monotone if and only if .
Thm. 3 means that , Eq. (19), is operator monotone for .
In Figure 1, we show , Eq. (19), for several different and highlight the regime of the monotone Petz functions by light cyan. Note that , Eq. (19), with are the Petz functions for the SLD and rRLD metrics, respectively. Furthermore, we also have [32].
Figure 1. , Eq. (19), with . Note that and yield the SLD and rRLD metrics, respectively. The regime of the monotone Petz functions is highlighted by light cyan.
Fisher metrics and speed of QNG
We here explain the relation between the quantum Fisher metrics and the speed of QNG. In general, the following theorem holds:
The proof of Thm. 4 is shown in Sec. A in Appendix A. We define the following diagonal metric:
In partial usages, Eq. (29) is of great importance and often used instead of Eq. (17) because the computation of the inverse matrix of Eq. (17) is numerically costly. Similarly to Thm. 4, the following theorem holds:
Theorem 5. The following relation holds for Eq. (29):
The proof of Thm. 5 is almost the same with Thm. 4. We then discuss the dependence of the speed of QNG on the quantum Fisher metrics. Metrics and satisfy the following relation: . In the case of Eq. (10), we have
And in the case of Eq. (11), we also have
Thus, for both of Eqs. (10) and (11), decrease the value of faster than when .
Numerical simulations
We show numerical simulations to support the main claim of this paper. We begin with the setup of numerical simulations. Using ,
we define the following parameterized quantum state: where , , and . We also define where is the identity operator and . Then, we consider the following minimization problem with respect to :
where . To stabilize the numerical calculations, we conduct the following operations to the density operator and metric: , and where is the identity matrix whose size is identical to that of , and and are tiny positive numbers.
In the following calculations, we set , , and . In Figure 2, we plot the time evolution of the cost function in the case of Eq. (10) and highlight the regime of the monotone Petz functions by light cyan. In Figure 3, we show the time evolution of the cost function in the case of Eq. (11). In both cases, QNG with outperforms SLD-based QNG proposed in Refs. [15, 16]. Furthermore, these figures show that the gaps between different values of becomes large in the case of Eq. (11) than Eq. (10). However, this phenomenon depends on the maximum eigenvalue of the quantum Fisher metric; thus, we have to choose either Eq. (10) or Eq. (11) depending on problems.
Figure 2. Cost functions for several in the case of Eq. (10). We set , , and . The regime of the monotone metrics is highlighted by light cyan.
Figure 3. Cost functions for several in the case of Eq. (11). We set , , and . The regime of the monotone metrics is highlighted by light cyan.
Conclusions
In this paper, we have presented a generalization of QNG based on QIG. Notably, the existing literature on QNG relies on the SLD metric, which is not the exclusive definition of the logarithmic derivative of density operators. In QNG, the e-representation, specified by the Petz function, aligns with the logarithmic derivative of density operators, and it identifies the quantum Fisher metric. In our work, we utilize the non-monotone quantum Fisher metrics to generalize QNG, successfully enhancing the speed of QNG. Monotonicity has been believed to be a necessary condition to construct a geometry in physics; however, we have clarified that it is not necessary; in fact, it should be eliminated for optimization purposes. Our findings have shed light on the fact that we need to be cautious even with mathematical principles to advance science.
Acknowledgments
H.M. was supported by JSPS KAKENHI Grant Number JP23H04489. H.M. thanks Akio Fujiwara, Lei Wang, Yusuke Nomura, and Shunsuke Daimon for fruitful discussions.
Proof of Thm. 4
The proof of Thm. 4 is as follows:
Proof. We show Eq. (28). From Eq. (17), we have
Since , we have
Thus, Eq. (28) is proved.
References
References
- [1]J. Nocedal and S. Wright, Numerical Optimization, Springer Series in Operations Research and Financial Engineering (Springer New York, 2009), ISBN 9781493937110, URL https://books.google.co.jp/books?id=en-YSWEaCAAJ.
- [2]S.-i. Amari and H. Nagaoka, Methods of information geometry, vol. 191 (American Mathematical Soc., 2000).
- [3]S.-i. Amari, Information Geometry and Its Applications, Applied Mathematical Sciences (Springer Japan, 2016), ISBN 9784431559788, URL https://books.google.co.jp/books?id=UkSFCwAAQBAJ.
- [4]S.-I. Amari, Numerical computation 10, 251 (1998).
- [5]C. M. Bishop, Pattern recognition and machine learning (springer, 2006).
- [6]K. Murphy, Probabilistic Machine Learning: An Introduction, Adaptive Computation and Machine Learning series (MIT Press, 2022), ISBN 9780262369305, URL https://books.google.co.jp/books?id=0yYueAAAQBAJ.
- [7]K. Murphy, Probabilistic Machine Learning: Advanced Topics, Adaptive Computation and Machine Learning series (MIT Press, 2023), ISBN 9780262048439, URL https://books.google.co.jp/books?id=8iycEAAAQBAJ.
- [8]J. Martens, The Journal of Machine Learning Research 21, 5776 (2020).
- [9]J. Preskill, Quantum 2, 79 (2018).
- [10]K. Mitarai, M. Negoro, M. Kitagawa, and K. Fujii, Phys. Rev. A 98, 032309 (2018), URL https://link.aps.org/doi/10.1103/PhysRevA.98.032309.DOI
- [11]M. Schuld, A. Bocharov, K. M. Svore, and N. Wiebe, Phys. Rev. A 101, 032308 (2020), URL https://link.aps.org/doi/10.1103/PhysRevA.101.032308.DOI
- [12]H. Miyahara and V. Roychowdhury, Scientific Reports 12, 19520 (2022).
- [13]P. O. Scherer and P. O. Scherer, Computational physics: simulation of classical and quantum systems (Springer, 2017).
- [14]R. Orús, Nature Reviews Physics 1, 538 (2019).
- [15]J. Stokes, J. Izaac, N. Killoran, and G. Carleo, Quantum 4, 269 (2020).
- [16]B. Koczor and S. C. Benjamin, Phys. Rev. A 106, 062416 (2022), URL https://link.aps.org/doi/10.1103/PhysRevA.106.062416.DOI
- [17]S. Sorella and L. Capriotti, Phys. Rev. B 61, 2599 (2000), URL https://link.aps.org/doi/10.1103/PhysRevB.61.2599.DOI
- [18]S. Sorella, Phys. Rev. B 64, 024512 (2001), URL https://link.aps.org/doi/10.1103/PhysRevB.64.024512.DOI
- [19]G. Mazzola, A. Zen, and S. Sorella, The Journal of Chemical Physics 137, 134112 (2012), ISSN 0021-9606, https://pubs.aip.org/aip/jcp/article-pdf/doi/10.1063/1.4755992/15454421/134112_1_online.pdf, URL https://doi.org/10.1063/1.4755992.
- [20]C.-Y. Park and M. J. Kastoryano, Phys. Rev. Res. 2, 023232 (2020), URL https://link.aps.org/doi/10.1103/PhysRevResearch.2.023232.DOI
- [21]H. Xie, L. Zhang, and L. Wang, SciPost Phys. 14, 154 (2023), URL https://scipost.org/10.21468/SciPostPhys.14.6.154.
- [22]A. Lopatnikova, M.-N. Tran, and S. A. Sisson, arXiv preprint arXiv:2112.06587 (2021).arxiv.org/abs/2112.06587
- [23]H. Miyahara, K. Aihara, and W. Lechner, Physical Review A 101, 012326 (2020).
- [24]H. Miyahara and Y. Sugihiyama, Physical Review A 98, 022330 (2018).
- [25]H. Miyahara and V. Roychowdhury, Proceedings of the National Academy of Sciences 120, e2212660120 (2023).
- [26]T. Van Erven and P. Harremos, IEEE Transactions on Information Theory 60, 3797 (2014).
- [27]T. Mizohata, T. J. Kobayashi, L.-S. Bouchard, and H. Miyahara, arXiv preprint arXiv:2309.10334 (2023).DOI
- [28]D. Petz, Linear Algebra and its Applications 244, 81 (1996), ISSN 0024-3795, URL https://www.sciencedirect.com/science/article/pii/0024379594002118.
- [29]D. Petz, Quantum information theory and quantum statistics (Springer Science & Business Media, 2007).DOI
- [30]D. Petz and C. Ghinea, in Quantum probability and related topics (World Scientific, 2011), pp. 261–281.
- [31]T. Sagawa, Entropy, Divergence, and Majorization in Classical and Quantum Thermodynamics, vol. 16 (Springer Nature, 2022).
- [32]K. Takahashi and A. Fujiwara, Journal of Physics A: Mathematical and Theoretical 50, 165301 (2017), URL https://dx.doi.org/10.1088/1751-8121/aa6326.
- [33]H. Umegaki, in Kodai Mathematical Seminar Reports (Department of Mathematics, Tokyo Institute of Technology, 1962), vol. 14, pp. 59–85.
- [34]F. Hiai and D. Petz, Introduction to matrix analysis and applications (Springer Science & Business Media, 2014).DOI
- [35]F. HIAI, Interdisciplinary Information Sciences 16, 139 (2010).
- [36]R. Bhatia, Matrix Analysis, Graduate Texts in Mathematics (Springer New York, 1996), ISBN 9780387948461, URL https://books.google.co.jp/books?id=F4hRy1F1M6QC.
- [37]R. Horn and C. Johnson, Matrix Analysis (Cambridge University Press, 2012), ISBN 9780521548236, URL https://books.google.co.jp/books?id=mlQ-PgAACAAJ.


