Introduction
The Wigner formalism of quantum mechanics presents a completely quantum mechanical phase space description, at the cost of losing the non-negativity of probability density functions from which observables will be calculated through integrals over space. The Wigner picture is very intuitive, as it presents itself as a quantum extension of typical classical statistical mechanics descriptions of probability distributions in phase space linked to Boltzmann equations and other families of kinetic equations as well. This Wigner quantum analog of a classical probability density function over phase space shows the similarities but also stresses the differences between the quantum and classical worlds (sometimes behaving as a false cognate if taken naively without care or physical intuition). This intuitive physical picture of the Wigner function renders itself helpful in the study of open quantum systems. Under the assumption of a Markovian noise, the density matrix picture for these quantum subsystems perturbed by their environment is given by the Lindblad master equation, which has the main drawback that the operators representing the noise effect coming from the environment are somehow obscure and hard to interpret in a physically intuitive way even though their math structure is well understood from the operator algebra point of view. On the other hand, the completely equivalent Wigner-Fokker-Planck equation (in continuous variables under the assumption that a phase space description is possible) is a more amenable and physically intuitive mathematical model. Under the assumption of continuous variables, the density matrix description of Markovian open quantum systems under a Lindblad master equation is completely equivalent to the phase-space formulation of open quantum systems via a Wigner-Fokker-Planck equation, since one can obtain the Wigner quasi-probability density function as the Fourier transform of the density matrix in convenient coordinates (and revert back from the Wigner function to the density matrix expressed in a position basis via an inverse Fourier transform). If one assumes that the potential is harmonic as in V = x 2 / 2 V=x^2/2 V = x 2 /2 , the Wigner-Fokker-Planck equation takes the form (taking units such that ℏ = 1 = m \hbar=1=m ℏ = 1 = m )
w t + p ⋅ w x − x ⋅ w p = ∇ ⋅ ( D ∇ w ) + γ ∇ p ⋅ ( w p ) (1) w_t+p\cdot w_x-x\cdot w_p=\nabla\cdot(D\nabla w)+\gamma\nabla_p\cdot(wp) \tag*{(1)} w t + p ⋅ w x − x ⋅ w p = ∇ ⋅ ( D ∇ w ) + γ ∇ p ⋅ ( w p ) (1)
where D D D is the diffusion matrix representing part of the noise process introduced by the environment, and γ \gamma γ is a friction coefficient for the other term introducing noise. Though this benchmark problem is already very useful since a good amount of the potentials that physicists use to model phenomena are linear combinations of harmonic potentials, one could ask the question on how to use a Wigner picture for more general non-harmonic potentials. The Wigner-Fokker-Planck equation for an arbitrary potential V V V is given by
w t + p ⋅ w x + Θ [ V ] ( w ) = ∇ ⋅ ( D ∇ w ) + γ ∇ p ⋅ ( w p ) , Θ [ V ] ( w ) = − i ℏ ( 2 π ) d ∫ R 2 d δ V ( x , η ) w ( x , p ′ ) e i η ⋅ ( p − p ′ ) d p ′ d η (2) \begin{aligned}
w_t+p\cdot w_x+\Theta[V](w)&=\nabla\cdot(D\nabla w)+\gamma\nabla_p\cdot(wp),\\
\Theta[V](w)&=\frac{-i}{\hbar(2\pi)^d}\int_{\mathbb{R}^{2d}}\delta V(x,\eta)w(x,p')e^{i\eta\cdot(p-p')}\,dp'\,d\eta \tag*{(2)}
\end{aligned} w t + p ⋅ w x + Θ [ V ] ( w ) Θ [ V ] ( w ) = ∇ ⋅ ( D ∇ w ) + γ ∇ p ⋅ ( w p ) , = ℏ ( 2 π ) d − i ∫ R 2 d δ V ( x , η ) w ( x , p ′ ) e i η ⋅ ( p − p ′ ) d p ′ d η (2)
where δ V ( x , η ) = V ( x + η 2 ) − V ( x − η 2 ) \delta V(x,\eta)=V\left(x+\frac{\eta}{2}\right)-V\left(x-\frac{\eta}{2}\right) δ V ( x , η ) = V ( x + 2 η ) − V ( x − 2 η ) . The right-hand-side terms have a clear interpretation as first a diffusion operator and then a friction term (that can be treated as a transport). In the left hand side, there is quantum transport over the phase space, with the pseudo-differential (integral) operator Θ [ V ] ( w ) \Theta[V](w) Θ [ V ] ( w ) . This operator is the result of the Fourier-transform of the commutator [ ρ ^ , V ^ ] [\hat{\rho},\hat{V}] [ ρ ^ , V ^ ] in the Lindblad picture for density matrices ρ ^ \hat{\rho} ρ ^ . As we have mentioned above, the relation between the density matrix operator ρ ^ \hat{\rho} ρ ^ in its position representation and the Wigner function is given by a Fourier transform in convenient coordinates, given below,
w ( x , p , t ) = 1 ℏ ( 2 π ) d ∫ R 2 d ρ ( x + y , x − y ) e − i y ⋅ p / ℏ d y ρ ( x , y ) = ⟨ x + y ∣ ρ ^ ∣ x − y ⟩ . (3) \begin{aligned}
w(x,p,t)&=\frac{1}{\hbar(2\pi)^d}\int_{\mathbb{R}^{2d}}\rho(x+y,x-y)e^{-iy\cdot p/\hbar}\,dy\\
\rho(x,y)&=\langle x+y|\hat{\rho}|x-y\rangle. \tag*{(3)}
\end{aligned} w ( x , p , t ) ρ ( x , y ) = ℏ ( 2 π ) d 1 ∫ R 2 d ρ ( x + y , x − y ) e − i y ⋅ p /ℏ d y = ⟨ x + y ∣ ρ ^ ∣ x − y ⟩ . (3)
Stochastic Gradient Descent
We focus on the study of stochastic gradient descent by means of their continuous time limit (Su, Wibisono, Jordan, Shi). Following (Liu, Su, Li), if f : R d → R d f:\mathbb{R}^{d} \to\mathbb{R}^{d} f : R d → R d is the objective function constructed with all data, and the Stochastic Gradient Descent (SGD) algorithm is given by
x k + 1 = x k − s ∇ ~ f ( x k ) x_{k+1}=x_k-s\widetilde{\nabla}f(x_k) x k + 1 = x k − s ∇ f ( x k )
where s s s is the learning rate and ∇ ~ f \widetilde{\nabla}f ∇ f is the estimated gradient (obtained from a mini-batch), then one can model SGD by the following iteration,
x k + 1 = x k − s ∇ f ( x k ) − s ξ k x_{k+1}=x_k-s\nabla f(x_k)-s\xi_k x k + 1 = x k − s ∇ f ( x k ) − s ξ k
where ξ k \xi_k ξ k is a given noise that obeys a normal distribution, as in the so-called “unadjusted Langevin dynamics”. Taking the continuous time limit one obtains a stochastic differential equation (SDE) that includes the learning rate s s s as a parameter, which models the discrete SGD algorithm. This SDE is
d x = − ∇ f ( x ) d t + s d W , dx=-\nabla f(x)dt+\sqrt{s}dW, d x = − ∇ f ( x ) d t + s d W ,
with W W W a Brownian motion.
There is a direct analogy with open quantum systems via this SDE. In the case where the environment introduces Markovian noise, the SDE version of the Wigner-Fokker-Planck model of open quantum systems is given by an SDE modeling the evolution of the Wigner quasi-distribution w ( x , p , t ) w(x,p,t) w ( x , p , t ) . Taking units such that ℏ = 1 = m \hbar=1=m ℏ = 1 = m , if we consider α \alpha α the quantum transport over the subsystem, σ \sigma σ the standard deviation related to the noise, 2 σ d W = 2 D d t N ( 0 , 1 ) 2\sigma dW=2Ddt\mathcal{N}(0,1) 2 σ d W = 2 D d t N ( 0 , 1 ) where N ( 0 , 1 ) \mathcal{N}(0,1) N ( 0 , 1 ) is a normal distribution with mean zero and variance 1, and D D D the diffusion matrix representing part of the noise introduced by the environment. The other part of the noise is given by a friction term which is included in the quantum transport. Namely, the friction term is γ ∇ p ⋅ ( w p ) \gamma\nabla_p\cdot(wp) γ ∇ p ⋅ ( w p ) , where γ \gamma γ is the friction coefficient and p p p is the momentum variable. The full transport term (including both pure quantum transport and the friction term representing noise but added to the transport term) is
α ( x , p , t ) = ∇ ⋅ ( w ( p , − x − γ p ) ) . \alpha(x,p,t)=\nabla\cdot(w(p,-x-\gamma p)). α ( x , p , t ) = ∇ ⋅ ( w ( p , − x − γ p )) .
From a particle viewpoint, where z = ( x , p ) z=(x,p) z = ( x , p ) represents the location in phase space of particle points to be sampled from a Wigner quasi-PDF, following D’Ariano et al., one can represent the motion of phase-space points according to the following model,
d z i = Q [ z i ( t ) ] d t + E [ z i ( t ) ] dz_i=Q[z_i(t)]dt+E[z_i(t)] d z i = Q [ z i ( t )] d t + E [ z i ( t )]
with i ∈ { 1 , N } i\in\{1,N\} i ∈ { 1 , N } an index representing the studied particle from the sample, Q [ z ( t ) ] = ( p , − x − γ p ) Q[z(t)]=(p,-x-\gamma p) Q [ z ( t )] = ( p , − x − γ p ) being the quantum transport, and E E E being a zero average Gaussian random variable, whose variance is D d t Ddt D d t . The closest expression as a stochastic gradient for our SDE related to open quantum systems is via the Hamiltonian+gradient flow as in
d z i = [ J ∇ i H ( z ) − ∇ i ( γ p 2 / 2 ) ] d t + E [ z i ( t ) ] , i ∈ { 1 , ⋯ , N } dz_i = [J\nabla_i H(z) - \nabla_i(\gamma p^2/2)]dt + E[z_i(t)], \quad i \in\{1,\cdots,N\} d z i = [ J ∇ i H ( z ) − ∇ i ( γ p 2 /2 )] d t + E [ z i ( t )] , i ∈ { 1 , ⋯ , N }
where part of the flow is indeed gradient (the friction one) but the other part is related to a Hamiltonian flow where the gradient is multiplied by the symplectic matrix J J J . Our problem has, regarding transport, both a symplectic flow structure (due to the Hamiltonian) and a gradient flow structure too (due to the friction term), which combined with the diffusion due to the environment noise, has a dynamics almost similar to the SGD counterpart (becoming more similar to a gradient flow as the friction term dominates in the limit γ → ∞ \gamma\to\infty γ → ∞ ). To fully express our problem then, we have then that
d ( x i , p i ) = [ J ∇ i ( p 2 + x 2 2 ) − γ ∇ i ( p 2 2 ) ] d t + E [ x i ( t ) , p i ( t ) ] , i ∈ { 1 , ⋯ , N } d(x_i,p_i) = [J\nabla_i\left(\frac{p^2+x^2}{2}\right) - \gamma\nabla_i\left(\frac{p^2}{2}\right)]dt + E[x_i(t),p_i(t)], \quad i \in\{1,\cdots,N\} d ( x i , p i ) = [ J ∇ i ( 2 p 2 + x 2 ) − γ ∇ i ( 2 p 2 ) ] d t + E [ x i ( t ) , p i ( t )] , i ∈ { 1 , ⋯ , N }
Our point: the Hamiltonian flow (for the case of the harmonic oscillator dynamics) plus a dominant gradient-flow (in the case where the friction dominates by making the damping coefficient γ → ∞ \gamma\to\infty γ → ∞ ) and diffusion terms (the last two due to subsystem-environment interaction) is almost equivalent to the continuous limit of Stochastic Gradient Descent (when friction dominates Hamiltonian transport), and therefore their behaviors (and performances) are similar (the similarity becoming stronger as friction dominates more the Hamiltonian transport). Since the numerics of SGD is similar to the Physics of open quantum systems, we would expect variations of SGD to a good numerical model available for mimicking quantum problems where the hamiltonian transport is subjected to environment noise represented in both transport due to friction and diffusion connected to random walks happening continuously in time.
Conjecture 1. For minimization problems that can be formulated as quantum transport perturbed by Markovian noise, then SGD must a good candidate for a numerical method available, since it’s reproducing exactly the problem it’s trying to solve.
Advantages of an SGD formulation as an open quantum system rather than a closed quantum dynamics
A closed quantum system description has the drawback that their evolution is unitary, so a Quantum Tunneling Walk (QTW) will not formally converge. This is not the behavior observed for SGD, whose dynamics is quite similar to an open quantum system: gradient-related transport plus noise, and convergence to a steady state solution (as proven for the benchmark problem of a harmonic potential). So an open quantum system dynamics does have truly a mixing time as exponential decay towards the steady state solution, for example in the case of a harmonic potential, where it has been proved by Sparber et al [6 ] .
Wigner-Fokker-Planck: harmonic case in arbitrary dimension, decay rate, mixing time
In classical machine learning, it is known [1 ] ,[2 ] ,[3 ] ,[8 ] that the continuous-time limit of gradient descent methods such as the Stochastic Gradient Descent (SGD) algorithm can be modeled by differential equations. For SGD, whose Stochastic Differential Equation (SDE) approximation is
d x = − ∇ f ( x ) d t + s d W , dx=-\nabla f(x)dt+\sqrt{s}dW, d x = − ∇ f ( x ) d t + s d W ,
with W W W a standard Brownian motion, the Fokker–Planck equation for the SGD pdf ρ S G D \rho_{\mathrm{SGD}} ρ SGD ,
∂ t ρ S G D = ∇ ⋅ ( ρ S G D ∇ f ) + s 2 ∇ 2 ρ S G D , \partial_t\rho_{\mathrm{SGD}}=\nabla\cdot(\rho_{\mathrm{SGD}}\nabla f)+\frac{s}{2}\nabla^2\rho_{\mathrm{SGD}}, ∂ t ρ SGD = ∇ ⋅ ( ρ SGD ∇ f ) + 2 s ∇ 2 ρ SGD ,
is the respective continuous-time limit equivalent [14 ] ,[12 ] ,[13 ] .
Quantum extensions of SGD have been proposed attempting to obtain a quantum advantage. Work in [14 ] explores quantum speedups for nonconvex optimization problems introducing a quantum tunneling walk (QTW) algorithm for nonconvex problems where local minima are "approximately global". This is obtained by analogy between classical Fokker-Planck and the Schroedinger equation as the quantum equivalent under certain periodicity conditions. They report quantum speedup over classical stochastic gradient descents (SGD) when the barriers between different local minima are high but thin and flat minima.
However, it is physically clear that the quantum version of the equation representing the SDE version of SGD should be Wigner-Fokker-Planck, which for the case of a harmonic potential V ( x ) = ω 0 2 x 2 / 2 V(x)=\omega_0^2x^2/2 V ( x ) = ω 0 2 x 2 /2 takes the form
∂ t w = − ∇ ⋅ ( w J ∇ H ) + γ ∇ p ⋅ ( w ∇ p p 2 2 ) + ∇ ⋅ ( D ∇ w ) , J = ( 0 I d − I d 0 ) , (4) \partial_t w=-\nabla\cdot(wJ\nabla H)+\gamma\nabla_p\cdot\left(w\nabla_p\frac{p^2}{2}\right)+\nabla\cdot(D\nabla w),\qquad J=\begin{pmatrix}0&I_d\\-I_d&0\end{pmatrix}, \tag*{(4)} ∂ t w = − ∇ ⋅ ( w J ∇ H ) + γ ∇ p ⋅ ( w ∇ p 2 p 2 ) + ∇ ⋅ ( D ∇ w ) , J = ( 0 − I d I d 0 ) , (4)
with the Hamiltonian H = T ( p ) + V ( x ) = p 2 2 m + ω 0 2 x 2 2 H=T(p)+V(x)=\frac{p^2}{2m}+\frac{\omega_0^2x^2}{2} H = T ( p ) + V ( x ) = 2 m p 2 + 2 ω 0 2 x 2 representing the (kinetic plus potential) energy, J J J the symplectic matrix, d d d the dimension of the physical space, the diffusion matrix D D D and friction coefficient γ \gamma γ , because the open quantum system model represented above is the quantum version of the system studied by the classical Fokker-Planck representing an SGD, particularly when the gradient term associated to friction dominates the Hamiltonian transport. Wigner-Fokker-Planck is completely equivalent to the dynamics of a noisy quantum transport system whose SDE form is
d ( x , p ) = ( J ∇ H − ∇ p γ p 2 2 ) d t + E ( D ) , (5) d(x,p)=\left(J\nabla H-\nabla_p\frac{\gamma p^2}{2}\right)dt+E(D), \tag*{(5)} d ( x , p ) = ( J ∇ H − ∇ p 2 γ p 2 ) d t + E ( D ) , (5)
where x x x is the position and p p p the momentum in quantum phase space, and E E E is a random variable sampled from a Gaussian distribution with covariance matrix 2 D d t 2Ddt 2 D d t (d t ≈ Δ t dt\approx\Delta t d t ≈ Δ t discretized). We illustrate in Figure 1 below the dynamics of the Wigner-Fokker-Planck model for an open quantum system as the balance between quantum transport (made of both a Hamiltonian flow and gradient transport) and diffusive noise.
Figure 1. A Wigner distribution is shown, where quantum transport & diffusion mechanisms over a point are highlighted. This process is the continuous quantum limit of Stochastic Gradient Descent.
There are previous studies of the harmonic benchmark for WFP [6 ] , whose results can be used. In particular, Sparber et al. prove in [6 ] the existence of a steady-state solution w ∞ ( x , p ) w_{\infty}(x,p) w ∞ ( x , p ) for this benchmark problem, providing its analytical form, and it also gives the exponential decay rate of any well-posed initial condition towards this analytical steady state for the harmonic case. Namely, [6 ] finds that
∥ w ( t , ⋅ ) − w ∞ ∥ p ≤ C exp ( − κ t ) , C > 0 , t > 0 , \lVert w(t,\cdot)-w_{\infty}\rVert_p \le C \exp(-\kappa t), \qquad C>0,\ t>0, ∥ w ( t , ⋅ ) − w ∞ ∥ p ≤ C exp ( − κ t ) , C > 0 , t > 0 ,
with 1 ≤ p ≤ 2 1 \le p \le2 1 ≤ p ≤ 2 depending on the relative entropy (logarithmic or quadratic), and that the respective decay rate is the smallest eigenvalue of the Hessian of the quadratic polynomial A A A such that w ∞ ∝ exp ( − A ( x , p ) ) w_{\infty} \propto\exp(-A(x,p)) w ∞ ∝ exp ( − A ( x , p )) (the Wigner steady state for this problem is a Gaussian state), so that
Hess A ≥ κ I 2 d , ( Hess A ) i j = ∂ i j 2 A , i , j = 1 , ⋯ , 2 d . \operatorname{Hess} A \ge\kappa I_{2d}, \qquad(\operatorname{Hess} A)_{ij} = \partial_{ij}^{2} A,\quad i,j=1,\cdots,2d. Hess A ≥ κ I 2 d , ( Hess A ) ij = ∂ ij 2 A , i , j = 1 , ⋯ , 2 d .
This exponential decay rate is advantageous for interpreting WFP as a quantum extension of SGD because it will provide the respective mixing time of a quantum SGD algorithm based on the WFP equation. Most importantly, nothing precludes interpreting WFP not only on physical dimensions such that d = 1 , 2 , 3 d=1,2,3 d = 1 , 2 , 3 , but actually for d ≫ 1 d \gg1 d ≫ 1 as is typically the case for Machine Learning, in which the problem is of high dimensionality given the amount of data for the problem. Our result is that, for WFP in a high dimension d ≫ 1 d \gg1 d ≫ 1 , the decay rate providing the mixing time for the related quantum SGD algorithm in the benchmark case of a harmonic potential is simply the smallest of only 2 possible roots (with high algebraic multiplicity) of the characteristic polynomial, which although of degree 2 d 2d 2 d , as we mentioned has only 2 possibly different roots of multiplicity d d d .
Our preliminary result indicates that the decay rate towards the steady state in WFP (and therefore the mixing time in its quantum version of SGD) does not depend on the dimension d d d of the problem for the harmonic potential benchmark problem, as it should be since this doesn’t happen either in the classical SGD optimization for a quadratic function (such as the parabola representing a harmonic potential). Namely, the decay rate does not slow down as the quantum space dimension d ≫ 1 d \gg1 d ≫ 1 increases for this benchmark case of a harmonic potential.
Preliminary Calculations
We show the preliminary calculations for high dimensions d ≫ 1 d \gg1 d ≫ 1 . We consider x ∈ R d x \in\mathbb{R}^{d} x ∈ R d , p ∈ R d p \in\mathbb{R}^{d} p ∈ R d , so ( x , p ) ∈ R 2 d (x,p) \in\mathbb{R}^{2d} ( x , p ) ∈ R 2 d , and therefore the diffusion matrix is the block matrix
D = ( D q q I d D p q I d D p q I d D p p I d ) , D p q = D q p , D = \begin{pmatrix} D_{qq}I_d & D_{pq}I_d \\ D_{pq}I_d & D_{pp}I_d \end{pmatrix}, \qquad D_{pq}=D_{qp}, D = ( D q q I d D pq I d D pq I d D pp I d ) , D pq = D q p ,
where D i j ∈ R d × d D_{ij} \in\mathbb{R}^{d \times d} D ij ∈ R d × d , i , j = q , p i,j=q,p i , j = q , p . As mentioned by Sparber et al. [6 ] , the optimal exponential rate κ \kappa κ is the smallest eigenvalue of the Hessian of A ~ \widetilde{A} A . The potential A ~ \widetilde{A} A is defined by
A ~ ( y ~ ) = A ( D y T ) , \widetilde{A}(\widetilde{y}) = A(\sqrt{D}y^{T}), A ( y ) = A ( D y T ) ,
with D \sqrt{D} D the square root of D D D in the sense of positive definite matrices, and y = ( x , p ) y=(x,p) y = ( x , p ) .
We know the form of A A A for the steady state, given by
A ( x , p ) = γ Q [ Q 11 ω 0 2 x 2 + 2 Q 12 ω 0 x ⋅ p + Q 22 p 2 ] A(x,p)=\frac{\gamma}{Q}\left[Q_{11}\omega_0^2x^2+2Q_{12}\omega_0x\cdot p+Q_{22}p^2\right] A ( x , p ) = Q γ [ Q 11 ω 0 2 x 2 + 2 Q 12 ω 0 x ⋅ p + Q 22 p 2 ]
Q 11 = D p p + ω 0 2 D q q Q_{11}=D_{pp}+\omega_0^2D_{qq} Q 11 = D pp + ω 0 2 D q q
Q 12 = 2 ω 0 γ D q q Q_{12}=2\omega_0\gamma D_{qq} Q 12 = 2 ω 0 γ D q q
Q 22 = D p p + ω 0 2 D q q + 4 γ ( D p q + γ D q q ) = Q 11 + 4 γ ( D p q + γ D q q ) Q_{22}=D_{pp}+\omega_0^2D_{qq}+4\gamma(D_{pq}+\gamma D_{qq})=Q_{11}+4\gamma(D_{pq}+\gamma D_{qq}) Q 22 = D pp + ω 0 2 D q q + 4 γ ( D pq + γ D q q ) = Q 11 + 4 γ ( D pq + γ D q q )
Q = Q 11 Q 22 − Q 12 2 Q=Q_{11}Q_{22}-Q_{12}^{2} Q = Q 11 Q 22 − Q 12 2
Since to compute the Hessian of the modified potential A ~ \widetilde{A} A we do it against the variables y ~ \widetilde{y} y , it is equivalent to compute the Hessian of A A A against the variable y y y . So we simply consider
H = Hessian ( A ) = γ Q ( Q 11 ω 0 2 Q 12 ω 0 Q 12 ω 0 Q 22 ) H=\operatorname{Hessian}(A)=\frac{\gamma}{Q}\begin{pmatrix} Q_{11}\omega_0^2 & Q_{12}\omega_0 \\ Q_{12}\omega_0 & Q_{22} \end{pmatrix} H = Hessian ( A ) = Q γ ( Q 11 ω 0 2 Q 12 ω 0 Q 12 ω 0 Q 22 )
Preliminary Calculations
H = γ Q ( [ D p p + ω 0 2 D q q ] ω 0 2 2 ω 0 2 γ D q q 2 ω 0 2 γ D q q D p p + ω 0 2 D q q + 4 γ ( D p q + γ D q q ) ) H=\frac{\gamma}{Q}\begin{pmatrix}
[D_{pp}+\omega_0^2D_{qq}]\omega_0^2 & 2\omega_0^2\gamma D_{qq}\\
2\omega_0^2\gamma D_{qq} & D_{pp}+\omega_0^2D_{qq}+4\gamma(D_{pq}+\gamma D_{qq})
\end{pmatrix} H = Q γ ( [ D pp + ω 0 2 D q q ] ω 0 2 2 ω 0 2 γ D q q 2 ω 0 2 γ D q q D pp + ω 0 2 D q q + 4 γ ( D pq + γ D q q ) )
with
Tr ( H ) = γ Q ( Q 11 ω 0 2 + Q 22 ) = γ Q ( Q 11 ω 0 2 + Q 11 + 4 γ ( D p q + γ D q q ) ) \operatorname{Tr}(H)=\frac{\gamma}{Q}(Q_{11}\omega_0^2+Q_{22})=\frac{\gamma}{Q}(Q_{11}\omega_0^2+Q_{11}+4\gamma(D_{pq}+\gamma D_{qq})) Tr ( H ) = Q γ ( Q 11 ω 0 2 + Q 22 ) = Q γ ( Q 11 ω 0 2 + Q 11 + 4 γ ( D pq + γ D q q ))
Tr ( H ) = γ Q ( [ D p p + ω 0 2 D q q ] ω 0 2 + D p p + ω 0 2 D q q + 4 γ ( D p q + γ D q q ) ) \operatorname{Tr}(H)=\frac{\gamma}{Q}([D_{pp}+\omega_0^2D_{qq}]\omega_0^2+D_{pp}+\omega_0^2D_{qq}+4\gamma(D_{pq}+\gamma D_{qq})) Tr ( H ) = Q γ ([ D pp + ω 0 2 D q q ] ω 0 2 + D pp + ω 0 2 D q q + 4 γ ( D pq + γ D q q ))
Tr ( H ) = γ Q ( [ D p p + ω 0 2 D q q ] ( ω 0 2 + 1 ) + 4 γ ( D p q + γ D q q ) ) \operatorname{Tr}(H)=\frac{\gamma}{Q}([D_{pp}+\omega_0^2D_{qq}](\omega_0^2+1)+4\gamma(D_{pq}+\gamma D_{qq})) Tr ( H ) = Q γ ([ D pp + ω 0 2 D q q ] ( ω 0 2 + 1 ) + 4 γ ( D pq + γ D q q ))
and
det ( H ) = ( γ Q ) 2 ( Q 11 Q 22 − Q 12 2 ) ω 0 2 = ( γ Q ω 0 ) 2 Q \det(H)=\left(\frac{\gamma}{Q}\right)^2(Q_{11}Q_{22}-Q_{12}^2)\omega_0^2=\left(\frac{\gamma}{Q}\omega_0\right)^2Q det ( H ) = ( Q γ ) 2 ( Q 11 Q 22 − Q 12 2 ) ω 0 2 = ( Q γ ω 0 ) 2 Q
So the characteristic polynomial for the Hessian of A A A is, for d = 1 d=1 d = 1 ,
λ 2 − Tr ( H ) λ + det ( H ) = 0 \lambda^2-\operatorname{Tr}(H)\lambda+\det(H)=0 λ 2 − Tr ( H ) λ + det ( H ) = 0
λ 2 − γ Q ( Q 11 ω 0 2 + Q 22 ) λ + ( γ Q ω 0 ) 2 ( Q 11 Q 22 − Q 12 2 ) = 0 \lambda^2-\frac{\gamma}{Q}(Q_{11}\omega_0^2+Q_{22})\lambda+\left(\frac{\gamma}{Q}\omega_0\right)^2(Q_{11}Q_{22}-Q_{12}^2)=0 λ 2 − Q γ ( Q 11 ω 0 2 + Q 22 ) λ + ( Q γ ω 0 ) 2 ( Q 11 Q 22 − Q 12 2 ) = 0
λ 2 − γ Q ( Q 11 ω 0 2 + Q 22 ) λ + ( γ Q ω 0 ) 2 Q = 0 \lambda^2-\frac{\gamma}{Q}(Q_{11}\omega_0^2+Q_{22})\lambda+\left(\frac{\gamma}{Q}\omega_0\right)^2Q=0 λ 2 − Q γ ( Q 11 ω 0 2 + Q 22 ) λ + ( Q γ ω 0 ) 2 Q = 0
therefore the roots are
λ ± = γ Q ( Q 11 ω 0 2 + Q 22 ) ± [ γ Q ( Q 11 ω 0 2 + Q 22 ) ] 2 − 4 ( γ Q ω 0 ) 2 Q 2 \lambda_{\pm}=\frac{\frac{\gamma}{Q}(Q_{11}\omega_0^2+Q_{22})\pm\sqrt{\left[\frac{\gamma}{Q}(Q_{11}\omega_0^2+Q_{22})\right]^2-4\left(\frac{\gamma}{Q}\omega_0\right)^2Q}}{2} λ ± = 2 Q γ ( Q 11 ω 0 2 + Q 22 ) ± [ Q γ ( Q 11 ω 0 2 + Q 22 ) ] 2 − 4 ( Q γ ω 0 ) 2 Q
λ ± = γ Q ( Q 11 ω 0 2 + Q 22 ) ± [ γ Q ( Q 11 ω 0 2 + Q 22 ) ] 2 − 4 ( γ Q ω 0 ) 2 ( Q 11 Q 22 − Q 12 2 ) 2 \lambda_{\pm}=\frac{\frac{\gamma}{Q}(Q_{11}\omega_0^2+Q_{22})\pm\sqrt{\left[\frac{\gamma}{Q}(Q_{11}\omega_0^2+Q_{22})\right]^2-4\left(\frac{\gamma}{Q}\omega_0\right)^2(Q_{11}Q_{22}-Q_{12}^2)}}{2} λ ± = 2 Q γ ( Q 11 ω 0 2 + Q 22 ) ± [ Q γ ( Q 11 ω 0 2 + Q 22 ) ] 2 − 4 ( Q γ ω 0 ) 2 ( Q 11 Q 22 − Q 12 2 )
λ ± = γ Q ( Q 11 ω 0 2 + Q 22 ) ± ∣ γ Q ∣ ( Q 11 ω 0 2 ) 2 + 2 Q 22 Q 11 ω 0 2 + Q 22 2 − 4 ω 0 2 ( Q 11 Q 22 − Q 12 2 ) 2 \lambda_{\pm}=\frac{\frac{\gamma}{Q}(Q_{11}\omega_0^2+Q_{22})\pm\left|\frac{\gamma}{Q}\right|\sqrt{(Q_{11}\omega_0^2)^2+2Q_{22}Q_{11}\omega_0^2+Q_{22}^2-4\omega_0^2(Q_{11}Q_{22}-Q_{12}^2)}}{2} λ ± = 2 Q γ ( Q 11 ω 0 2 + Q 22 ) ± Q γ ( Q 11 ω 0 2 ) 2 + 2 Q 22 Q 11 ω 0 2 + Q 22 2 − 4 ω 0 2 ( Q 11 Q 22 − Q 12 2 )
λ ± = γ Q ( Q 11 ω 0 2 + Q 22 ) ± ∣ γ Q ∣ ( Q 11 ω 0 2 ) 2 − 2 Q 22 Q 11 ω 0 2 + Q 22 2 + 4 ω 0 2 Q 12 2 2 \lambda_{\pm}=\frac{\frac{\gamma}{Q}(Q_{11}\omega_0^2+Q_{22})\pm\left|\frac{\gamma}{Q}\right|\sqrt{(Q_{11}\omega_0^2)^2-2Q_{22}Q_{11}\omega_0^2+Q_{22}^2+4\omega_0^2Q_{12}^2}}{2} λ ± = 2 Q γ ( Q 11 ω 0 2 + Q 22 ) ± Q γ ( Q 11 ω 0 2 ) 2 − 2 Q 22 Q 11 ω 0 2 + Q 22 2 + 4 ω 0 2 Q 12 2
λ ± = γ Q ( Q 11 ω 0 2 + Q 22 ) ± ∣ γ Q ∣ ( Q 11 ω 0 2 − Q 22 ) 2 + ( 2 ω 0 Q 12 ) 2 2 \lambda_{\pm}=\frac{\frac{\gamma}{Q}(Q_{11}\omega_0^2+Q_{22})\pm\left|\frac{\gamma}{Q}\right|\sqrt{(Q_{11}\omega_0^2-Q_{22})^2+(2\omega_0Q_{12})^2}}{2} λ ± = 2 Q γ ( Q 11 ω 0 2 + Q 22 ) ± Q γ ( Q 11 ω 0 2 − Q 22 ) 2 + ( 2 ω 0 Q 12 ) 2
λ ± = γ Q ( Q 11 ω 0 2 + Q 22 ) ± [ γ Q ( Q 11 ω 0 2 − Q 22 ) ] 2 + ( 2 γ Q ω 0 Q 12 ) 2 2 \lambda_{\pm}=\frac{\frac{\gamma}{Q}(Q_{11}\omega_0^2+Q_{22})\pm\sqrt{\left[\frac{\gamma}{Q}(Q_{11}\omega_0^2-Q_{22})\right]^2+\left(2\frac{\gamma}{Q}\omega_0Q_{12}\right)^2}}{2} λ ± = 2 Q γ ( Q 11 ω 0 2 + Q 22 ) ± [ Q γ ( Q 11 ω 0 2 − Q 22 ) ] 2 + ( 2 Q γ ω 0 Q 12 ) 2
λ ± = γ ( Q 11 ω 0 2 + Q 22 ) 2 Q ± [ γ 2 Q ( Q 11 ω 0 2 − Q 22 ) ] 2 + ( γ Q ω 0 Q 12 ) 2 \lambda_{\pm}=\frac{\gamma(Q_{11}\omega_0^2+Q_{22})}{2Q}\pm\sqrt{\left[\frac{\gamma}{2Q}(Q_{11}\omega_0^2-Q_{22})\right]^2+\left(\frac{\gamma}{Q}\omega_0Q_{12}\right)^2} λ ± = 2 Q γ ( Q 11 ω 0 2 + Q 22 ) ± [ 2 Q γ ( Q 11 ω 0 2 − Q 22 ) ] 2 + ( Q γ ω 0 Q 12 ) 2
so the optimal exponential rate κ \kappa κ is then the smallest eigenvalue,
κ = γ ( Q 11 ω 0 2 + Q 22 ) 2 Q − γ ( Q 11 ω 0 2 − Q 22 2 Q ) 2 + ( ω 0 Q 12 Q ) 2 \kappa=\frac{\gamma(Q_{11}\omega_0^2+Q_{22})}{2Q}-\gamma\sqrt{\left(\frac{Q_{11}\omega_0^2-Q_{22}}{2Q}\right)^2+\left(\frac{\omega_0Q_{12}}{Q}\right)^2} κ = 2 Q γ ( Q 11 ω 0 2 + Q 22 ) − γ ( 2 Q Q 11 ω 0 2 − Q 22 ) 2 + ( Q ω 0 Q 12 ) 2
Preliminary Calculations
then
κ = γ ( Q 11 ω 0 2 + Q 11 + 4 γ ( D p q + γ D q q ) ) 2 Q − γ [ Q 11 ω 0 2 − Q 11 − 4 γ ( D p q + γ D q q ) 2 Q ] 2 + ( 2 ω 0 2 γ D q q Q ) 2 \kappa= \frac{\gamma(Q_{11}\omega_0^2+Q_{11}+4\gamma(D_{pq}+\gamma D_{qq}))}{2Q}-\gamma\sqrt{\left[\frac{Q_{11}\omega_0^2-Q_{11}-4\gamma(D_{pq}+\gamma D_{qq})}{2Q}\right]^2+\left(\frac{2\omega_0^2\gamma D_{qq}}{Q}\right)^2} κ = 2 Q γ ( Q 11 ω 0 2 + Q 11 + 4 γ ( D pq + γ D q q )) − γ [ 2 Q Q 11 ω 0 2 − Q 11 − 4 γ ( D pq + γ D q q ) ] 2 + ( Q 2 ω 0 2 γ D q q ) 2
κ = γ ( Q 11 [ ω 0 2 + 1 ] + 4 γ [ D p q + γ D q q ] ) 2 Q − γ [ Q 11 ( ω 0 2 − 1 ) − 4 γ ( D p q + γ D q q ) 2 Q ] 2 + ( 4 ω 0 2 γ D q q 2 Q ) 2 \kappa= \frac{\gamma(Q_{11}[\omega_0^2+1]+4\gamma[D_{pq}+\gamma D_{qq}])}{2Q}-\gamma\sqrt{\left[\frac{Q_{11}(\omega_0^2-1)-4\gamma(D_{pq}+\gamma D_{qq})}{2Q}\right]^2+\left(\frac{4\omega_0^2\gamma D_{qq}}{2Q}\right)^2} κ = 2 Q γ ( Q 11 [ ω 0 2 + 1 ] + 4 γ [ D pq + γ D q q ]) − γ [ 2 Q Q 11 ( ω 0 2 − 1 ) − 4 γ ( D pq + γ D q q ) ] 2 + ( 2 Q 4 ω 0 2 γ D q q ) 2
κ = γ ( Q 11 [ ω 0 2 + 1 ] + 4 γ [ D p q + γ D q q ] ) 2 Q − γ [ Q 11 ( ω 0 2 − 1 ) − 4 γ ( D p q + γ D q q ) ] 2 + ( 4 ω 0 2 γ D q q ) 2 ( 2 Q ) 2 \kappa= \frac{\gamma(Q_{11}[\omega_0^2+1]+4\gamma[D_{pq}+\gamma D_{qq}])}{2Q}-\gamma\sqrt{\frac{[Q_{11}(\omega_0^2-1)-4\gamma(D_{pq}+\gamma D_{qq})]^2+(4\omega_0^2\gamma D_{qq})^2}{(2Q)^2}} κ = 2 Q γ ( Q 11 [ ω 0 2 + 1 ] + 4 γ [ D pq + γ D q q ]) − γ ( 2 Q ) 2 [ Q 11 ( ω 0 2 − 1 ) − 4 γ ( D pq + γ D q q ) ] 2 + ( 4 ω 0 2 γ D q q ) 2
κ = γ ( Q 11 [ ω 0 2 + 1 ] + 4 γ [ D p q + γ D q q ] ) 2 Q − γ [ Q 11 ( ω 0 2 − 1 ) ] 2 + [ 4 γ ( D p q + γ D q q ) ] 2 − 8 γ ( D p q + γ D q q ) Q 11 ( ω 0 2 − 1 ) + 16 ω 0 4 γ 2 D q q 2 ( 2 Q ) 2 \begin{aligned}
\kappa&= \frac{\gamma(Q_{11}[\omega_0^2+1]+4\gamma[D_{pq}+\gamma D_{qq}])}{2Q}\\
&\quad-\gamma\sqrt{\frac{[Q_{11}(\omega_0^2-1)]^2+[4\gamma(D_{pq}+\gamma D_{qq})]^2-8\gamma(D_{pq}+\gamma D_{qq})Q_{11}(\omega_0^2-1)+16\omega_0^4\gamma^2D_{qq}^2}{(2Q)^2}}
\end{aligned} κ = 2 Q γ ( Q 11 [ ω 0 2 + 1 ] + 4 γ [ D pq + γ D q q ]) − γ ( 2 Q ) 2 [ Q 11 ( ω 0 2 − 1 ) ] 2 + [ 4 γ ( D pq + γ D q q ) ] 2 − 8 γ ( D pq + γ D q q ) Q 11 ( ω 0 2 − 1 ) + 16 ω 0 4 γ 2 D q q 2
κ = γ ( Q 11 [ ω 0 2 + 1 ] + 4 γ [ D p q + γ D q q ] ) 2 Q − γ [ Q 11 ( ω 0 2 − 1 ) ] 2 + [ 4 γ ( D p q + γ D q q ) ] 2 − 8 γ Q 11 ( ω 0 2 − 1 ) D p q − 8 γ 2 Q 11 ( ω 0 2 − 1 ) D q q + 16 ω 0 4 γ 2 D q q 2 ( 2 Q ) 2 \begin{aligned}
\kappa&= \frac{\gamma(Q_{11}[\omega_0^2+1]+4\gamma[D_{pq}+\gamma D_{qq}])}{2Q}\\
&\quad-\gamma\sqrt{\frac{[Q_{11}(\omega_0^2-1)]^2+[4\gamma(D_{pq}+\gamma D_{qq})]^2-8\gamma Q_{11}(\omega_0^2-1)D_{pq}-8\gamma^2Q_{11}(\omega_0^2-1)D_{qq}+16\omega_0^4\gamma^2D_{qq}^2}{(2Q)^2}}
\end{aligned} κ = 2 Q γ ( Q 11 [ ω 0 2 + 1 ] + 4 γ [ D pq + γ D q q ]) − γ ( 2 Q ) 2 [ Q 11 ( ω 0 2 − 1 ) ] 2 + [ 4 γ ( D pq + γ D q q ) ] 2 − 8 γ Q 11 ( ω 0 2 − 1 ) D pq − 8 γ 2 Q 11 ( ω 0 2 − 1 ) D q q + 16 ω 0 4 γ 2 D q q 2
κ = γ ( Q 11 [ ω 0 2 + 1 ] + 4 γ [ D p q + γ D q q ] ) 2 Q − γ [ Q 11 ( ω 0 2 − 1 ) ] 2 + [ 4 γ ( D p q + γ D q q ) ] 2 − 8 γ Q 11 ( ω 0 2 − 1 ) D p q + 8 γ 2 D q q [ 2 ω 0 2 D q q − Q 11 ( ω 0 2 − 1 ) ] ( 2 Q ) 2 \begin{aligned}
\kappa&= \frac{\gamma(Q_{11}[\omega_0^2+1]+4\gamma[D_{pq}+\gamma D_{qq}])}{2Q}\\
&\quad-\gamma\sqrt{\frac{[Q_{11}(\omega_0^2-1)]^2+[4\gamma(D_{pq}+\gamma D_{qq})]^2-8\gamma Q_{11}(\omega_0^2-1)D_{pq}+8\gamma^2D_{qq}[2\omega_0^2D_{qq}-Q_{11}(\omega_0^2-1)]}{(2Q)^2}}
\end{aligned} κ = 2 Q γ ( Q 11 [ ω 0 2 + 1 ] + 4 γ [ D pq + γ D q q ]) − γ ( 2 Q ) 2 [ Q 11 ( ω 0 2 − 1 ) ] 2 + [ 4 γ ( D pq + γ D q q ) ] 2 − 8 γ Q 11 ( ω 0 2 − 1 ) D pq + 8 γ 2 D q q [ 2 ω 0 2 D q q − Q 11 ( ω 0 2 − 1 )]
and since
Q 11 = D p p + ω 0 2 D q q Q_{11}=D_{pp}+\omega_0^2D_{qq} Q 11 = D pp + ω 0 2 D q q
κ = γ ( Q 11 [ ω 0 2 + 1 ] + 4 γ [ D p q + γ D q q ] ) 2 Q − γ [ Q 11 ( ω 0 2 − 1 ) ] 2 + [ 4 γ ( D p q + γ D q q ) ] 2 − 8 γ Q 11 ( ω 0 2 − 1 ) D p q + 8 γ 2 D q q [ 2 ω 0 2 D q q − ( D p p + ω 0 2 D q q ) ( ω 0 2 − 1 ) ] ( 2 Q ) 2 \begin{aligned}
\kappa&= \frac{\gamma(Q_{11}[\omega_0^2+1]+4\gamma[D_{pq}+\gamma D_{qq}])}{2Q}\\
&\quad-\gamma\sqrt{\frac{[Q_{11}(\omega_0^2-1)]^2+[4\gamma(D_{pq}+\gamma D_{qq})]^2-8\gamma Q_{11}(\omega_0^2-1)D_{pq}+8\gamma^2D_{qq}[2\omega_0^2D_{qq}-(D_{pp}+\omega_0^2D_{qq})(\omega_0^2-1)]}{(2Q)^2}}
\end{aligned} κ = 2 Q γ ( Q 11 [ ω 0 2 + 1 ] + 4 γ [ D pq + γ D q q ]) − γ ( 2 Q ) 2 [ Q 11 ( ω 0 2 − 1 ) ] 2 + [ 4 γ ( D pq + γ D q q ) ] 2 − 8 γ Q 11 ( ω 0 2 − 1 ) D pq + 8 γ 2 D q q [ 2 ω 0 2 D q q − ( D pp + ω 0 2 D q q ) ( ω 0 2 − 1 )]
κ = γ ( Q 11 [ ω 0 2 + 1 ] + 4 γ [ D p q + γ D q q ] ) 2 Q − γ [ Q 11 ( ω 0 2 − 1 ) ] 2 + [ 4 γ ( D p q + γ D q q ) ] 2 − 8 γ Q 11 ( ω 0 2 − 1 ) D p q + 8 γ 2 D q q [ 2 ω 0 2 D q q − ω 0 2 D q q ( ω 0 2 − 1 ) − D p p ( ω 0 2 − 1 ) ] ( 2 Q ) 2 \begin{aligned}
\kappa&= \frac{\gamma(Q_{11}[\omega_0^2+1]+4\gamma[D_{pq}+\gamma D_{qq}])}{2Q}\\
&\quad-\gamma\sqrt{\frac{[Q_{11}(\omega_0^2-1)]^2+[4\gamma(D_{pq}+\gamma D_{qq})]^2-8\gamma Q_{11}(\omega_0^2-1)D_{pq}+8\gamma^2D_{qq}[2\omega_0^2D_{qq}-\omega_0^2D_{qq}(\omega_0^2-1)-D_{pp}(\omega_0^2-1)]}{(2Q)^2}}
\end{aligned} κ = 2 Q γ ( Q 11 [ ω 0 2 + 1 ] + 4 γ [ D pq + γ D q q ]) − γ ( 2 Q ) 2 [ Q 11 ( ω 0 2 − 1 ) ] 2 + [ 4 γ ( D pq + γ D q q ) ] 2 − 8 γ Q 11 ( ω 0 2 − 1 ) D pq + 8 γ 2 D q q [ 2 ω 0 2 D q q − ω 0 2 D q q ( ω 0 2 − 1 ) − D pp ( ω 0 2 − 1 )]
κ = γ ( Q 11 [ ω 0 2 + 1 ] + 4 γ [ D p q + γ D q q ] ) 2 Q − γ [ Q 11 ( ω 0 2 − 1 ) ] 2 + [ 4 γ ( D p q + γ D q q ) ] 2 − 8 γ Q 11 ( ω 0 2 − 1 ) D p q + 8 γ 2 D q q [ 2 ω 0 2 D q q − ω 0 2 D q q + ω 0 2 D q q − D p p ( ω 0 2 − 1 ) ] ( 2 Q ) 2 \begin{aligned}
\kappa&= \frac{\gamma(Q_{11}[\omega_0^2+1]+4\gamma[D_{pq}+\gamma D_{qq}])}{2Q}\\
&\quad-\gamma\sqrt{\frac{[Q_{11}(\omega_0^2-1)]^2+[4\gamma(D_{pq}+\gamma D_{qq})]^2-8\gamma Q_{11}(\omega_0^2-1)D_{pq}+8\gamma^2D_{qq}[2\omega_0^2D_{qq}-\omega_0^2D_{qq}+\omega_0^2D_{qq}-D_{pp}(\omega_0^2-1)]}{(2Q)^2}}
\end{aligned} κ = 2 Q γ ( Q 11 [ ω 0 2 + 1 ] + 4 γ [ D pq + γ D q q ]) − γ ( 2 Q ) 2 [ Q 11 ( ω 0 2 − 1 ) ] 2 + [ 4 γ ( D pq + γ D q q ) ] 2 − 8 γ Q 11 ( ω 0 2 − 1 ) D pq + 8 γ 2 D q q [ 2 ω 0 2 D q q − ω 0 2 D q q + ω 0 2 D q q − D pp ( ω 0 2 − 1 )]
κ = γ ( Q 11 [ ω 0 2 + 1 ] + 4 γ [ D p q + γ D q q ] ) 2 Q − γ [ Q 11 ( ω 0 2 − 1 ) ] 2 + [ 4 γ ( D p q + γ D q q ) ] 2 − 8 γ Q 11 ( ω 0 2 − 1 ) D p q + 8 γ 2 D q q [ ω 0 4 D q q + ω 0 2 D q q − D p p ( ω 0 2 − 1 ) ] ( 2 Q ) 2 \kappa= \frac{\gamma\left(Q_{11}\left[\omega_0^2+1\right]+4\gamma\left[D_{pq}+\gamma D_{qq}\right]\right)}{2Q}
-\gamma\sqrt{\frac{\left[Q_{11}\left(\omega_0^2-1\right)\right]^2+\left[4\gamma\left(D_{pq}+\gamma D_{qq}\right)\right]^2-8\gamma Q_{11}\left(\omega_0^2-1\right)D_{pq}+8\gamma^2D_{qq}\left[\omega_0^4D_{qq}+\omega_0^2D_{qq}-D_{pp}\left(\omega_0^2-1\right)\right]}{(2Q)^2}} κ = 2 Q γ ( Q 11 [ ω 0 2 + 1 ] + 4 γ [ D pq + γ D q q ] ) − γ ( 2 Q ) 2 [ Q 11 ( ω 0 2 − 1 ) ] 2 + [ 4 γ ( D pq + γ D q q ) ] 2 − 8 γ Q 11 ( ω 0 2 − 1 ) D pq + 8 γ 2 D q q [ ω 0 4 D q q + ω 0 2 D q q − D pp ( ω 0 2 − 1 ) ]
κ = γ ( Q 11 [ ω 0 2 + 1 ] + 4 γ [ D p q + γ D q q ] ) 2 Q − γ [ Q 11 ( ω 0 2 − 1 ) ] 2 + [ 4 γ ( D p q + γ D q q ) ] 2 − 8 γ Q 11 ( ω 0 2 − 1 ) D p q + 8 γ 2 D q q [ ω 0 2 D q q ( ω 0 2 + 1 ) − D p p ( ω 0 2 − 1 ) ] ( 2 Q ) 2 \kappa= \frac{\gamma\left(Q_{11}\left[\omega_0^2+1\right]+4\gamma\left[D_{pq}+\gamma D_{qq}\right]\right)}{2Q}
-\gamma\sqrt{\frac{\left[Q_{11}\left(\omega_0^2-1\right)\right]^2+\left[4\gamma\left(D_{pq}+\gamma D_{qq}\right)\right]^2-8\gamma Q_{11}\left(\omega_0^2-1\right)D_{pq}+8\gamma^2D_{qq}\left[\omega_0^2D_{qq}\left(\omega_0^2+1\right)-D_{pp}\left(\omega_0^2-1\right)\right]}{(2Q)^2}} κ = 2 Q γ ( Q 11 [ ω 0 2 + 1 ] + 4 γ [ D pq + γ D q q ] ) − γ ( 2 Q ) 2 [ Q 11 ( ω 0 2 − 1 ) ] 2 + [ 4 γ ( D pq + γ D q q ) ] 2 − 8 γ Q 11 ( ω 0 2 − 1 ) D pq + 8 γ 2 D q q [ ω 0 2 D q q ( ω 0 2 + 1 ) − D pp ( ω 0 2 − 1 ) ]
Special case of unit frequency
We consider the special case of ω 0 = 1 \omega_0=1 ω 0 = 1 for simplicity in the calculations. First we will do the d = 1 d=1 d = 1 case and then we will consider d ≫ 3 > 1 d\gg3>1 d ≫ 3 > 1 . In the special case considered, then
Q 11 = D p p + D q q Q_{11}=D_{pp}+D_{qq} Q 11 = D pp + D q q
Q 12 = 2 γ D q q Q_{12}=2\gamma D_{qq} Q 12 = 2 γ D q q
Q 22 = D p p + D q q + 4 γ ( D p q + γ D q q ) = Q 11 + 4 γ ( D p q + γ D q q ) Q_{22}=D_{pp}+D_{qq}+4\gamma(D_{pq}+\gamma D_{qq})=Q_{11}+4\gamma(D_{pq}+\gamma D_{qq}) Q 22 = D pp + D q q + 4 γ ( D pq + γ D q q ) = Q 11 + 4 γ ( D pq + γ D q q )
Q = Q 11 Q 22 − Q 12 2 Q=Q_{11}Q_{22}-Q_{12}^2 Q = Q 11 Q 22 − Q 12 2
Q = Q 11 [ Q 11 + 4 γ ( D p q + γ D q q ) ] − [ 2 γ D q q ] 2 Q=Q_{11}\left[Q_{11}+4\gamma(D_{pq}+\gamma D_{qq})\right]-\left[2\gamma D_{qq}\right]^2 Q = Q 11 [ Q 11 + 4 γ ( D pq + γ D q q ) ] − [ 2 γ D q q ] 2
Q = Q 11 2 + 4 γ ( D p q + γ D q q ) Q 11 − ( 2 γ D q q ) 2 Q=Q_{11}^2+4\gamma(D_{pq}+\gamma D_{qq})Q_{11}-(2\gamma D_{qq})^2 Q = Q 11 2 + 4 γ ( D pq + γ D q q ) Q 11 − ( 2 γ D q q ) 2
Q = ( D p p + D q q ) 2 + 4 γ ( D p q + γ D q q ) ( D p p + D q q ) − ( 2 γ D q q ) 2 Q=(D_{pp}+D_{qq})^2+4\gamma(D_{pq}+\gamma D_{qq})(D_{pp}+D_{qq})-(2\gamma D_{qq})^2 Q = ( D pp + D q q ) 2 + 4 γ ( D pq + γ D q q ) ( D pp + D q q ) − ( 2 γ D q q ) 2
Q = [ ( D p p + D q q ) + 4 γ ( D p q + γ D q q ) ] ( D p p + D q q ) − ( 2 γ D q q ) 2 Q=\left[(D_{pp}+D_{qq})+4\gamma(D_{pq}+\gamma D_{qq})\right](D_{pp}+D_{qq})-(2\gamma D_{qq})^2 Q = [ ( D pp + D q q ) + 4 γ ( D pq + γ D q q ) ] ( D pp + D q q ) − ( 2 γ D q q ) 2
Q = [ ( D p p + D q q ) + 4 γ ( D p q + γ D q q ) ] ( D p p + D q q ) − ( 2 γ D q q ) 2 Q=\left[(D_{pp}+D_{qq})+4\gamma(D_{pq}+\gamma D_{qq})\right](D_{pp}+D_{qq})-(2\gamma D_{qq})^2 Q = [ ( D pp + D q q ) + 4 γ ( D pq + γ D q q ) ] ( D pp + D q q ) − ( 2 γ D q q ) 2
In that case, since ω 0 = 1 \omega_0=1 ω 0 = 1 ,
κ = γ ( Q 11 [ ω 0 2 + 1 ] + 4 γ [ D p q + γ D q q ] ) 2 Q − γ [ Q 11 ( ω 0 2 − 1 ) ] 2 + [ 4 γ ( D p q + γ D q q ) ] 2 − 8 γ Q 11 ( ω 0 2 − 1 ) D p q + 8 γ 2 D q q [ 2 ω 0 4 D q q − Q 11 ( ω 0 2 − 1 ) ] ( 2 Q ) 2 \kappa=\frac{\gamma\left(Q_{11}\left[\omega_0^2+1\right]+4\gamma\left[D_{pq}+\gamma D_{qq}\right]\right)}{2Q}
-\gamma\sqrt{\frac{\left[Q_{11}\left(\omega_0^2-1\right)\right]^2+\left[4\gamma(D_{pq}+\gamma D_{qq})\right]^2-8\gamma Q_{11}(\omega_0^2-1)D_{pq}+8\gamma^2D_{qq}\left[2\omega_0^4D_{qq}-Q_{11}(\omega_0^2-1)\right]}{(2Q)^2}} κ = 2 Q γ ( Q 11 [ ω 0 2 + 1 ] + 4 γ [ D pq + γ D q q ] ) − γ ( 2 Q ) 2 [ Q 11 ( ω 0 2 − 1 ) ] 2 + [ 4 γ ( D pq + γ D q q ) ] 2 − 8 γ Q 11 ( ω 0 2 − 1 ) D pq + 8 γ 2 D q q [ 2 ω 0 4 D q q − Q 11 ( ω 0 2 − 1 ) ]
κ = γ ( 2 Q 11 + 4 γ [ D p q + γ D q q ] ) 2 Q − γ [ 4 γ ( D p q + γ D q q ) ] 2 + 8 γ 2 D q q × 2 ω 0 4 D q q ( 2 Q ) 2 \kappa=\frac{\gamma\left(2Q_{11}+4\gamma[D_{pq}+\gamma D_{qq}]\right)}{2Q}
-\gamma\sqrt{\frac{\left[4\gamma(D_{pq}+\gamma D_{qq})\right]^2+8\gamma^2D_{qq}\times2\omega_0^4D_{qq}}{(2Q)^2}} κ = 2 Q γ ( 2 Q 11 + 4 γ [ D pq + γ D q q ] ) − γ ( 2 Q ) 2 [ 4 γ ( D pq + γ D q q ) ] 2 + 8 γ 2 D q q × 2 ω 0 4 D q q
κ = γ ( Q 11 + 2 γ [ D p q + γ D q q ] ) Q − γ [ 2 γ ( D p q + γ D q q ) ] 2 + 4 γ 2 D q q 2 ω 0 4 Q 2 \kappa=\frac{\gamma\left(Q_{11}+2\gamma[D_{pq}+\gamma D_{qq}]\right)}{Q}
-\gamma\sqrt{\frac{\left[2\gamma(D_{pq}+\gamma D_{qq})\right]^2+4\gamma^2D_{qq}^2\omega_0^4}{Q^2}} κ = Q γ ( Q 11 + 2 γ [ D pq + γ D q q ] ) − γ Q 2 [ 2 γ ( D pq + γ D q q ) ] 2 + 4 γ 2 D q q 2 ω 0 4
κ = γ ( Q 11 + 2 γ [ D p q + γ D q q ] ) Q − γ [ 2 γ ( D p q + γ D q q ) ] 2 + ( 2 γ D q q ω 0 2 ) 2 Q 2 \kappa=\frac{\gamma\left(Q_{11}+2\gamma[D_{pq}+\gamma D_{qq}]\right)}{Q}
-\gamma\sqrt{\frac{\left[2\gamma(D_{pq}+\gamma D_{qq})\right]^2+(2\gamma D_{qq}\omega_0^2)^2}{Q^2}} κ = Q γ ( Q 11 + 2 γ [ D pq + γ D q q ] ) − γ Q 2 [ 2 γ ( D pq + γ D q q ) ] 2 + ( 2 γ D q q ω 0 2 ) 2
κ = γ ( Q 11 + 2 γ [ D p q + γ D q q ] ) Q − 2 γ 2 [ ( D p q + γ D q q ) 2 + ( D q q ω 0 2 ) 2 ] Q 2 \kappa= \frac{\gamma\left(Q_{11}+2\gamma[D_{pq}+\gamma D_{qq}]\right)}{Q}-2\gamma^2\sqrt{\frac{\left[(D_{pq}+\gamma D_{qq})^2+(D_{qq}\omega_0^2)^2\right]}{Q^2}} κ = Q γ ( Q 11 + 2 γ [ D pq + γ D q q ] ) − 2 γ 2 Q 2 [ ( D pq + γ D q q ) 2 + ( D q q ω 0 2 ) 2 ]
κ = γ ( Q 11 + 2 γ [ D p q + γ D q q ] ) Q − 2 γ 2 ∣ Q ∣ [ ( D p q + γ D q q ) 2 + ( D q q ω 0 2 ) 2 ] \kappa= \frac{\gamma\left(Q_{11}+2\gamma[D_{pq}+\gamma D_{qq}]\right)}{Q}-\frac{2\gamma^2}{\lvert Q\rvert}\sqrt{\left[(D_{pq}+\gamma D_{qq})^2+(D_{qq}\omega_0^2)^2\right]} κ = Q γ ( Q 11 + 2 γ [ D pq + γ D q q ] ) − ∣ Q ∣ 2 γ 2 [ ( D pq + γ D q q ) 2 + ( D q q ω 0 2 ) 2 ]
κ = γ ∣ Q ∣ [ sign ( Q ) ( Q 11 + 2 γ [ D p q + γ D q q ] ) − 2 γ [ ( D p q + γ D q q ) 2 + ( D q q ω 0 2 ) 2 ] ] \kappa= \frac{\gamma}{\lvert Q\rvert}\left[\operatorname{sign}(Q)\left(Q_{11}+2\gamma[D_{pq}+\gamma D_{qq}]\right)-2\gamma\sqrt{\left[(D_{pq}+\gamma D_{qq})^2+(D_{qq}\omega_0^2)^2\right]}\right] κ = ∣ Q ∣ γ [ sign ( Q ) ( Q 11 + 2 γ [ D pq + γ D q q ] ) − 2 γ [ ( D pq + γ D q q ) 2 + ( D q q ω 0 2 ) 2 ] ]
so we have the proportionality relation
κ ∝ [ sign ( Q ) ( Q 11 + 2 γ [ D p q + γ D q q ] ) − 2 γ [ ( D p q + γ D q q ) 2 + ( D q q ω 0 2 ) 2 ] ] \kappa\propto\left[\operatorname{sign}(Q)\left(Q_{11}+2\gamma[D_{pq}+\gamma D_{qq}]\right)-2\gamma\sqrt{\left[(D_{pq}+\gamma D_{qq})^2+(D_{qq}\omega_0^2)^2\right]}\right] κ ∝ [ sign ( Q ) ( Q 11 + 2 γ [ D pq + γ D q q ] ) − 2 γ [ ( D pq + γ D q q ) 2 + ( D q q ω 0 2 ) 2 ] ]
Mixing time: optimal exponential rate
We show the preliminary calculations for high dimensions d ≫ 1 d \gg1 d ≫ 1 . We consider x ∈ R d x \in\mathbb{R}^d x ∈ R d , p ∈ R d p \in\mathbb{R}^d p ∈ R d , so ( x , p ) ∈ R 2 d (x,p) \in\mathbb{R}^{2d} ( x , p ) ∈ R 2 d , and therefore the diffusion matrix is the block matrix
D = ( D q q I d D p q I d D p q I d D p p I d ) , D p q = D q p , D=\begin{pmatrix}D_{qq}I_d & D_{pq}I_d \\ D_{pq}I_d & D_{pp}I_d\end{pmatrix},\qquad D_{pq}=D_{qp}, D = ( D q q I d D pq I d D pq I d D pp I d ) , D pq = D q p ,
where D i j ∈ R d × d D_{ij}\in\mathbb{R}^{d\times d} D ij ∈ R d × d , i , j = q , p i,j=q,p i , j = q , p . As mentioned by Sparber et al. in [6 ] , the optimal exponential rate κ \kappa κ is the smalles eigenvalue of the Hessian of A ~ \widetilde{A} A . The potential A ~ \widetilde{A} A is defined by
A ~ ( y ~ ) = A ( D y T ) , \widetilde{A}(\widetilde{y})=A(\sqrt{D}y^T), A ( y ) = A ( D y T ) ,
with D \sqrt{D} D the square root of D D D in the sense of positive definite matrices, and y = ( x , p ) y=(x,p) y = ( x , p ) .
We know the form of A A A for the steady state, given by
A ( x , p ) = γ Q [ Q 11 ω 0 2 x 2 + 2 Q 12 ω 0 x ⋅ p + Q 22 p 2 ] A(x,p)=\frac{\gamma}{Q}\left[Q_{11}\omega_0^2x^2+2Q_{12}\omega_0x\cdot p+Q_{22}p^2\right] A ( x , p ) = Q γ [ Q 11 ω 0 2 x 2 + 2 Q 12 ω 0 x ⋅ p + Q 22 p 2 ]
Q 11 = D p p + ω 0 2 D q q Q_{11}=D_{pp}+\omega_0^2D_{qq} Q 11 = D pp + ω 0 2 D q q
Q 12 = 2 ω 0 γ D q q Q_{12}=2\omega_0\gamma D_{qq} Q 12 = 2 ω 0 γ D q q
Q 22 = D p p + ω 0 2 D q q + 4 γ ( D p q + γ D q q ) = Q 11 + 4 γ ( D p q + γ D q q ) Q_{22}=D_{pp}+\omega_0^2D_{qq}+4\gamma(D_{pq}+\gamma D_{qq})=Q_{11}+4\gamma(D_{pq}+\gamma D_{qq}) Q 22 = D pp + ω 0 2 D q q + 4 γ ( D pq + γ D q q ) = Q 11 + 4 γ ( D pq + γ D q q )
Q = Q 11 Q 22 − Q 12 2 Q=Q_{11}Q_{22}-Q_{12}^2 Q = Q 11 Q 22 − Q 12 2
Now, if we consider
A ( x , p ) = ( x , p ) [ γ Q ( Q 11 ω 0 2 Q 12 ω 0 Q 12 ω 0 Q 22 ) ] ( x , p ) T A(x,p)=(x,p)\left[\frac{\gamma}{Q}\begin{pmatrix}Q_{11}\omega_0^2 & Q_{12}\omega_0 \\ Q_{12}\omega_0 & Q_{22}\end{pmatrix}\right](x,p)^T A ( x , p ) = ( x , p ) [ Q γ ( Q 11 ω 0 2 Q 12 ω 0 Q 12 ω 0 Q 22 ) ] ( x , p ) T
A ( y T ) = y [ γ Q ( Q 11 ω 0 2 Q 12 ω 0 Q 12 ω 0 Q 22 ) ] y T , y = ( x , p ) A(y^T)=y\left[\frac{\gamma}{Q}\begin{pmatrix}Q_{11}\omega_0^2 & Q_{12}\omega_0 \\ Q_{12}\omega_0 & Q_{22}\end{pmatrix}\right]y^T,\qquad y=(x,p) A ( y T ) = y [ Q γ ( Q 11 ω 0 2 Q 12 ω 0 Q 12 ω 0 Q 22 ) ] y T , y = ( x , p )
and
A ~ ( y ~ ) = A ( D y T ) \widetilde{A}(\widetilde{y})=A(\sqrt{D}y^T) A ( y ) = A ( D y T )
then, using the one square root D \sqrt{D} D that is both positive semidefinite and symmetric, then
A ~ ( y ~ ) = y ~ [ γ Q ( Q 11 ω 0 2 Q 12 ω 0 Q 12 ω 0 Q 22 ) ] y ~ T \widetilde{A}(\widetilde{y})=\widetilde{y}\left[\frac{\gamma}{Q}\begin{pmatrix}Q_{11}\omega_0^2 & Q_{12}\omega_0 \\ Q_{12}\omega_0 & Q_{22}\end{pmatrix}\right]\widetilde{y}^{T} A ( y ) = y [ Q γ ( Q 11 ω 0 2 Q 12 ω 0 Q 12 ω 0 Q 22 ) ] y T
A ~ ( y ~ ) = y D [ γ Q ( Q 11 ω 0 2 Q 12 ω 0 Q 12 ω 0 Q 22 ) ] D y T \widetilde{A}(\widetilde{y})=y\sqrt{D}\left[\frac{\gamma}{Q}\begin{pmatrix}Q_{11}\omega_0^2 & Q_{12}\omega_0 \\ Q_{12}\omega_0 & Q_{22}\end{pmatrix}\right]\sqrt{D}y^{T} A ( y ) = y D [ Q γ ( Q 11 ω 0 2 Q 12 ω 0 Q 12 ω 0 Q 22 ) ] D y T
A ( D y T ) = ( x , p ) [ γ Q D ( Q 11 ω 0 2 Q 12 ω 0 Q 12 ω 0 Q 22 ) D ] ( x , p ) T A(\sqrt{D}y^{T})=(x,p)\left[\frac{\gamma}{Q}\sqrt{D}\begin{pmatrix}Q_{11}\omega_0^2 & Q_{12}\omega_0 \\ Q_{12}\omega_0 & Q_{22}\end{pmatrix}\sqrt{D}\right](x,p)^{T} A ( D y T ) = ( x , p ) [ Q γ D ( Q 11 ω 0 2 Q 12 ω 0 Q 12 ω 0 Q 22 ) D ] ( x , p ) T
and since we need
∂ 2 A ~ ( y ~ ) ∂ y ~ i y ~ j , i , j = 1 , … , 2 d \frac{\partial^{2}\widetilde{A}(\widetilde{y})}{\partial\widetilde{y}_{i}\widetilde{y}_{j}},\qquad i,j=1,\ldots,2d ∂ y i y j ∂ 2 A ( y ) , i , j = 1 , … , 2 d
to obtain the smallest eigenvalue of the Hessian of A ~ \widetilde{A} A under the transformed coordinates, we can just compute
∂ 2 A ( y ) ∂ y i y j \frac{\partial^{2}A(y)}{\partial y_{i}y_{j}} ∂ y i y j ∂ 2 A ( y )
as it will render the same Hessian and eigenvalue under the previous coordinates. So we have
H = Hessian ( A ) = γ Q ( Q 11 ω 0 2 I d Q 12 ω 0 I d Q 12 ω 0 I d Q 22 I d ) H=\operatorname{Hessian}(A)=\frac{\gamma}{Q}\begin{pmatrix}Q_{11}\omega_0^2 I_d & Q_{12}\omega_0 I_d \\ Q_{12}\omega_0 I_d & Q_{22}I_d\end{pmatrix} H = Hessian ( A ) = Q γ ( Q 11 ω 0 2 I d Q 12 ω 0 I d Q 12 ω 0 I d Q 22 I d )
with
Q 11 = D p p + ω 0 2 D q q Q_{11}=D_{pp}+\omega_0^2D_{qq} Q 11 = D pp + ω 0 2 D q q
Q 12 = 2 ω 0 γ D q q Q_{12}=2\omega_0\gamma D_{qq} Q 12 = 2 ω 0 γ D q q
Q 22 = D p p + ω 0 2 D q q + 4 γ ( D p q + γ D q q ) = Q 11 + 4 γ ( D p q + γ D q q ) Q_{22}=D_{pp}+\omega_0^2D_{qq}+4\gamma(D_{pq}+\gamma D_{qq})=Q_{11}+4\gamma(D_{pq}+\gamma D_{qq}) Q 22 = D pp + ω 0 2 D q q + 4 γ ( D pq + γ D q q ) = Q 11 + 4 γ ( D pq + γ D q q )
Q = Q 11 Q 22 − Q 12 2 Q=Q_{11}Q_{22}-Q_{12}^{2} Q = Q 11 Q 22 − Q 12 2
To find the smallest eigenvalue of the Hessian, we would need to solve the characteristic polynomial. Though this is manageable for the usual physical dimensions 1 ≤ d ≤ 3 1\le d\le3 1 ≤ d ≤ 3 , for high dimensions this might be unfeasible unless the structure of H H H is simple. Therefore, let’s study the case
0 = Q 12 = 2 ω 0 γ D q q 0=Q_{12}=2\omega_0\gamma D_{qq} 0 = Q 12 = 2 ω 0 γ D q q
and since we don’t want to make γ = 0 \gamma=0 γ = 0 since the steady state would be trivial, neither ω 0 = 0 \omega_0=0 ω 0 = 0 since the harmonic potential would vanish, let’s choose
which is somehow equivalent to a Caldeira-Legget model for γ > 0 \gamma>0 γ > 0 . Though this is unphysical, since the Caldeira-Legget model is known to not satisfy the Lindblad condition D p p D q q − D p q 2 = det ( D ) ≥ ( γ / 2 ) 2 D_{pp}D_{qq}-D_{pq}^{2}=\det(D)\ge(\gamma/2)^{2} D pp D q q − D pq 2 = det ( D ) ≥ ( γ /2 ) 2 , for the sake of simplicity we will proceed to perform its theoretical analysis. Then we have in that case
Q 11 = D p p Q_{11}=D_{pp} Q 11 = D pp
Q 22 = D p p + 4 γ D p q Q_{22}=D_{pp}+4\gamma D_{pq} Q 22 = D pp + 4 γ D pq
Q = Q 11 Q 22 = D p p 2 + 4 γ D p q D p p Q=Q_{11}Q_{22}=D_{pp}^{2}+4\gamma D_{pq}D_{pp} Q = Q 11 Q 22 = D pp 2 + 4 γ D pq D pp
but in that case
H = Hessian ( A ) = γ Q ( Q 11 ω 0 2 I d 0 I d 0 I d Q 22 I d ) H=\operatorname{Hessian}(\mathcal{A})=\frac{\gamma}{Q}\begin{pmatrix}Q_{11}\omega_{0}^{2}I_{d}&0I_{d}\\0I_{d}&Q_{22}I_{d}\end{pmatrix} H = Hessian ( A ) = Q γ ( Q 11 ω 0 2 I d 0 I d 0 I d Q 22 I d )
H = γ Q ( D p p ω 0 2 I d 0 I d 0 I d ( D p p 2 + 4 γ D p q D p p ) I d ) H=\frac{\gamma}{Q}\begin{pmatrix}D_{pp}\omega_{0}^{2}I_{d}&0I_{d}\\0I_{d}&(D_{pp}^{2}+4\gamma D_{pq}D_{pp})I_{d}\end{pmatrix} H = Q γ ( D pp ω 0 2 I d 0 I d 0 I d ( D pp 2 + 4 γ D pq D pp ) I d )
The eigenvalues of H H H are clearly
λ 1 = γ Q D p p ω 0 2 \lambda_{1}=\frac{\gamma}{Q}D_{pp}\omega_{0}^{2} λ 1 = Q γ D pp ω 0 2
λ 2 = γ Q ( D p p 2 + 4 γ D p q D p p ) \lambda_{2}=\frac{\gamma}{Q}(D_{pp}^{2}+4\gamma D_{pq}D_{pp}) λ 2 = Q γ ( D pp 2 + 4 γ D pq D pp )
Though we still have to determine the smallest root, it is clear that fact will be independent of the dimension d d d of the problem. So we proceed to compare by looking for the case in which they are equal
0 = γ Q D p p ( D p p + 4 γ D p q − ω 0 2 ) 0=\frac{\gamma}{Q}D_{pp}(D_{pp}+4\gamma D_{pq}-\omega_{0}^{2}) 0 = Q γ D pp ( D pp + 4 γ D pq − ω 0 2 )
and avoiding the trivial case D p p = 0 D_{pp}=0 D pp = 0 (since then the eigenvalues are equally zero, so there would be no exponential convergence), then which is the smallest root is determined by
D p p + 4 γ D p q vs. ω 0 2 D_{pp}+4\gamma D_{pq}\quad\text{vs.}\quad\omega_{0}^{2} D pp + 4 γ D pq vs. ω 0 2
depending on which case one would have
κ = λ i , i = 1 or 2. \kappa=\lambda_{i},\quad i=1\quad\text{or}\quad2. κ = λ i , i = 1 or 2.
Optimal exponential rate beyond Caldeira-Legget: case Q 11 = Q 22 Q_{11}=Q_{22} Q 11 = Q 22
Though formally we have
H = Hessian ( A ) = γ Q ( Q 11 ω 0 2 I d Q 12 ω 0 I d Q 12 ω 0 I d Q 22 I d ) H=\operatorname{Hessian}(\mathcal{A})=\frac{\gamma}{Q}\begin{pmatrix}Q_{11}\omega_{0}^{2}I_{d}&Q_{12}\omega_{0}I_{d}\\Q_{12}\omega_{0}I_{d}&Q_{22}I_{d}\end{pmatrix} H = Hessian ( A ) = Q γ ( Q 11 ω 0 2 I d Q 12 ω 0 I d Q 12 ω 0 I d Q 22 I d )
with
Q 11 = D p p + ω 0 2 D q q Q_{11}=D_{pp}+\omega_{0}^{2}D_{qq} Q 11 = D pp + ω 0 2 D q q
Q 12 = 2 ω 0 γ D q q Q_{12}=2\omega_{0}\gamma D_{qq} Q 12 = 2 ω 0 γ D q q
Q 22 = D p p + ω 0 2 D q q + 4 γ ( D p q + γ D q q ) = Q 11 + 4 γ ( D p q + γ D q q ) Q_{22}=D_{pp}+\omega_{0}^{2}D_{qq}+4\gamma(D_{pq}+\gamma D_{qq})=Q_{11}+4\gamma(D_{pq}+\gamma D_{qq}) Q 22 = D pp + ω 0 2 D q q + 4 γ ( D pq + γ D q q ) = Q 11 + 4 γ ( D pq + γ D q q )
Q = Q 11 Q 22 − Q 12 2 Q=Q_{11}Q_{22}-Q_{12}^{2} Q = Q 11 Q 22 − Q 12 2
the particular case with Q 11 = Q 22 Q_{11}=Q_{22} Q 11 = Q 22 , which entails (γ = 0 \gamma=0 γ = 0 is discarded as that trivial case makes the polynomial in the Gaussian vanish)
0 = 4 γ ( D p q + γ D q q ) ⟹ D p q = − γ D q q 0=4\gamma(D_{pq}+\gamma D_{qq})\Longrightarrow D_{pq}=-\gamma D_{qq} 0 = 4 γ ( D pq + γ D q q ) ⟹ D pq = − γ D q q
D = ( D q q − γ D q q − γ D q q D p p ) = D q q ( 1 − γ − γ D p p / D q q ) D=\begin{pmatrix} D_{qq} & -\gamma D_{qq} \\ -\gamma D_{qq} & D_{pp} \end{pmatrix}=D_{qq}\begin{pmatrix} 1 & -\gamma\\ -\gamma& D_{pp}/D_{qq} \end{pmatrix} D = ( D q q − γ D q q − γ D q q D pp ) = D q q ( 1 − γ − γ D pp / D q q )
would render a very simple matrix
H = γ Q ( Q 11 ω 0 2 I d Q 12 ω 0 I d Q 12 ω 0 I d Q 11 I d ) H=\frac{\gamma}{Q}\begin{pmatrix} Q_{11}\omega_0^2 I_d & Q_{12}\omega_0 I_d \\ Q_{12}\omega_0 I_d & Q_{11}I_d \end{pmatrix} H = Q γ ( Q 11 ω 0 2 I d Q 12 ω 0 I d Q 12 ω 0 I d Q 11 I d )
in which the transformation of coordinates (which amounts to some sort of linear combination of position and momentum operators: in properly normalized coordinates, is mixing the position variables with their respective canonically conjugated momentum variables)
w j = y j + y j + d , z j = y j − y j + d , j = 1 , … , d w_j=y_j+y_{j+d},\quad z_j=y_j-y_{j+d},\quad j=1,\ldots,d w j = y j + y j + d , z j = y j − y j + d , j = 1 , … , d
would render the matrix representation of H H H in the new basis as
[ H ] { w , z } = γ Q ( [ Q 11 ω 0 2 + Q 12 ω 0 ] I d 0 I d 0 I d [ Q 11 ω 0 2 − Q 12 ω 0 ] I d ) [H]_{\{w,z\}}=\frac{\gamma}{Q}\begin{pmatrix} [Q_{11}\omega_0^2+Q_{12}\omega_0]I_d & 0I_d \\ 0I_d & [Q_{11}\omega_0^2-Q_{12}\omega_0]I_d \end{pmatrix} [ H ] { w , z } = Q γ ( [ Q 11 ω 0 2 + Q 12 ω 0 ] I d 0 I d 0 I d [ Q 11 ω 0 2 − Q 12 ω 0 ] I d )
[ H ] { w , z } = γ Q ω 0 ( [ Q 11 ω 0 + Q 12 ] I d 0 I d 0 I d [ Q 11 ω 0 − Q 12 ] I d ) [H]_{\{w,z\}}=\frac{\gamma}{Q}\omega_0\begin{pmatrix} [Q_{11}\omega_0+Q_{12}]I_d & 0I_d \\ 0I_d & [Q_{11}\omega_0-Q_{12}]I_d \end{pmatrix} [ H ] { w , z } = Q γ ω 0 ( [ Q 11 ω 0 + Q 12 ] I d 0 I d 0 I d [ Q 11 ω 0 − Q 12 ] I d )
and in these new coordinates,
( w , z ) T = M − 1 y T (w,z)^T=M^{-1}y^T ( w , z ) T = M − 1 y T
M − 1 = ( I d I d I d − I d ) M^{-1}=\begin{pmatrix} I_d & I_d \\ I_d & -I_d \end{pmatrix} M − 1 = ( I d I d I d − I d )
y T = M ( w , z ) T y^T=M(w,z)^T y T = M ( w , z ) T
M = 2 M − 1 = 2 ( I d I d I d − I d ) , M = M T M=2M^{-1}=2\begin{pmatrix} I_d & I_d \\ I_d & -I_d \end{pmatrix},\quad M=M^T M = 2 M − 1 = 2 ( I d I d I d − I d ) , M = M T
A ( y T ) = y [ γ Q ( Q 11 ω 0 2 Q 12 ω 0 Q 12 ω 0 Q 22 ) ] y T , y = ( x , p ) A(y^T)=y\left[\frac{\gamma}{Q}\begin{pmatrix} Q_{11}\omega_0^2 & Q_{12}\omega_0 \\ Q_{12}\omega_0 & Q_{22} \end{pmatrix}\right]y^T,\quad y=(x,p) A ( y T ) = y [ Q γ ( Q 11 ω 0 2 Q 12 ω 0 Q 12 ω 0 Q 22 ) ] y T , y = ( x , p )
A ( y T ) = ( w , z ) M [ γ Q ( Q 11 ω 0 2 Q 12 ω 0 Q 12 ω 0 Q 22 ) ] M ( w , z ) T , y = ( x , p ) A(y^T)=(w,z)M\left[\frac{\gamma}{Q}\begin{pmatrix} Q_{11}\omega_0^2 & Q_{12}\omega_0 \\ Q_{12}\omega_0 & Q_{22} \end{pmatrix}\right]M(w,z)^T,\quad y=(x,p) A ( y T ) = ( w , z ) M [ Q γ ( Q 11 ω 0 2 Q 12 ω 0 Q 12 ω 0 Q 22 ) ] M ( w , z ) T , y = ( x , p )
and
A ~ ( y ~ ) = A ( D y T ) \widetilde{A}(\widetilde{y})=A(\sqrt{D}y^T) A ( y ) = A ( D y T )
then, using the one square root D \sqrt{D} D that is both positive semidefinite and symmetric, then
A ( D y T ) = ( w , z ) M D [ γ Q ( Q 11 ω 0 2 Q 12 ω 0 Q 12 ω 0 Q 22 ) ] D M ( w , z ) T , y = ( x , p ) A(\sqrt{D}y^T)=(w,z)M\sqrt{D}\left[\frac{\gamma}{Q}\begin{pmatrix} Q_{11}\omega_0^2 & Q_{12}\omega_0 \\ Q_{12}\omega_0 & Q_{22} \end{pmatrix}\right]\sqrt{D}M(w,z)^T,\quad y=(x,p) A ( D y T ) = ( w , z ) M D [ Q γ ( Q 11 ω 0 2 Q 12 ω 0 Q 12 ω 0 Q 22 ) ] D M ( w , z ) T , y = ( x , p )
Optimal exponential rate beyond Caldeira-Legget: case Q 11 = Q 22 Q_{11}=Q_{22} Q 11 = Q 22
A ~ ( y ~ ) = y ~ T γ Q ( Q 11 ω 0 2 Q 12 ω 0 Q 12 ω 0 Q 22 ) y ~ \widetilde{A}(\widetilde{y})=\widetilde{y}^{T}\frac{\gamma}{Q}\begin{pmatrix}Q_{11}\omega_{0}^{2} & Q_{12}\omega_{0}\\ Q_{12}\omega_{0} & Q_{22}\end{pmatrix}\widetilde{y} A ( y ) = y T Q γ ( Q 11 ω 0 2 Q 12 ω 0 Q 12 ω 0 Q 22 ) y
A ~ ( y ~ ) = y D γ Q ( Q 11 ω 0 2 Q 12 ω 0 Q 12 ω 0 Q 22 ) D y T \widetilde{A}(\widetilde{y})=y\sqrt{D}\frac{\gamma}{Q}\begin{pmatrix}Q_{11}\omega_{0}^{2} & Q_{12}\omega_{0}\\ Q_{12}\omega_{0} & Q_{22}\end{pmatrix}\sqrt{D}y^{T} A ( y ) = y D Q γ ( Q 11 ω 0 2 Q 12 ω 0 Q 12 ω 0 Q 22 ) D y T
A ( D y T ) = ( x , p ) [ γ Q D ( Q 11 ω 0 2 Q 12 ω 0 Q 12 ω 0 Q 22 ) D ] ( x , p ) T A(\sqrt{D}y^{T})=(x,p)\left[\frac{\gamma}{Q}\sqrt{D}\begin{pmatrix}Q_{11}\omega_{0}^{2} & Q_{12}\omega_{0}\\ Q_{12}\omega_{0} & Q_{22}\end{pmatrix}\sqrt{D}\right](x,p)^{T} A ( D y T ) = ( x , p ) [ Q γ D ( Q 11 ω 0 2 Q 12 ω 0 Q 12 ω 0 Q 22 ) D ] ( x , p ) T
and since we need
∂ 2 A ~ ( y ~ ) ∂ y ~ i y ~ j , i , j = 1 , … , 2 d \frac{\partial^{2}\widetilde{A}(\widetilde{y})}{\partial\widetilde{y}_{i}\widetilde{y}_{j}},\qquad i,j=1,\ldots,2d ∂ y i y j ∂ 2 A ( y ) , i , j = 1 , … , 2 d
to obtain the smallest eigenvalue of the Hessian of A ~ \widetilde{A} A under the transformed coordinates, we can just compute
∂ 2 A ( y ) ∂ y i y j \frac{\partial^{2}A(y)}{\partial y_{i}y_{j}} ∂ y i y j ∂ 2 A ( y )
as it will render the same Hessian and eigenvalue under the previous coordinates. So we have
H = Hessian ( A ) = γ Q ( Q 11 ω 0 2 I d Q 12 ω 0 I d Q 12 ω 0 I d Q 11 I d ) H=\operatorname{Hessian}(A)=\frac{\gamma}{Q}\begin{pmatrix}Q_{11}\omega_{0}^{2}I_{d} & Q_{12}\omega_{0}I_{d}\\ Q_{12}\omega_{0}I_{d} & Q_{11}I_{d}\end{pmatrix} H = Hessian ( A ) = Q γ ( Q 11 ω 0 2 I d Q 12 ω 0 I d Q 12 ω 0 I d Q 11 I d )
which by transformation of coordinates y → ( w , z ) y\to(w,z) y → ( w , z ) we can reorganize as
H = Hessian ( A ) = γ Q ( [ Q 11 ω 0 2 + Q 12 ω 0 ] I d 0 d 0 d [ Q 11 ω 0 2 − Q 12 ω 0 ] I d ) H=\operatorname{Hessian}(A)=\frac{\gamma}{Q}\begin{pmatrix}[Q_{11}\omega_{0}^{2}+Q_{12}\omega_{0}]I_{d} & 0_{d}\\ 0_{d} & [Q_{11}\omega_{0}^{2}-Q_{12}\omega_{0}]I_{d}\end{pmatrix} H = Hessian ( A ) = Q γ ( [ Q 11 ω 0 2 + Q 12 ω 0 ] I d 0 d 0 d [ Q 11 ω 0 2 − Q 12 ω 0 ] I d )
H = Hessian ( A ) = γ Q ( [ Q 11 ω 0 + Q 12 ] ω 0 I d 0 d 0 d [ Q 11 ω 0 − Q 12 ] ω 0 I d ) H=\operatorname{Hessian}(A)=\frac{\gamma}{Q}\begin{pmatrix}[Q_{11}\omega_{0}+Q_{12}]\omega_{0}I_{d} & 0_{d}\\ 0_{d} & [Q_{11}\omega_{0}-Q_{12}]\omega_{0}I_{d}\end{pmatrix} H = Hessian ( A ) = Q γ ( [ Q 11 ω 0 + Q 12 ] ω 0 I d 0 d 0 d [ Q 11 ω 0 − Q 12 ] ω 0 I d )
and in which case the characteristic polynomial of this Hessian would be
[ Q 11 ω 0 2 + Q 12 ω 0 − λ ] [ Q 11 ω 0 2 − Q 12 ω 0 − λ ] = 0 [Q_{11}\omega_{0}^{2}+Q_{12}\omega_{0}-\lambda][Q_{11}\omega_{0}^{2}-Q_{12}\omega_{0}-\lambda]=0 [ Q 11 ω 0 2 + Q 12 ω 0 − λ ] [ Q 11 ω 0 2 − Q 12 ω 0 − λ ] = 0
( λ − Q 11 ω 0 2 − Q 12 ω 0 ) ( λ − Q 11 ω 0 2 + Q 12 ω 0 ) = 0 (\lambda-Q_{11}\omega_{0}^{2}-Q_{12}\omega_{0})(\lambda-Q_{11}\omega_{0}^{2}+Q_{12}\omega_{0})=0 ( λ − Q 11 ω 0 2 − Q 12 ω 0 ) ( λ − Q 11 ω 0 2 + Q 12 ω 0 ) = 0
so the optimal exponential convergence rate (mixing time) is the smallest of the two possible roots
λ 1 = γ Q [ Q 11 ω 0 + Q 12 ] ω 0 \lambda_{1}=\frac{\gamma}{Q}[Q_{11}\omega_{0}+Q_{12}]\omega_{0} λ 1 = Q γ [ Q 11 ω 0 + Q 12 ] ω 0
λ 2 = γ Q [ Q 11 ω 0 − Q 12 ] ω 0 \lambda_{2}=\frac{\gamma}{Q}[Q_{11}\omega_{0}-Q_{12}]\omega_{0} λ 2 = Q γ [ Q 11 ω 0 − Q 12 ] ω 0
and since
Q 11 = D p p + ω 0 2 D q q Q_{11}=D_{pp}+\omega_{0}^{2}D_{qq} Q 11 = D pp + ω 0 2 D q q
Q 12 = 2 ω 0 γ D q q Q_{12}=2\omega_{0}\gamma D_{qq} Q 12 = 2 ω 0 γ D q q
then
λ 1 = γ Q [ ( D p p + ω 0 2 D q q ) ω 0 + 2 ω 0 γ D q q ] ω 0 \lambda_{1}=\frac{\gamma}{Q}[(D_{pp}+\omega_{0}^{2}D_{qq})\omega_{0}+2\omega_{0}\gamma D_{qq}]\omega_{0} λ 1 = Q γ [( D pp + ω 0 2 D q q ) ω 0 + 2 ω 0 γ D q q ] ω 0
λ 2 = γ Q [ ( D p p + ω 0 2 D q q ) ω 0 − 2 ω 0 γ D q q ] ω 0 \lambda_2 = \frac{\gamma}{Q}\left[(D_{pp}+\omega_0^2D_{qq})\omega_0-2\omega_0\gamma D_{qq}\right]\omega_0 λ 2 = Q γ [ ( D pp + ω 0 2 D q q ) ω 0 − 2 ω 0 γ D q q ] ω 0
so
λ 1 = γ Q [ D p p + ( ω 0 2 + 2 γ ) D q q ] ω 0 2 \lambda_1 = \frac{\gamma}{Q}\left[D_{pp}+(\omega_0^2+2\gamma)D_{qq}\right]\omega_0^2 λ 1 = Q γ [ D pp + ( ω 0 2 + 2 γ ) D q q ] ω 0 2
λ 2 = γ Q [ D p p + ( ω 0 2 − 2 γ ) D q q ] ω 0 2 \lambda_2 = \frac{\gamma}{Q}\left[D_{pp}+(\omega_0^2-2\gamma)D_{qq}\right]\omega_0^2 λ 2 = Q γ [ D pp + ( ω 0 2 − 2 γ ) D q q ] ω 0 2
therefore in this special case Q 11 = Q 22 ⟺ D p q = − γ D q q Q_{11}=Q_{22} \Longleftrightarrow D_{pq}=-\gamma D_{qq} Q 11 = Q 22 ⟺ D pq = − γ D q q we have that the optimal exponential convergence rate to the steady state is
κ = γ Q [ D p p + ( ω 0 2 ± 2 γ ) D q q ] ω 0 2 \kappa= \frac{\gamma}{Q}\left[D_{pp}+(\omega_0^2\pm2\gamma)D_{qq}\right]\omega_0^2 κ = Q γ [ D pp + ( ω 0 2 ± 2 γ ) D q q ] ω 0 2
which is also independent of the dimension d d d of the problem in this special case.
General case
General case for Hessian without perturbative approximations.
To consider the general case of
H = Hessian ( A ) = γ Q ( Q 11 ω 0 2 I d Q 12 ω 0 I d Q 12 ω 0 I d Q 22 I d ) H=\operatorname{Hessian}(A)=\frac{\gamma}{Q}
\begin{pmatrix}
Q_{11}\omega_0^2I_d & Q_{12}\omega_0I_d\\
Q_{12}\omega_0I_d & Q_{22}I_d
\end{pmatrix} H = Hessian ( A ) = Q γ ( Q 11 ω 0 2 I d Q 12 ω 0 I d Q 12 ω 0 I d Q 22 I d )
one can always do a change of variables (re-scaling mostly), such that, for z = L y z=Ly z = L y (where L L L is not only linear but diagonal and amounting to rescale each one of the respective coordinates), one could have (in fact part of the rescaling could help to make ω 0 = 1 \omega_0=1 ω 0 = 1 )
H = Hessian ( A ) = γ Q ( q 11 I d q 12 I d q 12 I d q 11 I d ) H=\operatorname{Hessian}(A)=\frac{\gamma}{Q}
\begin{pmatrix}
q_{11}I_d & q_{12}I_d\\
q_{12}I_d & q_{11}I_d
\end{pmatrix} H = Hessian ( A ) = Q γ ( q 11 I d q 12 I d q 12 I d q 11 I d )
in which case
κ = γ Q ( q 11 ± q 12 ) \kappa=\frac{\gamma}{Q}(q_{11}\pm q_{12}) κ = Q γ ( q 11 ± q 12 )
κ = γ Q [ D p p + ( 1 ± 2 γ ) D q q ] \kappa=\frac{\gamma}{Q}\left[D_{pp}+(1\pm2\gamma)D_{qq}\right] κ = Q γ [ D pp + ( 1 ± 2 γ ) D q q ]
which is independent of the dimension d d d of the vector variables. In fact, the rescaling can be performed in the momentum variables p j = y d + j p_j=y_{d+j} p j = y d + j , j = 1 , ⋯ , d j=1,\cdots,d j = 1 , ⋯ , d . By choosing L = diag ( 1 , ⋯ , 1 , Q 11 / Q 22 , ⋯ , Q 11 / Q 22 ) L=\operatorname{diag}(1,\cdots,1,Q_{11}/Q_{22},\cdots,Q_{11}/Q_{22}) L = diag ( 1 , ⋯ , 1 , Q 11 / Q 22 , ⋯ , Q 11 / Q 22 ) , one would get something equivalent to Q 11 = Q 22 Q_{11}=Q_{22} Q 11 = Q 22 from the beginning, as in q 11 = q 22 q_{11}=q_{22} q 11 = q 22 , and then this particular case could handle the general one. In fact, let’s go back to the expression of the Gaussian steady state in the Wigner formulation for WFP under a harmonic potential:
A ( x , p ) = γ Q [ Q 11 ω 0 2 x 2 + 2 Q 12 ω 0 x ⋅ p + Q 22 p 2 ] A(x,p)=\frac{\gamma}{Q}\left[Q_{11}\omega_0^2x^2+2Q_{12}\omega_0x\cdot p+Q_{22}p^2\right] A ( x , p ) = Q γ [ Q 11 ω 0 2 x 2 + 2 Q 12 ω 0 x ⋅ p + Q 22 p 2 ]
Q 11 = D p p + ω 0 2 D q q Q_{11}=D_{pp}+\omega_0^2D_{qq} Q 11 = D pp + ω 0 2 D q q
Q 12 = 2 ω 0 γ D q q Q_{12}=2\omega_0\gamma D_{qq} Q 12 = 2 ω 0 γ D q q
General case
Q 22 = D p p + ω 0 2 D q q + 4 γ ( D p q + γ D q q ) = Q 11 + 4 γ ( D p q + γ D q q ) Q_{22}=D_{pp}+\omega_0^2D_{qq}+4\gamma(D_{pq}+\gamma D_{qq})=Q_{11}+4\gamma(D_{pq}+\gamma D_{qq}) Q 22 = D pp + ω 0 2 D q q + 4 γ ( D pq + γ D q q ) = Q 11 + 4 γ ( D pq + γ D q q )
Q = Q 11 Q 22 − Q 12 2 Q=Q_{11}Q_{22}-Q_{12}^2 Q = Q 11 Q 22 − Q 12 2
Let’s consider the change of variables
in which case we have
A ( ω 0 x , p ) = γ Q [ Q 11 ( ω 0 x ) 2 + 2 Q 12 ( ω 0 x ) ⋅ p + Q 22 p 2 ] A(\omega_0x,p)=\frac{\gamma}{Q}[Q_{11}(\omega_0x)^2+2Q_{12}(\omega_0x)\cdot p+Q_{22}p^2] A ( ω 0 x , p ) = Q γ [ Q 11 ( ω 0 x ) 2 + 2 Q 12 ( ω 0 x ) ⋅ p + Q 22 p 2 ]
which is equivalent to take ω 0 = 1 \omega_0=1 ω 0 = 1 , in which case
A ( x , p ) = γ Q [ Q 11 x 2 + 2 Q 12 x ⋅ p + Q 22 p 2 ] A(x,p)=\frac{\gamma}{Q}[Q_{11}x^2+2Q_{12}x\cdot p+Q_{22}p^2] A ( x , p ) = Q γ [ Q 11 x 2 + 2 Q 12 x ⋅ p + Q 22 p 2 ]
Let’s assume for the moment the ratio Q 22 / Q 11 Q_{22}/Q_{11} Q 22 / Q 11 is non-negative (meaning that these two numbers don’t have opposite sign). In that case,
A ( x , p ) = γ Q 11 Q [ x 2 + 2 Q 12 Q 11 x ⋅ p + Q 22 Q 11 p 2 ] A(x,p)=\frac{\gamma Q_{11}}{Q}\left[x^2+2\frac{Q_{12}}{Q_{11}}x\cdot p+\frac{Q_{22}}{Q_{11}}p^2\right] A ( x , p ) = Q γ Q 11 [ x 2 + 2 Q 11 Q 12 x ⋅ p + Q 11 Q 22 p 2 ]
and, under the assumption Q 11 Q_{11} Q 11 and Q 22 Q_{22} Q 22 don’t have opposite sign, we propose the transformation
p → Q 22 Q 11 p p\to\sqrt{\frac{Q_{22}}{Q_{11}}}p p → Q 11 Q 22 p
in which case we have
A ( x , Q 22 Q 11 p ) = γ Q 11 Q [ x 2 + 2 Q 12 Q 11 x ⋅ Q 22 Q 11 p + Q 22 Q 11 p 2 ] A\left(x,\sqrt{\frac{Q_{22}}{Q_{11}}}p\right)=\frac{\gamma Q_{11}}{Q}\left[x^2+2\frac{Q_{12}}{Q_{11}}x\cdot\sqrt{\frac{Q_{22}}{Q_{11}}}p+\frac{Q_{22}}{Q_{11}}p^2\right] A ( x , Q 11 Q 22 p ) = Q γ Q 11 [ x 2 + 2 Q 11 Q 12 x ⋅ Q 11 Q 22 p + Q 11 Q 22 p 2 ]
A ( x , Q 22 Q 11 p ) = γ Q 11 Q [ x 2 + 2 Q 12 Q 11 Q 22 x ⋅ Q 22 Q 11 p + ( Q 22 Q 11 p ) 2 ] A\left(x,\sqrt{\frac{Q_{22}}{Q_{11}}}p\right)=\frac{\gamma Q_{11}}{Q}\left[x^2+2\frac{Q_{12}}{\sqrt{Q_{11}Q_{22}}}x\cdot\sqrt{\frac{Q_{22}}{Q_{11}}}p+\left(\sqrt{\frac{Q_{22}}{Q_{11}}}p\right)^2\right] A ( x , Q 11 Q 22 p ) = Q γ Q 11 x 2 + 2 Q 11 Q 22 Q 12 x ⋅ Q 11 Q 22 p + ( Q 11 Q 22 p ) 2
and, under these variables, the Hessian is
[ H ] { x , Q 22 p / Q 11 } = γ Q 11 Q ( 1 Q 12 Q 11 Q 22 Q 12 Q 11 Q 22 1 ) [H]_{\{x,\sqrt{Q_{22}}p/\sqrt{Q_{11}}\}}=\frac{\gamma Q_{11}}{Q}\begin{pmatrix}1&\frac{Q_{12}}{\sqrt{Q_{11}Q_{22}}}\\\frac{Q_{12}}{\sqrt{Q_{11}Q_{22}}}&1\end{pmatrix} [ H ] { x , Q 22 p / Q 11 } = Q γ Q 11 ( 1 Q 11 Q 22 Q 12 Q 11 Q 22 Q 12 1 )
[ H ] { x , Q 22 p / Q 11 } = γ Q ( Q 11 Q 12 Q 11 Q 22 Q 12 Q 11 Q 22 Q 11 ) [H]_{\{x,\sqrt{Q_{22}}p/\sqrt{Q_{11}}\}}=\frac{\gamma}{Q}\begin{pmatrix}Q_{11}&\frac{Q_{12}\sqrt{Q_{11}}}{\sqrt{Q_{22}}}\\\frac{Q_{12}\sqrt{Q_{11}}}{\sqrt{Q_{22}}}&Q_{11}\end{pmatrix} [ H ] { x , Q 22 p / Q 11 } = Q γ ( Q 11 Q 22 Q 12 Q 11 Q 22 Q 12 Q 11 Q 11 )
so, under the proposed transformation of variables,
( x , p ) → ( ω 0 x , Q 22 Q 11 p ) (x,p)\to\left(\omega_0x,\sqrt{\frac{Q_{22}}{Q_{11}}}p\right) ( x , p ) → ( ω 0 x , Q 11 Q 22 p )
General case
the Hessian is
[ H ] { x , Q 22 Q 11 p } = γ Q ( Q 11 Q 12 Q 11 Q 22 Q 12 Q 11 Q 22 Q 11 ) [H]_{\left\{x,\sqrt{\frac{Q_{22}}{Q_{11}}}p\right\}}=\frac{\gamma}{Q}\begin{pmatrix} Q_{11} & Q_{12}\sqrt{\frac{Q_{11}}{Q_{22}}} \\ Q_{12}\sqrt{\frac{Q_{11}}{Q_{22}}} & Q_{11} \end{pmatrix} [ H ] { x , Q 11 Q 22 p } = Q γ Q 11 Q 12 Q 22 Q 11 Q 12 Q 22 Q 11 Q 11
in which now the transformation of coordinates of linear combination can render a Hessian with diagonal structure: if we propose
( ω 0 x , Q 22 Q 11 p ) → ( ω 0 x + Q 22 Q 11 p , ω 0 x − Q 22 Q 11 p ) \left(\omega_0x,\sqrt{\frac{Q_{22}}{Q_{11}}}p\right)\to\left(\omega_0x+\sqrt{\frac{Q_{22}}{Q_{11}}}p,\omega_0x-\sqrt{\frac{Q_{22}}{Q_{11}}}p\right) ( ω 0 x , Q 11 Q 22 p ) → ( ω 0 x + Q 11 Q 22 p , ω 0 x − Q 11 Q 22 p )
the Hessian will have the diagonal structure
[ H ] { ω 0 x + Q 22 Q 11 p , ω 0 x − Q 22 Q 11 p } = γ Q ( Q 11 + Q 12 Q 11 Q 22 0 d 0 d Q 11 − Q 12 Q 11 Q 22 ) [H]_{\left\{\omega_0x+\sqrt{\frac{Q_{22}}{Q_{11}}}p,\omega_0x-\sqrt{\frac{Q_{22}}{Q_{11}}}p\right\}}=\frac{\gamma}{Q}\begin{pmatrix} Q_{11}+Q_{12}\sqrt{\frac{Q_{11}}{Q_{22}}} & 0_d \\ 0_d & Q_{11}-Q_{12}\sqrt{\frac{Q_{11}}{Q_{22}}} \end{pmatrix} [ H ] { ω 0 x + Q 11 Q 22 p , ω 0 x − Q 11 Q 22 p } = Q γ Q 11 + Q 12 Q 22 Q 11 0 d 0 d Q 11 − Q 12 Q 22 Q 11
and under this change of variables, the eigenvalues of the Hessian are only two, namely
κ ± = γ Q ( Q 11 ± Q 12 Q 11 Q 22 ) \kappa_{\pm}=\frac{\gamma}{Q}\left(Q_{11}\pm Q_{12}\sqrt{\frac{Q_{11}}{Q_{22}}}\right) κ ± = Q γ ( Q 11 ± Q 12 Q 22 Q 11 )
and the optimal convergence rate is the smallest of them. To determine which one is the smallest, we just have to compare them. So we have
κ ± = γ Q 11 Q ( Q 11 ± Q 12 Q 22 ) \kappa_{\pm}=\frac{\gamma\sqrt{Q_{11}}}{Q}\left(\sqrt{Q_{11}}\pm\frac{Q_{12}}{\sqrt{Q_{22}}}\right) κ ± = Q γ Q 11 ( Q 11 ± Q 22 Q 12 )
and recalling
Q 11 = D p p + ω 0 2 D q q Q_{11}=D_{pp}+\omega_0^2D_{qq} Q 11 = D pp + ω 0 2 D q q
Q 12 = 2 ω 0 γ D q q Q_{12}=2\omega_0\gamma D_{qq} Q 12 = 2 ω 0 γ D q q
Q 22 = D p p + ω 0 2 D q q + 4 γ ( D p q + γ D q q ) = Q 11 + 4 γ ( D p q + γ D q q ) Q_{22}=D_{pp}+\omega_0^2D_{qq}+4\gamma(D_{pq}+\gamma D_{qq})=Q_{11}+4\gamma(D_{pq}+\gamma D_{qq}) Q 22 = D pp + ω 0 2 D q q + 4 γ ( D pq + γ D q q ) = Q 11 + 4 γ ( D pq + γ D q q )
Q = Q 11 Q 22 − Q 12 2 Q=Q_{11}Q_{22}-Q_{12}^2 Q = Q 11 Q 22 − Q 12 2
the optimal convergence rate is the smallest of the following two roots:
κ ± = γ Q 11 Q 22 − Q 12 2 ( D p p + ω 0 2 D q q ± 2 ω 0 γ D q q 1 + 4 γ D p q + γ D q q Q 11 ) \kappa_{\pm}=\frac{\gamma}{Q_{11}Q_{22}-Q_{12}^2}\left(D_{pp}+\omega_0^2D_{qq}\pm\frac{2\omega_0\gamma D_{qq}}{\sqrt{1+4\gamma\frac{D_{pq}+\gamma D_{qq}}{Q_{11}}}}\right) κ ± = Q 11 Q 22 − Q 12 2 γ D pp + ω 0 2 D q q ± 1 + 4 γ Q 11 D pq + γ D q q 2 ω 0 γ D q q
κ ± = γ Q 11 [ Q 11 + 4 γ ( D p q + γ D q q ) ] − [ 2 ω 0 γ D q q ] 2 ( D p p + ω 0 2 D q q ± 1 1 ( 2 ω 0 γ D q q ) 2 + 4 γ ( 2 ω 0 γ D q q ) 2 D p q + γ D q q Q 11 ) \kappa_{\pm}=\frac{\gamma}{Q_{11}[Q_{11}+4\gamma(D_{pq}+\gamma D_{qq})]-[2\omega_0\gamma D_{qq}]^2}\left(D_{pp}+\omega_0^2D_{qq}\pm\frac{1}{\sqrt{\frac{1}{(2\omega_0\gamma D_{qq})^2}+\frac{4\gamma}{(2\omega_0\gamma D_{qq})^2}\frac{D_{pq}+\gamma D_{qq}}{Q_{11}}}}\right) κ ± = Q 11 [ Q 11 + 4 γ ( D pq + γ D q q )] − [ 2 ω 0 γ D q q ] 2 γ D pp + ω 0 2 D q q ± ( 2 ω 0 γ D q q ) 2 1 + ( 2 ω 0 γ D q q ) 2 4 γ Q 11 D pq + γ D q q 1
κ ± = γ Q 11 2 + 4 γ ( D p q + γ D q q ) Q 11 − 4 ω 0 2 γ 2 D q q 2 ( D p p + ω 0 2 D q q ± 1 1 ( 2 ω 0 γ D q q ) 2 + 1 ( ω 0 D q q ) 2 γ D p q + γ D q q Q 11 ) \kappa_{\pm}=\frac{\gamma}{Q_{11}^2+4\gamma(D_{pq}+\gamma D_{qq})Q_{11}-4\omega_0^2\gamma^2D_{qq}^2}\left(D_{pp}+\omega_0^2D_{qq}\pm\frac{1}{\sqrt{\frac{1}{(2\omega_0\gamma D_{qq})^2}+\frac{1}{(\omega_0D_{qq})^2\gamma}\frac{D_{pq}+\gamma D_{qq}}{Q_{11}}}}\right) κ ± = Q 11 2 + 4 γ ( D pq + γ D q q ) Q 11 − 4 ω 0 2 γ 2 D q q 2 γ D pp + ω 0 2 D q q ± ( 2 ω 0 γ D q q ) 2 1 + ( ω 0 D q q ) 2 γ 1 Q 11 D pq + γ D q q 1
and, under the rescaling that makes ω 0 = 1 \omega_0 = 1 ω 0 = 1 , we have
κ ± = γ Q 11 2 + 4 γ ( D p q + γ D q q ) Q 11 − 4 γ 2 D q q 2 ( D p p + D q q ± 1 1 ( 2 γ D q q ) 2 + 1 D q q 2 D p q + γ D q q Q 11 ) \kappa_{\pm} = \frac{\gamma}{Q_{11}^{2} + 4\gamma(D_{pq}+\gamma D_{qq})Q_{11} - 4\gamma^{2}D_{qq}^{2}}\left(D_{pp}+D_{qq}\pm\frac{1}{\sqrt{\frac{1}{(2\gamma D_{qq})^{2}}+\frac{1}{D_{qq}^{2}}\frac{D_{pq}+\gamma D_{qq}}{Q_{11}}}}\right) κ ± = Q 11 2 + 4 γ ( D pq + γ D q q ) Q 11 − 4 γ 2 D q q 2 γ D pp + D q q ± ( 2 γ D q q ) 2 1 + D q q 2 1 Q 11 D pq + γ D q q 1
κ ± = γ ( D p p + D q q ) 2 + 4 γ ( D p q + γ D q q ) ( D p p + D q q ) − 4 γ 2 D q q 2 ( D p p + D q q ± 1 1 ( 2 γ D q q ) 2 + 1 D q q 2 D p q + γ D q q D p p + D q q ) \kappa_{\pm} = \frac{\gamma}{(D_{pp}+D_{qq})^{2}+4\gamma(D_{pq}+\gamma D_{qq})(D_{pp}+D_{qq})-4\gamma^{2}D_{qq}^{2}}\left(D_{pp}+D_{qq}\pm\frac{1}{\sqrt{\frac{1}{(2\gamma D_{qq})^{2}}+\frac{1}{D_{qq}^{2}}\frac{D_{pq}+\gamma D_{qq}}{D_{pp}+D_{qq}}}}\right) κ ± = ( D pp + D q q ) 2 + 4 γ ( D pq + γ D q q ) ( D pp + D q q ) − 4 γ 2 D q q 2 γ D pp + D q q ± ( 2 γ D q q ) 2 1 + D q q 2 1 D pp + D q q D pq + γ D q q 1
κ ± = γ ( D p p + D q q ) 2 + ( 4 γ D p q + 4 γ 2 D q q ) ( D p p + D q q ) − 4 γ 2 D q q 2 ( D p p + D q q ± 1 1 ( 2 γ D q q ) 2 + 1 D q q 2 D p q + γ D q q D p p + D q q ) \kappa_{\pm} = \frac{\gamma}{(D_{pp}+D_{qq})^{2}+(4\gamma D_{pq}+4\gamma^{2}D_{qq})(D_{pp}+D_{qq})-4\gamma^{2}D_{qq}^{2}}\left(D_{pp}+D_{qq}\pm\frac{1}{\sqrt{\frac{1}{(2\gamma D_{qq})^{2}}+\frac{1}{D_{qq}^{2}}\frac{D_{pq}+\gamma D_{qq}}{D_{pp}+D_{qq}}}}\right) κ ± = ( D pp + D q q ) 2 + ( 4 γ D pq + 4 γ 2 D q q ) ( D pp + D q q ) − 4 γ 2 D q q 2 γ D pp + D q q ± ( 2 γ D q q ) 2 1 + D q q 2 1 D pp + D q q D pq + γ D q q 1
κ ± = γ ( D p p + D q q ) 2 + 4 γ D p q ( D p p + D q q ) + 4 γ 2 D q q ( D p p + D q q ) − 4 γ 2 D q q 2 ( D p p + D q q ± 1 1 ( 2 γ D q q ) 2 + 1 D q q 2 D p q + γ D q q D p p + D q q ) \kappa_{\pm} = \frac{\gamma}{(D_{pp}+D_{qq})^{2}+4\gamma D_{pq}(D_{pp}+D_{qq})+4\gamma^{2}D_{qq}(D_{pp}+D_{qq})-4\gamma^{2}D_{qq}^{2}}\left(D_{pp}+D_{qq}\pm\frac{1}{\sqrt{\frac{1}{(2\gamma D_{qq})^{2}}+\frac{1}{D_{qq}^{2}}\frac{D_{pq}+\gamma D_{qq}}{D_{pp}+D_{qq}}}}\right) κ ± = ( D pp + D q q ) 2 + 4 γ D pq ( D pp + D q q ) + 4 γ 2 D q q ( D pp + D q q ) − 4 γ 2 D q q 2 γ D pp + D q q ± ( 2 γ D q q ) 2 1 + D q q 2 1 D pp + D q q D pq + γ D q q 1
κ ± = γ ( D p p + D q q ) 2 + 4 γ D p q ( D p p + D q q ) + 4 γ 2 D q q D p p ( D p p + D q q ± 1 1 ( 2 γ D q q ) 2 + 1 D q q 1 + D p q / ( γ D q q ) D p p + D q q ) \kappa_{\pm} = \frac{\gamma}{(D_{pp}+D_{qq})^{2}+4\gamma D_{pq}(D_{pp}+D_{qq})+4\gamma^{2}D_{qq}D_{pp}}\left(D_{pp}+D_{qq}\pm\frac{1}{\sqrt{\frac{1}{(2\gamma D_{qq})^{2}}+\frac{1}{D_{qq}}\frac{1+D_{pq}/(\gamma D_{qq})}{D_{pp}+D_{qq}}}}\right) κ ± = ( D pp + D q q ) 2 + 4 γ D pq ( D pp + D q q ) + 4 γ 2 D q q D pp γ D pp + D q q ± ( 2 γ D q q ) 2 1 + D q q 1 D pp + D q q 1 + D pq / ( γ D q q ) 1
κ ± = γ ( D p p + D q q ) 2 + 4 γ D p q ( D p p + D q q ) + 4 γ 2 D q q D p p ( D p p + D q q ± [ 1 ( 2 γ D q q ) 2 + 1 D q q 1 + D p q / ( γ D q q ) D p p + D q q ] − 1 / 2 ) \kappa_{\pm} = \frac{\gamma}{(D_{pp}+D_{qq})^{2}+4\gamma D_{pq}(D_{pp}+D_{qq})+4\gamma^{2}D_{qq}D_{pp}}\left(D_{pp}+D_{qq}\pm\left[\frac{1}{(2\gamma D_{qq})^{2}}+\frac{1}{D_{qq}}\frac{1+D_{pq}/(\gamma D_{qq})}{D_{pp}+D_{qq}}\right]^{-1/2}\right) κ ± = ( D pp + D q q ) 2 + 4 γ D pq ( D pp + D q q ) + 4 γ 2 D q q D pp γ ( D pp + D q q ± [ ( 2 γ D q q ) 2 1 + D q q 1 D pp + D q q 1 + D pq / ( γ D q q ) ] − 1/2 )
κ ± = γ D p p 2 + D q q 2 + 2 ( 1 + 2 γ 2 ) D q q D p p + 4 γ D p q ( D p p + D q q ) ( D p p + D q q ± [ 1 ( 2 γ D q q ) 2 + 1 D q q 1 + D p q / ( γ D q q ) D p p + D q q ] − 1 / 2 ) \kappa_{\pm} = \frac{\gamma}{D_{pp}^{2}+D_{qq}^{2}+2(1+2\gamma^{2})D_{qq}D_{pp}+4\gamma D_{pq}(D_{pp}+D_{qq})}\left(D_{pp}+D_{qq}\pm\left[\frac{1}{(2\gamma D_{qq})^{2}}+\frac{1}{D_{qq}}\frac{1+D_{pq}/(\gamma D_{qq})}{D_{pp}+D_{qq}}\right]^{-1/2}\right) κ ± = D pp 2 + D q q 2 + 2 ( 1 + 2 γ 2 ) D q q D pp + 4 γ D pq ( D pp + D q q ) γ ( D pp + D q q ± [ ( 2 γ D q q ) 2 1 + D q q 1 D pp + D q q 1 + D pq / ( γ D q q ) ] − 1/2 )
κ ± = γ ( 1 + 2 γ 2 ) ( D p p + D q q ) 2 − 4 γ 2 D q q D p p + 4 γ D p q ( D p p + D q q ) ( D p p + D q q ± [ 1 ( 2 γ D q q ) 2 + 1 D q q 1 + D p q / ( γ D q q ) D p p + D q q ] − 1 / 2 ) \kappa_{\pm} = \frac{\gamma}{(1+2\gamma^{2})(D_{pp}+D_{qq})^{2}-4\gamma^{2}D_{qq}D_{pp}+4\gamma D_{pq}(D_{pp}+D_{qq})}\left(D_{pp}+D_{qq}\pm\left[\frac{1}{(2\gamma D_{qq})^{2}}+\frac{1}{D_{qq}}\frac{1+D_{pq}/(\gamma D_{qq})}{D_{pp}+D_{qq}}\right]^{-1/2}\right) κ ± = ( 1 + 2 γ 2 ) ( D pp + D q q ) 2 − 4 γ 2 D q q D pp + 4 γ D pq ( D pp + D q q ) γ ( D pp + D q q ± [ ( 2 γ D q q ) 2 1 + D q q 1 D pp + D q q 1 + D pq / ( γ D q q ) ] − 1/2 )
κ ± = D p p + D q q ± [ 1 ( 2 γ D q q ) 2 + ( 1 + D p q γ D q q ) 1 D q q ( D p p + D q q ) ] − 1 / 2 ( 1 γ + 2 γ ) ( D p p + D q q ) 2 − 4 γ D q q D p p + 4 D p q ( D p p + D q q ) \kappa_{\pm} = \frac{D_{pp}+D_{qq}\pm\left[\frac{1}{(2\gamma D_{qq})^{2}}+\left(1+\frac{D_{pq}}{\gamma D_{qq}}\right)\frac{1}{D_{qq}(D_{pp}+D_{qq})}\right]^{-1/2}}{\left(\frac{1}{\gamma}+2\gamma\right)(D_{pp}+D_{qq})^{2}-4\gamma D_{qq}D_{pp}+4D_{pq}(D_{pp}+D_{qq})} κ ± = ( γ 1 + 2 γ ) ( D pp + D q q ) 2 − 4 γ D q q D pp + 4 D pq ( D pp + D q q ) D pp + D q q ± [ ( 2 γ D q q ) 2 1 + ( 1 + γ D q q D pq ) D q q ( D pp + D q q ) 1 ] − 1/2
κ ± = D p p + D q q ± 1 1 ( 2 γ D q q ) 2 + ( 1 + D p q γ D q q ) 1 D q q ( D p p + D q q ) ( 1 γ + 2 γ ) ( D p p + D q q ) 2 + 4 [ D p q ( D p p + D q q ) − γ D q q D p p ] \kappa_{\pm} = \frac{D_{pp}+D_{qq}\pm\frac{1}{\sqrt{\frac{1}{(2\gamma D_{qq})^{2}}+\left(1+\frac{D_{pq}}{\gamma D_{qq}}\right)\frac{1}{D_{qq}(D_{pp}+D_{qq})}}}}{\left(\frac{1}{\gamma}+2\gamma\right)(D_{pp}+D_{qq})^{2}+4\left[D_{pq}(D_{pp}+D_{qq})-\gamma D_{qq}D_{pp}\right]} κ ± = ( γ 1 + 2 γ ) ( D pp + D q q ) 2 + 4 [ D pq ( D pp + D q q ) − γ D q q D pp ] D pp + D q q ± ( 2 γ D q q ) 2 1 + ( 1 + γ D q q D pq ) D q q ( D pp + D q q ) 1 1
κ ± = D p p + D q q ± 1 1 D q q 2 ( 1 γ ) 2 + 1 + D p q / D q q γ 1 + D p p / D q q ( 1 γ + 2 γ ) ( D p p + D q q ) 2 + 4 [ D p q ( D p p + D q q ) − γ D q q D p p ] \kappa_{\pm} = \frac{D_{pp}+D_{qq}\pm\frac{1}{\sqrt{\frac{1}{D_{qq}^{2}}\left(\frac{1}{\gamma}\right)^{2}+\frac{1+D_{pq}/D_{qq}\gamma}{1+D_{pp}/D_{qq}}}}}{\left(\frac{1}{\gamma}+2\gamma\right)(D_{pp}+D_{qq})^{2}+4\left[D_{pq}(D_{pp}+D_{qq})-\gamma D_{qq}D_{pp}\right]} κ ± = ( γ 1 + 2 γ ) ( D pp + D q q ) 2 + 4 [ D pq ( D pp + D q q ) − γ D q q D pp ] D pp + D q q ± D q q 2 1 ( γ 1 ) 2 + 1 + D pp / D q q 1 + D pq / D q q γ 1
Stochastic Gradient Descent in the continuous-time limit as an SDE: Classical vs. quantum regimes
We focus on the study of stochastic gradient descent by means of their continuous time limit (Su, Wibisono, Jordan, Shi).
We can understand the Stochastic Gradient Descent (SGD) algorithm by following [Liu, Su, Li], in which they consider f : R d → R d f:\mathbb{R}^{d}\to\mathbb{R}^{d} f : R d → R d the objective function constructed with all data, and the SGD algorithm being given by
x k + 1 = x k − s ∇ ~ f ( x k ) x_{k+1}=x_k-s\widetilde{\nabla}f(x_k) x k + 1 = x k − s ∇ f ( x k )
where s s s is the so-called “learning rate” and ∇ ~ f \widetilde{\nabla}f ∇ f is the estimated gradient obtained from a mini-batch. One can model SGD then by the following iterative method,
x k + 1 = x k − s ∇ ~ f ( x k ) − s ξ k x_{k+1}=x_k-s\widetilde{\nabla}f(x_k)-s\xi_k x k + 1 = x k − s ∇ f ( x k ) − s ξ k
where ξ k \xi_k ξ k is a given noise that obeys a normal distribution, as in the so-called “unadjusted Langevin dynamics”. Taking the continuous time limit one obtains a stochastic differential equation (SDE) that includes the learning rate s s s as a parameter, which models the discrete SGD algorithm. This SDE is
d x = − ∇ f ( x ) d t + s d W dx=-\nabla f(x)dt+\sqrt{s}dW d x = − ∇ f ( x ) d t + s d W
with W W W a Brownian motion. It is indicating that the transport is such that the objective function is minimized by following the direction opposite to the gradient, namely − ∇ f -\nabla f − ∇ f , but being also subjected to a noise following a normal distribution.
On the other hand, the main equation in quantum mechanics where both transport and noise are combined is a master equation, either in a Wigner formulation, such as Wigner-Fokker-Planck (integro-differential equation in continuous variables)
w t + p ⋅ w x + Θ [ V ] ( w ) = ∇ ⋅ ( D ∇ w ) + 2 γ ∇ p ⋅ ( w p ) w_t+p\cdot w_x+\Theta[V](w)=\nabla\cdot(D\nabla w)+2\gamma\nabla_p\cdot(wp) w t + p ⋅ w x + Θ [ V ] ( w ) = ∇ ⋅ ( D ∇ w ) + 2 γ ∇ p ⋅ ( w p )
or a density matrix one, such as the Lindblad master equation (an operator equation, which can be projected in any basis such as a position one),
ρ t = − i ℏ [ H , ρ ] + ∑ i γ i ( L i ρ L i † − 1 2 { L i † L i , ρ } ) \rho_t=-\frac{\mathrm{i}}{\hbar}[H,\rho]+\sum_i\gamma_i\left(L_i\rho L_i^\dagger-\frac{1}{2}\{L_i^\dagger L_i,\rho\}\right) ρ t = − ℏ i [ H , ρ ] + i ∑ γ i ( L i ρ L i † − 2 1 { L i † L i , ρ } )
and there is a third way to represent this noisy quantum transport process in an SDE form via a “Quantum Langevin” equation, where defining z = ( x , p ) z=(x,p) z = ( x , p ) as a phase-space position-momentum vector, is given by (for the case of a Harmonic potential)
d z = Q [ z ( t ) ] d t + E [ z i ( t ) ] dz=Q[z(t)]dt+E[z_i(t)] d z = Q [ z ( t )] d t + E [ z i ( t )]
with Q [ z ( t ) ] = ( p , − x − 2 γ p ) Q[z(t)]=(p,-x-2\gamma p) Q [ z ( t )] = ( p , − x − 2 γ p ) being the quantum transport, and E E E being a zero average Gaussian random variable, whose variance is D d t Ddt D d t .
The question is: are our quantum master equations above quantum analogs of the SDE equation with a gradient transport equivalent to a continuous time limit of SGD?
Let’s recall the classical limit of the WFP equation, as indicated in [6 ] . We have that [6 ] in the classical limit ℏ → 0 \hbar\to0 ℏ → 0 , D q q = 0 = D p q D_{qq}=0=D_{pq} D q q = 0 = D pq and the pseudo-differential operator becomes in this limit one of the phase-space Liouvillean transports, as in (taking m = 1 m=1 m = 1 )
Θ [ V ] { w } = − ∇ x V ⋅ ∇ p w \Theta[V]\{w\}=-\nabla_xV\cdot\nabla_pw Θ [ V ] { w } = − ∇ x V ⋅ ∇ p w
so the classical Fokker-Planck equation in phase space for the probability distribution f f f is recovered as w → f w\to f w → f when ℏ → 0 \hbar\to0 ℏ → 0 , namely
f t + p ⋅ f x − ∇ x V ⋅ f p = ∇ p ⋅ ( D p p ∇ p f ) + 2 γ ∇ p ⋅ ( f p ) f_t+p\cdot f_x-\nabla_xV\cdot f_p=\nabla_p\cdot(D_{pp}\nabla_pf)+2\gamma\nabla_p\cdot(fp) f t + p ⋅ f x − ∇ x V ⋅ f p = ∇ p ⋅ ( D pp ∇ p f ) + 2 γ ∇ p ⋅ ( f p )
and since we have the Hamiltonian
H ( x , p ) = p 2 2 + V ( x ) H(x,p) = \frac{p^2}{2} + V(x) H ( x , p ) = 2 p 2 + V ( x )
part of the transport has a symplectic structure, as in
f t + ∇ p H ⋅ f x − ∇ x H ⋅ f p = ∇ p ⋅ ( D p p ∇ p f ) + 2 γ ∇ p ⋅ ( f p ) f_t + \nabla_p H \cdot f_x - \nabla_x H \cdot f_p = \nabla_p \cdot(D_{pp}\nabla_p f) + 2\gamma\nabla_p \cdot(fp) f t + ∇ p H ⋅ f x − ∇ x H ⋅ f p = ∇ p ⋅ ( D pp ∇ p f ) + 2 γ ∇ p ⋅ ( f p )
and though we could express this as Poisson brackets, we can express the Hamiltonian transport as
f t + ∇ ⋅ ( f ( ∇ p H , − ∇ x H ) ) = ∇ p ⋅ ( D p p ∇ p f ) + 2 γ ∇ p ⋅ ( f p ) f_t + \nabla\cdot(f(\nabla_p H,-\nabla_x H)) = \nabla_p \cdot(D_{pp}\nabla_p f) + 2\gamma\nabla_p \cdot(fp) f t + ∇ ⋅ ( f ( ∇ p H , − ∇ x H )) = ∇ p ⋅ ( D pp ∇ p f ) + 2 γ ∇ p ⋅ ( f p )
and adding the friction to the Hamiltonian transport we get the classical kinetic Fokker-Planck equation as
f t + ∇ ⋅ ( f ( ∇ p H , − ∇ x H ) ) − 2 γ ∇ ⋅ ( f ( 0 , p ) ) = ∇ p ⋅ ( D p p ∇ p f ) f_t + \nabla\cdot(f(\nabla_p H,-\nabla_x H)) - 2\gamma\nabla\cdot(f(0,p)) = \nabla_p \cdot(D_{pp}\nabla_p f) f t + ∇ ⋅ ( f ( ∇ p H , − ∇ x H )) − 2 γ ∇ ⋅ ( f ( 0 , p )) = ∇ p ⋅ ( D pp ∇ p f )
f t + ∇ ⋅ ( f ( ∇ p H , − ∇ x H − 2 γ p ) ) = ∇ p ⋅ ( D p p ∇ p f ) f_t + \nabla\cdot(f(\nabla_p H,-\nabla_x H - 2\gamma p)) = \nabla_p \cdot(D_{pp}\nabla_p f) f t + ∇ ⋅ ( f ( ∇ p H , − ∇ x H − 2 γ p )) = ∇ p ⋅ ( D pp ∇ p f )
so again our transport is
α = ( p , − ∇ x V − 2 γ p ) \alpha= (p,-\nabla_x V - 2\gamma p) α = ( p , − ∇ x V − 2 γ p )
so for example, if V = x 2 / 2 V = x^2/2 V = x 2 /2 is harmonic, we have
α = ( p , − x − 2 γ p ) \alpha= (p,-x - 2\gamma p) α = ( p , − x − 2 γ p )
The problem is that a classical kinetic Fokker-Planck is just Hamiltonian transport and noise (with both diffusion and friction). On the other hand, the SDE for the continuous-time limit SGD is a gradient transport and noise (represented via diffusion). Namely, the SDE for the limit behavior of SGD has a steepest descent transport following the gradient with some added noise, whereas a Fokker-Planck equation has a Hamiltonian transport (the gradient multiplied by a symplectic matrix) with momentum noise and friction (the latter being the non-Hamiltonian contribution of the transport, rather in gradient form instead). The difference is clearly noticed if one thinks of the harmonic potential V = x 2 / 2 V = x^2/2 V = x 2 /2 , in which the equation without noise will just represent the harmonic oscillator orbiting the origin in phase space conserving the energy functional (on an ellipse or circle depending the scaling of coordinates), whereas the transport in the SGD goes in the opposite direction of the gradient for the objective function, minimizing the respective functional:
d x = − ∇ F ( x ) d t + s d W , dx = -\nabla F(x)dt + \sqrt{s}dW, d x = − ∇ F ( x ) d t + s d W ,
and even the kinetic diffusive PDE representing the SGD in continuous time limit is known by Jordan et al. [3 ] , as in
∂ t ρ S G D = ∇ ⋅ ( ρ S G D ∇ F ) + s 2 ∇ 2 ρ S G D \partial_t \rho_{SGD} = \nabla\cdot(\rho_{SGD}\nabla F) + \frac{s}{2}\nabla^2\rho_{SGD} ∂ t ρ S G D = ∇ ⋅ ( ρ S G D ∇ F ) + 2 s ∇ 2 ρ S G D
so the analog of a diffusion coefficient is s / 2 s/2 s /2 . Formally the variable in the equation above needs only be the position x x x : in the quantum version of SGD the problem dimensionality needs to be doubled by introducing a conjugate variable such as the momentum p p p . On the other hand, if we want to make analogies with the classical case it might be useful to think of a phase space setting even from the classical case, so we would have
∂ t ρ S G D = ∇ ( x , p ) ⋅ ( ρ S G D ∇ ( x , p ) F ) + ∇ ( x , p ) ⋅ ( s 2 I 2 d ∇ ( x , p ) ρ S G D ) \partial_t \rho_{SGD} = \nabla_{(x,p)} \cdot\left(\rho_{SGD} \nabla_{(x,p)} F\right) + \nabla_{(x,p)} \cdot\left(\frac{s}{2} I_{2d} \nabla_{(x,p)} \rho_{SGD}\right) ∂ t ρ S G D = ∇ ( x , p ) ⋅ ( ρ S G D ∇ ( x , p ) F ) + ∇ ( x , p ) ⋅ ( 2 s I 2 d ∇ ( x , p ) ρ S G D )
and if x , p ∈ R d x,p \in\mathbb{R}^d x , p ∈ R d then D = s I 2 d / 2 D=sI_{2d}/2 D = s I 2 d /2 in this case. If this Fokker-Planck for the limit SGD has an analog with the physics-motivated classical Fokker-Planck
f t + ∇ ⋅ ( f ( p , − x − 2 γ p ) ) = ∇ ⋅ ( D p p ( 0 d 0 d 0 d I d ) ∇ f ) f_t + \nabla\cdot\left(f(p,-x-2\gamma p)\right) = \nabla\cdot\left(D_{pp} \begin{pmatrix} 0_d & 0_d \\ 0_d & I_d \end{pmatrix} \nabla f\right) f t + ∇ ⋅ ( f ( p , − x − 2 γ p ) ) = ∇ ⋅ ( D pp ( 0 d 0 d 0 d I d ) ∇ f )
then we need that
∇ ( x , p ) F = 2 γ p , γ ≫ 1 \nabla_{(x,p)}F = 2\gamma p,\qquad\gamma\gg1 ∇ ( x , p ) F = 2 γ p , γ ≫ 1
so the analog is strong when the objective function is, for γ ≫ 1 \gamma\gg1 γ ≫ 1 ,
F = γ p 2 + C F = \gamma p^2 + C F = γ p 2 + C
with C C C a constant of integration to be defined. The analogy is not 100% direct, since D q q = 0 D_{qq}=0 D q q = 0 yet for the SGD-SDE limit equation one has D q q = s / 2 D_{qq}=s/2 D q q = s /2 . This would seem to indicate that the quantum analog of an SGD can be found for quadratic functionals dominant over the other noise term (diffusion) and with a negligible Hamiltonian transport by taking γ ≫ 1 \gamma\gg1 γ ≫ 1 .
Appendix
Asymptotic approximation to the optimal convergence rate (Perturbation theory approach)
We consider the general case of
H = Hessian ( A ) = γ Q ( Q 11 ω 0 2 I d Q 12 ω 0 I d Q 12 ω 0 I d Q 22 I d ) H = \operatorname{Hessian}(A) = \frac{\gamma}{Q} \begin{pmatrix} Q_{11}\omega_0^2 I_d & Q_{12}\omega_0 I_d \\ Q_{12}\omega_0 I_d & Q_{22}I_d \end{pmatrix} H = Hessian ( A ) = Q γ ( Q 11 ω 0 2 I d Q 12 ω 0 I d Q 12 ω 0 I d Q 22 I d )
and since
Q 11 = D p p + ω 0 2 D q q Q_{11}=D_{pp}+\omega_0^2D_{qq} Q 11 = D pp + ω 0 2 D q q
Q 12 = 2 ω 0 γ D q q Q_{12}=2\omega_0\gamma D_{qq} Q 12 = 2 ω 0 γ D q q
Q 22 = D p p + ω 0 2 D q q + 4 γ ( D p q + γ D q q ) = Q 11 + 4 γ ( D p q + γ D q q ) Q_{22}=D_{pp}+\omega_0^2D_{qq}+4\gamma(D_{pq}+\gamma D_{qq})=Q_{11}+4\gamma(D_{pq}+\gamma D_{qq}) Q 22 = D pp + ω 0 2 D q q + 4 γ ( D pq + γ D q q ) = Q 11 + 4 γ ( D pq + γ D q q )
Q = Q 11 Q 22 − Q 12 2 Q=Q_{11}Q_{22}-Q_{12}^2 Q = Q 11 Q 22 − Q 12 2
First, by a convenient change of coordinates
x → x / ω 0 x \to x/\omega_0 x → x / ω 0
which is equivalent to assuming ω 0 = 1 \omega_0 = 1 ω 0 = 1 , the Hessian matrix is rescaled to
H = γ Q ( Q 11 I d Q 12 I d Q 12 I d Q 22 I d ) H = \frac{\gamma}{Q}
\begin{pmatrix}
Q_{11}I_d & Q_{12}I_d \\
Q_{12}I_d & Q_{22}I_d
\end{pmatrix} H = Q γ ( Q 11 I d Q 12 I d Q 12 I d Q 22 I d )
which also renders the rescaling
Q 11 = D p p + D q q Q_{11} = D_{pp} + D_{qq} Q 11 = D pp + D q q
Q 12 = 2 γ D q q Q_{12} = 2\gamma D_{qq} Q 12 = 2 γ D q q
Q 22 = D p p + D q q + 4 γ ( D p q + γ D q q ) = Q 11 + 4 γ ( D p q + γ D q q ) Q_{22} = D_{pp} + D_{qq} + 4\gamma(D_{pq} + \gamma D_{qq}) = Q_{11} + 4\gamma(D_{pq} + \gamma D_{qq}) Q 22 = D pp + D q q + 4 γ ( D pq + γ D q q ) = Q 11 + 4 γ ( D pq + γ D q q )
Then we can split our matrix in a convenient way, as the sum of two matrices easier to analyze,
H = γ Q ( Q 11 I d Q 12 I d Q 12 I d [ Q 11 + 4 γ ( D p q + γ D q q ) ] I d ) H = \frac{\gamma}{Q}
\left(
\begin{array}{cc}
Q_{11}I_d & Q_{12}I_d \\
Q_{12}I_d & [Q_{11} + 4\gamma(D_{pq} + \gamma D_{qq})]I_d
\end{array}
\right) H = Q γ ( Q 11 I d Q 12 I d Q 12 I d [ Q 11 + 4 γ ( D pq + γ D q q )] I d )
H = γ Q [ Q 11 ( I d 0 d 0 d I d ) + ( 0 d Q 12 I d Q 12 I d 4 γ ( D p q + γ D q q ) I d ) ] H = \frac{\gamma}{Q}
\left[
Q_{11}
\begin{pmatrix}
I_d & 0_d \\
0_d & I_d
\end{pmatrix}
+
\begin{pmatrix}
0_d & Q_{12}I_d \\
Q_{12}I_d & 4\gamma(D_{pq} + \gamma D_{qq})I_d
\end{pmatrix}
\right] H = Q γ [ Q 11 ( I d 0 d 0 d I d ) + ( 0 d Q 12 I d Q 12 I d 4 γ ( D pq + γ D q q ) I d ) ]
H = γ Q [ Q 11 I 2 d + ( 0 d Q 12 I d Q 12 I d 4 γ ( D p q + γ D q q ) I d ) ] H = \frac{\gamma}{Q}
\left[
Q_{11}I_{2d}
+
\begin{pmatrix}
0_d & Q_{12}I_d \\
Q_{12}I_d & 4\gamma(D_{pq} + \gamma D_{qq})I_d
\end{pmatrix}
\right] H = Q γ [ Q 11 I 2 d + ( 0 d Q 12 I d Q 12 I d 4 γ ( D pq + γ D q q ) I d ) ]
If we assume
Q 11 = D p p + D q q ≠ 0 Q_{11} = D_{pp} + D_{qq} \ne0 Q 11 = D pp + D q q = 0
then
H = γ Q 11 Q [ I 2 d + ( 0 d I d Q 12 / Q 11 I d Q 12 / Q 11 I d 4 γ ( D p q + γ D q q ) / Q 11 ) ] H = \frac{\gamma Q_{11}}{Q}
\left[
I_{2d}
+
\begin{pmatrix}
0_d & I_d Q_{12}/Q_{11} \\
I_d Q_{12}/Q_{11} & I_d 4\gamma(D_{pq} + \gamma D_{qq})/Q_{11}
\end{pmatrix}
\right] H = Q γ Q 11 [ I 2 d + ( 0 d I d Q 12 / Q 11 I d Q 12 / Q 11 I d 4 γ ( D pq + γ D q q ) / Q 11 ) ]
and recalling
Q 11 = D p p + D q q Q_{11} = D_{pp} + D_{qq} Q 11 = D pp + D q q
Q 12 = 2 γ D q q Q_{12} = 2\gamma D_{qq} Q 12 = 2 γ D q q
then
H = γ Q 11 Q [ I 2 d + ( 0 d I d 2 γ D q q / ( D p p + D q q ) I d 2 γ D q q / ( D p p + D q q ) I d 4 γ ( D p q + γ D q q ) / ( D p p + D q q ) ) ] H = \frac{\gamma Q_{11}}{Q}
\left[
I_{2d}
+
\begin{pmatrix}
0_d & I_d 2\gamma D_{qq}/(D_{pp} + D_{qq}) \\
I_d 2\gamma D_{qq}/(D_{pp} + D_{qq}) & I_d 4\gamma(D_{pq} + \gamma D_{qq})/(D_{pp} + D_{qq})
\end{pmatrix}
\right] H = Q γ Q 11 [ I 2 d + ( 0 d I d 2 γ D q q / ( D pp + D q q ) I d 2 γ D q q / ( D pp + D q q ) I d 4 γ ( D pq + γ D q q ) / ( D pp + D q q ) ) ]
H = γ Q 11 Q [ I 2 d + 2 γ ( 0 d I d D q q / ( D p p + D q q ) I d D q q / ( D p p + D q q ) I d 2 ( D p q + γ D q q ) / ( D p p + D q q ) ) ] H = \frac{\gamma Q_{11}}{Q}
\left[
I_{2d}
+
2\gamma
\begin{pmatrix}
0_d & I_d D_{qq}/(D_{pp} + D_{qq}) \\
I_d D_{qq}/(D_{pp} + D_{qq}) & I_d 2(D_{pq} + \gamma D_{qq})/(D_{pp} + D_{qq})
\end{pmatrix}
\right] H = Q γ Q 11 [ I 2 d + 2 γ ( 0 d I d D q q / ( D pp + D q q ) I d D q q / ( D pp + D q q ) I d 2 ( D pq + γ D q q ) / ( D pp + D q q ) ) ]
assuming D q q ≠ 0 D_{qq} \ne0 D q q = 0 , then
H = γ Q 11 Q [ I 2 d + 2 γ ( 0 d I d / ( 1 + D p p / D q q ) I d / ( 1 + D p p / D q q ) I d 2 { 1 + ( γ − 1 ) / ( 1 + D p p / D q q ) } ) ] H = \frac{\gamma Q_{11}}{Q}
\left[
I_{2d}
+
2\gamma
\begin{pmatrix}
0_d & I_d/(1 + D_{pp}/D_{qq}) \\
I_d/(1 + D_{pp}/D_{qq}) & I_d 2\{1 + (\gamma- 1)/(1 + D_{pp}/D_{qq})\}
\end{pmatrix}
\right] H = Q γ Q 11 [ I 2 d + 2 γ ( 0 d I d / ( 1 + D pp / D q q ) I d / ( 1 + D pp / D q q ) I d 2 { 1 + ( γ − 1 ) / ( 1 + D pp / D q q )} ) ]
If we are able to obtain the eigenvalues of the second matrix, we can approximately obtain the eigenvalues of the full matrix by perturbation theory. Our matrix
( 0 d I d 1 + D p p / D q q I d 1 + D p p / D q q 2 [ 1 + γ − 1 1 + D p p / D q q ] I d ) \begin{pmatrix}
0_d & \dfrac{I_d}{1 + D_{pp}/D_{qq}} \\
\dfrac{I_d}{1 + D_{pp}/D_{qq}} & 2\left[1 + \dfrac{\gamma- 1}{1 + D_{pp}/D_{qq}}\right]I_d
\end{pmatrix} 0 d 1 + D pp / D q q I d 1 + D pp / D q q I d 2 [ 1 + 1 + D pp / D q q γ − 1 ] I d
which, if we define
r = 1 1 + D p p / D q q r = \frac{1}{1 + D_{pp}/D_{qq}} r = 1 + D pp / D q q 1
is given by
( 0 d r I d r I d 2 [ 1 + ( γ − 1 ) r ] I d ) \begin{pmatrix}
0_d & rI_d \\
rI_d & 2[1 + (\gamma- 1)r]I_d
\end{pmatrix} ( 0 d r I d r I d 2 [ 1 + ( γ − 1 ) r ] I d )
We will use the following result.
Lemma 2. If a real matrix 2 d × 2 d 2d \times2d 2 d × 2 d has the structure
T = ( 0 d a I d a I d b I d ) T = \begin{pmatrix}
0_d & aI_d \\
aI_d & bI_d
\end{pmatrix} T = ( 0 d a I d a I d b I d ) under the assumption
( a , b ) ≠ ( 0 , 0 ) (a,b) \ne(0,0) ( a , b ) = ( 0 , 0 ) to avoid trivial cases, there are only two eigenvalues for T T T of the form
b 2 ± ( b / 2 ) 2 + a 2 \frac{b}{2} \pm\sqrt{(b/2)^2 + a^2} 2 b ± ( b /2 ) 2 + a 2 Proof. If we consider a vector of the form
α e j + β e j + d , j = 1 , … d \alpha e_j + \beta e_{j+d}, \qquad j = 1,\ldots d α e j + β e j + d , j = 1 , … d when applying the transformation above, we have that
T ( α e j + β e j + d ) = α a e j + d + β ( a e j + b e j + d ) = a β e j + ( α a + β b ) e j + d T(\alpha e_j + \beta e_{j+d}) = \alpha a e_{j+d} + \beta(a e_j + b e_{j+d}) = a\beta e_j + (\alpha a + \beta b)e_{j+d} T ( α e j + β e j + d ) = α a e j + d + β ( a e j + b e j + d ) = a β e j + ( α a + β b ) e j + d and an eigenvalue would be such that the following ratios are equal
a β α = α a + β b β = a α β + b \frac{a\beta}{\alpha} = \frac{\alpha a + \beta b}{\beta} = a\frac{\alpha}{\beta} + b α a β = β α a + β b = a β α + b so
a ( β α − α β ) = b a\left(\frac{\beta}{\alpha} - \frac{\alpha}{\beta}\right) = b a ( α β − β α ) = b equivalent to
a ( β α ) 2 − b β α − a = 0 a\left(\frac{\beta}{\alpha}\right)^2 - b\frac{\beta}{\alpha} - a = 0 a ( α β ) 2 − b α β − a = 0 and therefore the two possible solutions for the ratio β / α \beta/\alpha β / α are
( β α ) ± = b ± b 2 + 4 a 2 2 a \left(\frac{\beta}{\alpha}\right)_{\pm} = \frac{b \pm\sqrt{b^2 + 4a^2}}{2a} ( α β ) ± = 2 a b ± b 2 + 4 a 2 which are different under the assumption that ( a , b ) ≠ ( 0 , 0 ) (a,b) \ne(0,0) ( a , b ) = ( 0 , 0 ) , therefore the eigenvalues are of the form
a β α = b ± b 2 + 4 a 2 2 = b 2 ± ( b 2 ) 2 + a 2 \frac{a\beta}{\alpha} = \frac{b \pm\sqrt{b^2 + 4a^2}}{2} = \frac{b}{2} \pm\sqrt{\left(\frac{b}{2}\right)^2 + a^2} α a β = 2 b ± b 2 + 4 a 2 = 2 b ± ( 2 b ) 2 + a 2
since
T ( α e j + β e j + d ) = α β α ( α e j + β e j + d ) T(\alpha e_j+\beta e_{j+d})=\frac{\alpha\beta}{\alpha}(\alpha e_j+\beta e_{j+d}) T ( α e j + β e j + d ) = α α β ( α e j + β e j + d )
with respective eigenvectors
α e j + β e j + d ∝ e j + ( β α ) ± e j + d , j = 1 , ⋯ d \alpha e_j+\beta e_{j+d}\propto e_j+\left(\frac{\beta}{\alpha}\right)_{\pm}e_{j+d},\quad j=1,\cdots d α e j + β e j + d ∝ e j + ( α β ) ± e j + d , j = 1 , ⋯ d
so we have found all our 2 d 2d 2 d possible eigenvalues (two cases ± \pm ± for each j = 1 , ⋯ , d j=1,\cdots,d j = 1 , ⋯ , d ). Under this eigenbasis, our operator is represented by the diagonal matrix
[ T ] { e j + ( β / α ) ± e j + d } = ( ( b 2 + ( b 2 ) 2 + a 2 ) I d 0 d 0 d ( b 2 − ( b 2 ) 2 + a 2 ) I d ) [T]_{\{e_j+(\beta/\alpha)_{\pm}e_{j+d}\}}=
\begin{pmatrix}
\left(\frac{b}{2}+\sqrt{\left(\frac{b}{2}\right)^2+a^2}\right)I_d & 0_d\\
0_d & \left(\frac{b}{2}-\sqrt{\left(\frac{b}{2}\right)^2+a^2}\right)I_d
\end{pmatrix} [ T ] { e j + ( β / α ) ± e j + d } = ( 2 b + ( 2 b ) 2 + a 2 ) I d 0 d 0 d ( 2 b − ( 2 b ) 2 + a 2 ) I d
□ \square □
By using the lemma above, if we consider our previous matrix
( 0 d r I d r I d 2 [ 1 + ( γ − 1 ) r ] I d ) \begin{pmatrix}
0_d & rI_d\\
rI_d & 2[1+(\gamma-1)r]I_d
\end{pmatrix} ( 0 d r I d r I d 2 [ 1 + ( γ − 1 ) r ] I d )
then
b = 2 [ 1 + ( γ − 1 ) r ] b=2[1+(\gamma-1)r] b = 2 [ 1 + ( γ − 1 ) r ]
so the eigenvalues of this matrix are
1 + ( γ − 1 ) r ± ( 1 + ( γ − 1 ) r ) 2 + r 2 1+(\gamma-1)r\pm\sqrt{(1+(\gamma-1)r)^2+r^2} 1 + ( γ − 1 ) r ± ( 1 + ( γ − 1 ) r ) 2 + r 2
and going back to our previous example, let’s consider the perturbative problem with parameter ε = O ( 1 / γ 2 ) \varepsilon=O(1/\gamma^2) ε = O ( 1/ γ 2 ) , so that γ ε = O ( 1 / γ ) ≪ 1 \gamma\varepsilon=O(1/\gamma)\ll1 γ ε = O ( 1/ γ ) ≪ 1 since γ ≫ 1 \gamma\gg1 γ ≫ 1 ,
I 2 d + 2 γ ( 0 d r I d r I d 2 [ 1 + ( γ − 1 ) r ] I d ) ε I_{2d}+2\gamma
\begin{pmatrix}
0_d & rI_d\\
rI_d & 2[1+(\gamma-1)r]I_d
\end{pmatrix}^{\varepsilon} I 2 d + 2 γ ( 0 d r I d r I d 2 [ 1 + ( γ − 1 ) r ] I d ) ε
Since the first matrix is the identity and the second one is symmetric (both real valued and 2 d × 2 d 2d\times2d 2 d × 2 d ), one can prove [Holmes, Intro. to Perturbation Methods] that a two-term perturbative expansion of the eigenvalues μ i \mu_i μ i of the matrix above are
μ i = 1 + ε λ i , i = 1 , ⋯ , 2 d \mu_i=1+\varepsilon\lambda_i,\quad i=1,\cdots,2d μ i = 1 + ε λ i , i = 1 , ⋯ , 2 d
with λ i \lambda_i λ i the eigenvalues of the symmetric matrix. For the case ε = 1 \varepsilon=1 ε = 1 , we approximately have that the eigenvalues of
I 2 d + 2 γ ( 0 d r I d r I d 2 [ 1 + ( γ − 1 ) r ] I d ) I_{2d}+2\gamma
\begin{pmatrix}
0_d & rI_d\\
rI_d & 2[1+(\gamma-1)r]I_d
\end{pmatrix} I 2 d + 2 γ ( 0 d r I d r I d 2 [ 1 + ( γ − 1 ) r ] I d )
are
2 + ( γ − 1 ) r ± ( 1 + ( γ − 1 ) r ) 2 + r 2 2+(\gamma-1)r\pm\sqrt{(1+(\gamma-1)r)^2+r^2} 2 + ( γ − 1 ) r ± ( 1 + ( γ − 1 ) r ) 2 + r 2
and if we finally go back to our original eigenproblem for the Hessian,
H = γ Q 11 Q [ I 2 d + 2 γ ( ( 0 d I d / ( 1 + D p p / D q q ) I d / ( 1 + D p p / D q q ) I d 2 { 1 + ( γ − 1 ) / ( 1 + D p p / D q q ) } ) ) ] H = \frac{\gamma Q_{11}}{Q}\left[I_{2d}+2\gamma\left(\begin{pmatrix} 0_d & I_d/(1+D_{pp}/D_{qq}) \\ I_d/(1+D_{pp}/D_{qq}) & I_{d2}\{1+(\gamma-1)/(1+D_{pp}/D_{qq})\} \end{pmatrix}\right)\right] H = Q γ Q 11 [ I 2 d + 2 γ ( ( 0 d I d / ( 1 + D pp / D q q ) I d / ( 1 + D pp / D q q ) I d 2 { 1 + ( γ − 1 ) / ( 1 + D pp / D q q )} ) ) ]
remembering
r = 1 1 + D p p / D q q = D q q D p p + D q q r=\frac{1}{1+D_{pp}/D_{qq}}=\frac{D_{qq}}{D_{pp}+D_{qq}} r = 1 + D pp / D q q 1 = D pp + D q q D q q
then the eigenvalues of H H H , up to first order in a perturbative expansion, are only two and are namely
γ Q 11 Q [ 2 + ( γ − 1 ) r ± ( 1 + ( γ − 1 ) r ) 2 + r 2 ] \frac{\gamma Q_{11}}{Q}\left[2+(\gamma-1)r\pm\sqrt{(1+(\gamma-1)r)^2+r^2}\right] Q γ Q 11 [ 2 + ( γ − 1 ) r ± ( 1 + ( γ − 1 ) r ) 2 + r 2 ]
and recalling
Q 11 = D p p + D q q Q_{11}=D_{pp}+D_{qq} Q 11 = D pp + D q q
Q 12 = 2 γ D q q Q_{12}=2\gamma D_{qq} Q 12 = 2 γ D q q
Q 11 + 4 γ ( D p q + γ D q q ) Q_{11}+4\gamma(D_{pq}+\gamma D_{qq}) Q 11 + 4 γ ( D pq + γ D q q )
then
γ Q [ 2 Q 11 + ( γ − 1 ) Q 11 r ± Q 11 ( 1 + ( γ − 1 ) r ) 2 + r 2 ] \frac{\gamma}{Q}\left[2Q_{11}+(\gamma-1)Q_{11}r\pm Q_{11}\sqrt{(1+(\gamma-1)r)^2+r^2}\right] Q γ [ 2 Q 11 + ( γ − 1 ) Q 11 r ± Q 11 ( 1 + ( γ − 1 ) r ) 2 + r 2 ]
γ Q [ 2 Q 11 + ( γ − 1 ) D q q ± sign ( Q 11 ) ( Q 11 + ( γ − 1 ) Q 11 r ) 2 + ( Q 11 r ) 2 ] \frac{\gamma}{Q}\left[2Q_{11}+(\gamma-1)D_{qq}\pm\operatorname{sign}(Q_{11})\sqrt{(Q_{11}+(\gamma-1)Q_{11}r)^2+(Q_{11}r)^2}\right] Q γ [ 2 Q 11 + ( γ − 1 ) D q q ± sign ( Q 11 ) ( Q 11 + ( γ − 1 ) Q 11 r ) 2 + ( Q 11 r ) 2 ]
γ Q [ 2 Q 11 + ( γ − 1 ) D q q ± ( D p p + D q q + ( γ − 1 ) D q q ) 2 + D q q 2 ] \frac{\gamma}{Q}\left[2Q_{11}+(\gamma-1)D_{qq}\pm\sqrt{(D_{pp}+D_{qq}+(\gamma-1)D_{qq})^2+D_{qq}^2}\right] Q γ [ 2 Q 11 + ( γ − 1 ) D q q ± ( D pp + D q q + ( γ − 1 ) D q q ) 2 + D q q 2 ]
γ Q [ 2 ( D p p + D q q ) + ( γ − 1 ) D q q ± ( D p p + D q q + ( γ − 1 ) D q q ) 2 + D q q 2 ] \frac{\gamma}{Q}\left[2(D_{pp}+D_{qq})+(\gamma-1)D_{qq}\pm\sqrt{(D_{pp}+D_{qq}+(\gamma-1)D_{qq})^2+D_{qq}^2}\right] Q γ [ 2 ( D pp + D q q ) + ( γ − 1 ) D q q ± ( D pp + D q q + ( γ − 1 ) D q q ) 2 + D q q 2 ]
γ Q [ 2 D p p + ( γ + 1 ) D q q ± ( D p p + γ D q q ) 2 + D q q 2 ] \frac{\gamma}{Q}\left[2D_{pp}+(\gamma+1)D_{qq}\pm\sqrt{(D_{pp}+\gamma D_{qq})^2+D_{qq}^2}\right] Q γ [ 2 D pp + ( γ + 1 ) D q q ± ( D pp + γ D q q ) 2 + D q q 2 ]
and
Q = Q 11 ( Q 11 + 4 γ ( D p q + γ D q q ) ) − ( 2 γ D q q ) 2 Q=Q_{11}(Q_{11}+4\gamma(D_{pq}+\gamma D_{qq}))-(2\gamma D_{qq})^2 Q = Q 11 ( Q 11 + 4 γ ( D pq + γ D q q )) − ( 2 γ D q q ) 2
Q = Q 11 2 + 4 γ ( D p q + γ D q q ) Q 11 − ( 2 γ D q q ) 2 Q=Q_{11}^2+4\gamma(D_{pq}+\gamma D_{qq})Q_{11}-(2\gamma D_{qq})^2 Q = Q 11 2 + 4 γ ( D pq + γ D q q ) Q 11 − ( 2 γ D q q ) 2
Q = ( D p p + D q q ) 2 + 4 γ ( D p q + γ D q q ) ( D p p + D q q ) − ( 2 γ D q q ) 2 Q=(D_{pp}+D_{qq})^2+4\gamma(D_{pq}+\gamma D_{qq})(D_{pp}+D_{qq})-(2\gamma D_{qq})^2 Q = ( D pp + D q q ) 2 + 4 γ ( D pq + γ D q q ) ( D pp + D q q ) − ( 2 γ D q q ) 2
In conclusion, by perturbative methods, up to first order, the eigenvalues of the Hessian
H = γ Q 11 Q [ I 2 d + 2 γ ( ( 0 d I d / ( 1 + D p p / D q q ) I d / ( 1 + D p p / D q q ) I d 2 { 1 + ( γ − 1 ) / ( 1 + D p p / D q q ) } ) ) ] H=\frac{\gamma Q_{11}}{Q}\left[I_{2d}+2\gamma\left(\begin{pmatrix} 0_d & I_d/(1+D_{pp}/D_{qq}) \\ I_d/(1+D_{pp}/D_{qq}) & I_{d2}\{1+(\gamma-1)/(1+D_{pp}/D_{qq})\} \end{pmatrix}\right)\right] H = Q γ Q 11 [ I 2 d + 2 γ ( ( 0 d I d / ( 1 + D pp / D q q ) I d / ( 1 + D pp / D q q ) I d 2 { 1 + ( γ − 1 ) / ( 1 + D pp / D q q )} ) ) ]
are only the following two:
γ [ 2 D p p + ( γ + 1 ) D q q ± ( D p p + γ D q q ) 2 + D q q 2 ] ( D p p + D q q ) 2 + 4 γ ( D p q + γ D q q ) ( D p p + D q q ) − ( 2 γ D q q ) 2 \frac{\gamma[2D_{pp}+(\gamma+1)D_{qq}\pm\sqrt{(D_{pp}+\gamma D_{qq})^2+D_{qq}^2}]}{(D_{pp}+D_{qq})^2+4\gamma(D_{pq}+\gamma D_{qq})(D_{pp}+D_{qq})-(2\gamma D_{qq})^2} ( D pp + D q q ) 2 + 4 γ ( D pq + γ D q q ) ( D pp + D q q ) − ( 2 γ D q q ) 2 γ [ 2 D pp + ( γ + 1 ) D q q ± ( D pp + γ D q q ) 2 + D q q 2 ]
under the renormalization of the position variable x x x equivalent to taking ω 0 = 1 \omega_0 = 1 ω 0 = 1 . Since the optimal exponential convergence rate κ \kappa κ is the smallest eigenvalue of the Hessian above, then up to first order the convergence rate is
κ = γ 2 D p p + ( γ + 1 ) D q q − ( D p p + γ D q q ) 2 + D q q 2 ( D p p + D q q ) 2 + 4 γ ( D p q + γ D q q ) ( D p p + D q q ) − ( 2 γ D q q ) 2 \kappa= \gamma\frac{2D_{pp} + (\gamma+ 1)D_{qq} - \sqrt{(D_{pp} + \gamma D_{qq})^2 + D_{qq}^2}}{(D_{pp} + D_{qq})^2 + 4\gamma(D_{pq} + \gamma D_{qq})(D_{pp} + D_{qq}) - (2\gamma D_{qq})^2} κ = γ ( D pp + D q q ) 2 + 4 γ ( D pq + γ D q q ) ( D pp + D q q ) − ( 2 γ D q q ) 2 2 D pp + ( γ + 1 ) D q q − ( D pp + γ D q q ) 2 + D q q 2
which (again, up to first order), is independent of the dimension d d d of the phase-space variables ( x , p ) ∈ R d × R d (x,p) \in\mathbb{R}^d \times\mathbb{R}^d ( x , p ) ∈ R d × R d of the problem.
Remark 3. The reason this perturbative calculation was kept as a preliminary calculation in the Appendix was because the perturbative argument splitting the matrix into an identity matrix and a remainder needs this remainder to be small (as for example with a perturbative parameter ε ≫ 1 \varepsilon\gg1 ε ≫ 1 ), however the remainder matrix is multiplied by the friction parameter γ ≫ 1 \gamma\gg1 γ ≫ 1 which we have assumed to be quite large in order to make the gradient transport due to the friction dominant over the Hamiltonian one. Therefore, though formally a correct calculation with the perturbative parameter ε \varepsilon ε , once removing ε \varepsilon ε then the assumption needed for the perturbative calculation does not apply anymore and therefore we keep this perturbative analysis as a formal calculation for the Appendix only, but not the definite approach for our final results, presented previously in the bulk of the text.