Statement on AI Usage. All ideas and technical contributions in this paper, as well as Part I [21 ] , were biologically generated. Claude Code (Opus 5.5) was used for language editing, proofreading, and transcribing a hand-drawn version of Figure 1 into TikZ code. The authors bear sole responsibility for the correctness of this manuscript.
Figure 1. Phase diagram for the recovery problem in tensor PCA in terms of λ \lambda λ
Introduction
This paper is the second of a two-part series on local Markov chains for the tensor principal components analysis (tensor PCA) problem, which involves recovering an unknown vector from a noisy observation of its tensor power. Originally introduced by Richard and Montanari [45 ] as a generalization of classical spiked matrix models in statistics and machine learning [30 , 6 ] , tensor PCA is a canonical planted problem in average-case complexity theory [26 , 23 , 24 , 13 , 5 , 16 , 20 , 34 , 43 , 32 WEM25, 35 , 14 ] . It is also widely studied in high-dimensional statistics [37 , 44 , 29 ] and in the theory of spin glasses in statistical physics [1 , 9 , 46 , 7 , 10 ] . Part I of this series [21 ] analyzes Glauber dynamics for Ising tensor PCA , in which the unknown vector lies on the signed Boolean hypercube { ± 1 } N \{\pm1\}^{N} { ± 1 } N spherical tensor PCA , in which the unknown vector lies on the sphere S N = { x ∈ R N : ∥ x ∥ 2 = N } \mathcal{S}_{N} = \{x \in\mathbb{R}^{N}: \lVert x\rVert^{2} = N\} S N = { x ∈ R N : ∥ x ∥ 2 = N }
Formally, for a fixed integer k ≥ 3 k \ge3 k ≥ 3 x ⋆ ∈ S N x^{\star} \in\mathcal{S}_{N} x ⋆ ∈ S N k k k
Y = λ N ( θ ⋆ ) ⊗ k + G , θ ⋆ = x ⋆ N . Y = \lambda\sqrt{N}(\theta^{\star})^{\otimes k} + G, \qquad\theta^{\star} = \frac{x^{\star}}{\sqrt{N}}. Y = λ N ( θ ⋆ ) ⊗ k + G , θ ⋆ = N x ⋆ . where λ ≥ 0 \lambda\ge0 λ ≥ 0 G ∈ ( R N ) ⊗ k G \in(\mathbb{R}^{N})^{\otimes k} G ∈ ( R N ) ⊗ k k k k N ( 0 , 1 ) \mathcal{N}(0,1) N ( 0 , 1 ) x ⋆ x^{\star} x ⋆ k k k single observation of the noisy tensor Y Y Y x ^ = x ^ ( Y ) ∈ S N \widehat{x} = \widehat{x}(Y) \in\mathcal{S}_{N} x = x ( Y ) ∈ S N x ⋆ x^{\star} x ⋆ weak (resp. strong ) recovery if its correlation N − 1 ⟨ x ^ , x ⋆ ⟩ N^{-1}\langle\widehat{x}, x^{\star}\rangle N − 1 ⟨ x , x ⋆ ⟩ = 1 − o N ( 1 ) = 1-o_{N}(1) = 1 − o N ( 1 ) k k k N − 1 ∣ ⟨ x ^ , x ⋆ ⟩ ∣ N^{-1}|\langle\widehat{x},x^{\star}\rangle| N − 1 ∣ ⟨ x , x ⋆ ⟩ ∣
As is discussed at length in Part I [21 Section 1] , the tensor PCA problem exhibits a well-studied statistical-computational gap (see Figure 1 ). Weak recovery is information-theoretically possible once λ \lambda λ λ I T = Θ k ( 1 ) \lambda_{\mathrm{IT}} = \Theta_{k}(1) λ IT = Θ k ( 1 ) [37 , 44 , 29 ] , whereas global algorithms, such as tensor unfolding [45 , 26 ] , partial traces [23 , 3 ] , and the sum-of-squares hierarchy [26 ] , achieve strong recovery in polynomial time above the polynomially larger algorithmic threshold λ A L G ≍ N k − 2 4 \lambda_{\mathrm{ALG}} \asymp N^{\frac{k-2}{4}} λ ALG ≍ N 4 k − 2 cold start . In particular, Ben Arous, Gheissari, and Jagannath [10 ] showed that spherical Langevin dynamics at any fixed temperature, initialized from the uniform measure on S N \mathcal{S}_{N} S N λ ≫ N k − 2 2 \lambda\gg N^{\frac{k-2}{2}} λ ≫ N 2 k − 2 N k − 2 4 N^{\frac{k-2}{4}} N 4 k − 2 [10 Section 1, see also Ban+23, Section 2(b)] , providing evidence via lower bounds for disorder-dependent Gibbs-type initializations. The same scale is also believed to be the threshold for other cold-started local algorithms, such as power iteration and approximate message passing [45 , 27 , 50 ] . This apparent weakness motivates the following central question of our two-part work:
“Can polynomial-time algorithms based on canonical local Markov chains initialized from a cold start achieve recovery down to the algorithmic threshold?”
For Ising tensor PCA, Part I [21 ] answered this question in the affirmative by establishing that Glauber dynamics, a canonical local Markov chain on the hypercube, achieves strong recovery in polynomial time above the algorithmic threshold from a uniformly random initialization when supplemented with annealing and ergodic averaging. In this paper, we show that the same holds for spherical tensor PCA, with Glauber dynamics replaced by spherical Langevin dynamics, the canonical local Markov chain on the sphere, which was analyzed at a fixed temperature in [10 ] . This extension is far from straightforward, as spherical tensor PCA differs substantially from its Ising counterpart. In particular, S N \mathcal{S}_N S N § 2 , the key conceptual ideas of Part I, which rely on only a few structural properties of the problem, carry over to the sphere once carefully adapted.
Our Results
Our first result is a simple, fully local algorithm for spherical tensor PCA that achieves polynomial-time strong recovery for all λ ≳ λ A L G \lambda\gtrsim\lambda_{\mathrm{ALG}} λ ≳ λ ALG [21 Section 1.1] , our algorithm is spherical Langevin dynamics from a uniform initialization, with a simple twist. Algorithm 1 first runs high temperature spherical Langevin dynamics and computes the ergodic average of its trajectory. It then normalizes this average to lie on S N \mathcal{S}_N S N k k k
Theorem 1.1 (Informal). Let k ≥ 3 k \ge3 k ≥ 3 H ( x ) = N − k − 1 2 ⟨ Y , x ⊗ k ⟩ H(x) = N^{-\frac{k-1}{2}} \langle Y, x^{\otimes k} \rangle H ( x ) = N − 2 k − 1 ⟨ Y , x ⊗ k ⟩ S N \mathcal{S}_N S N λ ≳ λ A L G \lambda\gtrsim\lambda_{\mathrm{ALG}} λ ≳ λ ALG T a v ≳ N 2 k − 2 + o ( 1 ) T_{\mathrm{av}} \gtrsim N^{2k-2+o(1)} T av ≳ N 2 k − 2 + o ( 1 ) 1 ≲ T ≲ poly ( N ) 1 \lesssim T \lesssim\operatorname{poly}(N) 1 ≲ T ≲ poly ( N ) 1 − exp ( − N 1 / 2 − o ( 1 ) ) 1-\exp(-N^{1/2-o(1)}) 1 − exp ( − N 1/2 − o ( 1 ) )
When the tensor order k k k F ( x ) = ( x , 1 ) ∈ R N + 1 F(x)=(x,1) \in\mathbb{R}^{N+1} F ( x ) = ( x , 1 ) ∈ R N + 1 k ≥ 3 k \ge3 k ≥ 3
Algorithm 1 Annealed Spherical Langevin Dynamics with Ergodic Averaging
Input: Hamiltonian H : S N → R H:\mathcal{S}_N \to\mathbb{R} H : S N → R X ~ 0 ∈ S N \widetilde{X}_0 \in\mathcal{S}_N X 0 ∈ S N β 0 , β > 0 \beta_0,\beta>0 β 0 , β > 0 T a v , T > 0 T_{\mathrm{av}},T>0 T av , T > 0
Run spherical Langevin dynamics with Hamiltonian H H H β 0 \beta_0 β 0 X ~ 0 \widetilde{X}_0 X 0 T a v T_{\mathrm{av}} T av ( X ~ t ) 0 ≤ t ≤ T a v (\widetilde{X}_t)_{0\le t\le T_{\mathrm{av}}} ( X t ) 0 ≤ t ≤ T av
d X ~ t = ( β 0 ⋅ Q X ~ t ∇ H ( X ~ t ) − N − 1 2 N X ~ t ) d t + Q X ~ t d B ~ t , Q x ≔ I − x x T ∥ x ∥ 2 . \mathrm{d}\widetilde{X}_t=\left(\beta_0\cdot Q_{\widetilde{X}_t}\nabla H(\widetilde{X}_t)-\frac{N-1}{2N}\widetilde{X}_t\right)\mathrm{d}t+Q_{\widetilde{X}_t}\mathrm{d}\widetilde{B}_t,\qquad Q_x\coloneqq I-\frac{xx^{\mathsf{T}}}{\lVert x\rVert^2}. d X t = ( β 0 ⋅ Q X t ∇ H ( X t ) − 2 N N − 1 X t ) d t + Q X t d B t , Q x : = I − ∥ x ∥ 2 x x T . Compute the ergodic average:
X ‾ T a v = 1 T a v ∫ 0 T a v X ~ t d t . \overline{X}_{T_{\mathrm{av}}}=\frac{1}{T_{\mathrm{av}}}\int_0^{T_{\mathrm{av}}}\widetilde{X}_t\mathrm{d}t. X T av = T av 1 ∫ 0 T av X t d t . Run spherical Langevin dynamics with Hamiltonian H H H β \beta β X 0 = N X ‾ T a v ∥ X ‾ T a v ∥ X_0=\frac{\sqrt{N}\overline{X}_{T_{\mathrm{av}}}}{\lVert\overline{X}_{T_{\mathrm{av}}}\rVert} X 0 = ∥ X T av ∥ N X T av T T T ( X t ) 0 ≤ t ≤ T (X_t)_{0\le t\le T} ( X t ) 0 ≤ t ≤ T
d X t = ( β ⋅ Q X t ∇ H ( X t ) − N − 1 2 N X t ) d t + Q X t d B t , Q x ≔ I − x x T ∥ x ∥ 2 . \mathrm{d}X_t=\left(\beta\cdot Q_{X_t}\nabla H(X_t)-\frac{N-1}{2N}X_t\right)\mathrm{d}t+Q_{X_t}\mathrm{d}B_t,\qquad Q_x\coloneqq I-\frac{xx^{\mathsf{T}}}{\lVert x\rVert^2}. d X t = ( β ⋅ Q X t ∇ H ( X t ) − 2 N N − 1 X t ) d t + Q X t d B t , Q x : = I − ∥ x ∥ 2 x x T . Output: X T X_T X T
Theorem 1.2 (Informal). Let k ≥ 3 k\ge3 k ≥ 3 H ( x ) = N − k − 1 2 ⟨ Y , x ⊗ k ⟩ H(x)=N^{-\frac{k-1}{2}}\langle Y,x^{\otimes k}\rangle H ( x ) = N − 2 k − 1 ⟨ Y , x ⊗ k ⟩ S N \mathcal{S}_N S N λ ≳ λ A L G \lambda\gtrsim\lambda_{\mathrm{ALG}} λ ≳ λ ALG T a v ≳ N 2 k + o ( 1 ) T_{\mathrm{av}}\gtrsim N^{2k+o(1)} T av ≳ N 2 k + o ( 1 ) 1 ≲ T ≲ poly ( N ) 1\lesssim T\lesssim\operatorname{poly}(N) 1 ≲ T ≲ poly ( N ) 1 − exp ( − N 1 / 2 − o ( 1 ) ) 1-\exp(-N^{1/2-o(1)}) 1 − exp ( − N 1/2 − o ( 1 ) )
Remark 1.3. Mirroring Part I [21 Remark 1.3] , the warm start in Algorithm 2 is computed via an ergodic statistic of the underlying local Markov chain itself. This differs from prior warm-started local algorithms [45 , 3 , 38 ] , which obtain their initializations from the observed tensor Y Y Y [42 , 33 ] alongside the high temperature trajectory,
z ˙ t = η t ⟨ F ( X ~ t ) , z t ⟩ F ( X ~ t ) , F ( x ) = ( x , 1 ) ∈ R N + 1 , z 0 ∼ Uniform ( S N ) , \dot{z}_t=\eta_t\langle F(\widetilde{X}_t),z_t\rangle F(\widetilde{X}_t),\qquad F(x)=(x,1)\in\mathbb{R}^{N+1},\qquad z_0\sim\operatorname{Uniform}(\mathcal{S}^N), z ˙ t = η t ⟨ F ( X t ) , z t ⟩ F ( X t ) , F ( x ) = ( x , 1 ) ∈ R N + 1 , z 0 ∼ Uniform ( S N ) , with step sizes η t > 0 \eta_t>0 η t > 0 ( v , a ) (v,a) ( v , a ) z T a v z_{T_{\mathrm{av}}} z T av z t z_t z t X ~ t \widetilde{X}_t X t
We highlight that Algorithms 1 and 2 are direct adaptations of the two algorithms of Part I [21 Section 1.1] , with spherical Langevin dynamics in place of Glauber dynamics. As such, they remain agnostic to the Hamiltonian, i.e., they are defined for any smooth H : S N → R H:\mathcal{S}_N\to\mathbb{R} H : S N → R
Algorithm 2 Annealed Spherical Langevin Dynamics with Augmented Ergodic Second Moment
Input: Hamiltonian H : S N → R H : \mathcal{S}_N \to\mathbb{R} H : S N → R X ~ 0 ∈ S N \widetilde{X}_0 \in\mathcal{S}_N X 0 ∈ S N β 0 , β > 0 \beta_0,\beta> 0 β 0 , β > 0 T a v , T > 0 T_{\mathrm{av}},T > 0 T av , T > 0
Run spherical Langevin dynamics with Hamiltonian H H H β 0 \beta_0 β 0 X ~ 0 \widetilde{X}_0 X 0 T a v T_{\mathrm{av}} T av ( X ~ t ) 0 ≤ t ≤ T a v (\widetilde{X}_t)_{0 \le t \le T_{\mathrm{av}}} ( X t ) 0 ≤ t ≤ T av
d X ~ t = ( β 0 ⋅ Q X ~ t ∇ H ( X ~ t ) − N − 1 2 N X ~ t ) d t + Q X ~ t d B ~ t , Q x ≔ I − x x T ∥ x ∥ 2 . \mathrm{d}\widetilde{X}_t=\left(\beta_0\cdot Q_{\widetilde{X}_t}\nabla H(\widetilde{X}_t)-\frac{N-1}{2N}\widetilde{X}_t\right)\mathrm{d}t+Q_{\widetilde{X}_t}\mathrm{d}\widetilde{B}_t,\qquad Q_x\coloneqq I-\frac{xx^{\mathsf{T}}}{\lVert x\rVert^2}. d X t = ( β 0 ⋅ Q X t ∇ H ( X t ) − 2 N N − 1 X t ) d t + Q X t d B t , Q x : = I − ∥ x ∥ 2 x x T . Compute the augmented ergodic second moment:
M ‾ T a v = 1 T a v ∫ 0 T a v F ( X ~ t ) ⊗ 2 d t , F ( x ) ≔ ( x , 1 ) ∈ R N + 1 . \overline{M}_{T_{\mathrm{av}}}=\frac{1}{T_{\mathrm{av}}}\int_0^{T_{\mathrm{av}}}F(\widetilde{X}_t)^{\otimes2}\,\mathrm{d}t,\qquad F(x)\coloneqq(x,1)\in\mathbb{R}^{N+1}. M T av = T av 1 ∫ 0 T av F ( X t ) ⊗ 2 d t , F ( x ) : = ( x , 1 ) ∈ R N + 1 . Compute a top unit eigenvector ( v , a ) ∈ R N × R (v,a)\in\mathbb{R}^N\times\mathbb{R} ( v , a ) ∈ R N × R M ‾ T a v \overline{M}_{T_{\mathrm{av}}} M T av a ≥ 0 a\ge0 a ≥ 0
Run spherical Langevin dynamics with Hamiltonian H H H β \beta β X 0 = N v ∥ v ∥ X_0=\frac{\sqrt{N}v}{\lVert v\rVert} X 0 = ∥ v ∥ N v T T T ( X t ) 0 ≤ t ≤ T (X_t)_{0 \le t \le T} ( X t ) 0 ≤ t ≤ T
d X t = ( β ⋅ Q X t ∇ H ( X t ) − N − 1 2 N X t ) d t + Q X t d B t , Q x ≔ I − x x T ∥ x ∥ 2 . \mathrm{d}X_t=\left(\beta\cdot Q_{X_t}\nabla H(X_t)-\frac{N-1}{2N}X_t\right)\mathrm{d}t+Q_{X_t}\mathrm{d}B_t,\qquad Q_x\coloneqq I-\frac{xx^{\mathsf{T}}}{\lVert x\rVert^2}. d X t = ( β ⋅ Q X t ∇ H ( X t ) − 2 N N − 1 X t ) d t + Q X t d B t , Q x : = I − ∥ x ∥ 2 x x T . Output: X T X_T X T
Acknowledgements
We thank Brice Huang for helpful discussions and feedback. The authors acknowledge the support of the NSF AI Institute for Foundations of Machine Learning (IFML) and of T.S.’s NSF CAREER award # 2143246.
Organization
We sketch the key ideas behind our proofs in § 2 , and review the necessary notation and preliminaries in § 3 . In § 4 , we carry out a signal-noise decomposition of the average spherical gradient and Hessian of the tensor PCA Hamiltonian, and § 5 shows that ergodic averages of high temperature spherical Langevin dynamics recover these averaged derivatives. We then analyze the second phase of Algorithm 1 in § 6 , proving a formal version of Theorem 1.1 . Finally, we extend this analysis to Algorithm 2 in § 7 , proving a formal version of Theorem 1.2 .
Proof Sketch
Before presenting our key ideas, we first describe the tensor PCA Hamiltonian in more detail and fix some notation. A natural approach to recovery is to solve the Maximum Likelihood Estimation (MLE) problem, defined as follows:
x ^ M L E ∈ argmax x ∈ S N ⟨ Y , x ⊗ k ⟩ . (1) \widehat{x}_{\mathrm{MLE}}\in\underset{x\in\mathcal{S}_N}{\operatorname{argmax}}\left\langle Y,x^{\otimes k}\right\rangle. \tag*{(1)} x MLE ∈ x ∈ S N argmax ⟨ Y , x ⊗ k ⟩ . (1) We approach the objective in Equation (1) indirectly, by running spherical Langevin dynamics with the corresponding Hamiltonian1 H : S N → R H:\mathcal{S}_N\to\mathbb{R} H : S N → R
H ( x ) ≔ N − k − 1 2 ⟨ Y , x ⊗ k ⟩ = N λ m ( x ) k + H 0 ( x ) , H(x) \coloneqq N^{-\frac{k-1}{2}}\left\langle Y,x^{\otimes k}\right\rangle= N\lambda m(x)^k+H_0(x), H ( x ) : = N − 2 k − 1 ⟨ Y , x ⊗ k ⟩ = N λm ( x ) k + H 0 ( x ) , m ( x ) ≔ 1 N ⟨ x ⋆ , x ⟩ = 1 N ⟨ θ ⋆ , x ⟩ , H 0 ( x ) ≔ N − k − 1 2 ⟨ G , x ⊗ k ⟩ , (2) m(x)\coloneqq\frac{1}{N}\langle x^\star,x\rangle=\frac{1}{\sqrt{N}}\langle\theta^\star,x\rangle,\qquad H_0(x)\coloneqq N^{-\frac{k-1}{2}}\left\langle G,x^{\otimes k}\right\rangle, \tag*{(2)} m ( x ) : = N 1 ⟨ x ⋆ , x ⟩ = N 1 ⟨ θ ⋆ , x ⟩ , H 0 ( x ) : = N − 2 k − 1 ⟨ G , x ⊗ k ⟩ , (2) where m : S N → [ − 1 , 1 ] m:\mathcal{S}_N\to[-1,1] m : S N → [ − 1 , 1 ] H 0 H_0 H 0 k k k [19 , 1 ] , a centered isotropic Gaussian process on S N \mathcal{S}_N S N
E [ H 0 ( x ) H 0 ( y ) ] = N ( ⟨ x , y ⟩ N ) k , x , y ∈ S N . (3) \mathbb{E}[H_0(x)H_0(y)]=N\left(\frac{\langle x,y\rangle}{N}\right)^k,\qquad x,y\in\mathcal{S}_N. \tag*{(3)} E [ H 0 ( x ) H 0 ( y )] = N ( N ⟨ x , y ⟩ ) k , x , y ∈ S N . (3) We refer to N λ m ( x ) k N\lambda m(x)^k N λm ( x ) k H 0 H_0 H 0 π \pi π S N \mathcal{S}_N S N
We now sketch the main ideas behind our results. Our analysis extends the proof techniques that were developed in Part I [21 ] for Glauber dynamics, to spherical Langevin dynamics, and each of its steps has a direct counterpart in Part I. At the level of a proof overview, the core ideas behind the two analyses are in many cases identical. Thus, to keep this paper self-contained, this section closely follows the corresponding section of Part I [21 Section 2] , often verbatim. The required technical modifications to the proof strategy stem from the Riemannian geometry of S N \mathcal{S}_N S N k k k § 2.1 to § 2.4 , and the ideas behind our analysis of Algorithm 2 in § 2.5 , highlighting these modifications along the way.
Escaping the Equator
The starting point of our analysis is the following heuristic picture of spherical Langevin dynamics proposed by Ben Arous, Gheissari, and Jagannath [10 Section 2.5.1] . Let ( X t ) t ≥ 0 (X_t)_{t\geq0} ( X t ) t ≥ 0 H H H β > 0 \beta>0 β > 0 f : R N → R f:\mathbb{R}^N\to\mathbb{R} f : R N → R ∇ S N f ( x ) = Q x ∇ f ( x ) \nabla_{\mathcal{S}_N}f(x)=Q_x\nabla f(x) ∇ S N f ( x ) = Q x ∇ f ( x ) x ∈ S N x\in\mathcal{S}_N x ∈ S N ∇ f \nabla f ∇ f Q x = I − x x T N Q_x=I-\frac{xx^\mathsf{T}}{N} Q x = I − N x x T S N \mathcal{S}_N S N x x x [10 Eq. (2.14)] for the correlation m t = m ( X t ) m_t=m(X_t) m t = m ( X t )
d m t d t ≈ β k λ m t k − 1 ( 1 − m t 2 ) ⏟ signal drift − N − 1 2 N m t ⏟ entropic drift + β N ⟨ x ⋆ , ∇ S N H 0 ( X t ) ⟩ ⏟ disorder drift . (4) \frac{\mathrm{d}m_t}{\mathrm{d}t}\approx\underbrace{\beta k\lambda m_t^{k-1}(1-m_t^2)}_{\text{signal}\\ \text{drift}}-\underbrace{\frac{N-1}{2N}m_t}_{\text{entropic}\\ \text{drift}}+\underbrace{\frac{\beta}{N}\left\langle x^\star,\nabla_{\mathcal{S}_N}H_0(X_t)\right\rangle}_{\text{disorder}\\ \text{drift}}. \tag*{(4)} d t d m t ≈ signal drift β k λ m t k − 1 ( 1 − m t 2 ) − entropic drift 2 N N − 1 m t + disorder drift N β ⟨ x ⋆ , ∇ S N H 0 ( X t ) ⟩ . (4) The disorder drift is a highly intricate random function of the trajectory, since X t X_t X t H 0 H_0 H 0
Setting aside the disorder drift for now, the evolution of m t m_t m t m t m_t m t β = 0 \beta= 0 β = 0 m t m_t m t { x ∈ S N : ∣ m ( x ) ∣ ≲ N − 1 / 2 } \{x \in\mathcal{S}_N : |m(x)| \lesssim N^{-1/2}\} { x ∈ S N : ∣ m ( x ) ∣ ≲ N − 1/2 } S N \mathcal{S}_N S N x ⋆ x^\star x ⋆ 2 β k λ m t k − 2 2\beta k\lambda m_t^{k-2} 2 β k λ m t k − 2 m t m_t m t 2 β k λ m t k − 2 > 1 2\beta k\lambda m_t^{k-2} > 1 2 β k λ m t k − 2 > 1 m 0 m_0 m 0 λ m 0 k − 2 ≳ 1 \lambda m_0^{k-2} \gtrsim1 λ m 0 k − 2 ≳ 1 m 0 ≳ λ − 1 / ( k − 2 ) m_0 \gtrsim\lambda^{-1/(k-2)} m 0 ≳ λ − 1/ ( k − 2 ) [9 , 46 ] , which show that spurious local maxima of the Hamiltonian are confined to the band ∣ m ∣ ≲ λ − 1 / ( k − 2 ) |m| \lesssim\lambda^{-1/(k-2)} ∣ m ∣ ≲ λ − 1/ ( k − 2 )
A uniformly random initialization has correlation of order N − 1 / 2 N^{-1/2} N − 1/2 λ ≳ N ( k − 2 ) / 2 \lambda\gtrsim N^{(k-2)/2} λ ≳ N ( k − 2 ) /2 [10 Theorem 2.1] that spherical Langevin dynamics initialized uniformly at random on the hemisphere { x ∈ S N : m ( x ) > 0 } \{x \in\mathcal{S}_N : m(x) > 0\} { x ∈ S N : m ( x ) > 0 } O ( 1 ) O(1) O ( 1 ) λ = N α \lambda= N^\alpha λ = N α α > k − 2 2 \alpha> \frac{k-2}{2} α > 2 k − 2 λ ≍ λ A L G \lambda\asymp\lambda_{\mathrm{ALG}} λ ≍ λ ALG λ A L G − 1 / ( k − 2 ) ≍ N − 1 / 4 \lambda_{\mathrm{ALG}}^{-1/(k-2)} \asymp N^{-1/4} λ ALG − 1/ ( k − 2 ) ≍ N − 1/4 natural warm start, which (i) achieves correlation Ω ( N − 1 / 4 ) \Omega(N^{-1/4}) Ω ( N − 1/4 ) λ ≍ λ A L G \lambda\asymp\lambda_{\mathrm{ALG}} λ ≍ λ ALG H H H S N \mathcal{S}_N S N
Warm Start via the Average Spherical Gradient
Recall that our goal is to construct a warm start from the trajectory of spherical Langevin dynamics itself. Since spherical Langevin dynamics depends on H H H ∇ S N H ( x ) = k λ m ( x ) k − 1 Q x x ⋆ + ∇ S N H 0 ( x ) \nabla_{\mathcal{S}_N} H(x) = k\lambda m(x)^{k-1}Q_x x^\star+ \nabla_{\mathcal{S}_N} H_0(x) ∇ S N H ( x ) = k λm ( x ) k − 1 Q x x ⋆ + ∇ S N H 0 ( x ) Q x x ⋆ Q_x x^\star Q x x ⋆ x x x m ( x ) k − 1 ≥ 0 m(x)^{k-1} \ge0 m ( x ) k − 1 ≥ 0 k k k λ N ( k − 2 ) / 2 \lambda N^{(k-2)/2} λ N ( k − 2 ) /2 N \sqrt{N} N
Fortunately, although H 0 H_0 H 0 [1 ] , its values at typical pairs of points are nearly independent, since its covariance only depends on the overlap ⟨ x , y ⟩ / N \langle x,y\rangle/N ⟨ x , y ⟩ / N x , y ∼ π x,y \sim\pi x , y ∼ π ⟨ x , y ⟩ / N \langle x,y\rangle/\sqrt{N} ⟨ x , y ⟩ / N N ( 0 , 1 ) \mathcal{N}(0,1) N ( 0 , 1 )
E [ H 0 ( x ) H 0 ( y ) ] = N ( ⟨ x , y ⟩ N ) k = N 1 − k 2 ( ⟨ x , y ⟩ N ) k ≲ N 1 − k 2 ≤ N − 1 / 2 . \mathbb{E}[H_0(x)H_0(y)] = N\left(\frac{\langle x,y\rangle}{N}\right)^k = N^{1-\frac{k}{2}}\left(\frac{\langle x,y\rangle}{\sqrt{N}}\right)^k \lesssim N^{1-\frac{k}{2}} \le N^{-1/2}. E [ H 0 ( x ) H 0 ( y )] = N ( N ⟨ x , y ⟩ ) k = N 1 − 2 k ( N ⟨ x , y ⟩ ) k ≲ N 1 − 2 k ≤ N − 1/2 . whereas E [ H 0 ( x ) 2 ] = N \mathbb{E}[H_0(x)^2] = N E [ H 0 ( x ) 2 ] = N π \pi π x ⋆ x^\star x ⋆ u H ≔ E π [ ∇ S N H ] u_H \coloneqq\mathbb{E}_{\pi}[\nabla_{\mathcal{S}_N} H] u H : = E π [ ∇ S N H ]
By Stein’s lemma for the uniform measure on S N \mathcal{S}_N S N § 3.1 , we have u H = κ N E π [ x H ( x ) ] u_H = \kappa_N\mathbb{E}_{\pi}[xH(x)] u H = κ N E π [ x H ( x )] κ N ≔ N − 1 N \kappa_N \coloneqq\frac{N-1}{N} κ N : = N N − 1 u H u_H u H Y Y Y § 4 , we use the rotation invariance of π \pi π H 0 H_0 H 0 u H u_H u H x ⋆ x^\star x ⋆
u H = κ N − 1 λ σ N 2 x ⋆ + ζ , ζ ∼ N ( 0 , σ N 2 I ) , σ N 2 = κ N 2 N E x , x ′ ∼ i . i . d . π [ ( ⟨ x , x ′ ⟩ N ) k + 1 ] ≍ k N − k − 1 2 . (5) u_H=\kappa_N^{-1}\lambda\sigma_N^2 x^\star+\zeta,\qquad\zeta\sim\mathcal{N}(0,\sigma_N^2 I),\qquad\sigma_N^2=\kappa_N^2 N\mathbb{E}_{x,x'\mathrel{\mathop{\sim}^{\mathrm{i.i.d.}}}\pi}\left[\left(\frac{\langle x,x'\rangle}{N}\right)^{k+1}\right]\asymp_k N^{-\frac{k-1}{2}}. \tag*{(5)} u H = κ N − 1 λ σ N 2 x ⋆ + ζ , ζ ∼ N ( 0 , σ N 2 I ) , σ N 2 = κ N 2 N E x , x ′ ∼ i.i.d. π [ ( N ⟨ x , x ′ ⟩ ) k + 1 ] ≍ k N − 2 k − 1 . (5) Thus, u H u_H u H κ N − 1 λ σ N ≍ k λ N − k − 1 4 \kappa_N^{-1}\lambda\sigma_N\asymp_k\lambda N^{-\frac{k-1}{4}} κ N − 1 λ σ N ≍ k λ N − 4 k − 1 N k − 1 4 N^{\frac{k-1}{4}} N 4 k − 1 u ^ H : = N u H ∥ u H ∥ \widehat{u}_H:=\frac{\sqrt{N}u_H}{\lVert u_H\rVert} u H := ∥ u H ∥ N u H sgn ( u H ) \operatorname{sgn}(u_H) sgn ( u H ) [21 ] ), satisfies the following with high probability:
m ( u ^ H ) ≍ k min { 1 , λ N − k − 1 4 } . m(\widehat{u}_H)\asymp_k\min\{1,\lambda N^{-\frac{k-1}{4}}\}. m ( u H ) ≍ k min { 1 , λ N − 4 k − 1 } . In particular, if λ ≤ N k − 1 4 \lambda\le N^{\frac{k-1}{4}} λ ≤ N 4 k − 1 λ m ( u ^ H ) k − 2 ≍ k ( λ N − k − 1 4 ) k − 1 \lambda m(\widehat{u}_H)^{k-2}\asymp_k(\lambda N^{-\frac{k-1}{4}})^{k-1} λm ( u H ) k − 2 ≍ k ( λ N − 4 k − 1 ) k − 1 u ^ H \widehat{u}_H u H λ ≳ λ A L G \lambda\gtrsim\lambda_{\mathrm{ALG}} λ ≳ λ ALG
This suggests initializing spherical Langevin dynamics at a constant temperature from u ^ H \widehat{u}_H u H u H u_H u H
The idea of averaging the gradient to suppress the disorder also underlies the averaged gradient descent algorithm of Biroli, Cammarota, and Ricci-Tersenghi [7 ] , which minimizes the energy − H -H − H Euclidean gradient ∇ H \nabla H ∇ H S N \mathbb{S}_N S N k = 3 k=3 k = 3 [7 ] provide non-rigorous evidence that their method succeeds above the algorithmic threshold. In contrast, our algorithms come with rigorous guarantees and, as we discuss below, approximate the average gradient u H u_H u H
Approximating the Average Gradient via High Temperature Langevin Dynamics
Our analysis of Glauber dynamics in Part I [21 ] suggests that u H u_H u H X ‾ T : = 1 T ∫ 0 T X ~ t d t \overline{X}_T:=\frac{1}{T}\int_0^T\widetilde{X}_t\,\mathrm{d}t X T := T 1 ∫ 0 T X t d t ( X ~ t ) t ≥ 0 (\widetilde{X}_t)_{t\ge0} ( X t ) t ≥ 0 H H H β 0 > 0 \beta_0>0 β 0 > 0 μ H , β 0 ∝ π e 2 β 0 H \mu_{H,\beta_0}\propto\pi e^{2\beta_0H} μ H , β 0 ∝ π e 2 β 0 H X ‾ T \overline{X}_T X T μ H , β 0 \mu_{H,\beta_0} μ H , β 0 T → ∞ T\to\infty T → ∞ μ H , β 0 \mu_{H,\beta_0} μ H , β 0 β 0 \beta_0 β 0 E π [ x ] = 0 \mathbb{E}_\pi[x]=0 E π [ x ] = 0
lim T → ∞ X ‾ T = E π [ x e 2 β 0 H ( x ) ] E π [ e 2 β 0 H ( x ) ] = 2 β 0 E π [ x H ( x ) ] + O ( β 0 2 ) = 2 β 0 κ N u H + O ( β 0 2 ) . \lim_{T\to\infty}\overline{X}_T=\frac{\mathbb{E}_\pi\left[x e^{2\beta_0H(x)}\right]}{\mathbb{E}_\pi\left[e^{2\beta_0H(x)}\right]}=2\beta_0\mathbb{E}_\pi[xH(x)]+O(\beta_0^2)=\frac{2\beta_0}{\kappa_N}u_H+O(\beta_0^2). T → ∞ lim X T = E π [ e 2 β 0 H ( x ) ] E π [ x e 2 β 0 H ( x ) ] = 2 β 0 E π [ x H ( x )] + O ( β 0 2 ) = κ N 2 β 0 u H + O ( β 0 2 ) . In other words, the mean of the Gibbs measure at high temperature is the linear response of the uniform measure to the field H H H u H u_H u H
Since β 0 u H \beta_0u_H β 0 u H β 0 → 0 \beta_0\to0 β 0 → 0 § 5 . Notably, our proofs circumvent the analysis of the stationary distribution or the mixing time of high temperature spherical Langevin dynamics, and instead proceed by comparing the diffusion directly with spherical Brownian motion on S N \mathcal{S}_N S N β 0 ∇ S N H ( x ) − K N 2 x \beta_0\nabla_{\mathcal{S}_N} H(x)-\frac{K_N}{2}x β 0 ∇ S N H ( x ) − 2 K N x X ˉ T \bar{X}_T X ˉ T 2 β 0 K N ∇ S N H ( X ~ t ) \frac{2\beta_0}{K_N}\nabla_{\mathcal{S}_N}H(\widetilde{X}_t) K N 2 β 0 ∇ S N H ( X t ) 2 β 0 K N E π [ ∇ S N H ] = 2 β 0 K N u H \frac{2\beta_0}{K_N}\mathbb{E}_{\pi}[\nabla_{\mathcal{S}_N}H]=\frac{2\beta_0}{K_N}u_H K N 2 β 0 E π [ ∇ S N H ] = K N 2 β 0 u H π \pi π [31 Ebe25] , with the Bakry–Émery curvature-dimension criterion [12 ] . For tensor PCA, the resulting bounds establish that β 0 = λ − 1 N − k − 1 2 − o ( 1 ) \beta_0=\lambda^{-1}N^{-\frac{k-1}{2}-o(1)} β 0 = λ − 1 N − 2 k − 1 − o ( 1 ) T a v = N 2 k − 2 + o ( 1 ) T_{\mathrm{av}}=N^{2k-2+o(1)} T av = N 2 k − 2 + o ( 1 ) X ^ T a v ≔ N X ˉ T a v ∥ X ˉ T a v ∥ \widehat{X}_{T_{\mathrm{av}}}\coloneqq\frac{\sqrt{N}\bar{X}_{T_{\mathrm{av}}}}{\lVert\bar{X}_{T_{\mathrm{av}}}\rVert} X T av : = ∥ X ˉ T av ∥ N X ˉ T av u ^ H \widehat{u}_H u H
Our non-asymptotic analysis of the ergodic average is established for generic smooth Hamiltonians satisfying mild regularity conditions, and could be of independent interest. It extends the corresponding analysis for Glauber dynamics in Part I [21 ] , which instead establishes a discrete version of the Bakry–Émery criterion on the hypercube using elementary Fourier analysis.
Computing the warm start and climbing towards the planted signal require different temperatures, which is why Algorithm 1 is annealed. At high temperature, the signal drift in Equation (4) is negligible, so the first phase cannot climb towards the planted signal, even though its ergodic average encodes u H u_H u H X ^ T a v \widehat{X}_{T_{\mathrm{av}}} X T av (4) . However, controlling this term now becomes all the more delicate since this warm start itself depends on the disorder.
Controlling the Disorder via Reverse Planting
To illustrate our ideas, we make two simplifications throughout this subsection. First, we ignore the estimation error of the ergodic average in the first phase, and assume that the second phase of Algorithm 1 is initialized exactly at u ^ H \widehat{u}_H u H x ⋆ x^\star x ⋆ S N \mathcal{S}_N S N § 6 dispense with both these simplifications, and apply to any initialization sufficiently close to u ^ H \widehat{u}_H u H x ⋆ ∈ S N x^\star\in\mathcal{S}_N x ⋆ ∈ S N S N \mathcal{S}_N S N u ^ H \widehat{u}_H u H
By Gaussian concentration, the disorder drift in Equation (4) is of order one uniformly over S N \mathcal{S}_N S N H 0 H_0 H 0 m t m_t m t critical correlation r N = λ − 1 / ( k − 1 ) r_N=\lambda^{-1/(k-1)} r N = λ − 1/ ( k − 1 ) m t m_t m t § 6 , the correlation then converges to one exponentially fast, such that spherical Langevin dynamics achieves strong recovery within O ( 1 ) O(1) O ( 1 ) § 2.2 that the warm start has correlation m 0 ≔ m ( u ^ H ) m_0\coloneqq m(\widehat{u}_H) m 0 : = m ( u H ) min { 1 , λ N − k − 1 4 } \min\{1,\lambda N^{-\frac{k-1}{4}}\} min { 1 , λ N − 4 k − 1 } m 0 ≥ C r N m_0\ge Cr_N m 0 ≥ C r N C C C λ ≳ N − k − 2 4 + 1 4 k \lambda\gtrsim N^{-\frac{k-2}{4}+\frac{1}{4k}} λ ≳ N − 4 k − 2 + 4 k 1 λ A L G ≲ λ ≲ λ A L G N 1 4 k \lambda_{\mathrm{ALG}}\lesssim\lambda\lesssim\lambda_{\mathrm{ALG}}N^{\frac{1}{4k}} λ ALG ≲ λ ≲ λ ALG N 4 k 1 λ m 0 k − 2 ≳ 1 \lambda m_0^{k-2} \gtrsim1 λ m 0 k − 2 ≳ 1 § 2.1 , but is subcritical, i.e., m 0 ≪ r N m_0 \ll r_N m 0 ≪ r N C r N Cr_N C r N O ( 1 ) O(1) O ( 1 )
However, neither worst-case nor typical bounds on the disorder drift suffice to justify this. Below the critical correlation, the signal drift can be much smaller than the worst-case magnitude of the disorder drift, which could thus overwhelm it. Moreover, although the disorder drift is O ( N − 1 / 2 ) O(N^{-1/2}) O ( N − 1/2 ) H 0 H_0 H 0 H 0 H_0 H 0 H 0 H_0 H 0 [1 ] , around which the disorder drift could dominate the signal drift and trap the dynamics at polynomial timescales.
We rule out such adversarial scenarios by showing that the dependence of the warm start on the disorder is benign. Recall from Equation (5) that u H = κ N − 1 λ σ N 2 x ⋆ + ζ u_H = \kappa_N^{-1}\lambda\sigma_N^2x^\star+\zeta u H = κ N − 1 λ σ N 2 x ⋆ + ζ ζ = E π [ ∇ S N H 0 ] \zeta=\mathbb{E}_{\pi}[\nabla_{\mathbb{S}^N}H_0] ζ = E π [ ∇ S N H 0 ] ζ = κ N E π [ x H 0 ( x ) ] \zeta=\kappa_N\mathbb{E}_{\pi}[xH_0(x)] ζ = κ N E π [ x H 0 ( x )] κ N − 1 ⟨ ζ , x ⟩ \kappa_N^{-1}\langle\zeta,x\rangle κ N − 1 ⟨ ζ , x ⟩ H 0 H_0 H 0 H 0 H_0 H 0 H 0 H_0 H 0 ζ \zeta ζ H 0 ( x ) − κ N − 1 ⟨ ζ , x ⟩ H_0(x)-\kappa_N^{-1}\langle\zeta,x\rangle H 0 ( x ) − κ N − 1 ⟨ ζ , x ⟩ H 0 H_0 H 0 planted law when the Hamiltonian is H H H null law when the Hamiltonian is H 0 H_0 H 0 λ = 0 \lambda=0 λ = 0 u ^ H \widehat{u}_H u H N ζ ∥ ζ ∥ \frac{\sqrt{N}\zeta}{\|\zeta\|} ∥ ζ ∥ N ζ (4) with λ = 0 \lambda=0 λ = 0
d m t d t ≃ 1 N ⟨ x ⋆ , β ∇ S N H 0 ( X t ) − N − 1 2 N X t ⟩ . (6) \frac{d m_t}{d t}\simeq\frac{1}{N}\left\langle x^\star,\beta\nabla_{\mathbb{S}^N}H_0(X_t)-\frac{N-1}{2N}X_t\right\rangle. \tag*{(6)} d t d m t ≃ N 1 ⟨ x ⋆ , β ∇ S N H 0 ( X t ) − 2 N N − 1 X t ⟩ . (6) We call the right-hand side of Equation (6) the null drift , since it is precisely the net contribution of the entropic and disorder drifts. Since x ⋆ x^\star x ⋆ x ⋆ x^\star x ⋆ O ( 1 ) O(1) O ( 1 ) x ⋆ / N x^\star/\sqrt{N} x ⋆ / N N − 1 / 2 N^{-1/2} N − 1/2
Thus, to control the disorder drift in the true dynamics, it remains to transfer this estimate to the planted law via a change of measure argument based on Girsanov’s theorem, which we call the reverse planting trick . As the name suggests, it runs opposite to the classical planting trick [2 ] , in which the null law is the true law of interest, and estimates established under the planted law are transferred to it via contiguity. However, executing this argument is quite challenging since the planted and null laws are far from contiguous. Indeed, their Hamiltonians differ by the signal N λ m ( x ) k N\lambda m(x)^k N λm ( x ) k C r N Cr_N C r N adaptive contiguity bound, which controls the cost of transferring an event up to any stopping time in terms of the signal accumulated before it.
We apply this adaptive contiguity bound up to the failure time , i.e., the first time before the correlation reaches C r N Cr_N C r N § 2.1 , and reaches C r N Cr_N C r N O ( 1 ) O(1) O ( 1 ) C r N Cr_N C r N O ( 1 ) O(1) O ( 1 ) Theorem 1.1 .
Strong Recovery for Any Tensor Order
We now turn our attention to Algorithm 2, and first focus on the case when k k k H ( − x ) = H ( x ) H(-x)=H(x) H ( − x ) = H ( x ) u H u_H u H ∇ S N 2 H \nabla^2_{\mathbb{S}_N}H ∇ S N 2 H
∇ S N 2 H ( x ) ≈ k ( k − 1 ) λ N m ( x ) k − 2 x ⋆ ( x ⋆ ) T + ∇ S N 2 H 0 ( x ) , \nabla^2_{\mathbb{S}_N}H(x)\approx\frac{k(k-1)\lambda}{N}m(x)^{k-2}x^\star(x^\star)^\mathsf{T}+\nabla^2_{\mathbb{S}_N}H_0(x), ∇ S N 2 H ( x ) ≈ N k ( k − 1 ) λ m ( x ) k − 2 x ⋆ ( x ⋆ ) T + ∇ S N 2 H 0 ( x ) , and therefore forms a noisy observation of x ⋆ ( x ⋆ ) T x^\star(x^\star)^\mathsf{T} x ⋆ ( x ⋆ ) T m ( x ) k − 2 ≥ 0 m(x)^{k-2}\ge0 m ( x ) k − 2 ≥ 0 k k k § 4 , we show that the average spherical Hessian has a spiked Wigner type structure with a planted spike in the direction of x ⋆ x^\star x ⋆
A H ≔ E π [ ∇ S N 2 H ] = λ γ N N ( x ⋆ ( x ⋆ ) T − I ) + W N , γ N ≍ k N − k − 2 2 , (7) A_H\coloneqq\mathbb{E}_{\pi}\left[\nabla^2_{\mathbb{S}_N}H\right]=\frac{\lambda\gamma_N}{N}\left(x^\star(x^\star)^\mathsf{T}-I\right)+W_N,\qquad\gamma_N\asymp_k N^{-\frac{k-2}{2}}, \tag*{(7)} A H : = E π [ ∇ S N 2 H ] = N λ γ N ( x ⋆ ( x ⋆ ) T − I ) + W N , γ N ≍ k N − 2 k − 2 , (7) where W N ≔ E π [ ∇ S N 2 H 0 ] W_N\coloneqq\mathbb{E}_{\pi}\left[\nabla^2_{\mathbb{S}_N}H_0\right] W N : = E π [ ∇ S N 2 H 0 ] (7) has eigenvalue of order λ γ N \lambda\gamma_N λ γ N ∥ W N ∥ o p ≲ γ N \lVert W_N\rVert_{\mathrm{op}}\lesssim\sqrt{\gamma_N} ∥ W N ∥ op ≲ γ N λ γ N ≍ k λ N − k − 2 4 ≳ 1 \lambda\sqrt{\gamma_N}\asymp_k\lambda N^{-\frac{k-2}{4}}\gtrsim1 λ γ N ≍ k λ N − 4 k − 2 ≳ 1 λ ≳ λ A L G \lambda\gtrsim\lambda_{\mathrm{ALG}} λ ≳ λ ALG [48 Theorem 4.5.5] , the top eigenvector of A H A_H A H S N \mathbb{S}_N S N x ⋆ x^\star x ⋆ 2 r N r_N r N § 2.4 , spherical Langevin dynamics achieves strong recovery within O ( 1 ) O(1) O ( 1 )
As before, despite being a global statistic, A H A_H A H Σ ˉ T ≔ 1 T ∫ 0 T X ~ t X ~ t T d t \bar{\Sigma}_T\coloneqq\frac{1}{T}\int_0^T\widetilde{X}_t\widetilde{X}_t^\mathsf{T}\,dt Σ ˉ T : = T 1 ∫ 0 T X t X t T d t Σ ˉ T \bar{\Sigma}_T Σ ˉ T μ H , β 0 \mu_{H,\beta_0} μ H , β 0 T → ∞ T\to\infty T → ∞ β 0 \beta_0 β 0 § 2.3 , we infer from E π [ x x T ] = I \mathbb{E}_{\pi}[xx^\mathsf{T}]=I E π [ x x T ] = I
lim T → ∞ Σ ˉ T = E π [ x x T e 2 β 0 H ( x ) ] E π [ e 2 β 0 H ( x ) ] = I + 2 β 0 E π [ ( x x T − I ) H ( x ) ] + O ( β 0 2 ) = I + 2 β 0 A H + O ( β 0 2 ) . \lim_{T\to\infty}\bar{\Sigma}_T=\frac{\mathbb{E}_{\pi}\left[xx^\mathsf{T}e^{2\beta_0H(x)}\right]}{\mathbb{E}_{\pi}\left[e^{2\beta_0H(x)}\right]}=I+2\beta_0\mathbb{E}_{\pi}\left[(xx^\mathsf{T}-I)H(x)\right]+O(\beta_0^2)=I+2\beta_0A_H+O(\beta_0^2). T → ∞ lim Σ ˉ T = E π [ e 2 β 0 H ( x ) ] E π [ x x T e 2 β 0 H ( x ) ] = I + 2 β 0 E π [ ( x x T − I ) H ( x ) ] + O ( β 0 2 ) = I + 2 β 0 A H + O ( β 0 2 ) . Since shifting by the identity does not change eigenvectors, the top eigenvector of Σ ˉ T \bar{\Sigma}_T Σ ˉ T A H A_H A H § 5 .
However, by symmetry, A H A_H A H k k k u H u_H u H k k k k k k F ( x ) ≔ ( x , 1 ) ∈ R N + 1 F(x)\coloneqq(x,1)\in\mathbb{R}^{N+1} F ( x ) : = ( x , 1 ) ∈ R N + 1 Σ ˉ T \bar{\Sigma}_T Σ ˉ T X ˉ T \bar{X}_T X ˉ T T T T β 0 \beta_0 β 0
M ˉ T ≔ 1 T ∫ 0 T F ( X ~ t ) ⊗ 2 d t = ( Σ ˉ T X ˉ T X ˉ T T 1 ) ≈ I N + 1 + 2 β 0 A H , A H ≔ ( A H κ N − 1 u H κ N − 1 u H T 0 ) . \bar{M}_T\coloneqq\frac{1}{T}\int_0^T F(\widetilde{X}_t)^{\otimes2}\,dt=\begin{pmatrix}\bar{\Sigma}_T&\bar{X}_T\\\bar{X}_T^\mathsf{T}&1\end{pmatrix}\approx I_{N+1}+2\beta_0\mathcal{A}_H,\qquad\mathcal{A}_H\coloneqq\begin{pmatrix}A_H&\kappa_N^{-1}u_H\\\kappa_N^{-1}u_H^\mathsf{T}&0\end{pmatrix}. M ˉ T : = T 1 ∫ 0 T F ( X t ) ⊗ 2 d t = ( Σ ˉ T X ˉ T T X ˉ T 1 ) ≈ I N + 1 + 2 β 0 A H , A H : = ( A H κ N − 1 u H T κ N − 1 u H 0 ) . When k k k A H = 0 A_H = 0 A H = 0 A H \mathcal{A}_H A H 1 2 ( u H / ∥ u H ∥ , 1 ) \frac{1}{\sqrt{2}}(u_H/\|u_H\|,1) 2 1 ( u H /∥ u H ∥ , 1 ) N N N S N \mathcal{S}_N S N u ^ H \hat{u}_H u ^ H § 2.4 applies. When k k k u H = 0 u_H = 0 u H = 0 A H \mathcal{A}_H A H A H A_H A H k k k k k k Theorem 1.2 .
Notation and Preliminaries
For d ≥ 2 d \ge2 d ≥ 2 M d ( R ) M_d(\mathbb{R}) M d ( R ) d × d d \times d d × d Sym d ⊆ M d ( R ) \operatorname{Sym}_d \subseteq M_d(\mathbb{R}) Sym d ⊆ M d ( R ) M ∈ M d ( R ) M \in M_d(\mathbb{R}) M ∈ M d ( R ) diag ( M ) \operatorname{diag}(M) diag ( M ) M M M a ∈ R d a \in\mathbb{R}^d a ∈ R d diag ( a ) \operatorname{diag}(a) diag ( a ) a 1 , … , a d a_1,\ldots,a_d a 1 , … , a d A ∈ Sym d A \in\operatorname{Sym}_d A ∈ Sym d λ 1 ( A ) ≥ λ 2 ( A ) ≥ ⋯ ≥ λ d ( A ) \lambda_1(A) \ge\lambda_2(A) \ge\cdots\ge\lambda_d(A) λ 1 ( A ) ≥ λ 2 ( A ) ≥ ⋯ ≥ λ d ( A )
For ℓ ≥ 1 \ell\ge1 ℓ ≥ 1 ( R d ) ⊗ ℓ (\mathbb{R}^d)^{\otimes\ell} ( R d ) ⊗ ℓ ℓ \ell ℓ R d \mathbb{R}^d R d ( R d ) ⊙ ℓ (\mathbb{R}^d)^{\odot\ell} ( R d ) ⊙ ℓ
Unless specified otherwise, ∥ ⋅ ∥ \|\cdot\| ∥ ⋅ ∥ S d − 1 \mathcal{S}^{d-1} S d − 1 R d \mathbb{R}^d R d A ∈ M d ( R ) A \in M_d(\mathbb{R}) A ∈ M d ( R ) ∥ A ∥ o p = sup x ∈ S d − 1 ∥ A x ∥ \|A\|_{\mathrm{op}} = \sup_{x \in\mathcal{S}^{d-1}} \|Ax\| ∥ A ∥ op = sup x ∈ S d − 1 ∥ A x ∥ T ∈ ( R d ) ⊗ ℓ T \in(\mathbb{R}^d)^{\otimes\ell} T ∈ ( R d ) ⊗ ℓ ∥ T ∥ o p = sup x 1 , … , x ℓ ∈ S d − 1 ∣ T ( x 1 , … , x ℓ ) ∣ \|T\|_{\mathrm{op}} = \sup_{x_1,\ldots,x_\ell\in\mathcal{S}^{d-1}} |T(x_1,\ldots,x_\ell)| ∥ T ∥ op = sup x 1 , … , x ℓ ∈ S d − 1 ∣ T ( x 1 , … , x ℓ ) ∣ e 1 , … , e N e_1,\ldots,e_N e 1 , … , e N R N \mathbb{R}^N R N
We use c , C > 0 c,C > 0 c , C > 0 c k , C k > 0 c_k,C_k > 0 c k , C k > 0 k k k a a a b b b a ≲ b a \lesssim b a ≲ b a = O ( b ) a = O(b) a = O ( b ) a ≤ C b a \le Cb a ≤ C b a ≳ b a \gtrsim b a ≳ b a = Ω ( b ) a = \Omega(b) a = Ω ( b ) a ≥ c b a \ge cb a ≥ c b a ≍ b a \asymp b a ≍ b a = Θ ( b ) a = \Theta(b) a = Θ ( b ) c b ≤ a ≤ C b cb \le a \le Cb c b ≤ a ≤ C b a ≪ b a \ll b a ≪ b a = o ( b ) a = o(b) a = o ( b ) a / b → 0 a/b \to0 a / b → 0 N → ∞ N \to\infty N → ∞ a ≫ b a \gg b a ≫ b b / a → 0 b/a \to0 b / a → 0 a ≲ k b a \lesssim_k b a ≲ k b a ≤ C k b a \le C_k b a ≤ C k b
Riemannian Geometry of the Sphere
We present a brief introduction to analysis on the sphere, and refer the readers to Lee [36 ] and Boumal [15 ] for a detailed treatment. Let S N = S N − 1 ( N ) = { x ∈ R N : ∥ x ∥ 2 = N } \mathcal{S}_N = \mathcal{S}^{N-1}(\sqrt{N}) = \{x \in\mathbb{R}^N : \|x\|^2 = N\} S N = S N − 1 ( N ) = { x ∈ R N : ∥ x ∥ 2 = N } N \sqrt{N} N π \pi π S N \mathcal{S}_N S N x , y ∈ S N x,y \in\mathcal{S}_N x , y ∈ S N x x x y y y d S N ( x , y ) = N arccos ( ⟨ x , y ⟩ / N ) d_{\mathcal{S}_N}(x,y) = \sqrt{N}\arccos(\langle x,y\rangle/N) d S N ( x , y ) = N arccos (⟨ x , y ⟩ / N ) [15 ] , Example 10.21]. Moreover, ∥ x − y ∥ 2 = 2 ( N − ⟨ x , y ⟩ ) \|x-y\|^2 = 2(N-\langle x,y\rangle) ∥ x − y ∥ 2 = 2 ( N − ⟨ x , y ⟩) d S N ( x , y ) ≤ C ∥ x − y ∥ d_{\mathcal{S}_N}(x,y) \le C\|x-y\| d S N ( x , y ) ≤ C ∥ x − y ∥ x , y ∈ S N x,y \in\mathcal{S}_N x , y ∈ S N C > 0 C > 0 C > 0
We equip S N \mathcal{S}_N S N S N \mathcal{S}_N S N x x x T x S N = { v ∈ R N : ⟨ v , x ⟩ = 0 } T_x\mathcal{S}_N = \{v \in\mathbb{R}^N : \langle v,x\rangle= 0\} T x S N = { v ∈ R N : ⟨ v , x ⟩ = 0 } x ∈ R N ∖ { 0 } x \in\mathbb{R}^N \setminus\{0\} x ∈ R N ∖ { 0 } Q x Q_x Q x κ N \kappa_N κ N
Q x ≔ I − x x T ∥ x ∥ 2 , κ N ≔ N − 1 N . Q_x \coloneqq I - \frac{xx^{\mathsf T}}{\|x\|^2}, \qquad\kappa_N \coloneqq\frac{N-1}{N}. Q x : = I − ∥ x ∥ 2 x x T , κ N : = N N − 1 . In particular, for every x ∈ S N x \in\mathcal{S}_N x ∈ S N Q x = I − x x T N Q_x = I - \frac{xx^{\mathsf T}}{N} Q x = I − N x x T T x S N T_x\mathcal{S}_N T x S N N ≥ 3 N \ge3 N ≥ 3 2 3 ≤ κ N < 1 \frac{2}{3} \le\kappa_N < 1 3 2 ≤ κ N < 1 κ N ≃ 1 \kappa_N \simeq1 κ N ≃ 1
We use ∇ \nabla ∇ ∇ 2 \nabla^2 ∇ 2 Δ \Delta Δ f : S N → R f:\mathbb{S}_N\to\mathbb{R} f : S N → R f f f S N \mathbb{S}_N S N R N \mathbb{R}^N R N ∇ S N f \nabla_{\mathbb{S}_N}f ∇ S N f ∇ S N 2 f \nabla^2_{\mathbb{S}_N}f ∇ S N 2 f Δ S N f \Delta_{\mathbb{S}_N}f Δ S N f
∇ S N f ( x ) = Q x ∇ f ( x ) , \nabla_{\mathbb{S}_N}f(x)=Q_x\nabla f(x), ∇ S N f ( x ) = Q x ∇ f ( x ) , ∇ S N 2 f ( x ) = Q x ∇ 2 f ( x ) Q x − ⟨ ∇ f ( x ) , x ⟩ N Q x , \nabla^2_{\mathbb{S}_N}f(x)=Q_x\nabla^2f(x)Q_x-\frac{\langle\nabla f(x),x\rangle}{N}Q_x, ∇ S N 2 f ( x ) = Q x ∇ 2 f ( x ) Q x − N ⟨ ∇ f ( x ) , x ⟩ Q x , Δ S N f ( x ) = Tr ( ∇ S N 2 f ( x ) ) = Δ f ( x ) − 1 N ⟨ x , ∇ 2 f ( x ) x ⟩ − κ N ⟨ ∇ f ( x ) , x ⟩ . (8) \Delta_{\mathbb{S}_N}f(x)=\operatorname{Tr}\left(\nabla^2_{\mathbb{S}_N}f(x)\right)=\Delta f(x)-\frac{1}{N}\langle x,\nabla^2f(x)x\rangle-\kappa_N\langle\nabla f(x),x\rangle. \tag*{(8)} Δ S N f ( x ) = Tr ( ∇ S N 2 f ( x ) ) = Δ f ( x ) − N 1 ⟨ x , ∇ 2 f ( x ) x ⟩ − κ N ⟨ ∇ f ( x ) , x ⟩ . (8) More generally, for any integer ℓ ≥ 1 \ell\geq1 ℓ ≥ 1 ∇ S N ℓ f ( x ) \nabla^\ell_{\mathbb{S}_N}f(x) ∇ S N ℓ f ( x ) ℓ \ell ℓ f f f x x x ℓ \ell ℓ T x S N T_x\mathbb{S}_N T x S N [36 Chapter 4, 15 Section 10.7] , whose operator norm is given by
∥ ∇ S N ℓ f ( x ) ∥ o p = max v 1 , … , v ℓ ∈ T x S N ∩ S N − 1 ∣ ∇ S N ℓ f ( x ) [ v 1 , … , v ℓ ] ∣ = max u 1 , … , u ℓ ∈ S N − 1 ∣ ∇ S N ℓ f ( x ) [ Q x u 1 , … , Q x u ℓ ] ∣ . \left\|\nabla^\ell_{\mathbb{S}_N}f(x)\right\|_{\mathrm{op}}=\max_{v_1,\ldots,v_\ell\in T_x\mathbb{S}_N\cap\mathbb{S}^{N-1}}\left|\nabla^\ell_{\mathbb{S}_N}f(x)[v_1,\ldots,v_\ell]\right|=\max_{u_1,\ldots,u_\ell\in\mathbb{S}^{N-1}}\left|\nabla^\ell_{\mathbb{S}_N}f(x)[Q_xu_1,\ldots,Q_xu_\ell]\right|. ∇ S N ℓ f ( x ) op = v 1 , … , v ℓ ∈ T x S N ∩ S N − 1 max ∇ S N ℓ f ( x ) [ v 1 , … , v ℓ ] = u 1 , … , u ℓ ∈ S N − 1 max ∇ S N ℓ f ( x ) [ Q x u 1 , … , Q x u ℓ ] . Accordingly, we identify ∇ S N ℓ f ( x ) \nabla^\ell_{\mathbb{S}_N}f(x) ∇ S N ℓ f ( x ) ( u 1 , … , u ℓ ) ↦ ∇ S N ℓ f ( x ) [ Q x u 1 , … , Q x u ℓ ] (u_1,\ldots,u_\ell)\mapsto\nabla^\ell_{\mathbb{S}_N}f(x)[Q_xu_1,\ldots,Q_xu_\ell] ( u 1 , … , u ℓ ) ↦ ∇ S N ℓ f ( x ) [ Q x u 1 , … , Q x u ℓ ] ( R N ) ⊗ ℓ (\mathbb{R}^N)^{\otimes\ell} ( R N ) ⊗ ℓ ℓ = 1 , 2 \ell=1,2 ℓ = 1 , 2 (8) , respectively.
We shall also use the product rules
∇ S N ( f g ) = f ∇ S N g + g ∇ S N f , \nabla_{\mathbb{S}_N}(fg)=f\nabla_{\mathbb{S}_N}g+g\nabla_{\mathbb{S}_N}f, ∇ S N ( f g ) = f ∇ S N g + g ∇ S N f , Δ S N ( f g ) = f Δ S N g + g Δ S N f + 2 ⟨ ∇ S N f , ∇ S N g ⟩ . \Delta_{\mathbb{S}_N}(fg)=f\Delta_{\mathbb{S}_N}g+g\Delta_{\mathbb{S}_N}f+2\langle\nabla_{\mathbb{S}_N}f,\nabla_{\mathbb{S}_N}g\rangle. Δ S N ( f g ) = f Δ S N g + g Δ S N f + 2 ⟨ ∇ S N f , ∇ S N g ⟩ . For any smooth F : S N → R m F:\mathbb{S}_N\to\mathbb{R}^m F : S N → R m ∇ S N F ( x ) ∈ R N × m \nabla_{\mathbb{S}_N}F(x)\in\mathbb{R}^{N\times m} ∇ S N F ( x ) ∈ R N × m j j j ∇ S N F j ( x ) \nabla_{\mathbb{S}_N}F_j(x) ∇ S N F j ( x ) j ∈ [ m ] j\in[m] j ∈ [ m ] ∇ S N F ( x ) = Q x ∇ F ( x ) \nabla_{\mathbb{S}_N}F(x)=Q_x\nabla F(x) ∇ S N F ( x ) = Q x ∇ F ( x ) ∇ F ( x ) ∈ R N × m \nabla F(x)\in\mathbb{R}^{N\times m} ∇ F ( x ) ∈ R N × m F F F ( i , j ) (i,j) ( i , j ) ∂ i F j ( x ) \partial_iF_j(x) ∂ i F j ( x )
We recall that d d t F ( γ ( t ) ) = ∇ S N F ( γ ( t ) ) ⊤ γ ˙ ( t ) \frac{\mathrm{d}}{\mathrm{d}t}F(\gamma(t))=\nabla_{\mathbb{S}_N}F(\gamma(t))^\top\dot{\gamma}(t) d t d F ( γ ( t )) = ∇ S N F ( γ ( t ) ) ⊤ γ ˙ ( t ) F : S N → R m F:\mathbb{S}_N\to\mathbb{R}^m F : S N → R m γ \gamma γ S N \mathbb{S}_N S N x x x y y y d S N ( x , y ) ≲ ∥ x − y ∥ d_{\mathbb{S}_N}(x,y)\lesssim\|x-y\| d S N ( x , y ) ≲ ∥ x − y ∥
∥ F ( x ) − F ( y ) ∥ ≤ [ sup z ∈ S N ∥ ∇ S N F ( z ) ∥ o p ] d S N ( x , y ) ≲ [ sup z ∈ S N ∥ ∇ S N F ( z ) ∥ o p ] ∥ x − y ∥ ∀ x , y ∈ S N . (9) \|F(x)-F(y)\|\leq\left[\sup_{z\in\mathbb{S}_N}\|\nabla_{\mathbb{S}_N}F(z)\|_{\mathrm{op}}\right]d_{\mathbb{S}_N}(x,y)\lesssim\left[\sup_{z\in\mathbb{S}_N}\|\nabla_{\mathbb{S}_N}F(z)\|_{\mathrm{op}}\right]\|x-y\|\qquad\forall x,y\in\mathbb{S}_N. \tag*{(9)} ∥ F ( x ) − F ( y ) ∥ ≤ [ z ∈ S N sup ∥ ∇ S N F ( z ) ∥ op ] d S N ( x , y ) ≲ [ z ∈ S N sup ∥ ∇ S N F ( z ) ∥ op ] ∥ x − y ∥ ∀ x , y ∈ S N . (9) We use the spherical integration by parts identity [12 Section 2.2] , which holds for any smooth f , g : S N → R f,g:\mathbb{S}_N\to\mathbb{R} f , g : S N → R
E π [ ⟨ ∇ S N f , ∇ S N g ⟩ ] = − E π [ f Δ S N g ] . (10) \mathbb{E}_{\pi}\left[\left\langle\nabla_{\mathbb{S}_N}f,\nabla_{\mathbb{S}_N}g\right\rangle\right]=-\mathbb{E}_{\pi}\left[f\Delta_{\mathbb{S}_N}g\right]. \tag*{(10)} E π [ ⟨ ∇ S N f , ∇ S N g ⟩ ] = − E π [ f Δ S N g ] . (10) The following lemma is a spherical analogue of the classical Stein’s lemma for Gaussian random variables [18 ] and will be used often in our proofs.
Lemma 3.1 (Stein’s Lemma for the sphere). For any smooth f : S N → R f:\mathbb{S}_N\to\mathbb{R} f : S N → R
E π [ ∇ S N f ( x ) ] = κ N E π [ x f ( x ) ] , \mathbb{E}_{\pi}[\nabla_{\mathbb{S}_N}f(x)]=\kappa_N\mathbb{E}_{\pi}[xf(x)], E π [ ∇ S N f ( x )] = κ N E π [ x f ( x )] , E π [ ∇ S N 2 f ( x ) ] = 1 2 E π [ x ∇ S N f ( x ) ⊤ + ∇ S N f ( x ) x ⊤ ] = E π [ ( x x ⊤ − I ) f ( x ) ] . \mathbb{E}_{\pi}[\nabla^2_{\mathbb{S}_N}f(x)]=\frac{1}{2}\mathbb{E}_{\pi}\left[x\nabla_{\mathbb{S}_N}f(x)^\top+\nabla_{\mathbb{S}_N}f(x)x^\top\right]=\mathbb{E}_{\pi}\left[(xx^\top-I)f(x)\right]. E π [ ∇ S N 2 f ( x )] = 2 1 E π [ x ∇ S N f ( x ) ⊤ + ∇ S N f ( x ) x ⊤ ] = E π [ ( x x ⊤ − I ) f ( x ) ] . Proof. Applying Equation (8) to the coordinate functions, we obtain ∇ S N x = Q x \nabla_{\mathcal{S}_N}x=Q_x ∇ S N x = Q x Δ S N x = − κ N x \Delta_{\mathcal{S}_N}x=-\kappa_Nx Δ S N x = − κ N x Δ S N \Delta_{\mathcal{S}_N} Δ S N i ∈ [ N ] i\in[N] i ∈ [ N ] ∇ S N f ( x ) ∈ T x S N \nabla_{\mathcal{S}_N}f(x)\in T_x\mathcal{S}_N ∇ S N f ( x ) ∈ T x S N ∇ S N x i = Q x e i \nabla_{\mathcal{S}_N}x_i=Q_xe_i ∇ S N x i = Q x e i ( ∇ S N f ) i = ⟨ ∇ S N f , ∇ S N x i ⟩ (\nabla_{\mathcal{S}_N}f)_i=\langle\nabla_{\mathcal{S}_N}f,\nabla_{\mathcal{S}_N}x_i\rangle ( ∇ S N f ) i = ⟨ ∇ S N f , ∇ S N x i ⟩
Applying Equation (10) to f f f x i x_i x i Δ S N x i = − κ N x i \Delta_{\mathcal{S}_N}x_i=-\kappa_Nx_i Δ S N x i = − κ N x i
E π [ ( ∇ S N f ) i ] = E π [ ⟨ ∇ S N f , ∇ S N x i ⟩ ] = − E π [ f Δ S N x i ] = κ N E π [ x i f ( x ) ] . \mathbb{E}_{\pi}[(\nabla_{\mathcal{S}_N}f)_i]=\mathbb{E}_{\pi}[\langle\nabla_{\mathcal{S}_N}f,\nabla_{\mathcal{S}_N}x_i\rangle]=-\mathbb{E}_{\pi}[f\Delta_{\mathcal{S}_N}x_i]=\kappa_N\mathbb{E}_{\pi}[x_if(x)]. E π [( ∇ S N f ) i ] = E π [⟨ ∇ S N f , ∇ S N x i ⟩] = − E π [ f Δ S N x i ] = κ N E π [ x i f ( x )] . which proves the first identity coordinatewise.
Now, consider any smooth F : S N → R N F:\mathcal{S}_N\to\mathbb{R}^N F : S N → R N F F F
E π [ ∇ S N F ] = κ N E π [ x F T ] . (11) \mathbb{E}_{\pi}[\nabla_{\mathcal{S}_N}F]=\kappa_N\mathbb{E}_{\pi}[xF^{\mathsf{T}}]. \tag*{(11)} E π [ ∇ S N F ] = κ N E π [ x F T ] . (11) Applying Equation (11) to F ( x ) = x f ( x ) F(x)=xf(x) F ( x ) = x f ( x ) ∇ S N x = Q x \nabla_{\mathcal{S}_N}x=Q_x ∇ S N x = Q x
κ N E π [ x x T f ( x ) ] = E π [ ∇ S N f x T + f Q x ] . \kappa_N\mathbb{E}_{\pi}[xx^{\mathsf{T}}f(x)]=\mathbb{E}_{\pi}[\nabla_{\mathcal{S}_N}f\,x^{\mathsf{T}}+fQ_x]. κ N E π [ x x T f ( x )] = E π [ ∇ S N f x T + f Q x ] . Rearranging terms and using Q x = I − x x T N Q_x=I-\frac{xx^{\mathsf{T}}}{N} Q x = I − N x x T κ N = 1 − 1 N \kappa_N=1-\frac{1}{N} κ N = 1 − N 1
E π [ ∇ S N f x T ] = κ N E π [ x x T f ( x ) ] − E π [ ( I − x x T N ) f ( x ) ] = E π [ ( x x T − I ) f ( x ) ] . \mathbb{E}_{\pi}[\nabla_{\mathcal{S}_N}f\,x^{\mathsf{T}}]=\kappa_N\mathbb{E}_{\pi}[xx^{\mathsf{T}}f(x)]-\mathbb{E}_{\pi}\left[\left(I-\frac{xx^{\mathsf{T}}}{N}\right)f(x)\right]=\mathbb{E}_{\pi}[(xx^{\mathsf{T}}-I)f(x)]. E π [ ∇ S N f x T ] = κ N E π [ x x T f ( x )] − E π [ ( I − N x x T ) f ( x ) ] = E π [( x x T − I ) f ( x )] . Since the right-hand side is symmetric, we conclude that
E π [ ( x x T − I ) f ( x ) ] = 1 2 E π [ x ∇ S N f ( x ) T + ∇ S N f ( x ) x T ] . (12) \mathbb{E}_{\pi}[(xx^{\mathsf{T}}-I)f(x)]=\frac{1}{2}\mathbb{E}_{\pi}[x\nabla_{\mathcal{S}_N}f(x)^{\mathsf{T}}+\nabla_{\mathcal{S}_N}f(x)x^{\mathsf{T}}]. \tag*{(12)} E π [( x x T − I ) f ( x )] = 2 1 E π [ x ∇ S N f ( x ) T + ∇ S N f ( x ) x T ] . (12) Now, let F = ∇ S N f F=\nabla_{\mathcal{S}_N}f F = ∇ S N f α ( x ) = 1 N ⟨ x , ∇ f ( x ) ⟩ \alpha(x)=\frac{1}{N}\langle x,\nabla f(x)\rangle α ( x ) = N 1 ⟨ x , ∇ f ( x )⟩ ∇ f − α x \nabla f-\alpha x ∇ f − α x F F F
∇ S N F = Q x [ ∇ 2 f − ( ∇ α ) x T − α I ] = Q x [ ∇ 2 f − 1 N ( ∇ f + ( ∇ 2 f ) x ) x T − α I ] = Q x ∇ 2 f Q x − 1 N ⟨ x , ∇ f ⟩ Q x − 1 N ∇ S N f x T = ∇ S N 2 f − 1 N ∇ S N f x T . (13) \begin{aligned}
\nabla_{\mathcal{S}_N}F=Q_x[\nabla^2f-(\nabla\alpha)x^{\mathsf{T}}-\alpha I]=Q_x\left[\nabla^2f-\frac{1}{N}(\nabla f+(\nabla^2f)x)x^{\mathsf{T}}-\alpha I\right] \\
&=Q_x\nabla^2fQ_x-\frac{1}{N}\langle x,\nabla f\rangle Q_x-\frac{1}{N}\nabla_{\mathcal{S}_N}f\,x^{\mathsf{T}}=\nabla_{\mathcal{S}_N}^2f-\frac{1}{N}\nabla_{\mathcal{S}_N}f\,x^{\mathsf{T}}. \tag*{(13)}
\end{aligned} ∇ S N F = Q x [ ∇ 2 f − ( ∇ α ) x T − α I ] = Q x [ ∇ 2 f − N 1 ( ∇ f + ( ∇ 2 f ) x ) x T − α I ] = Q x ∇ 2 f Q x − N 1 ⟨ x , ∇ f ⟩ Q x − N 1 ∇ S N f x T = ∇ S N 2 f − N 1 ∇ S N f x T . (13) Applying Equation (11) to F = ∇ S N f F=\nabla_{\mathcal{S}_N}f F = ∇ S N f (13) , we obtain
κ N E π [ x ∇ S N f T ] = E π [ ∇ S N 2 f ] − 1 N E π [ ∇ S N f x T ] . \kappa_N\mathbb{E}_{\pi}[x\nabla_{\mathcal{S}_N}f^{\mathsf{T}}]=\mathbb{E}_{\pi}[\nabla_{\mathcal{S}_N}^2f]-\frac{1}{N}\mathbb{E}_{\pi}[\nabla_{\mathcal{S}_N}f\,x^{\mathsf{T}}]. κ N E π [ x ∇ S N f T ] = E π [ ∇ S N 2 f ] − N 1 E π [ ∇ S N f x T ] . Averaging the above identity with its transpose using the symmetry of ∇ S N 2 f \nabla_{\mathcal{S}_N}^2f ∇ S N 2 f
E π [ ∇ S N 2 f ] = 1 2 ( κ N + 1 N ) E π [ ∇ S N f x T + x ∇ S N f T ] = 1 2 E π [ ∇ S N f x T + x ∇ S N f T ] . \mathbb{E}_{\pi}[\nabla_{\mathcal{S}_N}^2f]=\frac{1}{2}\left(\kappa_N+\frac{1}{N}\right)\mathbb{E}_{\pi}[\nabla_{\mathcal{S}_N}f\,x^{\mathsf{T}}+x\nabla_{\mathcal{S}_N}f^{\mathsf{T}}]=\frac{1}{2}\mathbb{E}_{\pi}[\nabla_{\mathcal{S}_N}f\,x^{\mathsf{T}}+x\nabla_{\mathcal{S}_N}f^{\mathsf{T}}]. E π [ ∇ S N 2 f ] = 2 1 ( κ N + N 1 ) E π [ ∇ S N f x T + x ∇ S N f T ] = 2 1 E π [ ∇ S N f x T + x ∇ S N f T ] . Together with Equation (12) , this proves the second identity.
We use O ( N , R ) O(N,\mathbb{R}) O ( N , R ) R ∈ O ( N , R ) R\in O(N,\mathbb{R}) R ∈ O ( N , R ) S N \mathcal{S}_N S N d S N d_{\mathcal{S}_N} d S N O ( N , R ) O(N,\mathbb{R}) O ( N , R ) ( S N , d S N ) (\mathcal{S}_N,d_{\mathcal{S}_N}) ( S N , d S N ) π \pi π O ( N , R ) O(N,\mathbb{R}) O ( N , R ) π \pi π x ↦ − x x\mapsto-x x ↦ − x ⟨ w , x ⟩ \langle w,x\rangle ⟨ w , x ⟩ π \pi π w ∈ S N − 1 w\in\mathcal{S}^{N-1} w ∈ S N − 1 O ( N , R ) O(N,\mathbb{R}) O ( N , R )
Lemma 3.2 (Commutants of O ( N , R ) O(N,\mathbb{R}) O ( N , R ) Any matrix that commutes with every element of O ( N , R ) O(N,\mathbb{R}) O ( N , R )
Proof. Let A ∈ R N × N A \in\mathbb{R}^{N \times N} A ∈ R N × N A M = M A AM = MA A M = M A M ∈ O ( N , R ) M \in O(N,\mathbb{R}) M ∈ O ( N , R ) i , j ∈ [ N ] i,j \in[N] i , j ∈ [ N ] D D D D i i = + 1 D_{ii}=+1 D ii = + 1 D j j = − 1 D_{jj}=-1 D j j = − 1 D ∈ O ( N , R ) D \in O(N,\mathbb{R}) D ∈ O ( N , R ) A D = D A AD=DA A D = D A
( A D ) i j = A i j D j j = − A i j , ( D A ) i j = D i i A i j = A i j . (AD)_{ij}=A_{ij}D_{jj}=-A_{ij}, \qquad(DA)_{ij}=D_{ii}A_{ij}=A_{ij}. ( A D ) ij = A ij D j j = − A ij , ( D A ) ij = D ii A ij = A ij . It follows that A i j = 0 A_{ij}=0 A ij = 0 i ≠ j i\ne j i = j A = diag ( a ) A=\operatorname{diag}(a) A = diag ( a ) a ∈ R N a\in\mathbb{R}^{N} a ∈ R N P P P i i i j j j P ∈ O ( N , R ) P\in O(N,\mathbb{R}) P ∈ O ( N , R ) A = P A P T A=PAP^{\mathsf{T}} A = P A P T a i = a j a_i=a_j a i = a j a 1 = ⋯ = a N a_1=\cdots=a_N a 1 = ⋯ = a N A = c I A=cI A = c I c ∈ R c\in\mathbb{R} c ∈ R
We shall also make use of the following result on the moments and symmetry of π \pi π
Lemma 3.3 (Moments and symmetry of the uniform measure on S N \mathcal{S}_N S N N ≥ 2 N\ge2 N ≥ 2 x ∼ π x\sim\pi x ∼ π w ∈ S N − 1 w\in\mathcal{S}^{N-1} w ∈ S N − 1 ℓ ≥ 1 \ell\ge1 ℓ ≥ 1
E π [ ( ⟨ w , x ⟩ N ) ℓ ] = { 0 , if ℓ is odd , ( ℓ − 1 ) ! ! N ( N + 2 ) ⋯ ( N + ℓ − 2 ) , if ℓ is even . \mathbb{E}_{\pi}\left[\left(\frac{\langle w,x\rangle}{\sqrt{N}}\right)^\ell\right]=\begin{cases}0, & \text{if }\ell\text{ is odd},\\\frac{(\ell-1)!!}{N(N+2)\cdots(N+\ell-2)}, & \text{if }\ell\text{ is even}.\end{cases} E π [ ( N ⟨ w , x ⟩ ) ℓ ] = { 0 , N ( N + 2 ) ⋯ ( N + ℓ − 2 ) ( ℓ − 1 )!! , if ℓ is odd , if ℓ is even . In particular, for fixed even ℓ \ell ℓ ( ℓ − 1 ) ! ! N − ℓ / 2 ( 1 + O ℓ ( N − 1 ) ) (\ell-1)!!\,N^{-\ell/2}(1+O_\ell(N^{-1})) ( ℓ − 1 )!! N − ℓ /2 ( 1 + O ℓ ( N − 1 )) z ∈ S N − 1 z\in\mathcal{S}^{N-1} z ∈ S N − 1 w w w g : R → R g:\mathbb{R}\to\mathbb{R} g : R → R h : R → R h:\mathbb{R}\to\mathbb{R} h : R → R
E π [ g ( ⟨ w , x ⟩ N ) h ( ⟨ z , x ⟩ N ) ] = 0. \mathbb{E}_{\pi}\left[g\left(\frac{\langle w,x\rangle}{\sqrt{N}}\right)h\left(\frac{\langle z,x\rangle}{\sqrt{N}}\right)\right]=0. E π [ g ( N ⟨ w , x ⟩ ) h ( N ⟨ z , x ⟩ ) ] = 0. Finally, for every integer ℓ ≥ 0 \ell\ge0 ℓ ≥ 0
E π [ x x T ( ⟨ w , x ⟩ N ) ℓ ] = N N + ℓ E π [ ( ⟨ w , x ⟩ N ) ℓ ] ( I + ℓ w w T ) . \mathbb{E}_{\pi}\left[xx^{\mathsf{T}}\left(\frac{\langle w,x\rangle}{\sqrt{N}}\right)^\ell\right]
=
\frac{N}{N+\ell}\mathbb{E}_{\pi}\left[\left(\frac{\langle w,x\rangle}{\sqrt{N}}\right)^\ell\right](I+\ell ww^{\mathsf{T}}). E π [ x x T ( N ⟨ w , x ⟩ ) ℓ ] = N + ℓ N E π [ ( N ⟨ w , x ⟩ ) ℓ ] ( I + ℓ w w T ) . Proof. The odd moments vanish by the symmetry x = d − x x \mathrel{\overset{\mathrm{d}}{=}}-x x = d − x ℓ ≥ 2 \ell\ge2 ℓ ≥ 2 Lemma 3.1 to f ( x ) = ⟨ w , x ⟩ ℓ − 1 f(x)=\langle w,x\rangle^{\ell-1} f ( x ) = ⟨ w , x ⟩ ℓ − 1 w w w
κ N E π [ ⟨ w , x ⟩ ℓ ] = ( ℓ − 1 ) E π [ ⟨ w , x ⟩ ℓ − 2 ⟨ w , Q x w ⟩ ] = ( ℓ − 1 ) E π [ ⟨ w , x ⟩ ℓ − 2 ( 1 − ⟨ w , x ⟩ 2 N ) ] . \kappa_N\mathbb{E}_{\pi}\left[\langle w,x\rangle^\ell\right]
=(\ell-1)\mathbb{E}_{\pi}\left[\langle w,x\rangle^{\ell-2}\langle w,Q_xw\rangle\right]
=(\ell-1)\mathbb{E}_{\pi}\left[\langle w,x\rangle^{\ell-2}\left(1-\frac{\langle w,x\rangle^2}{N}\right)\right]. κ N E π [ ⟨ w , x ⟩ ℓ ] = ( ℓ − 1 ) E π [ ⟨ w , x ⟩ ℓ − 2 ⟨ w , Q x w ⟩ ] = ( ℓ − 1 ) E π [ ⟨ w , x ⟩ ℓ − 2 ( 1 − N ⟨ w , x ⟩ 2 ) ] . Rearranging and using N κ N = N − 1 N\kappa_N=N-1 N κ N = N − 1
E π [ ( ⟨ w , x ⟩ N ) ℓ ] = ℓ − 1 N + ℓ − 2 E π [ ( ⟨ w , x ⟩ N ) ℓ − 2 ] . (14) \mathbb{E}_{\pi}\left[\left(\frac{\langle w,x\rangle}{\sqrt{N}}\right)^\ell\right]
=
\frac{\ell-1}{N+\ell-2}\mathbb{E}_{\pi}\left[\left(\frac{\langle w,x\rangle}{\sqrt{N}}\right)^{\ell-2}\right]. \tag*{(14)} E π [ ( N ⟨ w , x ⟩ ) ℓ ] = N + ℓ − 2 ℓ − 1 E π [ ( N ⟨ w , x ⟩ ) ℓ − 2 ] . (14) Iterating from the zeroth moment proves the first claim.
For the second claim, observe that π \pi π I − 2 w w T I-2ww^{\mathsf{T}} I − 2 w w T ⟨ w , x ⟩ \langle w,x\rangle ⟨ w , x ⟩ − ⟨ w , x ⟩ -\langle w,x\rangle − ⟨ w , x ⟩ ⟨ z , x ⟩ \langle z,x\rangle ⟨ z , x ⟩ g g g s = ⟨ w , x ⟩ / N s=\langle w,x\rangle/\sqrt{N} s = ⟨ w , x ⟩ / N F ( x ) = s ℓ x F(x)=s^{\ell}x F ( x ) = s ℓ x ∇ S N − 1 s = Q x w / N \nabla_{\mathbb{S}^{N-1}}s=Q_xw/\sqrt{N} ∇ S N − 1 s = Q x w / N Q x w = w − s N x Q_xw=w-\frac{s}{\sqrt{N}}x Q x w = w − N s x ∇ S N − 1 x = Q x \nabla_{\mathbb{S}^{N-1}}x=Q_x ∇ S N − 1 x = Q x
∇ S N − 1 F = ℓ N s ℓ − 1 ( Q x w ) x T + s ℓ Q x = ℓ N s ℓ − 1 w x T + s ℓ I − ℓ + 1 N s ℓ x x T . (15) \nabla_{\mathbb{S}^{N-1}}F=\frac{\ell}{\sqrt{N}}s^{\ell-1}(Q_xw)x^{\mathsf T}+s^\ell Q_x=\frac{\ell}{\sqrt{N}}s^{\ell-1}wx^{\mathsf T}+s^\ell I-\frac{\ell+1}{N}s^\ell xx^{\mathsf T}. \tag*{(15)} ∇ S N − 1 F = N ℓ s ℓ − 1 ( Q x w ) x T + s ℓ Q x = N ℓ s ℓ − 1 w x T + s ℓ I − N ℓ + 1 s ℓ x x T . (15) where the terms with the factor ℓ \ell ℓ ℓ = 0 \ell=0 ℓ = 0 ℓ ≥ 1 \ell\ge1 ℓ ≥ 1 w w w z z z E π [ s ℓ − 1 ⟨ z , x ⟩ ] = 0 \mathbb{E}_{\pi}[s^{\ell-1}\langle z,x\rangle]=0 E π [ s ℓ − 1 ⟨ z , x ⟩] = 0 z ∈ S N − 1 z\in\mathbb{S}^{N-1} z ∈ S N − 1 w w w
E π [ s ℓ − 1 x ] = E π [ s ℓ − 1 ⟨ w , x ⟩ ] w = N E π [ s ℓ ] w . (16) \mathbb{E}_{\pi}[s^{\ell-1}x]=\mathbb{E}_{\pi}[s^{\ell-1}\langle w,x\rangle]w=\sqrt{N}\mathbb{E}_{\pi}[s^\ell]w. \tag*{(16)} E π [ s ℓ − 1 x ] = E π [ s ℓ − 1 ⟨ w , x ⟩] w = N E π [ s ℓ ] w . (16) Applying Stein’s lemma (Lemma 3.1 ) to each coordinate of F F F
E π [ ∇ S N − 1 F ] = κ N E π [ x F T ] = κ N E π [ s ℓ x x T ] . (17) \mathbb{E}_{\pi}[\nabla_{\mathbb{S}^{N-1}}F]=\kappa_N\mathbb{E}_{\pi}[xF^{\mathsf T}]=\kappa_N\mathbb{E}_{\pi}[s^\ell xx^{\mathsf T}]. \tag*{(17)} E π [ ∇ S N − 1 F ] = κ N E π [ x F T ] = κ N E π [ s ℓ x x T ] . (17) Taking expectations in Equation (15) and using Equations (16) and (17) , we conclude that
κ N E π [ s ℓ x x T ] = E π [ s ℓ ] ( I + ℓ w w T ) − ℓ + 1 N E π [ s ℓ x x T ] . \kappa_N\mathbb{E}_{\pi}[s^\ell xx^{\mathsf T}]=\mathbb{E}_{\pi}[s^\ell](I+\ell ww^{\mathsf T})-\frac{\ell+1}{N}\mathbb{E}_{\pi}[s^\ell xx^{\mathsf T}]. κ N E π [ s ℓ x x T ] = E π [ s ℓ ] ( I + ℓ w w T ) − N ℓ + 1 E π [ s ℓ x x T ] . Since κ N + ℓ + 1 N = N + ℓ N \kappa_N+\frac{\ell+1}{N}=\frac{N+\ell}{N} κ N + N ℓ + 1 = N N + ℓ
Lastly, we shall make use of the following elementary estimate for the normalization of nonzero vectors.
Lemma 3.4 (Local Lipschitzness of normalization). For any nonzero a , b ∈ R N a,b\in\mathbb{R}^{N} a , b ∈ R N
∥ a ∥ a ∥ − b ∥ b ∥ ∥ ≤ 2 ∥ a − b ∥ max { ∥ a ∥ , ∥ b ∥ } . \left\|\frac{a}{\|a\|}-\frac{b}{\|b\|}\right\|\le\frac{2\|a-b\|}{\max\{\|a\|,\|b\|\}}. ∥ a ∥ a − ∥ b ∥ b ≤ max { ∥ a ∥ , ∥ b ∥ } 2∥ a − b ∥ . Proof. We write
a ∥ a ∥ − b ∥ b ∥ = a − b ∥ a ∥ + b ( 1 ∥ a ∥ − 1 ∥ b ∥ ) . \frac{a}{\|a\|}-\frac{b}{\|b\|}=\frac{a-b}{\|a\|}+b\left(\frac{1}{\|a\|}-\frac{1}{\|b\|}\right). ∥ a ∥ a − ∥ b ∥ b = ∥ a ∥ a − b + b ( ∥ a ∥ 1 − ∥ b ∥ 1 ) . Taking norms and using ∣ ∥ a ∥ − ∥ b ∥ ∣ ≤ ∥ a − b ∥ |\|a\|-\|b\||\le\|a-b\| ∣∥ a ∥ − ∥ b ∥∣ ≤ ∥ a − b ∥
∥ a ∥ a ∥ − b ∥ b ∥ ∥ ≤ ∥ a − b ∥ ∥ a ∥ + ∣ ∥ b ∥ − ∥ a ∥ ∣ ∥ a ∥ ≤ 2 ∥ a − b ∥ ∥ a ∥ . \left\|\frac{a}{\|a\|}-\frac{b}{\|b\|}\right\|\le\frac{\|a-b\|}{\|a\|}+\frac{\left|\|b\|-\|a\|\right|}{\|a\|}\le\frac{2\|a-b\|}{\|a\|}. ∥ a ∥ a − ∥ b ∥ b ≤ ∥ a ∥ ∥ a − b ∥ + ∥ a ∥ ∣ ∥ b ∥ − ∥ a ∥ ∣ ≤ ∥ a ∥ 2∥ a − b ∥ . Interchanging a a a b b b
Sobolev Norms for Spherical Spin Glass Hamiltonians
We now recall the Sobolev G \mathcal{G} G S N \mathbb{S}_N S N [11 , 10 ] .
Definition 3.5 (Sobolev G \mathcal{G} G n ≥ 0 n\ge0 n ≥ 0 f : S N → R f:\mathbb{S}_N\to\mathbb{R} f : S N → R G n \mathcal{G}^n G n [10 Definition 3.1] :
∥ f ∥ G n = ∑ ℓ = 0 n N ℓ / 2 max x ∈ S N ∥ ∇ S N ℓ f ( x ) ∥ o p , \|f\|_{\mathcal{G}^n}=\sum_{\ell=0}^{n}N^{\ell/2}\max_{x\in\mathbb{S}_N}\|\nabla_{\mathbb{S}_N}^{\ell}f(x)\|_{\mathrm{op}}, ∥ f ∥ G n = ℓ = 0 ∑ n N ℓ /2 x ∈ S N max ∥ ∇ S N ℓ f ( x ) ∥ op , where ∥ ∇ S N 0 f ( x ) ∥ o p = ∣ f ( x ) ∣ \|\nabla_{\mathbb{S}_N}^{0}f(x)\|_{\mathrm{op}}=|f(x)| ∥ ∇ S N 0 f ( x ) ∥ op = ∣ f ( x ) ∣
The following ladder relations [cf. BGJ20a, Lemma 3.5] are used to control the Sobolev norms of the action of differential operators.
Lemma 3.6 (Ladder relations). For every integer n ≥ 0 n \ge0 n ≥ 0 C n > 0 C_n > 0 C n > 0 N N N f , g : S N → R f,g:\mathbb{S}_N \to\mathbb{R} f , g : S N → R
∥ Δ S N f ∥ G n ≤ ∥ f ∥ G n + 2 , ∥ ⟨ ∇ S N g , ∇ S N f ⟩ ∥ G n ≤ C n N ∥ g ∥ G n + 1 ∥ f ∥ G n + 1 . \left\|\Delta_{\mathbb{S}_N}f\right\|_{G_n} \le\|f\|_{G_{n+2}}, \qquad\left\|\left\langle\nabla_{\mathbb{S}_N}g,\nabla_{\mathbb{S}_N}f\right\rangle\right\|_{G_n} \le\frac{C_n}{N}\|g\|_{G_{n+1}}\|f\|_{G_{n+1}}. ∥ Δ S N f ∥ G n ≤ ∥ f ∥ G n + 2 , ∥ ⟨ ∇ S N g , ∇ S N f ⟩ ∥ G n ≤ N C n ∥ g ∥ G n + 1 ∥ f ∥ G n + 1 . Moreover, fix integers n ≥ 0 n \ge0 n ≥ 0 ℓ ≥ 1 \ell\ge1 ℓ ≥ 1 c 1 , c 2 > 0 c_1,c_2 > 0 c 1 , c 2 > 0 g 1 , g 2 : S N → R g_1,g_2:\mathbb{S}_N \to\mathbb{R} g 1 , g 2 : S N → R ∥ g 1 ∥ G n + 2 ℓ ≤ c 1 N \|g_1\|_{G_{n+2\ell}} \le c_1N ∥ g 1 ∥ G n + 2 ℓ ≤ c 1 N ∥ g 2 ∥ G 1 ≤ c 2 N \|g_2\|_{G_1} \le c_2N ∥ g 2 ∥ G 1 ≤ c 2 N L = 1 2 Δ S N + ⟨ ∇ S N g 1 , ∇ S N ⋅ ⟩ L = \frac{1}{2}\Delta_{\mathbb{S}_N} + \langle\nabla_{\mathbb{S}_N}g_1,\nabla_{\mathbb{S}_N}\cdot\rangle L = 2 1 Δ S N + ⟨ ∇ S N g 1 , ∇ S N ⋅ ⟩ A = ⟨ ∇ S N g 2 , ∇ S N ⋅ ⟩ A = \langle\nabla_{\mathbb{S}_N}g_2,\nabla_{\mathbb{S}_N}\cdot\rangle A = ⟨ ∇ S N g 2 , ∇ S N ⋅ ⟩ C > 0 C > 0 C > 0 n n n ℓ \ell ℓ c 1 c_1 c 1 c 2 c_2 c 2 N N N f : S N → R f:\mathbb{S}_N \to\mathbb{R} f : S N → R
∥ L ℓ f ∥ G n ≤ C ∥ f ∥ G n + 2 ℓ , max x ∈ S N ∣ A L ℓ f ( x ) ∣ ≤ C ∥ f ∥ G 2 ℓ + 1 . \|L^\ell f\|_{G_n} \le C\|f\|_{G_{n+2\ell}}, \qquad\max_{x\in\mathbb{S}_N}|AL^\ell f(x)| \le C\|f\|_{G_{2\ell+1}}. ∥ L ℓ f ∥ G n ≤ C ∥ f ∥ G n + 2 ℓ , x ∈ S N max ∣ A L ℓ f ( x ) ∣ ≤ C ∥ f ∥ G 2 ℓ + 1 . The following lemma controls the Sobolev norms of the spherical pure k k k [11 , 10 Theorem 3.3] .
Lemma 3.7 (Sobolev norm concentration for the spherical pure k k k Let H 0 H_0 H 0 k k k (2) . For any integer n ≥ 0 n \ge0 n ≥ 0 C k , n , c k , n > 0 C_{k,n},c_{k,n} > 0 C k , n , c k , n > 0 r > 0 r > 0 r > 0
P [ ∥ H 0 ∥ G n ≥ C k , n N + r ] ≤ C k , n exp ( − c k , n r 2 N ) . \mathbb{P}\left[\|H_0\|_{G_n} \ge C_{k,n}N+r\right] \le C_{k,n}\exp\left(-\frac{c_{k,n}r^2}{N}\right). P [ ∥ H 0 ∥ G n ≥ C k , n N + r ] ≤ C k , n exp ( − N c k , n r 2 ) . Remark 3.8. As noted in [10 Corollary 3.6] , we conclude by a direct application of Lemmas 3.6 and 3.7 that ∥ Δ S N H 0 ∥ G n ≤ C k , n N \|\Delta_{\mathbb{S}_N}H_0\|_{G_n} \le C_{k,n}N ∥ Δ S N H 0 ∥ G n ≤ C k , n N ∥ ∥ ∇ S N H 0 ∥ 2 ∥ G n ≤ C k , n N \|\|\nabla_{\mathbb{S}_N}H_0\|^2\|_{G_n} \le C_{k,n}N ∥∥ ∇ S N H 0 ∥ 2 ∥ G n ≤ C k , n N n ≥ 0 n \ge0 n ≥ 0 1 − exp ( − c k , n N ) 1-\exp(-c_{k,n}N) 1 − exp ( − c k , n N )
The following lemma, which is a straightforward consequence of Eq. (3.4) in [10 ] , controls the Sobolev norms of the correlation with an arbitrary unit vector and its powers.
Lemma 3.9 (Sobolev norm bounds for the signal term). For any v ∈ S N − 1 v \in\mathbb{S}^{N-1} v ∈ S N − 1 g ( x ) = 1 N ⟨ v , x ⟩ g(x)=\frac{1}{\sqrt{N}}\langle v,x\rangle g ( x ) = N 1 ⟨ v , x ⟩ n ≥ 0 n \ge0 n ≥ 0 ∥ g ∥ G n ≤ C n \|g\|_{G_n} \le C_n ∥ g ∥ G n ≤ C n ∥ g k ∥ G n ≤ C k , n \|g^k\|_{G_n} \le C_{k,n} ∥ g k ∥ G n ≤ C k , n m m m (2) .
Remark 3.10. By Lemma 3.7 with r = N r=N r = N Lemma 3.9 , the tensor PCA Hamiltonian satisfies ∥ H ∥ G n ≤ C k , n N ( 1 + λ ) \|H\|_{G_n} \le C_{k,n}N(1+\lambda) ∥ H ∥ G n ≤ C k , n N ( 1 + λ ) n ≥ 0 n \ge0 n ≥ 0 1 − C k , n exp ( − c k , n N ) 1-C_{k,n}\exp(-c_{k,n}N) 1 − C k , n exp ( − c k , n N ) Definition 3.5 yields max x ∈ S N ∥ ∇ S N H ( x ) ∥ ≤ C k , n ( 1 + λ ) N \max_{x\in\mathbb{S}_N}\|\nabla_{\mathbb{S}_N}H(x)\| \le C_{k,n}(1+\lambda)\sqrt{N} max x ∈ S N ∥ ∇ S N H ( x ) ∥ ≤ C k , n ( 1 + λ ) N n ≥ 1 n \ge1 n ≥ 1 max x ∈ S N ∥ ∇ S N 2 H ( x ) ∥ o p ≤ C k , n ( 1 + λ ) \max_{x\in\mathbb{S}_N}\|\nabla_{\mathbb{S}_N}^2H(x)\|_{\mathrm{op}} \le C_{k,n}(1+\lambda) max x ∈ S N ∥ ∇ S N 2 H ( x ) ∥ op ≤ C k , n ( 1 + λ ) n ≥ 2 n \ge2 n ≥ 2
Spherical Langevin Dynamics
We use ( B t ) t ≥ 0 (B_t)_{t\ge0} ( B t ) t ≥ 0 R N \mathbb{R}^N R N B 0 = 0 B_0=0 B 0 = 0 B t − B s ∼ N ( 0 , ( t − s ) I ) B_t-B_s\sim\mathcal{N}(0,(t-s)I) B t − B s ∼ N ( 0 , ( t − s ) I ) 0 ≤ s < t 0\le s<t 0 ≤ s < t V : S N → R V:\mathbb{S}_N\to\mathbb{R} V : S N → R β ≥ 0 \beta\ge0 β ≥ 0 Z 0 ∈ S N Z_0 \in\mathbb{S}_N Z 0 ∈ S N ( Z t ) t ≥ 0 (Z_t)_{t\ge0} ( Z t ) t ≥ 0
d Z t = ( β ⋅ Q Z t ∇ V ( Z t ) − N − 1 2 N Z t ) d t + Q Z t d B t , (18) dZ_t=\left(\beta\cdot Q_{Z_t}\nabla V(Z_t)-\frac{N-1}{2N}Z_t\right)dt+Q_{Z_t}\,dB_t, \tag*{(18)} d Z t = ( β ⋅ Q Z t ∇ V ( Z t ) − 2 N N − 1 Z t ) d t + Q Z t d B t , (18) where the driving Brownian motion ( B t ) t ≥ 0 (B_t)_{t\ge0} ( B t ) t ≥ 0 V V V V V V V V V ( Z t ) t ≥ 0 (Z_t)_{t\ge0} ( Z t ) t ≥ 0
Since Q x x = 0 Q_xx=0 Q x x = 0 Tr ( Q x ) = N − 1 \operatorname{Tr}(Q_x)=N-1 Tr ( Q x ) = N − 1 [47 Section 4.4] yields d ∥ Z t ∥ 2 = 0 d\lVert Z_t\rVert^2=0 d ∥ Z t ∥ 2 = 0 Z t ∈ S N Z_t\in\mathbb{S}_N Z t ∈ S N t ≥ 0 t\ge0 t ≥ 0 μ V , β ( d x ) ∝ exp ( 2 β V ( x ) ) π ( d x ) \mu_{V,\beta}(dx)\propto\exp(2\beta V(x))\pi(dx) μ V , β ( d x ) ∝ exp ( 2 β V ( x )) π ( d x ) β = 0 \beta=0 β = 0 S N \mathbb{S}_N S N
The associated Markov semigroup P t ( V , β ) P_t^{(V,\beta)} P t ( V , β ) f : S N → R f:\mathbb{S}_N\to\mathbb{R} f : S N → R
P t ( V , β ) f ( x ) = E [ f ( Z t ) ∣ Z 0 = x ] . P_t^{(V,\beta)}f(x)=\mathbb{E}[f(Z_t)\mid Z_0=x]. P t ( V , β ) f ( x ) = E [ f ( Z t ) ∣ Z 0 = x ] . The generator L V , β \mathcal{L}_{V,\beta} L V , β f : S N → R f:\mathbb{S}_N\to\mathbb{R} f : S N → R
L V , β f ( x ) = lim t → 0 + P t ( V , β ) f ( x ) − f ( x ) t = 1 2 Δ S N f ( x ) + β ⟨ ∇ S N V ( x ) , ∇ S N f ( x ) ⟩ , (19) \mathcal{L}_{V,\beta}f(x)=\lim_{t\to0+}\frac{P_t^{(V,\beta)}f(x)-f(x)}{t}=\frac{1}{2}\Delta_{\mathbb{S}_N}f(x)+\beta\langle\nabla_{\mathbb{S}_N}V(x),\nabla_{\mathbb{S}_N}f(x)\rangle, \tag*{(19)} L V , β f ( x ) = t → 0 + lim t P t ( V , β ) f ( x ) − f ( x ) = 2 1 Δ S N f ( x ) + β ⟨ ∇ S N V ( x ) , ∇ S N f ( x )⟩ , (19) where the second equality follows from Itô’s formula applied to any smooth extension of f f f (8) and Tr ( Q x ) = N − 1 \operatorname{Tr}(Q_x)=N-1 Tr ( Q x ) = N − 1 1 2 Δ S N f ( x ) \frac{1}{2}\Delta_{\mathbb{S}_N}f(x) 2 1 Δ S N f ( x ) β ⟨ ∇ S N V ( x ) , ∇ S N f ( x ) ⟩ \beta\langle\nabla_{\mathbb{S}_N}V(x),\nabla_{\mathbb{S}_N}f(x)\rangle β ⟨ ∇ S N V ( x ) , ∇ S N f ( x )⟩ diffusion term and the drift term of the generator, respectively.
The Markov semigroup and its generator also satisfy the following Kolmogorov equations for every smooth f : S N → R f:\mathbb{S}_N\to\mathbb{R} f : S N → R
d d t P t ( V , β ) f = L V , β P t ( V , β ) f = P t ( V , β ) L V , β f . (20) \frac{d}{dt}P_t^{(V,\beta)}f=\mathcal{L}_{V,\beta}P_t^{(V,\beta)}f=P_t^{(V,\beta)}\mathcal{L}_{V,\beta}f. \tag*{(20)} d t d P t ( V , β ) f = L V , β P t ( V , β ) f = P t ( V , β ) L V , β f . (20) Intuitively, for any smooth observable f : S N → R f:\mathbb{S}_N\to\mathbb{R} f : S N → R L V , β f ( Z t ) \mathcal{L}_{V,\beta}f(Z_t) L V , β f ( Z t ) f ( Z t ) f(Z_t) f ( Z t )
Lemma 3.11 (Dynkin’s formula for spherical Langevin dynamics). Let ( Z t ) t ≥ 0 (Z_t)_{t\ge0} ( Z t ) t ≥ 0 V V V β \beta β ( B t ) t ≥ 0 (B_t)_{t\ge0} ( B t ) t ≥ 0 f : S N → R f:\mathbb{S}_N\to\mathbb{R} f : S N → R
M t f = f ( Z t ) − f ( Z 0 ) − ∫ 0 t L V , β f ( Z s ) d s = ∫ 0 t ⟨ ∇ S N f ( Z s ) , d B s ⟩ M_t^f=f(Z_t)-f(Z_0)-\int_0^t\mathcal{L}_{V,\beta}f(Z_s)\,ds=\int_0^t\langle\nabla_{\mathbb{S}_N}f(Z_s),dB_s\rangle M t f = f ( Z t ) − f ( Z 0 ) − ∫ 0 t L V , β f ( Z s ) d s = ∫ 0 t ⟨ ∇ S N f ( Z s ) , d B s ⟩ is a continuous martingale with M 0 f = 0 M_0^f=0 M 0 f = 0
⟨ M f ⟩ t = 2 ∫ 0 t Γ ( f , f ) ( Z s ) d s , \langle M^f\rangle_t=2\int_0^t\Gamma(f,f)(Z_s)\,ds, ⟨ M f ⟩ t = 2 ∫ 0 t Γ ( f , f ) ( Z s ) d s , where Γ \Gamma Γ L V , β \mathcal{L}_{V,\beta} L V , β
Γ ( f , g ) ( x ) = 1 2 [ L V , β ( f g ) − f L V , β g − g L V , β f ] ( x ) = 1 2 ⟨ ∇ S N f ( x ) , ∇ S N g ( x ) ⟩ . \Gamma(f,g)(x)=\frac{1}{2}\left[\mathcal{L}_{V,\beta}(fg)-f\mathcal{L}_{V,\beta}g-g\mathcal{L}_{V,\beta}f\right](x)=\frac{1}{2}\langle\nabla_{\mathbb{S}^{N}}f(x),\nabla_{\mathbb{S}^{N}}g(x)\rangle. Γ ( f , g ) ( x ) = 2 1 [ L V , β ( f g ) − f L V , β g − g L V , β f ] ( x ) = 2 1 ⟨ ∇ S N f ( x ) , ∇ S N g ( x )⟩ . In particular, Γ \Gamma Γ V V V β \beta β
The following uniform concentration guarantee for Dynkin martingales shall be used often in our proofs.
Lemma 3.12 (Concentration of Dynkin martingales). Let ( Z t ) t ≥ 0 (Z_t)_{t\geq0} ( Z t ) t ≥ 0 V V V β \beta β f : S N → R f:\mathbb{S}^{N}\to\mathbb{R} f : S N → R max x ∈ S N ∥ ∇ S N f ( x ) ∥ ≤ A \max_{x\in\mathbb{S}^{N}}\|\nabla_{\mathbb{S}^{N}}f(x)\|\leq A max x ∈ S N ∥ ∇ S N f ( x ) ∥ ≤ A A > 0 A>0 A > 0 M t f M^f_t M t f T , r > 0 T,r>0 T , r > 0
P [ sup 0 ≤ t ≤ T ∣ M t f ∣ ≥ r ] ≤ 2 exp ( − r 2 2 A 2 T ) . \mathbb{P}\left[\sup_{0\leq t\leq T}|M^f_t|\geq r\right]\leq2\exp\left(-\frac{r^2}{2A^2T}\right). P [ 0 ≤ t ≤ T sup ∣ M t f ∣ ≥ r ] ≤ 2 exp ( − 2 A 2 T r 2 ) . Proof. Since 2 Γ ( f , f ) = ∥ ∇ S N f ∥ 2 ≤ A 2 2\Gamma(f,f)=\|\nabla_{\mathbb{S}^{N}}f\|^2\leq A^2 2Γ ( f , f ) = ∥ ∇ S N f ∥ 2 ≤ A 2 Lemma 3.11 that M f M^f M f M 0 f = 0 M^f_0=0 M 0 f = 0 ⟨ M f ⟩ t ≤ A 2 T \langle M^f\rangle_t\leq A^2T ⟨ M f ⟩ t ≤ A 2 T t ≤ T t\leq T t ≤ T M f M^f M f − M f -M^f − M f a , b > 0 a,b>0 a , b > 0
P [ ∃ t ≥ 0 : ± M t f ≥ a + b ⟨ M f ⟩ t ] ≤ e − 2 a b . \mathbb{P}\left[\exists\,t\geq0:\pm M^f_t\geq a+b\langle M^f\rangle_t\right]\leq e^{-2ab}. P [ ∃ t ≥ 0 : ± M t f ≥ a + b ⟨ M f ⟩ t ] ≤ e − 2 ab . If sup 0 ≤ t ≤ T ∣ M t f ∣ ≥ r \sup_{0\leq t\leq T}|M^f_t|\geq r sup 0 ≤ t ≤ T ∣ M t f ∣ ≥ r ∣ M t f ∣ ≥ r |M^f_t|\geq r ∣ M t f ∣ ≥ r t ≤ T t\leq T t ≤ T a = r 2 a=\frac{r}{2} a = 2 r b = r 2 A 2 T b=\frac{r}{2A^2T} b = 2 A 2 T r a + b ⟨ M f ⟩ t ≤ r a+b\langle M^f\rangle_t\leq r a + b ⟨ M f ⟩ t ≤ r 2 a b = r 2 2 A 2 T 2ab=\frac{r^2}{2A^2T} 2 ab = 2 A 2 T r 2
Remark. By Definition 3.5 , max x ∈ S N ∥ ∇ S N f ( x ) ∥ ≤ N − 1 / 2 ∥ f ∥ G 1 \max_{x\in\mathbb{S}^{N}}\|\nabla_{\mathbb{S}^{N}}f(x)\|\leq N^{-1/2}\|f\|_{\mathcal{G}_1} max x ∈ S N ∥ ∇ S N f ( x ) ∥ ≤ N − 1/2 ∥ f ∥ G 1 Lemma 3.12 may always be applied with A = N − 1 / 2 ∥ f ∥ G 1 A=N^{-1/2}\|f\|_{\mathcal{G}_1} A = N − 1/2 ∥ f ∥ G 1
The Markov semigroup P t ( V , β ) P_t^{(V,\beta)} P t ( V , β ) L V , β \mathcal{L}_{V,\beta} L V , β Lemma 3.11 to each entry, and we obtain the following uniform concentration guarantee for operator norms of rectangular matrix valued Dynkin martingales via a direct application of Lemma 3.12 .
Lemma 3.13 (Concentration of matrix-valued Dynkin martingales). Let m , n ≥ 1 m,n\geq1 m , n ≥ 1 F : S N → R m × n F:\mathbb{S}^{N}\to\mathbb{R}^{m\times n} F : S N → R m × n max x ∈ S N ∥ ∇ S N ( u T F v ) ( x ) ∥ ≤ A \max_{x\in\mathbb{S}^{N}}\|\nabla_{\mathbb{S}^{N}}(u^{\mathsf{T}}Fv)(x)\|\leq A max x ∈ S N ∥ ∇ S N ( u T F v ) ( x ) ∥ ≤ A u ∈ S m − 1 u\in\mathbb{S}^{m-1} u ∈ S m − 1 v ∈ S n − 1 v\in\mathbb{S}^{n-1} v ∈ S n − 1 A > 0 A>0 A > 0 ( Z t ) t ≥ 0 (Z_t)_{t\geq0} ( Z t ) t ≥ 0 V V V β \beta β M t F M^F_t M t F T > 0 T>0 T > 0 δ ∈ ( 0 , 1 ) \delta\in(0,1) δ ∈ ( 0 , 1 ) 1 − δ 1-\delta 1 − δ
sup 0 ≤ t ≤ T ∥ M t F ∥ o p ≤ C A T ( m + n + log ( 1 / δ ) ) , \sup_{0\leq t\leq T}\|M^F_t\|_{\mathrm{op}}\leq CA\sqrt{T(m+n+\log(1/\delta))}, 0 ≤ t ≤ T sup ∥ M t F ∥ op ≤ C A T ( m + n + log ( 1/ δ )) , where C > 0 C>0 C > 0 n = 1 n=1 n = 1
Proof. Fix u ∈ S m − 1 u \in\mathbb{S}^{m-1} u ∈ S m − 1 v ∈ S n − 1 v \in\mathbb{S}^{n-1} v ∈ S n − 1 f u , v ( x ) = u T F ( x ) v f_{u,v}(x) = u^{\mathsf{T}}F(x)v f u , v ( x ) = u T F ( x ) v u T M t F v = M t f u , v u^{\mathsf{T}}M_t^Fv = M_t^{f_{u,v}} u T M t F v = M t f u , v max x ∈ S N ∥ ∇ S N f u , v ( x ) ∥ ≤ A \max_{x \in\mathbb{S}^{N}}\lVert\nabla_{\mathbb{S}^{N}}f_{u,v}(x)\rVert\le A max x ∈ S N ∥ ∇ S N f u , v ( x )∥ ≤ A Lemma 3.12 with r = A 2 T s r = A\sqrt{2Ts} r = A 2 T s s > 0 s > 0 s > 0
P [ sup 0 ≤ t ≤ T ∣ u T M t F v ∣ ≥ A 2 T s ] ≤ 2 e − s . \mathbb{P}\left[\sup_{0 \le t \le T}\left|u^{\mathsf{T}}M_t^Fv\right| \ge A\sqrt{2Ts}\right] \le2e^{-s}. P [ 0 ≤ t ≤ T sup u T M t F v ≥ A 2 T s ] ≤ 2 e − s . Let N m \mathcal{N}_m N m N n \mathcal{N}_n N n 1 / 4 1/4 1/4 S m − 1 \mathbb{S}^{m-1} S m − 1 S n − 1 \mathbb{S}^{n-1} S n − 1 ∣ N m ∣ ≤ 9 m \lvert\mathcal{N}_m\rvert\le9^m ∣ N m ∣ ≤ 9 m ∣ N n ∣ ≤ 9 n \lvert\mathcal{N}_n\rvert\le9^n ∣ N n ∣ ≤ 9 n ϵ \epsilon ϵ t ≥ 0 t \ge0 t ≥ 0
∥ M t F ∥ o p ≤ 2 max u ∈ N m , v ∈ N n ∣ u T M t F v ∣ . \lVert M_t^F\rVert_{\mathrm{op}} \le2\max_{u \in\mathcal{N}_m,v \in\mathcal{N}_n}\left|u^{\mathsf{T}}M_t^Fv\right|. ∥ M t F ∥ op ≤ 2 u ∈ N m , v ∈ N n max u T M t F v . The proof is concluded by taking s = ( m + n ) log 9 + log ( 2 / δ ) s = (m+n)\log9+\log(2/\delta) s = ( m + n ) log 9 + log ( 2/ δ ) N m × N n \mathcal{N}_m \times\mathcal{N}_n N m × N n
Remark 3.14. If m = n m=n m = n F F F Sym n \operatorname{Sym}_n Sym n w ∈ S n − 1 w \in\mathbb{S}^{n-1} w ∈ S n − 1 max x ∈ S N ∥ ∇ S N ( w T F w ) ( x ) ∥ ≤ A \max_{x \in\mathbb{S}^{N}}\lVert\nabla_{\mathbb{S}^{N}}(w^{\mathsf{T}}Fw)(x)\rVert\le A max x ∈ S N ∥ ∇ S N ( w T F w ) ( x )∥ ≤ A
Finally, the following change of measure theorem for spherical Langevin dynamics is a direct consequence of the Girsanov theorem [47 Section 6.4] .
Lemma 3.15 (Girsanov’s Theorem for Spherical Langevin Dynamics). Fix T > 0 T>0 T > 0 ( Z t ) 0 ≤ t ≤ T (Z_t)_{0 \le t \le T} ( Z t ) 0 ≤ t ≤ T V V V β \beta β ( B t ) 0 ≤ t ≤ T (B_t)_{0 \le t \le T} ( B t ) 0 ≤ t ≤ T Q T \mathbb{Q}^{T} Q T Z 0 Z_0 Z 0 ( B t ) 0 ≤ t ≤ T (B_t)_{0 \le t \le T} ( B t ) 0 ≤ t ≤ T ( θ t ) 0 ≤ t ≤ T (\theta_t)_{0 \le t \le T} ( θ t ) 0 ≤ t ≤ T R N \mathbb{R}^{N} R N ∥ θ t ∥ ≤ C \lVert\theta_t\rVert\le C ∥ θ t ∥ ≤ C C < ∞ C < \infty C < ∞ t ≤ T t \le T t ≤ T 0 ≤ t ≤ T 0 \le t \le T 0 ≤ t ≤ T
L t θ = exp ( ∫ 0 t ⟨ θ s , d B s ⟩ − 1 2 ∫ 0 t ∥ θ s ∥ 2 d s ) . L_t^\theta= \exp\left(\int_0^t \langle\theta_s,\mathrm{d}B_s\rangle-\frac{1}{2}\int_0^t\lVert\theta_s\rVert^2\mathrm{d}s\right). L t θ = exp ( ∫ 0 t ⟨ θ s , d B s ⟩ − 2 1 ∫ 0 t ∥ θ s ∥ 2 d s ) . Then ( L t θ ) 0 ≤ t ≤ T (L_t^\theta)_{0 \le t \le T} ( L t θ ) 0 ≤ t ≤ T Q T \mathbb{Q}^{T} Q T Q T , θ \mathbb{Q}^{T,\theta} Q T , θ d Q T , θ d Q T = L T θ \frac{\mathrm{d}\mathbb{Q}^{T,\theta}}{\mathrm{d}\mathbb{Q}^{T}}=L_T^\theta d Q T d Q T , θ = L T θ Q T , θ \mathbb{Q}^{T,\theta} Q T , θ B ~ t = B t − ∫ 0 t θ s d s \widetilde{B}_t=B_t-\int_0^t\theta_s\mathrm{d}s B t = B t − ∫ 0 t θ s d s [ 0 , T ] [0,T] [ 0 , T ] Z 0 Z_0 Z 0 Z 0 Z_0 Z 0 Q T \mathbb{Q}^{T} Q T
d Z t = ( β ⋅ Q Z t ∇ V ( Z t ) − N − 1 2 N Z t + Q Z t θ t ) d t + Q Z t d B ~ t . \mathrm{d}Z_t=\left(\beta\cdot Q_{Z_t}\nabla V(Z_t)-\frac{N-1}{2N}Z_t+Q_{Z_t}\theta_t\right)\mathrm{d}t+Q_{Z_t}\mathrm{d}\widetilde{B}_t. d Z t = ( β ⋅ Q Z t ∇ V ( Z t ) − 2 N N − 1 Z t + Q Z t θ t ) d t + Q Z t d B t . In particular, if θ t = β ∇ S N U ( Z t ) \theta_t=\beta\nabla_{\mathbb{S}^{N}}U(Z_t) θ t = β ∇ S N U ( Z t ) U : S N → R U:\mathbb{S}^{N}\to\mathbb{R} U : S N → R Q T , θ \mathbb{Q}^{T,\theta} Q T , θ ( Z t ) 0 ≤ t ≤ T (Z_t)_{0 \le t \le T} ( Z t ) 0 ≤ t ≤ T V + U V+U V + U β \beta β ( B ~ t ) 0 ≤ t ≤ T (\widetilde{B}_t)_{0 \le t \le T} ( B t ) 0 ≤ t ≤ T
Averaged Spherical Derivatives of the Tensor PCA Hamiltonian
In this section, we analyze the statistical properties of the average spherical gradient and Hessian of the tensor PCA Hamiltonian. In § 4.1 , we show that, for odd k k k § 4.2 , we show that, for even k k k
Analysis of the Average Spherical Gradient
We now establish a signal-noise decomposition of the average spherical gradient of the tensor PCA Hamiltonian.
Theorem 4.1 (Signal-Noise Decomposition of the Average Spherical Gradient). Let H H H (2) , and define the average spherical gradient u H ≔ E π [ ∇ S N H ( x ) ] \mathbf{u}_H \coloneqq\mathbb{E}_{\pi}[\nabla_{\mathbb{S}_N}H(\mathbf{x})] u H : = E π [ ∇ S N H ( x )]
If k k k u H = 0 \mathbf{u}_H=0 u H = 0
If k k k u H = N λ α N θ ⋆ + ζ \mathbf{u}_H=N\lambda\alpha_N\theta^\star+\zeta u H = N λ α N θ ⋆ + ζ ζ ≔ E π [ ∇ S N H 0 ( x ) ] ∼ N ( 0 , σ N 2 I ) \zeta\coloneqq\mathbb{E}_{\pi}[\nabla_{\mathbb{S}_N}H_0(\mathbf{x})]\sim\mathcal{N}(0,\sigma_N^2I) ζ : = E π [ ∇ S N H 0 ( x )] ∼ N ( 0 , σ N 2 I ) α N ≍ k N − k / 2 \alpha_N\asymp_k N^{-k/2} α N ≍ k N − k /2 σ N 2 = κ N N N α N ≍ k N − k − 1 2 \sigma_N^2=\kappa_NN\sqrt{N}\alpha_N\asymp_k N^{-\frac{k-1}{2}} σ N 2 = κ N N N α N ≍ k N − 2 k − 1
Proof. By Stein’s lemma (Lemma 3.1) and Equation (2) , we obtain
u H = κ N E π [ x H ( x ) ] = N λ κ N E π [ x m ( x ) k ] + ζ , ζ = κ N E π [ x H 0 ( x ) ] . (21) \mathbf{u}_H=\kappa_N\mathbb{E}_{\pi}[\mathbf{x}H(\mathbf{x})]=N\lambda\kappa_N\mathbb{E}_{\pi}[\mathbf{x}m(\mathbf{x})^k]+\zeta,\qquad\zeta=\kappa_N\mathbb{E}_{\pi}[\mathbf{x}H_0(\mathbf{x})]. \tag*{(21)} u H = κ N E π [ x H ( x )] = N λ κ N E π [ x m ( x ) k ] + ζ , ζ = κ N E π [ x H 0 ( x )] . (21) If k k k H ( − x ) = H ( x ) H(-\mathbf{x})=H(\mathbf{x}) H ( − x ) = H ( x ) π \pi π x ↦ − x \mathbf{x}\mapsto-\mathbf{x} x ↦ − x u H = 0 \mathbf{u}_H=0 u H = 0
Henceforth, assume k k k (21) . For every v ∈ S N − 1 \mathbf{v}\in\mathbb{S}^{N-1} v ∈ S N − 1 θ ⋆ \theta^\star θ ⋆ ( w , z ) = ( v , θ ⋆ ) (\mathbf{w},\mathbf{z})=(\mathbf{v},\theta^\star) ( w , z ) = ( v , θ ⋆ ) g ( t ) = t g(t)=t g ( t ) = t h ( t ) = t k h(t)=t^k h ( t ) = t k E π [ ⟨ v , x ⟩ m ( x ) k ] = 0 \mathbb{E}_{\pi}[\langle\mathbf{v},\mathbf{x}\rangle m(\mathbf{x})^k]=0 E π [⟨ v , x ⟩ m ( x ) k ] = 0 E π [ x m ( x ) k ] \mathbb{E}_{\pi}[\mathbf{x}m(\mathbf{x})^k] E π [ x m ( x ) k ] θ ⋆ \theta^\star θ ⋆ θ ⋆ \theta^\star θ ⋆ ⟨ θ ⋆ , x ⟩ = N m ( x ) \langle\theta^\star,\mathbf{x}\rangle=\sqrt{N}m(\mathbf{x}) ⟨ θ ⋆ , x ⟩ = N m ( x )
κ N E π [ x m ( x ) k ] = κ N E π [ ⟨ θ ⋆ , x ⟩ m ( x ) k ] θ ⋆ = α N θ ⋆ , α N ≔ κ N N E π [ m ( x ) k + 1 ] . (22) \kappa_N\mathbb{E}_{\pi}[\mathbf{x}m(\mathbf{x})^k]=\kappa_N\mathbb{E}_{\pi}[\langle\theta^\star,\mathbf{x}\rangle m(\mathbf{x})^k]\theta^\star=\alpha_N\theta^\star,\qquad\alpha_N\coloneqq\kappa_N\sqrt{N}\mathbb{E}_{\pi}[m(\mathbf{x})^{k+1}]. \tag*{(22)} κ N E π [ x m ( x ) k ] = κ N E π [⟨ θ ⋆ , x ⟩ m ( x ) k ] θ ⋆ = α N θ ⋆ , α N : = κ N N E π [ m ( x ) k + 1 ] . (22) Since k + 1 k+1 k + 1 α N ≍ k N − k / 2 \alpha_N\asymp_k N^{-k/2} α N ≍ k N − k /2 (21) is N λ α N θ ⋆ N\lambda\alpha_N\theta^\star N λ α N θ ⋆
We now analyze the noise term in Equation (21) . Since ζ \zeta ζ G G G Σ = E G [ ζ ζ T ] \Sigma=\mathbb{E}_G[\zeta\zeta^\mathsf{T}] Σ = E G [ ζ ζ T ] ζ = κ N E π [ x H 0 ( x ) ] \zeta=\kappa_N\mathbb{E}_{\pi}[\mathbf{x}H_0(\mathbf{x})] ζ = κ N E π [ x H 0 ( x )] (3) :
Σ = κ N 2 E x , y ∼ i . i . d . π [ x y T E G [ H 0 ( x ) H 0 ( y ) ] ] = κ N 2 N E x , y ∼ i . i . d . π [ x y T ( ⟨ x , y ⟩ N ) k ] . (23) \Sigma=\kappa_N^2\mathbb{E}_{\mathbf{x},\mathbf{y}\overset{\mathrm{i.i.d.}}{\sim}\pi}[\mathbf{x}\mathbf{y}^{\mathsf{T}}\mathbb{E}_G[H_0(\mathbf{x})H_0(\mathbf{y})]]
=\kappa_N^2N\mathbb{E}_{\mathbf{x},\mathbf{y}\overset{\mathrm{i.i.d.}}{\sim}\pi}\left[\mathbf{x}\mathbf{y}^{\mathsf{T}}\left(\frac{\langle\mathbf{x},\mathbf{y}\rangle}{N}\right)^k\right]. \tag*{(23)} Σ = κ N 2 E x , y ∼ i.i.d. π [ x y T E G [ H 0 ( x ) H 0 ( y )]] = κ N 2 N E x , y ∼ i.i.d. π [ x y T ( N ⟨ x , y ⟩ ) k ] . (23) Fix R ∈ O ( N , R ) R\in O(N,\mathbb{R}) R ∈ O ( N , R ) π \pi π O ( N , R ) O(N,\mathbb{R}) O ( N , R ) ⟨ R x , R y ⟩ = ⟨ x , y ⟩ \langle R\mathbf{x},R\mathbf{y}\rangle=\langle\mathbf{x},\mathbf{y}\rangle ⟨ R x , R y ⟩ = ⟨ x , y ⟩ ( x , y ) ↦ ( R x , R y ) (\mathbf{x},\mathbf{y})\mapsto(R\mathbf{x},R\mathbf{y}) ( x , y ) ↦ ( R x , R y ) (23) yields R Σ R T = Σ R\Sigma R^{\mathsf{T}}=\Sigma R Σ R T = Σ Σ \Sigma Σ O ( N , R ) O(N,\mathbb{R}) O ( N , R ) Σ = σ N 2 I \Sigma=\sigma_N^2I Σ = σ N 2 I σ N 2 ≥ 0 \sigma_N^2\ge0 σ N 2 ≥ 0
Observe that by O ( N , R ) O(N,\mathbb{R}) O ( N , R ) π \pi π x ∈ S N \mathbf{x}\in\mathbb{S}_N x ∈ S N y \mathbf{y} y
E y ∼ π [ ( ⟨ x , y ⟩ N ) k + 1 ] = E y ∼ π [ m ( y ) k + 1 ] . (24) \mathbb{E}_{\mathbf{y}\sim\pi}\left[\left(\frac{\langle\mathbf{x},\mathbf{y}\rangle}{N}\right)^{k+1}\right]=\mathbb{E}_{\mathbf{y}\sim\pi}[m(\mathbf{y})^{k+1}]. \tag*{(24)} E y ∼ π [ ( N ⟨ x , y ⟩ ) k + 1 ] = E y ∼ π [ m ( y ) k + 1 ] . (24) Taking the trace in Equation (23) and using Tr ( x y T ) = ⟨ x , y ⟩ \operatorname{Tr}(\mathbf{x}\mathbf{y}^{\mathsf{T}})=\langle\mathbf{x},\mathbf{y}\rangle Tr ( x y T ) = ⟨ x , y ⟩ (22) and (24) :
σ N 2 = 1 N Tr ( Σ ) = κ N 2 E x , y ∼ i . i . d . π [ ⟨ x , y ⟩ ( ⟨ x , y ⟩ N ) k ] . = κ N 2 N E x ∼ π [ E y ∼ π [ ( ⟨ x , y ⟩ N ) k + 1 ] ] = κ N 2 N E y ∼ π [ m ( y ) k + 1 ] = κ N N ⋅ κ N N E y ∼ π [ m ( y ) k + 1 ] = κ N N α N . \begin{aligned}
\sigma_N^2=\frac{1}{N}\operatorname{Tr}(\Sigma)=\kappa_N^2\mathbb{E}_{\mathbf{x},\mathbf{y}\overset{\mathrm{i.i.d.}}{\sim}\pi}\left[\langle\mathbf{x},\mathbf{y}\rangle\left(\frac{\langle\mathbf{x},\mathbf{y}\rangle}{N}\right)^k\right]. \\
&= \kappa_N^2 N \mathbb{E}_{x\sim\pi}\left[\mathbb{E}_{y\sim\pi}\left[\left(\frac{\langle x,y\rangle}{N}\right)^{k+1}\right]\right] = \kappa_N^2 N \mathbb{E}_{y\sim\pi}\left[m(y)^{k+1}\right] \\
&= \kappa_N\sqrt{N}\cdot\kappa_N\sqrt{N}\mathbb{E}_{y\sim\pi}\left[m(y)^{k+1}\right] = \kappa_N\sqrt{N}\alpha_N.
\end{aligned} σ N 2 = N 1 Tr ( Σ ) = κ N 2 E x , y ∼ i.i.d. π [ ⟨ x , y ⟩ ( N ⟨ x , y ⟩ ) k ] . = κ N 2 N E x ∼ π [ E y ∼ π [ ( N ⟨ x , y ⟩ ) k + 1 ] ] = κ N 2 N E y ∼ π [ m ( y ) k + 1 ] = κ N N ⋅ κ N N E y ∼ π [ m ( y ) k + 1 ] = κ N N α N . Thus, we conclude that ζ ∼ N ( 0 , σ N 2 I ) \zeta\sim\mathcal{N}(0,\sigma_N^2 I) ζ ∼ N ( 0 , σ N 2 I ) σ N 2 = κ N N α N ≍ k N − k − 1 2 \sigma_N^2=\kappa_N\sqrt{N}\alpha_N\asymp_k N^{-\frac{k-1}{2}} σ N 2 = κ N N α N ≍ k N − 2 k − 1
Remark 4.2. We make the following observations from Theorem 4.1 . For even k k k x ↦ − x x\mapsto-x x ↦ − x u H = 0 u_H=0 u H = 0 k k k σ N 2 = κ N N α N \sigma_N^2=\kappa_N\sqrt{N}\alpha_N σ N 2 = κ N N α N u H = κ N − 1 λ σ N 2 x ⋆ + ζ u_H=\kappa_N^{-1}\lambda\sigma_N^2x^\star+\zeta u H = κ N − 1 λ σ N 2 x ⋆ + ζ u H u_H u H x ⋆ x^\star x ⋆ κ N − 1 λ σ N ≍ k λ N − k − 1 4 \kappa_N^{-1}\lambda\sigma_N\asymp_k\lambda N^{-\frac{k-1}{4}} κ N − 1 λ σ N ≍ k λ N − 4 k − 1 λ ≳ k N k − 2 4 \lambda\gtrsim_k N^{\frac{k-2}{4}} λ ≳ k N 4 k − 2 Ω ( N − 1 / 4 ) \Omega(N^{-1/4}) Ω ( N − 1/4 )
Corollary 4.3 (Norm and Correlation of the Average Spherical Gradient). Let k ≥ 3 k\ge3 k ≥ 3 λ > 0 \lambda>0 λ > 0 u ^ H ≔ N u H ∥ u H ∥ \widehat{u}_H\coloneqq\frac{\sqrt{N}u_H}{\|u_H\|} u H : = ∥ u H ∥ N u H
P [ c k min { 1 , λ N − k − 1 4 } ≤ m ( u ^ H ) ≤ C k min { 1 , λ N − k − 1 4 } ] ≥ 1 − C exp ( − c k min { N , λ 2 N − k − 3 2 } ) . \mathbb{P}\left[c_k\min\left\{1,\lambda N^{-\frac{k-1}{4}}\right\}\le m(\widehat{u}_H)\le C_k\min\left\{1,\lambda N^{-\frac{k-1}{4}}\right\}\right]\ge1-C\exp\left(-c_k\min\left\{N,\lambda^2N^{-\frac{k-3}{2}}\right\}\right). P [ c k min { 1 , λ N − 4 k − 1 } ≤ m ( u H ) ≤ C k min { 1 , λ N − 4 k − 1 } ] ≥ 1 − C exp ( − c k min { N , λ 2 N − 2 k − 3 } ) . In particular, if λ ≥ C k N k − 2 4 \lambda\ge C_kN^{\frac{k-2}{4}} λ ≥ C k N 4 k − 2 N N N
P [ m ( u ^ H ) ≥ N − 1 / 4 ] ≥ 1 − exp ( − c k N ) . \mathbb{P}\left[m(\widehat{u}_H)\ge N^{-1/4}\right]\ge1-\exp(-c_k\sqrt{N}). P [ m ( u H ) ≥ N − 1/4 ] ≥ 1 − exp ( − c k N ) . Furthermore,
P [ ∥ u H ∥ ≥ c ( N λ α N + σ N N ) ] ≥ 1 − C exp ( − c N ) . \mathbb{P}\left[\|u_H\|\ge c(N\lambda\alpha_N+\sigma_N\sqrt{N})\right]\ge1-C\exp(-cN). P [ ∥ u H ∥ ≥ c ( N λ α N + σ N N ) ] ≥ 1 − C exp ( − c N ) . Proof. By Theorem 4.1 , u H = N λ α N θ ⋆ + ζ u_H=N\lambda\alpha_N\theta^\star+\zeta u H = N λ α N θ ⋆ + ζ ζ ∼ N ( 0 , σ N 2 I ) \zeta\sim\mathcal{N}(0,\sigma_N^2I) ζ ∼ N ( 0 , σ N 2 I ) Z = σ N − 1 ⟨ ζ , θ ⋆ ⟩ Z=\sigma_N^{-1}\langle\zeta,\theta^\star\rangle Z = σ N − 1 ⟨ ζ , θ ⋆ ⟩ ζ ⊥ = Q θ ⋆ ζ \zeta_\perp=Q_{\theta^\star}\zeta ζ ⊥ = Q θ ⋆ ζ Z ∼ N ( 0 , 1 ) Z\sim\mathcal{N}(0,1) Z ∼ N ( 0 , 1 ) σ N − 2 ∥ ζ ⊥ ∥ 2 \sigma_N^{-2}\|\zeta_\perp\|^2 σ N − 2 ∥ ζ ⊥ ∥ 2 χ 2 \chi^2 χ 2 N − 1 N-1 N − 1 u H = ( N λ α N + σ N Z ) θ ⋆ + ζ ⊥ u_H=(N\lambda\alpha_N+\sigma_NZ)\theta^\star+\zeta_\perp u H = ( N λ α N + σ N Z ) θ ⋆ + ζ ⊥
⟨ u H , θ ⋆ ⟩ = N λ α N + σ N Z , ∥ u H ∥ 2 = ( N λ α N + σ N Z ) 2 + ∥ ζ ⊥ ∥ 2 . (25) \langle u_H,\theta^\star\rangle=N\lambda\alpha_N+\sigma_NZ,\qquad\|u_H\|^2=(N\lambda\alpha_N+\sigma_NZ)^2+\|\zeta_\perp\|^2. \tag*{(25)} ⟨ u H , θ ⋆ ⟩ = N λ α N + σ N Z , ∥ u H ∥ 2 = ( N λ α N + σ N Z ) 2 + ∥ ζ ⊥ ∥ 2 . (25) By Gaussian concentration [cf. Ver18, Theorem 3.1.1], the event E ⊥ = { 1 4 σ N 2 N ≤ ∥ ζ ⊥ ∥ 2 ≤ 2 σ N 2 N } \mathcal{E}_\perp=\{\frac{1}{4}\sigma_N^2N\le\|\zeta_\perp\|^2\le2\sigma_N^2N\} E ⊥ = { 4 1 σ N 2 N ≤ ∥ ζ ⊥ ∥ 2 ≤ 2 σ N 2 N } 1 − 2 exp ( − c N ) 1-2\exp(-cN) 1 − 2 exp ( − c N )
We now prove the correlation bound. Define s N ≔ N λ α N σ N N s_N\coloneqq\frac{N\lambda\alpha_N}{\sigma_N\sqrt{N}} s N : = σ N N N λ α N σ N 2 = κ N N α N \sigma_N^2=\kappa_N\sqrt{N}\alpha_N σ N 2 = κ N N α N κ N ≍ 1 \kappa_N\asymp1 κ N ≍ 1 σ N ≍ k N − k − 1 4 \sigma_N\asymp_kN^{-\frac{k-1}{4}} σ N ≍ k N − 4 k − 1
s N = λ σ N κ N ≍ k λ N − k − 1 4 . s_N=\frac{\lambda\sigma_N}{\kappa_N}\asymp_k\lambda N^{-\frac{k-1}{4}}. s N = κ N λ σ N ≍ k λ N − 4 k − 1 . By a standard Gaussian tail bound applied to Z Z Z
P [ E ⊥ ∩ { ∣ Z ∣ ≤ 1 2 N min { s N , 1 } } ] ≥ 1 − 2 exp ( − c N ) − 2 exp ( − 1 8 N min { 1 , s N 2 } ) . \mathbb{P}\left[\mathcal{E}_\perp\cap\left\{|Z|\le\frac{1}{2}\sqrt{N}\min\{s_N,1\}\right\}\right]\ge1-2\exp(-cN)-2\exp\left(-\frac{1}{8}N\min\{1,s_N^2\}\right). P [ E ⊥ ∩ { ∣ Z ∣ ≤ 2 1 N min { s N , 1 } } ] ≥ 1 − 2 exp ( − c N ) − 2 exp ( − 8 1 N min { 1 , s N 2 } ) . On the above event, we have σ N ∣ Z ∣ ≤ 1 2 σ N N s N = 1 2 N λ α N \sigma_N|Z|\le\frac{1}{2}\sigma_N\sqrt{N}s_N=\frac{1}{2}N\lambda\alpha_N σ N ∣ Z ∣ ≤ 2 1 σ N N s N = 2 1 N λ α N (25) yields
1 2 N λ α N ≤ ⟨ u H , θ ⋆ ⟩ ≤ 3 2 N λ α N . \frac{1}{2}N\lambda\alpha_N\le\langle u_H,\theta^\star\rangle\le\frac{3}{2}N\lambda\alpha_N. 2 1 N λ α N ≤ ⟨ u H , θ ⋆ ⟩ ≤ 2 3 N λ α N . 1 4 ( N 2 λ 2 α N 2 + σ N 2 N ) ≤ ∥ u H ∥ 2 ≤ 9 4 ( N 2 λ 2 α N 2 + σ N 2 N ) . \frac{1}{4}\left(N^2\lambda^2\alpha_N^2+\sigma_N^2N\right)\leq\lVert u_H\rVert^2\leq\frac{9}{4}\left(N^2\lambda^2\alpha_N^2+\sigma_N^2N\right). 4 1 ( N 2 λ 2 α N 2 + σ N 2 N ) ≤ ∥ u H ∥ 2 ≤ 4 9 ( N 2 λ 2 α N 2 + σ N 2 N ) . In particular, u H ≠ 0 u_H\neq0 u H = 0 m ( u ^ H ) = ⟨ u H , θ ∗ ⟩ ∥ u H ∥ m(\widehat{u}_H)=\frac{\langle u_H,\theta^*\rangle}{\lVert u_H\rVert} m ( u H ) = ∥ u H ∥ ⟨ u H , θ ∗ ⟩
1 3 ⋅ s N 1 + s N 2 ≤ m ( u ^ H ) ≤ 3 ⋅ s N 1 + s N 2 . \frac{1}{3}\cdot\frac{s_N}{\sqrt{1+s_N^2}}\leq m(\widehat{u}_H)\leq3\cdot\frac{s_N}{\sqrt{1+s_N^2}}. 3 1 ⋅ 1 + s N 2 s N ≤ m ( u H ) ≤ 3 ⋅ 1 + s N 2 s N . Since s 1 + s 2 ≍ min { 1 , s } \frac{s}{\sqrt{1+s^2}}\asymp\min\{1,s\} 1 + s 2 s ≍ min { 1 , s } s > 0 s>0 s > 0 s N ≍ k λ N − k − 1 4 s_N\asymp_k\lambda N^{-\frac{k-1}{4}} s N ≍ k λ N − 4 k − 1 λ ≥ C k N k − 2 4 \lambda\geq C_kN^{\frac{k-2}{4}} λ ≥ C k N 4 k − 2 λ N − k − 1 4 ≥ C k N − 1 / 4 \lambda N^{-\frac{k-1}{4}}\geq C_kN^{-1/4} λ N − 4 k − 1 ≥ C k N − 1/4 N N N
Finally, we prove the norm bound. By a standard Gaussian tail bound, the event E ⊥ ∩ { Z ≥ − 1 2 N } \mathcal{E}_{\perp}\cap\{Z\geq-\frac{1}{2}\sqrt{N}\} E ⊥ ∩ { Z ≥ − 2 1 N } 1 − C e − c N 1-Ce^{-cN} 1 − C e − c N (25) :
∥ u H ∥ ≥ max { N λ α N + σ N Z , ∥ ζ ⊥ ∥ } ≥ max { N λ α N − 1 2 σ N N , 1 2 σ N N } ≥ 1 4 ( N λ α N + σ N N ) . \lVert u_H\rVert\geq\max\{N\lambda\alpha_N+\sigma_N Z,\lVert\zeta_{\perp}\rVert\}\geq\max\left\{N\lambda\alpha_N-\frac{1}{2}\sigma_N\sqrt{N},\frac{1}{2}\sigma_N\sqrt{N}\right\}\geq\frac{1}{4}\left(N\lambda\alpha_N+\sigma_N\sqrt{N}\right). ∥ u H ∥ ≥ max { N λ α N + σ N Z , ∥ ζ ⊥ ∥} ≥ max { N λ α N − 2 1 σ N N , 2 1 σ N N } ≥ 4 1 ( N λ α N + σ N N ) . where the last inequality holds since N λ α N − 1 2 σ N N ≥ 1 2 N λ α N N\lambda\alpha_N-\frac{1}{2}\sigma_N\sqrt{N}\geq\frac{1}{2}N\lambda\alpha_N N λ α N − 2 1 σ N N ≥ 2 1 N λ α N N λ α N ≥ σ N N N\lambda\alpha_N\geq\sigma_N\sqrt{N} N λ α N ≥ σ N N 1 2 σ N N ≥ 1 4 ( N λ α N + σ N N ) \frac{1}{2}\sigma_N\sqrt{N}\geq\frac{1}{4}(N\lambda\alpha_N+\sigma_N\sqrt{N}) 2 1 σ N N ≥ 4 1 ( N λ α N + σ N N )
Analysis of the Average Spherical Hessian
We now characterize the structure of the average spherical Hessian. In particular, we observe that for even k k k
Theorem 4.4 (Signal-Noise Decomposition of the Average Spherical Hessian). Let H H H (2) , and define the average spherical Hessian A H ≡ E π [ ∇ S N − 1 2 H ( x ) ] A_H\equiv\mathbb{E}_{\pi}[\nabla_{\mathbb{S}^{N-1}}^2H(x)] A H ≡ E π [ ∇ S N − 1 2 H ( x )]
If k k k A H = 0 A_H=0 A H = 0
If k k k
A H = λ γ N ( θ ∗ ( θ ∗ ) T − I N ) + W N , A_H=\lambda\gamma_N\left(\theta^*(\theta^*)^{\mathsf{T}}-\frac{I}{N}\right)+W_N, A H = λ γ N ( θ ∗ ( θ ∗ ) T − N I ) + W N , where γ N ≍ k N − k − 2 2 \gamma_N\asymp_k N^{-\frac{k-2}{2}} γ N ≍ k N − 2 k − 2 W N ≡ E π [ ∇ S N − 1 2 H 0 ( x ) ] W_N\equiv\mathbb{E}_{\pi}[\nabla_{\mathbb{S}^{N-1}}^2H_0(x)] W N ≡ E π [ ∇ S N − 1 2 H 0 ( x )]
P [ ∥ W N ∥ o p ≥ C k N − k − 2 4 ] ≤ exp ( − c N ) . \mathbb{P}\left[\lVert W_N\rVert_{\mathrm{op}}\geq C_kN^{-\frac{k-2}{4}}\right]\leq\exp(-cN). P [ ∥ W N ∥ op ≥ C k N − 4 k − 2 ] ≤ exp ( − c N ) . Proof. By Stein’s lemma (Lemma 3.1 ) and Equation (2) , we obtain
A H = E π [ ( x x T − I ) H ( x ) ] = A s i g + W N , A_H=\mathbb{E}_{\pi}[(xx^{\mathsf{T}}-I)H(x)]=A_{\mathrm{sig}}+W_N, A H = E π [( x x T − I ) H ( x )] = A sig + W N , A s i g = N λ E π [ ( x x T − I ) m ( x ) k ] , W N = E π [ ( x x T − I ) H 0 ( x ) ] . (26) A_{\mathrm{sig}}=N\lambda\mathbb{E}_{\pi}[(xx^{\mathsf{T}}-I)m(x)^k],\qquad W_N=\mathbb{E}_{\pi}[(xx^{\mathsf{T}}-I)H_0(x)]. \tag*{(26)} A sig = N λ E π [( x x T − I ) m ( x ) k ] , W N = E π [( x x T − I ) H 0 ( x )] . (26) If k k k H ( − x ) = − H ( x ) H(-x)=-H(x) H ( − x ) = − H ( x ) x x T − I xx^{\mathsf{T}}-I x x T − I x ↦ − x x\mapsto-x x ↦ − x π \pi π x ↦ − x x\mapsto-x x ↦ − x A H = 0 A_H=0 A H = 0
Henceforth, assume k k k (26) . Applying the last claim of Lemma 3.3 with w = θ ⋆ w=\theta^\star w = θ ⋆ ℓ = k \ell=k ℓ = k
N E π [ ( x x ⊤ − I ) m ( x ) k ] = N E π [ m ( x ) k ] [ N N + k ( I + k θ ⋆ ( θ ⋆ ) ⊤ ) − I ] = N E π [ m ( x ) k ] [ N k N + k θ ⋆ ( θ ⋆ ) ⊤ − k N + k I ] = γ N ( θ ⋆ ( θ ⋆ ) ⊤ − I N ) , (27) \begin{aligned}
N\mathbb{E}_{\pi}\left[(xx^\top-I)m(x)^k\right]
=N\mathbb{E}_{\pi}\left[m(x)^k\right]\left[\frac{N}{N+k}\left(I+k\theta^\star(\theta^\star)^\top\right)-I\right] \\
&=N\mathbb{E}_{\pi}\left[m(x)^k\right]\left[\frac{Nk}{N+k}\theta^\star(\theta^\star)^\top-\frac{k}{N+k}I\right] \\
&=\gamma_N\left(\theta^\star(\theta^\star)^\top-\frac{I}{N}\right), \tag*{(27)}
\end{aligned} N E π [ ( x x ⊤ − I ) m ( x ) k ] = N E π [ m ( x ) k ] [ N + k N ( I + k θ ⋆ ( θ ⋆ ) ⊤ ) − I ] = N E π [ m ( x ) k ] [ N + k N k θ ⋆ ( θ ⋆ ) ⊤ − N + k k I ] = γ N ( θ ⋆ ( θ ⋆ ) ⊤ − N I ) , (27) where γ N \gamma_N γ N
γ N ≔ k N 2 N + k E π [ m ( x ) k ] . \gamma_N\coloneqq\frac{kN^2}{N+k}\mathbb{E}_{\pi}\left[m(x)^k\right]. γ N : = N + k k N 2 E π [ m ( x ) k ] . Since k k k Lemma 3.3 implies that γ N ≍ k N − k − 2 2 \gamma_N\asymp_k N^{-\frac{k-2}{2}} γ N ≍ k N − 2 k − 2 (26) is A s i g = λ γ N ( θ ⋆ ( θ ⋆ ) ⊤ − I N ) A_{\mathrm{sig}}=\lambda\gamma_N\left(\theta^\star(\theta^\star)^\top-\frac{I}{N}\right) A sig = λ γ N ( θ ⋆ ( θ ⋆ ) ⊤ − N I )
We now analyze the noise term in Equation (26) . Since W N W_N W N G G G w ⊤ W N w w^\top W_Nw w ⊤ W N w w ∈ S N − 1 w\in\mathbb{S}^{N-1} w ∈ S N − 1 w ∈ S N − 1 w\in\mathbb{S}^{N-1} w ∈ S N − 1 w ⊤ W N w = E π [ ( ⟨ w , x ⟩ 2 − 1 ) H 0 ( x ) ] w^\top W_Nw=\mathbb{E}_{\pi}\left[(\langle w,x\rangle^2-1)H_0(x)\right] w ⊤ W N w = E π [ (⟨ w , x ⟩ 2 − 1 ) H 0 ( x ) ] (3) :
E G [ ( w ⊤ W N w ) 2 ] = E x , y ∼ i . i . d . π [ ( ⟨ w , x ⟩ 2 − 1 ) ( ⟨ w , y ⟩ 2 − 1 ) E G [ H 0 ( x ) H 0 ( y ) ] ] = N E x , y ∼ i . i . d . π [ ( ⟨ w , x ⟩ 2 − 1 ) ( ⟨ w , y ⟩ 2 − 1 ) ( ⟨ x , y ⟩ N ) k ] . (28) \begin{aligned}
\mathbb{E}_G\left[(w^\top W_Nw)^2\right]
=\mathbb{E}_{x,y\mathrel{\overset{\mathrm{i.i.d.}}{\sim}}\pi}\left[(\langle w,x\rangle^2-1)(\langle w,y\rangle^2-1)\mathbb{E}_G[H_0(x)H_0(y)]\right] \\
&=N\mathbb{E}_{x,y\mathrel{\overset{\mathrm{i.i.d.}}{\sim}}\pi}\left[(\langle w,x\rangle^2-1)(\langle w,y\rangle^2-1)\left(\frac{\langle x,y\rangle}{N}\right)^k\right]. \tag*{(28)}
\end{aligned} E G [ ( w ⊤ W N w ) 2 ] = E x , y ∼ i.i.d. π [ (⟨ w , x ⟩ 2 − 1 ) (⟨ w , y ⟩ 2 − 1 ) E G [ H 0 ( x ) H 0 ( y )] ] = N E x , y ∼ i.i.d. π [ (⟨ w , x ⟩ 2 − 1 ) (⟨ w , y ⟩ 2 − 1 ) ( N ⟨ x , y ⟩ ) k ] . (28) Observe that by O ( N , R ) O(N,\mathbb{R}) O ( N , R ) π \pi π (27) implies the following for any x ∈ S N x\in\mathbb{S}_N x ∈ S N y y y
E y ∼ π [ ( y y ⊤ − I ) ( ⟨ x , y ⟩ N ) k ] = γ N N 2 ( x x ⊤ − I ) . (29) \mathbb{E}_{y\sim\pi}\left[(yy^\top-I)\left(\frac{\langle x,y\rangle}{N}\right)^k\right]
=\frac{\gamma_N}{N^2}(xx^\top-I). \tag*{(29)} E y ∼ π [ ( y y ⊤ − I ) ( N ⟨ x , y ⟩ ) k ] = N 2 γ N ( x x ⊤ − I ) . (29) Using ⟨ w , y ⟩ 2 − 1 = w ⊤ ( y y ⊤ − I ) w \langle w,y\rangle^2-1=w^\top(yy^\top-I)w ⟨ w , y ⟩ 2 − 1 = w ⊤ ( y y ⊤ − I ) w Lemma 3.3 , we obtain the following from Equations (28) and (29) :
E G [ ( w ⊤ W N w ) 2 ] = N E x ∼ π [ ( ⟨ w , x ⟩ 2 − 1 ) w ⊤ E y ∼ π [ ( y y ⊤ − I ) ( ⟨ x , y ⟩ N ) k ] w ] = γ N N E π [ ( ⟨ w , x ⟩ 2 − 1 ) 2 ] = γ N N ( E π [ ⟨ w , x ⟩ 4 ] − 1 ) = 2 ( N − 1 ) γ N N ( N + 2 ) . \begin{aligned}
\mathbb{E}_G\left[(w^\top W_Nw)^2\right]
=N\mathbb{E}_{x\sim\pi}\left[(\langle w,x\rangle^2-1)w^\top\mathbb{E}_{y\sim\pi}\left[\left(yy^\top-I\right)\left(\frac{\langle x,y\rangle}{N}\right)^k\right]w\right] \\
&=\frac{\gamma_N}{N}\mathbb{E}_{\pi}\left[(\langle w,x\rangle^2-1)^2\right]
=\frac{\gamma_N}{N}\left(\mathbb{E}_{\pi}\left[\langle w,x\rangle^4\right]-1\right)
=\frac{2(N-1)\gamma_N}{N(N+2)}.
\end{aligned} E G [ ( w ⊤ W N w ) 2 ] = N E x ∼ π [ (⟨ w , x ⟩ 2 − 1 ) w ⊤ E y ∼ π [ ( y y ⊤ − I ) ( N ⟨ x , y ⟩ ) k ] w ] = N γ N E π [ (⟨ w , x ⟩ 2 − 1 ) 2 ] = N γ N ( E π [ ⟨ w , x ⟩ 4 ] − 1 ) = N ( N + 2 ) 2 ( N − 1 ) γ N . Thus, w ⊤ W N w w^\top W_Nw w ⊤ W N w 2 γ N N \frac{2\gamma_N}{N} N 2 γ N ε \varepsilon ε S N − 1 \mathbb{S}^{N-1} S N − 1
P [ ∥ W N ∥ o p ≥ 8 γ N ] ≤ e − N . \mathbb{P}\left[\lVert W_N\rVert_{\mathrm{op}}\ge8\sqrt{\gamma_N}\right]\le e^{-N}. P [ ∥ W N ∥ op ≥ 8 γ N ] ≤ e − N . Since γ N ≍ k N − k − 2 2 \gamma_N\asymp_k N^{-\frac{k-2}{2}} γ N ≍ k N − 2 k − 2
Ergodic Averages of High Temperature Langevin Dynamics
Our goal in this section is to establish the following recovery guarantee for ergodic averages of high temperature spherical Langevin dynamics.
Theorem 5.1 (Recovery by Ergodic Averaging for Tensor PCA). Let λ ≥ C k N k − 2 4 \lambda\ge C_kN^{\frac{k-2}{4}} λ ≥ C k N 4 k − 2 ( X ~ t ) t ≥ 0 (\widetilde{X}_t)_{t\ge0} ( X t ) t ≥ 0 H H H β 0 > 0 \beta_0>0 β 0 > 0 X ~ 0 ∼ π \widetilde{X}_0\sim\pi X 0 ∼ π
X ‾ T ≔ 1 T ∫ 0 T X ~ t d t , Σ ‾ T ≔ 1 T ∫ 0 T X ~ t X ~ t T d t . \overline{X}_T\coloneqq\frac{1}{T}\int_0^T\widetilde{X}_t\,\mathrm{d}t,\qquad\overline{\Sigma}_T\coloneqq\frac{1}{T}\int_0^T\widetilde{X}_t\widetilde{X}_t^{\mathsf{T}}\,\mathrm{d}t. X T : = T 1 ∫ 0 T X t d t , Σ T : = T 1 ∫ 0 T X t X t T d t . and write u H = E π [ ∇ S N H ] u_H=\mathbb{E}_{\pi}[\nabla_{\mathbb{S}^N}H] u H = E π [ ∇ S N H ] A H = E π [ ∇ S N 2 H ] A_H=\mathbb{E}_{\pi}[\nabla_{\mathbb{S}^N}^2H] A H = E π [ ∇ S N 2 H ] ϑ ∈ ( 0 , 1 ) \vartheta\in(0,1) ϑ ∈ ( 0 , 1 ) N N N 0 < β 0 ≲ k λ − 1 N − ⌊ k / 2 ⌋ − ϑ 0<\beta_0\lesssim_k\lambda^{-1}N^{-\lfloor k/2\rfloor-\vartheta} 0 < β 0 ≲ k λ − 1 N − ⌊ k /2 ⌋ − ϑ T a v ≳ k β 0 − 2 λ − 2 N 2 ⌊ k / 2 ⌋ + 2 ϑ T_{\mathrm{av}}\gtrsim_k\beta_0^{-2}\lambda^{-2}N^{2\lfloor k/2\rfloor+2\vartheta} T av ≳ k β 0 − 2 λ − 2 N 2 ⌊ k /2 ⌋ + 2 ϑ
If k k k 1 − exp ( − c k N ) 1-\exp(-c_kN) 1 − exp ( − c k N ) X ‾ T a v ≠ 0 \overline{X}_{T_{\mathrm{av}}}\ne0 X T av = 0
1 N ∥ X ‾ T a v − u ^ H ∥ 2 ≤ N − 2 ϑ min { 1 , λ 2 σ N 2 } , X ^ T a v = N X ‾ T a v ∥ X ‾ T a v ∥ . \frac{1}{N}\left\|\overline{X}_{T_{\mathrm{av}}}-\widehat{u}_H\right\|^2\le N^{-2\vartheta}\min\{1,\lambda^2\sigma_N^2\},\qquad\widehat{X}_{T_{\mathrm{av}}}=\frac{\sqrt{N}\,\overline{X}_{T_{\mathrm{av}}}}{\|\overline{X}_{T_{\mathrm{av}}}\|}. N 1 X T av − u H 2 ≤ N − 2 ϑ min { 1 , λ 2 σ N 2 } , X T av = ∥ X T av ∥ N X T av . where u ^ H \widehat{u}_H u H σ N ≍ k N k − 1 4 \sigma_N\asymp_kN^{\frac{k-1}{4}} σ N ≍ k N 4 k − 1 Theorem 4.1 .
If k k k v ^ \widehat{v} v Σ ‾ T a v \overline{\Sigma}_{T_{\mathrm{av}}} Σ T av 1 − exp ( − c k N ) 1-\exp(-c_kN) 1 − exp ( − c k N )
∣ m ( N v ^ ) ∣ ≥ 1 − C k ( N k − 2 2 λ 2 + N − 2 ϑ ) . \left|m(\sqrt{N}\widehat{v})\right|\ge1-C_k\left(\frac{N^{\frac{k-2}{2}}}{\lambda^2}+N^{-2\vartheta}\right). m ( N v ) ≥ 1 − C k ( λ 2 N 2 k − 2 + N − 2 ϑ ) . Theorem 5.1 , which is proved in § 5.4 , is obtained by specializing our results on ergodic averages of spherical Langevin dynamics for smooth potentials to tensor PCA. These results, presented in § 5.2 and § 5.3 , extend the framework of Part I [21 ] from Glauber dynamics on the hypercube to spherical Langevin dynamics: in both settings, we view the high temperature dynamics as a perturbation of its infinite temperature counterpart, and control the ergodic averages via their associated Green operators, as elucidated in § 5.1 . We believe that these results and their proof techniques, which require minor technical modifications for extending from the hypercube to the sphere, may be of independent interest.
Functional Analysis of Spherical Brownian Motion
We use ( S t ) t ≥ 0 (S_t)_{t\ge0} ( S t ) t ≥ 0 S N \mathcal{S}_N S N
d S t = − N − 1 2 N S t d t + Q S t d B t , S 0 ∈ S N . \mathrm{d}S_t=-\frac{N-1}{2N}S_t\,\mathrm{d}t+Q_{S_t}\,\mathrm{d}B_t,\qquad S_0\in\mathcal{S}_N. d S t = − 2 N N − 1 S t d t + Q S t d B t , S 0 ∈ S N . which corresponds to spherical Langevin dynamics with β = 0 \beta=0 β = 0 § 3.3 , S t ∈ S N S_t\in\mathcal{S}_N S t ∈ S N t ≥ 0 t\ge0 t ≥ 0 ( S t ) t ≥ 0 (S_t)_{t\ge0} ( S t ) t ≥ 0 π \pi π ( P t ) t ≥ 0 (P_t)_{t\ge0} ( P t ) t ≥ 0 P t f ( x ) = E [ f ( S t ) ∣ S 0 = x ] P_tf(x)=\mathbb{E}[f(S_t)\mid S_0=x] P t f ( x ) = E [ f ( S t ) ∣ S 0 = x ] f : S N → R f:\mathcal{S}_N\to\mathbb{R} f : S N → R (19) and Lemma 3.11 , the associated generator L \mathcal{L} L Γ \Gamma Γ f , g : S N → R f,g:\mathcal{S}_N\to\mathbb{R} f , g : S N → R
L f = 1 2 Δ S N f , Γ ( f , g ) = 1 2 ⟨ ∇ S N f , ∇ S N g ⟩ . \mathcal{L}f=\frac{1}{2}\Delta_{\mathcal{S}_N}f,\qquad\Gamma(f,g)=\frac{1}{2}\langle\nabla_{\mathcal{S}_N}f,\nabla_{\mathcal{S}_N}g\rangle. L f = 2 1 Δ S N f , Γ ( f , g ) = 2 1 ⟨ ∇ S N f , ∇ S N g ⟩ . An essential ingredient in our proofs is the following gradient estimate for the spherical Brownian semigroup, which is a corollary of the celebrated Bakry–Émery estimate for Markov diffusion semigroups on manifolds satisfying a curvature-dimension condition [12 Section 3.2.3] . Throughout, we write ρ N ≔ N − 2 2 N \rho_N\coloneqq\frac{N-2}{2N} ρ N : = 2 N N − 2 ρ N ≥ 1 6 \rho_N\ge\frac{1}{6} ρ N ≥ 6 1 N ≥ 3 N\ge3 N ≥ 3
Lemma 5.2 (Bakry–Émery Gradient Estimate for Spherical Brownian Motion). The following holds for every smooth f : S N → R m f:\mathcal{S}_N\to\mathbb{R}^m f : S N → R m x ∈ S N x\in\mathcal{S}_N x ∈ S N t ≥ 0 t\ge0 t ≥ 0
∥ ∇ S N ( P t f ) ( x ) ∥ o p ≤ e − ρ N t P t ( ∥ ∇ S N f ∥ o p ) ( x ) . \left\|\nabla_{\mathcal{S}_N}(P_tf)(x)\right\|_{\mathrm{op}}\le e^{-\rho_Nt}P_t\left(\left\|\nabla_{\mathcal{S}_N}f\right\|_{\mathrm{op}}\right)(x). ∥ ∇ S N ( P t f ) ( x ) ∥ op ≤ e − ρ N t P t ( ∥ ∇ S N f ∥ op ) ( x ) . Moreover, if max x ∈ S N ∥ ∇ S N f ( x ) ∥ o p ≤ L \max_{x\in\mathcal{S}_N}\left\|\nabla_{\mathcal{S}_N}f(x)\right\|_{\mathrm{op}}\le L max x ∈ S N ∥ ∇ S N f ( x ) ∥ op ≤ L
max x ∈ S N ∥ ∇ S N ( P t f ) ( x ) ∥ o p ≤ e − ρ N t L ∀ t ≥ 0. \max_{x\in\mathcal{S}_N}\left\|\nabla_{\mathcal{S}_N}(P_tf)(x)\right\|_{\mathrm{op}}\le e^{-\rho_Nt}L\qquad\forall\ t\ge0. x ∈ S N max ∥ ∇ S N ( P t f ) ( x ) ∥ op ≤ e − ρ N t L ∀ t ≥ 0. Proof. Since S N \mathcal{S}_N S N N − 1 N-1 N − 1 N \sqrt{N} N N − 2 N \frac{N-2}{N} N N − 2 [12 Section 2.2.4] . Consequently, L = 1 2 Δ S N \mathcal{L}=\frac{1}{2}\Delta_{\mathcal{S}_N} L = 2 1 Δ S N CD ( ρ N , ∞ ) \operatorname{CD}(\rho_N,\infty) CD ( ρ N , ∞ ) [12 Section C.6] . Fix u ∈ S m − 1 u\in\mathbb{S}^{m-1} u ∈ S m − 1 g ( x ) = ⟨ f ( x ) , u ⟩ g(x)=\langle f(x),u\rangle g ( x ) = ⟨ f ( x ) , u ⟩ P t P_t P t P t g = ⟨ P t f , u ⟩ P_tg=\langle P_tf,u\rangle P t g = ⟨ P t f , u ⟩ ∇ S N ( P t g ) = ∇ S N ( P t f ) u \nabla_{\mathcal{S}_N}(P_tg)=\nabla_{\mathcal{S}_N}(P_tf)u ∇ S N ( P t g ) = ∇ S N ( P t f ) u ∇ S N g = ( ∇ S N f ) u \nabla_{\mathcal{S}_N}g=(\nabla_{\mathcal{S}_N}f)u ∇ S N g = ( ∇ S N f ) u [12 Theorem 3.2.4] applied to g g g Γ ( g , g ) = 1 2 ∥ ∇ S N g ∥ 2 \Gamma(g,g)=\frac{1}{2}\|\nabla_{\mathcal{S}_N}g\|^2 Γ ( g , g ) = 2 1 ∥ ∇ S N g ∥ 2
∥ ∇ S N ( P t f ) ( x ) u ∥ = ∥ ∇ S N ( P t g ) ( x ) ∥ ≤ e − ρ N t P t ( ∥ ( ∇ S N f ) u ∥ ) ( x ) ≤ e − ρ N t P t ( ∥ ∇ S N f ∥ o p ) ( x ) . \left\|\nabla_{\mathcal{S}_N}(P_tf)(x)u\right\|=\left\|\nabla_{\mathcal{S}_N}(P_tg)(x)\right\|\le e^{-\rho_Nt}P_t\left(\left\|(\nabla_{\mathcal{S}_N}f)u\right\|\right)(x)\le e^{-\rho_Nt}P_t\left(\left\|\nabla_{\mathcal{S}_N}f\right\|_{\mathrm{op}}\right)(x). ∥ ∇ S N ( P t f ) ( x ) u ∥ = ∥ ∇ S N ( P t g ) ( x ) ∥ ≤ e − ρ N t P t ( ∥ ( ∇ S N f ) u ∥ ) ( x ) ≤ e − ρ N t P t ( ∥ ∇ S N f ∥ op ) ( x ) . Since u ∈ S m − 1 u\in\mathbb{S}^{m-1} u ∈ S m − 1
Finally, suppose ∥ ∇ S N f ( z ) ∥ o p ≤ L \left\|\nabla_{\mathcal{S}_N}f(z)\right\|_{\mathrm{op}}\le L ∥ ∇ S N f ( z ) ∥ op ≤ L z ∈ S N z\in\mathcal{S}_N z ∈ S N P t h ( x ) = E [ h ( S t ) ∣ S 0 = x ] P_th(x)=\mathbb{E}[h(S_t)\mid S_0=x] P t h ( x ) = E [ h ( S t ) ∣ S 0 = x ] h : S N → R h:\mathcal{S}_N\to\mathbb{R} h : S N → R x ∈ S N x\in\mathcal{S}_N x ∈ S N t ≥ 0 t\ge0 t ≥ 0
∥ ∇ S N ( P t f ) ( x ) ∥ o p ≤ e − ρ N t E [ ∥ ∇ S N f ( S t ) ∥ o p ∣ S 0 = x ] ≤ e − ρ N t L , \left\|\nabla_{\mathcal{S}_N}(P_tf)(x)\right\|_{\mathrm{op}}\le e^{-\rho_Nt}\mathbb{E}\left[\left\|\nabla_{\mathcal{S}_N}f(S_t)\right\|_{\mathrm{op}}\mid S_0=x\right]\le e^{-\rho_Nt}L, ∥ ∇ S N ( P t f ) ( x ) ∥ op ≤ e − ρ N t E [ ∥ ∇ S N f ( S t ) ∥ op ∣ S 0 = x ] ≤ e − ρ N t L , which proves the second claim.
We now introduce the Green operator (or restricted inverse of − L -\mathcal{L} − L L 2 ( π ) = { f : S N → R : E π [ ∣ f ∣ 2 ] < ∞ } L^{2}(\pi)=\{f:\mathcal{S}_{N}\to\mathbb{R}:\mathbb{E}_{\pi}[|f|^{2}]<\infty\} L 2 ( π ) = { f : S N → R : E π [ ∣ f ∣ 2 ] < ∞ } S N \mathcal{S}_{N} S N L 0 2 ( π ) = { f ∈ L 2 ( π ) : E π [ f ] = 0 } L_{0}^{2}(\pi)=\{f\in L^{2}(\pi):\mathbb{E}_{\pi}[f]=0\} L 0 2 ( π ) = { f ∈ L 2 ( π ) : E π [ f ] = 0 } S N \mathcal{S}_{N} S N R : L 0 2 ( π ) → L 0 2 ( π ) \mathcal{R}:L_{0}^{2}(\pi)\to L_{0}^{2}(\pi) R : L 0 2 ( π ) → L 0 2 ( π )
R f = ∫ 0 ∞ P t f d t ∀ f ∈ L 0 2 ( π ) . (30) \mathcal{R}f=\int_{0}^{\infty}P_{t}f\,dt \qquad\forall f\in L_{0}^{2}(\pi). \tag*{(30)} R f = ∫ 0 ∞ P t f d t ∀ f ∈ L 0 2 ( π ) . (30) For centered vector and matrix valued observables, R \mathcal{R} R R \mathcal{R} R
Lemma 5.3 (Properties of the Green operator). The Green operator R \mathcal{R} R L 0 2 ( π ) L_{0}^{2}(\pi) L 0 2 ( π ) f ∈ L 0 2 ( π ) f\in L_{0}^{2}(\pi) f ∈ L 0 2 ( π ) R f \mathcal{R}f R f
L ( R f ) = − f . (31) \mathcal{L}(\mathcal{R}f)=-f. \tag*{(31)} L ( R f ) = − f . (31) Furthermore, max x ∈ S N ∥ ∇ S N ( R f ) ( x ) ∥ ≤ ρ N − 1 max x ∈ S N ∥ ∇ S N f ( x ) ∥ \max_{x\in\mathcal{S}_{N}}\lVert\nabla_{\mathcal{S}_{N}}(\mathcal{R}f)(x)\rVert\leq\rho_{N}^{-1}\max_{x\in\mathcal{S}_{N}}\lVert\nabla_{\mathcal{S}_{N}}f(x)\rVert max x ∈ S N ∥ ∇ S N ( R f ) ( x )∥ ≤ ρ N − 1 max x ∈ S N ∥ ∇ S N f ( x )∥ f ∈ L 0 2 ( π ) f\in L_{0}^{2}(\pi) f ∈ L 0 2 ( π )
Proof. As noted in the proof of Lemma 5.2, L \mathcal{L} L CD ( ρ N , ∞ ) \operatorname{CD}(\rho_{N},\infty) CD ( ρ N , ∞ ) CD ( ρ N , ∞ ) \operatorname{CD}(\rho_{N},\infty) CD ( ρ N , ∞ ) f ∈ L 0 2 ( π ) f\in L_{0}^{2}(\pi) f ∈ L 0 2 ( π ) t ≥ 0 t\geq0 t ≥ 0 [12 Proposition 4.8.1 and Theorem 4.2.5] $:
∥ P t f ∥ L 2 ( π ) ≤ e − ρ N t ∥ f ∥ L 2 ( π ) . \lVert P_{t}f\rVert_{L^{2}(\pi)}\leq e^{-\rho_{N}t}\lVert f\rVert_{L^{2}(\pi)}. ∥ P t f ∥ L 2 ( π ) ≤ e − ρ N t ∥ f ∥ L 2 ( π ) . Integrating the above inequality over t ≥ 0 t\geq0 t ≥ 0 f ∈ L 0 2 ( π ) f\in L_{0}^{2}(\pi) f ∈ L 0 2 ( π )
∥ R f ∥ L 2 ( π ) ≤ ∫ 0 ∞ ∥ P t f ∥ L 2 ( π ) d t ≤ ∥ f ∥ L 2 ( π ) ∫ 0 ∞ e − ρ N t d t = ρ N − 1 ∥ f ∥ L 2 ( π ) . \lVert\mathcal{R}f\rVert_{L^{2}(\pi)}\leq\int_{0}^{\infty}\lVert P_{t}f\rVert_{L^{2}(\pi)}\,dt\leq\lVert f\rVert_{L^{2}(\pi)}\int_{0}^{\infty}e^{-\rho_{N}t}\,dt=\rho_{N}^{-1}\lVert f\rVert_{L^{2}(\pi)}. ∥ R f ∥ L 2 ( π ) ≤ ∫ 0 ∞ ∥ P t f ∥ L 2 ( π ) d t ≤ ∥ f ∥ L 2 ( π ) ∫ 0 ∞ e − ρ N t d t = ρ N − 1 ∥ f ∥ L 2 ( π ) . Hence, R \mathcal{R} R L 0 2 ( π ) L_{0}^{2}(\pi) L 0 2 ( π )
Now fix a smooth f ∈ L 0 2 ( π ) f\in L_{0}^{2}(\pi) f ∈ L 0 2 ( π ) (20) , we obtain
L ( R f ) = lim T → ∞ ∫ 0 T L ( P t f ) d t = lim T → ∞ ∫ 0 T d d t P t f d t = lim T → ∞ P T f − f = E π [ f ] − f = − f , \mathcal{L}(\mathcal{R}f)=\lim_{T\to\infty}\int_{0}^{T}\mathcal{L}(P_{t}f)\,dt=\lim_{T\to\infty}\int_{0}^{T}\frac{d}{dt}P_{t}f\,dt=\lim_{T\to\infty}P_{T}f-f=\mathbb{E}_{\pi}[f]-f=-f, L ( R f ) = T → ∞ lim ∫ 0 T L ( P t f ) d t = T → ∞ lim ∫ 0 T d t d P t f d t = T → ∞ lim P T f − f = E π [ f ] − f = − f , where we use the fact that π \pi π P t P_{t} P t f ∈ L 0 2 ( π ) f\in L_{0}^{2}(\pi) f ∈ L 0 2 ( π ) (31) . Finally, we infer the following from Lemma 5.2 for every x ∈ S N x\in\mathcal{S}_{N} x ∈ S N
∥ ∇ S N ( R f ) ( x ) ∥ ≤ ∫ 0 ∞ ∥ ∇ S N ( P t f ) ( x ) ∥ d t ≤ max z ∈ S N ∥ ∇ S N f ( z ) ∥ ∫ 0 ∞ e − ρ N t d t = ρ N − 1 max z ∈ S N ∥ ∇ S N f ( z ) ∥ , \lVert\nabla_{\mathcal{S}_{N} }(\mathcal{R}f)(x)\rVert\leq\int_{0}^{\infty}\lVert\nabla_{\mathcal{S}_{N}}(P_{t}f)(x)\rVert\,dt\leq\max_{z\in\mathcal{S}_{N}}\lVert\nabla_{\mathcal{S}_{N}}f(z)\rVert\int_{0}^{\infty}e^{-\rho_{N}t}\,dt=\rho_{N}^{-1}\max_{z\in\mathcal{S}_{N}}\lVert\nabla_{\mathcal{S}_{N}}f(z)\rVert, ∥ ∇ S N ( R f ) ( x )∥ ≤ ∫ 0 ∞ ∥ ∇ S N ( P t f ) ( x )∥ d t ≤ z ∈ S N max ∥ ∇ S N f ( z )∥ ∫ 0 ∞ e − ρ N t d t = ρ N − 1 z ∈ S N max ∥ ∇ S N f ( z )∥ , which proves the gradient bound.
Remark 5.4. By Equation (31) , we may interpret R \mathcal{R} R − L -\mathcal{L} − L L 0 2 ( π ) L_{0}^{2}(\pi) L 0 2 ( π ) [31 Ebe25] (see also [17 Section 4.1] for connections to Stein’s method of exchangeable pairs). In fact, Part I of our work [21 ] develops an analogous framework for the simple random walk on the hypercube. In subsequent sections, we shall use the Green operator to analyze ergodic averages along high-temperature spherical Langevin dynamics for Hamiltonians with bounded G 2 \mathcal{G}^{2} G 2
Recovering the Average Spherical Gradient
We now show that the ergodic average of high-temperature spherical Langevin dynamics approximates a scalar multiple of the average spherical gradient under π \pi π V : S N → R V:\mathcal{S}_N\to\mathbb{R} V : S N → R ( Z t ) t ≥ 0 (Z_t)_{t\geq0} ( Z t ) t ≥ 0 β 0 ≥ 0 \beta_0\geq0 β 0 ≥ 0 Z 0 ∼ π Z_0\sim\pi Z 0 ∼ π T > 0 T>0 T > 0
Z ˉ T = 1 T ∫ 0 T Z t d t . (32) \bar{Z}_T=\frac{1}{T}\int_0^T Z_t\,\mathrm{d}t. \tag*{(32)} Z ˉ T = T 1 ∫ 0 T Z t d t . (32) In § 2.3 , we argued heuristically that, for V = H V=H V = H Z ˉ T \bar{Z}_T Z ˉ T μ V , β 0 \mu_{V,\beta_0} μ V , β 0 T → ∞ T\to\infty T → ∞ 2 β 0 κ N E π [ ∇ S N V ] + O ( β 0 2 ) \frac{2\beta_0}{\kappa_N}\mathbb{E}_{\pi}[\nabla_{\mathcal{S}_N}V]+O(\beta_0^2) κ N 2 β 0 E π [ ∇ S N V ] + O ( β 0 2 ) Theorem 5.5 below.
Theorem 5.5 (Recovery of the Average Spherical Gradient). Suppose that ∥ V ∥ G 2 ≤ B N \|V\|_{\mathcal{G}_2}\leq BN ∥ V ∥ G 2 ≤ B N for some B > 0 B>0 B > 0 For every T ≥ 1 T\geq1 T ≥ 1 and δ ∈ ( 0 , 1 ) \delta\in(0,1) δ ∈ ( 0 , 1 ) the following holds with probability at least 1 − δ 1-\delta 1 − δ
∥ Z ˉ T − 2 β 0 κ N E π [ ∇ S N V ] ∥ ≲ ( 1 + β 0 B ) N + log ( 1 / δ ) T + β 0 2 B 2 N . \left\|\bar{Z}_T-\frac{2\beta_0}{\kappa_N}\mathbb{E}_{\pi}[\nabla_{\mathcal{S}_N}V]\right\|\lesssim(1+\beta_0B)\sqrt{\frac{N+\log(1/\delta)}{T}}+\beta_0^2B^2\sqrt{N}. Z ˉ T − κ N 2 β 0 E π [ ∇ S N V ] ≲ ( 1 + β 0 B ) T N + log ( 1/ δ ) + β 0 2 B 2 N . Proof. The proof proceeds in two steps. The first step combines the Poisson equation from Lemma 5.3 with Equation (19) to prove that the ergodic average of ∇ S N V ( Z t ) \nabla_{\mathcal{S}_N}V(Z_t) ∇ S N V ( Z t ) E π [ ∇ S N V ] \mathbb{E}_{\pi}[\nabla_{\mathcal{S}_N}V] E π [ ∇ S N V ] Z t Z_t Z t 2 β 0 κ N ∇ S N V ( Z t ) \frac{2\beta_0}{\kappa_N}\nabla_{\mathcal{S}_N}V(Z_t) κ N 2 β 0 ∇ S N V ( Z t )
Define F ( x ) = ∇ S N V ( x ) − E π [ ∇ S N V ] F(x)=\nabla_{\mathcal{S}_N}V(x)-\mathbb{E}_{\pi}[\nabla_{\mathcal{S}_N}V] F ( x ) = ∇ S N V ( x ) − E π [ ∇ S N V ] Φ = R F \Phi=\mathcal{R}F Φ = R F w ∈ S N − 1 w\in\mathcal{S}_N^{-1} w ∈ S N − 1 f w = ⟨ F , w ⟩ f_w=\langle F,w\rangle f w = ⟨ F , w ⟩ π \pi π f w ( x ) = ⟨ Q x ∇ V ( x ) , w ⟩ − ⟨ E π [ ∇ S N V ] , w ⟩ f_w(x)=\langle Q_x\nabla V(x),w\rangle-\langle\mathbb{E}_{\pi}[\nabla_{\mathcal{S}_N}V],w\rangle f w ( x ) = ⟨ Q x ∇ V ( x ) , w ⟩ − ⟨ E π [ ∇ S N V ] , w ⟩ (8) yields
∇ S N f w ( x ) = ∇ S N 2 V ( x ) w − ⟨ x , w ⟩ N ∇ S N V ( x ) . \nabla_{\mathcal{S}_N}f_w(x)=\nabla_{\mathcal{S}_N}^2V(x)w-\frac{\langle x,w\rangle}{N}\nabla_{\mathcal{S}_N}V(x). ∇ S N f w ( x ) = ∇ S N 2 V ( x ) w − N ⟨ x , w ⟩ ∇ S N V ( x ) . Since ∥ V ∥ G 2 ≤ B N \|V\|_{\mathcal{G}_2}\leq BN ∥ V ∥ G 2 ≤ B N ∥ ∇ S N 2 V ( x ) ∥ o p ≤ B \|\nabla_{\mathcal{S}_N}^2V(x)\|_{\mathrm{op}}\leq B ∥ ∇ S N 2 V ( x ) ∥ op ≤ B ∥ ∇ S N V ( x ) ∥ ≤ B N \|\nabla_{\mathcal{S}_N}V(x)\|\leq B\sqrt{N} ∥ ∇ S N V ( x ) ∥ ≤ B N x ∈ S N x\in\mathcal{S}_N x ∈ S N ∥ ∇ S N f w ( x ) ∥ ≤ 2 B \|\nabla_{\mathcal{S}_N}f_w(x)\|\leq2B ∥ ∇ S N f w ( x ) ∥ ≤ 2 B ⟨ Φ , w ⟩ = R f w \langle\Phi,w\rangle=\mathcal{R}f_w ⟨ Φ , w ⟩ = R f w R \mathcal{R} R Lemma 5.3 and ρ N ≥ 1 6 \rho_N\geq\frac{1}{6} ρ N ≥ 6 1
max x ∈ S N ∥ ∇ S N Φ ( x ) ∥ o p = max x ∈ S N sup w ∈ S N − 1 ∥ ∇ S N ( ⟨ Φ , w ⟩ ) ( x ) ∥ ≤ 2 B ρ N ≤ 12 B . (33) \max_{x\in\mathcal{S}_N}\|\nabla_{\mathcal{S}_N}\Phi(x)\|_{\mathrm{op}}=\max_{x\in\mathcal{S}_N}\sup_{w\in\mathcal{S}_N^{-1}}\|\nabla_{\mathcal{S}_N}(\langle\Phi,w\rangle)(x)\|\leq\frac{2B}{\rho_N}\leq12B. \tag*{(33)} x ∈ S N max ∥ ∇ S N Φ ( x ) ∥ op = x ∈ S N max w ∈ S N − 1 sup ∥ ∇ S N (⟨ Φ , w ⟩) ( x ) ∥ ≤ ρ N 2 B ≤ 12 B . (33) The Poisson equation Equation (31) from Lemma 5.3 implies L Φ = 1 2 Δ S N Φ = − F \mathcal{L}\Phi=\frac{1}{2}\Delta_{\mathcal{S}_N}\Phi=-F L Φ = 2 1 Δ S N Φ = − F (19) , we obtain
L V , β 0 Φ ( x ) = − F ( x ) + β 0 ∇ S N Φ ( x ) ⊤ ∇ S N V ( x ) . (34) \mathcal{L}_{V,\beta_0}\Phi(x)=-F(x)+\beta_0\nabla_{\mathcal{S}_N}\Phi(x)^{\top}\nabla_{\mathcal{S}_N}V(x). \tag*{(34)} L V , β 0 Φ ( x ) = − F ( x ) + β 0 ∇ S N Φ ( x ) ⊤ ∇ S N V ( x ) . (34) Let M t Φ M_t^\Phi M t Φ Φ \Phi Φ ( Z t ) t ≥ 0 (Z_t)_{t\geq0} ( Z t ) t ≥ 0 Lemma 3.11 ,
M T Φ = Φ ( Z T ) − Φ ( Z 0 ) − ∫ 0 T L V , β 0 Φ ( Z t ) d t . M_T^\Phi=\Phi(Z_T)-\Phi(Z_0)-\int_0^T\mathcal{L}_{V,\beta_0}\Phi(Z_t)\,\mathrm{d}t. M T Φ = Φ ( Z T ) − Φ ( Z 0 ) − ∫ 0 T L V , β 0 Φ ( Z t ) d t . By rearranging terms and using Equation (34) , we obtain
1 T ∫ 0 T ∇ S N V ( Z t ) d t − E π [ ∇ S N V ] = Φ ( Z 0 ) − Φ ( Z T ) T + M T Φ T + β 0 T ∫ 0 T ∇ S N Φ ( Z t ) T ∇ S N V ( Z t ) d t . (35) \frac{1}{T}\int_{0}^{T}\nabla_{\mathcal{S}_{N}}V(Z_t)\,\mathrm{d}t-\mathbb{E}_{\pi}[\nabla_{\mathcal{S}_{N}}V]=\frac{\Phi(Z_0)-\Phi(Z_T)}{T}+\frac{M_T^{\Phi}}{T}+\frac{\beta_0}{T}\int_{0}^{T}\nabla_{\mathcal{S}_{N}}\Phi(Z_t)^{\mathsf{T}}\nabla_{\mathcal{S}_{N}}V(Z_t)\,\mathrm{d}t. \tag*{(35)} T 1 ∫ 0 T ∇ S N V ( Z t ) d t − E π [ ∇ S N V ] = T Φ ( Z 0 ) − Φ ( Z T ) + T M T Φ + T β 0 ∫ 0 T ∇ S N Φ ( Z t ) T ∇ S N V ( Z t ) d t . (35) We control the three terms separately. Since ∥ Z 0 − Z T ∥ ≤ 2 N \lVert Z_0-Z_T\rVert\leq2\sqrt{N} ∥ Z 0 − Z T ∥ ≤ 2 N T ≥ 1 T\geq1 T ≥ 1 (9) and (33) :
∥ Φ ( Z 0 ) − Φ ( Z T ) T ∥ ≲ B N T ≲ B N T . (36) \left\lVert\frac{\Phi(Z_0)-\Phi(Z_T)}{T}\right\rVert\lesssim\frac{B\sqrt{N}}{T}\lesssim B\sqrt{\frac{N}{T}}. \tag*{(36)} T Φ ( Z 0 ) − Φ ( Z T ) ≲ T B N ≲ B T N . (36) For the drift term, since ∥ ∇ S N V ( x ) ∥ ≤ B N \lVert\nabla_{\mathcal{S}_{N}}V(x)\rVert\leq B\sqrt{N} ∥ ∇ S N V ( x )∥ ≤ B N (33) for every x ∈ S N x\in\mathcal{S}_{N} x ∈ S N
∥ β 0 ∇ S N Φ ( x ) T ∇ S N V ( x ) ∥ ≤ β 0 ∥ ∇ S N Φ ( x ) ∥ o p ∥ ∇ S N V ( x ) ∥ ≤ 12 β 0 B 2 N . (37) \left\lVert\beta_0\nabla_{\mathcal{S}_{N}}\Phi(x)^{\mathsf{T}}\nabla_{\mathcal{S}_{N}}V(x)\right\rVert\leq\beta_0\lVert\nabla_{\mathcal{S}_{N}}\Phi(x)\rVert_{\mathrm{op}}\lVert\nabla_{\mathcal{S}_{N}}V(x)\rVert\leq12\beta_0B^2\sqrt{N}. \tag*{(37)} β 0 ∇ S N Φ ( x ) T ∇ S N V ( x ) ≤ β 0 ∥ ∇ S N Φ ( x ) ∥ op ∥ ∇ S N V ( x )∥ ≤ 12 β 0 B 2 N . (37) It remains to control the martingale term. By Equation (33) , max x ∈ S N ∥ ∇ S N ⟨ Φ , w ⟩ ( x ) ∥ ≤ 12 B \max_{x\in\mathcal{S}_{N}}\lVert\nabla_{\mathcal{S}_{N}}\langle\Phi,w\rangle(x)\rVert\leq12B max x ∈ S N ∥ ∇ S N ⟨ Φ , w ⟩ ( x )∥ ≤ 12 B w ∈ S N − 1 w\in\mathcal{S}^{N-1} w ∈ S N − 1 Lemma 3.13 with m = N m=N m = N n = 1 n=1 n = 1 1 − δ / 2 1-\delta/2 1 − δ /2
∥ M T Φ ∥ T ≲ B N + log ( 1 / δ ) T . (38) \frac{\lVert M_T^{\Phi}\rVert}{T}\lesssim B\sqrt{\frac{N+\log(1/\delta)}{T}}. \tag*{(38)} T ∥ M T Φ ∥ ≲ B T N + log ( 1/ δ ) . (38) Substituting Equations (36) to (38) into Equation (35) , we obtain the following with probability at least 1 − δ / 2 1-\delta/2 1 − δ /2
∥ 1 T ∫ 0 T ∇ S N V ( Z t ) d t − E π [ ∇ S N V ] ∥ ≲ B N + log ( 1 / δ ) T + β 0 B 2 N . (39) \left\lVert\frac{1}{T}\int_{0}^{T}\nabla_{\mathcal{S}_{N}}V(Z_t)\,\mathrm{d}t-\mathbb{E}_{\pi}[\nabla_{\mathcal{S}_{N}}V]\right\rVert\lesssim B\sqrt{\frac{N+\log(1/\delta)}{T}}+\beta_0B^2\sqrt{N}. \tag*{(39)} T 1 ∫ 0 T ∇ S N V ( Z t ) d t − E π [ ∇ S N V ] ≲ B T N + log ( 1/ δ ) + β 0 B 2 N . (39) It remains to compare the respective ergodic averages of Z t Z_t Z t ∇ S N V ( Z t ) \nabla_{\mathcal{S}_{N}}V(Z_t) ∇ S N V ( Z t ) (8) to the coordinate functions, we obtain ∇ S N x = Q x \nabla_{\mathcal{S}_{N}}x=Q_x ∇ S N x = Q x Δ S N x = − κ N x \Delta_{\mathcal{S}_{N}}x=-\kappa_Nx Δ S N x = − κ N x Q x ∇ S N V ( x ) = ∇ S N V ( x ) Q_x\nabla_{\mathcal{S}_{N}}V(x)=\nabla_{\mathcal{S}_{N}}V(x) Q x ∇ S N V ( x ) = ∇ S N V ( x ) (19) :
L V , β 0 x = − κ N 2 x + β 0 ∇ S N V ( x ) . \mathcal{L}_{V,\beta_0}x=-\frac{\kappa_N}{2}x+\beta_0\nabla_{\mathcal{S}_{N}}V(x). L V , β 0 x = − 2 κ N x + β 0 ∇ S N V ( x ) . Let M t x M_t^x M t x Lemma 3.11 ,
M T x = Z T − Z 0 − ∫ 0 T L V , β 0 x ( Z t ) d t = Z T − Z 0 − ∫ 0 T [ β 0 ∇ S N V ( Z t ) − κ N 2 Z t ] d t . M_T^x=Z_T-Z_0-\int_{0}^{T}\mathcal{L}_{V,\beta_0}x(Z_t)\,\mathrm{d}t=Z_T-Z_0-\int_{0}^{T}\left[\beta_0\nabla_{\mathcal{S}_{N}}V(Z_t)-\frac{\kappa_N}{2}Z_t\right]\mathrm{d}t. M T x = Z T − Z 0 − ∫ 0 T L V , β 0 x ( Z t ) d t = Z T − Z 0 − ∫ 0 T [ β 0 ∇ S N V ( Z t ) − 2 κ N Z t ] d t . By rearranging terms and using Equation (32) , we obtain
Z ˉ T − 2 β 0 κ N ⋅ 1 T ∫ 0 T ∇ S N V ( Z t ) d t = 2 κ N [ Z 0 − Z T T + M T x T ] . (40) \bar{Z}_T-\frac{2\beta_0}{\kappa_N}\cdot\frac{1}{T}\int_{0}^{T}\nabla_{\mathcal{S}_{N}}V(Z_t)\,\mathrm{d}t=\frac{2}{\kappa_N}\left[\frac{Z_0-Z_T}{T}+\frac{M_T^x}{T}\right]. \tag*{(40)} Z ˉ T − κ N 2 β 0 ⋅ T 1 ∫ 0 T ∇ S N V ( Z t ) d t = κ N 2 [ T Z 0 − Z T + T M T x ] . (40) Since Z 0 , Z T ∈ S N Z_0,Z_T\in\mathcal{S}_{N} Z 0 , Z T ∈ S N κ N ≥ 2 3 \kappa_N\geq\frac{2}{3} κ N ≥ 3 2 T ≥ 1 T\geq1 T ≥ 1 6 N / T ≤ 6 N / T 6\sqrt{N}/T\leq6\sqrt{N/T} 6 N / T ≤ 6 N / T ∇ S N ⟨ x , w ⟩ = Q x w \nabla_{\mathcal{S}_{N}}\langle x,w\rangle=Q_xw ∇ S N ⟨ x , w ⟩ = Q x w 1 1 1 w ∈ S N − 1 w\in\mathcal{S}^{N-1} w ∈ S N − 1 Lemma 3.13 with m = N m=N m = N n = 1 n=1 n = 1 1 − δ / 2 1-\delta/2 1 − δ /2
∥ M T x ∥ T ≲ N + log ( 1 / δ ) T . \frac{\lVert M_T^x\rVert}{T}\lesssim\sqrt{\frac{N+\log(1/\delta)}{T}}. T ∥ M T x ∥ ≲ T N + log ( 1/ δ ) . Combining these estimates with Equation (40) , we conclude that, with probability at least 1 − δ / 2 1-\delta/2 1 − δ /2
∥ Z ˉ T − 2 β 0 κ N ⋅ 1 T ∫ 0 T ∇ S N V ( Z t ) d t ∥ ≲ N + log ( 1 / δ ) T . (41) \left\|\bar{Z}_T-\frac{2\beta_0}{\kappa_N}\cdot\frac{1}{T}\int_0^T\nabla_{\mathcal{S}_N}V(Z_t)\,\mathrm{d}t\right\|\lesssim\sqrt{\frac{N+\log(1/\delta)}{T}}. \tag*{(41)} Z ˉ T − κ N 2 β 0 ⋅ T 1 ∫ 0 T ∇ S N V ( Z t ) d t ≲ T N + log ( 1/ δ ) . (41) The proof is concluded by combining Equations (39) and (41) via the triangle inequality and a union bound, since 2 β 0 κ N ≤ 3 β 0 \frac{2\beta_0}{\kappa_N}\leq3\beta_0 κ N 2 β 0 ≤ 3 β 0
Recovering the Average Spherical Hessian
Borrowing the notation from § 5.2 , we use ( Z t ) t ≥ 0 (Z_t)_{t\geq0} ( Z t ) t ≥ 0 V : S N → R V:\mathcal{S}_N\to\mathbb{R} V : S N → R β 0 ≥ 0 \beta_0\geq0 β 0 ≥ 0 Z 0 ∼ π Z_0\sim\pi Z 0 ∼ π T > 0 T>0 T > 0
Z ˉ T ( 2 ) ≔ 1 T ∫ 0 T Z t Z t ⊤ d t . \bar{Z}_T^{(2)}\coloneqq\frac{1}{T}\int_0^T Z_tZ_t^\top\,\mathrm{d}t. Z ˉ T ( 2 ) : = T 1 ∫ 0 T Z t Z t ⊤ d t . The next theorem establishes a concentration guarantee for the ergodic second moment for potentials with bounded Sobolev norm.
Theorem 5.6 (Recovery of the Average Spherical Hessian). Suppose that ∥ V ∥ G 2 ≤ B N \|V\|_{\mathcal{G}_2}\leq BN ∥ V ∥ G 2 ≤ B N B > 0 B>0 B > 0 T ≥ 1 T\geq1 T ≥ 1 δ ∈ ( 0 , 1 ) \delta\in(0,1) δ ∈ ( 0 , 1 ) 1 − δ 1-\delta 1 − δ
∥ Z ˉ T ( 2 ) − I − 2 β 0 E π [ ∇ S N 2 V ] ∥ o p ≲ ( 1 + β 0 B ) N N + log ( 1 / δ ) T + β 0 2 B 2 N . \left\|\bar{Z}_T^{(2)}-I-2\beta_0\mathbb{E}_{\pi}\left[\nabla_{\mathcal{S}_N}^{2}V\right]\right\|_{\mathrm{op}}\lesssim(1+\beta_0B)\sqrt{N}\sqrt{\frac{N+\log(1/\delta)}{T}}+\beta_0^2B^2N. Z ˉ T ( 2 ) − I − 2 β 0 E π [ ∇ S N 2 V ] op ≲ ( 1 + β 0 B ) N T N + log ( 1/ δ ) + β 0 2 B 2 N . Proof. Define the matrix-valued observable J V : S N → Sym N J_V:\mathcal{S}_N\to\operatorname{Sym}_N J V : S N → Sym N
J V ( x ) ≔ 1 2 ( x ∇ S N V ( x ) ⊤ + ∇ S N V ( x ) x ⊤ ) . (42) J_V(x)\coloneqq\frac{1}{2}\left(x\nabla_{\mathcal{S}_N}V(x)^\top+\nabla_{\mathcal{S}_N}V(x)x^\top\right). \tag*{(42)} J V ( x ) : = 2 1 ( x ∇ S N V ( x ) ⊤ + ∇ S N V ( x ) x ⊤ ) . (42) By Stein’s lemma (Lemma 3.1 ), E π [ J V ] = E π [ ∇ S N 2 V ] \mathbb{E}_{\pi}[J_V]=\mathbb{E}_{\pi}[\nabla_{\mathcal{S}_N}^{2}V] E π [ J V ] = E π [ ∇ S N 2 V ] Theorem 5.5 , we first combine the Poisson equation from Lemma 5.3 with Equation (19) to show that the ergodic average of J V ( Z t ) J_V(Z_t) J V ( Z t ) E π [ ∇ S N 2 V ] \mathbb{E}_{\pi}[\nabla_{\mathcal{S}_N}^{2}V] E π [ ∇ S N 2 V ] x x ⊤ − I xx^\top-I x x ⊤ − I Z ˉ T ( 2 ) − I \bar{Z}_T^{(2)}-I Z ˉ T ( 2 ) − I 2 β 0 J V ( Z t ) 2\beta_0J_V(Z_t) 2 β 0 J V ( Z t )
Define F ( x ) = J V ( x ) − E π [ ∇ S N 2 V ] F(x)=J_V(x)-\mathbb{E}_{\pi}[\nabla_{\mathcal{S}_N}^{2}V] F ( x ) = J V ( x ) − E π [ ∇ S N 2 V ] Φ = R F \Phi=\mathcal{R}F Φ = R F Sym N \operatorname{Sym}_N Sym N w ∈ S N N − 1 w\in\mathcal{S}_N^{N-1} w ∈ S N N − 1 f w = w ⊤ F w f_w=w^\top Fw f w = w ⊤ F w π \pi π w ⊤ J V ( x ) w = ⟨ x , w ⟩ ⟨ Q x ∇ V ( x ) , w ⟩ w^\top J_V(x)w=\langle x,w\rangle\langle Q_x\nabla V(x),w\rangle w ⊤ J V ( x ) w = ⟨ x , w ⟩ ⟨ Q x ∇ V ( x ) , w ⟩ (8) and the product rule yields
∇ S N f w ( x ) = ⟨ ∇ S N V ( x ) , w ⟩ Q x w + ⟨ x , w ⟩ ( ∇ S N 2 V ( x ) w − ⟨ x , w ⟩ N ∇ S N V ( x ) ) . \nabla_{\mathcal{S}_N}f_w(x)=\langle\nabla_{\mathcal{S}_N}V(x),w\rangle Q_xw+\langle x,w\rangle\left(\nabla_{\mathcal{S}_N}^{2}V(x)w-\frac{\langle x,w\rangle}{N}\nabla_{\mathcal{S}_N}V(x)\right). ∇ S N f w ( x ) = ⟨ ∇ S N V ( x ) , w ⟩ Q x w + ⟨ x , w ⟩ ( ∇ S N 2 V ( x ) w − N ⟨ x , w ⟩ ∇ S N V ( x ) ) . Since ∥ V ∥ G 2 ≤ B N \|V\|_{\mathcal{G}_2}\leq BN ∥ V ∥ G 2 ≤ B N ∥ ∇ S N 2 V ( x ) ∥ o p ≤ B \|\nabla_{\mathcal{S}_N}^{2}V(x)\|_{\mathrm{op}}\leq B ∥ ∇ S N 2 V ( x ) ∥ op ≤ B ∥ ∇ S N V ( x ) ∥ ≤ B N \|\nabla_{\mathcal{S}_N}V(x)\|\leq B\sqrt{N} ∥ ∇ S N V ( x ) ∥ ≤ B N x ∈ S N x\in\mathcal{S}_N x ∈ S N ∥ ∇ S N f w ( x ) ∥ ≤ 3 B N \|\nabla_{\mathcal{S}_N}f_w(x)\|\leq3B\sqrt{N} ∥ ∇ S N f w ( x ) ∥ ≤ 3 B N w ⊤ Φ w = R f w w^\top\Phi w=\mathcal{R}f_w w ⊤ Φ w = R f w R \mathcal{R} R Lemma 5.3 and ρ N ≥ 1 6 \rho_N\geq\frac{1}{6} ρ N ≥ 6 1
max x ∈ S N sup w ∈ S N N − 1 ∥ ∇ S N ( w ⊤ Φ w ) ( x ) ∥ ≤ 3 B N ρ N ≤ 18 B N . (43) \max_{x\in\mathcal{S}_N}\sup_{w\in\mathcal{S}_N^{N-1}}\|\nabla_{\mathcal{S}_N}(w^\top\Phi w)(x)\|\leq\frac{3B\sqrt{N}}{\rho_N}\leq18B\sqrt{N}. \tag*{(43)} x ∈ S N max w ∈ S N N − 1 sup ∥ ∇ S N ( w ⊤ Φ w ) ( x ) ∥ ≤ ρ N 3 B N ≤ 18 B N . (43) The Poisson equation Equation (31) from Lemma 5.3 implies L Φ = 1 2 Δ S N Φ = − F \mathcal{L}\Phi= \frac{1}{2}\Delta_{\mathcal{S}_N}\Phi= -F L Φ = 2 1 Δ S N Φ = − F (19) , we obtain
L V , β 0 Φ ( x ) = − F ( x ) + β 0 D Φ ( x ) [ ∇ S N V ( x ) ] , (44) \mathcal{L}_{V,\beta_0}\Phi(x)=-F(x)+\beta_0D\Phi(x)[\nabla_{\mathcal{S}_N}V(x)], \tag*{(44)} L V , β 0 Φ ( x ) = − F ( x ) + β 0 D Φ ( x ) [ ∇ S N V ( x )] , (44) where D Φ ( x ) [ v ] ∈ Sym N D\Phi(x)[v]\in\operatorname{Sym}_N D Φ ( x ) [ v ] ∈ Sym N Φ \Phi Φ x x x v ∈ T x S N v\in T_x\mathcal{S}_N v ∈ T x S N ⟨ ∇ S N Φ i j ( x ) , v ⟩ \langle\nabla_{\mathcal{S}_N}\Phi_{ij}(x),v\rangle ⟨ ∇ S N Φ ij ( x ) , v ⟩ M t Φ M_t^\Phi M t Φ Φ \Phi Φ ( Z t ) t ≥ 0 (Z_t)_{t\ge0} ( Z t ) t ≥ 0 Lemma 3.11 ,
M T Φ = Φ ( Z T ) − Φ ( Z 0 ) − ∫ 0 T L V , β 0 Φ ( Z t ) d t . M_T^\Phi=\Phi(Z_T)-\Phi(Z_0)-\int_0^T\mathcal{L}_{V,\beta_0}\Phi(Z_t)\,dt. M T Φ = Φ ( Z T ) − Φ ( Z 0 ) − ∫ 0 T L V , β 0 Φ ( Z t ) d t . By rearranging terms and using Equation (44) , we obtain
1 T ∫ 0 T J V ( Z t ) d t − E π [ ∇ S N 2 V ] = Φ ( Z 0 ) − Φ ( Z T ) T + M T Φ T + β 0 T ∫ 0 T D Φ ( Z t ) [ ∇ S N V ( Z t ) ] d t . (45) \frac{1}{T}\int_0^T J_V(Z_t)\,dt-\mathbb{E}_{\pi}[\nabla_{\mathcal{S}_N}^2V]=\frac{\Phi(Z_0)-\Phi(Z_T)}{T}+\frac{M_T^\Phi}{T}+\frac{\beta_0}{T}\int_0^T D\Phi(Z_t)[\nabla_{\mathcal{S}_N}V(Z_t)]\,dt. \tag*{(45)} T 1 ∫ 0 T J V ( Z t ) d t − E π [ ∇ S N 2 V ] = T Φ ( Z 0 ) − Φ ( Z T ) + T M T Φ + T β 0 ∫ 0 T D Φ ( Z t ) [ ∇ S N V ( Z t )] d t . (45) We control the three terms separately. Since Φ \Phi Φ Sym N \operatorname{Sym}_N Sym N ∥ Φ ( Z 0 ) − Φ ( Z T ) ∥ o p = sup w ∈ S N − 1 ∣ w ⊤ Φ ( Z 0 ) w − w ⊤ Φ ( Z T ) w ∣ \|\Phi(Z_0)-\Phi(Z_T)\|_{\mathrm{op}}=\sup_{w\in\mathcal{S}^{N-1}}|w^\top\Phi(Z_0)w-w^\top\Phi(Z_T)w| ∥Φ ( Z 0 ) − Φ ( Z T ) ∥ op = sup w ∈ S N − 1 ∣ w ⊤ Φ ( Z 0 ) w − w ⊤ Φ ( Z T ) w ∣ ∥ Z 0 − Z T ∥ ≤ 2 N \|Z_0-Z_T\|\le2\sqrt{N} ∥ Z 0 − Z T ∥ ≤ 2 N T ≥ 1 T\ge1 T ≥ 1 (9) and (43) :
∥ Φ ( Z 0 ) − Φ ( Z T ) T ∥ o p ≲ B N T ≲ B N N T . (46) \left\|\frac{\Phi(Z_0)-\Phi(Z_T)}{T}\right\|_{\mathrm{op}}\lesssim\frac{BN}{T}\lesssim B\sqrt{N}\sqrt{\frac{N}{T}}. \tag*{(46)} T Φ ( Z 0 ) − Φ ( Z T ) op ≲ T B N ≲ B N T N . (46) For the drift term, observe that w ⊤ D Φ ( x ) [ ∇ S N V ( x ) ] w = ⟨ ∇ S N ( w ⊤ Φ w ) ( x ) , ∇ S N V ( x ) ⟩ w^\top D\Phi(x)[\nabla_{\mathcal{S}_N}V(x)]w=\langle\nabla_{\mathcal{S}_N}(w^\top\Phi w)(x),\nabla_{\mathcal{S}_N}V(x)\rangle w ⊤ D Φ ( x ) [ ∇ S N V ( x )] w = ⟨ ∇ S N ( w ⊤ Φ w ) ( x ) , ∇ S N V ( x )⟩ w ∈ S N − 1 w\in\mathcal{S}^{N-1} w ∈ S N − 1 ∥ ∇ S N V ( x ) ∥ ≤ B N \|\nabla_{\mathcal{S}_N}V(x)\|\le B\sqrt{N} ∥ ∇ S N V ( x ) ∥ ≤ B N (43) for every x ∈ S N x\in\mathcal{S}_N x ∈ S N
∥ β 0 D Φ ( x ) [ ∇ S N V ( x ) ] ∥ o p = β 0 sup w ∈ S N − 1 ∣ ⟨ ∇ S N ( w ⊤ Φ w ) ( x ) , ∇ S N V ( x ) ⟩ ∣ ≤ 18 β 0 B 2 N . (47) \|\beta_0D\Phi(x)[\nabla_{\mathcal{S}_N}V(x)]\|_{\mathrm{op}}=\beta_0\sup_{w\in\mathcal{S}^{N-1}}\left|\langle\nabla_{\mathcal{S}_N}(w^\top\Phi w)(x),\nabla_{\mathcal{S}_N}V(x)\rangle\right|\le18\beta_0B^2N. \tag*{(47)} ∥ β 0 D Φ ( x ) [ ∇ S N V ( x )] ∥ op = β 0 w ∈ S N − 1 sup ⟨ ∇ S N ( w ⊤ Φ w ) ( x ) , ∇ S N V ( x )⟩ ≤ 18 β 0 B 2 N . (47) It remains to control the martingale term. By Equation (43) and Remark 3.14, applying Lemma 3.13 with m = n = N m=n=N m = n = N A = 18 B N A=18B\sqrt{N} A = 18 B N 1 − δ / 2 1-\delta/2 1 − δ /2
∥ M T Φ ∥ o p T ≲ B N N + log ( 1 / δ ) T . (48) \frac{\|M_T^\Phi\|_{\mathrm{op}}}{T}\lesssim B\sqrt{N}\sqrt{\frac{N+\log(1/\delta)}{T}}. \tag*{(48)} T ∥ M T Φ ∥ op ≲ B N T N + log ( 1/ δ ) . (48) Substituting Equations (46) to (48) into Equation (45) , we obtain the following with probability at least 1 − δ / 2 1-\delta/2 1 − δ /2
∥ 1 T ∫ 0 T J V ( Z t ) d t − E π [ ∇ S N 2 V ] ∥ o p ≲ B N N + log ( 1 / δ ) T + β 0 B 2 N . (49) \left\|\frac{1}{T}\int_0^T J_V(Z_t)\,dt-\mathbb{E}_{\pi}[\nabla_{\mathcal{S}_N}^2V]\right\|_{\mathrm{op}}\lesssim B\sqrt{N}\sqrt{\frac{N+\log(1/\delta)}{T}}+\beta_0B^2N. \tag*{(49)} T 1 ∫ 0 T J V ( Z t ) d t − E π [ ∇ S N 2 V ] op ≲ B N T N + log ( 1/ δ ) + β 0 B 2 N . (49) It remains to compare Z ˉ ( 2 ) − I \bar{Z}^{(2)}-I Z ˉ ( 2 ) − I 2 β 0 J V ( Z t ) 2\beta_0J_V(Z_t) 2 β 0 J V ( Z t ) K ( x ) = x x ⊤ − I K(x)=xx^\top-I K ( x ) = x x ⊤ − I (8) to each entry of x x ⊤ xx^\top x x ⊤ Q x ∇ S N V ( x ) = ∇ S N V ( x ) Q_x\nabla_{\mathcal{S}_N}V(x)=\nabla_{\mathcal{S}_N}V(x) Q x ∇ S N V ( x ) = ∇ S N V ( x ) (42) :
1 2 Δ S N K ( x ) = I − x x ⊤ = − K ( x ) , D K ( x ) [ ∇ S N V ( x ) ] = x ∇ S N V ( x ) ⊤ + ∇ S N V ( x ) x ⊤ = 2 J V ( x ) . \frac{1}{2}\Delta_{\mathcal{S}_N}K(x)=I-xx^\top=-K(x),\qquad DK(x)[\nabla_{\mathcal{S}_N}V(x)]=x\nabla_{\mathcal{S}_N}V(x)^\top+\nabla_{\mathcal{S}_N}V(x)x^\top=2J_V(x). 2 1 Δ S N K ( x ) = I − x x ⊤ = − K ( x ) , D K ( x ) [ ∇ S N V ( x )] = x ∇ S N V ( x ) ⊤ + ∇ S N V ( x ) x ⊤ = 2 J V ( x ) . Combining the above identities with Equation (19) , where D K ( x ) [ v ] ∈ Sym N DK(x)[v] \in\operatorname{Sym}_{N} D K ( x ) [ v ] ∈ Sym N K K K x x x v ∈ T x S N v \in T_x\mathcal{S}_{N} v ∈ T x S N
L V , β 0 K ( x ) = 1 2 Δ S N K ( x ) + β 0 D K ( x ) [ ∇ S N V ( x ) ] = − K ( x ) + 2 β 0 J V ( x ) . (50) \mathcal{L}_{V,\beta_0}K(x)=\frac{1}{2}\Delta_{\mathcal{S}_{N}}K(x)+\beta_0DK(x)[\nabla_{\mathcal{S}_{N}}V(x)]=-K(x)+2\beta_0J_V(x). \tag*{(50)} L V , β 0 K ( x ) = 2 1 Δ S N K ( x ) + β 0 D K ( x ) [ ∇ S N V ( x )] = − K ( x ) + 2 β 0 J V ( x ) . (50) Let M t K M_t^K M t K (50) and Lemma 3.11 ,
M T K = K ( Z T ) − K ( Z 0 ) − ∫ 0 T L V , β 0 K ( Z t ) d t = K ( Z T ) − K ( Z 0 ) − ∫ 0 T [ 2 β 0 J V ( Z t ) − K ( Z t ) ] d t . M_T^K=K(Z_T)-K(Z_0)-\int_0^T\mathcal{L}_{V,\beta_0}K(Z_t)\,\mathrm{d}t=K(Z_T)-K(Z_0)-\int_0^T[2\beta_0J_V(Z_t)-K(Z_t)]\,\mathrm{d}t. M T K = K ( Z T ) − K ( Z 0 ) − ∫ 0 T L V , β 0 K ( Z t ) d t = K ( Z T ) − K ( Z 0 ) − ∫ 0 T [ 2 β 0 J V ( Z t ) − K ( Z t )] d t . By rearranging terms and using 1 T ∫ 0 T K ( Z t ) d t = Z ˉ T ( 2 ) − I \frac{1}{T}\int_0^T K(Z_t)\,\mathrm{d}t=\bar{Z}_T^{(2)}-I T 1 ∫ 0 T K ( Z t ) d t = Z ˉ T ( 2 ) − I
Z ˉ T ( 2 ) − I − 2 β 0 T ∫ 0 T J V ( Z t ) d t = K ( Z 0 ) − K ( Z T ) T + M T K T . (51) \bar{Z}_T^{(2)}-I-\frac{2\beta_0}{T}\int_0^T J_V(Z_t)\,\mathrm{d}t=\frac{K(Z_0)-K(Z_T)}{T}+\frac{M_T^K}{T}. \tag*{(51)} Z ˉ T ( 2 ) − I − T 2 β 0 ∫ 0 T J V ( Z t ) d t = T K ( Z 0 ) − K ( Z T ) + T M T K . (51) Since ∥ K ( x ) ∥ o p ≤ N \lVert K(x)\rVert_{\mathrm{op}}\leq N ∥ K ( x ) ∥ op ≤ N x ∈ S N x\in\mathcal{S}_N x ∈ S N T ≥ 1 T\geq1 T ≥ 1 2 N / T 2N/T 2 N / T w ∈ S N − 1 w\in\mathcal{S}^{N-1} w ∈ S N − 1 ∇ S N ( w ⊤ K w ) ( x ) = 2 ⟨ x , w ⟩ Q x w \nabla_{\mathcal{S}_N}(w^\top Kw)(x)=2\langle x,w\rangle Q_xw ∇ S N ( w ⊤ K w ) ( x ) = 2 ⟨ x , w ⟩ Q x w 2 N 2\sqrt{N} 2 N Lemma 3.13 with m = n = N m=n=N m = n = N 1 − δ / 2 1-\delta/2 1 − δ /2
∥ M T K ∥ o p T ≲ N N + log ( 1 / δ ) T . \frac{\lVert M_T^K\rVert_{\mathrm{op}}}{T}\lesssim\sqrt{N}\sqrt{\frac{N+\log(1/\delta)}{T}}. T ∥ M T K ∥ op ≲ N T N + log ( 1/ δ ) . Combining these estimates with Equation (51) , we conclude that, with probability at least 1 − δ / 2 1-\delta/2 1 − δ /2
∥ Z ˉ T ( 2 ) − I − 2 β 0 T ∫ 0 T J V ( Z t ) d t ∥ o p ≲ N N + log ( 1 / δ ) T . (52) \left\lVert\bar{Z}_T^{(2)}-I-\frac{2\beta_0}{T}\int_0^T J_V(Z_t)\,\mathrm{d}t\right\rVert_{\mathrm{op}}\lesssim\sqrt{N}\sqrt{\frac{N+\log(1/\delta)}{T}}. \tag*{(52)} Z ˉ T ( 2 ) − I − T 2 β 0 ∫ 0 T J V ( Z t ) d t op ≲ N T N + log ( 1/ δ ) . (52) The proof is concluded by combining Equations (49) and (52) via the triangle inequality and a union bound.
Proof of Theorem 5.1
Proof of Theorem 5.1 .
Let E = { ∥ H ∥ G 2 ≤ C k λ N } \mathcal{E}=\{\lVert H\rVert_{\mathcal{G}_2}\leq C_k\lambda N\} E = {∥ H ∥ G 2 ≤ C k λ N } λ ≥ 1 \lambda\geq1 λ ≥ 1 P [ E ] ≥ 1 − exp ( − c k N ) \mathbb{P}[\mathcal{E}]\geq1-\exp(-c_kN) P [ E ] ≥ 1 − exp ( − c k N ) T a v ≥ 1 T_{\mathrm{av}}\geq1 T av ≥ 1 H H H E \mathcal{E} E Theorems 5.5 and 5.6 with V = H V=H V = H B ≍ k λ B\asymp_k\lambda B ≍ k λ δ = exp ( − N ) \delta=\exp(-N) δ = exp ( − N ) κ N ≤ 1 \kappa_N\leq1 κ N ≤ 1 E \mathcal{E} E 1 − exp ( − c k N ) 1-\exp(-c_kN) 1 − exp ( − c k N )
∥ κ N 2 β 0 X ˉ T a v − u H ∥ ≤ ε N , ∥ Σ ˉ T a v 2 β 0 − I − A H ∥ o p ≤ N ε N , \left\lVert\frac{\kappa_N}{2\beta_0}\bar{X}T_{\mathrm{av}}-u_H\right\rVert\leq\varepsilon_N,\qquad
\left\lVert\frac{\bar{\Sigma}_{T_{\mathrm{av}}}}{2\beta_0}-I-A_H\right\rVert_{\mathrm{op}}\leq\sqrt{N}\varepsilon_N, 2 β 0 κ N X ˉ T av − u H ≤ ε N , 2 β 0 Σ ˉ T av − I − A H op ≤ N ε N , ε N ≔ C k ( β 0 − 1 + λ ) [ N T a v + β 0 λ 2 N ] . (53) \varepsilon_N\coloneqq C_k\left(\beta_0^{-1}+\lambda\right)\left[\sqrt{\frac{N}{T_{\mathrm{av}}}}+\beta_0\lambda^2\sqrt{N}\right]. \tag*{(53)} ε N : = C k ( β 0 − 1 + λ ) [ T av N + β 0 λ 2 N ] . (53) Since λ ≤ β 0 − 1 \lambda\leq\beta_0^{-1} λ ≤ β 0 − 1 β 0 \beta_0 β 0 T a v T_{\mathrm{av}} T av
ε N ≤ 2 C k [ β 0 − 1 N T a v + β 0 λ 2 N ] ≤ c k λ N 1 / 2 − ⌊ k / 2 ⌋ − ϑ . (54) \varepsilon_N\leq2C_k\left[\beta_0^{-1}\sqrt{\frac{N}{T_{\mathrm{av}}}}+\beta_0\lambda^2\sqrt{N}\right]\leq c_k\lambda N^{1/2-\lfloor k/2\rfloor-\vartheta}. \tag*{(54)} ε N ≤ 2 C k [ β 0 − 1 T av N + β 0 λ 2 N ] ≤ c k λ N 1/2 − ⌊ k /2 ⌋ − ϑ . (54) Here c k > 0 c_k > 0 c k > 0 β 0 \beta_0 β 0 T a v T_{\mathrm{av}} T av
For odd k k k ⌊ k / 2 ⌋ = k − 1 2 \lfloor k/2 \rfloor= \frac{k-1}{2} ⌊ k /2 ⌋ = 2 k − 1 (54) we have ε N ≤ c k λ N − k − 2 2 − ϑ \varepsilon_N \le c_k\lambda N^{-\frac{k-2}{2}-\vartheta} ε N ≤ c k λ N − 2 k − 2 − ϑ 1 − C exp ( − c N ) 1-C\exp(-cN) 1 − C exp ( − c N )
∥ u H ∥ ≥ c ( N λ α N + σ N N ) . (55) \lVert u_H\rVert\ge c(N\lambda\alpha_N+\sigma_N\sqrt{N}). \tag*{(55)} ∥ u H ∥ ≥ c ( N λ α N + σ N N ) . (55) where α N \alpha_N α N σ N \sigma_N σ N Theorem 4.1 . Henceforth, we assume that both Equations (53) and (55) hold, which occurs with probability at least 1 − exp ( − c k N ) 1-\exp(-c_kN) 1 − exp ( − c k N )
Combining Equations (54) and (55) with α N ≍ k N − k / 2 \alpha_N \asymp_k N^{-k/2} α N ≍ k N − k /2 σ N 2 = κ N N α N \sigma_N^2=\kappa_N\sqrt{N}\alpha_N σ N 2 = κ N N α N
ε N ∥ u H ∥ ≤ c k λ N − k − 2 2 − ϑ c ( N λ α N + σ N N ) ≲ k c k N − ϑ N λ α N N λ α N + σ N N ≤ c k N − ϑ min { 1 , N λ α N σ N N } = c k N − ϑ min { 1 , λ σ N κ N } . (56) \frac{\varepsilon_N}{\lVert u_H\rVert}
\le\frac{c_k\lambda N^{-\frac{k-2}{2}-\vartheta}}{c(N\lambda\alpha_N+\sigma_N\sqrt{N})}
\lesssim_k \frac{c_kN^{-\vartheta}N\lambda\alpha_N}{N\lambda\alpha_N+\sigma_N\sqrt{N}}
\le c_kN^{-\vartheta}\min\left\{1,\frac{N\lambda\alpha_N}{\sigma_N\sqrt{N}}\right\}
=c_kN^{-\vartheta}\min\left\{1,\frac{\lambda\sigma_N}{\kappa_N}\right\}. \tag*{(56)} ∥ u H ∥ ε N ≤ c ( N λ α N + σ N N ) c k λ N − 2 k − 2 − ϑ ≲ k N λ α N + σ N N c k N − ϑ N λ α N ≤ c k N − ϑ min { 1 , σ N N N λ α N } = c k N − ϑ min { 1 , κ N λ σ N } . (56) where the implicit constant does not depend on c k c_k c k c k c_k c k ε N < ∥ u H ∥ \varepsilon_N < \lVert u_H\rVert ε N < ∥ u H ∥ X ^ T a v ≠ 0 \widehat{X}_{T_{\mathrm{av}}}\ne0 X T av = 0 (53) . Now, applying Lemma 3.4 to κ N 2 β 0 X ^ T a v \frac{\kappa_N}{2\beta_0}\widehat{X}_{T_{\mathrm{av}}} 2 β 0 κ N X T av u H u_H u H (53) and (56) :
1 N ∥ X ^ T a v − u ^ H ∥ = ∥ X ^ T a v ∥ X ^ T a v ∥ − u H ∥ u H ∥ ∥ ≤ 2 ε N ∥ u H ∥ ≲ k c k N − ϑ min { 1 , λ σ N κ N } . \frac{1}{\sqrt{N}}\left\lVert\widehat{X}_{T_{\mathrm{av}}}-\widehat{u}_H\right\rVert
=\left\lVert\frac{\widehat{X}_{T_{\mathrm{av}}}}{\lVert\widehat{X}_{T_{\mathrm{av}}}\rVert}-\frac{u_H}{\lVert u_H\rVert}\right\rVert
\le\frac{2\varepsilon_N}{\lVert u_H\rVert}
\lesssim_k c_kN^{-\vartheta}\min\left\{1,\frac{\lambda\sigma_N}{\kappa_N}\right\}. N 1 X T av − u H = ∥ X T av ∥ X T av − ∥ u H ∥ u H ≤ ∥ u H ∥ 2 ε N ≲ k c k N − ϑ min { 1 , κ N λ σ N } . Since κ N ≥ 2 3 \kappa_N\ge\frac{2}{3} κ N ≥ 3 2 c k c_k c k k k k
For even k k k Theorem 4.4 that
A H = A s i g + W N , A s i g = λ γ N ( θ ⋆ ( θ ⋆ ) T − I N ) , γ N ≍ k N k − 2 2 , (57) A_H=A_{\mathrm{sig}}+W_N,\qquad A_{\mathrm{sig}}=\lambda\gamma_N\left(\theta^\star(\theta^\star)^\mathsf{T}-\frac{I}{N}\right),\qquad\gamma_N\asymp_k N^{\frac{k-2}{2}}, \tag*{(57)} A H = A sig + W N , A sig = λ γ N ( θ ⋆ ( θ ⋆ ) T − N I ) , γ N ≍ k N 2 k − 2 , (57) where ∥ W N ∥ o p ≤ C k N k − 2 4 \lVert W_N\rVert_{\mathrm{op}}\le C_kN^{\frac{k-2}{4}} ∥ W N ∥ op ≤ C k N 4 k − 2 1 − exp ( − c N ) 1-\exp(-cN) 1 − exp ( − c N ) ⌊ k / 2 ⌋ = k 2 \lfloor k/2\rfloor=\frac{k}{2} ⌊ k /2 ⌋ = 2 k (54) implies N ε N ≤ c k λ N − k − 2 2 − ϑ \sqrt{N}\varepsilon_N\le c_k\lambda N^{-\frac{k-2}{2}-\vartheta} N ε N ≤ c k λ N − 2 k − 2 − ϑ (53) and (57) and taking a union bound, we obtain the following with probability at least 1 − exp ( − c k N ) 1-\exp(-c_kN) 1 − exp ( − c k N )
∥ Σ ˉ T a v − ( I + 2 β 0 A s i g ) ∥ o p 2 β 0 λ γ N ≤ N ε N + ∥ W N ∥ o p λ γ N ≲ k N − ϑ + N k − 2 4 λ . (58) \frac{\left\lVert\bar{\Sigma}_{T_{\mathrm{av}}}-(I+2\beta_0A_{\mathrm{sig}})\right\rVert_{\mathrm{op}}}{2\beta_0\lambda\gamma_N}
\le\frac{\sqrt{N}\varepsilon_N+\lVert W_N\rVert_{\mathrm{op}}}{\lambda\gamma_N}
\lesssim_k N^{-\vartheta}+\frac{N^{\frac{k-2}{4}}}{\lambda}. \tag*{(58)} 2 β 0 λ γ N Σ ˉ T av − ( I + 2 β 0 A sig ) op ≤ λ γ N N ε N + ∥ W N ∥ op ≲ k N − ϑ + λ N 4 k − 2 . (58) The matrix I + 2 β 0 A s i g I+2\beta_0A_{\mathrm{sig}} I + 2 β 0 A sig θ ⋆ \theta^\star θ ⋆ 2 β 0 λ γ N 2\beta_0\lambda\gamma_N 2 β 0 λ γ N [48 Theorem 4.5.5] to Σ ˉ T a v \bar{\Sigma}_{T_{\mathrm{av}}} Σ ˉ T av I + 2 β 0 A s i g I+2\beta_0A_{\mathrm{sig}} I + 2 β 0 A sig (58) , we obtain
min s ∈ { ± 1 } ∥ v ^ − s θ ⋆ ∥ ≲ ∥ Σ ˉ T a v − ( I + 2 β 0 A s i g ) ∥ o p 2 β 0 λ γ N ≲ k N − ϑ + N k − 2 4 λ . \min_{s\in\{\pm1\}}\lVert\widehat{v}-s\theta^\star\rVert
\lesssim\frac{\left\lVert\bar{\Sigma}_{T_{\mathrm{av}}}-(I+2\beta_0A_{\mathrm{sig}})\right\rVert_{\mathrm{op}}}{2\beta_0\lambda\gamma_N}
\lesssim_k N^{-\vartheta}+\frac{N^{\frac{k-2}{4}}}{\lambda}. s ∈ { ± 1 } min ∥ v − s θ ⋆ ∥ ≲ 2 β 0 λ γ N Σ ˉ T av − ( I + 2 β 0 A sig ) op ≲ k N − ϑ + λ N 4 k − 2 . Since m ( N v ^ ) = ⟨ θ ⋆ , v ^ ⟩ m(\sqrt{N}\widehat{v})=\langle\theta^\star,\widehat{v}\rangle m ( N v ) = ⟨ θ ⋆ , v ⟩ ∥ v ^ − s θ ⋆ ∥ 2 = 2 − 2 s ⟨ θ ⋆ , v ^ ⟩ \lVert\widehat{v}-s\theta^\star\rVert^2=2-2s\langle\theta^\star,\widehat{v}\rangle ∥ v − s θ ⋆ ∥ 2 = 2 − 2 s ⟨ θ ⋆ , v ⟩ s ∈ { ± 1 } s\in\{\pm1\} s ∈ { ± 1 }
1 − ∣ m ( N v ^ ) ∣ = 1 2 min s ∈ { ± 1 } ∥ v ^ − s θ ⋆ ∥ 2 ≲ k N k − 2 2 λ 2 + N − 2 ϑ , 1-\left|m(\sqrt{N}\widehat{v})\right|
=\frac{1}{2}\min_{s\in\{\pm1\}}\lVert\widehat{v}-s\theta^\star\rVert^2
\lesssim_k \frac{N^{\frac{k-2}{2}}}{\lambda^2}+N^{-2\vartheta}, 1 − m ( N v ) = 2 1 s ∈ { ± 1 } min ∥ v − s θ ⋆ ∥ 2 ≲ k λ 2 N 2 k − 2 + N − 2 ϑ , which proves the claim for even k k k
Strong Recovery at the Algorithmic Threshold for Odd k k k
Throughout this section, we assume that k k k ( X t ) t ≥ 0 (X_t)_{t\ge0} ( X t ) t ≥ 0 H H H β > 0 \beta>0 β > 0 λ ≥ C β , k N γ \lambda\ge C_{\beta,k}N^\gamma λ ≥ C β , k N γ γ ≥ k − 2 4 \gamma\ge\frac{k-2}{4} γ ≥ 4 k − 2 C β , k C_{\beta,k} C β , k
Our goal in this section is to prove the following theorem, which is a formal version of Theorem 1.1 .
Theorem 6.1 (Persistent Strong Recovery above the Algorithmic Threshold). Let k ≥ 3 k\ge3 k ≥ 3 β > 0 \beta>0 β > 0 λ ≥ C β , k N k − 2 4 \lambda\ge C_{\beta,k}N^{\frac{k-2}{4}} λ ≥ C β , k N 4 k − 2 C β , k C_{\beta,k} C β , k ϑ ∈ ( 0 , 1 ) \vartheta\in(0,1) ϑ ∈ ( 0 , 1 ) η ∈ ( 0 , min { 1 2 , k − 2 4 } ) \eta\in(0,\min\{\frac{1}{2},\frac{k-2}{4}\}) η ∈ ( 0 , min { 2 1 , 4 k − 2 }) ( X t ) t ≥ 0 (X_t)_{t\ge0} ( X t ) t ≥ 0 X ~ 0 ∼ π \widetilde X_0\sim\pi X 0 ∼ π β 0 \beta_0 β 0 β \beta β β 0 ≍ k λ − 1 N − k − 1 2 − ϑ \beta_0\asymp_k \lambda^{-1}N^{-\frac{k-1}{2}-\vartheta} β 0 ≍ k λ − 1 N − 2 k − 1 − ϑ T a v ≍ k N 2 k − 2 + 4 ϑ T_{\mathrm{av}}\asymp_k N^{2k-2+4\vartheta} T av ≍ k N 2 k − 2 + 4 ϑ T s t r > 0 T_{\mathrm{str}}>0 T str > 0 β \beta β k k k N N N T T T T s t r ≤ T ≤ poly ( N ) T_{\mathrm{str}}\le T\le\operatorname{poly}(N) T str ≤ T ≤ poly ( N )
P [ inf t ∈ [ T s t r , T ] m ( X t ) ≥ 1 − N − η ] ≥ 1 − exp ( − c β , k N min { 1 2 , 1 − 2 η } ) . \mathbb{P}\left[\inf_{t\in[T_{\mathrm{str}},T]}m(X_t)\ge1-N^{-\eta}\right]\ge1-\exp\left(-c_{\beta,k}N^{\min\{\frac{1}{2},1-2\eta\}}\right). P [ t ∈ [ T str , T ] inf m ( X t ) ≥ 1 − N − η ] ≥ 1 − exp ( − c β , k N m i n { 2 1 , 1 − 2 η } ) . The behavior of the second phase depends on how warm its initialization is. Once the correlation exceeds a large constant multiple of the critical correlation λ − 1 k − 1 \lambda^{-\frac{1}{k-1}} λ − k − 1 1 S N \mathcal{S}_N S N § 6.1 . However, for λ \lambda λ § 6.5 , where we show that the warm start depends on the disorder only in a benign way, so that the signal drift still dominates the disorder drift along the trajectory until the dynamics achieves critical recovery. The complete proof is presented in § 6.6 .
Boosting Critical Recovery to Persistent Strong Recovery
Recall from § 2.4 that the signal drift dominates both the entropic and disorder drifts, even in the worst case, whenever the correlation exceeds a large constant multiple of the critical correlation λ − 1 k − 1 \lambda^{-\frac{1}{k-1}} λ − k − 1 1 λ m ( X t ) k − 1 \lambda m(X_t)^{k-1} λm ( X t ) k − 1 S N \mathcal{S}_N S N 1 − poly ( N ) 1-\operatorname{poly}(N) 1 − poly ( N ) poly ( N ) \operatorname{poly}(N) poly ( N )
Theorem 6.2 (Persistent Strong Recovery after Critical Correlation). Fix η ∈ ( 0 , min { 1 2 , k − 2 4 } ) \eta\in(0,\min\{\frac{1}{2},\frac{k-2}{4}\}) η ∈ ( 0 , min { 2 1 , 4 k − 2 }) s ∈ ( 0 , 1 2 ) s\in(0,\frac{1}{2}) s ∈ ( 0 , 2 1 ) r ˉ N = N − s \bar r_N=N^{-s} r ˉ N = N − s λ r ˉ N k − 1 ≥ C β , k \lambda\bar r_N^{k-1}\ge C_{\beta,k} λ r ˉ N k − 1 ≥ C β , k
τ C r ˉ N ≔ inf { t ≥ 0 : m ( X t ) ≥ C r ˉ N } , \tau_{C\bar r_N}\coloneqq\inf\{t\ge0:m(X_t)\ge C\bar r_N\}, τ C r ˉ N : = inf { t ≥ 0 : m ( X t ) ≥ C r ˉ N } , where C ≥ 1 C \ge1 C ≥ 1 N N N T T T 2 ≤ T ≤ poly ( N ) 2 \le T \le\operatorname{poly}(N) 2 ≤ T ≤ poly ( N ) X 0 X_0 X 0 H H H
P [ inf t ∈ [ 2 , T ] m ( X τ C r ˉ N + t ) ≥ 1 − N − η ] ≥ 1 − exp ( − c k N ) − 2 exp ( − c N min { r ˉ N , N − 2 η } ) . \mathbb{P}\left[\inf_{t\in[2,T]}m\left(X_{\tau_{C\bar r_N}+t}\right)\ge1-N^{-\eta}\right]\ge1-\exp(-c_kN)-2\exp\left(-cN\min\{\bar r_N,N^{-2\eta}\}\right). P [ t ∈ [ 2 , T ] inf m ( X τ C r ˉ N + t ) ≥ 1 − N − η ] ≥ 1 − exp ( − c k N ) − 2 exp ( − c N min { r ˉ N , N − 2 η } ) . Proof. Write q N ≔ N − η q_N \coloneqq N^{-\eta} q N : = N − η ε N ≔ 1 32 min { r ˉ N , q N } \varepsilon_N \coloneqq\frac{1}{32}\min\{\bar r_N,q_N\} ε N : = 32 1 min { r ˉ N , q N } N N N C β , k C_{\beta,k} C β , k S N \mathcal{S}_N S N τ C r ˉ N < ∞ \tau_{C\bar r_N}<\infty τ C r ˉ N < ∞ [40 ] . Hence, by the strong Markov property [47 Theorem 6.2.2] applied at τ C r ˉ N \tau_{C\bar r_N} τ C r ˉ N X 0 X_0 X 0 m ( X 0 ) ≥ C r ˉ N m(X_0)\ge C\bar r_N m ( X 0 ) ≥ C r ˉ N H H H
Since m ( x ) m(x) m ( x ) (8) implies ∇ S N m ( x ) = N − 1 / 2 Q x θ ⋆ \nabla_{\mathcal{S}_N}m(x)=N^{-1/2}Q_x\theta^\star ∇ S N m ( x ) = N − 1/2 Q x θ ⋆ Δ S N m = − κ N m \Delta_{\mathcal{S}_N}m=-\kappa_Nm Δ S N m = − κ N m N ∥ ∇ S N m ( x ) ∥ 2 = ∥ Q x θ ⋆ ∥ 2 = 1 − m ( x ) 2 N\lVert\nabla_{\mathcal{S}_N}m(x)\rVert^2=\lVert Q_x\theta^\star\rVert^2=1-m(x)^2 N ∥ ∇ S N m ( x ) ∥ 2 = ∥ Q x θ ⋆ ∥ 2 = 1 − m ( x ) 2 (19) :
L H , β m ( x ) = 1 2 Δ S N m ( x ) + β ⟨ ∇ S N H ( x ) , ∇ S N m ( x ) ⟩ = − κ N 2 m ( x ) + β ⟨ N λ k m ( x ) k − 1 ∇ S N m ( x ) + ∇ S N H 0 ( x ) , ∇ S N m ( x ) ⟩ = − κ N 2 m ( x ) + β λ k m ( x ) k − 1 ( 1 − m ( x ) 2 ) + β ⟨ ∇ S N H 0 ( x ) , ∇ S N m ( x ) ⟩ = ν ( x ) ( 1 − m ( x ) ) − b ( x ) , (59) \begin{aligned}
\mathcal{L}_{H,\beta}m(x)&=\frac{1}{2}\Delta_{\mathcal{S}_N}m(x)+\beta\langle\nabla_{\mathcal{S}_N}H(x),\nabla_{\mathcal{S}_N}m(x)\rangle\\
&=-\frac{\kappa_N}{2}m(x)+\beta\left\langle N\lambda_km(x)^{k-1}\nabla_{\mathcal{S}_N}m(x)+\nabla_{\mathcal{S}_N}H_0(x),\nabla_{\mathcal{S}_N}m(x)\right\rangle\\
&=-\frac{\kappa_N}{2}m(x)+\beta\lambda_km(x)^{k-1}\left(1-m(x)^2\right)+\beta\langle\nabla_{\mathcal{S}_N}H_0(x),\nabla_{\mathcal{S}_N}m(x)\rangle\\
&=\nu(x)\left(1-m(x)\right)-b(x), \tag*{(59)}
\end{aligned} L H , β m ( x ) = 2 1 Δ S N m ( x ) + β ⟨ ∇ S N H ( x ) , ∇ S N m ( x )⟩ = − 2 κ N m ( x ) + β ⟨ N λ k m ( x ) k − 1 ∇ S N m ( x ) + ∇ S N H 0 ( x ) , ∇ S N m ( x ) ⟩ = − 2 κ N m ( x ) + β λ k m ( x ) k − 1 ( 1 − m ( x ) 2 ) + β ⟨ ∇ S N H 0 ( x ) , ∇ S N m ( x )⟩ = ν ( x ) ( 1 − m ( x ) ) − b ( x ) , (59) where ν ( x ) \nu(x) ν ( x ) b ( x ) b(x) b ( x )
ν ( x ) ≔ β λ k m ( x ) k − 1 ( 1 + m ( x ) ) , b ( x ) ≔ κ N 2 m ( x ) − β ⟨ ∇ S N H 0 ( x ) , ∇ S N m ( x ) ⟩ . \nu(x)\coloneqq\beta\lambda_km(x)^{k-1}\left(1+m(x)\right),\qquad b(x)\coloneqq\frac{\kappa_N}{2}m(x)-\beta\langle\nabla_{\mathcal{S}_N}H_0(x),\nabla_{\mathcal{S}_N}m(x)\rangle. ν ( x ) : = β λ k m ( x ) k − 1 ( 1 + m ( x ) ) , b ( x ) : = 2 κ N m ( x ) − β ⟨ ∇ S N H 0 ( x ) , ∇ S N m ( x )⟩ . Let M t m M_t^m M t m m m m Lemma 3.11 ,
M t m = m ( X t ) − m ( X 0 ) − ∫ 0 t L H , β m ( X s ) d s . (60) M_t^m=m(X_t)-m(X_0)-\int_0^t\mathcal{L}_{H,\beta}m(X_s)\,ds. \tag*{(60)} M t m = m ( X t ) − m ( X 0 ) − ∫ 0 t L H , β m ( X s ) d s . (60) Consider the events
E ≔ { max x ∈ S N ∥ ∇ S N H 0 ( x ) ∥ ≤ C 1 N } , M j ≔ { sup j ≤ t ≤ j + 1 ∣ M t m − M j m ∣ ≤ ε N } , \mathcal{E}\coloneqq\left\{\max_{x\in\mathcal{S}_N}\lVert\nabla_{\mathcal{S}_N}H_0(x)\rVert\le C_1\sqrt{N}\right\},\qquad
\mathcal{M}_j\coloneqq\left\{\sup_{j\le t\le j+1}\left|M_t^m-M_j^m\right|\le\varepsilon_N\right\}, E : = { x ∈ S N max ∥ ∇ S N H 0 ( x )∥ ≤ C 1 N } , M j : = { j ≤ t ≤ j + 1 sup M t m − M j m ≤ ε N } , and let M ≔ ⋂ j = 0 ⌈ T ⌉ − 1 M j \mathcal{M}\coloneqq\bigcap_{j=0}^{\lceil T\rceil-1}\mathcal{M}_j M : = ⋂ j = 0 ⌈ T ⌉ − 1 M j Lemma 3.7 with n = 1 n=1 n = 1 r = N r=N r = N Definition 3.5 , we can choose C 1 C_1 C 1 P [ E c ] ≥ 1 − exp ( − c k N ) \mathbb{P}[\mathcal{E}^c]\ge1-\exp(-c_kN) P [ E c ] ≥ 1 − exp ( − c k N ) ∥ ∇ S N m ( x ) ∥ ≤ N − 1 / 2 \lVert\nabla_{\mathcal{S}_N}m(x)\rVert\le N^{-1/2} ∥ ∇ S N m ( x )∥ ≤ N − 1/2 Lemma 3.12 , and a union bound that
P [ M c ] ≤ ∑ j = 0 ⌈ T ⌉ − 1 P [ M j c ] ≤ 2 ⌈ T ⌉ exp ( − N ε N 2 2 ) . (61) \mathbb{P}[\mathcal{M}^c]\le\sum_{j=0}^{\lceil T\rceil-1}\mathbb{P}[\mathcal{M}_j^c]\le2\lceil T\rceil\exp\left(-\frac{N\varepsilon_N^2}{2}\right). \tag*{(61)} P [ M c ] ≤ j = 0 ∑ ⌈ T ⌉ − 1 P [ M j c ] ≤ 2 ⌈ T ⌉ exp ( − 2 N ε N 2 ) . (61) Henceforth, we work on the event E ∩ M \mathcal{E}\cap\mathcal{M} E ∩ M
We first express the deficit 1 − m ( X t ) 1-m(X_t) 1 − m ( X t ) b ( X t ) b(X_t) b ( X t ) R t ≔ ∫ 0 t ν ( X s ) d s R_t\coloneqq\int_0^t\nu(X_s)\,ds R t : = ∫ 0 t ν ( X s ) d s (59) and (60) ,
1 − m ( X t ) + M t m = 1 − m ( X 0 ) − ∫ 0 t [ ν ( X s ) ( 1 − m ( X s ) ) − b ( X s ) ] d s . 1-m(X_t)+M_t^m=1-m(X_0)-\int_0^t\left[\nu(X_s)\left(1-m(X_s)\right)-b(X_s)\right]\,ds. 1 − m ( X t ) + M t m = 1 − m ( X 0 ) − ∫ 0 t [ ν ( X s ) ( 1 − m ( X s ) ) − b ( X s ) ] d s . By the continuity of Brownian sample paths, the integrand is continuous in s s s t t t
d d t [ e R t ( 1 − m ( X t ) + M t m ) ] = e R t ν ( X t ) ( 1 − m ( X t ) + M t m ) − e R t [ ν ( X t ) ( 1 − m ( X t ) ) − b ( X t ) ] = e R t [ ν ( X t ) M t m + b ( X t ) ] . (62) \begin{aligned}
\frac{d}{dt}\left[e^{R_t}\left(1-m(X_t)+M_t^m\right)\right]
&=e^{R_t}\nu(X_t)\left(1-m(X_t)+M_t^m\right)-e^{R_t}\left[\nu(X_t)\left(1-m(X_t)\right)-b(X_t)\right]\\
&=e^{R_t}\left[\nu(X_t)M_t^m+b(X_t)\right]. \tag*{(62)}
\end{aligned} d t d [ e R t ( 1 − m ( X t ) + M t m ) ] = e R t ν ( X t ) ( 1 − m ( X t ) + M t m ) − e R t [ ν ( X t ) ( 1 − m ( X t ) ) − b ( X t ) ] = e R t [ ν ( X t ) M t m + b ( X t ) ] . (62) Integrating Equation (62) from 0 0 0 t t t M 0 m = 0 M_0^m=0 M 0 m = 0 e − R t e^{-R_t} e − R t
1 − m ( X t ) = e − R t ( 1 − m ( X 0 ) ) + ∫ 0 t e − ( R t − R s ) b ( X s ) d s + ξ t , 0 ≤ t ≤ T , (63) 1-m(X_t)=e^{-R_t}\left(1-m(X_0)\right)+\int_0^t e^{-(R_t-R_s)}b(X_s)\,ds+\xi_t,\qquad0\le t\le T, \tag*{(63)} 1 − m ( X t ) = e − R t ( 1 − m ( X 0 ) ) + ∫ 0 t e − ( R t − R s ) b ( X s ) d s + ξ t , 0 ≤ t ≤ T , (63) where the martingale error term ξ t \xi_t ξ t
ξ t = ∫ 0 t ν ( X s ) e − ( R t − R s ) M s m d s − M t m . (64) \xi_t=\int_0^t \nu(X_s)e^{-(R_t-R_s)}M_s^m\,ds-M_t^m. \tag*{(64)} ξ t = ∫ 0 t ν ( X s ) e − ( R t − R s ) M s m d s − M t m . (64) To control the integral involving b b b ξ t \xi_t ξ t (63) , we shall repeatedly use the following identity, which holds since R 0 = 0 R_0=0 R 0 = 0 d d s e − ( R t − R s ) = ν ( X s ) e − ( R t − R s ) \frac{d}{ds}e^{-(R_t-R_s)}=\nu(X_s)e^{-(R_t-R_s)} d s d e − ( R t − R s ) = ν ( X s ) e − ( R t − R s )
∫ 0 t ν ( X s ) e − ( R t − R s ) d s = 1 − e − R t . (65) \int_0^t \nu(X_s)e^{-(R_t-R_s)}\,ds=1-e^{-R_t}. \tag*{(65)} ∫ 0 t ν ( X s ) e − ( R t − R s ) d s = 1 − e − R t . (65) Multiplying Equation (65) by M t m M_t^m M t m
M t m = ∫ 0 t ν ( X s ) e − ( R t − R s ) M t m d s + e − R t M t m . (66) M_t^m=\int_0^t \nu(X_s)e^{-(R_t-R_s)}M_t^m\,ds+e^{-R_t}M_t^m. \tag*{(66)} M t m = ∫ 0 t ν ( X s ) e − ( R t − R s ) M t m d s + e − R t M t m . (66) Substituting Equation (66) into Equation (64) and using M 0 m = 0 M_0^m=0 M 0 m = 0 M m M^m M m
ξ t = ∫ 0 t ν ( X s ) e − ( R t − R s ) ( M s m − M t m ) d s − e − R t ( M t m − M 0 m ) . (67) \xi_t=\int_0^t \nu(X_s)e^{-(R_t-R_s)}\left(M_s^m-M_t^m\right)\,ds-e^{-R_t}\left(M_t^m-M_0^m\right). \tag*{(67)} ξ t = ∫ 0 t ν ( X s ) e − ( R t − R s ) ( M s m − M t m ) d s − e − R t ( M t m − M 0 m ) . (67) Since we work on M \mathcal{M} M
∣ M t m − M s m ∣ ≤ ∣ M t m − M ⌊ t ⌋ m ∣ + ∑ l = ⌊ s ⌋ + 1 ⌊ t ⌋ ∣ M l m − M l − 1 m ∣ + ∣ M s m − M ⌊ s ⌋ m ∣ ≤ ( ⌊ t ⌋ − ⌊ s ⌋ + 2 ) ϵ N ≤ ( t − s + 3 ) ϵ N , 0 ≤ s ≤ t ≤ T . (68) \begin{aligned}
\left|M_t^m-M_s^m\right|
&\le\left|M_t^m-M_{\lfloor t\rfloor}^m\right|+\sum_{l=\lfloor s\rfloor+1}^{\lfloor t\rfloor}\left|M_l^m-M_{l-1}^m\right|+\left|M_s^m-M_{\lfloor s\rfloor}^m\right|\\
&\le\left(\lfloor t\rfloor-\lfloor s\rfloor+2\right)\epsilon_N\le(t-s+3)\epsilon_N,\qquad0\le s\le t\le T. \tag*{(68)}
\end{aligned} ∣ M t m − M s m ∣ ≤ M t m − M ⌊ t ⌋ m + l = ⌊ s ⌋ + 1 ∑ ⌊ t ⌋ M l m − M l − 1 m + M s m − M ⌊ s ⌋ m ≤ ( ⌊ t ⌋ − ⌊ s ⌋ + 2 ) ϵ N ≤ ( t − s + 3 ) ϵ N , 0 ≤ s ≤ t ≤ T . (68) We now bound ν ( x ) \nu(x) ν ( x ) b ( x ) b(x) b ( x ) m ( x ) m(x) m ( x ) E \mathcal{E} E κ N ≤ 1 \kappa_N\le1 κ N ≤ 1 ∥ ∇ S N m ( x ) ∥ ≤ N − 1 / 2 \|\nabla_{\mathcal{S}_N}m(x)\|\le N^{-1/2} ∥ ∇ S N m ( x ) ∥ ≤ N − 1/2
b ( x ) ≤ κ N 2 + β ∥ ∇ S N H 0 ( x ) ∥ ∥ ∇ S N m ( x ) ∥ ≤ 1 2 + β C 1 , x ∈ S N . (69) b(x)\le\frac{\kappa_N}{2}+\beta\|\nabla_{\mathcal{S}_N}H_0(x)\|\|\nabla_{\mathcal{S}_N}m(x)\|\le\frac{1}{2}+\beta C_1,\qquad x\in\mathcal{S}_N. \tag*{(69)} b ( x ) ≤ 2 κ N + β ∥ ∇ S N H 0 ( x ) ∥∥ ∇ S N m ( x ) ∥ ≤ 2 1 + β C 1 , x ∈ S N . (69) Moreover, ν ( x ) ≥ β λ k m ( x ) k − 1 \nu(x)\ge\beta\lambda_km(x)^{k-1} ν ( x ) ≥ β λ k m ( x ) k − 1 m ( x ) > 0 m(x)>0 m ( x ) > 0 m ( x ) ≥ C r ˉ N / 2 m(x)\ge C\bar r_N/2 m ( x ) ≥ C r ˉ N /2 λ r ˉ N k − 1 ≥ C β , k \lambda_{\bar r_N}^{k-1}\ge C_{\beta,k} λ r ˉ N k − 1 ≥ C β , k C ≥ 1 C\ge1 C ≥ 1
ν ( x ) ≥ β λ k ( C r ˉ N 2 ) k − 1 ≥ β κ 2 k − 1 C β , k ≥ max { 1 , 16 ( 1 2 + β C 1 ) } . (70) \nu(x)\ge\beta\lambda_k\left(\frac{C\bar r_N}{2}\right)^{k-1}\ge\frac{\beta\kappa}{2^{k-1}}C_{\beta,k}\ge\max\left\{1,16\left(\frac{1}{2}+\beta C_1\right)\right\}. \tag*{(70)} ν ( x ) ≥ β λ k ( 2 C r ˉ N ) k − 1 ≥ 2 k − 1 β κ C β , k ≥ max { 1 , 16 ( 2 1 + β C 1 ) } . (70) and if m ( x ) ≥ 1 3 m(x) \ge\frac{1}{3} m ( x ) ≥ 3 1 λ ≥ C β , k N γ \lambda\ge C_{\beta,k}N^\gamma λ ≥ C β , k N γ
ν ( x ) ≥ ν ⋆ = β λ k 3 k − 1 ≥ β k 3 k − 1 C β , k N γ ≥ max { 1 , 64 q N ( 1 2 + β C 1 ) } . (71) \nu(x) \ge\nu_\star= \frac{\beta\lambda k}{3^{k-1}} \ge\frac{\beta k}{3^{k-1}} C_{\beta,k}N^\gamma\ge\max\left\{1,\frac{64}{q_N}\left(\frac{1}{2}+\beta C_1\right)\right\}. \tag*{(71)} ν ( x ) ≥ ν ⋆ = 3 k − 1 β λk ≥ 3 k − 1 β k C β , k N γ ≥ max { 1 , q N 64 ( 2 1 + β C 1 ) } . (71) Here, the last inequalities in Equations (70) and (71) hold since C β , k C_{\beta,k} C β , k (71) , since q N 2 = N η q_N^2=N^\eta q N 2 = N η η < k − 2 4 ≤ γ \eta<\frac{k-2}{4}\le\gamma η < 4 k − 2 ≤ γ N N N (70) and (71) with Equation (69) , we conclude that
ν ( x ) ≥ 1 and b ( x ) ≤ 1 16 ν ( x ) whenever m ( x ) ≥ C r ˉ N 2 , ν ( x ) ≥ ν ⋆ ≥ 1 and b ( x ) ≤ q N 64 ν ( x ) whenever m ( x ) ≥ 1 3 . (72) \begin{aligned}
\nu(x)&\ge1 &&\text{and}&& b(x)\le\frac{1}{16}\nu(x) &&\text{whenever }m(x)\ge\frac{C\bar r_N}{2},\\
\nu(x)&\ge\nu_\star\ge1 &&\text{and}&& b(x)\le\frac{q_N}{64}\nu(x) &&\text{whenever }m(x)\ge\frac{1}{3}. \tag*{(72)}
\end{aligned} ν ( x ) ν ( x ) ≥ 1 ≥ ν ⋆ ≥ 1 and and b ( x ) ≤ 16 1 ν ( x ) b ( x ) ≤ 64 q N ν ( x ) whenever m ( x ) ≥ 2 C r ˉ N , whenever m ( x ) ≥ 3 1 . (72) We now show that the correlation stays above C r ˉ N / 2 C\bar r_N/2 C r ˉ N /2 [ 0 , T ] [0,T] [ 0 , T ] ρ = inf { t ≥ 0 : m ( X t ) ≤ C r ˉ N / 2 } \rho=\inf\{t\ge0:m(X_t)\le C\bar r_N/2\} ρ = inf { t ≥ 0 : m ( X t ) ≤ C r ˉ N /2 } ρ > 0 \rho>0 ρ > 0 m ( X 0 ) ≥ C r ˉ N m(X_0)\ge C\bar r_N m ( X 0 ) ≥ C r ˉ N t ≤ ρ ∧ T t\le\rho\wedge T t ≤ ρ ∧ T m ( X s ) ≥ C r ˉ N / 2 m(X_s)\ge C\bar r_N/2 m ( X s ) ≥ C r ˉ N /2 s ∈ [ 0 , t ] s\in[0,t] s ∈ [ 0 , t ] (72) ,
ν ( X s ) ≥ 1 and b ( X s ) ≤ 1 16 ν ( X s ) , 0 ≤ s ≤ t . (73) \nu(X_s)\ge1\qquad\text{and}\qquad b(X_s)\le\frac{1}{16}\nu(X_s),\qquad0\le s\le t. \tag*{(73)} ν ( X s ) ≥ 1 and b ( X s ) ≤ 16 1 ν ( X s ) , 0 ≤ s ≤ t . (73) We use the two bounds in Equation (73) to control the martingale error term and the integral in Equation (63) , respectively. First, since ν ( X s ) ≥ 1 \nu(X_s)\ge1 ν ( X s ) ≥ 1 s ∈ [ 0 , t ] s\in[0,t] s ∈ [ 0 , t ] R t − R s ≥ t − s R_t-R_s\ge t-s R t − R s ≥ t − s (68) implies
∣ M t m − M s m ∣ ≤ ( R t − R s + 3 ) ε N , 0 ≤ s ≤ t . (74) \left|M_t^m-M_s^m\right|\le(R_t-R_s+3)\varepsilon_N,\qquad0\le s\le t. \tag*{(74)} ∣ M t m − M s m ∣ ≤ ( R t − R s + 3 ) ε N , 0 ≤ s ≤ t . (74) Substituting Equation (74) into Equation (67) and changing variables to u = R t − R s u=R_t-R_s u = R t − R s d u = − ν ( X s ) d s \mathrm{d}u=-\nu(X_s)\mathrm{d}s d u = − ν ( X s ) d s
∣ ξ t ∣ ≤ ε N ∫ 0 t ν ( X s ) e − ( R t − R s ) ( R t − R s + 3 ) d s + ε N e − R t ( R t + 3 ) = ε N ∫ 0 R t e − u ( u + 3 ) d u + ε N e − R t ( R t + 3 ) ≤ ε N ∫ 0 ∞ e − u ( u + 3 ) d u + ε N sup v ≥ 0 e − v ( v + 3 ) = 4 ε N + 3 ε N = 7 ε N . (75) \begin{aligned}
|\xi_t|&\le\varepsilon_N\int_0^t\nu(X_s)e^{-(R_t-R_s)}(R_t-R_s+3)\,\mathrm{d}s+\varepsilon_N e^{-R_t}(R_t+3)\\
&=\varepsilon_N\int_0^{R_t}e^{-u}(u+3)\,\mathrm{d}u+\varepsilon_N e^{-R_t}(R_t+3)\\
&\le\varepsilon_N\int_0^\infty e^{-u}(u+3)\,\mathrm{d}u+\varepsilon_N\sup_{v\ge0}e^{-v}(v+3)=4\varepsilon_N+3\varepsilon_N=7\varepsilon_N. \tag*{(75)}
\end{aligned} ∣ ξ t ∣ ≤ ε N ∫ 0 t ν ( X s ) e − ( R t − R s ) ( R t − R s + 3 ) d s + ε N e − R t ( R t + 3 ) = ε N ∫ 0 R t e − u ( u + 3 ) d u + ε N e − R t ( R t + 3 ) ≤ ε N ∫ 0 ∞ e − u ( u + 3 ) d u + ε N v ≥ 0 sup e − v ( v + 3 ) = 4 ε N + 3 ε N = 7 ε N . (75) The bound on b b b (73) , together with Equation (65) , implies the following
∫ 0 t e − ( R t − R s ) b ( X s ) d s ≤ 1 16 ∫ 0 t ν ( X s ) e − ( R t − R s ) d s = 1 16 ( 1 − e − R t ) . (76) \int_0^t e^{-(R_t-R_s)}b(X_s)\,\mathrm{d}s\le\frac{1}{16}\int_0^t\nu(X_s)e^{-(R_t-R_s)}\,\mathrm{d}s=\frac{1}{16}(1-e^{-R_t}). \tag*{(76)} ∫ 0 t e − ( R t − R s ) b ( X s ) d s ≤ 16 1 ∫ 0 t ν ( X s ) e − ( R t − R s ) d s = 16 1 ( 1 − e − R t ) . (76) Substituting Equations (75) and (76) into Equation (63) , we obtain the following for every t ≤ ρ ∧ T t\le\rho\wedge T t ≤ ρ ∧ T
1 − m ( X t ) ≤ e − R t ( 1 − m ( X 0 ) ) + 1 16 ( 1 − e − R t ) + 7 ε N (77) 1-m(X_t)\le e^{-R_t}(1-m(X_0))+\frac{1}{16}(1-e^{-R_t})+7\varepsilon_N \tag*{(77)} 1 − m ( X t ) ≤ e − R t ( 1 − m ( X 0 )) + 16 1 ( 1 − e − R t ) + 7 ε N (77) ≤ 1 − C r ˉ N + 7 ε N < 1 − C r ˉ N 2 , (78) \le1-C\bar r_N+7\varepsilon_N<1-\frac{C\bar r_N}{2}, \tag*{(78)} ≤ 1 − C r ˉ N + 7 ε N < 1 − 2 C r ˉ N , (78) where the second inequality uses 1 − m ( X 0 ) ≤ 1 − C r ˉ N 1-m(X_0)\le1-C\bar r_N 1 − m ( X 0 ) ≤ 1 − C r ˉ N 1 16 ≤ 1 − C r ˉ N \frac{1}{16}\le1-C\bar r_N 16 1 ≤ 1 − C r ˉ N 7 ε N ≤ 7 32 r ˉ N < C r ˉ N 2 7\varepsilon_N\le\frac{7}{32}\bar r_N<\frac{C\bar r_N}{2} 7 ε N ≤ 32 7 r ˉ N < 2 C r ˉ N ρ ≤ T \rho\le T ρ ≤ T m ( X ρ ) = C r ˉ N / 2 m(X_\rho)=C\bar r_N/2 m ( X ρ ) = C r ˉ N /2 (78) at t = ρ t=\rho t = ρ ρ > T \rho>T ρ > T (73) , (75) and (77) hold for every t ∈ [ 0 , T ] t\in[0,T] t ∈ [ 0 , T ] R t ≥ t R_t\ge t R t ≥ t (77) implies the following:
1 − m ( X t ) ≤ e − 1 + 1 16 + 7 ϵ N ≤ 2 3 , 1 ≤ t ≤ T . (79) 1-m(X_t)\le e^{-1}+\frac{1}{16}+7\epsilon_N\le\frac{2}{3},\qquad1\le t\le T. \tag*{(79)} 1 − m ( X t ) ≤ e − 1 + 16 1 + 7 ϵ N ≤ 3 2 , 1 ≤ t ≤ T . (79) Consequently, m ( X t ) > 1 / 3 m(X_t)>1/3 m ( X t ) > 1/3 t ∈ [ 1 , T ] t\in[1,T] t ∈ [ 1 , T ]
Finally, we show that the deficit falls below q N q_N q N t = 2 t=2 t = 2 q N q_N q N T T T t ∈ [ 2 , T ] t\in[2,T] t ∈ [ 2 , T ] m ( X s ) > 1 / 3 m(X_s)>1/3 m ( X s ) > 1/3 s ∈ [ 1 , t ] s\in[1,t] s ∈ [ 1 , t ] (79) , we infer from Equation (72) that
ν ( X s ) ≥ ν ⋆ and b ( X s ) ≤ q N 64 ν ( X s ) , 1 ≤ s ≤ t . (80) \nu(X_s)\ge\nu_\star\qquad\text{and}\qquad b(X_s)\le\frac{q_N}{64}\nu(X_s),\qquad1\le s\le t. \tag*{(80)} ν ( X s ) ≥ ν ⋆ and b ( X s ) ≤ 64 q N ν ( X s ) , 1 ≤ s ≤ t . (80) In particular, R t − R 1 = ∫ 1 t ν ( X s ) d s ≥ ν ⋆ ( t − 1 ) ≥ ν ⋆ R_t-R_1=\int_1^t\nu(X_s)\,ds\ge\nu_\star(t-1)\ge\nu_\star R t − R 1 = ∫ 1 t ν ( X s ) d s ≥ ν ⋆ ( t − 1 ) ≥ ν ⋆ t ≥ 2 t\ge2 t ≥ 2 b b b (80) , we split the integral in Equation (63) at s = 1 s=1 s = 1 (65) , the identity d d s e − ( R t − R s ) = ν ( X s ) e − ( R t − R s ) \frac{d}{ds}e^{-(R_t-R_s)}=\nu(X_s)e^{-(R_t-R_s)} d s d e − ( R t − R s ) = ν ( X s ) e − ( R t − R s )
∫ 0 1 ν ( X s ) e − ( R t − R s ) d s = e − ( R t − R 1 ) − e − R t , ∫ 1 t ν ( X s ) e − ( R t − R s ) d s = 1 − e − ( R t − R 1 ) . (81) \int_0^1\nu(X_s)e^{-(R_t-R_s)}\,ds=e^{-(R_t-R_1)}-e^{-R_t},\qquad
\int_1^t\nu(X_s)e^{-(R_t-R_s)}\,ds=1-e^{-(R_t-R_1)}. \tag*{(81)} ∫ 0 1 ν ( X s ) e − ( R t − R s ) d s = e − ( R t − R 1 ) − e − R t , ∫ 1 t ν ( X s ) e − ( R t − R s ) d s = 1 − e − ( R t − R 1 ) . (81) Combining Equation (63) with the bounds on b b b (73) and (80) , 1 − m ( X 0 ) ≤ 1 1-m(X_0)\le1 1 − m ( X 0 ) ≤ 1 ∣ ξ t ∣ ≤ 7 ϵ N ≤ 7 q N 32 |\xi_t|\le7\epsilon_N\le\frac{7q_N}{32} ∣ ξ t ∣ ≤ 7 ϵ N ≤ 32 7 q N (75) , we obtain
1 − m ( X t ) = e − R t ( 1 − m ( X 0 ) ) + ∫ 0 1 e − ( R t − R s ) b ( X s ) d s + ∫ 1 t e − ( R t − R s ) b ( X s ) d s + ξ t ≤ e − R t + 1 16 ∫ 0 1 ν ( X s ) e − ( R t − R s ) d s + q N 64 ∫ 1 t ν ( X s ) e − ( R t − R s ) d s + 7 q N 32 = 15 16 e − R t + 1 16 e − ( R t − R 1 ) + q N 64 ( 1 − e − ( R t − R 1 ) ) + 7 q N 32 ≤ e − ν ⋆ + q N 64 + 7 q N 32 < q N . (82) \begin{aligned}
1-m(X_t)&=e^{-R_t}(1-m(X_0))+\int_0^1e^{-(R_t-R_s)}b(X_s)\,ds+\int_1^te^{-(R_t-R_s)}b(X_s)\,ds+\xi_t\\
&\le e^{-R_t}+\frac{1}{16}\int_0^1\nu(X_s)e^{-(R_t-R_s)}\,ds+\frac{q_N}{64}\int_1^t\nu(X_s)e^{-(R_t-R_s)}\,ds+\frac{7q_N}{32}\\
&=\frac{15}{16}e^{-R_t}+\frac{1}{16}e^{-(R_t-R_1)}+\frac{q_N}{64}\left(1-e^{-(R_t-R_1)}\right)+\frac{7q_N}{32}\\
&\le e^{-\nu_\star}+\frac{q_N}{64}+\frac{7q_N}{32}<q_N. \tag*{(82)}
\end{aligned} 1 − m ( X t ) = e − R t ( 1 − m ( X 0 )) + ∫ 0 1 e − ( R t − R s ) b ( X s ) d s + ∫ 1 t e − ( R t − R s ) b ( X s ) d s + ξ t ≤ e − R t + 16 1 ∫ 0 1 ν ( X s ) e − ( R t − R s ) d s + 64 q N ∫ 1 t ν ( X s ) e − ( R t − R s ) d s + 32 7 q N = 16 15 e − R t + 16 1 e − ( R t − R 1 ) + 64 q N ( 1 − e − ( R t − R 1 ) ) + 32 7 q N ≤ e − ν ⋆ + 64 q N + 32 7 q N < q N . (82) Here, the second equality follows from Equation (81) , the second inequality uses e − R t ≤ e − ( R t − R 1 ) ≤ e − ν ⋆ e^{-R_t}\le e^{-(R_t-R_1)}\le e^{-\nu_\star} e − R t ≤ e − ( R t − R 1 ) ≤ e − ν ⋆ N N N ν ⋆ ≥ β k 3 1 − k C β , k N γ \nu_\star\ge\beta k 3^{1-k}C_{\beta,k}N^\gamma ν ⋆ ≥ β k 3 1 − k C β , k N γ e − ν ⋆ = o ( N − η ) e^{-\nu_\star}=o(N^{-\eta}) e − ν ⋆ = o ( N − η )
By Equation (82) , we have m ( X t ) > 1 − q N m(X_t)>1-q_N m ( X t ) > 1 − q N t ∈ [ 2 , T ] t\in[2,T] t ∈ [ 2 , T ] E ∩ M \mathcal E\cap\mathcal M E ∩ M (61) , we obtain
P [ inf t ∈ [ 2 , T ] m ( X t ) < 1 − q N ] ≤ P [ E c ] + P [ M c ] ≤ exp ( − c k N ) + 2 ⌈ T ⌉ exp ( − c N min { r ˉ N 2 , q N 2 } ) . \mathbb{P}\left[\inf_{t\in[2,T]}m(X_t)<1-q_N\right]\le\mathbb{P}[\mathcal E^c]+\mathbb{P}[\mathcal M^c]\le\exp(-c_kN)+2\lceil T\rceil\exp\left(-cN\min\{\bar r_N^2,q_N^2\}\right). P [ t ∈ [ 2 , T ] inf m ( X t ) < 1 − q N ] ≤ P [ E c ] + P [ M c ] ≤ exp ( − c k N ) + 2 ⌈ T ⌉ exp ( − c N min { r ˉ N 2 , q N 2 } ) . Since T ≤ poly ( N ) T\le\operatorname{poly}(N) T ≤ poly ( N ) N min { r ˉ N 2 , q N 2 } / log N → ∞ N\min\{\bar r_N^2,q_N^2\}/\log N\to\infty N min { r ˉ N 2 , q N 2 } / log N → ∞ s , η < 1 / 2 s,\eta<1/2 s , η < 1/2 ⌈ T ⌉ \lceil T\rceil ⌈ T ⌉
Remark 6.3. Although this section assumes that k k k Theorem 6.2 does not use this assumption. In § 7.2 , we use this fact to show that, for even k k k
A direct application of Theorems 5.1 and 6.2 establishes persistent strong recovery for Algorithm 1 in the regime λ ≳ N k − 2 4 + 1 4 k \lambda\gtrsim N^{\frac{k-2}{4}+\frac{1}{4k}} λ ≳ N 4 k − 2 + 4 k 1 N 1 4 k N^{\frac{1}{4k}} N 4 k 1
Corollary 6.4 (Persistent Strong Recovery Slightly above the Algorithmic Threshold). Suppose γ ≥ k − 2 4 + 1 4 k \gamma\ge\frac{k-2}{4}+\frac{1}{4k} γ ≥ 4 k − 2 + 4 k 1 Fix ϑ ∈ ( 0 , 1 ) \vartheta\in(0,1) ϑ ∈ ( 0 , 1 ) and η ∈ ( 0 , min { 1 2 , k − 2 4 } ) \eta\in(0,\min\{\frac{1}{2},\frac{k-2}{4}\}) η ∈ ( 0 , min { 2 1 , 4 k − 2 }) Let ( X t ) t ≥ 0 (X_t)_{t\ge0} ( X t ) t ≥ 0 denote the trajectory of the second phase of Algorithm 1, initialized at X ~ 0 ∼ π \widetilde{X}_0 \sim\pi X 0 ∼ π with inverse temperatures β 0 \beta_0 β 0 and β \beta β for the two respective phases, where β 0 \beta_0 β 0 and T a v T_{\mathrm{av}} T av are chosen as in Theorem 5.1 for this choice of ϑ \vartheta ϑ Then, for all sufficiently large N N N and every deterministic T T T satisfying 2 ≤ T ≤ poly ( N ) 2 \le T \le\operatorname{poly}(N) 2 ≤ T ≤ poly ( N )
P [ inf t ∈ [ 2 , T ] m ( X t ) ≥ 1 − N − η ] ≥ 1 − exp ( − c k N min { ( k + 1 ) / ( 2 k ) , 1 − 2 η } ) . \mathbb{P}\left[\inf_{t\in[2,T]}m(X_t)\ge1-N^{-\eta}\right]\ge1-\exp\left(-c_kN^{\min\{(k+1)/(2k),1-2\eta\}}\right). P [ t ∈ [ 2 , T ] inf m ( X t ) ≥ 1 − N − η ] ≥ 1 − exp ( − c k N m i n {( k + 1 ) / ( 2 k ) , 1 − 2 η } ) . Proof. Define α ≔ k − 1 4 k \alpha\coloneqq\frac{k-1}{4k} α : = 4 k k − 1 k − 2 4 + 1 4 k = ( k − 1 ) α \frac{k-2}{4}+\frac{1}{4k}=(k-1)\alpha 4 k − 2 + 4 k 1 = ( k − 1 ) α γ ≥ ( k − 1 ) α \gamma\ge(k-1)\alpha γ ≥ ( k − 1 ) α γ − k − 1 4 k ≥ − α \gamma-\frac{k-1}{4k}\ge-\alpha γ − 4 k k − 1 ≥ − α
Since λ ≥ C β , k N γ \lambda\ge C_{\beta,k}N^\gamma λ ≥ C β , k N γ σ N ≍ k N − k − 1 4 \sigma_N\asymp_k N^{-\frac{k-1}{4}} σ N ≍ k N − 4 k − 1 Theorem 4.1 , it follows that λ σ N ≳ k C β , k N γ − k − 1 4 ≥ C β , k N − α \lambda\sigma_N\gtrsim_k C_{\beta,k}N^{\gamma-\frac{k-1}{4}}\ge C_{\beta,k}N^{-\alpha} λ σ N ≳ k C β , k N γ − 4 k − 1 ≥ C β , k N − α
On the event of Theorem 5.1 , we have X ^ T a v ≠ 0 \widehat{X}_{T_{\mathrm{av}}}\ne0 X T av = 0 X 0 = X ^ T a v X_0=\widehat{X}_{T_{\mathrm{av}}} X 0 = X T av ∣ m ( x ) − m ( y ) ∣ ≤ N − 1 / 2 ∥ x − y ∥ |m(x)-m(y)|\le N^{-1/2}\|x-y\| ∣ m ( x ) − m ( y ) ∣ ≤ N − 1/2 ∥ x − y ∥ x , y ∈ R N x,y\in\mathbb{R}^N x , y ∈ R N Theorem 5.1 and Corollary 4.3 and a union bound that the following holds with probability at least 1 − exp ( − c k N ( k + 1 ) / ( 2 k ) ) 1-\exp(-c_kN^{(k+1)/(2k)}) 1 − exp ( − c k N ( k + 1 ) / ( 2 k ) ) C β , k C_{\beta,k} C β , k
m ( X 0 ) ≥ m ( u ^ H ) − 1 N ∥ X ^ T a v − u ^ H ∥ ≥ ( c k − N − ϑ ) min { 1 , λ σ N } ≥ N − α . (83) m(X_0)\ge m(\widehat{u}_H)-\frac{1}{\sqrt{N}}\|\widehat{X}_{T_{\mathrm{av}}}-\widehat{u}_H\|\ge(c_k-N^{-\vartheta})\min\{1,\lambda\sigma_N\}\ge N^{-\alpha}. \tag*{(83)} m ( X 0 ) ≥ m ( u H ) − N 1 ∥ X T av − u H ∥ ≥ ( c k − N − ϑ ) min { 1 , λ σ N } ≥ N − α . (83) Here, the second inequality combines the lower bound on m ( u ^ H ) m(\widehat{u}_H) m ( u H ) σ N ≍ k N − k − 1 4 \sigma_N\asymp_k N^{-\frac{k-1}{4}} σ N ≍ k N − 4 k − 1 ∥ X ^ T a v − u ^ H ∥ \|\widehat{X}_{T_{\mathrm{av}}}-\widehat{u}_H\| ∥ X T av − u H ∥ Theorem 5.1 , and the last inequality uses λ σ N ≳ k C β , k N − α \lambda\sigma_N\gtrsim_k C_{\beta,k}N^{-\alpha} λ σ N ≳ k C β , k N − α
By construction, ( X t ) t ≥ 0 (X_t)_{t\ge0} ( X t ) t ≥ 0 H H H β \beta β X 0 = X ^ T a v X_0=\widehat{X}_{T_{\mathrm{av}}} X 0 = X T av Theorem 6.2 allows the initial state to depend on H H H s = α s=\alpha s = α C = 1 C=1 C = 1 γ ≥ ( k − 1 ) α \gamma\ge(k-1)\alpha γ ≥ ( k − 1 ) α r ‾ N = N − α \overline{r}_N=N^{-\alpha} r N = N − α λ r ‾ N k − 1 ≥ C β , k \lambda\overline{r}_N^{k-1}\ge C_{\beta,k} λ r N k − 1 ≥ C β , k τ ‾ r ‾ N = 0 \overline{\tau}_{\overline{r}_N}=0 τ r N = 0 (83) . Finally, since N r ‾ N 2 = N 1 − 2 α = N k + 1 2 k N\overline{r}_N^2=N^{1-2\alpha}=N^{\frac{k+1}{2k}} N r N 2 = N 1 − 2 α = N 2 k k + 1 □ \square □
Achieving Critical Recovery above the Algorithmic Threshold from a Sufficiently Warm Start We now focus our attention on strong recovery in the regime N k − 2 4 ≲ λ ≲ N k − 2 4 + 1 4 k N^{\frac{k-2}{4}}\lesssim\lambda\lesssim N^{\frac{k-2}{4}+\frac{1}{4k}} N 4 k − 2 ≲ λ ≲ N 4 k − 2 + 4 k 1 λ = C β , k N γ \lambda=C_{\beta,k}N^\gamma λ = C β , k N γ k − 2 4 ≤ γ < k − 2 4 + 1 4 k \frac{k-2}{4}\le\gamma<\frac{k-2}{4}+\frac{1}{4k} 4 k − 2 ≤ γ < 4 k − 2 + 4 k 1 r N ≔ λ − 1 k − 1 ≍ N − γ k − 1 r_N\coloneqq\lambda^{-\frac{1}{k-1}}\asymp N^{-\frac{\gamma}{k-1}} r N : = λ − k − 1 1 ≍ N − k − 1 γ Algorithm 1 attains only subcritical correlation, of order λ N − k − 1 4 ≪ r N \lambda N^{-\frac{k-1}{4}}\ll r_N λ N − 4 k − 1 ≪ r N
In view of Theorem 6.2 , it suffices to show that the second phase of Algorithm 1 attains critical correlation in polynomial time. To this end, we prove a stronger guarantee in Theorem 6.5 , which establishes that, for any initialization sufficiently close to the normalized average gradient, possibly depending on the disorder, spherical Langevin dynamics attains critical correlation within O ( 1 ) O(1) O ( 1 )
For e ∈ S N e\in\mathbb{S}_N e ∈ S N ( X t ( e ) ) t ≥ 0 (X_t(e))_{t\ge0} ( X t ( e ) ) t ≥ 0 H H H β \beta β X 0 ( e ) = e X_0(e)=e X 0 ( e ) = e § 3.3 , this trajectory is a deterministic function of its initial point and its driving Brownian motion, and hence this definition extends
to any random initial state e e e H H H k k k Theorem 4.1 that the average spherical gradient u H = E π [ ∇ S N H ] u_H = \mathbb{E}_{\pi}[\nabla_{\mathcal{S}_N}H] u H = E π [ ∇ S N H ] N ( N λ α N θ ⋆ , σ N 2 I ) \mathcal{N}(N\lambda\alpha_N\theta^\star,\sigma_N^2 I) N ( N λ α N θ ⋆ , σ N 2 I ) σ N 2 = κ N N α N ≍ k N − k − 1 2 \sigma_N^2 = \kappa_N\sqrt{N}\alpha_N \asymp_k N^{-\frac{k-1}{2}} σ N 2 = κ N N α N ≍ k N − 2 k − 1
Theorem 6.5 (Critical Recovery from a Sufficiently Warm Start). Suppose k − 2 4 ≤ γ ≤ k − 2 4 + 1 4 k \frac{k-2}{4} \le\gamma\le\frac{k-2}{4} + \frac{1}{4k} 4 k − 2 ≤ γ ≤ 4 k − 2 + 4 k 1 Fix ϑ ∈ ( 0 , 1 ) \vartheta\in(0,1) ϑ ∈ ( 0 , 1 ) u H = E π [ ∇ S N H ] u_H = \mathbb{E}_{\pi}[\nabla_{\mathcal{S}_N}H] u H = E π [ ∇ S N H ] u ^ H = N u H ∥ u H ∥ \widehat{u}_H = \frac{\sqrt{N}u_H}{\lVert u_H\rVert} u H = ∥ u H ∥ N u H e ∈ S N e \in\mathcal{S}_N e ∈ S N
T ⋆ ( e ) ≔ inf { t ≥ 0 : m ( X t ( e ) ) ≥ C R N } , T_\star(e) \coloneqq\inf\{t \ge0 : m(X_t(e)) \ge C R_N\}, T ⋆ ( e ) : = inf { t ≥ 0 : m ( X t ( e )) ≥ C R N } , where C > 0 C > 0 C > 0 N N N e e e H H H
P [ 1 N ∥ e − u ^ H ∥ ≤ λ σ N N − ϑ and T ⋆ ( e ) > C 1 N − ( k − 1 ) ( γ − k − 2 4 ) ] ≤ exp ( − c N λ 2 σ N 2 ) , \mathbb{P}\left[\frac{1}{\sqrt{N}}\lVert e-\widehat{u}_H\rVert\le\lambda\sigma_N N^{-\vartheta}\ \text{and}\ T_\star(e)>C_1N^{-(k-1)\left(\gamma-\frac{k-2}{4}\right)}\right]\le\exp(-cN\lambda^2\sigma_N^2), P [ N 1 ∥ e − u H ∥ ≤ λ σ N N − ϑ and T ⋆ ( e ) > C 1 N − ( k − 1 ) ( γ − 4 k − 2 ) ] ≤ exp ( − c N λ 2 σ N 2 ) , where C 1 , c > 0 C_1,c>0 C 1 , c > 0 β \beta β k k k
The key challenge in establishing Theorem 6.5 is that the initialization may depend nontrivially on the disorder H 0 H_0 H 0 k − 2 4 ≤ γ < k − 2 4 + 1 4 k \frac{k-2}{4}\le\gamma<\frac{k-2}{4}+\frac{1}{4k} 4 k − 2 ≤ γ < 4 k − 2 + 4 k 1 N λ m k N\lambda m^k N λ m k H 0 H_0 H 0 § 2.4 , one can imagine a potentially dangerous initialization with nontrivial but subcritical correlation that lies close to one of the exponentially many local maxima of H 0 H_0 H 0 H 0 H_0 H 0
Our analysis rules out such a scenario for any initialization sufficiently close to u ^ H \widehat{u}_H u H u H u_H u H Lemma 3.1 ), u H u_H u H H 0 H_0 H 0 H 0 H_0 H 0 H 0 H_0 H 0 u H u_H u H H H H u ^ H \widehat{u}_H u H § 6.3 , we formalize the independence of u H u_H u H H H H § 6.4 develops the dynamical change of measure argument which we call the reverse planting trick. Finally, we prove Theorem 6.5 in § 6.5 .
Our proof of Theorem 6.5 extends the techniques that Part I [21 Section 6] develops to prove the analogous critical recovery guarantee for Glauber dynamics for Ising tensor PCA [21 Theorem 6.4] . As in Part I, we combine Gaussian conditioning, a gauge transformation of the warm start, and a reverse planting argument based on adaptive contiguity. The key differences stem from the Riemannian geometry of S N \mathcal{S}_N S N
The Average Gradient Gauge Transformation
The following lemma, which is a straightforward consequence of Gaussian conditioning, allows us to control the dependence of u H u_H u H
Lemma 6.6 (Decomposition of the Spherical k k k k ≥ 3 k \ge3 k ≥ 3 ζ ≔ E π [ ∇ S N H 0 ] \zeta\coloneqq\mathbb{E}_{\pi}[\nabla_{\mathbb{S}_N}H_0] ζ : = E π [ ∇ S N H 0 ] H 0 ( x ) = κ N − 1 ⟨ ζ , x ⟩ + H 0 ⊥ ( x ) H_0(x)=\kappa_N^{-1}\langle\zeta,x\rangle+H_0^{\perp}(x) H 0 ( x ) = κ N − 1 ⟨ ζ , x ⟩ + H 0 ⊥ ( x ) H 0 ⊥ : S N → R H_0^{\perp}:\mathbb{S}_N\to\mathbb{R} H 0 ⊥ : S N → R ζ \zeta ζ
E [ H 0 ⊥ ( x ) H 0 ⊥ ( y ) ] = N ( ⟨ x , y ⟩ N ) k − σ N 2 κ N 2 ⟨ x , y ⟩ , \mathbb{E}[H_0^{\perp}(x)H_0^{\perp}(y)]=N\left(\frac{\langle x,y\rangle}{N}\right)^k-\frac{\sigma_N^2}{\kappa_N^2}\langle x,y\rangle, E [ H 0 ⊥ ( x ) H 0 ⊥ ( y )] = N ( N ⟨ x , y ⟩ ) k − κ N 2 σ N 2 ⟨ x , y ⟩ , where σ N ≍ k N k − 1 4 \sigma_N\asymp_k N^{\frac{k-1}{4}} σ N ≍ k N 4 k − 1 Theorem 4.1 .
Proof. By Stein’s lemma (Lemma 3.1 ), ζ = κ N E y ∼ π [ y H 0 ( y ) ] \zeta=\kappa_N\mathbb{E}_{y\sim\pi}[yH_0(y)] ζ = κ N E y ∼ π [ y H 0 ( y )] x ∈ S N x\in\mathbb{S}_N x ∈ S N v ∈ S N − 1 v\in\mathbb{S}^{N-1} v ∈ S N − 1 x x x Lemma 3.3 , applied with ( w , z ) = ( v , x / N ) (w,z)=(v,x/\sqrt{N}) ( w , z ) = ( v , x / N ) g ( t ) = t g(t)=t g ( t ) = t h ( t ) = t k h(t)=t^k h ( t ) = t k E y ∼ π [ ⟨ v , y ⟩ ⟨ x , y ⟩ k ] = 0 \mathbb{E}_{y\sim\pi}[\langle v,y\rangle\langle x,y\rangle^k]=0 E y ∼ π [⟨ v , y ⟩ ⟨ x , y ⟩ k ] = 0 E y ∼ π [ y ⟨ x , y ⟩ k ] \mathbb{E}_{y\sim\pi}[y\langle x,y\rangle^k] E y ∼ π [ y ⟨ x , y ⟩ k ] x x x x x x (3) and Equations (22) and (24) in the proof of Theorem 4.1 :
E [ H 0 ( x ) ζ ] = κ N E y ∼ π [ y E [ H 0 ( x ) H 0 ( y ) ] ] = κ N N E y ∼ π [ y ( ⟨ x , y ⟩ N ) k ] = κ N N E y ∼ π [ ( ⟨ x , y ⟩ N ) k + 1 ] x = κ N N E y ∼ π [ m ( y ) k + 1 ] x = N α N x = σ N 2 κ N x , \begin{aligned}
\mathbb{E}[H_0(x)\zeta]=\kappa_N\mathbb{E}_{y\sim\pi}[y\mathbb{E}[H_0(x)H_0(y)]]=\kappa_NN\mathbb{E}_{y\sim\pi}\left[y\left(\frac{\langle x,y\rangle}{N}\right)^k\right] \\
&=\kappa_NN\mathbb{E}_{y\sim\pi}\left[\left(\frac{\langle x,y\rangle}{N}\right)^{k+1}\right]x=\kappa_NN\mathbb{E}_{y\sim\pi}[m(y)^{k+1}]x=\sqrt{N}\alpha_Nx=\frac{\sigma_N^2}{\kappa_N}x,
\end{aligned} E [ H 0 ( x ) ζ ] = κ N E y ∼ π [ y E [ H 0 ( x ) H 0 ( y )]] = κ N N E y ∼ π [ y ( N ⟨ x , y ⟩ ) k ] = κ N N E y ∼ π [ ( N ⟨ x , y ⟩ ) k + 1 ] x = κ N N E y ∼ π [ m ( y ) k + 1 ] x = N α N x = κ N σ N 2 x , where the last equality uses σ N 2 = κ N N α N \sigma_N^2=\kappa_N\sqrt{N}\alpha_N σ N 2 = κ N N α N Theorem 4.1 .
Since k k k Theorem 4.1 implies that σ N 2 > 0 \sigma_N^2>0 σ N 2 > 0 ( H 0 ( x ) ) x ∈ S N (H_0(x))_{x\in\mathcal{S}_N} ( H 0 ( x ) ) x ∈ S N ζ \zeta ζ E [ ζ ζ ⊤ ] = σ N 2 I \mathbb{E}[\zeta\zeta^\top]=\sigma_N^2I E [ ζ ζ ⊤ ] = σ N 2 I [28 Chapter 9] implies that, for every x ∈ S N x\in\mathcal{S}_N x ∈ S N E [ H 0 ( x ) ∣ ζ ] = κ N − 1 ⟨ ζ , x ⟩ \mathbb{E}[H_0(x)\mid\zeta]=\kappa_N^{-1}\langle\zeta,x\rangle E [ H 0 ( x ) ∣ ζ ] = κ N − 1 ⟨ ζ , x ⟩ H 0 ⊥ ( x ) = H 0 ( x ) − κ N − 1 ⟨ ζ , x ⟩ H_0^{\perp}(x)=H_0(x)-\kappa_N^{-1}\langle\zeta,x\rangle H 0 ⊥ ( x ) = H 0 ( x ) − κ N − 1 ⟨ ζ , x ⟩ ζ \zeta ζ H 0 ⊥ ( x ) = H 0 ( x ) − κ N − 1 ⟨ ζ , x ⟩ H_0^{\perp}(x)=H_0(x)-\kappa_N^{-1}\langle\zeta,x\rangle H 0 ⊥ ( x ) = H 0 ( x ) − κ N − 1 ⟨ ζ , x ⟩
E [ H 0 ⊥ ( x ) H 0 ⊥ ( y ) ] = E [ ( H 0 ( x ) − κ N − 1 ⟨ ζ , x ⟩ ) ( H 0 ( y ) − κ N − 1 ⟨ ζ , y ⟩ ) ] = E [ H 0 ( x ) H 0 ( y ) ] − ⟨ y , E [ H 0 ( x ) ζ ] ⟩ κ N − ⟨ x , E [ H 0 ( y ) ζ ] ⟩ κ N + x ⊤ E [ ζ ζ ⊤ ] y κ N 2 = N ( ⟨ x , y ⟩ N ) k − 2 σ N 2 κ N 2 ⟨ x , y ⟩ + σ N 2 κ N 2 ⟨ x , y ⟩ = N ( ⟨ x , y ⟩ N ) k − σ N 2 κ N 2 ⟨ x , y ⟩ , \begin{aligned}
\mathbb{E}[H_0^{\perp}(x)H_0^{\perp}(y)]=\mathbb{E}\left[(H_0(x)-\kappa_N^{-1}\langle\zeta,x\rangle)(H_0(y)-\kappa_N^{-1}\langle\zeta,y\rangle)\right] \\
&=\mathbb{E}[H_0(x)H_0(y)]-\frac{\langle y,\mathbb{E}[H_0(x)\zeta]\rangle}{\kappa_N}-\frac{\langle x,\mathbb{E}[H_0(y)\zeta]\rangle}{\kappa_N}+\frac{x^\top\mathbb{E}[\zeta\zeta^\top]y}{\kappa_N^2} \\
&=N\left(\frac{\langle x,y\rangle}{N}\right)^k-\frac{2\sigma_N^2}{\kappa_N^2}\langle x,y\rangle+\frac{\sigma_N^2}{\kappa_N^2}\langle x,y\rangle\\
&=N\left(\frac{\langle x,y\rangle}{N}\right)^k-\frac{\sigma_N^2}{\kappa_N^2}\langle x,y\rangle,
\end{aligned} E [ H 0 ⊥ ( x ) H 0 ⊥ ( y )] = E [ ( H 0 ( x ) − κ N − 1 ⟨ ζ , x ⟩) ( H 0 ( y ) − κ N − 1 ⟨ ζ , y ⟩) ] = E [ H 0 ( x ) H 0 ( y )] − κ N ⟨ y , E [ H 0 ( x ) ζ ]⟩ − κ N ⟨ x , E [ H 0 ( y ) ζ ]⟩ + κ N 2 x ⊤ E [ ζ ζ ⊤ ] y = N ( N ⟨ x , y ⟩ ) k − κ N 2 2 σ N 2 ⟨ x , y ⟩ + κ N 2 σ N 2 ⟨ x , y ⟩ = N ( N ⟨ x , y ⟩ ) k − κ N 2 σ N 2 ⟨ x , y ⟩ , which concludes the proof.
Since E π [ x x ⊤ ] = I \mathbb{E}_{\pi}[xx^\top]=I E π [ x x ⊤ ] = I Lemma 3.1 ) shows that κ N − 1 ⟨ ζ , x ⟩ = ⟨ E y ∼ π [ y H 0 ( y ) ] , x ⟩ \kappa_N^{-1}\langle\zeta,x\rangle=\langle\mathbb{E}_{y\sim\pi}[yH_0(y)],x\rangle κ N − 1 ⟨ ζ , x ⟩ = ⟨ E y ∼ π [ y H 0 ( y )] , x ⟩ H 0 H_0 H 0 Theorem 4.1 , we have u H = E π [ ∇ S N H ] = N λ α N θ ⋆ + ζ u_H=\mathbb{E}_{\pi}[\nabla_{\mathbb{S}_N}H]=N\lambda\alpha_N\theta^\star+\zeta u H = E π [ ∇ S N H ] = N λ α N θ ⋆ + ζ Lemma 6.6 shows that u H u_H u H H 0 ⊥ H_0^{\perp} H 0 ⊥ H 0 H_0 H 0
However, the initialization considered in Theorem 6.5 still depends on the disorder through u H u_H u H average gradient gauge transformation . This transformation maps the normalized average gradient u ^ H = N u H ∥ u H ∥ \widehat{u}_H = \frac{\sqrt{N}u_H}{\lVert u_H\rVert} u H = ∥ u H ∥ N u H N e 1 \sqrt{N}e_1 N e 1 [41 ] , except that we map the initial state of Langevin dynamics, rather than the planted signal, to N e 1 \sqrt{N}e_1 N e 1
R ≔ ∥ u H ∥ , Φ ≔ Ψ Λ ( u H ) , ϕ ≔ Φ θ ⋆ , (84) R \coloneqq\lVert u_H\rVert,\qquad\Phi\coloneqq\Psi\Lambda(u_H),\qquad\phi\coloneqq\Phi\theta^\star, \tag*{(84)} R : = ∥ u H ∥ , Φ : = ΨΛ ( u H ) , ϕ : = Φ θ ⋆ , (84) where Ψ \Psi Ψ { S ∈ O ( N , R ) : S e 1 = e 1 } ≅ O ( N − 1 , R ) \{S \in O(N,\mathbb{R}) : Se_1 = e_1\} \cong O(N-1,\mathbb{R}) { S ∈ O ( N , R ) : S e 1 = e 1 } ≅ O ( N − 1 , R ) Λ ( u ) \Lambda(u) Λ ( u ) u ∈ R N ∖ { 0 } u \in\mathbb{R}^N \setminus\{0\} u ∈ R N ∖ { 0 }
Λ ( u ) ≔ { I − 2 ( u − ∥ u ∥ e 1 ) ( u − ∥ u ∥ e 1 ) T ∥ u − ∥ u ∥ e 1 ∥ 2 , if u ≠ ∥ u ∥ e 1 , I , if u = ∥ u ∥ e 1 . \Lambda(u) \coloneqq
\begin{cases}
I - \dfrac{2(u-\lVert u\rVert e_1)(u-\lVert u\rVert e_1)^{\mathsf T}}{\lVert u-\lVert u\rVert e_1\rVert^2}, & \text{if }u \ne\lVert u\rVert e_1,\\
I, & \text{if }u = \lVert u\rVert e_1.
\end{cases} Λ ( u ) : = ⎩ ⎨ ⎧ I − ∥ u − ∥ u ∥ e 1 ∥ 2 2 ( u − ∥ u ∥ e 1 ) ( u − ∥ u ∥ e 1 ) T , I , if u = ∥ u ∥ e 1 , if u = ∥ u ∥ e 1 . By definition, Ψ e 1 = e 1 \Psi e_1=e_1 Ψ e 1 = e 1 Λ ( u ) u = ∥ u ∥ e 1 \Lambda(u)u=\lVert u\rVert e_1 Λ ( u ) u = ∥ u ∥ e 1 u ∈ R N u\in\mathbb{R}^N u ∈ R N u u u c u cu c u c > 0 c>0 c > 0 Λ ( u ) \Lambda(u) Λ ( u ) u u u u / ∥ u ∥ u/\lVert u\rVert u / ∥ u ∥ Φ u H = R e 1 \Phi u_H=Re_1 Φ u H = R e 1 Φ u ^ H = N e 1 ∈ S N \Phi\widehat{u}_H=\sqrt{N}e_1\in\mathcal{S}_N Φ u H = N e 1 ∈ S N ζ ∼ N ( 0 , σ N 2 I ) \zeta\sim\mathcal{N}(0,\sigma_N^2 I) ζ ∼ N ( 0 , σ N 2 I ) σ N 2 > 0 \sigma_N^2>0 σ N 2 > 0 u H u_H u H § 6.4 , the randomization over orthogonal transformations fixing e 1 e_1 e 1 λ = 0 \lambda=0 λ = 0 ϕ \phi ϕ S N − 1 \mathbb{S}^{N-1} S N − 1 R R R
y = y ( x ) ≔ Φ x , Y t ( e ) ≔ Φ X t ( Φ T e ) . (85) y=y(x)\coloneqq\Phi x,\qquad Y_t(e)\coloneqq\Phi X_t(\Phi^{\mathsf T}e). \tag*{(85)} y = y ( x ) : = Φ x , Y t ( e ) : = Φ X t ( Φ T e ) . (85) Observe that y ( u ^ H ) = N e 1 y(\widehat{u}_H)=\sqrt{N}e_1 y ( u H ) = N e 1 Y t ( N e 1 ) = Φ X t ( u ^ H ) Y_t(\sqrt{N}e_1)=\Phi X_t(\widehat{u}_H) Y t ( N e 1 ) = Φ X t ( u H )
Lemma 6.7 (Gauge-Transformed Hamiltonian). Let m ϕ ( y ) ≔ m ( Φ T y ) m_\phi(y)\coloneqq m(\Phi^{\mathsf T}y) m ϕ ( y ) : = m ( Φ T y ) H ϕ ( y ) ≔ H ( Φ T y ) H_\phi(y)\coloneqq H(\Phi^{\mathsf T}y) H ϕ ( y ) : = H ( Φ T y ) m ϕ ( y ) = 1 N ⟨ ϕ , y ⟩ m_\phi(y)=\frac{1}{\sqrt{N}}\langle\phi,y\rangle m ϕ ( y ) = N 1 ⟨ ϕ , y ⟩ H ϕ = H n u l l + H s i g H_\phi=H_{\mathrm{null}}+H_{\mathrm{sig}} H ϕ = H null + H sig
H n u l l ( y ) ≔ R κ N ⟨ e 1 , y ⟩ + W ( y ) , W ( y ) ≔ H 0 ⊥ ( Φ T y ) , H_{\mathrm{null}}(y)\coloneqq\frac{R}{\kappa_N}\langle e_1,y\rangle+W(y),\qquad W(y)\coloneqq H_0^\perp(\Phi^{\mathsf T}y), H null ( y ) : = κ N R ⟨ e 1 , y ⟩ + W ( y ) , W ( y ) : = H 0 ⊥ ( Φ T y ) , H s i g ( y ) ≔ N λ m ϕ ( y ) k − N λ σ N 2 κ N 2 m ϕ ( y ) . H_{\mathrm{sig}}(y)\coloneqq N\lambda m_\phi(y)^k-\frac{N\lambda\sigma_N^2}{\kappa_N^2}m_\phi(y). H sig ( y ) : = N λ m ϕ ( y ) k − κ N 2 N λ σ N 2 m ϕ ( y ) . Moreover, the following hold:
The Gaussian process W W W ( Φ , R ) (\Phi,R) ( Φ , R ) H 0 ⊥ H_0^\perp H 0 ⊥
For any initial state e ∈ S N e\in\mathcal{S}_N e ∈ S N ( Φ , R , W ) (\Phi,R,W) ( Φ , R , W ) ( Y t ( e ) ) t ≥ 0 (Y_t(e))_{t\ge0} ( Y t ( e ) ) t ≥ 0 H ϕ H_\phi H ϕ β \beta β Y 0 ( e ) = e Y_0(e)=e Y 0 ( e ) = e
Proof. Since Φ \Phi Φ ϕ = Φ θ ⋆ \phi=\Phi\theta^\star ϕ = Φ θ ⋆
m ϕ ( y ) = m ( Φ ⊤ y ) = 1 N ⟨ θ ⋆ , Φ ⊤ y ⟩ = 1 N ⟨ Φ θ ⋆ , y ⟩ = 1 N ⟨ ϕ , y ⟩ . m_\phi(y)=m(\Phi^\top y)=\frac{1}{\sqrt{N}}\langle\theta^\star,\Phi^\top y\rangle=\frac{1}{\sqrt{N}}\langle\Phi\theta^\star,y\rangle=\frac{1}{\sqrt{N}}\langle\phi,y\rangle. m ϕ ( y ) = m ( Φ ⊤ y ) = N 1 ⟨ θ ⋆ , Φ ⊤ y ⟩ = N 1 ⟨ Φ θ ⋆ , y ⟩ = N 1 ⟨ ϕ , y ⟩ . Since k k k Theorem 4.1 that u H = N λ α N θ ⋆ + ζ u_H=N\lambda\alpha_N\theta^\star+\zeta u H = N λ α N θ ⋆ + ζ σ N 2 = κ N N α N ≍ k N k − 1 2 \sigma_N^2=\kappa_N\sqrt{N}\alpha_N\asymp_k N^{\frac{k-1}{2}} σ N 2 = κ N N α N ≍ k N 2 k − 1 ⟨ θ ⋆ , x ⟩ = N m ( x ) \langle\theta^\star,x\rangle=\sqrt{N}m(x) ⟨ θ ⋆ , x ⟩ = N m ( x ) (2) and Lemma 6.6 that
H ( x ) = N λ m ( x ) k + κ N − 1 ⟨ ζ , x ⟩ + H 0 ⊥ ( x ) = κ N − 1 ⟨ u H , x ⟩ + N λ m ( x ) k − N λ σ N 2 κ N 2 m ( x ) + H 0 ⊥ ( x ) . \begin{aligned}
H(x)=N\lambda m(x)^k+\kappa_N^{-1}\langle\zeta,x\rangle+H_0^\perp(x) \\
&=\kappa_N^{-1}\langle u_H,x\rangle+N\lambda m(x)^k-\frac{N\lambda\sigma_N^2}{\kappa_N^2}m(x)+H_0^\perp(x).
\end{aligned} H ( x ) = N λm ( x ) k + κ N − 1 ⟨ ζ , x ⟩ + H 0 ⊥ ( x ) = κ N − 1 ⟨ u H , x ⟩ + N λm ( x ) k − κ N 2 N λ σ N 2 m ( x ) + H 0 ⊥ ( x ) . Moreover, since Φ u H = R e 1 \Phi u_H=Re_1 Φ u H = R e 1
κ N − 1 ⟨ u H , Φ ⊤ y ⟩ = κ N − 1 ⟨ Φ u H , y ⟩ = R κ N ⟨ e 1 , y ⟩ . \kappa_N^{-1}\langle u_H,\Phi^\top y\rangle=\kappa_N^{-1}\langle\Phi u_H,y\rangle=\frac{R}{\kappa_N}\langle e_1,y\rangle. κ N − 1 ⟨ u H , Φ ⊤ y ⟩ = κ N − 1 ⟨ Φ u H , y ⟩ = κ N R ⟨ e 1 , y ⟩ . Consequently,
H ϕ ( y ) = H ( Φ ⊤ y ) = R κ N ⟨ e 1 , y ⟩ + H 0 ⊥ ( Φ ⊤ y ) + N λ m ϕ ( y ) k − N λ σ N 2 κ N 2 m ϕ ( y ) = H n u l l ( y ) + H s i g ( y ) . \begin{aligned}
H_\phi(y)=H(\Phi^\top y)=\frac{R}{\kappa_N}\langle e_1,y\rangle+H_0^\perp(\Phi^\top y)+N\lambda m_\phi(y)^k-\frac{N\lambda\sigma_N^2}{\kappa_N^2}m_\phi(y) \\
&=H_{\mathrm{null}}(y)+H_{\mathrm{sig}}(y).
\end{aligned} H ϕ ( y ) = H ( Φ ⊤ y ) = κ N R ⟨ e 1 , y ⟩ + H 0 ⊥ ( Φ ⊤ y ) + N λ m ϕ ( y ) k − κ N 2 N λ σ N 2 m ϕ ( y ) = H null ( y ) + H sig ( y ) . We now prove the first claim. Since θ ⋆ \theta^\star θ ⋆ (84) shows that ( Φ , R ) (\Phi,R) ( Φ , R ) ( ζ , Ψ ) (\zeta,\Psi) ( ζ , Ψ ) Lemma 6.6 , H 0 ⊥ H_0^\perp H 0 ⊥ ζ \zeta ζ Ψ \Psi Ψ ( ζ , H 0 ⊥ ) (\zeta,H_0^\perp) ( ζ , H 0 ⊥ ) y ↦ Φ ⊤ y y\mapsto\Phi^\top y y ↦ Φ ⊤ y S N \mathbb{S}_N S N Φ ∈ O ( N , R ) \Phi\in O(N,\mathbb{R}) Φ ∈ O ( N , R ) H 0 ⊥ H_0^\perp H 0 ⊥ ⟨ x , y ⟩ \langle x,y\rangle ⟨ x , y ⟩ H 0 ⊥ H_0^\perp H 0 ⊥
Law ( W ∣ Φ , R ) = Law ( H 0 ⊥ ( Φ ⊤ ⋅ ) ∣ Φ , R ) = Law ( H 0 ⊥ ∣ Φ , R ) = Law ( H 0 ⊥ ) . \operatorname{Law}(W\mid\Phi,R)=\operatorname{Law}(H_0^\perp(\Phi^\top\cdot)\mid\Phi,R)=\operatorname{Law}(H_0^\perp\mid\Phi,R)=\operatorname{Law}(H_0^\perp). Law ( W ∣ Φ , R ) = Law ( H 0 ⊥ ( Φ ⊤ ⋅ ) ∣ Φ , R ) = Law ( H 0 ⊥ ∣ Φ , R ) = Law ( H 0 ⊥ ) . Thus, W W W ( Φ , R ) (\Phi,R) ( Φ , R ) H 0 ⊥ H_0^\perp H 0 ⊥
The second claim is a consequence of the rotation invariance of spherical Langevin dynamics. Let ( B t ) t ≥ 0 (B_t)_{t\geq0} ( B t ) t ≥ 0 ( X t ( ⋅ ) ) t ≥ 0 (X_t(\cdot))_{t\geq0} ( X t ( ⋅ ) ) t ≥ 0 X t = X t ( Φ ⊤ e ) X_t=X_t(\Phi^\top e) X t = X t ( Φ ⊤ e ) Y t = Y t ( e ) = Φ X t Y_t=Y_t(e)=\Phi X_t Y t = Y t ( e ) = Φ X t X t X_t X t (18) for the Hamiltonian H H H X 0 = Φ ⊤ e X_0=\Phi^\top e X 0 = Φ ⊤ e Φ \Phi Φ (18) by Φ \Phi Φ X t = Φ ⊤ Y t X_t=\Phi^\top Y_t X t = Φ ⊤ Y t
d Y t = Φ d X t = ( β ⋅ Φ Q X t ∇ H ( X t ) − N − 1 2 N Φ X t ) d t + Φ Q X t d B t = ( β ⋅ ( Φ Q X t Φ ⊤ ) ( Φ ∇ H ( Φ ⊤ Y t ) ) − N − 1 2 N Y t ) d t + ( Φ Q X t Φ ⊤ ) Φ d B t = ( β ⋅ Q Y t ∇ H ϕ ( Y t ) − N − 1 2 N Y t ) d t + Q Y t d ( Φ B t ) , \begin{aligned}
dY_t&=\Phi\,dX_t=\left(\beta\cdot\Phi Q_{X_t}\nabla H(X_t)-\frac{N-1}{2N}\Phi X_t\right)dt+\Phi Q_{X_t}\,dB_t\\
&=\left(\beta\cdot(\Phi Q_{X_t}\Phi^\top)(\Phi\nabla H(\Phi^\top Y_t))-\frac{N-1}{2N}Y_t\right)dt+(\Phi Q_{X_t}\Phi^\top)\Phi\,dB_t\\
&=\left(\beta\cdot Q_{Y_t}\nabla H_\phi(Y_t)-\frac{N-1}{2N}Y_t\right)dt+Q_{Y_t}\,d(\Phi B_t),
\end{aligned} d Y t = Φ d X t = ( β ⋅ Φ Q X t ∇ H ( X t ) − 2 N N − 1 Φ X t ) d t + Φ Q X t d B t = ( β ⋅ ( Φ Q X t Φ ⊤ ) ( Φ∇ H ( Φ ⊤ Y t )) − 2 N N − 1 Y t ) d t + ( Φ Q X t Φ ⊤ ) Φ d B t = ( β ⋅ Q Y t ∇ H ϕ ( Y t ) − 2 N N − 1 Y t ) d t + Q Y t d ( Φ B t ) , where the last equality uses Φ Q x Φ ⊤ = Q Φ x \Phi Q_x\Phi^\top=Q_{\Phi x} Φ Q x Φ ⊤ = Q Φ x x ∈ R N ∖ { 0 } x\in\mathbb{R}^N\setminus\{0\} x ∈ R N ∖ { 0 } ∇ H ϕ ( y ) = Φ ∇ H ( Φ ⊤ y ) \nabla H_\phi(y)=\Phi\nabla H(\Phi^\top y) ∇ H ϕ ( y ) = Φ∇ H ( Φ ⊤ y ) Y 0 = e Y_0=e Y 0 = e ( B t ) t ≥ 0 (B_t)_{t\geq0} ( B t ) t ≥ 0 ( Φ , R , W , e ) (\Phi,R,W,e) ( Φ , R , W , e ) ( Φ B t ) t ≥ 0 (\Phi B_t)_{t\geq0} ( Φ B t ) t ≥ 0 ( Φ , R , W , e ) (\Phi,R,W,e) ( Φ , R , W , e ) ( Y t ) t ≥ 0 (Y_t)_{t\geq0} ( Y t ) t ≥ 0 H ϕ H_\phi H ϕ β \beta β Y 0 = e Y_0=e Y 0 = e ( Φ B t ) t ≥ 0 (\Phi B_t)_{t\geq0} ( Φ B t ) t ≥ 0
Lemma 6.7 shows that, modulo the lower order linear terms κ N − 1 R ⟨ e 1 , y ⟩ \kappa_N^{-1}R\langle e_1,y\rangle κ N − 1 R ⟨ e 1 , y ⟩ − N λ κ N − 2 σ N 2 m ϕ ( y ) -N\lambda\kappa_N^{-2}\sigma_N^2m_\phi(y) − N λ κ N − 2 σ N 2 m ϕ ( y ) H H H N ϕ \sqrt{N}\phi N ϕ W W W ( Φ , R ) (\Phi,R) ( Φ , R ) H ϕ = H n u l l + H s i g H_\phi=H_{\mathrm{null}}+H_{\mathrm{sig}} H ϕ = H null + H sig H ϕ H_\phi H ϕ ϕ \phi ϕ H s i g H_{\mathrm{sig}} H sig H n u l l H_{\mathrm{null}} H null H H H κ N − 1 R ⟨ e 1 , y ⟩ \kappa_N^{-1}R\langle e_1,y\rangle κ N − 1 R ⟨ e 1 , y ⟩
The Reverse Planting Trick and Adaptive Contiguity
We use L p l ≔ L H ϕ , β \mathcal{L}_{\mathrm{pl}}\coloneqq\mathcal{L}_{H_\phi,\beta} L pl : = L H ϕ , β L n u l l ≔ L H n u l l , β \mathcal{L}_{\mathrm{null}}\coloneqq\mathcal{L}_{H_{\mathrm{null}},\beta} L null : = L H null , β β \beta β H ϕ H_\phi H ϕ H n u l l H_{\mathrm{null}} H null ( L n u l l m ϕ ) ( Y t ( e ) ) (\mathcal{L}_{\mathrm{null}}m_\phi)(Y_t(e)) ( L null m ϕ ) ( Y t ( e )) m ϕ m_\phi m ϕ ( L n u l l m ϕ ) ( Y t ( e ) ) (\mathcal{L}_{\mathrm{null}}m_\phi)(Y_t(e)) ( L null m ϕ ) ( Y t ( e )) m ϕ m_\phi m ϕ (8) implies ∇ S N m ϕ ( y ) = N − 1 / 2 Q y ϕ \nabla_{\mathbb{S}_N}m_\phi(y)=N^{-1/2}Q_y\phi ∇ S N m ϕ ( y ) = N − 1/2 Q y ϕ Δ S N m ϕ = − κ N m ϕ \Delta_{\mathbb{S}_N}m_\phi=-\kappa_Nm_\phi Δ S N m ϕ = − κ N m ϕ (19) for every y ∈ S N y\in\mathbb{S}_N y ∈ S N
( L n u l l m ϕ ) ( y ) = − κ N 2 m ϕ ( y ) + β ⟨ ∇ S N H n u l l ( y ) , ∇ S N m ϕ ( y ) ⟩ = 1 N ⟨ ϕ , β ∇ S N H n u l l ( y ) − κ N 2 y ⟩ . (\mathcal{L}_{\mathrm{null}}m_\phi)(y)=-\frac{\kappa_N}{2}m_\phi(y)+\beta\langle\nabla_{\mathbb{S}_N}H_{\mathrm{null}}(y),\nabla_{\mathbb{S}_N}m_\phi(y)\rangle=\frac{1}{\sqrt{N}}\left\langle\phi,\beta\nabla_{\mathbb{S}_N}H_{\mathrm{null}}(y)-\frac{\kappa_N}{2}y\right\rangle. ( L null m ϕ ) ( y ) = − 2 κ N m ϕ ( y ) + β ⟨ ∇ S N H null ( y ) , ∇ S N m ϕ ( y )⟩ = N 1 ⟨ ϕ , β ∇ S N H null ( y ) − 2 κ N y ⟩ . Although the law of ϕ \phi ϕ e 1 e_1 e 1 ϕ \phi ϕ ∇ S N H n u l l ( y ) = κ N − 1 R Q y e 1 + ∇ S N W ( y ) \nabla_{\mathbb{S}_N}H_{\mathrm{null}}(y)=\kappa_N^{-1}RQ_ye_1+\nabla_{\mathbb{S}_N}W(y) ∇ S N H null ( y ) = κ N − 1 R Q y e 1 + ∇ S N W ( y ) R R R (84) . Moreover, the trajectory Y t ( e ) Y_t(e) Y t ( e ) ϕ \phi ϕ H s i g H_{\mathrm{sig}} H sig e e e ϕ \phi ϕ
We circumvent these difficulties via a change of measure to a carefully constructed null law, under which ϕ \phi ϕ S N − 1 \mathbb{S}^{N-1} S N − 1 N e 1 \sqrt{N}e_1 N e 1
Since the initialization considered in Theorem 6.5 may depend on the disorder, we fix an initialization rule , i.e., a random initial state of the form e = e ( Φ , R , W , ω ) ∈ S N e=e(\Phi,R,W,\omega)\in\mathbb{S}_N e = e ( Φ , R , W , ω ) ∈ S N e e e ω \omega ω ( Φ , R , W ) (\Phi,R,W) ( Φ , R , W )
We formally define both the law of the true dynamics (which we call the planted law) and this null law, representing them as laws of spherical Langevin dynamics in the random environment ( Φ , R , W ) (\Phi,R,W) ( Φ , R , W )
Definition 6.8 (Planted Law). The planted law P \mathbb{P} P P = P e n v ( Φ , R , W ) ⋅ P d y n ( ⋅ ∣ Φ , R , W ) \mathbb{P}=\mathbb{P}_{\mathrm{env}}(\Phi,R,W)\cdot\mathbb{P}_{\mathrm{dyn}}(\cdot\mid\Phi,R,W) P = P env ( Φ , R , W ) ⋅ P dyn ( ⋅ ∣ Φ , R , W ) P e n v \mathbb{P}_{\mathrm{env}} P env P d y n \mathbb{P}_{\mathrm{dyn}} P dyn
R = ∥ U ∥ R=\lVert U\rVert R = ∥ U ∥ Φ = Ψ Λ ( U ) \Phi=\Psi\Lambda(U) Φ = ΨΛ ( U ) U ∼ N ( N λ α N θ ⋆ , σ N 2 I ) U\sim\mathcal{N}(N\lambda\alpha_N\theta^\star,\sigma_N^2I) U ∼ N ( N λ α N θ ⋆ , σ N 2 I ) Ψ \Psi Ψ { S ∈ O ( N , R ) : S e 1 = e 1 } \{S\in O(N,\mathbb{R}):Se_1=e_1\} { S ∈ O ( N , R ) : S e 1 = e 1 } U U U
W W W ( Φ , R ) (\Phi,R) ( Φ , R ) H 0 ⊥ H_0^\perp H 0 ⊥
Conditional on the environment ( Φ , R , W ) (\Phi,R,W) ( Φ , R , W ) P d y n \mathbb{P}_{\mathrm{dyn}} P dyn β \beta β
H ϕ ( y ) = κ N − 1 R ⟨ e 1 , y ⟩ + W ( y ) + N λ m ϕ ( y ) k − N λ κ N − 2 σ N 2 m ϕ ( y ) , ϕ = Φ Θ ⋆ , H_\phi(y)=\kappa_N^{-1}R\langle e_1,y\rangle+W(y)+N\lambda m_\phi(y)^k-N\lambda\kappa_N^{-2}\sigma_N^2m_\phi(y),\qquad\phi=\Phi\Theta^\star, H ϕ ( y ) = κ N − 1 R ⟨ e 1 , y ⟩ + W ( y ) + N λ m ϕ ( y ) k − N λ κ N − 2 σ N 2 m ϕ ( y ) , ϕ = Φ Θ ⋆ , initialized at Y 0 = e ( Φ , R , W , ω ) Y_0=e(\Phi,R,W,\omega) Y 0 = e ( Φ , R , W , ω ) L p l \mathcal{L}_{\mathrm{pl}} L pl
Definition 6.9 (Null Law). The null law Q \mathbb{Q} Q Q = Q e n v ( Φ , R , W ) ⋅ Q d y n ( ⋅ ∣ Φ , R , W ) \mathbb{Q}=\mathbb{Q}_{\mathrm{env}}(\Phi,R,W)\cdot\mathbb{Q}_{\mathrm{dyn}}(\cdot\mid\Phi,R,W) Q = Q env ( Φ , R , W ) ⋅ Q dyn ( ⋅ ∣ Φ , R , W ) Q e n v \mathbb{Q}_{\mathrm{env}} Q env Q d y n \mathbb{Q}_{\mathrm{dyn}} Q dyn
R = ∥ U ∥ R=\lVert U\rVert R = ∥ U ∥ Φ = Ψ Λ ( U ) \Phi=\Psi\Lambda(U) Φ = ΨΛ ( U ) U ∼ N ( 0 , σ N 2 I ) U\sim\mathcal{N}(0,\sigma_N^2 I) U ∼ N ( 0 , σ N 2 I ) Ψ \Psi Ψ { S ∈ O ( N , R ) : S e 1 = e 1 } \{S\in O(N,\mathbb{R}):Se_1=e_1\} { S ∈ O ( N , R ) : S e 1 = e 1 } U U U
W W W ( Φ , R ) (\Phi,R) ( Φ , R ) H 0 ⊥ H_0^\perp H 0 ⊥
Conditional on the environment ( Φ , R , W ) (\Phi,R,W) ( Φ , R , W ) Q d y n \mathbb{Q}_{\mathrm{dyn}} Q dyn β \beta β H n u l l ( y ) = κ N − 1 R ⟨ e 1 , y ⟩ + W ( y ) H_{\mathrm{null}}(y)=\kappa_N^{-1}R\langle e_1,y\rangle+W(y) H null ( y ) = κ N − 1 R ⟨ e 1 , y ⟩ + W ( y ) Y 0 = e ( Φ , R , W , ω ) Y_0=e(\Phi,R,W,\omega) Y 0 = e ( Φ , R , W , ω ) L n u l l \mathcal{L}_{\mathrm{null}} L null
Under both laws, we write Y t = Y t ( e ) Y_t=Y_t(e) Y t = Y t ( e ) Theorem 4.1 and Lemma 6.7 , P \mathbb{P} P e = N e 1 e=\sqrt{N}e_1 e = N e 1 P \mathbb{P} P Φ ⊤ Y t ( N e 1 ) \Phi^\top Y_t(\sqrt{N}e_1) Φ ⊤ Y t ( N e 1 ) H H H u ˉ H = N Φ ⊤ e 1 \bar{u}_H=\sqrt{N}\Phi^\top e_1 u ˉ H = N Φ ⊤ e 1 Q \mathbb{Q} Q Φ ⊤ Y t ( N e 1 ) \Phi^\top Y_t(\sqrt{N}e_1) Φ ⊤ Y t ( N e 1 )
H ~ ( x ) : = H n u l l ( Φ x ) = κ N − 1 ⟨ ζ ~ , x ⟩ + W ( Φ x ) , ζ ~ : = R Φ ⊤ e 1 , \widetilde{H}(x):=H_{\mathrm{null}}(\Phi x)=\kappa_N^{-1}\langle\widetilde{\zeta},x\rangle+W(\Phi x),\qquad\widetilde{\zeta}:=R\Phi^\top e_1, H ( x ) := H null ( Φ x ) = κ N − 1 ⟨ ζ , x ⟩ + W ( Φ x ) , ζ := R Φ ⊤ e 1 , initialized at N Φ ⊤ e 1 = N ζ ~ / ∥ ζ ~ ∥ \sqrt{N}\Phi^\top e_1=\sqrt{N}\widetilde{\zeta}/\lVert\widetilde{\zeta}\rVert N Φ ⊤ e 1 = N ζ / ∥ ζ ∥ Φ U = Ψ Λ ( U ) U = R e 1 \Phi U=\Psi\Lambda(U)U=Re_1 Φ U = ΨΛ ( U ) U = R e 1 ζ ~ = U ∼ N ( 0 , σ N 2 I ) \widetilde{\zeta}=U\sim\mathcal{N}(0,\sigma_N^2I) ζ = U ∼ N ( 0 , σ N 2 I ) Q \mathbb{Q} Q W W W ( Φ , R ) (\Phi,R) ( Φ , R ) W ( Φ ⋅ ) W(\Phi\cdot) W ( Φ ⋅ ) ζ ~ \widetilde{\zeta} ζ H 0 ⊥ H_0^\perp H 0 ⊥ Lemma 6.6 , we observe that Law ( H ~ , ζ ~ ) = Law ( H 0 , ζ ) \operatorname{Law}(\widetilde{H},\widetilde{\zeta})=\operatorname{Law}(H_0,\zeta) Law ( H , ζ ) = Law ( H 0 , ζ ) ζ = E π [ ∇ S N H 0 ] \zeta=\mathbb{E}_{\pi}[\nabla_{\mathbb{S}^N}H_0] ζ = E π [ ∇ S N H 0 ] ζ ~ \widetilde{\zeta} ζ H ~ \widetilde{H} H
Thus, we conclude that both the planted and null laws describe spherical Langevin dynamics initialized at the normalized average gradient of their respective underlying Hamiltonians, which correspond to the true tensor PCA Hamiltonian under P \mathbb{P} P Q \mathbb{Q} Q Q \mathbb{Q} Q P \mathbb{P} P λ = 0 \lambda=0 λ = 0 U U U H s i g H_{\mathrm{sig}} H sig θ ⋆ \theta^\star θ ⋆
Moreover, the randomization over Ψ \Psi Ψ Φ \Phi Φ ( R , W ) (R,W) ( R , W ) H n u l l H_{\mathrm{null}} H null ϕ \phi ϕ ∇ S N H n u l l \nabla_{\mathbb{S}^N}H_{\mathrm{null}} ∇ S N H null R R R
Lemma 6.10 (Decoupling of the Null Environment). Under the null law Q \mathbb{Q} Q Φ \Phi Φ O ( N , R ) O(N,\mathbb{R}) O ( N , R ) Φ \Phi Φ R R R W W W ϕ = Φ θ ⋆ \phi=\Phi\theta^\star ϕ = Φ θ ⋆ S N − 1 \mathbb{S}^{N-1} S N − 1 ( R , W ) (R,W) ( R , W )
Proof. Throughout, we work under Q \mathbb{Q} Q U ∼ N ( 0 , σ N 2 I ) U\sim\mathcal{N}(0,\sigma_N^2I) U ∼ N ( 0 , σ N 2 I ) U / R U/R U / R S N − 1 \mathbb{S}^{N-1} S N − 1 R R R [48 Exercise 3.3.7] . Moreover, as noted in § 6.3 , Λ ( U ) \Lambda(U) Λ ( U ) U U U U / R U/R U / R Φ = Ψ Λ ( U ) \Phi=\Psi\Lambda(U) Φ = ΨΛ ( U ) ( Ψ , U / R ) (\Psi,U/R) ( Ψ , U / R ) R R R Ψ \Psi Ψ U U U Λ ( U ) \Lambda(U) Λ ( U ) Λ ( U ) ( U / R ) = e 1 \Lambda(U)(U/R)=e_1 Λ ( U ) ( U / R ) = e 1 Λ ( U ) e 1 = U / R \Lambda(U)e_1=U/R Λ ( U ) e 1 = U / R Φ ⊤ = Λ ( U ) Ψ ⊤ \Phi^\top=\Lambda(U)\Psi^\top Φ ⊤ = Λ ( U ) Ψ ⊤ S N − 1 \mathbb{S}^{N-1} S N − 1 Ψ T \Psi^{\mathsf{T}} Ψ T { S ∈ O ( N , R ) : S e 1 = e 1 } ≅ O ( N − 1 , R ) \{S \in O(N,\mathbb{R}) : Se_1=e_1\} \cong O(N-1,\mathbb{R}) { S ∈ O ( N , R ) : S e 1 = e 1 } ≅ O ( N − 1 , R ) O ( N , R ) O(N,\mathbb{R}) O ( N , R ) Φ \Phi Φ O ( N , R ) O(N,\mathbb{R}) O ( N , R )
Hence, we conclude that Φ \Phi Φ R R R W W W ( Φ , R ) (\Phi,R) ( Φ , R ) Definition 6.9 , the random variables Φ \Phi Φ R R R W W W θ ⋆ ∈ S N − 1 \theta^\star\in\mathbb{S}^{N-1} θ ⋆ ∈ S N − 1 ϕ = Φ θ ⋆ \phi=\Phi\theta^\star ϕ = Φ θ ⋆ S N − 1 \mathbb{S}^{N-1} S N − 1 Φ \Phi Φ ( R , W ) (R,W) ( R , W ) ϕ \phi ϕ
By Lemma 6.10 , ϕ \phi ϕ N e 1 \sqrt{N}e_1 N e 1 H n u l l H_{\mathrm{null}} H null L n u l l m ϕ \mathcal{L}_{\mathrm{null}}m_\phi L null m ϕ
Lemma 6.11 (Uniform Concentration along the Null Trajectory). For every C 0 > 0 C_0>0 C 0 > 0 there exist constants c ⋆ , C ⋆ > 0 c_\star,C_\star>0 c ⋆ , C ⋆ > 0 depending only on β \beta β and C 0 C_0 C 0 such that the following holds for every T ∈ ( 0 , 1 ] T\in(0,1] T ∈ ( 0 , 1 ] every δ ≥ 0 \delta\geq0 δ ≥ 0 and every initialization rule e e e possibly randomized:
E Q [ exp ( c ⋆ N sup t ∈ [ 0 , T ] ∣ ( L n u l l m ϕ ) ( Y t ( e ) ) ∣ 2 ) 1 { ∥ H n u l l ∥ G 4 ≤ C 0 N , ∥ e − N e 1 ∥ ≤ δ N } ] ≤ C ⋆ exp ( C ⋆ N δ 2 ) . \mathbb{E}_{\mathbb{Q}}\left[\exp\left(c_\star N\sup_{t\in[0,T]}\left|(\mathcal{L}_{\mathrm{null}}m_\phi)(Y_t(e))\right|^2\right)\mathbf{1}\left\{\lVert H_{\mathrm{null}}\rVert_{\mathcal{G}_4}\leq C_0N,\ \lVert e-\sqrt{N}e_1\rVert\leq\delta\sqrt{N}\right\}\right]\leq C_\star\exp(C_\star N\delta^2). E Q [ exp ( c ⋆ N t ∈ [ 0 , T ] sup ∣ ( L null m ϕ ) ( Y t ( e )) ∣ 2 ) 1 { ∥ H null ∥ G 4 ≤ C 0 N , ∥ e − N e 1 ∥ ≤ δ N } ] ≤ C ⋆ exp ( C ⋆ N δ 2 ) . Proof. Throughout, we work under Q \mathbb{Q} Q E r e g = { ∥ H n u l l ∥ G 4 ≤ C 0 N } \mathcal{E}_{\mathrm{reg}}=\{\lVert H_{\mathrm{null}}\rVert_{\mathcal{G}_4}\leq C_0N\} E reg = {∥ H null ∥ G 4 ≤ C 0 N } N e 1 \sqrt{N}e_1 N e 1 ϕ \phi ϕ e e e ϕ \phi ϕ
We begin with uniform bounds on E r e g \mathcal{E}_{\mathrm{reg}} E reg
Since ∥ β H n u l l ∥ G 4 ≤ β C 0 N \lVert\beta H_{\mathrm{null}}\rVert_{\mathcal{G}_4}\leq\beta C_0N ∥ β H null ∥ G 4 ≤ β C 0 N E r e g \mathcal{E}_{\mathrm{reg}} E reg Lemma 3.6 with g 1 = β H n u l l g_1=\beta H_{\mathrm{null}} g 1 = β H null Lemma 3.9 applied to m ϕ m_\phi m ϕ ∥ L n u l l m ϕ ∥ G 1 ≤ K \lVert\mathcal{L}_{\mathrm{null}}m_\phi\rVert_{\mathcal{G}_1}\leq K ∥ L null m ϕ ∥ G 1 ≤ K ∥ L n u l l 2 m ϕ ∥ G 0 ≤ K \lVert\mathcal{L}_{\mathrm{null}}^2m_\phi\rVert_{\mathcal{G}_0}\leq K ∥ L null 2 m ϕ ∥ G 0 ≤ K E r e g \mathcal{E}_{\mathrm{reg}} E reg K > 0 K>0 K > 0 β \beta β C 0 C_0 C 0
Moreover, as m ϕ m_\phi m ϕ L n u l l \mathcal{L}_{\mathrm{null}} L null (8) and (19) that the following holds for every y ∈ S N y\in\mathbb{S}_N y ∈ S N
( L n u l l m ϕ ) ( y ) = 1 N ⟨ ϕ , F ( y ) ⟩ , F ( y ) = β ∇ S N H n u l l ( y ) − κ N 2 y , ( L n u l l 2 m ϕ ) ( y ) = 1 N ⟨ ϕ , ( L n u l l F ) ( y ) ⟩ , ∇ S N ( L n u l l m ϕ ) ( y ) = 1 N ∇ S N F ( y ) ϕ . (86) \begin{aligned}
(\mathcal{L}_{\mathrm{null}}m_\phi)(y)&=\frac{1}{\sqrt{N}}\langle\phi,F(y)\rangle,\qquad F(y)=\beta\nabla_{\mathbb{S}_N}H_{\mathrm{null}}(y)-\frac{\kappa_N}{2}y,\\
(\mathcal{L}_{\mathrm{null}}^2m_\phi)(y)&=\frac{1}{\sqrt{N}}\langle\phi,(\mathcal{L}_{\mathrm{null}}F)(y)\rangle,\qquad\nabla_{\mathbb{S}_N}(\mathcal{L}_{\mathrm{null}}m_\phi)(y)=\frac{1}{\sqrt{N}}\nabla_{\mathbb{S}_N}F(y)\phi. \tag*{(86)}
\end{aligned} ( L null m ϕ ) ( y ) ( L null 2 m ϕ ) ( y ) = N 1 ⟨ ϕ , F ( y )⟩ , F ( y ) = β ∇ S N H null ( y ) − 2 κ N y , = N 1 ⟨ ϕ , ( L null F ) ( y )⟩ , ∇ S N ( L null m ϕ ) ( y ) = N 1 ∇ S N F ( y ) ϕ . (86) Since the above bounds hold with any unit vector in place of ϕ \phi ϕ ϕ ∈ S N − 1 \phi\in\mathbb{S}^{N-1} ϕ ∈ S N − 1 (86) and using Definition 3.5 , we obtain the following on E r e g \mathcal{E}_{\mathrm{reg}} E reg y ∈ S N y\in\mathbb{S}_N y ∈ S N
∥ F ( y ) ∥ ≤ K N , ∥ ( L n u l l F ) ( y ) ∥ ≤ K N , ∥ ∇ S N F ( y ) ∥ o p ≤ K . (87) \lVert F(y)\rVert\leq K\sqrt{N},\qquad\lVert(\mathcal{L}_{\mathrm{null}}F)(y)\rVert\leq K\sqrt{N},\qquad\lVert\nabla_{\mathbb{S}_N}F(y)\rVert_{\mathrm{op}}\leq K. \tag*{(87)} ∥ F ( y )∥ ≤ K N , ∥( L null F ) ( y )∥ ≤ K N , ∥ ∇ S N F ( y ) ∥ op ≤ K . (87) For the first step, consider the null trajectory ( Y t ( N e 1 ) ) t ≥ 0 (Y_t(\sqrt{N}e_1))_{t\geq0} ( Y t ( N e 1 ) ) t ≥ 0 N e 1 \sqrt{N}e_1 N e 1 ( B t ) t ≥ 0 (B_t)_{t\geq0} ( B t ) t ≥ 0 Lemma 6.10 , ϕ \phi ϕ S N − 1 \mathbb{S}^{N-1} S N − 1 ( R , W , B ) (R,W,B) ( R , W , B ) E r e g \mathcal{E}_{\mathrm{reg}} E reg ( Y t ( N e 1 ) ) t ≥ 0 (Y_t(\sqrt{N}e_1))_{t\geq0} ( Y t ( N e 1 ) ) t ≥ 0 ( R , W , B ) (R,W,B) ( R , W , B ) H n u l l H_{\mathrm{null}} H null ( R , W ) (R,W) ( R , W ) N ϕ ∼ π \sqrt{N}\phi\sim\pi N ϕ ∼ π ( R , W , B ) (R,W,B) ( R , W , B ) v ∈ R N v \in\mathbb{R}^{N} v ∈ R N ∥ v ∥ ≤ K N \lVert v\rVert\le K\sqrt{N} ∥ v ∥ ≤ K N E r e g \mathcal{E}_{\mathrm{reg}} E reg c > 0 c>0 c > 0
E Φ [ exp ( c ⟨ ϕ , v ⟩ 2 K 2 ) 1 { E r e g } ] ≤ 2. (88) \mathbb{E}_{\Phi}\left[\exp\left(\frac{c\langle\phi,v\rangle^{2}}{K^{2}}\right)\mathbf{1}\{\mathcal{E}_{\mathrm{reg}}\}\right]\le2. \tag*{(88)} E Φ [ exp ( K 2 c ⟨ ϕ , v ⟩ 2 ) 1 { E reg } ] ≤ 2. (88) To control the supremum over t ∈ [ 0 , T ] t\in[0,T] t ∈ [ 0 , T ] M t M_t M t L n u l l m ϕ \mathcal{L}_{\mathrm{null}}m_{\phi} L null m ϕ Y t ( N e 1 ) Y_t(\sqrt{N}e_1) Y t ( N e 1 ) Lemma 3.11 conditionally on the environment, we obtain the following for every t ≥ 0 t\ge0 t ≥ 0
( L n u l l m ϕ ) ( Y t ( N e 1 ) ) = ( L n u l l m ϕ ) ( N e 1 ) + ∫ 0 t ( L n u l l 2 m ϕ ) ( Y s ( N e 1 ) ) d s + M t , (\mathcal{L}_{\mathrm{null}}m_{\phi})(Y_t(\sqrt{N}e_1))=(\mathcal{L}_{\mathrm{null}}m_{\phi})(\sqrt{N}e_1)+\int_{0}^{t}(\mathcal{L}_{\mathrm{null}}^{2}m_{\phi})(Y_s(\sqrt{N}e_1))\,\mathrm{d}s+M_t, ( L null m ϕ ) ( Y t ( N e 1 )) = ( L null m ϕ ) ( N e 1 ) + ∫ 0 t ( L null 2 m ϕ ) ( Y s ( N e 1 )) d s + M t , sup t ∈ [ 0 , T ] ∣ ( L n u l l m ϕ ) ( Y t ( N e 1 ) ) ∣ ≤ ∣ ( L n u l l m ϕ ) ( N e 1 ) ∣ + ∫ 0 T ∣ ( L n u l l 2 m ϕ ) ( Y s ( N e 1 ) ) ∣ d s + sup t ∈ [ 0 , T ] ∣ M t ∣ . (89) \sup_{t\in[0,T]}\left|(\mathcal{L}_{\mathrm{null}}m_{\phi})(Y_t(\sqrt{N}e_1))\right|\leq\left|(\mathcal{L}_{\mathrm{null}}m_{\phi})(\sqrt{N}e_1)\right|+\int_{0}^{T}\left|(\mathcal{L}_{\mathrm{null}}^{2}m_{\phi})(Y_s(\sqrt{N}e_1))\right|\,\mathrm{d}s+\sup_{t\in[0,T]}|M_t|. \tag*{(89)} t ∈ [ 0 , T ] sup ( L null m ϕ ) ( Y t ( N e 1 )) ≤ ( L null m ϕ ) ( N e 1 ) + ∫ 0 T ( L null 2 m ϕ ) ( Y s ( N e 1 )) d s + t ∈ [ 0 , T ] sup ∣ M t ∣. (89) By Equation (86) , the first term and the integrand of the second term on the right-hand side of Equation (89) are projections of vectors that are determined by ( R , W , B ) (R,W,B) ( R , W , B ) ϕ \phi ϕ N − 1 / 2 N^{-1/2} N − 1/2 K 2 T / N K^{2}T/N K 2 T / N E r e g \mathcal{E}_{\mathrm{reg}} E reg N − 1 / 2 N^{-1/2} N − 1/2 (88) with v = F ( N e 1 ) v=F(\sqrt{N}e_1) v = F ( N e 1 )
E Φ [ exp ( c N K 2 ∣ ( L n u l l m ϕ ) ( N e 1 ) ∣ 2 ) 1 { E r e g } ] ≤ 2. (90) \mathbb{E}_{\Phi}\left[\exp\left(\frac{cN}{K^{2}}\left|(\mathcal{L}_{\mathrm{null}}m_{\phi})(\sqrt{N}e_1)\right|^{2}\right)\mathbf{1}\{\mathcal{E}_{\mathrm{reg}}\}\right]\leq2. \tag*{(90)} E Φ [ exp ( K 2 c N ( L null m ϕ ) ( N e 1 ) 2 ) 1 { E reg } ] ≤ 2. (90) For the second term, the Cauchy–Schwarz inequality and Jensen’s inequality imply
exp ( c N K 2 ( ∫ 0 T ∣ ( L n u l l 2 m ϕ ) ( Y s ( N e 1 ) ) ∣ d s ) 2 ) ≤ exp ( c N T K 2 ∫ 0 T ∣ ( L n u l l 2 m ϕ ) ( Y s ( N e 1 ) ) ∣ 2 d s ) ≤ 1 T ∫ 0 T exp ( c N T 2 K 2 ( L n u l l 2 m ϕ ) ( Y s ( N e 1 ) ) 2 ) d s . (91) \begin{aligned}
\exp\left(\frac{cN}{K^{2}}\left(\int_{0}^{T}\left|(\mathcal{L}_{\mathrm{null}}^{2}m_{\phi})(Y_s(\sqrt{N}e_1))\right|\,\mathrm{d}s\right)^{2}\right)
\leq\exp\left(\frac{cNT}{K^{2}}\int_{0}^{T}\left|(\mathcal{L}_{\mathrm{null}}^{2}m_{\phi})(Y_s(\sqrt{N}e_1))\right|^{2}\,\mathrm{d}s\right) \\
&\leq\frac{1}{T}\int_{0}^{T}\exp\left(\frac{cNT^{2}}{K^{2}}\left(\mathcal{L}_{\mathrm{null}}^{2}m_{\phi}\right)(Y_s(\sqrt{N}e_1))^{2}\right)\,\mathrm{d}s. \tag*{(91)}
\end{aligned} exp K 2 c N ( ∫ 0 T ( L null 2 m ϕ ) ( Y s ( N e 1 )) d s ) 2 ≤ exp ( K 2 c N T ∫ 0 T ( L null 2 m ϕ ) ( Y s ( N e 1 )) 2 d s ) ≤ T 1 ∫ 0 T exp ( K 2 c N T 2 ( L null 2 m ϕ ) ( Y s ( N e 1 ) ) 2 ) d s . (91) Taking expectations on both sides and applying Equation (88) with v = ( L n u l l F ) ( Y s ( N e 1 ) ) v=(\mathcal{L}_{\mathrm{null}}F)(Y_s(\sqrt{N}e_1)) v = ( L null F ) ( Y s ( N e 1 )) s ∈ [ 0 , T ] s\in[0,T] s ∈ [ 0 , T ] T ≤ 1 T\leq1 T ≤ 1
E Φ [ exp ( c N K 2 ( ∫ 0 T ∣ ( L n u l l 2 m ϕ ) ( Y s ( N e 1 ) ) ∣ d s ) 2 ) 1 { E r e g } ] ≤ 2. (92) \mathbb{E}_{\Phi}\left[\exp\left(\frac{cN}{K^{2}}\left(\int_{0}^{T}\left|(\mathcal{L}_{\mathrm{null}}^{2}m_{\phi})(Y_s(\sqrt{N}e_1))\right|\,\mathrm{d}s\right)^{2}\right)\mathbf{1}\{\mathcal{E}_{\mathrm{reg}}\}\right]\leq2. \tag*{(92)} E Φ exp K 2 c N ( ∫ 0 T ( L null 2 m ϕ ) ( Y s ( N e 1 )) d s ) 2 1 { E reg } ≤ 2. (92) For the third term, since ∥ ∇ S N ( L n u l l m ϕ ) ∥ ≤ K / N \lVert\nabla_{\mathcal{S}_{N}}(\mathcal{L}_{\mathrm{null}}m_{\phi})\rVert\leq K/\sqrt{N} ∥ ∇ S N ( L null m ϕ )∥ ≤ K / N E r e g \mathcal{E}_{\mathrm{reg}} E reg Lemma 3.12 applied conditionally on the environment implies the following for every r > 0 r>0 r > 0 E r e g \mathcal{E}_{\mathrm{reg}} E reg
Q [ sup t ∈ [ 0 , T ] ∣ M t ∣ ≥ r | Φ , R , W ] ≤ 2 exp ( − N r 2 2 K 2 T ) . (93) \mathbb{Q}\left[\sup_{t\in[0,T]}|M_t|\geq r\;\middle|\;\Phi,R,W\right]\leq2\exp\left(-\frac{Nr^{2}}{2K^{2}T}\right). \tag*{(93)} Q [ t ∈ [ 0 , T ] sup ∣ M t ∣ ≥ r Φ , R , W ] ≤ 2 exp ( − 2 K 2 T N r 2 ) . (93) Since T ≤ 1 T\leq1 T ≤ 1 c c c (93) conditionally on the environment [cf. Ver18, Proposition 2.5.2], we conclude:
E Φ [ exp ( c N K 2 sup t ∈ [ 0 , T ] ∣ M t ∣ 2 ) 1 { E r e g } ] ≤ 2. (94) \mathbb{E}_{\Phi}\left[\exp\left(\frac{cN}{K^{2}}\sup_{t\in[0,T]}|M_t|^{2}\right)\mathbf{1}\{\mathcal{E}_{\mathrm{reg}}\}\right]\leq2. \tag*{(94)} E Φ [ exp ( K 2 c N t ∈ [ 0 , T ] sup ∣ M t ∣ 2 ) 1 { E reg } ] ≤ 2. (94) Let a 1 a_1 a 1 a 2 a_2 a 2 a 3 a_3 a 3 (89) . Then
E Q [ exp ( c N 9 K 2 sup t ∈ [ 0 , T ] ∣ ( L n u l l m ϕ ) ( Y t ( N e 1 ) ) ∣ 2 ) 1 { E r e g } ] ≤ E Q [ ∏ i = 1 3 ( exp ( c N 3 K 2 a i 2 ) 1 { E r e g } ) ] ≤ ∏ i = 1 3 E Q [ exp ( c N K 2 a i 2 ) 1 { E r e g } ] 1 / 3 ≤ 2. (95) \mathbb{E}_{\mathbb{Q}}\left[\exp\left(\frac{cN}{9K^2}\sup_{t\in[0,T]}\left|(\mathcal{L}_{\mathrm{null}}m_\phi)(Y_t(\sqrt{N}e_1))\right|^2\right)\mathbf{1}_{\{\mathcal{E}_{\mathrm{reg}}\}}\right]
\leq\mathbb{E}_{\mathbb{Q}}\left[\prod_{i=1}^{3}\left(\exp\left(\frac{cN}{3K^2}a_i^2\right)\mathbf{1}_{\{\mathcal{E}_{\mathrm{reg}}\}}\right)\right]
\leq\prod_{i=1}^{3}\mathbb{E}_{\mathbb{Q}}\left[\exp\left(\frac{cN}{K^2}a_i^2\right)\mathbf{1}_{\{\mathcal{E}_{\mathrm{reg}}\}}\right]^{1/3}\leq2. \tag*{(95)} E Q [ exp ( 9 K 2 c N t ∈ [ 0 , T ] sup ( L null m ϕ ) ( Y t ( N e 1 )) 2 ) 1 { E reg } ] ≤ E Q [ i = 1 ∏ 3 ( exp ( 3 K 2 c N a i 2 ) 1 { E reg } ) ] ≤ i = 1 ∏ 3 E Q [ exp ( K 2 c N a i 2 ) 1 { E reg } ] 1/3 ≤ 2. (95) where the first inequality follows from Equation (89) and ( a 1 + a 2 + a 3 ) 2 ≤ 3 ( a 1 2 + a 2 2 + a 3 2 ) (a_1+a_2+a_3)^2\leq3(a_1^2+a_2^2+a_3^2) ( a 1 + a 2 + a 3 ) 2 ≤ 3 ( a 1 2 + a 2 2 + a 3 2 ) (90) , (92) and (94) .
It remains to transfer Equation (95) to the initial state e e e (9) , (86) and (87) , the following holds on E r e g \mathcal{E}_{\mathrm{reg}} E reg y , y ′ ∈ S N y,y'\in\mathcal{S}_N y , y ′ ∈ S N C > 0 C>0 C > 0
∥ F ( y ) − F ( y ′ ) ∥ ≤ C K ∥ y − y ′ ∥ , ∣ ( L n u l l m ϕ ) ( y ) − ( L n u l l m ϕ ) ( y ′ ) ∣ ≤ C K N ∥ y − y ′ ∥ . (96) \left\|F(y)-F(y')\right\|\leq CK\left\|y-y'\right\|,\qquad\left|(\mathcal{L}_{\mathrm{null}}m_\phi)(y)-(\mathcal{L}_{\mathrm{null}}m_\phi)(y')\right|\leq\frac{CK}{\sqrt{N}}\left\|y-y'\right\|. \tag*{(96)} ∥ F ( y ) − F ( y ′ ) ∥ ≤ C K ∥ y − y ′ ∥ , ∣ ( L null m ϕ ) ( y ) − ( L null m ϕ ) ( y ′ ) ∣ ≤ N C K ∥ y − y ′ ∥ . (96) By the second bound in Equation (96) , the following holds on E r e g \mathcal{E}_{\mathrm{reg}} E reg ( y t ) t ∈ [ 0 , T ] (y_t)_{t\in[0,T]} ( y t ) t ∈ [ 0 , T ] ( y t ′ ) t ∈ [ 0 , T ] (y'_t)_{t\in[0,T]} ( y t ′ ) t ∈ [ 0 , T ] S N \mathcal{S}_N S N
sup t ∈ [ 0 , T ] ∣ ( L n u l l m ϕ ) ( y t ) ∣ ≤ sup t ∈ [ 0 , T ] ∣ ( L n u l l m ϕ ) ( y t ′ ) ∣ + C K N sup t ∈ [ 0 , T ] ∥ y t − y t ′ ∥ . (97) \sup_{t\in[0,T]}\left|(\mathcal{L}_{\mathrm{null}}m_\phi)(y_t)\right|\leq\sup_{t\in[0,T]}\left|(\mathcal{L}_{\mathrm{null}}m_\phi)(y'_t)\right|+\frac{CK}{\sqrt{N}}\sup_{t\in[0,T]}\left\|y_t-y'_t\right\|. \tag*{(97)} t ∈ [ 0 , T ] sup ∣ ( L null m ϕ ) ( y t ) ∣ ≤ t ∈ [ 0 , T ] sup ∣ ( L null m ϕ ) ( y t ′ ) ∣ + N C K t ∈ [ 0 , T ] sup ∥ y t − y t ′ ∥ . (97) Hence, to transfer Equation (95) to e e e e e e N e 1 \sqrt{N}e_1 N e 1 ∥ e − N e 1 ∥ \left\|e-\sqrt{N}e_1\right\| e − N e 1 T T T (97) is of order δ \delta δ { ∥ e − N e 1 ∥ ≤ δ N } \{\left\|e-\sqrt{N}e_1\right\|\leq\delta\sqrt{N}\} { e − N e 1 ≤ δ N }
For constructing such a coupling, driving both trajectories by the same Brownian motion in Equation (18) is inconvenient. This is because their noise terms are projections of a common increment d B t dB_t d B t Q Y t d B t Q_{Y_t}dB_t Q Y t d B t Y t Y_t Y t Y t Y_t Y t Q Y t d t Q_{Y_t}\,dt Q Y t d t
Precisely, let B \mathcal{B} B N × N N\times N N × N B j i = − B i j \mathcal{B}_{ji}=-\mathcal{B}_{ij} B j i = − B ij ( B i j ) i < j (\mathcal{B}_{ij})_{i<j} ( B ij ) i < j R \mathbb{R} R B 0 B^0 B 0 R \mathbb{R} R B \mathcal{B} B B 0 B^0 B 0 B \mathcal{B} B v ∈ R N v\in\mathbb{R}^N v ∈ R N
d B t v = ∑ i , j = 1 N v j ( d B t ) i j e i = ∑ i < j ( v j e i − v i e j ) ( d B t ) i j . d\mathcal{B}_t v=\sum_{i,j=1}^{N}v_j(d\mathcal{B}_t)_{ij}e_i=\sum_{i<j}(v_je_i-v_ie_j)(d\mathcal{B}_t)_{ij}. d B t v = i , j = 1 ∑ N v j ( d B t ) ij e i = i < j ∑ ( v j e i − v i e j ) ( d B t ) ij . Consequently, since the entries ( B i j ) i < j (\mathcal{B}_{ij})_{i<j} ( B ij ) i < j v ∈ R N v\in\mathbb{R}^N v ∈ R N
⟨ v , d B t v ⟩ = ∑ i < j ( v j v i − v i v j ) ( d B t ) i j = 0 , \langle v,d\mathcal{B}_t v\rangle=\sum_{i<j}(v_jv_i-v_iv_j)(d\mathcal{B}_t)_{ij}=0, ⟨ v , d B t v ⟩ = i < j ∑ ( v j v i − v i v j ) ( d B t ) ij = 0 , ( d B t v ) ( d B t v ) T = ∑ i < j ( v j e i − v i e j ) ( v j e i − v i e j ) T d t = 1 2 ∑ i , j = 1 N ( v j 2 e i e i T + v i 2 e j e j T − v i v j e i e j T − v i v j e j e i T ) d t = ( ∥ v ∥ 2 I − v v T ) d t . (98) (d\mathcal{B}_t v)(d\mathcal{B}_t v)^{\mathsf{T}}=\sum_{i<j}(v_je_i-v_ie_j)(v_je_i-v_ie_j)^{\mathsf{T}}dt
=\frac{1}{2}\sum_{i,j=1}^{N}\left(v_j^2e_ie_i^{\mathsf{T}}+v_i^2e_je_j^{\mathsf{T}}-v_iv_je_ie_j^{\mathsf{T}}-v_iv_je_je_i^{\mathsf{T}}\right)dt
=\left(\|v\|^2I-vv^{\mathsf{T}}\right)dt. \tag*{(98)} ( d B t v ) ( d B t v ) T = i < j ∑ ( v j e i − v i e j ) ( v j e i − v i e j ) T d t = 2 1 i , j = 1 ∑ N ( v j 2 e i e i T + v i 2 e j e j T − v i v j e i e j T − v i v j e j e i T ) d t = ( ∥ v ∥ 2 I − v v T ) d t . (98) For every initial state y ∈ S N y \in\mathcal{S}_N y ∈ S N ( Y t r o t ( y ) ) t ≥ 0 (Y_t^{\mathrm{rot}}(y))_{t\geq0} ( Y t rot ( y ) ) t ≥ 0
d Y t r o t ( y ) = F ( Y t r o t ( y ) ) d t + 1 N d B t Y t r o t ( y ) , Y 0 r o t ( y ) = y . (99) dY_t^{\mathrm{rot}}(y) = F(Y_t^{\mathrm{rot}}(y))dt + \frac{1}{\sqrt{N}}d\mathcal{B}_tY_t^{\mathrm{rot}}(y), \qquad Y_0^{\mathrm{rot}}(y)=y. \tag*{(99)} d Y t rot ( y ) = F ( Y t rot ( y )) d t + N 1 d B t Y t rot ( y ) , Y 0 rot ( y ) = y . (99) Since F F F (18) for H n u l l H_{\mathrm{null}} H null (99) differs from Equation (18) only in its noise term. We now verify that Y r o t ( y ) Y^{\mathrm{rot}}(y) Y rot ( y ) y y y Y t r o t ≔ Y t r o t ( y ) Y_t^{\mathrm{rot}} \coloneqq Y_t^{\mathrm{rot}}(y) Y t rot : = Y t rot ( y ) Y t r o t Y_t^{\mathrm{rot}} Y t rot S N \mathcal{S}_N S N (99) , we have
d ∥ Y t r o t ∥ 2 = 2 ⟨ Y t r o t , F ( Y t r o t ) ⟩ d t + 2 N ⟨ Y t r o t , d B t Y t r o t ⟩ + 1 N Tr ( ( d B t Y t r o t ) ( d B t Y t r o t ) T ) = − κ N ∥ Y t r o t ∥ 2 d t + κ N ∥ Y t r o t ∥ 2 d t = 0 , \begin{aligned}
d\left\|Y_t^{\mathrm{rot}}\right\|^2
&=2\langle Y_t^{\mathrm{rot}},F(Y_t^{\mathrm{rot}})\rangle dt+\frac{2}{\sqrt{N}}\langle Y_t^{\mathrm{rot}},d\mathcal{B}_tY_t^{\mathrm{rot}}\rangle+\frac{1}{N}\operatorname{Tr}\left((d\mathcal{B}_tY_t^{\mathrm{rot}})(d\mathcal{B}_tY_t^{\mathrm{rot}})^\mathsf{T}\right)\\
&=-\kappa_N\left\|Y_t^{\mathrm{rot}}\right\|^2dt+\kappa_N\left\|Y_t^{\mathrm{rot}}\right\|^2dt=0,
\end{aligned} d Y t rot 2 = 2 ⟨ Y t rot , F ( Y t rot )⟩ d t + N 2 ⟨ Y t rot , d B t Y t rot ⟩ + N 1 Tr ( ( d B t Y t rot ) ( d B t Y t rot ) T ) = − κ N Y t rot 2 d t + κ N Y t rot 2 d t = 0 , where the second equality follows from Equation (98) and ⟨ x , F ( x ) ⟩ = − κ N 2 ∥ x ∥ 2 \langle x,F(x)\rangle=-\frac{\kappa_N}{2}\|x\|^2 ⟨ x , F ( x )⟩ = − 2 κ N ∥ x ∥ 2 Q x x = 0 Q_xx=0 Q x x = 0 ∥ Y t r o t ∥ 2 = N \left\|Y_t^{\mathrm{rot}}\right\|^2=N ∥ Y t rot ∥ 2 = N t ≥ 0 t\geq0 t ≥ 0
We then identify the driving Brownian motion. Consider the process
B ^ t ≔ 1 N ∫ 0 t ( d B s Y s r o t + Y s r o t d B s 0 ) , \widehat{B}_t \coloneqq\frac{1}{\sqrt{N}}\int_0^t\left(d\mathcal{B}_sY_s^{\mathrm{rot}}+Y_s^{\mathrm{rot}}dB_s^0\right), B t : = N 1 ∫ 0 t ( d B s Y s rot + Y s rot d B s 0 ) , which is a continuous martingale with respect to the filtration generated by the environment, the auxiliary randomness, B \mathcal{B} B B 0 B^0 B 0 B \mathcal{B} B B 0 B^0 B 0 (98) and ∥ Y t r o t ∥ 2 = N \|Y_t^{\mathrm{rot}}\|^2=N ∥ Y t rot ∥ 2 = N
d ⟨ B ^ ⟩ t = 1 N ( ( d B t Y t r o t ) ( d B t Y t r o t ) T + Y t r o t ( Y t r o t ) T d t ) = 1 N ( ∥ Y t r o t ∥ 2 I − Y t r o t ( Y t r o t ) T + Y t r o t ( Y t r o t ) T ) d t = I d t . \begin{aligned}
d\langle\widehat{B}\rangle_t
&=\frac{1}{N}\left((d\mathcal{B}_tY_t^{\mathrm{rot}})(d\mathcal{B}_tY_t^{\mathrm{rot}})^\mathsf{T}+Y_t^{\mathrm{rot}}(Y_t^{\mathrm{rot}})^\mathsf{T}dt\right)\\
&=\frac{1}{N}\left(\|Y_t^{\mathrm{rot}}\|^2I-Y_t^{\mathrm{rot}}(Y_t^{\mathrm{rot}})^\mathsf{T}+Y_t^{\mathrm{rot}}(Y_t^{\mathrm{rot}})^\mathsf{T}\right)dt=I\,dt.
\end{aligned} d ⟨ B ⟩ t = N 1 ( ( d B t Y t rot ) ( d B t Y t rot ) T + Y t rot ( Y t rot ) T d t ) = N 1 ( ∥ Y t rot ∥ 2 I − Y t rot ( Y t rot ) T + Y t rot ( Y t rot ) T ) d t = I d t . Hence, by Lévy’s characterization of Brownian motion [cf. SV07, Theorem 4.1.1], B ^ \widehat{B} B B ^ \widehat{B} B
Lastly, we show that the noise term in Equation (99) is the projected Brownian increment Q Y t r o t d B ^ t Q_{Y_t^{\mathrm{rot}}}d\widehat{B}_t Q Y t rot d B t Q Y t r o t Y t r o t = 0 Q_{Y_t^{\mathrm{rot}}}Y_t^{\mathrm{rot}}=0 Q Y t rot Y t rot = 0 Q Y t r o t = I − 1 N Y t r o t ( Y t r o t ) T Q_{Y_t^{\mathrm{rot}}}=I-\frac{1}{N}Y_t^{\mathrm{rot}}(Y_t^{\mathrm{rot}})^\mathsf{T} Q Y t rot = I − N 1 Y t rot ( Y t rot ) T (98) that
Q Y t r o t d B ^ t = 1 N ( Q Y t r o t d B t Y t r o t + Q Y t r o t Y t r o t d B t 0 ) = 1 N ( d B t Y t r o t − 1 N Y t r o t ⟨ Y t r o t , d B t Y t r o t ⟩ ) = 1 N d B t Y t r o t . \begin{aligned}
Q_{Y_t^{\mathrm{rot}}}d\widehat{B}_t
&=\frac{1}{\sqrt{N}}\left(Q_{Y_t^{\mathrm{rot}}}d\mathcal{B}_tY_t^{\mathrm{rot}}+Q_{Y_t^{\mathrm{rot}}}Y_t^{\mathrm{rot}}dB_t^0\right)\\
&=\frac{1}{\sqrt{N}}\left(d\mathcal{B}_tY_t^{\mathrm{rot}}-\frac{1}{N}Y_t^{\mathrm{rot}}\langle Y_t^{\mathrm{rot}},d\mathcal{B}_tY_t^{\mathrm{rot}}\rangle\right)=\frac{1}{\sqrt{N}}d\mathcal{B}_tY_t^{\mathrm{rot}}.
\end{aligned} Q Y t rot d B t = N 1 ( Q Y t rot d B t Y t rot + Q Y t rot Y t rot d B t 0 ) = N 1 ( d B t Y t rot − N 1 Y t rot ⟨ Y t rot , d B t Y t rot ⟩ ) = N 1 d B t Y t rot . Substituting this into Equation (99) , we obtain
d Y t r o t = F ( Y t r o t ) d t + Q Y t r o t d B ^ t , Y 0 r o t = y , dY_t^{\mathrm{rot}}=F(Y_t^{\mathrm{rot}})dt+Q_{Y_t^{\mathrm{rot}}}d\widehat{B}_t,\qquad Y_0^{\mathrm{rot}}=y, d Y t rot = F ( Y t rot ) d t + Q Y t rot d B t , Y 0 rot = y , so that Y r o t Y^{\mathrm{rot}} Y rot (18) for H n u l l H_{\mathrm{null}} H null y y y B ^ \widehat{B} B (18) is a deterministic function of its initial point and its driving Brownian motion (§ 3.3 ). Moreover, both B ^ \widehat{B} B Y ( y ) Y(y) Y ( y ) H n u l l H_{\mathrm{null}} H null y y y
( Y t r o t ( y ) ) t ≥ 0 = d ( Y t ( y ) ) t ≥ 0 . (100) \left(Y_t^{\mathrm{rot}}(y)\right)_{t\geq0}\mathrel{\overset{\mathrm{d}}{=}}\left(Y_t(y)\right)_{t\geq0}. \tag*{(100)} ( Y t rot ( y ) ) t ≥ 0 = d ( Y t ( y ) ) t ≥ 0 . (100) We now show that the trajectories initialized at e e e N e 1 \sqrt{N}e_1 N e 1
Let D t = Y t r o t ( e ) − Y t r o t ( N e 1 ) D_t=Y_t^{\mathrm{rot}}(e)-Y_t^{\mathrm{rot}}(\sqrt{N}e_1) D t = Y t rot ( e ) − Y t rot ( N e 1 ) D 0 = e − N e 1 D_0=e-\sqrt{N}e_1 D 0 = e − N e 1 (99) is linear in Y t r o t ( y ) Y_t^{\mathrm{rot}}(y) Y t rot ( y )
d D t = ( F ( Y t r o t ( e ) ) − F ( Y t r o t ( N e 1 ) ) ) d t + 1 N d B t D t . \mathrm{d}D_t=\left(F(Y_t^{\mathrm{rot}}(e))-F(Y_t^{\mathrm{rot}}(\sqrt{N}e_1))\right)\mathrm{d}t+\frac{1}{\sqrt{N}}\mathrm{d}\mathcal{B}_tD_t. d D t = ( F ( Y t rot ( e )) − F ( Y t rot ( N e 1 )) ) d t + N 1 d B t D t . Consequently, by Itô’s formula, we obtain
d ∥ D t ∥ 2 = 2 ⟨ D t , F ( Y t r o t ( e ) ) − F ( Y t r o t ( N e 1 ) ) ⟩ d t + 2 N ⟨ D t , d B t D t ⟩ + 1 N Tr ( ( d B t D t ) ( d B t D t ) T ) = 2 ⟨ D t , F ( Y t r o t ( e ) ) − F ( Y t r o t ( N e 1 ) ) ⟩ d t + 1 N Tr ( ∥ D t ∥ 2 I − D t D t T ) d t = 2 ⟨ D t , F ( Y t r o t ( e ) ) − F ( Y t r o t ( N e 1 ) ) ⟩ d t + κ N ∥ D t ∥ 2 d t = 2 β ⟨ D t , ∇ S N H n u l l ( Y t r o t ( e ) ) − ∇ S N H n u l l ( Y t r o t ( N e 1 ) ) ⟩ d t , (101) \begin{aligned}
\mathrm{d}\lVert D_t\rVert^2
&=2\left\langle D_t,F(Y_t^{\mathrm{rot}}(e))-F(Y_t^{\mathrm{rot}}(\sqrt{N}e_1))\right\rangle\mathrm{d}t+\frac{2}{\sqrt{N}}\left\langle D_t,\mathrm{d}\mathcal{B}_tD_t\right\rangle\\
&\quad+\frac{1}{N}\operatorname{Tr}\left((\mathrm{d}\mathcal{B}_tD_t)(\mathrm{d}\mathcal{B}_tD_t)^{\mathsf{T}}\right)\\
&=2\left\langle D_t,F(Y_t^{\mathrm{rot}}(e))-F(Y_t^{\mathrm{rot}}(\sqrt{N}e_1))\right\rangle\mathrm{d}t+\frac{1}{N}\operatorname{Tr}\left(\lVert D_t\rVert^2I-D_tD_t^{\mathsf{T}}\right)\mathrm{d}t\\
&=2\left\langle D_t,F(Y_t^{\mathrm{rot}}(e))-F(Y_t^{\mathrm{rot}}(\sqrt{N}e_1))\right\rangle\mathrm{d}t+\kappa_N\lVert D_t\rVert^2\mathrm{d}t\\
&=2\beta\left\langle D_t,\nabla_{\mathbb{S}^N}H_{\mathrm{null}}(Y_t^{\mathrm{rot}}(e))-\nabla_{\mathbb{S}^N}H_{\mathrm{null}}(Y_t^{\mathrm{rot}}(\sqrt{N}e_1))\right\rangle\mathrm{d}t, \tag*{(101)}
\end{aligned} d ∥ D t ∥ 2 = 2 ⟨ D t , F ( Y t rot ( e )) − F ( Y t rot ( N e 1 )) ⟩ d t + N 2 ⟨ D t , d B t D t ⟩ + N 1 Tr ( ( d B t D t ) ( d B t D t ) T ) = 2 ⟨ D t , F ( Y t rot ( e )) − F ( Y t rot ( N e 1 )) ⟩ d t + N 1 Tr ( ∥ D t ∥ 2 I − D t D t T ) d t = 2 ⟨ D t , F ( Y t rot ( e )) − F ( Y t rot ( N e 1 )) ⟩ d t + κ N ∥ D t ∥ 2 d t = 2 β ⟨ D t , ∇ S N H null ( Y t rot ( e )) − ∇ S N H null ( Y t rot ( N e 1 )) ⟩ d t , (101) where the second equality follows from Equation (98) , and the last equality follows from the definition of F F F (86) . As anticipated, the same infinitesimal rotation is applied to both trajectories and preserves the distance between them, so that only the difference of the gradient drifts controls their separation.
Since Equation (101) contains no martingale term, t ↦ ∥ D t ∥ 2 t\mapsto\lVert D_t\rVert^2 t ↦ ∥ D t ∥ 2 (101) . Hence, by the Cauchy–Schwarz inequality, Equation (96) and κ N ≤ 1 \kappa_N\leq1 κ N ≤ 1 E r e g \mathcal{E}_{\mathrm{reg}} E reg t ≥ 0 t\geq0 t ≥ 0
d d t ∥ D t ∥ 2 = 2 ⟨ D t , F ( Y t r o t ( e ) ) − F ( Y t r o t ( N e 1 ) ) ⟩ + κ N ∥ D t ∥ 2 ≤ 2 ∥ D t ∥ ∥ F ( Y t r o t ( e ) ) − F ( Y t r o t ( N e 1 ) ) ∥ + ∥ D t ∥ 2 ≤ ( 2 C K + 1 ) ∥ D t ∥ 2 . (102) \begin{aligned}
\frac{\mathrm{d}}{\mathrm{d}t}\lVert D_t\rVert^2
&=2\left\langle D_t,F(Y_t^{\mathrm{rot}}(e))-F(Y_t^{\mathrm{rot}}(\sqrt{N}e_1))\right\rangle+\kappa_N\lVert D_t\rVert^2\\
&\leq2\lVert D_t\rVert\left\lVert F(Y_t^{\mathrm{rot}}(e))-F(Y_t^{\mathrm{rot}}(\sqrt{N}e_1))\right\rVert+\lVert D_t\rVert^2\leq(2CK+1)\lVert D_t\rVert^2. \tag*{(102)}
\end{aligned} d t d ∥ D t ∥ 2 = 2 ⟨ D t , F ( Y t rot ( e )) − F ( Y t rot ( N e 1 )) ⟩ + κ N ∥ D t ∥ 2 ≤ 2 ∥ D t ∥ F ( Y t rot ( e )) − F ( Y t rot ( N e 1 )) + ∥ D t ∥ 2 ≤ ( 2 C K + 1 ) ∥ D t ∥ 2 . (102) Applying Grönwall’s inequality to Equation (102) and using T ≤ 1 T\leq1 T ≤ 1 E r e g \mathcal{E}_{\mathrm{reg}} E reg
sup t ∈ [ 0 , T ] ∥ D t ∥ ≤ e C K + 1 / 2 ∥ D 0 ∥ = e C K + 1 / 2 ∥ e − N e 1 ∥ . (103) \sup_{t\in[0,T]}\lVert D_t\rVert\leq e^{CK+1/2}\lVert D_0\rVert=e^{CK+1/2}\lVert e-\sqrt{N}e_1\rVert. \tag*{(103)} t ∈ [ 0 , T ] sup ∥ D t ∥ ≤ e C K + 1/2 ∥ D 0 ∥ = e C K + 1/2 ∥ e − N e 1 ∥ . (103) Finally, we combine the two steps. By Equation (100) with y = N e 1 y=\sqrt{N}e_1 y = N e 1 (95) also holds with Y t r o t ( N e 1 ) Y_t^{\mathrm{rot}}(\sqrt{N}e_1) Y t rot ( N e 1 ) Y t ( N e 1 ) Y_t(\sqrt{N}e_1) Y t ( N e 1 ) c ⋆ ≔ c 18 K 2 c_\star\coloneqq\frac{c}{18K^2} c ⋆ : = 18 K 2 c C ⋆ ≔ max { 2 , c C 2 9 e 2 C K + 1 } C_\star\coloneqq\max\left\{2,\frac{cC^2}{9}e^{2CK+1}\right\} C ⋆ : = max { 2 , 9 c C 2 e 2 C K + 1 }
On E r e g ∩ { ∥ e − N e 1 ∥ ≤ δ N } \mathcal{E}_{\mathrm{reg}}\cap\{\lVert e-\sqrt{N}e_1\rVert\leq\delta\sqrt{N}\} E reg ∩ {∥ e − N e 1 ∥ ≤ δ N } (97) and (103) imply that
sup t ∈ [ 0 , T ] ∣ ( L n u l l m ϕ ) ( Y t r o t ( e ) ) ∣ ≤ sup t ∈ [ 0 , T ] ∣ ( L n u l l m ϕ ) ( Y t r o t ( N e 1 ) ) ∣ + C K e C K + 1 / 2 δ . \sup_{t\in[0,T]}\left|(\mathcal{L}_{\mathrm{null}}m_\phi)(Y_t^{\mathrm{rot}}(e))\right|
\leq\sup_{t\in[0,T]}\left|(\mathcal{L}_{\mathrm{null}}m_\phi)(Y_t^{\mathrm{rot}}(\sqrt{N}e_1))\right|+CKe^{CK+1/2}\delta. t ∈ [ 0 , T ] sup ( L null m ϕ ) ( Y t rot ( e )) ≤ t ∈ [ 0 , T ] sup ( L null m ϕ ) ( Y t rot ( N e 1 )) + C K e C K + 1/2 δ . Hence, the inequality ( a + b ) 2 ≤ 2 a 2 + 2 b 2 (a+b)^2 \le2a^2+2b^2 ( a + b ) 2 ≤ 2 a 2 + 2 b 2 (95) for Y t r o t ( N e 1 ) Y_t^{\mathrm{rot}}(\sqrt{N}e_1) Y t rot ( N e 1 )
E Q [ exp ( c ∗ N sup t ∈ [ 0 , T ] ∣ L n u l l m ϕ ( Y t r o t ( e ) ) ∣ 2 ) 1 { E r e g } 1 { ∥ e − N e 1 ∥ ≤ δ N } ] ≤ e C ∗ N δ 2 E Q [ exp ( c N 9 K N 2 sup t ∈ [ 0 , T ] ∣ L n u l l m ϕ ( Y t r o t ( N e 1 ) ) ∣ 2 ) 1 { E r e g } ] ≤ 2 e C ∗ N δ 2 ≤ C ∗ e C ∗ N δ 2 . \mathbb{E}_{\mathbb{Q}}\left[\exp\left(c_*N\sup_{t\in[0,T]}\left|\mathcal{L}_{\mathrm{null}}m_\phi\left(Y_t^{\mathrm{rot}}(e)\right)\right|^2\right)\mathbf{1}_{\{\mathcal{E}_{\mathrm{reg}}\}}\mathbf{1}_{\{\lVert e-\sqrt{N}e_1\rVert\le\delta\sqrt{N}\}}\right]
\le e^{C_*N\delta^2}\mathbb{E}_{\mathbb{Q}}\left[\exp\left(\frac{cN}{9K_N^2}\sup_{t\in[0,T]}\left|\mathcal{L}_{\mathrm{null}}m_\phi\left(Y_t^{\mathrm{rot}}(\sqrt{N}e_1)\right)\right|^2\right)\mathbf{1}_{\{\mathcal{E}_{\mathrm{reg}}\}}\right]\le2e^{C_*N\delta^2}\le C_*e^{C_*N\delta^2}. E Q [ exp ( c ∗ N t ∈ [ 0 , T ] sup L null m ϕ ( Y t rot ( e ) ) 2 ) 1 { E reg } 1 {∥ e − N e 1 ∥ ≤ δ N } ] ≤ e C ∗ N δ 2 E Q [ exp ( 9 K N 2 c N t ∈ [ 0 , T ] sup L null m ϕ ( Y t rot ( N e 1 ) ) 2 ) 1 { E reg } ] ≤ 2 e C ∗ N δ 2 ≤ C ∗ e C ∗ N δ 2 . By Equation (100) with y = e y=e y = e Y t r o t ( e ) Y_t^{\mathrm{rot}}(e) Y t rot ( e ) Y t ( e ) Y_t(e) Y t ( e )
By Markov’s inequality, Lemma 6.11 implies sharp subgaussian-style concentration for L n u l l m ϕ \mathcal{L}_{\mathrm{null}}m_\phi L null m ϕ N e 1 \sqrt{N}e_1 N e 1 reverse planting trick , as it runs in the direction opposite to canonical applications of the planting trick [2 ] , where the null model is the true distribution of interest, and statements proved under the planted model are transferred to it via contiguity.
However, the planted and null laws are far from contiguous, as their Hamiltonians differ by the signal term H s i g H_{\mathrm{sig}} H sig N λ m ϕ k N\lambda m_\phi^k N λ m ϕ k L n u l l m ϕ \mathcal{L}_{\mathrm{null}}m_\phi L null m ϕ
Theorem 6.12 (Adaptive Contiguity Estimate). Fix an initialization rule e e e T > 0 T>0 T > 0 ( F t ) 0 ≤ t ≤ T (\mathcal{F}_t)_{0\le t\le T} ( F t ) 0 ≤ t ≤ T ( Φ , R , W ) (\Phi,R,W) ( Φ , R , W ) ( Y t ( e ) ) 0 ≤ t ≤ T (Y_t(e))_{0\le t\le T} ( Y t ( e ) ) 0 ≤ t ≤ T τ \tau τ ( F t ) (\mathcal{F}_t) ( F t ) τ ≤ T \tau\le T τ ≤ T E ∈ F τ E\in\mathcal{F}_\tau E ∈ F τ
P ( E ) ≤ exp ( N λ 2 σ N 2 2 κ N 2 ) E Q [ exp ( β 2 ∫ 0 τ ∥ ∇ S N H s i g ( Y t ( e ) ) ∥ 2 d t ) 1 { E } ] 1 / 2 . \mathbb{P}(E)\le\exp\left(\frac{N\lambda^2\sigma_N^2}{2\kappa_N^2}\right)\mathbb{E}_{\mathbb{Q}}\left[\exp\left(\beta^2\int_0^\tau\left\|\nabla_{\mathbb{S}^N}H_{\mathrm{sig}}(Y_t(e))\right\|^2\,dt\right)\mathbf{1}_{\{E\}}\right]^{1/2}. P ( E ) ≤ exp ( 2 κ N 2 N λ 2 σ N 2 ) E Q [ exp ( β 2 ∫ 0 τ ∥ ∇ S N H sig ( Y t ( e )) ∥ 2 d t ) 1 { E } ] 1/2 . Proof. Throughout, we write Y t = Y t ( e ) Y_t=Y_t(e) Y t = Y t ( e ) t ≤ T t\le T t ≤ T P t \mathbb{P}^t P t Q t \mathbb{Q}^t Q t ( Φ , R , W ) (\Phi,R,W) ( Φ , R , W ) ( Y s ) 0 ≤ s ≤ t (Y_s)_{0\le s\le t} ( Y s ) 0 ≤ s ≤ t Definitions 6.8 and 6.9 , these laws factorize as P t = P e n v ⋅ P d y n t \mathbb{P}^t=\mathbb{P}_{\mathrm{env}}\cdot\mathbb{P}_{\mathrm{dyn}}^t P t = P env ⋅ P dyn t Q t = Q e n v ⋅ Q d y n t \mathbb{Q}^t=\mathbb{Q}_{\mathrm{env}}\cdot\mathbb{Q}_{\mathrm{dyn}}^t Q t = Q env ⋅ Q dyn t t t t t t t Z ( t ) Z(t) Z ( t )
Z ( t ) = Z e n v Z d y n ( t ) , Z e n v ≔ d P e n v d Q e n v , Z d y n ( t ) ≔ d P d y n t ( ⋅ ∣ Φ , R , W ) d Q d y n t ( ⋅ ∣ Φ , R , W ) . Z(t)=Z_{\mathrm{env}}Z_{\mathrm{dyn}}(t),\qquad Z_{\mathrm{env}}\coloneqq\frac{d\mathbb{P}_{\mathrm{env}}}{d\mathbb{Q}_{\mathrm{env}}},\qquad Z_{\mathrm{dyn}}(t)\coloneqq\frac{d\mathbb{P}_{\mathrm{dyn}}^t(\,\cdot\mid\Phi,R,W)}{d\mathbb{Q}_{\mathrm{dyn}}^t(\,\cdot\mid\Phi,R,W)}. Z ( t ) = Z env Z dyn ( t ) , Z env : = d Q env d P env , Z dyn ( t ) : = d Q dyn t ( ⋅ ∣ Φ , R , W ) d P dyn t ( ⋅ ∣ Φ , R , W ) . We begin by analyzing Z e n v Z_{\mathrm{env}} Z env Definitions 6.8 and 6.9 , W W W ( Φ , R ) (\Phi,R) ( Φ , R ) P e n v \mathbb{P}_{\mathrm{env}} P env Q e n v \mathbb{Q}_{\mathrm{env}} Q env ( Φ , R ) = ( Ψ A ( U ) , ∥ U ∥ ) (\Phi,R)=(\Psi A(U),\lVert U\rVert) ( Φ , R ) = ( Ψ A ( U ) , ∥ U ∥) U ∼ N ( N λ m ϕ k θ ∗ , σ N 2 I ) U\sim\mathcal{N}(N\lambda m_\phi^k\theta^*,\sigma_N^2I) U ∼ N ( N λ m ϕ k θ ∗ , σ N 2 I ) P e n v \mathbb{P}_{\mathrm{env}} P env U ∼ N ( 0 , σ N 2 I ) U\sim\mathcal{N}(0,\sigma_N^2I) U ∼ N ( 0 , σ N 2 I ) Q e n v \mathbb{Q}_{\mathrm{env}} Q env U = R Φ T e 1 U=R\Phi^\mathsf{T}e_1 U = R Φ T e 1 Ψ A ( U ) U = ∥ U ∥ e 1 \Psi A(U)U=\lVert U\rVert e_1 Ψ A ( U ) U = ∥ U ∥ e 1 Z e n v Z_{\mathrm{env}} Z env R Φ T e 1 R\Phi^\mathsf{T}e_1 R Φ T e 1
Gaussian MGF computation.
E Q [ Z e n v 2 ] = E U ∼ N ( 0 , σ N 2 I ) [ exp ( − 1 σ N 2 ( ∥ U − N λ α N θ ∗ ∥ 2 − ∥ U ∥ 2 ) ) ] = exp ( − N 2 λ 2 α N 2 σ N 2 ) E U ∼ N ( 0 , σ N 2 I ) [ exp ( 2 N λ α N σ N 2 ⟨ θ ∗ , U ⟩ ) ] = exp ( N 2 λ 2 α N 2 σ N 2 ) = exp ( N λ 2 σ N 2 κ N 2 ) , (104) \begin{aligned}
\mathbb{E}_{\mathbb{Q}}[Z_{\mathrm{env}}^2]
&=\mathbb{E}_{U\sim\mathcal{N}(0,\sigma_N^2 I)}\left[\exp\left(-\frac{1}{\sigma_N^2}\left(\left\|U-N\lambda\alpha_N\theta^*\right\|^2-\|U\|^2\right)\right)\right]\\
&=\exp\left(-\frac{N^2\lambda^2\alpha_N^2}{\sigma_N^2}\right)\mathbb{E}_{U\sim\mathcal{N}(0,\sigma_N^2 I)}\left[\exp\left(\frac{2N\lambda\alpha_N}{\sigma_N^2}\langle\theta^*,U\rangle\right)\right]\\
&=\exp\left(\frac{N^2\lambda^2\alpha_N^2}{\sigma_N^2}\right)=\exp\left(\frac{N\lambda^2\sigma_N^2}{\kappa_N^2}\right), \tag*{(104)}
\end{aligned} E Q [ Z env 2 ] = E U ∼ N ( 0 , σ N 2 I ) [ exp ( − σ N 2 1 ( ∥ U − N λ α N θ ∗ ∥ 2 − ∥ U ∥ 2 ) ) ] = exp ( − σ N 2 N 2 λ 2 α N 2 ) E U ∼ N ( 0 , σ N 2 I ) [ exp ( σ N 2 2 N λ α N ⟨ θ ∗ , U ⟩ ) ] = exp ( σ N 2 N 2 λ 2 α N 2 ) = exp ( κ N 2 N λ 2 σ N 2 ) , (104) where the last equality uses σ N 2 = κ N N α N \sigma_N^2=\kappa_N\sqrt{N}\alpha_N σ N 2 = κ N N α N
We now analyze Z d y n Z_{\mathrm{dyn}} Z dyn Definitions 6.8 and 6.9 , conditional on the environment, P d y n \mathbb{P}_{\mathrm{dyn}} P dyn Q d y n \mathbb{Q}_{\mathrm{dyn}} Q dyn β \beta β H ϕ H_\phi H ϕ H n u l l H_{\mathrm{null}} H null Y 0 = e Y_0=e Y 0 = e ( B t ) 0 ≤ t ≤ T (B_t)_{0\le t\le T} ( B t ) 0 ≤ t ≤ T Q d y n \mathbb{Q}_{\mathrm{dyn}} Q dyn m ϕ m_\phi m ϕ (8) implies ∇ S N m ϕ ( y ) = N − 1 / 2 Q y ϕ \nabla_{\mathcal{S}_N}m_\phi(y)=N^{-1/2}Q_y\phi ∇ S N m ϕ ( y ) = N − 1/2 Q y ϕ y ∈ S N y\in\mathcal{S}_N y ∈ S N
∇ S N H s i g ( y ) = N λ ( k m ϕ ( y ) k − 1 − σ N 2 κ N 2 ) Q y ϕ . (105) \nabla_{\mathcal{S}_N}H_{\mathrm{sig}}(y)=\sqrt{N}\lambda\left(km_\phi(y)^{k-1}-\frac{\sigma_N^2}{\kappa_N^2}\right)Q_y\phi. \tag*{(105)} ∇ S N H sig ( y ) = N λ ( k m ϕ ( y ) k − 1 − κ N 2 σ N 2 ) Q y ϕ . (105) For t ≤ T t\le T t ≤ T θ t = β ∇ S N H s i g ( Y t ) \theta_t=\beta\nabla_{\mathcal{S}_N}H_{\mathrm{sig}}(Y_t) θ t = β ∇ S N H sig ( Y t ) θ t \theta_t θ t ∣ m ϕ ∣ ≤ 1 |m_\phi|\le1 ∣ m ϕ ∣ ≤ 1 ∥ Q y ϕ ∥ ≤ 1 \|Q_y\phi\|\le1 ∥ Q y ϕ ∥ ≤ 1 (105) implies that ∥ θ t ∥ ≤ β N λ ( k + κ N − 2 σ N 2 ) \|\theta_t\|\le\beta\sqrt{N}\lambda(k+\kappa_N^{-2}\sigma_N^2) ∥ θ t ∥ ≤ β N λ ( k + κ N − 2 σ N 2 ) ( L t θ ) 0 ≤ t ≤ T (L_t^\theta)_{0\le t\le T} ( L t θ ) 0 ≤ t ≤ T Q d y n T , θ \mathbb{Q}_{\mathrm{dyn}}^{T,\theta} Q dyn T , θ
L t θ ≔ exp ( ∫ 0 t ⟨ θ s , d B s ⟩ − 1 2 ∫ 0 t ∥ θ s ∥ 2 d s ) , d Q d y n T , θ d Q d y n T = L T θ . (106) L_t^\theta\coloneqq\exp\left(\int_0^t\langle\theta_s,dB_s\rangle-\frac{1}{2}\int_0^t\|\theta_s\|^2\,ds\right),\qquad\frac{d\mathbb{Q}_{\mathrm{dyn}}^{T,\theta}}{d\mathbb{Q}_{\mathrm{dyn}}^T}=L_T^\theta. \tag*{(106)} L t θ : = exp ( ∫ 0 t ⟨ θ s , d B s ⟩ − 2 1 ∫ 0 t ∥ θ s ∥ 2 d s ) , d Q dyn T d Q dyn T , θ = L T θ . (106) Then, by Lemma 3.15 applied conditionally on the environment, L t θ L_t^\theta L t θ Q d y n \mathbb{Q}_{\mathrm{dyn}} Q dyn Q d y n T , θ \mathbb{Q}_{\mathrm{dyn}}^{T,\theta} Q dyn T , θ ( V , U ) = ( H n u l l , H s i g ) (V,U)=(H_{\mathrm{null}},H_{\mathrm{sig}}) ( V , U ) = ( H null , H sig ) ( Y t ) 0 ≤ t ≤ T (Y_t)_{0\le t\le T} ( Y t ) 0 ≤ t ≤ T Q d y n T , θ \mathbb{Q}_{\mathrm{dyn}}^{T,\theta} Q dyn T , θ β \beta β H n u l l + H s i g = H ϕ H_{\mathrm{null}}+H_{\mathrm{sig}}=H_\phi H null + H sig = H ϕ Y 0 = e Y_0=e Y 0 = e Q d y n T , θ \mathbb{Q}_{\mathrm{dyn}}^{T,\theta} Q dyn T , θ P d y n T \mathbb{P}_{\mathrm{dyn}}^T P dyn T θ t ∈ T Y t S N \theta_t\in T_{Y_t}\mathcal{S}_N θ t ∈ T Y t S N (18) :
⟨ θ t , d B t ⟩ = ⟨ θ t , Q Y t d B t ⟩ = ⟨ θ t , d Y t ⟩ − β ⟨ θ t , ∇ S N H n u l l ( Y t ) ⟩ d t . \langle\theta_t,dB_t\rangle=\langle\theta_t,Q_{Y_t}dB_t\rangle=\langle\theta_t,dY_t\rangle-\beta\langle\theta_t,\nabla_{\mathcal{S}_N}H_{\mathrm{null}}(Y_t)\rangle\,dt. ⟨ θ t , d B t ⟩ = ⟨ θ t , Q Y t d B t ⟩ = ⟨ θ t , d Y t ⟩ − β ⟨ θ t , ∇ S N H null ( Y t )⟩ d t . Hence L t θ L_t^\theta L t θ t t t
Z d y n ( t ) = L t θ = exp ( ∫ 0 t ⟨ θ s , d Y s ⟩ − β ∫ 0 t ⟨ θ s , ∇ S N H n u l l ( Y s ) ⟩ d s − 1 2 ∫ 0 t ∥ θ s ∥ 2 d s ) , t ≤ T . (107) Z_{\mathrm{dyn}}(t)=L_t^\theta=\exp\left(\int_0^t\langle\theta_s,dY_s\rangle-\beta\int_0^t\langle\theta_s,\nabla_{\mathcal{S}_N}H_{\mathrm{null}}(Y_s)\rangle\,ds-\frac{1}{2}\int_0^t\|\theta_s\|^2\,ds\right),\qquad t\le T. \tag*{(107)} Z dyn ( t ) = L t θ = exp ( ∫ 0 t ⟨ θ s , d Y s ⟩ − β ∫ 0 t ⟨ θ s , ∇ S N H null ( Y s )⟩ d s − 2 1 ∫ 0 t ∥ θ s ∥ 2 d s ) , t ≤ T . (107) Since L t θ L_t^\theta L t θ Q d y n \mathbb{Q}_{\mathrm{dyn}} Q dyn Z d y n ( t ) Z_{\mathrm{dyn}}(t) Z dyn ( t )
We now control Z d y n ( t ) 2 Z_{\mathrm{dyn}}(t)^2 Z dyn ( t ) 2 2 θ t 2\theta_t 2 θ t (107) and applying Lemma 3.15 with 2 θ 2\theta 2 θ θ \theta θ Q d y n \mathbb{Q}_{\mathrm{dyn}} Q dyn
L t ≔ L t 2 θ = exp ( 2 ∫ 0 t ⟨ θ s , d B s ⟩ − 2 ∫ 0 t ∥ θ s ∥ 2 d s ) = Z d y n ( t ) 2 exp ( − ∫ 0 t ∥ θ s ∥ 2 d s ) . (108) L_t\coloneqq L_t^{2\theta}=\exp\left(2\int_0^t\langle\theta_s,dB_s\rangle-2\int_0^t\|\theta_s\|^2\,ds\right)=Z_{\mathrm{dyn}}(t)^2\exp\left(-\int_0^t\|\theta_s\|^2\,ds\right). \tag*{(108)} L t : = L t 2 θ = exp ( 2 ∫ 0 t ⟨ θ s , d B s ⟩ − 2 ∫ 0 t ∥ θ s ∥ 2 d s ) = Z dyn ( t ) 2 exp ( − ∫ 0 t ∥ θ s ∥ 2 d s ) . (108) Finally, we combine the two factors via an optional stopping argument. Since Z e n v Z_{\mathrm{env}} Z env F 0 \mathcal{F}_0 F 0 Z d y n ( t ) Z_{\mathrm{dyn}}(t) Z dyn ( t ) L t L_t L t
E Q [ Z ( t ) ∣ F s ] = Z e n v E Q [ Z d y n ( t ) ∣ F s ] = Z e n v Z d y n ( s ) = Z ( s ) , E Q [ L t ∣ F s ] = L s . \mathbb{E}_{\mathbb{Q}}[Z(t)\mid\mathcal{F}_s]=Z_{\mathrm{env}}\mathbb{E}_{\mathbb{Q}}[Z_{\mathrm{dyn}}(t)\mid\mathcal{F}_s]=Z_{\mathrm{env}}Z_{\mathrm{dyn}}(s)=Z(s),\qquad\mathbb{E}_{\mathbb{Q}}[L_t\mid\mathcal{F}_s]=L_s. E Q [ Z ( t ) ∣ F s ] = Z env E Q [ Z dyn ( t ) ∣ F s ] = Z env Z dyn ( s ) = Z ( s ) , E Q [ L t ∣ F s ] = L s . and thus, Z ( t ) Z(t) Z ( t ) L t L_t L t ( F t ) (\mathcal{F}_t) ( F t ) Q \mathbb{Q} Q τ ≤ T \tau\le T τ ≤ T ( F t ) (\mathcal{F}_t) ( F t ) [47 Theorem 1.2.5] implies that E Q [ Z ( T ) ∣ F τ ] = Z ( τ ) \mathbb{E}_{\mathbb{Q}}[Z(T)\mid\mathcal{F}_\tau]=Z(\tau) E Q [ Z ( T ) ∣ F τ ] = Z ( τ ) E Q [ L τ ∣ F 0 ] = L 0 = 1 \mathbb{E}_{\mathbb{Q}}[L_\tau\mid\mathcal{F}_0]=L_0=1 E Q [ L τ ∣ F 0 ] = L 0 = 1 E ∈ F τ E\in\mathcal{F}_\tau E ∈ F τ
E Q [ Z ( τ ) 1 { E } ] = E Q [ E Q [ Z ( T ) ∣ F τ ] 1 { E } ] = E Q [ E Q [ Z ( T ) 1 { E } ∣ F τ ] ] = E Q [ Z ( T ) 1 { E } ] = E Q [ d P d Q 1 { E } ] = P ( E ) . (109) \begin{aligned}
\mathbb{E}_{\mathbb{Q}}[Z(\tau)\mathbf{1}\{E\}]&=\mathbb{E}_{\mathbb{Q}}\left[\mathbb{E}_{\mathbb{Q}}[Z(T)\mid\mathcal{F}_\tau]\mathbf{1}\{E\}\right]=\mathbb{E}_{\mathbb{Q}}\left[\mathbb{E}_{\mathbb{Q}}[Z(T)\mathbf{1}\{E\}\mid\mathcal{F}_\tau]\right]\\
&=\mathbb{E}_{\mathbb{Q}}[Z(T)\mathbf{1}\{E\}]=\mathbb{E}_{\mathbb{Q}}\left[\frac{\mathrm{d}\mathbb{P}}{\mathrm{d}\mathbb{Q}}\mathbf{1}\{E\}\right]=\mathbb{P}(E). \tag*{(109)}
\end{aligned} E Q [ Z ( τ ) 1 { E }] = E Q [ E Q [ Z ( T ) ∣ F τ ] 1 { E } ] = E Q [ E Q [ Z ( T ) 1 { E } ∣ F τ ] ] = E Q [ Z ( T ) 1 { E }] = E Q [ d Q d P 1 { E } ] = P ( E ) . (109) By Equation (109) and the Cauchy–Schwarz inequality, we obtain
P ( E ) = E Q [ Z ( τ ) exp ( − 1 2 ∫ 0 τ ∥ θ s ∥ 2 d s ) ⋅ exp ( 1 2 ∫ 0 τ ∥ θ s ∥ 2 d s ) 1 { E } ] ≤ E Q [ Z ( τ ) 2 exp ( − ∫ 0 τ ∥ θ s ∥ 2 d s ) ] 1 / 2 E Q [ exp ( ∫ 0 τ ∥ θ s ∥ 2 d s ) 1 { E } ] 1 / 2 . (110) \begin{aligned}
\mathbb{P}(E)&=\mathbb{E}_{\mathbb{Q}}\left[Z(\tau)\exp\left(-\frac{1}{2}\int_0^\tau\|\theta_s\|^2\,\mathrm{d}s\right)\cdot\exp\left(\frac{1}{2}\int_0^\tau\|\theta_s\|^2\,\mathrm{d}s\right)\mathbf{1}\{E\}\right]\\
&\le\mathbb{E}_{\mathbb{Q}}\left[Z(\tau)^2\exp\left(-\int_0^\tau\|\theta_s\|^2\,\mathrm{d}s\right)\right]^{1/2}\mathbb{E}_{\mathbb{Q}}\left[\exp\left(\int_0^\tau\|\theta_s\|^2\,\mathrm{d}s\right)\mathbf{1}\{E\}\right]^{1/2}. \tag*{(110)}
\end{aligned} P ( E ) = E Q [ Z ( τ ) exp ( − 2 1 ∫ 0 τ ∥ θ s ∥ 2 d s ) ⋅ exp ( 2 1 ∫ 0 τ ∥ θ s ∥ 2 d s ) 1 { E } ] ≤ E Q [ Z ( τ ) 2 exp ( − ∫ 0 τ ∥ θ s ∥ 2 d s ) ] 1/2 E Q [ exp ( ∫ 0 τ ∥ θ s ∥ 2 d s ) 1 { E } ] 1/2 . (110) For the first expectation on the right-hand side, since Z ( τ ) = Z e n v Z d y n ( τ ) Z(\tau)=Z_{\mathrm{env}}Z_{\mathrm{dyn}}(\tau) Z ( τ ) = Z env Z dyn ( τ ) Z e n v Z_{\mathrm{env}} Z env F 0 \mathcal{F}_0 F 0 (104) and (108) that
E Q [ Z ( τ ) 2 exp ( − ∫ 0 τ ∥ θ s ∥ 2 d s ) ] = E Q [ Z e n v 2 E Q [ Z d y n ( τ ) 2 exp ( − ∫ 0 τ ∥ θ s ∥ 2 d s ) | F 0 ] ] = E Q [ Z e n v 2 E Q [ L τ ∣ F 0 ] ] = E Q [ Z e n v 2 ] = exp ( N λ 2 σ N 2 κ N 2 ) , (111) \begin{aligned}
\mathbb{E}_{\mathbb{Q}}\left[Z(\tau)^2\exp\left(-\int_0^\tau\|\theta_s\|^2\,\mathrm{d}s\right)\right]
&=\mathbb{E}_{\mathbb{Q}}\left[Z_{\mathrm{env}}^2\mathbb{E}_{\mathbb{Q}}\left[Z_{\mathrm{dyn}}(\tau)^2\exp\left(-\int_0^\tau\|\theta_s\|^2\,\mathrm{d}s\right)\middle|\mathcal{F}_0\right]\right]\\
&=\mathbb{E}_{\mathbb{Q}}\left[Z_{\mathrm{env}}^2\mathbb{E}_{\mathbb{Q}}[L_\tau\mid\mathcal{F}_0]\right]=\mathbb{E}_{\mathbb{Q}}[Z_{\mathrm{env}}^2]=\exp\left(\frac{N\lambda^2\sigma_N^2}{\kappa_N^2}\right), \tag*{(111)}
\end{aligned} E Q [ Z ( τ ) 2 exp ( − ∫ 0 τ ∥ θ s ∥ 2 d s ) ] = E Q [ Z env 2 E Q [ Z dyn ( τ ) 2 exp ( − ∫ 0 τ ∥ θ s ∥ 2 d s ) F 0 ] ] = E Q [ Z env 2 E Q [ L τ ∣ F 0 ] ] = E Q [ Z env 2 ] = exp ( κ N 2 N λ 2 σ N 2 ) , (111) where the third equality uses E Q [ L τ ∣ F 0 ] = 1 \mathbb{E}_{\mathbb{Q}}[L_\tau\mid\mathcal{F}_0]=1 E Q [ L τ ∣ F 0 ] = 1 (111) into Equation (110) and recalling that ∥ θ s ∥ 2 = β 2 ∥ ∇ S N H s i g ( Y s ) ∥ 2 \|\theta_s\|^2=\beta^2\|\nabla_{\mathcal{S}_N}H_{\mathrm{sig}}(Y_s)\|^2 ∥ θ s ∥ 2 = β 2 ∥ ∇ S N H sig ( Y s ) ∥ 2 □ \square □
Remark 6.13. If ∫ 0 τ ∥ ∇ S N H s i g ( Y t ( e ) ) ∥ 2 d t ≤ Γ \int_0^\tau\|\nabla_{\mathcal{S}_N}H_{\mathrm{sig}}(Y_t(e))\|^2\,\mathrm{d}t\le\Gamma ∫ 0 τ ∥ ∇ S N H sig ( Y t ( e )) ∥ 2 d t ≤ Γ E E E Γ ≥ 0 \Gamma\ge0 Γ ≥ 0 Theorem 6.12 implies that P ( E ) ≤ exp ( N λ 2 σ N 2 2 κ N 2 + 1 2 β 2 Γ ) Q ( E ) 1 / 2 \mathbb{P}(E)\le\exp\left(\frac{N\lambda^2\sigma_N^2}{2\kappa_N^2}+\frac{1}{2}\beta^2\Gamma\right)\mathbb{Q}(E)^{1/2} P ( E ) ≤ exp ( 2 κ N 2 N λ 2 σ N 2 + 2 1 β 2 Γ ) Q ( E ) 1/2
Critical Recovery via the Reverse Planting Trick
In this subsection, we use the reverse planting trick to establish Theorem 6.5 . Our proof makes use of the following regularity estimate for the null Hamiltonian.
Lemma 6.14 (Regularity of the Null Hamiltonian). Suppose that λ σ N ≤ 1 \lambda\sigma_N\le1 λ σ N ≤ 1 C 0 , c > 0 C_0,c>0 C 0 , c > 0 k k k N N N
P ( ∥ H n u l l ∥ G 4 > C 0 N ) ≤ exp ( − c N ) . \mathbb{P}\left(\|H_{\mathrm{null}}\|_{\mathcal{G}_4}>C_0N\right)\le\exp(-cN). P ( ∥ H null ∥ G 4 > C 0 N ) ≤ exp ( − c N ) . Proof. Since σ N 2 ≍ k N − k − 1 2 \sigma_N^2 \asymp_k N^{-\frac{k-1}{2}} σ N 2 ≍ k N − 2 k − 1 N N N σ N ≤ 1 \sigma_N \le1 σ N ≤ 1 Lemmas 6.6 and 6.7 and Definition 6.8 , the null Hamiltonian takes the form
H n u l l = κ N − 1 R ⟨ e 1 , ⋅ ⟩ + W , R = ∥ U ∥ , W = d H 0 ⊥ = H 0 − κ N − 1 ⟨ ζ , ⋅ ⟩ . H_{\mathrm{null}}=\kappa_N^{-1}R\langle e_1,\cdot\rangle+W,\qquad R=\lVert U\rVert,\qquad W\overset{\mathrm{d}}{=}H_0^\perp=H_0-\kappa_N^{-1}\langle\zeta,\cdot\rangle. H null = κ N − 1 R ⟨ e 1 , ⋅ ⟩ + W , R = ∥ U ∥ , W = d H 0 ⊥ = H 0 − κ N − 1 ⟨ ζ , ⋅ ⟩ . where U ∼ N ( N λ α N θ ⋆ , σ N 2 I ) U\sim\mathcal{N}(N\lambda\alpha_N\theta^\star,\sigma_N^2 I) U ∼ N ( N λ α N θ ⋆ , σ N 2 I ) ζ ∼ N ( 0 , σ N 2 I ) \zeta\sim\mathcal{N}(0,\sigma_N^2 I) ζ ∼ N ( 0 , σ N 2 I ) v ∈ R N v\in\mathbb{R}^N v ∈ R N ∥ ⟨ v , ⋅ ⟩ ∥ G 4 ≲ N ∥ v ∥ \lVert\langle v,\cdot\rangle\rVert_{\mathcal{G}_4}\lesssim\sqrt{N}\lVert v\rVert ∥⟨ v , ⋅ ⟩ ∥ G 4 ≲ N ∥ v ∥ Lemma 3.9 applied to the unit vector v / ∥ v ∥ v/\lVert v\rVert v / ∥ v ∥ κ N ≥ 2 / 3 \kappa_N\ge2/3 κ N ≥ 2/3
∥ H n u l l ∥ G 4 ≲ N R + ∥ W ∥ G 4 , ∥ H 0 ⊥ ∥ G 4 ≲ ∥ H 0 ∥ G 4 + N ∥ ζ ∥ . (112) \lVert H_{\mathrm{null}}\rVert_{\mathcal{G}_4}\lesssim\sqrt{N}R+\lVert W\rVert_{\mathcal{G}_4},\qquad\lVert H_0^\perp\rVert_{\mathcal{G}_4}\lesssim\lVert H_0\rVert_{\mathcal{G}_4}+\sqrt{N}\lVert\zeta\rVert. \tag*{(112)} ∥ H null ∥ G 4 ≲ N R + ∥ W ∥ G 4 , ∥ H 0 ⊥ ∥ G 4 ≲ ∥ H 0 ∥ G 4 + N ∥ ζ ∥ . (112) We now show that R ≲ N R\lesssim\sqrt{N} R ≲ N ∥ W ∥ G 4 ≲ k N \lVert W\rVert_{\mathcal{G}_4}\lesssim_k N ∥ W ∥ G 4 ≲ k N σ N 2 = κ N N α N \sigma_N^2=\kappa_N\sqrt{N}\alpha_N σ N 2 = κ N N α N κ N ≥ 2 / 3 \kappa_N\ge2/3 κ N ≥ 2/3 λ σ N ≤ 1 \lambda\sigma_N\le1 λ σ N ≤ 1 σ N ≤ 1 \sigma_N\le1 σ N ≤ 1
R = ∥ U ∥ ≤ N λ α N + ∥ U − N λ α N θ ⋆ ∥ = N λ σ N 2 κ N + ∥ U − N λ α N θ ⋆ ∥ ≲ N + ∥ U − N λ α N θ ⋆ ∥ . R=\lVert U\rVert\le N\lambda\alpha_N+\lVert U-N\lambda\alpha_N\theta^\star\rVert=\frac{\sqrt{N}\lambda\sigma_N^2}{\kappa_N}+\lVert U-N\lambda\alpha_N\theta^\star\rVert\lesssim\sqrt{N}+\lVert U-N\lambda\alpha_N\theta^\star\rVert. R = ∥ U ∥ ≤ N λ α N + ∥ U − N λ α N θ ⋆ ∥ = κ N N λ σ N 2 + ∥ U − N λ α N θ ⋆ ∥ ≲ N + ∥ U − N λ α N θ ⋆ ∥ . Since U − N λ α N θ ⋆ ∼ N ( 0 , σ N 2 I ) U-N\lambda\alpha_N\theta^\star\sim\mathcal{N}(0,\sigma_N^2I) U − N λ α N θ ⋆ ∼ N ( 0 , σ N 2 I ) σ N ≤ 1 \sigma_N\le1 σ N ≤ 1
P ( R > C N ) ≤ exp ( − c N ) . (113) \mathbb{P}(R>C\sqrt{N})\le\exp(-cN). \tag*{(113)} P ( R > C N ) ≤ exp ( − c N ) . (113) We now bound ∥ W ∥ G 4 \lVert W\rVert_{\mathcal{G}_4} ∥ W ∥ G 4 ζ ∼ N ( 0 , σ N 2 I ) \zeta\sim\mathcal{N}(0,\sigma_N^2I) ζ ∼ N ( 0 , σ N 2 I ) σ N ≤ 1 \sigma_N\le1 σ N ≤ 1 Lemma 3.7 with n = 4 n=4 n = 4 r = N r=N r = N ∥ H 0 ∥ G 4 ≲ k N \lVert H_0\rVert_{\mathcal{G}_4}\lesssim_k N ∥ H 0 ∥ G 4 ≲ k N ∥ ζ ∥ ≲ N \lVert\zeta\rVert\lesssim\sqrt{N} ∥ ζ ∥ ≲ N 1 − exp ( − c k N ) 1-\exp(-c_kN) 1 − exp ( − c k N ) W = d H 0 ⊥ W\overset{\mathrm{d}}{=}H_0^\perp W = d H 0 ⊥ (112)
P ( ∥ W ∥ G 4 > C k N ) = P ( ∥ H 0 ⊥ ∥ G 4 > C k N ) ≤ exp ( − c k N ) . (114) \mathbb{P}(\lVert W\rVert_{\mathcal{G}_4}>C_kN)=\mathbb{P}(\lVert H_0^\perp\rVert_{\mathcal{G}_4}>C_kN)\le\exp(-c_kN). \tag*{(114)} P (∥ W ∥ G 4 > C k N ) = P (∥ H 0 ⊥ ∥ G 4 > C k N ) ≤ exp ( − c k N ) . (114) On the event { R ≤ C N } ∩ { ∥ W ∥ G 4 ≤ C k N } \{R\le C\sqrt{N}\}\cap\{\lVert W\rVert_{\mathcal{G}_4}\le C_kN\} { R ≤ C N } ∩ {∥ W ∥ G 4 ≤ C k N } (112) yields ∥ H n u l l ∥ G 4 ≲ k N \lVert H_{\mathrm{null}}\rVert_{\mathcal{G}_4}\lesssim_k N ∥ H null ∥ G 4 ≲ k N C 0 C_0 C 0 k k k ∥ H n u l l ∥ G 4 ≤ C 0 N \lVert H_{\mathrm{null}}\rVert_{\mathcal{G}_4}\le C_0N ∥ H null ∥ G 4 ≤ C 0 N (113) and (114)
P ( ∥ H n u l l ∥ G 4 > C 0 N ) ≤ P ( R > C N ) + P ( ∥ W ∥ G 4 > C k N ) ≤ exp ( − c N ) , \mathbb{P}(\lVert H_{\mathrm{null}}\rVert_{\mathcal{G}_4}>C_0N)\le\mathbb{P}(R>C\sqrt{N})+\mathbb{P}(\lVert W\rVert_{\mathcal{G}_4}>C_kN)\le\exp(-cN), P (∥ H null ∥ G 4 > C 0 N ) ≤ P ( R > C N ) + P (∥ W ∥ G 4 > C k N ) ≤ exp ( − c N ) , Our proof also makes use of the following Gronwall-type comparison inequality from Ben Arous, Gheissari, and Jagannath [10 ] .
Lemma 6.15 ([10 Lemma 5.1] ). Let a , c , T > 0 a,c,T>0 a , c , T > 0 f : [ 0 , T ] → R f:[0,T]\to\mathbb{R} f : [ 0 , T ] → R
f ( t ) ≥ a + c ∫ 0 t f ( s ) k − 1 d s ∀ t ∈ [ 0 , T ] . f(t)\ge a+c\int_0^t f(s)^{k-1}\,ds\qquad\forall t\in[0,T]. f ( t ) ≥ a + c ∫ 0 t f ( s ) k − 1 d s ∀ t ∈ [ 0 , T ] . Then, the following holds for every t ∈ [ 0 , T ] t\in[0,T] t ∈ [ 0 , T ] ( k − 2 ) c a k − 2 t < 1 (k-2)ca^{k-2}t<1 ( k − 2 ) c a k − 2 t < 1
f ( t ) ≥ [ a − ( k − 2 ) − ( k − 2 ) c t ] − 1 k − 2 . f(t)\ge\left[a^{-(k-2)}-(k-2)ct\right]^{-\frac{1}{k-2}}. f ( t ) ≥ [ a − ( k − 2 ) − ( k − 2 ) c t ] − k − 2 1 . Equipped with Lemmas 6.14 and 6.15 , we now establish Theorem 6.5 by applying the adaptive contiguity estimate up to a carefully chosen stopping time, thereby proving that the null drift is unlikely to overtake the signal drift before the dynamics achieves critical recovery.
Proof of Theorem 6.5
Proof. Throughout this proof, we work under the planted law. We assume N N N C β , k C_{\beta,k} C β , k β \beta β k k k e e e H H H e = e ( H , ω ) e=e(H,\omega) e = e ( H , ω ) e e e ω \omega ω H H H ( B t ) t ≥ 0 (B_t)_{t\geq0} ( B t ) t ≥ 0 ω \omega ω X ~ 0 \widetilde{X}_0 X 0 ( B ~ t ) 0 ≤ t ≤ T a v (\widetilde{B}_t)_{0\leq t\leq T_{\mathrm{av}}} ( B t ) 0 ≤ t ≤ T av
Consider the trajectory ( X t ( e ) ) t ≥ 0 (X_t(e))_{t\geq0} ( X t ( e ) ) t ≥ 0 H H H e e e (85) to this trajectory, we define
Y t ≔ Φ X t ( e ) , m t ≔ m ϕ ( Y t ) . (115) Y_t \coloneqq\Phi X_t(e), \qquad m_t \coloneqq m_\phi(Y_t). \tag*{(115)} Y t : = Φ X t ( e ) , m t : = m ϕ ( Y t ) . (115) Since e = Φ T Φ e e=\Phi^\mathsf{T}\Phi e e = Φ T Φ e (85) implies that Y t = Y t ( Φ e ) Y_t=Y_t(\Phi e) Y t = Y t ( Φ e ) Lemma 6.7 , ( Y t ) t ≥ 0 (Y_t)_{t\geq0} ( Y t ) t ≥ 0 H ϕ H_\phi H ϕ Y 0 = Φ e Y_0=\Phi e Y 0 = Φ e Lemma 6.7 defines m ϕ ( y ) = m ( Φ T y ) m_\phi(y)=m(\Phi^\mathsf{T}y) m ϕ ( y ) = m ( Φ T y ) m t = m ( X t ( e ) ) m_t=m(X_t(e)) m t = m ( X t ( e )) m t m_t m t H ( x ) = H ϕ ( Φ x ) H(x)=H_\phi(\Phi x) H ( x ) = H ϕ ( Φ x ) H ϕ H_\phi H ϕ ( Φ , R , W ) (\Phi,R,W) ( Φ , R , W ) Lemma 6.7 , the gauge-transformed initial state Φ e \Phi e Φ e ( Φ , R , W , ω ) (\Phi,R,W,\omega) ( Φ , R , W , ω ) § 6.4 . Now, let δ = λ σ N N − ϑ \delta=\lambda\sigma_NN^{-\vartheta} δ = λ σ N N − ϑ
A ≔ { ∥ Y 0 − N e 1 ∥ ≤ δ N } . \mathcal{A}\coloneqq\left\{\left\|Y_0-\sqrt{N}e_1\right\|\leq\delta\sqrt{N}\right\}. A : = { Y 0 − N e 1 ≤ δ N } . Recall that u H = E π [ ∇ S N H ] u_H=\mathbb{E}_{\pi}[\nabla_{\mathbb{S}_N}H] u H = E π [ ∇ S N H ] u ^ H = N u H ∥ u H ∥ \widehat{u}_H=\frac{\sqrt{N}u_H}{\|u_H\|} u H = ∥ u H ∥ N u H Φ ∈ O ( N , R ) \Phi\in O(N,\mathbb{R}) Φ ∈ O ( N , R ) Φ u ^ H = N e 1 \Phi\widehat{u}_H=\sqrt{N}e_1 Φ u H = N e 1 A = { ∥ e − u ^ H ∥ ≤ δ N } \mathcal{A}=\{\|e-\widehat{u}_H\|\leq\delta\sqrt{N}\} A = { ∥ e − u H ∥ ≤ δ N } Theorem 6.5 .
We begin by fixing some constants. Since σ N 2 ≍ k N k − 1 2 \sigma_N^2\asymp_k N^{\frac{k-1}{2}} σ N 2 ≍ k N 2 k − 1 k − 2 4 ≤ γ ≤ k − 2 4 + 1 4 k \frac{k-2}{4}\leq\gamma\leq\frac{k-2}{4}+\frac{1}{4k} 4 k − 2 ≤ γ ≤ 4 k − 2 + 4 k 1
N − 1 / 4 ≲ k λ σ N ≲ N − 1 4 + 1 4 k , min { 1 , λ N − 1 / 4 } ≍ k λ σ N , min { N , λ 2 N k − 3 2 } ≍ k N λ 2 σ N 2 → ∞ . N^{-1/4}\lesssim_k\lambda\sigma_N\lesssim N^{-\frac14+\frac{1}{4k}},\qquad\min\left\{1,\lambda N^{-1/4}\right\}\asymp_k\lambda\sigma_N,\qquad\min\left\{N,\lambda^2N^{\frac{k-3}{2}}\right\}\asymp_k N\lambda^2\sigma_N^2\to\infty. N − 1/4 ≲ k λ σ N ≲ N − 4 1 + 4 k 1 , min { 1 , λ N − 1/4 } ≍ k λ σ N , min { N , λ 2 N 2 k − 3 } ≍ k N λ 2 σ N 2 → ∞. Moreover, since Φ u ^ H = N e 1 \Phi\widehat{u}_H=\sqrt{N}e_1 Φ u H = N e 1 m ϕ ( N e 1 ) = m ( u ^ H ) m_\phi(\sqrt{N}e_1)=m(\widehat{u}_H) m ϕ ( N e 1 ) = m ( u H ) c 0 > 0 c_0>0 c 0 > 0 k k k
P [ m ϕ ( N e 1 ) ≥ 2 c 0 λ σ N ] ≥ 1 − exp ( − c N λ 2 σ N 2 ) . (116) \mathbb{P}\left[m_\phi(\sqrt{N}e_1)\geq2c_0\lambda\sigma_N\right]\geq1-\exp\left(-cN\lambda^2\sigma_N^2\right). \tag*{(116)} P [ m ϕ ( N e 1 ) ≥ 2 c 0 λ σ N ] ≥ 1 − exp ( − c N λ 2 σ N 2 ) . (116) We now define
a 0 ≔ c 0 λ σ N , η ≔ 3 β k 16 , T λ ≔ C T λ a 0 k − 2 , C T ≔ 2 k − 1 ( k − 2 ) η . a_0\coloneqq c_0\lambda\sigma_N,\qquad\eta\coloneqq\frac{3\beta k}{16},\qquad T_\lambda\coloneqq\frac{C_T}{\lambda a_0^{k-2}},\qquad C_T\coloneqq\frac{2^{k-1}}{(k-2)\eta}. a 0 : = c 0 λ σ N , η : = 16 3 β k , T λ : = λ a 0 k − 2 C T , C T : = ( k − 2 ) η 2 k − 1 . Since C β , k C_{\beta,k} C β , k
λ a 0 k − 2 = c 0 k − 2 λ k − 1 σ N k − 2 ≍ k C β , k k − 1 N ( k − 1 ) ( γ − k − 2 4 ) , \lambda a_0^{k-2}=c_0^{k-2}\lambda^{k-1}\sigma_N^{k-2}\asymp_k C_{\beta,k}^{k-1}N^{(k-1)\left(\gamma-\frac{k-2}{4}\right)}, λ a 0 k − 2 = c 0 k − 2 λ k − 1 σ N k − 2 ≍ k C β , k k − 1 N ( k − 1 ) ( γ − 4 k − 2 ) , we may assume that λ a 0 k − 2 ≥ C T \lambda a_0^{k-2}\geq C_T λ a 0 k − 2 ≥ C T T λ ≤ 1 T_\lambda\leq1 T λ ≤ 1
We now compute the signal drift. Since m ϕ m_\phi m ϕ (8) implies ∇ S N m ϕ ( y ) = N − 1 / 2 Q y ϕ \nabla_{\mathbb{S}^N}m_\phi(y)=N^{-1/2}Q_y\phi ∇ S N m ϕ ( y ) = N − 1/2 Q y ϕ ∥ Q y ϕ ∥ 2 = 1 − m ϕ ( y ) 2 \lVert Q_y\phi\rVert^2=1-m_\phi(y)^2 ∥ Q y ϕ ∥ 2 = 1 − m ϕ ( y ) 2 H ϕ = H n u l l + H s i g H_\phi=H_{\mathrm{null}}+H_{\mathrm{sig}} H ϕ = H null + H sig H s i g = N λ m ϕ k − N λ σ N 2 κ N 2 m ϕ H_{\mathrm{sig}}=N\lambda m_\phi^k-\frac{N\lambda\sigma_N^2}{\kappa_N^2}m_\phi H sig = N λ m ϕ k − κ N 2 N λ σ N 2 m ϕ Lemma 6.7 , we obtain the following from Equations (19) and (105) for every y ∈ S N y\in\mathbb{S}^N y ∈ S N
( L p l − L n u l l ) m ϕ ( y ) = β ⟨ ∇ S N H s i g ( y ) , ∇ S N m ϕ ( y ) ⟩ = β λ ( k m ϕ ( y ) k − 1 − σ N 2 κ N 2 ) ( 1 − m ϕ ( y ) 2 ) , ∥ ∇ S N H s i g ( y ) ∥ 2 = N λ 2 ( k m ϕ ( y ) k − 1 − σ N 2 κ N 2 ) 2 ( 1 − m ϕ ( y ) 2 ) . (117) \begin{aligned}
(\mathcal{L}_{\mathrm{pl}}-\mathcal{L}_{\mathrm{null}})m_\phi(y)&=\beta\langle\nabla_{\mathbb{S}^N}H_{\mathrm{sig}}(y),\nabla_{\mathbb{S}^N}m_\phi(y)\rangle=\beta\lambda\left(km_\phi(y)^{k-1}-\frac{\sigma_N^2}{\kappa_N^2}\right)(1-m_\phi(y)^2),\\
\lVert\nabla_{\mathbb{S}^N}H_{\mathrm{sig}}(y)\rVert^2&=N\lambda^2\left(km_\phi(y)^{k-1}-\frac{\sigma_N^2}{\kappa_N^2}\right)^2(1-m_\phi(y)^2). \tag*{(117)}
\end{aligned} ( L pl − L null ) m ϕ ( y ) ∥ ∇ S N H sig ( y ) ∥ 2 = β ⟨ ∇ S N H sig ( y ) , ∇ S N m ϕ ( y )⟩ = β λ ( k m ϕ ( y ) k − 1 − κ N 2 σ N 2 ) ( 1 − m ϕ ( y ) 2 ) , = N λ 2 ( k m ϕ ( y ) k − 1 − κ N 2 σ N 2 ) 2 ( 1 − m ϕ ( y ) 2 ) . (117) Since λ σ N ≳ N − 1 / 4 \lambda\sigma_N\gtrsim N^{-1/4} λ σ N ≳ N − 1/4 σ N 2 ≍ k N − k − 1 2 \sigma_N^2\asymp_k N^{-\frac{k-1}{2}} σ N 2 ≍ k N − 2 k − 1 σ N 2 κ N 2 ≤ k 2 ( a 0 / 2 ) k − 1 \frac{\sigma_N^2}{\kappa_N^2}\leq\frac{k}{2}(a_0/2)^{k-1} κ N 2 σ N 2 ≤ 2 k ( a 0 /2 ) k − 1 C r N ≤ 1 / 2 C r_N\leq1/2 C r N ≤ 1/2 r N → 0 r_N\to0 r N → 0 a 0 / 2 ≤ m ϕ ( y ) ≤ C r N a_0/2\leq m_\phi(y)\leq Cr_N a 0 /2 ≤ m ϕ ( y ) ≤ C r N σ N 2 κ N 2 ≤ k 2 m ϕ ( y ) k − 1 \frac{\sigma_N^2}{\kappa_N^2}\leq\frac{k}{2}m_\phi(y)^{k-1} κ N 2 σ N 2 ≤ 2 k m ϕ ( y ) k − 1 1 − m ϕ ( y ) 2 ≥ 3 / 4 1-m_\phi(y)^2\geq3/4 1 − m ϕ ( y ) 2 ≥ 3/4 (117) that
( L p l − L n u l l ) m ϕ ( y ) ≥ 2 η λ m ϕ ( y ) k − 1 , ∥ ∇ S N H s i g ( y ) ∥ 2 ≤ k 2 N λ 2 m ϕ ( y ) 2 k − 2 . (118) (\mathcal{L}_{\mathrm{pl}}-\mathcal{L}_{\mathrm{null}})m_\phi(y)\geq2\eta\lambda m_\phi(y)^{k-1},\qquad\lVert\nabla_{\mathbb{S}^N}H_{\mathrm{sig}}(y)\rVert^2\leq k^2N\lambda^2m_\phi(y)^{2k-2}. \tag*{(118)} ( L pl − L null ) m ϕ ( y ) ≥ 2 η λ m ϕ ( y ) k − 1 , ∥ ∇ S N H sig ( y ) ∥ 2 ≤ k 2 N λ 2 m ϕ ( y ) 2 k − 2 . (118) We now define the Dynkin martingale of m ϕ m_\phi m ϕ Definition 6.8 , conditional on the environment, ( Y t ) t ≥ 0 (Y_t)_{t\geq0} ( Y t ) t ≥ 0 H ϕ H_\phi H ϕ β \beta β Lemma 3.11 , the following process is a continuous martingale conditional on the environment:
M t ≔ m t − m 0 − ∫ 0 t ( L p l m ϕ ) ( Y s ) d s . (119) M_t\coloneqq m_t-m_0-\int_0^t(\mathcal{L}_{\mathrm{pl}}m_\phi)(Y_s)\,ds. \tag*{(119)} M t : = m t − m 0 − ∫ 0 t ( L pl m ϕ ) ( Y s ) d s . (119) We now define the following regularity events:
I ≔ { m 0 ≥ a 0 } , E r e g ≔ { ∥ H n u l l ∥ G 4 ≤ C 0 N } , M T λ ≔ { sup 0 ≤ t ≤ T λ ∣ M t ∣ ≤ a 0 32 } . \mathcal{I}\coloneqq\{m_0\geq a_0\},\qquad\mathcal{E}_{\mathrm{reg}}\coloneqq\{\lVert H_{\mathrm{null}}\rVert_{\mathcal{G}_4}\leq C_0N\},\qquad\mathcal{M}_{T_\lambda}\coloneqq\left\{\sup_{0\leq t\leq T_\lambda}|M_t|\leq\frac{a_0}{32}\right\}. I : = { m 0 ≥ a 0 } , E reg : = {∥ H null ∥ G 4 ≤ C 0 N } , M T λ : = { 0 ≤ t ≤ T λ sup ∣ M t ∣ ≤ 32 a 0 } . where C 0 > 0 C_0>0 C 0 > 0 Lemma 6.14 . We shall show that each of these events holds with high probability on A \mathcal{A} A
We begin with the event I \mathcal{I} I ϕ ∈ S N − 1 \phi\in\mathbb{S}^{N-1} ϕ ∈ S N − 1 m ϕ m_\phi m ϕ N − 1 / 2 N^{-1/2} N − 1/2 S N \mathbb{S}^N S N δ ≤ a 0 \delta\leq a_0 δ ≤ a 0 N N N δ = λ σ N N − ϑ \delta=\lambda\sigma_NN^{-\vartheta} δ = λ σ N N − ϑ a 0 = c 0 λ σ N a_0=c_0\lambda\sigma_N a 0 = c 0 λ σ N A ∩ { m ϕ ( N e 1 ) ≥ 2 a 0 } \mathcal{A}\cap\{m_\phi(\sqrt{N}e_1)\geq2a_0\} A ∩ { m ϕ ( N e 1 ) ≥ 2 a 0 }
m 0 = m ϕ ( Y 0 ) ≥ m ϕ ( N e 1 ) − 1 N ∥ Y 0 − N e 1 ∥ ≥ 2 a 0 − δ ≥ a 0 . m_0=m_\phi(Y_0)\geq m_\phi(\sqrt{N}e_1)-\frac{1}{\sqrt{N}}\lVert Y_0-\sqrt{N}e_1\rVert\geq2a_0-\delta\geq a_0. m 0 = m ϕ ( Y 0 ) ≥ m ϕ ( N e 1 ) − N 1 ∥ Y 0 − N e 1 ∥ ≥ 2 a 0 − δ ≥ a 0 . Consequently, A ∩ I c ⊆ { m ϕ ( N e 1 ) < 2 a 0 } \mathcal{A}\cap\mathcal{I}^c\subseteq\{m_\phi(\sqrt{N}e_1)<2a_0\} A ∩ I c ⊆ { m ϕ ( N e 1 ) < 2 a 0 } a 0 = c 0 λ σ N a_0=c_0\lambda\sigma_N a 0 = c 0 λ σ N (116) that
P ( A ∩ I c ) ≤ exp ( − c N λ 2 σ N 2 ) . (120) \mathbb{P}(\mathcal{A}\cap\mathcal{I}^c)\leq\exp(-cN\lambda^2\sigma_N^2). \tag*{(120)} P ( A ∩ I c ) ≤ exp ( − c N λ 2 σ N 2 ) . (120) We now consider the event E r e g \mathcal{E}_{\mathrm{reg}} E reg λ σ N ≤ 1 \lambda\sigma_N\leq1 λ σ N ≤ 1 Lemma 6.14 implies that
P ( E r e g c ) ≤ exp ( − c N ) . (121) \mathbb{P}(\mathcal{E}_{\mathrm{reg}}^c)\leq\exp(-cN). \tag*{(121)} P ( E reg c ) ≤ exp ( − c N ) . (121) Finally, we consider the event M T λ \mathcal{M}_{T_\lambda} M T λ ∇ S N m ϕ ( y ) = N − 1 / 2 Q y ϕ \nabla_{\mathbb{S}^N}m_\phi(y)=N^{-1/2}Q_y\phi ∇ S N m ϕ ( y ) = N − 1/2 Q y ϕ ∥ Q y ϕ ∥ ≤ 1 \lVert Q_y\phi\rVert\leq1 ∥ Q y ϕ ∥ ≤ 1 m ϕ m_\phi m ϕ Lemma 3.12 with A = N − 1 / 2 A=N^{-1/2} A = N − 1/2 Lemma 3.12 to the Dynkin martingale in Equation (119) conditionally on the environment, with T = T λ T=T_\lambda T = T λ r = a 0 / 32 r=a_0/32 r = a 0 /32
P ( M T λ c ) ≤ 2 exp ( − N ( a 0 / 32 ) 2 2 T λ ) ≤ 2 exp ( − c N λ 2 σ N 2 ) . (122) \mathbb{P}(\mathcal{M}_{T_\lambda}^c)\leq2\exp\left(-\frac{N(a_0/32)^2}{2T_\lambda}\right)\leq2\exp(-cN\lambda^2\sigma_N^2). \tag*{(122)} P ( M T λ c ) ≤ 2 exp ( − 2 T λ N ( a 0 /32 ) 2 ) ≤ 2 exp ( − c N λ 2 σ N 2 ) . (122) where the last step uses T λ ≤ 1 T_{\lambda} \le1 T λ ≤ 1 a 0 = c 0 λ σ N a_0 = c_0\lambda\sigma_N a 0 = c 0 λ σ N
We now define the stopping time up to which we apply the adaptive contiguity estimate. To this end, let ( F t ) t ≥ 0 (\mathcal{F}_t)_{t\ge0} ( F t ) t ≥ 0 ( Φ , R , W ) (\Phi,R,W) ( Φ , R , W ) ( Y t ) t ≥ 0 (Y_t)_{t\ge0} ( Y t ) t ≥ 0 Theorem 6.12 . Since m t m_t m t ( L n u l l m ϕ ) ( Y t ) (\mathcal{L}_{\mathrm{null}}m_\phi)(Y_t) ( L null m ϕ ) ( Y t ) ( F t ) (\mathcal{F}_t) ( F t ) ( F t ) (\mathcal{F}_t) ( F t )
T ∗ Y ≔ inf { t ≥ 0 : m t ≥ C r N } , T_*^Y \coloneqq\inf\{t\ge0:m_t\ge Cr_N\}, T ∗ Y : = inf { t ≥ 0 : m t ≥ C r N } , τ ↓ ≔ inf { t ≥ 0 : m t ≤ a 0 2 } , \tau^\downarrow\coloneqq\inf\left\{t\ge0:m_t\le\frac{a_0}{2}\right\}, τ ↓ : = inf { t ≥ 0 : m t ≤ 2 a 0 } , τ b a d ≔ inf { t ≥ 0 : ∣ ( L n u l l m ϕ ) ( Y t ) ∣ ≥ η λ m t k − 1 } , \tau^{\mathrm{bad}} \coloneqq\inf\left\{t\ge0:\left|(\mathcal{L}_{\mathrm{null}}m_\phi)(Y_t)\right|\ge\eta\lambda m_t^{k-1}\right\}, τ bad : = inf { t ≥ 0 : ∣ ( L null m ϕ ) ( Y t ) ∣ ≥ η λ m t k − 1 } , τ ≔ T λ ∧ T ∗ Y ∧ τ ↓ ∧ τ b a d . \tau\coloneqq T_\lambda\wedge T_*^Y\wedge\tau^\downarrow\wedge\tau^{\mathrm{bad}}. τ : = T λ ∧ T ∗ Y ∧ τ ↓ ∧ τ bad . For every t < τ t<\tau t < τ a 0 / 2 < m t < C r N a_0/2<m_t<Cr_N a 0 /2 < m t < C r N (118) implies the following:
∥ ∇ S N H s i g ( Y t ) ∥ 2 ≤ k 2 N λ 2 m t 2 k − 2 ∀ t < τ . (123) \left\|\nabla_{\mathcal{S}_N}H_{\mathrm{sig}}(Y_t)\right\|^2\le k^2N\lambda^2m_t^{2k-2}\qquad\forall\ t<\tau. \tag*{(123)} ∥ ∇ S N H sig ( Y t ) ∥ 2 ≤ k 2 N λ 2 m t 2 k − 2 ∀ t < τ . (123) Moreover, Equation (118) and τ ≤ τ b a d \tau\le\tau^{\mathrm{bad}} τ ≤ τ bad
( L p l m ϕ ) ( Y t ) = ( L n u l l m ϕ ) ( Y t ) + ( L p l − L n u l l ) m ϕ ( Y t ) (\mathcal{L}_{\mathrm{pl}}m_\phi)(Y_t)=(\mathcal{L}_{\mathrm{null}}m_\phi)(Y_t)+(\mathcal{L}_{\mathrm{pl}}-\mathcal{L}_{\mathrm{null}})m_\phi(Y_t) ( L pl m ϕ ) ( Y t ) = ( L null m ϕ ) ( Y t ) + ( L pl − L null ) m ϕ ( Y t ) ≥ 2 η λ m t k − 1 − ∣ ( L n u l l m ϕ ) ( Y t ) ∣ ≥ η λ m t k − 1 > 0 ∀ t < τ . (124) \ge2\eta\lambda m_t^{k-1}-|(\mathcal{L}_{\mathrm{null}}m_\phi)(Y_t)|\ge\eta\lambda m_t^{k-1}>0\qquad\forall\ t<\tau. \tag*{(124)} ≥ 2 η λ m t k − 1 − ∣ ( L null m ϕ ) ( Y t ) ∣ ≥ η λ m t k − 1 > 0 ∀ t < τ . (124) We now control the correlation up to time τ \tau τ M τ = { sup 0 ≤ t ≤ τ ∣ M t ∣ ≤ a 0 / 32 } \mathcal{M}_\tau=\{\sup_{0\le t\le\tau}|M_t|\le a_0/32\} M τ = { sup 0 ≤ t ≤ τ ∣ M t ∣ ≤ a 0 /32 } (119) and (124) , the following holds on I ∩ M τ \mathcal{I}\cap\mathcal{M}_\tau I ∩ M τ 0 ≤ s ≤ t ≤ τ 0\le s\le t\le\tau 0 ≤ s ≤ t ≤ τ
m t − m s = ∫ s t ( L p l m ϕ ) ( Y u ) d u + M t − M s ≥ M t − M s ≥ − a 0 16 . m_t-m_s=\int_s^t(\mathcal{L}_{\mathrm{pl}}m_\phi)(Y_u)\,\mathrm{d}u+M_t-M_s\ge M_t-M_s\ge-\frac{a_0}{16}. m t − m s = ∫ s t ( L pl m ϕ ) ( Y u ) d u + M t − M s ≥ M t − M s ≥ − 16 a 0 . Since M 0 = 0 M_0=0 M 0 = 0 m 0 ≥ a 0 m_0\ge a_0 m 0 ≥ a 0
m t ≥ m 0 + M t ≥ 31 a 0 32 ∀ t ≤ τ , m s ≤ m t + a 0 16 ≤ 2 m t ∀ s ≤ t ≤ τ . (125) m_t\ge m_0+M_t\ge\frac{31a_0}{32}\qquad\forall\ t\le\tau,\qquad m_s\le m_t+\frac{a_0}{16}\le2m_t\qquad\forall\ s\le t\le\tau. \tag*{(125)} m t ≥ m 0 + M t ≥ 32 31 a 0 ∀ t ≤ τ , m s ≤ m t + 16 a 0 ≤ 2 m t ∀ s ≤ t ≤ τ . (125) Similarly, we infer from Equations (119) and (124) that
m t = m 0 + ∫ 0 t ( L p l m ϕ ) ( Y s ) d s + M t ≥ 31 a 0 32 + η λ ∫ 0 t m s k − 1 d s ∀ t ≤ τ . (126) m_t=m_0+\int_0^t(\mathcal{L}_{\mathrm{pl}}m_\phi)(Y_s)\,\mathrm{d}s+M_t\ge\frac{31a_0}{32}+\eta\lambda\int_0^t m_s^{k-1}\,\mathrm{d}s\qquad\forall\ t\le\tau. \tag*{(126)} m t = m 0 + ∫ 0 t ( L pl m ϕ ) ( Y s ) d s + M t ≥ 32 31 a 0 + η λ ∫ 0 t m s k − 1 d s ∀ t ≤ τ . (126) We now show that the null drift is unlikely to dominate the signal drift before the dynamics achieves critical recovery. To this end, define the failure event
E f a i l ≔ A ∩ I ∩ E r e g ∩ M τ ∩ { τ = τ b a d < T ∗ Y } . \mathcal{E}_{\mathrm{fail}}\coloneqq\mathcal{A}\cap\mathcal{I}\cap\mathcal{E}_{\mathrm{reg}}\cap\mathcal{M}_\tau\cap\{\tau=\tau^{\mathrm{bad}}<T_*^Y\}. E fail : = A ∩ I ∩ E reg ∩ M τ ∩ { τ = τ bad < T ∗ Y } . Since A \mathcal{A} A I \mathcal{I} I Y 0 Y_0 Y 0 E r e g \mathcal{E}_{\mathrm{reg}} E reg τ \tau τ E f a i l ∈ F τ \mathcal{E}_{\mathrm{fail}}\in\mathcal{F}_\tau E fail ∈ F τ P ( E f a i l ) \mathbb{P}(\mathcal{E}_{\mathrm{fail}}) P ( E fail ) Theorem 6.12 , we first control the accumulated signal drift on E f a i l \mathcal{E}_{\mathrm{fail}} E fail (123) and (125) , the following holds on E f a i l \mathcal{E}_{\mathrm{fail}} E fail
∫ 0 τ ∥ ∇ S N H s i g ( Y t ) ∥ 2 d t ≤ k 2 N λ 2 ∫ 0 τ m t 2 k − 2 d t ≤ k 2 N λ 2 ( 2 m τ ) k − 1 ∫ 0 τ m t k − 1 d t . (127) \int_0^\tau\left\|\nabla_{\mathcal{S}_N}H_{\mathrm{sig}}(Y_t)\right\|^2\,\mathrm{d}t\le k^2N\lambda^2\int_0^\tau m_t^{2k-2}\,\mathrm{d}t\le k^2N\lambda^2(2m_\tau)^{k-1}\int_0^\tau m_t^{k-1}\,\mathrm{d}t. \tag*{(127)} ∫ 0 τ ∥ ∇ S N H sig ( Y t ) ∥ 2 d t ≤ k 2 N λ 2 ∫ 0 τ m t 2 k − 2 d t ≤ k 2 N λ 2 ( 2 m τ ) k − 1 ∫ 0 τ m t k − 1 d t . (127) Since m 0 + M τ ≥ 31 a 0 / 32 m_0+M_\tau\ge31a_0/32 m 0 + M τ ≥ 31 a 0 /32 (125) , we infer the following from Equations (119) and (124) :
η λ ∫ 0 τ m t k − 1 d t ≤ ∫ 0 τ ( L p l m ϕ ) ( Y t ) d t = m τ − m 0 − M τ ≤ m τ − a 0 2 . (128) \eta\lambda\int_0^\tau m_t^{k-1}\,\mathrm{d}t\le\int_0^\tau(\mathcal{L}_{\mathrm{pl}}m_\phi)(Y_t)\,\mathrm{d}t=m_\tau-m_0-M_\tau\le m_\tau-\frac{a_0}{2}. \tag*{(128)} η λ ∫ 0 τ m t k − 1 d t ≤ ∫ 0 τ ( L pl m ϕ ) ( Y t ) d t = m τ − m 0 − M τ ≤ m τ − 2 a 0 . (128) Moreover, since m τ ≥ a 0 / 2 m_\tau\ge a_0/2 m τ ≥ a 0 /2
m τ 2 k − 2 − ( a 0 2 ) 2 k − 2 ≥ m τ 2 k − 3 ( m τ − a 0 2 ) ≥ ( a 0 2 ) k − 2 m τ k − 1 ( m τ − a 0 2 ) . (129) m_\tau^{2k-2}-\left(\frac{a_0}{2}\right)^{2k-2}\ge m_\tau^{2k-3}\left(m_\tau-\frac{a_0}{2}\right)\ge\left(\frac{a_0}{2}\right)^{k-2}m_\tau^{k-1}\left(m_\tau-\frac{a_0}{2}\right). \tag*{(129)} m τ 2 k − 2 − ( 2 a 0 ) 2 k − 2 ≥ m τ 2 k − 3 ( m τ − 2 a 0 ) ≥ ( 2 a 0 ) k − 2 m τ k − 1 ( m τ − 2 a 0 ) . (129) Combining Equations (127) to (129) , we infer that the following holds on E f a i l \mathcal{E}_{\mathrm{fail}} E fail
∫ 0 τ ∥ ∇ S N H s i g ( Y t ) ∥ 2 d t ≤ 2 k − 1 k 2 N λ η m τ k − 1 ( m τ − a 0 2 ) ≤ 2 k − 1 k 2 N η 3 λ ( a 0 / 2 ) k − 2 [ η 2 λ 2 m τ 2 k − 2 − η 2 λ 2 ( a 0 2 ) 2 k − 2 ] . (130) \begin{aligned}
\int_0^\tau\left\|\nabla\mathcal{S}_N H_{\mathrm{sig}}(Y_t)\right\|^2\,\mathrm{d}t\le\frac{2^{k-1}k^2N\lambda}{\eta}m_\tau^{k-1}\left(m_\tau-\frac{a_0}{2}\right) \\
&\le\frac{2^{k-1}k^2N}{\eta^3\lambda(a_0/2)^{k-2}}\left[\eta^2\lambda^2m_\tau^{2k-2}-\eta^2\lambda^2\left(\frac{a_0}{2}\right)^{2k-2}\right]. \tag*{(130)}
\end{aligned} ∫ 0 τ ∥ ∇ S N H sig ( Y t ) ∥ 2 d t ≤ η 2 k − 1 k 2 N λ m τ k − 1 ( m τ − 2 a 0 ) ≤ η 3 λ ( a 0 /2 ) k − 2 2 k − 1 k 2 N [ η 2 λ 2 m τ 2 k − 2 − η 2 λ 2 ( 2 a 0 ) 2 k − 2 ] . (130) Since τ = τ b a d \tau=\tau^{\mathrm{bad}} τ = τ bad E f a i l \mathcal{E}_{\mathrm{fail}} E fail
η 2 λ 2 m τ 2 k − 2 ≤ ∣ ( L n u l l m ϕ ) ( Y τ ) ∣ 2 ≤ sup t ∈ [ 0 , T λ ] ∣ ( L n u l l m ϕ ) ( Y t ) ∣ 2 . (131) \eta^2\lambda^2m_\tau^{2k-2}\le\left|(\mathcal{L}_{\mathrm{null}}m_\phi)(Y_\tau)\right|^2\le\sup_{t\in[0,T_\lambda]}\left|(\mathcal{L}_{\mathrm{null}}m_\phi)(Y_t)\right|^2. \tag*{(131)} η 2 λ 2 m τ 2 k − 2 ≤ ∣ ( L null m ϕ ) ( Y τ ) ∣ 2 ≤ t ∈ [ 0 , T λ ] sup ∣ ( L null m ϕ ) ( Y t ) ∣ 2 . (131) Let c ∗ c_* c ∗ C ∗ C_* C ∗ Lemma 6.11 for our choice of C 0 C_0 C 0 β \beta β k k k C β , k C_{\beta,k} C β , k η 3 λ ( a 0 / 2 ) k − 2 ≥ 2 k − 1 β 2 k 2 / c ∗ \eta^3\lambda(a_0/2)^{k-2}\ge2^{k-1}\beta^2k^2/c_* η 3 λ ( a 0 /2 ) k − 2 ≥ 2 k − 1 β 2 k 2 / c ∗ c 3 = 2 k − 1 β 2 k 2 / η c_3=2^{k-1}\beta^2k^2/\eta c 3 = 2 k − 1 β 2 k 2 / η (130) and (131) , we conclude that the following holds on E f a i l \mathcal{E}_{\mathrm{fail}} E fail
β 2 ∫ 0 τ ∥ ∇ S N H s i g ( Y t ) ∥ 2 d t ≤ c ∗ N sup t ∈ [ 0 , T λ ] ∣ ( L n u l l m ϕ ) ( Y t ) ∣ 2 − c 3 N λ ( a 0 2 ) k . (132) \beta^2\int_0^\tau\left\|\nabla\mathcal{S}_N H_{\mathrm{sig}}(Y_t)\right\|^2\,\mathrm{d}t\le c_*N\sup_{t\in[0,T_\lambda]}\left|(\mathcal{L}_{\mathrm{null}}m_\phi)(Y_t)\right|^2-c_3N\lambda\left(\frac{a_0}{2}\right)^k. \tag*{(132)} β 2 ∫ 0 τ ∥ ∇ S N H sig ( Y t ) ∥ 2 d t ≤ c ∗ N t ∈ [ 0 , T λ ] sup ∣ ( L null m ϕ ) ( Y t ) ∣ 2 − c 3 N λ ( 2 a 0 ) k . (132) We now bound P ( E f a i l ) \mathbb{P}(\mathcal{E}_{\mathrm{fail}}) P ( E fail ) L n u l l m ϕ \mathcal{L}_{\mathrm{null}}m_\phi L null m ϕ E f a i l ∈ F τ \mathcal{E}_{\mathrm{fail}}\in\mathcal{F}_\tau E fail ∈ F τ τ ≤ T λ \tau\le T_\lambda τ ≤ T λ Theorem 6.12 with Φ e \Phi_e Φ e e e e T = T λ T=T_\lambda T = T λ E = E f a i l E=\mathcal{E}_{\mathrm{fail}} E = E fail (132) that
P ( E f a i l ) ≤ exp ( N λ 2 σ N 2 2 κ N 2 ) E Q [ exp ( β 2 ∫ 0 τ ∥ ∇ S N H s i g ( Y t ) ∥ 2 d t ) 1 { E f a i l } ] 1 / 2 ≤ exp ( N λ 2 σ N 2 2 κ N 2 − c 3 2 N λ ( a 0 2 ) k ) E Q [ exp ( c ∗ N sup t ∈ [ 0 , T λ ] ∣ ( L n u l l m ϕ ) ( Y t ) ∣ 2 ) 1 { E r e g ∩ A } ] 1 / 2 ≤ C ∗ 1 / 2 exp ( N λ 2 σ N 2 2 κ N 2 + C ∗ 2 N δ 2 − c 3 2 N λ ( a 0 2 ) k ) , \begin{aligned}
\mathbb{P}(\mathcal{E}_{\mathrm{fail}})\le\exp\left(\frac{N\lambda^2\sigma_N^2}{2\kappa_N^2}\right)\mathbb{E}_{\mathbb{Q}}\left[\exp\left(\beta^2\int_0^\tau\left\|\nabla\mathcal{S}_N H_{\mathrm{sig}}(Y_t)\right\|^2\,\mathrm{d}t\right)\mathbf{1}_{\{\mathcal{E}_{\mathrm{fail}}\}}\right]^{1/2} \\
&\le\exp\left(\frac{N\lambda^2\sigma_N^2}{2\kappa_N^2}-\frac{c_3}{2}N\lambda\left(\frac{a_0}{2}\right)^k\right)\mathbb{E}_{\mathbb{Q}}\left[\exp\left(c_*N\sup_{t\in[0,T_\lambda]}\left|(\mathcal{L}_{\mathrm{null}}m_\phi)(Y_t)\right|^2\right)\mathbf{1}_{\{\mathcal{E}_{\mathrm{reg}}\cap\mathcal{A}\}}\right]^{1/2} \\
&\le C_*^{1/2}\exp\left(\frac{N\lambda^2\sigma_N^2}{2\kappa_N^2}+\frac{C_*}{2}N\delta^2-\frac{c_3}{2}N\lambda\left(\frac{a_0}{2}\right)^k\right),
\end{aligned} P ( E fail ) ≤ exp ( 2 κ N 2 N λ 2 σ N 2 ) E Q [ exp ( β 2 ∫ 0 τ ∥ ∇ S N H sig ( Y t ) ∥ 2 d t ) 1 { E fail } ] 1/2 ≤ exp ( 2 κ N 2 N λ 2 σ N 2 − 2 c 3 N λ ( 2 a 0 ) k ) E Q [ exp ( c ∗ N t ∈ [ 0 , T λ ] sup ∣ ( L null m ϕ ) ( Y t ) ∣ 2 ) 1 { E reg ∩ A } ] 1/2 ≤ C ∗ 1/2 exp ( 2 κ N 2 N λ 2 σ N 2 + 2 C ∗ N δ 2 − 2 c 3 N λ ( 2 a 0 ) k ) , where the second inequality uses E f a i l ⊆ E r e g ∩ A \mathcal{E}_{\mathrm{fail}}\subseteq\mathcal{E}_{\mathrm{reg}}\cap\mathcal{A} E fail ⊆ E reg ∩ A Lemma 6.11 with Φ e \Phi_e Φ e e e e T = T λ ≤ 1 T=T_\lambda\le1 T = T λ ≤ 1 C β , k C_{\beta,k} C β , k c 3 c 0 2 λ ( a 0 / 2 ) k − 2 ≥ 32 c_3c_0^2\lambda(a_0/2)^{k-2}\ge32 c 3 c 0 2 λ ( a 0 /2 ) k − 2 ≥ 32
c 3 2 N λ ( a 0 2 ) k = c 3 c 0 2 8 λ ( a 0 2 ) k − 2 N λ 2 σ N 2 ≥ 4 N λ 2 σ N 2 . \frac{c_3}{2}N\lambda\left(\frac{a_0}{2}\right)^k=\frac{c_3c_0^2}{8}\lambda\left(\frac{a_0}{2}\right)^{k-2}N\lambda^2\sigma_N^2\ge4N\lambda^2\sigma_N^2. 2 c 3 N λ ( 2 a 0 ) k = 8 c 3 c 0 2 λ ( 2 a 0 ) k − 2 N λ 2 σ N 2 ≥ 4 N λ 2 σ N 2 . Since κ N ≥ 2 / 3 \kappa_N\ge2/3 κ N ≥ 2/3 N δ 2 = N − 2 ϑ N λ 2 σ N 2 N\delta^2=N^{-2\vartheta}N\lambda^2\sigma_N^2 N δ 2 = N − 2 ϑ N λ 2 σ N 2 N λ 2 σ N 2 ≳ N N\lambda^2\sigma_N^2\gtrsim\sqrt{N} N λ 2 σ N 2 ≳ N
P ( E f a i l ) ≤ C ∗ 1 / 2 exp ( − 2 N λ 2 σ N 2 ) ≤ exp ( − N λ 2 σ N 2 ) . (133) \mathbb{P}(\mathcal{E}_{\mathrm{fail}})\le C_*^{1/2}\exp(-2N\lambda^2\sigma_N^2)\le\exp(-N\lambda^2\sigma_N^2). \tag*{(133)} P ( E fail ) ≤ C ∗ 1/2 exp ( − 2 N λ 2 σ N 2 ) ≤ exp ( − N λ 2 σ N 2 ) . (133) We are now ready to show that the dynamics achieves critical recovery before time T λ T_\lambda T λ
E g o o d ≔ A ∩ I ∩ E r e g ∩ M T λ ∩ ( { τ < τ b a d } ∪ { τ = T ∗ Y } ) . \mathcal{E}_{\mathrm{good}} \coloneqq\mathcal{A} \cap\mathcal{I} \cap\mathcal{E}_{\mathrm{reg}} \cap\mathcal{M}_{T_\lambda} \cap\left(\{\tau< \tau^{\mathrm{bad}}\} \cup\{\tau= T_*^Y\}\right). E good : = A ∩ I ∩ E reg ∩ M T λ ∩ ( { τ < τ bad } ∪ { τ = T ∗ Y } ) . Observe that
{ τ = τ b a d < T ∗ Y } c = { τ < τ b a d } ∪ { τ = T ∗ Y } . \{\tau= \tau^{\mathrm{bad}} < T_*^Y\}^{c} = \{\tau< \tau^{\mathrm{bad}}\} \cup\{\tau= T_*^Y\}. { τ = τ bad < T ∗ Y } c = { τ < τ bad } ∪ { τ = T ∗ Y } . Moreover, since τ ≤ T λ \tau\le T_\lambda τ ≤ T λ M T λ ⊆ M τ \mathcal{M}_{T_\lambda} \subseteq\mathcal{M}_\tau M T λ ⊆ M τ E g o o d = A ∩ I ∩ E r e g ∩ M T λ ∩ E f a i l c \mathcal{E}_{\mathrm{good}} = \mathcal{A} \cap\mathcal{I} \cap\mathcal{E}_{\mathrm{reg}} \cap\mathcal{M}_{T_\lambda} \cap\mathcal{E}_{\mathrm{fail}}^{c} E good = A ∩ I ∩ E reg ∩ M T λ ∩ E fail c (120) to (122) and (133) that
P ( A ∖ E g o o d ) ≤ P ( A ∩ I c ) + P ( E r e g c ) + P ( M T λ c ) + P ( E f a i l ) ≤ 5 exp ( − c N λ 2 σ N 2 ) . \mathbb{P}(\mathcal{A} \setminus\mathcal{E}_{\mathrm{good}}) \le\mathbb{P}(\mathcal{A} \cap\mathcal{I}^{c}) + \mathbb{P}(\mathcal{E}_{\mathrm{reg}}^{c}) + \mathbb{P}(\mathcal{M}_{T_\lambda}^{c}) + \mathbb{P}(\mathcal{E}_{\mathrm{fail}}) \le5\exp(-cN\lambda^2\sigma_N^2). P ( A ∖ E good ) ≤ P ( A ∩ I c ) + P ( E reg c ) + P ( M T λ c ) + P ( E fail ) ≤ 5 exp ( − c N λ 2 σ N 2 ) . Henceforth, we work on E g o o d \mathcal{E}_{\mathrm{good}} E good M T λ ⊆ M τ \mathcal{M}_{T_\lambda} \subseteq\mathcal{M}_\tau M T λ ⊆ M τ (125) and (126) hold. We first turn Equation (126) into an explicit lower bound on the correlation. To this end, define the blow-up time
t ∞ ≔ ( a 0 / 2 ) − ( k − 2 ) ( k − 2 ) η λ = 2 k − 2 ( k − 2 ) η λ a 0 k − 2 . t_\infty\coloneqq\frac{(a_0/2)^{-(k-2)}}{(k-2)\eta\lambda} = \frac{2^{k-2}}{(k-2)\eta\lambda a_0^{k-2}}. t ∞ : = ( k − 2 ) η λ ( a 0 /2 ) − ( k − 2 ) = ( k − 2 ) η λ a 0 k − 2 2 k − 2 . By our choice of C T C_T C T t ∞ = T λ / 2 < T λ t_\infty= T_\lambda/2 < T_\lambda t ∞ = T λ /2 < T λ 31 a 0 / 32 ≥ a 0 / 2 31a_0/32 \ge a_0/2 31 a 0 /32 ≥ a 0 /2 t ↦ m t t \mapsto m_t t ↦ m t Lemma 6.15 to f ( t ) = m t f(t) = m_t f ( t ) = m t [ 0 , τ ] [0,\tau] [ 0 , τ ] a = a 0 / 2 a = a_0/2 a = a 0 /2 c = η λ c = \eta\lambda c = η λ (126) that
m t ≥ [ ( a 0 2 ) − ( k − 2 ) − ( k − 2 ) η λ t ] − 1 k − 2 ∀ t ∈ [ 0 , τ ∧ t ∞ ) . (134) m_t \ge\left[\left(\frac{a_0}{2}\right)^{-(k-2)} - (k-2)\eta\lambda t\right]^{-\frac{1}{k-2}} \qquad\forall t \in[0,\tau\wedge t_\infty). \tag*{(134)} m t ≥ [ ( 2 a 0 ) − ( k − 2 ) − ( k − 2 ) η λ t ] − k − 2 1 ∀ t ∈ [ 0 , τ ∧ t ∞ ) . (134) We now examine which of the stopping times attains the minimum in the definition of τ \tau τ (125) , we have m τ ≥ 31 a 0 / 32 > a 0 / 2 m_\tau\ge31a_0/32 > a_0/2 m τ ≥ 31 a 0 /32 > a 0 /2 τ ≠ τ ↓ \tau\ne\tau^\downarrow τ = τ ↓
Case 1: τ = T ∗ Y \tau= T_*^Y τ = T ∗ Y T ∗ Y = τ ≤ T λ T_*^Y = \tau\le T_\lambda T ∗ Y = τ ≤ T λ
Case 2: τ < T ∗ Y \tau< T_*^Y τ < T ∗ Y E g o o d \mathcal{E}_{\mathrm{good}} E good τ < τ b a d \tau< \tau^{\mathrm{bad}} τ < τ bad τ = T λ \tau= T_\lambda τ = T λ T ∗ Y > 0 T_*^Y > 0 T ∗ Y > 0 a 0 ≤ m 0 < C r N a_0 \le m_0 < Cr_N a 0 ≤ m 0 < C r N
t ∗ ≔ ( a 0 / 2 ) − ( k − 2 ) − ( 2 C r N ) − ( k − 2 ) ( k − 2 ) η λ . t_* \coloneqq\frac{(a_0/2)^{-(k-2)} - (2Cr_N)^{-(k-2)}}{(k-2)\eta\lambda}. t ∗ : = ( k − 2 ) η λ ( a 0 /2 ) − ( k − 2 ) − ( 2 C r N ) − ( k − 2 ) . Since a 0 / 2 < 2 C r N a_0/2 < 2Cr_N a 0 /2 < 2 C r N 0 < t ∗ < t ∞ < T λ = τ 0 < t_* < t_\infty< T_\lambda= \tau 0 < t ∗ < t ∞ < T λ = τ (134) holds at t = t ∗ t=t_* t = t ∗
m t ∗ ≥ [ ( a 0 2 ) − ( k − 2 ) − ( k − 2 ) η λ t ∗ ] − 1 k − 2 = 2 C r N . m_{t_*} \ge\left[\left(\frac{a_0}{2}\right)^{-(k-2)} - (k-2)\eta\lambda t_*\right]^{-\frac{1}{k-2}} = 2Cr_N. m t ∗ ≥ [ ( 2 a 0 ) − ( k − 2 ) − ( k − 2 ) η λ t ∗ ] − k − 2 1 = 2 C r N . Consequently, T ∗ Y ≤ t ∗ < τ T_*^Y \le t_* < \tau T ∗ Y ≤ t ∗ < τ τ < T ∗ Y \tau< T_*^Y τ < T ∗ Y
Combining the two cases, we have T ∗ Y ≤ T λ T_*^Y \le T_\lambda T ∗ Y ≤ T λ E g o o d \mathcal{E}_{\mathrm{good}} E good (115) . Since m t = m ( X t ( e ) ) m_t = m(X_t(e)) m t = m ( X t ( e )) T ∗ Y = T ∗ ( e ) T_*^Y = T_*(e) T ∗ Y = T ∗ ( e ) A = { ∥ e − u ^ H ∥ ≤ δ N } \mathcal{A} = \{\lVert e-\widehat{u}_H\rVert\le\delta\sqrt{N}\} A = {∥ e − u H ∥ ≤ δ N } δ = λ σ N N − ϑ \delta= \lambda\sigma_N N^{-\vartheta} δ = λ σ N N − ϑ
P [ 1 N ∥ e − u ^ H ∥ ≤ λ σ N N − ϑ and T ∗ ( e ) > T λ ] ≤ P ( A ∖ E g o o d ) ≤ exp ( − c N λ 2 σ N 2 ) . \mathbb{P}\left[\frac{1}{\sqrt{N}}\lVert e-\widehat{u}_H\rVert\le\lambda\sigma_N N^{-\vartheta}\ \text{and}\ T_*(e)>T_\lambda\right] \le\mathbb{P}(\mathcal{A}\setminus\mathcal{E}_{\mathrm{good}}) \le\exp(-cN\lambda^2\sigma_N^2). P [ N 1 ∥ e − u H ∥ ≤ λ σ N N − ϑ and T ∗ ( e ) > T λ ] ≤ P ( A ∖ E good ) ≤ exp ( − c N λ 2 σ N 2 ) . The proof is concluded by noting that T λ = C T / ( λ a 0 k − 2 ) ≲ N − ( k − 1 ) ( γ − k − 2 4 ) T_\lambda= C_T/(\lambda a_0^{k-2}) \lesssim N^{-(k-1)(\gamma-\frac{k-2}{4})} T λ = C T / ( λ a 0 k − 2 ) ≲ N − ( k − 1 ) ( γ − 4 k − 2 ) β \beta β k k k
Proof of Theorem 6.1
Proof. Throughout the proof, we assume that N N N T s t r = C 1 + 2 T_{\mathrm{str}}=C_1+2 T str = C 1 + 2 C 1 C_1 C 1 Theorem 6.5 , which depends only on β \beta β k k k k k k ⌊ k / 2 ⌋ = k − 1 2 \lfloor k/2\rfloor=\frac{k-1}{2} ⌊ k /2 ⌋ = 2 k − 1 β 0 − 2 λ − 2 N k − 1 + 2 ϑ ≍ k N 2 k − 2 + 4 ϑ \beta_0^{-2}\lambda^{-2}N^{k-1+2\vartheta}\asymp_k N^{2k-2+4\vartheta} β 0 − 2 λ − 2 N k − 1 + 2 ϑ ≍ k N 2 k − 2 + 4 ϑ β 0 \beta_0 β 0 T a v T_{\mathrm{av}} T av Theorem 5.1 . By construction, ( X t ) t ≥ 0 (X_t)_{t\ge0} ( X t ) t ≥ 0 H H H β \beta β Theorem 5.1 , we have X ‾ T a v ≠ 0 \overline{X}_{T_{\mathrm{av}}}\ne0 X T av = 0 X 0 = X ^ T a v X_0=\widehat{X}_{T_{\mathrm{av}}} X 0 = X T av λ ≥ C β , k N k − 2 4 + 1 4 k \lambda\ge C_{\beta,k}N^{\frac{k-2}{4}+\frac{1}{4k}} λ ≥ C β , k N 4 k − 2 + 4 k 1 Corollary 6.4 , since T s t r ≥ 2 T_{\mathrm{str}}\ge2 T str ≥ 2 k + 1 2 k > 1 2 \frac{k+1}{2k}>\frac12 2 k k + 1 > 2 1
Henceforth, we assume that λ = C β , k N γ \lambda=C_{\beta,k}N^\gamma λ = C β , k N γ k − 2 4 ≤ γ < k − 2 4 + 1 4 k \frac{k-2}{4}\le\gamma<\frac{k-2}{4}+\frac{1}{4k} 4 k − 2 ≤ γ < 4 k − 2 + 4 k 1 r N = λ − 1 k − 1 r_N=\lambda^{-\frac{1}{k-1}} r N = λ − k − 1 1 Theorem 5.1 implies that the following holds with probability at least 1 − exp ( − c k N ) 1-\exp(-c_kN) 1 − exp ( − c k N )
1 N ∥ X 0 − u ^ H ∥ ≤ N − ϑ min { 1 , λ σ N } ≤ λ σ N N − ϑ . \frac{1}{\sqrt{N}}\lVert X_0-\widehat{u}_H\rVert\le N^{-\vartheta}\min\{1,\lambda\sigma_N\}\le\lambda\sigma_NN^{-\vartheta}. N 1 ∥ X 0 − u H ∥ ≤ N − ϑ min { 1 , λ σ N } ≤ λ σ N N − ϑ . Let r ‾ N = N γ k − 1 \overline{r}_N=N^{\frac{\gamma}{k-1}} r N = N k − 1 γ r N r_N r N Theorem 6.5 allows the initial state to depend on H H H e = X 0 e=X_0 e = X 0 C C C C r N = r ‾ N Cr_N=\overline{r}_N C r N = r N γ ≥ k − 2 4 \gamma\ge\frac{k-2}{4} γ ≥ 4 k − 2 N λ 2 σ N 2 ≳ k N N\lambda^2\sigma_N^2\gtrsim_k\sqrt{N} N λ 2 σ N 2 ≳ k N 1 − exp ( − c β , k N ) 1-\exp(-c_{\beta,k}\sqrt{N}) 1 − exp ( − c β , k N )
T ∗ ( X 0 ) = inf { t ≥ 0 : m ( X t ) ≥ r ‾ N } ≤ C 1 . (135) T_*(X_0)=\inf\{t\ge0:m(X_t)\ge\overline{r}_N\}\le C_1. \tag*{(135)} T ∗ ( X 0 ) = inf { t ≥ 0 : m ( X t ) ≥ r N } ≤ C 1 . (135) Moreover, since λ r ‾ N k − 1 = C β , k \lambda\overline{r}_N^{k-1}=C_{\beta,k} λ r N k − 1 = C β , k γ k − 1 < 1 4 \frac{\gamma}{k-1}<\frac14 k − 1 γ < 4 1 Theorem 6.2 with s = γ k − 1 s=\frac{\gamma}{k-1} s = k − 1 γ τ r ‾ N = T ∗ ( X 0 ) \tau_{\overline{r}_N}=T_*(X_0) τ r N = T ∗ ( X 0 ) T s t r ≥ T ∗ ( X 0 ) + 2 T_{\mathrm{str}}\ge T_*(X_0)+2 T str ≥ T ∗ ( X 0 ) + 2 (135) and N r ‾ N 2 ≥ N N\overline{r}_N^2\ge\sqrt{N} N r N 2 ≥ N
P [ inf t ∈ [ T s t r , T ] m ( X t ) < 1 − N − η ] ≤ exp ( − c β , k N ) + exp ( − c k N min { 1 / 2 , 1 − 2 η } ) , \mathbb{P}\left[\inf_{t\in[T_{\mathrm{str}},T]}m(X_t)<1-N^{-\eta}\right]\le\exp(-c_{\beta,k}\sqrt{N})+\exp\left(-c_kN^{\min\{1/2,1-2\eta\}}\right), P [ t ∈ [ T str , T ] inf m ( X t ) < 1 − N − η ] ≤ exp ( − c β , k N ) + exp ( − c k N m i n { 1/2 , 1 − 2 η } ) , which proves the claim after adjusting constants appropriately. □ \square □
Strong Recovery at the Algorithmic Threshold for Every Tensor Order
In this section, we analyze Algorithm 2 and show that it achieves persistent strong recovery above the algorithmic threshold for any k ≥ 3 k\ge3 k ≥ 3 Theorem 1.2 . The complete proof is presented in § 7.3 .
Theorem 7.1 (Persistent Strong Recovery above the Algorithmic Threshold for Every Tensor Order). *Let k ≥ 3 k\ge3 k ≥ 3 β > 0 \beta>0 β > 0 λ ≥ C β , k N k − 2 4 \lambda\ge C_{\beta,k}N^{\frac{k-2}{4}} λ ≥ C β , k N 4 k − 2 C β , k C_{\beta,k} C β , k ϑ ∈ ( 0 , 1 ) \vartheta\in(0,1) ϑ ∈ ( 0 , 1 ) η ∈ ( 0 , min { 1 2 , k − 2 4 } ) \eta\in(0,\min\{\frac12,\frac{k-2}{4}\}) η ∈ ( 0 , min { 2 1 , 4 k − 2 }) ( X t ) t ≥ 0 (X_t)_{t\ge0} ( X t ) t ≥ 0 X ~ 0 ∼ π \widetilde{X}_0\sim\pi X 0 ∼ π β 0 \beta_0 β 0 β \beta β β 0 ≍ k λ − 1 N − k / 2 − ϑ \beta_0\asymp_k\lambda^{-1}N^{-k/2-\vartheta} β 0 ≍ k λ − 1 N − k /2 − ϑ T a v ≍ k N 2 k + 4 ϑ T_{\mathrm{av}}\asymp_kN^{2k+4\vartheta} T av ≍ k N 2 k + 4 ϑ T s t r > 0 T_{\mathrm{str}}>0 T str > 0 β \beta β k k k N N N T T T T s t r ≤ T ≤ poly ( N ) T_{\mathrm{str}}\le T\le\operatorname{poly}(N) T str ≤ T ≤ poly ( N )
P [ inf t ∈ [ T s t r , T ] ∣ m ( X t ) ∣ ≥ 1 − N − η ] ≥ 1 − exp ( − c β , k N min { 1 / 2 , 1 − 2 η } ) . \mathbb{P}\left[\inf_{t\in[T_{\mathrm{str}},T]}|m(X_t)|\ge1-N^{-\eta}\right]\ge1-\exp\left(-c_{\beta,k}N^{\min\{1/2,1-2\eta\}}\right). P [ t ∈ [ T str , T ] inf ∣ m ( X t ) ∣ ≥ 1 − N − η ] ≥ 1 − exp ( − c β , k N m i n { 1/2 , 1 − 2 η } ) . Moreover, if k k k m ( X t ) m(X_t) m ( X t ) ∣ m ( X t ) ∣ |m(X_t)| ∣ m ( X t ) ∣
Properties of the Augmented Ergodic Second Moment
In this section, we analyze the properties of the augmented ergodic second moment of the first phase of Algorithm 2, defined by M ˉ T ≔ 1 T ∫ 0 T F ( X ~ t ) ⊗ 2 d t \bar{M}_T \coloneqq\frac{1}{T}\int_0^T F(\widetilde{X}_t)^{\otimes2}\,\mathrm{d}t M ˉ T : = T 1 ∫ 0 T F ( X t ) ⊗ 2 d t F ( x ) = ( x , 1 ) ∈ R N + 1 F(x)=(x,1)\in\mathbb{R}^{N+1} F ( x ) = ( x , 1 ) ∈ R N + 1 ( X ~ t ) t ≥ 0 (\widetilde{X}_t)_{t\geq0} ( X t ) t ≥ 0 § 4 that the average spherical gradient and the average spherical Hessian of H H H u H = E π [ ∇ S N H ] u_H=\mathbb{E}_{\pi}[\nabla_{\mathbb{S}_N}H] u H = E π [ ∇ S N H ] A H = E π [ ∇ S N 2 H ] A_H=\mathbb{E}_{\pi}[\nabla_{\mathbb{S}_N}^2H] A H = E π [ ∇ S N 2 H ] F ( X ~ t ) ⊗ 2 F(\widetilde{X}_t)^{\otimes2} F ( X t ) ⊗ 2
M ˉ T = ( Σ ˉ T X ~ T X ~ T ⊤ 1 ) . \bar{M}_T=\begin{pmatrix}\bar{\Sigma}_T & \widetilde{X}_T\\ \widetilde{X}_T^{\top} & 1\end{pmatrix}. M ˉ T = ( Σ ˉ T X T ⊤ X T 1 ) . where X ~ T ≔ 1 T ∫ 0 T X ~ t d t \widetilde{X}_T\coloneqq\frac{1}{T}\int_0^T\widetilde{X}_t\,\mathrm{d}t X T : = T 1 ∫ 0 T X t d t Σ ˉ T ≔ 1 T ∫ 0 T X ~ t X ~ t ⊤ d t \bar{\Sigma}_T\coloneqq\frac{1}{T}\int_0^T\widetilde{X}_t\widetilde{X}_t^{\top}\,\mathrm{d}t Σ ˉ T : = T 1 ∫ 0 T X t X t ⊤ d t M ˉ T , a v \bar{M}_{T,\mathrm{av}} M ˉ T , av I N + 1 + 2 β 0 A H I_{N+1}+2\beta_0\mathcal{A}_H I N + 1 + 2 β 0 A H
A H = ( A H κ N − 1 u H κ N − 1 u H ⊤ 0 ) . \mathcal{A}_H=\begin{pmatrix}A_H & \kappa_N^{-1}u_H\\ \kappa_N^{-1}u_H^{\top} & 0\end{pmatrix}. A H = ( A H κ N − 1 u H ⊤ κ N − 1 u H 0 ) . Here, the factor κ N − 1 \kappa_N^{-1} κ N − 1 A H \mathcal{A}_H A H 2 β 0 κ N \frac{2\beta_0}{\kappa_N} κ N 2 β 0 2 β 0 2\beta_0 2 β 0
As discussed in § 2.5 , the parity of k k k A H \mathcal{A}_H A H k k k A H = 0 A_H=0 A H = 0 u H u_H u H x ⋆ x^\star x ⋆ A H \mathcal{A}_H A H 1 2 ( u H / ∥ u H ∥ , 1 ) \frac{1}{\sqrt{2}}(u_H/\lVert u_H\rVert,1) 2 1 ( u H / ∥ u H ∥ , 1 ) N N N S N \mathbb{S}_N S N u ^ H = N u H ∥ u H ∥ \widehat{u}_H=\frac{\sqrt{N}u_H}{\lVert u_H\rVert} u H = ∥ u H ∥ N u H u ^ H \widehat{u}_H u H
When k k k u H = 0 u_H=0 u H = 0 A H \mathcal{A}_H A H A H A_H A H A H A_H A H θ ⋆ \theta^\star θ ⋆ θ ⋆ \theta^\star θ ⋆
We note that Algorithm 2 does not need to know the parity of k k k A H \mathcal{A}_H A H
Theorem 7.2 (Recovery by the Augmented Ergodic Second Moment for Tensor PCA). Let λ ≥ C k N k − 2 4 \lambda\geq C_kN^{\frac{k-2}{4}} λ ≥ C k N 4 k − 2 C k C_k C k ( X ~ t ) t ≥ 0 (\widetilde{X}_t)_{t\geq0} ( X t ) t ≥ 0 H H H β 0 > 0 \beta_0>0 β 0 > 0 X ~ 0 ∼ π \widetilde{X}_0\sim\pi X 0 ∼ π
M ˉ T ≔ 1 T ∫ 0 T F ( X ~ t ) ⊗ 2 d t , F ( x ) = ( x , 1 ) ∈ R N + 1 . \bar{M}_T\coloneqq\frac{1}{T}\int_0^T F(\widetilde{X}_t)^{\otimes2}\,\mathrm{d}t,\qquad F(x)=(x,1)\in\mathbb{R}^{N+1}. M ˉ T : = T 1 ∫ 0 T F ( X t ) ⊗ 2 d t , F ( x ) = ( x , 1 ) ∈ R N + 1 . Define u H = E π [ ∇ S N H ] u_H=\mathbb{E}_{\pi}[\nabla_{\mathbb{S}_N}H] u H = E π [ ∇ S N H ] ( v , a ) ∈ R N × R (v,a)\in\mathbb{R}^N\times\mathbb{R} ( v , a ) ∈ R N × R M ˉ T , a v \bar{M}_{T,\mathrm{av}} M ˉ T , av a ≥ 0 a\geq0 a ≥ 0 ϑ ∈ ( 0 , 1 ) \vartheta\in(0,1) ϑ ∈ ( 0 , 1 ) N N N 0 < β 0 ≲ k λ − 1 N − k / 2 − ϑ 0<\beta_0\lesssim_k\lambda^{-1}N^{-k/2-\vartheta} 0 < β 0 ≲ k λ − 1 N − k /2 − ϑ T a v ≳ k β 0 − 2 λ − 2 N k + 2 ϑ T_{\mathrm{av}}\gtrsim_k\beta_0^{-2}\lambda^{-2}N^{k+2\vartheta} T av ≳ k β 0 − 2 λ − 2 N k + 2 ϑ
If k k k 1 − exp ( − c k N ) 1-\exp(-c_kN) 1 − exp ( − c k N ) v ≠ 0 v \ne0 v = 0
1 N ∥ N v ∥ v ∥ − u ^ H ∥ 2 ≤ N − 2 ϑ min { 1 , λ 2 σ N 2 } . \frac{1}{N}\left\lVert\frac{\sqrt{N}v}{\lVert v\rVert}-\widehat{u}_H\right\rVert^2 \le N^{-2\vartheta}\min\{1,\lambda^2\sigma_N^2\}. N 1 ∥ v ∥ N v − u H 2 ≤ N − 2 ϑ min { 1 , λ 2 σ N 2 } . where u ^ H = N u H ∥ u H ∥ \widehat{u}_H=\frac{\sqrt{N}u_H}{\lVert u_H\rVert} u H = ∥ u H ∥ N u H σ N ≍ k N − k − 1 4 \sigma_N\asymp_k N^{-\frac{k-1}{4}} σ N ≍ k N − 4 k − 1 Theorem 4.1 .
If k k k 1 − exp ( − c k N ) 1-\exp(-c_kN) 1 − exp ( − c k N ) v ≠ 0 v \ne0 v = 0
∣ m ( N v ∥ v ∥ ) ∣ ≥ 1 − C k ( N k − 2 2 λ 2 + N − 2 ϑ ) . \left|m\left(\frac{\sqrt{N}v}{\lVert v\rVert}\right)\right|\ge1-C_k\left(\frac{N^{\frac{k-2}{2}}}{\lambda^2}+N^{-2\vartheta}\right). m ( ∥ v ∥ N v ) ≥ 1 − C k ( λ 2 N 2 k − 2 + N − 2 ϑ ) . Proof. Throughout the proof, we assume that N N N A ^ ≔ ( 2 β 0 ) − 1 ( M ‾ T a v − I N + 1 ) \widehat{\mathcal A}\coloneqq(2\beta_0)^{-1}(\overline{M}_{T_{\mathrm{av}}}-I_{N+1}) A : = ( 2 β 0 ) − 1 ( M T av − I N + 1 ) A H \mathcal A_H A H β 0 > 0 \beta_0>0 β 0 > 0 ( v , a ) (v,a) ( v , a ) A ^ \widehat{\mathcal A} A Theorem 5.1 , we have
M ‾ T a v = ( Σ ‾ T a v X ‾ T a v X ‾ T a v T 1 ) . \overline{M}_{T_{\mathrm{av}}}=
\begin{pmatrix}
\overline{\Sigma}_{T_{\mathrm{av}}} & \overline{X}_{T_{\mathrm{av}}}\\
\overline{X}_{T_{\mathrm{av}}}^{\mathsf T} & 1
\end{pmatrix}. M T av = ( Σ T av X T av T X T av 1 ) . Since the bottom-right entries of M ‾ T a v − I N + 1 \overline{M}_{T_{\mathrm{av}}}-I_{N+1} M T av − I N + 1 A H \mathcal A_H A H
A ^ − A H = ( Σ ‾ T a v − I 2 β 0 − A H 0 0 0 ) + 1 κ N ( 0 κ N 2 β 0 X ‾ T a v − u H ( κ N 2 β 0 X ‾ T a v − u H ) T 0 ) . \widehat{\mathcal A}-\mathcal A_H=
\begin{pmatrix}
\frac{\overline{\Sigma}_{T_{\mathrm{av}}}-I}{2\beta_0}-\mathcal A_H & 0\\
0 & 0
\end{pmatrix}
+\frac{1}{\kappa_N}
\begin{pmatrix}
0 & \frac{\kappa_N}{2\beta_0}\overline{X}_{T_{\mathrm{av}}}-u_H\\
\left(\frac{\kappa_N}{2\beta_0}\overline{X}_{T_{\mathrm{av}}}-u_H\right)^{\mathsf T} & 0
\end{pmatrix}. A − A H = ( 2 β 0 Σ T av − I − A H 0 0 0 ) + κ N 1 0 ( 2 β 0 κ N X T av − u H ) T 2 β 0 κ N X T av − u H 0 . The first matrix on the right-hand side is block diagonal, and hence its operator norm equals that of its top-left block. The second matrix has zero diagonal blocks, and its operator norm equals the Euclidean norm of its top-right block. Consequently, the triangle inequality implies
∥ A ^ − A H ∥ o p ≤ ∥ Σ ‾ T a v − I 2 β 0 − A H ∥ o p + 1 κ N ∥ κ N 2 β 0 X ‾ T a v − u H ∥ . (136) \left\lVert\widehat{\mathcal A}-\mathcal A_H\right\rVert_{\mathrm{op}}
\le
\left\lVert\frac{\overline{\Sigma}_{T_{\mathrm{av}}}-I}{2\beta_0}-\mathcal A_H\right\rVert_{\mathrm{op}}
+\frac{1}{\kappa_N}\left\lVert\frac{\kappa_N}{2\beta_0}\overline{X}_{T_{\mathrm{av}}}-u_H\right\rVert. \tag*{(136)} A − A H op ≤ 2 β 0 Σ T av − I − A H op + κ N 1 2 β 0 κ N X T av − u H . (136) By Equation (53) in the proof of Theorem 5.1 , the following event holds with probability at least 1 − exp ( − c k N ) 1-\exp(-c_kN) 1 − exp ( − c k N )
E ≔ { ∥ κ N 2 β 0 X ‾ T a v − u H ∥ ≤ ε N } ∩ { ∥ Σ ‾ T a v − I 2 β 0 − A H ∥ o p ≤ N ε N } , \mathcal E\coloneqq
\left\{\left\lVert\frac{\kappa_N}{2\beta_0}\overline{X}_{T_{\mathrm{av}}}-u_H\right\rVert\le\varepsilon_N\right\}
\cap
\left\{\left\lVert\frac{\overline{\Sigma}_{T_{\mathrm{av}}}-I}{2\beta_0}-\mathcal A_H\right\rVert_{\mathrm{op}}\le\sqrt{N}\varepsilon_N\right\}, E : = { 2 β 0 κ N X T av − u H ≤ ε N } ∩ { 2 β 0 Σ T av − I − A H op ≤ N ε N } , ε N = C k [ ( β 0 − 1 + λ ) N T a v + β 0 λ 2 N ] . \varepsilon_N=C_k\left[(\beta_0^{-1}+\lambda)\sqrt{\frac{N}{T_{\mathrm{av}}}}+\beta_0\lambda^2\sqrt{N}\right]. ε N = C k [ ( β 0 − 1 + λ ) T av N + β 0 λ 2 N ] . Since our assumptions on β 0 \beta_0 β 0 T a v T_{\mathrm{av}} T av Theorem 5.1 with k / 2 k/2 k /2 ⌊ k / 2 ⌋ \lfloor k/2\rfloor ⌊ k /2 ⌋ (54) also shows that N ε N ≤ c k λ N − k − 2 2 − ϑ \sqrt{N}\varepsilon_N\le c_k\lambda N^{-\frac{k-2}{2}-\vartheta} N ε N ≤ c k λ N − 2 k − 2 − ϑ c k > 0 c_k>0 c k > 0 E \mathcal E E (136) and the definition of E \mathcal E E κ N − 1 ≤ 3 2 ≤ N \kappa_N^{-1}\le\frac{3}{2}\le\sqrt{N} κ N − 1 ≤ 2 3 ≤ N
∥ A ^ − A H ∥ o p ≤ ( N + 1 κ N ) ε N ≤ 2 c k λ N − k − 2 2 − ϑ . (137) \left\lVert\widehat{\mathcal A}-\mathcal A_H\right\rVert_{\mathrm{op}}
\le\left(\sqrt{N}+\frac{1}{\kappa_N}\right)\varepsilon_N
\le2c_k\lambda N^{-\frac{k-2}{2}-\vartheta}. \tag*{(137)} A − A H op ≤ ( N + κ N 1 ) ε N ≤ 2 c k λ N − 2 k − 2 − ϑ . (137) We first consider odd k k k Theorems 4.1 and 4.4 , A H = 0 A_H=0 A H = 0 u H = N λ α N θ ⋆ + ζ u_H=N\lambda\alpha_N\theta^\star+\zeta u H = N λ α N θ ⋆ + ζ ζ ∼ N ( 0 , σ N 2 I ) \zeta\sim\mathcal{N}(0,\sigma_N^2 I) ζ ∼ N ( 0 , σ N 2 I ) α N ≍ k N − k / 2 \alpha_N\asymp_k N^{-k/2} α N ≍ k N − k /2 σ N 2 = κ N N α N ≍ k N k − 1 2 \sigma_N^2=\kappa_N\sqrt{N}\alpha_N\asymp_k N^{\frac{k-1}{2}} σ N 2 = κ N N α N ≍ k N 2 k − 1 ∥ u H ∥ \|u_H\| ∥ u H ∥
P [ ∥ u H ∥ ≥ c ( N λ α N + σ N N ) ] ≥ 1 − C exp ( − c N ) . (138) \mathbb{P}\left[\|u_H\|\ge c(N\lambda\alpha_N+\sigma_N\sqrt{N})\right]\ge1-C\exp(-cN). \tag*{(138)} P [ ∥ u H ∥ ≥ c ( N λ α N + σ N N ) ] ≥ 1 − C exp ( − c N ) . (138) Henceforth, we also work on the event in Equation (138) , on which u H ≠ 0 u_H\ne0 u H = 0
We now show that v v v u H u_H u H A H = 0 A_H=0 A H = 0 A ^ H = κ N − 1 ( 0 u H u H T 0 ) \widehat{\mathcal A}_H=\kappa_N^{-1}\begin{pmatrix} 0 & u_H \\ u_H^{\mathsf T} & 0 \end{pmatrix} A H = κ N − 1 ( 0 u H T u H 0 ) w = 1 2 ( u H / ∥ u H ∥ , 1 ) w=\frac{1}{\sqrt{2}}(u_H/\|u_H\|,1) w = 2 1 ( u H /∥ u H ∥ , 1 ) κ N − 1 ∥ u H ∥ \kappa_N^{-1}\|u_H\| κ N − 1 ∥ u H ∥ 0 0 0 − κ N − 1 ∥ u H ∥ -\kappa_N^{-1}\|u_H\| − κ N − 1 ∥ u H ∥ [48 Theorem 4.5.5] to A ^ \widehat{\mathcal A} A A H \mathcal A_H A H
min s ∈ { ± 1 } ∥ ( v , a ) − s w ∥ ≲ κ N ∥ A ^ − A H ∥ o p ∥ u H ∥ ≲ c k λ N k − 2 2 − ϑ N λ α N + σ N N ≲ k c k N − ϑ min { 1 , λ σ N κ N } . (139) \min_{s\in\{\pm1\}}\|(v,a)-sw\|\lesssim\frac{\kappa_N\|\widehat{\mathcal A}-\mathcal A_H\|_{\mathrm{op}}}{\|u_H\|}\lesssim\frac{c_k\lambda N^{\frac{k-2}{2}-\vartheta}}{N\lambda\alpha_N+\sigma_N\sqrt{N}}\lesssim_k c_kN^{-\vartheta}\min\left\{1,\frac{\lambda\sigma_N}{\kappa_N}\right\}. \tag*{(139)} s ∈ { ± 1 } min ∥ ( v , a ) − s w ∥ ≲ ∥ u H ∥ κ N ∥ A − A H ∥ op ≲ N λ α N + σ N N c k λ N 2 k − 2 − ϑ ≲ k c k N − ϑ min { 1 , κ N λ σ N } . (139) where the implicit constants do not depend on c k c_k c k (137) and (138) and κ N ≤ 1 \kappa_N\le1 κ N ≤ 1 α N ≍ k N − k / 2 \alpha_N\asymp_k N^{-k/2} α N ≍ k N − k /2 σ N 2 = κ N N α N \sigma_N^2=\kappa_N\sqrt{N}\alpha_N σ N 2 = κ N N α N Theorem 4.1 .
Since the Davis–Kahan theorem determines ( v , a ) (v,a) ( v , a ) (139) is attained at s = 1 s=1 s = 1 a ≥ 0 a\ge0 a ≥ 0 w w w 1 / 2 1/\sqrt{2} 1/ 2 ( v , a ) + w (v,a)+w ( v , a ) + w 1 / 2 1/\sqrt{2} 1/ 2
∥ ( v , a ) + w ∥ ≥ a + 1 2 ≥ 1 2 . \|(v,a)+w\|\ge a+\frac{1}{\sqrt{2}}\ge\frac{1}{\sqrt{2}}. ∥ ( v , a ) + w ∥ ≥ a + 2 1 ≥ 2 1 . In contrast, the right-hand side of Equation (139) is smaller than 1 / 2 1/\sqrt{2} 1/ 2 N N N (139) is attained at s = 1 s=1 s = 1 N N N w w w u H 2 ∥ u H ∥ \frac{u_H}{\sqrt{2}\|u_H\|} 2 ∥ u H ∥ u H (139) that
∥ v − u H 2 ∥ u H ∥ ∥ ≤ ∥ ( v , a ) − w ∥ ≲ k c k N − ϑ min { 1 , λ σ N κ N } . (140) \left\|v-\frac{u_H}{\sqrt{2}\|u_H\|}\right\|\le\|(v,a)-w\|\lesssim_k c_kN^{-\vartheta}\min\left\{1,\frac{\lambda\sigma_N}{\kappa_N}\right\}. \tag*{(140)} v − 2 ∥ u H ∥ u H ≤ ∥ ( v , a ) − w ∥ ≲ k c k N − ϑ min { 1 , κ N λ σ N } . (140) We now compare the normalizations of v v v u H u_H u H ∥ u H 2 ∥ u H ∥ ∥ = 1 / 2 \left\|\frac{u_H}{\sqrt{2}\|u_H\|}\right\|=1/\sqrt{2} 2 ∥ u H ∥ u H = 1/ 2 (140) is smaller than 1 / 2 1/\sqrt{2} 1/ 2 N N N v ≠ 0 v\ne0 v = 0 u H 2 ∥ u H ∥ \frac{u_H}{\sqrt{2}\|u_H\|} 2 ∥ u H ∥ u H u H u_H u H u H ∥ u H ∥ = u ^ H N \frac{u_H}{\|u_H\|}=\frac{\widehat{u}_H}{\sqrt{N}} ∥ u H ∥ u H = N u H Lemma 3.4 to v v v u H 2 ∥ u H ∥ \frac{u_H}{\sqrt{2}\|u_H\|} 2 ∥ u H ∥ u H (140) :
1 N ∥ N v ∥ v ∥ − u ^ H ∥ = ∥ v ∥ v ∥ − u H ∥ u H ∥ ∥ ≤ 2 max { ∥ v ∥ , 1 / 2 } ∥ v − u H 2 ∥ u H ∥ ∥ \frac{1}{\sqrt{N}}\left\|\frac{\sqrt{N}v}{\|v\|}-\widehat{u}_H\right\|=\left\|\frac{v}{\|v\|}-\frac{u_H}{\|u_H\|}\right\|\le\frac{2}{\max\{\|v\|,1/\sqrt{2}\}}\left\|v-\frac{u_H}{\sqrt{2}\|u_H\|}\right\| N 1 ∥ v ∥ N v − u H = ∥ v ∥ v − ∥ u H ∥ u H ≤ max { ∥ v ∥ , 1/ 2 } 2 v − 2 ∥ u H ∥ u H ≤ 2 2 ∥ v − u H 2 ∥ u H ∥ ∥ ≲ k c k N − ϑ min { 1 , λ σ N κ N } . \le2\sqrt{2}\left\|v-\frac{u_H}{\sqrt{2}\|u_H\|}\right\|\lesssim_k c_kN^{-\vartheta}\min\left\{1,\frac{\lambda\sigma_N}{\kappa_N}\right\}. ≤ 2 2 v − 2 ∥ u H ∥ u H ≲ k c k N − ϑ min { 1 , κ N λ σ N } . Since κ N ≥ 2 3 \kappa_N\ge\frac{2}{3} κ N ≥ 3 2 c k c_k c k E \mathcal E E (138) proves the claim for odd k k k
We now consider even k k k Theorems 4.1 and 4.4 , we have u H = 0 u_H=0 u H = 0
A H = A s i g + W N , A s i g = λ γ N ( θ ⋆ ( θ ⋆ ) T − I N ) , γ N ≍ k N k − 2 2 . A_H=A_{\mathrm{sig}}+W_N,\qquad A_{\mathrm{sig}}=\lambda\gamma_N\left(\theta^\star(\theta^\star)^\mathsf{T}-\frac{I}{N}\right),\qquad\gamma_N\asymp_k N^{\frac{k-2}{2}}. A H = A sig + W N , A sig = λ γ N ( θ ⋆ ( θ ⋆ ) T − N I ) , γ N ≍ k N 2 k − 2 . where ∥ W N ∥ o p ≤ C k N k − 2 4 \lVert W_N\rVert_{\mathrm{op}}\le C_kN^{\frac{k-2}{4}} ∥ W N ∥ op ≤ C k N 4 k − 2 1 − exp ( − c N ) 1-\exp(-cN) 1 − exp ( − c N ) A H = ( A s i g + W N 0 0 0 ) \mathcal{A}_H=\left(\begin{smallmatrix}A_{\mathrm{sig}}+W_N&0\\0&0\end{smallmatrix}\right) A H = ( A sig + W N 0 0 0 ) k k k (137) :
∥ A ^ − ( A s i g 0 0 0 ) ∥ o p ≤ ∥ A ^ − A H ∥ o p + ∥ W N ∥ o p ≲ k λ N − k − 2 2 − ϑ + N k − 2 4 . \left\lVert\widehat{\mathcal{A}}-\begin{pmatrix}A_{\mathrm{sig}}&0\\0&0\end{pmatrix}\right\rVert_{\mathrm{op}}\le\left\lVert\widehat{\mathcal{A}}-\mathcal{A}_H\right\rVert_{\mathrm{op}}+\lVert W_N\rVert_{\mathrm{op}}\lesssim_k \lambda N^{-\frac{k-2}{2}-\vartheta}+N^{\frac{k-2}{4}}. A − ( A sig 0 0 0 ) op ≤ A − A H op + ∥ W N ∥ op ≲ k λ N − 2 k − 2 − ϑ + N 4 k − 2 . The matrix ( A s i g 0 0 0 ) \left(\begin{smallmatrix}A_{\mathrm{sig}}&0\\0&0\end{smallmatrix}\right) ( A sig 0 0 0 ) ( θ ⋆ , 0 ) (\theta^\star,0) ( θ ⋆ , 0 ) λ γ N ( 1 − 1 / N ) \lambda\gamma_N(1-1/N) λ γ N ( 1 − 1/ N ) 0 0 0 − λ γ N / N -\lambda\gamma_N/N − λ γ N / N A \mathcal{A} A
min s ∈ { ± 1 } ∥ v − s θ ⋆ ∥ ≤ min s ∈ { ± 1 } ∥ ( v , a ) − s ( θ ⋆ , 0 ) ∥ ≲ k λ N − k − 2 2 − ϑ + N k − 2 4 λ N − k − 2 2 = N − ϑ + N k − 2 4 λ . (141) \min_{s\in\{\pm1\}}\lVert v-s\theta^\star\rVert\le\min_{s\in\{\pm1\}}\lVert(v,a)-s(\theta^\star,0)\rVert\lesssim_k \frac{\lambda N^{-\frac{k-2}{2}-\vartheta}+N^{\frac{k-2}{4}}}{\lambda N^{-\frac{k-2}{2}}}=N^{-\vartheta}+\frac{N^{\frac{k-2}{4}}}{\lambda}. \tag*{(141)} s ∈ { ± 1 } min ∥ v − s θ ⋆ ∥ ≤ s ∈ { ± 1 } min ∥( v , a ) − s ( θ ⋆ , 0 )∥ ≲ k λ N − 2 k − 2 λ N − 2 k − 2 − ϑ + N 4 k − 2 = N − ϑ + λ N 4 k − 2 . (141) In particular, since the leading constant C k C_k C k λ ≥ C k N k − 2 4 \lambda\ge C_kN^{\frac{k-2}{4}} λ ≥ C k N 4 k − 2 min s ∈ { ± 1 } ∥ v − s θ ⋆ ∥ < 1 = ∥ θ ⋆ ∥ \min_{s\in\{\pm1\}}\lVert v-s\theta^\star\rVert<1=\lVert\theta^\star\rVert min s ∈ { ± 1 } ∥ v − s θ ⋆ ∥ < 1 = ∥ θ ⋆ ∥ v ≠ 0 v\ne0 v = 0 m ( N v / ∥ v ∥ ) = ⟨ θ ⋆ , v / ∥ v ∥ ⟩ m(\sqrt{N}v/\lVert v\rVert)=\langle\theta^\star,v/\lVert v\rVert\rangle m ( N v / ∥ v ∥) = ⟨ θ ⋆ , v / ∥ v ∥⟩ Lemma 3.4 to v v v s θ ⋆ s\theta^\star s θ ⋆ s ∈ { ± 1 } s\in\{\pm1\} s ∈ { ± 1 } (141) :
1 − ∣ m ( N v ∥ v ∥ ) ∣ = min s ∈ { ± 1 } ( 1 − s ⟨ θ ⋆ , v ∥ v ∥ ⟩ ) = 1 2 min s ∈ { ± 1 } ∥ v ∥ v ∥ − s θ ⋆ ∥ 2 ≤ 1 2 min s ∈ { ± 1 } ( 2 ∥ v − s θ ⋆ ∥ max { ∥ v ∥ , 1 } ) 2 ≤ 2 min s ∈ { ± 1 } ∥ v − s θ ⋆ ∥ 2 ≲ k N k − 2 2 λ 2 + N − 2 ϑ . \begin{aligned}
1-\left|m\left(\frac{\sqrt{N}v}{\lVert v\rVert}\right)\right|=\min_{s\in\{\pm1\}}\left(1-s\left\langle\theta^\star,\frac{v}{\lVert v\rVert}\right\rangle\right)=\frac{1}{2}\min_{s\in\{\pm1\}}\left\lVert\frac{v}{\lVert v\rVert}-s\theta^\star\right\rVert^2 \\
&\le\frac{1}{2}\min_{s\in\{\pm1\}}\left(\frac{2\lVert v-s\theta^\star\rVert}{\max\{\lVert v\rVert,1\}}\right)^2\le2\min_{s\in\{\pm1\}}\lVert v-s\theta^\star\rVert^2\lesssim_k \frac{N^{\frac{k-2}{2}}}{\lambda^2}+N^{-2\vartheta}.
\end{aligned} 1 − m ( ∥ v ∥ N v ) = s ∈ { ± 1 } min ( 1 − s ⟨ θ ⋆ , ∥ v ∥ v ⟩ ) = 2 1 s ∈ { ± 1 } min ∥ v ∥ v − s θ ⋆ 2 ≤ 2 1 s ∈ { ± 1 } min ( max {∥ v ∥ , 1 } 2 ∥ v − s θ ⋆ ∥ ) 2 ≤ 2 s ∈ { ± 1 } min ∥ v − s θ ⋆ ∥ 2 ≲ k λ 2 N 2 k − 2 + N − 2 ϑ . Here, the second equality holds since ∥ u − s θ ⋆ ∥ 2 = 2 − 2 s ⟨ θ ⋆ , u ⟩ \lVert u-s\theta^\star\rVert^2=2-2s\langle\theta^\star,u\rangle ∥ u − s θ ⋆ ∥ 2 = 2 − 2 s ⟨ θ ⋆ , u ⟩ u u u k k k
Remark 7.3. Compared to Theorem 5.1 , Theorem 7.2 requires, for odd k k k β 0 \beta_0 β 0 N \sqrt{N} N N 2 N^2 N 2 (53) in the proof of Theorem 5.1 , the estimation error of the ergodic second moment is larger than that of the ergodic average by a factor of N \sqrt{N} N M ‾ T a v \overline{M}_{T_{\mathrm{av}}} M T av k k k k k k Theorem 5.1 already relies on the ergodic second moment, and both theorems impose the same requirements on β 0 \beta_0 β 0 T a v T_{\mathrm{av}} T av
Boosting Weak Recovery to Persistent Strong Recovery
The following theorem shows that, for even k k k 1 − poly ( 1 / N ) 1-\operatorname{poly}(1/N) 1 − poly ( 1/ N ) poly ( N ) \operatorname{poly}(N) poly ( N )
The proof relies on the fact that, for even k k k x ⋆ x^\star x ⋆ ± x ⋆ \pm x^\star ± x ⋆ λ 1 k − 1 \lambda^{\frac{1}{k-1}} λ k − 1 1 S N \mathcal{S}_N S N Theorem 6.2 , which does not rely on the parity of k k k
Theorem 7.4 (Persistent Strong Recovery after Weak Recovery). Let k k k ( X t ) t ≥ 0 (X_t)_{t\geq0} ( X t ) t ≥ 0 H H H β > 0 \beta>0 β > 0 ρ ∈ ( 0 , 1 ) \rho\in(0,1) ρ ∈ ( 0 , 1 ) η ∈ ( 0 , 1 2 ) \eta\in(0,\frac{1}{2}) η ∈ ( 0 , 2 1 ) λ ≥ C β , k N k − 2 4 \lambda\geq C_{\beta,k}N^{\frac{k-2}{4}} λ ≥ C β , k N 4 k − 2 C β , k C_{\beta,k} C β , k
τ ρ ≔ inf { t ≥ 0 : ∣ m ( X t ) ∣ ≥ ρ } . \tau_\rho\coloneqq\inf\{t\geq0: |m(X_t)|\geq\rho\}. τ ρ : = inf { t ≥ 0 : ∣ m ( X t ) ∣ ≥ ρ } . Then, the following holds for all sufficiently large N N N T T T 2 ≤ T ≤ poly ( N ) 2\leq T\leq\operatorname{poly}(N) 2 ≤ T ≤ poly ( N ) X 0 X_0 X 0 H H H
P [ inf t ∈ [ 2 , T ] ∣ m ( X τ ρ + t ) ∣ ≥ 1 − N − η ] ≥ 1 − exp ( − c k N ) − 2 exp ( − c N 1 − 2 η ) . \mathbb{P}\left[\inf_{t\in[2,T]}|m(X_{\tau_\rho+t})|\geq1-N^{-\eta}\right]\geq1-\exp(-c_kN)-2\exp(-cN^{1-2\eta}). P [ t ∈ [ 2 , T ] inf ∣ m ( X τ ρ + t ) ∣ ≥ 1 − N − η ] ≥ 1 − exp ( − c k N ) − 2 exp ( − c N 1 − 2 η ) . Proof. Throughout the proof, we assume that N N N C β , k C_{\beta,k} C β , k Theorem 6.2 , τ ρ < ∞ \tau_\rho<\infty τ ρ < ∞ [40 ] . Hence, by the strong Markov property [47 Theorem 6.2.2] applied at τ ρ \tau_\rho τ ρ X 0 X_0 X 0 ∣ m ( X 0 ) ∣ ≥ ρ |m(X_0)|\geq\rho ∣ m ( X 0 ) ∣ ≥ ρ H H H
x ~ ⋆ ≔ sgn ( m ( X 0 ) ) x ⋆ , m ~ ( x ) ≔ 1 N ⟨ x ~ ⋆ , x ⟩ . \widetilde{x}^{\star}\coloneqq\operatorname{sgn}(m(X_0))x^{\star},\qquad\widetilde{m}(x)\coloneqq\frac{1}{N}\langle\widetilde{x}^{\star},x\rangle. x ⋆ : = sgn ( m ( X 0 )) x ⋆ , m ( x ) : = N 1 ⟨ x ⋆ , x ⟩ . Observe that m ~ ( X 0 ) = ∣ m ( X 0 ) ∣ ≥ ρ \widetilde{m}(X_0)=|m(X_0)|\geq\rho m ( X 0 ) = ∣ m ( X 0 ) ∣ ≥ ρ ∣ m ( x ) ∣ ≥ m ~ ( x ) |m(x)|\geq\widetilde{m}(x) ∣ m ( x ) ∣ ≥ m ( x ) x ∈ S N x\in\mathcal{S}_N x ∈ S N k k k H = N λ m ~ k + H 0 H=N\lambda\widetilde{m}^{k}+H_0 H = N λ m k + H 0 H H H x ~ ⋆ ∈ S N \widetilde{x}^{\star}\in\mathcal{S}_N x ⋆ ∈ S N H 0 H_0 H 0 m ~ \widetilde{m} m
We now repeat the steps of the proof of Theorem 6.2 with x ~ ⋆ \widetilde{x}^{\star} x ⋆ m ~ \widetilde{m} m x ⋆ x^{\star} x ⋆ m m m C = 1 C=1 C = 1 r ˉ N = N − s \bar r_N=N^{-s} r ˉ N = N − s s = min { η , k − 2 4 ( k − 1 ) } s=\min\{\eta,\frac{k-2}{4(k-1)}\} s = min { η , 4 ( k − 1 ) k − 2 } Theorem 6.2 , together with the standing assumption of § 6 with γ = k − 2 4 \gamma=\frac{k-2}{4} γ = 4 k − 2
η < min { 1 2 , k − 2 4 } , s < 1 2 , λ ≥ C β , k N k − 2 4 , λ r ˉ N k − 1 ≥ C β , k . \eta<\min\left\{\frac{1}{2},\frac{k-2}{4}\right\},\qquad s<\frac{1}{2},\qquad\lambda\geq C_{\beta,k}N^{\frac{k-2}{4}},\qquad\lambda\bar r_N^{k-1}\geq C_{\beta,k}. η < min { 2 1 , 4 k − 2 } , s < 2 1 , λ ≥ C β , k N 4 k − 2 , λ r ˉ N k − 1 ≥ C β , k . Here, the first condition holds since k ≥ 4 k\geq4 k ≥ 4 η < 1 / 2 \eta<1/2 η < 1/2 s ≤ k − 2 4 ( k − 1 ) < 1 / 2 s\leq\frac{k-2}{4(k-1)}<1/2 s ≤ 4 ( k − 1 ) k − 2 < 1/2 λ \lambda λ ( k − 1 ) s ≤ k − 2 4 (k-1)s\leq\frac{k-2}{4} ( k − 1 ) s ≤ 4 k − 2 r ˉ N → 0 \bar r_N\to0 r ˉ N → 0 m ~ ( X 0 ) ≥ ρ ≥ r ˉ N \widetilde{m}(X_0)\geq\rho\geq\bar r_N m ( X 0 ) ≥ ρ ≥ r ˉ N τ r ˉ N \tau_{\bar r_N} τ r ˉ N Theorem 6.2 , defined with m ~ \widetilde{m} m m m m
As noted in Remark 6.3, the proof of Theorem 6.2 does not use the parity of k k k H = N λ m ~ k + H 0 H=N\lambda\widetilde{m}^{k}+H_0 H = N λ m k + H 0 (59) with m ~ \widetilde{m} m m m m
L H , β m ~ ( x ) = v ~ ( x ) ( 1 − m ~ ( x ) ) − b ~ ( x ) , \mathcal{L}_{H,\beta}\widetilde{m}(x)=\widetilde{v}(x)(1-\widetilde{m}(x))-\widetilde{b}(x), L H , β m ( x ) = v ( x ) ( 1 − m ( x )) − b ( x ) , v ~ ( x ) ≔ β λ k m ~ ( x ) k − 1 ( 1 + m ~ ( x ) ) , b ~ ( x ) ≔ k N 2 m ~ ( x ) − β ⟨ ∇ S N H 0 ( x ) , ∇ S N m ~ ( x ) ⟩ . \widetilde{v}(x)\coloneqq\beta\lambda k\widetilde{m}(x)^{k-1}(1+\widetilde{m}(x)),\qquad\widetilde{b}(x)\coloneqq\frac{kN}{2}\widetilde{m}(x)-\beta\langle\nabla_{\mathcal{S}_N}H_0(x),\nabla_{\mathcal{S}_N}\widetilde{m}(x)\rangle. v ( x ) : = β λk m ( x ) k − 1 ( 1 + m ( x )) , b ( x ) : = 2 k N m ( x ) − β ⟨ ∇ S N H 0 ( x ) , ∇ S N m ( x )⟩ . Similarly, multiplying Equation (60) by sgn ( m ( X 0 ) ) \operatorname{sgn}(m(X_0)) sgn ( m ( X 0 )) t t t m ~ \widetilde m m sgn ( m ( X 0 ) ) M t m \operatorname{sgn}(m(X_0))M_t^m sgn ( m ( X 0 )) M t m
sgn ( m ( X 0 ) ) M t m = m ~ ( X t ) − m ~ ( X 0 ) − ∫ 0 t L H , β m ~ ( X s ) d s . \operatorname{sgn}(m(X_0))M_t^m=\widetilde m(X_t)-\widetilde m(X_0)-\int_0^t\mathcal{L}_{H,\beta}\widetilde m(X_s)\,\mathrm{d}s. sgn ( m ( X 0 )) M t m = m ( X t ) − m ( X 0 ) − ∫ 0 t L H , β m ( X s ) d s . As in the proof of Theorem 6.2 , we consider the events
E = { max x ∈ S N ∥ ∇ S N H 0 ( x ) ∥ ≤ C 1 N } , M = ⋂ j = 0 ⌈ T ⌉ − 1 { sup j ≤ t ≤ j + 1 ∣ M t m − M j m ∣ ≤ N − η 32 } . \mathcal{E}=\left\{\max_{x\in\mathbb{S}_N}\left\lVert\nabla_{\mathbb{S}_N}H_0(x)\right\rVert\leq C_1\sqrt{N}\right\},\qquad
\mathcal{M}=\bigcap_{j=0}^{\lceil T\rceil-1}\left\{\sup_{j\leq t\leq j+1}\left|M_t^m-M_j^m\right|\leq\frac{N^{-\eta}}{32}\right\}. E = { x ∈ S N max ∥ ∇ S N H 0 ( x ) ∥ ≤ C 1 N } , M = j = 0 ⋂ ⌈ T ⌉ − 1 { j ≤ t ≤ j + 1 sup M t m − M j m ≤ 32 N − η } . where the threshold 1 32 N − η \frac{1}{32}N^{-\eta} 32 1 N − η M \mathcal{M} M 1 32 min { r ‾ N , N − η } \frac{1}{32}\min\{\overline r_N,N^{-\eta}\} 32 1 min { r N , N − η } s ≤ η s\leq\eta s ≤ η X 0 X_0 X 0 H H H
P [ E c ] ≤ exp ( − c k N ) , P [ M c ] ≤ 2 ⌈ T ⌉ exp ( − c N 1 − 2 η ) . (142) \mathbb{P}[\mathcal{E}^{\mathrm{c}}]\leq\exp(-c_kN),\qquad\mathbb{P}[\mathcal{M}^{\mathrm{c}}]\leq2\lceil T\rceil\exp(-cN^{1-2\eta}). \tag*{(142)} P [ E c ] ≤ exp ( − c k N ) , P [ M c ] ≤ 2 ⌈ T ⌉ exp ( − c N 1 − 2 η ) . (142) Henceforth, we work on E ∩ M \mathcal{E}\cap\mathcal{M} E ∩ M
We emphasize that the initial state X 0 X_0 X 0 x ~ ⋆ \widetilde{x}^{\star} x ⋆ H 0 H_0 H 0 E \mathcal{E} E M \mathcal{M} M x ~ ⋆ \widetilde{x}^{\star} x ⋆ M \mathcal{M} M sgn ( m ( X 0 ) ) M t m \operatorname{sgn}(m(X_0))M_t^m sgn ( m ( X 0 )) M t m m ~ \widetilde m m M t m M_t^m M t m Theorem 6.2 are deterministic on E ∩ M \mathcal{E}\cap\mathcal{M} E ∩ M m ~ ( X t ) > 1 − N − η \widetilde m(X_t)>1-N^{-\eta} m ( X t ) > 1 − N − η t ∈ [ 2 , T ] t\in[2,T] t ∈ [ 2 , T ]
Finally, since ∣ m ∣ ≥ m ~ \lvert m\rvert\geq\widetilde m ∣ m ∣ ≥ m ∣ m ( X t ) ∣ > 1 − N − η \lvert m(X_t)\rvert>1-N^{-\eta} ∣ m ( X t )∣ > 1 − N − η t ∈ [ 2 , T ] t\in[2,T] t ∈ [ 2 , T ] E ∩ M \mathcal{E}\cap\mathcal{M} E ∩ M (142) via a union bound:
P [ inf t ∈ [ 2 , T ] ∣ m ( X t ) ∣ < 1 − N − η ] ≤ P [ E c ] + P [ M c ] ≤ exp ( − c k N ) + 2 ⌈ T ⌉ exp ( − c N 1 − 2 η ) . \mathbb{P}\left[\inf_{t\in[2,T]}\lvert m(X_t)\rvert<1-N^{-\eta}\right]\leq\mathbb{P}[\mathcal{E}^{\mathrm{c}}]+\mathbb{P}[\mathcal{M}^{\mathrm{c}}]\leq\exp(-c_kN)+2\lceil T\rceil\exp(-cN^{1-2\eta}). P [ t ∈ [ 2 , T ] inf ∣ m ( X t )∣ < 1 − N − η ] ≤ P [ E c ] + P [ M c ] ≤ exp ( − c k N ) + 2 ⌈ T ⌉ exp ( − c N 1 − 2 η ) . Since T ≤ poly ( N ) T\leq\operatorname{poly}(N) T ≤ poly ( N ) η < 1 / 2 \eta<1/2 η < 1/2 ⌈ T ⌉ \lceil T\rceil ⌈ T ⌉
Proof of Theorem 7.1
Proof. Throughout the proof, we assume that N N N T s t r = C 1 + 2 T_{\mathrm{str}}=C_1+2 T str = C 1 + 2 C 1 C_1 C 1 Theorem 6.5 , which depends only on β \beta β k k k β 0 − 2 λ 2 N k + 2 ϑ ≍ k N 2 k + 4 ϑ \beta_0^{-2}\lambda^2N^{k+2\vartheta}\asymp_kN^{2k+4\vartheta} β 0 − 2 λ 2 N k + 2 ϑ ≍ k N 2 k + 4 ϑ β 0 \beta_0 β 0 T a v T_{\mathrm{av}} T av Theorem 7.2 . By construction, ( X t ) t ≥ 0 (X_t)_{t\geq0} ( X t ) t ≥ 0 H H H β \beta β Theorem 7.2 , we have v ≠ 0 v\neq0 v = 0 X 0 = N v ∥ v ∥ X_0=\frac{\sqrt{N}v}{\lVert v\rVert} X 0 = ∥ v ∥ N v Theorems 6.2 , 6.5 and 7.4 allow the initial state to depend on H H H ( X t ) t ≥ 0 (X_t)_{t\geq0} ( X t ) t ≥ 0
We first analyze the case when k k k m ( X t ) m(X_t) m ( X t ) ∣ m ( X t ) ∣ \lvert m(X_t)\rvert ∣ m ( X t )∣ λ = C β , k N γ \lambda=C_{\beta,k}N^\gamma λ = C β , k N γ γ ≥ k − 2 4 \gamma\geq\frac{k-2}{4} γ ≥ 4 k − 2 Theorem 7.2 that the following holds with probability at least 1 − exp ( − c k N ) 1-\exp(-c_kN) 1 − exp ( − c k N )
1 N ∥ X 0 − u ^ H ∥ ≤ N − ϑ min { 1 , λ σ N } . (143) \frac{1}{\sqrt{N}}\lVert X_0-\widehat{u}_H\rVert\leq N^{-\vartheta}\min\{1,\lambda\sigma_N\}. \tag*{(143)} N 1 ∥ X 0 − u H ∥ ≤ N − ϑ min { 1 , λ σ N } . (143) We first consider the regime γ ≥ k − 2 4 + 1 4 k \gamma\geq\frac{k-2}{4}+\frac{1}{4k} γ ≥ 4 k − 2 + 4 k 1 X 0 X_0 X 0 Theorem 6.2 . Define a ≔ k − 1 4 k a\coloneqq\frac{k-1}{4k} a : = 4 k k − 1 γ ≥ ( k − 1 ) a \gamma\geq(k-1)a γ ≥ ( k − 1 ) a γ − k − 1 4 ≥ − a \gamma-\frac{k-1}{4}\geq-a γ − 4 k − 1 ≥ − a σ N ≍ k N − k − 1 4 \sigma_N \asymp_k N^{-\frac{k-1}{4}} σ N ≍ k N − 4 k − 1 Theorem 4.1 , it follows that λ σ N ≳ k C β , k N − α \lambda\sigma_N \gtrsim_k C_{\beta,k}N^{-\alpha} λ σ N ≳ k C β , k N − α ∣ m ( x ) − m ( y ) ∣ ≤ N − 1 / 2 ∥ x − y ∥ |m(x)-m(y)| \le N^{-1/2}\lVert x-y\rVert ∣ m ( x ) − m ( y ) ∣ ≤ N − 1/2 ∥ x − y ∥ x , y ∈ R N x,y\in\mathbb{R}^N x , y ∈ R N (143) , Corollary 4.3 and a union bound that the following holds with probability at least 1 − exp ( − c k N ( k + 1 ) / ( 2 k ) ) 1-\exp(-c_kN^{(k+1)/(2k)}) 1 − exp ( − c k N ( k + 1 ) / ( 2 k ) ) C β , k C_{\beta,k} C β , k
m ( X 0 ) ≥ m ( u ^ H ) − 1 N ∥ X 0 − u ^ H ∥ ≥ ( c k − N − ϑ ) min { 1 , λ σ N } ≥ N − α . (144) m(X_0)\ge m(\widehat{u}_H)-\frac{1}{\sqrt{N}}\lVert X_0-\widehat{u}_H\rVert\ge(c_k-N^{-\vartheta})\min\{1,\lambda\sigma_N\}\ge N^{-\alpha}. \tag*{(144)} m ( X 0 ) ≥ m ( u H ) − N 1 ∥ X 0 − u H ∥ ≥ ( c k − N − ϑ ) min { 1 , λ σ N } ≥ N − α . (144) Here, the second inequality combines the lower bound on m ( u ^ H ) m(\widehat{u}_H) m ( u H ) (143) , and the last inequality uses λ σ N ≳ k C β , k N − α \lambda\sigma_N\gtrsim_k C_{\beta,k}N^{-\alpha} λ σ N ≳ k C β , k N − α r ˉ N ≔ N − α \bar r_N\coloneqq N^{-\alpha} r ˉ N : = N − α γ ≥ ( k − 1 ) α \gamma\ge(k-1)\alpha γ ≥ ( k − 1 ) α λ r ˉ N k − 1 ≥ C β , k \lambda\bar r_N^{k-1}\ge C_{\beta,k} λ r ˉ N k − 1 ≥ C β , k Theorem 6.2 with s = α s=\alpha s = α C = 1 C=1 C = 1 τ r ˉ N = 0 \tau_{\bar r_N}=0 τ r ˉ N = 0 (144) , and using T s t r ≥ 2 T_{\rm str}\ge2 T str ≥ 2 N r ˉ N 2 k = N k + 1 2 k N\bar r_N^{2k}=N^{\frac{k+1}{2k}} N r ˉ N 2 k = N 2 k k + 1
P [ inf t ∈ [ T s t r , T ] m ( X t ) < 1 − N − η ] ≤ exp ( − c k N min { k + 1 2 k , 1 − 2 η } ) . \mathbb{P}\left[\inf_{t\in[T_{\rm str},T]}m(X_t)<1-N^{-\eta}\right]\le\exp\left(-c_kN^{\min\{\frac{k+1}{2k},1-2\eta\}}\right). P [ t ∈ [ T str , T ] inf m ( X t ) < 1 − N − η ] ≤ exp ( − c k N m i n { 2 k k + 1 , 1 − 2 η } ) . Since k + 1 2 k > 1 / 2 \frac{k+1}{2k}>1/2 2 k k + 1 > 1/2
We now consider the regime k − 2 4 ≤ γ < k − 2 4 + 1 4 k \frac{k-2}{4}\le\gamma<\frac{k-2}{4}+\frac{1}{4k} 4 k − 2 ≤ γ < 4 k − 2 + 4 k 1 r N ≔ λ − 1 k − 1 r_N\coloneqq\lambda^{-\frac{1}{k-1}} r N : = λ − k − 1 1 Theorem 6.1 . We first show that the second phase attains critical correlation in constant time. By Equation (143) , we have 1 N ∥ X 0 − u ^ H ∥ ≤ λ σ N N − ϑ \frac{1}{\sqrt{N}}\lVert X_0-\widehat{u}_H\rVert\le\lambda\sigma_NN^{-\vartheta} N 1 ∥ X 0 − u H ∥ ≤ λ σ N N − ϑ
Let r ˉ N ≔ N − γ k − 1 \bar r_N\coloneqq N^{-\frac{\gamma}{k-1}} r ˉ N : = N − k − 1 γ r N r_N r N Theorem 6.5 to e = X 0 e=X_0 e = X 0 C C C C r N = r ˉ N Cr_N=\bar r_N C r N = r ˉ N γ ≥ k − 2 4 \gamma\ge\frac{k-2}{4} γ ≥ 4 k − 2 N λ 2 σ N 2 ≳ k N N\lambda^2\sigma_N^2\gtrsim_k\sqrt{N} N λ 2 σ N 2 ≳ k N 1 − exp ( − c β , k N ) 1-\exp(-c_{\beta,k}\sqrt{N}) 1 − exp ( − c β , k N )
T ∗ ( X 0 ) = inf { t ≥ 0 : m ( X t ) ≥ r ˉ N } ≤ C 1 . (145) T_*(X_0)=\inf\{t\ge0:m(X_t)\ge\bar r_N\}\le C_1. \tag*{(145)} T ∗ ( X 0 ) = inf { t ≥ 0 : m ( X t ) ≥ r ˉ N } ≤ C 1 . (145) Moreover, since λ r ˉ N k − 1 = C β , k \lambda\bar r_N^{k-1}=C_{\beta,k} λ r ˉ N k − 1 = C β , k γ k − 1 ≤ 1 4 \frac{\gamma}{k-1}\le\frac14 k − 1 γ ≤ 4 1 Theorem 6.2 with s = γ k − 1 s=\frac{\gamma}{k-1} s = k − 1 γ C = 1 C=1 C = 1 τ r ˉ N = T ∗ ( X 0 ) \tau_{\bar r_N}=T_*(X_0) τ r ˉ N = T ∗ ( X 0 ) T s t r ≥ T ∗ ( X 0 ) + 2 T_{\rm str}\ge T_*(X_0)+2 T str ≥ T ∗ ( X 0 ) + 2 (145) and N r ˉ N 2 ≥ N N\bar r_N^2\ge\sqrt{N} N r ˉ N 2 ≥ N
P [ inf t ∈ [ T s t r , T ] m ( X t ) < 1 − N − η ] ≤ exp ( − c β , k N ) + exp ( − c k N min { 1 / 2 , 1 − 2 η } ) . \mathbb{P}\left[\inf_{t\in[T_{\rm str},T]}m(X_t)<1-N^{-\eta}\right]\le\exp(-c_{\beta,k}\sqrt{N})+\exp\left(-c_kN^{\min\{1/2,1-2\eta\}}\right). P [ t ∈ [ T str , T ] inf m ( X t ) < 1 − N − η ] ≤ exp ( − c β , k N ) + exp ( − c k N m i n { 1/2 , 1 − 2 η } ) . which proves the claim for odd k k k
Finally, we analyze the case when k k k C β , k C_{\beta,k} C β , k Theorem 7.2 that the following holds with probability at least 1 − exp ( − c k N ) 1-\exp(-c_kN) 1 − exp ( − c k N )
∣ m ( X 0 ) ∣ = ∣ m ( N u ∥ u ∥ ) ∣ ≥ 1 2 . (146) |m(X_0)|=\left|m\left(\frac{\sqrt{N}u}{\lVert u\rVert}\right)\right|\ge\frac12. \tag*{(146)} ∣ m ( X 0 ) ∣ = m ( ∥ u ∥ N u ) ≥ 2 1 . (146) In particular, the stopping time τ 1 / 2 \tau_{1/2} τ 1/2 Theorem 7.4 equals zero on the event in Equation (146) . Hence, applying Theorem 7.4 with ρ = 1 / 2 \rho=1/2 ρ = 1/2 T s t r ≥ 2 T_{\rm str}\ge2 T str ≥ 2
P [ inf t ∈ [ T s t r , T ] ∣ m ( X t ) ∣ < 1 − N − η ] ≤ exp ( − c k N ) + exp ( − c N 1 − 2 η ) . \mathbb{P}\left[\inf_{t\in[T_{\rm str},T]}|m(X_t)|<1-N^{-\eta}\right]\le\exp(-c_kN)+\exp(-cN^{1-2\eta}). P [ t ∈ [ T str , T ] inf ∣ m ( X t ) ∣ < 1 − N − η ] ≤ exp ( − c k N ) + exp ( − c N 1 − 2 η ) . Since 1 − 2 η ≥ min { 1 / 2 , 1 − 2 η } 1-2\eta\ge\min\{1/2,1-2\eta\} 1 − 2 η ≥ min { 1/2 , 1 − 2 η } □ \square □
