Introduction
A central topic in high-dimensional probability and statistics is to develop upper bounds on the distance
D n = sup A ∈ R ∣ P ( S n ∈ A ) − P ( G ∈ A ) ∣ (1) D_n = \sup_{A \in\mathcal{R}} \left| \mathbb{P}(S_n \in A) - \mathbb{P}(G \in A) \right| \tag*{(1)} D n = A ∈ R sup ∣ P ( S n ∈ A ) − P ( G ∈ A ) ∣ (1)
where S n = 1 n ∑ i = 1 n X i S_n = \frac{1}{\sqrt{n}}\sum_{i=1}^{n} X_i S n = n 1 ∑ i = 1 n X i is a normalized sum of centered i.i.d. random vectors in R p \mathbb{R}^p R p , G G G is a centered Gaussian vector with covariance matrix E ( X 1 X 1 ⊤ ) \mathbb{E}(X_1X_1^\top) E ( X 1 X 1 ⊤ ) , and R \mathcal{R} R is the class of all rectangles in R p \mathbb{R}^p R p . This measure of distance has attracted major interest because it strikes a special balance between theory and application. On one hand, it is known that accurate Gaussian approximation with respect to D n D_n D n is possible when p ≫ n p \gg n p ≫ n , while on the other hand, a wide range of high-dimensional inference procedures can be analyzed in terms of probabilities of the form P ( S n ∈ A ) \mathbb{P}(S_n \in A) P ( S n ∈ A ) . We refer to [5 , 9 ] (Chernozhukov et al. 2023) for overviews of these developments.
Prior work. Much of the contemporary interest in Berry-Esseen type bounds for D n D_n D n was initiated in Chernozhukov, Chetverikov and Kato [10 ] , where it was shown that if R \mathcal{R} R is replaced with a certain subclass of rectangles, and if the entries of X 1 X_1 X 1 have sufficiently light tails, then the corresponding distance is at most of order n − 1 / 8 log 7 / 8 ( p n ) n^{-1/8}\log^{7/8}(pn) n − 1/8 log 7/8 ( p n ) . This bound was pathbreaking in two key respects, because it featured mild logarithmic dependence on the dimension p p p , while also imposing no restriction on the off-diagonal entries of E ( X 1 X 1 ⊤ ) \mathbb{E}(X_1X_1^\top) E ( X 1 X 1 ⊤ ) . Moreover, the paper included similar results for Gaussian comparison and bootstrap approximation, which are topics that will be discussed in § 2 .
In the subsequent years, there was a cascade of theoretical advances seeking to extend and sharpen the results in Chernozhukov, Chetverikov and Kato [10 ] . However, before summarizing this line of work, a basic point to keep in mind is that for many years, there was not a clear consensus about whether near n − 1 / 2 n^{-1/2} n − 1/2 rates of Gaussian approximation over rectangles are even possible in a general sense when p ≫ n p \gg n p ≫ n . Indeed, at different times, conjectures were made that n − 1 / 6 n^{-1/6} n − 1/6 or n − 1 / 4 n^{-1/4} n − 1/4 rates of approximation may be optimal up to logarithmic
MSC2020 subject classifications. Primary 60F05; secondary 62E17, 62G09.
Keywords and phrases. high-dimensional central limit theorem, Berry–Esseen bound, Gaussian comparison, bootstrap, Gaussian surface measure. factors when Σ \Sigma Σ is unrestricted. Also, many contributions were made in improving the initial rates from near n − 1 / 8 n^{-1/8} n − 1/8 to near n − 1 / 4 n^{-1/4} n − 1/4 ([12 , 19 , 30 , 15 ] ), as well as in identifying conditions for ensuring asymptotically consistent Gaussian approximation when p ≫ n p \gg n p ≫ n ([17 , 30 , 27 , 18 ] ).
Broadly speaking, most of the progress that has been made in bounding D n D_n D n at faster than near n − 1 / 4 n^{-1/4} n − 1/4 rates has been based on additional structural assumptions. For example, the paper ([20 ] ) obtained near n − 1 / 3 n^{-1/3} n − 1/3 rates when the smallest eigenvalue λ min \lambda_{\min} λ m i n of the correlation matrix of X 1 X_1 X 1 satisfies λ min ≳ 1 \lambda_{\min} \gtrsim1 λ m i n ≳ 1 , and showed that this can be improved to near n − 1 / 2 n^{-1/2} n − 1/2 when X 1 X_1 X 1 also has a log-concave density. In turn, the achievability of n − 1 / 2 n^{-1/2} n − 1/2 rates without a log-concave density when λ min ≳ 1 \lambda_{\min} \gtrsim1 λ m i n ≳ 1 was shown in ([32 , 31 , 2 ] (Chernozhukov, Chetverikov and Koike 2023)). Furthermore, the paper ([22 ] ) obtained a similar result when a non-degeneracy assumption is satisfied by all of the 3 × 3 3 \times3 3 × 3 correlation submatrices, and the paper ([28 ] ) showed that Edgeworth-based approximations can yield near n − 1 n^{-1} n − 1 rates when λ min ≳ 1 \lambda_{\min} \gtrsim1 λ m i n ≳ 1 and X 1 X_1 X 1 has a Stein kernel. Meanwhile, it was shown in Fang and Koike [21 ] that a log-concave density for X 1 X_1 X 1 alone is sufficient to ensure near n − 1 / 2 n^{-1/2} n − 1/2 rates for general covariance matrices. In a different direction, the papers ([33 , 34 ] ) allow for certain degenerate covariance matrices, but instead work with a subclass of rectangles and assume the entries of X 1 X_1 X 1 satisfy a weak variance decay condition. Lastly, and most recently, the paper ([23 ] ) made progress without extra structure, showing that near n − 1 / 3 n^{-1/3} n − 1/3 rates are achievable for general covariance matrices.
Summary of results. Despite all of the advances just described, the question of whether D n D_n D n can be bounded at near n − 1 / 2 n^{-1/2} n − 1/2 rates for general covariance matrices has remained open, and the purpose of this work is to answer it affirmatively. More specifically, our first main result in Theorem 2.1 shows that if the entries of X 1 X_1 X 1 are normalized to have unit variance and if the sub-Gaussian condition ∥ X 1 j ∥ ψ 2 ≤ B \lVert X_{1j}\rVert_{\psi_2} \le B ∥ X 1 j ∥ ψ 2 ≤ B holds for all j ≤ p j \le p j ≤ p , then D n D_n D n is at most of order B n − 1 / 2 log 3 ( p n ) log ( n ) Bn^{-1/2}\log^3(pn)\log(n) B n − 1/2 log 3 ( p n ) log ( n ) .
As an extension of our Gaussian approximation result, we consider the related problem of comparing p p p -dimensional Gaussian vectors G ~ ∼ N ( 0 , Σ ~ ) \widetilde{G} \sim\mathcal{N}(0,\widetilde{\Sigma}) G ∼ N ( 0 , Σ ) and G ∼ N ( 0 , Σ ) G \sim\mathcal{N}(0,\Sigma) G ∼ N ( 0 , Σ ) over rectangles. Here, we show that the comparison can be controlled in terms of a special discrepancy measure d ( Σ ~ , Σ ) d(\widetilde{\Sigma},\Sigma) d ( Σ , Σ ) , which is defined in (2) and allows for Σ ~ \widetilde{\Sigma} Σ and Σ \Sigma Σ to be degenerate. In this notation, our Gaussian comparison result in Theorem 2.2 shows that if the entries of G G G are normalized to have unit variance, and if d ( Σ ~ , Σ ) ≤ δ d(\widetilde{\Sigma},\Sigma) \le\delta d ( Σ , Σ ) ≤ δ , then the distance sup A ∈ R ∣ P ( G ~ ∈ A ) − P ( G ∈ A ) ∣ \sup_{A \in\mathcal{R}} \left|\mathbb{P}(\widetilde{G} \in A) - \mathbb{P}(G \in A)\right| sup A ∈ R P ( G ∈ A ) − P ( G ∈ A ) is at most of order δ log ( p ) log ( 1 + 1 / δ ) \delta\log(p)\log(1+1/\delta) δ log ( p ) log ( 1 + 1/ δ ) . To our knowledge, the discrepancy measure d ( Σ ~ , Σ ) d(\widetilde{\Sigma},\Sigma) d ( Σ , Σ ) is novel in this context, and it is an important technical ingredient that enables Theorem 2.2 to improve upon other recent Gaussian comparison results in high dimensions. (See the remarks after that result for further discussion.) Also, in handling degenerate covariance matrices, this result may be of particular interest in connection with discretized versions of continuous Gaussian processes.
Our last main result deals with approximations involving the Gaussian multiplier bootstrap procedure ([10 ] ), which generates bootstrap samples of S n S_n S n defined by S n ∗ = 1 n ∑ i = 1 n ξ i ∗ X i S_n^* = \frac{1}{\sqrt{n}}\sum_{i=1}^n \xi_i^*X_i S n ∗ = n 1 ∑ i = 1 n ξ i ∗ X i , where ξ 1 ∗ , … , ξ n ∗ \xi_1^*,\ldots,\xi_n^* ξ 1 ∗ , … , ξ n ∗ are i.i.d. N ( 0 , 1 ) \mathcal{N}(0,1) N ( 0 , 1 ) random variables that are independent of X ‾ 1 , … , X n \overline{X}_1,\ldots,X_n X 1 , … , X n . If the entries of X 1 X_1 X 1 are normalized to have unit variance and satisfy a suitable sub-Gaussian type condition, then Theorem 2.3 shows that with high probability, the random distance sup A ∈ R ∣ P ∗ ( S n ∗ ∈ A ) − P ( S n ∈ A ) ∣ \sup_{A \in\mathcal{R}} \left|\mathbb{P}^*(S_n^* \in A) - \mathbb{P}(S_n \in A)\right| sup A ∈ R ∣ P ∗ ( S n ∗ ∈ A ) − P ( S n ∈ A ) ∣ satisfies a bound that is similar to the aforementioned one for D n D_n D n , where P ∗ \mathbb{P}^* P ∗ denotes probability conditional on X 1 , … , X n X_1,\ldots,X_n X 1 , … , X n . Likewise, Theorem 2.3 improves upon previous bootstrap approximation results in ways that are similar to the improvements of Theorem 2.1 in the Gaussian approximation context. (The bulk of comparable results on high-dimensional bootstrap approximation are analogous to the ones for Gaussian approximation, and are included in the works cited above.)
Proof techniques. The high-level framework of the proof is based on interpolation and induction, which have been foundational in previous work. If S n S_n S n and G G G are as in eqn. (1) , and N ∼ N ( 0 , I p ) N \sim\mathcal{N}(0,I_p) N ∼ N ( 0 , I p ) is independent of them, then our interpolation is built from the “smoothed orthant indicator” g u ( y ) = P ( y + u G + σ N ≤ β ) g_u(y) = \mathbb{P}(y + \sqrt{u}G + \sigma N \leq\beta) g u ( y ) = P ( y + u G + σ N ≤ β ) , where y ∈ R p y \in\mathbb{R}^p y ∈ R p is viewed as the argument, β ∈ R p \beta\in\mathbb{R}^p β ∈ R p is the corner point of the orthant, σ > 0 \sigma> 0 σ > 0 is a small smoothing parameter, and u = u ( t ) = 1 − t + σ 2 u = u(t) = 1 - t + \sigma^2 u = u ( t ) = 1 − t + σ 2 for t ∈ [ 0 , 1 ] t \in[0,1] t ∈ [ 0 , 1 ] . Because σ \sigma σ will be chosen to be negligible, it follows that the function Ψ ( t ) = E ( g u ( t S n ) ) \Psi(t) = \mathbb{E}(g_u(\sqrt{t}S_n)) Ψ ( t ) = E ( g u ( t S n )) satisfies Ψ ( 1 ) ≈ P ( S n ≤ β ) \Psi(1) \approx\mathbb{P}(S_n \leq\beta) Ψ ( 1 ) ≈ P ( S n ≤ β ) and Ψ ( 0 ) ≈ P ( G ≤ β ) \Psi(0) \approx\mathbb{P}(G \leq\beta) Ψ ( 0 ) ≈ P ( G ≤ β ) . Thus, the primary substance of the proof consists in controlling the difference P ( S n ≤ β ) − P ( G ≤ β ) ≈ ∫ 0 1 Ψ ′ ( t ) d t \mathbb{P}(S_n \leq\beta) - \mathbb{P}(G \leq\beta) \approx\int_0^1 \Psi'(t)dt P ( S n ≤ β ) − P ( G ≤ β ) ≈ ∫ 0 1 Ψ ′ ( t ) d t through the derivatives of g u ( y ) g_u(y) g u ( y ) .
The essential difficulty in quantifying these derivatives is due to the unrestricted correlation structure of G G G . To see this, note that g u ( y ) g_u(y) g u ( y ) can be rewritten as the probability that a standard Gaussian vector hits a generic polyhedron with p p p facets. In other words, the complexity of E ( X 1 X 1 ⊤ ) \mathbb{E}(X_1X_1^\top) E ( X 1 X 1 ⊤ ) translates into geometric complexity, which then appears in the derivatives of g u ( y ) g_u(y) g u ( y ) . Also, it should be emphasized that in the case when λ min ≳ 1 \lambda_{\min} \gtrsim1 λ m i n ≳ 1 , much of this complexity is concealed. Roughly speaking, this is because the function g u ( y ) g_u(y) g u ( y ) can be effectively replaced with the distribution function of N ( y , σ 2 I p ) \mathcal{N}(y,\sigma^2I_p) N ( y , σ 2 I p ) , which has special structure in its higher-order derivatives that can be leveraged in a non-trivial way when p p p is large [6 , 1 , 20 , 32 ] (Chernozhukov, Chetverikov and Koike 2023).
In light of the complexity of high-dimensional polyhedra, much prior work has pursued techniques that circumvent it. Two recent works that have stood out by dealing with polyhedra in a relatively direct way are [22 , 23 ] , and our work is similar in this regard. Accordingly, some of the key objects in our analysis are what we call “facet functions” ψ 1 ( y ) , … , ψ p ( y ) \psi_1(y),\ldots,\psi_p(y) ψ 1 ( y ) , … , ψ p ( y ) (see eqns. (10) and (13) ), which are proportional to the standard Gaussian surface measures of the p p p facets of a polyhedron P ( y ) P(y) P ( y ) parameterized by y ∈ R p y \in\mathbb{R}^p y ∈ R p . To a large extent, the problem of controlling Ψ ′ ( t ) \Psi'(t) Ψ ′ ( t ) when the matrix E ( X 1 X 1 ⊤ ) \mathbb{E}(X_1X_1^\top) E ( X 1 X 1 ⊤ ) is degenerate amounts to controlling the first several derivatives of ψ 1 , … , ψ p \psi_1,\ldots,\psi_p ψ 1 , … , ψ p . This connection is clarified by a quantity we call the “facet sensitivity” F ( y ) \mathcal{F}(y) F ( y ) of P ( y ) P(y) P ( y ) , which is a global measure of the sensitivity of the functions log ψ 1 , … , log ψ p \log\psi_1,\ldots,\log\psi_p log ψ 1 , … , log ψ p to translations of P ( y ) P(y) P ( y ) (see eqns. (11) and (12) and Proposition 3.4 ). On this point, it should be noted that the upper bounds on F ( y ) \mathcal{F}(y) F ( y ) that we present in Proposition 3.5 make use of Lemmas 26 and 30 in Gao and Zhang [23 ] . However, our introduction of F ( y ) \mathcal{F}(y) F ( y ) as a way to unify the quantities needed for upper bounding ∣ Ψ ′ ( t ) ∣ |\Psi'(t)| ∣ Ψ ′ ( t ) ∣ is a novel component of our work, and our interpolation is quite different from the two separate interpolations used in Gao and Zhang [23 ] .
Another novel part of the proof deals with bounding the curvature of the functions τ ↦ − log ψ j ( x + τ w ) \tau\mapsto-\log\psi_j(x+\tau w) τ ↦ − log ψ j ( x + τ w ) for τ ∈ R \tau\in\mathbb{R} τ ∈ R , where x , w ∈ R p x,w \in\mathbb{R}^p x , w ∈ R p are fixed. Because this curvature is difficult to control in a direct manner, we reformulate it in terms of the negative second derivative − A ′ ′ ( τ ) -A''(\tau) − A ′′ ( τ ) of the log-partition function A ( τ ) A(\tau) A ( τ ) of a certain Gibbs measure (see Proposition 4.7 and Lemma 5.1). The main benefit of this reformulation is that it becomes possible to express − A ′ ′ ( τ ) -A''(\tau) − A ′′ ( τ ) in terms of a variance computed with respect to the Gibbs measure—and there are a variety of tools for analyzing such variances. In particular, eqn. (34) and Lemma 5.3 lead to a variational representation of − A ′ ′ ( τ ) -A''(\tau) − A ′′ ( τ ) in eqn. (37) that ultimately yields the desired curvature bound. This representation could be considered a variant of the classical Helffer-Sjöstrand representation of covariances computed with respect to a Gibbs measure [26 p.352] , and is derived in the proof of Lemma 5.3 using a technique from Barthe and Cordero-Erausquin (2013, Section 2).
Outline. Section 2 states the main results. Section 3 gives a high-level proof of the main result on Gaussian approximation (Theorem 2.1 ), based on lower-level results whose proofs occupy Sections 4–8. The main results on Gaussian comparison (Theorem 2.2 ) and bootstrap approximation (Theorem 2.3 ) are proved in Sections 9 and 10 respectively. Section 11 is a statement on AI use.
Notation and conventions. If a a a and b b b are real numbers, we write a ≲ b a \lesssim b a ≲ b if a ≤ C b a \le Cb a ≤ C b for a positive absolute constant C C C whose value may change from line to line. Also, we write a = O ( b ) a = O(b) a = O ( b ) if ∣ a ∣ ≲ ∣ b ∣ |a| \lesssim|b| ∣ a ∣ ≲ ∣ b ∣ . The all-ones vector is 1 \mathbf{1} 1 , while the expressions 1 { ⋅ } \mathbf{1}\{\cdot\} 1 { ⋅ } and 1 A \mathbf{1}_A 1 A denote indicator functions. If x ∈ R p x \in\mathbb{R}^p x ∈ R p , we write ∥ x ∥ 2 \lVert x\rVert_2 ∥ x ∥ 2 for the Euclidean norm and let ∥ x ∥ ∞ = max j ≤ p ∣ x j ∣ \lVert x\rVert_\infty= \max_{j \le p} |x_j| ∥ x ∥ ∞ = max j ≤ p ∣ x j ∣ . If A A A is a matrix or tensor, then ∥ A ∥ ∞ \lVert A\rVert_\infty ∥ A ∥ ∞ refers to the maximum absolute value of the entries, while we write ∥ A ∥ F \lVert A\rVert_F ∥ A ∥ F for the Frobenius norm and ∥ A ∥ o p \lVert A\rVert_{\mathrm{op}} ∥ A ∥ op for the operator (spectral) norm. For a real-valued function F F F , let ∥ F ∥ s u p = sup x ∣ F ( x ) ∣ \lVert F\rVert_{\mathrm{sup}} = \sup_x |F(x)| ∥ F ∥ sup = sup x ∣ F ( x ) ∣ . When F F F takes values that are vectors, let ∥ F ∥ s u p = sup x ∥ F ( x ) ∥ 2 \lVert F\rVert_{\mathrm{sup}} = \sup_x \lVert F(x)\rVert_2 ∥ F ∥ sup = sup x ∥ F ( x ) ∥ 2 , and when F F F takes values that are matrices or tensors, let ∥ F ∥ s u p = sup x ∥ F ( x ) ∥ o p \lVert F\rVert_{\mathrm{sup}} = \sup_x \lVert F(x)\rVert_{\mathrm{op}} ∥ F ∥ sup = sup x ∥ F ( x ) ∥ op . If x x x and y y y are vectors in R p \mathbb{R}^p R p , then x ≤ y x \le y x ≤ y means x j ≤ y j x_j \le y_j x j ≤ y j for all j ≤ p j \le p j ≤ p . The Loewner partial order on symmetric matrices is denoted by ⪯ \preceq ⪯ . The standard Gaussian measure and density on R p \mathbb{R}^p R p are denoted by γ p \gamma_p γ p and ϕ p \phi_p ϕ p , while Φ \Phi Φ and ϕ \phi ϕ denote the distribution and density functions for N ( 0 , 1 ) \mathcal{N}(0,1) N ( 0 , 1 ) . For a linear subspace E ⊂ R p E \subset\mathbb{R}^p E ⊂ R p , the standard Gaussian measure on E E E is denoted as γ E \gamma_E γ E . We also write H d \mathcal{H}^d H d for the d d d -dimensional Hausdorff measure. The derivative in the direction of a vector w w w is denoted as ∂ w \partial_w ∂ w , and we write ∂ j = ∂ e j \partial_j = \partial_{e_j} ∂ j = ∂ e j , where e j e_j e j is the j j j th standard basis vector. If ζ \zeta ζ is a real-valued random variable, its ψ 2 \psi_2 ψ 2 -Orlicz norm is given by ∥ ζ ∥ ψ 2 = inf { t > 0 : E e ζ 2 / t 2 ≤ 2 } \lVert\zeta\rVert_{\psi_2} = \inf\{t > 0 : \mathbb{E}e^{\zeta^2/t^2} \le2\} ∥ ζ ∥ ψ 2 = inf { t > 0 : E e ζ 2 / t 2 ≤ 2 } . Finally, the collection of p p p -dimensional rectangles R \mathcal{R} R consists of all sets of the form A = ∏ j = 1 p I j A = \prod_{j=1}^p I_j A = ∏ j = 1 p I j , where I 1 , … , I p I_1,\ldots,I_p I 1 , … , I p are arbitrary intervals in R \mathbb{R} R that may be bounded or unbounded, and contain zero, one, or two endpoints.
Main results
This section states our three main results on Gaussian approximation, Gaussian comparison, and bootstrap approximation.
Gaussian approximation
Our first result concerns the normalized sum of centered i.i.d. random vectors whose coordinates have unit variance and sub-Gaussian tails. No assumption is made on the correlations between the coordinates.
Theorem 2.1. There is an absolute constant C C C such that the following holds. Let n , p ≥ 2 n,p \ge2 n , p ≥ 2 , and let X 1 , … , X n X_1,\ldots,X_n X 1 , … , X n be i.i.d. centered random vectors in R p \mathbb{R}^p R p with Var ( X 1 j ) = 1 \operatorname{Var}(X_{1j}) = 1 Var ( X 1 j ) = 1 and ∥ X 1 j ∥ ψ 2 ≤ B \lVert X_{1j}\rVert_{\psi_2} \le B ∥ X 1 j ∥ ψ 2 ≤ B for all j ≤ p j \le p j ≤ p . Also, let G G G be a centered Gaussian vector with covariance matrix E ( X 1 X 1 T ) \mathbb{E}(X_1X_1^{\mathsf{T}}) E ( X 1 X 1 T ) . Then,
sup A ∈ R ∣ P ( 1 n ∑ i = 1 n X i ∈ A ) − P ( G ∈ A ) ∣ ≤ C B log 3 ( p n ) log ( n ) n . \sup_{A \in\mathcal{R}} \left|\mathbb{P}\left(\frac{1}{\sqrt{n}}\sum_{i=1}^{n}X_i \in A\right)-\mathbb{P}(G \in A)\right| \le\frac{C B \log^3(pn)\log(n)}{\sqrt{n}}. A ∈ R sup P ( n 1 i = 1 ∑ n X i ∈ A ) − P ( G ∈ A ) ≤ n C B log 3 ( p n ) log ( n ) . This result is proven in Section 3, and it only depends on the distribution of X 1 X_1 X 1 through the parameter B B B . In particular, the covariance matrix Σ \Sigma Σ is allowed to be degenerate, and no lower bound on its eigenvalues is required. If the variances Var ( X 1 j ) \operatorname{Var}(X_{1j}) Var ( X 1 j ) are positive but otherwise arbitrary, then the theorem can be applied by replacing X 1 j X_{1j} X 1 j with X 1 j / Var ( X 1 j ) X_{1j}/\sqrt{\operatorname{Var}(X_{1j})} X 1 j / Var ( X 1 j ) for j ≤ p j \le p j ≤ p because rescaling the coordinates maps rectangles to rectangles.
Gaussian comparison
Our next result compares p p p -dimensional Gaussian vectors G ~ ∼ N ( 0 , Σ ~ ) \widetilde{G} \sim\mathcal{N}(0,\widetilde{\Sigma}) G ∼ N ( 0 , Σ ) and G ∼ N ( 0 , Σ ) G \sim\mathcal{N}(0,\Sigma) G ∼ N ( 0 , Σ ) over rectangles. The comparison is made in terms of the following measure of discrepancy between their covariance matrices
d ( Σ ~ , Σ ) = inf { t > 0 : ∣ ⟨ v , ( Σ ~ − Σ ) v ⟩ ∣ ≤ t ⟨ v , Σ v ⟩ for all v ∈ V } . (2) d(\widetilde{\Sigma},\Sigma)=\inf\left\{t>0:\left|\langle v,(\widetilde{\Sigma}-\Sigma)v\rangle\right|\leq t\langle v,\Sigma v\rangle\text{ for all }v\in\mathcal{V}\right\}. \tag*{(2)} d ( Σ , Σ ) = inf { t > 0 : ⟨ v , ( Σ − Σ ) v ⟩ ≤ t ⟨ v , Σ v ⟩ for all v ∈ V } . (2)
where V = { e 1 , … , e p } ∪ { e j ± e l : j < l } \mathcal{V}=\{e_1,\ldots,e_p\}\cup\{e_j\pm e_l:j<l\} V = { e 1 , … , e p } ∪ { e j ± e l : j < l } . From this definition, it is straightforward to check that d ( Σ ~ , Σ ) = ∞ d(\widetilde{\Sigma},\Sigma)=\infty d ( Σ , Σ ) = ∞ can only arise if G G G has an entry with 0 0 0 variance, or if it has a pair of (redundant) entries with correlation equal to ± 1 \pm1 ± 1 . If these exceptional cases do not occur, then the simpler formula
d ( Σ ~ , Σ ) = max v ∈ V ∣ ⟨ v , ( Σ ~ − Σ ) v ⟩ ∣ ⟨ v , Σ v ⟩ d(\widetilde{\Sigma},\Sigma)=\max_{v\in\mathcal{V}}\frac{\left|\langle v,(\widetilde{\Sigma}-\Sigma)v\rangle\right|}{\langle v,\Sigma v\rangle} d ( Σ , Σ ) = v ∈ V max ⟨ v , Σ v ⟩ ⟨ v , ( Σ − Σ ) v ⟩
can be used instead. Importantly, the following result only relies on the finiteness of d ( Σ ~ , Σ ) d(\widetilde{\Sigma},\Sigma) d ( Σ , Σ ) once the coordinates of G G G have been normalized. Its proof is given in § 9 .
Theorem 2.2. There is an absolute constant C C C such that the following holds. Let p ≥ 2 p\geq2 p ≥ 2 , let Σ \Sigma Σ and Σ ~ \widetilde{\Sigma} Σ be positive semidefinite p × p p\times p p × p matrices such that Σ j j = 1 \Sigma_{jj}=1 Σ j j = 1 for j ≤ p j\leq p j ≤ p . Also, let G ∼ N ( 0 , Σ ) G\sim\mathcal{N}(0,\Sigma) G ∼ N ( 0 , Σ ) and G ~ ∼ N ( 0 , Σ ~ ) \widetilde{G}\sim\mathcal{N}(0,\widetilde{\Sigma}) G ∼ N ( 0 , Σ ) . If d ( Σ ~ , Σ ) ≤ δ d(\widetilde{\Sigma},\Sigma)\leq\delta d ( Σ , Σ ) ≤ δ for some number δ > 0 \delta>0 δ > 0 , then
sup A ∈ R ∣ P ( G ~ ∈ A ) − P ( G ∈ A ) ∣ ≤ C δ log ( p ) log ( 1 + 1 / δ ) . \sup_{A\in\mathcal{R}}\left|\mathbb{P}(\widetilde{G}\in A)-\mathbb{P}(G\in A)\right|\leq C\delta\log(p)\log(1+1/\delta). A ∈ R sup P ( G ∈ A ) − P ( G ∈ A ) ≤ C δ log ( p ) log ( 1 + 1/ δ ) .
If the diagonal entries of Σ \Sigma Σ are positive but different from 1 1 1 , then Theorem 2.2 can still be used by conjugating Σ ~ \widetilde{\Sigma} Σ and Σ \Sigma Σ with Diag ( Σ ) − 1 / 2 \operatorname{Diag}(\Sigma)^{-1/2} Diag ( Σ ) − 1/2 . Apart from the generality of the covariance matrices, another valuable feature of Theorem 2.2 is that the bound has a near-linear dependence on the parameter δ \delta δ . Previously, Gaussian comparison results over rectangles have been established with near-linear dependence on the distinct discrepancy measure ∥ Σ ~ − Σ ∥ ∞ \lVert\widetilde{\Sigma}-\Sigma\rVert_\infty ∥ Σ − Σ ∥ ∞ ([20 , 32 ] ). However, such bounds have the limitation of depending on the reciprocal of the smallest eigenvalue λ min \lambda_{\min} λ m i n of the correlation matrix of G G G . To show that d ( Σ ~ , Σ ) d(\widetilde{\Sigma},\Sigma) d ( Σ , Σ ) compares favorably, it can be checked that if Σ j j = 1 \Sigma_{jj}=1 Σ j j = 1 for j ≤ p j\leq p j ≤ p and if λ min > 0 \lambda_{\min}>0 λ m i n > 0 , then the following bound holds for any choice of Σ ~ \widetilde{\Sigma} Σ ,
d ( Σ ~ , Σ ) ≤ 2 ∥ Σ ~ − Σ ∥ ∞ λ min . d(\widetilde{\Sigma},\Sigma)\leq\frac{2\lVert\widetilde{\Sigma}-\Sigma\rVert_\infty}{\lambda_{\min}}. d ( Σ , Σ ) ≤ λ m i n 2 ∥ Σ − Σ ∥ ∞ .
Earlier still, Gaussian comparison results were established in ([11 , 15 ] ) without placing constraints on the correlation matrices of G ~ \widetilde{G} G and G G G , but with rates of order ∥ Σ ~ − Σ ∥ ∞ 1 / 3 \lVert\widetilde{\Sigma}-\Sigma\rVert_\infty^{1/3} ∥ Σ − Σ ∥ ∞ 1/3 or ∥ Σ ~ − Σ ∥ ∞ 1 / 2 \lVert\widetilde{\Sigma}-\Sigma\rVert_\infty^{1/2} ∥ Σ − Σ ∥ ∞ 1/2 , up to logarithmic factors.
One further benefit of the discrepancy measure d ( Σ ~ , Σ ) d(\widetilde{\Sigma},\Sigma) d ( Σ , Σ ) is that it is well behaved when Σ ~ \widetilde{\Sigma} Σ is replaced by a sample covariance matrix Σ ^ \widehat{\Sigma} Σ of centered observations X 1 , … , X n X_1,\ldots,X_n X 1 , … , X n whose population covariance matrix is Σ \Sigma Σ . That is, even if ⟨ v , Σ v ⟩ = 0 \langle v,\Sigma v\rangle=0 ⟨ v , Σ v ⟩ = 0 for some v ∈ V v\in\mathcal{V} v ∈ V , then it follows that ⟨ v , X i ⟩ = 0 \langle v,X_i\rangle=0 ⟨ v , X i ⟩ = 0 almost surely for all i i i , and thus d ( Σ ^ , Σ ) d(\widehat{\Sigma},\Sigma) d ( Σ , Σ ) remains finite almost surely. This special property of d d d will make Theorem 2.2 useful toward obtaining a near 1 / n 1/\sqrt{n} 1/ n rate of bootstrap approximation below.
Bootstrap approximation
The Gaussian comparison result in Theorem 2.2 serves as a bridge to extend our Gaussian approximation result in Theorem 2.1 to a bootstrap approximation result. Let ξ 1 ∗ , … , ξ n ∗ \xi_1^*,\ldots,\xi_n^* ξ 1 ∗ , … , ξ n ∗ be i.i.d. N ( 0 , 1 ) \mathcal{N}(0,1) N ( 0 , 1 ) random variables that are independent of X 1 , … , X n X_1,\ldots,X_n X 1 , … , X n , and consider the Gaussian multiplier bootstrap version of S n S_n S n given by
S n ∗ = 1 n ∑ i = 1 n ξ i ∗ X i S_n^*=\frac{1}{\sqrt{n}}\sum_{i=1}^n \xi_i^*X_i S n ∗ = n 1 i = 1 ∑ n ξ i ∗ X i
which was popularized by Chernozhukov, Chetverikov and Kato [10 ] . Conditionally on X 1 , … , X n X_1,\ldots,X_n X 1 , … , X n , the vector S n ∗ S_n^* S n ∗ is centered Gaussian with covariance matrix Σ ^ = 1 n ∑ i = 1 n X i X i T \widehat{\Sigma}=\frac{1}{n}\sum_{i=1}^n X_iX_i^{\mathsf{T}} Σ = n 1 ∑ i = 1 n X i X i T . Likewise, we write P ∗ \mathbb{P}^* P ∗ for probability conditional on X 1 , … , X n X_1,\ldots,X_n X 1 , … , X n .
Theorem 2.3. There is an absolute constant C C C such that the following holds. Let n , p ≥ 2 n,p\geq2 n , p ≥ 2 , and let X 1 , … , X n X_1,\ldots,X_n X 1 , … , X n be i.i.d. centered random vectors in R p \mathbb{R}^p R p with covariance matrix Σ = E ( X 1 X 1 T ) \Sigma=\mathbb{E}(X_1X_1^{\mathsf{T}}) Σ = E ( X 1 X 1 T ) such that Σ j j = 1 \Sigma_{jj}=1 Σ j j = 1 for all j ≤ p j\leq p j ≤ p , and ∥ ⟨ X 1 , v ⟩ ∥ ψ 2 ≤ B ⟨ v , Σ v ⟩ \|\langle X_1,v\rangle\|_{\psi_2}\leq B\sqrt{\langle v,\Sigma v\rangle} ∥ ⟨ X 1 , v ⟩ ∥ ψ 2 ≤ B ⟨ v , Σ v ⟩ for all v ∈ V v\in\mathcal{V} v ∈ V . Then, for every α ∈ ( 0 , 1 ) \alpha\in(0,1) α ∈ ( 0 , 1 ) , the following event occurs with probability at least 1 − α 1-\alpha 1 − α ,
sup A ∈ A ∣ P ∗ ( S n ∗ ∈ A ) − P ( S n ∈ A ) ∣ ≤ C B log ( n ) n ( B log ( p / α ) log ( p ) + log 3 ( p n ) ) . \sup_{A\in\mathcal{A}}\left|\mathbb{P}^*(S_n^*\in A)-\mathbb{P}(S_n\in A)\right|
\leq\frac{CB\log(n)}{\sqrt{n}}\left(B\sqrt{\log(p/\alpha)}\log(p)+\log^3(pn)\right). A ∈ A sup ∣ P ∗ ( S n ∗ ∈ A ) − P ( S n ∈ A ) ∣ ≤ n C B log ( n ) ( B log ( p / α ) log ( p ) + log 3 ( p n ) ) .
The proof of Theorem 2.3 is given in § 10 . Analogously to Theorem 2.1 , the main novelty of this result is that it provides a near n − 1 / 2 n^{-1/2} n − 1/2 rate of approximation even when Σ \Sigma Σ may be far from invertible. The tail conditions on X 1 X_1 X 1 in this result are only modestly stronger than those in Theorem 2.1 , to the extent that the set V \mathcal{V} V expands { e 1 , … , e p } \{e_1,\ldots,e_p\} { e 1 , … , e p } by including the vectors in { e j ± e l : j < l } \{e_j\pm e_l:j<l\} { e j ± e l : j < l } . For context, it is common in the literature on high-dimensional probability and statistics to study settings where V \mathcal{V} V is replaced with R p \mathbb{R}^p R p i.e. where there is some B > 0 B>0 B > 0 such that ∥ ⟨ X 1 , v ⟩ ∥ ψ 2 ≤ B ⟨ v , Σ v ⟩ \|\langle X_1,v\rangle\|_{\psi_2}\leq B\sqrt{\langle v,\Sigma v\rangle} ∥ ⟨ X 1 , v ⟩ ∥ ψ 2 ≤ B ⟨ v , Σ v ⟩ holds for all v ∈ R p v\in\mathbb{R}^p v ∈ R p , (e.g. Koltchinskii and Lounici, [29 ] ; Vershynin, [40 ] ; Zhivotovskiy, [41 ] ).
Proof of Theorem 2.1
Standard arguments show that Theorem 2.1 can be reduced to the case when X 1 , … , X n X_1,\ldots,X_n X 1 , … , X n have bounded entries and the rectangles A A A are orthants, as recorded in Lemma 8.1 of the background results section. Hence, Theorem 2.1 is implied by the following result, whose proof will be covered in the rest of this section.
Theorem 3.1. There is an absolute constant C C C such that the following holds. Let p , n ≥ 2 p,n\geq2 p , n ≥ 2 , let B ‾ > 0 \overline{B}>0 B > 0 , and let X 1 , … , X n X_1,\ldots,X_n X 1 , … , X n be i.i.d. centered random vectors in R p \mathbb{R}^p R p with ∣ X 1 j ∣ ≤ B ‾ |X_{1j}|\leq\overline{B} ∣ X 1 j ∣ ≤ B almost surely and Var ( X 1 j ) = 1 \operatorname{Var}(X_{1j})=1 Var ( X 1 j ) = 1 for all j ≤ p j\leq p j ≤ p . Also, let G G G be a centered Gaussian vector with covariance matrix E ( X 1 X 1 T ) \mathbb{E}(X_1X_1^{\mathsf{T}}) E ( X 1 X 1 T ) , and for any k = 1 , … , n k=1,\ldots,n k = 1 , … , n , let
D ‾ k = sup β ∈ R p ∣ P ( 1 k ∑ i = 1 k X i ≤ β ) − P ( G ≤ β ) ∣ . \overline{D}_k=\sup_{\beta\in\mathbb{R}^p}\left|\mathbb{P}\left(\frac{1}{\sqrt{k}}\sum_{i=1}^k X_i\leq\beta\right)-\mathbb{P}(G\leq\beta)\right|. D k = β ∈ R p sup P ( k 1 i = 1 ∑ k X i ≤ β ) − P ( G ≤ β ) . Then, for any k = 1 , … , n k=1,\ldots,n k = 1 , … , n , we have
D ‾ k ≤ C B ‾ ( log ( p n ) ) 5 / 2 log ( n ) k . (3) \overline{D}_k\leq\frac{C\overline{B}(\log(pn))^{5/2}\log(n)}{\sqrt{k}}. \tag*{(3)} D k ≤ k C B ( log ( p n ) ) 5/2 log ( n ) . (3)
The proof of Theorem 3.1 is organized as follows. After some preliminary reductions, the distributions of 1 k ∑ i = 1 k X i \frac{1}{\sqrt{k}}\sum_{i=1}^k X_i k 1 ∑ i = 1 k X i and G G G are smoothed by a small Gaussian noise, and then connected by an interpolation (Lemma 3.2 ). The derivatives appearing in the interpolation are controlled through a special quantity we call the facet sensitivity, which is introduced and analyzed in § 3.1 –§ 3.2 . (The proofs of the lower-level results are deferred to § 4 –§ 8 .) Finally, the remaining ingredients are assembled in § 3.3 , where the proof of Theorem 3.1 is completed with an induction on k k k .
Technical preliminaries and further reductions. Throughout the proof of Theorem Theorem 3.1 , we may restrict our attention to the values of k = 1 , … , n k=1,\ldots,n k = 1 , … , n that satisfy
B ˉ ( log ( p n ) ) 5 / 2 log ( n ) k ≤ 1 4 , (4) \frac{\bar{B}(\log(pn))^{5/2}\log(n)}{\sqrt{k}} \le\frac{1}{4}, \tag*{(4)} k B ˉ ( log ( p n ) ) 5/2 log ( n ) ≤ 4 1 , (4)
for otherwise the conclusion (3) is trivially true. Similarly, we may assume that p n pn p n is large enough so that p n ≥ c pn \ge c p n ≥ c holds when c > 0 c>0 c > 0 is an absolute constant. To see this, recall that p , n ≥ 2 p,n\ge2 p , n ≥ 2 , and note that the almost-sure bound ∣ X i j ∣ ≤ B ˉ \lvert X_{ij}\rvert\le\bar{B} ∣ X ij ∣ ≤ B ˉ with Var ( X i j ) = 1 \operatorname{Var}(X_{ij})=1 Var ( X ij ) = 1 implies B ˉ ≥ 1 \bar{B}\ge1 B ˉ ≥ 1 . If the inequality p n < c pn<c p n < c held, then we would have k ≤ n ≤ c k\le n\le c k ≤ n ≤ c , implying [ c 1 / 2 ( log 4 ) − 5 / 2 ( log 2 ) − 1 ] B ˉ ( log ( p n ) ) 5 / 2 log ( n ) / k ≥ 1 \left[c^{1/2}(\log4)^{-5/2}(\log2)^{-1}\right]\bar{B}(\log(pn))^{5/2}\log(n)/\sqrt{k}\ge1 [ c 1/2 ( log 4 ) − 5/2 ( log 2 ) − 1 ] B ˉ ( log ( p n ) ) 5/2 log ( n ) / k ≥ 1 , and (3) would again be trivially true. For this reason, various intermediate results that depend on p n pn p n exceeding an absolute constant will use phrases such as “when p n pn p n is sufficiently large” in order to avoid the clutter of an extra recurring absolute constant.
From now until the induction step at the end of the proof in § 3.3 , we fix k = 2 , … , n k=2,\ldots,n k = 2 , … , n such that (4) holds, and we put
σ = C 0 B ˉ ( log ( p n ) ) 3 / 2 k , S = 1 k ∑ i = 1 k X i , S \ i = S − X i k , \sigma=\frac{C_0\bar{B}(\log(pn))^{3/2}}{\sqrt{k}},\qquad S=\frac{1}{\sqrt{k}}\sum_{i=1}^{k}X_i,\qquad S_{\backslash i}=S-\frac{X_i}{\sqrt{k}}, σ = k C 0 B ˉ ( log ( p n ) ) 3/2 , S = k 1 i = 1 ∑ k X i , S \ i = S − k X i ,
where C 0 ≥ 1 C_0\ge1 C 0 ≥ 1 is an absolute constant that is chosen large enough for the few steps below that require it. (We do not put a subscript k k k on σ \sigma σ and S S S above, because k k k is held fixed for almost the entire proof.) Unlike the generic absolute constant C C C , the value of C 0 C_0 C 0 stays fixed throughout the proof and does not change from line to line. Also, when we use the notation a ≲ b a\lesssim b a ≲ b to express the statement that a ≤ C b a\le Cb a ≤ C b for some absolute constant C > 0 C>0 C > 0 , the implicit constant C C C will not depend on C 0 C_0 C 0 .
Some frequently used relationships among the parameters are as follows. Since B ˉ ≥ 1 \bar{B}\ge1 B ˉ ≥ 1 , we have σ ≥ B ˉ / k ≥ n − 1 / 2 \sigma\ge\bar{B}/\sqrt{k}\ge n^{-1/2} σ ≥ B ˉ / k ≥ n − 1/2 , and (4) gives B ˉ ≤ k \bar{B}\le\sqrt{k} B ˉ ≤ k and σ ≤ C 0 ( log ( p n ) ) − 1 ( log ( n ) ) − 1 / 4 ≤ 1 \sigma\le C_0(\log(pn))^{-1}(\log(n))^{-1/4}\le1 σ ≤ C 0 ( log ( p n ) ) − 1 ( log ( n ) ) − 1/4 ≤ 1 , as p n pn p n is large. Moreover, each coordinate of X i / k X_i/\sqrt{k} X i / k is bounded almost surely in magnitude by
B ˉ k = σ C 0 ( log ( p n ) ) 3 / 2 , (5) \frac{\bar{B}}{\sqrt{k}}=\frac{\sigma}{C_0(\log(pn))^{3/2}}, \tag*{(5)} k B ˉ = C 0 ( log ( p n ) ) 3/2 σ , (5)
which is small compared to the smoothing parameter σ \sigma σ .
Next, let Σ = E ( X 1 X 1 T ) \Sigma=\mathbb{E}(X_1X_1^{\mathsf T}) Σ = E ( X 1 X 1 T ) so that G ∼ N ( 0 , Σ ) G\sim N(0,\Sigma) G ∼ N ( 0 , Σ ) . Also, let G ′ G' G ′ be an independent copy of G G G , and let N ∼ N ( 0 , I p ) N\sim N(0,I_p) N ∼ N ( 0 , I p ) , with the collection G , G ′ , N , X 1 , … , X n G,G',N,X_1,\ldots,X_n G , G ′ , N , X 1 , … , X n being independent. Both S S S and G G G will be perturbed by the noise vector σ G ′ + σ N \sigma G'+\sigma N σ G ′ + σ N . The importance of σ N \sigma N σ N is that its covariance matrix is equal to σ 2 I p \sigma^2 I_p σ 2 I p , which provides a small degree of smoothing in every direction, and thus ensures a basic level of differentiability even when Σ \Sigma Σ is degenerate. The smoothing effect of σ G ′ + σ N \sigma G'+\sigma N σ G ′ + σ N is quantified by a standard argument (Lemma Lemma 8.3 ) that gives
D ˉ k ≤ sup β ∈ R p ∣ P ( S + σ G ′ + σ N ≤ β ) − P ( G + σ G ′ + σ N ≤ β ) ∣ + C σ log ( p n ) . (6) \bar{D}_k\le\sup_{\beta\in\mathbb{R}^p}\left|\mathbb{P}\left(S+\sigma G'+\sigma N\le\beta\right)-\mathbb{P}\left(G+\sigma G'+\sigma N\le\beta\right)\right|+C\sigma\log(pn). \tag*{(6)} D ˉ k ≤ β ∈ R p sup ∣ P ( S + σ G ′ + σ N ≤ β ) − P ( G + σ G ′ + σ N ≤ β ) ∣ + C σ log ( p n ) . (6)
Thus, it remains to bound the supremum on the right side of (6) , and this will be done using the interpolation described below.
Interpolation. From now on β ∈ R p \beta\in\mathbb{R}^p β ∈ R p is fixed. We interpolate between the two smoothed distributions in (6) by means of the following functions. For u > 0 u>0 u > 0 and y ∈ R p y\in\mathbb{R}^p y ∈ R p , let
g u ( y ) = P ( y + u G + σ N ≤ β ) . g_u(y)=\mathbb{P}\left(y+\sqrt{u}G+\sigma N\le\beta\right). g u ( y ) = P ( y + u G + σ N ≤ β ) .
which may be viewed as an orthant indicator function that has been smoothed by convolution with a Gaussian density. Also, for t ∈ [ 0 , 1 ] t \in[0,1] t ∈ [ 0 , 1 ] , define the interpolation function
Ψ ( t ) = E u ( t ) ( t S ) , \Psi(t)=\mathbb{E}_{u(t)}(\sqrt{t}S), Ψ ( t ) = E u ( t ) ( t S ) ,
where we put u ( t ) = 1 − t + σ 2 u(t)=1-t+\sigma^2 u ( t ) = 1 − t + σ 2 . This leads to
P ( S + σ G ′ + σ N ≤ β ) − P ( G + σ G ′ + σ N ≤ β ) = ∫ 0 1 Ψ ′ ( t ) d t . (7) \mathbb{P}(S+\sigma G^{\prime}+\sigma N\leq\beta)-\mathbb{P}(G+\sigma G^{\prime}+\sigma N\leq\beta)=\int_{0}^{1}\Psi^{\prime}(t)\,\mathrm{d}t. \tag*{(7)} P ( S + σ G ′ + σ N ≤ β ) − P ( G + σ G ′ + σ N ≤ β ) = ∫ 0 1 Ψ ′ ( t ) d t . (7)
Strictly speaking, the integral is understood as a limit with the lower endpoint tending to 0 0 0 , which is justified by Proposition 3.7 . The following lemma provides an explicit expression for the derivative Ψ ′ ( t ) \Psi^{\prime}(t) Ψ ′ ( t ) in terms of the operator Δ Σ f = tr ( Σ ∇ 2 f ) \Delta_{\Sigma}f=\operatorname{tr}(\Sigma\nabla^{2}f) Δ Σ f = tr ( Σ ∇ 2 f ) acting on f ∈ C 2 ( R p ) f\in C^{2}(\mathbb{R}^{p}) f ∈ C 2 ( R p ) .
Lemma 3.2. If t ∈ ( 0 , 1 ] t\in(0,1] t ∈ ( 0 , 1 ] and u = u ( t ) u=u(t) u = u ( t ) , then the derivative Ψ ′ ( t ) \Psi^{\prime}(t) Ψ ′ ( t ) satisfies
Ψ ′ ( t ) = 1 2 t ∑ i = 1 k E ⟨ ∇ g u ( t S i + t k X i ) − ∇ g u ( t S i ) , X i k ⟩ − 1 2 E Δ Σ g u ( t S ) . (8) \Psi^{\prime}(t)=\frac{1}{2\sqrt{t}}\sum_{i=1}^{k}\mathbb{E}\left\langle\nabla g_{u}\left(\sqrt{t}S_{i}+\sqrt{\frac{t}{k}}X_{i}\right)-\nabla g_{u}\left(\sqrt{t}S_{i}\right),\frac{X_{i}}{\sqrt{k}}\right\rangle-\frac{1}{2}\mathbb{E}\Delta_{\Sigma}g_{u}(\sqrt{t}S). \tag*{(8)} Ψ ′ ( t ) = 2 t 1 i = 1 ∑ k E ⟨ ∇ g u ( t S i + k t X i ) − ∇ g u ( t S i ) , k X i ⟩ − 2 1 E Δ Σ g u ( t S ) . (8)
This formula is a basic consequence of Stein’s identity for Gaussian vectors and is proved in § 7 .
Polyhedra and the facet sensitivity
The terms in the formula (8) will eventually be controlled by a weighted sum of the Gaussian surface measures of the facets of certain polyhedra, which we now define.
Notation for polyhedra. A polyhedron in R p \mathbb{R}^{p} R p with linearly independent normal vectors n 1 , … , n p n_{1},\ldots,n_{p} n 1 , … , n p (with unit norm) and offset parameters a = ( a 1 , … , a p ) a=(a_{1},\ldots,a_{p}) a = ( a 1 , … , a p ) is denoted as
P = { z ∈ R p : ⟨ n j , z ⟩ ≤ a j for j = 1 , … , p } . (9) P=\{z\in\mathbb{R}^{p}:\langle n_{j},z\rangle\leq a_{j}\text{ for }j=1,\ldots,p\}. \tag*{(9)} P = { z ∈ R p : ⟨ n j , z ⟩ ≤ a j for j = 1 , … , p } . (9)
For each j ≤ p j\leq p j ≤ p , the j j j th facet of P P P is
F j = P ∩ { z ∈ R p : ⟨ n j , z ⟩ = a j } , F_{j}=P\cap\{z\in\mathbb{R}^{p}:\langle n_{j},z\rangle=a_{j}\}, F j = P ∩ { z ∈ R p : ⟨ n j , z ⟩ = a j } ,
and its Gaussian surface measure is
ν j ( P ) = ∫ F j φ p d H p − 1 , (10) \nu_{j}(P)=\int_{F_{j}}\varphi_{p}\,\mathrm{d}\mathcal{H}^{p-1}, \tag*{(10)} ν j ( P ) = ∫ F j φ p d H p − 1 , (10)
where H p − 1 \mathcal{H}^{p-1} H p − 1 denotes the ( p − 1 ) (p-1) ( p − 1 ) -dimensional Hausdorff measure. (An illustration of the notation describing polyhedra is given in Figure 1 later.) When P P P is fixed, we also write ν j = ν j ( P ) \nu_{j}=\nu_{j}(P) ν j = ν j ( P ) . For x ∈ R p x\in\mathbb{R}^{p} x ∈ R p , the translate P − x P-x P − x is a polyhedron of the same form, with the same normal vectors, while the j j j th offset and facet measure are respectively a j − ⟨ n j , x ⟩ a_{j}-\langle n_{j},x\rangle a j − ⟨ n j , x ⟩ and ν j ( P − x ) \nu_{j}(P-x) ν j ( P − x ) . For the sake of understanding various formulas given later, it is worth noting that the function x ↦ ν j ( P − x ) x\mapsto\nu_{j}(P-x) x ↦ ν j ( P − x ) is positive and C ∞ C^{\infty} C ∞ on R p \mathbb{R}^{p} R p (Lemma 8.7 ).
Figure 1. An illustration of the notation describing a polyhedron P P P in R 2 \mathbb{R}^2 R 2 with two linearly independent normal vectors. Note that P P P is unbounded, and so only part of it is shaded.
The following quantity plays a central role in our work. It quantifies the sensitivity of the facet measures ν j \nu_{j} ν j to translations of the polyhedron P P P .
Definition 3.3 (facet sensitivity). If P ⊂ R p P\subset\mathbb{R}^{p} P ⊂ R p is a polyhedron as in (9) with p p p facets having linearly independent normal vectors, then the facet sensitivity of P P P is defined as
F ( P ) = ∑ j = 1 p ν j ( P ) [ ∥ ∇ x log ν j ( P − x ) ∥ 2 2 + tr ( − ∇ x 2 log ν j ( P − x ) ) ] x = 0 . (11) \mathcal{F}(P)=\sum_{j=1}^{p}\nu_{j}(P)\left[\left\|\nabla_{x}\log\nu_{j}(P-x)\right\|_{2}^{2}+\operatorname{tr}\left(-\nabla_{x}^{2}\log\nu_{j}(P-x)\right)\right]_{x=0}. \tag*{(11)} F ( P ) = j = 1 ∑ p ν j ( P ) [ ∥ ∇ x log ν j ( P − x ) ∥ 2 2 + tr ( − ∇ x 2 log ν j ( P − x ) ) ] x = 0 . (11)
Importantly, the facet sensitivity satisfies F ( P ) ≥ 0 \mathcal{F}(P) \ge0 F ( P ) ≥ 0 , because the functions x ↦ ν j ( P − x ) x \mapsto\nu_j(P-x) x ↦ ν j ( P − x ) are log-concave (Lemma 8.7 ). Some interpretation for F ( P ) \mathcal{F}(P) F ( P ) is possible, since its terms are akin to the two classical forms of the trace of the Fisher information. An explicit formula for F ( P ) \mathcal{F}(P) F ( P ) in terms of the characteristics of P P P will be given in Lemma 4.2 .
Parameterized polyhedra. Recall that we let Σ = E ( X 1 X 1 T ) \Sigma= \mathbb{E}(X_1X_1^{\mathsf{T}}) Σ = E ( X 1 X 1 T ) , and for any u > 0 u > 0 u > 0 , let
Σ u = u Σ + σ 2 I p , \Sigma_u = u\Sigma+ \sigma^2 I_p, Σ u = u Σ + σ 2 I p ,
which is a positive definite matrix with diagonal entries u + σ 2 u+\sigma^2 u + σ 2 . For any j ≤ p j \le p j ≤ p and y ∈ R p y \in\mathbb{R}^p y ∈ R p , let
n j = Σ u 1 / 2 e j u + σ 2 , a j ( y ) = β j − y j u + σ 2 , n_j = \frac{\Sigma_u^{1/2}e_j}{\sqrt{u+\sigma^2}}, \qquad a_j(y) = \frac{\beta_j-y_j}{\sqrt{u+\sigma^2}}, n j = u + σ 2 Σ u 1/2 e j , a j ( y ) = u + σ 2 β j − y j ,
and denote the polyhedron with offsets a 1 ( y ) , … , a p ( y ) a_1(y),\ldots,a_p(y) a 1 ( y ) , … , a p ( y ) as
P ( y ) = { z ∈ R p : ⟨ n j , z ⟩ ≤ a j ( y ) for j ≤ p } = { z ∈ R p : Σ u 1 / 2 z ≤ β − y } . (12) P(y) = \{z \in\mathbb{R}^p : \langle n_j,z\rangle\le a_j(y)\text{ for }j \le p\}
= \{z \in\mathbb{R}^p : \Sigma_u^{1/2}z \le\beta-y\}. \tag*{(12)} P ( y ) = { z ∈ R p : ⟨ n j , z ⟩ ≤ a j ( y ) for j ≤ p } = { z ∈ R p : Σ u 1/2 z ≤ β − y } . (12)
The Gram matrix of n 1 , … , n p n_1,\ldots,n_p n 1 , … , n p is Σ u / ( u + σ 2 ) \Sigma_u/(u+\sigma^2) Σ u / ( u + σ 2 ) , which is invertible with unit diagonal entries. Hence, the vectors n 1 , … , n p n_1,\ldots,n_p n 1 , … , n p are linearly independent unit vectors with ⟨ n j , n l ⟩ = u Σ j l / ( u + σ 2 ) \langle n_j,n_l\rangle= u\Sigma_{jl}/(u+\sigma^2) ⟨ n j , n l ⟩ = u Σ j l / ( u + σ 2 ) for j ≠ l j \ne l j = l , and so P ( y ) P(y) P ( y ) is an instance of the polyhedron P P P discussed earlier. Likewise, we write F ( y ) = F ( P ( y ) ) \mathcal{F}(y)=\mathcal{F}(P(y)) F ( y ) = F ( P ( y )) for any y ∈ R p y \in\mathbb{R}^p y ∈ R p .
The polyhedron P ( y ) P(y) P ( y ) is directly connected to the smoothed orthant indicator g u ( y ) g_u(y) g u ( y ) , because (12) shows that
g u ( y ) = γ p ( P ( y ) ) , g_u(y)=\gamma_p(P(y)), g u ( y ) = γ p ( P ( y )) ,
where γ p \gamma_p γ p is the standard Gaussian measure on R p \mathbb{R}^p R p . Furthermore, the derivatives of g u g_u g u are linked to the facet measures ν j \nu_j ν j through the formula
− ∂ j g u ( y ) = ψ j ( y ) : = ν j ( P ( y ) ) u + σ 2 , (13) -\partial_j g_u(y)=\psi_j(y):=\frac{\nu_j(P(y))}{\sqrt{u+\sigma^2}}, \tag*{(13)} − ∂ j g u ( y ) = ψ j ( y ) := u + σ 2 ν j ( P ( y )) , (13)
which will be established in Lemma 4.3 (b), and provides some explanation as to why the facet sensitivity F ( y ) \mathcal{F}(y) F ( y ) will allow us to control the interpolation. We call ψ 1 , … , ψ p \psi_1,\ldots,\psi_p ψ 1 , … , ψ p the facet functions , and they will be a recurring part of our analysis. Lastly, note that because P ( y ) P(y) P ( y ) is defined in terms of Σ u \Sigma_u Σ u , the various objects associated with P ( y ) P(y) P ( y ) depend on u u u in general, but this will usually be suppressed for ease of notation.
Bounds involving the facet sensitivity
The following proposition is key, because it expresses the connection between the derivatives of the smoothed orthant indicator g u ( y ) g_u(y) g u ( y ) and the facet sensitivity F ( y ) \mathcal{F}(y) F ( y ) in quantitative terms. It is proved in § 4 .
Proposition 3.4. Let t ∈ ( 0 , 1 ] t \in(0,1] t ∈ ( 0 , 1 ] , u = u ( t ) u=u(t) u = u ( t ) and x ∈ R p x \in\mathbb{R}^p x ∈ R p . Then,
∣ E ⟨ ∇ g u ( x + t k X 1 ) − ∇ g u ( x ) , X 1 k ⟩ − t k Δ Σ g u ( x ) ∣ ≲ t B ‾ F ( x ) ( k u ) 3 / 2 + 1 ( p n ) 4 . \left|\mathbb{E}\left\langle\nabla g_u\left(x+\sqrt{\frac{t}{k}}X_1\right)-\nabla g_u(x),\frac{X_1}{\sqrt{k}}\right\rangle-\frac{\sqrt{t}}{k}\Delta_{\Sigma}g_u(x)\right|\lesssim\frac{t\overline{B}\mathcal{F}(x)}{(ku)^{3/2}}+\frac{1}{(pn)^4}. E ⟨ ∇ g u ( x + k t X 1 ) − ∇ g u ( x ) , k X 1 ⟩ − k t Δ Σ g u ( x ) ≲ ( k u ) 3/2 t B F ( x ) + ( p n ) 4 1 .
The next proposition provides upper bounds on the facet sensitivity. Notably, these bounds do not depend on the angles between the normal vectors, and they have well controlled dependence on p p p . Part (a) follows from Lemmas 26 and 30 in Gao and Zhang [23 ] . Part (b) is a new result, and part (c) combines parts (a) and (b) in the case of P = P ( y ) P=P(y) P = P ( y ) .
Proposition 3.5. If P P P is a polyhedron in R p \mathbb{R}^p R p of the form (9) having p p p facets with linearly independent normal vectors, then the following statements hold.
(a) The facet sensitivity satisfies F ( P ) ≲ ( log ( 2 p ) ) 3 / 2 \mathcal{F}(P) \lesssim(\log(2p))^{3/2} F ( P ) ≲ ( log ( 2 p ) ) 3/2 .
(b) If r > 0 r > 0 r > 0 and ∣ min j ≤ p a j ∣ ≥ r \lvert\min_{j\leq p} a_j\rvert\geq r ∣ min j ≤ p a j ∣ ≥ r , then F ( P ) ≲ p 2 e − r 2 / 16 \mathcal{F}(P) \lesssim p^2 e^{-r^2/16} F ( P ) ≲ p 2 e − r 2 /16 .
(c) If P = P ( y ) P=P(y) P = P ( y ) with u > 0 u>0 u > 0 and y ∈ R p y\in\mathbb{R}^p y ∈ R p as in (12) , then
F ( y ) ≲ ( log ( p n ) ) 3 / 2 { P ( min j ≤ p ( β j − y j ) < 8 ( u + σ 2 ) log ( p n ) ) + ( p n ) − 2 } . \mathcal{F}(y) \lesssim(\log(pn))^{3/2}\left\{\mathbb{P}\left(\min_{j\leq p}(\beta_j-y_j)<8\sqrt{(u+\sigma^2)\log(pn)}\right)+(pn)^{-2}\right\}. F ( y ) ≲ ( log ( p n ) ) 3/2 { P ( j ≤ p min ( β j − y j ) < 8 ( u + σ 2 ) log ( p n ) ) + ( p n ) − 2 } .
This result is proved in Section 6. For the polyhedron P ( y ) P(y) P ( y ) , the offset parameters are a j ( y ) = ( β j − y j ) / u + σ 2 a_j(y)=(\beta_j-y_j)/\sqrt{u+\sigma^2} a j ( y ) = ( β j − y j ) / u + σ 2 , and so the assumption of part (b) holds when y y y is “far” from the boundary of the orthant { y ≤ β } \{y\leq\beta\} { y ≤ β } with respect to the scale u + σ 2 \sqrt{u+\sigma^2} u + σ 2 .
The facet sensitivity also gives pointwise bounds on the third derivatives ∂ w Δ Σ g u \partial_w\Delta_{\Sigma}g_u ∂ w Δ Σ g u , as shown in the following lemma that is proved in Section 8.
Lemma 3.6. For all u > 0 u>0 u > 0 and y , w ∈ R p y,w\in\mathbb{R}^p y , w ∈ R p , we have
∣ ∂ w Δ Σ g u ( y ) ∣ ≤ ∥ w ∥ ∞ u 3 / 2 F ( y ) . \left\lvert\partial_w\Delta_{\Sigma}g_u(y)\right\rvert\leq\frac{\lVert w\rVert_\infty}{u^{3/2}}\mathcal{F}(y). ∣ ∂ w Δ Σ g u ( y ) ∣ ≤ u 3/2 ∥ w ∥ ∞ F ( y ) .
We can now obtain a pointwise bound for the derivative Ψ ′ \Psi' Ψ ′ in the interpolation.
Proposition 3.7. For t ∈ ( 0 , 1 ] t\in(0,1] t ∈ ( 0 , 1 ] and u ( t ) = 1 − t + σ 2 u(t)=1-t+\sigma^2 u ( t ) = 1 − t + σ 2 , let
r t = 20 u ( t ) log ( p n ) + B ‾ k and π t = P ( min j ≤ p ∣ ( β − t S \ 1 ) j ∣ < r t ) . r_t=20\sqrt{u(t)}\log(pn)+\frac{\overline{B}}{\sqrt{k}}
\qquad\text{and}\qquad
\pi_t=\mathbb{P}\left(\min_{j\leq p}\left\lvert(\beta-\sqrt{t}S_{\backslash1})_j\right\rvert<r_t\right). r t = 20 u ( t ) log ( p n ) + k B and π t = P ( j ≤ p min ( β − t S \1 ) j < r t ) . Then,
∣ Ψ ′ ( t ) ∣ ≲ t B ‾ ( log ( p n ) ) 3 / 2 k u ( t ) 3 / 2 ( π t + ( p n ) − 2 ) + 1 p n t . \left\lvert\Psi'(t)\right\rvert\lesssim\frac{\sqrt{t}\,\overline{B}(\log(pn))^{3/2}}{\sqrt{k}\,u(t)^{3/2}}\left(\pi_t+(pn)^{-2}\right)+\frac{1}{pn\sqrt{t}}. ∣ Ψ ′ ( t ) ∣ ≲ k u ( t ) 3/2 t B ( log ( p n ) ) 3/2 ( π t + ( p n ) − 2 ) + p n t 1 .
Proof. For each y ∈ R p y\in\mathbb{R}^p y ∈ R p , define the remainder function
R ( y ) = E ⟨ ∇ g u ( t ) ( y + t k X 1 ) − ∇ g u ( t ) ( y ) , X 1 k ⟩ − t k Δ Σ g u ( t ) ( y ) , R(y)=\mathbb{E}\left\langle\nabla g_{u(t)}\left(y+\sqrt{\frac{t}{k}}X_1\right)-\nabla g_{u(t)}(y),\frac{X_1}{\sqrt{k}}\right\rangle-\frac{\sqrt{t}}{k}\Delta_{\Sigma}g_{u(t)}(y), R ( y ) = E ⟨ ∇ g u ( t ) ( y + k t X 1 ) − ∇ g u ( t ) ( y ) , k X 1 ⟩ − k t Δ Σ g u ( t ) ( y ) ,
which is bounded in Proposition 3.4. Since S \ i S_{\backslash i} S \ i is independent of X i X_i X i , and X i X_i X i has the same distribution as X 1 X_1 X 1 , we may express the formula in Lemma 3.2 with this function and then rearrange the terms to obtain
Ψ ′ ( t ) = 1 2 t ∑ i = 1 k E ( R ( t S \ i ) + t k Δ Σ g u ( t ) ( t S \ i ) ) − 1 2 E Δ Σ g u ( t ) ( t S ) (14) \Psi'(t)=\frac{1}{2\sqrt{t}}\sum_{i=1}^{k}\mathbb{E}\left(R(\sqrt{t}S_{\backslash i})+\frac{\sqrt{t}}{k}\Delta_{\Sigma}g_{u(t)}(\sqrt{t}S_{\backslash i})\right)-\frac{1}{2}\mathbb{E}\Delta_{\Sigma}g_{u(t)}(\sqrt{t}S) \tag*{(14)} Ψ ′ ( t ) = 2 t 1 i = 1 ∑ k E ( R ( t S \ i ) + k t Δ Σ g u ( t ) ( t S \ i ) ) − 2 1 E Δ Σ g u ( t ) ( t S ) (14)
= 1 2 k ∑ i = 1 k E ( Δ Σ g u ( t ) ( t S \ i ) − Δ Σ g u ( t ) ( t S ) ) + 1 2 t ∑ i = 1 k E R ( t S \ i ) . (15) =\frac{1}{2k}\sum_{i=1}^{k}\mathbb{E}\left(\Delta_{\Sigma}g_{u(t)}(\sqrt{t}S_{\backslash i})-\Delta_{\Sigma}g_{u(t)}(\sqrt{t}S)\right)+\frac{1}{2\sqrt{t}}\sum_{i=1}^{k}\mathbb{E}R(\sqrt{t}S_{\backslash i}). \tag*{(15)} = 2 k 1 i = 1 ∑ k E ( Δ Σ g u ( t ) ( t S \ i ) − Δ Σ g u ( t ) ( t S ) ) + 2 t 1 i = 1 ∑ k E R ( t S \ i ) . (15)
We will control both of the sums in (15) using the facet sensitivity near the points t S \ i \sqrt{t}S_{\backslash i} t S \ i . To this end, put w i = t / k X i w_i=\sqrt{t/k}\,X_i w i = t / k X i , so that t S = t S \ i + w i \sqrt{t}S=\sqrt{t}S_{\backslash i}+w_i t S = t S \ i + w i and ∥ w i ∥ ∞ ≤ t B ‾ / k \lVert w_i\rVert_\infty\leq\sqrt{t}\,\overline{B}/\sqrt{k} ∥ w i ∥ ∞ ≤ t B / k . As a preparatory step, we first show that
E sup 0 ≤ θ ≤ 1 F ( t S \ i + θ w i ) ≲ ( log ( p n ) ) 3 / 2 π t + ( p n ) − 2 , i ≤ k . (16) \mathbb{E}\sup_{0\leq\theta\leq1}\mathcal{F}(\sqrt{t}S_{\backslash i}+\theta w_i)\lesssim(\log(pn))^{3/2}\pi_t+(pn)^{-2},\qquad i\leq k. \tag*{(16)} E 0 ≤ θ ≤ 1 sup F ( t S \ i + θ w i ) ≲ ( log ( p n ) ) 3/2 π t + ( p n ) − 2 , i ≤ k . (16)
Since u ( t ) + σ 2 ≤ 2 u ( t ) u(t)+\sigma^2 \le2u(t) u ( t ) + σ 2 ≤ 2 u ( t ) , the threshold in Proposition 3.5 (c) satisfies 8 u ( t ) + σ 2 log ( p n ) ≤ r t − B ˉ / k 8\sqrt{u(t)+\sigma^2}\log(pn) \le r_t-\bar{B}/\sqrt{k} 8 u ( t ) + σ 2 log ( p n ) ≤ r t − B ˉ / k . Due to this inequality and ∥ θ w i ∥ ∞ ≤ B ˉ / k \|\theta w_i\|_\infty\le\bar{B}/\sqrt{k} ∥ θ w i ∥ ∞ ≤ B ˉ / k , if we put y = t S ∖ i + θ w i y=\sqrt{t}S_{\setminus i}+\theta w_i y = t S ∖ i + θ w i inside the indicator of Proposition 3.5 (c), then it is at most 1 { ∣ min j ≤ p ( β − t S ∖ i ) j ∣ < r t } 1\{|\min_{j\le p}(\beta-\sqrt{t}S_{\setminus i})_j|<r_t\} 1 { ∣ min j ≤ p ( β − t S ∖ i ) j ∣ < r t } . Furthermore, the latter indicator has expectation equal to π t \pi_t π t , because S ∖ i S_{\setminus i} S ∖ i and S ∖ 1 S_{\setminus1} S ∖ 1 have the same distribution, and so (16) follows.
For the sum of remainders in (15) , we apply Proposition 3.4 with x = t S ∖ i x=\sqrt{t}S_{\setminus i} x = t S ∖ i , and then (16) with θ = 0 \theta=0 θ = 0 . In turn, summing over i ≤ k i\le k i ≤ k shows that
1 t ∑ i = 1 k ∣ E R ( t S ∖ i ) ∣ ≲ t B ˉ k u ( t ) 3 / 2 ( ( log ( p n ) ) 3 / 2 π t + ( p n ) − 2 ) + 1 p n t . \frac{1}{\sqrt{t}}\sum_{i=1}^{k}\left|\mathbb{E}R(\sqrt{t}S_{\setminus i})\right|
\lesssim\frac{\sqrt{t}\,\bar{B}}{\sqrt{k}u(t)^{3/2}}\left((\log(pn))^{3/2}\pi_t+(pn)^{-2}\right)+\frac{1}{pn\sqrt{t}}. t 1 i = 1 ∑ k E R ( t S ∖ i ) ≲ k u ( t ) 3/2 t B ˉ ( ( log ( p n ) ) 3/2 π t + ( p n ) − 2 ) + p n t 1 .
Turning to the first sum in (15) , the mean value theorem and Lemma 3.6 give
∣ Δ Σ g u ( t ) ( t S ∖ i + w i ) − Δ Σ g u ( t ) ( t S ∖ i ) ∣ ≤ sup 0 ≤ θ ≤ 1 ∣ ∂ w i Δ Σ g u ( t ) ( t S ∖ i + θ w i ) ∣ ≤ t B ˉ k u ( t ) 3 / 2 sup 0 ≤ θ ≤ 1 F ( t S ∖ i + θ w i ) . \begin{aligned}
\left|\Delta_{\Sigma}g_{u(t)}(\sqrt{t}S_{\setminus i}+w_i)-\Delta_{\Sigma}g_{u(t)}(\sqrt{t}S_{\setminus i})\right|
\le\sup_{0\le\theta\le1}\left|\partial_{w_i}\Delta_{\Sigma}g_{u(t)}(\sqrt{t}S_{\setminus i}+\theta w_i)\right| \\
&\le\frac{\sqrt{t}\,\bar{B}}{\sqrt{k}u(t)^{3/2}}\sup_{0\le\theta\le1}\mathcal{F}(\sqrt{t}S_{\setminus i}+\theta w_i).
\end{aligned} Δ Σ g u ( t ) ( t S ∖ i + w i ) − Δ Σ g u ( t ) ( t S ∖ i ) ≤ 0 ≤ θ ≤ 1 sup ∂ w i Δ Σ g u ( t ) ( t S ∖ i + θ w i ) ≤ k u ( t ) 3/2 t B ˉ 0 ≤ θ ≤ 1 sup F ( t S ∖ i + θ w i ) .
Taking expectations on both sides and applying (16) leads to the stated result. □ \square □
Concluding the proof of Theorem 3.1
The probability π t \pi_t π t in Proposition 3.7 involves only the k − 1 k-1 k − 1 vectors X 2 , … , X k X_2,\ldots,X_k X 2 , … , X k . This allows for a bound in terms of D ˉ k − 1 \bar{D}_{k-1} D ˉ k − 1 and eventually leads to a finite induction on k k k .
Lemma 3.8. Let t ∈ [ 1 / 2 , 1 ] t\in[1/2,1] t ∈ [ 1/2 , 1 ] , and let r t r_t r t and π t \pi_t π t be as in Proposition 3.7 . Then,
π t ≲ r t log ( p n ) + D ˉ k − 1 . \pi_t\lesssim r_t\sqrt{\log(pn)}+\bar{D}_{k-1}. π t ≲ r t log ( p n ) + D ˉ k − 1 .
Proof. Let S k − 1 = ( k − 1 ) − 1 / 2 ∑ i = 1 k − 1 X i S_{k-1}=(k-1)^{-1/2}\sum_{i=1}^{k-1}X_i S k − 1 = ( k − 1 ) − 1/2 ∑ i = 1 k − 1 X i , and note that t S ∖ 1 \sqrt{t}S_{\setminus1} t S ∖ 1 is equal in distribution to t ( k − 1 ) / k S k − 1 \sqrt{t(k-1)/k}S_{k-1} t ( k − 1 ) / k S k − 1 . Also observe that if the condition ∣ min j ( β j − y j ) ∣ < r t |\min_j(\beta_j-y_j)|<r_t ∣ min j ( β j − y j ) ∣ < r t holds, then so does y ≤ β + r t 1 y\le\beta+r_t\mathbf{1} y ≤ β + r t 1 but not y ≤ β − r t 1 y\le\beta-r_t\mathbf{1} y ≤ β − r t 1 . Hence, π t \pi_t π t is at most the difference of the probabilities that S k − 1 S_{k-1} S k − 1 lies in two orthants, and replacing S k − 1 S_{k-1} S k − 1 by G G G in each of them incurs an error of at most 2 D ˉ k − 1 2\bar{D}_{k-1} 2 D ˉ k − 1 . This leads to
π t ≤ P ( G ≤ k ( β + r t 1 ) t ( k − 1 ) ) − P ( G ≤ k ( β − r t 1 ) t ( k − 1 ) ) + 2 D ˉ k − 1 . \pi_t\le\mathbb{P}\left(G\le\frac{\sqrt{k}(\beta+r_t\mathbf{1})}{\sqrt{t(k-1)}}\right)-\mathbb{P}\left(G\le\frac{\sqrt{k}(\beta-r_t\mathbf{1})}{\sqrt{t(k-1)}}\right)+2\bar{D}_{k-1}. π t ≤ P ( G ≤ t ( k − 1 ) k ( β + r t 1 ) ) − P ( G ≤ t ( k − 1 ) k ( β − r t 1 ) ) + 2 D ˉ k − 1 .
The corners of the orthants in these two probabilities differ by 2 k / ( t ( k − 1 ) ) r t 1 ≤ 4 r t 1 2\sqrt{k/(t(k-1))}\,r_t\mathbf{1}\le4r_t\mathbf{1} 2 k / ( t ( k − 1 )) r t 1 ≤ 4 r t 1 . So, by Nazarov’s inequality (Lemma 8.2 ), the difference of the Gaussian probabilities is at most of order r t log ( p n ) r_t\sqrt{\log(pn)} r t log ( p n ) . □ \square □
We now derive a recursion that upper bounds D ˉ k \bar{D}_k D ˉ k in terms of D ˉ k − 1 \bar{D}_{k-1} D ˉ k − 1 . Let k = 2 , … , n k=2,\ldots,n k = 2 , … , n continue to satisfy (4) , so that the results of this section apply, and let β ∈ R p \beta\in\mathbb{R}^p β ∈ R p still be fixed. We will upper bound ∫ 0 1 ∣ Ψ ′ ( t ) ∣ d t \int_0^1|\Psi'(t)|\,dt ∫ 0 1 ∣ Ψ ′ ( t ) ∣ d t by integrating the derivative bound in Proposition 3.7 . Its last term contributes 1 p n ∫ 0 1 t − 1 / 2 d t ≤ B ˉ k \frac{1}{pn}\int_0^1t^{-1/2}\,dt\le\frac{\bar{B}}{\sqrt{k}} p n 1 ∫ 0 1 t − 1/2 d t ≤ k B ˉ , which is crude but adequate. Integrating the first term over ( 0 , 1 / 2 ] (0,1/2] ( 0 , 1/2 ] leads to a contribution of order ( B ˉ / k ) ( log ( p n ) ) 3 / 2 (\bar{B}/\sqrt{k})(\log(pn))^{3/2} ( B ˉ / k ) ( log ( p n ) ) 3/2 . When t ∈ [ 1 / 2 , 1 ] t\in[1/2,1] t ∈ [ 1/2 , 1 ] we use Lemma 3.8 , which gives
u ( t ) − 3 / 2 ( π t + ( p n ) − 2 ) ≲ log ( p n ) u ( t ) + B ˉ log ( p n ) k u ( t ) 3 / 2 + D ˉ k − 1 + ( p n ) − 2 u ( t ) 3 / 2 . u(t)^{-3/2}\left(\pi_t+(pn)^{-2}\right)\lesssim\frac{\log(pn)}{u(t)}+\frac{\bar{B}\sqrt{\log(pn)}}{\sqrt{k}u(t)^{3/2}}+\frac{\bar{D}_{k-1}+(pn)^{-2}}{u(t)^{3/2}}. u ( t ) − 3/2 ( π t + ( p n ) − 2 ) ≲ u ( t ) log ( p n ) + k u ( t ) 3/2 B ˉ log ( p n ) + u ( t ) 3/2 D ˉ k − 1 + ( p n ) − 2 .
Combining the last bound with ∫ 1 / 2 1 d t u ( t ) ≤ log ( n ) \int_{1/2}^{1}\frac{dt}{u(t)}\le\log(n) ∫ 1/2 1 u ( t ) d t ≤ log ( n ) and ∫ 1 / 2 1 d t u ( t ) 3 / 2 ≤ 2 σ \int_{1/2}^{1}\frac{dt}{u(t)^{3/2}}\le\frac{2}{\sigma} ∫ 1/2 1 u ( t ) 3/2 d t ≤ σ 2 , we have
∫ 0 1 ∣ Ψ ′ ( t ) ∣ d t ≲ B ˉ k ( log ( p n ) ) 3 / 2 ( log ( p n ) log ( n ) + B ˉ log ( p n ) k σ + D ˉ k − 1 + ( p n ) − 2 σ ) . \int_{0}^{1}|\Psi'(t)|\,dt\lesssim\frac{\bar{B}}{\sqrt{k}}(\log(pn))^{3/2}\left(\log(pn)\log(n)+\frac{\bar{B}\sqrt{\log(pn)}}{\sqrt{k}\sigma}+\frac{\bar{D}_{k-1}+(pn)^{-2}}{\sigma}\right). ∫ 0 1 ∣ Ψ ′ ( t ) ∣ d t ≲ k B ˉ ( log ( p n ) ) 3/2 ( log ( p n ) log ( n ) + k σ B ˉ log ( p n ) + σ D ˉ k − 1 + ( p n ) − 2 ) .
Furthermore, the smoothing error in (6) satisfies the bound
σ log ( p n ) ≤ C 0 log ( 2 ) B ˉ ( log ( p n ) ) 5 / 2 log ( n ) k , (17) \sigma\log(pn)\le\frac{C_{0}}{\log(2)}\frac{\bar{B}(\log(pn))^{5/2}\log(n)}{\sqrt{k}}, \tag*{(17)} σ log ( p n ) ≤ log ( 2 ) C 0 k B ˉ ( log ( p n ) ) 5/2 log ( n ) , (17)
because log ( n ) ≥ log ( 2 ) \log(n)\ge\log(2) log ( n ) ≥ log ( 2 ) . Combining this with (7) , the previous display, and C 0 ≥ 1 C_{0}\ge1 C 0 ≥ 1 , we obtain an absolute constant C 1 C_{1} C 1 , not depending on C 0 C_{0} C 0 , such that
D ˉ k ≤ C 1 C 0 B ˉ ( log ( p n ) ) 5 / 2 log ( n ) k + C 1 C 0 − 1 D ˉ k − 1 , (18) \bar{D}_{k}\le C_{1}C_{0}\frac{\bar{B}(\log(pn))^{5/2}\log(n)}{\sqrt{k}}+C_{1}C_{0}^{-1}\bar{D}_{k-1}, \tag*{(18)} D ˉ k ≤ C 1 C 0 k B ˉ ( log ( p n ) ) 5/2 log ( n ) + C 1 C 0 − 1 D ˉ k − 1 , (18)
for every k = 2 , … , n k=2,\ldots,n k = 2 , … , n satisfying (4) .
We conclude with a finite induction on k = 1 , … , n k=1,\ldots,n k = 1 , … , n . As C 1 C_{1} C 1 does not depend on C 0 C_{0} C 0 , we may increase the value of C 0 C_{0} C 0 if necessary so that C 1 C 0 − 1 ≤ 1 4 C_{1}C_{0}^{-1}\le\frac{1}{4} C 1 C 0 − 1 ≤ 4 1 . Letting M = max ( 4 , 2 C 1 C 0 ) M=\max(4,2C_{1}C_{0}) M = max ( 4 , 2 C 1 C 0 ) , we will show D ˉ k ≤ M B ˉ ( log ( p n ) ) 5 / 2 log ( n ) / k \bar{D}_{k}\le M\bar{B}(\log(pn))^{5/2}\log(n)/\sqrt{k} D ˉ k ≤ M B ˉ ( log ( p n ) ) 5/2 log ( n ) / k for k = 1 , … , n k=1,\ldots,n k = 1 , … , n , which will give the stated result. The base case of k = 1 k=1 k = 1 is clear. Now, let k ≥ 2 k\ge2 k ≥ 2 and assume the claim holds for k − 1 k-1 k − 1 . If (4) fails, then the claimed bound is at least M / 4 ≥ 1 ≥ D ˉ k M/4\ge1\ge\bar{D}_{k} M /4 ≥ 1 ≥ D ˉ k . Otherwise, the following bounds hold by (18) , C 1 C 0 − 1 ≤ 1 4 C_{1}C_{0}^{-1}\le\frac{1}{4} C 1 C 0 − 1 ≤ 4 1 , and the induction hypothesis
D ˉ k ≤ C 1 C 0 B ˉ ( log ( p n ) ) 5 / 2 log ( n ) k + 1 4 M B ˉ ( log ( p n ) ) 5 / 2 log ( n ) k − 1 ≤ ( 1 2 + 2 4 ) M B ˉ ( log ( p n ) ) 5 / 2 log ( n ) k ≤ M B ˉ ( log ( p n ) ) 5 / 2 log ( n ) k , \begin{aligned}
\bar{D}_{k}\le\frac{C_{1}C_{0}\bar{B}(\log(pn))^{5/2}\log(n)}{\sqrt{k}}+\frac{\frac{1}{4}M\bar{B}(\log(pn))^{5/2}\log(n)}{\sqrt{k-1}} \\
&\le\left(\frac{1}{2}+\frac{\sqrt{2}}{4}\right)\frac{M\bar{B}(\log(pn))^{5/2}\log(n)}{\sqrt{k}} \\
&\le\frac{M\bar{B}(\log(pn))^{5/2}\log(n)}{\sqrt{k}},
\end{aligned} D ˉ k ≤ k C 1 C 0 B ˉ ( log ( p n ) ) 5/2 log ( n ) + k − 1 4 1 M B ˉ ( log ( p n ) ) 5/2 log ( n ) ≤ ( 2 1 + 4 2 ) k M B ˉ ( log ( p n ) ) 5/2 log ( n ) ≤ k M B ˉ ( log ( p n ) ) 5/2 log ( n ) ,
where we have used k / ( k − 1 ) ≤ 2 \sqrt{k/(k-1)}\le\sqrt{2} k / ( k − 1 ) ≤ 2 and C 1 C 0 ≤ M / 2 C_{1}C_{0}\le M/2 C 1 C 0 ≤ M /2 in the second step. This completes the proof of Theorem 3.1 .
Proof of Proposition 3.4
Before proceeding directly to the proof of Proposition 3.4 in § 4.3 , we will develop a collection of preparatory results in § 4.1 and § 4.2 .
Properties of facet functions and the facet sensitivity
We begin by introducing some parameters associated with the sections of polyhedra.
Sections of polyhedra and related parameters. Let P P P be a polyhedron in R p \mathbb{R}^{p} R p with facets F 1 , … , F p F_{1},\ldots,F_{p} F 1 , … , F p having linearly independent normal vectors n 1 , … , n p n_{1},\ldots,n_{p} n 1 , … , n p . For each j ≤ p j\le p j ≤ p , let the ( p − 1 ) (p-1) ( p − 1 ) -dimensional subspace orthogonal to n j n_{j} n j be denoted as E j = n j ⊥ E_{j}=n_{j}^{\perp} E j = n j ⊥ , and let Π j \Pi_{j} Π j be the orthogonal projection onto E j E_{j} E j . The j j j th section of P P P and its Gaussian measure are denoted as
K j = { z ∈ E j : a j n j + z ∈ F j } and q j = γ E j ( K j ) . K_{j}=\{z\in E_{j}:a_{j}n_{j}+z\in F_{j}\}\qquad\text{and}\qquad q_{j}=\gamma_{E_{j}}(K_{j}). K j = { z ∈ E j : a j n j + z ∈ F j } and q j = γ E j ( K j ) .
It will be shown in Lemma 4.1 that q j > 0 q_{j}>0 q j > 0 . Also, let W W W be a random vector in E j E_{j} E j with distribution γ E j \gamma_{E_{j}} γ E j , and let
m j = E ( W ∣ W ∈ K j ) and G j = Cov ( W ∣ W ∈ K j ) . m_{j}=\mathbb{E}(W\mid W\in K_{j})\qquad\text{and}\qquad\mathcal{G}_{j}=\operatorname{Cov}(W\mid W\in K_{j}). m j = E ( W ∣ W ∈ K j ) and G j = Cov ( W ∣ W ∈ K j ) .
The next lemma relates ν j ( P ) \nu_{j}(P) ν j ( P ) to q j q_{j} q j and is proved in § 8.1 .
Lemma 4.1. Let P ⊂ R p P \subset\mathbb{R}^p P ⊂ R p be a polyhedron as in (9) with p p p facets having linearly independent normal vectors. Then, for any j ≤ p j \le p j ≤ p , the facet F j F_j F j can be represented as F j = a j n j + K j F_j = a_j n_j + K_j F j = a j n j + K j , and K j K_j K j is a closed convex subset of E j E_j E j with Gaussian measure q j > 0 q_j > 0 q j > 0 . Moreover,
ν j ( P ) = ϕ ( a j ) q j . \nu_j(P) = \phi(a_j)q_j. ν j ( P ) = ϕ ( a j ) q j .
Having introduced the parameters associated with sections, the facet sensitivity of Definition 3.3 can now be expressed in the following explicit form.
Lemma 4.2. If P P P is as in the statement of Lemma 4.1, then
F ( P ) = ∑ j = 1 p ( 1 + a j 2 + ∥ m j ∥ 2 2 + tr ( Π j − S j ) ) ν j ( P ) . \mathcal{F}(P) = \sum_{j=1}^{p} \left(1+a_j^2+\lVert m_j\rVert_2^2+\operatorname{tr}(\Pi_j-\mathcal{S}_j)\right)\nu_j(P). F ( P ) = j = 1 ∑ p ( 1 + a j 2 + ∥ m j ∥ 2 2 + tr ( Π j − S j ) ) ν j ( P ) .
Proof. Lemma 8.7 will show that the gradient and Hessian of the function x ↦ log ν j ( P − x ) x \mapsto\log\nu_j(P-x) x ↦ log ν j ( P − x ) at x = 0 x=0 x = 0 are respectively given by a j n j + m j a_j n_j+m_j a j n j + m j and − ( n j n j T + Π j − S j ) -(n_j n_j^{\mathsf T}+\Pi_j-\mathcal{S}_j) − ( n j n j T + Π j − S j ) . As m j ∈ E j m_j \in E_j m j ∈ E j is orthogonal to n j n_j n j , the squared norm of the gradient is a j 2 + ∥ m j ∥ 2 2 a_j^2+\lVert m_j\rVert_2^2 a j 2 + ∥ m j ∥ 2 2 , and the trace of minus the Hessian is 1 + tr ( Π j − S j ) 1+\operatorname{tr}(\Pi_j-\mathcal{S}_j) 1 + tr ( Π j − S j ) . Applying these observations to the formula (11) completes the proof.
We now return to the polyhedra P ( y ) P(y) P ( y ) of § 3.1 . For the objects of § 3.1 and § 4.1 attached to P ( y ) P(y) P ( y ) , we write
ν j ( y ) = ν j ( P ( y ) ) . \nu_j(y)=\nu_j(P(y)). ν j ( y ) = ν j ( P ( y )) .
and similarly for q j ( y ) q_j(y) q j ( y ) , m j ( y ) m_j(y) m j ( y ) and S j ( y ) \mathcal{S}_j(y) S j ( y ) . Also recall that ψ j ( y ) = ν j ( y ) / u + σ 2 \psi_j(y)=\nu_j(y)/\sqrt{u+\sigma^2} ψ j ( y ) = ν j ( y ) / u + σ 2 . The importance of the next lemma is that it expresses the derivatives of g u g_u g u in the interpolation through the polyhedron P ( y ) P(y) P ( y ) .
Lemma 4.3. Let u > 0 u>0 u > 0 and y , w ∈ R p y,w\in\mathbb{R}^p y , w ∈ R p . Also, let the normal vectors n 1 , … , n p n_1,\ldots,n_p n 1 , … , n p and the polyhedron P ( y ) P(y) P ( y ) be as in (12) . Then, the following statements hold.
(a) We have P ( y + w ) = P ( y ) − Σ u − 1 / 2 w P(y+w)=P(y)-\Sigma_u^{-1/2}w P ( y + w ) = P ( y ) − Σ u − 1/2 w .
(b) We have
∂ w g u ( y ) = − ∑ j = 1 p w j ψ j ( y ) , \partial_w g_u(y)=-\sum_{j=1}^{p}w_j\psi_j(y), ∂ w g u ( y ) = − j = 1 ∑ p w j ψ j ( y ) ,
Δ Σ g u ( y ) = − ∑ j = 1 p ( Σ ∇ ψ j ( y ) ) j = 1 u tr ( Γ ∇ x 2 γ p ( P ( y ) − x ) ∣ x = 0 ) , \Delta_{\Sigma}g_u(y)=-\sum_{j=1}^{p}(\Sigma\nabla\psi_j(y))_j=\frac{1}{u}\operatorname{tr}\left(\Gamma\nabla_x^2\gamma_p(P(y)-x)\big|_{x=0}\right), Δ Σ g u ( y ) = − j = 1 ∑ p ( Σ∇ ψ j ( y ) ) j = u 1 tr ( Γ ∇ x 2 γ p ( P ( y ) − x ) x = 0 ) ,
where Γ = u Σ u − 1 / 2 Σ Σ u − 1 / 2 \Gamma=u\Sigma_u^{-1/2}\Sigma\Sigma_u^{-1/2} Γ = u Σ u − 1/2 Σ Σ u − 1/2 satisfies 0 ⪯ Γ ⪯ I p 0\preceq\Gamma\preceq I_p 0 ⪯ Γ ⪯ I p .
Proof. Part (a) holds because z ∈ P ( y + w ) z\in P(y+w) z ∈ P ( y + w ) if and only if Σ u 1 / 2 ( z + Σ u − 1 / 2 w ) ≤ β − y \Sigma_u^{1/2}(z+\Sigma_u^{-1/2}w)\le\beta-y Σ u 1/2 ( z + Σ u − 1/2 w ) ≤ β − y . Turning to part (b), fix any y ∈ R p y\in\mathbb{R}^p y ∈ R p . From part (a) above and Lemma 8.6 (a), we have
∂ w g u ( y ) = − ∑ j = 1 p w j u + σ 2 ν j ( y ) = − ∑ j = 1 p w j ψ j ( y ) . \partial_wg_u(y)=-\sum_{j=1}^{p}\frac{w_j}{\sqrt{u+\sigma^2}}\nu_j(y)=-\sum_{j=1}^{p}w_j\psi_j(y). ∂ w g u ( y ) = − j = 1 ∑ p u + σ 2 w j ν j ( y ) = − j = 1 ∑ p w j ψ j ( y ) .
Hence Δ Σ g u = ∑ j , l Σ j l ∂ l ∂ j g u = − ∑ j ( Σ ∇ ψ j ) j \Delta_{\Sigma}g_u=\sum_{j,l}\Sigma_{jl}\partial_l\partial_jg_u=-\sum_j(\Sigma\nabla\psi_j)_j Δ Σ g u = ∑ j , l Σ j l ∂ l ∂ j g u = − ∑ j ( Σ∇ ψ j ) j . In addition, we have the Hessian formula
∇ 2 g u ( y ) = Σ u − 1 / 2 ∇ x 2 γ p ( P ( y ) − x ) ∣ x = 0 Σ u − 1 / 2 , \nabla^2g_u(y)=\Sigma_u^{-1/2}\nabla_x^2\gamma_p(P(y)-x)\big|_{x=0}\Sigma_u^{-1/2}, ∇ 2 g u ( y ) = Σ u − 1/2 ∇ x 2 γ p ( P ( y ) − x ) x = 0 Σ u − 1/2 ,
which gives the second formula for Δ Σ g u ( y ) = tr ( Σ ∇ 2 g u ( y ) ) \Delta_{\Sigma}g_u(y)=\operatorname{tr}(\Sigma\nabla^2g_u(y)) Δ Σ g u ( y ) = tr ( Σ ∇ 2 g u ( y )) . Finally, 0 ⪯ Γ ⪯ I p 0\preceq\Gamma\preceq I_p 0 ⪯ Γ ⪯ I p follows from 0 ⪯ u Σ ⪯ Σ u 0\preceq u\Sigma\preceq\Sigma_u 0 ⪯ u Σ ⪯ Σ u by conjugation with Σ u − 1 / 2 \Sigma_u^{-1/2} Σ u − 1/2 .
The following lemma records further properties of the facet functions and is proved in § 8.1 .
Lemma 4.4. If u > 0 u>0 u > 0 and j ≤ p j\le p j ≤ p , then for all y ∈ R p y\in\mathbb{R}^p y ∈ R p , we have
ψ j ( y ) = ϕ ( a j ( y ) ) q j ( y ) u + σ 2 , \psi_j(y)=\frac{\phi(a_j(y))q_j(y)}{\sqrt{u+\sigma^2}}, ψ j ( y ) = u + σ 2 ϕ ( a j ( y )) q j ( y ) , and the functions ψ j \psi_j ψ j and q j q_j q j are positive, C ∞ C^\infty C ∞ and log-concave on R p \mathbb{R}^p R p . In particular, for all y , w ∈ R p y,w\in\mathbb{R}^p y , w ∈ R p ,
∂ w log ψ j ( y ) = w j a j ( y ) u + σ 2 + ∂ w log q j ( y ) . (19) \partial_w\log\psi_j(y)=\frac{w_ja_j(y)}{\sqrt{u+\sigma^2}}+\partial_w\log q_j(y). \tag*{(19)} ∂ w log ψ j ( y ) = u + σ 2 w j a j ( y ) + ∂ w log q j ( y ) . (19) Moreover, for all y ∈ R p y\in\mathbb{R}^p y ∈ R p , we have
⟨ ∇ log ψ j ( y ) , Σ ∇ log ψ j ( y ) ⟩ ≤ a j ( y ) 2 + ∥ m j ( y ) ∥ 2 2 u , (20) \langle\nabla\log\psi_j(y),\Sigma\nabla\log\psi_j(y)\rangle\le\frac{a_j(y)^2+\lVert m_j(y)\rVert_2^2}{u}, \tag*{(20)} ⟨ ∇ log ψ j ( y ) , Σ∇ log ψ j ( y )⟩ ≤ u a j ( y ) 2 + ∥ m j ( y ) ∥ 2 2 , (20) tr ( Σ ( − ∇ 2 log ψ j ( y ) ) ) ≤ 1 + tr ( Π j − S j ( y ) ) u , (21) \operatorname{tr}\left(\Sigma(-\nabla^2\log\psi_j(y))\right)\le\frac{1+\operatorname{tr}(\Pi_j-\mathcal{S}_j(y))}{u}, \tag*{(21)} tr ( Σ ( − ∇ 2 log ψ j ( y )) ) ≤ u 1 + tr ( Π j − S j ( y )) , (21) as well as,
∑ j = 1 p ψ j ( y ) ≤ u − 1 / 2 F ( y ) , (22) \sum_{j=1}^{p}\psi_j(y)\le u^{-1/2}\mathcal{F}(y), \tag*{(22)} j = 1 ∑ p ψ j ( y ) ≤ u − 1/2 F ( y ) , (22) ∑ j = 1 p ψ j ( y ) ⟨ ∇ log ψ j ( y ) , Σ ∇ log ψ j ( y ) ⟩ ≤ u − 3 / 2 F ( y ) , (23) \sum_{j=1}^{p}\psi_j(y)\langle\nabla\log\psi_j(y),\Sigma\nabla\log\psi_j(y)\rangle\le u^{-3/2}\mathcal{F}(y), \tag*{(23)} j = 1 ∑ p ψ j ( y ) ⟨ ∇ log ψ j ( y ) , Σ∇ log ψ j ( y )⟩ ≤ u − 3/2 F ( y ) , (23) ∑ j = 1 p ψ j ( y ) tr ( Σ ( − ∇ 2 log ψ j ( y ) ) ) ≤ u − 3 / 2 F ( y ) . (24) \sum_{j=1}^{p}\psi_j(y)\operatorname{tr}\left(\Sigma(-\nabla^2\log\psi_j(y))\right)\le u^{-3/2}\mathcal{F}(y). \tag*{(24)} j = 1 ∑ p ψ j ( y ) tr ( Σ ( − ∇ 2 log ψ j ( y )) ) ≤ u − 3/2 F ( y ) . (24)
The second term in (19) is controlled by the following lemma, which is proved in § 5.7 .
Lemma 4.5. For all u ∈ ( 0 , 2 ] u \in(0,2] u ∈ ( 0 , 2 ] , y , w ∈ R p y,w \in\mathbb{R}^{p} y , w ∈ R p and j ≤ p j \le p j ≤ p ,
∣ ∂ w log q j ( y ) ∣ ≲ ∥ w ∥ ∞ σ ( log ( p n ) log 1 q j ( y ) + 1 ) . \left|\partial_{w}\log q_{j}(y)\right| \lesssim\frac{\|w\|_{\infty}}{\sigma}\left(\sqrt{\log(pn)}\log\frac{1}{q_{j}(y)}+1\right). ∣ ∂ w log q j ( y ) ∣ ≲ σ ∥ w ∥ ∞ ( log ( p n ) log q j ( y ) 1 + 1 ) .
Regular facets
This subsection distinguishes facets into two types that we call regular and non-regular (Definition 4.6). Roughly speaking, a facet F j F_j F j is regular when its offset parameter a j a_j a j is not very large, and its Gaussian section measure q j q_j q j is non-negligible. One reason why this matters is that if F j F_j F j is a regular facet, then it will turn out that the curvature of log ψ j \log\psi_j log ψ j can be controlled (Proposition 4.7), and this will be essential for the proof of Proposition 3.4 .
Definition 4.6 (regular facets). For u > 0 u>0 u > 0 , x ∈ R p x\in\mathbb{R}^{p} x ∈ R p , j ≤ p j\le p j ≤ p , we say the facet F j F_j F j (or the index j j j ) is regular at x x x if
∣ a j ( x ) ∣ ≤ 20 log ( p n ) and q j ( x ) ≥ ( p n ) − 10 . |a_j(x)| \le\sqrt{20\log(pn)}
\qquad\text{and}\qquad
q_j(x) \ge(pn)^{-10}. ∣ a j ( x ) ∣ ≤ 20 log ( p n ) and q j ( x ) ≥ ( p n ) − 10 .
If the j j j th facet is regular at x x x , then the proof of Proposition 3.4 relies on the curvature of log ψ j \log\psi_j log ψ j being well controlled at points near x x x , which is ensured by the following result.
Proposition 4.7. Let u ∈ ( 0 , 2 ] u\in(0,2] u ∈ ( 0 , 2 ] , j ≤ p j\le p j ≤ p and x , w ∈ R p x,w\in\mathbb{R}^{p} x , w ∈ R p be fixed with ∥ w ∥ ∞ ≤ B ‾ / k \|w\|_{\infty}\le\overline{B}/\sqrt{k} ∥ w ∥ ∞ ≤ B / k . If j j j is regular at x x x , then the function f j ( τ ) = log ψ j ( x + τ w ) f_j(\tau)=\log\psi_j(x+\tau w) f j ( τ ) = log ψ j ( x + τ w ) satisfies
− f j ′ ′ ( τ ) ≲ − f j ′ ′ ( 0 ) + B ‾ 2 k p n for all τ ∈ [ 0 , 1 ] . -f_j^{\prime\prime}(\tau)\lesssim-f_j^{\prime\prime}(0)+\frac{\overline{B}^{2}}{kpn}
\qquad\text{for all }\tau\in[0,1]. − f j ′′ ( τ ) ≲ − f j ′′ ( 0 ) + k p n B 2 for all τ ∈ [ 0 , 1 ] .
This result is crucial, and it will be explained in § 5.1 that it is a consequence of Proposition 5.2.
Concluding the proof of Proposition 3.4
Before starting the proof, we need the following lemma, which records some properties about regular facets and is proved in § 7 .
Lemma 4.8. Let j ≤ p j\le p j ≤ p , u ∈ ( 0 , 2 ] u\in(0,2] u ∈ ( 0 , 2 ] and x , w ∈ R p x,w\in\mathbb{R}^{p} x , w ∈ R p with ∥ w ∥ ∞ ≤ B ‾ / k \|w\|_{\infty}\le\overline{B}/\sqrt{k} ∥ w ∥ ∞ ≤ B / k .
(a) If j j j is regular at x x x , then ∣ ∂ w log ψ j ( x ) ∣ ≤ 1 100 \left|\partial_w\log\psi_j(x)\right|\le\frac{1}{100} ∣ ∂ w log ψ j ( x ) ∣ ≤ 100 1 .
(b) If j j j is not regular at x x x , then for all τ ∈ [ 0 , 1 ] \tau\in[0,1] τ ∈ [ 0 , 1 ] we have
( ∣ a j ( x ) ∣ + log 1 q j ( x ) + 1 ) ν j ( x ) ≲ ( p n ) − 5 and ν j ( x + τ w ) ≲ ( p n ) − 5 . \left(|a_j(x)|+\log\frac{1}{q_j(x)}+1\right)\nu_j(x)\lesssim(pn)^{-5}
\qquad\text{and}\qquad
\nu_j(x+\tau w)\lesssim(pn)^{-5}. ( ∣ a j ( x ) ∣ + log q j ( x ) 1 + 1 ) ν j ( x ) ≲ ( p n ) − 5 and ν j ( x + τ w ) ≲ ( p n ) − 5 .
For the purposes of proving Proposition 3.4 , the objects t ∈ ( 0 , 1 ] t\in(0,1] t ∈ ( 0 , 1 ] , u = u ( t ) u=u(t) u = u ( t ) and x ∈ R p x\in\mathbb{R}^{p} x ∈ R p will remain fixed throughout this subsection. For each j ≤ p j\le p j ≤ p and w ∈ R p w\in\mathbb{R}^{p} w ∈ R p , define the linear function h j ( w ) = ⟨ ∇ log ψ j ( x ) , w ⟩ h_j(w)=\langle\nabla\log\psi_j(x),w\rangle h j ( w ) = ⟨ ∇ log ψ j ( x ) , w ⟩ and the remainder function
R j ( w ) = ψ j ( x + t w ) − ψ j ( x ) − t ψ j ( x ) h j ( w ) . (25) R_j(w)=\psi_j(x+\sqrt{t}w)-\psi_j(x)-\sqrt{t}\,\psi_j(x)h_j(w). \tag*{(25)} R j ( w ) = ψ j ( x + t w ) − ψ j ( x ) − t ψ j ( x ) h j ( w ) . (25)
By Lemma 4.3 (b), we have
⟨ ∇ g u ( x + t w ) − ∇ g u ( x ) , w ⟩ = − t ∑ j = 1 p w j ψ j ( x ) h j ( w ) − ∑ j = 1 p w j R j ( w ) . (26) \left\langle\nabla g_u(x+\sqrt{t}w)-\nabla g_u(x),w\right\rangle
=-\sqrt{t}\sum_{j=1}^{p}w_j\psi_j(x)h_j(w)-\sum_{j=1}^{p}w_jR_j(w). \tag*{(26)} ⟨ ∇ g u ( x + t w ) − ∇ g u ( x ) , w ⟩ = − t j = 1 ∑ p w j ψ j ( x ) h j ( w ) − j = 1 ∑ p w j R j ( w ) . (26)
We will apply this with w = X 1 / k w = X_1/\sqrt{k} w = X 1 / k and take expectations. Since h j h_j h j is linear, we have E ( X 1 j h j ( X 1 ) ) = ( Σ ∇ log ψ j ( x ) ) j \mathbb{E}(X_{1j}h_j(X_1)) = (\Sigma\nabla\log\psi_j(x))_j E ( X 1 j h j ( X 1 )) = ( Σ∇ log ψ j ( x ) ) j . Together with ψ j ∇ log ψ j = ∇ ψ j \psi_j\nabla\log\psi_j = \nabla\psi_j ψ j ∇ log ψ j = ∇ ψ j and Lemma 4.3(b), this shows that the first term on the right of (26) has expectation
− t k ∑ j = 1 p ψ j ( x ) E ( X 1 j h j ( X 1 ) ) = − t k ∑ j = 1 p ( Σ ∇ ψ j ( x ) ) j = t k Δ Σ g u ( x ) . (27) -\frac{\sqrt{t}}{k}\sum_{j=1}^{p}\psi_j(x)\mathbb{E}(X_{1j}h_j(X_1))=-\frac{\sqrt{t}}{k}\sum_{j=1}^{p}(\Sigma\nabla\psi_j(x))_j=\frac{\sqrt{t}}{k}\Delta_\Sigma g_u(x). \tag*{(27)} − k t j = 1 ∑ p ψ j ( x ) E ( X 1 j h j ( X 1 )) = − k t j = 1 ∑ p ( Σ∇ ψ j ( x ) ) j = k t Δ Σ g u ( x ) . (27)
It therefore remains to bound the sum
1 k ∑ j = 1 p E ∣ X 1 j R j ( X 1 / k ) ∣ , \frac{1}{\sqrt{k}}\sum_{j=1}^{p}\mathbb{E}|X_{1j}R_j(X_1/\sqrt{k})|, k 1 j = 1 ∑ p E ∣ X 1 j R j ( X 1 / k ) ∣ ,
and this will be done by handling the regular and non-regular indices j j j separately.
Remainder bound for regular facets. Let j j j be regular at x x x , and first let w ∈ R p w\in\mathbb{R}^p w ∈ R p be deterministic with ∥ w ∥ ∞ ≤ B ˉ / k \|w\|_\infty\leq\bar{B}/\sqrt{k} ∥ w ∥ ∞ ≤ B ˉ / k . By Lemma 4.4, the function f j ( τ ) = log ψ j ( x + τ w ) f_j(\tau)=\log\psi_j(x+\tau w) f j ( τ ) = log ψ j ( x + τ w ) is C ∞ C^\infty C ∞ and concave with f j ′ ( 0 ) = h j ( w ) f_j'(0)=h_j(w) f j ′ ( 0 ) = h j ( w ) , so Taylor’s formula with integral remainder leads to
ψ j ( x + t w ) = ψ j ( x ) e t h j ( w ) − ε j ( w ) where ε j ( w ) = ∫ 0 t ( t − r ) ( − f j ′ ′ ( r ) ) d r ≥ 0 , \psi_j(x+\sqrt{t}w)=\psi_j(x)e^{\sqrt{t}h_j(w)-\varepsilon_j(w)}
\qquad\text{where}\qquad
\varepsilon_j(w)=\int_0^{\sqrt{t}}(\sqrt{t}-r)(-f_j''(r))\,dr\geq0, ψ j ( x + t w ) = ψ j ( x ) e t h j ( w ) − ε j ( w ) where ε j ( w ) = ∫ 0 t ( t − r ) ( − f j ′′ ( r )) d r ≥ 0 ,
where we recall that t ∈ ( 0 , 1 ] t\in(0,1] t ∈ ( 0 , 1 ] . Consequently, we may express the remainder R j ( w ) R_j(w) R j ( w ) defined in (25) as
R j ( w ) = ψ j ( x ) [ e t h j ( w ) − ε j ( w ) − 1 − t h j ( w ) ] . R_j(w)=\psi_j(x)\left[e^{\sqrt{t}h_j(w)-\varepsilon_j(w)}-1-\sqrt{t}h_j(w)\right]. R j ( w ) = ψ j ( x ) [ e t h j ( w ) − ε j ( w ) − 1 − t h j ( w ) ] .
By Lemma 4.8(a), we have ∣ h j ( w ) ∣ ≤ 1 100 |h_j(w)|\leq\frac{1}{100} ∣ h j ( w ) ∣ ≤ 100 1 , and then applying the inequality ∣ e a − b − 1 − a ∣ ≤ a 2 + 2 b |e^{a-b}-1-a|\leq a^2+2b ∣ e a − b − 1 − a ∣ ≤ a 2 + 2 b for ∣ a ∣ ≤ 1 100 |a|\leq\frac{1}{100} ∣ a ∣ ≤ 100 1 and b ≥ 0 b\geq0 b ≥ 0 gives
∣ R j ( w ) ∣ ≤ ψ j ( x ) [ t h j ( w ) 2 + 2 ε j ( w ) ] . (28) |R_j(w)|\leq\psi_j(x)\left[th_j(w)^2+2\varepsilon_j(w)\right]. \tag*{(28)} ∣ R j ( w ) ∣ ≤ ψ j ( x ) [ t h j ( w ) 2 + 2 ε j ( w ) ] . (28)
Next, a step that deserves special emphasis is that ε j ( w ) \varepsilon_j(w) ε j ( w ) can be bounded pointwise in w w w by Proposition 4.7. Combining that result with f j ′ ′ ( 0 ) = ⟨ w , ∇ 2 log ψ j ( x ) w ⟩ ≤ 0 f_j''(0)=\langle w,\nabla^2\log\psi_j(x)w\rangle\leq0 f j ′′ ( 0 ) = ⟨ w , ∇ 2 log ψ j ( x ) w ⟩ ≤ 0 gives
ε j ( w ) ≤ − t ⟨ w , ∇ 2 log ψ j ( x ) w ⟩ + t B ˉ 2 k p n . \varepsilon_j(w)\leq-t\langle w,\nabla^2\log\psi_j(x)w\rangle+\frac{t\bar{B}^2}{kpn}. ε j ( w ) ≤ − t ⟨ w , ∇ 2 log ψ j ( x ) w ⟩ + k p n t B ˉ 2 .
After applying the last bound to (28) and substituting w = X 1 / k w=X_1/\sqrt{k} w = X 1 / k , we take the expectation over X 1 X_1 X 1 to obtain
1 k E ∣ X 1 j R j ( X 1 / k ) ∣ ≲ t ψ j ( x ) B ˉ k E ( h j ( X 1 / k ) 2 − 1 k ⟨ X 1 , ∇ 2 log ψ j ( x ) X 1 ⟩ + B ˉ 2 k p n ) = t ψ j ( x ) B ˉ k 3 / 2 ( ⟨ ∇ log ψ j ( x ) , Σ ∇ log ψ j ( x ) ⟩ + tr ( Σ ( − ∇ 2 log ψ j ( x ) ) ) + B ˉ 2 p n ) , \begin{aligned}
\frac{1}{\sqrt{k}}\mathbb{E}|X_{1j}R_j(X_1/\sqrt{k})|
\lesssim\frac{t\psi_j(x)\bar{B}}{\sqrt{k}}\mathbb{E}\left(h_j(X_1/\sqrt{k})^2-\frac{1}{k}\langle X_1,\nabla^2\log\psi_j(x)X_1\rangle+\frac{\bar{B}^2}{kpn}\right) \\
&=\frac{t\psi_j(x)\bar{B}}{k^{3/2}}\left(\langle\nabla\log\psi_j(x),\Sigma\nabla\log\psi_j(x)\rangle+\operatorname{tr}\left(\Sigma(-\nabla^2\log\psi_j(x))\right)+\frac{\bar{B}^2}{pn}\right),
\end{aligned} k 1 E ∣ X 1 j R j ( X 1 / k ) ∣ ≲ k t ψ j ( x ) B ˉ E ( h j ( X 1 / k ) 2 − k 1 ⟨ X 1 , ∇ 2 log ψ j ( x ) X 1 ⟩ + k p n B ˉ 2 ) = k 3/2 t ψ j ( x ) B ˉ ( ⟨ ∇ log ψ j ( x ) , Σ∇ log ψ j ( x )⟩ + tr ( Σ ( − ∇ 2 log ψ j ( x )) ) + p n B ˉ 2 ) ,
where we have used ∥ X 1 ∥ ∞ ≤ B ˉ \|X_1\|_\infty\leq\bar{B} ∥ X 1 ∥ ∞ ≤ B ˉ in the first step. By summing the terms on the right over regular j j j and then applying the facet function bounds (22) –(24) of Lemma 4.4 (which hold for all j ≤ p j\leq p j ≤ p ), we conclude that
1 k ∑ j r e g u l a r E ∣ X 1 j R j ( X 1 / k ) ∣ ≲ t B ˉ F ( x ) ( k u ) 3 / 2 . (29) \frac{1}{\sqrt{k}}\sum_{j\ \mathrm{regular}}\mathbb{E}|X_{1j}R_j(X_1/\sqrt{k})|\lesssim\frac{t\bar{B}\,F(x)}{(ku)^{3/2}}. \tag*{(29)} k 1 j regular ∑ E ∣ X 1 j R j ( X 1 / k ) ∣ ≲ ( k u ) 3/2 t B ˉ F ( x ) . (29)
where we have simplified the last bound by using B ˉ 2 / ( p n ) ≤ 1 \bar{B}^{2}/(pn) \le1 B ˉ 2 / ( p n ) ≤ 1 and u − 1 / 2 ≤ 2 u − 3 / 2 u^{-1/2} \le2u^{-3/2} u − 1/2 ≤ 2 u − 3/2 (as u ≤ 2 u \le2 u ≤ 2 ).
Remainder bound for non-regular facets. Let j j j be non-regular at x x x , and again let w ∈ R p w \in\mathbb{R}^{p} w ∈ R p be deterministic with ∥ w ∥ ∞ ≤ B ˉ / k \lVert w\rVert_{\infty} \le\bar{B}/\sqrt{k} ∥ w ∥ ∞ ≤ B ˉ / k . Since the definition of ψ j \psi_j ψ j implies the pointwise upper bound ψ j = ν j / σ 2 + u ≤ ν j / σ \psi_j = \nu_j/\sqrt{\sigma^2+u} \le\nu_j/\sigma ψ j = ν j / σ 2 + u ≤ ν j / σ , we may apply Lemma 4.8(b) to the first two terms in the formula (25) . Together with t ≤ 1 t \le1 t ≤ 1 , this gives
∣ R j ( w ) ∣ ≲ 1 σ ( p n ) 5 + ψ j ( x ) ∣ h j ( w ) ∣ . \lvert R_j(w)\rvert\lesssim\frac{1}{\sigma(pn)^5}+\psi_j(x)\lvert h_j(w)\rvert. ∣ R j ( w )∣ ≲ σ ( p n ) 5 1 + ψ j ( x ) ∣ h j ( w )∣ .
The second term on the right can be bounded by combining the formula (19) for ∂ w log ψ j ( x ) = h j ( w ) \partial_w \log\psi_j(x)=h_j(w) ∂ w log ψ j ( x ) = h j ( w ) with Lemma 4.5, and using log ( p n ) ∥ w ∥ ∞ / σ ≤ 1 \sqrt{\log(pn)}\lVert w\rVert_{\infty}/\sigma\le1 log ( p n ) ∥ w ∥ ∞ / σ ≤ 1 from (5) . Along with the pointwise bound ψ j ≤ ν j / σ \psi_j \le\nu_j/\sigma ψ j ≤ ν j / σ and Lemma 4.8(b), this gives
ψ j ( x ) ∣ h j ( w ) ∣ ≲ ν j ( x ) σ ( ∣ a j ( x ) ∣ + log 1 q j ( x ) + 1 ) ≲ 1 σ ( p n ) 5 . \psi_j(x)\lvert h_j(w)\rvert\lesssim\frac{\nu_j(x)}{\sigma}\left(\lvert a_j(x)\rvert+\log\frac{1}{q_j(x)}+1\right)\lesssim\frac{1}{\sigma(pn)^5}. ψ j ( x ) ∣ h j ( w )∣ ≲ σ ν j ( x ) ( ∣ a j ( x )∣ + log q j ( x ) 1 + 1 ) ≲ σ ( p n ) 5 1 .
In turn, using the last two displays with ∥ w ∥ ∞ ≤ B ˉ / k ≤ σ \lVert w\rVert_{\infty} \le\bar{B}/\sqrt{k} \le\sigma ∥ w ∥ ∞ ≤ B ˉ / k ≤ σ leads to
∣ w j R j ( w ) ∣ ≲ 1 ( p n ) 5 . \lvert w_jR_j(w)\rvert\lesssim\frac{1}{(pn)^5}. ∣ w j R j ( w )∣ ≲ ( p n ) 5 1 .
Since this holds for every such w w w , it holds in particular for w = X 1 / k w=X_1/\sqrt{k} w = X 1 / k . Taking the expectation over X 1 X_1 X 1 , and summing over the set of (at most p p p ) non-regular indices j j j , we have
1 k ∑ j n o t r e g u l a r E ∣ X 1 j R j ( X 1 / k ) ∣ ≲ 1 ( p n ) 4 . \frac{1}{\sqrt{k}}\sum_{j\ \mathrm{not\ regular}}\mathbb{E}\lvert X_{1j}R_j(X_1/\sqrt{k})\rvert\lesssim\frac{1}{(pn)^4}. k 1 j not regular ∑ E ∣ X 1 j R j ( X 1 / k )∣ ≲ ( p n ) 4 1 .
Combining this with the bound (29) for the regular facets and the expression (27) for the main term, the proof of Proposition 3.4 is complete.
Proof of Proposition 4.7: control of curvature
The curvature of the log-facet functions log ψ j \log\psi_j log ψ j appearing in the statement of Proposition 4.7 is difficult to bound in a direct manner. This section reframes the curvature in terms of the log-partition function of a particular Gibbs measure, which is introduced in Section 5.1. The benefit of this reframing is that the curvature can be expressed in terms of a variance with respect to the Gibbs measure, and well-developed tools are available for analyzing such variances. Throughout this section, the quantities u u u , k k k , and σ = C 0 B ˉ ( log ( p n ) ) 3 / 2 / k \sigma=C_0\bar{B}(\log(pn))^{3/2}/\sqrt{k} σ = C 0 B ˉ ( log ( p n ) ) 3/2 / k will remain fixed as in Section 3, and so they will be suppressed in much of the notation.
Reframing in terms of log-partition functions of Gibbs measures
Let a = ( a 1 , … , a p − 1 ) \mathbf{a}=(a_1,\ldots,a_{p-1}) a = ( a 1 , … , a p − 1 ) be a tuple of real-valued affine functions on R p − 1 × R \mathbb{R}^{p-1}\times\mathbb{R} R p − 1 × R . That is, for each l = 1 , … , p − 1 l=1,\ldots,p-1 l = 1 , … , p − 1 , the function a l ( z , τ ) a_l(z,\tau) a l ( z , τ ) is jointly affine in z ∈ R p − 1 z\in\mathbb{R}^{p-1} z ∈ R p − 1 and τ ∈ R \tau\in\mathbb{R} τ ∈ R . For a given tuple a \mathbf{a} a , consider the associated function
ℓ τ ( z ) = − ∑ l = 1 p − 1 log Φ ( a l ( z , τ ) σ ) , (30) \ell_\tau(z)=-\sum_{l=1}^{p-1}\log\Phi\left(\frac{a_l(z,\tau)}{\sigma}\right), \tag*{(30)} ℓ τ ( z ) = − l = 1 ∑ p − 1 log Φ ( σ a l ( z , τ ) ) , (30)
which is non-negative and convex. As such, this function induces a Gibbs measure μ τ \mu_\tau μ τ whose density with respect to γ p − 1 \gamma_{p-1} γ p − 1 on R p − 1 \mathbb{R}^{p-1} R p − 1 is defined by
d μ τ ( z ) = e − ℓ τ ( z ) − A ( τ ) d γ p − 1 ( z ) , (31) d\mu_\tau(z)=e^{-\ell_\tau(z)-A(\tau)}d\gamma_{p-1}(z), \tag*{(31)} d μ τ ( z ) = e − ℓ τ ( z ) − A ( τ ) d γ p − 1 ( z ) , (31)
where A ( τ ) A(\tau) A ( τ ) is the log-partition function . More explicitly, the function A ( τ ) A(\tau) A ( τ ) is given by
A ( τ ) = log ∫ R p − 1 ∏ l = 1 p − 1 Φ ( a l ( z , τ ) σ ) d γ p − 1 ( z ) . (32) A(\tau)=\log\int_{\mathbb{R}^{p-1}}\prod_{l=1}^{p-1}\Phi\left(\frac{a_l(z,\tau)}{\sigma}\right)\,d\gamma_{p-1}(z). \tag*{(32)} A ( τ ) = log ∫ R p − 1 l = 1 ∏ p − 1 Φ ( σ a l ( z , τ ) ) d γ p − 1 ( z ) . (32)
Below, we will usually suppress the fact that μ τ \mu_\tau μ τ and A A A depend on a \mathsf{a} a , and will simply refer to them as the Gibbs measure and log-partition function associated with a \mathsf{a} a . The close connection between log-facet functions and log-partition functions is established in Lemma 5.1 below, which is proved in § 5.7 . As a clarification about notation, the gradient ∇ a l \nabla a_l ∇ a l below is understood with respect to the z z z argument of a l ( z , τ ) a_l(z,\tau) a l ( z , τ ) , and it should also be noted that the vectors ∇ a l \nabla a_l ∇ a l and ∂ τ a = ( ∂ τ a 1 , … , ∂ τ a p − 1 ) \partial_\tau\mathsf{a}=(\partial_\tau a_1,\ldots,\partial_\tau a_{p-1}) ∂ τ a = ( ∂ τ a 1 , … , ∂ τ a p − 1 ) are constant with respect to z z z and τ \tau τ .
Lemma 5.1. For any fixed u ∈ ( 0 , 2 ] u\in(0,2] u ∈ ( 0 , 2 ] , j ≤ p j\le p j ≤ p and y , w ∈ R p y,w\in\mathbb{R}^p y , w ∈ R p , there is a tuple of affine functions a = ( a 1 , … , a p − 1 ) \mathsf{a}=(a_1,\ldots,a_{p-1}) a = ( a 1 , … , a p − 1 ) on R p − 1 × R \mathbb{R}^{p-1}\times\mathbb{R} R p − 1 × R with an associated log-partition function A A A such that
∂ τ 2 log ϕ j ( y + τ w ) = − w j 2 u + σ 2 + A ′ ′ ( τ ) for all τ ∈ R . (33) \partial_\tau^2\log\phi_j(y+\tau w)=-\frac{w_j^2}{u+\sigma^2}+A''(\tau)\qquad\text{for all }\tau\in\mathbb{R}. \tag*{(33)} ∂ τ 2 log ϕ j ( y + τ w ) = − u + σ 2 w j 2 + A ′′ ( τ ) for all τ ∈ R . (33)
Furthermore, the tuple a \mathsf{a} a and function A A A satisfy ∥ ∂ τ a ∥ ∞ ≤ 2 ∥ w ∥ ∞ \|\partial_\tau\mathsf{a}\|_\infty\le2\|w\|_\infty ∥ ∂ τ a ∥ ∞ ≤ 2∥ w ∥ ∞ , max l ≤ p − 1 ∥ ∇ a l ∥ 2 ≤ 2 \max_{l\le p-1}\|\nabla a_l\|_2\le2 max l ≤ p − 1 ∥∇ a l ∥ 2 ≤ 2 , and A ( 0 ) = log q j ( y ) A(0)=\log q_j(y) A ( 0 ) = log q j ( y ) .
Based on this lemma, it is straightforward to check that Proposition 4.7 is implied by the following analogous statement involving the log-partition function, which is proven in § 5.3 .
Proposition 5.2. Let a = ( a 1 , … , a p − 1 ) \mathsf{a}=(a_1,\ldots,a_{p-1}) a = ( a 1 , … , a p − 1 ) be a tuple of affine functions on R p − 1 × R \mathbb{R}^{p-1}\times\mathbb{R} R p − 1 × R with associated log-partition function A A A . Also, let n ≥ 2 n\ge2 n ≥ 2 be an integer with p n pn p n sufficiently large. If ∥ ∂ τ a ∥ ∞ ≤ 2 B ‾ / k \|\partial_\tau\mathsf{a}\|_\infty\le2\overline{B}/\sqrt{k} ∥ ∂ τ a ∥ ∞ ≤ 2 B / k , max l ≤ p − 1 ∥ ∇ a l ∥ 2 ≤ 2 \max_{l\le p-1}\|\nabla a_l\|_2\le2 max l ≤ p − 1 ∥∇ a l ∥ 2 ≤ 2 , and A ( 0 ) ≥ − 10 log ( p n ) A(0)\ge-10\log(pn) A ( 0 ) ≥ − 10 log ( p n ) , then
− A ′ ′ ( τ ) ≲ − A ′ ′ ( 0 ) + B ‾ 2 k p n for all τ ∈ [ 0 , 1 ] . -A''(\tau)\lesssim-A''(0)+\frac{\overline{B}^2}{kpn}\qquad\text{for all }\tau\in[0,1]. − A ′′ ( τ ) ≲ − A ′′ ( 0 ) + k p n B 2 for all τ ∈ [ 0 , 1 ] .
Variational representation of − A ′ ′ ( τ ) -A''(\tau) − A ′′ ( τ )
The main goal of this subsection is to develop a formula showing that for a fixed τ ∈ R \tau\in\mathbb{R} τ ∈ R , the negative curvature − A ′ ′ ( τ ) -A''(\tau) − A ′′ ( τ ) can be represented as the minimum value of a functional of the form h ↦ E μ τ J τ h h\mapsto\mathbb{E}_{\mu_\tau}J_\tau h h ↦ E μ τ J τ h , where J τ J_\tau J τ is a certain differential operator and h h h ranges over a certain set of test functions (see (37) and (36) ). This representation of − A ′ ′ ( τ ) -A''(\tau) − A ′′ ( τ ) is useful, because a minimum value lends itself to deriving an upper bound on − A ′ ′ ( τ ) -A''(\tau) − A ′′ ( τ ) that is needed in Proposition 5.2, and because the functional will have a well-behaved dependence on τ \tau τ (which facilitates the comparison of A ′ ′ ( τ ) A''(\tau) A ′′ ( τ ) and A ′ ′ ( 0 ) A''(0) A ′′ ( 0 ) ). Throughout the rest of this section, a = ( a 1 , … , a p − 1 ) \mathsf{a}=(a_1,\ldots,a_{p-1}) a = ( a 1 , … , a p − 1 ) is a tuple of affine functions on R p − 1 × R \mathbb{R}^{p-1}\times\mathbb{R} R p − 1 × R , with associated function ℓ τ \ell_\tau ℓ τ , Gibbs measure μ τ \mu_\tau μ τ and log-partition function A A A as in § 5.1 .
The curvature of A ( τ ) A(\tau) A ( τ ) can be calculated by differentiating the identity e A ( τ ) = ∫ e − ℓ τ d γ p − 1 e^{A(\tau)}=\int e^{-\ell_\tau}\,d\gamma_{p-1} e A ( τ ) = ∫ e − ℓ τ d γ p − 1 twice, leading to
− A ′ ′ ( τ ) = E μ τ ∂ τ 2 ℓ τ − Var μ τ ( ℓ ˙ τ ) , (34) -A''(\tau)=\mathbb{E}_{\mu_\tau}\partial_\tau^2\ell_\tau-\operatorname{Var}_{\mu_\tau}(\dot{\ell}_\tau), \tag*{(34)} − A ′′ ( τ ) = E μ τ ∂ τ 2 ℓ τ − Var μ τ ( ℓ ˙ τ ) , (34)
where we use the notation ℓ ˙ τ = ∂ τ ℓ τ \dot{\ell}_\tau=\partial_\tau\ell_\tau ℓ ˙ τ = ∂ τ ℓ τ . When the density of μ τ \mu_\tau μ τ is computed with respect to Lebesgue measure on R p − 1 \mathbb{R}^{p-1} R p − 1 (rather than γ p − 1 \gamma_{p-1} γ p − 1 ), it is proportional to e − W τ e^{-W_\tau} e − W τ , where the function
W τ ( z ) = 1 2 ∥ z ∥ 2 2 + ℓ τ ( z ) W_\tau(z)=\frac{1}{2}\|z\|_2^2+\ell_\tau(z) W τ ( z ) = 2 1 ∥ z ∥ 2 2 + ℓ τ ( z )
is jointly convex in ( z , τ ) (z,\tau) ( z , τ ) , by the convexity of ℓ τ \ell_\tau ℓ τ . Another related classical fact is that μ τ \mu_\tau μ τ is the invariant measure of the Langevin diffusion (see, e.g., Pavliotis, [37 ] ; Bakry, Gentil and Ledoux, [3 ] ), whose generator is given by the following operator L τ L_\tau L τ defined in terms of W τ W_\tau W τ ,
L τ h = Δ h − ⟨ ∇ W τ , ∇ h ⟩ , L_\tau h = \Delta h - \langle\nabla W_\tau,\nabla h\rangle, L τ h = Δ h − ⟨ ∇ W τ , ∇ h ⟩ ,
where h h h is a suitable test function on R p − 1 \mathbb{R}^{p-1} R p − 1 . This operator and the function W τ W_\tau W τ are needed to obtain the variational representation for Var μ τ ( ℓ ˙ τ ) \operatorname{Var}_{\mu_\tau}(\dot{\ell}_\tau) Var μ τ ( ℓ ˙ τ ) in the lemma below, which is proved in Section 5.6. As a matter of notation, we use H \mathcal{H} H to denote the class of functions h ∈ C 3 ( R p − 1 ) h \in C^3(\mathbb{R}^{p-1}) h ∈ C 3 ( R p − 1 ) with bounded first and second derivatives.
Lemma 5.3. For any τ ∈ R \tau\in\mathbb{R} τ ∈ R and any h ∈ H h \in\mathcal{H} h ∈ H , we have
Var μ τ ( ℓ ˙ τ ) = E μ τ ( 2 ⟨ ∇ ℓ ˙ τ , ∇ h ⟩ − ⟨ ∇ h , ∇ 2 W τ ∇ h ⟩ − ∥ ∇ 2 h ∥ F 2 ) + E μ τ [ ( ℓ ˙ τ − E μ τ ℓ ˙ τ + L τ h ) 2 ] . (35) \operatorname{Var}_{\mu_\tau}(\dot{\ell}_\tau)
= \mathbb{E}_{\mu_\tau}\left(2\langle\nabla\dot{\ell}_\tau,\nabla h\rangle-\langle\nabla h,\nabla^2 W_\tau\nabla h\rangle-\|\nabla^2 h\|_F^2\right)
+\mathbb{E}_{\mu_\tau}\left[(\dot{\ell}_\tau-\mathbb{E}_{\mu_\tau}\dot{\ell}_\tau+L_\tau h)^2\right]. \tag*{(35)} Var μ τ ( ℓ ˙ τ ) = E μ τ ( 2 ⟨ ∇ ℓ ˙ τ , ∇ h ⟩ − ⟨ ∇ h , ∇ 2 W τ ∇ h ⟩ − ∥ ∇ 2 h ∥ F 2 ) + E μ τ [ ( ℓ ˙ τ − E μ τ ℓ ˙ τ + L τ h ) 2 ] . (35) In particular, Var μ τ ( ℓ ˙ τ ) \operatorname{Var}_{\mu_\tau}(\dot{\ell}_\tau) Var μ τ ( ℓ ˙ τ ) is lower bounded by the first term on the right, with equality holding when h h h solves the equation L τ h = E μ τ ℓ ˙ τ − ℓ ˙ τ L_\tau h=\mathbb{E}_{\mu_\tau}\dot{\ell}_\tau-\dot{\ell}_\tau L τ h = E μ τ ℓ ˙ τ − ℓ ˙ τ .
The formula (35) may be regarded as a variant of the Helffer–Sjöstrand representation for covariances computed with respect to a Gibbs measure (Helffer and Sjöstrand, [26 ] , p.352). This formula and (34) motivate defining the operator
J τ h = ∂ τ 2 ℓ τ − 2 ⟨ ∇ ℓ ˙ τ , ∇ h ⟩ + ⟨ ∇ h , ∇ 2 W τ ∇ h ⟩ + ∥ ∇ 2 h ∥ F 2 , (36) J_\tau h=\partial_\tau^2\ell_\tau-2\langle\nabla\dot{\ell}_\tau,\nabla h\rangle+\langle\nabla h,\nabla^2W_\tau\nabla h\rangle+\|\nabla^2h\|_F^2, \tag*{(36)} J τ h = ∂ τ 2 ℓ τ − 2 ⟨ ∇ ℓ ˙ τ , ∇ h ⟩ + ⟨ ∇ h , ∇ 2 W τ ∇ h ⟩ + ∥ ∇ 2 h ∥ F 2 , (36)
which allows − A ′ ′ ( τ ) -A''(\tau) − A ′′ ( τ ) to be expressed as
− A ′ ′ ( τ ) = E μ τ J τ h − E μ τ [ ( ℓ ˙ τ − E μ τ ℓ ˙ τ + L τ h ) 2 ] (37) -A''(\tau)=\mathbb{E}_{\mu_\tau}J_\tau h-\mathbb{E}_{\mu_\tau}\left[(\dot{\ell}_\tau-\mathbb{E}_{\mu_\tau}\dot{\ell}_\tau+L_\tau h)^2\right] \tag*{(37)} − A ′′ ( τ ) = E μ τ J τ h − E μ τ [ ( ℓ ˙ τ − E μ τ ℓ ˙ τ + L τ h ) 2 ] (37)
for every τ ∈ R \tau\in\mathbb{R} τ ∈ R and every h ∈ H h\in\mathcal{H} h ∈ H . This formula for − A ′ ′ ( τ ) -A''(\tau) − A ′′ ( τ ) now puts us in position to prove Proposition 5.2.
Proof of Proposition 5.2
The variational representation (37) shows that the upper bound
− A ′ ′ ( τ ) ≤ E μ τ J τ h (38) -A''(\tau)\leq\mathbb{E}_{\mu_\tau}J_\tau h \tag*{(38)} − A ′′ ( τ ) ≤ E μ τ J τ h (38)
holds for every h ∈ H h\in\mathcal{H} h ∈ H and τ ∈ [ 0 , 1 ] \tau\in[0,1] τ ∈ [ 0 , 1 ] . It also shows that if h ⋆ ∈ H h^\star\in\mathcal{H} h ⋆ ∈ H solves the equation L 0 h ⋆ = E μ 0 ℓ ˙ 0 − ℓ ˙ 0 L_0h^\star=\mathbb{E}_{\mu_0}\dot{\ell}_0-\dot{\ell}_0 L 0 h ⋆ = E μ 0 ℓ ˙ 0 − ℓ ˙ 0 , then we have the equality
− A ′ ′ ( 0 ) = E μ 0 J 0 h ⋆ . -A''(0)=\mathbb{E}_{\mu_0}J_0h^\star. − A ′′ ( 0 ) = E μ 0 J 0 h ⋆ .
(The fact that such a solution h ⋆ h^\star h ⋆ exists is ensured by Lemma 5.7 later.) To proceed, we take the test function in (38) to be h ⋆ h^\star h ⋆ , which gives
− A ′ ′ ( τ ) ≤ E μ τ J τ h ⋆ , -A''(\tau)\leq\mathbb{E}_{\mu_\tau}J_\tau h^\star, − A ′′ ( τ ) ≤ E μ τ J τ h ⋆ ,
for all τ ∈ [ 0 , 1 ] \tau\in[0,1] τ ∈ [ 0 , 1 ] . Thus, to prove Proposition 5.2, it remains to relate E μ τ J τ h ⋆ \mathbb{E}_{\mu_\tau}J_\tau h^\star E μ τ J τ h ⋆ to E μ 0 J 0 h ⋆ \mathbb{E}_{\mu_0}J_0h^\star E μ 0 J 0 h ⋆ . We do this in two steps, by replacing J τ J_\tau J τ with J 0 J_0 J 0 , and then replacing μ τ \mu_\tau μ τ with μ 0 \mu_0 μ 0 , while keeping track of the errors introduced by these substitutions. First, by choosing the test function in Lemma 5.5 to be h ⋆ h^\star h ⋆ , it follows that
E μ τ J τ h ⋆ ≤ 2 E μ τ J 0 h ⋆ + 2 ∥ ∂ τ a ∥ ∞ 2 ( p n ) 4 . (39) \mathbb{E}_{\mu_\tau}J_\tau h^\star\leq2\mathbb{E}_{\mu_\tau}J_0h^\star+\frac{2\|\partial_\tau a\|_\infty^2}{(pn)^4}. \tag*{(39)} E μ τ J τ h ⋆ ≤ 2 E μ τ J 0 h ⋆ + ( p n ) 4 2∥ ∂ τ a ∥ ∞ 2 . (39)
Next, it is shown in Lemma 5.6 that
E μ τ J 0 h ⋆ ≲ E μ 0 J 0 h ⋆ + ∥ J 0 h ⋆ ∥ s u p ( p n ) 10 . (40) \mathbb{E}_{\mu_\tau}J_0h^\star\lesssim\mathbb{E}_{\mu_0}J_0h^\star+\frac{\lVert J_0h^\star\rVert_{\mathrm{sup}}}{(pn)^{10}}. \tag*{(40)} E μ τ J 0 h ⋆ ≲ E μ 0 J 0 h ⋆ + ( p n ) 10 ∥ J 0 h ⋆ ∥ sup . (40)
Putting the last few pieces together gives
− A ′ ′ ( τ ) ≲ − A ′ ′ ( 0 ) + ∥ J 0 h ⋆ ∥ s u p ( p n ) 10 + ∥ ∂ τ a ∥ ∞ 2 ( p n ) 4 . -A^{\prime\prime}(\tau)\lesssim-A^{\prime\prime}(0)+\frac{\lVert J_0h^\star\rVert_{\mathrm{sup}}}{(pn)^{10}}+\frac{\lVert\partial_\tau a\rVert_\infty^2}{(pn)^4}. − A ′′ ( τ ) ≲ − A ′′ ( 0 ) + ( p n ) 10 ∥ J 0 h ⋆ ∥ sup + ( p n ) 4 ∥ ∂ τ a ∥ ∞ 2 .
In Lemma 5.7, it will be shown that the function h ⋆ h^\star h ⋆ can be taken to satisfy ∥ J 0 h ⋆ ∥ s u p ≲ ( p n ) 5 ∥ ∂ τ a ∥ ∞ 2 \lVert J_0h^\star\rVert_{\mathrm{sup}}\lesssim(pn)^5\lVert\partial_\tau a\rVert_\infty^2 ∥ J 0 h ⋆ ∥ sup ≲ ( p n ) 5 ∥ ∂ τ a ∥ ∞ 2 . Finally, applying the assumption ∥ ∂ τ a ∥ ∞ ≤ 2 B ‾ / k \lVert\partial_\tau a\rVert_\infty\le2\overline{B}/\sqrt{k} ∥ ∂ τ a ∥ ∞ ≤ 2 B / k completes the proof of Proposition 5.2.
The dependence of J on the parameter
This subsection will establish the upper bound {(5.10)} in Lemma 5.5, which relies on the explicit formula for J τ J_\tau J τ below. In particular, the formula shows that the dependence of J τ h J_\tau h J τ h on τ \tau τ is essentially governed by the function ρ : = − ( log Φ ) ′ ′ \rho:=-(\log\Phi)^{\prime\prime} ρ := − ( log Φ ) ′′ , which has well understood properties.
Lemma 5.4. For every tuple a a a of affine functions on R p − 1 × R \mathbb{R}^{p-1}\times\mathbb{R} R p − 1 × R , every τ ∈ R \tau\in\mathbb{R} τ ∈ R , and every function h ∈ C 2 ( R p − 1 ) h\in C^2(\mathbb{R}^{p-1}) h ∈ C 2 ( R p − 1 ) , we have
J τ h = ∥ ∇ h ∥ 2 2 + ∥ ∇ 2 h ∥ F 2 + ∑ l = 1 p − 1 ρ ( a l ( ⋅ , τ ) σ ) ( ∂ τ a l − ⟨ ∇ a l , ∇ h ⟩ ) 2 σ 2 . (41) J_\tau h=\lVert\nabla h\rVert_2^2+\lVert\nabla^2h\rVert_F^2+\sum_{l=1}^{p-1}\rho\left(\frac{a_l(\cdot,\tau)}{\sigma}\right)\frac{\left(\partial_\tau a_l-\langle\nabla a_l,\nabla h\rangle\right)^2}{\sigma^2}. \tag*{(41)} J τ h = ∥ ∇ h ∥ 2 2 + ∥ ∇ 2 h ∥ F 2 + l = 1 ∑ p − 1 ρ ( σ a l ( ⋅ , τ ) ) σ 2 ( ∂ τ a l − ⟨ ∇ a l , ∇ h ⟩ ) 2 . (41) Furthermore, J τ h ≥ 0 J_\tau h\ge0 J τ h ≥ 0 .
Proof. From looking at the definition of J τ J_\tau J τ in (36) and noting that ∇ 2 W τ = I + ∇ 2 ℓ τ \nabla^2W_\tau=I+\nabla^2\ell_\tau ∇ 2 W τ = I + ∇ 2 ℓ τ , we need to calculate ∂ τ 2 ℓ τ \partial_\tau^2\ell_\tau ∂ τ 2 ℓ τ , ∇ ℓ ˙ τ \nabla\dot{\ell}_\tau ∇ ℓ ˙ τ and ∇ 2 ℓ τ \nabla^2\ell_\tau ∇ 2 ℓ τ . Direct calculations with ℓ τ \ell_\tau ℓ τ give
∂ τ 2 ℓ τ = ∑ l = 1 p − 1 ρ ( a l ( ⋅ , τ ) σ ) ( ∂ τ a l ) 2 σ 2 , \partial_\tau^2\ell_\tau=\sum_{l=1}^{p-1}\rho\left(\frac{a_l(\cdot,\tau)}{\sigma}\right)\frac{(\partial_\tau a_l)^2}{\sigma^2}, ∂ τ 2 ℓ τ = l = 1 ∑ p − 1 ρ ( σ a l ( ⋅ , τ ) ) σ 2 ( ∂ τ a l ) 2 ,
∇ ℓ ˙ τ = ∑ l = 1 p − 1 ρ ( a l ( ⋅ , τ ) σ ) ( ∂ τ a l ) ∇ a l σ 2 , (42) \nabla\dot{\ell}_\tau=\sum_{l=1}^{p-1}\rho\left(\frac{a_l(\cdot,\tau)}{\sigma}\right)\frac{(\partial_\tau a_l)\nabla a_l}{\sigma^2}, \tag*{(42)} ∇ ℓ ˙ τ = l = 1 ∑ p − 1 ρ ( σ a l ( ⋅ , τ ) ) σ 2 ( ∂ τ a l ) ∇ a l , (42)
∇ 2 ℓ τ = ∑ l = 1 p − 1 ρ ( a l ( ⋅ , τ ) σ ) ∇ a l ∇ a l T σ 2 . \nabla^2\ell_\tau=\sum_{l=1}^{p-1}\rho\left(\frac{a_l(\cdot,\tau)}{\sigma}\right)\frac{\nabla a_l\nabla a_l^{\mathsf T}}{\sigma^2}. ∇ 2 ℓ τ = l = 1 ∑ p − 1 ρ ( σ a l ( ⋅ , τ ) ) σ 2 ∇ a l ∇ a l T .
Substituting these formulas into (36) and completing the square gives (41) . The non-negativity of J τ h J_\tau h J τ h follows from that of ρ \rho ρ , i.e. log-concavity of Φ \Phi Φ .
The next lemma compares J τ J_\tau J τ with J 0 J_0 J 0 .
Lemma 5.5. Let a a a be a tuple of affine functions on R p − 1 × R \mathbb{R}^{p-1}\times\mathbb{R} R p − 1 × R , and let p n pn p n be sufficiently large. If max l ≤ p − 1 ∥ ∇ a l ∥ 2 ≤ 2 \max_{l\le p-1}\lVert\nabla a_l\rVert_2\le2 max l ≤ p − 1 ∥ ∇ a l ∥ 2 ≤ 2 and ∥ ∂ τ a ∥ ∞ ≤ 2 B ‾ / k \lVert\partial_\tau a\rVert_\infty\le2\overline{B}/\sqrt{k} ∥ ∂ τ a ∥ ∞ ≤ 2 B / k , then for every τ ∈ [ 0 , 1 ] \tau\in[0,1] τ ∈ [ 0 , 1 ] , and every h ∈ C 2 ( R p − 1 ) h\in C^2(\mathbb{R}^{p-1}) h ∈ C 2 ( R p − 1 ) , the following inequality holds pointwise on R p − 1 \mathbb{R}^{p-1} R p − 1 ,
J τ h ≤ 2 J 0 h + 2 ∥ ∂ τ a ∥ ∞ 2 ( p n ) − 4 . J_\tau h\le2J_0h+2\lVert\partial_\tau a\rVert_\infty^2(pn)^{-4}. J τ h ≤ 2 J 0 h + 2 ∥ ∂ τ a ∥ ∞ 2 ( p n ) − 4 .
Proof. Because the functions a l a_l a l in the formula (41) are affine, the derivatives ∂ τ a l \partial_\tau a_l ∂ τ a l and ∇ a l \nabla a_l ∇ a l do not depend on τ \tau τ . Thus, a comparison of J τ h J_\tau h J τ h and J 0 h J_0h J 0 h reduces to comparing ρ ( a l ( z , τ ) / σ ) \rho(a_l(z,\tau)/\sigma) ρ ( a l ( z , τ ) / σ ) and ρ ( a l ( z , 0 ) / σ ) \rho(a_l(z,0)/\sigma) ρ ( a l ( z , 0 ) / σ ) at any fixed z ∈ R p − 1 z \in\mathbb{R}^{p-1} z ∈ R p − 1 . In Lemma 8.4 (c) it is shown that
ρ ( s ~ ) ≤ 2 ρ ( s ) + ( p n ) − 5 \rho(\widetilde{s}) \le2\rho(s) + (pn)^{-5} ρ ( s ) ≤ 2 ρ ( s ) + ( p n ) − 5
when s , s ~ ∈ R s,\widetilde{s} \in\mathbb{R} s , s ∈ R satisfy ∣ s ~ − s ∣ ≤ 2 ( log ( p n ) ) − 3 / 2 |\widetilde{s}-s| \le2(\log(pn))^{-3/2} ∣ s − s ∣ ≤ 2 ( log ( p n ) ) − 3/2 and p n pn p n is sufficiently large. Here, we will take s ~ = a l ( z , τ ) / σ \widetilde{s}=a_l(z,\tau)/\sigma s = a l ( z , τ ) / σ and s = a l ( z , 0 ) / σ s=a_l(z,0)/\sigma s = a l ( z , 0 ) / σ , and it can be checked that ∣ s ~ − s ∣ ≤ τ ∥ ∂ τ a ∥ ∞ / σ ≤ 2 ( log ( p n ) ) − 3 / 2 |\widetilde{s}-s| \le\tau\|\partial_\tau a\|_\infty/\sigma\le2(\log(pn))^{-3/2} ∣ s − s ∣ ≤ τ ∥ ∂ τ a ∥ ∞ / σ ≤ 2 ( log ( p n ) ) − 3/2 , due to the assumption on a a a , and the basic relation (5) . To proceed, we insert the bound on ρ ( s ~ ) \rho(\widetilde{s}) ρ ( s ) into (41) and compare with 2 J 0 h ( z ) 2J_0h(z) 2 J 0 h ( z ) , to obtain
J τ h ( z ) ≤ 2 J 0 h ( z ) − ∥ ∇ h ( z ) ∥ 2 2 + ( p n ) − 5 ∑ l = 1 p − 1 v l , (43) J_\tau h(z) \le2J_0h(z) - \|\nabla h(z)\|_2^2 + (pn)^{-5}\sum_{l=1}^{p-1}v_l, \tag*{(43)} J τ h ( z ) ≤ 2 J 0 h ( z ) − ∥∇ h ( z ) ∥ 2 2 + ( p n ) − 5 l = 1 ∑ p − 1 v l , (43)
where we put v l = ( ∂ τ a l − ⟨ ∇ a l , ∇ h ( z ) ⟩ ) 2 / σ 2 v_l=(\partial_\tau a_l-\langle\nabla a_l,\nabla h(z)\rangle)^2/\sigma^2 v l = ( ∂ τ a l − ⟨ ∇ a l , ∇ h ( z )⟩ ) 2 / σ 2 . Furthermore, we have v l ≤ ( 2 ∥ ∂ τ a ∥ ∞ 2 + 8 ∥ ∇ h ( z ) ∥ 2 2 ) n v_l\le(2\|\partial_\tau a\|_\infty^2+8\|\nabla h(z)\|_2^2)n v l ≤ ( 2∥ ∂ τ a ∥ ∞ 2 + 8∥∇ h ( z ) ∥ 2 2 ) n , because ∥ ∇ a l ∥ 2 ≤ 2 \|\nabla a_l\|_2\le2 ∥∇ a l ∥ 2 ≤ 2 and σ − 2 ≤ n \sigma^{-2}\le n σ − 2 ≤ n , and so the bound (43) leads to the stated result. □
Changing the measure to 0
This subsection establishes the bound (40) in the following lemma.
Lemma 5.6. Let a a a be a tuple of affine functions on R p − 1 × R \mathbb{R}^{p-1}\times\mathbb{R} R p − 1 × R , and let p n pn p n be sufficiently large. Suppose that max l ≤ p − 1 ∥ ∇ a l ∥ 2 ≤ 2 \max_{l\le p-1}\|\nabla a_l\|_2\le2 max l ≤ p − 1 ∥∇ a l ∥ 2 ≤ 2 , ∥ ∂ τ a ∥ ∞ ≤ 2 B ‾ / k \|\partial_\tau a\|_\infty\le2\overline{B}/\sqrt{k} ∥ ∂ τ a ∥ ∞ ≤ 2 B / k and A ( 0 ) ≥ − 10 log ( p n ) A(0)\ge-10\log(pn) A ( 0 ) ≥ − 10 log ( p n ) , and let h ⋆ ∈ H h^\star\in\mathcal{H} h ⋆ ∈ H be the function given by Lemma 5.7. Then, for every τ ∈ [ 0 , 1 ] \tau\in[0,1] τ ∈ [ 0 , 1 ] ,
E μ τ J 0 h ⋆ ≲ E μ 0 J 0 h ⋆ + ∥ J 0 h ⋆ ∥ s u p ( p n ) 10 . \mathbb{E}_{\mu_\tau}J_0h^\star\lesssim\mathbb{E}_{\mu_0}J_0h^\star+\frac{\|J_0h^\star\|_{\mathrm{sup}}}{(pn)^{10}}. E μ τ J 0 h ⋆ ≲ E μ 0 J 0 h ⋆ + ( p n ) 10 ∥ J 0 h ⋆ ∥ sup .
Proof. The proof is based on the formula
d μ τ d μ 0 = e ℓ 0 − ℓ τ + A ( 0 ) − A ( τ ) , (44) \frac{d\mu_\tau}{d\mu_0}=e^{\ell_0-\ell_\tau+A(0)-A(\tau)}, \tag*{(44)} d μ 0 d μ τ = e ℓ 0 − ℓ τ + A ( 0 ) − A ( τ ) , (44)
which follows from the definition d μ r = e − ℓ r − A ( r ) d γ p − 1 d\mu_r=e^{-\ell_r-A(r)}d\gamma_{p-1} d μ r = e − ℓ r − A ( r ) d γ p − 1 for r ∈ R r\in\mathbb{R} r ∈ R . We will show that the exponent in (44) is at most a constant multiple of ε ℓ 0 + 1 \varepsilon\ell_0+1 ε ℓ 0 + 1 , where we put ε = 1 / log ( p n ) \varepsilon=1/\log(pn) ε = 1/ log ( p n ) throughout the current proof. Consequently, the density d μ τ / d μ 0 d\mu_\tau/d\mu_0 d μ τ / d μ 0 is bounded by a constant, except on a set where ℓ 0 \ell_0 ℓ 0 is large, and the assumption A ( 0 ) ≥ − 10 log ( p n ) A(0)\ge-10\log(pn) A ( 0 ) ≥ − 10 log ( p n ) will turn out to make the μ 0 \mu_0 μ 0 -probability of that set negligible. To bound the exponent in (44) , we will proceed in two stages by comparing ℓ τ \ell_\tau ℓ τ with ℓ 0 \ell_0 ℓ 0 , and then comparing A ( τ ) A(\tau) A ( τ ) with A ( 0 ) A(0) A ( 0 ) .
Comparison of ℓ τ \ell_\tau ℓ τ and $\ell_0. Our aim in this stage is to show that
∣ ℓ τ − ℓ 0 ∣ ≲ ε ℓ 0 + 1 , (45) |\ell_\tau-\ell_0|\lesssim\varepsilon\ell_0+1, \tag*{(45)} ∣ ℓ τ − ℓ 0 ∣ ≲ ε ℓ 0 + 1 , (45)
and we will do this by developing a suitable bound on the derivative ℓ ˙ r = ∂ r ℓ r \dot{\ell}_r=\partial_r\ell_r ℓ ˙ r = ∂ r ℓ r that holds for any affine tuple a a a . Differentiating the formula for ℓ r \ell_r ℓ r in (30) gives
ℓ ˙ r = − ∑ l = 1 p − 1 λ ( a l ( ⋅ , r ) / σ ) ∂ τ a l σ , \dot{\ell}_r=-\sum_{l=1}^{p-1}\lambda(a_l(\mathord{\cdot},r)/\sigma)\frac{\partial_\tau a_l}{\sigma}, ℓ ˙ r = − l = 1 ∑ p − 1 λ ( a l ( ⋅ , r ) / σ ) σ ∂ τ a l ,
where we let λ = ( log Φ ) ′ = ϕ / Φ \lambda=(\log\Phi)'=\phi/\Phi λ = ( log Φ ) ′ = ϕ /Φ . The function λ \lambda λ is bounded pointwise in Lemma 8.4 (b) according to
λ ( s ) ≤ { − 2 ( 1 + s + ) log Φ ( s ) for all s ∈ R , 2 ϕ ( s ) for all s ≥ 0 , \lambda(s)\le
\begin{cases}
-2(1+s_+)\log\Phi(s) & \text{for all }s\in\mathbb{R},\\
2\phi(s) & \text{for all }s\ge0,
\end{cases} λ ( s ) ≤ { − 2 ( 1 + s + ) log Φ ( s ) 2 ϕ ( s ) for all s ∈ R , for all s ≥ 0 ,
where we write s + = max ( s , 0 ) s_{+}=\max(s,0) s + = max ( s , 0 ) . To lighten notation, we will also put s l = a l ( z , r ) / σ s_l=a_l(z,r)/\sigma s l = a l ( z , r ) / σ for fixed z ∈ R p − 1 z\in\mathbb{R}^{p-1} z ∈ R p − 1 and r ∈ R r\in\mathbb{R} r ∈ R . Applying the first bound when s l s_l s l is at most 2 log ( p n ) \sqrt{2\log(pn)} 2 log ( p n ) and applying the second bound when s l s_l s l exceeds 2 log ( p n ) \sqrt{2\log(pn)} 2 log ( p n ) , it follows that
∑ l = 1 p − 1 λ ( s l ) ≲ 1 + log ( p n ) ∑ l = 1 p − 1 ( − log Φ ( s l ) ) . \sum_{l=1}^{p-1}\lambda(s_l)\lesssim1+\sqrt{\log(pn)}\sum_{l=1}^{p-1}(-\log\Phi(s_l)). l = 1 ∑ p − 1 λ ( s l ) ≲ 1 + log ( p n ) l = 1 ∑ p − 1 ( − log Φ ( s l )) .
Noticing that the sum on the right is simply ℓ r ( z ) \ell_r(z) ℓ r ( z ) , combining the last few steps shows that for all r ∈ R r\in\mathbb{R} r ∈ R and z ∈ R p − 1 z\in\mathbb{R}^{p-1} z ∈ R p − 1 we have
∣ ℓ ˙ r ( z ) ∣ ≲ ∥ ∂ τ a ∥ ∞ σ ( log ( p n ) ℓ r ( z ) + 1 ) . (46) |\dot{\ell}_r(z)|\lesssim\frac{\|\partial_\tau a\|_\infty}{\sigma}\left(\sqrt{\log(pn)}\ell_r(z)+1\right). \tag*{(46)} ∣ ℓ ˙ r ( z ) ∣ ≲ σ ∥ ∂ τ a ∥ ∞ ( log ( p n ) ℓ r ( z ) + 1 ) . (46)
We now divide both sides by ℓ r + 1 \ell_r+1 ℓ r + 1 to get ∣ ∂ r log ( ℓ r + 1 ) ∣ ≲ log ( p n ) ∥ ∂ τ a ∥ ∞ / σ |\partial_r\log(\ell_r+1)|\lesssim\sqrt{\log(pn)}\|\partial_\tau a\|_\infty/\sigma ∣ ∂ r log ( ℓ r + 1 ) ∣ ≲ log ( p n ) ∥ ∂ τ a ∥ ∞ / σ and then integrate the latter bound over r ∈ [ 0 , τ ] r\in[0,\tau] r ∈ [ 0 , τ ] to obtain
∣ log ℓ τ + 1 ℓ 0 + 1 ∣ ≲ log ( p n ) τ ∥ ∂ τ a ∥ ∞ σ ≲ ε (47) \left|\log\frac{\ell_\tau+1}{\ell_0+1}\right|\lesssim\sqrt{\log(pn)}\frac{\tau\|\partial_\tau a\|_\infty}{\sigma}\lesssim\varepsilon \tag*{(47)} log ℓ 0 + 1 ℓ τ + 1 ≲ log ( p n ) σ τ ∥ ∂ τ a ∥ ∞ ≲ ε (47)
where we have used the upper bound on ∥ ∂ τ a ∥ ∞ \|\partial_\tau a\|_\infty ∥ ∂ τ a ∥ ∞ in the assumption, the basic relation (5) , and τ ≤ 1 \tau\leq1 τ ≤ 1 . Lastly, the inequality (47) leads to (45) by letting log ( ( ℓ τ + 1 ) / ( ℓ 0 + 1 ) ) \log((\ell_\tau+1)/(\ell_0+1)) log (( ℓ τ + 1 ) / ( ℓ 0 + 1 )) play the role of x x x in the bound ∣ e x − 1 ∣ ≲ ε e O ( ε ) |e^x-1|\lesssim\varepsilon e^{O(\varepsilon)} ∣ e x − 1∣ ≲ ε e O ( ε ) that holds for all ∣ x ∣ ≲ ε |x|\lesssim\varepsilon ∣ x ∣ ≲ ε . This finishes the comparison of ℓ τ \ell_\tau ℓ τ and ℓ 0 \ell_0 ℓ 0 .
*Comparison of A ( τ ) A(\tau) A ( τ ) and A ( 0 ) A(0) A ( 0 ) . Integrating the identity e − ℓ τ − A ( 0 ) = e ℓ 0 − ℓ τ d μ 0 d γ p − 1 e^{-\ell_\tau-A(0)}=e^{\ell_0-\ell_\tau}\frac{d\mu_0}{d\gamma_{p-1}} e − ℓ τ − A ( 0 ) = e ℓ 0 − ℓ τ d γ p − 1 d μ 0 against γ p − 1 \gamma_{p-1} γ p − 1 leads to
A ( 0 ) − A ( τ ) = − log E μ 0 ( e ℓ 0 − ℓ τ ) ≤ E μ 0 ( ℓ τ − ℓ 0 ) ≲ ε E μ 0 ( ℓ 0 ) + 1 \begin{aligned}
A(0)-A(\tau)&=-\log\mathbb{E}_{\mu_0}\left(e^{\ell_0-\ell_\tau}\right)\\
&\leq\mathbb{E}_{\mu_0}(\ell_\tau-\ell_0)\\
&\lesssim\varepsilon\mathbb{E}_{\mu_0}(\ell_0)+1
\end{aligned} A ( 0 ) − A ( τ ) = − log E μ 0 ( e ℓ 0 − ℓ τ ) ≤ E μ 0 ( ℓ τ − ℓ 0 ) ≲ ε E μ 0 ( ℓ 0 ) + 1
where the second step uses Jensen’s inequality and the third step uses (45) . Since the KL-divergence between μ 0 \mu_0 μ 0 and γ p − 1 \gamma_{p-1} γ p − 1 satisfies 0 ≤ ∫ log ( d μ 0 / d γ p − 1 ) d μ 0 = E μ 0 ( − ℓ 0 − A ( 0 ) ) 0\leq\int\log(d\mu_0/d\gamma_{p-1})d\mu_0=\mathbb{E}_{\mu_0}(-\ell_0-A(0)) 0 ≤ ∫ log ( d μ 0 / d γ p − 1 ) d μ 0 = E μ 0 ( − ℓ 0 − A ( 0 )) , we get
A ( 0 ) − A ( τ ) ≲ − ε A ( 0 ) + 1 ≲ 1 , \begin{aligned}
A(0)-A(\tau)&\lesssim-\varepsilon A(0)+1\\
&\lesssim1,
\end{aligned} A ( 0 ) − A ( τ ) ≲ − ε A ( 0 ) + 1 ≲ 1 ,
which uses the assumed lower bound on A ( 0 ) A(0) A ( 0 ) and the choice of ε = 1 / log ( p n ) \varepsilon=1/\log(pn) ε = 1/ log ( p n ) . Applying this and (45) to (44) shows that
d μ τ d μ 0 ≲ e O ( ε ℓ 0 ) . (48) \frac{d\mu_\tau}{d\mu_0}\lesssim e^{O(\varepsilon\ell_0)}. \tag*{(48)} d μ 0 d μ τ ≲ e O ( ε ℓ 0 ) . (48)
Combining. As a temporary shorthand, let φ = J 0 h ⋆ \varphi=J_0h^\star φ = J 0 h ⋆ . This function is nonnegative and has finite sup-norm due to Lemma 5.4 (as ρ < 1 \rho<1 ρ < 1 and h ⋆ ∈ H h^\star\in\mathcal{H} h ⋆ ∈ H ). To conclude the proof, note that E μ τ φ = E μ 0 ( φ d μ τ / d μ 0 ) \mathbb{E}_{\mu_\tau}\varphi=\mathbb{E}_{\mu_0}(\varphi\,d\mu_\tau/d\mu_0) E μ τ φ = E μ 0 ( φ d μ τ / d μ 0 ) and consider the set Ω = { ℓ 0 ≤ 20 log ( p n ) } \Omega=\{\ell_0\leq20\log(pn)\} Ω = { ℓ 0 ≤ 20 log ( p n )} . On the set Ω \Omega Ω , the density bound (48) implies d μ τ / d μ 0 ≲ 1 d\mu_\tau/d\mu_0\lesssim1 d μ τ / d μ 0 ≲ 1 . Meanwhile, on the set Ω c \Omega^c Ω c , we will use the same density bound to write ( d μ τ / d μ 0 ) d μ 0 = exp ( O ( ε ℓ 0 ) − ℓ 0 − A ( 0 ) ) d γ p − 1 (d\mu_\tau/d\mu_0)d\mu_0=\exp(O(\varepsilon\ell_0)-\ell_0-A(0))d\gamma_{p-1} ( d μ τ / d μ 0 ) d μ 0 = exp ( O ( ε ℓ 0 ) − ℓ 0 − A ( 0 )) d γ p − 1 . Hence,
E μ τ φ ≲ E μ 0 φ + ∥ φ ∥ s u p ∫ Ω c e O ( ε ℓ 0 ) − ℓ 0 − A ( 0 ) d γ p − 1 . (49) \mathbb{E}_{\mu_\tau}\varphi\lesssim\mathbb{E}_{\mu_0}\varphi+\|\varphi\|_{\mathrm{sup}}\int_{\Omega^c}e^{O(\varepsilon\ell_0)-\ell_0-A(0)}d\gamma_{p-1}. \tag*{(49)} E μ τ φ ≲ E μ 0 φ + ∥ φ ∥ sup ∫ Ω c e O ( ε ℓ 0 ) − ℓ 0 − A ( 0 ) d γ p − 1 . (49)
From the definition of Ω c \Omega^c Ω c and the assumed bound − A ( 0 ) ≤ 10 log ( p n ) -A(0)\leq10\log(pn) − A ( 0 ) ≤ 10 log ( p n ) , it follows that the exponent in (49) is at most − 10 log ( p n ) + O ( 1 ) -10\log(pn)+O(1) − 10 log ( p n ) + O ( 1 ) , which gives the stated result. □ \square □
The function h ⋆ h^\star h ⋆ and the proof of Lemma 5.3
Before proving Lemma 5.3 , we first prove Lemma 5.7 below, which establishes the existence and regularity of the function h ⋆ h^\star h ⋆ used in § 5.3 . Also, recall that H \mathcal{H} H denotes the class of functions h ∈ C 3 ( R p − 1 ) h \in C^3(\mathbb{R}^{p-1}) h ∈ C 3 ( R p − 1 ) with bounded first and second derivatives.
Lemma 5.7. Let a a a be a tuple of affine functions on R p − 1 × R \mathbb{R}^{p-1} \times\mathbb{R} R p − 1 × R with max l ≤ p − 1 ∥ ∇ a l ∥ 2 ≤ 2 \max_{l \le p-1}\|\nabla a_l\|_2 \le2 max l ≤ p − 1 ∥∇ a l ∥ 2 ≤ 2 . Then, there exists a function h ⋆ ∈ H h^\star\in\mathcal{H} h ⋆ ∈ H such that L 0 h ⋆ = E μ 0 ℓ ˙ 0 − ℓ ˙ 0 L_0h^\star= \mathbb{E}_{\mu_0}\dot{\ell}_0 - \dot{\ell}_0 L 0 h ⋆ = E μ 0 ℓ ˙ 0 − ℓ ˙ 0 and
∥ J 0 h ⋆ ∥ sup ≲ ( p n ) 5 ∥ ∂ τ a ∥ ∞ 2 . (50) \|J_0h^\star\|_{\sup} \lesssim(pn)^5\|\partial_\tau a\|_\infty^2. \tag*{(50)} ∥ J 0 h ⋆ ∥ s u p ≲ ( p n ) 5 ∥ ∂ τ a ∥ ∞ 2 . (50)
Proof. It is shown in Mackey and Gorham [35 ] that a solution h ⋆ ∈ C 2 ( R p − 1 ) h^\star\in C^2(\mathbb{R}^{p-1}) h ⋆ ∈ C 2 ( R p − 1 ) to the equation L 0 h = E μ 0 ℓ ˙ 0 − ℓ ˙ 0 L_0h = \mathbb{E}_{\mu_0}\dot{\ell}_0 - \dot{\ell}_0 L 0 h = E μ 0 ℓ ˙ 0 − ℓ ˙ 0 exists. Once a C 2 C^2 C 2 solution is known to exist, the differentiability can be upgraded to C ∞ C^\infty C ∞ , because the operator L 0 = Δ − ⟨ ∇ W 0 , ∇ ⋅ ⟩ L_0 = \Delta- \langle\nabla W_0,\nabla\cdot\rangle L 0 = Δ − ⟨ ∇ W 0 , ∇ ⋅ ⟩ has C ∞ C^\infty C ∞ coefficients and the right-hand side E μ 0 ℓ ˙ 0 − ℓ ˙ 0 \mathbb{E}_{\mu_0}\dot{\ell}_0 - \dot{\ell}_0 E μ 0 ℓ ˙ 0 − ℓ ˙ 0 is C ∞ C^\infty C ∞ (Gilbarg and Trudinger [24 ] ).
It remains to show that the first and second derivatives of h ⋆ h^\star h ⋆ are bounded and that (50) holds. After adapting the notation of Mackey and Gorham [35 ] to our setting, it provides the following quantitative derivative bounds
∥ ∇ h ⋆ ∥ sup ≲ ∥ ∇ ℓ ˙ 0 ∥ sup and ∥ ∇ 2 h ⋆ ∥ sup ≲ ∥ ∇ 3 ℓ 0 ∥ sup ∥ ∇ ℓ ˙ 0 ∥ sup + ∥ ∇ 2 ℓ ˙ 0 ∥ sup . \|\nabla h^\star\|_{\sup} \lesssim\|\nabla\dot{\ell}_0\|_{\sup}
\qquad\text{and}\qquad
\|\nabla^2h^\star\|_{\sup} \lesssim\|\nabla^3\ell_0\|_{\sup}\|\nabla\dot{\ell}_0\|_{\sup}+\|\nabla^2\dot{\ell}_0\|_{\sup}. ∥∇ h ⋆ ∥ s u p ≲ ∥∇ ℓ ˙ 0 ∥ s u p and ∥ ∇ 2 h ⋆ ∥ s u p ≲ ∥ ∇ 3 ℓ 0 ∥ s u p ∥∇ ℓ ˙ 0 ∥ s u p + ∥ ∇ 2 ℓ ˙ 0 ∥ s u p .
From the formula (30) , the derivatives of ℓ ˙ 0 \dot{\ell}_0 ℓ ˙ 0 and ℓ 0 \ell_0 ℓ 0 can be computed explicitly, and since the functions ρ \rho ρ , ρ ′ \rho' ρ ′ , ρ ′ ′ \rho'' ρ ′′ are bounded (Lemma 8.4 (a)), it can be checked that
∥ ∇ ℓ ˙ 0 ∥ sup ≤ 2 p ∥ ∂ τ a ∥ ∞ σ 2 , ∥ ∇ 2 ℓ ˙ 0 ∥ sup ≲ p ∥ ∂ τ a ∥ ∞ σ 3 , ∥ ∇ 3 ℓ 0 ∥ sup ≲ p σ 3 . \|\nabla\dot{\ell}_0\|_{\sup}\le\frac{2p\|\partial_\tau a\|_\infty}{\sigma^2},
\qquad
\|\nabla^2\dot{\ell}_0\|_{\sup}\lesssim\frac{p\|\partial_\tau a\|_\infty}{\sigma^3},
\qquad
\|\nabla^3\ell_0\|_{\sup}\lesssim\frac{p}{\sigma^3}. ∥∇ ℓ ˙ 0 ∥ s u p ≤ σ 2 2 p ∥ ∂ τ a ∥ ∞ , ∥ ∇ 2 ℓ ˙ 0 ∥ s u p ≲ σ 3 p ∥ ∂ τ a ∥ ∞ , ∥ ∇ 3 ℓ 0 ∥ s u p ≲ σ 3 p .
(Note that parts of (42) can be re-used here.) In particular, this shows that ∥ ∇ h ⋆ ∥ sup \|\nabla h^\star\|_{\sup} ∥∇ h ⋆ ∥ s u p and ∥ ∇ 2 h ⋆ ∥ sup \|\nabla^2h^\star\|_{\sup} ∥ ∇ 2 h ⋆ ∥ s u p are finite, and hence h ⋆ ∈ H h^\star\in\mathcal{H} h ⋆ ∈ H . Finally, for the bound on ∥ J 0 h ⋆ ∥ sup \|J_0h^\star\|_{\sup} ∥ J 0 h ⋆ ∥ s u p , applying the last two displays to (41) , and using the basic inequalities ρ < 1 \rho< 1 ρ < 1 and σ − 2 ≤ n \sigma^{-2} \le n σ − 2 ≤ n , leads to (50) . □
Proof of Lemma 5.3. The proof proceeds in a manner that is similar to Barthe and Cordero-Erausquin [4 ] . We begin by considering h ∈ C 3 ( R p − 1 ) h \in C^3(\mathbb{R}^{p-1}) h ∈ C 3 ( R p − 1 ) with compact support, and then we will extend the argument later to handle h ∈ H h \in\mathcal{H} h ∈ H . As notation that is specific to this proof, let φ ∈ C 1 ( R p − 1 ) \varphi\in C^1(\mathbb{R}^{p-1}) φ ∈ C 1 ( R p − 1 ) be a test function, and let g = ℓ ˙ τ − E μ τ ℓ ˙ τ g = \dot{\ell}_\tau- \mathbb{E}_{\mu_\tau}\dot{\ell}_\tau g = ℓ ˙ τ − E μ τ ℓ ˙ τ so that Var μ τ ( ℓ ˙ τ ) = E μ τ g 2 \operatorname{Var}_{\mu_\tau}(\dot{\ell}_\tau) = \mathbb{E}_{\mu_\tau}g^2 Var μ τ ( ℓ ˙ τ ) = E μ τ g 2 . The basic idea is to expand the square E μ τ ( g + L τ h ) 2 \mathbb{E}_{\mu_\tau}(g+L_\tau h)^2 E μ τ ( g + L τ h ) 2 to obtain E μ τ g 2 = − 2 E μ τ ( g L τ h ) − E μ τ ( L τ h ) 2 + E μ τ ( g + L τ h ) 2 \mathbb{E}_{\mu_\tau}g^2 = -2\mathbb{E}_{\mu_\tau}(gL_\tau h)-\mathbb{E}_{\mu_\tau}(L_\tau h)^2+\mathbb{E}_{\mu_\tau}(g+L_\tau h)^2 E μ τ g 2 = − 2 E μ τ ( g L τ h ) − E μ τ ( L τ h ) 2 + E μ τ ( g + L τ h ) 2 and then show that the right side has the form of the stated result in (35) .
By the divergence theorem and the compact support of h h h , the divergence of φ e − W τ ∇ h \varphi e^{-W_\tau}\nabla h φ e − W τ ∇ h integrates to 0 over R p − 1 \mathbb{R}^{p-1} R p − 1 . Rearranging that equation and noting that the density of μ τ \mu_\tau μ τ with respect to Lebesgue measure on R p − 1 \mathbb{R}^{p-1} R p − 1 is proportional to e − W τ e^{-W_\tau} e − W τ , it can be checked that
E μ τ ⟨ ∇ φ , ∇ h ⟩ = − E μ τ ( φ L τ h ) . (51) \mathbb{E}_{\mu_\tau}\langle\nabla\varphi,\nabla h\rangle=-\mathbb{E}_{\mu_\tau}(\varphi L_\tau h). \tag*{(51)} E μ τ ⟨ ∇ φ , ∇ h ⟩ = − E μ τ ( φ L τ h ) . (51)
We will use this identity three times with certain choices of φ \varphi φ and h h h . First, we take φ = L τ h \varphi=L_\tau h φ = L τ h , which lies in C 1 ( R p − 1 ) C^1(\mathbb{R}^{p-1}) C 1 ( R p − 1 ) because h ∈ C 3 ( R p − 1 ) h \in C^3(\mathbb{R}^{p-1}) h ∈ C 3 ( R p − 1 ) , and obtain E μ τ ( L τ h ) 2 = − E μ τ ⟨ ∇ L τ h , ∇ h ⟩ \mathbb{E}_{\mu_\tau}(L_\tau h)^2=-\mathbb{E}_{\mu_\tau}\langle\nabla L_\tau h,\nabla h\rangle E μ τ ( L τ h ) 2 = − E μ τ ⟨ ∇ L τ h , ∇ h ⟩ . Using ∂ j L τ h = L τ ∂ j h − ( ∇ 2 W τ ∇ h ) j \partial_jL_\tau h=L_\tau\partial_jh-(\nabla^2W_\tau\nabla h)_j ∂ j L τ h = L τ ∂ j h − ( ∇ 2 W τ ∇ h ) j , this becomes
E μ τ ( L τ h ) 2 = − ∑ j = 1 p − 1 E μ τ ( ∂ j h L τ ∂ j h ) + E μ τ ⟨ ∇ h , ∇ 2 W τ ∇ h ⟩ . \mathbb{E}_{\mu_\tau}(L_\tau h)^2=-\sum_{j=1}^{p-1}\mathbb{E}_{\mu_\tau}(\partial_jhL_\tau\partial_jh)+\mathbb{E}_{\mu_\tau}\langle\nabla h,\nabla^2W_\tau\nabla h\rangle. E μ τ ( L τ h ) 2 = − j = 1 ∑ p − 1 E μ τ ( ∂ j h L τ ∂ j h ) + E μ τ ⟨ ∇ h , ∇ 2 W τ ∇ h ⟩ .
Second, we apply (51) with ( φ , h ) (\varphi,h) ( φ , h ) replaced by ( ∂ j h , ∂ j h ) (\partial_jh,\partial_jh) ( ∂ j h , ∂ j h ) . This gives − E μ τ ( ∂ j h L τ ∂ j h ) = E μ τ ∥ ∇ ∂ j h ∥ 2 2 -\mathbb{E}_{\mu_\tau}(\partial_jhL_\tau\partial_jh)=\mathbb{E}_{\mu_\tau}\|\nabla\partial_jh\|_2^2 − E μ τ ( ∂ j h L τ ∂ j h ) = E μ τ ∥∇ ∂ j h ∥ 2 2 , and summing over j j j yields
E μ τ ( L τ h ) 2 = E μ τ ( ∥ ∇ 2 h ∥ F 2 + ⟨ ∇ h , ∇ 2 W τ ∇ h ⟩ ) . \mathbb{E}_{\mu_\tau}(L_\tau h)^2=\mathbb{E}_{\mu_\tau}\left(\|\nabla^2h\|_F^2+\langle\nabla h,\nabla^2W_\tau\nabla h\rangle\right). E μ τ ( L τ h ) 2 = E μ τ ( ∥ ∇ 2 h ∥ F 2 + ⟨ ∇ h , ∇ 2 W τ ∇ h ⟩ ) .
Third, taking φ = g \varphi=g φ = g in (51) gives − E μ τ ( g L τ h ) = E μ τ ⟨ ∇ g , ∇ h ⟩ = E μ τ ⟨ ∇ ℓ ˙ τ , ∇ h ⟩ -\mathbb{E}_{\mu_\tau}(gL_\tau h)=\mathbb{E}_{\mu_\tau}\langle\nabla g,\nabla h\rangle=\mathbb{E}_{\mu_\tau}\langle\nabla\dot{\ell}_\tau,\nabla h\rangle − E μ τ ( g L τ h ) = E μ τ ⟨ ∇ g , ∇ h ⟩ = E μ τ ⟨ ∇ ℓ ˙ τ , ∇ h ⟩ . Inserting the last two facts into the expression for E μ τ g 2 \mathbb{E}_{\mu_\tau}g^2 E μ τ g 2 at the beginning shows that it has the form of (35) .
Thus, it remains to extend to the case when h ∈ H h\in\mathcal{H} h ∈ H . For such h h h , let f r = χ ( ⋅ / r ) h f_r=\chi(\cdot/r)h f r = χ ( ⋅ / r ) h , where r > 0 r>0 r > 0 is a cutoff parameter and χ \chi χ is a smooth radial cutoff function on R p − 1 \mathbb{R}^{p-1} R p − 1 with 1 B ( 0 , 1 ) ≤ χ ≤ 1 B ( 0 , 2 ) \mathbf{1}_{B(0,1)}\leq\chi\leq\mathbf{1}_{B(0,2)} 1 B ( 0 , 1 ) ≤ χ ≤ 1 B ( 0 , 2 ) , and B ( 0 , r ) = { z ∈ R p − 1 : ∥ z ∥ 2 ≤ r } B(0,r)=\{z\in\mathbb{R}^{p-1}:\|z\|_2\leq r\} B ( 0 , r ) = { z ∈ R p − 1 : ∥ z ∥ 2 ≤ r } . Since f r f_r f r is C 3 C^3 C 3 with compact support, the first part of the proof shows that (35) holds with f r f_r f r in place of h h h . We now let r → ∞ r\to\infty r → ∞ . Since f r = h f_r=h f r = h on B ( 0 , r ) B(0,r) B ( 0 , r ) , the functions f r f_r f r , ∇ f r \nabla f_r ∇ f r , ∇ 2 f r \nabla^2f_r ∇ 2 f r and L τ f r L_\tau f_r L τ f r converge pointwise to h h h , ∇ h \nabla h ∇ h , ∇ 2 h \nabla^2h ∇ 2 h and L τ h L_\tau h L τ h as r → ∞ r\to\infty r → ∞ . To justify passing the limit under the expectation in (35) , we use dominated convergence, based on the following observations.
The first two derivatives of f r f_r f r are bounded uniformly in r ≥ 1 r\geq1 r ≥ 1 . Indeed, ∇ h \nabla h ∇ h and ∇ 2 h \nabla^2h ∇ 2 h are bounded, h h h grows at most linearly, and the k k k th derivative of χ ( ⋅ / r ) \chi(\cdot/r) χ ( ⋅ / r ) is bounded by a constant multiple of r − k r^{-k} r − k and vanishes outside B ( 0 , 2 r ) B(0,2r) B ( 0 , 2 r ) .
The functions ∥ ∇ W τ ∥ 2 \|\nabla W_\tau\|_2 ∥∇ W τ ∥ 2 , g g g and L τ f r L_\tau f_r L τ f r grow at most linearly in ∥ z ∥ 2 \|z\|_2 ∥ z ∥ 2 , uniformly in r ≥ 1 r\geq1 r ≥ 1 . For ∇ W τ \nabla W_\tau ∇ W τ , this follows from ∇ W τ = z − ∑ l ≤ p − 1 λ ( a l ( ⋅ , τ ) / σ ) ∇ a l / σ \nabla W_\tau=z-\sum_{l\leq p-1}\lambda(a_l(\cdot,\tau)/\sigma)\nabla a_l/\sigma ∇ W τ = z − ∑ l ≤ p − 1 λ ( a l ( ⋅ , τ ) / σ ) ∇ a l / σ and λ ( s ) ≤ 1 + ∣ s ∣ \lambda(s)\leq1+|s| λ ( s ) ≤ 1 + ∣ s ∣ (Lemma 8.4 (a)). For g g g , it follows from the boundedness of ∇ ℓ ˙ τ \nabla\dot{\ell}_\tau ∇ ℓ ˙ τ , which holds by (42) and ρ < 1 \rho<1 ρ < 1 . For L τ f r = Δ f r − ⟨ ∇ W τ , ∇ f r ⟩ L_\tau f_r=\Delta f_r-\langle\nabla W_\tau,\nabla f_r\rangle L τ f r = Δ f r − ⟨ ∇ W τ , ∇ f r ⟩ , it follows from (i).
The Hessian of W τ W_\tau W τ satisfies ∥ ∇ 2 W τ ∥ s u p ≤ 1 + ∥ ∇ 2 ℓ τ ∥ s u p < ∞ \|\nabla^2W_\tau\|_{\mathrm{sup}}\leq1+\|\nabla^2\ell_\tau\|_{\mathrm{sup}}<\infty ∥ ∇ 2 W τ ∥ sup ≤ 1 + ∥ ∇ 2 ℓ τ ∥ sup < ∞ , due to (42) and ρ < 1 \rho<1 ρ < 1 .
Since d μ τ ≤ e − A ( τ ) d γ p − 1 d\mu_\tau\leq e^{-A(\tau)}d\gamma_{p-1} d μ τ ≤ e − A ( τ ) d γ p − 1 , every function of z z z that grows at most polynomially is μ τ \mu_\tau μ τ -integrable.
By (i)–(iii), each integrand in (35) for f r f_r f r is bounded by a polynomial in ∥ z ∥ 2 \|z\|_2 ∥ z ∥ 2 that does not depend on r ≥ 1 r\geq1 r ≥ 1 , and so (iv) allows us to let r → ∞ r\to\infty r → ∞ . This proves (35) for all h ∈ H h\in\mathcal{H} h ∈ H .
Proofs of Lemmas 5.1 and 4.5
Proof of Lemma 5.1. Let s = u + σ 2 s=\sqrt{u+\sigma^2} s = u + σ 2 and observe that Lemma 4.4 gives
log ψ j ( y + τ w ) = log q j ( y + τ w ) − 1 2 a j ( y + τ w ) 2 − 1 2 log ( 2 π s 2 ) . \log\psi_j(y+\tau w)=\log q_j(y+\tau w)-\frac{1}{2}a_j(y+\tau w)^2-\frac{1}{2}\log(2\pi s^2). log ψ j ( y + τ w ) = log q j ( y + τ w ) − 2 1 a j ( y + τ w ) 2 − 2 1 log ( 2 π s 2 ) .
where a j ( y + τ w ) a_j(y+\tau w) a j ( y + τ w ) is affine in τ \tau τ with slope − w j / s -w_j/s − w j / s . It therefore suffices to find a tuple of affine functions satisfying the two stated bounds such that
log q j ( y + τ w ) = A ( τ ) for all τ ∈ R , (52) \log q_j(y+\tau w)=A(\tau)\qquad\text{for all }\tau\in\mathbb{R}, \tag*{(52)} log q j ( y + τ w ) = A ( τ ) for all τ ∈ R , (52)
since differentiating twice gives the curvature identity (33) , and setting τ = 0 \tau=0 τ = 0 gives A ( 0 ) = log q j ( y ) A(0)=\log q_j(y) A ( 0 ) = log q j ( y ) .
To proceed, let Z ∼ γ p Z\sim\gamma_p Z ∼ γ p and let Π j \Pi_j Π j denote the orthogonal projection onto E j E_j E j so that Π j Z ∼ γ E j \Pi_jZ\sim\gamma_{E_j} Π j Z ∼ γ E j . Consequently, the definitions of q j q_j q j and K j K_j K j give
q j ( y ) = P ( Π j Z ∈ K j ) = P ( a j ( y ) n j + Π j Z ∈ P ( y ) ) = P ( Σ u 1 / 2 ( a j ( y ) n j + Π j Z ) ≤ β − y ) , (53) \begin{aligned}
q_j(y)&=\mathbb{P}(\Pi_jZ\in K_j)\\
&=\mathbb{P}\bigl(a_j(y)n_j+\Pi_jZ\in P(y)\bigr)\\
&=\mathbb{P}\bigl(\Sigma_u^{1/2}(a_j(y)n_j+\Pi_jZ)\leq\beta-y\bigr), \tag*{(53)}
\end{aligned} q j ( y ) = P ( Π j Z ∈ K j ) = P ( a j ( y ) n j + Π j Z ∈ P ( y ) ) = P ( Σ u 1/2 ( a j ( y ) n j + Π j Z ) ≤ β − y ) , (53)
where we recall n j = 1 s Σ 1 / 2 u e j n_j = \frac{1}{s}\Sigma^{1/2}u\,e_j n j = s 1 Σ 1/2 u e j . Next, observe that the orthogonal projection Π j \Pi_j Π j can be expressed as Π j = I p − n j n j T \Pi_j = I_p - n_j n_j^{\mathsf T} Π j = I p − n j n j T , and define the deterministic vector v j = 1 s Σ 1 / 2 n j v_j = \frac{1}{s}\Sigma^{1/2}n_j v j = s 1 Σ 1/2 n j , as well as the Gaussian vector G ~ = Σ 1 / 2 Z \widetilde{G} = \Sigma^{1/2}Z G = Σ 1/2 Z . Thus, the last expression in (53) becomes
q j ( y ) = P ( [ s a j ( y ) − G ~ ] v j + G ~ ≤ β − y ) . (54) q_j(y) = \mathbf{P}\left(\left[s a_j(y)-\widetilde{G}\right]v_j+\widetilde{G}\leq\beta-y\right). \tag*{(54)} q j ( y ) = P ( [ s a j ( y ) − G ] v j + G ≤ β − y ) . (54)
Writing v j l v_{jl} v j l for the l l l th entry of v j v_j v j , it can be checked that v j j = 1 v_{jj}=1 v j j = 1 and ∣ v j l ∣ ≤ 1 |v_{jl}|\leq1 ∣ v j l ∣ ≤ 1 . Since s a j ( y ) = β j − y j s a_j(y)=\beta_j-y_j s a j ( y ) = β j − y j , the j j j th inequality in (54) holds trivially, and so
q j ( y ) = P ( G ~ l − v j l G ~ j ≤ b l for all l ≠ j ) , q_j(y)=\mathbf{P}\left(\widetilde{G}_l-v_{jl}\widetilde{G}_j\leq b_l\text{ for all }l\ne j\right), q j ( y ) = P ( G l − v j l G j ≤ b l for all l = j ) ,
where b l = β l − y l − v j l ( β j − y j ) b_l=\beta_l-y_l-v_{jl}(\beta_j-y_j) b l = β l − y l − v j l ( β j − y j ) .
Next, recall that G ∼ N ( 0 , Σ ) G\sim N(0,\Sigma) G ∼ N ( 0 , Σ ) and N ∼ N ( 0 , I p ) N\sim N(0,I_p) N ∼ N ( 0 , I p ) denote independent Gaussian vectors, which implies G ~ = d u G + σ N \widetilde{G}\mathrel{\overset{d}{=}}\sqrt{u}G+\sigma N G = d u G + σ N . Based on this, each G ~ l − v j l G ~ j \widetilde{G}_l-v_{jl}\widetilde{G}_j G l − v j l G j may be written in the form ζ l + σ N l \zeta_l+\sigma N_l ζ l + σ N l , where we put ζ l = u G l − v j l ( u G j + σ N j ) \zeta_l=\sqrt{u}G_l-v_{jl}(\sqrt{u}G_j+\sigma N_j) ζ l = u G l − v j l ( u G j + σ N j ) . Importantly, the vector ζ = ( ζ l ) l ≠ j \zeta=(\zeta_l)_{l\ne j} ζ = ( ζ l ) l = j only depends on N N N through its j j j th entry, and so ζ \zeta ζ is independent of ( N l ) l ≠ j (N_l)_{l\ne j} ( N l ) l = j . Taking advantage of this independence, as well as the fact that the entries of N N N are i.i.d. standard Gaussian, we have
q j ( y ) = P ( N l ≤ b l − ζ l σ for l ≠ j ) = E ∏ l : l ≠ j p Φ ( b l − ζ l σ ) = ∫ R p − 1 ∏ l : l ≠ j p Φ ( b l − ⟨ η l , z ⟩ σ ) d γ p − 1 ( z ) . (55) \begin{aligned}
q_j(y)&=\mathbf{P}\left(N_l\leq\frac{b_l-\zeta_l}{\sigma}\text{ for }l\ne j\right)\\
&=\mathbf{E}\prod_{l:l\ne j}^{p}\Phi\left(\frac{b_l-\zeta_l}{\sigma}\right)\\
&=\int_{\mathbb{R}^{p-1}}\prod_{l:l\ne j}^{p}\Phi\left(\frac{b_l-\langle\eta_l,z\rangle}{\sigma}\right)d\gamma_{p-1}(z). \tag*{(55)}
\end{aligned} q j ( y ) = P ( N l ≤ σ b l − ζ l for l = j ) = E l : l = j ∏ p Φ ( σ b l − ζ l ) = ∫ R p − 1 l : l = j ∏ p Φ ( σ b l − ⟨ η l , z ⟩ ) d γ p − 1 ( z ) . (55)
where we have used ζ l = d ⟨ η l , N ′ ⟩ \zeta_l\mathrel{\overset{d}{=}}\langle\eta_l,N'\rangle ζ l = d ⟨ η l , N ′ ⟩ , with N ′ ∼ γ p − 1 N'\sim\gamma_{p-1} N ′ ∼ γ p − 1 and the vectors ( η l ) l ≠ j (\eta_l)_{l\ne j} ( η l ) l = j corresponding to the rows of ( E ( ζ ζ T ) ) 1 / 2 (\mathbf{E}(\zeta\zeta^{\mathsf T}))^{1/2} ( E ( ζ ζ T ) ) 1/2 . To conclude the proof, we replace y y y with y + τ w y+\tau w y + τ w and define the affine functions
a l ( z , τ ) = b l − ⟨ η l , z ⟩ − τ ( w l − v j l w j ) , l ≠ j . a_l(z,\tau)=b_l-\langle\eta_l,z\rangle-\tau(w_l-v_{jl}w_j),\qquad l\ne j. a l ( z , τ ) = b l − ⟨ η l , z ⟩ − τ ( w l − v j l w j ) , l = j .
In this notation, the integral in (55) matches the form of the one that defines A ( τ ) A(\tau) A ( τ ) in equation (32) , provided that the affine functions are relabelled as a 1 , … , a p − 1 a_1,\ldots,a_{p-1} a 1 , … , a p − 1 . Finally, to verify that the affine functions satisfy the required conditions, observe that ∣ ∂ τ a l ∣ = ∣ w l − v j l w j ∣ ≤ 2 ∥ w ∥ ∞ |\partial_\tau a_l|=|w_l-v_{jl}w_j|\leq2\|w\|_\infty ∣ ∂ τ a l ∣ = ∣ w l − v j l w j ∣ ≤ 2∥ w ∥ ∞ . Furthermore, we have ∥ ∇ a l ∥ 2 2 = ∥ η l ∥ 2 2 = Var ( ζ l ) \|\nabla a_l\|_2^2=\|\eta_l\|_2^2=\operatorname{Var}(\zeta_l) ∥∇ a l ∥ 2 2 = ∥ η l ∥ 2 2 = Var ( ζ l ) , and a direct computation using v j l = u s 2 Σ j l v_{jl}=\frac{u}{s^2}\Sigma_{jl} v j l = s 2 u Σ j l and u ≤ 2 u\leq2 u ≤ 2 shows that Var ( ζ l ) = u − s 2 v j l 2 ≤ 2 \operatorname{Var}(\zeta_l)=u-s^2v_{jl}^2\leq2 Var ( ζ l ) = u − s 2 v j l 2 ≤ 2 . □ \square □
Proof of Lemma 4.5. Let a \mathbf{a} a be the tuple of affine functions given by Lemma 5.1 with associated log-partition function A A A . Also, let u u u , y y y , w w w and j j j be fixed as in the stated result. By (52) , we have log q j ( y + τ w ) = A ( τ ) \log q_j(y+\tau w)=A(\tau) log q j ( y + τ w ) = A ( τ ) for all τ ∈ R \tau\in\mathbb{R} τ ∈ R , and differentiating at τ = 0 \tau=0 τ = 0 gives
∂ w log q j ( y ) = A ′ ( 0 ) . \partial_w\log q_j(y)=A'(0). ∂ w log q j ( y ) = A ′ ( 0 ) .
So, it suffices to bound ∣ A ′ ( 0 ) ∣ |A'(0)| ∣ A ′ ( 0 ) ∣ . Differentiating the identity e A ( τ ) = ∫ e − ℓ τ d γ p − 1 e^{A(\tau)}=\int e^{-\ell_\tau}\,d\gamma_{p-1} e A ( τ ) = ∫ e − ℓ τ d γ p − 1 gives
A ′ ( 0 ) = − E μ 0 ℓ ˙ 0 . A'(0)=-\mathbf{E}_{\mu_0}\dot{\ell}_0. A ′ ( 0 ) = − E μ 0 ℓ ˙ 0 .
The derivative bound (46) in the proof of Lemma 5.6 holds for an arbitrary tuple a \mathbf{a} a , and so
∣ A ′ ( 0 ) ∣ ≤ E μ 0 ∣ ℓ ˙ 0 ∣ ≲ ∥ ∂ τ a ∥ ∞ σ ( log ( p n ) E μ 0 ℓ 0 + 1 ) . |A'(0)|\leq\mathbf{E}_{\mu_0}|\dot{\ell}_0|\lesssim\frac{\|\partial_\tau\mathbf{a}\|_\infty}{\sigma}\left(\sqrt{\log(pn)}\mathbf{E}_{\mu_0}\ell_0+1\right). ∣ A ′ ( 0 ) ∣ ≤ E μ 0 ∣ ℓ ˙ 0 ∣ ≲ σ ∥ ∂ τ a ∥ ∞ ( log ( p n ) E μ 0 ℓ 0 + 1 ) .
We will now express the right side in terms of w w w and q j ( y ) q_j(y) q j ( y ) . By Lemma 5.1, the tuple a \mathbf{a} a satisfies ∥ ∂ τ a ∥ ∞ ≤ 2 ∥ w ∥ ∞ \|\partial_\tau\mathbf{a}\|_\infty\leq2\|w\|_\infty ∥ ∂ τ a ∥ ∞ ≤ 2∥ w ∥ ∞ . Next, a simple calculation shows that the KL-divergence between μ 0 \mu_0 μ 0 and γ p − 1 \gamma_{p-1} γ p − 1 is equal to E μ 0 ( − ℓ 0 − A ( 0 ) ) \mathbf{E}_{\mu_0}(-\ell_0-A(0)) E μ 0 ( − ℓ 0 − A ( 0 )) , and so its non-negativity gives E μ 0 ℓ 0 ≤ − A ( 0 ) \mathbf{E}_{\mu_0}\ell_0\leq-A(0) E μ 0 ℓ 0 ≤ − A ( 0 ) . Lastly, Lemma 5.1 gives − A ( 0 ) = log ( 1 / q j ( y ) ) -A(0)=\log(1/q_j(y)) − A ( 0 ) = log ( 1/ q j ( y )) . Substituting these bounds into the last display completes the proof. □ \square □
Proof of Proposition 3.5: facet sensitivity bounds
For proving parts (a) and (b) of Proposition 3.5 , recall that P P P is a polyhedron in R p \mathbb{R}^{p} R p with p p p linearly independent normal vectors as in (9) , and that Lemma 4.2 gives the following explicit formula for the facet sensitivity,
F ( P ) = ∑ j = 1 p ( 1 + a j 2 + ∥ m j ∥ 2 2 + tr ( Π j − S j ) ) ν j ( P ) . (56) \mathcal{F}(P)=\sum_{j=1}^{p}\left(1+a_j^2+\lVert m_j\rVert_2^2+\operatorname{tr}(\Pi_j-\mathcal{S}_j)\right)\nu_j(P). \tag*{(56)} F ( P ) = j = 1 ∑ p ( 1 + a j 2 + ∥ m j ∥ 2 2 + tr ( Π j − S j ) ) ν j ( P ) . (56)
Part (a). By (56) , it suffices to show that
∑ j = 1 p ( 1 + a j 2 + ∥ m j ∥ 2 2 ) ν j ( P ) ≲ ( log ( 2 p ) ) 3 / 2 and ∑ j = 1 p tr ( Π j − S j ) ν j ( P ) ≲ ( log ( 2 p ) ) 3 / 2 . \sum_{j=1}^{p}\left(1+a_j^2+\lVert m_j\rVert_2^2\right)\nu_j(P)\lesssim(\log(2p))^{3/2}
\qquad\text{and}\qquad
\sum_{j=1}^{p}\operatorname{tr}(\Pi_j-\mathcal{S}_j)\nu_j(P)\lesssim(\log(2p))^{3/2}. j = 1 ∑ p ( 1 + a j 2 + ∥ m j ∥ 2 2 ) ν j ( P ) ≲ ( log ( 2 p ) ) 3/2 and j = 1 ∑ p tr ( Π j − S j ) ν j ( P ) ≲ ( log ( 2 p ) ) 3/2 .
These two bounds have been established by Gao and Zhang [23 ] for polyhedra that satisfy a few basic conditions, and so it is enough to check that they are satisfied by P P P . (For the purpose of referencing the cited results, the notation in Gao and Zhang [23 ] corresponds to ours via m P = p m_P=p m P = p , s j ↔ ν j ( P ) s_j\leftrightarrow\nu_j(P) s j ↔ ν j ( P ) , K j ↔ S j K_j\leftrightarrow\mathcal{S}_j K j ↔ S j , μ j ↔ m j \mu_j\leftrightarrow m_j μ j ↔ m j , Δ j ↔ tr ( Π j − S j ) \Delta_j\leftrightarrow\operatorname{tr}(\Pi_j-\mathcal{S}_j) Δ j ↔ tr ( Π j − S j ) , and Q j ↔ K j Q_j\leftrightarrow K_j Q j ↔ K j .)
The conditions for P P P that we must check are (1) that it is a proper subset of R p \mathbb{R}^{p} R p , (2) that it is non-empty, (3) that it has non-empty interior, and (4) that none of the constraint inequalities in (9) are redundant (i.e. none may be removed without changing P P P ). All four conditions can be checked using the observation that for any ( b 1 , … , b p ) ∈ R p (b_1,\ldots,b_p)\in\mathbb{R}^{p} ( b 1 , … , b p ) ∈ R p , there is some z ∈ R p z\in\mathbb{R}^{p} z ∈ R p such that
⟨ n j , z ⟩ = b j for all j ≤ p , (57) \langle n_j,z\rangle=b_j\qquad\text{for all }j\leq p, \tag*{(57)} ⟨ n j , z ⟩ = b j for all j ≤ p , (57)
because the unit normal vectors n 1 , … , n p n_1,\ldots,n_p n 1 , … , n p are linearly independent. The first two conditions are trivial and so we only give details for the latter two. For condition (3), let z z z correspond to the choice b j = a j − 1 b_j=a_j-1 b j = a j − 1 for all j ≤ p j\leq p j ≤ p , and notice that for any u ∈ R p u\in\mathbb{R}^{p} u ∈ R p with ∥ u ∥ 2 ≤ 1 \lVert u\rVert_2\leq1 ∥ u ∥ 2 ≤ 1 , we have ⟨ n j , z + u ⟩ ≤ a j − 1 + ∥ n j ∥ 2 ∥ u ∥ 2 ≤ a j \langle n_j,z+u\rangle\leq a_j-1+\lVert n_j\rVert_2\lVert u\rVert_2\leq a_j ⟨ n j , z + u ⟩ ≤ a j − 1 + ∥ n j ∥ 2 ∥ u ∥ 2 ≤ a j for all j j j . In other words, the unit ℓ 2 \ell_2 ℓ 2 -ball centered at z z z is contained in P P P , and so P P P has non-empty interior. For condition (4), fix any j 0 ∈ { 1 , … , p } j_0\in\{1,\ldots,p\} j 0 ∈ { 1 , … , p } and let z ′ z' z ′ be the vector that satisfies ⟨ n j , z ′ ⟩ = a j \langle n_j,z'\rangle=a_j ⟨ n j , z ′ ⟩ = a j for all j ≠ j 0 j\ne j_0 j = j 0 , as well as ⟨ n j 0 , z ′ ⟩ = a j 0 + 1 \langle n_{j_0},z'\rangle=a_{j_0}+1 ⟨ n j 0 , z ′ ⟩ = a j 0 + 1 . Hence, we have z ′ ∉ P z'\notin P z ′ ∈ / P , but if the constraint inequality corresponding to j 0 j_0 j 0 were removed from the definition of P P P , then z ′ z' z ′ would be included in P P P , and thus condition (4) is satisfied.
Part (b). We first bound F ( P ) \mathcal{F}(P) F ( P ) by two simpler sums. Recall that ν j ( P ) = ϕ ( a j ) q j \nu_j(P)=\phi(a_j)q_j ν j ( P ) = ϕ ( a j ) q j with 0 < q j ≤ 1 0<q_j\leq1 0 < q j ≤ 1 (Lemma 4.1 ). Also note that tr ( Π j − S j ) ≤ tr ( Π j ) = p − 1 \operatorname{tr}(\Pi_j-\mathcal{S}_j)\leq\operatorname{tr}(\Pi_j)=p-1 tr ( Π j − S j ) ≤ tr ( Π j ) = p − 1 and ∥ m j ∥ 2 2 ≤ 2 log ( 1 / q j ) \lVert m_j\rVert_2^2\leq2\log(1/q_j) ∥ m j ∥ 2 2 ≤ 2 log ( 1/ q j ) , where the latter bound follows from letting the subspace E j E_j E j play the role of Euclidean space in Lemma 8.5 (a). Applying these facts to (56) leads to
F ( P ) ≲ ∑ j = 1 p ( p + a j 2 ) ν j ( P ) + ∑ j = 1 p ϕ ( a j ) q j log ( 1 / q j ) . (58) \mathcal{F}(P)\lesssim\sum_{j=1}^{p}(p+a_j^2)\nu_j(P)+\sum_{j=1}^{p}\phi(a_j)q_j\log(1/q_j). \tag*{(58)} F ( P ) ≲ j = 1 ∑ p ( p + a j 2 ) ν j ( P ) + j = 1 ∑ p ϕ ( a j ) q j log ( 1/ q j ) . (58)
The main step is to show that every facet has small Gaussian surface measure, in the sense that
ν j ( P ) ≲ e − r 2 / 8 for all j ≤ p . (59) \nu_j(P)\lesssim e^{-r^2/8}\qquad\text{for all }j\leq p. \tag*{(59)} ν j ( P ) ≲ e − r 2 /8 for all j ≤ p . (59)
To prove this, we will consider an arbitrary fixed j j j and then separately address the two cases when ∣ a j ∣ ≥ r / 2 |a_j|\geq r/2 ∣ a j ∣ ≥ r /2 and ∣ a j ∣ < r / 2 |a_j|<r/2 ∣ a j ∣ < r /2 . In the first case, the bound (59) holds because
ν j ( P ) = ϕ ( a j ) q j ≤ ϕ ( a j ) ≤ ϕ ( r / 2 ) ≲ e − r 2 / 8 . \nu_j(P)=\phi(a_j)q_j\leq\phi(a_j)\leq\phi(r/2)\lesssim e^{-r^2/8}. ν j ( P ) = ϕ ( a j ) q j ≤ ϕ ( a j ) ≤ ϕ ( r /2 ) ≲ e − r 2 /8 .
The second case ∣ a j ∣ < r / 2 |a_j|<r/2 ∣ a j ∣ < r /2 is more involved. Let l ⋆ l^\star l ⋆ denote the minimizing index satisfying a l ⋆ = min l ≤ p a l a_{l^\star}=\min_{l\le p}a_l a l ⋆ = min l ≤ p a l and let W ∼ γ E j W\sim\gamma_{E_j} W ∼ γ E j . Observe that the event W ∈ K j W\in K_j W ∈ K j is the same as ( a j n j + W ) ∈ F j (a_jn_j+W)\in F_j ( a j n j + W ) ∈ F j , which implies that the point ( a j n j + W ) (a_jn_j+W) ( a j n j + W ) is in P P P , and hence satisfies the l ⋆ l^\star l ⋆ th inequality in P P P . Thus, { W ∈ K j } ⊂ { ⟨ n l ⋆ , a j n j + W ⟩ ≤ a l ⋆ } \{W\in K_j\}\subset\{\langle n_{l^\star},a_jn_j+W\rangle\le a_{l^\star}\} { W ∈ K j } ⊂ {⟨ n l ⋆ , a j n j + W ⟩ ≤ a l ⋆ } . Furthermore, because we are in the case that ∣ a j ∣ < r / 2 |a_j|<r/2 ∣ a j ∣ < r /2 , the assumption ∣ min l ≤ p a l ∣ ≥ r |\min_{l\le p}a_l|\ge r ∣ min l ≤ p a l ∣ ≥ r implies a l ⋆ ≤ − r a_{l^\star}\le-r a l ⋆ ≤ − r , and so the last few observations give
q j = γ E j ( K j ) = P ( W ∈ K j ) ≤ P ( ⟨ n l ⋆ , a j n j + W ⟩ ≤ − r ) . q_j=\gamma_{E_j}(K_j)=\mathbf{P}(W\in K_j)\le\mathbf{P}\left(\langle n_{l^\star},a_jn_j+W\rangle\le-r\right). q j = γ E j ( K j ) = P ( W ∈ K j ) ≤ P ( ⟨ n l ⋆ , a j n j + W ⟩ ≤ − r ) .
Furthermore, note that ⟨ n l ⋆ , a j n j + W ⟩ \langle n_{l^\star},a_jn_j+W\rangle ⟨ n l ⋆ , a j n j + W ⟩ is Gaussian with mean a j ⟨ n l ⋆ , n j ⟩ ≥ − r / 2 a_j\langle n_{l^\star},n_j\rangle\ge-r/2 a j ⟨ n l ⋆ , n j ⟩ ≥ − r /2 and variance ∥ Π j n l ⋆ ∥ 2 2 ≤ 1 \|\Pi_jn_{l^\star}\|_2^2\le1 ∥ Π j n l ⋆ ∥ 2 2 ≤ 1 , which means that the last probability is at most 1 − Φ ( r / 2 ) ≤ e − r 2 / 8 1-\Phi(r/2)\le e^{-r^2/8} 1 − Φ ( r /2 ) ≤ e − r 2 /8 . Combining this with ν j ( P ) = ϕ ( a j ) q j ≤ q j \nu_j(P)=\phi(a_j)q_j\le q_j ν j ( P ) = ϕ ( a j ) q j ≤ q j proves (59) .
It remains to bound the two sums in (58) . For the terms in the first sum, we take the geometric mean of the upper bounds (6.4) and ν j ( P ) ≤ ϕ ( a j ) \nu_j(P)\le\phi(a_j) ν j ( P ) ≤ ϕ ( a j ) to get
( p + a j 2 ) ν j ( P ) ≲ ( p + a j 2 ) ϕ ( a j ) e − r 2 / 8 (p+a_j^2)\nu_j(P)\lesssim(p+a_j^2)\sqrt{\phi(a_j)e^{-r^2/8}} ( p + a j 2 ) ν j ( P ) ≲ ( p + a j 2 ) ϕ ( a j ) e − r 2 /8
≲ ( p + sup a ∈ R ( a 2 e − a 2 / 4 ) ) e − r 2 / 16 \lesssim\left(p+\sup_{a\in\mathbb{R}}(a^2e^{-a^2/4})\right)e^{-r^2/16} ≲ ( p + a ∈ R sup ( a 2 e − a 2 /4 ) ) e − r 2 /16
≲ p e − r 2 / 16 . (60) \lesssim pe^{-r^2/16}. \tag*{(60)} ≲ p e − r 2 /16 . (60)
For the terms in the second sum in (58) , the elementary inequality q log ( 1 / q ) ≤ q q\log(1/q)\le\sqrt{q} q log ( 1/ q ) ≤ q for q ∈ ( 0 , 1 ] q\in(0,1] q ∈ ( 0 , 1 ] , and the formula q j = ν j ( P ) / ϕ ( a j ) \sqrt{q_j}=\sqrt{\nu_j(P)/\phi(a_j)} q j = ν j ( P ) / ϕ ( a j ) give
ϕ ( a j ) q j log ( 1 / q j ) ≤ ϕ ( a j ) ν j ( P ) ≤ e − r 2 / 16 , (61) \phi(a_j)q_j\log(1/q_j)\le\sqrt{\phi(a_j)\nu_j(P)}\le e^{-r^2/16}, \tag*{(61)} ϕ ( a j ) q j log ( 1/ q j ) ≤ ϕ ( a j ) ν j ( P ) ≤ e − r 2 /16 , (61)
where we have used (59) in the second step. Summing the bounds in (60) and (61) over j = 1 , … , p j=1,\ldots,p j = 1 , … , p produces the stated result for part (b).
Part (c). Let r > 0 r>0 r > 0 and consider multiplying F ( y ) \mathcal{F}(y) F ( y ) by the indicator 1 { ∣ min j ≤ p a j ( y ) ∣ ≥ r } 1\{|\min_{j\le p}a_j(y)|\ge r\} 1 { ∣ min j ≤ p a j ( y ) ∣ ≥ r } and its complementary version. Since the bound in part (a) holds uniformly with respect to all choices of P ( y ) P(y) P ( y ) , and since we can apply the bound in part (b) when the condition ∣ min j ≤ p ( a j ( y ) ) ∣ ≥ r |\min_{j\le p}(a_j(y))|\ge r ∣ min j ≤ p ( a j ( y )) ∣ ≥ r holds, it follows that
F ( y ) ≲ ( log ( p n ) ) 3 / 2 1 { ∣ min j ≤ p a j ( y ) ∣ < r } + p 2 e − r 2 / 16 . \mathcal{F}(y)\lesssim(\log(pn))^{3/2}1\left\{\left|\min_{j\le p}a_j(y)\right|<r\right\}+p^2e^{-r^2/16}. F ( y ) ≲ ( log ( p n ) ) 3/2 1 { j ≤ p min a j ( y ) < r } + p 2 e − r 2 /16 .
Taking r = 8 log ( p n ) r=8\sqrt{\log(pn)} r = 8 log ( p n ) and noting that min j ≤ p a j ( y ) = 1 u + σ 2 min j ≤ p ( β j − y j ) \min_{j\le p}a_j(y)=\frac{1}{\sqrt{u+\sigma^2}}\min_{j\le p}(\beta_j-y_j) min j ≤ p a j ( y ) = u + σ 2 1 min j ≤ p ( β j − y j ) leads to the stated result for part (c).
Proofs of Lemmas 3.2 and 4.8
Proof of Lemma 3.2. As a temporary shorthand, let f ( y ) = P ( y + σ N ≤ β ) = ∏ j Φ ( ( β j − y j ) / σ ) f(y)=\mathbf{P}(y+\sigma N\le\beta)=\prod_j\Phi((\beta_j-y_j)/\sigma) f ( y ) = P ( y + σ N ≤ β ) = ∏ j Φ (( β j − y j ) / σ ) , so that the smoothed orthant indicator is g u ( y ) = E f ( y + u G ) g_u(y)=\mathbf{E}f(y+\sqrt{u}G) g u ( y ) = E f ( y + u G ) . Differentiating in u u u and applying Stein’s identity [39 eqn. (A.17]
E ( G j φ ( G ) ) = ∑ l = 1 p Σ j l E ∂ l φ ( G ) (62) \mathbf{E}(G_j\varphi(G))=\sum_{l=1}^{p}\Sigma_{jl}\mathbf{E}\partial_l\varphi(G) \tag*{(62)} E ( G j φ ( G )) = l = 1 ∑ p Σ j l E ∂ l φ ( G ) (62)
to the function φ ( G ) = ∂ j f ( y + u G ) \varphi(G)=\partial_jf(y+\sqrt{u}G) φ ( G ) = ∂ j f ( y + u G ) , it follows that
∂ u g u ( y ) = 1 2 Δ Σ g u ( y ) . \partial_ug_u(y)=\frac{1}{2}\Delta_{\Sigma}g_u(y). ∂ u g u ( y ) = 2 1 Δ Σ g u ( y ) .
In turn, by the chain rule and recalling u ( t ) = 1 − t + σ 2 u(t)=1-t+\sigma^2 u ( t ) = 1 − t + σ 2 , we have
d d t g u ( t ) ( t S ) = 1 2 t ⟨ ∇ g u ( t ) ( t S ) , S ⟩ − 1 2 Δ Σ g u ( t ) ( t S ) . (63) \frac{d}{dt}g_{u(t)}(\sqrt{t}S)=\frac{1}{2\sqrt{t}}\langle\nabla g_{u(t)}(\sqrt{t}S),S\rangle-\frac{1}{2}\Delta\Sigma g_{u(t)}(\sqrt{t}S). \tag*{(63)} d t d g u ( t ) ( t S ) = 2 t 1 ⟨ ∇ g u ( t ) ( t S ) , S ⟩ − 2 1 ΔΣ g u ( t ) ( t S ) . (63)
Finally, note that the first term on the right side of (63) matches the sum on the right side of Lemma 3.2 because E ⟨ ∇ g u ( t ) ( t S \ i ) , X i / k ⟩ = 0 \mathbb{E}\langle\nabla g_{u(t)}(\sqrt{t}S_{\backslash i}),X_i/\sqrt{k}\rangle=0 E ⟨ ∇ g u ( t ) ( t S \ i ) , X i / k ⟩ = 0 for each i i i .
Proof of Lemma 4.8. Both parts rely on a bound for ∂ w log q j ( x ) \partial_w\log q_j(x) ∂ w log q j ( x ) , which we will derive first for any fixed j ≤ p j\le p j ≤ p . By (5) , we have ∥ w ∥ ∞ / σ ≤ C 0 − 1 ( log ( p n ) ) − 3 / 2 \lVert w\rVert_\infty/\sigma\le C_0^{-1}(\log(pn))^{-3/2} ∥ w ∥ ∞ / σ ≤ C 0 − 1 ( log ( p n ) ) − 3/2 , and so Lemma 4.5 gives
∣ ∂ w log q j ( x ) ∣ ≤ C C 0 ( log ( 1 / q j ( x ) ) log ( p n ) + ( log ( p n ) ) − 3 / 2 ) . (64) \lvert\partial_w\log q_j(x)\rvert\le\frac{C}{C_0}\left(\frac{\log(1/q_j(x))}{\log(pn)}+(\log(pn))^{-3/2}\right). \tag*{(64)} ∣ ∂ w log q j ( x )∣ ≤ C 0 C ( log ( p n ) log ( 1/ q j ( x )) + ( log ( p n ) ) − 3/2 ) . (64)
where C C C is the absolute constant implicit in Lemma 4.5.
Part (a). Here, j j j is assumed to be regular at x x x , and so if we let r = 20 log ( p n ) r=\sqrt{20\log(pn)} r = 20 log ( p n ) , then ∣ a j ( x ) ∣ ≤ r \lvert a_j(x)\rvert\le r ∣ a j ( x )∣ ≤ r and q j ( x ) ≥ ( p n ) − 10 q_j(x)\ge(pn)^{-10} q j ( x ) ≥ ( p n ) − 10 . We bound the two terms of ∂ w log ψ j ( x ) \partial_w\log\psi_j(x) ∂ w log ψ j ( x ) in (19) separately, which gives
∣ ∂ w log ψ j ( x ) ∣ ≤ ∥ w ∥ ∞ ∣ a j ( x ) ∣ u + σ 2 + ∣ ∂ w log q j ( x ) ∣ ≤ 20 C 0 log ( p n ) + 11 C C 0 , \lvert\partial_w\log\psi_j(x)\rvert\le\frac{\lVert w\rVert_\infty\lvert a_j(x)\rvert}{\sqrt{u+\sigma^2}}+\lvert\partial_w\log q_j(x)\rvert\le\frac{\sqrt{20}}{C_0\log(pn)}+\frac{11C}{C_0}, ∣ ∂ w log ψ j ( x )∣ ≤ u + σ 2 ∥ w ∥ ∞ ∣ a j ( x )∣ + ∣ ∂ w log q j ( x )∣ ≤ C 0 log ( p n ) 20 + C 0 11 C ,
where we used log ( 1 / q j ( x ) ) ≤ 10 log ( p n ) \log(1/q_j(x))\le10\log(pn) log ( 1/ q j ( x )) ≤ 10 log ( p n ) from (64) . Since C 0 C_0 C 0 may be taken to be large, the right side is at most 1 100 \frac{1}{100} 100 1 .
Part (b). Now suppose that j j j is not regular at x x x , and let τ ∈ [ 0 , 1 ] \tau\in[0,1] τ ∈ [ 0 , 1 ] . By the non-regularity, we may assume that either ∣ a j ( x ) ∣ > r \lvert a_j(x)\rvert>r ∣ a j ( x )∣ > r holds, or that the two conditions ∣ a j ( x ) ∣ ≤ r \lvert a_j(x)\rvert\le r ∣ a j ( x )∣ ≤ r and q j ( x ) < ( p n ) − 10 q_j(x)<(pn)^{-10} q j ( x ) < ( p n ) − 10 hold. We will treat these two cases separately, but in both cases we will use ν j = ϕ ( a j ) q j \nu_j=\phi(a_j)q_j ν j = ϕ ( a j ) q j with q j ≤ 1 q_j\le1 q j ≤ 1 (Lemma 4.1 ).
In the first case ∣ a j ( x ) ∣ > r \lvert a_j(x)\rvert>r ∣ a j ( x )∣ > r , the definition of the offset a j ( y ) = ( β j − y j ) / u + σ 2 a_j(y)=(\beta_j-y_j)/\sqrt{u+\sigma^2} a j ( y ) = ( β j − y j ) / u + σ 2 implies
∣ a j ( x + τ w ) − a j ( x ) ∣ = τ ∣ w j ∣ u + σ 2 ≤ ∥ w ∥ ∞ σ ≤ 1. \lvert a_j(x+\tau w)-a_j(x)\rvert=\frac{\tau\lvert w_j\rvert}{\sqrt{u+\sigma^2}}\le\frac{\lVert w\rVert_\infty}{\sigma}\le1. ∣ a j ( x + τ w ) − a j ( x )∣ = u + σ 2 τ ∣ w j ∣ ≤ σ ∥ w ∥ ∞ ≤ 1.
Hence ∣ a j ( x + τ w ) ∣ ≥ r − 1 \lvert a_j(x+\tau w)\rvert\ge r-1 ∣ a j ( x + τ w )∣ ≥ r − 1 , and so ν j ( x + τ w ) ≤ ϕ ( r − 1 ) ≤ ( p n ) − 5 \nu_j(x+\tau w)\le\phi(r-1)\le(pn)^{-5} ν j ( x + τ w ) ≤ ϕ ( r − 1 ) ≤ ( p n ) − 5 . For the other bound in (b), we use q log ( 1 / q ) ≤ 1 q\log(1/q)\le1 q log ( 1/ q ) ≤ 1 for q ∈ ( 0 , 1 ] q\in(0,1] q ∈ ( 0 , 1 ] , together with the fact that ( a + 2 ) ϕ ( a ) (a+2)\phi(a) ( a + 2 ) ϕ ( a ) is decreasing on [ 1 , ∞ ) [1,\infty) [ 1 , ∞ ) , to obtain
( ∣ a j ( x ) ∣ + log 1 q j ( x ) + 1 ) ν j ( x ) ≤ ( ∣ a j ( x ) ∣ + 2 ) ϕ ( a j ( x ) ) ≤ ( r + 2 ) ϕ ( r ) \begin{aligned}
\left(\lvert a_j(x)\rvert+\log\frac{1}{q_j(x)}+1\right)\nu_j(x)\le\left(\lvert a_j(x)\rvert+2\right)\phi(a_j(x)) \\
&\le(r+2)\phi(r)
\end{aligned} ( ∣ a j ( x )∣ + log q j ( x ) 1 + 1 ) ν j ( x ) ≤ ( ∣ a j ( x )∣ + 2 ) ϕ ( a j ( x )) ≤ ( r + 2 ) ϕ ( r )
≲ ( p n ) − 5 . (65) \lesssim(pn)^{-5}. \tag*{(65)} ≲ ( p n ) − 5 . (65)
Now we turn to the second case. Consider the function ϑ ( τ ) = log ( 1 / q j ( x + τ w ) ) \vartheta(\tau)=\log(1/q_j(x+\tau w)) ϑ ( τ ) = log ( 1/ q j ( x + τ w )) , which is convex because q j q_j q j is log-concave (Lemma 4.4 ). Its slope at the origin is ϑ ′ ( 0 ) = − ∂ w log q j ( x ) \vartheta'(0)=-\partial_w\log q_j(x) ϑ ′ ( 0 ) = − ∂ w log q j ( x ) , and so (64) gives ϑ ′ ( 0 ) ≥ − 1 4 ϑ ( 0 ) − 1 \vartheta'(0)\ge-\frac{1}{4}\vartheta(0)-1 ϑ ′ ( 0 ) ≥ − 4 1 ϑ ( 0 ) − 1 , as C 0 C_0 C 0 is large. (Here, the constants − 1 4 -\frac{1}{4} − 4 1 and − 1 -1 − 1 have been used for simplicity.) Thus,
ϑ ( τ ) ≥ ϑ ( 0 ) − τ ( 1 4 ϑ ( 0 ) + 1 ) ≥ 3 4 ϑ ( 0 ) − 1 \begin{aligned}
\vartheta(\tau)\ge\vartheta(0)-\tau\left(\frac{1}{4}\vartheta(0)+1\right) \\
&\ge\frac{3}{4}\vartheta(0)-1
\end{aligned} ϑ ( τ ) ≥ ϑ ( 0 ) − τ ( 4 1 ϑ ( 0 ) + 1 ) ≥ 4 3 ϑ ( 0 ) − 1
≥ 5 log ( p n ) , (66) \ge5\log(pn), \tag*{(66)} ≥ 5 log ( p n ) , (66)
where ϑ ( 0 ) > 10 log ( p n ) \vartheta(0)>10\log(pn) ϑ ( 0 ) > 10 log ( p n ) holds because we are considering the second case. Rewriting the previous display as an upper bound on q j ( x + τ w ) q_j(x+\tau w) q j ( x + τ w ) then leads to ν j ( x + τ w ) ≤ q j ( x + τ w ) ≤ ( p n ) − 5 \nu_j(x+\tau w)\le q_j(x+\tau w)\le(pn)^{-5} ν j ( x + τ w ) ≤ q j ( x + τ w ) ≤ ( p n ) − 5 . For the other bound in (b), we use ϕ ≤ 1 \phi\le1 ϕ ≤ 1 , and the fact that q ↦ ( r + 1 + log ( 1 / q ) ) q q\mapsto(r+1+\log(1/q))q q ↦ ( r + 1 + log ( 1/ q )) q is increasing on ( 0 , 1 ] (0,1] ( 0 , 1 ] to obtain
( ∣ a j ( x ) ∣ + log 1 q j ( x ) + 1 ) ν j ( x ) ≤ ( r + 1 + log 1 q j ( x ) ) q j ( x ) ≤ ( r + 1 + 10 log ( p n ) ) ( p n ) − 10 \begin{aligned}
\left(\lvert a_j(x)\rvert+\log\frac{1}{q_j(x)}+1\right)\nu_j(x)\le\left(r+1+\log\frac{1}{q_j(x)}\right)q_j(x) \\
&\le(r+1+10\log(pn))(pn)^{-10}
\end{aligned} ( ∣ a j ( x )∣ + log q j ( x ) 1 + 1 ) ν j ( x ) ≤ ( r + 1 + log q j ( x ) 1 ) q j ( x ) ≤ ( r + 1 + 10 log ( p n )) ( p n ) − 10
≲ ( p n ) − 5 . \lesssim(pn)^{-5}. ≲ ( p n ) − 5 .
Background results
This section collects the reduction lemma used in § 3 , results from the literature, a standard smoothing lemma, a lemma on the function ϕ / Φ \phi/\Phi ϕ /Φ , and elementary facts on the Gaussian measures of convex sets and of the polyhedra of § 3.1 .
Proof. First, the supremum in Theorem 2.1 can be replaced with a supremum over compact rectangles A = ∏ j [ α j , β j ] A=\prod_j[\alpha_j,\beta_j] A = ∏ j [ α j , β j ] with α j ≤ β j \alpha_j\le\beta_j α j ≤ β j . This is due to continuity of measure and the fact that taking a limit of compact rectangles approaching a general rectangle will not affect a bound that is uniform over compact rectangles.
For a compact rectangle A = ∏ j [ α j , β j ] A=\prod_j[\alpha_j,\beta_j] A = ∏ j [ α j , β j ] and a random vector U U U , observe that the event { U ∈ A } \{U\in A\} { U ∈ A } is equivalent to { ( U , − U ) ≤ ( β , − α ) } \{(U,-U)\le(\beta,-\alpha)\} {( U , − U ) ≤ ( β , − α )} , where α = ( α 1 , … , α p ) \alpha=(\alpha_1,\ldots,\alpha_p) α = ( α 1 , … , α p ) and β = ( β 1 , … , β p ) \beta=(\beta_1,\ldots,\beta_p) β = ( β 1 , … , β p ) . Furthermore, the vectors ( X 1 , − X 1 ) , … , ( X n , − X n ) (X_1,-X_1),\ldots,(X_n,-X_n) ( X 1 , − X 1 ) , … , ( X n , − X n ) are i.i.d. and centered in R 2 p \mathbb{R}^{2p} R 2 p , and their coordinates have unit variances and ψ 2 \psi_2 ψ 2 -norms at most B B B . Also, the Gaussian counterpart of these vectors is ( G , − G ) (G,-G) ( G , − G ) , and so it suffices to work with orthants in dimension 2 p 2p 2 p . (However, for simplicity we continue to write the dimension as p p p in the rest of this proof.)
It remains to show there is an absolute constant C > 0 C>0 C > 0 and centered i.i.d. random vectors Y 1 , … , Y n ∈ R p Y_1,\ldots,Y_n\in\mathbb{R}^p Y 1 , … , Y n ∈ R p with the following properties. These vectors must satisfy Var ( Y 1 j ) = 1 \operatorname{Var}(Y_{1j})=1 Var ( Y 1 j ) = 1 and ∣ Y 1 j ∣ ≤ B ‾ ≲ B log ( p n ) \lvert Y_{1j}\rvert\le\overline{B}\lesssim B\sqrt{\log(pn)} ∣ Y 1 j ∣ ≤ B ≲ B log ( p n ) almost surely for all j ≤ p j\le p j ≤ p , and also, if H H H is a centered Gaussian vector with covariance matrix E ( Y 1 Y 1 T ) \mathbb{E}(Y_1Y_1^\mathsf{T}) E ( Y 1 Y 1 T ) , then the bound below must hold for any β ∈ R p \beta\in\mathbb{R}^p β ∈ R p ,
∣ P ( 1 n ∑ i = 1 n X i ≤ β ) − P ( G ≤ β ) ∣ ≤ sup β ′ ∈ R p ∣ P ( 1 n ∑ i = 1 n Y i ≤ β ′ ) − P ( H ≤ β ′ ) ∣ + C n . \left\lvert\mathbb{P}\left(\frac{1}{\sqrt{n}}\sum_{i=1}^{n}X_i\le\beta\right)-\mathbb{P}(G\le\beta)\right\rvert
\le\sup_{\beta'\in\mathbb{R}^p}\left\lvert\mathbb{P}\left(\frac{1}{\sqrt{n}}\sum_{i=1}^{n}Y_i\le\beta'\right)-\mathbb{P}(H\le\beta')\right\rvert+\frac{C}{n}. P ( n 1 i = 1 ∑ n X i ≤ β ) − P ( G ≤ β ) ≤ β ′ ∈ R p sup P ( n 1 i = 1 ∑ n Y i ≤ β ′ ) − P ( H ≤ β ′ ) + n C .
We now construct each Y i Y_i Y i by truncation. We may assume without loss of generality that B 2 ≤ n B^2\le n B 2 ≤ n , because otherwise Theorem 2.1 holds trivially with C ≥ 1 C\ge1 C ≥ 1 . Also note that 1 = ( E X 1 j 2 ) 1 / 2 ≤ ( E X 1 j 4 ) 1 / 4 ≤ B 1=(\mathbb{E}X_{1j}^2)^{1/2}\le(\mathbb{E}X_{1j}^4)^{1/4}\le B 1 = ( E X 1 j 2 ) 1/2 ≤ ( E X 1 j 4 ) 1/4 ≤ B , and that if we define Ω i = { ∥ X i ∥ ∞ ≤ B 10 log ( p n ) } \Omega_i=\{\lVert X_i\rVert_\infty\le B\sqrt{10\log(pn)}\} Ω i = {∥ X i ∥ ∞ ≤ B 10 log ( p n ) } , then each Ω i \Omega_i Ω i fails with probability at most 2 ( p n ) − 9 2(pn)^{-9} 2 ( p n ) − 9 . To proceed, define the mean and variance parameters μ ~ = E ( X 1 1 Ω 1 ) \widetilde{\mu}=\mathbb{E}(X_1\mathbf{1}_{\Omega_1}) μ = E ( X 1 1 Ω 1 ) and σ ~ j 2 = Var ( X 1 j 1 Ω 1 ) \widetilde{\sigma}_j^2=\operatorname{Var}(X_{1j}\mathbf{1}_{\Omega_1}) σ j 2 = Var ( X 1 j 1 Ω 1 ) , and put
Y i j = 1 σ ~ j ( X i j 1 Ω i − μ ~ j ) . Y_{ij}=\frac{1}{\widetilde{\sigma}_j}\left(X_{ij}\mathbf{1}_{\Omega_i}-\widetilde{\mu}_j\right). Y ij = σ j 1 ( X ij 1 Ω i − μ j ) .
so that Y 1 , … , Y n Y_1,\ldots,Y_n Y 1 , … , Y n are centered and i.i.d. with standardized entries. Next, observe that the matrix
Δ = Σ − Cov ( X 1 1 Ω 1 ) = E ( X 1 X 1 T 1 Ω 1 c ) + μ ~ μ ~ T \Delta=\Sigma-\operatorname{Cov}(X_1\mathbf{1}_{\Omega_1})=\mathbb{E}(X_1X_1^\mathsf{T}\mathbf{1}_{\Omega_1^c})+\widetilde{\mu}\widetilde{\mu}^{\mathsf{T}} Δ = Σ − Cov ( X 1 1 Ω 1 ) = E ( X 1 X 1 T 1 Ω 1 c ) + μ μ T
is positive semidefinite, and the Cauchy–Schwarz inequality implies ∥ μ ~ ∥ ∞ ≤ ( p n ) − 4 \lVert\widetilde{\mu}\rVert_\infty\le(pn)^{-4} ∥ μ ∥ ∞ ≤ ( p n ) − 4 , as well as Δ j j ≤ 3 ( p n ) − 3 \Delta_{jj}\le3(pn)^{-3} Δ j j ≤ 3 ( p n ) − 3 . Since the diagonal entries of E ( X 1 X 1 T ) \mathbb{E}(X_1X_1^\mathsf{T}) E ( X 1 X 1 T ) are all equal to 1, it follows that σ ~ j 2 = 1 − Δ j j ≥ 1 4 \tilde{\sigma}_j^2 = 1 - \Delta_{jj} \ge\frac{1}{4} σ ~ j 2 = 1 − Δ j j ≥ 4 1 for all j ≤ p j \le p j ≤ p . Consequently, if we take B ˉ = 10 B log ( p n ) \bar{B} = 10B\sqrt{\log(pn)} B ˉ = 10 B log ( p n ) , then we have the almost-sure bound ∥ Y 1 ∥ ∞ ≤ B ˉ \lVert Y_1\rVert_\infty\le\bar{B} ∥ Y 1 ∥ ∞ ≤ B ˉ . Moreover, on the event ⋂ i ≤ n Ω i \bigcap_{i\le n}\Omega_i ⋂ i ≤ n Ω i , we have
{ 1 n ∑ i = 1 n X i ≤ β } = { 1 n ∑ i = 1 n Y i ≤ β ′ } , (67) \left\{\frac{1}{\sqrt{n}}\sum_{i=1}^{n} X_i \le\beta\right\} = \left\{\frac{1}{\sqrt{n}}\sum_{i=1}^{n} Y_i \le\beta'\right\}, \tag*{(67)} { n 1 i = 1 ∑ n X i ≤ β } = { n 1 i = 1 ∑ n Y i ≤ β ′ } , (67)
where we put β j ′ = 1 σ ~ j ( β j − n μ ~ j ) \beta'_j = \frac{1}{\tilde{\sigma}_j}(\beta_j-\sqrt{n}\tilde{\mu}_j) β j ′ = σ ~ j 1 ( β j − n μ ~ j ) for j ≤ p j \le p j ≤ p .
Due to (67) , the proof will be complete if we can show ∣ P ( H ≤ β ′ ) − P ( G ≤ β ) ∣ ≲ 1 / n \left|\mathbb{P}(H \le\beta')-\mathbb{P}(G \le\beta)\right| \lesssim1/n ∣ P ( H ≤ β ′ ) − P ( G ≤ β ) ∣ ≲ 1/ n . Since Δ ⪰ 0 \Delta\succeq0 Δ ⪰ 0 , we may represent G G G in distribution as G j = σ ~ j H j + D j G_j = \tilde{\sigma}_j H_j + D_j G j = σ ~ j H j + D j , where D ∼ N ( 0 , Δ ) D \sim\mathcal{N}(0,\Delta) D ∼ N ( 0 , Δ ) is independent of H H H . Thus, from the definition of β ′ \beta' β ′ , it is enough to show
∣ P ( H ≤ 1 σ ~ ( β − n μ ~ ) ) − P ( H ≤ 1 σ ~ ( β − D ) ) ∣ ≲ 1 n , (68) \left|\mathbb{P}\left(H \le\frac{1}{\tilde{\sigma}}(\beta-\sqrt{n}\tilde{\mu})\right)-\mathbb{P}\left(H \le\frac{1}{\tilde{\sigma}}(\beta-D)\right)\right| \lesssim\frac{1}{n}, \tag*{(68)} P ( H ≤ σ ~ 1 ( β − n μ ~ ) ) − P ( H ≤ σ ~ 1 ( β − D ) ) ≲ n 1 , (68)
where division by σ ~ \tilde{\sigma} σ ~ is understood in an entrywise sense. By construction, the coordinates of H H H satisfy Var ( H j ) = 1 \operatorname{Var}(H_j)=1 Var ( H j ) = 1 for j ≤ p j \le p j ≤ p , and it can be checked that ∥ 1 σ ~ ( D − n μ ~ ) ∥ ∞ > 4 ( p n ) − 1 \left\|\frac{1}{\tilde{\sigma}}(D-\sqrt{n}\tilde{\mu})\right\|_\infty> 4(pn)^{-1} σ ~ 1 ( D − n μ ~ ) ∞ > 4 ( p n ) − 1 occurs with probability at most O ( 1 / n ) O(1/n) O ( 1/ n ) . Consequently, the proof is completed by Nazarov’s inequality (Lemma 8.2). □
The following result is known as Nazarov’s inequality (Nazarov, [36 ] ; Chernozhukov, Chetverikov and Kato, [13 ] ).
Lemma 8.2. If G G G is a Gaussian vector in R d \mathbb{R}^d R d with min l ≤ d Var ( G l ) ≥ 1 \min_{l\le d}\operatorname{Var}(G_l)\ge1 min l ≤ d Var ( G l ) ≥ 1 , then for all y ∈ R d y\in\mathbb{R}^d y ∈ R d and δ > 0 \delta>0 δ > 0 we have
P ( G ≤ y + δ 1 ) − P ( G ≤ y ) ≤ δ ( 2 log d + 2 ) . \mathbb{P}(G\le y+\delta\mathbf{1})-\mathbb{P}(G\le y)\le\delta\left(\sqrt{2\log d}+2\right). P ( G ≤ y + δ 1 ) − P ( G ≤ y ) ≤ δ ( 2 log d + 2 ) .
The next result is a variant of a classical smoothing inequality. The proof is similar to that of Chernozhukov, Chetverikov and Koike (2023, Lemma 11.2) and hence omitted.
Lemma 8.3. Let p , n ≥ 2 p,n\ge2 p , n ≥ 2 and n − 1 / 2 ≤ σ ≤ 1 n^{-1/2}\le\sigma\le1 n − 1/2 ≤ σ ≤ 1 . Let S S S and G G G be random vectors in R p \mathbb{R}^p R p , where G G G is centered Gaussian with min j ≤ p Var ( G j ) ≥ 1 \min_{j\le p}\operatorname{Var}(G_j)\ge1 min j ≤ p Var ( G j ) ≥ 1 , and let W W W be a random vector in R p \mathbb{R}^p R p , independent of S S S and of G G G , with W j ∼ N ( 0 , 2 σ 2 ) W_j\sim\mathcal{N}(0,2\sigma^2) W j ∼ N ( 0 , 2 σ 2 ) for all j ≤ p j\le p j ≤ p . Then,
sup β ∈ R p ∣ P ( S ≤ β ) − P ( G ≤ β ) ∣ ≤ sup β ∈ R p ∣ P ( S + W ≤ β ) − P ( G + W ≤ β ) ∣ + C σ log ( p n ) . \sup_{\beta\in\mathbb{R}^p}\left|\mathbb{P}(S\le\beta)-\mathbb{P}(G\le\beta)\right|
\le
\sup_{\beta\in\mathbb{R}^p}\left|\mathbb{P}(S+W\le\beta)-\mathbb{P}(G+W\le\beta)\right|+C\sigma\log(pn). β ∈ R p sup ∣ P ( S ≤ β ) − P ( G ≤ β ) ∣ ≤ β ∈ R p sup ∣ P ( S + W ≤ β ) − P ( G + W ≤ β ) ∣ + C σ log ( p n ) .
The following elementary facts are used in § 5 . Below, we use s + = max ( s , 0 ) s_+=\max(s,0) s + = max ( s , 0 ) to denote the non-negative part of s ∈ R s\in\mathbb{R} s ∈ R .
Lemma 8.4. If λ = ( log Φ ) ′ = ϕ / Φ \lambda=(\log\Phi)'=\phi/\Phi λ = ( log Φ ) ′ = ϕ /Φ and ρ = − λ ′ \rho=-\lambda' ρ = − λ ′ , then the following statements hold.
(a) The function ρ \rho ρ is decreasing, with 0 < ρ < 1 0<\rho<1 0 < ρ < 1 . Also, the derivatives ρ ′ \rho' ρ ′ and ρ ′ ′ \rho'' ρ ′′ are bounded, and
∣ ρ ′ ( s ) ∣ ≤ ( 1 + s + ) ρ ( s ) and λ ( s ) ≤ 1 + ∣ s ∣ |\rho'(s)|\le(1+s_+)\rho(s) \qquad\text{and}\qquad\lambda(s)\le1+|s| ∣ ρ ′ ( s ) ∣ ≤ ( 1 + s + ) ρ ( s ) and λ ( s ) ≤ 1 + ∣ s ∣ for all real s s s .
(b) We have λ ( s ) ≤ 2 ( 1 + s + ) log ( 1 / Φ ( s ) ) \lambda(s)\le2(1+s_+)\log(1/\Phi(s)) λ ( s ) ≤ 2 ( 1 + s + ) log ( 1/Φ ( s )) for all real s s s , as well as
λ ( s ) ≤ 2 ϕ ( s ) and ρ ( s ) ≤ 2 ( 1 + s ) ϕ ( s ) \lambda(s)\le2\phi(s) \qquad\text{and}\qquad\rho(s)\le2(1+s)\phi(s) λ ( s ) ≤ 2 ϕ ( s ) and ρ ( s ) ≤ 2 ( 1 + s ) ϕ ( s ) for all s ≥ 0 s\ge0 s ≥ 0 .
(c) If p n pn p n is sufficiently large, and s , s ~ ∈ R s,\tilde{s}\in\mathbb{R} s , s ~ ∈ R satisfy ∣ s ~ − s ∣ ≤ 2 ( log ( p n ) ) − 3 / 2 |\tilde{s}-s|\le2(\log(pn))^{-3/2} ∣ s ~ − s ∣ ≤ 2 ( log ( p n ) ) − 3/2 , then
ρ ( s ~ ) ≤ 2 ρ ( s ) + ( p n ) − 5 . \rho(\tilde{s})\le2\rho(s)+(pn)^{-5}. ρ ( s ~ ) ≤ 2 ρ ( s ) + ( p n ) − 5 .
Proof. All three parts are consequences of the following standard facts, where ξ ∼ N ( 0 , 1 ) \xi\sim\mathcal{N}(0,1) ξ ∼ N ( 0 , 1 ) and m ( s ) = s + λ ( s ) m(s)=s+\lambda(s) m ( s ) = s + λ ( s ) . First, direct differentiation gives λ ′ = − λ m \lambda'=-\lambda m λ ′ = − λm , and so ρ = λ m \rho=\lambda m ρ = λm . Second, integration by parts gives
1 − ρ ( s ) = Var ( ξ ∣ ξ ≤ s ) . 1-\rho(s)=\operatorname{Var}(\xi\mid\xi\le s). 1 − ρ ( s ) = Var ( ξ ∣ ξ ≤ s ) .
Third, the Mills ratio satisfies ϕ ( x ) / ( 1 + x ) ≤ 1 − Φ ( x ) < ϕ ( x ) / x \phi(x)/(1+x)\le1-\Phi(x)<\phi(x)/x ϕ ( x ) / ( 1 + x ) ≤ 1 − Φ ( x ) < ϕ ( x ) / x for x > 0 x>0 x > 0 , where the lower bound is Birnbaum’s inequality [7 ] .
Part (a). The variance identity gives ρ < 1 \rho<1 ρ < 1 , and ρ \rho ρ is decreasing because Var ( ξ ∣ ξ ≤ s ) \operatorname{Var}(\xi\mid\xi\le s) Var ( ξ ∣ ξ ≤ s ) is increasing in s s s [25 Proposition 1] . The Mills ratio bounds give 0 < m ≤ 1 + s + 0<m\le1+s_+ 0 < m ≤ 1 + s + and λ ( s ) ≤ 1 + ∣ s ∣ \lambda(s)\le1+|s| λ ( s ) ≤ 1 + ∣ s ∣ , and so ρ = λ m > 0 \rho=\lambda m>0 ρ = λm > 0 . For the derivatives, note that m ′ = 1 − ρ m'=1-\rho m ′ = 1 − ρ . Differentiating ρ = λ m \rho=\lambda m ρ = λm then gives
ρ ′ = − λ ( m 2 − ( 1 − ρ ) ) and ρ ′ ′ = − ρ ′ ( m + λ ) − 2 ρ ( 1 − ρ ) . \rho'=-\lambda(m^2-(1-\rho)) \qquad\text{and}\qquad\rho''=-\rho'(m+\lambda)-2\rho(1-\rho). ρ ′ = − λ ( m 2 − ( 1 − ρ )) and ρ ′′ = − ρ ′ ( m + λ ) − 2 ρ ( 1 − ρ ) .
Since ρ \rho ρ is decreasing, the first formula gives ∣ ρ ′ ∣ ≤ λ m 2 = ρ m ≤ ( 1 + s + ) ρ |\rho'|\le\lambda m^2=\rho m\le(1+s_+)\rho ∣ ρ ′ ∣ ≤ λ m 2 = ρ m ≤ ( 1 + s + ) ρ . Using this in the second formula, together with λ m = ρ \lambda m=\rho λm = ρ and ρ < 1 \rho<1 ρ < 1 , gives
∣ ρ ′ ′ ∣ ≤ ρ m 2 + ρ 2 + 2 ρ ( 1 − ρ ) ≤ ( 1 + s + ) 2 ρ ( s ) + 1. |\rho''|\le\rho m^2+\rho^2+2\rho(1-\rho)\le(1+s_+)^2\rho(s)+1. ∣ ρ ′′ ∣ ≤ ρ m 2 + ρ 2 + 2 ρ ( 1 − ρ ) ≤ ( 1 + s + ) 2 ρ ( s ) + 1.
Both bounds are bounded functions of s s s , because ρ < 1 \rho<1 ρ < 1 , and because ρ ( s ) ≤ 2 ( 1 + s ) ϕ ( s ) \rho(s)\le2(1+s)\phi(s) ρ ( s ) ≤ 2 ( 1 + s ) ϕ ( s ) for s ≥ 0 s\ge0 s ≥ 0 by part (b), whose proof uses only m ≤ 1 + s + m\le1+s_+ m ≤ 1 + s + .
Part (b). We treat the cases s ≥ 0 s\ge0 s ≥ 0 and s < 0 s<0 s < 0 separately. First let s ≥ 0 s\ge0 s ≥ 0 . Then Φ ( s ) ≥ 1 2 \Phi(s)\ge\frac{1}{2} Φ ( s ) ≥ 2 1 gives λ ( s ) ≤ 2 ϕ ( s ) \lambda(s)\le2\phi(s) λ ( s ) ≤ 2 ϕ ( s ) , and together with m ( s ) ≤ 1 + s m(s)\le1+s m ( s ) ≤ 1 + s from (a), this gives ρ ( s ) = λ ( s ) m ( s ) ≤ 2 ( 1 + s ) ϕ ( s ) \rho(s)=\lambda(s)m(s)\le2(1+s)\phi(s) ρ ( s ) = λ ( s ) m ( s ) ≤ 2 ( 1 + s ) ϕ ( s ) . In addition, the lower Mills ratio bound and the inequality 1 − Φ ( s ) ≤ log ( 1 / Φ ( s ) ) 1-\Phi(s)\le\log(1/\Phi(s)) 1 − Φ ( s ) ≤ log ( 1/Φ ( s )) give ϕ ( s ) ≤ ( 1 + s ) log ( 1 / Φ ( s ) ) \phi(s)\le(1+s)\log(1/\Phi(s)) ϕ ( s ) ≤ ( 1 + s ) log ( 1/Φ ( s )) , which proves the first bound in (b) when s ≥ 0 s\ge0 s ≥ 0 .
Next, let s < 0 s<0 s < 0 . Since m > 0 m>0 m > 0 , we have λ ( r ) ≥ − r \lambda(r)\ge-r λ ( r ) ≥ − r for all r ∈ R r\in\mathbb{R} r ∈ R , and so
log Φ ( 0 ) − log Φ ( s ) = ∫ s 0 λ ( r ) d r ≥ s 2 2 . \log\Phi(0)-\log\Phi(s)=\int_s^0\lambda(r)\,dr\ge\frac{s^2}{2}. log Φ ( 0 ) − log Φ ( s ) = ∫ s 0 λ ( r ) d r ≥ 2 s 2 .
Combining this with the bound λ ( s ) ≤ 1 − s \lambda(s)\le1-s λ ( s ) ≤ 1 − s for negative s s s from (a), and the elementary inequality 1 − s ≤ 2 log 2 + s 2 1-s\le2\log2+s^2 1 − s ≤ 2 log 2 + s 2 , shows that λ ( s ) ≤ 2 log ( 1 / Φ ( s ) ) \lambda(s)\le2\log(1/\Phi(s)) λ ( s ) ≤ 2 log ( 1/Φ ( s )) , which proves the first bound in (b) when s < 0 s<0 s < 0 .
Part (c). Since ρ \rho ρ is decreasing, we may assume that s ~ = s − δ \widetilde{s}=s-\delta s = s − δ with 0 ≤ δ ≤ 2 ( log ( p n ) ) − 3 / 2 0\le\delta\le2(\log(pn))^{-3/2} 0 ≤ δ ≤ 2 ( log ( p n ) ) − 3/2 . We treat the cases s ≤ log ( p n ) s\le\log(pn) s ≤ log ( p n ) and s > log ( p n ) s>\log(pn) s > log ( p n ) separately. In the first case, the bound ∣ ρ ′ ( t ) ∣ ≤ ( 1 + log ( p n ) ) ρ ( t ) |\rho'(t)|\le(1+\log(pn))\rho(t) ∣ ρ ′ ( t ) ∣ ≤ ( 1 + log ( p n )) ρ ( t ) holds for all t ≤ s t\le s t ≤ s by (a), and integrating it over [ s − δ , s ] [s-\delta,s] [ s − δ , s ] gives
ρ ( s − δ ) ≤ ρ ( s ) e δ ( 1 + log ( p n ) ) ≤ 2 ρ ( s ) . \rho(s-\delta)\le\rho(s)e^{\delta(1+\log(pn))}\le2\rho(s). ρ ( s − δ ) ≤ ρ ( s ) e δ ( 1 + l o g ( p n )) ≤ 2 ρ ( s ) .
In the second case, we have s − δ ≥ log ( p n ) − 1 ≥ 1 s-\delta\ge\log(pn)-1\ge1 s − δ ≥ log ( p n ) − 1 ≥ 1 , and so the bound on ρ \rho ρ in (b) gives ρ ( s − δ ) ≲ log ( p n ) ϕ ( log ( p n ) − 1 ) ≤ ( p n ) − 5 \rho(s-\delta)\lesssim\log(pn)\phi(\log(pn)-1)\le(pn)^{-5} ρ ( s − δ ) ≲ log ( p n ) ϕ ( log ( p n ) − 1 ) ≤ ( p n ) − 5 . In both cases, the last inequality holds because p n pn p n is large.
Lemma 8.5. Let Z ∼ γ d Z\sim\gamma_d Z ∼ γ d , let A ⊂ R d A\subset\mathbb{R}^d A ⊂ R d be a closed convex set with γ d ( A ) > 0 \gamma_d(A)>0 γ d ( A ) > 0 , and let m A = E ( Z ∣ Z ∈ A ) m_A=\mathbb{E}(Z\mid Z\in A) m A = E ( Z ∣ Z ∈ A ) and S A = Cov ( Z ∣ Z ∈ A ) \mathcal{S}_A=\operatorname{Cov}(Z\mid Z\in A) S A = Cov ( Z ∣ Z ∈ A ) .
(a) The conditional mean satisfies ∥ m A ∥ 2 2 ≤ 2 log ( 1 / γ d ( A ) ) \|m_A\|_2^2\le2\log(1/\gamma_d(A)) ∥ m A ∥ 2 2 ≤ 2 log ( 1/ γ d ( A )) .
(b) The function x ↦ γ d ( A − x ) x\mapsto\gamma_d(A-x) x ↦ γ d ( A − x ) is C ∞ C^\infty C ∞ , positive and log-concave on R d \mathbb{R}^d R d . Also, the function x ↦ log γ d ( A − x ) x\mapsto\log\gamma_d(A-x) x ↦ log γ d ( A − x ) has gradient m A m_A m A and Hessian S A − I d ⪯ 0 \mathcal{S}_A-I_d\preceq0 S A − I d ⪯ 0 at x = 0 x=0 x = 0 .
The same statements hold for the standard Gaussian measure γ E \gamma_E γ E of a linear subspace E E E , with I d I_d I d replaced by the orthogonal projection onto E E E .
Proof. Part (a). Let ν \nu ν be the conditional distribution of Z Z Z given Z ∈ A Z \in A Z ∈ A , so that d ν / d γ d = 1 A / γ d ( A ) d\nu/d\gamma_d = \mathbf{1}_A/\gamma_d(A) d ν / d γ d = 1 A / γ d ( A ) and d K L ( ν ∥ γ d ) = log ( 1 / γ d ( A ) ) d_{\mathrm{KL}}(\nu\mathbin\Vert\gamma_d) = \log(1/\gamma_d(A)) d KL ( ν ∥ γ d ) = log ( 1/ γ d ( A )) . Applying the transportation inequality [8 Lemma 4.18] to the function z ↦ ⟨ m A , z ⟩ / ∥ m A ∥ 2 z \mapsto\langle m_A,z\rangle/\|m_A\|_2 z ↦ ⟨ m A , z ⟩ /∥ m A ∥ 2 (which is a standard Gaussian variable under γ d \gamma_d γ d ) it follows that ∥ m A ∥ 2 ≤ 2 log ( 1 / γ d ( A ) ) \|m_A\|_2 \le\sqrt{2\log(1/\gamma_d(A))} ∥ m A ∥ 2 ≤ 2 log ( 1/ γ d ( A )) .
Part (b). We write γ d ( A − x ) = ∫ A ϕ d ( z − x ) d z \gamma_d(A-x)=\int_A\phi_d(z-x)\,dz γ d ( A − x ) = ∫ A ϕ d ( z − x ) d z . The positivity and smoothness are clear. To address the log-concavity, observe that the convexity of A A A implies that the function ( x , z ) ↦ 1 A ( z ) ϕ d ( z − x ) (x,z)\mapsto\mathbf{1}_A(z)\phi_d(z-x) ( x , z ) ↦ 1 A ( z ) ϕ d ( z − x ) is log-concave. This means x ↦ γ d ( A − x ) x\mapsto\gamma_d(A-x) x ↦ γ d ( A − x ) is obtained from marginalizing a log-concave function, and is hence log-concave [38 ] . For the gradient and Hessian expressions, first note that when ∇ x ϕ d ( z − x ) \nabla_x\phi_d(z-x) ∇ x ϕ d ( z − x ) and ∇ x 2 ϕ d ( z − x ) \nabla_x^2\phi_d(z-x) ∇ x 2 ϕ d ( z − x ) are evaluated at x = 0 x=0 x = 0 , they become z ϕ d ( z ) z\phi_d(z) z ϕ d ( z ) and ( z z T − I d ) ϕ d ( z ) (zz^{\mathsf{T}}-I_d)\phi_d(z) ( z z T − I d ) ϕ d ( z ) . Second, let ν \nu ν be as in the proof of part (a) and recall that d ν / d γ d = 1 A / γ d ( A ) d\nu/d\gamma_d=\mathbf{1}_A/\gamma_d(A) d ν / d γ d = 1 A / γ d ( A ) . A direct calculation of ∇ x log γ d ( A − x ) ∣ x = 0 \nabla_x\log\gamma_d(A-x)|_{x=0} ∇ x log γ d ( A − x ) ∣ x = 0 then gives
∇ x log γ d ( A − x ) ∣ x = 0 = ∫ R d z ( d ν / d γ d ) d γ d ( z ) = ∫ R d z d ν ( z ) = m A . \nabla_x\log\gamma_d(A-x)|_{x=0}=\int_{\mathbb{R}^d}z(d\nu/d\gamma_d)\,d\gamma_d(z)=\int_{\mathbb{R}^d}z\,d\nu(z)=m_A. ∇ x log γ d ( A − x ) ∣ x = 0 = ∫ R d z ( d ν / d γ d ) d γ d ( z ) = ∫ R d z d ν ( z ) = m A .
and the calculation for the Hessian is similar. Meanwhile, the condition S A − I d ⪯ 0 S_A-I_d\preceq0 S A − I d ⪯ 0 is due to the log-concavity of x ↦ γ d ( A − x ) x\mapsto\gamma_d(A-x) x ↦ γ d ( A − x ) . Lastly, for a subspace E E E , the same arguments apply after identifying E E E with a Euclidean space of the same dimension. □ \square □
Polyhedra with linearly independent normal vectors
In this subsection, we prove Lemmas 4.1 and 4.4 , and two further lemmas, on the derivatives of the Gaussian measures of P P P and of its facets under translations.
Proof of Lemma 4.1. The representation of the j j j th facet as F j = a j n j + K j F_j=a_jn_j+K_j F j = a j n j + K j is straightforward to check, as is its closedness and convexity. To show that q j q_j q j is positive, first let H j = { z ∈ R p : ⟨ n j , z ⟩ = a j } H_j=\{z\in\mathbb{R}^p:\langle n_j,z\rangle=a_j\} H j = { z ∈ R p : ⟨ n j , z ⟩ = a j } denote the hyperplane containing the j j j th facet, so that F j = P ∩ H j F_j=P\cap H_j F j = P ∩ H j . By (57) there is a point z ∘ ∈ R p z^\circ\in\mathbb{R}^p z ∘ ∈ R p with ⟨ n j , z ∘ ⟩ = a j \langle n_j,z^\circ\rangle=a_j ⟨ n j , z ∘ ⟩ = a j and ⟨ n l , z ∘ ⟩ = a l − 1 \langle n_l,z^\circ\rangle=a_l-1 ⟨ n l , z ∘ ⟩ = a l − 1 for l ≠ j l\ne j l = j . Consequently, if a point z ∈ H j z\in H_j z ∈ H j satisfies ∥ z − z ∘ ∥ 2 ≤ 1 \|z-z^\circ\|_2\le1 ∥ z − z ∘ ∥ 2 ≤ 1 , then,
⟨ n l , z ⟩ ≤ ⟨ n l , z ∘ ⟩ + ∥ z − z ∘ ∥ 2 ≤ a l for all l ≠ j . \langle n_l,z\rangle\le\langle n_l,z^\circ\rangle+\|z-z^\circ\|_2\le a_l\qquad\text{for all }l\ne j. ⟨ n l , z ⟩ ≤ ⟨ n l , z ∘ ⟩ + ∥ z − z ∘ ∥ 2 ≤ a l for all l = j .
This means that the unit ℓ 2 \ell_2 ℓ 2 -ball of H j H_j H j centered at z ∘ z^\circ z ∘ is contained in F j F_j F j , and hence, that F j F_j F j has non-empty interior relative to H j H_j H j . Lastly, because K j K_j K j and E j E_j E j are obtained from a common shift of F j F_j F j and H j H_j H j , this implies that K j K_j K j has non-empty interior relative to E j E_j E j , and thus q j = γ E j ( K j ) q_j=\gamma_{E_j}(K_j) q j = γ E j ( K j ) is positive.
Finally, we verify the formula ν j ( P ) = ϕ ( a j ) q j \nu_j(P)=\phi(a_j)q_j ν j ( P ) = ϕ ( a j ) q j . For any z ′ ∈ E j z'\in E_j z ′ ∈ E j we have ∥ a j n j + z ′ ∥ 2 2 = a j 2 + ∥ z ′ ∥ 2 2 \|a_jn_j+z'\|_2^2=a_j^2+\|z'\|_2^2 ∥ a j n j + z ′ ∥ 2 2 = a j 2 + ∥ z ′ ∥ 2 2 , and so ϕ p ( a j n j + z ′ ) = ϕ ( a j ) ϕ p − 1 ( z ′ ) \phi_p(a_jn_j+z')=\phi(a_j)\phi_{p-1}(z') ϕ p ( a j n j + z ′ ) = ϕ ( a j ) ϕ p − 1 ( z ′ ) . Since the subspace E j E_j E j is isometric to the hyperplane H j H_j H j mentioned a moment ago, it follows that
ν j ( P ) = ∫ F j ϕ p d H p − 1 = ∫ K j ϕ p ( a j n j + z ′ ) d H p − 1 ( z ′ ) = ϕ ( a j ) ∫ K j ϕ p − 1 d H p − 1 = ϕ ( a j ) q j . \nu_j(P)=\int_{F_j}\phi_p\,d\mathcal{H}^{p-1}=\int_{K_j}\phi_p(a_jn_j+z')\,d\mathcal{H}^{p-1}(z')=\phi(a_j)\int_{K_j}\phi_{p-1}\,d\mathcal{H}^{p-1}=\phi(a_j)q_j. ν j ( P ) = ∫ F j ϕ p d H p − 1 = ∫ K j ϕ p ( a j n j + z ′ ) d H p − 1 ( z ′ ) = ϕ ( a j ) ∫ K j ϕ p − 1 d H p − 1 = ϕ ( a j ) q j .
The content of the following result might be regarded as classical, but we provide a proof since the statements do not seem to be easily referenced in the form below.
Lemma 8.6. Let P ⊂ R p P \subset\mathbb{R}^{p} P ⊂ R p be a polyhedron as in (9) with p p p facets having linearly independent normal vectors, and let h = π ϕ p h=\pi\phi_{p} h = π ϕ p with π \pi π being a polynomial on R p \mathbb{R}^{p} R p . Then, the function x ↦ ∫ P − x h d z x\mapsto\int_{P-x}h\,dz x ↦ ∫ P − x h d z is C ∞ C^{\infty} C ∞ on R p \mathbb{R}^{p} R p , and for all x , x ′ ∈ R p x,x'\in\mathbb{R}^{p} x , x ′ ∈ R p we have
∂ x ′ ∫ P − x h d z = − ∑ j = 1 p ⟨ n j , x ′ ⟩ ∫ F j − x h d H p − 1 , (69) \partial_{x'}\int_{P-x}h\,dz=-\sum_{j=1}^{p}\langle n_{j},x'\rangle\int_{F_{j}-x}h\,d\mathcal{H}^{p-1}, \tag*{(69)} ∂ x ′ ∫ P − x h d z = − j = 1 ∑ p ⟨ n j , x ′ ⟩ ∫ F j − x h d H p − 1 , (69) where F j − x F_{j}-x F j − x is the j j j th facet of P − x P-x P − x . Furthermore, the following hold.
(a) The first derivatives are given by ∂ x ′ γ p ( P − x ) = − ∑ j ⟨ n j , x ′ ⟩ ν j ( P − x ) \partial_{x'}\gamma_{p}(P-x)=-\sum_{j}\langle n_{j},x'\rangle\nu_{j}(P-x) ∂ x ′ γ p ( P − x ) = − ∑ j ⟨ n j , x ′ ⟩ ν j ( P − x ) .
(b) The second and third derivatives are given by
∇ x 2 γ p ( P − x ) = ∫ P − x ( z z T − I p ) ϕ p d z , \nabla_{x}^{2}\gamma_{p}(P-x)=\int_{P-x}(zz^{\mathsf{T}}-I_{p})\phi_{p}\,dz, ∇ x 2 γ p ( P − x ) = ∫ P − x ( z z T − I p ) ϕ p d z , ∂ x ′ ∇ x 2 γ p ( P − x ) = − ∑ j = 1 p ⟨ n j , x ′ ⟩ M j ( P − x ) , \partial_{x'}\nabla_{x}^{2}\gamma_{p}(P-x)=-\sum_{j=1}^{p}\langle n_{j},x'\rangle M_{j}(P-x), ∂ x ′ ∇ x 2 γ p ( P − x ) = − j = 1 ∑ p ⟨ n j , x ′ ⟩ M j ( P − x ) , where we put M j ( P ) = ∫ F j ( z z T − I p ) ϕ p d H p − 1 M_{j}(P)=\int_{F_{j}}(zz^{\mathsf{T}}-I_{p})\phi_{p}\,d\mathcal{H}^{p-1} M j ( P ) = ∫ F j ( z z T − I p ) ϕ p d H p − 1 .
(c) For every symmetric matrix Γ \Gamma Γ with 0 ⪯ Γ ⪯ I p 0\preceq\Gamma\preceq I_{p} 0 ⪯ Γ ⪯ I p and every j j j , we have
∣ tr ( Γ M j ( P ) ) ∣ ≤ ( 1 + a j 2 + ∥ m j ∥ 2 + tr ( Π j − S j ) ) ν j ( P ) . \left|\operatorname{tr}(\Gamma M_{j}(P))\right|\leq\left(1+a_{j}^{2}+\lVert m_{j}\rVert^{2}+\operatorname{tr}(\Pi_{j}-S_{j})\right)\nu_{j}(P). ∣ tr ( Γ M j ( P )) ∣ ≤ ( 1 + a j 2 + ∥ m j ∥ 2 + tr ( Π j − S j ) ) ν j ( P ) .
Proof. We first prove (69) . Since ∫ P − x h d z = ∫ P h ( z − x ) d z \int_{P-x}h\,dz=\int_{P}h(z-x)\,dz ∫ P − x h d z = ∫ P h ( z − x ) d z , differentiation under the integral shows that this function is C ∞ C^{\infty} C ∞ , with
∂ x ′ ∫ P − x h d z = − ∫ P − x ⟨ ∇ h , x ′ ⟩ d z . \partial_{x'}\int_{P-x}h\,dz=-\int_{P-x}\langle\nabla h,x'\rangle\,dz. ∂ x ′ ∫ P − x h d z = − ∫ P − x ⟨ ∇ h , x ′ ⟩ d z .
The boundary of P − x P-x P − x is the union of its facets F j − x F_{j}-x F j − x , and the outer unit normal on F j − x F_{j}-x F j − x is n j n_{j} n j . So the divergence theorem, applied to the vector field h x ′ hx' h x ′ on P − x P-x P − x , gives
∫ P − x ⟨ ∇ h , x ′ ⟩ d z = ∑ j = 1 p ⟨ n j , x ′ ⟩ ∫ F j − x h d H p − 1 , \int_{P-x}\langle\nabla h,x'\rangle\,dz=\sum_{j=1}^{p}\langle n_{j},x'\rangle\int_{F_{j}-x}h\,d\mathcal{H}^{p-1}, ∫ P − x ⟨ ∇ h , x ′ ⟩ d z = j = 1 ∑ p ⟨ n j , x ′ ⟩ ∫ F j − x h d H p − 1 ,
which is (69) . (Although P − x P-x P − x is unbounded, the divergence theorem can still be used by intersecting ( P − x ) (P-x) ( P − x ) with the ℓ 2 \ell_{2} ℓ 2 ball of radius r r r and then letting r → ∞ r\to\infty r → ∞ .)
Part (a). Taking π = 1 \pi=1 π = 1 in (69) proves (a).
Part (b). Since ∇ 2 ϕ p ( z ) = ( z z T − I p ) ϕ p ( z ) \nabla^{2}\phi_{p}(z)=(zz^{\mathsf{T}}-I_{p})\phi_{p}(z) ∇ 2 ϕ p ( z ) = ( z z T − I p ) ϕ p ( z ) , differentiating γ p ( P − x ) = ∫ P ϕ p ( z − x ) d z \gamma_{p}(P-x)=\int_{P}\phi_{p}(z-x)\,dz γ p ( P − x ) = ∫ P ϕ p ( z − x ) d z twice under the integral gives the first statement of (b). The second formula follows by applying (69) entrywise to the formula for ∇ x 2 γ ( P − x ) \nabla_{x}^{2}\gamma(P-x) ∇ x 2 γ ( P − x ) , while taking h ( z ) = π ( z ) ϕ p ( z ) h(z)=\pi(z)\phi_{p}(z) h ( z ) = π ( z ) ϕ p ( z ) with the polynomial π ( z ) = z i z l − 1 { i = l } \pi(z)=z_{i}z_{l}-1\{i=l\} π ( z ) = z i z l − 1 { i = l } .
Part (c). We first express M j ( P ) M_{j}(P) M j ( P ) in terms of the parameters of the j j j th section K j K_{j} K j . By Lemma 4.1 , every point of F j F_{j} F j can be written as z = a j n j + z ′ z=a_{j}n_{j}+z' z = a j n j + z ′ with z ′ ∈ K j z'\in K_{j} z ′ ∈ K j , and for such points we have ϕ p ( z ) = ϕ ( a j ) ϕ p − 1 ( z ′ ) \phi_{p}(z)=\phi(a_{j})\phi_{p-1}(z') ϕ p ( z ) = ϕ ( a j ) ϕ p − 1 ( z ′ ) . Changing variables from z z z to z ′ z' z ′ in the definition of M j ( P ) M_{j}(P) M j ( P ) gives
M j ( P ) = ϕ ( a j ) ∫ K j ( ( a j n j + z ′ ) ( a j n j + z ′ ) T − I p ) ϕ p − 1 ( z ′ ) d H p − 1 ( z ′ ) . M_{j}(P)=\phi(a_{j})\int_{K_{j}}\left((a_{j}n_{j}+z')(a_{j}n_{j}+z')^{\mathsf{T}}-I_{p}\right)\phi_{p-1}(z')\,d\mathcal{H}^{p-1}(z'). M j ( P ) = ϕ ( a j ) ∫ K j ( ( a j n j + z ′ ) ( a j n j + z ′ ) T − I p ) ϕ p − 1 ( z ′ ) d H p − 1 ( z ′ ) .
Next, let W ∼ γ E j W\sim\gamma_{E_{j}} W ∼ γ E j and U = a j n j + W U=a_{j}n_{j}+W U = a j n j + W . The integral above is the expectation of ( U U T − I p ) 1 { W ∈ K j } (UU^{\mathsf{T}}-I_{p})1\{W\in K_{j}\} ( U U T − I p ) 1 { W ∈ K j } , and so dividing and multiplying by q j = P ( W ∈ K j ) q_{j}=\mathbb{P}(W\in K_{j}) q j = P ( W ∈ K j ) and using ν j ( P ) = ϕ ( a j ) q j \nu_{j}(P)=\phi(a_{j})q_{j} ν j ( P ) = ϕ ( a j ) q j leads to
M j ( P ) = ν j ( P ) E ( U U T − I p ∣ W ∈ K j ) . M_{j}(P)=\nu_{j}(P)\mathbb{E}(UU^{\mathsf{T}}-I_{p}\mid W\in K_{j}). M j ( P ) = ν j ( P ) E ( U U T − I p ∣ W ∈ K j ) .
::: To evaluate the conditional expectation, recall that m j m_j m j and S j \mathfrak{S}_j S j are the mean and covariance of W W W given W ∈ K j W \in K_j W ∈ K j . Hence, the conditional mean and covariance of U U U are a j n j + m j a_j n_j + m_j a j n j + m j and S j \mathfrak{S}_j S j , which gives E ( U U T ∣ W ∈ K j ) = ( a j n j + m j ) ( a j n j + m j ) T + S j \mathbb{E}(UU^{\mathsf{T}} \mid W \in K_j) = (a_j n_j + m_j)(a_j n_j + m_j)^{\mathsf{T}} + \mathfrak{S}_j E ( U U T ∣ W ∈ K j ) = ( a j n j + m j ) ( a j n j + m j ) T + S j . Combining this with the algebraic identity I p = n j n j T + Π j I_p = n_j n_j^{\mathsf{T}} + \Pi_j I p = n j n j T + Π j , we obtain
M j ( P ) = ν j ( P ) [ ( a j n j + m j ) ( a j n j + m j ) T − n j n j T − ( Π j − S j ) ] . M_j(P) = \nu_j(P)\left[(a_j n_j + m_j)(a_j n_j + m_j)^{\mathsf{T}} - n_j n_j^{\mathsf{T}} - (\Pi_j - \mathfrak{S}_j)\right]. M j ( P ) = ν j ( P ) [ ( a j n j + m j ) ( a j n j + m j ) T − n j n j T − ( Π j − S j ) ] .
It remains to bound tr ( Γ M j ( P ) ) \operatorname{tr}(\Gamma M_j(P)) tr ( Γ M j ( P )) . The matrix in the brackets above has the form A 1 − A 2 A_1 - A_2 A 1 − A 2 , where A 1 = ( a j n j + m j ) ( a j n j + m j ) T A_1 = (a_j n_j + m_j)(a_j n_j + m_j)^{\mathsf{T}} A 1 = ( a j n j + m j ) ( a j n j + m j ) T and A 2 = n j n j T + ( Π j − S j ) A_2 = n_j n_j^{\mathsf{T}} + (\Pi_j - \mathfrak{S}_j) A 2 = n j n j T + ( Π j − S j ) are positive semidefinite, with Π j − S j ⪰ 0 \Pi_j - \mathfrak{S}_j \succeq0 Π j − S j ⪰ 0 (by Lemma 8.5 (b) on E j E_j E j ). Since 0 ⪯ Γ ⪯ I p 0 \preceq\Gamma\preceq I_p 0 ⪯ Γ ⪯ I p , every positive semidefinite matrix A A A satisfies 0 ≤ tr ( Γ A ) ≤ tr ( A ) 0 \leq\operatorname{tr}(\Gamma A) \leq\operatorname{tr}(A) 0 ≤ tr ( Γ A ) ≤ tr ( A ) , and so
∣ tr ( Γ M j ( P ) ) ∣ ν j ( P ) ≤ tr ( A 1 ) + tr ( A 2 ) = 1 + a j 2 + ∥ m j ∥ 2 2 + tr ( Π j − S j ) . \frac{\left|\operatorname{tr}(\Gamma M_j(P))\right|}{\nu_j(P)} \leq\operatorname{tr}(A_1) + \operatorname{tr}(A_2) = 1 + a_j^2 + \lVert m_j\rVert_2^2 + \operatorname{tr}(\Pi_j - \mathfrak{S}_j). ν j ( P ) ∣ tr ( Γ M j ( P )) ∣ ≤ tr ( A 1 ) + tr ( A 2 ) = 1 + a j 2 + ∥ m j ∥ 2 2 + tr ( Π j − S j ) .
where the last step uses the orthogonality of m j m_j m j and n j n_j n j .
Lemma 8.7
Lemma 8.7. Let P ⊂ R p P \subset\mathbb{R}^p P ⊂ R p be a polyhedron as in (9) with p p p facets having linearly independent normal vectors. Then, for all j ≤ p j \leq p j ≤ p and x ∈ R p x \in\mathbb{R}^p x ∈ R p , the j j j th section of P − x P - x P − x can be represented as K j − Π j x K_j - \Pi_jx K j − Π j x , and
ν j ( P − x ) = ϕ ( a j − ⟨ n j , x ⟩ ) q j ( P − x ) . \nu_j(P - x) = \phi(a_j - \langle n_j, x\rangle)q_j(P - x). ν j ( P − x ) = ϕ ( a j − ⟨ n j , x ⟩) q j ( P − x ) . Furthermore, both of the functions x ↦ q j ( P − x ) x \mapsto q_j(P - x) x ↦ q j ( P − x ) and x ↦ ν j ( P − x ) x \mapsto\nu_j(P - x) x ↦ ν j ( P − x ) are positive, C ∞ C^\infty C ∞ , and log-concave on R p \mathbb{R}^p R p . Lastly, at x = 0 x = 0 x = 0 , the function x ↦ log ν j ( P − x ) x \mapsto\log\nu_j(P - x) x ↦ log ν j ( P − x ) has gradient a j n j + m j a_j n_j + m_j a j n j + m j and Hessian S j − n j n j T − Π j ⪯ 0 \mathfrak{S}_j - n_j n_j^{\mathsf{T}} - \Pi_j \preceq0 S j − n j n j T − Π j ⪯ 0 .
Proof. We first identify the j j j th section of P − x P - x P − x . Note that the j j j th projection matrices of P − x P - x P − x and P P P are the same, because they have the same normal vectors. Also, it is simple to check that the j j j th section K j K_j K j and facet F j F_j F j of P P P are related by Π j F j = K j \Pi_j F_j = K_j Π j F j = K j . So, because the j j j th facet of P − x P - x P − x is given by F j − x F_j - x F j − x , it follows that the j j j th section of P − x P - x P − x is given by Π j ( F j − x ) = K j − Π j x \Pi_j(F_j - x) = K_j - \Pi_jx Π j ( F j − x ) = K j − Π j x . Taking Gaussian measures gives q j ( P − x ) = γ E j ( K j − Π j x ) q_j(P - x) = \gamma_{E_j}(K_j - \Pi_jx) q j ( P − x ) = γ E j ( K j − Π j x ) , and applying Lemma 4.1 to P − x P - x P − x gives the formula for ν j ( P − x ) \nu_j(P - x) ν j ( P − x ) .
It remains to establish the regularity properties and the derivatives at x = 0 x = 0 x = 0 . By the formula for ν j ( P − x ) \nu_j(P - x) ν j ( P − x ) , the function log ν j ( P − x ) \log\nu_j(P - x) log ν j ( P − x ) is the sum of log ϕ ( a j − ⟨ n j , x ⟩ ) \log\phi(a_j - \langle n_j, x\rangle) log ϕ ( a j − ⟨ n j , x ⟩) and log γ E j ( K j − Π j x ) \log\gamma_{E_j}(K_j - \Pi_jx) log γ E j ( K j − Π j x ) . The first term log ϕ ( a j − ⟨ n j , x ⟩ ) \log\phi(a_j - \langle n_j, x\rangle) log ϕ ( a j − ⟨ n j , x ⟩) is clearly concave, with gradient a j n j a_j n_j a j n j and Hessian − n j n j T -n_j n_j^{\mathsf{T}} − n j n j T at x = 0 x = 0 x = 0 . For the second term log γ E j ( K j − Π j x ) \log\gamma_{E_j}(K_j - \Pi_jx) log γ E j ( K j − Π j x ) , recall from Lemma 4.1 that K j K_j K j is closed and convex with q j > 0 q_j > 0 q j > 0 . So Lemma 8.5 (b) on E j E_j E j shows that x ′ ↦ γ E j ( K j − x ′ ) x' \mapsto\gamma_{E_j}(K_j - x') x ′ ↦ γ E j ( K j − x ′ ) is positive, C ∞ C^\infty C ∞ and log-concave on E j E_j E j , and that its logarithm has gradient m j m_j m j and Hessian S j − Π j \mathfrak{S}_j - \Pi_j S j − Π j at x ′ = 0 x' = 0 x ′ = 0 . Composing with the orthogonal projection Π j \Pi_j Π j of R p \mathbb{R}^p R p onto E j E_j E j preserves these properties, with the same gradient and Hessian at x = 0 x = 0 x = 0 . Adding the two contributions gives the claims.
Proof of Lemma 4.4. The formula for ψ j ( y ) \psi_j(y) ψ j ( y ) follows from its definition combined with ν j ( P ) = ϕ ( a j ) q j \nu_j(P) = \phi(a_j)q_j ν j ( P ) = ϕ ( a j ) q j (Lemma 4.1 ). Taking logarithms in this formula gives
log ψ j ( y ) = log q j ( y ) − 1 2 a j ( y ) 2 − 1 2 log ( 2 π ( u + σ 2 ) ) . \log\psi_j(y) = \log q_j(y) - \frac{1}{2}a_j(y)^2 - \frac{1}{2}\log(2\pi(u+\sigma^2)). log ψ j ( y ) = log q j ( y ) − 2 1 a j ( y ) 2 − 2 1 log ( 2 π ( u + σ 2 )) .
and applying ∂ w \partial_w ∂ w gives ((19) ). Next, since we have P ( y + w ) = P ( y ) − Σ u − 1 / 2 w P(y+w) = P(y) - \Sigma_u^{-1/2}w P ( y + w ) = P ( y ) − Σ u − 1/2 w by Lemma 4.3 (a), the positivity, smoothness, and log-concavity of ψ j \psi_j ψ j and q j q_j q j follow from Lemma 8.7 .
Before proceeding to the upper bounds in the statement of Lemma 4.4, we first compute formulas for the derivatives of log ψ j \log\psi_j log ψ j . The chain rule and the last statement of Lemma 8.7 (replacing x x x with Σ u − 1 / 2 w \Sigma_u^{-1/2}w Σ u − 1/2 w ) give
∇ log ψ j ( y ) = Σ u − 1 / 2 ( a j ( y ) n j + m j ( y ) ) and − ∇ 2 log ψ j ( y ) = Σ u − 1 / 2 Λ j ( y ) Σ u − 1 / 2 , \nabla\log\psi_j(y)=\Sigma_u^{-1/2}\left(a_j(y)n_j+m_j(y)\right)
\quad\text{and}\quad
-\nabla^2\log\psi_j(y)=\Sigma_u^{-1/2}\Lambda_j(y)\Sigma_u^{-1/2}, ∇ log ψ j ( y ) = Σ u − 1/2 ( a j ( y ) n j + m j ( y ) ) and − ∇ 2 log ψ j ( y ) = Σ u − 1/2 Λ j ( y ) Σ u − 1/2 ,
where we put Λ j ( y ) = n j n j T + Π j − S j ( y ) ⪰ 0 \Lambda_j(y)=n_jn_j^{\mathsf T}+\Pi_j-\mathcal{S}_j(y)\succeq0 Λ j ( y ) = n j n j T + Π j − S j ( y ) ⪰ 0 . To develop upper bounds, we use Σ ⪯ Σ u \Sigma\preceq\Sigma_u Σ ⪯ Σ u , which is the same as Σ u − 1 / 2 Σ Σ u − 1 / 2 ⪯ u − 1 I p \Sigma_u^{-1/2}\Sigma\Sigma_u^{-1/2}\preceq u^{-1}I_p Σ u − 1/2 Σ Σ u − 1/2 ⪯ u − 1 I p . Applying this matrix inequality to the gradient formula gives
⟨ ∇ log ψ j ( y ) , Σ ∇ log ψ j ( y ) ⟩ ≤ 1 u ∥ a j ( y ) n j + m j ( y ) ∥ 2 2 . \left\langle\nabla\log\psi_j(y),\Sigma\nabla\log\psi_j(y)\right\rangle
\le\frac{1}{u}\left\|a_j(y)n_j+m_j(y)\right\|_2^2. ⟨ ∇ log ψ j ( y ) , Σ∇ log ψ j ( y ) ⟩ ≤ u 1 ∥ a j ( y ) n j + m j ( y ) ∥ 2 2 .
which is (20) because m j ( y ) m_j(y) m j ( y ) and n j n_j n j are orthogonal. Similarly, combining the matrix inequality and the Hessian formula gives
tr ( Σ ( − ∇ 2 log ψ j ( y ) ) ) ≤ 1 u tr Λ j ( y ) , \operatorname{tr}\left(\Sigma(-\nabla^2\log\psi_j(y))\right)
\le\frac{1}{u}\operatorname{tr}\Lambda_j(y), tr ( Σ ( − ∇ 2 log ψ j ( y )) ) ≤ u 1 tr Λ j ( y ) ,
and this reduces to (21) because tr Λ j ( y ) = 1 + tr ( Π j − S j ( y ) ) \operatorname{tr}\Lambda_j(y)=1+\operatorname{tr}(\Pi_j-\mathcal{S}_j(y)) tr Λ j ( y ) = 1 + tr ( Π j − S j ( y )) .
Now we conclude the proof by applying the gradient and Hessian inequalities to prove (22) –(24) . Since tr ( Π j − S j ( y ) ) ≥ 0 \operatorname{tr}(\Pi_j-\mathcal{S}_j(y))\ge0 tr ( Π j − S j ( y )) ≥ 0 (Lemma 8.5(b)), the quantities ν j ( y ) \nu_j(y) ν j ( y ) , ( a j ( y ) 2 + ∥ m j ( y ) ∥ 2 2 ) ν j ( y ) (a_j(y)^2+\|m_j(y)\|_2^2)\nu_j(y) ( a j ( y ) 2 + ∥ m j ( y ) ∥ 2 2 ) ν j ( y ) and tr Λ j ( y ) ν j ( y ) \operatorname{tr}\Lambda_j(y)\nu_j(y) tr Λ j ( y ) ν j ( y ) are all at most the j j j th term of F ( y ) \mathcal{F}(y) F ( y ) in Lemma 4.2. Finally, due to the basic inequality ψ j ( y ) ≤ ν j ( y ) / u \psi_j(y)\le\nu_j(y)/\sqrt{u} ψ j ( y ) ≤ ν j ( y ) / u , the bounds (22) –(24) follow by multiplying (20) and (21) by ψ j ( y ) \psi_j(y) ψ j ( y ) and summing over j j j .
Proof of Lemma 3.6. Fix y y y , w w w and put x ′ = Σ u − 1 / 2 w x'=\Sigma_u^{-1/2}w x ′ = Σ u − 1/2 w . By Lemma 4.3, P ( y + τ w ) − x = P ( y ) − ( x + τ x ′ ) P(y+\tau w)-x=P(y)-(x+\tau x') P ( y + τ w ) − x = P ( y ) − ( x + τ x ′ ) and hence Δ Σ g u ( y + τ w ) = 1 u tr ( Γ ∇ x 2 γ p ( P ( y ) − x ) ) ∣ x = τ x ′ \Delta_{\Sigma}g_u(y+\tau w)=\frac{1}{u}\operatorname{tr}\left(\Gamma\nabla_x^2\gamma_p(P(y)-x)\right)|_{x=\tau x'} Δ Σ g u ( y + τ w ) = u 1 tr ( Γ ∇ x 2 γ p ( P ( y ) − x ) ) ∣ x = τ x ′ . Differentiating at τ = 0 \tau=0 τ = 0 with Lemma 8.6(b), applied to P ( y ) P(y) P ( y ) at x = 0 x=0 x = 0 ,
∂ w Δ Σ g u ( y ) = − 1 u ∑ j = 1 p ⟨ n j , x ′ ⟩ tr ( Γ M j ( P ( y ) ) ) . \partial_w\Delta_{\Sigma}g_u(y)
=-\frac{1}{u}\sum_{j=1}^{p}\langle n_j,x'\rangle\operatorname{tr}\left(\Gamma M_j(P(y))\right). ∂ w Δ Σ g u ( y ) = − u 1 j = 1 ∑ p ⟨ n j , x ′ ⟩ tr ( Γ M j ( P ( y )) ) .
where M j ( P ( y ) ) M_j(P(y)) M j ( P ( y )) is as defined in that lemma. Here ⟨ n j , x ′ ⟩ = w j / u + σ 2 \langle n_j,x'\rangle=w_j/\sqrt{u+\sigma^2} ⟨ n j , x ′ ⟩ = w j / u + σ 2 with u + σ 2 ≥ u u+\sigma^2\ge u u + σ 2 ≥ u , and Lemma 8.6(c) gives ∣ tr ( Γ M j ( P ( y ) ) ) ∣ ≤ ( 1 + a j ( y ) 2 + ∥ m j ( y ) ∥ 2 2 + tr ( Π j − S j ( y ) ) ) ν j ( y ) \left|\operatorname{tr}(\Gamma M_j(P(y)))\right|\le(1+a_j(y)^2+\|m_j(y)\|_2^2+\operatorname{tr}(\Pi_j-\mathcal{S}_j(y)))\nu_j(y) ∣ tr ( Γ M j ( P ( y ))) ∣ ≤ ( 1 + a j ( y ) 2 + ∥ m j ( y ) ∥ 2 2 + tr ( Π j − S j ( y ))) ν j ( y ) . Summing over j j j and using Lemma 4.2 gives the claim.
Proof of Theorem 2.2
It is sufficient to prove the result when the class of rectangles is replaced with the class of orthants, using essentially the same reasoning as in the proof of Lemma 8.1. Also, it can be checked that the discrepancy measure d ( Σ ~ , Σ ) d(\widetilde{\Sigma},\Sigma) d ( Σ , Σ ) between the covariance matrices of G ~ ∼ N ( 0 , Σ ~ ) \widetilde{G}\sim N(0,\widetilde{\Sigma}) G ∼ N ( 0 , Σ ) and G ∼ N ( 0 , Σ ) G\sim N(0,\Sigma) G ∼ N ( 0 , Σ ) does not change when Σ ~ \widetilde{\Sigma} Σ and Σ \Sigma Σ are replaced by the 2 p × 2 p 2p\times2p 2 p × 2 p covariance matrices of ( G , G ~ − G ~ ) (G,\widetilde{G}-\widetilde{G}) ( G , G − G ) and ( G , − G ) (G,-G) ( G , − G ) .
Thus, we will show that if d ( Σ ~ , Σ ) ≤ δ d(\widetilde{\Sigma},\Sigma)\le\delta d ( Σ , Σ ) ≤ δ for some δ > 0 \delta>0 δ > 0 , and if Σ \Sigma Σ has ones along the diagonal, then (with p p p in place of 2 p 2p 2 p )
sup β ∈ R p ∣ P ( G ~ ≤ β ) − P ( G ≤ β ) ∣ ≲ δ log ( p ) log ( 1 + 1 / δ ) . (70) \sup_{\beta\in\mathbb{R}^p}\left|\mathbb{P}(\widetilde{G}\le\beta)-\mathbb{P}(G\le\beta)\right|
\lesssim\delta\log(p)\log(1+1/\delta). \tag*{(70)} β ∈ R p sup P ( G ≤ β ) − P ( G ≤ β ) ≲ δ log ( p ) log ( 1 + 1/ δ ) . (70)
Also, we may assume going forward that δ ≤ 1 / 2 \delta\le1/2 δ ≤ 1/2 holds without loss of generality, for otherwise the statement of Theorem 2.2 is trivially true.
To proceed, we will re-use an initial portion of the proof of Theorem 3.1 by taking X 1 , … , X n X_1,\ldots,X_n X 1 , … , X n to be i.i.d. copies of G ~ \widetilde{G} G with n = ⌈ 1 / δ 2 ⌉ n=\lceil1/\delta^2\rceil n = ⌈ 1/ δ 2 ⌉ , so that the Gaussianity of G ~ \widetilde{G} G implies S = n − 1 / 2 ∑ i = 1 n X i → d G ~ S=n^{-1/2}\sum_{i=1}^{n}X_i \xrightarrow{d} \widetilde{G} S = n − 1/2 ∑ i = 1 n X i d G .1 Furthermore, if we let σ = δ \sigma=\delta σ = δ , then the arguments at the beginning of § 3 show that the supremum on the left side of (70) can be bounded as
sup β ∈ R p ∣ P ( G ~ ≤ β ) − P ( G ≤ β ) ∣ ≲ ∫ 0 1 ∣ Ψ ′ ( t ) ∣ d t + δ log ( p / δ ) , (71) \sup_{\beta\in\mathbb{R}^{p}}\left|\mathbb{P}(\widetilde{G}\le\beta)-\mathbb{P}(G\le\beta)\right|\lesssim\int_{0}^{1}\left|\Psi^{\prime}(t)\right|dt+\delta\log(p/\delta), \tag*{(71)} β ∈ R p sup P ( G ≤ β ) − P ( G ≤ β ) ≲ ∫ 0 1 ∣ Ψ ′ ( t ) ∣ d t + δ log ( p / δ ) , (71)
where Ψ ( t ) = E g u ( t ) ( t G ~ ) \Psi(t)=\mathbb{E}g_{u(t)}(\sqrt{t}\widetilde{G}) Ψ ( t ) = E g u ( t ) ( t G ) and u ( t ) = 1 − t + δ 2 u(t)=1-t+\delta^{2} u ( t ) = 1 − t + δ 2 . As in the proof of Lemma 3.2 , the derivative of Ψ \Psi Ψ can be calculated as Ψ ′ ( t ) = 1 2 t E ⟨ ∇ g u ( t G ~ ) , G ~ ⟩ − 1 2 E Δ Σ g u ( t ) ( t G ~ ) \Psi^{\prime}(t)=\frac{1}{2\sqrt{t}}\mathbb{E}\langle\nabla g_{u}(\sqrt{t}\widetilde{G}),\widetilde{G}\rangle-\frac{1}{2}\mathbb{E}\Delta_{\Sigma}g_{u(t)}(\sqrt{t}\widetilde{G}) Ψ ′ ( t ) = 2 t 1 E ⟨ ∇ g u ( t G ) , G ⟩ − 2 1 E Δ Σ g u ( t ) ( t G ) for t ∈ ( 0 , 1 ] t\in(0,1] t ∈ ( 0 , 1 ] , and applying Stein’s identity (62) to the first term gives
Ψ ′ ( t ) = 1 2 E tr ( ( Σ ~ − Σ ) ∇ 2 g u ( t ) ( t G ~ ) ) . \Psi^{\prime}(t)=\frac{1}{2}\mathbb{E}\operatorname{tr}\left((\widetilde{\Sigma}-\Sigma)\nabla^{2}g_{u(t)}(\sqrt{t}\widetilde{G})\right). Ψ ′ ( t ) = 2 1 E tr ( ( Σ − Σ ) ∇ 2 g u ( t ) ( t G ) ) .
In the later part of this section, Lemmas 9.2 and 9.3 will collectively show that the trace in this expression can be bounded as
∣ tr ( ( Σ ~ − Σ ) ∇ 2 g u ( y ) ) ∣ ≲ δ log ( 2 p ) u + δ 2 , (72) \left|\operatorname{tr}\left((\widetilde{\Sigma}-\Sigma)\nabla^{2}g_{u}(y)\right)\right|\lesssim\frac{\delta\log(2p)}{u+\delta^{2}}, \tag*{(72)} tr ( ( Σ − Σ ) ∇ 2 g u ( y ) ) ≲ u + δ 2 δ log ( 2 p ) , (72)
for all y ∈ R p y\in\mathbb{R}^{p} y ∈ R p and u > 0 u>0 u > 0 . Combining this with ∫ 0 1 ( u ( t ) + δ 2 ) − 1 d t ≤ 2 log ( 1 / δ ) \int_{0}^{1}(u(t)+\delta^{2})^{-1}dt\le2\log(1/\delta) ∫ 0 1 ( u ( t ) + δ 2 ) − 1 d t ≤ 2 log ( 1/ δ ) , we obtain
sup β ∈ R p ∣ P ( G ~ ≤ β ) − P ( G ≤ β ) ∣ ≲ δ log ( 2 p ) log ( 1 / δ ) + δ log ( p / δ ) , \sup_{\beta\in\mathbb{R}^{p}}\left|\mathbb{P}(\widetilde{G}\le\beta)-\mathbb{P}(G\le\beta)\right|\lesssim\delta\log(2p)\log(1/\delta)+\delta\log(p/\delta), β ∈ R p sup P ( G ≤ β ) − P ( G ≤ β ) ≲ δ log ( 2 p ) log ( 1/ δ ) + δ log ( p / δ ) ,
which implies the stated result.
Supporting results for the bound (9.3)
This subsection collects three lemmas that establish the bound (72) . Throughout the subsection, u > 0 u>0 u > 0 is fixed, and we write a ( y ) = ( a 1 ( y ) , … , a p ( y ) ) = ( β − y ) / u + σ 2 a(y)=(a_{1}(y),\ldots,a_{p}(y))=(\beta-y)/\sqrt{u+\sigma^{2}} a ( y ) = ( a 1 ( y ) , … , a p ( y )) = ( β − y ) / u + σ 2 for the vector of offsets of P ( y ) P(y) P ( y ) . Also, let R R R denote the correlation matrix R = 1 u + σ 2 Σ u R=\frac{1}{u+\sigma^{2}}\Sigma_{u} R = u + σ 2 1 Σ u , and let Y ∼ N ( 0 , R ) Y\sim\mathcal{N}(0,R) Y ∼ N ( 0 , R ) . Likewise, let ϕ R \phi_{R} ϕ R and Φ R \Phi_{R} Φ R denote the density and distribution function of Y Y Y so that Φ R ( a ) = P ( Y ≤ a ) \Phi_{R}(a)=\mathbb{P}(Y\le a) Φ R ( a ) = P ( Y ≤ a ) for all a ∈ R p a\in\mathbb{R}^{p} a ∈ R p .
The following lemma gives integral formulas for the first and second partial derivatives of Φ R \Phi_{R} Φ R , together with an identity for the diagonal entries of R ∇ 2 Φ R R\nabla^{2}\Phi_{R} R ∇ 2 Φ R .
Lemma 9.1. For all a ∈ R p a\in\mathbb{R}^{p} a ∈ R p and all 1 ≤ j ≠ l ≤ p 1\le j\ne l\le p 1 ≤ j = l ≤ p , we have
∂ j Φ R ( a ) = ∫ { z ≤ a , z j = a j } ϕ R ( z ) d H p − 1 ( z ) and ∂ j ∂ l Φ R ( a ) = ∫ { z ≤ a , z j = a j , z l = a l } ϕ R ( z ) d H p − 2 ( z ) . \partial_{j}\Phi_{R}(a)=\int_{\{z\le a,\ z_{j}=a_{j}\}}\phi_{R}(z)\,d\mathcal{H}^{p-1}(z)
\quad\text{and}\quad
\partial_{j}\partial_{l}\Phi_{R}(a)=\int_{\{z\le a,\ z_{j}=a_{j},\ z_{l}=a_{l}\}}\phi_{R}(z)\,d\mathcal{H}^{p-2}(z). ∂ j Φ R ( a ) = ∫ { z ≤ a , z j = a j } ϕ R ( z ) d H p − 1 ( z ) and ∂ j ∂ l Φ R ( a ) = ∫ { z ≤ a , z j = a j , z l = a l } ϕ R ( z ) d H p − 2 ( z ) . Moreover, for all a ∈ R p a\in\mathbb{R}^{p} a ∈ R p and j ≤ p j\le p j ≤ p ,
∑ l = 1 p R j l ∂ j ∂ l Φ R ( a ) = − a j ∂ j Φ R ( a ) . (73) \sum_{l=1}^{p}R_{jl}\partial_{j}\partial_{l}\Phi_{R}(a)=-a_{j}\partial_{j}\Phi_{R}(a). \tag*{(73)} l = 1 ∑ p R j l ∂ j ∂ l Φ R ( a ) = − a j ∂ j Φ R ( a ) . (73)
Proof. The first two formulas are straightforward. To prove the third formula (73) , we write Φ R ( a ) = ∫ { z ≤ 0 } ϕ R ( z + a ) d z \Phi_{R}(a)=\int_{\{z\le0\}}\phi_{R}(z+a)\,dz Φ R ( a ) = ∫ { z ≤ 0 } ϕ R ( z + a ) d z and differentiate under the integral sign, which gives ∇ Φ R ( a ) = ∫ { z ≤ a } ∇ ϕ R ( z ) d z \nabla\Phi_{R}(a)=\int_{\{z\le a\}}\nabla\phi_{R}(z)\,dz ∇ Φ R ( a ) = ∫ { z ≤ a } ∇ ϕ R ( z ) d z . The left side of (73) can be rewritten as
∑ l = 1 p R j l ∂ j ∂ l Φ R ( a ) = ∂ j ⟨ e j , R ∇ Φ R ( a ) ⟩ = ∫ { z ≤ a } ∂ j ( R ∇ ϕ R ) j ( z ) d z . \sum_{l=1}^{p}R_{jl}\partial_{j}\partial_{l}\Phi_{R}(a)=\partial_{j}\langle e_{j},R\nabla\Phi_{R}(a)\rangle=\int_{\{z\le a\}}\partial_{j}(R\nabla\phi_{R})_{j}(z)\,dz. l = 1 ∑ p R j l ∂ j ∂ l Φ R ( a ) = ∂ j ⟨ e j , R ∇ Φ R ( a )⟩ = ∫ { z ≤ a } ∂ j ( R ∇ ϕ R ) j ( z ) d z .
Furthermore, the basic formula R ∇ ϕ R ( z ) = − ϕ R ( z ) z R\nabla\phi_R(z)=-\phi_R(z)z R ∇ ϕ R ( z ) = − ϕ R ( z ) z allows the last integral to be calculated as
− ∫ { z ≤ a } ∂ j ( z j ϕ R ( z ) ) d z = − a j ∫ { z ≤ a , z j = a j } ϕ R ( z ) d H p − 1 ( z ) . -\int_{\{z\le a\}} \partial_j(z_j\phi_R(z))\,\mathrm{d}z=-a_j\int_{\{z\le a,\ z_j=a_j\}}\phi_R(z)\,\mathrm{d}\mathcal{H}^{p-1}(z). − ∫ { z ≤ a } ∂ j ( z j ϕ R ( z )) d z = − a j ∫ { z ≤ a , z j = a j } ϕ R ( z ) d H p − 1 ( z ) .
By the earlier formula for ∂ j Φ R ( a ) \partial_j\Phi_R(a) ∂ j Φ R ( a ) , the right side is − a j ∂ j Φ R ( a ) -a_j\partial_j\Phi_R(a) − a j ∂ j Φ R ( a ) .
As notation for the next lemma, we define for each 1 ≤ j ≠ l ≤ p 1\le j\ne l\le p 1 ≤ j = l ≤ p and y ∈ R p y\in\mathbb{R}^p y ∈ R p the function
ν j l ( y ) = ∂ j ∂ l Φ R ( a ( y ) ) . \nu_{jl}(y)=\partial_j\partial_l\Phi_R(a(y)). ν j l ( y ) = ∂ j ∂ l Φ R ( a ( y )) .
This notation is motivated by the formula ν j ( y ) = ∂ j Φ R ( a ( y ) ) \nu_j(y)=\partial_j\Phi_R(a(y)) ν j ( y ) = ∂ j Φ R ( a ( y )) , which will be shown below in (75) .
Lemma 9.2. Let Σ \Sigma Σ and Σ ~ \widetilde{\Sigma} Σ be positive semidefinite p × p p\times p p × p matrices such that Σ j j = 1 \Sigma_{jj}=1 Σ j j = 1 for all j ≤ p j\le p j ≤ p and d ( Σ ~ , Σ ) ≤ δ d(\widetilde{\Sigma},\Sigma)\le\delta d ( Σ , Σ ) ≤ δ for some δ > 0 \delta>0 δ > 0 . Then, for all y ∈ R p y\in\mathbb{R}^p y ∈ R p ,
∣ tr ( ( Σ ~ − Σ ) ∇ 2 g u ( y ) ) ∣ ≤ δ u + σ 2 ( ∑ j = 1 p ∣ a j ( y ) ∣ ∣ ν j ( y ) ∣ + 2 ∑ j < l ( 1 − R j l 2 ) ∣ ν j l ( y ) ∣ ) . \left|\operatorname{tr}\left((\widetilde{\Sigma}-\Sigma)\nabla^2g_u(y)\right)\right|\le\frac{\delta}{u+\sigma^2}\left(\sum_{j=1}^{p}|a_j(y)||\nu_j(y)|+2\sum_{j<l}(1-R_{jl}^2)|\nu_{jl}(y)|\right). tr ( ( Σ − Σ ) ∇ 2 g u ( y ) ) ≤ u + σ 2 δ j = 1 ∑ p ∣ a j ( y ) ∣∣ ν j ( y ) ∣ + 2 j < l ∑ ( 1 − R j l 2 ) ∣ ν j l ( y ) ∣ . Proof. To ease notation, we fix y y y and write a = a ( y ) a=a(y) a = a ( y ) , ν j = ν j ( y ) \nu_j=\nu_j(y) ν j = ν j ( y ) and ν j l = ν j l ( y ) \nu_{jl}=\nu_{jl}(y) ν j l = ν j l ( y ) . Noting that the covariance matrix of ( u G + σ N ) / u + σ 2 (\sqrt{u}G+\sigma N)/\sqrt{u+\sigma^2} ( u G + σ N ) / u + σ 2 is equal to R R R , it follows from the definition of g u g_u g u that g u ( y ) = Φ R ( a ( y ) ) g_u(y)=\Phi_R(a(y)) g u ( y ) = Φ R ( a ( y )) , and so the chain rule leads to
∇ 2 g u ( y ) = 1 u + σ 2 ∇ 2 Φ R ( a ) . \nabla^2g_u(y)=\frac{1}{u+\sigma^2}\nabla^2\Phi_R(a). ∇ 2 g u ( y ) = u + σ 2 1 ∇ 2 Φ R ( a ) . Furthermore, since the off-diagonal entries of ∇ 2 Φ R ( a ) \nabla^2\Phi_R(a) ∇ 2 Φ R ( a ) are the numbers ν j l \nu_{jl} ν j l , and since Σ j j = 1 \Sigma_{jj}=1 Σ j j = 1 , we have
tr ( ( Σ ~ − Σ ) ∇ 2 Φ R ( a ) ) = ∑ j = 1 p ( Σ ~ j j − 1 ) ∂ j 2 Φ R ( a ) + 2 ∑ j < l ( Σ ~ j l − Σ j l ) ν j l . (74) \operatorname{tr}\left((\widetilde{\Sigma}-\Sigma)\nabla^2\Phi_R(a)\right)=\sum_{j=1}^{p}(\widetilde{\Sigma}_{jj}-1)\partial_j^2\Phi_R(a)+2\sum_{j<l}(\widetilde{\Sigma}_{jl}-\Sigma_{jl})\nu_{jl}. \tag*{(74)} tr ( ( Σ − Σ ) ∇ 2 Φ R ( a ) ) = j = 1 ∑ p ( Σ j j − 1 ) ∂ j 2 Φ R ( a ) + 2 j < l ∑ ( Σ j l − Σ j l ) ν j l . (74) Towards computing a formula for ∂ j 2 Φ R ( a ) \partial_j^2\Phi_R(a) ∂ j 2 Φ R ( a ) , we show that the first derivative ∂ j Φ R ( a ) \partial_j\Phi_R(a) ∂ j Φ R ( a ) is related to ν j ( y ) \nu_j(y) ν j ( y ) . Indeed, the chain rule gives ∂ j g u ( y ) = − ∂ j Φ R ( a ) / u + σ 2 \partial_jg_u(y)=-\partial_j\Phi_R(a)/\sqrt{u+\sigma^2} ∂ j g u ( y ) = − ∂ j Φ R ( a ) / u + σ 2 , and comparing this with (13) gives
ν j ( y ) = ∂ j Φ R ( a ( y ) ) , (75) \nu_j(y)=\partial_j\Phi_R(a(y)), \tag*{(75)} ν j ( y ) = ∂ j Φ R ( a ( y )) , (75) for all j ≤ p j\le p j ≤ p . Now, to get ∂ j 2 Φ R ( a ) \partial_j^2\Phi_R(a) ∂ j 2 Φ R ( a ) , note that this quantity appears in the l = j l=j l = j term of the sum on the left side of (73) , and that R j j = 1 R_{jj}=1 R j j = 1 . Based on this observation, rearranging (73) and recalling the definition of ν j l \nu_{jl} ν j l leads to
∂ j 2 Φ R ( a ) = − a j ν j − ∑ l : l ≠ j R j l ν j l . \partial_j^2\Phi_R(a)=-a_j\nu_j-\sum_{l:l\ne j}R_{jl}\nu_{jl}. ∂ j 2 Φ R ( a ) = − a j ν j − l : l = j ∑ R j l ν j l . Substituting this into (74) gives
tr ( ( Σ ~ − Σ ) ∇ 2 Φ R ( a ) ) = − ∑ j = 1 p ( Σ ~ j j − 1 ) a j ν j − ∑ j ≠ l ( Σ ~ j j − 1 ) R j l ν j l + 2 ∑ j < l ( Σ ~ j l − Σ j l ) ν j l . \operatorname{tr}\left((\widetilde{\Sigma}-\Sigma)\nabla^2\Phi_R(a)\right)=-\sum_{j=1}^{p}(\widetilde{\Sigma}_{jj}-1)a_j\nu_j-\sum_{j\ne l}(\widetilde{\Sigma}_{jj}-1)R_{jl}\nu_{jl}+2\sum_{j<l}(\widetilde{\Sigma}_{jl}-\Sigma_{jl})\nu_{jl}. tr ( ( Σ − Σ ) ∇ 2 Φ R ( a ) ) = − j = 1 ∑ p ( Σ j j − 1 ) a j ν j − j = l ∑ ( Σ j j − 1 ) R j l ν j l + 2 j < l ∑ ( Σ j l − Σ j l ) ν j l . To convert the sum over j ≠ l j\ne l j = l into a sum over j < l j<l j < l , each term ( Σ ~ j j − 1 ) R j l ν j l (\widetilde{\Sigma}_{jj}-1)R_{jl}\nu_{jl} ( Σ j j − 1 ) R j l ν j l will become R j l ( Σ ~ j j + Σ ~ l l − 2 ) ν j l R_{jl}(\widetilde{\Sigma}_{jj}+\widetilde{\Sigma}_{ll}-2)\nu_{jl} R j l ( Σ j j + Σ l l − 2 ) ν j l . Thus, we obtain
tr ( ( Σ ~ − Σ ) ∇ 2 Φ R ( a ) ) = − ∑ j = 1 p ( Σ ~ j j − 1 ) a j ν j + ∑ j < l c j l ν j l . (76) \operatorname{tr}\left((\widetilde{\Sigma}-\Sigma)\nabla^2\Phi_R(a)\right)=-\sum_{j=1}^{p}(\widetilde{\Sigma}_{jj}-1)a_j\nu_j+\sum_{j<l}c_{jl}\nu_{jl}. \tag*{(76)} tr ( ( Σ − Σ ) ∇ 2 Φ R ( a ) ) = − j = 1 ∑ p ( Σ j j − 1 ) a j ν j + j < l ∑ c j l ν j l . (76)
where we define the coefficients c j l = 2 ( Σ ~ j l − Σ j l ) − R j l ( Σ ~ j j + Σ ~ l l − 2 ) c_{jl}=2(\widetilde{\Sigma}_{jl}-\Sigma_{jl})-R_{jl}(\widetilde{\Sigma}_{jj}+\widetilde{\Sigma}_{ll}-2) c j l = 2 ( Σ j l − Σ j l ) − R j l ( Σ j j + Σ l l − 2 ) . Because the quantities ν j \nu_j ν j and ν j l \nu_{jl} ν j l are all non-negative by Lemma 9.1, it remains to show that ∣ Σ ~ j j − 1 ∣ ≤ δ |\widetilde{\Sigma}_{jj}-1|\le\delta ∣ Σ j j − 1∣ ≤ δ and ∣ c j l ∣ ≤ 2 δ ( 1 − R j l 2 ) |c_{jl}|\le2\delta(1-R_{jl}^{2}) ∣ c j l ∣ ≤ 2 δ ( 1 − R j l 2 ) . By the assumption d ( Σ ~ , Σ ) ≤ δ d(\widetilde{\Sigma},\Sigma)\le\delta d ( Σ , Σ ) ≤ δ , we have
∣ ⟨ v , ( Σ ~ − Σ ) v ⟩ ∣ ≤ δ ⟨ v , Σ v ⟩ (77) |\langle v,(\widetilde{\Sigma}-\Sigma)v\rangle|\le\delta\langle v,\Sigma v\rangle \tag*{(77)} ∣ ⟨ v , ( Σ − Σ ) v ⟩ ∣ ≤ δ ⟨ v , Σ v ⟩ (77)
for every v ∈ V v\in\mathcal{V} v ∈ V . Taking v = e j v=e_j v = e j gives ∣ Σ ~ j j − 1 ∣ ≤ δ |\widetilde{\Sigma}_{jj}-1|\le\delta ∣ Σ j j − 1∣ ≤ δ , which handles the first sum in (76) . For the second sum, it will be convenient to introduce the following notation with the indices j < l j<l j < l held fixed,
ω ± = ⟨ e j ± e l , ( Σ ~ − Σ ) ( e j ± e l ) ⟩ . \omega_{\pm}=\langle e_j\pm e_l,(\widetilde{\Sigma}-\Sigma)(e_j\pm e_l)\rangle. ω ± = ⟨ e j ± e l , ( Σ − Σ ) ( e j ± e l )⟩ .
Taking v = e j ± e l v=e_j\pm e_l v = e j ± e l in the bound (77) gives ∣ ω ± ∣ ≤ 2 δ ( 1 ± Σ j l ) |\omega_{\pm}|\le2\delta(1\pm\Sigma_{jl}) ∣ ω ± ∣ ≤ 2 δ ( 1 ± Σ j l ) . To relate ω ± \omega_{\pm} ω ± to c j l c_{jl} c j l , we expand
ω ± = ( Σ ~ j j − 1 ) + ( Σ ~ l l − 1 ) ± 2 ( Σ ~ j l − Σ j l ) . \omega_{\pm}=(\widetilde{\Sigma}_{jj}-1)+(\widetilde{\Sigma}_{ll}-1)\pm2(\widetilde{\Sigma}_{jl}-\Sigma_{jl}). ω ± = ( Σ j j − 1 ) + ( Σ l l − 1 ) ± 2 ( Σ j l − Σ j l ) .
Comparing this with the definition of c j l c_{jl} c j l , it can be checked that
c j l = 1 2 ( 1 − R j l ) ω + − 1 2 ( 1 + R j l ) ω − . c_{jl}=\frac{1}{2}(1-R_{jl})\omega_{+}-\frac{1}{2}(1+R_{jl})\omega_{-}. c j l = 2 1 ( 1 − R j l ) ω + − 2 1 ( 1 + R j l ) ω − .
Since ∣ R j l ∣ ≤ 1 |R_{jl}|\le1 ∣ R j l ∣ ≤ 1 , the coefficients 1 2 ( 1 ∓ R j l ) \frac{1}{2}(1\mp R_{jl}) 2 1 ( 1 ∓ R j l ) are nonnegative, and so the bounds on ∣ ω ± ∣ |\omega_{\pm}| ∣ ω ± ∣ give
∣ c j l ∣ ≤ 1 2 ( 1 − R j l ) ∣ ω + ∣ + 1 2 ( 1 + R j l ) ∣ ω − ∣ ≤ δ ( 1 − R j l ) ( 1 + Σ j l ) + δ ( 1 + R j l ) ( 1 − Σ j l ) = 2 δ ( 1 − R j l Σ j l ) ≤ 2 δ ( 1 − R j l 2 ) , \begin{aligned}
|c_{jl}|\le\frac{1}{2}(1-R_{jl})|\omega_{+}|+\frac{1}{2}(1+R_{jl})|\omega_{-}| \\
&\le\delta(1-R_{jl})(1+\Sigma_{jl})+\delta(1+R_{jl})(1-\Sigma_{jl}) \\
&=2\delta(1-R_{jl}\Sigma_{jl})\le2\delta(1-R_{jl}^{2}),
\end{aligned} ∣ c j l ∣ ≤ 2 1 ( 1 − R j l ) ∣ ω + ∣ + 2 1 ( 1 + R j l ) ∣ ω − ∣ ≤ δ ( 1 − R j l ) ( 1 + Σ j l ) + δ ( 1 + R j l ) ( 1 − Σ j l ) = 2 δ ( 1 − R j l Σ j l ) ≤ 2 δ ( 1 − R j l 2 ) ,
where the last step holds because R j l Σ j l = u Σ j l 2 / ( u + σ 2 ) ≥ R j l 2 R_{jl}\Sigma_{jl}=u\Sigma_{jl}^{2}/(u+\sigma^{2})\ge R_{jl}^{2} R j l Σ j l = u Σ j l 2 / ( u + σ 2 ) ≥ R j l 2 .
Lemma 9.3. For all y ∈ R p y\in\mathbb{R}^{p} y ∈ R p ,
∑ j = 1 p ∣ a j ( y ) ∣ ν j ( y ) ≲ log ( 2 p ) , (78) \sum_{j=1}^{p}|a_j(y)|\nu_j(y)\lesssim\log(2p), \tag*{(78)} j = 1 ∑ p ∣ a j ( y ) ∣ ν j ( y ) ≲ log ( 2 p ) , (78) ∑ j < l ( 1 − R j l 2 ) ν j l ( y ) ≲ log ( 2 p ) . (79) \sum_{j<l}(1-R_{jl}^{2})\nu_{jl}(y)\lesssim\log(2p). \tag*{(79)} j < l ∑ ( 1 − R j l 2 ) ν j l ( y ) ≲ log ( 2 p ) . (79)
Proof. To ease notation as in the previous proof, we fix y y y , and write a = a ( y ) a=a(y) a = a ( y ) , ν j = ν j ( y ) \nu_j=\nu_j(y) ν j = ν j ( y ) and ν j l = ν j l ( y ) \nu_{jl}=\nu_{jl}(y) ν j l = ν j l ( y ) . In both (78) and (79) , the indices j j j corresponding to large values of ∣ a j ∣ |a_j| ∣ a j ∣ will be handled in part by the inequalities
ν j ≤ ϕ ( a j ) and ν j l ≤ ( 2 π ( 1 − R j l 2 ) ) − 1 / 2 ϕ ( a j ) for j ≠ l . (80) \nu_j\le\phi(a_j)\qquad\text{and}\qquad\nu_{jl}\le(2\pi(1-R_{jl}^{2}))^{-1/2}\phi(a_j)\qquad\text{for }j\ne l. \tag*{(80)} ν j ≤ ϕ ( a j ) and ν j l ≤ ( 2 π ( 1 − R j l 2 ) ) − 1/2 ϕ ( a j ) for j = l . (80)
The first of these follows from the formula ν j = ϕ ( a j ) q j \nu_j=\phi(a_j)q_j ν j = ϕ ( a j ) q j in Lemma 4.1 , and the second follows from the formula for ∂ j ∂ l Φ R ( a ) \partial_j\partial_l\Phi_R(a) ∂ j ∂ l Φ R ( a ) in Lemma 9.1 by expanding the domain of integration from { z ≤ a , z j = a j , z l = a l } \{z\le a,z_j=a_j,z_l=a_l\} { z ≤ a , z j = a j , z l = a l } to { z j = a j , z l = a l } \{z_j=a_j,z_l=a_l\} { z j = a j , z l = a l } . To see this, note that the latter domain corresponds to marginalizing out all the coordinates except for j j j and l l l , and then the integral is simply the bivariate normal density of ( Y j , Y l ) (Y_j,Y_l) ( Y j , Y l ) at ( a j , a l ) (a_j,a_l) ( a j , a l ) . In turn, that density can be upper bounded by factoring it as ϕ ( a j ) \phi(a_j) ϕ ( a j ) times the conditional density of Y l ∣ Y j Y_l\mid Y_j Y l ∣ Y j at a l a_l a l , which is at most ( 2 π ( 1 − R j l 2 ) ) − 1 / 2 (2\pi(1-R_{jl}^{2}))^{-1/2} ( 2 π ( 1 − R j l 2 ) ) − 1/2 .
Now we turn to the proof of (78) . Since Y Y Y has unit variances, Nazarov’s inequality (Lemma 8.2 ) gives Φ R ( a + ε 1 ) − Φ R ( a ) ≤ ε ( 2 log p + 2 ) \Phi_R(a+\varepsilon\mathbf{1})-\Phi_R(a)\le\varepsilon(\sqrt{2\log p}+2) Φ R ( a + ε 1 ) − Φ R ( a ) ≤ ε ( 2 log p + 2 ) for all ε > 0 \varepsilon>0 ε > 0 . Dividing by ε \varepsilon ε and letting ε ↓ 0 \varepsilon\downarrow0 ε ↓ 0 , we get ∑ j = 1 p ν j ≤ 2 log p + 2 \sum_{j=1}^{p}\nu_j\le\sqrt{2\log p}+2 ∑ j = 1 p ν j ≤ 2 log p + 2 . Next, we let r = 2 log ( 2 p ) r=2\sqrt{\log(2p)} r = 2 log ( 2 p ) and we split the indices j j j according to whether or not ∣ a j ∣ ≤ r |a_j| \le r ∣ a j ∣ ≤ r holds. Using (80) and the fact that t ϕ ( t ) t\phi(t) tϕ ( t ) is decreasing on [ 1 , ∞ ) [1,\infty) [ 1 , ∞ ) , we obtain
∑ j = 1 p ∣ a j ∣ ν j ≤ ∑ j : ∣ a j ∣ ≤ r r ν j + ∑ j : ∣ a j ∣ > r ∣ a j ∣ ϕ ( a j ) ≤ r ( 2 log p + 2 ) + p r ϕ ( r ) ≲ log ( 2 p ) , \sum_{j=1}^{p}|a_j|\nu_j \le\sum_{j:|a_j|\le r} r\nu_j+\sum_{j:|a_j|>r}|a_j|\phi(a_j)\le r(\sqrt{2\log p}+2)+pr\phi(r)\lesssim\log(2p), j = 1 ∑ p ∣ a j ∣ ν j ≤ j : ∣ a j ∣ ≤ r ∑ r ν j + j : ∣ a j ∣ > r ∑ ∣ a j ∣ ϕ ( a j ) ≤ r ( 2 log p + 2 ) + p r ϕ ( r ) ≲ log ( 2 p ) ,
which proves (78) .
The argument for (79) is more involved. We split the pairs j < l j<l j < l according to whether max ( a j , a l ) ≤ r \max(a_j,a_l)\le r max ( a j , a l ) ≤ r . If max ( a j , a l ) > r \max(a_j,a_l)>r max ( a j , a l ) > r , say a j > r a_j>r a j > r , then (80) gives ( 1 − R j l 2 ) ν j l ≤ ϕ ( a j ) ≤ ϕ ( r ) ≤ ( 2 p ) − 2 (1-R_{jl}^{2})\nu_{jl}\le\phi(a_j)\le\phi(r)\le(2p)^{-2} ( 1 − R j l 2 ) ν j l ≤ ϕ ( a j ) ≤ ϕ ( r ) ≤ ( 2 p ) − 2 , and since there are fewer than p 2 p^2 p 2 pairs, these pairs contribute at most O ( 1 ) O(1) O ( 1 ) to the sum in (79) . Hence it remains to show that
∑ j < l 1 { a j ≤ r , a l ≤ r } ( 1 − R j l 2 ) ν j l ≲ log ( 2 p ) . (81) \sum_{j<l}1\{a_j\le r,\ a_l\le r\}(1-R_{jl}^{2})\nu_{jl}\lesssim\log(2p). \tag*{(81)} j < l ∑ 1 { a j ≤ r , a l ≤ r } ( 1 − R j l 2 ) ν j l ≲ log ( 2 p ) . (81)
The idea is to thicken each edge E j l = { x ≤ a : x j = a j , x l = a l } E_{jl}=\{x\le a:x_j=a_j,\ x_l=a_l\} E j l = { x ≤ a : x j = a j , x l = a l } of the orthant { x ≤ a } \{x\le a\} { x ≤ a } into an “expanded edge” E ~ j l ⊂ R p \widetilde E_{jl}\subset\mathbb{R}^p E j l ⊂ R p , in such a way that the expanded edges are disjoint and satisfy
( 1 − R j l 2 ) ν j l ≲ r 2 P ( Y ∈ E ~ j l ) (82) (1-R_{jl}^{2})\nu_{jl}\lesssim r^2\mathbb{P}(Y\in\widetilde E_{jl}) \tag*{(82)} ( 1 − R j l 2 ) ν j l ≲ r 2 P ( Y ∈ E j l ) (82)
where a j , a l ≤ r a_j,a_l\le r a j , a l ≤ r and Y ∼ N ( 0 , R ) Y\sim N(0,R) Y ∼ N ( 0 , R ) . The disjointness of the expanded edges implies ∑ j < l P ( Y ∈ E ~ j l ) ≤ 1 \sum_{j<l}\mathbb{P}(Y\in\widetilde E_{jl})\le1 ∑ j < l P ( Y ∈ E j l ) ≤ 1 , and hence leads to (81) once (82) is established. To define the expanded edge for a pair of indices ( j , l ) (j,l) ( j , l ) with j < l j<l j < l , we let
E ~ j l = E j l + { s R e j + t R e l : s , t ∈ ( 0 , 1 / ( 2 r ) ] } , \widetilde E_{jl}=E_{jl}+\{sRe_j+tRe_l:s,t\in(0,1/(2r)]\}, E j l = E j l + { s R e j + tR e l : s , t ∈ ( 0 , 1/ ( 2 r )]} ,
which is understood in the sense of a Minkowski sum. (An illustration of these expanded edges is provided in Figure 2 .)
Figure 2. The expanded edges E ~ 12 \widetilde{\mathcal{E}}_{12} E 12 , E ~ 13 \widetilde{\mathcal{E}}_{13} E 13 and E ~ 23 \widetilde{\mathcal{E}}_{23} E 23 for the orthant { x ≤ a } \{x \le a\} { x ≤ a } are denoted by the solid gray regions in the case when p = 3 p=3 p = 3 and R = I 3 R=I_3 R = I 3 . The edges E 12 \mathcal{E}_{12} E 12 , E 13 \mathcal{E}_{13} E 13 and E 23 \mathcal{E}_{23} E 23 are denoted by the dashed lines.
We address the disjointness of the expanded edges first. We will show that if there are two points x ~ ∈ E ~ j l \widetilde x\in\widetilde E_{jl} x ∈ E j l and x ~ ′ ∈ E ~ j ′ l ′ \widetilde x'\in\widetilde E_{j'l'} x ′ ∈ E j ′ l ′ satisfying x ~ = x ~ ′ \widetilde x=\widetilde x' x = x ′ , then ( j , l ) = ( j ′ , l ′ ) (j,l)=(j',l') ( j , l ) = ( j ′ , l ′ ) . As an initial step, note that the equation x ~ = x ~ ′ \widetilde x=\widetilde x' x = x ′ can equivalently be written as x + s R e j + t R e l = x ′ + s ′ R e j ′ + t ′ R e l ′ x+sRe_j+tRe_l=x'+s'Re_{j'}+t'Re_{l'} x + s R e j + tR e l = x ′ + s ′ R e j ′ + t ′ R e l ′ , for some x ∈ E j l x\in E_{jl} x ∈ E j l , some x ′ ∈ E j ′ l ′ x'\in E_{j'l'} x ′ ∈ E j ′ l ′ , and some s , t , s ′ , t ′ > 0 s,t,s',t'>0 s , t , s ′ , t ′ > 0 . Rearranging the last equation so that x − x ′ x-x' x − x ′ is on the left, and then applying R − 1 R^{-1} R − 1 to both sides, we obtain R − 1 ( x − x ′ ) = s ′ e j ′ + t ′ e l ′ − s e j − t e l R^{-1}(x-x')=s'e_{j'}+t'e_{l'}-se_j-te_l R − 1 ( x − x ′ ) = s ′ e j ′ + t ′ e l ′ − s e j − t e l . Next, taking the inner product with x − x ′ x-x' x − x ′ directly and noting that x j = a j x_j=a_j x j = a j , x l = a l x_l=a_l x l = a l (and similarly for x ′ x' x ′ ), we have
⟨ x − x ′ , R − 1 ( x − x ′ ) ⟩ = s ′ ( x j ′ − a j ′ ) + t ′ ( x l ′ − a l ′ ) + s ( x j ′ − a j ) + t ( x l ′ − a l ) ≤ 0 , \left\langle x-x',R^{-1}(x-x')\right\rangle=s'(x_{j'}-a_{j'})+t'(x_{l'}-a_{l'})+s(x'_j-a_j)+t(x'_l-a_l)\le0, ⟨ x − x ′ , R − 1 ( x − x ′ ) ⟩ = s ′ ( x j ′ − a j ′ ) + t ′ ( x l ′ − a l ′ ) + s ( x j ′ − a j ) + t ( x l ′ − a l ) ≤ 0 ,
where the inequality holds due to x ≤ a x\le a x ≤ a and x ′ ≤ a x'\le a x ′ ≤ a . On the other hand, since R R R is positive definite, this forces x = x ′ x=x' x = x ′ , so that s e j + t e l = s ′ e j ′ + t ′ e l ′ se_j+te_l=s'e_{j'}+t'e_{l'} s e j + t e l = s ′ e j ′ + t ′ e l ′ . In turn, since j < l j<l j < l and j ′ < l ′ j'<l' j ′ < l ′ , this forces ( j , l ) = ( j ′ , l ′ ) (j,l)=(j',l') ( j , l ) = ( j ′ , l ′ ) . Thus the sets E ~ j l \widetilde E_{jl} E j l are disjoint.
Completing the proof is now reduced to showing (82) . This will be done by directly calculating the probability P ( Y ∈ E ~ j l ) \mathbb{P}(Y\in\widetilde E_{jl}) P ( Y ∈ E j l ) and showing that it is at least ( 1 − R j l 2 ) ν j l / r 2 (1-R_{jl}^{2})\nu_{jl}/r^2 ( 1 − R j l 2 ) ν j l / r 2 , up to an absolute constant. Throughout this calculation, we consider a fixed pair of indices j < l j<l j < l with a j , a l ≤ r a_j,a_l\le r a j , a l ≤ r . An argument similar to the one in the previous paragraph shows that the map T j l ( x , s , t ) = x + s R e j + t R e l T_{jl}(x,s,t)=x+sRe_j+tRe_l T j l ( x , s , t ) = x + s R e j + tR e l is a bijection between E j l × ( 0 , 1 / ( 2 r ) ] 2 E_{jl}\times(0,1/(2r)]^2 E j l × ( 0 , 1/ ( 2 r ) ] 2 and E ~ j l \widetilde E_{jl} E j l . Since every x ∈ E j l x\in E_{jl} x ∈ E j l with x j = a j x_j=a_j x j = a j and x l = a l x_l=a_l x l = a l , we may parametrize E ~ j l \widetilde E_{jl} E j l by the remaining coordinates x k x_k x k with k ≠ j , l k\ne j,l k = j , l . So, the change-of-variables formula gives
P ( Y ∈ E ~ j l ) = ∫ E ~ j l ϕ R ( z ) d z = ∣ det J j l ∣ ∫ E j l ∫ 0 1 / ( 2 r ) ∫ 0 1 / ( 2 r ) ϕ R ( T j l ( x , s , t ) ) d s d t d H p − 2 ( x ) , (83) \mathbb{P}(Y\in\widetilde E_{jl})=\int_{\widetilde E_{jl}}\phi_R(z)\,dz=|\det J_{jl}|\int_{E_{jl}}\int_0^{1/(2r)}\int_0^{1/(2r)}\phi_R(T_{jl}(x,s,t))\,ds\,dt\,d\mathcal{H}^{p-2}(x), \tag*{(83)} P ( Y ∈ E j l ) = ∫ E j l ϕ R ( z ) d z = ∣ det J j l ∣ ∫ E j l ∫ 0 1/ ( 2 r ) ∫ 0 1/ ( 2 r ) ϕ R ( T j l ( x , s , t )) d s d t d H p − 2 ( x ) , (83)
where J j l J_{jl} J j l is the Jacobian matrix of T j l T_{jl} T j l , which does not depend on ( x , s , t ) (x,s,t) ( x , s , t ) because T j l T_{jl} T j l is affine. Since ν j l = ∫ E j l ϕ R d H p − 2 \nu_{jl}=\int_{E_{jl}}\phi_R\,d\mathcal{H}^{p-2} ν j l = ∫ E j l ϕ R d H p − 2 by Lemma 9.1, it is enough to show $\phi_R(T_{jl}(x,s,t))\ge e − 3 / 2 ϕ R ( x ) e^{-3/2}\phi_R(x) e − 3/2 ϕ R ( x ) and ∣ det J j l ∣ = 1 − R j l 2 |\det J_{jl}|=1-R_{jl}^{2} ∣ det J j l ∣ = 1 − R j l 2 , because these two facts and (83) combine to give (82) .
For the determinant formula, note that the k k k th column of J j l J_{jl} J j l is equal to e k e_k e k except when k = j , l k=j,l k = j , l , and in these two cases the columns are R e j Re_j R e j and R e l Re_l R e l . Based on this, the formula ∣ det ( J j l ) ∣ = 1 − R j l 2 |\det(J_{jl})|=1-R_{jl}^{2} ∣ det ( J j l ) ∣ = 1 − R j l 2 can be obtained by reordering the columns of J j l J_{jl} J j l so that it has an upper-left block equal to I p − 2 I_{p-2} I p − 2 and it has a lower-right block with determinant 1 − R j l 2 1-R_{jl}^{2} 1 − R j l 2 . Finally, for the lower bound on the density, if we let x ∈ E j l x\in\mathcal{E}_{jl} x ∈ E j l and v = s e j + t e l v=se_j+te_l v = s e j + t e l with s , t ∈ ( 0 , 1 / ( 2 r ) ] s,t\in(0,1/(2r)] s , t ∈ ( 0 , 1/ ( 2 r )] , then
ϕ R ( T j l ( x , s , t ) ) = ϕ R ( x + R v ) = ϕ R ( x ) exp ( − ⟨ x , v ⟩ − 1 2 ⟨ v , R v ⟩ ) ≥ e − 3 / 2 ϕ R ( x ) . \phi_R(T_{jl}(x,s,t))=\phi_R(x+Rv)=\phi_R(x)\exp\left(-\langle x,v\rangle-\frac{1}{2}\langle v,Rv\rangle\right)\ge e^{-3/2}\phi_R(x). ϕ R ( T j l ( x , s , t )) = ϕ R ( x + R v ) = ϕ R ( x ) exp ( − ⟨ x , v ⟩ − 2 1 ⟨ v , R v ⟩ ) ≥ e − 3/2 ϕ R ( x ) .
where the last bound is due to ⟨ x , v ⟩ = s a j + t a l ≤ 1 \langle x,v\rangle=s a_j+t a_l\le1 ⟨ x , v ⟩ = s a j + t a l ≤ 1 , and ⟨ v , R v ⟩ = s 2 + 2 R j l s t + t 2 ≤ 1 \langle v,Rv\rangle=s^2+2R_{jl}st+t^2\le1 ⟨ v , R v ⟩ = s 2 + 2 R j l s t + t 2 ≤ 1 , because s , t ≤ 1 2 r s,t\le\frac{1}{2r} s , t ≤ 2 r 1 .
Proof of Theorem 2.3
Conditionally on X 1 , … , X n X_1,\ldots,X_n X 1 , … , X n , the bootstrap sum S n ∗ S_n^* S n ∗ is centered Gaussian with covariance matrix Σ ^ \widehat{\Sigma} Σ . So, once d ( Σ ^ , Σ ) d(\widehat{\Sigma},\Sigma) d ( Σ , Σ ) is controlled, Theorem 2.2 can be applied conditionally on X 1 , … , X n X_1,\ldots,X_n X 1 , … , X n with Σ ^ \widehat{\Sigma} Σ playing the role of Σ ~ \widetilde{\Sigma} Σ . This is handled by the next lemma.
Lemma 10.1. There is an absolute constant C C C such that the following holds. Let p , n ≥ 2 p,n\ge2 p , n ≥ 2 and let X 1 , … , X n X_1,\ldots,X_n X 1 , … , X n be i.i.d. centered random vectors in R p \mathbb{R}^p R p with covariance matrix Σ = E ( X 1 X 1 T ) \Sigma=\mathbb{E}(X_1X_1^{\mathsf T}) Σ = E ( X 1 X 1 T ) , such that ∥ ⟨ X 1 , v ⟩ ∥ ψ 2 ≤ B ⟨ v , Σ v ⟩ \|\langle X_1,v\rangle\|_{\psi_2}\le B\sqrt{\langle v,\Sigma v\rangle} ∥ ⟨ X 1 , v ⟩ ∥ ψ 2 ≤ B ⟨ v , Σ v ⟩ for all v ∈ V v\in\mathcal{V} v ∈ V , where V \mathcal{V} V is the set in (2) . Also, let Σ ^ = 1 n ∑ i = 1 n X i X i T \widehat{\Sigma}=\frac{1}{n}\sum_{i=1}^{n}X_iX_i^{\mathsf T} Σ = n 1 ∑ i = 1 n X i X i T . Then, for every α ∈ ( 0 , 1 ) \alpha\in(0,1) α ∈ ( 0 , 1 ) such that n ≥ C log ( p / α ) n\ge C\log(p/\alpha) n ≥ C log ( p / α ) , the event
d ( Σ ^ , Σ ) ≤ C B 2 log ( p / α ) n d(\widehat{\Sigma},\Sigma)\le CB^2\sqrt{\frac{\log(p/\alpha)}{n}} d ( Σ , Σ ) ≤ C B 2 n log ( p / α ) holds with probability at least 1 − α 1-\alpha 1 − α .
Proof. If ⟨ v , Σ v ⟩ = 0 \langle v,\Sigma v\rangle=0 ⟨ v , Σ v ⟩ = 0 holds for some v ∈ V v\in\mathcal{V} v ∈ V , then ⟨ X i , v ⟩ = 0 \langle X_i,v\rangle=0 ⟨ X i , v ⟩ = 0 almost surely for all i ≤ n i\le n i ≤ n , and hence ⟨ v , ( Σ ^ − Σ ) v ⟩ = 0 \langle v,(\widehat{\Sigma}-\Sigma)v\rangle=0 ⟨ v , ( Σ − Σ ) v ⟩ = 0 almost surely. Otherwise, we have ⟨ v , Σ v ⟩ > 0 \langle v,\Sigma v\rangle>0 ⟨ v , Σ v ⟩ > 0 , and in this case the variables ζ i ( v ) = ⟨ X i , v ⟩ / ⟨ v , Σ v ⟩ \zeta_i(v)=\langle X_i,v\rangle/\sqrt{\langle v,\Sigma v\rangle} ζ i ( v ) = ⟨ X i , v ⟩ / ⟨ v , Σ v ⟩ satisfy E ζ i 2 ( v ) = 1 \mathbb{E}\zeta_i^2(v)=1 E ζ i 2 ( v ) = 1 , ∥ ζ i ( v ) ∥ ψ 2 ≤ B \|\zeta_i(v)\|_{\psi_2}\le B ∥ ζ i ( v ) ∥ ψ 2 ≤ B , and
⟨ v , ( Σ ^ − Σ ) v ⟩ ⟨ v , Σ v ⟩ = 1 n ∑ i = 1 n ( ζ i ( v ) 2 − 1 ) . \frac{\langle v,(\widehat{\Sigma}-\Sigma)v\rangle}{\langle v,\Sigma v\rangle}
=\frac{1}{n}\sum_{i=1}^{n}\left(\zeta_i(v)^2-1\right). ⟨ v , Σ v ⟩ ⟨ v , ( Σ − Σ ) v ⟩ = n 1 i = 1 ∑ n ( ζ i ( v ) 2 − 1 ) .
By Bernstein’s inequality (Vershynin, [40 ] ), there is an absolute constant c > 0 c > 0 c > 0 such that P ( ∣ 1 n ∑ i ( ζ i 2 ( v ) − 1 ) ∣ > t ) ≤ 2 exp ( − c n min ( t 2 / B 4 , t / B 2 ) ) \mathbb{P}\left(\left|\frac{1}{n}\sum_i(\zeta_i^2(v)-1)\right| > t\right) \le2\exp\left(-cn\min\left(t^2/B^4,t/B^2\right)\right) P ( n 1 ∑ i ( ζ i 2 ( v ) − 1 ) > t ) ≤ 2 exp ( − c n min ( t 2 / B 4 , t / B 2 ) ) for t > 0 t > 0 t > 0 . Taking a union bound over the p 2 p^2 p 2 vectors in V \mathcal{V} V completes the proof.
Proof of Theorem 2.3. Let Σ = E ( X 1 X 1 T ) \Sigma= \mathbb{E}(X_1X_1^{\mathsf{T}}) Σ = E ( X 1 X 1 T ) and G ∼ N ( 0 , Σ ) G \sim\mathcal{N}(0,\Sigma) G ∼ N ( 0 , Σ ) . By Theorem 2.1 and the triangle inequality, it suffices to show that the following bound holds with probability at least 1 − α 1-\alpha 1 − α ,
sup A ∈ A ∣ P ∗ ( S n ∗ ∈ A ) − P ( G ∈ A ) ∣ ≲ B 2 log ( p / α ) n log ( p ) log ( n ) . (84) \sup_{A\in\mathcal{A}}\left|\mathbb{P}^*(S_n^* \in A)-\mathbb{P}(G\in A)\right| \lesssim B^2\sqrt{\frac{\log(p/\alpha)}{n}}\log(p)\log(n). \tag*{(84)} A ∈ A sup ∣ P ∗ ( S n ∗ ∈ A ) − P ( G ∈ A ) ∣ ≲ B 2 n log ( p / α ) log ( p ) log ( n ) . (84)
Note that we must have B ≥ 1 B \ge1 B ≥ 1 because the ψ 2 \psi_2 ψ 2 -norm upper bounds the L 2 L^2 L 2 -norm. Letting C C C be as in the statement of Lemma 10.1 , we may assume without loss of generality that n ≥ C log ( p / α ) n \ge C\log(p/\alpha) n ≥ C log ( p / α ) , because otherwise (84) is trivially true. Thus, we may apply Lemma 10.1 , and if we let δ = C B 2 log ( p / α ) / n \delta= CB^2\sqrt{\log(p/\alpha)/n} δ = C B 2 log ( p / α ) / n , then the event d ( Σ ^ , Σ ) ≤ δ d(\widehat{\Sigma},\Sigma) \le\delta d ( Σ , Σ ) ≤ δ holds with probability at least 1 − α 1-\alpha 1 − α . On this event, Theorem 2.2 , applied conditionally on X 1 , … , X n X_1,\ldots,X_n X 1 , … , X n with Σ ~ = Σ ^ \widetilde{\Sigma}=\widehat{\Sigma} Σ = Σ , gives
sup A ∈ A ∣ P ∗ ( S n ∗ ∈ A ) − P ( G ∈ A ) ∣ ≲ δ log ( p ) log ( 1 + 1 / δ ) , \sup_{A\in\mathcal{A}}\left|\mathbb{P}^*(S_n^* \in A)-\mathbb{P}(G\in A)\right| \lesssim\delta\log(p)\log(1+1/\delta), A ∈ A sup ∣ P ∗ ( S n ∗ ∈ A ) − P ( G ∈ A ) ∣ ≲ δ log ( p ) log ( 1 + 1/ δ ) ,
and since our choice of δ \delta δ implies log ( 1 + 1 / δ ) ≲ log n \log(1+1/\delta)\lesssim\log n log ( 1 + 1/ δ ) ≲ log n , the proof is complete.
Statement on AI use
Artificial intelligence was used in this work to develop proofs, check the arguments for correctness, and edit language. The authors confirm that the paper conforms with the Institute of Mathematical Statistics policy on use of AI. Essentially all of the text reflects the writing of the authors. The authors retain full responsibility for all aspects of the paper.
References