Corresponding authors: Tuan Pham (tuan.pham@utexas.edu) and Saptarshi Roy (Saptarshi.Roy@mbzuai.ac.ae).
Introduction
Generative models have gained substantial attention over the last few years due to their impressive performance in drawing approximate samples from complex probability distributions. Early successes in deep generative modeling include variational autoencoders (VAE) [27 ] and generative adversarial networks (GAN) [20 ] , which learn data distributions through latent-variable modeling and adversarial training, respectively. Diffusion and score-based models subsequently emerged as powerful approaches that generate samples by learning to reverse a process that gradually transforms data into noise [53 , 25 ] . Their continuous-time formulations connect generative modeling to stochastic and ordinary differential equations (ODEs) that transport a simple reference distribution to the target data distribution [54 ] .
In the past few years, Rectified Flow or Flow matching [36 , 38 , 33 ] has emerged as a prominent sampling methodology due to its state-of-the-art generative performance across various modalities including image generation [39 , 60 ] , audio generation [59 , 34 ] , and video generation [9 , 35 ] . Apart from applications, RF has also attracted several strands of theoretical research including low-dimensional adaptivity [5 , 50 , 28 ] , sampling performance [2 , 58 , 21 , 29 ] , alignment [17 , 48 ] , optimal distillation [49 ] and many more. The success of RF stems from two key factors: (1) RF is a deterministic ordinary differential equation (ODE)-based sampler that is easier to simulate compared to other stochastic samplers, and (2) RF learns almost straight trajectories between source and target data distributions via multiple reflow steps, leading to faster sampling. In particular, the straightness or near-straightness of its ODE trajectories is critical: flows with nearly straight trajectories require fewer sampling steps to generate new samples and are thus more efficient. Though this feature of RF is generally recognized as a main reason for its empirical successes and wide applicability, the theoretical understanding of the conditions under which the RF trajectories are guaranteed to be (near) straight or not has remained limited. In this paper we make a step forward in filling this gap by providing a rigorous framework for understanding the straightness of RFs. The key message is that our framework yields sharp conditions for straightness and identifies precisely when non-straightness can happen. We demonstrate the use of our framework on two common classes of data distributions: smooth, strongly log-concave distributions and Gaussian mixture models (GMMs).
Rectified flows Let us briefly recall the RF framework. In this generative modeling framework, we observe X 1 , … , X n ∼ p d a t a \mathbf{X}_1,\ldots,\mathbf{X}_n \sim p_{\mathrm{data}} X 1 , … , X n ∼ p data , where p d a t a p_{\mathrm{data}} p data is an unknown distribution on R d \mathbb{R}^d R d , and aim to generate new samples approximately from this distribution. The RF framework learns a velocity field v ( x , t ) \mathbf{v}(x,t) v ( x , t ) by minimizing the following objective:
v = arg min f ∫ 0 1 E ∥ X 1 − X 0 − f ( X t , t ) ∥ 2 d t , (1) \mathbf{v}=\underset{f}{\operatorname{arg\,min}}\int_{0}^{1}\mathbb{E}\left\|\mathbf{X}_1-\mathbf{X}_0-f(\mathbf{X}_t,t)\right\|^2\mathrm{d}t, \tag*{(1)} v = f arg min ∫ 0 1 E ∥ X 1 − X 0 − f ( X t , t ) ∥ 2 d t , (1)
where
X t = t X 1 + ( 1 − t ) X 0 , ( X 0 , t ) ∼ N ( 0 , I d ) × Unif ( 0 , 1 ) , \mathbf{X}_t=t\mathbf{X}_1+(1-t)\mathbf{X}_0,\qquad(\mathbf{X}_0,t)\sim\mathcal{N}(0,\mathbf{I}_d)\times\operatorname{Unif}(0,1), X t = t X 1 + ( 1 − t ) X 0 , ( X 0 , t ) ∼ N ( 0 , I d ) × Unif ( 0 , 1 ) ,
with ( X 0 , t ) (\mathbf{X}_0,t) ( X 0 , t ) independent of the data. In the objective (1) above, X t \mathbf{X}_t X t is formed by interpolating the same pair ( X 0 , X 1 ) (\mathbf{X}_0,\mathbf{X}_1) ( X 0 , X 1 ) that appears in the displacement X 1 − X 0 \mathbf{X}_1-\mathbf{X}_0 X 1 − X 0 . For each t t t , the above objective in (1) is minimized at
Figure 1. Schematic of the iterative reflow procedure for constructing K K K -RF.
( x , t ) ↦ v t ( x ) : = v ( x , t ) = E [ X 1 − X 0 ∣ X t = x ] . (2) (x,t) \mapsto v_t(x) := v(x,t) = \mathbb{E}[\mathbf{X}_1-\mathbf{X}_0 \mid X_t=x]. \tag*{(2)} ( x , t ) ↦ v t ( x ) := v ( x , t ) = E [ X 1 − X 0 ∣ X t = x ] . (2)
Under appropriate regularity conditions (investigated in Bansal et al. [ [2 ] ; Mena et al. [ [42 ] ) the velocity is differentiable and the ODE
d Z t = v t ( Z t ) d t , Z 0 ∼ p 0 = N ( 0 , I d ) (3) \mathrm{d}Z_t = v_t(Z_t)\,\mathrm{d}t,\qquad\mathbf{Z}_0 \sim p_0 = \mathcal{N}(\mathbf{0},I_d) \tag*{(3)} d Z t = v t ( Z t ) d t , Z 0 ∼ p 0 = N ( 0 , I d ) (3)
has a unique solution and satisfies the marginal preserving property, i.e., Law ( Z t ) = Law ( X t ) = p t \operatorname{Law}(\mathbf{Z}_t) = \operatorname{Law}(\mathbf{X}_t) = p_t Law ( Z t ) = Law ( X t ) = p t . Under a mild non-explosivity assumption on the ODE and the second moment assumption on p d a t a p_{\mathrm{data}} p data , Bansal et al. [ [2 ] showed that ODE (3) admits a unique solution. Throughout this paper, we assume that the above ODE has a unique solution, which is also necessary for a coupling to be rectifiable [36 , 38 ] .
The initial fitting procedure yields a learned velocity field, which we denote by v ( 1 ) v^{(1)} v ( 1 ) . More generally, after k k k rectification rounds we write v ( k ) v^{(k)} v ( k ) for the learned velocity field at round k k k . To simplify notation, when discussing the first round we will write v : = v ( 1 ) v := v^{(1)} v := v ( 1 ) . We refer to the endpoint coupling induced by v ( 1 ) v^{(1)} v ( 1 ) as the 1-RF coupling:
1 - R F : = ( X 0 ( 1 ) , Z 1 ( 1 ) ( X 0 ( 1 ) ) ) , X 0 ( 1 ) ∼ N ( 0 , I d ) , \mathrm{1\text{-}RF} := \left(\mathbf{X}_0^{(1)},\mathbf{Z}_1^{(1)}\left(\mathbf{X}_0^{(1)}\right)\right),\qquad\mathbf{X}_0^{(1)} \sim\mathcal{N}(\mathbf{0},I_d), 1 - RF := ( X 0 ( 1 ) , Z 1 ( 1 ) ( X 0 ( 1 ) ) ) , X 0 ( 1 ) ∼ N ( 0 , I d ) ,
where the flow map Z t ( 1 ) \mathbf{Z}_t^{(1)} Z t ( 1 ) solves
d d t Z t ( 1 ) ( x ) = v ( 1 ) ( Z t ( 1 ) ( x ) , t ) , Z 0 ( 1 ) ( x ) = x . \frac{\mathrm{d}}{\mathrm{d}t}\mathbf{Z}_t^{(1)}(x) = v^{(1)}\left(\mathbf{Z}_t^{(1)}(x),t\right),\qquad\mathbf{Z}_0^{(1)}(x)=x. d t d Z t ( 1 ) ( x ) = v ( 1 ) ( Z t ( 1 ) ( x ) , t ) , Z 0 ( 1 ) ( x ) = x .
This coupling is deterministic in the sense that, conditional on the learned velocity field, the terminal state is a deterministic function of the initial state X 0 ( 1 ) \mathbf{X}_0^{(1)} X 0 ( 1 ) . We can iterate this procedure to obtain K K K -RF by performing K − 1 K-1 K − 1 additional reflow rounds, as illustrated below: At each subsequent round, a new velocity field is fitted using the paired initial and terminal states generated by the preceding learned flow. We refer readers to Liu et al. [ [38 ] for a comprehensive description of the procedure and its implementation. Once the procedure stops after the K K K -th rectification, new samples are generated by
drawing X 0 ( K ) ∼ N ( 0 , I d ) and computing Z 1 ( K ) ( X 0 ( K ) ) . \text{drawing }\mathbf{X}_0^{(K)} \sim\mathcal{N}(\mathbf{0},I_d)\text{ and computing }\mathbf{Z}_1^{(K)}\left(\mathbf{X}_0^{(K)}\right). drawing X 0 ( K ) ∼ N ( 0 , I d ) and computing Z 1 ( K ) ( X 0 ( K ) ) .
This can be done through numerical integration of the ODE associated with the final learned velocity field v ^ ( K ) \hat{v}^{(K)} v ^ ( K ) . In practice, 2-RF or 3-RF can already produce high-quality samples [38 ] . Interestingly, repeated rectifications seem to increase the straightness; see Theorem D.7 in Liu et al. [ [38 ] .
Despite the impressive empirical success of RF, theoretical understanding and guarantees regarding its straightness remained limited until recently, when Bansal et al. [ [2 ] developed a framework for studying RF straightness. In particular, they showed that 2-RF produces straight flows under certain technical assumptions on the target distribution and the velocity field. A substantial limitation of these results is that they provide only a partial characterization of the conditions under which RF produces straight flows. Furthermore, they offer limited practical guidance for assessing the straightness of trajectories generated by a learned velocity field, which is an important consideration in determining how many rectification steps are needed and whether RF is an appropriate generative modeling framework for a given dataset.
The goal of the present article is to provide a general, simple framework for understanding the straightness property of RF under some common classes of data distributions via the concept of “maximum rotation” which borrows ideas from analysis. In particular, we will be concerned with the following questions:
(i) Given a data distribution p d a t a p_{\mathrm{data}} p data , when does RF produce straight couplings after two rectification steps, as one observes in practice? If not, can it be non-straight even for very regular data distributions?
(ii) Based on the data X 1 , … , X n ∈ R d X_1,\ldots,X_n \in\mathbb{R}^d X 1 , … , X n ∈ R d , how many rectification steps K ≥ 1 K \ge1 K ≥ 1 are needed so that the learned velocity fields after K K K steps give approximately straight couplings that are good enough to use for sampling?
Our results make both theoretical and methodological contributions, which can be summarized as follows.
We develop a general theory for proving the straightness of two-step RF by bounding the maximum angular rotation of the endpoint map. We apply our theory to two concrete, popular classes of distributions: (a) smooth, strongly log-concave distributions and (b) Gaussian mixture models with isotropic covariance matrices.
Using the maximum rotation framework, geometric conditions under which 2-RF produces exactly straight couplings (at the population level) are obtained for these two classes of distributions. We find that the condition number of the log-concave distribution and the maximum cluster mean separation in the case of GMM are the crucial quantities for characterizing the straightness of RF. Additionally, we show that our conditions are sharp by constructing non-straight flows when these geometric conditions fail. This answers question (i) in full generality. To the best of our knowledge, this is the first construction of non-straight couplings obtained from 2-RF.
We consider the problem of assessing the straightness of a learned velocity field, which is crucial for determining the number of rectification steps upon observing the data. We propose a notion of “approximate straightness” and construct a consistent estimator to quantify it. Using this estimator, we construct valid confidence intervals and develop a data-driven stopping rule based on a prescribed straightness tolerance.
On the technical level, the framework developed here is useful for understanding both straightness and non-straightness of the couplings produced by RF. In particular, our results provide the first sharp characterization of RF straightness. Proposition 4.1 below provides a route to proving that a coupling is non-straight, and we expect it to be relevant to other settings beyond those considered in this paper.
Notation. Throughout the paper, with some abuse of notation, we will identify a probability distribution with its Lebesgue density, which we will assume exists unless otherwise noted. For a distribution (Lebesgue density) p p p , write supp ( p ) \operatorname{supp}(p) supp ( p ) to denote the support of p p p . For a random variable X X X we use Law ( X ) \operatorname{Law}(X) Law ( X ) to denote its distribution. For a function f = ( f 1 , … , f d ) : R d → R d \boldsymbol{f} = (f_1,\ldots,f_d): \mathbb{R}^d \to\mathbb{R}^d f = ( f 1 , … , f d ) : R d → R d differentiable at x x x , we denote by D f ( x ) Df(x) D f ( x ) the Jacobian matrix of f f f at x x x
D f ( x ) : = ( ∂ f i ∂ x j ( x ) ) i , j = 1 d . Df(x) := \left(\frac{\partial f_i}{\partial x_j}(x)\right)_{i,j=1}^{d}. D f ( x ) := ( ∂ x j ∂ f i ( x ) ) i , j = 1 d .
When there are many variables, we will also write D x D_x D x to indicate that the Jacobian matrix is taken with respect to x x x . In the case where the function takes real values, we use ∇ \nabla ∇ to indicate its gradient and ∇ 2 \nabla^2 ∇ 2 to indicate its Hessian matrix.
For a set { x j } j ∈ [ K ] ⊂ R \{x_j\}_{j\in[K]} \subset\mathbb{R} { x j } j ∈ [ K ] ⊂ R and probability vector π : = ( π 1 , … , π K ) \boldsymbol{\pi} := (\pi_1,\ldots,\pi_K) π := ( π 1 , … , π K ) , we denote the variance of the set by
Var π ( { x j } j ∈ [ K ] ) = ∑ j ∈ [ K ] π j x j 2 − ( ∑ j ∈ [ K ] π j x j ) 2 . \operatorname{Var}_{\pi}(\{x_j\}_{j\in[K]}) = \sum_{j\in[K]} \pi_j x_j^2 - \left(\sum_{j\in[K]} \pi_j x_j\right)^2. Var π ({ x j } j ∈ [ K ] ) = j ∈ [ K ] ∑ π j x j 2 − j ∈ [ K ] ∑ π j x j 2 .
For two vectors u , v \boldsymbol{u},\boldsymbol{v} u , v , we denote by Angle ( u , v ) ∈ [ 0 , π ] \operatorname{Angle}(\boldsymbol{u},\boldsymbol{v}) \in[0,\pi] Angle ( u , v ) ∈ [ 0 , π ] the angle between u \boldsymbol{u} u and v \boldsymbol{v} v . Given a matrix M ∈ R d × d M \in\mathbb{R}^{d\times d} M ∈ R d × d , we also define the maximum angle
ϕ M : = sup u ∈ R d : M u ≠ 0 Angle ( u , M u ) , (4) \phi_M := \sup_{\boldsymbol{u}\in\mathbb{R}^d: M\boldsymbol{u}\ne0} \operatorname{Angle}(\boldsymbol{u},M\boldsymbol{u}), \tag*{(4)} ϕ M := u ∈ R d : M u = 0 sup Angle ( u , M u ) , (4)
which represents the maximum rotation achieved by the matrix M M M . For two square matrices M M M and N N N , we write their commutator as [ M , N ] = M N − N M [M,N] = MN - NM [ M , N ] = M N − N M . Finally, for a symmetric matrix M M M , we identify its maximum and minimum eigenvalues as λ max ( M ) \lambda_{\max}(M) λ m a x ( M ) and λ min ( M ) \lambda_{\min}(M) λ m i n ( M ) , respectively.
We use G L k ( R ) \mathrm{GL}_k(\mathbb{R}) GL k ( R ) to denote the group of real invertible matrices of size k k k , i.e.
G L k ( R ) : = { A ∈ R k × k : det ( A ) ≠ 0 } . \mathrm{GL}_k(\mathbb{R}) := \left\{A \in\mathbb{R}^{k\times k}: \det(A) \ne0\right\}. GL k ( R ) := { A ∈ R k × k : det ( A ) = 0 } .
Similarly, the space of positive-definite matrices of size k k k will be denoted by S + + k \mathbb{S}_{++}^k S ++ k :
S + + k : = { A : ∃ ε > 0 s.t. A ⪰ ε ⋅ I k } . \mathbb{S}_{++}^k := \left\{A : \exists\varepsilon> 0\ \text{s.t.}\ A \succeq\varepsilon\cdot I_k\right\}. S ++ k := { A : ∃ ε > 0 s.t. A ⪰ ε ⋅ I k } .
Both these spaces are regarded as open subsets of R k 2 \mathbb{R}^{k^2} R k 2 and R k ( k + 1 ) / 2 \mathbb{R}^{k(k+1)/2} R k ( k + 1 ) /2 , respectively, when we talk about differentiability. For example, a map T T T is said to be in C 1 ( [ 0 , 1 ] , G L k ( R ) ) C^1([0,1],\mathrm{GL}_k(\mathbb{R})) C 1 ([ 0 , 1 ] , GL k ( R )) if T T T takes values in G L k ( R ) \mathrm{GL}_k(\mathbb{R}) GL k ( R ) and is C 1 C^1 C 1 as a map from [ 0 , 1 ] [0,1] [ 0 , 1 ] to R k 2 \mathbb{R}^{k^2} R k 2 .
Background and Preliminaries
Straight coupling and straight flow
Rectified flow interpolates between the two distributions in straight line paths as X t = t X 1 + ( 1 − t ) X 0 X_t = tX_1 + (1-t)X_0 X t = t X 1 + ( 1 − t ) X 0 for t ∈ [ 0 , 1 ] t \in[0,1] t ∈ [ 0 , 1 ] , giving v t ( x ) v_t(x) v t ( x ) as in (2) . Therefore, the resulting sampling ODE (3) also approximates a geodesic path between p 0 p_0 p 0 and p 1 p_1 p 1 that allows faster sampling. In the context of RFs, the optimization step (1) is known as a rectification step, and the solution path
Z : = { Z t } t ∈ [ 0 , 1 ] (5) \mathcal{Z} := \{Z_t\}_{t\in[0,1]} \tag*{(5)} Z := { Z t } t ∈ [ 0 , 1 ] (5)
of the ODE (3) is known as the rectified flow, denoted by Z = Rectflow ( X 0 , X 1 ) \mathcal{Z} = \operatorname{Rectflow}(X_0,X_1) Z = Rectflow ( X 0 , X 1 ) . It is known that RF is marginal preserving [38 ] , i.e., Law ( X t ) = Law ( Z t ) \operatorname{Law}(X_t) = \operatorname{Law}(Z_t) Law ( X t ) = Law ( Z t ) . Hence, RF yields a new deterministic coupling ( Z 0 , Z 1 ) : = Rectify ( X 0 , X 1 ) (Z_0,Z_1) := \operatorname{Rectify}(X_0,X_1) ( Z 0 , Z 1 ) := Rectify ( X 0 , X 1 ) between p 0 p_0 p 0 and p 1 p_1 p 1 . In fact, ( Z 0 , Z t ) (Z_0,Z_t) ( Z 0 , Z t ) is also a deterministic coupling between p 0 p_0 p 0 and
p t : = density of Law ( X t ) , (6) p_t := \text{density of }\operatorname{Law}(X_t), \tag*{(6)} p t := density of Law ( X t ) , (6)
for any t ∈ [ 0 , 1 ] t \in[0,1] t ∈ [ 0 , 1 ] . To underscore this deterministic relation, we denote by Z t : R d → R d \mathbf{Z}_t:\mathbb{R}^d \to\mathbb{R}^d Z t : R d → R d the transport/flow map defined through the ODE
d d t Z t ( z 0 ) = v t ( Z t ( z 0 ) ) , Z 0 ( z 0 ) = z 0 , (7) \frac{d}{dt}\mathbf{Z}_t(z_0)=\mathbf{v}_t(\mathbf{Z}_t(z_0)),\quad\mathbf{Z}_0(z_0)=z_0, \tag*{(7)} d t d Z t ( z 0 ) = v t ( Z t ( z 0 )) , Z 0 ( z 0 ) = z 0 , (7)
for an initial point z 0 ∈ R d z_0 \in\mathbb{R}^d z 0 ∈ R d . It is easy to see that Z t = Z t ( Z 0 ) \mathbf{Z}_t=\mathbf{Z}_t(\mathbf{Z}_0) Z t = Z t ( Z 0 ) by comparing (1.3) and (7) . In particular, we have ( Z 0 , Z 1 ) = ( Z 0 , Z 1 ( Z 0 ) ) (Z_0,Z_1)=(Z_0,Z_1(Z_0)) ( Z 0 , Z 1 ) = ( Z 0 , Z 1 ( Z 0 )) for Z 0 ∼ N ( 0 , I d ) Z_0\sim\mathcal{N}(0,I_d) Z 0 ∼ N ( 0 , I d ) . For notational brevity, we also define the chord map
H t ( x ) : = ( 1 − t ) x + t Z 1 ( x ) , x ∈ R d . (8) \mathbf{H}_t(x):=(1-t)x+t\mathbf{Z}_1(x),\qquad x\in\mathbb{R}^d. \tag*{(8)} H t ( x ) := ( 1 − t ) x + t Z 1 ( x ) , x ∈ R d . (8)
Now, we recall the definitions of straight coupling and straight flow from Liu et al. [ [38 ] .
Definition 1 (Straight coupling and Straight flow). A coupling ( Z 0 , Z 1 ( Z 0 ) ) (Z_0,Z_1(Z_0)) ( Z 0 , Z 1 ( Z 0 )) such that E ∥ Z 1 ( Z 0 ) − Z 0 ∥ 2 < ∞ \mathbb{E}\lVert Z_1(Z_0)-Z_0\rVert^2<\infty E ∥ Z 1 ( Z 0 ) − Z 0 ∥ 2 < ∞ is called a straight coupling if
E [ Z 1 ( Z 0 ) − Z 0 ∣ t , H t ( Z 0 ) ] = Z 1 ( Z 0 ) − Z 0 , \mathbb{E}[Z_1(Z_0)-Z_0\mid t,\mathbf{H}_t(Z_0)]=Z_1(Z_0)-Z_0, E [ Z 1 ( Z 0 ) − Z 0 ∣ t , H t ( Z 0 )] = Z 1 ( Z 0 ) − Z 0 , almost surely with respect to independent draws of Z 0 ∼ p 0 Z_0\sim p_0 Z 0 ∼ p 0 and t ∼ Unif ( 0 , 1 ) t\sim\operatorname{Unif}(0,1) t ∼ Unif ( 0 , 1 ) . We call the Rectflow ( Z 0 , Z 1 ( Z 0 ) ) (Z_0,Z_1(Z_0)) ( Z 0 , Z 1 ( Z 0 )) generated by rectification of a straight coupling ( Z 0 , Z 1 ( Z 0 ) ) (Z_0,Z_1(Z_0)) ( Z 0 , Z 1 ( Z 0 )) a straight flow .
It is worth pointing out why rectifying a straight coupling yields constant velocity along each trajectory. Write D = Z 1 ( Z 0 ) − Z 0 D=Z_1(Z_0)-Z_0 D = Z 1 ( Z 0 ) − Z 0 and H t ( Z 0 ) = Z 0 + t D \mathbf{H}_t(Z_0)=Z_0+tD H t ( Z 0 ) = Z 0 + t D . By straightness, the 2-RF velocity satisfies
v t ( 2 ) ( H t ( Z 0 ) ) = E [ D ∣ t , H t ( Z 0 ) ] = D = d d t H t ( Z 0 ) \mathbf{v}^{(2)}_t(\mathbf{H}_t(Z_0))=\mathbb{E}[D\mid t,\mathbf{H}_t(Z_0)]=D=\frac{d}{dt}\mathbf{H}_t(Z_0) v t ( 2 ) ( H t ( Z 0 )) = E [ D ∣ t , H t ( Z 0 )] = D = d t d H t ( Z 0 )
almost surely in ( Z 0 , t ) (Z_0,t) ( Z 0 , t ) . Thus, the linear interpolation solves the 2-RF ODE and, under uniqueness, coincides with its trajectory. Consequently, 2-RF has constant velocity along almost every trajectory, for almost every time. This does not assert that the original 1-RF trajectories are straight, nor that v t ( 2 ) ( x ) \mathbf{v}^{(2)}_t(x) v t ( 2 ) ( x ) is independent of t t t at a fixed x x x . Following this, we state a stronger result that is also true:
Lemma 2.1 (Theorem D.6, Liu et al. [ [38 ] ). A coupling ( X 0 , X 1 ) (X_0,X_1) ( X 0 , X 1 ) is a straight coupling iff Rectflow ( X 0 , X 1 ) (X_0,X_1) ( X 0 , X 1 ) is a straight flow.
The above result is a subtle but crucial result that we use in proving the straightness of 2-RF. It is essentially an adaptation of Theorem D.6 of Liu et al. [ [38 ] , and we add the proof in Appendix § 10 to make the paper self-contained.
Throughout the paper, by 1-RF and 2-RF we mean the application of Rectflow ( ⋅ , ⋅ ) (\cdot,\cdot) ( ⋅ , ⋅ ) once and twice, respectively, starting from an independent coupling ( X 0 , X 1 ) (X_0,X_1) ( X 0 , X 1 ) . Then, provided that 1-RF produces a straight coupling, 2-RF will necessarily achieve straightness due to Lemma 2.1, i.e. it will yield a straight flow, so that samples from a target distribution of X 1 X_1 X 1 can be generated with a single Euler step.
Indeed, the main appeal of the RF machinery, in addition to its inherent conceptual simplicity among ODE-based sampling methods, is the fact that by design it seeks to produce trajectories with minimal curvature. Methodologically, this is a critical feature, as it allows for dramatically faster sampling. However, from a theoretical standpoint, very little is known in the literature about whether or when RF trajectories are provably straight or even approximately straight. To the best of our knowledge, sufficient conditions for straightness are known only for the case of Gaussian-to-Gaussian [42 , 2 , 24 ] and Gaussian-to-isotropic-Gaussian-mixtures, under appropriate (in fact, as we will see, sub-optimal) separation conditions [2 ] . In this paper, we partially fill this gap by fully characterizing the straightness of RF for two classical tasks: sampling from log-concave distributions and from (isotropic) Gaussian mixtures.
In their seminal paper, Liu et al. [ [38 ] showed that successive rectifications eventually lead to a near straight flow that allows faster sampling in the generation stage. Subsequently, Bansal et al. [ [2 ] formulated general conditions under which 2-RF leads to straight flows. Their main finding is that global injectivity of H t ( ⋅ ) \mathbf{H}_t(\cdot) H t ( ⋅ ) leads to straightness of 1-RF coupling. They also showed that the positive-definiteness of the symmetric part of the Jacobian J t ( z 0 ) : = D z 0 Z t ( z 0 ) \mathbf{J}_t(z_0) := D_{z_0}\mathbf{Z}_t(z_0) J t ( z 0 ) := D z 0 Z t ( z 0 ) for all z 0 ∈ R d z_0 \in\mathbb{R}^d z 0 ∈ R d is sufficient to ensure global invertibility of H t ( ⋅ ) \mathbf{H}_t(\cdot) H t ( ⋅ ) . In detail, the positive-definiteness condition1
u T J 1 ( z 0 ) u > 0 ; ∀ z 0 ∈ R d and u ∈ S d − 1 ⟺ φ J 1 ( z 0 ) < π / 2 ; ∀ z 0 ∈ R d (9) \mathbf{u}^{\mathsf{T}}\mathbf{J}_1(z_0)\mathbf{u} > 0;\quad\forall z_0 \in\mathbb{R}^d\text{ and }\mathbf{u} \in\mathbb{S}^{d-1} \Longleftrightarrow\varphi_{\mathbf{J}_1(z_0)} < \pi/2;\quad\forall z_0 \in\mathbb{R}^d \tag*{(9)} u T J 1 ( z 0 ) u > 0 ; ∀ z 0 ∈ R d and u ∈ S d − 1 ⟺ φ J 1 ( z 0 ) < π /2 ; ∀ z 0 ∈ R d (9)
ensures that H t ( ⋅ ) \mathbf{H}_t(\cdot) H t ( ⋅ ) is proper , i.e., lim ∥ z 0 ∥ 2 → ∞ ∥ H t ( z 0 ) ∥ 2 = ∞ \lim_{\lVert z_0\rVert_2\to\infty}\lVert\mathbf{H}_t(z_0)\rVert_2=\infty lim ∥ z 0 ∥ 2 → ∞ ∥ H t ( z 0 ) ∥ 2 = ∞ , and det ( D H t ( z 0 ) ) > 0 \det(D\mathbf{H}_t(z_0))>0 det ( D H t ( z 0 )) > 0 , which yield global invertibility of H t ( ⋅ ) \mathbf{H}_t(\cdot) H t ( ⋅ ) (cf. Plastock [1974]; Ruzhansky and Sugimoto [ [51 ] ). Furthermore, Bansal et al. [ [2 ] noted that the Jacobian J t ( z 0 ) \mathbf{J}_t(z_0) J t ( z 0 ) satisfies the ODE
d d t J t ( z 0 ) = D v t ( Z t ( z 0 ) ) J t ( z 0 ) , J 0 ( z 0 ) = I d , (10) \frac{d}{dt}\mathbf{J}_t(z_0)=D\mathbf{v}_t(\mathbf{Z}_t(z_0))\mathbf{J}_t(z_0),\quad\mathbf{J}_0(z_0)=\mathbf{I}_d, \tag*{(10)} d t d J t ( z 0 ) = D v t ( Z t ( z 0 )) J t ( z 0 ) , J 0 ( z 0 ) = I d , (10)
and used it to examine the properties of the Jacobian flow across different target models. In contrast, in this paper we investigate the injectivity of H t ( ⋅ ) \mathbf{H}_t(\cdot) H t ( ⋅ ) directly, without relying on the ODE (10) .
Related work Our results complement several recent works on sufficient and necessary conditions for straightness. Although Bansal et al. [ [2 ] established straightness of 1-RF under Condition (9) , they observed experimentally that this condition can fail for low-variance Gaussian mixtures. This failure was subsequently established rigorously by Pierret et al. [ [45 ] using a tessellation argument. Importantly, however, failure of the Jacobian condition alone does not imply that the resulting RF coupling is non-straight. Related positive results were obtained by Chambolle and Hertrich [ [7 ] , who introduced weak rectification and established straightness in a limiting sense under a generalization of Condition (9) , and by Tsimpos et al. [ [56 ] under similar structural assumptions.
On the negative side, Hertrich et al. [ [24 ] and Tsimpos et al. [ [56 ] showed that rectification can produce non-straight couplings when the underlying distributions have compact and disconnected supports. Such examples, however, are qualitatively different from the standard generative-modeling setting, where the source distribution is typically Gaussian and therefore has connected and unbounded support. Additionally, Mena et al. [ [42 ] showed that compact support may not even warrant existence of RF. Our results show that compact or disconnected support is not necessary for non-straightness. In Theorems 4.1 and 4.2 , we construct examples in which the source is Gaussian and the target is absolutely continuous (log-concave or a mixture of Gaussians), yet the coupling obtained after one rectification is non-straight. As a key ingredient, we develop a general criterion for certifying non-straightness (Proposition 4.1), which may be of independent interest.
Maximum rotation and straightness
In this section, we present our proposed criterion for straightness, which is based on the angle between the difference of two initial points and the difference of their images under the endpoint map; see Theorem 3.1 for the precise result. We apply this criterion in Subsections 3.2 and 3.3 to derive sufficient conditions for straightness for some popular types of data distributions. Sharpness of this condition will be studied in § 4 .
Maximum rotation
We begin by ensuring that distinct initial points remain distinct under the ODE flow. Throughout this section, we assume that the flow in (7) is defined over [ 0 , 1 ] [0,1] [ 0 , 1 ] for any initial point and that its Borel measurable velocity field satisfies
∫ 0 1 ( ∥ v t ( 0 ) ∥ 2 + sup ∥ z ∥ 2 ≤ R ∥ D v t ( z ) ∥ o p ) d t < ∞ , for every R > 0. (11) \int_{0}^{1}\left(\lVert v_t(0)\rVert_2+\sup_{\lVert z\rVert_2\le R}\lVert Dv_t(z)\rVert_{\mathrm{op}}\right)\,\mathrm{d}t<\infty,\qquad\text{for every }R>0. \tag*{(11)} ∫ 0 1 ( ∥ v t ( 0 ) ∥ 2 + ∥ z ∥ 2 ≤ R sup ∥ D v t ( z ) ∥ op ) d t < ∞ , for every R > 0. (11)
By applying uniqueness backward in time [1 Lemma 8.1.4] , the endpoint map Z 1 \mathbf{Z}_1 Z 1 is injective. For completeness, we add a relevant discussion in Appendix § 11 .§ 1 . Thus, H 0 = I d \mathbf{H}_0=\mathrm{Id} H 0 = Id and H 1 = Z 1 \mathbf{H}_1=\mathbf{Z}_1 H 1 = Z 1 are injective, and it remains to consider t ∈ ( 0 , 1 ) t\in(0,1) t ∈ ( 0 , 1 ) .
Suppose H t ( ⋅ ) \mathbf{H}_t(\cdot) H t ( ⋅ ) is not injective for some t ∈ ( 0 , 1 ) t\in(0,1) t ∈ ( 0 , 1 ) . Then there exist distinct points x , y \boldsymbol{x},\boldsymbol{y} x , y such that H t ( x ) = H t ( y ) \mathbf{H}_t(\boldsymbol{x})=\mathbf{H}_t(\boldsymbol{y}) H t ( x ) = H t ( y ) , i.e.,
Z 1 ( y ) − Z 1 ( x ) = − 1 − t t ( y − x ) = − 1 − t t ( Z 0 ( y ) − Z 0 ( x ) ) . (12) \mathbf{Z}_1(\boldsymbol{y})-\mathbf{Z}_1(\boldsymbol{x})=-\frac{1-t}{t}(\boldsymbol{y}-\boldsymbol{x})=-\frac{1-t}{t}\left(\mathbf{Z}_0(\boldsymbol{y})-\mathbf{Z}_0(\boldsymbol{x})\right). \tag*{(12)} Z 1 ( y ) − Z 1 ( x ) = − t 1 − t ( y − x ) = − t 1 − t ( Z 0 ( y ) − Z 0 ( x ) ) . (12)
Therefore, by writing
δ t ( x , y ) : = Z t ( y ) − Z t ( x ) , \boldsymbol{\delta}_t(\boldsymbol{x},\boldsymbol{y}):=\mathbf{Z}_t(\boldsymbol{y})-\mathbf{Z}_t(\boldsymbol{x}), δ t ( x , y ) := Z t ( y ) − Z t ( x ) ,
the identity (12) implies that the angle between δ 1 ( x , y ) \boldsymbol{\delta}_1(\boldsymbol{x},\boldsymbol{y}) δ 1 ( x , y ) and δ 0 ( x , y ) \boldsymbol{\delta}_0(\boldsymbol{x},\boldsymbol{y}) δ 0 ( x , y ) is π \pi π . Therefore, one equivalent condition for global injectivity of H t ( ⋅ ) \mathbf{H}_t(\cdot) H t ( ⋅ ) is
Angle ( δ 1 ( x , y ) , δ 0 ( x , y ) ) < π for all x ≠ y . (13) \operatorname{Angle}\left(\boldsymbol{\delta}_1(\boldsymbol{x},\boldsymbol{y}),\boldsymbol{\delta}_0(\boldsymbol{x},\boldsymbol{y})\right)<\pi\qquad\text{for all }\boldsymbol{x}\ne\boldsymbol{y}. \tag*{(13)} Angle ( δ 1 ( x , y ) , δ 0 ( x , y ) ) < π for all x = y . (13)
Lemma 3.1. If (11) and (13) hold, then H t \mathbf{H}_t H t is globally injective for all t ∈ [ 0 , 1 ] t\in[0,1] t ∈ [ 0 , 1 ] . Consequently, the 1-RF endpoint coupling ( Z 0 , Z 1 ( Z 0 ) ) (\mathbf{Z}_0,\mathbf{Z}_1(\mathbf{Z}_0)) ( Z 0 , Z 1 ( Z 0 )) is straight in the sense of Definition 1.
The proof can be found in Appendix § 11 .§ 2 . This is a slight modification of what was used in the proof of Theorem 6 in Bansal et al. [ [2 ] , which requires H t \mathbf{H}_t H t to be a diffeomorphism to obtain straightness. The rest of this subsection focuses on providing sufficient conditions for the inequality (13) to hold, which in turn yields a criterion for straightness. To set the stage, we begin by noticing the following direct consequence of the ODE (7) :
d d t δ t ( x , y ) = v t ( Z t ( y ) ) − v t ( Z t ( x ) ) = [ ∫ 0 1 D v t ( ξ ⋅ Z t ( y ) + ( 1 − ξ ) ⋅ Z t ( x ) ) d ξ ] δ t ( x , y ) . \begin{aligned}
\frac{\mathrm{d}}{\mathrm{d}t}\boldsymbol{\delta}_t(\boldsymbol{x},\boldsymbol{y})=v_t(\mathbf{Z}_t(\boldsymbol{y}))-v_t(\mathbf{Z}_t(\boldsymbol{x})) \\
&=\left[\int_{0}^{1}D v_t\left(\xi\cdot\mathbf{Z}_t(\boldsymbol{y})+(1-\xi)\cdot\mathbf{Z}_t(\boldsymbol{x})\right)\,\mathrm{d}\xi\right]\boldsymbol{\delta}_t(\boldsymbol{x},\boldsymbol{y}).
\end{aligned} d t d δ t ( x , y ) = v t ( Z t ( y )) − v t ( Z t ( x )) = [ ∫ 0 1 D v t ( ξ ⋅ Z t ( y ) + ( 1 − ξ ) ⋅ Z t ( x ) ) d ξ ] δ t ( x , y ) .
Next, setting
A t ( x , y ) : = ∫ 0 1 D v t ( ξ ⋅ Z t ( y ) + ( 1 − ξ ) ⋅ Z t ( x ) ) d ξ , (14) A_t(\boldsymbol{x},\boldsymbol{y}):=\int_{0}^{1}D v_t\left(\xi\cdot\mathbf{Z}_t(\boldsymbol{y})+(1-\xi)\cdot\mathbf{Z}_t(\boldsymbol{x})\right)\,\mathrm{d}\xi, \tag*{(14)} A t ( x , y ) := ∫ 0 1 D v t ( ξ ⋅ Z t ( y ) + ( 1 − ξ ) ⋅ Z t ( x ) ) d ξ , (14)
the ODE in the above display can be simply expressed as
d d t δ t ( x , y ) = A t ( x , y ) ⋅ δ t ( x , y ) . (15) \frac{\mathrm{d}}{\mathrm{d}t}\boldsymbol{\delta}_t(\boldsymbol{x},\boldsymbol{y})=A_t(\boldsymbol{x},\boldsymbol{y})\cdot\boldsymbol{\delta}_t(\boldsymbol{x},\boldsymbol{y}). \tag*{(15)} d t d δ t ( x , y ) = A t ( x , y ) ⋅ δ t ( x , y ) . (15)
The ODE (15) provides control on the maximum angular rotation of δ 1 ( x , y ) \boldsymbol{\delta}_1(\boldsymbol{x},\boldsymbol{y}) δ 1 ( x , y ) under some assumptions on A t ( x , y ) A_t(\boldsymbol{x},\boldsymbol{y}) A t ( x , y ) . It turns out that the spectral properties of A t A_t A t can be used to deduce sufficient conditions for (13) . Next, we present a heuristic argument for deriving (13) from the spectral properties of A t A_t A t . All of the derivations below can be made rigorous; see Appendix § 11 .§ 2 for more details.
Bounding the maximum rotation via the discretized solution The discretized solution of (15) takes the following form:
δ 1 d i s c ( x , y ) = [ I d + 1 N A t N − 1 ( x , y ) ] ⋯ [ I d + 1 N A t 1 ( x , y ) ] [ I d + 1 N A t 0 ( x , y ) ] δ 0 ( x , y ) . \delta^{\mathrm{disc}}_1(x,y)=\left[I_d+\frac{1}{N}A_{t_{N-1}}(x,y)\right]\cdots\left[I_d+\frac{1}{N}A_{t_1}(x,y)\right]\left[I_d+\frac{1}{N}A_{t_0}(x,y)\right]\delta_0(x,y). δ 1 disc ( x , y ) = [ I d + N 1 A t N − 1 ( x , y ) ] ⋯ [ I d + N 1 A t 1 ( x , y ) ] [ I d + N 1 A t 0 ( x , y ) ] δ 0 ( x , y ) .
For notational convenience, define
A ~ j = I d + 1 N A t j ( x , y ) ; and A [ j ] = A ~ j ⋯ A ~ 0 , j = 0 , … , N − 1. \widetilde{A}_j=I_d+\frac{1}{N}A_{t_j}(x,y);\qquad\text{and}\qquad A^{[j]}=\widetilde{A}_j\cdots\widetilde{A}_0,\qquad j=0,\ldots,N-1. A j = I d + N 1 A t j ( x , y ) ; and A [ j ] = A j ⋯ A 0 , j = 0 , … , N − 1.
Choose N N N sufficiently large such that the matrices
{ A ~ j } j = 0 N − 1 \left\{\widetilde{A}_j\right\}_{j=0}^{N-1} { A j } j = 0 N − 1
are symmetric and positive definite. Consider two cases:
Case 1: A ~ j \widetilde{A}_j A j ’s are commutative. In this case, Angle ( δ 1 d i s c ( x , y ) , δ 0 ( x , y ) ) < π / 2 \operatorname{Angle}\left(\delta^{\mathrm{disc}}_1(x,y),\delta_0(x,y)\right)<\pi/2 Angle ( δ 1 disc ( x , y ) , δ 0 ( x , y ) ) < π /2 as A [ N − 1 ] A^{[N-1]} A [ N − 1 ] is symmetric and positive definite.
Case 2: A ~ j \widetilde{A}_j A j ’s are not commutative. In this case, we can only bound the Angle ( u , A [ N − 1 ] u ) \operatorname{Angle}(u,A^{[N-1]}u) Angle ( u , A [ N − 1 ] u ) . First note that by elementary geodesic theory on S d − 1 \mathbb{S}^{d-1} S d − 1 , the following holds:
Angle ( δ 0 ( x , y ) , A [ N − 1 ] δ 0 ( x , y ) ) ≤ Angle ( δ 0 ( x , y ) , A [ 0 ] δ 0 ( x , y ) ) + ∑ j = 0 N − 2 Angle ( A [ j ] δ 0 ( x , y ) , A ~ j + 1 A [ j ] δ 0 ( x , y ) ) ≤ ∑ j = 0 N − 1 ϕ A ~ j . \begin{aligned}
\operatorname{Angle}\left(\delta_0(x,y),A^{[N-1]}\delta_0(x,y)\right)
&\leq\operatorname{Angle}\left(\delta_0(x,y),A^{[0]}\delta_0(x,y)\right)\\
&\quad+\sum_{j=0}^{N-2}\operatorname{Angle}\left(A^{[j]}\delta_0(x,y),\widetilde{A}_{j+1}A^{[j]}\delta_0(x,y)\right)\\
&\leq\sum_{j=0}^{N-1}\phi_{\widetilde{A}_j}.
\end{aligned} Angle ( δ 0 ( x , y ) , A [ N − 1 ] δ 0 ( x , y ) ) ≤ Angle ( δ 0 ( x , y ) , A [ 0 ] δ 0 ( x , y ) ) + j = 0 ∑ N − 2 Angle ( A [ j ] δ 0 ( x , y ) , A j + 1 A [ j ] δ 0 ( x , y ) ) ≤ j = 0 ∑ N − 1 ϕ A j .
Therefore, to ensure Angle ( δ 1 d i s c ( x , y ) , δ 0 ( x , y ) ) < π \operatorname{Angle}\left(\delta^{\mathrm{disc}}_1(x,y),\delta_0(x,y)\right)<\pi Angle ( δ 1 disc ( x , y ) , δ 0 ( x , y ) ) < π , we need ∑ j = 0 N − 1 ϕ A ~ j < π \sum_{j=0}^{N-1}\phi_{\widetilde{A}_j}<\pi ∑ j = 0 N − 1 ϕ A j < π . It turns out that this term can be bounded using
∑ j = 0 N − 1 ϕ A ~ j = 1 2 ∫ 0 1 { λ max [ A t ( x , y ) ] − λ min [ A t ( x , y ) ] } d t + o ( 1 ) . \sum_{j=0}^{N-1}\phi_{\widetilde{A}_j}=\frac{1}{2}\int_0^1\left\{\lambda_{\max}[A_t(x,y)]-\lambda_{\min}[A_t(x,y)]\right\}\,\mathrm{d}t+o(1). j = 0 ∑ N − 1 ϕ A j = 2 1 ∫ 0 1 { λ m a x [ A t ( x , y )] − λ m i n [ A t ( x , y )] } d t + o ( 1 ) .
By letting N → ∞ N\to\infty N → ∞ , we deduce that
Angle ( δ 1 ( x , y ) , δ 0 ( x , y ) ) ≈ lim N → ∞ Angle ( δ 1 d i s c ( x , y ) , δ 0 ( x , y ) ) ≤ 1 2 ∫ 0 1 { λ max [ A t ( x , y ) ] − λ min [ A t ( x , y ) ] } d t . \begin{aligned}
\operatorname{Angle}\left(\delta_1(x,y),\delta_0(x,y)\right)
&\approx\lim_{N\to\infty}\operatorname{Angle}\left(\delta^{\mathrm{disc}}_1(x,y),\delta_0(x,y)\right)\\
&\leq\frac{1}{2}\int_0^1\left\{\lambda_{\max}[A_t(x,y)]-\lambda_{\min}[A_t(x,y)]\right\}\,\mathrm{d}t.
\end{aligned} Angle ( δ 1 ( x , y ) , δ 0 ( x , y ) ) ≈ N → ∞ lim Angle ( δ 1 disc ( x , y ) , δ 0 ( x , y ) ) ≤ 2 1 ∫ 0 1 { λ m a x [ A t ( x , y )] − λ m i n [ A t ( x , y )] } d t .
The above heuristic can be proved rigorously and suggests our proposed criterion, as stated in the next theorem.
Theorem 3.1 (Straightness criterion). Let ( X 0 , X 1 ) ∼ N ( 0 , I d ) ⊗ p 1 (X_0,X_1)\sim\mathcal{N}(0,I_d)\otimes p_1 ( X 0 , X 1 ) ∼ N ( 0 , I d ) ⊗ p 1 with E ∥ X 1 ∥ 2 2 < ∞ \mathbb{E}\left\|X_1\right\|_2^2<\infty E ∥ X 1 ∥ 2 2 < ∞ , and let v t v_t v t be the associated RF velocity field. Assume that the flow Z t Z_t Z t defined in (7) exists on [ 0 , 1 ] [0,1] [ 0 , 1 ] for every initial point and satisfies the regularity condition (11) . With A t A_t A t defined in (14) , suppose that at least one of the following conditions holds:
(a) Commutativity. For every x ≠ y x \ne y x = y ,
[ A t ( x , y ) , A s ( x , y ) ] = 0 , s , t ∈ ( 0 , 1 ) . [A_t(x,y),A_s(x,y)] = 0, \qquad s,t \in(0,1). [ A t ( x , y ) , A s ( x , y )] = 0 , s , t ∈ ( 0 , 1 ) .
Here [ A , B ] [A,B] [ A , B ] denotes the commutator of two matrices, see the notation in § 1 for the precise definition.
(b) Bounded spectral spread. For every x ≠ y x \ne y x = y ,
1 2 ∫ 0 1 ( λ max [ A t ( x , y ) ] − λ min [ A t ( x , y ) ] ) d t < π . \frac{1}{2}\int_0^1 \left(\lambda_{\max}[A_t(x,y)]-\lambda_{\min}[A_t(x,y)]\right)\,\mathrm{d}t < \pi. 2 1 ∫ 0 1 ( λ m a x [ A t ( x , y )] − λ m i n [ A t ( x , y )] ) d t < π .
Then the angle condition (13) holds, and H t \mathbf{H}_t H t is globally injective for every t ∈ [ 0 , 1 ] t \in[0,1] t ∈ [ 0 , 1 ] . Consequently, the 1-RF endpoint coupling
( Z 0 , Z 1 ) = Rectify ( X 0 , X 1 ) (Z_0,Z_1)=\operatorname{Rectify}(X_0,X_1) ( Z 0 , Z 1 ) = Rectify ( X 0 , X 1 )
is straight in the sense of Definition 1.
The proof is deferred to Appendix B.2. Condition (a) (commutativity) holds trivially in the one-dimensional case. In Bansal et al. [ [2 ] , the authors showed the stronger commutativity condition [ D v t ( x ) , D v s ( y ) ] = 0 [Dv_t(x),Dv_s(y)]=0 [ D v t ( x ) , D v s ( y )] = 0 for all pairs of ( x , y ) (x,y) ( x , y ) and ( s , t ) (s,t) ( s , t ) when the target distribution is N ( μ , Σ ) \mathcal{N}(\mu,\Sigma) N ( μ , Σ ) or a two-component GMM with isotropic covariance. Therefore, 1-RF yields a straight coupling in the above cases. In the next subsections, we will derive model-specific conditions for straightness under other popular and more general types of data distributions.
Remark 3.1. Letting
Δ ( t ) : = sup z ∈ R d ( λ max [ D v t ( z ) ] − λ min [ D v t ( z ) ] ) , t ∈ [ 0 , 1 ] \Delta(t) := \sup_{z \in\mathbb{R}^{d}}\left(\lambda_{\max}[Dv_t(z)]-\lambda_{\min}[Dv_t(z)]\right), \qquad t \in[0,1] Δ ( t ) := z ∈ R d sup ( λ m a x [ D v t ( z )] − λ m i n [ D v t ( z )] ) , t ∈ [ 0 , 1 ]
and noting that λ max [ A t ( x , y ) ] − λ min [ A t ( x , y ) ] ≤ Δ ( t ) \lambda_{\max}[A_t(x,y)]-\lambda_{\min}[A_t(x,y)] \le\Delta(t) λ m a x [ A t ( x , y )] − λ m i n [ A t ( x , y )] ≤ Δ ( t ) for all t t t , Condition (b) in Theorem 3.1 is implied by the stronger condition
1 2 ∫ 0 1 Δ ( t ) d t < π . \frac{1}{2}\int_0^1 \Delta(t)\,\mathrm{d}t < \pi. 2 1 ∫ 0 1 Δ ( t ) d t < π .
Straightness when generating log-concave distributions
In this section we are going to investigate conditions under which straightness holds when the target distribution is log-concave. Towards that goal, suppose that the target distribution admits a Lebesgue density of the form
p 1 ( x ) = exp ( − V ( x ) ) , p_1(x)=\exp(-V(x)), p 1 ( x ) = exp ( − V ( x )) ,
for some C 3 C^3 C 3 function V : R d → R V:\mathbb{R}^{d}\to\mathbb{R} V : R d → R . We further assume that
m I d ⪯ ∇ 2 V ( x ) ⪯ L I d (16) mI_d \preceq\nabla^2 V(x) \preceq LI_d \tag*{(16)} m I d ⪯ ∇ 2 V ( x ) ⪯ L I d (16)
for some constants 0 < m < L 0<m<L 0 < m < L . Distributions satisfying the above condition (16) are known as smooth and strongly log-concave distributions and have been studied extensively in the optimization and machine learning literature; see, for example, Dalalyan [ [14 ] ; Durmus et al. [ [15 ] and the references therein. In our next result, we verify condition (b) of Theorem 3.1 to formulate a condition that guarantees the straightness of couplings obtained from 2-RF when the data distribution satisfies (16) .
Theorem 3.2. If L / m < exp ( 4 π ) L/m < \exp(4\pi) L / m < exp ( 4 π ) , then 2-RF is straight when applied to the independent coupling N ( 0 , I d ) ⊗ p 1 \mathcal{N}(0,I_d) \otimes p_1 N ( 0 , I d ) ⊗ p 1 .
Thus, Theorem 3.2 shows that the condition number can determine the straightness of RF starting from independent couplings. This should not come as a big surprise: the condition number is a central geometric parameter in the log-concave sampling literature; see, e.g., Chewi [ [11 ] and the references therein. It captures the interaction between concentration and the potential’s smoothness. This parameter also appears in recent high-accuracy sampling guarantees. For example, under their oracle model, Chen et al. [ [8 ] obtain a complexity bound that depends linearly on C L S I C_{\mathrm{LSI}} C LSI , where C L S I C_{\mathrm{LSI}} C LSI is the log-Sobolev constant of the target distribution; see Definition B.2 in Chen et al. [ [8 ] . Since C L S I ≤ 1 / m C_{\mathrm{LSI}} \le1/m C LSI ≤ 1/ m under strong log-concavity, this effective condition number is bounded by L / m L/m L / m . Our straightness result thus gives a complementary geometric interpretation of the same parameter.
The proof of Theorem 3.2 hinges on bounding the maximum rotation angle from equation (13) . In particular, we show that
sup x ≠ y { Angle ( δ 1 ( x , y ) , δ 0 ( x , y ) ) } ≤ 1 2 [ ∫ 0 1 s − m ( 1 − s ) s 2 + m ( 1 − s ) 2 d s − ∫ 0 1 s − L ( 1 − s ) s 2 + L ( 1 − s ) 2 d s ] = 1 4 log ( L m ) , \sup_{x \ne y} \left\{ \operatorname{Angle}(\delta_1(x,y),\delta_0(x,y)) \right\} \le\frac{1}{2}\left[\int_0^1 \frac{s-m(1-s)}{s^2+m(1-s)^2}\,ds-\int_0^1 \frac{s-L(1-s)}{s^2+L(1-s)^2}\,ds\right]
= \frac{1}{4}\log\left(\frac{L}{m}\right), x = y sup { Angle ( δ 1 ( x , y ) , δ 0 ( x , y )) } ≤ 2 1 [ ∫ 0 1 s 2 + m ( 1 − s ) 2 s − m ( 1 − s ) d s − ∫ 0 1 s 2 + L ( 1 − s ) 2 s − L ( 1 − s ) d s ] = 4 1 log ( m L ) ,
a bound implying that condition (b) in Theorem 3.1 holds if
L / m < exp ( 4 π ) ≈ 286751. L/m < \exp(4\pi) \approx286751. L / m < exp ( 4 π ) ≈ 286751.
In other words, if the condition number L / m L/m L / m of the distribution is smaller than exp ( 4 π ) \exp(4\pi) exp ( 4 π ) , then 2-RF always yields straight couplings when applied to the independent coupling N ( 0 , I d ) ⊗ p 1 \mathcal{N}(0,I_d) \otimes p_1 N ( 0 , I d ) ⊗ p 1 . To the best of our knowledge, this is the first straightness result under log-concave distributions. Interestingly, this upper bound on the distribution’s condition number is dimension-free .
At first glance, one might wonder how sharp this criterion based on condition numbers is. Since the bound is independent of the dimension, it does not capture any high-dimensional effects, and one might expect it to become loose as the dimension increases. This turns out not to be the case. Surprisingly, we show that in any dimension d ≥ 3 d \ge3 d ≥ 3 , this condition is necessary to guarantee straightness for all data distributions satisfying (3.6); see § 4.2 for more details.
Straightness when generating isotropic GMMs
Next, we consider another class of popular data distributions, the Gaussian mixture models, arguably one of the most popular and widely studied classes of mixture models, with huge expressive power. The readers are referred to McLachlan and Peel [ [41 ] ; Bishop [ [4 ] ; Nguyen et al. [ [44 ] ; Lee et al. [ [31 ] ; Chen et al. [ [10 ] for basic properties of GMMs and their applications in machine learning. Below, we assume that
X 1 ∼ p G M M : = ∑ j = 1 κ π j N ( μ j , σ 2 I d ) . (17) X_1 \sim p_{\mathrm{GMM}} := \sum_{j=1}^{\kappa} \pi_j \mathcal{N}(\mu_j,\sigma^2 I_d). \tag*{(17)} X 1 ∼ p GMM := j = 1 ∑ κ π j N ( μ j , σ 2 I d ) . (17)
To describe our straightness result, put
D max : = max i ≠ j ∥ μ i − μ j ∥ 2 . D_{\max} := \max_{i \ne j} \lVert\mu_i - \mu_j \rVert_2. D m a x := i = j max ∥ μ i − μ j ∥ 2 .
The quantity D max D_{\max} D m a x is the maximum interpoint distance (see Heiny and Kleemann [ [23 ] for more details) formed by the set { μ 1 , … , μ K } \{\mu_1,\ldots,\mu_K\} { μ 1 , … , μ K } . For data with distributions of GMM-types in (17) , our criterion for straightness reads
Theorem 3.3. If D max 2 < 16 π σ 2 D_{\max}^2 < 16\pi\sigma^2 D m a x 2 < 16 π σ 2 , then 2-RF is straight when applied to the independent coupling N ( 0 , I d ) ⊗ p G M M \mathcal{N}(0,I_d)\otimes p_{\mathrm{GMM}} N ( 0 , I d ) ⊗ p GMM .
Theorem 3.3 states that the dimension-free bound on the maximum interpoint distance
D max 2 < 16 π ⋅ σ 2 D_{\max}^2 < 16\pi\cdot\sigma^2 D m a x 2 < 16 π ⋅ σ 2
is enough to guarantee the straightness of 2-RF. We note that a weaker version of Theorem 3.3 was established in Bansal et al. [ [2 ] , where the authors obtained the sub-optimal condition D max 2 ≤ 4 σ 2 D_{\max}^2 \le4\sigma^2 D m a x 2 ≤ 4 σ 2 . Similar to the smooth, strongly log-concave case, our dimension-free condition is sharp; see § 4.3 for more details. Roughly speaking, for any constant greater than 16 π 16\pi 16 π , there exists a distribution of the form (17) such that 2-RF provably yields a non-straight coupling when applied to the independent coupling.
It is worth pointing out that a variant of the normalized maximal mean separation D max / σ D_{\max}/\sigma D m a x / σ appears in recent diffusion sampling guarantees. Under some score regularity and estimation assumptions, Gatmiry et al. [ [19 ] obtain high-accuracy sampling guarantees whose complexity depends quadratically on R / σ R/\sigma R / σ , up to logarithmic factors, where R R R bounds the radius of the mixing distribution. For isotropic GMMs, the smallest enclosing radius R ⋆ R_{\star} R ⋆ of the component means satisfies
D max 2 σ ≤ R ⋆ σ ≤ D max σ . \frac{D_{\max}}{2\sigma} \le\frac{R_{\star}}{\sigma} \le\frac{D_{\max}}{\sigma}. 2 σ D m a x ≤ σ R ⋆ ≤ σ D m a x .
Thus, after centering, their normalized radius is equivalent, up to a universal factor, to our normalized maximal mean separation. Therefore, our result gives a complementary geometric interpretation of this ratio by identifying a threshold that guarantees exact straightness.
Sharpness of the maximum rotation criterion
In § 3 , we established sufficient conditions for the straightness of 2-RF through the maximum-rotation framework. In particular, we showed that 2-RF generates a straight coupling for every smooth, strongly log-concave target distribution whose condition number is smaller than e 4 π e^{4\pi} e 4 π . The same argument guarantees straightness for isotropic Gaussian mixtures with identity component covariances whenever the squared maximum pairwise distance between the component means is smaller than 16 π 16\pi 16 π . These results naturally raise the question of whether the thresholds imposed by the maximum-rotation criterion are merely artifacts of our analysis, or whether they reflect a genuine obstruction to straightness. The purpose of this section is to show that the latter is the case. More precisely, we construct log-concave and Gaussian-mixture examples for which one reflow produces a non-straight coupling once the corresponding maximum-rotation criterion is violated. Thus, the thresholds obtained in § 3 are, in this sense, sharp.
Throughout this section, Z t Z_t Z t denotes the RF transport map defined in (7) , and Z 1 Z_1 Z 1 is thus the endpoint map. Therefore, ( Z 0 , Z 1 ( Z 0 ) ) (Z_0,Z_1(Z_0)) ( Z 0 , Z 1 ( Z 0 )) is the coupling obtained after the first rectification and, in turn, its rectification generates the second RF.
We will prescribe distributions for which 2-RF is provably not straight. We will detail the constructions for the case of d = 3 d=3 d = 3 ; as explained in § 4.2 this setting can be extended to arbitrary d ≥ 3 d\ge3 d ≥ 3 via a simple tensorization argument.
A criterion for non-straightness
Throughout this subsection, we assume that the endpoint map Z 1 : R 3 → R 3 \mathbf{Z}_1:\mathbb{R}^3 \to\mathbb{R}^3 Z 1 : R 3 → R 3 is C 1 C^1 C 1 and we write S + + k \mathbb{S}_{++}^k S ++ k for the set of real symmetric positive-definite k × k k \times k k × k matrices. Put
D ( x ) = Z 1 ( x ) − x , H t ( x ) = ( 1 − t ) ⋅ x + t ⋅ Z 1 ( x ) = x + t ⋅ D ( x ) . D(\boldsymbol{x})=\mathbf{Z}_1(\boldsymbol{x})-\boldsymbol{x}, \qquad\mathbf{H}_t(\boldsymbol{x})=(1-t)\cdot\boldsymbol{x}+t\cdot\mathbf{Z}_1(\boldsymbol{x})=\boldsymbol{x}+t\cdot D(\boldsymbol{x}). D ( x ) = Z 1 ( x ) − x , H t ( x ) = ( 1 − t ) ⋅ x + t ⋅ Z 1 ( x ) = x + t ⋅ D ( x ) .
For a square-integrable coupling ( X , Z 1 ( X ) ) (\boldsymbol{X},\mathbf{Z}_1(\boldsymbol{X})) ( X , Z 1 ( X )) , define the straightness defect
S t = S t ( X ) : = E ∥ D ( X ) − E [ D ( X ) ∣ H t ( X ) ] ∥ 2 . S_t=S_t(\boldsymbol{X}):=\mathbb{E}\left\|D(\boldsymbol{X})-\mathbb{E}\left[D(\boldsymbol{X})\mid\mathbf{H}_t(\boldsymbol{X})\right]\right\|^2. S t = S t ( X ) := E ∥ D ( X ) − E [ D ( X ) ∣ H t ( X ) ] ∥ 2 .
By Definition 1, the coupling is straight if and only if S t = 0 S_t=0 S t = 0 for almost every t ∈ ( 0 , 1 ) t\in(0,1) t ∈ ( 0 , 1 ) ; equivalently, D ( X ) D(\boldsymbol{X}) D ( X ) is measurable with respect to σ ( H t ( X ) ) \sigma(\mathbf{H}_t(\boldsymbol{X})) σ ( H t ( X )) for almost every such t t t . The goal of this subsection is to propose a sufficient condition for this coupling to be non-straight.
Since D ( X ) = t − 1 ( H t ( X ) − X ) D(\boldsymbol{X})=t^{-1}(\mathbf{H}_t(\boldsymbol{X})-\boldsymbol{X}) D ( X ) = t − 1 ( H t ( X ) − X ) , S t S_t S t can be written in terms of the chord map H t \mathbf{H}_t H t as
S t = t − 2 ⋅ E ∥ X − E [ X ∣ H t ( X ) ] ∥ 2 . (18) S_t=t^{-2}\cdot\mathbb{E}\left\|\boldsymbol{X}-\mathbb{E}\left[\boldsymbol{X}\mid\mathbf{H}_t(\boldsymbol{X})\right]\right\|^2. \tag*{(18)} S t = t − 2 ⋅ E ∥ X − E [ X ∣ H t ( X ) ] ∥ 2 . (18)
The following result provides the main technical tool for our construction of non-straight couplings. It shows that it suffices to formulate a data distribution for which the Jacobian of the endpoint map has two distinct negative eigenvalues at some point.
Proposition 4.1. Let Z 1 : R 3 → R 3 \mathbf{Z}_1:\mathbb{R}^3\to\mathbb{R}^3 Z 1 : R 3 → R 3 be C 1 C^1 C 1 such that
Z 1 ( x ) = B x + R ( x ) , B ∈ S + + 3 , sup x ∥ R ( x ) ∥ < ∞ . (19) \mathbf{Z}_1(\boldsymbol{x})=B\boldsymbol{x}+R(\boldsymbol{x}), \qquad B\in\mathbb{S}_{++}^3, \qquad\sup_{\boldsymbol{x}}\|R(\boldsymbol{x})\|<\infty. \tag*{(19)} Z 1 ( x ) = B x + R ( x ) , B ∈ S ++ 3 , x sup ∥ R ( x ) ∥ < ∞. (19) and D Z 1 D\mathbf{Z}_1 D Z 1 has real eigenvalues λ 1 < λ 2 < 0 < λ 3 \lambda_1<\lambda_2<0<\lambda_3 λ 1 < λ 2 < 0 < λ 3 at some point x ∗ \boldsymbol{x}_* x ∗ . Then, if X \boldsymbol{X} X has a strictly positive density and finite second moment,
on the interval ( ( 1 − λ 1 ) − 1 , ( 1 − λ 2 ) − 1 ) ((1-\lambda_1)^{-1},(1-\lambda_2)^{-1}) (( 1 − λ 1 ) − 1 , ( 1 − λ 2 ) − 1 ) , almost surely. As a result, the coupling ( X , Z 1 ( X ) ) (\boldsymbol{X},\mathbf{Z}_1(\boldsymbol{X})) ( X , Z 1 ( X )) is non-straight.
Proposition 4.1 provides a route to proving that a coupling is non-straight . To the best of our knowledge, the criterion introduced above is novel. The constructions below for the smooth, strongly log-concave data rely crucially on this result. We note that it is also relatively simple to generalize Proposition 4.1 to arbitrary dimension, under the assumption that the Jacobian matrix D Z 1 D\mathbf{Z}_1 D Z 1 has exactly two distinct negative eigenvalues (counting algebraic multiplicities) at some point.
Non-straight couplings under smooth & strongly log-concave data
We will now use Proposition 4.1 to show that the threshold e 4 π e^{4\pi} e 4 π in Theorem 3.2 is sharp. In fact, our results show that e 4 π e^{4\pi} e 4 π is the largest condition number possible to guarantee the straightness of 2-RF. Specifically, while Theorem 3.2 already shows that such a condition number must be no smaller than e 4 π e^{4\pi} e 4 π , Theorem 4.1 below shows that any condition number larger than this threshold will preclude straightness for one particular choice of the function V V V .
The proof of Theorem 4.1 entails constructing a limiting ODE where a suitably normalized version of the Jacobian map has exactly two negative eigenvalues at some point, for all large time. We then use Proposition 4.1 to finish the proof.
Theorem 4.1. For every ε > 0 \varepsilon> 0 ε > 0 and every d ≥ 3 d \ge3 d ≥ 3 , there exist V ∈ C ∞ ( R d ) V \in C^\infty(\mathbb{R}^d) V ∈ C ∞ ( R d ) and constants 0 < m < L < ∞ 0 < m < L < \infty 0 < m < L < ∞ such that
m I d ⪯ ∇ 2 V ( x ) ⪯ L I d , x ∈ R d , e 4 π < L m < e 4 π + ε , mI_d \preceq\nabla^2 V(\mathbf{x}) \preceq LI_d,\qquad\mathbf{x} \in\mathbb{R}^d,\qquad e^{4\pi} < \frac{L}{m} < e^{4\pi}+\varepsilon, m I d ⪯ ∇ 2 V ( x ) ⪯ L I d , x ∈ R d , e 4 π < m L < e 4 π + ε , and two-step RF applied to the independent coupling N ( 0 , I d ) ⊗ p 1 \mathcal{N}(\mathbf{0},I_d)\otimes p_1 N ( 0 , I d ) ⊗ p 1 yields a non-straight flow, where p 1 ( x ) ∝ exp ( − V ( x ) ) p_1(\mathbf{x}) \propto\exp(-V(\mathbf{x})) p 1 ( x ) ∝ exp ( − V ( x )) .
As noted above, we only need to consider the case d = 3 d=3 d = 3 . Indeed, for the case d ≥ 4 d \ge4 d ≥ 4 , let V 3 V_3 V 3 be the potential function in Theorem 4.1. Then, we can simply choose the new potential
V d ( x , y ) = V 3 ( x ) + m 2 ⋅ ∥ y ∥ 2 , x ∈ R 3 , y ∈ R d − 3 . V_d(\mathbf{x},\mathbf{y})=V_3(\mathbf{x})+\frac{m}{2}\cdot\|\mathbf{y}\|^2,\qquad\mathbf{x}\in\mathbb{R}^3,\ \mathbf{y}\in\mathbb{R}^{d-3}. V d ( x , y ) = V 3 ( x ) + 2 m ⋅ ∥ y ∥ 2 , x ∈ R 3 , y ∈ R d − 3 .
The target distribution is thus
exp ( − V 3 ) ⊗ N ( 0 , m − 1 ⋅ I d − 3 ) . \exp(-V_3)\otimes\mathcal{N}(\mathbf{0},m^{-1}\cdot I_{d-3}). exp ( − V 3 ) ⊗ N ( 0 , m − 1 ⋅ I d − 3 ) .
The distribution above has the same condition number as V 3 V_3 V 3 , and is non-straight because its endpoint map has the form
Z 1 V d ( x , y ) = ( Z 1 V 3 ( x ) , m − 1 / 2 ⋅ y ) , x ∈ R 3 , y ∈ R d − 3 . \mathbf{Z}_1^{V_d}(\mathbf{x},\mathbf{y})=\left(\mathbf{Z}_1^{V_3}(\mathbf{x}),m^{-1/2}\cdot\mathbf{y}\right),\qquad\mathbf{x}\in\mathbb{R}^3,\ \mathbf{y}\in\mathbb{R}^{d-3}. Z 1 V d ( x , y ) = ( Z 1 V 3 ( x ) , m − 1/2 ⋅ y ) , x ∈ R 3 , y ∈ R d − 3 .
where Z 1 V 3 \mathbf{Z}_1^{V_3} Z 1 V 3 is the endpoint map with respect to V 3 V_3 V 3 . Clearly, the straightness defect of V d V_d V d is the same as V 3 V_3 V 3 , which is positive and yields non-straightness.
The construction has two parts. First, we construct a smooth, strongly convex potential W W W and an associated auxiliary flow whose endpoint map has a Jacobian with two distinct negative eigenvalues. Next, we show that the RF endpoint map associated with the law p 1 β ∝ exp ( − β W ) p_1^\beta\propto\exp(-\beta W) p 1 β ∝ exp ( − β W ) also has the same eigenvalue property when β \beta β is sufficiently large (but finite). A crucial ingredient of the proof is a gluing argument that is used to guarantee that the condition number stays close to e 4 π e^{4\pi} e 4 π . The detailed proofs are presented in Appendix § 13 .
Non-straight couplings under Gaussian mixture models
For Gaussian mixtures, the relevant quantity that determines straightness is the ratio between the maximum interpoint distance between the component means and their common standard deviation. The next result shows that the bound in Theorem 3.3 , which guarantees straightness of 2-RF whenever the maximal separation among the means D max 2 D_{\max}^2 D m a x 2 is smaller than 16 π σ 2 16\pi\sigma^2 16 π σ 2 , is sharp. In detail, Theorem 4.2 below proves that, for any value of D max 2 D_{\max}^2 D m a x 2 larger than this threshold, there exists an isotropic GMM with variance parameter σ 2 \sigma^2 σ 2 and at least 4 components for which 2-RF provably fails to be straight.
In particular, our result improves and refines the bound in Bansal et al. [ [2 ] , who, in the same settings, show that 2-RF is straight as long as D max 2 / σ 2 ≤ 4 D_{\max}^2/\sigma^2 \le4 D m a x 2 / σ 2 ≤ 4 .
Theorem 4.2. Fix d ≥ 3 d \ge3 d ≥ 3 and σ > 0 \sigma> 0 σ > 0 . For any ε > 0 \varepsilon> 0 ε > 0 , there exist finitely many means μ 1 , … , μ K ∈ R d \boldsymbol{\mu}_1,\ldots,\boldsymbol{\mu}_K\in\mathbb{R}^d μ 1 , … , μ K ∈ R d , K ≥ 4 K\ge4 K ≥ 4 and positive weights π 1 , … , π K \pi_1,\ldots,\pi_K π 1 , … , π K summing to one such that the mixture
p 1 = ∑ j = 1 K π j N ( μ j , σ 2 I d ) p_1=\sum_{j=1}^{K}\pi_j\mathcal{N}(\boldsymbol{\mu}_j,\sigma^2I_d) p 1 = j = 1 ∑ K π j N ( μ j , σ 2 I d ) satisfies
16 π < D max 2 σ 2 < 16 π + ε , D max : = max i ≠ j ∥ μ i − μ j ∥ 2 , 16\pi< \frac{D_{\max}^2}{\sigma^2} < 16\pi+\varepsilon,\qquad D_{\max}:=\max_{i\ne j}\|\boldsymbol{\mu}_i-\boldsymbol{\mu}_j\|_2, 16 π < σ 2 D m a x 2 < 16 π + ε , D m a x := i = j max ∥ μ i − μ j ∥ 2 , while the 2-RF initialized from the independent coupling N ( 0 , I d ) ⊗ p 1 \mathcal{N}(\mathbf{0},I_d)\otimes p_1 N ( 0 , I d ) ⊗ p 1 produces a non-straight flow.
The strategy of the proof is generally similar to that of Theorem 4.1, in that we construct a limiting ODE whose endpoint map’s Jacobian has exactly two negative eigenvalues at some point. The number of components K K K chosen in the proof of Theorem 4.2 is huge and we have not attempted to make it explicit for clarity of the proof. The construction underlying Theorem 4.2 is illustrated in Figure S2 . We arrange the Gaussian component means in antipodal pairs on a cylinder around the z z z -axis, with their horizontal directions rotating as the height increases (Figure S2 ). The mixture weights are chosen so that, along a distinguished RF trajectory on the z z z -axis, the posterior mass progressively concentrates on successive pairs of components. This particular construction, coupled with the accumulated rotational effect of the components along the RF trajectory, produces an endpoint Jacobian with two distinct negative eigenvalues, corresponding to a net half-turn in the transverse directions. By Proposition 4.1, this implies non-straightness of the coupling obtained after the first rectification. The proof heuristics and details are deferred to Appendices § 14 .§ 1 and § 14 .§ 2 .
Figure S2. (a) Rotating antipodal component means. (b) Rotating posterior covariance along the distinguished axial trajectory. Figure S2: Geometric construction of the GMM. The mixture means are antipodal pairs with rotating horizontal directions. Along a chosen RF trajectory on the z z z -axis, the posterior progressively concentrates on successive pairs, inducing a rotating covariance in the transverse plane.
Approximate straightness and choosing the number of rectification steps
In the previous sections, Theorem 3.1 provided sufficient conditions for the endpoint coupling generated by 1-RF to be straight, and hence for 2-RF to produce a straight flow, when starting from an independent coupling. However, exact straightness is restrictive in practice as one often only has access to the data, and it is hard to verify the structural assumptions required for exact straightness just from the observed data.
From a practical perspective, each additional rectification requires training a new velocity field, which can be computationally expensive. Consequently, although additional rectification typically improves the straightness of the learned flow ([see 36 ] for more details] ), continuing the procedure indefinitely is neither necessary nor computationally desirable. Empirically, Lee et al. [ [32 ] demonstrate that a single reflow iteration can produce nearly straight trajectories, enabling accurate few-step generation. Related work by Wang et al. [ [57 ] has also explored efficient generation without requiring exactly straight trajectories. Motivated by these recent findings, we argue that a sufficiently straight flow is often adequate for achieving near-optimal sampling performance. This naturally raises the following practical question.
Given a trained K K K th rectified flow, can we determine from finite data whether the learned flow is sufficiently straight, or whether another rectification step is statistically warranted?
To give an answer to this question, we first define the concept of approximate straightness . Let ( Z 0 , Z 1 ( Z 0 ) ) (\mathbf{Z}_0,\mathbf{Z}_1(\mathbf{Z}_0)) ( Z 0 , Z 1 ( Z 0 )) be a deterministic population coupling between two densities p 0 p_0 p 0 and p 1 p_1 p 1 , with finite second moments . Recall that H t ( x ) = ( 1 − t ) x + t Z 1 ( x ) \mathbf{H}_t(x)=(1-t)x+t\mathbf{Z}_1(x) H t ( x ) = ( 1 − t ) x + t Z 1 ( x ) .
Definition 2 (Approximate straightness). For δ ≥ 0 \delta\ge0 δ ≥ 0 , we say that the coupling ( Z 0 , Z 1 ( Z 0 ) ) (\mathbf{Z}_0,\mathbf{Z}_1(\mathbf{Z}_0)) ( Z 0 , Z 1 ( Z 0 )) is δ \delta δ -approximately straight if
S ( Z ) : = E Z 0 , t [ ∥ Z 1 ( Z 0 ) − Z 0 − E [ Z 1 ( Z 0 ) − Z 0 ∣ H t ( Z 0 ) ] ∥ 2 2 ] ≤ δ , S(\mathbf{Z}) := \mathbb{E}_{\mathbf{Z}_0,t}\left[\left\|\mathbf{Z}_1(\mathbf{Z}_0)-\mathbf{Z}_0-\mathbb{E}\left[\mathbf{Z}_1(\mathbf{Z}_0)-\mathbf{Z}_0\mid\mathbf{H}_t(\mathbf{Z}_0)\right]\right\|_2^2\right] \le\delta, S ( Z ) := E Z 0 , t [ ∥ Z 1 ( Z 0 ) − Z 0 − E [ Z 1 ( Z 0 ) − Z 0 ∣ H t ( Z 0 ) ] ∥ 2 2 ] ≤ δ ,
almost surely with respect to independent draws of Z 0 ∼ p 0 \mathbf{Z}_0\sim p_0 Z 0 ∼ p 0 and t ∼ Unif [ 0 , 1 ] t\sim\operatorname{Unif}[0,1] t ∼ Unif [ 0 , 1 ] .
Clearly, when δ = 0 \delta=0 δ = 0 , this reduces to a straight coupling in the sense of Definition 1. For the population endpoint coupling obtained after K K K rectifications, we denote this defect by S ( K ) ( Z ) S^{(K)}(\mathbf{Z}) S ( K ) ( Z ) .
Liu et al. [ [38 ] showed that successive rectifications lead to a vanishing straightness defect, i.e., lim K → ∞ S ( K ) ( Z ) = 0 \lim_{K\to\infty} S^{(K)}(\mathcal{Z}) = 0 lim K → ∞ S ( K ) ( Z ) = 0 . To be precise, Theorem D.7 in Liu et al. [ [38 ] states that
∑ k = 0 K S ( k ) ( Z ) ≤ E ∥ X 1 − X 0 ∥ 2 2 < ∞ for all K , \sum_{k=0}^{K} S^{(k)}(\mathcal{Z}) \le\mathbb{E}\left\|X_1-X_0\right\|_2^2 < \infty\quad\text{for all } K, k = 0 ∑ K S ( k ) ( Z ) ≤ E ∥ X 1 − X 0 ∥ 2 2 < ∞ for all K ,
and hence, the limiting straightness claim mentioned above follows. In this section we consider this setting, with the goal of minimizing the number of reflow steps before obtaining a solution that, though not straight, has low enough defect to lead to practicable sampling methodologies. To this end, the concept of approximate straightness in Definition 2 is useful since one can construct a consistent estimator for it, given the learned vector field in a given rectification round, and construct a statistically valid stopping rule. In the case where the data belong to the class of smooth, strongly log-concave distributions, one can see a clear connection between the straightness result in Theorem 3.2 and the approximate straightness concept in Definition 2.
Proposition 5.1 (Approximate straightness under strong log-concavity). Suppose that p 1 p_1 p 1 satisfies (3.6). Let Z \mathcal{Z} Z be the population rectified flow starting from N ( 0 , I d ) ⊗ p 1 \mathcal{N}(0,I_d)\otimes p_1 N ( 0 , I d ) ⊗ p 1 , with endpoint coupling ( Z 0 , Z 1 ( Z 0 ) ) (Z_0,Z_1(Z_0)) ( Z 0 , Z 1 ( Z 0 )) . Then
S ( Z ) ≤ 2 d ( m L ) 1 / 4 ⋅ sinh ( [ log 2 ( L / m ) 16 − π 2 ] + ) (20) S(\mathcal{Z}) \le\frac{2d}{(mL)^{1/4}}\cdot\sinh\left(\sqrt{\left[\frac{\log^2(L/m)}{16}-\pi^2\right]_+}\right) \tag*{(20)} S ( Z ) ≤ ( m L ) 1/4 2 d ⋅ sinh ( [ 16 log 2 ( L / m ) − π 2 ] + ) (20) where [ a ] + : = max { a , 0 } [a]_+ := \max\{a,0\} [ a ] + := max { a , 0 } .
The main consequence of the last result is that, under the stated assumptions, the coupling is δ m , L \delta_{m,L} δ m , L -approximately straight, where δ m , L \delta_{m,L} δ m , L denotes the right-hand side of (5.1). It is immediate that when log ( L / m ) ≤ 4 π \log(L/m) \le4\pi log ( L / m ) ≤ 4 π , Proposition 5.1 yields the straightness of the coupling obtained from 2-RF. Interestingly, the bound in (5.1) is of order O ( d / m ) O(d/\sqrt{m}) O ( d / m ) for all large L , m L,m L , m . In fact, we can check that
2 d ( m L ) 1 / 4 ⋅ sinh ( [ log 2 ( L / m ) 16 − π 2 ] + ) ≤ d m \frac{2d}{(mL)^{1/4}}\cdot\sinh\left(\sqrt{\left[\frac{\log^2(L/m)}{16}-\pi^2\right]_+}\right)\le\frac{d}{\sqrt{m}} ( m L ) 1/4 2 d ⋅ sinh ( [ 16 log 2 ( L / m ) − π 2 ] + ) ≤ m d
when the ratio L / m L/m L / m is large. This implies that even when exact straightness fails, approximate straightness can still hold, as long as the data distribution is generated by a potential function with large strong-convexity parameter. The detailed proof is deferred to Appendix § 15 .
An important consideration in designing such a certification procedure is that its computational overhead should be negligible compared with the cost of training an additional rectified flow. Since the objective of the stopping rule is precisely to decide whether another expensive rectification step is necessary, the certification procedure itself should avoid solving additional optimization problems. Motivated by this principle, we introduce a trajectory-based straightness functional that can be evaluated directly from independently generated trajectories of the learned flow. Once the velocity field has been trained, estimating this functional requires only forward simulation of the learned ordinary differential equation together with evaluations of the learned velocity field along the generated trajectories. Consequently, the proposed certification procedure incurs only a modest computational overhead relative to the cost of training the rectified flow itself.
A trajectory-based straightness functional
Suppose that the K K K th rectified flow has been trained, yielding the learned velocity field v ^ ( K ) : R d × [ 0 , 1 ] → R d \hat{v}^{(K)}:\mathbb{R}^d\times[0,1]\to\mathbb{R}^d v ^ ( K ) : R d × [ 0 , 1 ] → R d . Throughout this section, we condition on the learned velocity field v ^ ( K ) \hat{v}^{(K)} v ^ ( K ) during the K K K th rectification stage, and regard it as fixed. All probabilities and expectations are therefore taken with respect to fresh trajectories generated from the learned rectified flow. The inferential target is thus the straightness of the learned rectified flow rather than that generated by the unknown population-optimal velocity field.
Figure 2 (b) summarizes the proposed certification procedure. Starting from a trained K K K th rectified flow, we generate fresh trajectories from the learned dynamics, evaluate the proposed trajectory-based straightness functional, construct a confidence interval for the resulting straightness measure, and determine whether another rectification step is statistically justified.
Figure 2. (a) The trajectory-based straightness functional compares the endpoint displacement with the instantaneous velocity evaluated at a uniformly sampled time along the learned trajectory. (b) After training the K K K th rectified flow, fresh trajectories are generated to estimate the straightness functional, construct a confidence bound, and determine whether another rectification step is statistically necessary.
Starting from an initial condition
Z 0 ( K ) ∼ N ( 0 , I d ) , Z_0^{(K)} \sim\mathcal{N}(0,I_d), Z 0 ( K ) ∼ N ( 0 , I d ) ,
let { Z t i ( K ) : 0 ≤ i ≤ N } \{Z_{t_i}^{(K)}: 0 \le i \le N\} { Z t i ( K ) : 0 ≤ i ≤ N } denote the solution to the following Euler discretized ODE:
0 = t 0 < t 1 < ⋯ < t N = 1 , Z t i + 1 ( K ) = Z t i ( K ) + ( t i + 1 − t i ) v ^ ( K ) ( Z t i ( K ) , t i ) , Z 0 ( K ) ∼ N ( 0 , I d ) , i ∈ { 0 , … , N − 1 } . (21) \begin{aligned}
0 &= t_0 < t_1 < \cdots< t_N = 1,\\
Z_{t_{i+1}}^{(K)} &= Z_{t_i}^{(K)} + (t_{i+1}-t_i)\widehat{v}^{(K)}\left(Z_{t_i}^{(K)},t_i\right), \qquad Z_0^{(K)} \sim\mathcal{N}(0,I_d), \quad i \in\{0,\ldots,N-1\}. \tag*{(21)}
\end{aligned} 0 Z t i + 1 ( K ) = t 0 < t 1 < ⋯ < t N = 1 , = Z t i ( K ) + ( t i + 1 − t i ) v ( K ) ( Z t i ( K ) , t i ) , Z 0 ( K ) ∼ N ( 0 , I d ) , i ∈ { 0 , … , N − 1 } . (21)
Motivated by the geometric characterization of straight flows (cf. Definition 2), we quantify the remaining deviation from straightness by comparing the endpoint displacement with the learned velocity evaluated at a uniformly sampled time along the trajectory. Specifically, let t ∼ Unif [ 0 , 1 ] t \sim\operatorname{Unif}[0,1] t ∼ Unif [ 0 , 1 ] , independent of Z 0 ( K ) Z_0^{(K)} Z 0 ( K ) , and define
T ( K ) : = E [ ∥ Z 1 ( K ) − Z 0 ( K ) − v ^ ( K ) ( Z t ( K ) , t ) ∥ 2 2 ] . (22) \mathcal{T}^{(K)} := \mathbb{E}\left[\left\|Z_1^{(K)} - Z_0^{(K)} - \widehat{v}^{(K)}\left(Z_t^{(K)},t\right)\right\|_2^2\right]. \tag*{(22)} T ( K ) := E [ Z 1 ( K ) − Z 0 ( K ) − v ( K ) ( Z t ( K ) , t ) 2 2 ] . (22)
The functional T ( K ) \mathcal{T}^{(K)} T ( K ) admits a natural geometric interpretation and has also been studied in Liu et al. [ [38 ] and Liu [ [36 ] before. It measures the variability of the velocity field along trajectories generated by the learned rectified flow. A perfectly straight flow has constant velocity along every trajectory, in which case these two quantities coincide almost surely and T ( K ) = 0 \mathcal{T}^{(K)}=0 T ( K ) = 0 . Larger values of T ( K ) \mathcal{T}^{(K)} T ( K ) indicate greater deviation from straightness.
An important advantage of the proposed functional is its computational simplicity. Once the learned velocity field has been obtained, evaluating T ( K ) \mathcal{T}^{(K)} T ( K ) requires only generating independent trajectories from the learned dynamics and evaluating the trained velocity field along these trajectories. No additional optimization or retraining is required, making the resulting certification procedure computationally lightweight relative to the training of the rectified flow itself.
Coupling straightness vs. trajectory straightness.
We now clarify the relationship between the coupling-based straightness defect S ( K ) ( Z ) S^{(K)}(\mathcal{Z}) S ( K ) ( Z ) (Definition 2) and the trajectory-based functional T ( K ) \mathcal{T}^{(K)} T ( K ) . For simplicity, we first consider the exact population velocity field v ( K ) v^{(K)} v ( K ) (cf. Figure 1), rather than its learned approximation v ^ ( K ) \hat{v}^{(K)} v ^ ( K ) . Accordingly, throughout this discussion, T ( K ) \mathcal{T}^{(K)} T ( K ) denotes the trajectory-based functional defined in (22) , with v ^ ( K ) \hat{v}^{(K)} v ^ ( K ) replaced by v ( K ) v^{(K)} v ( K ) .
The functional S ( K ) ( Z ) S^{(K)}(\mathcal{Z}) S ( K ) ( Z ) measures the deviation from straightness of the endpoint coupling ( Z 0 ( K ) , Z 1 ( K ) ) (Z_0^{(K)},Z_1^{(K)}) ( Z 0 ( K ) , Z 1 ( K ) ) , using the linear interpolation
H t ( Z 0 ( K ) , Z 1 ( K ) ) = ( 1 − t ) Z 0 ( K ) + t Z 1 ( K ) . \mathbf{H}_t\left(Z_0^{(K)},Z_1^{(K)}\right)=(1-t)Z_0^{(K)}+tZ_1^{(K)}. H t ( Z 0 ( K ) , Z 1 ( K ) ) = ( 1 − t ) Z 0 ( K ) + t Z 1 ( K ) .
In contrast, T ( K ) \mathcal{T}^{(K)} T ( K ) measures the variability of the instantaneous velocity v t ( K ) ( Z t ( K ) ) v_t^{(K)}\left(Z_t^{(K)}\right) v t ( K ) ( Z t ( K ) ) along the actual ODE trajectories. Although these functionals measure different aspects of straightness, they are quantitatively related under mild regularity conditions. In particular, the following result shows that small trajectory straightness defect guarantees small coupling straightness defect.
Theorem 5.1 (Trajectory straightness controls coupling straightness). Let { Z t ( K ) } t ∈ [ 0 , 1 ] \{Z_t^{(K)}\}_{t\in[0,1]} { Z t ( K ) } t ∈ [ 0 , 1 ] be an absolutely continuous solution of the RF ODE
d d t Z t ( K ) = v t ( K ) ( Z t ( K ) ) . \frac{\mathrm{d}}{\mathrm{d}t}Z_t^{(K)}=v_t^{(K)}\left(Z_t^{(K)}\right). d t d Z t ( K ) = v t ( K ) ( Z t ( K ) ) . Suppose that T ( K ) < ∞ \mathcal{T}^{(K)}<\infty T ( K ) < ∞ and that the velocity field v t ( K ) v_t^{(K)} v t ( K ) is uniformly ℓ K \ell_K ℓ K -Lipschitz in space. Then
S ( K ) ( Z ) ≤ ( 1 + ℓ K π ) 2 T ( K ) . (23) S^{(K)}(\mathcal{Z})\leq\left(1+\frac{\ell_K}{\pi}\right)^2\mathcal{T}^{(K)}. \tag*{(23)} S ( K ) ( Z ) ≤ ( 1 + π ℓ K ) 2 T ( K ) . (23) In particular, T ( K ) = 0 \mathcal{T}^{(K)}=0 T ( K ) = 0 implies S ( K ) ( Z ) = 0 S^{(K)}(\mathcal{Z})=0 S ( K ) ( Z ) = 0 .
Theorem 5.1 provides a theoretical justification for using T ( K ) \mathcal{T}^{(K)} T ( K ) as a surrogate for coupling straightness. In particular, controlling the trajectory-based defect also controls the coupling-based defect, up to a multiplicative factor determined by the spatial regularity of the velocity field. This conforms with the results in Lemma 2.1 . More importantly, the same comparison applies to a learned velocity field v ^ ( K ) \hat{v}^{(K)} v ^ ( K ) , provided that the corresponding exact learned ODE trajectories satisfy the assumptions of the theorem. The proof is deferred to Appendix § 15 .§ 2 .
Finally, we also point out that the Lipschitz condition, although strong, is a standard assumption in the existing RF literature [2 , 21 , 28 , 49 ] . In our settings, we start from a standard Gaussian distribution and the target distributions are isotropic GMMs or log-concave distributions absolutely continuous with respect to Lebesgue measure. Then, the Lipschitz condition holds in both cases. However, in general Mena et al. [ [42 ] showed that the uniform Lipschitz conditions may fail for irregular distributions.
Estimation and asymptotic inference
The characterization above suggests a simple Monte Carlo estimator of T ( K ) \mathcal{T}^{(K)} T ( K ) . After training the K K K th rectified flow, we generate N N N independent trajectories using the discretized sampler (21) starting from N N N samples { Z 0 , i ( K ) } i = 1 N ∼ i . i . d . N ( 0 , I d ) \{Z_{0,i}^{(K)}\}_{i=1}^{N} \overset{\mathrm{i.i.d.}}{\sim} \mathcal{N}(0,I_d) { Z 0 , i ( K ) } i = 1 N ∼ i.i.d. N ( 0 , I d ) . For each trajectory, an independent time t i ∼ Unif [ 0 , 1 ] t_i \sim\operatorname{Unif}[0,1] t i ∼ Unif [ 0 , 1 ] is sampled (cf. Figure 3 ), and the corresponding trajectory state Z t i , i ( K ) Z_{t_i,i}^{(K)} Z t i , i ( K ) is obtained by evaluating the numerical ODE solution at time t i t_i t i . The resulting trajectory residual is
Figure 3. Construction of the trajectory functionals.
ψ i ( K ) = ∥ Z 1 , i ( K ) − Z 0 , i ( K ) − v ^ ( K ) ( Z t i , i ( K ) , t i ) ∥ 2 2 . (24) \psi_i^{(K)} = \left\|Z_{1,i}^{(K)} - Z_{0,i}^{(K)} - \widehat{v}^{(K)}\left(Z_{t_i,i}^{(K)},t_i\right)\right\|_2^2. \tag*{(24)} ψ i ( K ) = Z 1 , i ( K ) − Z 0 , i ( K ) − v ( K ) ( Z t i , i ( K ) , t i ) 2 2 . (24)
Since the learned velocity field is regarded as fixed, the only randomness arises from the independently generated trajectories and the sampled times. Consequently, conditional on v ^ ( K ) \widehat{v}^{(K)} v ( K ) , the residuals { ψ i ( K ) } i = 1 N \{\psi_i^{(K)}\}_{i=1}^{N} { ψ i ( K ) } i = 1 N are independent and identically distributed with mean T ( K ) \mathcal{T}^{(K)} T ( K ) .
The trajectory-based straightness functional can therefore be estimated in a natural way by the sample average
T ^ N ( K ) = 1 N ∑ i = 1 N ψ i ( K ) . (25) \widehat{\mathcal{T}}_N^{(K)} = \frac{1}{N}\sum_{i=1}^{N}\psi_i^{(K)}. \tag*{(25)} T N ( K ) = N 1 i = 1 ∑ N ψ i ( K ) . (25)
We first establish consistency and asymptotic normality of the proposed estimator, which is a direct consequence of the weak law of large numbers and classical central limit theory.
Theorem 5.2 (Consistency and asymptotic normality). Let F K − 1 \mathcal{F}_{K-1} F K − 1 be the sigma-field generated by the samples obtained via ( K − 1 ) (K-1) ( K − 1 ) th rectified flow. Then, conditioned on F K − 1 \mathcal{F}_{K-1} F K − 1 , we have
T ^ N ( K ) ⟶ p T ( K ) . \widehat{\mathcal{T}}_N^{(K)} \overset{p}{\longrightarrow} \mathcal{T}^{(K)}. T N ( K ) ⟶ p T ( K ) .
Additionally, if σ K 2 : = Var ( ψ ( K ) ∣ F K − 1 ) ∈ ( 0 , ∞ ) \sigma_K^2 := \operatorname{Var}(\psi^{(K)} \mid\mathcal{F}_{K-1}) \in(0,\infty) σ K 2 := Var ( ψ ( K ) ∣ F K − 1 ) ∈ ( 0 , ∞ ) almost surely, then conditional on F K − 1 \mathcal{F}_{K-1} F K − 1 ,
N ( T ^ N ( K ) − T ( K ) ) σ K → d N ( 0 , 1 ) almost surely . \frac{\sqrt{N}\left(\widehat{\mathcal{T}}_N^{(K)}-\mathcal{T}^{(K)}\right)}{\sigma_K} \xrightarrow{d} \mathcal{N}(0,1) \quad\text{almost surely}. σ K N ( T N ( K ) − T ( K ) ) d N ( 0 , 1 ) almost surely .
The asymptotic normality property above eventually leads to the uncertainty quantification of the straightness functional. Towards that end, we begin with the natural plug-in estimator of the asymptotic variance.
σ ^ N , K 2 = 1 N − 1 ∑ i = 1 N ( ψ i ( K ) − T ^ N ( K ) ) 2 . \widehat{\sigma}_{N,K}^{2}=\frac{1}{N-1}\sum_{i=1}^{N}\left(\psi_i^{(K)}-\widehat{\mathcal{T}}_N^{(K)}\right)^2. σ N , K 2 = N − 1 1 i = 1 ∑ N ( ψ i ( K ) − T N ( K ) ) 2 .
The next result establishes its consistency.
Proposition 5.2 (Variance estimation). Let the assumptions of Theorem 5.2 hold. Then, conditional on F K − 1 \mathcal{F}_{K-1} F K − 1 ,
σ ^ N , K 2 → p σ K 2 . \widehat{\sigma}_{N,K}^{2}\xrightarrow{p}\sigma_K^2. σ N , K 2 p σ K 2 .
Consequently,
T ^ N ( K ) − T ( K ) σ ^ N , K / N → d N ( 0 , 1 ) . \frac{\widehat{\mathcal{T}}_N^{(K)}-\mathcal{T}^{(K)}}{\widehat{\sigma}_{N,K}/\sqrt{N}} \xrightarrow{d} \mathcal{N}(0,1). σ N , K / N T N ( K ) − T ( K ) d N ( 0 , 1 ) .
Let z 1 − α z_{1-\alpha} z 1 − α denote the ( 1 − α ) (1-\alpha) ( 1 − α ) th quantile of the standard Gaussian distribution. Based on the above result, the one-sided upper confidence bound
U N , K = T ^ N ( K ) + z 1 − α σ ^ N , K N U_{N,K}=\widehat{\mathcal{T}}_N^{(K)}+z_{1-\alpha}\frac{\widehat{\sigma}_{N,K}}{\sqrt{N}} U N , K = T N ( K ) + z 1 − α N σ N , K
will be used in the next subsection to construct a statistically principled stopping rule for determining the required number of rectification steps.
A statistically principled stopping rule
The confidence bound developed in the previous subsection naturally leads to a statistically principled criterion for determining whether an additional rectification step is warranted. Rather than seeking perfect straightness, we instead aim to identify the smallest number of rectification steps for which the learned flow is sufficiently straight.
Let δ > 0 \delta> 0 δ > 0 denote a user-specified tolerance representing the maximum acceptable deviation from trajectory straightness. At the K K K th rectification step, we consider the one-sided hypothesis
H 0 : T ( K ) > δ , versus H 1 : T ( K ) ≤ δ . H_0 : \mathcal{T}^{(K)} > \delta, \qquad\text{versus} \qquad H_1 : \mathcal{T}^{(K)} \le\delta. H 0 : T ( K ) > δ , versus H 1 : T ( K ) ≤ δ .
Since T ( K ) \mathcal{T}^{(K)} T ( K ) is unknown, we base the decision on the upper one-sided confidence bound
U N , K : = T ^ N ( K ) + z 1 − α σ ^ N , K N . U_{N,K}:=\widehat{\mathcal{T}}_N^{(K)}+z_{1-\alpha}\frac{\widehat{\sigma}_{N,K}}{\sqrt{N}}. U N , K := T N ( K ) + z 1 − α N σ N , K .
The null hypothesis is rejected whenever U N , K ≤ δ U_{N,K}\le\delta U N , K ≤ δ , indicating that the learned rectified flow is sufficiently straight with asymptotic confidence level 1 − α 1-\alpha 1 − α .
Proposition 5.3 (Validity of the stopping criterion). Under the assumptions of Theorem 5.2, the upper one-sided confidence bound
U N , K = T ^ N ( K ) + z 1 − α σ ^ N , K N U_{N,K} = \widehat{\mathcal{T}}_{N}^{(K)} + z_{1-\alpha}\frac{\widehat{\sigma}_{N,K}}{\sqrt{N}} U N , K = T N ( K ) + z 1 − α N σ N , K satisfies the coverage probability
lim N → ∞ P ( T ( K ) ≤ U N , K ) = 1 − α . \lim_{N \to\infty} \mathbb{P}\left(\mathcal{T}^{(K)} \le U_{N,K}\right) = 1-\alpha. N → ∞ lim P ( T ( K ) ≤ U N , K ) = 1 − α . Consequently, the stopping rule
U N , K ≤ δ U_{N,K} \le\delta U N , K ≤ δ ensures the following type-I error control:
lim N → ∞ P H 0 ( U N , K ≤ δ ) ≤ α , \lim_{N \to\infty} \mathbb{P}_{H_0}\left(U_{N,K} \le\delta\right) \le\alpha, N → ∞ lim P H 0 ( U N , K ≤ δ ) ≤ α , i.e., the proposed stopping rule falsely certifies a learned flow as sufficiently straight with asymptotic probability at most α \alpha α .
The proposed rule terminates the rectification procedure at the first iteration for which the remaining trajectory straightness is certified to be no larger than the prescribed tolerance. Consequently, the stopping decision explicitly accounts for the sampling variability of the estimated straightness functional T ( K ) \mathcal{T}^{(K)} T ( K ) , while avoiding unnecessary rectification steps. The steps are detailed in Algorithm 1 below.
Remark 5.1. Algorithm 1 repeatedly evaluates the same hypothesis (that the remaining straightness is below δ \delta δ ) across iterations. Using a fixed level α \alpha α in every iteration generally does not control the overall type-I error at α \alpha α .
If one enforces a maximum reflow budget K max K_{\max} K m a x , then family-wise type-I error can be controlled via an alpha-spending schedule: choose (possibly adaptive, data-dependent) levels { α k } k = 1 K max \{\alpha_k\}_{k=1}^{K_{\max}} { α k } k = 1 K m a x such that ∑ k = 1 K max α k ≤ α \sum_{k=1}^{K_{\max}}\alpha_k \le\alpha ∑ k = 1 K m a x α k ≤ α , and use z 1 − α k z_{1-\alpha_k} z 1 − α k at iteration k k k . A simple choice is Bonferroni (α k = α / K max \alpha_k = \alpha/K_{\max} α k = α / K m a x ), while geometric schedules (e.g., α k = α / 2 k \alpha_k = \alpha/2^k α k = α / 2 k ) spend more alpha early.
Numerical experiments
2D Gaussian mixtures experiment
We first illustrate the effect of iterative rectification on a simple two-dimensional example, where the geometry of the learned trajectories can be directly visualized. We consider the source distribution p 0 = N ( 0 , I 2 ) p_0 = \mathcal{N}(0, I_2) p 0 = N ( 0 , I 2 ) and the symmetric two-component Gaussian mixture
p 1 = 1 2 N ( μ , σ 2 I 2 ) + 1 2 N ( − μ , σ 2 I 2 ) , μ = ( 3 , 0 ) ⊤ , σ = 0.5. p_1 = \frac{1}{2}\mathcal{N}(\mu,\sigma^2 I_2) + \frac{1}{2}\mathcal{N}(-\mu,\sigma^2 I_2), \qquad\mu= (3,0)^\top, \qquad\sigma= 0.5. p 1 = 2 1 N ( μ , σ 2 I 2 ) + 2 1 N ( − μ , σ 2 I 2 ) , μ = ( 3 , 0 ) ⊤ , σ = 0.5.
The initial rectified flow (1-RF) is trained using independent samples X 0 ∼ p 0 \mathbf{X}_0 \sim p_0 X 0 ∼ p 0 and X 1 ∼ p 1 \mathbf{X}_1 \sim p_1 X 1 ∼ p 1 , with the usual linear interpolation
X t = ( 1 − t ) X 0 + t X 1 . \mathbf{X}_t = (1-t)\mathbf{X}_0 + t\mathbf{X}_1. X t = ( 1 − t ) X 0 + t X 1 .
We then perform two additional rounds of rectification. In each round, we generate endpoint pairs by solving the ODE associated with the previously learned velocity field and train the next velocity field using the resulting coupling.
Algorithm 1 Adaptive selection of the number of rectification steps
Choose confidence level 1 − α 1-\alpha 1 − α and tolerance δ \delta δ .
Initialize K = 1 K=1 K = 1 .
repeat
Train the K K K th rectified flow and obtain v ^ ( K ) \widehat{v}^{(K)} v ( K ) .
Generate independent trajectories from the learned ODE.
Compute the trajectory residuals { ψ i ( K ) } i = 1 N \{\psi_i^{(K)}\}_{i=1}^{N} { ψ i ( K ) } i = 1 N using (24) .
Compute T ^ N ( K ) \widehat{T}_N^{(K)} T N ( K ) and σ ^ N , K \widehat{\sigma}_{N,K} σ N , K .
Form
U N , K = T ^ N ( K ) + z 1 − α σ ^ N , K N . U_{N,K}=\widehat{T}_N^{(K)}+z_{1-\alpha}\frac{\widehat{\sigma}_{N,K}}{\sqrt{N}}. U N , K = T N ( K ) + z 1 − α N σ N , K .
if U N , K > δ U_{N,K}>\delta U N , K > δ then
Perform one additional rectification.
Set K ← K + 1 K\leftarrow K+1 K ← K + 1 .
end if
until U N , K ≤ δ U_{N,K}\leq\delta U N , K ≤ δ
return K ^ = K \widehat{K}=K K = K .
Trajectory straightening Figure 4 displays the trajectories produced by the three learned flows. To facilitate comparison, we use the same initial points Z 0 ∼ p 0 Z_0\sim p_0 Z 0 ∼ p 0 in all three panels. The effect of rectification is visually pronounced. The trajectories of 1-RF exhibit substantial curvature as the mass originating near the center is separated toward the two modes of the target distribution. After one additional rectification, the trajectories become nearly linear. The third round produces only a minor additional geometric improvement.
Figure 4. The figure shows that successive reflow steps straighten the trajectories for the 2D-GMM example.
We quantify this phenomenon using the trajectory-based straightness functional introduced in § 5.1 . For each K K K , we generate N = 5000 N=5000 N = 5000 fresh trajectories and compute
T ^ N ( K ) = 1 N ∑ i = 1 N ∥ Z 1 , i ( K ) − Z 0 , i ( K ) − v ^ ( K ) ( Z t i , i ( K ) , t i ) ∥ 2 2 , t i ∼ i . i . d . Unif [ 0 , 1 ] . \widehat{T}_N^{(K)}=\frac{1}{N}\sum_{i=1}^{N}\left\lVert Z_{1,i}^{(K)}-Z_{0,i}^{(K)}-\widehat{v}^{(K)}\left(Z_{t_i,i}^{(K)},t_i\right)\right\rVert_2^2,\qquad t_i\mathrel{\overset{\mathrm{i.i.d.}}{\sim}}\operatorname{Unif}[0,1]. T N ( K ) = N 1 i = 1 ∑ N Z 1 , i ( K ) − Z 0 , i ( K ) − v ( K ) ( Z t i , i ( K ) , t i ) 2 2 , t i ∼ i.i.d. Unif [ 0 , 1 ] .
We also report the corresponding standard error and the one-sided, per-round 95% upper confidence bound
U N , K = T ^ N ( K ) + z 0.95 σ ^ N , K N . U_{N,K} = \widehat{T}_N^{(K)} + z_{0.95}\frac{\widehat{\sigma}_{N,K}}{\sqrt{N}}. U N , K = T N ( K ) + z 0.95 N σ N , K .
The numerical results agree closely with the trajectory visualization. A single reflow step reduces the estimated straightness defect from 1.7665 to 5.75 × 10 − 4 5.75 \times10^{-4} 5.75 × 1 0 − 4 , a reduction of more than three orders of magnitude. A further rectification decreases the defect to 3.42 × 10 − 4 3.42 \times10^{-4} 3.42 × 1 0 − 4 , but the improvement is considerably smaller. Thus, in this example, most of the straightening effect occurs during the first reflow step, which conforms with the prior theoretical result in this paper.
Stopping rule The confidence bounds lead to the same conclusion. In particular,
U N , 1 = 1.8174 , U N , 2 = 8.04 × 10 − 4 , U N , 3 = 5.06 × 10 − 4 . U_{N,1} = 1.8174,\qquad U_{N,2} = 8.04 \times10^{-4},\qquad U_{N,3} = 5.06 \times10^{-4}. U N , 1 = 1.8174 , U N , 2 = 8.04 × 1 0 − 4 , U N , 3 = 5.06 × 1 0 − 4 .
Consequently, for each of the prescribed tolerances
δ ∈ { 0.50 , 0.20 , 0.10 , 0.05 } , \delta\in\{0.50, 0.20, 0.10, 0.05\}, δ ∈ { 0.50 , 0.20 , 0.10 , 0.05 } ,
the stopping rule in Algorithm 1 rejects H 0 : T ( K ) > δ H_0 : T^{(K)} > \delta H 0 : T ( K ) > δ for the first time at K = 2 K = 2 K = 2 . Hence, the per-round confidence rule selects 2-RF, in agreement with the visual observation in Figure 4 .
Finally, we examine whether the improvement in straightness comes at the expense of distributional accuracy. We compute an empirical Wasserstein-2 distance between the 1000 samples generated by each RF and the 1000 fresh samples from the target distribution, averaged over five independent runs. As shown in Table 1 , the empirical W 2 W_2 W 2 distances remain comparable across the three rounds, with 2-RF attaining the smallest average value. Thus, in this simple example, rectification dramatically straightens the trajectories without a corresponding deterioration in the quality of the generated distribution.
Table 1. Straightness and generation quality for the two-dimensional Gaussian-mixture experiment. The empirical W 2 W_2 W 2 distance is reported as mean ± \pm ± standard error over five independent runs.
This observation is consistent with the population interpretation of rectification: the rectified coupling preserves the endpoint marginals while progressively simplifying the geometry of the transport. However, in finite samples, the learned velocity fields and numerical ODE solutions only approximate this idealized operation. The empirical W 2 W_2 W 2 diagnostic therefore provides a complementary check that the observed reduction in the straightness defect is not merely obtained by sacrificing the target distribution.
MNIST data experiment
We next consider the MNIST dataset to study the proposed straightness criterion in a higher-dimensional setting. The images are normalized to [ − 1 , 1 ] [-1,1] [ − 1 , 1 ] , so that the target distribution is supported on [ − 1 , 1 ] 28 × 28 [-1,1]^{28 \times28} [ − 1 , 1 ] 28 × 28 , while the source distribution is p 0 = N ( 0 , I 784 ) p_0 = \mathcal{N}(0, I_{784}) p 0 = N ( 0 , I 784 ) . We first train a rectified flow between independent samples X 0 ∼ p 0 X_0 \sim p_0 X 0 ∼ p 0 and MNIST images X 1 ∼ p 1 X_1 \sim p_1 X 1 ∼ p 1 , and subsequently perform two additional rounds of rectification. At each round, the new RF is trained using endpoint pairs generated from the preceding learned flow.
Trajectory straightening and stopping Figure 5 shows the evolution of the same eight initial Gaussian samples under 1-RF, 2-RF, and 3-RF. We quantify the straightening using N = 2000 N = 2000 N = 2000 fresh trajectories at each round. The resulting straightness estimates, upper confidence bounds, and generation quality are reported in Table 2 .
Figure 5. Evolution of the same eight initial Gaussian samples under 1-RF, 2-RF, and 3-RF. It is seen that successive rectification visually degrades the quality of the generated images in Row 4. For Row 1, the generation quality is visually worse for 2-RF compared to 1-RF and 3-RF.
Table 2. Straightness and generation quality for MNIST.
Table 2 : Straightness and generation quality for MNIST. The MNIST feature FID is reported as mean ± \pm ± standard error over ten independent runs.
The first reflow reduces the estimated straightness defect by approximately 92.4%, from 56.89 to 4.31, whereas the second reflow gives a more modest reduction to 3.29. Thus, as in the Gaussian-mixture experiment, most of the straightening occurs after a single reflow step.
In high dimensions, directly specifying a fixed tolerance on the total straightness T ( K ) \mathcal{T}^{(K)} T ( K ) can be difficult to achieve, since it aggregates the squared discrepancy over all coordinates. We therefore consider the dimension-normalized criterion
T ( K ) d ≤ δ 0 , or equivalently T ( K ) ≤ d δ 0 . \frac{\mathcal{T}^{(K)}}{d} \le\delta_0, \qquad\text{or equivalently} \qquad\mathcal{T}^{(K)} \le d\delta_0. d T ( K ) ≤ δ 0 , or equivalently T ( K ) ≤ d δ 0 .
Here δ 0 \delta_0 δ 0 represents the desired average squared straightness defect per coordinate. Besides making the tolerance comparable across dimensions, such a criterion can be more practically meaningful in high-dimensional generative problems: one may only require the average coordinate-wise deviation from a straight trajectory to be small, rather than imposing a dimension-independent bound on the aggregate discrepancy. For MNIST, d = 28 × 28 = 784 d = 28 \times28 = 784 d = 28 × 28 = 784 is the full dimension. Taking δ 0 = 0.01 \delta_0 = 0.01 δ 0 = 0.01 , for example, gives δ = 7.84 \delta= 7.84 δ = 7.84 . Since
U N , 1 = 59.03 > 7.84 , U N , 2 = 5.36 < 7.84 , U_{N,1} = 59.03 > 7.84,\qquad U_{N,2} = 5.36 < 7.84, U N , 1 = 59.03 > 7.84 , U N , 2 = 5.36 < 7.84 ,
Algorithm 1 stops at K = 2 K = 2 K = 2 . Similarly, if δ 0 = 6 × 10 − 3 \delta_0 = 6 \times10^{-3} δ 0 = 6 × 1 0 − 3 , then δ ≈ 4.7 > 4.34 = U N , 3 \delta\approx4.7 > 4.34 = U_{N,3} δ ≈ 4.7 > 4.34 = U N , 3 , and Algorithm 1 stops at K = 3 K = 3 K = 3 .
Generation quality Table 2 also reveals an important trade-off. While successive reflow substantially improves straightness, the MNIST feature FID increases from 10.52 for 1-RF to 13.40 and 16.99 for 2-RF and 3-RF, respectively. This is consistent with recent observations that recursive reflow can accumulate approximation errors and lead to distributional drift or model collapse [61 , 26 ] . Several remedies have been proposed, including incorporating real data in subsequent reflow rounds [61 , 26 ] . An alternative approach is optimal self-distillation [49 ] , which avoids recursive replacement of endpoint pairs and instead mixes true and teacher-generated velocity targets on fixed interpolants, with the goal of improving velocity estimation and downstream generation. Our stopping rule provides a complementary perspective: rather than modifying the training procedure for subsequent rounds, it asks whether an additional reflow is necessary at all. In the present experiment, most of the improvement in straightness is already achieved by 2-RF, whereas an additional reflow provides only a modest reduction in the straightness defect while further increasing FID. Thus, statistically certifying approximate straightness can provide a principled mechanism for avoiding unnecessary reflow and the associated accumulation of distributional error.
Conclusion
In this paper, we have developed a criterion for the straightness of the couplings obtained from applying 2-RF to the independent couplings formed by the data and a standard Gaussian distribution. Our criterion reduces to interpretable conditions for some common classes of data distributions, such as the class of isotropic Gaussian mixture models and the class of smooth, strongly log-concave distributions. Furthermore, we show that this criterion is indeed sharp in the worst case by developing techniques for constructing non-straight couplings. Finally, we also develop statistical tests based on a trajectory-based functional to design stopping rules for reflow procedures.
Our work opens up the possibility for understanding further statistical properties of RFs. In particular, it would be interesting to study the behavior of couplings obtained from a higher number of rectification steps. We note that understanding a higher number of rectification steps requires more tools and new ideas than what we develop in the present article. One challenge is due to the fact that the solution of the ODE in (3) might not be unique for higher rectification steps; see recent papers Mena et al. [ [42 ] and Hertrich et al. [ [24 ] for more details.
Acknowledgments
We gratefully acknowledge NSF grants CCF-2019844, CCF-2505865, and 2217069.
AI statement. The authors use GPT 6-Astra for generating the construction of the limiting ODEs, for performing various calculations in the proof and for proof auditing. In particular, the use of Piola’s identity (Lemma G.6 ) was suggested by GPT 6-Astra as an alternative to using topological degree. The authors take full responsibility for all content and its accuracy.
Appendix
This supplement contains all the proofs and technical results for the main article.
Proof of Lemma 2.1
We have already discussed that if ( X 0 , X 1 ) (X_0,X_1) ( X 0 , X 1 ) is a straight coupling, then Rectflow ( X 0 , X 1 ) (X_0,X_1) ( X 0 , X 1 ) yields a straight flow. In fact, the flow X t : = ( 1 − t ) X 0 + t X 1 X_t := (1-t)X_0+tX_1 X t := ( 1 − t ) X 0 + t X 1 is exactly the solution to the 2-RF ODE, and it has a constant velocity almost surely.
For the other direction, we write { Z t } t ∈ [ 0 , 1 ] = Rectflow ( X 0 , X 1 ) \{Z_t\}_{t\in[0,1]}=\operatorname{Rectflow}(X_0,X_1) { Z t } t ∈ [ 0 , 1 ] = Rectflow ( X 0 , X 1 ) and assume that it is a straight flow. In other words, the flow has constant velocity and takes the form Z t = ( 1 − t ) Z 0 + t Z 1 Z_t=(1-t)Z_0+tZ_1 Z t = ( 1 − t ) Z 0 + t Z 1 . Also, due to the marginal preserving property of RF, we have Law ( Z t ) = Law ( X t ) \operatorname{Law}(Z_t)=\operatorname{Law}(X_t) Law ( Z t ) = Law ( X t ) . Collecting all of these facts, we can conclude
E ∥ Z t ∥ 2 2 = ( 1 − t ) E ∥ Z 0 ∥ 2 2 + t E ∥ Z 1 ∥ 2 2 − t ( 1 − t ) E ∥ Z 1 − Z 0 ∥ 2 2 \mathbb{E}\lVert Z_t\rVert_2^2=(1-t)\mathbb{E}\lVert Z_0\rVert_2^2+t\mathbb{E}\lVert Z_1\rVert_2^2-t(1-t)\mathbb{E}\lVert Z_1-Z_0\rVert_2^2 E ∥ Z t ∥ 2 2 = ( 1 − t ) E ∥ Z 0 ∥ 2 2 + t E ∥ Z 1 ∥ 2 2 − t ( 1 − t ) E ∥ Z 1 − Z 0 ∥ 2 2
and
E ∥ X t ∥ 2 2 = ( 1 − t ) E ∥ X 0 ∥ 2 2 + t E ∥ X 1 ∥ 2 2 − t ( 1 − t ) E ∥ X 1 − X 0 ∥ 2 2 . \mathbb{E}\lVert X_t\rVert_2^2=(1-t)\mathbb{E}\lVert X_0\rVert_2^2+t\mathbb{E}\lVert X_1\rVert_2^2-t(1-t)\mathbb{E}\lVert X_1-X_0\rVert_2^2. E ∥ X t ∥ 2 2 = ( 1 − t ) E ∥ X 0 ∥ 2 2 + t E ∥ X 1 ∥ 2 2 − t ( 1 − t ) E ∥ X 1 − X 0 ∥ 2 2 .
This shows that E ∥ X 1 − X 0 ∥ 2 2 = E ∥ Z 1 − Z 0 ∥ 2 2 \mathbb{E}\lVert X_1-X_0\rVert_2^2=\mathbb{E}\lVert Z_1-Z_0\rVert_2^2 E ∥ X 1 − X 0 ∥ 2 2 = E ∥ Z 1 − Z 0 ∥ 2 2 . By Jensen’s inequality, marginal preservation, and the conditional variance identity,
0 ≤ ∫ 0 1 E ∥ X 1 − X 0 − E [ X 1 − X 0 ∣ X t ] ∥ 2 2 d t ≤ E ∥ X 1 − X 0 ∥ 2 2 − E ∥ Z 1 − Z 0 ∥ 2 2 = 0. 0\leq\int_0^1\mathbb{E}\left\lVert X_1-X_0-\mathbb{E}[X_1-X_0\mid X_t]\right\rVert_2^2\,dt\leq\mathbb{E}\lVert X_1-X_0\rVert_2^2-\mathbb{E}\lVert Z_1-Z_0\rVert_2^2=0. 0 ≤ ∫ 0 1 E ∥ X 1 − X 0 − E [ X 1 − X 0 ∣ X t ] ∥ 2 2 d t ≤ E ∥ X 1 − X 0 ∥ 2 2 − E ∥ Z 1 − Z 0 ∥ 2 2 = 0.
Since the integrand is nonnegative, it follows that
E [ X 1 − X 0 ∣ X t ] = X 1 − X 0 almost surely \mathbb{E}[X_1-X_0\mid X_t]=X_1-X_0\qquad\text{almost surely} E [ X 1 − X 0 ∣ X t ] = X 1 − X 0 almost surely
for almost every t ∈ ( 0 , 1 ) t\in(0,1) t ∈ ( 0 , 1 ) . Thus ( X 0 , X 1 ) (X_0,X_1) ( X 0 , X 1 ) is a straight coupling, which completes the proof. □ \square □
Proof of the results in Section 3
Let us start with some useful connections between the velocity v t ( x ) v_t(x) v t ( x ) and the score function
s t ( x ) : = ∇ log p t ( x ) s_t(x):=\nabla\log p_t(x) s t ( x ) := ∇ log p t ( x )
where p t p_t p t is defined as in (6) .
For t ∈ ( 0 , 1 ) t\in(0,1) t ∈ ( 0 , 1 ) , Equation (2) can be rewritten as
v t ( x ) = E [ X 1 − X 0 ∣ X t = x ] = 1 1 − t ( E [ X 1 ∣ X t = x ] − x ) = 1 1 − t [ x t + ( 1 − t ) 2 t ⋅ s t ( x ) − x ] . (26) v_t(x)=\mathbb{E}[X_1-X_0\mid X_t=x]=\frac{1}{1-t}\left(\mathbb{E}[X_1\mid X_t=x]-x\right)=\frac{1}{1-t}\left[\frac{x}{t}+\frac{(1-t)^2}{t}\cdot s_t(x)-x\right]. \tag*{(26)} v t ( x ) = E [ X 1 − X 0 ∣ X t = x ] = 1 − t 1 ( E [ X 1 ∣ X t = x ] − x ) = 1 − t 1 [ t x + t ( 1 − t ) 2 ⋅ s t ( x ) − x ] . (26)
Indeed, the above follows from first-order Tweedie’s formula [16 , 43 ] , which guarantees that for any choice of t ∈ ( 0 , 1 ) t\in(0,1) t ∈ ( 0 , 1 ) and x ∈ R d x\in\mathbb{R}^d x ∈ R d ,
E [ X 1 ∣ X t = x ] = x t + ( 1 − t ) 2 t ⋅ s t ( x ) , \mathbb{E}[X_1\mid X_t=x]=\frac{x}{t}+\frac{(1-t)^2}{t}\cdot s_t(x), E [ X 1 ∣ X t = x ] = t x + t ( 1 − t ) 2 ⋅ s t ( x ) ,
Also, the second-order Tweedie’s formula gives
Cov μ x ( X 1 ) = ( 1 − t ) 2 t 2 ⋅ I d + ( 1 − t ) 4 t 2 ⋅ D x [ s t ( x ) ] \operatorname{Cov}_{\mu_x}(\mathbf{X}_1) = \frac{(1-t)^2}{t^2} \cdot I_d + \frac{(1-t)^4}{t^2} \cdot D_x[s_t(x)] Cov μ x ( X 1 ) = t 2 ( 1 − t ) 2 ⋅ I d + t 2 ( 1 − t ) 4 ⋅ D x [ s t ( x )]
where μ x : = Law ( X 1 ∣ X t = x ) \mu_x := \operatorname{Law}(\mathbf{X}_1 \mid X_t = x) μ x := Law ( X 1 ∣ X t = x ) .
Substituting this in (B.1), we obtain that
v t ( x ) = x t + 1 − t t ⋅ s t ( x ) , and D x [ v t ( x ) ] = − I d 1 − t + t ( 1 − t ) 3 ⋅ Cov μ x ( X 1 ) . (27) v_t(x) = \frac{x}{t} + \frac{1-t}{t} \cdot s_t(x), \qquad\text{and} \qquad D_x[v_t(x)] = -\frac{I_d}{1-t} + \frac{t}{(1-t)^3} \cdot\operatorname{Cov}_{\mu_x}(\mathbf{X}_1). \tag*{(27)} v t ( x ) = t x + t 1 − t ⋅ s t ( x ) , and D x [ v t ( x )] = − 1 − t I d + ( 1 − t ) 3 t ⋅ Cov μ x ( X 1 ) . (27)
The reformulation of v t v_t v t essentially shows that estimating v t v_t v t is equivalent to estimating the score s t s_t s t for t ∈ ( 0 , 1 ) t \in(0,1) t ∈ ( 0 , 1 ) .
Injectivity of terminal flow map
In this subsection, we establish a basic injectivity property of the terminal flow map associated with an ODE. Specifically, given a (possibly time-dependent) velocity field v : [ 0 , 1 ] × R d → R d \mathbf{v} : [0,1] \times\mathbb{R}^d \to\mathbb{R}^d v : [ 0 , 1 ] × R d → R d , we write v t ( x ) : = v ( t , x ) v_t(x) := v(t,x) v t ( x ) := v ( t , x ) and let Z t ( z ) Z_t(z) Z t ( z ) denote the solution at time t t t to the initial value problem starting from z z z . Under mild local integrability and spatial regularity assumptions ensuring local uniqueness of solutions, we show that the endpoint map Z 1 : R d → R d Z_1 : \mathbb{R}^d \to\mathbb{R}^d Z 1 : R d → R d is injective.
Lemma B.1 (Injectivity of the endpoint map). Let v : [ 0 , 1 ] × R d → R d v : [0,1] \times\mathbb{R}^d \to\mathbb{R}^d v : [ 0 , 1 ] × R d → R d be Borel measurable and continuously differentiable in space for almost every t t t . Suppose that, for every R > 0 R > 0 R > 0 ,
∫ 0 1 ( ∥ v t ( 0 ) ∥ 2 + sup ∥ z ∥ 2 ≤ R ∥ D v t ( z ) ∥ op ) d t < ∞ . \int_0^1 \left(\lVert v_t(0)\rVert_2 + \sup_{\lVert z\rVert_2 \le R} \lVert Dv_t(z)\rVert_{\operatorname{op}}\right) \, dt < \infty. ∫ 0 1 ( ∥ v t ( 0 ) ∥ 2 + ∥ z ∥ 2 ≤ R sup ∥ D v t ( z ) ∥ op ) d t < ∞. Assume that, for every z ∈ R d z \in\mathbb{R}^d z ∈ R d , the ODE
d d t Z t ( z ) = v t ( Z t ( z ) ) , Z 0 ( z ) = z , \frac{d}{dt} Z_t(z) = v_t\bigl(Z_t(z)\bigr), \qquad Z_0(z) = z, d t d Z t ( z ) = v t ( Z t ( z ) ) , Z 0 ( z ) = z , admits an absolutely continuous solution on [ 0 , 1 ] [0,1] [ 0 , 1 ] . Then Z 1 : R d → R d Z_1 : \mathbb{R}^d \to\mathbb{R}^d Z 1 : R d → R d is injective.
Proof. We begin by recalling the mean-value theorem for vector-valued functions:
v t ( x ) − v t ( y ) = [ ∫ 0 1 D v t ( y + ξ ( x − y ) ) d ξ ] ( x − y ) . v_t(x) - v_t(y) = \left[\int_0^1 Dv_t\bigl(y+\xi(x-y)\bigr)\,d\xi\right](x-y). v t ( x ) − v t ( y ) = [ ∫ 0 1 D v t ( y + ξ ( x − y ) ) d ξ ] ( x − y ) .
Then, for a fixed z z z with ∥ z ∥ 2 ≤ R \lVert z\rVert_2 \le R ∥ z ∥ 2 ≤ R , the mean-value theorem, by setting x = z x=z x = z and y = 0 y=0 y = 0 , leads to
∥ v t ( z ) − v t ( 0 ) ∥ 2 ≤ ∥ z ∥ 2 sup ξ ∈ [ 0 , 1 ] ∥ D v t ( ξ z ) ∥ op . \lVert v_t(z) - v_t(0)\rVert_2 \le\lVert z\rVert_2 \sup_{\xi\in[0,1]} \lVert Dv_t(\xi z)\rVert_{\operatorname{op}}. ∥ v t ( z ) − v t ( 0 ) ∥ 2 ≤ ∥ z ∥ 2 ξ ∈ [ 0 , 1 ] sup ∥ D v t ( ξ z ) ∥ op .
Therefore, a simple rearrangement of terms yields
sup ∥ z ∥ 2 ≤ R ∥ v t ( z ) ∥ 2 ≤ ∥ v t ( 0 ) ∥ 2 + R sup ∥ z ∥ 2 ≤ R ∥ D v t ( z ) ∥ op , Lip ( v t ; { ∥ z ∥ 2 ≤ R } ) ≤ sup ∥ z ∥ 2 ≤ R ∥ D v t ( z ) ∥ op , \sup_{\lVert z\rVert_2 \le R} \lVert v_t(z)\rVert_2 \le\lVert v_t(0)\rVert_2 + R \sup_{\lVert z\rVert_2 \le R} \lVert Dv_t(z)\rVert_{\operatorname{op}}, \qquad\operatorname{Lip}(v_t;\{\lVert z\rVert_2 \le R\}) \le\sup_{\lVert z\rVert_2 \le R} \lVert Dv_t(z)\rVert_{\operatorname{op}}, ∥ z ∥ 2 ≤ R sup ∥ v t ( z ) ∥ 2 ≤ ∥ v t ( 0 ) ∥ 2 + R ∥ z ∥ 2 ≤ R sup ∥ D v t ( z ) ∥ op , Lip ( v t ; {∥ z ∥ 2 ≤ R }) ≤ ∥ z ∥ 2 ≤ R sup ∥ D v t ( z ) ∥ op ,
where Lip ( v t ; { ∥ z ∥ 2 ≤ R } ) \operatorname{Lip}(v_t;\{\lVert z\rVert_2 \le R\}) Lip ( v t ; {∥ z ∥ 2 ≤ R }) denotes the uniform Lipschitz constant of v t ( ⋅ ) v_t(\cdot) v t ( ⋅ ) in the ball { z : ∥ z ∥ 2 ≤ R } \{z : \lVert z\rVert_2 \le R\} { z : ∥ z ∥ 2 ≤ R } . Hence, the local integrability hypothesis of Ambrosio et al. [ [1 ] holds. In particular, local uniqueness holds through every ( s , x ) ∈ [ 0 , 1 ] × R d (s,x) \in[0,1] \times\mathbb{R}^d ( s , x ) ∈ [ 0 , 1 ] × R d , on a relatively open time interval $I(s,x) \ni s. Suppose Z 1 ( x ) = Z 1 ( y ) Z_1(x)=Z_1(y) Z 1 ( x ) = Z 1 ( y ) , and define
A : = { t ∈ [ 0 , 1 ] : Z t ( x ) = Z t ( y ) } . A:=\{t\in[0,1]:Z_t(x)=Z_t(y)\}. A := { t ∈ [ 0 , 1 ] : Z t ( x ) = Z t ( y )} .
The continuity of the trajectories implies that A A A is relatively closed, and 1 ∈ A 1\in A 1 ∈ A . For any s ∈ A s\in A s ∈ A , the two trajectories solve the same initial-value problem at time s s s . Local uniqueness therefore gives an ε > 0 \varepsilon>0 ε > 0 such that
( s − ε , s + ε ) ∩ [ 0 , 1 ] ⊆ A . (s-\varepsilon,s+\varepsilon)\cap[0,1]\subseteq A. ( s − ε , s + ε ) ∩ [ 0 , 1 ] ⊆ A .
Thus A A A is also relatively open. Since [ 0 , 1 ] [0,1] [ 0 , 1 ] is connected, and A A A is a non-empty clopen subset of [ 0 , 1 ] [0,1] [ 0 , 1 ] , we can conclude A = [ 0 , 1 ] A=[0,1] A = [ 0 , 1 ] . In particular,
x = Z 0 ( x ) = Z 0 ( y ) = y , x=Z_0(x)=Z_0(y)=y, x = Z 0 ( x ) = Z 0 ( y ) = y ,
which proves injectivity.
Proof of Lemma 3.1 and Theorem 3.1
Proof of Lemma 3.1. Injectivity is obvious. If H t \mathbf{H}_t H t is continuous and injective, then its inverse on its image is measurable. Hence the displacement Z 1 ( Z 0 ) − Z 0 \mathbf{Z}_1(\mathbf{Z}_0)-\mathbf{Z}_0 Z 1 ( Z 0 ) − Z 0 is measurable with respect to H t ( Z 0 ) \mathbf{H}_t(\mathbf{Z}_0) H t ( Z 0 ) , and most importantly the σ \sigma σ –field generated by H t ( Z 0 ) \mathbf{H}_t(\mathbf{Z}_0) H t ( Z 0 ) coincides with that of Z 0 \mathbf{Z}_0 Z 0 . Therefore, it follows that
E [ Z 1 ( Z 0 ) − Z 0 ∣ t , H t ( Z 0 ) ] = E [ Z 1 ( Z 0 ) − Z 0 ∣ Z 0 ] = Z 1 ( Z 0 ) − Z 0 almost surely . \mathbb{E}[\mathbf{Z}_1(\mathbf{Z}_0)-\mathbf{Z}_0\mid t,\mathbf{H}_t(\mathbf{Z}_0)]
=\mathbb{E}[\mathbf{Z}_1(\mathbf{Z}_0)-\mathbf{Z}_0\mid\mathbf{Z}_0]
=\mathbf{Z}_1(\mathbf{Z}_0)-\mathbf{Z}_0\quad\text{almost surely}. E [ Z 1 ( Z 0 ) − Z 0 ∣ t , H t ( Z 0 )] = E [ Z 1 ( Z 0 ) − Z 0 ∣ Z 0 ] = Z 1 ( Z 0 ) − Z 0 almost surely .
This is precisely the straightness property in Definition 1. □ \square □
Before proving the theorem, we state the following elementary bound.
Lemma B.2. Let A A A be a symmetric matrix and u u u be a unit vector. Then, we have
u T A 2 u − ( u T A u ) 2 ≤ ( λ max ( A ) − λ min ( A ) ) 2 4 . u^{\mathsf{T}}A^2u-(u^{\mathsf{T}}Au)^2\le\frac{(\lambda_{\max}(A)-\lambda_{\min}(A))^2}{4}. u T A 2 u − ( u T A u ) 2 ≤ 4 ( λ m a x ( A ) − λ m i n ( A ) ) 2 .
Proof of Lemma B.2. Let us begin with the spectral form of A A A , i.e., A = ∑ j ∈ [ d ] λ j p j p j T A=\sum_{j\in[d]}\lambda_jp_jp_j^{\mathsf{T}} A = ∑ j ∈ [ d ] λ j p j p j T , where { p j } j ∈ [ d ] \{p_j\}_{j\in[d]} { p j } j ∈ [ d ] is the orthonormal set of eigenvectors and λ 1 ≥ ⋯ ≥ λ d \lambda_1\ge\dots\ge\lambda_d λ 1 ≥ ⋯ ≥ λ d are the corresponding eigenvalues. Let us define α j = ( u T p j ) 2 \alpha_j=(u^{\mathsf{T}}p_j)^2 α j = ( u T p j ) 2 , and note that ∑ j ∈ [ d ] α j = u T ( ∑ j ∈ [ d ] p j p j T ) u = u T u = 1 \sum_{j\in[d]}\alpha_j=u^{\mathsf{T}}(\sum_{j\in[d]}p_jp_j^{\mathsf{T}})u=u^{\mathsf{T}}u=1 ∑ j ∈ [ d ] α j = u T ( ∑ j ∈ [ d ] p j p j T ) u = u T u = 1 . Using the above spectral form, we have
u T A 2 u − ( u T A u ) 2 = ∑ j ∈ [ d ] λ j 2 ( u T p j ) 2 − ( ∑ j ∈ [ d ] λ j ( u T p j ) 2 ) 2 = Var α ( { λ j } j ∈ [ d ] ) ≤ ( λ max ( A ) − λ min ( A ) ) 2 4 , \begin{aligned}
u^{\mathsf{T}}A^2u-(u^{\mathsf{T}}Au)^2
&=\sum_{j\in[d]}\lambda_j^2(u^{\mathsf{T}}p_j)^2-\left(\sum_{j\in[d]}\lambda_j(u^{\mathsf{T}}p_j)^2\right)^2\\
&=\operatorname{Var}_{\alpha}(\{\lambda_j\}_{j\in[d]})\\
&\le\frac{(\lambda_{\max}(A)-\lambda_{\min}(A))^2}{4},
\end{aligned} u T A 2 u − ( u T A u ) 2 = j ∈ [ d ] ∑ λ j 2 ( u T p j ) 2 − j ∈ [ d ] ∑ λ j ( u T p j ) 2 2 = Var α ({ λ j } j ∈ [ d ] ) ≤ 4 ( λ m a x ( A ) − λ m i n ( A ) ) 2 ,
where the last inequality follows from the well-known Popoviciu inequality [47 , 3 ] . □ \square □
We are now ready to prove Theorem 3.1.
Proof of Theorem 3.1. For notational brevity, we define A t : = A t ( x , y ) A_t:=A_t(x,y) A t := A t ( x , y ) and u t : = δ t ( x , y ) u_t:=\delta_t(x,y) u t := δ t ( x , y ) for a fixed pair of distinct points x , y x,y x , y . Then, we recall ODE (3.5):
d u t d t = A t u t , u 0 = δ 0 ( x , y ) = y − x . (28) \frac{du_t}{dt}=A_tu_t,\qquad u_0=\delta_0(x,y)=y-x. \tag*{(28)} d t d u t = A t u t , u 0 = δ 0 ( x , y ) = y − x . (28)
The main strategy of the proof is to track the evolution of the angle θ t : = Angle ( u t , u 0 ) \theta_t:=\operatorname{Angle}(u_t,u_0) θ t := Angle ( u t , u 0 ) .
Assuming Condition (a): If we assume that [ A t , A s ] = 0 [A_t,A_s]=0 [ A t , A s ] = 0 , then using standard theory [40 ] , it follows that
u t = exp ( ∫ 0 t A s d s ) u 0 . u_t=\exp\left(\int_0^t A_s\,\mathrm{d}s\right)u_0. u t = exp ( ∫ 0 t A s d s ) u 0 .
Therefore, θ 1 u < π / 2 < π \theta_1^u<\pi/2<\pi θ 1 u < π /2 < π . Then the proof concludes by using Lemma 3.1.
Assuming Condition (b): We begin from the basic inequality, valid for all t ∈ [ 0 , 1 ] t\in[0,1] t ∈ [ 0 , 1 ] ,
d ∥ u t ∥ 2 2 d t = 2 u t T u ˙ t = 2 u t T A t u t ≥ 2 λ min ( A t ) ∥ u t ∥ 2 2 , (29) \frac{\mathrm{d}\lVert u_t\rVert_2^2}{\mathrm{d}t}=2u_t^\mathsf{T}\dot{u}_t=2u_t^\mathsf{T}A_tu_t\geq2\lambda_{\min}(A_t)\lVert u_t\rVert_2^2, \tag*{(29)} d t d ∥ u t ∥ 2 2 = 2 u t T u ˙ t = 2 u t T A t u t ≥ 2 λ m i n ( A t ) ∥ u t ∥ 2 2 , (29)
which, by Gronwall’s lemma, yields that ∥ u t ∥ 2 2 ≥ ∥ u 0 ∥ 2 2 exp ( 2 ∫ 0 t λ min ( A s ) d s ) > 0 \lVert u_t\rVert_2^2\geq\lVert u_0\rVert_2^2\exp\left(2\int_0^t\lambda_{\min}(A_s)\,\mathrm{d}s\right)>0 ∥ u t ∥ 2 2 ≥ ∥ u 0 ∥ 2 2 exp ( 2 ∫ 0 t λ m i n ( A s ) d s ) > 0 . This and (B.3) ensure that t ↦ ∥ u t ∥ 2 t\mapsto\lVert u_t\rVert_2 t ↦ ∥ u t ∥ 2 is a differentiable map over [ 0 , 1 ] [0,1] [ 0 , 1 ] . Now, we define the unit vector n t : = u t / ∥ u t ∥ 2 n_t:=u_t/\lVert u_t\rVert_2 n t := u t / ∥ u t ∥ 2 , so we can trivially write n t ∥ u t ∥ 2 = u t n_t\lVert u_t\rVert_2=u_t n t ∥ u t ∥ 2 = u t . Differentiating both sides of this identity and using (B.3), we obtain
u ˙ t = A t u t = n ˙ t ∥ u t ∥ 2 + n t d d t ∥ u t ∥ 2 = n ˙ t ∥ u t ∥ 2 + n t × 1 2 ∥ u t ∥ 2 × 2 ⟨ u ˙ t , u t ⟩ = n ˙ t ∥ u t ∥ 2 + n t ⟨ A t u t , n t ⟩ . \begin{aligned}
\dot{u}_t&=A_tu_t=\dot{n}_t\lVert u_t\rVert_2+n_t\frac{\mathrm{d}}{\mathrm{d}t}\lVert u_t\rVert_2\\
&=\dot{n}_t\lVert u_t\rVert_2+n_t\times\frac{1}{2\lVert u_t\rVert_2}\times2\langle\dot{u}_t,u_t\rangle\\
&=\dot{n}_t\lVert u_t\rVert_2+n_t\langle A_tu_t,n_t\rangle.
\end{aligned} u ˙ t = A t u t = n ˙ t ∥ u t ∥ 2 + n t d t d ∥ u t ∥ 2 = n ˙ t ∥ u t ∥ 2 + n t × 2 ∥ u t ∥ 2 1 × 2 ⟨ u ˙ t , u t ⟩ = n ˙ t ∥ u t ∥ 2 + n t ⟨ A t u t , n t ⟩ .
Dividing both sides by ∥ u t ∥ 2 > 0 \lVert u_t\rVert_2>0 ∥ u t ∥ 2 > 0 yields
A t n t = n ˙ t + n t n t T A t n t A_tn_t=\dot{n}_t+n_tn_t^\mathsf{T}A_tn_t A t n t = n ˙ t + n t n t T A t n t
and, rearranging,
n ˙ t = ( I d − n t n t T ) A t n t . (30) \dot{n}_t=(I_d-n_tn_t^\mathsf{T})A_tn_t. \tag*{(30)} n ˙ t = ( I d − n t n t T ) A t n t . (30)
Next, the endpoint angle θ t u \theta_t^u θ t u is bounded above by the length of the normalized trajectory on the unit sphere; that is,
θ t u ≤ ∫ 0 t ∥ n ˙ s ∥ 2 d s . \theta_t^u\leq\int_0^t\lVert\dot{n}_s\rVert_2\,\mathrm{d}s. θ t u ≤ ∫ 0 t ∥ n ˙ s ∥ 2 d s .
Therefore, using the ODE (B.5),
θ t u ≤ ∫ 0 t ∥ ( I d − n s n s T ) A s n s ∥ 2 d s = ∫ 0 t ∥ ( I d − n s n s T ) A s n s ∥ 2 2 d s = ∫ 0 t { n s T A s 2 n s − ( n s T A s n s ) 2 } 1 / 2 d s ≤ ∫ 0 t ( λ max ( A s ) − λ min ( A s ) ) 2 4 d s = ∫ 0 t λ max ( A s ) − λ min ( A s ) 2 d s , (31) \begin{aligned}
\theta_t^u&\leq\int_0^t\lVert(I_d-n_sn_s^\mathsf{T})A_sn_s\rVert_2\,\mathrm{d}s\\
&=\int_0^t\sqrt{\lVert(I_d-n_sn_s^\mathsf{T})A_sn_s\rVert_2^2}\,\mathrm{d}s\\
&=\int_0^t\left\{n_s^\mathsf{T}A_s^2n_s-\left(n_s^\mathsf{T}A_sn_s\right)^2\right\}^{1/2}\,\mathrm{d}s\\
&\leq\int_0^t\sqrt{\frac{\left(\lambda_{\max}(A_s)-\lambda_{\min}(A_s)\right)^2}{4}}\,\mathrm{d}s\\
&=\int_0^t\frac{\lambda_{\max}(A_s)-\lambda_{\min}(A_s)}{2}\,\mathrm{d}s, \tag*{(31)}
\end{aligned} θ t u ≤ ∫ 0 t ∥( I d − n s n s T ) A s n s ∥ 2 d s = ∫ 0 t ∥( I d − n s n s T ) A s n s ∥ 2 2 d s = ∫ 0 t { n s T A s 2 n s − ( n s T A s n s ) 2 } 1/2 d s ≤ ∫ 0 t 4 ( λ m a x ( A s ) − λ m i n ( A s ) ) 2 d s = ∫ 0 t 2 λ m a x ( A s ) − λ m i n ( A s ) d s , (31)
where the inequality is due to Lemma B.2. The claimed result under condition (b) follows from Lemma 3.1. □
Proofs of Theorems 3.2 and 3.3
Before proving Theorem 3.2, we collect a useful Jacobian estimate.
Lemma B.3. Recall v ( x , t ) v(x,t) v ( x , t ) in (B.1). Then, for all t ∈ [ 0 , 1 ] t \in[0,1] t ∈ [ 0 , 1 ] , it holds that
t − L ( 1 − t ) t 2 + L ( 1 − t ) 2 ⋅ I d ⪯ D x v ( x , t ) ⪯ t − m ( 1 − t ) t 2 + m ( 1 − t ) 2 ⋅ I d (32) \frac{t-L(1-t)}{t^2+L(1-t)^2}\cdot I_d \preceq D_xv(x,t) \preceq\frac{t-m(1-t)}{t^2+m(1-t)^2}\cdot I_d \tag*{(32)} t 2 + L ( 1 − t ) 2 t − L ( 1 − t ) ⋅ I d ⪯ D x v ( x , t ) ⪯ t 2 + m ( 1 − t ) 2 t − m ( 1 − t ) ⋅ I d (32) for all x ∈ R d x \in\mathbb{R}^d x ∈ R d .
Proof of Lemma B.3. Recall that v v v has the exact formula
v ( x , t ) = ∇ x ( ∥ x ∥ 2 2 t + 1 − t t log p t ( x ) ) (33) v(x,t)=\nabla_x\left(\frac{\lVert x\rVert^2}{2t}+\frac{1-t}{t}\log p_t(x)\right) \tag*{(33)} v ( x , t ) = ∇ x ( 2 t ∥ x ∥ 2 + t 1 − t log p t ( x ) ) (33)
where
p t ( x ) ∝ ∫ R d exp [ − ∥ x − t y ∥ 2 2 ( 1 − t ) 2 − V ( y ) ] d y . p_t(x)\propto\int_{\mathbb{R}^d}\exp\left[-\frac{\lVert x-ty\rVert^2}{2(1-t)^2}-V(y)\right]\,dy. p t ( x ) ∝ ∫ R d exp [ − 2 ( 1 − t ) 2 ∥ x − t y ∥ 2 − V ( y ) ] d y .
From the expression above, we find that
∇ x 2 log p t ( x ) = − 1 ( 1 − t ) 2 I d + t 2 ( 1 − t ) 4 Cov μ x ( Y ) (34) \nabla_x^2\log p_t(x)=-\frac{1}{(1-t)^2}I_d+\frac{t^2}{(1-t)^4}\operatorname{Cov}_{\mu_x}(Y) \tag*{(34)} ∇ x 2 log p t ( x ) = − ( 1 − t ) 2 1 I d + ( 1 − t ) 4 t 2 Cov μ x ( Y ) (34)
where Y ∼ μ x Y\sim\mu_x Y ∼ μ x and μ x \mu_x μ x is the measure
μ x ( d y ) ∝ exp [ − V ( y ) − ∥ x − t y ∥ 2 2 ( 1 − t ) 2 ] d y . \mu_x(dy)\propto\exp\left[-V(y)-\frac{\lVert x-ty\rVert^2}{2(1-t)^2}\right]\,dy. μ x ( d y ) ∝ exp [ − V ( y ) − 2 ( 1 − t ) 2 ∥ x − t y ∥ 2 ] d y .
It is easy to see that for every x x x , the measure μ x ( d y ) \mu_x(dy) μ x ( d y ) is a strongly log-concave distribution, and
∇ y 2 [ V ( y ) + ∥ x − t y ∥ 2 2 ( 1 − t ) 2 ] = ∇ 2 V ( y ) + t 2 ( 1 − t ) 2 I d . \nabla_y^2\left[V(y)+\frac{\lVert x-ty\rVert^2}{2(1-t)^2}\right]=\nabla^2V(y)+\frac{t^2}{(1-t)^2}I_d. ∇ y 2 [ V ( y ) + 2 ( 1 − t ) 2 ∥ x − t y ∥ 2 ] = ∇ 2 V ( y ) + ( 1 − t ) 2 t 2 I d .
Therefore, the Brascamp–Lieb inequality and the Crámer–Rao lower bound (see Lemma 7 in Chewi and Pooladian [ [12 ] ) give
[ ( 1 − t ) 2 t 2 + L ( 1 − t ) 2 ] ⋅ I d ⪯ Cov μ x ( Y ) ⪯ [ ( 1 − t ) 2 t 2 + m ( 1 − t ) 2 ] ⋅ I d . \left[\frac{(1-t)^2}{t^2+L(1-t)^2}\right]\cdot I_d\preceq\operatorname{Cov}_{\mu_x}(Y)\preceq\left[\frac{(1-t)^2}{t^2+m(1-t)^2}\right]\cdot I_d. [ t 2 + L ( 1 − t ) 2 ( 1 − t ) 2 ] ⋅ I d ⪯ Cov μ x ( Y ) ⪯ [ t 2 + m ( 1 − t ) 2 ( 1 − t ) 2 ] ⋅ I d .
The proof is completed by plugging the above expression into (B.9) and then (B.8). □
Proof of Theorem 3.2. Using Lemma B.3, we deduce that
Angle ( δ 1 ( x , y ) , δ 0 ( x , y ) ) ≤ 1 2 [ ∫ 0 1 s − m ( 1 − s ) s 2 + m ( 1 − s ) 2 d s − ∫ 0 1 s − L ( 1 − s ) s 2 + L ( 1 − s ) 2 d s ] = 1 4 log ( L m ) . \operatorname{Angle}(\delta_1(x,y),\delta_0(x,y))\leq\frac{1}{2}\left[\int_0^1\frac{s-m(1-s)}{s^2+m(1-s)^2}\,ds-\int_0^1\frac{s-L(1-s)}{s^2+L(1-s)^2}\,ds\right]=\frac{1}{4}\log\left(\frac{L}{m}\right). Angle ( δ 1 ( x , y ) , δ 0 ( x , y )) ≤ 2 1 [ ∫ 0 1 s 2 + m ( 1 − s ) 2 s − m ( 1 − s ) d s − ∫ 0 1 s 2 + L ( 1 − s ) 2 s − L ( 1 − s ) d s ] = 4 1 log ( m L ) .
As a result, Condition (b) in Theorem 3.1 is verified provided L / m < exp ( 4 π ) L/m<\exp(4\pi) L / m < exp ( 4 π ) . □
Proof of Theorem 3.3. In this setting the score takes the form
s t ( x ) = ∑ i = 1 K π i , t ( x ) ( t μ i − x σ t 2 ) . s_t(x)=\sum_{i=1}^{K}\pi_{i,t}(x)\left(\frac{t\mu_i-x}{\sigma_t^2}\right). s t ( x ) = i = 1 ∑ K π i , t ( x ) ( σ t 2 t μ i − x ) .
where σ t 2 = ( 1 − t ) 2 + t 2 σ 2 \sigma_t^2=(1-t)^2+t^2\sigma^2 σ t 2 = ( 1 − t ) 2 + t 2 σ 2 and
π i , t ( x ) = π i exp ( − ∥ x − t μ i ∥ 2 2 σ t 2 ) ∑ j = 1 K π j exp ( − ∥ x − t μ j ∥ 2 2 σ t 2 ) \pi_{i,t}(x)=\frac{\pi_i\exp\left(-\frac{\lVert x-t\mu_i\rVert^2}{2\sigma_t^2}\right)}{\sum_{j=1}^{K}\pi_j\exp\left(-\frac{\lVert x-t\mu_j\rVert^2}{2\sigma_t^2}\right)} π i , t ( x ) = ∑ j = 1 K π j exp ( − 2 σ t 2 ∥ x − t μ j ∥ 2 ) π i exp ( − 2 σ t 2 ∥ x − t μ i ∥ 2 )
Substituting in (B.2), we get
v t ( x ) = ( 1 + σ 2 ) t − 1 σ t 2 x + 1 − t σ t 2 ∑ i ∈ [ K ] π i , t ( x ) μ i , (35) v_t(x)=\frac{(1+\sigma^2)t-1}{\sigma_t^2}x+\frac{1-t}{\sigma_t^2}\sum_{i\in[K]}\pi_{i,t}(x)\mu_i, \tag*{(35)} v t ( x ) = σ t 2 ( 1 + σ 2 ) t − 1 x + σ t 2 1 − t i ∈ [ K ] ∑ π i , t ( x ) μ i , (35)
D x v t ( x ) = ( 1 + σ 2 ) t − 1 σ t 2 I d + t ( 1 − t ) 2 σ t 4 ∑ i ≠ j π i , t ( x ) π j , t ( x ) ( μ i − μ j ) ( μ i − μ j ) T . (36) D_xv_t(x)=\frac{(1+\sigma^2)t-1}{\sigma_t^2}I_d+\frac{t(1-t)}{2\sigma_t^4}\sum_{i\ne j}\pi_{i,t}(x)\pi_{j,t}(x)(\mu_i-\mu_j)(\mu_i-\mu_j)^{\mathsf{T}}. \tag*{(36)} D x v t ( x ) = σ t 2 ( 1 + σ 2 ) t − 1 I d + 2 σ t 4 t ( 1 − t ) i = j ∑ π i , t ( x ) π j , t ( x ) ( μ i − μ j ) ( μ i − μ j ) T . (36)
Next, note that B t ( x ) B_t(x) B t ( x ) in (B.11) is positive semi-definite uniformly over x ∈ R d x\in\mathbb{R}^d x ∈ R d . Therefore
λ max ( D x v t ( x ) ) − λ min ( D x v t ( x ) ) ≤ t ( 1 − t ) σ t 4 sup α : ∥ α ∥ 2 = 1 Var π t ( { α T μ i } i ∈ [ K ] ) ≤ t ( 1 − t ) 4 σ t 4 D max 2 . \lambda_{\max}(D_xv_t(x))-\lambda_{\min}(D_xv_t(x))\le\frac{t(1-t)}{\sigma_t^4}\sup_{\alpha:\lVert\alpha\rVert_2=1}\operatorname{Var}_{\pi_t}\left(\{\alpha^{\mathsf{T}}\mu_i\}_{i\in[K]}\right)\le\frac{t(1-t)}{4\sigma_t^4}D_{\max}^2. λ m a x ( D x v t ( x )) − λ m i n ( D x v t ( x )) ≤ σ t 4 t ( 1 − t ) α : ∥ α ∥ 2 = 1 sup Var π t ( { α T μ i } i ∈ [ K ] ) ≤ 4 σ t 4 t ( 1 − t ) D m a x 2 .
Using the identity
∫ 0 1 t ( 1 − t ) σ t 4 d t = 1 2 σ 2 , \int_0^1\frac{t(1-t)}{\sigma_t^4}\,dt=\frac{1}{2\sigma^2}, ∫ 0 1 σ t 4 t ( 1 − t ) d t = 2 σ 2 1 ,
Condition (b) in Theorem 3.1 holds as long as D max 2 < 16 π σ 2 D_{\max}^2<16\pi\sigma^2 D m a x 2 < 16 π σ 2 . The result then follows from Theorem 3.1. □
Proof of Proposition 4.1
Before proving Proposition 4.1, we need a technical result. We assume that the endpoint map Z 1 Z_1 Z 1 is C 1 C^1 C 1 and that the data’s law has a strictly positive density with finite second moment.
Lemma C.1. Suppose that the endpoint map has the form
Z 1 ( x ) = B x + R ( x ) , B ∈ S + + 3 , (37) Z_1(x)=Bx+R(x),\qquad B\in\mathbb{S}_{++}^3, \tag*{(37)} Z 1 ( x ) = B x + R ( x ) , B ∈ S ++ 3 , (37) where sup x ∥ R ( x ) ∥ < ∞ \sup_x\lVert R(x)\rVert<\infty sup x ∥ R ( x )∥ < ∞ . Recall the chord map H t H_t H t and the straightness defect S t S_t S t in (2.4) and (4.1), respectively. Assume that the law of X X X admits a strictly positive density. If det D H t ( x ∗ ) < 0 \det DH_t(x_*)<0 det D H t ( x ∗ ) < 0 for some x ∗ x_* x ∗ and some t ∈ ( 0 , 1 ) t\in(0,1) t ∈ ( 0 , 1 ) , then S t > 0 S_t>0 S t > 0 .
Proof of Lemma C.1. All the diffeomorphisms in the proof below are understood as C 1 C^1 C 1 . The argument below is based on topological degree, although our proof is elementary and does not make use of such a concept. The argument below is adapted from Simader [ [52 ] . Write
F : = H t , A t : = ( 1 − t ) I 3 + t B , q t : = t R , F:=H_t,\qquad A_t:=(1-t)I_3+tB,\qquad q_t:=tR, F := H t , A t := ( 1 − t ) I 3 + tB , q t := tR ,
so that
F ( x ) = A t x + q t ( x ) , A t ∈ S + + 3 , sup x ∥ q t ( x ) ∥ < ∞ . (38) F(x)=A_tx+q_t(x),\qquad A_t\in\mathbb{S}_{++}^3,\qquad\sup_x\lVert q_t(x)\rVert<\infty. \tag*{(38)} F ( x ) = A t x + q t ( x ) , A t ∈ S ++ 3 , x sup ∥ q t ( x )∥ < ∞. (38)
Step 1: Signed change of variables. We first prove the signed change-of-variables
∫ R 3 φ ( F ( x ) ) ⋅ det D F ( x ) d x = ∫ R 3 φ ( z ) d z , for all φ ∈ C c ∞ ( R 3 ) , (39) \int_{\mathbb{R}^{3}} \varphi(F(x)) \cdot\det DF(x)\,\mathrm{d}x = \int_{\mathbb{R}^{3}} \varphi(z)\,\mathrm{d}z, \qquad\text{for all } \varphi\in C^{\infty}_{c}(\mathbb{R}^{3}), \tag*{(39)} ∫ R 3 φ ( F ( x )) ⋅ det D F ( x ) d x = ∫ R 3 φ ( z ) d z , for all φ ∈ C c ∞ ( R 3 ) , (39)
and for all F F F satisfying (C.2).
Let us pause for a moment before proving (C.3) and explain what it means. If F F F is a diffeomorphism on R 3 \mathbb{R}^{3} R 3 , then the usual change of variable formula (Theorem 2.47, section 2.6 in Folland [ [18 ] ) gives
∫ R 3 φ ( F ( x ) ) ⋅ ∣ det D F ( x ) ∣ d x = ∫ R 3 φ ( z ) d z , for all φ ∈ C c ∞ ( R 3 ) . \int_{\mathbb{R}^{3}} \varphi(F(x)) \cdot\lvert\det DF(x)\rvert\,\mathrm{d}x = \int_{\mathbb{R}^{3}} \varphi(z)\,\mathrm{d}z, \qquad\text{for all } \varphi\in C^{\infty}_{c}(\mathbb{R}^{3}). ∫ R 3 φ ( F ( x )) ⋅ ∣ det D F ( x )∣ d x = ∫ R 3 φ ( z ) d z , for all φ ∈ C c ∞ ( R 3 ) .
The above display differs from (C.3) in the absolute value sign, and that it requires F F F to be a C 1 C^{1} C 1 diffeomorphism. The point of (C.3) is that the identity holds without the absolute value sign and the diffeomorphism requirement on F F F . We instead assume only (C.2).
To prove (C.3), keep t ∈ ( 0 , 1 ) t \in(0,1) t ∈ ( 0 , 1 ) fixed and define the interpolation
F s ( x ) = A t x + s q t ( x ) , 0 ≤ s ≤ 1. F_{s}(x) = A_{t}x + s q_{t}(x), \qquad0 \le s \le1. F s ( x ) = A t x + s q t ( x ) , 0 ≤ s ≤ 1.
and set
I ( s ) = ∫ R 3 φ ( F s ( x ) ) det D F s ( x ) d x . I(s) = \int_{\mathbb{R}^{3}} \varphi(F_{s}(x)) \det DF_{s}(x)\,\mathrm{d}x. I ( s ) = ∫ R 3 φ ( F s ( x )) det D F s ( x ) d x .
Because q t q_{t} q t is bounded and A t A_{t} A t is invertible, the integrands are supported in one fixed bounded set, uniformly in s s s . Thus I ( s ) < ∞ I(s) < \infty I ( s ) < ∞ for all s ∈ [ 0 , 1 ] s \in[0,1] s ∈ [ 0 , 1 ] . On the other hand, Lemma G.6 gives
d d s [ φ ( F s ) det D F s ] = div [ φ ( F s ) cof ( D F s ) T q t ] . \frac{\mathrm{d}}{\mathrm{d}s}\left[\varphi(F_{s})\det DF_{s}\right] = \operatorname{div}\left[\varphi(F_{s})\operatorname{cof}(DF_{s})^{\mathsf T}q_{t}\right]. d s d [ φ ( F s ) det D F s ] = div [ φ ( F s ) cof ( D F s ) T q t ] .
Hence
I ′ ( s ) = ∫ R 3 d d s [ φ ( F s ) det D F s ] d x = ∫ R 3 div [ φ ( F s ) cof ( D F s ) T q t ] d x = 0. I'(s) = \int_{\mathbb{R}^{3}} \frac{\mathrm{d}}{\mathrm{d}s}\left[\varphi(F_{s})\det DF_{s}\right]\,\mathrm{d}x = \int_{\mathbb{R}^{3}} \operatorname{div}\left[\varphi(F_{s})\operatorname{cof}(DF_{s})^{\mathsf T}q_{t}\right]\,\mathrm{d}x = 0. I ′ ( s ) = ∫ R 3 d s d [ φ ( F s ) det D F s ] d x = ∫ R 3 div [ φ ( F s ) cof ( D F s ) T q t ] d x = 0.
Let us supply more details on the last equality. The function G ( x ) : = φ ( F s ) cof ( D F s ) T q t G(x) := \varphi(F_{s})\operatorname{cof}(DF_{s})^{\mathsf T}q_{t} G ( x ) := φ ( F s ) cof ( D F s ) T q t inside the divergence has compact support because φ \varphi φ has compact support and
∥ F s ( x ) ∥ ≥ λ min ( A t ) ⋅ ∥ x ∥ − ∥ q t ∥ ∞ → ∞ , as ∥ x ∥ → ∞ . \lVert F_{s}(x)\rVert\ge\lambda_{\min}(A_{t}) \cdot\lVert x\rVert- \lVert q_{t}\rVert_{\infty} \to\infty, \qquad\text{as } \lVert x\rVert\to\infty. ∥ F s ( x )∥ ≥ λ m i n ( A t ) ⋅ ∥ x ∥ − ∥ q t ∥ ∞ → ∞ , as ∥ x ∥ → ∞.
Therefore, one can pick a ball B ( 0 , R ) B(0,R) B ( 0 , R ) that contains the support of G G G and use integration by parts to obtain
∫ R 3 div ( G ( x ) ) d x = ∫ { x ∈ R 3 : ∥ x ∥ = R } G ⋅ n d S = 0. \int_{\mathbb{R}^{3}} \operatorname{div}(G(x))\,\mathrm{d}x = \int_{\{x\in\mathbb{R}^{3}:\lVert x\rVert=R\}} G \cdot n\,\mathrm{d}S = 0. ∫ R 3 div ( G ( x )) d x = ∫ { x ∈ R 3 : ∥ x ∥ = R } G ⋅ n d S = 0.
Thus I ′ ( s ) = 0 I'(s)=0 I ′ ( s ) = 0 for all s ∈ [ 0 , 1 ] s \in[0,1] s ∈ [ 0 , 1 ] . Consequently,
I ( 1 ) = I ( 0 ) = ∫ R 3 φ ( A t x ) det A t d x = ∫ R 3 φ ( z ) d z , I(1) = I(0) = \int_{\mathbb{R}^{3}} \varphi(A_{t}x)\det A_{t}\,\mathrm{d}x = \int_{\mathbb{R}^{3}} \varphi(z)\,\mathrm{d}z, I ( 1 ) = I ( 0 ) = ∫ R 3 φ ( A t x ) det A t d x = ∫ R 3 φ ( z ) d z ,
which proves (C.3).
Note that the argument above implicitly uses that q t q_{t} q t is C 2 C^{2} C 2 (so that Lemma G.6 applies). This does not affect the conclusion in (C.3) even if q t q_{t} q t is only C 1 C^{1} C 1 since one can smooth out q t q_{t} q t by taking the convolution with a sequence of smooth mollifiers, see the argument in Lemma G.6 .
Step 2: Non-injectivity of F F F . Since det D F ( x ∗ ) < 0 \det DF(x_{*}) < 0 det D F ( x ∗ ) < 0 , the inverse function theorem gives the existence of open sets U ∋ x ∗ U \ni x_{*} U ∋ x ∗ and O ∋ F ( x ∗ ) O \ni F(x_{*}) O ∋ F ( x ∗ ) such that
F : U ⟶ O F: U \longrightarrow O F : U ⟶ O
is a diffeomorphism,
and
det D F < 0 on U . \det DF < 0 \quad\text{on } U. det D F < 0 on U .
Choose φ ∈ C c ∞ ( O ) \varphi\in C^\infty_c(O) φ ∈ C c ∞ ( O ) with φ ≥ 0 \varphi\ge0 φ ≥ 0 and φ ≢ 0 \varphi\not\equiv0 φ ≡ 0 . The usual change of variables formula gives
∫ U φ ( F ( x ) ) det D F ( x ) d x = − ∫ U φ ( F ( x ) ) ∣ det D F ( x ) ∣ d x = − ∫ O φ ( z ) d z . \int_U \varphi(F(x))\det DF(x)\,dx=-\int_U \varphi(F(x))\lvert\det DF(x)\rvert\,dx=-\int_O\varphi(z)\,dz. ∫ U φ ( F ( x )) det D F ( x ) d x = − ∫ U φ ( F ( x )) ∣ det D F ( x )∣ d x = − ∫ O φ ( z ) d z .
Combining this with (C.3),
∫ R 3 ∖ U φ ( F ( x ) ) det D F ( x ) d x = 2 ∫ O φ ( z ) d z > 0. \int_{\mathbb{R}^3\setminus U}\varphi(F(x))\det DF(x)\,dx=2\int_O\varphi(z)\,dz>0. ∫ R 3 ∖ U φ ( F ( x )) det D F ( x ) d x = 2 ∫ O φ ( z ) d z > 0.
Consequently, there exists x + ∉ U x_+ \notin U x + ∈ / U such that
φ ( F ( x + ) ) > 0 , det D F ( x + ) > 0 \varphi(F(x_+))>0,\qquad\det DF(x_+)>0 φ ( F ( x + )) > 0 , det D F ( x + ) > 0
and z 0 : = F ( x + ) ∈ O z_0:=F(x_+)\in O z 0 := F ( x + ) ∈ O . Next, let
x − : = ( F ∣ U ) − 1 ( z 0 ) , x_-:=(F|_U)^{-1}(z_0), x − := ( F ∣ U ) − 1 ( z 0 ) ,
which is well defined since F ∣ U F|_U F ∣ U is a diffeomorphism into O O O . Observe that
x − ≠ x + , F ( x − ) = F ( x + ) = z 0 , x_-\ne x_+,\qquad F(x_-)=F(x_+)=z_0, x − = x + , F ( x − ) = F ( x + ) = z 0 ,
and both Jacobians are invertible. The inverse function theorem once again gives, after possibly shrinking the neighborhoods, three open sets U 1 , U 2 , O 1 U_1,U_2,O_1 U 1 , U 2 , O 1 such that
U 1 ⊂ U , U 2 ⊂ R 3 ∖ U , O 1 ⊂ O U_1\subset U,\qquad U_2\subset\mathbb{R}^3\setminus U,\qquad O_1\subset O U 1 ⊂ U , U 2 ⊂ R 3 ∖ U , O 1 ⊂ O
and
F : U 1 ⟶ O 1 and F : U 2 ⟶ O 1 F:U_1\longrightarrow O_1\qquad\text{and}\qquad F:U_2\longrightarrow O_1 F : U 1 ⟶ O 1 and F : U 2 ⟶ O 1
are diffeomorphisms. Thus, F F F is non-injective on an open set.
Step 3: Non-injectivity yields positive straightness defect. Reasoning by contradiction, assume that S t = 0 S_t=0 S t = 0 . By (18) , there exists a Borel map g : R 3 → R 3 g:\mathbb{R}^3\to\mathbb{R}^3 g : R 3 → R 3 such that
X = g ( F ( X ) ) = g ( H t ( X ) ) , X=g(F(X))=g(H_t(X)), X = g ( F ( X )) = g ( H t ( X )) ,
almost surely. Since X X X has a strictly positive density, x = g ( F ( x ) ) x=g(F(x)) x = g ( F ( x )) for Lebesgue-almost every x x x in each U i U_i U i , i = 1 , 2 i=1,2 i = 1 , 2 . The fact that F : U 1 → O 1 F:U_1\to O_1 F : U 1 → O 1 and F : U 2 → O 1 F:U_2\to O_1 F : U 2 → O 1 are both diffeomorphisms forces g ( z ) ∈ U 1 ∩ U 2 g(z)\in U_1\cap U_2 g ( z ) ∈ U 1 ∩ U 2 for almost every z ∈ O 1 z\in O_1 z ∈ O 1 , which is a contradiction. Therefore, S t > 0 S_t>0 S t > 0 . □
Proof of Proposition 4.1 For t ∈ ( 0 , 1 ) t\in(0,1) t ∈ ( 0 , 1 ) ,
det D H t ( x ∗ ) = ∏ j = 1 3 ( 1 − t + t λ j ) . \det DH_t(x_*)=\prod_{j=1}^{3}(1-t+t\lambda_j). det D H t ( x ∗ ) = j = 1 ∏ 3 ( 1 − t + t λ j ) .
If ( 1 − λ 1 ) − 1 < t < ( 1 − λ 2 ) − 1 (1-\lambda_1)^{-1}<t<(1-\lambda_2)^{-1} ( 1 − λ 1 ) − 1 < t < ( 1 − λ 2 ) − 1 , the first factor is negative and the other two are positive. Hence, the determinant is negative throughout this nonempty open interval. Lemma C.1 therefore gives S t > 0 S_t>0 S t > 0 on that interval, so the coupling is not straight. □
Proof of Theorem 4.1
Throughout, S + + k \mathbb{S}_{++}^{k} S ++ k denotes the set of real symmetric positive-definite k × k k \times k k × k matrices. Write points of R 3 \mathbb{R}^{3} R 3 as ( x , z ) ∈ R 2 × R (x,z) \in\mathbb{R}^{2} \times\mathbb{R} ( x , z ) ∈ R 2 × R and set
ζ ( r ) = r 1 + r 2 . (40) \zeta(r) = \frac{r}{\sqrt{1+r^{2}}}. \tag*{(40)} ζ ( r ) = 1 + r 2 r . (40)
Our construction has two ingredients. We first show that a sufficiently nice potential function with a desirable condition number exists, namely an “admissible potential” as in Definition 3 below. The second step is to show that starting with an admissible potential, one can get the desired potential function by taking its “zero-temperature limit”, see Theorem D.2 for the precise statement. Theorem 4.1 follows as a corollary of Theorems D.1 and D.2.
Definition 3 (Admissible potential). We say that a potential function W ∈ C ∞ ( R 3 ) W \in C^{\infty}(\mathbb{R}^{3}) W ∈ C ∞ ( R 3 ) is ( κ , Λ ) (\kappa,\Lambda) ( κ , Λ ) -admissible if
(I) κ I 3 ⪯ ∇ 2 W ⪯ Λ I 3 \kappa I_{3} \preceq\nabla^{2}W \preceq\Lambda I_{3} κ I 3 ⪯ ∇ 2 W ⪯ Λ I 3 on R 3 \mathbb{R}^{3} R 3 , for some 0 < κ ≤ Λ < ∞ 0 < \kappa\leq\Lambda< \infty 0 < κ ≤ Λ < ∞ ;
(II) for some Q ∈ S + + 3 Q \in\mathbb{S}_{++}^{3} Q ∈ S ++ 3 , it holds that
sup y ∣ W ( y ) − 1 2 y T Q y ∣ < ∞ . \sup_{y} \left|W(y)-\frac{1}{2}y^{\mathsf{T}}Qy\right|<\infty. y sup W ( y ) − 2 1 y T Q y < ∞. (III) ∇ W ( 0 , z ) = ( 0 , z ) \nabla W(0,z)=(0,z) ∇ W ( 0 , z ) = ( 0 , z ) for all z ∈ R z \in\mathbb{R} z ∈ R ;
(IV) for M W M_W M W the solution to the differential equation
M W ′ ( r ) = r [ G W ( ζ ( r ) ) + r 2 ⋅ I 2 ] − 1 ⋅ M W ( r ) , M W ( 0 ) = I 2 , (41) M_W'(r)=r\left[G_W(\zeta(r))+r^{2}\cdot I_2\right]^{-1}\cdot M_W(r), \qquad M_W(0)=I_2, \tag*{(41)} M W ′ ( r ) = r [ G W ( ζ ( r )) + r 2 ⋅ I 2 ] − 1 ⋅ M W ( r ) , M W ( 0 ) = I 2 , (41) with G W G_W G W defined by the identity
∇ 2 W ( 0 , z ) = ( G W ( z ) 0 0 T 1 ) , z ∈ R , (42) \nabla^{2}W(0,z)=
\begin{pmatrix}
G_W(z) & 0\\
0^{\mathsf{T}} & 1
\end{pmatrix},
\qquad z\in\mathbb{R}, \tag*{(42)} ∇ 2 W ( 0 , z ) = ( G W ( z ) 0 T 0 1 ) , z ∈ R , (42) it holds that the matrix Π W = lim r → ∞ M W ( r ) / r \Pi_W=\lim_{r\to\infty}M_W(r)/r Π W = lim r → ∞ M W ( r ) / r is well-defined and has two distinct negative eigenvalues.
The first condition (I) requires W W W to be smooth and strongly convex with parameters Λ \Lambda Λ and κ \kappa κ , respectively. The second condition (II) requires that W W W is within an O ( 1 ) O(1) O ( 1 ) distance from a quadratic form, which will later be used to show that the endpoint map corresponding to N ( 0 , I 3 ) ⊗ exp ( − β W ( x ) ) \mathcal{N}(0,I_3)\otimes\exp(-\beta W(x)) N ( 0 , I 3 ) ⊗ exp ( − β W ( x )) is proper, for all β > 0 \beta>0 β > 0 . Conditions (III) and (IV) are needed to guarantee the existence of two distinct negative eigenvalues so that Proposition 4.1 is applicable.
Theorem D.1. For any S > 2 π S>2\pi S > 2 π and ε ∈ ( 0 , 1 / 10 ) \varepsilon\in(0,1/10) ε ∈ ( 0 , 1/10 ) , there exists a potential function W ∈ C ∞ ( R 3 ) W\in C^{\infty}(\mathbb{R}^{3}) W ∈ C ∞ ( R 3 ) that is ( κ , Λ ) (\kappa,\Lambda) ( κ , Λ ) -admissible in the sense of Definition 3 with
e 2 S < Λ κ < ( 1 + ε ) e 2 S . (43) e^{2S}<\frac{\Lambda}{\kappa}<(1+\varepsilon)e^{2S}. \tag*{(43)} e 2 S < κ Λ < ( 1 + ε ) e 2 S . (43)
Given an admissible potential W W W , let p 1 β ( x ) ∝ exp ( − β W ( x ) ) p_{1}^{\beta}(x)\propto\exp(-\beta W(x)) p 1 β ( x ) ∝ exp ( − β W ( x )) . The following theorem transfers non-straightness from the zero-temperature limit to a finite value of β \beta β .
Theorem D.2. Suppose W W W is ( κ , Λ ) (\kappa,\Lambda) ( κ , Λ ) -admissible in the sense of Definition 3 and satisfies (D.4). Then, for all sufficiently large β > 0 \beta>0 β > 0 , 2-RF yields a non-straight flow from the independent coupling N ( 0 , I 3 ) ⊗ p 1 β \mathcal{N}(0,I_3)\otimes p_{1}^{\beta} N ( 0 , I 3 ) ⊗ p 1 β , and the condition number of the potential β W \beta W β W satisfies (D.4).
Theorems D.1 and D.2 give Theorem 4.1 by taking S S S sufficiently close to 2 π 2\pi 2 π . It remains to prove Theorems D.1 and D.2. Before doing so, we need some technical results.
Technical preliminaries
We need some technical results for the construction of admissible potential functions. The first result is the construction of an appropriate convex, smooth function that is needed for the gluing argument used later on.
Lemma D.1. For every α > 0 \alpha> 0 α > 0 and β ∈ ( 0 , 1 / 10 ) \beta\in(0,1/10) β ∈ ( 0 , 1/10 ) , there exist δ ± > 0 \delta_{\pm} > 0 δ ± > 0 and χ ∈ C ∞ ( R ) \chi\in C^{\infty}(\mathbb{R}) χ ∈ C ∞ ( R ) that depend only on α \alpha α , β \beta β such that
(i) χ ′ ′ ≥ 0 \chi^{\prime\prime} \ge0 χ ′′ ≥ 0 on R \mathbb{R} R , so that χ \chi χ is convex;
(ii) χ ( s ) = ∣ s ∣ \chi(s) = |s| χ ( s ) = ∣ s ∣ , for s ∉ ( − δ − , δ + ) s \notin(-\delta_{-},\delta_{+}) s ∈ / ( − δ − , δ + ) ;
(iii) ∣ χ ′ ′ ( s ) ∣ ⋅ ∣ α − s ∣ ≤ β |\chi^{\prime\prime}(s)| \cdot|\alpha-s| \le\beta ∣ χ ′′ ( s ) ∣ ⋅ ∣ α − s ∣ ≤ β for every s ∈ R s \in\mathbb{R} s ∈ R ,
and with
α β e 2 / β ≤ α − δ + , δ − ≤ 4 α β . (44) \frac{\alpha}{\beta e^{2/\beta}} \le\alpha-\delta_{+}, \qquad\delta_{-} \le\frac{4\alpha}{\beta}. \tag*{(44)} β e 2/ β α ≤ α − δ + , δ − ≤ β 4 α . (44)
Proof of Lemma D.1. Put p ( x ) = χ ′ ′ ( x ) p(x) = \chi^{\prime\prime}(x) p ( x ) = χ ′′ ( x ) . Suppose we have chosen δ − \delta_{-} δ − and δ + \delta_{+} δ + , to satisfy conditions (i) and (ii), we only need
p ≥ 0 , ∫ R p ( x ) d x = 2 , ∫ R x p ( x ) d x = 0 , supp ( p ) ⊂ ( − δ − , δ + ) . (45) p \ge0,\qquad\int_{\mathbb{R}} p(x)\,dx = 2,\qquad\int_{\mathbb{R}} xp(x)\,dx = 0,\qquad\operatorname{supp}(p) \subset(-\delta_{-},\delta_{+}). \tag*{(45)} p ≥ 0 , ∫ R p ( x ) d x = 2 , ∫ R x p ( x ) d x = 0 , supp ( p ) ⊂ ( − δ − , δ + ) . (45)
Indeed, (i) follows from p ≥ 0 p \ge0 p ≥ 0 , and
χ ( s ) = − s + ∫ − ∞ s ( s − t ) ⋅ p ( t ) d t = ∣ s ∣ , if s ∉ ( − δ − , δ + ) . \chi(s) = -s + \int_{-\infty}^{s}(s-t)\cdot p(t)\,dt = |s|,\qquad\text{if }s\notin(-\delta_{-},\delta_{+}). χ ( s ) = − s + ∫ − ∞ s ( s − t ) ⋅ p ( t ) d t = ∣ s ∣ , if s ∈ / ( − δ − , δ + ) .
It suffices to pick δ ± \delta_{\pm} δ ± and p p p so that (D.5) and (iii) hold. Let η ∈ C ∞ ( ( − 1 , 1 ) ) \eta\in C^{\infty}((-1,1)) η ∈ C ∞ (( − 1 , 1 )) be an even, non-negative mollifier. For example, we can take
η ( x ) = c η { exp [ − 1 1 − 4 x 2 ] , ∣ x ∣ < 1 / 2 , 0 , ∣ x ∣ ≥ 1 / 2 , c η : = ( ∫ − 1 / 2 1 / 2 exp [ − 1 1 − 4 u 2 ] d u ) − 1 . \eta(x) = c_{\eta}
\begin{cases}
\exp\left[-\frac{1}{1-4x^{2}}\right], & |x|<1/2,\\
0, & |x|\ge1/2,
\end{cases}
\qquad
c_{\eta} := \left(\int_{-1/2}^{1/2}\exp\left[-\frac{1}{1-4u^{2}}\right]\,du\right)^{-1}. η ( x ) = c η { exp [ − 1 − 4 x 2 1 ] , 0 , ∣ x ∣ < 1/2 , ∣ x ∣ ≥ 1/2 , c η := ( ∫ − 1/2 1/2 exp [ − 1 − 4 u 2 1 ] d u ) − 1 .
Pick a , b , γ > 0 a,b,\gamma> 0 a , b , γ > 0 with b < α b < \alpha b < α , all to be chosen later, and choose p p p of the form
p ( s ) = q ∗ η ε ( s ) p(s) = q * \eta_{\varepsilon}(s) p ( s ) = q ∗ η ε ( s )
where
η ε ( x ) : = 1 ε ⋅ η ( x ε ) , q ( s ) : = γ α − s ⋅ 1 [ − a , b ] ( s ) . \eta_{\varepsilon}(x) := \frac{1}{\varepsilon}\cdot\eta\left(\frac{x}{\varepsilon}\right),\qquad q(s) := \frac{\gamma}{\alpha-s}\cdot\mathbf{1}_{[-a,b]}(s). η ε ( x ) := ε 1 ⋅ η ( ε x ) , q ( s ) := α − s γ ⋅ 1 [ − a , b ] ( s ) .
Now, observe that, since convolution preserves L 1 L_{1} L 1 norm and also first moment, if the mollifier is even, we have
∫ R p ( s ) d s = ∫ R q ∗ η ε ( x ) d x = ∫ − a b γ α − s d s = γ ⋅ log ( α + a α − b ) , \int_{\mathbb{R}}p(s)\,ds = \int_{\mathbb{R}}q*\eta_{\varepsilon}(x)\,dx = \int_{-a}^{b}\frac{\gamma}{\alpha-s}\,ds = \gamma\cdot\log\left(\frac{\alpha+a}{\alpha-b}\right), ∫ R p ( s ) d s = ∫ R q ∗ η ε ( x ) d x = ∫ − a b α − s γ d s = γ ⋅ log ( α − b α + a ) ,
and
∫ R s ⋅ p ( s ) d s = ∫ R s ⋅ q ( s ) d s = γ ( α ⋅ log ( α + a α − b ) − a − b ) . \int_{\mathbb{R}}s\cdot p(s)\,ds = \int_{\mathbb{R}}s\cdot q(s)\,ds = \gamma\left(\alpha\cdot\log\left(\frac{\alpha+a}{\alpha-b}\right)-a-b\right). ∫ R s ⋅ p ( s ) d s = ∫ R s ⋅ q ( s ) d s = γ ( α ⋅ log ( α − b α + a ) − a − b ) .
Thus, to get the second and third constraints in (D.6), we need
γ ⋅ log ( α + a α − b ) = 2 , α ⋅ log α + a α − b − a − b = 0. \gamma\cdot\log\left(\frac{\alpha+a}{\alpha-b}\right)=2,\qquad\alpha\cdot\log\frac{\alpha+a}{\alpha-b}-a-b=0. γ ⋅ log ( α − b α + a ) = 2 , α ⋅ log α − b α + a − a − b = 0.
Plugging the first equation into the second one, we have a + b = ( 2 α ) / γ a+b=(2\alpha)/\gamma a + b = ( 2 α ) / γ . Solving these equations, we get
b = α ⋅ [ 1 − 2 γ ( e 2 / γ − 1 ) ] , a = α ⋅ [ 2 γ + 2 γ ( e 2 / γ − 1 ) − 1 ] . (46) b=\alpha\cdot\left[1-\frac{2}{\gamma(e^{2/\gamma}-1)}\right],\qquad a=\alpha\cdot\left[\frac{2}{\gamma}+\frac{2}{\gamma(e^{2/\gamma}-1)}-1\right]. \tag*{(46)} b = α ⋅ [ 1 − γ ( e 2/ γ − 1 ) 2 ] , a = α ⋅ [ γ 2 + γ ( e 2/ γ − 1 ) 2 − 1 ] . (46)
Note that supp ( η ε ) ⊂ ( − ε , ε ) \operatorname{supp}(\eta_\varepsilon)\subset(-\varepsilon,\varepsilon) supp ( η ε ) ⊂ ( − ε , ε ) and supp ( p ) ⊂ [ − a − ε , b + ε ] \operatorname{supp}(p)\subset[-a-\varepsilon,b+\varepsilon] supp ( p ) ⊂ [ − a − ε , b + ε ] . It suffices to choose a , b , ε a,b,\varepsilon a , b , ε for given α , β \alpha,\beta α , β . The term ε \varepsilon ε is harmless since we can make it as small as possible once a a a and b b b are chosen. We will then set δ − = a + ε \delta_-=a+\varepsilon δ − = a + ε and δ + = b + ε \delta_+=b+\varepsilon δ + = b + ε .
To get (iii), note that
( α − s ) p ( s ) = γ ∫ R η ε ( u ) ⋅ α − s α − s + u ⋅ 1 [ − a , b ] ( s − u ) d u ≤ γ ∫ R η ε ( u ) ⋅ ( 1 + ε α − b ) d u = γ ⋅ ( 1 + ε α − b ) , \begin{aligned}
(\alpha-s)p(s)=\gamma\int_{\mathbb{R}}\eta_\varepsilon(u)\cdot\frac{\alpha-s}{\alpha-s+u}\cdot\mathbf{1}_{[-a,b]}(s-u)\,du \\
&\le\gamma\int_{\mathbb{R}}\eta_\varepsilon(u)\cdot\left(1+\frac{\varepsilon}{\alpha-b}\right)\,du=\gamma\cdot\left(1+\frac{\varepsilon}{\alpha-b}\right),
\end{aligned} ( α − s ) p ( s ) = γ ∫ R η ε ( u ) ⋅ α − s + u α − s ⋅ 1 [ − a , b ] ( s − u ) d u ≤ γ ∫ R η ε ( u ) ⋅ ( 1 + α − b ε ) d u = γ ⋅ ( 1 + α − b ε ) ,
where the last inequality follows from the fact that the function inside the integral takes non-zero value only when ∣ u ∣ ≤ ε |u|\le\varepsilon ∣ u ∣ ≤ ε and s − u ∈ [ − a , b ] s-u\in[-a,b] s − u ∈ [ − a , b ] , which in turn yields
α − s α − s + u = 1 − u α − s + u ≤ 1 + ∣ u α − s + u ∣ ≤ 1 + ε α − b \frac{\alpha-s}{\alpha-s+u}=1-\frac{u}{\alpha-s+u}\le1+\left|\frac{u}{\alpha-s+u}\right|\le1+\frac{\varepsilon}{\alpha-b} α − s + u α − s = 1 − α − s + u u ≤ 1 + α − s + u u ≤ 1 + α − b ε
on the set { u : η ε ( u ) ⋅ α − s α − s + u ⋅ 1 [ − a , b ] ( s − u ) ≠ 0 } \left\{u:\eta_\varepsilon(u)\cdot\frac{\alpha-s}{\alpha-s+u}\cdot\mathbf{1}_{[-a,b]}(s-u)\ne0\right\} { u : η ε ( u ) ⋅ α − s + u α − s ⋅ 1 [ − a , b ] ( s − u ) = 0 } .
Finally, with a , b a,b a , b chosen as in (D.7), for (i)–(iii) and (D.5) to hold, we need
γ ⋅ ( 1 + ε α − b ) ≤ β , α − b − ε ≥ α β ⋅ e 2 / β , a + ε ≤ 4 α β . \gamma\cdot\left(1+\frac{\varepsilon}{\alpha-b}\right)\le\beta,\qquad\alpha-b-\varepsilon\ge\frac{\alpha}{\beta\cdot e^{2/\beta}},\qquad a+\varepsilon\le\frac{4\alpha}{\beta}. γ ⋅ ( 1 + α − b ε ) ≤ β , α − b − ε ≥ β ⋅ e 2/ β α , a + ε ≤ β 4 α .
Such a choice of ( γ , ε ) (\gamma,\varepsilon) ( γ , ε ) exists because we can first choose γ \gamma γ to approach β \beta β from the left and then take ε \varepsilon ε sufficiently small. The proof is completed. □
We will also need the following elementary ODE result.
Lemma D.2. For all S > 2 π S>2\pi S > 2 π , there exists a smooth function H : ( 0 , ∞ ) → S + + 2 H:(0,\infty)\to S^2_{++} H : ( 0 , ∞ ) → S ++ 2 , which can be constructed explicitly as in (D.9) below, such that the ODE
M ′ ( r ) = r ( H ( r ) + r 2 I 2 ) − 1 ⋅ M ( r ) , M ( 0 ) = I 2 M'(r)=r\left(H(r)+r^2I_2\right)^{-1}\cdot M(r),\qquad M(0)=I_2 M ′ ( r ) = r ( H ( r ) + r 2 I 2 ) − 1 ⋅ M ( r ) , M ( 0 ) = I 2 has a unique solution M ∈ C 1 ( [ 0 , ∞ ) , G L 2 ( R ) ) M\in C^1([0,\infty),\mathrm{GL}_2(\mathbb{R})) M ∈ C 1 ([ 0 , ∞ ) , GL 2 ( R )) with
Π = lim r → ∞ M ( r ) r = − e − S / 2 ⋅ exp ( S π − π 0 ) . \Pi=\lim_{r\to\infty}\frac{M(r)}{r}=-e^{-S/2}\cdot\exp\begin{pmatrix} S & \pi\\ -\pi& 0 \end{pmatrix}. Π = r → ∞ lim r M ( r ) = − e − S /2 ⋅ exp ( S − π π 0 ) . In particular, Π \Pi Π has two distinct negative eigenvalues: − exp ( − S 2 − 4 π 2 2 ) -\exp\left(-\frac{\sqrt{S^2-4\pi^2}}{2}\right) − exp ( − 2 S 2 − 4 π 2 ) and − exp ( S 2 − 4 π 2 2 ) -\exp\left(\frac{\sqrt{S^2-4\pi^2}}{2}\right) − exp ( 2 S 2 − 4 π 2 ) .
We note that the assumption S > 2 π S > 2\pi S > 2 π is crucial here. For 0 < S < 2 π 0 < S < 2\pi 0 < S < 2 π , the matrix Π \Pi Π has nonreal eigenvalues; at S = 2 π S = 2\pi S = 2 π , it has the repeated negative eigenvalue − 1 -1 − 1 . Neither case is useful for our construction of an admissible potential function. It is also worth noting that for the construction of an admissible potential function to work, the solution M M M of the ODE above has to be non-symmetric, at least for some r > 0 r > 0 r > 0 .
Proof of Lemma D.2. Consider the functions over [ 0 , ∞ ) [0,\infty) [ 0 , ∞ )
b ( r ) : = r r 2 + e S , δ ( r ) : = r r 2 + e − S − r r 2 + e S and ϑ ( r ) = π S ∫ 0 r δ ( s ) d s . b(r) := \frac{r}{r^2 + e^S}, \qquad\delta(r) := \frac{r}{r^2 + e^{-S}} - \frac{r}{r^2 + e^S}
\qquad\text{and}\qquad
\vartheta(r) = \frac{\pi}{S}\int_0^r \delta(s)\,ds. b ( r ) := r 2 + e S r , δ ( r ) := r 2 + e − S r − r 2 + e S r and ϑ ( r ) = S π ∫ 0 r δ ( s ) d s .
Then,
ϑ ( 0 ) = 0 , ϑ ( ∞ ) = π , ϑ ′ ( r ) = π S ⋅ δ ( r ) \vartheta(0) = 0,\qquad\vartheta(\infty) = \pi,\qquad\vartheta'(r) = \frac{\pi}{S}\cdot\delta(r) ϑ ( 0 ) = 0 , ϑ ( ∞ ) = π , ϑ ′ ( r ) = S π ⋅ δ ( r )
and
∫ 0 ∞ δ ( x ) d x = lim r → ∞ ∫ 0 r δ ( x ) d x = lim r → ∞ { S + 1 2 log ( r 2 + e − S r 2 + e S ) } = S . \int_0^\infty\delta(x)\,dx = \lim_{r\to\infty}\int_0^r \delta(x)\,dx = \lim_{r\to\infty}\left\{S + \frac{1}{2}\log\left(\frac{r^2 + e^{-S}}{r^2 + e^S}\right)\right\} = S. ∫ 0 ∞ δ ( x ) d x = r → ∞ lim ∫ 0 r δ ( x ) d x = r → ∞ lim { S + 2 1 log ( r 2 + e S r 2 + e − S ) } = S .
Let u ϕ = ( cos ϕ , sin ϕ ) T \boldsymbol{u}_{\phi} = (\cos\phi,\sin\phi)^{\mathsf T} u ϕ = ( cos ϕ , sin ϕ ) T and define the rank-one matrix
P ϕ = u ϕ u ϕ T . (47) P_{\phi} = \boldsymbol{u}_{\phi}\boldsymbol{u}_{\phi}^{\mathsf T}. \tag*{(47)} P ϕ = u ϕ u ϕ T . (47)
Set
H ( r ) = e − S ⋅ P ϑ ( r ) + e S ⋅ ( I 2 − P ϑ ( r ) ) . (48) H(r) = e^{-S}\cdot P_{\vartheta(r)} + e^S\cdot\left(I_2 - P_{\vartheta(r)}\right). \tag*{(48)} H ( r ) = e − S ⋅ P ϑ ( r ) + e S ⋅ ( I 2 − P ϑ ( r ) ) . (48)
It is easy to check that for all r ≥ 0 r \ge0 r ≥ 0 , spec H ( r ) = { e − S , e S } \operatorname{spec} H(r) = \{e^{-S},e^S\} spec H ( r ) = { e − S , e S } , so that
e − S I 2 ⪯ H ( r ) ⪯ e S I 2 , for all r ≥ 0. e^{-S}I_2 \preceq H(r) \preceq e^S I_2,\qquad\text{for all } r \ge0. e − S I 2 ⪯ H ( r ) ⪯ e S I 2 , for all r ≥ 0.
Thus, Lemma G.2 yields the existence and uniqueness of a solution M ∈ C 1 ( [ 0 , ∞ ) , GL 2 ( R ) ) M \in C^1([0,\infty),\operatorname{GL}_2(\mathbb{R})) M ∈ C 1 ([ 0 , ∞ ) , GL 2 ( R )) . Next, using the Sherman–Morrison formula
( a I − b u u T ) − 1 = 1 a ⋅ I + b a 2 − a b ⋅ u u T , \left(aI - b\boldsymbol{u}\boldsymbol{u}^{\mathsf T}\right)^{-1}
= \frac{1}{a}\cdot I + \frac{b}{a^2-ab}\cdot\boldsymbol{u}\boldsymbol{u}^{\mathsf T}, ( a I − b u u T ) − 1 = a 1 ⋅ I + a 2 − ab b ⋅ u u T ,
we get
r ⋅ ( H ( r ) + r 2 ⋅ I 2 ) − 1 = r ⋅ [ ( r 2 + e S ) ⋅ I 2 − ( e S − e − S ) ⋅ P ϑ ( r ) ] − 1 = r r 2 + e S ⋅ I 2 + ( r r 2 + e − S − r r 2 + e S ) ⋅ P ϑ ( r ) = b ( r ) ⋅ I 2 + δ ( r ) ⋅ P ϑ ( r ) . \begin{aligned}
r\cdot\left(H(r)+r^2\cdot I_2\right)^{-1}
&= r\cdot\left[(r^2+e^S)\cdot I_2-(e^S-e^{-S})\cdot P_{\vartheta(r)}\right]^{-1}\\
&= \frac{r}{r^2+e^S}\cdot I_2+
\left(\frac{r}{r^2+e^{-S}}-\frac{r}{r^2+e^S}\right)\cdot P_{\vartheta(r)}\\
&= b(r)\cdot I_2+\delta(r)\cdot P_{\vartheta(r)}.
\end{aligned} r ⋅ ( H ( r ) + r 2 ⋅ I 2 ) − 1 = r ⋅ [ ( r 2 + e S ) ⋅ I 2 − ( e S − e − S ) ⋅ P ϑ ( r ) ] − 1 = r 2 + e S r ⋅ I 2 + ( r 2 + e − S r − r 2 + e S r ) ⋅ P ϑ ( r ) = b ( r ) ⋅ I 2 + δ ( r ) ⋅ P ϑ ( r ) .
Thus, the ODE for M M M becomes
M ′ ( r ) = [ b ( r ) ⋅ I 2 + δ ( r ) ⋅ P ϑ ( r ) ] ⋅ M ( r ) , r ≥ 0. M'(r)=\left[b(r)\cdot I_2+\delta(r)\cdot P_{\vartheta(r)}\right]\cdot M(r),\qquad r\ge0. M ′ ( r ) = [ b ( r ) ⋅ I 2 + δ ( r ) ⋅ P ϑ ( r ) ] ⋅ M ( r ) , r ≥ 0.
Its solution is
M ( r ) = 1 + e − S r 2 ⋅ R ϑ ( r ) ⋅ exp [ ϑ ( r ) ( S π P 0 − Ω ) ] , r ≥ 0. M(r)=\sqrt{1+e^{-S}r^2}\cdot R_{\vartheta(r)}\cdot\exp\left[\vartheta(r)\left(\frac{S}{\pi}P_0-\Omega\right)\right],\qquad r\ge0. M ( r ) = 1 + e − S r 2 ⋅ R ϑ ( r ) ⋅ exp [ ϑ ( r ) ( π S P 0 − Ω ) ] , r ≥ 0.
where
Ω = ( 0 − 1 1 0 ) and R ϕ = exp ( ϕ Ω ) = ( cos ϕ − sin ϕ sin ϕ cos ϕ ) . (49) \boldsymbol{\Omega}=\begin{pmatrix}0&-1\\1&0\end{pmatrix}
\qquad\text{and}\qquad
R_{\phi}=\exp(\phi\boldsymbol{\Omega})=\begin{pmatrix}\cos\phi&-\sin\phi\\\sin\phi&\cos\phi\end{pmatrix}. \tag*{(49)} Ω = ( 0 1 − 1 0 ) and R ϕ = exp ( ϕ Ω ) = ( cos ϕ sin ϕ − sin ϕ cos ϕ ) . (49)
Indeed, it is easy to check that M ( 0 ) = I 2 M(0)=I_2 M ( 0 ) = I 2 and, setting Q : = S π P 0 − Ω Q:=\frac{S}{\pi}P_0-\boldsymbol{\Omega} Q := π S P 0 − Ω ,
M ′ ( r ) = d d r [ 1 + e − S r 2 ⋅ R ϑ ( r ) ⋅ exp [ ϑ ( r ) Q ] ] + 1 + e − S r 2 ⋅ d d r [ R ϑ ( r ) ] ⋅ exp [ ϑ ( r ) Q ] + 1 + e − S r 2 ⋅ R ϑ ( r ) ⋅ d d r { exp [ ϑ ( r ) Q ] } = b ( r ) ⋅ M ( r ) + 1 + e − S r 2 ⋅ ϑ ′ ( r ) Ω R ϑ ( r ) ⋅ exp [ ϑ ( r ) Q ] + 1 + e − S r 2 ⋅ R ϑ ( r ) ⋅ ϑ ′ ( r ) Q ⋅ exp [ ϑ ( r ) Q ] = b ( r ) ⋅ M ( r ) + 1 + e − S r 2 ⋅ ϑ ′ ( r ) R ϑ ( r ) ( Ω + Q ) ⋅ exp [ ϑ ( r ) Q ] = [ b ( r ) ⋅ I 2 + δ ( r ) ⋅ P ϑ ( r ) ] ⋅ M ( r ) , \begin{aligned}
M'(r)
&=\frac{d}{dr}\left[\sqrt{1+e^{-Sr^2}}\cdot R_{\vartheta(r)}\cdot\exp[\vartheta(r)Q]\right]+\sqrt{1+e^{-Sr^2}}\cdot\frac{d}{dr}\left[R_{\vartheta(r)}\right]\cdot\exp[\vartheta(r)Q]\\
&\quad+\sqrt{1+e^{-Sr^2}}\cdot R_{\vartheta(r)}\cdot\frac{d}{dr}\{\exp[\vartheta(r)Q]\}\\
&=b(r)\cdot M(r)+\sqrt{1+e^{-Sr^2}}\cdot\vartheta'(r)\boldsymbol{\Omega}R_{\vartheta(r)}\cdot\exp[\vartheta(r)Q]\\
&\quad+\sqrt{1+e^{-Sr^2}}\cdot R_{\vartheta(r)}\cdot\vartheta'(r)Q\cdot\exp[\vartheta(r)Q]\\
&=b(r)\cdot M(r)+\sqrt{1+e^{-Sr^2}}\cdot\vartheta'(r)R_{\vartheta(r)}(\boldsymbol{\Omega}+Q)\cdot\exp[\vartheta(r)Q]\\
&=\left[b(r)\cdot I_2+\delta(r)\cdot P_{\vartheta(r)}\right]\cdot M(r),
\end{aligned} M ′ ( r ) = d r d [ 1 + e − S r 2 ⋅ R ϑ ( r ) ⋅ exp [ ϑ ( r ) Q ] ] + 1 + e − S r 2 ⋅ d r d [ R ϑ ( r ) ] ⋅ exp [ ϑ ( r ) Q ] + 1 + e − S r 2 ⋅ R ϑ ( r ) ⋅ d r d { exp [ ϑ ( r ) Q ]} = b ( r ) ⋅ M ( r ) + 1 + e − S r 2 ⋅ ϑ ′ ( r ) Ω R ϑ ( r ) ⋅ exp [ ϑ ( r ) Q ] + 1 + e − S r 2 ⋅ R ϑ ( r ) ⋅ ϑ ′ ( r ) Q ⋅ exp [ ϑ ( r ) Q ] = b ( r ) ⋅ M ( r ) + 1 + e − S r 2 ⋅ ϑ ′ ( r ) R ϑ ( r ) ( Ω + Q ) ⋅ exp [ ϑ ( r ) Q ] = [ b ( r ) ⋅ I 2 + δ ( r ) ⋅ P ϑ ( r ) ] ⋅ M ( r ) ,
where the last equality follows from the fact that R ϕ R_{\phi} R ϕ commutes with Ω \boldsymbol{\Omega} Ω and from P ϑ ( r ) = R ϑ ( r ) P 0 R − ϑ ( r ) P_{\vartheta(r)}=R_{\vartheta(r)}P_0R_{-\vartheta(r)} P ϑ ( r ) = R ϑ ( r ) P 0 R − ϑ ( r ) . The proof is completed by noting that
lim r → ∞ M ( r ) r = e − S / 2 ⋅ R ϑ ( ∞ ) ⋅ exp [ ϑ ( ∞ ) ⋅ ( S π P 0 − Ω ) ] = − e − S / 2 ⋅ exp ( S π − π 0 ) . \lim_{r\to\infty}\frac{M(r)}{r}=e^{-S/2}\cdot R_{\vartheta(\infty)}\cdot\exp\left[\vartheta(\infty)\cdot\left(\frac{S}{\pi}P_0-\boldsymbol{\Omega}\right)\right]=-e^{-S/2}\cdot\exp\begin{pmatrix}S&\pi\\-\pi&0\end{pmatrix}. r → ∞ lim r M ( r ) = e − S /2 ⋅ R ϑ ( ∞ ) ⋅ exp [ ϑ ( ∞ ) ⋅ ( π S P 0 − Ω ) ] = − e − S /2 ⋅ exp ( S − π π 0 ) .
because ϑ ( ∞ ) = π \vartheta(\infty)=\pi ϑ ( ∞ ) = π and R π = − I 2 R_{\pi}=-I_2 R π = − I 2 . □
We will also need the following perturbation result for the endpoint map. Let Z 1 β , W Z_1^{\beta,W} Z 1 β , W be the flow generated by the ODE
d d t Z t β , W ( x ) = v t β ( Z t β , W ( x ) ) , Z 0 β , W ( x ) = x . \frac{d}{dt}Z_t^{\beta,W}(x)=v_t^\beta\left(Z_t^{\beta,W}(x)\right),\qquad Z_0^{\beta,W}(x)=x. d t d Z t β , W ( x ) = v t β ( Z t β , W ( x ) ) , Z 0 β , W ( x ) = x .
where
v t β ( x ) : = E ( X , Y ~ ) ∼ N ( 0 , I 3 ) ⊗ p 1 β [ Y ~ − X ∣ t Y ~ + ( 1 − t ) X = x ] . v_t^\beta(x):=\mathbb{E}_{(X,\widetilde{Y})\sim N(0,I_3)\otimes p_1^\beta}\left[\widetilde{Y}-X\mid t\widetilde{Y}+(1-t)X=x\right]. v t β ( x ) := E ( X , Y ) ∼ N ( 0 , I 3 ) ⊗ p 1 β [ Y − X ∣ t Y + ( 1 − t ) X = x ] .
The next lemma asserts that bounded perturbation of the potential function results in bounded perturbation of the endpoint map.
Lemma D.3. Let W W W be a function on R 3 \mathbb{R}^3 R 3 of the form
W ( y ) = 1 2 y ⊤ Q y + U ( y ) , y ∈ R 3 , W(y)=\frac{1}{2}y^\top Qy+U(y),\qquad y\in\mathbb{R}^3, W ( y ) = 2 1 y ⊤ Q y + U ( y ) , y ∈ R 3 , with
∥ U ∥ ∞ < ∞ , Q ∈ S + + 3 and q − = λ min ( Q ) . \lVert U\rVert_{\infty}<\infty,\qquad Q\in\mathbb{S}_{++}^3\qquad\text{and}\qquad q_-=\lambda_{\min}(Q). ∥ U ∥ ∞ < ∞ , Q ∈ S ++ 3 and q − = λ m i n ( Q ) . Assume also that W W W satisfies (I) and (II) in Definition 3. Then, for each fixed β ≥ 1 \beta\ge1 β ≥ 1 ,
Z 1 β , W ( x ) = ( β Q ) − 1 / 2 x + R β ( x ) and sup x ∈ R 3 ∥ R β ( x ) ∥ < ∞ . (50) Z_1^{\beta,W}(x)=(\beta Q)^{-1/2}x+R_\beta(x)\qquad\text{and}\qquad\sup_{x\in\mathbb{R}^3}\lVert R_\beta(x)\rVert<\infty. \tag*{(50)} Z 1 β , W ( x ) = ( β Q ) − 1/2 x + R β ( x ) and x ∈ R 3 sup ∥ R β ( x )∥ < ∞. (50)
Proof of Lemma D.3. The idea of the proof is to compare the end-point maps obtained for the Gaussian distribution N ( 0 , ( β Q ) − 1 ) N(0,(\beta Q)^{-1}) N ( 0 , ( β Q ) − 1 ) and the distribution p 1 β ( y ) ∝ exp ( − β W ( y ) ) p_1^\beta(y)\propto\exp(-\beta W(y)) p 1 β ( y ) ∝ exp ( − β W ( y )) . When X , Y ~ ∼ N ( 0 , I 3 ) ⊗ p 1 β \mathbf{X},\widetilde{\mathbf{Y}} \sim N(\mathbf{0},I_3)\otimes p_1^\beta X , Y ∼ N ( 0 , I 3 ) ⊗ p 1 β and t < 1 t<1 t < 1 , the posterior distribution of Y ~ \widetilde{\mathbf{Y}} Y given t Y ~ + ( 1 − t ) X = x t\widetilde{\mathbf{Y}}+(1-t)\mathbf{X}=\mathbf{x} t Y + ( 1 − t ) X = x takes the form
π t , x ( d y ) ∝ exp { − β W ( y ) − ∥ x − t y ∥ 2 2 ( 1 − t ) 2 } d y \pi_{t,\mathbf{x}}(d\mathbf{y}) \propto\exp\left\{-\beta W(\mathbf{y})-\frac{\lVert\mathbf{x}-t\mathbf{y}\rVert^2}{2(1-t)^2}\right\}d\mathbf{y} π t , x ( d y ) ∝ exp { − β W ( y ) − 2 ( 1 − t ) 2 ∥ x − t y ∥ 2 } d y
∝ e − β U ( y ) ⋅ exp { − β 2 y T Q y − ∥ x − t y ∥ 2 2 ( 1 − t ) 2 } d y . \propto e^{-\beta U(\mathbf{y})}\cdot\exp\left\{-\frac{\beta}{2}\mathbf{y}^{\mathsf{T}}Q\mathbf{y}-\frac{\lVert\mathbf{x}-t\mathbf{y}\rVert^2}{2(1-t)^2}\right\}d\mathbf{y}. ∝ e − β U ( y ) ⋅ exp { − 2 β y T Q y − 2 ( 1 − t ) 2 ∥ x − t y ∥ 2 } d y .
Using this posterior, we can conveniently express v t β v_t^\beta v t β as
v t β ( x ) = 1 1 − t E Y ∼ π t , x [ Y ] − x 1 − t , 0 ≤ t < 1. v_t^\beta(\mathbf{x})=\frac{1}{1-t}\mathbb{E}_{\mathbf{Y}\sim\pi_{t,\mathbf{x}}}[\mathbf{Y}]-\frac{\mathbf{x}}{1-t},\qquad0\leq t<1. v t β ( x ) = 1 − t 1 E Y ∼ π t , x [ Y ] − 1 − t x , 0 ≤ t < 1.
Next, let Y ∼ N ( 0 , ( β Q ) − 1 ) \mathbf{Y}\sim N\left(\mathbf{0},(\beta Q)^{-1}\right) Y ∼ N ( 0 , ( β Q ) − 1 ) and let Z t G ∈ C 1 ( [ 0 , 1 ] , R 3 ) Z_t^G\in C^1([0,1],\mathbb{R}^3) Z t G ∈ C 1 ([ 0 , 1 ] , R 3 ) be the flow generated by the ODE
d d t Z t G ( x ) = v t G ( Z t G ( x ) ) , Z 0 G ( x ) = x , \frac{d}{dt}Z_t^G(\mathbf{x})=v_t^G\left(Z_t^G(\mathbf{x})\right),\qquad Z_0^G(\mathbf{x})=\mathbf{x}, d t d Z t G ( x ) = v t G ( Z t G ( x ) ) , Z 0 G ( x ) = x ,
where
v t G ( x ) : = E ( X , Y ) ∼ N ( 0 , I 3 ) ⊗ N ( 0 , ( β Q ) − 1 ) [ Y ∣ t Y + ( 1 − t ) X = x ] = E [ Y ∣ t Y + ( 1 − t ) X = x ] − E [ X ∣ t Y + ( 1 − t ) X = x ] = t β Q − 1 ⋅ [ ( 1 − t ) 2 I 3 + t 2 β Q − 1 ] − 1 ⋅ x − ( 1 − t ) [ ( 1 − t ) 2 I 3 + t 2 β Q − 1 ] − 1 ⋅ x = [ t β ⋅ Q − 1 − ( 1 − t ) I 3 ] ⋅ [ ( 1 − t ) 2 I 3 + t 2 β Q − 1 ] − 1 x . \begin{aligned}
v_t^G(\mathbf{x}):=\mathbb{E}_{(\mathbf{X},\mathbf{Y})\sim N(\mathbf{0},I_3)\otimes N(\mathbf{0},(\beta Q)^{-1})}\left[\mathbf{Y}\mid t\mathbf{Y}+(1-t)\mathbf{X}=\mathbf{x}\right] \\
&=\mathbb{E}\left[\mathbf{Y}\mid t\mathbf{Y}+(1-t)\mathbf{X}=\mathbf{x}\right]-\mathbb{E}\left[\mathbf{X}\mid t\mathbf{Y}+(1-t)\mathbf{X}=\mathbf{x}\right] \\
&=\frac{t}{\beta}Q^{-1}\cdot\left[(1-t)^2I_3+\frac{t^2}{\beta}Q^{-1}\right]^{-1}\cdot\mathbf{x}-(1-t)\left[(1-t)^2I_3+\frac{t^2}{\beta}Q^{-1}\right]^{-1}\cdot\mathbf{x} \\
&=\left[\frac{t}{\beta}\cdot Q^{-1}-(1-t)I_3\right]\cdot\left[(1-t)^2I_3+\frac{t^2}{\beta}Q^{-1}\right]^{-1}\mathbf{x}.
\end{aligned} v t G ( x ) := E ( X , Y ) ∼ N ( 0 , I 3 ) ⊗ N ( 0 , ( β Q ) − 1 ) [ Y ∣ t Y + ( 1 − t ) X = x ] = E [ Y ∣ t Y + ( 1 − t ) X = x ] − E [ X ∣ t Y + ( 1 − t ) X = x ] = β t Q − 1 ⋅ [ ( 1 − t ) 2 I 3 + β t 2 Q − 1 ] − 1 ⋅ x − ( 1 − t ) [ ( 1 − t ) 2 I 3 + β t 2 Q − 1 ] − 1 ⋅ x = [ β t ⋅ Q − 1 − ( 1 − t ) I 3 ] ⋅ [ ( 1 − t ) 2 I 3 + β t 2 Q − 1 ] − 1 x .
Above, the third line follows from the Gaussian conditional mean formula
E [ Y ∣ t Y + ( 1 − t ) X = x ] = Cov [ Y , t Y + ( 1 − t ) X ] ⋅ { Cov [ t Y + ( 1 − t ) X ] } − 1 ⋅ x = t β Q − 1 ⋅ [ ( 1 − t ) 2 I 3 + t 2 β Q − 1 ] − 1 ⋅ x . \begin{aligned}
\mathbb{E}\left[\mathbf{Y}\mid t\mathbf{Y}+(1-t)\mathbf{X}=\mathbf{x}\right]=\operatorname{Cov}\left[\mathbf{Y},t\mathbf{Y}+(1-t)\mathbf{X}\right]\cdot\left\{\operatorname{Cov}\left[t\mathbf{Y}+(1-t)\mathbf{X}\right]\right\}^{-1}\cdot\mathbf{x} \\
&=\frac{t}{\beta}Q^{-1}\cdot\left[(1-t)^2I_3+\frac{t^2}{\beta}Q^{-1}\right]^{-1}\cdot\mathbf{x}.
\end{aligned} E [ Y ∣ t Y + ( 1 − t ) X = x ] = Cov [ Y , t Y + ( 1 − t ) X ] ⋅ { Cov [ t Y + ( 1 − t ) X ] } − 1 ⋅ x = β t Q − 1 ⋅ [ ( 1 − t ) 2 I 3 + β t 2 Q − 1 ] − 1 ⋅ x .
and
E [ X ∣ t Y + ( 1 − t ) X = x ] = Cov [ X , t Y + ( 1 − t ) X ] ⋅ { Cov [ t Y + ( 1 − t ) X ] } − 1 ⋅ x = ( 1 − t ) I 3 ⋅ [ ( 1 − t ) 2 I 3 + t 2 β Q − 1 ] − 1 ⋅ x = ( 1 − t ) ⋅ [ ( 1 − t ) 2 I 3 + t 2 β Q − 1 ] − 1 ⋅ x . \begin{aligned}
\mathbb{E}\left[\mathbf{X}\mid t\mathbf{Y}+(1-t)\mathbf{X}=\mathbf{x}\right]=\operatorname{Cov}\left[\mathbf{X},t\mathbf{Y}+(1-t)\mathbf{X}\right]\cdot\left\{\operatorname{Cov}\left[t\mathbf{Y}+(1-t)\mathbf{X}\right]\right\}^{-1}\cdot\mathbf{x} \\
&=(1-t)I_3\cdot\left[(1-t)^2I_3+\frac{t^2}{\beta}Q^{-1}\right]^{-1}\cdot\mathbf{x} \\
&=(1-t)\cdot\left[(1-t)^2I_3+\frac{t^2}{\beta}Q^{-1}\right]^{-1}\cdot\mathbf{x}.
\end{aligned} E [ X ∣ t Y + ( 1 − t ) X = x ] = Cov [ X , t Y + ( 1 − t ) X ] ⋅ { Cov [ t Y + ( 1 − t ) X ] } − 1 ⋅ x = ( 1 − t ) I 3 ⋅ [ ( 1 − t ) 2 I 3 + β t 2 Q − 1 ] − 1 ⋅ x = ( 1 − t ) ⋅ [ ( 1 − t ) 2 I 3 + β t 2 Q − 1 ] − 1 ⋅ x .
In particular, for some L β , Q < ∞ L_{\beta,Q}<\infty L β , Q < ∞ , we have
sup t ∈ [ 0 , 1 ] ∥ v t G ( x ) − v t G ( y ) ∥ ≤ L β , Q ⋅ ∥ x − y ∥ \sup_{t\in[0,1]}\left\lVert v_t^G(\mathbf{x})-v_t^G(\mathbf{y})\right\rVert\leq L_{\beta,Q}\cdot\lVert\mathbf{x}-\mathbf{y}\rVert t ∈ [ 0 , 1 ] sup v t G ( x ) − v t G ( y ) ≤ L β , Q ⋅ ∥ x − y ∥
for all x , y ∈ R 3 \mathbf{x},\mathbf{y}\in\mathbb{R}^3 x , y ∈ R 3 .
For X , Y ∼ N ( 0 , I 3 ) ⊗ N ( 0 , ( β Q ) − 1 ) \mathbf{X},\mathbf{Y} \sim N(0,I_3)\otimes N(0,(\beta Q)^{-1}) X , Y ∼ N ( 0 , I 3 ) ⊗ N ( 0 , ( β Q ) − 1 ) and t < 1 t<1 t < 1 , denote by π ~ t , x \widetilde{\pi}_{t,x} π t , x the posterior distribution of Y \mathbf{Y} Y given t Y + ( 1 − t ) X = x t\mathbf{Y}+(1-t)\mathbf{X}=x t Y + ( 1 − t ) X = x , which is given by
π ~ t , x ( y ) ∝ exp { − β 2 y ⊤ Q y − ∥ x − t y ∥ 2 2 ( 1 − t ) 2 } d y ∼ N ( μ t , x , A t , x − 1 ) . \widetilde{\pi}_{t,x}(\mathbf{y})\propto\exp\left\{-\frac{\beta}{2}\mathbf{y}^{\top}Q\mathbf{y}-\frac{\|x-t\mathbf{y}\|^2}{2(1-t)^2}\right\}d\mathbf{y}\sim N\left(\mu_{t,x},A_{t,x}^{-1}\right). π t , x ( y ) ∝ exp { − 2 β y ⊤ Q y − 2 ( 1 − t ) 2 ∥ x − t y ∥ 2 } d y ∼ N ( μ t , x , A t , x − 1 ) .
where
A t , x : = β Q + t 2 ( 1 − t ) 2 I 3 , μ t , x = ( A t , x ) − 1 ⋅ t x ( 1 − t ) 2 , A_{t,x}:=\beta Q+\frac{t^2}{(1-t)^2}I_3,\qquad\mu_{t,x}=(A_{t,x})^{-1}\cdot\frac{tx}{(1-t)^2}, A t , x := β Q + ( 1 − t ) 2 t 2 I 3 , μ t , x = ( A t , x ) − 1 ⋅ ( 1 − t ) 2 t x ,
using the explicit formulae for the conditional distribution of a sub-vector of a Gaussian vector. Similarly, for all 0 ≤ t < 1 0\le t<1 0 ≤ t < 1 ,
v t G ( x ) = 1 1 − t E Y ∼ π ~ t , x [ Y ] − x 1 − t . v_t^G(x)=\frac{1}{1-t}\mathbb{E}_{Y\sim\widetilde{\pi}_{t,x}}[Y]-\frac{x}{1-t}. v t G ( x ) = 1 − t 1 E Y ∼ π t , x [ Y ] − 1 − t x .
From the form of the posterior distributions, we obtain that their Radon-Nikodym derivative at any y ∈ R 3 y\in\mathbb{R}^3 y ∈ R 3 is
d π t , x d π ~ t , x ( y ) = e − β U ( y ) E π ~ t , x [ e − β U ( Y ) ] ≤ e 2 β ∥ U ∥ ∞ . \frac{d\pi_{t,x}}{d\widetilde{\pi}_{t,x}}(y)=\frac{e^{-\beta U(y)}}{\mathbb{E}_{\widetilde{\pi}_{t,x}}[e^{-\beta U(Y)}]}\le e^{2\beta\|U\|_{\infty}}. d π t , x d π t , x ( y ) = E π t , x [ e − β U ( Y ) ] e − β U ( y ) ≤ e 2 β ∥ U ∥ ∞ .
Putting all the pieces together, for all t < 1 t<1 t < 1 and x ∈ R 3 x\in\mathbb{R}^3 x ∈ R 3 ,
∥ v t β ( x ) − v t G ( x ) ∥ = 1 1 − t ⋅ ∥ E Y ∼ π t , x [ Y ] − E Y ∼ π ~ t , x [ Y ] ∥ ≤ e 2 β ∥ U ∥ ∞ + 1 1 − t ⋅ E Y ∼ π ~ t , x [ ∥ Y − E Y ∼ π ~ t , x [ Y ] ∥ ] ≤ 2 e 2 β ∥ U ∥ ∞ 1 − t ⋅ tr ( A t , x − 1 ) ≤ 2 e 2 β ∥ U ∥ ∞ 1 − t ⋅ 3 β q − + t 2 ( 1 − t ) − 2 ≤ 6 e 2 β ∥ U ∥ ∞ ⋅ 1 + 1 β q − = : ψ ( β , q − ) , \begin{aligned}
\left\|v_t^\beta(x)-v_t^G(x)\right\|
&=\frac{1}{1-t}\cdot\left\|\mathbb{E}_{Y\sim\pi_{t,x}}[Y]-\mathbb{E}_{Y\sim\widetilde{\pi}_{t,x}}[Y]\right\|\\
&\le\frac{e^{2\beta\|U\|_{\infty}}+1}{1-t}\cdot\mathbb{E}_{Y\sim\widetilde{\pi}_{t,x}}\left[\left\|Y-\mathbb{E}_{Y\sim\widetilde{\pi}_{t,x}}[Y]\right\|\right]\\
&\le\frac{2e^{2\beta\|U\|_{\infty}}}{1-t}\cdot\sqrt{\operatorname{tr}\left(A_{t,x}^{-1}\right)}\\
&\le\frac{2e^{2\beta\|U\|_{\infty}}}{1-t}\cdot\frac{\sqrt{3}}{\sqrt{\beta q_-+t^2(1-t)^{-2}}}\le6e^{2\beta\|U\|_{\infty}}\cdot\sqrt{1+\frac{1}{\beta q_-}}=:\psi(\beta,q_-),
\end{aligned} v t β ( x ) − v t G ( x ) = 1 − t 1 ⋅ E Y ∼ π t , x [ Y ] − E Y ∼ π t , x [ Y ] ≤ 1 − t e 2 β ∥ U ∥ ∞ + 1 ⋅ E Y ∼ π t , x [ Y − E Y ∼ π t , x [ Y ] ] ≤ 1 − t 2 e 2 β ∥ U ∥ ∞ ⋅ tr ( A t , x − 1 ) ≤ 1 − t 2 e 2 β ∥ U ∥ ∞ ⋅ β q − + t 2 ( 1 − t ) − 2 3 ≤ 6 e 2 β ∥ U ∥ ∞ ⋅ 1 + β q − 1 =: ψ ( β , q − ) ,
where q − = λ min ( Q ) q_-=\lambda_{\min}(Q) q − = λ m i n ( Q ) . The second inequality follows from Jensen’s inequality, while the last follows from the fact that
A t , x ⪰ [ β q − + t 2 ( 1 − t ) 2 ] ⋅ I 3 A_{t,x}\succeq\left[\beta q_-+\frac{t^2}{(1-t)^2}\right]\cdot I_3 A t , x ⪰ [ β q − + ( 1 − t ) 2 t 2 ] ⋅ I 3
or, equivalently,
A t , x − 1 ⪯ [ β q − + t 2 ( 1 − t ) 2 ] − 1 ⋅ I 3 , A_{t,x}^{-1}\preceq\left[\beta q_-+\frac{t^2}{(1-t)^2}\right]^{-1}\cdot I_3, A t , x − 1 ⪯ [ β q − + ( 1 − t ) 2 t 2 ] − 1 ⋅ I 3 ,
which implies
tr ( A t , x − 1 ) ≤ 3 β q − + t 2 ⋅ ( 1 − t ) − 2 . \operatorname{tr}\left(A_{t,x}^{-1}\right)\le\frac{3}{\beta q_-+t^2\cdot(1-t)^{-2}}. tr ( A t , x − 1 ) ≤ β q − + t 2 ⋅ ( 1 − t ) − 2 3 .
To finish the proof, put E t : = ∥ Z t β , W ( x ) − Z t G ( x ) ∥ E_t := \left\|Z_t^{\beta,W}(x)-Z_t^G(x)\right\| E t := Z t β , W ( x ) − Z t G ( x ) and write
E t = ∥ Z t β , W ( x ) − x − [ Z t G ( x ) − x ] ∥ = ∥ ∫ 0 t v s β ( Z s β , W ) d s − ∫ 0 t v s G ( Z s G ) d s ∥ ≤ ∫ 0 t ∥ v s β ( Z s β , W ) − v s G ( Z s β , W ) ∥ d s + ∫ 0 t ∥ v s G ( Z s β , W ) − v s G ( Z s G ) ∥ d s ≤ ψ ( β , q − ) + L β , Q ∫ 0 t E s d s \begin{aligned}
E_t
&=\left\|Z_t^{\beta,W}(x)-x-\left[Z_t^G(x)-x\right]\right\|\\
&=\left\|\int_0^t v_s^{\beta}\left(Z_s^{\beta,W}\right)\,ds-\int_0^t v_s^G\left(Z_s^G\right)\,ds\right\|\\
&\leq\int_0^t\left\|v_s^{\beta}\left(Z_s^{\beta,W}\right)-v_s^G\left(Z_s^{\beta,W}\right)\right\|\,ds+\int_0^t\left\|v_s^G\left(Z_s^{\beta,W}\right)-v_s^G\left(Z_s^G\right)\right\|\,ds\\
&\leq\psi\left(\beta,q_-\right)+L_{\beta,Q}\int_0^t E_s\,ds
\end{aligned} E t = Z t β , W ( x ) − x − [ Z t G ( x ) − x ] = ∫ 0 t v s β ( Z s β , W ) d s − ∫ 0 t v s G ( Z s G ) d s ≤ ∫ 0 t v s β ( Z s β , W ) − v s G ( Z s β , W ) d s + ∫ 0 t v s G ( Z s β , W ) − v s G ( Z s G ) d s ≤ ψ ( β , q − ) + L β , Q ∫ 0 t E s d s
Gronwall’s inequality then yields
E t ≤ ψ ( β , q − ) ⋅ e L β , Q , ∀ t ∈ [ 0 , 1 ) . E_t\leq\psi\left(\beta,q_-\right)\cdot e^{L_{\beta,Q}},\qquad\forall t\in[0,1). E t ≤ ψ ( β , q − ) ⋅ e L β , Q , ∀ t ∈ [ 0 , 1 ) .
Letting t ↑ 1 t\uparrow1 t ↑ 1 finishes the proof.
Proof of Theorem D.1
Take S > 2 π S>2\pi S > 2 π and ε ∈ ( 0 , 1 / 10 ) \varepsilon\in(0,1/10) ε ∈ ( 0 , 1/10 ) . We first use Lemma D.2 to construct the 2 × 2 2\times2 2 × 2 block G W ( z ) G_W(z) G W ( z ) in (D.3). For simplicity, we will drop the subscript W W W and simply write G ( z ) G(z) G ( z ) . Define
Θ ( s ) = π S { S + 1 2 ⋅ log [ s + e − S ( 1 − s ) s + e S ( 1 − s ) ] } , − e − S < s < 1 + e − S . \Theta(s)=\frac{\pi}{S}\left\{S+\frac{1}{2}\cdot\log\left[\frac{s+e^{-S}(1-s)}{s+e^S(1-s)}\right]\right\},\qquad-e^{-S}<s<1+e^{-S}. Θ ( s ) = S π { S + 2 1 ⋅ log [ s + e S ( 1 − s ) s + e − S ( 1 − s ) ] } , − e − S < s < 1 + e − S .
Take a smooth, compactly supported function Θ ~ ( s ) \widetilde{\Theta}(s) Θ ( s ) that agrees with Θ ( s ) \Theta(s) Θ ( s ) in a neighborhood of [ 0 , 1 ] [0,1] [ 0 , 1 ] and define
G ( z ) : = e − S ⋅ P Θ ~ ( e z 2 ) + e S ⋅ ( I 2 − P Θ ~ ( e z 2 ) ) (51) G(z):=e^{-S}\cdot P_{\widetilde{\Theta}(e^{z^2})}+e^S\cdot\left(I_2-P_{\widetilde{\Theta}(e^{z^2})}\right) \tag*{(51)} G ( z ) := e − S ⋅ P Θ ( e z 2 ) + e S ⋅ ( I 2 − P Θ ( e z 2 ) ) (51)
where P ϕ P_\phi P ϕ is the rank-one map defined in (D.8). In other words, with ζ ( r ) \zeta(r) ζ ( r ) as in (D.1), our choice G ( z ) G(z) G ( z ) gives
G ( ζ ( r ) ) = H ( r ) (52) G(\zeta(r))=H(r) \tag*{(52)} G ( ζ ( r )) = H ( r ) (52)
where H ( r ) H(r) H ( r ) is the matrix-valued function constructed in Lemma D.2, see (D.9). Note that this construction gives spec G ( z ) = { e − S , e S } \operatorname{spec}G(z)=\{e^{-S},e^S\} spec G ( z ) = { e − S , e S } for all z ∈ R z\in\mathbb{R} z ∈ R .
The reason we choose Θ ~ ( s ) \widetilde{\Theta}(s) Θ ( s ) instead of Θ ( s ) \Theta(s) Θ ( s ) is to extend G ( z ) G(z) G ( z ) smoothly to all z ∈ R z\in\mathbb{R} z ∈ R . Note that the construction yields spec G ( z ) = { e − S , e S } \operatorname{spec}G(z)=\{e^{-S},e^S\} spec G ( z ) = { e − S , e S } for all z ∈ R z\in\mathbb{R} z ∈ R , and thus
e − S ⋅ I 2 ⪯ G ( z ) ⪯ e S ⋅ I 2 , ∀ z ∈ R . (53) e^{-S}\cdot I_2\preceq G(z)\preceq e^S\cdot I_2,\qquad\forall z\in\mathbb{R}. \tag*{(53)} e − S ⋅ I 2 ⪯ G ( z ) ⪯ e S ⋅ I 2 , ∀ z ∈ R . (53)
For x ∈ R 2 x\in\mathbb{R}^2 x ∈ R 2 and α , ε ~ , ε ~ 1 > 0 \alpha,\widetilde{\varepsilon},\widetilde{\varepsilon}_1>0 α , ε , ε 1 > 0 to be chosen later, define
W 0 ( x , z ) : = z 2 2 + 1 2 x ⊤ G ( z ) x , Q ( x , z ) : = ( 1 + ε ~ ) ⋅ e S ⋅ ∥ x ∥ 2 2 + z 2 2 − α , W_0(x,z):=\frac{z^2}{2}+\frac{1}{2}x^\top G(z)x,\qquad Q(x,z):=(1+\widetilde{\varepsilon})\cdot e^S\cdot\frac{\|x\|^2}{2}+\frac{z^2}{2}-\alpha, W 0 ( x , z ) := 2 z 2 + 2 1 x ⊤ G ( z ) x , Q ( x , z ) := ( 1 + ε ) ⋅ e S ⋅ 2 ∥ x ∥ 2 + 2 z 2 − α ,
and
Δ ( x , z ) : = W 0 ( x , z ) − Q ( x , z ) = α − 1 2 x ⊤ [ ( 1 + ε ~ ) ⋅ e S ⋅ I 2 − G ( z ) ] x . \Delta(x,z):=W_0(x,z)-Q(x,z)=\alpha-\frac{1}{2}x^\top\left[(1+\widetilde{\varepsilon})\cdot e^S\cdot I_2-G(z)\right]x. Δ ( x , z ) := W 0 ( x , z ) − Q ( x , z ) = α − 2 1 x ⊤ [ ( 1 + ε ) ⋅ e S ⋅ I 2 − G ( z ) ] x .
We will take W W W to be of the form
W = 1 2 [ W 0 + Q + χ ( Δ ) ] . W = \frac{1}{2}\left[W_0+Q+\chi(\Delta)\right]. W = 2 1 [ W 0 + Q + χ ( Δ ) ] .
where χ \chi χ is the function from Lemma D.1, applied with parameters α \alpha α and β = ε ~ e 1 \beta=\tilde{\varepsilon}e_1 β = ε ~ e 1 .
Before going further, let us explain why W W W is chosen to be of this form. Firstly, W 0 W_0 W 0 satisfies (III), but is not necessarily convex globally because G ( z ) G(z) G ( z ) varies with z z z . Fortunately, it is strongly convex with condition number roughly equal to e 2 S e^{2S} e 2 S in a small cylinder { ( x , z ) : ∥ x ∥ ≤ ς } \{(x,z):\|x\|\leq\varsigma\} {( x , z ) : ∥ x ∥ ≤ ς } , for some small ς > 0 \varsigma>0 ς > 0 , due to (D.14) and the fact that
∇ 2 W 0 ( x , z ) = ( G ( z ) G ′ ( z ) x x T G ′ ( z ) 1 + 1 2 x T G ′ ′ ( z ) x ) = ( G ( z ) 0 2 × 1 0 1 × 2 1 ) + O ( ∥ x ∥ + ∥ x ∥ 2 ) . (54) \nabla^2 W_0(x,z)=\begin{pmatrix}G(z)&G'(z)x\\x^{\mathsf T}G'(z)&1+\frac{1}{2}x^{\mathsf T}G''(z)x\end{pmatrix}
=\begin{pmatrix}G(z)&0_{2\times1}\\0_{1\times2}&1\end{pmatrix}+O\left(\|x\|+\|x\|^2\right). \tag*{(54)} ∇ 2 W 0 ( x , z ) = ( G ( z ) x T G ′ ( z ) G ′ ( z ) x 1 + 2 1 x T G ′′ ( z ) x ) = ( G ( z ) 0 1 × 2 0 2 × 1 1 ) + O ( ∥ x ∥ + ∥ x ∥ 2 ) . (54)
Therefore, we will let W W W equal W 0 W_0 W 0 only in the cylinder where it can be shown to be strongly convex, and let W W W equal Q Q Q outside a larger neighborhood. Generally speaking, W W W glues W 0 W_0 W 0 and Q Q Q together in such a way that (I)–(III) are preserved and the condition number of W 0 W_0 W 0 is only slightly perturbed (these properties are justified below).
We visualize this gluing construction of W W W in Figure S1 below. With δ ± = δ ± ( α , ε ~ 1 ) \delta_{\pm}=\delta_{\pm}(\alpha,\tilde{\varepsilon}_1) δ ± = δ ± ( α , ε ~ 1 ) as in Lemma D.1 with δ + < α < δ − \delta_+<\alpha<\delta_- δ + < α < δ − , we have
Figure S1. Geometry of the gluing construction.
W = { W 0 , if Δ ≥ δ + , Q , if Δ ≤ − δ − . W=\begin{cases}
W_0,&\text{if }\Delta\geq\delta_+,\\
Q,&\text{if }\Delta\leq-\delta_-.
\end{cases} W = { W 0 , Q , if Δ ≥ δ + , if Δ ≤ − δ − .
The region { ( x , z ) : Δ ≥ δ + } \{(x,z):\Delta\geq\delta_+\} {( x , z ) : Δ ≥ δ + } is the green twisted elliptical tube depicted in Figure S1 . In the region { ( x , z ) : − δ − ≤ Δ ( ( x , z ) ) ≤ δ + } \{(x,z):-\delta_-\leq\Delta((x,z))\leq\delta_+\} {( x , z ) : − δ − ≤ Δ (( x , z )) ≤ δ + } , W W W interpolates smoothly between W 0 W_0 W 0 and Q Q Q and is depicted as the complement of the green region in the cylinder. Put
D ( z ) : = ( 1 + ε ~ ) ⋅ e S ⋅ I 2 − G ( z ) D(z):=(1+\tilde{\varepsilon})\cdot e^S\cdot I_2-G(z) D ( z ) := ( 1 + ε ~ ) ⋅ e S ⋅ I 2 − G ( z )
and note that for all z ∈ R z\in\mathbb{R} z ∈ R ,
ε ~ e S ⋅ I 2 ⪯ D ( z ) ⪯ [ ( 1 + ε ~ ) ⋅ e S − e − S ] ⋅ I 2 . (55) \tilde{\varepsilon}e^S\cdot I_2\preceq D(z)\preceq\left[(1+\tilde{\varepsilon})\cdot e^S-e^{-S}\right]\cdot I_2. \tag*{(55)} ε ~ e S ⋅ I 2 ⪯ D ( z ) ⪯ [ ( 1 + ε ~ ) ⋅ e S − e − S ] ⋅ I 2 . (55)
To finish the proof, we check that W W W satisfies (I)–(IV) with a condition number satisfying (D.4).
Checking (IV). G W ( z ) G_W(z) G W ( z ) in (D.3) has the form as in (D.12) by our construction, so (IV) follows from (D.13) and Lemma D.2.
Checking (III). This is obvious because W = W 0 W=W_0 W = W 0 along a cylinder such that its interior contains the z z z -axis, which yields
∇ W ( 0 , z ) = ∇ W 0 ( 0 , z ) = ( 0 , z ) . \nabla W(0,z)=\nabla W_0(0,z)=(0,z). ∇ W ( 0 , z ) = ∇ W 0 ( 0 , z ) = ( 0 , z ) .
Checking (II). Take Q = diag { ( 1 + ε ~ ) ⋅ e S ; ( 1 + ε ~ ) ⋅ e S ; 1 } Q=\operatorname{diag}\{(1+\tilde{\varepsilon})\cdot e^S;(1+\tilde{\varepsilon})\cdot e^S;1\} Q = diag {( 1 + ε ~ ) ⋅ e S ; ( 1 + ε ~ ) ⋅ e S ; 1 } . Writing y = ( x , z ) ∈ R 3 y=(x,z)\in\mathbb{R}^3 y = ( x , z ) ∈ R 3 , we have
Q ( x , z ) = 1 2 y T Q y − α Q(x,z)=\frac{1}{2}y^{\mathsf T}Qy-\alpha Q ( x , z ) = 2 1 y T Q y − α
for all y ∈ R 3 y\in\mathbb{R}^3 y ∈ R 3 . We will verify that
sup y ∈ R 3 ∣ W ( y ) − 1 2 y T Q y ∣ < ∞ . \sup_{y\in\mathbb{R}^3}\left|W(y)-\frac{1}{2}y^{\mathsf T}Qy\right|<\infty. y ∈ R 3 sup W ( y ) − 2 1 y T Q y < ∞.
In the region { y : Δ ( y ) ≤ − δ − } \{y:\Delta(y)\leq-\delta_-\} { y : Δ ( y ) ≤ − δ − } , we have W ( y ) = Q ( y ) W(y)=Q(y) W ( y ) = Q ( y ) , so
sup y ∈ { y : Δ ≤ − δ − } ∣ W ( y ) − 1 2 y T Q y ∣ = α . \sup_{y\in\{y:\Delta\leq-\delta_-\}}\left|W(y)-\frac{1}{2}y^{\mathsf T}Qy\right|=\alpha. y ∈ { y : Δ ≤ − δ − } sup W ( y ) − 2 1 y T Q y = α .
In the region { y : Δ ( y ) ≥ δ + } \{y:\Delta(y)\ge\delta_{+}\} { y : Δ ( y ) ≥ δ + } , we have
0 ≤ x T D ( z ) x ≤ 2 ( α − δ + ) 0\le x^{\mathsf{T}}D(z)x\le2(\alpha-\delta_{+}) 0 ≤ x T D ( z ) x ≤ 2 ( α − δ + )
Thus
sup y ∈ { y : Δ ≥ δ + } ∣ W ( y ) − 1 2 y T Q y ∣ ≤ α + sup y ∈ { y : Δ ≥ δ + } x T D ( z ) x ≤ α + 2 ( α − δ + ) . \sup_{y\in\{y:\Delta\ge\delta_{+}\}}\left|W(y)-\frac{1}{2}y^{\mathsf{T}}Qy\right|\le\alpha+\sup_{y\in\{y:\Delta\ge\delta_{+}\}}x^{\mathsf{T}}D(z)x\le\alpha+2(\alpha-\delta_{+}). y ∈ { y : Δ ≥ δ + } sup W ( y ) − 2 1 y T Q y ≤ α + y ∈ { y : Δ ≥ δ + } sup x T D ( z ) x ≤ α + 2 ( α − δ + ) .
Finally, in the region { y : − δ − ≤ Δ ≤ δ + } \{y:-\delta_{-}\le\Delta\le\delta_{+}\} { y : − δ − ≤ Δ ≤ δ + } , we have
2 ( α + δ − ) ≥ x T D ( z ) x ≥ 2 ( α − δ + ) . 2(\alpha+\delta_{-})\ge x^{\mathsf{T}}D(z)x\ge2(\alpha-\delta_{+}). 2 ( α + δ − ) ≥ x T D ( z ) x ≥ 2 ( α − δ + ) .
Consequently,
sup y ∈ { y : − δ − ≤ Δ ≤ δ + } ∣ W ( y ) − 1 2 y T Q y ∣ = sup y ∈ { y : − δ − ≤ Δ ≤ δ + } ∣ 1 2 ( W 0 + Q + χ ( Δ ) ) − Q − α ∣ ≤ 1 2 sup y ∈ { y : − δ − ≤ Δ ≤ δ + } ∣ W 0 ( y ) − Q ( y ) ∣ + α + 1 2 sup − δ − ≤ t ≤ δ + ∣ χ ( t ) ∣ ≤ 1 2 sup y ∈ { y : − δ − ≤ Δ ≤ δ + } ∣ x T D ( z ) x ∣ + 3 α 2 + 1 2 sup − δ − ≤ t ≤ δ + ∣ χ ( t ) ∣ ≤ α + δ − + 3 α 2 + 1 2 sup − δ − ≤ t ≤ δ + ∣ χ ( t ) ∣ < ∞ . \begin{aligned}
\sup_{y\in\{y:-\delta_{-}\le\Delta\le\delta_{+}\}}\left|W(y)-\frac{1}{2}y^{\mathsf{T}}Qy\right|
&=\sup_{y\in\{y:-\delta_{-}\le\Delta\le\delta_{+}\}}\left|\frac{1}{2}(W_{0}+Q+\chi(\Delta))-Q-\alpha\right|\\
&\le\frac{1}{2}\sup_{y\in\{y:-\delta_{-}\le\Delta\le\delta_{+}\}}|W_{0}(y)-Q(y)|+\alpha+\frac{1}{2}\sup_{-\delta_{-}\le t\le\delta_{+}}|\chi(t)|\\
&\le\frac{1}{2}\sup_{y\in\{y:-\delta_{-}\le\Delta\le\delta_{+}\}}|x^{\mathsf{T}}D(z)x|+\frac{3\alpha}{2}+\frac{1}{2}\sup_{-\delta_{-}\le t\le\delta_{+}}|\chi(t)|\\
&\le\alpha+\delta_{-}+\frac{3\alpha}{2}+\frac{1}{2}\sup_{-\delta_{-}\le t\le\delta_{+}}|\chi(t)|<\infty.
\end{aligned} y ∈ { y : − δ − ≤ Δ ≤ δ + } sup W ( y ) − 2 1 y T Q y = y ∈ { y : − δ − ≤ Δ ≤ δ + } sup 2 1 ( W 0 + Q + χ ( Δ )) − Q − α ≤ 2 1 y ∈ { y : − δ − ≤ Δ ≤ δ + } sup ∣ W 0 ( y ) − Q ( y ) ∣ + α + 2 1 − δ − ≤ t ≤ δ + sup ∣ χ ( t ) ∣ ≤ 2 1 y ∈ { y : − δ − ≤ Δ ≤ δ + } sup ∣ x T D ( z ) x ∣ + 2 3 α + 2 1 − δ − ≤ t ≤ δ + sup ∣ χ ( t ) ∣ ≤ α + δ − + 2 3 α + 2 1 − δ − ≤ t ≤ δ + sup ∣ χ ( t ) ∣ < ∞.
Checking (I). We first note that due to the construction of G \mathbf{G} G and the fact that Θ ~ \widetilde{\Theta} Θ has compact support, we have
max { sup z ∈ R ∥ G ′ ( z ) ∥ o p ; sup z ∈ R ∥ G ′ ′ ( z ) ∥ o p } = C S < ∞ , \max\left\{\sup_{z\in\mathbb{R}}\lVert G'(z)\rVert_{\mathrm{op}};\sup_{z\in\mathbb{R}}\lVert G''(z)\rVert_{\mathrm{op}}\right\}=C_{S}<\infty, max { z ∈ R sup ∥ G ′ ( z ) ∥ op ; z ∈ R sup ∥ G ′′ ( z ) ∥ op } = C S < ∞ ,
where C S C_{S} C S depends only on S S S .
With y = ( x , z ) ∈ R 3 \mathbf{y}=(x,z)\in\mathbb{R}^{3} y = ( x , z ) ∈ R 3 , and omitting for readability the argument y \mathbf{y} y in the functions W 0 W_{0} W 0 , Q Q Q and Δ \Delta Δ ,
∇ 2 W ( y ) = 1 2 ⋅ ∇ 2 [ W 0 ( y ) + Q ( y ) + χ ( W 0 ( y ) − Q ( y ) ) ] = 1 2 ⋅ ∇ [ ∇ W 0 + ∇ Q + χ ′ ( W 0 − Q ) ⋅ ∇ ( W 0 − Q ) ] = 1 2 ⋅ [ ∇ 2 W 0 + ∇ 2 Q + χ ′ ( W 0 − Q ) ⋅ ∇ 2 ( W 0 − Q ) + χ ′ ′ ( W 0 − Q ) ⋅ ∇ ( W 0 − Q ) ⋅ ∇ ( W 0 − Q ) T ] = 1 2 ⋅ { ∇ 2 W 0 + ∇ 2 Q + χ ′ ( Δ ) ⋅ ∇ 2 ( Δ ) + χ ′ ′ ( Δ ) ⋅ [ ∇ Δ ⋅ ( ∇ Δ ) T ] } = 1 + χ ′ ( Δ ) 2 ⋅ ∇ 2 W 0 + 1 − χ ′ ( Δ ) 2 ⋅ ∇ 2 Q + 1 2 χ ′ ′ ( Δ ) ⋅ [ ∇ Δ ⋅ ( ∇ Δ ) T ] = L y + N y , \begin{aligned}
\nabla^{2}W(\mathbf{y})
&=\frac{1}{2}\cdot\nabla^{2}\left[W_{0}(\mathbf{y})+Q(\mathbf{y})+\chi\left(W_{0}(\mathbf{y})-Q(\mathbf{y})\right)\right]\\
&=\frac{1}{2}\cdot\nabla\left[\nabla W_{0}+\nabla Q+\chi'\left(W_{0}-Q\right)\cdot\nabla\left(W_{0}-Q\right)\right]\\
&=\frac{1}{2}\cdot\left[\nabla^{2}W_{0}+\nabla^{2}Q+\chi'\left(W_{0}-Q\right)\cdot\nabla^{2}\left(W_{0}-Q\right)+\chi''\left(W_{0}-Q\right)\cdot\nabla\left(W_{0}-Q\right)\cdot\nabla\left(W_{0}-Q\right)^{\mathsf{T}}\right]\\
&=\frac{1}{2}\cdot\left\{\nabla^{2}W_{0}+\nabla^{2}Q+\chi'(\Delta)\cdot\nabla^{2}(\Delta)+\chi''(\Delta)\cdot\left[\nabla\Delta\cdot(\nabla\Delta)^{\mathsf{T}}\right]\right\}\\
&=\frac{1+\chi'(\Delta)}{2}\cdot\nabla^{2}W_{0}+\frac{1-\chi'(\Delta)}{2}\cdot\nabla^{2}Q+\frac{1}{2}\chi''(\Delta)\cdot\left[\nabla\Delta\cdot(\nabla\Delta)^{\mathsf{T}}\right]\\
&=L_{y}+N_{y},
\end{aligned} ∇ 2 W ( y ) = 2 1 ⋅ ∇ 2 [ W 0 ( y ) + Q ( y ) + χ ( W 0 ( y ) − Q ( y ) ) ] = 2 1 ⋅ ∇ [ ∇ W 0 + ∇ Q + χ ′ ( W 0 − Q ) ⋅ ∇ ( W 0 − Q ) ] = 2 1 ⋅ [ ∇ 2 W 0 + ∇ 2 Q + χ ′ ( W 0 − Q ) ⋅ ∇ 2 ( W 0 − Q ) + χ ′′ ( W 0 − Q ) ⋅ ∇ ( W 0 − Q ) ⋅ ∇ ( W 0 − Q ) T ] = 2 1 ⋅ { ∇ 2 W 0 + ∇ 2 Q + χ ′ ( Δ ) ⋅ ∇ 2 ( Δ ) + χ ′′ ( Δ ) ⋅ [ ∇Δ ⋅ ( ∇Δ ) T ] } = 2 1 + χ ′ ( Δ ) ⋅ ∇ 2 W 0 + 2 1 − χ ′ ( Δ ) ⋅ ∇ 2 Q + 2 1 χ ′′ ( Δ ) ⋅ [ ∇Δ ⋅ ( ∇Δ ) T ] = L y + N y ,
where
L y : = 1 + χ ′ ( Δ ( y ) ) 2 ⋅ ∇ 2 W 0 ( y ) + 1 − χ ′ ( Δ ( y ) ) 2 ⋅ ∇ 2 Q ( y ) L_{y}:=\frac{1+\chi'(\Delta(\mathbf{y}))}{2}\cdot\nabla^{2}W_{0}(\mathbf{y})+\frac{1-\chi'(\Delta(\mathbf{y}))}{2}\cdot\nabla^{2}Q(\mathbf{y}) L y := 2 1 + χ ′ ( Δ ( y )) ⋅ ∇ 2 W 0 ( y ) + 2 1 − χ ′ ( Δ ( y )) ⋅ ∇ 2 Q ( y )
and
N y : = 1 2 χ ′ ′ ( Δ ( y ) ) ⋅ [ ∇ Δ ( y ) ⋅ ( ∇ Δ ( y ) ) T ] . N_{y}:=\frac{1}{2}\chi''(\Delta(\mathbf{y}))\cdot\left[\nabla\Delta(\mathbf{y})\cdot(\nabla\Delta(\mathbf{y}))^{\mathsf{T}}\right]. N y := 2 1 χ ′′ ( Δ ( y )) ⋅ [ ∇Δ ( y ) ⋅ ( ∇Δ ( y ) ) T ] .
The term L y L_{y} L y , the convex combination of two PSD matrices, is the dominant term in this sum since, due to our construction, − 1 ≤ χ ′ ( s ) ≤ 1 -1\leq\chi'(s)\leq1 − 1 ≤ χ ′ ( s ) ≤ 1 for all s ∈ R s\in\mathbb{R} s ∈ R . We will separately bound the spectrum of L y L_{y} L y and N y N_{y} N y in the following. In particular, we will show that
spec L y ⊂ [ e − S − O ε ~ , ε ~ 1 ( α ) , ( 1 + ε ~ ) ⋅ e S + O ε ~ , ε ~ 1 ( α ) ] . (56) \operatorname{spec}L_{y}\subset\left[e^{-S}-O_{\widetilde{\varepsilon},\widetilde{\varepsilon}_{1}}(\sqrt{\alpha}),(1+\widetilde{\varepsilon})\cdot e^{S}+O_{\widetilde{\varepsilon},\widetilde{\varepsilon}_{1}}(\sqrt{\alpha})\right]. \tag*{(56)} spec L y ⊂ [ e − S − O ε , ε 1 ( α ) , ( 1 + ε ) ⋅ e S + O ε , ε 1 ( α ) ] . (56)
and
spec N y ⊂ [ 0 , ( 1 + ε ~ ) ε ~ 1 e S + O ε ~ , ε ~ 1 ( α ) ] . (57) \operatorname{spec}N_{y}\subset\left[0,(1+\widetilde{\varepsilon})\widetilde{\varepsilon}_{1}e^{S}+O_{\widetilde{\varepsilon},\widetilde{\varepsilon}_{1}}(\alpha)\right]. \tag*{(57)} spec N y ⊂ [ 0 , ( 1 + ε ) ε 1 e S + O ε , ε 1 ( α ) ] . (57)
Given (D.17) and (D.18), both (I) and (D.4) follow because
sup y ∈ R 3 λ max ( ∇ 2 W ( y ) ) inf y ∈ R 3 λ min ( ∇ 2 W ( y ) ) ≤ ( 1 + ε ~ ) ( 1 + ε ~ 1 ) e S + O ε ~ , ε ~ 1 ( α ) e − S − O ε ~ , ε ~ 1 ( α ) → α → 0 ( 1 + ε ~ ) ( 1 + ε ~ 1 ) e 2 S . \frac{\sup_{\mathbf{y}\in\mathbb{R}^{3}}\lambda_{\max}\left(\nabla^{2}W(\mathbf{y})\right)}{\inf_{\mathbf{y}\in\mathbb{R}^{3}}\lambda_{\min}\left(\nabla^{2}W(\mathbf{y})\right)}
\leq\frac{(1+\widetilde{\varepsilon})(1+\widetilde{\varepsilon}_{1})e^{S}+O_{\widetilde{\varepsilon},\widetilde{\varepsilon}_{1}}(\sqrt{\alpha})}{e^{-S}-O_{\widetilde{\varepsilon},\widetilde{\varepsilon}_{1}}(\sqrt{\alpha})}\xrightarrow[\alpha\to0]{}(1+\widetilde{\varepsilon})(1+\widetilde{\varepsilon}_{1})e^{2S}. inf y ∈ R 3 λ m i n ( ∇ 2 W ( y ) ) sup y ∈ R 3 λ m a x ( ∇ 2 W ( y ) ) ≤ e − S − O ε , ε 1 ( α ) ( 1 + ε ) ( 1 + ε 1 ) e S + O ε , ε 1 ( α ) α → 0 ( 1 + ε ) ( 1 + ε 1 ) e 2 S .
We then get (D.4) by picking ε ~ , ε ~ 1 \widetilde{\varepsilon},\widetilde{\varepsilon}_{1} ε , ε 1 such that
( 1 + ε ~ ) ( 1 + ε ~ 1 ) < 1 + ε . (1+\widetilde{\varepsilon})(1+\widetilde{\varepsilon}_{1})<1+\varepsilon. ( 1 + ε ) ( 1 + ε 1 ) < 1 + ε .
For the strict lower bound, choose κ \kappa κ and Λ \Lambda Λ as the global infimum and supremum of the Hessian eigenvalues, respectively. At a point on the z z z -axis the Hessian has the eigenvalue e − S e^{-S} e − S , whereas in the exterior region where W = Q W=Q W = Q it has the eigenvalue ( 1 + ε ~ ) e S (1+\widetilde{\varepsilon})e^{S} ( 1 + ε ) e S . Hence
Λ κ ≥ ( 1 + ε ~ ) e 2 S > e 2 S . \frac{\Lambda}{\kappa}\geq(1+\widetilde{\varepsilon})e^{2S}>e^{2S}. κ Λ ≥ ( 1 + ε ) e 2 S > e 2 S .
Together with the preceding upper bound, this proves (D.4). It remains to prove (D.17) and (D.18). Let us start with (D.17). Note that on the set { y : χ ( Δ ) ≠ − 1 } \{y : \chi(\Delta) \ne-1\} { y : χ ( Δ ) = − 1 } , we have Δ > − δ − \Delta> -\delta_{-} Δ > − δ − , or equivalently
x T D ( z ) x ≤ 2 ( α + δ − ) . x^{\mathsf{T}}D(z)x \le2(\alpha+\delta_{-}). x T D ( z ) x ≤ 2 ( α + δ − ) .
The above, (D.16) and (D.5) yield
∥ x ∥ 2 ≤ 2 ( α + δ − ) ε ~ e S ≤ 2 ( α + 4 α / ε ~ 1 ) ε ~ e S \lVert x\rVert^2 \le\frac{2(\alpha+\delta_{-})}{\widetilde{\varepsilon}e^S} \le\frac{2(\alpha+4\alpha/\widetilde{\varepsilon}_1)}{\widetilde{\varepsilon}e^S} ∥ x ∥ 2 ≤ ε e S 2 ( α + δ − ) ≤ ε e S 2 ( α + 4 α / ε 1 )
on { Δ > − δ − } \{\Delta> -\delta_{-}\} { Δ > − δ − } .
Moreover,
spec G ( z ) ∪ spec ∇ 2 Q ( y ) = { e − S , 1 , e S , ( 1 + ε ~ ) ⋅ e S } . \operatorname{spec}G(z)\cup\operatorname{spec}\nabla^2Q(y)=\{e^{-S},1,e^S,(1+\widetilde{\varepsilon})\cdot e^S\}. spec G ( z ) ∪ spec ∇ 2 Q ( y ) = { e − S , 1 , e S , ( 1 + ε ) ⋅ e S } .
Thus Lemma G.7, the above display and (D.15) yield (D.17). We now prove (D.18). It is clear that N y ⪰ 0 N_y \succeq0 N y ⪰ 0 and that N y ≠ 0 N_y \ne0 N y = 0 only when Δ ∈ ( − δ − , δ + ) \Delta\in(-\delta_{-},\delta_{+}) Δ ∈ ( − δ − , δ + ) .
Observe that
∇ Δ ( x , z ) = ( − D ( z ) x 1 2 x T G ′ ( z ) x ) . \nabla\Delta(x,z)=\begin{pmatrix}
-D(z)x\\
\frac{1}{2}x^{\mathsf{T}}G'(z)x
\end{pmatrix}. ∇Δ ( x , z ) = ( − D ( z ) x 2 1 x T G ′ ( z ) x ) .
Thus
∥ ∇ Δ ( x , z ) ∥ 2 = ∥ D ( z ) x ∥ 2 + 1 4 [ x T G ′ ( z ) x ] 2 ≤ x T D ( z ) 2 x + O ( ∥ x ∥ 4 ⋅ 1 { Δ ∈ ( − δ − , δ + ) } ) \lVert\nabla\Delta(x,z)\rVert^2=\lVert D(z)x\rVert^2+\frac{1}{4}\left[x^{\mathsf{T}}G'(z)x\right]^2\le x^{\mathsf{T}}D(z)^2x+O\left(\lVert x\rVert^4\cdot1_{\{\Delta\in(-\delta_{-},\delta_{+})\}}\right) ∥ ∇Δ ( x , z ) ∥ 2 = ∥ D ( z ) x ∥ 2 + 4 1 [ x T G ′ ( z ) x ] 2 ≤ x T D ( z ) 2 x + O ( ∥ x ∥ 4 ⋅ 1 { Δ ∈ ( − δ − , δ + )} )
≤ 2 ( 1 + ε ~ ) ⋅ e S ⋅ ( α − Δ ) + O ε ~ , ε ~ 1 ( α 2 ) , \le2(1+\widetilde{\varepsilon})\cdot e^S\cdot(\alpha-\Delta)+O_{\widetilde{\varepsilon},\widetilde{\varepsilon}_1}(\alpha^2), ≤ 2 ( 1 + ε ) ⋅ e S ⋅ ( α − Δ ) + O ε , ε 1 ( α 2 ) ,
where the last inequality follows from x T D ( z ) x = 2 ( α − Δ ) x^{\mathsf{T}}D(z)x=2(\alpha-\Delta) x T D ( z ) x = 2 ( α − Δ ) and
D ( z ) 2 ⪯ ( 1 + ε ~ ) ⋅ e S ⋅ D ( z ) , D(z)^2\preceq(1+\widetilde{\varepsilon})\cdot e^S\cdot D(z), D ( z ) 2 ⪯ ( 1 + ε ) ⋅ e S ⋅ D ( z ) ,
which is due to (D.16).
Consequently,
1 2 ⋅ χ ′ ′ ( Δ ) ⋅ ∥ ∇ Δ ∥ 2 ≤ χ ′ ′ ( Δ ) ⋅ [ ( 1 + ε ~ ) ⋅ e S ⋅ ( α − Δ ) + O ε ~ , ε ~ 1 ( α 2 ) ] ≤ ε ~ 1 ⋅ ( 1 + ε ~ ) ⋅ e S + O ε ~ , ε ~ 1 ( α 2 α − Δ ) , \begin{aligned}
\frac{1}{2}\cdot\chi''(\Delta)\cdot\lVert\nabla\Delta\rVert^2\le\chi''(\Delta)\cdot\left[(1+\widetilde{\varepsilon})\cdot e^S\cdot(\alpha-\Delta)+O_{\widetilde{\varepsilon},\widetilde{\varepsilon}_1}(\alpha^2)\right] \\
&\le\widetilde{\varepsilon}_1\cdot(1+\widetilde{\varepsilon})\cdot e^S+O_{\widetilde{\varepsilon},\widetilde{\varepsilon}_1}\left(\frac{\alpha^2}{\alpha-\Delta}\right),
\end{aligned} 2 1 ⋅ χ ′′ ( Δ ) ⋅ ∥ ∇Δ ∥ 2 ≤ χ ′′ ( Δ ) ⋅ [ ( 1 + ε ) ⋅ e S ⋅ ( α − Δ ) + O ε , ε 1 ( α 2 ) ] ≤ ε 1 ⋅ ( 1 + ε ) ⋅ e S + O ε , ε 1 ( α − Δ α 2 ) ,
where the second line follows from property (iii) of χ \chi χ in Lemma D.1. Furthermore, on { ( x , z ) : Δ ∈ ( − δ − , δ + ) } \{(x,z):\Delta\in(-\delta_{-},\delta_{+})\} {( x , z ) : Δ ∈ ( − δ − , δ + )} , we get, by using (D.5),
α 2 α − Δ ≤ α 2 α − δ + ≤ ε ~ 1 e 2 / ε ~ 1 ⋅ α . \frac{\alpha^2}{\alpha-\Delta}\le\frac{\alpha^2}{\alpha-\delta_{+}}\le\widetilde{\varepsilon}_1e^{2/\widetilde{\varepsilon}_1}\cdot\alpha. α − Δ α 2 ≤ α − δ + α 2 ≤ ε 1 e 2/ ε 1 ⋅ α .
Thus
spec N y = { 0 ; 1 2 ⋅ χ ′ ′ ( Δ ) ⋅ ∥ ∇ Δ ∥ 2 } ⊂ [ 0 , ( 1 + ε ~ ) ε ~ 1 e S + O ε ~ , ε ~ 1 ( α ) ] , \operatorname{spec}N_y=\left\{0;\frac{1}{2}\cdot\chi''(\Delta)\cdot\lVert\nabla\Delta\rVert^2\right\}\subset\left[0,(1+\widetilde{\varepsilon})\widetilde{\varepsilon}_1e^S+O_{\widetilde{\varepsilon},\widetilde{\varepsilon}_1}(\alpha)\right], spec N y = { 0 ; 2 1 ⋅ χ ′′ ( Δ ) ⋅ ∥ ∇Δ ∥ 2 } ⊂ [ 0 , ( 1 + ε ) ε 1 e S + O ε , ε 1 ( α ) ] ,
which is (D.18). The proof is completed. ∎
Proof of Theorem D.2
Let e 3 = ( 0 , 0 , 1 ) T ∈ R 3 e_3=(0,0,1)^{\mathsf{T}}\in\mathbb{R}^3 e 3 = ( 0 , 0 , 1 ) T ∈ R 3 and put
P β ( x ) : = Z 1 β , W ( β x ) . \mathcal{P}_{\beta}(x):=\mathbf{Z}^{\beta,W}_1\left(\sqrt{\beta}x\right). P β ( x ) := Z 1 β , W ( β x ) .
where Z 1 β , W \mathbf{Z}^{\beta,W}_1 Z 1 β , W is the endpoint of the flow obtained from N ( 0 , I 3 ) ⊗ p 1 β N(0,I_3)\otimes p^\beta_1 N ( 0 , I 3 ) ⊗ p 1 β . The reader is referred to Section G.1 for some basic terminologies and properties of Z 1 β , W \mathbf{Z}^{\beta,W}_1 Z 1 β , W .
By Proposition 4.1 and Lemma D.3, it suffices to show that for all β > 0 \beta>0 β > 0 sufficiently large, the Jacobian D Z 1 β , W ( β e 3 ) D\mathbf{Z}^{\beta,W}_1\left(\sqrt{\beta}e_3\right) D Z 1 β , W ( β e 3 ) has two distinct negative eigenvalues. By using the fact
D P β ( e 3 ) = β ⋅ D Z 1 β , W ( β e 3 ) D\mathcal{P}_{\beta}(e_3)=\sqrt{\beta}\cdot D\mathbf{Z}^{\beta,W}_1\left(\sqrt{\beta}e_3\right) D P β ( e 3 ) = β ⋅ D Z 1 β , W ( β e 3 )
along with Proposition G.2(iii), we only need to check that D P ( e 3 ) D\mathcal{P}(e_3) D P ( e 3 ) has two distinct negative eigenvalues.
Indeed, given such a property on D P ( e 3 ) D\mathcal{P}(e_3) D P ( e 3 ) , Proposition G.2(iii) implies that D Z 1 β , W ( β e 3 ) D\mathbf{Z}^{\beta,W}_1\left(\sqrt{\beta}e_3\right) D Z 1 β , W ( β e 3 ) also has two distinct negative eigenvalues (with the remaining eigenvalue positive), for all β \beta β sufficiently large. We then get non-straightness from Lemma D.3, (II) and Proposition 4.1.
Define
K ( r , x ) : = D x h ( r , x ) and K ( r ) : = D x h ( r , x ) ∣ x = e 3 . K(r,x):=D_xh(r,x)\qquad\text{and}\qquad K(r):=D_xh(r,x)\big|_{x=e_3}. K ( r , x ) := D x h ( r , x ) and K ( r ) := D x h ( r , x ) x = e 3 .
Now, by Proposition G.2(ii),
D P ( e 3 ) = lim r → ∞ K ( r ) r . (58) D\mathcal{P}(e_3)=\lim_{r\to\infty}\frac{K(r)}{r}. \tag*{(58)} D P ( e 3 ) = r → ∞ lim r K ( r ) . (58)
Next, let us get an ODE for K ( r ) K(r) K ( r ) . Observe that
d d r K ( r , x ) = d d r [ D x h ( r , x ) ] = D x [ d d r h ( r , x ) ] = D x [ y ( r , h ( r , x ) ) ] = D h y ( r , h ) ⋅ D x h ( r , x ) = r ⋅ [ ∇ 2 W ( y ( r , h ) ) + r 2 I 3 ] − 1 K ( r , x ) , \begin{aligned}
\frac{d}{dr}K(r,x)&=\frac{d}{dr}\left[D_xh(r,x)\right]=D_x\left[\frac{d}{dr}h(r,x)\right]\\
&=D_x\left[\mathbf{y}(r,h(r,x))\right]\\
&=D_h\mathbf{y}(r,h)\cdot D_xh(r,x)\\
&=r\cdot\left[\nabla^2W(\mathbf{y}(r,h))+r^2I_3\right]^{-1}K(r,x),
\end{aligned} d r d K ( r , x ) = d r d [ D x h ( r , x ) ] = D x [ d r d h ( r , x ) ] = D x [ y ( r , h ( r , x )) ] = D h y ( r , h ) ⋅ D x h ( r , x ) = r ⋅ [ ∇ 2 W ( y ( r , h )) + r 2 I 3 ] − 1 K ( r , x ) ,
where the last equality follows from (G.13).
By setting x = e 3 x=e_3 x = e 3 , we deduce that
K ′ ( r ) = r ⋅ [ ∇ 2 W ( y ( r , h ) ) + r 2 I 3 ] − 1 ⋅ K ( r ) , K ( 0 ) = I 3 . K'(r)=r\cdot\left[\nabla^2W(\mathbf{y}(r,h))+r^2I_3\right]^{-1}\cdot K(r),\qquad K(0)=I_3. K ′ ( r ) = r ⋅ [ ∇ 2 W ( y ( r , h )) + r 2 I 3 ] − 1 ⋅ K ( r ) , K ( 0 ) = I 3 .
By (III), the trajectory ( h ( r ) , y ( r ) ) (h(r),\mathbf{y}(r)) ( h ( r ) , y ( r )) starting at e 3 e_3 e 3 is precisely
h ( r , e 3 ) = ( 0 , 1 + r 2 ) = 1 + r 2 ⋅ e 3 and y ( r ) = ( 0 , r 1 + r 2 ) = r 1 + r 2 ⋅ e 3 . h(r,e_3)=(0,\sqrt{1+r^2})=\sqrt{1+r^2}\cdot e_3\qquad\text{and}\qquad\mathbf{y}(r)=\left(0,\frac{r}{\sqrt{1+r^2}}\right)=\frac{r}{\sqrt{1+r^2}}\cdot e_3. h ( r , e 3 ) = ( 0 , 1 + r 2 ) = 1 + r 2 ⋅ e 3 and y ( r ) = ( 0 , 1 + r 2 r ) = 1 + r 2 r ⋅ e 3 .
Indeed, one can check from ∇ W ( 0 , z ) = ( 0 , z ) \nabla W(0,z)=(0,z) ∇ W ( 0 , z ) = ( 0 , z ) that
d d r h ( r , e 3 ) = y ( r , h ( r , e 3 ) ) and ∇ W ( y ( r ) ) + r 2 y ( r ) − r h ( r ) = 0. \frac{d}{dr}h(r,e_3)=\mathbf{y}(r,h(r,e_3))\qquad\text{and}\qquad\nabla W(\mathbf{y}(r))+r^2\mathbf{y}(r)-rh(r)=0. d r d h ( r , e 3 ) = y ( r , h ( r , e 3 )) and ∇ W ( y ( r )) + r 2 y ( r ) − r h ( r ) = 0.
Together with (IV), we find that
[ ∇ 2 W ( y ( r , h ) ) + r 2 I 3 ] − 1 = [ ∇ 2 W ( r 1 + r 2 ⋅ e 3 ) + r 2 I 3 ] − 1 = ( G W ( ζ ( r ) ) + r 2 I 2 0 0 1 r 2 + 1 ) \left[\nabla^2W(\mathbf{y}(r,h))+r^2I_3\right]^{-1}
=
\left[\nabla^2W\left(\frac{r}{\sqrt{1+r^2}}\cdot e_3\right)+r^2I_3\right]^{-1}
=
\begin{pmatrix}
G_W(\zeta(r))+r^2I_2 & 0\\
0 & \frac{1}{r^2+1}
\end{pmatrix} [ ∇ 2 W ( y ( r , h )) + r 2 I 3 ] − 1 = [ ∇ 2 W ( 1 + r 2 r ⋅ e 3 ) + r 2 I 3 ] − 1 = ( G W ( ζ ( r )) + r 2 I 2 0 0 r 2 + 1 1 )
where ζ ( r ) = r / 1 + r 2 \zeta(r) = r/\sqrt{1+r^2} ζ ( r ) = r / 1 + r 2 .
Consequently, if we write K ( r ) = ( M W ( r ) , k ( r ) ) K(r) = (M_W(r), k(r)) K ( r ) = ( M W ( r ) , k ( r )) with M W ∈ R 2 × 2 M_W \in\mathbb{R}^{2 \times2} M W ∈ R 2 × 2 , then
M W ′ ( r ) = r ⋅ ( G W ( ζ ( r ) ) + r 2 I 2 ) − 1 ⋅ M W ( r ) , M W ( 0 ) = I 2 . M_W'(r) = r \cdot\left(G_W(\zeta(r)) + r^2 I_2\right)^{-1} \cdot M_W(r), \qquad M_W(0) = I_2. M W ′ ( r ) = r ⋅ ( G W ( ζ ( r )) + r 2 I 2 ) − 1 ⋅ M W ( r ) , M W ( 0 ) = I 2 .
and
k ′ ( r ) = r 1 + r 2 ⋅ k ( r ) , k ( 0 ) = 1. k'(r) = \frac{r}{1+r^2} \cdot k(r), \qquad k(0) = 1. k ′ ( r ) = 1 + r 2 r ⋅ k ( r ) , k ( 0 ) = 1.
It is easy to check that k ( r ) = 1 + r 2 k(r) = \sqrt{1+r^2} k ( r ) = 1 + r 2 , which yields the block form
K ( r ) = ( M W ( r ) 0 0 1 + r 2 ) . K(r) = \begin{pmatrix} M_W(r) & 0 \\ 0 & \sqrt{1+r^2} \end{pmatrix}. K ( r ) = ( M W ( r ) 0 0 1 + r 2 ) .
By (IV), K ( r ) / r K(r)/r K ( r ) / r has two distinct negative eigenvalues for all r r r large enough. This finishes the proof.
Proof of Theorem 4.2
Heuristic construction
We explain the construction in dimension three with σ = 1 \sigma= 1 σ = 1 . Choose an integer N ≥ 2 N \ge2 N ≥ 2 , a horizontal radius R > 0 R > 0 R > 0 , and a vertical scale η > 0 \eta> 0 η > 0 . The parameters R , η R,\eta R , η will be used to control how close D max 2 D_{\max}^2 D m a x 2 is to 16 π 16\pi 16 π . Arrange the means of the 2 N 2N 2 N components as
μ j ± = ( ± R u π θ j ⊥ , η θ j ) , θ j = 2 j − 1 2 N , u ϕ = ( cos ϕ sin ϕ ) , 1 ≤ j ≤ N . \mu_j^\pm= (\pm R u_{\pi\theta_j}^\perp, \eta\theta_j), \qquad\theta_j = \frac{2j-1}{2N}, \qquad u_\phi= \begin{pmatrix} \cos\phi\\ \sin\phi\end{pmatrix}, \qquad1 \le j \le N. μ j ± = ( ± R u π θ j ⊥ , η θ j ) , θ j = 2 N 2 j − 1 , u ϕ = ( cos ϕ sin ϕ ) , 1 ≤ j ≤ N .
The mixture is chosen to be
∑ j = 1 N ( p j + ⋅ N ( μ j + , I d ) + p j − ⋅ N ( μ j − , I d ) ) , \sum_{j=1}^{N} \left(p_j^+ \cdot\mathcal{N}(\mu_j^+, I_d) + p_j^- \cdot\mathcal{N}(\mu_j^-, I_d)\right), j = 1 ∑ N ( p j + ⋅ N ( μ j + , I d ) + p j − ⋅ N ( μ j − , I d ) ) ,
for some appropriately chosen weights { p j ± ; 1 ≤ j ≤ N } \{p_j^\pm; 1 \le j \le N\} { p j ± ; 1 ≤ j ≤ N } .
By pairwise symmetry of the means, any trajectory that begins on the z z z -axis stays on the z z z -axis at all times. Moreover, it can be checked that 4 R 2 ≤ D max 2 ≤ 4 R 2 + η 2 4R^2 \le D_{\max}^2 \le4R^2 + \eta^2 4 R 2 ≤ D m a x 2 ≤ 4 R 2 + η 2 , and we pick R 2 R^2 R 2 to be close to 4 π 4\pi 4 π so that the requirement that D max 2 D_{\max}^2 D m a x 2 be close to 16 π 16\pi 16 π holds.
Write p 1 p_1 p 1 for this mixture and let Z t p 1 Z_t^{p_1} Z t p 1 denote the corresponding RF flow map. To get a cleaner form of the endpoint Jacobian, we introduce the rescaled flow q q q satisfying
Z t p 1 ( x ) = ρ t q ( a ( t ) , x ) , Z_t^{p_1}(x) = \rho_t q(a(t),x), Z t p 1 ( x ) = ρ t q ( a ( t ) , x ) ,
where
ρ t : = ( 1 − t ) 2 + t 2 and a ( t ) : = t ρ t . \rho_t := \sqrt{(1-t)^2+t^2} \qquad\text{and} \qquad a(t) := \frac{t}{\rho_t}. ρ t := ( 1 − t ) 2 + t 2 and a ( t ) := ρ t t .
As shown in (E.5) below, the Jacobian of this rescaled flow K ( a , x ) : = D x q ( a , x ) K(a,x) := D_xq(a,x) K ( a , x ) := D x q ( a , x ) solves the matrix ODE
∂ a K ( a , x ) = a C ( a , q ( a , x ) ) ⋅ K ( a , x ) , K ( 0 , x ) = I 3 . \partial_a K(a,x) = a C(a,q(a,x)) \cdot K(a,x), \qquad K(0,x) = I_3. ∂ a K ( a , x ) = a C ( a , q ( a , x )) ⋅ K ( a , x ) , K ( 0 , x ) = I 3 .
Here,
C ( a , y ) : = Cov ( μ ∣ a μ + G = y ) C(a,y) := \operatorname{Cov}(\mu\mid a\mu+ G = y) C ( a , y ) := Cov ( μ ∣ a μ + G = y )
where μ \mu μ is the random component mean with P ( μ = μ j ± ) = p j ± \mathbb{P}(\mu= \mu_j^{\pm}) = p_j^{\pm} P ( μ = μ j ± ) = p j ± , and G ∼ N ( 0 , I 3 ) G \sim\mathcal{N}(0,I_3) G ∼ N ( 0 , I 3 ) is independent of μ \mu μ . Since ρ 1 = a ( 1 ) = 1 \rho_1 = a(1) = 1 ρ 1 = a ( 1 ) = 1 , the endpoint Jacobians of the two flows coincide, i.e.
D Z 1 ρ 1 ( x ) = K ( 1 , x ) . D Z_1^{\rho_1}(x) = K(1,x). D Z 1 ρ 1 ( x ) = K ( 1 , x ) .
Using Proposition 4.1, it is sufficient to choose the mixture weights { p j ± ; 1 ≤ j ≤ N } \left\{p_j^{\pm}; 1 \le j \le N\right\} { p j ± ; 1 ≤ j ≤ N } so that the matrix ODE of K K K approximates one whose solution has two distinct negative eigenvalues at the endpoint a = 1 a = 1 a = 1 , for some initial location x ⋆ x_\star x ⋆ . Towards that goal, we choose
p j ± = exp ( − B θ j 2 ) 2 ∑ ℓ = 1 N exp ( − B θ ℓ 2 ) , x ⋆ = 2 B η e 3 and e 3 = ( 0 , 0 , 1 ) T , p_j^{\pm} = \frac{\exp(-B\theta_j^2)}{2\sum_{\ell=1}^{N}\exp(-B\theta_\ell^2)}, \qquad x_\star= \frac{2B}{\eta}e_3 \qquad\text{and}\qquad e_3 = (0,0,1)^{\mathsf{T}}, p j ± = 2 ∑ ℓ = 1 N exp ( − B θ ℓ 2 ) exp ( − B θ j 2 ) , x ⋆ = η 2 B e 3 and e 3 = ( 0 , 0 , 1 ) T ,
for a sufficiently large B > 0 B > 0 B > 0 .
The evolution of the posterior covariance C ( a , ⋅ ) C(a,\cdot) C ( a , ⋅ ) over the time variable a > 0 a > 0 a > 0 can be described via the posterior distribution
α j ( a ) : = P ( μ ∈ { μ j + , μ j − } ∣ a μ + G = q ( a , x ⋆ ) ) , ∑ j = 1 N α j ( a ) = 1. \alpha_j(a) := \mathbb{P}\left(\mu\in\{\mu_j^+,\mu_j^-\}\mid a\mu+ G = q(a,x_\star)\right), \qquad\sum_{j=1}^{N}\alpha_j(a)=1. α j ( a ) := P ( μ ∈ { μ j + , μ j − } ∣ a μ + G = q ( a , x ⋆ ) ) , j = 1 ∑ N α j ( a ) = 1.
More precisely, due to the fact that the trajectory stays on the z z z -axis, we get
C ( a , q ( a , x ⋆ ) ) = ( C ⊥ ( a ) 0 2 × 1 0 1 × 2 c 3 ( a ) ) . C(a,q(a,x_\star)) =
\begin{pmatrix}
C_\perp(a) & 0_{2\times1}\\
0_{1\times2} & c_3(a)
\end{pmatrix}. C ( a , q ( a , x ⋆ )) = ( C ⊥ ( a ) 0 1 × 2 0 2 × 1 c 3 ( a ) ) .
where
C ⊥ ( a ) = R 2 ∑ j = 1 N α j ( a ) P θ j 2 , P ϕ : = u ϕ u ϕ T = ( cos 2 ϕ cos ϕ sin ϕ cos ϕ sin ϕ sin 2 ϕ ) . C_\perp(a) = R^2\sum_{j=1}^{N}\alpha_j(a)P_{\theta_j^2}, \qquad P_\phi:= u_\phi u_\phi^{\mathsf{T}} =
\begin{pmatrix}
\cos^2\phi& \cos\phi\sin\phi\\
\cos\phi\sin\phi& \sin^2\phi
\end{pmatrix}. C ⊥ ( a ) = R 2 j = 1 ∑ N α j ( a ) P θ j 2 , P ϕ := u ϕ u ϕ T = ( cos 2 ϕ cos ϕ sin ϕ cos ϕ sin ϕ sin 2 ϕ ) .
and
c 3 ( a ) = η 2 [ ∑ j = 1 N α j ( a ) θ j 2 − ( ∑ j = 1 N α j ( a ) θ j ) 2 ] . c_3(a)=\eta^2\left[\sum_{j=1}^{N}\alpha_j(a)\theta_j^2-\left(\sum_{j=1}^{N}\alpha_j(a)\theta_j\right)^2\right]. c 3 ( a ) = η 2 j = 1 ∑ N α j ( a ) θ j 2 − ( j = 1 ∑ N α j ( a ) θ j ) 2 .
See the two panels in Figure S2 . The choice of the initial mixture weights { p j ± ; 1 ≤ j ≤ N } \{p_j^{\pm};1\le j\le N\} { p j ± ; 1 ≤ j ≤ N } gives, as N , B → ∞ N,B\to\infty N , B → ∞ ,
C ⊥ ( a ) → R 2 P π a 2 , c 3 ( a ) → 0 , C_{\perp}(a)\to R^2P_{\pi a^2},\qquad c_3(a)\to0, C ⊥ ( a ) → R 2 P π a 2 , c 3 ( a ) → 0 ,
uniformly for a ∈ [ 0 , 1 ] a\in[0,1] a ∈ [ 0 , 1 ] .
The limiting 2 × 2 2\times2 2 × 2 upper left block Jacobian therefore solves
M ′ ( a ) = a R 2 P π a 2 ⋅ M ( a ) , M ( 0 ) = I 2 . M'(a)=aR^2P_{\pi a^2}\cdot M(a),\qquad M(0)=I_2. M ′ ( a ) = a R 2 P π a 2 ⋅ M ( a ) , M ( 0 ) = I 2 .
We solve this ODE explicitly in Lemma E.1 and show that, when R 2 > 4 π R^2>4\pi R 2 > 4 π , M ( 1 ) M(1) M ( 1 ) has real, distinct and negative eigenvalues
− exp ( R 2 4 ± R 4 16 − π 2 ) . -\exp\left(\frac{R^2}{4}\pm\sqrt{\frac{R^4}{16}-\pi^2}\right). − exp ( 4 R 2 ± 16 R 4 − π 2 ) .
Note that the negative eigenvalues happen because the projections in the family { P a ; a ∈ R } \{P_a;a\in\mathbb{R}\} { P a ; a ∈ R } generally do not commute. By choosing N , B N,B N , B large (but finite), we get (using standard perturbation theory for ODEs) that
D Z 1 p 1 ( x ⋆ ) = K ( 1 , x ⋆ ) ≈ ( M ( 1 ) 0 0 1 ) . \mathrm{D}Z_1^{p_1}(\mathbf{x}_\star)=K(1,\mathbf{x}_\star)\approx
\begin{pmatrix}
M(1) & 0\\
0 & 1
\end{pmatrix}. D Z 1 p 1 ( x ⋆ ) = K ( 1 , x ⋆ ) ≈ ( M ( 1 ) 0 0 1 ) .
Here, both the mixture p 1 p_1 p 1 and the starting point x ⋆ \mathbf{x}_\star x ⋆ depend on N , B N,B N , B . The final step connects this endpoint reversal to the folding of straight endpoint chords. That is, for
H t ( x ) = ( 1 − t ) x + t Z 1 p 1 ( x ) , \mathbf{H}_t(x)=(1-t)x+tZ_1^{p_1}(x), H t ( x ) = ( 1 − t ) x + t Z 1 p 1 ( x ) ,
we have
det D H t ( x ⋆ ) = ∏ i = 1 3 ( 1 − t + t λ i ) < 0 when 1 1 − λ 1 < t < 1 1 − λ 2 . \det\mathrm{D}\mathbf{H}_t(\mathbf{x}_\star)=\prod_{i=1}^{3}(1-t+t\lambda_i)<0\qquad\text{when}\qquad\frac{1}{1-\lambda_1}<t<\frac{1}{1-\lambda_2}. det D H t ( x ⋆ ) = i = 1 ∏ 3 ( 1 − t + t λ i ) < 0 when 1 − λ 1 1 < t < 1 − λ 2 1 .
Lemma H.3 verifies that Z P 1 ( x ) − x Z^{\mathrm{P1}}(x)-x Z P1 ( x ) − x is bounded. Under this global condition, Proposition 4.1 shows that the negative determinant forces overlapping inverse branches of H t \mathbf{H}_t H t and hence S t > 0 S_t>0 S t > 0 . This gives the non-straightness mechanism, as illustrated in Figure S3 .
Figure S3. Mechanism of non-straightness. The rotating posterior covariance induces non-commuting positive semidefinite transverse stretches. Their accumulation produces an endpoint Jacobian with two distinct negative eigenvalues. By Proposition 4.1, the resulting coupling is non-straight.
Detailed proof of Theorem 4.2
Before proving Theorem 4.2 , we first construct an explicit ODE whose solution has a Jacobian with two distinct, negative eigenvalues. Recall P θ P_\theta P θ , R θ R_\theta R θ and Ω \Omega Ω in (D.8) and (D.10), respectively. Also recall the relation
P θ = R θ P 0 R θ T . P_\theta=R_\theta P_0R_\theta^{\mathsf{T}}. P θ = R θ P 0 R θ T .
Lemma E.1. Suppose R 2 > 4 π R^2>4\pi R 2 > 4 π . The equation
M ′ ( a ) = a R 2 P π a 2 ⋅ M ( a ) , M ( 0 ) = I 2 , 0 ≤ a ≤ 1 , M'(a)=aR^2P_{\pi a^2}\cdot M(a),\qquad M(0)=I_2,\qquad0\leq a\leq1, M ′ ( a ) = a R 2 P π a 2 ⋅ M ( a ) , M ( 0 ) = I 2 , 0 ≤ a ≤ 1 , has a unique solution M ∈ C 1 ( [ 0 , 1 ] , GL 2 ( R ) ) M\in C^1\left([0,1],\operatorname{GL}_2(\mathbb{R})\right) M ∈ C 1 ( [ 0 , 1 ] , GL 2 ( R ) ) such that M ( 1 ) M(1) M ( 1 ) has two distinct negative eigenvalues.
Proof of Lemma E.1. The existence, uniqueness and C 1 C^1 C 1 regularity follow easily from Theorem 3.9 in Teschl [ [55 ] since the coefficients are smooth. Its solution can be computed explicitly as
M ( a ) : = R π a 2 ⋅ exp [ π a 2 ( R 2 2 π P 0 − Ω ) ] . M(a):=R_{\pi a^2}\cdot\exp\left[\pi a^2\left(\frac{R^2}{2\pi}P_0-\Omega\right)\right]. M ( a ) := R π a 2 ⋅ exp [ π a 2 ( 2 π R 2 P 0 − Ω ) ] .
Indeed, this can be checked directly by differentiating the above expression. Using the representation R θ = exp ( θ Ω ) R_\theta=\exp(\theta\Omega) R θ = exp ( θ Ω ) (for all real θ \theta θ ) and setting K : = ( R 2 / 2 π ) P 0 − Ω K:=\left(R^2/2\pi\right)P_0-\Omega K := ( R 2 /2 π ) P 0 − Ω , we can write
M ′ ( a ) = d d a [ exp ( π a 2 Ω ) ⋅ exp ( π a 2 ⋅ K ) ] = 2 π a ⋅ Ω ⋅ M ( a ) + exp ( π a 2 Ω ) ⋅ 2 π a ⋅ K ⋅ exp ( π a 2 ⋅ K ) = 2 π a ⋅ [ Ω + exp ( π a 2 Ω ) ⋅ K ⋅ exp ( − π a 2 Ω ) ] ⋅ M ( a ) = 2 π a ⋅ [ Ω + R π a 2 K R π a 2 T ] ⋅ M ( a ) = 2 π a ⋅ [ Ω + R 2 2 π P π a 2 − Ω ] ⋅ M ( a ) = a R 2 P π a 2 ⋅ M ( a ) . \begin{aligned}
M'(a)&=\frac{d}{da}\left[\exp(\pi a^2\Omega)\cdot\exp(\pi a^2\cdot K)\right]\\
&=2\pi a\cdot\Omega\cdot M(a)+\exp(\pi a^2\Omega)\cdot2\pi a\cdot K\cdot\exp(\pi a^2\cdot K)\\
&=2\pi a\cdot\left[\Omega+\exp(\pi a^2\Omega)\cdot K\cdot\exp(-\pi a^2\Omega)\right]\cdot M(a)\\
&=2\pi a\cdot\left[\Omega+R_{\pi a^2}KR_{\pi a^2}^{\mathsf{T}}\right]\cdot M(a)\\
&=2\pi a\cdot\left[\Omega+\frac{R^2}{2\pi}P_{\pi a^2}-\Omega\right]\cdot M(a)
=aR^2P_{\pi a^2}\cdot M(a).
\end{aligned} M ′ ( a ) = d a d [ exp ( π a 2 Ω ) ⋅ exp ( π a 2 ⋅ K ) ] = 2 π a ⋅ Ω ⋅ M ( a ) + exp ( π a 2 Ω ) ⋅ 2 π a ⋅ K ⋅ exp ( π a 2 ⋅ K ) = 2 π a ⋅ [ Ω + exp ( π a 2 Ω ) ⋅ K ⋅ exp ( − π a 2 Ω ) ] ⋅ M ( a ) = 2 π a ⋅ [ Ω + R π a 2 K R π a 2 T ] ⋅ M ( a ) = 2 π a ⋅ [ Ω + 2 π R 2 P π a 2 − Ω ] ⋅ M ( a ) = a R 2 P π a 2 ⋅ M ( a ) .
where the second last equality follows from the fact that Ω \Omega Ω commutes with R θ R_\theta R θ , for all θ ∈ R \theta\in\mathbb{R} θ ∈ R . At a = 1 a=1 a = 1 , we have R π = − I 2 R_\pi=-I_2 R π = − I 2 , so
M ( 1 ) = − exp [ π ( R 2 / ( 2 π ) 1 − 1 0 ) ] . M(1)=-\exp\left[\pi\begin{pmatrix}R^2/(2\pi)&1\\-1&0\end{pmatrix}\right]. M ( 1 ) = − exp [ π ( R 2 / ( 2 π ) − 1 1 0 ) ] .
The eigenvalues of M ( 1 ) M(1) M ( 1 ) are
− exp [ π 2 ( R 2 2 π + R 4 4 π 2 − 4 ) ] , − exp [ π 2 ( R 2 2 π − R 4 4 π 2 − 4 ) ] . -\exp\left[\frac{\pi}{2}\left(\frac{R^2}{2\pi}+\sqrt{\frac{R^4}{4\pi^2}-4}\right)\right],
\qquad
-\exp\left[\frac{\pi}{2}\left(\frac{R^2}{2\pi}-\sqrt{\frac{R^4}{4\pi^2}-4}\right)\right]. − exp [ 2 π ( 2 π R 2 + 4 π 2 R 4 − 4 ) ] , − exp [ 2 π ( 2 π R 2 − 4 π 2 R 4 − 4 ) ] .
Proof of Theorem 4.2. Without loss of generality, we may assume that σ = 1 \sigma=1 σ = 1 . Take ε > 0 \varepsilon>0 ε > 0 and choose
R 2 : = 4 π + ε 8 and η : = ε 2 . R^2:=4\pi+\frac{\varepsilon}{8}
\qquad\text{and}\qquad
\eta:=\frac{\sqrt{\varepsilon}}{2}. R 2 := 4 π + 8 ε and η := 2 ε .
Recall u θ u_\theta u θ from (D.8). Let N ∈ N N \in\mathbb{N} N ∈ N and B > 0 B > 0 B > 0 be two parameters sufficiently large to be chosen later and define
θ j : = 2 j − 1 2 N ; μ j ± : = ( ± R u π θ j 2 η θ j ) ∈ R 3 , p j ± : = exp ( − B θ j 2 ) 2 ∑ k = 1 N exp ( − B θ k 2 ) . (59) \theta_j := \frac{2j-1}{2N}; \qquad\mu_j^\pm:= \begin{pmatrix} \pm R u_{\pi\theta_j^2} \\ \eta\theta_j \end{pmatrix} \in\mathbb{R}^3, \qquad p_j^\pm:= \frac{\exp\left(-B\theta_j^2\right)}{2\sum_{k=1}^{N}\exp\left(-B\theta_k^2\right)}. \tag*{(59)} θ j := 2 N 2 j − 1 ; μ j ± := ( ± R u π θ j 2 η θ j ) ∈ R 3 , p j ± := 2 ∑ k = 1 N exp ( − B θ k 2 ) exp ( − B θ j 2 ) . (59)
The desired target distribution is
p 1 : = ∑ j = 1 N [ p j + ⋅ N ( μ j + , I 3 ) + p j − ⋅ N ( μ j − , I 3 ) ] . p_1 := \sum_{j=1}^{N}\left[p_j^+ \cdot\mathcal{N}\left(\mu_j^+, I_3\right) + p_j^- \cdot\mathcal{N}\left(\mu_j^-, I_3\right)\right]. p 1 := j = 1 ∑ N [ p j + ⋅ N ( μ j + , I 3 ) + p j − ⋅ N ( μ j − , I 3 ) ] .
Roughly speaking, p 1 p_1 p 1 is a mixture with 2 N 2N 2 N components, with equal weights within each antipodal pair. By Lemma H.1 and the definitions of R , η R,\eta R , η , we get
16 π ≤ D max 2 ≤ 16 π + ε . 16\pi\le D_{\max}^2 \le16\pi+\varepsilon. 16 π ≤ D m a x 2 ≤ 16 π + ε .
Therefore, by using Proposition 4.1, we only need to show that the endpoint map’s Jacobian of Z 1 p 1 Z_1^{p_1} Z 1 p 1 , where
{ Z t p 1 } 0 ≤ t ≤ 1 : = Rectflow ( N ( 0 , I 3 ) , p 1 ) , (60) \left\{Z_t^{p_1}\right\}_{0\le t\le1} := \operatorname{Rectflow}\left(\mathcal{N}(0,I_3),p_1\right), \tag*{(60)} { Z t p 1 } 0 ≤ t ≤ 1 := Rectflow ( N ( 0 , I 3 ) , p 1 ) , (60)
has exactly two distinct negative eigenvalues at some point. Note that the condition (19) is met with B = I 3 B=I_3 B = I 3 and by using Lemma H.3 .
We split the proof into a few steps.
Step 1: Rescale the endpoint map. Make the change of variable
ρ t : = ( 1 − t ) 2 + t 2 , a ( t ) : = t ρ t and Z t p 1 ( x ) : = ρ t ⋅ q ( a ( t ) , x ) . (61) \rho_t := \sqrt{(1-t)^2+t^2}, \qquad a(t):=\frac{t}{\rho_t} \qquad\text{and}\qquad Z_t^{p_1}(x):=\rho_t\cdot q(a(t),x). \tag*{(61)} ρ t := ( 1 − t ) 2 + t 2 , a ( t ) := ρ t t and Z t p 1 ( x ) := ρ t ⋅ q ( a ( t ) , x ) . (61)
The last change of variable is valid because a ( t ) a(t) a ( t ) is a C 1 C^1 C 1 homeomorphism on [ 0 , 1 ] [0,1] [ 0 , 1 ] . Roughly speaking, we will change the time scale from t t t to a a a . The point of this change of variable is that q ( a , x ) q(a,x) q ( a , x ) satisfies a convenient ODE that is easier to analyze.
Letting
K ( a , x ) : = D x q ( a , x ) , K(a,x):=D_xq(a,x), K ( a , x ) := D x q ( a , x ) ,
observe that
D x [ Z t p 1 ( x ) ] = ρ t ⋅ K ( a ( t ) , x ) . D_x\left[Z_t^{p_1}(x)\right]=\rho_t\cdot K(a(t),x). D x [ Z t p 1 ( x ) ] = ρ t ⋅ K ( a ( t ) , x ) .
Hence, D x Z 1 p 1 ( x ) = K ( 1 , x ) D_xZ_1^{p_1}(x)=K(1,x) D x Z 1 p 1 ( x ) = K ( 1 , x ) .
Let us derive the ODE for K ( ⋅ , x ) K(\cdot,x) K ( ⋅ , x ) . A direct calculation shows that the velocity field in (B.1) with respect to N ( 0 , I 3 ) ⊗ p 1 \mathcal{N}(0,I_3)\otimes p_1 N ( 0 , I 3 ) ⊗ p 1 has the form
v t p 1 ( z ) = ρ t ′ ρ t ⋅ z + 1 − t ρ t 2 ⋅ m ( a ( t ) , z ρ t ) , v_t^{p_1}(z)=\frac{\rho_t'}{\rho_t}\cdot z+\frac{1-t}{\rho_t^2}\cdot m\left(a(t),\frac{z}{\rho_t}\right), v t p 1 ( z ) = ρ t ρ t ′ ⋅ z + ρ t 2 1 − t ⋅ m ( a ( t ) , ρ t z ) ,
where m m m is the posterior mean
m ( a , q ) : = ∑ ℓ = 1 N [ w ℓ + ( a , q ) ⋅ μ ℓ + + w ℓ − ( a , q ) ⋅ μ ℓ − ] . m(a,q):=\sum_{\ell=1}^{N}\left[w_\ell^+(a,q)\cdot\mu_\ell^+ + w_\ell^-(a,q)\cdot\mu_\ell^-\right]. m ( a , q ) := ℓ = 1 ∑ N [ w ℓ + ( a , q ) ⋅ μ ℓ + + w ℓ − ( a , q ) ⋅ μ ℓ − ] .
with
w ℓ ± ( a , q ) : = p ℓ + ⋅ exp ( a q ⊤ μ ℓ ± − a 2 2 ⋅ ∥ μ ℓ ± ∥ 2 ) ∑ k = 1 N ∑ ϵ ∈ { + , − } p k ϵ ⋅ exp ( a q ⊤ μ k ϵ − a 2 2 ⋅ ∥ μ k ϵ ∥ 2 ) . w_{\ell}^{\pm}(a,q):=\frac{p_{\ell}^{+}\cdot\exp\left(aq^{\top}\mu_{\ell}^{\pm}-\frac{a^{2}}{2}\cdot\left\|\mu_{\ell}^{\pm}\right\|^{2}\right)}{\sum_{k=1}^{N}\sum_{\epsilon\in\{+,-\}}p_{k}^{\epsilon}\cdot\exp\left(aq^{\top}\mu_{k}^{\epsilon}-\frac{a^{2}}{2}\cdot\left\|\mu_{k}^{\epsilon}\right\|^{2}\right)}. w ℓ ± ( a , q ) := ∑ k = 1 N ∑ ϵ ∈ { + , − } p k ϵ ⋅ exp ( a q ⊤ μ k ϵ − 2 a 2 ⋅ ∥ μ k ϵ ∥ 2 ) p ℓ + ⋅ exp ( a q ⊤ μ ℓ ± − 2 a 2 ⋅ μ ℓ ± 2 ) .
Here the probability weights p j ± p_{j}^{\pm} p j ± ’s are as in (E.1). One can think of m \boldsymbol{m} m as the posterior mean since ∑ ℓ = 1 N w ℓ + ( a , q ) + w ℓ − ( a , q ) = 1 \sum_{\ell=1}^{N}w_{\ell}^{+}(a,q)+w_{\ell}^{-}(a,q)=1 ∑ ℓ = 1 N w ℓ + ( a , q ) + w ℓ − ( a , q ) = 1 for all ( a , q ) (a,q) ( a , q ) .
From these representations, we get
∂ a q ( a , x ) = m ( a , q ( a , x ) ) , q ( 0 , x ) = x . (62) \partial_{a}\boldsymbol{q}(a,\boldsymbol{x})=\boldsymbol{m}(a,\boldsymbol{q}(a,\boldsymbol{x})),\qquad\boldsymbol{q}(0,\boldsymbol{x})=\boldsymbol{x}. \tag*{(62)} ∂ a q ( a , x ) = m ( a , q ( a , x )) , q ( 0 , x ) = x . (62)
Thus, taking the Jacobian D x D_{\boldsymbol{x}} D x in the above yields
∂ a K ( a , x ) = a ⋅ C ( a , q ( a , x ) ) ⋅ K ( a , x ) , K ( 0 , x ) = I 3 (63) \partial_{a}K(a,\boldsymbol{x})=a\cdot C(a,\boldsymbol{q}(a,\boldsymbol{x}))\cdot K(a,\boldsymbol{x}),\qquad K(0,\boldsymbol{x})=I_{3} \tag*{(63)} ∂ a K ( a , x ) = a ⋅ C ( a , q ( a , x )) ⋅ K ( a , x ) , K ( 0 , x ) = I 3 (63)
where
C ( a , q ) = ∑ ℓ = 1 N ∑ ϵ ∈ { + , − } w ℓ ϵ ( a , q ) ⋅ [ μ ℓ ϵ − m ( a , q ) ] ⋅ [ μ ℓ ϵ − m ( a , q ) ] ⊤ . C(a,q)=\sum_{\ell=1}^{N}\sum_{\epsilon\in\{+,-\}}w_{\ell}^{\epsilon}(a,q)\cdot\left[\mu_{\ell}^{\epsilon}-\boldsymbol{m}(a,q)\right]\cdot\left[\mu_{\ell}^{\epsilon}-\boldsymbol{m}(a,q)\right]^{\top}. C ( a , q ) = ℓ = 1 ∑ N ϵ ∈ { + , − } ∑ w ℓ ϵ ( a , q ) ⋅ [ μ ℓ ϵ − m ( a , q ) ] ⋅ [ μ ℓ ϵ − m ( a , q ) ] ⊤ .
To finish the proof, we will show that K ( 1 , ⋅ ) K(1,\cdot) K ( 1 , ⋅ ) has two distinct negative eigenvalues at some point, for all sufficiently large N , B N,B N , B .
Step 2: Covariance concentration. Define
L : = 2 B η , x ⋆ = ( 0 , 0 , L ) = L e 3 . L:=\frac{2B}{\eta},\qquad\boldsymbol{x}_{\star}=(0,0,L)=L\boldsymbol{e}_{3}. L := η 2 B , x ⋆ = ( 0 , 0 , L ) = L e 3 .
Starting at x ⋆ \boldsymbol{x}_{\star} x ⋆ , the ODE in (E.4) has an explicit solution given by
q ( a , x ⋆ ) = ( 0 , 0 , Q ( a ) ) = Q ( a ) ⋅ e 3 (64) \boldsymbol{q}(a,\boldsymbol{x}_{\star})=(0,0,Q(a))=Q(a)\cdot\boldsymbol{e}_{3} \tag*{(64)} q ( a , x ⋆ ) = ( 0 , 0 , Q ( a )) = Q ( a ) ⋅ e 3 (64)
where Q Q Q solves the one-dimensional ODE
Q ′ ( a ) = η 2 N ⋅ ∑ j = 1 N [ ( 2 j − 1 ) ⋅ α j ( a , Q ( a ) ) ] = η ⋅ ∑ j = 1 N [ θ j ⋅ α j ( a , Q ( a ) ) ] , Q ( 0 ) = L , (65) Q'(a)=\frac{\eta}{2N}\cdot\sum_{j=1}^{N}\left[(2j-1)\cdot\alpha_{j}(a,Q(a))\right]=\eta\cdot\sum_{j=1}^{N}\left[\theta_{j}\cdot\alpha_{j}(a,Q(a))\right],\qquad Q(0)=L, \tag*{(65)} Q ′ ( a ) = 2 N η ⋅ j = 1 ∑ N [ ( 2 j − 1 ) ⋅ α j ( a , Q ( a )) ] = η ⋅ j = 1 ∑ N [ θ j ⋅ α j ( a , Q ( a )) ] , Q ( 0 ) = L , (65)
with
α j ( a , Q ( a ) ) : = 2 w j + ( a , Q ( a ) ⋅ e 3 ) = exp ( − B θ j 2 + a η ⋅ Q ( a ) ⋅ θ j − a 2 η 2 2 ⋅ θ j 2 ) ∑ k = 1 N exp ( − B θ k 2 + a η ⋅ Q ( a ) ⋅ θ k − a 2 η 2 2 ⋅ θ k 2 ) . \alpha_{j}(a,Q(a)):=2w_{j}^{+}(a,Q(a)\cdot\boldsymbol{e}_{3})=\frac{\exp\left(-B\theta_{j}^{2}+a\eta\cdot Q(a)\cdot\theta_{j}-\frac{a^{2}\eta^{2}}{2}\cdot\theta_{j}^{2}\right)}{\sum_{k=1}^{N}\exp\left(-B\theta_{k}^{2}+a\eta\cdot Q(a)\cdot\theta_{k}-\frac{a^{2}\eta^{2}}{2}\cdot\theta_{k}^{2}\right)}. α j ( a , Q ( a )) := 2 w j + ( a , Q ( a ) ⋅ e 3 ) = ∑ k = 1 N exp ( − B θ k 2 + a η ⋅ Q ( a ) ⋅ θ k − 2 a 2 η 2 ⋅ θ k 2 ) exp ( − B θ j 2 + a η ⋅ Q ( a ) ⋅ θ j − 2 a 2 η 2 ⋅ θ j 2 ) .
In fact, one can derive the ODE in (E.7) and the solution in (E.6) by observing that
∂ a q ( a , x ⋆ ) = m ( a , Q ( a ) ⋅ e 3 ) = ∑ j = 1 N α j ( a , Q ( a ) ) 2 ⋅ [ ( R u π θ j 2 η θ j ) + ( − R u π θ j 2 η θ j ) ] = ( 0 0 η ⋅ ∑ j = 1 N α j ( a , Q ( a ) ) ⋅ θ j ) . \begin{aligned}
\partial_{a}\boldsymbol{q}(a,\boldsymbol{x}_{\star})=\boldsymbol{m}(a,Q(a)\cdot\boldsymbol{e}_{3})=\sum_{j=1}^{N}\frac{\alpha_{j}(a,Q(a))}{2}\cdot\left[\begin{pmatrix}Ru\pi\theta_{j}^{2}\\ \eta\theta_{j}\end{pmatrix}+\begin{pmatrix}-Ru\pi\theta_{j}^{2}\\ \eta\theta_{j}\end{pmatrix}\right] \\
&=\begin{pmatrix}
0\\
0\\
\eta\cdot\sum_{j=1}^{N}\alpha_{j}(a,Q(a))\cdot\theta_{j}
\end{pmatrix}.
\end{aligned} ∂ a q ( a , x ⋆ ) = m ( a , Q ( a ) ⋅ e 3 ) = j = 1 ∑ N 2 α j ( a , Q ( a )) ⋅ [ ( R u π θ j 2 η θ j ) + ( − R u π θ j 2 η θ j ) ] = 0 0 η ⋅ ∑ j = 1 N α j ( a , Q ( a )) ⋅ θ j .
Plug ((E.6)) into the formula of C C C , we get
C ( a , q ( a , x ⋆ ) ) = ( R 2 ⋅ ∑ j = 1 N α j ( a ) ⋅ P π θ j 2 0 2 × 1 0 1 × 2 η 2 ⋅ Var α ( a ) ( { θ j } 1 ≤ j ≤ N ) ) C(a,q(a,x_\star))=
\begin{pmatrix}
R^2\cdot\sum_{j=1}^{N}\alpha_j(a)\cdot P_{\pi\theta_j^2} & 0_{2\times1}\\
0_{1\times2} & \eta^2\cdot\operatorname{Var}_{\alpha(a)}\left(\{\theta_j\}_{1\leq j\leq N}\right)
\end{pmatrix} C ( a , q ( a , x ⋆ )) = ( R 2 ⋅ ∑ j = 1 N α j ( a ) ⋅ P π θ j 2 0 1 × 2 0 2 × 1 η 2 ⋅ Var α ( a ) ( { θ j } 1 ≤ j ≤ N ) )
where we used the short-hand notation α j ( a ) : = α j ( a , Q ( a ) ) \alpha_j(a):=\alpha_j(a,Q(a)) α j ( a ) := α j ( a , Q ( a )) and α ( a ) : = ( α 1 ( a ) , … , α N ( a ) ) \alpha(a):=(\alpha_1(a),\ldots,\alpha_N(a)) α ( a ) := ( α 1 ( a ) , … , α N ( a )) .
Due to the form of C ( a , q ( a , x ⋆ ) ) C(a,q(a,x_\star)) C ( a , q ( a , x ⋆ )) , the matrix-valued ODE in ((E.5)) has solution of block type
K ( a , x ⋆ ) = ( M ~ ( a , x ⋆ ) 0 2 × 1 0 1 × 2 k ( a ) ) K(a,x_\star)=
\begin{pmatrix}
\widetilde{M}(a,x_\star) & 0_{2\times1}\\
0_{1\times2} & k(a)
\end{pmatrix} K ( a , x ⋆ ) = ( M ( a , x ⋆ ) 0 1 × 2 0 2 × 1 k ( a ) )
where M ~ ( a , x ⋆ ) \widetilde{M}(a,x_\star) M ( a , x ⋆ ) and k ( a ) k(a) k ( a ) solve the ODEs
d d a M ~ ( a , x ⋆ ) = a R 2 ⋅ ( ∑ j = 1 N α j ( a ) ⋅ P π θ j 2 ) ⋅ M ~ ( a , x ⋆ ) , M ~ ( 0 , x ⋆ ) = I 2 , \frac{d}{da}\widetilde{M}(a,x_\star)
=aR^2\cdot\left(\sum_{j=1}^{N}\alpha_j(a)\cdot P_{\pi\theta_j^2}\right)\cdot\widetilde{M}(a,x_\star),
\qquad\widetilde{M}(0,x_\star)=I_2, d a d M ( a , x ⋆ ) = a R 2 ⋅ ( j = 1 ∑ N α j ( a ) ⋅ P π θ j 2 ) ⋅ M ( a , x ⋆ ) , M ( 0 , x ⋆ ) = I 2 ,
and
k ′ ( a ) = a η 2 ⋅ Var α ( a ) ( { θ j } 1 ≤ j ≤ N ) ⋅ k ( a ) , k ( 0 ) = 1 , k'(a)=a\eta^2\cdot\operatorname{Var}_{\alpha(a)}\left(\{\theta_j\}_{1\leq j\leq N}\right)\cdot k(a),
\qquad k(0)=1, k ′ ( a ) = a η 2 ⋅ Var α ( a ) ( { θ j } 1 ≤ j ≤ N ) ⋅ k ( a ) , k ( 0 ) = 1 ,
respectively.
With M ∈ C 1 ( [ 0 , 1 ] , GL 2 ( R ) ) M\in C^1([0,1],\operatorname{GL}_2(\mathbb{R})) M ∈ C 1 ([ 0 , 1 ] , GL 2 ( R )) being the solution of the ODE in Lemma E.1, Lemma H.2 (applied with C C C being the matrix in the ODE of M ~ \widetilde{M} M ) yields
∥ M ~ ( 1 , x ⋆ ) − M ( 1 , x ⋆ ) ∥ ≤ exp ( R 2 / 2 ) ⋅ { exp [ R 2 2 ⋅ sup a ∈ [ 0 , 1 ] ∥ ∑ j = 1 N α j ( a ) ⋅ P π θ j 2 − P π a 2 ∥ ] − 1 } ≤ exp ( 2 π + ε 16 ) ⋅ { exp [ ( 2 π + ε 16 ) ⋅ ( π N + π 2 B + π ε 4 B ) ] − 1 } . (66) \begin{aligned}
\left\|\widetilde{M}(1,x_\star)-M(1,x_\star)\right\|
\leq\exp(R^2/2)\cdot\left\{\exp\left[\frac{R^2}{2}\cdot\sup_{a\in[0,1]}\left\|\sum_{j=1}^{N}\alpha_j(a)\cdot P_{\pi\theta_j^2}-P_{\pi a^2}\right\|\right]-1\right\} \\
&\leq\exp\left(2\pi+\frac{\varepsilon}{16}\right)\cdot\left\{\exp\left[\left(2\pi+\frac{\varepsilon}{16}\right)\cdot\left(\frac{\pi}{N}+\frac{\pi\sqrt{2}}{\sqrt{B}}+\frac{\pi\varepsilon}{4B}\right)\right]-1\right\}. \tag*{(66)}
\end{aligned} M ( 1 , x ⋆ ) − M ( 1 , x ⋆ ) ≤ exp ( R 2 /2 ) ⋅ { exp [ 2 R 2 ⋅ a ∈ [ 0 , 1 ] sup j = 1 ∑ N α j ( a ) ⋅ P π θ j 2 − P π a 2 ] − 1 } ≤ exp ( 2 π + 16 ε ) ⋅ { exp [ ( 2 π + 16 ε ) ⋅ ( N π + B π 2 + 4 B π ε ) ] − 1 } . (66)
where the second bound comes from Lemma H.4 and that η 2 = ε / 4 \eta^2=\varepsilon/4 η 2 = ε /4 .
By choosing N , B N,B N , B sufficiently large so that the last term in ((E.8)) is sufficiently small, we conclude that M ~ ( 1 , x ⋆ ) \widetilde{M}(1,x_\star) M ( 1 , x ⋆ ) has two distinct negative eigenvalues for such ( N , B ) (N,B) ( N , B ) . Keep in mind that k ( 1 ) > 0 k(1)>0 k ( 1 ) > 0 for finite N , B N,B N , B , so D x Z P 1 ( x ⋆ ) = K ( 1 , x ⋆ ) D_xZ^{\mathrm{P1}}(x_\star)=K(1,x_\star) D x Z P1 ( x ⋆ ) = K ( 1 , x ⋆ ) has two distinct negative eigenvalues and one positive eigenvalue. The proof is completed. □
Proofs of approximate straightness results
Proof of Proposition 5.1
Let
Γ : = 1 4 log ( L / m ) , ρ : = ( m L ) − 1 / 4 and η : = [ Γ 2 − π 2 ] + . \Gamma:=\frac{1}{4}\log(L/m),\qquad\rho:=(mL)^{-1/4}\qquad\text{and}\qquad\eta:=\sqrt{\left[\Gamma^2-\pi^2\right]_+}. Γ := 4 1 log ( L / m ) , ρ := ( m L ) − 1/4 and η := [ Γ 2 − π 2 ] + .
We first show that if non-injectivity happens for some t ∈ [ 0 , 1 ] t\in[0,1] t ∈ [ 0 , 1 ] , then
π 2 + log 2 ( 1 − t t ρ ) ≤ Γ . \sqrt{\pi^2+\log^2\left(\frac{1-t}{t\rho}\right)}\leq\Gamma. π 2 + log 2 ( tρ 1 − t ) ≤ Γ.
Towards that end, suppose that H t ( x ) = H t ( y ) \mathbf{H}_t(\mathbf{x})=\mathbf{H}_t(\mathbf{y}) H t ( x ) = H t ( y ) for some distinct x , y \mathbf{x},\mathbf{y} x , y and some t ∈ ( 0 , 1 ) t\in(0,1) t ∈ ( 0 , 1 ) . Then, as argued in (12) ,
δ 1 ( x , y ) = − 1 − t t ⋅ δ 0 ( x , y ) . \delta_1(\mathbf{x},\mathbf{y})=-\frac{1-t}{t}\cdot\delta_0(\mathbf{x},\mathbf{y}). δ 1 ( x , y ) = − t 1 − t ⋅ δ 0 ( x , y ) .
For any s ∈ [ 0 , 1 ] s\in[0,1] s ∈ [ 0 , 1 ] , write u s : = δ s ( x , y ) \mathbf{u}_s:=\delta_s(\mathbf{x},\mathbf{y}) u s := δ s ( x , y ) and n s : = u s / ∥ u s ∥ 2 \mathbf{n}_s:=\mathbf{u}_s/\lVert\mathbf{u}_s\rVert_2 n s := u s / ∥ u s ∥ 2 . Then n 1 = − n 0 \mathbf{n}_1=-\mathbf{n}_0 n 1 = − n 0 and
∥ u 1 ∥ 2 ∥ u 0 ∥ 2 = 1 − t t . \frac{\lVert\mathbf{u}_1\rVert_2}{\lVert\mathbf{u}_0\rVert_2}=\frac{1-t}{t}. ∥ u 0 ∥ 2 ∥ u 1 ∥ 2 = t 1 − t .
Define
b s : = f m ( s ) + f L ( s ) 2 , and r s : = f m ( s ) − f L ( s ) 2 , b_s:=\frac{f_m(s)+f_L(s)}{2},\qquad\text{and}\qquad r_s:=\frac{f_m(s)-f_L(s)}{2}, b s := 2 f m ( s ) + f L ( s ) , and r s := 2 f m ( s ) − f L ( s ) ,
where, for any a > 0 a>0 a > 0 , f a f_a f a is the function
s ∈ [ 0 , 1 ] ↦ f a ( s ) : = s − a ( 1 − s ) s 2 + a ( 1 − s ) 2 . s\in[0,1]\mapsto f_a(s):=\frac{s-a(1-s)}{s^2+a(1-s)^2}. s ∈ [ 0 , 1 ] ↦ f a ( s ) := s 2 + a ( 1 − s ) 2 s − a ( 1 − s ) .
Let A s A_s A s be the matrix in (14) and set a s : = n s T A s n s a_s:=\mathbf{n}_s^{\mathsf T}A_s\mathbf{n}_s a s := n s T A s n s . Using the normalized-vector calculation in the proof of Theorem 3.1, we obtain
a s = u s T A s u s ∥ u s ∥ 2 2 = 1 2 ∥ u s ∥ 2 2 ⋅ d d s ( ∥ u s ∥ 2 2 ) = d d s ( log ∥ u s ∥ 2 ) . a_s=\frac{\mathbf{u}_s^{\mathsf T}A_s\mathbf{u}_s}{\lVert\mathbf{u}_s\rVert_2^2}
=\frac{1}{2\lVert\mathbf{u}_s\rVert_2^2}\cdot\frac{d}{ds}\left(\lVert\mathbf{u}_s\rVert_2^2\right)
=\frac{d}{ds}\left(\log\lVert\mathbf{u}_s\rVert_2\right). a s = ∥ u s ∥ 2 2 u s T A s u s = 2 ∥ u s ∥ 2 2 1 ⋅ d s d ( ∥ u s ∥ 2 2 ) = d s d ( log ∥ u s ∥ 2 ) .
where the second equality follows from the second equality in (B.4). The above display also yields
∫ 0 1 a s d s = { log ∥ u s ∥ 2 } s = 0 s = 1 = log ( ∥ u 1 ∥ 2 ∥ u 0 ∥ 2 ) . \int_0^1a_s\,ds=\left\{\log\lVert\mathbf{u}_s\rVert_2\right\}_{s=0}^{s=1}
=\log\left(\frac{\lVert\mathbf{u}_1\rVert_2}{\lVert\mathbf{u}_0\rVert_2}\right). ∫ 0 1 a s d s = { log ∥ u s ∥ 2 } s = 0 s = 1 = log ( ∥ u 0 ∥ 2 ∥ u 1 ∥ 2 ) .
By using the ODE (B.5) and the fact that n s \mathbf{n}_s n s is orthogonal to n ˙ s \dot{\mathbf{n}}_s n ˙ s , we have
∥ ( A s − b s I d ) n s ∥ 2 2 = ∥ n ˙ s ∥ 2 2 + ( a s − b s ) 2 ⋅ ∥ n s ∥ 2 2 = ∥ n ˙ s ∥ 2 2 + ( a s − b s ) 2 . \lVert(A_s-b_sI_d)\mathbf{n}_s\rVert_2^2
=\lVert\dot{\mathbf{n}}_s\rVert_2^2+(a_s-b_s)^2\cdot\lVert\mathbf{n}_s\rVert_2^2
=\lVert\dot{\mathbf{n}}_s\rVert_2^2+(a_s-b_s)^2. ∥( A s − b s I d ) n s ∥ 2 2 = ∥ n ˙ s ∥ 2 2 + ( a s − b s ) 2 ⋅ ∥ n s ∥ 2 2 = ∥ n ˙ s ∥ 2 2 + ( a s − b s ) 2 .
Additionally, the bound f L ( s ) ⋅ I d ⪯ A s ⪯ f m ( s ) ⋅ I d f_L(s)\cdot I_d\preceq A_s\preceq f_m(s)\cdot I_d f L ( s ) ⋅ I d ⪯ A s ⪯ f m ( s ) ⋅ I d (due to Lemma B.3 ) yields
− r s ⋅ I d ⪯ A s − b s ⋅ I d ⪯ r s ⋅ I d . -r_s\cdot I_d\preceq A_s-b_s\cdot I_d\preceq r_s\cdot I_d. − r s ⋅ I d ⪯ A s − b s ⋅ I d ⪯ r s ⋅ I d .
Consequently,
r s 2 ≥ ∥ ( A s − b s I d ) n s ∥ 2 2 ≥ ∥ n ˙ s ∥ 2 2 + ( a s − b s ) 2 . (67) r_s^2\geq\lVert(A_s-b_sI_d)\mathbf{n}_s\rVert_2^2\geq\lVert\dot{\mathbf{n}}_s\rVert_2^2+(a_s-b_s)^2. \tag*{(67)} r s 2 ≥ ∥( A s − b s I d ) n s ∥ 2 2 ≥ ∥ n ˙ s ∥ 2 2 + ( a s − b s ) 2 . (67)
Moreover, since n 1 = − n 0 \mathbf{n}_1=-\mathbf{n}_0 n 1 = − n 0 , the fact that the geodesic distance achieves the shortest length among all smooth curves connecting two points gives
π ≤ ∫ 0 1 ∥ n ˙ s ∥ 2 d s . (68) \pi\leq\int_0^1\lVert\dot{\mathbf{n}}_s\rVert_2\,ds. \tag*{(68)} π ≤ ∫ 0 1 ∥ n ˙ s ∥ 2 d s . (68)
Putting the bounds together, we get from triangle inequality (for vector-valued integral) that
Γ ≥ ( i ) ∫ 0 1 r s d s ≥ ( i i ) ∫ 0 1 ∥ n ˙ s ∥ 2 2 + ( a s − b s ) 2 d s ≥ ( i i i ) ( ∫ 0 1 ∥ n ˙ s ∥ 2 d s ) 2 + [ ∫ 0 1 ( a s − b s ) d s ] 2 ≥ ( i v ) π 2 + [ ∫ 0 1 ( a s − b s ) d s ] 2 . \begin{aligned}
\Gamma\mathop{\geq}^{(i)}\int_0^1r_s\,ds
&\mathop{\geq}^{(ii)}\int_0^1\sqrt{\lVert\dot{\mathbf{n}}_s\rVert_2^2+(a_s-b_s)^2}\,ds \\
&\mathop{\geq}^{(iii)}\sqrt{\left(\int_0^1\lVert\dot{\mathbf{n}}_s\rVert_2\,ds\right)^2+\left[\int_0^1(a_s-b_s)\,ds\right]^2} \\
&\mathop{\geq}^{(iv)}\sqrt{\pi^2+\left[\int_0^1(a_s-b_s)\,ds\right]^2}.
\end{aligned} Γ ≥ ( i ) ∫ 0 1 r s d s ≥ ( ii ) ∫ 0 1 ∥ n ˙ s ∥ 2 2 + ( a s − b s ) 2 d s ≥ ( iii ) ( ∫ 0 1 ∥ n ˙ s ∥ 2 d s ) 2 + [ ∫ 0 1 ( a s − b s ) d s ] 2 ≥ ( i v ) π 2 + [ ∫ 0 1 ( a s − b s ) d s ] 2 .
Here (i) follows from a simple calculation, (ii) follows from (F.1), (iv) follows from (F.2) and (iii) follows from Minkowski’s integral inequality (Theorem 6.19 in Folland [ [18 ] ): for the vector-valued function s ∈ [ 0 , 1 ] ↦ w s : = ( ∥ n ˙ s ∥ 2 , a s − b s ) T s \in[0,1] \mapsto w_s := (\lVert\dot{n}_s\rVert_2,a_s-b_s)^{\mathsf{T}} s ∈ [ 0 , 1 ] ↦ w s := (∥ n ˙ s ∥ 2 , a s − b s ) T , Minkowski’s integral inequality for the Euclidean norm (and integrating over the space { 1 , 2 } × [ 0 , 1 ] \{1,2\}\times[0,1] { 1 , 2 } × [ 0 , 1 ] with respect to the product of the counting and Lebesgue measures) yields
( ∫ 0 1 ∥ n ˙ s ∥ 2 d s ) 2 + [ ∫ 0 1 ( a s − b s ) d s ] 2 = ∥ ∫ 0 1 w s d s ∥ ≤ ∫ 0 1 ∥ w s ∥ 2 d s = ∫ 0 1 ∥ n ˙ s ∥ 2 2 + ( a s − b s ) 2 d s . \sqrt{\left(\int_0^1 \lVert\dot{n}_s\rVert_2\,ds\right)^2+\left[\int_0^1(a_s-b_s)\,ds\right]^2}=\left\lVert\int_0^1w_s\,ds\right\rVert\leq\int_0^1\lVert w_s\rVert_2\,ds=\int_0^1\sqrt{\lVert\dot{n}_s\rVert_2^2+(a_s-b_s)^2}\,ds. ( ∫ 0 1 ∥ n ˙ s ∥ 2 d s ) 2 + [ ∫ 0 1 ( a s − b s ) d s ] 2 = ∫ 0 1 w s d s ≤ ∫ 0 1 ∥ w s ∥ 2 d s = ∫ 0 1 ∥ n ˙ s ∥ 2 2 + ( a s − b s ) 2 d s .
Let us evaluate the integral on the right-hand side of (iv). Write
∫ 0 1 ( a s − b s ) d s = ∫ 0 1 a s d s − ∫ 0 1 b s d s = log ( ∥ u 1 ∥ 2 ∥ u 0 ∥ 2 ) − log ρ = log ( 1 − t t ρ ) . \int_0^1(a_s-b_s)\,ds=\int_0^1a_s\,ds-\int_0^1b_s\,ds=\log\left(\frac{\lVert u_1\rVert_2}{\lVert u_0\rVert_2}\right)-\log\rho=\log\left(\frac{1-t}{t\rho}\right). ∫ 0 1 ( a s − b s ) d s = ∫ 0 1 a s d s − ∫ 0 1 b s d s = log ( ∥ u 0 ∥ 2 ∥ u 1 ∥ 2 ) − log ρ = log ( tρ 1 − t ) .
Therefore,
Γ ≥ π 2 + log 2 ( 1 − t t ρ ) . \Gamma\geq\sqrt{\pi^2+\log^2\left(\frac{1-t}{t\rho}\right)}. Γ ≥ π 2 + log 2 ( tρ 1 − t ) .
Consequently, every non-injective time belongs to
I : = [ 1 1 + ρ e η , 1 1 + ρ e − η ] . I:=\left[\frac{1}{1+\rho e^\eta},\frac{1}{1+\rho e^{-\eta}}\right]. I := [ 1 + ρ e η 1 , 1 + ρ e − η 1 ] .
Finally, put
D : = Z 1 ( Z 0 ) − Z 0 , H t : = H t ( Z 0 ) and S t : = E ∥ D − E [ D ∣ H t ] ∥ 2 2 . D:=Z_1(Z_0)-Z_0,\qquad H_t:=H_t(Z_0)\qquad\text{and}\qquad S_t:=\mathbb{E}\lVert D-\mathbb{E}[D\mid H_t]\rVert_2^2. D := Z 1 ( Z 0 ) − Z 0 , H t := H t ( Z 0 ) and S t := E ∥ D − E [ D ∣ H t ] ∥ 2 2 .
By Definition 2, S ( Z ) = ∫ 0 1 S t d t S(\mathcal{Z})=\int_0^1S_t\,dt S ( Z ) = ∫ 0 1 S t d t . Outside I I I , the continuous injective map H t H_t H t has a measurable inverse on its image, so S t = 0 S_t=0 S t = 0 . For t > 0 t>0 t > 0 , the identity D = ( H t − Z 0 ) / t D=(H_t-Z_0)/t D = ( H t − Z 0 ) / t gives
S t = 1 t 2 E ∥ Z 0 − E [ Z 0 ∣ H t ] ∥ 2 2 ≤ d t 2 . S_t=\frac{1}{t^2}\mathbb{E}\lVert Z_0-\mathbb{E}[Z_0\mid H_t]\rVert_2^2\leq\frac{d}{t^2}. S t = t 2 1 E ∥ Z 0 − E [ Z 0 ∣ H t ] ∥ 2 2 ≤ t 2 d .
Therefore,
S ( Z ) ≤ d ∫ I d t t 2 = d ρ ( e η − e − η ) = 2 d ρ sinh η . S(\mathcal{Z})\leq d\int_I\frac{dt}{t^2}=d\rho(e^\eta-e^{-\eta})=2d\rho\sinh\eta. S ( Z ) ≤ d ∫ I t 2 d t = d ρ ( e η − e − η ) = 2 d ρ sinh η .
which is the asserted bound. This also covers Γ ≤ π \Gamma\leq\pi Γ ≤ π , since then I I I is a singleton. □
Proof of Theorem 5.1
To simplify notation, we suppress the superscript K K K throughout the proof. Write
D = Z 1 − Z 0 , H t = ( 1 − t ) Z 0 + t Z 1 . D=Z_1-Z_0,\qquad H_t=(1-t)Z_0+tZ_1. D = Z 1 − Z 0 , H t = ( 1 − t ) Z 0 + t Z 1 .
By the optimality of the conditional expectation, we have
S ( Z ) ≤ E ∫ 0 1 ∥ D − v t ( H t ) ∥ 2 2 d t . S(\mathcal{Z})\leq\mathbb{E}\int_0^1\lVert D-v_t(H_t)\rVert_2^2\,dt. S ( Z ) ≤ E ∫ 0 1 ∥ D − v t ( H t ) ∥ 2 2 d t .
Using Minkowski’s inequality with respect to the product measure P Z 0 , Z 1 ⊗ Unif [ 0 , 1 ] \mathbb{P}_{Z_0,Z_1} \otimes\operatorname{Unif}[0,1] P Z 0 , Z 1 ⊗ Unif [ 0 , 1 ] and the ℓ K \ell_K ℓ K -Lipschitz continuity of v t v_t v t , it follows that
S ( Z ) ≤ ( E ∫ 0 1 ∥ D − v t ( Z t ) ∥ 2 2 d t ) 1 / 2 + ℓ K ( E ∫ 0 1 ∥ Z t − H t ∥ 2 2 d t ) 1 / 2 = T + ℓ K ( E ∫ 0 1 ∥ Z t − H t ∥ 2 2 d t ) 1 / 2 . \sqrt{S(Z)} \leq\left(\mathbb{E}\int_0^1 \lVert D-v_t(Z_t)\rVert_2^2\,dt\right)^{1/2}+\ell_K\left(\mathbb{E}\int_0^1 \lVert Z_t-H_t\rVert_2^2\,dt\right)^{1/2}
= \sqrt{\mathcal{T}}+\ell_K\left(\mathbb{E}\int_0^1 \lVert Z_t-H_t\rVert_2^2\,dt\right)^{1/2}. S ( Z ) ≤ ( E ∫ 0 1 ∥ D − v t ( Z t ) ∥ 2 2 d t ) 1/2 + ℓ K ( E ∫ 0 1 ∥ Z t − H t ∥ 2 2 d t ) 1/2 = T + ℓ K ( E ∫ 0 1 ∥ Z t − H t ∥ 2 2 d t ) 1/2 .
Next, define h t = Z t − H t h_t=Z_t-H_t h t = Z t − H t . Since h 0 = h 1 = 0 h_0=h_1=0 h 0 = h 1 = 0 and
d d t h t = v t ( Z t ) − D . \frac{d}{dt}h_t=v_t(Z_t)-D. d t d h t = v t ( Z t ) − D .
Then, Wirtinger’s inequality (cf. Hardy et al. [ [22 ] , p. 184 – 187) yields
∫ 0 1 ∥ h t ∥ 2 2 d t ≤ 1 π 2 ∫ 0 1 ∥ v t ( Z t ) − D ∥ 2 2 d t . \int_0^1 \lVert h_t\rVert_2^2\,dt \leq\frac{1}{\pi^2}\int_0^1 \lVert v_t(Z_t)-D\rVert_2^2\,dt. ∫ 0 1 ∥ h t ∥ 2 2 d t ≤ π 2 1 ∫ 0 1 ∥ v t ( Z t ) − D ∥ 2 2 d t .
Taking expectations gives
E ∫ 0 1 ∥ Z t − H t ∥ 2 2 d t ≤ T π 2 . \mathbb{E}\int_0^1 \lVert Z_t-H_t\rVert_2^2\,dt \leq\frac{\mathcal{T}}{\pi^2}. E ∫ 0 1 ∥ Z t − H t ∥ 2 2 d t ≤ π 2 T .
Consequently,
S ( Z ) ≤ ( 1 + ℓ K π ) T , \sqrt{S(Z)} \leq\left(1+\frac{\ell_K}{\pi}\right)\sqrt{\mathcal{T}}, S ( Z ) ≤ ( 1 + π ℓ K ) T ,
which proves the claim. □
Proof of Theorem 5.2
Let n : = N n:=N n := N , and recall from the discussion preceding (25) that, conditional on F K − 1 \mathcal{F}_{K-1} F K − 1 (equivalently, conditional on the learned velocity field), the residuals { ψ i ( K ) } i = 1 n \{\psi_i^{(K)}\}_{i=1}^n { ψ i ( K ) } i = 1 n are i.i.d. with conditional mean
E [ ψ 1 ( K ) ∣ F K − 1 ] = T ( K ) . \mathbb{E}[\psi_1^{(K)}\mid\mathcal{F}_{K-1}]=\mathcal{T}^{(K)}. E [ ψ 1 ( K ) ∣ F K − 1 ] = T ( K ) .
Consistency. Assume E [ ∣ ψ 1 ( K ) ∣ ∣ F K − 1 ] < ∞ \mathbb{E}[|\psi_1^{(K)}|\mid\mathcal{F}_{K-1}]<\infty E [ ∣ ψ 1 ( K ) ∣ ∣ F K − 1 ] < ∞ almost surely. Then, by the conditional weak law of large numbers (applied on the conditional probability space given F K − 1 \mathcal{F}_{K-1} F K − 1 ),
T ^ N ( K ) = 1 n ∑ i = 1 n ψ i ( K ) ⟶ p E [ ψ 1 ( K ) ∣ F K − 1 ] = T ( K ) . \widehat{\mathcal{T}}_N^{(K)}=\frac{1}{n}\sum_{i=1}^n\psi_i^{(K)}\mathrel{\overset{p}{\longrightarrow}}\mathbb{E}[\psi_1^{(K)}\mid\mathcal{F}_{K-1}]=\mathcal{T}^{(K)}. T N ( K ) = n 1 i = 1 ∑ n ψ i ( K ) ⟶ p E [ ψ 1 ( K ) ∣ F K − 1 ] = T ( K ) .
Asymptotic normality. If in addition σ K 2 : = Var ( ψ 1 ( K ) ∣ F K − 1 ) < ∞ \sigma_K^2:=\operatorname{Var}(\psi_1^{(K)}\mid\mathcal{F}_{K-1})<\infty σ K 2 := Var ( ψ 1 ( K ) ∣ F K − 1 ) < ∞ almost surely, then by the (conditional) Lindeberg–Lévy central limit theorem,
n ( T ^ N ( K ) − T ( K ) ) σ K ⟶ d N ( 0 , 1 ) conditional on F K − 1 a.s. \frac{\sqrt{n}\left(\widehat{\mathcal{T}}_N^{(K)}-\mathcal{T}^{(K)}\right)}{\sigma_K}\mathrel{\overset{d}{\longrightarrow}}\mathcal{N}(0,1)\quad\text{conditional on }\mathcal{F}_{K-1}\text{ a.s.} σ K n ( T N ( K ) − T ( K ) ) ⟶ d N ( 0 , 1 ) conditional on F K − 1 a.s.
Since n = N n=N n = N , this yields the stated convergence. □
Proof of Proposition 5.2
Let ψ ˉ N , K : = T ^ N ( K ) = 1 N ∑ i = 1 N ψ i ( K ) \bar{\psi}_{N,K}:=\widehat{\mathcal{T}}_N^{(K)}=\frac{1}{N}\sum_{i=1}^N\psi_i^{(K)} ψ ˉ N , K := T N ( K ) = N 1 ∑ i = 1 N ψ i ( K ) . Conditional on F K − 1 \mathcal{F}_{K-1} F K − 1 , the variables { ψ i ( K ) } i ≥ 1 \{\psi_i^{(K)}\}_{i\geq1} { ψ i ( K ) } i ≥ 1 are i.i.d. with E [ ψ 1 ( K ) ∣ F K − 1 ] = T ( K ) \mathbb{E}[\psi_1^{(K)}\mid\mathcal{F}_{K-1}]=\mathcal{T}^{(K)} E [ ψ 1 ( K ) ∣ F K − 1 ] = T ( K ) and Var ( ψ 1 ( K ) ∣ F K − 1 ) = σ K 2 < ∞ \operatorname{Var}(\psi_1^{(K)}\mid\mathcal{F}_{K-1})=\sigma_K^2<\infty Var ( ψ 1 ( K ) ∣ F K − 1 ) = σ K 2 < ∞ .
Consistency of σ ^ N , K 2 \hat{\sigma}^{2}_{N,K} σ ^ N , K 2 . Using the standard identity,
σ ^ N , K 2 = 1 N − 1 ∑ i = 1 N ( ψ i ( K ) − ψ ˉ N , K ) 2 = N N − 1 ( 1 N ∑ i = 1 N ( ψ i ( K ) ) 2 − ψ ˉ N , K 2 ) . \hat{\sigma}^{2}_{N,K}=\frac{1}{N-1}\sum_{i=1}^{N}\left(\psi_{i}^{(K)}-\bar{\psi}_{N,K}\right)^{2}=\frac{N}{N-1}\left(\frac{1}{N}\sum_{i=1}^{N}\left(\psi_{i}^{(K)}\right)^{2}-\bar{\psi}_{N,K}^{2}\right). σ ^ N , K 2 = N − 1 1 i = 1 ∑ N ( ψ i ( K ) − ψ ˉ N , K ) 2 = N − 1 N ( N 1 i = 1 ∑ N ( ψ i ( K ) ) 2 − ψ ˉ N , K 2 ) .
By the (conditional) weak law of large numbers,
ψ ˉ N , K → p T ( K ) , 1 N ∑ i = 1 N ( ψ i ( K ) ) 2 → p E [ ( ψ 1 ( K ) ) 2 ∣ F K − 1 ] , \bar{\psi}_{N,K}\xrightarrow{p}\mathcal{T}^{(K)},\qquad\frac{1}{N}\sum_{i=1}^{N}\left(\psi_{i}^{(K)}\right)^{2}\xrightarrow{p}\mathbb{E}\left[\left(\psi_{1}^{(K)}\right)^{2}\mid\mathcal{F}_{K-1}\right], ψ ˉ N , K p T ( K ) , N 1 i = 1 ∑ N ( ψ i ( K ) ) 2 p E [ ( ψ 1 ( K ) ) 2 ∣ F K − 1 ] ,
where the second convergence uses E [ ( ψ 1 ( K ) ) 2 ∣ F K − 1 ] < ∞ \mathbb{E}\left[\left(\psi_{1}^{(K)}\right)^{2}\mid\mathcal{F}_{K-1}\right]<\infty E [ ( ψ 1 ( K ) ) 2 ∣ F K − 1 ] < ∞ almost everywhere. By the continuous mapping theorem, ψ ˉ N , K 2 → p ( T ( K ) ) 2 \bar{\psi}_{N,K}^{2}\xrightarrow{p}\left(\mathcal{T}^{(K)}\right)^{2} ψ ˉ N , K 2 p ( T ( K ) ) 2 , hence
1 N ∑ i = 1 N ( ψ i ( K ) ) 2 − ψ ˉ N , K 2 → p E [ ( ψ 1 ( K ) ) 2 ∣ F K − 1 ] − ( T ( K ) ) 2 = σ K 2 . \frac{1}{N}\sum_{i=1}^{N}\left(\psi_{i}^{(K)}\right)^{2}-\bar{\psi}_{N,K}^{2}\xrightarrow{p}\mathbb{E}\left[\left(\psi_{1}^{(K)}\right)^{2}\mid\mathcal{F}_{K-1}\right]-\left(\mathcal{T}^{(K)}\right)^{2}=\sigma_{K}^{2}. N 1 i = 1 ∑ N ( ψ i ( K ) ) 2 − ψ ˉ N , K 2 p E [ ( ψ 1 ( K ) ) 2 ∣ F K − 1 ] − ( T ( K ) ) 2 = σ K 2 .
Finally, N N − 1 → 1 \frac{N}{N-1}\to1 N − 1 N → 1 , so σ ^ N , K 2 → p σ K 2 \hat{\sigma}^{2}_{N,K}\xrightarrow{p}\sigma_{K}^{2} σ ^ N , K 2 p σ K 2 .
Studentization. By Theorem 5.2, conditional on F K − 1 \mathcal{F}_{K-1} F K − 1 ,
N ( ψ ˉ N , K − T ( K ) ) σ K → d N ( 0 , 1 ) . \frac{\sqrt{N}\left(\bar{\psi}_{N,K}-\mathcal{T}^{(K)}\right)}{\sigma_{K}}\xrightarrow{d}\mathcal{N}(0,1). σ K N ( ψ ˉ N , K − T ( K ) ) d N ( 0 , 1 ) .
Moreover, σ ^ N , K 2 → p σ K 2 \hat{\sigma}^{2}_{N,K}\xrightarrow{p}\sigma_{K}^{2} σ ^ N , K 2 p σ K 2 implies σ ^ N , K → p σ K \hat{\sigma}_{N,K}\xrightarrow{p}\sigma_{K} σ ^ N , K p σ K (if σ K = 0 \sigma_{K}=0 σ K = 0 , the conclusion is trivial). Therefore,
ψ ˉ N , K − T ( K ) σ ^ N , K / N = N ( ψ ˉ N , K − T ( K ) ) σ K ⋅ σ K σ ^ N , K → d N ( 0 , 1 ) \frac{\bar{\psi}_{N,K}-\mathcal{T}^{(K)}}{\hat{\sigma}_{N,K}/\sqrt{N}}=\frac{\sqrt{N}\left(\bar{\psi}_{N,K}-\mathcal{T}^{(K)}\right)}{\sigma_{K}}\cdot\frac{\sigma_{K}}{\hat{\sigma}_{N,K}}\xrightarrow{d}\mathcal{N}(0,1) σ ^ N , K / N ψ ˉ N , K − T ( K ) = σ K N ( ψ ˉ N , K − T ( K ) ) ⋅ σ ^ N , K σ K d N ( 0 , 1 )
by Slutsky’s theorem. □
Proof of Proposition 5.3
The results follow immediately from Theorem 5.2, Proposition 5.2, and the dominated convergence theorem (DCT).
Coverage probability. Indeed, due to Proposition 5.2 we have the following.
lim N → ∞ P ( T ( K ) ≤ U N , K ∣ F K − 1 ) = 1 − α . \lim_{N\to\infty}\mathbb{P}\left(\mathcal{T}^{(K)}\leq U_{N,K}\mid\mathcal{F}_{K-1}\right)=1-\alpha. N → ∞ lim P ( T ( K ) ≤ U N , K ∣ F K − 1 ) = 1 − α .
Now, let us denote by E K − 1 ( ⋅ ) \mathbb{E}_{K-1}(\cdot) E K − 1 ( ⋅ ) the expectation with respect to the randomness in F K − 1 \mathcal{F}_{K-1} F K − 1 . Since P ( T ( K ) ≤ U N , K ) = E K − 1 [ P ( T ( K ) ≤ U N , K ∣ F K − 1 ) ] \mathbb{P}\left(\mathcal{T}^{(K)}\leq U_{N,K}\right)=\mathbb{E}_{K-1}\left[\mathbb{P}\left(\mathcal{T}^{(K)}\leq U_{N,K}\mid\mathcal{F}_{K-1}\right)\right] P ( T ( K ) ≤ U N , K ) = E K − 1 [ P ( T ( K ) ≤ U N , K ∣ F K − 1 ) ] , we can directly apply DCT to conclude
lim N → ∞ P ( T ( K ) ≤ U N , K ) = E K − 1 [ lim N → ∞ P ( T ( K ) ≤ U N , K ∣ F K − 1 ) ] = 1 − α . \lim_{N\to\infty}\mathbb{P}\left(\mathcal{T}^{(K)}\leq U_{N,K}\right)=\mathbb{E}_{K-1}\left[\lim_{N\to\infty}\mathbb{P}\left(\mathcal{T}^{(K)}\leq U_{N,K}\mid\mathcal{F}_{K-1}\right)\right]=1-\alpha. N → ∞ lim P ( T ( K ) ≤ U N , K ) = E K − 1 [ N → ∞ lim P ( T ( K ) ≤ U N , K ∣ F K − 1 ) ] = 1 − α .
Type-I error control: Similar to before, conditional on F K − 1 \mathcal{F}_{K-1} F K − 1 , we have
P H 0 ( U N , K ≤ δ ∣ F K − 1 ) = P H 0 ( T ^ ( K ) − T ( K ) σ ^ N , K / N ≤ − z 1 − α + δ − T ( K ) σ ^ N , K / N | F K − 1 ) ≤ P H 0 ( T ^ ( K ) − T ( K ) σ ^ N , K / N ≤ − z 1 − α | F K − 1 ) → N → ∞ α . (69) \begin{aligned}
\mathbb{P}_{H_0}(U_{N,K}\leq\delta\mid\mathcal{F}_{K-1})=\mathbb{P}_{H_0}\left(\frac{\widehat{T}^{(K)}-T^{(K)}}{\widehat{\sigma}_{N,K}/\sqrt{N}}\leq-z_{1-\alpha}+\frac{\delta-T^{(K)}}{\widehat{\sigma}_{N,K}/\sqrt{N}}\middle|\mathcal{F}_{K-1}\right) \\
&\leq\mathbb{P}_{H_0}\left(\frac{\widehat{T}^{(K)}-T^{(K)}}{\widehat{\sigma}_{N,K}/\sqrt{N}}\leq-z_{1-\alpha}\middle|\mathcal{F}_{K-1}\right)\xrightarrow{N\to\infty}\alpha. \tag*{(69)}
\end{aligned} P H 0 ( U N , K ≤ δ ∣ F K − 1 ) = P H 0 ( σ N , K / N T ( K ) − T ( K ) ≤ − z 1 − α + σ N , K / N δ − T ( K ) F K − 1 ) ≤ P H 0 ( σ N , K / N T ( K ) − T ( K ) ≤ − z 1 − α F K − 1 ) N → ∞ α . (69)
Therefore, we can again apply the DCT on (F.3) to conclude
lim N → ∞ P H 0 ( U N , K ≤ δ ) = E K − 1 [ lim N → ∞ P H 0 ( U N , K ≤ δ ∣ F K − 1 ) ] ≤ α . \lim_{N\to\infty}\mathbb{P}_{H_0}(U_{N,K}\leq\delta)=\mathbb{E}_{K-1}\left[\lim_{N\to\infty}\mathbb{P}_{H_0}(U_{N,K}\leq\delta\mid\mathcal{F}_{K-1})\right]\leq\alpha. N → ∞ lim P H 0 ( U N , K ≤ δ ) = E K − 1 [ N → ∞ lim P H 0 ( U N , K ≤ δ ∣ F K − 1 ) ] ≤ α .
Technical results used in the proof of Theorem 4.1
Endpoint maps under zero-temperature limit
Suppose W ∈ C 3 ( R 3 ) W\in C^3(\mathbb{R}^3) W ∈ C 3 ( R 3 ) such that (I) in Definition 3 holds. Let Y β , W Y_{\beta,W} Y β , W be the random vector taking values in R 3 \mathbb{R}^3 R 3 with law p 1 β ∝ exp ( − β W ) p_1^\beta\propto\exp(-\beta W) p 1 β ∝ exp ( − β W ) . The velocity field v t β ( z ) v_t^\beta(z) v t β ( z ) corresponding to the independent coupling N ( 0 , I 3 ) ⊗ p 1 β N(0,I_3)\otimes p_1^\beta N ( 0 , I 3 ) ⊗ p 1 β has the form
v t β ( z ) = E [ Y β , W − X | ( 1 − t ) X + t Y β , W = z ] , X ∼ N ( 0 , I 3 ) v_t^\beta(z)=\mathbb{E}\left[Y_{\beta,W}-X\middle|(1-t)X+tY_{\beta,W}=z\right],\qquad X\sim N(0,I_3) v t β ( z ) = E [ Y β , W − X ∣ ( 1 − t ) X + t Y β , W = z ] , X ∼ N ( 0 , I 3 )
Let Z t β , W ( x ) Z_t^{\beta,W}(x) Z t β , W ( x ) be the flow generated by v t β v_t^\beta v t β , starting at x x x . The corresponding endpoint map is Z 1 β , W Z_1^{\beta,W} Z 1 β , W , obtained by evaluating the flow at t = 1 t=1 t = 1 . We refer to the limit β → ∞ \beta\to\infty β → ∞ as the zero-temperature limit. We will show below in Proposition G.2(iii) that, under appropriate rescalings, both the endpoint map and its Jacobian converge as β → ∞ \beta\to\infty β → ∞ .
Define the posterior measure
π β , r , h ( d y ) = exp { − β ⋅ [ W ( y ) + 1 2 ∥ h − r y ∥ 2 ] } ∫ R 3 exp { − β ⋅ [ W ( y ) + 1 2 ∥ h − r y ∥ 2 ] } d y . \pi_{\beta,r,h}(dy)=\frac{\exp\left\{-\beta\cdot\left[W(y)+\frac{1}{2}\lVert h-ry\rVert^2\right]\right\}}{\int_{\mathbb{R}^3}\exp\left\{-\beta\cdot\left[W(y)+\frac{1}{2}\lVert h-ry\rVert^2\right]\right\}\,dy}. π β , r , h ( d y ) = ∫ R 3 exp { − β ⋅ [ W ( y ) + 2 1 ∥ h − r y ∥ 2 ] } d y exp { − β ⋅ [ W ( y ) + 2 1 ∥ h − r y ∥ 2 ] } .
Proposition G.1. Let r = r ( t ) : = t β ( 1 − t ) r=r(t):=\frac{t}{\sqrt{\beta(1-t)}} r = r ( t ) := β ( 1 − t ) t and set
h β ( r , x ) : = ( 1 + r β ) ⋅ Z 1 β , W ( β r 1 + β r ( β x ) ) β . h_\beta(r,x):=(1+r\sqrt{\beta})\cdot\frac{Z_1^{\beta,W}\left(\frac{\sqrt{\beta}r}{1+\sqrt{\beta}r}(\sqrt{\beta}x)\right)}{\sqrt{\beta}}. h β ( r , x ) := ( 1 + r β ) ⋅ β Z 1 β , W ( 1 + β r β r ( β x ) ) . It holds that
d d r h β ( r , x ) = y β ( r , h β ( r , x ) ) , h β ( 0 , x ) = x , y β = E Y ∼ π β , r , h [ Y ] . (70) \frac{d}{dr}h_\beta(r,x)=y_\beta(r,h_\beta(r,x)),\qquad h_\beta(0,x)=x,\qquad y_\beta=\mathbb{E}_{Y\sim\pi_{\beta,r,h}}[Y]. \tag*{(70)} d r d h β ( r , x ) = y β ( r , h β ( r , x )) , h β ( 0 , x ) = x , y β = E Y ∼ π β , r , h [ Y ] . (70) Z 1 β , W ( β x ) = lim r → ∞ h β ( r , x ) r , and β ⋅ D Z 1 β , W ( β x ) = lim r → ∞ D h β ( r , x ) r . Z_1^{\beta,W}(\sqrt{\beta}x)=\lim_{r\to\infty}\frac{h_\beta(r,x)}{r},\qquad\text{and}\qquad\sqrt{\beta}\cdot DZ_1^{\beta,W}(\sqrt{\beta}x)=\lim_{r\to\infty}\frac{Dh_\beta(r,x)}{r}. Z 1 β , W ( β x ) = r → ∞ lim r h β ( r , x ) , and β ⋅ D Z 1 β , W ( β x ) = r → ∞ lim r D h β ( r , x ) .
Proof of Proposition G.1. Fix β > 0 \beta> 0 β > 0 and write
t = t ( r ) : = β r 1 + β r , and d t d r = β ( 1 − t ) 2 . t=t(r):=\frac{\sqrt{\beta}\,r}{1+\sqrt{\beta}\,r}, \qquad\text{and} \qquad\frac{dt}{dr}=\sqrt{\beta}(1-t)^2. t = t ( r ) := 1 + β r β r , and d r d t = β ( 1 − t ) 2 .
By definition, we have
v t β ( z ) = y β ( r , h ) − z 1 − t . v_t^\beta(z)=\frac{\mathbf{y}_\beta(r,h)-z}{1-t}. v t β ( z ) = 1 − t y β ( r , h ) − z .
Thus
∂ r h β ( r , x ) = Z t β , W ( β x ) + ( 1 − t ) ⋅ v t β ( Z t β , W ( β x ) ) = y β ( r , h β ( r , x ) ) . \partial_r h_\beta(r,x)=Z_t^{\beta,W}(\sqrt{\beta}\,x)+(1-t)\cdot v_t^\beta\left(Z_t^{\beta,W}(\sqrt{\beta}\,x)\right)=y_\beta(r,h_\beta(r,x)). ∂ r h β ( r , x ) = Z t β , W ( β x ) + ( 1 − t ) ⋅ v t β ( Z t β , W ( β x ) ) = y β ( r , h β ( r , x )) .
Also,
h β ( 0 , x ) = Z 0 β , W ( β x ) β = x . h_\beta(0,x)=\frac{Z_0^{\beta,W}(\sqrt{\beta}\,x)}{\sqrt{\beta}}=x. h β ( 0 , x ) = β Z 0 β , W ( β x ) = x .
This proves (G.1). We now prove the second claim. Lemmas G.3 and G.5 give
∥ y β ( r , h ) ∥ ≤ E π β , r , h ∥ Y ∥ ≤ r ∥ h ∥ + ∥ ∇ W ( 0 ) ∥ κ + r 2 + 3 β ( κ + r 2 ) . (71) \lVert y_\beta(r,h)\rVert\leq\mathbb{E}_{\pi_{\beta,r,h}}\lVert Y\rVert\leq\frac{r\lVert h\rVert+\lVert\nabla W(0)\rVert}{\kappa+r^2}+\sqrt{\frac{3}{\beta(\kappa+r^2)}}. \tag*{(71)} ∥ y β ( r , h )∥ ≤ E π β , r , h ∥ Y ∥ ≤ κ + r 2 r ∥ h ∥ + ∥ ∇ W ( 0 )∥ + β ( κ + r 2 ) 3 . (71)
Together with the Lipschitz bound in Lemma G.5, this ensures that the ODE (G.1) has a unique global solution, and is C 1 C^1 C 1 with respect to x x x . We split the rest of the proof into a few steps. Step 1: Bounding ∥ h β ( r , x ) ∥ \lVert h_\beta(r,x)\rVert ∥ h β ( r , x )∥ . We will show that for all fixed x ∈ R 3 x\in\mathbb{R}^3 x ∈ R 3 ,
∥ h β ( r , x ) ∥ ≤ κ + r 2 κ ( ∥ x ∥ + ∥ ∇ W ( 0 ) ∥ κ + π 2 3 β ) . (72) \lVert h_\beta(r,x)\rVert\leq\sqrt{\frac{\kappa+r^2}{\kappa}}\left(\lVert x\rVert+\frac{\lVert\nabla W(0)\rVert}{\sqrt{\kappa}}+\frac{\pi}{2}\sqrt{\frac{3}{\beta}}\right). \tag*{(72)} ∥ h β ( r , x )∥ ≤ κ κ + r 2 ( ∥ x ∥ + κ ∥ ∇ W ( 0 )∥ + 2 π β 3 ) . (72)
To show this, put f ( r ) : = ∥ h β ( r , x ) ∥ f(r):=\lVert h_\beta(r,x)\rVert f ( r ) := ∥ h β ( r , x )∥ . Triangle inequality yields, for all δ > 0 \delta>0 δ > 0 ,
∣ f ( r + δ ) − f ( r ) ∣ ≤ ∥ h β ( r + δ , x ) − h β ( r , x ) ∥ ≤ ∫ r r + δ ∥ ∂ s h β ( s , x ) ∥ d s = ∫ r r + δ ∥ y β ( s , h β ( s , x ) ) ∥ d s . \begin{aligned}
\lvert f(r+\delta)-f(r)\rvert\leq\lVert h_\beta(r+\delta,x)-h_\beta(r,x)\rVert\leq\int_r^{r+\delta}\lVert\partial_s h_\beta(s,x)\rVert\,ds \\
&=\int_r^{r+\delta}\lVert y_\beta(s,h_\beta(s,x))\rVert\,ds.
\end{aligned} ∣ f ( r + δ ) − f ( r )∣ ≤ ∥ h β ( r + δ , x ) − h β ( r , x )∥ ≤ ∫ r r + δ ∥ ∂ s h β ( s , x )∥ d s = ∫ r r + δ ∥ y β ( s , h β ( s , x ))∥ d s .
Dividing both sides by δ \delta δ , letting δ → 0 \delta\to0 δ → 0 , and using (G.2), we deduce that, for almost every r > 0 r>0 r > 0 ,
∣ f ′ ( r ) ∣ ≤ ∥ y β ( r , h β ( r , x ) ) ∥ ≤ r ∥ h β ( r , x ) ∥ + ∥ ∇ W ( 0 ) ∥ κ + r 2 + 3 β ( κ + r 2 ) = r κ + r 2 ⋅ f ( r ) + ∥ ∇ W ( 0 ) ∥ κ + r 2 + 3 β ( κ + r 2 ) . \begin{aligned}
\lvert f'(r)\rvert\leq\lVert y_\beta(r,h_\beta(r,x))\rVert\leq\frac{r\lVert h_\beta(r,x)\rVert+\lVert\nabla W(0)\rVert}{\kappa+r^2}+\sqrt{\frac{3}{\beta(\kappa+r^2)}} \\
&=\frac{r}{\kappa+r^2}\cdot f(r)+\frac{\lVert\nabla W(0)\rVert}{\kappa+r^2}+\sqrt{\frac{3}{\beta(\kappa+r^2)}}.
\end{aligned} ∣ f ′ ( r )∣ ≤ ∥ y β ( r , h β ( r , x ))∥ ≤ κ + r 2 r ∥ h β ( r , x )∥ + ∥ ∇ W ( 0 )∥ + β ( κ + r 2 ) 3 = κ + r 2 r ⋅ f ( r ) + κ + r 2 ∥ ∇ W ( 0 )∥ + β ( κ + r 2 ) 3 .
Lemma G.1 then gives
f ( r ) ≤ ∥ x ∥ ⋅ exp { ∫ 0 r u κ + u 2 d u } + ∫ 0 r [ ∥ ∇ W ( 0 ) ∥ κ + s 2 + 3 β ( κ + s 2 ) ] ⋅ exp { ∫ s r u κ + u 2 d u } d s = ∥ x ∥ ⋅ κ + r 2 κ + κ + r 2 ⋅ ∫ 0 r [ ∥ ∇ W ( 0 ) ∥ ( κ + s 2 ) 3 / 2 + 3 / β κ + s 2 ] d s . \begin{aligned}
f(r)\leq\lVert x\rVert\cdot\exp\left\{\int_0^r\frac{u}{\kappa+u^2}\,du\right\}+\int_0^r\left[\frac{\lVert\nabla W(0)\rVert}{\kappa+s^2}+\sqrt{\frac{3}{\beta(\kappa+s^2)}}\right]\cdot\exp\left\{\int_s^r\frac{u}{\kappa+u^2}\,du\right\}\,ds \\
&=\lVert x\rVert\cdot\sqrt{\frac{\kappa+r^2}{\kappa}}+\sqrt{\kappa+r^2}\cdot\int_0^r\left[\frac{\lVert\nabla W(0)\rVert}{(\kappa+s^2)^{3/2}}+\frac{\sqrt{3/\beta}}{\kappa+s^2}\right]\,ds.
\end{aligned} f ( r ) ≤ ∥ x ∥ ⋅ exp { ∫ 0 r κ + u 2 u d u } + ∫ 0 r [ κ + s 2 ∥ ∇ W ( 0 )∥ + β ( κ + s 2 ) 3 ] ⋅ exp { ∫ s r κ + u 2 u d u } d s = ∥ x ∥ ⋅ κ κ + r 2 + κ + r 2 ⋅ ∫ 0 r [ ( κ + s 2 ) 3/2 ∥ ∇ W ( 0 )∥ + κ + s 2 3/ β ] d s .
Finally, the elementary bounds
∫ 0 r d s ( κ + s 2 ) 3 / 2 ≤ 1 κ , ∫ 0 r d s κ + s 2 ≤ π 2 κ , \int_{0}^{r} \frac{ds}{(\kappa+s^{2})^{3/2}} \le\frac{1}{\kappa}, \qquad\int_{0}^{r} \frac{ds}{\kappa+s^{2}} \le\frac{\pi}{2\sqrt{\kappa}}, ∫ 0 r ( κ + s 2 ) 3/2 d s ≤ κ 1 , ∫ 0 r κ + s 2 d s ≤ 2 κ π ,
give (G.3).
Step 2: Local convergence of h β h_{\beta} h β . By using Stein’s identity (see Lemma 2.2 in Liu et al. [ [37 ] )
E X ∼ p [ ∇ log p ( X ) ] = 0 \mathbb{E}_{X\sim p}[\nabla\log p(X)] = 0 E X ∼ p [ ∇ log p ( X )] = 0
on π β , r , h \pi_{\beta,r,h} π β , r , h , we obtain
E π β , r , h β ( r , x ) [ ∇ W ( Y ) ] + r 2 ⋅ y β ( r , h β ( r , x ) ) − r ⋅ h β ( r , x ) = 0. \mathbb{E}_{\pi_{\beta,r,h_{\beta}(r,x)}}[\nabla W(Y)] + r^{2}\cdot y_{\beta}(r,h_{\beta}(r,x)) - r\cdot h_{\beta}(r,x) = 0. E π β , r , h β ( r , x ) [ ∇ W ( Y )] + r 2 ⋅ y β ( r , h β ( r , x )) − r ⋅ h β ( r , x ) = 0.
Consequently, along the transformed flow,
∂ r [ h β ( r , x ) r ] = ∂ r h β ( r , x ) r − h β ( r , x ) r 2 = − 1 r 3 ⋅ E π β , r , h β ( r , x ) [ ∇ W ( Y ) ] . (73) \partial_{r}\left[\frac{h_{\beta}(r,x)}{r}\right] = \frac{\partial_{r}h_{\beta}(r,x)}{r} - \frac{h_{\beta}(r,x)}{r^{2}} = -\frac{1}{r^{3}}\cdot\mathbb{E}_{\pi_{\beta,r,h_{\beta}(r,x)}}[\nabla W(Y)]. \tag*{(73)} ∂ r [ r h β ( r , x ) ] = r ∂ r h β ( r , x ) − r 2 h β ( r , x ) = − r 3 1 ⋅ E π β , r , h β ( r , x ) [ ∇ W ( Y )] . (73)
Recall that y ( r , h ) y(r,h) y ( r , h ) is the mode of the posterior distribution π β , r , h \pi_{\beta,r,h} π β , r , h , defined as in (G.6) below. For any compact K ⊂ R 3 K \subset\mathbb{R}^{3} K ⊂ R 3 , (G.3) and Lemma G.5 yield
sup r ≥ 1 x ∈ K E π β , r , h β ( r , x ) ∥ Y ∥ ≤ sup r ≥ 1 x ∈ K ∥ y ( r , h β ( r , x ) ) ∥ + 3 β ( κ + 1 ) ≤ C K + 3 β ( κ + 1 ) < ∞ . \begin{aligned}
\sup_{\substack{r\ge1\\x\in K}}\mathbb{E}_{\pi_{\beta,r,h_{\beta}(r,x)}}\|Y\| \le\sup_{\substack{r\ge1\\x\in K}}\|y(r,h_{\beta}(r,x))\|+\sqrt{\frac{3}{\beta(\kappa+1)}} \\
&\le C_{K}+\sqrt{\frac{3}{\beta(\kappa+1)}}<\infty.
\end{aligned} r ≥ 1 x ∈ K sup E π β , r , h β ( r , x ) ∥ Y ∥ ≤ r ≥ 1 x ∈ K sup ∥ y ( r , h β ( r , x )) ∥ + β ( κ + 1 ) 3 ≤ C K + β ( κ + 1 ) 3 < ∞.
On the other hand, since
∥ ∇ W ( u ) ∥ ≤ ∥ ∇ W ( 0 ) ∥ + Λ ∥ u ∥ , \|\nabla W(u)\|\le\|\nabla W(0)\|+\Lambda\|u\|, ∥∇ W ( u ) ∥ ≤ ∥∇ W ( 0 ) ∥ + Λ∥ u ∥ ,
the expectation on the right-hand side of (G.4) is also uniformly bounded. Hence, for s ≥ R ≥ 1 s\ge R\ge1 s ≥ R ≥ 1 ,
sup x ∈ K ∥ h β ( s , x ) s − h β ( R , x ) R ∥ ≤ ∫ R s sup x ∈ K ∥ ∂ u [ h β ( u , x ) u ] ∥ d u ≤ C β , K , κ , Λ ∫ R s u − 3 d u ≤ C β , K , κ , Λ 2 R 2 . \begin{aligned}
\sup_{x\in K}\left\|\frac{h_{\beta}(s,x)}{s}-\frac{h_{\beta}(R,x)}{R}\right\|\le\int_{R}^{s}\sup_{x\in K}\left\|\partial_{u}\left[\frac{h_{\beta}(u,x)}{u}\right]\right\|du \\
&\le C_{\beta,K,\kappa,\Lambda}\int_{R}^{s}u^{-3}du\le\frac{C_{\beta,K,\kappa,\Lambda}}{2R^{2}}.
\end{aligned} x ∈ K sup s h β ( s , x ) − R h β ( R , x ) ≤ ∫ R s x ∈ K sup ∂ u [ u h β ( u , x ) ] d u ≤ C β , K , κ , Λ ∫ R s u − 3 d u ≤ 2 R 2 C β , K , κ , Λ .
Thus h β ( r , ⋅ ) / r h_{\beta}(r,\cdot)/r h β ( r , ⋅ ) / r converges locally uniformly.
By definition of h β h_{\beta} h β ,
Z t ( r ) β , W ( β x ) = t ( r ) h β ( r , x ) r . \mathsf{Z}_{t(r)}^{\beta,W}(\sqrt{\beta}\,x)=t(r)\frac{h_{\beta}(r,x)}{r}. Z t ( r ) β , W ( β x ) = t ( r ) r h β ( r , x ) .
Since t ( r ) → 1 t(r)\to1 t ( r ) → 1 , the preceding convergence establishes the continuous endpoint and gives
Z 1 β , W ( β x ) = lim t ( r ) → 1 Z t ( r ) β , W ( β x ) = lim r → ∞ h β ( r , x ) r , \mathsf{Z}_{1}^{\beta,W}(\sqrt{\beta}\,x)=\lim_{t(r)\to1}\mathsf{Z}_{t(r)}^{\beta,W}(\sqrt{\beta}\,x)=\lim_{r\to\infty}\frac{h_{\beta}(r,x)}{r}, Z 1 β , W ( β x ) = t ( r ) → 1 lim Z t ( r ) β , W ( β x ) = r → ∞ lim r h β ( r , x ) ,
locally uniformly in x x x .
Step 3: Local convergence of the normalized spatial derivatives. Put K β ( r , x ) : = D x h β ( r , x ) K_{\beta}(r,x):=D_{x}h_{\beta}(r,x) K β ( r , x ) := D x h β ( r , x ) . Differentiating (G.1) with respect to x x x gives
∂ r K β ( r , x ) = D h y β ( r , h β ( r , x ) ) ⋅ K β ( r , x ) , K β ( 0 , x ) = I 3 . \partial_{r}K_{\beta}(r,x)=D_{h}y_{\beta}(r,h_{\beta}(r,x))\cdot K_{\beta}(r,x),\qquad K_{\beta}(0,x)=I_{3}. ∂ r K β ( r , x ) = D h y β ( r , h β ( r , x )) ⋅ K β ( r , x ) , K β ( 0 , x ) = I 3 .
By Lemma G.5,
r Λ + r 2 I 3 ⪯ D h β ( r , h β ( r , x ) ) ⪯ r κ + r 2 I 3 . \frac{r}{\Lambda+r^2}I_3 \preceq Dh_{\beta}(r,h_{\beta}(r,\mathbf{x})) \preceq\frac{r}{\kappa+r^2}I_3. Λ + r 2 r I 3 ⪯ D h β ( r , h β ( r , x )) ⪯ κ + r 2 r I 3 .
Consequently, for every fixed v ∈ R 3 \mathbf{v}\in\mathbb{R}^3 v ∈ R 3 ,
d d r ∥ K β ( r , x ) v ∥ 2 = 2 v T ⋅ K β T ( r , x ) ⋅ D h β ( r , h β ( r , x ) ) ⋅ K β ( r , x ) ⋅ v ≤ 2 r κ + r 2 ⋅ ∥ K β ( r , x ) v ∥ 2 . \frac{d}{dr}\lVert K_{\beta}(r,\mathbf{x})\mathbf{v}\rVert^2
=2\mathbf{v}^{\mathsf{T}}\cdot K_{\beta}^{\mathsf{T}}(r,\mathbf{x})\cdot Dh_{\beta}(r,h_{\beta}(r,\mathbf{x}))\cdot K_{\beta}(r,\mathbf{x})\cdot\mathbf{v}
\leq\frac{2r}{\kappa+r^2}\cdot\lVert K_{\beta}(r,\mathbf{x})\mathbf{v}\rVert^2. d r d ∥ K β ( r , x ) v ∥ 2 = 2 v T ⋅ K β T ( r , x ) ⋅ D h β ( r , h β ( r , x )) ⋅ K β ( r , x ) ⋅ v ≤ κ + r 2 2 r ⋅ ∥ K β ( r , x ) v ∥ 2 .
Gronwall’s inequality yields
∥ K β ( r , x ) v ∥ 2 ≤ exp ( ∫ 0 r 2 s κ + s 2 d s ) ∥ v ∥ 2 = κ + r 2 κ ∥ v ∥ 2 . \lVert K_{\beta}(r,\mathbf{x})\mathbf{v}\rVert^2
\leq\exp\left(\int_0^r\frac{2s}{\kappa+s^2}\,ds\right)\lVert\mathbf{v}\rVert^2
=\frac{\kappa+r^2}{\kappa}\lVert\mathbf{v}\rVert^2. ∥ K β ( r , x ) v ∥ 2 ≤ exp ( ∫ 0 r κ + s 2 2 s d s ) ∥ v ∥ 2 = κ κ + r 2 ∥ v ∥ 2 .
Taking the supremum over unit vectors v \mathbf{v} v gives
∥ K β ( r , x ) ∥ ≤ κ + r 2 κ . \lVert K_{\beta}(r,\mathbf{x})\rVert\leq\sqrt{\frac{\kappa+r^2}{\kappa}}. ∥ K β ( r , x )∥ ≤ κ κ + r 2 .
Moreover, the sandwich bound above also implies, for r > 0 r>0 r > 0 ,
0 ⪯ 1 r I 3 − D h β ( r , h β ( r , x ) ) ⪯ Λ r ( Λ + r 2 ) I 3 . 0\preceq\frac{1}{r}I_3-Dh_{\beta}(r,h_{\beta}(r,\mathbf{x}))\preceq\frac{\Lambda}{r(\Lambda+r^2)}I_3. 0 ⪯ r 1 I 3 − D h β ( r , h β ( r , x )) ⪯ r ( Λ + r 2 ) Λ I 3 .
Thus
∥ D h β ( r , h β ( r , x ) ) − 1 r I 3 ∥ ≤ Λ r ( Λ + r 2 ) . \left\lVert Dh_{\beta}(r,h_{\beta}(r,\mathbf{x}))-\frac{1}{r}I_3\right\rVert\leq\frac{\Lambda}{r(\Lambda+r^2)}. D h β ( r , h β ( r , x )) − r 1 I 3 ≤ r ( Λ + r 2 ) Λ .
Differentiating the normalized matrix K β / r K_{\beta}/r K β / r therefore gives
∥ ∂ r { K β ( r , x ) r } ∥ = ∥ { D h β ( r , h β ( r , x ) ) − 1 r I 3 } ⋅ K β ( r , x ) r ∥ ≤ Λ r 2 ( Λ + r 2 ) κ + r 2 κ ≤ C r 3 , \left\lVert\partial_r\left\{\frac{K_{\beta}(r,\mathbf{x})}{r}\right\}\right\rVert
=\left\lVert\left\{Dh_{\beta}(r,h_{\beta}(r,\mathbf{x}))-\frac{1}{r}I_3\right\}\cdot\frac{K_{\beta}(r,\mathbf{x})}{r}\right\rVert
\leq\frac{\Lambda}{r^2(\Lambda+r^2)}\sqrt{\frac{\kappa+r^2}{\kappa}}\leq\frac{C}{r^3}, ∂ r { r K β ( r , x ) } = { D h β ( r , h β ( r , x )) − r 1 I 3 } ⋅ r K β ( r , x ) ≤ r 2 ( Λ + r 2 ) Λ κ κ + r 2 ≤ r 3 C ,
for all r ≥ 1 r\geq1 r ≥ 1 , where C κ , Λ < ∞ C_{\kappa,\Lambda}<\infty C κ , Λ < ∞ depends only on κ \kappa κ and Λ \Lambda Λ .
Thus, for S > R ≥ 1 S>R\geq1 S > R ≥ 1 ,
sup x ∈ R 3 ∥ K β ( S , x ) S − K β ( R , x ) R ∥ ≤ ∫ R S C κ , Λ r 3 d r ≤ C κ , Λ 2 R 2 . (74) \sup_{\mathbf{x}\in\mathbb{R}^3}\left\lVert\frac{K_{\beta}(S,\mathbf{x})}{S}-\frac{K_{\beta}(R,\mathbf{x})}{R}\right\rVert
\leq\int_R^S\frac{C_{\kappa,\Lambda}}{r^3}\,dr\leq\frac{C_{\kappa,\Lambda}}{2R^2}. \tag*{(74)} x ∈ R 3 sup S K β ( S , x ) − R K β ( R , x ) ≤ ∫ R S r 3 C κ , Λ d r ≤ 2 R 2 C κ , Λ . (74)
Hence K β ( r , ⋅ ) / r K_{\beta}(r,\cdot)/r K β ( r , ⋅ ) / r converges uniformly, and in particular locally uniformly. Let this limit be
L β ( x ) : = lim r → ∞ K β ( r , x ) r . L_{\beta}(\mathbf{x}):=\lim_{r\to\infty}\frac{K_{\beta}(r,\mathbf{x})}{r}. L β ( x ) := r → ∞ lim r K β ( r , x ) .
It suffices to show that L β ( x ) = β ⋅ D Z 1 β , W ( β x ) L_{\beta}(\mathbf{x})=\sqrt{\beta}\cdot DZ_1^{\beta,W}(\sqrt{\beta}\mathbf{x}) L β ( x ) = β ⋅ D Z 1 β , W ( β x ) . Consider a coordinate vector e j \mathbf{e}_j e j . For all r , t > 0 r,t>0 r , t > 0 , we have
h β ( r , x + t e j ) − h β ( r , x ) r = ∫ 0 t K β ( r , x + s e j ) r ⋅ e j d s . \frac{h_{\beta}(r,\mathbf{x}+t\mathbf{e}_j)-h_{\beta}(r,\mathbf{x})}{r}
=\int_0^t\frac{K_{\beta}(r,\mathbf{x}+s\mathbf{e}_j)}{r}\cdot\mathbf{e}_j\,ds. r h β ( r , x + t e j ) − h β ( r , x ) = ∫ 0 t r K β ( r , x + s e j ) ⋅ e j d s .
Let r → ∞ r\to\infty r → ∞ , using the local convergence in Step 2, we deduce that
Z 1 β , W ( β ( x + t e j ) ) − Z 1 β , W ( β x ) = ∫ 0 t L β ( x + s e j ) e j d s . Z_1^{\beta,W}\left(\sqrt{\beta}(\mathbf{x}+t\mathbf{e}_j)\right)-Z_1^{\beta,W}\left(\sqrt{\beta}\mathbf{x}\right)
=\int_0^tL_{\beta}(\mathbf{x}+s\mathbf{e}_j)\mathbf{e}_j\,ds. Z 1 β , W ( β ( x + t e j ) ) − Z 1 β , W ( β x ) = ∫ 0 t L β ( x + s e j ) e j d s .
Divide both sides by t t t and send t → 0 t \to0 t → 0 , we get
∂ x j { Z 1 β , W ( β x ) } = L β ( x ) ⋅ e j . \partial_{x_j}\left\{Z_1^{\beta,W}\left(\sqrt{\beta}x\right)\right\}=L_\beta(x)\cdot e_j. ∂ x j { Z 1 β , W ( β x ) } = L β ( x ) ⋅ e j .
Since j j j is an arbitrary coordinate, the conclusion follows. □ \square □
Define
y ( r , h ) : = arg min u ∈ R 3 { W ( u ) + 1 2 ∥ h − r u ∥ 2 } . (75) \boldsymbol{y}(r,\boldsymbol{h}):=\underset{\boldsymbol{u}\in\mathbb{R}^3}{\arg\min}\left\{W(\boldsymbol{u})+\frac{1}{2}\left\|\boldsymbol{h}-r\boldsymbol{u}\right\|^2\right\}. \tag*{(75)} y ( r , h ) := u ∈ R 3 arg min { W ( u ) + 2 1 ∥ h − r u ∥ 2 } . (75)
Equivalently, y ( r , h ) \boldsymbol{y}(r,\boldsymbol{h}) y ( r , h ) is the unique solution of
∇ W ( y ) + r 2 y − r h = 0. \nabla W(\boldsymbol{y})+r^2\boldsymbol{y}-r\boldsymbol{h}=0. ∇ W ( y ) + r 2 y − r h = 0.
We interpret the above as y ( r , h ) \boldsymbol{y}(r,\boldsymbol{h}) y ( r , h ) being a function of r r r and h \boldsymbol{h} h : such a function is well-defined because due to strong convexity of W W W , there exists a unique y \boldsymbol{y} y satisfying the above equality for fixed r , h r,\boldsymbol{h} r , h . Lemma G.4 yields that, as a function of ( r , h ) (r,\boldsymbol{h}) ( r , h ) , y ( r , h ) \boldsymbol{y}(r,\boldsymbol{h}) y ( r , h ) is of class C 1 C^1 C 1 .
Proposition G.2. Under the same assumptions as in Proposition G.1, it holds that
(i) There exists a unique map h ∈ C 1 ( [ 0 , ∞ ) × R 3 ; R 3 ) \boldsymbol{h}\in C^1([0,\infty)\times\mathbb{R}^3;\mathbb{R}^3) h ∈ C 1 ([ 0 , ∞ ) × R 3 ; R 3 ) satisfying
∂ ∂ r h ( r , x ) = y ( r , h ( r , x ) ) , h ( 0 , x ) = x . (76) \frac{\partial}{\partial r}\boldsymbol{h}(r,\boldsymbol{x})=\boldsymbol{y}(r,\boldsymbol{h}(r,\boldsymbol{x})),\qquad\boldsymbol{h}(0,\boldsymbol{x})=\boldsymbol{x}. \tag*{(76)} ∂ r ∂ h ( r , x ) = y ( r , h ( r , x )) , h ( 0 , x ) = x . (76) Moreover, h β → h \boldsymbol{h}_\beta\to\boldsymbol{h} h β → h in C l o c 1 ( [ 0 , ∞ ) × R 3 ; R 3 ) C^1_{\mathrm{loc}}([0,\infty)\times\mathbb{R}^3;\mathbb{R}^3) C loc 1 ([ 0 , ∞ ) × R 3 ; R 3 ) .
(ii) The limits
P ( x ) : = lim r → ∞ h ( r , x ) r , D P ( x ) = lim r → ∞ D x h ( r , x ) r \mathcal{P}(\boldsymbol{x}):=\lim_{r\to\infty}\frac{\boldsymbol{h}(r,\boldsymbol{x})}{r},\qquad D\mathcal{P}(\boldsymbol{x})=\lim_{r\to\infty}\frac{D_{\boldsymbol{x}}\boldsymbol{h}(r,\boldsymbol{x})}{r} P ( x ) := r → ∞ lim r h ( r , x ) , D P ( x ) = r → ∞ lim r D x h ( r , x ) exist and the convergences above hold locally uniformly in x \boldsymbol{x} x . Moreover, we have P ∈ C 1 ( R 3 ; R 3 ) \mathcal{P}\in C^1(\mathbb{R}^3;\mathbb{R}^3) P ∈ C 1 ( R 3 ; R 3 ) .
(iii) If P β ( x ) : = Z 1 β , W ( β x ) \mathcal{P}_\beta(\boldsymbol{x}):=Z_1^{\beta,W}(\sqrt{\beta}\boldsymbol{x}) P β ( x ) := Z 1 β , W ( β x ) , then
P β ⟶ P in C l o c 1 ( R 3 ; R 3 ) . \mathcal{P}_\beta\longrightarrow\mathcal{P}\qquad\text{in }C^1_{\mathrm{loc}}(\mathbb{R}^3;\mathbb{R}^3). P β ⟶ P in C loc 1 ( R 3 ; R 3 ) .
To be precise, Proposition G.2(i) claims that for every R < ∞ R<\infty R < ∞ and every compact K ⊂ R 3 K\subset\mathbb{R}^3 K ⊂ R 3 ,
sup 0 ≤ r ≤ R x ∈ K { ∥ h β ( r , x ) − h ( r , x ) ∥ + ∥ ∂ r h β ( r , x ) − ∂ r h ( r , x ) ∥ + ∥ D x h β ( r , x ) − D x h ( r , x ) ∥ } → 0. \sup_{\substack{0\le r\le R\\ \boldsymbol{x}\in K}}\left\{\left\|\boldsymbol{h}_\beta(r,\boldsymbol{x})-\boldsymbol{h}(r,\boldsymbol{x})\right\|+\left\|\partial_r\boldsymbol{h}_\beta(r,\boldsymbol{x})-\partial_r\boldsymbol{h}(r,\boldsymbol{x})\right\|+\left\|D_{\boldsymbol{x}}\boldsymbol{h}_\beta(r,\boldsymbol{x})-D_{\boldsymbol{x}}\boldsymbol{h}(r,\boldsymbol{x})\right\|\right\}\to0. 0 ≤ r ≤ R x ∈ K sup { ∥ h β ( r , x ) − h ( r , x ) ∥ + ∥ ∂ r h β ( r , x ) − ∂ r h ( r , x ) ∥ + ∥ D x h β ( r , x ) − D x h ( r , x ) ∥ } → 0.
Similarly, Proposition G.2(iii) yields that D P β → D P D\mathcal{P}_\beta\to D\mathcal{P} D P β → D P uniformly over any compact set K ⊂ R 3 K\subset\mathbb{R}^3 K ⊂ R 3
sup x ∈ K ∥ D P β ( x ) − D P ( x ) ∥ o p → 0. \sup_{\boldsymbol{x}\in K}\left\|D\mathcal{P}_\beta(\boldsymbol{x})-D\mathcal{P}(\boldsymbol{x})\right\|_{\mathrm{op}}\to0. x ∈ K sup ∥ D P β ( x ) − D P ( x ) ∥ op → 0.
Let us explain briefly why these results should be expected. The ODE in Proposition G.2(i) has a C 1 C^1 C 1 solution h \boldsymbol{h} h due to Lemma G.3 and the classical existence results for ODEs. The convergence h β → h \boldsymbol{h}_\beta\to\boldsymbol{h} h β → h is suggested by taking β → ∞ \beta\to\infty β → ∞ in (G.1) and observing that the posterior mean under π β , r , h \pi_{\beta,r,\boldsymbol{h}} π β , r , h converges to the minimizer:
y β ( r , h ) = E Y ∼ π β , r , h [ Y ] → β → ∞ y ( r , h ) = arg min u ∈ R 3 { W ( u ) + 1 2 ∥ h − r u ∥ 2 } . \boldsymbol{y}_\beta(r,\boldsymbol{h})=\mathbb{E}_{Y\sim\pi_{\beta,r,\boldsymbol{h}}}[Y]\xrightarrow{\beta\to\infty}\boldsymbol{y}(r,\boldsymbol{h})=\underset{\boldsymbol{u}\in\mathbb{R}^3}{\arg\min}\left\{W(\boldsymbol{u})+\frac{1}{2}\left\|\boldsymbol{h}-r\boldsymbol{u}\right\|^2\right\}. y β ( r , h ) = E Y ∼ π β , r , h [ Y ] β → ∞ y ( r , h ) = u ∈ R 3 arg min { W ( u ) + 2 1 ∥ h − r u ∥ 2 } .
uniformly over compact subsets. Propositions G.2(ii) and (iii) also follow from taking the derivatives and sending β → ∞ \beta\to\infty β → ∞ , which can be justified rigorously by bounding the derivatives of y \mathbf{y} y . We note that the pair ( y , h ) (\mathbf{y},h) ( y , h ) is best understood in the ODE sense. If one views y , h \mathbf{y},h y , h as functions of r r r and considers a flow h = h ( r ) h=h(r) h = h ( r ) from the ODE (G.7), then the joint trajectory ( y ( r ) , h ( r ) ) (\mathbf{y}(r),h(r)) ( y ( r ) , h ( r )) satisfies
∇ W ( y ( r ) ) + r 2 ⋅ y ( r ) − r ⋅ h ( r , x ) = 0 \nabla W(\mathbf{y}(r)) + r^{2}\cdot\mathbf{y}(r) - r\cdot h(r,\mathbf{x}) = 0 ∇ W ( y ( r )) + r 2 ⋅ y ( r ) − r ⋅ h ( r , x ) = 0
with y ( r ) = y ( r , h ( r , x ) ) \mathbf{y}(r)=\mathbf{y}(r,h(r,\mathbf{x})) y ( r ) = y ( r , h ( r , x )) .
Proof of Proposition G.2. Put
Φ ( u ) : = W ( u ) + 1 2 ∥ h − r u ∥ 2 , A ( r , h ) : = ∇ 2 Φ y ( y ( r , h ) ) = ∇ 2 W ( y ( r , h ) ) + r 2 I 3 . \Phi(u) := W(u) + \frac{1}{2}\lVert h-ru\rVert^{2}, \qquad A(r,h) := \nabla^{2}\Phi_{\mathbf{y}}(\mathbf{y}(r,h)) = \nabla^{2}W(\mathbf{y}(r,h)) + r^{2}I_{3}. Φ ( u ) := W ( u ) + 2 1 ∥ h − r u ∥ 2 , A ( r , h ) := ∇ 2 Φ y ( y ( r , h )) = ∇ 2 W ( y ( r , h )) + r 2 I 3 .
All expectations below are taken with respect to π β , r , h \pi_{\beta,r,h} π β , r , h .
Proof of (i). The existence and uniqueness of h ∈ C 1 ( [ 0 , ∞ ) × R 3 ; R 3 ) h\in C^{1}([0,\infty)\times\mathbb{R}^{3};\mathbb{R}^{3}) h ∈ C 1 ([ 0 , ∞ ) × R 3 ; R 3 ) follow from Lemma G.5, which yields the Lipschitz property of y \mathbf{y} y , and Lemma G.4 , which yields the differentiability of y \mathbf{y} y . Put
K β ( r , x ) : = D x h β ( r , x ) , K ( r , x ) : = D x h ( r , x ) . K_{\beta}(r,\mathbf{x}) := D_{\mathbf{x}}h_{\beta}(r,\mathbf{x}), \qquad K(r,\mathbf{x}) := D_{\mathbf{x}}h(r,\mathbf{x}). K β ( r , x ) := D x h β ( r , x ) , K ( r , x ) := D x h ( r , x ) .
The pair ( h β , K β ) (h_{\beta},K_{\beta}) ( h β , K β ) solves the coupled ODE
d d r ( h β ( r , x ) K β ( r , x ) ) = ( y β ( r , h β ) D y β ( r , h β ) ⋅ K β ) , ( h β K β ) ( 0 , x ) = ( x I 3 ) T . \frac{d}{dr}\begin{pmatrix} h_{\beta}(r,\mathbf{x}) \\ K_{\beta}(r,\mathbf{x}) \end{pmatrix}
=
\begin{pmatrix} \mathbf{y}_{\beta}(r,h_{\beta}) \\ D\mathbf{y}_{\beta}(r,h_{\beta})\cdot K_{\beta} \end{pmatrix},
\qquad
\begin{pmatrix} h_{\beta} \\ K_{\beta} \end{pmatrix}(0,\mathbf{x})
=
\begin{pmatrix} \mathbf{x} \\ I_{3} \end{pmatrix}^{\mathsf{T}}. d r d ( h β ( r , x ) K β ( r , x ) ) = ( y β ( r , h β ) D y β ( r , h β ) ⋅ K β ) , ( h β K β ) ( 0 , x ) = ( x I 3 ) T .
On the other hand, ( h , K ) (h,K) ( h , K ) solves the coupled ODE
d d r ( h ( r , x ) K ( r , x ) ) = ( y ( r , h ) D y ( r , h ) ⋅ K ) , ( h K ) ( 0 , x ) = ( x I 3 ) T . \frac{d}{dr}\begin{pmatrix} h(r,\mathbf{x}) \\ K(r,\mathbf{x}) \end{pmatrix}
=
\begin{pmatrix} \mathbf{y}(r,h) \\ D\mathbf{y}(r,h)\cdot K \end{pmatrix},
\qquad
\begin{pmatrix} h \\ K \end{pmatrix}(0,\mathbf{x})
=
\begin{pmatrix} \mathbf{x} \\ I_{3} \end{pmatrix}^{\mathsf{T}}. d r d ( h ( r , x ) K ( r , x ) ) = ( y ( r , h ) D y ( r , h ) ⋅ K ) , ( h K ) ( 0 , x ) = ( x I 3 ) T .
By perturbation results for ODEs (see, for example, Theorem 2.8 in Teschl [ [55 ] ), we only need to show that uniformly over (space-time) compact sets,
y β ( r , h ) → β → ∞ y ( r , h ) and D y β ( r , h ) → β → ∞ D y ( r , h ) . \mathbf{y}_{\beta}(r,h) \xrightarrow[\beta\to\infty]{} \mathbf{y}(r,h)
\qquad\text{and}\qquad
D\mathbf{y}_{\beta}(r,h) \xrightarrow[\beta\to\infty]{} D\mathbf{y}(r,h). y β ( r , h ) β → ∞ y ( r , h ) and D y β ( r , h ) β → ∞ D y ( r , h ) .
The first convergence is immediate from Lemma G.5, because
∥ y β ( r , h ) − y ( r , h ) ∥ ≤ 3 β ( κ + r 2 ) . \lVert\mathbf{y}_{\beta}(r,h)-\mathbf{y}(r,h)\rVert\le\sqrt{\frac{3}{\beta(\kappa+r^{2})}}. ∥ y β ( r , h ) − y ( r , h )∥ ≤ β ( κ + r 2 ) 3 .
To prove the convergence of the Jacobian, note that
D h y β ( r , h ) = β r ⋅ Cov ( Y ) , D h y ( r , h ) = r ⋅ [ ∇ 2 W ( y ( r , h ) ) + r 2 I 3 ] − 1 = r ⋅ A ( r , h ) − 1 . D_{h}\mathbf{y}_{\beta}(r,h)=\beta r\cdot\operatorname{Cov}(Y), \qquad D_{h}\mathbf{y}(r,h)=r\cdot\left[\nabla^{2}W(\mathbf{y}(r,h))+r^{2}I_{3}\right]^{-1}=r\cdot A(r,h)^{-1}. D h y β ( r , h ) = β r ⋅ Cov ( Y ) , D h y ( r , h ) = r ⋅ [ ∇ 2 W ( y ( r , h )) + r 2 I 3 ] − 1 = r ⋅ A ( r , h ) − 1 .
It suffices to show that
β ⋅ Cov ( Y ) → β → ∞ A ( r , h ) − 1 . \beta\cdot\operatorname{Cov}(Y)\xrightarrow[\beta\to\infty]{}A(r,h)^{-1}. β ⋅ Cov ( Y ) β → ∞ A ( r , h ) − 1 .
The Brascamp–Lieb inequality and the Cramér–Rao lower bound yield
{ E Y ∼ π r , β , h [ ∇ 2 W ( Y ) + r 2 I 3 ] } − 1 ⪯ β ⋅ Cov ( Y ) ⪯ E Y ∼ π r , β , h { [ ∇ 2 W ( Y ) + r 2 I 3 ] − 1 } . \left\{\mathbb{E}_{Y\sim\pi_{r,\beta,h}}\left[\nabla^{2}W(Y)+r^{2}I_{3}\right]\right\}^{-1}
\preceq\beta\cdot\operatorname{Cov}(Y)
\preceq
\mathbb{E}_{Y\sim\pi_{r,\beta,h}}\left\{\left[\nabla^{2}W(Y)+r^{2}I_{3}\right]^{-1}\right\}. { E Y ∼ π r , β , h [ ∇ 2 W ( Y ) + r 2 I 3 ] } − 1 ⪯ β ⋅ Cov ( Y ) ⪯ E Y ∼ π r , β , h { [ ∇ 2 W ( Y ) + r 2 I 3 ] − 1 } .
Put H = ∇ 2 W ( Y ) + r 2 I 3 \boldsymbol{H}=\nabla^{2}W(\boldsymbol{Y})+r^{2}I_{3} H = ∇ 2 W ( Y ) + r 2 I 3 . By using Lemma G.8, we obtain
∥ ( E Y ∼ π r , β , h H ) − 1 − A ( r , h ) − 1 ∥ ≤ E Y ∼ π r , β , h ∥ ∇ 2 W ( Y ) − ∇ 2 W ( y ( r , h ) ) ∥ κ 2 \left\|\left(\mathbb{E}_{Y\sim\pi_{r,\beta,h}}\boldsymbol{H}\right)^{-1}-A(r,h)^{-1}\right\|
\leq\frac{\mathbb{E}_{Y\sim\pi_{r,\beta,h}}\left\|\nabla^{2}W(\boldsymbol{Y})-\nabla^{2}W(y(r,h))\right\|}{\kappa^{2}} ( E Y ∼ π r , β , h H ) − 1 − A ( r , h ) − 1 ≤ κ 2 E Y ∼ π r , β , h ∇ 2 W ( Y ) − ∇ 2 W ( y ( r , h ))
∥ E Y ∼ π r , β , h [ H − 1 ] − A ( r , h ) − 1 ∥ ≤ E Y ∼ π r , β , h ∥ ∇ 2 W ( Y ) − ∇ 2 W ( y ( r , h ) ) ∥ κ 2 \left\|\mathbb{E}_{Y\sim\pi_{r,\beta,h}}\left[\boldsymbol{H}^{-1}\right]-A(r,h)^{-1}\right\|
\leq\frac{\mathbb{E}_{Y\sim\pi_{r,\beta,h}}\left\|\nabla^{2}W(\boldsymbol{Y})-\nabla^{2}W(y(r,h))\right\|}{\kappa^{2}} E Y ∼ π r , β , h [ H − 1 ] − A ( r , h ) − 1 ≤ κ 2 E Y ∼ π r , β , h ∇ 2 W ( Y ) − ∇ 2 W ( y ( r , h ))
Consequently,
− E Y ∼ π r , β , h ∥ ∇ 2 W ( Y ) − ∇ 2 W ( y ( r , h ) ) ∥ κ 2 ⋅ I 3 ⪯ β ⋅ Cov ( Y ) − A ( r , h ) − 1 -\frac{\mathbb{E}_{Y\sim\pi_{r,\beta,h}}\left\|\nabla^{2}W(\boldsymbol{Y})-\nabla^{2}W(y(r,h))\right\|}{\kappa^{2}}\cdot I_{3}\preceq\beta\cdot\operatorname{Cov}(\boldsymbol{Y})-A(r,h)^{-1} − κ 2 E Y ∼ π r , β , h ∇ 2 W ( Y ) − ∇ 2 W ( y ( r , h )) ⋅ I 3 ⪯ β ⋅ Cov ( Y ) − A ( r , h ) − 1
⪯ E Y ∼ π r , β , h ∥ ∇ 2 W ( Y ) − ∇ 2 W ( y ( r , h ) ) ∥ κ 2 ⋅ I 3 \preceq\frac{\mathbb{E}_{Y\sim\pi_{r,\beta,h}}\left\|\nabla^{2}W(\boldsymbol{Y})-\nabla^{2}W(y(r,h))\right\|}{\kappa^{2}}\cdot I_{3} ⪯ κ 2 E Y ∼ π r , β , h ∇ 2 W ( Y ) − ∇ 2 W ( y ( r , h )) ⋅ I 3
It remains to bound the expectation term in the last display. Fix R < ∞ R<\infty R < ∞ and a compact set B ⊂ R 3 B\subset\mathbb{R}^{3} B ⊂ R 3 . Consider the compact set { y ( r , h ) : r ∈ [ 0 , R ] , h ∈ B } \{y(r,h):r\in[0,R],\,h\in B\} { y ( r , h ) : r ∈ [ 0 , R ] , h ∈ B } . By uniform continuity of ∇ 2 W \nabla^{2}W ∇ 2 W on a compact neighborhood of this set, for every ε > 0 \varepsilon>0 ε > 0 there exists a common δ = δ ( ε , R , B ) > 0 \delta=\delta(\varepsilon,R,B)>0 δ = δ ( ε , R , B ) > 0 such that
∥ u − y ( r , h ) ∥ ≤ δ ⟹ ∥ ∇ 2 W ( u ) − ∇ 2 W ( y ( r , h ) ) ∥ ≤ ε \|\boldsymbol{u}-y(r,h)\|\leq\delta\quad\Longrightarrow\quad\|\nabla^{2}W(\boldsymbol{u})-\nabla^{2}W(y(r,h))\|\leq\varepsilon ∥ u − y ( r , h ) ∥ ≤ δ ⟹ ∥ ∇ 2 W ( u ) − ∇ 2 W ( y ( r , h )) ∥ ≤ ε
for all r ∈ [ 0 , R ] r\in[0,R] r ∈ [ 0 , R ] and h ∈ B h\in B h ∈ B . Consequently,
sup r ∈ [ 0 , R ] h ∈ B E ∥ ∇ 2 W ( Y ) − ∇ 2 W ( y ( r , h ) ) ∥ ≤ ε + 2 Λ sup r ∈ [ 0 , R ] h ∈ B P ( ∥ Y − y ( r , h ) ∥ ≥ δ ) ≤ ε + 6 Λ β κ δ 2 . \begin{aligned}
\sup_{\substack{r\in[0,R]\\h\in B}}\mathbb{E}\left\|\nabla^{2}W(\boldsymbol{Y})-\nabla^{2}W(y(r,h))\right\|
\leq\varepsilon+2\Lambda\sup_{\substack{r\in[0,R]\\h\in B}}\mathbb{P}\left(\|\boldsymbol{Y}-y(r,h)\|\geq\delta\right) \\
&\leq\varepsilon+\frac{6\Lambda}{\beta\kappa\delta^{2}}.
\end{aligned} r ∈ [ 0 , R ] h ∈ B sup E ∇ 2 W ( Y ) − ∇ 2 W ( y ( r , h )) ≤ ε + 2Λ r ∈ [ 0 , R ] h ∈ B sup P ( ∥ Y − y ( r , h ) ∥ ≥ δ ) ≤ ε + β κ δ 2 6Λ .
where the second inequality follows from Markov’s inequality and the second claim in Lemma G.5. Using the preceding covariance bounds, we obtain
sup r ∈ [ 0 , R ] h ∈ B ∥ D h y β ( r , h ) − D h y ( r , h ) ∥ ≤ R κ 2 ( ε + 6 Λ β κ δ 2 ) . \sup_{\substack{r\in[0,R]\\h\in B}}\|D_{h}y_{\beta}(r,h)-D_{h}y(r,h)\|
\leq\frac{R}{\kappa^{2}}\left(\varepsilon+\frac{6\Lambda}{\beta\kappa\delta^{2}}\right). r ∈ [ 0 , R ] h ∈ B sup ∥ D h y β ( r , h ) − D h y ( r , h ) ∥ ≤ κ 2 R ( ε + β κ δ 2 6Λ ) .
Taking lim sup β → ∞ \limsup_{\beta\to\infty} lim sup β → ∞ and then letting ε ↓ 0 \varepsilon\downarrow0 ε ↓ 0 proves that D h y β → D h y D_{h}y_{\beta}\to D_{h}y D h y β → D h y locally uniformly in ( r , h ) (r,h) ( r , h ) .
Proof of (ii) and (iii). For a compact set K ⊂ R 3 K\subset\mathbb{R}^{3} K ⊂ R 3 , write
∥ F ∥ C 1 ( K ) : = sup x ∈ K ∥ F ( x ) ∥ + sup x ∈ K ∥ D F ( x ) ∥ . \|F\|_{C^{1}(K)}:=\sup_{x\in K}\|F(x)\|+\sup_{x\in K}\|DF(x)\|. ∥ F ∥ C 1 ( K ) := x ∈ K sup ∥ F ( x ) ∥ + x ∈ K sup ∥ D F ( x ) ∥.
The estimates in Steps 1–3 of the proof of Proposition G.1 give
∥ h β ( S , ⋅ ) S − h β ( R , ⋅ ) R ∥ C 1 ( K ) ≤ C K R 2 , S ≥ R ≥ 1 , \left\|\frac{h_{\beta}(S,\cdot)}{S}-\frac{h_{\beta}(R,\cdot)}{R}\right\|_{C^{1}(K)}
\leq\frac{C_{K}}{R^{2}},\qquad S\geq R\geq1, S h β ( S , ⋅ ) − R h β ( R , ⋅ ) C 1 ( K ) ≤ R 2 C K , S ≥ R ≥ 1 ,
with C K C_{K} C K independent of β ≥ 1 \beta\geq1 β ≥ 1 . Indeed, (G.3) and (G.5) are uniform over β ≥ 1 \beta\geq1 β ≥ 1 . For fixed S S S and R R R , letting β → ∞ \beta\to\infty β → ∞ and using (i), we obtain
∥ h ( S , ⋅ ) S − h ( R , ⋅ ) R ∥ C 1 ( K ) ≤ C K R 2 . \left\|\frac{h(S,\cdot)}{S}-\frac{h(R,\cdot)}{R}\right\|_{C^{1}(K)}
\leq\frac{C_{K}}{R^{2}}. S h ( S , ⋅ ) − R h ( R , ⋅ ) C 1 ( K ) ≤ R 2 C K .
Hence the normalized flow and its Jacobians are locally uniformly Cauchy. The same fundamental theorem of calculus argument as in Proposition G.1 identifies the derivative limit with the Jacobian of the function limit. Thus P ∈ C 1 \mathcal{P} \in C^1 P ∈ C 1 and
h ( R , ⋅ ) R ⟶ P in C l o c 1 , \frac{h(R,\cdot)}{R} \longrightarrow\mathcal{P} \quad\text{in } C^1_{\mathrm{loc}}, R h ( R , ⋅ ) ⟶ P in C loc 1 ,
proving (ii).
Letting S → ∞ S \to\infty S → ∞ in the two estimates above gives
∥ P β − h β ( R , ⋅ ) R ∥ C 1 ( K ) ≤ C K R 2 , ∥ P − h ( R , ⋅ ) R ∥ C 1 ( K ) ≤ C K R 2 . \left\|\mathcal{P}_{\beta}-\frac{h_{\beta}(R,\cdot)}{R}\right\|_{C^1(K)} \leq\frac{C_K}{R^2}, \qquad\left\|\mathcal{P}-\frac{h(R,\cdot)}{R}\right\|_{C^1(K)} \leq\frac{C_K}{R^2}. P β − R h β ( R , ⋅ ) C 1 ( K ) ≤ R 2 C K , P − R h ( R , ⋅ ) C 1 ( K ) ≤ R 2 C K .
Therefore,
∥ P β − P ∥ C 1 ( K ) ≤ 2 C K R 2 + 1 R ∥ h β ( R , ⋅ ) − h ( R , ⋅ ) ∥ C 1 ( K ) . \|\mathcal{P}_{\beta}-\mathcal{P}\|_{C^1(K)} \leq\frac{2C_K}{R^2}+\frac{1}{R}\|h_{\beta}(R,\cdot)-h(R,\cdot)\|_{C^1(K)}. ∥ P β − P ∥ C 1 ( K ) ≤ R 2 2 C K + R 1 ∥ h β ( R , ⋅ ) − h ( R , ⋅ ) ∥ C 1 ( K ) .
For fixed R R R , the last term tends to zero by (i). Taking lim sup β → ∞ \limsup_{\beta\to\infty} lim sup β → ∞ and then letting R → ∞ R \to\infty R → ∞ proves (iii). □ \square □
ODE results
Lemma G.1 (Gronwall-type inequality). Let T > 0 T>0 T > 0 , let a , b ∈ L 1 ( [ 0 , T ] ; R ) a,b\in L^1([0,T];\mathbb{R}) a , b ∈ L 1 ([ 0 , T ] ; R ) , and suppose f : [ 0 , T ] → R f:[0,T]\to\mathbb{R} f : [ 0 , T ] → R is absolutely continuous. Assume that a ( r ) ≥ 0 a(r)\geq0 a ( r ) ≥ 0 for almost every r ∈ [ 0 , T ] r\in[0,T] r ∈ [ 0 , T ] and
f ′ ( r ) ≤ a ( r ) f ( r ) + b ( r ) for almost every r ∈ [ 0 , T ] . f'(r)\leq a(r)f(r)+b(r) \qquad\text{for almost every }r\in[0,T]. f ′ ( r ) ≤ a ( r ) f ( r ) + b ( r ) for almost every r ∈ [ 0 , T ] . Then, for every r ∈ [ 0 , T ] r\in[0,T] r ∈ [ 0 , T ] ,
f ( r ) ≤ f ( 0 ) ⋅ exp { ∫ 0 r a ( s ) d s } + ∫ 0 r b ( s ) ⋅ exp { ∫ s r a ( u ) d u } d s . f(r)\leq f(0)\cdot\exp\left\{\int_0^r a(s)\,ds\right\}+\int_0^r b(s)\cdot\exp\left\{\int_s^r a(u)\,du\right\}\,ds. f ( r ) ≤ f ( 0 ) ⋅ exp { ∫ 0 r a ( s ) d s } + ∫ 0 r b ( s ) ⋅ exp { ∫ s r a ( u ) d u } d s .
Proof of Lemma G.1. Set α ( r ) : = f ( 0 ) + ∫ 0 r b ( s ) d s \alpha(r):=f(0)+\int_0^r b(s)\,ds α ( r ) := f ( 0 ) + ∫ 0 r b ( s ) d s . Integrating the differential inequality gives
f ( r ) ≤ α ( r ) + ∫ 0 r a ( s ) f ( s ) d s . f(r)\leq\alpha(r)+\int_0^r a(s)f(s)\,ds. f ( r ) ≤ α ( r ) + ∫ 0 r a ( s ) f ( s ) d s .
By Lemma 2.7 in Teschl [ [55 ] ,
f ( r ) ≤ α ( r ) + ∫ 0 r α ( s ) ⋅ a ( s ) ⋅ exp { ∫ s r a ( u ) d u } d s = f ( 0 ) ⋅ exp { ∫ 0 r a ( u ) d u } + ∫ 0 r b ( s ) ⋅ exp { ∫ s r a ( u ) d u } d s . \begin{aligned}
f(r)\leq\alpha(r)+\int_0^r \alpha(s)\cdot a(s)\cdot\exp\left\{\int_s^r a(u)\,du\right\}\,ds \\
&= f(0)\cdot\exp\left\{\int_0^r a(u)\,du\right\}+\int_0^r b(s)\cdot\exp\left\{\int_s^r a(u)\,du\right\}\,ds.
\end{aligned} f ( r ) ≤ α ( r ) + ∫ 0 r α ( s ) ⋅ a ( s ) ⋅ exp { ∫ s r a ( u ) d u } d s = f ( 0 ) ⋅ exp { ∫ 0 r a ( u ) d u } + ∫ 0 r b ( s ) ⋅ exp { ∫ s r a ( u ) d u } d s .
To justify the last equality, fix r r r and observe that
d d s exp { ∫ s r a ( u ) d u } = − a ( s ) exp { ∫ s r a ( u ) d u } \frac{d}{ds}\exp\left\{\int_s^r a(u)\,du\right\}=-a(s)\exp\left\{\int_s^r a(u)\,du\right\} d s d exp { ∫ s r a ( u ) d u } = − a ( s ) exp { ∫ s r a ( u ) d u }
for almost every s ∈ [ 0 , r ] s\in[0,r] s ∈ [ 0 , r ] . Hence integration by parts gives
∫ 0 r α ( s ) a ( s ) exp { ∫ s r a ( u ) d u } d s = − [ α ( s ) exp { ∫ s r a ( u ) d u } ] s = 0 s = r + ∫ 0 r α ′ ( s ) exp { ∫ s r a ( u ) d u } d s = − α ( r ) + f ( 0 ) exp { ∫ 0 r a ( u ) d u } + ∫ 0 r b ( s ) exp { ∫ s r a ( u ) d u } d s . \begin{aligned}
\int_0^r \alpha(s)a(s)\exp\left\{\int_s^r a(u)\,du\right\}\,ds
= -\left[\alpha(s)\exp\left\{\int_s^r a(u)\,du\right\}\right]_{s=0}^{s=r}
+\int_0^r \alpha'(s)\exp\left\{\int_s^r a(u)\,du\right\}\,ds \\
&= -\alpha(r)+f(0)\exp\left\{\int_0^r a(u)\,du\right\}
+\int_0^r b(s)\exp\left\{\int_s^r a(u)\,du\right\}\,ds.
\end{aligned} ∫ 0 r α ( s ) a ( s ) exp { ∫ s r a ( u ) d u } d s = − [ α ( s ) exp { ∫ s r a ( u ) d u } ] s = 0 s = r + ∫ 0 r α ′ ( s ) exp { ∫ s r a ( u ) d u } d s = − α ( r ) + f ( 0 ) exp { ∫ 0 r a ( u ) d u } + ∫ 0 r b ( s ) exp { ∫ s r a ( u ) d u } d s .
where we used α ( 0 ) = f ( 0 ) \alpha(0)=f(0) α ( 0 ) = f ( 0 ) and α ′ = b \alpha'=b α ′ = b almost everywhere. Adding α ( r ) \alpha(r) α ( r ) to both sides gives the asserted equality.
Lemma G.2. Let H : [ 0 , ∞ ) → S + + n H:[0,\infty)\to\mathbb{S}_{++}^{n} H : [ 0 , ∞ ) → S ++ n , n ∈ { 2 , 3 } n\in\{2,3\} n ∈ { 2 , 3 } , be continuous with κ I n ⪯ H ⪯ Λ I n \kappa I_n\preceq H\preceq\Lambda I_n κ I n ⪯ H ⪯ Λ I n . The following statements hold
M ′ ( r ) = r ( H + r 2 I n ) − 1 ⋅ M ( r ) , M ( 0 ) = I n . M'(r)=r\left(\mathbf{H}+r^2I_n\right)^{-1}\cdot M(r),\qquad M(0)=I_n. M ′ ( r ) = r ( H + r 2 I n ) − 1 ⋅ M ( r ) , M ( 0 ) = I n . has a unique solution M ∈ C 1 ( [ 0 , ∞ ) , R n × n ) M\in C^1([0,\infty),\mathbb{R}^{n\times n}) M ∈ C 1 ([ 0 , ∞ ) , R n × n ) .
∥ M ( r ) ∥ ≤ κ + r 2 κ . \|M(r)\|\le\sqrt{\frac{\kappa+r^2}{\kappa}}. ∥ M ( r ) ∥ ≤ κ κ + r 2 . ∥ Π − M ( R ) R ∥ ≤ Λ 2 R 2 ⋅ ∥ M ( R ) R ∥ . (77) \left\|\boldsymbol{\Pi}-\frac{M(R)}{R}\right\|\le\frac{\Lambda}{2R^2}\cdot\left\|\frac{M(R)}{R}\right\|. \tag*{(77)} Π − R M ( R ) ≤ 2 R 2 Λ ⋅ R M ( R ) . (77)
Proof of Lemma G.2. It is easy to see that A ( r ) : = r ( H ( r ) + r 2 I n ) − 1 A(r):=r\left(H(r)+r^2I_n\right)^{-1} A ( r ) := r ( H ( r ) + r 2 I n ) − 1 is continuous and symmetric with
r Λ + r 2 I n ⪯ A ( r ) ⪯ r κ + r 2 I n . (78) \frac{r}{\Lambda+r^2}I_n\preceq A(r)\preceq\frac{r}{\kappa+r^2}I_n. \tag*{(78)} Λ + r 2 r I n ⪯ A ( r ) ⪯ κ + r 2 r I n . (78)
A linear system with continuous coefficients has a unique global solution (Theorem 5.1, chapter I of Coddington and Levinson [ [13 ] ). Thus the first claim follows.
We now prove the second claim. Fix v ∈ R n v\in\mathbb{R}^n v ∈ R n and put
f v ( r ) : = ∥ M ( r ) v ∥ 2 . f_v(r):=\|M(r)v\|^2. f v ( r ) := ∥ M ( r ) v ∥ 2 .
Taking derivative in r r r , we obtain
f v ′ ( r ) = 2 ⟨ M ( r ) v , A ( r ) M ( r ) v ⟩ . f_v'(r)=2\langle M(r)v,A(r)M(r)v\rangle. f v ′ ( r ) = 2 ⟨ M ( r ) v , A ( r ) M ( r ) v ⟩ .
Consequently, (G.9) gives
2 r Λ + r 2 f v ( r ) ≤ f v ′ ( r ) ≤ 2 r κ + r 2 f v ( r ) . \frac{2r}{\Lambda+r^2}f_v(r)\le f_v'(r)\le\frac{2r}{\kappa+r^2}f_v(r). Λ + r 2 2 r f v ( r ) ≤ f v ′ ( r ) ≤ κ + r 2 2 r f v ( r ) .
Equivalently,
d d r [ f v ( r ) Λ + r 2 ] ≥ 0 , d d r [ f v ( r ) κ + r 2 ] ≤ 0. \frac{\mathrm{d}}{\mathrm{d}r}\left[\frac{f_v(r)}{\Lambda+r^2}\right]\ge0,\qquad
\frac{\mathrm{d}}{\mathrm{d}r}\left[\frac{f_v(r)}{\kappa+r^2}\right]\le0. d r d [ Λ + r 2 f v ( r ) ] ≥ 0 , d r d [ κ + r 2 f v ( r ) ] ≤ 0.
Integrating from 0 0 0 to r r r , and using M ( 0 ) = I n M(0)=I_n M ( 0 ) = I n , yields
Λ + r 2 Λ ∥ v ∥ 2 ≤ ∥ M ( r ) v ∥ 2 ≤ κ + r 2 κ ∥ v ∥ 2 . \frac{\Lambda+r^2}{\Lambda}\|v\|^2\le\|M(r)v\|^2\le\frac{\kappa+r^2}{\kappa}\|v\|^2. Λ Λ + r 2 ∥ v ∥ 2 ≤ ∥ M ( r ) v ∥ 2 ≤ κ κ + r 2 ∥ v ∥ 2 .
Thus M ( r ) M(r) M ( r ) is invertible for every $r\ge 0, and taking the supremum over ∥ v ∥ = 1 \|v\|=1 ∥ v ∥ = 1 in the upper bound gives
∥ M ( r ) ∥ ≤ κ + r 2 κ . \|M(r)\|\le\sqrt{\frac{\kappa+r^2}{\kappa}}. ∥ M ( r ) ∥ ≤ κ κ + r 2 .
as claimed.
To prove the third claim, take r > 0 r > 0 r > 0 and define
L ( r ) : = M ( r ) r . L(r) := \frac{M(r)}{r}. L ( r ) := r M ( r ) .
Differentiating gives
L ′ ( r ) = M ′ ( r ) r − M ( r ) r 2 = C ( r ) L ( r ) . L'(r) = \frac{M'(r)}{r} - \frac{M(r)}{r^2} = C(r)L(r). L ′ ( r ) = r M ′ ( r ) − r 2 M ( r ) = C ( r ) L ( r ) .
where
C ( r ) : = A ( r ) − 1 r I n = r ( H ( r ) + r 2 I n ) − 1 − 1 r I n = − 1 r H ( r ) ( H ( r ) + r 2 I n ) − 1 . \begin{aligned}
C(r) &:= A(r) - \frac{1}{r}I_n = r\left(H(r) + r^2 I_n\right)^{-1} - \frac{1}{r}I_n \\
&= -\frac{1}{r}H(r)\left(H(r) + r^2 I_n\right)^{-1}.
\end{aligned} C ( r ) := A ( r ) − r 1 I n = r ( H ( r ) + r 2 I n ) − 1 − r 1 I n = − r 1 H ( r ) ( H ( r ) + r 2 I n ) − 1 .
Therefore, for each fixed r > 0 r > 0 r > 0 , C ( r ) C(r) C ( r ) is symmetric and has eigenvalues
− h r ( h + r 2 ) , h ∈ spec ( H ( r ) ) ⊂ [ κ , Λ ] . -\frac{h}{r(h+r^2)}, \qquad h \in\operatorname{spec}(H(r)) \subset[\kappa,\Lambda]. − r ( h + r 2 ) h , h ∈ spec ( H ( r )) ⊂ [ κ , Λ ] .
In particular, C ( r ) C(r) C ( r ) is negative definite. Moreover, since h ↦ h / ( h + r 2 ) h \mapsto h/(h+r^2) h ↦ h / ( h + r 2 ) is increasing on ( 0 , ∞ ) (0,\infty) ( 0 , ∞ ) ,
∥ C ( r ) ∥ ≤ Λ r ( Λ + r 2 ) , ∫ R ∞ ∥ C ( r ) ∥ d r ≤ ∫ R ∞ Λ r ( Λ + r 2 ) d r ≤ 1 2 log ( 1 + Λ R 2 ) ≤ Λ 2 R 2 , (79) \|C(r)\| \le\frac{\Lambda}{r(\Lambda+r^2)}, \qquad\int_R^\infty\|C(r)\|\,\mathrm{d}r \le\int_R^\infty\frac{\Lambda}{r(\Lambda+r^2)}\,\mathrm{d}r \le\frac{1}{2}\log\left(1+\frac{\Lambda}{R^2}\right) \le\frac{\Lambda}{2R^2}, \tag*{(79)} ∥ C ( r ) ∥ ≤ r ( Λ + r 2 ) Λ , ∫ R ∞ ∥ C ( r ) ∥ d r ≤ ∫ R ∞ r ( Λ + r 2 ) Λ d r ≤ 2 1 log ( 1 + R 2 Λ ) ≤ 2 R 2 Λ , (79)
for all R > 0 R > 0 R > 0 .
Next, for every fixed v ∈ R n v \in\mathbb{R}^n v ∈ R n ,
d d r ∥ L ( r ) v ∥ 2 = 2 ⟨ L ( r ) v , C ( r ) L ( r ) v ⟩ ≤ 0. \frac{\mathrm{d}}{\mathrm{d}r}\|L(r)v\|^2 = 2\langle L(r)v,C(r)L(r)v\rangle\le0. d r d ∥ L ( r ) v ∥ 2 = 2 ⟨ L ( r ) v , C ( r ) L ( r ) v ⟩ ≤ 0.
Thus, for r ≥ R > 0 r \ge R > 0 r ≥ R > 0 ,
∥ L ( r ) v ∥ ≤ ∥ L ( R ) v ∥ . \|L(r)v\| \le\|L(R)v\|. ∥ L ( r ) v ∥ ≤ ∥ L ( R ) v ∥.
Taking the supremum over all unit vectors gives
∥ L ( r ) ∥ ≤ ∥ L ( R ) ∥ , r ≥ R . \|L(r)\| \le\|L(R)\|, \qquad r \ge R. ∥ L ( r ) ∥ ≤ ∥ L ( R ) ∥ , r ≥ R .
Fix R > 0 R > 0 R > 0 . For s ≥ t ≥ R s \ge t \ge R s ≥ t ≥ R , the fundamental theorem of calculus and the preceding estimates imply
∥ L ( s ) − L ( t ) ∥ ≤ ∫ t s ∥ L ′ ( r ) ∥ d r ≤ ∫ t s ∥ C ( r ) ∥ ⋅ ∥ L ( r ) ∥ d r ≤ ∥ L ( R ) ∥ ⋅ ∫ t ∞ ∥ C ( r ) ∥ d r ≤ Λ 2 t 2 ⋅ ∥ L ( R ) ∥ . \begin{aligned}
\|L(s)-L(t)\| &\le\int_t^s \|L'(r)\|\,\mathrm{d}r \le\int_t^s \|C(r)\|\cdot\|L(r)\|\,\mathrm{d}r \\
&\le\|L(R)\|\cdot\int_t^\infty\|C(r)\|\,\mathrm{d}r \le\frac{\Lambda}{2t^2}\cdot\|L(R)\|.
\end{aligned} ∥ L ( s ) − L ( t ) ∥ ≤ ∫ t s ∥ L ′ ( r ) ∥ d r ≤ ∫ t s ∥ C ( r ) ∥ ⋅ ∥ L ( r ) ∥ d r ≤ ∥ L ( R ) ∥ ⋅ ∫ t ∞ ∥ C ( r ) ∥ d r ≤ 2 t 2 Λ ⋅ ∥ L ( R ) ∥.
The last expression tends to zero as t → ∞ t \to\infty t → ∞ , uniformly over s ≥ t ≥ R s \ge t \ge R s ≥ t ≥ R , with R R R fixed. Hence L ( r ) L(r) L ( r ) is Cauchy as r → ∞ r \to\infty r → ∞ , so there exists Π ∈ R n × n \Pi\in\mathbb{R}^{n\times n} Π ∈ R n × n such that
L ( r ) ⟶ Π . L(r) \longrightarrow\Pi. L ( r ) ⟶ Π.
Taking t = R t=R t = R and letting s → ∞ s \to\infty s → ∞ in the same estimate yields
∥ Π − M ( R ) R ∥ ≤ Λ 2 R 2 ∥ M ( R ) R ∥ , \left\|\Pi-\frac{M(R)}{R}\right\| \le\frac{\Lambda}{2R^2}\left\|\frac{M(R)}{R}\right\|, Π − R M ( R ) ≤ 2 R 2 Λ R M ( R ) ,
which proves (G.8). The proof is completed. □
Lemma G.3. Recall y ( r , h ) y(r,h) y ( r , h ) from (G.6). We have the estimates
∥ y ( r , h 1 ) − y ( r , h 2 ) ∥ ≤ r κ + r 2 ∥ h 1 − h 2 ∥ ≤ ∥ h 1 − h 2 ∥ 2 κ , ∥ y ( r , h ) ∥ ≤ r ∥ h ∥ + ∥ ∇ W ( 0 ) ∥ κ + r 2 . (80) \lVert y(r,h_1)-y(r,h_2)\rVert\le\frac{r}{\kappa+r^2}\lVert h_1-h_2\rVert\le\frac{\lVert h_1-h_2\rVert}{2\sqrt{\kappa}}, \qquad\lVert y(r,h)\rVert\le\frac{r\lVert h\rVert+\lVert\nabla W(0)\rVert}{\kappa+r^2}. \tag*{(80)} ∥ y ( r , h 1 ) − y ( r , h 2 )∥ ≤ κ + r 2 r ∥ h 1 − h 2 ∥ ≤ 2 κ ∥ h 1 − h 2 ∥ , ∥ y ( r , h )∥ ≤ κ + r 2 r ∥ h ∥ + ∥ ∇ W ( 0 )∥ . (80)
Proof of Lemma G.3. By (I), for every u , v ∈ R 3 \mathbf{u},\mathbf{v}\in\mathbb{R}^3 u , v ∈ R 3 ,
⟨ ∇ W ( u ) − ∇ W ( v ) , u − v ⟩ ≥ κ ∥ u − v ∥ 2 . \langle\nabla W(\mathbf{u})-\nabla W(\mathbf{v}),\mathbf{u}-\mathbf{v}\rangle\ge\kappa\lVert\mathbf{u}-\mathbf{v}\rVert^2. ⟨ ∇ W ( u ) − ∇ W ( v ) , u − v ⟩ ≥ κ ∥ u − v ∥ 2 .
Fix r ≥ 0 r\ge0 r ≥ 0 and h 1 , h 2 ∈ R 3 \mathbf{h}_1,\mathbf{h}_2\in\mathbb{R}^3 h 1 , h 2 ∈ R 3 , and write y i : = y ( r , h i ) \mathbf{y}_i:=y(r,\mathbf{h}_i) y i := y ( r , h i ) for i = 1 , 2 i=1,2 i = 1 , 2 . The first-order optimality conditions give
∇ W ( y i ) + r 2 y i = r h i , i = 1 , 2. \nabla W(\mathbf{y}_i)+r^2\mathbf{y}_i=r\mathbf{h}_i,\qquad i=1,2. ∇ W ( y i ) + r 2 y i = r h i , i = 1 , 2.
Subtracting these equations and taking the inner product with y 1 − y 2 \mathbf{y}_1-\mathbf{y}_2 y 1 − y 2 , we obtain
( κ + r 2 ) ∥ y 1 − y 2 ∥ 2 ≤ ⟨ ∇ W ( y 1 ) − ∇ W ( y 2 ) , y 1 − y 2 ⟩ + r 2 ∥ y 1 − y 2 ∥ 2 = r ⟨ h 1 − h 2 , y 1 − y 2 ⟩ ≤ r ∥ h 1 − h 2 ∥ ∥ y 1 − y 2 ∥ . \begin{aligned}
(\kappa+r^2)\lVert\mathbf{y}_1-\mathbf{y}_2\rVert^2
&\le\langle\nabla W(\mathbf{y}_1)-\nabla W(\mathbf{y}_2),\mathbf{y}_1-\mathbf{y}_2\rangle+r^2\lVert\mathbf{y}_1-\mathbf{y}_2\rVert^2\\
&=r\langle\mathbf{h}_1-\mathbf{h}_2,\mathbf{y}_1-\mathbf{y}_2\rangle\\
&\le r\lVert\mathbf{h}_1-\mathbf{h}_2\rVert\lVert\mathbf{y}_1-\mathbf{y}_2\rVert.
\end{aligned} ( κ + r 2 ) ∥ y 1 − y 2 ∥ 2 ≤ ⟨ ∇ W ( y 1 ) − ∇ W ( y 2 ) , y 1 − y 2 ⟩ + r 2 ∥ y 1 − y 2 ∥ 2 = r ⟨ h 1 − h 2 , y 1 − y 2 ⟩ ≤ r ∥ h 1 − h 2 ∥ ∥ y 1 − y 2 ∥ .
If y 1 = y 2 \mathbf{y}_1=\mathbf{y}_2 y 1 = y 2 , the desired bound is immediate. Otherwise, dividing by ( κ + r 2 ) ∥ y 1 − y 2 ∥ (\kappa+r^2)\lVert\mathbf{y}_1-\mathbf{y}_2\rVert ( κ + r 2 ) ∥ y 1 − y 2 ∥ gives
∥ y ( r , h 1 ) − y ( r , h 2 ) ∥ ≤ r κ + r 2 ∥ h 1 − h 2 ∥ . \lVert y(r,\mathbf{h}_1)-y(r,\mathbf{h}_2)\rVert\le\frac{r}{\kappa+r^2}\lVert\mathbf{h}_1-\mathbf{h}_2\rVert. ∥ y ( r , h 1 ) − y ( r , h 2 )∥ ≤ κ + r 2 r ∥ h 1 − h 2 ∥ .
This yields the first estimate. The second estimate follows from
r κ + r 2 ≤ 1 2 κ . \frac{r}{\kappa+r^2}\le\frac{1}{2\sqrt{\kappa}}. κ + r 2 r ≤ 2 κ 1 .
For the remaining estimate, fix h ∈ R 3 \mathbf{h}\in\mathbb{R}^3 h ∈ R 3 and write y : = y ( r , h ) \mathbf{y}:=y(r,\mathbf{h}) y := y ( r , h ) . Its optimality condition can be rewritten as
∇ W ( y ) − ∇ W ( 0 ) + r 2 y = r h − ∇ W ( 0 ) . \nabla W(\mathbf{y})-\nabla W(0)+r^2\mathbf{y}=r\mathbf{h}-\nabla W(0). ∇ W ( y ) − ∇ W ( 0 ) + r 2 y = r h − ∇ W ( 0 ) .
Taking the inner product with y \mathbf{y} y and applying the strong-convexity inequality above with u = y \mathbf{u}=\mathbf{y} u = y and v = 0 \mathbf{v}=0 v = 0 , we obtain
( κ + r 2 ) ∥ y ∥ 2 ≤ ⟨ ∇ W ( y ) − ∇ W ( 0 ) , y ⟩ + r 2 ∥ y ∥ 2 = ⟨ r h − ∇ W ( 0 ) , y ⟩ ≤ ( r ∥ h ∥ + ∥ ∇ W ( 0 ) ∥ ) ⋅ ∥ y ∥ . \begin{aligned}
(\kappa+r^2)\lVert\mathbf{y}\rVert^2
&\le\langle\nabla W(\mathbf{y})-\nabla W(0),\mathbf{y}\rangle+r^2\lVert\mathbf{y}\rVert^2\\
&=\langle r\mathbf{h}-\nabla W(0),\mathbf{y}\rangle
\le(r\lVert\mathbf{h}\rVert+\lVert\nabla W(0)\rVert)\cdot\lVert\mathbf{y}\rVert.
\end{aligned} ( κ + r 2 ) ∥ y ∥ 2 ≤ ⟨ ∇ W ( y ) − ∇ W ( 0 ) , y ⟩ + r 2 ∥ y ∥ 2 = ⟨ r h − ∇ W ( 0 ) , y ⟩ ≤ ( r ∥ h ∥ + ∥ ∇ W ( 0 )∥) ⋅ ∥ y ∥ .
Dividing both sides by ( κ + r 2 ) ∥ y ∥ (\kappa+r^2)\lVert\mathbf{y}\rVert ( κ + r 2 ) ∥ y ∥ completes the proof. □
Other technical results
Lemma G.4. Recall y ( r , h ) y(r,h) y ( r , h ) from (G.6). As a function of ( r , h ) (r,h) ( r , h ) , we have y ∈ C 1 ( ( 0 , ∞ ) × R 3 , R 3 ) \mathbf{y}\in C^1((0,\infty)\times\mathbb{R}^3,\mathbb{R}^3) y ∈ C 1 (( 0 , ∞ ) × R 3 , R 3 ) .
Proof of Lemma G.4. For r > 0 r>0 r > 0 and h ∈ R 3 \mathbf{h}\in\mathbb{R}^3 h ∈ R 3 , define
Φ r , h ( u ) : = W ( u ) + 1 2 ∥ h − r u ∥ 2 . \Phi_{r,\mathbf{h}}(\mathbf{u}):=W(\mathbf{u})+\frac{1}{2}\lVert\mathbf{h}-r\mathbf{u}\rVert^2. Φ r , h ( u ) := W ( u ) + 2 1 ∥ h − r u ∥ 2 .
By (I), Φ r , h \Phi_{r,\mathbf{h}} Φ r , h is strongly convex and coercive. Consequently, it has a unique minimizer, which is precisely y ( r , h ) y(r,\mathbf{h}) y ( r , h ) . The first-order optimality condition is thus
∇ W ( y ( r , h ) ) + r 2 y ( r , h ) − r h = 0. (81) \nabla W(y(r,\mathbf{h}))+r^2y(r,\mathbf{h})-r\mathbf{h}=0. \tag*{(81)} ∇ W ( y ( r , h )) + r 2 y ( r , h ) − r h = 0. (81)
Consider the map
F : R 3 × ( 0 , ∞ ) × R 3 ⟶ R 3 , F ( u , r , h ) = ∇ W ( u ) + r 2 u − r h . F:\mathbb{R}^{3}\times(0,\infty)\times\mathbb{R}^{3}\longrightarrow\mathbb{R}^{3},\qquad F(u,r,h)=\nabla W(u)+r^{2}u-rh. F : R 3 × ( 0 , ∞ ) × R 3 ⟶ R 3 , F ( u , r , h ) = ∇ W ( u ) + r 2 u − r h .
Since W ∈ C 3 ( R 3 ) W\in C^{3}(\mathbb{R}^{3}) W ∈ C 3 ( R 3 ) , the map F F F is C 2 C^{2} C 2 . A direct computation yields
D u F ( u , r , h ) = ∇ 2 W ( u ) + r 2 I 3 . D_uF(u,r,h)=\nabla^{2}W(u)+r^{2}I_{3}. D u F ( u , r , h ) = ∇ 2 W ( u ) + r 2 I 3 .
By (I), D u F ( u , r , h ) ⪰ ( κ + r 2 ) I 3 D_uF(u,r,h)\succeq(\kappa+r^{2})I_{3} D u F ( u , r , h ) ⪰ ( κ + r 2 ) I 3 , and hence D u F ( u , r , h ) D_uF(u,r,h) D u F ( u , r , h ) is invertible for every ( u , r , h ) (u,r,h) ( u , r , h ) . In particular, it is invertible at u = y ( r , h ) u=y(r,h) u = y ( r , h ) . The implicit function theorem therefore implies that, in a neighborhood of every ( r 0 , h 0 ) ∈ ( 0 , ∞ ) × R 3 (r_{0},h_{0})\in(0,\infty)\times\mathbb{R}^{3} ( r 0 , h 0 ) ∈ ( 0 , ∞ ) × R 3 , the equation
F ( u , r , h ) = 0 F(u,r,h)=0 F ( u , r , h ) = 0
has a C 1 C^{1} C 1 solution u = y ~ ( r , h ) u=\widetilde{y}(r,h) u = y ( r , h ) .
For later use, differentiating (G.12) with respect to h h h gives
[ ∇ 2 W ( y ( r , h ) ) + r 2 I 3 ] D h y ( r , h ) = r I 3 , \left[\nabla^{2}W\left(y(r,h)\right)+r^{2}I_{3}\right]D_hy(r,h)=rI_{3}, [ ∇ 2 W ( y ( r , h ) ) + r 2 I 3 ] D h y ( r , h ) = r I 3 ,
and therefore
D h y ( r , h ) = r [ ∇ 2 W ( y ( r , h ) ) + r 2 I 3 ] − 1 . (82) D_hy(r,h)=r\left[\nabla^{2}W\left(y(r,h)\right)+r^{2}I_{3}\right]^{-1}. \tag*{(82)} D h y ( r , h ) = r [ ∇ 2 W ( y ( r , h ) ) + r 2 I 3 ] − 1 . (82)
The proof is completed. □
Lemma G.5 (Posterior covariance estimates). For all β ≥ 1 \beta\geq1 β ≥ 1 , r > 0 r>0 r > 0 , and h ∈ R 3 h\in\mathbb{R}^{3} h ∈ R 3 ,
r Λ + r 2 I 3 ⪯ D h y β ( r , h ) = β r ⋅ Cov π β , r , h ( Y ) ⪯ r κ + r 2 I 3 , (83) \frac{r}{\Lambda+r^{2}}I_{3}\preceq D_hy_{\beta}(r,h)=\beta r\cdot\operatorname{Cov}_{\pi_{\beta,r,h}}(Y)\preceq\frac{r}{\kappa+r^{2}}I_{3}, \tag*{(83)} Λ + r 2 r I 3 ⪯ D h y β ( r , h ) = β r ⋅ Cov π β , r , h ( Y ) ⪯ κ + r 2 r I 3 , (83) and
E π β , r , h ∥ Y − y ( r , h ) ∥ 2 ≤ 3 β ( κ + r 2 ) . \mathbb{E}_{\pi_{\beta,r,h}}\left\|Y-y(r,h)\right\|^{2}\leq\frac{3}{\beta(\kappa+r^{2})}. E π β , r , h ∥ Y − y ( r , h ) ∥ 2 ≤ β ( κ + r 2 ) 3 .
Proof of Lemma G.5. Fix β ≥ 1 \beta\geq1 β ≥ 1 , r > 0 r>0 r > 0 , and h ∈ R 3 h\in\mathbb{R}^{3} h ∈ R 3 , and put
Φ ( u ) : = W ( u ) + 1 2 ∥ h − r u ∥ 2 , y ∗ : = y ( r , h ) . \Phi(u):=W(u)+\frac{1}{2}\left\|h-ru\right\|^{2},\qquad y_{*}:=y(r,h). Φ ( u ) := W ( u ) + 2 1 ∥ h − r u ∥ 2 , y ∗ := y ( r , h ) .
Throughout the proof, expectations are taken with respect to Y ∼ π β , r , h Y\sim\pi_{\beta,r,h} Y ∼ π β , r , h . By (I),
( κ + r 2 ) I 3 ⪯ ∇ 2 Φ ( u ) = ∇ 2 W ( u ) + r 2 I 3 ⪯ ( Λ + r 2 ) I 3 . (84) (\kappa+r^{2})I_{3}\preceq\nabla^{2}\Phi(u)=\nabla^{2}W(u)+r^{2}I_{3}\preceq(\Lambda+r^{2})I_{3}. \tag*{(84)} ( κ + r 2 ) I 3 ⪯ ∇ 2 Φ ( u ) = ∇ 2 W ( u ) + r 2 I 3 ⪯ ( Λ + r 2 ) I 3 . (84)
Recall that y β ( r , h ) y_{\beta}(r,h) y β ( r , h ) is the conditional mean of the posterior distribution π β , r , h \pi_{\beta,r,h} π β , r , h . A direct differentiation yields
D h y β ( r , h ) = β r ⋅ Cov ( Y ) . D_hy_{\beta}(r,h)=\beta r\cdot\operatorname{Cov}(Y). D h y β ( r , h ) = β r ⋅ Cov ( Y ) .
Moreover, the Cramér–Rao and Brascamp–Lieb inequalities give, respectively,
1 β ( E ∇ 2 Φ ( Y ) ) − 1 ⪯ Cov ( Y ) ⪯ 1 β E [ ( ∇ 2 Φ ( Y ) ) − 1 ] . \frac{1}{\beta}\left(\mathbb{E}\nabla^{2}\Phi(Y)\right)^{-1}\preceq\operatorname{Cov}(Y)\preceq\frac{1}{\beta}\mathbb{E}\left[\left(\nabla^{2}\Phi(Y)\right)^{-1}\right]. β 1 ( E ∇ 2 Φ ( Y ) ) − 1 ⪯ Cov ( Y ) ⪯ β 1 E [ ( ∇ 2 Φ ( Y ) ) − 1 ] .
Using (G.15), we therefore obtain
1 β ( Λ + r 2 ) I 3 ⪯ Cov ( Y ) ⪯ 1 β ( κ + r 2 ) I 3 . \frac{1}{\beta(\Lambda+r^{2})}I_{3}\preceq\operatorname{Cov}(Y)\preceq\frac{1}{\beta(\kappa+r^{2})}I_{3}. β ( Λ + r 2 ) 1 I 3 ⪯ Cov ( Y ) ⪯ β ( κ + r 2 ) 1 I 3 .
Multiplying by β r \beta r β r and using D h ψ = β r Cov ( Y ) D_h\psi=\beta r\operatorname{Cov}(Y) D h ψ = β r Cov ( Y ) proves (G.14).
We now prove the second moment bound. For each i ∈ { 1 , 2 , 3 } i\in\{1,2,3\} i ∈ { 1 , 2 , 3 } , integration by parts gives
0 = ∫ R 3 ∂ u i [ ( u i − ( y ∗ ) i ) e − β Φ ( u ) ] d u . 0=\int_{\mathbb{R}^{3}}\partial_{u_i}\left[(u_i-(y_*)_i)e^{-\beta\Phi(u)}\right]\,\mathrm{d}u. 0 = ∫ R 3 ∂ u i [ ( u i − ( y ∗ ) i ) e − β Φ ( u ) ] d u .
Dividing by the normalizing constant and summing over i i i yields
3 = β ⋅ E [ ⟨ Y − y ∗ , ∇ Φ ( Y ) ⟩ ] . 3=\beta\cdot\mathbb{E}\left[\langle Y-y_*,\nabla\Phi(Y)\rangle\right]. 3 = β ⋅ E [ ⟨ Y − y ∗ , ∇Φ ( Y )⟩ ] .
On the other hand, strong convexity and ∇ Φ ( y ∗ ) = 0 \nabla\Phi(y_*)=0 ∇Φ ( y ∗ ) = 0 imply
⟨ u − y ∗ , ∇ Φ ( u ) ⟩ = ⟨ u − y ∗ , ∇ Φ ( u ) − ∇ Φ ( y ∗ ) ⟩ ≥ ( κ + r 2 ) ∥ u − y ∗ ∥ 2 . \langle u-y_*,\nabla\Phi(u)\rangle=\langle u-y_*,\nabla\Phi(u)-\nabla\Phi(y_*)\rangle\geq(\kappa+r^{2})\lVert u-y_*\rVert^{2}. ⟨ u − y ∗ , ∇Φ ( u )⟩ = ⟨ u − y ∗ , ∇Φ ( u ) − ∇Φ ( y ∗ )⟩ ≥ ( κ + r 2 ) ∥ u − y ∗ ∥ 2 .
Therefore,
β ( κ + r 2 ) ⋅ E ∥ Y − y ∗ ∥ 2 ≤ 3. \beta(\kappa+r^{2})\cdot\mathbb{E}\lVert Y-y_*\rVert^{2}\leq3. β ( κ + r 2 ) ⋅ E ∥ Y − y ∗ ∥ 2 ≤ 3.
The proof is completed. □ \square □
Lemma G.6. Let F : [ 0 , 1 ] × R d → R d F:[0,1]\times\mathbb{R}^{d}\to\mathbb{R}^{d} F : [ 0 , 1 ] × R d → R d be a C 2 C^{2} C 2 function and write
F s ( x ) : = F ( s , x ) , q s ( x ) : = ∂ s F s ( x ) . F_s(x):=F(s,x),\qquad q_s(x):=\partial_sF_s(x). F s ( x ) := F ( s , x ) , q s ( x ) := ∂ s F s ( x ) . For all φ ∈ C c 2 ( R d ) \varphi\in C_c^{2}(\mathbb{R}^{d}) φ ∈ C c 2 ( R d ) , we have
d d s [ φ ( F s ( x ) ) ⋅ det D F s ( x ) ] = div x [ φ ( F s ( x ) ) ⋅ cof ( D F s ( x ) ) T ⋅ q s ( x ) ] . \frac{\mathrm{d}}{\mathrm{d}s}\left[\varphi(F_s(x))\cdot\det DF_s(x)\right]=\operatorname{div}_x\left[\varphi(F_s(x))\cdot\operatorname{cof}(DF_s(x))^{\mathsf{T}}\cdot q_s(x)\right]. d s d [ φ ( F s ( x )) ⋅ det D F s ( x ) ] = div x [ φ ( F s ( x )) ⋅ cof ( D F s ( x ) ) T ⋅ q s ( x ) ] . where cof ( A ) \operatorname{cof}(A) cof ( A ) denotes the cofactor matrix of a matrix A A A .
The identity above is a variant of the well-known Piola identity (see [see 30 ] and the references therein] ).
Proof of Lemma G.6. We first show the identity for F F F 's that are smooth in both space and time. We compute
d d s [ φ ( F s ( x ) ) ⋅ det D F s ( x ) ] = d d s [ φ ( F s ( x ) ) ] ⋅ det D F s ( x ) + φ ( F s ( x ) ) ⋅ d d s [ det D F s ( x ) ] = ∇ φ ( F s ( x ) ) T ⋅ ∂ s F s ( x ) ⋅ det D F s ( x ) + φ ( F s ( x ) ) ⋅ ⟨ cof ( D F s ( x ) ) , D q s ( x ) ⟩ \begin{aligned}
\frac{\mathrm{d}}{\mathrm{d}s}\left[\varphi(F_s(x))\cdot\det DF_s(x)\right]
=\frac{\mathrm{d}}{\mathrm{d}s}\left[\varphi(F_s(x))\right]\cdot\det DF_s(x)+\varphi(F_s(x))\cdot\frac{\mathrm{d}}{\mathrm{d}s}\left[\det DF_s(x)\right] \\
&=\nabla\varphi(F_s(x))^{\mathsf{T}}\cdot\partial_sF_s(x)\cdot\det DF_s(x)+\varphi(F_s(x))\cdot\langle\operatorname{cof}(DF_s(x)),Dq_s(x)\rangle
\end{aligned} d s d [ φ ( F s ( x )) ⋅ det D F s ( x ) ] = d s d [ φ ( F s ( x )) ] ⋅ det D F s ( x ) + φ ( F s ( x )) ⋅ d s d [ det D F s ( x ) ] = ∇ φ ( F s ( x ) ) T ⋅ ∂ s F s ( x ) ⋅ det D F s ( x ) + φ ( F s ( x )) ⋅ ⟨ cof ( D F s ( x )) , D q s ( x )⟩
where we have used the facts that
d d s [ φ ( F s ( x ) ) ] = ∇ φ ( F s ( x ) ) T ⋅ ∂ s F s ( x ) and d d s [ det D F s ( x ) ] = ⟨ cof ( D F s ( x ) ) , D q s ( x ) ⟩ . \frac{\mathrm{d}}{\mathrm{d}s}\left[\varphi(F_s(x))\right]=\nabla\varphi(F_s(x))^{\mathsf{T}}\cdot\partial_sF_s(x)
\qquad\text{and}\qquad
\frac{\mathrm{d}}{\mathrm{d}s}\left[\det DF_s(x)\right]=\langle\operatorname{cof}(DF_s(x)),Dq_s(x)\rangle. d s d [ φ ( F s ( x )) ] = ∇ φ ( F s ( x ) ) T ⋅ ∂ s F s ( x ) and d s d [ det D F s ( x ) ] = ⟨ cof ( D F s ( x )) , D q s ( x )⟩ .
To proceed, let C s ( x ) : = cof ( D F s ( x ) ) C_s(x):=\operatorname{cof}(DF_s(x)) C s ( x ) := cof ( D F s ( x )) . We will simply write C s C_s C s in some calculations below. The Piola identity yields
Div x ( C s ( x ) ) = 0 (85) \operatorname{Div}_x(C_s(x))=0 \tag*{(85)} Div x ( C s ( x )) = 0 (85)
Here the operator Div ( ⋅ ) \operatorname{Div}(\cdot) Div ( ⋅ ) acts on matrices and is defined as
Div x C s : = ( div x (first row of C s ) ⋮ div x ( d -th row of C s ) ) = ( ∑ j = 1 d ∂ x j ( C s ) 1 j ⋮ ∑ j = 1 d ∂ x j ( C s ) d j ) ∈ R d . \operatorname{Div}_x C_s:=
\begin{pmatrix}
\operatorname{div}_x\text{ (first row of }C_s\text{)}\\
\vdots\\
\operatorname{div}_x\text{ (}d\text{-th row of }C_s\text{)}
\end{pmatrix}
=
\begin{pmatrix}
\sum_{j=1}^{d}\partial_{x_j}(C_s)_{1j}\\
\vdots\\
\sum_{j=1}^{d}\partial_{x_j}(C_s)_{dj}
\end{pmatrix}
\in\mathbb{R}^{d}. Div x C s := div x (first row of C s ) ⋮ div x ( d -th row of C s ) = ∑ j = 1 d ∂ x j ( C s ) 1 j ⋮ ∑ j = 1 d ∂ x j ( C s ) d j ∈ R d .
Thus
div x ( C s T q s ) = ∑ j = 1 d ∂ j [ ∑ i = 1 d ( C s ) i j ( q s ) i ] = ∑ 1 ≤ i , j ≤ d ∂ x j ( C s ) i j ( q s ) i + ∑ 1 ≤ i , j ≤ d ( C s ) i j ⋅ ∂ x j ( q s ) i = ∑ i = 1 d ( Div C s ) i ⋅ ( q s ) i ⏟ = 0 by x r e f ‘ ( G .16 ) ‘ + ⟨ C s , D q s ⟩ H S = ⟨ C s , D q s ⟩ H S . \begin{aligned}
\operatorname{div}_{x}\left(C_s^{\mathsf T}q_s\right)=\sum_{j=1}^{d}\partial_j\left[\sum_{i=1}^{d}(C_s)_{ij}(q_s)_i\right]
=\sum_{1\le i,j\le d}\partial_{x_j}(C_s)_{ij}(q_s)_i+\sum_{1\le i,j\le d}(C_s)_{ij}\cdot\partial_{x_j}(q_s)_i \\
&=\sum_{i=1}^{d}\underbrace{(\operatorname{Div}C_s)_i\cdot(q_s)_i}_{=0\text{ by }{xref}`(G.16)`}+\langle C_s,Dq_s\rangle_{\mathrm{HS}}=\langle C_s,Dq_s\rangle_{\mathrm{HS}}.
\end{aligned} div x ( C s T q s ) = j = 1 ∑ d ∂ j [ i = 1 ∑ d ( C s ) ij ( q s ) i ] = 1 ≤ i , j ≤ d ∑ ∂ x j ( C s ) ij ( q s ) i + 1 ≤ i , j ≤ d ∑ ( C s ) ij ⋅ ∂ x j ( q s ) i = i = 1 ∑ d = 0 by x r e f ‘ ( G .16 ) ‘ ( Div C s ) i ⋅ ( q s ) i + ⟨ C s , D q s ⟩ HS = ⟨ C s , D q s ⟩ HS .
Therefore,
d d s [ φ ( F s ( x ) ) ⋅ det D F s ( x ) ] = ∇ φ ( F s ( x ) ) T ⋅ q s ⋅ det D F s ( x ) + φ ( F s ( x ) ) ⋅ div x ( C s T q s ) \frac{d}{ds}\left[\varphi(F_s(x))\cdot\det DF_s(x)\right]=\nabla\varphi(F_s(x))^{\mathsf T}\cdot q_s\cdot\det DF_s(x)+\varphi(F_s(x))\cdot\operatorname{div}_{x}\left(C_s^{\mathsf T}q_s\right) d s d [ φ ( F s ( x )) ⋅ det D F s ( x ) ] = ∇ φ ( F s ( x ) ) T ⋅ q s ⋅ det D F s ( x ) + φ ( F s ( x )) ⋅ div x ( C s T q s )
On the other hand, we have
div x [ φ ( F s ( x ) ) ⋅ cof ( D F s ( x ) ) T ⋅ q s ( x ) ] = ∇ x [ φ ( F s ( x ) ) ] T ⋅ C s T q s + φ ( F s ( x ) ) ⋅ div x ( C s T q s ) = ∇ φ ( F s ) T ⋅ D F s ⋅ C s T q s + φ ( F s ( x ) ) ⋅ div x ( C s T q s ) \begin{aligned}
\operatorname{div}_{x}\left[\varphi(F_s(x))\cdot\operatorname{cof}\left(DF_s(x)\right)^{\mathsf T}\cdot q_s(x)\right]=\nabla_x\left[\varphi(F_s(x))\right]^{\mathsf T}\cdot C_s^{\mathsf T}q_s+\varphi(F_s(x))\cdot\operatorname{div}_{x}\left(C_s^{\mathsf T}q_s\right) \\
&=\nabla\varphi(F_s)^{\mathsf T}\cdot DF_s\cdot C_s^{\mathsf T}q_s+\varphi(F_s(x))\cdot\operatorname{div}_{x}\left(C_s^{\mathsf T}q_s\right)
\end{aligned} div x [ φ ( F s ( x )) ⋅ cof ( D F s ( x ) ) T ⋅ q s ( x ) ] = ∇ x [ φ ( F s ( x )) ] T ⋅ C s T q s + φ ( F s ( x )) ⋅ div x ( C s T q s ) = ∇ φ ( F s ) T ⋅ D F s ⋅ C s T q s + φ ( F s ( x )) ⋅ div x ( C s T q s )
Moreover, using the cofactor identity
A ⋅ cof ( A ) T = det ( A ) ⋅ I d , A\cdot\operatorname{cof}(A)^{\mathsf T}=\det(A)\cdot I_d, A ⋅ cof ( A ) T = det ( A ) ⋅ I d ,
with A ≡ D F s A\equiv DF_s A ≡ D F s , we arrive at
∇ φ ( F s ) T ⋅ D F s ⋅ C s T q s = ∇ φ ( F s ) T ⋅ q s ⋅ det D F s ( x ) . \nabla\varphi(F_s)^{\mathsf T}\cdot DF_s\cdot C_s^{\mathsf T}q_s=\nabla\varphi(F_s)^{\mathsf T}\cdot q_s\cdot\det DF_s(x). ∇ φ ( F s ) T ⋅ D F s ⋅ C s T q s = ∇ φ ( F s ) T ⋅ q s ⋅ det D F s ( x ) .
Thus the conclusion follows when F F F is smooth in both space and time. For a general C 1 C^1 C 1 map, take smooth approximations F ε → F F^\varepsilon\to F F ε → F in C l o c 1 C^1_{\mathrm{loc}} C loc 1 . This can be done by taking the convolution
F ε = F ∗ ρ ε . F^\varepsilon=F*\rho_\varepsilon. F ε = F ∗ ρ ε .
where ρ ε : R d + 1 → R \rho_\varepsilon:\mathbb{R}^{d+1}\to\mathbb{R} ρ ε : R d + 1 → R is a mollifier indexed by ε > 0 \varepsilon>0 ε > 0 , and the convolution above is understood in the coordinate-wise sense (applying the convolution to each coordinate of F F F , including the time coordinate). The readers are referred to Folland [ [18 ] for a comprehensive, compact exposition of mollifiers and their smoothing properties. Both φ ( F ε ) × det D x F ε \varphi(F^\varepsilon)\times\det D_xF^\varepsilon φ ( F ε ) × det D x F ε and φ ( F ε ) × cof ( D x F ε ) T × ∂ s F ε \varphi(F^\varepsilon)\times\operatorname{cof}(D_xF^\varepsilon)^{\mathsf T}\times\partial_sF^\varepsilon φ ( F ε ) × cof ( D x F ε ) T × ∂ s F ε converge locally uniformly to their corresponding limits as ε → 0 \varepsilon\to0 ε → 0 . □
Lemma G.7. Let A , B ∈ S + + d A,B\in\mathbb{S}^{d}_{++} A , B ∈ S ++ d and put
C t : = t A + ( 1 − t ) B , t ∈ [ 0 , 1 ] . C_t:=tA+(1-t)B,\qquad t\in[0,1]. C t := t A + ( 1 − t ) B , t ∈ [ 0 , 1 ] . It holds that
t λ min ( A ) + ( 1 − t ) λ min ( B ) ≤ λ min ( C t ) ≤ λ max ( C t ) ≤ t λ max ( A ) + ( 1 − t ) λ max ( B ) . t\lambda_{\min}(A)+(1-t)\lambda_{\min}(B)\le\lambda_{\min}(C_t)\le\lambda_{\max}(C_t)\le t\lambda_{\max}(A)+(1-t)\lambda_{\max}(B). t λ m i n ( A ) + ( 1 − t ) λ m i n ( B ) ≤ λ m i n ( C t ) ≤ λ m a x ( C t ) ≤ t λ m a x ( A ) + ( 1 − t ) λ m a x ( B ) .
Proof of Lemma G.7. The proof follows directly from the facts that the maximum eigenvalue is a convex functional on S + + d \mathbb{S}^{d}_{++} S ++ d , while the minimum eigenvalue is a concave functional on the same space. □
Lemma G.8. Let κ > 0 \kappa> 0 κ > 0 and let U , V ∈ R n × n U,V \in\mathbb{R}^{n \times n} U , V ∈ R n × n be symmetric matrices satisfying
U , V ⪰ κ I n . U,V \succeq\kappa I_n. U , V ⪰ κ I n .
Then, in the operator norm,
∥ U − 1 − V − 1 ∥ ≤ κ − 2 ∥ U − V ∥ . \left\|U^{-1}-V^{-1}\right\| \leq\kappa^{-2}\left\|U-V\right\|. U − 1 − V − 1 ≤ κ − 2 ∥ U − V ∥ .
Proof of Lemma G.8. The lower eigenvalue bounds imply that both matrices are invertible, with ∥ U − 1 ∥ , ∥ V − 1 ∥ ≤ κ − 1 \left\|U^{-1}\right\|,\left\|V^{-1}\right\| \leq\kappa^{-1} U − 1 , V − 1 ≤ κ − 1 . Using
U − 1 − V − 1 = U − 1 ( V − U ) V − 1 , U^{-1}-V^{-1}=U^{-1}(V-U)V^{-1}, U − 1 − V − 1 = U − 1 ( V − U ) V − 1 ,
we obtain
∥ U − 1 − V − 1 ∥ ≤ ∥ U − 1 ∥ ⋅ ∥ V − U ∥ ⋅ ∥ V − 1 ∥ ≤ κ − 2 ∥ U − V ∥ . \left\|U^{-1}-V^{-1}\right\| \leq\left\|U^{-1}\right\|\cdot\left\|V-U\right\|\cdot\left\|V^{-1}\right\| \leq\kappa^{-2}\left\|U-V\right\|. U − 1 − V − 1 ≤ U − 1 ⋅ ∥ V − U ∥ ⋅ V − 1 ≤ κ − 2 ∥ U − V ∥ .
The proof is completed. □
Technical results used in the proof of Theorem 4.2
Lemma H.1. Recall the mean vectors μ j ± \mu_j^{\pm} μ j ± in (E.1). Then
4 R 2 ≤ max 1 ≤ i , j ≤ N s , t ∈ { + , − } ( i , s ) ≠ ( j , t ) ∥ μ i s − μ j t ∥ 2 2 ≤ 4 R 2 + η 2 . 4R^2 \leq\max_{\substack{1\leq i,j\leq N\\s,t\in\{+,-\}\\(i,s)\ne(j,t)}}\left\|\mu_i^s-\mu_j^t\right\|_2^2 \leq4R^2+\eta^2. 4 R 2 ≤ 1 ≤ i , j ≤ N s , t ∈ { + , − } ( i , s ) = ( j , t ) max μ i s − μ j t 2 2 ≤ 4 R 2 + η 2 .
Proof of Lemma H.1. First note that the two means in the same pair satisfy
∥ μ j + − μ j − ∥ 2 = 2 R ∥ u π θ j 2 ∥ 2 = 2 R . \left\|\mu_j^+-\mu_j^-\right\|_2=2R\left\|u_{\pi\theta_j^2}\right\|_2=2R. μ j + − μ j − 2 = 2 R u π θ j 2 2 = 2 R .
Thus the maximum interpoint distance is at least 2 R 2R 2 R . On the other hand, for arbitrary indices i , j i,j i , j and signs s , t s,t s , t , we have
∥ μ i s − μ j t ∥ 2 = R 2 ∥ sign ( s ) ⋅ u π θ i 2 − sign ( t ) ⋅ u π θ j 2 ∥ 2 + η 2 ( θ i − θ j ) 2 ≤ 4 R 2 + η 2 . \left\|\mu_i^s-\mu_j^t\right\|^2=R^2\left\|\operatorname{sign}(s)\cdot u_{\pi\theta_i^2}-\operatorname{sign}(t)\cdot u_{\pi\theta_j^2}\right\|^2+\eta^2(\theta_i-\theta_j)^2\leq4R^2+\eta^2. μ i s − μ j t 2 = R 2 sign ( s ) ⋅ u π θ i 2 − sign ( t ) ⋅ u π θ j 2 2 + η 2 ( θ i − θ j ) 2 ≤ 4 R 2 + η 2 .
The proof is completed. □
The next result is a perturbation result for matrix-valued ODEs. Although it is standard, its non-asymptotic nature is helpful for the proof of Theorem 4.2.
Lemma H.2. Let C , C ~ : [ 0 , 1 ] → R d × d C,\widetilde{C}:[0,1]\to\mathbb{R}^{d\times d} C , C : [ 0 , 1 ] → R d × d be continuous matrix-valued mappings and suppose M , M ~ M,\widetilde{M} M , M solve the ODEs
d d t M ( t ) = C ( t ) ⋅ M ( t ) , d d t M ~ ( t ) = C ~ ( t ) ⋅ M ~ ( t ) , M ( 0 ) = M ~ ( 0 ) = I d . \frac{d}{dt}M(t)=C(t)\cdot M(t),\qquad\frac{d}{dt}\widetilde{M}(t)=\widetilde{C}(t)\cdot\widetilde{M}(t),\qquad M(0)=\widetilde{M}(0)=I_d. d t d M ( t ) = C ( t ) ⋅ M ( t ) , d t d M ( t ) = C ( t ) ⋅ M ( t ) , M ( 0 ) = M ( 0 ) = I d .
Then, it holds that
sup t ∈ [ 0 , 1 ] ∥ M ( t ) − M ~ ( t ) ∥ ≤ exp ( ∫ 0 1 ∥ C ( t ) ∥ d t ) ⋅ { exp [ ∫ 0 1 ∥ C ~ ( s ) − C ( s ) ∥ d s ] − 1 } . \sup_{t\in[0,1]}\left\|M(t)-\widetilde{M}(t)\right\|\leq\exp\left(\int_0^1\left\|C(t)\right\|\,dt\right)\cdot\left\{\exp\left[\int_0^1\left\|\widetilde{C}(s)-C(s)\right\|\,ds\right]-1\right\}. t ∈ [ 0 , 1 ] sup M ( t ) − M ( t ) ≤ exp ( ∫ 0 1 ∥ C ( t ) ∥ d t ) ⋅ { exp [ ∫ 0 1 C ( s ) − C ( s ) d s ] − 1 } .
Proof of Lemma H.2. Set
Δ ( t ) : = C ~ ( t ) − C ( t ) , A ( t ) : = ∫ 0 t ∥ C ( s ) ∥ d s , D ( t ) : = ∫ 0 t ∥ Δ ( s ) ∥ d s . \Delta(t) := \widetilde{C}(t) - C(t), \qquad A(t) := \int_{0}^{t} \lVert C(s)\rVert\,\mathrm{d}s, \qquad D(t) := \int_{0}^{t} \lVert\Delta(s)\rVert\,\mathrm{d}s. Δ ( t ) := C ( t ) − C ( t ) , A ( t ) := ∫ 0 t ∥ C ( s )∥ d s , D ( t ) := ∫ 0 t ∥ Δ ( s )∥ d s .
By Grönwall’s inequality,
∥ M ~ ( t ) ∥ ≤ exp ( ∫ 0 t ∥ C ~ ( s ) ∥ d s ) ≤ e A ( t ) + D ( t ) . \lVert\widetilde{M}(t)\rVert\leq\exp\left(\int_{0}^{t} \lVert\widetilde{C}(s)\rVert\,\mathrm{d}s\right) \leq e^{A(t)+D(t)}. ∥ M ( t )∥ ≤ exp ( ∫ 0 t ∥ C ( s )∥ d s ) ≤ e A ( t ) + D ( t ) .
Let E ( t ) : = M ~ ( t ) − M ( t ) E(t) := \widetilde{M}(t) - M(t) E ( t ) := M ( t ) − M ( t ) . Subtracting the two ODEs gives
E ′ ( t ) = C ( t ) E ( t ) + Δ ( t ) M ~ ( t ) , E ( 0 ) = 0 d × d . E'(t) = C(t)E(t) + \Delta(t)\widetilde{M}(t), \qquad E(0) = 0_{d\times d}. E ′ ( t ) = C ( t ) E ( t ) + Δ ( t ) M ( t ) , E ( 0 ) = 0 d × d .
Integrating the above and taking norms on both sides, we obtain
∥ E ( t ) ∥ ≤ ∫ 0 t [ ∥ C ( s ) ∥ ⋅ ∥ E ( s ) ∥ + ∥ Δ ( s ) ∥ ⋅ e A ( s ) + D ( s ) ] d s = : F ( t ) . \lVert E(t)\rVert\leq\int_{0}^{t} \left[\lVert C(s)\rVert\cdot\lVert E(s)\rVert+ \lVert\Delta(s)\rVert\cdot e^{A(s)+D(s)}\right]\,\mathrm{d}s =: F(t). ∥ E ( t )∥ ≤ ∫ 0 t [ ∥ C ( s )∥ ⋅ ∥ E ( s )∥ + ∥ Δ ( s )∥ ⋅ e A ( s ) + D ( s ) ] d s =: F ( t ) .
Since ∥ E ( t ) ∥ ≤ F ( t ) \lVert E(t)\rVert\leq F(t) ∥ E ( t )∥ ≤ F ( t ) , the function F F F satisfies
F ′ ( t ) ≤ A ′ ( t ) F ( t ) + D ′ ( t ) e A ( t ) + D ( t ) , F ( 0 ) = 0. F'(t) \leq A'(t)F(t) + D'(t)e^{A(t)+D(t)}, \qquad F(0) = 0. F ′ ( t ) ≤ A ′ ( t ) F ( t ) + D ′ ( t ) e A ( t ) + D ( t ) , F ( 0 ) = 0.
Multiplying by the integrating factor e − A ( t ) e^{-A(t)} e − A ( t ) yields
d d t [ e − A ( t ) F ( t ) ] ≤ D ′ ( t ) e D ( t ) = d d t e D ( t ) . \frac{\mathrm{d}}{\mathrm{d}t}\left[e^{-A(t)}F(t)\right] \leq D'(t)e^{D(t)} = \frac{\mathrm{d}}{\mathrm{d}t}e^{D(t)}. d t d [ e − A ( t ) F ( t ) ] ≤ D ′ ( t ) e D ( t ) = d t d e D ( t ) .
Integrating from 0 0 0 to t t t therefore gives
∥ E ( t ) ∥ ≤ F ( t ) ≤ e A ( t ) ( e D ( t ) − 1 ) . \lVert E(t)\rVert\leq F(t) \leq e^{A(t)}\left(e^{D(t)}-1\right). ∥ E ( t )∥ ≤ F ( t ) ≤ e A ( t ) ( e D ( t ) − 1 ) .
Finally, A A A and D D D are nondecreasing, so
sup t ∈ [ 0 , 1 ] ∥ M ~ ( t ) − M ( t ) ∥ ≤ exp ( ∫ 0 1 ∥ C ( s ) ∥ d s ) ⋅ [ exp ( ∫ 0 1 ∥ C ~ ( s ) − C ( s ) ∥ d s ) − 1 ] . \sup_{t\in[0,1]}\lVert\widetilde{M}(t)-M(t)\rVert\leq\exp\left(\int_{0}^{1}\lVert C(s)\rVert\,\mathrm{d}s\right)\cdot\left[\exp\left(\int_{0}^{1}\lVert\widetilde{C}(s)-C(s)\rVert\,\mathrm{d}s\right)-1\right]. t ∈ [ 0 , 1 ] sup ∥ M ( t ) − M ( t )∥ ≤ exp ( ∫ 0 1 ∥ C ( s )∥ d s ) ⋅ [ exp ( ∫ 0 1 ∥ C ( s ) − C ( s )∥ d s ) − 1 ] .
This completes the proof. □
Lemma H.3. Recall Z 1 p 1 ( x ) Z^{\mathrm{p1}}_{1}(x) Z 1 p1 ( x ) in (E.2). It holds that
sup x ∈ R 3 ∥ Z 1 p 1 ( x ) − x ∥ < ∞ . \sup_{x\in\mathbb{R}^{3}}\lVert Z^{\mathrm{p1}}_{1}(x)-x\rVert< \infty. x ∈ R 3 sup ∥ Z 1 p1 ( x ) − x ∥ < ∞.
Proof of Lemma H.3. Recall the rescaled flow q ( a ( t ) , x ) q(a(t),x) q ( a ( t ) , x ) in (E.3). Integrating (E.4) gives
Z 1 p 1 ( x ) − x = q ( 1 , x ) − x = ∫ 0 1 m ( a , q ( a , x ) ) d a . Z^{\mathrm{p1}}_{1}(x)-x=q(1,x)-x=\int_{0}^{1}m(a,q(a,x))\,\mathrm{d}a. Z 1 p1 ( x ) − x = q ( 1 , x ) − x = ∫ 0 1 m ( a , q ( a , x )) d a .
On the other hand,
∥ m ( a , q ) ∥ ≤ ∑ j = 1 N ∑ s ∈ { + , − } w j s ( a , q ) ∥ μ j s ∥ ≤ max 1 ≤ j ≤ N s ∈ { + , − } ∥ μ j s ∥ = max 1 ≤ j ≤ N R 2 + η 2 θ j 2 ≤ R 2 + η 2 = 4 π + 3 ε 8 . \begin{aligned}
\lVert m(a,q)\rVert\leq\sum_{j=1}^{N}\sum_{s\in\{+,-\}}w_{j}^{s}(a,q)\lVert\mu_{j}^{s}\rVert\\
&\leq\max_{\substack{1\leq j\leq N\\s\in\{+,-\}}}\lVert\mu_{j}^{s}\rVert\\
&= \max_{1\leq j\leq N}\sqrt{R^{2}+\eta^{2}\theta_{j}^{2}} \\
&\leq\sqrt{R^{2}+\eta^{2}}=\sqrt{4\pi+\frac{3\varepsilon}{8}}.
\end{aligned} ∥ m ( a , q )∥ ≤ j = 1 ∑ N s ∈ { + , − } ∑ w j s ( a , q ) ∥ μ j s ∥ ≤ 1 ≤ j ≤ N s ∈ { + , − } max ∥ μ j s ∥ = 1 ≤ j ≤ N max R 2 + η 2 θ j 2 ≤ R 2 + η 2 = 4 π + 8 3 ε .
Thus
sup x ∈ R 3 ∥ P 1 p ( x ) − x ∥ ≤ 4 π + ε . \sup_{x\in\mathbb{R}^{3}}\left\|P_{1}^{p}(x)-x\right\|\le4\pi+\varepsilon. x ∈ R 3 sup ∥ P 1 p ( x ) − x ∥ ≤ 4 π + ε .
The proof is completed. □
Lemma H.4. With the notation as in the proof of Theorem 4.2 , we have
sup a ∈ [ 0 , 1 ] ∥ ∑ j = 1 N α j ( a ) ⋅ P π θ j 2 − P π a 2 ∥ ≤ min { 1 ; π N + π 2 B + π η 2 B } . \sup_{a\in[0,1]}\left\|\sum_{j=1}^{N}\alpha_j(a)\cdot P_{\pi\theta_j^2}-P_{\pi a^2}\right\|\le\min\left\{1;\frac{\pi}{N}+\frac{\pi\sqrt{2}}{\sqrt{B}}+\frac{\pi\eta^2}{B}\right\}. a ∈ [ 0 , 1 ] sup j = 1 ∑ N α j ( a ) ⋅ P π θ j 2 − P π a 2 ≤ min { 1 ; N π + B π 2 + B π η 2 } .
Proof of Lemma H.4. Let Θ a \Theta_a Θ a be a random variable taking values in { θ 1 , … , θ N } \{\theta_1,\ldots,\theta_N\} { θ 1 , … , θ N } with the law
P ( Θ a = θ i ) = α i ( a ) = exp { − β ( a ) ⋅ [ θ i − c ( a ) ] 2 } ∑ k = 1 N exp { − β ( a ) ⋅ [ θ k − c ( a ) ] 2 } . \mathbb{P}(\Theta_a=\theta_i)=\alpha_i(a)=\frac{\exp\left\{-\beta(a)\cdot[\theta_i-c(a)]^2\right\}}{\sum_{k=1}^{N}\exp\left\{-\beta(a)\cdot[\theta_k-c(a)]^2\right\}}. P ( Θ a = θ i ) = α i ( a ) = ∑ k = 1 N exp { − β ( a ) ⋅ [ θ k − c ( a ) ] 2 } exp { − β ( a ) ⋅ [ θ i − c ( a ) ] 2 } .
where
β ( a ) : = B + a 2 η 2 2 , c ( a ) : = a η ⋅ Q ( a ) 2 B + a 2 η 2 . \beta(a):=B+\frac{a^2\eta^2}{2},\qquad c(a):=\frac{a\eta\cdot Q(a)}{2B+a^2\eta^2}. β ( a ) := B + 2 a 2 η 2 , c ( a ) := 2 B + a 2 η 2 a η ⋅ Q ( a ) .
Here Q Q Q is the solution of the ODE in (E.7). Note that the representations of the α i ( a ) \alpha_i(a) α i ( a ) follow directly from completing the squares in their original definitions; see the proof of Theorem 4.2 .
We first collect some elementary properties. It is clear from definitions that for all a ∈ [ 0 , 1 ] a\in[0,1] a ∈ [ 0 , 1 ] ,
β ( a ) ≥ B , c ( a ) ∈ [ 0 , 1 ] , ∣ c ( a ) − a ∣ ≤ η 2 2 B . \beta(a)\ge B,\qquad c(a)\in[0,1],\qquad|c(a)-a|\le\frac{\eta^2}{2B}. β ( a ) ≥ B , c ( a ) ∈ [ 0 , 1 ] , ∣ c ( a ) − a ∣ ≤ 2 B η 2 .
The last bound follows from the fact that L ≤ Q ( a ) ≤ L + a η L\le Q(a)\le L+a\eta L ≤ Q ( a ) ≤ L + a η , where we recall that L = 2 B / η L=2B/\eta L = 2 B / η . Next, observe that
sup a ∈ [ 0 , 1 ] ∥ ∑ j = 1 N α j ( a ) ⋅ P π θ j 2 − P π a 2 ∥ = sup a ∈ [ 0 , 1 ] ∥ E [ P π Θ a 2 − P π a 2 ] ∥ ≤ sup a ∈ [ 0 , 1 ] ∥ P π Θ a 2 − P π a 2 ∥ . \sup_{a\in[0,1]}\left\|\sum_{j=1}^{N}\alpha_j(a)\cdot P_{\pi\theta_j^2}-P_{\pi a^2}\right\|=\sup_{a\in[0,1]}\left\|\mathbb{E}\left[P_{\pi\Theta_a^2}-P_{\pi a^2}\right]\right\|\le\sup_{a\in[0,1]}\left\|P_{\pi\Theta_a^2}-P_{\pi a^2}\right\|. a ∈ [ 0 , 1 ] sup j = 1 ∑ N α j ( a ) ⋅ P π θ j 2 − P π a 2 = a ∈ [ 0 , 1 ] sup E [ P π Θ a 2 − P π a 2 ] ≤ a ∈ [ 0 , 1 ] sup P π Θ a 2 − P π a 2 .
A simple calculation yields
∥ P π Θ a 2 − P π a 2 ∥ = ∣ sin ( π Θ a 2 − π a 2 ) ∣ ≤ 2 π ⋅ ∣ Θ a − a ∣ \left\|P_{\pi\Theta_a^2}-P_{\pi a^2}\right\|=|\sin(\pi\Theta_a^2-\pi a^2)|\le2\pi\cdot|\Theta_a-a| P π Θ a 2 − P π a 2 = ∣ sin ( π Θ a 2 − π a 2 ) ∣ ≤ 2 π ⋅ ∣ Θ a − a ∣
where the last bound follows from the fact that ∣ sin ( x ) ∣ ≤ ∣ x ∣ |\sin(x)|\le|x| ∣ sin ( x ) ∣ ≤ ∣ x ∣ . Thus
sup a ∈ [ 0 , 1 ] ∥ ∑ j = 1 N α j ( a ) ⋅ P π θ j 2 − P π a 2 ∥ ≤ 2 π ⋅ sup a ∈ [ 0 , 1 ] E ∣ Θ a − a ∣ ≤ 2 π ⋅ sup a ∈ [ 0 , 1 ] { E ∣ Θ a − c ( a ) ∣ + ∣ c ( a ) − a ∣ } \sup_{a\in[0,1]}\left\|\sum_{j=1}^{N}\alpha_j(a)\cdot P_{\pi\theta_j^2}-P_{\pi a^2}\right\|\le2\pi\cdot\sup_{a\in[0,1]}\mathbb{E}|\Theta_a-a|\le2\pi\cdot\sup_{a\in[0,1]}\left\{\mathbb{E}|\Theta_a-c(a)|+|c(a)-a|\right\} a ∈ [ 0 , 1 ] sup j = 1 ∑ N α j ( a ) ⋅ P π θ j 2 − P π a 2 ≤ 2 π ⋅ a ∈ [ 0 , 1 ] sup E ∣ Θ a − a ∣ ≤ 2 π ⋅ a ∈ [ 0 , 1 ] sup { E ∣ Θ a − c ( a ) ∣ + ∣ c ( a ) − a ∣ }
≤ 2 π ⋅ sup a ∈ [ 0 , 1 ] E ∣ Θ a − c ( a ) ∣ + π η 2 B . \le2\pi\cdot\sup_{a\in[0,1]}\mathbb{E}|\Theta_a-c(a)|+\frac{\pi\eta^2}{B}. ≤ 2 π ⋅ a ∈ [ 0 , 1 ] sup E ∣ Θ a − c ( a ) ∣ + B π η 2 .
To finish the proof, it suffices to bound the first term in the last display. We will show that
E ∣ Θ a − c ( a ) ∣ ≤ 1 2 N + 1 2 B . (86) \mathbb{E}|\Theta_a-c(a)|\le\frac{1}{2N}+\frac{1}{\sqrt{2B}}. \tag*{(86)} E ∣ Θ a − c ( a ) ∣ ≤ 2 N 1 + 2 B 1 . (86)
To prove (H.1), we partition [ 0 , 1 ] [0,1] [ 0 , 1 ] into N N N intervals I j : = [ ( j − 1 ) / N , j / N ] I_j := [(j-1)/N,j/N] I j := [( j − 1 ) / N , j / N ] . Note that the partition yields θ j ∈ I j \theta_j \in I_j θ j ∈ I j . Put
H ( x , a ) : = ∫ − 1 / ( 2 N ) 1 / ( 2 N ) e − β ( a ) u 2 ⋅ cosh ( 2 β ( a ) ⋅ x u ) d u . H(x,a) := \int_{-1/(2N)}^{1/(2N)} e^{-\beta(a)u^2}\cdot\cosh\left(2\beta(a)\cdot xu\right)\,du. H ( x , a ) := ∫ − 1/ ( 2 N ) 1/ ( 2 N ) e − β ( a ) u 2 ⋅ cosh ( 2 β ( a ) ⋅ xu ) d u .
Write
∫ I j exp { − β ( a ) ⋅ [ y − c ( a ) ] 2 } d y = ∫ ( j − 1 ) / N j / N exp { − β ( a ) ⋅ ( y − θ j ) 2 } ⋅ exp { − 2 β ( a ) ⋅ ( y − θ j ) ( θ j − c ( a ) ) } d y × exp { − β ( a ) ⋅ [ θ j − c ( a ) ] 2 } = exp { − β ( a ) ⋅ [ θ j − c ( a ) ] 2 } × ∫ − 1 / ( 2 N ) 1 / ( 2 N ) exp { − β ( a ) ⋅ u 2 } ⋅ cosh [ 2 β ( a ) ⋅ u ( θ j − c ( a ) ) ] d u = exp { − β ( a ) ⋅ [ θ j − c ( a ) ] 2 } ⋅ H ( ∣ θ j − c ( a ) ∣ , a ) . (87) \begin{aligned}
\int_{I_j}\exp\left\{-\beta(a)\cdot[y-c(a)]^2\right\}\,dy
&=\int_{(j-1)/N}^{j/N}\exp\left\{-\beta(a)\cdot(y-\theta_j)^2\right\}\cdot\exp\left\{-2\beta(a)\cdot(y-\theta_j)(\theta_j-c(a))\right\}\,dy\\
&\qquad\times\exp\left\{-\beta(a)\cdot[\theta_j-c(a)]^2\right\}\\
&=\exp\left\{-\beta(a)\cdot[\theta_j-c(a)]^2\right\}\times\int_{-1/(2N)}^{1/(2N)}\exp\left\{-\beta(a)\cdot u^2\right\}\cdot\cosh\left[2\beta(a)\cdot u(\theta_j-c(a))\right]\,du\\
&=\exp\left\{-\beta(a)\cdot[\theta_j-c(a)]^2\right\}\cdot H\left(|\theta_j-c(a)|,a\right). \tag*{(87)}
\end{aligned} ∫ I j exp { − β ( a ) ⋅ [ y − c ( a ) ] 2 } d y = ∫ ( j − 1 ) / N j / N exp { − β ( a ) ⋅ ( y − θ j ) 2 } ⋅ exp { − 2 β ( a ) ⋅ ( y − θ j ) ( θ j − c ( a )) } d y × exp { − β ( a ) ⋅ [ θ j − c ( a ) ] 2 } = exp { − β ( a ) ⋅ [ θ j − c ( a ) ] 2 } × ∫ − 1/ ( 2 N ) 1/ ( 2 N ) exp { − β ( a ) ⋅ u 2 } ⋅ cosh [ 2 β ( a ) ⋅ u ( θ j − c ( a )) ] d u = exp { − β ( a ) ⋅ [ θ j − c ( a ) ] 2 } ⋅ H ( ∣ θ j − c ( a ) ∣ , a ) . (87)
Note that the second equality follows from the facts that exp ( − x ) = cosh ( x ) − sinh ( x ) \exp(-x)=\cosh(x)-\sinh(x) exp ( − x ) = cosh ( x ) − sinh ( x ) , sinh ( x ) \sinh(x) sinh ( x ) is an odd function, and a change of variable.
Since H ( x , a ) H(x,a) H ( x , a ) is non-decreasing on [ 0 , ∞ ) [0,\infty) [ 0 , ∞ ) for each a ∈ [ 0 , 1 ] a\in[0,1] a ∈ [ 0 , 1 ] , Chebyshev’s association inequality (Theorem 2.14 from [6 ] ) gives
0 ≤ Cov ( ∣ Θ a − c ( a ) ∣ , H ( ∣ Θ a − c ( a ) ∣ , a ) ) , ∀ a ∈ [ 0 , 1 ] . 0\leq\operatorname{Cov}\left(|\Theta_a-c(a)|,H\left(|\Theta_a-c(a)|,a\right)\right),\qquad\forall a\in[0,1]. 0 ≤ Cov ( ∣ Θ a − c ( a ) ∣ , H ( ∣ Θ a − c ( a ) ∣ , a ) ) , ∀ a ∈ [ 0 , 1 ] .
Equivalently, for all a ∈ [ 0 , 1 ] a\in[0,1] a ∈ [ 0 , 1 ] ,
E ( ∣ Θ a − c ( a ) ∣ ) ≤ E [ ∣ Θ a − c ( a ) ∣ ⋅ H ( ∣ Θ a − c ( a ) ∣ , a ) ] E H ( ∣ Θ a − c ( a ) ∣ , a ) \mathbb{E}\left(|\Theta_a-c(a)|\right)\leq\frac{\mathbb{E}\left[|\Theta_a-c(a)|\cdot H\left(|\Theta_a-c(a)|,a\right)\right]}{\mathbb{E}H\left(|\Theta_a-c(a)|,a\right)} E ( ∣ Θ a − c ( a ) ∣ ) ≤ E H ( ∣ Θ a − c ( a ) ∣ , a ) E [ ∣ Θ a − c ( a ) ∣ ⋅ H ( ∣ Θ a − c ( a ) ∣ , a ) ]
= ∑ j = 1 N ∣ θ j − c ( a ) ∣ ⋅ ∫ I j exp { − β ( a ) ⋅ [ y − c ( a ) ] 2 } d y ∫ 0 1 exp { − β ( a ) ⋅ [ y − c ( a ) ] 2 } d y ≤ (ii) 1 2 N + ∫ 0 1 ∣ y − c ( a ) ∣ ⋅ exp { − β ( a ) ⋅ [ y − c ( a ) ] 2 } d y ∫ 0 1 exp { − β ( a ) ⋅ [ y − c ( a ) ] 2 } d y . \begin{aligned}
&=\frac{\sum_{j=1}^{N}|\theta_j-c(a)|\cdot\int_{I_j}\exp\left\{-\beta(a)\cdot[y-c(a)]^2\right\}\,dy}{\int_0^1\exp\left\{-\beta(a)\cdot[y-c(a)]^2\right\}\,dy}\\
&\underset{\text{(ii)}}{\leq}\frac{1}{2N}+\frac{\int_0^1|y-c(a)|\cdot\exp\left\{-\beta(a)\cdot[y-c(a)]^2\right\}\,dy}{\int_0^1\exp\left\{-\beta(a)\cdot[y-c(a)]^2\right\}\,dy}.
\end{aligned} = ∫ 0 1 exp { − β ( a ) ⋅ [ y − c ( a ) ] 2 } d y ∑ j = 1 N ∣ θ j − c ( a ) ∣ ⋅ ∫ I j exp { − β ( a ) ⋅ [ y − c ( a ) ] 2 } d y (ii) ≤ 2 N 1 + ∫ 0 1 exp { − β ( a ) ⋅ [ y − c ( a ) ] 2 } d y ∫ 0 1 ∣ y − c ( a ) ∣ ⋅ exp { − β ( a ) ⋅ [ y − c ( a ) ] 2 } d y .
Here (i) follows from (H.2) and (ii) follows from the triangle inequality
∣ θ j − c ( a ) ∣ ≤ ∣ y − c ( a ) ∣ + ∣ θ j − y ∣ ≤ ∣ y − c ( a ) ∣ + 1 2 N |\theta_j-c(a)|\leq|y-c(a)|+|\theta_j-y|\leq|y-c(a)|+\frac{1}{2N} ∣ θ j − c ( a ) ∣ ≤ ∣ y − c ( a ) ∣ + ∣ θ j − y ∣ ≤ ∣ y − c ( a ) ∣ + 2 N 1
for all y ∈ I j y\in I_j y ∈ I j .
To finish the proof, we bound the second term in (ii). The Cauchy–Schwarz inequality yields
∫ 0 1 ∣ y − c ( a ) ∣ ⋅ exp { − β ( a ) ⋅ [ y − c ( a ) ] 2 } d y ∫ 0 1 exp { − β ( a ) ⋅ [ y − c ( a ) ] 2 } d y ≤ ∫ 0 1 ( y − c ( a ) ) 2 ⋅ exp { − β ( a ) ⋅ [ y − c ( a ) ] 2 } d y ∫ 0 1 exp { − β ( a ) ⋅ [ y − c ( a ) ] 2 } d y . \frac{\int_0^1|y-c(a)|\cdot\exp\left\{-\beta(a)\cdot[y-c(a)]^2\right\}\,dy}{\int_0^1\exp\left\{-\beta(a)\cdot[y-c(a)]^2\right\}\,dy}
\leq\sqrt{\frac{\int_0^1(y-c(a))^2\cdot\exp\left\{-\beta(a)\cdot[y-c(a)]^2\right\}\,dy}{\int_0^1\exp\left\{-\beta(a)\cdot[y-c(a)]^2\right\}\,dy}}. ∫ 0 1 exp { − β ( a ) ⋅ [ y − c ( a ) ] 2 } d y ∫ 0 1 ∣ y − c ( a ) ∣ ⋅ exp { − β ( a ) ⋅ [ y − c ( a ) ] 2 } d y ≤ ∫ 0 1 exp { − β ( a ) ⋅ [ y − c ( a ) ] 2 } d y ∫ 0 1 ( y − c ( a ) ) 2 ⋅ exp { − β ( a ) ⋅ [ y − c ( a ) ] 2 } d y .
To bound the last term, we do integration by parts to get
∫ 0 1 ( y − c ( a ) ) 2 ⋅ exp { − β ( a ) ⋅ [ y − c ( a ) ] 2 } d y = − 1 2 β ( a ) { ( y − c ( a ) ) ⋅ exp [ − β ( a ) ⋅ ( y − c ( a ) ) 2 ] } ∣ y = 0 y = 1 + 1 2 β ( a ) ⋅ ∫ 0 1 exp { − β ( a ) ⋅ [ y − c ( a ) ] 2 } d y ≤ 1 2 β ( a ) ⋅ ∫ 0 1 exp { − β ( a ) ⋅ [ y − c ( a ) ] 2 } d y . \begin{aligned}
\int_{0}^{1} (y-c(a))^{2}\cdot\exp\left\{-\beta(a)\cdot[y-c(a)]^{2}\right\}\,dy
&=-\frac{1}{2\beta(a)}\left.\left\{(y-c(a))\cdot\exp\left[-\beta(a)\cdot(y-c(a))^{2}\right]\right\}\right|_{y=0}^{y=1}\\
&\quad+\frac{1}{2\beta(a)}\cdot\int_{0}^{1}\exp\left\{-\beta(a)\cdot[y-c(a)]^{2}\right\}\,dy\\
&\leq\frac{1}{2\beta(a)}\cdot\int_{0}^{1}\exp\left\{-\beta(a)\cdot[y-c(a)]^{2}\right\}\,dy.
\end{aligned} ∫ 0 1 ( y − c ( a ) ) 2 ⋅ exp { − β ( a ) ⋅ [ y − c ( a ) ] 2 } d y = − 2 β ( a ) 1 { ( y − c ( a )) ⋅ exp [ − β ( a ) ⋅ ( y − c ( a ) ) 2 ] } y = 0 y = 1 + 2 β ( a ) 1 ⋅ ∫ 0 1 exp { − β ( a ) ⋅ [ y − c ( a ) ] 2 } d y ≤ 2 β ( a ) 1 ⋅ ∫ 0 1 exp { − β ( a ) ⋅ [ y − c ( a ) ] 2 } d y .
Consequently,
∫ 0 1 ∣ y − c ( a ) ∣ ⋅ exp { − β ( a ) ⋅ [ y − c ( a ) ] 2 } d y ∫ 0 1 exp { − β ( a ) ⋅ [ y − c ( a ) ] 2 } d y ≤ 1 2 β ( a ) ≤ 1 2 B . \frac{\int_{0}^{1}|y-c(a)|\cdot\exp\left\{-\beta(a)\cdot[y-c(a)]^{2}\right\}\,dy}{\int_{0}^{1}\exp\left\{-\beta(a)\cdot[y-c(a)]^{2}\right\}\,dy}
\leq\frac{1}{\sqrt{2\beta(a)}}\leq\frac{1}{\sqrt{2B}}. ∫ 0 1 exp { − β ( a ) ⋅ [ y − c ( a ) ] 2 } d y ∫ 0 1 ∣ y − c ( a ) ∣ ⋅ exp { − β ( a ) ⋅ [ y − c ( a ) ] 2 } d y ≤ 2 β ( a ) 1 ≤ 2 B 1 .
This yields (H.1) and in turn completes the proof. □ \square □