Introduction
Muon and its variants have emerged as practical optimizers for large-scale language-model training [23 ] (Kimi Team 2025), using Newton–Schulz iterations to approximately orthogonalize matrix-valued updates [13 ] . Their design is closely related to steepest descent under the spectral norm, where the polar factor provides an idealized update direction [2 ] . In practice, however, orthogonalization is performed by a fixed, small number of polynomial iterations. This distinction is particularly relevant near rank deficiency: exact polar orthogonalization maps every nonzero singular value to one, whereas finite Newton–Schulz iterations have a bounded amplification factor near zero at each fixed depth. Analyses of idealized spectral updates have identified learning dynamics distinct from gradient descent in concrete matrix models [29 ] (Ma et al. 2026; Li et al. 2026), while finite Newton–Schulz updates have been studied through convergence guarantees and their smoothing effect [14 ] (Li and Tsuchiya 2026). These developments motivate a direct study of how the finite number of iterations shapes the optimization trajectory.
Polynomial iterations for matrix orthogonalization have a long history in numerical linear algebra [16 , 4 , 8 , 9 ] . More recently, optimization analyses that explicitly retain finitely many Newton–Schulz iterations have investigated convergence to stationary points, the dependence on iteration depth, and the effects of stabilization and momentum [14 ] (Li and Tsuchiya 2026).
©2026 Yanlin Long.
License: CC-BY 4.0, see https://creativecommons.org/licenses/by/4.0/. 2026; Peng et al., [28 ] ; Li and Tan (2026). These results provide a useful foundation for studying the dynamical consequences of a fixed-depth spectral transformation.
Gradient-based training of factorized models from small initialization can exhibit incremental learning, with different components becoming appreciable at successive stages [30 , 10 , 3 , 12 ] (Li et al. 2021). In matrix factorization, this behavior can take the form of dominant directions being learned first, followed by weaker directions that increase the effective rank of the predictor. The trajectory may consequently pass near intermediate low-rank stationary points. Related analyses of homogeneous networks describe directional convergence near the origin and saddle points, as well as the subsequent trajectory toward the first saddle after escape [17 , 18 ] . A complementary approach studies training dynamics in high-dimensional limits, extracting effective structure from the evolving parameters or deriving dynamical mean-field descriptions of gradient flow [25 , 26 ] . Related work on feature learning and the distinction from lazy training provides another perspective on how different directions become accessible during optimization [6 , 7 ] .
Since Newton–Schulz iterations amplify small singular values of the normalized gradient, a natural question is whether a finite increase in the number of iterations can make weaker directions emerge early enough to change which stationary points the trajectory approaches. Answering this question requires tracking both singular-value amplification and the coupled evolution of different directions, as the gradient and its singular subspaces evolve with the factorized parameters.
We consider a two-layer neural network with quadratic activation and all second-layer weights fixed to one. This model admits an equivalent representation as a positive-semidefinite matrix factorization. Such quadratic networks provide a tractable setting for analyzing nonlinear training dynamics, and have also been studied through gradient flow and spectral gradient methods [25 , 26 , 5 ] . We study continuous-time optimization dynamics in which K K K K K K
For the analysis, we use a two-scale Gaussian initialization: one hidden row is initialized at scale ε \varepsilon ε ε b \varepsilon^{b} ε b b > 1 b > 1 b > 1 ε \varepsilon ε ε \varepsilon ε
Another question concerns the extent to which small-initialization asymptotics can determine the subsequent training trajectory. Under suitable structural conditions, gradient flow admits a precise description in which the initialization scale can be used to reverse-engineer the full training dynamics, leading also to a characterization of time-dependent implicit regularization [3 ] . For finite-depth Newton–Schulz dynamics, the nonlinear matrix iteration prevents this reconstruction argument from carrying over directly. Although the dynamics can be analyzed in a small neighborhood of the origin, the resulting local description does not by itself determine the trajectory after leaving that neighborhood. This creates two distinct dynamical scales: the entrance dynamics near the origin and the subsequent evolution away from it. Establishing how the entrance dynamics are transmitted to the later trajectory is one of the main results of this paper.
Following the framework of Li et al. (2021), which characterizes the first nonzero stationary point selected by gradient flow under suitable assumptions, we study the evolution from small initialization through the first stationary point encountered after escape. This allows us to compare the stationary points approached by gradient descent and by finite-depth Newton–Schulz dynamics. Under an additional structural assumption, we establish a sufficient large-depth guarantee: for all sufficiently large spectral depths, the selected Newton–Schulz continuation approaches a first stationary point whose loss is strictly smaller than that of the first stationary point selected by gradient descent. For smaller depths, our results do not provide a general criterion determining whether the trajectory visits or bypasses the gradient-descent stationary point. Moreover, without the additional assumption, the first stationary points reached by finite-depth Newton–Schulz dynamics and gradient descent may have the same loss. Such equal-loss behavior may reveal further dynamical structures, which we leave for future work.
The rest of this paper is organized as follows. § 2 introduces the model and pathwise comparison criteria. § 3 –§ 5 establish the Gaussian entrance law, continuation criteria, and eventual depth guarantee. § 6 presents an exact empirical task, and § 7 discusses the scope of the results and open questions. Detailed proofs are given in the appendices.
Notations. We write S r \mathbb{S}^{r} S r r × r r \times r r × r S + r \mathbb{S}_{+}^{r} S + r ⟨ A , B ⟩ = tr ( A T B ) \langle A, B\rangle= \operatorname{tr}(A^{\mathsf{T}}B) ⟨ A , B ⟩ = tr ( A T B ) A A A n + ( A ) n_{+}(A) n + ( A ) O ( m ) = { Q ∈ R m × m : Q T Q = I m } O(m) = \{Q \in\mathbb{R}^{m \times m} : Q^{\mathsf{T}}Q = I_{m}\} O ( m ) = { Q ∈ R m × m : Q T Q = I m }
Main Results
In this section, we introduce the empirical quadratic model and finite-depth Newton–Schulz dynamics, and formulate the pathwise comparison with gradient flow from small initialization.
Empirical quadratic learning
Fix m ≥ 2 m \ge2 m ≥ 2 r ≥ 3 r \ge3 r ≥ 3 { ( x l , y l ) } l = 1 n \{(x_l, y_l)\}_{l=1}^{n} {( x l , y l ) } l = 1 n x l ∈ R r x_l \in\mathbb{R}^{r} x l ∈ R r R r \mathbb{R}^{r} R r y l ∈ R y_l \in\mathbb{R} y l ∈ R q = min { m , r } q = \min\{m,r\} q = min { m , r } W ∈ R m × r W \in\mathbb{R}^{m \times r} W ∈ R m × r
M = W T W , Φ ( M ) = 1 2 n ∑ l = 1 n ( y l − x l T M x l ) 2 , f ( W ) = Φ ( W T W ) . (1) M = W^{\mathsf{T}}W,\qquad\Phi(M) = \frac{1}{2n}\sum_{l=1}^{n}(y_l - x_l^{\mathsf{T}} M x_l)^2,\qquad f(W) = \Phi(W^{\mathsf{T}}W). \tag*{(1)} M = W T W , Φ ( M ) = 2 n 1 l = 1 ∑ n ( y l − x l T M x l ) 2 , f ( W ) = Φ ( W T W ) . (1) This is the predictor of the quadratic network x ↦ ∥ W x ∥ 2 2 x \mapsto\lVert W x\rVert_{2}^{2} x ↦ ∥ W x ∥ 2 2
C = 1 n ∑ l y l x l x l T , L ( H ) = 1 n ∑ l ( x l T H x l ) x l x l T , R ( M ) = C − L ( M ) . (2) C = \frac{1}{n}\sum_l y_l x_l x_l^{\mathsf{T}},\qquad\mathcal{L}(H) = \frac{1}{n}\sum_l (x_l^{\mathsf{T}} H x_l)x_l x_l^{\mathsf{T}},\qquad R(M) = C - \mathcal{L}(M). \tag*{(2)} C = n 1 l ∑ y l x l x l T , L ( H ) = n 1 l ∑ ( x l T H x l ) x l x l T , R ( M ) = C − L ( M ) . (2) The gradient is
G ( W ) : = − ∇ f ( W ) = 2 W R ( W T W ) . G(W) := -\nabla f(W) = 2W R(W^{\mathsf{T}}W). G ( W ) := − ∇ f ( W ) = 2 W R ( W T W ) . Finite spectral depth and two-scale initialization
We use the cubic Newton–Schulz polynomial iteration (Björck and Bowie, [4 ] ; Higham, [8 ] )
P 0 ( X ) = X , P k + 1 ( X ) = 3 2 P k ( X ) − 1 2 P k ( X ) P k ( X ) T P k ( X ) . (3) P_0(X) = X,\qquad P_{k+1}(X) = \frac{3}{2}P_k(X) - \frac{1}{2}P_k(X)P_k(X)^{\mathsf{T}}P_k(X). \tag*{(3)} P 0 ( X ) = X , P k + 1 ( X ) = 2 3 P k ( X ) − 2 1 P k ( X ) P k ( X ) T P k ( X ) . (3) On the unit Frobenius ball, P k P_k P k p k p_k p k
p 0 ( s ) = s , p k + 1 ( s ) = 3 2 p k ( s ) − 1 2 p k ( s ) 3 , a K = p K ′ ( 0 ) = ( 3 / 2 ) K . (4) p_0(s) = s,\qquad p_{k+1}(s) = \frac{3}{2}p_k(s) - \frac{1}{2}p_k(s)^3,\qquad a_K = p_K'(0) = (3/2)^K. \tag*{(4)} p 0 ( s ) = s , p k + 1 ( s ) = 2 3 p k ( s ) − 2 1 p k ( s ) 3 , a K = p K ′ ( 0 ) = ( 3/2 ) K . (4) Fix b > 1 b > 1 b > 1 c η > 0 c_\eta> 0 c η > 0 K ∈ N 0 = { 0 , 1 , 2 , … } K \in\mathbb{N}_0 = \{0,1,2,\ldots\} K ∈ N 0 = { 0 , 1 , 2 , … } U ∈ R m × r U \in\mathbb{R}^{m \times r} U ∈ R m × r
W K ε , b ( 0 ; U ) = D ε , b U , D ε , b = diag ( ε , ε b , … , ε b ) , η ε = c η ε . (5) W_K^{\varepsilon,b}(0;U) = D_{\varepsilon,b}U,\qquad D_{\varepsilon,b} = \operatorname{diag}(\varepsilon,\varepsilon^b,\ldots,\varepsilon^b),\qquad\eta_\varepsilon= c_\eta\varepsilon. \tag*{(5)} W K ε , b ( 0 ; U ) = D ε , b U , D ε , b = diag ( ε , ε b , … , ε b ) , η ε = c η ε . (5) The actual trajectory, in physical time t t t
d W d t = P K ( G ( W ) ∥ G ( W ) ∥ F + η ε ) . (6) \frac{\mathrm{d}W}{\mathrm{d}t} = P_K\left(\frac{G(W)}{\lVert G(W)\rVert_F + \eta_\varepsilon}\right). \tag*{(6)} d t d W = P K ( ∥ G ( W ) ∥ F + η ε G ( W ) ) . (6) The additive stabilization in the denominator follows the original Muon implementation, which uses a fixed numerical constant of 10 − 7 10^{-7} 1 0 − 7 [13 ] ). The choice η ε = c η ε \eta_\varepsilon= c_\eta\varepsilon η ε = c η ε
Throughout the paper, we fix a finite spectral depth K K K ε ↓ 0 \varepsilon\downarrow0 ε ↓ 0 U U U
Since p K p_K p K q K q_K q K
p K ( s ) = s q K ( s 2 ) , q K ( 0 ) = a K . p_K(s) = s q_K(s^2),\qquad q_K(0) = a_K. p K ( s ) = s q K ( s 2 ) , q K ( 0 ) = a K . For G ∈ R m × r G \in\mathbb{R}^{m \times r} G ∈ R m × r η ≥ 0 \eta\ge0 η ≥ 0
H K , η ( G ) = { ( ∥ G ∥ F + η ) P K ( G / ( ∥ G ∥ F + η ) ) , ∥ G ∥ F + η > 0 , 0 , G = 0 , η = 0. (7) H_{K,\eta}(G) =
\begin{cases}
(\lVert G\rVert_F + \eta)P_K\left(G/(\lVert G\rVert_F + \eta)\right), & \lVert G\rVert_F + \eta> 0,\\
0, & G = 0,\ \eta= 0.
\end{cases} \tag*{(7)} H K , η ( G ) = { (∥ G ∥ F + η ) P K ( G / (∥ G ∥ F + η ) ) , 0 , ∥ G ∥ F + η > 0 , G = 0 , η = 0. (7) We use τ \tau τ d W / d τ = H K , η ( G ( W ) ) \mathrm{d}W/\mathrm{d}\tau= H_{K,\eta}(G(W)) d W / d τ = H K , η ( G ( W ))
For the actual predictor, write
M K ε , b ( t ; U ) = W K ε , b ( t ; U ) T W K ε , b ( t ; U ) . M_K^{\varepsilon,b}(t;U) = W_K^{\varepsilon,b}(t;U)^{\mathsf{T}}W_K^{\varepsilon,b}(t;U). M K ε , b ( t ; U ) = W K ε , b ( t ; U ) T W K ε , b ( t ; U ) . Continuation and the gradient-flow comparison
Multiplying the physical vector field by its positive denominator gives a convenient time parameter for continuation. The following bounds keep that change of time and all later finite-interval comparisons on the same ordered path.
Proposition 1 (Continuation of finite-depth dynamics). For a spanning empirical task, the following assertions hold in arbitrary positive factor dimensions. Fix finite K K K η ≥ 0 \eta\ge0 η ≥ 0
d W d τ = H K , η ( G ( W ) ) \frac{\mathrm{d}W}{\mathrm{d}\tau}=H_{K,\eta}(G(W)) d τ d W = H K , η ( G ( W )) is uniquely defined for all τ ≥ 0 \tau\ge0 τ ≥ 0 W s W_s W s Φ \Phi Φ f f f
− d f d τ ≥ ∥ ∇ f ∥ F 2 , ∥ d W d τ ∥ F ≤ a K ∥ ∇ f ∥ F , − d f d τ ≥ a K − 1 ∥ ∇ f ∥ F ∥ d W d τ ∥ F . (8) -\frac{\mathrm{d}f}{\mathrm{d}\tau}\ge\lVert\nabla f\rVert_F^2,\qquad\left\lVert\frac{\mathrm{d}W}{\mathrm{d}\tau}\right\rVert_F\le a_K\lVert\nabla f\rVert_F,\qquad-\frac{\mathrm{d}f}{\mathrm{d}\tau}\ge a_K^{-1}\lVert\nabla f\rVert_F\left\lVert\frac{\mathrm{d}W}{\mathrm{d}\tau}\right\rVert_F. \tag*{(8)} − d τ d f ≥ ∥ ∇ f ∥ F 2 , d τ d W F ≤ a K ∥ ∇ f ∥ F , − d τ d f ≥ a K − 1 ∥ ∇ f ∥ F d τ d W F . (8) For η > 0 \eta>0 η > 0 ∥ G ( W ) ∥ F + η \lVert G(W)\rVert_F+\eta ∥ G ( W ) ∥ F + η t ≥ 0 t\ge0 t ≥ 0 W s W_s W s
rank W ( t ) = rank W ( 0 ) ( 0 ≤ t < ∞ ) . \operatorname{rank}W(t)=\operatorname{rank}W(0)\qquad(0\le t<\infty). rank W ( t ) = rank W ( 0 ) ( 0 ≤ t < ∞ ) . The change of time is d t / d τ = ∥ G ( W ) ∥ F + η \mathrm{d}t/\mathrm{d}\tau=\lVert G(W)\rVert_F+\eta d t / d τ = ∥ G ( W ) ∥ F + η C 1 C^1 C 1
At depth zero, H 0 , η ( G ) = G H_{0,\eta}(G)=G H 0 , η ( G ) = G C C C
Assumption 2 (Task spectrum). The task matrix C C C
C v l = λ l v l , λ 1 > λ 2 ≥ ⋯ ≥ λ r , λ 2 > 0. Cv_l=\lambda_l v_l,\qquad\lambda_1>\lambda_2\ge\cdots\ge\lambda_r,\qquad\lambda_2>0. C v l = λ l v l , λ 1 > λ 2 ≥ ⋯ ≥ λ r , λ 2 > 0. Spectral separation conditions are common in analyses of incremental learning in matrix factorization [11 ] . Here, strict gap between the two largest eigenvalues, allowing multiplicities among the remaining ones. Removing the leading spectral gap may give rise to other interesting dynamical structures, which we leave for future work.
Proposition 3 (First gradient-flow stage). Under Assumption 2, there is a task-dependent nonconstant rank-one predictor trajectory M G ( s ) \mathcal{M}^{G}(s) M G ( s )
M 1 = lim s → ∞ M G ( s ) , M 1 R ( M 1 ) = 0. (9) M_1=\lim_{s\to\infty}\mathcal{M}^{G}(s),\qquad M_1R(M_1)=0. \tag*{(9)} M 1 = s → ∞ lim M G ( s ) , M 1 R ( M 1 ) = 0. (9) The trajectory is selected by the leading positive eigendirection of C C C b > 1 b>1 b > 1
lim ε ↓ 0 inf t ≥ 0 ∥ M 0 ε , b ( t ; U ) − M 1 ∥ F = 0 for Gaussian-almost every U . (10) \lim_{\varepsilon\downarrow0}\inf_{t\ge0}\left\lVert M_0^{\varepsilon,b}(t;U)-M_1\right\rVert_F=0\qquad\text{for Gaussian-almost every }U. \tag*{(10)} ε ↓ 0 lim t ≥ 0 inf M 0 ε , b ( t ; U ) − M 1 F = 0 for Gaussian-almost every U . (10) The explicit time-shifted selection law for a general positive semidefinite initial family is given in Appendix C. The endpoint M 1 M_1 M 1 R 1 = R ( M 1 ) R_1=R(M_1) R 1 = R ( M 1 ) ϕ 1 = Φ ( M 1 ) \phi_1=\Phi(M_1) ϕ 1 = Φ ( M 1 )
All-time distance and loss crossing
A visit is measured over the whole physical-time path. The second observable asks whether training reaches a level below the comparison loss, providing a quantitative loss improvement alongside separation.
Definition 4 (First nonzero stationary-point bypass). For the predictor M 1 M_1 M 1 Δ \Delta Δ
d 1 ( K , b , ε , U ) = inf t ≥ 0 ∥ M K ε , b ( t ; U ) − M 1 ∥ F ∥ M 1 ∥ F , (11) d_1(K,b,\varepsilon,U)=\inf_{t\geq0}\frac{\left\|M_K^{\varepsilon,b}(t;U)-M_1\right\|_{\mathrm{F}}}{\left\|M_1\right\|_{\mathrm{F}}}, \tag*{(11)} d 1 ( K , b , ε , U ) = t ≥ 0 inf ∥ M 1 ∥ F M K ε , b ( t ; U ) − M 1 F , (11) T Δ ( K , b , ε , U ) = inf { t ≥ 0 : Φ ( M K ε , b ( t ; U ) ) ≤ ϕ 1 − Δ } . (12) T_{\Delta}(K,b,\varepsilon,U)=\inf\left\{t\geq0:\Phi\left(M_K^{\varepsilon,b}(t;U)\right)\leq\phi_1-\Delta\right\}. \tag*{(12)} T Δ ( K , b , ε , U ) = inf { t ≥ 0 : Φ ( M K ε , b ( t ; U ) ) ≤ ϕ 1 − Δ } . (12) An empty hitting set has time + ∞ +\infty + ∞
A finite depth K K K M 1 M_1 M 1 U U U
lim inf ε ↓ 0 d 1 ( K , b , ε , U ) > 0 , lim sup ε ↓ 0 inf t ≥ 0 Φ ( M K ε , b ( t ; U ) ) < ϕ 1 . \liminf_{\varepsilon\downarrow0}d_1(K,b,\varepsilon,U)>0,\qquad\limsup_{\varepsilon\downarrow0}\inf_{t\geq0}\Phi\left(M_K^{\varepsilon,b}(t;U)\right)<\phi_1. ε ↓ 0 lim inf d 1 ( K , b , ε , U ) > 0 , ε ↓ 0 lim sup t ≥ 0 inf Φ ( M K ε , b ( t ; U ) ) < ϕ 1 . The second condition is equivalent to eventual finite crossing of a common loss level strictly below ϕ 1 \phi_1 ϕ 1 K b y p a s s ( b ) K_{\mathrm{bypass}}(b) K bypass ( b ) min ∅ = + ∞ \min\varnothing=+\infty min ∅ = + ∞
The infimum over time is taken separately at each positive ε \varepsilon ε
Gaussian Entrance at Finite Spectral Depth
An independent weak component can be suppressed during one stage and amplified during a later stage. The entrance rank records how many such components reach a fixed rescaled size before physical time becomes independent of initialization. This section first gives the finite recursion computing that rank, then identifies its limit in the actual Gaussian dynamics.
Rescaled shapes and normal growth rates
For σ = t / ε \sigma=t/\varepsilon σ = t / ε v = log t v=\log t v = log t
Y ε ( σ ) = W K ε , b ( ε σ ; U ) ε , Q ε ( t ) = W K ε , b ( t ; U ) t . Y^\varepsilon(\sigma)=\frac{W_K^{\varepsilon,b}(\varepsilon\sigma;U)}{\varepsilon},\qquad Q^\varepsilon(t)=\frac{W_K^{\varepsilon,b}(t;U)}{t}. Y ε ( σ ) = ε W K ε , b ( ε σ ; U ) , Q ε ( t ) = t W K ε , b ( t ; U ) . The exact relation is Q ϵ ( ϵ σ ) = Y ϵ ( σ ) / σ Q^{\epsilon}(\epsilon\sigma)=Y^{\epsilon}(\sigma)/\sigma Q ϵ ( ϵ σ ) = Y ϵ ( σ ) / σ
F K ( Q ) = P K ( Q C ∥ Q C ∥ F ) − Q , Q ≠ 0. (13) F_K(Q)=P_K\left(\frac{QC}{\lVert QC\rVert_F}\right)-Q,\qquad Q\ne0. \tag*{(13)} F K ( Q ) = P K ( ∥ QC ∥ F QC ) − Q , Q = 0. (13) Nonsingularity of C C C C 1 C^1 C 1 t 2 + ϵ / t t^2+\epsilon/t t 2 + ϵ / t
A matrix satisfying F K ( Q I , K ) = 0 F_K(Q_{I,K})=0 F K ( Q I , K ) = 0 Ran ( Q I , K T ) = span { v l : l ∈ I } \operatorname{Ran}(Q_{I,K}^{\mathsf T})=\operatorname{span}\{v_l:l\in I\} Ran ( Q I , K T ) = span { v l : l ∈ I } Q I , K Q_{I,K} Q I , K s I , K = ∥ Q I , K C ∥ F s_{I,K}=\lVert Q_{I,K}C\rVert_F s I , K = ∥ Q I , K C ∥ F { O Q I , K : O ∈ O ( m ) } \{OQ_{I,K}:O\in O(m)\} { O Q I , K : O ∈ O ( m )} d d d Q d , K = Q { 1 , … , d } , K Q_{d,K}=Q_{\{1,\ldots,d\},K} Q d , K = Q { 1 , … , d } , K s d , K = s { 1 , … , d } , K s_{d,K}=s_{\{1,\ldots,d\},K} s d , K = s { 1 , … , d } , K
For K ≥ 1 K\ge1 K ≥ 1 q l > 0 q_l>0 q l > 0
Q I , K T Q I , K = ∑ l ∈ I q l 2 v l v l T , Q_{I,K}^{\mathsf T}Q_{I,K}=\sum_{l\in I}q_l^2v_lv_l^{\mathsf T}, Q I , K T Q I , K = l ∈ I ∑ q l 2 v l v l T , q l = p K ( λ l q l s I , K ) , l ∈ I , s I , K 2 = ∑ l ∈ I λ l 2 q l 2 . (14) q_l=p_K\left(\frac{\lambda_lq_l}{s_{I,K}}\right),\quad l\in I,\qquad s_{I,K}^2=\sum_{l\in I}\lambda_l^2q_l^2. \tag*{(14)} q l = p K ( s I , K λ l q l ) , l ∈ I , s I , K 2 = l ∈ I ∑ λ l 2 q l 2 . (14) The rank-one shape is Q 1 , K = e 1 v 1 T Q_{1,K}=e_1v_1^{\mathsf T} Q 1 , K = e 1 v 1 T s 1 , K = λ 1 s_{1,K}=\lambda_1 s 1 , K = λ 1 K = 0 K=0 K = 0 v l v_l v l
g l ( d ) = a K λ l s d , K − 1 , l > d . (15) g_l^{(d)}=\frac{a_K\lambda_l}{s_{d,K}}-1,\qquad l>d. \tag*{(15)} g l ( d ) = s d , K a K λ l − 1 , l > d . (15) A positive next rate constructs the next leading support by Lemma 34. Negative task eigenvalues give rates below − 1 -1 − 1
For a symmetric matrix A A A n + ( A ) n_+(A) n + ( A )
p = min { m , n + ( C ) } , 2 ≤ p ≤ q = min { m , r } . (16) p=\min\{m,n_+(C)\},\qquad2\le p\le q=\min\{m,r\}. \tag*{(16)} p = min { m , n + ( C )} , 2 ≤ p ≤ q = min { m , r } . (16) The entrance may have rank below p p p q q q
Assumption 5 (Separated capacity prefix). The ordered eigenvalues in Assumption 2 satisfy
λ k > λ k + 1 ( 1 ≤ k ≤ min { p , r − 1 } ) . (17) \lambda_k>\lambda_{k+1}\qquad(1\le k\le\min\{p,r-1\}). \tag*{(17)} λ k > λ k + 1 ( 1 ≤ k ≤ min { p , r − 1 }) . (17) The gaps in this assumption distinguish the leading exterior column at each retained order. Eigenvalues beyond the tested prefix may repeat. When p = n + ( C ) < r p=n_+(C)<r p = n + ( C ) < r C C C
Executable clocks and exceptional exponents
The clock x d x_d x d t = O ( ε ) t = O(\varepsilon) t = O ( ε )
Definition 6 (Candidate entrance-rank computation). Fix K K K b > 1 b > 1 b > 1 p p p (16) .
Start at rank one with x 1 = 0 x_1 = 0 x 1 = 0 s 1 , K = λ 1 s_{1,K} = \lambda_1 s 1 , K = λ 1
x 2 = { b − 1 a K λ 2 / λ 1 − 1 , a K λ 2 / λ 1 > 1 , + ∞ , a K λ 2 / λ 1 ≤ 1. (18) x_2 =
\begin{cases}
\dfrac{b-1}{a_K\lambda_2/\lambda_1 - 1}, & a_K\lambda_2/\lambda_1 > 1,\\
+\infty, & a_K\lambda_2/\lambda_1 \le1.
\end{cases} \tag*{(18)} x 2 = ⎩ ⎨ ⎧ a K λ 2 / λ 1 − 1 b − 1 , + ∞ , a K λ 2 / λ 1 > 1 , a K λ 2 / λ 1 ≤ 1. (18) At each subsequently reached rank 2 ≤ d < p 2 \le d < p 2 ≤ d < p (14) and the next growth rate from (15) . Define
x d + 1 = { x d + ( b − 1 + x d ) ( 1 − λ d + 1 / λ d ) g d + 1 ( d ) , g d + 1 ( d ) > 0 , + ∞ , g d + 1 ( d ) ≤ 0. (19) x_{d+1} =
\begin{cases}
x_d + \dfrac{(b-1+x_d)(1-\lambda_{d+1}/\lambda_d)}{g^{(d)}_{d+1}}, & g^{(d)}_{d+1} > 0,\\
+\infty, & g^{(d)}_{d+1} \le0.
\end{cases} \tag*{(19)} x d + 1 = ⎩ ⎨ ⎧ x d + g d + 1 ( d ) ( b − 1 + x d ) ( 1 − λ d + 1 / λ d ) , + ∞ , g d + 1 ( d ) > 0 , g d + 1 ( d ) ≤ 0. (19) Proceed to rank d + 1 d+1 d + 1 x d + 1 < 1 x_{d+1} < 1 x d + 1 < 1 p p p x d + 1 ≥ 1 x_{d+1} \ge1 x d + 1 ≥ 1 d K ( b ) d_K(b) d K ( b )
To separate the exponent from this procedure, first build the longest leading support allowed by positive addition rates, without applying the test x d < 1 x_d < 1 x d < 1 Q 1 , K Q_{1,K} Q 1 , K d < p d < p d < p g d + 1 ( d ) > 0 g^{(d)}_{d+1} > 0 g d + 1 ( d ) > 0 p p p c 1 , K = 0 c_{1,K} = 0 c 1 , K = 0
c 2 , K = 1 a K λ 2 / λ 1 − 1 , c d + 1 , K = c d , K + ( 1 + c d , K ) ( 1 − λ d + 1 / λ d ) g d + 1 ( d ) , ( 2 ≤ d < p ) . (20) \begin{aligned}
c_{2,K} &= \frac{1}{a_K\lambda_2/\lambda_1 - 1},\\
c_{d+1,K} &= c_{d,K} + \frac{(1+c_{d,K})(1-\lambda_{d+1}/\lambda_d)}{g^{(d)}_{d+1}}, \qquad(2 \le d < p). \tag*{(20)}
\end{aligned} c 2 , K c d + 1 , K = a K λ 2 / λ 1 − 1 1 , = c d , K + g d + 1 ( d ) ( 1 + c d , K ) ( 1 − λ d + 1 / λ d ) , ( 2 ≤ d < p ) . (20) A displayed quotient is evaluated only when its addition rate is positive. At the first nonpositive rate, that next coefficient and all later coefficients are set to + ∞ +\infty + ∞
B C , K = { 1 + 1 / c d , K : 2 ≤ d ≤ p , 0 < c d , K < ∞ } , B C = ⋃ K ∈ N 0 B C , K . (21) \mathcal{B}_{C,K} = \{1 + 1/c_{d,K} : 2 \le d \le p,\ 0 < c_{d,K} < \infty\}, \qquad\mathcal{B}_C = \bigcup_{K\in\mathbb{N}_0} \mathcal{B}_{C,K}. \tag*{(21)} B C , K = { 1 + 1/ c d , K : 2 ≤ d ≤ p , 0 < c d , K < ∞ } , B C = K ∈ N 0 ⋃ B C , K . (21) The finite candidate equations determine B C , K \mathcal{B}_{C,K} B C , K Proposition 37 proves existence of each evaluated support and the identity x d = ( b − 1 ) c d , K x_d = (b-1)c_{d,K} x d = ( b − 1 ) c d , K ∣ B C , K ∣ ≤ p − 1 |\mathcal{B}_{C,K}| \le p-1 ∣ B C , K ∣ ≤ p − 1 B C \mathcal{B}_C B C
d K ( b ) = max { d ≤ p : ( b − 1 ) c d , K < 1 } . (22) d_K(b) = \max\{d \le p : (b-1)c_{d,K} < 1\}. \tag*{(22)} d K ( b ) = max { d ≤ p : ( b − 1 ) c d , K < 1 } . (22) Here a positive number times + ∞ +\infty + ∞ + ∞ +\infty + ∞ b < a K λ 2 / λ 1 b < a_K\lambda_2/\lambda_1 b < a K λ 2 / λ 1
The actual first-radius limit
For fixed K K K b b b ρ > 0 \rho> 0 ρ > 0
T ρ ε = inf { t ≥ 0 : ∥ M K ε , b ( t ; U ) ∥ F = ρ } , inf ∅ = + ∞ . (23) T_\rho^\varepsilon= \inf\{t \ge0 : \lVert M_K^{\varepsilon,b}(t;U)\rVert_F = \rho\}, \qquad\inf\varnothing= +\infty. \tag*{(23)} T ρ ε = inf { t ≥ 0 : ∥ M K ε , b ( t ; U ) ∥ F = ρ } , inf ∅ = + ∞. (23) The first-radius entrance rank is the rank of the predictor limit at this first hit, whenever the hit and limit exist. The theorem identifies both and fixes the continuation selected at the boundary t = 0 t = 0 t = 0
Theorem 7 (Gaussian first-radius entrance) Fix Assumptions 2 and 5, m ≥ 2 m \ge2 m ≥ 2 r ≥ 3 r \ge3 r ≥ 3 c η > 0 c_\eta> 0 c η > 0 K K K b > 1 b > 1 b > 1 b ∉ B C , K b \notin\mathcal{B}_{C,K} b ∈ / B C , K d = d K ( b ) d = d_K(b) d = d K ( b ) d d d W ‾ K , d \overline{W}_{K,d} W K , d
W ˙ = P K ( G ( W ) / ∥ G ( W ) ∥ F ) , W ( t ) / t ⟶ Q d , K , \dot{W} = P_K(G(W)/\lVert G(W)\rVert_F), \qquad W(t)/t \longrightarrow Q_{d,K}, W ˙ = P K ( G ( W ) / ∥ G ( W ) ∥ F ) , W ( t ) / t ⟶ Q d , K , unique in this fixed-rank boundary class up to a constant left orthogonal transformation. For some fixed t ∗ > 0 t_* > 0 t ∗ > 0
W ‾ K , d ( t ) = t Q d , K + O ( t 3 ) , W ‾ ˙ K , d ( t ) = Q d , K + O ( t 2 ) . (24) \overline{W}_{K,d}(t) = tQ_{d,K} + O(t^3), \qquad\dot{\overline{W}}_{K,d}(t) = Q_{d,K} + O(t^2). \tag*{(24)} W K , d ( t ) = t Q d , K + O ( t 3 ) , W ˙ K , d ( t ) = Q d , K + O ( t 2 ) . (24) There is a measurable Gaussian-full set G K , b \mathcal{G}_{K,b} G K , b U ∈ G K , b U \in\mathcal{G}_{K,b} U ∈ G K , b
lim ε ↓ 0 inf O ∈ O ( m ) ∥ O W K ε , b ( t ; U ) − W ‾ K , d ( t ) ∥ F = 0 locally uniformly on ( 0 , t ∗ ) . (25) \lim_{\varepsilon\downarrow0}\inf_{O\in O(m)}\lVert O W_K^{\varepsilon,b}(t;U) - \overline{W}_{K,d}(t)\rVert_F = 0 \quad\text{locally uniformly on }(0,t_*). \tag*{(25)} ε ↓ 0 lim O ∈ O ( m ) inf ∥ O W K ε , b ( t ; U ) − W K , d ( t ) ∥ F = 0 locally uniformly on ( 0 , t ∗ ) . (25) For each sufficiently small fixed ρ > 0 \rho> 0 ρ > 0 t ρ t_\rho t ρ ∥ W ‾ K , d ( t ) ⊤ W ‾ K , d ( t ) ∥ F = ρ \lVert\overline{W}_{K,d}(t)^\top\overline{W}_{K,d}(t)\rVert_F = \rho ∥ W K , d ( t ) ⊤ W K , d ( t ) ∥ F = ρ U ∈ G K , b U \in\mathcal{G}_{K,b} U ∈ G K , b ε \varepsilon ε t ρ t_\rho t ρ
M K ε , b ( T ρ ε ; U ) ⟶ W ‾ K , d ( t ρ ) ⊤ W ‾ K , d ( t ρ ) . (26) M_K^{\varepsilon,b}(T_\rho^\varepsilon;U) \longrightarrow\overline{W}_{K,d}(t_\rho)^\top\overline{W}_{K,d}(t_\rho). \tag*{(26)} M K ε , b ( T ρ ε ; U ) ⟶ W K , d ( t ρ ) ⊤ W K , d ( t ρ ) . (26) This predictor is independent of U U U d K ( b ) d_K(b) d K ( b ) b ∉ B C b \notin\mathcal{B}_C b ∈ / B C K K K
The time interval and admissible radii are determined by the task and selected shape. Constants may depend on K K K b > 1 b > 1 b > 1
Proposition 8 (Depth monotonicity of entrance rank) Under Assumptions 2 and 5, the algebraically computed rank d K ( b ) d_K(b) d K ( b ) K K K Theorem 7 .
The normalization of an existing support of rank at least two increases with depth, but its ratio to a K a_K a K
How Gaussian independent scales determine the clocks
The initial-layer equations separate the leading row from the weak rows on every fixed finite σ \sigma σ σ 0 ( U ) \sigma_0(U) σ 0 ( U ) ε ↓ 0 \varepsilon\downarrow0 ε ↓ 0
In logarithmic initial-layer time
μ = log σ = log ( t / ε ) , (27) \mu= \log\sigma= \log(t/\varepsilon), \tag*{(27)} μ = log σ = log ( t / ε ) , (27) the leading independent rates are g l ( 1 ) = a K λ l / λ 1 − 1 g_l^{(1)} = a_K\lambda_l/\lambda_1 - 1 g l ( 1 ) = a K λ l / λ 1 − 1 t = ε σ 0 ( U ) t = \varepsilon\sigma_0(U) t = ε σ 0 ( U ) h h h h ( b − 1 ) log ε + O U ( 1 ) h(b-1)\log\varepsilon+ O_U(1) h ( b − 1 ) log ε + O U ( 1 )
For a later rank-d d d Z Z Z Z Z Z ε / t \varepsilon/t ε / t
The first waiting fraction already illustrates the balance. On the rank-one stage, a weak independent amplitude begins at order ε b − 1 \varepsilon^{b-1} ε b − 1 g 2 ( 1 ) g_2^{(1)} g 2 ( 1 ) − ( b − 1 ) log ε / g 2 ( 1 ) + O U ( 1 ) -(b-1)\log\varepsilon/g_2^{(1)} + O_U(1) − ( b − 1 ) log ε / g 2 ( 1 ) + O U ( 1 )
log t 2 ε = ( 1 − b − 1 g 2 ( 1 ) ) log ε + O U ( 1 ) . \log t_2^\varepsilon= \left(1-\frac{b-1}{g_2^{(1)}}\right)\log\varepsilon+ O_U(1). log t 2 ε = ( 1 − g 2 ( 1 ) b − 1 ) log ε + O U ( 1 ) . This crossing remains on a vanishing physical scale exactly when x 2 < 1 x_2 < 1 x 2 < 1
For two completed additions, consider an unused task label l > 2 l > 2 l > 2
( b − 1 ) − g l ( 1 ) x 2 = ( b − 1 + x 2 ) ( 1 − λ l λ 2 ) . (b-1)-g_l^{(1)}x_2=(b-1+x_2)\left(1-\frac{\lambda_l}{\lambda_2}\right). ( b − 1 ) − g l ( 1 ) x 2 = ( b − 1 + x 2 ) ( 1 − λ 2 λ l ) . The equality follows from ( b − 1 ) = g 2 ( 1 ) x 2 (b-1)=g_2^{(1)}x_2 ( b − 1 ) = g 2 ( 1 ) x 2 g l ( 1 ) + 1 = ( λ l / λ 2 ) ( g 2 ( 1 ) + 1 ) g_l^{(1)}+1=(\lambda_l/\lambda_2)(g_2^{(1)}+1) g l ( 1 ) + 1 = ( λ l / λ 2 ) ( g 2 ( 1 ) + 1 ) d d d ( b − 1 + x d ) ( 1 − λ l / λ d ) (b-1+x_d)(1-\lambda_l/\lambda_d) ( b − 1 + x d ) ( 1 − λ l / λ d ) l = d + 1 l=d+1 l = d + 1
The residence interval can have logarithmic length proportional to log ( 1 / ε ) \log(1/\varepsilon) log ( 1/ ε ) Lemma 39 give this bound uniformly in residence length. For the two explicit rescaling terms, their accumulated contribution from t = ε σ 0 t=\varepsilon\sigma_0 t = ε σ 0 t ∗ t_* t ∗
∫ ε σ 0 t ∗ ( t 2 + ε t ) d t t = t ∗ 2 − ε 2 σ 0 2 2 + 1 σ 0 − ε t ∗ ≤ t ∗ 2 2 + 1 σ 0 . \int_{\varepsilon\sigma_0}^{t_*}\left(t^2+\frac{\varepsilon}{t}\right)\frac{\mathrm{d}t}{t}=\frac{t_*^2-\varepsilon^2\sigma_0^2}{2}+\frac{1}{\sigma_0}-\frac{\varepsilon}{t_*}\leq\frac{t_*^2}{2}+\frac{1}{\sigma_0}. ∫ ε σ 0 t ∗ ( t 2 + t ε ) t d t = 2 t ∗ 2 − ε 2 σ 0 2 + σ 0 1 − t ∗ ε ≤ 2 t ∗ 2 + σ 0 1 . The stable tangent equation and the increasing normal component give corresponding integrated bounds for the remaining coefficient terms. The resulting control is multiplicative on each independent volume: it preserves a positive lower bound relative to its incoming value, even when that value is a high power of ε \varepsilon ε
log t d + 1 ε = ( 1 − x d + 1 ) log ε + O U ( 1 ) . \log t_{d+1}^{\varepsilon}=(1-x_{d+1})\log\varepsilon+O_U(1). log t d + 1 ε = ( 1 − x d + 1 ) log ε + O U ( 1 ) . After that crossing, projection away from the new direction and relative transport on a fixed connection segment preserve every remaining independent exponent. This produces the numerator ( b − 1 + x d ) ( 1 − λ d + 1 / λ d ) (b-1+x_d)(1-\lambda_{d+1}/\lambda_d) ( b − 1 + x d ) ( 1 − λ d + 1 / λ d ) (19) .
At the final stage, the unused component tends to zero on each compact positive-time interval. A stable Gram equation with a zero-time boundary then selects the rank-d K ( b ) d_K(b) d K ( b ) (24) , a speed bound before a fixed early time, and transverse crossing at the selected section identify the actual first hit. The definitions and estimates for these steps appear in Appendix B.
The selected entrance solves the empirical equation at positive time. In its rescaled Gram equation, the stable tangent correction has order t 2 t^2 t 2 O ( t 3 ) O(t^3) O ( t 3 ) (24) . The loss operator therefore enters the selected continuation even though the finite clock recursion uses only the task spectrum. This separates two computations: the spectrum and initialization exponent determine the entrance rank, and the empirical operator determines the continuation from its boundary shape.
The argument also distinguishes convergence at a prescribed positive time from convergence at the first section. The uniform speed bound excludes an earlier actual hit on an initial interval. On the remaining compact interval before the selected hit, the limiting predictor stays strictly below the chosen radius. Uniform convergence preserves that margin. Transversality then places the actual first hit in every small neighborhood of the selected hitting time. This gives the time and predictor conclusions of Theorem 7 on the same seed event.
Selected Continuation and All-Time Bypass
The entrance theorem supplies a deterministic predictor at a small section. Continuing from that selected entrance gives a finite comparison segment. A strict loss decrease on such a segment subsequently controls all later actual times. At equal terminal loss, control instead comes from the finite length of trajectories near an analytic stationary point.
The selected empirical continuation
Write φ K ( τ , Z ) \varphi_K(\tau,Z) φ K ( τ , Z ) Z Z Z
d W d τ = H K , 0 ( G ( W ) ) . (28) \frac{\mathrm{d}W}{\mathrm{d}\tau}=H_{K,0}(G(W)). \tag*{(28)} d τ d W = H K , 0 ( G ( W )) . (28) For a depth and exponent covered by Theorem 7 , choose a factor Z Z Z W ‾ K , d K ( b ) \overline{W}_{K,d_K(b)} W K , d K ( b )
M s , K = lim τ → ∞ φ K ( τ , Z ) T φ K ( τ , Z ) , ℓ K , d K ( b ) = Φ ( M s , K ) . M_{s,K}=\lim_{\tau\to\infty}\varphi_K(\tau,Z)^{\mathsf{T}}\varphi_K(\tau,Z),\qquad\ell_{K,d_K(b)}=\Phi(M_{s,K}). M s , K = τ → ∞ lim φ K ( τ , Z ) T φ K ( τ , Z ) , ℓ K , d K ( b ) = Φ ( M s , K ) . Proposition 1 gives existence. Uniqueness of continuation, its positive time change, and left orthogonal equivariance make these objects independent of the chosen positive entrance point and hidden frame. The starting point is fixed by the selected entrance. The notation ℓ K , d \ell_{K,d} ℓ K , d d d d
Assumption 9 (A nonglobal comparison predictor). The first gradient-flow endpoint satisfies
λ max ( R 1 ) > 0. (29) \lambda_{\max}(R_1)>0. \tag*{(29)} λ m a x ( R 1 ) > 0. (29) Under this condition every factor of M 1 M_1 M 1 Corollary 47 . It also implies that the minimum PSD loss is strictly below ϕ 1 \phi_1 ϕ 1 § 5 .
Theorem 10 (Strict selected loss implies actual bypass). Under the pointwise hypotheses of Theorem 7 and Assumption 9, suppose ℓ K , d K ( b ) < ϕ 1 \ell_{K,d_K(b)}<\phi_1 ℓ K , d K ( b ) < ϕ 1 d ∗ , Δ ∗ > 0 d_*,\Delta_*>0 d ∗ , Δ ∗ > 0 U U U
d 1 ( K , b , ε , U ) ≥ d ∗ , T Δ ∗ ( K , b , ε , U ) < ∞ d_1(K,b,\varepsilon,U)\ge d_*,\qquad T_{\Delta_*}(K,b,\varepsilon,U)<\infty d 1 ( K , b , ε , U ) ≥ d ∗ , T Δ ∗ ( K , b , ε , U ) < ∞ for every sufficiently small ε > 0 \varepsilon>0 ε > 0 U U U
The entry-set formulation in Appendix § 10 gives the same selected terminal loss at every sufficiently small section. It also retains a section-independent terminal-loss test and its minimum successful depth.
A finite comparison segment and the entire tail
Before the first-radius hit, the predictor norm is small, giving a positive distance from M 1 M_1 M 1 ϕ 1 \phi_1 ϕ 1 M 1 M_1 M 1 M 1 M_1 M 1
Φ ( M ) − ϕ 1 = − ⟨ R 1 , M − M 1 ⟩ + 1 2 ⟨ M − M 1 , L ( M − M 1 ) ⟩ . \Phi(M)-\phi_1=-\langle R_1,M-M_1\rangle+\frac{1}{2}\langle M-M_1,\mathcal{L}(M-M_1)\rangle. Φ ( M ) − ϕ 1 = − ⟨ R 1 , M − M 1 ⟩ + 2 1 ⟨ M − M 1 , L ( M − M 1 )⟩ . Its nonnegative quadratic term converts the preserved loss decrease into a permanent predictor-distance bound. In particular, whenever Φ ( M ) ≤ ϕ 1 − Δ \Phi(M)\le\phi_1-\Delta Φ ( M ) ≤ ϕ 1 − Δ Δ > 0 \Delta>0 Δ > 0
Δ ≤ ⟨ R 1 , M − M 1 ⟩ ≤ ∥ R 1 ∥ F ∥ M − M 1 ∥ F . \Delta\le\langle R_1,M-M_1\rangle\le\lVert R_1\rVert_F\lVert M-M_1\rVert_F. Δ ≤ ⟨ R 1 , M − M 1 ⟩ ≤ ∥ R 1 ∥ F ∥ M − M 1 ∥ F . After the crossing, the same inequality applies at every later actual time. The small-initialization comparison is used only to reach this fixed loss level; the algebraic loss identity then supplies the tail bound without an initialization-dependent comparison interval. Combining the early interval, the fixed comparison segment, and this tail proves the all-time result. The comparison time is chosen before taking the initialization limit.
Equal loss, selected capture, and full width
A selected endpoint can have loss ϕ 1 \phi_1 ϕ 1 M 1 M_1 M 1 M 1 M_1 M 1
Corollary 11 (Selected endpoints and actual paths) The following assertions have separate hypotheses.
Under the pointwise hypotheses of Theorem 7 , if M s , K = M 1 M_{s,K}=M_1 M s , K = M 1
lim ε ↓ 0 d 1 ( K , b , ε , U ) = 0. \lim_{\varepsilon\downarrow0}d_1(K,b,\varepsilon,U)=0. ε ↓ 0 lim d 1 ( K , b , ε , U ) = 0. Under the pointwise hypotheses of Theorem 7 and Assumption 9, suppose ℓ K , d K ( b ) = ϕ 1 \ell_{K,d_K}(b)=\phi_1 ℓ K , d K ( b ) = ϕ 1 M s , K ≠ M 1 M_{s,K}\ne M_1 M s , K = M 1 d ∗ > 0 d_*>0 d ∗ > 0 d 1 ( K , b , ε , U ) ≥ d ∗ d_1(K,b,\varepsilon,U)\ge d_* d 1 ( K , b , ε , U ) ≥ d ∗ ε > 0 \varepsilon>0 ε > 0 m ≥ r m\ge r m ≥ r
Under the empirical model, m ≥ r m\ge r m ≥ r Φ ∗ : = min M ⪰ 0 Φ ( M ) \Phi_*:=\min_{M\succeq0}\Phi(M) Φ ∗ := min M ⪰ 0 Φ ( M ) Φ ∗ < ϕ 1 \Phi_*<\phi_1 Φ ∗ < ϕ 1 ε > 0 \varepsilon>0 ε > 0 K K K
lim t → ∞ Φ ( M K ε , b ( t ; U ) ) = Φ ∗ . \lim_{t\to\infty}\Phi\left(M_K^{\varepsilon,b}(t;U)\right)=\Phi_*. t → ∞ lim Φ ( M K ε , b ( t ; U ) ) = Φ ∗ . Thus the strict loss-crossing part of Definition 4 holds at every finite depth. The crossing times may depend on ε \varepsilon ε
Under the pointwise hypotheses of Theorem 7 , m ≥ r m\ge r m ≥ r ℓ K , d K ( b ) ≤ ϕ 1 \ell_{K,d_K}(b)\le\phi_1 ℓ K , d K ( b ) ≤ ϕ 1
K gives almost-sure small-initialization bypass ⟺ M s , K ≠ M 1 . K\text{ gives almost-sure small-initialization bypass}\quad\Longleftrightarrow\quad M_{s,K}\ne M_1. K gives almost-sure small-initialization bypass ⟺ M s , K = M 1 . Part (iii) requires the empirical model and full width at positive initialization. Its probability-one event is the full-column-rank event for U U U
For part (ii), choose a small factor ball around the selected terminal factor whose predictors stay separated from M 1 M_1 M 1 (8) control the length of actual trajectories inside a small loss interval around ϕ 1 \phi_1 ϕ 1 ϕ 1 \phi_1 ϕ 1
The distinction between parts (ii) and (iii) is useful for the order of limits. In part (ii), the selected endpoint and its neighborhood are fixed before ε ↓ 0 \varepsilon\downarrow0 ε ↓ 0
Residual Inertia and an Eventual Depth Guarantee
A large entrance rank can rule out a high-loss stationary endpoint. The relevant restriction uses the rank at finite time, which the flow preserves, and the positive eigenvalues of the residual at the limit. This distinction allows the selected predictor to lose rank as it converges.
Preserved rank and terminal residual inertia
Lemma 12 (Rank–inertia bound). Let B ( τ ) B(\tau) B ( τ ) I ⪯ B ( τ ) ⪯ a I I \preceq B(\tau) \preceq aI I ⪯ B ( τ ) ⪯ a I a ≥ 1 a \ge1 a ≥ 1
W ′ = 2 W R ( W ⊤ W ) B ( τ ) W^\prime= 2W R(W^\top W)B(\tau) W ′ = 2 W R ( W ⊤ W ) B ( τ ) for almost every τ \tau τ
Assume the coefficients are locally bounded and the trajectory converges to W s W_s W s rank W ( 0 ) = d \operatorname{rank} W(0) = d rank W ( 0 ) = d
d + n + ( R ( W s ⊤ W s ) ) ≤ r . (30) d + n_+\left(R(W_s^\top W_s)\right) \le r. \tag*{(30)} d + n + ( R ( W s ⊤ W s ) ) ≤ r . (30) In particular this applies to W ′ = H K , η ( G ( W ) ) W^\prime= H_{K,\eta}(G(W)) W ′ = H K , η ( G ( W )) K K K
The proof uses the actual right-multiplication equation. A positive quadratic cone associated with the terminal residual is forward invariant once the residual is sufficiently close to its limit. Uniform positivity of the multiplier forces exponential growth of every row combination in that cone. A dimension argument supplies such a combination whenever the inequality in (30) fails, contradicting convergence. The derivative estimate keeps the order of all matrix products.
For the empirical loss, a separate stationary identity gives rank M + n + ( R ( M ) ) ≤ n + ( C ) \operatorname{rank} M + n_+(R(M)) \le n_+(C) rank M + n + ( R ( M )) ≤ n + ( C ) M ⪰ 0 M \succeq0 M ⪰ 0 M R ( M ) = 0 MR(M) = 0 M R ( M ) = 0 L ( M ) \mathcal{L}(M) L ( M ) Ran M \operatorname{Ran} M Ran M
A critical stationary loss band
Under Assumption 9, define
C ≥ ϕ 1 ( q ) = { M ⪰ 0 : rank M ≤ q , M R ( M ) = 0 , ϕ 1 ≤ Φ ( M ) ≤ Φ ( 0 ) } , \mathcal{C}_{\geq\phi_1}^{(q)}=\{M \succeq0:\operatorname{rank} M\leq q,\ MR(M)=0,\ \phi_1\leq\Phi(M)\leq\Phi(0)\}, C ≥ ϕ 1 ( q ) = { M ⪰ 0 : rank M ≤ q , M R ( M ) = 0 , ϕ 1 ≤ Φ ( M ) ≤ Φ ( 0 )} , p ∗ ( q ) = min M ∈ C ≥ ϕ 1 ( q ) n + ( R ( M ) ) . (31) p_*^{(q)}=\min_{M\in\mathcal{C}_{\geq\phi_1}^{(q)}}n_+(R(M)). \tag*{(31)} p ∗ ( q ) = M ∈ C ≥ ϕ 1 ( q ) min n + ( R ( M )) . (31) This set is nonempty and compact, and 1 ≤ p ∗ ( q ) ≤ n + ( R 1 ) ≤ n + ( C ) − 1 1\leq p_*^{(q)}\leq n_+(R_1)\leq n_+(C)-1 1 ≤ p ∗ ( q ) ≤ n + ( R 1 ) ≤ n + ( C ) − 1 Lemma 53 ). Its lower bound follows because M 1 M_1 M 1
Every selected continuation starts below Φ ( 0 ) \Phi(0) Φ ( 0 ) d K ( b ) d_K(b) d K ( b ) ϕ 1 \phi_1 ϕ 1 Lemma 12 would then require d K ( b ) + p ∗ ( q ) ≤ r d_K(b)+p_*^{(q)}\leq r d K ( b ) + p ∗ ( q ) ≤ r Theorem 10 . This argument is formalized in Corollary 54 .
Capacity saturation and the depth tail
The next bound makes every clock complete before the fixed physical scale. It depends on the input spectrum, hidden width, and initialization exponent. The sharper selected loss remains a property of the full empirical continuation.
Proposition 13 (Saturation at an explicit finite depth) Under Assumptions 2 and 5, define
K c a p ( b ) = 1 + ⌈ log 3 / 2 [ 1 + 2 ( p − 1 ) max { 1 , b − 1 } ] ( ∑ i = 1 p λ i 2 ) 1 / 2 λ p ⌉ . (32) K_{\mathrm{cap}}(b)=1+\left\lceil\log_{3/2}\frac{\left[1+2(p-1)\max\{1,b-1\}\right]\left(\sum_{i=1}^{p}\lambda_i^2\right)^{1/2}}{\lambda_p}\right\rceil. \tag*{(32)} K cap ( b ) = 1 + ⌈ log 3/2 λ p [ 1 + 2 ( p − 1 ) max { 1 , b − 1 } ] ( ∑ i = 1 p λ i 2 ) 1/2 ⌉ . (32) For every fixed b > 1 b>1 b > 1 K ≥ K c a p ( b ) K\geq K_{\mathrm{cap}}(b) K ≥ K cap ( b ) p p p Theorem 7 applies with d K ( b ) = p d_K(b)=p d K ( b ) = p b , c η b,c_\eta b , c η B > 1 B>1 B > 1 B C ∩ ( 1 , B ] \mathcal{B}_{C}\cap(1,B] B C ∩ ( 1 , B ] ( p − 1 ) K c a p ( B ) (p-1)K_{\mathrm{cap}}(B) ( p − 1 ) K cap ( B )
Theorem 14 (Eventual avoidance under a residual-inertia condition) Fix a task and width satisfying Assumptions 2, 5, and 9. Suppose, in addition, that
p + p ∗ ( q ) > r , p = min { m , n + ( C ) } , q = min { m , r } . (33) p+p_*^{(q)}>r,\qquad p=\min\{m,n_+(C)\},\qquad q=\min\{m,r\}. \tag*{(33)} p + p ∗ ( q ) > r , p = min { m , n + ( C )} , q = min { m , r } . (33) For every fixed b > 1 b>1 b > 1 c η > 0 c_\eta>0 c η > 0 K ≥ K c a p ( b ) K\geq K_{\mathrm{cap}}(b) K ≥ K cap ( b )
d K ( b ) = p , ℓ K , p < ϕ 1 , d_K(b)=p,\qquad\ell_{K,p}<\phi_1, d K ( b ) = p , ℓ K , p < ϕ 1 , and gives almost-sure small-initialization bypass in Definition 4 . More precisely, for every finite K ≥ K c a p ( b ) K\geq K_{\mathrm{cap}}(b) K ≥ K cap ( b ) d ∗ , K , Δ ∗ , K > 0 d_{*,K},\Delta_{*,K}>0 d ∗ , K , Δ ∗ , K > 0
such that, for every U U U K K K ε > 0 \varepsilon> 0 ε > 0 K K K U U U
d 1 ( K , b , ε , U ) ≥ d ∗ , K , T Δ ∗ , K ( K , b , ε , U ) < ∞ . (34) d_1(K,b,\varepsilon,U) \ge d_{\ast,K}, \qquad T_{\Delta_{\ast,K}}(K,b,\varepsilon,U) < \infty. \tag*{(34)} d 1 ( K , b , ε , U ) ≥ d ∗ , K , T Δ ∗ , K ( K , b , ε , U ) < ∞. (34) Here K c a p ( b ) K_{\mathrm{cap}}(b) K cap ( b )
For each finite depth in the asserted tail, the proof first selects its rank-p p p
Numerical Experiments
We compare gradient descent, finite-depth Newton–Schulz dynamics, and polar updates on the same empirical quadratic task. The experiment follows the approach to the first gradient-flow stationary point and measures the closest point reached along each training trajectory.
We use m = r = 3 m=r=3 m = r = 3
x l = 3 1 / 4 ( ± 1 , ± 1 , ± 3 / 20 ) , y l = 3 3 , x_l=3^{1/4}(\pm1,\pm1,\pm\sqrt{3}/20), \qquad y_l=3\sqrt{3}, x l = 3 1/4 ( ± 1 , ± 1 , ± 3 /20 ) , y l = 3 3 , with all independent sign choices. The remaining samples consist of four copies of each signed pair
x l = ( 0 , ± 3 , 0 ) , y l = − 1 , x_l=(0,\pm\sqrt{3},0), \qquad y_l=-1, x l = ( 0 , ± 3 , 0 ) , y l = − 1 , x l = ( 0 , 0 , ± ( 1173 / 400 ) 1 / 4 ) , y l = 11 3 / 391 . x_l=(0,0,\pm(1173/400)^{1/4}), \qquad y_l=11\sqrt{3/391}. x l = ( 0 , 0 , ± ( 1173/400 ) 1/4 ) , y l = 11 3/391 . For these data, C = diag ( 3 , 2 , 1 ) C=\operatorname{diag}(3,2,1) C = diag ( 3 , 2 , 1 ) Proposition 3 is M 1 = diag ( 3 , 0 , 0 ) M_1=\operatorname{diag}(3,0,0) M 1 = diag ( 3 , 0 , 0 ) Φ ( M 1 ) = 377 / 1173 \Phi(M_1)=377/1173 Φ ( M 1 ) = 377/1173 1 / 6 1/6 1/6
We initialize with b = 5 / 4 b=5/4 b = 5/4 c η = 1 c_\eta=1 c η = 1 ε ∈ { 10 − 3 , 10 − 5 , … , 10 − 15 } \varepsilon\in\{10^{-3},10^{-5},\ldots,10^{-15}\} ε ∈ { 1 0 − 3 , 1 0 − 5 , … , 1 0 − 15 } K = 1 , … , 6 K=1,\ldots,6 K = 1 , … , 6 U U U Proposition 1 , so the left panel uses the normalized clock of (6). The polar branch uses the polar factor of the full factor gradient at each numerical step. All methods update the complete 3 × 3 3\times3 3 × 3
Theorem 7 gives entrance rank one at K = 1 K=1 K = 1 K = 2 , 3 K=2,3 K = 2 , 3 K ≥ 4 K\ge4 K ≥ 4 span ( e 1 , e 2 ) \operatorname{span}(e_1,e_2) span ( e 1 , e 2 ) M 1 M_1 M 1 M 1 M_1 M 1 Figure 1 follows this classification: the distance decreases with initialization scale for GD and depths one to three, while depths four to six remain separated from M 1 M_1 M 1 Φ ( M 1 ) \Phi(M_1) Φ ( M 1 ) Corollary 11 for GD and finite depth.
Figure 1. Finite-initialization trajectories on the 24-sample task. (a) Loss at ε = 10 − 9 \varepsilon= 10^{-9} ε = 1 0 − 9 M 1 M_1 M 1
The GD and finite-depth equations are integrated with DOP853 in continuation time while accumulating the normalized time. Polar is integrated by explicit SVD updates with step-size control and backtracking. Integration stops within 10 − 11 10^{-11} 1 0 − 11 10 − 9 10^{-9} 1 0 − 9
Conclusion
In this paper, we have studied how finitely many iterations of a nonlinear matrix transformation can reshape optimization dynamics. In a quadratic two-layer network, we have characterized the evolution from small-initialization escape near the origin to the first nonzero stationary stage. Our results provide a precise description of how the number of Newton–Schulz iterations controls early rank selection and regulates the subsequent optimization path. Beyond this particular model, the analysis offers a framework combining spectral dynamics with small-initialization techniques, which may help understand certain training phenomena arising in large-scale neural networks, including language models.
An immediate open question is whether similar dynamical descriptions can be established for more general models and under weaker assumptions. Of particular interest are networks with nonlinear activations such as ReLU, high-dimensional training regimes, and deeper architectures, where the interaction between spectral updates and feature learning may lead to richer dynamics.
Foundations and Shared Structural Tools
The empirical identities provide compact sublevels and positive multipliers. These yield continuation, finite-time comparison, and rank preservation. The last subsection proves the rank–inertia bound used both for full-width optimality and for the sufficient depth theorem.
Empirical identities, update bounds, and changes of time
The next three statements give the algebraic and analytic properties of the update before any spectral-prefix or comparison assumption is used.
Lemma 15 (Empirical loss identities). The empirical loss satisfies
Φ ( M ) = Φ ( 0 ) − ⟨ C , M ⟩ + 1 2 ⟨ M , L ( M ) ⟩ , ⟨ H , L ( H ) ⟩ = 1 n ∑ l ( x l T H x l ) 2 ≥ 0. (35) \Phi(M) = \Phi(0) - \langle C, M\rangle+ \frac{1}{2}\langle M,\mathcal{L}(M)\rangle,\qquad\langle H,\mathcal{L}(H)\rangle= \frac{1}{n}\sum_l (x_l^{\mathsf T} Hx_l)^2 \ge0. \tag*{(35)} Φ ( M ) = Φ ( 0 ) − ⟨ C , M ⟩ + 2 1 ⟨ M , L ( M )⟩ , ⟨ H , L ( H )⟩ = n 1 l ∑ ( x l T H x l ) 2 ≥ 0. (35) Moreover, Φ \Phi Φ L \mathcal{L} L L ( M ) ⪰ 0 \mathcal{L}(M) \succeq0 L ( M ) ⪰ 0 M ⪰ 0 M \succeq0 M ⪰ 0 ∇ Φ ( M ) = − R ( M ) \nabla\Phi(M) = -R(M) ∇Φ ( M ) = − R ( M ) G ( W ) = 2 W R ( W T W ) G(W) = 2W R(W^{\mathsf T}W) G ( W ) = 2 W R ( W T W )
Proof (of Lemma 15). Expand each summand as y l 2 − 2 y l x l T M x l + ( x l T M x l ) 2 y_l^2 - 2y_lx_l^{\mathsf T}Mx_l + (x_l^{\mathsf T}Mx_l)^2 y l 2 − 2 y l x l T M x l + ( x l T M x l ) 2 x l T M x l = ⟨ x l x l T , M ⟩ x_l^{\mathsf T}Mx_l = \langle x_lx_l^{\mathsf T}, M\rangle x l T M x l = ⟨ x l x l T , M ⟩ − ⟨ C , M ⟩ -\langle C, M\rangle − ⟨ C , M ⟩ ⟨ M , L ( M ) ⟩ / 2 \langle M,\mathcal{L}(M)\rangle/2 ⟨ M , L ( M )⟩ /2 A , B A,B A , B
⟨ A , L ( B ) ⟩ = 1 n ∑ l ( x l T A x l ) ( x l T B x l ) = ⟨ L ( A ) , B ⟩ . \langle A,\mathcal{L}(B)\rangle= \frac{1}{n}\sum_l (x_l^{\mathsf T}Ax_l)(x_l^{\mathsf T}Bx_l) = \langle\mathcal{L}(A), B\rangle. ⟨ A , L ( B )⟩ = n 1 l ∑ ( x l T A x l ) ( x l T B x l ) = ⟨ L ( A ) , B ⟩ . This proves self-adjointness and the second identity. If M ⪰ 0 M \succeq0 M ⪰ 0 x l x l T x_lx_l^{\mathsf T} x l x l T L ( M ) \mathcal{L}(M) L ( M ) ∇ Φ ( M ) = L ( M ) − C \nabla\Phi(M) = \mathcal{L}(M)-C ∇Φ ( M ) = L ( M ) − C E E E
D f ( W ) [ E ] = ⟨ − R ( W T W ) , E T W + W T E ⟩ = ⟨ − 2 W R ( W T W ) , E ⟩ , Df(W)[E] = \langle-R(W^{\mathsf T}W), E^{\mathsf T}W + W^{\mathsf T}E\rangle= \langle-2W R(W^{\mathsf T}W), E\rangle, D f ( W ) [ E ] = ⟨ − R ( W T W ) , E T W + W T E ⟩ = ⟨ − 2 W R ( W T W ) , E ⟩ , which gives the stated negative gradient. In particular Φ \Phi Φ
Lemma 16 (Bounds and continuity of the update map). Fix finite K K K [ 0 , 1 ] [0,1] [ 0 , 1 ]
s ≤ p K ( s ) ≤ a K s , 0 ≤ p K ( s ) ≤ 1 , 1 ≤ q K ( x ) ≤ a K . s \le p_K(s) \le a_Ks,\qquad0 \le p_K(s) \le1,\qquad1 \le q_K(x) \le a_K. s ≤ p K ( s ) ≤ a K s , 0 ≤ p K ( s ) ≤ 1 , 1 ≤ q K ( x ) ≤ a K . Moreover p K ′ ( s ) > 0 p_K'(s) > 0 p K ′ ( s ) > 0 0 < s < 1 0 < s < 1 0 < s < 1 K ≥ 1 K \ge1 K ≥ 1 q K ′ ( x ) < 0 q_K'(x) < 0 q K ′ ( x ) < 0 0 < x < 1 0 < x < 1 0 < x < 1 q K q_K q K [ 0 , 1 ] [0,1] [ 0 , 1 ] [ 1 , a K ] [1,a_K] [ 1 , a K ] X X X
P K ( X ) = X q K ( X T X ) = q K ( X X T ) X . P_K(X) = Xq_K(X^{\mathsf T}X) = q_K(XX^{\mathsf T})X. P K ( X ) = X q K ( X T X ) = q K ( X X T ) X . The maps H K , η H_{K,\eta} H K , η η ≥ 0 \eta\ge0 η ≥ 0 G G G η \eta η
H K , η ( G ) = G B R , I ⪯ B R ⪯ a K I , (36) H_{K,\eta}(G) = GB^R,\qquad I \preceq B^R \preceq a_KI, \tag*{(36)} H K , η ( G ) = G B R , I ⪯ B R ⪯ a K I , (36) ⟨ G , H K , η ( G ) ⟩ ≥ ∥ G ∥ F 2 , ∥ H K , η ( G ) ∥ F ≤ a K ∥ G ∥ F , (37) \langle G,H_{K,\eta}(G)\rangle\ge\lVert G\rVert_F^2,\qquad\lVert H_{K,\eta}(G)\rVert_F \le a_K\lVert G\rVert_F, \tag*{(37)} ⟨ G , H K , η ( G )⟩ ≥ ∥ G ∥ F 2 , ∥ H K , η ( G ) ∥ F ≤ a K ∥ G ∥ F , (37) sup G ∥ H K , η ( G ) − H K , 0 ( G ) ∥ F ≤ C K η . (38) \sup_G\lVert H_{K,\eta}(G)-H_{K,0}(G)\rVert_F \le C_K\eta. \tag*{(38)} G sup ∥ H K , η ( G ) − H K , 0 ( G ) ∥ F ≤ C K η . (38) Here the identity matrix has the appropriate size, and
B R = q K ( G T G ( ∥ G ∥ F + η ) 2 ) ∈ S r . B^{R}=q_{K}\left(\frac{G^{T}G}{(\lVert G\rVert_{F}+\eta)^{2}}\right)\in\mathbb{S}^{r}. B R = q K ( (∥ G ∥ F + η ) 2 G T G ) ∈ S r . The constant C K < ∞ C_{K}<\infty C K < ∞ G = 0 , η = 0 G=0,\eta=0 G = 0 , η = 0 B R = a K I r B^{R}=a_{K}I_{r} B R = a K I r
Proof (of Lemma 16). The polynomial identity in the statement follows by expanding p K p_{K} p K
For 0 ≤ s ≤ 1 0\leq s\leq1 0 ≤ s ≤ 1 s ↦ ( 3 s − s 3 ) / 2 s\mapsto(3s-s^{3})/2 s ↦ ( 3 s − s 3 ) /2 [ 0 , 1 ] [0,1] [ 0 , 1 ] s s s 3 s / 2 3s/2 3 s /2
s ≤ p K ( s ) ≤ a K s , 0 ≤ p K ( s ) ≤ 1. s\leq p_{K}(s)\leq a_{K}s,\qquad0\leq p_{K}(s)\leq1. s ≤ p K ( s ) ≤ a K s , 0 ≤ p K ( s ) ≤ 1. Thus 1 ≤ q K ( x ) ≤ a K 1\leq q_{K}(x)\leq a_{K} 1 ≤ q K ( x ) ≤ a K [ 0 , 1 ] [0,1] [ 0 , 1 ] ( 0 , 1 ) (0,1) ( 0 , 1 ) p K ′ ( s ) > 0 p_{K}'(s)>0 p K ′ ( s ) > 0 q 1 ( x ) = 3 / 2 − x / 2 q_{1}(x)=3/2-x/2 q 1 ( x ) = 3/2 − x /2
q K + 1 ( x ) = q K ( x ) { 3 2 − 1 2 p K ( x 2 ) } q_{K+1}(x)=q_{K}(x)\left\{\frac{3}{2}-\frac{1}{2}p_{K}(\sqrt{x}^{\,2})\right\} q K + 1 ( x ) = q K ( x ) { 2 3 − 2 1 p K ( x 2 ) } has a positive second factor; differentiating it shows that both summands in its derivative are negative whenever q K ′ ( x ) < 0 q_{K}'(x)<0 q K ′ ( x ) < 0 0 < x < 1 0<x<1 0 < x < 1 q K ( 0 ) = a K q_{K}(0)=a_{K} q K ( 0 ) = a K q K ( 1 ) = p K ( 1 ) = 1 q_{K}(1)=p_{K}(1)=1 q K ( 1 ) = p K ( 1 ) = 1
The normalized singular values of G G G
⟨ G , H K , η ( G ) ⟩ = tr ( G T G B R ) ≥ tr ( G T G ) , ∥ G B R ∥ F ≤ a K ∥ G ∥ F , \langle G,H_{K,\eta}(G)\rangle=\operatorname{tr}(G^{T}GB^{R})\geq\operatorname{tr}(G^{T}G),\qquad\lVert GB^{R}\rVert_{F}\leq a_{K}\lVert G\rVert_{F}, ⟨ G , H K , η ( G )⟩ = tr ( G T G B R ) ≥ tr ( G T G ) , ∥ G B R ∥ F ≤ a K ∥ G ∥ F , so (37) follows.
For the Lipschitz bound, P K P_{K} P K G ≠ 0 G\neq0 G = 0
D H K , η ( G ) [ E ] = D P K ( G ∥ G ∥ F + η ) [ E ] + ⟨ G , E ⟩ ∥ G ∥ F { P K ( G ∥ G ∥ F + η ) − D P K ( G ∥ G ∥ F + η ) [ G ∥ G ∥ F + η ] } . (39) \begin{aligned}
DH_{K,\eta}(G)[E]
={}&DP_{K}\left(\frac{G}{\lVert G\rVert_{F}+\eta}\right)[E]\\
&+\frac{\langle G,E\rangle}{\lVert G\rVert_{F}}\left\{P_{K}\left(\frac{G}{\lVert G\rVert_{F}+\eta}\right)-DP_{K}\left(\frac{G}{\lVert G\rVert_{F}+\eta}\right)\left[\frac{G}{\lVert G\rVert_{F}+\eta}\right]\right\}. \tag*{(39)}
\end{aligned} D H K , η ( G ) [ E ] = D P K ( ∥ G ∥ F + η G ) [ E ] + ∥ G ∥ F ⟨ G , E ⟩ { P K ( ∥ G ∥ F + η G ) − D P K ( ∥ G ∥ F + η G ) [ ∥ G ∥ F + η G ] } . (39) The normalized argument has norm at most one, and ∣ ⟨ G , E ⟩ ∣ / ∥ G ∥ F ≤ ∥ E ∥ F \lvert\langle G,E\rangle\rvert/\lVert G\rVert_{F}\leq\lVert E\rVert_{F} ∣⟨ G , E ⟩∣ / ∥ G ∥ F ≤ ∥ E ∥ F C K ∥ E ∥ F C_{K}\lVert E\rVert_{F} C K ∥ E ∥ F η \eta η H K , η H_{K,\eta} H K , η
For G ≠ 0 G\neq0 G = 0
H K , η ( G ) − H K , 0 ( G ) = η P K ( G ∥ G ∥ F + η ) + ∥ G ∥ F { P K ( G ∥ G ∥ F + η ) − P K ( G ∥ G ∥ F ) } , \begin{aligned}
H_{K,\eta}(G)-H_{K,0}(G)
={}&\eta P_{K}\left(\frac{G}{\lVert G\rVert_{F}+\eta}\right)\\
&+\lVert G\rVert_{F}\left\{P_{K}\left(\frac{G}{\lVert G\rVert_{F}+\eta}\right)-P_{K}\left(\frac{G}{\lVert G\rVert_{F}}\right)\right\},
\end{aligned} H K , η ( G ) − H K , 0 ( G ) = η P K ( ∥ G ∥ F + η G ) + ∥ G ∥ F { P K ( ∥ G ∥ F + η G ) − P K ( ∥ G ∥ F G ) } , ∥ G ∥ G ∥ F + η − G ∥ G ∥ F ∥ F = η ∥ G ∥ F + η . \left\lVert\frac{G}{\lVert G\rVert_{F}+\eta}-\frac{G}{\lVert G\rVert_{F}}\right\rVert_{F}=\frac{\eta}{\lVert G\rVert_{F}+\eta}. ∥ G ∥ F + η G − ∥ G ∥ F G F = ∥ G ∥ F + η η . The first term is O K ( η ) O_K(\eta) O K ( η ) C K ∥ G ∥ F η / ( ∥ G ∥ F + η ) ≤ C K η C_K\|G\|_F\eta/(\|G\|_F+\eta)\le C_K\eta C K ∥ G ∥ F η / ( ∥ G ∥ F + η ) ≤ C K η G = 0 G=0 G = 0 (38) .
Lemma 17 (Positive change of time). Fix η > 0 \eta>0 η > 0 W ( t ) W(t) W ( t ) physical-time equation with denominator ∥ G ( W ) ∥ F + η \|G(W)\|_F+\eta ∥ G ( W ) ∥ F + η
τ ( t ) = ∫ 0 t d s ∥ G ( W ( s ) ) ∥ F + η . \tau(t)=\int_0^t\frac{\mathrm{d}s}{\|G(W(s))\|_F+\eta}. τ ( t ) = ∫ 0 t ∥ G ( W ( s )) ∥ F + η d s . Then τ ( t ) \tau(t) τ ( t )
d t d τ = ∥ G ( W ( t ( τ ) ) ) ∥ F + η , d d τ W ( t ( τ ) ) = H K , η ( G ( W ( t ( τ ) ) ) ) . (40) \frac{\mathrm{d}t}{\mathrm{d}\tau}=\|G(W(t(\tau)))\|_F+\eta,\qquad
\frac{\mathrm{d}}{\mathrm{d}\tau}W(t(\tau))=H_{K,\eta}(G(W(t(\tau)))). \tag*{(40)} d τ d t = ∥ G ( W ( t ( τ ))) ∥ F + η , d τ d W ( t ( τ )) = H K , η ( G ( W ( t ( τ )))) . (40) On bounded forward solutions this change of time maps the whole forward half-line onto itself and preserves finite hitting times and the infimum of distance over the path. At K = 0 K=0 K = 0 H 0 , η ( G ) = G H_{0,\eta}(G)=G H 0 , η ( G ) = G η ≥ 0 \eta\ge0 η ≥ 0
Proof (of Lemma 17). The integrand is positive and continuous, so τ ( t ) \tau(t) τ ( t )
d W ( t ( τ ) ) d τ = d W d t d t d τ = ( ∥ G ( W ( t ( τ ) ) ) ∥ F + η ) P K ( G ( W ( t ( τ ) ) ) ∥ G ( W ( t ( τ ) ) ) ∥ F + η ) . \frac{\mathrm{d}W(t(\tau))}{\mathrm{d}\tau}
=\frac{\mathrm{d}W}{\mathrm{d}t}\frac{\mathrm{d}t}{\mathrm{d}\tau}
=\left(\|G(W(t(\tau)))\|_F+\eta\right)P_K\left(\frac{G(W(t(\tau)))}{\|G(W(t(\tau)))\|_F+\eta}\right). d τ d W ( t ( τ )) = d t d W d τ d t = ( ∥ G ( W ( t ( τ ))) ∥ F + η ) P K ( ∥ G ( W ( t ( τ ))) ∥ F + η G ( W ( t ( τ ))) ) . This proves (40). On a bounded trajectory G ( W ) G(W) G ( W ) η ≤ d t / d τ ≤ sup τ ∥ G ( W ( τ ) ) ∥ F + η \eta\le\mathrm{d}t/\mathrm{d}\tau\le\sup_\tau\|G(W(\tau))\|_F+\eta η ≤ d t / d τ ≤ sup τ ∥ G ( W ( τ )) ∥ F + η P 0 ( X ) = X P_0(X)=X P 0 ( X ) = X H 0 , η ( G ) = G H_{0,\eta}(G)=G H 0 , η ( G ) = G
The two rescalings introduced in § 3 retain the empirical operator and denominator at their respective scales. The error bound below is local in a region with a positive limiting denominator.
Lemma 18 (Exact rescaling and local error). The initial-layer and shape equations are
d Y ε d σ = P K ( 2 Y ε [ C − ε 2 L ( ( Y ε ) ⊤ Y ε ) ] 2 ∥ Y ε [ C − ε 2 L ( ( Y ε ) ⊤ Y ε ) ] ∥ F + c η ) , Y ε ( 0 ) = diag ( 1 , ε b − 1 , … , ε b − 1 ) U , (41) \frac{\mathrm{d}Y^\varepsilon}{\mathrm{d}\sigma}
=P_K\left(\frac{2Y^\varepsilon[C-\varepsilon^2\mathcal{L}((Y^\varepsilon)^\top Y^\varepsilon)]}{2\|Y^\varepsilon[C-\varepsilon^2\mathcal{L}((Y^\varepsilon)^\top Y^\varepsilon)]\|_F+c\eta}\right),
\qquad
Y^\varepsilon(0)=\operatorname{diag}(1,\varepsilon^{b-1},\ldots,\varepsilon^{b-1})U, \tag*{(41)} d σ d Y ε = P K ( 2∥ Y ε [ C − ε 2 L (( Y ε ) ⊤ Y ε )] ∥ F + cη 2 Y ε [ C − ε 2 L (( Y ε ) ⊤ Y ε )] ) , Y ε ( 0 ) = diag ( 1 , ε b − 1 , … , ε b − 1 ) U , (41) and
d Q ε d ν = P K ( Q ε [ C − t 2 L ( ( Q ε ) ⊤ Q ε ) ] ∥ Q ε [ C − t 2 L ( ( Q ε ) ⊤ Q ε ) ] ∥ F + c η ε / ( 2 t ) ) − Q ε , t = e ν . (42) \frac{\mathrm{d}Q^\varepsilon}{\mathrm{d}\nu}
=P_K\left(\frac{Q^\varepsilon[C-t^2\mathcal{L}((Q^\varepsilon)^\top Q^\varepsilon)]}{\|Q^\varepsilon[C-t^2\mathcal{L}((Q^\varepsilon)^\top Q^\varepsilon)]\|_F+c\eta\varepsilon/(2t)}\right)-Q^\varepsilon,
\qquad t=e^\nu. \tag*{(42)} d ν d Q ε = P K ( ∥ Q ε [ C − t 2 L (( Q ε ) ⊤ Q ε )] ∥ F + cη ε / ( 2 t ) Q ε [ C − t 2 L (( Q ε ) ⊤ Q ε )] ) − Q ε , t = e ν . (42) On a fixed bounded Q Q Q inf Q ∥ Q C ∥ F > 0 \inf_Q\|QC\|_F>0 inf Q ∥ QC ∥ F > 0
∥ right side of (42) − F K ∥ C 1 = O ( t 2 + ε / t ) (43) \|\text{right side of (42)}-F_K\|_{C^1}=O(t^2+\varepsilon/t) \tag*{(43)} ∥ right side of (42) − F K ∥ C 1 = O ( t 2 + ε / t ) (43) for all sufficiently small t > 0 t>0 t > 0 0 ≤ ε / t ≤ 1 0\le\varepsilon/t\le1 0 ≤ ε / t ≤ 1 t t t
Proof (of Lemma 18). The chain rule gives
d Q d v = t d d t ( W / t ) = W ˙ − Q . \frac{\mathrm{d}Q}{\mathrm{d}v}=t\frac{\mathrm{d}}{\mathrm{d}t}(W/t)=\dot{W}-Q. d v d Q = t d t d ( W / t ) = W ˙ − Q . The exact gradient substitution is
G ( t Q ) = 2 t Q { C − t 2 L ( Q T Q ) } , G ( t Q ) ∥ G ( t Q ) ∥ F + c η ε = Q [ C − t 2 L ( Q T Q ) ] ∥ Q [ C − t 2 L ( Q T Q ) ] ∥ F + c η ε / ( 2 t ) . G(tQ)=2tQ\{C-t^{2}\mathcal{L}(Q^{\mathsf{T}}Q)\},\qquad
\frac{G(tQ)}{\lVert G(tQ)\rVert_{F}+c_{\eta}\varepsilon}
=
\frac{Q[C-t^{2}\mathcal{L}(Q^{\mathsf{T}}Q)]}{\lVert Q[C-t^{2}\mathcal{L}(Q^{\mathsf{T}}Q)]\rVert_{F}+c_{\eta}\varepsilon/(2t)}. G ( tQ ) = 2 tQ { C − t 2 L ( Q T Q )} , ∥ G ( tQ ) ∥ F + c η ε G ( tQ ) = ∥ Q [ C − t 2 L ( Q T Q )] ∥ F + c η ε / ( 2 t ) Q [ C − t 2 L ( Q T Q )] . Together with Q ′ = W ˙ − Q Q'=\dot{W}-Q Q ′ = W ˙ − Q (42) .
For the estimate, choose κ > 0 \kappa>0 κ > 0 ∥ Q C ∥ F ≥ κ \lVert QC\rVert_{F}\ge\kappa ∥ QC ∥ F ≥ κ L \mathcal{L} L ∥ Q ∥ F ≤ B \lVert Q\rVert_{F}\le B ∥ Q ∥ F ≤ B
∥ Q L ( Q T Q ) ∥ F ≤ ∥ L ∥ B 3 , D [ Q L ( Q T Q ) ] [ E ] = E L ( Q T Q ) + Q L ( E T Q + Q T E ) . \lVert Q\mathcal{L}(Q^{\mathsf{T}}Q)\rVert_{F}\le\lVert\mathcal{L}\rVert B^{3},\qquad
D[Q\mathcal{L}(Q^{\mathsf{T}}Q)][E]=E\mathcal{L}(Q^{\mathsf{T}}Q)+Q\mathcal{L}(E^{\mathsf{T}}Q+Q^{\mathsf{T}}E). ∥ Q L ( Q T Q ) ∥ F ≤ ∥ L ∥ B 3 , D [ Q L ( Q T Q )] [ E ] = E L ( Q T Q ) + Q L ( E T Q + Q T E ) . The derivative has norm at most 3 ∥ L ∥ B 2 ∥ E ∥ F 3\lVert\mathcal{L}\rVert B^{2}\lVert E\rVert_{F} 3 ∥ L ∥ B 2 ∥ E ∥ F Q [ C − t 2 L ( Q T Q ) ] − Q C Q[C-t^{2}\mathcal{L}(Q^{\mathsf{T}}Q)]-QC Q [ C − t 2 L ( Q T Q )] − QC O ( t 2 ) O(t^{2}) O ( t 2 ) t t t ∥ Q [ C − t 2 L ( Q T Q ) ] ∥ F ≥ κ / 2 \lVert Q[C-t^{2}\mathcal{L}(Q^{\mathsf{T}}Q)]\rVert_{F}\ge\kappa/2 ∥ Q [ C − t 2 L ( Q T Q )] ∥ F ≥ κ /2
On matrices A A A ∥ A ∥ F ≥ κ / 2 \lVert A\rVert_{F}\ge\kappa/2 ∥ A ∥ F ≥ κ /2 0 ≤ z ≤ c η / 2 0\le z\le c_{\eta}/2 0 ≤ z ≤ c η /2 A / ( ∥ A ∥ F + z ) A/(\lVert A\rVert_{F}+z) A / (∥ A ∥ F + z ) ( A , z ) (A,z) ( A , z ) E E E
E ∥ A ∥ F + z − A ⟨ A , E ⟩ ∥ A ∥ F ( ∥ A ∥ F + z ) 2 . \frac{E}{\lVert A\rVert_{F}+z}-\frac{A\langle A,E\rangle}{\lVert A\rVert_{F}(\lVert A\rVert_{F}+z)^{2}}. ∥ A ∥ F + z E − ∥ A ∥ F (∥ A ∥ F + z ) 2 A ⟨ A , E ⟩ . The denominator lower bound controls this expression and its next derivatives. Applying the mean value estimate to A = Q [ C − t 2 L ( Q T Q ) ] A=Q[C-t^{2}\mathcal{L}(Q^{\mathsf{T}}Q)] A = Q [ C − t 2 L ( Q T Q )] z = c η ε / ( 2 t ) z=c_{\eta}\varepsilon/(2t) z = c η ε / ( 2 t ) A = Q C A=QC A = QC z = 0 z=0 z = 0 C 1 C^{1} C 1 t 2 + ε / t t^{2}+\varepsilon/t t 2 + ε / t P K P_{K} P K (43) . The same argument applies to every fixed bounded range of ε / t \varepsilon/t ε / t
For the initial scale, the chain rule gives
d Y ε d σ = 1 ε d W ( ε σ ) d σ = W ˙ ( ε σ ) . \frac{\mathrm{d}Y^{\varepsilon}}{\mathrm{d}\sigma}=\frac{1}{\varepsilon}\frac{\mathrm{d}W(\varepsilon\sigma)}{\mathrm{d}\sigma}=\dot{W}(\varepsilon\sigma). d σ d Y ε = ε 1 d σ d W ( ε σ ) = W ˙ ( ε σ ) . Substituting W = ε Y W=\varepsilon Y W = ε Y G ( ε Y ) = 2 ε Y [ C − ε 2 L ( Y T Y ) ] G(\varepsilon Y)=2\varepsilon Y[C-\varepsilon^{2}\mathcal{L}(Y^{\mathsf{T}}Y)] G ( ε Y ) = 2 ε Y [ C − ε 2 L ( Y T Y )] ε \varepsilon ε ε − 1 D ε , b = diag ( 1 , ε b − 1 , … , ε b − 1 ) \varepsilon^{-1}D_{\varepsilon,b}=\operatorname{diag}(1,\varepsilon^{b-1},\ldots,\varepsilon^{b-1}) ε − 1 D ε , b = diag ( 1 , ε b − 1 , … , ε b − 1 )
Global continuation and finite-time comparison
Compact empirical sublevels bound the factors. The update inequalities then supply both a descent angle and a uniform finite-interval perturbation estimate in the regularization parameter.
Proof (of Proposition 1). The spanning assumption gives n − 1 ∑ l x l x l T ≻ 0 n^{-1}\sum_l x_lx_l^{\mathsf{T}}\succ0 n − 1 ∑ l x l x l T ≻ 0 M ⪰ 0 M\succeq0 M ⪰ 0 x l T M x l x_l^{\mathsf{T}}Mx_l x l T M x l
λ min ( 1 n ∑ l x l x l T ) tr M ≤ 1 n ∑ l x l T M x l ≤ ( 1 n ∑ l y l 2 ) 1 / 2 + 2 Φ ( M ) . (44) \lambda_{\min}\left(\frac{1}{n}\sum_l x_lx_l^{\mathsf{T}}\right)\operatorname{tr}M
\le\frac{1}{n}\sum_l x_l^{\mathsf{T}}Mx_l
\le\left(\frac{1}{n}\sum_l y_l^{2}\right)^{1/2}+\sqrt{2\Phi(M)}. \tag*{(44)} λ m i n ( n 1 l ∑ x l x l T ) tr M ≤ n 1 l ∑ x l T M x l ≤ ( n 1 l ∑ y l 2 ) 1/2 + 2Φ ( M ) . (44) Since ∥ M ∥ F ≤ tr M \lVert M\rVert_F \le\operatorname{tr} M ∥ M ∥ F ≤ tr M ∥ W ∥ F 2 = tr ( W T W ) \lVert W\rVert_F^2 = \operatorname{tr}(W^{\mathsf{T}}W) ∥ W ∥ F 2 = tr ( W T W ) G = − ∇ f G=-\nabla f G = − ∇ f
− f ′ = ⟨ G , H K , η ( G ) ⟩ ≥ ∥ G ∥ F 2 , ∥ W ′ ∥ F ≤ a K ∥ G ∥ F . -f'=\langle G,H_{K,\eta}(G)\rangle\ge\lVert G\rVert_F^2,\qquad\lVert W'\rVert_F\le a_K\lVert G\rVert_F. − f ′ = ⟨ G , H K , η ( G )⟩ ≥ ∥ G ∥ F 2 , ∥ W ′ ∥ F ≤ a K ∥ G ∥ F . Their combination proves all three inequalities in (8). The vector field is locally Lipschitz by Lemma 16. Each solution remains in its compact initial sublevel, so the continuation theorem of Teschl [31 ] gives global existence.
For any bounded C 1 C^1 C 1
d d τ f ( W ( τ ) ) ≤ − a K − 1 ∥ ∇ f ( W ( τ ) ) ∥ F ∥ W ′ ( τ ) ∥ F . \frac{\mathrm{d}}{\mathrm{d}\tau}f(W(\tau))\le-a_K^{-1}\lVert\nabla f(W(\tau))\rVert_F\lVert W'(\tau)\rVert_F. d τ d f ( W ( τ )) ≤ − a K − 1 ∥ ∇ f ( W ( τ )) ∥ F ∥ W ′ ( τ ) ∥ F . A zero loss derivative forces the gradient and then the velocity to vanish. The angle and weak-decrease hypotheses of Absil et al. [1 ] therefore hold. Analyticity on a neighborhood of the compact closure and boundedness give convergence. The limit is stationary: otherwise continuity bounds the gradient norm away from zero on the tail, and − f ′ ≥ ∥ ∇ f ∥ F 2 -f'\ge\lVert\nabla f\rVert_F^2 − f ′ ≥ ∥ ∇ f ∥ F 2
For a nonstationary solution, local uniqueness implies G ( W ( τ ) ) ≠ 0 G(W(\tau))\ne0 G ( W ( τ )) = 0
W ′ = 2 W R ( W T W ) B R W'=2WR(W^{\mathsf{T}}W)B^R W ′ = 2 W R ( W T W ) B R is continuous and bounded on finite intervals, including when η = 0 \eta=0 η = 0 W W W
For η > 0 \eta>0 η > 0 d t / d τ \mathrm{d}t/\mathrm{d}\tau d t / d τ η \eta η K = 0 K=0 K = 0
Corollary 19 (Finite-time comparison). Let W η W_\eta W η W ‾ \overline{W} W
W η ′ = H K , η ( G ( W η ) ) , W ‾ ′ = H K , 0 ( G ( W ‾ ) ) . W_\eta'=H_{K,\eta}(G(W_\eta)),\qquad\overline{W}'=H_{K,0}(G(\overline{W})). W η ′ = H K , η ( G ( W η )) , W ′ = H K , 0 ( G ( W )) . Fix finite K K K η ≥ 0 \eta\ge0 η ≥ 0 0 ≤ τ ≤ S 0\le\tau\le S 0 ≤ τ ≤ S S < ∞ S<\infty S < ∞ L L L
sup 0 ≤ τ ≤ S ∥ W η ( τ ) − W ‾ ( τ ) ∥ F ≤ e L S ( ∥ W η ( 0 ) − W ‾ ( 0 ) ∥ F + C K η S ) . (45) \sup_{0\le\tau\le S}\lVert W_\eta(\tau)-\overline{W}(\tau)\rVert_F\le e^{LS}\left(\lVert W_\eta(0)-\overline{W}(0)\rVert_F+C_K\eta S\right). \tag*{(45)} 0 ≤ τ ≤ S sup ∥ W η ( τ ) − W ( τ ) ∥ F ≤ e L S ( ∥ W η ( 0 ) − W ( 0 ) ∥ F + C K η S ) . (45) Proof (of Corollary 19). The polynomial map G G G L L L η \eta η H K , η ( G ( W ‾ ) ) H_{K,\eta}(G(\overline{W})) H K , η ( G ( W ))
∥ W η ( τ ) − W ‾ ( τ ) ∥ F ≤ ∥ W η ( 0 ) − W ‾ ( 0 ) ∥ F + ∫ 0 τ ( L ∥ W η ( u ) − W ‾ ( u ) ∥ F + C K η ) d u . \lVert W_\eta(\tau)-\overline{W}(\tau)\rVert_F\le\lVert W_\eta(0)-\overline{W}(0)\rVert_F+\int_0^\tau\left(L\lVert W_\eta(u)-\overline{W}(u)\rVert_F+C_K\eta\right)\,\mathrm{d}u. ∥ W η ( τ ) − W ( τ ) ∥ F ≤ ∥ W η ( 0 ) − W ( 0 ) ∥ F + ∫ 0 τ ( L ∥ W η ( u ) − W ( u ) ∥ F + C K η ) d u . The nonhomogeneous form of Teschl [31 ] gives
∥ W η ( τ ) − W ‾ ( τ ) ∥ F ≤ e L τ ∥ W η ( 0 ) − W ‾ ( 0 ) ∥ F + C K η ∫ 0 τ e L ( τ − u ) d u ≤ e L S ( ∥ W η ( 0 ) − W ‾ ( 0 ) ∥ F + C K η S ) . \lVert W_{\eta}(\tau)-\overline{W}(\tau)\rVert_{F} \le e^{L\tau}\lVert W_{\eta}(0)-\overline{W}(0)\rVert_{F}+C_{K}\eta\int_{0}^{\tau}e^{L(\tau-u)}\,du
\le e^{LS}\left(\lVert W_{\eta}(0)-\overline{W}(0)\rVert_{F}+C_{K}\eta S\right). ∥ W η ( τ ) − W ( τ ) ∥ F ≤ e Lτ ∥ W η ( 0 ) − W ( 0 ) ∥ F + C K η ∫ 0 τ e L ( τ − u ) d u ≤ e L S ( ∥ W η ( 0 ) − W ( 0 ) ∥ F + C K η S ) . Taking the supremum proves (45) .
Lemma 20 (Linearization at a stationary factor) Fix η > 0 \eta>0 η > 0 K K K f f f W s W_s W s
− a K η ∇ 2 f ( W s ) . -\frac{a_K}{\eta}\nabla^2 f(W_s). − η a K ∇ 2 f ( W s ) . Thus its linearized eigenvalue signs agree with those of negative gradient flow, including zero eigenvalues.
Proof (of Lemma 20). For stationarity, the normalized singular values lie in [ 0 , 1 ) [0,1) [ 0 , 1 ) p K ( s ) > 0 p_K(s)>0 p K ( s ) > 0 s > 0 s>0 s > 0 G = 0 G=0 G = 0 G / ( ∥ G ∥ F + η ) G/(\lVert G\rVert_F+\eta) G / (∥ G ∥ F + η ) D G / η DG/\eta D G / η D P K ( 0 ) = a K I DP_K(0)=a_KI D P K ( 0 ) = a K I D G ( W s ) = − ∇ 2 f ( W s ) DG(W_s)=-\nabla^2 f(W_s) D G ( W s ) = − ∇ 2 f ( W s ) a K / η a_K/\eta a K / η
Lemma 21 (Fixed-denominator expansion) For fixed finite K K K η > 0 \eta>0 η > 0
P K ( G ∥ G ∥ F + η ) = a K η G + O K ( ∥ G ∥ F 2 η 2 ) ( ∥ G ∥ F / η → 0 ) . P_K\left(\frac{G}{\lVert G\rVert_F+\eta}\right)=\frac{a_K}{\eta}G+O_K\left(\frac{\lVert G\rVert_F^2}{\eta^2}\right)\qquad(\lVert G\rVert_F/\eta\to0). P K ( ∥ G ∥ F + η G ) = η a K G + O K ( η 2 ∥ G ∥ F 2 ) (∥ G ∥ F / η → 0 ) . Proof (of Lemma 21). Each cubic composition is odd and has derivative a K I a_KI a K I P K ( X ) = a K X + O K ( ∥ X ∥ F 3 ) P_K(X)=a_KX+O_K(\lVert X\rVert_F^3) P K ( X ) = a K X + O K (∥ X ∥ F 3 )
∥ G ∥ G ∥ F + η − G η ∥ F = ∥ G ∥ F 2 η ( ∥ G ∥ F + η ) ≤ ∥ G ∥ F 2 η 2 . \left\lVert\frac{G}{\lVert G\rVert_F+\eta}-\frac{G}{\eta}\right\rVert_F
=\frac{\lVert G\rVert_F^2}{\eta(\lVert G\rVert_F+\eta)}
\le\frac{\lVert G\rVert_F^2}{\eta^2}. ∥ G ∥ F + η G − η G F = η (∥ G ∥ F + η ) ∥ G ∥ F 2 ≤ η 2 ∥ G ∥ F 2 . The nonlinear polynomial remainder is O K ( ( ∥ G ∥ F / η ) 3 ) O_K((\lVert G\rVert_F/\eta)^3) O K ((∥ G ∥ F / η ) 3 ) O K ( ( ∥ G ∥ F / η ) 2 ) O_K((\lVert G\rVert_F/\eta)^2) O K ((∥ G ∥ F / η ) 2 ) ∥ G ∥ F / η → 0 \lVert G\rVert_F/\eta\to0 ∥ G ∥ F / η → 0
Cited analytic tools and the normalized leading-direction limit
Local solution, uniqueness, dependence, and continuation results are used in the forms of Teschl [31 ] . At each application the vector field is locally Lipschitz on the indicated region, and compact confinement extends the solution as needed. The integral comparison is Teschl [31 ] ; the nonhomogeneous form follows by including the accumulated forcing in the additive term and interchanging the integrals. For measurable coefficients the integrating-factor argument applies to absolutely continuous primitives.
The weighted function spaces used below are Banach spaces after multiplication by their displayed weight. Each application of Teschl [31 ] verifies invariance of a closed ball and a strict contraction estimate. Nonzero determinant polynomials have null zero sets by Mityagin [27 ] ; an identity submatrix witnesses their nontriviality, and Gaussian density transfers nullity to probability zero.
The next lemma states the normalization of the local exit result needed in the empirical passage argument. Its proof retains the asymptotic, uniqueness, and compact-interval comparison steps around the cited shifted-limit theorem.
Lemma 22 (Limit in a simple leading direction). Let g : R N → R N g:\mathbb{R}^{N}\to\mathbb{R}^{N} g : R N → R N C 2 C^{2} C 2 g ( 0 ) = 0 g(0)=0 g ( 0 ) = 0 ψ ( t , z ) \psi(t,z) ψ ( t , z ) z ′ = g ( z ) z'=g(z) z ′ = g ( z ) ( t , z ) ∈ R × R N (t,z)\in\mathbb{R}\times\mathbb{R}^{N} ( t , z ) ∈ R × R N D g ( 0 ) Dg(0) D g ( 0 ) α ∗ > 0 \alpha_{*}>0 α ∗ > 0 r ∗ r_{*} r ∗ l ∗ l_{*} l ∗ l ∗ T r ∗ = 1 l_{*}^{\mathsf{T}}r_{*}=1 l ∗ T r ∗ = 1 e − α ∗ t z ∗ ( t ) → r ∗ e^{-\alpha_{*}t}z_{*}(t)\to r_{*} e − α ∗ t z ∗ ( t ) → r ∗ t → − ∞ t\to-\infty t → − ∞
z ∗ ( t ) = lim α → 0 ψ ( t + α ∗ − 1 log ( 1 / α ) , α r ∗ ) . z_{*}(t)=\lim_{\alpha\to0}\psi\left(t+\alpha_{*}^{-1}\log(1/\alpha),\alpha r_{*}\right). z ∗ ( t ) = α → 0 lim ψ ( t + α ∗ − 1 log ( 1/ α ) , α r ∗ ) . For every family z α → 0 z_{\alpha}\to0 z α → 0
lim inf α → 0 l ∗ T z α ∥ z α ∥ > 0 , \liminf_{\alpha\to0}\frac{l_{*}^{\mathsf{T}}z_{\alpha}}{\lVert z_{\alpha}\rVert}>0, α → 0 lim inf ∥ z α ∥ l ∗ T z α > 0 , and every fixed t ∈ R t\in\mathbb{R} t ∈ R
ψ ( t + α ∗ − 1 log 1 l ∗ T z α , z α ) ⟶ z ∗ ( t ) . \psi\left(t+\alpha_{*}^{-1}\log\frac{1}{l_{*}^{\mathsf{T}}z_{\alpha}},z_{\alpha}\right)\longrightarrow z_{*}(t). ψ ( t + α ∗ − 1 log l ∗ T z α 1 , z α ) ⟶ z ∗ ( t ) . Proof (of Lemma 22). Theorem E.7 of Li et al. (2021) gives the common shifted limit under the stated positive left-projection condition. It allows non-diagonalizable remaining spectral blocks. To relate its normalization to the one used here, choose C ≥ max { 1 , ∥ r ∗ ∥ } C\geq\max\{1,\lVert r_{*}\rVert\} C ≥ max { 1 , ∥ r ∗ ∥}
∥ e s D g ( 0 ) ∥ ≤ C e α ∗ s ( s ≥ 0 ) , ∥ g ( z ) − D g ( 0 ) z ∥ ≤ C ∥ z ∥ 2 \lVert e^{sDg(0)}\rVert\leq Ce^{\alpha_{*}s}\quad(s\geq0),\qquad\lVert g(z)-Dg(0)z\rVert\leq C\lVert z\rVert^{2} ∥ e sD g ( 0 ) ∥ ≤ C e α ∗ s ( s ≥ 0 ) , ∥ g ( z ) − D g ( 0 ) z ∥ ≤ C ∥ z ∥ 2 in a fixed ball about zero for the second estimate. The first estimate follows from the simple leading eigenvalue and the strict gap in real parts, including all nonleading Jordan blocks. Variation of constants gives
∥ ψ ( s , α r ∗ ) − α e α ∗ s r ∗ ∥ ≤ C 2 ∫ 0 s e α ∗ ( s − u ) ∥ ψ ( u , α r ∗ ) ∥ 2 d u . \lVert\psi(s,\alpha r_{*})-\alpha e^{\alpha_{*}s}r_{*}\rVert\leq C^{2}\int_{0}^{s}e^{\alpha_{*}(s-u)}\lVert\psi(u,\alpha r_{*})\rVert^{2}\,du. ∥ ψ ( s , α r ∗ ) − α e α ∗ s r ∗ ∥ ≤ C 2 ∫ 0 s e α ∗ ( s − u ) ∥ ψ ( u , α r ∗ ) ∥ 2 d u . Under the temporary bound ∥ ψ ( u , α r ∗ ) ∥ ≤ 2 C α e α ∗ u \lVert\psi(u,\alpha r_{*})\rVert\leq2C\alpha e^{\alpha_{*}u} ∥ ψ ( u , α r ∗ )∥ ≤ 2 C α e α ∗ u
4 C 4 α ∗ α 2 e α ∗ s ( e α ∗ s − 1 ) . \frac{4C^{4}}{\alpha_{*}}\alpha^{2}e^{\alpha_{*}s}(e^{\alpha_{*}s}-1). α ∗ 4 C 4 α 2 e α ∗ s ( e α ∗ s − 1 ) . If α e α ∗ s \alpha e^{\alpha_{*}s} α e α ∗ s s = t + α ∗ − 1 log ( 1 / α ) s=t+\alpha_*^{-1}\log(1/\alpha) s = t + α ∗ − 1 log ( 1/ α ) α ↓ 0 \alpha\downarrow0 α ↓ 0 t t t
∥ z ∗ ( t ) − e α ∗ t r ∗ ∥ ≤ 4 C 4 α ∗ e 2 α ∗ t . \lVert z_*(t)-e^{\alpha_*t}r_*\rVert\leq\frac{4C^4}{\alpha_*}e^{2\alpha_*t}. ∥ z ∗ ( t ) − e α ∗ t r ∗ ∥ ≤ α ∗ 4 C 4 e 2 α ∗ t . This proves the normalized leading asymptotic and nonconstancy. The limit solves the differential equation: convergence at a fixed time, together with finite-time continuous dependence, identifies all other limits with the solution from that limiting value. Completeness gives its extension to every real time. For any compact interval, apply Theorem E.7 at its left endpoint and then finite-time continuous dependence on the whole interval. This upgrades the convergence to uniform convergence on each compact time interval.
The normalized asymptotic identifies the solution uniquely. If z z z e − α ∗ s z ( s ) → r ∗ e^{-\alpha_*s}z(s)\to r_* e − α ∗ s z ( s ) → r ∗ s → − ∞ s\to-\infty s → − ∞ ℓ ∗ T z ( s ) = e α ∗ s ( 1 + o ( 1 ) ) \ell_*^{\mathsf T}z(s)=e^{\alpha_*s}(1+o(1)) ℓ ∗ T z ( s ) = e α ∗ s ( 1 + o ( 1 )) ℓ ∗ T z ( s ) / ∥ z ( s ) ∥ → 1 / ∥ r ∗ ∥ > 0 \ell_*^{\mathsf T}z(s)/\lVert z(s)\rVert\to1/\lVert r_*\rVert>0 ℓ ∗ T z ( s ) / ∥ z ( s )∥ → 1/ ∥ r ∗ ∥ > 0 z ( s ) z(s) z ( s ) s → − ∞ s\to-\infty s → − ∞ t t t
ψ ( t + α ∗ − 1 log 1 ℓ ∗ T z ( s ) , z ( s ) ) = z ( s + t + α ∗ − 1 log 1 ℓ ∗ T z ( s ) ) = z ( t + o ( 1 ) ) . \psi\left(t+\alpha_*^{-1}\log\frac{1}{\ell_*^{\mathsf T}z(s)},z(s)\right)=z\left(s+t+\alpha_*^{-1}\log\frac{1}{\ell_*^{\mathsf T}z(s)}\right)=z(t+o(1)). ψ ( t + α ∗ − 1 log ℓ ∗ T z ( s ) 1 , z ( s ) ) = z ( s + t + α ∗ − 1 log ℓ ∗ T z ( s ) 1 ) = z ( t + o ( 1 )) . Its limit is both z ∗ ( t ) z_*(t) z ∗ ( t ) z ( t ) z(t) z ( t )
Rank–inertia and PSD optimality
Finite-time row relations are preserved by a right-linear equation. The terminal residual then determines a quadratic cone in which a row would have to grow. The following proof applies with a measurable positive multiplier and retains its noncommuting products.
Proof (of Lemma 12 ). For any fixed coefficient vector c ∈ R m c\in\mathbb{R}^{m} c ∈ R m c T W c^{\mathsf T}W c T W W W W 2 R ( W T W ) B 2R(W^{\mathsf T}W)B 2 R ( W T W ) B d d d
Write D = R ( W s T W s ) D=R(W_s^{\mathsf T}W_s) D = R ( W s T W s ) n + ( D ) = 0 n_+(D)=0 n + ( D ) = 0 d ≤ r d\leq r d ≤ r P 0 P_0 P 0 ker D \ker D ker D σ > 0 \sigma>0 σ > 0 D D D
α = min { 1 , σ 2 16 a 2 ∥ D ∥ o p } , J = D − α P 0 . \alpha=\min\left\{1,\frac{\sigma^2}{16a^2\lVert D\rVert_{\mathrm{op}}}\right\},\qquad J=D-\alpha P_0. α = min { 1 , 16 a 2 ∥ D ∥ op σ 2 } , J = D − α P 0 . The matrix J J J n + ( D ) n_+(D) n + ( D ) α ≤ ∥ D ∥ o p \alpha\leq\lVert D\rVert_{\mathrm{op}} α ≤ ∥ D ∥ op ∥ J ∥ o p = ∥ D ∥ o p \lVert J\rVert_{\mathrm{op}}=\lVert D\rVert_{\mathrm{op}} ∥ J ∥ op = ∥ D ∥ op Q ( z ) = z J z T Q(z)=zJz^{\mathsf T} Q ( z ) = z J z T c T W c^{\mathsf T}W c T W c c c Q ( c T W ) ≥ 0 Q(c^{\mathsf T}W)\geq0 Q ( c T W ) ≥ 0
α ∥ c T W P 0 ∥ 2 ≤ c T W D W T c ≤ ∥ D ∥ o p ∥ c T W ( I − P 0 ) ∥ 2 , ∥ c T W ( I − P 0 ) ∥ ≤ σ − 1 ∥ c T W D ∥ . \alpha\lVert c^{\mathsf T}WP_0\rVert^2\leq c^{\mathsf T}WDW^{\mathsf T}c\leq\lVert D\rVert_{\mathrm{op}}\lVert c^{\mathsf T}W(I-P_0)\rVert^2,\qquad
\lVert c^{\mathsf T}W(I-P_0)\rVert\leq\sigma^{-1}\lVert c^{\mathsf T}WD\rVert. α ∥ c T W P 0 ∥ 2 ≤ c T W D W T c ≤ ∥ D ∥ op ∥ c T W ( I − P 0 ) ∥ 2 , ∥ c T W ( I − P 0 )∥ ≤ σ − 1 ∥ c T W D ∥ . Consequently,
∥ c T W ∥ 2 ≤ 1 + ∥ D ∥ o p / α σ 2 ∥ c T W D ∥ 2 . \lVert c^{\mathsf T}W\rVert^2\leq\frac{1+\lVert D\rVert_{\mathrm{op}}/\alpha}{\sigma^2}\lVert c^{\mathsf T}WD\rVert^2. ∥ c T W ∥ 2 ≤ σ 2 1 + ∥ D ∥ op / α ∥ c T W D ∥ 2 . Differentiation using the actual residual and D P 0 = P 0 D = 0 DP_0=P_0D=0 D P 0 = P 0 D = 0
d d τ Q ( c T W ) = 4 c T W D B D W T c − 4 α c T W D B P 0 W T c + 4 c T W ( R ( W T W ) − D ) B J W T c . (46) \frac{\mathrm{d}}{\mathrm{d}\tau}Q(c^{\mathsf T}W)=4c^{\mathsf T}WDBDW^{\mathsf T}c-4\alpha c^{\mathsf T}WDBP_0W^{\mathsf T}c
+4c^{\mathsf T}W\bigl(R(W^{\mathsf T}W)-D\bigr)BJW^{\mathsf T}c. \tag*{(46)} d τ d Q ( c T W ) = 4 c T W D B D W T c − 4 α c T W D B P 0 W T c + 4 c T W ( R ( W T W ) − D ) B J W T c . (46) The first term is at least 4 ∥ c T W D ∥ 2 4\lVert c^{\mathsf T}WD\rVert^2 4 ∥ c T W D ∥ 2
4 α ∥ c T W D ∥ ∥ c T W P 0 ∥ ≤ 4 α α ∥ D ∥ o p σ ∥ c T W D ∥ 2 ≤ ∥ c T W D ∥ 2 . 4\alpha\lVert c^{\mathsf T}WD\rVert\lVert c^{\mathsf T}WP_0\rVert\le\frac{4\alpha\sqrt{\alpha}\lVert D\rVert_{\mathrm{op}}}{\sigma}\lVert c^{\mathsf T}WD\rVert^2\le\lVert c^{\mathsf T}WD\rVert^2. 4 α ∥ c T W D ∥ ∥ c T W P 0 ∥ ≤ σ 4 α α ∥ D ∥ op ∥ c T W D ∥ 2 ≤ ∥ c T W D ∥ 2 . The last term is bounded in absolute value by
4 α ∥ R ( W T W ) − D ∥ o p ∥ D ∥ o p 1 + ∥ D ∥ o p / α σ 2 ∥ c T W D ∥ 2 . 4\alpha\lVert R(W^{\mathsf T}W)-D\rVert_{\mathrm{op}}\lVert D\rVert_{\mathrm{op}}\frac{1+\lVert D\rVert_{\mathrm{op}}/\alpha}{\sigma^2}\lVert c^{\mathsf T}WD\rVert^2. 4 α ∥ R ( W T W ) − D ∥ op ∥ D ∥ op σ 2 1 + ∥ D ∥ op / α ∥ c T W D ∥ 2 . Its coefficient tends to zero. After a sufficiently late time, chosen independently of c c c
d d τ Q ( c T W ) ≥ 2 ∥ c T W D ∥ 2 ≥ 2 σ Q ( c T W ) . (47) \frac{\mathrm{d}}{\mathrm{d}\tau}Q(c^{\mathsf T}W)\ge2\lVert c^{\mathsf T}WD\rVert^2\ge2\sigma Q(c^{\mathsf T}W). \tag*{(47)} d τ d Q ( c T W ) ≥ 2 ∥ c T W D ∥ 2 ≥ 2 σ Q ( c T W ) . (47) For the second inequality, diagonalization gives D 2 ⪰ σ D D^2\succeq\sigma D D 2 ⪰ σ D σ \sigma σ c T W D W T c ≥ Q ( c T W ) c^{\mathsf T}WDW^{\mathsf T}c\ge Q(c^{\mathsf T}W) c T W D W T c ≥ Q ( c T W )
On an interval where this quadratic form is positive, its logarithmic derivative is at least 2 σ 2\sigma 2 σ d > r − n + ( D ) d>r-n_+(D) d > r − n + ( D ) J J J J J J c T W c^{\mathsf T}W c T W c c c d + n + ( D ) ≤ r d+n_+(D)\le r d + n + ( D ) ≤ r
Lemma 23 (Stationary rank and residual inertia) For a spanning empirical task in any positive factor dimensions, every M ⪰ 0 M\succeq0 M ⪰ 0 M R ( M ) = 0 MR(M)=0 M R ( M ) = 0
rank M + n + ( R ( M ) ) ≤ n + ( C ) . (48) \operatorname{rank} M+n_+\bigl(R(M)\bigr)\le n_+(C). \tag*{(48)} rank M + n + ( R ( M ) ) ≤ n + ( C ) . (48) In particular, under Assumption 9, 1 ≤ n + ( R 1 ) ≤ n + ( C ) − 1 1\le n_+(R_1)\le n_+(C)-1 1 ≤ n + ( R 1 ) ≤ n + ( C ) − 1
Proof (of Lemma 23). The empirical formula gives L ( M ) ⪰ 0 \mathcal{L}(M)\succeq0 L ( M ) ⪰ 0 u ∈ Ran M u\in\operatorname{Ran}M u ∈ Ran M x l T u ≠ 0 x_l^{\mathsf T}u\ne0 x l T u = 0 x l T M x l > 0 x_l^{\mathsf T}Mx_l>0 x l T M x l > 0 M x l = 0 Mx_l=0 M x l = 0 x l x_l x l Ran M \operatorname{Ran}M Ran M
u T L ( M ) u = 1 n ∑ l ( x l T M x l ) ( x l T u ) 2 > 0. u^{\mathsf T}\mathcal{L}(M)u=\frac{1}{n}\sum_l(x_l^{\mathsf T}Mx_l)(x_l^{\mathsf T}u)^2>0. u T L ( M ) u = n 1 l ∑ ( x l T M x l ) ( x l T u ) 2 > 0. Symmetry and M R ( M ) = 0 MR(M)=0 M R ( M ) = 0 R ( M ) M = 0 R(M)M=0 R ( M ) M = 0 Ran M ⊂ ker R ( M ) \operatorname{Ran} M \subset\ker R(M) Ran M ⊂ ker R ( M ) u ∈ Ran M u \in\operatorname{Ran} M u ∈ Ran M v v v R ( M ) R(M) R ( M )
( u + v ) ⊤ C ( u + v ) = v ⊤ R ( M ) v + ( u + v ) ⊤ L ( M ) ( u + v ) . (u+v)^{\top}C(u+v)=v^{\top}R(M)v+(u+v)^{\top}L(M)(u+v). ( u + v ) ⊤ C ( u + v ) = v ⊤ R ( M ) v + ( u + v ) ⊤ L ( M ) ( u + v ) . This is positive whenever ( u , v ) ≠ ( 0 , 0 ) (u,v)\ne(0,0) ( u , v ) = ( 0 , 0 ) v = 0 v=0 v = 0 v ≠ 0 v\ne0 v = 0 C C C C C C C C C rank M + n + ( R ( M ) ) ≤ n + ( C ) \operatorname{rank} M+n_{+}(R(M))\le n_{+}(C) rank M + n + ( R ( M )) ≤ n + ( C ) M = M 1 M=M_1 M = M 1 n + ( R 1 ) ≥ 1 n_{+}(R_1)\ge1 n + ( R 1 ) ≥ 1
Lemma 24 (PSD optimality conditions) If M s ⪰ 0 M_s\succeq0 M s ⪰ 0 M s R ( M s ) = 0 M_sR(M_s)=0 M s R ( M s ) = 0 R ( M s ) ⪯ 0 R(M_s)\preceq0 R ( M s ) ⪯ 0 Φ ( M ) ≥ Φ ( M s ) \Phi(M)\ge\Phi(M_s) Φ ( M ) ≥ Φ ( M s ) M ⪰ 0 M\succeq0 M ⪰ 0
Proof (of Lemma 24). For any M ⪰ 0 M\succeq0 M ⪰ 0 ∇ Φ ( M s ) = − R ( M s ) \nabla\Phi(M_s)=-R(M_s) ∇Φ ( M s ) = − R ( M s )
Φ ( M ) ≥ Φ ( M s ) + ⟨ − R ( M s ) , M − M s ⟩ ≥ Φ ( M s ) . \Phi(M)\ge\Phi(M_s)+\langle-R(M_s),M-M_s\rangle\ge\Phi(M_s). Φ ( M ) ≥ Φ ( M s ) + ⟨ − R ( M s ) , M − M s ⟩ ≥ Φ ( M s ) . The second inequality uses − R ( M s ) ⪰ 0 -R(M_s)\succeq0 − R ( M s ) ⪰ 0 ⟨ R ( M s ) , M s ⟩ = 0 \langle R(M_s),M_s\rangle=0 ⟨ R ( M s ) , M s ⟩ = 0
Corollary 25 (Full-column-rank convergence to a PSD optimum) Fix finite K K K η ≥ 0 \eta\ge0 η ≥ 0 W ( 0 ) W(0) W ( 0 ) r r r W ′ = H K , η ( G ( W ) ) W'=H_{K,\eta}(G(W)) W ′ = H K , η ( G ( W )) W s W_s W s W s ⊤ W s W_s^{\top}W_s W s ⊤ W s Φ \Phi Φ S + \mathcal{S}_{+} S + ε > 0 \varepsilon>0 ε > 0 r r r
Proof (of Corollary 25). Proposition 1 gives a stationary limit W s W_s W s r r r R ( W s ⊤ W s ) ⪯ 0 R(W_s^{\top}W_s)\preceq0 R ( W s ⊤ W s ) ⪯ 0 ( W s ⊤ W s ) R ( W s ⊤ W s ) = 0 (W_s^{\top}W_s)R(W_s^{\top}W_s)=0 ( W s ⊤ W s ) R ( W s ⊤ W s ) = 0 ε > 0 \varepsilon>0 ε > 0
Gaussian Entrance and Actual Passages
The proof first establishes independent Gaussian coefficients on the fixed initial scale. It then propagates their relative volumes through each actual passage, including the empirical and regularization terms. The final residence selects the boundary-value entrance of Theorem 7.
Initial rows and independent Gaussian components
The row expansion is taken on a fixed finite initial-layer interval. Its leading row determines the coefficients of a linear equation for the weaker rows; eliminating their parallel components preserves a nonsingular conditional Gaussian law. Write the rows of U U U u ⊤ , u 2 ⊤ , … , u m ⊤ u^\top, u_2^\top, \ldots, u_m^\top u ⊤ , u 2 ⊤ , … , u m ⊤ Y ε ( σ ) Y^\varepsilon(\sigma) Y ε ( σ ) y a ε ( σ ) ⊤ y_a^\varepsilon(\sigma)^\top y a ε ( σ ) ⊤ w ( σ ) , h a ( σ ) ∈ R r w(\sigma), h_a(\sigma) \in\mathbb{R}^r w ( σ ) , h a ( σ ) ∈ R r
d w ( σ ) ⊤ d σ = w ( σ ) ⊤ A ( σ ) , w ( 0 ) = u , d h a ( σ ) ⊤ d σ = h a ( σ ) ⊤ A ( σ ) , h a ( 0 ) = u a ( 2 ≤ a ≤ m ) . (49) \begin{aligned}
\frac{\mathrm{d}w(\sigma)^\top}{\mathrm{d}\sigma} &= w(\sigma)^\top A(\sigma), \qquad w(0)=u,\\
\frac{\mathrm{d}h_a(\sigma)^\top}{\mathrm{d}\sigma} &= h_a(\sigma)^\top A(\sigma), \qquad h_a(0)=u_a \quad(2 \le a \le m). \tag*{(49)}
\end{aligned} d σ d w ( σ ) ⊤ d σ d h a ( σ ) ⊤ = w ( σ ) ⊤ A ( σ ) , w ( 0 ) = u , = h a ( σ ) ⊤ A ( σ ) , h a ( 0 ) = u a ( 2 ≤ a ≤ m ) . (49) where
A ( σ ) = 2 C 2 ∥ w ( σ ) ⊤ C ∥ + c η p K ( 4 C w ( σ ) w ( σ ) ⊤ C ( 2 ∥ w ( σ ) ⊤ C ∥ + c η ) 2 ) . (50) A(\sigma)=\frac{2C}{2\lVert w(\sigma)^\top C\rVert+c_\eta}p_K\left(\frac{4Cw(\sigma)w(\sigma)^\top C}{\left(2\lVert w(\sigma)^\top C\rVert+c_\eta\right)^2}\right). \tag*{(50)} A ( σ ) = 2 ∥ w ( σ ) ⊤ C ∥ + c η 2 C p K ( ( 2 ∥ w ( σ ) ⊤ C ∥ + c η ) 2 4 C w ( σ ) w ( σ ) ⊤ C ) . (50) In the first row, the same equation reads
d w ( σ ) ⊤ d σ = p K ( 2 ∥ w ( σ ) ⊤ C ∥ 2 ∥ w ( σ ) ⊤ C ∥ + c η ) w ( σ ) ⊤ C ∥ w ( σ ) ⊤ C ∥ . (51) \frac{\mathrm{d}w(\sigma)^\top}{\mathrm{d}\sigma}
=p_K\left(\frac{2\lVert w(\sigma)^\top C\rVert}{2\lVert w(\sigma)^\top C\rVert+c_\eta}\right)\frac{w(\sigma)^\top C}{\lVert w(\sigma)^\top C\rVert}. \tag*{(51)} d σ d w ( σ ) ⊤ = p K ( 2 ∥ w ( σ ) ⊤ C ∥ + c η 2 ∥ w ( σ ) ⊤ C ∥ ) ∥ w ( σ ) ⊤ C ∥ w ( σ ) ⊤ C . (51) Lemma 26 (Finite-interval row expansion) Under Assumption 2, fix K K K b b b c η c_\eta c η u ⊤ v 1 ≠ 0 u^\top v_1 \ne0 u ⊤ v 1 = 0 S > 0 S>0 S > 0 [ 0 , ∞ ) [0,\infty) [ 0 , ∞ ) w ( σ ) ⊤ v 1 ≠ 0 w(\sigma)^\top v_1 \ne0 w ( σ ) ⊤ v 1 = 0 σ ≥ 0 \sigma\ge0 σ ≥ 0 0 ≤ σ ≤ S 0 \le\sigma\le S 0 ≤ σ ≤ S
y 1 ε ( σ ) = w ( σ ) + O U , S ( ε 2 ( b − 1 ) + ε 2 ) , y a ε ( σ ) = ε b − 1 [ h a ( σ ) + O U , S ( ε 2 ( b − 1 ) + ε 2 ) ] , 2 ≤ a ≤ m . (52) \begin{aligned}
y_1^\varepsilon(\sigma)&=w(\sigma)+O_{U,S}\left(\varepsilon^{2(b-1)}+\varepsilon^2\right),\\
y_a^\varepsilon(\sigma)&=\varepsilon^{b-1}\left[h_a(\sigma)+O_{U,S}\left(\varepsilon^{2(b-1)}+\varepsilon^2\right)\right], \qquad2\le a\le m. \tag*{(52)}
\end{aligned} y 1 ε ( σ ) y a ε ( σ ) = w ( σ ) + O U , S ( ε 2 ( b − 1 ) + ε 2 ) , = ε b − 1 [ h a ( σ ) + O U , S ( ε 2 ( b − 1 ) + ε 2 ) ] , 2 ≤ a ≤ m . (52) Proof (of Lemma 26). First consider the first equation in (49). The matrix polynomial identity in Lemma 16, applied to a one-row matrix, gives (51). Introduce a scalar time ϑ \vartheta ϑ
d ϑ d σ = p K ( 2 ∥ w ( σ ) ⊤ C ∥ / ( 2 ∥ w ( σ ) ⊤ C ∥ + c η ) ) ∥ w ( σ ) ⊤ C ∥ , ϑ ( 0 ) = 0. \frac{\mathrm{d}\vartheta}{\mathrm{d}\sigma}
=\frac{p_K\left(2\lVert w(\sigma)^\top C\rVert/(2\lVert w(\sigma)^\top C\rVert+c_\eta)\right)}{\lVert w(\sigma)^\top C\rVert},
\qquad\vartheta(0)=0. d σ d ϑ = ∥ w ( σ ) ⊤ C ∥ p K ( 2 ∥ w ( σ ) ⊤ C ∥ / ( 2 ∥ w ( σ ) ⊤ C ∥ + c η ) ) , ϑ ( 0 ) = 0. As long as the solution is defined, this number is positive and
w ( σ ) ⊤ = u ⊤ e ϑ ( σ ) C , w ( σ ) ⊤ v 1 = ( u ⊤ v 1 ) e λ 1 ϑ ( σ ) . w(\sigma)^\top=u^\top e^{\vartheta(\sigma)C},\qquad
w(\sigma)^\top v_1=(u^\top v_1)e^{\lambda_1\vartheta(\sigma)}. w ( σ ) ⊤ = u ⊤ e ϑ ( σ ) C , w ( σ ) ⊤ v 1 = ( u ⊤ v 1 ) e λ 1 ϑ ( σ ) . In particular the last quantity never vanishes, and ∥ w ( σ ) ⊤ C ∥ > 0 \lVert w(\sigma)^\top C\rVert>0 ∥ w ( σ ) ⊤ C ∥ > 0 p K ( s ) ≤ 1 p_K(s)\le1 p K ( s ) ≤ 1 [ 0 , 1 ] [0,1] [ 0 , 1 ] ∥ d w / d σ ∥ ≤ 1 \lVert\mathrm{d}w/\mathrm{d}\sigma\rVert\le1 ∥ d w / d σ ∥ ≤ 1 ∥ w ( σ ) ∥ ≤ ∥ u ∥ + σ \lVert w(\sigma)\rVert\le\lVert u\rVert+\sigma ∥ w ( σ )∥ ≤ ∥ u ∥ + σ ∣ w ( σ ) ⊤ v 1 ∣ ≥ ∣ u ⊤ v 1 ∣ > 0 \lvert w(\sigma)^\top v_1\rvert\ge\lvert u^\top v_1\rvert>0 ∣ w ( σ ) ⊤ v 1 ∣ ≥ ∣ u ⊤ v 1 ∣ > 0 [31 ] . Thus w w w A ( σ ) A(\sigma) A ( σ ) h a h_a h a
∥ h a ( σ ) ∥ ≤ ∥ u a ∥ exp ( ∫ 0 σ ∥ A ( s ) ∥ o p d s ) . \lVert h_a(\sigma)\rVert\le\lVert u_a\rVert\exp\left(\int_0^\sigma\lVert A(s)\rVert_{\mathrm{op}}\,\mathrm{d}s\right). ∥ h a ( σ )∥ ≤ ∥ u a ∥ exp ( ∫ 0 σ ∥ A ( s ) ∥ op d s ) . This proves the asserted existence of all row equations.
We next quantify their relation to the actual matrix. Use two independent real parameters a , z a,z a , z Y a , z Y^{a,z} Y a , z
d Y a , z d σ = P K ( 2 Y a , z [ C − z L ( ( Y a , z ) ⊤ Y a , z ) ] 2 ∥ Y a , z [ C − z L ( ( Y a , z ) ⊤ Y a , z ) ] ∥ F + c η ) , Y a , z ( 0 ) = diag ( 1 , a , … , a ) U . \frac{\mathrm{d}Y^{a,z}}{\mathrm{d}\sigma}
=P_K\left(\frac{2Y^{a,z}\left[C-zL\left((Y^{a,z})^\top Y^{a,z}\right)\right]}{2\left\lVert Y^{a,z}\left[C-zL\left((Y^{a,z})^\top Y^{a,z}\right)\right]\right\rVert_F+c_\eta}\right),
\qquad Y^{a,z}(0)=\operatorname{diag}(1,a,\ldots,a)U. d σ d Y a , z = P K ( 2 ∥ Y a , z [ C − z L ( ( Y a , z ) ⊤ Y a , z ) ] ∥ F + c η 2 Y a , z [ C − z L ( ( Y a , z ) ⊤ Y a , z ) ] ) , Y a , z ( 0 ) = diag ( 1 , a , … , a ) U . At ( a , z ) = ( 0 , 0 ) (a,z)=(0,0) ( a , z ) = ( 0 , 0 ) e 1 w ( σ ) T e_1w(\sigma)^{\mathsf T} e 1 w ( σ ) T S S S e 1 w ( σ ) T C e_1w(\sigma)^{\mathsf T}C e 1 w ( σ ) T C 0 ≤ σ ≤ S 0\leq\sigma\leq S 0 ≤ σ ≤ S ( a , z ) = ( 0 , 0 ) (a,z)=(0,0) ( a , z ) = ( 0 , 0 ) C ∞ C^\infty C ∞ c η c_\eta c η [31 ] , on a finite cover of [ 0 , S ] [0,S] [ 0 , S ] Y a , z Y^{a,z} Y a , z
The transformation Y ↦ diag ( 1 , − 1 , … , − 1 ) Y Y\mapsto\operatorname{diag}(1,-1,\ldots,-1)Y Y ↦ diag ( 1 , − 1 , … , − 1 ) Y Y T Y Y^{\mathsf T}Y Y T Y P K P_K P K Y − a , z = diag ( 1 , − 1 , … , − 1 ) Y a , z Y^{-a,z}=\operatorname{diag}(1,-1,\ldots,-1)Y^{a,z} Y − a , z = diag ( 1 , − 1 , … , − 1 ) Y a , z a a a a a a a = 0 a=0 a = 0 z z z Y Y Y [ 0 , S ] [0,S] [ 0 , S ]
( Y a , z ) 1 , : = w ( σ ) T + O U , S ( a 2 + ∣ z ∣ ) , \left(Y^{a,z}\right)_{1,:}=w(\sigma)^{\mathsf T}+O_{U,S}(a^2+|z|), ( Y a , z ) 1 , : = w ( σ ) T + O U , S ( a 2 + ∣ z ∣ ) , ( Y a , z ) a 0 , : = a [ ∂ a ( Y a , 0 ) a 0 , : ∣ a = 0 + O U , S ( a 2 + ∣ z ∣ ) ] , 2 ≤ a 0 ≤ m . \left(Y^{a,z}\right)_{a_0,:}=a\left[\partial_a\left(Y^{a,0}\right)_{a_0,:}\big|_{a=0}+O_{U,S}(a^2+|z|)\right],\qquad2\leq a_0\leq m. ( Y a , z ) a 0 , : = a [ ∂ a ( Y a , 0 ) a 0 , : a = 0 + O U , S ( a 2 + ∣ z ∣ ) ] , 2 ≤ a 0 ≤ m . For each weak-row entry, oddness eliminates its constant and quadratic Taylor terms in a a a a = 0 a=0 a = 0 O ( a 3 ) O(a^3) O ( a 3 ) z z z 0 0 0 O ( ∣ z ∣ ) O(|z|) O ( ∣ z ∣ )
It remains to identify the derivative in this formula. At Y = e 1 w T Y=e_1w^{\mathsf T} Y = e 1 w T
δ ( Y T Y ) = ( δ Y ) T e 1 w T + w e 1 T δ Y = 0 , δ ∥ Y C ∥ F 2 = tr ( C δ ( Y T Y ) C ) = 0. \delta(Y^{\mathsf T}Y)=(\delta Y)^{\mathsf T}e_1w^{\mathsf T}+we_1^{\mathsf T}\delta Y=0,\qquad\delta\lVert YC\rVert_F^2=\operatorname{tr}\bigl(C\delta(Y^{\mathsf T}Y)C\bigr)=0. δ ( Y T Y ) = ( δ Y ) T e 1 w T + w e 1 T δ Y = 0 , δ ∥ Y C ∥ F 2 = tr ( C δ ( Y T Y ) C ) = 0. At z = 0 z=0 z = 0
Y 2 C 2 ∥ Y C ∥ F + c η q K ( 4 C Y T Y C ( 2 ∥ Y C ∥ F + c η ) 2 ) . Y\frac{2C}{2\lVert YC\rVert_F+c_\eta}{}^{q_K}\left(\frac{4CY^{\mathsf T}YC}{(2\lVert YC\rVert_F+c_\eta)^2}\right). Y 2 ∥ Y C ∥ F + c η 2 C q K ( ( 2 ∥ Y C ∥ F + c η ) 2 4 C Y T Y C ) . The displayed first variations make the derivative of this entire right coefficient zero in a weak-row perturbation. Its coefficient at the base solution is exactly A ( σ ) A(\sigma) A ( σ ) a a a a 0 a_0 a 0 d h a 0 T / d σ = h a 0 T A dh_{a_0}^{\mathsf T}/d\sigma=h_{a_0}^{\mathsf T}A d h a 0 T / d σ = h a 0 T A h a 0 ( 0 ) = u a 0 h_{a_0}(0)=u_{a_0} h a 0 ( 0 ) = u a 0 a = ε b − 1 a=\varepsilon^{b-1} a = ε b − 1 z = ε 2 z=\varepsilon^2 z = ε 2 S S S
Let V ⊥ = ( v 2 , … , v r ) V_\perp=(v_2,\ldots,v_r) V ⊥ = ( v 2 , … , v r )
ζ a ( σ ) = h a ( σ ) T V ⊥ − h a ( σ ) T v 1 w ( σ ) T v 1 w ( σ ) T V ⊥ , (53) \zeta_a(\sigma)=h_a(\sigma)^{\mathsf T}V_\perp-\frac{h_a(\sigma)^{\mathsf T}v_1}{w(\sigma)^{\mathsf T}v_1}w(\sigma)^{\mathsf T}V_\perp, \tag*{(53)} ζ a ( σ ) = h a ( σ ) T V ⊥ − w ( σ ) T v 1 h a ( σ ) T v 1 w ( σ ) T V ⊥ , (53) B ( σ ) = V ⊥ T A ( σ ) V ⊥ − V ⊥ T A ( σ ) v 1 w ( σ ) T V ⊥ w ( σ ) T v 1 . (54) B(\sigma)=V_\perp^{\mathsf T}A(\sigma)V_\perp-\frac{V_\perp^{\mathsf T}A(\sigma)v_1w(\sigma)^{\mathsf T}V_\perp}{w(\sigma)^{\mathsf T}v_1}. \tag*{(54)} B ( σ ) = V ⊥ T A ( σ ) V ⊥ − w ( σ ) T v 1 V ⊥ T A ( σ ) v 1 w ( σ ) T V ⊥ . (54) Lemma 27 (Projected weak-row transport). For the data of Lemma 26,
d ζ a d σ = ζ a B ( σ ) . (55) \frac{\mathrm{d}\zeta_a}{\mathrm{d}\sigma}=\zeta_a B(\sigma). \tag*{(55)} d σ d ζ a = ζ a B ( σ ) . (55) The matrix solution
Ψ ′ ( σ ) = Ψ ( σ ) B ( σ ) , Ψ ( 0 ) = I r − 1 (56) \Psi'(\sigma)=\Psi(\sigma)B(\sigma),\qquad\Psi(0)=I_{r-1} \tag*{(56)} Ψ ′ ( σ ) = Ψ ( σ ) B ( σ ) , Ψ ( 0 ) = I r − 1 (56) is invertible at every finite σ \sigma σ
ζ a ( σ ) = ζ a ( 0 ) Ψ ( σ ) , ζ a ( 0 ) = u a T V ⊥ − u a T v 1 u T v 1 u T V ⊥ . (57) \zeta_a(\sigma)=\zeta_a(0)\Psi(\sigma),\qquad\zeta_a(0)=u_a^{\mathsf{T}}V_\perp-\frac{u_a^{\mathsf{T}}v_1}{u^{\mathsf{T}}v_1}u^{\mathsf{T}}V_\perp. \tag*{(57)} ζ a ( σ ) = ζ a ( 0 ) Ψ ( σ ) , ζ a ( 0 ) = u a T V ⊥ − u T v 1 u a T v 1 u T V ⊥ . (57) Conditional on u u u σ \sigma σ
Cov ( ζ a ( σ ) T ∣ u ) = Ψ ( σ ) T [ I r − 1 + ( V ⊥ T u ) ( V ⊥ T u ) T ( u T v 1 ) 2 ] Ψ ( σ ) ≻ 0. (58) \operatorname{Cov}(\zeta_a(\sigma)^{\mathsf{T}}\mid u)=\Psi(\sigma)^{\mathsf{T}}\left[I_{r-1}+\frac{(V_\perp^{\mathsf{T}}u)(V_\perp^{\mathsf{T}}u)^{\mathsf{T}}}{(u^{\mathsf{T}}v_1)^2}\right]\Psi(\sigma)\succ0. \tag*{(58)} Cov ( ζ a ( σ ) T ∣ u ) = Ψ ( σ ) T [ I r − 1 + ( u T v 1 ) 2 ( V ⊥ T u ) ( V ⊥ T u ) T ] Ψ ( σ ) ≻ 0. (58) For every fixed finite S S S [ 0 , S ] [0,S] [ 0 , S ]
y a ε ( σ ) T V ⊥ − y a ε ( σ ) T v 1 y 1 ε ( σ ) T v 1 y 1 ε ( σ ) T V ⊥ = ε b − 1 [ ζ a ( σ ) + O U , S ( ε 2 ( b − 1 ) + ε 2 ) ] . (59) y_a^\varepsilon(\sigma)^{\mathsf{T}}V_\perp-\frac{y_a^\varepsilon(\sigma)^{\mathsf{T}}v_1}{y_1^\varepsilon(\sigma)^{\mathsf{T}}v_1}y_1^\varepsilon(\sigma)^{\mathsf{T}}V_\perp
=\varepsilon^{b-1}\left[\zeta_a(\sigma)+O_{U,S}\left(\varepsilon^{2(b-1)}+\varepsilon^2\right)\right]. \tag*{(59)} y a ε ( σ ) T V ⊥ − y 1 ε ( σ ) T v 1 y a ε ( σ ) T v 1 y 1 ε ( σ ) T V ⊥ = ε b − 1 [ ζ a ( σ ) + O U , S ( ε 2 ( b − 1 ) + ε 2 ) ] . (59) The denominator on the left is nonzero throughout [ 0 , S ] [0,S] [ 0 , S ] ε > 0 \varepsilon>0 ε > 0
Proof (of Lemma 27). The denominator w ( σ ) T v 1 w(\sigma)^{\mathsf{T}}v_1 w ( σ ) T v 1
h a ( σ ) T − h a ( σ ) T v 1 w ( σ ) T v 1 w ( σ ) T h_a(\sigma)^{\mathsf{T}}-\frac{h_a(\sigma)^{\mathsf{T}}v_1}{w(\sigma)^{\mathsf{T}}v_1}w(\sigma)^{\mathsf{T}} h a ( σ ) T − w ( σ ) T v 1 h a ( σ ) T v 1 w ( σ ) T annihilates v 1 v_1 v 1 ζ a ( σ ) V ⊥ T \zeta_a(\sigma)V_\perp^{\mathsf{T}} ζ a ( σ ) V ⊥ T
d d σ h a ( σ ) T v 1 w ( σ ) T v 1 = [ h a ( σ ) T A ( σ ) v 1 ] [ w ( σ ) T v 1 ] − [ h a ( σ ) T v 1 ] [ w ( σ ) T A ( σ ) v 1 ] [ w ( σ ) T v 1 ] 2 = ζ a ( σ ) V ⊥ T A ( σ ) v 1 w ( σ ) T v 1 . \frac{\mathrm{d}}{\mathrm{d}\sigma}\frac{h_a(\sigma)^{\mathsf{T}}v_1}{w(\sigma)^{\mathsf{T}}v_1}
=
\frac{[h_a(\sigma)^{\mathsf{T}}A(\sigma)v_1][w(\sigma)^{\mathsf{T}}v_1]-[h_a(\sigma)^{\mathsf{T}}v_1][w(\sigma)^{\mathsf{T}}A(\sigma)v_1]}{[w(\sigma)^{\mathsf{T}}v_1]^2}
=
\frac{\zeta_a(\sigma)V_\perp^{\mathsf{T}}A(\sigma)v_1}{w(\sigma)^{\mathsf{T}}v_1}. d σ d w ( σ ) T v 1 h a ( σ ) T v 1 = [ w ( σ ) T v 1 ] 2 [ h a ( σ ) T A ( σ ) v 1 ] [ w ( σ ) T v 1 ] − [ h a ( σ ) T v 1 ] [ w ( σ ) T A ( σ ) v 1 ] = w ( σ ) T v 1 ζ a ( σ ) V ⊥ T A ( σ ) v 1 . Differentiating (53), grouping the two terms that contain a differentiated row, and then using this quotient identity gives
d ζ a d σ = [ h a ( σ ) T − h a ( σ ) T v 1 w ( σ ) T v 1 w ( σ ) T ] A ( σ ) V ⊥ − ζ a ( σ ) V ⊥ T A ( σ ) v 1 w ( σ ) T v 1 w ( σ ) T V ⊥ = ζ a ( σ ) B ( σ ) . \frac{\mathrm{d}\zeta_a}{\mathrm{d}\sigma}
=
\left[h_a(\sigma)^{\mathsf{T}}-\frac{h_a(\sigma)^{\mathsf{T}}v_1}{w(\sigma)^{\mathsf{T}}v_1}w(\sigma)^{\mathsf{T}}\right]A(\sigma)V_\perp
-\frac{\zeta_a(\sigma)V_\perp^{\mathsf{T}}A(\sigma)v_1}{w(\sigma)^{\mathsf{T}}v_1}w(\sigma)^{\mathsf{T}}V_\perp
=\zeta_a(\sigma)B(\sigma). d σ d ζ a = [ h a ( σ ) T − w ( σ ) T v 1 h a ( σ ) T v 1 w ( σ ) T ] A ( σ ) V ⊥ − w ( σ ) T v 1 ζ a ( σ ) V ⊥ T A ( σ ) v 1 w ( σ ) T V ⊥ = ζ a ( σ ) B ( σ ) . The quotient derivative accounts for the motion of the occupied row. The coefficient B B B Ψ \Psi Ψ (56) . A row solution that vanishes at a finite time vanishes initially by backward uniqueness. Since Ψ ( 0 ) = I \Psi(0)=I Ψ ( 0 ) = I Ψ \Psi Ψ ζ a ( 0 ) Ψ \zeta_a(0)\Psi ζ a ( 0 ) Ψ ζ a \zeta_a ζ a (57) .
Conditional on u u u u T v 1 u^{\mathsf T}v_1 u T v 1 u T V ⊥ u^{\mathsf T}V_\perp u T V ⊥ 1 1 1 I r − 1 I_{r-1} I r − 1
I r − 1 + ( V ⊥ T u ) ( V ⊥ T u ) T ( u T v 1 ) 2 . I_{r-1}+\frac{(V_\perp^{\mathsf T}u)(V_\perp^{\mathsf T}u)^{\mathsf T}}{(u^{\mathsf T}v_1)^2}. I r − 1 + ( u T v 1 ) 2 ( V ⊥ T u ) ( V ⊥ T u ) T . Its quadratic form at nonzero x x x ∥ x ∥ 2 + ( u T V ⊥ x ) 2 / ( u T v 1 ) 2 > 0 \lVert x\rVert^2+(u^{\mathsf T}V_\perp x)^2/(u^{\mathsf T}v_1)^2>0 ∥ x ∥ 2 + ( u T V ⊥ x ) 2 / ( u T v 1 ) 2 > 0 A A A B B B Ψ \Psi Ψ u u u σ \sigma σ
Ψ ( σ ) [ I r − 1 + ( V ⊥ T u ) ( V ⊥ T u ) T ( u T v 1 ) 2 ] Ψ ( σ ) T ≻ 0. \Psi(\sigma)\left[I_{r-1}+\frac{(V_\perp^{\mathsf T}u)(V_\perp^{\mathsf T}u)^{\mathsf T}}{(u^{\mathsf T}v_1)^2}\right]\Psi(\sigma)^{\mathsf T}\succ0. Ψ ( σ ) [ I r − 1 + ( u T v 1 ) 2 ( V ⊥ T u ) ( V ⊥ T u ) T ] Ψ ( σ ) T ≻ 0. Conditional independence of the weak rows is preserved by these separate deterministic linear transformations.
Finally fix S S S inf 0 ≤ σ ≤ S ∣ w ( σ ) T v 1 ∣ > 0 \inf_{0\leq\sigma\leq S}|w(\sigma)^{\mathsf T}v_1|>0 inf 0 ≤ σ ≤ S ∣ w ( σ ) T v 1 ∣ > 0 Lemma 26 gives, uniformly there,
1 y 1 ε ( σ ) T v 1 = 1 w ( σ ) T v 1 + O U , S ( ε 2 ( b − 1 ) + ε 2 ) , \frac{1}{\mathsf{y}_1^\varepsilon(\sigma)^{\mathsf T}v_1}
=\frac{1}{w(\sigma)^{\mathsf T}v_1}+O_{U,S}\left(\varepsilon^{2(b-1)}+\varepsilon^2\right), y 1 ε ( σ ) T v 1 1 = w ( σ ) T v 1 1 + O U , S ( ε 2 ( b − 1 ) + ε 2 ) , y a ε ( σ ) T v 1 y 1 ε ( σ ) T v 1 = ε b − 1 [ h a ( σ ) T v 1 w ( σ ) T v 1 + O U , S ( ε 2 ( b − 1 ) + ε 2 ) ] . \frac{\mathsf{y}_a^\varepsilon(\sigma)^{\mathsf T}v_1}{\mathsf{y}_1^\varepsilon(\sigma)^{\mathsf T}v_1}
=\varepsilon^{b-1}\left[\frac{h_a(\sigma)^{\mathsf T}v_1}{w(\sigma)^{\mathsf T}v_1}+O_{U,S}\left(\varepsilon^{2(b-1)}+\varepsilon^2\right)\right]. y 1 ε ( σ ) T v 1 y a ε ( σ ) T v 1 = ε b − 1 [ w ( σ ) T v 1 h a ( σ ) T v 1 + O U , S ( ε 2 ( b − 1 ) + ε 2 ) ] . The first equality follows by subtracting the reciprocals and bounding both denominators below; all row factors in the second are bounded on [ 0 , S ] [0,S] [ 0 , S ] y 1 ε ( σ ) T V ⊥ = w ( σ ) T V ⊥ + O U , S ( ε 2 ( b − 1 ) + ε 2 ) \mathsf{y}_1^\varepsilon(\sigma)^{\mathsf T}V_\perp=w(\sigma)^{\mathsf T}V_\perp+O_{U,S}(\varepsilon^{2(b-1)}+\varepsilon^2) y 1 ε ( σ ) T V ⊥ = w ( σ ) T V ⊥ + O U , S ( ε 2 ( b − 1 ) + ε 2 )
Put κ = ( λ 1 − λ 2 ) / λ 1 ∈ ( 0 , 1 ) \kappa=(\lambda_1-\lambda_2)/\lambda_1\in(0,1) κ = ( λ 1 − λ 2 ) / λ 1 ∈ ( 0 , 1 )
Lemma 28 (Leading-row asymptotic scale) Under Assumption 2, for every fixed u u u u T v 1 ≠ 0 u^{\mathsf T}v_1\ne0 u T v 1 = 0
w ( σ ) T σ = sign ( u T v 1 ) v 1 T + O u ( σ − κ ) ( σ → ∞ ) . (60) \frac{w(\sigma)^{\mathsf T}}{\sigma}
=\operatorname{sign}(u^{\mathsf T}v_1)v_1^{\mathsf T}+O_u(\sigma^{-\kappa})
\qquad(\sigma\to\infty). \tag*{(60)} σ w ( σ ) T = sign ( u T v 1 ) v 1 T + O u ( σ − κ ) ( σ → ∞ ) . (60) Consequently, for every prescribed neighborhood of { O e 1 v 1 T : O ∈ O ( m ) } \{Oe_1v_1^{\mathsf T}:O\in O(m)\} { O e 1 v 1 T : O ∈ O ( m )} 1 / σ 0 1/\sigma_0 1/ σ 0 σ 0 ( U ) \sigma_0(U) σ 0 ( U ) Q ε ( ε σ 0 ) Q^\varepsilon(\varepsilon\sigma_0) Q ε ( ε σ 0 ) ε \varepsilon ε ε ↓ 0 \varepsilon\downarrow0 ε ↓ 0
Proof (of Lemma 28). Use the scalar time ϑ \vartheta ϑ
∥ u T e ϑ C C ∥ 2 = λ 1 2 ( u T v 1 ) 2 e 2 λ 1 ϑ + ∑ l = 2 r λ l 2 ( u T v l ) 2 e 2 λ l ϑ , \lVert u^{\mathsf T}e^{\vartheta C}C\rVert^2
=\lambda_1^2(u^{\mathsf T}v_1)^2e^{2\lambda_1\vartheta}
+\sum_{l=2}^{r}\lambda_l^2(u^{\mathsf T}v_l)^2e^{2\lambda_l\vartheta}, ∥ u T e ϑC C ∥ 2 = λ 1 2 ( u T v 1 ) 2 e 2 λ 1 ϑ + l = 2 ∑ r λ l 2 ( u T v l ) 2 e 2 λ l ϑ , ∥ u T e ϑ C ∥ = λ 1 ∣ u T v 1 ∣ e λ 1 ϑ [ 1 + O u ( e − 2 ( λ 1 − λ 2 ) ϑ ) ] . \lVert u^{\mathsf T}e^{\vartheta C}\rVert
=\lambda_1|u^{\mathsf T}v_1|e^{\lambda_1\vartheta}\left[1+O_u\left(e^{-2(\lambda_1-\lambda_2)\vartheta}\right)\right]. ∥ u T e ϑC ∥ = λ 1 ∣ u T v 1 ∣ e λ 1 ϑ [ 1 + O u ( e − 2 ( λ 1 − λ 2 ) ϑ ) ] . The second equality follows by dividing by the first summand and using ∣ 1 + x − 1 ∣ ≤ x \lvert\sqrt{1+x}-1\rvert\le x ∣ 1 + x − 1 ∣ ≤ x x ≥ 0 x\ge0 x ≥ 0
s = 2 ∥ u T e ϑ C C ∥ 2 ∥ u T e ϑ C C ∥ + c η . s=\frac{2\lVert u^{\mathsf{T}}e^{\vartheta C}C\rVert}{2\lVert u^{\mathsf{T}}e^{\vartheta C}C\rVert+c_{\eta}}. s = 2 ∥ u T e ϑC C ∥ + c η 2 ∥ u T e ϑC C ∥ . As ϑ → ∞ \vartheta\to\infty ϑ → ∞ 1 − s = O u ( e − λ 1 ϑ ) 1-s=O_u(e^{-\lambda_1\vartheta}) 1 − s = O u ( e − λ 1 ϑ ) p K ( 1 ) = 1 p_K(1)=1 p K ( 1 ) = 1 p K ( s ) = 1 + O K , u ( e − λ 1 ϑ ) p_K(s)=1+O_{K,u}(e^{-\lambda_1\vartheta}) p K ( s ) = 1 + O K , u ( e − λ 1 ϑ )
d σ d ϑ = ∥ u T e ϑ C C ∥ p K ( s ) = λ 1 ∣ u T v 1 ∣ e λ 1 ϑ + O u ( e ( 2 λ 2 − λ 1 ) ϑ ) + O u ( 1 ) . \frac{\mathrm{d}\sigma}{\mathrm{d}\vartheta}=\frac{\lVert u^{\mathsf{T}}e^{\vartheta C}C\rVert}{p_K(s)}=\lambda_1\lvert u^{\mathsf{T}}v_1\rvert e^{\lambda_1\vartheta}+O_u(e^{(2\lambda_2-\lambda_1)\vartheta})+O_u(1). d ϑ d σ = p K ( s ) ∥ u T e ϑC C ∥ = λ 1 ∣ u T v 1 ∣ e λ 1 ϑ + O u ( e ( 2 λ 2 − λ 1 ) ϑ ) + O u ( 1 ) . This derivative is positive and continuous at finite ϑ \vartheta ϑ 0 < λ 2 < λ 1 0<\lambda_2<\lambda_1 0 < λ 2 < λ 1 O u ( e λ 2 ϑ ) O_u(e^{\lambda_2\vartheta}) O u ( e λ 2 ϑ )
σ = ∣ u T v 1 ∣ e λ 1 ϑ + O u ( e λ 2 ϑ ) . \sigma=\lvert u^{\mathsf{T}}v_1\rvert e^{\lambda_1\vartheta}+O_u(e^{\lambda_2\vartheta}). σ = ∣ u T v 1 ∣ e λ 1 ϑ + O u ( e λ 2 ϑ ) . In particular σ \sigma σ e λ 1 ϑ e^{\lambda_1\vartheta} e λ 1 ϑ
u T e ϑ C = ( u T v 1 ) e λ 1 ϑ v 1 T + O u ( e λ 2 ϑ ) . u^{\mathsf{T}}e^{\vartheta C}=(u^{\mathsf{T}}v_1)e^{\lambda_1\vartheta}v_1^{\mathsf{T}}+O_u(e^{\lambda_2\vartheta}). u T e ϑC = ( u T v 1 ) e λ 1 ϑ v 1 T + O u ( e λ 2 ϑ ) . Subtracting sign ( u T v 1 ) σ v 1 T \operatorname{sign}(u^{\mathsf{T}}v_1)\sigma v_1^{\mathsf{T}} sign ( u T v 1 ) σ v 1 T σ \sigma σ O u ( e − ( λ 1 − λ 2 ) ϑ ) = O u ( σ − κ ) O_u(e^{-(\lambda_1-\lambda_2)\vartheta})=O_u(\sigma^{-\kappa}) O u ( e − ( λ 1 − λ 2 ) ϑ ) = O u ( σ − κ ) σ 0 \sigma_0 σ 0 Q ε ( ε σ 0 ) → e 1 w ( σ 0 ) T / σ 0 Q^{\varepsilon}(\varepsilon\sigma_0)\to e_1w(\sigma_0)^{\mathsf{T}}/\sigma_0 Q ε ( ε σ 0 ) → e 1 w ( σ 0 ) T / σ 0 1 / σ 0 1/\sigma_0 1/ σ 0
Lemma 29 (Asymptotic independent growth coefficients). Under Assumption 2, for every fixed u u u u T v 1 ≠ 0 , u^{\mathsf{T}}v_1\ne0, u T v 1 = 0 ,
σ B ( σ ) = a K λ 1 diag ( λ 2 , … , λ r ) + O u ( σ − κ ) . (61) \sigma B(\sigma)=\frac{a_K}{\lambda_1}\operatorname{diag}(\lambda_2,\ldots,\lambda_r)+O_u(\sigma^{-\kappa}). \tag*{(61)} σ B ( σ ) = λ 1 a K diag ( λ 2 , … , λ r ) + O u ( σ − κ ) . (61) In the logarithmic initial-layer time μ \mu μ
d d μ [ e − μ ζ a ( e μ ) ] = e − μ ζ a ( e μ ) [ diag 2 ≤ l ≤ r ( g l ( 1 ) ) + E u ( μ ) ] , \frac{\mathrm{d}}{\mathrm{d}\mu}\left[e^{-\mu}\zeta_a(e^\mu)\right]
=e^{-\mu}\zeta_a(e^\mu)\left[\operatorname{diag}_{2\le l\le r}\left(g_l^{(1)}\right)+E_u(\mu)\right], d μ d [ e − μ ζ a ( e μ ) ] = e − μ ζ a ( e μ ) [ diag 2 ≤ l ≤ r ( g l ( 1 ) ) + E u ( μ ) ] , g l ( 1 ) = a K λ l λ 1 − 1. (62) g_l^{(1)}=\frac{a_K\lambda_l}{\lambda_1}-1. \tag*{(62)} g l ( 1 ) = λ 1 a K λ l − 1. (62) where E u E_u E u u u u ∥ E u ( μ ) ∥ o p = O u ( e − κ μ ) \lVert E_u(\mu)\rVert_{\mathrm{op}}=O_u(e^{-\kappa\mu}) ∥ E u ( μ ) ∥ op = O u ( e − κ μ ) μ \mu μ
Proof (of Lemma 29). Divide the numerator and denominator in (50) by the appropriate powers of σ \sigma σ
σ A ( σ ) = 2 C 2 ∥ w ( σ ) T C / σ ∥ + c η / σ q K ( 4 C [ w ( σ ) / σ ] [ w ( σ ) / σ ] T C ( 2 ∥ w ( σ ) T C / σ ∥ + c η / σ ) 2 ) . \sigma A(\sigma)=\frac{2C}{2\lVert w(\sigma)^{\mathsf{T}}C/\sigma\rVert+c_{\eta}/\sigma}\,q_K\left(\frac{4C[w(\sigma)/\sigma][w(\sigma)/\sigma]^{\mathsf{T}}C}{(2\lVert w(\sigma)^{\mathsf{T}}C/\sigma\rVert+c_{\eta}/\sigma)^2}\right). σ A ( σ ) = 2 ∥ w ( σ ) T C / σ ∥ + c η / σ 2 C q K ( ( 2 ∥ w ( σ ) T C / σ ∥ + c η / σ ) 2 4 C [ w ( σ ) / σ ] [ w ( σ ) / σ ] T C ) . By Lemma 28, its row argument differs from sign ( u T v 1 ) v 1 \operatorname{sign}(u^{\mathsf{T}}v_1)v_1 sign ( u T v 1 ) v 1 O u ( σ − κ ) O_u(\sigma^{-\kappa}) O u ( σ − κ ) 2 λ 1 2\lambda_1 2 λ 1 σ − 1 = O ( σ − κ ) \sigma^{-1}=O(\sigma^{-\kappa}) σ − 1 = O ( σ − κ ) 0 < κ < 1 0<\kappa<1 0 < κ < 1 X j − Y j = ∑ i = 0 j − 1 X i ( X − Y ) Y j − 1 − i X^j-Y^j=\sum_{i=0}^{j-1}X^i(X-Y)Y^{j-1-i} X j − Y j = ∑ i = 0 j − 1 X i ( X − Y ) Y j − 1 − i q K q_K q K
σ A ( σ ) = C λ 1 q K ( v 1 v 1 T ) + O u ( σ − κ ) = C λ 1 [ a K I + ( 1 − a K ) v 1 v 1 T ] + O u ( σ − κ ) . \sigma_A(\sigma)=\frac{C}{\lambda_1}q_K(v_1v_1^{\mathsf T})+O_u(\sigma^{-\kappa})=\frac{C}{\lambda_1}\left[a_KI+(1-a_K)v_1v_1^{\mathsf T}\right]+O_u(\sigma^{-\kappa}). σ A ( σ ) = λ 1 C q K ( v 1 v 1 T ) + O u ( σ − κ ) = λ 1 C [ a K I + ( 1 − a K ) v 1 v 1 T ] + O u ( σ − κ ) . In the last equality a projection has eigenvalues zero and one, q K ( 0 ) = a K q_K(0)=a_K q K ( 0 ) = a K q K ( 1 ) = 1 q_K(1)=1 q K ( 1 ) = 1 V ⊥ V_\perp V ⊥ V ⊥ V_\perp V ⊥ v 1 v_1 v 1 σ V ⊥ T A ( σ ) v 1 = O u ( σ − κ ) \sigma V_\perp^{\mathsf T}A(\sigma)v_1=O_u(\sigma^{-\kappa}) σ V ⊥ T A ( σ ) v 1 = O u ( σ − κ )
w ( σ ) T V ⊥ w ( σ ) T v 1 = O u ( σ − κ ) . \frac{w(\sigma)^{\mathsf T}V_\perp}{w(\sigma)^{\mathsf T}v_1}=O_u(\sigma^{-\kappa}). w ( σ ) T v 1 w ( σ ) T V ⊥ = O u ( σ − κ ) . Thus the subtracted term in σ B \sigma B σ B O u ( σ − 2 κ ) O_u(\sigma^{-2\kappa}) O u ( σ − 2 κ ) σ B \sigma B σ B μ = log σ \mu=\log\sigma μ = log σ
d d μ ( e − μ ζ a ( e μ ) ) = e − μ ζ a ( e μ ) [ e μ B ( e μ ) − I ] . \frac{\mathrm{d}}{\mathrm{d}\mu}\left(e^{-\mu}\zeta_a(e^\mu)\right)=e^{-\mu}\zeta_a(e^\mu)\left[e^\mu B(e^\mu)-I\right]. d μ d ( e − μ ζ a ( e μ ) ) = e − μ ζ a ( e μ ) [ e μ B ( e μ ) − I ] . Substitution of the coefficient estimate proves the normalized equation with the diagonal entries a K λ l / λ 1 − 1 a_K\lambda_l/\lambda_1-1 a K λ l / λ 1 − 1 O u ( e − κ μ ) O_u(e^{-\kappa\mu}) O u ( e − κ μ )
The coefficient error is integrable on every sufficiently late half-line in μ \mu μ σ \sigma σ
Asymptotic coefficients, minors, and first entry
The integrable error in the normalized independent-row equation permits a measurable asymptotic basis. Its Gaussian coefficients are fixed before choosing the seed-dependent matching time. Exterior products then identify all retained independent scales at that time.
Lemma 30 (Asymptotic basis of the independent-row equation) Under Assumption 2, fix u u u u T v 1 ≠ 0 u^{\mathsf T}v_1\ne0 u T v 1 = 0 F u ( μ ) ∈ R ( r − 1 ) × ( r − 1 ) F_u(\mu)\in\mathbb{R}^{(r-1)\times(r-1)} F u ( μ ) ∈ R ( r − 1 ) × ( r − 1 )
F u ( μ ) = diag 2 ≤ l ≤ r ( e g l ( 1 ) μ ) [ I r − 1 + o u ( 1 ) ] ( μ → ∞ ) . (63) F_u(\mu)=\operatorname{diag}_{2\le l\le r}(e^{g_l^{(1)}\mu})[I_{r-1}+o_u(1)]\qquad(\mu\to\infty). \tag*{(63)} F u ( μ ) = diag 2 ≤ l ≤ r ( e g l ( 1 ) μ ) [ I r − 1 + o u ( 1 )] ( μ → ∞ ) . (63) It can be chosen measurably as a function of u u u μ \mu μ Ξ ∈ R ( m − 1 ) × ( r − 1 ) \Xi\in\mathbb{R}^{(m-1)\times(r-1)} Ξ ∈ R ( m − 1 ) × ( r − 1 )
Ξ a − 1 , : = ζ a ( 1 ) F u ( 0 ) − 1 , e − μ ζ a ( e μ ) = Ξ a − 1 , : F u ( μ ) , 2 ≤ a ≤ m . (64) \Xi_{a-1,:}=\zeta_a(1)F_u(0)^{-1},\qquad e^{-\mu}\zeta_a(e^\mu)=\Xi_{a-1,:}F_u(\mu),\quad2\le a\le m. \tag*{(64)} Ξ a − 1 , : = ζ a ( 1 ) F u ( 0 ) − 1 , e − μ ζ a ( e μ ) = Ξ a − 1 , : F u ( μ ) , 2 ≤ a ≤ m . (64) Conditional on u u u Ξ \Xi Ξ
F u ( 0 ) − T Ψ ( 1 ) T [ I r − 1 + ( V ⊥ T u ) ( V ⊥ T u ) T ( u T v 1 ) 2 ] Ψ ( 1 ) F u ( 0 ) − 1 ≻ 0. F_u(0)^{-\mathsf T}\Psi(1)^{\mathsf T}\left[I_{r-1}+\frac{(V_\perp^{\mathsf T}u)(V_\perp^{\mathsf T}u)^{\mathsf T}}{(u^{\mathsf T}v_1)^2}\right]\Psi(1)F_u(0)^{-1}\succ0. F u ( 0 ) − T Ψ ( 1 ) T [ I r − 1 + ( u T v 1 ) 2 ( V ⊥ T u ) ( V ⊥ T u ) T ] Ψ ( 1 ) F u ( 0 ) − 1 ≻ 0. Proof (of Lemma 30). Choose μ b \mu_b μ b
r − 1 ∫ μ b ∞ ∥ E u ( s ) ∥ o p d s < 1 2 . \sqrt{r-1}\int_{\mu_b}^{\infty}\lVert E_u(s)\rVert_{\mathrm{op}}\,\mathrm{d}s < \frac{1}{2}. r − 1 ∫ μ b ∞ ∥ E u ( s ) ∥ op d s < 2 1 . For each task label l l l x x x [ μ b , ∞ ) [\mu_b,\infty) [ μ b , ∞ ) k k k μ \mu μ
{ ∫ μ b μ e ( g k ( 1 ) − g l ( 1 ) ) ( μ − s ) [ ( e l T + x ( s ) ) E u ( s ) ] k d s , g k ( 1 ) < g l ( 1 ) , − ∫ μ ∞ e ( g k ( 1 ) − g l ( 1 ) ) ( μ − s ) [ ( e l T + x ( s ) ) E u ( s ) ] k d s , g k ( 1 ) ≥ g l ( 1 ) . \begin{cases}
\int_{\mu_b}^{\mu}e^{(g_k^{(1)}-g_l^{(1)})(\mu-s)}\left[(e_l^{\mathsf T}+x(s))E_u(s)\right]_k\,\mathrm{d}s, & g_k^{(1)}<g_l^{(1)},\\[6pt]
-\int_{\mu}^{\infty}e^{(g_k^{(1)}-g_l^{(1)})(\mu-s)}\left[(e_l^{\mathsf T}+x(s))E_u(s)\right]_k\,\mathrm{d}s, & g_k^{(1)}\ge g_l^{(1)}.
\end{cases} ⎩ ⎨ ⎧ ∫ μ b μ e ( g k ( 1 ) − g l ( 1 ) ) ( μ − s ) [ ( e l T + x ( s )) E u ( s ) ] k d s , − ∫ μ ∞ e ( g k ( 1 ) − g l ( 1 ) ) ( μ − s ) [ ( e l T + x ( s )) E u ( s ) ] k d s , g k ( 1 ) < g l ( 1 ) , g k ( 1 ) ≥ g l ( 1 ) . Here x x x r − 1 r-1 r − 1 r − 1 \sqrt{r-1} r − 1 [31 Theorem 2.1] gives a unique fixed point. Define row l l l F u ( μ ) F_u(\mu) F u ( μ ) e l T e_l^{\mathsf T} e l T e g l ( 1 ) μ e^{g_l^{(1)}\mu} e g l ( 1 ) μ
The normalized correction e − g l ( 1 ) μ ( F u ( μ ) ) l , : − e l T e^{-g_l^{(1)}\mu}(F_u(\mu))_{l,:}-e_l^{\mathsf T} e − g l ( 1 ) μ ( F u ( μ ) ) l , : − e l T g k ( 1 ) ≥ g l ( 1 ) g_k^{(1)}\ge g_l^{(1)} g k ( 1 ) ≥ g l ( 1 ) k k k
( 1 + sup ν ≥ μ b ∥ e − g l ( 1 ) ν ( F u ( ν ) ) l , : − e l T ∥ ) ∫ μ ∞ ∥ E u ( s ) ∥ o p d s . \left(1+\sup_{\nu\ge\mu_b}\left\lVert e^{-g_l^{(1)}\nu}(F_u(\nu))_{l,:}-e_l^{\mathsf T}\right\rVert\right)\int_{\mu}^{\infty}\lVert E_u(s)\rVert_{\mathrm{op}}\,\mathrm{d}s. ( 1 + ν ≥ μ b sup e − g l ( 1 ) ν ( F u ( ν ) ) l , : − e l T ) ∫ μ ∞ ∥ E u ( s ) ∥ op d s . which tends to zero. For g k ( 1 ) < g l ( 1 ) g_k^{(1)}<g_l^{(1)} g k ( 1 ) < g l ( 1 ) ( μ + μ b ) / 2 (\mu+\mu_b)/2 ( μ + μ b ) /2
e ( g k ( 1 ) − g l ( 1 ) ) ( μ − μ b ) / 2 ∫ μ b ∞ ∥ E u ( s ) ∥ o p d s + ∫ ( μ + μ b ) / 2 ∞ ∥ E u ( s ) ∥ o p d s . e^{(g_k^{(1)}-g_l^{(1)})(\mu-\mu_b)/2}\int_{\mu_b}^{\infty}\lVert E_u(s)\rVert_{\mathrm{op}}\,\mathrm{d}s+\int_{(\mu+\mu_b)/2}^{\infty}\lVert E_u(s)\rVert_{\mathrm{op}}\,\mathrm{d}s. e ( g k ( 1 ) − g l ( 1 ) ) ( μ − μ b ) /2 ∫ μ b ∞ ∥ E u ( s ) ∥ op d s + ∫ ( μ + μ b ) /2 ∞ ∥ E u ( s ) ∥ op d s . Both terms tend to zero. The normalized rows of F u F_u F u F u ′ = F u [ diag 2 ≤ l ≤ r ( g l ( 1 ) ) + E u ] F_u'=F_u[\operatorname{diag}_{2\le l\le r}(g_l^{(1)})+E_u] F u ′ = F u [ diag 2 ≤ l ≤ r ( g l ( 1 ) ) + E u ]
The tail integral is a measurable limit of finite integrals in u u u μ b \mu_b μ b F u F_u F u
The time μ = 0 \mu=0 μ = 0 σ = 1 \sigma=1 σ = 1 e − μ ζ a ( e μ ) e^{-\mu\zeta_a(e^\mu)} e − μ ζ a ( e μ ) ζ a ( 1 ) F u ( 0 ) − 1 F u ( μ ) \zeta_a(1)F_u(0)^{-1}F_u(\mu) ζ a ( 1 ) F u ( 0 ) − 1 F u ( μ )
e − μ ζ a ( e μ ) = Ξ a − 1 , : F u ( μ ) , Ξ a − 1 , : = ζ a ( 1 ) F u ( 0 ) − 1 . e^{-\mu}\zeta_a(e^\mu)=\Xi_{a-1,:}F_u(\mu),\qquad\Xi_{a-1,:}=\zeta_a(1)F_u(0)^{-1}. e − μ ζ a ( e μ ) = Ξ a − 1 , : F u ( μ ) , Ξ a − 1 , : = ζ a ( 1 ) F u ( 0 ) − 1 . By Lemma 27, ζ a ( 1 ) = ζ a ( 0 ) Ψ ( 1 ) \zeta_a(1)=\zeta_a(0)\Psi(1) ζ a ( 1 ) = ζ a ( 0 ) Ψ ( 1 ) u u u
F u ( 0 ) − T Ψ ( 1 ) T [ I r − 1 + ( V ⊥ T u ) ( V ⊥ T u ) T ( u T v 1 ) 2 ] Ψ ( 1 ) F u ( 0 ) − 1 ≻ 0. F_u(0)^{-\mathsf T}\Psi(1)^{\mathsf T}\left[I_{r-1}+\frac{(V_{\perp}^{\mathsf T}u)(V_{\perp}^{\mathsf T}u)^{\mathsf T}}{(u^{\mathsf T}v_1)^2}\right]\Psi(1)F_u(0)^{-1}\succ0. F u ( 0 ) − T Ψ ( 1 ) T [ I r − 1 + ( u T v 1 ) 2 ( V ⊥ T u ) ( V ⊥ T u ) T ] Ψ ( 1 ) F u ( 0 ) − 1 ≻ 0. The bracket has strictly positive quadratic form on nonzero vectors, and both multiplying matrices are invertible and measurable in u u u ■ \blacksquare ■
For a matrix Z = ( z 2 , … , z r ) ∈ R ( m − 1 ) × ( r − 1 ) Z=(z_2,\ldots,z_r)\in\mathbb{R}^{(m-1)\times(r-1)} Z = ( z 2 , … , z r ) ∈ R ( m − 1 ) × ( r − 1 )
V h ( Z ) = ∥ z 2 ∧ ⋯ ∧ z h + 1 ∥ , V 0 ( Z ) = 1. V_h(Z)=\lVert z_2\wedge\cdots\wedge z_{h+1}\rVert,\qquad V_0(Z)=1. V h ( Z ) = ∥ z 2 ∧ ⋯ ∧ z h + 1 ∥ , V 0 ( Z ) = 1. The induced matrix ∧ h Z \wedge^h Z ∧ h Z h h h Z Z Z ζ a ( σ ) / σ \zeta_a(\sigma)/\sigma ζ a ( σ ) / σ 2 ≤ a ≤ m 2\leq a\leq m 2 ≤ a ≤ m Ξ F u ( log σ ) \Xi F_u(\log\sigma) Ξ F u ( log σ ) h h h J J J ( ∧ h Ξ ) J (\wedge^h\Xi)_J ( ∧ h Ξ ) J
Lemma 31 (Nonzero Gaussian exterior coefficients). Under Assumption 2, there is a measurable Gaussian-full event on which u ⊤ v 1 ≠ 0 u^\top v_1\ne0 u ⊤ v 1 = 0 ( ∧ h Ξ ) J (\wedge^h\Xi)_J ( ∧ h Ξ ) J 1 ≤ h ≤ min { m − 1 , r − 1 } 1\leq h\leq\min\{m-1,r-1\} 1 ≤ h ≤ min { m − 1 , r − 1 } h h h J ⊂ { 2 , … , r } J\subset\{2,\ldots,r\} J ⊂ { 2 , … , r } 1 ≤ h ≤ p − 1 1\leq h\leq p-1 1 ≤ h ≤ p − 1 J h = { 2 , … , h + 1 } J_h=\{2,\ldots,h+1\} J h = { 2 , … , h + 1 }
e − μ ∑ l = 2 h + 1 g l ( 1 ) ∧ h [ Ξ F u ( μ ) ] ⟶ ( ∧ h Ξ ) J h e J h T . (65) e^{-\mu\sum_{l=2}^{h+1}g_l^{(1)}}\wedge^h[\Xi F_u(\mu)]\longrightarrow(\wedge^h\Xi)_{J_h}e_{J_h}^{\mathsf{T}}. \tag*{(65)} e − μ ∑ l = 2 h + 1 g l ( 1 ) ∧ h [ Ξ F u ( μ )] ⟶ ( ∧ h Ξ ) J h e J h T . (65) Here e J h e_{J_h} e J h V h ( Ξ F u ( log σ ) ) > 0 V_h(\Xi F_u(\log\sigma))>0 V h ( Ξ F u ( log σ )) > 0 σ \sigma σ
∥ ∧ h Ξ F u ( log σ ) ∥ F 2 V h ( Ξ F u ( log σ ) ) 2 ⟶ 1. \frac{\lVert\wedge^h\Xi F_u(\log\sigma)\rVert_F^2}{V_h(\Xi F_u(\log\sigma))^2}\longrightarrow1. V h ( Ξ F u ( log σ ) ) 2 ∥ ∧ h Ξ F u ( log σ ) ∥ F 2 ⟶ 1. For h = r − 1 h=r-1 h = r − 1
Proof (of Lemma 31). Conditional on u u u Ξ \Xi Ξ h h h J J J ( ∧ h Ξ ) J (\wedge^h\Xi)_J ( ∧ h Ξ ) J h h h J J J [27 ] makes its zero set Lebesgue null. There are finitely many relevant pairs ( h , J ) (h,J) ( h , J ) u ⊤ v 1 = 0 u^\top v_1=0 u ⊤ v 1 = 0
Assume the separated-prefix hypothesis and fix 1 ≤ h ≤ p − 1 1\leq h\leq p-1 1 ≤ h ≤ p − 1
∧ h [ Ξ F u ( μ ) ] = ( ∧ h Ξ ) diag ∣ J ∣ = h ( e μ ∑ l ∈ J g l ( 1 ) ) [ I + o u ( 1 ) ] . \wedge^h[\Xi F_u(\mu)]=(\wedge^h\Xi)\operatorname{diag}_{\lvert J\rvert=h}\left(e^{\mu\sum_{l\in J}g_l^{(1)}}\right)[I+o_u(1)]. ∧ h [ Ξ F u ( μ )] = ( ∧ h Ξ ) diag ∣ J ∣ = h ( e μ ∑ l ∈ J g l ( 1 ) ) [ I + o u ( 1 )] . The last factor tends to the identity because fixed exterior powers are polynomial in the entries. The set J h = { 2 , … , h + 1 } J_h=\{2,\ldots,h+1\} J h = { 2 , … , h + 1 } h < r − 1 h<r-1 h < r − 1 g h + 1 ( 1 ) g_{h+1}^{(1)} g h + 1 ( 1 ) g h + 2 ( 1 ) g_{h+2}^{(1)} g h + 2 ( 1 ) g h + 1 ( 1 ) − g h + 2 ( 1 ) g_{h+1}^{(1)}-g_{h+2}^{(1)} g h + 1 ( 1 ) − g h + 2 ( 1 )
e − μ ∑ l = 2 h + 1 g l ( 1 ) ∧ h [ Ξ F u ( μ ) ] ⟶ ( ∧ h Ξ ) J h e J h T . e^{-\mu\sum_{l=2}^{h+1}g_l^{(1)}}\wedge^h[\Xi F_u(\mu)]\longrightarrow(\wedge^h\Xi)_{J_h}e_{J_h}^{\mathsf{T}}. e − μ ∑ l = 2 h + 1 g l ( 1 ) ∧ h [ Ξ F u ( μ )] ⟶ ( ∧ h Ξ ) J h e J h T . This is (65) . Its leading column is nonzero on the established event, while every competing column is negligible relative to it. Thus the leading volume is eventually positive and the squared full-exterior to leading-volume ratio tends to one. For h = r − 1 h=r-1 h = r − 1
For a matrix Q Q Q Q v 1 Qv_1 Q v 1 Ran ( Q v 1 ) ⊥ \operatorname{Ran}(Qv_1)^\perp Ran ( Q v 1 ) ⊥ Z ( Q ) Z(Q) Z ( Q ) Z ε ( t ) = Z ( Q ε ( t ) ) Z^\varepsilon(t)=Z(Q^\varepsilon(t)) Z ε ( t ) = Z ( Q ε ( t ))
Lemma 32 (Gaussian independent scales at first entry) Under Assumption 2 , for every fixed seed with u ⊤ v 1 ≠ 0 u^\top v_1\ne0 u ⊤ v 1 = 0 σ > 0 \sigma>0 σ > 0
∥ Z ε ( ε σ ) ∥ F = O U ( ε b − 1 ) . (66) \lVert Z^\varepsilon(\varepsilon\sigma)\rVert_F=O_U(\varepsilon^{b-1}). \tag*{(66)} ∥ Z ε ( ε σ ) ∥ F = O U ( ε b − 1 ) . (66) Under Assumption 5, on the Gaussian-full event of Lemma 31 , for every ζ > 0 \zeta>0 ζ > 0 σ 0 ( U ) \sigma_0(U) σ 0 ( U ) ε > 0 \varepsilon>0 ε > 0 1 ≤ h ≤ p − 1 1\le h\le p-1 1 ≤ h ≤ p − 1
log V h ( Z ε ( ε σ 0 ) ) = h ( b − 1 ) log ε + O U ( 1 ) , (67) \log V_h(Z^\varepsilon(\varepsilon\sigma_0))=h(b-1)\log\varepsilon+O_U(1), \tag*{(67)} log V h ( Z ε ( ε σ 0 )) = h ( b − 1 ) log ε + O U ( 1 ) , (67) ∥ ∧ h Z ε ( ε σ 0 ) ∥ F 2 ≤ ( 1 + ζ 2 ) V h ( Z ε ( ε σ 0 ) ) 2 . (68) \lVert\wedge^h Z^\varepsilon(\varepsilon\sigma_0)\rVert_F^2\le(1+\zeta^2)V_h(Z^\varepsilon(\varepsilon\sigma_0))^2. \tag*{(68)} ∥ ∧ h Z ε ( ε σ 0 ) ∥ F 2 ≤ ( 1 + ζ 2 ) V h ( Z ε ( ε σ 0 ) ) 2 . (68) The same σ 0 ( U ) \sigma_0(U) σ 0 ( U ) Q ε ( ε σ 0 ) Q^\varepsilon(\varepsilon\sigma_0) Q ε ( ε σ 0 ) { O e 1 v 1 ⊤ : O ∈ O ( m ) } \{Oe_1v_1^\top:O\in\mathrm{O}(m)\} { O e 1 v 1 ⊤ : O ∈ O ( m )} 1 / σ 0 1/\sigma_0 1/ σ 0 ε ↓ 0 \varepsilon\downarrow0 ε ↓ 0
Proof (of Lemma 32). Use ordinary entries and submatrices in task coordinates. When Q 11 ≠ 0 Q_{11}\ne0 Q 11 = 0
( − Q 2 : m , 1 ⊤ / Q 11 I m − 1 ) ( I m − 1 + Q 2 : m , 1 Q 2 : m , 1 ⊤ Q 11 2 ) − 1 / 2 \begin{pmatrix}
-Q_{2:m,1}^\top/Q_{11}\\
I_{m-1}
\end{pmatrix}
\left(I_{m-1}+\frac{Q_{2:m,1}Q_{2:m,1}^\top}{Q_{11}^2}\right)^{-1/2} ( − Q 2 : m , 1 ⊤ / Q 11 I m − 1 ) ( I m − 1 + Q 11 2 Q 2 : m , 1 Q 2 : m , 1 ⊤ ) − 1/2 form an orthonormal frame perpendicular to the first column of Q Q Q
Z ( Q ) = ( I m − 1 + Q 2 : m , 1 Q 2 : m , 1 ⊤ Q 11 2 ) − 1 / 2 ( Q 2 : m , 2 : r − Q 2 : m , 1 Q 1 , 2 : r Q 11 ) . Z(Q)=\left(I_{m-1}+\frac{Q_{2:m,1}Q_{2:m,1}^\top}{Q_{11}^2}\right)^{-1/2}\left(Q_{2:m,2:r}-\frac{Q_{2:m,1}Q_{1,2:r}}{Q_{11}}\right). Z ( Q ) = ( I m − 1 + Q 11 2 Q 2 : m , 1 Q 2 : m , 1 ⊤ ) − 1/2 ( Q 2 : m , 2 : r − Q 11 Q 2 : m , 1 Q 1 , 2 : r ) . For Q = Y ε ( σ ) / σ Q=Y^\varepsilon(\sigma)/\sigma Q = Y ε ( σ ) / σ σ > 0 \sigma>0 σ > 0 Q 2 : m , 1 / Q 11 = O U ( ε b − 1 ) Q_{2:m,1}/Q_{11}=O_U(\varepsilon^{b-1}) Q 2 : m , 1 / Q 11 = O U ( ε b − 1 ) O U ( ε 2 ( b − 1 ) ) O_U(\varepsilon^{2(b-1)}) O U ( ε 2 ( b − 1 ) ) 1 / σ 1/\sigma 1/ σ
Z ε ( ε σ ) = ε b − 1 [ Ξ F u ( log σ ) + O U , σ ( ε 2 ( b − 1 ) + ε 2 ) ] . Z^\varepsilon(\varepsilon\sigma)=\varepsilon^{b-1}\left[\Xi F_u(\log\sigma)+O_{U,\sigma}\left(\varepsilon^{2(b-1)}+\varepsilon^2\right)\right]. Z ε ( ε σ ) = ε b − 1 [ Ξ F u ( log σ ) + O U , σ ( ε 2 ( b − 1 ) + ε 2 ) ] . e n d s m i d = 1 {ends_mid=1} e n d s m i d = 1 This also proves the fixed-time norm estimate.
Fix a seed in the Gaussian-full event of Lemma 31 . There are finitely many retained orders. Choose a sufficiently large finite σ 0 \sigma_0 σ 0 Ξ F u ( log σ 0 ) \Xi_{F_u}(\log\sigma_0) Ξ F u ( log σ 0 ) 1 + ζ 2 1+\zeta^2 1 + ζ 2 1 / σ 0 1/\sigma_0 1/ σ 0 ε ↓ 0 \varepsilon\downarrow0 ε ↓ 0
ε − h ( b − 1 ) ⋀ h Z ε ( ε σ 0 ) ⟶ ⋀ h [ Ξ F u ( log σ 0 ) ] . \varepsilon^{-h(b-1)}\bigwedge^{h} Z^{\varepsilon}(\varepsilon\sigma_0) \longrightarrow\bigwedge^{h}\left[\Xi_{F_u}(\log\sigma_0)\right]. ε − h ( b − 1 ) ⋀ h Z ε ( ε σ 0 ) ⟶ ⋀ h [ Ξ F u ( log σ 0 ) ] . The leading-column norm on the right is positive and finite. Its normalized counterpart is bounded above and below by positive seed-dependent constants, giving
log V h ( Z ε ( ε σ 0 ) ) = h ( b − 1 ) log ε + O U ( 1 ) . \log V_h\left(Z^{\varepsilon}(\varepsilon\sigma_0)\right)=h(b-1)\log\varepsilon+\mathcal{O}_U(1). log V h ( Z ε ( ε σ 0 ) ) = h ( b − 1 ) log ε + O U ( 1 ) . The strict ratio margins pass to the actual matrix for sufficiently small ε \varepsilon ε
The conclusions concern finitely many nonzero volumes at a fixed initial-layer time. The next sections prove their relative propagation over the much longer actual time intervals on which independent components reach a fixed size.
Shapes, full linearization, and connections
The local calculations below apply to a symmetric nonsingular task matrix in arbitrary positive factor dimensions. They establish the shape equations used in the main recursion, the contracting tangent blocks, and a connecting solution between successive leading ranks.
Lemma 33 (Gram matrices at zeros of F K F_K F K Every nonzero Q Q Q F K ( Q ) = 0 F_K(Q)=0 F K ( Q ) = 0
C Ran ( Q T ) = Ran ( Q T ) , z T C z > 0 for z ∈ Ran ( Q T ) ∖ { 0 } . C\operatorname{Ran}(Q^{\mathsf T})=\operatorname{Ran}(Q^{\mathsf T}),\qquad z^{\mathsf T}Cz>0\quad\text{for }z\in\operatorname{Ran}(Q^{\mathsf T})\setminus\{0\}. C Ran ( Q T ) = Ran ( Q T ) , z T C z > 0 for z ∈ Ran ( Q T ) ∖ { 0 } . For K ≥ 1 K\geq1 K ≥ 1 Ran ( Q T ) \operatorname{Ran}(Q^{\mathsf T}) Ran ( Q T ) Q T Q Q^{\mathsf T}Q Q T Q Q = Q I , K Q=Q_{I,K} Q = Q I , K q l = ∥ Q I , K v l ∥ q_l=\lVert Q_{I,K}v_l\rVert q l = ∥ Q I , K v l ∥ l ∈ I l\in I l ∈ I (14) . Conversely, for K ≥ 1 K\geq1 K ≥ 1 ∣ I ∣ ≤ m |I|\leq m ∣ I ∣ ≤ m λ l > 0 \lambda_l>0 λ l > 0 I I I q l q_l q l s I , K s_{I,K} s I , K
At rank one, Q = u v T Q=uv^{\mathsf T} Q = u v T u , v u,v u , v C v = λ v Cv=\lambda v C v = λ v λ > 0 \lambda>0 λ > 0 ∥ Q C ∥ F = λ \lVert QC\rVert_F=\lambda ∥ QC ∥ F = λ
At K = 0 K=0 K = 0 C C C Q 1 , K = e 1 v 1 T Q_{1,K}=e_1v_1^{\mathsf T} Q 1 , K = e 1 v 1 T K K K s 1 , K = λ 1 s_{1,K}=\lambda_1 s 1 , K = λ 1
Proof (of Lemma 33). Since Q ≠ 0 Q\ne0 Q = 0 C C C ∥ Q C ∥ F > 0 \lVert QC\rVert_F>0 ∥ QC ∥ F > 0
Q C ∥ Q C ∥ F = ∑ a = 1 d σ a u a z a T , σ a > 0 , ∑ a = 1 d σ a 2 = 1. \frac{QC}{\lVert QC\rVert_F}=\sum_{a=1}^{d}\sigma_a u_a z_a^{\mathsf T},\qquad\sigma_a>0,\qquad\sum_{a=1}^{d}\sigma_a^2=1. ∥ QC ∥ F QC = a = 1 ∑ d σ a u a z a T , σ a > 0 , a = 1 ∑ d σ a 2 = 1. The equation F K ( Q ) = 0 F_K(Q)=0 F K ( Q ) = 0 Q = ∑ a p K ( σ a ) u a z a T Q=\sum_a p_K(\sigma_a)u_a z_a^{\mathsf{T}} Q = ∑ a p K ( σ a ) u a z a T C C C u a u_a u a
p K ( σ a ) z ˉ a T C = ∥ Q C ∥ F σ a z ˉ a T , C z a = ∥ Q C ∥ F σ a p K ( σ a ) z a . p_K(\sigma_a)\bar{z}_a^{\mathsf{T}}C=\lVert QC\rVert_F\sigma_a\bar{z}_a^{\mathsf{T}},\qquad Cz_a=\frac{\lVert QC\rVert_F\sigma_a}{p_K(\sigma_a)}z_a. p K ( σ a ) z ˉ a T C = ∥ QC ∥ F σ a z ˉ a T , C z a = p K ( σ a ) ∥ QC ∥ F σ a z a . The displayed eigenvalue is positive. These vectors span Ran ( Q T ) \operatorname{Ran}(Q^{\mathsf{T}}) Ran ( Q T )
For K ≥ 1 K\geq1 K ≥ 1 q K q_K q K a K a_K a K [ 0 , 1 ] [0,1] [ 0 , 1 ] λ \lambda λ
σ 2 = q K − 1 ( ∥ Q C ∥ F / λ ) . \sigma^2=q_K^{-1}(\lVert QC\rVert_F/\lambda). σ 2 = q K − 1 (∥ QC ∥ F / λ ) . Each such value strictly decreases as ∥ Q C ∥ F \lVert QC\rVert_F ∥ QC ∥ F ∥ Q C ∥ F \lVert QC\rVert_F ∥ QC ∥ F
For the prescribed row space indexed by I I I v l v_l v l q l = p K ( σ l ) = ∥ Q I , K v l ∥ q_l=p_K(\sigma_l)=\lVert Q_{I,K}v_l\rVert q l = p K ( σ l ) = ∥ Q I , K v l ∥ Q I , K C Q_{I,K}C Q I , K C
λ l q l = s I , K σ l , ∑ l ∈ I σ l 2 = 1. \lambda_lq_l=s_{I,K}\sigma_l,\qquad\sum_{l\in I}\sigma_l^2=1. λ l q l = s I , K σ l , l ∈ I ∑ σ l 2 = 1. Substitution proves all three equations in (14). For later scalar calculations, they imply
∑ l ∈ I q K − 1 ( s I , K / λ l ) = 1 , max l ∈ I λ l ≤ s I , K < a K min l ∈ I λ l \sum_{l\in I}q_K^{-1}(s_{I,K}/\lambda_l)=1,\qquad\max_{l\in I}\lambda_l\leq s_{I,K}<a_K\min_{l\in I}\lambda_l l ∈ I ∑ q K − 1 ( s I , K / λ l ) = 1 , l ∈ I max λ l ≤ s I , K < a K l ∈ I min λ l when all normalized components are positive. At rank one s I , K s_{I,K} s I , K
Conversely, given a positive solution of (14), choose orthonormal u l ∈ R m u_l\in\mathbb{R}^m u l ∈ R m Q = ∑ l ∈ I q l u l v l T Q=\sum_{l\in I}q_lu_lv_l^{\mathsf{T}} Q = ∑ l ∈ I q l u l v l T λ l q l / s I , K > 0 \lambda_lq_l/s_{I,K}>0 λ l q l / s I , K > 0 P K P_K P K q l q_l q l F K ( Q ) = 0 F_K(Q)=0 F K ( Q ) = 0
At rank one the normalized singular value is one. Since p K ( 1 ) = 1 p_K(1)=1 p K ( 1 ) = 1 Q Q Q λ > 0 \lambda>0 λ > 0 K = 0 K=0 K = 0 Q C / ∥ Q C ∥ F = Q QC/\lVert QC\rVert_F=Q QC / ∥ QC ∥ F = Q Q C = s Q QC=sQ QC = s Q ∥ Q C ∥ F > 0 \lVert QC\rVert_F>0 ∥ QC ∥ F > 0 ∥ Q C ∥ F = ∥ Q C ∥ F = ∥ Q C ∥ F ∥ Q ∥ F \lVert QC\rVert_F=\lVert QC\rVert_F=\lVert QC\rVert_F\lVert Q\rVert_F ∥ QC ∥ F = ∥ QC ∥ F = ∥ QC ∥ F ∥ Q ∥ F ∥ Q ∥ F = 1 \lVert Q\rVert_F=1 ∥ Q ∥ F = 1 F 0 ( Q ) = 0 F_0(Q)=0 F 0 ( Q ) = 0
Lemma 34 (Existence after adding an input direction). Suppose K ≥ 1 K \ge1 K ≥ 1 Q I , K Q_{I,K} Q I , K m > ∣ I ∣ m > \lvert I\rvert m > ∣ I ∣ 0 < λ l < min a ∈ I λ a 0 < \lambda_l < \min_{a \in I}\lambda_a 0 < λ l < min a ∈ I λ a l ∉ I l \notin I l ∈ / I F K F_K F K span { v a : a ∈ I ∪ { l } } \operatorname{span}\{v_a : a \in I \cup\{l\}\} span { v a : a ∈ I ∪ { l }}
a K λ l > s I , K . a_K\lambda_l > s_{I,K}. a K λ l > s I , K . For K = 0 K = 0 K = 0 λ l < min a ∈ I λ a \lambda_l < \min_{a \in I}\lambda_a λ l < min a ∈ I λ a λ d > λ d + 1 > 0 \lambda_d > \lambda_{d+1} > 0 λ d > λ d + 1 > 0 d < m d < m d < m Q d + 1 , K Q_{d+1,K} Q d + 1 , K a K λ d + 1 / s d , K − 1 > 0 a_K\lambda_{d+1}/s_{d,K} - 1 > 0 a K λ d + 1 / s d , K − 1 > 0
Proof (of Lemma 34). First suppose K ≥ 1 K \ge1 K ≥ 1 (14) and the inverse in Lemma 16 give ∑ a ∈ I q K − 1 ( s I , K / λ a ) = 1 \sum_{a \in I}q_K^{-1}(s_{I,K}/\lambda_a) = 1 ∑ a ∈ I q K − 1 ( s I , K / λ a ) = 1 a K λ l > s I , K a_K\lambda_l > s_{I,K} a K λ l > s I , K a K λ l a_K\lambda_l a K λ l λ l \lambda_l λ l s I , K s_{I,K} s I , K a K λ l a_K\lambda_l a K λ l
For K = 0 K = 0 K = 0 F K F_K F K λ l \lambda_l λ l a 0 λ l = λ l < s I , 0 a_0\lambda_l = \lambda_l < s_{I,0} a 0 λ l = λ l < s I , 0 I = { 1 , … , d } I = \{1,\ldots,d\} I = { 1 , … , d }
Lemma 35 (Full derivative at a zero of F K F_K F K Let Q I , K Q_{I,K} Q I , K d d d v l v_l v l u ⊥ u_\perp u ⊥ Q I , K Q_{I,K} Q I , K
D F K ( Q I , K ) [ u ⊥ v l T ] = ( a K λ l s I , K − 1 ) u ⊥ v l T . DF_K(Q_{I,K})[u_\perp v_l^{\mathsf T}] = \left(\frac{a_K\lambda_l}{s_{I,K}} - 1\right)u_\perp v_l^{\mathsf T}. D F K ( Q I , K ) [ u ⊥ v l T ] = ( s I , K a K λ l − 1 ) u ⊥ v l T . For K ≥ 1 K \ge1 K ≥ 1 span { Q I , K v a v a T : a ∈ I } \operatorname{span}\{Q_{I,K}v_av_a^{\mathsf T}: a \in I\} span { Q I , K v a v a T : a ∈ I } K ≥ 1 K \ge1 K ≥ 1 m r mr m r a ∈ I a \in I a ∈ I l ∉ I l \notin I l ∈ / I
Table 1.
The zeros in the active–active and unused-hidden/active-input blocks are tangent to { O Q I , K : O ∈ O ( m ) } \{OQ_{I,K} : O \in O(m)\} { O Q I , K : O ∈ O ( m )} d d d K = 0 K = 0 K = 0 λ l / λ 1 − 1 \lambda_l/\lambda_1 - 1 λ l / λ 1 − 1
:::
Proof (of Lemma 35). Use left and right bases in which Q I , K Q_{I,K} Q I , K q a = ∥ Q I , K v a ∥ q_a=\lVert Q_{I,K}v_a\rVert q a = ∥ Q I , K v a ∥ Q I , K C / s I , K Q_{I,K}C/s_{I,K} Q I , K C / s I , K s a = λ a q a / s I , K s_a=\lambda_aq_a/s_{I,K} s a = λ a q a / s I , K (14) gives q a = p K ( s a ) q_a=p_K(s_a) q a = p K ( s a )
D ( Q C ∥ Q C ∥ F ) Q I , K [ E ] = E C s I , K − Q I , K C s I , K 3 ⟨ Q I , K C , E C ⟩ . D\left(\frac{QC}{\lVert QC\rVert_F}\right)_{Q_{I,K}}[E]=\frac{EC}{s_{I,K}}-\frac{Q_{I,K}C}{s_{I,K}^3}\langle Q_{I,K}C,EC\rangle. D ( ∥ QC ∥ F QC ) Q I , K [ E ] = s I , K E C − s I , K 3 Q I , K C ⟨ Q I , K C , E C ⟩ . Differentiating the first cubic iterate gives
D P 1 ( X ) [ E ] = 3 2 E − 1 2 ( E X T X + X E T X + X X T E ) . (69) DP_1(X)[E]=\frac{3}{2}E-\frac{1}{2}\left(EX^{\mathsf T}X+XE^{\mathsf T}X+XX^{\mathsf T}E\right). \tag*{(69)} D P 1 ( X ) [ E ] = 2 3 E − 2 1 ( E X T X + X E T X + X X T E ) . (69) These two formulas account for all blocks.
Unused hidden and input directions.
For E = u ⊥ v l T E=u_{\perp}v_l^{\mathsf T} E = u ⊥ v l T u ⊥ u_{\perp} u ⊥ l ∉ I l\notin I l ∈ / I
E C = λ l E , ⟨ Q I , K C , E C ⟩ = 0 , D ( Q C / ∥ Q C ∥ F ) Q I , K [ E ] = λ l s I , K E . EC=\lambda_lE,\qquad\langle Q_{I,K}C,EC\rangle=0,\qquad D(QC/\lVert QC\rVert_F)_{Q_{I,K}}[E]=\frac{\lambda_l}{s_{I,K}}E. E C = λ l E , ⟨ Q I , K C , E C ⟩ = 0 , D ( QC / ∥ QC ∥ F ) Q I , K [ E ] = s I , K λ l E . Orthogonality on both sides gives
E ( Q I , K C ) T ( Q I , K C ) = 0 , ( Q I , K C ) E T ( Q I , K C ) = 0 , ( Q I , K C ) ( Q I , K C ) T E = 0. E(Q_{I,K}C)^{\mathsf T}(Q_{I,K}C)=0,\qquad(Q_{I,K}C)E^{\mathsf T}(Q_{I,K}C)=0,\qquad(Q_{I,K}C)(Q_{I,K}C)^{\mathsf T}E=0. E ( Q I , K C ) T ( Q I , K C ) = 0 , ( Q I , K C ) E T ( Q I , K C ) = 0 , ( Q I , K C ) ( Q I , K C ) T E = 0. Positive scalar normalization preserves these equalities. Each successive polynomial iterate preserves the active left and right singular subspaces, so the cubic correction vanishes at every differentiation step. The increment is multiplied by 3 / 2 3/2 3/2 a K a_K a K λ l / s I , K \lambda_l/s_{I,K} λ l / s I , K E E E a K λ l / s I , K − 1 a_K\lambda_l/s_{I,K}-1 a K λ l / s I , K − 1
One-sided changes of orientation.
On an active hidden direction and unused input direction, the derivative of P K P_K P K p K ( s a ) / s a p_K(s_a)/s_a p K ( s a ) / s a
λ l s I , K p K ( s a ) s a − 1 = λ l λ a − 1 , \frac{\lambda_l}{s_{I,K}}\frac{p_K(s_a)}{s_a}-1=\frac{\lambda_l}{\lambda_a}-1, s I , K λ l s a p K ( s a ) − 1 = λ a λ l − 1 , using the identity in (14) . A rotation of the left singular vector gives the same multiplier with λ a \lambda_a λ a λ l \lambda_l λ l ( m − d ) d (m-d)d ( m − d ) d
Amplitude changes.
For a diagonal perturbation, d s I , K = ∑ b λ b s b d q b ds_{I,K}=\sum_b\lambda_bs_b\,dq_b d s I , K = ∑ b λ b s b d q b
d s a = λ a s I , K d q a − s a s I , K d s I , K . ds_a=\frac{\lambda_a}{s_{I,K}}\,dq_a-\frac{s_a}{s_{I,K}}\,ds_{I,K}. d s a = s I , K λ a d q a − s I , K s a d s I , K . Consequently the amplitude derivative is
diag ( s a p K ′ ( s a ) / q a − 1 ) − α β T , α a = p K ′ ( s a ) s a s I , K , β a = λ a s a . \operatorname{diag}\left(s_ap'_K(s_a)/q_a-1\right)-\alpha\beta^{\mathsf T},\qquad\alpha_a=\frac{p'_K(s_a)s_a}{s_{I,K}},\qquad\beta_a=\lambda_as_a. diag ( s a p K ′ ( s a ) / q a − 1 ) − α β T , α a = s I , K p K ′ ( s a ) s a , β a = λ a s a . For d ≥ 2 d\ge2 d ≥ 2 s a ∈ ( 0 , 1 ) s_a\in(0,1) s a ∈ ( 0 , 1 ) 0 < s a p K ′ ( s a ) / q a < 1 0<s_ap'_K(s_a)/q_a<1 0 < s a p K ′ ( s a ) / q a < 1 q K ′ ( s a 2 ) < 0 q'_K(s_a^2)<0 q K ′ ( s a 2 ) < 0 diag ( β a / α a ) \operatorname{diag}(\sqrt{\beta_a/\alpha_a}) diag ( β a / α a ) α a β a α b β b \sqrt{\alpha_a\beta_a}\sqrt{\alpha_b\beta_b} α a β a α b β b − 1 -1 − 1
An active off-diagonal pair.
For distinct active singular values s a , s b s_a, s_b s a , s b X q K ( X T X ) Xq_K(X^\mathsf{T}X) X q K ( X T X ) ( E a b , E b a ) (E_{ab},E_{ba}) ( E ab , E ba )
A a b = s a q a − s b q b s a 2 − s b 2 , B a b = s b q a − s a q b s a 2 − s b 2 . A_{ab}=\frac{s_aq_a-s_bq_b}{s_a^2-s_b^2},\qquad B_{ab}=\frac{s_bq_a-s_aq_b}{s_a^2-s_b^2}. A ab = s a 2 − s b 2 s a q a − s b q b , B ab = s a 2 − s b 2 s b q a − s a q b . Indeed the divided difference of q K q_K q K { q a / s a − q b / s b } / ( s a 2 − s b 2 ) \{q_a/s_a-q_b/s_b\}/(s_a^2-s_b^2) { q a / s a − q b / s b } / ( s a 2 − s b 2 ) X q K ( X T X ) Xq_K(X^\mathsf{T}X) X q K ( X T X ) D F K DF_K D F K
( A a b λ b / s I , K − 1 B a b λ a / s I , K B a b λ b / s I , K A a b λ a / s I , K − 1 ) . \begin{pmatrix}
A_{ab}\lambda_b/s_{I,K}-1 & B_{ab}\lambda_a/s_{I,K}\\
B_{ab}\lambda_b/s_{I,K} & A_{ab}\lambda_a/s_{I,K}-1
\end{pmatrix}. ( A ab λ b / s I , K − 1 B ab λ b / s I , K B ab λ a / s I , K A ab λ a / s I , K − 1 ) . The vector ( q b , − q a ) (q_b,-q_a) ( q b , − q a ) λ a / s I , K = s a / q a \lambda_a/s_{I,K}=s_a/q_a λ a / s I , K = s a / q a λ b / s I , K = s b / q b \lambda_b/s_{I,K}=s_b/q_b λ b / s I , K = s b / q b
− 1 + s a s b ( q a 2 − q b 2 ) q a q b ( s a 2 − s b 2 ) . (70) -1+\frac{s_as_b(q_a^2-q_b^2)}{q_aq_b(s_a^2-s_b^2)}. \tag*{(70)} − 1 + q a q b ( s a 2 − s b 2 ) s a s b ( q a 2 − q b 2 ) . (70) If s a > s b s_a>s_b s a > s b 1 < q a / q b < s a / s b 1<q_a/q_b<s_a/s_b 1 < q a / q b < s a / s b z ↦ z − z − 1 z\mapsto z-z^{-1} z ↦ z − z − 1 ( 1 , ∞ ) (1,\infty) ( 1 , ∞ ) (70) lies in ( 0 , 1 ) (0,1) ( 0 , 1 ) s a = s b s_a=s_b s a = s b q a = q b q_a=q_b q a = q b
A a b = 1 2 ( p K ′ ( s a ) + q a s a ) , B a b = 1 2 ( p K ′ ( s a ) − q a s a ) . A_{ab}=\frac{1}{2}\left(p'_K(s_a)+\frac{q_a}{s_a}\right),\qquad B_{ab}=\frac{1}{2}\left(p'_K(s_a)-\frac{q_a}{s_a}\right). A ab = 2 1 ( p K ′ ( s a ) + s a q a ) , B ab = 2 1 ( p K ′ ( s a ) − s a q a ) . Indeed, differentiation of s ↦ s p K ( s ) s\mapsto sp_K(s) s ↦ s p K ( s ) s ↦ p K ( s ) / s s\mapsto p_K(s)/s s ↦ p K ( s ) / s λ a / s I , K = λ b / s I , K = s a / q a \lambda_a/s_{I,K}=\lambda_b/s_{I,K}=s_a/q_a λ a / s I , K = λ b / s I , K = s a / q a − 1 + s a p K ′ ( s a ) / q a < 0 -1+s_ap'_K(s_a)/q_a<0 − 1 + s a p K ′ ( s a ) / q a < 0 0 < s a < 1 0<s_a<1 0 < s a < 1 q K ′ ( s a 2 ) < 0 q'_K(s_a^2)<0 q K ′ ( s a 2 ) < 0
The block dimensions sum to
d + d ( d − 1 ) + ( m − d ) d + d ( r − d ) + ( m − d ) ( r − d ) = m r . d+d(d-1)+(m-d)d+d(r-d)+(m-d)(r-d)=mr. d + d ( d − 1 ) + ( m − d ) d + d ( r − d ) + ( m − d ) ( r − d ) = m r . To identify the Gram derivative, the differential of Q ↦ Q T Q Q\mapsto Q^\mathsf{T}Q Q ↦ Q T Q E ↦ Q I , K T E + E T Q I , K E\mapsto Q_{I,K}^\mathsf{T}E+E^\mathsf{T}Q_{I,K} E ↦ Q I , K T E + E T Q I , K d d d d d d E 11 = Ω 11 diag ( q a ) E_{11}=\Omega_{11}\operatorname{diag}(q_a) E 11 = Ω 11 diag ( q a ) Ω 11 T = − Ω 11 \Omega_{11}^\mathsf{T}=-\Omega_{11} Ω 11 T = − Ω 11 Ω 21 diag ( q a ) \Omega_{21}\operatorname{diag}(q_a) Ω 21 diag ( q a ) E = Ω Q I , K E=\Omega Q_{I,K} E = Ω Q I , K Ω \Omega Ω K = 0 K = 0 K = 0 Q C / ∥ Q C ∥ F − Q Q C/\lVert Q C\rVert_F - Q QC / ∥ QC ∥ F − Q − 1 -1 − 1 λ l / λ 1 − 1 \lambda_l/\lambda_1 - 1 λ l / λ 1 − 1
Fix K ≥ 1 K \ge1 K ≥ 1 1 ≤ d < min { m , r } 1 \le d < \min\{m,r\} 1 ≤ d < min { m , r } Q d , K Q_{d,K} Q d , K
λ d > λ d + 1 > 0 , g d + 1 ( d ) > 0. \lambda_d > \lambda_{d+1} > 0,\qquad g^{(d)}_{d+1} > 0. λ d > λ d + 1 > 0 , g d + 1 ( d ) > 0. For 1 ≤ i ≤ d 1 \le i \le d 1 ≤ i ≤ d u i = Q d , K v i / ∥ Q d , K v i ∥ u_i = Q_{d,K}v_i/\lVert Q_{d,K}v_i\rVert u i = Q d , K v i / ∥ Q d , K v i ∥ u d + 1 u_{d+1} u d + 1 u d + 1 v d + 1 T u_{d+1}v_{d+1}^{\mathsf T} u d + 1 v d + 1 T Q d , K Q_{d,K} Q d , K v → − ∞ v \to-\infty v → − ∞ e g d + 1 ( d ) v e^{g^{(d)}_{d+1}v} e g d + 1 ( d ) v
Lemma 36 (Connection to the next rank). Under these conditions, the connection exists in { ∑ i = 1 d + 1 q i u i v i T : q i > 0 } \{\sum_{i=1}^{d+1}q_i u_i v_i^{\mathsf T}:q_i>0\} { ∑ i = 1 d + 1 q i u i v i T : q i > 0 } Q d + 1 , K Q_{d+1,K} Q d + 1 , K l > d + 1 l>d+1 l > d + 1 [ v a , v b ] [v_a,v_b] [ v a , v b ]
exp { ∫ v a v b ( a K λ l ∥ Q ( v ) C ∥ F − 1 ) d v } > 0. \exp\left\{\int_{v_a}^{v_b}\left(\frac{a_K\lambda_l}{\lVert Q(v)C\rVert_F}-1\right)\,\mathrm{d}v\right\}>0. exp { ∫ v a v b ( ∥ Q ( v ) C ∥ F a K λ l − 1 ) d v } > 0. Proof (of Lemma 36). In the diagonal factor subspace supported on { 1 , … , d + 1 } \{1,\ldots,d+1\} { 1 , … , d + 1 } g d + 1 ( d ) g^{(d)}_{d+1} g d + 1 ( d ) Q d , K Q_{d,K} Q d , K C 2 C^2 C 2
All diagonal amplitudes are positive. Define their normalized squared coordinates
x i = λ i 2 ∥ Q ( v ) v i ∥ 2 ∥ Q ( v ) C ∥ F 2 , 1 ≤ i ≤ d + 1. x_i=\frac{\lambda_i^2\lVert Q(v)v_i\rVert^2}{\lVert Q(v)C\rVert_F^2},\qquad1\le i\le d+1. x i = ∥ Q ( v ) C ∥ F 2 λ i 2 ∥ Q ( v ) v i ∥ 2 , 1 ≤ i ≤ d + 1. The amplitude equation is
d d v ∥ Q ( v ) v i ∥ = ∥ Q ( v ) v i ∥ { λ i q K ( x i ) ∥ Q ( v ) C ∥ F − 1 } . \frac{\mathrm{d}}{\mathrm{d}v}\lVert Q(v)v_i\rVert
=
\lVert Q(v)v_i\rVert
\left\{\frac{\lambda_i q_K(x_i)}{\lVert Q(v)C\rVert_F}-1\right\}. d v d ∥ Q ( v ) v i ∥ = ∥ Q ( v ) v i ∥ { ∥ Q ( v ) C ∥ F λ i q K ( x i ) − 1 } . Its multiplicative form preserves positivity at finite times. Differentiating the norm and the normalized coordinates gives
x i ′ = 2 x i ∥ Q ( v ) C ∥ F { λ i q K ( x i ) − ∑ a = 1 d + 1 λ a x a q K ( x a ) } , d d v ∥ Q ( v ) C ∥ F = ∑ i = 1 d + 1 λ i x i q K ( x i ) − ∥ Q ( v ) C ∥ F . (71) \begin{aligned}
x_i'&=\frac{2x_i}{\lVert Q(v)C\rVert_F}
\left\{\lambda_i q_K(x_i)-\sum_{a=1}^{d+1}\lambda_a x_a q_K(x_a)\right\},\\
\frac{\mathrm{d}}{\mathrm{d}v}\lVert Q(v)C\rVert_F
&=\sum_{i=1}^{d+1}\lambda_i x_iq_K(x_i)-\lVert Q(v)C\rVert_F. \tag*{(71)}
\end{aligned} x i ′ d v d ∥ Q ( v ) C ∥ F = ∥ Q ( v ) C ∥ F 2 x i { λ i q K ( x i ) − a = 1 ∑ d + 1 λ a x a q K ( x a ) } , = i = 1 ∑ d + 1 λ i x i q K ( x i ) − ∥ Q ( v ) C ∥ F . (71) Thus x i > 0 x_i > 0 x i > 0 ∑ i x i = 1 \sum_i x_i = 1 ∑ i x i = 1 x ↦ ∑ i λ i x i ∫ 0 x i q K ( u ) d u x \mapsto\sum_i \lambda_i x_i \int_0^{x_i} q_K(u)\,\mathrm{d}u x ↦ ∑ i λ i x i ∫ 0 x i q K ( u ) d u λ i q K ′ ( x i ) < 0 \lambda_i q_K'(x_i) < 0 λ i q K ′ ( x i ) < 0 q K − 1 ( s d + 1 , K / λ i ) q_K^{-1}(s_{d+1,K}/\lambda_i) q K − 1 ( s d + 1 , K / λ i ) s d + 1 , K s_{d+1,K} s d + 1 , K
d d v ∑ i q K − 1 ( s d + 1 , K / λ i ) log q K − 1 ( s d + 1 , K / λ i ) x i = − 2 ∥ Q ( v ) C ∥ F 2 ∑ i [ q K − 1 ( s d + 1 , K / λ i ) − x i ] λ i q K ( x i ) ≤ − 2 ∥ Q ( v ) C ∥ F 2 ∑ i λ i ∫ x i q K − 1 ( s d + 1 , K / λ i ) q K ( u ) d u . (72) \begin{aligned}
\frac{\mathrm{d}}{\mathrm{d}v}\sum_i q_K^{-1}(s_{d+1,K}/\lambda_i)\log\frac{q_K^{-1}(s_{d+1,K}/\lambda_i)}{x_i}
= -\frac{2}{\lVert Q(v)C\rVert_F^2}\sum_i\left[q_K^{-1}(s_{d+1,K}/\lambda_i)-x_i\right]\lambda_i q_K(x_i) \\
&\leq-\frac{2}{\lVert Q(v)C\rVert_F^2}\sum_i\lambda_i\int_{x_i}^{q_K^{-1}(s_{d+1,K}/\lambda_i)}q_K(u)\,\mathrm{d}u. \tag*{(72)}
\end{aligned} d v d i ∑ q K − 1 ( s d + 1 , K / λ i ) log x i q K − 1 ( s d + 1 , K / λ i ) = − ∥ Q ( v ) C ∥ F 2 2 i ∑ [ q K − 1 ( s d + 1 , K / λ i ) − x i ] λ i q K ( x i ) ≤ − ∥ Q ( v ) C ∥ F 2 2 i ∑ λ i ∫ x i q K − 1 ( s d + 1 , K / λ i ) q K ( u ) d u . (72) The last line uses the concave tangent inequality. Positive weights make an entropy sublevel bounded away from every simplex boundary.
The scalar polynomial bounds also give
λ d + 1 λ 1 − ∥ Q ( v ) ∥ F ≤ d d v ∥ Q ( v ) ∥ F ≤ d + 1 − ∥ Q ( v ) ∥ F . \frac{\lambda_{d+1}}{\lambda_1}-\lVert Q(v)\rVert_F\leq\frac{d}{d_v}\lVert Q(v)\rVert_F\leq\sqrt{d+1}-\lVert Q(v)\rVert_F. λ 1 λ d + 1 − ∥ Q ( v ) ∥ F ≤ d v d ∥ Q ( v ) ∥ F ≤ d + 1 − ∥ Q ( v ) ∥ F . For the lower bound use p K ( x i ) ≥ x i p_K(\sqrt{x_i})\geq\sqrt{x_i} p K ( x i ) ≥ x i ∥ Q ( v ) C ∥ F ≤ λ 1 ∥ Q ( v ) ∥ F \lVert Q(v)C\rVert_F\leq\lambda_1\lVert Q(v)\rVert_F ∥ Q ( v ) C ∥ F ≤ λ 1 ∥ Q ( v ) ∥ F p K ≤ 1 p_K\leq1 p K ≤ 1
x i ( v ) ⟶ q K − 1 ( s d + 1 , K / λ i ) . x_i(v)\longrightarrow q_K^{-1}(s_{d+1,K}/\lambda_i). x i ( v ) ⟶ q K − 1 ( s d + 1 , K / λ i ) . The sum ∑ i λ i x i ( v ) q K ( x i ( v ) ) \sum_i\lambda_i x_i(v)q_K(x_i(v)) ∑ i λ i x i ( v ) q K ( x i ( v )) s d + 1 , K s_{d+1,K} s d + 1 , K
∥ Q ( v ) C ∥ F − s d + 1 , K = e − v ( ∥ Q ( 0 ) C ∥ F − s d + 1 , K ) + ∫ 0 v e − ( v − u ) [ ∑ i λ i x i ( u ) q K ( x i ( u ) ) − s d + 1 , K ] d u . \lVert Q(v)C\rVert_F-s_{d+1,K}
=e^{-v}\left(\lVert Q(0)C\rVert_F-s_{d+1,K}\right)
+\int_0^v e^{-(v-u)}\left[\sum_i\lambda_i x_i(u)q_K(x_i(u))-s_{d+1,K}\right]\,\mathrm{d}u. ∥ Q ( v ) C ∥ F − s d + 1 , K = e − v ( ∥ Q ( 0 ) C ∥ F − s d + 1 , K ) + ∫ 0 v e − ( v − u ) [ i ∑ λ i x i ( u ) q K ( x i ( u )) − s d + 1 , K ] d u . Splitting at a sufficiently late fixed time proves convergence of the scale. Since ∥ Q ( v ) v i ∥ = ∥ Q ( v ) C ∥ F x i ( v ) / λ i \lVert Q(v)v_i\rVert=\lVert Q(v)C\rVert_F\sqrt{x_i(v)}/\lambda_i ∥ Q ( v ) v i ∥ = ∥ Q ( v ) C ∥ F x i ( v ) / λ i Q d + 1 , K Q_{d+1,K} Q d + 1 , K
For each remaining unused input label l l l a K λ l / ∥ Q ( v ) C ∥ F − 1 a_K\lambda_l/\lVert Q(v)C\rVert_F-1 a K λ l / ∥ Q ( v ) C ∥ F − 1
exp { ∫ v a v b ( a K λ l ∥ Q ( v ) C ∥ F − 1 ) d v } > 0. (73) \exp\left\{\int_{v_a}^{v_b}\left(\frac{a_K\lambda_l}{\lVert Q(v)C\rVert_F}-1\right)\,\mathrm{d}v\right\}>0. \tag*{(73)} exp { ∫ v a v b ( ∥ Q ( v ) C ∥ F a K λ l − 1 ) d v } > 0. (73) Labels evolve independently. The product for h h h a K / ∥ Q ( v ) C ∥ F a_K/\lVert Q(v)C\rVert_F a K / ∥ Q ( v ) C ∥ F
The solution connects the two local neighborhoods. The Gaussian data and the subsequent passage results identify the independent coefficients and show how the actual trajectory follows the prescribed additions.
The algebraic construction in § 3 uses only existing supports. The following proposition verifies the finite procedure and identifies its exceptional exponent set.
Proposition 37 (Clocks and exceptional exponents) Under Assumptions 2 and 5, the two algebraic procedures in § 3 are well defined. Every candidate support used by either procedure exists; all its normalizations are uniquely determined. The finite coefficients c d , K c_{d,K} c d , K b b b d ≥ 2 d \ge2 d ≥ 2
x d = ( b − 1 ) c d , K , ∣ B C , K ∣ ≤ p − 1. x_d = (b-1)c_{d,K}, \qquad\lvert B_{C,K}\rvert\le p-1. x d = ( b − 1 ) c d , K , ∣ B C , K ∣ ≤ p − 1. The set B C B_C B C b ∉ B C , K b \notin B_{C,K} b ∈ / B C , K b ∉ B C b \notin B_C b ∈ / B C
Proof (of Proposition 37). The initial zero Q 1 , K = e 1 v 1 T Q_{1,K}=e_1v_1^\mathsf{T} Q 1 , K = e 1 v 1 T K K K s 1 , K = λ 1 s_{1,K}=\lambda_1 s 1 , K = λ 1 d < p d<p d < p λ d + 1 \lambda_{d+1} λ d + 1 λ d > λ d + 1 \lambda_d>\lambda_{d+1} λ d > λ d + 1 Lemma 34 constructs the next support and Lemma 33 gives its unique Gram matrix and scale. The hidden dimension permits the addition since d < p ≤ m d<p\le m d < p ≤ m K = 0 K=0 K = 0 λ 2 / λ 1 − 1 < 0 \lambda_2/\lambda_1-1<0 λ 2 / λ 1 − 1 < 0 p − 1 p-1 p − 1
The coefficient c 2 , K c_{2,K} c 2 , K 1 − λ d + 1 / λ d > 0 1-\lambda_{d+1}/\lambda_d>0 1 − λ d + 1 / λ d > 0 (20) is positive, proving strict increase and positivity of every finite c d , K c_{d,K} c d , K d ≥ 2 d\ge2 d ≥ 2 C , K C,K C , K b b b
For the actual candidate-time procedure, the first finite quotient is ( b − 1 ) c 2 , K (b-1)c_{2,K} ( b − 1 ) c 2 , K x d = ( b − 1 ) c d , K x_d=(b-1)c_{d,K} x d = ( b − 1 ) c d , K
x d + 1 = ( b − 1 ) c d , K + ( b − 1 ) ( 1 + c d , K ) ( 1 − λ d + 1 / λ d ) g d + 1 ( d ) = ( b − 1 ) c d + 1 , K . x_{d+1}=(b-1)c_{d,K}+
\frac{(b-1)(1+c_{d,K})(1-\lambda_{d+1}/\lambda_d)}
{g_{d+1}^{(d)}}
=(b-1)c_{d+1,K}. x d + 1 = ( b − 1 ) c d , K + g d + 1 ( d ) ( b − 1 ) ( 1 + c d , K ) ( 1 − λ d + 1 / λ d ) = ( b − 1 ) c d + 1 , K . Its used supports form an initial segment of the previously constructed algebraic sequence; stopping at a clock at least one creates no new normalization. At a nonpositive rate both constructions stop. This proves the identity at every evaluated finite candidate.
A finite candidate can equal one only if b = 1 + 1 / c d , K b=1+1/c_{d,K} b = 1 + 1/ c d , K p − 1 p-1 p − 1 K K K K ≥ 0 K\ge0 K ≥ 0 min { p , r − 1 } \min\{p,r-1\} min { p , r − 1 }
Exact normal equation and integrated relative estimates
Local Gram coordinates separate stable tangent motion from the independent unused columns. The exact projected equation is homogeneous in the full normal matrix. This permits relative volume estimates even when those columns are arbitrarily small. For I ⊂ { 1 , … , r } I \subset\{1,\ldots,r\} I ⊂ { 1 , … , r } Q I = ( Q V l ) l ∈ I Q_I=(Q_{Vl})_{l\in I} Q I = ( Q V l ) l ∈ I d d d Ran ( Q { 1 , … , d } ) ⊥ \operatorname{Ran}(Q_{\{1,\ldots,d\}})^\perp Ran ( Q { 1 , … , d } ) ⊥
Z ( Q ) = ( z d + 1 , … , z r ) ∈ R ( m − d ) × ( r − d ) . Z(Q)=(z_{d+1},\ldots,z_r)\in\mathbb{R}^{(m-d)\times(r-d)}. Z ( Q ) = ( z d + 1 , … , z r ) ∈ R ( m − d ) × ( r − d ) . This extends the rank-one definition in Section B.2. For 0 ≤ h ≤ min { m − d , r − d } 0\le h\le\min\{m-d,r-d\} 0 ≤ h ≤ min { m − d , r − d }
V h ( Z ) = ∥ z d + 1 ∧ ⋯ ∧ z d + h ∥ , V 0 ( Z ) = 1. (74) V_h(Z)=\lVert z_{d+1}\wedge\cdots\wedge z_{d+h}\rVert,\qquad V_0(Z)=1. \tag*{(74)} V h ( Z ) = ∥ z d + 1 ∧ ⋯ ∧ z d + h ∥ , V 0 ( Z ) = 1. (74) At rank d d d
1 ≤ h ≤ p − d , J d , h = { d + 1 , … , d + h } , p = min { m , n + ( C ) } . (75) 1\le h\le p-d,\qquad J_{d,h}=\{d+1,\ldots,d+h\},\qquad p=\min\{m,n_+(C)\}. \tag*{(75)} 1 ≤ h ≤ p − d , J d , h = { d + 1 , … , d + h } , p = min { m , n + ( C )} . (75) At d = p d=p d = p
σ d , h = ∑ l = d + 1 d + h g l ( d ) − max J ⊂ { d + 1 , … , r } , ∣ J ∣ = h J ≠ J d , h ∑ l ∈ J g l ( d ) . (76) \sigma_{d,h}=\sum_{l=d+1}^{d+h}g_l^{(d)}-\max_{\substack{J\subset\{d+1,\ldots,r\},\ \lvert J\rvert=h\\J\ne J_{d,h}}}\sum_{l\in J}g_l^{(d)}. \tag*{(76)} σ d , h = l = d + 1 ∑ d + h g l ( d ) − J ⊂ { d + 1 , … , r } , ∣ J ∣ = h J = J d , h max l ∈ J ∑ g l ( d ) . (76) Set σ d , h = + ∞ \sigma_{d,h}=+\infty σ d , h = + ∞ V h / V h − 1 V_h/V_{h-1} V h / V h − 1 V h / V h − 1 V_h/V_{h-1} V h / V h − 1
Lemma 38 (Leading exterior gaps) For an existing Q d , K Q_{d,K} Q d , K 1 ≤ h ≤ min { m − d , r − d } 1\le h\le\min\{m-d,r-d\} 1 ≤ h ≤ min { m − d , r − d } d + h < r d+h<r d + h < r
σ d , h = a K s d , K ( λ d + h − λ d + h + 1 ) ≥ 0. \sigma_{d,h}=\frac{a_K}{s_{d,K}}(\lambda_{d+h}-\lambda_{d+h+1})\ge0. σ d , h = s d , K a K ( λ d + h − λ d + h + 1 ) ≥ 0. Under Assumption 5, this gap is strictly positive for every retained order 1 ≤ h ≤ p − d 1\le h\le p-d 1 ≤ h ≤ p − d λ d > λ d + 1 \lambda_d>\lambda_{d+1} λ d > λ d + 1 d ≤ p d\le p d ≤ p d < r d<r d < r d + h = r d+h=r d + h = r + ∞ +\infty + ∞
Proof (of Lemma 38). The rates have the same signed order as the task eigenvalues because a K / s d , K > 0 a_K/s_{d,K}>0 a K / s d , K > 0 h h h h h h d + h ≤ p d+h\le p d + h ≤ p
The coordinate identities and norm estimates in this subsection apply in any positive matrix dimensions under their stated active-column and spectral-boundary conditions. A neighborhood of Q d , K Q_{d,K} Q d , K Q ↦ Q T Q Q\mapsto Q^{\mathsf T}Q Q ↦ Q T Q Q d , K T Q d , K Q_{d,K}^{\mathsf T}Q_{d,K} Q d , K T Q d , K S = Q T Q S=Q^{\mathsf T}Q S = Q T Q { 1 , … , d } \{1,\ldots,d\} { 1 , … , d } { d + 1 , … , r } \{d+1,\ldots,r\} { d + 1 , … , r }
y = ( S 11 − ( Q d , K T Q d , K ) 11 , S 12 ) , H = S 22 − S 21 S 11 − 1 S 12 = Z T Z . y=(S_{11}-(Q_{d,K}^{\mathsf T}Q_{d,K})_{11},S_{12}),\qquad H=S_{22}-S_{21}S_{11}^{-1}S_{12}=Z^{\mathsf T}Z. y = ( S 11 − ( Q d , K T Q d , K ) 11 , S 12 ) , H = S 22 − S 21 S 11 − 1 S 12 = Z T Z . Choose an orthonormal complementary frame U ⊥ U_{\perp} U ⊥
U ⊥ ′ = − Q { 1 , … , d } ( Q { 1 , … , d } T Q { 1 , … , d } ) − 1 ( Q { 1 , … , d } T ) ′ U ⊥ . U_{\perp}'=-Q_{\{1,\ldots,d\}}\left(Q_{\{1,\ldots,d\}}^{\mathsf T}Q_{\{1,\ldots,d\}}\right)^{-1}\left(Q_{\{1,\ldots,d\}}^{\mathsf T}\right)'U_{\perp}. U ⊥ ′ = − Q { 1 , … , d } ( Q { 1 , … , d } T Q { 1 , … , d } ) − 1 ( Q { 1 , … , d } T ) ′ U ⊥ . Here and below the prime denotes differentiation in v = log t v=\log t v = log t Q { 1 , … , d } T U ⊥ = 0 Q_{\{1,\ldots,d\}}^{\mathsf T}U_{\perp}=0 Q { 1 , … , d } T U ⊥ = 0 U ⊥ T U ⊥ = I U_{\perp}^{\mathsf T}U_{\perp}=I U ⊥ T U ⊥ = I U ⊥ T U ⊥ ′ = 0 U_{\perp}^{\mathsf T}U_{\perp}'=0 U ⊥ T U ⊥ ′ = 0 Z = U ⊥ T Q { d + 1 , … , r } Z=U_{\perp}^{\mathsf T}Q_{\{d+1,\ldots,r\}} Z = U ⊥ T Q { d + 1 , … , r }
R t = C − t 2 L ( S ) , s t = ∥ Q R t ∥ F + c η ε 2 t , T = R t s t q K ( R t S R t s t 2 ) . R_t=C-t^2\mathcal{L}(S),\qquad s_t=\lVert QR_t\rVert_F+\frac{c_{\eta}\varepsilon}{2t},\qquad T=\frac{R_t}{s_t}q_K\left(\frac{R_tSR_t}{s_t^2}\right). R t = C − t 2 L ( S ) , s t = ∥ Q R t ∥ F + 2 t c η ε , T = s t R t q K ( s t 2 R t S R t ) . With Γ d = diag d < l ≤ r ( g l ( d ) ) \Gamma_d=\operatorname{diag}_{d<l\le r}(g_l^{(d)}) Γ d = diag d < l ≤ r ( g l ( d ) )
E = T 22 − I − T 21 S 11 − 1 S 12 − Γ d . E=T_{22}-I-T_{21}S_{11}^{-1}S_{12}-\Gamma_d. E = T 22 − I − T 21 S 11 − 1 S 12 − Γ d . The identity ∥ Q R t ∥ F 2 = tr ( R t S R t ) \lVert QR_t\rVert_F^2=\operatorname{tr}(R_tSR_t) ∥ Q R t ∥ F 2 = tr ( R t S R t ) t 2 t^2 t 2 ε / t \varepsilon/t ε / t
The limiting Gram equation used below is
T ( S ) = C tr ( C S C ) q K ( C S C tr ( C S C ) ) − I , S ′ = T ( S ) T S + S T ( S ) . (77) T(S)=\frac{C}{\sqrt{\operatorname{tr}(CSC)}}q_K\left(\frac{CSC}{\operatorname{tr}(CSC)}\right)-I,\qquad S'=T(S)^{\mathsf T}S+ST(S). \tag*{(77)} T ( S ) = tr ( C S C ) C q K ( tr ( C S C ) C S C ) − I , S ′ = T ( S ) T S + S T ( S ) . (77) The prime denotes differentiation in v = log t v=\log t v = log t J d J_d J d y y y y y y ( y , H ) = ( 0 , 0 ) (y,H)=(0,0) ( y , H ) = ( 0 , 0 ) y y y g = g d + 1 ( d ) g=g_{d+1}^{(d)} g = g d + 1 ( d ) h h h ζ \zeta ζ
∥ ∧ h Z ∥ F 2 ≤ ( 1 + ζ 2 ) V h ( Z ) 2 . \lVert\wedge^h Z\rVert_F^2\le(1+\zeta^2)V_h(Z)^2. ∥ ∧ h Z ∥ F 2 ≤ ( 1 + ζ 2 ) V h ( Z ) 2 . Only orders required in a given result are retained. The local equations and tangent contraction use λ d > λ d + 1 \lambda_d>\lambda_{d+1} λ d > λ d + 1 d < r d<r d < r σ d , h > 0 \sigma_{d,h} > 0 σ d , h > 0 ∥ Z ∥ F \lVert Z\rVert_{F} ∥ Z ∥ F g g g
D + a ( v ) = lim sup h → 0 a ( v + h ) − a ( v ) h D^{+}a(v) = \limsup_{h \to0} \frac{a(v+h)-a(v)}{h} D + a ( v ) = h → 0 lim sup h a ( v + h ) − a ( v ) for the upper right Dini derivative of a scalar function.
Lemma 39 (Integrated relative volume estimates) In a sufficiently small coordinate neighborhood of Q d , K Q_{d,K} Q d , K
y ′ = J d y + O ( ∥ y ∥ 2 + ∥ Z ∥ F 2 + t 2 + ε / t ) , Z ′ = Z ( Γ d + E ) , ∥ E ∥ o p ≤ C ( ∥ y ∥ + ∥ Z ∥ F 2 + t 2 + ε / t ) , D + ∥ y ∥ ≤ − ω ∥ y ∥ + C ( ∥ Z ∥ F 2 + t 2 + ε / t ) (78) \begin{aligned}
y' &= J_d y + O(\lVert y\rVert^2 + \lVert Z\rVert_F^2 + t^2 + \varepsilon/t),\\
Z' &= Z(\Gamma_d + E),\\
\lVert E\rVert_{\mathrm{op}} &\le C(\lVert y\rVert+ \lVert Z\rVert_F^2 + t^2 + \varepsilon/t),\\
D^{+}\lVert y\rVert&\le-\omega\lVert y\rVert+ C(\lVert Z\rVert_F^2 + t^2 + \varepsilon/t) \tag*{(78)}
\end{aligned} y ′ Z ′ ∥ E ∥ op D + ∥ y ∥ = J d y + O (∥ y ∥ 2 + ∥ Z ∥ F 2 + t 2 + ε / t ) , = Z ( Γ d + E ) , ≤ C (∥ y ∥ + ∥ Z ∥ F 2 + t 2 + ε / t ) , ≤ − ω ∥ y ∥ + C (∥ Z ∥ F 2 + t 2 + ε / t ) (78) The identity for Z Z Z Q d , K Q_{d,K} Q d , K ω > 0 \omega> 0 ω > 0 [ t a , t b ] [t_a,t_b] [ t a , t b ] t a ≥ c U ε t_a \ge c_U\varepsilon t a ≥ c U ε c U > 0 c_U > 0 c U > 0 t b ≤ t ∗ t_b \le t_\ast t b ≤ t ∗ t ∗ > 0 t_\ast> 0 t ∗ > 0 t a t_a t a t ∗ t_\ast t ∗ ε / t a \varepsilon/t_a ε / t a V h ( Z ( t a ) ) > 0 V_h(Z(t_a)) > 0 V h ( Z ( t a )) > 0
∣ log V h ( Z ( t ) ) V h ( Z ( t a ) ) − ( ∑ ℓ ∈ J d , h g ℓ ( d ) ) log ( t / t a ) ∣ ≤ C h ∫ log t a log t ∥ E ∥ o p d v , for each retained order. (79) \left\lvert\log\frac{V_h(Z(t))}{V_h(Z(t_a))} - \left(\sum_{\ell\in J_{d,h}} g_{\ell}^{(d)}\right)\log(t/t_a)\right\rvert\le C_h\int_{\log t_a}^{\log t}\lVert E\rVert_{\mathrm{op}}\,dv,\qquad\text{for each retained order.} \tag*{(79)} log V h ( Z ( t a )) V h ( Z ( t )) − ℓ ∈ J d , h ∑ g ℓ ( d ) log ( t / t a ) ≤ C h ∫ l o g t a l o g t ∥ E ∥ op d v , for each retained order. (79) Here the constants are independent of the size of the leading volumes. Suppose g > 0 g > 0 g > 0 g / 2 g/2 g /2 t a ≤ s ≤ t b t_a \le s \le t_b t a ≤ s ≤ t b
∥ Z ( s ) ∥ F ≤ 1 + ζ 2 ∥ Z ( t b ) ∥ F ( s / t b ) g / 2 , ∫ log t a log t b ∥ Z ∥ F 2 d v ≤ C g ∥ Z ( t b ) ∥ F 2 , ∫ log t a log t b ( ∥ y ∥ + ∥ E ∥ o p ) d v ≤ C U . (80) \begin{aligned}
\lVert Z(s)\rVert_F &\le\sqrt{1+\zeta^2}\,\lVert Z(t_b)\rVert_F(s/t_b)^{g/2},\\
\int_{\log t_a}^{\log t_b}\lVert Z\rVert_F^2\,dv &\le\frac{C}{g}\lVert Z(t_b)\rVert_F^2,\\
\int_{\log t_a}^{\log t_b}(\lVert y\rVert+\lVert E\rVert_{\mathrm{op}})\,dv &\le C_U. \tag*{(80)}
\end{aligned} ∥ Z ( s ) ∥ F ∫ l o g t a l o g t b ∥ Z ∥ F 2 d v ∫ l o g t a l o g t b (∥ y ∥ + ∥ E ∥ op ) d v ≤ 1 + ζ 2 ∥ Z ( t b ) ∥ F ( s / t b ) g /2 , ≤ g C ∥ Z ( t b ) ∥ F 2 , ≤ C U . (80) The endpoint may be any stopping time for which the stated local conditions hold. These constants are independent of the interval length. For a sufficiently narrow first-column cone and sufficiently small coefficient error, there is a fixed C ≥ 1 C \ge1 C ≥ 1
C − 1 tr H ≤ ( tr H ) ′ ≤ C tr H C^{-1}\operatorname{tr} H \le(\operatorname{tr} H)' \le C\operatorname{tr} H C − 1 tr H ≤ ( tr H ) ′ ≤ C tr H on the same positive-growth interval.
For the following norm estimate, assume the local equations and tangent contraction above, with any multiplicities among the unused rates; no volume-cone hypothesis is imposed. Suppose
∥ Z ( t a ) ∥ F ≤ C U ε e , e > 0 , log ( t ∗ / t a ) = θ log ( 1 / ε ) + O U ( 1 ) , θ ≥ 0. \lVert Z(t_a)\rVert_F \le C_U\varepsilon^e,\qquad e>0,\qquad\log(t_*/t_a)=\theta\log(1/\varepsilon)+O_U(1),\qquad\theta\ge0. ∥ Z ( t a ) ∥ F ≤ C U ε e , e > 0 , log ( t ∗ / t a ) = θ log ( 1/ ε ) + O U ( 1 ) , θ ≥ 0. If e − g θ > 0 e-g\theta>0 e − g θ > 0 ν > 0 \nu>0 ν > 0 e − ( g + ν ) θ > 0 e-(g+\nu)\theta>0 e − ( g + ν ) θ > 0 ∥ E ∥ o p ≤ ν \lVert E\rVert_{\mathrm{op}}\le\nu ∥ E ∥ op ≤ ν t ∗ t_* t ∗
∥ Z ( t ) ∥ F ≤ C U ε e ( t / t a ) g + ν , t a ≤ t ≤ t ∗ . \lVert Z(t)\rVert_F\le C_U\varepsilon^e(t/t_a)^{g+\nu},\qquad t_a\le t\le t_*. ∥ Z ( t ) ∥ F ≤ C U ε e ( t / t a ) g + ν , t a ≤ t ≤ t ∗ . The supremum of ∥ Z ∥ F \lVert Z\rVert_F ∥ Z ∥ F ε \varepsilon ε g ≤ 0 g\le0 g ≤ 0
∥ Z ( t ) ∥ F ≤ ∥ Z ( t a ) ∥ F ( t / t a ) − 1 / 2 ; \lVert Z(t)\rVert_F\le\lVert Z(t_a)\rVert_F(t/t_a)^{-1/2}; ∥ Z ( t ) ∥ F ≤ ∥ Z ( t a ) ∥ F ( t / t a ) − 1/2 ; the squared-normal and coefficient integrals are bounded without a cone condition or an incoming power-law assumption. If the normal block is empty, the conclusions hold with Z = 0 Z=0 Z = 0
Proof (of Lemma 39). Fix Q d , K Q_{d,K} Q d , K d d d v = log t v=\log t v = log t
The horizontal-frame equation prescribed before the lemma has continuous coefficients while the active columns are independent. Differentiation gives
( Q { 1 , … , d } T U ⊥ ) ′ = ( Q { 1 , … , d } ′ ) T U ⊥ − ( Q { 1 , … , d } ) T U ⊥ ′ = 0 , (Q_{\{1,\ldots,d\}}^{\mathsf T}U_\perp)'=(Q'_{\{1,\ldots,d\}})^{\mathsf T}U_\perp-(Q_{\{1,\ldots,d\}})^{\mathsf T}U'_\perp=0, ( Q { 1 , … , d } T U ⊥ ) ′ = ( Q { 1 , … , d } ′ ) T U ⊥ − ( Q { 1 , … , d } ) T U ⊥ ′ = 0 , U ⊥ T U ⊥ ′ = 0 , ( U ⊥ T U ⊥ ) ′ = 0. U_\perp^{\mathsf T}U'_\perp=0,\qquad(U_\perp^{\mathsf T}U_\perp)'=0. U ⊥ T U ⊥ ′ = 0 , ( U ⊥ T U ⊥ ) ′ = 0. Thus U ⊥ U_\perp U ⊥ Q ′ = Q ( T − I ) Q'=Q(T-I) Q ′ = Q ( T − I ) R t R_t R t s t s_t s t Z = U ⊥ T Q { d + 1 , … , r } Z=U_\perp^{\mathsf T}Q_{\{d+1,\ldots,r\}} Z = U ⊥ T Q { d + 1 , … , r }
Z ′ = U ⊥ T Q { d + 1 , … , r } ′ − U ⊥ T Q { 1 , … , d } ′ S 11 − 1 S 12 , Z'=U_\perp^{\mathsf T}Q'_{\{d+1,\ldots,r\}}-U_\perp^{\mathsf T}Q'_{\{1,\ldots,d\}}S_{11}^{-1}S_{12}, Z ′ = U ⊥ T Q { d + 1 , … , r } ′ − U ⊥ T Q { 1 , … , d } ′ S 11 − 1 S 12 , U ⊥ T Q { 1 , … , d } ′ = Z T 21 , U ⊥ T Q { d + 1 , … , r } ′ = Z ( T 22 − I ) . U_\perp^{\mathsf T}Q'_{\{1,\ldots,d\}}=ZT_{21},\qquad U_\perp^{\mathsf T}Q'_{\{d+1,\ldots,r\}}=Z(T_{22}-I). U ⊥ T Q { 1 , … , d } ′ = Z T 21 , U ⊥ T Q { d + 1 , … , r } ′ = Z ( T 22 − I ) . Consequently,
Z ′ = Z ( T 22 − I − T 21 S 11 − 1 S 12 ) = Z ( Γ d + E ) . (81) Z'=Z(T_{22}-I-T_{21}S_{11}^{-1}S_{12})=Z(\Gamma_d+E). \tag*{(81)} Z ′ = Z ( T 22 − I − T 21 S 11 − 1 S 12 ) = Z ( Γ d + E ) . (81) This identity preserves every vanishing independent component under the complete empirical equation.
The limiting Gram equation follows by differentiating S = Q T Q S=Q^{\mathsf T}Q S = Q T Q tr ( C Q d , K T Q d , K C ) = s d , K 2 > 0 \operatorname{tr}(CQ_{d,K}^{\mathsf T}Q_{d,K}C)=s_{d,K}^2>0 tr ( C Q d , K T Q d , K C ) = s d , K 2 > 0 S 11 S_{11} S 11 S S S
( S 22 − S 21 S 11 − 1 S 12 ) ′ = S 22 ′ − S 21 ′ S 11 − 1 S 12 − S 21 S 11 − 1 S 12 ′ + S 21 S 11 − 1 S 11 ′ S 11 − 1 S 12 . (S_{22}-S_{21}S_{11}^{-1}S_{12})'
=S'_{22}-S'_{21}S_{11}^{-1}S_{12}-S_{21}S_{11}^{-1}S'_{12}
+S_{21}S_{11}^{-1}S'_{11}S_{11}^{-1}S_{12}. ( S 22 − S 21 S 11 − 1 S 12 ) ′ = S 22 ′ − S 21 ′ S 11 − 1 S 12 − S 21 S 11 − 1 S 12 ′ + S 21 S 11 − 1 S 11 ′ S 11 − 1 S 12 . Substitute S ′ = ( T − I ) T S + S ( T − I ) S'=(T-I)^{\mathsf{T}}S+S(T-I) S ′ = ( T − I ) T S + S ( T − I ) T 12 T_{12} T 12 T 11 − I T_{11}-I T 11 − I
H ′ = ( Γ d + E ) T H + H ( Γ d + E ) , H = S 22 − S 21 S 11 − 1 S 12 . (82) H'=(\Gamma_d+E)^{\mathsf{T}}H+H(\Gamma_d+E),\qquad H=S_{22}-S_{21}S_{11}^{-1}S_{12}. \tag*{(82)} H ′ = ( Γ d + E ) T H + H ( Γ d + E ) , H = S 22 − S 21 S 11 − 1 S 12 . (82) This algebraic identity holds throughout that symmetric neighborhood. For actual factors, orthogonal projection gives H = Z T Z ⪰ 0 H=Z^{\mathsf{T}}Z\succeq0 H = Z T Z ⪰ 0
In the coordinates y y y S 22 = H + S 21 S 11 − 1 S 12 S_{22}=H+S_{21}S_{11}^{-1}S_{12} S 22 = H + S 21 S 11 − 1 S 12 rank S = d + rank H \operatorname{rank}S=d+\operatorname{rank}H rank S = d + rank H S 11 ≻ 0 S_{11}\succ0 S 11 ≻ 0 H = 0 H=0 H = 0 J d J_d J d Lemma 35 , Taylor expansion, and Lemma 18 prove the first three relations in (78) ; the squared-normal term comes from H = Z T Z H=Z^{\mathsf{T}}Z H = Z T Z
To obtain a contracting tangent norm for the Hurwitz matrix J d J_d J d
P d = ∫ 0 ∞ e J d T u e J d u d u , ∥ y ∥ = ( y T P d y ) 1 / 2 . P_d=\int_{0}^{\infty}e^{J_d^{\mathsf{T}}u}e^{J_du}\,\mathrm{d}u,\qquad\lVert y\rVert=(y^{\mathsf{T}}P_dy)^{1/2}. P d = ∫ 0 ∞ e J d T u e J d u d u , ∥ y ∥ = ( y T P d y ) 1/2 . The integral converges and is positive definite; integration of its derivative gives J d T P d + P d J d = − I J_d^{\mathsf{T}}P_d+P_dJ_d=-I J d T P d + P d J d = − I y ′ = J d y y'=J_dy y ′ = J d y − ∥ y ∥ 2 -\lVert y\rVert^2 − ∥ y ∥ 2 ω > 0 \omega>0 ω > 0 (78) . These constants are uniform on the fixed chart.
The determinant identity
V h ( Z ) 2 = det ( ( z d + a T z d + b ) a , b = 1 h ) V_h(Z)^2=\det\left((z_{d+a}^\mathsf{T}z_{d+b})_{a,b=1}^{h}\right) V h ( Z ) 2 = det ( ( z d + a T z d + b ) a , b = 1 h ) characterizes column independence. Orthogonal elimination gives V h / V h − 1 V_h/V_{h-1} V h / V h − 1 diag ( ε , ε 3 , ε 6 ) \operatorname{diag}(\varepsilon,\varepsilon^3,\varepsilon^6) diag ( ε , ε 3 , ε 6 ) I + ε e 2 e 3 T I+\varepsilon e_2e_3^{\mathsf{T}} I + ε e 2 e 3 T ( 0 , ε 4 , ε 6 ) T (0,\varepsilon^4,\varepsilon^6)^{\mathsf{T}} ( 0 , ε 4 , ε 6 ) T ε 6 \varepsilon^6 ε 6 Lemma 43 provide two-sided estimates for these independent scales.
The matrix ⋀ h Z \bigwedge^h Z ⋀ h Z h h h
( ⋀ h Z ) ′ = ( ⋀ h Z ) ( Γ d [ h ] + E [ h ] ) , ∥ E [ h ] ∥ o p ≤ h ∥ E ∥ o p . (83) (\bigwedge^h Z)'=(\bigwedge^h Z)(\Gamma_d^{[h]}+E^{[h]}),\qquad\lVert E^{[h]}\rVert_{\mathrm{op}}\le h\lVert E\rVert_{\mathrm{op}}. \tag*{(83)} ( ⋀ h Z ) ′ = ( ⋀ h Z ) ( Γ d [ h ] + E [ h ] ) , ∥ E [ h ] ∥ op ≤ h ∥ E ∥ op . (83) For any square matrix, the superscript [ h ] [h] [ h ] Γ d [ h ] \Gamma_d^{[h]} Γ d [ h ] ∑ l ∈ J g l ( d ) \sum_{l\in J}g_l^{(d)} ∑ l ∈ J g l ( d ) J J J
For use throughout this subsection, let B h B_h B h ⋀ h Z \bigwedge^h Z ⋀ h Z J d , h J_{d,h} J d , h V h ( Z ) V_h(Z) V h ( Z )
D + ∥ B h ∥ F ≤ ( ∑ l ∈ J d , h g l ( d ) − σ d , h ) ∥ B h ∥ F + h ∥ E ∥ o p ( V h ( Z ) + ∥ B h ∥ F ) , D^+\lVert B_h\rVert_F\le\left(\sum_{l\in J_{d,h}}g_l^{(d)}-\sigma_{d,h}\right)\lVert B_h\rVert_F+h\lVert E\rVert_{\mathrm{op}}\left(V_h(Z)+\lVert B_h\rVert_F\right), D + ∥ B h ∥ F ≤ l ∈ J d , h ∑ g l ( d ) − σ d , h ∥ B h ∥ F + h ∥ E ∥ op ( V h ( Z ) + ∥ B h ∥ F ) , D − V h ( Z ) ≥ ( ∑ l ∈ J d , h g l ( d ) ) V h ( Z ) − h ∥ E ∥ o p ( V h ( Z ) + ∥ B h ∥ F ) . (84) D^-V_h(Z)\ge\left(\sum_{l\in J_{d,h}}g_l^{(d)}\right)V_h(Z)-h\lVert E\rVert_{\mathrm{op}}\left(V_h(Z)+\lVert B_h\rVert_F\right). \tag*{(84)} D − V h ( Z ) ≥ l ∈ J d , h ∑ g l ( d ) V h ( Z ) − h ∥ E ∥ op ( V h ( Z ) + ∥ B h ∥ F ) . (84) Here D − D^{-} D −
D + ∥ B h ∥ F V h ( Z ) ≤ − σ d , h ∥ B h ∥ F V h ( Z ) + h ∥ E ∥ op ( 1 + ∥ B h ∥ F V h ( Z ) ) 2 . D^{+}\frac{\lVert B_h\rVert_F}{V_h(Z)}\leq-\sigma_{d,h}\frac{\lVert B_h\rVert_F}{V_h(Z)}+h\lVert E\rVert_{\operatorname{op}}\left(1+\frac{\lVert B_h\rVert_F}{V_h(Z)}\right)^2. D + V h ( Z ) ∥ B h ∥ F ≤ − σ d , h V h ( Z ) ∥ B h ∥ F + h ∥ E ∥ op ( 1 + V h ( Z ) ∥ B h ∥ F ) 2 . Choose the coefficient error small enough that the right side is negative at ratio ζ \zeta ζ h 1 + ζ 2 ∥ E ∥ op h\sqrt{1+\zeta^2}\lVert E\rVert_{\operatorname{op}} h 1 + ζ 2 ∥ E ∥ op
∣ log V h ( Z ( t ) ) V h ( Z ( t a ) ) − ( ∑ ℓ ∈ J d , h g ℓ ( d ) ) log ( t / t a ) ∣ ≤ h 1 + ζ 2 ∫ t a t ∥ E ( s ) ∥ op d s s . (85) \left\lvert\log\frac{V_h(Z(t))}{V_h(Z(t_a))}-\left(\sum_{\ell\in J_{d,h}}g_{\ell}^{(d)}\right)\log(t/t_a)\right\rvert\leq h\sqrt{1+\zeta^2}\int_{t_a}^{t}\lVert E(s)\rVert_{\operatorname{op}}\frac{\mathrm{d}s}{s}. \tag*{(85)} log V h ( Z ( t a )) V h ( Z ( t )) − ℓ ∈ J d , h ∑ g ℓ ( d ) log ( t / t a ) ≤ h 1 + ζ 2 ∫ t a t ∥ E ( s ) ∥ op s d s . (85) This proves (79). If the distinguished group is the sole group, B h B_h B h
A residence interval can have length proportional to log ( 1 / ε ) \log(1/\varepsilon) log ( 1/ ε )
For positive growth, the first cone gives V 1 ( Z ) ≤ ∥ Z ∥ F ≤ 1 + ζ 2 V 1 ( Z ) V_1(Z)\leq\lVert Z\rVert_F\leq\sqrt{1+\zeta^2}V_1(Z) V 1 ( Z ) ≤ ∥ Z ∥ F ≤ 1 + ζ 2 V 1 ( Z ) ( log V 1 ) ′ ≥ g / 2 (\log V_1)'\geq g/2 ( log V 1 ) ′ ≥ g /2 t b t_b t b
∥ Z ( s ) ∥ F ≤ 1 + ζ 2 ∥ Z ( t b ) ∥ F ( s / t b ) g / 2 , t a ≤ s ≤ t b . \lVert Z(s)\rVert_F\leq\sqrt{1+\zeta^2}\lVert Z(t_b)\rVert_F(s/t_b)^{g/2},\qquad t_a\leq s\leq t_b. ∥ Z ( s ) ∥ F ≤ 1 + ζ 2 ∥ Z ( t b ) ∥ F ( s / t b ) g /2 , t a ≤ s ≤ t b . Thus
∫ t a t b ∥ Z ( s ) ∥ F 2 d s s ≤ 1 + ζ 2 g ∥ Z ( t b ) ∥ F 2 . \int_{t_a}^{t_b}\lVert Z(s)\rVert_F^2\frac{\mathrm{d}s}{s}\leq\frac{1+\zeta^2}{g}\lVert Z(t_b)\rVert_F^2. ∫ t a t b ∥ Z ( s ) ∥ F 2 s d s ≤ g 1 + ζ 2 ∥ Z ( t b ) ∥ F 2 . The tangent inequality integrates to
∥ y ( t ) ∥ ≤ ( t a / t ) ω ∥ y ( t a ) ∥ + C ∫ t a t ( s / t ) ω ( ∥ Z ( s ) ∥ F 2 + s 2 + ε / s ) d s s . \lVert y(t)\rVert\leq(t_a/t)^\omega\lVert y(t_a)\rVert+C\int_{t_a}^{t}(s/t)^\omega\left(\lVert Z(s)\rVert_F^2+s^2+\varepsilon/s\right)\frac{\mathrm{d}s}{s}. ∥ y ( t )∥ ≤ ( t a / t ) ω ∥ y ( t a )∥ + C ∫ t a t ( s / t ) ω ( ∥ Z ( s ) ∥ F 2 + s 2 + ε / s ) s d s . A second integration and interchange of nonnegative integrals give
∫ t a t b ∥ y ( t ) ∥ d t t ≤ ∥ y ( t a ) ∥ ω + C ω ∫ t a t b ( ∥ Z ( t ) ∥ F 2 + t 2 + ε / t ) d t t . \int_{t_a}^{t_b}\lVert y(t)\rVert\frac{\mathrm{d}t}{t}\leq\frac{\lVert y(t_a)\rVert}{\omega}+\frac{C}{\omega}\int_{t_a}^{t_b}\left(\lVert Z(t)\rVert_F^2+t^2+\varepsilon/t\right)\frac{\mathrm{d}t}{t}. ∫ t a t b ∥ y ( t )∥ t d t ≤ ω ∥ y ( t a )∥ + ω C ∫ t a t b ( ∥ Z ( t ) ∥ F 2 + t 2 + ε / t ) t d t . The remaining integral is explicit:
∫ t a t b ( t 2 + ε / t ) d t t = t b 2 − t a 2 2 + ε ( 1 t a − 1 t b ) ≤ t ∗ 2 2 + 1 c U . (86) \int_{t_a}^{t_b}(t^2+\varepsilon/t)\frac{\mathrm{d}t}{t}=\frac{t_b^2-t_a^2}{2}+\varepsilon\left(\frac{1}{t_a}-\frac{1}{t_b}\right)\leq\frac{t_*^2}{2}+\frac{1}{c_U}. \tag*{(86)} ∫ t a t b ( t 2 + ε / t ) t d t = 2 t b 2 − t a 2 + ε ( t a 1 − t b 1 ) ≤ 2 t ∗ 2 + c U 1 . (86) The bound for E E E t a ≥ c U ε t_a\geq c_U\varepsilon t a ≥ c U ε ε / t a \varepsilon/t_a ε / t a
For the trace estimate, positivity and the first cone imply
tr ( Γ d H ) ≥ g − ζ 2 ∥ Γ d ∥ op 1 + ζ 2 tr H . \operatorname{tr}(\Gamma_dH)\geq\frac{g-\zeta^2\lVert\Gamma_d\rVert_{\operatorname{op}}}{1+\zeta^2}\operatorname{tr}H. tr ( Γ d H ) ≥ 1 + ζ 2 g − ζ 2 ∥ Γ d ∥ op tr H . Choose ζ \zeta ζ (82) and ∣ tr ( H E ) ∣ ≤ ∥ E ∥ o p tr H |\operatorname{tr}(HE)| \le\|E\|_{\mathrm{op}}\operatorname{tr} H ∣ tr ( H E ) ∣ ≤ ∥ E ∥ op tr H E E E Γ d + E \Gamma_d+E Γ d + E H H H
For the final norm estimate choose ν > 0 \nu> 0 ν > 0 ϵ − ( g + ν ) θ > 0 \epsilon-(g+\nu)\theta>0 ϵ − ( g + ν ) θ > 0 ∥ E ∥ o p ≤ ν \|E\|_{\mathrm{op}}\le\nu ∥ E ∥ op ≤ ν
D + ∥ Z ∥ F ≤ ( g + ν ) ∥ Z ∥ F , ∥ Z ( t ) ∥ F ≤ C U ϵ e ( t / t a ) g + ν . D^{+}\|Z\|_{F}\le(g+\nu)\|Z\|_{F},\qquad\|Z(t)\|_{F}\le C_{U}\epsilon^{e(t/t_a)^{g+\nu}}. D + ∥ Z ∥ F ≤ ( g + ν ) ∥ Z ∥ F , ∥ Z ( t ) ∥ F ≤ C U ϵ e ( t / t a ) g + ν . If g + ν ≥ 0 g+\nu\ge0 g + ν ≥ 0 O U ( ϵ e − ( g + ν ) θ ) O_{U}(\epsilon^{e^{-(g+\nu)\theta}}) O U ( ϵ e − ( g + ν ) θ ) g + ν < 0 g+\nu<0 g + ν < 0 C U ϵ C_{U}\epsilon C U ϵ ϵ \epsilon ϵ O U ( log ( 1 / ϵ ) ) O_{U}(\log(1/\epsilon)) O U ( log ( 1/ ϵ )) t ∗ t_{*} t ∗ ϵ / t a \epsilon/t_a ϵ / t a λ l \lambda_l λ l g l ( d ) < − 1 g_l^{(d)}<-1 g l ( d ) < − 1 ∥ E ∥ o p ≤ 1 / 2 \|E\|_{\mathrm{op}}\le1/2 ∥ E ∥ op ≤ 1/2
D + ∥ Z ∥ F ≤ − ∥ Z ∥ F / 2 , D^{+}\|Z\|_{F}\le-\|Z\|_{F}/2, D + ∥ Z ∥ F ≤ − ∥ Z ∥ F /2 , whose squared integral is bounded directly. For an empty normal block, set Z = 0 Z=0 Z = 0
First sections and propagation through one passage
A single outgoing section must work for the actual trajectories and for the limiting exit. Transversality fixes its first crossing, after which projection and a positive relative transfer preserve the remaining independent exponents.
Lemma 40 (Uniform first-section limit). Suppose g d + 1 ( d ) > 0 g_{d+1}^{(d)}>0 g d + 1 ( d ) > 0 ζ > 0 \zeta>0 ζ > 0 C 0 > 0 C_0>0 C 0 > 0 h 0 > 0 h_0>0 h 0 > 0 0 < h < h 0 0<h<h_0 0 < h < h 0
A h = { ( y , H ) : H ⪰ 0 , tr H = h 2 , H d + 1 , d + 1 ≥ h 2 1 + ζ 2 , ∥ y ∥ ≤ C 0 h 2 } . (87) \mathcal{A}_h=\left\{(y,H):H\succeq0,\ \operatorname{tr}H=h^2,\ H_{d+1,d+1}\ge\frac{h^2}{1+\zeta^2},\ \|y\|\le C_0h^2\right\}. \tag*{(87)} A h = { ( y , H ) : H ⪰ 0 , tr H = h 2 , H d + 1 , d + 1 ≥ 1 + ζ 2 h 2 , ∥ y ∥ ≤ C 0 h 2 } . (87) There is no rank restriction on H H H A h \mathcal{A}_h A h tr H = h 0 2 \operatorname{tr}H=h_0^2 tr H = h 0 2 h ↓ 0 h\downarrow0 h ↓ 0 A h \mathcal{A}_h A h C 0 C_0 C 0 Q d , K Q_{d,K} Q d , K K K K h h h
Proof (of Lemma 40).
Identification of the connecting Gram trajectory.
Let e l e_l e l R r − d \mathbb{R}^{r-d} R r − d l l l ( y , H ) (y,H) ( y , H )
\begin{pmatrix}
J_d & [D_Hy']_{(0,0)}\\
0 & [H\mapsto\Gamma_dH+H\Gamma_d]
\end{pmatrix}. \tag*{(88)}\end{pmatrix}
The lower eigenvalues are g l ( d ) + g k ( d ) g_l^{(d)}+g_k^{(d)} g l ( d ) + g k ( d ) g d + 1 ( d ) g_{d+1}^{(d)} g d + 1 ( d ) 2 g d + 1 ( d ) 2g_{d+1}^{(d)} 2 g d + 1 ( d )
( y , H ) ↦ H d + 1 , d + 1 . (89) (y,H)\mapsto H_{d+1,d+1}. \tag*{(89)} ( y , H ) ↦ H d + 1 , d + 1 . (89) The right eigenvector normalized by H d + 1 , d + 1 = 1 H_{d+1,d+1}=1 H d + 1 , d + 1 = 1
( ( 2 g d + 1 ( d ) I − J d ) − 1 [ D H y ′ ] ( 0 , 0 ) ( e d + 1 e d + 1 T ) , e d + 1 e d + 1 T ) \left((2g_{d+1}^{(d)}I-J_d)^{-1}[DH y']_{(0,0)}(e_{d+1}e_{d+1}^{\mathsf T}),e_{d+1}e_{d+1}^{\mathsf T}\right) ( ( 2 g d + 1 ( d ) I − J d ) − 1 [ D H y ′ ] ( 0 , 0 ) ( e d + 1 e d + 1 T ) , e d + 1 e d + 1 T ) The connecting trajectory of Lemma 36 gives this Gram solution, up to time translation. Indeed its new factor amplitude is proportional to e g d + 1 ( d ) v e^{g_{d+1}^{(d)}v} e g d + 1 ( d ) v v → − ∞ v\to-\infty v → − ∞ H d + 1 , d + 1 H_{d+1,d+1} H d + 1 , d + 1 e 2 g d + 1 ( d ) v e^{2g_{d+1}^{(d)}v} e 2 g d + 1 ( d ) v
The simplicity of 2 g d + 1 ( d ) 2g_{d+1}^{(d)} 2 g d + 1 ( d ) m − d m-d m − d
For H ⪰ 0 H\succeq0 H ⪰ 0 H = Z T Z H=Z^{\mathsf T}Z H = Z T Z (87) is the first column cone in Gram form, since tr H = ∥ Z ∥ F 2 \operatorname{tr}H=\lVert Z\rVert_{F}^{2} tr H = ∥ Z ∥ F 2 H d + 1 , d + 1 = ∥ z d + 1 ∥ 2 H_{d+1,d+1}=\lVert z_{d+1}\rVert^{2} H d + 1 , d + 1 = ∥ z d + 1 ∥ 2 ζ \zeta ζ ∥ Γ d ∥ o p ζ 2 < g d + 1 ( d ) / 4 \lVert\Gamma_d\rVert_{\mathrm{op}}\zeta^{2}<g_{d+1}^{(d)}/4 ∥ Γ d ∥ op ζ 2 < g d + 1 ( d ) /4 g d + 1 ( d ) g_{d+1}^{(d)} g d + 1 ( d )
tr ( Γ d H ) ≥ g d + 1 ( d ) H d + 1 , d + 1 − ∥ Γ d ∥ o p ∑ ℓ ≠ d + 1 H ℓ ℓ ≥ ( g d + 1 ( d ) − ∥ Γ d ∥ o p ζ 2 ) H d + 1 , d + 1 . \operatorname{tr}(\Gamma_dH)\ge g_{d+1}^{(d)}H_{d+1,d+1}-\lVert\Gamma_d\rVert_{\mathrm{op}}\sum_{\ell\ne d+1}H_{\ell\ell}\ge\left(g_{d+1}^{(d)}-\lVert\Gamma_d\rVert_{\mathrm{op}}\zeta^{2}\right)H_{d+1,d+1}. tr ( Γ d H ) ≥ g d + 1 ( d ) H d + 1 , d + 1 − ∥ Γ d ∥ op ℓ = d + 1 ∑ H ℓℓ ≥ ( g d + 1 ( d ) − ∥ Γ d ∥ op ζ 2 ) H d + 1 , d + 1 . Differentiating H = Z T Z H=Z^{\mathsf T}Z H = Z T Z Lemma 39 gives H ′ = ( Γ d + E ) T H + H ( Γ d + E ) H'=(\Gamma_d+E)^{\mathsf T}H+H(\Gamma_d+E) H ′ = ( Γ d + E ) T H + H ( Γ d + E ) 2 tr ( H E ) 2\operatorname{tr}(HE) 2 tr ( H E ) 2 ∥ E ∥ o p tr H 2\lVert E\rVert_{\mathrm{op}}\operatorname{tr}H 2 ∥ E ∥ op tr H H H H ( tr H ) ′ ≥ ( g d + 1 ( d ) / 2 ) tr H (\operatorname{tr}H)'\ge(g_{d+1}^{(d)}/2)\operatorname{tr}H ( tr H ) ′ ≥ ( g d + 1 ( d ) /2 ) tr H Lemma 39 . Cone invariance follows from Lemma 39 . The stable tangent estimate keeps y y y tr H ≤ h 0 2 \operatorname{tr}H\le h_{0}^{2} tr H ≤ h 0 2 C 0 h 2 C_{0}h^{2} C 0 h 2 tr H \operatorname{tr}H tr H
For Lemma 22 , the field must be C 2 C^{2} C 2 (77) gives an analytic extension to an open symmetric-matrix neighborhood. A smooth compactly supported cutoff supplies a complete auxiliary field agreeing with it on the chart. Equations (88) –(89) give the largest eigenvalue 2 g d + 1 ( d ) 2g_{d+1}^{(d)} 2 g d + 1 ( d ) H d + 1 , d + 1 H_{d+1,d+1} H d + 1 , d + 1 A h \mathcal A_h A h
∥ ( y , H ) ∥ ≤ ( C 0 + 1 ) h 2 , H d + 1 , d + 1 ≥ h 2 1 + ζ 2 . \lVert(y,H)\rVert\le(C_0+1)h^{2},\qquad H_{d+1,d+1}\ge\frac{h^{2}}{1+\zeta^{2}}. ∥( y , H )∥ ≤ ( C 0 + 1 ) h 2 , H d + 1 , d + 1 ≥ 1 + ζ 2 h 2 . In particular
H d + 1 , d + 1 ∥ ( y , H ) ∥ ≥ 1 ( 1 + ζ 2 ) ( C 0 + 1 ) > 0 , \frac{H_{d+1,d+1}}{\lVert(y,H)\rVert}\ge\frac{1}{(1+\zeta^{2})(C_0+1)}>0, ∥( y , H )∥ H d + 1 , d + 1 ≥ ( 1 + ζ 2 ) ( C 0 + 1 ) 1 > 0 , which is the required relative projection bound independent of h h h
Apply Lemma 22 with phase shift ( 2 g d + 1 ( d ) ) − 1 log ( 1 / H d + 1 , d + 1 ) (2g_{d+1}^{(d)})^{-1}\log(1/H_{d+1,d+1}) ( 2 g d + 1 ( d ) ) − 1 log ( 1/ H d + 1 , d + 1 ) A h \mathcal{A}_h A h tr H = h 0 2 \operatorname{tr} H=h_0^2 tr H = h 0 2
Lemma 41 (Actual first-section limit) Fix Q d , K Q_{d,K} Q d , K g d + 1 ( d ) > 0 g_{d+1}^{(d)}>0 g d + 1 ( d ) > 0 C 0 C_0 C 0 Lemma 40 at least as large as the uniform coefficient of h 2 h^2 h 2 Lemma 39 . Then choose h 0 h_0 h 0 C 0 C_0 C 0 Lemma 40 . Consider any actual trajectory family on [ t a ε , t b ε ] [t_a^\varepsilon,t_b^\varepsilon] [ t a ε , t b ε ] y ( t a ε ) y(t_a^\varepsilon) y ( t a ε )
0 < ∥ Z ( t a ε ) ∥ F ⟶ 0 , t a ε ≥ c U ε 0<\lVert Z(t_a^\varepsilon)\rVert_F\longrightarrow0,\qquad t_a^\varepsilon\ge c_U\varepsilon 0 < ∥ Z ( t a ε ) ∥ F ⟶ 0 , t a ε ≥ c U ε for fixed c U > 0 c_U>0 c U > 0 t b ε t_b^\varepsilon t b ε ∥ Z ∥ F = h 0 \lVert Z\rVert_F=h_0 ∥ Z ∥ F = h 0
t b ε ⟶ 0 , ε / t b ε ⟶ 0. t_b^\varepsilon\longrightarrow0,\qquad\varepsilon/t_b^\varepsilon\longrightarrow0. t b ε ⟶ 0 , ε / t b ε ⟶ 0. The outgoing Gram converges to the Gram matrix at the first crossing of the connecting solution of Lemma 36 . With that crossing assigned phase zero in logarithmic time, convergence holds on each fixed compact phase interval within the coordinate neighborhood or its fixed continuation segment. For every fixed 0 < h < h 0 0<h<h_0 0 < h < h 0 t h ε t_h^\varepsilon t h ε ∥ Z ∥ F = h \lVert Z\rVert_F=h ∥ Z ∥ F = h C ≥ 1 C\geq1 C ≥ 1 h h h
lim sup ε ↓ 0 ∥ y ( t h ε ) ∥ ≤ C h 2 , H d + 1 , d + 1 ( t h ε ) ≥ h 2 1 + ζ 2 . (90) \limsup_{\varepsilon\downarrow0}\lVert y(t_h^\varepsilon)\rVert\leq Ch^2,\qquad H_{d+1,d+1}(t_h^\varepsilon)\geq\frac{h^2}{1+\zeta^2}. \tag*{(90)} ε ↓ 0 lim sup ∥ y ( t h ε )∥ ≤ C h 2 , H d + 1 , d + 1 ( t h ε ) ≥ 1 + ζ 2 h 2 . (90) and throughout the residence
C − 1 tr H ≤ d d log t tr H ≤ C tr H . (91) C^{-1}\operatorname{tr}H\leq\frac{d}{d\log t}\operatorname{tr}H\leq C\operatorname{tr}H. \tag*{(91)} C − 1 tr H ≤ d log t d tr H ≤ C tr H . (91) Proof (of Lemma 41). Fix a coordinate neighborhood and the contraction rate ω > 0 \omega>0 ω > 0 Lemma 39 . Use a fixed C ≥ 1 C\geq1 C ≥ 1 ∥ Z ∥ F = h \lVert Z\rVert_F=h ∥ Z ∥ F = h ∥ Z ( v ) ∥ F 2 ≤ h 2 e ( v − log t h ε ) / C \lVert Z(v)\rVert_F^2\leq h^2e^{(v-\log t_h^\varepsilon)/C} ∥ Z ( v ) ∥ F 2 ≤ h 2 e ( v − l o g t h ε ) / C C h 2 / ( ω + C − 1 ) Ch^2/(\omega+C^{-1}) C h 2 / ( ω + C − 1 ) C 0 C_0 C 0 h 0 h_0 h 0 Lemma 40 . Shrinking this outgoing level preserves the constants fixed on the larger neighborhood. The remaining incoming and nonautonomous terms are shown below to vanish at every fixed h h h Lemma 39 compare ∥ Z ∥ F \lVert Z\rVert_F ∥ Z ∥ F 0 < h < h 0 0<h<h_0 0 < h < h 0 t h ε t_h^\varepsilon t h ε ∥ Z ∥ F = h \lVert Z\rVert_F=h ∥ Z ∥ F = h h h h h 0 h_0 h 0 h , h 0 h,h_0 h , h 0 t h ε / t b ε t_h^\varepsilon/t_b^\varepsilon t h ε / t b ε h h h
t h ε → 0 , ε / t h ε → 0 , t a ε / t h ε → 0. t_h^\varepsilon\to0,\qquad\varepsilon/t_h^\varepsilon\to0,\qquad t_a^\varepsilon/t_h^\varepsilon\to0. t h ε → 0 , ε / t h ε → 0 , t a ε / t h ε → 0. The first cone at the auxiliary crossing gives
H d + 1 , d + 1 = ∥ z d + 1 ∥ 2 ≥ h 2 1 + ζ 2 . (92) H_{d+1,d+1}=\lVert z_{d+1}\rVert^2\ge\frac{h^2}{1+\zeta^2}. \tag*{(92)} H d + 1 , d + 1 = ∥ z d + 1 ∥ 2 ≥ 1 + ζ 2 h 2 . (92) We next estimate the tangent coordinates at this crossing. The backward normal estimate from Lemma 39, now ending at level h h h C h 2 Ch^2 C h 2 C C C h h h C ( t a ε / t h ε ) ω C(t_a^\varepsilon/t_h^\varepsilon)^\omega C ( t a ε / t h ε ) ω t 2 t^2 t 2 C ( t h ε ) 2 C(t_h^\varepsilon)^2 C ( t h ε ) 2
C ε ( t h ε ) − ω ∫ t a ε t h ε s ω − 2 d s . C_\varepsilon(t_h^\varepsilon)^{-\omega}\int_{t_a^\varepsilon}^{t_h^\varepsilon}s^{\omega-2}\,ds. C ε ( t h ε ) − ω ∫ t a ε t h ε s ω − 2 d s . For 0 < ω < 1 0<\omega<1 0 < ω < 1 ( ε / t h ε ) ( t h ε / t a ε ) ω (\varepsilon/t_h^\varepsilon)(t_h^\varepsilon/t_a^\varepsilon)^\omega ( ε / t h ε ) ( t h ε / t a ε ) ω ω > 1 \omega>1 ω > 1 O ( ε / t a ε ) O(\varepsilon/t_a^\varepsilon) O ( ε / t a ε ) ω = 1 \omega=1 ω = 1 O ( ( ε / t h ε ) log ( t h ε / t a ε ) ) = o ( 1 ) O((\varepsilon/t_h^\varepsilon)\log(t_h^\varepsilon/t_a^\varepsilon))=o(1) O (( ε / t h ε ) log ( t h ε / t a ε )) = o ( 1 ) t a ε ≥ c U ε t_a^\varepsilon\ge c_U\varepsilon t a ε ≥ c U ε ∥ y ( t h ε ) ∥ ≤ C h 2 + o U , ε ( 1 ) \lVert y(t_h^\varepsilon)\rVert\le Ch^2+o_{U,\varepsilon}(1) ∥ y ( t h ε )∥ ≤ C h 2 + o U , ε ( 1 ) h h h
∥ y ∥ ≤ C 0 h 2 , H d + 1 , d + 1 ≥ h 2 / ( 1 + ζ 2 ) > 0 , (93) \lVert y\rVert\le C_0h^2,\qquad H_{d+1,d+1}\ge h^2/(1+\zeta^2)>0, \tag*{(93)} ∥ y ∥ ≤ C 0 h 2 , H d + 1 , d + 1 ≥ h 2 / ( 1 + ζ 2 ) > 0 , (93) and lies in (87) with a deterministic constant C 0 C_0 C 0
Take any sequence ε n ↓ 0 \varepsilon_n\downarrow0 ε n ↓ 0 h h h t t t t b ε n t_b^{\varepsilon_n} t b ε n t 2 + ε n / t → 0 t^2+\varepsilon_n/t\to0 t 2 + ε n / t → 0 C 1 C^1 C 1 [31 ] (Lemma 2.7) identify the outgoing limit as the first limiting-equation exit of that predecessor. Transversality identifies the limiting exit as the first crossing of the section.
Lemma 40 bounds its distance to the same crossing of the connecting solution by a quantity tending to zero with h h h h ↓ 0 h\downarrow0 h ↓ 0 h h h ε \varepsilon ε
Lemma 42 (Relative compound transfer) Let V V V V 0 V_0 V 0 ∥ V ∥ F 2 ≤ ( 1 + ζ 2 ) ∥ V 0 ∥ 2 \lVert V\rVert_F^2\le(1+\zeta^2)\lVert V_0\rVert^2 ∥ V ∥ F 2 ≤ ( 1 + ζ 2 ) ∥ V 0 ∥ 2 D D D F F F 0 0 0
∥ ( V ( D + F ) ) 0 ∥ ≥ [ D 00 − 1 + ζ 2 ∥ F ∥ o p ] ∥ V 0 ∥ . (94) \lVert(V(D+F))_0\rVert\ge\left[D_{00}-\sqrt{1+\zeta^2}\lVert F\rVert_{\mathrm{op}}\right]\lVert V_0\rVert. \tag*{(94)} ∥( V ( D + F ) ) 0 ∥ ≥ [ D 00 − 1 + ζ 2 ∥ F ∥ op ] ∥ V 0 ∥ . (94) Proof (of Lemma 42). The output column equals D 00 V 0 + V F e 0 D_{00}V_0+VFe_0 D 00 V 0 + V F e 0
∥ D 00 V 0 + V F e 0 ∥ ≥ D 00 ∥ V 0 ∥ − ∥ V ∥ o p ∥ F ∥ o p . \left\|D_{00}V_0+VFe_0\right\|\ge D_{00}\left\|V_0\right\|-\left\|V\right\|_{\mathrm{op}}\left\|F\right\|_{\mathrm{op}}. ∥ D 00 V 0 + V F e 0 ∥ ≥ D 00 ∥ V 0 ∥ − ∥ V ∥ op ∥ F ∥ op . The hypothesis implies ∥ V ∥ o p ≤ ∥ V ∥ F ≤ 1 + ζ 2 ∥ V 0 ∥ \left\|V\right\|_{\mathrm{op}}\le\left\|V\right\|_{F}\le\sqrt{1+\zeta^2}\left\|V_0\right\| ∥ V ∥ op ≤ ∥ V ∥ F ≤ 1 + ζ 2 ∥ V 0 ∥
The next statement applies these local estimates to incoming powers of ε \varepsilon ε
Lemma 43 (Propagation of independent scales through an actual passage) Fix Q d , K Q_{d,K} Q d , K F K ( Q d , K ) = 0 F_K(Q_{d,K})=0 F K ( Q d , K ) = 0 h max h_{\max} h m a x
1 ≤ h max ≤ min { m − d , r − d } . 1\le h_{\max}\le\min\{m-d,r-d\}. 1 ≤ h m a x ≤ min { m − d , r − d } . Assume the active spectral boundary is strict and that each retained leading group is separated:
λ d > λ d + 1 , λ d + h > λ d + h + 1 for 1 ≤ h ≤ h max with d + h < r . \lambda_d>\lambda_{d+1},\qquad\lambda_{d+h}>\lambda_{d+h+1}\quad\text{for }1\le h\le h_{\max}\text{ with }d+h<r. λ d > λ d + 1 , λ d + h > λ d + h + 1 for 1 ≤ h ≤ h m a x with d + h < r . Suppose an actual trajectory enters a sufficiently small neighbourhood of Q d , K Q_{d,K} Q d , K t i n ε t_{\mathrm{in}}^\varepsilon t in ε
log t i n ε = ( 1 − x ) log ε + O U ( 1 ) , 0 ≤ x < 1 , \log t_{\mathrm{in}}^\varepsilon=(1-x)\log\varepsilon+O_U(1),\qquad0\le x<1, log t in ε = ( 1 − x ) log ε + O U ( 1 ) , 0 ≤ x < 1 , and ε / t i n ε \varepsilon/t_{\mathrm{in}}^\varepsilon ε / t in ε
log V h ( Z ε ( t i n ε ) ) V h − 1 ( Z ε ( t i n ε ) ) = β h log ε + O U ( 1 ) , 0 < β 1 ≤ ⋯ ≤ β h max . \log\frac{V_h(Z^\varepsilon(t_{\mathrm{in}}^\varepsilon))}{V_{h-1}(Z^\varepsilon(t_{\mathrm{in}}^\varepsilon))}=\beta_h\log\varepsilon+O_U(1),\qquad0<\beta_1\le\cdots\le\beta_{h_{\max}}. log V h − 1 ( Z ε ( t in ε )) V h ( Z ε ( t in ε )) = β h log ε + O U ( 1 ) , 0 < β 1 ≤ ⋯ ≤ β h m a x . Assume also that each retained leading volume satisfies
∥ ⋀ h Z ε ( t i n ε ) ∥ F 2 ≤ ( 1 + ζ 2 ) V h ( Z ε ( t i n ε ) ) 2 , 1 ≤ h ≤ h max , \left\|\bigwedge^h Z^\varepsilon(t_{\mathrm{in}}^\varepsilon)\right\|_F^2\le(1+\zeta^2)V_h(Z^\varepsilon(t_{\mathrm{in}}^\varepsilon))^2,\qquad1\le h\le h_{\max}, ⋀ h Z ε ( t in ε ) F 2 ≤ ( 1 + ζ 2 ) V h ( Z ε ( t in ε ) ) 2 , 1 ≤ h ≤ h m a x , for a sufficiently small fixed ζ > 0 \zeta>0 ζ > 0
Put g = g d + 1 ( d ) g=g_{d+1}^{(d)} g = g d + 1 ( d ) ε \varepsilon ε
log V h ( Z ε ( t ) ) = ( ∑ a = 1 h β a ) log ε + ( ∑ a = 1 h g d + a ( d ) ) log t t i n ε + O U ( 1 ) . \log V_h(Z^\varepsilon(t))=\left(\sum_{a=1}^{h}\beta_a\right)\log\varepsilon+\left(\sum_{a=1}^{h}g_{d+a}^{(d)}\right)\log\frac{t}{t_{\mathrm{in}}^\varepsilon}+O_U(1). log V h ( Z ε ( t )) = ( a = 1 ∑ h β a ) log ε + ( a = 1 ∑ h g d + a ( d ) ) log t in ε t + O U ( 1 ) . The error is uniform over the passage interval. The integrated bounds of Lemma 39 give
∫ passage interval ( ∥ y ∥ + ∥ Z ∥ F 2 + ∥ E ∥ o p ) d log t ≤ C U , (95) \int_{\text{passage interval}}\left(\|y\|+\|Z\|_F^2+\|E\|_{\mathrm{op}}\right)\,d\log t\le C_U, \tag*{(95)} ∫ passage interval ( ∥ y ∥ + ∥ Z ∥ F 2 + ∥ E ∥ op ) d log t ≤ C U , (95) The interval ends at the outgoing time in the first case and at a fixed sufficiently small t ∗ > 0 t_*>0 t ∗ > 0
If g > 0 g>0 g > 0 x + β 1 / g < 1 x+\beta_1/g<1 x + β 1 / g < 1 ∥ Z ε ∥ F = h d > 0 \lVert Z^\varepsilon\rVert_F=h_d>0 ∥ Z ε ∥ F = h d > 0 t o u t ε t^\varepsilon_{\mathrm{out}} t out ε
log t o u t ε = ( 1 − x − β 1 g ) log ε + O U ( 1 ) . \log t^\varepsilon_{\mathrm{out}}=\left(1-x-\frac{\beta_1}{g}\right)\log\varepsilon+O_U(1). log t out ε = ( 1 − x − g β 1 ) log ε + O U ( 1 ) . The crossing is transverse. After shifting its logarithmic time so that the crossing occurs at zero, the actual Gram converges on each fixed compact time interval to Q ( v ) ⊤ Q ( v ) Q(v)^\top Q(v) Q ( v ) ⊤ Q ( v ) Q ( v ) Q(v) Q ( v ) Q d , K Q_{d,K} Q d , K Q d + 1 , K Q_{d+1,K} Q d + 1 , K
β a + 1 − g d + a + 1 ( d ) β 1 g , 1 ≤ a < h max . \beta_{a+1}-g^{(d)}_{d+a+1}\frac{\beta_1}{g},\qquad1\leq a<h_{\max}. β a + 1 − g d + a + 1 ( d ) g β 1 , 1 ≤ a < h m a x . Each retained volume at the next entry is bounded above and below by positive constants times its outgoing projected volume, uniformly in ε \varepsilon ε
If
β 1 − g ( 1 − x ) > 0 , \beta_1-g(1-x)>0, β 1 − g ( 1 − x ) > 0 , fix ν > 0 \nu>0 ν > 0 β 1 − ( g + ν ) ( 1 − x ) > 0 \beta_1-(g+\nu)(1-x)>0 β 1 − ( g + ν ) ( 1 − x ) > 0 ε / t i n ε \varepsilon/t^\varepsilon_{\mathrm{in}} ε / t in ε t ∗ t_* t ∗ ν \nu ν
∥ Z ε ( t ) ∥ F ≤ C U ε β 1 ( t / t i n ε ) g + ν , t i n ε ≤ t ≤ t ∗ . \lVert Z^\varepsilon(t)\rVert_F\leq CU\varepsilon^{\beta_1}(t/t^\varepsilon_{\mathrm{in}})^{g+\nu},\qquad t^\varepsilon_{\mathrm{in}}\leq t\leq t_*. ∥ Z ε ( t ) ∥ F ≤ C U ε β 1 ( t / t in ε ) g + ν , t in ε ≤ t ≤ t ∗ . In particular, the supremum on this interval tends to zero as a positive power of ε \varepsilon ε d log t \mathrm{d}\log t d log t t ∗ t_* t ∗
∥ Z ( t ) ∥ F ≤ ∥ Z ( t i n ε ) ∥ F ( t / t i n ε ) − 1 / 2 , t i n ε ≤ t ≤ t ∗ . \lVert Z(t)\rVert_F\leq\lVert Z(t^\varepsilon_{\mathrm{in}})\rVert_F(t/t^\varepsilon_{\mathrm{in}})^{-1/2},\qquad t^\varepsilon_{\mathrm{in}}\leq t\leq t_*. ∥ Z ( t ) ∥ F ≤ ∥ Z ( t in ε ) ∥ F ( t / t in ε ) − 1/2 , t in ε ≤ t ≤ t ∗ . Thus the unused component tends to zero at every fixed t ∈ ( 0 , t ∗ ] t\in(0,t_*] t ∈ ( 0 , t ∗ ] Z = 0 Z=0 Z = 0
Proof (of Lemma 43). Put g = g d + 1 ( d ) g=g^{(d)}_{d+1} g = g d + 1 ( d )
Existence and time of the actual crossing.
Suppose first that g > 0 g>0 g > 0 x + β 1 / g < 1 x+\beta_1/g<1 x + β 1 / g < 1 t ∗ t_* t ∗ h = 1 h=1 h = 1 h = 1 h=1 h = 1 ( log V 1 ) ′ ≥ g / 2 (\log V_1)'\geq g/2 ( log V 1 ) ′ ≥ g /2 ∥ Z ∥ F ≤ h d \lVert Z\rVert_F \le h_d ∥ Z ∥ F ≤ h d h d h_d h d t ∗ t_* t ∗ (79) and Lemma 39 therefore give, on this interval,
log V h ( Z ε ( t ) ) = ( ∑ a = 1 h β a ) log ε + ( ∑ a = 1 h g d + a ( d ) ) log ( t / t i n ε ) + O U ( 1 ) . \log V_h(Z^\varepsilon(t)) = \left(\sum_{a=1}^{h} \beta_a\right) \log\varepsilon+ \left(\sum_{a=1}^{h} g_{d+a}^{(d)}\right) \log(t/t_{\mathrm{in}}^\varepsilon) + O_U(1). log V h ( Z ε ( t )) = ( a = 1 ∑ h β a ) log ε + ( a = 1 ∑ h g d + a ( d ) ) log ( t / t in ε ) + O U ( 1 ) . If g > 0 g > 0 g > 0 x + β 1 / g < 1 x + \beta_1/g < 1 x + β 1 / g < 1 t ∗ t_* t ∗
log V 1 ( Z ε ( t ∗ ) ) = { β 1 − g ( 1 − x ) } log ε + O U ( 1 ) ⟶ + ∞ , \log V_1(Z^\varepsilon(t_*)) = \{\beta_1 - g(1-x)\}\log\varepsilon+ O_U(1) \longrightarrow+\infty, log V 1 ( Z ε ( t ∗ )) = { β 1 − g ( 1 − x )} log ε + O U ( 1 ) ⟶ + ∞ , contradicting the stopped normal bound. Thus a first crossing occurs. There h d / 1 + ζ 2 ≤ V 1 ≤ h d h_d/\sqrt{1+\zeta^2} \le V_1 \le h_d h d / 1 + ζ 2 ≤ V 1 ≤ h d
log ( t o u t ε / t i n ε ) = β 1 g log ( 1 / ε ) + O U ( 1 ) . (96) \log(t_{\mathrm{out}}^\varepsilon/t_{\mathrm{in}}^\varepsilon) = \frac{\beta_1}{g}\log(1/\varepsilon) + O_U(1). \tag*{(96)} log ( t out ε / t in ε ) = g β 1 log ( 1/ ε ) + O U ( 1 ) . (96) This proves the outgoing-time formula and gives t o u t ε → 0 t_{\mathrm{out}}^\varepsilon\to0 t out ε → 0 ε / t o u t ε → 0 \varepsilon/t_{\mathrm{out}}^\varepsilon\to0 ε / t out ε → 0 β 1 > 0 \beta_1 > 0 β 1 > 0 Lemma 41 have now been verified, so it identifies the outgoing Gram and the limit on each fixed phase interval. The trace-growth inequality makes this crossing transverse.
Use the complementary exterior columns B h B_h B h Lemma 39 . By Lemma 41 , the time-shifted actual Gram paths converge to the diagonal connecting Gram path on every fixed backward window [ − A , 0 ] [-A,0] [ − A , 0 ] T 22 − I − T 21 S 11 − 1 S 12 T_{22}-I-T_{21}S_{11}^{-1}S_{12} T 22 − I − T 21 S 11 − 1 S 12 t 2 t^2 t 2 ε / t \varepsilon/t ε / t ∥ Q R t ∥ F 2 = tr ( R t S R t ) \lVert QR_t\rVert_F^2=\operatorname{tr}(R_tSR_t) ∥ Q R t ∥ F 2 = tr ( R t S R t ) E − diag ( E ) E-\operatorname{diag}(E) E − diag ( E ) Q d , K Q_{d,K} Q d , K
Within the coarse cone, define the ratio r h = ∥ B h ∥ F / V h ( Z ) r_h=\lVert B_h\rVert_F/V_h(Z) r h = ∥ B h ∥ F / V h ( Z )
D + r h ≤ − σ d , h r h / 2 + h ( 1 + ζ ) 2 ∥ E − diag ( E ) ∥ o p . D^+r_h \le-\sigma_{d,h}r_h/2 + h(1+\zeta)^2\lVert E-\operatorname{diag}(E)\rVert_{\mathrm{op}}. D + r h ≤ − σ d , h r h /2 + h ( 1 + ζ ) 2 ∥ E − diag ( E ) ∥ op . Hence
r h ( 0 ) ≤ ζ e − σ d , h A / 2 + h ( 1 + ζ ) 2 σ d , h / 2 ∫ − A 0 e σ d , h u / 2 ∥ E ( u ) − diag ( E ( u ) ) ∥ o p d u ⟶ 0 , (97) r_h(0) \le\zeta e^{-\sigma_{d,h}A/2} + \frac{h(1+\zeta)^2}{\sigma_{d,h}/2}\int_{-A}^{0}e^{\sigma_{d,h}u/2}\lVert E(u)-\operatorname{diag}(E(u))\rVert_{\mathrm{op}}\,du \longrightarrow0, \tag*{(97)} r h ( 0 ) ≤ ζ e − σ d , h A /2 + σ d , h /2 h ( 1 + ζ ) 2 ∫ − A 0 e σ d , h u /2 ∥ E ( u ) − diag ( E ( u )) ∥ op d u ⟶ 0 , (97) where ε ↓ 0 \varepsilon\downarrow0 ε ↓ 0 A A A A → ∞ A \to\infty A → ∞
At the outgoing section adjoin z d + 1 ≠ 0 z_{d+1}\ne0 z d + 1 = 0 J J J
∥ z d + 1 ∧ ⋀ ℓ ∈ J z ℓ ∥ = ∥ z d + 1 ∥ ∥ ⋀ ℓ ∈ J ( I − z d + 1 z d + 1 T ∥ z d + 1 ∥ 2 ) z ℓ ∥ . (98) \left\lVert z_{d+1}\wedge\bigwedge_{\ell\in J}z_\ell\right\rVert
=
\lVert z_{d+1}\rVert
\left\lVert\bigwedge_{\ell\in J}\left(I-\frac{z_{d+1}z_{d+1}^{\mathsf T}}{\lVert z_{d+1}\rVert^2}\right)z_\ell\right\rVert. \tag*{(98)} z d + 1 ∧ ℓ ∈ J ⋀ z ℓ = ∥ z d + 1 ∥ ℓ ∈ J ⋀ ( I − ∥ z d + 1 ∥ 2 z d + 1 z d + 1 T ) z ℓ . (98) Indeed, choose the first basis vector along z d + 1 z_{d+1} z d + 1 h h h V h + 1 ( Z ) / V 1 ( Z ) V_{h+1}(Z)/V_1(Z) V h + 1 ( Z ) / V 1 ( Z ) V 1 ( Z ) 2 V_1(Z)^2 V 1 ( Z ) 2 V 1 ( Z ) ≥ h d / 1 + ζ 2 V_1(Z) \ge h_d/\sqrt{1+\zeta^2} V 1 ( Z ) ≥ h d / 1 + ζ 2
On a fixed connecting segment [ v a , v b ] [v_a,v_b] [ v a , v b ] d + 1 d+1 d + 1 Ψ ε \Psi_\varepsilon Ψ ε v a v_a v a v b v_b v b Z ( v b ) = Z ( v a ) Ψ ε Z(v_b)=Z(v_a)\Psi_\varepsilon Z ( v b ) = Z ( v a ) Ψ ε h h h
D = diag J ⊂ { d + 2 , … , r } ∣ J ∣ = h exp { ∫ v a v b ∑ l ∈ J ( a K λ l ∥ Q ( u ) C ∥ F − 1 ) d u } , F = ⋀ h Ψ ε − D . D=\operatorname{diag}_{\substack{J\subset\{d+2,\ldots,r\}\\ |J|=h}}\exp\left\{\int_{v_a}^{v_b}\sum_{l\in J}\left(\frac{a_K\lambda_l}{\lVert Q(u)C\rVert_F}-1\right)\,du\right\},\qquad F=\bigwedge\nolimits^h\Psi_\varepsilon-D. D = diag J ⊂ { d + 2 , … , r } ∣ J ∣ = h exp { ∫ v a v b l ∈ J ∑ ( ∥ Q ( u ) C ∥ F a K λ l − 1 ) d u } , F = ⋀ h Ψ ε − D . Here Q Q Q ∥ F ∥ o p → 0 \lVert F\rVert_{\mathrm{op}}\to0 ∥ F ∥ op → 0 ⋀ h Z ( v a ) \bigwedge^h Z(v_a) ⋀ h Z ( v a )
V h ( Z ( v b ) ) ≥ [ D 00 − 1 + ζ 2 ∥ F ∥ o p ] V h ( Z ( v a ) ) . V_h(Z(v_b))\ge\left[D_{00}-\sqrt{1+\zeta^2}\lVert F\rVert_{\mathrm{op}}\right]V_h(Z(v_a)). V h ( Z ( v b )) ≥ [ D 00 − 1 + ζ 2 ∥ F ∥ op ] V h ( Z ( v a )) . For sufficiently small ε \varepsilon ε D 00 / 2 > 0 D_{00}/2>0 D 00 /2 > 0 max j ≠ 0 D j j ≤ D 00 \max_{j\ne0}D_{jj}\le D_{00} max j = 0 D j j ≤ D 00
D 00 ∥ B h ( v a ) ∥ F / V h ( Z ( v a ) ) + 1 + ζ 2 ∥ F ∥ o p D 00 − 1 + ζ 2 ∥ F ∥ o p . \frac{D_{00}\lVert B_h(v_a)\rVert_F/V_h(Z(v_a))+\sqrt{1+\zeta^2}\lVert F\rVert_{\mathrm{op}}}{D_{00}-\sqrt{1+\zeta^2}\lVert F\rVert_{\mathrm{op}}}. D 00 − 1 + ζ 2 ∥ F ∥ op D 00 ∥ B h ( v a ) ∥ F / V h ( Z ( v a )) + 1 + ζ 2 ∥ F ∥ op . The incoming fine-cone ratio and ∥ F ∥ o p \lVert F\rVert_{\mathrm{op}} ∥ F ∥ op
Outgoing independent exponents and the final interval.
For the remaining scales, the new principal h h h V h + 1 ( Z ) / V 1 ( Z ) V_{h+1}(Z)/V_1(Z) V h + 1 ( Z ) / V 1 ( Z ) β a + 1 − g d + a + 1 ( d ) β 1 / g \beta_{a+1}-g_{d+a+1}^{(d)}\beta_1/g β a + 1 − g d + a + 1 ( d ) β 1 / g a a a
If β 1 − g ( 1 − x ) > 0 \beta_1-g(1-x)>0 β 1 − g ( 1 − x ) > 0 Z Z Z t ∗ t_* t ∗ D + ∥ Z ∥ F ≤ − ∥ Z ∥ F / 2 D^+\lVert Z\rVert_F\le-\lVert Z\rVert_F/2 D + ∥ Z ∥ F ≤ − ∥ Z ∥ F /2
Actual clocks and the deterministic entrance
The preceding estimates are now iterated in the finite order prescribed by the clocks. Each entry carries the independent suppression left by all earlier stages. The last residence gives convergence to a fixed-rank empirical boundary solution. Fix a task under Assumptions 2 and 5, a finite K K K b > 1 b>1 b > 1 b ∉ B C , K b\notin\mathcal{B}_{C,K} b ∈ / B C , K D = d K ( b ) D=d_K(b) D = d K ( b ) 1 ≤ h ≤ p − d 1\le h\le p-d 1 ≤ h ≤ p − d J d , h J_{d,h} J d , h V h V_h V h (75) and (74) . In each rank-d d d y y y Z Z Z E E E Lemma 43 on Q ε ( t ) Q^\varepsilon(t) Q ε ( t ) Z d ε ( t ) Z_d^\varepsilon(t) Z d ε ( t ) l l l
e l ( 1 ) = ( b − 1 ) , e l ( d ) = ( ( b − 1 ) + x d ) ( 1 − λ l / λ d ) ( 2 ≤ d ≤ D ) . (99) e_l^{(1)}=(b-1),\qquad e_l^{(d)}=((b-1)+x_d)(1-\lambda_l/\lambda_d)\qquad(2\le d\le D). \tag*{(99)} e l ( 1 ) = ( b − 1 ) , e l ( d ) = (( b − 1 ) + x d ) ( 1 − λ l / λ d ) ( 2 ≤ d ≤ D ) . (99) The lemma below identifies these numbers as the actual incoming independent exponents. For an index set I I I P Ran Q I P_{\operatorname{Ran} Q_I} P Ran Q I Q I Q_I Q I h d > 0 h_d>0 h d > 0 d < D d<D d < D
∥ ( I m − P Ran Q { 1 , … , d } ( t ) ) Q { d + 1 , … , r } ε ( t ) ∥ F = h d . (100) \left\|\left(I_m-P_{\operatorname{Ran} Q_{\{1,\ldots,d\}}}(t)\right)Q^\varepsilon_{\{d+1,\ldots,r\}}(t)\right\|_F=h_d. \tag*{(100)} ( I m − P Ran Q { 1 , … , d } ( t ) ) Q { d + 1 , … , r } ε ( t ) F = h d . (100) Its first crossing after entry into the rank-d d d t d + 1 ε t_{d+1}^\varepsilon t d + 1 ε Lemma 43 , the left side is ∥ Z d ε ∥ F \lVert Z_d^\varepsilon\rVert_F ∥ Z d ε ∥ F Proposition 1 .
Lemma 44 (Actual transition times and remaining independent scales) For the fixed data above, there are finitely many coordinate neighborhoods and cone widths, positive outgoing levels h d h_d h d d < D d<D d < D Δ v d > 0 \Delta v_d>0 Δ v d > 0 t ∗ > 0 t_*>0 t ∗ > 0 Lemma 32 , there are σ 0 ( U ) < ∞ \sigma_0(U)<\infty σ 0 ( U ) < ∞ C U < ∞ C_U<\infty C U < ∞ ε > 0 \varepsilon>0 ε > 0 ε \varepsilon ε U U U
The first inlet is t i n , 1 ε = ε σ 0 ( U ) t^\varepsilon_{\mathrm{in},1}=\varepsilon\sigma_0(U) t in , 1 ε = ε σ 0 ( U ) t i n , d ε = e Δ v d − 1 t d ε t^\varepsilon_{\mathrm{in},d}=e^{\Delta v_{d-1}}t^\varepsilon_d t in , d ε = e Δ v d − 1 t d ε
∣ log t i n , d ε − ( 1 − x d ) log ε ∣ ≤ C U , 1 ≤ d ≤ D . (101) \left|\log t^\varepsilon_{\mathrm{in},d}-(1-x_d)\log\varepsilon\right|\le C_U,\qquad1\le d\le D. \tag*{(101)} log t in , d ε − ( 1 − x d ) log ε ≤ C U , 1 ≤ d ≤ D . (101) For d < D d<D d < D t d + 1 ε t^\varepsilon_{d+1} t d + 1 ε d = D d=D d = D t ∗ t_* t ∗
∣ log V h ( Z d ε ( t ) ) − ( ∑ l ∈ J d , h e l ( d ) ) log ε − ( ∑ l ∈ J d , h g l ( d ) ) log t t i n , d ε ∣ ≤ C U , 1 ≤ h ≤ p − d , (102) \left|\log V_h(Z_d^\varepsilon(t))-\left(\sum_{l\in J_{d,h}}e_l^{(d)}\right)\log\varepsilon-\left(\sum_{l\in J_{d,h}}g_l^{(d)}\right)\log\frac{t}{t^\varepsilon_{\mathrm{in},d}}\right|\le C_U,\qquad1\le h\le p-d, \tag*{(102)} log V h ( Z d ε ( t )) − l ∈ J d , h ∑ e l ( d ) log ε − l ∈ J d , h ∑ g l ( d ) log t in , d ε t ≤ C U , 1 ≤ h ≤ p − d , (102) and its coefficient error satisfies
∫ residence at d ( ∥ y ∥ + ∥ Z d ε ∥ F 2 + ∥ E ∥ o p ) d log t ≤ C U . (103) \int_{\text{residence at }d}\left(\lVert y\rVert+\lVert Z_d^\varepsilon\rVert_F^2+\lVert E\rVert_{\mathrm{op}}\right)\,d\log t\le C_U. \tag*{(103)} ∫ residence at d ( ∥ y ∥ + ∥ Z d ε ∥ F 2 + ∥ E ∥ op ) d log t ≤ C U . (103) Each completed crossing is transverse and obeys
log t d + 1 ε = ( 1 − x d + 1 ) log ε + O U ( 1 ) , 1 ≤ d < D . (104) \log t^\varepsilon_{d+1}=(1-x_{d+1})\log\varepsilon+O_U(1),\qquad1\le d<D. \tag*{(104)} log t d + 1 ε = ( 1 − x d + 1 ) log ε + O U ( 1 ) , 1 ≤ d < D . (104) Its Gram matrix, with logarithmic time shifted to put the crossing at zero, converges to the Gram matrix of the connecting trajectory of Lemma 36 . Projection and transport along the fixed segment give the stated inlet conditions at rank d + 1 d+1 d + 1 Z D ε ( t ) → 0 Z_D^\varepsilon(t)\to0 Z D ε ( t ) → 0 t ∈ ( 0 , t ∗ ] t\in(0,t_*] t ∈ ( 0 , t ∗ ] d = p d=p d = p
Proof (of Lemma 44 ). The candidate rank sequence is finite. Use D = d K ( b ) D=d_K(b) D = d K ( b ) ε ↓ 0 \varepsilon\downarrow0 ε ↓ 0
Geometric choices.
The candidate ranks form a finite sequence. Assumption 5 and Lemma 38 give the strict boundaries and the positive exterior gaps required by Lemma 43 at every rank in this sequence. Choose the neighborhoods and outgoing levels supplied by that lemma. By Lemma 36 , each connecting solution reaches the interior of the next chosen neighborhood after a fixed finite logarithmic duration Δ v d \Delta v_d Δ v d t ∗ > 0 t_* > 0 t ∗ > 0
Fix a seed in the measurable Gaussian-full event of Lemma 32 . Choose its finite σ 0 ( U ) \sigma_0(U) σ 0 ( U ) 1 / σ 0 1/\sigma_0 1/ σ 0
First entry.
At t i n , 1 ε = ε σ 0 t^\varepsilon_{\mathrm{in},1}=\varepsilon\sigma_0 t in , 1 ε = ε σ 0 x 1 = 0 x_1=0 x 1 = 0 (67) gives
log V h ( Z ε ( ε σ 0 ) ) V h − 1 ( Z ε ( ε σ 0 ) ) = ( b − 1 ) log ε + O U ( 1 ) , 1 ≤ h ≤ p − 1. \log\frac{V_h\left(Z^\varepsilon(\varepsilon\sigma_0)\right)}{V_{h-1}\left(Z^\varepsilon(\varepsilon\sigma_0)\right)}=(b-1)\log\varepsilon+O_U(1),\qquad1\le h\le p-1. log V h − 1 ( Z ε ( ε σ 0 ) ) V h ( Z ε ( ε σ 0 ) ) = ( b − 1 ) log ε + O U ( 1 ) , 1 ≤ h ≤ p − 1. These exponents are positive because b > 1 b>1 b > 1 (68) supplies the required leading volume bounds. Thus the input conditions of Lemma 43 hold before its first application.
A completed stage.
Suppose the entry conditions have been established at rank d < D d<D d < D Lemma 43 to the established rank-d d d p − d p-d p − d e d + h ( d ) e_{d+h}^{(d)} e d + h ( d ) d = 1 d=1 d = 1 ( b − 1 ) (b-1) ( b − 1 ) d ≥ 2 d\ge2 d ≥ 2
e d + h ( d ) = ( ( b − 1 ) + x d ) ( 1 − λ d + h λ d ) > 0. e_{d+h}^{(d)}=((b-1)+x_d)\left(1-\frac{\lambda_{d+h}}{\lambda_d}\right)>0. e d + h ( d ) = (( b − 1 ) + x d ) ( 1 − λ d λ d + h ) > 0. Their order is nondecreasing in h h h 0 ≤ x d < 1 0\le x_d<1 0 ≤ x d < 1 t i n , d ε / ε t^\varepsilon_{\mathrm{in},d}/\varepsilon t in , d ε / ε x d > 0 x_d>0 x d > 0 ε / t i n , d ε → 0 \varepsilon/t^\varepsilon_{\mathrm{in},d}\to0 ε / t in , d ε → 0
g d + 1 ( d ) > 0 , x d + e d + 1 ( d ) g d + 1 ( d ) = x d + 1 < 1. g_{d+1}^{(d)}>0,\qquad x_d+\frac{e_{d+1}^{(d)}}{g_{d+1}^{(d)}}=x_{d+1}<1. g d + 1 ( d ) > 0 , x d + g d + 1 ( d ) e d + 1 ( d ) = x d + 1 < 1. All hypotheses have now been checked. The lemma gives the first transverse crossing, its Gram limit, the estimates (102)–(103), and
log t d + 1 ε = ( 1 − x d + 1 ) log ε + O U ( 1 ) . \log t^\varepsilon_{d+1}=(1-x_{d+1})\log\varepsilon+O_U(1). log t d + 1 ε = ( 1 − x d + 1 ) log ε + O U ( 1 ) . In particular t d + 1 ε → 0 t^\varepsilon_{d+1}\to0 t d + 1 ε → 0 ε / t d + 1 ε → 0 \varepsilon/t^\varepsilon_{d+1}\to0 ε / t d + 1 ε → 0
The exponents at the next entry.
At the first completed stage, x 2 = ( b − 1 ) / g 2 ( 1 ) x_2=(b-1)/g_2^{(1)} x 2 = ( b − 1 ) / g 2 ( 1 ) Lemma 43 gives, for every remaining retained label l l l
e l ( 2 ) = ( b − 1 ) − g l ( 1 ) x 2 = ( b − 1 ) + x 2 − a K λ l λ 1 x 2 = ( ( b − 1 ) + x 2 ) ( 1 − λ l λ 2 ) , (105) \begin{aligned}
e_l^{(2)}=(b-1)-g_l^{(1)}x_2=(b-1)+x_2-\frac{a_K\lambda_l}{\lambda_1}x_2 \\
&=((b-1)+x_2)\left(1-\frac{\lambda_l}{\lambda_2}\right), \tag*{(105)}
\end{aligned} e l ( 2 ) = ( b − 1 ) − g l ( 1 ) x 2 = ( b − 1 ) + x 2 − λ 1 a K λ l x 2 = (( b − 1 ) + x 2 ) ( 1 − λ 2 λ l ) , (105) because ( b − 1 ) + x 2 = ( a K λ 2 / λ 1 ) x 2 (b-1)+x_2=(a_K\lambda_2/\lambda_1)x_2 ( b − 1 ) + x 2 = ( a K λ 2 / λ 1 ) x 2
( ( b − 1 ) + x d ) ( 1 − λ d + 1 λ d ) = ( x d + 1 − x d ) ( a K λ d + 1 s d , K − 1 ) . ((b-1)+x_d)\left(1-\frac{\lambda_{d+1}}{\lambda_d}\right)=(x_{d+1}-x_d)\left(\frac{a_K\lambda_{d+1}}{s_{d,K}}-1\right). (( b − 1 ) + x d ) ( 1 − λ d λ d + 1 ) = ( x d + 1 − x d ) ( s d , K a K λ d + 1 − 1 ) . Rearranging gives
( b − 1 ) + x d λ d + a K ( x d + 1 − x d ) s d , K = ( b − 1 ) + x d + 1 λ d + 1 . \frac{(b-1)+x_d}{\lambda_d}+\frac{a_K(x_{d+1}-x_d)}{s_{d,K}}=\frac{(b-1)+x_{d+1}}{\lambda_{d+1}}. λ d ( b − 1 ) + x d + s d , K a K ( x d + 1 − x d ) = λ d + 1 ( b − 1 ) + x d + 1 . For a remaining label, the transfer formula therefore becomes
e l ( d + 1 ) = e l ( d ) − g l ( d ) ( x d + 1 − x d ) = ( b − 1 ) + x d + 1 − λ l ( ( b − 1 ) + x d λ d + a K ( x d + 1 − x d ) s d , K ) = ( ( b − 1 ) + x d + 1 ) ( 1 − λ l λ d + 1 ) > 0. (106) \begin{aligned}
e_l^{(d+1)}=e_l^{(d)}-g_l^{(d)}(x_{d+1}-x_d) \\
&=(b-1)+x_{d+1}-\lambda_l\left(\frac{(b-1)+x_d}{\lambda_d}+\frac{a_K(x_{d+1}-x_d)}{s_{d,K}}\right) \\
&=((b-1)+x_{d+1})\left(1-\frac{\lambda_l}{\lambda_{d+1}}\right)>0. \tag*{(106)}
\end{aligned} e l ( d + 1 ) = e l ( d ) − g l ( d ) ( x d + 1 − x d ) = ( b − 1 ) + x d + 1 − λ l ( λ d ( b − 1 ) + x d + s d , K a K ( x d + 1 − x d ) ) = (( b − 1 ) + x d + 1 ) ( 1 − λ d + 1 λ l ) > 0. (106) The identity p − ( d + 1 ) = ( p − d ) − 1 p-(d+1)=(p-d)-1 p − ( d + 1 ) = ( p − d ) − 1 Lemma 43 , give the next entry’s volume and cone conditions. The fixed segment adds Δ v d \Delta v_d Δ v d t d + 1 ε = e Δ v d t d + 1 ε t_{d+1}^{\varepsilon}=e^{\Delta v_d}t_{d+1}^{\varepsilon} t d + 1 ε = e Δ v d t d + 1 ε (101) . This completes the induction step.
The final interval.
If D < p D<p D < p Lemma 43 to the final entry with exponent e D + 1 ( D ) e_{D+1}^{(D)} e D + 1 ( D ) x D x_D x D
e D + 1 ( D ) − g D + 1 ( D ) ( 1 − x D ) > 0. e_{D+1}^{(D)}-g_{D+1}^{(D)}(1-x_D)>0. e D + 1 ( D ) − g D + 1 ( D ) ( 1 − x D ) > 0. For a positive rate this is equivalent to the next candidate being larger than one. For a nonpositive rate it follows from the positivity of e D + 1 ( D ) e_{D+1}^{(D)} e D + 1 ( D ) t ∗ t_* t ∗ D = p = m D=p=m D = p = m D = p < m D=p<m D = p < m p = min { m , n + ( C ) } p=\min\{m,n_+(\mathcal{C})\} p = min { m , n + ( C )} p = n + ( C ) p=n_+(\mathcal{C}) p = n + ( C ) C C C p = r < m p=r<m p = r < m Z D ε ( t ) → 0 Z_D^{\varepsilon}(t)\to0 Z D ε ( t ) → 0 t > 0 t>0 t > 0
There are finitely many applications. Taking the smallest of their seed-dependent cutoffs and the largest of their constants gives one range of ε \varepsilon ε C U C_U C U
The bound O U ( 1 ) OU(1) O U ( 1 ) log ( t d + 1 ε / ε ) / log ( 1 / ε ) → x d + 1 \log(t^{\varepsilon}_{d+1}/\varepsilon)/\log(1/\varepsilon) \to x_{d+1} log ( t d + 1 ε / ε ) / log ( 1/ ε ) → x d + 1 ( 0 , 1 ) (0,1) ( 0 , 1 ) t d + 1 ε / ε t^{\varepsilon}_{d+1}/\varepsilon t d + 1 ε / ε
The next lemma identifies an entrance from the decay of the unused component. Its conditions concern a fixed rank and apply in any positive factor dimensions. They are also used for the rank-one conclusion in Proposition 51.
Lemma 45 (Deterministic entrance at a fixed rank). Consider a spanning empirical quadratic task, finite K K K Q d , K Q_{d,K} Q d , K F K ( Q d , K ) = 0 F_K(Q_{d,K}) = 0 F K ( Q d , K ) = 0 Ran ( Q d , K ⊤ ) = span { v 1 , … , v d } \operatorname{Ran}(Q_{d,K}^{\top}) = \operatorname{span}\{v_1,\ldots,v_d\} Ran ( Q d , K ⊤ ) = span { v 1 , … , v d } C C C 1 ≤ d ≤ min { m , r } 1 \le d \le\min\{m,r\} 1 ≤ d ≤ min { m , r } λ d > λ d + 1 \lambda_d > \lambda_{d+1} λ d > λ d + 1 d < r d < r d < r K ≥ 1 K \ge1 K ≥ 1 d = 1 d = 1 d = 1 d d d W ‾ K , d \overline{W}_{K,d} W K , d
W ˙ = P K ( G ( W ) / ∥ G ( W ) ∥ F ) , W ( t ) / t ⟶ Q d , K , \dot{W} = P_K(G(W)/\|G(W)\|_F), \qquad W(t)/t \longrightarrow Q_{d,K}, W ˙ = P K ( G ( W ) /∥ G ( W ) ∥ F ) , W ( t ) / t ⟶ Q d , K , unique in this fixed-rank class up to a constant left orthogonal transformation. For some t ∗ > 0 t_* > 0 t ∗ > 0
W ‾ K , d ( t ) = t Q d , K + O ( t 3 ) , W ‾ ˙ K , d ( t ) = Q d , K + O ( t 2 ) . \overline{W}_{K,d}(t) = tQ_{d,K} + O(t^3), \qquad\dot{\overline{W}}_{K,d}(t) = Q_{d,K} + O(t^2). W K , d ( t ) = t Q d , K + O ( t 3 ) , W ˙ K , d ( t ) = Q d , K + O ( t 2 ) . Let W ε W^\varepsilon W ε (6) with η ε = c η ε \eta_\varepsilon= c_\eta\varepsilon η ε = c η ε W ε ( 0 ) → 0 W^\varepsilon(0) \to0 W ε ( 0 ) → 0 t a ε → 0 t^\varepsilon_a \to0 t a ε → 0 t a ε ≥ ε t^\varepsilon_a \ge\varepsilon t a ε ≥ ε W ε ( t ) / t W^\varepsilon(t)/t W ε ( t ) / t Q d , K Q_{d,K} Q d , K t a ε ≤ t ≤ t ∗ t^\varepsilon_a \le t \le t_* t a ε ≤ t ≤ t ∗ Z ε ( t ) → 0 Z^\varepsilon(t) \to0 Z ε ( t ) → 0 t ∈ ( 0 , t ∗ ] t \in(0,t_*] t ∈ ( 0 , t ∗ ] t ∈ ( 0 , t ∗ ) t \in(0,t_*) t ∈ ( 0 , t ∗ )
inf O ∈ O ( m ) ∥ O W ε ( t ) − W ‾ K , d ( t ) ∥ F ⟶ 0. \inf_{O\in O(m)}\|OW^\varepsilon(t)-\overline{W}_{K,d}(t)\|_F \longrightarrow0. O ∈ O ( m ) inf ∥ O W ε ( t ) − W K , d ( t ) ∥ F ⟶ 0. At every sufficiently small fixed predictor radius ρ > 0 \rho> 0 ρ > 0 W ‾ K , d \overline{W}_{K,d} W K , d d d d t ∗ t_* t ∗ K K K Q d , K Q_{d,K} Q d , K
Proof (of Lemma 45). Use the Gram coordinates fixed before Lemma 39 near Q d , K Q_{d,K} Q d , K H = 0 H = 0 H = 0 S 22 = S 21 S 11 − 1 S 12 S_{22} = S_{21}S_{11}^{-1}S_{12} S 22 = S 21 S 11 − 1 S 12 d d d S 11 ≻ 0 S_{11} \succ0 S 11 ≻ 0
y ′ = J d y + R d ( y , t 2 ) , ∥ R d ( y , t 2 ) ∥ ≤ C ( ∥ y ∥ 2 + t 2 ) , y' = J_dy + R_d(y,t^2), \qquad\|R_d(y,t^2)\| \le C(\|y\|^2+t^2), y ′ = J d y + R d ( y , t 2 ) , ∥ R d ( y , t 2 ) ∥ ≤ C ( ∥ y ∥ 2 + t 2 ) , where R d R_d R d
∥ R d ( y , t 2 ) − R d ( z , t 2 ) ∥ ≤ C ( ∥ y ∥ + ∥ z ∥ + t 2 ) ∥ y − z ∥ . \|R_d(y,t^2)-R_d(z,t^2)\| \le C(\|y\|+\|z\|+t^2)\|y-z\|. ∥ R d ( y , t 2 ) − R d ( z , t 2 ) ∥ ≤ C ( ∥ y ∥ + ∥ z ∥ + t 2 ) ∥ y − z ∥. Choose the tangent norm and common contraction rate ω \omega ω ∥ ( t / s ) J d ∥ ≤ ( s / t ) ω \|(t/s)^{J_d}\| \le(s/t)^\omega ∥ ( t / s ) J d ∥ ≤ ( s / t ) ω 0 < s ≤ t 0 < s \le t 0 < s ≤ t
For any bounded solution tending to zero, multiplication by t − J d t^{-J_d} t − J d t − 1 − J d R d ( y ( t ) , t 2 ) t^{-1-J_d}R_d(y(t),t^2) t − 1 − J d R d ( y ( t ) , t 2 ) C t ω − 1 ( ∥ y ( t ) ∥ 2 + t 2 ) Ct^{\omega-1}(\|y(t)\|^2+t^2) C t ω − 1 ( ∥ y ( t ) ∥ 2 + t 2 ) ∥ t − J d y ( t ) ∥ ≤ t ω sup 0 ≤ s ≤ t ∗ ∥ y ( s ) ∥ → 0 \lVert t^{-J_d}y(t)\rVert\le t^\omega\sup_{0\le s\le t_\ast}\lVert y(s)\rVert\to0 ∥ t − J d y ( t )∥ ≤ t ω sup 0 ≤ s ≤ t ∗ ∥ y ( s )∥ → 0
y ∗ ( t ) = ∫ 0 t ( t / s ) J d R d ( y ∗ ( s ) , s 2 ) d s s . (107) y_\ast(t)=\int_0^t (t/s)^{J_d}R_d(y_\ast(s),s^2)\,\frac{ds}{s}. \tag*{(107)} y ∗ ( t ) = ∫ 0 t ( t / s ) J d R d ( y ∗ ( s ) , s 2 ) s d s . (107) On the Banach space of continuous functions on ( 0 , t ∗ ] (0,t_\ast] ( 0 , t ∗ ] sup t − 2 ∥ y ( t ) ∥ \sup t^{-2}\lVert y(t)\rVert sup t − 2 ∥ y ( t )∥ R = 2 C / ( ω + 2 ) R=2C/(\omega+2) R = 2 C / ( ω + 2 ) C t 2 / ( ω + 2 ) Ct^2/(\omega+2) C t 2 / ( ω + 2 ) C R 2 t 4 / ( ω + 4 ) CR^2t^4/(\omega+4) C R 2 t 4 / ( ω + 4 ) C ( 2 R + 1 ) t ∗ ω C(2R+1)t_\ast^\omega C ( 2 R + 1 ) t ∗ ω t ∗ t_\ast t ∗ [31 Theorem 2.1] gives a unique y ∗ = O ( t 2 ) y_\ast=O(t^2) y ∗ = O ( t 2 )
Every solution tending to zero satisfies the same integral equation. Taking suprema up to t t t
sup 0 < s ≤ t ∥ y ( s ) ∥ ≤ C ( sup 0 < s ≤ t ∥ y ( s ) ∥ ) 2 + C t 2 . \sup_{0<s\le t}\lVert y(s)\rVert\le C\left(\sup_{0<s\le t}\lVert y(s)\rVert\right)^2+Ct^2. 0 < s ≤ t sup ∥ y ( s )∥ ≤ C ( 0 < s ≤ t sup ∥ y ( s )∥ ) 2 + C t 2 . For small t t t y = O ( t 2 ) y=O(t^2) y = O ( t 2 )
Let S ( t ) S(t) S ( t ) d d d t ∗ > 0 t_\ast>0 t ∗ > 0 t ∗ 2 S ( t ∗ ) t_\ast^2S(t_\ast) t ∗ 2 S ( t ∗ ) W ‾ K , d \overline{W}_{K,d} W K , d S ( t ) S(t) S ( t ) t t t C − t 2 L ( S ( t ) ) C-t^2\mathcal{L}(S(t)) C − t 2 L ( S ( t )) ∥ G ( W ‾ K , d ( t ) ) ∥ F \lVert G(\overline{W}_{K,d}(t))\rVert_F ∥ G ( W K , d ( t )) ∥ F t t t ( 0 , t ∗ ] (0,t_\ast] ( 0 , t ∗ ]
The factor
( S 11 1 / 2 S 11 − 1 / 2 S 12 0 0 ) \begin{pmatrix}
S_{11}^{1/2} & S_{11}^{-1/2}S_{12}\\
0 & 0
\end{pmatrix} ( S 11 1/2 0 S 11 − 1/2 S 12 0 ) has Gram matrix S S S y y y t − 1 W ‾ K , d ( t ) t^{-1}\overline{W}_{K,d}(t) t − 1 W K , d ( t ) O ( t 2 ) O(t^2) O ( t 2 ) { O Q d , K : O ∈ O ( m ) } \{OQ_{d,K}:O\in O(m)\} { O Q d , K : O ∈ O ( m )} t = 0 t=0 t = 0 O ( t 2 ) O(t^2) O ( t 2 ) ∥ ∂ v ( t − 1 W ‾ K , d ( t ) ) ∥ F = O ( t 2 ) \lVert\partial_v(t^{-1}\overline{W}_{K,d}(t))\rVert_F=O(t^2) ∥ ∂ v ( t − 1 W K , d ( t )) ∥ F = O ( t 2 ) v = − ∞ v=-\infty v = − ∞ O ( t 2 ) O(t^2) O ( t 2 )
W ‾ K , d ( t ) = t Q d , K + O ( t 3 ) , W ‾ ˙ K , d ( t ) = Q d , K + O ( t 2 ) . (108) \overline{W}_{K,d}(t)=tQ_{d,K}+O(t^3),\qquad\dot{\overline{W}}_{K,d}(t)=Q_{d,K}+O(t^2). \tag*{(108)} W K , d ( t ) = t Q d , K + O ( t 3 ) , W ˙ K , d ( t ) = Q d , K + O ( t 2 ) . (108) Factors with the same Gram at a positive time differ by a left orthogonal transformation. Equivariance and factor uniqueness keep this transformation constant along the solutions, proving the fixed-rank uniqueness.
For the actual family, smooth dependence of the exact Gram field on H H H ε / t \varepsilon/t ε / t
( y ε ) ′ = J d y ε + R d ( y ε , t 2 ) + O ( ∥ Z ε ∥ F 2 + ε / t ) . (y^\varepsilon)'=J_dy^\varepsilon+R_d(y^\varepsilon,t^2)+O(\lVert Z^\varepsilon\rVert_F^2+\varepsilon/t). ( y ε ) ′ = J d y ε + R d ( y ε , t 2 ) + O (∥ Z ε ∥ F 2 + ε / t ) . Subtracting the constructed equation and absorbing its small Lipschitz remainder in the common tangent norm gives
D + ∥ y ε − y ∗ ∥ ≤ − ω ∥ y ε − y ∗ ∥ + C ∥ Z ε ∥ F 2 + C ε / t . (109) D^+\lVert y^\varepsilon-y_\ast\rVert\le-\omega\lVert y^\varepsilon-y_\ast\rVert+C\lVert Z^\varepsilon\rVert_F^2+C\varepsilon/t. \tag*{(109)} D + ∥ y ε − y ∗ ∥ ≤ − ω ∥ y ε − y ∗ ∥ + C ∥ Z ε ∥ F 2 + C ε / t . (109) Consequently,
∥ y ε ( t ) − y ∗ ( t ) ∥ ≤ C ( t a ε t ) ω ∥ y ε ( t a ε ) − y ∗ ( t a ε ) ∥ + C t − ω ∫ t a ε t s ω − 1 ∥ Z ε ( s ) ∥ F 2 d s + C ε t − ω ∫ t a ε t s ω − 2 d s . \left\|y^{\varepsilon}(t)-y_*(t)\right\| \leq C\left(\frac{t_a^{\varepsilon}}{t}\right)^{\omega}\left\|y^{\varepsilon}\left(t_a^{\varepsilon}\right)-y_*\left(t_a^{\varepsilon}\right)\right\|+Ct^{-\omega}\int_{t_a^{\varepsilon}}^t s^{\omega-1}\left\|Z^{\varepsilon}(s)\right\|_F^2\,\mathrm{d}s+C\varepsilon t^{-\omega}\int_{t_a^{\varepsilon}}^t s^{\omega-2}\,\mathrm{d}s. ∥ y ε ( t ) − y ∗ ( t ) ∥ ≤ C ( t t a ε ) ω ∥ y ε ( t a ε ) − y ∗ ( t a ε ) ∥ + C t − ω ∫ t a ε t s ω − 1 ∥ Z ε ( s ) ∥ F 2 d s + C ε t − ω ∫ t a ε t s ω − 2 d s . On any fixed compact positive-time interval, the initial term vanishes because its mismatch is bounded and t a ε → 0 t_a^{\varepsilon}\to0 t a ε → 0 Z ε Z^{\varepsilon} Z ε t a ε t_a^{\varepsilon} t a ε C s ω − 1 Cs^{\omega-1} C s ω − 1 t a ε ≥ ε t_a^{\varepsilon}\geq\varepsilon t a ε ≥ ε O ( ε ω ) O(\varepsilon^{\omega}) O ( ε ω ) 0 < ω < 1 0<\omega<1 0 < ω < 1 O ( ε log ( 1 / ε ) ) O(\varepsilon\log(1/\varepsilon)) O ( ε log ( 1/ ε )) ω = 1 \omega=1 ω = 1 O ( ε ) O(\varepsilon) O ( ε ) ω > 1 \omega>1 ω > 1 y ε → y ∗ y^{\varepsilon}\to y_* y ε → y ∗ Z ε Z^{\varepsilon} Z ε S S S y y y H = ( Z ε ) ⊤ Z ε H=(Z^{\varepsilon})^{\top}Z^{\varepsilon} H = ( Z ε ) ⊤ Z ε
The factor conclusion follows by compactness: any sequence violating it has convergent subsequences of bounded factors at positive times with equal limiting Grams. Their limits differ by a left orthogonal transformation, contradicting the assumed positive distance from that orthogonal class.
Equation (108) yields
d d t ∥ W ‾ K , d ( t ) ⊤ W ‾ K , d ( t ) ∥ F = 2 t ∥ Q d , K ⊤ Q d , K ∥ F + O ( t 3 ) > 0. (110) \frac{\mathrm{d}}{\mathrm{d}t}\left\|\overline{W}_{K,d}(t)^{\top}\overline{W}_{K,d}(t)\right\|_F=2t\left\|Q_{d,K}^{\top}Q_{d,K}\right\|_F+O(t^3)>0. \tag*{(110)} d t d W K , d ( t ) ⊤ W K , d ( t ) F = 2 t Q d , K ⊤ Q d , K F + O ( t 3 ) > 0. (110) Every sufficiently small fixed ρ > 0 \rho>0 ρ > 0 t ρ t_{\rho} t ρ ∥ W ˙ ε ∥ F ≤ min { m , r } \left\|\dot{W}^{\varepsilon}\right\|_F\leq\sqrt{\min\{m,r\}} W ˙ ε F ≤ min { m , r } ε \varepsilon ε ∥ W ε ( 0 ) ∥ F ≤ ρ / 4 \left\|W^{\varepsilon}(0)\right\|_F\leq\sqrt{\rho}/4 ∥ W ε ( 0 ) ∥ F ≤ ρ /4 ρ / 4 \rho/4 ρ /4
0 ≤ t ≤ ρ 4 min { m , r } . 0\leq t\leq\frac{\sqrt{\rho}}{4\sqrt{\min\{m,r\}}}. 0 ≤ t ≤ 4 min { m , r } ρ . The deterministic entrance has the same speed bound from zero, giving t ρ ≥ ρ / min { m , r } t_{\rho}\geq\sqrt{\rho}/\sqrt{\min\{m,r\}} t ρ ≥ ρ / min { m , r } t ρ t_{\rho} t ρ ( 0 , t ∗ ) (0,t_*) ( 0 , t ∗ ) ρ \rho ρ ρ \rho ρ ρ \rho ρ t ρ t_{\rho} t ρ
Lemma 46 (Isometric embeddings of a fixed-rank entrance). Let V ( t ) ∈ R d × r V(t)\in\mathbb{R}^{d\times r} V ( t ) ∈ R d × r E ∈ R m × d E\in\mathbb{R}^{m\times d} E ∈ R m × d E ⊤ E = I d E^{\top}E=I_d E ⊤ E = I d m ≥ d m\geq d m ≥ d E V ( t ) EV(t) E V ( t ) m m m E Q d , K EQ_{d,K} E Q d , K V V V d d d
Proof (of Lemma 46 ). Since E T E = I d E^{\mathsf{T}}E=I_d E T E = I d V ∈ R d × r V\in\mathbb{R}^{d\times r} V ∈ R d × r X ∈ R d × r X\in\mathbb{R}^{d\times r} X ∈ R d × r
( E V ) T E V = V T V , (EV)^{\mathsf{T}}EV=V^{\mathsf{T}}V, ( E V ) T E V = V T V , G ( E V ) = 2 E V R ( V T V ) = E G ( V ) , G(EV)=2EVR(V^{\mathsf{T}}V)=EG(V), G ( E V ) = 2 E V R ( V T V ) = E G ( V ) , ∥ E X ∥ F 2 = tr ( X T E T E X ) = ∥ X ∥ F 2 . \lVert EX\rVert_F^2=\operatorname{tr}(X^{\mathsf{T}}E^{\mathsf{T}}EX)=\lVert X\rVert_F^2. ∥ E X ∥ F 2 = tr ( X T E T E X ) = ∥ X ∥ F 2 . We prove P k ( E X ) = E P k ( X ) P_k(EX)=EP_k(X) P k ( E X ) = E P k ( X ) k = 0 k=0 k = 0 k k k
( E P k ( X ) ) ( E P k ( X ) ) T E P k ( X ) = E P k ( X ) P k ( X ) T E T E P k ( X ) = E P k ( X ) P k ( X ) T P k ( X ) . (EP_k(X))(EP_k(X))^{\mathsf{T}}EP_k(X)=EP_k(X)P_k(X)^{\mathsf{T}}E^{\mathsf{T}}EP_k(X)=EP_k(X)P_k(X)^{\mathsf{T}}P_k(X). ( E P k ( X )) ( E P k ( X ) ) T E P k ( X ) = E P k ( X ) P k ( X ) T E T E P k ( X ) = E P k ( X ) P k ( X ) T P k ( X ) . Substitution in (3) proves the induction step. Both the zero-stabilization physical field where G ≠ 0 G\ne0 G = 0 H K , η ( G ) H_{K,\eta}(G) H K , η ( G ) E E E G = 0 G=0 G = 0 E V ( t ) EV(t) E V ( t ) m × r m\times r m × r d d d
Lemma 45 identifies it with the local m m m d d d
Lemma 44 verifies these conditions for the rank d K ( b ) d_K(b) d K ( b )
Proof (of Theorem 7 ). Fix the task, K K K b b b c η c_\eta c η d = d K ( b ) d=d_K(b) d = d K ( b ) Q d , K Q_{d,K} Q d , K d < r d<r d < r K = 0 K=0 K = 0 d = 1 d=1 d = 1 d d d
For a seed in the measurable Gaussian-full event of Lemma 32, apply Lemma 44. Its final incoming time tends to zero. At rank one choose the initial-layer cutoff at least one; at every later rank the time law has x d > 0 x_d>0 x d > 0 t i n , d ≥ ε t_{\mathrm{in},d}\ge\varepsilon t in , d ≥ ε ε \varepsilon ε Z d ε ( t ) → 0 Z_d^\varepsilon(t)\to0 Z d ε ( t ) → 0 D ε , b U → 0 D_{\varepsilon,b}U\to0 D ε , b U → 0 (25) and (26) .
The task and the final zero determine the constructed Gram solution, so the limit is independent of the seed and has rank d d d B C \mathcal{B}_C B C b > 1 b>1 b > 1 b b b
Dependence on initialization and depth
The algebraic identity (22) separates the initialization exponent from the spectral coefficients. For every already existing support of rank at least two and finite K ≥ 1 K \ge1 K ≥ 1 s I , K s_{I,K} s I , K K K K s I , K / a K s_{I,K}/a_K s I , K / a K c d , K c_{d,K} c d , K d ≥ 2 d \ge2 d ≥ 2
Proof (the identity (22) ). Substitute x d = ( b − 1 ) c d , K x_d=(b-1)c_{d,K} x d = ( b − 1 ) c d , K (18) –(19) . The common factor b − 1 b-1 b − 1 (20) . At each candidate with a positive addition rate, induction therefore identifies the computed time fraction as x d = ( b − 1 ) c d , K x_d=(b-1)c_{d,K} x d = ( b − 1 ) c d , K + ∞ +\infty + ∞ x d < 1 x_d<1 x d < 1 ( b − 1 ) c d , K < 1 (b-1)c_{d,K}<1 ( b − 1 ) c d , K < 1 (22) . In particular the second direction enters precisely when 1 < b < a K λ 2 / λ 1 1<b<a_K\lambda_2/\lambda_1 1 < b < a K λ 2 / λ 1
Proof (of Proposition 8). For 0 < s < 1 0<s<1 0 < s < 1
p K + 1 ( s ) = 3 2 p K ( s ) ( 1 − p K ( s ) 2 3 ) p_{K+1}(s)=\frac{3}{2}p_K(s)\left(1-\frac{p_K(s)^2}{3}\right) p K + 1 ( s ) = 2 3 p K ( s ) ( 1 − 3 p K ( s ) 2 ) Since 0 < p K ( s ) < 1 0<p_K(s)<1 0 < p K ( s ) < 1 p K + 1 ( s ) > p K ( s ) p_{K+1}(s)>p_K(s) p K + 1 ( s ) > p K ( s )
q K + 1 ( s 2 ) > q K ( s 2 ) , q K + 1 ( s 2 ) a K + 1 = q K ( s 2 ) a K ( 1 − p K ( s ) 2 3 ) < q K ( s 2 ) a K . q_{K+1}(s^2)>q_K(s^2), \qquad\frac{q_{K+1}(s^2)}{a_{K+1}}=\frac{q_K(s^2)}{a_K}\left(1-\frac{p_K(s)^2}{3}\right)<\frac{q_K(s^2)}{a_K}. q K + 1 ( s 2 ) > q K ( s 2 ) , a K + 1 q K + 1 ( s 2 ) = a K q K ( s 2 ) ( 1 − 3 p K ( s ) 2 ) < a K q K ( s 2 ) . First fix an existing support I I I ∣ I ∣ ≥ 2 |I|\ge2 ∣ I ∣ ≥ 2 K ≥ 1 K\ge1 K ≥ 1 ( 0 , 1 ) (0,1) ( 0 , 1 ) (14) gives
∑ a ∈ I q K − 1 ( s I , K / λ a ) = 1 , max a ∈ I λ a < s I , K < a K min a ∈ I λ a . \sum_{a\in I}q_K^{-1}(s_{I,K}/\lambda_a)=1,\qquad\max_{a\in I}\lambda_a<s_{I,K}<a_K\min_{a\in I}\lambda_a. a ∈ I ∑ q K − 1 ( s I , K / λ a ) = 1 , a ∈ I max λ a < s I , K < a K a ∈ I min λ a . The strict lower inequality follows because if the scale equalled the largest active eigenvalue, one inverse term would be one and all other terms would be positive. At the upper endpoint ( a K + 1 / a K ) s I , K (a_{K+1}/a_K)s_{I,K} ( a K + 1 / a K ) s I , K
q K + 1 − 1 ( a K + 1 s I , K a K λ a ) < q K − 1 ( s I , K / λ a ) . q_{K+1}^{-1}\left(\frac{a_{K+1}s_{I,K}}{a_K\lambda_a}\right)<q_K^{-1}(s_{I,K}/\lambda_a). q K + 1 − 1 ( a K λ a a K + 1 s I , K ) < q K − 1 ( s I , K / λ a ) . All displayed inverse arguments lie in their domains, by the strict upper bound on s I , K s_{I,K} s I , K max a ∈ I λ a \max_{a\in I}\lambda_a max a ∈ I λ a s I , K + 1 s_{I,K+1} s I , K + 1 K + 1 K+1 K + 1
At the old unscaled root s I , K s_{I,K} s I , K q K + 1 > q K q_{K+1}>q_K q K + 1 > q K
s I , K < s I , K + 1 < a K + 1 a K s I , K . s_{I,K}<s_{I,K+1}<\frac{a_{K+1}}{a_K}s_{I,K}. s I , K < s I , K + 1 < a K a K + 1 s I , K . Iterating proves both comparisons for all larger finite depths. At the leading rank-one support, s 1 , K = λ 1 s_{1,K}=\lambda_1 s 1 , K = λ 1 K K K s 1 , K / a K s_{1,K}/a_K s 1 , K / a K
The candidate process at K = 0 K=0 K = 0 λ 2 / λ 1 − 1 < 0 \lambda_2/\lambda_1-1<0 λ 2 / λ 1 − 1 < 0 a K λ l / s I , K − 1 a_K\lambda_l/s_{I,K}-1 a K λ l / s I , K − 1 c 2 , K c_{2,K} c 2 , K K K K (20) , the map
c ⟼ c + ( 1 + c ) ( 1 − λ d + 1 / λ d ) g d + 1 ( d ) c\longmapsto c+\frac{(1+c)(1-\lambda_{d+1}/\lambda_d)}{g_{d+1}^{(d)}} c ⟼ c + g d + 1 ( d ) ( 1 + c ) ( 1 − λ d + 1 / λ d ) is strictly increasing in c ≥ 0 c\ge0 c ≥ 0 c d , K c_{d,K} c d , K K K K (22) proves the algebraic rank monotonicity. At depths satisfying the pointwise conditions of Theorem 7 , the same integers are the actual first-radius entrance ranks, which proves the last assertion. ■
Comparison and All-Time Path Separation
We first identify the gradient-flow comparison predictor. Entry sets then allow the selected continuation to be compared at any sufficiently small section. Strict loss and equal loss have separate tail estimates, which are combined with the finite-time comparison to give all-time statements.
Gradient-flow comparison and its stationary endpoint
The precise selection law used in Proposition 3 is as follows. For any family A ε ⪰ 0 A_\varepsilon\succeq0 A ε ⪰ 0
d M ε d s = − 2 { M ε ∇ Φ ( M ε ) + ∇ Φ ( M ε ) M ε } , M ε ( 0 ) = A ε . (111) \frac{\mathrm{d}M_\varepsilon}{\mathrm{d}s}=-2\{M_\varepsilon\nabla\Phi(M_\varepsilon)+\nabla\Phi(M_\varepsilon)M_\varepsilon\},\qquad M_\varepsilon(0)=A_\varepsilon. \tag*{(111)} d s d M ε = − 2 { M ε ∇Φ ( M ε ) + ∇Φ ( M ε ) M ε } , M ε ( 0 ) = A ε . (111) If some fixed c > 0 c>0 c > 0
⟨ A ε , v 1 v 1 T ⟩ ≥ c ∥ A ε ∥ F > 0 for every sufficiently small ε , (112) \langle A_\varepsilon,v_1v_1^{\mathsf{T}}\rangle\ge c\lVert A_\varepsilon\rVert_F>0\qquad\text{for every sufficiently small }\varepsilon, \tag*{(112)} ⟨ A ε , v 1 v 1 T ⟩ ≥ c ∥ A ε ∥ F > 0 for every sufficiently small ε , (112) then for every fixed s ∈ R s\in\mathbb{R} s ∈ R
M ε ( 1 4 λ 1 log 1 ⟨ A ε , v 1 v 1 T ⟩ + s ) ⟶ M G ( s ) . (113) M_\varepsilon\left(\frac{1}{4\lambda_1}\log\frac{1}{\langle A_\varepsilon,v_1v_1^{\mathsf{T}}\rangle}+s\right)\longrightarrow M^{\mathrm{G}}(s). \tag*{(113)} M ε ( 4 λ 1 1 log ⟨ A ε , v 1 v 1 T ⟩ 1 + s ) ⟶ M G ( s ) . (113) The time s s s
Proof (of Proposition 3). Differentiating M = W T W M=W^{\mathsf{T}}W M = W T W d W / d s = − 2 W ∇ Φ ( W T W ) \mathrm{d}W/\mathrm{d}s=-2W\nabla\Phi(W^{\mathsf{T}}W) d W / d s = − 2 W ∇Φ ( W T W ) (111) . Its derivative at M = 0 M=0 M = 0 H ↦ 2 ( C H + H C ) H\mapsto2(CH+HC) H ↦ 2 ( C H + H C ) C C C 2 ( λ a + λ b ) 2(\lambda_a+\lambda_b) 2 ( λ a + λ b ) a ≤ b a\le b a ≤ b 4 λ 1 4\lambda_1 4 λ 1 v 1 v 1 T v_1v_1^{\mathsf{T}} v 1 v 1 T H ↦ ⟨ H , v 1 v 1 T ⟩ H \mapsto\langle H,v_1v_1^{\mathsf{T}}\rangle H ↦ ⟨ H , v 1 v 1 T ⟩
The predictor field is analytic. For each A ⪰ 0 A \succeq0 A ⪰ 0 A A A
d d s Φ ( W T W ) = − 4 ∥ W ∇ Φ ( W T W ) ∥ F 2 ≤ 0. \frac{\mathrm{d}}{\mathrm{d}s}\Phi(W^{\mathsf{T}}W)=-4\lVert W\nabla\Phi(W^{\mathsf{T}}W)\rVert_F^2\le0. d s d Φ ( W T W ) = − 4 ∥ W ∇Φ ( W T W ) ∥ F 2 ≤ 0. Compactness of the PSD sublevels bounds the predictor and hence the factor, because ∥ W ∥ F 2 = tr ( W T W ) \lVert W\rVert_F^2=\operatorname{tr}(W^{\mathsf{T}}W) ∥ W ∥ F 2 = tr ( W T W ) [31 ] therefore gives a global forward factor solution and a PSD predictor solution. Choose a compact PSD sublevel containing 0 0 0 A ε A_\varepsilon A ε C 2 C^2 C 2 2 Φ 2\Phi 2Φ
For the initial values α v 1 v 1 T \alpha v_1v_1^{\mathsf{T}} α v 1 v 1 T Φ \Phi Φ a 0 = 1 a_0=1 a 0 = 1 Φ ( 0 ) \Phi(0) Φ ( 0 ) M 1 R ( M 1 ) = 0 M_1R(M_1)=0 M 1 R ( M 1 ) = 0
The empirical loss in Lemma 15 and compactness in Proposition 1 supply the hypotheses used in the preceding argument. We now apply that argument to the prescribed Gaussian initialization.
For the actual initialization, write u a T u_a^{\mathsf{T}} u a T a a a U U U
A ε = U T D ε , b 2 U = ε 2 u 1 u 1 T + ε 2 b ∑ a = 2 m u a u a T , ε − 2 A ε → u 1 u 1 T . A_\varepsilon=U^{\mathsf{T}}D_{\varepsilon,b}^2U=\varepsilon^2u_1u_1^{\mathsf{T}}+\varepsilon^{2b}\sum_{a=2}^{m}u_au_a^{\mathsf{T}},\qquad\varepsilon^{-2}A_\varepsilon\to u_1u_1^{\mathsf{T}}. A ε = U T D ε , b 2 U = ε 2 u 1 u 1 T + ε 2 b a = 2 ∑ m u a u a T , ε − 2 A ε → u 1 u 1 T . Since b > 1 b>1 b > 1 ∥ u 1 u 1 T ∥ F = ∥ u 1 ∥ 2 \lVert u_1u_1^{\mathsf{T}}\rVert_F=\lVert u_1\rVert^2 ∥ u 1 u 1 T ∥ F = ∥ u 1 ∥ 2 u 1 u_1 u 1 u 1 T v 1 ≠ 0 u_1^{\mathsf{T}}v_1\ne0 u 1 T v 1 = 0
⟨ A ε , v 1 v 1 T ⟩ ∥ A ε ∥ F ⟶ ( u 1 T v 1 ) 2 ∥ u 1 ∥ 2 > 0. (114) \frac{\langle A_\varepsilon,v_1v_1^{\mathsf{T}}\rangle}{\lVert A_\varepsilon\rVert_F}\longrightarrow\frac{(u_1^{\mathsf{T}}v_1)^2}{\lVert u_1\rVert^2}>0. \tag*{(114)} ∥ A ε ∥ F ⟨ A ε , v 1 v 1 T ⟩ ⟶ ∥ u 1 ∥ 2 ( u 1 T v 1 ) 2 > 0. (114) For each such fixed seed, half the positive limit gives a constant c c c K = 0 K=0 K = 0 M 1 M_1 M 1 s s s M G ( s ) M^G(s) M G ( s ) M 1 M_1 M 1 ε \varepsilon ε
Corollary 47 (Negative curvature at the comparison factor). Under Assumption 9 , every factor W 1 ∈ R m × r W_1 \in\mathbb{R}^{m \times r} W 1 ∈ R m × r W 1 T W 1 = M 1 W_1^{\mathsf{T}}W_1=M_1 W 1 T W 1 = M 1 f f f X X X
⟨ X , ∇ 2 f ( W 1 ) [ X ] ⟩ = − 2 λ max ( R 1 ) < 0. \langle X,\nabla^2 f(W_1)[X]\rangle=-2\lambda_{\max}(R_1)<0. ⟨ X , ∇ 2 f ( W 1 ) [ X ]⟩ = − 2 λ m a x ( R 1 ) < 0. Thus the comparison factor is a strict saddle in every hidden width m ≥ 2 m \ge2 m ≥ 2
Proof (of Corollary 47). Proposition 3 gives M 1 R 1 = 0 M_1R_1=0 M 1 R 1 = 0 W 1 W_1 W 1 M 1 M_1 M 1 ∥ W 1 R 1 ∥ F 2 = tr ( R 1 M 1 R 1 ) = 0 \lVert W_1R_1\rVert_F^2=\operatorname{tr}(R_1M_1R_1)=0 ∥ W 1 R 1 ∥ F 2 = tr ( R 1 M 1 R 1 ) = 0 G ( W 1 ) = 0 G(W_1)=0 G ( W 1 ) = 0 m ≥ 2 m \ge2 m ≥ 2 X X X ker W 1 T \ker W_1^{\mathsf{T}} ker W 1 T R 1 R_1 R 1 λ max ( R 1 ) \lambda_{\max}(R_1) λ m a x ( R 1 )
∥ X ∥ F = 1 , W 1 T X = 0 , tr ( X R 1 X T ) = λ max ( R 1 ) . \lVert X\rVert_F=1,\qquad W_1^{\mathsf{T}}X=0,\qquad\operatorname{tr}(XR_1X^{\mathsf{T}})=\lambda_{\max}(R_1). ∥ X ∥ F = 1 , W 1 T X = 0 , tr ( X R 1 X T ) = λ m a x ( R 1 ) . For every real s s s M 1 + s 2 X T X ⪰ 0 M_1+s^2X^{\mathsf{T}}X\succeq0 M 1 + s 2 X T X ⪰ 0
f ( W 1 + s X ) − f ( W 1 ) = − λ max ( R 1 ) s 2 + s 4 2 ⟨ X T X , L ( X T X ) ⟩ . f(W_1+sX)-f(W_1)=-\lambda_{\max}(R_1)s^2+\frac{s^4}{2}\langle X^{\mathsf{T}}X,\mathcal{L}(X^{\mathsf{T}}X)\rangle. f ( W 1 + s X ) − f ( W 1 ) = − λ m a x ( R 1 ) s 2 + 2 s 4 ⟨ X T X , L ( X T X )⟩ . The second derivative at zero is the claimed negative value. The variational characterization of the least Hessian eigenvalue proves the strict-saddle assertion. The same display supplies a PSD predictor of loss strictly below ϕ 1 \phi_1 ϕ 1 s s s
Entry sets and section-independent terminal loss
For the continuation (28) , put ℓ ∞ ( Z ) = lim τ → ∞ f ( φ K ( τ , Z ) ) \ell_\infty(Z)=\lim_{\tau\to\infty}f(\varphi_K(\tau,Z)) ℓ ∞ ( Z ) = lim τ → ∞ f ( φ K ( τ , Z )) K K K
Lemma 48 (Maximum terminal loss on a compact set). For any nonempty compact set E ⊂ R m × r E\subset\mathbb{R}^{m\times r} E ⊂ R m × r max Z ∈ E f ( φ K ( T , Z ) ) \max_{Z\in E}f(\varphi_K(T,Z)) max Z ∈ E f ( φ K ( T , Z )) T ≥ 0 T\ge0 T ≥ 0 T T T
lim T → ∞ max Z ∈ E f ( φ K ( T , Z ) ) = max Z ∈ E ℓ ∞ ( Z ) . (115) \lim_{T\to\infty}\max_{Z\in E}f(\varphi_K(T,Z))=\max_{Z\in E}\ell_\infty(Z). \tag*{(115)} T → ∞ lim Z ∈ E max f ( φ K ( T , Z )) = Z ∈ E max ℓ ∞ ( Z ) . (115) Proof (of Lemma 48). Proposition 1 gives global existence and decreasing loss. At each finite time, continuous dependence and compactness of E E E T ≥ 0 T\ge0 T ≥ 0 E E E T T T
{ Z ∈ E : f ( φ K ( T , Z ) ) ≥ lim s → ∞ max Y ∈ E f ( φ K ( s , Y ) ) } . \left\{Z\in E:f(\varphi_K(T,Z))\ge\lim_{s\to\infty}\max_{Y\in E}f(\varphi_K(s,Y))\right\}. { Z ∈ E : f ( φ K ( T , Z )) ≥ s → ∞ lim Y ∈ E max f ( φ K ( s , Y )) } . These closed subsets are nested as T T T E E E
⋂ T ≥ 0 { Z ∈ E : f ( φ K ( T , Z ) ) ≥ lim s → ∞ max Y ∈ E f ( φ K ( s , Y ) ) } ≠ ∅ . \bigcap_{T\ge0}\left\{Z\in E:f(\varphi_K(T,Z))\ge\lim_{s\to\infty}\max_{Y\in E}f(\varphi_K(s,Y))\right\}\ne\varnothing. T ≥ 0 ⋂ { Z ∈ E : f ( φ K ( T , Z )) ≥ s → ∞ lim Y ∈ E max f ( φ K ( s , Y )) } = ∅ . For every element of this intersection, passing to the limit along its own trajectory shows that its terminal loss is at least the limit on the right. Conversely, for every Z ∈ E Z \in E Z ∈ E T ≥ 0 T \ge0 T ≥ 0
ℓ ∞ ( Z ) ≤ f ( φ K ( T , Z ) ) ≤ max Y ∈ E f ( φ K ( T , Y ) ) . \ell_{\infty}(Z) \le f(\varphi_K(T,Z)) \le\max_{Y \in E} f(\varphi_K(T,Y)). ℓ ∞ ( Z ) ≤ f ( φ K ( T , Z )) ≤ Y ∈ E max f ( φ K ( T , Y )) . Taking the infimum over T T T
Definition 49 (Entry set and terminal-loss test). Fix a finite depth K K K b b b U U U ε \varepsilon ε
E ρ ( K , b , U ) : = { Z : W K ε n , b ( T ρ ε n ; U ) → Z for some ε n ↓ 0 } , (116) E_{\rho}(K,b,U) := \{Z : W_{K}^{\varepsilon_n,b}(T_{\rho}^{\varepsilon_n};U) \to Z \text{ for some } \varepsilon_n \downarrow0\}, \tag*{(116)} E ρ ( K , b , U ) := { Z : W K ε n , b ( T ρ ε n ; U ) → Z for some ε n ↓ 0 } , (116) and its terminal-loss test by
J ρ ( K , b , U ) : = lim T → ∞ max Z ∈ E ρ ( K , b , U ) f ( φ K ( T , Z ) ) . (117) \mathcal{J}_{\rho}(K,b,U) := \lim_{T \to\infty} \max_{Z \in E_{\rho}(K,b,U)} f(\varphi_K(T,Z)). \tag*{(117)} J ρ ( K , b , U ) := T → ∞ lim Z ∈ E ρ ( K , b , U ) max f ( φ K ( T , Z )) . (117) Choose a radius bound satisfying
0 < ρ ˉ < min { σ min ( C ) 2 n − 1 ∑ l ∥ x l ∥ 4 , ∥ M 1 ∥ F 2 } . (118) 0 < \bar{\rho} < \min\left\{\frac{\sigma_{\min}(C)}{2n^{-1}\sum_l \lVert x_l\rVert^4},\frac{\lVert M_1\rVert_F}{2}\right\}. \tag*{(118)} 0 < ρ ˉ < min { 2 n − 1 ∑ l ∥ x l ∥ 4 σ m i n ( C ) , 2 ∥ M 1 ∥ F } . (118) Here σ min ( C ) > 0 \sigma_{\min}(C) > 0 σ m i n ( C ) > 0 C C C
K l o s s ( b ) : = min { K ∈ N 0 : J ρ ( K , b , U ) < ϕ 1 for almost every U } . (119) K_{\mathrm{loss}}(b) := \min\{K \in\mathbb{N}_0 : \mathcal{J}_{\rho}(K,b,U) < \phi_1 \text{ for almost every } U\}. \tag*{(119)} K loss ( b ) := min { K ∈ N 0 : J ρ ( K , b , U ) < ϕ 1 for almost every U } . (119) Eligible depths have almost-sure first hits at every sufficiently small fixed section. The test uses any such radius below ρ ˉ \bar{\rho} ρ ˉ min ∅ = + ∞ \min\varnothing= +\infty min ∅ = + ∞
Lemma 50 (Well-defined terminal-loss test). Under Definition 49, each entry set E ρ ( K , b , U ) E_{\rho}(K,b,U) E ρ ( K , b , U ) 0 < ρ 1 < ρ 2 < ρ ˉ 0 < \rho_1 < \rho_2 < \bar{\rho} 0 < ρ 1 < ρ 2 < ρ ˉ ε \varepsilon ε ε \varepsilon ε
{ ℓ ∞ ( Z ) : Z ∈ E ρ 1 ( K , b , U ) } = { ℓ ∞ ( Z ) : Z ∈ E ρ 2 ( K , b , U ) } . \{\ell_{\infty}(Z) : Z \in E_{\rho_1}(K,b,U)\} = \{\ell_{\infty}(Z) : Z \in E_{\rho_2}(K,b,U)\}. { ℓ ∞ ( Z ) : Z ∈ E ρ 1 ( K , b , U )} = { ℓ ∞ ( Z ) : Z ∈ E ρ 2 ( K , b , U )} . Consequently J ρ 1 ( K , b , U ) = J ρ 2 ( K , b , U ) \mathcal{J}_{\rho_1}(K,b,U) = \mathcal{J}_{\rho_2}(K,b,U) J ρ 1 ( K , b , U ) = J ρ 2 ( K , b , U )
Proof (of Lemma 50). E ρ E_{\rho} E ρ M ⪰ 0 M \succeq0 M ⪰ 0 M ≤ q M \le q M ≤ q ∥ M ∥ F = ρ \lVert M\rVert_F = \rho ∥ M ∥ F = ρ
∥ W ∥ F 2 = tr M ≤ q ρ . \lVert W\rVert_F^2 = \operatorname{tr} M \le\sqrt{q}\rho. ∥ W ∥ F 2 = tr M ≤ q ρ . Thus every sequence of hit factors has a convergent subsequence. The set of their subsequential limits as ε ↓ 0 \varepsilon\downarrow0 ε ↓ 0 j j j 0 < ε < 1 / j 0 < \varepsilon< 1/j 0 < ε < 1/ j σ min ( C ) > 0 \sigma_{\min}(C) > 0 σ m i n ( C ) > 0 ρ ˉ \bar{\rho} ρ ˉ (118) . The measurement formula gives
∥ L ( M ) ∥ o p ≤ 1 n ∑ l ∣ x l T M x l ∣ ∥ x l ∥ 2 ≤ ( n − 1 ∑ l ∥ x l ∥ 4 ) ∥ M ∥ F . \lVert\mathcal{L}(M)\rVert_{\mathrm{op}} \le\frac{1}{n}\sum_l \lvert x_l^{\mathsf{T}} M x_l\rvert\lVert x_l\rVert^2 \le\left(n^{-1}\sum_l \lVert x_l\rVert^4\right)\lVert M\rVert_F. ∥ L ( M ) ∥ op ≤ n 1 l ∑ ∣ x l T M x l ∣ ∥ x l ∥ 2 ≤ ( n − 1 l ∑ ∥ x l ∥ 4 ) ∥ M ∥ F . For ∥ M ∥ F ≤ ρ ˉ \lVert M\rVert_F \le\bar{\rho} ∥ M ∥ F ≤ ρ ˉ
σ min ( R ( M ) ) ≥ σ min ( C ) − ∥ L ( M ) ∥ o p ≥ σ min ( C ) / 2 , ∥ G ( W ) ∥ F = 2 ∥ W R ( M ) ∥ F ≥ σ min ( C ) ∥ W ∥ F . \begin{aligned}
\sigma_{\min}(R(M)) &\ge\sigma_{\min}(C)-\lVert\mathcal{L}(M)\rVert_{\mathrm{op}} \ge\sigma_{\min}(C)/2,\\
\lVert G(W)\rVert_F &= 2\lVert W R(M)\rVert_F \ge\sigma_{\min}(C)\lVert W\rVert_F.
\end{aligned} σ m i n ( R ( M )) ∥ G ( W ) ∥ F ≥ σ m i n ( C ) − ∥ L ( M ) ∥ op ≥ σ m i n ( C ) /2 , = 2 ∥ W R ( M ) ∥ F ≥ σ m i n ( C ) ∥ W ∥ F . All derivatives in the next calculation use continuation time τ \tau τ
− f ′ ≥ a κ − 1 ∥ G ∥ F ∥ W ′ ∥ F ≥ σ min ( C ) a κ ∥ W ∥ F ∥ W ′ ∥ F ≥ σ min ( C ) 2 a κ ∣ d d τ tr ( W T W ) ∣ . -f' \ge a_{\kappa}^{-1}\lVert G\rVert_F\lVert W'\rVert_F \ge\frac{\sigma_{\min}(C)}{a_{\kappa}}\lVert W\rVert_F\lVert W'\rVert_F \ge\frac{\sigma_{\min}(C)}{2a_{\kappa}}\left\lvert\frac{d}{d\tau}\operatorname{tr}(W^{\mathsf{T}}W)\right\rvert. − f ′ ≥ a κ − 1 ∥ G ∥ F ∥ W ′ ∥ F ≥ a κ σ m i n ( C ) ∥ W ∥ F ∥ W ′ ∥ F ≥ 2 a κ σ m i n ( C ) d τ d tr ( W T W ) . Integrating the last inequality and then using the signed change in the trace, which is at most its total variation, proves
f ( W ( T ρ ε ) ) ≤ f ( W ( 0 ) ) − σ min ( C ) 2 a κ { tr M ( T ρ ε ) − tr M ( 0 ) } . f(W(T_{\rho}^{\varepsilon})) \le f(W(0))-\frac{\sigma_{\min}(C)}{2a_{\kappa}}\{\operatorname{tr}M(T_{\rho}^{\varepsilon})-\operatorname{tr}M(0)\}. f ( W ( T ρ ε )) ≤ f ( W ( 0 )) − 2 a κ σ m i n ( C ) { tr M ( T ρ ε ) − tr M ( 0 )} . The point denoted by T ρ ε T_{\rho}^{\varepsilon} T ρ ε
Take 0 < ρ 1 < ρ 2 < ρ ˉ 0<\rho_1<\rho_2<\bar{\rho} 0 < ρ 1 < ρ 2 < ρ ˉ tr M ( T ρ 1 ε ) ≥ ρ 1 \operatorname{tr}M(T_{\rho_1}^{\varepsilon})\ge\rho_1 tr M ( T ρ 1 ε ) ≥ ρ 1 tr M ( 0 ) → 0 \operatorname{tr}M(0)\to0 tr M ( 0 ) → 0 f ( W ( 0 ) ) → Φ ( 0 ) f(W(0))\to\Phi(0) f ( W ( 0 )) → Φ ( 0 ) ε \varepsilon ε
f ( W ( T ρ 1 ε ) ) ≤ Φ ( 0 ) − σ min ( C ) ρ 1 4 a κ . f(W(T_{\rho_1}^{\varepsilon}))\le\Phi(0)-\frac{\sigma_{\min}(C)\rho_1}{4a_{\kappa}}. f ( W ( T ρ 1 ε )) ≤ Φ ( 0 ) − 4 a κ σ m i n ( C ) ρ 1 . After this inner hit, and for as long as the outer hit has not occurred, loss monotonicity and (35) imply
σ min ( C ) ρ 1 4 a κ ≤ Φ ( 0 ) − Φ ( M ) ≤ ⟨ C , M ⟩ ≤ ∥ C ∥ F ∥ M ∥ F . \frac{\sigma_{\min}(C)\rho_1}{4a_{\kappa}}\le\Phi(0)-\Phi(M)\le\langle C,M\rangle\le\lVert C\rVert_F\lVert M\rVert_F. 4 a κ σ m i n ( C ) ρ 1 ≤ Φ ( 0 ) − Φ ( M ) ≤ ⟨ C , M ⟩ ≤ ∥ C ∥ F ∥ M ∥ F . Consequently the trajectory cannot return arbitrarily close to zero on that interval. Its loss decreases at a rate bounded away from zero:
− f ′ ≥ ∥ G ∥ F 2 ≥ σ min ( C ) 2 tr M ≥ σ min ( C ) 3 ρ 1 4 a κ ∥ C ∥ F . -f' \ge\lVert G\rVert_F^2\ge\sigma_{\min}(C)^2\operatorname{tr}M\ge\frac{\sigma_{\min}(C)^3\rho_1}{4a_{\kappa}\lVert C\rVert_F}. − f ′ ≥ ∥ G ∥ F 2 ≥ σ m i n ( C ) 2 tr M ≥ 4 a κ ∥ C ∥ F σ m i n ( C ) 3 ρ 1 . Because the loss is nonnegative, the elapsed continuation time between the two sections is at most 4 a κ Φ ( 0 ) ∥ C ∥ F / [ σ min ( C ) 3 ρ 1 ] 4a_{\kappa}\Phi(0)\lVert C\rVert_F/[\sigma_{\min}(C)^3\rho_1] 4 a κ Φ ( 0 ) ∥ C ∥ F / [ σ m i n ( C ) 3 ρ 1 ] ε \varepsilon ε
Extracting convergent hit factors and elapsed times from a sequence at either radius gives the two image-set relations
E ρ 2 ⊆ ⋃ Z ∈ E ρ 1 { φ κ ( s , Z ) : 0 ≤ s ≤ 4 a κ Φ ( 0 ) ∥ C ∥ F σ min ( C ) 3 ρ 1 } , E_{\rho_2}\subseteq\bigcup_{Z\in E_{\rho_1}}\left\{\varphi_{\kappa}(s,Z):0\le s\le\frac{4a_{\kappa}\Phi(0)\lVert C\rVert_F}{\sigma_{\min}(C)^3\rho_1}\right\}, E ρ 2 ⊆ Z ∈ E ρ 1 ⋃ { φ κ ( s , Z ) : 0 ≤ s ≤ σ m i n ( C ) 3 ρ 1 4 a κ Φ ( 0 ) ∥ C ∥ F } , and, for every Z ∈ E ρ 1 Z \in E_{\rho_1} Z ∈ E ρ 1
{ φ K ( s , Z ) : 0 ≤ s ≤ 4 a K Φ ( 0 ) ∥ C ∥ F σ min ( C ) ρ 1 } ∩ E ρ 2 ≠ ∅ . \left\{\varphi_K(s,Z): 0 \le s \le\frac{4a_K\Phi(0)\lVert C\rVert_F}{\sigma_{\min}(C)\rho_1}\right\} \cap E_{\rho_2} \ne\varnothing. { φ K ( s , Z ) : 0 ≤ s ≤ σ m i n ( C ) ρ 1 4 a K Φ ( 0 ) ∥ C ∥ F } ∩ E ρ 2 = ∅ . To verify these relations, apply (45) with η = c η ε \eta= c_\eta\varepsilon η = c η ε J ρ 1 = J ρ 2 \mathcal{J}_{\rho_1} = \mathcal{J}_{\rho_2} J ρ 1 = J ρ 2 ρ ˉ \bar{\rho} ρ ˉ
Strict loss transfer and all-time separation
Under the pointwise hypotheses of Theorem 7, the selected continuation has the same terminal predictor and loss for every positive entrance point and hidden frame. On its Gaussian-full event, at every sufficiently small fixed section,
J ρ ( K , b , U ) = ℓ K , d K ( b ) . (120) \mathcal{J}_{\rho}(K,b,U) = \ell_{K,d_K}(b). \tag*{(120)} J ρ ( K , b , U ) = ℓ K , d K ( b ) . (120) Every entry factor has this terminal loss. If K l o s s ( b ) < ∞ K_{\mathrm{loss}}(b) < \infty K loss ( b ) < ∞ K = K l o s s ( b ) K = K_{\mathrm{loss}}(b) K = K loss ( b ) K b y p a s s ( b ) ≤ K l o s s ( b ) K_{\mathrm{bypass}}(b) \le K_{\mathrm{loss}}(b) K bypass ( b ) ≤ K loss ( b )
Proof (of Theorem 10 and the entry-set characterization). For two positive points of the same local selected entrance, the change of time d t / d τ = ∥ G ( W ) ∥ F \mathrm{d}t/\mathrm{d}\tau= \lVert G(W)\rVert_F d t / d τ = ∥ G ( W ) ∥ F
φ K ( τ , φ K ( s , Z ) ) = φ K ( τ + s , Z ) \varphi_K(\tau,\varphi_K(s,Z)) = \varphi_K(\tau+s,Z) φ K ( τ , φ K ( s , Z )) = φ K ( τ + s , Z ) therefore identifies their continuations after a time shift. For O ∈ O ( m ) O \in O(m) O ∈ O ( m ) G ( O W ) = O G ( W ) G(OW) = OG(W) G ( O W ) = O G ( W ) H K , 0 ( O G ) = O H K , 0 ( G ) H_{K,0}(OG) = OH_{K,0}(G) H K , 0 ( O G ) = O H K , 0 ( G ) φ K ( τ , O Z ) = O φ K ( τ , Z ) \varphi_K(\tau,OZ) = O\varphi_K(\tau,Z) φ K ( τ , O Z ) = O φ K ( τ , Z ) M s , K M_{s,K} M s , K ℓ K , d K ( b ) \ell_{K,d_K}(b) ℓ K , d K ( b ) Φ ( M s , K ) = ℓ K , d K ( b ) \Phi(M_{s,K}) = \ell_{K,d_K}(b) Φ ( M s , K ) = ℓ K , d K ( b )
At a sufficiently small first-radius section, Theorem 7 gives the same limiting Gram for every entry factor. Two factors with the same Gram differ by a left orthogonal transformation: their corresponding columns define an isometry between column spans, which extends to R m \mathbb{R}^m R m
Assume now (29) and ℓ K , d K ( b ) < ϕ 1 \ell_{K,d_K}(b) < \phi_1 ℓ K , d K ( b ) < ϕ 1 Δ ∗ > 0 \Delta_* > 0 Δ ∗ > 0 ℓ K , d K ( b ) < ϕ 1 − 4 Δ ∗ \ell_{K,d_K}(b) < \phi_1 - 4\Delta_* ℓ K , d K ( b ) < ϕ 1 − 4 Δ ∗ ρ < ∥ M 1 ∥ F / 2 \rho< \lVert M_1\rVert_F/2 ρ < ∥ M 1 ∥ F /2 ϕ 1 − 4 Δ ∗ \phi_1 - 4\Delta_* ϕ 1 − 4 Δ ∗ Φ ( 0 ) > ϕ 1 \Phi(0) > \phi_1 Φ ( 0 ) > ϕ 1 ϕ 1 − 4 Δ ∗ \phi_1 - 4\Delta_* ϕ 1 − 4 Δ ∗
The predictor image of this compact segment is disjoint from M 1 M_1 M 1 W W W M 1 M_1 M 1
∥ W R 1 ∥ F 2 = tr ( R 1 M 1 R 1 ) = 0 \lVert WR_1\rVert_F^2 = \operatorname{tr}(R_1M_1R_1) = 0 ∥ W R 1 ∥ F 2 = tr ( R 1 M 1 R 1 ) = 0 and is stationary. Local uniqueness prevents a nonconstant solution from meeting such a factor at finite time. The segment is nonconstant because its endpoint has loss below ϕ 1 \phi_1 ϕ 1 M 1 M_1 M 1
Before the first radius hit the normalized distance is at least 1 − ρ / ∥ M 1 ∥ F 1-\rho/\lVert M_1\rVert_F 1 − ρ / ∥ M 1 ∥ F ϕ 1 − Δ ∗ \phi_1-\Delta_* ϕ 1 − Δ ∗
Φ ( M ) − ϕ 1 = − ⟨ R 1 , M − M 1 ⟩ + 1 2 ⟨ M − M 1 , L ( M − M 1 ) ⟩ (121) \Phi(M)-\phi_1=-\langle R_1,M-M_1\rangle+\frac{1}{2}\langle M-M_1,\mathcal{L}(M-M_1)\rangle \tag*{(121)} Φ ( M ) − ϕ 1 = − ⟨ R 1 , M − M 1 ⟩ + 2 1 ⟨ M − M 1 , L ( M − M 1 )⟩ (121) gives
Δ ∗ ≤ ⟨ R 1 , M − M 1 ⟩ ≤ ∥ R 1 ∥ F ∥ M − M 1 ∥ F . \Delta_*\le\langle R_1,M-M_1\rangle\le\lVert R_1\rVert_F\lVert M-M_1\rVert_F. Δ ∗ ≤ ⟨ R 1 , M − M 1 ⟩ ≤ ∥ R 1 ∥ F ∥ M − M 1 ∥ F . Choose the theorem constant d ∗ > 0 d_*>0 d ∗ > 0 1 − ρ / ∥ M 1 ∥ F 1-\rho/\lVert M_1\rVert_F 1 − ρ / ∥ M 1 ∥ F Δ ∗ / ( ∥ R 1 ∥ F ∥ M 1 ∥ F ) \Delta_*/(\lVert R_1\rVert_F\lVert M_1\rVert_F) Δ ∗ / (∥ R 1 ∥ F ∥ M 1 ∥ F )
For each fixed seed in the entrance event, apply one matching left orthogonal transformation to its actual path at the section and keep that transformation fixed during comparison. Theorem 7 and (45) , with η = c η ε \eta=c_\eta\varepsilon η = c η ε
∥ A ⊤ A − B ⊤ B ∥ F ≤ ( ∥ A ∥ F + ∥ B ∥ F ) ∥ A − B ∥ F . \lVert A^\top A-B^\top B\rVert_F\le(\lVert A\rVert_F+\lVert B\rVert_F)\lVert A-B\rVert_F. ∥ A ⊤ A − B ⊤ B ∥ F ≤ (∥ A ∥ F + ∥ B ∥ F ) ∥ A − B ∥ F . For sufficiently small ε \varepsilon ε d ∗ d_* d ∗ ϕ 1 − Δ ∗ \phi_1-\Delta_* ϕ 1 − Δ ∗ (121) gives the required distance bound. The pre-section estimate covers all earlier times. This proves both assertions, with seed-dependent initialization cutoff. The positive time change covers the forward half-line, so the loss crossing is finite in physical time as well. Applying the conclusion at K = K l o s s ( b ) K=K_{\mathrm{loss}}(b) K = K loss ( b ) K b y p a s s ( b ) ≤ K l o s s ( b ) K_{\mathrm{bypass}}(b)\le K_{\mathrm{loss}}(b) K bypass ( b ) ≤ K loss ( b )
Selected capture, equal loss, and the combined corollary
The full-width loss statement uses only the full-rank seed event. Capture follows from a finite selected segment. For equal loss at a different endpoint, a local analytic length estimate controls the remaining tail. We prove these parts in an order that makes the full-width loss improvement available when needed.
Proof (of Corollary 11 ).
(iii) The PSD rank-one perturbation in the proof of Corollary 47 has loss below ϕ 1 \phi_1 ϕ 1 Φ ∗ < ϕ 1 \Phi_*<\phi_1 Φ ∗ < ϕ 1 r r r U U U [27 ] . On this one Gaussian-full event, U U U r r r D ε , b D_{\varepsilon,b} D ε , b ε > 0 \varepsilon>0 ε > 0 b > 1 b>1 b > 1 Corollary 25 applies to each positive initialization and each finite K K K Φ ∗ \Phi_* Φ ∗ ( ϕ 1 + Φ ∗ ) / 2 (\phi_1+\Phi_*)/2 ( ϕ 1 + Φ ∗ ) /2 Φ ∗ \Phi_* Φ ∗
(i) Fix a seed in the entrance event and restart at a small fixed section, using constant left transformations as above. Fix any open predictor ball about M 1 M_1 M 1 ε \varepsilon ε d 1 d_1 d 1 d 1 → 0 d_1 \to0 d 1 → 0
(ii) Fix a small first-radius section and its selected factor Z Z Z Proposition 1 gives the limit W s W_s W s φ K ( ⋅ , Z ) \varphi_K(\cdot,Z) φ K ( ⋅ , Z ) W s T W s = M s , K W_s^{\mathsf{T}}W_s=M_{s,K} W s T W s = M s , K f ( W s ) = ϕ 1 f(W_s)=\phi_1 f ( W s ) = ϕ 1 M s , K ≠ M 1 M_{s,K}\ne M_1 M s , K = M 1 R > 0 R>0 R > 0 B R ( W s ) B_R(W_s) B R ( W s ) 1 2 ∥ M s , K − M 1 ∥ F \frac{1}{2}\lVert M_{s,K}-M_1\rVert_F 2 1 ∥ M s , K − M 1 ∥ F R R R [1 ] Lemma 2.1 gives c > 0 c>0 c > 0 0 ≤ θ < 1 0\le\theta<1 0 ≤ θ < 1
∥ ∇ f ( W ) ∥ F ≥ c ∣ f ( W ) − ϕ 1 ∣ θ \lVert\nabla f(W)\rVert_F \ge c\lvert f(W)-\phi_1\rvert^\theta ∥ ∇ f ( W ) ∥ F ≥ c ∣ f ( W ) − ϕ 1 ∣ θ throughout this ball. Choose a loss width
0 < a ≤ ( c ( 1 − θ ) R 8 a K ) 1 / ( 1 − θ ) . 0<a\le\left(\frac{c(1-\theta)R}{8a_K}\right)^{1/(1-\theta)}. 0 < a ≤ ( 8 a K c ( 1 − θ ) R ) 1/ ( 1 − θ ) . These constants concern only the selected continuation.
On a segment in this ball with ∣ f − ϕ 1 ∣ ≤ a \lvert f-\phi_1\rvert\le a ∣ f − ϕ 1 ∣ ≤ a − f ′ ≥ a K − 1 ∥ ∇ f ∥ F ∥ W ′ ∥ F -f'\ge a_K^{-1}\lVert\nabla f\rVert_F\lVert W'\rVert_F − f ′ ≥ a K − 1 ∥ ∇ f ∥ F ∥ W ′ ∥ F ϕ 1 \phi_1 ϕ 1 ( f − ϕ 1 ) 1 − θ (f-\phi_1)^{1-\theta} ( f − ϕ 1 ) 1 − θ a K a 1 − θ / [ c ( 1 − θ ) ] ≤ R / 8 a_Ka^{1-\theta}/[c(1-\theta)]\le R/8 a K a 1 − θ / [ c ( 1 − θ )] ≤ R /8 ϕ 1 \phi_1 ϕ 1 ( ϕ 1 − f ) 1 − θ (\phi_1-f)^{1-\theta} ( ϕ 1 − f ) 1 − θ θ < 1 \theta<1 θ < 1 R / 4 R/4 R /4
Choose a finite point of φ K ( ⋅ ; Z ) \varphi_K(\cdot;Z) φ K ( ⋅ ; Z ) B R / 8 ( W s ) B_{R/8}(W_s) B R /8 ( W s ) a / 4 a/4 a /4 ε ↓ 0 \varepsilon\downarrow0 ε ↓ 0 Z Z Z Z T Z Z^{\mathsf{T}}Z Z T Z
For each seed in the entrance event and sufficiently small ε \varepsilon ε B R / 4 ( W s ) B_{R/4}(W_s) B R /4 ( W s ) a / 2 a/2 a /2 ϕ 1 − a \phi_1-a ϕ 1 − a B R ( W s ) B_R(W_s) B R ( W s ) 3 R / 4 3R/4 3 R /4 R / 4 R/4 R /4 1 2 ∥ M s , K − M 1 ∥ F \frac{1}{2}\lVert M_{s,K}-M_1\rVert_F 2 1 ∥ M s , K − M 1 ∥ F ϕ 1 − a \phi_1-a ϕ 1 − a (121) and monotonicity give, at every later time,
a ≤ ⟨ R 1 , M − M 1 ⟩ − 1 2 ⟨ M − M 1 , L ( M − M 1 ) ⟩ ≤ ∥ R 1 ∥ F ∥ M − M 1 ∥ F . a\le\langle R_{1,M}-M_1\rangle-\frac{1}{2}\langle M-M_1,\mathcal{L}(M-M_1)\rangle\le\lVert R_{1}\rVert_F\lVert M-M_1\rVert_F. a ≤ ⟨ R 1 , M − M 1 ⟩ − 2 1 ⟨ M − M 1 , L ( M − M 1 )⟩ ≤ ∥ R 1 ∥ F ∥ M − M 1 ∥ F . Before the first section, take its radius below ∥ M 1 ∥ F / 2 \lVert M_1\rVert_F/2 ∥ M 1 ∥ F /2 ∥ M 1 ∥ F / 2 \lVert M_1\rVert_F/2 ∥ M 1 ∥ F /2 M 1 M_1 M 1 M 1 M_1 M 1 a / ∥ R 1 ∥ F a/\lVert R_1\rVert_F a / ∥ R 1 ∥ F ∥ M 1 ∥ F \lVert M_1\rVert_F ∥ M 1 ∥ F d ∗ > 0 d_*>0 d ∗ > 0 m ≥ r m \ge r m ≥ r
(iv) If ℓ K , d K ( b ) < ϕ 1 \ell_{K,d_K}(b) < \phi_1 ℓ K , d K ( b ) < ϕ 1 Theorem 10 gives bypass, and Φ ( M s , K ) = ℓ K , d K ( b ) \Phi(M_{s,K}) = \ell_{K,d_K}(b) Φ ( M s , K ) = ℓ K , d K ( b ) M s , K ≠ M 1 M_{s,K} \ne M_1 M s , K = M 1 ℓ K , d K ( b ) = ϕ 1 \ell_{K,d_K}(b) = \phi_1 ℓ K , d K ( b ) = ϕ 1 M s , K ≠ M 1 M_{s,K} \ne M_1 M s , K = M 1 Corollary 11 (ii) gives bypass. These cover the entire premise ℓ K , d K ( b ) ≤ ϕ 1 \ell_{K,d_K}(b) \le\phi_1 ℓ K , d K ( b ) ≤ ϕ 1 M 1 M_1 M 1 Corollary 11 (i) gives visitation, which contradicts the positive all-time distance in the definition of bypass. This proves the reverse implication and the equivalence.
Rank-one and constant-rank consequences
A rank-one continuation follows the ordinary gradient-flow path after a positive change of time. This gives a lower bound on the loss-test depth, including repeated unused eigenvalues. Fixed-rank entrance uniqueness also makes the selected loss constant on an admissible exponent interval where the entrance rank is unchanged.
Proposition 51 (The rank-one lower bound on loss-test depth) Assume the empirical task satisfies Assumption 2 and (29) , with b > 1 b > 1 b > 1 c η > 0 c_\eta> 0 c η > 0 W ( t ) / t → Q 1 , K W(t)/t \to Q_{1,K} W ( t ) / t → Q 1 , K ϕ 1 \phi_1 ϕ 1 K K K a K λ 2 / λ 1 < b a_K\lambda_2/\lambda_1 < b a K λ 2 / λ 1 < b ϕ 1 \phi_1 ϕ 1 λ 2 , … , λ r \lambda_2,\ldots,\lambda_r λ 2 , … , λ r
K l o s s ( b ) ≥ ⌈ log 3 / 2 ( b λ 1 / λ 2 ) ⌉ . (122) K_{\mathrm{loss}}(b) \ge\left\lceil\log_{3/2}(b\lambda_1/\lambda_2) \right\rceil. \tag*{(122)} K loss ( b ) ≥ ⌈ log 3/2 ( b λ 1 / λ 2 ) ⌉ . (122) Proof (of Proposition 51).
Rank-one continuation.
For a rank-one factor, G ( W ) G(W) G ( W ) p K ( 1 ) = 1 p_K(1) = 1 p K ( 1 ) = 1
P K ( G ( W ) ∥ G ( W ) ∥ F ) = G ( W ) ∥ G ( W ) ∥ F , H K , 0 ( G ( W ) ) = G ( W ) . P_K\left(\frac{G(W)}{\lVert G(W)\rVert_F}\right) = \frac{G(W)}{\lVert G(W)\rVert_F}, \qquad H_{K,0}(G(W)) = G(W). P K ( ∥ G ( W ) ∥ F G ( W ) ) = ∥ G ( W ) ∥ F G ( W ) , H K , 0 ( G ( W )) = G ( W ) . The second identity also holds at G ( W ) = 0 G(W) = 0 G ( W ) = 0 Q 1 , K = e 1 v 1 T Q_{1,K} = e_1v_1^{\mathsf T} Q 1 , K = e 1 v 1 T K K K Lemma 45 identifies a single such rank-one trajectory.
To match its gradient-flow parametrization to Proposition 3 , use (24) to obtain
∥ G ( W ( t ) ) ∥ F = 2 λ 1 t + O ( t 3 ) , d τ d t = 1 2 λ 1 t + O ( t ) . \lVert G(W(t))\rVert_F = 2\lambda_1 t + O(t^3), \qquad\frac{\mathrm{d}\tau}{\mathrm{d}t} = \frac{1}{2\lambda_1 t} + O(t). ∥ G ( W ( t )) ∥ F = 2 λ 1 t + O ( t 3 ) , d t d τ = 2 λ 1 t 1 + O ( t ) . Choose the additive origin of continuation time so that τ = ( 2 λ 1 ) − 1 log t + O ( t 2 ) \tau= (2\lambda_1)^{-1}\log t + O(t^2) τ = ( 2 λ 1 ) − 1 log t + O ( t 2 )
M ( τ ) = e 4 λ 1 τ v 1 v 1 T + O ( e 8 λ 1 τ ) ( τ → − ∞ ) . M(\tau) = e^{4\lambda_1\tau}v_1v_1^{\mathsf T} + O(e^{8\lambda_1\tau}) \qquad(\tau\to-\infty). M ( τ ) = e 4 λ 1 τ v 1 v 1 T + O ( e 8 λ 1 τ ) ( τ → − ∞ ) . Starting this solution at successively earlier times gives initial predictors whose normalized projection on v 1 v 1 T v_1v_1^{\mathsf T} v 1 v 1 T Proposition 3 therefore identifies it, up to a constant time shift, with M G \mathcal{M}^{G} M G ( 4 λ 1 ) − 1 log ( 1 / ⟨ M ( τ ) , v 1 v 1 T ⟩ ) (4\lambda_1)^{-1}\log(1/\langle M(\tau),v_1v_1^{\mathsf T}\rangle) ( 4 λ 1 ) − 1 log ( 1/ ⟨ M ( τ ) , v 1 v 1 T ⟩) − τ + o ( 1 ) -\tau+ o(1) − τ + o ( 1 ) M 1 M_1 M 1 ϕ 1 \phi_1 ϕ 1
Actual rank-one selection below the threshold.
Fix K K K a K λ 2 / λ 1 < b a_K\lambda_2/\lambda_1 < b a K λ 2 / λ 1 < b u ⊤ v 1 ≠ 0 u^\top v_1 \ne0 u ⊤ v 1 = 0
g 2 ( 1 ) = a K λ 2 / λ 1 − 1 , ( b − 1 ) − g 2 ( 1 ) = b − a K λ 2 / λ 1 > 0. g_2^{(1)} = a_K\lambda_2/\lambda_1 - 1, \qquad(b-1)-g_2^{(1)} = b-a_K\lambda_2/\lambda_1 > 0. g 2 ( 1 ) = a K λ 2 / λ 1 − 1 , ( b − 1 ) − g 2 ( 1 ) = b − a K λ 2 / λ 1 > 0. By Lemmas 28 and 32 , a sufficiently large fixed σ 0 ≥ 1 \sigma_0 \ge1 σ 0 ≥ 1 ϵ σ 0 \epsilon\sigma_0 ϵ σ 0
∥ Z ε ( ϵ σ 0 ) ∥ F = O U ( ϵ b − 1 ) . \left\|Z^\varepsilon(\epsilon\sigma_0)\right\|_F = O_U(\epsilon^{b-1}). ∥ Z ε ( ϵ σ 0 ) ∥ F = O U ( ϵ b − 1 ) . The rank-one tangent equation is stable by Lemma 35 , using λ 1 > λ 2 \lambda_1 > \lambda_2 λ 1 > λ 2 Lemma 39 , which uses only the largest unused rate, on the interval stopped at the first chart boundary or at a fixed small t ∗ t_* t ∗ 1 / σ 0 1/\sigma_0 1/ σ 0 t ∗ t_* t ∗ ∥ E ∥ o p ≤ ( b − a K λ 2 / λ 1 ) / 2 \|E\|_{\mathrm{op}} \le(b-a_K\lambda_2/\lambda_1)/2 ∥ E ∥ op ≤ ( b − a K λ 2 / λ 1 ) /2
∥ Z ε ( t ) ∥ F = O U ( ϵ b − 1 ( t / ( ϵ σ 0 ) ) g 2 ( 1 ) + ( b − a K λ 2 / λ 1 ) / 2 ) . \left\|Z^\varepsilon(t)\right\|_F = O_U\left(\epsilon^{b-1}\left(t/(\epsilon\sigma_0)\right)^{g_2^{(1)}+(b-a_K\lambda_2/\lambda_1)/2}\right). ∥ Z ε ( t ) ∥ F = O U ( ϵ b − 1 ( t / ( ϵ σ 0 ) ) g 2 ( 1 ) + ( b − a K λ 2 / λ 1 ) /2 ) . After the task, K , b K,b K , b σ 0 \sigma_0 σ 0 t ∗ t_* t ∗ ϵ \epsilon ϵ
ϵ min { b − 1 , ( b − a K λ 2 / λ 1 ) / 2 } , \epsilon^{\min\{b-1,(b-a_K\lambda_2/\lambda_1)/2\}}, ϵ m i n { b − 1 , ( b − a K λ 2 / λ 1 ) /2 } , which tends to zero. Its squared logarithmic-time integral also tends to zero, since the interval length is O U ( log ( 1 / ϵ ) ) O_U(\log(1/\epsilon)) O U ( log ( 1/ ϵ ))
The tangent inequality in Lemma 39 excludes a first chart exit: after multiplication by e ω v e^{\omega v} e ω v ∥ y ∥ \|y\| ∥ y ∥ C / ω C/\omega C / ω ∥ Z ∥ F + t ∗ 2 + 1 / σ 0 \|Z\|_F+t_*^2+1/\sigma_0 ∥ Z ∥ F + t ∗ 2 + 1/ σ 0 t ∗ 2 t_*^2 t ∗ 2 1 / σ 0 1/\sigma_0 1/ σ 0 ϵ \epsilon ϵ t ∗ t_* t ∗
We have established ( ϵ σ 0 ) → 0 (\epsilon\sigma_0)\to0 ( ϵ σ 0 ) → 0 ( ϵ σ 0 ) ≥ ϵ (\epsilon\sigma_0)\ge\epsilon ( ϵ σ 0 ) ≥ ϵ Z ε ( t ) → 0 Z^\varepsilon(t)\to0 Z ε ( t ) → 0 t > 0 t>0 t > 0 W K ε ( 0 ; U ) → 0 W_K^\varepsilon(0;U)\to0 W K ε ( 0 ; U ) → 0 Lemma 45 now gives the rank-one first-radius limit and existence of all its sufficiently small actual sections. Every limiting factor on such a section has the same Gram matrix as the rank-one solution already identified. Left orthogonal equivariance and Lemma 48 give J ρ ( K , b , U ) = ϕ 1 J_\rho(K,b,U)=\phi_1 J ρ ( K , b , U ) = ϕ 1
Every nonnegative integer strictly below the ceiling in (122) satisfies the strict inequality just proved and hence fails the terminal-loss test. This proves the bound, while retaining the original strict inequality at the boundary.
Proposition 52 (Terminal loss on a constant-rank exponent interval) Fix a finite depth K K K b b b Theorem 7 hold and d K ( b ) d_K(b) d K ( b ) b b b
Proof (of Proposition 52). The boundary matrix Q d K ( b ) , K Q_{d_K(b),K} Q d K ( b ) , K K K K d K ( b ) d_K(b) d K ( b ) Theorem 7 identifies the entrance with the unique fixed-rank solution having this boundary condition. Hence the selected continuation is the same throughout the exponent interval, up to its positive starting point and hidden orthogonal frame. The continuation identity proved with (120) gives independence of terminal loss from these choices. ■
Residual-Inertia Certificate and the Depth Tail
A spectral estimate first gives an explicit cutoff for entrance capacity. The critical loss band then converts preserved finite-time rank into a strict selected loss decrease. The final proof combines these two inputs with the all-time transfer theorem.
Capacity saturation and an explicit sufficient cutoff
For 2 ≤ d ≤ p 2 \le d \le p 2 ≤ d ≤ p
a K λ d > ( 1 + 2 ( d − 1 ) max { 1 , b − 1 } ) ( ∑ h = 1 d λ h 2 ) 1 / 2 ⟹ d K ( b ) ≥ d . (123) a_K\lambda_d > \left(1 + 2(d-1)\max\{1,b-1\}\right)\left(\sum_{h=1}^{d}\lambda_h^2\right)^{1/2}
\quad\Longrightarrow\quad d_K(b) \ge d. \tag*{(123)} a K λ d > ( 1 + 2 ( d − 1 ) max { 1 , b − 1 } ) ( h = 1 ∑ d λ h 2 ) 1/2 ⟹ d K ( b ) ≥ d . (123) The estimate proves that every intermediate clock completes; the strict integer convention in (32) then applies it with d = p d=p d = p
Proof (the sufficient bound (123)). At any existing support { 1 , … , h } \{1,\ldots,h\} { 1 , … , h }
s h , K 2 = ∑ a ≤ h λ a 2 p K ( s a ) 2 ≤ ∑ a = 1 h λ a 2 ≤ ∑ a = 1 d λ a 2 . s_{h,K}^2=\sum_{a\le h}\lambda_a^2p_K(s_a)^2\le\sum_{a=1}^{h}\lambda_a^2\le\sum_{a=1}^{d}\lambda_a^2. s h , K 2 = a ≤ h ∑ λ a 2 p K ( s a ) 2 ≤ a = 1 ∑ h λ a 2 ≤ a = 1 ∑ d λ a 2 . Condition (123) and Lemma 34 therefore give successive existence of all supports up to d d d 2 ( d − 1 ) max { 1 , b − 1 } 2(d-1)\max\{1,b-1\} 2 ( d − 1 ) max { 1 , b − 1 } 1 − λ h + 1 / λ h < 1 1-\lambda_{h+1}/\lambda_h<1 1 − λ h + 1 / λ h < 1
x h + 1 ≤ x h + b − 1 + x h 2 ( d − 1 ) max { 1 , b − 1 } . x_{h+1}\le x_h+\frac{b-1+x_h}{2(d-1)\max\{1,b-1\}}. x h + 1 ≤ x h + 2 ( d − 1 ) max { 1 , b − 1 } b − 1 + x h . Starting at x 1 = 0 x_1=0 x 1 = 0
x h ≤ ( b − 1 ) { ( 1 + 1 2 ( d − 1 ) max { 1 , b − 1 } ) h − 1 − 1 } . x_h\le(b-1)\left\{\left(1+\frac{1}{2(d-1)\max\{1,b-1\}}\right)^{h-1}-1\right\}. x h ≤ ( b − 1 ) { ( 1 + 2 ( d − 1 ) max { 1 , b − 1 } 1 ) h − 1 − 1 } . For 2 ≤ h ≤ d 2\le h\le d 2 ≤ h ≤ d
( 1 + 1 2 ( d − 1 ) max { 1 , b − 1 } ) h − 1 ≤ exp ( h − 1 2 ( d − 1 ) max { 1 , b − 1 } ) < 1 + h − 1 ( d − 1 ) max { 1 , b − 1 } . \left(1+\frac{1}{2(d-1)\max\{1,b-1\}}\right)^{h-1}
\le\exp\left(\frac{h-1}{2(d-1)\max\{1,b-1\}}\right)
<1+\frac{h-1}{(d-1)\max\{1,b-1\}}. ( 1 + 2 ( d − 1 ) max { 1 , b − 1 } 1 ) h − 1 ≤ exp ( 2 ( d − 1 ) max { 1 , b − 1 } h − 1 ) < 1 + ( d − 1 ) max { 1 , b − 1 } h − 1 . The last exponent is at most 1 / 2 1/2 1/2 e z − 1 < 2 z e^z-1<2z e z − 1 < 2 z 0 < z ≤ 1 / 2 0<z\le1/2 0 < z ≤ 1/2 e s ≤ e 1 / 2 < 2 e^s\le e^{1/2}<2 e s ≤ e 1/2 < 2 x h x_h x h ( b − 1 ) / max { 1 , b − 1 } ≤ 1 (b-1)/\max\{1,b-1\}\le1 ( b − 1 ) / max { 1 , b − 1 } ≤ 1
Proof (of Proposition 13 ). The strict floor-plus-one convention in (32) implies, for K ≥ K c a p ( b ) K \ge K_{\mathrm{cap}}(b) K ≥ K cap ( b )
a K λ p > [ 1 + 2 ( p − 1 ) max { 1 , b − 1 } ] ( ∑ i = 1 p λ i 2 ) 1 / 2 . a_K\lambda_p > \left[1+2(p-1)\max\{1,b-1\}\right]\left(\sum_{i=1}^{p}\lambda_i^2\right)^{1/2}. a K λ p > [ 1 + 2 ( p − 1 ) max { 1 , b − 1 } ] ( i = 1 ∑ p λ i 2 ) 1/2 . Apply the sufficient bound (123) with d = p d=p d = p p p p Theorem 7 gives the actual rank-p p p K ≥ K c a p ( b ) K \ge K_{\mathrm{cap}}(b) K ≥ K cap ( b )
The function K c a p ( b ) K_{\mathrm{cap}}(b) K cap ( b ) b b b 1 < b ≤ B 1<b\le B 1 < b ≤ B K ≥ K c a p ( B ) K\ge K_{\mathrm{cap}}(B) K ≥ K cap ( B ) B C , K \mathcal{B}_{C,K} B C , K ( 1 , B ] (1,B] ( 1 , B ] K c a p ( B ) K_{\mathrm{cap}}(B) K cap ( B ) Proposition 37 gives at most p − 1 p-1 p − 1
The critical loss band and sufficient rank condition
The definitions in (31) concern stationary PSD predictors with loss between ϕ 1 \phi_1 ϕ 1 Φ ( 0 ) \Phi(0) Φ ( 0 )
Lemma 53 (Positive residual directions above the comparison loss). Under (29) , C ≥ ϕ 1 ( q ) \mathcal{C}^{(q)}_{\ge\phi_1} C ≥ ϕ 1 ( q ) p ∗ ( q ) p_*^{(q)} p ∗ ( q ) 1 ≤ p ∗ ( q ) ≤ n + ( R 1 ) ≤ n + ( C ) − 1 1\le p_*^{(q)}\le n_+(R_1)\le n_+(C)-1 1 ≤ p ∗ ( q ) ≤ n + ( R 1 ) ≤ n + ( C ) − 1
Proof (of Lemma 53 ). The set is closed and lies in a compact PSD sublevel, so it is compact; it is nonempty because it contains M 1 M_1 M 1 p ∗ ( q ) p_*^{(q)} p ∗ ( q ) 0 0 0 r r r
The positive residual eigenvalue at M 1 M_1 M 1 ϕ 1 \phi_1 ϕ 1 Corollary 47 . Thus the PSD-optimal loss is strictly below ϕ 1 \phi_1 ϕ 1 (31) is consequently nonglobal. If one had R ( M ) ⪯ 0 R(M)\preceq0 R ( M ) ⪯ 0 Lemma 24 , a contradiction. It must have at least one positive residual eigenvalue.
Finally M 1 ∈ C ≥ ϕ 1 ( q ) M_1\in\mathcal{C}^{(q)}_{\ge\phi_1} M 1 ∈ C ≥ ϕ 1 ( q ) p ∗ ( q ) ≤ n + ( R 1 ) p_*^{(q)}\le n_+(R_1) p ∗ ( q ) ≤ n + ( R 1 ) M 1 M_1 M 1 n + ( R 1 ) ≤ n + ( C ) − 1 n_+(R_1)\le n_+(C)-1 n + ( R 1 ) ≤ n + ( C ) − 1
Corollary 54 (A sufficient rank condition for loss crossing). Under Theorem 7 and (29) , if d K ( b ) + p ∗ ( q ) > r d_K(b)+p_*^{(q)}>r d K ( b ) + p ∗ ( q ) > r ℓ K , d K ( b ) < ϕ 1 \ell_{K,d_K(b)}<\phi_1 ℓ K , d K ( b ) < ϕ 1 K K K (119) .
Proof (of Corollary 54 ). Choose a small positive time on the selected entrance. By (24) , its predictor has the form
M ( t ) = t 2 Q d K ( b ) , K Q d K ( b ) , K T + O ( t 4 ) . M(t)=t^2Q_{d_K(b),K}Q_{d_K(b),K}^{\mathsf{T}}+O(t^4). M ( t ) = t 2 Q d K ( b ) , K Q d K ( b ) , K T + O ( t 4 ) . The row space of Q d K ( b ) , K Q_{d_K(b),K} Q d K ( b ) , K ⟨ C , Q d K ( b ) , K ⊤ Q d K ( b ) , K ⟩ = ∑ i = 1 d K ( b ) λ i q i 2 > 0 \langle C,Q_{d_K(b),K}^{\top}Q_{d_K(b),K}\rangle= \sum_{i=1}^{d_K(b)}\lambda_iq_i^2 > 0 ⟨ C , Q d K ( b ) , K ⊤ Q d K ( b ) , K ⟩ = ∑ i = 1 d K ( b ) λ i q i 2 > 0 (35) now gives
Φ ( M ( t ) ) = Φ ( 0 ) − t 2 ⟨ C , Q d K ( b ) , K ⊤ Q d K ( b ) , K ⟩ + O ( t 4 ) < Φ ( 0 ) \Phi(M(t)) = \Phi(0) - t^2\langle C,Q_{d_K(b),K}^{\top}Q_{d_K(b),K}\rangle+ O(t^4) < \Phi(0) Φ ( M ( t )) = Φ ( 0 ) − t 2 ⟨ C , Q d K ( b ) , K ⊤ Q d K ( b ) , K ⟩ + O ( t 4 ) < Φ ( 0 ) for a sufficiently small fixed positive time. Start the continuation there. Proposition 1 gives convergence of the selected factor and preserves its initial rank d K ( b ) d_K(b) d K ( b ) M s , K M_{s,K} M s , K d K ( b ) ≤ q d_K(b) \le q d K ( b ) ≤ q M s , K R ( M s , K ) = 0 M_{s,K}R(M_{s,K}) = 0 M s , K R ( M s , K ) = 0 Φ ( 0 ) \Phi(0) Φ ( 0 )
If Φ ( M s , K ) ≥ ϕ 1 \Phi(M_{s,K}) \ge\phi_1 Φ ( M s , K ) ≥ ϕ 1 M s , K M_{s,K} M s , K C ≥ ϕ 1 ( q ) \mathcal{C}_{\ge\phi_1}^{(q)} C ≥ ϕ 1 ( q ) p ∗ ( q ) p_*^{(q)} p ∗ ( q ) Lemma 12 , applied with the initial rank d K ( b ) d_K(b) d K ( b )
d K ( b ) + n + ( R ( M s , K ) ) ≤ r . d_K(b) + n_+(R(M_{s,K})) \le r. d K ( b ) + n + ( R ( M s , K )) ≤ r . This contradicts d K ( b ) + p ∗ ( q ) > r d_K(b) + p_*^{(q)} > r d K ( b ) + p ∗ ( q ) > r ℓ K , d K ( b ) = Φ ( M s , K ) < ϕ 1 \ell_{K,d_K(b)} = \Phi(M_{s,K}) < \phi_1 ℓ K , d K ( b ) = Φ ( M s , K ) < ϕ 1 (120) makes the same strict inequality hold for the terminal-loss test at every sufficiently small section, proving success in (119) . ■
The eventual finite-depth theorem
The proof is pointwise in each finite depth. Its final countable intersection combines the probability-one events while retaining the individual constants and initialization cutoffs.
Proof (of Theorem 14 ). Fix b > 1 b > 1 b > 1 c η > 0 c_\eta> 0 c η > 0 K ≥ K c a p ( b ) K \ge K_{\mathrm{cap}}(b) K ≥ K cap ( b ) Proposition 13 proves existence of all candidate supports, strict completion of every clock, and the pointwise applicability of Theorem 7 . In particular the actual selected entrance has rank d K ( b ) = p d_K(b) = p d K ( b ) = p
The inequality (33) is therefore the hypothesis d K ( b ) + p ∗ ( q ) > r d_K(b) + p_*^{(q)} > r d K ( b ) + p ∗ ( q ) > r Corollary 54 . That corollary gives ℓ K , p < ϕ 1 \ell_{K,p} < \phi_1 ℓ K , p < ϕ 1 p p p Theorem 10 now supplies positive constants d ∗ , K d_{*,K} d ∗ , K Δ ∗ , K \Delta_{*,K} Δ ∗ , K (34) . Its proof compares a fixed finite continuation segment, and then uses loss monotonicity for the full tail, so the second inequality in (34) holds at a finite physical time for each positive initialization.
Perform this argument separately for each integer K ≥ K c a p ( b ) K \ge K_{\mathrm{cap}}(b) K ≥ K cap ( b ) K K K d ∗ , K d_{*,K} d ∗ , K Δ ∗ , K \Delta_{*,K} Δ ∗ , K K K K
