Introduction
Finite-sum minimax problems arise when a learning objective is optimized against an adversary or against a worst-case weighting of the training examples, with one component per example, as in adversarial training (Madry et al., [13 ] ), distributionally robust optimization (Namkoong and Duchi, [15 ] ) and agnostic federated learning (Mohri et al., [14 ] ). When the adversary’s objective is concave and carries a strongly convex penalty, the inner maximization
©2026 Yuxing Peng, Zhiqing Tang and Weijia Jia.
License: CC-BY 4.0, see https://creativecommons.org/licenses/by/4.0/. is strongly concave; gradient descent ascent and its variance-reduced variants are standard methods for the resulting nonconvex–strongly-concave (NC–SC) problems [10 , 11 ] . The oracle complexity of such a problem is the number of component queries that any method must make. Finite-sum lower bounds have been proved for linear-span algorithms [27 ] , a restricted proximal protocol [7 ] and zero-respecting algorithms [30 ] ; none of them covers arbitrary randomized adaptive algorithms. An exact component proximal query also carries more information than a gradient. It is granted at every legal parameter, for which the regularized component problem is globally convex–concave and has a saddle point; it then returns a global saddle point, which can expose hidden directions that gradients alone cannot reveal. We ask whether such information changes the complexity of the finite-sum problem
min p ∈ R d V ( p ) , V ( p ) = max y ∈ R d y F ( p , y ) , F = 1 n ∑ i = 1 n F i , (1) \min_{p\in\mathbb{R}^{d}} V(p), \qquad V(p)=\max_{y\in\mathbb{R}^{d_y}} F(p,y), \qquad F=\frac{1}{n}\sum_{i=1}^{n}F_i, \tag*{(1)} p ∈ R d min V ( p ) , V ( p ) = y ∈ R d y max F ( p , y ) , F = n 1 i = 1 ∑ n F i , (1)
where F F F is nonconvex in p p p , μ \mu μ -strongly concave in y y y , and the component gradients have joint mean-squared smoothness L L L . The condition number κ = L / μ \kappa=L/\mu κ = L / μ measures the difficulty of the inner maximization. The output criterion is a small gradient of the value function V V V , also called the primal function [27 , 30 ] .
For primal gap Δ \Delta Δ and accuracy ε \varepsilon ε , put S = L Δ / ε 2 S=L\Delta/\varepsilon^2 S = L Δ/ ε 2 and h ( n , κ ) = min { n κ , n 3 / 4 κ } h(n,\kappa)=\min\{\sqrt{n}\,\kappa,n^{3/4}\sqrt{\kappa} \} h ( n , κ ) = min { n κ , n 3/4 κ } . On the normalized-start class of § 2 and for ε 2 \varepsilon^2 ε 2 below a universal multiple of L Δ L\Delta L Δ , the lower and upper bounds on the oracle complexity are
Ω ( n + h ( n , κ ) S ) and O ( ( n + h ( n , κ ) S ) log 2 ( 2 + κ ) ) , \Omega\bigl(n+h(n,\kappa)S\bigr)\quad\text{and}\quad O\bigl((n+h(n,\kappa)S)\log^2(2+\kappa)\bigr), Ω ( n + h ( n , κ ) S ) and O ( ( n + h ( n , κ ) S ) log 2 ( 2 + κ ) ) ,
so they match up to the factor log 2 ( 2 + κ ) \log^2(2+\kappa) log 2 ( 2 + κ ) . Both bounds cover every κ ≥ 1 \kappa\ge1 κ ≥ 1 in the stated accuracy range, with primal-gap and joint-gradient data supplied at initialization.
For large condition numbers, our hard instances chain M ≍ S M\asymp S M ≍ S nonlinear primal stages through dual paths with m ≍ κ / n 1 / 4 m\asymp\sqrt{\kappa}/n^{1/4} m ≍ κ / n 1/4 nodes, and each edge of a path is assigned independently to one fixed component, its owner. A query reveals an edge only if it is addressed to the owner, so locating each owner takes on the order of n n n component queries, giving
n ⋅ m ⋅ M ≳ n 3 / 4 κ L Δ ε 2 . n\cdot m\cdot M\gtrsim n^{3/4}\sqrt{\kappa}\,\frac{L\Delta}{\varepsilon^2}. n ⋅ m ⋅ M ≳ n 3/4 κ ε 2 L Δ .
Proximal responses threaten this count in two ways. A response is a global saddle point, so one query could reveal edges far beyond the current one; and if the set of legal parameters depended on the component, testing legality alone would reveal owners. A public coordinate, with the same curvature in every component, makes the legal parameters identical for all components. Edge interactions that vanish while the hidden coordinates are small, together with a reflection symmetry, confine the unique saddle to the spans of the current query centers and the blocks already revealed. A query to a component then reveals only its run of consecutive edges and stops at the first edge with another owner. These constructions keep the primal scale L L L and the scale L / n L/\sqrt{n} L / n of the dual edge interactions.
Our contributions are as follows.
We construct hard instances for exact proximal oracles whose components are individually strongly concave, whose legal responses are unique, and whose legal domain is common to all components (Appendices A–D).
We prove the lower bound for every condition number against arbitrary randomized algorithms with exact values, gradients, and legal saddle-proximal responses, using long dual paths, two-node paths, and a flat primal coupling (Theorem 1); it also holds under the protocol of Han et al. [7 ] (Corollary 4).
We match it up to log 2 ( 2 + κ ) \log^{2}(2+\kappa) log 2 ( 2 + κ ) with a component-gradient method that pays for initialization, snapshots, refreshes, and refinement (Theorem 2).
Concurrent work by Zhou [30 ] establishes the same polynomial rate for randomized zero-respecting incremental first-order methods. Our lower bound removes the zero-respecting restriction and adds exact values and proximal responses; our upper bound starts from a joint-gradient bound instead of an exact dual maximizer and charges the resulting initialization cost. Wu et al. [23 ] prove lower bounds for arbitrary randomized algorithms in the related finite-sum nonconvex–concave (NC–C) problem over a bounded dual domain, with values and gradients only. Table 1 compares the lower bounds, and § 2.2 compares our assumptions with those of prior work.
Table 1. Bounds on component queries for finite-sum NC–SC problems with joint mean-squared smoothness, for small ε \varepsilon ε and every κ ≥ 1 \kappa\ge1 κ ≥ 1 unless stated; h = min { n κ , n 3 / 4 κ } h=\min\{\sqrt{n\kappa},n^{3/4}\sqrt{\kappa}\} h = min { nκ , n 3/4 κ } and S = L Δ / ε 2 S=L\Delta/\varepsilon^{2} S = L Δ/ ε 2 . Shaded: this paper. The first row combines Zhang et al. [27 ] , for linear-span methods, and Han et al. [7 ] , for their restricted proximal protocol with primal nonconvexity parameter L L L ; the second is the concurrent result of Zhou [30 ] , which also attains this rate up to logarithmic factors from an exact dual maximizer. A dash marks a combination not treated separately. Down the first column the algorithm class grows and the bound reaches n + h S n+hS n + h S ; the stronger oracle leaves it unchanged, and component gradients attain it up to the factor log 2 ( 2 + κ ) \log^{2}(2+\kappa) log 2 ( 2 + κ ) .
Models and Main Results
This section defines the problem class and the oracles, states our lower and upper bounds, and compares our assumptions with those of prior work.
Strongly Concave Finite Sums and Exact Indexed Oracles
Write [ n ] = { 1 , … , n } [n]=\{1,\ldots,n\} [ n ] = { 1 , … , n } , w = ( p , y ) w=(p,y) w = ( p , y ) and S = L Δ / ε 2 S=L\Delta/\varepsilon^2 S = L Δ/ ε 2 . For Y = R d y \mathcal{Y}=\mathbb{R}^{d_y} Y = R d y , the class F ( n , L , μ , Δ ) \mathcal{F}(n,L,\mu,\Delta) F ( n , L , μ , Δ ) consists of continuously differentiable finite sums satisfying, globally,
1 n ∑ i ∥ ∇ F i ( w ) − ∇ F i ( w ′ ) ∥ 2 ≤ L 2 ∥ w − w ′ ∥ 2 , (2) \frac{1}{n}\sum_i\lVert\nabla F_i(w)-\nabla F_i(w')\rVert^2\le L^2\lVert w-w'\rVert^2, \tag*{(2)} n 1 i ∑ ∥ ∇ F i ( w ) − ∇ F i ( w ′ ) ∥ 2 ≤ L 2 ∥ w − w ′ ∥ 2 , (2)
F ( p , ⋅ ) is μ -strongly concave , (3) F(p,\cdot)\text{ is }\mu\text{-strongly concave}, \tag*{(3)} F ( p , ⋅ ) is μ -strongly concave , (3)
V ( 0 ) − inf p V ( p ) ≤ Δ , (4) V(0)-\inf_p V(p)\le\Delta, \tag*{(4)} V ( 0 ) − p inf V ( p ) ≤ Δ , (4)
∥ ∇ F ( 0 , 0 ) ∥ 2 ≤ 2 L Δ . (5) \lVert\nabla F(0,0)\rVert^2\le2L\Delta. \tag*{(5)} ∥ ∇ F ( 0 , 0 ) ∥ 2 ≤ 2 L Δ. (5)
All norms are Euclidean; for matrices and multilinear maps, ∥ ⋅ ∥ \lVert\cdot\rVert ∥ ⋅ ∥ is the induced operator norm. (2) is joint mean-squared smoothness, abbreviated as root-mean-square (RMS) smoothness. It implies mean L L L -smoothness and component smoothness at most n L \sqrt{n}L n L . Strong concavity gives a unique maximizer y ∗ ( p ) y^*(p) y ∗ ( p ) and ∇ V ( p ) = ∇ p F ( p , y ∗ ( p ) ) \nabla V(p)=\nabla_pF(p,y^*(p)) ∇ V ( p ) = ∇ p F ( p , y ∗ ( p )) . We call the combination of (4) and (5) a normalized start. No bound on the initial primal–dual gap is assumed; that gap is G 0 = sup y F ( 0 , y ) − inf p F ( p , 0 ) G_0=\sup_y F(0,y)-\inf_p F(p,0) G 0 = sup y F ( 0 , y ) − inf p F ( p , 0 ) , distinct from (4).
An exact indexed first-order query returns
O ( i , w ) = ( F i ( w ) , ∇ p F i ( w ) , ∇ y F i ( w ) ) . (6) \mathcal{O}(i,w)=\left(F_i(w),\nabla_pF_i(w),\nabla_yF_i(w)\right). \tag*{(6)} O ( i , w ) = ( F i ( w ) , ∇ p F i ( w ) , ∇ y F i ( w ) ) . (6)
The component saddle-proximal oracle, a proximal point step [18 ] on one component, additionally returns a saddle pair of
H i , γ ( p , y ) = F i ( p , y ) + ∥ p − p ˉ ∥ 2 − ∥ y − y ˉ ∥ 2 2 γ (7) H_{i,\gamma}(p,y)=F_i(p,y)+\frac{\lVert p-\bar p\rVert^2-\lVert y-\bar y\rVert^2}{2\gamma} \tag*{(7)} H i , γ ( p , y ) = F i ( p , y ) + 2 γ ∥ p − p ˉ ∥ 2 − ∥ y − y ˉ ∥ 2 (7)
at the queried center ( p ˉ , y ˉ ) (\bar p,\bar y) ( p ˉ , y ˉ ) . A finite γ > 0 \gamma>0 γ > 0 is legal when this problem has a saddle and H i , γ H_{i,\gamma} H i , γ is globally convex in p p p and concave in y y y , as in (2) and Remark 12(iv) of Han et al. [7 ] . A saddle pair satisfies H i , γ ( p + , y ) ≤ H i , γ ( p + , y + ) ≤ H i , γ ( p , y + ) H_{i,\gamma}(p^+,y)\le H_{i,\gamma}(p^+,y^+)\le H_{i,\gamma}(p,y^+) H i , γ ( p + , y ) ≤ H i , γ ( p + , y + ) ≤ H i , γ ( p , y + ) for all p , y p,y p , y .
One query to either oracle costs one unit, a full gradient costs n n n , and Q Q Q denotes the total cost of a run. Algorithms may choose arbitrary adaptive indices, finite query points, and legal proximal parameters using the entire exact transcript and a private random seed. Their computation and primal output are unrestricted, and stopping may be random. Only the displayed oracle information and common valid class parameters are supplied.
Theorem 1 (Unrestricted lower bound for every condition number). *There are universal constants c ⋆ ∈ ( 0 , 1 / 2 ] c_\star\in(0,1/2] c ⋆ ∈ ( 0 , 1/2 ] and c > 0 c>0 c > 0 such that, for n ≥ 2 n\ge2 n ≥ 2 , L , μ , Δ > 0 L,\mu,\Delta>0 L , μ , Δ > 0 , κ = L / μ ≥ 1 \kappa=L/\mu\ge1 κ = L / μ ≥ 1 , and
0 < ε 2 ≤ 2 − 52 c ⋆ 2 L Δ , (8) 0<\varepsilon^2\le2^{-52}c_\star^2L\Delta, \tag*{(8)} 0 < ε 2 ≤ 2 − 52 c ⋆ 2 L Δ , (8) every algorithm using either exact indexed oracle above, terminating almost surely and guaranteeing E ∥ ∇ V ( p o u t ) ∥ ≤ ε \mathbb{E}\lVert\nabla V(p_{\mathrm{out}})\rVert\le\varepsilon E ∥ ∇ V ( p out )∥ ≤ ε uniformly on F ( n , L , μ , Δ ) \mathcal{F}(n,L,\mu,\Delta) F ( n , L , μ , Δ ) has worst-case expected cost at
least
c ( n + min { n κ , n 3 / 4 κ } ) L Δ ε 2 . (9) c\left(n+\min\left\{\sqrt{n}\kappa,n^{3/4}\sqrt{\kappa}\right\}\right)\frac{L\Delta}{\varepsilon^2}. \tag*{(9)} c ( n + min { n κ , n 3/4 κ } ) ε 2 L Δ . (9)
The same lower order holds for success probability 2/3 under a deterministic query cap. The hard instances have polynomial dimension, C 1 , 1 C^{1,1} C 1 , 1 components that are individually μ \mu μ -strongly concave in y y y , y ∗ ( 0 ) = 0 y^{\ast}(0)=0 y ∗ ( 0 ) = 0 , finite initial primal–dual gap G 0 G_0 G 0 , and a unique response at every legal proximal parameter. Their exact legal domain is common to all components, all hidden realizations and all proximal centers.
We prove Theorem 1 in Appendix D, where we obtain c = c ⋆ 2 / 2 96 c=c_{\star}^2/2^{96} c = c ⋆ 2 / 2 96 ; the next theorem gives the matching upper rate.
Theorem 2 (Upper bound for all condition numbers and matching polynomial rate). For n ≥ 2 n\ge2 n ≥ 2 , κ ≥ 1 \kappa\ge1 κ ≥ 1 and S = L Δ / ε 2 ≥ 2 S=L\Delta/\varepsilon^2\ge2 S = L Δ/ ε 2 ≥ 2 , an indexed component-gradient algorithm on F ( n , L , μ , Δ ) \mathcal{F}(n,L,\mu,\Delta) F ( n , L , μ , Δ ) terminates almost surely and satisfies
E ∥ ∇ V ( p o u t ) ∥ 2 ≤ ε 2 , (10) \mathbb{E}\lVert\nabla V(p_{\mathrm{out}})\rVert^2\le\varepsilon^2, \tag*{(10)} E ∥ ∇ V ( p out ) ∥ 2 ≤ ε 2 , (10) E Q ≤ C [ n + h ( n , κ ) ( 1 + S ) ] log 2 ( 2 + κ ) , (11) \mathbb{E}Q\le C\left[n+h(n,\kappa)(1+S)\right]\log^2(2+\kappa), \tag*{(11)} E Q ≤ C [ n + h ( n , κ ) ( 1 + S ) ] log 2 ( 2 + κ ) , (11) h ( n , κ ) = min { n κ , n 3 / 4 κ } . h(n,\kappa)=\min\left\{\sqrt{n}\kappa,n^{3/4}\sqrt{\kappa}\right\}. h ( n , κ ) = min { n κ , n 3/4 κ } . Here C > 0 C>0 C > 0 is universal, and all initialization, snapshots and final refinement calls are included. The same order holds for a deterministic query cap with success probability 2/3. Under (8), we have S ≥ 2 S\ge2 S ≥ 2 and E ∥ ∇ V ( p o u t ) ∥ ≤ ( E ∥ ∇ V ( p o u t ) ∥ 2 ) 1 / 2 ≤ ε \mathbb{E}\lVert\nabla V(p_{\mathrm{out}})\rVert\le\left(\mathbb{E}\lVert\nabla V(p_{\mathrm{out}})\rVert^2\right)^{1/2}\le\varepsilon E ∥ ∇ V ( p out )∥ ≤ ( E ∥ ∇ V ( p out ) ∥ 2 ) 1/2 ≤ ε , so this upper bound exceeds the lower bound of Theorem 1 by at most a constant multiple of log 2 ( 2 + κ ) \log^2(2+\kappa) log 2 ( 2 + κ ) .
We prove Theorem 2 in Appendix E.8. The method of Appendix E covers κ ≤ 2 18 n \kappa\le2^{18}\sqrt{n} κ ≤ 2 18 n , and that of the next theorem covers larger κ \kappa κ . We state this regime separately because in it the construction with long paths gives a larger constant c c c , the bound G 0 ≤ 1025 Δ G_0\le1025\Delta G 0 ≤ 1025Δ , the dimension estimate (16), and one hard instance for all early horizons under the protocol of Han et al. [7 ] .
Theorem 3 (Special case of large condition numbers and matching upper bound). There are universal constants c ⋆ ∈ ( 0 , 1 / 2 ] c_{\star}\in(0,1/2] c ⋆ ∈ ( 0 , 1/2 ] and c , C > 0 c,C>0 c , C > 0 such that, for n ≥ 2 n\ge2 n ≥ 2 , L , μ , Δ > 0 L,\mu,\Delta>0 L , μ , Δ > 0 , and
κ = L / μ ≥ 2 18 n , 0 < ε 2 ≤ 2 − 52 c ⋆ 2 L Δ , (12) \kappa=L/\mu\ge2^{18}\sqrt{n},\qquad0<\varepsilon^2\le2^{-52}c_{\star}^2L\Delta, \tag*{(12)} κ = L / μ ≥ 2 18 n , 0 < ε 2 ≤ 2 − 52 c ⋆ 2 L Δ , (12) every algorithm using either oracle above, terminating almost surely and guaranteeing E ∥ ∇ V ( p o u t ) ∥ ≤ ε \mathbb{E}\lVert\nabla V(p_{\mathrm{out}})\rVert\le\varepsilon E ∥ ∇ V ( p out )∥ ≤ ε uniformly on F ( n , L , μ , Δ ) \mathcal{F}(n,L,\mu,\Delta) F ( n , L , μ , Δ ) , has worst-case expected query cost at least
c n 3 / 4 κ S . (13) cn^{3/4}\sqrt{\kappa}S. \tag*{(13)} c n 3/4 κ S . (13) The fixed hard instance can be chosen with C 1 , 1 C^{1,1} C 1 , 1 components, each μ \mu μ -strongly concave in y y y , and with y ∗ ( 0 ) = 0 y^{\ast}(0)=0 y ∗ ( 0 ) = 0 and G 0 ≤ 1025 Δ G_0\le1025\Delta G 0 ≤ 1025Δ .
For every ε > 0 \varepsilon>0 ε > 0 and κ ≥ 2 18 n \kappa\ge2^{18}\sqrt{n} κ ≥ 2 18 n , without the accuracy condition in Equation (12) , our method of Appendix F, which combines existing first-order methods, achieves
E ∥ ∇ V ( p o u t ) ∥ 2 ≤ ε 2 , (14) \mathbb{E}\lVert\nabla V(p_{\mathrm{out}})\rVert^2\le\varepsilon^2, \tag*{(14)} E ∥ ∇ V ( p out ) ∥ 2 ≤ ε 2 , (14) E Q ≤ C n 3 / 4 κ ( 1 + S ) log 2 ( 2 + κ ) . (15) \mathbb{E}Q\le Cn^{3/4}\sqrt{\kappa}(1+S)\log^2(2+\kappa). \tag*{(15)} E Q ≤ C n 3/4 κ ( 1 + S ) log 2 ( 2 + κ ) . (15) The same orders hold for success probability 2/3 under a deterministic query cap.
Our proof gives c = 133 c ⋆ 2 / ( 3 ⋅ 2 73 ) c = 133c_{\star}^{2}/(3 \cdot2^{73}) c = 133 c ⋆ 2 / ( 3 ⋅ 2 73 ) , where c ⋆ c_{\star} c ⋆ is determined by the minima of continuous residuals over three fixed-dimensional compact sets (Lemma 10 ). The primal and dual dimensions d p d_p d p and d y d_y d y are fixed before execution and satisfy
d p + d y = O ( ( S n 3 / 4 κ + S 2 κ 3 n − 3 / 2 ) log ( n κ S ) ) . (16) d_p + d_y = O\left(\left(Sn^{3/4}\sqrt{\kappa} + S^2\kappa^3n^{-3/2}\right)\log(n\kappa S)\right). \tag*{(16)} d p + d y = O ( ( S n 3/4 κ + S 2 κ 3 n − 3/2 ) log ( nκ S ) ) . (16)
Corollary 4 (The Han et al. proximal protocol). In the regime of Theorem 3 , the lower order also holds for the proximal incremental first-order oracle (PIFO) algorithms of Han et al. [7 ] , with their complexity sup F inf t \sup_F \inf_t sup F inf t of Definition 13 measured by the expected gradient norm.
In Appendix C.§ 2 , one fixed hard instance serves every early output horizon, and both component calls and full-gradient refreshes are charged. The upper bound in Theorem 3 is stated in the indexed oracle model.
Comparison with Prior Assumptions
Smoothness. The joint mean-squared condition (2) is Assumption 1 of Zhou [30 ] and the smoothness condition of the finite-sum lower bounds of Zhang et al. [27 ] and Han et al. [7 ] . It is weaker than L L L -smoothness of every component, which Wu et al. [23 ] use for their lower bounds, and stronger than L L L -smoothness of the mean alone. The upper bounds need only (2). Our hard components use the extra freedom. Their individual smoothness constants grow proportionally to n L \sqrt{n}L n L .
Dual variable. As in Zhou [30 ] , the dual space is R d y \mathbb{R}^{d_y} R d y and the mean is μ \mu μ -strongly concave; our hard components are individually μ \mu μ -strongly concave. Wu et al. [23 ] instead use a known compact convex dual set of diameter at most D y D_y D y with a concave mean, and measure stationarity through the Moreau envelope of the value function.
Initialization. The normalized start (4)–(5) bounds the primal gap and the joint gradient at the origin. Zhou [30 ] assumes a primal gap and an exact dual maximizer y ⋆ ( 0 ) = 0 y^{\star}(0) = 0 y ⋆ ( 0 ) = 0 , and Wu et al. [23 ] assume only a primal gap, the bounded dual domain controlling the dual side. Neither of the first two conditions implies the other. Exact dual initialization does not bound the primal gradient of the value function, and a bounded joint gradient does not give y ⋆ ( 0 ) = 0 y^{\star}(0) = 0 y ⋆ ( 0 ) = 0 . Our hard instances satisfy both, so Theorem 1 also holds under the start of Zhou [30 ] . Every hard instance also has a finite initial primal–dual gap (Appendix § 10 .§ 4 ).
Oracle and algorithms. Theorem 1 allows arbitrary randomized adaptive algorithms with exact component values, gradients and saddle-proximal responses at legal parameters. Zhou [30 ] considers randomized zero-respecting algorithms with component gradients, and Wu et al. [23 ] consider arbitrary randomized algorithms with exact values and gradients but no proximal map. Han et al. [7 ] allow proximal steps within a restricted protocol, which Corollary 4 covers.
Accuracy and cost. Theorem 1 bounds the expected cost under E ∥ ∇ V ( p o u t ) ∥ ≤ ε \mathbb{E}\lVert\nabla V(p_{\mathrm{out}})\rVert\le\varepsilon E ∥ ∇ V ( p out )∥ ≤ ε and the deterministic cap needed for confidence 2 / 3 2/3 2/3 ; Theorem 2 guarantees E ∥ ∇ V ( p o u t ) ∥ 2 ≤ ε 2 \mathbb{E}\lVert\nabla V(p_{\mathrm{out}})\rVert^2 \le\varepsilon^2 E ∥ ∇ V ( p out ) ∥ 2 ≤ ε 2 . Zhou [30 ] characterizes expected query counts under E ∥ ∇ V ∥ 2 ≤ ε 2 \mathbb{E}\lVert\nabla V\rVert^2 \le\varepsilon^2 E ∥ ∇ V ∥ 2 ≤ ε 2 , with a fixed budget for κ ≤ n \kappa\le\sqrt{n} κ ≤ n , and that lower bound also covers the other two criteria; Wu et al. [23 ] use fixed budgets and confidence 2 / 3 2/3 2/3 . The remaining gap is log 2 ( 2 + κ ) \log^2(2+\kappa) log 2 ( 2 + κ ) here and logarithmic for Zhou [30 ] ; for Wu et al. [23 ] there is no gap when L D y / ε ≤ n 1 / 4 LD_y/\varepsilon\le n^{1/4} L D y / ε ≤ n 1/4 and one logarithm otherwise.
Technical Overview
We couple a chain of nonconvex primal stages through dual paths whose internal edges are owned by random components; the symbols of this section are local to the lower bounds.
Dual Paths with Component Assignments
A dual path connects two primal stages while preserving their nonlinear amplitude. For large condition numbers we use Q = P m + m − 2 I Q = P_m + m^{-2}I Q = P m + m − 2 I , where P m P_m P m is the Laplacian of the path graph on m m m vertices. Lemma 5 gives entries of the Green matrix u = ( Q − 1 ) 11 ≍ m u = (Q^{-1})_{11} \asymp m u = ( Q − 1 ) 11 ≍ m , v = ( Q − 1 ) 1 m ≍ m v = (Q^{-1})_{1m} \asymp m v = ( Q − 1 ) 1 m ≍ m , and 1 < u / v ≤ cosh 1 1 < u/v \le\cosh1 1 < u / v ≤ cosh 1 . We take a = L / ( 2 16 n ) a = L/(2^{16}\sqrt{n}) a = L / ( 2 16 n ) , λ = 2 − 39 L v / u \lambda= 2^{-39}Lv/u λ = 2 − 39 Lv / u , and m = ⌊ κ / ( 2 16 n ) ⌋ m = \lfloor\sqrt{\kappa}/(2^{16}\sqrt{n}) \rfloor m = ⌊ κ / ( 2 16 n )⌋ . Each node has an independent hidden axis; public identity maps pair primal and dual endpoint blocks, and independent uniform component owners label the internal edges.
The primal variable P = ( P 1 , … , P M ) P = (P_1,\ldots,P_M) P = ( P 1 , … , P M ) has one block P j = ( X j , Z j + 1 ) P_j = (X_j,Z_{j+1}) P j = ( X j , Z j + 1 ) per stage, whose two subblocks are publicly paired with the first and last nodes of path j j j , and η ≥ 0 \eta\ge0 η ≥ 0 is the width of the flat zones of § 3.2 . Each component has the form
f i , η ( P , y ) = P ( P ) − ν 2 ∥ y ∥ 2 + b χ ∗ ( Y ∂ ) ⟨ A ( P ) , Y ∂ ⟩ + c ∑ v χ ∗ ( y v ) φ η ( t v ) − n a ∑ e : Σ e = i A η ( Y e ) + (public radial terms) ; f_{i,\eta}(P,y)=\mathcal{P}(P)-\frac{\nu}{2}\lVert y\rVert^2+b\chi_*(Y_{\partial})\langle A(P),Y_{\partial}\rangle+c\sum_v\chi_*(y_v)\varphi_\eta(t_v)
-na\sum_{e:\Sigma_e=i}A_\eta(Y_e)+\text{(public radial terms)}; f i , η ( P , y ) = P ( P ) − 2 ν ∥ y ∥ 2 + b χ ∗ ( Y ∂ ) ⟨ A ( P ) , Y ∂ ⟩ + c v ∑ χ ∗ ( y v ) φ η ( t v ) − na e : Σ e = i ∑ A η ( Y e ) + (public radial terms) ;
see Equation (44) . The primal part P \mathcal{P} P collects a scaled source chain built from the scalar functions of Carmon et al. [4 ] , the compensation − d ∥ A ∥ 2 -d\lVert A\rVert^2 − d ∥ A ∥ 2 and a radial penalty. The forcing A ( P ) A(P) A ( P ) stacks the gated, capped primal blocks and acts on the concatenated path endpoints Y ∂ Y_{\partial} Y ∂ . Each dual node v v v has a hidden coordinate t v t_v t v along its axis, φ η ( t ) = 1 2 ( ∣ t ∣ − η ) + 2 \varphi_\eta(t)=\frac{1}{2}(\lvert t\rvert-\eta)_+^2 φ η ( t ) = 2 1 (∣ t ∣ − η ) + 2 , and χ ∗ \chi_* χ ∗ is a cutoff. The owned term A η ( Y e ) = 1 2 ( ∣ t e ∣ − h η ( ∥ Y e ∥ ) ) + 2 A_\eta(Y_e)=\frac{1}{2}(\lvert t_e\rvert-h_\eta(\lVert Y_e\rVert))_+^2 A η ( Y e ) = 2 1 (∣ t e ∣ − h η (∥ Y e ∥) ) + 2 acts on the difference t e t_e t e of the two hidden coordinates of an edge e e e with endpoint pair Y e Y_e Y e , and only the owner Σ e \Sigma_e Σ e of e e e contains it; a , b , c , d , ν > 0 a,b,c,d,\nu>0 a , b , c , d , ν > 0 are constants.
With ridge ν = a / u \nu=a/u ν = a / u , coupling b 2 = 12 a λ / v b^2=12a\lambda/v b 2 = 12 aλ / v , and compensation d = 6 λ ( u / v − 1 ) d=6\lambda(u/v-1) d = 6 λ ( u / v − 1 ) , dual elimination gives, for each stage j j j with gate weight h j 2 h_j^2 h j 2 , the completed quadratic
6 λ ( x j − z j + 1 / 2 ) 2 + 6 λ ∥ A j , ⊥ ∥ 2 + 3 2 λ ∥ B j + 1 , ⊥ ∥ 2 (17) 6\lambda(x_j-z_{j+1}/2)^2+6\lambda\lVert A_{j,\perp}\rVert^2+\frac{3}{2}\lambda\lVert B_{j+1,\perp}\rVert^2 \tag*{(17)} 6 λ ( x j − z j + 1 /2 ) 2 + 6 λ ∥ A j , ⊥ ∥ 2 + 2 3 λ ∥ B j + 1 , ⊥ ∥ 2 (17)
where x j x_j x j and z j + 1 z_{j+1} z j + 1 are the coordinates of the capped subblocks of P j P_j P j along the first and last hidden axes of path j j j , and A j , ⊥ A_{j,\perp} A j , ⊥ and B j + 1 , ⊥ B_{j+1,\perp} B j + 1 , ⊥ are their orthogonal parts. Thus dual elimination preserves the same normalized primal quadratic, including in the orthogonal directions available to unrestricted outputs. The path and source ingredients follow Li et al. [8 ] and Carmon et al. [4 ] .
Global Geometry and Exact Hiding
Against zero-respecting algorithms [30 ] , every query lies in the span of the coordinates that earlier gradients have touched, so the unexposed coordinates are exactly zero at every query. Arbitrary queries and outputs may instead have nonzero projections onto unexposed hidden axes and arbitrary norms, and an exact proximal response is a global saddle point; the following devices keep the hidden structure invisible in that case. Smooth radial maps (cf. Carmon et al. [4 ] , Section 5.2) compress the primal blocks to bounded images and hence bound the forcing. Dual cutoffs with bounded support control the primal Hessian of the coupling. The region where an owned edge interaction vanishes (a flat edge, cf. Woodworth and Srebro [21 ] , Appendix C) has a width depending on the radius r = ∥ Y e ∥ r = \lVert Y_e\rVert r = ∥ Y e ∥ of the edge pair and a localization radius R ∗ R_\ast R ∗ ,
h η ( r ) = η ϱ ( r / R ∗ ) , (18) h_\eta(r) = \eta\varrho(r/R_\ast), \tag*{(18)} h η ( r ) = η ϱ ( r / R ∗ ) , (18)
ϱ ( u ) = { 1 , u ≤ 1 , 1 + ( u − 1 ) 2 / 2 , 1 < u < 2 , u − 1 / 2 , u ≥ 2. (19) \varrho(u) =
\begin{cases}
1, & u \le1, \\
1 + (u - 1)^2/2, & 1 < u < 2, \\
u - 1/2, & u \ge2.
\end{cases} \tag*{(19)} ϱ ( u ) = ⎩ ⎨ ⎧ 1 , 1 + ( u − 1 ) 2 /2 , u − 1/2 , u ≤ 1 , 1 < u < 2 , u ≥ 2. (19)
Since ϱ ( u ) ≥ u / 2 \varrho(u) \ge u/2 ϱ ( u ) ≥ u /2 , flatness holds on a cone at every radius. Common public radial penalties dominate the positive cutoff curvature without revealing owners. The uncut optimizer lies inside 3 R ∗ / 16 3R_\ast/16 3 R ∗ /16 , where these corrections vanish and the cutoffs equal one; strong concavity identifies it with the actual optimizer. With the primal length scale s = 8 ε / ( c ⋆ λ ) s = 8\varepsilon/(c_\star\lambda) s = 8 ε / ( c ⋆ λ ) , the zero-width barrier on the last stage coordinate z M + 1 z_{M+1} z M + 1 survives this perturbation:
∣ z M + 1 ∣ ≤ s / 8 ⟹ ∥ ∇ V ( P ) ∥ ≥ 6 ε . \lvert z_{M+1}\rvert\le s/8 \quad\Longrightarrow\quad\lVert\nabla V(P)\rVert\ge6\varepsilon. ∣ z M + 1 ∣ ≤ s /8 ⟹ ∥ ∇ V ( P )∥ ≥ 6 ε .
Exact Oracle Responses and Component Search
We add one public primal coordinate ξ \xi ξ with the common term ( L / 8 ) ( 1 − cos ξ ) (L/8)(1 - \cos\xi) ( L /8 ) ( 1 − cos ξ ) . It preserves all initial budgets and gradient barriers, while making the exact legal domain τ ≥ L / 8 \tau\ge L/8 τ ≥ L /8 for the inverse step τ = 1 / γ \tau= 1/\gamma τ = 1/ γ . In the regularized problem the variables ( P , y ) (P,y) ( P , y ) are strongly convex–concave; the separate scalar problem is strictly convex even at the boundary.
A legal call can reveal all consecutive edges with its component as owner (Figure 1 ). After that run, we set the unrevealed hidden axes to zero in every term and keep all public terms; this censored problem on the full space has a unique saddle. We reflect unrevealed blocks through the spans of their current query centers, matching the primal and dual endpoint reflections. Invariance and uniqueness place the saddle in these spans, each of dimension at most two: one for an interior node, and two at an endpoint, whose span also contains the paired primal center. This step uses no property of the hidden axes; their random directions enter only in the next step. The hidden axes have small projections onto these spans, so the actual and censored problems have the same values and full gradients there, and the censored saddle is the actual response. A separate projection argument handles the arbitrary final output.
Figure 1. A dual path. A query to component i i i reveals the entire run of consecutive edges owned by i i i and stops at the first edge with another owner, revealing no later owner or axis. There are order M m Mm M m serial searches, each costing order n n n .
There are E = M ( m − 1 ) E = M(m - 1) E = M ( m − 1 ) serial owner searches. Given everything revealed so far, the owner of the first unrevealed edge is uniform over the components not yet excluded. A nonnegative posterior potential starts at E ( n − 1 ) / 2 E(n-1)/2 E ( n − 1 ) /2 and decreases in conditional expectation by at most one per call, including consecutive hits (Lemma 14 ). Its completion bound and the terminal barrier give order n M m nMm n M m expected queries under random stopping. Since this bound holds on average over the random owners and axes, some fixed choice of component functions inherits it, as in Yao’s minimax principle [25 ] .
The Remaining Condition Numbers
For 64 ≤ κ ≤ 2 18 n 64 \le\kappa\le2^{18}\sqrt{n} 64 ≤ κ ≤ 2 18 n , a two-node path with r = μ / a r = \mu/a r = μ / a uses the same calibration λ = 2 − 39 L / ν = 2 − 39 L / ( 1 + r ) \lambda= 2^{-39}L/\nu= 2^{-39}L/(1+r) λ = 2 − 39 L / ν = 2 − 39 L / ( 1 + r ) and preserves (17); its cost n M nM n M of owner search has order n κ S \sqrt{n}\kappa S n κ S . We use paths only for κ ≥ 64 \kappa\ge64 κ ≥ 64 : their dual ridge and cutoff barriers add about 47 ν 47\nu 47 ν to our smoothness bound (Lemma 9), and ν ≥ μ = L / κ \nu\ge\mu= L/\kappa ν ≥ μ = L / κ . For 1 ≤ κ < 64 1 \le\kappa< 64 1 ≤ κ < 64 , a separable strongly concave dual instead leaves a primal bridge C η ( t 1 , t 2 ) = ϕ η ( t 1 + t 2 ) − ϕ η ( t 1 ) − ϕ η ( t 2 ) C_\eta(t_1,t_2) = \phi_\eta(t_1+t_2)-\phi_\eta(t_1)-\phi_\eta(t_2) C η ( t 1 , t 2 ) = ϕ η ( t 1 + t 2 ) − ϕ η ( t 1 ) − ϕ η ( t 2 ) , where ϕ η ( t ) = 1 2 ( ∣ t ∣ − η ) + 2 \phi_\eta(t) = \frac{1}{2}(|t|-\eta)_+^2 ϕ η ( t ) = 2 1 ( ∣ t ∣ − η ) + 2 . This equals t 1 t 2 t_1t_2 t 1 t 2 at zero width and is flat near the hidden origin. Independent owners cost only n \sqrt{n} n in RMS smoothness, but a single component then has primal curvature of order n L \sqrt{n}L n L . Each component i i i therefore has its own public coordinate ξ i \xi_i ξ i with the term B ( 1 − cos ξ i ) B(1-\cos\xi_i) B ( 1 − cos ξ i ) , B = n L / 4 B = \sqrt{n}L/4 B = n L /4 ; these orthogonal coordinates cost only B / n = L / 4 B/\sqrt{n} = L/4 B / n = L /4 in RMS smoothness and give every component the exact legal domain τ ≥ B \tau\ge B τ ≥ B . A separate instance, in which one hidden component carries a random sign, gives the n n n term (Lemma 21).
Table 2 lists, for each main statement, the results its proof uses in order.
Table 2. Where each main statement is proved, with the results its proof uses in order. In the first row, Lemmas 5 and 6 calibrate the paths and the forcing, Lemmas 7–9 give class membership, Lemma 10 the terminal barrier, Lemma 11 the width and initial budgets, and Lemmas 12–14 the legal domain, localization and owner search. The rows below the heavy rule concern upper bounds, which use only component gradients; the accelerated method also uses Catalyst [9 ] and Zhang et al. (2021, Theorems 4.1 and 4.2).
Matching Upper Bound and Initialization Cost
We match the two branches of h ( n , κ ) h(n,\kappa) h ( n , κ ) using component gradients under mean strong concavity: a single-loop method for κ ≤ 2 18 n \kappa\le2^{18}\sqrt{n} κ ≤ 2 18 n and an accelerated method above. Appendix § 11 combines the stochastic projected damped extragradient (SPDE) updates of Zhang et al. [26 ] with recursive finite-sum gradient estimates (Nguyen et al. [16 ] ; Fang et al. [6 ] ) and loopless stochastic variance-reduced extragradient (L-SVRE) steps (Alacaoglu and Malitsky [1 ] ; Luo et al. [12 ] ), which need strong monotonicity only of the mean operator (Algorithm 1). Its energy combines the gap of the Moreau envelope M ( z ) = min p { V ( p ) + L ∥ p − z ∥ 2 } \mathcal{M}(z) = \min_p\{V(p) + L\lVert p-z\rVert^2\} M ( z ) = min p { V ( p ) + L ∥ p − z ∥ 2 } at a moving primal center z z z with the distance to the saddle point of F ( p , y ) + L ∥ p − z ∥ 2 F(p,y) + L\lVert p-z\rVert^2 F ( p , y ) + L ∥ p − z ∥ 2 . We prove the L-SVRE contraction directly and use it to initialize this energy. The same solver then refines a sampled endpoint, whose distance to its proximal saddle is already controlled by the SPDE dissipation. A pathwise error bound charges every full snapshot and both gradients in each recursive difference, giving the bound O ( n + n κ S + ( n + n κ ) log ( 2 κ ) ) O(n+\sqrt{n}\kappa S+(n+\sqrt{n}\kappa)\log(2\kappa)) O ( n + n κ S + ( n + n κ ) log ( 2 κ )) of Theorem 22 in Appendix § 11 , with no logarithm on the accuracy term. Zhang et al. [26 ] analyze SPDE on compact domains; we apply their one-step inequalities on the whole space, where each step stays in balls on which the projections are inactive (Lemma 23 ).
For large κ \kappa κ , acceleration gives the other branch (Appendix F). We normalize f i = F i / L f_i = F_i/L f i = F i / L , ν = μ / L \nu= \mu/L ν = μ / L , D 0 = Δ / L D_0 = \Delta/L D 0 = Δ/ L , and e 0 = ε / L e_0 = \varepsilon/L e 0 = ε / L , and set
s = n − 1 / 2 , τ = s = ν , q = ν / s , ρ = 0.9 q . (20) s = n^{-1/2}, \qquad\tau= s = \nu, \qquad q = \nu/s, \qquad\rho= 0.9\sqrt{q}. \tag*{(20)} s = n − 1/2 , τ = s = ν , q = ν / s , ρ = 0.9 q . (20)
Catalyst adds the dual regularizer τ 2 ∥ y − z k ∥ 2 \frac{\tau}{2}\lVert y-z_k\rVert^2 2 τ ∥ y − z k ∥ 2 at dual centers z k z_k z k , so each subproblem is s s s -strongly concave; q q q is its inverse condition number, and the accuracy targets decrease like ( 1 − ρ ) k (1-\rho)^k ( 1 − ρ ) k . The regime of Theorem 3 ensures τ > 0 \tau>0 τ > 0 and q ≤ 1 / 64 q\le1/64 q ≤ 1/64 . The same L-SVRE solver with an exact residual stopping test, combined with Catalyst for strongly convex problems [9 ] , computes a dual warm start satisfying
∥ y w − y ∗ ( 0 ) ∥ 2 ≤ D 0 (21) \lVert y_{\mathrm{w}}-y^*(0)\rVert^2 \le D_0 \tag*{(21)} ∥ y w − y ∗ ( 0 ) ∥ 2 ≤ D 0 (21)
on every terminating run, at expected cost O ( n q − 1 / 2 log 2 ( 2 + κ ) ) O(nq^{-1/2}\log^2(2+\kappa)) O ( n q − 1/2 log 2 ( 2 + κ )) .
At an outer point w t w_t w t , let f ^ t ( p , y ) = f ( p , y ) + ∥ p − p t ∥ 2 \widehat{f}_t(p,y)=f(p,y)+\lVert p-p_t\rVert^2 f t ( p , y ) = f ( p , y ) + ∥ p − p t ∥ 2 , let G t G_t G t be its unknown saddle gap, and let R t 2 = ∥ ∇ f ^ t ( w t ) ∥ 2 R_t^2=\lVert\nabla\widehat{f}_t(w_t)\rVert^2 R t 2 = ∥ ∇ f t ( w t ) ∥ 2 . If R t = 0 R_t=0 R t = 0 , the point is stationary. Otherwise
B t : = ν R t 2 72 ≤ G t , G t B t ≤ 36 ν 2 . (22) B_t := \frac{\nu R_t^2}{72} \le G_t,\qquad\frac{G_t}{B_t}\le\frac{36}{\nu^2}. \tag*{(22)} B t := 72 ν R t 2 ≤ G t , B t G t ≤ ν 2 36 . (22)
Replacing the unavailable gap by B t B_t B t in the acceleration tolerances preserves the bounds of Zhang et al. (2021, Theorems 4.1–4.3) with a logarithmic cost change. The residual estimate also controls each auxiliary warm start directly. Each of O ( 1 + S ) O(1+S) O ( 1 + S ) outer solves costs O ( n q − 1 / 2 log 2 ( 2 + κ ) ) O(nq^{-1/2}\log^2(2+\kappa)) O ( n q − 1/2 log 2 ( 2 + κ )) , and n q − 1 / 2 = n 3 / 4 κ nq^{-1/2}=n^{3/4}\sqrt{\kappa} n q − 1/2 = n 3/4 κ . Appendix F includes the cold starts, exact residual checks, and confidence conversion. Appendix E.8 combines the two regimes.
Related Work
For finite-sum monotone inclusions under mean-squared Lipschitz continuity, Zhou [29 ] gives an O ( n + n L R / ε ) O(n+\sqrt{n}LR/\varepsilon) O ( n + n L R / ε ) method and a matching Ω ( n + n L R / ε ) \Omega(n+\sqrt{n}LR/\varepsilon) Ω ( n + n L R / ε ) lower bound, with R R R an initial distance to a solution. The lower bound concerns randomized linear-span algorithms with component operator evaluations, and the resolvent there belongs to the common operator, not to components.
The lower bounds of Wu et al. [23 ] for finite-sum nonconvex–concave problems differ from ours in the dual geometry and the stationarity measure. As their Section 4 notes, substituting the effective condition number of a regularized dual into the NC–SC rate is only a formal comparison, and converting their Moreau envelope guarantee into a bound on the gradient of the value function can lose a factor of order κ \kappa κ .
The path and nonlinear source have established precedents. Li et al. [8 ] use dual paths and endpoint compensation with the source functions of Carmon et al. [4 ] , and Zhou [30 ] assigns internal dual edges to random component owners. Han et al. [7 ] develop component grouping and convex finite-sum path bounds. Our construction preserves the primal and dual scales while satisfying global smoothness and hiding the component assignments from arbitrary exact proximal queries.
Exact hiding also has distinct antecedents. Woodworth and Srebro [21 ] use flat edge penalties for proximal information. Carmon et al. [4 ] bound the effect of unrestricted query norms by radial compaction, and Bubeck et al. [3 ] use a convex wall outside correlation cones. Arjevani et al. [2 ] condition on preceding projected queries in randomized information arguments. Peng et al. [17 ] develop dense weak hiding for individually smooth nonconvex and PL finite sums with exact values and gradients. Our construction combines blockwise exact transcript coupling with radius-dependent flat edges and componentwise strong concavity, and it controls the entire saddle-proximal response. Wu et al. [22 ] treat the related NC–C setting with a bounded dual domain, with a deterministic zero-respecting class and a different stationarity measure.
Stochastic NC–SC problems with fresh samples form a separate line. Wu et al. [24 ] obtain matching upper and lower bounds, up to logarithmic factors, when every query draws a fresh stochastic gradient, and Zhang et al. [28 ] prove lower bounds of order κ L Δ σ 2 ε − 4 \kappa L\Delta\sigma^{2}\varepsilon^{-4} κ L Δ σ 2 ε − 4 under bounded variance and κ 3 / 2 L Δ σ ε − 3 \kappa^{3/2}L\Delta\sigma\varepsilon^{-3} κ 3/2 L Δ σ ε − 3 under averaged smoothness for zero-respecting methods. For zero-respecting methods with an unbiased stochastic oracle on instances with a bounded dual box, Zhou [31 ] proves the lower bound Ω ( κ L G σ 2 ε − 4 ) \Omega(\kappa LG\sigma^{2}\varepsilon^{-4}) Ω ( κ L G σ 2 ε − 4 ) for σ ≥ ε \sigma\ge\varepsilon σ ≥ ε under Moreau-envelope stationarity, where G G G bounds the initial primal–dual gap, and Ω ( L Δ ( κ ε − 2 + κ σ 2 ε − 4 ) ) \Omega(L\Delta(\sqrt{\kappa}\varepsilon^{-2}+\kappa\sigma^{2}\varepsilon^{-4})) Ω ( L Δ ( κ ε − 2 + κ σ 2 ε − 4 )) for the gradient of the primal function. Song and Xu [19 ] obtain lower bounds with the same stochastic terms for zero-respecting methods that may use variance reduction, and their Theorem 4.8 treats the stochastic NC–C case with a bounded dual domain. In these models a sample is used only within its own round, whereas every finite-sum component can be queried repeatedly; neither model includes proximal access.
Discussion
Exact proximal access and unrestricted adaptivity do not change the polynomial complexity of the problem for any condition number. An exact component proximal step solves a saddle problem in each call, yet on this class it saves at most a factor O ( log 2 ( 2 + κ ) ) O(\log^{2}(2+\kappa)) O ( log 2 ( 2 + κ )) in worst-case component queries over component gradients, for ε \varepsilon ε in the range of Theorem 1 . Two parts of our upper bounds apply to other finite-sum minimax methods: the shared L-SVRE solver needs strong concavity only of the mean, not of each component, and the residual estimate B t B_t B t of § 4 makes the subproblem tolerances of the accelerated scheme of Zhang et al. [27 ] computable.
The results are worst-case oracle bounds in a dimension that grows with the parameters. As in the lower bound of Zhou [30 ] , the lower theorem requires ε 2 / ( L Δ ) \varepsilon^{2}/(L\Delta) ε 2 / ( L Δ ) to be below a universal constant; we have not optimized this constant or the other universal constants. For κ ≤ 2 18 n \kappa\le2^{18}\sqrt{n} κ ≤ 2 18 n and S ≥ log ( 2 κ ) S \ge\log(2\kappa) S ≥ log ( 2 κ ) , Theorem 22 gives O ( n log ( 2 κ ) + h ( n , κ ) S ) O(n\log(2\kappa)+h(n,\kappa)S) O ( n log ( 2 κ ) + h ( n , κ ) S ) , so the bounds match in the accuracy term and differ by at most log ( 2 κ ) \log(2\kappa) log ( 2 κ ) in the term n n n ; for larger κ \kappa κ the accelerated method of Appendix F leaves a factor log 2 ( 2 + κ ) \log^{2}(2+\kappa) log 2 ( 2 + κ ) . Individually L L L -smooth components form a stricter class than joint mean-squared smoothness; for zero-respecting methods, Zhou [30 ] , Theorem 4 proves Ω ( n + min { κ , n κ } L Δ / ε 2 ) \Omega(n+\min\{\kappa,\sqrt{n\kappa}\}L\Delta/\varepsilon^{2}) Ω ( n + min { κ , nκ } L Δ/ ε 2 ) on a bilinear subclass, and the unrestricted complexity of this class remains open.
For nonconvex–concave finite sums over bounded dual domains, Wu et al. [23 ] determine the complexity with exact values and gradients up to one logarithm; whether exact component proximal access changes that complexity remains open.
Formal verification. The Lean supplement, provided as supplementary material and as ancillary files of the arXiv version, verifies the existential lower assertions of Theorem 1 : the expected query cost and the fixed-confidence complexity under a deterministic cap, in all three ranges of the condition number and for the indexed baseline. The formal proof includes the path calibration of Lemma 5 , the posterior owner cost of Lemma 14 , and the public cosine terms in (73) and (90) with their exact legal boundary. It uses a finite Rademacher construction in place of the Haar-distributed axes, and obtains a barrier constant in the role of that of Lemma 10 by the same compactness argument. Its CLAIMS and TRUSTED DEFINITIONS files state the formal theorems, the dimension bounds and the remaining differences from the written proof. The Haar conditioning argument, zero-axis extension, the identity of the two barrier constants, Corollary 4 , upper bounds and dimension estimate (16) are not formalized.
The Hard Family and Its Global Geometry
We use one path family for every κ ≥ 64 \kappa\ge64 κ ≥ 64 : long paths prove Theorem 3 , and two-node paths cover the intermediate range in Appendix § 10 . The scalar source functions are those of Carmon et al. [4 ] , and the dual paths with endpoint compensation follow Li et al. [8 ] . Symbols defined in Appendices A–§ 10 are local to the lower-bound proofs.
Let P m P_m P m be the Laplacian of the m m m -vertex path, with diagonal ( 1 , 2 , … , 2 , 1 ) (1,2,\ldots,2,1) ( 1 , 2 , … , 2 , 1 ) . Fix
a = L 2 16 n , t = a / μ , m = max { 2 , ⌊ t ⌋ } , r = { μ / a , t < 2 , m − 2 , t ≥ 2 , μ 0 = a r , Q = P m + r I . (23) a = \frac{L}{2^{16}\sqrt{n}}, \qquad t = \sqrt{a/\mu}, \qquad m = \max\{2,\lfloor t\rfloor\},
\qquad
r =
\begin{cases}
\mu/a, & t < 2,\\
m^{-2}, & t \ge2,
\end{cases}
\qquad
\mu_0 = ar, \qquad Q = P_m + rI. \tag*{(23)} a = 2 16 n L , t = a / μ , m = max { 2 , ⌊ t ⌋} , r = { μ / a , m − 2 , t < 2 , t ≥ 2 , μ 0 = a r , Q = P m + r I . (23)
Write u = ( Q − 1 ) 11 u=(Q^{-1})_{11} u = ( Q − 1 ) 11 , v = ( Q − 1 ) 1 m v=(Q^{-1})_{1m} v = ( Q − 1 ) 1 m and g = u / v g=u/v g = u / v , and put
δ 0 = 2 − 39 , λ = δ 0 L g , ν = a u , c = ν − μ 0 , b 2 = 12 a λ v , d = 6 λ ( g − 1 ) . (24) \delta_0 = 2^{-39}, \qquad\lambda= \frac{\delta_0 L}{g}, \qquad\nu= \frac{a}{u}, \qquad c = \nu-\mu_0, \qquad b^2 = \frac{12a\lambda}{v}, \qquad d = 6\lambda(g-1). \tag*{(24)} δ 0 = 2 − 39 , λ = g δ 0 L , ν = u a , c = ν − μ 0 , b 2 = v 12 aλ , d = 6 λ ( g − 1 ) . (24)
Instantiate the path family only when
X : = c ⋆ 2 λ Δ 2048 ε 2 ≥ 2 , X := \frac{c_\star^2\lambda\Delta}{2048\varepsilon^2} \ge2, X := 2048 ε 2 c ⋆ 2 λ Δ ≥ 2 ,
equivalently ε 2 ≤ c ⋆ 2 λ Δ / 4096 \varepsilon^2 \le c_\star^2\lambda\Delta/4096 ε 2 ≤ c ⋆ 2 λ Δ/4096 .
{#eq:25}
Outside this window the indexed baseline will suffice. Within it set
s = 8 ε c ⋆ λ , M = ⌊ X ⌋ , K = M m , E = M ( m − 1 ) . (25) s = \frac{8\varepsilon}{c_\star\lambda}, \qquad M=\lfloor X\rfloor, \qquad K=Mm, \qquad E=M(m-1). \tag*{(25)} s = c ⋆ λ 8 ε , M = ⌊ X ⌋ , K = M m , E = M ( m − 1 ) . (25)
Thus m , M ≥ 2 m,M\ge2 m , M ≥ 2 and
M ≥ c ⋆ 2 λ Δ 4096 ε 2 , M λ s 2 ≤ Δ / 32 , λ s 2 ≤ Δ / 64. (26) M \ge\frac{c_\star^2\lambda\Delta}{4096\varepsilon^2}, \qquad M\lambda s^2 \le\Delta/32, \qquad\lambda s^2 \le\Delta/64. \tag*{(26)} M ≥ 4096 ε 2 c ⋆ 2 λ Δ , M λ s 2 ≤ Δ/32 , λ s 2 ≤ Δ/64. (26)
The width and hiding dimension are fixed by (65) and (79) . With A 1 = 2433 A_1=2433 A 1 = 2433 , the bound on ∥ D A ∥ \lVert D_A\rVert ∥ D A ∥ in Lemma 6 , and C 0 = 294912 A 1 / c ⋆ C_0=294912A_1/c_\star C 0 = 294912 A 1 / c ⋆ , their ratio is
R ⋆ η = max { C 0 g r K , 16 n } , R ⋆ η 0 = C 0 g r K > 128. (27) \frac{R_\star}{\eta} = \max\left\{C_0\frac{g}{r}\sqrt{K},16\sqrt{n}\right\}, \qquad\frac{R_\star}{\eta_0}=C_0\frac{g}{r}\sqrt{K}>128. \tag*{(27)} η R ⋆ = max { C 0 r g K , 16 n } , η 0 R ⋆ = C 0 r g K > 128. (27)
In particular H n = max { C 0 2 ( g / r ) 2 K , 256 n } H_n=\max\{C_0^2(g/r)^2K,256n\} H n = max { C 0 2 ( g / r ) 2 K , 256 n } , and the dimension is fixed before execution, independently of the stopping time.
Path Geometry, Hidden Axes, and the Scalar Components
There are M M M publicly identified dual paths. Every node y j , k ∈ R D y_{j,k}\in\mathbb{R}^D y j , k ∈ R D has an independent Haar-uniform hidden unit axis U j , k U_{j,k} U j , k . The primal coordinate block P j = ( X j , Z j + 1 ) ∈ R D × R D P_j=(X_j,Z_{j+1})\in\mathbb{R}^D\times\mathbb{R}^D P j = ( X j , Z j + 1 ) ∈ R D × R D is publicly identified with the two endpoint coordinate blocks ( j , 1 ) (j,1) ( j , 1 ) and ( j , m ) (j,m) ( j , m ) , respectively. Thus its source and outgoing connector axes are U j , 1 U_{j,1} U j , 1 and U j , m U_{j,m} U j , m . Interior axes are independent of all endpoint axes. For each edge ( j , k ) (j,k) ( j , k ) , k < m k<m k < m , draw a fixed independent owner Σ j , k \Sigma_{j,k} Σ j , k uniformly from [ n ] [n] [ n ] . These draws and the axes are independent of the algorithm’s private seed; they will be fixed before invoking its oracle.
Lemma 5 (Common path bounds and branch calibration). In both branches,
μ 0 ≥ μ , 0 ≤ c ≤ a , ν ≤ μ + a , 0 < v ≤ u , b 2 v = 12 δ 0 L , 0 ≤ d = 6 δ 0 L ( 1 − 1 / g ) ≤ 6 δ 0 L . (28) \mu_0 \ge\mu,\quad0 \le c \le a,\quad\nu\le\mu+a,\quad0 < v \le u,
\qquad\frac{b^2}{v}=12\delta_0L,\quad0 \le d=6\delta_0L(1-1/g)\le6\delta_0L. \tag*{(28)} μ 0 ≥ μ , 0 ≤ c ≤ a , ν ≤ μ + a , 0 < v ≤ u , v b 2 = 12 δ 0 L , 0 ≤ d = 6 δ 0 L ( 1 − 1/ g ) ≤ 6 δ 0 L . (28) Every entry in either endpoint inverse column is at most u u u . For t ≥ 2 t \ge2 t ≥ 2 , that is, κ ≥ 2 18 n \kappa\ge2^{18}\sqrt{n} κ ≥ 2 18 n ,
m ≤ u ≤ m 2 , m sinh 1 ≤ v ≤ m , 1 < g ≤ cosh 1 < 5 / 3. (29) m \le u \le m^2,\qquad\frac{m}{\sinh1}\le v\le m,\qquad1<g\le\cosh1<5/3. \tag*{(29)} m ≤ u ≤ m 2 , sinh 1 m ≤ v ≤ m , 1 < g ≤ cosh 1 < 5/3. (29) Then μ ≤ μ 0 ≤ 4 μ \mu\le\mu_0\le4\mu μ ≤ μ 0 ≤ 4 μ , ν , c ≤ a / 2 \nu,c\le a/2 ν , c ≤ a /2 , d < 4 λ d<4\lambda d < 4 λ and λ ≥ 2 − 40 L \lambda\ge2^{-40}L λ ≥ 2 − 40 L . Hence the window (12) of Theorem 3 implies (25) and
m ≥ κ 512 n 1 / 4 , M ≥ c ∗ 2 S 2 52 . (30) m\ge\frac{\sqrt{\kappa}}{512n^{1/4}},\qquad M\ge\frac{c_*^2S}{2^{52}}. \tag*{(30)} m ≥ 512 n 1/4 κ , M ≥ 2 52 c ∗ 2 S . (30) For t < 2 t<2 t < 2 , m = 2 m=2 m = 2 , r = μ / a > 1 / 4 r=\mu/a>1/4 r = μ / a > 1/4 , μ 0 = μ \mu_0=\mu μ 0 = μ , and
Q − 1 = 1 r ( r + 2 ) ( r + 1 1 1 r + 1 ) , g = 1 + r , ν = μ + a r r + 1 , c = a r r + 1 . (31) Q^{-1}=\frac{1}{r(r+2)}
\begin{pmatrix}
r+1 & 1\\
1 & r+1
\end{pmatrix},\qquad
g=1+r,\qquad
\nu=\mu+\frac{ar}{r+1},\qquad
c=\frac{ar}{r+1}. \tag*{(31)} Q − 1 = r ( r + 2 ) 1 ( r + 1 1 1 r + 1 ) , g = 1 + r , ν = μ + r + 1 a r , c = r + 1 a r . (31) The two calibrations agree at t = 2 t=2 t = 2 .
Proof. The two-node formula is direct. For the long path and any mass r > 0 r>0 r > 0 , put θ = arcosh ( 1 + r / 2 ) \theta=\operatorname{arcosh}(1+r/2) θ = arcosh ( 1 + r /2 ) . The leading-minor recurrence d j = ( 2 + r ) d j − 1 − d j − 2 d_j=(2+r)d_{j-1}-d_{j-2} d j = ( 2 + r ) d j − 1 − d j − 2 , with first endpoint entry 1 + r 1+r 1 + r , gives d j = cosh ( ( j + 1 / 2 ) θ ) / cosh ( θ / 2 ) d_j=\cosh((j+1/2)\theta)/\cosh(\theta/2) d j = cosh (( j + 1/2 ) θ ) / cosh ( θ /2 ) before the last endpoint, and det ( P m + r I ) = r sinh ( m θ ) / sinh θ \det(P_m+rI)=r\sinh(m\theta)/\sinh\theta det ( P m + r I ) = r sinh ( m θ ) / sinh θ . The endpoint cofactors therefore give
v = sinh θ r sinh ( m θ ) , g = cosh ( ( m − 1 / 2 ) θ ) cosh ( θ / 2 ) . v=\frac{\sinh\theta}{r\sinh(m\theta)},\qquad
g=\frac{\cosh((m-1/2)\theta)}{\cosh(\theta/2)}. v = r sinh ( m θ ) sinh θ , g = cosh ( θ /2 ) cosh (( m − 1/2 ) θ ) .
For r = m − 2 r=m^{-2} r = m − 2 , θ = 2 asinh ( 1 / ( 2 m ) ) \theta=2\operatorname{asinh}(1/(2m)) θ = 2 asinh ( 1/ ( 2 m )) , m θ ≤ 1 m\theta\le1 m θ ≤ 1 , sinh θ ≥ 1 / m \sinh\theta\ge1/m sinh θ ≥ 1/ m and sinh ( m θ ) ≥ m sinh θ \sinh(m\theta)\ge m\sinh\theta sinh ( m θ ) ≥ m sinh θ , proving the bounds on v , g v,g v , g . The constant eigenvector contributes m m m to each diagonal entry of Q − 1 Q^{-1} Q − 1 , and ∥ Q − 1 ∥ = m 2 \lVert Q^{-1}\rVert=m^2 ∥ Q − 1 ∥ = m 2 , giving the bounds on u u u . The endpoint column maximum principle gives the entrywise claim in both branches. The remaining bounds follow by substitution and ⌊ t ⌋ ≥ t / 2 \lfloor t\rfloor\ge t/2 ⌊ t ⌋ ≥ t /2 for t ≥ 2 t\ge2 t ≥ 2 .
For the high branch, r = m − 2 r=m^{-2} r = m − 2 and K = M m K=Mm K = M m , so (27) specializes to
H n = max { C 0 2 g 2 M m 5 , 256 n } , H_n=\max\{C_0^2g^2Mm^5,256n\}, H n = max { C 0 2 g 2 M m 5 , 256 n } ,
D = O ( ( n + S κ 5 / 2 n − 5 / 4 ) log ( n κ S ) ) . D=O\left((n+S\kappa^{5/2}n^{-5/4})\log(n\kappa S)\right). D = O ( ( n + S κ 5/2 n − 5/4 ) log ( nκ S ) ) .
Indeed M = O ( S ) M=O(S) M = O ( S ) , m = O ( κ n − 1 / 4 ) m=O(\sqrt{\kappa}n^{-1/4}) m = O ( κ n − 1/4 ) and g ≤ cosh 1 g\le\cosh1 g ≤ cosh 1 . The primal dimension is 2 M D + 1 2MD+1 2 M D + 1 , including the public coordinate in (73) , and the dual dimension is K D KD K D , so the total dimension ( 2 M + K ) D + 1 (2M+K)D+1 ( 2 M + K ) D + 1 is (16) .
The source uses the scalar functions of Carmon et al. [4 ] and Li et al. [8 ] :
Ψ ( t ) = { 0 , t ≤ 1 / 2 , e 1 − ( 2 t − 1 ) − 2 , t > 1 / 2 , Φ ( t ) = e ∫ − ∞ t e − r 2 / 2 d r . (32) \Psi(t)=
\begin{cases}
0, & t\le1/2,\\
e^{1-(2t-1)^{-2}}, & t>1/2,
\end{cases}
\qquad
\Phi(t)=\sqrt{e}\int_{-\infty}^{t}e^{-r^2/2}\,dr. \tag*{(32)} Ψ ( t ) = { 0 , e 1 − ( 2 t − 1 ) − 2 , t ≤ 1/2 , t > 1/2 , Φ ( t ) = e ∫ − ∞ t e − r 2 /2 d r . (32)
The cap and primal penalty are
w j = ρ ( P j ) = P j / ( 1 + ∥ P j ∥ 2 / R 2 ) 1 / 2 , R = 128 s , (33) w_j=\rho(P_j)=P_j/\left(1+\lVert P_j\rVert^2/R^2\right)^{1/2}, \qquad R=128s, \tag*{(33)} w j = ρ ( P j ) = P j / ( 1 + ∥ P j ∥ 2 / R 2 ) 1/2 , R = 128 s , (33)
p ( P j ) = λ 1024 ( ∥ P j ∥ − R ) + 2 , x j = ⟨ U j , 1 , w j , X ⟩ , z j + 1 = ⟨ U j , m , w j , Z ⟩ . (34) p(P_j)=\frac{\lambda}{1024}\left(\lVert P_j\rVert-R\right)_+^2, \qquad x_j=\langle U_{j,1},w_{j,X}\rangle,\qquad z_{j+1}=\langle U_{j,m},w_{j,Z}\rangle. \tag*{(34)} p ( P j ) = 1024 λ ( ∥ P j ∥ − R ) + 2 , x j = ⟨ U j , 1 , w j , X ⟩ , z j + 1 = ⟨ U j , m , w j , Z ⟩ . (34)
Write w j , X = x j U j , 1 + A j , ⊥ w_{j,X}=x_jU_{j,1}+A_{j,\perp} w j , X = x j U j , 1 + A j , ⊥ and w j , Z = z j + 1 U j , m + B j + 1 , ⊥ w_{j,Z}=z_{j+1}U_{j,m}+B_{j+1,\perp} w j , Z = z j + 1 U j , m + B j + 1 , ⊥ . Let
J M = − Φ ( x 1 / s ) + ∑ j = 2 M [ Ψ ( − z j / s ) Φ ( − x j / s ) − Ψ ( z j / s ) Φ ( x j / s ) ] . (35) J_M=-\Phi(x_1/s)+\sum_{j=2}^{M}\left[\Psi(-z_j/s)\Phi(-x_j/s)-\Psi(z_j/s)\Phi(x_j/s)\right]. \tag*{(35)} J M = − Φ ( x 1 / s ) + j = 2 ∑ M [ Ψ ( − z j / s ) Φ ( − x j / s ) − Ψ ( z j / s ) Φ ( x j / s ) ] . (35)
For 1 / 2 < t < 1 1/2<t<1 1/2 < t < 1 define
w ( t ) = Ψ ( t ) , ω ( w ) = 1 − ( 1 − w ) 3 , w(t)=\sqrt{\Psi(t)},\qquad\omega(w)=1-(1-w)^3, w ( t ) = Ψ ( t ) , ω ( w ) = 1 − ( 1 − w ) 3 ,
θ + ( t ) = π 2 ω ( w ( t ) ) , a + ( t ) = sin θ + ( t ) , b + ( t ) = cos θ + ( t ) . (36) \theta_+(t)=\frac{\pi}{2}\omega(w(t)),\qquad a_+(t)=\sin\theta_+(t),\qquad b_+(t)=\cos\theta_+(t). \tag*{(36)} θ + ( t ) = 2 π ω ( w ( t )) , a + ( t ) = sin θ + ( t ) , b + ( t ) = cos θ + ( t ) . (36)
extending θ + \theta_+ θ + by 0 0 0 for t ≤ 1 / 2 t\leq1/2 t ≤ 1/2 and by π / 2 \pi/2 π /2 for t ≥ 1 t\geq1 t ≥ 1 . Set
α = a + 2 , q j = ∏ k = 2 j α ( z k / s ) , q 1 = 1 , (37) \alpha=a_+^2,\qquad q_j=\prod_{k=2}^{j}\alpha(z_k/s),\qquad q_1=1, \tag*{(37)} α = a + 2 , q j = k = 2 ∏ j α ( z k / s ) , q 1 = 1 , (37)
h j = ( ∏ k = 2 j a + ( z k / s ) ) b + ( z j + 1 / s ) , A j = h j Γ w j , Γ = diag ( I , − I / 2 ) . (38) h_j=\left(\prod_{k=2}^{j}a_+(z_k/s)\right)b_+(z_{j+1}/s),\qquad A_j=h_j\Gamma w_j,\qquad\Gamma=\operatorname{diag}(I,-I/2). \tag*{(38)} h j = ( k = 2 ∏ j a + ( z k / s ) ) b + ( z j + 1 / s ) , A j = h j Γ w j , Γ = diag ( I , − I /2 ) . (38)
The concatenated endpoint forcing obeys
h j 2 = q j [ 1 − α ( z j + 1 / s ) ] , ∑ j h j 2 ≤ 1 , ∥ A ∥ ≤ R . h_j^2=q_j[1-\alpha(z_{j+1}/s)],\qquad\sum_j h_j^2\leq1,\qquad\lVert A\rVert\leq R. h j 2 = q j [ 1 − α ( z j + 1 / s )] , j ∑ h j 2 ≤ 1 , ∥ A ∥ ≤ R .
Define the common primal part
P ( P ) = λ s 2 J M ( P ) − d ∥ A ( P ) ∥ 2 + ∑ j p ( P j ) . (39) \mathcal{P}(P)=\lambda s^2J_M(P)-d\lVert A(P)\rVert^2+\sum_j p(P_j). \tag*{(39)} P ( P ) = λ s 2 J M ( P ) − d ∥ A ( P ) ∥ 2 + j ∑ p ( P j ) . (39)
Let t j , k = ⟨ U j , k , y j , k ⟩ t_{j,k}=\langle U_{j,k},y_{j,k}\rangle t j , k = ⟨ U j , k , y j , k ⟩ , ϕ η ( t ) = 1 2 ( ∣ t ∣ − η ) + 2 \phi_\eta(t)=\frac{1}{2}(|t|-\eta)_+^2 ϕ η ( t ) = 2 1 ( ∣ t ∣ − η ) + 2 , and write Y ∂ Y_{\partial} Y ∂ for the concatenation of all endpoint coordinate blocks, ordered like A A A . For an edge e = ( ( j , k ) , ( j , k + 1 ) ) e=((j,k),(j,k+1)) e = (( j , k ) , ( j , k + 1 )) , write Y e = ( y j , k , y j , k + 1 ) Y_e=(y_{j,k},y_{j,k+1}) Y e = ( y j , k , y j , k + 1 ) and t e = t j , k − t j , k + 1 t_e=t_{j,k}-t_{j,k+1} t e = t j , k − t j , k + 1 . For a node v v v write y v , t v y_v,t_v y v , t v .
The cutoff functions are
χ ( t ) = { 1 t ≤ 1 , 1 − 10 r 3 + 15 r 4 − 6 r 5 t = 1 + r , 0 < r < 1 , 0 t ≥ 2 , (40) \chi(t)=
\begin{cases}
1 & t\leq1,\\
1-10r^3+15r^4-6r^5 & t=1+r,\ 0<r<1,\\
0 & t\geq2,
\end{cases} \tag*{(40)} χ ( t ) = ⎩ ⎨ ⎧ 1 1 − 10 r 3 + 15 r 4 − 6 r 5 0 t ≤ 1 , t = 1 + r , 0 < r < 1 , t ≥ 2 , (40)
ψ ( t ) = { 0 t ≤ 1 , ( t − 1 ) 3 / 6 1 < t < 2 , ( t − 2 ) 2 / 2 + ( t − 2 ) / 2 + 1 / 6 t ≥ 2. (41) \psi(t)=
\begin{cases}
0 & t\leq1,\\
(t-1)^3/6 & 1<t<2,\\
(t-2)^2/2+(t-2)/2+1/6 & t\geq2.
\end{cases} \tag*{(41)} ψ ( t ) = ⎩ ⎨ ⎧ 0 ( t − 1 ) 3 /6 ( t − 2 ) 2 /2 + ( t − 2 ) /2 + 1/6 t ≤ 1 , 1 < t < 2 , t ≥ 2. (41)
We localize the cutoff χ \chi χ and the dual radial profile ψ \psi ψ at the radius R ∗ R_{*} R ∗ by setting
R ∗ = 16 b R / ν , χ ∗ ( z ) = χ ( ∥ z ∥ / R ∗ ) , ψ ∗ ( z ) = R ∗ 2 ψ ( ∥ z ∥ / R ∗ ) . (42) R_{*}=16bR/\nu,\qquad\chi_{*}(z)=\chi(\lVert z\rVert/R_{*}),\qquad\psi_{*}(z)=R_{*}^{2}\psi(\lVert z\rVert/R_{*}). \tag*{(42)} R ∗ = 16 b R / ν , χ ∗ ( z ) = χ (∥ z ∥ / R ∗ ) , ψ ∗ ( z ) = R ∗ 2 ψ (∥ z ∥ / R ∗ ) . (42)
For the owned edges use the profile ϱ \varrho ϱ in (19) and define
h η ( r ) = η ϱ ( r / R ∗ ) , A η ( Y e ) = 1 2 ( ∣ t e ∣ − h η ( ∥ Y e ∥ ) ) + 2 , B T ( z ) = 1 2 ( ∥ z ∥ − T ) + 2 . (43) h_{\eta}(r)=\eta\varrho(r/R_{*}),\qquad A_{\eta}(Y_{e})=\frac{1}{2}\left(\lvert t_{e}\rvert-h_{\eta}(\lVert Y_{e}\rVert)\right)_{+}^{2},\qquad B_{T}(z)=\frac{1}{2}(\lVert z\rVert-T)_{+}^{2}. \tag*{(43)} h η ( r ) = η ϱ ( r / R ∗ ) , A η ( Y e ) = 2 1 ( ∣ t e ∣ − h η (∥ Y e ∥) ) + 2 , B T ( z ) = 2 1 (∥ z ∥ − T ) + 2 . (43)
Node and endpoint cutoffs keep radius R ∗ R_{*} R ∗ ; we use the positive width (65) and width zero.
The globally defined components are precisely
f i , η ( P , y ) = P ( P ) − ν 2 ∥ y ∥ 2 + b χ ∗ ( Y ∂ ) ⟨ A ( P ) , Y ∂ ⟩ − 45 ν ψ ∗ ( Y ∂ ) + ∑ v [ c χ ∗ ( y v ) φ η ( t v ) − 720 c ψ ∗ ( y v ) ] − n a ∑ e : Σ e = i A η ( Y e ) − a n ∑ e B R ∗ / 2 ( Y e ) . (44) \begin{aligned}
f_{i,\eta}(P,y)={}&\mathcal{P}(P)-\frac{\nu}{2}\lVert y\rVert^{2}+b\chi_{*}(Y_{\partial})\langle A(P),Y_{\partial}\rangle-45\nu\psi_{*}(Y_{\partial})\\
&+\sum_{v}\left[c\chi_{*}(y_{v})\varphi_{\eta}(t_{v})-720c\psi_{*}(y_{v})\right]\\
&-na\sum_{e:\Sigma_{e}=i}A_{\eta}(Y_{e})-a\sqrt{n}\sum_{e}B_{R_{*}/2}(Y_{e}). \tag*{(44)}
\end{aligned} f i , η ( P , y ) = P ( P ) − 2 ν ∥ y ∥ 2 + b χ ∗ ( Y ∂ ) ⟨ A ( P ) , Y ∂ ⟩ − 45 ν ψ ∗ ( Y ∂ ) + v ∑ [ c χ ∗ ( y v ) φ η ( t v ) − 720 c ψ ∗ ( y v ) ] − na e : Σ e = i ∑ A η ( Y e ) − a n e ∑ B R ∗ /2 ( Y e ) . (44)
Only the edge sum with a flat region depends on owners; every radial correction uses public norms and is common to all components.
The radii are nested as follows. Since ∥ A ∥ ≤ R \lVert\mathcal{A}\rVert\le R ∥ A ∥ ≤ R , every node, edge pair and endpoint tuple of the uncut dual maximizer has norm at most 2 b R / ν = R ∗ / 8 2bR/\nu=R_{*}/8 2 b R / ν = R ∗ /8 (Lemma 7 ), and after the width perturbation these stay inside 3 R ∗ / 16 3R_{*}/16 3 R ∗ /16 (Lemma 11 ). There the cutoffs equal one and the barriers ψ ∗ \psi_{*} ψ ∗ and B R ∗ / 2 B_{R_{*}/2} B R ∗ /2 vanish, so localization does not move the maximizer. The barriers act only beyond R ∗ / 2 R_{*}/2 R ∗ /2 , where B R ∗ / 2 B_{R_{*}/2} B R ∗ /2 dominates the curvature of the flat edges (Lemma 8 ).
Forcing Derivatives
Lemma 6 (Gate and forcing estimates). The nonnegative amplitudes a + , b + a_{+},b_{+} a + , b + are C 2 C^{2} C 2 , flat to second order at both joins, and satisfy a + 2 + b + 2 = 1 a_{+}^{2}+b_{+}^{2}=1 a + 2 + b + 2 = 1 . With C = 23 C=23 C = 23 ,
Ψ ( t ) ≤ α ( t ) ≤ C Ψ ( t ) , 1 2 < t < 1 , (45) \Psi(t)\le\alpha(t)\le C\Psi(t),\qquad\frac{1}{2}<t<1, \tag*{(45)} Ψ ( t ) ≤ α ( t ) ≤ C Ψ ( t ) , 2 1 < t < 1 , (45) α ( t ) / C ≤ Ψ ( t ) ≤ e α ( t ) , t ≥ 0 , (46) \alpha(t)/C\le\Psi(t)\le e\alpha(t),\qquad t\ge0, \tag*{(46)} α ( t ) / C ≤ Ψ ( t ) ≤ e α ( t ) , t ≥ 0 , (46) 0 ≤ α ′ ( t ) ≤ C Ψ ′ ( t ) , 1 2 < t < 1. (47) 0\le\alpha^{\prime}(t)\le C\Psi^{\prime}(t),\qquad\frac{1}{2}<t<1. \tag*{(47)} 0 ≤ α ′ ( t ) ≤ C Ψ ′ ( t ) , 2 1 < t < 1. (47) Outside the transition α ′ = 0 \alpha^{\prime}=0 α ′ = 0 . Uniformly in length, dimension, owners, signs and raw radii,
∥ D A ∥ ≤ 2433 , ∥ D 2 A ∥ < 231000 / s , (48) \lVert D\mathcal{A}\rVert\le2433,\qquad\lVert D^{2}\mathcal{A}\rVert<231000/s, \tag*{(48)} ∥ D A ∥ ≤ 2433 , ∥ D 2 A ∥ < 231000/ s , (48) where the second norm is bilinear-to-vector in Euclidean norms.
Proof. Gate. At onset w = e 1 / 2 − [ 2 ( 2 t − 1 ) 2 ] − 1 w=e^{1/2-[2(2t-1)^{2}]^{-1}} w = e 1/2 − [ 2 ( 2 t − 1 ) 2 ] − 1 is infinitely flat; at completion 1 − ω ( w ) = ( 1 − w ) 3 1-\omega(w)=(1-w)^{3} 1 − ω ( w ) = ( 1 − w ) 3 , so b + = O ( ( 1 − t ) 3 ) b_{+}=O((1-t)^{3}) b + = O (( 1 − t ) 3 ) and 1 − a + = O ( ( 1 − t ) 6 ) 1-a_{+}=O((1-t)^{6}) 1 − a + = O (( 1 − t ) 6 ) . For 0 ≤ w ≤ 1 0\le w\le1 0 ≤ w ≤ 1 , w ≤ ω ( w ) ≤ 3 w w\le\omega(w)\le3w w ≤ ω ( w ) ≤ 3 w and ω ( w ) ≤ sin ( π ω ( w ) / 2 ) ≤ π ω ( w ) / 2 \omega(w)\le\sin(\pi\omega(w)/2)\le\pi\omega(w)/2 ω ( w ) ≤ sin ( π ω ( w ) /2 ) ≤ π ω ( w ) /2 . Hence the first two comparisons hold, using 9 π 2 / 4 < C 9\pi^{2}/4<C 9 π 2 /4 < C , α = 1 \alpha=1 α = 1 and 1 ≤ Ψ < e 1\le\Psi<e 1 ≤ Ψ < e on t ≥ 1 t\ge1 t ≥ 1 . Differentiating gives α ′ / Ψ ′ = π ω ′ ( w ) sin θ + + cos θ + / ( 2 w ) < C \alpha^{\prime}/\Psi^{\prime}=\pi\omega^{\prime}(w)\sin\theta_{+}+\cos\theta_{+}/(2w)<C α ′ / Ψ ′ = π ω ′ ( w ) sin θ + + cos θ + / ( 2 w ) < C .
Angle and radial maps. For v = ( 2 t − 1 ) − 2 ≥ 1 v=(2t-1)^{-2}\ge1 v = ( 2 t − 1 ) − 2 ≥ 1 , w ′ = 2 e ( 1 − v ) / 2 v 3 / 2 w^{\prime}=2e^{(1-v)/2}v^{3/2} w ′ = 2 e ( 1 − v ) /2 v 3/2 and w ′ ′ = 4 e ( 1 − v ) / 2 v − 2 ( v − 3 ) w^{\prime\prime}=4e^{(1-v)/2}v^{-2}(v-3) w ′′ = 4 e ( 1 − v ) /2 v − 2 ( v − 3 ) . Maximizing v − v / 2 v^{-v/2} v − v /2 gives ∣ w ′ ∣ < 4 \lvert w^{\prime}\rvert<4 ∣ w ′ ∣ < 4 , ∣ w ′ ′ ∣ < 120 \lvert w^{\prime\prime}\rvert<120 ∣ w ′′ ∣ < 120 and thus ∣ θ + ′ ∣ < 19 \lvert\theta_{+}^{\prime}\rvert<19 ∣ θ + ′ ∣ < 19 , ∣ θ + ′ ′ ∣ < 720 \lvert\theta_{+}^{\prime\prime}\rvert<720 ∣ θ + ′′ ∣ < 720 . Writing c = ( 1 + ∥ P ∥ 2 / R 2 ) − 1 / 2 c=(1+\lVert P\rVert^{2}/R^{2})^{-1/2} c = ( 1 + ∥ P ∥ 2 / R 2 ) − 1/2 gives D ρ = c I − c 3 P P T / R 2 D\rho=cI-c^{3}PP^{\mathsf{T}}/R^{2} D ρ = c I − c 3 P P T / R 2 , with eigenvalues c c c , c 3 c^3 c 3 . The three second-derivative terms linear in P P P total at most 3 r ( 1 + r 2 ) − 3 / 2 / R 3r(1+r^2)^{-3/2}/R 3 r ( 1 + r 2 ) − 3/2 / R , and the cubic term is at most 3 r 3 ( 1 + r 2 ) − 5 / 2 / R 3r^3(1+r^2)^{-5/2}/R 3 r 3 ( 1 + r 2 ) − 5/2 / R , where r = ∥ P ∥ / R r=\|P\|/R r = ∥ P ∥/ R . Each scalar factor is at most 1 / 2 1/2 1/2 , so ∥ D ρ ∥ ≤ 1 \|D\rho\|\le1 ∥ D ρ ∥ ≤ 1 , ∥ D 2 ρ ∥ ≤ 3 / R \|D^2\rho\|\le3/R ∥ D 2 ρ ∥ ≤ 3/ R . The disjoint-block angle map Θ j ( P ) = θ + ( z j + 1 / s ) \Theta_j(P)=\theta+(z_{j+1}/s) Θ j ( P ) = θ + ( z j + 1 / s ) therefore obeys
∥ D Θ ∥ < 19 / s , ∥ D 2 Θ ∥ < 721 / s 2 . \|D\Theta\|<19/s,\qquad\|D^2\Theta\|<721/s^2. ∥ D Θ∥ < 19/ s , ∥ D 2 Θ∥ < 721/ s 2 .
For the second bound use ∑ j ∥ U j ∥ 2 ∥ W j ∥ 2 ≤ ∥ U ∥ 2 ∥ W ∥ 2 \sum_j\|U_j\|^2\|W_j\|^2\le\|U\|^2\|W\|^2 ∑ j ∥ U j ∥ 2 ∥ W j ∥ 2 ≤ ∥ U ∥ 2 ∥ W ∥ 2 .
Spherical coordinates. To avoid a length factor, put p 0 = 1 p_0=1 p 0 = 1 , p k = ∏ i ≤ k sin ϑ i p_k=\prod_{i\le k}\sin\vartheta_i p k = ∏ i ≤ k sin ϑ i and
H ( ϑ ) = ( cos ϑ 1 , p 1 cos ϑ 2 , … , p M − 1 cos ϑ M , p M ) . H(\vartheta)=(\cos\vartheta_1,p_1\cos\vartheta_2,\ldots,p_{M-1}\cos\vartheta_M,p_M). H ( ϑ ) = ( cos ϑ 1 , p 1 cos ϑ 2 , … , p M − 1 cos ϑ M , p M ) .
Let T k T_k T k be the unit tail starting with cos ϑ k \cos\vartheta_k cos ϑ k , padded by zeros, and Q k = ∂ k T k Q_k=\partial_kT_k Q k = ∂ k T k . The Q k Q_k Q k are orthonormal, since earlier derivatives are multiples of a later tail and tangents are orthogonal to that tail. Moreover,
∂ k H = p k − 1 Q k , ∥ ∇ p k − 1 ∥ 2 = ∑ i < k cos 2 ϑ i ∏ r < k r ≠ i sin 2 ϑ r ≤ 1. \partial_kH=p_{k-1}Q_k,\qquad\|\nabla p_{k-1}\|^2=\sum_{i<k}\cos^2\vartheta_i\prod_{\substack{r<k\\r\ne i}}\sin^2\vartheta_r\le1. ∂ k H = p k − 1 Q k , ∥∇ p k − 1 ∥ 2 = i < k ∑ cos 2 ϑ i r < k r = i ∏ sin 2 ϑ r ≤ 1.
The last inequality follows by expanding ∏ i < k ( sin 2 ϑ i + cos 2 ϑ i ) = 1 \prod_{i<k}(\sin^2\vartheta_i+\cos^2\vartheta_i)=1 ∏ i < k ( sin 2 ϑ i + cos 2 ϑ i ) = 1 . Thus ∥ D H ∥ ≤ 1 \|DH\|\le1 ∥ D H ∥ ≤ 1 , and
D 2 H [ U , W ] = − ∑ k U k W k p k − 1 T k + ∑ k [ W k ⟨ ∇ p k − 1 , U < k ⟩ + U k ⟨ ∇ p k − 1 , W < k ⟩ ] Q k \begin{aligned}
D^2H[U,W]={}&-\sum_kU_kW_kp_{k-1}T_k\\
&+\sum_k[W_k\langle\nabla p_{k-1},U_{<k}\rangle+U_k\langle\nabla p_{k-1},W_{<k}\rangle]Q_k
\end{aligned} D 2 H [ U , W ] = − k ∑ U k W k p k − 1 T k + k ∑ [ W k ⟨ ∇ p k − 1 , U < k ⟩ + U k ⟨ ∇ p k − 1 , W < k ⟩] Q k
has norm at most 3 ∥ U ∥ ∥ W ∥ 3\|U\|\|W\| 3∥ U ∥∥ W ∥ , by Cauchy–Schwarz in the first sum and orthogonality in each of the other two.
Forcing map. Since h h h contains the first M M M coordinates of H ( Θ ) H(\Theta) H ( Θ ) , ∥ D h ∥ ≤ 19 / s \|Dh\|\le19/s ∥ D h ∥ ≤ 19/ s and ∥ D 2 h ∥ < 1804 / s 2 \|D^2h\|<1804/s^2 ∥ D 2 h ∥ < 1804/ s 2 . Differentiating A j = h j Γ ρ ( P j ) A_j=h_j\Gamma\rho(P_j) A j = h j Γ ρ ( P j ) in orthogonal output blocks gives
∥ D A ∥ ≤ 1 + R ∥ D h ∥ ≤ 2433 , ∥ D 2 A ∥ ≤ 3 / R + 2 ∥ D h ∥ + R ∥ D 2 h ∥ < 231000 / s . \|DA\|\le1+R\|Dh\|\le2433,\qquad\|D^2A\|\le3/R+2\|Dh\|+R\|D^2h\|<231000/s. ∥ D A ∥ ≤ 1 + R ∥ D h ∥ ≤ 2433 , ∥ D 2 A ∥ ≤ 3/ R + 2∥ D h ∥ + R ∥ D 2 h ∥ < 231000/ s .
Class Membership and Exact Zero-Width Elimination
Lemma 7 (Uncut energy and Schur identity). Define
E η ( y ) = ν 2 ∥ y ∥ 2 − c ∑ v ϕ η ( t v ) + a ∑ e ϕ η ( t e ) . (49) \mathcal{E}_\eta(y)=\frac{\nu}{2}\|y\|^2-c\sum_v\phi_\eta(t_v)+a\sum_e\phi_\eta(t_e). \tag*{(49)} E η ( y ) = 2 ν ∥ y ∥ 2 − c v ∑ ϕ η ( t v ) + a e ∑ ϕ η ( t e ) . (49) It is μ 0 \mu_0 μ 0 -strongly convex. At η = 0 \eta=0 η = 0 , its hidden-axis Hessian on every path is a Q aQ a Q , and its complementary Hessian is ν I \nu I ν I . Maximizing P ( P ) + b ⟨ A ( P ) , Y ∂ ⟩ − E 0 ( y ) \mathcal{P}(P)+b\langle\mathcal{A}(P),Y_\partial\rangle-\mathcal{E}_0(y) P ( P ) + b ⟨ A ( P ) , Y ∂ ⟩ − E 0 ( y ) gives exactly
V 0 ( P ) = λ s 2 J M ( P ) + ∑ j h j 2 Q j ( w j ) + ∑ j p ( P j ) , (50) V_0(P)=\lambda s^2J_M(P)+\sum_j h_j^2Q_j(w_j)+\sum_jp(P_j), \tag*{(50)} V 0 ( P ) = λ s 2 J M ( P ) + j ∑ h j 2 Q j ( w j ) + j ∑ p ( P j ) , (50) where
Q j ( w ) = 6 λ ( x j − z j + 1 / 2 ) 2 + 6 λ ∥ A j , ⊥ ∥ 2 + 3 λ 2 ∥ B j + 1 , ⊥ ∥ 2 . (51) Q_j(w)=6\lambda(x_j-z_{j+1/2})^2+6\lambda\|A_{j,\perp}\|^2+\frac{3\lambda}{2}\|B_{j+1,\perp}\|^2. \tag*{(51)} Q j ( w ) = 6 λ ( x j − z j + 1/2 ) 2 + 6 λ ∥ A j , ⊥ ∥ 2 + 2 3 λ ∥ B j + 1 , ⊥ ∥ 2 . (51) Its unique uncut maximizer obeys
∥ Y ∂ , 0 ∥ ≤ 2 b R / ν , ∥ y v , 0 ∥ ≤ 2 b R / ν , ∥ Y e , 0 ∥ ≤ 2 b R / ν . (52) \|Y_{\partial,0}\|\le2bR/\nu,\qquad\|y_{v,0}\|\le2bR/\nu,\qquad\|Y_{e,0}\|\le2bR/\nu. \tag*{(52)} ∥ Y ∂ , 0 ∥ ≤ 2 b R / ν , ∥ y v , 0 ∥ ≤ 2 b R / ν , ∥ Y e , 0 ∥ ≤ 2 b R / ν . (52)
Proof. Let U U U map the vector of hidden scalar coordinates to their orthogonal ambient node axes, and let D U D_U D U be the incidence map on those scalars; ∥ D U ∥ ≤ 2 \lVert D_U\rVert\le2 ∥ D U ∥ ≤ 2 . Almost everywhere,
∇ 2 E η = ν I − c U diag ( active nodes ) U T + a D U T diag ( active edges ) D U ⪰ ( ν − c ) I = μ 0 I . \nabla^2\mathcal{E}_{\eta}=\nu I-cU\operatorname{diag}(\text{active nodes})U^{\mathsf{T}}+aD_U^{\mathsf{T}}\operatorname{diag}(\text{active edges})D_U\succeq(\nu-c)I=\mu_0 I. ∇ 2 E η = ν I − c U diag ( active nodes ) U T + a D U T diag ( active edges ) D U ⪰ ( ν − c ) I = μ 0 I .
The gradient is continuous at the hinges, so line integration gives strong convexity everywhere. At zero width the hidden Hessian is μ 0 I + a P m = a Q \mu_0I+aP_m=aQ μ 0 I + a P m = a Q , while the orthogonal complements have Hessian ν I \nu I ν I . For one path its eliminated contribution before compensation is
b 2 h j 2 2 a ( u x j 2 − v x j z j + 1 + u z j + 1 2 / 4 ) + b 2 h j 2 2 ν ( ∥ A j , ∥ ∥ 2 + ∥ B j + 1 , ∥ ∥ 2 / 4 ) . \frac{b^2h_j^2}{2a}\left(ux_j^2-vx_jz_{j+1}+uz_{j+1}^2/4\right)+\frac{b^2h_j^2}{2\nu}\left(\lVert A_{j,\parallel}\rVert^2+\lVert B_{j+1,\parallel}\rVert^2/4\right). 2 a b 2 h j 2 ( u x j 2 − v x j z j + 1 + u z j + 1 2 /4 ) + 2 ν b 2 h j 2 ( ∥ A j , ∥ ∥ 2 + ∥ B j + 1 , ∥ ∥ 2 /4 ) .
Both endpoint diagonal coefficients after subtracting d h j 2 ( ∥ w j , X ∥ 2 + ∥ w j , Z ∥ 2 / 4 ) dh_j^2(\lVert w_{j,X}\rVert^2+\lVert w_{j,Z}\rVert^2/4) d h j 2 (∥ w j , X ∥ 2 + ∥ w j , Z ∥ 2 /4 ) are fixed by b 2 u / ( 2 a ) − d = b 2 / ( 2 ν ) − d = 6 λ b^2u/(2a)-d=b^2/(2\nu)-d=6\lambda b 2 u / ( 2 a ) − d = b 2 / ( 2 ν ) − d = 6 λ . This proves the identity.
The hidden endpoint Green matrix has norm u + v ≤ 2 u u+v\le2u u + v ≤ 2 u and the complementary inverse is 1 / ν 1/\nu 1/ ν . Summing over the orthogonal bridges and using ∑ j ∥ A j ∥ 2 ≤ R 2 \sum_j\lVert A_j\rVert^2\le R^2 ∑ j ∥ A j ∥ 2 ≤ R 2 proves the first optimizer bound. Any two interior rows, such as the two nodes of an edge, restricted to the two source columns form a 2 × 2 2\times2 2 × 2 matrix with entries at most u u u , hence norm at most 2 u 2u 2 u . The complementary sources occur only at endpoints, where the inverse has norm 1 / ν 1/\nu 1/ ν . Taking the maximum of these orthogonal operators proves both local bounds. ■
Lemma 8 (Radial calculus). The following bounds hold globally, where Hessians are understood almost everywhere.
(i) If a C 1 , 1 C^{1,1} C 1 , 1 scalar A A A satisfies ∣ A ∣ / R ∗ 2 , ∥ ∇ A ∥ / R ∗ ≤ 4 A 0 |A|/R_*^2,\lVert\nabla A\rVert/R_*\le4A_0 ∣ A ∣/ R ∗ 2 , ∥ ∇ A ∥ / R ∗ ≤ 4 A 0 on R ∗ < ∥ z ∥ < 2 R ∗ R_*<\lVert z\rVert<2R_* R ∗ < ∥ z ∥ < 2 R ∗ , then
∇ 2 ( χ ∗ A ) − χ ∗ ∇ 2 A ⪯ 720 A 0 ∇ 2 ψ ∗ . \nabla^2(\chi_*A)-\chi_*\nabla^2A\preceq720A_0\nabla^2\psi_*. ∇ 2 ( χ ∗ A ) − χ ∗ ∇ 2 A ⪯ 720 A 0 ∇ 2 ψ ∗ . (ii) For every η ≥ 0 \eta\ge0 η ≥ 0 , the radial threshold h η ( ∥ z ∥ ) h_\eta(\lVert z\rVert) h η (∥ z ∥) is convex and globally C 1 , 1 C^{1,1} C 1 , 1 , constant on ∥ z ∥ ≤ R ∗ \lVert z\rVert\le R_* ∥ z ∥ ≤ R ∗ , and satisfies ∥ ∇ h η ∥ ≤ β : = η / R ∗ \lVert\nabla h_\eta\rVert\le\beta:=\eta/R_* ∥ ∇ h η ∥ ≤ β := η / R ∗ and h η ( r ) ≥ max { η , η r / ( 2 R ∗ ) } h_\eta(r)\ge\max\{\eta,\eta r/(2R_*)\} h η ( r ) ≥ max { η , η r / ( 2 R ∗ )} . For r = ∥ z ∥ > 0 r=\lVert z\rVert>0 r = ∥ z ∥ > 0 , u = z / r u=z/r u = z / r ,
∇ 2 h η = η R ∗ 2 ϱ ′ ′ ( r / R ∗ ) u u T + η R ∗ r ϱ ′ ( r / R ∗ ) ( I − u u T ) ⪰ 0. (53) \nabla^2h_\eta=\frac{\eta}{R_*^2}\varrho^{\prime\prime}(r/R_*)uu^{\mathsf{T}}+\frac{\eta}{R_*r}\varrho^\prime(r/R_*)(I-uu^{\mathsf{T}})\succeq0. \tag*{(53)} ∇ 2 h η = R ∗ 2 η ϱ ′′ ( r / R ∗ ) u u T + R ∗ r η ϱ ′ ( r / R ∗ ) ( I − u u T ) ⪰ 0. (53) Here 0 ≤ ϱ ′ ≤ 1 0\le\varrho^\prime\le1 0 ≤ ϱ ′ ≤ 1 and ϱ ′ ′ = 1 ( 1 , 2 ) \varrho^{\prime\prime}=\mathbb{1}_{(1,2)} ϱ ′′ = 1 ( 1 , 2 ) almost everywhere.
(iii) If n ≥ 2 n\ge2 n ≥ 2 , 0 ≤ η ≤ R ∗ / ( 16 n ) 0\le\eta\le R_*/(16\sqrt{n}) 0 ≤ η ≤ R ∗ / ( 16 n ) and a vector e U e_U e U satisfies ∥ e U ∥ ≤ 2 \lVert e_U\rVert\le\sqrt{2} ∥ e U ∥ ≤ 2 , the owned edge
G ( z ) = − n a 2 ( ∣ ⟨ e U , z ⟩ ∣ − h η ( ∥ z ∥ ) ) + 2 G(z)=-\frac{na}{2}\left(\lvert\langle e_U,z\rangle\rvert-h_\eta(\lVert z\rVert)\right)_+^2 G ( z ) = − 2 na ( ∣⟨ e U , z ⟩∣ − h η (∥ z ∥) ) + 2 is globally C 1 , 1 C^{1,1} C 1 , 1 , with
∥ ∇ 2 G ∥ ≤ 9 4 n a , G − a n R ∗ / 2 globally concave. (54) \lVert\nabla^2G\rVert\le\frac{9}{4}na,\qquad G-a\sqrt{n}R_*/2\text{ globally concave.} \tag*{(54)} ∥ ∇ 2 G ∥ ≤ 4 9 na , G − a n R ∗ /2 globally concave. (54)
Proof. On ∥ z ∥ / R ∗ = 1 + r \lVert z\rVert/R_*=1+r ∥ z ∥ / R ∗ = 1 + r , 0 < r < 1 0<r<1 0 < r < 1 , one has ∣ χ ′ ∣ ≤ 30 r 2 |\chi'|\le30r^2 ∣ χ ′ ∣ ≤ 30 r 2 , ∣ χ ′ ′ ∣ ≤ 60 r |\chi''|\le60r ∣ χ ′′ ∣ ≤ 60 r . In direction t e r + w te_r+w t e r + w , w ⊥ e r w\perp e_r w ⊥ e r , the two extra product rule terms are bounded above by
A 0 ( 480 r t 2 + 240 r 2 ∣ t ∣ ∥ w ∥ + 120 r 2 ∥ w ∥ 2 ) ≤ 720 A 0 ( r t 2 + r 2 ∥ w ∥ 2 / 4 ) . A_0\left(480rt^2+240r^2|t|\lVert w\rVert+120r^2\lVert w\rVert^2\right)\le720A_0\left(rt^2+r^2\lVert w\rVert^2/4\right). A 0 ( 480 r t 2 + 240 r 2 ∣ t ∣ ∥ w ∥ + 120 r 2 ∥ w ∥ 2 ) ≤ 720 A 0 ( r t 2 + r 2 ∥ w ∥ 2 /4 ) .
The difference is positive semidefinite, since its matrix has diagonals 240 r 240r 240 r , 60 r 2 60r^{2} 60 r 2 and off-diagonal − 120 r 2 -120r^{2} − 120 r 2 , with determinant 14400 r 3 ( 1 − r ) ≥ 0 14400r^{3}(1-r) \ge0 14400 r 3 ( 1 − r ) ≥ 0 . The radial and tangential eigenvalues of ∇ 2 ψ ∗ \nabla^{2}\psi_{*} ∇ 2 ψ ∗ are r r r and r 2 / [ 2 ( 1 + r ) ] ≥ r 2 / 4 r^{2}/[2(1+r)] \ge r^{2}/4 r 2 / [ 2 ( 1 + r )] ≥ r 2 /4 . Outside this annulus the cutoff is constant, proving (i).
The derivative of ϱ \varrho ϱ is 0 0 0 , t − 1 t-1 t − 1 , and 1 1 1 on its three intervals, continuous and 1 1 1 -Lipschitz. The radial Hessian formula gives (ii), including regularity at the origin since the threshold is locally constant. The lower bound uses ϱ ≥ 1 ≥ t / 2 \varrho\ge1 \ge t/2 ϱ ≥ 1 ≥ t /2 on [ 0 , 2 ] [0,2] [ 0 , 2 ] and ϱ ( t ) = t − 1 / 2 ≥ t / 2 \varrho(t)=t-1/2 \ge t/2 ϱ ( t ) = t − 1/2 ≥ t /2 for every t ≥ 2 t \ge2 t ≥ 2 .
For (iii), write r = ∥ z ∥ r=\lVert z\rVert r = ∥ z ∥ and, on the active set, put q = ∣ ⟨ e U , z ⟩ ∣ − h η ( r ) > 0 q=\lvert\langle e_U,z\rangle\rvert-h_\eta(r)>0 q = ∣⟨ e U , z ⟩∣ − h η ( r ) > 0 and v = sign ( ⟨ e U , z ⟩ ) e U − ∇ h η v=\operatorname{sign}(\langle e_U,z\rangle)e_U-\nabla h_\eta v = sign (⟨ e U , z ⟩) e U − ∇ h η . Then
∇ G = − n a q v , ∇ 2 G = − n a v v T + n a q ∇ 2 h η . \nabla G=-naqv,\qquad\nabla^{2}G=-na\,vv^{\mathsf T}+na\,q\nabla^{2}h_\eta. ∇ G = − na q v , ∇ 2 G = − na v v T + na q ∇ 2 h η .
For η > 0 \eta>0 η > 0 , away from ⟨ e U , z ⟩ = 0 \langle e_U,z\rangle=0 ⟨ e U , z ⟩ = 0 this is the locally C 1 , 1 C^{1,1} C 1 , 1 composition − n a ( q + ) 2 / 2 -na(q_{+})^{2}/2 − na ( q + ) 2 /2 ; the gradient vanishes continuously at activation. Near ⟨ e U , z ⟩ = 0 \langle e_U,z\rangle=0 ⟨ e U , z ⟩ = 0 the function is locally zero. For η = 0 \eta=0 η = 0 , G = − n a ⟨ e U , z ⟩ 2 / 2 G=-na\langle e_U,z\rangle^{2}/2 G = − na ⟨ e U , z ⟩ 2 /2 directly. Since q ≤ 2 r q\le\sqrt{2}r q ≤ 2 r , the radial and tangential eigenvalues of q ∇ 2 h η q\nabla^{2}h_\eta q ∇ 2 h η are at most 2 2 β 2\sqrt{2}\beta 2 2 β and 2 β \sqrt{2}\beta 2 β , respectively; the radial term is supported on R ∗ < r < 2 R ∗ R_{*}<r<2R_{*} R ∗ < r < 2 R ∗ . Thus
0 ⪯ n a q ∇ 2 h η ⪯ 2 2 n a β I ⪯ ( 2 / 8 ) a n I , (55) 0\preceq naq\nabla^{2}h_\eta\preceq2\sqrt{2}na\beta I\preceq(\sqrt{2}/8)a\sqrt{n}I, \tag*{(55)} 0 ⪯ na q ∇ 2 h η ⪯ 2 2 na β I ⪯ ( 2 /8 ) a n I , (55)
vanishing on r < R ∗ r<R_{*} r < R ∗ . Also, since β ≤ 1 / ( 16 n ) \beta\le1/(16\sqrt{n}) β ≤ 1/ ( 16 n ) and n ≥ 2 n\ge2 n ≥ 2 , ∥ v ∥ 2 ≤ ( 2 + β ) 2 ≤ ( 2 + 1 / ( 16 2 ) ) 2 = 1089 / 512 < 9 / 4 \lVert v\rVert^{2}\le(\sqrt{2}+\beta)^{2}\le(\sqrt{2}+1/(16\sqrt{2}))^{2}=1089/512<9/4 ∥ v ∥ 2 ≤ ( 2 + β ) 2 ≤ ( 2 + 1/ ( 16 2 ) ) 2 = 1089/512 < 9/4 and 2 2 β ≤ 1 / 8 2\sqrt{2}\beta\le1/8 2 2 β ≤ 1/8 . The norm of the difference of two positive semidefinite matrices is at most the larger norm, proving the edge bound. For r > R ∗ / 2 r>R_{*}/2 r > R ∗ /2 , ∇ 2 B R ∗ / 2 = u u T + ( 1 − R ∗ / ( 2 r ) ) ( I − u u T ) \nabla^{2}B_{R_{*}/2}=uu^{\mathsf T}+(1-R_{*}/(2r))(I-uu^{\mathsf T}) ∇ 2 B R ∗ /2 = u u T + ( 1 − R ∗ / ( 2 r )) ( I − u u T ) ; it is nonnegative everywhere and at least I / 2 I/2 I /2 on r ≥ R ∗ r\ge R_{*} r ≥ R ∗ , which dominates (55) after multiplication by a n a\sqrt{n} a n . All gradients match at the profile, cutoff and activation joins. Bounded weak Hessians, mollification and line integration therefore give the stated global C 1 , 1 C^{1,1} C 1 , 1 and curvature claims. ■
Lemma 9. (Componentwise strong concavity and RMS smoothness). The components (45) are globally C 1 , 1 C^{1,1} C 1 , 1 , each component is μ 0 \mu_{0} μ 0 -strongly concave in y y y , and their global joint RMS smoothness constant is at most
L r o b ≤ 601 λ + 10 8 d + 32 ⋅ 128 ⋅ 231000 b 2 / ν + 13000 b + 13 2 a n + 741 c + 47 ν < 4 L / 5. (56) L_{\mathrm{rob}}\le601\lambda+10^{8}d+32\cdot128\cdot231000\,b^{2}/\nu+13000b+\frac{13}{2}a\sqrt{n}+741c+47\nu<4L/5. \tag*{(56)} L rob ≤ 601 λ + 1 0 8 d + 32 ⋅ 128 ⋅ 231000 b 2 / ν + 13000 b + 2 13 a n + 741 c + 47 ν < 4 L /5. (56) The pure primal Hessian also obeys, almost everywhere,
∥ ∇ P P 2 f i , η ∥ ≤ K P : = 601 λ + 10 8 d + 32 ⋅ 128 ⋅ 231000 b 2 / ν < L / 32. (57) \lVert\nabla^{2}_{PP}f_{i,\eta}\rVert\le K_{P}:=601\lambda+10^{8}d+32\cdot128\cdot231000\,b^{2}/\nu<L/32. \tag*{(57)} ∥ ∇ P P 2 f i , η ∥ ≤ K P := 601 λ + 1 0 8 d + 32 ⋅ 128 ⋅ 231000 b 2 / ν < L /32. (57) These bounds hold for every owner assignment, all dimensions and 0 ≤ η ≤ R ∗ / ( 16 n ) 0\le\eta\le R_{*}/(16\sqrt{n}) 0 ≤ η ≤ R ∗ / ( 16 n ) , including when any subset of axes is replaced by zero consistently in its primal and dual occurrences.
Proof Strong concavity. Lemma 8 (iii) makes each owned edge plus its shared correction concave; unowned corrections are concave. Apply part (i) to ( A , A 0 ) = ( c ϕ η ( ⟨ U , y v ⟩ ) , c ) (A,A_{0})=(c\phi_{\eta}(\langle U,y_{v}\rangle),c) ( A , A 0 ) = ( c ϕ η (⟨ U , y v ⟩) , c ) at each node and ( b A ( P ) , Y ∂ ) , b R / R ∗ ) (b\mathcal{A}(P),Y_{\partial}),bR/R_{*}) ( b A ( P ) , Y ∂ ) , b R / R ∗ ) at the endpoints. Since b R / R ∗ = ν / 16 bR/R_{*}=\nu/16 b R / R ∗ = ν /16 , the barriers remove the extra curvature and leave ∇ y y 2 f i , η ⪯ − ν I + c U U T ⪯ − μ 0 I \nabla^{2}_{yy}f_{i,\eta}\preceq-\nu I+cUU^{\mathsf T}\preceq-\mu_{0}I ∇ y y 2 f i , η ⪯ − ν I + c U U T ⪯ − μ 0 I .
Common terms. For the norm estimates, put A 1 = 2433 A_1 = 2433 A 1 = 2433 , A 2 = 231000 A_2 = 231000 A 2 = 231000 , C 1 = 15 / 8 C_1 = 15/8 C 1 = 15/8 and C 2 = 10 / 3 C_2 = 10/\sqrt{3} C 2 = 10/ 3 . Direct differentiation gives ∥ ∇ χ ∗ ∥ ≤ C 1 / R ∗ \lVert\nabla\chi_*\rVert\le C_1/R_* ∥ ∇ χ ∗ ∥ ≤ C 1 / R ∗ , ∥ ∇ 2 χ ∗ ∥ ≤ C 2 / R ∗ 2 \lVert\nabla^2\chi_*\rVert\le C_2/R_*^2 ∥ ∇ 2 χ ∗ ∥ ≤ C 2 / R ∗ 2 and ∥ ∇ 2 ψ ∗ ∥ ≤ 1 \lVert\nabla^2\psi_*\rVert\le1 ∥ ∇ 2 ψ ∗ ∥ ≤ 1 . Also 0 ≤ Ψ < 3 0 \le\Psi< 3 0 ≤ Ψ < 3 , ∣ Ψ ′ ∣ < 5 |\Psi'| < 5 ∣ Ψ ′ ∣ < 5 , ∣ Ψ ′ ′ ∣ < 100 |\Psi''| < 100 ∣ Ψ ′′ ∣ < 100 and 0 < Φ < 5 0 < \Phi< 5 0 < Φ < 5 , ∣ Φ ′ ∣ < 2 |\Phi'| < 2 ∣ Φ ′ ∣ < 2 , ∣ Φ ′ ′ ∣ ≤ 1 |\Phi''| \le1 ∣ Φ ′′ ∣ ≤ 1 . In particular, for v = ( 2 t − 1 ) − 2 v=(2t-1)^{-2} v = ( 2 t − 1 ) − 2 , Ψ ′ ′ = 8 e 1 − v v 2 ( 2 v − 3 ) \Psi''=8e^{1-v}v^2(2v-3) Ψ ′′ = 8 e 1 − v v 2 ( 2 v − 3 ) gives ∣ Ψ ′ ′ ∣ ≤ 432 / e 2 + 96 / e < 100 |\Psi''| \le432/e^2+96/e<100 ∣ Ψ ′′ ∣ ≤ 432/ e 2 + 96/ e < 100 by maximizing powers times exponentials. The source-pair Hessian entries are bounded by 500, 3, 10, hence its norm is below 510; composition with the caps adds less than ( 3 / 128 ) 6 2 + 25 2 < 1 (3/128)\sqrt{6^2}+25^2<1 ( 3/128 ) 6 2 + 2 5 2 < 1 and the penalty adds λ / 512 \lambda/512 λ /512 . Together with Lemma 6, this gives the following bounds for the common terms (the endpoint forcing includes its cutoff):
Table 1 (PDF p. 20).
On node cutoff supports, ∣ ϕ η ∣ ≤ 2 R ∗ 2 |\phi_\eta|\le2R_*^2 ∣ ϕ η ∣ ≤ 2 R ∗ 2 and ∥ ∇ ϕ η ∥ ≤ 2 R ∗ \lVert\nabla\phi_\eta\rVert\le2R_* ∥ ∇ ϕ η ∥ ≤ 2 R ∗ ; node blocks are disjoint. The effective endpoint vector has norm at most 2 R ∗ 2R_* 2 R ∗ . The symmetric mixed block has norm equal to that of its rectangular P y Py P y block.
Owned edges. Write H e H_e H e for the unamplified pair Hessian, ∥ H e ∥ ≤ C a = 9 a / 4 \lVert H_e\rVert\le C_a=9a/4 ∥ H e ∥ ≤ C a = 9 a /4 . Degree two and blockwise Cauchy–Schwarz give
∑ i ∥ ∑ e : Σ e = i H e w e ∥ 2 ≤ 2 ∑ e ∥ H e w e ∥ 2 ≤ 4 C a 2 ∥ w ∥ 2 . \sum_i\left\lVert\sum_{e:\,\Sigma_e=i}H_ew_e\right\rVert^2\le2\sum_e\lVert H_ew_e\rVert^2\le4C_a^2\lVert w\rVert^2. i ∑ e : Σ e = i ∑ H e w e 2 ≤ 2 e ∑ ∥ H e w e ∥ 2 ≤ 4 C a 2 ∥ w ∥ 2 .
Amplification by n n n and RMS averaging cost 2 n C a 2\sqrt{n}C_a 2 n C a ; the shared pair barriers cost 2 a n 2a\sqrt{n} 2 a n by the same degree bound. Their sum is ( 13 / 2 ) a n (13/2)a\sqrt{n} ( 13/2 ) a n , uniformly over owner assignments and lengths. Minkowski’s inequality in the component-indexed Hilbert space now proves (56) ; the three P P PP P P rows give (57) . All terms are locally C 1 , 1 C^{1,1} C 1 , 1 . Mollification preserves both component curvature and the stacked-gradient bound for n − 1 / 2 ( ∇ f 1 , … , ∇ f n ) n^{-1/2}(\nabla f_1,\ldots,\nabla f_n) n − 1/2 ( ∇ f 1 , … , ∇ f n ) ; line integration and passage to the limit give the global claims across every hinge and join.
Numerical constants. (28) and λ ≤ δ 0 L \lambda\le\delta_0L λ ≤ δ 0 L bound λ \lambda λ , d / 6 d/6 d /6 and b 2 / ( 12 ν ) b^2/(12\nu) b 2 / ( 12 ν ) by δ 0 L \delta_0L δ 0 L , so K P / L < ( 601 + 6 ⋅ 10 8 + 12 ⋅ 32 ⋅ 128 ⋅ 231000 ) δ 0 < 2 34 δ 0 = 1 / 32 K_P/L<(601+6\cdot10^8+12\cdot32\cdot128\cdot231000)\delta_0<2^{34}\delta_0=1/32 K P / L < ( 601 + 6 ⋅ 1 0 8 + 12 ⋅ 32 ⋅ 128 ⋅ 231000 ) δ 0 < 2 34 δ 0 = 1/32 . Since κ ≥ 64 \kappa\ge64 κ ≥ 64 and ν / L ≤ 1 / 64 + 2 − 16 < 1 / 32 \nu/L\le1/64+2^{-16}<1/32 ν / L ≤ 1/64 + 2 − 16 < 1/32 , the joint bound is
L r o b L ≤ 1 32 + 13000 12 δ 0 / 32 + 47 64 + 788 + 13 / 2 2 16 < 0.79 < 4 5 . \frac{L_{\mathrm{rob}}}{L}\le\frac{1}{32}+13000\sqrt{12\delta_0}/32+\frac{47}{64}+\frac{788+13/2}{2^{16}}<0.79<\frac45. L L rob ≤ 32 1 + 13000 12 δ 0 /32 + 64 47 + 2 16 788 + 13/2 < 0.79 < 5 4 .
This includes the public dual ridge and endpoint correction 47 ν 47\nu 47 ν .
Zeroed axes. Every estimate used only axis norms at most one; zeroing axes preserves the source, forcing and derivative bounds, while node curvature remains at most c I cI c I and edge vectors have norm at most 2 \sqrt{2} 2 . Retain all public radial corrections and the full ambient endpoint forcing. The same curvature and RMS bounds hold for these zero-axis formulas, used as censored objectives in Appendix C.
The Value Function and Stationarity Barrier
We first bound the gradient of the zero-width value function V 0 V_0 V 0 near the end of the stage chain (Lemma 10). A positive width then preserves this barrier, keeps the optimizer inside radius 3 R ∗ / 16 3R_*/16 3 R ∗ /16 , and gives the initial budgets (Lemma 11).
Lemma 10 (Uniform terminal-band barrier). There is a universal c ⋆ > 0 c_\star> 0 c ⋆ > 0 , depending only on the displayed scalar functions and normalized radius 128, such that, for every finite length, dimension, owner assignment, and finite uncompressed primal point,
∣ z M + 1 ∣ ≤ s / 8 ⟹ ∥ ∇ V 0 ( P ) ∥ ≥ c ⋆ λ s . (58) \lvert z_{M+1}\rvert\le s/8 \quad\Longrightarrow\quad\lVert\nabla V_0(P)\rVert\ge c_\star\lambda s. \tag*{(58)} ∣ z M + 1 ∣ ≤ s /8 ⟹ ∥ ∇ V 0 ( P )∥ ≥ c ⋆ λ s . (58) The gradient is in the uncompressed primal coordinates, with arbitrary radii, signs, and orthogonal complements of the axes.
Proof
Outline. After normalization the value is a chain in which block j j j carries a mass q j ∈ [ 0 , 1 ] q_j \in[0,1] q j ∈ [ 0 , 1 ] that can only decrease along the chain. We show that at an approximately stationary point the mass never falls below a fixed level ℓ \ell ℓ , and that a block of mass at least ℓ \ell ℓ cannot be approximately stationary when its last coordinate lies in the terminal band. Step 3 proves both facts for one or two blocks with exactly vanishing residuals; compactness makes them quantitative (Step 4), and induction along the chain concludes (Step 5).
Step 1: residuals in compressed coordinates. Normalize lengths by s s s and values by λ s 2 \lambda s^2 λ s 2 , keeping the same symbols for the normalized quantities, and write R 0 = 128 R_0 = 128 R 0 = 128 . The cap is a diffeomorphism onto the open radius-R 0 R_0 R 0 ball. In compressed coordinates its primal penalty is
π ( r ) = 1 1024 ( r 1 − r 2 / R 0 2 − R 0 ) 2 , 0 ≤ r < R 0 . \pi(r) = \frac{1}{1024}\left(\frac{r}{\sqrt{1-r^2/R_0^2}}-R_0\right)^2,\qquad0 \le r < R_0. π ( r ) = 1024 1 ( 1 − r 2 / R 0 2 r − R 0 ) 2 , 0 ≤ r < R 0 .
Set τ ( r ) = π ′ ( r ) / r \tau(r)=\pi'(r)/r τ ( r ) = π ′ ( r ) / r , with τ ( 0 ) = 0 \tau(0)=0 τ ( 0 ) = 0 . It is continuous, nonnegative, and positive exactly for r > R 0 / 2 r>R_0/\sqrt{2} r > R 0 / 2 . The value is
V = J M + ∑ j q j [ 1 − α ( z j + 1 ) ] Q j + ∑ j π ( ∥ w j ∥ ) , V = J_M + \sum_j q_j\left[1-\alpha(z_{j+1})\right]Q_j + \sum_j \pi(\lVert w_j\rVert), V = J M + j ∑ q j [ 1 − α ( z j + 1 ) ] Q j + j ∑ π (∥ w j ∥) ,
Q j = 6 ( x j − z j + 1 / 2 ) 2 + 6 ∥ A j , ⊥ ∥ 2 + 3 2 ∥ B j + 1 , ⊥ ∥ 2 . Q_j = 6\left(x_j-z_{j+1}/2\right)^2+6\lVert A_{j,\perp}\rVert^2+\frac{3}{2}\lVert B_{j+1,\perp}\rVert^2. Q j = 6 ( x j − z j + 1 /2 ) 2 + 6 ∥ A j , ⊥ ∥ 2 + 2 3 ∥ B j + 1 , ⊥ ∥ 2 .
Define conditional energies by
E j = [ 1 − α ( z j + 1 ) ] Q j + α ( z j + 1 ) E j + 1 , E M + 1 = 0. E_j = \left[1-\alpha(z_{j+1})\right]Q_j+\alpha(z_{j+1})E_{j+1},\qquad E_{M+1}=0. E j = [ 1 − α ( z j + 1 ) ] Q j + α ( z j + 1 ) E j + 1 , E M + 1 = 0.
The quadratic matrix has norm at most 12, so telescoping gives 0 ≤ Q j , E j ≤ E max : = 12 R 0 2 0 \le Q_j, E_j \le E_{\max} := 12R_0^2 0 ≤ Q j , E j ≤ E m a x := 12 R 0 2 , including arbitrary complements. The source coefficient in block j j j is β 1 = 1 \beta_1 = 1 β 1 = 1 and β j = Ψ ( z j ) + Ψ ( − z j ) \beta_j = \Psi(z_j) + \Psi(-z_j) β j = Ψ ( z j ) + Ψ ( − z j ) for j ≥ 2 j \ge2 j ≥ 2 . At a nonterminal connector t = z j + 1 t = z_{j+1} t = z j + 1 , with Y = x j + 1 Y = x_{j+1} Y = x j + 1 , its source derivative is − B ( t , Y ) -B(t,Y) − B ( t , Y ) , where
B ( t , Y ) = Ψ ′ ( − t ) Φ ( − Y ) + Ψ ′ ( t ) Φ ( Y ) ∈ [ 0 , 25 ] . B(t,Y) = \Psi'(-t)\Phi(-Y) + \Psi'(t)\Phi(Y) \in[0,25]. B ( t , Y ) = Ψ ′ ( − t ) Φ ( − Y ) + Ψ ′ ( t ) Φ ( Y ) ∈ [ 0 , 25 ] .
At the terminal connector it is zero. Put q 0 = 1 q_0 = 1 q 0 = 1 . If q j > 0 q_j > 0 q j > 0 , every incoming factor is positive and its connector exceeds 1 / 2 1/2 1/2 ; hence, also for block one,
0 < q j ≤ q j − 1 ≤ 1 , q j C q j − 1 ≤ β j ≤ e q j q j − 1 , C = 23. 0 < q_j \le q_{j-1} \le1,\qquad\frac{q_j}{Cq_{j-1}} \le\beta_j \le\frac{\mathrm e q_j}{q_{j-1}},\qquad C = 23. 0 < q j ≤ q j − 1 ≤ 1 , C q j − 1 q j ≤ β j ≤ q j − 1 e q j , C = 23.
At a current block abbreviate x = x j x = x_j x = x j , t = z j + 1 t = z_{j+1} t = z j + 1 , q = q j q = q_j q = q j , q p r e v = q j − 1 q_{\mathrm{prev}} = q_{j-1} q prev = q j − 1 , H = q [ 1 − α ( t ) ] H = q[1-\alpha(t)] H = q [ 1 − α ( t )] , D = x − t / 2 D = x-t/2 D = x − t /2 , E = E j + 1 E = E_{j+1} E = E j + 1 , and τ = τ ( ∥ w j ∥ ) \tau= \tau(\lVert w_j\rVert) τ = τ (∥ w j ∥) . All compressed residuals are as follows; the suffix enters only through the bounded scalar E E E :
R x = − β Φ ′ ( x ) + 12 H D + τ x , R t = − B − 6 H D + q α ′ ( t ) ( E − Q ) + τ t , ∇ A ⊥ V = ( 12 H + τ ) A ⊥ , ∇ B ⊥ V = ( 3 H + τ ) B ⊥ . \begin{aligned}
R_x &= -\beta\Phi'(x) + 12HD + \tau x,\\
R_t &= -B - 6HD + q\alpha'(t)(E-Q) + \tau t,\\
\nabla_{A_\perp} V &= (12H+\tau)A_\perp,\qquad\nabla_{B_\perp} V = (3H+\tau)B_\perp.
\end{aligned} R x R t ∇ A ⊥ V = − β Φ ′ ( x ) + 12 H D + τ x , = − B − 6 H D + q α ′ ( t ) ( E − Q ) + τ t , = ( 12 H + τ ) A ⊥ , ∇ B ⊥ V = ( 3 H + τ ) B ⊥ .
Step 2: a priori bounds on the block states. Before the inverse-cap penalty, the gradient of each compressed block has norm at most B 0 = 26 + 24 R 0 + ( 23 ⋅ 5 ) ( 12 R 0 2 ) B_0 = 26 + 24R_0 + (23\cdot5)(12R_0^2) B 0 = 26 + 24 R 0 + ( 23 ⋅ 5 ) ( 12 R 0 2 ) , the sum of bounds for the source, the direct quadratic and the gate derivative. If the full normalized uncompressed gradient has norm at most one, ∥ D ρ ∥ ≤ 1 \lVert D\rho\rVert\le1 ∥ D ρ ∥ ≤ 1 implies p ′ ( ∥ P j ∥ ) ≤ B 0 + 1 p'(\lVert P_j\rVert) \le B_0+1 p ′ (∥ P j ∥) ≤ B 0 + 1 , and therefore
∥ P j ∥ ≤ R ‾ : = R 0 + 512 ( B 0 + 1 ) , ∥ w j ∥ ≤ r ‾ : = R 0 R ‾ R 0 2 + R ‾ 2 < R 0 , \lVert P_j\rVert\le\overline R := R_0 + 512(B_0+1),\qquad\lVert w_j\rVert\le\overline r := \frac{R_0\overline R}{\sqrt{R_0^2+\overline R^2}} < R_0, ∥ P j ∥ ≤ R := R 0 + 512 ( B 0 + 1 ) , ∥ w j ∥ ≤ r := R 0 2 + R 2 R 0 R < R 0 ,
σ min ( D ρ ( P j ) ) ≥ σ 0 : = ( 1 + R ‾ 2 / R 0 2 ) − 3 / 2 > 0. \sigma_{\min}(D\rho(P_j)) \ge\sigma_0 := (1+\overline R^2/R_0^2)^{-3/2} > 0. σ m i n ( D ρ ( P j )) ≥ σ 0 := ( 1 + R 2 / R 0 2 ) − 3/2 > 0.
Every compressed block gradient norm is consequently at most ∥ ∇ V ∥ / σ 0 \lVert\nabla V\rVert/\sigma_0 ∥ ∇ V ∥ / σ 0 . Represent a compressed block by ( x , t , ξ , ζ ) (x,t,\xi,\zeta) ( x , t , ξ , ζ ) , with ξ = ∥ A ⊥ ∥ \xi= \lVert A_\perp\rVert ξ = ∥ A ⊥ ∥ and ζ = ∥ B ⊥ ∥ \zeta= \lVert B_\perp\rVert ζ = ∥ B ⊥ ∥ :
ξ , ζ ≥ 0 , r 2 = x 2 + t 2 + ξ 2 + ζ 2 ≤ r ‾ 2 , τ = π ′ ( r ) / r , Q = 6 ( x − t / 2 ) 2 + 6 ξ 2 + 3 2 ζ 2 . (59) \xi,\zeta\ge0,\qquad r^2 = x^2+t^2+\xi^2+\zeta^2 \le\overline r^2,\qquad\tau= \pi'(r)/r,\qquad Q = 6(x-t/2)^2 + 6\xi^2 + \frac{3}{2}\zeta^2. \tag*{(59)} ξ , ζ ≥ 0 , r 2 = x 2 + t 2 + ξ 2 + ζ 2 ≤ r 2 , τ = π ′ ( r ) / r , Q = 6 ( x − t /2 ) 2 + 6 ξ 2 + 2 3 ζ 2 . (59)
Let R = max { ∣ R x ∣ , ∣ R t ∣ , ( 12 H + τ ) ξ , ( 3 H + τ ) ζ } \mathcal R = \max\{|R_x|,|R_t|,(12H+\tau)\xi,(3H+\tau)\zeta\} R = max { ∣ R x ∣ , ∣ R t ∣ , ( 12 H + τ ) ξ , ( 3 H + τ ) ζ } . Allowing both complementary norms only enlarges this dimension-free set. The statements below use the relations
0 < q ≤ q p r e v ≤ 1 , q C q p r e v ≤ β ≤ e q q p r e v . (60) 0 < q \le q_{\mathrm{prev}} \le1,\qquad\frac{q}{Cq_{\mathrm{prev}}} \le\beta\le\frac{\mathrm e q}{q_{\mathrm{prev}}}. \tag*{(60)} 0 < q ≤ q prev ≤ 1 , C q prev q ≤ β ≤ q prev e q . (60)
Fix
q ∗ = ( 24 C R 0 2 ) − 1 , β ∗ = e − 99 , m ∗ = q ∗ β ∗ , ℓ = m ∗ / 4 , E R = e − 4095 , p e x t = e E R / 540. q_* = (24CR_0^2)^{-1},\qquad\beta_* = \mathrm e^{-99},\qquad m_* = q_*\beta_*,\qquad\ell= m_*/4,\qquad E_R = \mathrm e^{-4095},\qquad p_{\mathrm{ext}} = \mathrm e^{E_R}/540. q ∗ = ( 24 C R 0 2 ) − 1 , β ∗ = e − 99 , m ∗ = q ∗ β ∗ , ℓ = m ∗ /4 , E R = e − 4095 , p ext = e E R /540.
Here β ∗ = Ψ ( 11 / 20 ) \beta_* = \Psi(11/20) β ∗ = Ψ ( 11/20 ) , and E R E_R E R bounds Φ ′ ( x ) = e e − x 2 / 2 \Phi'(x) = \sqrt{\mathrm e}\,\mathrm e^{-x^2/2} Φ ′ ( x ) = e e − x 2 /2 when x 2 > R 0 2 / 2 − 1 x^2 > R_0^2/2 - 1 x 2 > R 0 2 /2 − 1 , the region where an aligned block meets the active penalty. In particular 0 < p e x t < e − 4100 < ℓ < m ∗ / 2 < m ∗ < q ∗ < 1 0 < p_{\mathrm{ext}} < \mathrm e^{-4100} < \ell< m_*/2 < m_* < q_* < 1 0 < p ext < e − 4100 < ℓ < m ∗ /2 < m ∗ < q ∗ < 1 .
Step 3: exact local statements. We prove three statements about blocks whose residuals vanish exactly: (a) if ℓ ≤ q ≤ q ∗ \ell\le q \le q_* ℓ ≤ q ≤ q ∗ , q p r e v ≥ ℓ q_{\mathrm{prev}} \ge\ell q prev ≥ ℓ and the residuals of a nonterminal block and its successor vanish, then t > 1 t > 1 t > 1 , so the mass does not drop; (b) if q , q p r e v ≥ q ∗ q, q_{\mathrm{prev}} \ge q_* q , q prev ≥ q ∗ and the residuals of a nonterminal block vanish, then q α ( t ) ≥ m ∗ q\alpha(t) \ge m_* q α ( t ) ≥ m ∗ ; (c) if q , q p r e v ≥ ℓ q, q_{\mathrm{prev}} \ge\ell q , q prev ≥ ℓ and ∣ t ∣ ≤ 1 / 8 |t| \le1/8 ∣ t ∣ ≤ 1/8 , the residuals of the terminal block do not all vanish. We begin with facts about a single block. Assume that all current residuals vanish and q , β > 0 q,\beta> 0 q , β > 0 . For t < 1 t < 1 t < 1 , H > 0 H > 0 H > 0 eliminates the complements. For 0 < t < 1 0 < t < 1 0 < t < 1 the source equation forces x > t / 2 x > t/2 x > t /2 . Indeed, x ≤ 0 x \le0 x ≤ 0 is impossible, and 0 < x ≤ t / 2 0 < x \le t/2 0 < x ≤ t /2 has radius below two, hence τ = 0 \tau= 0 τ = 0 , also impossible. At t = 1 t = 1 t = 1 , τ x = β Φ ′ ( x ) > 0 \tau x = \beta\Phi'(x) > 0 τ x = β Φ ′ ( x ) > 0 forces x , τ > 0 x,\tau> 0 x , τ > 0 and zero complements. For t ≤ 0 t \le0 t ≤ 0 , α = α ′ = 0 \alpha= \alpha' = 0 α = α ′ = 0 and the source equation forces D > 0 D > 0 D > 0 , whence
R t = − B − 6 q D + τ t < 0. R_t = -B - 6qD + \tau t < 0. R t = − B − 6 q D + τ t < 0.
For 0 < t ≤ 1 / 2 0 < t \le1/2 0 < t ≤ 1/2 , B = 0 B = 0 B = 0 and τ t = 6 q D \tau t = 6qD τ t = 6 q D , so τ > 0 \tau> 0 τ > 0 . Every block with 0 < t < 1 0 < t < 1 0 < t < 1 and τ > 0 \tau> 0 τ > 0 satisfies
x > R 0 2 / 2 − 1 , D > x − 1 / 2 > 90 , Φ ′ ( x ) < E R . (61) x > \sqrt{R_0^2/2 - 1}, \qquad D > x - 1/2 > 90, \qquad\Phi'(x) < E_R. \tag*{(61)} x > R 0 2 /2 − 1 , D > x − 1/2 > 90 , Φ ′ ( x ) < E R . (61)
Substitute 12 q ( 1 − α ) D ≤ β Φ ′ ( x ) 12q(1-\alpha)D \le\beta\Phi'(x) 12 q ( 1 − α ) D ≤ β Φ ′ ( x ) and β ≤ e q / q p r e v \beta\le eq/q_{\mathrm{prev}} β ≤ e q / q prev to obtain
q p r e v [ 1 − α ( t ) ] < e E R / 1080 , α ( t ) ≤ 1 / 2 ⟹ q p r e v < p e x t . (62) q_{\mathrm{prev}}[1-\alpha(t)] < eE_R/1080, \qquad\alpha(t) \le1/2 \Longrightarrow q_{\mathrm{prev}} < p_{\mathrm{ext}}. \tag*{(62)} q prev [ 1 − α ( t )] < e E R /1080 , α ( t ) ≤ 1/2 ⟹ q prev < p ext . (62)
At a nonterminal connector t > 1 / 2 t > 1/2 t > 1/2 , the adjacent data are
q ′ = q α ( t ) > 0 , β ′ = Ψ ( t ) + Ψ ( − t ) > 0. (63) q' = q\alpha(t) > 0, \qquad\beta' = \Psi(t) + \Psi(-t) > 0. \tag*{(63)} q ′ = q α ( t ) > 0 , β ′ = Ψ ( t ) + Ψ ( − t ) > 0. (63)
Use ( Y , u , ξ ′ , ζ ′ ) (Y,u,\xi',\zeta') ( Y , u , ξ ′ , ζ ′ ) for the next block, with its actual τ ′ , Q ′ , H ′ = q ′ [ 1 − α ( u ) ] \tau',Q',H' = q'[1-\alpha(u)] τ ′ , Q ′ , H ′ = q ′ [ 1 − α ( u )] and arbitrary farther B ′ ∈ [ 0 , 25 ] B' \in[0,25] B ′ ∈ [ 0 , 25 ] , E ′ ∈ [ 0 , E max ] E' \in[0,E_{\max}] E ′ ∈ [ 0 , E m a x ] . If its residuals vanish, then Y > 0 Y > 0 Y > 0 . For Y ≤ 0 , u ≥ 0 Y \le0,u \ge0 Y ≤ 0 , u ≥ 0 its source residual cannot vanish; for Y ≤ 0 , u < 0 Y \le0,u < 0 Y ≤ 0 , u < 0 that equation forces D ′ = Y − u / 2 > 0 D' = Y-u/2 > 0 D ′ = Y − u /2 > 0 , making the connector residual − B ′ − 6 q ′ D ′ + τ ′ u < 0 -B' - 6q'D' + \tau'u < 0 − B ′ − 6 q ′ D ′ + τ ′ u < 0 .
Proof of (a). Suppose ℓ ≤ q ≤ q ∗ \ell\le q \le q_* ℓ ≤ q ≤ q ∗ , q p r e v ≥ ℓ q_{\mathrm{prev}} \ge\ell q prev ≥ ℓ , and both blocks have zero residuals. We show t > 1 t > 1 t > 1 . Nonpositive t t t is excluded above; 0 < t ≤ 1 / 2 0 < t \le1/2 0 < t ≤ 1/2 forces q p r e v < p e x t < ℓ q_{\mathrm{prev}} < p_{\mathrm{ext}} < \ell q prev < p ext < ℓ . For 1 / 2 < t < 1 1/2 < t < 1 1/2 < t < 1 , Y > 0 Y > 0 Y > 0 gives Φ ( Y ) > 1 \Phi(Y) > 1 Φ ( Y ) > 1 , whereas q α ′ ( t ) ∣ E − Q ∣ ≤ q ∗ C Ψ ′ ( t ) E max = Ψ ′ ( t ) / 2 q\alpha'(t)|E-Q| \le q_*C\Psi'(t)E_{\max} = \Psi'(t)/2 q α ′ ( t ) ∣ E − Q ∣ ≤ q ∗ C Ψ ′ ( t ) E m a x = Ψ ′ ( t ) /2 . The connector equation and D > 0 D > 0 D > 0 imply τ t > Ψ ′ ( t ) / 2 \tau t > \Psi'(t)/2 τ t > Ψ ′ ( t ) /2 . Then (62) and the source equation imply τ ≤ e E R / 90 \tau\le eE_R/90 τ ≤ e E R /90 and Ψ ′ ( t ) ≤ e E R / 45 \Psi'(t) \le eE_R/45 Ψ ′ ( t ) ≤ e E R /45 . Since Ψ ′ = 4 Ψ / ( 2 t − 1 ) 3 \Psi' = 4\Psi/(2t-1)^3 Ψ ′ = 4Ψ/ ( 2 t − 1 ) 3 , we have α ( t ) ≤ C Ψ ′ ( t ) / 4 < 1 / 2 \alpha(t) \le C\Psi'(t)/4 < 1/2 α ( t ) ≤ C Ψ ′ ( t ) /4 < 1/2 , contradicting (63). At t = 1 t = 1 t = 1 , zero complements and the source equation give x > 90 x > 90 x > 90 and τ ≤ e E R / 90 < 1 \tau\le eE_R/90 < 1 τ ≤ e E R /90 < 1 . But α ( 1 ) = 1 \alpha(1) = 1 α ( 1 ) = 1 and α ′ ( 1 ) = 0 \alpha'(1) = 0 α ′ ( 1 ) = 0 give H = 0 H = 0 H = 0 , so the connector equation yields
τ = B ( 1 , Y ) = Ψ ′ ( 1 ) Φ ( Y ) = 4 Φ ( Y ) > 4. \tau= \mathcal{B}(1,Y) = \Psi'(1)\Phi(Y) = 4\Phi(Y) > 4. τ = B ( 1 , Y ) = Ψ ′ ( 1 ) Φ ( Y ) = 4Φ ( Y ) > 4.
Proof of (b). If q , q p r e v ≥ q ∗ q,q_{\mathrm{prev}} \ge q_* q , q prev ≥ q ∗ and only the current nonterminal residuals vanish (with actual B ( t , Y ) \mathcal{B}(t,Y) B ( t , Y ) ), then
q α ( t ) ≥ m ∗ . (64) q\alpha(t) \ge m_*. \tag*{(64)} q α ( t ) ≥ m ∗ . (64)
For α ≥ 1 / 2 \alpha\ge1/2 α ≥ 1/2 this is immediate. Otherwise the single-block facts above force 1 / 2 < t < 1 1/2 < t < 1 1/2 < t < 1 , τ = 0 \tau= 0 τ = 0 , D > 0 D > 0 D > 0 . The source equation with β / q ≥ 1 / C \beta/q \ge1/C β / q ≥ 1/ C gives D ≥ 1 / ( 24 C ) D \ge1/(24C) D ≥ 1/ ( 24 C ) , since 0 < x < 3 / 2 0 < x < 3/2 0 < x < 3/2 and Φ ′ ( x ) > 1 / 2 \Phi'(x) > 1/2 Φ ′ ( x ) > 1/2 when D < 1 D < 1 D < 1 . Since α ( t ) < 1 / 2 \alpha(t) < 1/2 α ( t ) < 1/2 gives H > q / 2 H > q/2 H > q /2 and B ≥ 0 \mathcal{B} \ge0 B ≥ 0 , the connector equation with τ = 0 \tau= 0 τ = 0 gives q α ′ ( t ) ( E − Q ) = B + 6 H D ≥ 3 q D q\alpha'(t)(E-Q) = \mathcal{B} + 6HD \ge3qD q α ′ ( t ) ( E − Q ) = B + 6 H D ≥ 3 q D , hence α ′ ( t ) ≥ D / ( 4 R 0 2 ) ≥ 1 / ( 96 C R 0 2 ) \alpha'(t) \ge D/(4R_0^2) \ge1/(96CR_0^2) α ′ ( t ) ≥ D / ( 4 R 0 2 ) ≥ 1/ ( 96 C R 0 2 ) . But for t ≤ 11 / 20 t \le11/20 t ≤ 11/20 , α ′ ( t ) ≤ 4000 C e − 99 \alpha'(t) \le4000Ce^{-99} α ′ ( t ) ≤ 4000 C e − 99 is smaller. Thus t > 11 / 20 t > 11/20 t > 11/20 and α ( t ) ≥ Ψ ( t ) > e − 99 \alpha(t) \ge\Psi(t) > e^{-99} α ( t ) ≥ Ψ ( t ) > e − 99 . Proof of (c). Here B = E = 0 \mathcal{B}=E=0 B = E = 0 . For q , q p r e v ≥ ℓ q,q_{\mathrm{prev}}\ge\ell q , q prev ≥ ℓ and ∣ t ∣ ≤ 1 / 8 |t|\le1/8 ∣ t ∣ ≤ 1/8 , zero residuals are impossible, since nonpositive t t t was excluded and positive t t t is flat and forces q p r e v < p e x t q_{\mathrm{prev}}<p_{\mathrm{ext}} q prev < p ext .
Step 4: quantitative versions by compactness. Every current block obeys (59) and (60) , with Q , τ Q,\tau Q , τ the actual functions of that block. Define:
K J \mathcal{K}_{J} K J : q ∗ ≤ q ≤ q p r e v ≤ 1 q_{*}\le q\le q_{\mathrm{prev}}\le1 q ∗ ≤ q ≤ q prev ≤ 1 , q α ( t ) ≤ m ∗ / 2 q\alpha(t)\le m_{*}/2 q α ( t ) ≤ m ∗ /2 , 0 ≤ E ≤ E max 0\le E\le E_{\max} 0 ≤ E ≤ E m a x , ∣ Y ∣ ≤ r ˉ |Y|\le\bar r ∣ Y ∣ ≤ r ˉ , and B = B ( t , Y ) \mathcal{B}=\mathcal{B}(t,Y) B = B ( t , Y ) . Its residual is R \mathcal{R} R ; statement (b), in the form of (64) , excludes a zero.
K L \mathcal{K}_{L} K L : ℓ ≤ q ≤ q ∗ \ell\le q\le q_{*} ℓ ≤ q ≤ q ∗ , q ≤ q p r e v ≤ 1 q\le q_{\mathrm{prev}}\le1 q ≤ q prev ≤ 1 , t ≤ 1 t\le1 t ≤ 1 , 0 ≤ E ≤ E max 0\le E\le E_{\max} 0 ≤ E ≤ E m a x , and a second block ( Y , u , ξ ′ , ζ ′ ) (Y,u,\xi',\zeta') ( Y , u , ξ ′ , ζ ′ ) satisfying the same compact radius condition. Retain B = B ( t , Y ) \mathcal{B}=\mathcal{B}(t,Y) B = B ( t , Y ) and (63) ; allow independent E ′ ∈ [ 0 , E max ] E'\in[0,E_{\max}] E ′ ∈ [ 0 , E m a x ] , B ′ ∈ [ 0 , 25 ] \mathcal{B}'\in[0,25] B ′ ∈ [ 0 , 25 ] . The next residuals are
R Y = − β ′ Φ ′ ( Y ) + 12 H ′ ( Y − u / 2 ) + τ ′ Y , R u = − B ′ − 6 H ′ ( Y − u / 2 ) + q ′ α ′ ( u ) ( E ′ − Q ′ ) + τ ′ u , R ξ ′ = ( 12 H ′ + τ ′ ) ξ ′ , R ζ ′ = ( 3 H ′ + τ ′ ) ζ ′ , H ′ = q ′ [ 1 − α ( u ) ] . \begin{aligned}
R_Y&=-\beta'\Phi'(Y)+12H'(Y-u/2)+\tau'Y,\\
R_u&=-\mathcal{B}'-6H'(Y-u/2)+q'\alpha'(u)(E'-Q')+\tau'u,\\
R_{\xi'}&=(12H'+\tau')\xi',\qquad R_{\zeta'}=(3H'+\tau')\zeta',\qquad H'=q'[1-\alpha(u)].
\end{aligned} R Y R u R ξ ′ = − β ′ Φ ′ ( Y ) + 12 H ′ ( Y − u /2 ) + τ ′ Y , = − B ′ − 6 H ′ ( Y − u /2 ) + q ′ α ′ ( u ) ( E ′ − Q ′ ) + τ ′ u , = ( 12 H ′ + τ ′ ) ξ ′ , R ζ ′ = ( 3 H ′ + τ ′ ) ζ ′ , H ′ = q ′ [ 1 − α ( u )] .
Its residual is the maximum of R \mathcal{R} R and the absolute values of these four quantities. For t ≤ 1 / 2 t\le1/2 t ≤ 1/2 the current flat exclusions apply; otherwise next-block positivity and statement (a), including t = 1 t=1 t = 1 , rule out a zero. The coefficient β ′ = Ψ ( t ) + Ψ ( − t ) \beta'=\Psi(t)+\Psi(-t) β ′ = Ψ ( t ) + Ψ ( − t ) is defined also at q ′ = 0 q'=0 q ′ = 0 .
K T \mathcal{K}_{T} K T : ℓ ≤ q ≤ q p r e v ≤ 1 \ell\le q\le q_{\mathrm{prev}}\le1 ℓ ≤ q ≤ q prev ≤ 1 , ∣ t ∣ ≤ 1 / 8 |t|\le1/8 ∣ t ∣ ≤ 1/8 and B = E = 0 \mathcal{B}=E=0 B = E = 0 ; by (c), R \mathcal{R} R has no zero.
These are fixed-dimensional compact sets. Masses in denominators are bounded away from zero, r ˉ < R 0 \bar r<R_{0} r ˉ < R 0 , and residuals are continuous at gate joins, penalty onset and zero complements. They cover every actual selected window, including a terminal next block through B ′ = E ′ = 0 \mathcal{B}'=E'=0 B ′ = E ′ = 0 ; independent farther data only enlarge the sets. Let ϵ J , ϵ L , ϵ T > 0 \epsilon_J,\epsilon_L,\epsilon_T>0 ϵ J , ϵ L , ϵ T > 0 be their residual minima, with min ∅ = 1 \min\varnothing=1 min ∅ = 1 ; they are positive by the no-zero arguments. Thus residual below ϵ J \epsilon_J ϵ J forces q α ( t ) > m ∗ / 2 q\alpha(t)>m_{*}/2 q α ( t ) > m ∗ /2 in the high-mass range, two-block residual below ϵ L \epsilon_L ϵ L forces t > 1 t>1 t > 1 in the low-mass range, and the terminal-band residual is at least ϵ T \epsilon_T ϵ T . Set
ϵ l o c = min { 1 , ϵ J , ϵ L , ϵ T } , c ⋆ = 1 2 min { 1 , σ 0 ϵ l o c } > 0. \epsilon_{\mathrm{loc}}=\min\{1,\epsilon_J,\epsilon_L,\epsilon_T\},\qquad c_{\star}=\frac{1}{2}\min\{1,\sigma_{0}\epsilon_{\mathrm{loc}}\}>0. ϵ loc = min { 1 , ϵ J , ϵ L , ϵ T } , c ⋆ = 2 1 min { 1 , σ 0 ϵ loc } > 0.
Step 5: induction along the chain. Suppose ∥ ∇ V ∥ < c ⋆ \|\nabla V\|<c_{\star} ∥∇ V ∥ < c ⋆ and ∣ z M + 1 ∣ ≤ 1 / 8 |z_{M+1}|\le1/8 ∣ z M + 1 ∣ ≤ 1/8 . Radius control makes every compressed residual smaller than ϵ l o c \epsilon_{\mathrm{loc}} ϵ loc . We inductively prove q j ≥ ℓ q_j\ge\ell q j ≥ ℓ , starting from q 1 = 1 q_1=1 q 1 = 1 . For j < M j<M j < M , if q j > q ∗ q_j>q_{*} q j > q ∗ , separation of K J \mathcal{K}_{J} K J gives q j + 1 > m ∗ / 2 > ℓ q_{j+1}>m_{*}/2>\ell q j + 1 > m ∗ /2 > ℓ . Otherwise ℓ ≤ q j ≤ q ∗ \ell\le q_j\le q_{*} ℓ ≤ q j ≤ q ∗ ; since q j − 1 ≥ q j q_{j-1}\ge q_j q j − 1 ≥ q j , separation of K L \mathcal{K}_{L} K L gives z j + 1 > 1 z_{j+1}>1 z j + 1 > 1 , so q j + 1 = q j α ( z j + 1 ) = q j ≥ ℓ q_{j+1}=q_j\alpha(z_{j+1})=q_j\ge\ell q j + 1 = q j α ( z j + 1 ) = q j ≥ ℓ . Thus q M − 1 ≥ q M ≥ ℓ q_{M-1}\ge q_M\ge\ell q M − 1 ≥ q M ≥ ℓ (with q 0 = q 1 = 1 q_0=q_1=1 q 0 = q 1 = 1 for M = 1 M=1 M = 1 ), and the terminal band contradicts separation of K T \mathcal{K}_{T} K T . Restoring units multiplies the primal gradient by λ s \lambda s λ s , proving the result. ■
Positive Width, Exact Optimizer Localization, and Initialization
Fix the constant c ⋆ c_{\star} c ⋆ of the preceding barrier proof. Take
η 0 = c ⋆ λ s μ 0 12 A 1 a b K , η = min { η 0 , R ∗ / ( 16 n ) } > 0 , A 1 = 2433 , K = M m . (65) \eta_0=\frac{c_{\star}\lambda s\mu_0}{12A_1ab\sqrt{K}},\qquad\eta=\min\{\eta_0,R_{*}/(16\sqrt{n})\}>0,\qquad A_1=2433,\qquad K=Mm. \tag*{(65)} η 0 = 12 A 1 ab K c ⋆ λ s μ 0 , η = min { η 0 , R ∗ / ( 16 n )} > 0 , A 1 = 2433 , K = M m . (65)
The width meets both the perturbation budget and the curvature condition of Lemma 8(iii), without changing the number of stages.
Lemma 11 (Uniform perturbation and initial budgets). For all finite raw P P P , the uncut maximizer of P ( P ) + b ⟨ A ( P ) , Y ∂ ⟩ − E η ( y ) \mathcal{P}(P) + b\langle A(P), Y_{\partial}\rangle- \mathcal{E}_{\eta}(y) P ( P ) + b ⟨ A ( P ) , Y ∂ ⟩ − E η ( y ) has every relevant node, edge tuple, and complete endpoint tuple strictly inside radius 3 R ∗ / 16 3R_*/16 3 R ∗ /16 . The node and endpoint cutoffs χ ∗ \chi_* χ ∗ equal one, the radial edge width equals the same constant η \eta η , and every radial barrier vanishes on a neighborhood of this point. It is therefore the unique maximizer of the modified mean as well. Its value function V η V_{\eta} V η obeys
∥ ∇ V η ( P ) − ∇ V 0 ( P ) ∥ ≤ c ∗ λ s / 4 , (66) \lVert\nabla V_{\eta}(P)-\nabla V_{0}(P)\rVert\leq c_*\lambda s/4, \tag*{(66)} ∥ ∇ V η ( P ) − ∇ V 0 ( P )∥ ≤ c ∗ λ s /4 , (66) ∣ V η ( P ) − V 0 ( P ) ∣ ≤ ( 32 c ∗ / A 1 ) λ s 2 < 0.014 λ s 2 . (67) \lvert V_{\eta}(P)-V_{0}(P)\rvert\leq(32c_*/A_1)\lambda s^2<0.014\lambda s^2. \tag*{(67)} ∣ V η ( P ) − V 0 ( P )∣ ≤ ( 32 c ∗ / A 1 ) λ s 2 < 0.014 λ s 2 . (67) Consequently
∣ z M + 1 ∣ ≤ s / 8 ⟹ ∥ ∇ V η ( P ) ∥ ≥ 3 4 c ∗ λ s = 6 ϵ . (68) \lvert z_{M+1}\rvert\leq s/8\quad\Longrightarrow\quad\lVert\nabla V_{\eta}(P)\rVert\geq\frac{3}{4}c_*\lambda s=6\epsilon. \tag*{(68)} ∣ z M + 1 ∣ ≤ s /8 ⟹ ∥ ∇ V η ( P )∥ ≥ 4 3 c ∗ λ s = 6 ϵ . (68) At the supplied origin y ∗ ( 0 ) = 0 y^*(0)=0 y ∗ ( 0 ) = 0 ,
∥ ∇ F η ( 0 , 0 ) ∥ 2 = e λ 2 s 2 , (69) \lVert\nabla F_{\eta}(0,0)\rVert^2=e\lambda^2s^2, \tag*{(69)} ∥ ∇ F η ( 0 , 0 ) ∥ 2 = e λ 2 s 2 , (69) V η ( 0 ) − inf P V η ( P ) ≤ ( 5 e M + 0.014 ) λ s 2 , (70) V_{\eta}(0)-\inf_P V_{\eta}(P)\leq(5eM+0.014)\lambda s^2, \tag*{(70)} V η ( 0 ) − P inf V η ( P ) ≤ ( 5 e M + 0.014 ) λ s 2 , (70) G 0 : = V η ( 0 ) − inf P F η ( P , 0 ) ≤ 5 e M λ s 2 + d R 2 < ∞ . (71) G_0:=V_{\eta}(0)-\inf_P F_{\eta}(P,0)\leq5eM\lambda s^2+dR^2<\infty. \tag*{(71)} G 0 := V η ( 0 ) − P inf F η ( P , 0 ) ≤ 5 e M λ s 2 + d R 2 < ∞. (71) The primal–dual gap G 0 G_0 G 0 is independent of η \eta η .
Proof. Localization. The clipping identity ϕ η ′ ( t ) = t − clip η ( t ) \phi'_{\eta}(t)=t-\operatorname{clip}_{\eta}(t) ϕ η ′ ( t ) = t − clip η ( t ) and ∥ D U ∥ ≤ 2 \lVert DU\rVert\leq2 ∥ D U ∥ ≤ 2 give uniformly on the entire dual space
∥ ∇ E η − ∇ E 0 ∥ ≤ ( c + 2 a ) η K ≤ 3 a η K . \lVert\nabla\mathcal{E}_{\eta}-\nabla\mathcal{E}_{0}\rVert\leq(c+2a)\eta\sqrt{K}\leq3a\eta\sqrt{K}. ∥ ∇ E η − ∇ E 0 ∥ ≤ ( c + 2 a ) η K ≤ 3 a η K .
Strong monotonicity of either uncut energy therefore yields
∥ y η u n c u t ( P ) − y 0 ( P ) ∥ ≤ 3 a η K μ 0 ≤ c ∗ λ s 4 A 1 b . (72) \lVert y_{\eta}^{\mathrm{uncut}}(P)-y_0(P)\rVert\leq\frac{3a\eta\sqrt{K}}{\mu_0}\leq\frac{c_*\lambda s}{4A_1b}. \tag*{(72)} ∥ y η uncut ( P ) − y 0 ( P )∥ ≤ μ 0 3 a η K ≤ 4 A 1 b c ∗ λ s . (72)
The ratio of this error to b R / ν bR/\nu b R / ν is at most c ∗ / ( 6144 A 1 ) < 1 c_*/(6144A_1)<1 c ∗ / ( 6144 A 1 ) < 1 , since R = 128 s R=128s R = 128 s and b 2 / ν ≥ 12 λ b^2/\nu\geq12\lambda b 2 / ν ≥ 12 λ . Lemma 7 now puts every relevant node, pair and complete endpoint tuple strictly within 3 b R / ν = 3 R ∗ / 16 < R ∗ / 2 3bR/\nu=3R_*/16<R_*/2 3 b R / ν = 3 R ∗ /16 < R ∗ /2 . All node and endpoint cutoffs χ ∗ \chi_* χ ∗ are identically one, every edge threshold is exactly η \eta η , and all radial barriers vanish on a neighborhood of this point. Its uncut first-order condition is thus also the localized first-order condition. Global mean strong concavity makes it the unique global localized maximizer.
Gradients. Both maxima are unique and the objective is continuously differentiable, including at all hinges, so Danskin’s theorem gives the value-function gradients. Their exact difference is
b [ D A ( P ) ] ⊤ Π ∂ [ y η ( P ) − y 0 ( P ) ] , b[DA(P)]^\top\Pi_{\partial}[y_{\eta}(P)-y_0(P)], b [ D A ( P ) ] ⊤ Π ∂ [ y η ( P ) − y 0 ( P )] ,
where Π ∂ \Pi_{\partial} Π ∂ has norm one. The forcing bound and (72) prove (66) , and the preceding barrier proves (68) .
Values. Since ∣ ϕ η ( t ) − t 2 / 2 ∣ ≤ η ∣ t ∣ \lvert\phi_{\eta}(t)-t^2/2\rvert\leq\eta\lvert t\rvert ∣ ϕ η ( t ) − t 2 /2 ∣ ≤ η ∣ t ∣ , we have ∣ E η ( y ) − E 0 ( y ) ∣ ≤ 3 a η K ∥ y ∥ \lvert\mathcal{E}_{\eta}(y)-\mathcal{E}_{0}(y)\rvert\leq3a\eta\sqrt{K}\lVert y\rVert ∣ E η ( y ) − E 0 ( y )∣ ≤ 3 a η K ∥ y ∥ . The gradients of both energies vanish at zero; strong monotonicity therefore bounds each uncut optimizer’s norm by b R / μ 0 bR/\mu_0 b R / μ 0 . Evaluating either variational problem at the other’s optimizer proves (67) . Initial budgets. At the origin A = 0 \mathcal{A}=0 A = 0 and the unique maximizer is zero; the only nonzero initial derivative is − e λ s U 1 , 1 -\sqrt{e}\lambda sU_{1,1} − e λ s U 1 , 1 in the first primal subblock. The values with positive and zero width coincide there. Since J M ≥ − 5 e M J_M \ge-5eM J M ≥ − 5 e M , the nonnegative quadratic and penalty terms in (50) , combined with (67) , give (70) . Finally
F η ( P , 0 ) = λ s 2 J M ( P ) − d ∥ A ( P ) ∥ 2 + ∑ j p ( P j ) , F_\eta(P,0)=\lambda s^2J_M(P)-d\lVert\mathcal{A}(P)\rVert^2+\sum_j p(P_j), F η ( P , 0 ) = λ s 2 J M ( P ) − d ∥ A ( P ) ∥ 2 + j ∑ p ( P j ) ,
which is width-independent and bounded below by − 5 e M λ s 2 − d R 2 -5eM\lambda s^2-dR^2 − 5 e M λ s 2 − d R 2 . Its value at the origin is negative. This proves (71) .
Normalization and class membership. All realizations meet the global smoothness and mean strong concavity requirements by Lemma 9, since μ 0 ≥ μ \mu_0\ge\mu μ 0 ≥ μ . By (26) , the primal gap is at most ( 5 e 32 + 0.896 4096 ) Δ < Δ / 2 (\frac{5e}{32}+\frac{0.896}{4096})\Delta<\Delta/2 ( 32 5 e + 4096 0.896 ) Δ < Δ/2 . The initial gradient satisfies e λ 2 s 2 = 64 e ε 2 / c ⋆ 2 ≤ e λ Δ / 64 < 2 L Δ e\lambda^2s^2=64e\varepsilon^2/c_\star^2\le e\lambda\Delta/64<2L\Delta e λ 2 s 2 = 64 e ε 2 / c ⋆ 2 ≤ e λ Δ/64 < 2 L Δ . Only in the high branch, d < 4 λ d<4\lambda d < 4 λ gives the additional check G 0 ≤ ( 5 e 32 + 1024 ) Δ < 1025 Δ G_0\le(\frac{5e}{32}+1024)\Delta<1025\Delta G 0 ≤ ( 32 5 e + 1024 ) Δ < 1025Δ . Thus the hard family also has finite initial primal–dual gap.
Unrestricted Oracle Lower Bounds
A public convexification coordinate. For exact oracle access, augment the components already constructed by one common public primal coordinate:
f ~ i , η ( P , ξ p u b , y ) = f i , η ( P , y ) + B ( 1 − cos ξ p u b ) , B = L / 8. (73) \widetilde f_{i,\eta}(P,\xi_{\mathrm{pub}},y)=f_{i,\eta}(P,y)+B(1-\cos\xi_{\mathrm{pub}}),\qquad B=L/8. \tag*{(73)} f i , η ( P , ξ pub , y ) = f i , η ( P , y ) + B ( 1 − cos ξ pub ) , B = L /8. (73)
We keep V η ( P ) V_\eta(P) V η ( P ) for the value studied above; the augmented value and gradient satisfy
V ~ η ( P , ξ p u b ) = V η ( P ) + B ( 1 − cos ξ p u b ) , ∥ ∇ V ~ η ( P , ξ p u b ) ∥ 2 = ∥ ∇ V η ( P ) ∥ 2 + B 2 sin 2 ξ p u b . \widetilde V_\eta(P,\xi_{\mathrm{pub}})=V_\eta(P)+B(1-\cos\xi_{\mathrm{pub}}),\qquad\lVert\nabla\widetilde V_\eta(P,\xi_{\mathrm{pub}})\rVert^2=\lVert\nabla V_\eta(P)\rVert^2+B^2\sin^2\xi_{\mathrm{pub}}. V η ( P , ξ pub ) = V η ( P ) + B ( 1 − cos ξ pub ) , ∥ ∇ V η ( P , ξ pub ) ∥ 2 = ∥ ∇ V η ( P ) ∥ 2 + B 2 sin 2 ξ pub .
The added term is nonnegative and vanishes with its gradient at zero. Both initial gaps, the initial gradient and the dual maximizer are unchanged. Orthogonality gives joint RMS constant max { L r o b , B } < 4 L / 5 \max\{L_{\mathrm{rob}},B\}<4L/5 max { L rob , B } < 4 L /5 , and one extra primal dimension. All preceding widths, constants and value-transfer differences are unchanged. For every finite ( P , ξ p u b ) (P,\xi_{\mathrm{pub}}) ( P , ξ pub ) ,
∣ z M + 1 ( P ) ∣ ≤ s / 8 ⟹ ∥ ∇ V ~ η ( P , ξ p u b ) ∥ ≥ 6 ε . (74) |z_{M+1}(P)|\le s/8\quad\Longrightarrow\quad\lVert\nabla\widetilde V_\eta(P,\xi_{\mathrm{pub}})\rVert\ge6\varepsilon. \tag*{(74)} ∣ z M + 1 ( P ) ∣ ≤ s /8 ⟹ ∥ ∇ V η ( P , ξ pub )∥ ≥ 6 ε . (74)
Proximal notation. At a finite center w ˉ = ( P ˉ , ξ ˉ p u b , y ˉ ) \bar w=(\bar P,\bar\xi_{\mathrm{pub}},\bar y) w ˉ = ( P ˉ , ξ ˉ pub , y ˉ ) and inverse step τ = 1 / γ > 0 \tau=1/\gamma>0 τ = 1/ γ > 0 , the regularized objective is
H ~ i = f ~ i , η + τ 2 ( ∥ P − P ˉ ∥ 2 + ∣ ξ p u b − ξ ˉ p u b ∣ 2 − ∥ y − y ˉ ∥ 2 ) . (75) \widetilde H_i=\widetilde f_{i,\eta}+\frac{\tau}{2}\left(\lVert P-\bar P\rVert^2+|\xi_{\mathrm{pub}}-\bar\xi_{\mathrm{pub}}|^2-\lVert y-\bar y\rVert^2\right). \tag*{(75)} H i = f i , η + 2 τ ( ∥ P − P ˉ ∥ 2 + ∣ ξ pub − ξ ˉ pub ∣ 2 − ∥ y − y ˉ ∥ 2 ) . (75)
Exact Proximal Localization and Oracle Complexity
Lemma 12 (Public legality and unique responses). The exact legal domain of (75) is τ ∈ [ B , ∞ ) \tau\in[B,\infty) τ ∈ [ B , ∞ ) , common to all components, hidden realizations and centers. Every legal response exists and is unique. The same conclusions hold for every consistent zero-axis formula that keeps all public terms; its response is measurable in its coefficients and center.
Proof. The scalar curvature is τ + B cos ξ p u b \tau+ B \cos\xi_{\mathrm{pub}} τ + B cos ξ pub , negative at ξ p u b = π \xi_{\mathrm{pub}} = \pi ξ pub = π when τ < B \tau< B τ < B . For τ ≥ B \tau\ge B τ ≥ B , its derivative τ ( ξ p u b − ξ ˉ p u b ) + B sin ξ p u b \tau(\xi_{\mathrm{pub}}-\bar{\xi}_{\mathrm{pub}})+B\sin\xi_{\mathrm{pub}} τ ( ξ pub − ξ ˉ pub ) + B sin ξ pub tends to ± ∞ \pm\infty ± ∞ and is strictly increasing. At τ = B \tau= B τ = B , its increment on [ ξ 1 , ξ 2 ] [\xi_1,\xi_2] [ ξ 1 , ξ 2 ] , ξ 1 < ξ 2 \xi_1 < \xi_2 ξ 1 < ξ 2 , is B ∫ ξ 1 ξ 2 ( 1 + cos t ) d t > 0 B\int_{\xi_1}^{\xi_2}(1+\cos t)\,\mathrm{d}t > 0 B ∫ ξ 1 ξ 2 ( 1 + cos t ) d t > 0 . Thus the scalar has a unique minimizer, measurably given by the inverse of ξ p u b + ( B / τ ) sin ξ p u b = ξ ˉ p u b \xi_{\mathrm{pub}}+(B/\tau)\sin\xi_{\mathrm{pub}}=\bar{\xi}_{\mathrm{pub}} ξ pub + ( B / τ ) sin ξ pub = ξ ˉ pub . It is not strongly convex at τ = B \tau= B τ = B .
For the remaining variables z = ( P , y ) z=(P,y) z = ( P , y ) , let H H H be the corresponding regularized objective and put T = ( ∇ P H , − ∇ y H ) T=(\nabla_P H,-\nabla_y H) T = ( ∇ P H , − ∇ y H ) . Lemma 9 gives strong monotonicity with m 0 = min { τ − K P , μ 0 + τ } > 0 m_0=\min\{\tau-K_P,\mu_0+\tau\}>0 m 0 = min { τ − K P , μ 0 + τ } > 0 and a Lipschitz bound M 0 = n L + τ M_0=\sqrt{n}L+\tau M 0 = n L + τ . Mixed Hessian terms cancel in the monotonicity quadratic form. For a 0 = m 0 / M 0 2 a_0=m_0/M_0^2 a 0 = m 0 / M 0 2 ,
∥ ( I − a 0 T ) z − ( I − a 0 T ) z ′ ∥ 2 ≤ ( 1 − m 0 2 M 0 2 ) ∥ z − z ′ ∥ 2 . \left\|(I-a_0T)z-(I-a_0T)z'\right\|^2\le\left(1-\frac{m_0^2}{M_0^2}\right)\left\|z-z'\right\|^2. ∥ ( I − a 0 T ) z − ( I − a 0 T ) z ′ ∥ 2 ≤ ( 1 − M 0 2 m 0 2 ) ∥ z − z ′ ∥ 2 .
Contraction gives a unique zero, hence a unique global saddle by convexity–concavity. Iteration from zero also makes this root measurable. The same bounds hold for zeroed axes by Lemma 9. Combining this with the scalar minimizer proves all claims, including those at the legal boundary.
Full-Space Localization by Reflection
We grant the first source axis and maintain a serial frontier through E = M ( m − 1 ) E=M(m-1) E = M ( m − 1 ) fixed owned edges. A component-i i i call tests the frontier: a hit reveals its successor axis and continues with i i i ; completing a path also grants the next source axis. The first mismatch reveals only that the owner is not i i i and ends the run. Granting this entire run can only help the algorithm.
After exposure we retain full ambient blocks for revealed axes. For an unrevealed interior dual block retain its current-center span; for an unrevealed endpoint retain the span of its current dual and paired primal centers on both sides of the public identity. Their product is S = S P × R ξ p u b × S y S=S_P\times\mathbb{R}_{\xi_{\mathrm{pub}}}\times S_y S = S P × R ξ pub × S y , with each unrevealed span S b S_b S b of dimension at most two. We rebuild it at each call and replace each unrevealed axis by zero in every primal and dual occurrence of the full-space component, retaining all public norms, corrections and identity maps. Let j j j be the stage whose path contains the frontier. Its outgoing axis is unrevealed, so its gate factor becomes b + ( 0 ) = 1 b_+(0)=1 b + ( 0 ) = 1 , and we retain it. We call the resulting function f ~ i 0 \tilde{f}^{\,0}_i f ~ i 0 . The first missing edge is absent because its owner is known to differ from i i i ; every later edge has two zero axes and is identically zero, regardless of its owner, and unrevealed node terms vanish. Later sources have Ψ ( ± 0 ) = 0 \Psi(\pm0)=0 Ψ ( ± 0 ) = 0 , and the forcing of every stage after j j j has an incoming factor a + ( 0 ) = 0 a_+(0)=0 a + ( 0 ) = 0 . The ambient forcing of stage j j j and its square are retained. Thus the censored full-space formula reads only the exposed prefix.
Lemma 13 (Exact localization of the unique proximal response). For every legal call, the unique full-space censored saddle lies in S S S . If the actual axes obey
∥ Proj S b U b ∥ ≤ δ , δ = min { s 8 R , η 8 R ∗ } , (76) \left\|\operatorname{Proj}_{S_b}U_b\right\|\le\delta,\qquad\delta=\min\left\{\frac{s}{8R},\frac{\eta}{8R_*}\right\}, \tag*{(76)} Proj S b U b ≤ δ , δ = min { 8 R s , 8 R ∗ η } , (76) then it is also the unique actual proximal response. Actual and censored center values and full joint gradients agree as well.
Proof. The full-space censored saddle exists uniquely by Lemma 12. Reflect each unrevealed block through S b S_b S b , using the same reflection on paired primal and dual endpoints; fix all revealed blocks and ξ p u b \xi_{\mathrm{pub}} ξ pub . Norms and radial terms are invariant, caps and the full ambient forcing are equivariant, the paired endpoint inner product is invariant, and zeroed projections stay zero. The center is fixed, so this reflection maps the censored saddle to another censored saddle. Uniqueness therefore places it in S S S .
Under (77), capped primal projections on the entire S S S have magnitude at most R δ ≤ s / 8 R\delta\le s/8 R δ ≤ s /8 . The profile satisfies ρ ( u ) ≥ u / 2 \rho(u) \ge u/2 ρ ( u ) ≥ u /2 for all u ≥ 0 u \ge0 u ≥ 0 , since on [ 1 , 2 ] [1,2] [ 1 , 2 ] this is equivalent to u 2 − 3 u + 3 ≥ 0 u^2-3u+3 \ge0 u 2 − 3 u + 3 ≥ 0 and the other intervals are immediate. For a hidden–hidden edge with r = ∥ Y e ∥ > 0 r = \lVert Y_e\rVert> 0 r = ∥ Y e ∥ > 0 , every point of the retained subspace therefore satisfies
∣ t e ∣ ≤ δ ( ∥ y v ∥ + ∥ y w ∥ ) ≤ 2 δ r ≤ 2 η r 8 R ∗ ≤ η r 2 R ∗ ≤ h η ( r ) . |t_e| \le\delta(\lVert y_v\rVert+\lVert y_w\rVert) \le\sqrt{2}\delta r \le\frac{\sqrt{2}\eta r}{8R_*} \le\frac{\eta r}{2R_*} \le h_\eta(r). ∣ t e ∣ ≤ δ (∥ y v ∥ + ∥ y w ∥) ≤ 2 δ r ≤ 8 R ∗ 2 η r ≤ 2 R ∗ η r ≤ h η ( r ) .
At r = 0 r=0 r = 0 the margin is η > 0 \eta>0 η > 0 . Thus the edge and its full gradient vanish on an open neighborhood of every such point, at every radius. For an unrevealed node with ∥ y v ∥ ≤ 2 R ∗ \lVert y_v\rVert\le2R_* ∥ y v ∥ ≤ 2 R ∗ , its hidden projection is at most 2 R ∗ δ ≤ η / 4 2R_*\delta\le\eta/4 2 R ∗ δ ≤ η /4 ; outside that radius its cutoff vanishes. The strict margin of the flat region also covers the cutoff boundary. Hence all discarded terms are locally zero in the full space. The actual and censored functions have the same values and full gradients on S S S . The censored full-space saddle is therefore stationary for the actual convex–concave objective, hence is its unique saddle. The center itself belongs to S S S , proving exact scalar and joint-gradient equality there as well. ■
A Total Ideal Transcript and Uniform Subspace Hiding
Fix the finite comparison horizon
E = M ( m − 1 ) , H = E ( n − 1 ) , q P = ⌊ H / 16 ⌋ , p = 1 / 64. (77) E=M(m-1),\qquad H=E(n-1),\qquad q_P=\lfloor H/16\rfloor,\qquad p=1/64. \tag*{(77)} E = M ( m − 1 ) , H = E ( n − 1 ) , q P = ⌊ H /16 ⌋ , p = 1/64. (77)
We compare the algorithm with an ideal process that sees only what the owner runs have exposed. The instance and the private seed are sampled before execution. The algorithm’s query, stopping and output rules are defined on the histories the real oracle can produce. We fix a public extension of them, measurable and using the same seed, to every history of length at most q P q_P q P the ideal process can produce, with valid indices, finite centers and outputs, and finite positive inverse steps; the extension need not terminate. At each call the ideal process chooses its query from the past, exposes the owner run, and returns the full-space censored tuple; if τ < B \tau< B τ < B , it returns a fixed dummy instead. This public fallback reads no hidden data and reveals nothing about the real oracle. All other responses are unique and measurable by Lemma 12. The algorithm is legal almost surely on each fixed instance; averaging over the prior and taking a countable union over calls gives one null event over instances and seeds, so the owner and Haar laws are never conditioned on legality or on projection success.
Conditional on the ideal history and current run exposure, every unexposed axis is independent Haar. Its current-center span is measurable from the prefix and private seed and has dimension at most two; earlier projection successes are not conditioned upon (cf. [2 Appendix B.1.2] ). For Haar U ∈ S D − 1 U\in S^{D-1} U ∈ S D − 1 , rotational symmetry and the beta integral give E ⟨ U , v ⟩ 2 r = ( 2 r − 1 ) ! ! ∥ v ∥ 2 r / [ D ( D + 2 ) ⋯ ( D + 2 r − 2 ) ] \mathbb{E}\langle U,v\rangle^{2r}=(2r-1)!!\|v\|^{2r}/[D(D+2)\cdots(D+2r-2)] E ⟨ U , v ⟩ 2 r = ( 2 r − 1 )!! ∥ v ∥ 2 r / [ D ( D + 2 ) ⋯ ( D + 2 r − 2 )] and zero odd moments. Bounding denominators by D r D^r D r in the exponential series yields E e θ ⟨ U , v ⟩ ≤ e θ 2 ∥ v ∥ 2 / ( 2 D ) \mathbb{E}e^{\theta\langle U,v\rangle}\le e^{\theta^2\|v\|^2/(2D)} E e θ ⟨ U , v ⟩ ≤ e θ 2 ∥ v ∥ 2 / ( 2 D ) , so Chernoff gives P { ∣ ⟨ U , v ⟩ ∣ > t } ≤ 2 e − D t 2 / ( 2 ∥ v ∥ 2 ) \mathbb{P}\{|\langle U,v\rangle|>t\}\le2e^{-Dt^2/(2\|v\|^2)} P { ∣ ⟨ U , v ⟩ ∣ > t } ≤ 2 e − D t 2 / ( 2∥ v ∥ 2 ) . A 1 / 2 1/2 1/2 -net of a d d d -dimensional unit sphere has at most 5 d 5^d 5 d points ([20 Corollary 4.2.13] ); net comparison ([20 Lemma 4.4.1] ) therefore gives, for any fixed subspace S S S ,
P { ∥ Proj S U ∥ > δ } ≤ 2 5 d exp ( − D δ 2 / 8 ) . (78) \mathbb{P}\{\|\operatorname{Proj}_{S}U\|>\delta\}\le2\,5^d\exp(-D\delta^2/8). \tag*{(78)} P { ∥ Proj S U ∥ > δ } ≤ 2 5 d exp ( − D δ 2 /8 ) . (78)
Conditioning on the ideal history and averaging extends this to the random current span.
We include the actual proposed output when stopping by q P q_P q P , and a fixed virtual output otherwise. In each unrevealed endpoint coordinate block, its capped primal output subblock spans at most one dimension and is determined by the ideal history, and we test that span separately. There are at most K ( q P + 1 ) K(q_P+1) K ( q P + 1 ) coordinate block tests in total, with d ≤ 2 d\le2 d ≤ 2 , including the output when q P = 0 q_P=0 q P = 0 . With
H n = max { 1 , R 2 / s 2 , R ∗ 2 / η 2 } , δ − 2 ≤ 64 H n , H_n=\max\{1,R^2/s^2,R_*^2/\eta^2\},\qquad\delta^{-2}\le64H_n, H n = max { 1 , R 2 / s 2 , R ∗ 2 / η 2 } , δ − 2 ≤ 64 H n ,
a union bound yields a common event G q P G_{q_P} G q P of probability at least 1 − p 1-p 1 − p whenever
D ≥ max { 3 , ⌈ 512 H n [ 2 log 5 + log 2 K ( q P + 1 ) p ] ⌉ } . (79) D\ge\max\left\{3,\left\lceil512H_n\left[2\log5+\log\frac{2K(q_P+1)}{p}\right]\right\rceil\right\}. \tag*{(79)} D ≥ max { 3 , ⌈ 512 H n [ 2 log 5 + log p 2 K ( q P + 1 ) ] ⌉ } . (79)
Localization uses only the current-center subspace; the transcript keeps all earlier responses.
We choose D = ⌈ 1024 H n log ( 1024 K ( q P + 1 ) ) ⌉ D=\lceil1024H_n\log(1024K(q_P+1))\rceil D = ⌈ 1024 H n log ( 1024 K ( q P + 1 ))⌉ . For p = 1 / 64 p=1/64 p = 1/64 the logarithmic bracket in (79) is log ( 3200 K ( q P + 1 ) ) \log(3200K(q_P+1)) log ( 3200 K ( q P + 1 )) . Since K ≥ 4 K\ge4 K ≥ 4 ,
( 1024 K ( q P + 1 ) ) 2 ≥ 3200 K ( q P + 1 ) . (80) (1024K(q_P+1))^2\ge3200K(q_P+1). \tag*{(80)} ( 1024 K ( q P + 1 ) ) 2 ≥ 3200 K ( q P + 1 ) . (80)
Taking logarithms proves (79) , including D ≥ 3 D\ge3 D ≥ 3 .
On G q P G_{q_P} G q P , induction using Lemma 13 couples all exact real and ideal responses through q P q_P q P , since equal pasts give the same legal query, localization gives its same unique tuple, and fallback is unused. The output test gives (74) when the terminal axis is hidden. The proof covers all raw radii because primal vectors are capped, node cutoffs vanish outside radius 2 R ∗ 2R_* 2 R ∗ , and edge flatness holds on the entire retained cone.
Posterior Owner Cost and Integer Stopping
Keep the owners fixed before execution, independently of the axes and private seed. The augmented ideal history contains the seed, exposed axes, internal hits and misses, full censored responses, and stop/output decisions; it is not conditioned on geometric success or legality of the real oracle. Induction over internal tests leaves the frontier owner uniform among its unexcluded labels and all successors independent and uniform, since a miss removes only its label, a hit fixes the current owner, and a new axis is independent. Full values, gradients and proximal tuples read only the exposed prefix; the fallback reads none.
Lemma 14 (Posterior cost of serial owner exposure). For E ≥ 1 E\ge1 E ≥ 1 independent uniform owners on [ n ] [n] [ n ] , n ≥ 2 n\ge2 n ≥ 2 , assume the posterior interface above. A call traverses matching owners until its first mismatch. Freeze at termination and let C q C_q C q be complete chain exposure within q q q calls, for integer q ≥ 0 q\ge0 q ≥ 0 . Then
P ( C q ) ≤ 2 q H , H = E ( n − 1 ) , q ≥ 0. (81) \mathbb{P}(C_q)\le\frac{2q}{H},\qquad H=\mathbb{E}(n-1),\qquad q\ge0. \tag*{(81)} P ( C q ) ≤ H 2 q , H = E ( n − 1 ) , q ≥ 0. (81)
Proof. With r r r frontier candidates and s s s unreached successors, set W = ( r − 1 ) / 2 + s ( n − 1 ) / 2 W=(r-1)/2+s(n-1)/2 W = ( r − 1 ) /2 + s ( n − 1 ) /2 , and W = 0 W=0 W = 0 after completion. Repeated failed labels leave W W W unchanged. A new label misses with probability ( r − 1 ) / r (r-1)/r ( r − 1 ) / r , decreasing W W W by 1 / 2 1/2 1/2 , or hits with probability 1 / r 1/r 1/ r , removing ( r − 1 ) / 2 (r-1)/2 ( r − 1 ) /2 and continuing at an independent successor. The conditional expected decrease during the entire call therefore obeys
d ( r , 0 ) = r − 1 r , d ( r , s ) = r − 1 r + d ( n , s − 1 ) r ≤ 1 ( s ≥ 1 ) , d(r,0)=\frac{r-1}{r}, \qquad d(r,s)=\frac{r-1}{r}+\frac{d(n,s-1)}{r}\leq1 \quad(s\geq1), d ( r , 0 ) = r r − 1 , d ( r , s ) = r r − 1 + r d ( n , s − 1 ) ≤ 1 ( s ≥ 1 ) ,
by induction, including all consecutive hits in one call. The potential is nonnegative and nonincreasing, starts at W 0 = H / 2 W_0=H/2 W 0 = H /2 , and freezes at termination. Summing conditional expectations gives E [ W 0 − W q ] ≤ q \mathbb{E}[W_0-W_q]\leq q E [ W 0 − W q ] ≤ q . Completion implies W q = 0 W_q=0 W q = 0 . Thus P ( C q ) W 0 ≤ E [ W 0 − W q ] ≤ q \mathbb{P}(C_q)W_0\leq\mathbb{E}[W_0-W_q]\leq q P ( C q ) W 0 ≤ E [ W 0 − W q ] ≤ q .
■
For q P = ⌊ H / 16 ⌋ q_P=\lfloor H/16\rfloor q P = ⌊ H /16 ⌋ , the lemma gives
P ( C q P c ) ≥ 7 8 . (82) \mathbb{P}(C_{q_P}^{c})\geq\frac{7}{8}. \tag*{(82)} P ( C q P c ) ≥ 8 7 . (82)
On C q P c ∩ { N ≤ q P } ∩ G q P C_{q_P}^{c}\cap\{N\leq q_P\}\cap G_{q_P} C q P c ∩ { N ≤ q P } ∩ G q P the terminal axis remains unexposed, the actual and ideal outputs coincide, and the output barrier is at least 6 ε 6\varepsilon 6 ε . Uniform expected accuracy on every fixed instance, averaged over the hard prior, implies by Markov’s inequality that this event has probability at most 1 / 6 1/6 1/6 . Thus
P { N > q P } ≥ 7 8 − 1 64 − 1 6 = 133 192 . (83) \mathbb{P}\{N>q_P\}\geq\frac{7}{8}-\frac{1}{64}-\frac{1}{6}=\frac{133}{192}. \tag*{(83)} P { N > q P } ≥ 8 7 − 64 1 − 6 1 = 192 133 . (83)
This argument also covers q P = 0 q_P=0 q P = 0 , where the only required geometric test is the proposed zero-call output, already included above. Since the query count is integer,
E N ≥ ( q P + 1 ) P { N > q P } ≥ 133 H 3072 = 133 M ( m − 1 ) ( n − 1 ) 3072 ≥ 133 n M m 12288 . (84) \begin{aligned}
\mathbb{E}N\geq(q_P+1)\mathbb{P}\{N>q_P\}\geq\frac{133H}{3072} \\
&=\frac{133M(m-1)(n-1)}{3072}\geq\frac{133nMm}{12288}. \tag*{(84)}
\end{aligned} E N ≥ ( q P + 1 ) P { N > q P } ≥ 3072 133 H = 3072 133 M ( m − 1 ) ( n − 1 ) ≥ 12288 133 n M m . (84)
The last inequality uses m , n ≥ 2 m,n\geq2 m , n ≥ 2 ; infinite expected cost is trivial. Since the instance prior is seed-independent, averaging the real cost selects one fixed realization with this expected cost, while the assumed accuracy still holds there.
In the high branch, using (30) in (84) , with 12288 = 3 ⋅ 2 12 12288=3\cdot2^{12} 12288 = 3 ⋅ 2 12 and 512 = 2 9 512=2^9 512 = 2 9 , gives
E N ≥ 133 c ∗ 2 3 ⋅ 2 73 n 3 / 4 κ L Δ ε 2 , \mathbb{E}N\geq\frac{133c_*^2}{3\cdot2^{73}}n^{3/4}\sqrt{\kappa}\frac{L\Delta}{\varepsilon^2}, E N ≥ 3 ⋅ 2 73 133 c ∗ 2 n 3/4 κ ε 2 L Δ ,
proving Theorem 3’s lower bound.
Fixed confidence. For a deterministic cap q P q_P q P , the failure probability is at least 7 / 8 − 1 / 64 = 55 / 64 > 1 / 3 7/8-1/64=55/64>1/3 7/8 − 1/64 = 55/64 > 1/3 , proving the fixed-confidence lower bound.
Han’s Protocol and All Early Horizons
Corollary 15 (Han’s protocol and one fixed instance for all early horizons). In the window (12) , the same family gives the lower bound under Han et al. [7 ] , whose full-gradient refreshes occur with probability at most c 0 / n c_0/n c 0 / n . More precisely, put
E = M ( m − 1 ) , H = E ( n − 1 ) , C = 2 + c 0 , T 0 = ⌊ H 512 C ⌋ . E = M(m-1), \qquad H = E(n-1), \qquad C = 2+c_0, \qquad T_0 = \left\lfloor\frac{H}{512C} \right\rfloor. E = M ( m − 1 ) , H = E ( n − 1 ) , C = 2 + c 0 , T 0 = ⌊ 512 C H ⌋ . For every algorithm in that protocol, there is one fixed hard realization for which, simultaneously at every step 0 ≤ t ≤ T 0 0 \le t \le T_0 0 ≤ t ≤ T 0 ,
E s e e d ∥ ∇ ~ η ( P t , ξ p u b , t ) ∥ ≥ 159 32 ε > ε . (85) \mathbb{E}_{\mathrm{seed}}\left\|\widetilde{\nabla}_{\eta}(P_t,\xi_{\mathrm{pub},t})\right\| \ge\frac{159}{32}\varepsilon> \varepsilon. \tag*{(85)} E seed ∇ η ( P t , ξ pub , t ) ≥ 32 159 ε > ε . (85) Consequently its first accurate horizon, and hence the inf A sup f inf t \inf_A \sup_f \inf_t inf A sup f inf t complexity of Definition 13, is at least
H 512 ( 2 + c 0 ) ≥ n M m 2048 ( 2 + c 0 ) ≥ c ∗ 2 2 72 ( 2 + c 0 ) n 3 / 4 κ L Δ ε 2 . (86) \frac{H}{512(2+c_0)} \ge\frac{nMm}{2048(2+c_0)} \ge\frac{c_*^2}{2^{72}(2+c_0)}n^{3/4}\sqrt{\kappa}\frac{L\Delta}{\varepsilon^2}. \tag*{(86)} 512 ( 2 + c 0 ) H ≥ 2048 ( 2 + c 0 ) n M m ≥ 2 72 ( 2 + c 0 ) c ∗ 2 n 3/4 κ ε 2 L Δ . (86) The dimension D D D chosen in Appendix C.§ 1 suffices.
Proof Outline. We embed Han’s protocol in our oracle model, add a subspace test for every early iterate to the ideal simulator, and find one event of probability at least 53 / 64 53/64 53/64 on which the terminal axis stays unexposed at every early step; the terminal barrier then bounds every early gradient.
Embedding the protocol. Whole-space projections are identities, so Han’s fixed sampling and linear-span operations restrict our model. Supply the common component smoothness bound and grant the owner-independent initial full gradient, whose only nonzero block is − ε λ s U 1 , 1 -\sqrt{\varepsilon}\lambda sU_{1,1} − ε λ s U 1 , 1 in the first primal subblock; the public coordinate has zero derivative. It therefore reveals only the already free source axis. Simulate a full gradient by n n n component calls, ignoring their proximal responses. A Han step has at most two component PIFO calls and an independent refresh coin A j A_j A j with mean π ≤ c 0 / n \pi\le c_0/n π ≤ c 0 / n . The predrawn budget
B T = 2 T + n ∑ j ≤ T A j B_T = 2T+n\sum_{j\le T}A_j B T = 2 T + n j ≤ T ∑ A j
dominates its cost, even under mid-step truncation, and satisfies E B T ≤ C T \mathbb{E}B_T \le CT E B T ≤ C T .
Tested early outputs. Write d = H / 16 d=H/16 d = H /16 and q P = ⌊ d ⌋ q_P=\lfloor d\rfloor q P = ⌊ d ⌋ . Use the exact ideal component simulator for its first q P q_P q P calls. In addition to its rank-at-most-two current-center tests, test the rank-one capped output span for each of the iterates P 0 , … , P T 0 P_0,\ldots,P_{T_0} P 0 , … , P T 0 produced within that call budget; assign a fixed virtual output to any step not completed within the budget. This keeps every test measurable from the ideal history. On B T 0 ≤ d B_{T_0}\le d B T 0 ≤ d , all T 0 T_0 T 0 steps have completed within q P q_P q P calls, so none of their outputs is virtual.
Subspace tests. Use the same δ \delta δ , H n H_n H n as in (79) . Each rank-at-most-two test fails with probability at most 50 e − D / ( 512 H n ) 50e^{-D/(512H_n)} 50 e − D / ( 512 H n ) . There are at most K ( q P + T 0 + 1 ) ≤ 2 K ( q P + 1 ) K(q_P+T_0+1)\le2K(q_P+1) K ( q P + T 0 + 1 ) ≤ 2 K ( q P + 1 ) tests since T 0 ≤ q P T_0\le q_P T 0 ≤ q P . The same dimension therefore gives an event G G G with
P ( G c ) ≤ 100 K ( q P + 1 ) ( 1024 K ( q P + 1 ) ) 2 = 100 1024 2 K ( q P + 1 ) < 1 64 . \mathbb{P}(G^c)\le\frac{100K(q_P+1)}{(1024K(q_P+1))^2}=\frac{100}{1024^2K(q_P+1)}<\frac{1}{64}. P ( G c ) ≤ ( 1024 K ( q P + 1 ) ) 2 100 K ( q P + 1 ) = 102 4 2 K ( q P + 1 ) 100 < 64 1 .
This includes T 0 = q P = 0 T_0=q_P=0 T 0 = q P = 0 , since K ≥ 4 K\ge4 K ≥ 4 . One common event. Lemma 14 gives P ( C q p c ) ≥ 7 / 8 \mathbb{P}(C_{\mathrm{qp}}^{c}) \ge7/8 P ( C qp c ) ≥ 7/8 without conditioning on G G G . Markov’s inequality and the definition of T 0 T_0 T 0 give P ( B T 0 > d ) ≤ C T 0 / d ≤ 1 / 32 \mathbb{P}(B_{T_0} > d) \le CT_0/d \le1/32 P ( B T 0 > d ) ≤ C T 0 / d ≤ 1/32 . Thus the single common event
E = C q p c ∩ { B T 0 ≤ d } ∩ G \mathcal{E} = C_{\mathrm{qp}}^{c} \cap\{B_{T_0} \le d\} \cap G E = C qp c ∩ { B T 0 ≤ d } ∩ G
has probability at least 7 / 8 − 1 / 32 − 1 / 64 = 53 / 64 7/8 - 1/32 - 1/64 = 53/64 7/8 − 1/32 − 1/64 = 53/64 . On this one event the terminal axis is still unexposed after all T 0 T_0 T 0 steps, and therefore also at every earlier step. Exact transcript coupling and the separately tested early outputs give ∥ ∇ V ~ η ( P t , ξ p u b , t ) ∥ ≥ 6 ε \lVert\nabla\widetilde{V}_{\eta}(P_t,\xi_{\mathrm{pub},t}) \rVert\ge6\varepsilon ∥ ∇ V η ( P t , ξ pub , t )∥ ≥ 6 ε simultaneously for all 0 ≤ t ≤ T 0 0 \le t \le T_0 0 ≤ t ≤ T 0 .
A fixed realization. Let B B B be the event, over real instances and seeds, that ∥ ∇ V ~ η ( P t , ξ p u b , t ) ∥ ≥ 6 ε \lVert\nabla\widetilde{V}_{\eta}(P_t,\xi_{\mathrm{pub},t}) \rVert\ge6\varepsilon ∥ ∇ V η ( P t , ξ pub , t )∥ ≥ 6 ε for all 0 ≤ t ≤ T 0 0 \le t \le T_0 0 ≤ t ≤ T 0 . The coupling gives E ⊆ B \mathcal{E} \subseteq B E ⊆ B , so P ( B ) ≥ 53 / 64 \mathbb{P}(B) \ge53/64 P ( B ) ≥ 53/64 . Averaging B B B selects one fixed realization with this seed probability. Every early expected gradient there is therefore at least 6 ε ( 53 / 64 ) = 159 ε / 32 6\varepsilon(53/64) = 159\varepsilon/32 6 ε ( 53/64 ) = 159 ε /32 . Definition 13 indexes the step-( T − 1 ) (T-1) ( T − 1 ) point by T T T ; thus the first successful horizon is at least T 0 + 1 ≥ H / ( 512 C ) T_0 + 1 \ge H/(512C) T 0 + 1 ≥ H / ( 512 C ) . The floor bounds on m m m , M M M give (86) .
Membership in Han’s class. The mean is L L L -smooth by the RMS assumption, so its primal weak-convexity parameter is at most L L L . Together with the already proved gap and strong concavity, this places the witness in Han’s nonconvex–strongly-concave class with μ x = L \mu_x = L μ x = L . Because every legal proximal subproblem has a unique saddle, the response does not depend on a selection rule. ■
The Strongly Concave Lower Bound for Every Condition Number
To complete the proof of Theorem 1 , we specialize the common path family to 64 ≤ κ ≤ 2 18 n 64 \le\kappa\le2^{18}\sqrt{n} 64 ≤ κ ≤ 2 18 n , and then add a primal family for 1 ≤ κ < 64 1 \le\kappa< 64 1 ≤ κ < 64 and an indexed-search baseline.
The Two-Node Specialization
The common parameters (23) –(24) give m = 2 m = 2 m = 2 , r = μ / a ≥ 1 / 4 r = \mu/a \ge1/4 r = μ / a ≥ 1/4 , g = 1 + r g = 1 + r g = 1 + r , and
λ = 2 − 39 L κ κ + 2 16 n . (87) \lambda= \frac{2^{-39}L\kappa}{\kappa+ 2^{16}\sqrt{n}}. \tag*{(87)} λ = κ + 2 16 n 2 − 39 L κ . (87)
As before, use this family only in the path window (25), with the components, widths and oracle simulator of Appendices A–C.
Corollary 16 (Intermediate admissibility and oracle cost). The augmented two-node family has joint RMS smoothness below 4 L / 5 4L/5 4 L /5 , componentwise μ \mu μ -strong dual concavity, the 6 ε 6\varepsilon 6 ε terminal barrier, y ∗ ( 0 ) = 0 y^{*}(0) = 0 y ∗ ( 0 ) = 0 , initial squared gradient norm at most 2 L Δ 2L\Delta 2 L Δ and primal gap less than Δ / 2 \Delta/2 Δ/2 . Its common exact legal domain is τ ≥ L / 8 \tau\ge L/8 τ ≥ L /8 , including a unique response at the boundary. A fixed polynomial dimension yields a fixed hard realization with
E N ≥ 133 M ( n − 1 ) 3072 ≥ 133 n M 6144 , (88) \mathbb{E}N \ge\frac{133M(n-1)}{3072} \ge\frac{133nM}{6144}, \tag*{(88)} E N ≥ 3072 133 M ( n − 1 ) ≥ 6144 133 n M , (88) and the same fixed-confidence lower bound. By (71) and d = 6 λ r d = 6\lambda r d = 6 λ r , the initial primal–dual gap satisfies G 0 ≤ ( 5 e / 32 + 1536 μ / a ) Δ G_0 \le(5e/32 + 1536\mu/a)\Delta G 0 ≤ ( 5 e /32 + 1536 μ / a ) Δ .
Proof Lemmas 9, 11 and 12 apply directly to this branch. The exact localization, total ideal transcript, posterior owner cost of Lemma 14 , and arbitrary-output test in Appendix C.§ 1 then apply without changes. Here E = M E=M E = M , so (84) gives the sharper first inequality in (88) ; the capped failure probability is still 55 / 64 > 1 / 3 55/64>1/3 55/64 > 1/3 . Finally g / r = ( 1 + r ) / r ≤ 5 g/r=(1+r)/r\leq5 g / r = ( 1 + r ) / r ≤ 5 in (27) , hence H n = O ( n + M ) H_n=O(n+M) H n = O ( n + M ) and D = O ( ( n + M ) log ( n M ) ) D=O((n+M)\log(nM)) D = O (( n + M ) log ( n M )) with D D D chosen as in Appendix C.§ 1 . Total dimension is 4 M D + 1 4MD+1 4 M D + 1 , fixed before execution.
A Primal Family with a Public Exact Convexification Threshold
The next family works for every κ ≥ 1 \kappa\geq1 κ ≥ 1 and will be used when κ < 64 \kappa<64 κ < 64 . It has no hidden dual path. Retain the primal blocks P j = ( X j , Z j + 1 ) P_j=(X_j,Z_{j+1}) P j = ( X j , Z j + 1 ) , independent Haar axes U j , X , U j , Z U_{j,X},U_{j,Z} U j , X , U j , Z , and the same w j , x j , z j + 1 , J M , h j , A j = h j Γ w j w_j,x_j,z_{j+1},J_M,h_j,A_j=h_j\Gamma w_j w j , x j , z j + 1 , J M , h j , A j = h j Γ w j . Define hidden forcing coordinates
t j , 1 = ⟨ U j , X , A j , X ⟩ = h j x j , t j , 2 = ⟨ U j , Z , A j , Z ⟩ = − h j z j + 1 / 2 , t_{j,1}=\langle U_{j,X},A_{j,X}\rangle=h_jx_j,\qquad t_{j,2}=\langle U_{j,Z},A_{j,Z}\rangle=-h_jz_{j+1}/2, t j , 1 = ⟨ U j , X , A j , X ⟩ = h j x j , t j , 2 = ⟨ U j , Z , A j , Z ⟩ = − h j z j + 1 /2 ,
and the flat cross term
C η ( t 1 , t 2 ) = ϕ η ( t 1 + t 2 ) − ϕ η ( t 1 ) − ϕ η ( t 2 ) , ϕ η ( t ) = 1 2 ( ∣ t ∣ − η ) + 2 . (89) C_\eta(t_1,t_2)=\phi_\eta(t_1+t_2)-\phi_\eta(t_1)-\phi_\eta(t_2),\qquad\phi_\eta(t)=\frac{1}{2}(|t|-\eta)_+^2. \tag*{(89)} C η ( t 1 , t 2 ) = ϕ η ( t 1 + t 2 ) − ϕ η ( t 1 ) − ϕ η ( t 2 ) , ϕ η ( t ) = 2 1 ( ∣ t ∣ − η ) + 2 . (89)
Choose independent fixed owners Σ j ∼ Unif [ n ] \Sigma_j\sim\operatorname{Unif}[n] Σ j ∼ Unif [ n ] . With public auxiliary primal coordinates ξ ∈ R n \xi\in\mathbb{R}^n ξ ∈ R n , set
g i ( P , ξ ) = λ s 2 J M ( P ) + ∑ j p ( P j ) + 6 λ ∥ A ( P ) ∥ 2 + 12 n λ ∑ j : Σ j = i C η ( t j , 1 , t j , 2 ) + B ( 1 − cos ξ i ) , (90) g_i(P,\xi)=\lambda s^2J_M(P)+\sum_j p(P_j)+6\lambda\lVert A(P)\rVert^2
+\;12n\lambda\sum_{j:\Sigma_j=i}C_\eta(t_{j,1},t_{j,2})+B(1-\cos\xi_i), \tag*{(90)} g i ( P , ξ ) = λ s 2 J M ( P ) + j ∑ p ( P j ) + 6 λ ∥ A ( P ) ∥ 2 + 12 nλ j : Σ j = i ∑ C η ( t j , 1 , t j , 2 ) + B ( 1 − cos ξ i ) , (90)
F i ( P , ξ , y ) = g i ( P , ξ ) − μ y 2 / 2 , y ∈ R . (91) F_i(P,\xi,y)=g_i(P,\xi)-\mu y^2/2,\qquad y\in\mathbb{R}. \tag*{(91)} F i ( P , ξ , y ) = g i ( P , ξ ) − μ y 2 /2 , y ∈ R . (91)
The calibration is
λ = L 2 34 n , B = 2 32 n λ , s = 8 ε c ⋆ λ , R = 128 s , (92) \lambda=\frac{L}{2^{34}\sqrt{n}},\qquad B=2^{32}n\lambda,\qquad s=\frac{8\varepsilon}{c_\star\lambda},\qquad R=128s, \tag*{(92)} λ = 2 34 n L , B = 2 32 nλ , s = c ⋆ λ 8 ε , R = 128 s , (92)
M = ⌈ c ⋆ 2 λ Δ 2048 ε 2 ⌉ , η = c ⋆ s 96 2 M A 1 , A 1 = 2433 , A 2 = 231000. (93) M=\left\lceil\frac{c_\star^2\lambda\Delta}{2048\varepsilon^2}\right\rceil,\qquad\eta=\frac{c_\star s}{96\sqrt{2}MA_1},\qquad A_1=2433,\qquad A_2=231000. \tag*{(93)} M = ⌈ 2048 ε 2 c ⋆ 2 λ Δ ⌉ , η = 96 2 M A 1 c ⋆ s , A 1 = 2433 , A 2 = 231000. (93)
Again assume (25), with this new λ \lambda λ , so M ≥ 2 M\geq2 M ≥ 2 .
Lemma 17 (Global smoothness and a common exact legal domain). The components (92) satisfy joint RMS smoothness L L L and componentwise μ \mu μ -strong dual concavity. Their exact legal domain of inverse step sizes is τ ∈ [ B , ∞ ) \tau\in[B,\infty) τ ∈ [ B , ∞ ) , independent of all hidden axes, owners, component indices and centers, and every legal response is unique. These statements also hold for censored formulas obtained by setting any unexposed scalar projections to zero.
Proof Almost everywhere, ∇ 2 C η = a 0 ( 1 1 1 1 ) − diag ( b 0 , c 0 ) \nabla^2C_\eta=a_0\begin{pmatrix}1&1\\1&1\end{pmatrix}-\operatorname{diag}(b_0,c_0) ∇ 2 C η = a 0 ( 1 1 1 1 ) − diag ( b 0 , c 0 ) for some a 0 , b 0 , c 0 ∈ { 0 , 1 } a_0,b_0,c_0\in\{0,1\} a 0 , b 0 , c 0 ∈ { 0 , 1 } ; hence ∥ ∇ 2 C η ∥ ≤ 3 \lVert\nabla^2C_\eta\rVert\leq3 ∥ ∇ 2 C η ∥ ≤ 3 and ∥ ∇ C η ( t ) ∥ ≤ 3 ∥ t ∥ \lVert\nabla C_\eta(t)\rVert\leq3\lVert t\rVert ∥ ∇ C η ( t )∥ ≤ 3 ∥ t ∥ . Put K A = A 1 2 + 128 A 2 < 3.6 ⋅ 10 7 K_A=A_1^2+128A_2<3.6\cdot10^7 K A = A 1 2 + 128 A 2 < 3.6 ⋅ 1 0 7 . The public forcing square costs 12 K A λ 12K_A\lambda 12 K A λ in Hessian norm; the source and penalty cost 601 λ 601\lambda 601 λ by
Lemma 9. As functions of the forcing vector A = A ( P ) A=A(P) A = A ( P ) , the owned terms are T i ( A ) = 12 n λ ∑ j : Σ j = i C η ( ⟨ U j , X , A j , X ⟩ , ⟨ U j , Z , A j , Z ⟩ ) T_i(A)=12n\lambda\sum_{j:\Sigma_j=i} C_\eta(\langle U_{j,X},A_{j,X}\rangle,\langle U_{j,Z},A_{j,Z}\rangle) T i ( A ) = 12 nλ ∑ j : Σ j = i C η (⟨ U j , X , A j , X ⟩ , ⟨ U j , Z , A j , Z ⟩) . Before composition, the stage pairs have disjoint output coordinates, norm-one hidden maps, and exactly one owner. Thus, for every direction v v v ,
( n − 1 ∑ i ∥ D 2 T i ( A ) v ∥ 2 ) 1 / 2 ≤ 36 n λ ∥ v ∥ \left(n^{-1}\sum_i\lVert D^2T_i(A)v\rVert^2\right)^{1/2}\leq36\sqrt{n}\lambda\lVert v\rVert ( n − 1 i ∑ ∥ D 2 T i ( A ) v ∥ 2 ) 1/2 ≤ 36 n λ ∥ v ∥ and
( n − 1 ∑ i ∥ ∇ T i ( A ) ∥ 2 ) 1 / 2 ≤ 36 n λ ∥ A ∥ . \left(n^{-1}\sum_i\lVert\nabla T_i(A)\rVert^2\right)^{1/2}\leq36\sqrt{n}\lambda\lVert A\rVert. ( n − 1 i ∑ ∥ ∇ T i ( A ) ∥ 2 ) 1/2 ≤ 36 n λ ∥ A ∥ . The two chain-rule terms, with ∥ A ∥ ≤ R \lVert A\rVert\leq R ∥ A ∥ ≤ R , cost 36 n λ ( A 1 2 + R A 2 / s ) = 36 K A n λ 36\sqrt{n}\lambda(A_1^2+RA_2/s)=36K_A\sqrt{n}\lambda 36 n λ ( A 1 2 + R A 2 / s ) = 36 K A n λ . Minkowski's inequality gives
L P ≤ ( 601 + 48 K A ) n λ < 2 31 n λ . L_P\leq(601+48K_A)\sqrt{n}\lambda<2^{31}\sqrt{n}\lambda. L P ≤ ( 601 + 48 K A ) n λ < 2 31 n λ . Without averaging the same argument gives each P P P -Hessian norm below 2 31 n λ = B / 2 2^{31}n\lambda=B/2 2 31 nλ = B /2 . The C 1 , 1 C^{1,1} C 1 , 1 components therefore have these global gradient bounds. The auxiliary gradients occupy distinct coordinates, so their RMS constant is B / n = L / 4 B/\sqrt{n}=L/4 B / n = L /4 . Orthogonality of the P , ξ , y P,\xi,y P , ξ , y blocks gives total RMS constant max { L P , B / n , μ } ≤ L \max\{L_P,B/\sqrt{n},\mu\}\leq L max { L P , B / n , μ } ≤ L , including μ = L \mu=L μ = L .
For τ < B \tau<B τ < B , the i i i th auxiliary curvature at ξ i = π \xi_i=\pi ξ i = π is τ − B < 0 \tau-B<0 τ − B < 0 , ruling out global primal convexity at every center. For τ ≥ B \tau\geq B τ ≥ B the P P P part is ( τ − B / 2 ) (\tau-B/2) ( τ − B /2 ) -strongly convex. The i i i th auxiliary curvature τ + B cos ξ i \tau+B\cos\xi_i τ + B cos ξ i is nonnegative; at τ = B \tau=B τ = B its isolated zeros still give a positive integral on every nontrivial interval, hence strict convexity. The other coordinates have positive proximal quadratics. All capped and auxiliary terms are bounded, p ≥ 0 p\geq0 p ≥ 0 , and the proximal quadratic is positive, giving coercivity and a unique primal minimizer. The unique dual maximizer is
y + = τ y ˉ μ + τ . (94) y^+=\frac{\tau\bar y}{\mu+\tau}. \tag*{(94)} y + = μ + τ τ y ˉ . (94) Censoring replaces norm-one hidden maps by contractions; all estimates and the argument for the public auxiliary coordinates persist. This proves the exact domain and unique responses for both actual and censored formulas.
Lemma 18 (Value identity, perturbation and initialization). The value of (92) satisfies the terminal barrier ∥ ∇ V η ( P , ξ ) ∥ ≥ 6 ϵ \lVert\nabla V_\eta(P,\xi)\rVert\geq6\epsilon ∥ ∇ V η ( P , ξ )∥ ≥ 6 ϵ whenever ∣ z M + 1 ∣ ≤ s / 8 \lvert z_{M+1}\rvert\leq s/8 ∣ z M + 1 ∣ ≤ s /8 , and belongs to the stated class. Moreover y ∗ ( P , ξ ) = 0 y^*(P,\xi)=0 y ∗ ( P , ξ ) = 0 everywhere and G 0 = V ( 0 ) − inf V ≤ Δ G_0=V(0)-\inf V\leq\Delta G 0 = V ( 0 ) − inf V ≤ Δ .
Proof At width zero, C 0 ( t 1 , t 2 ) = t 1 t 2 C_0(t_1,t_2)=t_1t_2 C 0 ( t 1 , t 2 ) = t 1 t 2 . Hence
6 λ ∥ A j ∥ 2 + 12 λ t j , 1 t j , 2 = h j 2 [ 6 λ ( x j − z j + 1 / 2 ) 2 + 6 λ ∥ A j , ⊥ ∥ 2 + 3 2 λ ∥ B j + 1 , ⊥ ∥ 2 ] . 6\lambda\lVert A_j\rVert^2+12\lambda t_{j,1}t_{j,2}
=h_j^2\left[6\lambda(x_j-z_{j+1}/2)^2+6\lambda\lVert A_{j,\perp}\rVert^2+\frac{3}{2}\lambda\lVert B_{j+1,\perp}\rVert^2\right]. 6 λ ∥ A j ∥ 2 + 12 λ t j , 1 t j , 2 = h j 2 [ 6 λ ( x j − z j + 1 /2 ) 2 + 6 λ ∥ A j , ⊥ ∥ 2 + 2 3 λ ∥ B j + 1 , ⊥ ∥ 2 ] .
Since averaging leaves exactly one copy of each bridge, the P P P value is exactly V 0 V_0 V 0 in (51). The auxiliary mean B ∑ i ( 1 − cos ξ i ) / n B\sum_i(1-\cos\xi_i)/n B ∑ i ( 1 − cos ξ i ) / n is nonnegative and vanishes with its gradient at 0 0 0 .
Use ϕ η ′ ( t ) = t − clip η ( t ) \phi_\eta'(t)=t-\operatorname{clip}_\eta(t) ϕ η ′ ( t ) = t − clip η ( t ) and ∣ ϕ η ( t ) − t 2 / 2 ∣ ≤ η ∣ t ∣ \lvert\phi_\eta(t)-t^2/2\rvert\leq\eta\lvert t\rvert ∣ ϕ η ( t ) − t 2 /2 ∣ ≤ η ∣ t ∣ . The map taking each pair to its sum has norm 2 \sqrt{2} 2 . Let T η , m e a n = n − 1 ∑ i T i T_{\eta,\mathrm{mean}}=n^{-1}\sum_iT_i T η , mean = n − 1 ∑ i T i , with T i T_i T i from the proof of Lemma 17, and let T 0 , m e a n T_{0,\mathrm{mean}} T 0 , mean be its value with C 0 C_0 C 0 . The M M M pair-sum clips and the 2 M 2M 2 M individual clips give, before composition,
∥ ∇ A ( T η , m e a n − T 0 , m e a n ) ∥ ≤ 24 2 M λ η , \lVert\nabla_A(T_{\eta,\mathrm{mean}}-T_{0,\mathrm{mean}})\rVert\leq24\sqrt{2M}\lambda\eta, ∥ ∇ A ( T η , mean − T 0 , mean )∥ ≤ 24 2 M λ η ,
∣ T η , m e a n − T 0 , m e a n ∣ ≤ 24 2 M λ η ∥ A ∥ . \lvert T_{\eta,\mathrm{mean}}-T_{0,\mathrm{mean}}\rvert\leq24\sqrt{2M}\lambda\eta\lVert A\rVert. ∣ T η , mean − T 0 , mean ∣ ≤ 24 2 M λ η ∥ A ∥ .
After composition and the choice of η \eta η ,
∥ ∇ P V η − ∇ V 0 ∥ ≤ c ⋆ λ s / 4 , \lVert\nabla_P V_\eta-\nabla V_0\rVert\le c_\star\lambda s/4, ∥ ∇ P V η − ∇ V 0 ∥ ≤ c ⋆ λ s /4 ,
∣ V η ( P , 0 ) − V 0 ( P ) ∣ ≤ ( 32 c ⋆ / A 1 ) λ s 2 < 0.014 λ s 2 . \lvert V_\eta(P,0)-V_0(P)\rvert\le(32c_\star/A_1)\lambda s^2 < 0.014\lambda s^2. ∣ V η ( P , 0 ) − V 0 ( P )∣ ≤ ( 32 c ⋆ / A 1 ) λ s 2 < 0.014 λ s 2 .
Lemma 10 now gives the terminal bound; the orthogonal auxiliary gradient cannot cancel the P P P gradient. At the origin, the sole nonzero derivative has norm e λ s \sqrt{e}\lambda s e λ s . The source bound J M ≥ − 5 e M J_M\ge-5eM J M ≥ − 5 e M and (26) imply
V η ( 0 ) − inf V η ≤ ( 5 e M + 0.014 ) λ s 2 < Δ / 2 , e λ 2 s 2 ≤ e λ Δ / 64 < 2 L Δ . V_\eta(0)-\inf V_\eta\le(5eM+0.014)\lambda s^2<\Delta/2,\qquad e\lambda^2s^2\le e\lambda\Delta/64<2L\Delta. V η ( 0 ) − inf V η ≤ ( 5 e M + 0.014 ) λ s 2 < Δ/2 , e λ 2 s 2 ≤ e λ Δ/64 < 2 L Δ.
The dual is separable and maximized at zero, proving the identity for G 0 G_0 G 0 .
Exact Proximal Hiding for the Primal Family
Lemma 19 (Localization of the complete exact response). The family (92) admits an exact owner-prefix simulation for arbitrary finite centers and arbitrary legal proximal parameters. Every unexposed primal subblock of its censored proximal response lies in the span of that subblock’s current center. If each unexposed axis has projection norm at most δ h = η / ( 4 R ) \delta_h=\eta/(4R) δ h = η / ( 4 R ) onto that span, the actual and censored value, full gradient, and unique proximal response agree.
Proof Grant U 1 , X U_{1,X} U 1 , X and complete each queried component’s same-owner run: a hit reveals U j , Z U_{j,Z} U j , Z and the next source U j + 1 , X U_{j+1,X} U j + 1 , X ; the first mismatch reveals only failure of that label. Censor every unexposed scalar projection, retaining public norms and maps. For an unexposed subblock the censored regularized objective is invariant under all orthogonal transformations fixing its center. Its unique minimizer (Lemma 17 ) therefore lies in that center’s span, or is zero when the center is zero.
On the projection event, both center and censored minimizer have every unexposed capped coordinate bounded by R δ h = η / 4 < s / 8 R\delta_h=\eta/4<s/8 R δ h = η /4 < s /8 , independently of raw radii, so termwise equality follows. Earlier bridges use exposed axes. Let j j j be the first unexposed stage. Its source and incoming connector are known, its outgoing gate factor is locally b + ( z j + 1 / s ) = 1 b_+(z_{j+1}/s)=1 b + ( z j + 1 / s ) = 1 , so its forcing square is radial, and its owned interaction is absent because its owner mismatched. The gate weights h k h_k h k of later stages contain the factor α ( z j + 1 / s ) \alpha(z_{j+1}/s) α ( z j + 1 / s ) , which vanishes locally. Each later source vanishes through its own unexposed connector, where both Ψ ( ± z k / s ) \Psi(\pm z_k/s) Ψ ( ± z k / s ) and their derivatives vanish. For any fully hidden pair, ∣ t 1 ∣ , ∣ t 2 ∣ ≤ η / 4 \lvert t_1\rvert,\lvert t_2\rvert\le\eta/4 ∣ t 1 ∣ , ∣ t 2 ∣ ≤ η /4 and ∣ t 1 + t 2 ∣ ≤ η / 2 \lvert t_1+t_2\rvert\le\eta/2 ∣ t 1 + t 2 ∣ ≤ η /2 , so all three terms of (89) vanish with their full gradients. These strict neighborhood margins establish equality of center tuples and of full gradients at the censored minimizer, including omitted directions. It is thus stationary for the actual globally convex regularized component and equals its unique minimizer. The dual response is the public (94) .
Lemma 20 (Primal-family information bound). Under (25), there is a fixed realization of (91) with polynomial dimension such that E N ≥ 133 n M / 6144 \mathbb{E}N\ge133nM/6144 E N ≥ 133 n M /6144 . The assertion includes arbitrary randomized query points, legal proximal parameters, stopping times and outputs. Up to the same smaller constant, it also holds for success probability 2 / 3 2/3 2/3 under a deterministic cap on the number of queries.
Proof. We run the argument of Appendix C.1 with three changes. First, the localizer is Lemma 19, the public legality test is τ < B \tau< B τ < B , and all admissible censored problems have unique responses by Lemma 17; fallback and exposed owner runs therefore read only the revealed prefix, and the null-set argument handles almost-sure legality as before. Second, each current-center span has rank at most one, there are K = 2 M K = 2M K = 2 M blocks, and we put H = M ( n − 1 ) H = M(n-1) H = M ( n − 1 ) , q = ⌊ H / 16 ⌋ q = \lfloor H/16 \rfloor q = ⌊ H /16 ⌋ and
D = ⌈ 1024 δ h − 2 log ( 1024 K ( q + 1 ) ) ⌉ . (95) D = \left\lceil1024\delta_h^{-2}\log(1024K(q+1)) \right\rceil. \tag*{(95)} D = ⌈ 1024 δ h − 2 log ( 1024 K ( q + 1 )) ⌉ . (95)
The spherical bound (79) and a union bound over at most K ( q + 1 ) K(q+1) K ( q + 1 ) tests, including a stopped or virtual output, give geometric failure below 1 / 64 1/64 1/64 , because D δ h 2 / 8 ≥ 128 log ( 1024 K ( q + 1 ) ) ≥ log ( 640 K ( q + 1 ) ) D\delta_h^2/8 \ge128\log(1024K(q+1)) \ge\log(640K(q+1)) D δ h 2 /8 ≥ 128 log ( 1024 K ( q + 1 )) ≥ log ( 640 K ( q + 1 )) . Off this event an unexposed terminal axis has ∣ z M + 1 ∣ ≤ R δ h = η / 4 < s / 8 \lvert z_{M+1}\rvert\le R\delta_h = \eta/4 < s/8 ∣ z M + 1 ∣ ≤ R δ h = η /4 < s /8 , hence gradient at least 6 ϵ 6\epsilon 6 ϵ . Third, each same-owner run stops at its first miss, and each miss response reads only the prefix and the failed label, so the augmented ideal history satisfies the posterior interface of Lemma 14 with E = M E = M E = M , without conditioning on geometric success.
With these inputs, (83), (84) and (85) give P ( N > q ) ≥ 133 / 192 \mathbb{P}(N > q) \ge133/192 P ( N > q ) ≥ 133/192 and E N ≥ 133 H / 3072 ≥ 133 n M / 6144 \mathbb{E}N \ge133H/3072 \ge133nM/6144 E N ≥ 133 H /3072 ≥ 133 n M /6144 , including q = 0 q = 0 q = 0 . Averaging over the hard prior selects one realization, and under a deterministic cap q q q the failure probability is at least 7 / 8 − 1 / 64 = 55 / 64 > 1 / 3 7/8 - 1/64 = 55/64 > 1/3 7/8 − 1/64 = 55/64 > 1/3 . Since δ h − 2 = O ( M ) \delta_h^{-2} = O(M) δ h − 2 = O ( M ) , the dimension is 2 M D + n + 1 = O ( M 2 log ( n M ) + n ) 2MD + n + 1 = O(M^2\log(nM) + n) 2 M D + n + 1 = O ( M 2 log ( n M ) + n ) . ■
The Indexed Baseline and Completion of All Accuracy Windows
Lemma 21 (An indexed baseline with exact proximal responses). For n ≥ 2 n \ge2 n ≥ 2 , κ ≥ 1 \kappa\ge1 κ ≥ 1 , and ϵ 2 ≤ 2 L Δ / 9 \epsilon^2 \le2L\Delta/9 ϵ 2 ≤ 2 L Δ/9 , the class contains fixed instances requiring worst-case expected cost at least n / 16 n/16 n /16 for expected-norm accuracy. The same lower bound holds for success probability 2 / 3 2/3 2/3 under a deterministic query cap.
Proof. Choose independent uniform I ∈ [ n ] I \in[n] I ∈ [ n ] and sign σ ∈ { − 1 , 1 } \sigma\in\{-1,1\} σ ∈ { − 1 , 1 } , and set (cf. the hidden-component quadratic of Zhou, 2026b, Appendix D.3)
F i ( x , y ) = L x 2 / 2 − 3 n σ ϵ 1 { i = I } x − μ y 2 / 2. F_i(x,y) = Lx^2/2 - 3n\sigma\epsilon\mathbf{1}_{\{i=I\}}x - \mu y^2/2. F i ( x , y ) = L x 2 /2 − 3 nσ ϵ 1 { i = I } x − μ y 2 /2.
The joint RMS constant is L L L , every component is μ \mu μ -strongly concave in y y y , and every τ > 0 \tau> 0 τ > 0 is legal with a unique response. The mean initial gap and squared joint gradient are 9 ϵ 2 / ( 2 L ) 9\epsilon^2/(2L) 9 ϵ 2 / ( 2 L ) and 9 ϵ 2 9\epsilon^2 9 ϵ 2 , respectively, within the budgets. Before index I I I is queried, center values, gradients, and proximal responses are all sign-independent. At any sign-independent output,
1 2 ∣ L x − 3 ϵ ∣ + 1 2 ∣ L x + 3 ϵ ∣ ≥ 3 ϵ ≥ 2 ϵ . \frac{1}{2}\lvert Lx - 3\epsilon\rvert+ \frac{1}{2}\lvert Lx + 3\epsilon\rvert\ge3\epsilon\ge2\epsilon. 2 1 ∣ Lx − 3 ϵ ∣ + 2 1 ∣ Lx + 3 ϵ ∣ ≥ 3 ϵ ≥ 2 ϵ .
For q = ⌊ n / 4 ⌋ q = \lfloor n/4 \rfloor q = ⌊ n /4 ⌋ , adaptive search hits I I I within q q q calls with probability at most q / n q/n q / n , since before the hit the label sequence depends only on misses. Thus expected accuracy implies
ϵ ≥ 2 ϵ [ 1 − P ( N > q ) − q / n ] , E N ≥ ( q + 1 ) / 4 ≥ n / 16. \epsilon\ge2\epsilon\left[1-\mathbb{P}(N>q)-q/n\right], \qquad\mathbb{E}N \ge(q+1)/4 \ge n/16. ϵ ≥ 2 ϵ [ 1 − P ( N > q ) − q / n ] , E N ≥ ( q + 1 ) /4 ≥ n /16.
For a deterministic cap q q q , the success intervals for the two signs are disjoint and success is at most 1 / 2 + q / ( 2 n ) ≤ 5 / 8 < 2 / 3 1/2 + q/(2n) \le5/8 < 2/3 1/2 + q / ( 2 n ) ≤ 5/8 < 2/3 . Averaging over the prior fixes a hard instance. ■
Proof of Theorem 1 Write S = L Δ / ε 2 S = L\Delta/\varepsilon^2 S = L Δ/ ε 2 and h ( n , κ ) = min { n κ , n 3 / 4 κ } h(n,\kappa) = \min\{\sqrt{n}\kappa,n^{3/4}\sqrt{\kappa}\} h ( n , κ ) = min { n κ , n 3/4 κ } . The common accuracy window is stronger than that of Lemma 21, so the n / 16 n/16 n /16 baseline is always available. If κ ≥ 2 18 n \kappa\ge2^{18}\sqrt{n} κ ≥ 2 18 n , Theorem 3 gives the accuracy term, which is the second branch of h h h .
For 64 ≤ κ ≤ 2 18 n 64 \le\kappa\le2^{18}\sqrt{n} 64 ≤ κ ≤ 2 18 n , the two-node coefficient obeys
λ = 2 − 39 L κ κ + 2 16 n ≥ L κ 5 ⋅ 2 55 n . \lambda= \frac{2^{-39}L\kappa}{\kappa+2^{16}\sqrt{n}} \ge\frac{L\kappa}{5\cdot2^{55}\sqrt{n}}. λ = κ + 2 16 n 2 − 39 L κ ≥ 5 ⋅ 2 55 n L κ .
If its path window holds, Corollary 16 and the floor bound give
E N ≥ 133 c ⋆ 2 6144 ⋅ 4096 n λ Δ ε 2 ≥ 133 c ⋆ 2 15 ⋅ 2 78 n κ S ≥ 133 c ⋆ 2 15 ⋅ 2 78 h ( n , κ ) S . \mathbb{E}N \ge\frac{133c_{\star}^{2}}{6144\cdot4096}\frac{n\lambda\Delta}{\varepsilon^{2}} \ge\frac{133c_{\star}^{2}}{15\cdot2^{78}}\sqrt{n\kappa}S \ge\frac{133c_{\star}^{2}}{15\cdot2^{78}}h(n,\kappa)S. E N ≥ 6144 ⋅ 4096 133 c ⋆ 2 ε 2 nλ Δ ≥ 15 ⋅ 2 78 133 c ⋆ 2 nκ S ≥ 15 ⋅ 2 78 133 c ⋆ 2 h ( n , κ ) S .
If the path window fails, instead
h ( n , κ ) S ≤ n κ S < 5 ⋅ 2 67 c ⋆ 2 n . h(n,\kappa)S \le\sqrt{n\kappa}S < \frac{5\cdot2^{67}}{c_{\star}^{2}}n. h ( n , κ ) S ≤ nκ S < c ⋆ 2 5 ⋅ 2 67 n .
The indexed baseline then supplies the requested sum up to a universal constant. For 1 ≤ κ < 64 1\le\kappa<64 1 ≤ κ < 64 , use the primal family. In its path window, Lemma 20 gives
E N ≥ 133 c ⋆ 2 6144 ⋅ 4096 ⋅ 2 34 n S = 133 c ⋆ 2 3 ⋅ 2 57 n S . \mathbb{E}N \ge\frac{133c_{\star}^{2}}{6144\cdot4096\cdot2^{34}}\sqrt{n}S = \frac{133c_{\star}^{2}}{3\cdot2^{57}}\sqrt{n}S. E N ≥ 6144 ⋅ 4096 ⋅ 2 34 133 c ⋆ 2 n S = 3 ⋅ 2 57 133 c ⋆ 2 n S .
Since h ( n , κ ) ≤ 64 n h(n,\kappa)\le64\sqrt{n} h ( n , κ ) ≤ 64 n , this bounds the required accuracy term. If its path window fails, n S < 2 46 n / c ⋆ 2 \sqrt{n}S<2^{46}n/c_{\star}^{2} n S < 2 46 n / c ⋆ 2 , so h ( n , κ ) S < 2 52 n / c ⋆ 2 h(n,\kappa)S<2^{52}n/c_{\star}^{2} h ( n , κ ) S < 2 52 n / c ⋆ 2 , again covered by the indexed baseline. In a path window, the maximum of the baseline and accuracy lower bounds is at least half their sum. Outside it, the accuracy term is at most the displayed universal multiple of n n n . The common constant c = c ⋆ 2 / 2 96 c=c_{\star}^{2}/2^{96} c = c ⋆ 2 / 2 96 is valid in all cases. Both lower bounds are worst cases over instances with the same public parameters.
The confidence statements of the three families and the baseline give the same rate with this smaller common constant. All components are C 1 , 1 C^{1,1} C 1 , 1 and μ \mu μ -strongly concave, have y ⋆ ( 0 ) = 0 y^{\star}(0)=0 y ⋆ ( 0 ) = 0 , and have unique responses on common exact legal domains. The long-path dimension is (16); the intermediate and primal dimensions were proved above and are polynomial since M = O ( S ) M=O(S) M = O ( S ) . For G 0 G_{0} G 0 , the long-path family has G 0 ≤ 1025 Δ G_{0}\le1025\Delta G 0 ≤ 1025Δ , the primal family has G 0 ≤ Δ G_{0}\le\Delta G 0 ≤ Δ , and the intermediate family has the bound of Corollary 16.
A Normalized-Start Upper Bound for Every Condition Number
We use the stochastic projected damped extragradient (SPDE) update of Zhang et al. [26 ] and the solver for the mean operator of Alacaoglu and Malitsky [1 ] and Luo et al. [12 ] . Fixed differentiable components satisfy (2), with mean μ \mu μ -strong concavity, n ≥ 2 n\ge2 n ≥ 2 , κ ≥ 1 \kappa\ge1 κ ≥ 1 and S = L Δ / ε 2 ≥ 2 S=L\Delta/\varepsilon^{2}\ge2 S = L Δ/ ε 2 ≥ 2 . The finite supplied pair u i n = ( x i n , y i n ) u_{\mathrm{in}}=(x_{\mathrm{in}},y_{\mathrm{in}}) u in = ( x in , y in ) obeys V ( x i n ) − V ⋆ ≤ Δ V(x_{\mathrm{in}})-V_{\star}\le\Delta V ( x in ) − V ⋆ ≤ Δ and ∥ ∇ F ( u i n ) ∥ 2 ≤ 2 L Δ \lVert\nabla F(u_{\mathrm{in}})\rVert^{2}\le2L\Delta ∥ ∇ F ( u in ) ∥ 2 ≤ 2 L Δ ; in this appendix we write x x x for the primal variable and M \mathcal{M} M for the Moreau envelope below; as in the lower-bound appendices, symbols defined in Appendices E and F are local. Proximal regularizers use exact arithmetic; each gradient call queries one component F i F_i F i .
Algorithm 1 Normalized-start method of Theorem 22
Require: $L$, $\mu$, $\Delta$, $\varepsilon$; a start $u_{\mathrm{in}}=(x_{\mathrm{in}},y_{\mathrm{in}})$ with $V(x_{\mathrm{in}})-V_* \le \Delta$ and $\|\nabla F(u_{\mathrm{in}})\|^2 \le 2L\Delta$
1: set the parameters (100), (112) and (114), and the budgets $j_w$ of (110) and $j_p$ of (116)
2: $u_w \gets$ endpoint of $j_w$ L-SVRE iterations (108) for $Q_{x_{\mathrm{in}}}$, from $u_{\mathrm{in}}$
3: $(x_0,y_0)\gets u_w,\quad z_0\gets x_{\mathrm{in}},\quad v_0\gets 0$
4: for $t=0,\ldots,T-1$ do
5: apply the updates (101), whose gradient requests $j=3t,3t+1,3t+2$ use
$\widehat g_j=\nabla F(u_j)$ if $j\equiv 0\pmod q$; otherwise (111) with $b$ fresh uniform indices
6: end for
7: draw $J$ uniformly from $\{0,\ldots,T-1\}$
8: $\widehat u\gets$ endpoint of $j_p$ L-SVRE iterations for $Q_{z_J}$ with fresh randomness, from $(x_{J+1},y_{J+1})$
9: return the primal coordinate $\widehat x$ of $\widehat u$
Theorem 22 (Normalized-start algorithm for all condition numbers). Under these assumptions, a randomized component-gradient algorithm achieves E ∥ ∇ V ( x ^ ) ∥ 2 ≤ ε 2 \mathbb{E}\|\nabla V(\widehat{x})\|^2\le\varepsilon^2 E ∥∇ V ( x ) ∥ 2 ≤ ε 2 with expected cost at most
C N [ n + n κ S + ( n + n κ ) log ( 2 κ ) ] . C_N\left[n+\sqrt{n\kappa}S+(n+\sqrt{n\kappa})\log(2\kappa)\right]. C N [ n + nκ S + ( n + nκ ) log ( 2 κ ) ] . for a universal constant C N C_N C N . The iteration budgets are finite and fixed from supplied parameters. This statement holds for every κ ≥ 1 \kappa\ge1 κ ≥ 1 .
We prove the theorem by initializing the proximal energy, running a fixed SPDE budget with recursive estimates, and refining one uniformly selected endpoint (Algorithm 1). All three stages use component gradients.
The component inequality is
n − 1 ∑ i = 1 n ∥ ∇ F i ( u ) − ∇ F i ( u ′ ) ∥ 2 ≤ L 2 ∥ u − u ′ ∥ 2 . (96) n^{-1}\sum_{i=1}^{n}\|\nabla F_i(u)-\nabla F_i(u')\|^2\le L^2\|u-u'\|^2. \tag*{(96)} n − 1 i = 1 ∑ n ∥∇ F i ( u ) − ∇ F i ( u ′ ) ∥ 2 ≤ L 2 ∥ u − u ′ ∥ 2 . (96)
Unconstrained Proximal Geometry
Strong concavity makes F ( x , ⋅ ) F(x,\cdot) F ( x , ⋅ ) coercive from above, so its maximum is attained uniquely at y ∗ ( x ) y^*(x) y ∗ ( x ) . Joint mean L L L -smoothness follows from Jensen’s inequality in (96) . The two dual optimality conditions imply ∥ y ∗ ( x ) − y ∗ ( x ′ ) ∥ ≤ κ ∥ x − x ′ ∥ \|y^*(x)-y^*(x')\|\le\kappa\|x-x'\| ∥ y ∗ ( x ) − y ∗ ( x ′ ) ∥ ≤ κ ∥ x − x ′ ∥ . Consequently V V V is differentiable, L ( 1 + κ ) L(1+\kappa) L ( 1 + κ ) -smooth and L L L -weakly convex (cf. Lin et al., [10 ] , Lemma 4.3, stated there for bounded dual sets). Its infimum V ∗ > − ∞ V_*>-\infty V ∗ > − ∞ by the supplied primal gap. Define the proximal objective Q z Q_z Q z , the Moreau envelope M \mathcal{M} M of V V V (Davis and Drusvyatskiy, [5 ] ), and the proximal saddle u z ∗ u_z^* u z ∗ by
Q z ( x , y ) = F ( x , y ) + L ∥ x − z ∥ 2 , M ( z ) = min x { V ( x ) + L ∥ x − z ∥ 2 } , Q_z(x,y)=F(x,y)+L\|x-z\|^2,\qquad\mathcal{M}(z)=\min_x\{V(x)+L\|x-z\|^2\}, Q z ( x , y ) = F ( x , y ) + L ∥ x − z ∥ 2 , M ( z ) = x min { V ( x ) + L ∥ x − z ∥ 2 } ,
u z ∗ = ( x z ∗ , y z ∗ ) = ( arg min x { V ( x ) + L ∥ x − z ∥ 2 } , y ∗ ( x z ∗ ) ) , u_z^*=(x_z^*,y_z^*)=\left(\mathop{\arg\min}_x\{V(x)+L\|x-z\|^2\},y^*(x_z^*)\right), u z ∗ = ( x z ∗ , y z ∗ ) = ( arg min x { V ( x ) + L ∥ x − z ∥ 2 } , y ∗ ( x z ∗ ) ) ,
inf M = V ∗ , ∇ M ( z ) = 2 L ( z − x z ∗ ) = ∇ x F ( u z ∗ ) , ∇ y F ( u z ∗ ) = 0. (97) \inf\mathcal{M}=V_*,\qquad\nabla\mathcal{M}(z)=2L(z-x_z^*)=\nabla_xF(u_z^*),\qquad\nabla_yF(u_z^*)=0. \tag*{(97)} inf M = V ∗ , ∇ M ( z ) = 2 L ( z − x z ∗ ) = ∇ x F ( u z ∗ ) , ∇ y F ( u z ∗ ) = 0. (97)
The proximal objective is coercive and L L L -strongly convex, hence has a unique minimizer. Its first-order condition and dual optimality make u z ∗ u_z^* u z ∗ the unique unconstrained saddle. Strong monotonicity of the mean proximal saddle operator gives
L ∥ x z ∗ − x z ′ ∗ ∥ 2 + μ ∥ y z ∗ − y z ′ ∗ ∥ 2 ≤ 2 L ⟨ z − z ′ , x z ∗ − x z ′ ∗ ⟩ . L\|x_z^*-x_{z'}^*\|^2+\mu\|y_z^*-y_{z'}^*\|^2\le2L\langle z-z',x_z^*-x_{z'}^*\rangle. L ∥ x z ∗ − x z ′ ∗ ∥ 2 + μ ∥ y z ∗ − y z ′ ∗ ∥ 2 ≤ 2 L ⟨ z − z ′ , x z ∗ − x z ′ ∗ ⟩ .
Young’s inequality therefore yields
∥ x z ∗ − x z ′ ∗ ∥ ≤ 2 ∥ z − z ′ ∥ , ∥ y z ∗ − y z ′ ∗ ∥ ≤ 2 L / μ ∥ z − z ′ ∥ , Lip ( ∇ M ) ≤ 6 L . (98) \lVert x_z^* - x_{z'}^* \rVert\le2\lVert z-z'\rVert,\qquad\lVert y_z^* - y_{z'}^* \rVert\le\sqrt{2L/\mu}\lVert z-z'\rVert,\qquad\operatorname{Lip}(\nabla\mathcal{M}) \le6L. \tag*{(98)} ∥ x z ∗ − x z ′ ∗ ∥ ≤ 2 ∥ z − z ′ ∥ , ∥ y z ∗ − y z ′ ∗ ∥ ≤ 2 L / μ ∥ z − z ′ ∥ , Lip ( ∇ M ) ≤ 6 L . (98)
Pathwise SPDE Descent and Motion Bounds
Set
s = 2 L μ , h = ( 192 L ) − 1 , d = 1 / 64 , α = ( 1 + h s / 16 ) − 1 , s=\sqrt{2L\mu},\qquad h=(192L)^{-1},\qquad d=1/64,\qquad\alpha=(1+hs/16)^{-1}, s = 2 Lμ , h = ( 192 L ) − 1 , d = 1/64 , α = ( 1 + h s /16 ) − 1 ,
κ = h ( 8 L − s ) 16 + h s , β = h s / 4096 , C = 140. (99) \kappa=\frac{h(8L-s)}{16+hs},\qquad\beta=hs/4096,\qquad C=140. \tag*{(99)} κ = 16 + h s h ( 8 L − s ) , β = h s /4096 , C = 140. (99)
From any initial state, use three gradient requests g ^ j \widehat g_j g j at points u j u_j u j per iteration:
u 3 t = ( x t , y t ) , x ~ t = x t − h [ g ^ x , 3 t + 2 L ( x t − z t ) ] , u_{3t}=(x_t,y_t),\qquad\widetilde{x}_t=x_t-h[\widehat g_{x,3t}+2L(x_t-z_t)], u 3 t = ( x t , y t ) , x t = x t − h [ g x , 3 t + 2 L ( x t − z t )] ,
y ~ t = y t + h ( g ^ y , 3 t + v t ) , u 3 t + 1 = ( x ~ t , y ~ t ) , \widetilde{y}_t=y_t+h(\widehat g_{y,3t}+v_t),\qquad u_{3t+1}=(\widetilde{x}_t,\widetilde{y}_t), y t = y t + h ( g y , 3 t + v t ) , u 3 t + 1 = ( x t , y t ) ,
v ‾ t = α v t + κ g ^ y , 3 t + 1 , x t + 1 = x t − h [ g ^ x , 3 t + 1 + 2 L ( x ~ t − z t ) ] , \overline{v}_t=\alpha v_t+\kappa\widehat g_{y,3t+1},\qquad x_{t+1}=x_t-h[\widehat g_{x,3t+1}+2L(\widetilde{x}_t-z_t)], v t = α v t + κ g y , 3 t + 1 , x t + 1 = x t − h [ g x , 3 t + 1 + 2 L ( x t − z t )] ,
y t + 1 = y t + h ( g ^ y , 3 t + 1 + v ‾ t ) , u 3 t + 2 = ( x t + 1 , y t + 1 ) , y_{t+1}=y_t+h(\widehat g_{y,3t+1}+\overline{v}_t),\qquad u_{3t+2}=(x_{t+1},y_{t+1}), y t + 1 = y t + h ( g y , 3 t + 1 + v t ) , u 3 t + 2 = ( x t + 1 , y t + 1 ) ,
v t + 1 = α v t + κ g ^ y , 3 t + 2 , z t + 1 = z t + β ( x t + 1 − z t ) . (100) v_{t+1}=\alpha v_t+\kappa\widehat g_{y,3t+2},\qquad z_{t+1}=z_t+\beta(x_{t+1}-z_t). \tag*{(100)} v t + 1 = α v t + κ g y , 3 t + 2 , z t + 1 = z t + β ( x t + 1 − z t ) . (100)
Always u 3 t = u 3 t − 1 u_{3t}=u_{3t-1} u 3 t = u 3 t − 1 for t ≥ 1 t\ge1 t ≥ 1 . Proximal gradients are evaluated by exact arithmetic, outside the gradient estimator.
For w = ( x , y , v ) w=(x,y,v) w = ( x , y , v ) at center z z z , write X = x − x z ∗ X=x-x_z^* X = x − x z ∗ , Y ′ = y − y z ∗ Y'=y-y_z^* Y ′ = y − y z ∗ , P = ∇ x Q z ( x , y ) P=\nabla_xQ_z(x,y) P = ∇ x Q z ( x , y ) , R = − ∇ y F ( x , y ) R=-\nabla_yF(x,y) R = − ∇ y F ( x , y ) and Q = R − v Q=R-v Q = R − v . The common potential is
E z ( w ) = − ( 1 − s 16 L ) ( Q z ( x , y ) − M ( z ) ) + ∥ P ∥ 2 + ∥ Q ∥ 2 L + s 16 L ⟨ v , Y ′ ⟩ + μ 256 ∥ Y ′ ∥ 2 , E_z(w)=-\left(1-\frac{s}{16L}\right)(Q_z(x,y)-\mathcal{M}(z))+\frac{\lVert P\rVert^2+\lVert Q\rVert^2}{L}+\frac{s}{16L}\langle v,Y'\rangle+\frac{\mu}{256}\lVert Y'\rVert^2, E z ( w ) = − ( 1 − 16 L s ) ( Q z ( x , y ) − M ( z )) + L ∥ P ∥ 2 + ∥ Q ∥ 2 + 16 L s ⟨ v , Y ′ ⟩ + 256 μ ∥ Y ′ ∥ 2 ,
H z ( w ) = E z ( w ) + s 256 L 2 ∥ v ∥ 2 , V ( z , w ) = M ( z ) − V ∗ + H z ( w ) . (101) H_z(w)=E_z(w)+\frac{s}{256L^2}\lVert v\rVert^2,\qquad\mathcal{V}(z,w)=\mathcal{M}(z)-V_*+H_z(w). \tag*{(101)} H z ( w ) = E z ( w ) + 256 L 2 s ∥ v ∥ 2 , V ( z , w ) = M ( z ) − V ∗ + H z ( w ) . (101)
Strong saddle inequalities give
Q z ( x , y ) − M ( z ) + ⟨ R , Y ′ ⟩ ≥ L 2 ∥ X ∥ 2 + μ 2 ∥ Y ′ ∥ 2 , Q z ( x , y ) − M ( z ) ≤ ∥ P ∥ 2 2 L − μ 2 ∥ Y ′ ∥ 2 . Q_z(x,y)-\mathcal{M}(z)+\langle R,Y'\rangle\ge\frac{L}{2}\lVert X\rVert^2+\frac{\mu}{2}\lVert Y'\rVert^2,\qquad Q_z(x,y)-\mathcal{M}(z)\le\frac{\lVert P\rVert^2}{2L}-\frac{\mu}{2}\lVert Y'\rVert^2. Q z ( x , y ) − M ( z ) + ⟨ R , Y ′ ⟩ ≥ 2 L ∥ X ∥ 2 + 2 μ ∥ Y ′ ∥ 2 , Q z ( x , y ) − M ( z ) ≤ 2 L ∥ P ∥ 2 − 2 μ ∥ Y ′ ∥ 2 .
Completing the Q , Y ′ Q,Y' Q , Y ′ square yields the following bound; in particular V ≥ M − V ∗ ≥ 0 \mathcal{V}\ge\mathcal{M}-V_*\ge0 V ≥ M − V ∗ ≥ 0 .
E z ( w ) ≥ ∥ P ∥ 2 2 L + s 32 ∥ X ∥ 2 + μ 2 ∥ Y ′ ∥ 2 + 1 L ∥ Q − s Y ′ / 32 ∥ 2 ≥ 0. (102) E_z(w)\ge\frac{\lVert P\rVert^2}{2L}+\frac{s}{32}\lVert X\rVert^2+\frac{\mu}{2}\lVert Y'\rVert^2+\frac{1}{L}\lVert Q-sY'/32\rVert^2\ge0. \tag*{(102)} E z ( w ) ≥ 2 L ∥ P ∥ 2 + 32 s ∥ X ∥ 2 + 2 μ ∥ Y ′ ∥ 2 + L 1 ∥ Q − s Y ′ /32 ∥ 2 ≥ 0. (102)
Let η j = g ^ j − ∇ F ( u j ) \eta_j=\widehat g_j-\nabla F(u_j) η j = g j − ∇ F ( u j ) and ( a t , b t , c t ) = ( η 3 t , η 3 t + 1 , η 3 t + 2 ) (a_t,b_t,c_t)=(\eta_{3t},\eta_{3t+1},\eta_{3t+2}) ( a t , b t , c t ) = ( η 3 t , η 3 t + 1 , η 3 t + 2 ) . The proof uses
v t + 1 ∘ = α v t + κ ∇ y F ( x t + 1 , y t + 1 ) , v t + 1 = v t + 1 ∘ + κ c y , t . (103) v^\circ_{t+1}=\alpha v_t+\kappa\nabla_yF(x_{t+1},y_{t+1}),\qquad v_{t+1}=v^\circ_{t+1}+\kappa c_{y,t}. \tag*{(103)} v t + 1 ∘ = α v t + κ ∇ y F ( x t + 1 , y t + 1 ) , v t + 1 = v t + 1 ∘ + κ c y , t . (103)
This is the true-gradient momentum at the realized endpoint. Fix center z t z_t z t when defining P t , Q t P_t,Q_t P t , Q t as above, and set
P t + = ∇ x Q z t ( u 3 t + 2 ) , Q t + = − ∇ y F ( u 3 t + 2 ) − v t + 1 ∘ , X t + = x t + 1 − x z t ∗ , Y t + = y t + 1 − y z t ∗ , P_t^+=\nabla_xQ_{z_t}(u_{3t+2}),\qquad Q_t^+=-\nabla_yF(u_{3t+2})-v^\circ_{t+1},\qquad X_t^+=x_{t+1}-x_{z_t}^*,\qquad Y_t^+=y_{t+1}-y_{z_t}^*, P t + = ∇ x Q z t ( u 3 t + 2 ) , Q t + = − ∇ y F ( u 3 t + 2 ) − v t + 1 ∘ , X t + = x t + 1 − x z t ∗ , Y t + = y t + 1 − y z t ∗ ,
I t = ∥ P t + − P t ∥ 2 + ∥ Q t + − Q t ∥ 2 , \mathcal{I}_t=\lVert P_t^+-P_t\rVert^2+\lVert Q_t^+-Q_t\rVert^2, I t = ∥ P t + − P t ∥ 2 + ∥ Q t + − Q t ∥ 2 ,
D t = h 4 ∥ P t + ∥ 2 + 24 β L ∥ Q t + ∥ 2 + 7 β L ∥ v t + 1 ∘ ∥ 2 + 32 μ β ∥ Y t + ∥ 2 + 4 β ( 16 L − s ) ∥ X t + ∥ 2 + I t 4 L . (104) D_t=\frac{h}{4}\lVert P_t^+\rVert^2+\frac{24\beta}{L}\lVert Q_t^+\rVert^2+\frac{7\beta}{L}\lVert v^\circ_{t+1}\rVert^2+32\mu\beta\lVert Y_t^+\rVert^2+4\beta(16L-s)\lVert X_t^+\rVert^2+\frac{\mathcal{I}_t}{4L}. \tag*{(104)} D t = 4 h ∥ P t + ∥ 2 + L 24 β ∥ Q t + ∥ 2 + L 7 β ∥ v t + 1 ∘ ∥ 2 + 32 μ β ∥ Y t + ∥ 2 + 4 β ( 16 L − s ) ∥ X t + ∥ 2 + 4 L I t . (104)
Lemma 23 (Pathwise drift and joint motion). For arbitrary realized vector errors and every finite initial state, with V t = V ( z t , w t ) V_t = \mathcal{V}(z_t,w_t) V t = V ( z t , w t ) , the updates satisfy
V t + 1 − V t ≤ − 1 4 D t − β 16 L ∥ ∇ M ( z t ) ∥ 2 + C L β ( ∥ a t ∥ 2 + ∥ b t ∥ 2 + ∥ c t ∥ 2 ) . (105) \mathcal{V}_{t+1}-\mathcal{V}_t \le-\frac{1}{4}D_t-\frac{\beta}{16L}\lVert\nabla\mathcal{M}(z_t)\rVert^2+\frac{C}{L\beta}\left(\lVert a_t\rVert^2+\lVert b_t\rVert^2+\lVert c_t\rVert^2\right). \tag*{(105)} V t + 1 − V t ≤ − 4 1 D t − 16 L β ∥ ∇ M ( z t ) ∥ 2 + L β C ( ∥ a t ∥ 2 + ∥ b t ∥ 2 + ∥ c t ∥ 2 ) . (105) ∥ u 3 t + 1 − u 3 t ∥ 2 + ∥ u 3 t + 2 − u 3 t + 1 ∥ 2 ≤ D t L β + ∥ a t ∥ 2 + ∥ b t ∥ 2 L 2 . (106) \lVert u_{3t+1}-u_{3t}\rVert^2+\lVert u_{3t+2}-u_{3t+1}\rVert^2 \le\frac{D_t}{L\beta}+\frac{\lVert a_t\rVert^2+\lVert b_t\rVert^2}{L^2}. \tag*{(106)} ∥ u 3 t + 1 − u 3 t ∥ 2 + ∥ u 3 t + 2 − u 3 t + 1 ∥ 2 ≤ L β D t + L 2 ∥ a t ∥ 2 + ∥ b t ∥ 2 . (106) Proof. The drift and joint-motion estimates are the deterministic inequalities of Zhang et al. [26 ] (Lemmas 11–12, Eqs. (116) and (121) }. Use their strongly concave substitution from the proof of Lemma 10, {xref}Eq. (100), as explicitly applied to these lemmas in the proof of {xref}Lemma 15: $G_{\mathrm{sc}}=Q_z$, $p_{\mathrm{sc}}=\mathcal{M}$, all normal-cone vectors are zero, the virtual momentum is $v_{t+1}^{\circ}$, and their $D_t^0$ is {xref}(105). Their numerical parameters are exactly {xref}(100)`, including C = 140 C=140 C = 140 .
The cited lemmas are stated on compact domains. Equations (97) –(98) and (102) supply the all-space saddles, envelope and nonnegative energy. For one finite realized step, choose compact primal and dual balls whose interiors contain its centers, its query points and the saddle path between the two centers; such balls exist by (98) . On these balls the projections are inactive, and the saddles and envelope values coincide with their all-space counterparts. The lower energy floor changes by an additive constant, which cancels in one-step differences. Their fixed-center Lemma 5 starts from arbitrary feasible states, and the subsequent center and endpoint estimates, including the endpoint expansion Eq. (87) and the Young bounds (117) –(120) , are algebraic in the realized errors. Hence both estimates hold for arbitrary finite initial states and correlated errors.
A Shared Solver with Mean Strong Monotonicity
We use loopless stochastic variance-reduced extragradient (L-SVRE) (Alacaoglu and Malitsky [1 ] ; Luo et al. [12 ] ), with the following self-contained guarantee.
Lemma 24 (L-SVRE contraction under mean strong monotonicity). Let n ≥ 2 n\ge2 n ≥ 2 and A = n − 1 ∑ i A i A=n^{-1}\sum_i A_i A = n − 1 ∑ i A i on R d \mathbb{R}^d R d have a zero z ∗ z^* z ∗ , be α \alpha α -strongly monotone, and satisfy n − 1 ∑ i ∥ A i ( z ) − A i ( z ′ ) ∥ 2 ≤ M s 2 ∥ z − z ′ ∥ 2 n^{-1}\sum_i\lVert A_i(z)-A_i(z')\rVert^2\le M_s^2\lVert z-z'\rVert^2 n − 1 ∑ i ∥ A i ( z ) − A i ( z ′ ) ∥ 2 ≤ M s 2 ∥ z − z ′ ∥ 2 . No component monotonicity is assumed. Starting from r 0 = w 0 = u r_0=w_0=u r 0 = w 0 = u , pay n n n calls for A ( w 0 ) A(w_0) A ( w 0 ) and, with a fresh uniform index i i i , perform
r ˉ = ( 1 − p s ) r + p s w , b = r ˉ − τ A ( w ) , \bar r=(1-p_s)r+p_sw,\qquad b=\bar r-\tau A(w), r ˉ = ( 1 − p s ) r + p s w , b = r ˉ − τ A ( w ) , v s = A ( w ) + A i ( b ) − A i ( w ) , r + = r ˉ − τ v s , v_s=A(w)+A_i(b)-A_i(w),\qquad r^+=\bar r-\tau v_s, v s = A ( w ) + A i ( b ) − A i ( w ) , r + = r ˉ − τ v s , p s = 1 2 n , τ = 1 4 n M s . (107) p_s=\frac{1}{2n},\qquad\tau=\frac{1}{4\sqrt{n}M_s}. \tag*{(107)} p s = 2 n 1 , τ = 4 n M s 1 . (107) Independently refresh w + = r + w^+=r^+ w + = r + with probability p s p_s p s , paying n n n calls for its mean, and otherwise retain w w w . Then, for every j ≥ 0 j\ge0 j ≥ 0 ,
E ∥ r j − z ∗ ∥ 2 ≤ 4 ( 1 − θ ) j ∥ u − z ∗ ∥ 2 , θ − 1 = 4 n + 8 n M s / α . (108) \mathbb{E}\lVert r_j-z^*\rVert^2\le4(1-\theta)^j\lVert u-z^*\rVert^2,\qquad\theta^{-1}=4n+8\sqrt{n}M_s/\alpha. \tag*{(108)} E ∥ r j − z ∗ ∥ 2 ≤ 4 ( 1 − θ ) j ∥ u − z ∗ ∥ 2 , θ − 1 = 4 n + 8 n M s / α . (108)
Each iteration costs at most 2 + n p s = 5 / 2 2+n_{\mathrm{ps}}=5/2 2 + n ps = 5/2 calls in conditional expectation; the guarantee holds conditionally on preceding history, with fresh randomness.
Proof Condition on ( r , w ) (r,w) ( r , w ) and write p = p s p=p_s p = p s , t = τ α t=\tau\alpha t = τ α , R = ∥ r − z ∗ ∥ 2 R=\lVert r-z^*\rVert^2 R = ∥ r − z ∗ ∥ 2 , W = ∥ w − z ∗ ∥ 2 W=\lVert w-z^*\rVert^2 W = ∥ w − z ∗ ∥ 2 . Unbiasedness gives E v s = A ( b ) \mathbb{E}v_s=A(b) E v s = A ( b ) and E ∥ r + − b ∥ 2 ≤ τ 2 M s 2 ∥ b − w ∥ 2 \mathbb{E}\lVert r^+-b\rVert^2\le\tau^2M_s^2\lVert b-w\rVert^2 E ∥ r + − b ∥ 2 ≤ τ 2 M s 2 ∥ b − w ∥ 2 . The identity
E R + = ∥ r ˉ − z ∗ ∥ 2 − 2 τ ⟨ A ( b ) , b − z ∗ ⟩ − ∥ r ˉ − b ∥ 2 + E ∥ r + − b ∥ 2 \mathbb{E}R^+=\lVert\bar r-z^*\rVert^2-2\tau\langle A(b),b-z^*\rangle-\lVert\bar r-b\rVert^2+\mathbb{E}\lVert r^+-b\rVert^2 E R + = ∥ r ˉ − z ∗ ∥ 2 − 2 τ ⟨ A ( b ) , b − z ∗ ⟩ − ∥ r ˉ − b ∥ 2 + E ∥ r + − b ∥ 2
and R + ≤ 2 ∥ b − z ∗ ∥ 2 + 2 ∥ r + − b ∥ 2 R^+\le2\lVert b-z^*\rVert^2+2\lVert r^+-b\rVert^2 R + ≤ 2 ∥ b − z ∗ ∥ 2 + 2 ∥ r + − b ∥ 2 imply
( 1 + t ) E R + ≤ ∥ r ˉ − z ∗ ∥ 2 − ∥ r ˉ − b ∥ 2 + ( 1 + 2 t ) τ 2 M s 2 ∥ b − w ∥ 2 . (1+t)\mathbb{E}R^+\le\lVert\bar r-z^*\rVert^2-\lVert\bar r-b\rVert^2+(1+2t)\tau^2M_s^2\lVert b-w\rVert^2. ( 1 + t ) E R + ≤ ∥ r ˉ − z ∗ ∥ 2 − ∥ r ˉ − b ∥ 2 + ( 1 + 2 t ) τ 2 M s 2 ∥ b − w ∥ 2 .
Mean Lipschitzness and strong monotonicity give α ≤ M s \alpha\le M_s α ≤ M s ; hence p , t ≤ 1 / 4 p,t\le1/4 p , t ≤ 1/4 and τ 2 M s 2 = p / 8 \tau^2M_s^2=p/8 τ 2 M s 2 = p /8 . Expand ∥ r ˉ − z ∗ ∥ 2 = ( 1 − p ) R + p W − p ( 1 − p ) ∥ r − w ∥ 2 \lVert\bar r-z^*\rVert^2=(1-p)R+pW-p(1-p)\lVert r-w\rVert^2 ∥ r ˉ − z ∗ ∥ 2 = ( 1 − p ) R + p W − p ( 1 − p ) ∥ r − w ∥ 2 and use ∥ b − w ∥ 2 ≤ 2 ∥ b − r ˉ ∥ 2 + 2 ( 1 − p ) 2 ∥ r − w ∥ 2 \lVert b-w\rVert^2\le2\lVert b-\bar r\rVert^2+2(1-p)^2\lVert r-w\rVert^2 ∥ b − w ∥ 2 ≤ 2 ∥ b − r ˉ ∥ 2 + 2 ( 1 − p ) 2 ∥ r − w ∥ 2 . Both motion coefficients are nonpositive, so
( 1 + t ) E R + ≤ ( 1 − p ) R + p W . (1+t)\mathbb{E}R^+\le(1-p)R+pW. ( 1 + t ) E R + ≤ ( 1 − p ) R + p W .
Set ω = 2 ( p + t ) / [ ( 2 p + t ) ( 1 − p ) ] \omega=2(p+t)/[(2p+t)(1-p)] ω = 2 ( p + t ) / [( 2 p + t ) ( 1 − p )] , Λ = R + ω W \Lambda=R+\omega W Λ = R + ω W and θ = p t / [ 2 ( p + t ) ] \theta=pt/[2(p+t)] θ = pt / [ 2 ( p + t )] . Since E W + = p E R + + ( 1 − p ) W \mathbb{E}W^+=p\mathbb{E}R^++(1-p)W E W + = p E R + + ( 1 − p ) W , the relative decreases of the R R R and ω W \omega W ω W coefficients are
t ( p + t ) ( 2 p + t ) ( 1 + t ) ≥ θ , p t ( 1 + p + 2 t ) 2 ( p + t ) ( 1 + t ) ≥ θ . \frac{t(p+t)}{(2p+t)(1+t)}\ge\theta,\qquad
\frac{pt(1+p+2t)}{2(p+t)(1+t)}\ge\theta. ( 2 p + t ) ( 1 + t ) t ( p + t ) ≥ θ , 2 ( p + t ) ( 1 + t ) pt ( 1 + p + 2 t ) ≥ θ .
The first inequality is equivalent to t [ 3 p − 2 p 2 + ( 2 − p ) t ] ≥ 0 t[3p-2p^2+(2-p)t]\ge0 t [ 3 p − 2 p 2 + ( 2 − p ) t ] ≥ 0 ; the second is immediate. Thus
E Λ + ≤ ( 1 − θ ) Λ . \mathbb{E}\Lambda^+\le(1-\theta)\Lambda. E Λ + ≤ ( 1 − θ ) Λ.
As 1 + ω ≤ 1 + 2 / ( 1 − p ) ≤ 11 / 3 < 4 1+\omega\le1+2/(1-p)\le11/3<4 1 + ω ≤ 1 + 2/ ( 1 − p ) ≤ 11/3 < 4 , iteration proves (108) . ▪
For SPDE, use A i = ( ∇ x F i + 2 L ( x − z ) , − ∇ y F i ) A_i=(\nabla_xF_i+2L(x-z),-\nabla_yF_i) A i = ( ∇ x F i + 2 L ( x − z ) , − ∇ y F i ) . Minkowski’s inequality gives RMS constant M s = 3 L M_s=3L M s = 3 L ; proximal geometry gives mean curvature α = μ \alpha=\mu α = μ , and the zero of the operator is u z ∗ u_z^* u z ∗ . Thus θ − 1 = 4 ( n + 6 n κ ) \theta^{-1}=4(n+6\sqrt{n\kappa}) θ − 1 = 4 ( n + 6 nκ ) in (108) .
For a normalized start, use an initial proximal solve for Q x i n = F + L ∥ x − x i n ∥ 2 Q_{x_{\mathrm{in}}}=F+L\lVert x-x_{\mathrm{in}}\rVert^2 Q x in = F + L ∥ x − x in ∥ 2 . Its unique saddle u ∗ = ( x ∗ , y ∗ ) u_*=(x_*,y_*) u ∗ = ( x ∗ , y ∗ ) exists by (97) using only V ∗ > − ∞ V_*>-\infty V ∗ > − ∞ , and ∇ Q x i n ( u ∗ ) = 0 \nabla Q_{x_{\mathrm{in}}}(u_*)=0 ∇ Q x in ( u ∗ ) = 0 . At u i n u_{\mathrm{in}} u in the regularizer gradient vanishes, so strong monotonicity implies the deterministic bound
∥ u i n − u ∗ ∥ 2 ≤ ∥ ∇ F ( u i n ) ∥ 2 μ 2 ≤ 2 κ 2 Δ L . \lVert u_{\mathrm{in}}-u_*\rVert^2\le\frac{\lVert\nabla F(u_{\mathrm{in}})\rVert^2}{\mu^2}\le\frac{2\kappa^2\Delta}{L}. ∥ u in − u ∗ ∥ 2 ≤ μ 2 ∥ ∇ F ( u in ) ∥ 2 ≤ L 2 κ 2 Δ .
Run (107) on the whole space for
j w = ⌈ 4 ( n + 6 n κ ) log ( 512 κ 2 ) ⌉ , E ∥ u w − u ∗ ∥ 2 ≤ Δ 64 L . (109) j_w=\left\lceil4(n+6\sqrt{n\kappa})\log(512\kappa^2)\right\rceil,\qquad\mathbb{E}\lVert u_w-u_*\rVert^2\le\frac{\Delta}{64L}. \tag*{(109)} j w = ⌈ 4 ( n + 6 nκ ) log ( 512 κ 2 ) ⌉ , E ∥ u w − u ∗ ∥ 2 ≤ 64 L Δ . (109)
By (108) , the cost including the snapshot is O ( ( n + n κ ) log ( 2 κ ) ) O((n+\sqrt{n\kappa})\log(2\kappa)) O (( n + nκ ) log ( 2 κ )) , with all budgets supplied.
Initialize at u w u_w u w , with
z 0 = x i n , v 0 = 0. z_0=x_{\mathrm{in}},\qquad v_0=0. z 0 = x in , v 0 = 0.
Since Q x i n ( u ∗ ) = M ( x i n ) Q_{x_{\mathrm{in}}}(u_*)=\mathcal{M}(x_{\mathrm{in}}) Q x in ( u ∗ ) = M ( x in ) , joint 3 L 3L 3 L -smoothness and ∇ Q x i n ( u ∗ ) = 0 \nabla Q_{x_{\mathrm{in}}}(u_*)=0 ∇ Q x in ( u ∗ ) = 0 imply
∣ Q x i n ( u w ) − M ( x i n ) ∣ ≤ ( 3 L / 2 ) ∥ u w − u ∗ ∥ 2 , ∥ ∇ Q x i n ( u w ) ∥ 2 ≤ 9 L 2 ∥ u w − u ∗ ∥ 2 . \left|Q_{x_{\mathrm{in}}}(u_w)-\mathcal{M}(x_{\mathrm{in}})\right|\le(3L/2)\lVert u_w-u_*\rVert^2,\qquad
\lVert\nabla Q_{x_{\mathrm{in}}}(u_w)\rVert^2\le9L^2\lVert u_w-u_*\rVert^2. ∣ Q x in ( u w ) − M ( x in ) ∣ ≤ ( 3 L /2 ) ∥ u w − u ∗ ∥ 2 , ∥ ∇ Q x in ( u w ) ∥ 2 ≤ 9 L 2 ∥ u w − u ∗ ∥ 2 .
Using (102) gives
0 ≤ H z 0 ( u w , 0 ) ≤ 11 L ∥ u w − u ∗ ∥ 2 , E V 0 ≤ Δ + 11 Δ / 64 < 2 Δ . 0\le H_{z_0}(u_w,0)\le11L\lVert u_w-u_*\rVert^2,\qquad
\mathbb{E}V_0\le\Delta+11\Delta/64<2\Delta. 0 ≤ H z 0 ( u w , 0 ) ≤ 11 L ∥ u w − u ∗ ∥ 2 , E V 0 ≤ Δ + 11Δ/64 < 2Δ.
Main-stage batches are fresh after this fixed-budget initialization.
Recursive Gradient Estimation with Full Gradient Refreshes
Write A m = ∑ t < m E D t A_m = \sum_{t<m} \mathbb{E}D_t A m = ∑ t < m E D t and E ( m ) = ∑ j < 3 m E ∥ η j ∥ 2 E(m) = \sum_{j<3m} \mathbb{E}\lVert\eta_j\rVert^2 E ( m ) = ∑ j < 3 m E ∥ η j ∥ 2 . Let H j \mathcal{H}_j H j be the pre-query history, which includes the chosen point, all previous estimates and all past randomness, but excludes the fresh batches at that request. Fixed finite budgets, finite-support component draws and Bernoulli refreshes leave finitely many states at each finite horizon, including initialization, so all conditional expectations below exist.
Full-gradient refreshes and recursive updates. At j ≡ 0 ( m o d q ) j \equiv0 \pmod q j ≡ 0 ( mod q ) , use g ^ j = ∇ F ( u j ) \widehat{g}_j = \nabla F(u_j) g j = ∇ F ( u j ) , at cost n n n . Otherwise, with b b b fresh uniform indices sampled with replacement, use the recursive estimate of Nguyen et al. [16 ] and Fang et al. [6 ] ,
g ^ j = g ^ j − 1 + 1 b ∑ i ∈ I j [ ∇ F i ( u j ) − ∇ F i ( u j − 1 ) ] . (110) \widehat{g}_j = \widehat{g}_{j-1} + \frac{1}{b}\sum_{i\in I_j}\left[\nabla F_i(u_j) - \nabla F_i(u_{j-1})\right]. \tag*{(110)} g j = g j − 1 + b 1 i ∈ I j ∑ [ ∇ F i ( u j ) − ∇ F i ( u j − 1 ) ] . (110)
Although E [ η j ∣ H j ] = η j − 1 \mathbb{E}[\eta_j \mid\mathcal{H}_j] = \eta_{j-1} E [ η j ∣ H j ] = η j − 1 , the centered new increment and (96) give
E [ ∥ η j ∥ 2 ∣ H j ] ≤ ∥ η j − 1 ∥ 2 + ( L 2 / b ) ∥ u j − u j − 1 ∥ 2 ; \mathbb{E}[\lVert\eta_j\rVert^2 \mid\mathcal{H}_j] \le\lVert\eta_{j-1}\rVert^2 + (L^2/b)\lVert u_j-u_{j-1}\rVert^2; E [∥ η j ∥ 2 ∣ H j ] ≤ ∥ η j − 1 ∥ 2 + ( L 2 / b ) ∥ u j − u j − 1 ∥ 2 ;
at refreshes η j = 0 \eta_j = 0 η j = 0 . Summing within epochs counts each motion at most q q q times. Since consecutive iterations share their boundary points, (106) gives E ( m ) ≤ q L b β A m + q b E ( m ) E(m) \le\frac{qL}{b\beta}A_m + \frac{q}{b}E(m) E ( m ) ≤ b β q L A m + b q E ( m ) . Set
A = 16 C / β 2 , q = max { 1 , ⌊ n / A ⌋ } , b = ⌈ A q ⌉ . (111) A = 16C/\beta^2,\qquad q = \max\{1,\lfloor\sqrt{n/A}\rfloor\},\qquad b = \lceil Aq\rceil. \tag*{(111)} A = 16 C / β 2 , q = max { 1 , ⌊ n / A ⌋} , b = ⌈ A q ⌉ . (111)
Since q / b ≤ 1 / A < 1 / 2 q/b \le1/A < 1/2 q / b ≤ 1/ A < 1/2 , the preceding recursion gives E ( m ) ≤ 2 q L A m / ( b β ) E(m) \le2qLA_m/(b\beta) E ( m ) ≤ 2 q L A m / ( b β ) and C E ( m ) / ( L β ) ≤ 2 C A m / ( A β 2 ) = A m / 8 CE(m)/(L\beta) \le2CA_m/(A\beta^2) = A_m/8 C E ( m ) / ( L β ) ≤ 2 C A m / ( A β 2 ) = A m /8 . When q = 1 q = 1 q = 1 , every request is exact, E ( m ) = 0 E(m) = 0 E ( m ) = 0 , and the difference batch is unused. Telescoping (105) from any initialization with E V 0 ≤ B 0 : = 2 Δ \mathbb{E}V_0 \le B_0 := 2\Delta E V 0 ≤ B 0 := 2Δ gives
E V m + 1 8 A m + β 16 L ∑ t < m E ∥ ∇ M ( z t ) ∥ 2 ≤ B 0 . (112) \mathbb{E}V_m + \frac{1}{8}A_m + \frac{\beta}{16L}\sum_{t<m}\mathbb{E}\lVert\nabla\mathcal{M}(z_t)\rVert^2 \le B_0. \tag*{(112)} E V m + 8 1 A m + 16 L β t < m ∑ E ∥ ∇ M ( z t ) ∥ 2 ≤ B 0 . (112)
Common Budgets, Dissipation, and Output Refinement
For the main stage choose
δ = ε / 16 , B 0 = 2 Δ , T = ⌈ 128 L B 0 β δ 2 ⌉ . (113) \delta= \varepsilon/16,\qquad B_0 = 2\Delta,\qquad T = \left\lceil\frac{128LB_0}{\beta\delta^2}\right\rceil. \tag*{(113)} δ = ε /16 , B 0 = 2Δ , T = ⌈ β δ 2 128 L B 0 ⌉ . (113)
Choose J J J independently and uniformly from { 0 , … , T − 1 } \{0,\ldots,T-1\} { 0 , … , T − 1 } . Then (112) gives
E ∥ ∇ M ( z J ) ∥ 2 ≤ δ 2 / 8 = ε 2 / 2048 , A T ≤ 16 Δ , E [ M ( z J ) − V ∗ ] ≤ 2 Δ . (114) \mathbb{E}\lVert\nabla\mathcal{M}(z_J)\rVert^2 \le\delta^2/8 = \varepsilon^2/2048,\qquad A_T \le16\Delta,\qquad\mathbb{E}[\mathcal{M}(z_J)-V_*] \le2\Delta. \tag*{(114)} E ∥ ∇ M ( z J ) ∥ 2 ≤ δ 2 /8 = ε 2 /2048 , A T ≤ 16Δ , E [ M ( z J ) − V ∗ ] ≤ 2Δ. (114)
The seed for the final proximal solve is u s e e d = ( x J + 1 , y J + 1 ) u_{\mathrm{seed}} = (x_{J+1},y_{J+1}) u seed = ( x J + 1 , y J + 1 ) at center z J z_J z J . The distance terms in (104) already give, since μ ≤ L \mu\le L μ ≤ L and s ≤ 2 L s \le2L s ≤ 2 L ,
D t ≥ 32 μ β ( ∥ X t + ∥ 2 + ∥ Y t + ∥ 2 ) = 32 μ β ∥ u 3 t + 2 − u z t ∗ ∥ 2 . D_t \ge32\mu\beta(\lVert X_t^+\rVert^2+\lVert Y_t^+\rVert^2) = 32\mu\beta\lVert u_{3t+2}-u_{zt}^*\rVert^2. D t ≥ 32 μ β (∥ X t + ∥ 2 + ∥ Y t + ∥ 2 ) = 32 μ β ∥ u 3 t + 2 − u z t ∗ ∥ 2 .
Here X t + , Y t + X_t^+,Y_t^+ X t + , Y t + are measured from the saddle at the center z t z_t z t . As A T / T ≤ β δ 2 / ( 16 L ) A_T/T \le\beta\delta^2/(16L) A T / T ≤ β δ 2 / ( 16 L ) , averaging over J J J yields
E ∥ u s e e d − u z J ∗ ∥ 2 ≤ A T 32 μ β T ≤ κ δ 2 512 L 2 ≤ κ 2 ε 2 L 2 . \mathbb{E}\lVert u_{\mathrm{seed}}-u_{z_J}^*\rVert^2 \le\frac{A_T}{32\mu\beta T} \le\frac{\kappa\delta^2}{512L^2} \le\frac{\kappa^2\varepsilon^2}{L^2}. E ∥ u seed − u z J ∗ ∥ 2 ≤ 32 μ β T A T ≤ 512 L 2 κ δ 2 ≤ L 2 κ 2 ε 2 .
Use fresh solver randomness at this center and seed, with the fixed budget
j p = ⌈ 4 ( n + 6 n κ ) log [ 1024 κ 2 ( 1 + κ ) 2 ] ⌉ , E ∥ u ~ − u z J ∗ ∥ 2 ≤ ε 2 256 L 2 ( 1 + κ ) 2 . (115) j_{\mathrm{p}}=\left\lceil4(n+6\sqrt{n\kappa})\log\left[1024\kappa^{2}(1+\kappa)^{2}\right]\right\rceil,\qquad\mathbb{E}\lVert\widetilde{u}-u^{*}_{z_J}\rVert^{2}\le\frac{\varepsilon^{2}}{256L^{2}(1+\kappa)^{2}}. \tag*{(115)} j p = ⌈ 4 ( n + 6 nκ ) log [ 1024 κ 2 ( 1 + κ ) 2 ] ⌉ , E ∥ u − u z J ∗ ∥ 2 ≤ 256 L 2 ( 1 + κ ) 2 ε 2 . (115)
Indeed, condition on the complete history of the main stage and J J J . Equation (108) bounds the conditional mean-square distance by 4 e − θ j p ∥ u s e e d − u z J ∗ ∥ 2 4e^{-\theta j_{\mathrm{p}}}\lVert u_{\mathrm{seed}}-u^{*}_{z_J}\rVert^{2} 4 e − θ j p ∥ u seed − u z J ∗ ∥ 2 ; then use 4 e − θ j p ≤ [ 256 κ 2 ( 1 + κ ) 2 ] − 1 4e^{-\theta j_{\mathrm{p}}}\le[256\kappa^{2}(1+\kappa)^{2}]^{-1} 4 e − θ j p ≤ [ 256 κ 2 ( 1 + κ ) 2 ] − 1 and average. This solve includes its own n n n -call snapshot.
Return the primal coordinate x ^ \widehat{x} x of u ^ \widehat{u} u . The geometry above gives ∇ V ( x z J ∗ ) = ∇ M ( z J ) \nabla V(x^{*}_{z_J})=\nabla\mathcal{M}(z_J) ∇ V ( x z J ∗ ) = ∇ M ( z J ) and Lip ( ∇ V ) ≤ L ( 1 + κ ) \operatorname{Lip}(\nabla V)\le L(1+\kappa) Lip ( ∇ V ) ≤ L ( 1 + κ ) , so pathwise
∥ ∇ V ( x ^ ) ∥ ≤ ∥ ∇ M ( z J ) ∥ + L ( 1 + κ ) ∥ u ^ − u z J ∗ ∥ . \lVert\nabla V(\widehat{x})\rVert\le\lVert\nabla\mathcal{M}(z_J)\rVert+L(1+\kappa)\lVert\widehat{u}-u^{*}_{z_J}\rVert. ∥ ∇ V ( x )∥ ≤ ∥ ∇ M ( z J )∥ + L ( 1 + κ ) ∥ u − u z J ∗ ∥ .
Squaring, taking expectations, and using (114) and (115) proves
E ∥ ∇ V ( x ^ ) ∥ 2 ≤ ( 1 / 1024 + 1 / 128 ) ε 2 ≤ ε 2 . \mathbb{E}\lVert\nabla V(\widehat{x})\rVert^{2}\le(1/1024+1/128)\varepsilon^{2}\le\varepsilon^{2}. E ∥ ∇ V ( x ) ∥ 2 ≤ ( 1/1024 + 1/128 ) ε 2 ≤ ε 2 .
Oracle Costs and Completion of the Upper Bounds
With integer rounding, a recursive estimator with refresh cost M 0 ≥ 1 M_{0}\ge1 M 0 ≥ 1 , cost two per paired difference, ratio parameter A ≥ 1 A\ge1 A ≥ 1 , q = max { 1 , ⌈ M 0 / A ⌉ } q=\max\{1,\lceil\sqrt{M_{0}/A}\rceil\} q = max { 1 , ⌈ M 0 / A ⌉} , and b = ⌈ A q ⌉ b=\lceil Aq\rceil b = ⌈ A q ⌉ has N r = ⌈ 3 T / q ⌉ N_{r}=\lceil3T/q\rceil N r = ⌈ 3 T / q ⌉ refreshes and cost
M 0 N r + 2 b ( 3 T − N r ) ≤ M 0 + 18 T min { M 0 , A M 0 } . (116) M_{0}N_{r}+2b(3T-N_{r})\le M_{0}+18T\min\{M_{0},\sqrt{AM_{0}}\}. \tag*{(116)} M 0 N r + 2 b ( 3 T − N r ) ≤ M 0 + 18 T min { M 0 , A M 0 } . (116)
For q ≥ 2 q\ge2 q ≥ 2 , use q ≥ 1 2 M 0 / A q\ge\frac{1}{2}\sqrt{M_{0}/A} q ≥ 2 1 M 0 / A , b ≤ 2 A M 0 b\le2\sqrt{AM_{0}} b ≤ 2 A M 0 and A M 0 ≤ M 0 / 2 \sqrt{AM_{0}}\le M_{0}/2 A M 0 ≤ M 0 /2 . For q = 1 q=1 q = 1 , the cost is 3 T M 0 3TM_{0} 3 T M 0 , while M 0 < 4 A M_{0}<4A M 0 < 4 A implies min { M 0 , A M 0 } ≥ M 0 / 2 \min\{M_{0},\sqrt{AM_{0}}\}\ge M_{0}/2 min { M 0 , A M 0 } ≥ M 0 /2 .
Taking M 0 = n M_{0}=n M 0 = n and A = 16 C / β 2 A=16C/\beta^{2} A = 16 C / β 2 in (116) bounds the main cost by n + 18 T min { n , A n } n+18T\min\{n,\sqrt{An}\} n + 18 T min { n , A n } . Since β − 1 = O ( κ ) \beta^{-1}=O(\sqrt{\kappa}) β − 1 = O ( κ ) , T = O ( κ S ) T=O(\sqrt{\kappa}S) T = O ( κ S ) and A = O ( κ ) A=O(\kappa) A = O ( κ ) , this is O ( n + n κ S ) O(n+\sqrt{n\kappa}S) O ( n + nκ S ) ; the ceiling contributes at most another O ( n ) O(n) O ( n ) term. Adding the initial proximal solve and the final refinement proves Theorem 22. The bound holds for every κ ≥ 1 \kappa\ge1 κ ≥ 1 because min { n , A n } ≤ A n \min\{n,\sqrt{An}\}\le\sqrt{An} min { n , A n } ≤ A n ; S ≥ 2 S\ge2 S ≥ 2 absorbs the ceiling.
For the explicit coefficient, use S = L Δ / ε 2 S=L\Delta/\varepsilon^{2} S = L Δ/ ε 2 and β − 1 = ( 3 ⋅ 2 18 / 2 ) κ \beta^{-1}=(3\cdot2^{18}/\sqrt{2})\sqrt{\kappa} β − 1 = ( 3 ⋅ 2 18 / 2 ) κ :
β − 1 < 2 20 κ , A < 2 52 κ , T = ⌈ 2 16 S / β ⌉ < 2 37 κ S . \beta^{-1}<2^{20}\sqrt{\kappa},\qquad A<2^{52}\kappa,\qquad T=\lceil2^{16}S/\beta\rceil<2^{37}\sqrt{\kappa}S. β − 1 < 2 20 κ , A < 2 52 κ , T = ⌈ 2 16 S / β ⌉ < 2 37 κ S .
Thus the cost of the main stage is at most n + 2 68 n κ S n+2^{68}\sqrt{n\kappa}S n + 2 68 nκ S . Each solver iteration costs at most 5 / 2 5/2 5/2 in expectation, including refreshes. Since
log ( 512 κ 2 ) ≤ 9 log ( 2 κ ) , log [ 1024 κ 2 ( 1 + κ ) 2 ] ≤ 12 log ( 2 κ ) , \log(512\kappa^{2})\le9\log(2\kappa),\qquad\log[1024\kappa^{2}(1+\kappa)^{2}]\le12\log(2\kappa), log ( 512 κ 2 ) ≤ 9 log ( 2 κ ) , log [ 1024 κ 2 ( 1 + κ ) 2 ] ≤ 12 log ( 2 κ ) ,
the warm and final solves together cost at most 2 n + 2 10 ( n + 6 n κ ) log ( 2 κ ) + 5 ≤ 2 12 ( n + n κ ) log ( 2 κ ) 2n+2^{10}(n+6\sqrt{n\kappa})\log(2\kappa)+5\le2^{12}(n+\sqrt{n\kappa})\log(2\kappa) 2 n + 2 10 ( n + 6 nκ ) log ( 2 κ ) + 5 ≤ 2 12 ( n + nκ ) log ( 2 κ ) , including both initial snapshots and both iteration ceilings. Consequently C N = 2 72 C_{N}=2^{72} C N = 2 72 is valid in Theorem 22.
Assembly over All Condition Numbers
Write h ( n , κ ) = min { n κ , n 3 / 4 κ } h(n,\kappa)=\min\{\sqrt{n\kappa},n^{3/4}\sqrt{\kappa}\} h ( n , κ ) = min { nκ , n 3/4 κ } . For 1 ≤ κ ≤ n 1\le\kappa\le\sqrt{n} 1 ≤ κ ≤ n , h = n κ h=\sqrt{n\kappa} h = nκ . For n < κ ≤ 2 18 n \sqrt{n}<\kappa\le2^{18}\sqrt{n} n < κ ≤ 2 18 n ,
n κ h ( n , κ ) = κ n ≤ 2 9 . \frac{\sqrt{n\kappa}}{h(n,\kappa)}=\sqrt{\frac{\kappa}{\sqrt{n}}}\le2^{9}. h ( n , κ ) nκ = n κ ≤ 2 9 .
Use Theorem 22 throughout 1 ≤ κ ≤ 2 18 n 1\le\kappa\le2^{18}\sqrt{n} 1 ≤ κ ≤ 2 18 n . As log 2 ( 2 + κ ) ≥ 1 \log^{2}(2+\kappa)\ge1 log 2 ( 2 + κ ) ≥ 1 and
Algorithm 2 Method of Theorem 3 for large condition numbers
Require: f i = F i / L f_i = F_i/L f i = F i / L , ν = μ / L \nu= \mu/L ν = μ / L , D 0 = Δ / L D_0 = \Delta/L D 0 = Δ/ L , e 0 = ε / L e_0 = \varepsilon/L e 0 = ε / L ; the parameters (20) and β = ( 1 − q ) / ( 1 + q ) \beta= (1-\sqrt{q})/(1+\sqrt{q}) β = ( 1 − q ) / ( 1 + q ) ; J J J of (122); K K K and α \alpha α of Appendix F.3
y 0 ← 0 y_0 \leftarrow0 y 0 ← 0 , z 1 ← 0 z_1 \leftarrow0 z 1 ← 0
for k = 1 , … , J k = 1,\ldots,J k = 1 , … , J do ▷ dual warm start, Appendix F.2
from y = 0 y = 0 y = 0 , run certified L-SVRE epochs (Lemma 25 ) on g k g_k g k of (120) until ∥ ∇ g k ( y k ) ∥ 2 ≤ 2 s η k \lVert\nabla g_k(y_k)\rVert^2 \le2s\eta_k ∥ ∇ g k ( y k ) ∥ 2 ≤ 2 s η k
z k + 1 ← y k + β ( y k − y k − 1 ) z_{k+1} \leftarrow y_k + \beta(y_k-y_{k-1}) z k + 1 ← y k + β ( y k − y k − 1 )
end for
w 1 = ( p 1 , y 1 ) ← ( 0 , y J ) w_1 = (p_1,y_1) \leftarrow(0,y_J) w 1 = ( p 1 , y 1 ) ← ( 0 , y J )
for t = 1 , … , T o u t = ⌈ 60 D 0 / e 0 2 ⌉ t = 1,\ldots,T_{\mathrm{out}} = \lceil60D_0/e_0^2\rceil t = 1 , … , T out = ⌈ 60 D 0 / e 0 2 ⌉ do ▷ outer steps, Appendices F.3–F.4
compute R t 2 = ∥ ∇ f ^ t ( w t ) ∥ 2 R_t^2 = \lVert\nabla\widehat{f}_t(w_t)\rVert^2 R t 2 = ∥ ∇ f t ( w t ) ∥ 2 with n n n calls; if R t = 0 R_t = 0 R t = 0 then return p t p_t p t
B t ← ν R t 2 / 72 B_t \leftarrow\nu R_t^2/72 B t ← ν R t 2 /72 , w ~ 0 = ( p ~ 0 , y ~ 0 ) ← w t \widetilde{w}_0 = (\widetilde{p}_0,\widetilde{y}_0) \leftarrow w_t w 0 = ( p 0 , y 0 ) ← w t , z 1 ← y t z_1 \leftarrow y_t z 1 ← y t
for k = 1 , … , K k = 1,\ldots,K k = 1 , … , K do
from w ~ k − 1 \widetilde{w}_{k-1} w k − 1 , run certified L-SVRE epochs on f ^ t , k \widehat{f}_{t,k} f t , k of (124) until ∥ ∇ f ^ t , k ( w ~ k ) ∥ 2 ≤ δ t , k \lVert\nabla\widehat{f}_{t,k}(\widetilde{w}_k)\rVert^2 \le\delta_{t,k} ∥ ∇ f t , k ( w k ) ∥ 2 ≤ δ t , k
z k + 1 ← y ~ k + β ( y ~ k − y ~ k − 1 ) z_{k+1} \leftarrow\widetilde{y}_k + \beta(\widetilde{y}_k-\widetilde{y}_{k-1}) z k + 1 ← y k + β ( y k − y k − 1 )
end for
w t + 1 ← w ~ K w_{t+1} \leftarrow\widetilde{w}_K w t + 1 ← w K ▷ ∥ ∇ f ^ t ( w ~ K ) ∥ 2 ≤ α R t 2 \lVert\nabla\widehat{f}_t(\widetilde{w}_K)\rVert^2 \le\alpha R_t^2 ∥ ∇ f t ( w K ) ∥ 2 ≤ α R t 2 by (125)
end for
return p t + 1 p_{t+1} p t + 1 for t t t drawn uniformly from { 1 , … , T o u t } \{1,\ldots,T_{\mathrm{out}}\} { 1 , … , T out }
log ( 2 κ ) ≤ 2 log 2 ( 2 + κ ) \log(2\kappa) \le2\log^2(2+\kappa) log ( 2 κ ) ≤ 2 log 2 ( 2 + κ ) , its cost is at most 2 83 [ n + h ( n , κ ) ( 1 + S ) ] log 2 ( 2 + κ ) 2^{83}[n+h(n,\kappa)(1+S)]\log^2(2+\kappa) 2 83 [ n + h ( n , κ ) ( 1 + S )] log 2 ( 2 + κ ) . For κ > 2 18 n \kappa> 2^{18}\sqrt{n} κ > 2 18 n , Appendix F proves the upper bound of Theorem 3 for every S > 0 S>0 S > 0 , with schedules that use only n n n , κ \kappa κ , S S S and T o u t = ⌈ 60 S ⌉ T_{\mathrm{out}}=\lceil60S\rceil T out = ⌈ 60 S ⌉ . Taking C = 2 83 + C H C=2^{83}+C_H C = 2 83 + C H , where C H C_H C H is the universal constant in (15), proves Theorem 2. Both branches terminate almost surely with finite expected cost, and their output satisfies the expected squared-gradient criterion. For a deterministic cap with success probability 2 / 3 2/3 2/3 , use target ε 2 / 6 \varepsilon^2/6 ε 2 /6 , cap at six times the resulting bound on the expected cost, and apply Markov’s inequality separately to cost and squared gradient norm. The two failure probabilities are at most 1 / 6 1/6 1/6 each.
Matching Upper Bound Including All Initialization Costs
We combine the shared L-SVRE solver with Lin et al. [9 ] and Zhang et al. [27 ] to prove (14)–(15). Algorithm 2 lists the steps; Appendices F.2–F.4 analyze them.
Residual Certification with the Shared Solver
Lemma 25 (Certified L-SVRE epochs). Under Lemma 24, restart from r = w = u r=w=u r = w = u for j 0 = ⌈ θ − 1 log 8 ⌉ j_0=\lceil\theta^{-1}\log8\rceil j 0 = ⌈ θ − 1 log 8 ⌉ updates per epoch, paying n n n calls to test ∥ A ( u + ) ∥ 2 ≤ δ \lVert A(u^+)\rVert^2 \le\delta ∥ A ( u + ) ∥ 2 ≤ δ afterward. If the test fails, restart from u + u^+ u + . For δ > 0 \delta>0 δ > 0 , this terminates almost surely with a certified output. Writing H = M s 2 ∥ u 0 − z ∗ ∥ 2 / δ H=M_s^2\lVert u_0-z^*\rVert^2/\delta H = M s 2 ∥ u 0 − z ∗ ∥ 2 / δ and log + 0 : = 0 \log^+0:=0 log + 0 := 0 , the cost Q c e r t Q_{\mathrm{cert}} Q cert from the given start satisfies
E Q c e r t ≤ [ 2 n + 5 2 ⌈ θ − 1 log 8 ⌉ ] [ 2 + ⌈ log 2 + H ⌉ ] (117) \mathbb{E}Q_{\mathrm{cert}} \le\left[2n+\frac{5}{2}\lceil\theta^{-1}\log8\rceil\right]\left[2+\lceil\log_2^+ H\rceil\right] \tag*{(117)} E Q cert ≤ [ 2 n + 2 5 ⌈ θ − 1 log 8 ⌉ ] [ 2 + ⌈ log 2 + H ⌉ ] (117) \(=O\left((n+\sqrt{n}M_{s}/\alpha)[1+\log^{+} H]\right)\)
Proof Each epoch halves conditional mean-squared distance by (109). Continue virtually after acceptance, and let N e p N_{\mathrm{ep}} N ep be the number of epochs. If N e p > r N_{\mathrm{ep}}>r N ep > r , the r r r th virtual output fails its residual test; mean Lipschitzness and Markov’s inequality therefore give P ( N e p > r ) ≤ min { 1 , H 2 − r } \mathbb{P}(N_{\mathrm{ep}}>r)\le\min\{1,H2^{-r}\} P ( N ep > r ) ≤ min { 1 , H 2 − r } for r ≥ 1 r\ge1 r ≥ 1 . Summation gives E N e p ≤ 2 + ⌈ log 2 + H ⌉ \mathbb{E}N_{\mathrm{ep}}\le2+\lceil\log_2^+ H\rceil E N ep ≤ 2 + ⌈ log 2 + H ⌉ and almost-sure termination. Each epoch costs at most C e p = 2 n + ( 5 / 2 ) j 0 C_{\mathrm{ep}}=2n+(5/2)j_0 C ep = 2 n + ( 5/2 ) j 0 in conditional expectation, including its initial snapshot, random refreshes, and final test. The epoch-start indicator is measurable before that epoch. Multiply its uniform conditional cost bound by this indicator and sum by the tower property to obtain E Q c e r t ≤ C e p E N e p \mathbb{E}Q_{\mathrm{cert}}\le C_{\mathrm{ep}}\mathbb{E}N_{\mathrm{ep}} E Q cert ≤ C ep E N ep .
Both applications below use α = s = n − 1 / 2 \alpha=s=n^{-1/2} α = s = n − 1/2 and M s ≤ 3 M_s\le3 M s ≤ 3 , so their uniform conditional epoch charge is at most
C e p H : = 2 n + 5 2 ⌈ 28 n log 8 ⌉ . (118) C_{\mathrm{ep}}^H:=2n+\frac{5}{2}\lceil28n\log8\rceil. \tag*{(118)} C ep H := 2 n + 2 5 ⌈ 28 n log 8 ⌉ . (118)
Initialization Cost and a Pathwise Certificate
We use the normalization and parameters of § 4 , and write f = F / L f=F/L f = F / L and V f ( p ) = max y f ( p , y ) = V ( p ) / L V_f(p)=\max_y f(p,y)=V(p)/L V f ( p ) = max y f ( p , y ) = V ( p ) / L . For g ( y ) = − f ( 0 , y ) g(y)=-f(0,y) g ( y ) = − f ( 0 , y ) let g ∗ = min y g ( y ) g^*=\min_y g(y) g ∗ = min y g ( y ) and B = D 0 / ν B=D_0/\nu B = D 0 / ν . Mean strong convexity and (5) imply
g ( 0 ) − g ∗ ≤ ∥ ∇ g ( 0 ) ∥ 2 2 ν ≤ B , ∥ y ∗ ( 0 ) ∥ 2 ≤ 2 B ν . g(0)-g^*\le\frac{\lVert\nabla g(0)\rVert^2}{2\nu}\le B,\qquad\lVert y^*(0)\rVert^2\le\frac{2B}{\nu}. g ( 0 ) − g ∗ ≤ 2 ν ∥ ∇ g ( 0 ) ∥ 2 ≤ B , ∥ y ∗ ( 0 ) ∥ 2 ≤ ν 2 B .
Starting at y 0 = z 1 = 0 y_0=z_1=0 y 0 = z 1 = 0 , set
g k ( y ) = g ( y ) + τ 2 ∥ y − z k ∥ 2 , η k = 2 9 B ( 1 − ρ ) k , z k + 1 = y k + β ( y k − y k − 1 ) , β = 1 − q 1 + q . (119) \begin{aligned}
g_k(y)&=g(y)+\frac{\tau}{2}\lVert y-z_k\rVert^2,\qquad&&\eta_k=\frac{2}{9}B(1-\rho)^k,\\
z_{k+1}&=y_k+\beta(y_k-y_{k-1}),\qquad&&\beta=\frac{1-\sqrt q}{1+\sqrt q}. \tag*{(119)}
\end{aligned} g k ( y ) z k + 1 = g ( y ) + 2 τ ∥ y − z k ∥ 2 , = y k + β ( y k − y k − 1 ) , η k = 9 2 B ( 1 − ρ ) k , β = 1 + q 1 − q . (119)
For each k k k , solve from y = 0 y=0 y = 0 using Lemma 25 until the computable condition ∥ ∇ g k ( y k ) ∥ 2 ≤ 2 s η k \lVert\nabla g_k(y_k)\rVert^2\le2s\eta_k ∥ ∇ g k ( y k ) ∥ 2 ≤ 2 s η k holds. Each subproblem has strong convexity s s s and RMS smoothness constant at most 1 + τ ≤ 2 1+\tau\le2 1 + τ ≤ 2 , so this certifies g k ( y k ) − min g k ≤ η k g_k(y_k)-\min g_k\le\eta_k g k ( y k ) − min g k ≤ η k .
The Catalyst estimate for strongly convex objectives [9 Proposition 5] states that the recurrence (119) , with these subproblem errors and 0 < ρ < q 0<\rho<\sqrt q 0 < ρ < q , satisfies
g ( y k ) − g ∗ ≤ 8 B ( q − ρ ) 2 ( 1 − ρ ) k + 1 ≤ 800 B q ( 1 − ρ ) k . (120) g(y_k)-g^*\le\frac{8B}{(\sqrt q-\rho)^2}(1-\rho)^{k+1}\le\frac{800B}{q}(1-\rho)^k. \tag*{(120)} g ( y k ) − g ∗ ≤ ( q − ρ ) 2 8 B ( 1 − ρ ) k + 1 ≤ q 800 B ( 1 − ρ ) k . (120)
The error sum in the proof of Proposition 5 is monotone in the initial error and potential, bounded here by B B B and 2 ( 1 − q ) B 2(1-\sqrt q)B 2 ( 1 − q ) B ; prescribing all errors with the same B B B therefore permits this supplied upper bound in place of the unknown initial gap.
Choose the deterministic number of accepted subproblems
J = ⌈ log ( 1600 / ( q ν 2 ) ) − log ( 1 − ρ ) ⌉ . (121) J=\left\lceil\frac{\log(1600/(q\nu^2))}{-\log(1-\rho)}\right\rceil. \tag*{(121)} J = ⌈ − log ( 1 − ρ ) log ( 1600/ ( q ν 2 )) ⌉ . (121)
Then (120) gives g ( y J ) − g ∗ ≤ ν D 0 / 2 g(y_J)-g^*\le\nu D_0/2 g ( y J ) − g ∗ ≤ ν D 0 /2 , so ∥ y J − y ∗ ( 0 ) ∥ 2 ≤ D 0 \lVert y_J-y^*(0)\rVert^2\le D_0 ∥ y J − y ∗ ( 0 ) ∥ 2 ≤ D 0 on every terminating run. Each certified subproblem terminates almost surely; the finite composition therefore yields the pathwise certificate (21) for the warm start y w = y J y_w=y_J y w = y J .
To charge the cold starts, (120) and strong convexity bound ∥ y k − y ∗ ( 0 ) ∥ 2 ≤ 1600 B / ( q ν ) \lVert y_k-y^*(0)\rVert^2\le1600B/(q\nu) ∥ y k − y ∗ ( 0 ) ∥ 2 ≤ 1600 B / ( q ν ) for all accepted k ≤ J k\le J k ≤ J . Since $0\le\beta\le 1$, ∥ z k − y ∗ ( 0 ) ∥ 2 ≤ 14400 B / ( q ν ) \lVert z_k-y^*(0)\rVert^2\le14400B/(q\nu) ∥ z k − y ∗ ( 0 ) ∥ 2 ≤ 14400 B / ( q ν ) . If y k ∗ = a r g m i n g k y_k^*=\mathop{\rm argmin} g_k y k ∗ = argmin g k , contraction of its proximal map under strong convexity gives $\lVert y_k^-y^ (0)\rVert\le \frac{\tau}{\nu+\tau}\lVert z_k-y^*(0)\rVert\le ∥ z k − y ∗ ( 0 ) ∥ \|z_k-y^*(0)\| ∥ z k − y ∗ ( 0 ) ∥ . Thus ∥ 0 − y k ∗ ∥ 2 ≤ C B / ( q ν ) \|0-y_k^*\|^2 \le CB/(q\nu) ∥0 − y k ∗ ∥ 2 ≤ C B / ( q ν ) . The threshold is 2 s η k = ( 4 / 9 ) s B ( 1 − ρ ) k 2s\eta_k=(4/9)sB(1-\rho)^k 2 s η k = ( 4/9 ) s B ( 1 − ρ ) k . Because q s = ν qs=\nu q s = ν , the ratio in (117) is at most H k ≤ C ν − 2 ( 1 − ρ ) − k ≤ C κ 5 H_k\le C\nu^{-2}(1-\rho)^{-k}\le C\kappa^5 H k ≤ C ν − 2 ( 1 − ρ ) − k ≤ C κ 5 . The last bound follows from (121) , q ≥ ν q\ge\nu q ≥ ν , and 1 − ρ ≥ 0.1 1-\rho\ge0.1 1 − ρ ≥ 0.1 . Each subproblem costs O ( n log ( 2 + κ ) ) O(n\log(2+\kappa)) O ( n log ( 2 + κ )) expected calls, uniformly conditional on all previous accepted outputs. Since J = O ( q − 1 / 2 log ( 2 + κ ) ) J=O(q^{-1/2}\log(2+\kappa)) J = O ( q − 1/2 log ( 2 + κ )) , startup costs
O ( n q − 1 / 2 log 2 ( 2 + κ ) ) . (122) O\left(nq^{-1/2}\log^2(2+\kappa)\right). \tag*{(122)} O ( n q − 1/2 log 2 ( 2 + κ ) ) . (122)
Auxiliary Saddle Solves with Computable Tolerances
At outer iterate w t = ( p t , y t ) w_t=(p_t,y_t) w t = ( p t , y t ) let f ^ t ( p , y ) = f ( p , y ) + ∥ p − p t ∥ 2 \widehat f_t(p,y)=f(p,y)+\|p-p_t\|^2 f t ( p , y ) = f ( p , y ) + ∥ p − p t ∥ 2 and R t 2 = ∥ ∇ f ^ t ( w t ) ∥ 2 R_t^2=\|\nabla\widehat f_t(w_t)\|^2 R t 2 = ∥∇ f t ( w t ) ∥ 2 . The mean is ( 1 , ν ) (1,\nu) ( 1 , ν ) -strongly convex–strongly concave and has joint smoothness at most three. Let w t ∗ w_t^* w t ∗ be its saddle point and G t G_t G t its saddle gap at w t w_t w t . Then
G t ≥ ν 2 ∥ w t − w t ∗ ∥ 2 ≥ ν R t 2 18 , G t ≤ R t 2 2 ν . G_t\ge\frac{\nu}{2}\|w_t-w_t^*\|^2\ge\frac{\nu R_t^2}{18},\qquad G_t\le\frac{R_t^2}{2\nu}. G t ≥ 2 ν ∥ w t − w t ∗ ∥ 2 ≥ 18 ν R t 2 , G t ≤ 2 ν R t 2 .
The first uses strong curvature and R t ≤ 3 ∥ w t − w t ∗ ∥ R_t\le3\|w_t-w_t^*\| R t ≤ 3∥ w t − w t ∗ ∥ ; the second applies strong convexity/concavity separately to the two gap terms. Thus the computable B t = ν R t 2 / 72 B_t=\nu R_t^2/72 B t = ν R t 2 /72 satisfies (22) . If R t = 0 R_t=0 R t = 0 , uniqueness of the dual maximizer gives y t = y ∗ ( p t ) y_t=y^*(p_t) y t = y ∗ ( p t ) and ∇ V f ( p t ) = 0 \nabla V_f(p_t)=0 ∇ V f ( p t ) = 0 .
Otherwise start the dual acceleration at w ^ 0 = ( p ~ 0 , y ~ 0 ) = w t \widehat w_0=(\widetilde p_0,\widetilde y_0)=w_t w 0 = ( p 0 , y 0 ) = w t , z 1 = y t z_1=y_t z 1 = y t , write w ^ k = ( p ~ k , y ~ k ) \widehat w_k=(\widetilde p_k,\widetilde y_k) w k = ( p k , y k ) for its iterates, and, with the same β \beta β as in (119) , form
f ^ t , k ( p , y ) = f ^ t ( p , y ) − τ 2 ∥ y − z k ∥ 2 , \widehat f_{t,k}(p,y)=\widehat f_t(p,y)-\frac{\tau}{2}\|y-z_k\|^2, f t , k ( p , y ) = f t ( p , y ) − 2 τ ∥ y − z k ∥ 2 ,
δ t , k = ν 2 ( 1 − ρ ) k B t , (123) \delta_{t,k}=\frac{\nu}{\sqrt{2}}(1-\rho)^kB_t, \tag*{(123)} δ t , k = 2 ν ( 1 − ρ ) k B t , (123)
z k + 1 = y ~ k + β ( y ~ k − y ~ k − 1 ) . z_{k+1}=\widetilde y_k+\beta(\widetilde y_k-\widetilde y_{k-1}). z k + 1 = y k + β ( y k − y k − 1 ) .
Solve f ^ t , k \widehat f_{t,k} f t , k from the preceding accepted w ^ k − 1 \widehat w_{k-1} w k − 1 using Lemma 25 , until ∥ ∇ f ^ t , k ( w ^ k ) ∥ 2 ≤ δ t , k \|\nabla\widehat f_{t,k}(\widehat w_k)\|^2\le\delta_{t,k} ∥∇ f t , k ( w k ) ∥ 2 ≤ δ t , k . Its mean strong monotonicity is s s s and its RMS smoothness constant is at most three, because the regularizer Hessian has norm at most two and the RMS smoothness constant of f f f is one. Since B t ≤ G t B_t\le G_t B t ≤ G t , our accepted residual thresholds are no larger than ( ν / 2 ) ( 1 − ρ ) k G t (\nu/\sqrt{2})(1-\rho)^kG_t ( ν / 2 ) ( 1 − ρ ) k G t . The residual estimate of Zhang et al. [27 ] therefore gives, pathwise for every accepted finite prefix,
∥ ∇ f ^ t ( w ^ k ) ∥ 2 ≤ C R ( 1 − ρ ) k R t 2 , C R = 5508 ν 2 ( q − ρ ) 2 + 18 2 ν . (124) \|\nabla\widehat f_t(\widehat w_k)\|^2\le C_R(1-\rho)^kR_t^2,\qquad C_R=\frac{5508}{\nu^2(\sqrt q-\rho)^2}+\frac{18\sqrt{2}}{\nu}. \tag*{(124)} ∥∇ f t ( w k ) ∥ 2 ≤ C R ( 1 − ρ ) k R t 2 , C R = ν 2 ( q − ρ ) 2 5508 + ν 18 2 . (124)
The same residual estimate directly controls the next solver’s starting distance. Write a = 1 − ρ a=1-\rho a = 1 − ρ and A = ( ∇ p f ^ t , − ∇ y f ^ t ) A=(\nabla_p\widehat f_t,-\nabla_y\widehat f_t) A = ( ∇ p f t , − ∇ y f t ) . Its ν \nu ν -strong monotonicity gives
d j : = ∥ w ^ j − w t ∗ ∥ ≤ ν − 1 C R a j / 2 R t ( j ≥ 0 ) , d_j:=\|\widehat w_j-w_t^*\|\le\nu^{-1}\sqrt{C_R}\,a^{j/2}R_t\qquad(j\ge0), d j := ∥ w j − w t ∗ ∥ ≤ ν − 1 C R a j /2 R t ( j ≥ 0 ) ,
where j = 0 j=0 j = 0 follows from ∥ A ( w ^ 0 ) ∥ = R t \|A(\widehat w_0)\|=R_t ∥ A ( w 0 ) ∥ = R t and C R ≥ 1 C_R\ge1 C R ≥ 1 . Writing w t ∗ = ( p t ∗ , y t ∗ ) w_t^*=(p_t^*,y_t^*) w t ∗ = ( p t ∗ , y t ∗ ) , the auxiliary operator A k = A + ( 0 , τ ( y − z k ) ) A_k=A+(0,\tau(y-z_k)) A k = A + ( 0 , τ ( y − z k )) is s s s -strongly monotone, so
∥ w ^ t , k ∗ − w t ∗ ∥ ≤ ( τ / s ) ∥ z k − y t ∗ ∥ ≤ ∥ z k − y t ∗ ∥ . \|\widehat w_{t,k}^*-w_t^*\|\le(\tau/s)\|z_k-y_t^*\|\le\|z_k-y_t^*\|. ∥ w t , k ∗ − w t ∗ ∥ ≤ ( τ / s ) ∥ z k − y t ∗ ∥ ≤ ∥ z k − y t ∗ ∥.
For k ≥ 2 k \ge2 k ≥ 2 , z k = y ~ k − 1 + β ( y ~ k − 1 − y ~ k − 2 ) z_k = \tilde{y}_{k-1} + \beta(\tilde{y}_{k-1} - \tilde{y}_{k-2}) z k = y ~ k − 1 + β ( y ~ k − 1 − y ~ k − 2 ) and 0 ≤ β ≤ 1 0 \le\beta\le1 0 ≤ β ≤ 1 consequently give ∥ w ~ k − 1 − w t , k ∗ ∥ ≤ 3 d k − 1 + d k − 2 \lVert\tilde{w}_{k-1} - w^*_{t,k}\rVert\le3d_{k-1} + d_{k-2} ∥ w ~ k − 1 − w t , k ∗ ∥ ≤ 3 d k − 1 + d k − 2 . For k = 1 k = 1 k = 1 , z 1 = y ~ 0 z_1 = \tilde{y}_0 z 1 = y ~ 0 gives A 1 ( w ~ 0 ) = A ( w ~ 0 ) A_1(\tilde{w}_0) = A(\tilde{w}_0) A 1 ( w ~ 0 ) = A ( w ~ 0 ) . Thus
∥ w ~ k − 1 − w t , k ∗ ∥ 2 ≤ { R t 2 / s 2 , k = 1 , 16 C R ν − 2 a k − 2 R t 2 , k ≥ 2. \lVert\tilde{w}_{k-1} - w^*_{t,k}\rVert^2 \le\begin{cases} R_t^2/s^2, & k = 1, \\ 16C_R\nu^{-2}a^{k-2}R_t^2, & k \ge2. \end{cases} ∥ w ~ k − 1 − w t , k ∗ ∥ 2 ≤ { R t 2 / s 2 , 16 C R ν − 2 a k − 2 R t 2 , k = 1 , k ≥ 2.
Since δ t , k = ν 2 R t 2 a k / ( 72 2 ) \delta_{t,k} = \nu^2 R_t^2 a^k/(72\sqrt{2}) δ t , k = ν 2 R t 2 a k / ( 72 2 ) , a ≥ 1 / 10 a \ge1/10 a ≥ 1/10 , s ≥ ν s \ge\nu s ≥ ν , and C R = 550800 / ( ν 2 q ) + 18 2 / ν ≤ C ν − 3 C_R = 550800/(\nu^2q) + 18\sqrt{2}/\nu\le C\nu^{-3} C R = 550800/ ( ν 2 q ) + 18 2 / ν ≤ C ν − 3 , the certification ratio obeys
9 ∥ w ~ k − 1 − w t , k ∗ ∥ 2 δ t , k ≤ C κ 7 ( k ≥ 1 ) . \frac{9\lVert\tilde{w}_{k-1} - w^*_{t,k}\rVert^2}{\delta_{t,k}} \le C\kappa^7 \qquad(k \ge1). δ t , k 9 ∥ w ~ k − 1 − w t , k ∗ ∥ 2 ≤ C κ 7 ( k ≥ 1 ) .
Here q = ν / s ≥ ν q = \nu/s \ge\nu q = ν / s ≥ ν was used. Conditional on the complete history at each auxiliary start, all these inputs are fixed and subsequent solver indices are fresh. Lemma 25 therefore gives almost-sure termination and expected cost
O ( ( n + 3 n / s ) [ 1 + log + ( C κ 7 ) ] ) = O ( n log ( 2 + κ ) ) O\bigl((n + 3\sqrt{n}/s)[1 + \log^+(C\kappa^7)]\bigr) = O(n\log(2+\kappa)) O ( ( n + 3 n / s ) [ 1 + log + ( C κ 7 )] ) = O ( n log ( 2 + κ ))
for that call, uniformly over the preceding accepted history, by induction from k = 1 k = 1 k = 1 . Take α = ν 5 / 576 \alpha= \nu^5/576 α = ν 5 /576 and K = ⌈ log ( C R / α ) − log ( 1 − ρ ) ⌉ = O ( q − 1 / 2 log ( 2 + κ ) ) K = \left\lceil\frac{\log(C_R/\alpha)}{-\log(1-\rho)} \right\rceil= O(q^{-1/2}\log(2+\kappa)) K = ⌈ − l o g ( 1 − ρ ) l o g ( C R / α ) ⌉ = O ( q − 1/2 log ( 2 + κ )) . Equation (124) guarantees ∥ ∇ f ^ t ( w ~ K ) ∥ 2 ≤ α R t 2 \lVert\nabla\hat{f}_t(\tilde{w}_K)\rVert^2 \le\alpha R_t^2 ∥ ∇ f ^ t ( w ~ K ) ∥ 2 ≤ α R t 2 on every run. The choice of α \alpha α is no larger than the two outer tolerances in Zhang et al. [27 ] . Consequently one outer step costs
O ( n q − 1 / 2 log 2 ( 2 + κ ) ) (125) O(nq^{-1/2}\log^2(2+\kappa)) \tag*{(125)} O ( n q − 1/2 log 2 ( 2 + κ )) (125)
in conditional expectation.
Stationarity, Total Cost, and Fixed Confidence
Start the outer method at ( 0 , y w ) (0,y_{\mathrm{w}}) ( 0 , y w ) and run T o u t = ⌈ 60 D 0 / ϵ 0 2 ⌉ T_{\mathrm{out}} = \lceil60D_0/\epsilon_0^2\rceil T out = ⌈ 60 D 0 / ϵ 0 2 ⌉ outer steps, unless an exactly zero residual permits early return. The outer theorem of Zhang et al. [27 ] , under the relative residual conditions used above, bounds the gradients at the outputs p t + 1 p_{t+1} p t + 1 of the outer steps:
1 T o u t ∑ t = 1 T o u t ∥ ∇ V f ( p t + 1 ) ∥ 2 ≤ 268 D 0 5 T o u t + 28 ∥ y w − y ∗ ( 0 ) ∥ 2 5 T o u t ≤ 296 D 0 5 T o u t ≤ ϵ 0 2 . \frac{1}{T_{\mathrm{out}}}\sum_{t=1}^{T_{\mathrm{out}}}\lVert\nabla V_f(p_{t+1})\rVert^2 \le\frac{268D_0}{5T_{\mathrm{out}}} + \frac{28\lVert y_{\mathrm{w}} - y^*(0)\rVert^2}{5T_{\mathrm{out}}} \le\frac{296D_0}{5T_{\mathrm{out}}} \le\epsilon_0^2. T out 1 t = 1 ∑ T out ∥ ∇ V f ( p t + 1 ) ∥ 2 ≤ 5 T out 268 D 0 + 5 T out 28 ∥ y w − y ∗ ( 0 ) ∥ 2 ≤ 5 T out 296 D 0 ≤ ϵ 0 2 .
This holds pathwise for certified accepted outputs. Returning p t + 1 p_{t+1} p t + 1 for a uniform t ∈ { 1 , … , T o u t } t \in\{1,\ldots,T_{\mathrm{out}}\} t ∈ { 1 , … , T out } and scaling by L L L give (14) for the gradient of the unnormalized value function. The conditional expected costs (122) and (125) are uniform. The tower property therefore bounds total expected queries by
C n q − 1 / 2 ( 1 + S ) log 2 ( 2 + κ ) = C n 3 / 4 κ ( 1 + S ) log 2 ( 2 + κ ) . Cnq^{-1/2}(1+S)\log^2(2+\kappa) = Cn^{3/4}\sqrt{\kappa}(1+S)\log^2(2+\kappa). C n q − 1/2 ( 1 + S ) log 2 ( 2 + κ ) = C n 3/4 κ ( 1 + S ) log 2 ( 2 + κ ) .
All auxiliary saddle gaps are finite by strong convexity–concavity.
For a deterministic cap with success probability 2 / 3 2/3 2/3 , run with target squared gradient norm ϵ 2 / 6 \epsilon^2/6 ϵ 2 /6 . Choose a universal C c e r t ≥ 1 C_{\mathrm{cert}} \ge1 C cert ≥ 1 bounding both certification ratios above by C c e r t κ 7 C_{\mathrm{cert}}\kappa^7 C cert κ 7 . With T 6 = ⌈ 360 D 0 / ϵ 0 2 ⌉ T_6 = \lceil360D_0/\epsilon_0^2\rceil T 6 = ⌈ 360 D 0 / ϵ 0 2 ⌉ , a valid expected-cost bound is
B Q = C e p H ( J + T 6 K ) [ 2 + ⌈ log 2 + ( C c e r t κ 7 ) ⌉ ] + n T 6 . (126) B_Q = C_{\mathrm{ep}}^H(J + T_6K)[2 + \lceil\log_2^+(C_{\mathrm{cert}}\kappa^7)\rceil] + nT_6. \tag*{(126)} B Q = C ep H ( J + T 6 K ) [ 2 + ⌈ log 2 + ( C cert κ 7 )⌉] + n T 6 . (126)
This uses the L-SVRE epoch charge (119) and includes the residual R t R_t R t computed at each outer step. Cap execution at 6 B Q 6B_Q 6 B Q , returning the origin on timeout; Markov’s inequality then bounds the failure probability by 1 / 3 1/3 1/3 as in Appendix E.8, at the same rate.
AI Usage Statement
We used AI assistants, including OpenAI’s GPT models and Anthropic’s Claude, to help develop and check proofs, formalize them in Lean, and edit the text. The authors take full responsibility for the content of this paper.