Introduction
Determining the optimal complexity of minimizing uniformly convex functions is a fundamental problem in optimization. Uniform convexity generalizes strong convexity through curvature bounds involving higher powers of the distance between points, with applications in high-order optimization [26 , 8 ] and machine learning [30 , 8 , 4 ] . The optimal complexity depends jointly on the order of derivative oracle access, the smoothness of the function, and its uniform-convexity degree.
We study this problem in a general setting, with p t h p^{\mathrm{th}} p th -order derivative oracle access to functions F ( x ) F(x) F ( x ) whose p t h p^{\mathrm{th}} p th -order derivatives ∇ p F ( x ) \nabla^p F(x) ∇ p F ( x ) are Hölder continuous with exponent ν \nu ν and parameter H H H , and that are uniformly convex with degree q q q and modulus σ \sigma σ . We focus on the matched regime q = p + ν q=p+\nu q = p + ν , which includes the classical smooth strongly convex case when p = 1 , ν = 1 , q = 2 p=1,\nu=1,q=2 p = 1 , ν = 1 , q = 2 , in which H σ \frac{H}{\sigma} σ H is the condition number. In this case, Nesterov’s accelerated gradient method [24 ] converges linearly, attaining an optimality gap F ( x T ) − F ( x ∗ ) ≤ ϵ F(x_T)-F(x^*)\leq\epsilon F ( x T ) − F ( x ∗ ) ≤ ϵ in T ∈ O ( ( H σ ) 1 2 log ( Δ 0 ϵ ) ) T\in\mathcal{O}\left(\left(\frac{H}{\sigma}\right)^{\frac{1}{2}}\log\left(\frac{\Delta_0}{\epsilon}\right)\right) T ∈ O ( ( σ H ) 2 1 log ( ϵ Δ 0 ) ) gradient oracle calls [25 , 7 ] , where x ∗ x^* x ∗ is the minimizer and Δ 0 : = F ( x 0 ) − F ( x ∗ ) \Delta_0:=F(x_0)-F(x^*) Δ 0 := F ( x 0 ) − F ( x ∗ ) denotes the initial optimality gap.
On the upper-bound side, [29 ] establishes linear convergence in the matched higher-order regime q = p + ν > 2 q=p+\nu>2 q = p + ν > 2 , with iteration complexity O p , ν ( ( H σ ) 2 3 q − 2 log ( Δ 0 ϵ ) ) \mathcal{O}_{p,\nu}\left(\left(\frac{H}{\sigma}\right)^{\frac{2}{3q-2}}\log\left(\frac{\Delta_0}{\epsilon}\right)\right) O p , ν ( ( σ H ) 3 q − 2 2 log ( ϵ Δ 0 ) ) . These methods use derivatives through order p p p , and H σ \frac{H}{\sigma} σ H is the high-order condition number [17 , 8 , 16 ] . The logarithmic accuracy dependence follows from a restarting mechanism that reapplies an accelerated convex method at smaller accuracy scales, reducing the optimality gap by a constant factor at each stage [17 , 29 ] .
On the lower-bound side, Nesterov’s classical first-order construction [25 ] establishes the optimality of the first-order rate above, while a tight characterization for the matched higher-order setting has remained open.
For general combinations of p , q , ν p,q,\nu p , q , ν , [4 ] establishes tight deterministic lower bounds for both asymmetric regimes, with polynomial dependence on 1 ϵ \frac{1}{\epsilon} ϵ 1 when q > p + ν q>p+\nu q > p + ν and doubly logarithmic dependence on 1 ϵ \frac{1}{\epsilon} ϵ 1 when q < p + ν q<p+\nu q < p + ν , matching the corresponding upper bounds [29 ] . These complementary results leave q = p + ν q=p+\nu q = p + ν as the only missing case in the general complexity characterization, as summarized in Table 1 , under the respective function-class assumptions.
Table 1. Complexity overview. κ : = H σ \kappa:= \frac{H}{\sigma} κ := σ H ; Δ 0 \Delta_0 Δ 0 : initial optimality gap; ϵ \epsilon ϵ : target accuracy.
Our Result
In this paper, we resolve this open problem by establishing a lower bound matching the upper bound for deterministic algorithms with oracle access to all derivatives up to order p p p . Our result establishes a higher-order uniformly convex counterpart to Nesterov’s classical first-order lower bound [25 ] and complements the asymmetric results of [4 ] .
Theorem 1.1 (Matched-order lower bound). *Fix an integer p ≥ 2 p\geq2 p ≥ 2 , 0 < ν ≤ 1 0<\nu\leq1 0 < ν ≤ 1 , and q = p + ν q=p+\nu q = p + ν . There are constants c p , ν , K p , ν > 0 c_{p,\nu},K_{p,\nu}>0 c p , ν , K p , ν > 0 such that, for H , σ , Δ 0 > 0 H,\sigma,\Delta_0>0 H , σ , Δ 0 > 0 with sufficiently large H σ \frac{H}{\sigma} σ H and 0 < ϵ ≤ c p , ν Δ 0 0<\epsilon\leq c_{p,\nu}\Delta_0 0 < ϵ ≤ c p , ν Δ 0 , and every dimension d ≥ K p , ν ( H σ ) 2 3 q − 2 log ( Δ 0 ϵ ) d\geq K_{p,\nu}\left(\frac{H}{\sigma}\right)^{\frac{2}{3q-2}}\log\left(\frac{\Delta_0}{\epsilon}\right) d ≥ K p , ν ( σ H ) 3 q − 2 2 log ( ϵ Δ 0 ) , every deterministic p t h p^{\mathrm{th}} p th -order oracle algorithm initialized at x 0 ∈ R d x_0\in\mathbb{R}^d x 0 ∈ R d admits a p p p times continuously differentiable function F : R d → R F:\mathbb{R}^d\to\mathbb{R} F : R d → R with minimizer x ∗ x^* x ∗ and F ( x 0 ) − F ( x ∗ ) = Δ 0 F(x_0)-F(x^*)=\Delta_0 F ( x 0 ) − F ( x ∗ ) = Δ 0 , such that, for all x , y ∈ R d x,y\in\mathbb{R}^d x , y ∈ R d : (a) p t h p^{\mathrm{th}} p th -order Hölder smoothness. ∥ ∇ p F ( y ) − ∇ p F ( x ) ∥ ≤ H ∥ y − x ∥ ν \|\nabla^pF(y)-\nabla^pF(x)\|\leq H\|y-x\|^\nu ∥ ∇ p F ( y ) − ∇ p F ( x ) ∥ ≤ H ∥ y − x ∥ ν . (b) Uniform convexity. F ( y ) ≥ F ( x ) + ⟨ ∇ F ( x ) , y − x ⟩ + σ q ∥ y − x ∥ q F(y)\geq F(x)+\langle\nabla F(x),y-x\rangle+\frac{\sigma}{q}\|y-x\|^q F ( y ) ≥ F ( x ) + ⟨ ∇ F ( x ) , y − x ⟩ + q σ ∥ y − x ∥ q .
For a candidate x ^ T \widehat{x}_T x T after T T T oracle calls, F ( x ^ T ) − F ( x ∗ ) ≤ ϵ F(\widehat{x}_T)-F(x^*)\leq\epsilon F ( x T ) − F ( x ∗ ) ≤ ϵ requires T ∈ Ω p , ν ( ( H σ ) 2 3 q − 2 log ( Δ 0 ϵ ) ) T\in\Omega_{p,\nu}\left(\left(\frac{H}{\sigma}\right)^{\frac{2}{3q-2}}\log\left(\frac{\Delta_0}{\epsilon}\right)\right) T ∈ Ω p , ν ( ( σ H ) 3 q − 2 2 log ( ϵ Δ 0 ) ) .*
Motivation
To prove an oracle lower bound, we construct, for any given deterministic algorithm, a hard function for which reaching the target optimality gap requires the number of oracle calls stated in Theorem 1.1 . The construction limits the information that each oracle output provides about the minimizer, so that fewer oracle calls cannot identify an accurate solution.
Classical Construction. Nesterov’s classical first-order lower-bound construction [25 ] uses a quadratic chain: a quadratic function whose terms couple neighboring coordinates, such as ( x i − x i + 1 ) 2 (x_i-x_{i+1})^2 ( x i − x i + 1 ) 2 . The coordinate directions can be chosen so that gradient oracle outputs reveal the minimizer gradually. Its coordinates decay geometrically, so the unrevealed tail prevents the algorithm from reaching the target accuracy until sufficiently many directions have been determined. This construction does not directly extend to higher-order oracles, because one Hessian oracle call reveals the entire quadratic form.
Norm Mismatch. A natural extension is therefore to replace the quadratic terms with absolute q th q^{\text{th}} q th powers, such as ∣ x i − x i + 1 ∣ q |x_i-x_{i+1}|^q ∣ x i − x i + 1 ∣ q . However, a long chain of scalar q q q -power residuals naturally controls the ℓ q \ell_q ℓ q norm, while our uniform-convexity condition uses the Euclidean norm. For q > 2 q>2 q > 2 , ∥ x ∥ q q ≥ d 1 − q 2 ∥ x ∥ 2 q \lVert x\rVert_q^q \ge d^{1-\frac{q}{2}}\lVert x\rVert_2^q ∥ x ∥ q q ≥ d 1 − 2 q ∥ x ∥ 2 q . This norm conversion introduces the dimension-dependent factor d 1 − q 2 d^{1-\frac{q}{2}} d 1 − 2 q into the Euclidean uniform-convexity modulus, leading to a suboptimal lower-bound rate for this naive extension.1
Successive Phases. We therefore need a different construction to recover both factors in the upper bound. To identify the condition-number cost that this construction should preserve, we examine the asymmetric lower bound in [4 Theorem 1] . For q > p + ν q>p+\nu q > p + ν , their rate is Ω ( ( H σ ) 2 ( q − p − ν ) 3 ( p + ν ) − 2 ( σ ϵ ) 2 q ( 3 ( p + ν ) − 2 ) ) \Omega\left(\left(\frac{H}{\sigma}\right)^{\frac{2(q-p-\nu)}{3(p+\nu)-2}}\left(\frac{\sigma}{\epsilon}\right)^{\frac{2}{q(3(p+\nu)-2)}}\right) Ω ( ( σ H ) 3 ( p + ν ) − 2 2 ( q − p − ν ) ( ϵ σ ) q ( 3 ( p + ν ) − 2 ) 2 ) . Although their theorem treats only q > p + ν q>p+\nu q > p + ν , formally substituting q = p + ν q=p+\nu q = p + ν in this expression gives ( H σ ) 2 3 q − 2 \left(\frac{H}{\sigma}\right)^{\frac{2}{3q-2}} ( σ H ) 3 q − 2 2 . This expression has the desired condition-number dependence but is independent of ϵ \epsilon ϵ , and therefore misses the logarithmic factor in the upper bound. In light of the restarting mechanism in the upper-bound analyses [17 , 29 ] , we seek one hard function that forces this condition-number cost at each constant-factor reduction of the optimality gap. The successive-scale constructions used in earlier logarithmic lower bounds [23 , 5 , 15 ] motivate arranging these repeated hard instances into phases at decreasing accuracy scales. Each phase must require a further Ω p , ν ( ( H σ ) 2 3 q − 2 ) \Omega_{p,\nu}\left(\left(\frac{H}{\sigma}\right)^{\frac{2}{3q-2}}\right) Ω p , ν ( ( σ H ) 3 q − 2 2 ) calls even after the algorithm has observed all oracle outputs from the preceding phases. To obtain the sharp product in the upper bound, we must realize these phases in one objective without increasing H H H or decreasing σ \sigma σ with their number.
Techniques and Proof Strategy
Our technical novelty is a multi-center base B B B that allows us to separate Euclidean uniform convexity from the chains supplying oracle hardness, avoiding the norm mismatch. Figure 1 illustrates this separation. We construct L L L successive phases, each with a short power chain of length m m m , where
Figure 1. Geometry and hardness at successive accuracy scales (schematic).
m ≍ p , ν ( H σ ) 2 3 q − 2 , L ≍ p , ν log ( Δ 0 ϵ ) . (1) m \asymp_{p,\nu} \left(\frac{H}{\sigma}\right)^{\frac{2}{3q-2}}, \qquad L \asymp_{p,\nu} \log\left(\frac{\Delta_0}{\epsilon}\right). \tag*{(1)} m ≍ p , ν ( σ H ) 3 q − 2 2 , L ≍ p , ν log ( ϵ Δ 0 ) . (1)
Multi-phase Uniform Convexity. For geometry, B B B must reproduce the same local q th q^{\text{th}} q th -power behavior at each scale. A fixed-center regularizer ∥ x ∥ 2 q \lVert x\rVert_2^q ∥ x ∥ 2 q has positive quadratic curvature at nonzero centers, so it cannot provide this behavior there. We therefore construct B B B around successive reference centers c 0 , … , c L \mathbf{c}_0,\ldots,\mathbf{c}_L c 0 , … , c L , whose consecutive distances decrease by a fixed factor to match the decreasing accuracy scales. Near every center, the base has the exact form
B ( x ) = B ( c j ) + ⟨ ∇ B ( c j ) , x − c j ⟩ + μ q ∥ x − c j ∥ 2 q B(x) = B(\mathbf{c}_j) + \langle\nabla B(\mathbf{c}_j),x-\mathbf{c}_j\rangle+ \frac{\mu}{q}\lVert x-\mathbf{c}_j\rVert_2^q B ( x ) = B ( c j ) + ⟨ ∇ B ( c j ) , x − c j ⟩ + q μ ∥ x − c j ∥ 2 q
with the same μ > 0 \mu> 0 μ > 0 . Its higher-order derivatives through order p p p consequently vanish at every center, while B B B remains globally high-order Hölder smooth and Euclidean uniformly convex, with bounds independent of L L L .
Hardness Construction. For hardness, we add the short power chains and an initial linear term giving a nonzero starting gradient. To initiate the next phase, we join each chain to an affine function near later centers. Its slope makes the complete objective’s gradient at the next center point along the next chain’s starting direction, while the full oracle output at that center does not reveal the remaining chain directions. Reducing the gap further therefore requires resolving another short chain. Convexity preserves the base’s uniform-convexity bound, while orthogonality of nonadjacent phases prevents smoothness costs from accumulating with L L L . The local base replacements and affine continuations must also retain their prescribed formulas and derivatives exactly. An ordinary maximum preserves convexity but need not be smooth. We therefore use a smooth maximum [3 Lemma 1] that remains convex and equals the larger input whenever it exceeds the other by at least the smoothing width.
Oracle Consistency. We enforce these restrictions for arbitrary deterministic oracle algorithms by choosing unrevealed chain directions as the algorithm makes its oracle calls, while preserving all earlier oracle outputs. The final choices define one fixed objective requiring Ω ( m ) \Omega(m) Ω ( m ) calls per phase, hence Ω ( m L ) \Omega(mL) Ω ( m L ) calls in total.
Related Work
Upper Bounds. In the matched regime, [17 ] obtains the upper bound in tensor iterations for ν = 1 \nu= 1 ν = 1 , and [29 ] obtains the general Hölder-smooth iteration rate using a restarting mechanism for accelerated high-order methods. For p = 2 p = 2 p = 2 and q = 2 + ν q = 2+\nu q = 2 + ν , [16 ] establishes linear convergence with the larger condition-number exponent 1 1 + ν \frac{1}{1+\nu} 1 + ν 1 , compared with the sharp 2 4 + 3 ν \frac{2}{4+3\nu} 4 + 3 ν 2 . [13 ] develops general non-Euclidean high-order oracle bounds. Their Euclidean convex rate, combined with the restarting mechanism, also gives the upper bound in full p th p^{\text{th}} p th -order oracle calls.
Lower Bounds. Before this work, a tight lower bound for the matched regime was known only in Nesterov’s classical first-order setting p = 1 p = 1 p = 1 , ν = 1 \nu= 1 ν = 1 , q = 2 q = 2 q = 2 [25 ] . Beyond extending this classical result, our bound completes the general p , q , ν p,q,\nu p , q , ν characterization in Table 1 , complementing the asymmetric results of [4 ] . Smaller entries record earlier special cases.
Notations
We use bold lowercase letters for vectors (e.g., x , y \mathbf{x},\mathbf{y} x , y ), with subscripts for vectors at different iterations (e.g., x T \mathbf{x}_T x T ). We use regular lowercase letters for scalars, and x i x_i x i for the i th i^{\text{th}} i th coordinate of x \mathbf{x} x . We use ∥ ⋅ ∥ \|\cdot\| ∥ ⋅ ∥ to denote the Euclidean norm for vectors and the induced operator norm for multilinear forms, and ⟨ ⋅ , ⋅ ⟩ \langle\cdot,\cdot\rangle ⟨ ⋅ , ⋅ ⟩ for the Euclidean inner product. For a k k k -linear form T \mathcal{T} T on R d \mathbb{R}^d R d , its operator norm is ∥ T ∥ : = sup ∥ z i ∥ ≤ 1 , 1 ≤ i ≤ k ∣ T [ z 1 , … , z k ] ∣ \|\mathcal{T}\| := \sup_{\|z_i\|\le1,\,1\le i\le k}|\mathcal{T}[z_1,\ldots,z_k]| ∥ T ∥ := sup ∥ z i ∥ ≤ 1 , 1 ≤ i ≤ k ∣ T [ z 1 , … , z k ] ∣ . We use ∇ F ( x ) \nabla F(x) ∇ F ( x ) for the gradient vector and ∇ k F ( x ) \nabla^k F(x) ∇ k F ( x ) for the k th k^{\text{th}} k th derivative of F F F at x x x , a symmetric k k k -linear form for k ≥ 2 k \ge2 k ≥ 2 . We set ∇ 0 F : = F \nabla^0F := F ∇ 0 F := F . For a differentiable F : R d → R F:\mathbb{R}^d\to\mathbb{R} F : R d → R , we denote its Bregman divergence by D F ( y , x ) : = F ( y ) − F ( x ) − ⟨ ∇ F ( x ) , y − x ⟩ D_F(y,x):=F(y)-F(x)-\langle\nabla F(x),y-x\rangle D F ( y , x ) := F ( y ) − F ( x ) − ⟨ ∇ F ( x ) , y − x ⟩ for x , y ∈ R d x,y\in\mathbb{R}^d x , y ∈ R d . We use C p ( R d ) C^p(\mathbb{R}^d) C p ( R d ) for the p p p times continuously differentiable functions and C p , ν ( R d ) C^{p,\nu}(\mathbb{R}^d) C p , ν ( R d ) for those whose p th p^{\text{th}} p th derivatives are globally ν \nu ν -Hölder continuous. We use I I I for the identity matrix and P E P_E P E for the orthogonal projection onto a linear subspace E E E . For pairwise orthogonal subspaces E 1 , … , E n E_1,\ldots,E_n E 1 , … , E n , i.e., ⟨ u i , u j ⟩ = 0 \langle u_i,u_j\rangle=0 ⟨ u i , u j ⟩ = 0 whenever u i ∈ E i u_i\in E_i u i ∈ E i , u j ∈ E j u_j\in E_j u j ∈ E j , and i ≠ j i\ne j i = j , we denote their orthogonal direct sum by ⨁ i = 1 n E i : = { ∑ i = 1 n u i : u i ∈ E i } \bigoplus_{i=1}^n E_i:=\{\sum_{i=1}^n u_i:u_i\in E_i\} ⨁ i = 1 n E i := { ∑ i = 1 n u i : u i ∈ E i } , with ∥ ∑ i = 1 n u i ∥ 2 = ∑ i = 1 n ∥ u i ∥ 2 \|\sum_{i=1}^n u_i\|^2=\sum_{i=1}^n\|u_i\|^2 ∥ ∑ i = 1 n u i ∥ 2 = ∑ i = 1 n ∥ u i ∥ 2 . We denote the closure of a set S S S by S ‾ \overline{S} S . For a nonempty set S S S , we denote dist ( x , S ) : = inf z ∈ S ∥ x − z ∥ \operatorname{dist}(x,S):=\inf_{z\in S}\|x-z\| dist ( x , S ) := inf z ∈ S ∥ x − z ∥ .
Lower Bound for the q = p + ν q = p + \nu q = p + ν Case
We prove Theorem 1.1 by constructing a hard function whose successive phases each require additional oracle calls. Following the separation of geometry and hardness in § 1.3 , we first build a uniformly convex base and then add convex phase functions. Combining these components requires the smooth maximum introduced below.
Smooth Maximum
A standard lower-bound construction takes the pointwise maximum of convex functions to restrict the information revealed by each oracle output. Since this maximum need not be differentiable, a smoothing operator supplies the required derivative regularity while preserving that restriction [19 , 1 , 18 , 4 ] . Our construction additionally requires exact agreement with the base and chain formulas on specified regions, to preserve their derivatives and earlier oracle outputs. Averaging over nearby points can alter nonlinear norm-power formulas even where one input dominates. We therefore use the smooth maximum [3 ] , which smoothly compares input values and equals the larger input whenever their difference exceeds the smoothing width.
Definition 4.1 (Smooth maximum). We use the even smooth probability density φ ( u ) = c φ exp ( − 1 1 − u 2 ) \varphi(u) = c_{\varphi}\exp\left(-\frac{1}{1-u^2}\right) φ ( u ) = c φ exp ( − 1 − u 2 1 ) for ∣ u ∣ < 1 |u| < 1 ∣ u ∣ < 1 and φ ( u ) = 0 \varphi(u) = 0 φ ( u ) = 0 otherwise, with c φ > 0 c_{\varphi} > 0 c φ > 0 chosen so that ∫ − 1 1 φ ( u ) d u = 1 \int_{-1}^{1}\varphi(u)\,\mathrm{d}u = 1 ∫ − 1 1 φ ( u ) d u = 1 . For t + : = max { t , 0 } t_{+} := \max\{t,0\} t + := max { t , 0 } and smoothing width δ > 0 \delta> 0 δ > 0 , define
ψ δ ( t ) : = ∫ − 1 1 ( t − δ u ) + φ ( u ) d u , M δ ( a , b ) : = a + ψ δ ( b − a ) ( a , b ∈ R ) . (2) \psi_{\delta}(t) := \int_{-1}^{1}(t-\delta u)_{+}\varphi(u)\,\mathrm{d}u,\qquad M_{\delta}(a,b) := a+\psi_{\delta}(b-a)\qquad(a,b\in\mathbb{R}). \tag*{(2)} ψ δ ( t ) := ∫ − 1 1 ( t − δ u ) + φ ( u ) d u , M δ ( a , b ) := a + ψ δ ( b − a ) ( a , b ∈ R ) . (2)
Lemma 4.2 ([3 Lemma 1] ). For every δ > 0 \delta> 0 δ > 0 , the function M δ M_{\delta} M δ in Definition 4.1 is C ∞ C^{\infty} C ∞ , convex, and nondecreasing in each argument. For all a , b ∈ R a,b\in\mathbb{R} a , b ∈ R ,
max { a , b } ≤ M δ ( a , b ) ≤ max { a , b } + δ 2 , M δ ( a , b ) = { a , a − b ≥ δ , b , b − a ≥ δ . \max\{a,b\}\le M_{\delta}(a,b)\le\max\{a,b\}+\frac{\delta}{2},\qquad M_{\delta}(a,b)=
\begin{cases}
a, & a-b\ge\delta,\\
b, & b-a\ge\delta.
\end{cases} max { a , b } ≤ M δ ( a , b ) ≤ max { a , b } + 2 δ , M δ ( a , b ) = { a , b , a − b ≥ δ , b − a ≥ δ .
Step 1: Construct the Uniformly Convex Base
To force the condition-number cost at successive accuracy scales, we construct L ≥ 1 L\ge1 L ≥ 1 phases. We first give the base B B B the same local uniform-convexity geometry at each phase.
Step 1.1: Choose the phase centers. To give each phase the same local uniform-convexity geometry, we anchor phase j j j at a center c j c_j c j , with endpoint c j + 1 c_{j+1} c j + 1 and c 0 = 0 c_0=0 c 0 = 0 . We define its phase displacement by h j = c j + 1 − c j h_j=c_{j+1}-c_j h j = c j + 1 − c j for 0 ≤ j < L 0\le j<L 0 ≤ j < L . Later chain directions must be chosen without altering earlier chains, so we choose mutually orthogonal phase subspaces, with each h j h_j h j in its corresponding subspace. We will specify the coordinates of h j h_j h j in Step 2.
To impose Euclidean uniform convexity, we begin with B 0 ( x ) = μ q ∥ x ∥ q B_0(x)=\frac{\mu}{q}\lVert x\rVert^q B 0 ( x ) = q μ ∥ x ∥ q , with μ > 0 \mu>0 μ > 0 , the simplest norm-power function. We then reproduce its geometry at every phase center through successive local replacements. Let B j B_j B j denote the base after j j j replacements, and define B = B L B=B_L B = B L . We require
B j ( x ) = B j ( c j ) + ⟨ ∇ B j ( c j ) , x − c j ⟩ + μ q ∥ x − c j ∥ q (3) B_j(x)=B_j(c_j)+\langle\nabla B_j(c_j),x-c_j\rangle+\frac{\mu}{q}\lVert x-c_j\rVert^q \tag*{(3)} B j ( x ) = B j ( c j ) + ⟨ ∇ B j ( c j ) , x − c j ⟩ + q μ ∥ x − c j ∥ q (3)
on a ball containing c j + 1 c_{j+1} c j + 1 . Here growth above the tangent is measured by the Euclidean q t h q^{\mathrm{th}} q th power, avoiding the norm mismatch. Since q > p q>p q > p , the higher-order derivatives satisfy ∇ k B j ( c j ) = 0 \nabla^k B_j(c_j)=0 ∇ k B j ( c j ) = 0 for 2 ≤ k ≤ p 2\le k\le p 2 ≤ k ≤ p .
Step 1.2: Preserve the completed phase’s gradient. At the end of phase j j j , the objective’s gradient should point only in the next phase’s directions. The objective’s gradient at c j c_j c j , the increment from the base, and the gradient of the hardness function A j A_j A j , which we will construct in Step 2, must therefore cancel in the completed phase’s directions. Define R j = ∥ h j ∥ R_j=\lVert h_j\rVert R j = ∥ h j ∥ . Differentiating Eq. (3) at c j + 1 = c j + h j c_{j+1}=c_j+h_j c j + 1 = c j + h j identifies the base increment as ∇ B j ( c j + 1 ) − ∇ B j ( c j ) = μ R j q − 2 h j \nabla B_j(c_{j+1})-\nabla B_j(c_j)=\mu R_j^{q-2}h_j ∇ B j ( c j + 1 ) − ∇ B j ( c j ) = μ R j q − 2 h j .
We will make A j A_j A j equal an affine function near later centers, so its gradient there is fixed. For the combined gradient to remain canceled along h j h_j h j , the base gradient component along h j h_j h j must also stay unchanged.
To check whether this cancellation survives movement in the next phase, we evaluate the gradient given by Eq. (3) at c j + 1 + z c_{j+1}+z c j + 1 + z near the next center, with z ≠ 0 z\ne0 z = 0 in the next phase’s directions and hence z ⊥ h j z\perp h_j z ⊥ h j . Where Eq. (3) holds, Eq. (C.5) in Appx. C gives
⟨ ∇ B j ( c j + 1 + z ) − ∇ B j ( c j + 1 ) , h j ⟩ = μ R j 2 ( ( R j 2 + ∥ z ∥ 2 ) q − 2 2 − R j q − 2 ) > 0. \left\langle\nabla B_j(c_{j+1}+z)-\nabla B_j(c_{j+1}),h_j\right\rangle=\mu R_j^2\left(\left(R_j^2+\lVert z\rVert^2\right)^{\frac{q-2}{2}}-R_j^{q-2}\right)>0. ⟨ ∇ B j ( c j + 1 + z ) − ∇ B j ( c j + 1 ) , h j ⟩ = μ R j 2 ( ( R j 2 + ∥ z ∥ 2 ) 2 q − 2 − R j q − 2 ) > 0.
The positive inner product shows that movement in the next phase changes the base gradient component along h j h_j h j . Therefore, we need to replace Eq. (3) with an alternative formula N j B N_j^B N j B , centered at c j + 1 c_{j+1} c j + 1 , which we define below in Eq. (4) . Its gradient increment at c j + 1 + z c_{j+1}+z c j + 1 + z should be μ ∥ z ∥ q − 2 z \mu\lVert z\rVert^{q-2}z μ ∥ z ∥ q − 2 z , orthogonal to h j h_j h j as required. To make the smooth maximum equal N j B N_j^B N j B near c j + 1 c_{j+1} c j + 1 , we give N j B N_j^B N j B value B j ( c j + 1 ) + C j B_j(c_{j+1})+C_j B j ( c j + 1 ) + C j there, with C j > 0 C_j>0 C j > 0 , while retaining the endpoint gradient. therefore define
N j B ( x ) = B j ( c j + 1 ) + C j + ⟨ ∇ B j ( c j + 1 ) , x − c j + 1 ⟩ + μ q ∥ x − c j + 1 ∥ q . (4) N_j^B(x)=B_j(c_{j+1})+C_j+\langle\nabla B_j(c_{j+1}),x-c_{j+1}\rangle+\frac{\mu}{q}\lVert x-c_{j+1}\rVert^q. \tag*{(4)} N j B ( x ) = B j ( c j + 1 ) + C j + ⟨ ∇ B j ( c j + 1 ) , x − c j + 1 ⟩ + q μ ∥ x − c j + 1 ∥ q . (4)
Step 1.3: Join the formulas near the next center. To preserve earlier values and derivatives, we confine the change to a small ball S j = { x : ∥ x − c j + 1 ∥ < θ R j } S_j=\{x:\lVert x-c_{j+1}\rVert<\theta R_j\} S j = { x : ∥ x − c j + 1 ∥ < θ R j } , with radius parameter θ > 0 \theta>0 θ > 0 . We denote its closure by S ‾ j = { x : ∥ x − c j + 1 ∥ ≤ θ R j } \overline{S}_j=\{x:\lVert x-c_{j+1}\rVert\le\theta R_j\} S j = { x : ∥ x − c j + 1 ∥ ≤ θ R j } and define
B j + 1 ( x ) = { M δ j B ( B j ( x ) , N j B ( x ) ) , ∥ x − c j + 1 ∥ < θ R j , B j ( x ) , ∥ x − c j + 1 ∥ ≥ θ R j . (5) B_{j+1}(x)=
\begin{cases}
M_{\delta_j^B}\left(B_j(x),N_j^B(x)\right), & \lVert x-c_{j+1}\rVert<\theta R_j,\\
B_j(x), & \lVert x-c_{j+1}\rVert\ge\theta R_j.
\end{cases} \tag*{(5)} B j + 1 ( x ) = { M δ j B ( B j ( x ) , N j B ( x ) ) , B j ( x ) , ∥ x − c j + 1 ∥ < θ R j , ∥ x − c j + 1 ∥ ≥ θ R j . (5)
where M δ M_\delta M δ is defined in Eq. (2) . Smoothness and convexity across the boundary require the smooth maximum to equal B j B_j B j there. The next phase’s geometry instead requires equality with N j B N_j^B N j B near the center. Let γ > 0 \gamma>0 γ > 0 denote the inner radius parameter. By Lemma 4.2 , these equalities follow from
B j ( x ) − N j B ( x ) ≥ δ j B if θ R j 2 ≤ ∥ x − c j + 1 ∥ ≤ θ R j , B_j(x)-N_j^B(x)\ge\delta_j^B\quad\text{if}\quad\frac{\theta R_j}{2}\le\lVert x-c_{j+1}\rVert\le\theta R_j, B j ( x ) − N j B ( x ) ≥ δ j B if 2 θ R j ≤ ∥ x − c j + 1 ∥ ≤ θ R j ,
N j B ( x ) − B j ( x ) ≥ δ j B if ∥ x − c j + 1 ∥ ≤ γ R j . (6) N_j^B(x)-B_j(x)\ge\delta_j^B\quad\text{if}\quad\lVert x-c_{j+1}\rVert\le\gamma R_j. \tag*{(6)} N j B ( x ) − B j ( x ) ≥ δ j B if ∥ x − c j + 1 ∥ ≤ γ R j . (6)
Above B j B_j B j ’s tangent at c j + 1 c_{j+1} c j + 1 , B j B_j B j grows quadratically with distance, whereas N j B N_j^B N j B adds C j + μ q ∥ x − c j + 1 ∥ q C_j+\frac{\mu}{q}\lVert x-c_{j+1}\rVert^q C j + q μ ∥ x − c j + 1 ∥ q . Since q > 2 q>2 q > 2 , choosing the value increase and radii together allows B j B_j B j to be larger near the boundary while N j B N_j^B N j B is larger near the center. We choose
a 0 = 1 2 ( 3 4 ) q − 2 , a 1 = q − 1 2 ( 5 4 ) q − 2 , a l i f t = a 0 8 , a_0=\frac{1}{2}\left(\frac{3}{4}\right)^{q-2},\qquad
a_1=\frac{q-1}{2}\left(\frac{5}{4}\right)^{q-2},\qquad
a_{\mathrm{lift}}=\frac{a_0}{8}, a 0 = 2 1 ( 4 3 ) q − 2 , a 1 = 2 q − 1 ( 4 5 ) q − 2 , a lift = 8 a 0 ,
θ = min { 1 16 , ( 3 q a 0 32 ) 1 q − 2 } , γ = θ 3 a 0 32 a 1 , C j = a l i f t μ θ 2 R j q , δ j B = C j 4 . (7) \theta=\min\left\{\frac{1}{16},\left(\frac{3qa_0}{32}\right)^{\frac{1}{q-2}}\right\},\qquad
\gamma=\theta\sqrt{\frac{3a_0}{32a_1}},\qquad
C_j=a_{\mathrm{lift}}\mu\theta^2R_j^q,\qquad
\delta_j^B=\frac{C_j}{4}. \tag*{(7)} θ = min { 16 1 , ( 32 3 q a 0 ) q − 2 1 } , γ = θ 32 a 1 3 a 0 , C j = a lift μ θ 2 R j q , δ j B = 4 C j . (7)
Lemma C.11 in Appx. C derives these choices and proves Eq. (6) . Consequently, B j + 1 = N j B B_{j+1}=N_j^B B j + 1 = N j B for ∥ x − c j + 1 ∥ ≤ γ R j \lVert x-c_{j+1}\rVert\le\gamma R_j ∥ x − c j + 1 ∥ ≤ γ R j , and B j + 1 = B j B_{j+1}=B_j B j + 1 = B j for ∥ x − c j + 1 ∥ ≥ θ R j 2 \lVert x-c_{j+1}\rVert\ge\frac{\theta R_j}{2} ∥ x − c j + 1 ∥ ≥ 2 θ R j . Thus the next phase has its required geometry and starting gradient, while the earlier formula remains unchanged outside the replacement ball.
Step 1.4: Repeat at smaller accuracy scales. With the same chain and normalization parameters across phases, the assembled function F F F will satisfy F ( x ) − min F ≥ μ R j q F(x)-\min F\ge\mu R_j^q F ( x ) − min F ≥ μ R j q on the phase-j j j point sets specified in Lemma 4.5. Here μ R j q = μ ∥ c j + 1 − c j ∥ q \mu R_j^q=\mu\lVert c_{j+1}-c_j\rVert^q μ R j q = μ ∥ c j + 1 − c j ∥ q . A fixed-factor decrease in these quantities therefore supplies logarithmically many phases before the gap reaches ϵ \epsilon ϵ . We obtain this decrease by defining
R j = R 0 η j , μ R j q = μ R 0 q η q j , μ R j + 1 q μ R j q = η q . R_j=R_0\eta^j,\qquad
\mu R_j^q=\mu R_0^q\eta^{qj},\qquad
\frac{\mu R_{j+1}^q}{\mu R_j^q}=\eta^q. R j = R 0 η j , μ R j q = μ R 0 q η q j , μ R j q μ R j + 1 q = η q .
Each subsequent replacement must lie inside the ball where B j + 1 = N j B B_{j+1}=N_j^B B j + 1 = N j B , so that its starting formula has the geometry of Eq. (4) . The triangle inequality gives this containment if ( 1 + θ ) R j + 1 ≤ γ R j 2 (1+\theta)R_{j+1}\le\frac{\gamma R_j}{2} ( 1 + θ ) R j + 1 ≤ 2 γ R j . We consequently require 0 < η ≤ γ 2 ( 1 + θ ) 0<\eta\le\frac{\gamma}{2(1+\theta)} 0 < η ≤ 2 ( 1 + θ ) γ . The next lemma establishes the resulting local formulas, retains earlier values and derivatives, and verifies global smoothness and Euclidean uniform convexity independently of L L L .
Lemma 4.3 (Uniformly convex base). For the constants chosen in Step 1.3, there is C B > 0 C_B>0 C B > 0 , depending only on p , ν p,\nu p , ν , such that the following holds. We have θ ≤ 1 16 \theta\le\frac{1}{16} θ ≤ 16 1 and 0 < γ < θ 2 0<\gamma<\frac{\theta}{2} 0 < γ < 2 θ . For any L ≥ 1 L\ge1 L ≥ 1 , μ , R 0 > 0 \mu,R_0>0 μ , R 0 > 0 , 0 < η ≤ γ 2 ( 1 + θ ) 0<\eta\le\frac{\gamma}{2(1+\theta)} 0 < η ≤ 2 ( 1 + θ ) γ , and mutually orthogonal h 0 , … , h L − 1 h_0,\ldots,h_{L-1} h 0 , … , h L − 1 with ∥ h j ∥ = R 0 η j \lVert h_j\rVert=R_0\eta^j ∥ h j ∥ = R 0 η j , Eq. (5) produces convex B j ∈ C p ( R d ) B_j\in C^p(\mathbb{R}^d) B j ∈ C p ( R d ) and B = B L B=B_L B = B L satisfying:
(a) Smoothness and uniform convexity. *For c B = 1 2 q 16 q − 2 c_B=\frac{1}{2q16^{q-2}} c B = 2 q 1 6 q − 2 1 , all x , y ∈ R d x,y\in\mathbb{R}^d x , y ∈ R d , and 0 ≤ j ≤ L 0\le j\le L 0 ≤ j ≤ L ,
∥ ∇ B j ( y ) − ∇ B j ( x ) ∥ ≤ C B μ ∥ y − x ∥ q − 1 , D B ( y , x ) ≥ c B μ ∥ y − x ∥ q . \lVert\nabla B_j(y)-\nabla B_j(x)\rVert\le C_B\mu\lVert y-x\rVert^{q-1},\qquad
D_B(y,x)\ge c_B\mu\lVert y-x\rVert^q. ∥ ∇ B j ( y ) − ∇ B j ( x )∥ ≤ C B μ ∥ y − x ∥ q − 1 , D B ( y , x ) ≥ c B μ ∥ y − x ∥ q .
(b) Values and gradients at the centers. B ( 0 ) = 0 B(0)=0 B ( 0 ) = 0 , ∇ B ( 0 ) = 0 \nabla B(0)=0 ∇ B ( 0 ) = 0 , and, for 0 ≤ j < L 0 \le j < L 0 ≤ j < L ,
∇ B ( c j + 1 ) − ∇ B ( c j ) = μ R j q − 2 h j , B ( c j + 1 ) − B ( c j ) = β q μ R j q , (8) \nabla B(c_{j+1})-\nabla B(c_j)=\mu R_j^{q-2}h_j,\qquad B(c_{j+1})-B(c_j)=\beta_q\mu R_j^q, \tag*{(8)} ∇ B ( c j + 1 ) − ∇ B ( c j ) = μ R j q − 2 h j , B ( c j + 1 ) − B ( c j ) = β q μ R j q , (8)
where β q = 1 q + a l i f t θ 2 \beta_q=\frac{1}{q}+a_{\mathrm{lift}}\theta^2 β q = q 1 + a lift θ 2 .
(c) Exact local formulas. For 0 ≤ j < L 0 \le j < L 0 ≤ j < L ,
B j + 1 ( x ) = { N j B ( x ) , ∥ x − c j + 1 ∥ ≤ γ R j , B j ( x ) , ∥ x − c j + 1 ∥ ≥ θ R j 2 , B_{j+1}(x)=
\begin{cases}
N_j^B(x), & \lVert x-c_{j+1}\rVert\le\gamma R_j,\\
B_j(x), & \lVert x-c_{j+1}\rVert\ge\frac{\theta R_j}{2},
\end{cases} B j + 1 ( x ) = { N j B ( x ) , B j ( x ) , ∥ x − c j + 1 ∥ ≤ γ R j , ∥ x − c j + 1 ∥ ≥ 2 θ R j ,
Eq. (3) holds globally for j = 0 j=0 j = 0 and on ∥ x − c j ∥ ≤ γ R j − 1 \lVert x-c_j\rVert\le\gamma R_{j-1} ∥ x − c j ∥ ≤ γ R j − 1 for 1 ≤ j ≤ L 1 \le j \le L 1 ≤ j ≤ L .
(d) Preservation of earlier formulas. For the replacement ball S j S_j S j defined in Step 1.3 and 0 ≤ j < L 0 \le j < L 0 ≤ j < L , B ( x ) = B j ( x ) B(x)=B_j(x) B ( x ) = B j ( x ) and ∇ k B ( x ) = ∇ k B j ( x ) \nabla^kB(x)=\nabla^kB_j(x) ∇ k B ( x ) = ∇ k B j ( x ) for x ∉ S j x\notin S_j x ∈ / S j and 1 ≤ k ≤ p 1 \le k \le p 1 ≤ k ≤ p . Also, B = B j B=B_j B = B j on a neighborhood of c j c_j c j for $0 \le j \le L. The partial base B j B_j B j depends only on the previously chosen centers c 0 , … , c j c_0,\ldots,c_j c 0 , … , c j .
Proof sketch. (a) Uniform convexity requires curvature along every segment, even though our local geometry has zero Hessian at the centers. For C = { c 0 , … , c L } C=\{c_0,\ldots,c_L\} C = { c 0 , … , c L } , d = y − x ≠ 0 d=y-x\ne0 d = y − x = 0 , and ℓ = ∥ d ∥ \ell=\lVert d\rVert ℓ = ∥ d ∥ , Lemma C.14 gives ∇ 2 B ( z ) [ d , d ] ≥ μ ℓ 2 dist ( z , C ) q − 2 \nabla^2B(z)[d,d]\ge\mu\ell^2\operatorname{dist}(z,C)^{q-2} ∇ 2 B ( z ) [ d , d ] ≥ μ ℓ 2 dist ( z , C ) q − 2 . Lemma C.15 guarantees that at least half the segment stays at distance ℓ 16 \frac{\ell}{16} 16 ℓ from every center. Integrating the curvature over this part gives gradient growth. A further integration yields D B ( y , x ) ≥ c B μ ∥ y − x ∥ q D_B(y,x)\ge c_B\mu\lVert y-x\rVert^q D B ( y , x ) ≥ c B μ ∥ y − x ∥ q . For smoothness, each correction B j + 1 − B j B_{j+1}-B_j B j + 1 − B j has p t h p^{\mathrm{th}} p th -derivative magnitude bounded by C μ R j ν C\mu R_j^\nu C μ R j ν . The geometric decrease of R j R_j R j bounds their sum independently of L L L ; the exact local formulas handle nearby points. The complete proof is given by Corollary C.5 in Appx. C.
Step 2: Add Hardness to the Uniformly Convex Base
The base B B B supplies the required uniform convexity and smoothness, but does not yet force oracle calls. In particular, the initial point c 0 = 0 c_0=0 c 0 = 0 already minimizes B B B , since
B ( x ) ≥ B ( 0 ) + ⟨ ∇ B ( 0 ) , x ⟩ + c B μ ∥ x ∥ q = c B μ ∥ x ∥ q , (9) B(x)\ge B(0)+\langle\nabla B(0),x\rangle+c_B\mu\lVert x\rVert^q=c_B\mu\lVert x\rVert^q, \tag*{(9)} B ( x ) ≥ B ( 0 ) + ⟨ ∇ B ( 0 ) , x ⟩ + c B μ ∥ x ∥ q = c B μ ∥ x ∥ q , (9)
where the inequality follows from Lemma 4.3 (a) and the equality from Lemma 4.3 (b). Thus returning 0 0 0 attains zero gap without an oracle call. We therefore add convex chains that restrict the directions revealed by each oracle output and leave a positive gap at points using only the first few chain coordinates, while preserving the base’s uniform convexity.
Step 2.1: Define the short chains. Since each phase should force the same condition-number cost, we use a common length m ≥ 1 m\ge1 m ≥ 1 , independent of L L L . To give each phase fresh chain directions and keep the center increments orthogonal, we choose mutually orthogonal blocks for 0 ≤ j < L 0\le j<L 0 ≤ j < L , E j = span { v j , u j , 1 , … , u j , m } E_j=\operatorname{span}\{v_j,u_{j,1},\ldots,u_{j,m}\} E j = span { v j , u j , 1 , … , u j , m } , with orthonormal bases and starting directions v j v_j v j . We refer to the subspace U j , b : = span { v j , u j , 1 , … , u j , b } U_{j,b}:=\operatorname{span}\{v_j,u_{j,1},\ldots,u_{j,b}\} U j , b := span { v j , u j , 1 , … , u j , b } , for b ∈ { 0 , … , m } b\in\{0,\ldots,m\} b ∈ { 0 , … , m } , as a chain prefix , and call it a short prefix when b ≤ ⌊ m 4 ⌋ b\le\left\lfloor\frac{m}{4}\right\rfloor b ≤ ⌊ 4 m ⌋ . Following the power chain in [20 ] , Eq. (5.4), we adapt its scaling to our phase geometry. For w ∈ E j w\in E_j w ∈ E j , we denote its coordinates in this orthonormal basis by t = ⟨ v j , w ⟩ t=\langle v_j,w\rangle t = ⟨ v j , w ⟩ and x i = ⟨ u j , i , w ⟩ x_i=\langle u_{j,i},w\rangle x i = ⟨ u j , i , w ⟩ , and define, for λ > 0 \lambda>0 λ > 0 ,
Q j ( w ) = λ q ( ∣ t m − x 1 ∣ q + ∑ i = 1 m − 1 ∣ x i − x i + 1 ∣ q + ∣ x m ∣ q ) . (10) Q_j(w)=\frac{\lambda}{q}\left(\left|\frac{t}{\sqrt{m}}-x_1\right|^q+\sum_{i=1}^{m-1}|x_i-x_{i+1}|^q+|x_m|^q\right). \tag*{(10)} Q j ( w ) = q λ ( m t − x 1 q + i = 1 ∑ m − 1 ∣ x i − x i + 1 ∣ q + ∣ x m ∣ q ) . (10)
Choose the phase endpoint. To pass from one phase to the next, we need an endpoint at which the chain’s gradient can be balanced with the base’s gradient increment. We choose one with no chain gradient in the internal directions u j , i u_{j,i} u j , i . We refer to the differences inside Eq. (10) as chain residuals . Since each x i x_i x i appears with opposite signs in consecutive residuals, we cancel the internal gradient components by requiring
t m − x 1 = x 1 − x 2 = ⋯ = x m − 1 − x m = x m = τ j > 0. \frac{t}{\sqrt{m}}-x_1=x_1-x_2=\cdots=x_{m-1}-x_m=x_m=\tau_j>0. m t − x 1 = x 1 − x 2 = ⋯ = x m − 1 − x m = x m = τ j > 0.
The middle differences are omitted when m = 1 m=1 m = 1 . This follows the minimizer calculation of [20 Lemma 5.2] . Solving these equalities gives the center increment
t j ⋆ = m ( m + 1 ) τ j , h j = t j ⋆ v j + ∑ i = 1 m ( m − i + 1 ) τ j u j , i , R j = ∥ h j ∥ . (11) t_j^\star=\sqrt{m}(m+1)\tau_j,\qquad\boldsymbol{h}_j=t_j^\star\boldsymbol{v}_j+\sum_{i=1}^{m}(m-i+1)\tau_j\boldsymbol{u}_{j,i},\qquad R_j=\lVert\boldsymbol{h}_j\rVert. \tag*{(11)} t j ⋆ = m ( m + 1 ) τ j , h j = t j ⋆ v j + i = 1 ∑ m ( m − i + 1 ) τ j u j , i , R j = ∥ h j ∥ . (11)
The remaining chain gradient is ∇ Q j ( h j ) = Γ j v j \nabla Q_j(\boldsymbol{h}_j)=\Gamma_j\boldsymbol{v}_j ∇ Q j ( h j ) = Γ j v j , with Γ j = λ τ j q − 1 m \Gamma_j=\frac{\lambda\tau_j^{q-1}}{\sqrt{m}} Γ j = m λ τ j q − 1 . The base adds μ R j q − 2 h j \mu R_j^{q-2}\boldsymbol{h}_j μ R j q − 2 h j by Eq. (8) . To bound the base-to-chain gradient ratio in Step 3 and keep earlier evaluation points outside later replacement balls in Step 5, we need the endpoint length in terms of τ j \tau_j τ j and m m m . Lemma D.3 in Appx. D proves the chain gradient above and gives R j = β m τ j m 3 2 R_j=\beta_m\tau_jm^{\frac{3}{2}} R j = β m τ j m 2 3 with 1 ≤ β m ≤ 5 1\leq\beta_m\leq\sqrt{5} 1 ≤ β m ≤ 5 .
Limit the revealed directions. The endpoint involves all m m m internal directions, so the oracle must not reveal all of them in a single oracle output. For an evaluation point w ∈ U j , b \boldsymbol{w}\in U_{j,b} w ∈ U j , b , we have x i = 0 x_i=0 x i = 0 for i > b i>b i > b . For b < m b<m b < m , its last possibly nonzero residual uses u j , b + 1 \boldsymbol{u}_{j,b+1} u j , b + 1 . Since q > p q>p q > p , the ∣ ⋅ ∣ q \lvert\cdot\rvert^q ∣ ⋅ ∣ q terms have zero derivatives of orders 1 , … , p 1,\ldots,p 1 , … , p at zero. Thus the value and derivatives through order p p p depend only on this prefix and next direction, regardless of ∥ w ∥ \lVert\boldsymbol{w}\rVert ∥ w ∥ .
Step 2.2: Create a nonzero starting gradient. The same vanishing derivatives give ∇ k Q j ( 0 ) = 0 \nabla^k Q_j(0)=0 ∇ k Q j ( 0 ) = 0 for 1 ≤ k ≤ p 1\leq k\leq p 1 ≤ k ≤ p , so the chain alone creates no starting slope. Our phase functions will retain these derivative values at their local zero coordinates. Since c j ∈ ⨁ k < j E k \boldsymbol{c}_j\in\bigoplus_{k<j}E_k c j ∈ ⨁ k < j E k and E k ⊥ E ℓ E_k\perp E_\ell E k ⊥ E ℓ for k ≠ ℓ k\ne\ell k = ℓ , we have P E j ( 0 − c j ) = 0 P_{E_j}(0-\boldsymbol{c}_j)=0 P E j ( 0 − c j ) = 0 and ⟨ v j + 1 , 0 − c j ⟩ = 0 \langle\boldsymbol{v}_{j+1},0-\boldsymbol{c}_j\rangle=0 ⟨ v j + 1 , 0 − c j ⟩ = 0 when j < L − 1 j<L-1 j < L − 1 , so adding the phase functions alone would still leave 0 0 0 optimal. We therefore include ℓ i n i t ( x ) = − s 0 ⟨ v 0 , x ⟩ \ell_{\mathrm{init}}(\boldsymbol{x})=-s_0\langle\boldsymbol{v}_0,\boldsymbol{x}\rangle ℓ init ( x ) = − s 0 ⟨ v 0 , x ⟩ , with s 0 > 0 s_0>0 s 0 > 0 to be chosen later, to give starting gradient − s 0 v 0 -s_0\boldsymbol{v}_0 − s 0 v 0 . By Lemma B.20 in Appx. B, this affine term preserves the Bregman divergence and, since p ≥ 2 p\geq2 p ≥ 2 , leaves ∇ p B \nabla^p B ∇ p B unchanged.
Step 2.3: Keep the gradient of each completed phase fixed. To repeat the same chain hardness in each phase, we require the starting gradient at c j \boldsymbol{c}_j c j to be − s j v j -s_j\boldsymbol{v}_j − s j v j , with no component in any earlier block E k E_k E k , k < j k<j k < j . At c j + 1 \boldsymbol{c}_{j+1} c j + 1 , the base adds b j = μ R j q − 2 h j \boldsymbol{b}_j=\mu R_j^{q-2}\boldsymbol{h}_j b j = μ R j q − 2 h j to its gradient. To cancel this increment and transfer the starting slope from v j \boldsymbol{v}_j v j to v j + 1 \boldsymbol{v}_{j+1} v j + 1 , the phase function must contribute a gradient g j \boldsymbol{g}_j g j satisfying − s j v j + b j + g j = − s j + 1 v j + 1 -s_j\boldsymbol{v}_j+\boldsymbol{b}_j+\boldsymbol{g}_j=-s_{j+1}\boldsymbol{v}_{j+1} − s j v j + b j + g j = − s j + 1 v j + 1 . We match ⟨ g j , v j ⟩ \langle\boldsymbol{g}_j,\boldsymbol{v}_j\rangle ⟨ g j , v j ⟩ to the chain’s endpoint gradient component Γ j \Gamma_j Γ j , which determines s j s_j s j :
g j = s j v j − s j + 1 v j + 1 − b j , s j = Γ j + G j t j ⋆ R j , G j = ∥ b j ∥ . (12) \boldsymbol{g}_j=s_j\boldsymbol{v}_j-s_{j+1}\boldsymbol{v}_{j+1}-\boldsymbol{b}_j,\qquad s_j=\Gamma_j+\frac{G_jt_j^\star}{R_j},\qquad G_j=\lVert\boldsymbol{b}_j\rVert. \tag*{(12)} g j = s j v j − s j + 1 v j + 1 − b j , s j = Γ j + R j G j t j ⋆ , G j = ∥ b j ∥ . (12)
For j < L − 1 j<L-1 j < L − 1 , we denote the coordinate along v j + 1 \boldsymbol{v}_{j+1} v j + 1 by t j + 1 t_{j+1} t j + 1 , distinguishing this variable from the fixed endpoint coordinate t j + 1 ⋆ t_{j+1}^\star t j + 1 ⋆ . Relative to c j \boldsymbol{c}_j c j , the center c j + 1 \boldsymbol{c}_{j+1} c j + 1 has coordinates ( h j , 0 ) (\boldsymbol{h}_j,0) ( h j , 0 ) , and all subsequent centers have ( h j , t j + 1 ⋆ ) (\boldsymbol{h}_j,t_{j+1}^\star) ( h j , t j + 1 ⋆ ) . Since the completed phase must contribute g j \boldsymbol{g}_j g j at each of them, we define the affine function
I j ( w , t j + 1 ) = ⟨ g j , w − h j + t j + 1 v j + 1 ⟩ . (13) I_j(\boldsymbol{w},t_{j+1})=\langle\boldsymbol{g}_j,\boldsymbol{w}-\boldsymbol{h}_j+t_{j+1}\boldsymbol{v}_{j+1}\rangle. \tag*{(13)} I j ( w , t j + 1 ) = ⟨ g j , w − h j + t j + 1 v j + 1 ⟩ . (13)
To bound the derivatives of the smooth maximum, we need to control the chain’s values where it meets the affine function. However, Q j − I j Q_j-I_j Q j − I j contains the term s j + 1 t j + 1 s_{j+1}t_{j+1} s j + 1 t j + 1 , which is unbounded below as t j + 1 → − ∞ t_{j+1}\to-\infty t j + 1 → − ∞ . To control this direction as well, we add a convex regularizer,
P j ( w , t j + 1 ) = Q j ( w ) + α q ∣ t j + 1 ∣ q , α = λ m 1 − 3 q 2 . (14) P_j(\boldsymbol{w},t_{j+1})=Q_j(\boldsymbol{w})+\frac{\alpha}{q}\lvert t_{j+1}\rvert^q,\qquad\alpha=\lambda m^{1-\frac{3q}{2}}. \tag*{(14)} P j ( w , t j + 1 ) = Q j ( w ) + q α ∣ t j + 1 ∣ q , α = λ m 1 − 2 3 q . (14)
This choice makes the added term comparable to λ m τ j q \lambda m\tau_j^q λm τ j q when ∣ t j + 1 ∣ = τ j m 3 2 \lvert t_{j+1}\rvert=\tau_jm^{\frac{3}{2}} ∣ t j + 1 ∣ = τ j m 2 3 . To make the affine formula larger at ( h j , 0 ) (\boldsymbol{h}_j,0) ( h j , 0 ) , we choose δ j = c 1 λ m τ j q \delta_j=c_1\lambda m\tau_j^q δ j = c 1 λm τ j q , with c 0 = c 1 = 2 − q − 3 q c_0=c_1=\frac{2-q^{-3}}{q} c 0 = c 1 = q 2 − q − 3 , and shift the regularized chain down:
O j ( w , t j + 1 ) = P j ( w , t j + 1 ) − P j ( h j , 0 ) − 4 δ j . (15) O_j(\boldsymbol{w},t_{j+1})=P_j(\boldsymbol{w},t_{j+1})-P_j(\boldsymbol{h}_j,0)-4\delta_j. \tag*{(15)} O j ( w , t j + 1 ) = P j ( w , t j + 1 ) − P j ( h j , 0 ) − 4 δ j . (15)
Now I j ( h j , 0 ) = 0 I_j(\boldsymbol{h}_j,0)=0 I j ( h j , 0 ) = 0 and O j ( h j , 0 ) = − 4 δ j O_j(\boldsymbol{h}_j,0)=-4\delta_j O j ( h j , 0 ) = − 4 δ j . The smooth maximum therefore agrees with I j I_j I j near this point, while retaining O j O_j O j wherever O j − I j > δ j O_j-I_j>\delta_j O j − I j > δ j . We define the phase function by
A j ( w , t j + 1 ) = M δ j ( I j , O j ) . (16) A_j(w,t_{j+1})=M_{\delta_j}(I_j,O_j). \tag*{(16)} A j ( w , t j + 1 ) = M δ j ( I j , O j ) . (16)
For j = L − 1 j=L-1 j = L − 1 , we omit t L , v L t_L,v_L t L , v L and set s L = t L ⋆ = 0 s_L=t_L^\star=0 s L = t L ⋆ = 0 , with s L v L : = 0 s_Lv_L:=0 s L v L := 0 . These functions are expressed relative to the center c j c_j c j . To add them to B B B , we extend each T ∈ { P , I , O , A } T\in\{P,I,O,A\} T ∈ { P , I , O , A } to R d \mathbb{R}^d R d using the projection onto E j E_j E j and the next starting direction:
T j ( x ) : = T j ( P E j ( x − c j ) , ⟨ v j + 1 , x − c j ⟩ ) , (17) T_j(x):=T_j\left(P_{E_j}(x-c_j),\langle v_{j+1},x-c_j\rangle\right), \tag*{(17)} T j ( x ) := T j ( P E j ( x − c j ) , ⟨ v j + 1 , x − c j ⟩ ) , (17)
with the second input omitted when j = L − 1 j=L-1 j = L − 1 . It remains to verify that the smooth maximum uses the chain at short prefixes and the affine function near completed centers, while preserving global smoothness. For ρ j A = ( δ j λ ) 1 q \rho_j^A=\left(\frac{\delta_j}{\lambda}\right)^{\frac{1}{q}} ρ j A = ( λ δ j ) q 1 , the critical relation q = p + ν q=p+\nu q = p + ν gives δ j ( ρ j A ) p + ν = λ \frac{\delta_j}{(\rho_j^A)^{p+\nu}}=\lambda ( ρ j A ) p + ν δ j = λ , so the smoothing normalization is independent of τ j \tau_j τ j . The following lemma verifies these formulas and starting values, with a global smoothness bound independent of m m m and dimension.
Lemma 4.4 (The phase function). There is a constant C A > 0 C_A>0 C A > 0 , depending only on p , ν p,\nu p , ν , with the following property. In Eqs. (10) –(4.12), take m ≥ 1 m\geq1 m ≥ 1 , λ , τ j > 0 \lambda,\tau_j>0 λ , τ j > 0 , and G j , s j + 1 , t j + 1 ⋆ ≥ 0 G_j,s_{j+1},t_{j+1}^\star\geq0 G j , s j + 1 , t j + 1 ⋆ ≥ 0 . Define b j = G j R j h j b_j=\frac{G_j}{R_j}h_j b j = R j G j h j and the slope by Eq. (12) , using the scalar parameters above. Assume
G j ≤ c 0 Γ j , α − 1 q − 1 s j + 1 q q − 1 ≤ c 0 λ m τ j q , s j + 1 t j + 1 ⋆ + α q ( t j + 1 ⋆ ) q ≤ c 0 λ m τ j q . (18) G_j\leq c_0\Gamma_j,\qquad\alpha^{-\frac{1}{q-1}}s_{j+1}^{\frac{q}{q-1}}\leq c_0\lambda m\tau_j^q,\qquad s_{j+1}t_{j+1}^\star+\frac{\alpha}{q}(t_{j+1}^\star)^q\leq c_0\lambda m\tau_j^q. \tag*{(18)} G j ≤ c 0 Γ j , α − q − 1 1 s j + 1 q − 1 q ≤ c 0 λm τ j q , s j + 1 t j + 1 ⋆ + q α ( t j + 1 ⋆ ) q ≤ c 0 λm τ j q . (18) (a) Convexity and smoothness. A j A_j A j is convex and C p , ν C^{p,\nu} C p , ν , with ∥ ∇ p A j ( x ) − ∇ p A j ( y ) ∥ ≤ C A λ ∥ x − y ∥ ν \lVert\nabla^p A_j(x)-\nabla^p A_j(y)\rVert\leq C_A\lambda\lVert x-y\rVert^\nu ∥ ∇ p A j ( x ) − ∇ p A j ( y )∥ ≤ C A λ ∥ x − y ∥ ν for all x , y ∈ R d x,y\in\mathbb{R}^d x , y ∈ R d , where C A C_A C A is independent of m m m and dimension.
(b) Chain formulas and oracle outputs. For every w ∈ U j , b w\in U_{j,b} w ∈ U j , b and 0 ≤ b ≤ ⌊ m 4 ⌋ 0\leq b\leq\left\lfloor\frac{m}{4}\right\rfloor 0 ≤ b ≤ ⌊ 4 m ⌋ , O j ( w , 0 ) − I j ( w , 0 ) ≥ 7 δ j O_j(w,0)-I_j(w,0)\geq7\delta_j O j ( w , 0 ) − I j ( w , 0 ) ≥ 7 δ j , and the functions on R d \mathbb{R}^d R d defined by Eq. (17) satisfy A j = O j A_j=O_j A j = O j near c j + w c_j+w c j + w . In particular, A j ( c j + w ) = Q j ( w ) − λ ( m + 1 ) τ j q q − 4 δ j A_j(c_j+w)=Q_j(w)-\frac{\lambda(m+1)\tau_j^q}{q}-4\delta_j A j ( c j + w ) = Q j ( w ) − q λ ( m + 1 ) τ j q − 4 δ j . For 1 ≤ k ≤ p 1\leq k\leq p 1 ≤ k ≤ p and all a 1 , … , a k ∈ R d a_1,\ldots,a_k\in\mathbb{R}^d a 1 , … , a k ∈ R d , the derivative tensors on R d \mathbb{R}^d R d satisfy
∇ k A j ( c j + w ) [ a 1 , … , a k ] = ∇ k Q j ( w ) [ P U j , b + 1 a 1 , … , P U j , b + 1 a k ] . \nabla^k A_j(c_j+w)[a_1,\ldots,a_k]=\nabla^k Q_j(w)[P_{U_{j,b+1}}a_1,\ldots,P_{U_{j,b+1}}a_k]. ∇ k A j ( c j + w ) [ a 1 , … , a k ] = ∇ k Q j ( w ) [ P U j , b + 1 a 1 , … , P U j , b + 1 a k ] . The value and these tensors are unchanged by varying the remaining chain directions and, when present, v j + 1 v_{j+1} v j + 1 in an orthonormal completion.
(c) Gradient after completing the phase. A j = I j A_j=I_j A j = I j near { ( h j , t j + 1 ) : 0 ≤ t j + 1 ≤ t j + 1 ⋆ } \{(h_j,t_{j+1}):0\leq t_{j+1}\leq t_{j+1}^\star\} {( h j , t j + 1 ) : 0 ≤ t j + 1 ≤ t j + 1 ⋆ } . On a neighborhood of { c j + h j + t v j + 1 : 0 ≤ t ≤ t j + 1 ⋆ } \{c_j+h_j+tv_{j+1}:0\leq t\leq t_{j+1}^\star\} { c j + h j + t v j + 1 : 0 ≤ t ≤ t j + 1 ⋆ } in R d \mathbb{R}^d R d (with the last term omitted when j = L − 1 j=L-1 j = L − 1 ), ∇ A j = g j \nabla A_j=g_j ∇ A j = g j and ∇ k A j = 0 \nabla^k A_j=0 ∇ k A j = 0 for 2 ≤ k ≤ p 2\leq k\leq p 2 ≤ k ≤ p . Its values at the two endpoints are 0 0 0 and − s j + 1 t j + 1 ⋆ -s_{j+1}t_{j+1}^\star − s j + 1 t j + 1 ⋆ .
(d) Starting and terminal data. A j ( 0 , 0 ) = − λ ( m + 1 ) τ j q q − 4 δ j A_j(0,0)=-\frac{\lambda(m+1)\tau_j^q}{q}-4\delta_j A j ( 0 , 0 ) = − q λ ( m + 1 ) τ j q − 4 δ j and ∇ k A j ( 0 , 0 ) = 0 \nabla^kA_j(0,0)=0 ∇ k A j ( 0 , 0 ) = 0 for 1 ≤ k ≤ p 1\leq k\leq p 1 ≤ k ≤ p . The terminal phase satisfies the same conclusions with the next coordinate omitted and the completed segment reduced to { c L } \{c_L\} { c L } .
The proof is given by Theorem D.11 in Appx. D.
Step 3: Choose Compatible Parameters and Assemble the Function
Selecting O j O_j O j on short chain prefixes requires G j ≤ c 0 Γ j G_j\leq c_0\Gamma_j G j ≤ c 0 Γ j in Eq. (18) . Since R j = β m τ j m 3 2 R_j=\beta_m\tau_jm^{\frac{3}{2}} R j = β m τ j m 2 3 , the ratio of the base increment G j G_j G j to Γ j \Gamma_j Γ j is independent of τ j \tau_j τ j . We choose τ 0 > 0 \tau_0>0 τ 0 > 0 and
a q = 3 q − 2 2 , λ = K μ m a q , α = K μ , τ j = η j τ 0 , c j = ∑ k < j h k . (19) a_q=\frac{3q-2}{2},\qquad\lambda=K\mu m^{a_q},\qquad\alpha=K\mu,\qquad\tau_j=\eta^j\tau_0,\qquad c_j=\sum_{k<j}h_k. \tag*{(19)} a q = 2 3 q − 2 , λ = K μ m a q , α = K μ , τ j = η j τ 0 , c j = k < j ∑ h k . (19)
Then
G j Γ j = μ R j q − 1 m λ τ j q − 1 = μ λ β m q − 1 m α q = β m q − 1 K . (20) \frac{G_j}{\Gamma_j}=\frac{\mu R_j^{q-1}\sqrt{m}}{\lambda\tau_j^{q-1}}=\frac{\mu}{\lambda}\beta_m^{q-1}m^{\alpha q}=\frac{\beta_m^{q-1}}{K}. \tag*{(20)} Γ j G j = λ τ j q − 1 μ R j q − 1 m = λ μ β m q − 1 m α q = K β m q − 1 . (20)
Thus K = 5 q − 1 2 c 0 K=\frac{5^{\frac{q-1}{2}}}{c_0} K = c 0 5 2 q − 1 gives G j ≤ c 0 Γ j G_j\le c_0\Gamma_j G j ≤ c 0 Γ j for every phase. Choosing η = γ 2 ( 1 + θ ) \eta=\frac{\gamma}{2(1+\theta)} η = 2 ( 1 + θ ) γ also satisfies the next-phase slope, endpoint, and base-nesting conditions by Proposition E.5 in Appx. E. All constants depend only on p , ν p,\nu p , ν .
We then assemble
F ( x ) = B ( x ) + ℓ i n i t ( x ) + ∑ j = 0 L − 1 A j ( x ) . (21) F(x)=B(x)+\ell_{\mathrm{init}}(x)+\sum_{j=0}^{L-1}A_j(x). \tag*{(21)} F ( x ) = B ( x ) + ℓ init ( x ) + j = 0 ∑ L − 1 A j ( x ) . (21)
An algorithm may retain completed-phase information, so we include those subspaces in the current-prefix space W j , b : = ( ⨁ k < j E k ) ⊕ U j , b W_{j,b}:=\left(\bigoplus_{k<j}E_k\right)\oplus U_{j,b} W j , b := ( ⨁ k < j E k ) ⊕ U j , b , for 0 ≤ j < L 0\le j<L 0 ≤ j < L and 0 ≤ b ≤ m 0\le b\le m 0 ≤ b ≤ m . The sum over k < j k<j k < j is { 0 } \{0\} { 0 } when j = 0 j=0 j = 0 .
Lemma 4.5 (The complete objective). Let m , L ≥ 1 m,L\ge1 m , L ≥ 1 and μ , τ 0 > 0 \mu,\tau_0>0 μ , τ 0 > 0 . Choose the parameters in Eq. (19) with K = 5 q − 1 2 c 0 K=\frac{5^{\frac{q-1}{2}}}{c_0} K = c 0 5 2 q − 1 and η = γ 2 ( 1 + θ ) \eta=\frac{\gamma}{2(1+\theta)} η = 2 ( 1 + θ ) γ , and construct B , A j B,A_j B , A j by Eqs. (5) and (4.12). Then F F F in Eq. (21) satisfies:
(a) Smoothness and uniform convexity. For all x , y ∈ R d x,y\in\mathbb{R}^d x , y ∈ R d ,
D F ( y , x ) ≥ c B μ ∥ y − x ∥ q , ∥ ∇ p F ( x ) − ∇ p F ( y ) ∥ ≤ C H μ m α q ∥ x − y ∥ ν , D_F(y,x)\ge c_B\mu\lVert y-x\rVert^q,\qquad\lVert\nabla^pF(x)-\nabla^pF(y)\rVert\le C_H\mu m^{\alpha q}\lVert x-y\rVert^\nu, D F ( y , x ) ≥ c B μ ∥ y − x ∥ q , ∥ ∇ p F ( x ) − ∇ p F ( y )∥ ≤ C H μ m α q ∥ x − y ∥ ν , where C H : = C B + 2 C A K C_H:=C_B+2C_AK C H := C B + 2 C A K depends only on p , ν p,\nu p , ν .
(b) Center gradients and minimizer. ∇ F ( c j ) = − s j v j \nabla F(c_j)=-s_jv_j ∇ F ( c j ) = − s j v j for j < L j<L j < L , ∇ F ( c L ) = 0 \nabla F(c_L)=0 ∇ F ( c L ) = 0 , and c L c_L c L is the unique minimizer.
(c) Optimality gap on chain prefixes. Every x ∈ W j , b x\in W_{j,b} x ∈ W j , b with b ≤ ⌊ m 4 ⌋ b\le\left\lfloor\frac{m}{4}\right\rfloor b ≤ ⌊ 4 m ⌋ satisfies F ( x ) − F ( c L ) ≥ 7 δ j F(x)-F(c_L)\ge7\delta_j F ( x ) − F ( c L ) ≥ 7 δ j , where δ j = c 1 λ m τ 0 q \delta_j=c_1\lambda m\tau_0^q δ j = c 1 λm τ 0 q and c 1 = 2 − q − a − 3 c_1=2^{-q-a-3} c 1 = 2 − q − a − 3 .
(d) Objective decay and initial gap. Define the decrease between consecutive centers by χ j : = F ( c j ) − F ( c j + 1 ) \chi_j:=F(c_j)-F(c_{j+1}) χ j := F ( c j ) − F ( c j + 1 ) . The initial optimality gap D m , L : = F ( c 0 ) − F ( c L ) D_{m,L}:=F(c_0)-F(c_L) D m , L := F ( c 0 ) − F ( c L ) satisfies
χ j = η q j χ 0 , D m , L = χ 0 1 − η q L 1 − η q . (22) \chi_j=\eta^{qj}\chi_0,\qquad D_{m,L}=\chi_0\frac{1-\eta^{qL}}{1-\eta^q}. \tag*{(22)} χ j = η q j χ 0 , D m , L = χ 0 1 − η q 1 − η q L . (22) Here 0 < χ 0 ≤ 2 ( 1 + c 0 ) λ m τ 0 q 0<\chi_0\le2(1+c_0)\lambda m\tau_0^q 0 < χ 0 ≤ 2 ( 1 + c 0 ) λm τ 0 q . The value of D m , L D_{m,L} D m , L is independent of the orthonormal phase directions.
Proof sketch. (a) Adding convex phase functions preserves the uniform convexity of B B B . To avoid accumulating a smoothness cost proportional to L L L , we group phases by parity. Each A j A_j A j depends only on V j = E j ⊕ span { v j + 1 } V_j=E_j\oplus\operatorname{span}\{v_{j+1}\} V j = E j ⊕ span { v j + 1 } , with V L − 1 = E L − 1 V_{L-1}=E_{L-1} V L − 1 = E L − 1 , and these subspaces are orthogonal within each group. The norm of each group’s derivative sum is therefore its largest component norm, giving a total phase Hölder bound 2 C A λ 2C_A\lambda 2 C A λ . (b) The center gradients follow by telescoping b j + g j = s j v j − s j + 1 v j + 1 b_j+g_j=s_jv_j-s_{j+1}v_{j+1} b j + g j = s j v j − s j + 1 v j + 1 . (c) To keep the other components from offsetting the chain gap, consider the convex function Ψ j = F − A j + I j \Psi_j=F-A_j+I_j Ψ j = F − A j + I j , which equals F F F near c j + 1 c_{j+1} c j + 1 . For x ∈ W j , b x\in W_{j,b} x ∈ W j , b , the center gradient is perpendicular to x − c j + 1 x-c_{j+1} x − c j + 1 ; it is zero in the terminal phase. Convexity therefore gives Ψ j ( x ) ≥ F ( c j + 1 ) \Psi_j(x)\ge F(c_{j+1}) Ψ j ( x ) ≥ F ( c j + 1 ) . Adding A j ( x ) − I j ( x ) ≥ 7 δ j A_j(x)-I_j(x)\ge7\delta_j A j ( x ) − I j ( x ) ≥ 7 δ j gives F ( x ) ≥ min F + 7 δ j F(x)\ge\min F+7\delta_j F ( x ) ≥ min F + 7 δ j . The full proof follows this order in Appx. E.3.
Steps 4–5: Generate the Trajectory and Prove Oracle Indistinguishability
Step 4: Generate the trajectory. To apply Lemma 4.5(c) to an arbitrary deterministic algorithm A A A , we must place its candidate solutions in short prefixes W j , b W_{j,b} W j , b . We therefore choose the unfixed chain directions as the algorithm runs, while fixing the scalar parameters of Step 3 and d ≥ L ( m + 1 ) d \ge L(m+1) d ≥ L ( m + 1 ) beforehand. For m ≥ 32 m \ge32 m ≥ 32 , define n m = ⌊ m 16 ⌋ n_m=\left\lfloor\frac{m}{16}\right\rfloor n m = ⌊ 16 m ⌋ . Group j j j consists of oracle calls j n m + 1 , … , ( j + 1 ) n m jn_m+1,\ldots,(j+1)n_m j n m + 1 , … , ( j + 1 ) n m .
For oracle call s s s , denote the evaluation point by x s x_s x s , the returned value and full derivative tensors by O s = ( a s , 0 , … , a s , p ) O_s=(a_{s,0},\ldots,a_{s,p}) O s = ( a s , 0 , … , a s , p ) , and the candidate solution after this call by x ^ s \hat{x}_s x ^ s . We impose no norm bound on these points, and a candidate solution need not be an evaluation point. Lemma F.3 in Appx. F allows any point to be included in W j , b W_{j,b} W j , b by choosing at most one additional chain direction. We begin by choosing v 0 v_0 v 0 and including x ^ 0 \hat{x}_0 x ^ 0 , using at most one direction. Within group j j j , let b b b count the chosen directions u j , 1 , … , u j , b u_{j,1},\ldots,u_{j,b} u j , 1 , … , u j , b . Each call consists of three ordered selections, with b b b increased whenever a direction is chosen.
The algorithm determines x s + 1 x_{s+1} x s + 1 from the earlier oracle outputs. We include it in W j , b W_{j,b} W j , b using at most one direction.
The current chain can involve u j , b + 1 u_{j,b+1} u j , b + 1 in its full derivative tensors, so we fix one further unit direction perpendicular to W j , b W_{j,b} W j , b . Any orthonormal completion gives an objective F ~ \widetilde{F} F , from which we return a s + 1 , k = ∇ k F ~ ( x s + 1 ) a_{s+1,k}=\nabla^k\widetilde{F}(x_{s+1}) a s + 1 , k = ∇ k F ( x s + 1 ) for 0 ≤ k ≤ p 0\le k\le p 0 ≤ k ≤ p . Step 5 proves independence from the remaining completion.
The new oracle output determines x ^ s + 1 \hat{x}_{s+1} x ^ s + 1 . We include this candidate solution in W j , b W_{j,b} W j , b using at most one further direction.
Thus each oracle call fixes at most three chain directions. After the last call in group j j j , we complete its block E j E_j E j . If j < L − 1 j<L-1 j < L − 1 , we choose v j + 1 v_{j+1} v j + 1 perpendicular to the completed blocks and begin the next group with b = 0 b=0 b = 0 .
Step 5: Prove oracle indistinguishability and complete one instance. We prove that one fixed objective reproduces the adaptive trajectory in Step 4. Let F s \mathcal{F}_s F s be the family of objectives in Eq. (21) obtained by completing the directions fixed after s s s oracle calls, including those chosen for x ^ s \hat{x}_s x ^ s and preparation of the next group. Equality of the recorded oracle outputs across these completions will make the deterministic algorithm reproduce the trajectory, allowing us to apply the prefix gaps from Step 3.
Proposition 4.6 (Oracle indistinguishability and one fixed instance). Fix the scalar parameters specified in Step 3, m ≥ 32 m\ge32 m ≥ 32 , L ≥ 1 L\ge1 L ≥ 1 , and d ≥ L ( m + 1 ) d\ge L(m+1) d ≥ L ( m + 1 ) . Put n m = ⌊ m 16 ⌋ n_m=\left\lfloor\frac{m}{16}\right\rfloor n m = ⌊ 16 m ⌋ . For every deterministic p t h p^{\mathrm{th}} p th -order oracle algorithm, the ordered orthonormal directions in Step 4 can be chosen so that:
(a) Oracle indistinguishability. For every 0 ≤ s ≤ n m L 0\le s\le n_mL 0 ≤ s ≤ n m L ,
F s ≠ ∅ , ∇ k F ( x ℓ ) = a ℓ , k ( F ∈ F s , 1 ≤ ℓ ≤ s , 0 ≤ k ≤ p ) . (23) \mathcal{F}_s\ne\varnothing,\qquad\nabla^kF(x_\ell)=a_{\ell,k}\quad(F\in\mathcal{F}_s,\ 1\le\ell\le s,\ 0\le k\le p). \tag*{(23)} F s = ∅ , ∇ k F ( x ℓ ) = a ℓ , k ( F ∈ F s , 1 ≤ ℓ ≤ s , 0 ≤ k ≤ p ) . (23) (b) One fixed objective and candidate gaps. Every F ∈ F n m L F\in\mathcal{F}_{n_mL} F ∈ F n m L reproduces the recorded evaluation points, oracle outputs, and candidate solutions when the algorithm runs on it through n m L n_mL n m L oracle calls, and satisfies, for j n m ≤ s ≤ ( j + 1 ) n m jn_m\le s\le(j+1)n_m j n m ≤ s ≤ ( j + 1 ) n m and 0 ≤ j < L 0\le j<L 0 ≤ j < L ,
F ( x ^ s ) − min F ≥ 7 δ j , F ( x ^ 0 ) − min F ≥ 7 δ 0 . (24) F(\hat{x}_s)-\min F\ge7\delta_j,\qquad F(\hat{x}_0)-\min F\ge7\delta_0. \tag*{(24)} F ( x ^ s ) − min F ≥ 7 δ j , F ( x ^ 0 ) − min F ≥ 7 δ 0 . (24)
Proof sketch. (a) Step 5.1: Available directions. The selections in Step 4 leave candidate solutions in short prefixes, with unused directions for completion. Step 5.2: Identical oracle outputs. Since dist ( c j + 1 , W j , b ) > θ R j \operatorname{dist}(c_{j+1},W_{j,b})>\theta R_j dist ( c j + 1 , W j , b ) > θ R j , prefix points miss all later replacement balls, so the base’s values and derivatives depend only on fixed centers. The current phase uses only the prefix and the additional direction fixed in Step 4, while future and completed phases give fixed values and derivative tensors. Thus every completion gives the same full oracle output, and further selections preserve earlier outputs by restricting the completion family. (b) Step 5.3: One fixed objective. Determinism reproduces the trajectory on any final completion, and Lemma 4.5(c) gives the stated gaps. The full construction and proof are given in Proposition F.5 in Appx. F.
Step 6: Choose the Parameters and Prove the Main Theorem
We now formally prove Theorem 1.1 by combining Lemma 4.5 with Proposition 4.6. We choose the chain length to meet H , σ H,\sigma H , σ , the phase count to keep the candidate gaps above ϵ \epsilon ϵ , and rescale to the prescribed initial gap Δ 0 \Delta_0 Δ 0 .
Proof. High-Order Hölder Smoothness and Uniform Convexity. With τ 0 = 1 \tau_0=1 τ 0 = 1 and κ = H σ \kappa=\frac{H}{\sigma} κ = σ H , Lemma 4.5 (a) gives the prescribed class parameters provided q c B μ ≥ σ q c_B \mu\geq\sigma q c B μ ≥ σ and C H μ a q ≤ H C_H \mu^{a_q} \leq H C H μ a q ≤ H . Taking μ = σ q c B \mu=\frac{\sigma}{q c_B} μ = q c B σ leaves μ a q ≤ q c B κ C H \mu^{a_q}\leq\frac{q c_B \kappa}{C_H} μ a q ≤ C H q c B κ as the smoothness requirement. We therefore choose the admissible integer m ≍ p , ν κ 2 3 q − 2 m\asymp_{p,\nu}\kappa^{\frac{2}{3q-2}} m ≍ p , ν κ 3 q − 2 2 given by Lemma G.1 (a) in Appx. G.1. For sufficiently large κ \kappa κ , this choice also gives m ≥ 32 m\geq32 m ≥ 32 and n m ≍ m n_m\asymp m n m ≍ m , fixing the condition-number cost of each phase.
Phase Count. Because these class bounds are independent of L L L , we can repeat this cost at successively smaller gaps. Comparing δ j = c 1 λ m η q j \delta_j=c_1\lambda m\eta^{qj} δ j = c 1 λm η q j with the initial gap in Lemma 4.5 (d) gives, for a constant c ∗ ∈ ( 0 , 1 ) c_*\in(0,1) c ∗ ∈ ( 0 , 1 ) depending only on p , ν p,\nu p , ν ,
7 δ j D m , L ≥ c ∗ η q j . (25) \frac{7\delta_j}{D_{m,L}}\geq c_*\eta^{qj}. \tag*{(25)} D m , L 7 δ j ≥ c ∗ η q j . (25)
Normalizing the initial gap to Δ 0 \Delta_0 Δ 0 therefore gives a phase-j j j candidate gap of at least c ∗ Δ 0 η q j c_*\Delta_0\eta^{qj} c ∗ Δ 0 η q j . We choose the largest integer L ≥ 1 L\geq1 L ≥ 1 such that c ∗ Δ 0 η q ( L − 1 ) ≥ 2 ϵ c_*\Delta_0\eta^{q(L-1)}\geq2\epsilon c ∗ Δ 0 η q ( L − 1 ) ≥ 2 ϵ . Since 0 < η < 1 0<\eta<1 0 < η < 1 , this keeps every earlier phase gap above 2 ϵ 2\epsilon 2 ϵ as well. Lemma G.1 (b)–(c) verifies L ≍ p , ν log ( Δ 0 ϵ ) L\asymp_{p,\nu}\log\left(\frac{\Delta_0}{\epsilon}\right) L ≍ p , ν log ( ϵ Δ 0 ) and L ( m + 1 ) ≤ d L(m+1)\leq d L ( m + 1 ) ≤ d under the theorem’s dimension bound.
Initial Gap. To apply this comparison to the prescribed initial gap, we take a = ( Δ 0 D m , L ) 1 q a=\left(\frac{\Delta_0}{D_{m,L}}\right)^{\frac{1}{q}} a = ( D m , L Δ 0 ) q 1 and define F ^ ( x ) = a q F ( x − x 0 a ) \widehat{F}(x)=a^qF\left(\frac{x-x_0}{a}\right) F ( x ) = a q F ( a x − x 0 ) . Lemma 4.5 (d) makes D m , L D_{m,L} D m , L independent of the chain directions, so this normalization is fixed before the oracle trajectory. Since q = p + ν q=p+\nu q = p + ν , Lemma G.2 preserves both class parameters and gives F ^ ( x 0 ) − min F ^ = Δ 0 \widehat{F}(x_0)-\min\widehat{F}=\Delta_0 F ( x 0 ) − min F = Δ 0 .
Oracle Lower Bound. Applying Proposition 4.6 to the algorithm in coordinates x − x 0 a \frac{x-x_0}{a} a x − x 0 , with the derivative transformation in Lemma G.2 , gives one fixed F ^ \widehat{F} F reproducing its oracle trajectory. For a candidate solution x ^ s \widehat{x}_s x s in group j j j , Proposition 4.6 (b), Eq. (25) , and our choice of L L L give
F ^ ( x ^ s ) − min F ^ ≥ 7 a q δ j ≥ c ∗ Δ 0 η q j ≥ 2 ϵ > ϵ . \widehat{F}(\widehat{x}_s)-\min\widehat{F}\geq7a^q\delta_j\geq c_*\Delta_0\eta^{qj}\geq2\epsilon>\epsilon. F ( x s ) − min F ≥ 7 a q δ j ≥ c ∗ Δ 0 η q j ≥ 2 ϵ > ϵ .
The zero-call candidate solution satisfies the same bound. Thus reaching the target gap after T T T oracle calls requires
T > n m L = Ω p , ν ( ( H σ ) 2 3 q − 2 log ( Δ 0 ϵ ) ) . T>n_mL=\Omega_{p,\nu}\left(\left(\frac{H}{\sigma}\right)^{\frac{2}{3q-2}}\log\left(\frac{\Delta_0}{\epsilon}\right)\right). T > n m L = Ω p , ν ( ( σ H ) 3 q − 2 2 log ( ϵ Δ 0 ) ) .
Choosing c p , ν > 0 c_{p,\nu}>0 c p , ν > 0 sufficiently small therefore proves both the theorem’s accuracy range and lower bound.
Conclusion and Future Work
We establish a tight deterministic high-order oracle lower bound of Ω p , ν ( ( H σ ) 2 3 q − 2 log ( Δ 0 ϵ ) ) \Omega_{p,\nu}\left(\left(\frac{H}{\sigma}\right)^{\frac{2}{3q-2}}\log\left(\frac{\Delta_0}{\epsilon}\right)\right) Ω p , ν ( ( σ H ) 3 q − 2 2 log ( ϵ Δ 0 ) ) calls in the matched case q = p + ν q=p+\nu q = p + ν . The construction combines Euclidean curvature at successive centers with short hard chains, while preserving earlier oracle outputs and keeping the global class parameters independent of the number of phases. Together with the asymmetric results, it resolves the remaining equality case, with the stated function-class distinctions. Randomized algorithms and matched lower bounds in other norm geometries remain separate questions.
AI Statement
We used OpenAI Codex 6 Astra and 6.1 Sol to assist with literature searches, identifying suitable constants, verifying proof arguments, and refining the presentation of proofs. This assistance materially influenced the main text, appendices, and figures. The author independently verified all mathematical arguments and takes full responsibility for the content of the paper.
Norm Mismatch in a Long Residual Chain
To quantify the norm mismatch, we consider a single long residual chain. Let q = p + ν > 2 q=p+\nu>2 q = p + ν > 2 and let S ( x 1 , … , x n ) = ( 0 , x 1 , … , x n − 1 ) S(x_1,\ldots,x_n)=(0,x_1,\ldots,x_{n-1}) S ( x 1 , … , x n ) = ( 0 , x 1 , … , x n − 1 ) be the lower shift. For α , a > 0 \alpha,a>0 α , a > 0 and 0 < ξ < 1 0<\xi<1 0 < ξ < 1 , define
A ξ = I − ξ S , G ( x ) = α q ∥ A ξ x − a e 1 ∥ q q . (26) A_\xi=I-\xi S,\qquad G(x)=\frac{\alpha}{q}\lVert A_\xi x-ae_1\rVert_q^q. \tag*{(26)} A ξ = I − ξ S , G ( x ) = q α ∥ A ξ x − a e 1 ∥ q q . (26)
Here ∥ z ∥ q q = ∑ i = 1 n ∣ z i ∣ q \lVert z\rVert_q^q=\sum_{i=1}^n|z_i|^q ∥ z ∥ q q = ∑ i = 1 n ∣ z i ∣ q and e 1 e_1 e 1 is the first standard basis vector. The minimizer of G G G is x ⋆ = a ( 1 , ξ , … , ξ n − 1 ) x^\star=a(1,\xi,\ldots,\xi^{n-1}) x ⋆ = a ( 1 , ξ , … , ξ n − 1 ) , since A ξ x ⋆ = a e 1 A_\xi x^\star=ae_1 A ξ x ⋆ = a e 1 and A ξ A_\xi A ξ is invertible. In particular, G ( x ⋆ ) = 0 G(x^\star)=0 G ( x ⋆ ) = 0 and ∇ G ( x ⋆ ) = 0 \nabla G(x^\star)=0 ∇ G ( x ⋆ ) = 0 by Lemma B.16 (b) in Appx. B.
For ϕ ( t ) = ∣ t ∣ q q \phi(t)=\frac{|t|^q}{q} ϕ ( t ) = q ∣ t ∣ q , the scalar divergence bound in Lemma B.16 (e) is expressed in the residual ℓ q \ell_q ℓ q norm. Converting it to a Euclidean bound uses
∥ z ∥ 2 2 = ∑ i = 1 n ∣ z i ∣ 2 ⋅ 1 ≤ ( ∑ i = 1 n ∣ z i ∣ q ) 2 q n q − 2 q \lVert z\rVert_2^2=\sum_{i=1}^n |z_i|^2\cdot1
\le\left(\sum_{i=1}^n |z_i|^q\right)^{\frac{2}{q}}n^{\frac{q-2}{q}} ∥ z ∥ 2 2 = i = 1 ∑ n ∣ z i ∣ 2 ⋅ 1 ≤ ( i = 1 ∑ n ∣ z i ∣ q ) q 2 n q q − 2
by Hölder’s inequality,
∥ z ∥ q q ≥ n 1 − q 2 ∥ z ∥ 2 q \lVert z\rVert_q^q\ge n^{1-\frac{q}{2}}\lVert z\rVert_2^q ∥ z ∥ q q ≥ n 1 − 2 q ∥ z ∥ 2 q
by raising to the power q 2 \frac{q}{2} 2 q .
The shift satisfies ∥ S z ∥ 2 ≤ ∥ z ∥ 2 \lVert Sz\rVert_2\le\lVert z\rVert_2 ∥ S z ∥ 2 ≤ ∥ z ∥ 2 , so the reverse triangle inequality gives ∥ A ξ z ∥ 2 ≥ ∥ z ∥ 2 − ξ ∥ S z ∥ 2 ≥ ( 1 − ξ ) ∥ z ∥ 2 \lVert A_\xi z\rVert_2\ge\lVert z\rVert_2-\xi\lVert Sz\rVert_2\ge(1-\xi)\lVert z\rVert_2 ∥ A ξ z ∥ 2 ≥ ∥ z ∥ 2 − ξ ∥ S z ∥ 2 ≥ ( 1 − ξ ) ∥ z ∥ 2 . For x , y ∈ R n x,y\in\mathbb{R}^n x , y ∈ R n , define their residual vectors u = A ξ x − a e 1 u=A_\xi x-ae_1 u = A ξ x − a e 1 and v = A ξ y − a e 1 v=A_\xi y-ae_1 v = A ξ y − a e 1 . Applying the two norm bounds to v − u = A ξ ( y − x ) v-u=A_\xi(y-x) v − u = A ξ ( y − x ) gives
D G ( y , x ) = α ∑ i = 1 n D ϕ ( v i , u i ) ≥ 2 2 − q α q ∥ A ξ ( y − x ) ∥ q q ≥ 2 2 − q α q n 1 − q 2 ∥ A ξ ( y − x ) ∥ 2 q ≥ 2 2 − q α q n 1 − q 2 ( 1 − ξ ) q ∥ y − x ∥ 2 q . D_G(y,x)=\alpha\sum_{i=1}^n D_\phi(v_i,u_i)
\ge\frac{2^{2-q}\alpha}{q}\lVert A_\xi(y-x)\rVert_q^q
\ge\frac{2^{2-q}\alpha}{q}n^{1-\frac{q}{2}}\lVert A_\xi(y-x)\rVert_2^q
\ge\frac{2^{2-q}\alpha}{q}n^{1-\frac{q}{2}}(1-\xi)^q\lVert y-x\rVert_2^q. D G ( y , x ) = α i = 1 ∑ n D ϕ ( v i , u i ) ≥ q 2 2 − q α ∥ A ξ ( y − x ) ∥ q q ≥ q 2 2 − q α n 1 − 2 q ∥ A ξ ( y − x ) ∥ 2 q ≥ q 2 2 − q α n 1 − 2 q ( 1 − ξ ) q ∥ y − x ∥ 2 q .
by the chain rule and Bregman definition, by Lemma B.16 (e), by the preceding norm comparison, by the lower bound for A ξ A_\xi A ξ .
Thus G G G is uniformly convex with modulus
σ c e r t = 2 2 − q α n 1 − q 2 ( 1 − ξ ) q . (27) \sigma_{\mathrm{cert}}=2^{2-q}\alpha n^{1-\frac{q}{2}}(1-\xi)^q. \tag*{(27)} σ cert = 2 2 − q α n 1 − 2 q ( 1 − ξ ) q . (27)
The following calculation shows that the factor n 1 − q 2 n^{1-\frac{q}{2}} n 1 − 2 q is also present in the best Euclidean modulus of this family.
Lemma A.1 (A direction witnessing the Euclidean modulus loss). If n ≥ 2 ⌈ ( 1 − ξ ) − 1 ⌉ n\ge2\left\lceil(1-\xi)^{-1}\right\rceil n ≥ 2 ⌈ ( 1 − ξ ) − 1 ⌉ , every valid degree-q q q Euclidean uniform-convexity modulus σ G \sigma_G σ G of G G G in Eq. (A.1) satisfies
σ G ≤ 2 q 2 ( 1 − e − 1 ) q α n 1 − q 2 ( 1 − ξ ) q . (28) \sigma_G\le\frac{2^{\frac{q}{2}}}{(1-e^{-1})^q}\alpha n^{1-\frac{q}{2}}(1-\xi)^q. \tag*{(28)} σ G ≤ ( 1 − e − 1 ) q 2 2 q α n 1 − 2 q ( 1 − ξ ) q . (28)
Proof. To bound the modulus from above, we denote δ = 1 − ξ \delta= 1 - \xi δ = 1 − ξ and choose w = ( w 1 , … , w n ) \mathbf{w} = (w_1,\ldots,w_n) w = ( w 1 , … , w n ) with w i = 1 − ξ i w_i = 1 - \xi^i w i = 1 − ξ i . This choice makes all residual coordinates equal to δ \delta δ , since
( A ξ w ) 1 = 1 − ξ = δ , (A_{\xi}\mathbf{w})_1 = 1-\xi= \delta, ( A ξ w ) 1 = 1 − ξ = δ ,
( A ξ w ) i = ( 1 − ξ i ) − ξ ( 1 − ξ i − 1 ) = 1 − ξ i − ξ + ξ i = δ ( 2 ≤ i ≤ n ) . \begin{aligned}
(A_{\xi}\mathbf{w})_i &= (1-\xi^i)-\xi(1-\xi^{i-1}) \\
&= 1-\xi^i-\xi+\xi^i = \delta\qquad(2\le i\le n).
\end{aligned} ( A ξ w ) i = ( 1 − ξ i ) − ξ ( 1 − ξ i − 1 ) = 1 − ξ i − ξ + ξ i = δ ( 2 ≤ i ≤ n ) .
For i ≥ ⌈ n 2 ⌉ i\ge\left\lceil\frac{n}{2}\right\rceil i ≥ ⌈ 2 n ⌉ , the length assumption gives i δ ≥ n δ 2 ≥ 1 i\delta\ge\frac{n\delta}{2}\ge1 i δ ≥ 2 n δ ≥ 1 . Consequently,
w i = 1 − ( 1 − δ ) i ≥ 1 − e − i δ since 1 − δ ≤ e − δ , ≥ 1 − e − 1 since i δ ≥ 1 , \begin{aligned}
w_i &= 1-(1-\delta)^i \\
&\ge1-e^{-i\delta} \qquad\text{since }1-\delta\le e^{-\delta},\\
&\ge1-e^{-1} \qquad\text{since }i\delta\ge1,
\end{aligned} w i = 1 − ( 1 − δ ) i ≥ 1 − e − i δ since 1 − δ ≤ e − δ , ≥ 1 − e − 1 since i δ ≥ 1 ,
∥ w ∥ 2 2 = ∑ i = 1 n w i 2 ≥ n 2 ( 1 − e − 1 ) 2 since at least n 2 coordinates satisfy this bound. \|\mathbf{w}\|_2^2=\sum_{i=1}^{n}w_i^2
\ge\frac{n}{2}(1-e^{-1})^2
\qquad\text{since at least }\frac{n}{2}\text{ coordinates satisfy this bound.} ∥ w ∥ 2 2 = i = 1 ∑ n w i 2 ≥ 2 n ( 1 − e − 1 ) 2 since at least 2 n coordinates satisfy this bound.
Testing uniform convexity at ( x , y ) = ( x ⋆ , x ⋆ + w ) (\mathbf{x},\mathbf{y})=(\mathbf{x}^{\star},\mathbf{x}^{\star}+\mathbf{w}) ( x , y ) = ( x ⋆ , x ⋆ + w ) gives
σ G q ∥ w ∥ q q ≤ D G ( x ⋆ + w , x ⋆ ) by uniform convexity, = G ( x ⋆ + w ) since G ( x ⋆ ) = 0 , ∇ G ( x ⋆ ) = 0 , = α q ∥ A ξ w ∥ q q since A ξ x ⋆ = a e 1 , = α q n δ q since A ξ w = δ 1 , \begin{aligned}
\frac{\sigma_G}{q}\|\mathbf{w}\|_q^q
&\le D_G(\mathbf{x}^{\star}+\mathbf{w},\mathbf{x}^{\star})
&&\text{by uniform convexity,}\\
&=G(\mathbf{x}^{\star}+\mathbf{w})
&&\text{since }G(\mathbf{x}^{\star})=0,\ \nabla G(\mathbf{x}^{\star})=0,\\
&=\frac{\alpha}{q}\|A_{\xi}\mathbf{w}\|_q^q
&&\text{since }A_{\xi}\mathbf{x}^{\star}=a\mathbf{e}_1,\\
&=\frac{\alpha}{q}n\delta^q
&&\text{since }A_{\xi}\mathbf{w}=\delta\mathbf{1},
\end{aligned} q σ G ∥ w ∥ q q ≤ D G ( x ⋆ + w , x ⋆ ) = G ( x ⋆ + w ) = q α ∥ A ξ w ∥ q q = q α n δ q by uniform convexity, since G ( x ⋆ ) = 0 , ∇ G ( x ⋆ ) = 0 , since A ξ x ⋆ = a e 1 , since A ξ w = δ 1 ,
σ G ≤ α n δ q ∥ w ∥ 2 q ≤ α n δ q ( n 2 ) q / 2 ( 1 − e − 1 ) q = 2 q / 2 ( 1 − e − 1 ) q α n 1 − q / 2 ( 1 − ξ ) q . \sigma_G\le\frac{\alpha n\delta^q}{\|\mathbf{w}\|_2^q}
\le\frac{\alpha n\delta^q}{\left(\frac{n}{2}\right)^{q/2}(1-e^{-1})^q}
=\frac{2^{q/2}}{(1-e^{-1})^q}\alpha n^{1-q/2}(1-\xi)^q. σ G ≤ ∥ w ∥ 2 q α n δ q ≤ ( 2 n ) q /2 ( 1 − e − 1 ) q α n δ q = ( 1 − e − 1 ) q 2 q /2 α n 1 − q /2 ( 1 − ξ ) q .
The last line is Eq. (A.3).
This dimension dependence cannot be canceled by a dimension-dependent improvement in the smoothness estimate. Since A ξ e n = e n A_{\xi}\mathbf{e}_n=\mathbf{e}_n A ξ e n = e n , G ( x ⋆ + t e n ) = α ∣ t ∣ q q G(\mathbf{x}^{\star}+t\mathbf{e}_n)=\frac{\alpha|t|^q}{q} G ( x ⋆ + t e n ) = q α ∣ t ∣ q . For S q , p : = ∏ ℓ = 1 p − 1 ( q − ℓ ) S_{q,p}:=\prod_{\ell=1}^{p-1}(q-\ell) S q , p := ∏ ℓ = 1 p − 1 ( q − ℓ ) and t > 0 t>0 t > 0 , Lemma B.16 (a) therefore gives
∥ ∇ p G ( x ⋆ + t e n ) − ∇ p G ( x ⋆ ) ∥ o p ≥ ∣ ( ∇ p G ( x ⋆ + t e n ) − ∇ p G ( x ⋆ ) ) [ e n , … , e n ] ∣ = α S q , p t ν . \begin{aligned}
\|\nabla^pG(\mathbf{x}^{\star}+t\mathbf{e}_n)-\nabla^pG(\mathbf{x}^{\star})\|_{\mathrm{op}}
\ge\left|(\nabla^pG(\mathbf{x}^{\star}+t\mathbf{e}_n)-\nabla^pG(\mathbf{x}^{\star}))[\mathbf{e}_n,\ldots,\mathbf{e}_n]\right| \\
&=\alpha S_{q,p}t^\nu.
\end{aligned} ∥ ∇ p G ( x ⋆ + t e n ) − ∇ p G ( x ⋆ ) ∥ op ≥ ∣ ( ∇ p G ( x ⋆ + t e n ) − ∇ p G ( x ⋆ )) [ e n , … , e n ] ∣ = α S q , p t ν .
Every valid Hölder parameter H G H_G H G consequently satisfies H G ≥ α S q , p H_G\ge\alpha S_{q,p} H G ≥ α S q , p . Together with Eq. (A.3), this gives
H G σ G ≥ S q , p ( 1 − e − 1 ) q 2 q / 2 n q / 2 − 2 ( 1 − ξ ) − q . \frac{H_G}{\sigma_G}\ge\frac{S_{q,p}(1-e^{-1})^q}{2^{q/2}}n^{q/2-2}(1-\xi)^{-q}. σ G H G ≥ 2 q /2 S q , p ( 1 − e − 1 ) q n q /2 − 2 ( 1 − ξ ) − q .
The obstruction concerns this residual family; it does not assert that every lower-bound construction must use multiple phases.
Definitions and Elementary Tools
Additional notation and oracle model
We use the notation and Bregman divergence of § 3 , and the class assumptions of Theorem 1.1 . For derivative evaluations, this gives
∇ f ( x ) [ z ] = ⟨ ∇ f ( x ) , z ⟩ , ∇ 2 f ( x ) [ u , v ] = u T ∇ 2 f ( x ) v . \nabla f(x)[z] = \langle\nabla f(x),z\rangle,\qquad\nabla^2 f(x)[u,v] = u^{\mathsf T}\nabla^2 f(x)v. ∇ f ( x ) [ z ] = ⟨ ∇ f ( x ) , z ⟩ , ∇ 2 f ( x ) [ u , v ] = u T ∇ 2 f ( x ) v .
For symmetric matrices, we use A ⪰ B A \succeq B A ⪰ B to mean that z T ( A − B ) z ≥ 0 z^{\mathsf T}(A-B)z \ge0 z T ( A − B ) z ≥ 0 for every z z z . For a normed-space-valued map g g g on U U U , we denote ∥ g ∥ ∞ ; U : = sup x ∈ U ∥ g ( x ) ∥ \lVert g\rVert_{\infty;U} := \sup_{x\in U}\lVert g(x)\rVert ∥ g ∥ ∞ ; U := sup x ∈ U ∥ g ( x )∥ , omitting U U U when the domain is fixed.
If E E E has orthonormal basis e 1 , … , e k e_1,\ldots,e_k e 1 , … , e k , its projection is
P E x = ∑ i = 1 k ⟨ e i , x ⟩ e i . P_E x = \sum_{i=1}^{k}\langle e_i,x\rangle e_i. P E x = i = 1 ∑ k ⟨ e i , x ⟩ e i .
The open and closed balls are
B ( c , R ) = { x : ∥ x − c ∥ < R } , B ‾ ( c , R ) = { x : ∥ x − c ∥ ≤ R } . B(c,R)=\{x:\lVert x-c\rVert<R\},\qquad\overline{B}(c,R)=\{x:\lVert x-c\rVert\le R\}. B ( c , R ) = { x : ∥ x − c ∥ < R } , B ( c , R ) = { x : ∥ x − c ∥ ≤ R } .
For any x , y ∈ R d x,y\in\mathbb{R}^d x , y ∈ R d , the closed line segment joining them is
[ x , y ] : = { ( 1 − t ) x + t y : 0 ≤ t ≤ 1 } . [x,y] := \{(1-t)x+ty:0\le t\le1\}. [ x , y ] := {( 1 − t ) x + t y : 0 ≤ t ≤ 1 } .
Its length is ∥ y − x ∥ \lVert y-x\rVert ∥ y − x ∥ . The factor 1 q \frac{1}{q} q 1 in Theorem 1.1 ’s uniform-convexity convention means that a bound D f ( y , x ) ≥ c μ ∥ y − x ∥ q D_f(y,x)\ge c\mu\lVert y-x\rVert^q D f ( y , x ) ≥ c μ ∥ y − x ∥ q gives modulus q c μ qc\mu q c μ .
Definition B.1 (Deterministic algorithm with p th p^{\text{th}} p th -order oracle access). An oracle call at x x x returns the oracle output ( F ( x ) , ∇ F ( x ) , … , ∇ p F ( x ) ) (F(x),\nabla F(x),\ldots,\nabla^pF(x)) ( F ( x ) , ∇ F ( x ) , … , ∇ p F ( x )) . Each evaluation point is a deterministic function of the prescribed parameters, initialization, and preceding evaluation points and oracle outputs. After s s s calls, a candidate solution x ^ s \widehat{x}_s x s is any deterministic function of this information; it need not be an evaluation point. A candidate solution x ^ 0 \widehat{x}_0 x 0 before the first oracle call is also allowed.
Evaluation points and candidate solutions can have arbitrary norms. In particular, no restriction to the span of previous gradients is assumed.
Elementary functions used in the construction
Throughout, p ≥ 2 p\ge2 p ≥ 2 , 0 < ν ≤ 1 0<\nu\le1 0 < ν ≤ 1 , and q = p + ν > 2 q=p+\nu>2 q = p + ν > 2 ; vector norms are Euclidean and derivative norms are operator norms.
Scalar and Euclidean functions
We define
ϕ ( t ) = ∣ t ∣ q q , f ( x ) = ∥ x ∥ q q . \phi(t)=\frac{|t|^q}{q},\qquad f(x)=\frac{\lVert x\rVert^q}{q}. ϕ ( t ) = q ∣ t ∣ q , f ( x ) = q ∥ x ∥ q .
The factor 1 q \frac{1}{q} q 1 simplifies the first derivative. Differentiation p p p times reduces the degree to q − p = ν q-p=\nu q − p = ν . The norm-power theorems cited in Lemma B.16 justify this scaling globally, including at zero.
For scalar arguments, D ϕ ( a , b ) D_\phi(a,b) D ϕ ( a , b ) denotes the divergence defined in § 3 with d = 1 d=1 d = 1 ; the affine approximation is taken at the second argument b b b .
The fixed smooth-maximum profile
We use the smooth maximum in Definition 4.1 , whose scalar convexity and equality properties are given in Lemma 4.2 . With its fixed density φ \varphi φ , we define Θ ( v ) : = ∫ − 1 1 ∣ v − u ∣ φ ( u ) d u \Theta(v) := \int_{-1}^{1} |v-u|\varphi(u)\,\mathrm{d}u Θ ( v ) := ∫ − 1 1 ∣ v − u ∣ φ ( u ) d u . This function is smooth, convex, even, and 1-Lipschitz, with Θ ( v ) = ∣ v ∣ \Theta(v)=|v| Θ ( v ) = ∣ v ∣ if and only if ∣ v ∣ ≥ 1 |v|\ge1 ∣ v ∣ ≥ 1 , as in [Azal13, Lemma 1 and its proof]. It is fixed independently of the dimension and all phase parameters. For δ > 0 \delta>0 δ > 0 , the definitions in Eq. (2) equivalently give
ψ δ ( v ) : = v + δ Θ ( v δ ) 2 , M δ ( s , t ) : = s + ψ δ ( t − s ) . (29) \psi_\delta(v):=\frac{v+\delta\Theta\left(\frac{v}{\delta}\right)}{2},\qquad M_\delta(s,t):=s+\psi_\delta(t-s). \tag*{(29)} ψ δ ( v ) := 2 v + δ Θ ( δ v ) , M δ ( s , t ) := s + ψ δ ( t − s ) . (29)
Thus ψ δ ( v ) = δ ψ 1 ( v δ ) \psi_\delta(v)=\delta\psi_1\left(\frac{v}{\delta}\right) ψ δ ( v ) = δ ψ 1 ( δ v ) , and evenness gives
M δ ( s , t ) = s + t + δ Θ ( s − t δ ) 2 . M_\delta(s,t)=\frac{s+t+\delta\Theta\left(\frac{s-t}{\delta}\right)}{2}. M δ ( s , t ) = 2 s + t + δ Θ ( δ s − t ) .
Lemma B.17 records the positive-part derivative and translation properties used below; Lemma B.21 gives the Hessian and joining rules for its composition with convex functions. All smoothing constants refer to this one fixed choice of Θ \Theta Θ .
Theorem B.2 (Taylor’s formula [14 Theorem 5.1] ). Let V , W V,W V , W be finite-dimensional real normed spaces, let U ⊂ V U\subset V U ⊂ V be open, and let f : U → W f:U\to W f : U → W be C k C^k C k , where k ≥ 1 k\ge1 k ≥ 1 is an integer. Fix x ∈ U x\in U x ∈ U and r > 0 r>0 r > 0 such that B ( x , r ) ⊂ U B(x,r)\subset U B ( x , r ) ⊂ U . For ∥ h ∥ < r \lVert h\rVert<r ∥ h ∥ < r ,
f ( x + h ) = ∑ j = 0 k 1 j ! ∇ j f ( x ) [ h , … , h ] + E k ( x , h ) , f(x+h)=\sum_{j=0}^{k}\frac{1}{j!}\nabla^j f(x)[h,\ldots,h]+\mathcal{E}_k(x,h), f ( x + h ) = j = 0 ∑ k j ! 1 ∇ j f ( x ) [ h , … , h ] + E k ( x , h ) , E k ( x , h ) = 1 ( k − 1 ) ! ∫ 0 1 ( 1 − t ) k − 1 ( ∇ k f ( x + t h ) − ∇ k f ( x ) ) [ h , … , h ] d t . \mathcal{E}_k(x,h)=\frac{1}{(k-1)!}\int_0^1(1-t)^{k-1}\left(\nabla^k f(x+th)-\nabla^k f(x)\right)[h,\ldots,h]\,\mathrm{d}t. E k ( x , h ) = ( k − 1 )! 1 ∫ 0 1 ( 1 − t ) k − 1 ( ∇ k f ( x + t h ) − ∇ k f ( x ) ) [ h , … , h ] d t . The j = 0 j=0 j = 0 summand means f ( x ) f(x) f ( x ) . With
C k , h , x : = 1 k ! sup 0 ≤ t ≤ 1 ∥ ∇ k f ( x + t h ) − ∇ k f ( x ) ∥ , C_{k,h,x}:=\frac{1}{k!}\sup_{0\le t\le1}\left\lVert\nabla^k f(x+th)-\nabla^k f(x)\right\rVert, C k , h , x := k ! 1 0 ≤ t ≤ 1 sup ∇ k f ( x + t h ) − ∇ k f ( x ) , one has ∥ E k ( x , h ) ∥ ≤ C k , h , x ∥ h ∥ k \lVert\mathcal{E}_k(x,h)\rVert\le C_{k,h,x}\lVert h\rVert^k ∥ E k ( x , h )∥ ≤ C k , h , x ∥ h ∥ k and C k , h , x → 0 C_{k,h,x}\to0 C k , h , x → 0 as h → 0 h\to0 h → 0 . This convergence is uniform when x x x ranges over a compact subset of U U U . Equivalently,
f ( x + h ) = ∑ j = 0 k − 1 1 j ! ∇ j f ( x ) [ h , … , h ] + 1 ( k − 1 ) ! ∫ 0 1 ( 1 − t ) k − 1 ∇ k f ( x + t h ) [ h , … , h ] d t . (30) f(x+h)=\sum_{j=0}^{k-1}\frac{1}{j!}\nabla^j f(x)[h,\ldots,h]+\frac{1}{(k-1)!}\int_0^1(1-t)^{k-1}\nabla^k f(x+th)[h,\ldots,h]\,\mathrm{d}t. \tag*{(30)} f ( x + h ) = j = 0 ∑ k − 1 j ! 1 ∇ j f ( x ) [ h , … , h ] + ( k − 1 )! 1 ∫ 0 1 ( 1 − t ) k − 1 ∇ k f ( x + t h ) [ h , … , h ] d t . (30) For f ∈ C 2 f\in C^2 f ∈ C 2 and d = y − x d=y-x d = y − x , Theorem B.2 gives
D f ( y , x ) = ∫ 0 1 ( 1 − t ) ∇ 2 f ( x + t d ) [ d , d ] d t . (31) D_f(y,x)=\int_0^1(1-t)\nabla^2 f(x+td)[d,d]\,\mathrm{d}t. \tag*{(31)} D f ( y , x ) = ∫ 0 1 ( 1 − t ) ∇ 2 f ( x + t d ) [ d , d ] d t . (31)
Mean-value estimates
Theorem B.3 (Mean-value inequality [11 Theorem 3.1.3, p. 39] ). Let a < b a < b a < b be real. If v : [ a , b ] → X v : [a,b] \to X v : [ a , b ] → X and g : [ a , b ] → R g : [a,b] \to\mathbb{R} g : [ a , b ] → R are continuous, X X X is a Banach space, and outside a countable exceptional set their right derivatives exist on [ a , b ) [a,b) [ a , b ) and satisfy ∥ v + ′ ( t ) ∥ ≤ g + ′ ( t ) \lVert v_{+}'(t)\rVert\le g_{+}'(t) ∥ v + ′ ( t )∥ ≤ g + ′ ( t ) , then
∥ v ( b ) − v ( a ) ∥ ≤ g ( b ) − g ( a ) . \lVert v(b)-v(a)\rVert\le g(b)-g(a). ∥ v ( b ) − v ( a )∥ ≤ g ( b ) − g ( a ) .
Lemma B.4 (Mean-value bound on a convex domain). Let U ⊂ R d U \subset\mathbb{R}^{d} U ⊂ R d be open and convex, and let f : U → R f : U \to\mathbb{R} f : U → R be differentiable. If ∥ ∇ f ( z ) ∥ ≤ M \lVert\nabla f(z)\rVert\le M ∥ ∇ f ( z )∥ ≤ M for every z ∈ U z \in U z ∈ U , where M ≥ 0 M \ge0 M ≥ 0 , then
∥ f ( y ) − f ( x ) ∥ ≤ M ∥ y − x ∥ ( x , y ∈ U ) . \lVert f(y)-f(x)\rVert\le M\lVert y-x\rVert\qquad(x,y \in U). ∥ f ( y ) − f ( x )∥ ≤ M ∥ y − x ∥ ( x , y ∈ U ) . In particular, ∇ f = 0 \nabla f = 0 ∇ f = 0 on U U U implies that f f f is constant on U U U .
Proof. Fix x , y ∈ U x,y \in U x , y ∈ U and define, for t ∈ [ 0 , 1 ] t \in[0,1] t ∈ [ 0 , 1 ] , v ( t ) = f ( x + t ( y − x ) ) v(t)=f(x+t(y-x)) v ( t ) = f ( x + t ( y − x )) and g ( t ) = M t ∥ y − x ∥ g(t)=Mt\lVert y-x\rVert g ( t ) = M t ∥ y − x ∥ . Convexity of U U U places this segment in the domain, and differentiability gives
∥ v + ′ ( t ) ∥ = ∥ ∇ f ( x + t ( y − x ) ) [ y − x ] ∥ ≤ M ∥ y − x ∥ \lVert v_{+}'(t)\rVert= \lVert\nabla f(x+t(y-x))[y-x]\rVert
\le M\lVert y-x\rVert ∥ v + ′ ( t )∥ = ∥ ∇ f ( x + t ( y − x )) [ y − x ]∥ ≤ M ∥ y − x ∥
by the derivative bound,
= g + ′ ( t ) . = g_{+}'(t). = g + ′ ( t ) .
The continuous maps v , g v,g v , g therefore satisfy the hypotheses of Theorem B.3. It gives
∥ f ( y ) − f ( x ) ∥ = ∥ v ( 1 ) − v ( 0 ) ∥ ≤ g ( 1 ) − g ( 0 ) \lVert f(y)-f(x)\rVert= \lVert v(1)-v(0)\rVert
\le g(1)-g(0) ∥ f ( y ) − f ( x )∥ = ∥ v ( 1 ) − v ( 0 )∥ ≤ g ( 1 ) − g ( 0 )
by Theorem B.3,
= M ∥ y − x ∥ . = M\lVert y-x\rVert. = M ∥ y − x ∥ .
Taking M = 0 M=0 M = 0 proves the final assertion.
A quantitative Hölder composition theorem
Definition B.5 (Lipschitz jets [6 Definition 1.1] ). Let p ≥ 0 p \ge0 p ≥ 0 be an integer, 0 < ν ≤ 1 0 < \nu\le1 0 < ν ≤ 1 , and q = p + ν q=p+\nu q = p + ν . Let X , Y X,Y X , Y be normed spaces and U ⊂ X U \subset X U ⊂ X . A jet f = ( f 0 , f 1 , … , f p ) f=(f_{0},f_{1},\ldots,f_{p}) f = ( f 0 , f 1 , … , f p ) consists of f 0 : U → Y f_{0}:U \to Y f 0 : U → Y and symmetric k k k -linear maps f k ( x ) : X k → Y f_{k}(x):X^{k} \to Y f k ( x ) : X k → Y for 1 ≤ k ≤ p 1 \le k \le p 1 ≤ k ≤ p . Regard each multilinear map as its linear map on the tensor power. For 0 ≤ k ≤ p 0 \le k \le p 0 ≤ k ≤ p , x , y ∈ U x,y \in U x , y ∈ U , and every u ∈ X ⊗ k u \in X^{\otimes k} u ∈ X ⊗ k , define the remainder by
f k ( x ) [ u ] = ∑ a = 0 p − k 1 a ! f k + a ( y ) [ u ⊗ ( x − y ) ⊗ a ] + R k ( x , y ) [ u ] . f_{k}(x)[u]=\sum_{a=0}^{p-k}\frac{1}{a!}f_{k+a}(y)\left[u\otimes(x-y)^{\otimes a}\right]+R_{k}(x,y)[u]. f k ( x ) [ u ] = a = 0 ∑ p − k a ! 1 f k + a ( y ) [ u ⊗ ( x − y ) ⊗ a ] + R k ( x , y ) [ u ] . For u = z 1 ⊗ ⋯ ⊗ z k u=z_{1}\otimes\cdots\otimes z_{k} u = z 1 ⊗ ⋯ ⊗ z k , the bracket is the original multilinear evaluation, with a a a additional copies of x − y x-y x − y ; for k = 0 k=0 k = 0 use X ⊗ 0 = R X^{\otimes0}=\mathbb{R} X ⊗ 0 = R . The norm ∥ f ∥ Lip ( q ) \lVert f\rVert_{\operatorname{Lip}(q)} ∥ f ∥ Lip ( q ) is the least C C C such that, for all 0 ≤ k ≤ p 0 \le k \le p 0 ≤ k ≤ p , x , y ∈ U x,y \in U x , y ∈ U , and z 1 , … , z k ∈ X z_{1},\ldots,z_{k}\in X z 1 , … , z k ∈ X ,
∥ f k ( x ) [ z 1 , … , z k ] ∥ ≤ C ∥ z 1 ⊗ ⋯ ⊗ z k ∥ , \lVert f_{k}(x)[z_{1},\ldots,z_{k}]\rVert\le C\lVert z_{1}\otimes\cdots\otimes z_{k}\rVert, ∥ f k ( x ) [ z 1 , … , z k ]∥ ≤ C ∥ z 1 ⊗ ⋯ ⊗ z k ∥ , ∥ R k ( x , y ) [ z 1 , … , z k ] ∥ ≤ C ∥ x − y ∥ q − k ∥ z 1 ⊗ ⋯ ⊗ z k ∥ . \lVert R_{k}(x,y)[z_{1},\ldots,z_{k}]\rVert\le C\lVert x-y\rVert^{q-k}\lVert z_{1}\otimes\cdots\otimes z_{k}\rVert. ∥ R k ( x , y ) [ z 1 , … , z k ]∥ ≤ C ∥ x − y ∥ q − k ∥ z 1 ⊗ ⋯ ⊗ z k ∥ . The tensor norms have the projective property ∥ a ⊗ b ∥ ≤ ∥ a ∥ ∥ b ∥ \lVert a\otimes b\rVert\le\lVert a\rVert\lVert b\rVert ∥ a ⊗ b ∥ ≤ ∥ a ∥ ∥ b ∥ whenever the tensor orders match. No openness or convexity of U U U is required.
Theorem B.6 (Characterization of Lipschitz jets [6 Theorem 1.20] ). Let p ≥ 0 p \ge0 p ≥ 0 be an integer, 0 < ν ≤ 1 0 < \nu\le1 0 < ν ≤ 1 , C ≥ 0 C \ge0 C ≥ 0 , let X , Y X,Y X , Y be normed spaces, and let U ⊂ X U \subset X U ⊂ X . Suppose the tensor norms on X ⊗ k X^{\otimes k} X ⊗ k , 1 ≤ k ≤ p 1 \le k \le p 1 ≤ k ≤ p , are the canonical projective tensor norms. Given a jet f = ( f 0 , … , f p ) f=(f_0,\ldots,f_p) f = ( f 0 , … , f p ) as in Definition B.5, consider the assertions
(A1) f f f is Lip ( p + ν ) (p+\nu) ( p + ν ) and ∥ f ∥ Lip ( p + ν ) ≤ C \lVert f\rVert_{\operatorname{Lip}(p+\nu)} \le C ∥ f ∥ Lip ( p + ν ) ≤ C .
(A2) f 0 f_0 f 0 is p p p times differentiable, its successive derivatives are f 1 , … , f p f_1,\ldots,f_p f 1 , … , f p , all ∥ f k ∥ ∞ ≤ C \lVert f_k\rVert_\infty\le C ∥ f k ∥ ∞ ≤ C , and ∥ f p ( x ) − f p ( y ) ∥ ≤ C ∥ x − y ∥ ν \lVert f_p(x)-f_p(y)\rVert\le C\lVert x-y\rVert^\nu ∥ f p ( x ) − f p ( y )∥ ≤ C ∥ x − y ∥ ν for x , y ∈ U x,y \in U x , y ∈ U .
If U U U is open, (A1) implies (A2). If U U U is also convex, the two assertions are equivalent.
Theorem B.7 (Composition of Lipschitz jets [6 Theorem 1.48] ). Let X , Y , Z X,Y,Z X , Y , Z be normed spaces, U ⊂ X U \subset X U ⊂ X , V ⊂ Y V \subset Y V ⊂ Y , and q > 1 q>1 q > 1 . Define p = ⌈ q ⌉ − 1 p=\lceil q\rceil-1 p = ⌈ q ⌉ − 1 and ν = q − p \nu=q-p ν = q − p , so p ≥ 1 p \ge1 p ≥ 1 and 0 < ν ≤ 1 0<\nu\le1 0 < ν ≤ 1 . Equip the tensor powers of X X X and Y Y Y with the canonical projective tensor norms, and use the Lipschitz-jet convention of Definition B.5. If f : U → Y f:U \to Y f : U → Y and h : V → Z h:V \to Z h : V → Z are Lip ( q ) (q) ( q ) maps with f ( U ) ⊂ V f(U) \subset V f ( U ) ⊂ V , then the jet of h ∘ f h \circ f h ∘ f obtained by formal successive chain rules is Lip ( q ) (q) ( q ) , and
∥ h ∘ f ∥ Lip ( q ) ≤ C q comp ∥ h ∥ Lip ( q ) max { ∥ f ∥ Lip ( q ) q , 1 } . \lVert h \circ f\rVert_{\operatorname{Lip}(q)} \le C_q^{\operatorname{comp}}\lVert h\rVert_{\operatorname{Lip}(q)}\max\left\{\lVert f\rVert_{\operatorname{Lip}(q)}^q,1\right\}. ∥ h ∘ f ∥ Lip ( q ) ≤ C q comp ∥ h ∥ Lip ( q ) max { ∥ f ∥ Lip ( q ) q , 1 } . The constant depends only on q q q ; neither U , V U,V U , V nor the dimensions enter it.
Lemma B.8 (Bounded Hölder norms under scalar composition). Let p ≥ 1 p \ge1 p ≥ 1 be an integer, 0 < ν ≤ 1 0<\nu\le1 0 < ν ≤ 1 , and q = p + ν q=p+\nu q = p + ν . For a C p C^p C p scalar function on an open set W W W , define the bounded norm
∥ f ∥ p , ν ; W : = max { max 0 ≤ k ≤ p ∥ ∇ k f ∥ ∞ ; W , sup x , y ∈ W x ≠ y ∥ ∇ p f ( x ) − ∇ p f ( y ) ∥ ∥ x − y ∥ ν } . \lVert f\rVert_{p,\nu;W}:=\max\left\{\max_{0\le k\le p}\lVert\nabla^k f\rVert_{\infty;W},\sup_{\substack{x,y\in W\\x\ne y}}\frac{\lVert\nabla^p f(x)-\nabla^p f(y)\rVert}{\lVert x-y\rVert^\nu}\right\}. ∥ f ∥ p , ν ; W := max ⎩ ⎨ ⎧ 0 ≤ k ≤ p max ∥ ∇ k f ∥ ∞ ; W , x , y ∈ W x = y sup ∥ x − y ∥ ν ∥ ∇ p f ( x ) − ∇ p f ( y )∥ ⎭ ⎬ ⎫ . The domain subscript restricts both suprema to $W. The first term bounds the function and all its derivatives through order p p p ; the second bounds variations of the p p p th derivative. Let U ⊂ R d U \subset\mathbb{R}^d U ⊂ R d be convex and open, let J ⊂ R J \subset\mathbb{R} J ⊂ R be an open interval, and suppose f ∈ C p ( U ) f \in C^p(U) f ∈ C p ( U ) , h ∈ C p ( J ) h \in C^p(J) h ∈ C p ( J ) , f ( U ) ⊂ J f(U) \subset J f ( U ) ⊂ J , and both bounded norms below are finite. Then
∥ h ∘ f ∥ p , ν ; U ≤ C q comp ∥ h ∥ p , ν ; J max { 1 , ∥ f ∥ p , ν ; U q } , \lVert h\circ f\rVert_{p,\nu;U}\le C_q^{\operatorname{comp}}\lVert h\rVert_{p,\nu;J}\max\left\{1,\lVert f\rVert_{p,\nu;U}^q\right\}, ∥ h ∘ f ∥ p , ν ; U ≤ C q comp ∥ h ∥ p , ν ; J max { 1 , ∥ f ∥ p , ν ; U q } , where C q comp C_q^{\operatorname{comp}} C q comp depends only on q q q , not on the dimension or the domains. This includes ν = 1 \nu=1 ν = 1 and uses derivatives only through $p.
Proof. Equip each tensor power of R d \mathbb{R}^d R d with the projective norm
∥ u ∥ π : = inf u = ∑ ℓ z ℓ , 1 ⊗ ⋯ ⊗ z ℓ , k ∑ ℓ ∏ i = 1 k ∥ z ℓ , i ∥ . \lVert u\rVert_\pi:=\inf_{\substack{u=\sum_\ell z_{\ell,1}\otimes\cdots\otimes z_{\ell,k}}}\sum_\ell\prod_{i=1}^k\lVert z_{\ell,i}\rVert. ∥ u ∥ π := u = ∑ ℓ z ℓ , 1 ⊗ ⋯ ⊗ z ℓ , k inf ℓ ∑ i = 1 ∏ k ∥ z ℓ , i ∥ .
These norms satisfy ∥ u ⊗ z ∥ π = ∥ u ∥ π ∥ z ∥ \lVert u\otimes z\rVert_\pi=\lVert u\rVert_\pi\lVert z\rVert ∥ u ⊗ z ∥ π = ∥ u ∥ π ∥ z ∥ . Indeed, the projective property gives the upper bound. For the reverse bound, suppose z ≠ 0 z\ne0 z = 0 and define ℓ ( w ) = ⟨ z ∥ z ∥ , w ⟩ \ell(w)=\left\langle\frac{z}{\lVert z\rVert},w\right\rangle ℓ ( w ) = ⟨ ∥ z ∥ z , w ⟩ . Applying I ⊗ ℓ I\otimes\ell I ⊗ ℓ to any pure-tensor decomposition and using ∥ ℓ ∥ = 1 \lVert\ell\rVert=1 ∥ ℓ ∥ = 1 gives ∥ I ⊗ ℓ ∥ ≤ 1 \lVert I\otimes\ell\rVert\le1 ∥ I ⊗ ℓ ∥ ≤ 1 in the projective norms. Therefore
∥ u ∥ π ∥ z ∥ = ∥ ( I ⊗ ℓ ) ( u ⊗ z ) ∥ π since ℓ ( z ) = ∥ z ∥ , ≤ ∥ u ⊗ z ∥ π since ∥ I ⊗ ℓ ∥ ≤ 1. \begin{aligned}
\lVert u\rVert_\pi\lVert z\rVert=\lVert(I\otimes\ell)(u\otimes z)\rVert_\pi\quad\text{since }\ell(z)=\lVert z\rVert, \\
&\le\lVert u\otimes z\rVert_\pi\quad\text{since }\lVert I\otimes\ell\rVert\le1.
\end{aligned} ∥ u ∥ π ∥ z ∥ = ∥( I ⊗ ℓ ) ( u ⊗ z ) ∥ π since ℓ ( z ) = ∥ z ∥ , ≤ ∥ u ⊗ z ∥ π since ∥ I ⊗ ℓ ∥ ≤ 1.
The case z = 0 z = 0 z = 0 is immediate. For a k k k -linear form Q Q Q , linearity gives, for every such decomposition,
∣ Q [ u ] ∣ = ∣ ∑ ℓ Q [ z ℓ , 1 , … , z ℓ , k ] ∣ by linearity, ≤ ∑ ℓ ∣ Q [ z ℓ , 1 , … , z ℓ , k ] ∣ by the triangle inequality, ≤ ∥ Q ∥ ∑ ℓ ∏ i = 1 k ∥ z ℓ , i ∥ by the operator norm. \begin{aligned}
\lvert Q[u]\rvert= \left\lvert\sum_{\ell} Q[z_{\ell,1},\ldots,z_{\ell,k}] \right\rvert\qquad\text{by linearity,} \\
&\le\sum_{\ell} \lvert Q[z_{\ell,1},\ldots,z_{\ell,k}]\rvert\qquad\text{by the triangle inequality,} \\
&\le\lVert Q\rVert\sum_{\ell}\prod_{i=1}^{k}\lVert z_{\ell,i}\rVert\qquad\text{by the operator norm.}
\end{aligned} ∣ Q [ u ]∣ = ℓ ∑ Q [ z ℓ , 1 , … , z ℓ , k ] by linearity, ≤ ℓ ∑ ∣ Q [ z ℓ , 1 , … , z ℓ , k ]∣ by the triangle inequality, ≤ ∥ Q ∥ ℓ ∑ i = 1 ∏ k ∥ z ℓ , i ∥ by the operator norm.
Taking the infimum gives ∣ Q [ u ] ∣ ≤ ∥ Q ∥ ∥ u ∥ π \lvert Q[u]\rvert\le\lVert Q\rVert\lVert u\rVert_{\pi} ∣ Q [ u ]∣ ≤ ∥ Q ∥ ∥ u ∥ π . Conversely, testing the associated linear map on pure tensors and using ∥ z 1 ⊗ ⋯ ⊗ z k ∥ π = ∏ i ∥ z i ∥ \lVert z_1\otimes\cdots\otimes z_k\rVert_{\pi}=\prod_i\lVert z_i\rVert ∥ z 1 ⊗ ⋯ ⊗ z k ∥ π = ∏ i ∥ z i ∥ recovers the full multilinear operator norm. Thus the pure-tensor bounds in Definition B.5 extend to arbitrary tensors with exactly the same constant.
Define f ˉ = ( f , ∇ f , … , ∇ p f ) \bar{f}=(f,\nabla f,\ldots,\nabla^p f) f ˉ = ( f , ∇ f , … , ∇ p f ) and h ˉ = ( h , h ′ , … , h ( p ) ) \bar{h}=(h,h',\ldots,h^{(p)}) h ˉ = ( h , h ′ , … , h ( p ) ) . Since U , J U,J U , J are convex and open, Theorem B.6 gives
∥ f ˉ ∥ Lip ( q ) = ∥ f ∥ p , ν ; U , ∥ h ˉ ∥ Lip ( q ) = ∥ h ∥ p , ν ; J . \lVert\bar{f}\rVert_{\operatorname{Lip}(q)}=\lVert f\rVert_{p,\nu;U},\qquad\lVert\bar{h}\rVert_{\operatorname{Lip}(q)}=\lVert h\rVert_{p,\nu;J}. ∥ f ˉ ∥ Lip ( q ) = ∥ f ∥ p , ν ; U , ∥ h ˉ ∥ Lip ( q ) = ∥ h ∥ p , ν ; J .
The assumption f ( U ) ⊂ J f(U)\subset J f ( U ) ⊂ J verifies the range hypothesis of Theorem B.7 . Its composition jet therefore satisfies
∥ h ∘ f ∥ p , ν ; U = ∥ h ˉ ∘ f ˉ ∥ Lip ( q ) by xref‘Theorem B.6‘, ≤ C q comp ∥ h ˉ ∥ Lip ( q ) max { 1 , ∥ f ˉ ∥ Lip ( q ) q } by xref‘Theorem B.7‘, = C q comp ∥ h ∥ p , ν ; J max { 1 , ∥ f ∥ p , ν ; U q } by the two norm identities. \begin{aligned}
\lVert h\circ f\rVert_{p,\nu;U}=\lVert\bar{h}\circ\bar{f}\rVert_{\operatorname{Lip}(q)} \qquad\text{by {xref}`Theorem B.6`,} \\
&\le C_q^{\operatorname{comp}}\lVert\bar{h}\rVert_{\operatorname{Lip}(q)}\max\left\{1,\lVert\bar{f}\rVert_{\operatorname{Lip}(q)}^q\right\} \qquad\text{by {xref}`Theorem B.7`,} \\
&= C_q^{\operatorname{comp}}\lVert h\rVert_{p,\nu;J}\max\left\{1,\lVert f\rVert_{p,\nu;U}^q\right\} \qquad\text{by the two norm identities.}
\end{aligned} ∥ h ∘ f ∥ p , ν ; U = ∥ h ˉ ∘ f ˉ ∥ Lip ( q ) by xref ‘Theorem B.6‘, ≤ C q comp ∥ h ˉ ∥ Lip ( q ) max { 1 , ∥ f ˉ ∥ Lip ( q ) q } by xref ‘Theorem B.7‘, = C q comp ∥ h ∥ p , ν ; J max { 1 , ∥ f ∥ p , ν ; U q } by the two norm identities.
When ν = 1 \nu=1 ν = 1 , we still have p = q − 1 p=q-1 p = q − 1 , so Theorems B.6 and B.7 require derivatives only through p p p .
Orthogonal multilinear forms
Lemma B.9 (Tensor direct sums [21 Definition 2.2 and Lemma 3.1, Eq. (20)] ). Let k ≥ 3 k\ge3 k ≥ 3 and n ≥ 1 n\ge1 n ≥ 1 be integers, and let d j ( i ) d_j^{(i)} d j ( i ) be positive integers for 1 ≤ j ≤ k 1\le j\le k 1 ≤ j ≤ k and 1 ≤ i ≤ n 1\le i\le n 1 ≤ i ≤ n . Let Q i ∈ R d 1 ( i ) × ⋯ × d k ( i ) Q_i\in\mathbb{R}^{d_1^{(i)}\times\cdots\times d_k^{(i)}} Q i ∈ R d 1 ( i ) × ⋯ × d k ( i ) be real tensors. For their tensor direct sum
Q = Q 1 ⊕ ⋯ ⊕ Q n ∈ R ( ∑ i d 1 ( i ) ) × ⋯ × ( ∑ i d k ( i ) ) , Q=Q_1\oplus\cdots\oplus Q_n\in\mathbb{R}^{\left(\sum_i d_1^{(i)}\right)\times\cdots\times\left(\sum_i d_k^{(i)}\right)}, Q = Q 1 ⊕ ⋯ ⊕ Q n ∈ R ( ∑ i d 1 ( i ) ) × ⋯ × ( ∑ i d k ( i ) ) , the operator norm satisfies
∥ Q ∥ = max 1 ≤ i ≤ n ∥ Q i ∥ . \lVert Q\rVert=\max_{1\le i\le n}\lVert Q_i\rVert. ∥ Q ∥ = 1 ≤ i ≤ n max ∥ Q i ∥ . The spectral norm is defined by
∥ Q ∥ = max ∥ z 1 ∥ = ⋯ = ∥ z k ∥ = 1 ∣ Q [ z 1 , … , z k ] ∣ . \lVert Q\rVert=\max_{\lVert z_1\rVert=\cdots=\lVert z_k\rVert=1}\lvert Q[z_1,\ldots,z_k]\rvert. ∥ Q ∥ = ∥ z 1 ∥ = ⋯ = ∥ z k ∥ = 1 max ∣ Q [ z 1 , … , z k ]∣ . The test vectors in the k k k modes are independent. The direct sum places each tensor in its matching diagonal block and makes all mixed-block entries zero.
Lemma B.10 (Orthogonal sums of multilinear forms). Let k ≥ 2 k \ge2 k ≥ 2 be an integer, let V 1 , … , V n V_1,\ldots,V_n V 1 , … , V n be mutually orthogonal Euclidean subspaces, and let Q i Q_i Q i be a k k k -linear form on V i V_i V i . Define the ambient form
Q [ z 1 , … , z k ] = ∑ i = 1 n Q i [ P V i z 1 , … , P V i z k ] . Q[z_1,\ldots,z_k] = \sum_{i=1}^{n} Q_i[P_{V_i}z_1,\ldots,P_{V_i}z_k]. Q [ z 1 , … , z k ] = i = 1 ∑ n Q i [ P V i z 1 , … , P V i z k ] . Then its full multilinear operator norm satisfies
∥ Q ∥ = max 1 ≤ i ≤ n ∥ Q i ∥ . \lVert Q\rVert= \max_{1\le i\le n}\lVert Q_i\rVert. ∥ Q ∥ = 1 ≤ i ≤ n max ∥ Q i ∥ . Proof. Zero blocks contribute nothing. If only one nonzero block is present, restricting all test vectors to that block proves the identity. For k ≥ 3 k \ge3 k ≥ 3 and at least two nonzero blocks, choose orthonormal coordinates on each nonzero V i V_i V i . The ambient form is the tensor direct sum of the coordinate representations of Q i Q_i Q i , with all mixed-block entries zero. If the ambient space contains an orthogonal complement of ⨁ i V i \bigoplus_i V_i ⨁ i V i , include its zero tensor block in this direct sum. Lemma B.9 therefore gives the identity.
For k = 2 k=2 k = 2 , define M = max i ∥ Q i ∥ M=\max_i\lVert Q_i\rVert M = max i ∥ Q i ∥ . For arbitrary unit vectors z 1 , z 2 z_1,z_2 z 1 , z 2 ,
∣ Q [ z 1 , z 2 ] ∣ ≤ M ∑ i = 1 n ∥ P V i z 1 ∥ ∥ P V i z 2 ∥ by the block operator norms, \lvert Q[z_1,z_2]\rvert\le M\sum_{i=1}^{n}\lVert P_{V_i}z_1\rVert\lVert P_{V_i}z_2\rVert\qquad\text{by the block operator norms,} ∣ Q [ z 1 , z 2 ]∣ ≤ M i = 1 ∑ n ∥ P V i z 1 ∥ ∥ P V i z 2 ∥ by the block operator norms, ≤ M ( ∑ i ∥ P V i z 1 ∥ 2 ) 1 / 2 ( ∑ i ∥ P V i z 2 ∥ 2 ) 1 / 2 by Cauchy–Schwarz, \le M\left(\sum_i\lVert P_{V_i}z_1\rVert^2\right)^{1/2}\left(\sum_i\lVert P_{V_i}z_2\rVert^2\right)^{1/2} \qquad\text{by Cauchy--Schwarz,} ≤ M ( i ∑ ∥ P V i z 1 ∥ 2 ) 1/2 ( i ∑ ∥ P V i z 2 ∥ 2 ) 1/2 by Cauchy–Schwarz, ≤ M by orthogonality and ∥ z 1 ∥ = ∥ z 2 ∥ = 1. \le M \qquad\text{by orthogonality and }\lVert z_1\rVert=\lVert z_2\rVert=1. ≤ M by orthogonality and ∥ z 1 ∥ = ∥ z 2 ∥ = 1. Taking the supremum gives ∥ Q ∥ ≤ M \lVert Q\rVert\le M ∥ Q ∥ ≤ M . Conversely, choose an index i i i attaining the largest block norm and restrict both test vectors to V i V_i V i . The other blocks vanish, so ∥ Q ∥ ≥ ∥ Q i ∥ = M \lVert Q\rVert\ge\lVert Q_i\rVert=M ∥ Q ∥ ≥ ∥ Q i ∥ = M . This proves equality.
Estimates for ∣ t ∣ q q \frac{|t|^q}{q} q ∣ t ∣ q and ∥ x ∥ q q \frac{\lVert x\rVert^q}{q} q ∥ x ∥ q
The scalar divergence D ϕ ( a , b ) D_\phi(a,b) D ϕ ( a , b ) is defined in Appx. B.2.1.
Definition B.11 (Derivative polynomials [28 Definition 3.1] ). For real q q q and integers k ≥ 0 k\ge0 k ≥ 0 , define the polynomials by
g 0 , q ( u ) = 1 , g k , q ( u ) = ( 1 − u 2 ) g k − 1 , q ′ ( u ) + ( q − k + 1 ) u g k − 1 , q ( u ) ( k ≥ 1 ) . g_{0,q}(u)=1,\qquad g_{k,q}(u)=(1-u^2)g'_{k-1,q}(u)+(q-k+1)ug_{k-1,q}(u)\qquad(k\ge1). g 0 , q ( u ) = 1 , g k , q ( u ) = ( 1 − u 2 ) g k − 1 , q ′ ( u ) + ( q − k + 1 ) u g k − 1 , q ( u ) ( k ≥ 1 ) .
Lemma B.12 (Symmetric derivative norms and norm-power derivatives [28 Sec. 2, Eq. (1), Theorem 3.1, and Proposition 4.5] ). Equip R d \mathbb{R}^d R d with its Euclidean norm. For q > 0 q>0 q > 0 , define f q ( x ) = ∥ x ∥ q f_q(x)=\lVert x\rVert^q f q ( x ) = ∥ x ∥ q , and let g k , q g_{k,q} g k , q be the polynomials in Definition B.11.
(a) Symmetric operator norm. For a symmetric k k k -linear form A A A on R d \mathbb{R}^d R d , k ≥ 1 k\ge1 k ≥ 1 ,
∥ A ∥ = max ∥ z ∥ = 1 ∣ A [ z , … , z ] ∣ . \lVert A\rVert=\max_{\lVert z\rVert=1}\lvert A[z,\ldots,z]\rvert. ∥ A ∥ = ∥ z ∥ = 1 max ∣ A [ z , … , z ]∣ .
(b) Derivative formula. For real q > 0 q>0 q > 0 and every integer 0 ≤ k < q 0\le k<q 0 ≤ k < q , f q f_q f q is k k k times differentiable, and for every unit vector z z z ,
∇ k f q ( x ) [ z , … , z ] = ∥ x ∥ q − k g k , q ( τ z ( x ) ) , τ z ( x ) : = { ⟨ x , z ⟩ ∥ x ∥ , x ≠ 0 , 0 , x = 0. \nabla^k f_q(x)[z,\ldots,z]=\lVert x\rVert^{q-k}g_{k,q}(\tau_z(x)),\qquad\tau_z(x):=\begin{cases}\dfrac{\langle x,z\rangle}{\lVert x\rVert},&x\ne0,\\0,&x=0.\end{cases} ∇ k f q ( x ) [ z , … , z ] = ∥ x ∥ q − k g k , q ( τ z ( x )) , τ z ( x ) := ⎩ ⎨ ⎧ ∥ x ∥ ⟨ x , z ⟩ , 0 , x = 0 , x = 0.
At k = 0 k=0 k = 0 the displayed derivative means f q f_q f q itself.
(c) Polynomial bound. For every integer k ≥ 0 k\ge0 k ≥ 0 and real q ≥ k q\ge k q ≥ k ,
max ∣ u ∣ ≤ 1 ∣ g k , q ( u ) ∣ = ∏ ℓ = 0 k − 1 ( q − ℓ ) . \max_{\lvert u\rvert\le1}\lvert g_{k,q}(u)\rvert=\prod_{\ell=0}^{k-1}(q-\ell). ∣ u ∣ ≤ 1 max ∣ g k , q ( u )∣ = ℓ = 0 ∏ k − 1 ( q − ℓ ) .
An empty product equals one.
Lemma B.13 (Hölder continuity of norm-power derivatives [28 Theorems 6.2–6.3 and Eq. (24)] ). Let p ≥ 0 p\ge0 p ≥ 0 be an integer, 0 < ν ≤ 1 0<\nu\le1 0 < ν ≤ 1 , q = p + ν q=p+\nu q = p + ν , and f q ( x ) = ∥ x ∥ q f_q(x)=\lVert x\rVert^q f q ( x ) = ∥ x ∥ q on Euclidean R d \mathbb{R}^d R d . Let g k , q g_{k,q} g k , q be the polynomials in Definition B.11 .
(a) Restriction to a line through the origin. The restriction of ∇ p f q \nabla^p f_q ∇ p f q to any line through the origin is ν \nu ν -Hölder continuous with constant
C p , ν : = { ∏ i = 1 p ( ν + i ) , p even , 2 1 − ν ∏ i = 1 p ( ν + i ) , p odd . C_{p,\nu}:=\begin{cases}
\prod_{i=1}^{p}(\nu+i),&p\text{ even},\\
2^{1-\nu}\prod_{i=1}^{p}(\nu+i),&p\text{ odd}.
\end{cases} C p , ν := { ∏ i = 1 p ( ν + i ) , 2 1 − ν ∏ i = 1 p ( ν + i ) , p even , p odd .
(b) Global Hölder continuity. The map ∇ p f q \nabla^p f_q ∇ p f q is globally ν \nu ν -Hölder continuous with constant
A p , ν : = { ( p − 1 ) ! ! ∏ i = 1 p / 2 ( ν + 2 i ) + H p , ν , p even , 2 1 − ν ∏ i = 1 p ( ν + i ) , p odd . A_{p,\nu}:=\begin{cases}
(p-1)!!\prod_{i=1}^{p/2}(\nu+2i)+H_{p,\nu},&p\text{ even},\\
2^{1-\nu}\prod_{i=1}^{p}(\nu+i),&p\text{ odd}.
\end{cases} A p , ν := { ( p − 1 )!! ∏ i = 1 p /2 ( ν + 2 i ) + H p , ν , 2 1 − ν ∏ i = 1 p ( ν + i ) , p even , p odd .
where H p , ν H_{p,\nu} H p , ν denotes the ν \nu ν -Hölder constant of g p , p + ν g_{p,p+\nu} g p , p + ν on [ 0 , 1 ] [0,1] [ 0 , 1 ] :
H p , ν : = sup 0 ≤ u < v ≤ 1 ∣ g p , p + ν ( v ) − g p , p + ν ( u ) ∣ ∣ v − u ∣ ν . H_{p,\nu}:=\sup_{0\le u<v\le1}\frac{\lvert g_{p,p+\nu}(v)-g_{p,p+\nu}(u)\rvert}{\lvert v-u\rvert^\nu}. H p , ν := 0 ≤ u < v ≤ 1 sup ∣ v − u ∣ ν ∣ g p , p + ν ( v ) − g p , p + ν ( u )∣ .
In particular, a valid global constant is
A ~ p , ν : = { ( p − 1 ) ! ! ∏ i = 1 p / 2 ( ν + 2 i ) + ∏ i = 1 p ( ν + i ) , p even , 2 1 − ν ∏ i = 1 p ( ν + i ) , p odd . \widetilde{A}_{p,\nu}:=\begin{cases}
(p-1)!!\prod_{i=1}^{p/2}(\nu+2i)+\prod_{i=1}^{p}(\nu+i),&p\text{ even},\\
2^{1-\nu}\prod_{i=1}^{p}(\nu+i),&p\text{ odd}.
\end{cases} A p , ν := { ( p − 1 )!! ∏ i = 1 p /2 ( ν + 2 i ) + ∏ i = 1 p ( ν + i ) , 2 1 − ν ∏ i = 1 p ( ν + i ) , p even , p odd .
For even p p p , ( p − 1 ) ! ! = ∏ i = 1 p / 2 ( 2 i − 1 ) (p-1)!!=\prod_{i=1}^{p/2}(2i-1) ( p − 1 )!! = ∏ i = 1 p /2 ( 2 i − 1 ) , including ( − 1 ) ! ! = 1 (-1)!!=1 ( − 1 )!! = 1 . Global Hölder continuity here means ∥ ∇ p f q ( x ) − ∇ p f q ( y ) ∥ ≤ A p , ν ∥ x − y ∥ ν \lVert\nabla^p f_q(x)-\nabla^p f_q(y)\rVert\le A_{p,\nu}\lVert x-y\rVert^\nu ∥ ∇ p f q ( x ) − ∇ p f q ( y )∥ ≤ A p , ν ∥ x − y ∥ ν for all x , y x,y x , y , and likewise with A ~ p , ν \widetilde{A}_{p,\nu} A p , ν .
Lemma B.14. (Uniform monotonicity implies uniform convexity [16 Lemma 2.1] ). Let C C C be a convex subset of the open domain of a differentiable function f f f . If, for some σ > 0 \sigma> 0 σ > 0 and real q ≥ 2 q \ge2 q ≥ 2 ,
⟨ ∇ f ( y ) − ∇ f ( x ) , y − x ⟩ ≥ σ ∥ y − x ∥ q ( x , y ∈ C ) , \langle\nabla f(y)-\nabla f(x),y-x\rangle\ge\sigma\lVert y-x\rVert^{q} \qquad(x,y\in C), ⟨ ∇ f ( y ) − ∇ f ( x ) , y − x ⟩ ≥ σ ∥ y − x ∥ q ( x , y ∈ C ) , then f f f is uniformly convex of degree q q q on C C C with parameter σ \sigma σ , i.e.,
f ( y ) ≥ f ( x ) + ⟨ ∇ f ( x ) , y − x ⟩ + σ q ∥ y − x ∥ q ( x , y ∈ C ) . f(y) \ge f(x)+\langle\nabla f(x),y-x\rangle+\frac{\sigma}{q}\lVert y-x\rVert^{q} \qquad(x,y\in C). f ( y ) ≥ f ( x ) + ⟨ ∇ f ( x ) , y − x ⟩ + q σ ∥ y − x ∥ q ( x , y ∈ C ) .
Lemma B.15. (Uniform monotonicity of the Euclidean norm power [16 Lemma 2.5(1), Eq. (16)] ). For a fixed real q ≥ 2 q \ge2 q ≥ 2 , the function f ( x ) = ∥ x ∥ q q f(x)=\frac{\lVert x\rVert^{q}}{q} f ( x ) = q ∥ x ∥ q , x ∈ R d x\in\mathbb{R}^{d} x ∈ R d , is uniformly convex of degree q q q and satisfies
⟨ ∇ f ( y ) − ∇ f ( x ) , y − x ⟩ ≥ 2 2 − q ∥ y − x ∥ q ( x , y ∈ R d ) . \langle\nabla f(y)-\nabla f(x),y-x\rangle\ge2^{2-q}\lVert y-x\rVert^{q} \qquad(x,y\in\mathbb{R}^{d}). ⟨ ∇ f ( y ) − ∇ f ( x ) , y − x ⟩ ≥ 2 2 − q ∥ y − x ∥ q ( x , y ∈ R d ) .
Lemma B.16. (Scalar and Euclidean norm estimates). For every d ≥ 1 d\ge1 d ≥ 1 , let ϕ ( t ) = ∣ t ∣ q q \phi(t)=\frac{|t|^{q}}{q} ϕ ( t ) = q ∣ t ∣ q and f ( x ) = ∥ x ∥ q q f(x)=\frac{\lVert x\rVert^{q}}{q} f ( x ) = q ∥ x ∥ q . The constants below depend only on q q q and the indicated derivative order.
(a) Derivatives away from zero. The function f f f is smooth on R d ∖ { 0 } \mathbb{R}^{d}\setminus\{0\} R d ∖ { 0 } . With C q , k C_{q,k} C q , k defined in Eq. (B.11), every integer 1 ≤ k ≤ p 1\le k\le p 1 ≤ k ≤ p satisfies
∥ ∇ k f ( x ) ∥ ≤ C q , k ∥ x ∥ q − k ( x ≠ 0 ) . (32) \lVert\nabla^{k}f(x)\rVert\le C_{q,k}\lVert x\rVert^{q-k}\qquad(x\ne0). \tag*{(32)} ∥ ∇ k f ( x )∥ ≤ C q , k ∥ x ∥ q − k ( x = 0 ) . (32) For 1 ≤ k ≤ p 1\le k\le p 1 ≤ k ≤ p , the scalar derivatives are
S q , k : = ∏ ℓ = 1 k − 1 ( q − ℓ ) , S q , 1 = 1 , S_{q,k}:=\prod_{\ell=1}^{k-1}(q-\ell),\qquad S_{q,1}=1, S q , k := ℓ = 1 ∏ k − 1 ( q − ℓ ) , S q , 1 = 1 , ϕ ( k ) ( a ) = S q , k sign ( a ) k ∣ a ∣ q − k ( a ≠ 0 ) . (33) \phi^{(k)}(a)=S_{q,k}\operatorname{sign}(a)^{k}|a|^{q-k}\qquad(a\ne0). \tag*{(33)} ϕ ( k ) ( a ) = S q , k sign ( a ) k ∣ a ∣ q − k ( a = 0 ) . (33) (b) Derivatives at zero. Both functions belong to C p C^{p} C p , and all their derivatives of orders 1 , … , p 1,\ldots,p 1 , … , p vanish at zero. For 1 ≤ k ≤ p 1\le k\le p 1 ≤ k ≤ p , Eq. (B.4) also holds at x = 0 x=0 x = 0 . For every linear map r : R d → R r:\mathbb{R}^{d}\to\mathbb{R} r : R d → R , all derivatives of ϕ ∘ r \phi\circ r ϕ ∘ r through order p p p vanish at any point where r ( x ) = 0 r(x)=0 r ( x ) = 0 .
(c) Hessian lower bound. For every x ∈ R d x\in\mathbb{R}^{d} x ∈ R d ,
∇ 2 f ( x ) ⪰ ∥ x ∥ q − 2 I . (34) \nabla^{2}f(x)\succeq\lVert x\rVert^{q-2}I. \tag*{(34)} ∇ 2 f ( x ) ⪰ ∥ x ∥ q − 2 I . (34) In particular, the Hessian is positive definite when x ≠ 0 x\ne0 x = 0 .
(d) Global Hölder estimates. With H p o w H_{\mathrm{pow}} H pow defined in Eq. (B.13) and L ϕ : = 2 1 − ν S q , p L_{\phi}:=2^{1-\nu}S_{q,p} L ϕ := 2 1 − ν S q , p , for every x , y ∈ R d x,y\in\mathbb{R}^{d} x , y ∈ R d and a , b ∈ R a,b\in\mathbb{R} a , b ∈ R ,
∥ ∇ p f ( x ) − ∇ p f ( y ) ∥ ≤ H p o w ∥ x − y ∥ ν , (35) \lVert\nabla^{p}f(x)-\nabla^{p}f(y)\rVert\le H_{\mathrm{pow}}\lVert x-y\rVert^{\nu}, \tag*{(35)} ∥ ∇ p f ( x ) − ∇ p f ( y )∥ ≤ H pow ∥ x − y ∥ ν , (35) ∣ ϕ ( p ) ( a ) − ϕ ( p ) ( b ) ∣ ≤ L ϕ ∣ a − b ∣ ν . (36) |\phi^{(p)}(a)-\phi^{(p)}(b)|\le L_{\phi}|a-b|^{\nu}. \tag*{(36)} ∣ ϕ ( p ) ( a ) − ϕ ( p ) ( b ) ∣ ≤ L ϕ ∣ a − b ∣ ν . (36) (e) Scalar divergence lower bound. With c q : = 2 2 − q q > 0 c_q:=\frac{2^{2-q}}{q}>0 c q := q 2 2 − q > 0 ,
D ϕ ( a , b ) ≥ c q ∣ a − b ∣ q ( a , b ∈ R ) . (37) D_{\phi}(a,b)\ge c_q|a-b|^{q}\qquad(a,b\in\mathbb{R}). \tag*{(37)} D ϕ ( a , b ) ≥ c q ∣ a − b ∣ q ( a , b ∈ R ) . (37)
Proof. • Proof of (a). Away from zero, f = h ∘ s f=h\circ s f = h ∘ s with s ( x ) = ⟨ x , x ⟩ s(x)=\langle x,x\rangle s ( x ) = ⟨ x , x ⟩ and h ( v ) = v q / 2 q h(v)=\frac{v^{q/2}}{q} h ( v ) = q v q /2 for v > 0 v>0 v > 0 , so f f f is smooth. Direct differentiation gives ∇ f ( x ) = r q − 2 x \nabla f(x)=r^{q-2}x ∇ f ( x ) = r q − 2 x , where r = ∥ x ∥ r=\lVert x\rVert r = ∥ x ∥ , and
∇ 2 f ( x ) = r q − 2 I + ( q − 2 ) r q − 4 x x T . (38) \nabla^{2}f(x)=r^{q-2}I+(q-2)r^{q-4}xx^{\mathsf{T}}. \tag*{(38)} ∇ 2 f ( x ) = r q − 2 I + ( q − 2 ) r q − 4 x x T . (38)
For $1 \le k \le p < q$, Lemma B.12 gives the derivative norm bound with the constants
C q , k : = S q , k = ∏ ℓ = 1 k − 1 ( q − ℓ ) ( 1 ≤ k ≤ p ) , (39) C_{q,k} := S_{q,k} = \prod_{\ell=1}^{k-1} (q-\ell) \qquad(1 \le k \le p), \tag*{(39)} C q , k := S q , k = ℓ = 1 ∏ k − 1 ( q − ℓ ) ( 1 ≤ k ≤ p ) , (39)
where the empty product is 1. Lemma B.12 (a) identifies the diagonal norm of a symmetric form with its full operator norm, giving
∥ ∇ k f ( x ) ∥ ≤ C q , k ∥ x ∥ q − k ( x ≠ 0 , 1 ≤ k ≤ p ) . (40) \lVert\nabla^{k} f(x) \rVert\le C_{q,k} \lVert x \rVert^{q-k} \qquad(x \ne0,\ 1 \le k \le p). \tag*{(40)} ∥ ∇ k f ( x )∥ ≤ C q , k ∥ x ∥ q − k ( x = 0 , 1 ≤ k ≤ p ) . (40)
Dividing the derivative product by q q q gives C q , k C_{q,k} C q , k . Ordinary differentiation on the two half-lines gives Eq. (B.5).
Proof of (b). Lemma B.12 (b) gives derivatives of ∥ x ∥ q \lVert x \rVert^{q} ∥ x ∥ q for every integer k < q k < q k < q , with those of positive order zero at the origin. Applying this formula for $1 \le k \le p < q$ and dividing by q q q proves the vanishing derivatives of f f f . Eq. (B.12), with q − k > 0 q-k>0 q − k > 0 , gives continuity there and extends the bound to zero. The scalar case is d = 1 d=1 d = 1 . For a linear r r r , the chain rule gives
∇ k ( ϕ ∘ r ) ( x ) [ z 1 , … , z k ] = ϕ ( k ) ( r ( x ) ) ∏ i = 1 k r ( z i ) , \nabla^{k}(\phi\circ r)(x)[z_{1},\ldots,z_{k}] = \phi^{(k)}(r(x)) \prod_{i=1}^{k} r(z_i), ∇ k ( ϕ ∘ r ) ( x ) [ z 1 , … , z k ] = ϕ ( k ) ( r ( x )) i = 1 ∏ k r ( z i ) ,
which is zero whenever r ( x ) = 0 r(x)=0 r ( x ) = 0 .
∇ 2 f ( x ) [ z , z ] = ∥ x ∥ q − 2 ∥ z ∥ 2 + ( q − 2 ) ∥ x ∥ q − 4 ⟨ x , z ⟩ 2 ≥ ∥ x ∥ q − 2 ∥ z ∥ 2 since q > 2. \begin{aligned}
\nabla^{2} f(x)[z,z] = \lVert x \rVert^{q-2}\lVert z\rVert^{2} + (q-2)\lVert x\rVert^{q-4}\langle x,z\rangle^{2} \\
&\ge\lVert x\rVert^{q-2}\lVert z\rVert^{2} \qquad\text{since } q>2.
\end{aligned} ∇ 2 f ( x ) [ z , z ] = ∥ x ∥ q − 2 ∥ z ∥ 2 + ( q − 2 ) ∥ x ∥ q − 4 ⟨ x , z ⟩ 2 ≥ ∥ x ∥ q − 2 ∥ z ∥ 2 since q > 2.
This is the matrix inequality Eq. (B.6). At zero both sides vanish by part (b).
Proof of (d). Lemma B.13 (b) gives the dimension-independent constant A ~ p , ν \widetilde{A}_{p,\nu} A p , ν . For even p p p , compare each odd factor separately:
( p − 1 ) ! ! ∏ i = 1 p / 2 ( ν + 2 i ) = ∏ i = 1 p / 2 ( 2 i − 1 ) ( ν + 2 i ) ≤ ∏ i = 1 p / 2 ( ν + 2 i − 1 ) ( ν + 2 i ) since ν > 0 , = ∏ i = 1 p ( ν + i ) . \begin{aligned}
(p-1)!! \prod_{i=1}^{p/2}(\nu+2i) = \prod_{i=1}^{p/2}(2i-1)(\nu+2i) \\
&\le\prod_{i=1}^{p/2}(\nu+2i-1)(\nu+2i) \qquad\text{since } \nu>0, \\
&= \prod_{i=1}^{p}(\nu+i).
\end{aligned} ( p − 1 )!! i = 1 ∏ p /2 ( ν + 2 i ) = i = 1 ∏ p /2 ( 2 i − 1 ) ( ν + 2 i ) ≤ i = 1 ∏ p /2 ( ν + 2 i − 1 ) ( ν + 2 i ) since ν > 0 , = i = 1 ∏ p ( ν + i ) .
and hence A ~ p , ν ≤ 2 ∏ i = 1 p ( ν + i ) \widetilde{A}_{p,\nu} \le2\prod_{i=1}^{p}(\nu+i) A p , ν ≤ 2 ∏ i = 1 p ( ν + i ) . For odd p p p , the same inequality follows from 2 1 − ν ≤ 2 2^{1-\nu}\le2 2 1 − ν ≤ 2 . Thus the ν \nu ν -Hölder constant of ∇ p ∥ x ∥ q \nabla^{p}\lVert x\rVert^{q} ∇ p ∥ x ∥ q is at most 2 ∏ i = 1 p ( ν + i ) = 2 q S q , p 2\prod_{i=1}^{p}(\nu+i)=2qS_{q,p} 2 ∏ i = 1 p ( ν + i ) = 2 q S q , p for 0 < ν ≤ 1 0<\nu\le1 0 < ν ≤ 1 . Dividing by q q q gives Eq. (B.7) with
H p o w : = 2 S q , p . (41) H_{\mathrm{pow}} := 2S_{q,p}. \tag*{(41)} H pow := 2 S q , p . (41)
For the scalar estimate, Lemma B.13 (a), restricted to d = 1 d=1 d = 1 and divided by q q q , gives the constant S q , p S_{q,p} S q , p for even p p p and 2 1 − ν S q , p 2^{1-\nu}S_{q,p} 2 1 − ν S q , p for odd p p p . Both are at most the stated L ϕ = 2 1 − ν S q , p L_{\phi}=2^{1-\nu}S_{q,p} L ϕ = 2 1 − ν S q , p , proving Eq. (B.8).
( ϕ ′ ( a ) − ϕ ′ ( b ) ) ( a − b ) ≥ 2 2 − q ∣ a − b ∣ q . (\phi'(a)-\phi'(b))(a-b) \ge2^{2-q}|a-b|^{q}. ( ϕ ′ ( a ) − ϕ ′ ( b )) ( a − b ) ≥ 2 2 − q ∣ a − b ∣ q .
Lemma B.14 then gives
D ϕ ( a , b ) ≥ ( 2 2 − q q ) ∣ a − b ∣ q , D_{\phi}(a,b) \ge\left(\frac{2^{2-q}}{q}\right)|a-b|^{q}, D ϕ ( a , b ) ≥ ( q 2 2 − q ) ∣ a − b ∣ q ,
which is Eq. (B.9). ∊pp
Positive-part derivatives and common translation
We use the scaled functions in Eq. (B.1).
Lemma B.17 (Positive-part derivatives and common translation). For δ > 0 \delta> 0 δ > 0 , let ψ δ \psi_\delta ψ δ and M δ M_\delta M δ be defined by Eq. (B.1).
(a) Exact formulas for ψ δ \psi_\delta ψ δ .
ψ δ ( v ) = 0 ( v ≤ − δ ) , (42) \psi_\delta(v) = 0 \qquad(v \le-\delta), \tag*{(42)} ψ δ ( v ) = 0 ( v ≤ − δ ) , (42) ψ δ ( v ) = v ( v ≥ δ ) . (43) \psi_\delta(v) = v \qquad(v \ge\delta). \tag*{(43)} ψ δ ( v ) = v ( v ≥ δ ) . (43) (b) Smoothness and derivatives of ψ δ \psi_\delta ψ δ . The function ψ δ \psi_\delta ψ δ belongs to C ∞ ( R ) C^\infty(\mathbb{R}) C ∞ ( R ) and is convex and nondecreasing, with
0 ≤ ψ δ ′ ( v ) ≤ 1. (44) 0 \le\psi_\delta'(v) \le1. \tag*{(44)} 0 ≤ ψ δ ′ ( v ) ≤ 1. (44) (c) Common translation. For every a ∈ R a \in\mathbb{R} a ∈ R ,
M δ ( s + a , t + a ) = M δ ( s , t ) + a . (45) M_\delta(s+a,t+a) = M_\delta(s,t) + a. \tag*{(45)} M δ ( s + a , t + a ) = M δ ( s , t ) + a . (45) Proof. • Proof of (a). Substitute Θ ( v δ ) = ∣ v ∣ δ \Theta\left(\frac{v}{\delta}\right)=\frac{|v|}{\delta} Θ ( δ v ) = δ ∣ v ∣ into Eq. (B.1) when ∣ v ∣ ≥ δ |v| \ge\delta ∣ v ∣ ≥ δ . Then
ψ δ ( v ) = v + δ ( ∣ v ∣ δ ) 2 \psi_\delta(v) = \frac{v+\delta\left(\frac{|v|}{\delta}\right)}{2} ψ δ ( v ) = 2 v + δ ( δ ∣ v ∣ ) by Eq. (B.1),
= v + ∣ v ∣ 2 = { 0 , v ≤ − δ , v , v ≥ δ . = \frac{v+|v|}{2} =
\begin{cases}
0, & v \le-\delta,\\
v, & v \ge\delta.
\end{cases} = 2 v + ∣ v ∣ = { 0 , v , v ≤ − δ , v ≥ δ . • Proof of (b). Differentiate the fixed smooth profile:
ψ δ ′ ( v ) = 1 + Θ ′ ( v δ ) 2 , ψ δ ′ ′ ( v ) = Θ ′ ′ ( v δ ) 2 δ . (46) \psi_\delta'(v) = \frac{1+\Theta'\left(\frac{v}{\delta}\right)}{2}, \qquad\psi_\delta''(v) = \frac{\Theta''\left(\frac{v}{\delta}\right)}{2\delta}. \tag*{(46)} ψ δ ′ ( v ) = 2 1 + Θ ′ ( δ v ) , ψ δ ′′ ( v ) = 2 δ Θ ′′ ( δ v ) . (46) The bounds ∣ Θ ′ ∣ ≤ 1 |\Theta'| \le1 ∣ Θ ′ ∣ ≤ 1 and Θ ′ ′ ≥ 0 \Theta'' \ge0 Θ ′′ ≥ 0 give Eq. (B.16), monotonicity, and convexity.
• Proof of (c). By Eq. (B.1),
M δ ( s + a , t + a ) = ( s + a ) + ψ δ ( ( t + a ) − ( s + a ) ) M_\delta(s+a,t+a) = (s+a) + \psi_\delta((t+a)-(s+a)) M δ ( s + a , t + a ) = ( s + a ) + ψ δ (( t + a ) − ( s + a )) = s + ψ δ ( t − s ) + a = M δ ( s , t ) + a . = s+\psi_\delta(t-s)+a = M_\delta(s,t)+a. = s + ψ δ ( t − s ) + a = M δ ( s , t ) + a .
Composition and joining
Corollary B.18 (Smooth convex replacement). Let U ⊂ R d U \subset\mathbb{R}^d U ⊂ R d be open and convex, let f , g ∈ C p ( U ) f,g \in C^p(U) f , g ∈ C p ( U ) be convex, and fix δ > 0 \delta> 0 δ > 0 . Define h ( x ) : = M δ ( f ( x ) , g ( x ) ) h(x) := M_\delta(f(x),g(x)) h ( x ) := M δ ( f ( x ) , g ( x )) for x ∈ U x \in U x ∈ U . Then:
(a) h h h is convex and belongs to C p ( U ) C^p(U) C p ( U ) .
(b) If ∣ f ( x ) − g ( x ) ∣ > δ |f(x)-g(x)| > \delta ∣ f ( x ) − g ( x ) ∣ > δ at x ∈ U x \in U x ∈ U , there is an open neighborhood V ⊂ U V \subset U V ⊂ U of x x x such that, for every y ∈ V y \in V y ∈ V and 0 ≤ k ≤ p 0 \le k \le p 0 ≤ k ≤ p ,
∇ k h ( y ) = { ∇ k f ( y ) , f ( x ) > g ( x ) + δ , ∇ k g ( y ) , g ( x ) > f ( x ) + δ . \nabla^k h(y) =
\begin{cases}
\nabla^k f(y), & f(x) > g(x)+\delta,\\
\nabla^k g(y), & g(x) > f(x)+\delta.
\end{cases} ∇ k h ( y ) = { ∇ k f ( y ) , ∇ k g ( y ) , f ( x ) > g ( x ) + δ , g ( x ) > f ( x ) + δ .
Proof of Corollary B.18. (a) Convexity follows from convexity and coordinatewise monotonicity of M δ M_\delta M δ ; C p C^p C p regularity follows by the chain rule.
(b) Continuity of f − g f-g f − g preserves whichever strict inequality holds at x x x on an open neighborhood V ⊂ U V \subset U V ⊂ U . Lemma 4.2 then gives h = f h=f h = f or h = g h=g h = g throughout V V V , so their derivatives through order p p p agree there.
Lemma B.19 (Convexity criteria and operations [9 Secs. 3.1.3–3.1.4 and 3.2.1–3.2.2] ). For real-valued functions on Euclidean spaces, the following statements hold.
(a) Suppose f f f is differentiable on its open domain U U U . Then f f f is convex if and only if U U U is convex and
f ( y ) ≥ f ( x ) + ⟨ ∇ f ( x ) , y − x ⟩ ( x , y ∈ U ) . f(y) \ge f(x) + \langle\nabla f(x), y-x\rangle\qquad(x,y \in U). f ( y ) ≥ f ( x ) + ⟨ ∇ f ( x ) , y − x ⟩ ( x , y ∈ U ) . Equivalently, D f ( y , x ) ≥ 0 D_f(y,x) \ge0 D f ( y , x ) ≥ 0 . For convex f f f , x , y ∈ U x,y \in U x , y ∈ U , and 0 ≤ t ≤ 1 0 \le t \le1 0 ≤ t ≤ 1 , the equivalent value inequality is
f ( ( 1 − t ) x + t y ) ≤ ( 1 − t ) f ( x ) + t f ( y ) . f\bigl((1-t)x+ty\bigr) \le(1-t)f(x)+tf(y). f ( ( 1 − t ) x + t y ) ≤ ( 1 − t ) f ( x ) + t f ( y ) . (b) Suppose f f f is twice differentiable on its open domain U U U . Then f f f is convex if and only if U U U is convex and ∇ 2 f ( x ) ⪰ 0 \nabla^2 f(x) \succeq0 ∇ 2 f ( x ) ⪰ 0 for all x ∈ U x \in U x ∈ U .
(c) A nonnegative weighted sum ∑ i = 1 n a i f i \sum_{i=1}^{n} a_i f_i ∑ i = 1 n a i f i of convex functions is convex, and a nonnegative weighted sum of concave functions is concave.
(d) For a linear map L : R d → R n L:\mathbb{R}^d \to\mathbb{R}^n L : R d → R n and b ∈ R n b \in\mathbb{R}^n b ∈ R n , set g ( x ) = f ( L x + b ) g(x)=f(Lx+b) g ( x ) = f ( Lx + b ) with dom g = { x : L x + b ∈ dom f } \operatorname{dom}g=\{x:Lx+b \in\operatorname{dom}f\} dom g = { x : Lx + b ∈ dom f } . If f f f is convex, then g g g is convex; if f f f is concave, then g g g is concave.
Lemma B.20 (Convex additions and affine terms). Let F , G : R d → R F,G:\mathbb{R}^d \to\mathbb{R} F , G : R d → R be differentiable, with G G G convex, and let ℓ : R d → R \ell:\mathbb{R}^d \to\mathbb{R} ℓ : R d → R be affine. For all x , y ∈ R d x,y \in\mathbb{R}^d x , y ∈ R d ,
D F + G + ℓ ( y , x ) = D F ( y , x ) + D G ( y , x ) ≥ D F ( y , x ) . D_{F+G+\ell}(y,x)=D_F(y,x)+D_G(y,x)\ge D_F(y,x). D F + G + ℓ ( y , x ) = D F ( y , x ) + D G ( y , x ) ≥ D F ( y , x ) .
Proof. Expanding the Bregman definition in Sec. 3 gives additivity and D ℓ = 0 D_\ell=0 D ℓ = 0 . Lemma B.19 (a) gives D G ≥ 0 D_G \ge0 D G ≥ 0 .
Lemma B.21 (Composition and exact joining). Let U ⊆ R d U \subseteq\mathbb{R}^d U ⊆ R d be open and convex, let r ≥ 2 r \ge2 r ≥ 2 be an integer, and let f 1 , f 2 ∈ C r ( U ) f_1,f_2 \in C^r(U) f 1 , f 2 ∈ C r ( U ) be convex. For δ > 0 \delta>0 δ > 0 , define
F ( x ) : = M δ ( f 1 ( x ) , f 2 ( x ) ) , g : = f 2 − f 1 , a ( x ) : = ψ δ ′ ( g ( x ) ) . F(x):=M_\delta\bigl(f_1(x),f_2(x)\bigr), \qquad g:=f_2-f_1, \qquad a(x):=\psi_\delta'(g(x)). F ( x ) := M δ ( f 1 ( x ) , f 2 ( x ) ) , g := f 2 − f 1 , a ( x ) := ψ δ ′ ( g ( x )) . (a) Smooth convex composition. The function F F F belongs to C r ( U ) C^r(U) C r ( U ) and is convex. Its gradient and Hessian satisfy Eq. (B.19) and (B.20).
(b) Preservation of a common Hessian bound. If ∇ 2 f i ( x ) ⪰ K ( x ) \nabla^2 f_i(x)\succeq K(x) ∇ 2 f i ( x ) ⪰ K ( x ) for i = 1 , 2 i=1,2 i = 1 , 2 , where K ( x ) K(x) K ( x ) is symmetric, then ∇ 2 F ( x ) ⪰ K ( x ) \nabla^2F(x)\succeq K(x) ∇ 2 F ( x ) ⪰ K ( x ) .
(c) Agreement with the larger input. On any open region where f i − f 3 − i ≥ δ f_i-f_{3-i}\ge\delta f i − f 3 − i ≥ δ , the equality F = f i F=f_i F = f i holds, with ∇ k F = ∇ k f i \nabla^kF=\nabla^k f_i ∇ k F = ∇ k f i for $0\le k\le r.
(d) Joining to the unchanged function. Suppose B ‾ ( c , R ) ⊂ U \overline{B}(c,R)\subset U B ( c , R ) ⊂ U and f 1 − f 2 ≥ δ f_1-f_2\ge\delta f 1 − f 2 ≥ δ on an open neighborhood of the sphere { x : ∥ x − c ∥ = R } \{x:\lVert x-c\rVert=R\} { x : ∥ x − c ∥ = R } . Then
F ~ ( x ) = { M δ ( f 1 ( x ) , f 2 ( x ) ) , x ∈ B ( c , R ) , f 1 ( x ) , x ∈ U ∖ B ( c , R ) \widetilde{F}(x)=
\begin{cases}
M_\delta\bigl(f_1(x),f_2(x)\bigr), & x\in B(c,R),\\
f_1(x), & x\in U\setminus B(c,R)
\end{cases} F ( x ) = { M δ ( f 1 ( x ) , f 2 ( x ) ) , f 1 ( x ) , x ∈ B ( c , R ) , x ∈ U ∖ B ( c , R ) is convex and belongs to $C^r(U).
Proof. • Proof of (a). Corollary B.18 , applied with p = r p=r p = r , gives convexity and C r C^r C r regularity of F = M δ ( f 1 , f 2 ) F=M_\delta(f_1,f_2) F = M δ ( f 1 , f 2 ) on the convex open set U U U . Equivalently, F = f 1 + ψ δ ( g ) F=f_1+\psi_\delta(g) F = f 1 + ψ δ ( g ) , with g = f 2 − f 1 g=f_2-f_1 g = f 2 − f 1 . The chain rule gives
∇ F = ∇ f 1 + ψ δ ′ ( g ) ∇ g = ∇ f 1 + a ( ∇ f 2 − ∇ f 1 ) by the definitions of a , g , = ( 1 − a ) ∇ f 1 + a ∇ f 2 . (47) \begin{aligned}
\nabla F &= \nabla f_1+\psi_\delta'(g)\nabla g\\
&=\nabla f_1+a(\nabla f_2-\nabla f_1) \quad\text{by the definitions of }a,g,\\
&=(1-a)\nabla f_1+a\nabla f_2. \tag*{(47)}
\end{aligned} ∇ F = ∇ f 1 + ψ δ ′ ( g ) ∇ g = ∇ f 1 + a ( ∇ f 2 − ∇ f 1 ) by the definitions of a , g , = ( 1 − a ) ∇ f 1 + a ∇ f 2 . (47)
∇ 2 F = ∇ 2 f 1 + ψ δ ′ ( g ) ∇ 2 g + ψ δ ′ ′ ( g ) ∇ g ∇ g T by the chain rule, = ∇ 2 f 1 + a ( ∇ 2 f 2 − ∇ 2 f 1 ) + ψ δ ′ ′ ( g ) ∇ g ∇ g T by the definitions of a , g , = ( 1 − a ) ∇ 2 f 1 + a ∇ 2 f 2 + ψ δ ′ ′ ( g ) ∇ g ∇ g T . (48) \begin{aligned}
\nabla^2F &= \nabla^2f_1+\psi_\delta'(g)\nabla^2g+\psi_\delta''(g)\nabla g\nabla g^{\mathsf T} &&\text{by the chain rule,}\\
&=\nabla^2f_1+a(\nabla^2f_2-\nabla^2f_1)+\psi_\delta''(g)\nabla g\nabla g^{\mathsf T} &&\text{by the definitions of }a,g,\\
&=(1-a)\nabla^2f_1+a\nabla^2f_2+\psi_\delta''(g)\nabla g\nabla g^{\mathsf T}. \tag*{(48)}
\end{aligned} ∇ 2 F = ∇ 2 f 1 + ψ δ ′ ( g ) ∇ 2 g + ψ δ ′′ ( g ) ∇ g ∇ g T = ∇ 2 f 1 + a ( ∇ 2 f 2 − ∇ 2 f 1 ) + ψ δ ′′ ( g ) ∇ g ∇ g T = ( 1 − a ) ∇ 2 f 1 + a ∇ 2 f 2 + ψ δ ′′ ( g ) ∇ g ∇ g T . by the chain rule, by the definitions of a , g , (48)
• Proof of (b). For any z ∈ R d z\in\mathbb{R}^d z ∈ R d , Eq. (B.20) gives
z T ( ∇ 2 F − K ) z = ( 1 − a ) z T ( ∇ 2 f 1 − K ) z + a z T ( ∇ 2 f 2 − K ) z + ψ δ ′ ′ ( g ) ⟨ ∇ g , z ⟩ 2 ≥ 0. \begin{aligned}
z^{\mathsf T}(\nabla^2F-K)z
&=(1-a)z^{\mathsf T}(\nabla^2f_1-K)z+az^{\mathsf T}(\nabla^2f_2-K)z\\
&\qquad+\psi_\delta''(g)\langle\nabla g,z\rangle^2\\
&\ge0.
\end{aligned} z T ( ∇ 2 F − K ) z = ( 1 − a ) z T ( ∇ 2 f 1 − K ) z + a z T ( ∇ 2 f 2 − K ) z + ψ δ ′′ ( g ) ⟨ ∇ g , z ⟩ 2 ≥ 0.
The last line uses the two assumed Hessian bounds and 0 ≤ a ≤ 1 0\le a\le1 0 ≤ a ≤ 1 , ψ δ ′ ′ ≥ 0 \psi_\delta''\ge0 ψ δ ′′ ≥ 0 from Lemma B.17 (b). Thus ∇ 2 F ⪰ K \nabla^2F\succeq K ∇ 2 F ⪰ K .
• Proof of (c). Lemma 4.2 gives F = f i F=f_i F = f i throughout the stated open region, since f i − f 3 − i ≥ δ f_i-f_{3-i}\ge\delta f i − f 3 − i ≥ δ there. Differentiating this equality gives agreement through order r r r .
• Proof of (d). Part (c) makes the two definitions agree on an open neighborhood of the joining sphere. Thus F ˉ ∈ C r ( U ) \bar F\in C^r(U) F ˉ ∈ C r ( U ) and its Hessian is positive semidefinite everywhere, using part (a) inside and convexity of f 1 f_1 f 1 outside. Lemma B.19 (b), applied on the convex set U U U , proves convexity. By continuity, the strict inequality f 1 − f 2 > δ f_1-f_2>\delta f 1 − f 2 > δ on the sphere suffices for the neighborhood hypothesis.
Supporting Proofs for the Uniformly Convex Base
Constructing and verifying the uniformly convex component
To give each of L L L phases the required Euclidean q t h q^{\mathrm{th}} q th -power local formula, we anchor phase j j j at a center c j c_j c j , as in Step 1. We use q = p + ν > 2 q=p+\nu>2 q = p + ν > 2 , μ > 0 \mu>0 μ > 0 , and consecutive centers satisfying
c 0 = 0 , c j + 1 = c j + h j , R j : = ∥ h j ∥ > 0 , c_0=0,\qquad c_{j+1}=c_j+h_j,\qquad R_j:=\lVert h_j\rVert>0, c 0 = 0 , c j + 1 = c j + h j , R j := ∥ h j ∥ > 0 ,
where the phase displacements h j h_j h j are mutually orthogonal, and their lengths will decrease geometrically. The first center is the origin, so we start with B 0 ( x ) = μ ∥ x ∥ q q B_0(x)=\frac{\mu\lVert x\rVert^q}{q} B 0 ( x ) = q μ ∥ x ∥ q . Let B j B_j B j denote the partial base after adding the local formula at c j c_j c j , and set B = B L B=B_L B = B L .
Requirements for the base
For the cancellation Eq. (E.5), we choose the base increment below. We also require uniform convexity and Hölder smoothness:
∇ B ( c j + 1 ) − ∇ B ( c j ) = μ R j q − 2 h j ( 0 ≤ j < L ) , \nabla B(c_{j+1})-\nabla B(c_j)=\mu R_j^{q-2}h_j \qquad(0\le j<L), ∇ B ( c j + 1 ) − ∇ B ( c j ) = μ R j q − 2 h j ( 0 ≤ j < L ) ,
D B ( y , x ) ≥ c B μ ∥ y − x ∥ q ( x , y ∈ R d ) , D_B(y,x)\ge c_B\mu\lVert y-x\rVert^q \qquad(x,y\in\mathbb{R}^d), D B ( y , x ) ≥ c B μ ∥ y − x ∥ q ( x , y ∈ R d ) ,
∥ ∇ p B ( y ) − ∇ p B ( x ) ∥ ≤ C B μ ∥ y − x ∥ ν ( x , y ∈ R d ) . (49) \lVert\nabla^pB(y)-\nabla^pB(x)\rVert\le C_B\mu\lVert y-x\rVert^\nu\qquad(x,y\in\mathbb{R}^d). \tag*{(49)} ∥ ∇ p B ( y ) − ∇ p B ( x )∥ ≤ C B μ ∥ y − x ∥ ν ( x , y ∈ R d ) . (49)
The positive constants c B c_B c B , C B C_B C B , independent of dimension and the number of phases, will be derived in the proofs. Lemma C.1 establishes the common construction. We then verify Hölder smoothness, preservation of oracle outputs, and uniform convexity in separate subsections.
Preserving earlier oracle outputs is a separate requirement. If the contribution of B j B_j B j has already been used at an evaluation point x s x_s x s , we require
B k ( x s ) = B j ( x s ) , ∇ r B k ( x s ) = ∇ r B j ( x s ) ( 1 ≤ r ≤ p , k > j ) . (50) B_k(x_s)=B_j(x_s),\qquad\nabla^rB_k(x_s)=\nabla^rB_j(x_s)\qquad(1\le r\le p,k>j). \tag*{(50)} B k ( x s ) = B j ( x s ) , ∇ r B k ( x s ) = ∇ r B j ( x s ) ( 1 ≤ r ≤ p , k > j ) . (50)
We will change B j B_j B j only where ∥ x − c j + 1 ∥ < θ R j \lVert x-c_{j+1}\rVert<\theta R_j ∥ x − c j + 1 ∥ < θ R j , with θ > 0 \theta>0 θ > 0 to be chosen later so that this ball misses the previous evaluation points x s x_s x s . All later changes will lie inside that same ball. Lemma C.3 (a) proves that this gives neighborhood agreement at every earlier evaluation point. Centers index phases and need not be evaluation points. Thus Eq. (C.2) does not prescribe the gradient at a new center that has not been evaluated.
Choose a local formula that can be repeated at successive centers
To obtain the first line of Eq. (C.1), we impose the following local gradient rule on a ball about c j c_j c j :
∇ B j ( x ) − ∇ B j ( c j ) = μ ∥ x − c j ∥ q − 2 ( x − c j ) . (51) \nabla B_j(x)-\nabla B_j(c_j)=\mu\lVert x-c_j\rVert^{q-2}(x-c_j). \tag*{(51)} ∇ B j ( x ) − ∇ B j ( c j ) = μ ∥ x − c j ∥ q − 2 ( x − c j ) . (51)
Lemma C.6 (a), proved in Appx. C.2, shows that this is equivalent to
B j ( x ) = B j ( c j ) + ⟨ ∇ B j ( c j ) , x − c j ⟩ + μ q ∥ x − c j ∥ q . (52) B_j(x)=B_j(c_j)+\langle\nabla B_j(c_j),x-c_j\rangle+\frac{\mu}{q}\lVert x-c_j\rVert^q. \tag*{(52)} B j ( x ) = B j ( c j ) + ⟨ ∇ B j ( c j ) , x − c j ⟩ + q μ ∥ x − c j ∥ q . (52)
When this ball contains c j + 1 c_{j+1} c j + 1 , evaluating the gradient there gives the desired increment μ R j q − 2 ( c j + 1 − c j ) \mu R_j^{q-2}(c_{j+1}-c_j) μ R j q − 2 ( c j + 1 − c j ) . Keeping this same expression for the next phase would also change the preceding block’s gradient component. Indeed, when x x x and c j + 1 c_{j+1} c j + 1 lie in the region where Eq. (C.4) holds and x − c j + 1 ⊥ c j + 1 − c j x-c_{j+1}\perp c_{j+1}-c_j x − c j + 1 ⊥ c j + 1 − c j , differentiation gives
∇ B j ( x ) − ∇ B j ( c j + 1 ) = μ ( ∥ c j + 1 − c j ∥ 2 + ∥ x − c j + 1 ∥ 2 ) q − 2 2 ( x − c j + 1 ) + μ [ ( ∥ c j + 1 − c j ∥ 2 + ∥ x − c j + 1 ∥ 2 ) q − 2 2 − ∥ c j + 1 − c j ∥ q − 2 ] ( c j + 1 − c j ) . (53) \begin{aligned}
\nabla B_j(x)-\nabla B_j(c_{j+1})
={}&\mu\left(\lVert c_{j+1}-c_j\rVert^2+\lVert x-c_{j+1}\rVert^2\right)^{\frac{q-2}{2}}(x-c_{j+1})\\
&+\mu\left[\left(\lVert c_{j+1}-c_j\rVert^2+\lVert x-c_{j+1}\rVert^2\right)^{\frac{q-2}{2}}-\lVert c_{j+1}-c_j\rVert^{q-2}\right](c_{j+1}-c_j). \tag*{(53)}
\end{aligned} ∇ B j ( x ) − ∇ B j ( c j + 1 ) = μ ( ∥ c j + 1 − c j ∥ 2 + ∥ x − c j + 1 ∥ 2 ) 2 q − 2 ( x − c j + 1 ) + μ [ ( ∥ c j + 1 − c j ∥ 2 + ∥ x − c j + 1 ∥ 2 ) 2 q − 2 − ∥ c j + 1 − c j ∥ q − 2 ] ( c j + 1 − c j ) . (53)
The coefficient on the last line is positive when x − c j + 1 ≠ 0 x-c_{j+1}\ne0 x − c j + 1 = 0 , because q > 2 q>2 q > 2 . Thus movement away from c j + 1 c_{j+1} c j + 1 changes the gradient component along h j = c j + 1 − c j h_j=c_{j+1}-c_j h j = c j + 1 − c j . To keep this component fixed during the next phase, we require
B j + 1 ( x ) = B j + 1 ( c j + 1 ) + ⟨ ∇ B j + 1 ( c j + 1 ) , x − c j + 1 ⟩ + μ q ∥ x − c j + 1 ∥ q . (54) B_{j+1}(x)=B_{j+1}(c_{j+1})+\langle\nabla B_{j+1}(c_{j+1}),x-c_{j+1}\rangle+\frac{\mu}{q}\lVert x-c_{j+1}\rVert^q. \tag*{(54)} B j + 1 ( x ) = B j + 1 ( c j + 1 ) + ⟨ ∇ B j + 1 ( c j + 1 ) , x − c j + 1 ⟩ + q μ ∥ x − c j + 1 ∥ q . (54)
To retain the gradient increment just obtained, we set
∇ B j + 1 ( c j + 1 ) = ∇ B j ( c j + 1 ) . (55) \nabla B_{j+1}(c_{j+1})=\nabla B_j(c_{j+1}). \tag*{(55)} ∇ B j + 1 ( c j + 1 ) = ∇ B j ( c j + 1 ) . (55)
We also set B j + 1 ( c j + 1 ) = B j ( c j + 1 ) + C j B_{j+1}(c_{j+1})=B_j(c_{j+1})+C_j B j + 1 ( c j + 1 ) = B j ( c j + 1 ) + C j , where C j > 0 C_j>0 C j > 0 will allow the two expressions to join. These choices give
B j + 1 ( x ) = B j ( c j + 1 ) + C j + ⟨ ∇ B j ( c j + 1 ) , x − c j + 1 ⟩ + μ q ∥ x − c j + 1 ∥ q . (56) B_{j+1}(x)=B_j(c_{j+1})+C_j+\langle\nabla B_j(c_{j+1}),x-c_{j+1}\rangle+\frac{\mu}{q}\lVert x-c_{j+1}\rVert^q. \tag*{(56)} B j + 1 ( x ) = B j ( c j + 1 ) + C j + ⟨ ∇ B j ( c j + 1 ) , x − c j + 1 ⟩ + q μ ∥ x − c j + 1 ∥ q . (56)
Lemma C.7 and Corollaries C.8–C.9 verify the resulting changes in value and derivatives. We next construct a function that has this formula near c j + 1 c_{j+1} c j + 1 while leaving B j B_j B j unchanged outside a small ball.
Define the replacement and its two radii
For each j j j , we extend the desired expression to a function on R d \mathbb{R}^d R d :
N j B ( x ) : = B j ( c j + 1 ) + C j + ⟨ ∇ B j ( c j + 1 ) , x − c j + 1 ⟩ + μ q ∥ x − c j + 1 ∥ q . N_j^B(x):=B_j(c_{j+1})+C_j+\langle\nabla B_j(c_{j+1}),x-c_{j+1}\rangle+\frac{\mu}{q}\lVert x-c_{j+1}\rVert^q. N j B ( x ) := B j ( c j + 1 ) + C j + ⟨ ∇ B j ( c j + 1 ) , x − c j + 1 ⟩ + q μ ∥ x − c j + 1 ∥ q .
Using the smooth maximum M δ j B M_{\delta_j^B} M δ j B from Eq. (B.1), define
B j + 1 ( x ) : = { M δ j B ( B j ( x ) , N j B ( x ) ) , ∥ x − c j + 1 ∥ < ρ j , B j ( x ) , ∥ x − c j + 1 ∥ ≥ ρ j . (57) B_{j+1}(x):=
\begin{cases}
M_{\delta_j^B}\left(B_j(x),N_j^B(x)\right), & \lVert x-c_{j+1}\rVert<\rho_j,\\
B_j(x), & \lVert x-c_{j+1}\rVert\ge\rho_j.
\end{cases} \tag*{(57)} B j + 1 ( x ) := { M δ j B ( B j ( x ) , N j B ( x ) ) , B j ( x ) , ∥ x − c j + 1 ∥ < ρ j , ∥ x − c j + 1 ∥ ≥ ρ j . (57)
The replacement must equal N j B N_j^B N j B near c j + 1 c_{j+1} c j + 1 and B j B_j B j near the sphere ∥ x − c j + 1 ∥ = ρ j \lVert x-c_{j+1}\rVert=\rho_j ∥ x − c j + 1 ∥ = ρ j , so the inside and outside formulas join smoothly. We therefore scale the support radius, value increase, and smoothing width by
ρ j : = θ R j , C j : = a l i f t θ 2 μ R j q , δ j B : = C j 4 . (58) \rho_j:=\theta R_j,\qquad C_j:=a_{\mathrm{lift}}\theta^2\mu R_j^q,\qquad\delta_j^B:=\frac{C_j}{4}. \tag*{(58)} ρ j := θ R j , C j := a lift θ 2 μ R j q , δ j B := 4 C j . (58)
The value comparisons derived in Lemma C.11 in Appx. C.2 determine the constants needed for these two equalities. They give the following explicit choices, all depending only on q q q :
a 0 : = 1 2 ( 3 4 ) q − 2 , a 1 : = q − 1 2 ( 5 4 ) q − 2 , (59) a_0:=\frac{1}{2}\left(\frac{3}{4}\right)^{q-2},\qquad
a_1:=\frac{q-1}{2}\left(\frac{5}{4}\right)^{q-2}, \tag*{(59)} a 0 := 2 1 ( 4 3 ) q − 2 , a 1 := 2 q − 1 ( 4 5 ) q − 2 , (59)
a l i f t : = a 0 8 , d ∗ : = a 0 32 = a l i f t 4 . (60) a_{\mathrm{lift}}:=\frac{a_0}{8},\qquad
d_*:=\frac{a_0}{32}=\frac{a_{\mathrm{lift}}}{4}. \tag*{(60)} a lift := 8 a 0 , d ∗ := 32 a 0 = 4 a lift . (60)
θ : = min { 1 16 , ( 3 q a 0 32 ) 1 q − 2 } . (61) \theta:=\min\left\{\frac{1}{16},\left(\frac{3qa_0}{32}\right)^{\frac{1}{q-2}}\right\}. \tag*{(61)} θ := min { 16 1 , ( 32 3 q a 0 ) q − 2 1 } . (61)
γ : = θ 3 a 0 32 a 1 . (62) \gamma:=\theta\sqrt{\frac{3a_0}{32a_1}}. \tag*{(62)} γ := θ 32 a 1 3 a 0 . (62)
Define
S j : = B ( c j + 1 , θ R j ) = { x : ∥ x − c j + 1 ∥ < θ R j } , S_j:=B(c_{j+1},\theta R_j)=\{x:\lVert x-c_{j+1}\rVert<\theta R_j\}, S j := B ( c j + 1 , θ R j ) = { x : ∥ x − c j + 1 ∥ < θ R j } ,
U j : = B ( c j + 1 , γ R j ) = { x : ∥ x − c j + 1 ∥ < γ R j } , U_j:=B(c_{j+1},\gamma R_j)=\{x:\lVert x-c_{j+1}\rVert<\gamma R_j\}, U j := B ( c j + 1 , γ R j ) = { x : ∥ x − c j + 1 ∥ < γ R j } ,
Δ j B : = B j + 1 − B j . \Delta_j^B:=B_{j+1}-B_j. Δ j B := B j + 1 − B j .
For one replacement, assume that B j B_j B j is globally convex and C p C^p C p and that Eq. (C.4) holds on an open set containing S ‾ j \overline{S}_j S j . Lemma C.1 (a) below verifies these hypotheses at every step.
The support ball S j S_j S j is the region where Eq. (C.9) uses M δ S j B M_{\delta_{S_j}^B} M δ S j B . The inner ball U j U_j U j is the smaller region where B j + 1 = N j B B_{j+1}=N_j^B B j + 1 = N j B . Lemma C.11 (a)–(b) gives
B j + 1 = N j B on U ‾ j , B j + 1 = B j where ∥ x − c j + 1 ∥ ≥ θ R j 2 . B_{j+1}=N_j^B\quad\text{on }\overline{U}_j,\qquad B_{j+1}=B_j\quad\text{where }\lVert x-c_{j+1}\rVert\geq\frac{\theta R_j}{2}. B j + 1 = N j B on U j , B j + 1 = B j where ∥ x − c j + 1 ∥ ≥ 2 θ R j .
Thus the two definitions agree on an open region containing the joining sphere. Lemmas C.12 and C.13 bound the derivatives and Hölder constant of Δ j B \Delta_j^B Δ j B . The following main results verify that the replacements can be repeated and that the completed base has all the properties in Eq. (C.1)–(C.2).
Repeat the construction and obtain the center recurrences
The next replacement must lie where the current base already has its exact local formula. We therefore place its support ball inside U j U_j U j and, to control later corrections for nearby pairs, require the smaller containment below.
To shrink the accuracy scales geometrically, we choose R j = R 0 η j R_j=R_0\eta^j R j = R 0 η j and require
S ‾ j + 1 ⊆ B ‾ ( c j + 1 , γ R j 2 ) ⊂ U j . \overline{S}_{j+1}\subseteq\overline{B}\left(c_{j+1},\frac{\gamma R_j}{2}\right)\subset U_j. S j + 1 ⊆ B ( c j + 1 , 2 γ R j ) ⊂ U j .
A pair less than γ R j 2 \frac{\gamma R_j}{2} 2 γ R j apart with one point outside U j U_j U j then misses this smaller ball. For x ∈ S ‾ j + 1 x\in\overline{S}_{j+1} x ∈ S j + 1 , the triangle inequality gives ∥ x − c j + 1 ∥ ≤ ( 1 + θ ) R j + 1 = ( 1 + θ ) η R j \lVert x-c_{j+1}\rVert\leq(1+\theta)R_{j+1}=(1+\theta)\eta R_j ∥ x − c j + 1 ∥ ≤ ( 1 + θ ) R j + 1 = ( 1 + θ ) η R j . Thus the required containment follows from
0 < η ≤ γ 2 ( 1 + θ ) . (63) 0<\eta\leq\frac{\gamma}{2(1+\theta)}. \tag*{(63)} 0 < η ≤ 2 ( 1 + θ ) γ . (63)
Lemma C.11 gives γ < θ 2 \gamma<\frac{\theta}{2} γ < 2 θ and θ ≤ 1 16 \theta\leq\frac{1}{16} θ ≤ 16 1 , so this already implies η < 1 64 < 1 8 \eta<\frac{1}{64}<\frac{1}{8} η < 64 1 < 8 1 , as needed for geometric tails and center spacing. No additional minimum is required.
Lemma C.1 (Nested replacements preserve earlier centers and give explicit recurrences). Assume R j = R 0 η j R_j=R_0\eta^j R j = R 0 η j and Eq. (C.15), and start from B 0 ( x ) = μ ∥ x ∥ q q B_0(x)=\frac{\mu\lVert x\rVert^q}{q} B 0 ( x ) = q μ ∥ x ∥ q . For the functions defined recursively by Eq. (C.9), define
B : = B L , d j : = ∇ B ( c j ) , ω j : = B ( c j ) , β q : = 1 q + a l i f t θ 2 . B:=B_L,\qquad d_j:=\nabla B(c_j),\qquad\omega_j:=B(c_j),\qquad\beta_q:=\frac{1}{q}+a_{\mathrm{lift}}\theta^2. B := B L , d j := ∇ B ( c j ) , ω j := B ( c j ) , β q := q 1 + a lift θ 2 . (a) (i) Local formula and containment of the next support. For 1 ≤ j ≤ L 1\leq j\leq L 1 ≤ j ≤ L ,
B j ( x ) = B j ( c j ) + ⟨ ∇ B j ( c j ) , x − c j ⟩ + μ q ∥ x − c j ∥ q B_j(x)=B_j(c_j)+\langle\nabla B_j(c_j),x-c_j\rangle+\frac{\mu}{q}\lVert x-c_j\rVert^q B j ( x ) = B j ( c j ) + ⟨ ∇ B j ( c j ) , x − c j ⟩ + q μ ∥ x − c j ∥ q whenever ∥ x − c j ∥ ≤ γ R j − 1 \lVert x-c_j\rVert\leq\gamma R_{j-1} ∥ x − c j ∥ ≤ γ R j − 1 .
For j = 0 j=0 j = 0 , the same function identity holds on all of R d \mathbb{R}^d R d . For 1 ≤ j < L 1\leq j<L 1 ≤ j < L ,
S ‾ j = { x : ∥ x − c j + 1 ∥ ≤ θ R j } \overline{S}_j=\{x:\lVert x-c_{j+1}\rVert\leq\theta R_j\} S j = { x : ∥ x − c j + 1 ∥ ≤ θ R j } ⊆ B ‾ ( c j , γ R j − 1 2 ) \subseteq\overline{B}\left(c_j,\frac{\gamma R_{j-1}}{2}\right) ⊆ B ( c j , 2 γ R j − 1 ) ⊂ { x : ∥ x − c j ∥ < γ R j − 1 } = U j − 1 . \subset\{x:\lVert x-c_j\rVert<\gamma R_{j-1}\}=U_{j-1}. ⊂ { x : ∥ x − c j ∥ < γ R j − 1 } = U j − 1 . Thus the displayed formula for B j B_j B j holds on an open set containing S ‾ j \overline{S}_j S j , as assumed in Lemma C.11 .
(ii) Global C p C^p C p regularity and convexity.
B j ∈ C p ( R d ) , B j is convex ( 0 ≤ j ≤ L ) . B_j \in C^p\left(\mathbb{R}^d\right), \qquad B_j\text{ is convex} \qquad(0 \le j \le L). B j ∈ C p ( R d ) , B j is convex ( 0 ≤ j ≤ L ) .
The Hölder bound is proved in Lemma C.2.
(iii) Preservation around every earlier center. For every 0 ≤ i < L 0 \le i < L 0 ≤ i < L , the functions agree on the open ball with center c i c_i c i and radius ( 1 − θ ) R i > 0 (1-\theta)R_i>0 ( 1 − θ ) R i > 0 :
B k ( x ) = B i ( x ) ( i ≤ k ≤ L , ∥ x − c i ∥ < ( 1 − θ ) R i ) . B_k(x)=B_i(x) \qquad(i\le k\le L,\ \lVert x-c_i\rVert<(1-\theta)R_i). B k ( x ) = B i ( x ) ( i ≤ k ≤ L , ∥ x − c i ∥ < ( 1 − θ ) R i ) .
(iv) Preservation outside the replacement ball. For 0 ≤ i < L 0\le i<L 0 ≤ i < L and i ≤ k ≤ L i\le k\le L i ≤ k ≤ L ,
B k ( x ) = B i ( x ) , ∇ r B k ( x ) = ∇ r B i ( x ) ( x ∉ S ‾ i , 1 ≤ r ≤ p ) . B_k(x)=B_i(x), \qquad\nabla^r B_k(x)=\nabla^r B_i(x) \qquad(x\notin\overline{S}_i,\ 1\le r\le p). B k ( x ) = B i ( x ) , ∇ r B k ( x ) = ∇ r B i ( x ) ( x ∈ / S i , 1 ≤ r ≤ p ) .
The partial base B j B_j B j depends only on c 0 , … , c j c_0,\ldots,c_j c 0 , … , c j .
(b) The center recurrences are
d j + 1 − d j = μ R j q − 2 h j , ω j + 1 − ω j = ⟨ d j , h j ⟩ + β q μ R j q . (64) d_{j+1}-d_j=\mu R_j^{q-2}h_j,\qquad\omega_{j+1}-\omega_j=\langle d_j,h_j\rangle+\beta_q\mu R_j^q. \tag*{(64)} d j + 1 − d j = μ R j q − 2 h j , ω j + 1 − ω j = ⟨ d j , h j ⟩ + β q μ R j q . (64)
(c) The center values and derivatives are
d j = μ ∑ i < j R i q − 2 h i , ω j = β q μ ∑ i < j R i q , ∇ k B ( c j ) = 0 ( 2 ≤ k ≤ p ) . (65) d_j=\mu\sum_{i<j}R_i^{q-2}h_i,\qquad\omega_j=\beta_q\mu\sum_{i<j}R_i^q,\qquad\nabla^k B(c_j)=0\quad(2\le k\le p). \tag*{(65)} d j = μ i < j ∑ R i q − 2 h i , ω j = β q μ i < j ∑ R i q , ∇ k B ( c j ) = 0 ( 2 ≤ k ≤ p ) . (65)
(d) The final function satisfies
B ( x ) = B ( c j ) + ⟨ ∇ B ( c j ) , x − c j ⟩ + μ q ∥ x − c j ∥ q . (66) B(x)=B(c_j)+\langle\nabla B(c_j),x-c_j\rangle+\frac{\mu}{q}\lVert x-c_j\rVert^q. \tag*{(66)} B ( x ) = B ( c j ) + ⟨ ∇ B ( c j ) , x − c j ⟩ + q μ ∥ x − c j ∥ q . (66)
For L ≥ 1 L\ge1 L ≥ 1 , this identity holds on the following explicit balls:
{ ∥ x − c j ∥ < ( 1 − θ ) R j , 0 ≤ j < L , ∥ x − c L ∥ ≤ γ R L − 1 , j = L . \begin{cases}
\lVert x-c_j\rVert<(1-\theta)R_j, & 0\le j<L,\\
\lVert x-c_L\rVert\le\gamma R_{L-1}, & j=L.
\end{cases} { ∥ x − c j ∥ < ( 1 − θ ) R j , ∥ x − c L ∥ ≤ γ R L − 1 , 0 ≤ j < L , j = L .
For L = 0 L=0 L = 0 , it holds on all of R d \mathbb{R}^d R d .
Proof. • Proof of (a). Proof of (i) and (ii). Lemma C.11 gives the two exact formulas for one replacement. To repeat the construction, we place each subsequent replacement inside the region where B j + 1 = N B B_{j+1}=N^B B j + 1 = N B . This containment depends only on the centers, radii, and Eq. (C.15), so we can verify it before constructing the functions. We then prove the local formulas and global convexity and C p C^p C p regularity together by induction.
Geometric preparation for (i). To apply Lemma C.11 to a replacement B j + 1 ↦ B j + 2 B_{j+1}\mapsto B_{j+2} B j + 1 ↦ B j + 2 , the local formula for B j + 1 B_{j+1} B j + 1 must hold on an open set containing the next closed support S ‾ j + 1 \overline{S}_{j+1} S j + 1 . The formula in (i), with j + 1 j+1 j + 1 in place of j j j , is required on U j U_j U j . Thus the geometric condition we need is S ‾ j + 1 ⊂ U j \overline{S}_{j+1}\subset U_j S j + 1 ⊂ U j .
Step 1. Identify the two balls and their centers. Fix 0 ≤ j < L − 1 0\le j<L-1 0 ≤ j < L − 1 . By the definitions of S ‾ j + 1 \overline{S}_{j+1} S j + 1 and U j U_j U j ,
S ‾ j + 1 = { x : ∥ x − c j + 2 ∥ ≤ θ R j + 1 } , center c j + 2 , radius θ R j + 1 , \overline{S}_{j+1}=\{x:\lVert x-c_{j+2}\rVert\le\theta R_{j+1}\},\qquad\text{center }c_{j+2},\ \text{radius }\theta R_{j+1}, S j + 1 = { x : ∥ x − c j + 2 ∥ ≤ θ R j + 1 } , center c j + 2 , radius θ R j + 1 ,
U j = { x : ∥ x − c j + 1 ∥ < γ R j } , center c j + 1 , radius γ R j . U_j=\{x:\lVert x-c_{j+1}\rVert<\gamma R_j\},\qquad\text{center }c_{j+1},\ \text{radius }\gamma R_j. U j = { x : ∥ x − c j + 1 ∥ < γ R j } , center c j + 1 , radius γ R j .
Their centers are separated by
∥ c j + 2 − c j + 1 ∥ = R j + 1 = η R j . \lVert c_{j+2}-c_{j+1}\rVert= R_{j+1} = \eta R_j. ∥ c j + 2 − c j + 1 ∥ = R j + 1 = η R j .
For an arbitrary x ∈ S ‾ j + 1 x \in\overline{S}_{j+1} x ∈ S j + 1 , we know its distance from c j + 2 c_{j+2} c j + 2 and need an upper bound on its distance from c j + 1 c_{j+1} c j + 1 :
∥ x − c j + 2 ∥ ≤ θ R j + 1 ⏟ known from membership in S ‾ j + 1 ⟹ ∥ x − c j + 1 ∥ ≤ γ R j 2 ⏟ bound to prove \underbrace{\lVert x-c_{j+2}\rVert\le\theta R_{j+1}}_{\text{known from membership in }\overline{S}_{j+1}}
\quad\Longrightarrow\quad
\underbrace{\lVert x-c_{j+1}\rVert\le\frac{\gamma R_j}{2}}_{\text{bound to prove}} known from membership in S j + 1 ∥ x − c j + 2 ∥ ≤ θ R j + 1 ⟹ bound to prove ∥ x − c j + 1 ∥ ≤ 2 γ R j
Step 2. Bound the distance to c j + 1 c_{j+1} c j + 1 and obtain the containment. The vector identity
x − c j + 1 = ( x − c j + 2 ) + ( c j + 2 − c j + 1 ) x-c_{j+1}=(x-c_{j+2})+(c_{j+2}-c_{j+1}) x − c j + 1 = ( x − c j + 2 ) + ( c j + 2 − c j + 1 )
and the triangle inequality give
∥ x − c j + 1 ∥ ≤ ∥ x − c j + 2 ∥ ⏟ ≤ θ R j + 1 + ∥ c j + 2 − c j + 1 ∥ ⏟ = R j + 1 ≤ θ R j + 1 + R j + 1 = ( 1 + θ ) R j + 1 = ( 1 + θ ) η R j ≤ γ R j 2 . \begin{aligned}
\lVert x-c_{j+1}\rVert
&\le\underbrace{\lVert x-c_{j+2}\rVert}_{\le\theta R_{j+1}}
+\underbrace{\lVert c_{j+2}-c_{j+1}\rVert}_{=R_{j+1}}\\
&\le\theta R_{j+1}+R_{j+1}\\
&=(1+\theta)R_{j+1}\\
&=(1+\theta)\eta R_j\\
&\le\frac{\gamma R_j}{2}.
\end{aligned} ∥ x − c j + 1 ∥ ≤ ≤ θ R j + 1 ∥ x − c j + 2 ∥ + = R j + 1 ∥ c j + 2 − c j + 1 ∥ ≤ θ R j + 1 + R j + 1 = ( 1 + θ ) R j + 1 = ( 1 + θ ) η R j ≤ 2 γ R j .
where the last line uses Eq. (C.15). Since x x x was arbitrary, the whole ball centered at c j + 2 c_{j+2} c j + 2 lies in the displayed ball centered at c j + 1 c_{j+1} c j + 1 . The remaining inclusions compare radii around the same center, using γ R j 2 < γ R j < θ R j \frac{\gamma R_j}{2}<\gamma R_j<\theta R_j 2 γ R j < γ R j < θ R j :
S ‾ j + 1 = B ‾ ( c j + 2 , θ R j + 1 ) ⊂ B ‾ ( c j + 1 , γ R j 2 ) ⊂ B ( c j + 1 , γ R j ) = U j ⊂ B ( c j + 1 , θ R j ) = S j . (67) \begin{aligned}
\overline{S}_{j+1}
&=\overline{B}(c_{j+2},\theta R_{j+1})\\
&\subset\overline{B}\left(c_{j+1},\frac{\gamma R_j}{2}\right)\\
&\subset B(c_{j+1},\gamma R_j)=U_j\\
&\subset B(c_{j+1},\theta R_j)=S_j. \tag*{(67)}
\end{aligned} S j + 1 = B ( c j + 2 , θ R j + 1 ) ⊂ B ( c j + 1 , 2 γ R j ) ⊂ B ( c j + 1 , γ R j ) = U j ⊂ B ( c j + 1 , θ R j ) = S j . (67)
Replacing j + 1 j+1 j + 1 by j j j gives S ‾ j ⊂ U j − 1 \overline{S}_j\subset U_{j-1} S j ⊂ U j − 1 for 1 ≤ j < L 1\le j<L 1 ≤ j < L , as asserted in (i).
Joint induction for the local formula in (i) and the properties in (ii).
1. Base case: j = 0 j=0 j = 0 . By definition, B 0 ( x ) = μ q ∥ x ∥ q B_0(x)=\frac{\mu}{q}\lVert x\rVert^q B 0 ( x ) = q μ ∥ x ∥ q and c 0 = 0 c_0=0 c 0 = 0 . Lemma B.16 (b) in Appx. B gives B 0 ( c 0 ) = 0 B_0(c_0)=0 B 0 ( c 0 ) = 0 and ∇ B 0 ( c 0 ) = 0 \nabla B_0(c_0)=0 ∇ B 0 ( c 0 ) = 0 . Hence
B 0 ( x ) = B 0 ( c 0 ) + ⟨ ∇ B 0 ( c 0 ) , x − c 0 ⟩ + μ q ∥ x − c 0 ∥ q ( x ∈ R d ) , B_0(x)=B_0(c_0)+\langle\nabla B_0(c_0),x-c_0\rangle+\frac{\mu}{q}\lVert x-c_0\rVert^q
\qquad(x\in\mathbb{R}^d), B 0 ( x ) = B 0 ( c 0 ) + ⟨ ∇ B 0 ( c 0 ) , x − c 0 ⟩ + q μ ∥ x − c 0 ∥ q ( x ∈ R d ) ,
which proves the local-formula claim (i) at j = 0 j=0 j = 0 . Parts (b) and (c) of the same lemma give
B 0 ∈ C p ( R d ) , B 0 is convex , B_0\in C^p(\mathbb{R}^d),\qquad B_0\text{ is convex}, B 0 ∈ C p ( R d ) , B 0 is convex ,
proving (ii) at $j=0. 2. Inductive hypothesis: fix 0 ≤ j < L 0 \le j < L 0 ≤ j < L and assume both claims for B j B_j B j . For (i), assume
B j ( x ) = B j ( c j ) + ⟨ ∇ B j ( c j ) , x − c j ⟩ + μ q ∥ x − c j ∥ q , B_j(x) = B_j(c_j) + \langle\nabla B_j(c_j),x-c_j\rangle+ \frac{\mu}{q}\lVert x-c_j\rVert^q, B j ( x ) = B j ( c j ) + ⟨ ∇ B j ( c j ) , x − c j ⟩ + q μ ∥ x − c j ∥ q ,
on all of R d \mathbb{R}^d R d if j = 0 j=0 j = 0 , and on ∥ x − c j ∥ ≤ γ R j − 1 \lVert x-c_j\rVert\le\gamma R_{j-1} ∥ x − c j ∥ ≤ γ R j − 1 if j ≥ 1 j \ge1 j ≥ 1 . For (ii), assume
B j ∈ C p ( R d ) , B j is convex . B_j \in C^p(\mathbb{R}^d), \qquad B_j\text{ is convex}. B j ∈ C p ( R d ) , B j is convex .
These are assumptions at index j j j . The induction step must establish both conclusions at index j + 1 j+1 j + 1 .
3. Induction step: construct B j + 1 B_{j+1} B j + 1 and prove both claims.
(a) Verify the assumptions of Lemma C.11. The inductive hypothesis (ii) supplies global convexity and C p C^p C p regularity of B j B_j B j . For j ≥ 1 j \ge1 j ≥ 1 , the inductive hypothesis (i) supplies its local formula on the open set
U j − 1 = { x : ∥ x − c j ∥ < γ R j − 1 } . U_{j-1}=\{x:\lVert x-c_j\rVert<\gamma R_{j-1}\}. U j − 1 = { x : ∥ x − c j ∥ < γ R j − 1 } .
The geometric containment Eq. (C.19), with j − 1 j-1 j − 1 in place of j j j , gives
S j ‾ = B ‾ ( c j + 1 , θ R j ) ⊂ U j − 1 . \overline{S_j}=\overline{B}(c_{j+1},\theta R_j)\subset U_{j-1}. S j = B ( c j + 1 , θ R j ) ⊂ U j − 1 .
Thus the local formula holds on an open set containing the entire closed support used for B j ↦ B j + 1 B_j\mapsto B_{j+1} B j ↦ B j + 1 . For j = 0 j=0 j = 0 , the global formula gives the same requirement. All assumptions of Lemma C.11 are therefore verified using the hypotheses at index j j j and the independently proved geometric containment.
(b) Prove (i) for B j + 1 B_{j+1} B j + 1 . Define B j + 1 B_{j+1} B j + 1 by Eq. (C.9). Lemma C.11(b) gives B j + 1 = N j B B_{j+1}=N_j^B B j + 1 = N j B on ∥ x − c j + 1 ∥ ≤ γ R j \lVert x-c_{j+1}\rVert\le\gamma R_j ∥ x − c j + 1 ∥ ≤ γ R j . Part (c) of that lemma identifies its center value and gradient, yielding
B j + 1 ( x ) = N j B ( x ) = B j + 1 ( c j + 1 ) + ⟨ ∇ B j + 1 ( c j + 1 ) , x − c j + 1 ⟩ + μ q ∥ x − c j + 1 ∥ q ( ∥ x − c j + 1 ∥ ≤ γ R j ) . B_{j+1}(x)=N_j^B(x)=B_{j+1}(c_{j+1})+\langle\nabla B_{j+1}(c_{j+1}),x-c_{j+1}\rangle
+\frac{\mu}{q}\lVert x-c_{j+1}\rVert^q
\qquad(\lVert x-c_{j+1}\rVert\le\gamma R_j). B j + 1 ( x ) = N j B ( x ) = B j + 1 ( c j + 1 ) + ⟨ ∇ B j + 1 ( c j + 1 ) , x − c j + 1 ⟩ + q μ ∥ x − c j + 1 ∥ q (∥ x − c j + 1 ∥ ≤ γ R j ) .
This is exactly the local formula in (i) at index j + 1 j+1 j + 1 . When j < L − 1 j<L-1 j < L − 1 , Eq. (C.19) also gives S j + 1 ‾ ⊂ U j \overline{S_{j+1}}\subset U_j S j + 1 ⊂ U j , so this formula is available on an open set containing the following closed support.
(c) Prove (ii) for B j + 1 B_{j+1} B j + 1 . Apply Lemma C.11(d) and part (e), whose assumptions were verified in (a). They give
B j + 1 ∈ C p ( R d ) , B j + 1 is convex , B_{j+1}\in C^p(\mathbb{R}^d),\qquad B_{j+1}\text{ is convex}, B j + 1 ∈ C p ( R d ) , B j + 1 is convex ,
which proves (ii) at index j + 1 j+1 j + 1 .
The base case establishes both claims at j = 0 j=0 j = 0 , and the induction step establishes both at j + 1 j+1 j + 1 from both at j j j . They therefore hold for every 0 ≤ j ≤ L 0\le j\le L 0 ≤ j ≤ L . Together with the geometric containment already proved, this completes (i) and (ii).
Proof of (iii) and (iv): preservation of earlier formulas. We fix 0 ≤ i < L 0\le i<L 0 ≤ i < L . Since the later replacement balls are contained in S i S_i S i by Eq. (C.19), every later modification vanishes on R d ∖ S i ‾ \mathbb{R}^d\setminus\overline{S_i} R d ∖ S i , an open set. For x x x in this set and i ≤ k ≤ L i\le k\le L i ≤ k ≤ L , the definition Eq. (C.9) therefore gives
B k ( x ) − B i ( x ) = ∑ j = i k − 1 ( B j + 1 ( x ) − B j ( x ) ) = 0. \begin{aligned}
B_k(x)-B_i(x)=\sum_{j=i}^{k-1}(B_{j+1}(x)-B_j(x)) \\
&=0.
\end{aligned} B k ( x ) − B i ( x ) = j = i ∑ k − 1 ( B j + 1 ( x ) − B j ( x )) = 0.
Equality of the functions on this open set gives equality of their derivatives, since B i , B k ∈ C p B_i, B_k \in C^p B i , B k ∈ C p by (ii):
∇ r B k ( x ) = ∇ r B i ( x ) ( x ∉ S i ‾ , 1 ≤ r ≤ p , i ≤ k ≤ L ) . \nabla^r B_k(x)=\nabla^r B_i(x) \qquad(x \notin\overline{S_i},\ 1\le r\le p,\ i\le k\le L). ∇ r B k ( x ) = ∇ r B i ( x ) ( x ∈ / S i , 1 ≤ r ≤ p , i ≤ k ≤ L ) .
The same agreement leaves a neighborhood of every earlier center unchanged. Indeed, if ∥ x − c i ∥ < ( 1 − θ ) R i \lVert x-c_i\rVert< (1-\theta)R_i ∥ x − c i ∥ < ( 1 − θ ) R i , then
∥ x − c i + 1 ∥ ≥ R i − ∥ x − c i ∥ by the reverse triangle inequality, > θ R i , \begin{aligned}
\lVert x-c_{i+1}\rVert\ge R_i-\lVert x-c_i\rVert\quad\text{by the reverse triangle inequality,} \\
&> \theta R_i,
\end{aligned} ∥ x − c i + 1 ∥ ≥ R i − ∥ x − c i ∥ by the reverse triangle inequality, > θ R i ,
so x ∉ S i ‾ x\notin\overline{S_i} x ∈ / S i . This proves (iii) and (iv), including the preservation in Lemma 4.3 (d). The recursion uses only its previous base and next center, so B j B_j B j depends only on c 0 , … , c j c_0,\ldots,c_j c 0 , … , c j .
Proof of (b). Each replacement retains its endpoint gradient and adds C j C_j C j to its value. To obtain the center recurrences, we evaluate the local formula at c j + 1 c_{j+1} c j + 1 and then account for this value increase. The preservation just proved identifies the partial and final center data: for 0 ≤ j < L 0\le j<L 0 ≤ j < L , part (a)(iii), with i = j i=j i = j and k = L k=L k = L , gives
B j ( x ) = B L ( x ) = B ( x ) whenever ∥ x − c j ∥ < ( 1 − θ ) R j . B_j(x)=B_L(x)=B(x) \qquad\text{whenever }\lVert x-c_j\rVert< (1-\theta)R_j. B j ( x ) = B L ( x ) = B ( x ) whenever ∥ x − c j ∥ < ( 1 − θ ) R j .
Thus the equality holds throughout the open ball B ( c j , ( 1 − θ ) R j ) B(c_j,(1-\theta)R_j) B ( c j , ( 1 − θ ) R j ) , whose radius is positive. Evaluating and differentiating this equality at its center c j c_j c j yields
B j ( c j ) = B ( c j ) = ω j , B_j(c_j)=B(c_j)=\omega_j, B j ( c j ) = B ( c j ) = ω j ,
∇ B j ( c j ) = ∇ B ( c j ) = d j . \nabla B_j(c_j)=\nabla B(c_j)=d_j. ∇ B j ( c j ) = ∇ B ( c j ) = d j .
These identities compare the partial function B j B_j B j and the completed function B = B L B=B_L B = B L at the same center c j c_j c j . Substituting them into the local formula in part (a)(i) gives, on its stated region,
B j ( x ) = B j ( c j ) + ⟨ ∇ B j ( c j ) , x − c j ⟩ + μ q ∥ x − c j ∥ q = ω j + ⟨ d j , x − c j ⟩ + μ q ∥ x − c j ∥ q . \begin{aligned}
B_j(x)=B_j(c_j)+\langle\nabla B_j(c_j),x-c_j\rangle+\frac{\mu}{q}\lVert x-c_j\rVert^q \\
&=\omega_j+\langle d_j,x-c_j\rangle+\frac{\mu}{q}\lVert x-c_j\rVert^q.
\end{aligned} B j ( x ) = B j ( c j ) + ⟨ ∇ B j ( c j ) , x − c j ⟩ + q μ ∥ x − c j ∥ q = ω j + ⟨ d j , x − c j ⟩ + q μ ∥ x − c j ∥ q .
The point c j + 1 = c j + h j c_{j+1}=c_j+h_j c j + 1 = c j + h j belongs to the local-formula region: for j ≥ 1 j\ge1 j ≥ 1 , Eq. (C.15) gives R j = η R j − 1 < γ R j − 1 R_j=\eta R_{j-1}<\gamma R_{j-1} R j = η R j − 1 < γ R j − 1 , and for j = 0 j=0 j = 0 the formula is global. Evaluating and differentiating it there gives
B j ( c j + 1 ) = ω j + ⟨ d j , h j ⟩ + μ q R j q , B_j(c_{j+1})=\omega_j+\langle d_j,h_j\rangle+\frac{\mu}{q}R_j^q, B j ( c j + 1 ) = ω j + ⟨ d j , h j ⟩ + q μ R j q ,
∇ B j ( c j + 1 ) = d j + μ ∥ h j ∥ q − 2 h j = d j + μ R j q − 2 h j . \nabla B_j(c_{j+1})=d_j+\mu\lVert h_j\rVert^{q-2}h_j=d_j+\mu R_j^{q-2}h_j. ∇ B j ( c j + 1 ) = d j + μ ∥ h j ∥ q − 2 h j = d j + μ R j q − 2 h j .
Lemma C.11 (c) and the neighborhood preservation in part (a)(iii) now give the following identities. At j + 1 = L j+1=L j + 1 = L , use B = B L B=B_L B = B L in place of that preservation property.
d j + 1 = ∇ B j + 1 ( c j + 1 ) = ∇ B j ( c j + 1 ) = d j + μ R j q − 2 h j , d_{j+1}=\nabla B_{j+1}(c_{j+1})=\nabla B_j(c_{j+1})=d_j+\mu R_j^{q-2}h_j, d j + 1 = ∇ B j + 1 ( c j + 1 ) = ∇ B j ( c j + 1 ) = d j + μ R j q − 2 h j ,
ω j + 1 = B j + 1 ( c j + 1 ) = B j ( c j + 1 ) + C j = ω j + ⟨ d j , h j ⟩ + μ q R j q + α l i f t θ 2 μ R j q = ω j + ⟨ d j , h j ⟩ + ( 1 q + α l i f t θ 2 ) μ R j q = ω j + ⟨ d j , h j ⟩ + β q μ R j q . \begin{aligned}
\omega_{j+1}&=B_{j+1}(c_{j+1})=B_j(c_{j+1})+C_j\\
&=\omega_j+\langle d_j,h_j\rangle+\frac{\mu}{q}R_j^q+\alpha_{\mathrm{lift}}\theta^2\mu R_j^q\\
&=\omega_j+\langle d_j,h_j\rangle+\left(\frac{1}{q}+\alpha_{\mathrm{lift}}\theta^2\right)\mu R_j^q\\
&=\omega_j+\langle d_j,h_j\rangle+\beta_q\mu R_j^q.
\end{aligned} ω j + 1 = B j + 1 ( c j + 1 ) = B j ( c j + 1 ) + C j = ω j + ⟨ d j , h j ⟩ + q μ R j q + α lift θ 2 μ R j q = ω j + ⟨ d j , h j ⟩ + ( q 1 + α lift θ 2 ) μ R j q = ω j + ⟨ d j , h j ⟩ + β q μ R j q .
These are exactly the two recurrences in Eq. (C.16).
Proof of (c). The initial value and gradient remain zero, so d 0 = 0 d_0=0 d 0 = 0 and ω 0 = 0 \omega_0=0 ω 0 = 0 . Since successive gradient increments are parallel to h i h_i h i , orthogonality gives ⟨ d j , h j ⟩ = 0 \langle d_j,h_j\rangle=0 ⟨ d j , h j ⟩ = 0 for 0 ≤ j < L 0\le j<L 0 ≤ j < L . We obtain the explicit center data by telescoping the recurrences in Eq. (C.16):
d j = d 0 + ∑ i = 0 j − 1 ( d i + 1 − d i ) = μ ∑ i < j R i q − 2 h i , d_j=d_0+\sum_{i=0}^{j-1}(d_{i+1}-d_i)=\mu\sum_{i<j}R_i^{q-2}h_i, d j = d 0 + i = 0 ∑ j − 1 ( d i + 1 − d i ) = μ i < j ∑ R i q − 2 h i ,
⟨ d j , h j ⟩ = μ ∑ i < j R i q − 2 ⟨ h i , h j ⟩ = 0 , \langle d_j,h_j\rangle=\mu\sum_{i<j}R_i^{q-2}\langle h_i,h_j\rangle=0, ⟨ d j , h j ⟩ = μ i < j ∑ R i q − 2 ⟨ h i , h j ⟩ = 0 ,
ω j = ω 0 + ∑ i < j ( ω i + 1 − ω i ) = ∑ i < j ( ⟨ d i , h i ⟩ + β q μ R i q ) = β q μ ∑ i < j R i q . \begin{aligned}
\omega_j=\omega_0+\sum_{i<j}(\omega_{i+1}-\omega_i) \\
&=\sum_{i<j}\left(\langle d_i,h_i\rangle+\beta_q\mu R_i^q\right)=\beta_q\mu\sum_{i<j}R_i^q.
\end{aligned} ω j = ω 0 + i < j ∑ ( ω i + 1 − ω i ) = i < j ∑ ( ⟨ d i , h i ⟩ + β q μ R i q ) = β q μ i < j ∑ R i q .
For 1 ≤ j ≤ L 1\le j\le L 1 ≤ j ≤ L , Lemma C.11 (c) gives ∇ k B j ( c j ) = 0 \nabla^kB_j(c_j)=0 ∇ k B j ( c j ) = 0 for 2 ≤ k ≤ p 2\le k\le p 2 ≤ k ≤ p . If j < L j<L j < L , part (a)(iii) gives B = B j B=B_j B = B j on a neighborhood of c j c_j c j , so
∇ k B ( c j ) = ∇ k B j ( c j ) = 0 ( 2 ≤ k ≤ p ) . \nabla^kB(c_j)=\nabla^kB_j(c_j)=0\qquad(2\le k\le p). ∇ k B ( c j ) = ∇ k B j ( c j ) = 0 ( 2 ≤ k ≤ p ) .
For j = L j=L j = L , the same equality follows from B = B L B=B_L B = B L . At c 0 c_0 c 0 , use Lemma B.16 (b) for B 0 B_0 B 0 and the same neighborhood preservation. This proves every identity in Eq. (C.17).
Earlier centers: 0 ≤ j < L 0\le j<L 0 ≤ j < L . For 1 ≤ j < L 1\le j<L 1 ≤ j < L , Eq. (C.15) gives
( 1 − θ ) R j = ( 1 − θ ) η R j − 1 < γ R j − 1 . (1-\theta)R_j=(1-\theta)\eta R_{j-1}<\gamma R_{j-1}. ( 1 − θ ) R j = ( 1 − θ ) η R j − 1 < γ R j − 1 .
Thus the formula in part (a)(i) holds on ∥ x − c j ∥ < ( 1 − θ ) R j \lVert x-c_j\rVert<(1-\theta)R_j ∥ x − c j ∥ < ( 1 − θ ) R j ; for j = 0 j=0 j = 0 , that formula holds globally. On this ball, part (a)(iii) gives B = B j B=B_j B = B j , hence their center values and gradients agree by part (a)(ii). Substitution into part (a)(i) yields
B ( x ) − B ( c j ) − ⟨ ∇ B ( c j ) , x − c j ⟩ = B j ( x ) − B j ( c j ) − ⟨ ∇ B j ( c j ) , x − c j ⟩ = μ q ∥ x − c j ∥ q , \begin{aligned}
B(x)-B(c_j)-\langle\nabla B(c_j),x-c_j\rangle\\
&=B_j(x)-B_j(c_j)-\langle\nabla B_j(c_j),x-c_j\rangle\\
&=\frac{\mu}{q}\lVert x-c_j\rVert^q,
\end{aligned} B ( x ) − B ( c j ) − ⟨ ∇ B ( c j ) , x − c j ⟩ = B j ( x ) − B j ( c j ) − ⟨ ∇ B j ( c j ) , x − c j ⟩ = q μ ∥ x − c j ∥ q ,
which is Eq. (C.18).
Last center: j = L j=L j = L . Since B = B L B=B_L B = B L , part (a)(i) gives Eq. (C.18) directly on ∥ x − c L ∥ ≤ γ R L − 1 \lVert x-c_L\rVert\le\gamma R_{L-1} ∥ x − c L ∥ ≤ γ R L − 1 if L ≥ 1 L\ge1 L ≥ 1 , and on all of R d \mathbb{R}^d R d if L = 0 L=0 L = 0 .
By Lemma C.1 (b), the original base increment and its magnitude are
b j : = ∇ B ( c j + 1 ) − ∇ B ( c j ) = μ R j q − 2 h j , G j : = ∥ b j ∥ = μ R j q − 1 . (68) b_j:=\nabla B(c_{j+1})-\nabla B(c_j)=\mu R_j^{q-2}h_j,\qquad G_j:=\lVert b_j\rVert=\mu R_j^{q-1}. \tag*{(68)} b j := ∇ B ( c j + 1 ) − ∇ B ( c j ) = μ R j q − 2 h j , G j := ∥ b j ∥ = μ R j q − 1 . (68)
Before the replacement, Eq. (C.4) holds along the full segment from c j c_j c j to c j + 1 c_{j+1} c j + 1 . The final neighborhood formula Eq. (C.18) need not hold at c j + 1 c_{j+1} c j + 1 . Thus across consecutive centers the final tangent remainder is β q μ R j q = μ R j q q + C j \beta_q\mu R_j^q=\frac{\mu R_j^q}{q}+C_j β q μ R j q = q μ R j q + C j , whereas within the final neighborhood of a center it is given by Eq. (C.18).
Hölder smoothness: compare a pair at the first relevant scale
For smoothness, each correction B j + 1 − B j B_{j+1}-B_j B j + 1 − B j has p th p^{\text{th}} p th -derivative magnitude at most a fixed multiple of μ R j ν \mu R_j^\nu μ R j ν . Their geometric decrease bounds the sum beyond any scale by its first term times a constant, avoiding a factor of L L L . To turn this magnitude bound into the third line of Eq. (C.1), we combine the correction sum for distant pairs with the exact local norm-power formulas for close pairs. The two local estimates from Lemma C.13, with the second holding for all x , y ∈ R d x,y\in\mathbb{R}^d x , y ∈ R d , are
∥ ∇ p Δ j B ∥ ∞ ≤ M p Δ μ R j ν , \lVert\nabla^p\Delta_j^B\rVert_\infty\le M_p^\Delta\mu R_j^\nu, ∥ ∇ p Δ j B ∥ ∞ ≤ M p Δ μ R j ν ,
∥ ∇ p Δ j B ( x ) − ∇ p Δ j B ( y ) ∥ ≤ H Δ μ ∥ x − y ∥ ν . \lVert\nabla^p\Delta_j^B(x)-\nabla^p\Delta_j^B(y)\rVert\le H_\Delta\mu\lVert x-y\rVert^\nu. ∥ ∇ p Δ j B ( x ) − ∇ p Δ j B ( y )∥ ≤ H Δ μ ∥ x − y ∥ ν .
Their proofs, including the estimates for N j B − B j N_j^B-B_j N j B − B j in Lemma C.12, are grouped in Appx. C.2. Combined with the exact formulas and nesting in Lemma C.1, they give the following bound independent of the number of replacements.
Lemma C.2 (Hölder smoothness independent of the number of replacements).
(a) For all x , y ∈ R d x,y\in\mathbb{R}^d x , y ∈ R d , the final base satisfies
∥ ∇ p B ( x ) − ∇ p B ( y ) ∥ ≤ C B μ ∥ x − y ∥ ν . \lVert\nabla^pB(x)-\nabla^pB(y)\rVert\le C_B\mu\lVert x-y\rVert^\nu. ∥ ∇ p B ( x ) − ∇ p B ( y )∥ ≤ C B μ ∥ x − y ∥ ν . The finite constant in Eq. (C.24) is independent of dimension, L L L , R 0 R_0 R 0 , and every allowed smaller value of η \eta η . For 0 ≤ j ≤ L 0\le j\le L 0 ≤ j ≤ L , every partial base belongs to C p , ν ( R d ) C^{p,\nu}(\mathbb{R}^d) C p , ν ( R d ) and satisfies, for all x , y ∈ R d x,y\in\mathbb{R}^d x , y ∈ R d ,
∥ ∇ p B j ( x ) − ∇ p B j ( y ) ∥ ≤ C B μ ∥ x − y ∥ ν . \lVert\nabla^pB_j(x)-\nabla^pB_j(y)\rVert\le C_B\mu\lVert x-y\rVert^\nu. ∥ ∇ p B j ( x ) − ∇ p B j ( y )∥ ≤ C B μ ∥ x − y ∥ ν .
Proof. • Proof of (a). Fix the completed function B = B L B=B_L B = B L and the constant C B C_B C B in Eq. (C.24). We use backward induction on j = L , L − 1 , … , 0 j=L,L-1,\ldots,0 j = L , L − 1 , … , 0 , since exact agreement in the next inner ball lets us apply the induction hypothesis to the same pair without an additional Hölder estimate.
Induction claim P j P_j P j (0 ≤ j ≤ L 0\le j\le L 0 ≤ j ≤ L ). For every pair of distinct x , y ∈ R d x,y\in\mathbb{R}^d x , y ∈ R d , put d 0 = ∥ x − y ∥ d_0=\lVert x-y\rVert d 0 = ∥ x − y ∥ . We prove the following implication.
Assumption: the local formula Eq. (C.4) holds on a neighborhood of each point:
B j ( w ) = B j ( c j ) + ⟨ ∇ B j ( c j ) , w − c j ⟩ + μ q ∥ w − c j ∥ q B_j(w)=B_j(c_j)+\langle\nabla B_j(c_j),w-c_j\rangle+\frac{\mu}{q}\lVert w-c_j\rVert^q B j ( w ) = B j ( c j ) + ⟨ ∇ B j ( c j ) , w − c j ⟩ + q μ ∥ w − c j ∥ q
near both x , y x,y x , y .
Conclusion: the final function B B B satisfies
∥ ∇ p B ( x ) − ∇ p B ( y ) ∥ op ≤ C B μ d 0 ν . \lVert\nabla^pB(x)-\nabla^pB(y)\rVert_{\operatorname{op}}\le C_B\mu d_0^\nu. ∥ ∇ p B ( x ) − ∇ p B ( y ) ∥ op ≤ C B μ d 0 ν .
The symbol P j P_j P j denotes this entire implication. The final function B B B and constant C B C_B C B are the same at every index.
Bound the size of all remaining corrections. Lemma C.13 (a) gives ∥ ∇ p Δ k B ∥ ∞ ≤ M p Δ μ R k ν \lVert\nabla^p\Delta_k^B\rVert_\infty\le M_p^\Delta\mu R_k^\nu ∥ ∇ p Δ k B ∥ ∞ ≤ M p Δ μ R k ν . For 0 ≤ j < L 0\le j<L 0 ≤ j < L , let k = j + a k=j+a k = j + a and use R k ν = R j ν η a ν R_k^\nu=R_j^\nu\eta^{a\nu} R k ν = R j ν η a ν :
∥ ∑ k = j L − 1 ∇ p Δ k B ∥ ∞ ≤ ∑ k = j L − 1 ∥ ∇ p Δ k B ∥ ∞ ≤ M p Δ μ R j ν ∑ a = 0 L − 1 − j η a ν ≤ M p Δ μ R j ν ∑ a = 0 ∞ η a ν = M p Δ μ R j ν 1 − η ν . \begin{aligned}
\left\lVert\sum_{k=j}^{L-1}\nabla^p\Delta_k^B\right\rVert_\infty
\le\sum_{k=j}^{L-1}\lVert\nabla^p\Delta_k^B\rVert_\infty\\
&\le M_p^\Delta\mu R_j^\nu\sum_{a=0}^{L-1-j}\eta^{a\nu} \\
&\le M_p^\Delta\mu R_j^\nu\sum_{a=0}^{\infty}\eta^{a\nu}
=\frac{M_p^\Delta\mu R_j^\nu}{1-\eta^\nu}.
\end{aligned} k = j ∑ L − 1 ∇ p Δ k B ∞ ≤ k = j ∑ L − 1 ∥ ∇ p Δ k B ∥ ∞ ≤ M p Δ μ R j ν a = 0 ∑ L − 1 − j η a ν ≤ M p Δ μ R j ν a = 0 ∑ ∞ η a ν = 1 − η ν M p Δ μ R j ν .
The first line is the triangle inequality; the second uses the displayed derivative bound, and the last line uses 0 < η ν < 1 0<\eta^\nu<1 0 < η ν < 1 . Thus
∥ ∑ k = j L − 1 ∇ p Δ k B ∥ ∞ ≤ M p Δ μ R j ν 1 − η ν . (69) \left\|\sum_{k=j}^{L-1}\nabla^p\Delta_k^B\right\|_\infty\le\frac{M_p^\Delta\mu R_j^\nu}{1-\eta^\nu}. \tag*{(69)} k = j ∑ L − 1 ∇ p Δ k B ∞ ≤ 1 − η ν M p Δ μ R j ν . (69)
An estimate from the local formula. Fix 0 ≤ j ≤ L 0\le j\le L 0 ≤ j ≤ L and suppose Eq. (C.4) holds near both x x x and y y y . Differentiating that formula at the two points, with p ≥ 2 p\ge2 p ≥ 2 , removes its affine terms:
∇ p B j ( z ) = μ ∇ w p ( ∥ w − c j ∥ q q ) ∣ w = z ( z ∈ { x , y } ) . (70) \nabla^p B_j(z)=\left.\mu\nabla_w^p\left(\frac{\lVert w-c_j\rVert^q}{q}\right)\right|_{w=z}\qquad(z\in\{x,y\}). \tag*{(70)} ∇ p B j ( z ) = μ ∇ w p ( q ∥ w − c j ∥ q ) w = z ( z ∈ { x , y }) . (70)
Lemma B.16 (d) in Appx. B therefore gives
∥ ∇ p B j ( x ) − ∇ p B j ( y ) ∥ o p ≤ H p o w μ d 0 ν . (71) \lVert\nabla^p B_j(x)-\nabla^p B_j(y)\rVert_{\mathrm{op}}\le H_{\mathrm{pow}}\mu d_0^\nu. \tag*{(71)} ∥ ∇ p B j ( x ) − ∇ p B j ( y ) ∥ op ≤ H pow μ d 0 ν . (71)
Base case: j = L j=L j = L . Take arbitrary distinct x , y x,y x , y and assume Eq. (C.4), with j = L j=L j = L , holds near both points. Put f ( u ) = ∥ u ∥ q q f(u)=\frac{\lVert u\rVert^q}{q} f ( u ) = q ∥ u ∥ q . Eq. (C.22), obtained by differentiating this local formula, reads
∇ p B L ( z ) = μ ∇ p f ( z − c L ) ( z ∈ { x , y } ) . \nabla^p B_L(z)=\mu\nabla^p f(z-c_L)\qquad(z\in\{x,y\}). ∇ p B L ( z ) = μ ∇ p f ( z − c L ) ( z ∈ { x , y }) .
Since B = B L B=B_L B = B L , Lemma B.16 (d) gives
∥ ∇ p B ( x ) − ∇ p B ( y ) ∥ o p = μ ∥ ∇ p f ( x − c L ) − ∇ p f ( y − c L ) ∥ o p ≤ H p o w μ ∥ ( x − c L ) − ( y − c L ) ∥ ν = H p o w μ d 0 ν ≤ C B μ d 0 ν . \begin{aligned}
\lVert\nabla^p B(x)-\nabla^p B(y)\rVert_{\mathrm{op}}
&=\mu\lVert\nabla^p f(x-c_L)-\nabla^p f(y-c_L)\rVert_{\mathrm{op}}\\
&\le H_{\mathrm{pow}}\mu\lVert(x-c_L)-(y-c_L)\rVert^\nu\\
&=H_{\mathrm{pow}}\mu d_0^\nu\le C_B\mu d_0^\nu.
\end{aligned} ∥ ∇ p B ( x ) − ∇ p B ( y ) ∥ op = μ ∥ ∇ p f ( x − c L ) − ∇ p f ( y − c L ) ∥ op ≤ H pow μ ∥( x − c L ) − ( y − c L ) ∥ ν = H pow μ d 0 ν ≤ C B μ d 0 ν .
We have derived the claimed bound from the assumed local formula for every such pair. This proves the implication P L \mathcal{P}_L P L .
Inductive hypothesis. Fix 0 ≤ j < L 0\le j<L 0 ≤ j < L and assume the implication P j + 1 \mathcal{P}_{j+1} P j + 1 has been proved for every distinct pair: if Eq. (C.4), with j + 1 j+1 j + 1 in place of j j j , holds near both points, then the displayed bound for B B B follows. The hypothesis is applied only in Case 2 below ; Cases 1 and 3 give direct estimates.
Induction step: prove P j \mathcal{P}_j P j . Take arbitrary distinct x , y x,y x , y and assume Eq. (C.4) for B j B_j B j holds near both points. Put d 0 = ∥ x − y ∥ d_0=\lVert x-y\rVert d 0 = ∥ x − y ∥ . Eq. (C.23) follows from this local formula. It remains to control the corrections in
B = B j + Δ j B ⏟ current correction + ∑ k = j + 1 L − 1 Δ k B ⏟ later corrections . B=B_j+\underbrace{\Delta_j^B}_{\text{current correction}}+\underbrace{\sum_{k=j+1}^{L-1}\Delta_k^B}_{\text{later corrections}}. B = B j + current correction Δ j B + later corrections k = j + 1 ∑ L − 1 Δ k B .
Why split at d 0 = γ R j 2 d_0=\frac{\gamma R_j}{2} d 0 = 2 γ R j ? Lemma C.1 (a)(i) gives the exact formula for B j + 1 B_{j+1} B j + 1 on U j = B ( c j + 1 , γ R j ) U_j=B(c_{j+1},\gamma R_j) U j = B ( c j + 1 , γ R j ) . Its containment Eq. (C.19) confines all later corrections to the concentric closed ball of radius γ R j 2 \frac{\gamma R_j}{2} 2 γ R j . The gap between these radii is γ R j 2 \frac{\gamma R_j}{2} 2 γ R j . A pair closer than this gap either lies entirely in U j U_j U j (Case 2), or has a point outside U j U_j U j , forcing both points outside the smaller ball (Case 3). For d 0 ≥ γ R j 2 d_0\ge\frac{\gamma R_j}{2} d 0 ≥ 2 γ R j , the size bound Eq. (C.21) suffices (Case 1).
Case 1. Pairs at distance at least γ R j 2 \frac{\gamma R_j}{2} 2 γ R j . If d 0 ≥ γ R j 2 d_0 \ge\frac{\gamma R_j}{2} d 0 ≥ 2 γ R j , then
R j ν ≤ ( 2 γ ) ν d 0 ν . R_j^\nu\le\left(\frac{2}{\gamma}\right)^\nu d_0^\nu. R j ν ≤ ( γ 2 ) ν d 0 ν .
Use the exact decomposition B = B j + ∑ k = j L − 1 Δ k B B = B_j + \sum_{k=j}^{L-1} \Delta_k^B B = B j + ∑ k = j L − 1 Δ k B . Differentiating and subtracting at the two points gives
∇ p B ( x ) − ∇ p B ( y ) = ∇ p B j ( x ) − ∇ p B j ( y ) + ∑ k = j L − 1 ∇ p Δ k B ( x ) − ∑ k = j L − 1 ∇ p Δ k B ( y ) . \nabla^p B(x) - \nabla^p B(y) = \nabla^p B_j(x) - \nabla^p B_j(y) + \sum_{k=j}^{L-1} \nabla^p \Delta_k^B(x) - \sum_{k=j}^{L-1} \nabla^p \Delta_k^B(y). ∇ p B ( x ) − ∇ p B ( y ) = ∇ p B j ( x ) − ∇ p B j ( y ) + k = j ∑ L − 1 ∇ p Δ k B ( x ) − k = j ∑ L − 1 ∇ p Δ k B ( y ) .
Apply the triangle inequality, then Eq. (C.23) to the B j B_j B j difference and Eq. (C.21) to each correction sum:
∥ ∇ p B ( x ) − ∇ p B ( y ) ∥ o p ≤ ∥ ∇ p B j ( x ) − ∇ p B j ( y ) ∥ o p + ∥ ∑ k = j L − 1 ∇ p Δ k B ( x ) ∥ o p + ∥ ∑ k = j L − 1 ∇ p Δ k B ( y ) ∥ o p ≤ H p o w μ d 0 ν + M p Δ μ R j ν 1 − η ν + M p Δ μ R j ν 1 − η ν = H p o w μ d 0 ν + 2 M p Δ μ R j ν 1 − η ν ≤ ( H p o w + 2 M p Δ ( 2 γ ) ν 1 − 8 − ν ) μ d 0 ν ≤ C B μ d 0 ν . \begin{aligned}
\|\nabla^p B(x) - \nabla^p B(y)\|_{\mathrm{op}}
&\le\|\nabla^p B_j(x) - \nabla^p B_j(y)\|_{\mathrm{op}} \\
&\quad+ \left\|\sum_{k=j}^{L-1} \nabla^p \Delta_k^B(x)\right\|_{\mathrm{op}}
+ \left\|\sum_{k=j}^{L-1} \nabla^p \Delta_k^B(y)\right\|_{\mathrm{op}} \\
&\le H_{\mathrm{pow}}\mu d_0^\nu
+ \frac{M_p^\Delta\mu R_j^\nu}{1-\eta^\nu}
+ \frac{M_p^\Delta\mu R_j^\nu}{1-\eta^\nu} \\
&= H_{\mathrm{pow}}\mu d_0^\nu
+ \frac{2M_p^\Delta\mu R_j^\nu}{1-\eta^\nu} \\
&\le\left(H_{\mathrm{pow}}+
\frac{2M_p^\Delta\left(\frac{2}{\gamma}\right)^\nu}{1-8^{-\nu}}\right)\mu d_0^\nu
\le C_B\mu d_0^\nu.
\end{aligned} ∥ ∇ p B ( x ) − ∇ p B ( y ) ∥ op ≤ ∥ ∇ p B j ( x ) − ∇ p B j ( y ) ∥ op + k = j ∑ L − 1 ∇ p Δ k B ( x ) op + k = j ∑ L − 1 ∇ p Δ k B ( y ) op ≤ H pow μ d 0 ν + 1 − η ν M p Δ μ R j ν + 1 − η ν M p Δ μ R j ν = H pow μ d 0 ν + 1 − η ν 2 M p Δ μ R j ν ≤ H pow + 1 − 8 − ν 2 M p Δ ( γ 2 ) ν μ d 0 ν ≤ C B μ d 0 ν .
The two applications of Eq. (C.21) give the factor 2; the last line uses R j ν ≤ ( 2 γ ) ν d 0 ν R_j^\nu\le\left(\frac{2}{\gamma}\right)^\nu d_0^\nu R j ν ≤ ( γ 2 ) ν d 0 ν and η ≤ 1 8 \eta\le\frac{1}{8} η ≤ 8 1 . This proves P j \mathcal{P}_j P j in Case 1. For d 0 < γ R j 2 d_0 < \frac{\gamma R_j}{2} d 0 < 2 γ R j , the ratio R j ν d 0 ν \frac{R_j^\nu}{d_0^\nu} d 0 ν R j ν can be arbitrarily large, so we use the locations of the points in the next two cases.
Case 2. Both points lie in U j U_j U j : use the exact formula for B j + 1 B_{j+1} B j + 1 . Assume d 0 < γ R j 2 d_0 < \frac{\gamma R_j}{2} d 0 < 2 γ R j and x , y ∈ U j x,y \in U_j x , y ∈ U j . Lemma C.1 (a)(i), applied at index j + 1 j+1 j + 1 , gives
B j + 1 ( w ) = B j + 1 ( c j + 1 ) + ⟨ ∇ B j + 1 ( c j + 1 ) , w − c j + 1 ⟩ + μ q ∥ w − c j + 1 ∥ q ( ∥ w − c j + 1 ∥ ≤ γ R j ) . B_{j+1}(w) = B_{j+1}(c_{j+1}) + \langle\nabla B_{j+1}(c_{j+1}), w-c_{j+1}\rangle+ \frac{\mu}{q}\|w-c_{j+1}\|^q
\qquad(\|w-c_{j+1}\| \le\gamma R_j). B j + 1 ( w ) = B j + 1 ( c j + 1 ) + ⟨ ∇ B j + 1 ( c j + 1 ) , w − c j + 1 ⟩ + q μ ∥ w − c j + 1 ∥ q ( ∥ w − c j + 1 ∥ ≤ γ R j ) .
For each z ∈ { x , y } z \in\{x,y\} z ∈ { x , y } ,
r z : = γ R j − ∥ z − c j + 1 ∥ > 0 , B ( z , r z ) ⊂ U j . r_z := \gamma R_j-\|z-c_{j+1}\|>0,\qquad B(z,r_z)\subset U_j. r z := γ R j − ∥ z − c j + 1 ∥ > 0 , B ( z , r z ) ⊂ U j .
Thus the displayed formula can be differentiated at both points. Recall f ( u ) = ∥ u ∥ q q f(u)=\frac{\|u\|^q}{q} f ( u ) = q ∥ u ∥ q . Since p ≥ 2 p\ge2 p ≥ 2 , the affine terms have zero p t h p^{\mathrm{th}} p th derivative, so
∇ p B j + 1 ( z ) = μ ∇ p f ( z − c j + 1 ) ( z ∈ { x , y } ) . \nabla^p B_{j+1}(z)=\mu\nabla^p f(z-c_{j+1})\qquad(z\in\{x,y\}). ∇ p B j + 1 ( z ) = μ ∇ p f ( z − c j + 1 ) ( z ∈ { x , y }) .
This is Eq. (C.22) at index j + 1 j+1 j + 1 . Lemma B.16 (d) now gives
∥ ∇ p B j + 1 ( x ) − ∇ p B j + 1 ( y ) ∥ o p = μ ∥ ∇ p f ( x − c j + 1 ) − ∇ p f ( y − c j + 1 ) ∥ o p ≤ μ H p o w ∥ ( x − c j + 1 ) − ( y − c j + 1 ) ∥ = H p o w μ ∥ x − y ∥ ν = H p o w μ d 0 ν . \left\|\nabla^p B_{j+1}(x)-\nabla^p B_{j+1}(y)\right\|_{\mathrm{op}}
= \mu\left\|\nabla^p f(x-c_{j+1})-\nabla^p f(y-c_{j+1})\right\|_{\mathrm{op}}
\le\mu H_{\mathrm{pow}}\left\|(x-c_{j+1})-(y-c_{j+1})\right\|
=H_{\mathrm{pow}}\mu\left\|x-y\right\|^\nu
=H_{\mathrm{pow}}\mu d_0^\nu. ∥ ∇ p B j + 1 ( x ) − ∇ p B j + 1 ( y ) ∥ op = μ ∥ ∇ p f ( x − c j + 1 ) − ∇ p f ( y − c j + 1 ) ∥ op ≤ μ H pow ∥ ( x − c j + 1 ) − ( y − c j + 1 ) ∥ = H pow μ ∥ x − y ∥ ν = H pow μ d 0 ν .
This proves Eq. (C.23) at index j + 1 j+1 j + 1 with the same coefficient; no bound for Δ j B \Delta_j^B Δ j B has been added.
Apply the induction hypothesis P j + 1 \mathcal{P}_{j+1} P j + 1 . We have verified Eq. (C.4) for B j + 1 B_{j+1} B j + 1 near the same x , y \boldsymbol{x},\boldsymbol{y} x , y . Applying P j + 1 \mathcal{P}_{j+1} P j + 1 to this pair gives the bound for the final function B B B :
∥ ∇ p B ( x ) − ∇ p B ( y ) ∥ o p ≤ C B μ d 0 ν \left\|\nabla^p B(x)-\nabla^p B(y)\right\|_{\mathrm{op}}\le C_B\mu d_0^\nu ∥ ∇ p B ( x ) − ∇ p B ( y ) ∥ op ≤ C B μ d 0 ν
by P j + 1 \mathcal{P}_{j+1} P j + 1 .
This proves P j \mathcal{P}_j P j in Case 2, including j = L − 1 j=L-1 j = L − 1 , where the hypothesis is the base case P L \mathcal{P}_L P L . No correction estimate is added to the induction bound.
Case 3. Remaining close pairs: at least one point lies outside U j U_j U j . Assume d 0 < γ R j 2 d_0<\frac{\gamma R_j}{2} d 0 < 2 γ R j and, exchanging the points if needed, x ∉ U j \boldsymbol{x}\notin U_j x ∈ / U j . The key identity is
B = B j + 1 , B − B j = Δ j B B=B_{j+1},\qquad B-B_j=\Delta_j^B B = B j + 1 , B − B j = Δ j B
near both x \boldsymbol{x} x and y \boldsymbol{y} y .
For j = L − 1 j=L-1 j = L − 1 , it holds globally. For j ≤ L − 2 j\le L-2 j ≤ L − 2 , apply Lemma F.2 (a) in Appx. F at starting index j + 1 j+1 j + 1 . The required distance bounds are
∥ x − c j + 1 ∥ ≥ γ R j , ∥ y − c j + 1 ∥ ≥ γ R j − d 0 > γ R j 2 . \left\|x-c_{j+1}\right\|\ge\gamma R_j,\qquad
\left\|y-c_{j+1}\right\|\ge\gamma R_j-d_0>\frac{\gamma R_j}{2}. ∥ x − c j + 1 ∥ ≥ γ R j , ∥ y − c j + 1 ∥ ≥ γ R j − d 0 > 2 γ R j .
Lemma C.1 (a)(i), through Eq. (C.19), gives
S ‾ k ⊆ S ‾ j + 1 ⊆ B ‾ ( c j + 1 , γ R j 2 ) ( j + 1 ≤ k ≤ L − 1 ) . \overline{S}_k\subseteq\overline{S}_{j+1}\subseteq\overline{B}\left(c_{j+1},\frac{\gamma R_j}{2}\right)
\qquad(j+1\le k\le L-1). S k ⊆ S j + 1 ⊆ B ( c j + 1 , 2 γ R j ) ( j + 1 ≤ k ≤ L − 1 ) .
Thus both points have positive distance from S ‾ j + 1 \overline{S}_{j+1} S j + 1 . Every correction Δ k B \Delta_k^B Δ k B with k ≥ j + 1 k\ge j+1 k ≥ j + 1 vanishes outside S k S_k S k by Eq. (C.9); part (a)(ii) supplies C p C^p C p regularity. These verify the hypotheses of Lemma F.2 and prove the neighborhood identity. Differentiating that identity gives
∇ p ( B − B j ) ( z ) = ∇ p Δ j B ( z ) ( z ∈ { x , y } ) . \nabla^p(B-B_j)(z)=\nabla^p\Delta_j^B(z)\qquad(z\in\{x,y\}). ∇ p ( B − B j ) ( z ) = ∇ p Δ j B ( z ) ( z ∈ { x , y }) .
Consequently, Eq. (C.23) and Lemma C.13 (b) yield
∥ ∇ p B ( x ) − ∇ p B ( y ) ∥ o p = ∥ ∇ p B j ( x ) − ∇ p B j ( y ) + ∇ p Δ j B ( x ) − ∇ p Δ j B ( y ) ∥ o p ≤ ∥ ∇ p B j ( x ) − ∇ p B j ( y ) ∥ o p + ∥ ∇ p Δ j B ( x ) − ∇ p Δ j B ( y ) ∥ o p ≤ H p o w μ d 0 ν + H Δ μ d 0 ν = ( H p o w + H Δ ) μ d 0 ν ≤ C B μ d 0 ν by xref‘Eq. (C.24)‘. \begin{aligned}
\left\|\nabla^p B(x)-\nabla^p B(y)\right\|_{\mathrm{op}}
&=\left\|\nabla^p B_j(x)-\nabla^p B_j(y)+\nabla^p\Delta_j^B(x)-\nabla^p\Delta_j^B(y)\right\|_{\mathrm{op}}\\
&\le\left\|\nabla^p B_j(x)-\nabla^p B_j(y)\right\|_{\mathrm{op}}
+\left\|\nabla^p\Delta_j^B(x)-\nabla^p\Delta_j^B(y)\right\|_{\mathrm{op}}\\
&\le H_{\mathrm{pow}}\mu d_0^\nu+H_\Delta\mu d_0^\nu
=(H_{\mathrm{pow}}+H_\Delta)\mu d_0^\nu\\
&\le C_B\mu d_0^\nu
\qquad\text{by {xref}`Eq. (C.24)`.}
\end{aligned} ∥ ∇ p B ( x ) − ∇ p B ( y ) ∥ op = ∇ p B j ( x ) − ∇ p B j ( y ) + ∇ p Δ j B ( x ) − ∇ p Δ j B ( y ) op ≤ ∥ ∇ p B j ( x ) − ∇ p B j ( y ) ∥ op + ∇ p Δ j B ( x ) − ∇ p Δ j B ( y ) op ≤ H pow μ d 0 ν + H Δ μ d 0 ν = ( H pow + H Δ ) μ d 0 ν ≤ C B μ d 0 ν by xref ‘Eq. (C.24)‘.
This proves P j \mathcal{P}_j P j in Case 3.
The three cases prove P j + 1 ⟹ P j \mathcal{P}_{j+1} \Longrightarrow\mathcal{P}_j P j + 1 ⟹ P j . The base case and all three estimates use the single constant
C B : = H p o w + max { H Δ , 2 M p Δ γ ( 2 γ ) ν 1 1 − 8 − ν } . (72) C_B := H_{\mathrm{pow}}+\max\left\{H_\Delta,\frac{2M_p^\Delta}{\gamma}\left(\frac{2}{\gamma}\right)^\nu\frac{1}{1-8^{-\nu}}\right\}. \tag*{(72)} C B := H pow + max { H Δ , γ 2 M p Δ ( γ 2 ) ν 1 − 8 − ν 1 } . (72)
These constants are finite and dimension independent by Lemmas B.16 (d) and C.13 (a), (b).
Conclusion of the induction. Backward induction proves the implication P j \mathcal{P}_j P j for every 0 ≤ j ≤ L 0\le j\le L 0 ≤ j ≤ L . Since B 0 ( w ) = μ ∥ w ∥ q q B_0(w)=\frac{\mu\lVert w\rVert^q}{q} B 0 ( w ) = q μ ∥ w ∥ q globally, Eq. (C.4) at j = 0 j=0 j = 0 holds near every pair of points. Applying the proved implication P 0 \mathcal{P}_0 P 0 to every distinct pair gives
∥ ∇ p B ( x ) − ∇ p B ( y ) ∥ ≤ C B μ ∥ x − y ∥ ν ( x , y ∈ R d ) . (73) \lVert\nabla^p B(x)-\nabla^p B(y)\rVert\le C_B\mu\lVert x-y\rVert^\nu\qquad(x,y\in\mathbb{R}^d). \tag*{(73)} ∥ ∇ p B ( x ) − ∇ p B ( y )∥ ≤ C B μ ∥ x − y ∥ ν ( x , y ∈ R d ) . (73)
Stopping the construction after any j ≤ L j\le L j ≤ L gives the same bound for B j B_j B j . Together with Lemma C.1 (a)(ii), this proves B j ∈ C p , ν B_j\in C^{p,\nu} B j ∈ C p , ν .
Preserve every base oracle output on a separated subspace
To verify Eq. (C.2), we separate evaluation points x ∈ W x\in W x ∈ W from all future corrections Δ k B \Delta_k^B Δ k B , k ≥ j k\ge j k ≥ j .
Lemma C.3 (Future base changes are invisible on a separated subspace). Fix 0 ≤ j < L 0\le j<L 0 ≤ j < L , and let W W W be a linear subspace. If
dist ( c j + 1 , W ) > θ R j , (74) \operatorname{dist}(c_{j+1},W)>\theta R_j, \tag*{(74)} dist ( c j + 1 , W ) > θ R j , (74) then:
(a) B = B j B=B_j B = B j on a neighborhood of every x ∈ W x\in W x ∈ W .
B = B j on a neighborhood of every x ∈ W . (75) B=B_j\quad\text{on a neighborhood of every }x\in W. \tag*{(75)} B = B j on a neighborhood of every x ∈ W . (75) (b) The values and all derivative tensors through order p p p agree at every x ∈ W x\in W x ∈ W .
Proof. • Proof of (a). For every x ∈ W x\in W x ∈ W , the separation assumption Eq. (C.26) and the triangle inequality give
dist ( x , S j ‾ ) ≥ dist ( c j + 1 , W ) − θ R j > 0. \operatorname{dist}(x,\overline{S_j})\ge\operatorname{dist}(c_{j+1},W)-\theta R_j>0. dist ( x , S j ) ≥ dist ( c j + 1 , W ) − θ R j > 0.
Lemma C.1 (a) gives C p C^p C p regularity of all partial bases and places every later support S k S_k S k (k ≥ j k\ge j k ≥ j ) inside S j = B ( c j + 1 , θ R j ) S_j=B(c_{j+1},\theta R_j) S j = B ( c j + 1 , θ R j ) . Lemma C.11 (a) gives agreement B k + 1 = B k B_{k+1}=B_k B k + 1 = B k outside each support S k = B ( c k + 1 , θ R k ) S_k=B(c_{k+1},\theta R_k) S k = B ( c k + 1 , θ R k ) . These verify the hypotheses of Lemma F.2 (a) in Appx. F, which proves Eq. (C.27).
• Proof of (b). Fix x ∈ W x\in W x ∈ W . Part (a) gives B = B j B=B_j B = B j on an open set containing x x x , and Lemma C.1 (a)(ii) gives B , B j ∈ C p B,B_j\in C^p B , B j ∈ C p . Applying Lemma F.2 (b) to this equality yields
B ( x ) = B j ( x ) , ∇ k B ( x ) = ∇ k B j ( x ) ( 1 ≤ k ≤ p ) . B(x)=B_j(x),\qquad\nabla^k B(x)=\nabla^k B_j(x)\quad(1\le k\le p). B ( x ) = B j ( x ) , ∇ k B ( x ) = ∇ k B j ( x ) ( 1 ≤ k ≤ p ) .
:::
The formula for B j B_j B j uses only previously chosen centers, their values and gradients, and Euclidean distances. It is therefore unchanged by rotations of the remaining, unchosen directions. Some derivative tensors act on the orthogonal complement of the known span, but that action identifies no particular direction there. Lemma C.3 (b) proves the base part of oracle consistency. The phase functions A j A_j A j require their separate calculation on the chain-prefix subspaces W j , b W_{j,b} W j , b from Step 3. Appx. D.1.6 verifies Eq. (C.26) for an incomplete phase using the endpoint defined in Appx. D.1.2.
Prove uniform convexity for every pair of points
Let C : = { c 0 , … , c L } C := \{c_0, \ldots, c_L\} C := { c 0 , … , c L } . The distance from a point x x x to this finite set is
dist ( x , C ) : = min 0 ≤ i ≤ L ∥ x − c i ∥ , \operatorname{dist}(x,C) := \min_{0 \le i \le L} \lVert x-c_i\rVert, dist ( x , C ) := 0 ≤ i ≤ L min ∥ x − c i ∥ ,
the Euclidean distance to the nearest center. To prove the second line of Eq. (C.1), we use the standard implication from uniform monotonicity of the gradient to uniform convexity. Lemmas C.14 and C.15 in Appx. C.2.4 verify its hypothesis for this base.
Lemma C.4 (Uniform convexity of the completed base). For the constant c B c_B c B computed below:
(a)
D B ( y , x ) ≥ c B μ ∥ y − x ∥ q ( x , y ∈ R d ) . (76) D_B(y,x) \ge c_B\mu\lVert y-x\rVert^q \qquad(x,y \in\mathbb{R}^d). \tag*{(76)} D B ( y , x ) ≥ c B μ ∥ y − x ∥ q ( x , y ∈ R d ) . (76) (b) The uniform-convexity modulus, under the convention with factor 1 q \frac{1}{q} q 1 , is at least q c B μ qc_B\mu q c B μ .
Proof. • Proof of (a). Uniform convexity requires sufficient curvature along every segment from x x x to y y y . Here B ∈ C 2 B \in C^2 B ∈ C 2 by Lemma C.1 (a)(ii), since p ≥ 2 p \ge2 p ≥ 2 . For distinct x , y ∈ R d x,y \in\mathbb{R}^d x , y ∈ R d , define d = y − x d=y-x d = y − x and ℓ = ∥ d ∥ \ell=\lVert d\rVert ℓ = ∥ d ∥ . To identify how this curvature gives growth above the tangent, we view B B B along the segment as the curve g ( s ) : = B ( x + s u ) g(s):=B(x+su) g ( s ) := B ( x + s u ) for u = d ℓ u=\frac{d}{\ell} u = ℓ d and 0 ≤ s ≤ ℓ 0 \le s \le\ell 0 ≤ s ≤ ℓ . The tangent at its starting point has height g ( 0 ) + s g ′ ( 0 ) g(0)+sg'(0) g ( 0 ) + s g ′ ( 0 ) , and its gap from the curve at the endpoint is exactly
g ( ℓ ) − g ( 0 ) − ℓ g ′ ( 0 ) = D B ( y , x ) . g(\ell)-g(0)-\ell g'(0)=D_B(y,x). g ( ℓ ) − g ( 0 ) − ℓ g ′ ( 0 ) = D B ( y , x ) .
Thus we need a lower bound of order μ ℓ q \mu\ell^q μ ℓ q on this gap, uniformly over the starting point and direction.
Lemma C.14 bounds directional curvature by ∇ 2 B ( z ) [ d , d ] ≥ μ ℓ 2 dist ( z , C ) q − 2 \nabla^2B(z)[d,d] \ge\mu\ell^2\operatorname{dist}(z,C)^{q-2} ∇ 2 B ( z ) [ d , d ] ≥ μ ℓ 2 dist ( z , C ) q − 2 . Equivalently, along the curve, g ′ ′ ( s ) ≥ μ dist ( x + s u , C ) q − 2 g''(s) \ge\mu\operatorname{dist}(x+su,C)^{q-2} g ′′ ( s ) ≥ μ dist ( x + s u , C ) q − 2 . This can vanish at the centers, but Lemma C.15 guarantees that at least half the segment stays at distance ℓ 16 \frac{\ell}{16} 16 ℓ from every center, i.e., there is a measurable Ω ⊂ [ 0 , 1 ] \Omega\subset[0,1] Ω ⊂ [ 0 , 1 ] with ∣ Ω ∣ ≥ 1 2 |\Omega| \ge\frac{1}{2} ∣Ω∣ ≥ 2 1 and dist ( x + t d , C ) ≥ ℓ 16 \operatorname{dist}(x+td,C) \ge\frac{\ell}{16} dist ( x + t d , C ) ≥ 16 ℓ on Ω \Omega Ω . Elsewhere the curvature is nonnegative, so the accumulated increase in slope cannot be undone. Integrating the curvature over this part gives
⟨ ∇ B ( y ) − ∇ B ( x ) , d ⟩ = ∫ 0 1 ∇ 2 B ( x + t d ) [ d , d ] d t by Theorem B.2 , ≥ μ ℓ 2 ∫ 0 1 dist ( x + t d , C ) q − 2 d t by Lemma C.14 , ≥ μ ℓ 2 ∫ Ω dist ( x + t d , C ) q − 2 d t by nonnegativity , ≥ μ ℓ 2 ∣ Ω ∣ ( ℓ 16 ) q − 2 by Lemma C.15 , ≥ μ 2 ⋅ 16 q − 2 ℓ q since ∣ Ω ∣ ≥ 1 2 . \begin{aligned}
\langle\nabla B(y)-\nabla B(x),d\rangle
&=\int_0^1 \nabla^2B(x+td)[d,d]\,dt && \text{by Theorem B.2},\\
&\ge\mu\ell^2\int_0^1 \operatorname{dist}(x+td,C)^{q-2}\,dt && \text{by Lemma C.14},\\
&\ge\mu\ell^2\int_\Omega\operatorname{dist}(x+td,C)^{q-2}\,dt && \text{by nonnegativity},\\
&\ge\mu\ell^2|\Omega|\left(\frac{\ell}{16}\right)^{q-2} && \text{by Lemma C.15},\\
&\ge\frac{\mu}{2\cdot16^{q-2}}\ell^q && \text{since }|\Omega|\ge\frac{1}{2}.
\end{aligned} ⟨ ∇ B ( y ) − ∇ B ( x ) , d ⟩ = ∫ 0 1 ∇ 2 B ( x + t d ) [ d , d ] d t ≥ μ ℓ 2 ∫ 0 1 dist ( x + t d , C ) q − 2 d t ≥ μ ℓ 2 ∫ Ω dist ( x + t d , C ) q − 2 d t ≥ μ ℓ 2 ∣Ω∣ ( 16 ℓ ) q − 2 ≥ 2 ⋅ 1 6 q − 2 μ ℓ q by Theorem B.2 , by Lemma C.14 , by nonnegativity , by Lemma C.15 , since ∣Ω∣ ≥ 2 1 .
In terms of the curve, this gradient increase is ℓ ( g ′ ( ℓ ) − g ′ ( 0 ) ) \ell\bigl(g'(\ell)-g'(0)\bigr) ℓ ( g ′ ( ℓ ) − g ′ ( 0 ) ) . Dividing by ℓ \ell ℓ gives the accumulated excess slope
g ′ ( ℓ ) − g ′ ( 0 ) ≥ ℓ 2 ⏟ good length μ ( ℓ 16 ) q − 2 ⏟ curvature there = μ ℓ q − 1 2 ⋅ 16 q − 2 . g'(\ell)-g'(0)\ge\underbrace{\frac{\ell}{2}}_{\text{good length}}\underbrace{\mu\left(\frac{\ell}{16}\right)^{q-2}}_{\text{curvature there}}=\frac{\mu\ell^{q-1}}{2\cdot16^{q-2}}. g ′ ( ℓ ) − g ′ ( 0 ) ≥ good length 2 ℓ curvature there μ ( 16 ℓ ) q − 2 = 2 ⋅ 1 6 q − 2 μ ℓ q − 1 .
The same estimate applies to every shorter subsegment [ x , x + s u ] [x, x + su] [ x , x + s u ] , giving excess slope at least μ s q − 1 2 ⋅ 16 q − 2 \frac{\mu s^{q-1}}{2\cdot16^{q-2}} 2 ⋅ 1 6 q − 2 μ s q − 1 . The height above the initial tangent accumulates this excess slope. Formally, the uniform-monotonicity criterion [16 Lemma 2.1] , restated as Lemma B.14, gives
D B ( y , x ) ≥ μ 2 q ⋅ 16 q − 2 ∥ y − x ∥ q . D_B(y,x) \ge\frac{\mu}{2q\cdot16^{q-2}}\lVert y-x\rVert^q. D B ( y , x ) ≥ 2 q ⋅ 1 6 q − 2 μ ∥ y − x ∥ q .
Thus the Hessian bound supplies curvature, and the geometric lemma ensures that enough of it is encountered between every pair of points. Eq. (C.28) holds with
c B : = 1 2 q ⋅ 16 q − 2 . (77) c_B := \frac{1}{2q\cdot16^{q-2}}. \tag*{(77)} c B := 2 q ⋅ 1 6 q − 2 1 . (77)
The case x = y x=y x = y is immediate.
D B ( y , x ) ≥ c B μ ∥ y − x ∥ q = q c B μ q ∥ y − x ∥ q . D_B(y,x) \ge c_B\mu\lVert y-x\rVert^q = \frac{qc_B\mu}{q}\lVert y-x\rVert^q. D B ( y , x ) ≥ c B μ ∥ y − x ∥ q = q q c B μ ∥ y − x ∥ q .
Comparison with the uniform-convexity condition in Theorem 1.1 therefore gives modulus at least q c B μ qc_B\mu q c B μ .
Corollary C.5. (All properties of the uniformly convex base.) Under the hypotheses and constant choices of Lemma 4.3, the construction in Eq. (5) satisfies all conclusions of that lemma. We may take C B C_B C B from Eq. (C.24) and c B c_B c B from Eq. (C.29).
Proof. (a) Lemma C.1 (a)(ii) gives convexity and C p C^p C p regularity of every B j B_j B j . Lemma C.2 gives their common global Hölder bound, and Lemma C.4 gives the Euclidean uniform-convexity bound for B = B L B=B_L B = B L .
(b) The gradient and value recurrences in Lemma C.1 (b), with the orthogonality computation in part (c), give Eq. (8) . Their initial value and gradient are zero.
(c) The exterior and interior conclusions of Lemma C.11 give the two exact formulas for B j + 1 B_{j+1} B j + 1 . Lemma C.1 (a)(i) verifies the local formula for each subsequent B j B_j B j .
(d) Lemma C.1 (a)(iii)–(iv) gives neighborhood agreement at the earlier centers, equality of values and derivatives outside S j ‾ \overline{S_j} S j , and dependence only on the previously chosen centers. For j = L j=L j = L , the neighborhood agreement is the identity B = B L B=B_L B = B L .
Technical lemmas for the uniformly convex component
Local gradient formulas and recentering
Lemma C.6 (A prescribed local gradient increment determines the local function). Let B j B_j B j be differentiable on B ( c j , r ) = { x : ∥ x − c j ∥ < r } B(c_j,r)=\{x:\lVert x-c_j\rVert<r\} B ( c j , r ) = { x : ∥ x − c j ∥ < r } for some r > 0 r>0 r > 0 . The following statements hold.
(a) On that ball, the gradient rule Eq. (C.3) is equivalent to the function identity Eq. (C.4).
(b) When either equivalent condition holds and ∥ c j + 1 − c j ∥ < r \lVert c_{j+1}-c_j\rVert<r ∥ c j + 1 − c j ∥ < r , the gradient increment is in the direction c j + 1 − c j c_{j+1}-c_j c j + 1 − c j and has magnitude μ ∥ c j + 1 − c j ∥ q − 1 \mu\lVert c_{j+1}-c_j\rVert^{q-1} μ ∥ c j + 1 − c j ∥ q − 1 .
Proof. • Proof of (a). Define the difference
f j ( x ) : = B j ( x ) − B j ( c j ) − ⟨ ∇ B j ( c j ) , x − c j ⟩ − μ q ∥ x − c j ∥ q . f_j(x) := B_j(x) - B_j(c_j) - \langle\nabla B_j(c_j), x-c_j\rangle- \frac{\mu}{q}\lVert x-c_j\rVert^q. f j ( x ) := B j ( x ) − B j ( c j ) − ⟨ ∇ B j ( c j ) , x − c j ⟩ − q μ ∥ x − c j ∥ q .
Lemma B.16(a) in Appx. B, including its extension at zero in part (b), gives
∇ f j ( x ) = ∇ B j ( x ) − ∇ B j ( c j ) − μ ∥ x − c j ∥ q − 2 ( x − c j ) = 0 \nabla f_j(x)=\nabla B_j(x)-\nabla B_j(c_j)-\mu\lVert x-c_j\rVert^{q-2}(x-c_j)=0 ∇ f j ( x ) = ∇ B j ( x ) − ∇ B j ( c j ) − μ ∥ x − c j ∥ q − 2 ( x − c j ) = 0
by Eq. (C.3).
Fix x ∈ B ( c j , r ) x\in B(c_j,r) x ∈ B ( c j , r ) and set v ( t ) = f j ( c j + t ( x − c j ) ) v(t)=f_j(c_j+t(x-c_j)) v ( t ) = f j ( c j + t ( x − c j )) and g ( t ) = 0 g(t)=0 g ( t ) = 0 on [ 0 , 1 ] [0,1] [ 0 , 1 ] . The segment stays in this convex ball, and ∣ v ′ ( t ) ∣ = ∣ ∇ f j ( c j + t ( x − c j ) ) [ x − c j ] ∣ = 0 = g ′ ( t ) |v'(t)|=|\nabla f_j(c_j+t(x-c_j))[x-c_j]|=0=g'(t) ∣ v ′ ( t ) ∣ = ∣∇ f j ( c j + t ( x − c j )) [ x − c j ] ∣ = 0 = g ′ ( t ) . Theorem B.3 therefore gives
∣ f j ( x ) − f j ( c j ) ∣ = ∣ v ( 1 ) − v ( 0 ) ∣ ≤ g ( 1 ) − g ( 0 ) = 0. |f_j(x)-f_j(c_j)|=|v(1)-v(0)|\leq g(1)-g(0)=0. ∣ f j ( x ) − f j ( c j ) ∣ = ∣ v ( 1 ) − v ( 0 ) ∣ ≤ g ( 1 ) − g ( 0 ) = 0.
Since f j ( c j ) = 0 f_j(c_j)=0 f j ( c j ) = 0 , this proves Eq. (C.4). Conversely, differentiation of Eq. (C.4) gives
∇ B j ( x ) = ∇ B j ( c j ) + μ ∥ x − c j ∥ q − 2 ( x − c j ) , \nabla B_j(x)=\nabla B_j(c_j)+\mu\lVert x-c_j\rVert^{q-2}(x-c_j), ∇ B j ( x ) = ∇ B j ( c j ) + μ ∥ x − c j ∥ q − 2 ( x − c j ) ,
∇ B j ( x ) − ∇ B j ( c j ) = μ ∥ x − c j ∥ q − 2 ( x − c j ) , \nabla B_j(x)-\nabla B_j(c_j)=\mu\lVert x-c_j\rVert^{q-2}(x-c_j), ∇ B j ( x ) − ∇ B j ( c j ) = μ ∥ x − c j ∥ q − 2 ( x − c j ) ,
by the same norm-power derivative formula. This is Eq. (C.3), including at x = c j x=c_j x = c j .
• Proof of (b). Set h = c j + 1 − c j ≠ 0 h=c_{j+1}-c_j\ne0 h = c j + 1 − c j = 0 and evaluate Eq. (C.3) at c j + 1 c_{j+1} c j + 1 :
∇ B j ( c j + 1 ) − ∇ B j ( c j ) = μ ∥ h ∥ q − 2 h , \nabla B_j(c_{j+1})-\nabla B_j(c_j)=\mu\lVert h\rVert^{q-2}h, ∇ B j ( c j + 1 ) − ∇ B j ( c j ) = μ ∥ h ∥ q − 2 h ,
∥ ∇ B j ( c j + 1 ) − ∇ B j ( c j ) ∥ = μ ∥ h ∥ q − 2 ∥ h ∥ = μ ∥ h ∥ q − 1 . \lVert\nabla B_j(c_{j+1})-\nabla B_j(c_j)\rVert=\mu\lVert h\rVert^{q-2}\lVert h\rVert=\mu\lVert h\rVert^{q-1}. ∥ ∇ B j ( c j + 1 ) − ∇ B j ( c j )∥ = μ ∥ h ∥ q − 2 ∥ h ∥ = μ ∥ h ∥ q − 1 .
The coefficient μ ∥ h ∥ q − 2 \mu\lVert h\rVert^{q-2} μ ∥ h ∥ q − 2 is positive, so the increment has the direction of h h h , as claimed.
Lemma C.7 (Expressing B j B_j B j relative to the next center). Suppose Eq. (C.4) holds on a neighborhood containing c j + 1 c_{j+1} c j + 1 and x x x . Then:
(a) The function B j B_j B j satisfies
B j ( x ) = B j ( c j + 1 ) + ⟨ ∇ B j ( c j + 1 ) , x − c j + 1 ⟩ + μ q ( ∥ ( c j + 1 − c j ) + ( x − c j + 1 ) ∥ q − ∥ c j + 1 − c j ∥ q ) − μ ∥ c j + 1 − c j ∥ q − 2 ⟨ c j + 1 − c j , x − c j + 1 ⟩ . (78) B_j(x)=B_j(c_{j+1})+\langle\nabla B_j(c_{j+1}),x-c_{j+1}\rangle
+\frac{\mu}{q}\left(\lVert(c_{j+1}-c_j)+(x-c_{j+1})\rVert^q-\lVert c_{j+1}-c_j\rVert^q\right)
-\mu\lVert c_{j+1}-c_j\rVert^{q-2}\langle c_{j+1}-c_j,x-c_{j+1}\rangle. \tag*{(78)} B j ( x ) = B j ( c j + 1 ) + ⟨ ∇ B j ( c j + 1 ) , x − c j + 1 ⟩ + q μ ( ∥( c j + 1 − c j ) + ( x − c j + 1 ) ∥ q − ∥ c j + 1 − c j ∥ q ) − μ ∥ c j + 1 − c j ∥ q − 2 ⟨ c j + 1 − c j , x − c j + 1 ⟩ . (78) Its last two lines generally differ from μ ∥ x − c j + 1 ∥ q q \frac{\mu\lVert x-c_{j+1}\rVert^q}{q} q μ ∥ x − c j + 1 ∥ q .
Proof. • Proof of (a). Set h = c j + 1 − c j h=c_{j+1}-c_j h = c j + 1 − c j and z = x − c j + 1 z=x-c_{j+1} z = x − c j + 1 . Eq. (C.4) and Lemma C.6(a) give
B j ( x ) − B j ( c j + 1 ) = ⟨ ∇ B j ( c j ) , z ⟩ + μ q ( ∥ h + z ∥ q − ∥ h ∥ q ) , B_j(x)-B_j(c_{j+1})=\langle\nabla B_j(c_j),z\rangle+\frac{\mu}{q}\left(\lVert h+z\rVert^q-\lVert h\rVert^q\right), B j ( x ) − B j ( c j + 1 ) = ⟨ ∇ B j ( c j ) , z ⟩ + q μ ( ∥ h + z ∥ q − ∥ h ∥ q ) ,
∇ B j ( c j ) = ∇ B j ( c j + 1 ) − μ ∥ h ∥ q − 2 h . \nabla B_j(c_j)=\nabla B_j(c_{j+1})-\mu\lVert h\rVert^{q-2}h. ∇ B j ( c j ) = ∇ B j ( c j + 1 ) − μ ∥ h ∥ q − 2 h .
Substituting the second equality into the first, and distributing the inner product, gives
B j ( x ) − B j ( c j + 1 ) = ⟨ ∇ B j ( c j + 1 ) − μ ∥ h ∥ q − 2 h , z ⟩ + μ q ( ∥ h + z ∥ q − ∥ h ∥ q ) = ⟨ ∇ B j ( c j + 1 ) , z ⟩ − μ ∥ h ∥ q − 2 ⟨ h , z ⟩ + μ q ( ∥ h + z ∥ q − ∥ h ∥ q ) . \begin{aligned}
B_j(x)-B_j(c_{j+1})=\langle\nabla B_j(c_{j+1})-\mu\lVert h\rVert^{q-2}h,z\rangle+\frac{\mu}{q}\left(\lVert h+z\rVert^q-\lVert h\rVert^q\right) \\
&=\langle\nabla B_j(c_{j+1}),z\rangle-\mu\lVert h\rVert^{q-2}\langle h,z\rangle
+\frac{\mu}{q}\left(\lVert h+z\rVert^q-\lVert h\rVert^q\right).
\end{aligned} B j ( x ) − B j ( c j + 1 ) = ⟨ ∇ B j ( c j + 1 ) − μ ∥ h ∥ q − 2 h , z ⟩ + q μ ( ∥ h + z ∥ q − ∥ h ∥ q ) = ⟨ ∇ B j ( c j + 1 ) , z ⟩ − μ ∥ h ∥ q − 2 ⟨ h , z ⟩ + q μ ( ∥ h + z ∥ q − ∥ h ∥ q ) .
Add B j ( c j + 1 ) B_j(c_{j+1}) B j ( c j + 1 ) and substitute the definitions of h , z h,z h , z . The resulting expression is exactly Eq. (C.30).
Corollary C.8 (The specified replacement changes the remainder and preserves the gradient). Suppose Eq. (C.30) and Eq. (C.8) hold on the neighborhood under consideration. Then:
(a) Their complete difference is
B j + 1 ( x ) − B j ( x ) = C j + μ ∥ c j + 1 − c j ∥ q − 2 ⟨ c j + 1 − c j , x − c j + 1 ⟩ + μ q [ ∥ x − c j + 1 ∥ q + ∥ c j + 1 − c j ∥ q − ∥ ( c j + 1 − c j ) + ( x − c j + 1 ) ∥ q ] . (79) B_{j+1}(x)-B_j(x)=C_j+\mu\lVert c_{j+1}-c_j\rVert^{q-2}\langle c_{j+1}-c_j,x-c_{j+1}\rangle
+\frac{\mu}{q}\left[\lVert x-c_{j+1}\rVert^q+\lVert c_{j+1}-c_j\rVert^q-\lVert(c_{j+1}-c_j)+(x-c_{j+1})\rVert^q\right]. \tag*{(79)} B j + 1 ( x ) − B j ( x ) = C j + μ ∥ c j + 1 − c j ∥ q − 2 ⟨ c j + 1 − c j , x − c j + 1 ⟩ + q μ [ ∥ x − c j + 1 ∥ q + ∥ c j + 1 − c j ∥ q − ∥( c j + 1 − c j ) + ( x − c j + 1 ) ∥ q ] . (79) (b) The new local gradient rule is
∇ B j + 1 ( x ) − ∇ B j + 1 ( c j + 1 ) = μ ∥ x − c j + 1 ∥ q − 2 ( x − c j + 1 ) . \nabla B_{j+1}(x)-\nabla B_{j+1}(c_{j+1})=\mu\lVert x-c_{j+1}\rVert^{q-2}(x-c_{j+1}). ∇ B j + 1 ( x ) − ∇ B j + 1 ( c j + 1 ) = μ ∥ x − c j + 1 ∥ q − 2 ( x − c j + 1 ) . (c) The preceding center-to-center increment is retained:
∇ B j + 1 ( c j + 1 ) − ∇ B j ( c j ) = μ ∥ c j + 1 − c j ∥ q − 2 ( c j + 1 − c j ) . (80) \nabla B_{j+1}(c_{j+1})-\nabla B_j(c_j)=\mu\lVert c_{j+1}-c_j\rVert^{q-2}(c_{j+1}-c_j). \tag*{(80)} ∇ B j + 1 ( c j + 1 ) − ∇ B j ( c j ) = μ ∥ c j + 1 − c j ∥ q − 2 ( c j + 1 − c j ) . (80)
Proof. • Proof of (a). Set h = c j + 1 − c j h=c_{j+1}-c_j h = c j + 1 − c j and z = x − c j + 1 z=x-c_{j+1} z = x − c j + 1 . Subtract Eq. (C.30) from Eq. (C.8); both contain the same term B j ( c j + 1 ) + ⟨ ∇ B j ( c j + 1 ) , z ⟩ B_j(c_{j+1})+\langle\nabla B_j(c_{j+1}),z\rangle B j ( c j + 1 ) + ⟨ ∇ B j ( c j + 1 ) , z ⟩ . After canceling that term, the remaining expressions are
B j + 1 ( x ) − B j ( x ) = C j + μ q ∥ z ∥ q − μ q ( ∥ h + z ∥ q − ∥ h ∥ q ) + μ ∥ h ∥ q − 2 ⟨ h , z ⟩ = C j + μ ∥ h ∥ q − 2 ⟨ h , z ⟩ + μ q ∥ z ∥ q + μ q ∥ h ∥ q − μ q ∥ h + z ∥ q = C j + μ ∥ h ∥ q − 2 ⟨ h , z ⟩ + μ q ( ∥ z ∥ q + ∥ h ∥ q − ∥ h + z ∥ q ) . \begin{aligned}
B_{j+1}(x)-B_j(x)&=C_j+\frac{\mu}{q}\lVert z\rVert^q-\frac{\mu}{q}\left(\lVert h+z\rVert^q-\lVert h\rVert^q\right)+\mu\lVert h\rVert^{q-2}\langle h,z\rangle\\
&=C_j+\mu\lVert h\rVert^{q-2}\langle h,z\rangle+\frac{\mu}{q}\lVert z\rVert^q+\frac{\mu}{q}\lVert h\rVert^q-\frac{\mu}{q}\lVert h+z\rVert^q\\
&=C_j+\mu\lVert h\rVert^{q-2}\langle h,z\rangle\\
&\quad+\frac{\mu}{q}\left(\lVert z\rVert^q+\lVert h\rVert^q-\lVert h+z\rVert^q\right).
\end{aligned} B j + 1 ( x ) − B j ( x ) = C j + q μ ∥ z ∥ q − q μ ( ∥ h + z ∥ q − ∥ h ∥ q ) + μ ∥ h ∥ q − 2 ⟨ h , z ⟩ = C j + μ ∥ h ∥ q − 2 ⟨ h , z ⟩ + q μ ∥ z ∥ q + q μ ∥ h ∥ q − q μ ∥ h + z ∥ q = C j + μ ∥ h ∥ q − 2 ⟨ h , z ⟩ + q μ ( ∥ z ∥ q + ∥ h ∥ q − ∥ h + z ∥ q ) .
Substituting h , z h,z h , z gives exactly Eq. (C.31).
• Proof of (b). Differentiate Eq. (C.8) using Lemma B.16 (a), with its extension at the center in part (b). With z = x − c j + 1 z=x-c_{j+1} z = x − c j + 1 ,
∇ B j + 1 ( x ) = ∇ B j ( c j + 1 ) + μ ∥ z ∥ q − 2 z , ∇ B j + 1 ( c j + 1 ) = ∇ B j ( c j + 1 ) , ∇ B j + 1 ( x ) − ∇ B j + 1 ( c j + 1 ) = μ ∥ z ∥ q − 2 z . \begin{aligned}
\nabla B_{j+1}(x)&=\nabla B_j(c_{j+1})+\mu\lVert z\rVert^{q-2}z,\\
\nabla B_{j+1}(c_{j+1})&=\nabla B_j(c_{j+1}),\\
\nabla B_{j+1}(x)-\nabla B_{j+1}(c_{j+1})&=\mu\lVert z\rVert^{q-2}z.
\end{aligned} ∇ B j + 1 ( x ) ∇ B j + 1 ( c j + 1 ) ∇ B j + 1 ( x ) − ∇ B j + 1 ( c j + 1 ) = ∇ B j ( c j + 1 ) + μ ∥ z ∥ q − 2 z , = ∇ B j ( c j + 1 ) , = μ ∥ z ∥ q − 2 z .
The second line uses the vanishing norm-power gradient at z = 0 z=0 z = 0 .
• Proof of (c). Using Eq. (C.7) and then Lemma C.6 (b),
∇ B j + 1 ( c j + 1 ) − ∇ B j ( c j ) = ∇ B j ( c j + 1 ) − ∇ B j ( c j ) = μ ∥ c j + 1 − c j ∥ q − 2 ( c j + 1 − c j ) . \begin{aligned}
\nabla B_{j+1}(c_{j+1})-\nabla B_j(c_j)&=\nabla B_j(c_{j+1})-\nabla B_j(c_j)\\
&=\mu\lVert c_{j+1}-c_j\rVert^{q-2}(c_{j+1}-c_j).
\end{aligned} ∇ B j + 1 ( c j + 1 ) − ∇ B j ( c j ) = ∇ B j ( c j + 1 ) − ∇ B j ( c j ) = μ ∥ c j + 1 − c j ∥ q − 2 ( c j + 1 − c j ) .
This is Eq. (C.32).
Corollary C.9 (The replacement cancels old curvature at its center). Under the preceding local specifications:
(a) The higher derivatives satisfy
∇ k B j + 1 ( c j + 1 ) = 0 ( 2 ≤ k ≤ p ) . \nabla^{k} B_{j+1}(c_{j+1})=0 \qquad(2\le k\le p). ∇ k B j + 1 ( c j + 1 ) = 0 ( 2 ≤ k ≤ p ) . (b) The correction satisfies
∇ 2 ( B j + 1 − B j ) ( c j + 1 ) ≺ 0 , \nabla^{2}(B_{j+1}-B_j)(c_{j+1})\prec0, ∇ 2 ( B j + 1 − B j ) ( c j + 1 ) ≺ 0 , so it cannot be a convex function added to the old base.
(c) For every x x x in the neighborhood where Eq. (C.8) holds,
B j + 1 ( x ) − B j + 1 ( c j + 1 ) − ⟨ ∇ B j + 1 ( c j + 1 ) , x − c j + 1 ⟩ = μ q ∥ x − c j + 1 ∥ q . (81) B_{j+1}(x)-B_{j+1}(c_{j+1})-\langle\nabla B_{j+1}(c_{j+1}),x-c_{j+1}\rangle
=\frac{\mu}{q}\lVert x-c_{j+1}\rVert^{q}. \tag*{(81)} B j + 1 ( x ) − B j + 1 ( c j + 1 ) − ⟨ ∇ B j + 1 ( c j + 1 ) , x − c j + 1 ⟩ = q μ ∥ x − c j + 1 ∥ q . (81)
Proof. • Proof of (a). The affine terms in Eq. (C.8) have zero derivatives of order at least two. The derivatives of ∥ x − c j + 1 ∥ q q \frac{\lVert x-c_{j+1}\rVert^q}{q} q ∥ x − c j + 1 ∥ q through order p p p vanish there by Lemma B.16 (b), since q > p q>p q > p .
• Proof of (b). For the old function, differentiating Eq. (C.4) with the Hessian formula in Lemma B.16 (a) gives
∇ 2 B j ( c j + 1 ) = μ ∥ c j + 1 − c j ∥ q − 2 I + μ ( q − 2 ) ∥ c j + 1 − c j ∥ q − 4 ( c j + 1 − c j ) ( c j + 1 − c j ) T ≻ 0. (82) \nabla^{2}B_j(c_{j+1})=\mu\lVert c_{j+1}-c_j\rVert^{q-2}I
+\mu(q-2)\lVert c_{j+1}-c_j\rVert^{q-4}(c_{j+1}-c_j)(c_{j+1}-c_j)^{\mathsf{T}}\succ0. \tag*{(82)} ∇ 2 B j ( c j + 1 ) = μ ∥ c j + 1 − c j ∥ q − 2 I + μ ( q − 2 ) ∥ c j + 1 − c j ∥ q − 4 ( c j + 1 − c j ) ( c j + 1 − c j ) T ≻ 0. (82)
Set h = c j + 1 − c j ≠ 0 h=c_{j+1}-c_j\ne0 h = c j + 1 − c j = 0 . For every nonzero u u u , Eq. (C.34) and part (a) give
∇ 2 ( B j + 1 − B j ) ( c j + 1 ) [ u , u ] = 0 − ∇ 2 B j ( c j + 1 ) [ u , u ] = − μ ∥ h ∥ q − 2 ∥ u ∥ 2 − μ ( q − 2 ) ∥ h ∥ q − 4 ⟨ h , u ⟩ 2 ≤ − μ ∥ h ∥ q − 2 ∥ u ∥ 2 < 0. \begin{aligned}
\nabla^{2}(B_{j+1}-B_j)(c_{j+1})[u,u]
=0-\nabla^{2}B_j(c_{j+1})[u,u] \\
&=-\mu\lVert h\rVert^{q-2}\lVert u\rVert^2-\mu(q-2)\lVert h\rVert^{q-4}\langle h,u\rangle^2 \\
&\le-\mu\lVert h\rVert^{q-2}\lVert u\rVert^2<0.
\end{aligned} ∇ 2 ( B j + 1 − B j ) ( c j + 1 ) [ u , u ] = 0 − ∇ 2 B j ( c j + 1 ) [ u , u ] = − μ ∥ h ∥ q − 2 ∥ u ∥ 2 − μ ( q − 2 ) ∥ h ∥ q − 4 ⟨ h , u ⟩ 2 ≤ − μ ∥ h ∥ q − 2 ∥ u ∥ 2 < 0.
The inequality uses q > 2 q>2 q > 2 ; its strictness uses μ > 0 \mu>0 μ > 0 and h , u ≠ 0 h,u\ne0 h , u = 0 . A twice differentiable convex function has a positive semidefinite Hessian, so the correction is not convex.
• Proof of (c). In Eq. (C.8), the center value is B j ( c j + 1 ) + C j B_j(c_{j+1})+C_j B j ( c j + 1 ) + C j and its gradient is ∇ B j ( c j + 1 ) \nabla B_j(c_{j+1}) ∇ B j ( c j + 1 ) . Thus, with z = x − c j + 1 z=x-c_{j+1} z = x − c j + 1 ,
B j + 1 ( x ) − B j + 1 ( c j + 1 ) − ⟨ ∇ B j + 1 ( c j + 1 ) , z ⟩ = [ B j ( c j + 1 ) + C j + ⟨ ∇ B j ( c j + 1 ) , z ⟩ + μ q ∥ z ∥ q ] − [ B j ( c j + 1 ) + C j ] − ⟨ ∇ B j ( c j + 1 ) , z ⟩ = μ q ∥ z ∥ q . \begin{aligned}
&B_{j+1}(x)-B_{j+1}(c_{j+1})-\langle\nabla B_{j+1}(c_{j+1}),z\rangle\\
&=\left[B_j(c_{j+1})+C_j+\langle\nabla B_j(c_{j+1}),z\rangle+\frac{\mu}{q}\lVert z\rVert^q\right]\\
&\quad-[B_j(c_{j+1})+C_j]-\langle\nabla B_j(c_{j+1}),z\rangle\\
&=\frac{\mu}{q}\lVert z\rVert^q.
\end{aligned} B j + 1 ( x ) − B j + 1 ( c j + 1 ) − ⟨ ∇ B j + 1 ( c j + 1 ) , z ⟩ = [ B j ( c j + 1 ) + C j + ⟨ ∇ B j ( c j + 1 ) , z ⟩ + q μ ∥ z ∥ q ] − [ B j ( c j + 1 ) + C j ] − ⟨ ∇ B j ( c j + 1 ) , z ⟩ = q μ ∥ z ∥ q .
This is Eq. (C.33).
:::
Joining one replacement to the old function
Lemma C.10 (Local quadratic remainder). Let V ⊂ R d V\subset\mathbb{R}^d V ⊂ R d be open and convex, let f ∈ C 2 ( V ) f\in C^2(V) f ∈ C 2 ( V ) , and suppose that m − I ⪯ ∇ 2 f ( z ) ⪯ m + I m_{-}I\preceq\nabla^2f(z)\preceq m_{+}I m − I ⪯ ∇ 2 f ( z ) ⪯ m + I for all z ∈ V z\in V z ∈ V , where m − ≤ m + m_{-}\le m_{+} m − ≤ m + are finite real constants. Then
m − 2 ∥ y − x ∥ 2 ≤ D f ( y , x ) ≤ m + 2 ∥ y − x ∥ 2 ( x , y ∈ V ) . (83) \frac{m_{-}}{2}\lVert y-x\rVert^2\le D_f(y,x)\le\frac{m_{+}}{2}\lVert y-x\rVert^2 \qquad(x,y\in V). \tag*{(83)} 2 m − ∥ y − x ∥ 2 ≤ D f ( y , x ) ≤ 2 m + ∥ y − x ∥ 2 ( x , y ∈ V ) . (83)
Proof. Fix x , y ∈ V x,y \in V x , y ∈ V and put d = y − x d=y-x d = y − x . Convexity of V V V places the entire segment x + t d x+td x + t d in V V V . For v ( t ) = f ( x + t d ) v(t)=f(x+td) v ( t ) = f ( x + t d ) , the chain rule gives v ′ ( 0 ) = ⟨ ∇ f ( x ) , d ⟩ v'(0)=\langle\nabla f(x),d\rangle v ′ ( 0 ) = ⟨ ∇ f ( x ) , d ⟩ and v ′ ′ ( t ) = ∇ 2 f ( x + t d ) [ d , d ] v''(t)=\nabla^2 f(x+td)[d,d] v ′′ ( t ) = ∇ 2 f ( x + t d ) [ d , d ] . Thus
D f ( y , x ) = v ( 1 ) − v ( 0 ) − v ′ ( 0 ) by the Bregman definition, = ∫ 0 1 ( v ′ ( t ) − v ′ ( 0 ) ) d t by the fundamental theorem of calculus, = ∫ 0 1 ∫ 0 t v ′ ′ ( s ) d s d t by the same theorem, = ∫ 0 1 ( 1 − s ) ∇ 2 f ( x + s d ) [ d , d ] d s by interchanging the continuous integrals. \begin{aligned}
D_f(y,x) &= v(1)-v(0)-v'(0) && \text{by the Bregman definition,}\\
&= \int_0^1 \bigl(v'(t)-v'(0)\bigr)\,dt && \text{by the fundamental theorem of calculus,}\\
&= \int_0^1\int_0^t v''(s)\,ds\,dt && \text{by the same theorem,}\\
&= \int_0^1 (1-s)\nabla^2 f(x+sd)[d,d]\,ds && \text{by interchanging the continuous integrals.}
\end{aligned} D f ( y , x ) = v ( 1 ) − v ( 0 ) − v ′ ( 0 ) = ∫ 0 1 ( v ′ ( t ) − v ′ ( 0 ) ) d t = ∫ 0 1 ∫ 0 t v ′′ ( s ) d s d t = ∫ 0 1 ( 1 − s ) ∇ 2 f ( x + s d ) [ d , d ] d s by the Bregman definition, by the fundamental theorem of calculus, by the same theorem, by interchanging the continuous integrals.
The assumed Hessian bounds give m − ∥ d ∥ 2 ≤ ∇ 2 f ( x + s d ) [ d , d ] ≤ m + ∥ d ∥ 2 m_{-}\lVert d\rVert^2 \le\nabla^2 f(x+sd)[d,d] \le m_{+}\lVert d\rVert^2 m − ∥ d ∥ 2 ≤ ∇ 2 f ( x + s d ) [ d , d ] ≤ m + ∥ d ∥ 2 for every s ∈ [ 0 , 1 ] s \in[0,1] s ∈ [ 0 , 1 ] . Since 1 − s ≥ 0 1-s \ge0 1 − s ≥ 0 and ∫ 0 1 ( 1 − s ) d s = 1 2 \int_0^1(1-s)\,ds=\frac{1}{2} ∫ 0 1 ( 1 − s ) d s = 2 1 , substitution yields
m − 2 ∥ d ∥ 2 ≤ D f ( y , x ) ≤ m + 2 ∥ d ∥ 2 . \frac{m_{-}}{2}\lVert d\rVert^2 \le D_f(y,x) \le\frac{m_{+}}{2}\lVert d\rVert^2. 2 m − ∥ d ∥ 2 ≤ D f ( y , x ) ≤ 2 m + ∥ d ∥ 2 .
This is Eq. (C.35).
Fix j j j and R j = ∥ c j + 1 − c j ∥ R_j=\lVert c_{j+1}-c_j\rVert R j = ∥ c j + 1 − c j ∥ . Assume that B j B_j B j is globally convex and C p C^p C p and that Eq. (C.4) holds on an open set containing S j ‾ = B ‾ ( c j + 1 , θ R j ) \overline{S_j}=\overline{B}(c_{j+1},\theta R_j) S j = B ( c j + 1 , θ R j ) . Use N j B N_j^B N j B and B j + 1 B_{j+1} B j + 1 from Appx. C.1.3, and define
ℓ j B ( x ) : = B j ( c j + 1 ) + ⟨ ∇ B j ( c j + 1 ) , x − c j + 1 ⟩ , \ell_j^B(x):=B_j(c_{j+1})+\langle\nabla B_j(c_{j+1}),x-c_{j+1}\rangle, ℓ j B ( x ) := B j ( c j + 1 ) + ⟨ ∇ B j ( c j + 1 ) , x − c j + 1 ⟩ ,
u j B ( x ) : = B j ( x ) − B j ( c j + 1 ) − ⟨ ∇ B j ( c j + 1 ) , x − c j + 1 ⟩ . u_j^B(x):=B_j(x)-B_j(c_{j+1})-\langle\nabla B_j(c_{j+1}),x-c_{j+1}\rangle. u j B ( x ) := B j ( x ) − B j ( c j + 1 ) − ⟨ ∇ B j ( c j + 1 ) , x − c j + 1 ⟩ .
v j B ( x ) : = C j + μ q ∥ x − c j + 1 ∥ q , ℓ j B + u j B = B j , ℓ j B + v j B = N j B . (84) v_j^B(x):=C_j+\frac{\mu}{q}\lVert x-c_{j+1}\rVert^q,\qquad\ell_j^B+u_j^B=B_j,\qquad\ell_j^B+v_j^B=N_j^B. \tag*{(84)} v j B ( x ) := C j + q μ ∥ x − c j + 1 ∥ q , ℓ j B + u j B = B j , ℓ j B + v j B = N j B . (84)
The scalar function v j B v_j^B v j B is distinct from the vector v j v_j v j .
Lemma C.11 (One replacement gives the new local formula and preserves the exterior). Under the assumptions above, there are constants θ , γ , a l i f t > 0 \theta,\gamma,a_{\mathrm{lift}}>0 θ , γ , a lift > 0 , depending only on q q q , with 0 < γ < θ 2 0<\gamma<\frac{\theta}{2} 0 < γ < 2 θ and θ ≤ 1 16 \theta\le\frac{1}{16} θ ≤ 16 1 , such that the choices in Eq. (C.10) give the following conclusions. Their explicit values are Eq. (C.11), Eq. (C.12), Eq. (C.13), and Eq. (C.14).
(a) Agreement with B j B_j B j outside radius ρ j 2 = θ R j 2 \frac{\rho_j}{2}=\frac{\theta R_j}{2} 2 ρ j = 2 θ R j .
B j + 1 ( x ) = B j ( x ) if ∥ x − c j + 1 ∥ ≥ ρ j 2 . B_{j+1}(x)=B_j(x)\quad\text{if }\lVert x-c_{j+1}\rVert\ge\frac{\rho_j}{2}. B j + 1 ( x ) = B j ( x ) if ∥ x − c j + 1 ∥ ≥ 2 ρ j . In particular, the formulas defining B j + 1 B_{j+1} B j + 1 on ∥ x − c j + 1 ∥ < ρ j \lVert x-c_{j+1}\rVert<\rho_j ∥ x − c j + 1 ∥ < ρ j and ∥ x − c j + 1 ∥ ≥ ρ j \lVert x-c_{j+1}\rVert\ge\rho_j ∥ x − c j + 1 ∥ ≥ ρ j agree near the sphere ∥ x − c j + 1 ∥ = ρ j = θ R j \lVert x-c_{j+1}\rVert=\rho_j=\theta R_j ∥ x − c j + 1 ∥ = ρ j = θ R j .
(b) The prescribed expression near the new center.
B j + 1 ( x ) = N j B ( x ) if ∥ x − c j + 1 ∥ ≤ γ R j . B_{j+1}(x)=N_j^B(x)\quad\text{if }\lVert x-c_{j+1}\rVert\le\gamma R_j. B j + 1 ( x ) = N j B ( x ) if ∥ x − c j + 1 ∥ ≤ γ R j . (c) Value and derivatives at the new center.
B j + 1 ( c j + 1 ) = B j ( c j + 1 ) + C j , ∇ B j + 1 ( c j + 1 ) = ∇ B j ( c j + 1 ) , ∇ k B j + 1 ( c j + 1 ) = 0 ( 2 ≤ k ≤ p ) . \begin{aligned}
B_{j+1}(c_{j+1})&=B_j(c_{j+1})+C_j,\\
\nabla B_{j+1}(c_{j+1})&=\nabla B_j(c_{j+1}),\\
\nabla^k B_{j+1}(c_{j+1})&=0\qquad(2\le k\le p).
\end{aligned} B j + 1 ( c j + 1 ) ∇ B j + 1 ( c j + 1 ) ∇ k B j + 1 ( c j + 1 ) = B j ( c j + 1 ) + C j , = ∇ B j ( c j + 1 ) , = 0 ( 2 ≤ k ≤ p ) .
The Hölder bound for the correction Δ j B = B j + 1 − B j \Delta_j^B = B_{j+1} - B_j Δ j B = B j + 1 − B j is proved separately in Lemma C.13.
Proof. We now choose C j C_j C j , δ j B \delta_j^B δ j B , θ \theta θ , and γ \gamma γ to obtain the two agreement inequalities in Eq. (6) . Both formulas have gradient ∇ B j ( c j + 1 ) \nabla B_j(c_{j+1}) ∇ B j ( c j + 1 ) at the next center, so subtracting B j B_j B j ’s tangent there reduces their comparison to their values above B j B_j B j ’s tangent. For z = x − c j + 1 z=x-c_{j+1} z = x − c j + 1 , we have
N j B ( c j + 1 + z ) − B j ( c j + 1 + z ) = C j + μ q ∥ z ∥ q − D B j ( c j + 1 + z , c j + 1 ) = u j B ( c j + 1 + z ) − u j B ( c j + 1 + z ) . (85) \begin{aligned}
N_j^B(c_{j+1}+z)-B_j(c_{j+1}+z)=C_j+\frac{\mu}{q}\lVert z\rVert^q-D_{B_j}(c_{j+1}+z,c_{j+1}) \\
&=u_j^B(c_{j+1}+z)-u_j^B(c_{j+1}+z). \tag*{(85)}
\end{aligned} N j B ( c j + 1 + z ) − B j ( c j + 1 + z ) = C j + q μ ∥ z ∥ q − D B j ( c j + 1 + z , c j + 1 ) = u j B ( c j + 1 + z ) − u j B ( c j + 1 + z ) . (85)
Proof of (a): agreement with B j B_j B j near the boundary of S j S_j S j . We first bound the Bregman growth of Eq. (C.4), which determines how much the constant C j C_j C j and the new q q q th-power term can add while leaving B j B_j B j larger near the boundary. We initially require ρ j = θ R j \rho_j=\theta R_j ρ j = θ R j with 0 < θ ≤ 1 4 0<\theta\le\frac{1}{4} 0 < θ ≤ 4 1 and define h = c j + 1 − c j h=c_{j+1}-c_j h = c j + 1 − c j . For ∥ z ∥ ≤ ρ j \lVert z\rVert\le\rho_j ∥ z ∥ ≤ ρ j and every t ∈ [ 0 , 1 ] t\in[0,1] t ∈ [ 0 , 1 ] , put w t = c j + 1 + t z w_t=c_{j+1}+tz w t = c j + 1 + t z . The entire segment lies in S j S_j S j , and
∥ w t − c j ∥ = ∥ h + t z ∥ ≥ R j − t ∥ z ∥ ≥ ( 1 − θ ) R j ≥ 3 R j 4 , \begin{aligned}
\lVert w_t-c_j\rVert=\lVert h+tz\rVert\\
&\ge R_j-t\lVert z\rVert\ge(1-\theta)R_j\ge\frac{3R_j}{4},
\end{aligned} ∥ w t − c j ∥ = ∥ h + t z ∥ ≥ R j − t ∥ z ∥ ≥ ( 1 − θ ) R j ≥ 4 3 R j ,
∥ w t − c j ∥ ≤ R j + t ∥ z ∥ ≤ ( 1 + θ ) R j ≤ 5 R j 4 . \lVert w_t-c_j\rVert\le R_j+t\lVert z\rVert\le(1+\theta)R_j\le\frac{5R_j}{4}. ∥ w t − c j ∥ ≤ R j + t ∥ z ∥ ≤ ( 1 + θ ) R j ≤ 4 5 R j .
The bounds follow from the reverse triangle and triangle inequalities. The local formula Eq. (C.4) holds on a neighborhood of this segment; its affine terms have zero Hessian. For any u ∈ R d u\in\mathbb{R}^d u ∈ R d , Eq. (B.10) in Appx. B therefore gives
∇ 2 B j ( w t ) [ u , u ] = μ ∥ w t − c j ∥ q − 2 ∥ u ∥ 2 + μ ( q − 2 ) ∥ w t − c j ∥ q − 4 ⟨ w t − c j , u ⟩ 2 , \nabla^2B_j(w_t)[u,u]=\mu\lVert w_t-c_j\rVert^{q-2}\lVert u\rVert^2+\mu(q-2)\lVert w_t-c_j\rVert^{q-4}\langle w_t-c_j,u\rangle^2, ∇ 2 B j ( w t ) [ u , u ] = μ ∥ w t − c j ∥ q − 2 ∥ u ∥ 2 + μ ( q − 2 ) ∥ w t − c j ∥ q − 4 ⟨ w t − c j , u ⟩ 2 ,
μ ∥ w t − c j ∥ q − 2 ∥ u ∥ 2 ≤ ∇ 2 B j ( w t ) [ u , u ] ≤ μ ( q − 1 ) ∥ w t − c j ∥ q − 2 ∥ u ∥ 2 . \mu\lVert w_t-c_j\rVert^{q-2}\lVert u\rVert^2\le\nabla^2B_j(w_t)[u,u]\le\mu(q-1)\lVert w_t-c_j\rVert^{q-2}\lVert u\rVert^2. μ ∥ w t − c j ∥ q − 2 ∥ u ∥ 2 ≤ ∇ 2 B j ( w t ) [ u , u ] ≤ μ ( q − 1 ) ∥ w t − c j ∥ q − 2 ∥ u ∥ 2 .
Here q > 2 q>2 q > 2 makes the second Hessian term nonnegative, and Cauchy–Schwarz bounds it by μ ( q − 2 ) ∥ w t − c j ∥ q − 2 ∥ u ∥ 2 \mu(q-2)\lVert w_t-c_j\rVert^{q-2}\lVert u\rVert^2 μ ( q − 2 ) ∥ w t − c j ∥ q − 2 ∥ u ∥ 2 . The preceding segment bounds and Eq. (C.11) now imply
2 a 0 μ R j q − 2 I ⪯ ∇ 2 B j ( x ) ⪯ 2 a 1 μ R j q − 2 I , 2a_0\mu R_j^{q-2}I\preceq\nabla^2B_j(x)\preceq2a_1\mu R_j^{q-2}I, 2 a 0 μ R j q − 2 I ⪯ ∇ 2 B j ( x ) ⪯ 2 a 1 μ R j q − 2 I ,
with a 0 , a 1 a_0,a_1 a 0 , a 1 from Eq. (C.11). To bound growth above B j B_j B j ’s tangent, we apply Lemma C.10 with f = B j f=B_j f = B j , V = S j V=S_j V = S j , x = c j + 1 x=c_{j+1} x = c j + 1 , y = c j + 1 + z y=c_{j+1}+z y = c j + 1 + z , and m − = 2 a 0 μ R j q − 2 m_-=2a_0\mu R_j^{q-2} m − = 2 a 0 μ R j q − 2 , m + = 2 a 1 μ R j q − 2 m_+=2a_1\mu R_j^{q-2} m + = 2 a 1 μ R j q − 2 . Since u j B ( c j + 1 + z ) = D B j ( c j + 1 + z , c j + 1 ) u_j^B(c_{j+1}+z)=D_{B_j}(c_{j+1}+z,c_{j+1}) u j B ( c j + 1 + z ) = D B j ( c j + 1 + z , c j + 1 ) , and continuity extends the bound to ∥ z ∥ = θ R j \lVert z\rVert=\theta R_j ∥ z ∥ = θ R j , we obtain
a 0 μ R j q − 2 ∥ z ∥ 2 ≤ u j B ( c j + 1 + z ) ≤ a 1 μ R j q − 2 ∥ z ∥ 2 . (86) a_0\mu R_j^{q-2}\lVert z\rVert^2\le u_j^B(c_{j+1}+z)\le a_1\mu R_j^{q-2}\lVert z\rVert^2. \tag*{(86)} a 0 μ R j q − 2 ∥ z ∥ 2 ≤ u j B ( c j + 1 + z ) ≤ a 1 μ R j q − 2 ∥ z ∥ 2 . (86)
Thus B j B_j B j grows quadratically above its tangent at c j + 1 c_{j+1} c j + 1 , whereas N j B N_j^B N j B adds C j + μ q ∥ z ∥ q C_j+\frac{\mu}{q}\lVert z\rVert^q C j + q μ ∥ z ∥ q above B j B_j B j ’s tangent. Since q > 2 q>2 q > 2 , we can choose C j C_j C j so that B j B_j B j is larger near the boundary of S j S_j S j while N j B N_j^B N j B is larger near its center.
We choose δ j B = C j 4 \delta_j^B = \frac{C_j}{4} δ j B = 4 C j . On the annulus θ R j 2 ≤ ∥ z ∥ ≤ θ R j \frac{\theta R_j}{2} \le\lVert z\rVert\le\theta R_j 2 θ R j ≤ ∥ z ∥ ≤ θ R j , Eq. (C.38) shows that B j − N j B ≥ δ j B B_j-N_j^B \ge\delta_j^B B j − N j B ≥ δ j B follows if
a 0 4 μ θ 2 R j q ⏟ growth bound for B j ≥ C j + μ q ( θ R j ) q ⏟ growth bound for N j B + C j 4 ⏟ smoothing width \underbrace{\frac{a_0}{4}\mu\theta^2R_j^q}_{\text{growth bound for }B_j} \ge C_j+\underbrace{\frac{\mu}{q}(\theta R_j)^q}_{\text{growth bound for }N_j^B}+\underbrace{\frac{C_j}{4}}_{\text{smoothing width}} growth bound for B j 4 a 0 μ θ 2 R j q ≥ C j + growth bound for N j B q μ ( θ R j ) q + smoothing width 4 C j
We choose C j = a l i f t μ θ 2 R j q C_j=a_{\mathrm{lift}}\mu\theta^2R_j^q C j = a lift μ θ 2 R j q , with a l i f t = a 0 8 a_{\mathrm{lift}}=\frac{a_0}{8} a lift = 8 a 0 , and denote the common positive scale by
K j B : = μ R j q − 2 ρ j 2 = μ θ 2 R j q , C j = a l i f t K j B , δ j B = d ∗ K j B , d ∗ = a 0 32 . K_j^B:=\mu R_j^{q-2}\rho_j^2=\mu\theta^2R_j^q,\qquad C_j=a_{\mathrm{lift}}K_j^B,\qquad\delta_j^B=d_*K_j^B,\qquad d_*=\frac{a_0}{32}. K j B := μ R j q − 2 ρ j 2 = μ θ 2 R j q , C j = a lift K j B , δ j B = d ∗ K j B , d ∗ = 32 a 0 .
Dividing the required inequality by K j B = μ θ 2 R j q K_j^B=\mu\theta^2R_j^q K j B = μ θ 2 R j q leaves
a l i f t + θ q − 2 q + d ∗ ≤ a 0 4 . (87) a_{\mathrm{lift}}+\frac{\theta^{q-2}}{q}+d_* \le\frac{a_0}{4}. \tag*{(87)} a lift + q θ q − 2 + d ∗ ≤ 4 a 0 . (87)
Using the choices of a l i f t a_{\mathrm{lift}} a lift and d ∗ d_* d ∗ , this leaves
θ q − 2 q ≤ a 0 4 − a l i f t − d ∗ = a 0 4 − a 0 8 − a 0 32 by Eq. ( C.12 ) , = 3 a 0 32 . \begin{aligned}
\frac{\theta^{q-2}}{q}\le\frac{a_0}{4}-a_{\mathrm{lift}}-d_*
=\frac{a_0}{4}-\frac{a_0}{8}-\frac{a_0}{32}\quad\text{by Eq. }(\text{C.12}), \\
&=\frac{3a_0}{32}.
\end{aligned} q θ q − 2 ≤ 4 a 0 − a lift − d ∗ = 4 a 0 − 8 a 0 − 32 a 0 by Eq. ( C.12 ) , = 32 3 a 0 .
We therefore choose θ = min { 1 16 , ( 3 q a 0 32 ) 1 q − 2 } \theta=\min\left\{\frac{1}{16},\left(\frac{3qa_0}{32}\right)^{\frac{1}{q-2}}\right\} θ = min { 16 1 , ( 32 3 q a 0 ) q − 2 1 } , as in Eq. (C.13). This is positive because q > 2 q>2 q > 2 . The cap 1 16 \frac{1}{16} 16 1 meets the initial requirement θ ≤ 1 4 \theta\le\frac{1}{4} θ ≤ 4 1 and will also keep evaluation points outside the replacement ball in Step 5, where θ < 1 120 \theta<\frac{1}{\sqrt{120}} θ < 120 1 is sufficient. Consequently, on ρ j 2 ≤ ∥ z ∥ ≤ ρ j \frac{\rho_j}{2}\le\lVert z\rVert\le\rho_j 2 ρ j ≤ ∥ z ∥ ≤ ρ j ,
u j B − v j B K j B ≥ a 0 4 − a 0 8 − 3 a 0 32 = a 0 32 = δ j B K j B . \frac{u_j^B-v_j^B}{K_j^B}\ge\frac{a_0}{4}-\frac{a_0}{8}-\frac{3a_0}{32}=\frac{a_0}{32}=\frac{\delta_j^B}{K_j^B}. K j B u j B − v j B ≥ 4 a 0 − 8 a 0 − 32 3 a 0 = 32 a 0 = K j B δ j B .
Thus B j − N j B = u j B − v j B ≥ δ j B B_j-N_j^B=u_j^B-v_j^B\ge\delta_j^B B j − N j B = u j B − v j B ≥ δ j B on the annulus ρ j 2 ≤ ∥ x − c j + 1 ∥ ≤ ρ j \frac{\rho_j}{2}\le\lVert x-c_{j+1}\rVert\le\rho_j 2 ρ j ≤ ∥ x − c j + 1 ∥ ≤ ρ j , equivalently θ R j 2 ≤ ∥ z ∥ ≤ θ R j \frac{\theta R_j}{2}\le\lVert z\rVert\le\theta R_j 2 θ R j ≤ ∥ z ∥ ≤ θ R j . Lemma 4.2 gives B j + 1 = B j B_{j+1}=B_j B j + 1 = B j there; beyond radius ρ j \rho_j ρ j , equality is part of definition Eq. (C.9). Together these prove
B j + 1 ( x ) = B j ( x ) if ∥ x − c j + 1 ∥ ≥ ρ j 2 . (88) B_{j+1}(x)=B_j(x)\quad\text{if }\lVert x-c_{j+1}\rVert\ge\frac{\rho_j}{2}. \tag*{(88)} B j + 1 ( x ) = B j ( x ) if ∥ x − c j + 1 ∥ ≥ 2 ρ j . (88)
This includes equality near the joining sphere ∥ x − c j + 1 ∥ = ρ j = θ R j \lVert x-c_{j+1}\rVert=\rho_j=\theta R_j ∥ x − c j + 1 ∥ = ρ j = θ R j , as claimed in part (a).
• Proof of (b): agreement with N j B N_j^B N j B near c j + 1 c_{j+1} c j + 1 . On an inner ball ∥ z ∥ ≤ γ R j \lVert z\rVert\le\gamma R_j ∥ z ∥ ≤ γ R j , the value increase C j C_j C j must exceed B j B_j B j ’s growth above its tangent by δ j B \delta_j^B δ j B . The upper quadratic bound in Eq. (C.38) therefore gives the sufficient condition
a 1 μ R j q γ 2 ≤ C j − δ j B = 3 C j 4 a_1\mu R_j^q\gamma^2\le C_j-\delta_j^B=\frac{3C_j}{4} a 1 μ R j q γ 2 ≤ C j − δ j B = 4 3 C j
since δ j B = C j 4 \delta_j^B=\frac{C_j}{4} δ j B = 4 C j ,
a 1 γ 2 ≤ 3 a 0 θ 2 32 (89) a_1\gamma^2\le\frac{3a_0\theta^2}{32} \tag*{(89)} a 1 γ 2 ≤ 32 3 a 0 θ 2 (89)
after division by μ R j q \mu R_j^q μ R j q ,
a 1 ( γ θ ) 2 + d ∗ ≤ a l i f t a_1\left(\frac{\gamma}{\theta}\right)^2+d_*\le a_{\mathrm{lift}} a 1 ( θ γ ) 2 + d ∗ ≤ a lift
by Eq. (C.12). We therefore define γ = θ 3 a 0 32 a 1 \gamma= \theta\sqrt{\frac{3a_0}{32a_1}} γ = θ 32 a 1 3 a 0 , as in Eq. (C.14). The required restriction γ < θ 2 \gamma< \frac{\theta}{2} γ < 2 θ follows without another minimum, since Eq. (C.11) gives
a 0 a 1 = ( 3 5 ) q − 2 q − 1 < 1 , ( γ θ ) 2 < 3 32 < 1 4 . \frac{a_0}{a_1} = \frac{\left(\frac{3}{5}\right)^{q-2}}{q-1} < 1,\qquad\left(\frac{\gamma}{\theta}\right)^2 < \frac{3}{32} < \frac{1}{4}. a 1 a 0 = q − 1 ( 5 3 ) q − 2 < 1 , ( θ γ ) 2 < 32 3 < 4 1 .
These choices give Eq. (C.10) and 0 < γ < θ 2 0 < \gamma< \frac{\theta}{2} 0 < γ < 2 θ . For ∥ z ∥ ≤ γ R j \lVert z\rVert\leq\gamma R_j ∥ z ∥ ≤ γ R j ,
v j B − u j B K j B ≥ a 0 8 − 3 a 0 32 = a 0 32 = δ j B K j B . \frac{v_j^B-u_j^B}{K_j^B} \geq\frac{a_0}{8}-\frac{3a_0}{32}=\frac{a_0}{32}=\frac{\delta_j^B}{K_j^B}. K j B v j B − u j B ≥ 8 a 0 − 32 3 a 0 = 32 a 0 = K j B δ j B .
Therefore N j B − B j ≥ δ j B N_j^B-B_j\geq\delta_j^B N j B − B j ≥ δ j B throughout the closed inner ball ∥ x − c j + 1 ∥ ≤ γ R j \lVert x-c_{j+1}\rVert\leq\gamma R_j ∥ x − c j + 1 ∥ ≤ γ R j . Lemma 4.2 and definition Eq. (C.9) give
B j + 1 ( x ) = N j B ( x ) if ∥ x − c j + 1 ∥ ≤ γ R j , (90) B_{j+1}(x)=N_j^B(x)\quad\text{if }\lVert x-c_{j+1}\rVert\leq\gamma R_j, \tag*{(90)} B j + 1 ( x ) = N j B ( x ) if ∥ x − c j + 1 ∥ ≤ γ R j , (90)
which proves part (c).
Proof of (c): the value and derivatives at the center. By part (b), B j + 1 B_{j+1} B j + 1 and N j B N_j^B N j B agree on a neighborhood of c j + 1 c_{j+1} c j + 1 . Evaluate and differentiate the defining formula for N j B N_j^B N j B . Its constant term is B j ( c j + 1 ) + C j B_j(c_{j+1})+C_j B j ( c j + 1 ) + C j , its affine term has gradient ∇ B j ( c j + 1 ) \nabla B_j(c_{j+1}) ∇ B j ( c j + 1 ) , and every derivative of μ ∥ x − c j + 1 ∥ q q \frac{\mu\lVert x-c_{j+1}\rVert^q}{q} q μ ∥ x − c j + 1 ∥ q of order 1 , … , p 1,\ldots,p 1 , … , p vanishes at the center by Lemma B.16 (b), since q > p q>p q > p . Hence
B j + 1 ( c j + 1 ) = B j ( c j + 1 ) + C j , ∇ B j + 1 ( c j + 1 ) = ∇ B j ( c j + 1 ) , ∇ k B j + 1 ( c j + 1 ) = 0 ( 2 ≤ k ≤ p ) , (91) \begin{aligned}
B_{j+1}(c_{j+1})&=B_j(c_{j+1})+C_j,\\
\nabla B_{j+1}(c_{j+1})&=\nabla B_j(c_{j+1}),\\
\nabla^k B_{j+1}(c_{j+1})&=0\qquad(2\leq k\leq p), \tag*{(91)}
\end{aligned} B j + 1 ( c j + 1 ) ∇ B j + 1 ( c j + 1 ) ∇ k B j + 1 ( c j + 1 ) = B j ( c j + 1 ) + C j , = ∇ B j ( c j + 1 ) , = 0 ( 2 ≤ k ≤ p ) , (91)
proving part (c).
Proof of (d): global C p C^p C p smoothness. The function B j B_j B j is convex and C p C^p C p by assumption. Lemma B.16 (b), together with part (c), gives the same properties for N j B N_j^B N j B , which is a translated norm expression plus an affine function. To verify the open-neighborhood hypothesis for joining, use Eq. (C.38) at ∥ z ∥ = ρ j \lVert z\rVert=\rho_j ∥ z ∥ = ρ j and Eq. (C.13):
B j − N j B K j B = u j B − v j B K j B ≥ a 0 − a 0 8 − 3 a 0 32 = 25 a 0 32 > a 0 32 = δ j B K j B . \frac{B_j-N_j^B}{K_j^B}=\frac{u_j^B-v_j^B}{K_j^B}\geq a_0-\frac{a_0}{8}-\frac{3a_0}{32}=\frac{25a_0}{32}>\frac{a_0}{32}=\frac{\delta_j^B}{K_j^B}. K j B B j − N j B = K j B u j B − v j B ≥ a 0 − 8 a 0 − 32 3 a 0 = 32 25 a 0 > 32 a 0 = K j B δ j B .
By continuity, this strict inequality holds on an open neighborhood of the joining sphere ∥ x − c j + 1 ∥ = ρ j \lVert x-c_{j+1}\rVert=\rho_j ∥ x − c j + 1 ∥ = ρ j . Thus Lemma B.21 (d) applies with U = R d U=\mathbb{R}^d U = R d , r = p r=p r = p , f 1 = B j f_1=B_j f 1 = B j , f 2 = N j B f_2=N_j^B f 2 = N j B , and ball B ( c j + 1 , ρ j ) B(c_{j+1},\rho_j) B ( c j + 1 , ρ j ) . Its C p C^p C p conclusion proves this part.
Derivative and Hölder estimates for Δ B j \Delta B_j Δ B j
We use Δ B j B = B j + 1 − B j \Delta B_j^B=B_{j+1}-B_j Δ B j B = B j + 1 − B j , S j = B ( c j + 1 , θ R j ) S_j=B(c_{j+1},\theta R_j) S j = B ( c j + 1 , θ R j ) , and U j = B ( c j + 1 , γ R j ) U_j=B(c_{j+1},\gamma R_j) U j = B ( c j + 1 , γ R j ) from Appx. C.1.3. By Eq. (C.9), the definition M δ ( s , t ) = s + ψ δ ( t − s ) M_\delta(s,t)=s+\psi_\delta(t-s) M δ ( s , t ) = s + ψ δ ( t − s ) , and Lemma C.11 (a)–(b),
Δ j B ( x ) = { v j B ( x ) − u j B ( x ) , ∥ x − c j + 1 ∥ ≤ γ R j , ψ δ j B ( v j B ( x ) − u j B ( x ) ) , γ R j < ∥ x − c j + 1 ∥ < ρ j 2 , 0 , ∥ x − c j + 1 ∥ ≥ ρ j 2 . (92) \Delta_j^B(x)=
\begin{cases}
v_j^B(x)-u_j^B(x), & \lVert x-c_{j+1}\rVert\leq\gamma R_j,\\
\psi_{\delta_j^B}\bigl(v_j^B(x)-u_j^B(x)\bigr), & \gamma R_j<\lVert x-c_{j+1}\rVert<\frac{\rho_j}{2},\\
0, & \lVert x-c_{j+1}\rVert\geq\frac{\rho_j}{2}.
\end{cases} \tag*{(92)} Δ j B ( x ) = ⎩ ⎨ ⎧ v j B ( x ) − u j B ( x ) , ψ δ j B ( v j B ( x ) − u j B ( x ) ) , 0 , ∥ x − c j + 1 ∥ ≤ γ R j , γ R j < ∥ x − c j + 1 ∥ < 2 ρ j , ∥ x − c j + 1 ∥ ≥ 2 ρ j . (92)
The composition ψ j δ B ( v j B − u j B ) \psi^{B}_{j\delta}(v^{B}_{j}-u^{B}_{j}) ψ j δ B ( v j B − u j B ) also equals Δ j B \Delta^{B}_{j} Δ j B throughout the closed support ball S ‾ j \overline{S}_{j} S j , where ∥ x − c j + 1 ∥ ≤ θ R j \lVert x-c_{j+1}\rVert\leq\theta R_{j} ∥ x − c j + 1 ∥ ≤ θ R j . Subtracting the two expressions in Eq. (C.36) and inserting Eq. (C.4) gives on that same set,
v j B ( x ) − u j B ( x ) = N j B ( x ) − B j ( x ) = C j + μ q ∥ x − c j + 1 ∥ q − μ q ( ∥ x − c j ∥ q − R j q ) + μ R j q − 2 ⟨ c j + 1 − c j , x − c j + 1 ⟩ . (93) \begin{aligned}
v^{B}_{j}(x)-u^{B}_{j}(x)&=N^{B}_{j}(x)-B_{j}(x)\\
&=C_{j}+\frac{\mu}{q}\lVert x-c_{j+1}\rVert^{q}-\frac{\mu}{q}\left(\lVert x-c_{j}\rVert^{q}-R_{j}^{q}\right)\\
&\quad+\mu R_{j}^{q-2}\langle c_{j+1}-c_{j},x-c_{j+1}\rangle. \tag*{(93)}
\end{aligned} v j B ( x ) − u j B ( x ) = N j B ( x ) − B j ( x ) = C j + q μ ∥ x − c j + 1 ∥ q − q μ ( ∥ x − c j ∥ q − R j q ) + μ R j q − 2 ⟨ c j + 1 − c j , x − c j + 1 ⟩ . (93)
The inherited value B j ( c j ) B_{j}(c_{j}) B j ( c j ) and gradient ∇ B j ( c j ) \nabla B_{j}(c_{j}) ∇ B j ( c j ) have canceled.
Lemma C.12 (Distance, regularity, and derivative bounds for v j B − u j B v^{B}_{j}-u^{B}_{j} v j B − u j B ). Under the assumptions and parameter choices of Lemma C.11 , the difference in Eq. (C.45) has the following properties. The constants L k B L^{B}_{k} L k B are defined explicitly in the corresponding proof parts and depend only on p , ν p,\nu p , ν .
(a) Distance from the old center. On the closed support ball S ‾ j = { x : ∥ x − c j + 1 ∥ ≤ θ R j } \overline{S}_{j}=\{x:\lVert x-c_{j+1}\rVert\leq\theta R_{j}\} S j = { x : ∥ x − c j + 1 ∥ ≤ θ R j } ,
( 1 − θ ) R j ≤ ∥ x − c j ∥ ≤ ( 1 + θ ) R j . (1-\theta)R_{j}\leq\lVert x-c_{j}\rVert\leq(1+\theta)R_{j}. ( 1 − θ ) R j ≤ ∥ x − c j ∥ ≤ ( 1 + θ ) R j . (b) Regularity of the difference. The function v j B − u j B v^{B}_{j}-u^{B}_{j} v j B − u j B belongs to C p ( R d ) C^{p}(\mathbb{R}^{d}) C p ( R d ) and to C ∞ ( S j ∖ { c j + 1 } ) C^{\infty}(S_{j}\setminus\{c_{j+1}\}) C ∞ ( S j ∖ { c j + 1 }) , where
S j ∖ { c j + 1 } = { x : 0 < ∥ x − c j + 1 ∥ < θ R j } . S_{j}\setminus\{c_{j+1}\}=\{x:0<\lVert x-c_{j+1}\rVert<\theta R_{j}\}. S j ∖ { c j + 1 } = { x : 0 < ∥ x − c j + 1 ∥ < θ R j } . (c) Bounds through order p p p on the closed support ball. For 0 ≤ k ≤ p 0\leq k\leq p 0 ≤ k ≤ p ,
∥ ∇ k ( v j B − u j B ) ( x ) ∥ ≤ L k B μ R j q − k when ∥ x − c j + 1 ∥ ≤ θ R j . \lVert\nabla^{k}(v^{B}_{j}-u^{B}_{j})(x)\rVert\leq L^{B}_{k}\mu R_{j}^{q-k}\qquad\text{when }\lVert x-c_{j+1}\rVert\leq\theta R_{j}. ∥ ∇ k ( v j B − u j B ) ( x )∥ ≤ L k B μ R j q − k when ∥ x − c j + 1 ∥ ≤ θ R j . At k = 0 k=0 k = 0 , the norm denotes absolute value.
Proof. Put z = x − c j + 1 z=x-c_{j+1} z = x − c j + 1 and h = c j + 1 − c j h=c_{j+1}-c_{j} h = c j + 1 − c j , so ∥ h ∥ = R j \lVert h\rVert=R_{j} ∥ h ∥ = R j . With f ( u ) = ∥ u ∥ q q f(u)=\frac{\lVert u\rVert^{q}}{q} f ( u ) = q ∥ u ∥ q , Eq. (C.45) reads on S ‾ j \overline{S}_{j} S j
v j B − u j B = C j + μ ( f ( z ) − f ( h + z ) + f ( h ) + ⟨ ∇ f ( h ) , z ⟩ ) . v^{B}_{j}-u^{B}_{j}=C_{j}+\mu\left(f(z)-f(h+z)+f(h)+\langle\nabla f(h),z\rangle\right). v j B − u j B = C j + μ ( f ( z ) − f ( h + z ) + f ( h ) + ⟨ ∇ f ( h ) , z ⟩ ) .
Proof of (b). Globally, u j B = B j − ℓ j B u^{B}_{j}=B_{j}-\ell^{B}_{j} u j B = B j − ℓ j B and v j B = C j + μ f ( z ) v^{B}_{j}=C_{j}+\mu f(z) v j B = C j + μ f ( z ) are C p C^{p} C p , by the hypothesis on B j B_{j} B j and Lemma B.16 (b) in Appx. B. On S j ∖ { c j + 1 } S_{j}\setminus\{c_{j+1}\} S j ∖ { c j + 1 } , both z z z and h + z h+z h + z are nonzero by part (a); the displayed formula and Lemma B.16 (a) give C ∞ C^{\infty} C ∞ regularity there.
∣ v j B ( x ) − u j B ( x ) ∣ ≤ C j + μ q ( ∥ z ∥ q + ∥ h + z ∥ q + ∥ h ∥ q ) + μ ∥ h ∥ q − 1 ∥ z ∥ ≤ μ R j q ( a l i f t θ q + θ q + ( 1 + θ ) q + 1 q + θ ) . \begin{aligned}
\left|v^{B}_{j}(x)-u^{B}_{j}(x)\right|\leq C_{j}+\frac{\mu}{q}\left(\lVert z\rVert^{q}+\lVert h+z\rVert^{q}+\lVert h\rVert^{q}\right)+\mu\lVert h\rVert^{q-1}\lVert z\rVert\\
&\leq\mu R_{j}^{q}\left(a_{\mathrm{lift}}\theta^{q}+\frac{\theta^{q}+(1+\theta)^{q}+1}{q}+\theta\right).
\end{aligned} v j B ( x ) − u j B ( x ) ≤ C j + q μ ( ∥ z ∥ q + ∥ h + z ∥ q + ∥ h ∥ q ) + μ ∥ h ∥ q − 1 ∥ z ∥ ≤ μ R j q ( a lift θ q + q θ q + ( 1 + θ ) q + 1 + θ ) .
The last line uses C j = a l i f t θ 2 μ R j q C_{j}=a_{\mathrm{lift}}\theta^{2}\mu R_{j}^{q} C j = a lift θ 2 μ R j q and the distance bounds in part (a). For k = 1 k=1 k = 1 , differentiate the same formula:
∇ ( v j B − u j B ) ( x ) = μ ( ∇ f ( z ) − ∇ f ( h + z ) + ∇ f ( h ) ) , \nabla(v^{B}_{j}-u^{B}_{j})(x)=\mu\left(\nabla f(z)-\nabla f(h+z)+\nabla f(h)\right), ∇ ( v j B − u j B ) ( x ) = μ ( ∇ f ( z ) − ∇ f ( h + z ) + ∇ f ( h ) ) ,
∥ ∇ ( v j B − u j B ) ( x ) ∥ ≤ μ ( ∥ z ∥ q − 1 + ∥ h + z ∥ q − 1 + ∥ h ∥ q − 1 ) ≤ μ R j q − 1 ( θ q − 1 + ( 1 + θ ) q − 1 + 1 ) . \begin{aligned}
\lVert\nabla(v^{B}_{j}-u^{B}_{j})(x)\rVert\leq\mu\left(\lVert z\rVert^{q-1}+\lVert h+z\rVert^{q-1}+\lVert h\rVert^{q-1}\right) \\
&\leq\mu R_{j}^{q-1}\left(\theta^{q-1}+(1+\theta)^{q-1}+1\right).
\end{aligned} ∥ ∇ ( v j B − u j B ) ( x )∥ ≤ μ ( ∥ z ∥ q − 1 + ∥ h + z ∥ q − 1 + ∥ h ∥ q − 1 ) ≤ μ R j q − 1 ( θ q − 1 + ( 1 + θ ) q − 1 + 1 ) .
The norm-power gradient formula is Lemma B.16 (a), with its extension at zero. For 2 ≤ k < p 2 \le k < p 2 ≤ k < p , the constant and affine terms disappear, so that lemma and the triangle inequality give
∇ k ( v j B − u j B ) ( x ) = μ ( ∇ k f ( z ) − ∇ k f ( h + z ) ) , \nabla^{k}\left(v_{j}^{B}-u_{j}^{B}\right)(x)=\mu\left(\nabla^{k}f(z)-\nabla^{k}f(h+z)\right), ∇ k ( v j B − u j B ) ( x ) = μ ( ∇ k f ( z ) − ∇ k f ( h + z ) ) ,
∥ ∇ k ( v j B − u j B ) ( x ) ∥ ≤ μ C q , k ( ∥ z ∥ q − k + ∥ h + z ∥ q − k ) ≤ μ R j q − k C q , k ( θ q − k + ( 1 + θ ) q − k ) . \begin{aligned}
\left\|\nabla^{k}\left(v_{j}^{B}-u_{j}^{B}\right)(x)\right\|\leq\mu C_{q,k}\left(\|z\|^{q-k}+\|h+z\|^{q-k}\right) \\
&\leq\mu R_{j}^{q-k}C_{q,k}\left(\theta^{q-k}+(1+\theta)^{q-k}\right).
\end{aligned} ∇ k ( v j B − u j B ) ( x ) ≤ μ C q , k ( ∥ z ∥ q − k + ∥ h + z ∥ q − k ) ≤ μ R j q − k C q , k ( θ q − k + ( 1 + θ ) q − k ) .
Finally, for k = p k=p k = p , use the Hölder estimate in Lemma B.16 (d):
∥ ∇ p ( v j B − u j B ) ( x ) ∥ = μ ∥ ∇ p f ( z ) − ∇ p f ( h + z ) ∥ ≤ μ H p o w ∥ z − ( h + z ) ∥ ν = μ H p o w R j ν = μ H p o w R j q − p . \begin{aligned}
\left\|\nabla^{p}\left(v_{j}^{B}-u_{j}^{B}\right)(x)\right\|=\mu\left\|\nabla^{p}f(z)-\nabla^{p}f(h+z)\right\| \\
&\leq\mu H_{\mathrm{pow}}\|z-(h+z)\|^{\nu} \\
&=\mu H_{\mathrm{pow}}R_{j}^{\nu}=\mu H_{\mathrm{pow}}R_{j}^{q-p}.
\end{aligned} ∇ p ( v j B − u j B ) ( x ) = μ ∥ ∇ p f ( z ) − ∇ p f ( h + z ) ∥ ≤ μ H pow ∥ z − ( h + z ) ∥ ν = μ H pow R j ν = μ H pow R j q − p .
These four calculations give the stated bounds with
L k B : = { a l i f t θ 2 + θ q + ( 1 + θ ) q + 1 q + θ , k = 0 , θ q − 1 + ( 1 + θ ) q − 1 + 1 , k = 1 , C q , k [ θ q − k + ( 1 + θ ) q − k ] , 2 ≤ k < p , H p o w , k = p . L_{k}^{B}:=
\begin{cases}
a_{\mathrm{lift}}\theta^{2}+\dfrac{\theta^{q}+(1+\theta)^{q+1}}{q}+\theta, & k=0,\\
\theta^{q-1}+(1+\theta)^{q-1}+1, & k=1,\\
C_{q,k}\left[\theta^{q-k}+(1+\theta)^{q-k}\right], & 2\leq k<p,\\
H_{\mathrm{pow}}, & k=p.
\end{cases} L k B := ⎩ ⎨ ⎧ a lift θ 2 + q θ q + ( 1 + θ ) q + 1 + θ , θ q − 1 + ( 1 + θ ) q − 1 + 1 , C q , k [ θ q − k + ( 1 + θ ) q − k ] , H pow , k = 0 , k = 1 , 2 ≤ k < p , k = p .
The third range is empty for p = 2 p=2 p = 2 . All bounds include the center and boundary by C p C^{p} C p continuity.
Lemma C.13 (Derivative and Hölder bounds for one replacement). Under the assumptions and parameter choices of Lemma C.11 , there is a finite constant C b a s e C_{\mathrm{base}} C base , depending only on p p p , ν \nu ν and the fixed smoothing function, for which the following hold.
(a) Derivative bounds at scale R j R_j R j . The function Δ j B \Delta_{j}^{B} Δ j B is C p C^{p} C p , and
∥ ∇ k Δ j B ∥ ∞ ≤ M k Δ μ R j q − k , M k Δ : = C b a s e ( 0 ≤ k ≤ p ) . (94) \|\nabla^{k}\Delta_{j}^{B}\|_{\infty}\leq M_{k}^{\Delta}\mu R_{j}^{q-k},\qquad M_{k}^{\Delta}:=C_{\mathrm{base}}\quad(0\leq k\leq p). \tag*{(94)} ∥ ∇ k Δ j B ∥ ∞ ≤ M k Δ μ R j q − k , M k Δ := C base ( 0 ≤ k ≤ p ) . (94) For k = 0 k=0 k = 0 , the norm is absolute value.
(b) Hölder continuity of the p t h p^{\mathrm{th}} p th derivative. For all x , y ∈ R d x,y\in\mathbb{R}^{d} x , y ∈ R d ,
∥ ∇ p Δ j B ( x ) − ∇ p Δ j B ( y ) ∥ ≤ H Δ μ ∥ x − y ∥ ν , H Δ : = C b a s e . (95) \|\nabla^{p}\Delta_{j}^{B}(x)-\nabla^{p}\Delta_{j}^{B}(y)\|\leq H_{\Delta}\mu\|x-y\|^{\nu},\qquad H_{\Delta}:=C_{\mathrm{base}}. \tag*{(95)} ∥ ∇ p Δ j B ( x ) − ∇ p Δ j B ( y ) ∥ ≤ H Δ μ ∥ x − y ∥ ν , H Δ := C base . (95) If also B j ∈ C p , ν B_j\in C^{p,\nu} B j ∈ C p , ν , then B j + 1 = B j + Δ j B B_{j+1}=B_j+\Delta_{j}^{B} B j + 1 = B j + Δ j B belongs to C p , ν C^{p,\nu} C p , ν .
Proof. We prove parts (a) and (b) together by one bounded composition estimate. Rotation and rescaling by R j = ∥ c j + 1 − c j ∥ R_j=\|c_{j+1}-c_j\| R j = ∥ c j + 1 − c j ∥ and μ R j q \mu R_j^q μ R j q give a correction whose derivative bounds do not depend on the phase. Let e 1 e_1 e 1 be the first coordinate vector and choose an orthogonal matrix Q j Q_j Q j with Q j e 1 = c j + 1 − c j R j Q_je_1=\dfrac{c_{j+1}-c_j}{R_j} Q j e 1 = R j c j + 1 − c j . The normalized input difference and resulting correction are
δ ∗ : = a l i f t θ 2 4 , G d ( z ) : = a l i f t θ q + ∥ z ∥ q − ∥ e 1 + z ∥ q + 1 q + ⟨ e 1 , z ⟩ , \delta_*:=\dfrac{a_{\mathrm{lift}}\theta^2}{4},\qquad G_d(z):=a_{\mathrm{lift}}\theta^q+\dfrac{\|z\|^q-\|e_1+z\|^q+1}{q}+\langle e_1,z\rangle, δ ∗ := 4 a lift θ 2 , G d ( z ) := a lift θ q + q ∥ z ∥ q − ∥ e 1 + z ∥ q + 1 + ⟨ e 1 , z ⟩ ,
D d ( z ) : = { ψ δ ∗ ( G d ( z ) ) , ∥ z ∥ < θ , 0 , ∥ z ∥ ≥ θ . D_d(z):=
\begin{cases}
\psi_{\delta_*}(G_d(z)), & \|z\|<\theta,\\
0, & \|z\|\geq\theta.
\end{cases} D d ( z ) := { ψ δ ∗ ( G d ( z )) , 0 , ∥ z ∥ < θ , ∥ z ∥ ≥ θ .
Define a = μ R j q > 0 a=\mu R_j^q>0 a = μ R j q > 0 . Orthogonality of Q j Q_j Q j , its defining identity Q j e 1 = c j + 1 − c j R j Q_je_1=\frac{c_{j+1}-c_j}{R_j} Q j e 1 = R j c j + 1 − c j , and Eq. (C.45) give
( v j B − u j B ) ( c j + 1 + R j Q j z ) = a α l i f t θ 2 + a q ( ∥ Q j z ∥ q − ∥ Q j ( e 1 + z ) ∥ q + 1 ) + a ⟨ Q j e 1 , Q j z ⟩ = a ( α l i f t θ 2 + ∥ z ∥ q − ∥ e 1 + z ∥ q + 1 q + ⟨ e 1 , z ⟩ ) = a G d ( z ) . (v_j^B-u_j^B)(c_{j+1}+R_jQ_jz)
= a\alpha_{\mathrm{lift}}\theta^2+\frac{a}{q}\left(\lVert Q_jz\rVert^q-\lVert Q_j(e_1+z)\rVert^q+1\right)+a\langle Q_je_1,Q_jz\rangle
= a\left(\alpha_{\mathrm{lift}}\theta^2+\frac{\lVert z\rVert^q-\lVert e_1+z\rVert^q+1}{q}+\langle e_1,z\rangle\right)=aG_d(z). ( v j B − u j B ) ( c j + 1 + R j Q j z ) = a α lift θ 2 + q a ( ∥ Q j z ∥ q − ∥ Q j ( e 1 + z ) ∥ q + 1 ) + a ⟨ Q j e 1 , Q j z ⟩ = a ( α lift θ 2 + q ∥ z ∥ q − ∥ e 1 + z ∥ q + 1 + ⟨ e 1 , z ⟩ ) = a G d ( z ) .
Also δ j B = a δ ∗ \delta_j^B=a\delta_* δ j B = a δ ∗ , by Eq. (C.10). The scalar definition Eq. (B.1) in Appx. B gives
ψ a δ ∗ ( a t ) = a t + a δ ∗ Θ ( a t a δ ∗ ) 2 = a t + δ ∗ Θ ( t δ ∗ ) 2 = a ψ δ ∗ ( t ) . \psi_{a\delta_*}(at)=\frac{at+a\delta_*\Theta\left(\frac{at}{a\delta_*}\right)}{2}
=a\frac{t+\delta_*\Theta\left(\frac{t}{\delta_*}\right)}{2}
=a\psi_{\delta_*}(t). ψ a δ ∗ ( a t ) = 2 a t + a δ ∗ Θ ( a δ ∗ a t ) = a 2 t + δ ∗ Θ ( δ ∗ t ) = a ψ δ ∗ ( t ) .
Substituting these two identities into Eq. (C.44) proves
Δ j B ( c j + 1 + R j Q j z ) = μ R j q D d ( z ) . \Delta_j^B(c_{j+1}+R_jQ_jz)=\mu R_j^qD_d(z). Δ j B ( c j + 1 + R j Q j z ) = μ R j q D d ( z ) .
Lemma C.11 (d) and part (a) imply that D d ∈ C p \mathcal{D}_d\in C^p D d ∈ C p and D d = 0 \mathcal{D}_d=0 D d = 0 on ∥ z ∥ ≥ θ 2 \lVert z\rVert\ge\frac{\theta}{2} ∥ z ∥ ≥ 2 θ .
On U : = B ( 0 , θ ) U:=B(0,\theta) U := B ( 0 , θ ) , Lemma C.12 (c) gives ∥ ∇ k G d ∥ ∞ ; U ≤ L k B \lVert\nabla^kG_d\rVert_{\infty;U}\le L_k^B ∥ ∇ k G d ∥ ∞ ; U ≤ L k B for 0 ≤ k ≤ p 0\le k\le p 0 ≤ k ≤ p . The p t h p^{\mathrm{th}} p th derivatives of the two norm powers in G d G_d G d are each Hölder continuous with parameter H p o w H_{\mathrm{pow}} H pow by Lemma B.16 (d); its affine terms contribute zero. More explicitly, for distinct z , w ∈ U z,w\in U z , w ∈ U ,
∥ ∇ p G d ( z ) − ∇ p G d ( w ) ∥ ≤ ∥ ∇ p f ( z ) − ∇ p f ( w ) ∥ + ∥ ∇ p f ( e 1 + z ) − ∇ p f ( e 1 + w ) ∥ ≤ H p o w ∥ z − w ∥ ν + H p o w ∥ ( e 1 + z ) − ( e 1 + w ) ∥ ν = 2 H p o w ∥ z − w ∥ ν , \begin{aligned}
\lVert\nabla^pG_d(z)-\nabla^pG_d(w)\rVert
&\le\lVert\nabla^pf(z)-\nabla^pf(w)\rVert+\lVert\nabla^pf(e_1+z)-\nabla^pf(e_1+w)\rVert\\
&\le H_{\mathrm{pow}}\lVert z-w\rVert^\nu+H_{\mathrm{pow}}\lVert(e_1+z)-(e_1+w)\rVert^\nu\\
&=2H_{\mathrm{pow}}\lVert z-w\rVert^\nu,
\end{aligned} ∥ ∇ p G d ( z ) − ∇ p G d ( w )∥ ≤ ∥ ∇ p f ( z ) − ∇ p f ( w )∥ + ∥ ∇ p f ( e 1 + z ) − ∇ p f ( e 1 + w )∥ ≤ H pow ∥ z − w ∥ ν + H pow ∥( e 1 + z ) − ( e 1 + w ) ∥ ν = 2 H pow ∥ z − w ∥ ν ,
where f ( u ) = ∥ u ∥ q q f(u)=\frac{\lVert u\rVert^q}{q} f ( u ) = q ∥ u ∥ q . Together with the bounds through order p p p , this gives
C G : = max { 1 , L 0 B , … , L p B , 2 H p o w } , ∥ G d ∥ p , ν ; U ≤ C G . C_G:=\max\left\{1,L_0^B,\ldots,L_p^B,2H_{\mathrm{pow}}\right\},\qquad\lVert G_d\rVert_{p,\nu;U}\le C_G. C G := max { 1 , L 0 B , … , L p B , 2 H pow } , ∥ G d ∥ p , ν ; U ≤ C G .
Set J : = ( − L 0 B − 1 , L 0 B + 1 ) J:=(-L_0^B-1,L_0^B+1) J := ( − L 0 B − 1 , L 0 B + 1 ) and C ψ : = ∥ ψ δ ∗ ∥ p , ν ; J < ∞ C_\psi:=\lVert\psi_{\delta_*}\rVert_{p,\nu;J}<\infty C ψ := ∥ ψ δ ∗ ∥ p , ν ; J < ∞ . The interval J J J contains G d ( U ) G_d(U) G d ( U ) , so Lemma B.8 on the convex open sets U U U and J J J gives
∥ D d ∥ p , ν ; U ≤ C b a s e , C b a s e : = C q c o m p C ψ C G q . \lVert\mathcal{D}_d\rVert_{p,\nu;U}\le C_{\mathrm{base}},\qquad C_{\mathrm{base}}:=C_q^{\mathrm{comp}}C_\psi C_G^q. ∥ D d ∥ p , ν ; U ≤ C base , C base := C q comp C ψ C G q .
The bounded interval and smooth outer function make C ψ C_\psi C ψ finite; all constants are independent of d d d and the phase scale.
The same bound holds globally. Derivative suprema extend because D d = 0 \mathcal{D}_d=0 D d = 0 outside U U U . For Hölder continuity, pairs in U U U are already covered. For a pair with one point outside U U U , exchange the names if needed so that y ∉ U y\notin U y ∈ / U . If ∥ x ∥ < θ 2 \lVert x\rVert<\frac{\theta}{2} ∥ x ∥ < 2 θ , choose the point z ∈ [ x , y ] ∩ ∂ B ( 0 , θ 2 ) z\in[x,y]\cap\partial B(0,\frac{\theta}{2}) z ∈ [ x , y ] ∩ ∂ B ( 0 , 2 θ ) . Both ∇ p D d ( z ) \nabla^p\mathcal{D}_d(z) ∇ p D d ( z ) and ∇ p D d ( y ) \nabla^p\mathcal{D}_d(y) ∇ p D d ( y ) vanish, including at the sphere by C p C^p C p continuity. Since x , z ∈ U x,z\in U x , z ∈ U ,
∥ ∇ p D d ( x ) − ∇ p D d ( y ) ∥ = ∥ ∇ p D d ( x ) − ∇ p D d ( z ) ∥ ≤ C b a s e ∥ x − z ∥ ν ≤ C b a s e ∥ x − y ∥ ν . \begin{aligned}
\lVert\nabla^p\mathcal{D}_d(x)-\nabla^p\mathcal{D}_d(y)\rVert
&=\lVert\nabla^p\mathcal{D}_d(x)-\nabla^p\mathcal{D}_d(z)\rVert\\
&\le C_{\mathrm{base}}\lVert x-z\rVert^\nu\\
&\le C_{\mathrm{base}}\lVert x-y\rVert^\nu.
\end{aligned} ∥ ∇ p D d ( x ) − ∇ p D d ( y )∥ = ∥ ∇ p D d ( x ) − ∇ p D d ( z )∥ ≤ C base ∥ x − z ∥ ν ≤ C base ∥ x − y ∥ ν .
Every remaining pair has both tensors zero. Thus ∥ D d ∥ p , ν ; R d ≤ C b a s e \lVert D_d\rVert_{p,\nu;\mathbb{R}^d} \le C_{\mathrm{base}} ∥ D d ∥ p , ν ; R d ≤ C base . Finally, put z = R j − 1 Q j T ( x − c j + 1 ) z = R_j^{-1}Q_j^{\mathsf T}(x-c_{j+1}) z = R j − 1 Q j T ( x − c j + 1 ) . The scaling identity and the chain rule imply, for arbitrary vectors u 1 , … , u k u_1,\ldots,u_k u 1 , … , u k and 1 ≤ k ≤ p 1 \le k \le p 1 ≤ k ≤ p ,
∇ k Δ j B ( x ) [ u 1 , … , u k ] = μ R j q ∇ k D d ( z ) [ R j − 1 Q j T u 1 , … , R j − 1 Q j T u k ] = μ R j q − k ∇ k D d ( z ) [ Q j T u 1 , … , Q j T u k ] . \nabla^k\Delta_j^B(x)[u_1,\ldots,u_k]
= \mu R_j^q\nabla^kD_d(z)[R_j^{-1}Q_j^{\mathsf T}u_1,\ldots,R_j^{-1}Q_j^{\mathsf T}u_k]
= \mu R_j^{q-k}\nabla^kD_d(z)[Q_j^{\mathsf T}u_1,\ldots,Q_j^{\mathsf T}u_k]. ∇ k Δ j B ( x ) [ u 1 , … , u k ] = μ R j q ∇ k D d ( z ) [ R j − 1 Q j T u 1 , … , R j − 1 Q j T u k ] = μ R j q − k ∇ k D d ( z ) [ Q j T u 1 , … , Q j T u k ] .
The map Q j T Q_j^{\mathsf T} Q j T preserves norms and maps the unit sphere onto itself. Taking the full multilinear operator norm therefore gives equality of the two derivative norms. For k = 0 k=0 k = 0 , use the value identity directly. For y = c j + 1 + R j Q j w y=c_{j+1}+R_jQ_jw y = c j + 1 + R j Q j w with w ≠ z w\ne z w = z , the same change of coordinates gives
∥ ∇ p Δ j B ( x ) − ∇ p Δ j B ( y ) ∥ ∥ x − y ∥ ν = μ R j q − p ∥ ∇ p D d ( z ) − ∇ p D d ( w ) ∥ R j ν ∥ z − w ∥ ν = μ R j q − p − ν ∥ ∇ p D d ( z ) − ∇ p D d ( w ) ∥ ∥ z − w ∥ ν . \frac{\lVert\nabla^p\Delta_j^B(x)-\nabla^p\Delta_j^B(y)\rVert}{\lVert x-y\rVert^\nu}
= \mu R_j^{q-p}\frac{\lVert\nabla^pD_d(z)-\nabla^pD_d(w)\rVert}{R_j^\nu\lVert z-w\rVert^\nu}
= \mu R_j^{q-p-\nu}\frac{\lVert\nabla^pD_d(z)-\nabla^pD_d(w)\rVert}{\lVert z-w\rVert^\nu}. ∥ x − y ∥ ν ∥ ∇ p Δ j B ( x ) − ∇ p Δ j B ( y )∥ = μ R j q − p R j ν ∥ z − w ∥ ν ∥ ∇ p D d ( z ) − ∇ p D d ( w )∥ = μ R j q − p − ν ∥ z − w ∥ ν ∥ ∇ p D d ( z ) − ∇ p D d ( w )∥ .
Every point and every distinct pair occur under this invertible change of coordinates. Taking suprema in the derivative identities and using q = p + ν q=p+\nu q = p + ν in the pairwise ratio yields
∥ ∇ k Δ j B ∥ ∞ = μ R j q − k ∥ ∇ k D d ∥ ∞ ( 0 ≤ k ≤ p ) , \lVert\nabla^k\Delta_j^B\rVert_\infty=\mu R_j^{q-k}\lVert\nabla^kD_d\rVert_\infty\qquad(0\le k\le p), ∥ ∇ k Δ j B ∥ ∞ = μ R j q − k ∥ ∇ k D d ∥ ∞ ( 0 ≤ k ≤ p ) ,
∥ ∇ p Δ j B ( x ) − ∇ p Δ j B ( y ) ∥ ∥ x − y ∥ ν = μ ∥ ∇ p D d ( z ) − ∇ p D d ( w ) ∥ ∥ z − w ∥ ν ≤ μ C b a s e . (96) \frac{\lVert\nabla^p\Delta_j^B(x)-\nabla^p\Delta_j^B(y)\rVert}{\lVert x-y\rVert^\nu}
=\mu\frac{\lVert\nabla^pD_d(z)-\nabla^pD_d(w)\rVert}{\lVert z-w\rVert^\nu}\le\mu C_{\mathrm{base}}. \tag*{(96)} ∥ x − y ∥ ν ∥ ∇ p Δ j B ( x ) − ∇ p Δ j B ( y )∥ = μ ∥ z − w ∥ ν ∥ ∇ p D d ( z ) − ∇ p D d ( w )∥ ≤ μ C base . (96)
These prove both claims with M k Δ = H Δ = C b a s e M_k^\Delta=H_\Delta=C_{\mathrm{base}} M k Δ = H Δ = C base . If also B j ∈ C p , ν B_j\in C^{p,\nu} B j ∈ C p , ν , the triangle inequality gives, for all x , y ∈ R d x,y\in\mathbb{R}^d x , y ∈ R d ,
∥ ∇ p B j + 1 ( x ) − ∇ p B j + 1 ( y ) ∥ ≤ ∥ ∇ p B j ( x ) − ∇ p B j ( y ) ∥ + H Δ μ ∥ x − y ∥ ν . \lVert\nabla^pB_{j+1}(x)-\nabla^pB_{j+1}(y)\rVert
\le\lVert\nabla^pB_j(x)-\nabla^pB_j(y)\rVert+H_\Delta\mu\lVert x-y\rVert^\nu. ∥ ∇ p B j + 1 ( x ) − ∇ p B j + 1 ( y )∥ ≤ ∥ ∇ p B j ( x ) − ∇ p B j ( y )∥ + H Δ μ ∥ x − y ∥ ν .
Thus B j + 1 ∈ C p , ν B_{j+1}\in C^{p,\nu} B j + 1 ∈ C p , ν .
Hessian and segment-length bounds for uniform convexity
Let B = B L B=B_L B = B L be as in Lemma C.1 , and set C = { c 0 , … , c L } \mathcal{C}=\{c_0,\ldots,c_L\} C = { c 0 , … , c L } and dist ( x , C ) = min i ∥ x − c i ∥ \operatorname{dist}(x,\mathcal{C})=\min_i\lVert x-c_i\rVert dist ( x , C ) = min i ∥ x − c i ∥ .
Lemma C.14 (The Hessian is bounded below by distance to the centers). (a) The final base satisfies
∇ 2 B ( x ) ⪰ μ dist ( x , C ) q − 2 I ( x ∈ R d ) . (97) \nabla^2B(x)\succeq\mu\operatorname{dist}(x,\mathcal{C})^{q-2}I\qquad(x\in\mathbb{R}^d). \tag*{(97)} ∇ 2 B ( x ) ⪰ μ dist ( x , C ) q − 2 I ( x ∈ R d ) . (97)
Proof. • Proof of (a). Keep the full set C \mathcal{C} C fixed and put K ( x ) = μ dist ( x , C ) q − 2 I K(x)=\mu\operatorname{dist}(x,\mathcal{C})^{q-2}I K ( x ) = μ dist ( x , C ) q − 2 I . For every center c i c_i c i , Lemma B.16 (c) in Appx. B gives
∇ 2 ( μ q ∥ x − c i ∥ q ) ⪰ μ ∥ x − c i ∥ q − 2 I ⪰ K ( x ) . \nabla^2\left(\frac{\mu}{q}\lVert x-c_i\rVert^q\right)\succeq\mu\lVert x-c_i\rVert^{q-2}I\succeq K(x). ∇ 2 ( q μ ∥ x − c i ∥ q ) ⪰ μ ∥ x − c i ∥ q − 2 I ⪰ K ( x ) .
In particular ∇ 2 B 0 ⪰ K \nabla^2B_0\succeq K ∇ 2 B 0 ⪰ K . Assume ∇ 2 B j ⪰ K \nabla^2B_j\succeq K ∇ 2 B j ⪰ K ; the displayed bound also gives ∇ 2 N j B ⪰ K \nabla^2N_j^B\succeq K ∇ 2 N j B ⪰ K , since the remaining terms of N j B N_j^B N j B are affine. On S j S_j S j , set s = N j B ( x ) − B j ( x ) s=N_j^B(x)-B_j(x) s = N j B ( x ) − B j ( x ) , ϑ = ψ δ j B ′ ( s ) \vartheta=\psi_{\delta_j^B}'(s) ϑ = ψ δ j B ′ ( s ) , and a = ∇ N j B ( x ) − ∇ B j ( x ) a=\nabla N^{B}_{j}(x)-\nabla B_{j}(x) a = ∇ N j B ( x ) − ∇ B j ( x ) . Lemma B.17 (b) gives 0 ≤ ϑ ≤ 1 0\leq\vartheta\leq1 0 ≤ ϑ ≤ 1 and ψ δ j B ′ ′ ( s ) ≥ 0 \psi^{\prime\prime}_{\delta^{B}_{j}}(s)\geq0 ψ δ j B ′′ ( s ) ≥ 0 . Both inputs have lower curvature K K K , and the smooth maximum’s additional positive semidefinite term can only increase it. The Hessian formula in Lemma B.21 (b) in that appendix gives
∇ 2 B j + 1 ( x ) = ( 1 − ϑ ) ∇ 2 B j ( x ) + ϑ ∇ 2 N j B ( x ) + ψ δ j B ′ ′ ( s ) a a T , \nabla^{2}B_{j+1}(x)=(1-\vartheta)\nabla^{2}B_{j}(x)+\vartheta\nabla^{2}N^{B}_{j}(x)+\psi^{\prime\prime}_{\delta^{B}_{j}}(s)aa^{\mathsf{T}}, ∇ 2 B j + 1 ( x ) = ( 1 − ϑ ) ∇ 2 B j ( x ) + ϑ ∇ 2 N j B ( x ) + ψ δ j B ′′ ( s ) a a T ,
∇ 2 B j + 1 ( x ) − K ( x ) = ( 1 − ϑ ) ( ∇ 2 B j ( x ) − K ( x ) ) + ϑ ( ∇ 2 N j B ( x ) − K ( x ) ) + ψ δ j B ′ ′ ( s ) a a T ⪰ 0. \begin{aligned}
\nabla^{2}B_{j+1}(x)-K(x)
&=(1-\vartheta)\left(\nabla^{2}B_{j}(x)-K(x)\right)\\
&\quad+\vartheta\left(\nabla^{2}N^{B}_{j}(x)-K(x)\right)+\psi^{\prime\prime}_{\delta^{B}_{j}}(s)aa^{\mathsf{T}}\succeq0.
\end{aligned} ∇ 2 B j + 1 ( x ) − K ( x ) = ( 1 − ϑ ) ( ∇ 2 B j ( x ) − K ( x ) ) + ϑ ( ∇ 2 N j B ( x ) − K ( x ) ) + ψ δ j B ′′ ( s ) a a T ⪰ 0.
Each term on the last two lines is positive semidefinite. Hence
∇ 2 B j + 1 = ∇ 2 M δ j B ( B j , N j B ) ⪰ K . \nabla^{2}B_{j+1}=\nabla^{2}M_{\delta^{B}_{j}}(B_{j},N^{B}_{j})\succeq K. ∇ 2 B j + 1 = ∇ 2 M δ j B ( B j , N j B ) ⪰ K .
Outside S j S_{j} S j , including its boundary, Lemma C.11 (a) gives neighborhood equality B j + 1 = B j B_{j+1}=B_{j} B j + 1 = B j , so the same bound holds. Induction and B = B L B=B_{L} B = B L prove Eq. (C.49).
Lemma C.15 (A segment spends a fixed fraction away from all centers). (a) For every pair of distinct points x , y ∈ R d \boldsymbol{x},\boldsymbol{y}\in\mathbb{R}^{d} x , y ∈ R d , set ℓ = ∥ y − x ∥ \ell=\|\boldsymbol{y}-\boldsymbol{x}\| ℓ = ∥ y − x ∥ . The segment [ x , y ] [\boldsymbol{x},\boldsymbol{y}] [ x , y ] has a measurable subset of length at least ℓ 2 \frac{\ell}{2} 2 ℓ such that every point z \boldsymbol{z} z in that subset satisfies
dist ( z , C ) ≥ ℓ 16 . \operatorname{dist}(\boldsymbol{z},\mathcal{C})\geq\frac{\ell}{16}. dist ( z , C ) ≥ 16 ℓ .
Proof. • Proof of (a). Fix distinct x , y ∈ R d \boldsymbol{x},\boldsymbol{y}\in\mathbb{R}^{d} x , y ∈ R d , set ℓ = ∥ y − x ∥ \ell=\|\boldsymbol{y}-\boldsymbol{x}\| ℓ = ∥ y − x ∥ and ρ ℓ = ℓ 16 \rho_{\ell}=\frac{\ell}{16} ρ ℓ = 16 ℓ , and let u = y − x ℓ \boldsymbol{u}=\frac{\boldsymbol{y}-\boldsymbol{x}}{\ell} u = ℓ y − x . Parameterize the supporting line by x + t u \boldsymbol{x}+t\boldsymbol{u} x + t u ; its speed is one, and the segment corresponds to 0 ≤ t ≤ ℓ 0\leq t\leq\ell 0 ≤ t ≤ ℓ .
Step 1. Bound the separation of three center projections. Define
α i : = ⟨ c i − x , u ⟩ , z i : = c i − x − α i u , I : = { i : ∥ z i ∥ ≤ ρ ℓ } . \alpha_{i}:=\langle\boldsymbol{c}_{i}-\boldsymbol{x},\boldsymbol{u}\rangle,\qquad\boldsymbol{z}_{i}:=\boldsymbol{c}_{i}-\boldsymbol{x}-\alpha_{i}\boldsymbol{u},\qquad I:=\{i:\|\boldsymbol{z}_{i}\|\leq\rho_{\ell}\}. α i := ⟨ c i − x , u ⟩ , z i := c i − x − α i u , I := { i : ∥ z i ∥ ≤ ρ ℓ } .
The vector z i \boldsymbol{z}_{i} z i points from the projection x + α i u \boldsymbol{x}+\alpha_{i}\boldsymbol{u} x + α i u to the center c i \boldsymbol{c}_{i} c i , as illustrated in Figure 2 . Thus z i ⊥ u \boldsymbol{z}_{i}\perp\boldsymbol{u} z i ⊥ u , and only centers indexed by I I I can have their radius-ρ ℓ \rho_{\ell} ρ ℓ balls meet the line. For i < j < k i<j<k i < j < k in I I I , orthogonality of the phase displacements h a \boldsymbol{h}_{a} h a gives
Figure 2. Geometry for Lemma C.15 , with ρ ℓ = ℓ 16 \rho_\ell= \frac{\ell}{16} ρ ℓ = 16 ℓ . Left: orthogonal center increments at illustrative scales and radius-ρ ℓ \rho_\ell ρ ℓ balls, with gold showing the parts of the segment inside their union. Right: line projections p i = x + α i u p_i=x+\alpha_i u p i = x + α i u and residual z j = c j − p j z_j=c_j-p_j z j = c j − p j . The three projections are pairwise more than 2 ρ ℓ 2\rho_\ell 2 ρ ℓ apart, while the middle center lies outside the radius-ρ ℓ \rho_\ell ρ ℓ strip. These fixed-radius balls differ from the replacement neighborhoods S j S_j S j .
⟨ c i − c j , c k − c j ⟩ = ⟨ − ∑ a = i j − 1 h a , ∑ b = j k − 1 h b ⟩ = 0. \left\langle\boldsymbol{c}_{i}-\boldsymbol{c}_{j},\boldsymbol{c}_{k}-\boldsymbol{c}_{j}\right\rangle
=\left\langle-\sum_{a=i}^{j-1}\boldsymbol{h}_{a},\sum_{b=j}^{k-1}\boldsymbol{h}_{b}\right\rangle=0. ⟨ c i − c j , c k − c j ⟩ = ⟨ − a = i ∑ j − 1 h a , b = j ∑ k − 1 h b ⟩ = 0.
Substitute c i − c j = ( α i − α j ) u + ( z i − z j ) \boldsymbol{c}_{i}-\boldsymbol{c}_{j}=(\alpha_{i}-\alpha_{j})\boldsymbol{u}+(\boldsymbol{z}_{i}-\boldsymbol{z}_{j}) c i − c j = ( α i − α j ) u + ( z i − z j ) and its counterpart with k k k in place of i i i . Since the residuals are orthogonal to u \boldsymbol{u} u and ∥ u ∥ = 1 \|\boldsymbol{u}\|=1 ∥ u ∥ = 1 ,
0 = ⟨ c i − c j , c k − c j ⟩ = ( α i − α j ) ( α k − α j ) + ⟨ z i − z j , z k − z j ⟩ , ∣ ( α i − α j ) ( α k − α j ) ∣ = ∣ ⟨ z i − z j , z k − z j ⟩ ∣ ≤ ∥ z i − z j ∥ ∥ z k − z j ∥ ≤ ( ∥ z i ∥ + ∥ z j ∥ ) ( ∥ z k ∥ + ∥ z j ∥ ) ≤ ( 2 ρ ℓ ) ( 2 ρ ℓ ) = 4 ρ ℓ 2 . \begin{aligned}
0&=\left\langle\boldsymbol{c}_{i}-\boldsymbol{c}_{j},\boldsymbol{c}_{k}-\boldsymbol{c}_{j}\right\rangle\\
&=(\alpha_{i}-\alpha_{j})(\alpha_{k}-\alpha_{j})+\left\langle\boldsymbol{z}_{i}-\boldsymbol{z}_{j},\boldsymbol{z}_{k}-\boldsymbol{z}_{j}\right\rangle,\\
\left|(\alpha_{i}-\alpha_{j})(\alpha_{k}-\alpha_{j})\right|
&=\left|\left\langle\boldsymbol{z}_{i}-\boldsymbol{z}_{j},\boldsymbol{z}_{k}-\boldsymbol{z}_{j}\right\rangle\right|\\
&\leq\|\boldsymbol{z}_{i}-\boldsymbol{z}_{j}\|\|\boldsymbol{z}_{k}-\boldsymbol{z}_{j}\|\\
&\leq(\|\boldsymbol{z}_{i}\|+\|\boldsymbol{z}_{j}\|)(\|\boldsymbol{z}_{k}\|+\|\boldsymbol{z}_{j}\|)\\
&\leq(2\rho_{\ell})(2\rho_{\ell})=4\rho_{\ell}^{2}.
\end{aligned} 0 ∣ ( α i − α j ) ( α k − α j ) ∣ = ⟨ c i − c j , c k − c j ⟩ = ( α i − α j ) ( α k − α j ) + ⟨ z i − z j , z k − z j ⟩ , = ∣ ⟨ z i − z j , z k − z j ⟩ ∣ ≤ ∥ z i − z j ∥∥ z k − z j ∥ ≤ ( ∥ z i ∥ + ∥ z j ∥ ) ( ∥ z k ∥ + ∥ z j ∥ ) ≤ ( 2 ρ ℓ ) ( 2 ρ ℓ ) = 4 ρ ℓ 2 .
The inequalities are Cauchy–Schwarz, the triangle inequality, and membership in I I I , respectively. Consequently no three projections from I I I can be pairwise more than 2 ρ ℓ 2\rho_{\ell} 2 ρ ℓ apart: ordering their center indices would contradict this bound.
Step 2. Cover the projections by at most two short intervals. If I \mathcal{I} I is empty, no center ball meets the line. Otherwise let a 1 a_1 a 1 be the smallest projection, and cover all projections in [ a 1 , a 1 + 2 ρ ℓ ] [a_1,a_1+2\rho_\ell] [ a 1 , a 1 + 2 ρ ℓ ] . If any remain, let a 2 a_2 a 2 be the smallest remaining projection. Every remaining projection lies in [ a 2 , a 2 + 2 ρ ℓ ] [a_2,a_2+2\rho_\ell] [ a 2 , a 2 + 2 ρ ℓ ] ; a projection larger than a 2 + 2 ρ ℓ a_2+2\rho_\ell a 2 + 2 ρ ℓ would form three pairwise separated projections with a 1 , a 2 a_1,a_2 a 1 , a 2 , contrary to Step 1.
Step 3. Bound the length within distance ρ ℓ \rho_\ell ρ ℓ of a center. If ∥ x + t u − c i ∥ < ρ ℓ \lVert x+tu-c_i\rVert<\rho_\ell ∥ x + t u − c i ∥ < ρ ℓ , then i ∈ I i\in\mathcal{I} i ∈ I and ∣ t − α i ∣ < ρ ℓ \lvert t-\alpha_i\rvert<\rho_\ell ∣ t − α i ∣ < ρ ℓ . Each interval from Step 2 therefore contributes only within
[ a b − ρ ℓ , a b + 3 ρ ℓ ] , [a_b-\rho_\ell,a_b+3\rho_\ell], [ a b − ρ ℓ , a b + 3 ρ ℓ ] ,
an interval of length 4 ρ ℓ 4\rho_\ell 4 ρ ℓ .
There are at most two such intervals, so the part of [ x , y ] [x,y] [ x , y ] at distance less than ρ ℓ \rho_\ell ρ ℓ from C C C has length at most 8 ρ ℓ 8\rho_\ell 8 ρ ℓ . Its complement is measurable and has length at least
ℓ − 8 ρ ℓ = ℓ 2 . \ell-8\rho_\ell=\frac{\ell}{2}. ℓ − 8 ρ ℓ = 2 ℓ .
Every point in this complement has distance at least ρ ℓ = ℓ 16 \rho_\ell=\frac{\ell}{16} ρ ℓ = 16 ℓ from C C C , as required.
Supporting Proofs for the Phase Functions
Constructing and verifying the convex phase functions
Requirements for one phase
The uniformly convex base from Appx. C gives the geometry that the phase functions must preserve. Lemma C.1 (b) gives
b j : = ∇ B ( c j + 1 ) − ∇ B ( c j ) = μ R j q − 2 ( c j + 1 − c j ) . (98) b_j := \nabla B(c_{j+1})-\nabla B(c_j)=\mu R_j^{q-2}(c_{j+1}-c_j). \tag*{(98)} b j := ∇ B ( c j + 1 ) − ∇ B ( c j ) = μ R j q − 2 ( c j + 1 − c j ) . (98)
For F = B + ℓ i n i t + ∑ k A k F=B+\ell_{\mathrm{init}}+\sum_k A_k F = B + ℓ init + ∑ k A k , with ℓ i n i t ( x ) = − s 0 ⟨ v 0 , x ⟩ \ell_{\mathrm{init}}(x)=-s_0\langle v_0,x\rangle ℓ init ( x ) = − s 0 ⟨ v 0 , x ⟩ , the phase conditions are:
R1. Center gradients. The target gradients are
∇ F ( c j ) = − s j v j , ∇ F ( c j + 1 ) = − s j + 1 v j + 1 (99) \nabla F(c_j)=-s_jv_j,\qquad\nabla F(c_{j+1})=-s_{j+1}v_{j+1} \tag*{(99)} ∇ F ( c j ) = − s j v j , ∇ F ( c j + 1 ) = − s j + 1 v j + 1 (99)
with ∇ A j ( c j ) = 0 \nabla A_j(c_j)=0 ∇ A j ( c j ) = 0 and A j = I j A_j=I_j A j = I j near c j + 1 c_{j+1} c j + 1 , where I j ( x ) = ⟨ g j , x − c j + 1 ⟩ I_j(x)=\langle g_j,x-c_{j+1}\rangle I j ( x ) = ⟨ g j , x − c j + 1 ⟩ and
g j = s j v j − s j + 1 v j + 1 − b j . (100) g_j=s_jv_j-s_{j+1}v_{j+1}-b_j. \tag*{(100)} g j = s j v j − s j + 1 v j + 1 − b j . (100)
Lemma E.4 (b) in Appx. E derives this cancellation under R6. We use s L v L = 0 s_Lv_L=0 s L v L = 0 in the terminal phase.
R2. Optimality gap on chain prefixes. For the subspaces W j , b W_{j,b} W j , b containing early evaluation points,
r j : = A j − I j = 0 near c j + 1 , r_j:=A_j-I_j=0\qquad\text{near }c_{j+1}, r j := A j − I j = 0 near c j + 1 ,
r j ( x ) ≥ e j : = 7 δ j > 0 ( x ∈ W j , b ) . (101) r_j(x)\ge e_j:=7\delta_j>0\qquad(x\in W_{j,b}). \tag*{(101)} r j ( x ) ≥ e j := 7 δ j > 0 ( x ∈ W j , b ) . (101)
Appx. E.2.4 uses this excess and the orthogonal center gradient to prove F ( x ) − min F ≥ 7 δ j F(x)-\min F\ge7\delta_j F ( x ) − min F ≥ 7 δ j .
R3. Convexity: A j A_j A j is convex, so Lemma B.20 ensures that adding it preserves the base’s Bregman lower bound.
R4. Oracle consistency: future base changes miss W j , b W_{j,b} W j , b , and the current phase’s oracle output uses at most one new direction.
R5. Smoothness: A j ∈ C p , ν A_j\in C^{p,\nu} A j ∈ C p , ν and ∥ ∇ p A j ( x ) − ∇ p A j ( y ) ∥ ≤ C A λ ∥ x − y ∥ ν \|\nabla^pA_j(x)-\nabla^pA_j(y)\|\le C_A\lambda\|x-y\|^\nu ∥ ∇ p A j ( x ) − ∇ p A j ( y ) ∥ ≤ C A λ ∥ x − y ∥ ν for all x , y ∈ R d x,y\in\mathbb{R}^d x , y ∈ R d , with C A C_A C A independent of m m m .
R6. Other phases: ∇ A k ( c j + 1 ) = ∇ A k ( c j ) \nabla A_k(c_{j+1})=\nabla A_k(c_j) ∇ A k ( c j + 1 ) = ∇ A k ( c j ) for k ≠ j k\ne j k = j . Appx. E.2 uses E j ⊥ E k E_j\perp E_k E j ⊥ E k for j ≠ k j\ne k j = k , h j ∈ E j h_j\in E_j h j ∈ E j , and v k + 1 ∈ E k + 1 v_{k+1}\in E_{k+1} v k + 1 ∈ E k + 1 when the next direction is present. For k = j − 1 k=j-1 k = j − 1 , the gradient remains g j − 1 g_{j-1} g j − 1 along the completed-center segment by Lemma 4.4 (c).
Throughout, p ≥ 2 p\ge2 p ≥ 2 , 0 < ν ≤ 1 0<\nu\le1 0 < ν ≤ 1 , q = p + ν q=p+\nu q = p + ν , and q ∗ : = q q − 1 q^*:=\frac{q}{q-1} q ∗ := q − 1 q . Define ∥ a ∥ r = ( ∑ i ∣ a i ∣ r ) 1 / r \|a\|_r=\left(\sum_i|a_i|^r\right)^{1/r} ∥ a ∥ r = ( ∑ i ∣ a i ∣ r ) 1/ r ; unqualified norms are Euclidean and derivative norms are operator norms.
Adapt the classical power chain to one phase
The phase chain follows the construction of [20 Sec. 5.1, Eq. (5.4)] , whose definition, matrix estimate, and minimizer formula are restated in Lemma D.1 below. The same chain structure appears in [4 Sec. 5.1] .2
Lemma D.1 (Power chain [20 Lemmas 5.1–5.2 and their proofs] ). Let p ≥ 2 p \ge2 p ≥ 2 be an integer, 0 < ν ≤ 1 0 < \nu\le1 0 < ν ≤ 1 , q = p + ν q = p+\nu q = p + ν , and let 2 ≤ k ≤ d 2 \le k \le d 2 ≤ k ≤ d be integers. Define
f k ( a ) : = 1 q [ ∑ i = 1 k − 1 ∣ a i − a i + 1 ∣ q + ∑ i = k d ∣ a i ∣ q ] − a 1 ( a ∈ R d ) . f_k(a) := \frac{1}{q}\left[\sum_{i=1}^{k-1}|a_i-a_{i+1}|^q+\sum_{i=k}^{d}|a_i|^q\right]-a_1 \qquad(a \in\mathbb{R}^d). f k ( a ) := q 1 [ i = 1 ∑ k − 1 ∣ a i − a i + 1 ∣ q + i = k ∑ d ∣ a i ∣ q ] − a 1 ( a ∈ R d ) . Let U k U_k U k have diagonal entries 1 1 1 , superdiagonal entries − 1 -1 − 1 , and all other entries zero. Then
A k = diag ( U k , I d − k ) , f k ( a ) = 1 q ∥ A k a ∥ q q − a 1 , ∥ A k ∥ 2 → 2 ≤ 2. (102) A_k = \operatorname{diag}(U_k,I_{d-k}), \qquad f_k(a)=\frac{1}{q}\lVert A_k a\rVert_q^q-a_1, \qquad\lVert A_k\rVert_{2\to2}\le2. \tag*{(102)} A k = diag ( U k , I d − k ) , f k ( a ) = q 1 ∥ A k a ∥ q q − a 1 , ∥ A k ∥ 2 → 2 ≤ 2. (102) For k = d k=d k = d , the second diagonal block is absent. The function f k f_k f k has a unique global minimizer a k ∗ a_k^* a k ∗ and
min a ∈ R d f k ( a ) = − q − 1 q k , ∥ a k ∗ ∥ < ( k + 1 ) 3 2 3 . \min_{a\in\mathbb{R}^d}f_k(a)=-\frac{q-1}{q}k, \qquad\lVert a_k^*\rVert<\frac{(k+1)^{\frac{3}{2}}}{\sqrt{3}}. a ∈ R d min f k ( a ) = − q q − 1 k , ∥ a k ∗ ∥ < 3 ( k + 1 ) 2 3 . The minimizer satisfies
( a k ∗ ) i = ( k − i + 1 ) + ( 1 ≤ i ≤ d ) , A k a k ∗ = ( 1 , … , 1 ⏟ k , 0 , … , 0 ) T . (103) (a_k^*)_i=(k-i+1)_+ \qquad(1\le i\le d), \qquad A_k a_k^*=\left(\underbrace{1,\ldots,1}_{k},0,\ldots,0\right)^{\mathsf T}. \tag*{(103)} ( a k ∗ ) i = ( k − i + 1 ) + ( 1 ≤ i ≤ d ) , A k a k ∗ = k 1 , … , 1 , 0 , … , 0 T . (103) where ( a ) + : = max { a , 0 } (a)_+:=\max\{a,0\} ( a ) + := max { a , 0 } .
Definition D.2 (p t h p^{th} p t h -order zero-chain [12 Sec. 3.1, Definition 3] ). For a tensor Q Q Q , let supp Q \operatorname{supp}Q supp Q contain every coordinate index appearing in a nonzero entry of Q Q Q ; for a vector this is its usual support. For an integer p ≥ 1 p\ge1 p ≥ 1 , a function f : R d → R f:\mathbb{R}^d\to\mathbb{R} f : R d → R with derivatives through order p p p is a p t h p^{th} p t h -order zero-chain if, for every x ∈ R d x\in\mathbb{R}^d x ∈ R d and 1 ≤ i ≤ d 1\le i\le d 1 ≤ i ≤ d ,
supp x ⊆ { 1 , … , i − 1 } ⟹ ⋃ r = 1 p supp ∇ r f ( x ) ⊆ { 1 , … , i } . \operatorname{supp}x\subseteq\{1,\ldots,i-1\}\quad\Longrightarrow\quad\bigcup_{r=1}^{p}\operatorname{supp}\nabla^r f(x)\subseteq\{1,\ldots,i\}. supp x ⊆ { 1 , … , i − 1 } ⟹ r = 1 ⋃ p supp ∇ r f ( x ) ⊆ { 1 , … , i } . If this holds for every positive integer p p p , it is called a zero-chain.
Definition (Phase chain and endpoint). Fix 0 ≤ j < L 0\le j<L 0 ≤ j < L , an integer m ≥ 1 m\ge1 m ≥ 1 , and λ , τ j > 0 \lambda,\tau_j>0 λ , τ j > 0 . Let v j , u j , 1 , … , u j , m v_j,u_{j,1},\ldots,u_{j,m} v j , u j , 1 , … , u j , m be orthonormal, and put
E j : = span { v j , u j , 1 , … , u j , m } , w = t v j + ∑ i = 1 m x i u j , i ∈ E j . E_j:=\operatorname{span}\{v_j,u_{j,1},\ldots,u_{j,m}\}, \qquad w=tv_j+\sum_{i=1}^{m}x_i u_{j,i}\in E_j. E j := span { v j , u j , 1 , … , u j , m } , w = t v j + i = 1 ∑ m x i u j , i ∈ E j . Matrix products use coordinates in the ordered basis ( v j , u j , 1 , … , u j , m ) (v_j,u_{j,1},\ldots,u_{j,m}) ( v j , u j , 1 , … , u j , m ) . Thus
D m w : = D m ( t , x 1 , … , x m ) T , e 0 = ( 1 , 0 , … , 0 ) T , D_mw:=D_m(t,x_1,\ldots,x_m)^{\mathsf T}, \qquad e_0=(1,0,\ldots,0)^{\mathsf T}, D m w := D m ( t , x 1 , … , x m ) T , e 0 = ( 1 , 0 , … , 0 ) T , where e 0 e_0 e 0 represents v j v_j v j . Define the residual matrix by
D m = ( m − 1 2 − 1 0 ⋯ 0 0 1 − 1 ⋱ ⋮ ⋮ ⋱ ⋱ ⋱ 0 0 ⋯ 0 1 − 1 0 ⋯ ⋯ 0 1 ) , D m ( t x 1 ⋮ x m ) = ( t m − x 1 x 1 − x 2 ⋮ x m − 1 − x m x m ) . (104) D_m=
\begin{pmatrix}
m^{-\frac12} & -1 & 0 & \cdots& 0\\
0 & 1 & -1 & \ddots& \vdots\\
\vdots& \ddots& \ddots& \ddots& 0\\
0 & \cdots& 0 & 1 & -1\\
0 & \cdots& \cdots& 0 & 1
\end{pmatrix},
\qquad
D_m
\begin{pmatrix}
t\\x_1\\\vdots\\x_m
\end{pmatrix}
=
\begin{pmatrix}
\frac{t}{\sqrt{m}}-x_1\\
x_1-x_2\\
\vdots\\
x_{m-1}-x_m\\
x_m
\end{pmatrix}. \tag*{(104)} D m = m − 2 1 0 ⋮ 0 0 − 1 1 ⋱ ⋯ ⋯ 0 − 1 ⋱ 0 ⋯ ⋯ ⋱ ⋱ 1 0 0 ⋮ 0 − 1 1 , D m t x 1 ⋮ x m = m t − x 1 x 1 − x 2 ⋮ x m − 1 − x m x m . (104)
For m = 1 m=1 m = 1 , D m = ( 1 − 1 0 1 ) D_m=\begin{pmatrix}1&-1\\0&1\end{pmatrix} D m = ( 1 0 − 1 1 ) . With ϕ ( a ) = ∣ a ∣ q q \phi(a)=\frac{|a|^q}{q} ϕ ( a ) = q ∣ a ∣ q , define the chain on the ambient space by
Q j ( x ) : = λ q ∥ D m P E j x ∥ q q ( x ∈ R d ) . Q_j(x):=\frac{\lambda}{q}\lVert D_mP_{E_j}x\rVert_q^q \qquad(x\in\mathbb{R}^d). Q j ( x ) := q λ ∥ D m P E j x ∥ q q ( x ∈ R d ) .
Thus Q j ( w ) = λ ∑ i = 0 m ϕ ( ( D m w ) i ) Q_j(w)=\lambda\sum_{i=0}^{m}\phi((D_mw)_i) Q j ( w ) = λ ∑ i = 0 m ϕ (( D m w ) i ) on E j E_j E j ; rows are indexed 0 , … , m 0,\ldots,m 0 , … , m . Define the endpoint by
t j ⋆ : = m ( m + 1 ) τ j , h j : = t j ⋆ v j + ∑ i = 1 m ( m − i + 1 ) τ j u j , i , R j : = ∥ h j ∥ . (105) t_j^\star:=\sqrt{m}(m+1)\tau_j,\qquad h_j:=t_j^\star v_j+\sum_{i=1}^{m}(m-i+1)\tau_j u_{j,i},\qquad R_j:=\lVert h_j\rVert. \tag*{(105)} t j ⋆ := m ( m + 1 ) τ j , h j := t j ⋆ v j + i = 1 ∑ m ( m − i + 1 ) τ j u j , i , R j := ∥ h j ∥ . (105)
Lemma D.3 (The chain matrix and its endpoint). For the chain Q j Q_j Q j and endpoint defined above:
(a) Residuals and gradient. D m D_m D m is invertible, and
D m ( t j ⋆ m τ j ⋮ τ j ) = τ j 1 , D m T 1 = e 0 m . (106) D_m\begin{pmatrix}t_j^\star\\m\tau_j\\ \vdots\\ \tau_j\end{pmatrix}=\tau_j\mathbf{1},\qquad D_m^{\mathsf T}\mathbf{1}=\frac{e_0}{\sqrt{m}}. \tag*{(106)} D m t j ⋆ m τ j ⋮ τ j = τ j 1 , D m T 1 = m e 0 . (106) ∇ Q j ( h j ) = λ τ j q − 1 m v j . \nabla Q_j(h_j)=\frac{\lambda\tau_j^{q-1}}{\sqrt{m}}v_j. ∇ Q j ( h j ) = m λ τ j q − 1 v j . (b) Operator norm.
∥ D m ∥ 2 → 2 ≤ 2 , ∥ D m w ∥ r ≤ 2 ∥ w ∥ ( w ∈ E j , r ≥ 2 ) . (107) \lVert D_m\rVert_{2\to2}\le2,\qquad\lVert D_mw\rVert_r\le2\lVert w\rVert\qquad(w\in E_j,\ r\ge2). \tag*{(107)} ∥ D m ∥ 2 → 2 ≤ 2 , ∥ D m w ∥ r ≤ 2 ∥ w ∥ ( w ∈ E j , r ≥ 2 ) . (107) (c) Endpoint lengths.
R j 2 = τ j 2 m ( m + 1 ) ( 8 m + 7 ) 6 , (108) R_j^2=\tau_j^2\frac{m(m+1)(8m+7)}{6}, \tag*{(108)} R j 2 = τ j 2 6 m ( m + 1 ) ( 8 m + 7 ) , (108) τ j m 3 2 ≤ R j ≤ 5 τ j m 3 2 , 3 2 R j ≤ t j ⋆ ≤ R j , ∑ i = 1 m ( m − i + 1 ) τ j ≤ m 2 τ j . (109) \tau_jm^{\frac{3}{2}}\le R_j\le\sqrt{5}\tau_jm^{\frac{3}{2}},\qquad\frac{\sqrt{3}}{2}R_j\le t_j^\star\le R_j,\qquad\sum_{i=1}^{m}(m-i+1)\tau_j\le m^2\tau_j. \tag*{(109)} τ j m 2 3 ≤ R j ≤ 5 τ j m 2 3 , 2 3 R j ≤ t j ⋆ ≤ R j , i = 1 ∑ m ( m − i + 1 ) τ j ≤ m 2 τ j . (109) R j 5 ≤ ∥ h j − t j ⋆ v j ∥ ≤ R j 2 . \frac{R_j}{\sqrt{5}}\le\lVert h_j-t_j^\star v_j\rVert\le\frac{R_j}{2}. 5 R j ≤ ∥ h j − t j ⋆ v j ∥ ≤ 2 R j . (d) One additional direction. For every integer 0 ≤ k < m 0\le k<m 0 ≤ k < m and w ∈ span { v j , u j , 1 , … , u j , k } w\in\operatorname{span}\{v_j,u_{j,1},\ldots,u_{j,k}\} w ∈ span { v j , u j , 1 , … , u j , k } ,
Q j ( w ) , ∇ Q j ( w ) , … , ∇ p Q j ( w ) depend only on v j , u j , 1 , … , u j , k + 1 , m , λ , q , w . Q_j(w),\ \nabla Q_j(w),\ldots,\nabla^pQ_j(w)\quad\text{depend only on }v_j,u_{j,1},\ldots,u_{j,k+1},m,\lambda,q,w. Q j ( w ) , ∇ Q j ( w ) , … , ∇ p Q j ( w ) depend only on v j , u j , 1 , … , u j , k + 1 , m , λ , q , w . These are full ambient derivative tensors.
Proof. • Proof of (a). Define v j , i = ( D m w ) i v_{j,i}=(D_mw)_i v j , i = ( D m w ) i . Consecutive residuals contain x i x_i x i with opposite signs, so
∂ Q j ∂ x i ( w ) = λ [ ϕ ′ ( v j , i ) − ϕ ′ ( v j , i − 1 ) ] ( 1 ≤ i ≤ m ) , \frac{\partial Q_j}{\partial x_i}(w)=\lambda[\phi'(v_{j,i})-\phi'(v_{j,i-1})]\qquad(1\le i\le m), ∂ x i ∂ Q j ( w ) = λ [ ϕ ′ ( v j , i ) − ϕ ′ ( v j , i − 1 )] ( 1 ≤ i ≤ m ) ,
∂ Q j ∂ t ( w ) = λ m ϕ ′ ( v j , 0 ) . \frac{\partial Q_j}{\partial t}(w)=\frac{\lambda}{\sqrt{m}}\phi'(v_{j,0}). ∂ t ∂ Q j ( w ) = m λ ϕ ′ ( v j , 0 ) .
Equal residuals therefore cancel every link-gradient component. Solving for a common value τ j \tau_j τ j gives
x m = τ j , x i − x i + 1 = τ j ( 1 ≤ i < m ) ⟹ x i = ( m − i + 1 ) τ j ( 1 ≤ i ≤ m ) , x_m=\tau_j,\qquad x_i-x_{i+1}=\tau_j\ (1\le i<m)\quad\Longrightarrow\quad x_i=(m-i+1)\tau_j\ (1\le i\le m), x m = τ j , x i − x i + 1 = τ j ( 1 ≤ i < m ) ⟹ x i = ( m − i + 1 ) τ j ( 1 ≤ i ≤ m ) ,
t m − x 1 = τ j ⟹ t = m ( m + 1 ) τ j = t j ⋆ . \frac{t}{\sqrt{m}}-x_1=\tau_j\quad\Longrightarrow\quad t=\sqrt{m}(m+1)\tau_j=t_j^\star. m t − x 1 = τ j ⟹ t = m ( m + 1 ) τ j = t j ⋆ .
This is exactly h j h_j h j in Eq. (D.8), so
D m h j = τ j 1 , ∇ Q j ( h j ) = λ m ϕ ′ ( τ j ) v j = λ τ j q − 1 m v j . D_m h_j=\tau_j\mathbf{1},\qquad\nabla Q_j(h_j)=\frac{\lambda}{\sqrt{m}}\phi'(\tau_j)v_j=\frac{\lambda\tau_j^{q-1}}{\sqrt{m}}v_j. D m h j = τ j 1 , ∇ Q j ( h j ) = m λ ϕ ′ ( τ j ) v j = m λ τ j q − 1 v j .
To represent full ambient derivatives, define the row vectors by
a j , 0 = v j m − u j , 1 , a j , i = u j , i − u j , i + 1 ( 1 ≤ i < m ) , a j , m = u j , m . a_{j,0}=\frac{v_j}{\sqrt{m}}-u_{j,1},\qquad a_{j,i}=u_{j,i}-u_{j,i+1}\ (1\le i<m),\qquad a_{j,m}=u_{j,m}. a j , 0 = m v j − u j , 1 , a j , i = u j , i − u j , i + 1 ( 1 ≤ i < m ) , a j , m = u j , m .
Then ( D m P E j x ) i = ⟨ a j , i , x ⟩ (D_mP_{E_j}x)_i=\langle a_{j,i},x\rangle ( D m P E j x ) i = ⟨ a j , i , x ⟩ , and the link vectors cancel consecutively:
∑ i = 0 m a j , i = v j m − u j , 1 + ∑ i = 1 m − 1 ( u j , i − u j , i + 1 ) + u j , m = v j m − u j , 1 + ( u j , 1 − u j , m ) + u j , m = v j m . \begin{aligned}
\sum_{i=0}^{m}a_{j,i}
&=\frac{v_j}{\sqrt{m}}-u_{j,1}+\sum_{i=1}^{m-1}(u_{j,i}-u_{j,i+1})+u_{j,m}\\
&=\frac{v_j}{\sqrt{m}}-u_{j,1}+(u_{j,1}-u_{j,m})+u_{j,m}
=\frac{v_j}{\sqrt{m}}.
\end{aligned} i = 0 ∑ m a j , i = m v j − u j , 1 + i = 1 ∑ m − 1 ( u j , i − u j , i + 1 ) + u j , m = m v j − u j , 1 + ( u j , 1 − u j , m ) + u j , m = m v j .
For m = 1 m=1 m = 1 , the middle sum is empty and the same cancellation holds. This proves the second identity in Eq. (D.9). The triangular matrix D m D_m D m has nonzero diagonal, so it is invertible.
Proof of (b). The matrix factorization below reduces the norm bound to Eq. (D.5) applied to U m + 1 U_{m+1} U m + 1 . Put S m : = diag ( m − 1 / 2 , 1 , … , 1 ) S_m:=\operatorname{diag}(m^{-1/2},1,\ldots,1) S m := diag ( m − 1/2 , 1 , … , 1 ) . Then
D m = U m + 1 S m , ∥ S m ∥ 2 → 2 = 1 , D_m=U_{m+1}S_m,\qquad\lVert S_m\rVert_{2\to2}=1, D m = U m + 1 S m , ∥ S m ∥ 2 → 2 = 1 ,
∥ D m ∥ 2 → 2 ≤ ∥ U m + 1 ∥ 2 → 2 ∥ S m ∥ 2 → 2 ≤ 2 , \lVert D_m\rVert_{2\to2}\le\lVert U_{m+1}\rVert_{2\to2}\lVert S_m\rVert_{2\to2}\le2, ∥ D m ∥ 2 → 2 ≤ ∥ U m + 1 ∥ 2 → 2 ∥ S m ∥ 2 → 2 ≤ 2 ,
∥ D m w ∥ r ≤ ∥ D m w ∥ 2 ≤ 2 ∥ w ∥ ( r ≥ 2 ) . \lVert D_mw\rVert_r\le\lVert D_mw\rVert_2\le2\lVert w\rVert\qquad(r\ge2). ∥ D m w ∥ r ≤ ∥ D m w ∥ 2 ≤ 2 ∥ w ∥ ( r ≥ 2 ) .
The last line uses the comparison of ℓ r \ell_r ℓ r and Euclidean norms.
( t j ⋆ ) 2 = m ( m + 1 ) 2 τ j 2 , (t_j^\star)^2=m(m+1)^2\tau_j^2, ( t j ⋆ ) 2 = m ( m + 1 ) 2 τ j 2 ,
∥ h j − t j ⋆ v j ∥ 2 = τ j 2 ∑ a = 1 m a 2 = m ( m + 1 ) ( 2 m + 1 ) 6 τ j 2 . \lVert h_j-t_j^\star v_j\rVert^2=\tau_j^2\sum_{a=1}^{m}a^2=\frac{m(m+1)(2m+1)}{6}\tau_j^2. ∥ h j − t j ⋆ v j ∥ 2 = τ j 2 a = 1 ∑ m a 2 = 6 m ( m + 1 ) ( 2 m + 1 ) τ j 2 .
Hence
R j 2 = ( t j ⋆ ) 2 + ∥ h j − t j ⋆ v j ∥ 2 = τ j 2 [ m ( m + 1 ) 2 + m ( m + 1 ) ( 2 m + 1 ) 6 ] = τ j 2 m ( m + 1 ) 6 [ 6 ( m + 1 ) + ( 2 m + 1 ) ] = m ( m + 1 ) ( 8 m + 7 ) 6 τ j 2 , \begin{aligned}
R_j^2
&=(t_j^\star)^2+\lVert h_j-t_j^\star v_j\rVert^2\\
&=\tau_j^2\left[m(m+1)^2+\frac{m(m+1)(2m+1)}{6}\right]\\
&=\frac{\tau_j^2m(m+1)}{6}[6(m+1)+(2m+1)]\\
&=\frac{m(m+1)(8m+7)}{6}\tau_j^2,
\end{aligned} R j 2 = ( t j ⋆ ) 2 + ∥ h j − t j ⋆ v j ∥ 2 = τ j 2 [ m ( m + 1 ) 2 + 6 m ( m + 1 ) ( 2 m + 1 ) ] = 6 τ j 2 m ( m + 1 ) [ 6 ( m + 1 ) + ( 2 m + 1 )] = 6 m ( m + 1 ) ( 8 m + 7 ) τ j 2 ,
R j 2 τ j 2 m 3 = ( m + 1 ) ( 8 m + 7 ) 6 m 2 = 8 m 2 + 15 m + 7 6 m 2 = 4 3 + 5 2 m + 7 6 m 2 ∈ [ 1 , 5 ] , \begin{aligned}
\frac{R_j^2}{\tau_j^2m^3}
&=\frac{(m+1)(8m+7)}{6m^2}
=\frac{8m^2+15m+7}{6m^2}\\
&=\frac{4}{3}+\frac{5}{2m}+\frac{7}{6m^2}\in[1,5],
\end{aligned} τ j 2 m 3 R j 2 = 6 m 2 ( m + 1 ) ( 8 m + 7 ) = 6 m 2 8 m 2 + 15 m + 7 = 3 4 + 2 m 5 + 6 m 2 7 ∈ [ 1 , 5 ] ,
∥ h j − t j ⋆ v j ∥ 2 ( t j ⋆ ) 2 = 2 m + 1 6 ( m + 1 ) ∈ [ 1 4 , 1 3 ] . \frac{\lVert h_j-t_j^\star v_j\rVert^2}{(t_j^\star)^2}
=\frac{2m+1}{6(m+1)}\in\left[\frac{1}{4},\frac{1}{3}\right]. ( t j ⋆ ) 2 ∥ h j − t j ⋆ v j ∥ 2 = 6 ( m + 1 ) 2 m + 1 ∈ [ 4 1 , 3 1 ] .
Define r = ∥ h j − t j ⋆ v j ∥ 2 ( t j ⋆ ) 2 ∈ [ 1 4 , 1 3 ] r = \frac{\lVert h_j-t_j^\star v_j\rVert^2}{(t_j^\star)^2} \in\left[\frac{1}{4},\frac{1}{3}\right] r = ( t j ⋆ ) 2 ∥ h j − t j ⋆ v j ∥ 2 ∈ [ 4 1 , 3 1 ] . Since R j 2 = ( t j ⋆ ) 2 ( 1 + r ) R_j^2 = (t_j^\star)^2(1+r) R j 2 = ( t j ⋆ ) 2 ( 1 + r ) , the two fractions are
( t j ⋆ ) 2 R j 2 = 1 1 + r ∈ [ 3 4 , 4 5 ] , ∥ h j − t j ⋆ v j ∥ 2 R j 2 = r 1 + r ∈ [ 1 5 , 1 4 ] . \frac{(t_j^\star)^2}{R_j^2} = \frac{1}{1+r} \in\left[\frac{3}{4},\frac{4}{5}\right],
\qquad
\frac{\lVert h_j-t_j^\star v_j\rVert^2}{R_j^2} = \frac{r}{1+r} \in\left[\frac{1}{5},\frac{1}{4}\right]. R j 2 ( t j ⋆ ) 2 = 1 + r 1 ∈ [ 4 3 , 5 4 ] , R j 2 ∥ h j − t j ⋆ v j ∥ 2 = 1 + r r ∈ [ 5 1 , 4 1 ] .
Taking square roots proves the length bounds. Also ∑ i = 1 m ( m − i + 1 ) τ j = m ( m + 1 ) τ j 2 ≤ m 2 τ j \sum_{i=1}^{m}(m-i+1)\tau_j = \frac{m(m+1)\tau_j}{2} \le m^2\tau_j ∑ i = 1 m ( m − i + 1 ) τ j = 2 m ( m + 1 ) τ j ≤ m 2 τ j . This explains the coefficient m − 1 2 m^{-\frac{1}{2}} m − 2 1 in the first residual: using β t − x 1 \beta t-x_1 β t − x 1 with β > 0 \beta>0 β > 0 instead would give t = ( m + 1 ) τ j β t=\frac{(m+1)\tau_j}{\beta} t = β ( m + 1 ) τ j at equal residuals; β = m − 1 2 \beta=m^{-\frac{1}{2}} β = m − 2 1 makes the first coordinate and link vector comparable in length. The first coordinate supplies s j t j ⋆ s_jt_j^\star s j t j ⋆ in Lemma E.15 in Appx. E; the unexposed links give the separation proved in Lemma D.12 (a).
( D m w ) i = 0 ( i > k ) , ϕ ( r ) ( 0 ) = 0 ( 0 ≤ r ≤ p ) , (D_mw)_i=0 \quad(i>k),\qquad\phi^{(r)}(0)=0 \quad(0\le r\le p), ( D m w ) i = 0 ( i > k ) , ϕ ( r ) ( 0 ) = 0 ( 0 ≤ r ≤ p ) ,
where ϕ ( 0 ) = ϕ \phi^{(0)}=\phi ϕ ( 0 ) = ϕ and the second equality follows from Lemma B.16 (b) in Appx. B. Using the row vectors from part (a), for 1 ≤ r ≤ p 1\le r\le p 1 ≤ r ≤ p and arbitrary d 1 , … , d r ∈ R d d_1,\ldots,d_r\in\mathbb{R}^d d 1 , … , d r ∈ R d we obtain
Q j ( w ) = λ ∑ i = 0 k ϕ ( ( D m w ) i ) , Q_j(w)=\lambda\sum_{i=0}^{k}\phi\bigl((D_mw)_i\bigr), Q j ( w ) = λ i = 0 ∑ k ϕ ( ( D m w ) i ) ,
∇ r Q j ( w ) [ d 1 , … , d r ] = λ ∑ i = 0 k ϕ ( r ) ( ( D m w ) i ) ∏ ℓ = 1 r ⟨ a j , i , d ℓ ⟩ . \nabla^r Q_j(w)[d_1,\ldots,d_r]
=\lambda\sum_{i=0}^{k}\phi^{(r)}\bigl((D_mw)_i\bigr)\prod_{\ell=1}^{r}\langle a_{j,i},d_\ell\rangle. ∇ r Q j ( w ) [ d 1 , … , d r ] = λ i = 0 ∑ k ϕ ( r ) ( ( D m w ) i ) ℓ = 1 ∏ r ⟨ a j , i , d ℓ ⟩ .
Rows 0 , … , k 0,\ldots,k 0 , … , k use only v j , u j , 1 , … , u j , k + 1 v_j,u_{j,1},\ldots,u_{j,k+1} v j , u j , 1 , … , u j , k + 1 , proving the claim. At w = 0 w=0 w = 0 all residuals vanish, so ∇ r Q j ( 0 ) = 0 \nabla^rQ_j(0)=0 ∇ r Q j ( 0 ) = 0 for 1 ≤ r ≤ p 1\le r\le p 1 ≤ r ≤ p . Together these statements give the p th p^{\text{th}} p th -order zero-chain property in the ordered coordinates of E j E_j E j , in the sense of [12 Definition 3] , restated as Definition D.2.
The affine formula supplies gradient g j g_j g j after completing the phase, whereas the chain formula gives positive excess at early evaluation points. We use the smooth maximum to retain each formula on its required region.
Definition D.4 (Affine and chain formulas). For the phase-j j j chain and endpoint from Appx. D.1.2, let
λ , α , δ j > 0 , G j , s j + 1 , t j + 1 ⋆ ≥ 0. \lambda,\alpha,\delta_j>0,\qquad G_j,s_{j+1},t_{j+1}^\star\ge0. λ , α , δ j > 0 , G j , s j + 1 , t j + 1 ⋆ ≥ 0. The local origin is c j c_j c j and h j = c j + 1 − c j h_j=c_{j+1}-c_j h j = c j + 1 − c j . For j < L − 1 j<L-1 j < L − 1 , choose a unit v j + 1 ⊥ E j v_{j+1}\perp E_j v j + 1 ⊥ E j ; local inputs are
( w , t j + 1 ) = ( P E j ( x − c j ) , ⟨ v j + 1 , x − c j ⟩ ) ∈ E j × R . (w,t_{j+1})=\bigl(P_{E_j}(x-c_j),\langle v_{j+1},x-c_j\rangle\bigr)\in E_j\times\mathbb{R}. ( w , t j + 1 ) = ( P E j ( x − c j ) , ⟨ v j + 1 , x − c j ⟩ ) ∈ E j × R . The same symbols denote the ambient functions obtained by this substitution. Here s j + 1 , t j + 1 ⋆ s_{j+1},t_{j+1}^\star s j + 1 , t j + 1 ⋆ are the next phase’s starting-gradient magnitude and first endpoint coordinate. In the terminal phase set s L = t L ⋆ = 0 s_L=t_L^\star=0 s L = t L ⋆ = 0 , omit v L v_L v L , and restrict to t j + 1 = 0 t_{j+1}=0 t j + 1 = 0 . Set
x j ⋆ : = ( h j , 0 ) , Γ j : = λ τ j q − 1 m , b j : = G j h j R j , E j : = λ m τ j q , (110) x_j^\star:=(h_j,0),\qquad\Gamma_j:=\frac{\lambda\tau_j^{q-1}}{\sqrt{m}},\qquad b_j:=\frac{G_jh_j}{R_j},\qquad\mathcal{E}_j:=\lambda m\tau_j^q, \tag*{(110)} x j ⋆ := ( h j , 0 ) , Γ j := m λ τ j q − 1 , b j := R j G j h j , E j := λm τ j q , (110)
s j : = Γ j + G j t j ⋆ R j , g j : = s j v j − s j + 1 v j + 1 − b j . (111) s_j := \Gamma_j + \frac{G_j t_j^\star}{R_j}, \qquad g_j := s_j v_j - s_{j+1}v_{j+1} - b_j. \tag*{(111)} s j := Γ j + R j G j t j ⋆ , g j := s j v j − s j + 1 v j + 1 − b j . (111)
Define
P j ( w , t j + 1 ) : = Q j ( w ) + α q ∣ t j + 1 ∣ q = λ q ∥ D m w ∥ q q + α q ∣ t j + 1 ∣ q , (112) P_j(w,t_{j+1}) := Q_j(w) + \frac{\alpha}{q}|t_{j+1}|^q = \frac{\lambda}{q}\lVert D_m w\rVert_q^q + \frac{\alpha}{q}|t_{j+1}|^q, \tag*{(112)} P j ( w , t j + 1 ) := Q j ( w ) + q α ∣ t j + 1 ∣ q = q λ ∥ D m w ∥ q q + q α ∣ t j + 1 ∣ q , (112)
P j ( h j , 0 ) = λ q ( m + 1 ) τ j q , P_j(h_j,0) = \frac{\lambda}{q}(m+1)\tau_j^q, P j ( h j , 0 ) = q λ ( m + 1 ) τ j q ,
I j ( w , t j + 1 ) : = ⟨ g j , w − h j + t j + 1 v j + 1 ⟩ , (113) I_j(w,t_{j+1}) := \langle g_j,w-h_j+t_{j+1}v_{j+1}\rangle, \tag*{(113)} I j ( w , t j + 1 ) := ⟨ g j , w − h j + t j + 1 v j + 1 ⟩ , (113)
O j : = P j − P j ( h j , 0 ) − 4 δ j , ζ j : = O j − I j , (114) O_j := P_j-P_j(h_j,0)-4\delta_j,\qquad\zeta_j := O_j-I_j, \tag*{(114)} O j := P j − P j ( h j , 0 ) − 4 δ j , ζ j := O j − I j , (114)
A j : = M δ j ( I j , O j ) = I j + ψ δ j ( ζ j ) . (115) A_j := M_{\delta_j}(I_j,O_j)=I_j+\psi_{\delta_j}(\zeta_j). \tag*{(115)} A j := M δ j ( I j , O j ) = I j + ψ δ j ( ζ j ) . (115)
The base construction will supply G j = μ R j q − 1 G_j=\mu R_j^{q-1} G j = μ R j q − 1 .
The term α ∣ t j + 1 ∣ q q \frac{\alpha|t_{j+1}|^q}{q} q α ∣ t j + 1 ∣ q controls the linear term s j + 1 t j + 1 s_{j+1}t_{j+1} s j + 1 t j + 1 in ζ j \zeta_j ζ j ; Corollary D.10 quantifies this bound.
Lemma D.5 (The affine gradient in residual coordinates). With Definition D.4 , put u j : = λ τ j q − 1 = m Γ j u_j:=\lambda\tau_j^{q-1}=\sqrt{m}\,\Gamma_j u j := λ τ j q − 1 = m Γ j .
(a) Gradient difference.
e j : = g j − ∇ P j ( x j ∗ ) = − G j R j ∑ i = 1 m x j , i ∗ u j , i − s j + 1 v j + 1 , x j , i ∗ = ( m − i + 1 ) τ j . (116) e_j:=g_j-\nabla P_j(x_j^\ast)=-\frac{G_j}{R_j}\sum_{i=1}^{m}x_{j,i}^\ast u_{j,i}-s_{j+1}v_{j+1},\qquad x_{j,i}^\ast=(m-i+1)\tau_j. \tag*{(116)} e j := g j − ∇ P j ( x j ∗ ) = − R j G j i = 1 ∑ m x j , i ∗ u j , i − s j + 1 v j + 1 , x j , i ∗ = ( m − i + 1 ) τ j . (116) I j ( h j , 0 ) = 0 , I j ( h j , t j + 1 ⋆ ) = − s j + 1 t j + 1 ⋆ , ∇ I j = g j . I_j(h_j,0)=0,\qquad I_j(h_j,t_{j+1}^\star)=-s_{j+1}t_{j+1}^\star,\qquad\nabla I_j=g_j. I j ( h j , 0 ) = 0 , I j ( h j , t j + 1 ⋆ ) = − s j + 1 t j + 1 ⋆ , ∇ I j = g j . (b) Residual coordinates. The coefficients
π j , 0 = u j , π j , i = u j − G j R j ∑ k = 1 i x j , k ∗ ( 1 ≤ i ≤ m ) (117) \pi_{j,0}=u_j,\qquad\pi_{j,i}=u_j-\frac{G_j}{R_j}\sum_{k=1}^{i}x_{j,k}^\ast\qquad(1\le i\le m) \tag*{(117)} π j , 0 = u j , π j , i = u j − R j G j k = 1 ∑ i x j , k ∗ ( 1 ≤ i ≤ m ) (117) satisfy D m T π j = P E j g j D_m^{\mathsf T}\pi_j=P_{E_j}g_j D m T π j = P E j g j in the coordinates of Appx. D.1.2, and
ζ j ( w , t j + 1 ) = P j ( w , t j + 1 ) − ⟨ π j , D m w ⟩ + s j + 1 t j + 1 + k j , a f f , \zeta_j(w,t_{j+1})=P_j(w,t_{j+1})-\langle\pi_j,D_mw\rangle+s_{j+1}t_{j+1}+k_{j,\mathrm{aff}}, ζ j ( w , t j + 1 ) = P j ( w , t j + 1 ) − ⟨ π j , D m w ⟩ + s j + 1 t j + 1 + k j , aff , k j , a f f : = τ j ∑ i = 0 m π j , i − λ q ( m + 1 ) τ j q − 4 δ j . (118) k_{j,\mathrm{aff}}:=\tau_j\sum_{i=0}^{m}\pi_{j,i}-\frac{\lambda}{q}(m+1)\tau_j^q-4\delta_j. \tag*{(118)} k j , aff := τ j i = 0 ∑ m π j , i − q λ ( m + 1 ) τ j q − 4 δ j . (118) (c) Coefficient bounds. For any 0 < c 0 < 1 0<c_0<1 0 < c 0 < 1 , if G j ≤ c 0 Γ j G_j\le c_0\Gamma_j G j ≤ c 0 Γ j , then
( 1 − c 0 ) u j ≤ π j , i ≤ u j ( 0 ≤ i ≤ m ) , (1-c_0)u_j\le\pi_{j,i}\le u_j\qquad(0\le i\le m), ( 1 − c 0 ) u j ≤ π j , i ≤ u j ( 0 ≤ i ≤ m ) , k j , a f f ≥ ( m + 1 ) ( 1 q ∗ − c 0 ) λ τ j q − 4 δ j , (119) k_{j,\mathrm{aff}}\ge(m+1)\left(\frac{1}{q^\ast}-c_0\right)\lambda\tau_j^q-4\delta_j, \tag*{(119)} k j , aff ≥ ( m + 1 ) ( q ∗ 1 − c 0 ) λ τ j q − 4 δ j , (119) k j , a f f > 0 under the scalar choice xref‘Eq. (D.24)‘ below. k_{j,\mathrm{aff}}>0\quad\text{under the scalar choice {xref}`Eq. (D.24)` below.} k j , aff > 0 under the scalar choice xref ‘Eq. (D.24)‘ below.
Proof. • Proof of (a). Lemma D.3(a) gives ∇ P j ( x j ⋆ ) = Γ j v j \nabla P_j(x_j^\star)=\Gamma_jv_j ∇ P j ( x j ⋆ ) = Γ j v j . Subtract it from g j g_j g j :
g j − ∇ P j ( x j ⋆ ) = ( s j − Γ j − G j t j + 1 ⋆ R j ) ⏟ 0 by Eq. ( D .13 ) v j − G j R j ∑ i = 1 m x j , i ⋆ u j , i − s j + 1 v j + 1 . g_j-\nabla P_j(x_j^\star)=\underbrace{\left(s_j-\Gamma_j-\frac{G_jt_{j+1}^\star}{R_j}\right)}_{0\text{ by Eq. }{{(D.13)}}}v_j-\frac{G_j}{R_j}\sum_{i=1}^{m}x_{j,i}^\star u_{j,i}-s_{j+1}v_{j+1}. g j − ∇ P j ( x j ⋆ ) = 0 by Eq. ( D .13 ) ( s j − Γ j − R j G j t j + 1 ⋆ ) v j − R j G j i = 1 ∑ m x j , i ⋆ u j , i − s j + 1 v j + 1 .
Thus the choice of s j s_j s j removes the v j v_j v j -component, proving Eq. (D.18). For j < L − 1 j<L-1 j < L − 1 , the affine definition Eq. (D.15) also gives
I j ( h j , t j + 1 ) = ⟨ g j , h j − h j + t j + 1 v j + 1 ⟩ = t j + 1 ⟨ g j , v j + 1 ⟩ = − s j + 1 t j + 1 , \begin{aligned}
I_j(h_j,t_{j+1})=\langle g_j,h_j-h_j+t_{j+1}v_{j+1}\rangle\\
&=t_{j+1}\langle g_j,v_{j+1}\rangle=-s_{j+1}t_{j+1},
\end{aligned} I j ( h j , t j + 1 ) = ⟨ g j , h j − h j + t j + 1 v j + 1 ⟩ = t j + 1 ⟨ g j , v j + 1 ⟩ = − s j + 1 t j + 1 ,
I j ( h j , 0 ) = 0 , I j ( h j , t j + 1 ⋆ ) = − s j + 1 t j + 1 ⋆ , ∇ I j = g j . I_j(h_j,0)=0,\qquad I_j(h_j,t_{j+1}^\star)=-s_{j+1}t_{j+1}^\star,\qquad\nabla I_j=g_j. I j ( h j , 0 ) = 0 , I j ( h j , t j + 1 ⋆ ) = − s j + 1 t j + 1 ⋆ , ∇ I j = g j .
The third equality uses v j + 1 ⊥ E j v_{j+1}\perp E_j v j + 1 ⊥ E j and b j , v j ∈ E j b_j,v_j\in E_j b j , v j ∈ E j . In the terminal phase t j + 1 = 0 t_{j+1}=0 t j + 1 = 0 and the next-direction term is omitted; the affine definition directly gives I j ( h j , 0 ) = 0 I_j(h_j,0)=0 I j ( h j , 0 ) = 0 and ∇ I j = g j \nabla I_j=g_j ∇ I j = g j .
• Proof of (b). Using the columns of D m D_m D m and the gradient in part (a), the equation D m ⊤ π j = P E j g j D_m^\top\pi_j=P_{E_j}g_j D m ⊤ π j = P E j g j is
π j , 0 m = Γ j , π j , i − π j , i − 1 = − G j x j , i ⋆ R j ( 1 ≤ i ≤ m ) . \frac{\pi_{j,0}}{\sqrt{m}}=\Gamma_j,\qquad\pi_{j,i}-\pi_{j,i-1}=-\frac{G_jx_{j,i}^\star}{R_j}\qquad(1\le i\le m). m π j , 0 = Γ j , π j , i − π j , i − 1 = − R j G j x j , i ⋆ ( 1 ≤ i ≤ m ) .
The first equation gives π j , 0 = m Γ j = u j \pi_{j,0}=\sqrt{m}\,\Gamma_j=u_j π j , 0 = m Γ j = u j . Telescoping the next i i i equations gives
π j , i − π j , 0 = ∑ k = 1 i ( π j , k − π j , k − 1 ) = − G j R j ∑ k = 1 i x j , k ⋆ , \pi_{j,i}-\pi_{j,0}=\sum_{k=1}^{i}(\pi_{j,k}-\pi_{j,k-1})=-\frac{G_j}{R_j}\sum_{k=1}^{i}x_{j,k}^\star, π j , i − π j , 0 = k = 1 ∑ i ( π j , k − π j , k − 1 ) = − R j G j k = 1 ∑ i x j , k ⋆ ,
π j , i = π j , 0 − G j R j ∑ k = 1 i x j , k ⋆ = u j − G j R j ∑ k = 1 i x j , k ⋆ . \pi_{j,i}=\pi_{j,0}-\frac{G_j}{R_j}\sum_{k=1}^{i}x_{j,k}^\star=u_j-\frac{G_j}{R_j}\sum_{k=1}^{i}x_{j,k}^\star. π j , i = π j , 0 − R j G j k = 1 ∑ i x j , k ⋆ = u j − R j G j k = 1 ∑ i x j , k ⋆ .
This is Eq. (D.19). Since D m h j = τ j 1 D_mh_j=\tau_j\mathbf{1} D m h j = τ j 1 , the affine and chain formulas become
I j ( w , t j + 1 ) = ⟨ π j , D m ( w − h j ) ⟩ − s j + 1 t j + 1 = ⟨ π j , D m w ⟩ − τ j ∑ i = 0 m π j , i − s j + 1 t j + 1 , \begin{aligned}
I_j(w,t_{j+1})=\langle\pi_j,D_m(w-h_j)\rangle-s_{j+1}t_{j+1} \\
&=\langle\pi_j,D_mw\rangle-\tau_j\sum_{i=0}^{m}\pi_{j,i}-s_{j+1}t_{j+1},
\end{aligned} I j ( w , t j + 1 ) = ⟨ π j , D m ( w − h j )⟩ − s j + 1 t j + 1 = ⟨ π j , D m w ⟩ − τ j i = 0 ∑ m π j , i − s j + 1 t j + 1 ,
O j ( w , t j + 1 ) = P j ( w , t j + 1 ) − λ q ( m + 1 ) τ j q − 4 δ j . O_j(w,t_{j+1})=P_j(w,t_{j+1})-\frac{\lambda}{q}(m+1)\tau_j^q-4\delta_j. O j ( w , t j + 1 ) = P j ( w , t j + 1 ) − q λ ( m + 1 ) τ j q − 4 δ j .
Subtracting the affine formula from the chain formula and distributing the negative sign gives
ζ j ( w , t j + 1 ) = P j ( w , t j + 1 ) − λ q ( m + 1 ) τ j q − 4 δ j − ( ⟨ π j , D m w ⟩ − τ j ∑ i = 0 m π j , i − s j + 1 t j + 1 ) = P j ( w , t j + 1 ) − ⟨ π j , D m w ⟩ + s j + 1 t j + 1 + τ j ∑ i = 0 m π j , i − λ q ( m + 1 ) τ j q − 4 δ j = P j ( w , t j + 1 ) − ⟨ π j , D m w ⟩ + s j + 1 t j + 1 + k j , a f f . \begin{aligned}
\zeta_j(w,t_{j+1})=P_j(w,t_{j+1})-\frac{\lambda}{q}(m+1)\tau_j^q-4\delta_j
-\left(\langle\pi_j,D_mw\rangle-\tau_j\sum_{i=0}^{m}\pi_{j,i}-s_{j+1}t_{j+1}\right) \\
&=P_j(w,t_{j+1})-\langle\pi_j,D_mw\rangle+s_{j+1}t_{j+1}
+\tau_j\sum_{i=0}^{m}\pi_{j,i}-\frac{\lambda}{q}(m+1)\tau_j^q-4\delta_j \\
&=P_j(w,t_{j+1})-\langle\pi_j,D_mw\rangle+s_{j+1}t_{j+1}+k_{j,\mathrm{aff}}.
\end{aligned} ζ j ( w , t j + 1 ) = P j ( w , t j + 1 ) − q λ ( m + 1 ) τ j q − 4 δ j − ( ⟨ π j , D m w ⟩ − τ j i = 0 ∑ m π j , i − s j + 1 t j + 1 ) = P j ( w , t j + 1 ) − ⟨ π j , D m w ⟩ + s j + 1 t j + 1 + τ j i = 0 ∑ m π j , i − q λ ( m + 1 ) τ j q − 4 δ j = P j ( w , t j + 1 ) − ⟨ π j , D m w ⟩ + s j + 1 t j + 1 + k j , aff .
The last two constant terms and the coefficient sum give precisely k j , a f f k_{j,\mathrm{aff}} k j , aff in Eq. (D.20).
Proof of (c). Lemma D.3 (c) gives ∑ k x j , k ∗ ≤ τ j m 2 \sum_k x^*_{j,k} \le\tau_j m^2 ∑ k x j , k ∗ ≤ τ j m 2 and R j ≥ τ j m 3 2 R_j \ge\tau_j m^{\frac{3}{2}} R j ≥ τ j m 2 3 . Hence
0 ≤ u j − π j , i = G j R j ∑ k = 1 i x j , k ∗ ≤ G j τ j m 3 2 τ j m 2 = G j m ≤ c 0 G j m = c 0 u j . \begin{aligned}
0 \le u_j-\pi_{j,i}=\frac{G_j}{R_j}\sum_{k=1}^{i}x^*_{j,k} \\
&\le\frac{G_j}{\tau_jm^{\frac{3}{2}}}\tau_jm^2=G_j\sqrt{m} \\
&\le c_0G_j\sqrt{m}=c_0u_j.
\end{aligned} 0 ≤ u j − π j , i = R j G j k = 1 ∑ i x j , k ∗ ≤ τ j m 2 3 G j τ j m 2 = G j m ≤ c 0 G j m = c 0 u j .
Insert ( 1 − c 0 ) u j ≤ π j , i ≤ u j (1-c_0)u_j\le\pi_{j,i}\le u_j ( 1 − c 0 ) u j ≤ π j , i ≤ u j into the constant formula:
k j , a f f = τ j ∑ i = 0 m π j , i − λ q ( m + 1 ) τ j q − 4 δ j ≥ ( m + 1 ) ( 1 − c 0 ) τ j u j − λ q ( m + 1 ) τ j q − 4 δ j = ( m + 1 ) ( 1 − c 0 − 1 q ) λ τ j q − 4 δ j = ( m + 1 ) ( 1 q ∗ − c 0 ) λ τ j q − 4 δ j . \begin{aligned}
k_{j,\mathrm{aff}}=\tau_j\sum_{i=0}^{m}\pi_{j,i}-\frac{\lambda}{q}(m+1)\tau_j^q-4\delta_j \\
&\ge(m+1)(1-c_0)\tau_ju_j-\frac{\lambda}{q}(m+1)\tau_j^q-4\delta_j \\
&=(m+1)\left(1-c_0-\frac{1}{q}\right)\lambda\tau_j^q-4\delta_j \\
&=(m+1)\left(\frac{1}{q^*}-c_0\right)\lambda\tau_j^q-4\delta_j.
\end{aligned} k j , aff = τ j i = 0 ∑ m π j , i − q λ ( m + 1 ) τ j q − 4 δ j ≥ ( m + 1 ) ( 1 − c 0 ) τ j u j − q λ ( m + 1 ) τ j q − 4 δ j = ( m + 1 ) ( 1 − c 0 − q 1 ) λ τ j q − 4 δ j = ( m + 1 ) ( q ∗ 1 − c 0 ) λ τ j q − 4 δ j .
The third line uses u j = λ τ j q − 1 u_j=\lambda\tau_j^{q-1} u j = λ τ j q − 1 , and the last uses 1 − 1 q = 1 q ∗ 1-\frac{1}{q}=\frac{1}{q^*} 1 − q 1 = q ∗ 1 . Under the later choice Eq. (D.24), c 0 = c 1 < 1 64 c_0=c_1<\frac{1}{64} c 0 = c 1 < 64 1 , δ j = c 1 E j \delta_j=c_1\mathcal{E}_j δ j = c 1 E j , and 1 q ∗ > 1 2 \frac{1}{q^*}>\frac{1}{2} q ∗ 1 > 2 1 . Consequently
k j , a f f ≥ ( m + 1 ) ( 1 q ∗ − c 0 ) λ τ j q − 4 c 1 E j ≥ ( 1 q ∗ − c 0 ) E j − 4 c 1 E j = E j ( 1 q ∗ − c 0 − 4 c 1 ) > E j ( 1 2 − 5 64 ) > 0. \begin{aligned}
k_{j,\mathrm{aff}}\ge(m+1)\left(\frac{1}{q^*}-c_0\right)\lambda\tau_j^q-4c_1\mathcal{E}_j \\
&\ge\left(\frac{1}{q^*}-c_0\right)\mathcal{E}_j-4c_1\mathcal{E}_j \\
&=\mathcal{E}_j\left(\frac{1}{q^*}-c_0-4c_1\right) \\
&>\mathcal{E}_j\left(\frac{1}{2}-\frac{5}{64}\right)>0.
\end{aligned} k j , aff ≥ ( m + 1 ) ( q ∗ 1 − c 0 ) λ τ j q − 4 c 1 E j ≥ ( q ∗ 1 − c 0 ) E j − 4 c 1 E j = E j ( q ∗ 1 − c 0 − 4 c 1 ) > E j ( 2 1 − 64 5 ) > 0.
Here 1 q ∗ − c 0 > 0 \frac{1}{q^*}-c_0>0 q ∗ 1 − c 0 > 0 permits m + 1 ≥ m m+1\ge m m + 1 ≥ m , and the last line uses c 0 = c 1 < 1 64 c_0=c_1<\frac{1}{64} c 0 = c 1 < 64 1 .
Value differences and the next-coordinate bound
Lemma D.6 (Fenchel–Young inequality [10 Proposition 2.3.1] ). Let f : R n → ( − ∞ , + ∞ ] f:\mathbb{R}^n\to(-\infty,+\infty] f : R n → ( − ∞ , + ∞ ] , let a ∈ dom f a\in\operatorname{dom}f a ∈ dom f , and let b ∈ R n b\in\mathbb{R}^n b ∈ R n . Define
f ∗ ( b ) = sup z ∈ R n { ⟨ b , z ⟩ − f ( z ) } . f^*(b)=\sup_{z\in\mathbb{R}^n}\{\langle b,z\rangle-f(z)\}. f ∗ ( b ) = z ∈ R n sup {⟨ b , z ⟩ − f ( z )} . Then
f ( a ) + f ∗ ( b ) ≥ ⟨ b , a ⟩ , f ( a ) + f ∗ ( b ) = ⟨ b , a ⟩ ⟺ b ∈ ∂ f ( a ) . f(a)+f^*(b)\ge\langle b,a\rangle,\qquad f(a)+f^*(b)=\langle b,a\rangle\Longleftrightarrow b\in\partial f(a). f ( a ) + f ∗ ( b ) ≥ ⟨ b , a ⟩ , f ( a ) + f ∗ ( b ) = ⟨ b , a ⟩ ⟺ b ∈ ∂ f ( a ) .
Here
dom f = { z : f ( z ) < + ∞ } , ∂ f ( a ) = { b : f ( z ) ≥ f ( a ) + ⟨ b , z − a ⟩ for every z ∈ R n } . \operatorname{dom} f = \{z : f(z) < +\infty\}, \qquad\partial f(a) = \{b : f(z) \ge f(a) + \langle b, z-a\rangle\text{ for every } z \in\mathbb{R}^{n}\}. dom f = { z : f ( z ) < + ∞ } , ∂ f ( a ) = { b : f ( z ) ≥ f ( a ) + ⟨ b , z − a ⟩ for every z ∈ R n } .
The proposition does not require convexity or lower semicontinuity of f f f .
Lemma D.7 (Power conjugates and positive scaling [10 Tables 2.1–2.2, pp. 45–46] ). For q > 1 q > 1 q > 1 and q ∗ = q q − 1 q^{*} = \frac{q}{q-1} q ∗ = q − 1 q , the conjugate pair is
ϕ ( a ) = ∣ a ∣ q q , ϕ ∗ ( b ) = ∣ b ∣ q ∗ q ∗ , dom ϕ = dom ϕ ∗ = R . \phi(a) = \frac{\lvert a\rvert^{q}}{q}, \qquad\phi^{*}(b) = \frac{\lvert b\rvert^{q^{*}}}{q^{*}}, \qquad\operatorname{dom}\phi= \operatorname{dom}\phi^{*} = \mathbb{R}. ϕ ( a ) = q ∣ a ∣ q , ϕ ∗ ( b ) = q ∗ ∣ b ∣ q ∗ , dom ϕ = dom ϕ ∗ = R . For h : R → ( − ∞ , + ∞ ] h : \mathbb{R} \to(-\infty,+\infty] h : R → ( − ∞ , + ∞ ] with conjugate h ∗ ( b ) = sup a ∈ R { b a − h ( a ) } h^{*}(b) = \sup_{a\in\mathbb{R}}\{ba-h(a)\} h ∗ ( b ) = sup a ∈ R { ba − h ( a )} and ℓ > 0 \ell> 0 ℓ > 0 ,
( ℓ h ) ∗ ( b ) = ℓ h ∗ ( b ℓ ) . (\ell h)^{*}(b) = \ell h^{*}\left(\frac{b}{\ell}\right). ( ℓ h ) ∗ ( b ) = ℓ h ∗ ( ℓ b ) .
Lemma D.8 (Scaled scalar and separable power inequalities). Let q > 1 q > 1 q > 1 , q ∗ = q q − 1 q^{*} = \frac{q}{q-1} q ∗ = q − 1 q , and ℓ > 0 \ell> 0 ℓ > 0 . Then
b a ≤ ℓ q ∣ a ∣ q + ℓ − 1 q − 1 q ∗ ∣ b ∣ q ∗ ( a , b ∈ R ) . (120) ba \le\frac{\ell}{q}\lvert a\rvert^{q} + \frac{\ell^{-\frac{1}{q-1}}}{q^{*}}\lvert b\rvert^{q^{*}} \qquad(a,b \in\mathbb{R}). \tag*{(120)} ba ≤ q ℓ ∣ a ∣ q + q ∗ ℓ − q − 1 1 ∣ b ∣ q ∗ ( a , b ∈ R ) . (120) Consequently, for any finite collection of real a i a_i a i , b i b_i b i and positive ℓ i \ell_i ℓ i ,
∑ i b i a i ≤ ∑ i ℓ i q ∣ a i ∣ q + 1 q ∗ ∑ i ℓ i − 1 q − 1 ∣ b i ∣ q ∗ . (121) \sum_i b_i a_i \le\sum_i \frac{\ell_i}{q}\lvert a_i\rvert^{q} + \frac{1}{q^{*}}\sum_i \ell_i^{-\frac{1}{q-1}}\lvert b_i\rvert^{q^{*}}. \tag*{(121)} i ∑ b i a i ≤ i ∑ q ℓ i ∣ a i ∣ q + q ∗ 1 i ∑ ℓ i − q − 1 1 ∣ b i ∣ q ∗ . (121) Proof. Put ϕ ( a ) = ∣ a ∣ q q \phi(a) = \frac{\lvert a\rvert^{q}}{q} ϕ ( a ) = q ∣ a ∣ q and h = ℓ ϕ h = \ell\phi h = ℓ ϕ . The conjugate pair and positive-scaling rule in Lemma D.7 give
h ∗ ( b ) = ℓ ϕ ∗ ( b ℓ ) by the positive-scaling rule, h^{*}(b) = \ell\phi^{*}\left(\frac{b}{\ell}\right) \qquad\text{by the positive-scaling rule,} h ∗ ( b ) = ℓ ϕ ∗ ( ℓ b ) by the positive-scaling rule, = ℓ q ∗ ∣ b ℓ ∣ q ∗ by the conjugate of ϕ , = \frac{\ell}{q^{*}}\left\lvert\frac{b}{\ell}\right\rvert^{q^{*}} \qquad\text{by the conjugate of }\phi, = q ∗ ℓ ℓ b q ∗ by the conjugate of ϕ , = ℓ 1 − q ∗ q ∗ ∣ b ∣ q ∗ , = \frac{\ell^{1-q^{*}}}{q^{*}}\lvert b\rvert^{q^{*}}, = q ∗ ℓ 1 − q ∗ ∣ b ∣ q ∗ , = ℓ − 1 q − 1 q ∗ ∣ b ∣ q ∗ since 1 − q ∗ = − 1 q − 1 . = \frac{\ell^{-\frac{1}{q-1}}}{q^{*}}\lvert b\rvert^{q^{*}} \qquad\text{since } 1-q^{*}=-\frac{1}{q-1}. = q ∗ ℓ − q − 1 1 ∣ b ∣ q ∗ since 1 − q ∗ = − q − 1 1 . Apply Lemma D.6 with f = h f=h f = h :
b a ≤ h ( a ) + h ∗ ( b ) = ℓ q ∣ a ∣ q + ℓ − 1 q − 1 q ∗ ∣ b ∣ q ∗ . ba \le h(a) + h^{*}(b) = \frac{\ell}{q}\lvert a\rvert^{q} + \frac{\ell^{-\frac{1}{q-1}}}{q^{*}}\lvert b\rvert^{q^{*}}. ba ≤ h ( a ) + h ∗ ( b ) = q ℓ ∣ a ∣ q + q ∗ ℓ − q − 1 1 ∣ b ∣ q ∗ . This is Eq. (D.22). Applying it with ( a , b , ℓ ) = ( a i , b i , ℓ i ) (a,b,\ell) = (a_i,b_i,\ell_i) ( a , b , ℓ ) = ( a i , b i , ℓ i ) for each index and summing gives Eq. (D.23). \hfill □ \square □
To determine which function equals A j A_j A j , Lemma 4.2 gives
ζ j > δ j ⟹ A j = O j , ζ j < − δ j ⟹ A j = I j . \zeta_j > \delta_j \Longrightarrow A_j = O_j, \qquad\zeta_j < -\delta_j \Longrightarrow A_j = I_j. ζ j > δ j ⟹ A j = O j , ζ j < − δ j ⟹ A j = I j .
Lemma D.9 (Constants giving the required value differences). Use Definition D.4 . Choose
c 0 = c 1 = 2 − q − 3 q , δ j = c 1 E j , (122) c_0=c_1=\frac{2^{-q-3}}{q},\qquad\delta_j=c_1\mathcal{E}_j, \tag*{(122)} c 0 = c 1 = q 2 − q − 3 , δ j = c 1 E j , (122) and assume
G j ≤ c 0 Γ j , (123) G_j\le c_0\Gamma_j, \tag*{(123)} G j ≤ c 0 Γ j , (123) s j + 1 t j + 1 ⋆ + α q ( t j + 1 ⋆ ) q ≤ c 0 E j . (124) s_{j+1}t_{j+1}^{\star}+\frac{\alpha}{q}\left(t_{j+1}^{\star}\right)^q\le c_0\mathcal{E}_j. \tag*{(124)} s j + 1 t j + 1 ⋆ + q α ( t j + 1 ⋆ ) q ≤ c 0 E j . (124) Use the chain-prefix subspace U j , b U_{j,b} U j , b from Step 2.1, where the integer b b b counts links and the coordinate ⟨ v j , w ⟩ \langle v_j,w\rangle ⟨ v j , w ⟩ is unrestricted.
(a) Value difference on chain prefixes.
ζ j ( w , 0 ) ≥ 7 δ j ( w ∈ U j , b , 0 ≤ b ≤ ⌊ m 4 ⌋ ) . \zeta_j(w,0)\ge7\delta_j\qquad\left(w\in U_{j,b},\ 0\le b\le\left\lfloor\frac{m}{4}\right\rfloor\right). ζ j ( w , 0 ) ≥ 7 δ j ( w ∈ U j , b , 0 ≤ b ≤ ⌊ 4 m ⌋ ) . (b) Value differences at the endpoints.
ζ j ( h j , 0 ) = − 4 δ j , ζ j ( h j , t j + 1 ⋆ ) ≤ − 3 δ j . \zeta_j(h_j,0)=-4\delta_j,\qquad\zeta_j\left(h_j,t_{j+1}^{\star}\right)\le-3\delta_j. ζ j ( h j , 0 ) = − 4 δ j , ζ j ( h j , t j + 1 ⋆ ) ≤ − 3 δ j .
Proof. • Proof of (a). Lemma D.7 gives the scalar conjugate formula
sup α ∈ R { v α − ℓ q ∣ α ∣ q } = ℓ − 1 q − 1 q ⋆ ∣ v ∣ q ⋆ ( ℓ > 0 , v ∈ R ) . (125) \sup_{\alpha\in\mathbb{R}}\left\{v\alpha-\frac{\ell}{q}|\alpha|^q\right\}=\frac{\ell^{-\frac{1}{q-1}}}{q^{\star}}|v|^{q^{\star}}\qquad(\ell>0,\ v\in\mathbb{R}). \tag*{(125)} α ∈ R sup { v α − q ℓ ∣ α ∣ q } = q ⋆ ℓ − q − 1 1 ∣ v ∣ q ⋆ ( ℓ > 0 , v ∈ R ) . (125)
The chosen constants satisfy
2 c 0 + 11 c 1 = 13 c 1 < 13 64 < 3 8 < 3 4 q ⋆ . 2c_0+11c_1=13c_1<\frac{13}{64}<\frac{3}{8}<\frac{3}{4q^{\star}}. 2 c 0 + 11 c 1 = 13 c 1 < 64 13 < 8 3 < 4 q ⋆ 3 .
since c 1 < 1 64 c_1<\frac{1}{64} c 1 < 64 1 and q ⋆ < 2 q^{\star}<2 q ⋆ < 2 . Fix w ∈ U j , b w\in U_{j,b} w ∈ U j , b , 0 ≤ b ≤ ⌊ m 4 ⌋ 0\le b\le\left\lfloor\frac{m}{4}\right\rfloor 0 ≤ b ≤ ⌊ 4 m ⌋ , and put a = D m w a=D_mw a = D m w . Then a i = 0 a_i=0 a i = 0 for i > b i>b i > b . For 0 ≤ i ≤ b 0\le i\le b 0 ≤ i ≤ b , applying Eq. (D.22) with coefficient π j , i \pi_{j,i} π j , i and weight λ \lambda λ gives
λ q ∣ a i ∣ q − π j , i a i ≥ − λ − 1 q − 1 q ⋆ π j , i q ⋆ ≥ − λ − 1 q − 1 q ⋆ ( λ τ j q − 1 ) q ⋆ = − λ τ j q q ⋆ . \begin{aligned}
\frac{\lambda}{q}|a_i|^q-\pi_{j,i}a_i\ge-\frac{\lambda^{-\frac{1}{q-1}}}{q^{\star}}\pi_{j,i}^{q^{\star}} \\
&\ge-\frac{\lambda^{-\frac{1}{q-1}}}{q^{\star}}\left(\lambda\tau_j^{q-1}\right)^{q^{\star}}=-\frac{\lambda\tau_j^q}{q^{\star}}.
\end{aligned} q λ ∣ a i ∣ q − π j , i a i ≥ − q ⋆ λ − q − 1 1 π j , i q ⋆ ≥ − q ⋆ λ − q − 1 1 ( λ τ j q − 1 ) q ⋆ = − q ⋆ λ τ j q .
Here 0 < π j , i ≤ λ τ j q − 1 0<\pi_{j,i}\le\lambda\tau_j^{q-1} 0 < π j , i ≤ λ τ j q − 1 by Lemma D.5 (c). The power simplification on the last line is
λ − 1 q − 1 ( λ τ j q − 1 ) q ⋆ = λ q ⋆ − 1 q − 1 τ j ( q − 1 ) q ⋆ = λ τ j q , \lambda^{-\frac{1}{q-1}}\left(\lambda\tau_j^{q-1}\right)^{q^{\star}}=\lambda^{q^{\star}-\frac{1}{q-1}}\tau_j^{(q-1)q^{\star}}=\lambda\tau_j^q, λ − q − 1 1 ( λ τ j q − 1 ) q ⋆ = λ q ⋆ − q − 1 1 τ j ( q − 1 ) q ⋆ = λ τ j q ,
since q ∗ − 1 q − 1 = 1 q^*-\frac{1}{q-1}=1 q ∗ − q − 1 1 = 1 and ( q − 1 ) q ∗ = q (q-1)q^*=q ( q − 1 ) q ∗ = q . Thus Eq. (D.20) and Eq. (D.21) give
ζ j ( w , 0 ) = k j , a f f + ∑ i = 0 b ( ∣ a i ∣ q q − π j , i a i ) ≥ k j , a f f − b + 1 q ∗ λ τ j q ≥ [ m + 1 q ∗ ⏟ all rows − b + 1 q ∗ ⏟ rows 0 , … , b − c 0 ( m + 1 ) ⏟ base error ] λ τ j q − 4 δ j = ( m − b q ∗ − c 0 ( m + 1 ) ) λ τ j q − 4 δ j ≥ ( 3 4 q ∗ − 2 c 0 − 4 c 1 ) E j ( b ≤ m 4 , m + 1 ≤ 2 m , δ j = c 1 E j ) ≥ 7 c 1 E j = 7 δ j ( 2 c 0 + 11 c 1 ≤ 3 4 q ∗ ) . \begin{aligned}
\zeta_j(w,0)&=k_{j,\mathrm{aff}}+\sum_{i=0}^{b}\left(\frac{\lvert a_i\rvert^q}{q}-\pi_{j,i}a_i\right)\\
&\ge k_{j,\mathrm{aff}}-\frac{b+1}{q^*}\lambda\tau_j^q\\
&\ge\left[\underbrace{\frac{m+1}{q^*}}_{\text{all rows}}-\underbrace{\frac{b+1}{q^*}}_{\text{rows }0,\ldots,b}-\underbrace{c_0(m+1)}_{\text{base error}}\right]\lambda\tau_j^q-4\delta_j\\
&=\left(\frac{m-b}{q^*}-c_0(m+1)\right)\lambda\tau_j^q-4\delta_j\\
&\ge\left(\frac{3}{4q^*}-2c_0-4c_1\right)\mathcal{E}_j\qquad\left(b\le\frac{m}{4},\ m+1\le2m,\ \delta_j=c_1\mathcal{E}_j\right)\\
&\ge7c_1\mathcal{E}_j=7\delta_j\qquad\left(2c_0+11c_1\le\frac{3}{4q^*}\right).
\end{aligned} ζ j ( w , 0 ) = k j , aff + i = 0 ∑ b ( q ∣ a i ∣ q − π j , i a i ) ≥ k j , aff − q ∗ b + 1 λ τ j q ≥ all rows q ∗ m + 1 − rows 0 , … , b q ∗ b + 1 − base error c 0 ( m + 1 ) λ τ j q − 4 δ j = ( q ∗ m − b − c 0 ( m + 1 ) ) λ τ j q − 4 δ j ≥ ( 4 q ∗ 3 − 2 c 0 − 4 c 1 ) E j ( b ≤ 4 m , m + 1 ≤ 2 m , δ j = c 1 E j ) ≥ 7 c 1 E j = 7 δ j ( 2 c 0 + 11 c 1 ≤ 4 q ∗ 3 ) .
Hence
ζ j ( w , 0 ) ≥ 7 δ j ( w ∈ U j , b , b ≤ ⌊ m 4 ⌋ ) . (126) \zeta_j(w,0)\ge7\delta_j\qquad\left(w\in U_{j,b},\ b\le\left\lfloor\frac{m}{4}\right\rfloor\right). \tag*{(126)} ζ j ( w , 0 ) ≥ 7 δ j ( w ∈ U j , b , b ≤ ⌊ 4 m ⌋ ) . (126)
O j ( h j , t j + 1 ) = P j ( h j , 0 ) + α q t j + 1 q − P j ( h j , 0 ) − 4 δ j = α q t j + 1 q − 4 δ j , O_j(h_j,t_{j+1})=P_j(h_j,0)+\frac{\alpha}{q}t_{j+1}^q-P_j(h_j,0)-4\delta_j=\frac{\alpha}{q}t_{j+1}^q-4\delta_j, O j ( h j , t j + 1 ) = P j ( h j , 0 ) + q α t j + 1 q − P j ( h j , 0 ) − 4 δ j = q α t j + 1 q − 4 δ j ,
I j ( h j , t j + 1 ) = − s j + 1 t j + 1 , I_j(h_j,t_{j+1})=-s_{j+1}t_{j+1}, I j ( h j , t j + 1 ) = − s j + 1 t j + 1 ,
ζ j ( h j , t j + 1 ) = O j ( h j , t j + 1 ) − I j ( h j , t j + 1 ) = α q t j + 1 q + s j + 1 t j + 1 − 4 δ j . \begin{aligned}
\zeta_j(h_j,t_{j+1})&=O_j(h_j,t_{j+1})-I_j(h_j,t_{j+1})\\
&=\frac{\alpha}{q}t_{j+1}^q+s_{j+1}t_{j+1}-4\delta_j.
\end{aligned} ζ j ( h j , t j + 1 ) = O j ( h j , t j + 1 ) − I j ( h j , t j + 1 ) = q α t j + 1 q + s j + 1 t j + 1 − 4 δ j .
At t j + 1 = 0 t_{j+1}=0 t j + 1 = 0 , the same identity holds in the terminal phase with the next-coordinate terms omitted. The definitions and c 0 E j = δ j c_0\mathcal{E}_j=\delta_j c 0 E j = δ j therefore give
ζ j ( h j , 0 ) = − 4 δ j , \zeta_j(h_j,0)=-4\delta_j, ζ j ( h j , 0 ) = − 4 δ j ,
ζ j ( h j , t j + 1 ∗ ) = s j + 1 t j + 1 ∗ + α q ( t j + 1 ∗ ) q − 4 δ j ≤ − 3 δ j , \zeta_j(h_j,t^*_{j+1})=s_{j+1}t^*_{j+1}+\frac{\alpha}{q}(t^*_{j+1})^q-4\delta_j\le-3\delta_j, ζ j ( h j , t j + 1 ∗ ) = s j + 1 t j + 1 ∗ + q α ( t j + 1 ∗ ) q − 4 δ j ≤ − 3 δ j ,
using Eq. (D.26). The subtraction 4 δ j 4\delta_j 4 δ j thus permits a δ j \delta_j δ j increase while keeping ζ j < − δ j \zeta_j<-\delta_j ζ j < − δ j . For j < L − 1 j<L-1 j < L − 1 , the assembly in Appx. E identifies the second input with the following center:
P E j ( c j + 2 − c j ) = P E j ( h j + h j + 1 ) = h j , P_{E_j}(c_{j+2}-c_j)=P_{E_j}(h_j+h_{j+1})=h_j, P E j ( c j + 2 − c j ) = P E j ( h j + h j + 1 ) = h j ,
⟨ v j + 1 , c j + 2 − c j ⟩ = ⟨ v j + 1 , h j + 1 ⟩ = t j + 1 ∗ . \langle v_{j+1},c_{j+2}-c_j\rangle=\langle v_{j+1},h_{j+1}\rangle=t^*_{j+1}. ⟨ v j + 1 , c j + 2 − c j ⟩ = ⟨ v j + 1 , h j + 1 ⟩ = t j + 1 ∗ .
Keeping the same affine formula there leaves the completed phase’s gradient unchanged.
Corollary D.10 (Fenchel–Young bound for the next coordinate). For α > 0 \alpha>0 α > 0 and s j + 1 ≥ 0 s_{j+1}\ge0 s j + 1 ≥ 0 ,
inf t j + 1 ∈ R { α q ∣ t j + 1 ∣ q + s j + 1 t j + 1 } = − α − 1 q − 1 s j + 1 q ∗ q ∗ . \inf_{t_{j+1}\in\mathbb{R}}\left\{\frac{\alpha}{q}\lvert t_{j+1}\rvert^q+s_{j+1}t_{j+1}\right\}=-\frac{\alpha^{-\frac{1}{q-1}}s_{j+1}^{q^*}}{q^*}. t j + 1 ∈ R inf { q α ∣ t j + 1 ∣ q + s j + 1 t j + 1 } = − q ∗ α − q − 1 1 s j + 1 q ∗ .
Consequently, if
α 1 q − 1 s j + 1 ∗ ≤ c 0 E j , (127) \alpha^{\frac{1}{q-1}}s_{j+1}^{*}\le c_0\mathcal{E}_j, \tag*{(127)} α q − 1 1 s j + 1 ∗ ≤ c 0 E j , (127)
then
α q ∣ t j + 1 ∣ q + s j + 1 t j + 1 ≥ − c 0 E j q ∗ ( t j + 1 ∈ R ) . \frac{\alpha}{q}|t_{j+1}|^q+s_{j+1}t_{j+1}\ge-\frac{c_0\mathcal{E}_j}{q^*}\qquad(t_{j+1}\in\mathbb{R}). q α ∣ t j + 1 ∣ q + s j + 1 t j + 1 ≥ − q ∗ c 0 E j ( t j + 1 ∈ R ) .
Proof. Apply the power-conjugate formula Eq. (D.27) with ( a , v , ℓ ) = ( t j + 1 , − s j + 1 , α ) (a,v,\ell)=(t_{j+1},-s_{j+1},\alpha) ( a , v , ℓ ) = ( t j + 1 , − s j + 1 , α ) :
inf t j + 1 { α q ∣ t j + 1 ∣ q + s j + 1 t j + 1 } = − sup t j + 1 { − s j + 1 t j + 1 − α q ∣ t j + 1 ∣ q } = − α − 1 q − 1 s j + 1 ∗ q ∗ q ∗ . \inf_{t_{j+1}}\left\{\frac{\alpha}{q}|t_{j+1}|^q+s_{j+1}t_{j+1}\right\}
=-\sup_{t_{j+1}}\left\{-s_{j+1}t_{j+1}-\frac{\alpha}{q}|t_{j+1}|^q\right\}
=-\frac{\alpha^{-\frac{1}{q-1}}s_{j+1}^{*q^*}}{q^*}. t j + 1 inf { q α ∣ t j + 1 ∣ q + s j + 1 t j + 1 } = − t j + 1 sup { − s j + 1 t j + 1 − q α ∣ t j + 1 ∣ q } = − q ∗ α − q − 1 1 s j + 1 ∗ q ∗ .
For each t j + 1 ∈ R t_{j+1}\in\mathbb{R} t j + 1 ∈ R , Eq. (D.29) therefore implies
α q ∣ t j + 1 ∣ q + s j + 1 t j + 1 ≥ inf a ∈ R { α q ∣ a ∣ q + s j + 1 a } = − α − 1 q − 1 s j + 1 ∗ q ∗ q ∗ ≥ − c 0 E j q ∗ . \frac{\alpha}{q}|t_{j+1}|^q+s_{j+1}t_{j+1}
\ge\inf_{a\in\mathbb{R}}\left\{\frac{\alpha}{q}|a|^q+s_{j+1}a\right\}
=-\frac{\alpha^{-\frac{1}{q-1}}s_{j+1}^{*q^*}}{q^*}
\ge-\frac{c_0\mathcal{E}_j}{q^*}. q α ∣ t j + 1 ∣ q + s j + 1 t j + 1 ≥ a ∈ R inf { q α ∣ a ∣ q + s j + 1 a } = − q ∗ α − q − 1 1 s j + 1 ∗ q ∗ ≥ − q ∗ c 0 E j .
Set
α : = E j ( τ j m 3 2 ) q = λ m 1 − 3 q 2 . (128) \alpha:=\frac{\mathcal{E}_j}{\left(\tau_jm^{\frac{3}{2}}\right)^q}
=\lambda m^{1-\frac{3q}{2}}. \tag*{(128)} α := ( τ j m 2 3 ) q E j = λ m 1 − 2 3 q . (128)
Lemma E.7 in Appx. E verifies both next-coordinate bounds with this choice and the base’s ratio η \eta η .
Verify the phase function
Theorem D.11 (Phase function with A j − I j ≥ 7 δ j A_j-I_j\ge7\delta_j A j − I j ≥ 7 δ j on short chain prefixes). Fix a phase 0 ≤ j < L 0\le j<L 0 ≤ j < L , p ≥ 2 p\ge2 p ≥ 2 , 0 < ν ≤ 1 0<\nu\le1 0 < ν ≤ 1 , q = p + ν q=p+\nu q = p + ν , and an orthonormal block as in Eq. (D.7)–Eq. (D.8). Take
m ≥ 1 , λ , τ j > 0 , G j , s j + 1 , t j + 1 ∗ ≥ 0 , m\ge1,\qquad\lambda,\tau_j>0,\qquad G_j,s_{j+1},t_{j+1}^{*}\ge0, m ≥ 1 , λ , τ j > 0 , G j , s j + 1 , t j + 1 ∗ ≥ 0 , with m , p m,p m , p integers. For j < L − 1 j<L-1 j < L − 1 , take a unit v j + 1 ⊥ E j v_{j+1}\perp E_j v j + 1 ⊥ E j ; part ( f ) (f) ( f ) specifies the terminal convention. Use c 0 , c 1 c_0,c_1 c 0 , c 1 from Eq. (D.24), and set
Γ j = λ τ j q − 1 m , b j = G j h j R j , s j = Γ j + G j t j ∗ R j , \Gamma_j=\frac{\lambda\tau_j^{q-1}}{\sqrt{m}},\qquad
b_j=\frac{G_jh_j}{R_j},\qquad
s_j=\Gamma_j+\frac{G_jt_j^{*}}{R_j}, Γ j = m λ τ j q − 1 , b j = R j G j h j , s j = Γ j + R j G j t j ∗ , α = λ m 1 − 3 q 2 , δ j = c 1 E j , E j = λ m τ j q , \alpha=\lambda m^{1-\frac{3q}{2}},\qquad
\delta_j=c_1\mathcal{E}_j,\qquad
\mathcal{E}_j=\lambda m\tau_j^q, α = λ m 1 − 2 3 q , δ j = c 1 E j , E j = λm τ j q , g j = s j v j − s j + 1 v j + 1 − b j . g_j=s_jv_j-s_{j+1}v_{j+1}-b_j. g j = s j v j − s j + 1 v j + 1 − b j . Assume
G j ≤ c 0 Γ j , G_j\le c_0\Gamma_j, G j ≤ c 0 Γ j , s j + 1 t j + 1 ∗ + α ( t j + 1 ∗ ) q q ≤ c 0 E j , x r e f ‘ E q . ( D .26 ) ‘ , s_{j+1}t_{j+1}^{*}+\frac{\alpha\left(t_{j+1}^{*}\right)^q}{q}\le c_0\mathcal{E}_j,\qquad{xref}`Eq. (D.26)`, s j + 1 t j + 1 ∗ + q α ( t j + 1 ∗ ) q ≤ c 0 E j , x r e f ‘ E q . ( D .26 ) ‘ , α − 1 q − 1 s j + 1 ∗ q ∗ ≤ c 0 E j , x r e f ‘ E q . ( D .29 ) ‘ . \alpha^{-\frac{1}{q-1}}s_{j+1}^{*q^*}\le c_0\mathcal{E}_j,\qquad{xref}`Eq. (D.29)`. α − q − 1 1 s j + 1 ∗ q ∗ ≤ c 0 E j , x r e f ‘ E q . ( D .29 ) ‘. For I j , O j , A j I_j,O_j,A_j I j , O j , A j from Eq. (D.15), Eq. (D.16), Eq. (D.17), and U j , b = span { v j , u j , 1 , … , u j , b } U_{j,b}=\operatorname{span}\{v_j,u_{j,1},\ldots,u_{j,b}\} U j , b = span { v j , u j , 1 , … , u j , b } :
(a) Convexity. A j A_j A j is convex and A j ≥ max { I j , O j } A_j \ge\max\{I_j,O_j\} A j ≥ max { I j , O j } .
(b) Chain-prefix excess and oracle output. For every integer 0 ≤ b ≤ ⌊ m / 4 ⌋ 0 \le b \le\lfloor m/4 \rfloor 0 ≤ b ≤ ⌊ m /4 ⌋ and every w ∈ U j , b w \in U_{j,b} w ∈ U j , b , without a bound on ∥ w ∥ \lVert w\rVert ∥ w ∥ ,
O j ( w , 0 ) − I j ( w , 0 ) ≥ 7 δ j , A j = O j near ( w , 0 ) . O_j(w,0)-I_j(w,0) \ge7\delta_j,\qquad A_j=O_j\ \text{near }(w,0). O j ( w , 0 ) − I j ( w , 0 ) ≥ 7 δ j , A j = O j near ( w , 0 ) .
The value and full ambient derivatives through order p p p , given by Eq. (D.36)–(D.37), depend only on v j , u j , 1 , … , u j , b + 1 v_j,u_{j,1},\ldots,u_{j,b+1} v j , u j , 1 , … , u j , b + 1 and fixed scalars.
(c) Affine region. A j = I j A_j=I_j A j = I j near { ( h j , t j + 1 ) : 0 ≤ t j + 1 ≤ t j + 1 ⋆ } \{(h_j,t_{j+1}):0\le t_{j+1}\le t^\star_{j+1}\} {( h j , t j + 1 ) : 0 ≤ t j + 1 ≤ t j + 1 ⋆ } , and
A j ( h j , 0 ) = 0 , A j ( h j , t j + 1 ⋆ ) = − s j + 1 t j + 1 ⋆ . (129) A_j(h_j,0)=0,\qquad A_j(h_j,t^\star_{j+1})=-s_{j+1}t^\star_{j+1}. \tag*{(129)} A j ( h j , 0 ) = 0 , A j ( h j , t j + 1 ⋆ ) = − s j + 1 t j + 1 ⋆ . (129)
∇ A j ( h j , 0 ) = g j , ∇ A j ( h j , t j + 1 ⋆ ) = g j . (130) \nabla A_j(h_j,0)=g_j,\qquad\nabla A_j(h_j,t^\star_{j+1})=g_j. \tag*{(130)} ∇ A j ( h j , 0 ) = g j , ∇ A j ( h j , t j + 1 ⋆ ) = g j . (130)
∇ k A j ( h j , 0 ) = 0 , ∇ k A j ( h j , t j + 1 ⋆ ) = 0 ( 2 ≤ k ≤ p ) . (131) \nabla^k A_j(h_j,0)=0,\qquad\nabla^k A_j(h_j,t^\star_{j+1})=0\quad(2\le k\le p). \tag*{(131)} ∇ k A j ( h j , 0 ) = 0 , ∇ k A j ( h j , t j + 1 ⋆ ) = 0 ( 2 ≤ k ≤ p ) . (131)
(d) Starting derivatives and gradient change.
A j ( 0 , 0 ) = − λ q ( m + 1 ) τ j q − 4 δ j , ∇ k A j ( 0 , 0 ) = 0 ( 1 ≤ k ≤ p ) . (132) A_j(0,0)=-\frac{\lambda}{q}(m+1)\tau_j^q-4\delta_j,\qquad\nabla^k A_j(0,0)=0\quad(1\le k\le p). \tag*{(132)} A j ( 0 , 0 ) = − q λ ( m + 1 ) τ j q − 4 δ j , ∇ k A j ( 0 , 0 ) = 0 ( 1 ≤ k ≤ p ) . (132)
Consequently,
b j + ( ∇ A j ( h j , 0 ) − ∇ A j ( 0 , 0 ) ) = b j + g j = s j v j − s j + 1 v j + 1 . (133) b_j+\bigl(\nabla A_j(h_j,0)-\nabla A_j(0,0)\bigr)=b_j+g_j=s_jv_j-s_{j+1}v_{j+1}. \tag*{(133)} b j + ( ∇ A j ( h j , 0 ) − ∇ A j ( 0 , 0 ) ) = b j + g j = s j v j − s j + 1 v j + 1 . (133)
(e) Smoothness. A j ∈ C p , ν A_j\in C^{p,\nu} A j ∈ C p , ν and, for all x , y ∈ R d x,y\in\mathbb{R}^d x , y ∈ R d ,
∥ ∇ p A j ( x ) − ∇ p A j ( y ) ∥ ≤ C A λ ∥ x − y ∥ ν , \lVert\nabla^p A_j(x)-\nabla^p A_j(y)\rVert\le C_A\lambda\lVert x-y\rVert^\nu, ∥ ∇ p A j ( x ) − ∇ p A j ( y )∥ ≤ C A λ ∥ x − y ∥ ν ,
where C A C_A C A in Eq. (D.47) depends only on p , ν p,\nu p , ν and the fixed smoothing function.
(f) Terminal phase. For j = L − 1 j=L-1 j = L − 1 , parts (a)–(e) hold with s L = t L ⋆ = 0 s_L=t^\star_L=0 s L = t L ⋆ = 0 , v L v_L v L omitted, and the functions restricted to t j + 1 = 0 t_{j+1}=0 t j + 1 = 0 .
Proof. Define E j = λ m τ j q \mathcal{E}_j=\lambda m\tau_j^q E j = λm τ j q and ζ j = O j − I j \zeta_j=O_j-I_j ζ j = O j − I j .
Proof of (a). The scalar function ϕ ( a ) = ∣ a ∣ q q \phi(a)=\frac{|a|^q}{q} ϕ ( a ) = q ∣ a ∣ q is convex and C p C^p C p by Lemma B.16 (b), (c) in Appx. B. Sums and affine substitutions preserve convexity [9 Secs. 3.2.1–3.2.2] (Lemma B.19 (c)–(d)), so P j , O j , ζ j P_j,O_j,\zeta_j P j , O j , ζ j are convex and C p C^p C p . Since I j I_j I j is affine, Lemma B.21 (a) gives A j = M δ j ( I j , O j ) A_j=M_{\delta_j}(I_j,O_j) A j = M δ j ( I j , O j ) convex and C p C^p C p ; Lemma 4.2 gives A j ≥ I j , O j A_j\ge I_j,O_j A j ≥ I j , O j .
Proof of (b). The affine gradient g j g_j g j can involve unrevealed directions, so the oracle output at every w ∈ U j , b w\in U_{j,b} w ∈ U j , b with b ≤ ⌊ m / 4 ⌋ b\le\lfloor m/4\rfloor b ≤ ⌊ m /4 ⌋ must come from O j O_j O j . Fix w ∈ U j , b w\in U_{j,b} w ∈ U j , b , 0 ≤ b ≤ ⌊ m / 4 ⌋ 0\le b\le\lfloor m/4\rfloor 0 ≤ b ≤ ⌊ m /4 ⌋ . The theorem’s choices of c 0 , c 1 , δ j c_0,c_1,\delta_j c 0 , c 1 , δ j and its base and endpoint assumptions are precisely those of Lemma D.9 . Its part (a) gives
ζ j ( w , 0 ) ≥ 7 δ j ( w ∈ U j , b , 0 ≤ b ≤ ⌊ m / 4 ⌋ ) . \zeta_j(w,0)\ge7\delta_j\qquad(w\in U_{j,b},\ 0\le b\le\lfloor m/4\rfloor). ζ j ( w , 0 ) ≥ 7 δ j ( w ∈ U j , b , 0 ≤ b ≤ ⌊ m /4 ⌋) .
Continuity and 7 δ j > δ j 7\delta_j>\delta_j 7 δ j > δ j give A j = O j A_j=O_j A j = O j near ( w , 0 ) (w,0) ( w , 0 ) by Lemma B.21 (c); hence A j ( w , 0 ) − I j ( w , 0 ) ≥ 7 δ j A_j(w,0)-I_j(w,0)\ge7\delta_j A j ( w , 0 ) − I j ( w , 0 ) ≥ 7 δ j .
The full oracle output. Apply Lemma D.3 (d) to Q j Q_j Q j with prefix index k = b k=b k = b , which is permitted because b ≤ ⌊ m / 4 ⌋ < m b\le\lfloor m/4\rfloor<m b ≤ ⌊ m /4 ⌋ < m . The added term α ϕ ( t j + 1 ) \alpha\phi(t_{j+1}) α ϕ ( t j + 1 ) and its derivatives through order p p p vanish at t j + 1 = 0 t_{j+1}=0 t j + 1 = 0 by Lemma B.16 (b). Thus A j = O j A_j=O_j A j = O j near the evaluation point and the formulas in Lemma D.3 (d) give, for 1 ≤ k ≤ p 1\le k\le p 1 ≤ k ≤ p and arbitrary ambient d 1 , … , d k d_1,\ldots,d_k d 1 , … , d k ,
∇ k A j ( w , 0 ) [ d 1 , … , d k ] = λ ∑ i = 0 b ϕ ( k ) ( ( D m w ) i ) ∏ ℓ = 1 k ⟨ a j , i , d ℓ ⟩ . (134) \nabla^k A_j(w,0)[d_1,\ldots,d_k]=\lambda\sum_{i=0}^{b}\phi^{(k)}\bigl((D_mw)_i\bigr)\prod_{\ell=1}^{k}\langle a_{j,i},d_\ell\rangle. \tag*{(134)} ∇ k A j ( w , 0 ) [ d 1 , … , d k ] = λ i = 0 ∑ b ϕ ( k ) ( ( D m w ) i ) ℓ = 1 ∏ k ⟨ a j , i , d ℓ ⟩ . (134)
The value is
A j ( w , 0 ) = λ q ∑ i = 0 b ∣ ( D m w ) i ∣ q − λ q ( m + 1 ) τ j q − 4 δ j . (135) A_j(w,0)=\frac{\lambda}{q}\sum_{i=0}^{b}\lvert(D_mw)_i\rvert^q-\frac{\lambda}{q}(m+1)\tau_j^q-4\delta_j. \tag*{(135)} A j ( w , 0 ) = q λ i = 0 ∑ b ∣( D m w ) i ∣ q − q λ ( m + 1 ) τ j q − 4 δ j . (135)
The row vectors a j , 0 , … , a j , b a_{j,0},\ldots,a_{j,b} a j , 0 , … , a j , b involve only v j , u j , 1 , … , u j , b + 1 v_j,u_{j,1},\ldots,u_{j,b+1} v j , u j , 1 , … , u j , b + 1 , so they belong to U j , b + 1 U_{j,b+1} U j , b + 1 . For the ambient functions in Eq. (17) , the point c j + w c_j+w c j + w has local inputs ( w , 0 ) (w,0) ( w , 0 ) . Orthogonal projection therefore leaves every inner product in Eq. (D.36) unchanged, giving
∇ k A j ( c j + w ) [ d 1 , … , d k ] = λ ∑ i = 0 b ϕ ( k ) ( ( D m w ) i ) ∏ ℓ = 1 k ⟨ a j , i , P U j , b + 1 d ℓ ⟩ = ∇ k Q j ( w ) [ P U j , b + 1 d 1 , … , P U j , b + 1 d k ] \begin{aligned}
\nabla^k A_j(c_j+w)[d_1,\ldots,d_k]=\lambda\sum_{i=0}^{b}\phi^{(k)}\bigl((D_mw)_i\bigr)\prod_{\ell=1}^{k}\langle a_{j,i},P_{U_{j,b+1}}d_\ell\rangle\\
&=\nabla^k Q_j(w)[P_{U_{j,b+1}}d_1,\ldots,P_{U_{j,b+1}}d_k]
\end{aligned} ∇ k A j ( c j + w ) [ d 1 , … , d k ] = λ i = 0 ∑ b ϕ ( k ) ( ( D m w ) i ) ℓ = 1 ∏ k ⟨ a j , i , P U j , b + 1 d ℓ ⟩ = ∇ k Q j ( w ) [ P U j , b + 1 d 1 , … , P U j , b + 1 d k ]
by Lemma D.3 (d).
Thus the value and full ambient tensors agree for all remaining completions. This is pointwise agreement at the evaluation point; completions need not agree nearby.
Proof of (c). After the chain is complete, A j A_j A j must equal I j I_j I j near the endpoint segment to supply the fixed gradient g j g_j g j . Lemma D.9 (b) gives ζ j ( h j , 0 ) = − 4 δ j \zeta_j(h_j,0)=-4\delta_j ζ j ( h j , 0 ) = − 4 δ j and ζ j ( h j , t j + 1 ⋆ ) ≤ − 3 δ j \zeta_j(h_j,t_{j+1}^{\star})\le-3\delta_j ζ j ( h j , t j + 1 ⋆ ) ≤ − 3 δ j . The function ζ j = P j − I j − P j ( h j , 0 ) − 4 δ j \zeta_j=P_j-I_j-P_j(h_j,0)-4\delta_j ζ j = P j − I j − P j ( h j , 0 ) − 4 δ j is convex by Lemma B.19 (c). Its value at an intermediate point is therefore at most the weighted average of the endpoint values, hence at most − 3 δ j -3\delta_j − 3 δ j , as proved in Lemma D.13 (d). The segment therefore lies in the open set { ζ j < − δ j } \{\zeta_j<-\delta_j\} { ζ j < − δ j } , where Lemma B.21 (c) gives A j = I j A_j=I_j A j = I j . Evaluating and differentiating Eq. (D.15) on this region gives
I j ( h j , t j + 1 ) = − s j + 1 t j + 1 , ∇ I j = g j , ∇ k I j = 0 ( 2 ≤ k ≤ p ) , I_j(h_j,t_{j+1})=-s_{j+1}t_{j+1},\qquad\nabla I_j=g_j,\qquad\nabla^k I_j=0\quad(2\le k\le p), I j ( h j , t j + 1 ) = − s j + 1 t j + 1 , ∇ I j = g j , ∇ k I j = 0 ( 2 ≤ k ≤ p ) ,
which gives Eq. (D.31)–(D.33).
Proof of (d). At ( 0 , 0 ) (0,0) ( 0 , 0 ) , part (b) gives A j = O j A_j=O_j A j = O j on a neighborhood. Evaluating Eq. (D.16) with the power terms and their derivatives zero gives the starting data. More explicitly, D m h j = τ j 1 D_mh_j=\tau_j\mathbf{1} D m h j = τ j 1 , so
A j ( 0 , 0 ) = P j ( 0 , 0 ) − P j ( h j , 0 ) − 4 δ j = λ q ( m + 1 ) τ j q − 4 δ j , \begin{aligned}
A_j(0,0)=P_j(0,0)-P_j(h_j,0)-4\delta_j \\
&=\frac{\lambda}{q}(m+1)\tau_j^q-4\delta_j,
\end{aligned} A j ( 0 , 0 ) = P j ( 0 , 0 ) − P j ( h j , 0 ) − 4 δ j = q λ ( m + 1 ) τ j q − 4 δ j ,
∇ k A j ( 0 , 0 ) = ∇ k P j ( 0 , 0 ) = 0 \nabla^k A_j(0,0)=\nabla^k P_j(0,0)=0 ∇ k A j ( 0 , 0 ) = ∇ k P j ( 0 , 0 ) = 0
by Lemma B.16 (b), for 1 ≤ k ≤ p 1\le k\le p 1 ≤ k ≤ p .
Using the endpoint gradient from part (c),
b j + ( ∇ A j ( h j , 0 ) − ∇ A j ( 0 , 0 ) ) = b j + g j = b j + ( s j v j − s j + 1 v j + 1 − b j ) = s j v j − s j + 1 v j + 1 . \begin{aligned}
b_j+\bigl(\nabla A_j(h_j,0)-\nabla A_j(0,0)\bigr)=b_j+g_j \\
&=b_j+\bigl(s_jv_j-s_{j+1}v_{j+1}-b_j\bigr) \\
&=s_jv_j-s_{j+1}v_{j+1}.
\end{aligned} b j + ( ∇ A j ( h j , 0 ) − ∇ A j ( 0 , 0 ) ) = b j + g j = b j + ( s j v j − s j + 1 v j + 1 − b j ) = s j v j − s j + 1 v j + 1 .
A j = O j + K j , K j : = ψ δ j ( ζ j ) − ζ j , T j : = { x : ζ j ( x ) ≤ δ j } . A_j=O_j+K_j,\qquad K_j:=\psi_{\delta_j}(\zeta_j)-\zeta_j,\qquad T_j:=\{x:\zeta_j(x)\le\delta_j\}. A j = O j + K j , K j := ψ δ j ( ζ j ) − ζ j , T j := { x : ζ j ( x ) ≤ δ j } .
Lemma B.17 (a) gives K j = 0 K_j=0 K j = 0 on T j c T_j^c T j c . On the remaining region, Lemma D.17 controls the values of P j P_j P j by a fixed multiple of δ j \delta_j δ j . To keep the smoothing correction’s Hölder parameter independent of the phase scale, we normalize its lengths by
ρ j A ≔ ( δ j λ ) 1 q , λ ( ρ j A ) q = δ j , \rho_j^A \coloneqq\left(\frac{\delta_j}{\lambda}\right)^{\frac{1}{q}},
\qquad\lambda\left(\rho_j^A\right)^q=\delta_j, ρ j A : = ( λ δ j ) q 1 , λ ( ρ j A ) q = δ j ,
and values by δ j \delta_j δ j . The composition estimate needs these bounds on a neighborhood, so Lemma D.17 (b) also controls the points within distance ρ j A \rho_j^A ρ j A of T j T_j T j . On this neighborhood,
q P j ( w , t j + 1 ) = λ ∥ D m w ∥ q + α ∣ t j + 1 ∣ q ≤ C N δ j qP_j\left(w,t_{j+1}\right)=\lambda\lVert D_mw\rVert^q+\alpha\lvert t_{j+1}\rvert^q\le C_N\delta_j q P j ( w , t j + 1 ) = λ ∥ D m w ∥ q + α ∣ t j + 1 ∣ q ≤ C N δ j
by Lemma D.17 (b),
∥ ∇ k ζ j ∥ ≤ M k λ ( ρ j A ) q − k , 1 ≤ k ≤ p \lVert\nabla^k\zeta_j\rVert\le M_k\lambda\left(\rho_j^A\right)^{q-k},\qquad1\le k\le p ∥ ∇ k ζ j ∥ ≤ M k λ ( ρ j A ) q − k , 1 ≤ k ≤ p
by Lemma D.18.
Here Eq. (D.29) controls the linear term s j + 1 t j + 1 s_{j+1}t_{j+1} s j + 1 t j + 1 in ζ j \zeta_j ζ j , so the constants are independent of the next phase’s direction and scale. With the normalized value and derivative bounds, Lemma D.19 (a) gives, for all x , y ∈ R d x,y\in\mathbb{R}^d x , y ∈ R d ,
∥ ∇ p K j ( x ) − ∇ p K j ( y ) ∥ ≤ C K δ j ( ρ j A ) p + ν ∥ x − y ∥ ν = C K δ j ( ρ j A ) q ∥ x − y ∥ ν \begin{aligned}
\lVert\nabla^pK_j(x)-\nabla^pK_j(y)\rVert\le C_K\frac{\delta_j}{\left(\rho_j^A\right)^{p+\nu}}\lVert x-y\rVert^\nu\\
&= C_K\frac{\delta_j}{\left(\rho_j^A\right)^q}\lVert x-y\rVert^\nu
\end{aligned} ∥ ∇ p K j ( x ) − ∇ p K j ( y )∥ ≤ C K ( ρ j A ) p + ν δ j ∥ x − y ∥ ν = C K ( ρ j A ) q δ j ∥ x − y ∥ ν
since q = p + ν q=p+\nu q = p + ν ,
= C K λ ∥ x − y ∥ ν = C_K\lambda\lVert x-y\rVert^\nu = C K λ ∥ x − y ∥ ν
by the definition of ρ j A \rho_j^A ρ j A .
It remains to add P j P_j P j ’s global bound. Since O j − P j O_j-P_j O j − P j is constant and p ≥ 2 p\ge2 p ≥ 2 , ∇ p O j = ∇ p P j \nabla^pO_j=\nabla^pP_j ∇ p O j = ∇ p P j . With α = λ m 1 − 2 q ≤ λ \alpha=\lambda m^{1-\frac{2}{q}}\le\lambda α = λ m 1 − q 2 ≤ λ , Lemma D.15 (a) gives Hölder parameter L P λ L_P\lambda L P λ for P j P_j P j . The decomposition A j = O j + K j A_j=O_j+K_j A j = O j + K j therefore gives
∥ ∇ p A j ( x ) − ∇ p A j ( y ) ∥ ≤ ∥ ∇ p O j ( x ) − ∇ p O j ( y ) ∥ + ∥ ∇ p K j ( x ) − ∇ p K j ( y ) ∥ ≤ L P λ ∥ x − y ∥ ν + C K λ ∥ x − y ∥ ν = ( L P + C K ) λ ∥ x − y ∥ ν = C A λ ∥ x − y ∥ ν . \begin{aligned}
\lVert\nabla^pA_j(x)-\nabla^pA_j(y)\rVert
&\le\lVert\nabla^pO_j(x)-\nabla^pO_j(y)\rVert+\lVert\nabla^pK_j(x)-\nabla^pK_j(y)\rVert\\
&\le L_P\lambda\lVert x-y\rVert^\nu+C_K\lambda\lVert x-y\rVert^\nu\\
&=(L_P+C_K)\lambda\lVert x-y\rVert^\nu=C_A\lambda\lVert x-y\rVert^\nu.
\end{aligned} ∥ ∇ p A j ( x ) − ∇ p A j ( y )∥ ≤ ∥ ∇ p O j ( x ) − ∇ p O j ( y )∥ + ∥ ∇ p K j ( x ) − ∇ p K j ( y )∥ ≤ L P λ ∥ x − y ∥ ν + C K λ ∥ x − y ∥ ν = ( L P + C K ) λ ∥ x − y ∥ ν = C A λ ∥ x − y ∥ ν .
The constants L P , C K , C A L_P,C_K,C_A L P , C K , C A depend only on p , ν p,\nu p , ν and the fixed smooth maximum, giving the claimed global bound independently of m m m , dimension, and phase scale.
Proof of (f). For s j + 1 = t j + 1 ∗ = 0 s_{j+1}=t_{j+1}^*=0 s j + 1 = t j + 1 ∗ = 0 , both next-coordinate bounds reduce to 0 ≤ c 0 E j 0\le c_0\mathcal{E}_j 0 ≤ c 0 E j . Restrict the construction on E j × R E_j\times\mathbb{R} E j × R by the linear isometry J j w = ( w , 0 ) J_jw=(w,0) J j w = ( w , 0 ) . Then
∇ k ( A j ∘ J j ) ( w ) [ d 1 , … , d k ] = ∇ k A j ( J j w ) [ J j d 1 , … , J j d k ] , \nabla^k(A_j\circ J_j)(w)[d_1,\ldots,d_k]=\nabla^kA_j(J_jw)[J_jd_1,\ldots,J_jd_k], ∇ k ( A j ∘ J j ) ( w ) [ d 1 , … , d k ] = ∇ k A j ( J j w ) [ J j d 1 , … , J j d k ] ,
so convexity and the Hölder bound restrict to E j E_j E j . The formulas above restrict by setting t j + 1 = 0 t_{j+1}=0 t j + 1 = 0 , and g j = s j v j − b j ∈ E j g_j=s_jv_j-b_j\in E_j g j = s j v j − b j ∈ E j . No additional ambient direction v j + 1 v_{j+1} v j + 1 is needed.
Prove preservation of the oracle outputs
Theorem D.11 (b) proves R4 for the phase function. For the base, we must separate W j , b W_{j,b} W j , b from the next modification ball.
Apply the endpoint construction in mutually orthogonal blocks E j = span { v j , u j , 1 , … , u j , m } E_j=\operatorname{span}\{v_j,u_{j,1},\ldots,u_{j,m}\} E j = span { v j , u j , 1 , … , u j , m } :
t j ⋆ = ( m + 1 ) m τ j , h j = t j ⋆ v j + ∑ i = 1 m ( m − i + 1 ) τ j u j , i . (136) t_j^\star=(m+1)\sqrt{m}\,\tau_j,\qquad h_j=t_j^\star v_j+\sum_{i=1}^{m}(m-i+1)\tau_j u_{j,i}. \tag*{(136)} t j ⋆ = ( m + 1 ) m τ j , h j = t j ⋆ v j + i = 1 ∑ m ( m − i + 1 ) τ j u j , i . (136)
Set c 0 = 0 c_0=0 c 0 = 0 , c j + 1 = c j + h j c_{j+1}=c_j+h_j c j + 1 = c j + h j , and R j = ∥ h j ∥ R_j=\lVert h_j\rVert R j = ∥ h j ∥ . In particular,
R j 2 = τ j 2 [ m ( m + 1 ) 2 + ∑ k = 1 m k 2 ] ≤ 5 τ j 2 m 3 . (137) R_j^2=\tau_j^2\left[m(m+1)^2+\sum_{k=1}^{m}k^2\right]\leq5\tau_j^2m^3. \tag*{(137)} R j 2 = τ j 2 [ m ( m + 1 ) 2 + k = 1 ∑ m k 2 ] ≤ 5 τ j 2 m 3 . (137)
Thus τ j = η j τ 0 \tau_j=\eta^j\tau_0 τ j = η j τ 0 gives R j = η j R 0 R_j=\eta^jR_0 R j = η j R 0 .
Lemma D.12 (Future base changes miss every early prefix). Let W j , b W_{j,b} W j , b be the span of all earlier blocks, v j , u j , 1 , … , u j , b v_j,u_{j,1},\ldots,u_{j,b} v j , u j , 1 , … , u j , b , and any additional directions orthogonal to all hard blocks. Suppose 0 ≤ b ≤ ⌊ m 2 ⌋ 0\leq b\leq\left\lfloor\frac{m}{2}\right\rfloor 0 ≤ b ≤ ⌊ 2 m ⌋ and θ ≤ 1 16 \theta\leq\frac{1}{16} θ ≤ 16 1 .
(a) Separation.
dist ( c j + 1 , W j , b ) ≥ R j 120 > θ R j . (138) \operatorname{dist}(c_{j+1},W_{j,b})\geq\frac{R_j}{\sqrt{120}}>\theta R_j. \tag*{(138)} dist ( c j + 1 , W j , b ) ≥ 120 R j > θ R j . (138) (b) Base agreement. If every modification from B j B_j B j onward lies in S j = B ( c j + 1 , θ R j ) S_j=B(c_{j+1},\theta R_j) S j = B ( c j + 1 , θ R j ) , then
B = B j near every x ∈ W j , b . B=B_j\quad\text{near every }x\in W_{j,b}. B = B j near every x ∈ W j , b .
Proof. • Proof of (a). The previous center c j c_j c j belongs to W j , b W_{j,b} W j , b . From Eq. (D.38), the component of c j + 1 = c j + h j c_{j+1}=c_j+h_j c j + 1 = c j + h j perpendicular to W j , b W_{j,b} W j , b is
∑ i = b + 1 m ( m − i + 1 ) τ j u j , i . \sum_{i=b+1}^{m}(m-i+1)\tau_j u_{j,i}. i = b + 1 ∑ m ( m − i + 1 ) τ j u j , i .
Orthogonality and b ≤ m 2 b\leq\frac{m}{2} b ≤ 2 m therefore give
dist ( c j + 1 , W j , b ) 2 = τ j 2 ∑ i = b + 1 m ( m − i + 1 ) 2 = τ j 2 ∑ a = 1 m − b a 2 = τ j 2 ( m − b ) ( m − b + 1 ) ( 2 ( m − b ) + 1 ) 6 = τ j 2 6 ( 2 ( m − b ) 3 + 3 ( m − b ) 2 + ( m − b ) ) ≥ τ j 2 ( m − b ) 3 3 ≥ τ j 2 ( m 2 ) 3 3 = τ j 2 m 3 24 . \begin{aligned}
\operatorname{dist}(c_{j+1},W_{j,b})^2
=\tau_j^2\sum_{i=b+1}^{m}(m-i+1)^2 \\
&=\tau_j^2\sum_{a=1}^{m-b}a^2 \\
&=\tau_j^2\frac{(m-b)(m-b+1)(2(m-b)+1)}{6} \\
&=\frac{\tau_j^2}{6}\left(2(m-b)^3+3(m-b)^2+(m-b)\right) \\
&\geq\frac{\tau_j^2(m-b)^3}{3} \\
&\geq\frac{\tau_j^2\left(\frac{m}{2}\right)^3}{3}
=\frac{\tau_j^2m^3}{24}.
\end{aligned} dist ( c j + 1 , W j , b ) 2 = τ j 2 i = b + 1 ∑ m ( m − i + 1 ) 2 = τ j 2 a = 1 ∑ m − b a 2 = τ j 2 6 ( m − b ) ( m − b + 1 ) ( 2 ( m − b ) + 1 ) = 6 τ j 2 ( 2 ( m − b ) 3 + 3 ( m − b ) 2 + ( m − b ) ) ≥ 3 τ j 2 ( m − b ) 3 ≥ 3 τ j 2 ( 2 m ) 3 = 24 τ j 2 m 3 .
Apply Lemma D.3 (c) to the j t h j^{\mathit{th}} j th endpoint. Its bound R j 2 ≤ 5 τ j 2 m 3 R_j^2\leq5\tau_j^2m^3 R j 2 ≤ 5 τ j 2 m 3 then yields
dist ( c j + 1 , W j , b ) 2 ≥ τ j 2 m 3 24 ≥ R j 2 5 ⋅ 24 = R j 2 120 , \operatorname{dist}(c_{j+1},W_{j,b})^2\geq\frac{\tau_j^2m^3}{24}\geq\frac{R_j^2}{5\cdot24}=\frac{R_j^2}{120}, dist ( c j + 1 , W j , b ) 2 ≥ 24 τ j 2 m 3 ≥ 5 ⋅ 24 R j 2 = 120 R j 2 ,
dist ( c j + 1 , W j , b ) ≥ R j 120 > θ R j , (139) \operatorname{dist}(c_{j+1},W_{j,b})\geq\frac{R_j}{\sqrt{120}}>\theta R_j, \tag*{(139)} dist ( c j + 1 , W j , b ) ≥ 120 R j > θ R j , (139)
because θ ≤ 1 16 < 1 120 \theta\le\frac{1}{16}<\frac{1}{\sqrt{120}} θ ≤ 16 1 < 120 1 . This is Eq. (D.40).
Proof of (b). For any x ∈ W j , b x\in W_{j,b} x ∈ W j , b , part (a) gives
dist ( x , S j ‾ ) ≥ dist ( c j + 1 , W j , b ) − θ R j > 0. \operatorname{dist}(x,\overline{S_j})\ge\operatorname{dist}(c_{j+1},W_{j,b})-\theta R_j>0. dist ( x , S j ) ≥ dist ( c j + 1 , W j , b ) − θ R j > 0.
The assumed containment of every later modification in S j S_j S j and this separation verify the hypotheses of Lemma C.3 (a) in Appx. C. Applying it with W = W j , b W=W_{j,b} W = W j , b gives
on a neighborhood of every x ∈ W j , b x\in W_{j,b} x ∈ W j , b .
The displayed distance bound is independent of ∥ x ∥ \lVert x\rVert ∥ x ∥ .
Since B j B_j B j uses only selected centers and Euclidean distances, this neighborhood agreement makes the base’s oracle output independent of remaining directions. A fixed fraction of missing links is essential: with one link left, dist ( c j + 1 , W j , m − 1 ) = τ j \operatorname{dist}(c_{j+1},W_{j,m-1})=\tau_j dist ( c j + 1 , W j , m − 1 ) = τ j , whereas θ R j ≥ θ τ j m 3 2 \theta R_j\ge\theta\tau_jm^{\frac{3}{2}} θ R j ≥ θ τ j m 2 3 .
The scalar assumptions of Theorem D.11 are checked in Appx. E. In particular, Corollary E.6 derives the coefficient λ = K μ m 3 q − 2 2 \lambda=K\mu m^{\frac{3q-2}{2}} λ = K μ m 2 3 q − 2 needed to control the base increment.
Technical lemmas for the phase construction
Fix 0 ≤ j < L 0\le j<L 0 ≤ j < L . Use x = ( w , t j + 1 ) ∈ E j × R x=(w,t_{j+1})\in E_j\times\mathbb{R} x = ( w , t j + 1 ) ∈ E j × R and the ordered basis ( v j , u j , 1 , … , u j , m ) (v_j,u_{j,1},\ldots,u_{j,m}) ( v j , u j , 1 , … , u j , m ) for D m w D_mw D m w . The terminal phase omits t j + 1 t_{j+1} t j + 1 and all next-phase terms.
Value and derivative formulas under weaker assumptions
Lemma D.13 (Convexity, the residual gap, and the endpoint derivatives). Under the assumptions of Lemma D.9 :
(a) Convexity. A j A_j A j is convex and A j ≥ I j , O j A_j\ge I_j,O_j A j ≥ I j , O j .
(b) Prefix. For w ∈ U j , b w\in U_{j,b} w ∈ U j , b and 0 ≤ b ≤ ⌊ m 4 ⌋ 0\le b\le\left\lfloor\frac{m}{4}\right\rfloor 0 ≤ b ≤ ⌊ 4 m ⌋ ,
A j = O j near ( w , 0 ) , A j ( w , 0 ) − I j ( w , 0 ) ≥ 7 δ j . A_j=O_j\ \text{near }(w,0),\qquad A_j(w,0)-I_j(w,0)\ge7\delta_j. A j = O j near ( w , 0 ) , A j ( w , 0 ) − I j ( w , 0 ) ≥ 7 δ j .
(c) Origin. As in Eq. (D.34),
A j ( 0 , 0 ) = − λ q ( m + 1 ) τ j q − 4 δ j , ∇ k A j ( 0 , 0 ) = 0 ( 1 ≤ k ≤ p ) . A_j(0,0)=-\frac{\lambda}{q}(m+1)\tau_j^q-4\delta_j,\qquad\nabla^k A_j(0,0)=0\quad(1\le k\le p). A j ( 0 , 0 ) = − q λ ( m + 1 ) τ j q − 4 δ j , ∇ k A j ( 0 , 0 ) = 0 ( 1 ≤ k ≤ p ) .
(d) Affine region. A j = I j A_j=I_j A j = I j near [ ( h j , 0 ) , ( h j , t j + 1 ⋆ ) ] [(h_j,0),(h_j,t_{j+1}^{\star})] [( h j , 0 ) , ( h j , t j + 1 ⋆ )] . The endpoint data are Eq. (D.31)–(D.33); the gradient increment is Eq. (D.35).
Proof. Proof of (a). The positive weighted sum defining P j P_j P j is convex, since ϕ ( a ) = ∣ a ∣ q q \phi(a)=\frac{|a|^q}{q} ϕ ( a ) = q ∣ a ∣ q is convex by Lemma B.16 (c) in Appx. B. Hence O j O_j O j is convex and I j I_j I j is affine. Lemma B.21 (a) gives convexity of A j A_j A j , and Lemma 4.2 gives A j ≥ I j , O j A_j\ge I_j,O_j A j ≥ I j , O j .
Proof of (b). Lemma D.9 (a) gives ζ j ( w , 0 ) ≥ 7 δ j > δ j \zeta_j(w,0) \ge7\delta_j > \delta_j ζ j ( w , 0 ) ≥ 7 δ j > δ j . Continuity preserves the strict last inequality on a neighborhood of ( w , 0 ) (w,0) ( w , 0 ) . Lemma B.21 (c) then gives
A j ( w , 0 ) − I j ( w , 0 ) = O j ( w , 0 ) − I j ( w , 0 ) = ζ j ( w , 0 ) ≥ 7 δ j . A_j(w,0)-I_j(w,0)=O_j(w,0)-I_j(w,0)=\zeta_j(w,0)\ge7\delta_j. A j ( w , 0 ) − I j ( w , 0 ) = O j ( w , 0 ) − I j ( w , 0 ) = ζ j ( w , 0 ) ≥ 7 δ j .
Proof of (c). Apply the preceding part with w = 0 ∈ U j , 0 w=0\in U_{j,0} w = 0 ∈ U j , 0 . The power value and derivatives through order p p p vanish at zero by Lemma B.16 (b). Thus
A j ( 0 , 0 ) = O j ( 0 , 0 ) = P j ( 0 , 0 ) − P j ( h j , 0 ) − 4 δ j = − λ q ( m + 1 ) τ j q − 4 δ j , ∇ k A j ( 0 , 0 ) = ∇ k O j ( 0 , 0 ) = ∇ k P j ( 0 , 0 ) = 0 ( 1 ≤ k ≤ p ) . \begin{aligned}
A_j(0,0)&=O_j(0,0)=P_j(0,0)-P_j(h_j,0)-4\delta_j\\
&=-\frac{\lambda}{q}(m+1)\tau_j^q-4\delta_j,\\
\nabla^k A_j(0,0)&=\nabla^k O_j(0,0)=\nabla^k P_j(0,0)=0\qquad(1\le k\le p).
\end{aligned} A j ( 0 , 0 ) ∇ k A j ( 0 , 0 ) = O j ( 0 , 0 ) = P j ( 0 , 0 ) − P j ( h j , 0 ) − 4 δ j = − q λ ( m + 1 ) τ j q − 4 δ j , = ∇ k O j ( 0 , 0 ) = ∇ k P j ( 0 , 0 ) = 0 ( 1 ≤ k ≤ p ) .
The derivative equalities follow from neighborhood agreement, rather than only the equality of values at the origin.
Proof of (d). Lemma D.9 (b) gives ζ j ( h j , 0 ) = − 4 δ j \zeta_j(h_j,0)=-4\delta_j ζ j ( h j , 0 ) = − 4 δ j and ζ j ( h j , t j + 1 ⋆ ) ≤ − 3 δ j \zeta_j(h_j,t^\star_{j+1})\le-3\delta_j ζ j ( h j , t j + 1 ⋆ ) ≤ − 3 δ j . Since P j P_j P j is convex and I j I_j I j is affine, ζ j = P j − I j − P j ( h j , 0 ) − 4 δ j \zeta_j=P_j-I_j-P_j(h_j,0)-4\delta_j ζ j = P j − I j − P j ( h j , 0 ) − 4 δ j is convex. For 0 ≤ r ≤ 1 0\le r\le1 0 ≤ r ≤ 1 , its value at an intermediate point is at most the weighted average of these endpoint values, i.e.,
ζ j ( h j , r t j + 1 ⋆ ) ≤ ( 1 − r ) ζ j ( h j , 0 ) + r ζ j ( h j , t j + 1 ⋆ ) by convexity of ζ j , ≤ − 4 ( 1 − r ) δ j − 3 r δ j by Lemma D.9 (b), = ( − 4 + r ) δ j ≤ − 3 δ j < − δ j since 0 ≤ r ≤ 1. \begin{aligned}
\zeta_j(h_j,rt^\star_{j+1})&\le(1-r)\zeta_j(h_j,0)+r\zeta_j(h_j,t^\star_{j+1}) &&\text{by convexity of }\zeta_j\text{,}\\
&\le-4(1-r)\delta_j-3r\delta_j &&\text{by Lemma D.9 (b),}\\
&=(-4+r)\delta_j\le-3\delta_j<-\delta_j &&\text{since }0\le r\le1.
\end{aligned} ζ j ( h j , r t j + 1 ⋆ ) ≤ ( 1 − r ) ζ j ( h j , 0 ) + r ζ j ( h j , t j + 1 ⋆ ) ≤ − 4 ( 1 − r ) δ j − 3 r δ j = ( − 4 + r ) δ j ≤ − 3 δ j < − δ j by convexity of ζ j , by Lemma D.9 (b), since 0 ≤ r ≤ 1.
Thus the entire segment lies in the open set { ζ j < − δ j } \{\zeta_j<-\delta_j\} { ζ j < − δ j } . Lemma B.21 (c) gives A j = I j A_j=I_j A j = I j on this set, including equality of derivatives. Evaluating and differentiating Eq. (D.15) gives
A j ( h j , t j + 1 ) = ⟨ g j , t j + 1 v j + 1 ⟩ = − s j + 1 t j + 1 ( 0 ≤ t j + 1 ≤ t j + 1 ⋆ ) , A_j(h_j,t_{j+1})=\langle g_j,t_{j+1}v_{j+1}\rangle=-s_{j+1}t_{j+1}\qquad(0\le t_{j+1}\le t^\star_{j+1}), A j ( h j , t j + 1 ) = ⟨ g j , t j + 1 v j + 1 ⟩ = − s j + 1 t j + 1 ( 0 ≤ t j + 1 ≤ t j + 1 ⋆ ) ,
∇ A j = ∇ I j = g j , ∇ k A j = ∇ k I j = 0 ( 2 ≤ k ≤ p ) . \nabla A_j=\nabla I_j=g_j,\qquad\nabla^k A_j=\nabla^k I_j=0\qquad(2\le k\le p). ∇ A j = ∇ I j = g j , ∇ k A j = ∇ k I j = 0 ( 2 ≤ k ≤ p ) .
These identities give Eq. (D.31)–(D.33). Combining the fixed endpoint gradient with the zero starting gradient in part (c) gives
b j + ∇ A j ( h j , 0 ) − ∇ A j ( 0 , 0 ) = b j + g j = b j + s j v j − s j + 1 v j + 1 − b j = s j v j − s j + 1 v j + 1 . \begin{aligned}
b_j+\nabla A_j(h_j,0)-\nabla A_j(0,0)&=b_j+g_j\\
&=b_j+s_jv_j-s_{j+1}v_{j+1}-b_j\\
&=s_jv_j-s_{j+1}v_{j+1}.
\end{aligned} b j + ∇ A j ( h j , 0 ) − ∇ A j ( 0 , 0 ) = b j + g j = b j + s j v j − s j + 1 v j + 1 − b j = s j v j − s j + 1 v j + 1 .
None of these calculations uses Eq. (D.29) or the particular choice of α \alpha α .
Details for preservation of the oracle outputs
Lemma D.14 (Only one additional direction enters a phase’s oracle output). Under the assumptions of Lemma D.9 :
(a) For w ∈ U j , b w \in U_{j,b} w ∈ U j , b and 0 ≤ b ≤ ⌊ m 4 ⌋ 0 \le b \le\left\lfloor\frac{m}{4} \right\rfloor 0 ≤ b ≤ ⌊ 4 m ⌋ , the full tensors
∇ k A j ( w , 0 ) , 0 ≤ k ≤ p ( ∇ 0 A j = A j ) \nabla^{k} A_j(w,0), \qquad0 \le k \le p \qquad\left(\nabla^{0} A_j = A_j\right) ∇ k A j ( w , 0 ) , 0 ≤ k ≤ p ( ∇ 0 A j = A j ) given by Eq. (D.36)–(D.37), depend only on v j , u j , 1 , … , u j , b + 1 v_j,u_{j,1},\ldots,u_{j,b+1} v j , u j , 1 , … , u j , b + 1 and fixed scalars.
Proof. Lemma D.13 (b) gives A j = O j A_j = O_j A j = O j near the evaluation point, so every derivative through order p p p there agrees with that of O j O_j O j . Apply Lemma D.3 (d) with k = b < m k=b<m k = b < m . For 1 ≤ r ≤ p 1 \le r \le p 1 ≤ r ≤ p and arbitrary ambient vectors d 1 , … , d r d_1,\ldots,d_r d 1 , … , d r , it gives
∇ r A j ( w , 0 ) [ d 1 , … , d r ] = ∇ r P j ( w , 0 ) [ d 1 , … , d r ] = λ ∑ i = 0 b ϕ ( r ) ( ( D m w ) i ) ∏ ℓ = 1 r ⟨ a j , i , d ℓ ⟩ . \nabla^{r} A_j(w,0)[d_1,\ldots,d_r]
=
\nabla^{r} P_j(w,0)[d_1,\ldots,d_r]
=
\lambda\sum_{i=0}^{b}\phi^{(r)}\left((D_m w)_i\right)\prod_{\ell=1}^{r}\langle a_{j,i},d_\ell\rangle. ∇ r A j ( w , 0 ) [ d 1 , … , d r ] = ∇ r P j ( w , 0 ) [ d 1 , … , d r ] = λ i = 0 ∑ b ϕ ( r ) ( ( D m w ) i ) ℓ = 1 ∏ r ⟨ a j , i , d ℓ ⟩ .
The term α ϕ ( t j + 1 ) \alpha\phi(t_{j+1}) α ϕ ( t j + 1 ) contributes zero because ϕ ( r ) ( 0 ) = 0 \phi^{(r)}(0)=0 ϕ ( r ) ( 0 ) = 0 by Lemma B.16 (b) in Appx. B. The same statement at order zero, and the constant in O j O_j O j , give Eq. (D.37). Rows 0 , … , b 0,\ldots,b 0 , … , b contain only v j , u j , 1 , … , u j , b + 1 v_j,u_{j,1},\ldots,u_{j,b+1} v j , u j , 1 , … , u j , b + 1 , so these are full ambient tensor identities independent of all remaining directions. They recover Eq. (D.36)–(D.37) in Theorem D.11 (b) under the weaker assumptions of Lemma D.9. □
Hölder estimates
Use Eq. (D.13), Eq. (D.24), and Eq. (D.30), and assume Eq. (D.25), Eq. (D.26), and Eq. (D.29); in particular, 0 < α ≤ λ 0<\alpha\le\lambda 0 < α ≤ λ .
Lemma D.15 (Derivative bounds for the residual expression). Define
R j x : = ( D m w , ( α λ ) 1 q t j + 1 ) , P j ( x ) = λ q ∥ R j x ∥ q q , ∥ R j ∥ 2 → 2 ≤ 2. \mathcal{R}_j x := \left(D_m w,\left(\frac{\alpha}{\lambda}\right)^{\frac{1}{q}}t_{j+1}\right), \qquad
P_j(x)=\frac{\lambda}{q}\|\mathcal{R}_j x\|_q^q,\qquad
\|\mathcal{R}_j\|_{2\to2}\le2. R j x := ( D m w , ( λ α ) q 1 t j + 1 ) , P j ( x ) = q λ ∥ R j x ∥ q q , ∥ R j ∥ 2 → 2 ≤ 2. Omit the last coordinate in the terminal phase. Then:
(a) Global Hölder bound. For all x , y ∈ E j × R x,y\in E_j\times\mathbb{R} x , y ∈ E j × R ,
∥ ∇ p P j ( x ) − ∇ p P j ( y ) ∥ ≤ L P λ ∥ x − y ∥ ν , L P = L ϕ ( 2 p + ν + 1 ) . (140) \|\nabla^p P_j(x)-\nabla^p P_j(y)\|\le L_P\lambda\|x-y\|^\nu,
\qquad
L_P=L_\phi(2^{p+\nu}+1). \tag*{(140)} ∥ ∇ p P j ( x ) − ∇ p P j ( y ) ∥ ≤ L P λ ∥ x − y ∥ ν , L P = L ϕ ( 2 p + ν + 1 ) . (140) (b) Difference between the formulas. For all x , y ∈ E j × R x,y\in E_j\times\mathbb{R} x , y ∈ E j × R ,
∥ ∇ p ζ j ( x ) − ∇ p ζ j ( y ) ∥ ≤ L P λ ∥ x − y ∥ ν . \|\nabla^p\zeta_j(x)-\nabla^p\zeta_j(y)\|\le L_P\lambda\|x-y\|^\nu. ∥ ∇ p ζ j ( x ) − ∇ p ζ j ( y ) ∥ ≤ L P λ ∥ x − y ∥ ν . (c) Orders 1 through p p p .
∥ ∇ k P j ( x ) ∥ ≤ 2 k S q , k λ ∥ R j x ∥ q q − k ( x ∈ E j × R , 1 ≤ k ≤ p ) . \|\nabla^k P_j(x)\|\le2^k S_{q,k}\lambda\|\mathcal{R}_j x\|_q^{q-k}
\qquad(x\in E_j\times\mathbb{R},\ 1\le k\le p). ∥ ∇ k P j ( x ) ∥ ≤ 2 k S q , k λ ∥ R j x ∥ q q − k ( x ∈ E j × R , 1 ≤ k ≤ p ) .
Proof. By definition,
λ ∥ R j x ∥ q q = λ ∥ D m w ∥ q q + λ ( α λ ) q ∣ t j + 1 ∣ q = λ ∥ D m w ∥ q q + α ∣ t j + 1 ∣ q . \lambda\lVert R_jx\rVert_q^q=\lambda\lVert D_mw\rVert_q^q+\lambda\left(\frac{\alpha}{\lambda}\right)^q|t_{j+1}|^q=\lambda\lVert D_mw\rVert_q^q+\alpha|t_{j+1}|^q. λ ∥ R j x ∥ q q = λ ∥ D m w ∥ q q + λ ( λ α ) q ∣ t j + 1 ∣ q = λ ∥ D m w ∥ q q + α ∣ t j + 1 ∣ q .
By α ≤ λ \alpha\le\lambda α ≤ λ and Lemma D.3 (b),
∥ R j x ∥ 2 2 = ∥ D m w ∥ 2 2 + ( α λ ) 2 ∣ t j + 1 ∣ 2 ≤ 4 ∥ w ∥ 2 + ∣ t j + 1 ∣ 2 ≤ 4 ∥ x ∥ 2 . \lVert R_jx\rVert_2^2=\lVert D_mw\rVert_2^2+\left(\frac{\alpha}{\lambda}\right)^2|t_{j+1}|^2\le4\lVert w\rVert^2+|t_{j+1}|^2\le4\lVert x\rVert^2. ∥ R j x ∥ 2 2 = ∥ D m w ∥ 2 2 + ( λ α ) 2 ∣ t j + 1 ∣ 2 ≤ 4 ∥ w ∥ 2 + ∣ t j + 1 ∣ 2 ≤ 4 ∥ x ∥ 2 .
Hence ∥ R j ∥ 2 → 2 ≤ 2 \lVert R_j\rVert_{2\to2}\le2 ∥ R j ∥ 2 → 2 ≤ 2 .
Put Φ ( a ) = ∑ i ϕ ( a i ) \Phi(a)=\sum_i\phi(a_i) Φ ( a ) = ∑ i ϕ ( a i ) , so P j = λ Φ ∘ R j P_j=\lambda\Phi\circ R_j P j = λ Φ ∘ R j . For 2 ≤ k ≤ p 2\le k\le p 2 ≤ k ≤ p , the k t h k^{\mathrm{th}} k th derivative is diagonal:
∇ k Φ ( a ) [ z 1 , … , z k ] = ∑ i ϕ ( k ) ( a i ) ∏ ℓ = 1 k z ℓ , i . \nabla^k\Phi(a)[z_1,\ldots,z_k]=\sum_i\phi^{(k)}(a_i)\prod_{\ell=1}^k z_{\ell,i}. ∇ k Φ ( a ) [ z 1 , … , z k ] = i ∑ ϕ ( k ) ( a i ) ℓ = 1 ∏ k z ℓ , i .
Lemma B.10 in Appx. B, applied to the one-dimensional orthogonal coordinate spaces for every k ≥ 2 k\ge2 k ≥ 2 , gives
∥ ∇ k Φ ( a ) ∥ = max i ∣ ϕ ( k ) ( a i ) ∣ , \lVert\nabla^k\Phi(a)\rVert=\max_i|\phi^{(k)}(a_i)|, ∥ ∇ k Φ ( a )∥ = i max ∣ ϕ ( k ) ( a i ) ∣ ,
∥ ∇ p Φ ( a ) − ∇ p Φ ( b ) ∥ = max i ∣ ϕ ( p ) ( a i ) − ϕ ( p ) ( b i ) ∣ . \lVert\nabla^p\Phi(a)-\nabla^p\Phi(b)\rVert=\max_i|\phi^{(p)}(a_i)-\phi^{(p)}(b_i)|. ∥ ∇ p Φ ( a ) − ∇ p Φ ( b )∥ = i max ∣ ϕ ( p ) ( a i ) − ϕ ( p ) ( b i ) ∣.
There is no factor m m m . For 1 ≤ k ≤ p 1\le k\le p 1 ≤ k ≤ p , the linear chain rule gives
∇ k P j ( x ) [ z 1 , … , z k ] = λ ∇ k Φ ( R j x ) [ R j z 1 , … , R j z k ] . \nabla^kP_j(x)[z_1,\ldots,z_k]=\lambda\nabla^k\Phi(R_jx)[R_jz_1,\ldots,R_jz_k]. ∇ k P j ( x ) [ z 1 , … , z k ] = λ ∇ k Φ ( R j x ) [ R j z 1 , … , R j z k ] .
∥ ∇ p P j ( x ) − ∇ p P j ( x ′ ) ∥ ≤ λ ∥ R j ∥ 2 → 2 p max i ∣ ϕ ( p ) ( ( R j x ) i ) − ϕ ( p ) ( ( R j x ′ ) i ) ∣ ≤ 2 p L ϕ λ ∥ R j ( x − x ′ ) ∥ ∞ ν ≤ 2 p L ϕ λ ( 2 ∥ x − x ′ ∥ ) ν = 2 p + ν L ϕ λ ∥ x − x ′ ∥ ν . \begin{aligned}
\lVert\nabla^pP_j(x)-\nabla^pP_j(x')\rVert
&\le\lambda\lVert R_j\rVert_{2\to2}^p\max_i\left|\phi^{(p)}((R_jx)_i)-\phi^{(p)}((R_jx')_i)\right|\\
&\le2^pL_\phi\lambda\lVert R_j(x-x')\rVert_\infty^\nu\\
&\le2^pL_\phi\lambda(2\lVert x-x'\rVert)^\nu\\
&=2^{p+\nu}L_\phi\lambda\lVert x-x'\rVert^\nu.
\end{aligned} ∥ ∇ p P j ( x ) − ∇ p P j ( x ′ )∥ ≤ λ ∥ R j ∥ 2 → 2 p i max ϕ ( p ) (( R j x ) i ) − ϕ ( p ) (( R j x ′ ) i ) ≤ 2 p L ϕ λ ∥ R j ( x − x ′ ) ∥ ∞ ν ≤ 2 p L ϕ λ ( 2 ∥ x − x ′ ∥ ) ν = 2 p + ν L ϕ λ ∥ x − x ′ ∥ ν .
Here ∥ R j z ∥ ∞ ≤ ∥ R j z ∥ 2 ≤ 2 ∥ z ∥ \lVert R_jz\rVert_\infty\le\lVert R_jz\rVert_2\le2\lVert z\rVert ∥ R j z ∥ ∞ ≤ ∥ R j z ∥ 2 ≤ 2 ∥ z ∥ and 2 p + ν L ϕ ≤ L P 2^{p+\nu}L_\phi\le L_P 2 p + ν L ϕ ≤ L P . The scalar bound includes ν = 1 \nu=1 ν = 1 .
Proof of (c). For 2 ≤ k ≤ p 2\le k\le p 2 ≤ k ≤ p , use the chain rule, diagonal norm identity, and Lemma B.16 (a):
∥ ∇ k P j ( x ) ∥ ≤ λ ∥ R j ∥ 2 → 2 k max i ∣ ϕ ( k ) ( ( R j x ) i ) ∣ ≤ 2 k S q , k λ ∥ R j x ∥ ∞ q − k ≤ 2 k S q , k λ ∥ R j x ∥ q q − k . \lVert\nabla^kP_j(x)\rVert\le\lambda\lVert R_j\rVert_{2\to2}^k\max_i\left|\phi^{(k)}((R_jx)_i)\right|
\le2^kS_{q,k}\lambda\lVert R_jx\rVert_\infty^{q-k}\le2^kS_{q,k}\lambda\lVert R_jx\rVert_q^{q-k}. ∥ ∇ k P j ( x )∥ ≤ λ ∥ R j ∥ 2 → 2 k i max ϕ ( k ) (( R j x ) i ) ≤ 2 k S q , k λ ∥ R j x ∥ ∞ q − k ≤ 2 k S q , k λ ∥ R j x ∥ q q − k .
Here q − k > 0 q-k>0 q − k > 0 , with zero arguments covered by Lemma B.16 (b). For k = 1 k=1 k = 1 , set a = R j x a=R_jx a = R j x and use ∇ P j = λ R j ⊤ ( ∣ a ∣ q − 2 a ) \nabla P_j=\lambda R_j^\top(|a|^{q-2}a) ∇ P j = λ R j ⊤ ( ∣ a ∣ q − 2 a ) (componentwise powers):
∥ ∇ P j ( x ) ∥ ≤ 2 λ ( ∑ i ∣ a i ∣ 2 ( q − 1 ) ) 1 2 = 2 λ ∥ a ∥ 2 ( q − 1 ) q − 1 ≤ 2 λ ∥ a ∥ q q − 1 = 2 S q , 1 λ ∥ R j x ∥ q q − 1 . \begin{aligned}
\lVert\nabla P_j(x)\rVert
&\le2\lambda\left(\sum_i|a_i|^{2(q-1)}\right)^{\frac12}
=2\lambda\lVert a\rVert_{2(q-1)}^{q-1}\\
&\le2\lambda\lVert a\rVert_q^{q-1}
=2S_{q,1}\lambda\lVert R_jx\rVert_q^{q-1}.
\end{aligned} ∥ ∇ P j ( x )∥ ≤ 2 λ ( i ∑ ∣ a i ∣ 2 ( q − 1 ) ) 2 1 = 2 λ ∥ a ∥ 2 ( q − 1 ) q − 1 ≤ 2 λ ∥ a ∥ q q − 1 = 2 S q , 1 λ ∥ R j x ∥ q q − 1 .
Here 2 ( q − 1 ) ≥ q 2(q-1)\ge q 2 ( q − 1 ) ≥ q and S q , 1 = 1 S_{q,1}=1 S q , 1 = 1 .
Set χ δ ( a ) : = ψ δ ( a ) − a \chi_\delta(a) := \psi_\delta(a) - a χ δ ( a ) := ψ δ ( a ) − a for δ > 0 \delta> 0 δ > 0 and K j : = χ δ j ∘ ζ j = A j − O j K_j := \chi_{\delta_j} \circ\zeta_j = A_j - O_j K j := χ δ j ∘ ζ j = A j − O j . Define
ρ j A = ( δ j λ ) 1 q , T j = { x : ζ j ( x ) ≤ δ j } . (141) \rho_j^A = \left(\frac{\delta_j}{\lambda}\right)^{\frac{1}{q}}, \qquad\mathcal{T}_j = \{x : \zeta_j(x) \le\delta_j\}. \tag*{(141)} ρ j A = ( λ δ j ) q 1 , T j = { x : ζ j ( x ) ≤ δ j } . (141)
Let T j , ρ j A : = { x : dist ( x , T j ) ≤ ρ j A } \mathcal{T}_{j,\rho_j^A} := \{x : \operatorname{dist}(x,\mathcal{T}_j) \le\rho_j^A\} T j , ρ j A := { x : dist ( x , T j ) ≤ ρ j A } be its closed Euclidean ρ j A \rho_j^A ρ j A -neighborhood; ρ j A \rho_j^A ρ j A differs from the base support radius ρ j = θ R j \rho_j = \theta R_j ρ j = θ R j in Eq. (C.10).
Lemma D.16 (Nearest point in a nonempty closed set [9 Sec. 8.1, p. 397] ). If S ⊂ R d S \subset\mathbb{R}^d S ⊂ R d is nonempty and closed, then for every x ∈ R d x \in\mathbb{R}^d x ∈ R d there exists a ∈ S a \in S a ∈ S such that
∥ x − a ∥ = dist ( x , S ) . \lVert x-a\rVert= \operatorname{dist}(x,S). ∥ x − a ∥ = dist ( x , S ) . Such a point is called a projection; there can be more than one. If S S S is also convex, the Euclidean projection is unique.
Lemma D.17 (Residual bounds on the enlarged region). Under the assumptions stated before Lemma D.15, set
C d u a l : = 2 1 q − 1 ( 2 + c 0 ) q ∗ c 1 , C T : = 2 q ( 1 + C d u a l ) , C N : = 2 q − 1 ( C T + 2 q + 1 ) . C_{\mathrm{dual}} := \frac{2^{\frac{1}{q-1}}(2+c_0)}{q^*c_1}, \qquad C_T := 2q(1+C_{\mathrm{dual}}), \qquad C_N := 2^{q-1}(C_T+2^q+1). C dual := q ∗ c 1 2 q − 1 1 ( 2 + c 0 ) , C T := 2 q ( 1 + C dual ) , C N := 2 q − 1 ( C T + 2 q + 1 ) . These constants depend only on q q q .
(a) Sublevel set.
ζ j ( x ) ≥ − C d u a l δ j ( x ∈ E j × R ) , \zeta_j(x) \ge-C_{\mathrm{dual}}\delta_j \qquad(x \in E_j \times\mathbb{R}), ζ j ( x ) ≥ − C dual δ j ( x ∈ E j × R ) , λ ∥ D m w ∥ q q + α ∣ t j + 1 ∣ q ≤ C T δ j ( ( w , t j + 1 ) ∈ T j ) . \lambda\lVert D_mw\rVert_q^q+\alpha\lvert t_{j+1}\rvert^q \le C_T\delta_j \qquad((w,t_{j+1}) \in\mathcal{T}_j). λ ∥ D m w ∥ q q + α ∣ t j + 1 ∣ q ≤ C T δ j (( w , t j + 1 ) ∈ T j ) . (b) Enlarged region.
λ ∥ D m w ∥ q q + α ∣ t j + 1 ∣ q ≤ C N δ j ( ( w , t j + 1 ) ∈ T j , ρ j A ) . (142) \lambda\lVert D_mw\rVert_q^q+\alpha\lvert t_{j+1}\rvert^q \le C_N\delta_j \qquad((w,t_{j+1}) \in\mathcal{T}_{j,\rho_j^A}). \tag*{(142)} λ ∥ D m w ∥ q q + α ∣ t j + 1 ∣ q ≤ C N δ j (( w , t j + 1 ) ∈ T j , ρ j A ) . (142)
Proof. • Proof of (a). To bound the residuals where ζ j \zeta_j ζ j is small, we first bound its linear terms by P j 2 \frac{P_j}{2} 2 P j and a constant. The remaining P j 2 \frac{P_j}{2} 2 P j then bounds the residual energy. In Eq. (D.27), substitute
( a , b , ℓ ) = ( ( D m w ) i , π j , i , λ 2 ) ( 0 ≤ i ≤ m ) . (a,b,\ell)=\left((D_mw)_i,\pi_{j,i},\frac{\lambda}{2}\right) \qquad(0 \le i \le m). ( a , b , ℓ ) = ( ( D m w ) i , π j , i , 2 λ ) ( 0 ≤ i ≤ m ) .
Each chain row gives
π j , i ( D m w ) i ≤ λ 2 q ∣ ( D m w ) i ∣ q + 2 1 q − 1 q ∗ λ − 1 q − 1 π j , i ∗ q , \pi_{j,i}(D_mw)_i \le\frac{\lambda}{2q}\lvert(D_mw)_i\rvert^q+\frac{2^{\frac{1}{q-1}}}{q^*}\lambda^{-\frac{1}{q-1}}\pi_{j,i}^{*q}, π j , i ( D m w ) i ≤ 2 q λ ∣( D m w ) i ∣ q + q ∗ 2 q − 1 1 λ − q − 1 1 π j , i ∗ q ,
where ( λ 2 ) − 1 q − 1 = 2 1 q − 1 λ − 1 q − 1 \left(\frac{\lambda}{2}\right)^{-\frac{1}{q-1}}=2^{\frac{1}{q-1}}\lambda^{-\frac{1}{q-1}} ( 2 λ ) − q − 1 1 = 2 q − 1 1 λ − q − 1 1 . For the next coordinate, apply Corollary D.10 with α 2 \frac{\alpha}{2} 2 α in place of α \alpha α . Its infimum identity gives, for every t j + 1 ∈ R t_{j+1}\in\mathbb{R} t j + 1 ∈ R ,
− s j + 1 t j + 1 ≤ α 2 q ∣ t j + 1 ∣ q + 2 1 q − 1 q ∗ α − 1 q − 1 s j + 1 ∗ q . -s_{j+1}t_{j+1}\le\frac{\alpha}{2q}\lvert t_{j+1}\rvert^q+\frac{2^{\frac{1}{q-1}}}{q^*}\alpha^{-\frac{1}{q-1}}s_{j+1}^{*q}. − s j + 1 t j + 1 ≤ 2 q α ∣ t j + 1 ∣ q + q ∗ 2 q − 1 1 α − q − 1 1 s j + 1 ∗ q .
Summing these inequalities gives
⟨ π j , D m w ⟩ − s j + 1 t j + 1 ≤ 1 2 P j ( w , t j + 1 ) + 2 1 q − 1 q ∗ ( λ − 1 q − 1 ∑ i = 0 m π j , i q ∗ + α − 1 q − 1 s j + 1 q ∗ ) . \langle\pi_j,D_mw\rangle-s_{j+1}t_{j+1}\le\frac{1}{2}P_j(w,t_{j+1})
+\frac{2^{\frac{1}{q-1}}}{q^*}\left(\lambda^{-\frac{1}{q-1}}\sum_{i=0}^{m}\pi_{j,i}^{q^*}+\alpha^{-\frac{1}{q-1}}s_{j+1}^{q^*}\right). ⟨ π j , D m w ⟩ − s j + 1 t j + 1 ≤ 2 1 P j ( w , t j + 1 ) + q ∗ 2 q − 1 1 ( λ − q − 1 1 i = 0 ∑ m π j , i q ∗ + α − q − 1 1 s j + 1 q ∗ ) .
By Eq. (D.21) and ( q − 1 ) q ∗ = q (q-1)q^*=q ( q − 1 ) q ∗ = q ,
λ − 1 q − 1 ∑ i = 0 m π j , i q ∗ ≤ ( m + 1 ) λ − 1 q − 1 ( λ τ j q − 1 ) q ∗ = ( m + 1 ) λ q ∗ − 1 q − 1 τ j ( q − 1 ) q ∗ = ( m + 1 ) λ τ j q ≤ 2 E j . \begin{aligned}
\lambda^{-\frac{1}{q-1}}\sum_{i=0}^{m}\pi_{j,i}^{q^*}\le(m+1)\lambda^{-\frac{1}{q-1}}\left(\lambda\tau_j^{q-1}\right)^{q^*} \\
&=(m+1)\lambda^{q^*-\frac{1}{q-1}}\tau_j^{(q-1)q^*}=(m+1)\lambda\tau_j^q\le2\mathcal{E}_j.
\end{aligned} λ − q − 1 1 i = 0 ∑ m π j , i q ∗ ≤ ( m + 1 ) λ − q − 1 1 ( λ τ j q − 1 ) q ∗ = ( m + 1 ) λ q ∗ − q − 1 1 τ j ( q − 1 ) q ∗ = ( m + 1 ) λ τ j q ≤ 2 E j .
Use Eq. (D.29) for the other term and E j = δ j c 1 \mathcal{E}_j=\frac{\delta_j}{c_1} E j = c 1 δ j : the total remainder is at most
2 1 q − 1 q ∗ ( 2 + c 0 ) E j = 2 1 q − 1 ( 2 + c 0 ) q ∗ c 1 δ j = C d u a l δ j . \frac{2^{\frac{1}{q-1}}}{q^*}(2+c_0)\mathcal{E}_j
=\frac{2^{\frac{1}{q-1}}(2+c_0)}{q^*c_1}\delta_j=C_{\mathrm{dual}}\delta_j. q ∗ 2 q − 1 1 ( 2 + c 0 ) E j = q ∗ c 1 2 q − 1 1 ( 2 + c 0 ) δ j = C dual δ j .
Substitute in Eq. (D.20), using k j , a f f > 0 k_{j,\mathrm{aff}}>0 k j , aff > 0 from Eq. (D.21):
ζ j = P j − ( ⟨ π j , D m w ⟩ − s j + 1 t j + 1 ) + k j , a f f ≥ 1 2 P j − C d u a l δ j + k j , a f f ≥ 1 2 P j − C d u a l δ j ≥ − C d u a l δ j . \begin{aligned}
\zeta_j=P_j-\left(\langle\pi_j,D_mw\rangle-s_{j+1}t_{j+1}\right)+k_{j,\mathrm{aff}} \\
&\ge\frac{1}{2}P_j-C_{\mathrm{dual}}\delta_j+k_{j,\mathrm{aff}}
\ge\frac{1}{2}P_j-C_{\mathrm{dual}}\delta_j\ge-C_{\mathrm{dual}}\delta_j.
\end{aligned} ζ j = P j − ( ⟨ π j , D m w ⟩ − s j + 1 t j + 1 ) + k j , aff ≥ 2 1 P j − C dual δ j + k j , aff ≥ 2 1 P j − C dual δ j ≥ − C dual δ j .
For x ∈ T j x\in T_j x ∈ T j , ζ j ( x ) ≤ δ j \zeta_j(x)\le\delta_j ζ j ( x ) ≤ δ j therefore gives
1 2 P j ( x ) ≤ ( 1 + C d u a l ) δ j , λ ∥ D m w ∥ q q + α ∣ t j + 1 ∣ q = q P j ( x ) ≤ C T δ j . \frac{1}{2}P_j(x)\le(1+C_{\mathrm{dual}})\delta_j,\qquad
\lambda\lVert D_mw\rVert_q^q+\alpha|t_{j+1}|^q=qP_j(x)\le C_T\delta_j. 2 1 P j ( x ) ≤ ( 1 + C dual ) δ j , λ ∥ D m w ∥ q q + α ∣ t j + 1 ∣ q = q P j ( x ) ≤ C T δ j .
Omit the t j + 1 t_{j+1} t j + 1 -terms in the terminal phase.
Proof of (b). Continuity of ζ j \zeta_j ζ j and ζ j ( x j ∗ ) = − 4 δ j \zeta_j(x_j^*)=-4\delta_j ζ j ( x j ∗ ) = − 4 δ j make T j T_j T j closed and nonempty. For x ∈ T j , ρ j A x\in T_{j,\rho_j^A} x ∈ T j , ρ j A , Lemma D.16 supplies
x = a + d , ∥ d ∥ ≤ ρ j A , a = ( w a , a j + 1 ) , d = ( d E , d j + 1 ) . x=a+d,\qquad\lVert d\rVert\le\rho_j^A,\qquad a=(w_a,a_{j+1}),\quad d=(d_E,d_{j+1}). x = a + d , ∥ d ∥ ≤ ρ j A , a = ( w a , a j + 1 ) , d = ( d E , d j + 1 ) .
Here a ∈ T j a\in T_j a ∈ T j , w a = P E j a w_a=P_{E_j}a w a = P E j a , and d E = P E j d d_E=P_{E_j}d d E = P E j d . For j < L − 1 j<L-1 j < L − 1 , we define a j + 1 = ⟨ v j + 1 , a ⟩ a_{j+1}=\langle v_{j+1},a\rangle a j + 1 = ⟨ v j + 1 , a ⟩ and d j + 1 = ⟨ v j + 1 , d ⟩ d_{j+1}=\langle v_{j+1},d\rangle d j + 1 = ⟨ v j + 1 , d ⟩ . Thus w = w a + d E w=w_a+d_E w = w a + d E and t j + 1 = a j + 1 + d j + 1 t_{j+1}=a_{j+1}+d_{j+1} t j + 1 = a j + 1 + d j + 1 . The terminal phase omits the scalar components. Apply ∣ a + b ∣ q ≤ 2 q − 1 ( ∣ a ∣ q + ∣ b ∣ q ) |a+b|^q\le2^{q-1}(|a|^q+|b|^q) ∣ a + b ∣ q ≤ 2 q − 1 ( ∣ a ∣ q + ∣ b ∣ q ) coordinatewise:
λ ∥ D m w ∥ q q + α ∣ t j + 1 ∣ q ≤ 2 q − 1 [ λ ∥ D m w a ∥ q q + α ∣ a j + 1 ∣ q + λ ∥ D m d E ∥ q q + α ∣ d j + 1 ∣ q ] \lambda\lVert D_mw\rVert_q^q+\alpha|t_{j+1}|^q
\le2^{q-1}\left[\lambda\lVert D_mw_a\rVert_q^q+\alpha|a_{j+1}|^q+\lambda\lVert D_md_E\rVert_q^q+\alpha|d_{j+1}|^q\right] λ ∥ D m w ∥ q q + α ∣ t j + 1 ∣ q ≤ 2 q − 1 [ λ ∥ D m w a ∥ q q + α ∣ a j + 1 ∣ q + λ ∥ D m d E ∥ q q + α ∣ d j + 1 ∣ q ]
≤ 2 q − 1 [ C T δ j + 2 q λ ∥ d E ∥ q + λ ∣ d j + 1 ∣ q ] \le2^{q-1}\left[C_T\delta_j+2^q\lambda\lVert d_E\rVert^q+\lambda|d_{j+1}|^q\right] ≤ 2 q − 1 [ C T δ j + 2 q λ ∥ d E ∥ q + λ ∣ d j + 1 ∣ q ]
≤ 2 q − 1 [ C T δ j + ( 2 q + 1 ) λ ( ρ j A ) q ] = 2 q − 1 ( C T + 2 q + 1 ) δ j = C N δ j . \begin{aligned}
\le2^{q-1}\left[C_T\delta_j+(2^q+1)\lambda(\rho_j^A)^q\right] \\
&=2^{q-1}(C_T+2^q+1)\delta_j=C_N\delta_j.
\end{aligned} ≤ 2 q − 1 [ C T δ j + ( 2 q + 1 ) λ ( ρ j A ) q ] = 2 q − 1 ( C T + 2 q + 1 ) δ j = C N δ j .
The bounds used are part (a) at a a a , ∥ D m d E ∥ q ≤ 2 ∥ d E ∥ \lVert D_md_E\rVert_q\le2\lVert d_E\rVert ∥ D m d E ∥ q ≤ 2 ∥ d E ∥ (Lemma D.3 (b)), α ≤ λ \alpha\le\lambda α ≤ λ , ∥ d E ∥ , ∣ d j + 1 ∣ ≤ ρ j A \lVert d_E\rVert,|d_{j+1}|\le\rho_j^A ∥ d E ∥ , ∣ d j + 1 ∣ ≤ ρ j A , and λ ( ρ j A ) q = δ j \lambda(\rho_j^A)^q=\delta_j λ ( ρ j A ) q = δ j .
Lemma D.18 (Bounds for each derivative order). Assume Lemma D.17 ’s hypotheses. The constants M k M_k M k in Eq. (D.45) and Eq. (D.46) depend only on p , ν p,\nu p , ν ; k k k is the derivative order and j j j the phase index.
(a) Orders 2 through p p p .
∥ ∇ k ζ j ( x ) ∥ o p ≤ M k λ ( ρ j , ρ j A A ) q − k ( x ∈ T j , ρ j A A , 2 ≤ k ≤ p ) . (143) \lVert\nabla^{k}\zeta_j(x)\rVert_{\mathrm{op}} \le M_k\lambda\left(\rho_{j,\rho_j^A}^{A}\right)^{q-k}\qquad\left(x\in\mathcal{T}_{j,\rho_j^A}^{A},\ 2\le k\le p\right). \tag*{(143)} ∥ ∇ k ζ j ( x ) ∥ op ≤ M k λ ( ρ j , ρ j A A ) q − k ( x ∈ T j , ρ j A A , 2 ≤ k ≤ p ) . (143) (b) Gradient.
∥ ∇ ζ j ( x ) ∥ ≤ M 1 λ ( ρ j A ) q − 1 ( x ∈ T j , ρ j A A ) . \lVert\nabla\zeta_j(x)\rVert\le M_1\lambda\left(\rho_j^A\right)^{q-1}\qquad\left(x\in\mathcal{T}_{j,\rho_j^A}^{A}\right). ∥ ∇ ζ j ( x )∥ ≤ M 1 λ ( ρ j A ) q − 1 ( x ∈ T j , ρ j A A ) . Proof. For x ∈ T j , ρ j A A x\in\mathcal{T}_{j,\rho_j^A}^{A} x ∈ T j , ρ j A A , Lemma D.17 (b) gives
λ ∥ R j x ∥ q q = λ ∥ D m w ∥ q q + α ∣ t j + 1 ∣ q ≤ C N δ j = C N λ ( ρ j A ) q . \lambda\lVert R_jx\rVert_q^q=\lambda\lVert D_mw\rVert_q^q+\alpha\lvert t_{j+1}\rvert^q\le C_N\delta_j=C_N\lambda\left(\rho_j^A\right)^q. λ ∥ R j x ∥ q q = λ ∥ D m w ∥ q q + α ∣ t j + 1 ∣ q ≤ C N δ j = C N λ ( ρ j A ) q . Taking the q t h q^{\mathrm{th}} q th root after division by λ > 0 \lambda>0 λ > 0 , we obtain
∥ R j x ∥ q ≤ V ρ j A , V : = C N 1 q . \lVert R_jx\rVert_q\le V\rho_j^A,\qquad V:=C_N^{\frac{1}{q}}. ∥ R j x ∥ q ≤ V ρ j A , V := C N q 1 . Lemma D.15 (c) then gives
∥ ∇ k P j ( x ) ∥ ≤ 2 k S q , k λ ( V ρ j A ) q − k = 2 k S q , k V q − k λ ( ρ j A ) q − k ( 1 ≤ k ≤ p ) . \lVert\nabla^kP_j(x)\rVert\le2^kS_{q,k}\lambda\left(V\rho_j^A\right)^{q-k}=2^kS_{q,k}V^{q-k}\lambda\left(\rho_j^A\right)^{q-k}\qquad(1\le k\le p). ∥ ∇ k P j ( x )∥ ≤ 2 k S q , k λ ( V ρ j A ) q − k = 2 k S q , k V q − k λ ( ρ j A ) q − k ( 1 ≤ k ≤ p ) . M k : = 2 k S q , k V q − k ( 2 ≤ k ≤ p ) . (144) M_k:=2^kS_{q,k}V^{q-k}\qquad(2\le k\le p). \tag*{(144)} M k := 2 k S q , k V q − k ( 2 ≤ k ≤ p ) . (144) g j = Γ j v j − G j R j ∑ i = 1 m x j , i ∗ u j , i − s j + 1 v j + 1 . g_j=\Gamma_jv_j-\frac{G_j}{R_j}\sum_{i=1}^{m}x_{j,i}^*u_{j,i}-s_{j+1}v_{j+1}. g j = Γ j v j − R j G j i = 1 ∑ m x j , i ∗ u j , i − s j + 1 v j + 1 . The link projection has norm at most R j R_j R j . Hence, using G j ≤ c 0 Γ j G_j\le c_0\Gamma_j G j ≤ c 0 Γ j ,
∥ g j ∥ ≤ Γ j ∥ v j ∥ + G j R j ∥ ∑ i = 1 m x j , i ∗ u j , i ∥ + s j + 1 \lVert g_j\rVert\le\Gamma_j\lVert v_j\rVert+\frac{G_j}{R_j}\left\lVert\sum_{i=1}^{m}x_{j,i}^*u_{j,i}\right\rVert+s_{j+1} ∥ g j ∥ ≤ Γ j ∥ v j ∥ + R j G j i = 1 ∑ m x j , i ∗ u j , i + s j + 1 ≤ Γ j + G j R j R j + s j + 1 \le\Gamma_j+\frac{G_j}{R_j}R_j+s_{j+1} ≤ Γ j + R j G j R j + s j + 1 = Γ j + G j + s j + 1 =\Gamma_j+G_j+s_{j+1} = Γ j + G j + s j + 1 ≤ ( 1 + c 0 ) Γ j + s j + 1 . \le(1+c_0)\Gamma_j+s_{j+1}. ≤ ( 1 + c 0 ) Γ j + s j + 1 . Since ρ j A = ( δ j λ ) 1 q = ( c 1 m ) 1 q τ j \rho_j^A=\left(\frac{\delta_j}{\lambda}\right)^{\frac{1}{q}}=(c_1m)^{\frac{1}{q}}\tau_j ρ j A = ( λ δ j ) q 1 = ( c 1 m ) q 1 τ j and m ≥ 1 m\ge1 m ≥ 1 ,
Γ j λ ( ρ j A ) q − 1 = λ τ j q − 1 m − 1 2 λ ( c 1 m ) q − 1 q τ j q − 1 = c 1 − q − 1 q m − 1 2 − q − 1 q ≤ c 1 − q − 1 q . \frac{\Gamma_j}{\lambda\left(\rho_j^A\right)^{q-1}}
=\frac{\lambda\tau_j^{q-1}m^{-\frac{1}{2}}}{\lambda(c_1m)^{\frac{q-1}{q}}\tau_j^{q-1}}
=c_1^{-\frac{q-1}{q}}m^{-\frac{1}{2}-\frac{q-1}{q}}
\le c_1^{-\frac{q-1}{q}}. λ ( ρ j A ) q − 1 Γ j = λ ( c 1 m ) q q − 1 τ j q − 1 λ τ j q − 1 m − 2 1 = c 1 − q q − 1 m − 2 1 − q q − 1 ≤ c 1 − q q − 1 .
From Eq. (D.29) and c 0 = c 1 c_0=c_1 c 0 = c 1 , α 1 q − 1 s j + 1 ∗ ≤ δ j \alpha^{\frac{1}{q-1}}s_{j+1}^{*}\le\delta_j α q − 1 1 s j + 1 ∗ ≤ δ j . Its ( q ∗ ) t h (q^*)^{\mathrm{th}} ( q ∗ ) th root gives
s j + 1 ≤ α 1 q δ j 1 q ∗ = α 1 q λ q − 1 q ( ρ j A ) q − 1 ≤ λ ( ρ j A ) q − 1 . s_{j+1}\le\alpha^{\frac{1}{q}}\delta_j^{\frac{1}{q^*}}=\alpha^{\frac{1}{q}}\lambda^{\frac{q-1}{q}}\left(\rho_j^A\right)^{q-1}
\le\lambda\left(\rho_j^A\right)^{q-1}. s j + 1 ≤ α q 1 δ j q ∗ 1 = α q 1 λ q q − 1 ( ρ j A ) q − 1 ≤ λ ( ρ j A ) q − 1 .
using ( q − 1 ) q ∗ = q (q-1)q^*=q ( q − 1 ) q ∗ = q and α ≤ λ \alpha\le\lambda α ≤ λ . Add the k = 1 k=1 k = 1 bound for P j P_j P j :
∥ ∇ ζ j ∥ ≤ ∥ ∇ P j ∥ + ∥ g j ∥ ≤ 2 V q − 1 λ ( ρ j A ) q − 1 + ( 1 + c 0 ) Γ j + s j + 1 ≤ 2 V q − 1 λ ( ρ j A ) q − 1 + ( 1 + c 0 ) c 1 q − 1 q λ ( ρ j A ) q − 1 + λ ( ρ j A ) q − 1 ≤ [ 2 V q − 1 + ( 1 + c 0 ) c 1 q − 1 q + 1 ] λ ( ρ j A ) q − 1 . \begin{aligned}
\lVert\nabla\zeta_j\rVert&\le\lVert\nabla P_j\rVert+\lVert g_j\rVert\\
&\le2V^{q-1}\lambda\left(\rho_j^A\right)^{q-1}+(1+c_0)\Gamma_j+s_{j+1}\\
&\le2V^{q-1}\lambda\left(\rho_j^A\right)^{q-1}+(1+c_0)c_1^{\frac{q-1}{q}}\lambda\left(\rho_j^A\right)^{q-1}+\lambda\left(\rho_j^A\right)^{q-1}\\
&\le\left[2V^{q-1}+(1+c_0)c_1^{\frac{q-1}{q}}+1\right]\lambda\left(\rho_j^A\right)^{q-1}.
\end{aligned} ∥ ∇ ζ j ∥ ≤ ∥ ∇ P j ∥ + ∥ g j ∥ ≤ 2 V q − 1 λ ( ρ j A ) q − 1 + ( 1 + c 0 ) Γ j + s j + 1 ≤ 2 V q − 1 λ ( ρ j A ) q − 1 + ( 1 + c 0 ) c 1 q q − 1 λ ( ρ j A ) q − 1 + λ ( ρ j A ) q − 1 ≤ [ 2 V q − 1 + ( 1 + c 0 ) c 1 q q − 1 + 1 ] λ ( ρ j A ) q − 1 .
Take
M 1 : = 2 V q − 1 + ( 1 + c 0 ) c 1 q − 1 q + 1. (145) M_1:=2V^{q-1}+(1+c_0)c_1^{\frac{q-1}{q}}+1. \tag*{(145)} M 1 := 2 V q − 1 + ( 1 + c 0 ) c 1 q q − 1 + 1. (145)
All constants depend only on p , ν p,\nu p , ν ; the terminal case has s j + 1 = 0 s_{j+1}=0 s j + 1 = 0 .
Lemma D.19 (The smoothing correction has a uniform Hölder bound). Under the assumptions of Lemma D.17:
(a) Correction. K j = χ δ j ∘ ζ j ∈ C p , ν K_j=\chi_{\delta_j}\circ\zeta_j\in C^{p,\nu} K j = χ δ j ∘ ζ j ∈ C p , ν and, for all x , y ∈ R d x,y\in\mathbb{R}^d x , y ∈ R d ,
∥ ∇ p K j ( x ) − ∇ p K j ( y ) ∥ ≤ C K λ ∥ x − y ∥ ν . \lVert\nabla^pK_j(x)-\nabla^pK_j(y)\rVert\le C_K\lambda\lVert x-y\rVert^\nu. ∥ ∇ p K j ( x ) − ∇ p K j ( y )∥ ≤ C K λ ∥ x − y ∥ ν . (b) Phase function. A j ∈ C p , ν A_j\in C^{p,\nu} A j ∈ C p , ν and, for all x , y ∈ R d x,y\in\mathbb{R}^d x , y ∈ R d ,
∥ ∇ p A j ( x ) − ∇ p A j ( y ) ∥ ≤ C A λ ∥ x − y ∥ ν , \lVert\nabla^pA_j(x)-\nabla^pA_j(y)\rVert\le C_A\lambda\lVert x-y\rVert^\nu, ∥ ∇ p A j ( x ) − ∇ p A j ( y )∥ ≤ C A λ ∥ x − y ∥ ν , with C A C_A C A given by Eq. (D.47).
The constants depend only on p , ν p,\nu p , ν and the fixed smoothing function, independently of m m m , dimension, and phase scales. No ∇ p + 1 P j \nabla^{p+1}P_j ∇ p + 1 P j is required, including at ν = 1 \nu=1 ν = 1 .
Proof. • Proof of (a). Since ζ j ∈ C p \zeta_j\in C^p ζ j ∈ C p and χ δ j ∈ C ∞ \chi_{\delta_j}\in C^\infty χ δ j ∈ C ∞ , K j ∈ C p K_j\in C^p K j ∈ C p . Lemma B.8 in Appx. B requires a convex open domain and bounds on the value and derivatives of the inner function.
Domain and value bound. Since ζ j \zeta_j ζ j is continuous and convex, T j T_j T j is closed and convex; ζ j ( x j ∗ ) = − 4 δ j \zeta_j(x_j^*)=-4\delta_j ζ j ( x j ∗ ) = − 4 δ j makes it nonempty. Set
U j : = { x : dist ( x , T j ) < ρ j A } = T j + B ( 0 , ρ j A ) . U_j:=\{x:\operatorname{dist}(x,T_j)<\rho_j^A\}=T_j+B(0,\rho_j^A). U j := { x : dist ( x , T j ) < ρ j A } = T j + B ( 0 , ρ j A ) .
The sum is convex and open, with T j ⊂ U j ⊂ T j , ρ j A T_j\subset U_j\subset T_{j,\rho_j^A} T j ⊂ U j ⊂ T j , ρ j A . For x ∈ U j x\in U_j x ∈ U j , choose a nearest a ∈ T j a\in T_j a ∈ T j ; ∥ x − a ∥ < ρ j A \lVert x-a\rVert<\rho_j^A ∥ x − a ∥ < ρ j A . Set
M : = M 1 λ ( ρ j A ) q − 1 , v ( t ) : = ζ j ( a + t ( x − a ) ) , g ( t ) : = M t ∥ x − a ∥ ( 0 ≤ t ≤ 1 ) . M:=M_1\lambda\left(\rho_j^A\right)^{q-1},\qquad v(t):=\zeta_j\left(a+t(x-a)\right),\qquad g(t):=Mt\lVert x-a\rVert\quad(0\le t\le1). M := M 1 λ ( ρ j A ) q − 1 , v ( t ) := ζ j ( a + t ( x − a ) ) , g ( t ) := M t ∥ x − a ∥ ( 0 ≤ t ≤ 1 ) .
The segment lies in the open convex set U j U_j U j . Lemma D.18 (b) gives
∣ v ′ ( t ) ∣ ≤ ∥ ∇ ζ j ( a + t ( x − a ) ) ∥ ∥ x − a ∥ ≤ M ∥ x − a ∥ = g ′ ( t ) . |v'(t)| \le\|\nabla\zeta_j(a+t(x-a))\|\|x-a\|
\le M\|x-a\|=g'(t). ∣ v ′ ( t ) ∣ ≤ ∥∇ ζ j ( a + t ( x − a )) ∥∥ x − a ∥ ≤ M ∥ x − a ∥ = g ′ ( t ) .
Theorem B.3 therefore implies ∣ ζ j ( x ) − ζ j ( a ) ∣ ≤ M ∥ x − a ∥ |\zeta_j(x)-\zeta_j(a)|\le M\|x-a\| ∣ ζ j ( x ) − ζ j ( a ) ∣ ≤ M ∥ x − a ∥ , so
ζ j ( x ) ≤ ζ j ( a ) + M 1 λ ( ρ j A ) q − 1 ∥ x − a ∥ ≤ δ j + M 1 λ ( ρ j A ) q = ( 1 + M 1 ) δ j . \zeta_j(x)\le\zeta_j(a)+M_1\lambda\left(\rho_j^A\right)^{q-1}\|x-a\|
\le\delta_j+M_1\lambda\left(\rho_j^A\right)^q=(1+M_1)\delta_j. ζ j ( x ) ≤ ζ j ( a ) + M 1 λ ( ρ j A ) q − 1 ∥ x − a ∥ ≤ δ j + M 1 λ ( ρ j A ) q = ( 1 + M 1 ) δ j .
Here ζ j ( a ) ≤ δ j \zeta_j(a)\le\delta_j ζ j ( a ) ≤ δ j and λ ( ρ j A ) q = δ j \lambda\left(\rho_j^A\right)^q=\delta_j λ ( ρ j A ) q = δ j . Together with ζ j ≥ − C d u a l δ j \zeta_j\ge-C_{\mathrm{dual}}\delta_j ζ j ≥ − C dual δ j (Lemma D.17 (a)), this gives ∣ ζ j ∣ ≤ M 0 δ j |\zeta_j|\le M_0\delta_j ∣ ζ j ∣ ≤ M 0 δ j on U j U_j U j , where M 0 : = max { C d u a l , 1 + M 1 } M_0:=\max\{C_{\mathrm{dual}},1+M_1\} M 0 := max { C dual , 1 + M 1 } .
Normalized bounds. Define
U ~ j : = ( ρ j A ) − 1 U j , ζ ~ j ( u ) : = ζ j ( ρ j A u ) δ j . \widetilde{U}_j:=\left(\rho_j^A\right)^{-1}U_j,\qquad\widetilde{\zeta}_j(u):=\frac{\zeta_j\left(\rho_j^A u\right)}{\delta_j}. U j := ( ρ j A ) − 1 U j , ζ j ( u ) := δ j ζ j ( ρ j A u ) .
The domain remains convex and open, and ∥ ζ ~ j ∥ ∞ ≤ M 0 \|\widetilde{\zeta}_j\|_\infty\le M_0 ∥ ζ j ∥ ∞ ≤ M 0 . For 1 ≤ k ≤ p 1\le k\le p 1 ≤ k ≤ p , the chain rule and Lemma D.18 give
∇ k ζ ~ j ( u ) = ( ρ j A ) k δ j ∇ k ζ j ( ρ j A u ) , \nabla^k\widetilde{\zeta}_j(u)=\frac{\left(\rho_j^A\right)^k}{\delta_j}\nabla^k\zeta_j\left(\rho_j^A u\right), ∇ k ζ j ( u ) = δ j ( ρ j A ) k ∇ k ζ j ( ρ j A u ) ,
∥ ∇ k ζ ~ j ∥ ∞ ; U ~ j ≤ ( ρ j A ) k δ j M k λ ( ρ j A ) q − k = M k . \|\nabla^k\widetilde{\zeta}_j\|_{\infty;\widetilde{U}_j}\le\frac{\left(\rho_j^A\right)^k}{\delta_j}M_k\lambda\left(\rho_j^A\right)^{q-k}=M_k. ∥ ∇ k ζ j ∥ ∞ ; U j ≤ δ j ( ρ j A ) k M k λ ( ρ j A ) q − k = M k .
Here δ j = λ ( ρ j A ) q \delta_j=\lambda\left(\rho_j^A\right)^q δ j = λ ( ρ j A ) q . For distinct u , v ∈ U ~ j u,v\in\widetilde{U}_j u , v ∈ U j , Lemma D.15 (b) gives
∥ ∇ p ζ ~ j ( u ) − ∇ p ζ ~ j ( v ) ∥ ∥ u − v ∥ ν ≤ ( ρ j A ) p δ j L P ∥ ρ j A ( u − v ) ∥ ν ∥ u − v ∥ ν = L P λ ( ρ j A ) p + ν δ j = L P . \begin{aligned}
\frac{\|\nabla^p\widetilde{\zeta}_j(u)-\nabla^p\widetilde{\zeta}_j(v)\|}{\|u-v\|^\nu}
\le\frac{\left(\rho_j^A\right)^p}{\delta_j}L_P\frac{\|\rho_j^A(u-v)\|^\nu}{\|u-v\|^\nu} \\
&=L_P\frac{\lambda\left(\rho_j^A\right)^{p+\nu}}{\delta_j}=L_P.
\end{aligned} ∥ u − v ∥ ν ∥ ∇ p ζ j ( u ) − ∇ p ζ j ( v ) ∥ ≤ δ j ( ρ j A ) p L P ∥ u − v ∥ ν ∥ ρ j A ( u − v ) ∥ ν = L P δ j λ ( ρ j A ) p + ν = L P .
Since p + ν = q p+\nu=q p + ν = q , the bounded norm therefore satisfies
∥ ζ ~ j ∥ p , ν ; U ~ j ≤ C ζ : = max { 1 , M 0 , M 1 , … , M p , L P } . \|\widetilde{\zeta}_j\|_{p,\nu;\widetilde{U}_j}\le C_\zeta:=\max\{1,M_0,M_1,\ldots,M_p,L_P\}. ∥ ζ j ∥ p , ν ; U j ≤ C ζ := max { 1 , M 0 , M 1 , … , M p , L P } .
Choose J : = ( − M 0 − 1 , M 0 + 1 ) J:=(-M_0-1,M_0+1) J := ( − M 0 − 1 , M 0 + 1 ) , so that ζ ~ j ( U ~ j ) ⊂ [ − M 0 , M 0 ] ⊂ J \widetilde{\zeta}_j(\widetilde{U}_j)\subset[-M_0,M_0]\subset J ζ j ( U j ) ⊂ [ − M 0 , M 0 ] ⊂ J . Smoothness of χ 1 \chi_1 χ 1 on J ‾ \overline{J} J makes C p + 1 : = ∥ χ 1 ( p + 1 ) ∥ ∞ ; J C_{p+1}:=\|\chi_1^{(p+1)}\|_{\infty;J} C p + 1 := ∥ χ 1 ( p + 1 ) ∥ ∞ ; J finite. For any distinct a , b ∈ J a,b\in J a , b ∈ J , the mean-value theorem on the interval between them gives
∣ χ 1 ( p ) ( a ) − χ 1 ( p ) ( b ) ∣ ≤ C p + 1 ∣ a − b ∣ , |\chi_1^{(p)}(a)-\chi_1^{(p)}(b)|\le C_{p+1}|a-b|, ∣ χ 1 ( p ) ( a ) − χ 1 ( p ) ( b ) ∣ ≤ C p + 1 ∣ a − b ∣ ,
∣ χ 1 ( p ) ( a ) − χ 1 ( p ) ( b ) ∣ ∣ a − b ∣ ν ≤ C p + 1 ∣ a − b ∣ 1 − ν ≤ C p + 1 ∣ J ∣ 1 − ν . \frac{|\chi_1^{(p)}(a)-\chi_1^{(p)}(b)|}{|a-b|^\nu}\le C_{p+1}|a-b|^{1-\nu}
\le C_{p+1}|J|^{1-\nu}. ∣ a − b ∣ ν ∣ χ 1 ( p ) ( a ) − χ 1 ( p ) ( b ) ∣ ≤ C p + 1 ∣ a − b ∣ 1 − ν ≤ C p + 1 ∣ J ∣ 1 − ν .
The last line uses ∣ a − b ∣ ≤ ∣ J ∣ |a-b|\leq|\mathcal{J}| ∣ a − b ∣ ≤ ∣ J ∣ and 1 − ν ≥ 0 1-\nu\geq0 1 − ν ≥ 0 , including ν = 1 \nu=1 ν = 1 . Thus, for all a , b ∈ J a,b\in\mathcal{J} a , b ∈ J ,
∣ χ 1 ( p ) ( a ) − χ 1 ( p ) ( b ) ∣ ≤ ∥ χ 1 ( p + 1 ) ∥ ∞ ; J ∣ J ∣ 1 − ν ∣ a − b ∣ ν . |\chi_1^{(p)}(a)-\chi_1^{(p)}(b)|\leq\|\chi_1^{(p+1)}\|_{\infty;\mathcal{J}}|\mathcal{J}|^{1-\nu}|a-b|^\nu. ∣ χ 1 ( p ) ( a ) − χ 1 ( p ) ( b ) ∣ ≤ ∥ χ 1 ( p + 1 ) ∥ ∞ ; J ∣ J ∣ 1 − ν ∣ a − b ∣ ν .
where ∣ J ∣ = 2 M 0 + 2 |\mathcal{J}|=2M_0+2 ∣ J ∣ = 2 M 0 + 2 ; thus C χ : = ∥ χ 1 ∥ p , ν ; J < ∞ C_{\chi}:=\|\chi_1\|_{p,\nu;\mathcal{J}}<\infty C χ := ∥ χ 1 ∥ p , ν ; J < ∞ . Apply Lemma B.8 with
U = U ~ j , f = ζ ~ j , h = χ 1 , J = ( − M 0 − 1 , M 0 + 1 ) . U=\widetilde{U}_j,\qquad f=\widetilde{\zeta}_j,\qquad h=\chi_1,\qquad J=(-M_0-1,M_0+1). U = U j , f = ζ j , h = χ 1 , J = ( − M 0 − 1 , M 0 + 1 ) .
The bounds ∥ f ∥ p , ν ; U ≤ C ζ \|f\|_{p,\nu;U}\leq C_{\zeta} ∥ f ∥ p , ν ; U ≤ C ζ and ∥ h ∥ p , ν ; J = C χ \|h\|_{p,\nu;J}=C_{\chi} ∥ h ∥ p , ν ; J = C χ yield
∥ χ 1 ∘ ζ ~ j ∥ p , ν ; U ~ j ≤ C q c o m p C χ max { 1 , C ζ q } = C c o m p , C c o m p : = C q c o m p C χ C ζ q . \|\chi_1\circ\widetilde{\zeta}_j\|_{p,\nu;\widetilde{U}_j}\leq C_q^{\mathrm{comp}}C_{\chi}\max\{1,C_{\zeta}^{q}\}=C_{\mathrm{comp}},\qquad C_{\mathrm{comp}}:=C_q^{\mathrm{comp}}C_{\chi}C_{\zeta}^{q}. ∥ χ 1 ∘ ζ j ∥ p , ν ; U j ≤ C q comp C χ max { 1 , C ζ q } = C comp , C comp := C q comp C χ C ζ q .
At ν = 1 \nu=1 ν = 1 the lemma uses the Lipschitz ∇ p ζ ~ j \nabla^p\widetilde{\zeta}_j ∇ p ζ j ; only the smooth outer function χ 1 \chi_1 χ 1 uses a ( p + 1 ) t h (p+1)^{th} ( p + 1 ) t h derivative.
Return to the original scale. By Eq. (B.1),
λ δ j ( a ) = ψ δ j ( a ) − a = a + δ j Θ ( a δ j ) 2 − a = δ j ( a δ j + Θ ( a δ j ) 2 − a δ j ) = δ j χ 1 ( a δ j ) . \begin{aligned}
\lambda_{\delta_j}(a)=\psi_{\delta_j}(a)-a=\frac{a+\delta_j\Theta\left(\frac{a}{\delta_j}\right)}{2}-a \\
&=\delta_j\left(\frac{\frac{a}{\delta_j}+\Theta\left(\frac{a}{\delta_j}\right)}{2}-\frac{a}{\delta_j}\right)=\delta_j\chi_1\left(\frac{a}{\delta_j}\right).
\end{aligned} λ δ j ( a ) = ψ δ j ( a ) − a = 2 a + δ j Θ ( δ j a ) − a = δ j 2 δ j a + Θ ( δ j a ) − δ j a = δ j χ 1 ( δ j a ) .
Put H j = χ 1 ∘ ζ ~ j H_j=\chi_1\circ\widetilde{\zeta}_j H j = χ 1 ∘ ζ j . The normalized definition then gives
K j ( ρ j A u ) = λ δ j ( ζ j ( ρ j A u ) ) = δ j χ 1 ( ζ j ( ρ j A u ) δ j ) = δ j H j ( u ) . \begin{aligned}
\mathcal{K}_j\left(\rho_j^A\boldsymbol{u}\right)=\lambda_{\delta_j}\left(\zeta_j\left(\rho_j^A\boldsymbol{u}\right)\right) \\
&=\delta_j\chi_1\left(\frac{\zeta_j\left(\rho_j^A\boldsymbol{u}\right)}{\delta_j}\right)=\delta_jH_j(\boldsymbol{u}).
\end{aligned} K j ( ρ j A u ) = λ δ j ( ζ j ( ρ j A u ) ) = δ j χ 1 ( δ j ζ j ( ρ j A u ) ) = δ j H j ( u ) .
Differentiating with respect to u \boldsymbol{u} u gives ( ρ j A ) p ∇ p K j ( ρ j A u ) = δ j ∇ p H j ( u ) \left(\rho_j^A\right)^p\nabla^p\mathcal{K}_j\left(\rho_j^A\boldsymbol{u}\right)=\delta_j\nabla^pH_j(\boldsymbol{u}) ( ρ j A ) p ∇ p K j ( ρ j A u ) = δ j ∇ p H j ( u ) . Thus, for distinct u , v ∈ U ~ j \boldsymbol{u},\boldsymbol{v}\in\widetilde{U}_j u , v ∈ U j ,
∥ ∇ p K j ( ρ j A u ) − ∇ p K j ( ρ j A v ) ∥ ∥ ρ j A u − ρ j A v ∥ ν = δ j ( ρ j A ) p ∥ ∇ p H j ( u ) − ∇ p H j ( v ) ∥ ( ρ j A ) ν ∥ u − v ∥ ν = δ j ( ρ j A ) p + ν ∥ ∇ p H j ( u ) − ∇ p H j ( v ) ∥ ∥ u − v ∥ ν . \begin{aligned}
\frac{\|\nabla^p\mathcal{K}_j\left(\rho_j^A\boldsymbol{u}\right)-\nabla^p\mathcal{K}_j\left(\rho_j^A\boldsymbol{v}\right)\|}{\|\rho_j^A\boldsymbol{u}-\rho_j^A\boldsymbol{v}\|^\nu}
=
\frac{\delta_j}{\left(\rho_j^A\right)^p}\frac{\|\nabla^pH_j(\boldsymbol{u})-\nabla^pH_j(\boldsymbol{v})\|}{\left(\rho_j^A\right)^\nu\|\boldsymbol{u}-\boldsymbol{v}\|^\nu} \\
&=\frac{\delta_j}{\left(\rho_j^A\right)^{p+\nu}}\frac{\|\nabla^pH_j(\boldsymbol{u})-\nabla^pH_j(\boldsymbol{v})\|}{\|\boldsymbol{u}-\boldsymbol{v}\|^\nu}.
\end{aligned} ∥ ρ j A u − ρ j A v ∥ ν ∥ ∇ p K j ( ρ j A u ) − ∇ p K j ( ρ j A v ) ∥ = ( ρ j A ) p δ j ( ρ j A ) ν ∥ u − v ∥ ν ∥ ∇ p H j ( u ) − ∇ p H j ( v ) ∥ = ( ρ j A ) p + ν δ j ∥ u − v ∥ ν ∥ ∇ p H j ( u ) − ∇ p H j ( v ) ∥ .
The composition bound and the bijection u ↦ ρ j A u \boldsymbol{u}\mapsto\rho_j^A\boldsymbol{u} u ↦ ρ j A u therefore give, for all x , y ∈ U j \boldsymbol{x},\boldsymbol{y}\in U_j x , y ∈ U j ,
∥ ∇ p K j ( x ) − ∇ p K j ( y ) ∥ ≤ C c o m p δ j ( ρ j A ) p + ν ∥ x − y ∥ ν = C c o m p λ ∥ x − y ∥ ν ( p + ν = q ) . \begin{aligned}
\|\nabla^p\mathcal{K}_j(\boldsymbol{x})-\nabla^p\mathcal{K}_j(\boldsymbol{y})\|\leq C_{\mathrm{comp}}\frac{\delta_j}{\left(\rho_j^A\right)^{p+\nu}}\|\boldsymbol{x}-\boldsymbol{y}\|^\nu\\
&=C_{\mathrm{comp}}\lambda\|\boldsymbol{x}-\boldsymbol{y}\|^\nu\qquad(p+\nu=q).
\end{aligned} ∥ ∇ p K j ( x ) − ∇ p K j ( y ) ∥ ≤ C comp ( ρ j A ) p + ν δ j ∥ x − y ∥ ν = C comp λ ∥ x − y ∥ ν ( p + ν = q ) .
Global extension. Lemma B.17 (a) and smoothness give χ δ j ( k ) = 0 \chi_{\delta_j}^{(k)}=0 χ δ j ( k ) = 0 on [ δ j , ∞ ) [\delta_j,\infty) [ δ j , ∞ ) for 0 ≤ k ≤ p 0\leq k\leq p 0 ≤ k ≤ p ; the chain rule gives ∇ p K j = 0 \nabla^p\mathcal{K}_j=0 ∇ p K j = 0 on { ζ j ≥ δ j } \{\zeta_j\geq\delta_j\} { ζ j ≥ δ j } . Pairs in T j T_j T j satisfy the bound on U j U_j U j ; pairs outside have zero tensors. For x ∈ T j x \in\mathcal{T}_j x ∈ T j and y ∉ T j y \notin\mathcal{T}_j y ∈ / T j , continuity supplies z ∈ [ x , y ] z \in[x,y] z ∈ [ x , y ] with ζ j ( z ) = δ j \zeta_j(z)=\delta_j ζ j ( z ) = δ j . Thus x , z ∈ T j ⊂ U j x,z \in\mathcal{T}_j \subset U_j x , z ∈ T j ⊂ U j , ∇ p K j ( z ) = ∇ p K j ( y ) = 0 \nabla^p K_j(z)=\nabla^p K_j(y)=0 ∇ p K j ( z ) = ∇ p K j ( y ) = 0 , and
∥ ∇ p K j ( x ) − ∇ p K j ( y ) ∥ = ∥ ∇ p K j ( x ) − ∇ p K j ( z ) ∥ ≤ C c o m p λ ∥ x − z ∥ ν ≤ C c o m p λ ∥ x − y ∥ ν . \left\|\nabla^p K_j(x)-\nabla^p K_j(y)\right\|=\left\|\nabla^p K_j(x)-\nabla^p K_j(z)\right\|
\le C_{\mathrm{comp}}\lambda\left\|x-z\right\|^\nu
\le C_{\mathrm{comp}}\lambda\left\|x-y\right\|^\nu. ∥ ∇ p K j ( x ) − ∇ p K j ( y ) ∥ = ∥ ∇ p K j ( x ) − ∇ p K j ( z ) ∥ ≤ C comp λ ∥ x − z ∥ ν ≤ C comp λ ∥ x − y ∥ ν .
Hence C K : = C c o m p C_K:=C_{\mathrm{comp}} C K := C comp is independent of m m m , dimension, and phase scale.
Proof of (b). Use A j = O j + K j A_j=O_j+K_j A j = O j + K j , ∇ p O j = ∇ p P j \nabla^p O_j=\nabla^p P_j ∇ p O j = ∇ p P j , and Lemma D.15 (a). For all x , y ∈ R d x,y\in\mathbb{R}^d x , y ∈ R d ,
∥ ∇ p A j ( x ) − ∇ p A j ( y ) ∥ ≤ ∥ ∇ p P j ( x ) − ∇ p P j ( y ) ∥ + ∥ ∇ p K j ( x ) − ∇ p K j ( y ) ∥ ≤ ( L P + C c o m p ) λ ∥ x − y ∥ ν = C A λ ∥ x − y ∥ ν , C A : = L P + C c o m p . (146) \begin{aligned}
\left\|\nabla^p A_j(x)-\nabla^p A_j(y)\right\|
&\le\left\|\nabla^p P_j(x)-\nabla^p P_j(y)\right\|+\left\|\nabla^p K_j(x)-\nabla^p K_j(y)\right\|\\
&\le(L_P+C_{\mathrm{comp}})\lambda\left\|x-y\right\|^\nu\\
&=C_A\lambda\left\|x-y\right\|^\nu,\qquad C_A:=L_P+C_{\mathrm{comp}}. \tag*{(146)}
\end{aligned} ∥ ∇ p A j ( x ) − ∇ p A j ( y ) ∥ ≤ ∥ ∇ p P j ( x ) − ∇ p P j ( y ) ∥ + ∥ ∇ p K j ( x ) − ∇ p K j ( y ) ∥ ≤ ( L P + C comp ) λ ∥ x − y ∥ ν = C A λ ∥ x − y ∥ ν , C A := L P + C comp . (146)
Compatible Parameters and Assembly
Parameter choices and the complete function
Phase coordinates and scales
Definition E.1 (Phase coordinates and scales). Choose m , L ∈ N m,L\in\mathbb{N} m , L ∈ N , μ , τ 0 > 0 \mu,\tau_0>0 μ , τ 0 > 0 , and mutually orthogonal subspaces
E j = span { v j , u j , 1 , … , u j , m } , 0 ≤ j < L , E_j=\operatorname{span}\{v_j,u_{j,1},\ldots,u_{j,m}\},\qquad0\le j<L, E j = span { v j , u j , 1 , … , u j , m } , 0 ≤ j < L , with the displayed orthonormal bases. For a contraction η ∈ ( 0 , 1 ) \eta\in(0,1) η ∈ ( 0 , 1 ) to be chosen later, put
τ j = η j τ 0 , t j ⋆ = m ( m + 1 ) τ j , h j = t j ⋆ v j + ∑ i = 1 m ( m − i + 1 ) τ j u j , i , c j = ∑ k < j h k . \tau_j=\eta^j\tau_0,\qquad t_j^\star=\sqrt{m}(m+1)\tau_j,\qquad h_j=t_j^\star v_j+\sum_{i=1}^{m}(m-i+1)\tau_j u_{j,i},\qquad c_j=\sum_{k<j}h_k. τ j = η j τ 0 , t j ⋆ = m ( m + 1 ) τ j , h j = t j ⋆ v j + i = 1 ∑ m ( m − i + 1 ) τ j u j , i , c j = k < j ∑ h k . Thus h j = c j + 1 − c j h_j=c_{j+1}-c_j h j = c j + 1 − c j . Lemma D.3 (c) in Appx. D gives
R j : = ∥ h j ∥ = β m τ j m 3 / 2 , β m 2 = 4 3 + 5 2 m + 7 6 m 2 , 1 ≤ β m ≤ 5 . R_j:=\left\|h_j\right\|=\beta_m\tau_jm^{3/2},\qquad\beta_m^2=\frac{4}{3}+\frac{5}{2m}+\frac{7}{6m^2},\qquad1\le\beta_m\le\sqrt{5}. R j := ∥ h j ∥ = β m τ j m 3/2 , β m 2 = 3 4 + 2 m 5 + 6 m 2 7 , 1 ≤ β m ≤ 5 . In particular R j = η j R 0 R_j=\eta^jR_0 R j = η j R 0 .
For η \eta η satisfying the base nesting inequality Eq. (C.15), Lemma C.1 (b) in Appx. C gives
b j : = ∇ B ( c j + 1 ) − ∇ B ( c j ) = μ R j q − 2 h j , G j : = ∥ b j ∥ = μ R j q − 1 . b_j:=\nabla B(c_{j+1})-\nabla B(c_j)=\mu R_j^{q-2}h_j,\qquad G_j:=\left\|b_j\right\|=\mu R_j^{q-1}. b j := ∇ B ( c j + 1 ) − ∇ B ( c j ) = μ R j q − 2 h j , G j := ∥ b j ∥ = μ R j q − 1 . To match the chain’s endpoint gradient with its affine continuation, the starting slope must compensate the base increment’s projection G j t j ⋆ R j \frac{G_jt_j^\star}{R_j} R j G j t j ⋆ along v j v_j v j . For the chain coefficient λ > 0 \lambda>0 λ > 0 , we therefore define
Γ j : = λ τ j q − 1 m , s j : = Γ j + G j t j ⋆ R j . \Gamma_j:=\frac{\lambda\tau_j^{q-1}}{\sqrt{m}},\qquad s_j:=\Gamma_j+\frac{G_jt_j^\star}{R_j}. Γ j := m λ τ j q − 1 , s j := Γ j + R j G j t j ⋆ .
Center-gradient conditions
For the mutually orthogonal phase subspaces E 0 , … , E L − 1 E_0,\ldots,E_{L-1} E 0 , … , E L − 1 and unit vectors v j ∈ E j v_j \in E_j v j ∈ E j , the centers satisfy
c 0 = 0 , c j + 1 − c j ∈ E j . c_0 = 0,\qquad c_{j+1}-c_j \in E_j. c 0 = 0 , c j + 1 − c j ∈ E j .
Let the phase-j j j prefix be a subspace W j , b ⊂ E 0 ⊕ ⋯ ⊕ E j W_{j,b} \subset E_0 \oplus\cdots\oplus E_j W j , b ⊂ E 0 ⊕ ⋯ ⊕ E j . For j < L − 1 j < L-1 j < L − 1 ,
v j + 1 ⊥ W j , b , v j + 1 ⊥ c j + 1 . v_{j+1} \perp W_{j,b},\qquad v_{j+1} \perp c_{j+1}. v j + 1 ⊥ W j , b , v j + 1 ⊥ c j + 1 .
Consequently the choice ∇ F ( c j + 1 ) = − s j + 1 v j + 1 \nabla F(c_{j+1})=-s_{j+1}v_{j+1} ∇ F ( c j + 1 ) = − s j + 1 v j + 1 gives ⟨ ∇ F ( c j + 1 ) , x − c j + 1 ⟩ = 0 \langle\nabla F(c_{j+1}),x-c_{j+1}\rangle=0 ⟨ ∇ F ( c j + 1 ) , x − c j + 1 ⟩ = 0 for every x ∈ W j , b x\in W_{j,b} x ∈ W j , b , as required by Lemma E.12 (b), regardless of ∥ x ∥ \lVert x\rVert ∥ x ∥ .
Design condition E.2 (Center gradients for the gap proof). Choose positive s j s_j s j and require
∇ F ( c j ) = − s j v j ( 0 ≤ j < L ) , ∇ F ( c L ) = 0. (147) \nabla F(c_j)=-s_jv_j \qquad(0\leq j<L),\qquad\nabla F(c_L)=0. \tag*{(147)} ∇ F ( c j ) = − s j v j ( 0 ≤ j < L ) , ∇ F ( c L ) = 0. (147)
The terminal equality also identifies the minimizer once uniform convexity is proved. We use s L v L = 0 s_Lv_L=0 s L v L = 0 as a convention; no additional direction v L v_L v L is needed.
Phase decomposition
Fix a phase j j j and e j , δ j > 0 e_j,\delta_j>0 e j , δ j > 0 . Let O j O_j O j be convex and C p C^p C p , let I j I_j I j be affine, and suppose
O j − I j ≥ e j on W j , b , O j − I j < − δ j near c j + 1 . O_j-I_j\geq e_j\ \text{on }W_{j,b},\qquad O_j-I_j<-\delta_j\ \text{near }c_{j+1}. O j − I j ≥ e j on W j , b , O j − I j < − δ j near c j + 1 .
The smooth positive part from Appx. B.2.2 satisfies
ψ δ j ( t ) ≥ t + , ψ δ j ( t ) = 0 ( t ≤ − δ j ) , ψ δ j ( t ) = t ( t ≥ δ j ) . \psi_{\delta_j}(t)\geq t_+,\qquad\psi_{\delta_j}(t)=0\ (t\leq-\delta_j),\qquad\psi_{\delta_j}(t)=t\ (t\geq\delta_j). ψ δ j ( t ) ≥ t + , ψ δ j ( t ) = 0 ( t ≤ − δ j ) , ψ δ j ( t ) = t ( t ≥ δ j ) .
For a differentiable function Ψ j \Psi_j Ψ j , define
r j : = ψ δ j ( O j − I j ) , r_j:=\psi_{\delta_j}(O_j-I_j), r j := ψ δ j ( O j − I j ) ,
A j : = I j + r j = M δ j ( I j , O j ) , A_j:=I_j+r_j=M_{\delta_j}(I_j,O_j), A j := I j + r j = M δ j ( I j , O j ) ,
F : = Ψ j + r j = Ψ j + A j − I j . (148) F:=\Psi_j+r_j=\Psi_j+A_j-I_j. \tag*{(148)} F := Ψ j + r j = Ψ j + A j − I j . (148)
Then r j ≥ e j r_j\geq e_j r j ≥ e j on W j , b W_{j,b} W j , b and r j = 0 r_j=0 r j = 0 near c j + 1 c_{j+1} c j + 1 . Lemma B.21 (a) in Appx. B gives convexity of A j A_j A j .
Design condition E.3 (Two choices that isolate the current gradient change). Let g j : = ∇ I j g_j:=\nabla I_j g j := ∇ I j . In addition to Eq. (E.2), assume:
(a) Zero starting gradient for the current phase function.
∇ A j ( c j ) = 0. (149) \nabla A_j(c_j)=0. \tag*{(149)} ∇ A j ( c j ) = 0. (149)
(b) The remaining terms contribute no gradient change.
∇ ( Ψ j − B ) ( c j + 1 ) = ∇ ( Ψ j − B ) ( c j ) . (150) \nabla(\Psi_j-B)(c_{j+1})=\nabla(\Psi_j-B)(c_j). \tag*{(150)} ∇ ( Ψ j − B ) ( c j + 1 ) = ∇ ( Ψ j − B ) ( c j ) . (150)
Part (a) preserves the starting gradient of Ψ j − I j \Psi_j-I_j Ψ j − I j when A j A_j A j is added. Part (b) leaves the gradient change of Ψ j \Psi_j Ψ j equal to that of B B B .
For b j : = ∇ B ( c j + 1 ) − ∇ B ( c j ) b_j:=\nabla B(c_{j+1})-\nabla B(c_j) b j := ∇ B ( c j + 1 ) − ∇ B ( c j ) , the target gradient relation is
− s j v j + b j + g j = − s j + 1 v j + 1 , -s_jv_j+b_j+g_j=-s_{j+1}v_{j+1}, − s j v j + b j + g j = − s j + 1 v j + 1 ,
equivalently g j = s j v j − s j + 1 v j + 1 − b j . (151) \text{equivalently}\qquad g_j=s_jv_j-s_{j+1}v_{j+1}-b_j. \tag*{(151)} equivalently g j = s j v j − s j + 1 v j + 1 − b j . (151)
The affine-gradient identity
Lemma E.4 (The two choices determine the affine gradient). Assume Condition E.3, the decomposition Eq. (E.2), A j = I j A_j=I_j A j = I j near c j + 1 c_{j+1} c j + 1 , and ∇ F ( c j ) = − s j v j \nabla F(c_j)=-s_jv_j ∇ F ( c j ) = − s j v j . Keep the definition b j : = ∇ B ( c j + 1 ) − ∇ B ( c j ) b_j:=\nabla B(c_{j+1})-\nabla B(c_j) b j := ∇ B ( c j + 1 ) − ∇ B ( c j ) . Then:
(a) The actual gradient changes are
∇ Ψ j ( c j + 1 ) − ∇ Ψ j ( c j ) = b j , ∇ r j ( c j + 1 ) − ∇ r j ( c j ) = g j . \nabla\Psi_j(c_{j+1})-\nabla\Psi_j(c_j)=b_j,\qquad\nabla r_j(c_{j+1})-\nabla r_j(c_j)=g_j. ∇ Ψ j ( c j + 1 ) − ∇ Ψ j ( c j ) = b j , ∇ r j ( c j + 1 ) − ∇ r j ( c j ) = g j . (b) The next-gradient target determines the affine slope. The target ∇ F ( c j + 1 ) = − s j + 1 v j + 1 \nabla F(c_{j+1})=-s_{j+1}v_{j+1} ∇ F ( c j + 1 ) = − s j + 1 v j + 1 holds if and only if the affine gradient satisfies Eq. (E.5).
Proof. • Proof of (a). For the first identity, split off the base term:
∇ Ψ j ( c j + 1 ) − ∇ Ψ j ( c j ) = ∇ B ( c j + 1 ) − ∇ B ( c j ) + ∇ ( Ψ j − B ) ( c j + 1 ) − ∇ ( Ψ j − B ) ( c j ) = ∇ B ( c j + 1 ) − ∇ B ( c j ) by xref‘Condition E.3 (b)‘ , = b j . \begin{aligned}
\nabla\Psi_j(c_{j+1})-\nabla\Psi_j(c_j)
&=\nabla B(c_{j+1})-\nabla B(c_j)\\
&\quad+\nabla(\Psi_j-B)(c_{j+1})-\nabla(\Psi_j-B)(c_j)\\
&=\nabla B(c_{j+1})-\nabla B(c_j) \qquad\text{by {xref}`Condition E.3 (b)`},\\
&=b_j.
\end{aligned} ∇ Ψ j ( c j + 1 ) − ∇ Ψ j ( c j ) = ∇ B ( c j + 1 ) − ∇ B ( c j ) + ∇ ( Ψ j − B ) ( c j + 1 ) − ∇ ( Ψ j − B ) ( c j ) = ∇ B ( c j + 1 ) − ∇ B ( c j ) by xref ‘Condition E.3 (b)‘ , = b j .
For the second, use r j = A j − I j r_j=A_j-I_j r j = A j − I j from Eq. (E.2), ∇ I j = g j \nabla I_j=g_j ∇ I j = g j , Condition E.3 (a), and the assumed neighborhood equality A j = I j A_j=I_j A j = I j :
∇ r j ( c j ) = ∇ A j ( c j ) − ∇ I j ( c j ) = 0 − g j , \nabla r_j(c_j)=\nabla A_j(c_j)-\nabla I_j(c_j)=0-g_j, ∇ r j ( c j ) = ∇ A j ( c j ) − ∇ I j ( c j ) = 0 − g j ,
∇ r j ( c j + 1 ) = ∇ A j ( c j + 1 ) − ∇ I j ( c j + 1 ) = g j − g j = 0 , \nabla r_j(c_{j+1})=\nabla A_j(c_{j+1})-\nabla I_j(c_{j+1})=g_j-g_j=0, ∇ r j ( c j + 1 ) = ∇ A j ( c j + 1 ) − ∇ I j ( c j + 1 ) = g j − g j = 0 ,
∇ r j ( c j + 1 ) − ∇ r j ( c j ) = 0 − ( − g j ) = g j . \nabla r_j(c_{j+1})-\nabla r_j(c_j)=0-(-g_j)=g_j. ∇ r j ( c j + 1 ) − ∇ r j ( c j ) = 0 − ( − g j ) = g j .
• Proof of (b). Differentiate F = Ψ j + r j F=\Psi_j+r_j F = Ψ j + r j at the two centers. Part (a) and the assumed starting gradient yield
∇ F ( c j + 1 ) = ∇ F ( c j ) + b j + g j = − s j v j + b j + g j . \nabla F(c_{j+1})=\nabla F(c_j)+b_j+g_j=-s_jv_j+b_j+g_j. ∇ F ( c j + 1 ) = ∇ F ( c j ) + b j + g j = − s j v j + b j + g j .
Thus the next-center target in Condition E.2 holds exactly when − s j v j + b j + g j = − s j + 1 v j + 1 -s_jv_j+b_j+g_j=-s_{j+1}v_{j+1} − s j v j + b j + g j = − s j + 1 v j + 1 . Solving for g j g_j g j gives Eq. (E.5).
:::
The assembled phase functions therefore use
g j : = s j v j − s j + 1 v j + 1 − b j . (152) g_j:=s_jv_j-s_{j+1}v_{j+1}-b_j. \tag*{(152)} g j := s j v j − s j + 1 v j + 1 − b j . (152)
In the terminal phase, we use s L = t L ⋆ = 0 s_L=t_L^\star=0 s L = t L ⋆ = 0 and omit v L v_L v L . The identity ⟨ g j , v j ⟩ = Γ j \langle g_j,v_j\rangle=\Gamma_j ⟨ g j , v j ⟩ = Γ j follows from Lemma D.5 (a).
One parameter choice satisfying all requirements
Proposition E.5 (Parameters satisfying all phase conditions). Let c 0 , c 1 c_0,c_1 c 0 , c 1 be the constants in Theorem D.11 in Appx. D, and set
a q : = 3 q − 2 2 , K : = 5 q − 1 2 c 0 , η : = γ 2 ( 1 + θ ) . a_q:=\frac{3q-2}{2},\qquad K:=\frac{5^{\frac{q-1}{2}}}{c_0},\qquad\eta:=\frac{\gamma}{2(1+\theta)}. a q := 2 3 q − 2 , K := c 0 5 2 q − 1 , η := 2 ( 1 + θ ) γ .
These constants depend only on p p p , ν \nu ν , and 0 < η < 1 0 < \eta< 1 0 < η < 1 . For every m m m , L ∈ N L \in\mathbb{N} L ∈ N and μ , τ 0 > 0 \mu,\tau_0 > 0 μ , τ 0 > 0 , use Definition E.1 and choose
λ : = K μ m a q , α : = λ m − a q = K μ , δ j : = c 1 λ m τ j q . \lambda:= K\mu m^{a_q}, \qquad\alpha:= \lambda m^{-a_q} = K\mu, \qquad\delta_j := c_1\lambda m\tau_j^q. λ := K μ m a q , α := λ m − a q = K μ , δ j := c 1 λm τ j q .
Parts (a)–(b) hold for every 0 ≤ j < L 0 \le j < L 0 ≤ j < L :
(a) Base gradient.
G j ≤ c 0 Γ j , Γ j ≤ s j ≤ ( 1 + c 0 ) Γ j . (153) G_j \le c_0\Gamma_j, \qquad\Gamma_j \le s_j \le(1+c_0)\Gamma_j. \tag*{(153)} G j ≤ c 0 Γ j , Γ j ≤ s j ≤ ( 1 + c 0 ) Γ j . (153)
(b) Next-coordinate bounds.
α − 1 q − 1 s j + 1 q q − 1 ≤ c 0 λ m τ j q , (154) \alpha^{-\frac{1}{q-1}}s_{j+1}^{\frac{q}{q-1}} \le c_0\lambda m\tau_j^q, \tag*{(154)} α − q − 1 1 s j + 1 q − 1 q ≤ c 0 λm τ j q , (154)
s j + 1 t j + 1 ∗ + α q ( t j + 1 ∗ ) q ≤ c 0 λ m τ j q . (155) s_{j+1}t_{j+1}^{*}+\frac{\alpha}{q}(t_{j+1}^{*})^q \le c_0\lambda m\tau_j^q. \tag*{(155)} s j + 1 t j + 1 ∗ + q α ( t j + 1 ∗ ) q ≤ c 0 λm τ j q . (155)
(c) Nested base supports.
0 < η = γ 2 ( 1 + θ ) , S ‾ j + 1 ⊆ B ‾ ( c j + 1 , γ R j 2 ) ⊂ U j ( 0 ≤ j < L − 1 ) . 0 < \eta= \frac{\gamma}{2(1+\theta)}, \qquad\overline{S}_{j+1} \subseteq\overline{B}\left(c_{j+1},\frac{\gamma R_j}{2}\right) \subset U_j \quad(0 \le j < L-1). 0 < η = 2 ( 1 + θ ) γ , S j + 1 ⊆ B ( c j + 1 , 2 γ R j ) ⊂ U j ( 0 ≤ j < L − 1 ) .
The base-gradient estimate
Corollary E.6 (The coefficient required for a block of length m m m ). Suppose G j = μ R j q − 1 G_j=\mu R_j^{q-1} G j = μ R j q − 1 , and choose
λ = K μ m 3 q − 2 2 , K ≥ 5 q − 1 2 c 0 . \lambda= K\mu m^{\frac{3q-2}{2}}, \qquad K \ge\frac{5^{\frac{q-1}{2}}}{c_0}. λ = K μ m 2 3 q − 2 , K ≥ c 0 5 2 q − 1 . Then the base increment satisfies G j ≤ c 0 Γ j G_j \le c_0\Gamma_j G j ≤ c 0 Γ j .
Proof. Substitute R j = β m τ j m 3 2 R_j=\beta_m\tau_jm^{\frac{3}{2}} R j = β m τ j m 2 3 from Definition E.1 :
G j Γ j = μ ( β m τ j m 3 2 ) q − 1 λ τ j q − 1 m − 1 2 \frac{G_j}{\Gamma_j}
= \frac{\mu\left(\beta_m\tau_jm^{\frac{3}{2}}\right)^{q-1}}{\lambda\tau_j^{q-1}m^{-\frac{1}{2}}} Γ j G j = λ τ j q − 1 m − 2 1 μ ( β m τ j m 2 3 ) q − 1 by Definition E.1 ,
= μ λ β m q − 1 τ j q − 1 τ j q − 1 m 3 ( q − 1 ) 2 + 1 2 = \frac{\mu}{\lambda}\beta_m^{q-1}\frac{\tau_j^{q-1}}{\tau_j^{q-1}}m^{\frac{3(q-1)}{2}+\frac{1}{2}} = λ μ β m q − 1 τ j q − 1 τ j q − 1 m 2 3 ( q − 1 ) + 2 1 = μ λ β m q − 1 m 3 q − 2 2 = \frac{\mu}{\lambda}\beta_m^{q-1}m^{\frac{3q-2}{2}} = λ μ β m q − 1 m 2 3 q − 2 since 3 ( q − 1 ) + 1 = 3 q − 2 3(q-1)+1=3q-2 3 ( q − 1 ) + 1 = 3 q − 2 .
With λ = K μ m 3 q − 2 2 \lambda=K\mu m^{\frac{3q-2}{2}} λ = K μ m 2 3 q − 2 and β m ≤ 5 \beta_m\le\sqrt{5} β m ≤ 5 from Lemma D.3 (c) in Appendix § 10 ,
G j Γ j ≤ μ λ 5 q − 1 2 m 3 q − 2 2 = 5 q − 1 2 K ≤ c 0 . \frac{G_j}{\Gamma_j}\le\frac{\mu}{\lambda}5^{\frac{q-1}{2}}m^{\frac{3q-2}{2}}=\frac{5^{\frac{q-1}{2}}}{K}\le c_0. Γ j G j ≤ λ μ 5 2 q − 1 m 2 3 q − 2 = K 5 2 q − 1 ≤ c 0 .
The two next-coordinate estimates
Lemma E.7 (A common coefficient for the two next-center bounds). Fix a nonterminal phase 0 ≤ j < L − 1 0 \le j < L - 1 0 ≤ j < L − 1 . Assume
G j ≤ c 0 Γ j , s j = Γ j + G j t j ⋆ R j , α = λ m 1 − 3 q 2 , G_j \le c_0\Gamma_j,\qquad s_j = \Gamma_j + \frac{G_jt_j^\star}{R_j},\qquad\alpha= \lambda m^{1-\frac{3q}{2}}, G j ≤ c 0 Γ j , s j = Γ j + R j G j t j ⋆ , α = λ m 1 − 2 3 q , s j + 1 = η q − 1 s j , t j + 1 ⋆ = η t j ⋆ , s_{j+1} = \eta^{q-1}s_j,\qquad t_{j+1}^\star= \eta t_j^\star, s j + 1 = η q − 1 s j , t j + 1 ⋆ = η t j ⋆ , η q ( 1 + c 0 ) q ⋆ ≤ c 0 , η q ( 2 ( 1 + c 0 ) + 2 q q ) ≤ c 0 . \eta^q(1+c_0)^{q^\star} \le c_0,\qquad\eta^q\left(2(1+c_0)+\frac{2^q}{q}\right) \le c_0. η q ( 1 + c 0 ) q ⋆ ≤ c 0 , η q ( 2 ( 1 + c 0 ) + q 2 q ) ≤ c 0 . The restrictions on η \eta η depend only on q q q . Then:
(a) Dual bound Eq. Eq. (D.29).
α − 1 q − 1 s j + 1 q ⋆ ≤ c 0 E j . \alpha^{-\frac{1}{q-1}}s_{j+1}^{q^\star} \le c_0\mathcal{E}_j. α − q − 1 1 s j + 1 q ⋆ ≤ c 0 E j . (b) Endpoint bound Eq. Eq. (D.26).
s j + 1 t j + 1 ⋆ + α q ( t j + 1 ⋆ ) q ≤ c 0 E j . s_{j+1}t_{j+1}^\star+\frac{\alpha}{q}(t_{j+1}^\star)^q \le c_0\mathcal{E}_j. s j + 1 t j + 1 ⋆ + q α ( t j + 1 ⋆ ) q ≤ c 0 E j .
Proof. • Proof of (a). Substitute α = λ m 1 − 3 q 2 \alpha= \lambda m^{1-\frac{3q}{2}} α = λ m 1 − 2 3 q from Eq. (D.30) and Γ j = λ τ j q − 1 m \Gamma_j = \frac{\lambda\tau_j^{q-1}}{\sqrt{m}} Γ j = m λ τ j q − 1 :
α − 1 q − 1 Γ j q ⋆ = ( λ m 1 − 3 q 2 ) − 1 q − 1 ( λ τ j q − 1 m − 1 2 ) q q − 1 = λ − 1 q − 1 + q q − 1 τ j ( q − 1 ) q q − 1 m 3 q 2 ( q − 1 ) − q 2 ( q − 1 ) = λ τ j q m q q − 1 = λ m τ j q = E j . (156) \begin{aligned}
\alpha^{-\frac{1}{q-1}}\Gamma_j^{q^\star}
&=\left(\lambda m^{1-\frac{3q}{2}}\right)^{-\frac{1}{q-1}}\left(\lambda\tau_j^{q-1}m^{-\frac{1}{2}}\right)^{\frac{q}{q-1}}\\
&=\lambda^{-\frac{1}{q-1}+\frac{q}{q-1}}\tau_j^{(q-1)\frac{q}{q-1}}m^{\frac{3q}{2(q-1)}-\frac{q}{2(q-1)}}\\
&=\lambda\tau_j^q m^{\frac{q}{q-1}}\\
&=\lambda m\tau_j^q=\mathcal{E}_j. \tag*{(156)}
\end{aligned} α − q − 1 1 Γ j q ⋆ = ( λ m 1 − 2 3 q ) − q − 1 1 ( λ τ j q − 1 m − 2 1 ) q − 1 q = λ − q − 1 1 + q − 1 q τ j ( q − 1 ) q − 1 q m 2 ( q − 1 ) 3 q − 2 ( q − 1 ) q = λ τ j q m q − 1 q = λm τ j q = E j . (156)
The third line uses − 1 + q = q − 1 -1+q=q-1 − 1 + q = q − 1 for the power of λ \lambda λ and 3 q 2 − 1 − q 2 = q − 1 \frac{3q}{2}-1-\frac{q}{2}=q-1 2 3 q − 1 − 2 q = q − 1 for the power of m m m .
By Lemma D.3 (c) in Appx. D, 0 < t j ⋆ R j ≤ 1 0 < \frac{t_j^\star}{R_j} \le1 0 < R j t j ⋆ ≤ 1 , so
Γ j ≤ s j = Γ j + G j t j ⋆ R j ≤ Γ j + G j ≤ ( 1 + c 0 ) Γ j . \Gamma_j \le s_j = \Gamma_j+\frac{G_jt_j^\star}{R_j}\le\Gamma_j+G_j\le(1+c_0)\Gamma_j. Γ j ≤ s j = Γ j + R j G j t j ⋆ ≤ Γ j + G j ≤ ( 1 + c 0 ) Γ j .
Using s j + 1 = η q − 1 s j s_{j+1}=\eta^{q-1}s_j s j + 1 = η q − 1 s j and Eq. (E.10),
α − 1 q − 1 s j + 1 q ⋆ = η ( q − 1 ) q ⋆ α − 1 q − 1 s j q ⋆ ≤ η q ( 1 + c 0 ) q ⋆ α − 1 q − 1 Γ j q ⋆ since ( q − 1 ) q ⋆ = q , = η q ( 1 + c 0 ) q ⋆ E j by Eq. (E.10) , ≤ c 0 E j by the first bound on η . \begin{aligned}
\alpha^{-\frac{1}{q-1}}s_{j+1}^{q^\star}
&=\eta^{(q-1)q^\star}\alpha^{-\frac{1}{q-1}}s_j^{q^\star}\\
&\le\eta^q(1+c_0)^{q^\star}\alpha^{-\frac{1}{q-1}}\Gamma_j^{q^\star}
&&\text{since }(q-1)q^\star=q,\\
&=\eta^q(1+c_0)^{q^\star}\mathcal{E}_j
&&\text{by Eq. (E.10)},\\
&\le c_0\mathcal{E}_j
&&\text{by the first bound on }\eta.
\end{aligned} α − q − 1 1 s j + 1 q ⋆ = η ( q − 1 ) q ⋆ α − q − 1 1 s j q ⋆ ≤ η q ( 1 + c 0 ) q ⋆ α − q − 1 1 Γ j q ⋆ = η q ( 1 + c 0 ) q ⋆ E j ≤ c 0 E j since ( q − 1 ) q ⋆ = q , by Eq. (E.10) , by the first bound on η .
• Proof of (b). The bound s j ≤ ( 1 + c 0 ) Γ j s_j\le(1+c_0)\Gamma_j s j ≤ ( 1 + c 0 ) Γ j proved in part (a), and the definitions of Γ j \Gamma_j Γ j , t j ⋆ t_j^\star t j ⋆ , α \alpha α , E j \mathcal{E}_j E j , give
Γ j t j ⋆ E j = ( λ j q − 1 m ) ( m ( m + 1 ) τ j ) λ m τ j q = m + 1 m ≤ 2 , \begin{aligned}
\frac{\Gamma_j t_j^\star}{\mathcal{E}_j}
= \frac{\left(\frac{\lambda_j^{q-1}}{\sqrt{m}}\right)\left(\sqrt{m}(m+1)\tau_j\right)}{\lambda m \tau_j^q} \\
&= \frac{m+1}{m} \le2,
\end{aligned} E j Γ j t j ⋆ = λm τ j q ( m λ j q − 1 ) ( m ( m + 1 ) τ j ) = m m + 1 ≤ 2 ,
α ( t j ⋆ ) q E j = λ m 1 − 3 q 2 m q 2 ( m + 1 ) q τ j q λ m τ j q = m − q ( m + 1 ) q = ( m + 1 m ) q ≤ 2 q . \begin{aligned}
\frac{\alpha\left(t_j^\star\right)^q}{\mathcal{E}_j}
= \frac{\lambda m^{1-\frac{3q}{2}}m^{\frac{q}{2}}(m+1)^q\tau_j^q}{\lambda m\tau_j^q} \\
&= m^{-q}(m+1)^q
= \left(\frac{m+1}{m}\right)^q \le2^q.
\end{aligned} E j α ( t j ⋆ ) q = λm τ j q λ m 1 − 2 3 q m 2 q ( m + 1 ) q τ j q = m − q ( m + 1 ) q = ( m m + 1 ) q ≤ 2 q .
Since t j + 1 ⋆ = η t j ⋆ t_{j+1}^\star=\eta t_j^\star t j + 1 ⋆ = η t j ⋆ , it follows that
s j + 1 t j + 1 ⋆ + α q ( t j + 1 ⋆ ) q = η q ( s j t j ⋆ + α q ( t j ⋆ ) q ) ≤ η q ( ( 1 + c 0 ) Γ j t j ⋆ + α q ( t j ⋆ ) q ) ≤ η q ( 2 ( 1 + c 0 ) + 2 q q ) E j ≤ c 0 E j . \begin{aligned}
s_{j+1}t_{j+1}^\star+\frac{\alpha}{q}\left(t_{j+1}^\star\right)^q
= \eta^q\left(s_jt_j^\star+\frac{\alpha}{q}\left(t_j^\star\right)^q\right) \\
&\le\eta^q\left((1+c_0)\Gamma_jt_j^\star+\frac{\alpha}{q}\left(t_j^\star\right)^q\right) \\
&\le\eta^q\left(2(1+c_0)+\frac{2^q}{q}\right)\mathcal{E}_j \\
&\le c_0\mathcal{E}_j.
\end{aligned} s j + 1 t j + 1 ⋆ + q α ( t j + 1 ⋆ ) q = η q ( s j t j ⋆ + q α ( t j ⋆ ) q ) ≤ η q ( ( 1 + c 0 ) Γ j t j ⋆ + q α ( t j ⋆ ) q ) ≤ η q ( 2 ( 1 + c 0 ) + q 2 q ) E j ≤ c 0 E j .
The second line uses the bound on s j s_j s j , the third uses the two ratios just proved, and the last uses the second hypothesis on η \eta η .
Verify the common parameter choice
Proof of Proposition E.5 . • Proof of (a). Corollary E.6 gives G j ≤ c 0 Γ j G_j\le c_0\Gamma_j G j ≤ c 0 Γ j . Since 0 < t j ⋆ R j ≤ 1 0<\frac{t_j^\star}{R_j}\le1 0 < R j t j ⋆ ≤ 1 ,
Γ j ≤ s j = Γ j + G j t j ⋆ R j ≤ Γ j + G j ≤ ( 1 + c 0 ) Γ j . \Gamma_j\le s_j=\Gamma_j+\frac{G_jt_j^\star}{R_j}\le\Gamma_j+G_j\le(1+c_0)\Gamma_j. Γ j ≤ s j = Γ j + R j G j t j ⋆ ≤ Γ j + G j ≤ ( 1 + c 0 ) Γ j .
• Proof of (b). The choice α = λ m − a q = λ m 1 − 3 q 2 \alpha=\lambda m^{-a_q}=\lambda m^{1-\frac{3q}{2}} α = λ m − a q = λ m 1 − 2 3 q is exactly Eq. (D.30). For j < L − 1 j<L-1 j < L − 1 , Definition E.1 gives
G j + 1 = η q − 1 G j , Γ j + 1 = η q − 1 Γ j , t j + 1 ⋆ = η t j ⋆ , R j + 1 = η R j . G_{j+1}=\eta^{q-1}G_j,\qquad
\Gamma_{j+1}=\eta^{q-1}\Gamma_j,\qquad
t_{j+1}^\star=\eta t_j^\star,\qquad
R_{j+1}=\eta R_j. G j + 1 = η q − 1 G j , Γ j + 1 = η q − 1 Γ j , t j + 1 ⋆ = η t j ⋆ , R j + 1 = η R j .
Substituting all four relations into the definition of s j + 1 s_{j+1} s j + 1 gives
s j + 1 = Γ j + 1 + G j + 1 t j + 1 ⋆ R j + 1 = η q − 1 Γ j + η q − 1 G j η t j ⋆ η R j = η q − 1 ( Γ j + G j t j ⋆ R j ) = η q − 1 s j . \begin{aligned}
s_{j+1}=\Gamma_{j+1}+G_{j+1}\frac{t_{j+1}^\star}{R_{j+1}} \\
&=\eta^{q-1}\Gamma_j+\eta^{q-1}G_j\frac{\eta t_j^\star}{\eta R_j} \\
&=\eta^{q-1}\left(\Gamma_j+G_j\frac{t_j^\star}{R_j}\right)=\eta^{q-1}s_j.
\end{aligned} s j + 1 = Γ j + 1 + G j + 1 R j + 1 t j + 1 ⋆ = η q − 1 Γ j + η q − 1 G j η R j η t j ⋆ = η q − 1 ( Γ j + G j R j t j ⋆ ) = η q − 1 s j .
This verifies the scaling assumptions of Lemma E.7. It remains to check its two bounds on η \eta η . Lemma C.11 in Appx. C gives γ < θ 2 \gamma< \frac{\theta}{2} γ < 2 θ and θ ≤ 1 16 \theta\le\frac{1}{16} θ ≤ 16 1 , so
η = γ 2 ( 1 + θ ) < γ 2 < θ 4 ≤ 1 64 < 1 32 . \eta= \frac{\gamma}{2(1+\theta)} < \frac{\gamma}{2} < \frac{\theta}{4} \le\frac{1}{64} < \frac{1}{32}. η = 2 ( 1 + θ ) γ < 2 γ < 4 θ ≤ 64 1 < 32 1 .
Also c 0 = 2 q − 3 q < 1 c_0 = \frac{2^{q-3}}{q} < 1 c 0 = q 2 q − 3 < 1 by Eq. (D.24), and q ∗ < 2 q^* < 2 q ∗ < 2 . Hence
( 1 + c 0 ) q ∗ ≤ 4 ≤ 2 q + 1 , 2 ( 1 + c 0 ) + 2 q q ≤ 4 + 2 q q ≤ 2 q + 1 . (1+c_0)^{q^*} \le4 \le2^{q+1}, \qquad2(1+c_0) + \frac{2^q}{q} \le4 + \frac{2^q}{q} \le2^{q+1}. ( 1 + c 0 ) q ∗ ≤ 4 ≤ 2 q + 1 , 2 ( 1 + c 0 ) + q 2 q ≤ 4 + q 2 q ≤ 2 q + 1 .
Both hypotheses of Lemma E.7 follow from η < 2 − 5 \eta< 2^{-5} η < 2 − 5 , c 0 − 1 = q 2 q + 3 c_0^{-1} = q2^{q+3} c 0 − 1 = q 2 q + 3 , and
η q c 0 max { ( 1 + c 0 ) q ∗ , 2 ( 1 + c 0 ) + 2 q q } ≤ 2 − 5 q q 2 q + 3 2 q + 1 = q 2 4 − 3 q ≤ 2 3 − 2 q < 1. \frac{\eta^q}{c_0}\max\left\{(1+c_0)^{q^*},\,2(1+c_0)+\frac{2^q}{q}\right\}
\le2^{-5q}q2^{q+3}2^{q+1}=q2^{4-3q}\le2^{3-2q}<1. c 0 η q max { ( 1 + c 0 ) q ∗ , 2 ( 1 + c 0 ) + q 2 q } ≤ 2 − 5 q q 2 q + 3 2 q + 1 = q 2 4 − 3 q ≤ 2 3 − 2 q < 1.
The penultimate inequality uses q ≤ 2 q − 1 q \le2^{q-1} q ≤ 2 q − 1 for q ≥ 2 q \ge2 q ≥ 2 . Apply Lemma E.7 (a)–(b) to obtain Eq. (E.8)–(E.9). For j = L − 1 j=L-1 j = L − 1 , s L = t L ∗ = 0 s_L=t_L^*=0 s L = t L ∗ = 0 , so the left sides of Eq. (E.8) and (E.9) are zero.
Define the complete objective
Definition E.8 (The complete objective). With the parameters of Proposition E.5 , define
ℓ i n i t ( x ) : = − s 0 ⟨ v 0 , x ⟩ , F ( x ) : = B ( x ) + ℓ i n i t ( x ) + ∑ j = 0 L − 1 A j ( x ) . (157) \ell_{\mathrm{init}}(x):=-s_0\langle v_0,x\rangle,\qquad F(x):=B(x)+\ell_{\mathrm{init}}(x)+\sum_{j=0}^{L-1}A_j(x). \tag*{(157)} ℓ init ( x ) := − s 0 ⟨ v 0 , x ⟩ , F ( x ) := B ( x ) + ℓ init ( x ) + j = 0 ∑ L − 1 A j ( x ) . (157) Here B = B L B=B_L B = B L , and each A j A_j A j is the phase function of Appx. D.1.3 evaluated at the local coordinates
w = P E j ( x − c j ) = t j ( x ) v j + ∑ i = 1 m x j , i ( x ) u j , i . w=P_{E_j}(x-c_j)=t_j(x)v_j+\sum_{i=1}^{m}x_{j,i}(x)u_{j,i}. w = P E j ( x − c j ) = t j ( x ) v j + i = 1 ∑ m x j , i ( x ) u j , i . where, for x ∈ R d x\in\mathbb{R}^d x ∈ R d ,
t j ( x ) : = ⟨ v j , x − c j ⟩ , t_j(x):=\langle v_j,x-c_j\rangle, t j ( x ) := ⟨ v j , x − c j ⟩ , x j , i ( x ) : = ⟨ u j , i , x − c j ⟩ ( 1 ≤ i ≤ m ) . x_{j,i}(x):=\langle u_{j,i},x-c_j\rangle\qquad(1\le i\le m). x j , i ( x ) := ⟨ u j , i , x − c j ⟩ ( 1 ≤ i ≤ m ) . For j < L − 1 j<L-1 j < L − 1 , orthogonality of h j h_j h j and v j + 1 v_{j+1} v j + 1 gives t j + 1 ( x ) = ⟨ v j + 1 , x − c j + 1 ⟩ = ⟨ v j + 1 , x − c j ⟩ t_{j+1}(x)=\langle v_{j+1},x-c_{j+1}\rangle=\langle v_{j+1},x-c_j\rangle t j + 1 ( x ) = ⟨ v j + 1 , x − c j + 1 ⟩ = ⟨ v j + 1 , x − c j ⟩ . We retain P j , I j , O j , A j P_j,I_j,O_j,A_j P j , I j , O j , A j for the ambient functions obtained from the local inputs ( w , t j + 1 ( x ) ) (w,t_{j+1}(x)) ( w , t j + 1 ( x )) in Eq. (17) . Their formulas are
P j ( x ) : = λ q ( ∣ t j ( x ) m − x j , 1 ( x ) ∣ q + ∑ i = 1 m − 1 ∣ x j , i ( x ) − x j , i + 1 ( x ) ∣ q + ∣ x j , m ( x ) ∣ q ) + α q ∣ t j + 1 ( x ) ∣ q , P_j(x):=\frac{\lambda}{q}\left(\left|\frac{t_j(x)}{\sqrt{m}}-x_{j,1}(x)\right|^q+\sum_{i=1}^{m-1}|x_{j,i}(x)-x_{j,i+1}(x)|^q+|x_{j,m}(x)|^q\right)+\frac{\alpha}{q}|t_{j+1}(x)|^q, P j ( x ) := q λ ( m t j ( x ) − x j , 1 ( x ) q + i = 1 ∑ m − 1 ∣ x j , i ( x ) − x j , i + 1 ( x ) ∣ q + ∣ x j , m ( x ) ∣ q ) + q α ∣ t j + 1 ( x ) ∣ q , I j ( x ) : = ⟨ g j , x − c j + 1 ⟩ , I_j(x):=\langle g_j,x-c_{j+1}\rangle, I j ( x ) := ⟨ g j , x − c j + 1 ⟩ , O j ( x ) : = P j ( x ) − P j ( c j + 1 ) − 4 δ j , O_j(x):=P_j(x)-P_j(c_{j+1})-4\delta_j, O j ( x ) := P j ( x ) − P j ( c j + 1 ) − 4 δ j , A j ( x ) : = I j ( x ) + ψ δ j ( O j ( x ) − I j ( x ) ) . A_j(x):=I_j(x)+\psi_{\delta_j}(O_j(x)-I_j(x)). A j ( x ) := I j ( x ) + ψ δ j ( O j ( x ) − I j ( x )) .
Here P j ( c j + 1 ) = λ ( m + 1 ) t j q q P_j(c_{j+1}) = \frac{\lambda(m+1)t_j^q}{q} P j ( c j + 1 ) = q λ ( m + 1 ) t j q and A j = M δ j ( I j , O j ) A_j = M_{\delta_j}(I_j,O_j) A j = M δ j ( I j , O j ) . Since g j ∈ E j ⊕ span { v j + 1 } g_j \in E_j \oplus\operatorname{span}\{v_{j+1}\} g j ∈ E j ⊕ span { v j + 1 } , with the last summand omitted in the terminal phase, the displayed I j I_j I j agrees with the local affine function under the same coordinate map. The last phase omits t L t_L t L and its term; when m = 1 m=1 m = 1 , the sum over i i i is empty.
Orthogonality gives the following local inputs:
Table 2.
For the terminal phase the last coordinate and last row are omitted. The endpoint value of the first coordinate is t j ( c j + 1 ) = ⟨ v j , h j ⟩ = t j ⋆ t_j(c_{j+1})=\langle v_j,h_j\rangle=t_j^\star t j ( c j + 1 ) = ⟨ v j , h j ⟩ = t j ⋆ .
Verification of the complete objective
Retain uniform convexity and smoothness
Lemma E.9 (Uniform convexity of the complete objective). The complete objective F F F in Eq. (E.11) satisfies
D F ( y , x ) ≥ c B μ ∥ y − x ∥ q , c B = 1 2 q ⋅ 16 q − 2 . D_F(y,x) \ge c_B\mu\lVert y-x\rVert^q,\qquad c_B=\frac{1}{2q\cdot16^{q-2}}. D F ( y , x ) ≥ c B μ ∥ y − x ∥ q , c B = 2 q ⋅ 1 6 q − 2 1 . Proof. Theorem D.11 (a) gives convexity in the local phase coordinates. Since the coordinate map is affine, Lemma B.19 (d) in Appx. B makes each ambient A j A_j A j convex. Lemma B.20 applies to the convex sum G = ∑ j A j G=\sum_j A_j G = ∑ j A j and affine term ℓ i n i t \ell_{\mathrm{init}} ℓ init . Together with Lemma C.4 (a) in Appx. C, it gives
D F ( y , x ) = D B ( y , x ) + D ℓ i n i t ( y , x ) + ∑ j D A j ( y , x ) D_F(y,x)=D_B(y,x)+D_{\ell_{\mathrm{init}}}(y,x)+\sum_jD_{A_j}(y,x) D F ( y , x ) = D B ( y , x ) + D ℓ init ( y , x ) + j ∑ D A j ( y , x ) by Eq. (E.11) and the Bregman definition,
= D B ( y , x ) + ∑ j D A j ( y , x ) =D_B(y,x)+\sum_jD_{A_j}(y,x) = D B ( y , x ) + j ∑ D A j ( y , x ) since ℓ i n i t \ell_{\mathrm{init}} ℓ init is affine,
≥ D B ( y , x ) \ge D_B(y,x) ≥ D B ( y , x ) by Lemma B.20 ,
≥ c B μ ∥ y − x ∥ q \ge c_B\mu\lVert y-x\rVert^q ≥ c B μ ∥ y − x ∥ q by Lemma C.4 (a).
The modulus is therefore at least q c B μ qc_B\mu q c B μ under the uniform-convexity convention of Theorem 1.1 . □
Lemma E.10 (Smoothness independent of the number of phases). For F F F in Eq. (E.11), we have F ∈ C p , ν ( R d ) F\in C^{p,\nu}(\mathbb{R}^d) F ∈ C p , ν ( R d ) and, for all x , y ∈ R d x,y\in\mathbb{R}^d x , y ∈ R d ,
∥ ∇ p F ( x ) − ∇ p F ( y ) ∥ ≤ C H μ m 3 q − 2 2 ∥ x − y ∥ ν . \lVert\nabla^pF(x)-\nabla^pF(y)\rVert\le C_H\mu m^{\frac{3q-2}{2}}\lVert x-y\rVert^\nu. ∥ ∇ p F ( x ) − ∇ p F ( y )∥ ≤ C H μ m 2 3 q − 2 ∥ x − y ∥ ν . The constant C H : = C B + 2 C A K C_H:=C_B+2C_AK C H := C B + 2 C A K depends only on p , ν p,\nu p , ν .
Proof. All terms are C p C^p C p by Lemma C.1 (a) in Appx. C and Theorem D.11 (e) in Appx. D. Set
V j = E j ⊕ span { v j + 1 } ( j < L − 1 ) , V L − 1 = E L − 1 , Π j = P V j . V_j=E_j\oplus\operatorname{span}\{v_{j+1}\}\quad(j<L-1),\qquad V_{L-1}=E_{L-1},\qquad\Pi_j=P_{V_j}. V j = E j ⊕ span { v j + 1 } ( j < L − 1 ) , V L − 1 = E L − 1 , Π j = P V j . Then ∣ j − k ∣ ≥ 2 ⇒ V j ⊥ V k |j-k|\ge2\Rightarrow V_j\perp V_k ∣ j − k ∣ ≥ 2 ⇒ V j ⊥ V k . Thus each parity class J = { j : 0 ≤ j < L , j ≡ a ( m o d 2 ) } J=\{j:0\le j<L,\ j\equiv a\pmod2\} J = { j : 0 ≤ j < L , j ≡ a ( mod 2 )} , a ∈ { 0 , 1 } a\in\{0,1\} a ∈ { 0 , 1 } , is orthogonal.
Fix x x x , y y y and a nonempty parity class J J J . Define the tensor on V j V_j V j by
Q j : = ( ∇ p A j ( x ) − ∇ p A j ( y ) ) ∣ V j p . Q_j := \left.(\nabla^p A_j(x)-\nabla^p A_j(y))\right|_{V_j^p}. Q j := ( ∇ p A j ( x ) − ∇ p A j ( y )) ∣ V j p .
The linear chain rule and the phase bound in Theorem D.11 (e) give
( ∇ p A j ( x ) − ∇ p A j ( y ) ) [ z 1 , … , z p ] = Q j [ Π j z 1 , … , Π j z p ] , (\nabla^p A_j(x)-\nabla^p A_j(y))[z_1,\ldots,z_p]=Q_j[\Pi_jz_1,\ldots,\Pi_jz_p], ( ∇ p A j ( x ) − ∇ p A j ( y )) [ z 1 , … , z p ] = Q j [ Π j z 1 , … , Π j z p ] ,
∥ Q j ∥ ≤ C A λ ∥ Π j ( x − y ) ∥ ν . \lVert Q_j\rVert\le C_A\lambda\lVert\Pi_j(x-y)\rVert^\nu. ∥ Q j ∥ ≤ C A λ ∥ Π j ( x − y ) ∥ ν .
Each tensor acts only on V j V_j V j , and these subspaces are orthogonal within J J J . Lemma B.10 in Appx. B therefore controls their sum by the largest phase contribution, avoiding an L L L factor. Applied with order p p p and tensors Q j Q_j Q j on V j p V_j^p V j p , it gives
∥ ∑ j ∈ J ( ∇ p A j ( x ) − ∇ p A j ( y ) ) ∥ = max j ∈ J ∥ Q j ∥ ≤ C A λ max j ∈ J ∥ Π j ( x − y ) ∥ ν ≤ C A λ ∥ x − y ∥ ν . \begin{aligned}
\left\lVert\sum_{j\in J}(\nabla^p A_j(x)-\nabla^p A_j(y))\right\rVert
=\max_{j\in J}\lVert Q_j\rVert\\
&\le C_A\lambda\max_{j\in J}\lVert\Pi_j(x-y)\rVert^\nu\\
&\le C_A\lambda\lVert x-y\rVert^\nu.
\end{aligned} j ∈ J ∑ ( ∇ p A j ( x ) − ∇ p A j ( y )) = j ∈ J max ∥ Q j ∥ ≤ C A λ j ∈ J max ∥ Π j ( x − y ) ∥ ν ≤ C A λ ∥ x − y ∥ ν .
For J = ∅ J=\varnothing J = ∅ , the sum is 0 and satisfies the same bound. Adding the two parity bounds and the base bound from Lemma C.2 (a), with ∇ p ℓ i n i t = 0 \nabla^p\ell_{\mathrm{init}}=0 ∇ p ℓ init = 0 , yields
∥ ∇ p F ( x ) − ∇ p F ( y ) ∥ ≤ ( C B μ + 2 C A λ ) ∥ x − y ∥ ν = ( C B μ + 2 C A K μ a q ) ∥ x − y ∥ ν ≤ ( C B + 2 C A K ) μ a q ∥ x − y ∥ ν ( μ a q ≥ 1 ) , = C H μ a q ∥ x − y ∥ ν . (158) \begin{aligned}
\lVert\nabla^pF(x)-\nabla^pF(y)\rVert
\le(C_B\mu+2C_A\lambda)\lVert x-y\rVert^\nu\\
&= (C_B\mu+2C_AK\mu^{aq})\lVert x-y\rVert^\nu\\
&\le(C_B+2C_AK)\mu^{aq}\lVert x-y\rVert^\nu
\qquad(\mu^{aq}\ge1), \\
&= C_H\mu^{aq}\lVert x-y\rVert^\nu. \tag*{(158)}
\end{aligned} ∥ ∇ p F ( x ) − ∇ p F ( y )∥ ≤ ( C B μ + 2 C A λ ) ∥ x − y ∥ ν = ( C B μ + 2 C A K μ a q ) ∥ x − y ∥ ν ≤ ( C B + 2 C A K ) μ a q ∥ x − y ∥ ν ( μ a q ≥ 1 ) , = C H μ a q ∥ x − y ∥ ν . (158)
Verify the center gradients and identify the minimizer
The affine slopes telescope in the center gradients. These gradients also provide the orthogonality used in the prefix-gap estimate.
Lemma E.11 (All center gradients and the minimizer). For the objective in Definition E.8 and 0 ≤ k < L 0\le k<L 0 ≤ k < L , 0 ≤ j ≤ L 0\le j\le L 0 ≤ j ≤ L :
(a) Phase derivatives.
∇ A k ( c j ) = { g k , k < j , 0 , k ≥ j , ∇ ℓ A k ( c j ) = 0 ( 2 ≤ ℓ ≤ p ) . \nabla A_k(c_j)=
\begin{cases}
g_k, & k<j,\\
0, & k\ge j,
\end{cases}
\qquad
\nabla^\ell A_k(c_j)=0\quad(2\le\ell\le p). ∇ A k ( c j ) = { g k , 0 , k < j , k ≥ j , ∇ ℓ A k ( c j ) = 0 ( 2 ≤ ℓ ≤ p ) . (b) Full gradients.
∇ F ( c j ) = { − s j v j , j < L , 0 , j = L . \nabla F(c_j)=
\begin{cases}
-s_jv_j, & j<L,\\
0, & j=L.
\end{cases} ∇ F ( c j ) = { − s j v j , 0 , j < L , j = L . (c) Minimizer.
argmin x ∈ R d F ( x ) = { c L } . \underset{x\in\mathbb{R}^d}{\operatorname{argmin}}\,F(x)=\{c_L\}. x ∈ R d argmin F ( x ) = { c L } .
Proof. • Proof of (a). Definition E.1 and orthogonality give three cases for the arguments of A k A_k A k at c j c_j c j :
Omit the second coordinate for k = L − 1 k=L-1 k = L − 1 , as in Theorem D.11 (f) in Appx. D. Apply its origin formula (part (d)) and affine endpoint formula (part (c)):
k ≥ j : ∇ ℓ A k ( c j ) = 0 ( 1 ≤ ℓ ≤ p ) , k < j : A k = I k near c j , ∇ A k ( c j ) = g k , ∇ ℓ A k ( c j ) = 0 ( 2 ≤ ℓ ≤ p ) . \begin{aligned}
k\ge j &: \quad\nabla^\ell A_k(c_j)=0 &&(1\le\ell\le p),\\
k<j &: \quad A_k=I_k &&\text{near }c_j,\\
&\quad\nabla A_k(c_j)=g_k,\quad\nabla^\ell A_k(c_j)=0 &&(2\le\ell\le p).
\end{aligned} k ≥ j k < j : ∇ ℓ A k ( c j ) = 0 : A k = I k ∇ A k ( c j ) = g k , ∇ ℓ A k ( c j ) = 0 ( 1 ≤ ℓ ≤ p ) , near c j , ( 2 ≤ ℓ ≤ p ) .
• Proof of (b). For 1 ≤ j ≤ L 1\le j\le L 1 ≤ j ≤ L , use Lemma C.1 (c) in Appx. C, part (a), and Eq. (E.6):
∇ F ( c j ) = ∑ k = 0 j − 1 b k − s 0 v 0 + ∑ k = 0 j − 1 g k = − s 0 v 0 + ∑ k = 0 j − 1 ( b k + s k v k − s k + 1 v k + 1 − b k ) = − s 0 v 0 + ∑ k = 0 j − 1 s k v k − ∑ k = 0 j − 1 s k + 1 v k + 1 = − s 0 v 0 + ( s 0 v 0 + ∑ k = 1 j − 1 s k v k ) − ( ∑ k = 1 j − 1 s k v k + s j v j ) = − s j v j . \begin{aligned}
\nabla F(c_j)&=\sum_{k=0}^{j-1}b_k-s_0v_0+\sum_{k=0}^{j-1}g_k\\
&=-s_0v_0+\sum_{k=0}^{j-1}(b_k+s_kv_k-s_{k+1}v_{k+1}-b_k)\\
&=-s_0v_0+\sum_{k=0}^{j-1}s_kv_k-\sum_{k=0}^{j-1}s_{k+1}v_{k+1}\\
&=-s_0v_0+\left(s_0v_0+\sum_{k=1}^{j-1}s_kv_k\right)-\left(\sum_{k=1}^{j-1}s_kv_k+s_jv_j\right)\\
&=-s_jv_j.
\end{aligned} ∇ F ( c j ) = k = 0 ∑ j − 1 b k − s 0 v 0 + k = 0 ∑ j − 1 g k = − s 0 v 0 + k = 0 ∑ j − 1 ( b k + s k v k − s k + 1 v k + 1 − b k ) = − s 0 v 0 + k = 0 ∑ j − 1 s k v k − k = 0 ∑ j − 1 s k + 1 v k + 1 = − s 0 v 0 + ( s 0 v 0 + k = 1 ∑ j − 1 s k v k ) − ( k = 1 ∑ j − 1 s k v k + s j v j ) = − s j v j .
In the fourth line, the second sum is reindexed from k + 1 k+1 k + 1 to k k k . At j = L j=L j = L , the convention s L v L = 0 s_Lv_L=0 s L v L = 0 gives ∇ F ( c L ) = 0 \nabla F(c_L)=0 ∇ F ( c L ) = 0 . At j = 0 j=0 j = 0 , both sums in the first line are empty and ∇ B ( c 0 ) = 0 \nabla B(c_0)=0 ∇ B ( c 0 ) = 0 , giving ∇ F ( c 0 ) = − s 0 v 0 \nabla F(c_0)=-s_0v_0 ∇ F ( c 0 ) = − s 0 v 0 directly.
• Proof of (c). Lemma E.9 at c L c_L c L , together with the zero gradient in part (b), gives
F ( x ) ≥ F ( c L ) + c B μ ∥ x − c L ∥ q . F(x)\ge F(c_L)+c_B\mu\lVert x-c_L\rVert^q. F ( x ) ≥ F ( c L ) + c B μ ∥ x − c L ∥ q .
The right side is strictly larger than F ( c L ) F(c_L) F ( c L ) when x ≠ c L x\ne c_L x = c L .
A convex gap comparison
Lemma E.12 (A convex comparison proves an optimality gap). Let W ⊂ R d W\subset\mathbb{R}^d W ⊂ R d , e > 0 e>0 e > 0 , and c ∈ R d c\in\mathbb{R}^d c ∈ R d . Suppose F = Ψ + r F=\Psi+r F = Ψ + r , Ψ \Psi Ψ is convex and differentiable, and r = 0 r=0 r = 0 on a neighborhood of c c c . Assume also that r ( x ) ≥ e r(x)\ge e r ( x ) ≥ e and ⟨ ∇ F ( c ) , x − c ⟩ = 0 \langle\nabla F(c),x-c\rangle=0 ⟨ ∇ F ( c ) , x − c ⟩ = 0 for every x ∈ W x\in W x ∈ W . Then:
(a) Ψ ( c ) = F ( c ) \Psi(c)=F(c) Ψ ( c ) = F ( c ) and ∇ Ψ ( c ) = ∇ F ( c ) \nabla\Psi(c)=\nabla F(c) ∇Ψ ( c ) = ∇ F ( c ) .
(b) F ( x ) ≥ F ( c ) + e ≥ inf F + e F(x)\ge F(c)+e\ge\inf F+e F ( x ) ≥ F ( c ) + e ≥ inf F + e for every $x\in W. Proof. • Proof of (a). The neighborhood identity F = Ψ F = \Psi F = Ψ gives equality of values and gradients at c c c .
• Proof of (b). By the first-order convexity inequality in Lemma B.19 (a), for x ∈ W x \in W x ∈ W , part (a) and the two assumptions on W W W give
F ( x ) = Ψ ( x ) + r ( x ) by the decomposition, ≥ Ψ ( c ) + ⟨ ∇ Ψ ( c ) , x − c ⟩ + r ( x ) by xref‘Lemma B.19 (a)‘, = F ( c ) + ⟨ ∇ F ( c ) , x − c ⟩ + r ( x ) by part (a), ≥ F ( c ) + e by the two assumptions on W , ≥ inf F + e . \begin{aligned}
F(x) &= \Psi(x) + r(x) && \text{by the decomposition,}\\
&\ge\Psi(c) + \langle\nabla\Psi(c), x-c\rangle+ r(x) && \text{by {xref}`Lemma B.19 (a)`,}\\
&= F(c) + \langle\nabla F(c), x-c\rangle+ r(x) && \text{by part (a),}\\
&\ge F(c) + e && \text{by the two assumptions on } W,\\
&\ge\inf F + e.
\end{aligned} F ( x ) = Ψ ( x ) + r ( x ) ≥ Ψ ( c ) + ⟨ ∇Ψ ( c ) , x − c ⟩ + r ( x ) = F ( c ) + ⟨ ∇ F ( c ) , x − c ⟩ + r ( x ) ≥ F ( c ) + e ≥ inf F + e . by the decomposition, by xref ‘Lemma B.19 (a)‘, by part (a), by the two assumptions on W ,
Optimality gaps on short prefixes
For each 0 ≤ j < L 0 \le j < L 0 ≤ j < L and 0 ≤ b ≤ ⌊ m 4 ⌋ 0 \le b \le\left\lfloor\frac{m}{4} \right\rfloor 0 ≤ b ≤ ⌊ 4 m ⌋ , define the subspace containing all earlier phases and the selected part of phase j j j :
W j , b : = ( ⨁ k < j E k ) ⊕ span { v j , u j , 1 , … , u j , b } . W_{j,b} := \left(\bigoplus_{k<j} E_k\right) \oplus\operatorname{span}\{v_j,u_{j,1},\ldots,u_{j,b}\}. W j , b := k < j ⨁ E k ⊕ span { v j , u j , 1 , … , u j , b } .
Using A j A_j A j and I j I_j I j from Definition E.8, define
r j : = A j − I j , Ψ j : = F − A j + I j = B + ℓ i n i t + ∑ k ≠ j A k + I j . r_j := A_j-I_j,\qquad\Psi_j := F-A_j+I_j = B+\ell_{\mathrm{init}}+\sum_{k\ne j} A_k+I_j. r j := A j − I j , Ψ j := F − A j + I j = B + ℓ init + k = j ∑ A k + I j .
Convex comparison for the complete objective
Lemma E.13 (Convex comparison with local equality). For 0 ≤ j < L 0 \le j < L 0 ≤ j < L , use F F F , r j = A j − I j r_j=A_j-I_j r j = A j − I j , and Ψ j = F − A j + I j \Psi_j=F-A_j+I_j Ψ j = F − A j + I j as defined in Secs. E.1.9 and E.2.4. Then
Ψ j is convex and C p , r j = 0 near c j + 1 , \Psi_j\text{ is convex and }C^p,\qquad r_j=0\text{ near }c_{j+1}, Ψ j is convex and C p , r j = 0 near c j + 1 ,
Ψ j ( c j + 1 ) = F ( c j + 1 ) , ∇ Ψ j ( c j + 1 ) = ∇ F ( c j + 1 ) . \Psi_j(c_{j+1})=F(c_{j+1}),\qquad\nabla\Psi_j(c_{j+1})=\nabla F(c_{j+1}). Ψ j ( c j + 1 ) = F ( c j + 1 ) , ∇ Ψ j ( c j + 1 ) = ∇ F ( c j + 1 ) .
Proof. Lemma C.1 (a) in Appx. C and Theorem D.11 (a), (e) in Appx. D give convexity and C p C^p C p regularity of B B B and the A k A_k A k . The convex-sum rule in Lemma B.19 (c) applies to
Ψ j = B + ∑ k ≠ j A k + ℓ i n i t + I j , \Psi_j = B+\sum_{k\ne j}A_k+\ell_{\mathrm{init}}+I_j, Ψ j = B + k = j ∑ A k + ℓ init + I j ,
where ℓ i n i t + I j \ell_{\mathrm{init}}+I_j ℓ init + I j is affine, so Ψ j \Psi_j Ψ j is convex and C p C^p C p . Theorem D.11 (c) gives A j = I j A_j=I_j A j = I j near c j + 1 c_{j+1} c j + 1 . On that neighborhood,
r j = A j − I j = 0 , Ψ j = F − r j = F . r_j=A_j-I_j=0,\qquad\Psi_j=F-r_j=F. r j = A j − I j = 0 , Ψ j = F − r j = F .
Evaluating and differentiating the second equality proves the center identities.
Lemma E.14 (Gap on every short prefix). For the objective F F F in Definition E.8:
(a) Gap for every point in an early prefix.
x ∈ W j , b , b ≤ ⌊ m 4 ⌋ ⟹ F ( x ) − min F ≥ 7 δ j . (159) x \in W_{j,b},\quad b \le\left\lfloor\frac{m}{4} \right\rfloor\quad\Longrightarrow\quad F(x)-\min F \ge7\delta_j. \tag*{(159)} x ∈ W j , b , b ≤ ⌊ 4 m ⌋ ⟹ F ( x ) − min F ≥ 7 δ j . (159) Proof. • Proof of (a). Fix x ∈ W j , b x \in W_{j,b} x ∈ W j , b . Lemma E.13 gives the comparison hypotheses:
F = Ψ j + r j , Ψ j is convex and C 1 , r j = 0 near c j + 1 , F=\Psi_j+r_j,\qquad\Psi_j\text{ is convex and }C^1,\qquad r_j=0\text{ near }c_{j+1}, F = Ψ j + r j , Ψ j is convex and C 1 , r j = 0 near c j + 1 , Ψ j ( c j + 1 ) = F ( c j + 1 ) , ∇ Ψ j ( c j + 1 ) = ∇ F ( c j + 1 ) . \Psi_j(c_{j+1})=F(c_{j+1}),\qquad\nabla\Psi_j(c_{j+1})=\nabla F(c_{j+1}). Ψ j ( c j + 1 ) = F ( c j + 1 ) , ∇ Ψ j ( c j + 1 ) = ∇ F ( c j + 1 ) . The phase coordinates of x x x satisfy
P E j ( x − c j ) ∈ span { v j , u j , 1 , … , u j , b } , t j + 1 ( x ) = 0 , P_{E_j}(x-c_j)\in\operatorname{span}\{v_j,u_{j,1},\ldots,u_{j,b}\},\qquad t_{j+1}(x)=0, P E j ( x − c j ) ∈ span { v j , u j , 1 , … , u j , b } , t j + 1 ( x ) = 0 , with the next coordinate omitted in the terminal phase. Thus Theorem D.11 (b) gives r j ( x ) ≥ 7 δ j r_j(x)\ge7\delta_j r j ( x ) ≥ 7 δ j . For j < L − 1 j<L-1 j < L − 1 , we have v j + 1 ∈ E j + 1 v_{j+1}\in E_{j+1} v j + 1 ∈ E j + 1 , whereas x , c j + 1 ∈ ⨁ k ≤ j E k x,c_{j+1}\in\bigoplus_{k\le j}E_k x , c j + 1 ∈ ⨁ k ≤ j E k . Thus ⟨ v j + 1 , x ⟩ = ⟨ v j + 1 , c j + 1 ⟩ = 0 \langle v_{j+1},x\rangle=\langle v_{j+1},c_{j+1}\rangle=0 ⟨ v j + 1 , x ⟩ = ⟨ v j + 1 , c j + 1 ⟩ = 0 . Lemma E.11 (b) then gives
⟨ ∇ F ( c j + 1 ) , x − c j + 1 ⟩ = { − s j + 1 ⟨ v j + 1 , x − c j + 1 ⟩ = 0 , j < L − 1 , 0 , j = L − 1. \left\langle\nabla F(c_{j+1}),x-c_{j+1}\right\rangle=
\begin{cases}
-s_{j+1}\left\langle v_{j+1},x-c_{j+1}\right\rangle=0, & j<L-1,\\
0, & j=L-1.
\end{cases} ⟨ ∇ F ( c j + 1 ) , x − c j + 1 ⟩ = { − s j + 1 ⟨ v j + 1 , x − c j + 1 ⟩ = 0 , 0 , j < L − 1 , j = L − 1. In the first line v j + 1 v_{j+1} v j + 1 is orthogonal to both points; in the second ∇ F ( c L ) = 0 \nabla F(c_L)=0 ∇ F ( c L ) = 0 . The first-order convexity inequality for Ψ j \Psi_j Ψ j therefore has zero linear term on W j , b W_{j,b} W j , b . Lemma E.12 (b) applies with
( Ψ , r , W , c , e ) = ( Ψ j , r j , W j , b , c j + 1 , 7 δ j ) . (\Psi,r,W,c,e)=(\Psi_j,r_j,W_{j,b},c_{j+1},7\delta_j). ( Ψ , r , W , c , e ) = ( Ψ j , r j , W j , b , c j + 1 , 7 δ j ) . Its conclusion is
F ( x ) ≥ F ( c j + 1 ) + 7 δ j ≥ min F + 7 δ j . F(x)\ge F(c_{j+1})+7\delta_j\ge\min F+7\delta_j. F ( x ) ≥ F ( c j + 1 ) + 7 δ j ≥ min F + 7 δ j .
Compute the initial gap for normalization
To normalize the objective to the prescribed initial gap, we first compute its decrease between consecutive centers. The base and phase value changes give an exact formula for each decrease. Their geometric decay then determines D m , L D_{m,L} D m , L .
Lemma E.15 (Exact decrease between consecutive centers). For 0 ≤ j < L 0\le j<L 0 ≤ j < L , use the decrease χ j : = F ( c j ) − F ( c j + 1 ) \chi_j:=F(c_j)-F(c_{j+1}) χ j := F ( c j ) − F ( c j + 1 ) from Lemma 4.5 (d). Then:
(a) Exact decrease.
χ j = s j t j ⋆ − β q μ R j q − λ ( m + 1 ) τ j q q − 4 δ j , (160) \chi_j=s_jt_j^\star-\beta_q\mu R_j^q-\frac{\lambda(m+1)\tau_j^q}{q}-4\delta_j, \tag*{(160)} χ j = s j t j ⋆ − β q μ R j q − q λ ( m + 1 ) τ j q − 4 δ j , (160) where β q = 1 q + a l i f t θ 2 \beta_q=\frac{1}{q}+a_{\mathrm{lift}}\theta^2 β q = q 1 + a lift θ 2 .
Proof. We compare the base, the phase functions, and the initial affine term at c j c_j c j and c j + 1 = c j + h j c_{j+1}=c_j+h_j c j + 1 = c j + h j . The base gradient at c j c_j c j lies in the completed phase subspaces, so its linear contribution along h j h_j h j vanishes. Indeed, Lemma C.1 (c) in Appx. C gives
⟨ ∇ B ( c j ) , h j ⟩ = ⟨ ∑ k < j b k , h j ⟩ = ∑ k < j μ R k q − 2 ⟨ h k , h j ⟩ since b k = μ R k q − 2 h k , = 0 since E k ⊥ E j for k < j . \left\langle\nabla B(c_j),h_j\right\rangle=\left\langle\sum_{k<j}b_k,h_j\right\rangle
=\sum_{k<j}\mu R_k^{q-2}\left\langle h_k,h_j\right\rangle\quad\text{since }b_k=\mu R_k^{q-2}h_k,
=0 \quad\text{since }E_k\perp E_j\text{ for }k<j. ⟨ ∇ B ( c j ) , h j ⟩ = ⟨ k < j ∑ b k , h j ⟩ = k < j ∑ μ R k q − 2 ⟨ h k , h j ⟩ since b k = μ R k q − 2 h k , = 0 since E k ⊥ E j for k < j .
The center-value recurrence in Lemma C.1 (b) therefore identifies the base contribution as
B ( c j ) − B ( c j + 1 ) = − ⟨ ∇ B ( c j ) , h j ⟩ − β q μ R j q = − β q μ R j q by the zero inner product above. B(c_j)-B(c_{j+1})=-\left\langle\nabla B(c_j),h_j\right\rangle-\beta_q\mu R_j^q
=-\beta_q\mu R_j^q \quad\text{by the zero inner product above.} B ( c j ) − B ( c j + 1 ) = − ⟨ ∇ B ( c j ) , h j ⟩ − β q μ R j q = − β q μ R j q by the zero inner product above.
For the phase functions, moving from c j c_j c j to c j + 1 c_{j+1} c j + 1 changes only the inputs of A j − 1 A_{j-1} A j − 1 , when j ≥ 1 j\geq1 j ≥ 1 , and A j A_j A j . The other completed phases and the later phases have the same inputs at both centers. The center evaluations in Lemma E.11 (a), using Theorem D.11 (d), (c) in Appx. D, give
A k ( c j ) − A k ( c j + 1 ) = { 0 , k ≤ j − 2 or k ≥ j + 1 (same inputs) , 0 − ( − s j t j ⋆ ) , k = j − 1 , j ≥ 1 , ( − λ ( m + 1 ) τ j q q − 4 δ j ) − 0 , k = j . A_k(c_j)-A_k(c_{j+1})=
\begin{cases}
0, & k\leq j-2\text{ or }k\geq j+1\quad\text{(same inputs)},\\
0-\left(-s_jt_j^\star\right), & k=j-1,\ j\geq1,\\
\left(-\dfrac{\lambda(m+1)\tau_j^q}{q}-4\delta_j\right)-0, & k=j.
\end{cases} A k ( c j ) − A k ( c j + 1 ) = ⎩ ⎨ ⎧ 0 , 0 − ( − s j t j ⋆ ) , ( − q λ ( m + 1 ) τ j q − 4 δ j ) − 0 , k ≤ j − 2 or k ≥ j + 1 (same inputs) , k = j − 1 , j ≥ 1 , k = j .
When j = 0 j=0 j = 0 , there is no preceding phase, and the initial affine term supplies the same starting-slope contribution, since
ℓ i n i t ( c j ) − ℓ i n i t ( c j + 1 ) = s 0 ⟨ v 0 , h j ⟩ = { s 0 t 0 ⋆ , j = 0 , 0 , j ≥ 1. \ell_{\mathrm{init}}(c_j)-\ell_{\mathrm{init}}(c_{j+1})=s_0\left\langle v_0,h_j\right\rangle=
\begin{cases}
s_0t_0^\star, & j=0,\\
0, & j\geq1.
\end{cases} ℓ init ( c j ) − ℓ init ( c j + 1 ) = s 0 ⟨ v 0 , h j ⟩ = { s 0 t 0 ⋆ , 0 , j = 0 , j ≥ 1.
Adding these value changes and then arranging them in the stated order gives
χ j = ( B ( c j ) − B ( c j + 1 ) ) + ( ℓ i n i t ( c j ) − ℓ i n i t ( c j + 1 ) ) + ∑ k = 0 L − 1 ( A k ( c j ) − A k ( c j + 1 ) ) = − β q μ R j q + s j t j ⋆ − λ ( m + 1 ) τ j q q − 4 δ j by the value changes above, = s j t j ⋆ − β q μ R j q − λ ( m + 1 ) τ j q q − 4 δ j . \begin{aligned}
\chi_j&=(B(c_j)-B(c_{j+1}))+(\ell_{\mathrm{init}}(c_j)-\ell_{\mathrm{init}}(c_{j+1}))\\
&\quad+\sum_{k=0}^{L-1}(A_k(c_j)-A_k(c_{j+1}))\\
&=-\beta_q\mu R_j^q+s_jt_j^\star-\frac{\lambda(m+1)\tau_j^q}{q}-4\delta_j
\quad\text{by the value changes above,}\\
&=s_jt_j^\star-\beta_q\mu R_j^q-\frac{\lambda(m+1)\tau_j^q}{q}-4\delta_j.
\end{aligned} χ j = ( B ( c j ) − B ( c j + 1 )) + ( ℓ init ( c j ) − ℓ init ( c j + 1 )) + k = 0 ∑ L − 1 ( A k ( c j ) − A k ( c j + 1 )) = − β q μ R j q + s j t j ⋆ − q λ ( m + 1 ) τ j q − 4 δ j by the value changes above, = s j t j ⋆ − β q μ R j q − q λ ( m + 1 ) τ j q − 4 δ j .
The last line is Eq. (E.14).
Lemma E.16 (Bounds and exact normalization of the initial gap). For the decreases χ j \chi_j χ j in Lemma E.15:
(a) Decrease bounds.
0 < c B μ R j q ≤ χ j ≤ 2 ( 1 + c 0 ) λ m τ j q ( 0 ≤ j < L ) . 0<c_B\mu R_j^q\leq\chi_j\leq2(1+c_0)\lambda m\tau_j^q \qquad(0\leq j<L). 0 < c B μ R j q ≤ χ j ≤ 2 ( 1 + c 0 ) λm τ j q ( 0 ≤ j < L ) .
Initial gap
χ j = η q j χ 0 , D m , L : = F ( 0 ) − min F = χ 0 1 − η q L 1 − η q > 0. \chi_j = \eta^{qj}\chi_0,\qquad D_{m,L} := F(0)-\min F = \chi_0\frac{1-\eta^{qL}}{1-\eta^q}>0. χ j = η q j χ 0 , D m , L := F ( 0 ) − min F = χ 0 1 − η q 1 − η q L > 0.
For every orthogonal matrix Q Q Q , replacing all v j , u j , i v_j,u_{j,i} v j , u j , i by Q v j , Q u j , i Qv_j,Qu_{j,i} Q v j , Q u j , i leaves D m , L D_{m,L} D m , L unchanged.
Proof. • Proof of (a). Uniform convexity gives a positive decrease between consecutive centers, while convexity bounds that decrease by the starting slope. Lemma E.11 (b) and orthogonality give ⟨ ∇ F ( c j + 1 ) , h j ⟩ = 0 \langle\nabla F(c_{j+1}),h_j\rangle=0 ⟨ ∇ F ( c j + 1 ) , h j ⟩ = 0 , including the terminal phase. Hence
D F ( c j , c j + 1 ) = F ( c j ) − F ( c j + 1 ) − ⟨ ∇ F ( c j + 1 ) , c j − c j + 1 ⟩ = χ j + ⟨ ∇ F ( c j + 1 ) , h j ⟩ = χ j . D_F(c_j,c_{j+1})=F(c_j)-F(c_{j+1})-\langle\nabla F(c_{j+1}),c_j-c_{j+1}\rangle
=\chi_j+\langle\nabla F(c_{j+1}),h_j\rangle=\chi_j. D F ( c j , c j + 1 ) = F ( c j ) − F ( c j + 1 ) − ⟨ ∇ F ( c j + 1 ) , c j − c j + 1 ⟩ = χ j + ⟨ ∇ F ( c j + 1 ) , h j ⟩ = χ j .
Lemma E.9 now gives
χ j = D F ( c j , c j + 1 ) ≥ c B μ ∥ c j − c j + 1 ∥ q = c B μ R j q > 0. \chi_j=D_F(c_j,c_{j+1})\ge c_B\mu\lVert c_j-c_{j+1}\rVert^q=c_B\mu R_j^q>0. χ j = D F ( c j , c j + 1 ) ≥ c B μ ∥ c j − c j + 1 ∥ q = c B μ R j q > 0.
The first-order convexity inequality in Lemma B.19 (a), now based at c j c_j c j , gives
F ( c j + 1 ) ≥ F ( c j ) + ⟨ ∇ F ( c j ) , h j ⟩ , F(c_{j+1})\ge F(c_j)+\langle\nabla F(c_j),h_j\rangle, F ( c j + 1 ) ≥ F ( c j ) + ⟨ ∇ F ( c j ) , h j ⟩ ,
χ j ≤ − ⟨ ∇ F ( c j ) , h j ⟩ = s j ⟨ v j , h j ⟩ = s j t j ⋆ . \chi_j\le-\langle\nabla F(c_j),h_j\rangle
=s_j\langle v_j,h_j\rangle=s_jt_j^\star. χ j ≤ − ⟨ ∇ F ( c j ) , h j ⟩ = s j ⟨ v j , h j ⟩ = s j t j ⋆ .
By Proposition E.5 (a),
s j t j ⋆ ≤ ( 1 + c 0 ) Γ j t j ⋆ = ( 1 + c 0 ) λ ( m + 1 ) τ j q ≤ 2 ( 1 + c 0 ) λ m τ j q . s_jt_j^\star\le(1+c_0)\Gamma_jt_j^\star=(1+c_0)\lambda(m+1)\tau_j^q\le2(1+c_0)\lambda m\tau_j^q. s j t j ⋆ ≤ ( 1 + c 0 ) Γ j t j ⋆ = ( 1 + c 0 ) λ ( m + 1 ) τ j q ≤ 2 ( 1 + c 0 ) λm τ j q .
Therefore
0 < c B μ R j q ≤ F ( c j ) − F ( c j + 1 ) ≤ s j t j ⋆ ≤ 2 ( 1 + c 0 ) λ m τ j q . (161) 0<c_B\mu R_j^q\le F(c_j)-F(c_{j+1})\le s_jt_j^\star\le2(1+c_0)\lambda m\tau_j^q. \tag*{(161)} 0 < c B μ R j q ≤ F ( c j ) − F ( c j + 1 ) ≤ s j t j ⋆ ≤ 2 ( 1 + c 0 ) λm τ j q . (161)
• Proof of (b). The phase parameters have the same scaling in every phase. Definition E.1 and the formulas for s j s_j s j and δ j \delta_j δ j give
s j = η ( q − 1 ) j s 0 , t j ⋆ = η j t 0 ⋆ , τ j = η j τ 0 , R j = η j R 0 , δ j = η q j δ 0 . s_j=\eta^{(q-1)j}s_0,\qquad t_j^\star=\eta^jt_0^\star,\qquad\tau_j=\eta^j\tau_0,\qquad R_j=\eta^jR_0,\qquad\delta_j=\eta^{qj}\delta_0. s j = η ( q − 1 ) j s 0 , t j ⋆ = η j t 0 ⋆ , τ j = η j τ 0 , R j = η j R 0 , δ j = η q j δ 0 .
Each term in Eq. E.14 therefore scales by η q j \eta^{qj} η q j , giving
χ j = ( η ( q − 1 ) j s 0 ) ( η j t 0 ⋆ ) − λ q ( m + 1 ) ( η j τ 0 ) q − 4 η q j δ 0 − β q μ ( η j R 0 ) q = η q j ( s 0 t 0 ⋆ − λ q ( m + 1 ) τ 0 q − 4 δ 0 − β q μ R 0 q ) = η q j χ 0 \begin{aligned}
\chi_j=\left(\eta^{(q-1)j}s_0\right)\left(\eta^jt_0^\star\right)-\frac{\lambda}{q}(m+1)\left(\eta^j\tau_0\right)^q-4\eta^{qj}\delta_0-\beta_q\mu\left(\eta^jR_0\right)^q \\
&=\eta^{qj}\left(s_0t_0^\star-\frac{\lambda}{q}(m+1)\tau_0^q-4\delta_0-\beta_q\mu R_0^q\right) \\
&=\eta^{qj}\chi_0
\end{aligned} χ j = ( η ( q − 1 ) j s 0 ) ( η j t 0 ⋆ ) − q λ ( m + 1 ) ( η j τ 0 ) q − 4 η q j δ 0 − β q μ ( η j R 0 ) q = η q j ( s 0 t 0 ⋆ − q λ ( m + 1 ) τ 0 q − 4 δ 0 − β q μ R 0 q ) = η q j χ 0
by Eq. E.14 at j = 0 j=0 j = 0 .
Since c 0 = 0 c_0=0 c 0 = 0 and c L c_L c L is the minimizer by Lemma E.11 (c), summing these decreases gives the exact initial gap
D m , L : = F ( 0 ) − F ( c L ) = ∑ j = 0 L − 1 ( F ( c j ) − F ( c j + 1 ) ) since c 0 = 0 , = ∑ j = 0 L − 1 χ j = χ 0 ∑ j = 0 L − 1 η q j = χ 0 1 − η q L 1 − η q (162) \begin{aligned}
D_{m,L}:=F(0)-F(c_L)=\sum_{j=0}^{L-1}\left(F(c_j)-F(c_{j+1})\right)\quad\text{since }c_0=0, \\
&=\sum_{j=0}^{L-1}\chi_j=\chi_0\sum_{j=0}^{L-1}\eta^{qj} \\
&=\chi_0\frac{1-\eta^{qL}}{1-\eta^q} \tag*{(162)}
\end{aligned} D m , L := F ( 0 ) − F ( c L ) = j = 0 ∑ L − 1 ( F ( c j ) − F ( c j + 1 ) ) since c 0 = 0 , = j = 0 ∑ L − 1 χ j = χ 0 j = 0 ∑ L − 1 η q j = χ 0 1 − η q 1 − η q L (162)
by the finite geometric sum,
Positivity follows from part (a), L ≥ 1 L \ge1 L ≥ 1 , and 0 < η < 1 0 < \eta< 1 0 < η < 1 . The values s 0 s_0 s 0 , t 0 ⋆ t_0^\star t 0 ⋆ , R 0 R_0 R 0 , τ 0 \tau_0 τ 0 , δ 0 \delta_0 δ 0 in the formula for χ 0 \chi_0 χ 0 depend only on the fixed scalar parameters, by Definition E.1 . Consequently, the formula for D m , L D_{m,L} D m , L is independent of the orthonormal directions, proving the stated invariance under Q Q Q .
Proof of Lemma 4.5
Proof of Lemma 4.5. Proof of (a). Since the added phase functions are convex, Lemma B.20 with G = ∑ j = 0 L − 1 A j G = \sum_{j=0}^{L-1} A_j G = ∑ j = 0 L − 1 A j and ℓ = ℓ i n i t \ell= \ell_{\mathrm{init}} ℓ = ℓ init gives D F ≥ D B D_F \ge D_B D F ≥ D B . The bound for the base in Lemma 4.3 (a) therefore yields
D F ( y , x ) ≥ D B ( y , x ) ≥ c B μ ∥ y − x ∥ q . D_F(y,x) \ge D_B(y,x) \ge c_B \mu\lVert y-x\rVert^q. D F ( y , x ) ≥ D B ( y , x ) ≥ c B μ ∥ y − x ∥ q .
Summing phase smoothness bounds would cost a factor L L L . Instead, each A j A_j A j depends only on V j = E j ⊕ span { v j + 1 } V_j = E_j \oplus\operatorname{span}\{v_{j+1}\} V j = E j ⊕ span { v j + 1 } , with V L − 1 = E L − 1 V_{L-1}=E_{L-1} V L − 1 = E L − 1 . These subspaces are mutually orthogonal within each parity group J a = { 0 ≤ j < L : j ≡ a ( m o d 2 ) } J_a = \{0 \le j < L : j \equiv a \pmod2\} J a = { 0 ≤ j < L : j ≡ a ( mod 2 )} for a ∈ { 0 , 1 } a \in\{0,1\} a ∈ { 0 , 1 } , allowing a maximum bound on the derivative differences Q j = ∇ p A j ( x ) − ∇ p A j ( y ) Q_j = \nabla_p A_j(x) - \nabla_p A_j(y) Q j = ∇ p A j ( x ) − ∇ p A j ( y ) . Each Q j Q_j Q j acts only on the projections of its arguments onto V j V_j V j , as verified in Lemma E.10 . For nonempty J a J_a J a ,
∥ ∑ j ∈ J a Q j ∥ = max j ∈ J a ∥ Q j ∥ by Lemma B.10, ≤ C A λ ∥ x − y ∥ ν by Lemma 4.4 (a). \begin{aligned}
\left\lVert\sum_{j\in J_a} Q_j \right\rVert= \max_{j\in J_a}\lVert Q_j\rVert\qquad\text{by Lemma B.10,} \\
&\le C_A\lambda\lVert x-y\rVert^\nu\qquad\text{by Lemma 4.4 (a).}
\end{aligned} j ∈ J a ∑ Q j = j ∈ J a max ∥ Q j ∥ by Lemma B.10, ≤ C A λ ∥ x − y ∥ ν by Lemma 4.4 (a).
The same bound holds with a zero sum when J a = ∅ J_a=\varnothing J a = ∅ . Adding both groups and the base bound gives
∥ ∇ p F ( x ) − ∇ p F ( y ) ∥ ≤ ( C B μ + 2 C A λ ) ∥ x − y ∥ ν by Lemma 4.3 (a), ≤ C H μ m a q ∥ x − y ∥ ν since λ = K μ m a q , m ≥ 1. \begin{aligned}
\lVert\nabla^p F(x)-\nabla^p F(y)\rVert\le(C_B\mu+2C_A\lambda)\lVert x-y\rVert^\nu\qquad\text{by Lemma 4.3 (a),} \\
&\le C_H\mu m^{a_q}\lVert x-y\rVert^\nu\qquad\text{since }\lambda=K\mu m^{a_q},\ m\ge1.
\end{aligned} ∥ ∇ p F ( x ) − ∇ p F ( y )∥ ≤ ( C B μ + 2 C A λ ) ∥ x − y ∥ ν by Lemma 4.3 (a), ≤ C H μ m a q ∥ x − y ∥ ν since λ = K μ m a q , m ≥ 1.
All terms of F F F are C p C^p C p by Lemmas 4.3 and 4.4 , so this proves both stated regularity bounds.
Proof of (b). By Lemma 4.4 (c)–(d), completed phases contribute g k g_k g k and the others 0 0 0 at c j c_j c j . Adding the base increments gives
∇ F ( c j ) = − s 0 v 0 + ∑ k < j ( b k + g k ) by Lemma 4.3 (b), = − s 0 v 0 + ∑ k < j ( s k v k − s k + 1 v k + 1 ) by xref‘Eq. (4.11), = − s j v j by telescoping ( 0 ≤ j < L ) . \begin{aligned}
\nabla F(c_j)=-s_0v_0+\sum_{k<j}(b_k+g_k) \qquad\text{by Lemma 4.3 (b),} \\
&=-s_0v_0+\sum_{k<j}(s_kv_k-s_{k+1}v_{k+1}) \qquad\text{by {xref}`Eq. (4.11),} \\
&=-s_jv_j \qquad\text{by telescoping}\qquad(0\le j<L).
\end{aligned} ∇ F ( c j ) = − s 0 v 0 + k < j ∑ ( b k + g k ) by Lemma 4.3 (b), = − s 0 v 0 + k < j ∑ ( s k v k − s k + 1 v k + 1 ) by xref ‘Eq. (4.11), = − s j v j by telescoping ( 0 ≤ j < L ) .
For j = L j=L j = L , the same telescope ends with the omitted terminal term s L v L = 0 s_Lv_L=0 s L v L = 0 , giving ∇ F ( c L ) = 0 \nabla F(c_L)=0 ∇ F ( c L ) = 0 . Convexity makes c L c_L c L a minimizer, and the positive uniform-convexity bound from part (a) makes it unique. Lemma E.11 verifies the individual phase evaluations and this terminal case.
Proof of (c). To prevent other components from offsetting A j − I j ≥ 7 δ j A_j-I_j \ge7\delta_j A j − I j ≥ 7 δ j , we define Ψ j : = F − A j + I j \Psi_j := F-A_j+I_j Ψ j := F − A j + I j and r j ( x ) : = A j ( x ) − I j ( x ) r_j(x):=A_j(x)-I_j(x) r j ( x ) := A j ( x ) − I j ( x ) . The function Ψ j = B + ℓ i n i t + ∑ k ≠ j A k + I j \Psi_j=B+\ell_{\mathrm{init}}+\sum_{k\ne j}A_k+I_j Ψ j = B + ℓ init + ∑ k = j A k + I j is convex by Lemma B.19 (c). Lemma 4.4 (c) gives Ψ j = F \Psi_j=F Ψ j = F near c j + 1 c_{j+1} c j + 1 , hence equality of their values and gradients there. For x ∈ W j , b x\in W_{j,b} x ∈ W j , b with b ≤ ⌊ m 4 ⌋ b\le\left\lfloor\frac{m}{4}\right\rfloor b ≤ ⌊ 4 m ⌋ , define w = P E j ( x − c j ) ∈ U j , b w=P_{E_j}(x-c_j)\in U_{j,b} w = P E j ( x − c j ) ∈ U j , b . The next coordinate ⟨ v j + 1 , x − c j ⟩ \langle v_{j+1},x-c_j\rangle ⟨ v j + 1 , x − c j ⟩ is 0 0 0 when j < L − 1 j<L-1 j < L − 1 . Thus Eq. (17) and Lemma 4.4 (b) give
r j ( x ) = r j ( c j + w ) ≥ 7 δ j . r_j(x)=r_j(c_j+w)\ge7\delta_j. r j ( x ) = r j ( c j + w ) ≥ 7 δ j .
For j < L − 1 j < L-1 j < L − 1 , x − c j + 1 ∈ ⨁ k ≤ j E k x-c_{j+1} \in\bigoplus_{k\le j} E_k x − c j + 1 ∈ ⨁ k ≤ j E k is perpendicular to v j + 1 v_{j+1} v j + 1 . By part (b), ⟨ ∇ F ( c j + 1 ) , x − c j + 1 ⟩ = 0 \langle\nabla F(c_{j+1}),x-c_{j+1}\rangle=0 ⟨ ∇ F ( c j + 1 ) , x − c j + 1 ⟩ = 0 . The same equality holds for j = L − 1 j=L-1 j = L − 1 because ∇ F ( c L ) = 0 \nabla F(c_L)=0 ∇ F ( c L ) = 0 . The first-order convexity inequality therefore yields
Ψ j ( x ) ≥ Ψ j ( c j + 1 ) + ⟨ ∇ Ψ j ( c j + 1 ) , x − c j + 1 ⟩ by Lemma B.19 (a) , = F ( c j + 1 ) by Ψ j = F nearby and the zero inner product, F ( x ) = Ψ j ( x ) + r j ( x ) by the definition of Ψ j , ≥ F ( c j + 1 ) + 7 δ j by the preceding bounds, ≥ F ( c L ) + 7 δ j since c L is the minimizer. \begin{aligned}
\Psi_j(x) &\ge\Psi_j(c_{j+1})+\langle\nabla\Psi_j(c_{j+1}),x-c_{j+1}\rangle&&\text{by Lemma \(\text{B.19 (a)}\),}\\
&=F(c_{j+1}) &&\text{by \(\Psi_j=F\) nearby and the zero inner product,}\\
F(x)&=\Psi_j(x)+r_j(x) &&\text{by the definition of \(\Psi_j\),}\\
&\ge F(c_{j+1})+7\delta_j &&\text{by the preceding bounds,}\\
&\ge F(c_L)+7\delta_j &&\text{since \(c_L\) is the minimizer.}
\end{aligned} Ψ j ( x ) F ( x ) ≥ Ψ j ( c j + 1 ) + ⟨ ∇ Ψ j ( c j + 1 ) , x − c j + 1 ⟩ = F ( c j + 1 ) = Ψ j ( x ) + r j ( x ) ≥ F ( c j + 1 ) + 7 δ j ≥ F ( c L ) + 7 δ j by Lemma B.19 (a) , by Ψ j = F nearby and the zero inner product, by the definition of Ψ j , by the preceding bounds, since c L is the minimizer.
This proves the prefix-gap conclusion for all x ∈ W j , b x\in W_{j,b} x ∈ W j , b .
Proof of (d). Lemmas E.15 and E.16 give the base and phase value changes, their geometric decay, and 0 < χ 0 ≤ 2 ( 1 + c 0 ) λ m τ 0 q 0<\chi_0\le2(1+c_0)\lambda m\tau_0^q 0 < χ 0 ≤ 2 ( 1 + c 0 ) λm τ 0 q . Summing these decreases gives
D m , L = F ( c 0 ) − F ( c L ) = ∑ j = 0 L − 1 χ j D_{m,L}=F(c_0)-F(c_L)=\sum_{j=0}^{L-1}\chi_j D m , L = F ( c 0 ) − F ( c L ) = j = 0 ∑ L − 1 χ j
by the definition of χ j \chi_j χ j ,
= χ 0 ∑ j = 0 L − 1 η q j =\chi_0\sum_{j=0}^{L-1}\eta^{qj} = χ 0 j = 0 ∑ L − 1 η q j
by Lemma E.16 (b),
= χ 0 1 − η q L 1 − η q =\chi_0\frac{1-\eta^{qL}}{1-\eta^q} = χ 0 1 − η q 1 − η q L
by the finite geometric sum.
The value-change formula in Lemma E.15 depends only on s j s_j s j , t j ⋆ t_j^\star t j ⋆ , τ j \tau_j τ j , δ j \delta_j δ j , R j R_j R j and the fixed scalar coefficients. These quantities are independent of the orthonormal phase directions, so both χ 0 \chi_0 χ 0 and D m , L D_{m,L} D m , L have the claimed independence.
The Oracle Transcript and One Fixed Completion
Preservation of earlier oracle outputs
To apply Lemma E.14 (a) to a deterministic algorithm A \mathcal{A} A as in Definition B.1 , we must place its candidate solutions in short prefixes W j , b W_{j,b} W j , b . We therefore choose the unfixed chain directions as the algorithm runs, while fixing the dimension and all scalar parameters beforehand. For oracle call s s s , denote the evaluation point by x s x_s x s , the returned value and derivative tensors by a s , k a_{s,k} a s , k , and the candidate solution after this call by x ^ s \widehat{x}_s x s . We impose no norm bound on these points, and a candidate solution need not be an evaluation point. We collect the oracle outputs and available information as
O s = ( a s , 0 , … , a s , p ) , H s = ( ( x ℓ , O ℓ ) ) ℓ = 1 s , H 0 = ∅ . (163) \mathcal{O}_s=(a_{s,0},\ldots,a_{s,p}),\qquad\mathcal{H}_s=((x_\ell,\mathcal{O}_\ell))_{\ell=1}^{s},\qquad\mathcal{H}_0=\varnothing. \tag*{(163)} O s = ( a s , 0 , … , a s , p ) , H s = (( x ℓ , O ℓ ) ) ℓ = 1 s , H 0 = ∅ . (163)
With fixed parameters and initialization, determinism gives
x s + 1 = A s + 1 ( H s ) , x ^ s = A ^ s ( H s ) . (164) x_{s+1}=\mathcal{A}_{s+1}(\mathcal{H}_s),\qquad\widehat{x}_s=\widehat{\mathcal{A}}_s(\mathcal{H}_s). \tag*{(164)} x s + 1 = A s + 1 ( H s ) , x s = A s ( H s ) . (164)
These rules define the evaluation trajectory x 1 , … , x n m L x_1,\ldots,x_{nm_L} x 1 , … , x n m L and candidate solutions x ^ 0 , … , x ^ n m L \widehat{x}_0,\ldots,\widehat{x}_{nm_L} x 0 , … , x n m L . To obtain one fixed objective that reproduces this adaptive trajectory, we must preserve every recorded value and derivative tensor when choosing the remaining directions. For the nonempty completion family F s \mathcal{F}_s F s in Eq. (23) , preservation of the recorded oracle outputs requires
∇ k F ( x ℓ ) = a ℓ , k ( F ∈ F s , 1 ≤ ℓ ≤ s , 0 ≤ k ≤ p ) , (165) \nabla^k F(x_\ell)=a_{\ell,k}\quad(F\in\mathcal{F}_s,\ 1\leq\ell\leq s,\ 0\leq k\leq p), \tag*{(165)} ∇ k F ( x ℓ ) = a ℓ , k ( F ∈ F s , 1 ≤ ℓ ≤ s , 0 ≤ k ≤ p ) , (165)
where ∇ 0 F = F \nabla^0 F=F ∇ 0 F = F and a ℓ , 0 a_{\ell,0} a ℓ , 0 is the returned value.
Replay and preservation of earlier oracle outputs
Lemma F.1 (Completing directions gives one fixed oracle instance). Suppose the dimension and all scalar parameters are fixed before directions are chosen adaptively. After each oracle call, suppose the nonempty family F s \mathcal{F}_s F s of final completions satisfies the oracle consistency requirement Eq. (F.3). Running the algorithm on any final completion has both properties:
(a) It generates the recorded evaluation points and oracle outputs.
(b) It generates the recorded deterministic candidate solutions, including the zero-call candidate solution.
Proof. Fix any final completion F F F . Its oracle agrees with every recorded oracle output by Eq. (F.3). The deterministic rules in Eq. (F.2) then reproduce both the evaluation trajectory and the candidate solutions, as follows.
Proof of (a). Eq. (F.2) gives x 1 = A 1 ( H 0 ) x_1=\mathcal{A}_1(\mathcal{H}_0) x 1 = A 1 ( H 0 ) from the fixed input. If the first ℓ − 1 \ell-1 ℓ − 1 oracle outputs agree, their transcript H ℓ − 1 \mathcal{H}_{\ell-1} H ℓ − 1 is the same, so x ℓ = A ℓ ( H ℓ − 1 ) x_\ell=\mathcal{A}_\ell(\mathcal{H}_{\ell-1}) x ℓ = A ℓ ( H ℓ − 1 ) is unchanged. Eq. (F.3) fixes every component a ℓ , k a_{\ell,k} a ℓ , k of O ℓ \mathcal{O}_\ell O ℓ . Induction therefore reproduces the recorded transcript H s \mathcal{H}_s H s .
Proof of (b). Part (a) fixes each H s \mathcal{H}_s H s . Eq. (F.2) then gives the same candidate solution x ^ s = A s ( H s ) \widehat{x}_s=\mathcal{A}_s(\mathcal{H}_s) x s = A s ( H s ) , including s = 0 s=0 s = 0 , when H 0 = ∅ \mathcal{H}_0=\varnothing H 0 = ∅ .
Lemma F.2 (A local change preserves earlier oracle outputs of the base). Suppose all partial bases belong to C p C^p C p , B j + 1 = B j B_{j+1}=B_j B j + 1 = B j outside an open ball S j S_j S j , all later changes up to the finite final base are contained in S j S_j S j , and dist ( x s , S j ‾ ) > 0 \operatorname{dist}(x_s,\overline{S_j})>0 dist ( x s , S j ) > 0 . Then:
(a) The final base B B B equals B j B_j B j on a neighborhood of x s x_s x s .
(b) ∇ k B ( x s ) = ∇ k B j ( x s ) \nabla^kB(x_s)=\nabla^kB_j(x_s) ∇ k B ( x s ) = ∇ k B j ( x s ) for 0 ≤ k ≤ p 0\leq k\leq p 0 ≤ k ≤ p .
Proof. • Proof of (a). Define d s : = dist ( x s , S j ‾ ) > 0 d_s:=\operatorname{dist}(x_s,\overline{S_j})>0 d s := dist ( x s , S j ) > 0 . For z ∈ B ( x s , d s / 2 ) z\in B(x_s,d_s/2) z ∈ B ( x s , d s /2 ) and w ∈ S j ‾ w\in\overline{S_j} w ∈ S j ,
∥ z − w ∥ ≥ ∥ x s − w ∥ − ∥ z − x s ∥ by the reverse triangle inequality, ≥ d s − ∥ z − x s ∥ > d s 2 . \begin{aligned}
\lVert z-w\rVert\geq\lVert x_s-w\rVert-\lVert z-x_s\rVert\quad\text{by the reverse triangle inequality,} \\
&\geq d_s-\lVert z-x_s\rVert>\frac{d_s}{2}.
\end{aligned} ∥ z − w ∥ ≥ ∥ x s − w ∥ − ∥ z − x s ∥ by the reverse triangle inequality, ≥ d s − ∥ z − x s ∥ > 2 d s .
Thus this ball is disjoint from every later modification. If B = B L B=B_L B = B L is the final base, then at every point z z z of the ball,
B ( z ) − B j ( z ) = ∑ k = j L − 1 ( B k + 1 ( z ) − B k ( z ) ) = 0 , \begin{aligned}
B(z)-B_j(z)=\sum_{k=j}^{L-1}\left(B_{k+1}(z)-B_k(z)\right) \\
&=0,
\end{aligned} B ( z ) − B j ( z ) = k = j ∑ L − 1 ( B k + 1 ( z ) − B k ( z ) ) = 0 ,
where the second line uses the assumed exterior agreement at each step. This proves equality throughout the neighborhood.
One fixed instance for deterministic algorithms
Two facts for selecting directions
Lemma F.3 (One direction includes an arbitrary point). Let V ⊆ R d V \subseteq\mathbb{R}^{d} V ⊆ R d be a linear subspace and x ∈ R d x \in\mathbb{R}^{d} x ∈ R d . At most one additional unit direction perpendicular to V V V is needed to include x x x in the enlarged span.
Proof. To include x x x in the enlarged span, we isolate its component perpendicular to V V V . Using the orthogonal projection from § 3 , define ω = x − P V x \omega= x - P_{V}x ω = x − P V x . The projection identity P V 2 = P V P_{V}^{2} = P_{V} P V 2 = P V gives
P V ω = P V x − P V 2 x = 0 , P_{V}\omega= P_{V}x - P_{V}^{2}x = 0, P V ω = P V x − P V 2 x = 0 , so ω ⊥ V \omega\perp V ω ⊥ V . If ω = 0 \omega= 0 ω = 0 , then x = P V x ∈ V x = P_{V}x \in V x = P V x ∈ V . Otherwise u = ω ∥ ω ∥ u = \frac{\omega}{\lVert\omega\rVert} u = ∥ ω ∥ ω is a unit vector perpendicular to V V V , and
x = P V x + ∥ ω ∥ u ∈ V ⊕ span { u } . x = P_{V}x + \lVert\omega\rVert u \in V \oplus\operatorname{span}\{u\}. x = P V x + ∥ ω ∥ u ∈ V ⊕ span { u } . In particular, for V = W j , b V = W_{j,b} V = W j , b , the additional chain direction is chosen by
ω = x − P W j , b x , u j , b + 1 = ω ∥ ω ∥ ( ω ≠ 0 ) . (166) \omega= x - P_{W_{j,b}}x,\qquad u_{j,b+1} = \frac{\omega}{\lVert\omega\rVert}\quad(\omega\ne0). \tag*{(166)} ω = x − P W j , b x , u j , b + 1 = ∥ ω ∥ ω ( ω = 0 ) . (166) Then x ∈ W j , b + 1 x \in W_{j,b+1} x ∈ W j , b + 1 , so we increase b b b by one. If ω = 0 \omega= 0 ω = 0 , no direction is needed.
Lemma F.4 (One additional direction specifies a prefix oracle output). Use the scalar parameters of Definition E.8 in Appx. E. Fix indices 0 ≤ j < L 0 \le j < L 0 ≤ j < L and 0 ≤ b ≤ ⌊ L 4 ⌋ 0 \le b \le\left\lfloor\frac{L}{4} \right\rfloor 0 ≤ b ≤ ⌊ 4 L ⌋ , the ordered orthonormal bases of E 0 , … , E j − 1 E_{0},\ldots,E_{j-1} E 0 , … , E j − 1 , and v j , u j , 1 , … , u j , b + 1 v_{j},u_{j,1},\ldots,u_{j,b+1} v j , u j , 1 , … , u j , b + 1 . The objectives F , F ~ F,\widetilde{F} F , F arising from any two remaining orthonormal completions satisfy
∇ k F ( x ) = ∇ k F ~ ( x ) ( x ∈ W j , b , 0 ≤ k ≤ p ) , ∇ 0 F : = F . \nabla^{k}F(x) = \nabla^{k}\widetilde{F}(x)\qquad(x \in W_{j,b},\ 0 \le k \le p),\qquad\nabla^{0}F := F. ∇ k F ( x ) = ∇ k F ( x ) ( x ∈ W j , b , 0 ≤ k ≤ p ) , ∇ 0 F := F . Proof. Fix x ∈ W j , b x \in W_{j,b} x ∈ W j , b . To show that its oracle output is unchanged by the remaining completion, we check each component of F = B + ℓ i n i t + ∑ k A k F = B + \ell_{\mathrm{init}} + \sum_{k} A_{k} F = B + ℓ init + ∑ k A k from Definition E.8. The terminal convention is supplied by Theorem D.11 (f).
Case 1. Base term B B B . The change from B j B_{j} B j to B j + 1 B_{j+1} B j + 1 is confined to S j S_{j} S j , and all subsequent replacement balls lie inside S j S_{j} S j by Lemma C.1 (a). Lemma D.12 (a) gives dist ( c j + 1 , W j , b ) > θ R j \operatorname{dist}(c_{j+1},W_{j,b}) > \theta R_{j} dist ( c j + 1 , W j , b ) > θ R j , so x x x lies outside S ‾ j \overline{S}_{j} S j . Lemma C.3 (a) therefore gives B = B j B = B_{j} B = B j near x x x , and hence
∇ k B ( x ) = ∇ k B j ( x ) ( 0 ≤ k ≤ p ) . \nabla^{k}B(x) = \nabla^{k}B_{j}(x)\qquad(0 \le k \le p). ∇ k B ( x ) = ∇ k B j ( x ) ( 0 ≤ k ≤ p ) . These derivatives use only the fixed centers c 0 , … , c j c_{0},\ldots,c_{j} c 0 , … , c j and Euclidean distances, so they are independent of all remaining directions.
Case 2. Current phase A j A_j A j . The evaluation point has phase coordinates
w = P E j ( x − c j ) ∈ span { v j , u j , 1 , … , u j , b } , ⟨ v j + 1 , x − c j ⟩ = 0 , w = P_{E_j}(x-c_j) \in\operatorname{span}\{v_j,u_{j,1},\ldots,u_{j,b}\}, \qquad\langle v_{j+1},x-c_j\rangle= 0, w = P E j ( x − c j ) ∈ span { v j , u j , 1 , … , u j , b } , ⟨ v j + 1 , x − c j ⟩ = 0 ,
with the second coordinate omitted in the terminal phase. The coordinate definition in Eq. (17) therefore gives
∇ k A j ( x ) = ∇ k A j ( c j + w ) ( 0 ≤ k ≤ p ) . \nabla^k A_j(x) = \nabla^k A_j(c_j+w) \qquad(0\leq k\leq p). ∇ k A j ( x ) = ∇ k A j ( c j + w ) ( 0 ≤ k ≤ p ) .
By Lemma D.14 , these tensors and the value depend only on the prefix and its next direction u j , b + 1 u_{j,b+1} u j , b + 1 . Fixing this direction determines the full ambient tensors, including their action on test vectors outside the selected span.
Case 3. Future phases A k A_k A k , k > j k>j k > j . Both x ∈ W j , b x\in W_{j,b} x ∈ W j , b and c k = ∑ ℓ < k h ℓ c_k=\sum_{\ell<k}h_\ell c k = ∑ ℓ < k h ℓ belong to ⨁ ℓ < k E ℓ \bigoplus_{\ell<k}E_\ell ⨁ ℓ < k E ℓ . Hence
P E k ( x − c k ) = 0 , ⟨ v k + 1 , x − c k ⟩ = 0 , P_{E_k}(x-c_k)=0,\qquad\langle v_{k+1},x-c_k\rangle=0, P E k ( x − c k ) = 0 , ⟨ v k + 1 , x − c k ⟩ = 0 ,
with the second coordinate omitted for k = L − 1 k=L-1 k = L − 1 . Theorem D.11 (d) at these zero inputs gives
A k ( x ) = − λ ( m + 1 ) τ k q q − 4 δ k , ∇ ℓ A k ( x ) = 0 ( 1 ≤ ℓ ≤ p ) . A_k(x)=-\frac{\lambda(m+1)\tau_k^q}{q}-4\delta_k,\qquad\nabla^\ell A_k(x)=0 \quad(1\leq\ell\leq p). A k ( x ) = − q λ ( m + 1 ) τ k q − 4 δ k , ∇ ℓ A k ( x ) = 0 ( 1 ≤ ℓ ≤ p ) .
Thus unchosen future directions enter neither the derivatives nor the value returned now.
Case 4. Completed phases and ℓ i n i t \ell_{\mathrm{init}} ℓ init . For k < j k<j k < j , the coordinates of A k A_k A k use only the fixed E k E_k E k and v k + 1 v_{k+1} v k + 1 . The initial affine term uses only the fixed v 0 v_0 v 0 .
Adding all components gives the same value and full derivative tensors for every remaining completion and every 0 ≤ k ≤ p 0\leq k\leq p 0 ≤ k ≤ p . The base agrees on a neighborhood; for the phase functions, equality of the value and derivatives at the evaluation point is sufficient for Eq. (F.3). □ \square □
Construct the transcript and complete one fixed instance
Proposition F.5 (Direction selection and fixed-function realization). Before the first oracle call, fix the scalar parameters of Definition E.8 in Appx. E, with
m , L ∈ N , m ≥ 32 , L ≥ 1 , d ≥ L ( m + 1 ) , n m : = ⌊ m 16 ⌋ . m,L\in\mathbb{N},\qquad m\geq32,\qquad L\geq1,\qquad d\geq L(m+1),\qquad n_m:=\left\lfloor\frac{m}{16}\right\rfloor. m , L ∈ N , m ≥ 32 , L ≥ 1 , d ≥ L ( m + 1 ) , n m := ⌊ 16 m ⌋ . For every deterministic algorithm, the ordered bases can be chosen so that:
(a) Prefix placement. n m ≥ m 32 n_m\geq\frac{m}{32} n m ≥ 32 m . For 0 ≤ j < L 0\leq j<L 0 ≤ j < L and j n m < s ≤ ( j + 1 ) n m jn_m<s\leq(j+1)n_m j n m < s ≤ ( j + 1 ) n m , there is an index 0 ≤ b ≤ ⌊ m 4 ⌋ 0\leq b\leq\left\lfloor\frac{m}{4}\right\rfloor 0 ≤ b ≤ ⌊ 4 m ⌋ with
x s , x ^ s ∈ W j , b . x_s,\widehat{x}_s\in W_{j,b}. x s , x s ∈ W j , b . Also x ^ 0 ∈ W 0 , b \widehat{x}_0\in W_{0,b} x 0 ∈ W 0 , b for some 0 ≤ b ≤ 1 0\leq b\leq1 0 ≤ b ≤ 1 .
(b) One fixed objective. The complete bases define one F F F by Eq. (E.11). Running the algorithm with its oracle reproduces all recorded evaluation points, oracle outputs, and candidate solutions through oracle call n m L n_mL n m L .
(c) Gaps at candidate solutions for that same F F F .
F ( x ^ s ) − min F ≥ 7 δ j ( 0 ≤ j < L , j n m < s ≤ ( j + 1 ) n m ) , F(\widehat{x}_s)-\min F\geq7\delta_j \qquad(0\leq j<L,\;jn_m<s\leq(j+1)n_m), F ( x s ) − min F ≥ 7 δ j ( 0 ≤ j < L , j n m < s ≤ ( j + 1 ) n m ) , F ( x ^ 0 ) − min F ≥ 7 δ 0 . F(\widehat{x}_0)-\min F\geq7\delta_0. F ( x 0 ) − min F ≥ 7 δ 0 .
Proof. We prove Proposition 4.6 together with the prefix-placement claim. Steps 5.1–5.2 establish its completion-family invariant, and Step 5.3 gives the fixed objective and candidate gaps.
Proof of (a). Work at initialization zero, with the normalization in Appx. G.2 fixed before selecting directions. The initial gap D m , L D_{m,L} D m , L is independent of the orthonormal directions by Lemma E.16 (b), so this normalization does not depend on later choices. When d > L ( m + 1 ) d > L(m+1) d > L ( m + 1 ) , the blocks need not span R d \mathbb{R}^{d} R d . The base starts from B 0 ( x ) = μ q ∥ x ∥ q B_0(x)=\frac{\mu}{q}\lVert x\rVert^{q} B 0 ( x ) = q μ ∥ x ∥ q , and its replacements use the full ambient Euclidean norm. Thus its uniform-convexity bound controls the unused directions as well, by Lemma 4.3 (a). Step 5.1: Available directions. Group j j j consists of oracle calls j n m + 1 , … , ( j + 1 ) n m jn_m+1,\ldots,(j+1)n_m j n m + 1 , … , ( j + 1 ) n m , and b b b counts the chosen current-block chain directions. To keep every candidate solution within a prefix where the gap bound applies, we leave room for the zero-call candidate solution and at most three new chain directions per oracle call. For n m = ⌊ m 16 ⌋ n_m=\left\lfloor\frac{m}{16}\right\rfloor n m = ⌊ 16 m ⌋ ,
1 + 3 n m ≤ 3 n m + 2 ≤ 3 m 16 + 2 since n m = ⌊ m 16 ⌋ , ≤ 3 m 16 + m 16 = m 4 since m ≥ 32. \begin{aligned}
1+3n_m\le3n_m+2\le\frac{3m}{16}+2 \qquad\text{since } n_m=\left\lfloor\frac{m}{16}\right\rfloor, \\
&\le\frac{3m}{16}+\frac{m}{16}=\frac{m}{4}
\qquad\text{since }m\ge32.
\end{aligned} 1 + 3 n m ≤ 3 n m + 2 ≤ 16 3 m + 2 since n m = ⌊ 16 m ⌋ , ≤ 16 3 m + 16 m = 4 m since m ≥ 32.
Since 1 + 3 n m 1+3n_m 1 + 3 n m is an integer, this gives 1 + 3 n m ≤ ⌊ m 4 ⌋ 1+3n_m\le\left\lfloor\frac{m}{4}\right\rfloor 1 + 3 n m ≤ ⌊ 4 m ⌋ . Also m 16 ≥ 2 \frac{m}{16}\ge2 16 m ≥ 2 , so the inequality ⌊ x ⌋ ≥ x 2 \lfloor x\rfloor\ge\frac{x}{2} ⌊ x ⌋ ≥ 2 x for x ≥ 2 x\ge2 x ≥ 2 gives
n m = ⌊ m 16 ⌋ ≥ m 16 ⋅ 1 2 = m 32 . n_m=\left\lfloor\frac{m}{16}\right\rfloor\ge\frac{m}{16}\cdot\frac{1}{2}=\frac{m}{32}. n m = ⌊ 16 m ⌋ ≥ 16 m ⋅ 2 1 = 32 m .
All choices below are orthonormally completable: whenever a new direction is required in group j j j ,
dim W j , b ⊥ = d − j ( m + 1 ) − ( b + 1 ) ≥ ( L − j ) ( m + 1 ) − ( b + 1 ) since d ≥ L ( m + 1 ) , ≥ m − b since j ≤ L − 1 , ≥ m − ( 1 + 3 n m ) ≥ 13 m 16 − 1 > 0 since n m ≤ m 16 , m ≥ 32. \begin{aligned}
\dim W_{j,b}^{\perp}
&=d-j(m+1)-(b+1)\\
&\ge(L-j)(m+1)-(b+1) && \text{since }d\ge L(m+1),\\
&\ge m-b && \text{since }j\le L-1,\\
&\ge m-(1+3n_m)\\
&\ge\frac{13m}{16}-1>0 && \text{since }n_m\le\frac{m}{16},\ m\ge32.
\end{aligned} dim W j , b ⊥ = d − j ( m + 1 ) − ( b + 1 ) ≥ ( L − j ) ( m + 1 ) − ( b + 1 ) ≥ m − b ≥ m − ( 1 + 3 n m ) ≥ 16 13 m − 1 > 0 since d ≥ L ( m + 1 ) , since j ≤ L − 1 , since n m ≤ 16 m , m ≥ 32.
Consequently all link selections admit an orthonormal completion. Step 5.2: Identical oracle outputs. Induction invariant. Before oracle call s + 1 s+1 s + 1 , where 0 ≤ s < n m L 0\le s<n_mL 0 ≤ s < n m L , the current group is
j = ⌊ s n m ⌋ . j=\left\lfloor\frac{s}{n_m}\right\rfloor. j = ⌊ n m s ⌋ .
The earlier blocks are complete, and the current prefix is W j , b W_{j,b} W j , b . Let F s \mathcal{F}_s F s be the family of objectives obtained by completing the directions selected after s s s oracle calls, including the direction for the candidate solution and any preparation for the next group. Equality of the recorded oracle outputs across these completions will make Eq. (F.2) reproduce the trajectory, allowing us to apply the prefix gaps to its candidate solutions. We therefore maintain the invariant in Eq. (23) :
F s ≠ ∅ , ∇ k F ( x ℓ ) = a ℓ , k ( F ∈ F s , 1 ≤ ℓ ≤ s , 0 ≤ k ≤ p ) . \mathcal{F}_s\ne\varnothing,\qquad\nabla^k F(x_\ell)=a_{\ell,k}\quad(F\in\mathcal{F}_s,\ 1\le\ell\le s,\ 0\le k\le p). F s = ∅ , ∇ k F ( x ℓ ) = a ℓ , k ( F ∈ F s , 1 ≤ ℓ ≤ s , 0 ≤ k ≤ p ) .
Base case: s = 0 s=0 s = 0 . Initially, we choose v 0 v_0 v 0 , set b = 0 b=0 b = 0 , and use Eq. (F.4) to include x ^ 0 \hat{x}_0 x ^ 0 . Lemma F.3 gives x ^ 0 ∈ W 0 , b \hat{x}_0\in W_{0,b} x ^ 0 ∈ W 0 , b with b ≤ 1 b\le1 b ≤ 1 . Orthonormal completion gives F 0 ≠ ∅ \mathcal{F}_0\ne\varnothing F 0 = ∅ ; there are no oracle output equalities to check.
Induction step: from s s s oracle calls to s + 1 s+1 s + 1 oracle calls. Assume the invariant for F s \mathcal{F}_s F s and use j = ⌊ s n m ⌋ j=\left\lfloor\frac{s}{n_m}\right\rfloor j = ⌊ n m s ⌋ as above. The procedure consists of the following three selections, with b b b updated after each selection. Denote the families after the evaluation point and oracle output selections by F q , F a \mathcal{F}^{q},\mathcal{F}^{a} F q , F a .
Include the evaluation point. The common transcript H s \mathcal{H}_s H s determines x s + 1 = A s + 1 ( H s ) x_{s+1}=\mathcal{A}_{s+1}(\mathcal{H}_s) x s + 1 = A s + 1 ( H s ) by Eq. (F.2). Set ω = x s + 1 − P W j , b x s + 1 \omega=x_{s+1}-P_{W_{j,b}}x_{s+1} ω = x s + 1 − P W j , b x s + 1 . Lemma F.3 gives
{ F q : = F s , b unchanged , ω = 0 , F q : = { F ∈ F s : u j , b + 1 = ω ∥ ω ∥ } , b ← b + 1 , ω ≠ 0. \begin{cases}
\mathcal{F}^{q}:=\mathcal{F}_s,\quad b\text{ unchanged}, & \omega=0,\\
\mathcal{F}^{q}:=\left\{F\in\mathcal{F}_s:u_{j,b+1}=\dfrac{\omega}{\lVert\omega\rVert}\right\},\quad b\leftarrow b+1, & \omega\ne0.
\end{cases} ⎩ ⎨ ⎧ F q := F s , b unchanged , F q := { F ∈ F s : u j , b + 1 = ∥ ω ∥ ω } , b ← b + 1 , ω = 0 , ω = 0.
Thus x s + 1 ∈ W j , b x_{s+1}\in W_{j,b} x s + 1 ∈ W j , b .
Determine and return the oracle output. The evaluation point already lies in W j , b W_{j,b} W j , b , but the current chain can involve u j , b + 1 u_{j,b+1} u j , b + 1 in its derivative tensors. We therefore fix one further unit direction e ⊥ W j , b e\perp W_{j,b} e ⊥ W j , b before returning the oracle output, restricting to
F a : = { F ∈ F q : u j , b + 1 = e } . \mathcal{F}^{a}:=\left\{F\in\mathcal{F}^{q}:u_{j,b+1}=e\right\}. F a := { F ∈ F q : u j , b + 1 = e } .
Lemma F.4 , with the current b b b , gives
∇ k F ( x s + 1 ) = ∇ k F ~ ( x s + 1 ) = : a s + 1 , k ( F , F ~ ∈ F a , 0 ≤ k ≤ p ) . \nabla^{k}F(x_{s+1})=\nabla^{k}\widetilde{F}(x_{s+1})=:a_{s+1,k}\qquad\left(F,\widetilde{F}\in\mathcal{F}^{a},\ 0\le k\le p\right). ∇ k F ( x s + 1 ) = ∇ k F ( x s + 1 ) =: a s + 1 , k ( F , F ∈ F a , 0 ≤ k ≤ p ) .
Return these values and tensors as O s + 1 \mathcal{O}_{s+1} O s + 1 and append ( x s + 1 , O s + 1 ) (x_{s+1},\mathcal{O}_{s+1}) ( x s + 1 , O s + 1 ) to H s \mathcal{H}_s H s , giving H s + 1 \mathcal{H}_{s+1} H s + 1 as in Eq. (F.1). Then set b ← b + 1 b\leftarrow b+1 b ← b + 1 .
Include the candidate solution. The enlarged transcript gives x ^ s + 1 = A ^ s + 1 ( H s + 1 ) \widehat{x}_{s+1}=\widehat{\mathcal{A}}_{s+1}(\mathcal{H}_{s+1}) x s + 1 = A s + 1 ( H s + 1 ) by Eq. (F.2). Set ω = x ^ s + 1 − P W j , b x ^ s + 1 \omega=\widehat{x}_{s+1}-P_{W_{j,b}}\widehat{x}_{s+1} ω = x s + 1 − P W j , b x s + 1 . Lemma F.3 gives
{ F s + 1 : = F a , b unchanged , ω = 0 , F s + 1 : = { F ∈ F a : u j , b + 1 = ω ∥ ω ∥ } , b ← b + 1 , ω ≠ 0. \begin{cases}
\mathcal{F}_{s+1}:=\mathcal{F}^{a},\quad b\text{ unchanged}, & \omega=0,\\
\mathcal{F}_{s+1}:=\left\{F\in\mathcal{F}^{a}:u_{j,b+1}=\dfrac{\omega}{\lVert\omega\rVert}\right\},\quad b\leftarrow b+1, & \omega\ne0.
\end{cases} ⎩ ⎨ ⎧ F s + 1 := F a , b unchanged , F s + 1 := { F ∈ F a : u j , b + 1 = ∥ ω ∥ ω } , b ← b + 1 , ω = 0 , ω = 0.
Hence x s + 1 , x ^ s + 1 ∈ W j , b x_{s+1},\widehat{x}_{s+1}\in W_{j,b} x s + 1 , x s + 1 ∈ W j , b . Each oracle call adds at most three links. Including the zero-call candidate solution when j = 0 j=0 j = 0 , the final prefix length is at most 1 + 3 n m ≤ ⌊ m 4 ⌋ 1+3n_m\le\left\lfloor\frac{m}{4}\right\rfloor 1 + 3 n m ≤ ⌊ 4 m ⌋ .
Further direction choices restrict the completion family, so the induction hypothesis preserves all earlier oracle outputs. The second selection supplies the new oracle output, giving
∅ ≠ F s + 1 ⊆ F a ⊆ F q ⊆ F s , \varnothing\ne\mathcal{F}_{s+1}\subseteq\mathcal{F}^{a}\subseteq\mathcal{F}^{q}\subseteq\mathcal{F}_s, ∅ = F s + 1 ⊆ F a ⊆ F q ⊆ F s ,
∇ k F ( x ℓ ) = a ℓ , k ( F ∈ F s + 1 , 1 ≤ ℓ ≤ s + 1 , 0 ≤ k ≤ p ) . \nabla^{k}F(x_\ell)=a_{\ell,k}\qquad\left(F\in\mathcal{F}_{s+1},\ 1\le\ell\le s+1,\ 0\le k\le p\right). ∇ k F ( x ℓ ) = a ℓ , k ( F ∈ F s + 1 , 1 ≤ ℓ ≤ s + 1 , 0 ≤ k ≤ p ) .
Prepare the next oracle call. If s + 1 < ( j + 1 ) n m s+1<(j+1)n_m s + 1 < ( j + 1 ) n m , retain the current group and prefix. If s + 1 = ( j + 1 ) n m s+1=(j+1)n_m s + 1 = ( j + 1 ) n m , complete E j E_j E j ; when j < L − 1 j<L-1 j < L − 1 , choose v j + 1 v_{j+1} v j + 1 perpendicular to the completed blocks and start group j + 1 j+1 j + 1 with b = 0 b=0 b = 0 . There are at least ( L − j − 1 ) ( m + 1 ) (L-j-1)(m+1) ( L − j − 1 ) ( m + 1 ) dimensions for the remaining blocks. Restrict F s + 1 \mathcal{F}_{s+1} F s + 1 to these choices: it remains nonempty, and the displayed oracle output equalities persist. The recorded evaluation point and candidate solution remain in their group-j j j prefix, whose directions are now fixed. This proves the invariant at s + 1 s+1 s + 1 , including group boundaries. Induction covers all n m L n_mL n m L oracle calls and completes all bases.
∇ k F ( x s ) = a s , k ( 1 ≤ s ≤ n m L , 0 ≤ k ≤ p ) . \nabla^{k}F(x_s)=a_{s,k}\qquad\left(1\le s\le n_mL,\ 0\le k\le p\right). ∇ k F ( x s ) = a s , k ( 1 ≤ s ≤ n m L , 0 ≤ k ≤ p ) .
Lemma F.1 (a) reproduces the recorded evaluation points and oracle outputs on this fixed F F F ; part (b) reproduces every candidate solution, including x ^ 0 \widehat{x}_0 x 0 . If the algorithm stops early, extend its rule arbitrarily after stopping to finish the construction; its recorded stopping candidate solution is unchanged.
x ^ s ∈ W j , b , b ≤ ⌊ m 4 ⌋ ⟹ F ( x ^ s ) − min F ≥ 7 δ j . \widehat{x}_s \in W_{j,b},\quad b \le\left\lfloor\frac{m}{4} \right\rfloor\quad\Longrightarrow\quad F(\widehat{x}_s)-\min F \ge7\delta_j. x s ∈ W j , b , b ≤ ⌊ 4 m ⌋ ⟹ F ( x s ) − min F ≥ 7 δ j .
For s = 0 s=0 s = 0 , use j = 0 j=0 j = 0 and b ≤ 1 b \le1 b ≤ 1 .
Detailed Parameter Calibration and Critical Rescaling
Integer calibration of the chain length and phase count
Lemma G.1 (Calibrating the oracle lower bound). Let a q = 3 q − 2 2 a_q = \frac{3q-2}{2} a q = 2 3 q − 2 and κ = H σ \kappa= \frac{H}{\sigma} κ = σ H . With the constants of Lemma 4.5 , define
c m = ( q C B C H ) 1 a q , κ p , ν = ( 32 c m ) a q , c_m = \left(\frac{qC_B}{C_H}\right)^{\frac{1}{a_q}}, \qquad\kappa_{p,\nu} = \left(\frac{32}{c_m}\right)^{a_q}, c m = ( C H q C B ) a q 1 , κ p , ν = ( c m 32 ) a q ,
c ∗ = 2 c 1 ( 1 − η q ) 2 ( 1 + c 0 ) , c L = 1 q log ( 1 η ) + log ( 2 c ∗ ) . c_* = \frac{2c_1(1-\eta^q)}{2(1+c_0)}, \qquad c_L = \frac{1}{q\log\left(\frac{1}{\eta}\right)+\log\left(\frac{2}{c_*}\right)}. c ∗ = 2 ( 1 + c 0 ) 2 c 1 ( 1 − η q ) , c L = q log ( η 1 ) + log ( c ∗ 2 ) 1 .
Suppose κ ≥ κ p , ν \kappa\ge\kappa_{p,\nu} κ ≥ κ p , ν and 0 < ϵ ≤ c ∗ Δ 0 2 0<\epsilon\le\frac{c_*\Delta_0}{2} 0 < ϵ ≤ 2 c ∗ Δ 0 . Take τ 0 = 1 \tau_0=1 τ 0 = 1 and define
μ = σ q C B , m = ⌊ c m κ 1 a q ⌋ , L = 1 + ⌊ log ( c ∗ Δ 0 ϵ ) q log ( 1 η ) ⌋ . \mu= \frac{\sigma}{qC_B}, \qquad m = \left\lfloor c_m\kappa^{\frac{1}{a_q}}\right\rfloor, \qquad L = 1+\left\lfloor\frac{\log\left(\frac{c_*\Delta_0}{\epsilon}\right)}{q\log\left(\frac{1}{\eta}\right)}\right\rfloor. μ = q C B σ , m = ⌊ c m κ a q 1 ⌋ , L = 1 + q log ( η 1 ) log ( ϵ c ∗ Δ 0 ) .
Then, with n m = ⌊ m 16 ⌋ n_m=\left\lfloor\frac{m}{16}\right\rfloor n m = ⌊ 16 m ⌋ ,
(a) Smoothness, uniform convexity, and per-phase cost. m ≥ 32 m\ge32 m ≥ 32 , q C B μ = σ qC_B\mu=\sigma q C B μ = σ , C H μ a q ≤ H C_H\mu^{a_q}\le H C H μ a q ≤ H , and n m ≥ c m 32 κ 1 a q n_m\ge\frac{c_m}{32}\kappa^{\frac{1}{a_q}} n m ≥ 32 c m κ a q 1 .
(b) Normalized phase gaps and phase count. For 0 ≤ j < L 0\le j<L 0 ≤ j < L ,
7 δ j D m , L ≥ c ∗ η q j , c ∗ Δ 0 η q ( L − 1 ) ≥ 2 ϵ , \frac{7\delta_j}{D_{m,L}}\ge c_*\eta^{qj}, \qquad c_*\Delta_0\eta^{q(L-1)}\ge2\epsilon, D m , L 7 δ j ≥ c ∗ η q j , c ∗ Δ 0 η q ( L − 1 ) ≥ 2 ϵ ,
L ≲ p , ν log ( Δ 0 ϵ ) . L\lesssim_{p,\nu}\log\left(\frac{\Delta_0}{\epsilon}\right). L ≲ p , ν log ( ϵ Δ 0 ) .
(c) Ambient dimension. For K p , ν : = 2 c m ( 1 + 1 q log ( 1 η ) ) K_{p,\nu}:=2c_m\left(1+\frac{1}{q\log\left(\frac{1}{\eta}\right)}\right) K p , ν := 2 c m ( 1 + q l o g ( η 1 ) 1 ) ,
L ( m + 1 ) ≤ K p , ν ( H σ ) 2 3 q − 2 log ( Δ 0 ϵ ) . L(m+1)\le K_{p,\nu}\left(\frac{H}{\sigma}\right)^{\frac{2}{3q-2}}\log\left(\frac{\Delta_0}{\epsilon}\right). L ( m + 1 ) ≤ K p , ν ( σ H ) 3 q − 2 2 log ( ϵ Δ 0 ) .
Proof. Proof of (a). The initial scale τ 0 \tau_0 τ 0 will be adjusted by rescaling, so we first take τ 0 = 1 \tau_0=1 τ 0 = 1 and define κ = H σ \kappa=\frac{H}{\sigma} κ = σ H . By Lemma 4.5 (a), the theorem’s two conditions hold provided q C B μ ≥ σ qC_B\mu\ge\sigma q C B μ ≥ σ and C H μ m a q ≤ H C_H\mu m^{a_q} \le H C H μ m a q ≤ H . Equating the first coefficient to σ \sigma σ allows the largest chain length under the second inequality. Indeed, for μ = σ q C B \mu= \frac{\sigma}{qC_B} μ = q C B σ , the smoothness requirement becomes
C H μ m a q ≤ H ⟺ m a q ≤ H C H μ ⟺ m a q ≤ q C B H C H σ ⟺ m ≤ c m κ 1 a q \begin{aligned}
C_H\mu m^{a_q} &\le H \\
&\Longleftrightarrow\quad m^{a_q} \le\frac{H}{C_H\mu} \\
&\Longleftrightarrow\quad m^{a_q} \le\frac{qC_B H}{C_H\sigma} \\
&\Longleftrightarrow\quad m \le c_m\kappa^{\frac{1}{a_q}}
\end{aligned} C H μ m a q ≤ H ⟺ m a q ≤ C H μ H ⟺ m a q ≤ C H σ q C B H ⟺ m ≤ c m κ a q 1
by the definitions of c m , κ c_m,\kappa c m , κ .
Our choice m = ⌊ c m κ 1 a q ⌋ m = \left\lfloor c_m\kappa^{\frac{1}{a_q}}\right\rfloor m = ⌊ c m κ a q 1 ⌋ is the largest integer allowed by this bound. Proposition E.5 in Appx. E supplies compatible phase coefficients for this choice. For κ ≥ κ p , ν \kappa\ge\kappa_{p,\nu} κ ≥ κ p , ν , define x : = c m κ 1 a q ≥ 32 x := c_m\kappa^{\frac{1}{a_q}} \ge32 x := c m κ a q 1 ≥ 32 . Then m = ⌊ x ⌋ ≥ 32 m = \lfloor x\rfloor\ge32 m = ⌊ x ⌋ ≥ 32 and, for all u , v ∈ R d u,v \in\mathbb{R}^d u , v ∈ R d ,
∥ ∇ p F ( u ) − ∇ p F ( v ) ∥ ≤ C H μ m a q ∥ u − v ∥ ν ≤ H ∥ u − v ∥ ν \begin{aligned}
\left\|\nabla^p F(u)-\nabla^p F(v)\right\| &\le C_H\mu m^{a_q}\left\|u-v\right\|^\nu\\
&\le H\left\|u-v\right\|^\nu
\end{aligned} ∥ ∇ p F ( u ) − ∇ p F ( v ) ∥ ≤ C H μ m a q ∥ u − v ∥ ν ≤ H ∥ u − v ∥ ν
by the chosen m m m ,
D F ( v , u ) ≥ c B μ ∥ v − u ∥ q = σ q ∥ v − u ∥ q D_F(v,u) \ge c_B\mu\left\|v-u\right\|^q = \frac{\sigma}{q}\left\|v-u\right\|^q D F ( v , u ) ≥ c B μ ∥ v − u ∥ q = q σ ∥ v − u ∥ q
since μ = σ q C B \mu= \frac{\sigma}{qC_B} μ = q C B σ .
Thus Proposition 4.6 applies, with the per-phase number of calls bounded by
n m = ⌊ ⌊ x ⌋ 16 ⌋ = ⌊ x 16 ⌋ ≥ x 32 \begin{aligned}
n_m &= \left\lfloor\frac{\lfloor x\rfloor}{16}\right\rfloor\\
&= \left\lfloor\frac{x}{16}\right\rfloor\\
&\ge\frac{x}{32}
\end{aligned} n m = ⌊ 16 ⌊ x ⌋ ⌋ = ⌊ 16 x ⌋ ≥ 32 x
since x 16 ≥ 2 \frac{x}{16}\ge2 16 x ≥ 2 , ⌊ t ⌋ ≥ t 2 \lfloor t\rfloor\ge\frac{t}{2} ⌊ t ⌋ ≥ 2 t ,
= c m 32 κ 3 q − 2 = \frac{c_m}{32\kappa^{3q-2}} = 32 κ 3 q − 2 c m
since 1 a q = 2 3 q − 2 \frac{1}{a_q}=\frac{2}{3q-2} a q 1 = 3 q − 2 2 .
For the second equality, define n = ⌊ x 16 ⌋ n=\left\lfloor\frac{x}{16}\right\rfloor n = ⌊ 16 x ⌋ . Then 16 n ≤ x < 16 ( n + 1 ) 16n\le x<16(n+1) 16 n ≤ x < 16 ( n + 1 ) , and the integer endpoints imply 16 n ≤ ⌊ x ⌋ < 16 ( n + 1 ) 16n\le\lfloor x\rfloor<16(n+1) 16 n ≤ ⌊ x ⌋ < 16 ( n + 1 ) . Dividing by 16 and taking floors gives ⌊ ⌊ x ⌋ 16 ⌋ = n \left\lfloor\frac{\lfloor x\rfloor}{16}\right\rfloor=n ⌊ 16 ⌊ x ⌋ ⌋ = n .
Proof of (b). The choices above meet the smoothness and uniform-convexity bounds and fix the number of calls required in each phase, independently of L L L . We can therefore increase the total oracle cost by taking as many phases as possible while keeping their candidate gaps above ϵ \epsilon ϵ after normalization. Rescaling to Δ 0 \Delta_0 Δ 0 multiplies each candidate gap by Δ 0 D m , L \frac{\Delta_0}{D_{m,L}} D m , L Δ 0 , so we compare 7 δ j 7\delta_j 7 δ j from Proposition 4.6 to D m , L D_{m,L} D m , L .
Compare the phase gaps with the initial gap. Lemma E.16 (a) in Appx. E and part (b) give, for every L ≥ 1 L \ge1 L ≥ 1 ,
D m , L = ∑ j = 0 L − 1 χ j by xref‘Eq. (E.16)‘ , ≤ 2 ( 1 + c 0 ) λ m ∑ j = 0 L − 1 τ j q by xref‘Eq. (E.15)‘ , = 2 ( 1 + c 0 ) λ m τ 0 q ∑ j = 0 L − 1 η q j ≤ 2 ( 1 + c 0 ) 1 − η q λ m τ 0 q since 0 < η q < 1 , 7 δ j D m , L = 7 c 1 λ m τ 0 q η q j D m , L by the definition of δ j , ≥ 7 c 1 ( 1 − η q ) 2 ( 1 + c 0 ) η q j = c ∗ η q j ( 0 ≤ j < L ) . \begin{aligned}
D_{m,L} &= \sum_{j=0}^{L-1} \chi_j && \text{by {xref}`Eq. (E.16)`},\\
&\le2(1+c_0)\lambda m\sum_{j=0}^{L-1}\tau_j^q && \text{by {xref}`Eq. (E.15)`},\\
&=2(1+c_0)\lambda m\tau_0^q\sum_{j=0}^{L-1}\eta^{qj}\\
&\le\frac{2(1+c_0)}{1-\eta^q}\lambda m\tau_0^q && \text{since }0<\eta^q<1,\\
\frac{7\delta_j}{D_{m,L}}&=\frac{7c_1\lambda m\tau_0^q\eta^{qj}}{D_{m,L}} && \text{by the definition of }\delta_j,\\
&\ge\frac{7c_1(1-\eta^q)}{2(1+c_0)}\eta^{qj}=c_*\eta^{qj} && (0\le j<L).
\end{aligned} D m , L D m , L 7 δ j = j = 0 ∑ L − 1 χ j ≤ 2 ( 1 + c 0 ) λm j = 0 ∑ L − 1 τ j q = 2 ( 1 + c 0 ) λm τ 0 q j = 0 ∑ L − 1 η q j ≤ 1 − η q 2 ( 1 + c 0 ) λm τ 0 q = D m , L 7 c 1 λm τ 0 q η q j ≥ 2 ( 1 + c 0 ) 7 c 1 ( 1 − η q ) η q j = c ∗ η q j by xref ‘Eq. (E.16)‘ , by xref ‘Eq. (E.15)‘ , since 0 < η q < 1 , by the definition of δ j , ( 0 ≤ j < L ) .
Thus c ∗ c_* c ∗ is independent of L L L , and 0 < c ∗ < 7 128 < 1 0<c_*<\frac{7}{128}<1 0 < c ∗ < 128 7 < 1 because c 1 < 1 64 c_1<\frac{1}{64} c 1 < 64 1 . Lemma G.2 verifies the stated effect of rescaling. Since 0 < η < 1 0<\eta<1 0 < η < 1 , a last-phase gap above 2 ϵ 2\epsilon 2 ϵ also keeps all earlier gaps above 2 ϵ 2\epsilon 2 ϵ , as
7 δ j D m , L Δ 0 ≥ c ∗ Δ 0 η q j ≥ c ∗ Δ 0 η q ( L − 1 ) ( 0 ≤ j < L ) . \frac{7\delta_j}{D_{m,L}}\Delta_0\ge c_*\Delta_0\eta^{qj}\ge c_*\Delta_0\eta^{q(L-1)}\qquad(0\le j<L). D m , L 7 δ j Δ 0 ≥ c ∗ Δ 0 η q j ≥ c ∗ Δ 0 η q ( L − 1 ) ( 0 ≤ j < L ) .
Choose the largest phase count keeping the last gap above 2 ϵ 2\epsilon 2 ϵ . For ϵ ≤ c ∗ Δ 0 2 \epsilon\le\frac{c_*\Delta_0}{2} ϵ ≤ 2 c ∗ Δ 0 , solving c ∗ Δ 0 η q ( L − 1 ) ≥ 2 ϵ c_*\Delta_0\eta^{q(L-1)}\ge2\epsilon c ∗ Δ 0 η q ( L − 1 ) ≥ 2 ϵ for the largest integer L L L gives
c ∗ Δ 0 η q ( L − 1 ) ≥ 2 ϵ ⟺ q ( L − 1 ) log η ≥ log ( 2 ϵ c ∗ Δ 0 ) c_*\Delta_0\eta^{q(L-1)}\ge2\epsilon
\quad\Longleftrightarrow\quad
q(L-1)\log\eta\ge\log\left(\frac{2\epsilon}{c_*\Delta_0}\right) c ∗ Δ 0 η q ( L − 1 ) ≥ 2 ϵ ⟺ q ( L − 1 ) log η ≥ log ( c ∗ Δ 0 2 ϵ )
⟺ L − 1 ≤ log ( c ∗ Δ 0 2 ϵ ) q log ( 1 η ) since log η < 0. \Longleftrightarrow\quad
L-1\le\frac{\log\left(\frac{c_*\Delta_0}{2\epsilon}\right)}{q\log\left(\frac{1}{\eta}\right)}
\qquad\text{since }\log\eta<0. ⟺ L − 1 ≤ q log ( η 1 ) log ( 2 ϵ c ∗ Δ 0 ) since log η < 0.
Denoting the upper bound by u u u , this choice is
u : = log ( c ∗ Δ 0 2 ϵ ) q log ( 1 η ) ≥ 0 , L : = 1 + ⌊ u ⌋ ≥ 1. u:=\frac{\log\left(\frac{c_*\Delta_0}{2\epsilon}\right)}{q\log\left(\frac{1}{\eta}\right)}\ge0,\qquad L:=1+\lfloor u\rfloor\ge1. u := q log ( η 1 ) log ( 2 ϵ c ∗ Δ 0 ) ≥ 0 , L := 1 + ⌊ u ⌋ ≥ 1.
Indeed, L − 1 ≤ u L-1\le u L − 1 ≤ u gives
η q ( L − 1 ) ≥ η q u = exp ( − log ( c ∗ Δ 0 2 ϵ ) ) = 2 ϵ c ∗ Δ 0 , \eta^{q(L-1)}\ge\eta^{qu}=\exp\left(-\log\left(\frac{c_*\Delta_0}{2\epsilon}\right)\right)=\frac{2\epsilon}{c_*\Delta_0}, η q ( L − 1 ) ≥ η q u = exp ( − log ( 2 ϵ c ∗ Δ 0 ) ) = c ∗ Δ 0 2 ϵ ,
which proves the lower bound on the last normalized phase gap.
Verify the logarithmic phase count. This choice also gives L ≥ c L log ( Δ 0 ϵ ) L\ge c_L\log\left(\frac{\Delta_0}{\epsilon}\right) L ≥ c L log ( ϵ Δ 0 ) . To verify it, define z : = log ( Δ 0 ϵ ) z := \log\left(\frac{\Delta_0}{\epsilon}\right) z := log ( ϵ Δ 0 ) and b ∗ : = log ( 2 c ∗ ) > 0 b_* := \log\left(\frac{2}{c_*}\right) > 0 b ∗ := log ( c ∗ 2 ) > 0 . Then z ≥ b ∗ z \ge b_* z ≥ b ∗ , u = z − b ∗ q log ( 1 η ) < L u = \frac{z-b_*}{q\log\left(\frac{1}{\eta}\right)} < L u = q l o g ( η 1 ) z − b ∗ < L , and L ≥ 1 L \ge1 L ≥ 1 imply
z = q log ( 1 η ) u + b ∗ < q log ( 1 η ) L + b ∗ ≤ ( q log ( 1 η ) + b ∗ ) L = L c L . \begin{aligned}
z = q\log\left(\frac{1}{\eta}\right)u+b_* \\
&< q\log\left(\frac{1}{\eta}\right)L+b_* \\
&\le\left(q\log\left(\frac{1}{\eta}\right)+b_*\right)L = \frac{L}{c_L}.
\end{aligned} z = q log ( η 1 ) u + b ∗ < q log ( η 1 ) L + b ∗ ≤ ( q log ( η 1 ) + b ∗ ) L = c L L .
Hence L ≥ c L log ( Δ 0 ϵ ) L \ge c_L\log\left(\frac{\Delta_0}{\epsilon}\right) L ≥ c L log ( ϵ Δ 0 ) , proving the phase-count bound. For the corresponding upper bound, u ≤ z q log ( 1 η ) u \le\frac{z}{q\log\left(\frac{1}{\eta}\right)} u ≤ q l o g ( η 1 ) z and z ≥ b ∗ z \ge b_* z ≥ b ∗ give
L = 1 + ⌊ u ⌋ ≤ 1 + u ≤ ( 1 b ∗ + 1 q log ( 1 η ) ) z . L = 1+\lfloor u\rfloor\le1+u \le\left(\frac{1}{b_*}+\frac{1}{q\log\left(\frac{1}{\eta}\right)}\right)z. L = 1 + ⌊ u ⌋ ≤ 1 + u ≤ b ∗ 1 + q log ( η 1 ) 1 z .
Thus L ≲ p , ν log ( Δ 0 ϵ ) L \lesssim_{p,\nu}\log\left(\frac{\Delta_0}{\epsilon}\right) L ≲ p , ν log ( ϵ Δ 0 ) , as used in Step 6.
Proof of (c). The direction-selection argument requires L ( m + 1 ) L(m+1) L ( m + 1 ) dimensions. With x = c m κ 1 a q q ≥ 32 x=c_m\kappa^{\frac{1}{a_q}q} \ge32 x = c m κ a q 1 q ≥ 32 and z = log ( Δ 0 ϵ ) z=\log\left(\frac{\Delta_0}{\epsilon}\right) z = log ( ϵ Δ 0 ) as above, the accuracy restriction gives z ≥ log ( 2 c ∗ ) > 1 z\ge\log\left(\frac{2}{c_*}\right)>1 z ≥ log ( c ∗ 2 ) > 1 because c ∗ < 7 128 c_*<\frac{7}{128} c ∗ < 128 7 . Consequently,
m + 1 ≤ x + 1 ≤ 2 x since m = ⌊ x ⌋ , x ≥ 32 , m+1 \le x+1 \le2x \qquad\text{since } m=\lfloor x\rfloor,\ x\ge32, m + 1 ≤ x + 1 ≤ 2 x since m = ⌊ x ⌋ , x ≥ 32 ,
L ≤ 1 + z q log ( 1 η ) ≤ ( 1 + 1 q log ( 1 η ) ) z since z > 1 , L \le1+\frac{z}{q\log\left(\frac{1}{\eta}\right)} \le\left(1+\frac{1}{q\log\left(\frac{1}{\eta}\right)}\right)z \qquad\text{since } z>1, L ≤ 1 + q log ( η 1 ) z ≤ 1 + q log ( η 1 ) 1 z since z > 1 ,
L ( m + 1 ) ≤ 2 c m ( 1 + 1 q log ( 1 η ) ) κ 1 a q q z = K p , ν ( H σ ) 2 3 q − 2 log ( Δ 0 ϵ ) by 1 a q = 2 3 q − 2 . \begin{aligned}
L(m+1) \le2c_m\left(1+\frac{1}{q\log\left(\frac{1}{\eta}\right)}\right)\kappa^{\frac{1}{a_q}q}z \\
&= K_{p,\nu}\left(\frac{H}{\sigma}\right)^{\frac{2}{3q-2}}\log\left(\frac{\Delta_0}{\epsilon}\right) \qquad\text{by } \frac{1}{a_q}=\frac{2}{3q-2}.
\end{aligned} L ( m + 1 ) ≤ 2 c m 1 + q log ( η 1 ) 1 κ a q 1 q z = K p , ν ( σ H ) 3 q − 2 2 log ( ϵ Δ 0 ) by a q 1 = 3 q − 2 2 .
□ \square □
To use one constant in the accuracy range and oracle lower bound of Theorem 1.1 , choose c p , ν : = min { c ∗ 2 , c m c L 32 } c_{p,\nu}:=\min\left\{\frac{c_*}{2},\frac{c_mc_L}{32}\right\} c p , ν := min { 2 c ∗ , 32 c m c L } . The first term ensures ϵ ≤ c ∗ Δ 0 2 \epsilon\le\frac{c_*\Delta_0}{2} ϵ ≤ 2 c ∗ Δ 0 ; the second is the coefficient in the product of the two count bounds.
Critical rescaling
The choice of L L L compares the phase gaps with the target ϵ \epsilon ϵ for an initial gap of Δ 0 \Delta_0 Δ 0 , whereas the current objective has initial gap D m , L D_{m,L} D m , L . We now rescale the objective to make this comparison apply. Eq. (E.16) determines D m , L D_{m,L} D m , L independently of the chain directions, so we can fix
a = ( Δ 0 D m , L ) 1 q a=\left(\frac{\Delta_0}{D_{m,L}}\right)^{\frac{1}{q}} a = ( D m , L Δ 0 ) q 1
before applying the trajectory construction. The following lemma verifies that this normalization adjusts the initial gap while preserving both parameters because q = p + ν q=p+\nu q = p + ν .
Lemma G.2 (Rescaling preserves the critical class). Let F F F be the objective in Definition E.8. For a > 0 a>0 a > 0 and an initialization x 0 \mathbf{x}_0 x 0 , define
F a ( x ) : = a q F ( x a ) , F ^ ( x ) : = F a ( x − x 0 ) . F_a(\mathbf{x}) := a^q F\left(\frac{\mathbf{x}}{a}\right), \qquad\widehat{F}(\mathbf{x}) := F_a(\mathbf{x}-\mathbf{x}_0). F a ( x ) := a q F ( a x ) , F ( x ) := F a ( x − x 0 ) . For 0 ≤ k ≤ p 0\leq k\leq p 0 ≤ k ≤ p , with ∇ 0 F = F \nabla^0 F=F ∇ 0 F = F ,
∇ k F ^ ( x ) = a q − k ∇ k F ( x − x 0 a ) . \nabla^k \widehat{F}(\mathbf{x}) = a^{q-k}\nabla^k F\left(\frac{\mathbf{x}-\mathbf{x}_0}{a}\right). ∇ k F ( x ) = a q − k ∇ k F ( a x − x 0 ) . Every p th p^{\text{th}} p th -order Hölder smoothness parameter H H H of F F F is preserved: for all x , y ∈ R d \mathbf{x},\mathbf{y}\in\mathbb{R}^d x , y ∈ R d ,
∥ ∇ p F ^ ( x ) − ∇ p F ^ ( y ) ∥ ≤ H ∥ x − y ∥ ν , \left\|\nabla^p\widehat{F}(\mathbf{x})-\nabla^p\widehat{F}(\mathbf{y})\right\|\leq H\|\mathbf{x}-\mathbf{y}\|^\nu, ∇ p F ( x ) − ∇ p F ( y ) ≤ H ∥ x − y ∥ ν , D F ^ ( y , x ) = a q D F ( y − x 0 a , x − x 0 a ) . (167) D_{\widehat{F}}(\mathbf{y},\mathbf{x})=a^qD_F\left(\frac{\mathbf{y}-\mathbf{x}_0}{a},\frac{\mathbf{x}-\mathbf{x}_0}{a}\right). \tag*{(167)} D F ( y , x ) = a q D F ( a y − x 0 , a x − x 0 ) . (167) Every admissible degree-q q q uniform-convexity modulus is preserved, and
argmin x ∈ R d F ^ ( x ) = { x 0 + a c L } , F ^ ( x 0 ) − min F ^ = a q D m , L . \underset{\mathbf{x}\in\mathbb{R}^d}{\operatorname{argmin}}\widehat{F}(\mathbf{x})=\{\mathbf{x}_0+a c_L\}, \qquad\widehat{F}(\mathbf{x}_0)-\min\widehat{F}=a^qD_{m,L}. x ∈ R d argmin F ( x ) = { x 0 + a c L } , F ( x 0 ) − min F = a q D m , L . In particular, a = ( Δ 0 D m , L ) 1 q a=\left(\frac{\Delta_0}{D_{m,L}}\right)^{\frac{1}{q}} a = ( D m , L Δ 0 ) q 1 gives F ^ ( x 0 ) − min F ^ = Δ 0 \widehat{F}(\mathbf{x}_0)-\min\widehat{F}=\Delta_0 F ( x 0 ) − min F = Δ 0 .
Proof. Define u = x − x 0 a \mathbf{u}=\frac{\mathbf{x}-\mathbf{x}_0}{a} u = a x − x 0 and v = y − x 0 a \mathbf{v}=\frac{\mathbf{y}-\mathbf{x}_0}{a} v = a y − x 0 . For 1 ≤ k ≤ p 1\leq k\leq p 1 ≤ k ≤ p and arbitrary test vectors z 1 , … , z k \mathbf{z}_1,\ldots,\mathbf{z}_k z 1 , … , z k ,
∇ k F ^ ( x ) [ z 1 , … , z k ] = a q ∇ k F ( u ) [ z 1 a , … , z k a ] \nabla^k\widehat{F}(\mathbf{x})[\mathbf{z}_1,\ldots,\mathbf{z}_k]
=a^q\nabla^kF(\mathbf{u})\left[\frac{\mathbf{z}_1}{a},\ldots,\frac{\mathbf{z}_k}{a}\right] ∇ k F ( x ) [ z 1 , … , z k ] = a q ∇ k F ( u ) [ a z 1 , … , a z k ] by the linear chain rule,
= a q − k ∇ k F ( u ) [ z 1 , … , z k ] =a^{q-k}\nabla^kF(\mathbf{u})[\mathbf{z}_1,\ldots,\mathbf{z}_k] = a q − k ∇ k F ( u ) [ z 1 , … , z k ] by multilinearity. This proves the full derivative identity; k = 0 k=0 k = 0 is the definition of F ^ \widehat{F} F . For any smoothness parameter H H H of F F F ,
∥ ∇ p F ^ ( x ) − ∇ p F ^ ( y ) ∥ = a q − p ∥ ∇ p F ( u ) − ∇ p F ( v ) ∥ by the derivative identity above, ≤ a q − p H ∥ u − v ∥ ν by the H o ¨ lder smoothness assumption on F , = a q − p − ν H ∥ x − y ∥ ν since u − v = x − y a , = H ∥ x − y ∥ ν since q = p + ν . \begin{aligned}
\left\|\nabla^p\widehat{F}(\mathbf{x})-\nabla^p\widehat{F}(\mathbf{y})\right\|
&=a^{q-p}\left\|\nabla^pF(\mathbf{u})-\nabla^pF(\mathbf{v})\right\| &&\text{by the derivative identity above,}\\
&\leq a^{q-p}H\|\mathbf{u}-\mathbf{v}\|^\nu&&\text{by the Hölder smoothness assumption on }F,\\
&=a^{q-p-\nu}H\|\mathbf{x}-\mathbf{y}\|^\nu&&\text{since }\mathbf{u}-\mathbf{v}=\frac{\mathbf{x}-\mathbf{y}}{a},\\
&=H\|\mathbf{x}-\mathbf{y}\|^\nu&&\text{since }q=p+\nu.
\end{aligned} ∇ p F ( x ) − ∇ p F ( y ) = a q − p ∥ ∇ p F ( u ) − ∇ p F ( v ) ∥ ≤ a q − p H ∥ u − v ∥ ν = a q − p − ν H ∥ x − y ∥ ν = H ∥ x − y ∥ ν by the derivative identity above, by the H o ¨ lder smoothness assumption on F , since u − v = a x − y , since q = p + ν . For uniform convexity, we first expand the Bregman divergence. Using y − x = a ( v − u ) \mathbf{y}-\mathbf{x}=a(\mathbf{v}-\mathbf{u}) y − x = a ( v − u ) gives
D F ^ ( y , x ) = a q ( F ( v ) − F ( u ) ) − a q − 1 ⟨ ∇ F ( u ) , y − x ⟩ by the definition and chain rule, = a q ( F ( v ) − F ( u ) − ⟨ ∇ F ( u ) , v − u ⟩ ) since y − x = a ( v − u ) , = a q D F ( v , u ) by the Bregman definition. \begin{aligned}
D_{\widehat{F}}(\mathbf{y},\mathbf{x})
&=a^q\bigl(F(\mathbf{v})-F(\mathbf{u})\bigr)-a^{q-1}\langle\nabla F(\mathbf{u}),\mathbf{y}-\mathbf{x}\rangle&&\text{by the definition and chain rule,}\\
&=a^q\bigl(F(\mathbf{v})-F(\mathbf{u})-\langle\nabla F(\mathbf{u}),\mathbf{v}-\mathbf{u}\rangle\bigr) &&\text{since }\mathbf{y}-\mathbf{x}=a(\mathbf{v}-\mathbf{u}),\\
&=a^qD_F(\mathbf{v},\mathbf{u}) &&\text{by the Bregman definition.}
\end{aligned} D F ( y , x ) = a q ( F ( v ) − F ( u ) ) − a q − 1 ⟨ ∇ F ( u ) , y − x ⟩ = a q ( F ( v ) − F ( u ) − ⟨ ∇ F ( u ) , v − u ⟩ ) = a q D F ( v , u ) by the definition and chain rule, since y − x = a ( v − u ) , by the Bregman definition.
If F F F has modulus σ \sigma σ , this identity gives
D F ^ ( y , x ) = a q D F ( v , u ) by the identity above, ≥ a q σ q ∥ v − u ∥ q by uniform convexity of F , = a q σ a q q ∥ y − x ∥ q since v − u = y − x a , = σ q ∥ y − x ∥ q . \begin{aligned}
D_{\widehat{F}}(\mathbf{y},\mathbf{x}) = a^{q}D_{F}(\mathbf{v},\mathbf{u}) \qquad\text{by the identity above,} \\
&\ge a^{q}\frac{\sigma}{q}\lVert\mathbf{v}-\mathbf{u}\rVert^{q} \qquad\text{by uniform convexity of }F, \\
&= \frac{a^{q}\sigma}{a^{q}q}\lVert\mathbf{y}-\mathbf{x}\rVert^{q} \qquad\text{since }\mathbf{v}-\mathbf{u}=\frac{\mathbf{y}-\mathbf{x}}{a}, \\
&= \frac{\sigma}{q}\lVert\mathbf{y}-\mathbf{x}\rVert^{q}.
\end{aligned} D F ( y , x ) = a q D F ( v , u ) by the identity above, ≥ a q q σ ∥ v − u ∥ q by uniform convexity of F , = a q q a q σ ∥ y − x ∥ q since v − u = a y − x , = q σ ∥ y − x ∥ q .
The affine change of variables is bijective and a q > 0 a^{q}>0 a q > 0 . Lemma E.11 (c) therefore gives
argmin x F ^ ( x ) = { x 0 + a u : u ∈ argmin u F ( u ) } = { x 0 + a c L } . \begin{aligned}
\underset{\mathbf{x}}{\operatorname{argmin}}\ \widehat{F}(\mathbf{x})=
\left\{\mathbf{x}_{0}+a\mathbf{u}:\mathbf{u}\in
\underset{\mathbf{u}}{\operatorname{argmin}}\ F(\mathbf{u})\right\} \\
&=\{\mathbf{x}_{0}+a\mathbf{c}_{L}\}.
\end{aligned} x argmin F ( x ) = { x 0 + a u : u ∈ u argmin F ( u ) } = { x 0 + a c L } .
Using this minimizer and the definition of D m , L D_{m,L} D m , L ,
F ^ ( x 0 ) − min F ^ = a q F ( 0 ) − a q F ( c L ) = a q ( F ( 0 ) − F ( c L ) ) by factoring a q , = a q D m , L by xref‘Eq. (E.16)’. \begin{aligned}
\widehat{F}(\mathbf{x}_{0})-\min\widehat{F}
=a^{q}F(\mathbf{0})-a^{q}F(\mathbf{c}_{L}) \\
&=a^{q}\bigl(F(\mathbf{0})-F(\mathbf{c}_{L})\bigr)
\qquad\text{by factoring }a^{q}, \\
&=a^{q}D_{m,L}
\qquad\text{by {xref}`Eq. (E.16)'.}
\end{aligned} F ( x 0 ) − min F = a q F ( 0 ) − a q F ( c L ) = a q ( F ( 0 ) − F ( c L ) ) by factoring a q , = a q D m , L by xref ‘Eq. (E.16)’.
For the normalizing scale fixed before the trajectory construction, the last expression is
a q D m , L = ( Δ 0 D m , L ) D m , L by the choice of a , = Δ 0 since D m , L > 0 by xref‘Lemma E.16 (a)’. \begin{aligned}
a^{q}D_{m,L}=\left(\frac{\Delta_{0}}{D_{m,L}}\right)D_{m,L}
\qquad\text{by the choice of }a, \\
&=\Delta_{0}
\qquad\text{since }D_{m,L}>0\text{ by {xref}`Lemma E.16 (a)'.}
\end{aligned} a q D m , L = ( D m , L Δ 0 ) D m , L by the choice of a , = Δ 0 since D m , L > 0 by xref ‘Lemma E.16 (a)’.