Introduction
We consider online convex optimization (OCO) over a convex compact decision set K ⊂ R n K \subset\mathbb{R}^{n} K ⊂ R n (also called, for brevity, a body) of diameter at most D D D , where a learner and an adversary play for T T T rounds. At round t t t , the learner plays x t ∈ K x_t \in K x t ∈ K , the adversary reveals a convex L L L -Lipschitz loss f t f_t f t , and the learner suffers f t ( x t ) f_t(x_t) f t ( x t ) and observes a subgradient g t ∈ ∂ f t ( x t ) g_t \in\partial f_t(x_t) g t ∈ ∂ f t ( x t ) . The quality of the sequence x 0 , x 1 , … , x T − 1 x_0,x_1,\ldots,x_{T-1} x 0 , x 1 , … , x T − 1 is then quantified by its regret against the best fixed point in hindsight,
Reg T = ∑ t < T f t ( x t ) − min u ∈ K ∑ t < T f t ( u ) . \operatorname{Reg}_{T} = \sum_{t<T} f_t(x_t) - \min_{u\in K} \sum_{t<T} f_t(u). Reg T = t < T ∑ f t ( x t ) − u ∈ K min t < T ∑ f t ( u ) .
OCO is detailed in [8 , 18 ] . Online gradient descent [23 ] guarantees the minimax rate Reg T = O ( L D T ) \operatorname{Reg}_{T} = O(LD\sqrt{T}) Reg T = O ( L D T ) but requires a Euclidean projection onto K K K at every round. For many sets of interest, such as nuclear-norm balls and flow, matching or matroid polytopes, the projection is the bottleneck, while a linear minimization oracle (LMO) over K K K is cheap, e.g., via a leading singular pair or a combinatorial subroutine. Projection-free methods replace the projection by an LMO, which returns a minimizer of ⟨ d , ⋅ ⟩ \langle d,\cdot\rangle ⟨ d , ⋅ ⟩ over K K K for a given direction d d d . In offline convex optimization, this yields the classical Frank–Wolfe method [4 ] . The Online Frank–Wolfe (OFW), introduced by Hazan and Kale [9 ] , calls the LMO once per round using the gradient of a regularized cumulative loss, and moves a fraction of the way towards the returned vertex. OFW guarantees Reg T = O ( L D T 3 / 4 ) \operatorname{Reg}_{T} = O(LD T^{3/4}) Reg T = O ( L D T 3/4 ) for convex Lipschitz losses [9 , 8 ] .
The central question of this paper is whether the gap between the O ( T ) O(\sqrt{T}) O ( T ) and the O ( T 3 / 4 ) O(T^{3/4}) O ( T 3/4 ) can be bridged. In particular, is the O ( T 3 / 4 ) O(T^{3/4}) O ( T 3/4 ) precisely the price to pay for accessing K K K only through m = O ( 1 ) m=O(1) m = O ( 1 ) linear optimization calls per round, or is it an artifact of the current analysis and parameter choices in OFW? The question is stated as open by Mhammedi [15 ] and by Weibel et al. [20 ] . The latter reports an optimized variant of the OFW obtained through performance estimation problems [7 ] , along with strong numerical evidence that no method within the class of OFW, that performs no more than m m m LMO calls per round beats O ( T 3 / 4 ) O(T^{3/4}) O ( T 3/4 ) . This work answers positively: in high dimension (n ≥ T n \ge T n ≥ T ), Ω ( T 3 / 4 m − 1 / 4 ) \Omega(T^{3/4}m^{-1/4}) Ω ( T 3/4 m − 1/4 ) is the price of the LMO model.
Related work
In projection-free online learning, beating the O ( T 3 / 4 ) O(T^{3/4}) O ( T 3/4 ) rate with m = 1 m = 1 m = 1 requires either stronger oracles than LMOs or extra assumptions on the decision set K K K or on the losses. Membership oracles (Levy and Krause, [12 ] ; Mhammedi, [14 ] ) and separation oracles (Garber and Kretzu, [6 ] ; Mhammedi, [15 ] ) lead to T \sqrt{T} T rates. Here, we assume that the learner only has access to an LMO. With one LMO call per round, strongly convex decision sets allow T 2 / 3 T^{2/3} T 2/3 for convex losses (Wan and Zhang, [19 ] ), and curvature of the set can be exploited further (Mhammedi, [13 ] ). On polytopes, local LMOs give T \sqrt{T} T with constants depending on the dimension and geometry (Garber and Hazan, [5 ] ).
For decision sets of finite dimension n n n , Follow-the-Perturbed-Leader (FTPL) achieves O ( n 1 / 4 T ) O(n^{1/4}\sqrt{T}) O ( n 1/4 T ) (Kalai and Vempala, [11 ] ; Abernethy et al., [1 ] ) and O ( n T 2 / 3 ) O(\sqrt{n}T^{2/3}) O ( n T 2/3 ) (Hazan and Minasyan, [10 ] ) on linear and smooth losses respectively, but with dimension-dependent multiplicative constants that blow up with n n n . In particular, FTPL may achieve o ( T 3 / 4 ) o(T^{3/4}) o ( T 3/4 ) regret when n = o ( T ) n = o(T) n = o ( T ) . So no lower bound of order T 3 / 4 T^{3/4} T 3/4 can hold for randomized learners on bodies of dimension o ( T ) o(T) o ( T ) , and the dimension of a hard instance must grow with T T T . In fixed dimension, polynomially many oracle calls also simulate a separation oracle by cutting-plane methods. Our lower bounds are therefore valid under a classical large-scale assumption : the dimension must be larger than 2 m T 2mT 2 m T for span methods (see definition in § 1.3 ) and ≳ ( m T ) 7 / 2 \gtrsim(mT)^{7/2} ≳ ( m T ) 7/2 for arbitrary learners.
Our lower-bound hard instance for span methods adapts the chain principle of the classical lower bounds for first-order methods restricted to the span of past gradients (Nemirovski and Yudin, [16 ] ; Nesterov, [17 ] ): the oracle reveals the instance one element at a time, so that after k k k queries a span method knows nothing beyond the k k k -th element while the comparator sits at the end of the chain. Our extension to all learners follows Woodworth and Srebro [21 ] and Carmon et al. [3 ] : a random rotation hides the chain in a high-dimensional space, so that, with high probability, no query of the learner has a significant component along the directions not yet revealed, and the oracle answers as it would to a span method.
Setting
We consider the following online learning framework: a learner plays against an adversary over T T T rounds, and the goal of the learner is to minimize its regret. At the start of the game, the adversary chooses an instance of the game.
Instance, oracle and learner. An instance of the game is given by a triple ( K , L M O , ( g t ) t < T ) (K,\mathrm{LMO},(g_t)_{t<T}) ( K , LMO , ( g t ) t < T ) , fixed before the game (oblivious adversary), where:
– K K K is a convex compact decision set in R n \mathbb{R}^n R n ;
– L M O : R n → K \mathrm{LMO} : \mathbb{R}^n \to K LMO : R n → K is a linear minimization oracle, that solves L M O ( d ) ∈ arg min x ∈ K ⟨ d , x ⟩ \mathrm{LMO}(d) \in\operatorname*{arg\,min}_{x\in K}\langle d,x\rangle LMO ( d ) ∈ arg min x ∈ K ⟨ d , x ⟩ . When the minimizer is not unique, the oracle may return any minimizer, and our instances fix a tie-breaking rule that the learner may know.
– ( g t ) t < T (g_t)_{t<T} ( g t ) t < T is a sequence of loss vectors in R n \mathbb{R}^n R n .
The starting point x 0 ∈ K x_0 \in K x 0 ∈ K is part of the instance, hence chosen by the adversary. Since the problem is invariant under translations, we fix x 0 = 0 x_0 = 0 x 0 = 0 .
In each round t = 0 , … , T − 1 t = 0,\ldots,T - 1 t = 0 , … , T − 1 , the learner:
makes (if t ≥ 1 t \ge1 t ≥ 1 ) at most m m m oracle calls, taking the form: for 1 ≤ k ≤ m 1 \le k \le m 1 ≤ k ≤ m , it chooses a query d t , k ∈ R n d_{t,k} \in\mathbb{R}^n d t , k ∈ R n and receives L M O ( d t , k ) ∈ K \mathrm{LMO}(d_{t,k}) \in K LMO ( d t , k ) ∈ K ,
plays a point x t x_t x t that is certified to lie in K K K ,
observes the loss vector g t g_t g t .
The goal of the learner is to minimize its regret
Reg T = ∑ t < T ⟨ g t , x t ⟩ − min u ∈ K ∑ t < T ⟨ g t , u ⟩ . (1) \operatorname{Reg}_T = \sum_{t<T}\langle g_t,x_t\rangle- \min_{u\in K}\sum_{t<T}\langle g_t,u\rangle. \tag*{(1)} Reg T = t < T ∑ ⟨ g t , x t ⟩ − u ∈ K min t < T ∑ ⟨ g t , u ⟩ . (1)
It is worth emphasizing that the learner cannot project onto K K K . We nevertheless make the learner slightly more powerful by granting it two extra abilities: in our lower bound (recall we fixed x 0 = 0 x_0 = 0 x 0 = 0 ) it knows a radius r > 0 r > 0 r > 0 such that B r = { x ∈ R n : ∥ x ∥ ≤ r } ⊆ K B_r = \{x \in\mathbb{R}^n : \lVert x\rVert\le r\} \subseteq K B r = { x ∈ R n : ∥ x ∥ ≤ r } ⊆ K , and it may use internal randomness ξ \xi ξ , independent of the instance. Queries and plays may be arbitrary measurable functions of ξ \xi ξ and of everything observed so far. A point is certified if it lies in K K K for every instance consistent with information collected so far: for example, any point of the convex hull of B r B_r B r and of the oracle’s answers so far is certified.
Why linear losses? A linear loss x ↦ ⟨ g t , x ⟩ x \mapsto\langle g_t,x\rangle x ↦ ⟨ g t , x ⟩ with ∥ g t ∥ ≤ L \lVert g_t\rVert\le L ∥ g t ∥ ≤ L is convex and L L L -Lipschitz, and observing g t g_t g t is equivalent to observing the whole loss f t f_t f t . So, a lower bound proved with linear losses and full information also holds for convex L L L -Lipschitz losses under every feedback model that includes a subgradient at the played point. Hence, we assume linear losses.
Contributions
Lower bound for span methods (Section 2). Our first contribution (Theorem 2.2 ) is a single fixed instance, ( K , L M O , ( g t ) t < T ) (K,\mathrm{LMO},(g_t)_{t<T}) ( K , LMO , ( g t ) t < T ) , that may be known beforehand by the learner, and that is hard for a large subclass of learners called span methods . This subclass restrict the learner to query LMOs in the span of past observations and play points x t x_t x t in the convex hull of the starting point and the past LMOs’ outputs (see Theorem 2.1). In particular, this subclass contains all OFW tunings considered by Weibel et al. [20 ] , thereby answering their open question.
More precisely, the hard instance is defined by a set K K K of dimension 2 m T 2mT 2 m T with diameter 1 1 1 , and gradients g t g_t g t with norms bounded by 1 1 1 , such that any span method suffers a regret
Reg T ≥ T 3 / 4 ( 4 m ) − 1 / 4 . \operatorname{Reg}_T \ge T^{3/4}(4m)^{-1/4}. Reg T ≥ T 3/4 ( 4 m ) − 1/4 .
We restrict only the span of its queries and plays, which excludes randomized algorithms such as FTPL.
Lower bound for all learners (Section 3). Our second result and main contribution removes the span restriction on the learner, at the price of a random instance and a larger dimension n n n . More precisely, Theorem 3.1 states that there exists a fixed distribution of instances with K K K of dimension n ≳ ( m T ) 7 / 2 n \gtrsim(mT)^{7/2} n ≳ ( m T ) 7/2 and L = D = 1 L=D=1 L = D = 1 such that every learner that makes at most m m m oracle calls per round satisfies
E [ Reg T ] ≥ 0.23 T 3 / 4 m − 1 / 4 , \mathbb{E}[\operatorname{Reg}_T] \ge0.23T^{3/4}m^{-1/4}, E [ Reg T ] ≥ 0.23 T 3/4 m − 1/4 ,
where the expectation is over both the instance and the learner’s randomness.
Upper bound for OFW with m m m LMO calls per round (Section 4). Weibel et al. [20 ] gave numerical evidence that no OFW tuning making m = O ( 1 ) m=O(1) m = O ( 1 ) LMO calls per round beats the T 3 / 4 T^{3/4} T 3/4 regret, but left open how the regret depends on m m m . Our third result shows that the right dependence is m − 1 / 4 m^{-1/4} m − 1/4 . We introduce m m m -step OFW (Algorithm 1), which generalizes OFW to m m m LMO calls per round, and prove that its regret matches the lower bound of Theorem 2.2 for m ≤ T m \le T m ≤ T up to a multiplicative constant. Adapting the potential-based proof of Weibel et al. [20 ] , we show
Reg T ≤ 4.8 L D max { T 3 / 4 m − 1 / 4 , T } . (2) \operatorname{Reg}_T \le4.8LD\max\left\{T^{3/4}m^{-1/4},\sqrt{T}\right\}. \tag*{(2)} Reg T ≤ 4.8 L D max { T 3/4 m − 1/4 , T } . (2)
for well-chosen hyperparameters. When m = 1 m=1 m = 1 , it recovers the rate of Weibel et al. [20 ] , while when m ≥ T m \ge T m ≥ T , it recovers the O ( T ) O(\sqrt{T}) O ( T ) minimax rate of OCO. We also show that (2) holds, with a larger constant, for an anytime tuning of m m m -step OFW, which does not need to know the horizon T T T .
Lower bound for span methods
In this section, we prove the lower bound for the specific subclass of learners called span methods . The latter is large enough to contain OFW (Hazan and Kale [9 ] ) with any number of LMO calls per round, its variants with adaptive step sizes or line search, away-step and pairwise variants, and the generalized OFW tunings of Weibel et al. [20 ] , whose queries combine past gradients and vertices with arbitrary fixed coefficients. However, it excludes methods that compute projections, such as online gradient descent, or that randomly perturb their queries, such as FTPL.
Definition 2.1 (Span methods). A learner is a span method if:
its LMO queries d t , k d_{t,k} d t , k are linear combination of the starting point x 0 x_0 x 0 , past LMO answers v s , j v_{s,j} v s , j and observed loss vectors g s g_s g s : for all t ≥ 1 t \ge1 t ≥ 1 and 1 ≤ k ≤ m 1 \le k \le m 1 ≤ k ≤ m ,
d t , k ∈ span ( x 0 , v 1 , 1 , … , v 1 , m , … , v t , k − 1 , g 0 , … , g t − 1 ) d_{t,k} \in\operatorname{span}(x_0,v_{1,1},\ldots,v_{1,m},\ldots,v_{t,k-1},g_0,\ldots,g_{t-1}) d t , k ∈ span ( x 0 , v 1 , 1 , … , v 1 , m , … , v t , k − 1 , g 0 , … , g t − 1 ) v t , k = LMO ( d t , k ) ; v_{t,k}=\operatorname{LMO}(d_{t,k}); v t , k = LMO ( d t , k ) ; x t ∈ conv ( { x 0 } ∪ { v 1 , 1 , … , v 1 , m , … , v t , m } ) . x_t \in\operatorname{conv}\left(\{x_0\}\cup\{v_{1,1},\ldots,v_{1,m},\ldots,v_{t,m}\}\right). x t ∈ conv ( { x 0 } ∪ { v 1 , 1 , … , v 1 , m , … , v t , m } ) .
Note that only the sets of vectors from which d t , k d_{t,k} d t , k and x t x_t x t are formed are restricted; the linear and convex coefficients used to form d t , k d_{t,k} d t , k and x t x_t x t may be any functions of everything the learner knows, including its internal randomness, the decision set, the oracle’s rule, or even future losses.
Theorem 2.2 (Lower bound for span methods). Let m , T ≥ 1 m,T \ge1 m , T ≥ 1 and N = m ( T − 1 ) + 1 N=m(T-1)+1 N = m ( T − 1 ) + 1 . There is a fixed instance with K ⊂ R 2 N K \subset\mathbb{R}^{2N} K ⊂ R 2 N , D = 1 D=1 D = 1 , and ∥ g t ∥ = 1 \lVert g_t\rVert=1 ∥ g t ∥ = 1 , on which every span method making at most m m m oracle calls per round has regret
Reg T ≥ T ( 4 N ) − 1 / 4 ≥ T 3 / 4 ( 4 m ) − 1 / 4 . \operatorname{Reg}_T \ge T(4N)^{-1/4} \ge T^{3/4}(4m)^{-1/4}. Reg T ≥ T ( 4 N ) − 1/4 ≥ T 3/4 ( 4 m ) − 1/4 .
The proof, deferred to Appendix § 6 .§ 5 , is based on constructing a single instance that is hard for all span methods. The hard-instance construction for our lower bound adapts the chain principle of the classical lower bounds for first-order methods restricted to the span of past gradients (Nemirovski and Yudin [16 ] ; Nesterov [17 ] ). Scaling the instance, for all L , D ≥ 0 L,D \ge0 L , D ≥ 0 there is an instance with diam K ≤ D \operatorname{diam} K \le D diam K ≤ D and ∥ g t ∥ ≤ L \lVert g_t\rVert\le L ∥ g t ∥ ≤ L on which every span method making at most m m m oracle calls per round has regret at least L D T 3 / 4 ( 4 m ) − 1 / 4 LD\,T^{3/4}(4m)^{-1/4} L D T 3/4 ( 4 m ) − 1/4 .
The rest of this section is devoted to providing the key ideas of the proof of Theorem 2.2 .
Design of the hard instance
We start by constructing the hard instance of Theorem 2.2 . The body is a simplex whose vertices are the origin x 0 = 0 x_0 = 0 x 0 = 0 and N = m ( T − 1 ) + 1 N = m(T-1) + 1 N = m ( T − 1 ) + 1 points called the atoms . The atoms form a chain: they share a common walk with orthogonal steps, a deterministic analogue of a random walk, to which each atom adds a private orthogonal coordinate. The losses reward all atoms beyond a moving index by the same amount. The oracle breaks ties in favor of earlier atoms to force the learner to walk the chain one atom at a time.
Body and its atoms. Let u 0 , ν 0 , … , u N − 1 , ν N − 1 u_0,\nu_0,\ldots,u_{N-1},\nu_{N-1} u 0 , ν 0 , … , u N − 1 , ν N − 1 be the standard basis of R 2 N \mathbb{R}^{2N} R 2 N . The vectors u i u_i u i carry the steps of the common walk, and ν j \nu_j ν j carries the private coordinate of atom w ˉ j \bar{w}_j w ˉ j :
w ˉ j = M j + f j ν j , M j = ∑ i ≤ j δ i u i . \bar{w}_j = M_j + \sqrt{f_j}\nu_j,\qquad M_j = \sum_{i\le j}\sqrt{\delta_i}\,u_i. w ˉ j = M j + f j ν j , M j = i ≤ j ∑ δ i u i .
where the increments δ j = e j − e j − 1 \delta_j = e_j-e_{j-1} δ j = e j − e j − 1 and noise sizes f j = min ( e j , 1 − e j ) f_j = \min(e_j,1-e_j) f j = min ( e j , 1 − e j ) depend on a profile 0 = e − 1 < e 0 < e 1 < ⋯ < e N − 1 0=e_{-1}<e_0<e_1<\cdots<e_{N-1} 0 = e − 1 < e 0 < e 1 < ⋯ < e N − 1 which we define in the next paragraph. The body is the following simplex
K ˉ = conv ( { 0 } ∪ { w ˉ 0 , … , w ˉ N − 1 } ) . \bar{K}=\operatorname{conv}\left(\{0\}\cup\{\bar{w}_0,\ldots,\bar{w}_{N-1}\}\right). K ˉ = conv ( { 0 } ∪ { w ˉ 0 , … , w ˉ N − 1 } ) .
The starting point imposed on the learner is x 0 = 0 x_0=0 x 0 = 0 . Any two atoms share the beginning of the walk, which gives the martingale identity ⟨ w ˉ i , w ˉ j ⟩ = e i \langle\bar{w}_i,\bar{w}_j\rangle=e_i ⟨ w ˉ i , w ˉ j ⟩ = e i for i < j i<j i < j , and ∥ w ˉ j ∥ 2 = e j + f j ≤ 1 \lVert\bar{w}_j\rVert^2=e_j+f_j\le1 ∥ w ˉ j ∥ 2 = e j + f j ≤ 1 . Each atom is the only generating point with a component along its ν j \nu_j ν j , so the N + 1 N+1 N + 1 points are affinely independent and K ˉ \bar{K} K ˉ is a simplex; its diameter is at most 1 1 1 (Theorem A.1 (f)). The oracle LMO returns the first minimizer of ⟨ d , ⋅ ⟩ \langle d,\cdot\rangle ⟨ d , ⋅ ⟩ in the list 0 , w ˉ 0 , w ˉ 1 , … , w ˉ N − 1 0,\bar{w}_0,\bar{w}_1,\ldots,\bar{w}_{N-1} 0 , w ˉ 0 , w ˉ 1 , … , w ˉ N − 1 .
The profile. For k ≥ 0 k\ge0 k ≥ 0 let h k = 2 k + 1 h_k=\sqrt{2k+1} h k = 2 k + 1 and h ^ k = h k / ( 2 N ) \hat{h}_k=h_k/(2\sqrt{N}) h ^ k = h k / ( 2 N ) . For 0 ≤ j ≤ N − 1 0\le j\le N-1 0 ≤ j ≤ N − 1 the profile e j e_j e j is
e j = { h ^ j , j < N / 2 , 1 − h ^ N − 1 − j , j ≥ N / 2. e_j =
\begin{cases}
\hat{h}_j, & j<N/2,\\
1-\hat{h}_{N-1-j}, & j\ge N/2.
\end{cases} e j = { h ^ j , 1 − h ^ N − 1 − j , j < N /2 , j ≥ N /2.
The profile rises like a square root from 0 0 0 to 1 2 \frac{1}{2} 2 1 and then symmetrically up to 1 1 1 . It is strictly increasing (Theorem A.1 ), and δ j f j \delta_jf_j δ j f j is of order 1 / N 1/N 1/ N at every index, which is what makes every round cost roughly the same amount, of order N − 1 / 4 N^{-1/4} N − 1/4 (Section 2.3).
The losses. Let W ˉ \bar{W} W ˉ be the matrix with the atoms as columns, A = W ˉ T W ˉ A=\bar{W}^{\mathsf{T}}\bar{W} A = W ˉ T W ˉ the Gram matrix of the atoms, and 1 κ \mathbf{1}_\kappa 1 κ the indicator vector of the indices j ≥ κ j\ge\kappa j ≥ κ . The atoms are linearly independent, so A A A is invertible, and for 0 ≤ κ ≤ N − 1 0\le\kappa\le N-1 0 ≤ κ ≤ N − 1 we set
c ˉ κ = ( 1 κ T A − 1 1 κ ) − 1 / 2 , g ˉ ( κ ) = − c ˉ κ W ˉ A − 1 1 κ . \bar{c}_\kappa= \left(\mathbf{1}_\kappa^{\mathsf{T}}A^{-1}\mathbf{1}_\kappa\right)^{-1/2},\qquad
\bar{g}^{(\kappa)}=-\bar{c}_\kappa\bar{W}A^{-1}\mathbf{1}_\kappa. c ˉ κ = ( 1 κ T A − 1 1 κ ) − 1/2 , g ˉ ( κ ) = − c ˉ κ W ˉ A − 1 1 κ .
Then W ˉ T g ˉ ( κ ) = − c ˉ κ 1 κ \bar{W}^{\mathsf{T}}\bar{g}^{(\kappa)}=-\bar{c}_\kappa\mathbf{1}_\kappa W ˉ T g ˉ ( κ ) = − c ˉ κ 1 κ and ∥ g ˉ ( κ ) ∥ 2 = c ˉ κ 2 1 κ T A − 1 1 κ = 1 \lVert\bar{g}^{(\kappa)}\rVert^2=\bar{c}_\kappa^2\mathbf{1}_\kappa^{\mathsf{T}}A^{-1}\mathbf{1}_\kappa=1 ∥ g ˉ ( κ ) ∥ 2 = c ˉ κ 2 1 κ T A − 1 1 κ = 1 : g ˉ ( κ ) \bar{g}^{(\kappa)} g ˉ ( κ ) is the unit vector in the span of the atoms with
⟨ g ˉ ( κ ) , w ˉ j ⟩ = { 0 , j < κ , − c ˉ κ , j ≥ κ . (3) \langle\bar{g}^{(\kappa)},\bar{w}_j\rangle=
\begin{cases}
0, & j<\kappa,\\
-\bar{c}_\kappa, & j\ge\kappa.
\end{cases} \tag*{(3)} ⟨ g ˉ ( κ ) , w ˉ j ⟩ = { 0 , − c ˉ κ , j < κ , j ≥ κ . (3)
It gives no reward to the atoms before κ \kappa κ , and it rewards every atom from κ \kappa κ on by the same amount. The learner faces the fixed loss schedule
g t = g ˉ ( κ t ) , κ t = m t , t = 0 , … , T − 1 , g_t=\bar{g}^{(\kappa_t)},\qquad\kappa_t=mt,\qquad t=0,\ldots,T-1, g t = g ˉ ( κ t ) , κ t = m t , t = 0 , … , T − 1 ,
every κ t \kappa_t κ t being a valid index because m ( T − 1 ) = N − 1 m(T-1)=N-1 m ( T − 1 ) = N − 1 . The schedule is matched to the oracle budget: when it plays in round t t t , the learner has made at most κ t \kappa_t κ t oracle calls. The comparator is the last atom x ∗ = w ˉ N − 1 x^*=\bar{w}_{N-1} x ∗ = w ˉ N − 1 and is rewarded in every round. Everything is deterministic and may be known to the learner.
Every round costs c ˉ κ t \bar{c}_{\kappa_t} c ˉ κ t
The key idea is that a query built from past losses and past atoms gives the same value to all atoms beyond the current index, so the oracle, which breaks ties in favor of the earliest atom, advances along the chain by at most one atom per call.
Lemma 2.3. Let t ≥ 1 t\ge1 t ≥ 1 and J ≥ κ t − 1 = m ( t − 1 ) J\ge\kappa_{t-1}=m(t-1) J ≥ κ t − 1 = m ( t − 1 ) . Let d ∈ span ( 0 , w ˉ 0 , … , w ˉ j − 1 , g 0 , … , g t − 1 ) d\in\operatorname{span}(0,\bar{w}_0,\ldots,\bar{w}_{j-1},g_0,\ldots,g_{t-1}) d ∈ span ( 0 , w ˉ 0 , … , w ˉ j − 1 , g 0 , … , g t − 1 ) . Then, for all j ≥ J j\ge J j ≥ J , we have ⟨ d , w ˉ j ⟩ = ⟨ d , w ˉ J ⟩ \langle d,\bar{w}_j\rangle=\langle d,\bar{w}_J\rangle ⟨ d , w ˉ j ⟩ = ⟨ d , w ˉ J ⟩ .
In particular, the oracle tie-breaking rule gives LMO ( d ) ∈ { 0 , w ˉ 0 , … , w ˉ J } \operatorname{LMO}(d)\in\{0,\bar{w}_0,\ldots,\bar{w}_J\} LMO ( d ) ∈ { 0 , w ˉ 0 , … , w ˉ J } .
Proof sketch. (The complete proof is in Section A.2.) A received loss g s = g ˉ ( κ s ) g_s=\bar{g}^{(\kappa_s)} g s = g ˉ ( κ s ) has κ s ≤ J \kappa_s\le J κ s ≤ J and gives the value − c ˉ κ s -\bar{c}_{\kappa_s} − c ˉ κ s to every atom of index at least J J J by (3) , a returned atom w ˉ i \bar{w}_i w ˉ i has i < J i<J i < J and gives them the value e i e_i e i by the martingale identity, and linear combinations preserve this.
Using Theorem 2.3, we get that all span methods incur the same exact regret on this instance.
Proposition 2.4. On the instance of § 2.1 , every span method making at most m m m oracle calls per round has regret exactly Reg T = ∑ t < T c ˉ κ t \operatorname{Reg}_T=\sum_{t<T}\bar{c}_{\kappa_t} Reg T = ∑ t < T c ˉ κ t .
Proof sketch. (The complete proof is in Section A.3.) By induction on t t t and Theorem 2.3, every atom returned before the play of round t t t has index below κ t = m t \kappa_t=mt κ t = m t : round 0 0 0 makes no call, the oracle advances by at most one atom per call, at most m m m calls are made per round, and the schedule advances by m m m atoms per round. So the play x t x_t x t combines the origin and atoms of index below κ t \kappa_t κ t , which all have loss 0 0 0 against g t g_t g t by (3) , while the comparator has loss − c ˉ κ t -\bar{c}_{\kappa_t} − c ˉ κ t and is the best point in hindsight. This is where m m m enters: the schedule has to advance m m m atoms per round, which costs the adversary a factor m m m in the number of atoms and hence m − 1 / 4 m^{-1/4} m − 1/4 in the regret.
A lower bound on per-round costs c ˉ κ \bar{c}_{\kappa} c ˉ κ
We prove the following lower bound on c ˉ κ \bar{c}_{\kappa} c ˉ κ , which combined with Theorem 2.4 gives Theorem 2.2 .
Lemma 2.5. For 0 ≤ κ ≤ N − 1 0 \le\kappa\le N-1 0 ≤ κ ≤ N − 1 , we have
c ˉ κ = dist ( aff ( w ˉ κ , … , w ˉ N − 1 ) , span ( w ˉ 0 , … , w ˉ κ − 1 ) ) , \bar{c}_{\kappa}=\operatorname{dist}\left(\operatorname{aff}(\bar{w}_{\kappa},\ldots,\bar{w}_{N-1}),\operatorname{span}(\bar{w}_{0},\ldots,\bar{w}_{\kappa-1})\right), c ˉ κ = dist ( aff ( w ˉ κ , … , w ˉ N − 1 ) , span ( w ˉ 0 , … , w ˉ κ − 1 ) ) ,
and
c ˉ κ ≥ dist ( M κ , span ( w ˉ 0 , … , w ˉ κ − 1 ) ) ≥ ( 4 N ) − 1 / 4 . \bar{c}_{\kappa}\ge\operatorname{dist}\left(M_{\kappa},\operatorname{span}(\bar{w}_{0},\ldots,\bar{w}_{\kappa-1})\right)\ge(4N)^{-1/4}. c ˉ κ ≥ dist ( M κ , span ( w ˉ 0 , … , w ˉ κ − 1 ) ) ≥ ( 4 N ) − 1/4 .
Proof sketch. (The complete proof is in Section A.4.) The distance. Let y y y be in the affine hull of the tail atoms and z z z in the span of the head atoms. By (3) , ⟨ g ˉ ( κ ) , y − z ⟩ = − c ˉ κ \langle\bar{g}^{(\kappa)},y-z\rangle=-\bar{c}_{\kappa} ⟨ g ˉ ( κ ) , y − z ⟩ = − c ˉ κ , so ∥ y − z ∥ ≥ c ˉ κ \lVert y-z\rVert\ge\bar{c}_{\kappa} ∥ y − z ∥ ≥ c ˉ κ because g ˉ ( κ ) \bar{g}^{(\kappa)} g ˉ ( κ ) is a unit vector. Equality is attained: g ˉ ( κ ) = ∑ j β j w ˉ j \bar{g}^{(\kappa)}=\sum_j\beta_j\bar{w}_j g ˉ ( κ ) = ∑ j β j w ˉ j with β = − c ˉ κ A − 1 1 κ \beta=-\bar{c}_{\kappa}A^{-1}\mathbf{1}_{\kappa} β = − c ˉ κ A − 1 1 κ , and 1 = ∥ g ˉ ( κ ) ∥ 2 = ∑ j β j ⟨ g ˉ ( κ ) , w ˉ j ⟩ = − c ˉ κ ∑ j ≥ κ β j 1=\lVert\bar{g}^{(\kappa)}\rVert^2=\sum_j\beta_j\langle\bar{g}^{(\kappa)},\bar{w}_j\rangle=-\bar{c}_{\kappa}\sum_{j\ge\kappa}\beta_j 1 = ∥ g ˉ ( κ ) ∥ 2 = ∑ j β j ⟨ g ˉ ( κ ) , w ˉ j ⟩ = − c ˉ κ ∑ j ≥ κ β j by (3) , so y ∗ = − c ˉ κ ∑ j ≥ κ β j w ˉ j y^*=-\bar{c}_{\kappa}\sum_{j\ge\kappa}\beta_j\bar{w}_j y ∗ = − c ˉ κ ∑ j ≥ κ β j w ˉ j is in the affine hull of the tail, z ∗ = c ˉ κ ∑ j < κ β j w ˉ j z^*=\bar{c}_{\kappa}\sum_{j<\kappa}\beta_j\bar{w}_j z ∗ = c ˉ κ ∑ j < κ β j w ˉ j is in the span of the head, and y ∗ − z ∗ = − c ˉ κ g ˉ ( κ ) y^*-z^*=-\bar{c}_{\kappa}\bar{g}^{(\kappa)} y ∗ − z ∗ = − c ˉ κ g ˉ ( κ ) has norm c ˉ κ \bar{c}_{\kappa} c ˉ κ .
The first inequality. Every tail atom is M κ M_{\kappa} M κ plus a vector orthogonal to M κ M_{\kappa} M κ and to the head atoms, so ∥ y − z ∥ 2 = ∥ M κ − z ∥ 2 + ∥ y − M κ ∥ 2 \lVert y-z\rVert^2=\lVert M_{\kappa}-z\rVert^2+\lVert y-M_{\kappa}\rVert^2 ∥ y − z ∥ 2 = ∥ M κ − z ∥ 2 + ∥ y − M κ ∥ 2 : the tail is never closer to the head span than M κ M_{\kappa} M κ is.
The second inequality. Expand M κ − ∑ s < κ y s w ˉ s M_{\kappa}-\sum_{s<\kappa}y_s\bar{w}_s M κ − ∑ s < κ y s w ˉ s along the vectors u i u_i u i and ν s \nu_s ν s and set Z i = 1 − ∑ s = i κ − 1 y s Z_i=1-\sum_{s=i}^{\kappa-1}y_s Z i = 1 − ∑ s = i κ − 1 y s (so that Z κ = 1 Z_{\kappa}=1 Z κ = 1 ): its squared norm is a discrete Dirichlet energy on the chain, and dist ( M κ , span ( w ˉ 0 , … , w ˉ κ − 1 ) ) 2 \operatorname{dist}(M_{\kappa},\operatorname{span}(\bar{w}_0,\ldots,\bar{w}_{\kappa-1}))^2 dist ( M κ , span ( w ˉ 0 , … , w ˉ κ − 1 ) ) 2 is the minimum of this energy,
q κ : = min Z : Z κ = 1 ∑ i ≤ κ δ i Z i 2 + ∑ s < κ f s ( Z s − Z s + 1 ) 2 . q_{\kappa}:=\min_{Z:Z_{\kappa}=1}\sum_{i\le\kappa}\delta_i Z_i^2+\sum_{s<\kappa}f_s(Z_s-Z_{s+1})^2. q κ := Z : Z κ = 1 min i ≤ κ ∑ δ i Z i 2 + s < κ ∑ f s ( Z s − Z s + 1 ) 2 .
Minimizing over Z 0 , Z 1 , … Z_0,Z_1,\ldots Z 0 , Z 1 , … one variable at a time, each step being min t [ x t 2 + y ( t − 1 ) 2 ] = x y / ( x + y ) \min_t[x t^2+y(t-1)^2]=xy/(x+y) min t [ x t 2 + y ( t − 1 ) 2 ] = x y / ( x + y ) (the formula for two resistances in parallel), gives the recursion q 0 = δ 0 q_0=\delta_0 q 0 = δ 0 , q k + 1 = δ k + 1 + q k f k / ( q k + f k ) q_{k+1}=\delta_{k+1}+q_kf_k/(q_k+f_k) q k + 1 = δ k + 1 + q k f k / ( q k + f k ) , and on our profile N q k ≥ 1 2 \sqrt{N}q_k\ge\frac{1}{2} N q k ≥ 2 1 for every k k k (Theorem A.2). Only the head of the chain enters this bound.
This completes the proof of Theorem 2.2 .
Remarks
Tie-breaking. The instance uses the oracle’s tie rule, because the tail ties exactly. The tie rule can be removed (up to ties with the origin) by shrinking atom j j j by ρ j = 1 − ε j / ( N − 1 ) \rho_j=1-\varepsilon j/(N-1) ρ j = 1 − ε j / ( N − 1 ) (that is replacing w ˉ j \bar{w}_j w ˉ j by w ~ j = ρ j w ˉ j \widetilde{w}_j=\rho_j\bar{w}_j w j = ρ j w ˉ j ) for a small ε > 0 \varepsilon>0 ε > 0 , at the price of a factor 1 − ε 1-\varepsilon 1 − ε in the regret bound.
Indeed, two shrunk atoms satisfy ⟨ w ~ i , w ~ j ⟩ = ρ i ρ j e i \langle\widetilde{w}_i,\widetilde{w}_j\rangle=\rho_i\rho_j e_i ⟨ w i , w j ⟩ = ρ i ρ j e i for i < j i<j i < j , and ⟨ g ˉ ( κ ) , w ~ j ⟩ = − ρ j c ˉ κ \langle\bar{g}^{(\kappa)},\widetilde{w}_j\rangle=-\rho_j\bar{c}_{\kappa} ⟨ g ˉ ( κ ) , w j ⟩ = − ρ j c ˉ κ for j ≥ κ j\ge\kappa j ≥ κ . So every query of a span method gives each tail atom ρ j \rho_j ρ j times a common number. If the common number is negative, the first tail atom is the strict best of the tail; if it is positive, the whole tail loses to the origin. Otherwise the tail ties with the origin (e.g., for the query 0 0 0 ), which the oracle breaks in favor of the origin.
On the multiplicative constant 4 − 1 / 4 4^{-1/4} 4 − 1/4 . Comparing the lower bound of Theorem 2.2 to the upper bounds of Weibel et al. [20 ] for m = 1 m=1 m = 1 and of § 4 for m > 1 m>1 m > 1 , we get that the gap is a multiplicative factor 4 × 3 − 3 / 4 2 < 2.5 4\times3^{-3/4}\sqrt{2}<2.5 4 × 3 − 3/4 2 < 2.5 . Numerically, for m = 1 m=1 m = 1 , ∑ t < T c ˉ t \sum_{t<T}\bar{c}_t ∑ t < T c ˉ t exceeds T 3 / 4 T^{3/4} T 3/4 at every horizon we computed (Table 1 ), so the constant 4 − 1 / 4 4^{-1/4} 4 − 1/4 of Theorem 2.2 is not sharp and the true constant appears to be around 1 1 1 .
Table 1. Regret lower bound for span methods with m = 1 m=1 m = 1 LMO call per round, together with regret of FTPL (with theoretical horizon-dependent tuning) on the hard instance of § 2.1 .
The instance is hard only for span methods: FTPL beats it. FTPL uses one oracle call per round but perturbs its query with a full-dimensional random noise, so it is not a span method. On the convex hull of N + 1 N+1 N + 1 known points, FTPL with Gaussian perturbations has regret O ( L D T ) O(LD\sqrt{T}) O ( L D T ) up to a factor logarithmic in N N N (Abernethy et al., [1 ] ), and Table 1 illustrates this numerically on the instance of § 2.1 . To design a lower bound for a larger class of learners that includes FTPL, the chain of atoms has to be hidden by a random rotation in a space of much larger dimension: this is the instance of § 3 .
Lower bound for all learners
A general learner may query the LMO in arbitrary (possibly random) directions, and may play outside the convex hull of the oracle’s answers, by combining with elements of B r ⊂ K B_r\subset K B r ⊂ K .
To handle such learners, we adapt the random rotation construction (Woodworth and Srebro, [21 ] ; Carmon et al., [3 ] ) to hide the orientation of the meaningful hard body of § 2 in a uniformly random frame of a high-dimensional space, and add a shrinkage and a small ball. This makes each oracle call reveal at most one new atom whatever the query.
Theorem 3.1 (Lower bound for every learner). Let m ≥ 1 m\ge1 m ≥ 1 , T ≥ 2 T\ge2 T ≥ 2 , N = m ( T − 1 ) + 1 N=m(T-1)+1 N = m ( T − 1 ) + 1 and n ≥ 1.1 ⋅ 10 6 N 7 / 2 n\ge1.1\cdot10^6N^{7/2} n ≥ 1.1 ⋅ 1 0 6 N 7/2 . There is a probability distribution over instances with K ⊂ R n K\subset\mathbb{R}^n K ⊂ R n , 0 ∈ K 0\in K 0 ∈ K , diam K ≤ 5 / 2 \operatorname{diam}K\le\sqrt{5}/2 diam K ≤ 5 /2 and ∥ g t ∥ = 1 \lVert g_t\rVert=1 ∥ g t ∥ = 1 , such that every learner making at most m m m oracle calls per round satisfies
Reg T ≥ 0.25 T N − 1 / 4 \operatorname{Reg}_{T} \ge0.25T N^{-1/4} Reg T ≥ 0.25 T N − 1/4
with probability at least 1 − 1 100 N − 1 / 4 1-\frac{1}{100}N^{-1/4} 1 − 100 1 N − 1/4 , and
E [ Reg T ] ≥ 0.23 T N − 1 / 4 , \mathbb{E}[\operatorname{Reg}_{T}] \ge0.23T N^{-1/4}, E [ Reg T ] ≥ 0.23 T N − 1/4 ,
where the probability and the expectation are over the instance and the learner’s randomness.
The instance adversary is still oblivious: the instance distribution is the same for all learners, and the instance randomness is independent of the learner’s randomness (as a lower bound for randomized learners requires). Because of this, by Yao’s principle Yao [22 ] it suffices to treat deterministic learners.
As a corollary of Theorem 3.1, choosing a learner-dependent rotation, we get a (deterministic) instance that depends on the learner’s distribution but not on its randomness: it is a lower bound in the usual sense for randomized algorithms.
Corollary 3.2 (A fixed instance for each learner). Under the assumptions of Theorem 3.1, for all L , D > 0 L,D>0 L , D > 0 and every learner making at most m m m oracle calls per round, there is a fixed instance with diam K ≤ D \operatorname{diam} K\le D diam K ≤ D and ∥ g t ∥ ≤ L \lVert g_t\rVert\le L ∥ g t ∥ ≤ L on which E ξ [ Reg T ] ≥ 0.2 L D T N − 1 / 4 \mathbb{E}_{\xi}[\operatorname{Reg}_{T}]\ge0.2LD T N^{-1/4} E ξ [ Reg T ] ≥ 0.2 L D T N − 1/4 .*
The rest of the section is devoted to sketching the key ideas of the proof of Theorem 3.1, starting with the design of the hard hidden instance. The full proofs are deferred to Appendix § 7 .
The hidden instance
We start by constructing the hard instance of Theorem 3.1. The instance is the chain of § 2.1 hidden by a random rotation in a space of dimension n ≫ N n\gg N n ≫ N , so that the learner cannot compute the atoms. Two additions ensure that, with high probability, each oracle call reveals at most one new atom of the chain. A small n n n -dimensional ball is added to the body, so that a query unrelated to the revealed atoms is answered by a ball point that reveals nothing. The atoms are also slightly shrunk, so that a query with a tiny component on the unrevealed atoms can reveal only the first unrevealed atom.
Body and its atoms. Let n ≫ 2 N n\gg2N n ≫ 2 N to be chosen later (see the end of § 3.2 ), and let ( u 0 , ν 0 , u 1 , ν 1 , … , u N − 1 , ν N − 1 ) (u_0,\nu_0,u_1,\nu_1,\ldots,u_{N-1},\nu_{N-1}) ( u 0 , ν 0 , u 1 , ν 1 , … , u N − 1 , ν N − 1 ) be a uniformly random orthonormal family in R n \mathbb{R}^n R n , called the frame and listed in the order in which it will be revealed to the learner. The atoms are those of § 2.1 in this frame, slightly shrunk:
w j = ρ j M j + f j ν j , ρ j = 1 − j 2 ( N − 1 ) , w_j=\rho_jM_j+\sqrt{f_j}\nu_j,\qquad\rho_j=1-\frac{j}{2(N-1)}, w j = ρ j M j + f j ν j , ρ j = 1 − 2 ( N − 1 ) j ,
where
M j = ∑ i ≤ j δ i u i M_j=\sum_{i\le j}\sqrt{\delta_i}u_i M j = i ≤ j ∑ δ i u i
is the walk position in the hidden frame. The shrinkage factor decreases from 1 1 1 to 1 2 \frac{1}{2} 2 1 by the same amount γ = 1 / ( 2 ( N − 1 ) ) \gamma=1/(2(N-1)) γ = 1/ ( 2 ( N − 1 )) at every step. Its only purpose is to break ties between atoms in favor of the earliest one. The body is
K = conv ( { 0 } ∪ { w 0 , … , w N − 1 } ∪ B r ) , K=\operatorname{conv}\left(\{0\}\cup\{w_0,\ldots,w_{N-1}\}\cup B_r\right), K = conv ( { 0 } ∪ { w 0 , … , w N − 1 } ∪ B r ) ,
with B r B_r B r the closed ball of radius r r r and r = 1 40 N − 1 / 4 r=\frac{1}{40}N^{-1/4} r = 40 1 N − 1/4 . The starting point imposed on the learner is x 0 = 0 x_0=0 x 0 = 0 . For d ≠ 0 d\ne0 d = 0 the oracle LMO returns the first candidate in the list 0 , w 0 , … , w N − 1 , − r d / ∥ d ∥ 0,w_0,\ldots,w_{N-1},-r\,d/\lVert d\rVert 0 , w 0 , … , w N − 1 , − r d / ∥ d ∥ that minimizes ⟨ d , ⋅ ⟩ \langle d,\cdot\rangle ⟨ d , ⋅ ⟩ (and 0 0 0 for d = 0 d=0 d = 0 ). A linear function is minimized over a convex hull at a generating point and over the ball at − r d / ∥ d ∥ -r\,d/\lVert d\rVert − r d / ∥ d ∥ , so this is a valid oracle.
The ball plays two roles. For queries, it is a decoy: a query with no significant component along the revealed atoms is answered by the ball point − r d / ∥ d ∥ -r\,d/\lVert d\rVert − r d / ∥ d ∥ , which reveals nothing (Step 1 of Section 3.2). For plays, it lets the learner leave the hull of the oracle’s answers, but a ball point earns at most r r r per round, a small fraction of what the comparator will earn. Finally, since the frame is random and n ≫ N n\gg N n ≫ N , the unrevealed atoms are nearly orthogonal to any vector the learner can form: a query cannot aim at them, and a play cannot put significant weight on them.
Losses. The canonical loss g ˉ ( κ ) \bar{g}^{(\kappa)} g ˉ ( κ ) of § 2.1 depends on all atoms through A − 1 A^{-1} A − 1 , so it cannot be computed from the part of the frame revealed up to index κ \kappa κ . We use instead the unit vector pointing against the part of the walk position M κ M_\kappa M κ that the first κ \kappa κ atoms cannot explain: with Π κ \Pi_\kappa Π κ the orthogonal projection onto span ( w 0 , … , w κ − 1 ) \operatorname{span}(w_0,\ldots,w_{\kappa-1}) span ( w 0 , … , w κ − 1 ) ,
c κ = ∥ M κ − Π κ M κ ∥ , g ( κ ) = − M κ − Π κ M κ c κ , c_\kappa=\lVert M_\kappa-\Pi_\kappa M_\kappa\rVert,\qquad
g^{(\kappa)}=-\frac{M_\kappa-\Pi_\kappa M_\kappa}{c_\kappa}, c κ = ∥ M κ − Π κ M κ ∥ , g ( κ ) = − c κ M κ − Π κ M κ ,
and g t = g ( κ t ) g_t=g^{(\kappa_t)} g t = g ( κ t ) with κ t = m t \kappa_t=mt κ t = m t as before. Note that c κ > 0 c_\kappa>0 c κ > 0 since M κ ∉ span ( w 0 , … , w κ − 1 ) M_\kappa\notin\operatorname{span}(w_0,\ldots,w_{\kappa-1}) M κ ∈ / span ( w 0 , … , w κ − 1 ) , and thus g ( κ ) g^{(\kappa)} g ( κ ) is well defined. This vector lies in E κ = span ( u 0 , … , u κ , ν 0 , … , ν κ − 1 ) E_\kappa=\operatorname{span}(u_0,\ldots,u_\kappa,\nu_0,\ldots,\nu_{\kappa-1}) E κ = span ( u 0 , … , u κ , ν 0 , … , ν κ − 1 ) and has the same pattern as before up to the shrinkage (Theorem B.1): ⟨ g ( κ ) , w j ⟩ = 0 \langle g^{(\kappa)},w_j\rangle=0 ⟨ g ( κ ) , w j ⟩ = 0 for j < κ j<\kappa j < κ and − ρ j c κ -\rho_jc_\kappa − ρ j c κ for j ≥ κ j\ge\kappa j ≥ κ , while every ball point has value at least − r -r − r . The comparator x ∗ = w N − 1 x^*=w_{N-1} x ∗ = w N − 1 earns exactly c κ / 2 c_\kappa/2 c κ /2 per round, and c κ ≥ ( 4 N ) − 1 / 4 c_\kappa\ge(4N)^{-1/4} c κ ≥ ( 4 N ) − 1/4 by a recursion similar to that of Theorem 2.5 (the shrinkage only helping, Theorem B.3).
Proof outline
As noted after Theorem 3.1, the instance is oblivious, and we can assume without loss of generality that the learner is deterministic. Below, we sketch the main ingredients of the proof of Theorem 3.1; all steps are formalized as lemmas in Appendix § 7 . In round t t t only atoms of index at least κ t \kappa_t κ t are useful, each with loss of order − N − 1 / 4 -N^{-1/4} − N − 1/4 , the comparator included. A learner that knows only atoms of index below κ t \kappa_t κ t loses about N − 1 / 4 N^{-1/4} N − 1/4 against x ⋆ x^\star x ⋆ in round t t t , as in § 2 .
Step 1. We define a clean oracle and a clean process that reveal atoms one at a time. The process is at level L L L when the first 2 L + 1 2L+1 2 L + 1 frame vectors u 0 , ν 0 , … , ν L − 1 , u L u_0,\nu_0,\ldots,\nu_{L-1},u_L u 0 , ν 0 , … , ν L − 1 , u L are revealed. The atoms w 0 , … , w L − 1 w_0,\ldots,w_{L-1} w 0 , … , w L − 1 and the walk position M L M_L M L are then functions of revealed vectors, and every unrevealed atom splits into a known part and a hidden part,
w j = ρ j M L + a j ( L ) , a j ( L ) ∈ U L , w_j=\rho_j M_L+a_j^{(L)}, \qquad a_j^{(L)}\in U_L, w j = ρ j M L + a j ( L ) , a j ( L ) ∈ U L ,
where U L : = span ( ν L , u L + 1 , ν L + 1 , … , ν N − 1 ) U_L:=\operatorname{span}(\nu_L,u_{L+1},\nu_{L+1},\ldots,\nu_{N-1}) U L := span ( ν L , u L + 1 , ν L + 1 , … , ν N − 1 ) is the span of the unrevealed frame vectors and ∥ a j ( L ) ∥ ≤ 1 \lVert a_j^{(L)}\rVert\le1 ∥ a j ( L ) ∥ ≤ 1 . To the learner, unrevealed atoms look alike except for the shrinkage. Before its play in round t t t it has made at most κ t \kappa_t κ t calls, so it can know an atom of index κ t \kappa_t κ t or more only if some past oracle answer skipped an atom in the chain. Two features prevent this:
Shrinkage: for a query d d d nearly orthogonal to U L U_L U L , every unrevealed atom has value ρ j ⟨ d , M L ⟩ \rho_j\langle d,M_L\rangle ρ j ⟨ d , M L ⟩ up to a tiny error, and if this is negative the next atom w L w_L w L is the strict winner among unrevealed atoms by a margin γ ∣ ⟨ d , M L ⟩ ∣ \gamma\lvert\langle d,M_L\rangle\rvert γ ∣⟨ d , M L ⟩∣ .
The ball: an unrevealed atom can beat the ball point only if ∣ ⟨ d , M L ⟩ ∣ ≳ r ∥ d ∥ \lvert\langle d,M_L\rangle\rvert\gtrsim r\lVert d\rVert ∣⟨ d , M L ⟩∣ ≳ r ∥ d ∥ , and then the shrinkage margin γ r ∥ d ∥ \gamma r\lVert d\rVert γ r ∥ d ∥ exceeds the error from the hidden parts. Otherwise no unrevealed atom is returned, and the oracle answer (the origin, a revealed atom, or − r d / ∥ d ∥ -r\,d/\lVert d\rVert − r d / ∥ d ∥ ) tells the learner nothing new.
A clean oracle that answers using only the revealed vectors and raises the level by at most one per query (Theorem B.6) formalizes this. It agrees with the true oracle on every query d d d such that ∥ Π U L d ∥ < τ ∥ d ∥ \lVert\Pi_{U_L}d\rVert<\tau\lVert d\rVert ∥ Π U L d ∥ < τ ∥ d ∥ , where Π U L \Pi_{U_L} Π U L is the orthogonal projection onto U L U_L U L and τ = γ r / 5 \tau=\gamma r/5 τ = γ r /5 (Theorem B.8). The clean process is the run of the learner against the clean oracle: when the learner plays in round t t t , its level is at most κ t \kappa_t κ t (Theorem B.7).
Step 2. The clean and true processes coincide (with high probability). In the clean process, where the true oracle is replaced by the clean oracle, every message of the learner, query or play, is a function of the revealed vectors (Theorem B.9). The messages are thus determined by the revealed vectors, but they can be outside their span.
Conditionally on them, the unrevealed frame is a uniformly random orthonormal family of the orthogonal complement, so U L U_L U L is a uniformly random subspace of dimension at most 2 N 2N 2 N in a space of dimension at least n − 2 N n-2N n − 2 N , and a fixed vector has projection of order N / n \sqrt{N/n} N / n onto it (Theorem B.10). A union bound over the at most 2 N 2N 2 N messages shows that, with probability at least 1 − δ 1-\delta 1 − δ , where δ : = 1 100 N − 1 / 4 \delta:=\frac{1}{100}N^{-1/4} δ := 100 1 N − 1/4 , every message v v v satisfies ∥ Π U L v ∥ < τ ∥ v ∥ \lVert\Pi_{U_L}v\rVert<\tau\lVert v\rVert ∥ Π U L v ∥ < τ ∥ v ∥ (Theorem B.11). We call this event the good event . On this good event the learner’s process is the clean process.
Step 3. On the good event, each certifiable play for round t t t earns at most 0.11 c κ t + r 0.11\,c_{\kappa_t}+r 0.11 c κ t + r , against c κ t / 2 c_{\kappa_t}/2 c κ t /2 for the comparator. The learner may play any point of K K K it can certify, for instance a mixture of unrevealed atoms and a ball point whose hidden parts cancel. This fails because the hidden parts of unrevealed atoms cannot cancel each other: a nonnegative combination of them with total weight Λ \Lambda Λ has norm at least Λ / ( 4 N 1 / 4 ) \Lambda/(4N^{1/4}) Λ/ ( 4 N 1/4 ) (Theorem B.4). A play with hidden atom component at most τ \tau τ therefore puts weight at most 4 N 1 / 4 ( τ + r ) ≤ 0.11 4N^{1/4}(\tau+r)\le0.11 4 N 1/4 ( τ + r ) ≤ 0.11 on unrevealed atoms and earns at most 0.11 c κ t + r 0.11\,c_{\kappa_t}+r 0.11 c κ t + r (Theorem B.5), against c κ t / 2 c_{\kappa_t}/2 c κ t /2 for the comparator.
Conclusion. On the good event each round costs at least 0.39 c κ t − r > 0.25 N − 1 / 4 0.39\,c_{\kappa_t}-r>0.25N^{-1/4} 0.39 c κ t − r > 0.25 N − 1/4 . And outside the good event, the regret is at least − D T ≥ − 1.12 T -DT\ge-1.12T − D T ≥ − 1.12 T . As δ = 1 100 N − 1 / 4 \delta=\frac{1}{100}N^{-1/4} δ = 100 1 N − 1/4 , this gives E [ Reg T ] ≥ 0.23 T N − 1 / 4 \mathbb{E}[\operatorname{Reg}_T]\ge0.23TN^{-1/4} E [ Reg T ] ≥ 0.23 T N − 1/4 and proves Theorem 3.1. The dimension n n n comes from the condition N / n ≲ τ \sqrt{N/n}\lesssim\tau N / n ≲ τ : the typical size of the projection of a message onto the hidden subspace U L U_L U L is N / n \sqrt{N/n} N / n , and it must be below the margin τ ≈ N − 5 / 4 / 400 \tau\approx N^{-5/4}/400 τ ≈ N − 5/4 /400 , which gives n ≳ N ⋅ N 5 / 2 = N 7 / 2 n\gtrsim N\cdot N^{5/2}=N^{7/2} n ≳ N ⋅ N 5/2 = N 7/2 (indeed we get n = 1.1 ⋅ 10 6 N 7 / 2 n=1.1\cdot10^6N^{7/2} n = 1.1 ⋅ 1 0 6 N 7/2 to accommodate small values of N N N ). We have not tried to optimize this exponent.
OFW with m m m LMO calls per round
We design a generalization of OFW (Hazan and Kale, [9 ] ) with m m m LMO calls per round (Algorithm 1). We show that its regret is O ( L D T 3 / 4 m − 1 / 4 ) O(LD\,T^{3/4}m^{-1/4}) O ( L D T 3/4 m − 1/4 ) when m = O ( T ) m=O(T) m = O ( T ) (Theorem 4.2 ), and sketch the potential-function proof, which follows Weibel et al. [20 ] . We then give an anytime tuning (which does not depend on T T T ) with the same rate (Theorem 4.3 ). The minimax rate O ( L D max { T 3 / 4 m − 1 / 4 , T } ) O(LD\max\{T^{3/4}m^{-1/4},\sqrt{T}\}) O ( L D max { T 3/4 m − 1/4 , T }) is attained for all m m m and T T T by a tuning capped at m = T m=T m = T (Theorem 4.4 ).
In this section rounds are numbered t = 1 , … , T t=1,\ldots,T t = 1 , … , T , the starting point is x 1 ∈ K x_1\in K x 1 ∈ K , G t : = ∑ s = 1 t g s G_t:=\sum_{s=1}^{t}g_s G t := ∑ s = 1 t g s , and for a fixed comparator x ⋆ ∈ K x^\star\in K x ⋆ ∈ K we write Reg T : = ∑ t = 1 T ⟨ g t , x t − x ⋆ ⟩ \operatorname{Reg}_T:=\sum_{t=1}^{T}\langle g_t,x_t-x^\star\rangle Reg T := ∑ t = 1 T ⟨ g t , x t − x ⋆ ⟩ .
Assumption 4.1. The body K K K has diameter at most D D D , the loss vectors satisfy ∥ g t ∥ ≤ L \lVert g_t\rVert\le L ∥ g t ∥ ≤ L for all t t t , and the LMO may break ties arbitrarily.
These are the conventions of Weibel et al. [20 ] , so that for m = 1 m = 1 m = 1 every statement reduces to the corresponding statement there.
Algorithm 1 m m m -step Online Frank–Wolfe
Require: x 1 ∈ K x_1 \in K x 1 ∈ K , η > 0 \eta> 0 η > 0 , σ ∈ ( 0 , 1 2 ] \sigma\in(0,\frac{1}{2}] σ ∈ ( 0 , 2 1 ]
for t = 1 , … , T t = 1,\ldots,T t = 1 , … , T do
play x t x_t x t , observe g t g_t g t ; set z t , 0 ← x t z_{t,0} \leftarrow x_t z t , 0 ← x t
for k = 1 , … , m k = 1,\ldots,m k = 1 , … , m do
d t , k ← η G t + ( z t , k − 1 − x 1 ) d_{t,k} \leftarrow\eta G_t + (z_{t,k-1} - x_1) d t , k ← η G t + ( z t , k − 1 − x 1 )
v t , k ← LMO ( d t , k ) v_{t,k} \leftarrow\operatorname{LMO}(d_{t,k}) v t , k ← LMO ( d t , k )
z t , k ← ( 1 − σ ) z t , k − 1 + σ v t , k z_{t,k} \leftarrow(1-\sigma)z_{t,k-1} + \sigma v_{t,k} z t , k ← ( 1 − σ ) z t , k − 1 + σ v t , k
end for
x t + 1 ← z t , m x_{t+1} \leftarrow z_{t,m} x t + 1 ← z t , m
end for
In Algorithm 1, the query d t , k d_{t,k} d t , k is the gradient at z t , k − 1 z_{t,k-1} z t , k − 1 of F t ( x ) = η ⟨ G t , x ⟩ + 1 2 ∥ x − x 1 ∥ 2 F_t(x) = \eta\langle G_t, x\rangle+ \frac{1}{2}\lVert x - x_1\rVert^2 F t ( x ) = η ⟨ G t , x ⟩ + 2 1 ∥ x − x 1 ∥ 2 , so a round consists of m m m Frank–Wolfe steps with constant step size σ \sigma σ on F t F_t F t , started at x t x_t x t . Every iterate is a convex combination of x 1 x_1 x 1 and of returned atoms, and every query is a linear combination of x 1 x_1 x 1 and of past gradients and atoms, so the method is a span method and Theorem 2.2 applies to it. Algorithm 1 enjoys a regret upper bound, stated in Theorem 4.2 , that matches the lower bound of Theorem 2.2 for m = O ( T ) m = O(T) m = O ( T ) .
Theorem 4.2 (Upper bound for OFW with m m m calls per round). Under Theorem 4.1, for every m , T ≥ 1 m,T \ge1 m , T ≥ 1 with m T ≥ 12 mT \ge12 m T ≥ 12 , and for every x ⋆ ∈ K x^\star\in K x ⋆ ∈ K , m m m -step Online Frank–Wolfe (Algorithm 1) with parameters
η = D 2 L ( 3 T ) 3 / 4 m 1 / 4 , σ = 3 m T , (4) \eta= \frac{D}{2L}\left(\frac{3}{T}\right)^{3/4}m^{1/4}, \qquad\sigma= \sqrt{\frac{3}{mT}}, \tag*{(4)} η = 2 L D ( T 3 ) 3/4 m 1/4 , σ = m T 3 , (4) has regret
Reg T ≤ 4 3 3 / 4 L D T 3 / 4 m − 1 / 4 + 4 3 1 / 4 L D ( m T ) 1 / 4 . \operatorname{Reg}_T \le\frac{4}{3^{3/4}}LD\,T^{3/4}m^{-1/4} + \frac{4}{3^{1/4}}LD\,(mT)^{1/4}. Reg T ≤ 3 3/4 4 L D T 3/4 m − 1/4 + 3 1/4 4 L D ( m T ) 1/4 . For m = 1 m = 1 m = 1 the bound Reg T ≤ 4 ⋅ 3 − 3 / 4 L D T 3 / 4 \operatorname{Reg}_T \le4\cdot3^{-3/4}LD\,T^{3/4} Reg T ≤ 4 ⋅ 3 − 3/4 L D T 3/4 holds without the second term.
The proof is in Appendix § 8 .§ 1 . Note that, by the standard linearization trick for OCO, the theorem extends to convex L L L -Lipschitz losses f t f_t f t : run the algorithm on the subgradients g t ∈ ∂ f t ( x t ) g_t \in\partial f_t(x_t) g t ∈ ∂ f t ( x t ) , and use that, by convexity, f t ( x t ) − f t ( u ) ≤ ⟨ g t , x t − u ⟩ f_t(x_t) - f_t(u) \le\langle g_t, x_t-u\rangle f t ( x t ) − f t ( u ) ≤ ⟨ g t , x t − u ⟩ for all u ∈ K u \in K u ∈ K .
The potential-function proof. Let y t + 1 : = arg min y ∈ K F t ( y ) y_{t+1} := \arg\min_{y\in K} F_t(y) y t + 1 := arg min y ∈ K F t ( y ) , with y 1 : = x 1 y_1 := x_1 y 1 := x 1 , be the exact follow-the-regularized-leader (FTRL) iterates. Weibel et al. [20 ] prove the constant 4 ⋅ 3 − 3 / 4 4\cdot3^{-3/4} 4 ⋅ 3 − 3/4 for m = 1 m = 1 m = 1 by tracking the potential
ϕ t : = ∑ s = 1 t ⟨ g s , x s − y t + 1 ⟩ − 1 2 η ∥ y t + 1 − x 1 ∥ 2 + 1 6 η ∥ x t + 1 − y t + 1 ∥ 2 . \phi_t := \sum_{s=1}^{t}\langle g_s,x_s-y_{t+1}\rangle- \frac{1}{2\eta}\lVert y_{t+1}-x_1\rVert^2 + \frac{1}{6\eta}\lVert x_{t+1}-y_{t+1}\rVert^2. ϕ t := s = 1 ∑ t ⟨ g s , x s − y t + 1 ⟩ − 2 η 1 ∥ y t + 1 − x 1 ∥ 2 + 6 η 1 ∥ x t + 1 − y t + 1 ∥ 2 .
Their proof is based on upper bounding the potential increase ϕ t − ϕ t − 1 \phi_t-\phi_{t-1} ϕ t − ϕ t − 1 for every round t ≥ 1 t\ge1 t ≥ 1 (with ϕ 0 = 0 \phi_0=0 ϕ 0 = 0 ). The LMO step enters their proof only through the squared distance ∥ x t + 1 − y t + 1 ∥ 2 \lVert x_{t+1}-y_{t+1}\rVert^2 ∥ x t + 1 − y t + 1 ∥ 2 , and is handled with the following one-step contraction argument. For a step z ′ = ( 1 − σ ) z + σ v z'=(1-\sigma)z+\sigma v z ′ = ( 1 − σ ) z + σ v , where v v v minimizes the linearization of F t F_t F t at z z z , the optimality of v v v and of y t + 1 y_{t+1} y t + 1 yields the one-step contraction
∥ z ′ − y t + 1 ∥ 2 ≤ ( 1 − 2 σ ) ∥ z − y t + 1 ∥ 2 + σ 2 ∥ z − v ∥ 2 . (5) \lVert z'-y_{t+1}\rVert^2 \le(1-2\sigma)\lVert z-y_{t+1}\rVert^2+\sigma^2\lVert z-v\rVert^2. \tag*{(5)} ∥ z ′ − y t + 1 ∥ 2 ≤ ( 1 − 2 σ ) ∥ z − y t + 1 ∥ 2 + σ 2 ∥ z − v ∥ 2 . (5)
For m = 1 m=1 m = 1 it is applied once, to z = x t z=x_t z = x t and z ′ = x t + 1 z'=x_{t+1} z ′ = x t + 1 ; the rest of the proof involves only x t x_t x t and the exact iterates y t , y t + 1 y_t,y_{t+1} y t , y t + 1 and does not depend on how x t + 1 x_{t+1} x t + 1 was produced. With m m m calls, the m m m steps of round t t t are LMO steps on the same F t F_t F t , towards the same y t + 1 y_{t+1} y t + 1 , so each of them satisfies (5) with ( z , v , z ′ ) = ( z t , k − 1 , v t , k , z t , k ) (z,v,z')=(z_{t,k-1},v_{t,k},z_{t,k}) ( z , v , z ′ ) = ( z t , k − 1 , v t , k , z t , k ) . Chaining the m m m inequalities, which requires 1 − 2 σ ≥ 0 1-2\sigma\ge0 1 − 2 σ ≥ 0 and is the reason for σ ≤ 1 2 \sigma\le\frac{1}{2} σ ≤ 2 1 , gives
∥ x t + 1 − y t + 1 ∥ 2 ≤ ( 1 − 2 σ ) m ∥ x t − y t + 1 ∥ 2 + σ 2 ∑ k = 1 m ( 1 − 2 σ ) m − k ∥ z t , k − 1 − v t , k ∥ 2 . (6) \lVert x_{t+1}-y_{t+1}\rVert^2 \le(1-2\sigma)^m\lVert x_t-y_{t+1}\rVert^2+\sigma^2\sum_{k=1}^{m}(1-2\sigma)^{m-k}\lVert z_{t,k-1}-v_{t,k}\rVert^2. \tag*{(6)} ∥ x t + 1 − y t + 1 ∥ 2 ≤ ( 1 − 2 σ ) m ∥ x t − y t + 1 ∥ 2 + σ 2 k = 1 ∑ m ( 1 − 2 σ ) m − k ∥ z t , k − 1 − v t , k ∥ 2 . (6)
A round with m m m LMO calls thus contracts the squared distance by the factor 1 − u 1-u 1 − u , where
u : = 1 − ( 1 − 2 σ ) m ∈ ( 0 , 1 ] . (7) u := 1-(1-2\sigma)^m \in(0,1]. \tag*{(7)} u := 1 − ( 1 − 2 σ ) m ∈ ( 0 , 1 ] . (7)
exactly as one LMO call contracts it by 1 − 2 σ 1-2\sigma 1 − 2 σ . The error of step k k k is damped by the m − k m-k m − k contractions that follow it. Replacing (5) by (6) in the proof of Weibel et al. [20 ] , and 2 σ 2\sigma 2 σ by u u u in their constants, gives the per-round inequality (Theorem C.1)
ϕ t − ϕ t − 1 ≤ 8 η 3 u ∥ g t ∥ 2 + σ 2 6 η ∑ k = 1 m ( 1 − 2 σ ) m − k ∥ z t , k − 1 − v t , k ∥ 2 . \phi_t-\phi_{t-1}\le\frac{8\eta}{3u}\lVert g_t\rVert^2+\frac{\sigma^2}{6\eta}\sum_{k=1}^{m}(1-2\sigma)^{m-k}\lVert z_{t,k-1}-v_{t,k}\rVert^2. ϕ t − ϕ t − 1 ≤ 3 u 8 η ∥ g t ∥ 2 + 6 η σ 2 k = 1 ∑ m ( 1 − 2 σ ) m − k ∥ z t , k − 1 − v t , k ∥ 2 .
Summing over t t t and using the optimality of y T + 1 y_{T+1} y T + 1 , ∥ z t , k − 1 − v t , k ∥ ≤ D \lVert z_{t,k-1}-v_{t,k}\rVert\le D ∥ z t , k − 1 − v t , k ∥ ≤ D and ∑ k = 1 m ( 1 − 2 σ ) m − k = u / ( 2 σ ) \sum_{k=1}^{m}(1-2\sigma)^{m-k}=u/(2\sigma) ∑ k = 1 m ( 1 − 2 σ ) m − k = u / ( 2 σ ) gives, for every x ⋆ ∈ K x_\star\in K x ⋆ ∈ K ,
Reg T ≤ 8 η L 2 T 3 u + σ u D 2 T 12 η + D 2 2 η . \operatorname{Reg}_T \le\frac{8\eta L^2T}{3u}+\frac{\sigma uD^2T}{12\eta}+\frac{D^2}{2\eta}. Reg T ≤ 3 u 8 η L 2 T + 12 η σ u D 2 T + 2 η D 2 .
(Theorem C.2 ), the same bound as in Weibel et al. [20 ] for single-call OFW, with 2 σ 2\sigma 2 σ replaced by u u u .
Since u ≈ 2 m σ u \approx2m\sigma u ≈ 2 mσ at first order, a round of m m m LMO steps tracks y t + 1 y_{t+1} y t + 1 like a single step of size m σ m\sigma mσ (same first error term), while its atom error for the second term pays a factor m m m instead of m 2 m^2 m 2 . For u = 2 m σ u=2m\sigma u = 2 mσ the three terms would be 2, 1 and 1 times 3 − 3 / 4 L D T 3 / 4 m − 1 / 4 3^{-3/4}LD T^{3/4}m^{-1/4} 3 − 3/4 L D T 3/4 m − 1/4 , whence the constant 4 ⋅ 3 − 3 / 4 4\cdot3^{-3/4} 4 ⋅ 3 − 3/4 for every m m m . The lower-order term 4 ⋅ 3 − 1 / 4 L D ( m T ) 1 / 4 4\cdot3^{-1/4}LD(mT)^{1/4} 4 ⋅ 3 − 1/4 L D ( m T ) 1/4 accounts for u ≠ 2 m σ u\ne2m\sigma u = 2 mσ through 1 / u ≤ 1 / ( 2 m σ ) + 1 1/u\le1/(2m\sigma)+1 1/ u ≤ 1/ ( 2 mσ ) + 1 .
Anytime tuning. An adaptation of this proof (Section C.2) handles time-varying parameters, that is, Algorithm 1 with η , σ \eta,\sigma η , σ replaced in round t − 1 t-1 t − 1 by
η t = D 2 L ( 3 t ) 3 / 4 m 1 / 4 , σ t = min { 1 2 , 3 m t } . (8) \eta_t=\frac{D}{2L}\left(\frac{3}{t}\right)^{3/4}m^{1/4},\qquad\sigma_t=\min\left\{\frac{1}{2},\sqrt{\frac{3}{mt}}\right\}. \tag*{(8)} η t = 2 L D ( t 3 ) 3/4 m 1/4 , σ t = min { 2 1 , m t 3 } . (8)
to compute x t x_t x t . This method does not depend on T T T .
Theorem 4.3 (Anytime m m m -step OFW, Upper bound). Under Theorem 4.1, for every m , T ≥ 1 m,T\ge1 m , T ≥ 1 and x ⋆ ∈ K x_\star\in K x ⋆ ∈ K , Algorithm 1 with the time-varying parameters (8) has regret
Reg T ≤ 5 3 3 / 4 L D T 3 / 4 m − 1 / 4 + 57 L D ( m T ) 1 / 4 . \operatorname{Reg}_T\le\frac{5}{3^{3/4}}LD T^{3/4}m^{-1/4}+57LD(mT)^{1/4}. Reg T ≤ 3 3/4 5 L D T 3/4 m − 1/4 + 57 L D ( m T ) 1/4 .
The lower-order constant is crude. For m = 1 m=1 m = 1 and the schedule σ t = min { 1 , 3 / t } \sigma_t=\min\{1,\sqrt{3/t}\} σ t = min { 1 , 3/ t } , Weibel et al. [20 ] prove Reg T ≤ 5 ⋅ 3 − 3 / 4 L D T 3 / 4 \operatorname{Reg}_T\le5\cdot3^{-3/4}LD T^{3/4} Reg T ≤ 5 ⋅ 3 − 3/4 L D T 3/4 for every T T T , without the lower-order term.
The regime m ≥ T m\ge T m ≥ T . The above tunings of Theorems 4.2 and 4.3 are suboptimal for m ≥ T m\ge T m ≥ T since the term ( m T ) 1 / 4 (mT)^{1/4} ( m T ) 1/4 dominates. In this case, replacing m m m with min { m , t } \min\{m,t\} min { m , t } in parameter η t \eta_t η t , (and with min { m , T } \min\{m,T\} min { m , T } in the fixed-horizon parameter η = η T \eta=\eta_T η = η T ), that is setting
η ~ t = D 2 L ( 3 t ) 3 / 4 min { m , t } 1 / 4 , (9) \widetilde{\eta}_t=\frac{D}{2L}\left(\frac{3}{t}\right)^{3/4}\min\{m,t\}^{1/4}, \tag*{(9)} η t = 2 L D ( t 3 ) 3/4 min { m , t } 1/4 , (9)
recovers the minimax T \sqrt{T} T rate of OCO with projection. With m ≥ T m\ge T m ≥ T LMO calls per round, the OFW iterate tracks the exact FTRL iterate closely enough that the method inherits its T \sqrt{T} T regret.
Corollary 4.4 (m m m -step OFW with capped tuning).
Fixed-horizon case: Under Theorem 4.1, for every m , T ≥ 1 m,T\ge1 m , T ≥ 1 with m T ≥ 12 mT\ge12 m T ≥ 12 , and every x ⋆ ∈ K x_\star\in K x ⋆ ∈ K , Algorithm 1 with parameters σ \sigma σ as in (4) and η = η ~ T \eta=\widetilde{\eta}_T η = η T as in (9) has regret
Reg T ≤ 4.8 L D max { T 3 / 4 m − 1 / 4 , T } . \operatorname{Reg}_T\le4.8LD\max\{T^{3/4}m^{-1/4},\sqrt{T}\}. Reg T ≤ 4.8 L D max { T 3/4 m − 1/4 , T } . Anytime case: Under Theorem 4.1, for every m , T ≥ 1 m,T\ge1 m , T ≥ 1 , and every x ⋆ ∈ K x_\star\in K x ⋆ ∈ K , Algorithm 1 with anytime parameters σ t \sigma_t σ t as in (8) and η t = η ~ t \eta_t=\widetilde{\eta}_t η t = η t as in (9) has regret
Reg T ≤ 60 L D max { T 3 / 4 m − 1 / 4 , T } . \operatorname{Reg}_T\le60LD\max\{T^{3/4}m^{-1/4},\sqrt{T}\}. Reg T ≤ 60 L D max { T 3/4 m − 1/4 , T } .
The complete proof is in Section C.3. Note that the constant 60 of the anytime bound is crude.
Conclusion
In online convex optimization, the T 3 / 4 T^{3/4} T 3/4 regret of OFW is the price of accessing a high-dimensional body through O ( 1 ) O(1) O ( 1 ) linear optimization calls per round: one known simplex in dimension 2 m ( T − 1 ) + 2 2m(T-1)+2 2 m ( T − 1 ) + 2 enforces it on every span method; and a random rotation of the same simplex in dimension ≳ ( m T ) 7 / 2 \gtrsim(mT)^{7/2} ≳ ( m T ) 7/2 , with a small ball as decoy, enforces it on every learner. With m m m calls per round the minimax regret is Θ ( L D max { T 3 / 4 m − 1 / 4 , T } ) \Theta(LD\max\{T^{3/4}m^{-1/4},\sqrt{T}\}) Θ ( L D max { T 3/4 m − 1/4 , T }) , attained by a generalization of OFW with m m m LMO calls per round.
Acknowledgments
This work took place in the context of the associate team 4TUNE within CWI-Inria international lab . Julien Weibel and Adrien Taylor are supported by the European Union (ERC grant CASPER 101162889). Views and opinions expressed are however those of the author(s) only and do not necessarily reflect those of the European Union or the European Research Council. Neither the European Union nor the granting authority can be held responsible for them. The French government also partly funded this work under the management of Agence Nationale de la Recherche as part of the “France 2030” program, reference ANR-23-IACL-0008 (PR[AI]RIE-PSAI) and reference ANR-23-IACL-0006 (MIAI Cluster).
AI use statement
Generative AI tools were used in this work to draft and revise the text, to carry out numerical and symbolic verification of the constants, and to search the literature. All AI-assisted proofs were checked line by line by the authors, and the literature search was complemented by a manual survey. We take responsibility for the final content of this work, including text, claims, and artifacts produced with the aid of generative AI.
References
Proofs for Section 2: span methods
This section contains the complete proofs of all the results in § 2 , and in particular Theorem 2.2 . Throughout this section, the instance we consider is the one defined in § 2.1 : unshrunk atoms w ˉ j \bar{w}_j w ˉ j in R 2 N \mathbb{R}^{2N} R 2 N , the simplex K ˉ \bar{K} K ˉ , the canonical losses g ˉ ( κ ) \bar{g}^{(\kappa)} g ˉ ( κ ) with the pattern (3) , the schedule κ t = m t \kappa_t = mt κ t = m t , and the oracle returning the first minimizer in the list 0 , w ˉ 0 , … , w ˉ N − 1 0,\bar{w}_0,\ldots,\bar{w}_{N-1} 0 , w ˉ 0 , … , w ˉ N − 1 .
Preliminary bounds on the profile e j e_j e j
Recall from § 2.1 that h k = 2 k + 1 h_k = \sqrt{2k+1} h k = 2 k + 1 , h ^ k = h k / ( 2 N ) \hat{h}_k = h_k/(2\sqrt{N}) h ^ k = h k / ( 2 N ) , and that the profile is e j = h ^ j e_j = \hat{h}_j e j = h ^ j in the lower half j < N / 2 j < N/2 j < N /2 and e j = 1 − h ^ N − 1 − j e_j = 1 - \hat{h}_{N-1-j} e j = 1 − h ^ N − 1 − j in the upper half j ≥ N / 2 j \ge N/2 j ≥ N /2 , with increments δ j = e j − e j − 1 \delta_j = e_j - e_{j-1} δ j = e j − e j − 1 and noise sizes f j = min ( e j , 1 − e j ) f_j = \min(e_j,1-e_j) f j = min ( e j , 1 − e j ) .
To prove preliminary bounds on the profile e j e_j e j , we will need the following auxiliary sequences
a j = min ( j , N − j ) , b j = min ( j , N − 1 − j ) , a_j = \min(j,N-j), \qquad b_j = \min(j,N-1-j), a j = min ( j , N − j ) , b j = min ( j , N − 1 − j ) ,
ψ 0 = 2 , ψ k = h k + h k − 1 ( k ≥ 1 ) , Ψ j = ψ a j . \psi_0 = 2, \qquad\psi_k = h_k + h_{k-1}\ (k \ge1), \qquad\Psi_j = \psi_{a_j}. ψ 0 = 2 , ψ k = h k + h k − 1 ( k ≥ 1 ) , Ψ j = ψ a j .
Since h k 2 − h k − 1 2 = 2 h_k^2-h_{k-1}^2=2 h k 2 − h k − 1 2 = 2 , we have ψ k = 2 / ( h k − h k − 1 ) \psi_k = 2/(h_k-h_{k-1}) ψ k = 2/ ( h k − h k − 1 ) for k ≥ 1 k \ge1 k ≥ 1 .
Lemma A.1 (Properties of the profile e j e_j e j ). The following statements hold:
(a) Monotonicity: The profile is strictly increasing and lies in ( 0 , 1 ) (0,1) ( 0 , 1 ) ; in particular all δ j \delta_j δ j and f j f_j f j are positive.
(b) Noise sizes: N f j = h b j / 2 \sqrt{N}f_j = h_{b_j}/2 N f j = h b j /2 .
(c) Increments: N δ j ≥ 1 / Ψ j \sqrt{N}\delta_j \ge1/\Psi_j N δ j ≥ 1/ Ψ j .
(d) Index functions: for 0 ≤ i ≤ N − 2 0 \le i \le N-2 0 ≤ i ≤ N − 2 , a i + 1 ≥ 1 a_{i+1} \ge1 a i + 1 ≥ 1 and b i ∈ { a i + 1 − 1 , a i + 1 } b_i \in\{a_{i+1}-1,a_{i+1}\} b i ∈ { a i + 1 − 1 , a i + 1 } .
(e) Gaps: h 0 = 1 h_0=1 h 0 = 1 and 0 < h k − h k − 1 ≤ 1 0 < h_k-h_{k-1} \le1 0 < h k − h k − 1 ≤ 1 for k ≥ 1 k \ge1 k ≥ 1 .
(f) Distances: e j + f j ≤ 1 e_j+f_j \le1 e j + f j ≤ 1 for every j j j , and e j − e i + f i + f j ≤ 1 e_j-e_i+f_i+f_j \le1 e j − e i + f i + f j ≤ 1 for i < j i<j i < j .
Facts (b) and (c) express the noise sizes and increments, after multiplication by N \sqrt{N} N , through the explicit numbers h k h_k h k . Together with (d) and (e), they are the inputs of the two recursions that lower bound the per-round costs: Theorem A.2 below, used for both instances through Theorems B.3 and 2.5, and Theorem B.4 for the hidden instance. Fact (a) gives the simplex structure and fact (f) the diameter bound of span-method instance. The proof of the lemma is an elementary case analysis.
Proof of Theorem A.1. The facts are checked one at a time.
(a) Monotonicity. In the lower half j < N / 2 j<N/2 j < N /2 , we have 2 j + 1 ≤ N 2j+1\le N 2 j + 1 ≤ N , so e j = h ^ j ≤ 1 2 e_j=\hat{h}_j\le\frac{1}{2} e j = h ^ j ≤ 2 1 . In the upper half j ≥ N / 2 j\ge N/2 j ≥ N /2 , we have 2 ( N − 1 − j ) + 1 ≤ N − 1 2(N-1-j)+1\le N-1 2 ( N − 1 − j ) + 1 ≤ N − 1 , so h ^ N − 1 − j < 1 2 \hat{h}_{N-1-j}<\frac{1}{2} h ^ N − 1 − j < 2 1 and e j = 1 − h ^ N − 1 − j > 1 2 e_j=1-\hat{h}_{N-1-j}>\frac{1}{2} e j = 1 − h ^ N − 1 − j > 2 1 . The profile is increasing within each half, since h ^ k \hat{h}_k h ^ k is increasing in k k k , and it is increasing across the middle because every lower value is at most 1 2 \frac{1}{2} 2 1 and every upper value is above 1 2 \frac{1}{2} 2 1 . Finally e 0 = h ^ 0 > 0 e_0=\hat{h}_0>0 e 0 = h ^ 0 > 0 and e N − 1 = 1 − h ^ 0 < 1 e_{N-1}=1-\hat{h}_0<1 e N − 1 = 1 − h ^ 0 < 1 . Hence, all δ j \delta_j δ j and f j f_j f j are positive, since the profile is strictly increasing with e − 1 = 0 e_{-1}=0 e − 1 = 0 and lies in ( 0 , 1 ) (0,1) ( 0 , 1 ) . (b) Noise sizes. By the bounds of (a), the noise size is the hatted term in both halves:
f j = min ( e j , 1 − e j ) = { h ^ j , j < N / 2 , h ^ N − 1 − j , j ≥ N / 2. f_j=\min(e_j,1-e_j)=
\begin{cases}
\hat{h}_j, & j<N/2,\\
\hat{h}_{N-1-j}, & j\geq N/2.
\end{cases} f j = min ( e j , 1 − e j ) = { h ^ j , h ^ N − 1 − j , j < N /2 , j ≥ N /2.
which is h ^ b j \hat{h}_{b_j} h ^ b j , because b j = j b_j=j b j = j in the lower half and b j = N − 1 − j b_j=N-1-j b j = N − 1 − j in the upper half. Multiplying by N \sqrt{N} N gives (b).
(c) Increments. We show 1 / δ j ≤ N Ψ j 1/\delta_j\leq\sqrt{N}\,\Psi_j 1/ δ j ≤ N Ψ j by distinguishing four cases according to the value of j j j .
j = 0 j=0 j = 0 : We have δ 0 = h ^ 0 = 1 / ( 2 N ) \delta_0=\hat{h}_0=1/(2\sqrt{N}) δ 0 = h ^ 0 = 1/ ( 2 N ) , so 1 / δ 0 = 2 N = N ψ 0 1/\delta_0=2\sqrt{N}=\sqrt{N}\,\psi_0 1/ δ 0 = 2 N = N ψ 0 as a 0 = 0 a_0=0 a 0 = 0 .
1 ≤ j < N / 2 1\leq j<N/2 1 ≤ j < N /2 : Both j j j and j − 1 j-1 j − 1 are in the lower half, δ j = ( h j − h j − 1 ) / ( 2 N ) \delta_j=(h_j-h_{j-1})/(2\sqrt{N}) δ j = ( h j − h j − 1 ) / ( 2 N ) , so 1 / δ j = N ψ j 1/\delta_j=\sqrt{N}\,\psi_j 1/ δ j = N ψ j as a j = j a_j=j a j = j .
j > N / 2 j>N/2 j > N /2 : Then we have δ j = h ^ N − j − h ^ N − 1 − j \delta_j=\hat{h}_{N-j}-\hat{h}_{N-1-j} δ j = h ^ N − j − h ^ N − 1 − j . This is clear when j − 1 j-1 j − 1 is also in the upper half. Otherwise N N N is odd and j = ( N + 1 ) / 2 j=(N+1)/2 j = ( N + 1 ) /2 , and it still holds because e j − 1 = h ^ ( N − 1 ) / 2 = 1 / 2 = 1 − h ^ N − j e_{j-1}=\hat{h}_{(N-1)/2}=1/2=1-\hat{h}_{N-j} e j − 1 = h ^ ( N − 1 ) /2 = 1/2 = 1 − h ^ N − j , since j − 1 = ( N − 1 ) / 2 = N − j j-1=(N-1)/2=N-j j − 1 = ( N − 1 ) /2 = N − j . So in both cases, we have 1 / δ j = N ψ N − j 1/\delta_j=\sqrt{N}\,\psi_{N-j} 1/ δ j = N ψ N − j as a j = N − j a_j=N-j a j = N − j .
j = N / 2 j=N/2 j = N /2 (N N N even): Here δ j = 1 − 2 h ^ j − 1 = 1 − ( N − 1 ) / N \delta_j=1-2\hat{h}_{j-1}=1-\sqrt{(N-1)/N} δ j = 1 − 2 h ^ j − 1 = 1 − ( N − 1 ) / N , and as a j = N / 2 a_j=N/2 a j = N /2 , we have
1 δ j = N ( N + N − 1 ) ≤ N ( N + 1 + N − 1 ) = N ψ N / 2 . \frac{1}{\delta_j}=\sqrt{N}(\sqrt{N}+\sqrt{N-1})\leq\sqrt{N}(\sqrt{N+1}+\sqrt{N-1})=\sqrt{N}\,\psi_{N/2}. δ j 1 = N ( N + N − 1 ) ≤ N ( N + 1 + N − 1 ) = N ψ N /2 .
(d) Index functions. For 0 ≤ i ≤ N − 2 0\leq i\leq N-2 0 ≤ i ≤ N − 2 , a i + 1 = min ( i + 1 , N − 1 − i ) ≥ 1 a_{i+1}=\min(i+1,N-1-i)\geq1 a i + 1 = min ( i + 1 , N − 1 − i ) ≥ 1 . If i + 1 ≤ N − 1 − i i+1\leq N-1-i i + 1 ≤ N − 1 − i , then a i + 1 = i + 1 a_{i+1}=i+1 a i + 1 = i + 1 and b i = i b_i=i b i = i . Otherwise N − 1 − i ≤ i N-1-i\leq i N − 1 − i ≤ i , and a i + 1 = N − 1 − i = b i a_{i+1}=N-1-i=b_i a i + 1 = N − 1 − i = b i .
(e) Gaps of h h h . We have h 0 = 1 h_0=1 h 0 = 1 . For k ≥ 1 k\geq1 k ≥ 1 the gap is h k − h k − 1 = 2 / ( h k + h k − 1 ) h_k-h_{k-1}=2/(h_k+h_{k-1}) h k − h k − 1 = 2/ ( h k + h k − 1 ) , which is at most 2 / ( 1 + 3 ) < 1 2/(1+\sqrt{3})<1 2/ ( 1 + 3 ) < 1 .
(f) Distances. For every j j j , we have e j + f j = min ( 2 e j , 1 ) ≤ 1 e_j+f_j=\min(2e_j,1)\leq1 e j + f j = min ( 2 e j , 1 ) ≤ 1 . For i < j i<j i < j there are three cases. If both e i e_i e i and e j e_j e j are at most 1 / 2 1/2 1/2 , the sum e j − e i + f i + f j e_j-e_i+f_i+f_j e j − e i + f i + f j equals 2 e j ≤ 1 2e_j\leq1 2 e j ≤ 1 . If both are at least 1 / 2 1/2 1/2 , it equals 2 − 2 e i ≤ 1 2-2e_i\leq1 2 − 2 e i ≤ 1 (recall e i < 1 e_i<1 e i < 1 ). If e i ≤ 1 / 2 ≤ e j e_i\leq1/2\leq e_j e i ≤ 1/2 ≤ e j , it equals 1 1 1 . In all three cases, the sum is at most 1 1 1 .
Proof of lem:ties
Proof of Theorem 2.3. By linearity, it suffices to check ⟨ d , w ˉ j ⟩ = ⟨ d , w ˉ J ⟩ \langle d,\bar{w}_j\rangle=\langle d,\bar{w}_J\rangle ⟨ d , w ˉ j ⟩ = ⟨ d , w ˉ J ⟩ for j ≥ J j\geq J j ≥ J when d d d is one of the generating vectors. For d = 0 d=0 d = 0 both sides vanish. For d = w ˉ i d=\bar{w}_i d = w ˉ i with i < J ≤ j i<J\leq j i < J ≤ j , the martingale identity gives ⟨ w ˉ i , w ˉ j ⟩ = e i = ⟨ w ˉ i , w ˉ J ⟩ \langle\bar{w}_i,\bar{w}_j\rangle=e_i=\langle\bar{w}_i,\bar{w}_J\rangle ⟨ w ˉ i , w ˉ j ⟩ = e i = ⟨ w ˉ i , w ˉ J ⟩ . For d = g s = g ˉ ( κ s ) d=g_s=\bar{g}^{(\kappa_s)} d = g s = g ˉ ( κ s ) with s ≤ t − 1 s\leq t-1 s ≤ t − 1 , we have κ s ≤ κ t − 1 ≤ J ≤ j \kappa_s\leq\kappa_{t-1}\leq J\leq j κ s ≤ κ t − 1 ≤ J ≤ j , so (3) gives ⟨ g s , w ˉ j ⟩ = − c ˉ κ s = ⟨ g s , w ˉ J ⟩ \langle g_s,\bar{w}_j\rangle=-\bar{c}_{\kappa_s}=\langle g_s,\bar{w}_J\rangle ⟨ g s , w ˉ j ⟩ = − c ˉ κ s = ⟨ g s , w ˉ J ⟩ . Hence all atoms of index at least J J J have the same value ⟨ d , w ˉ J ⟩ \langle d,\bar{w}_J\rangle ⟨ d , w ˉ J ⟩ , which proves the first statement of the lemma.
In particular, for any query d ∈ span ( 0 , w ˉ 0 , … , w ˉ J − 1 , g 0 , … , g t − 1 ) d\in\operatorname{span}(0,\bar{w}_0,\ldots,\bar{w}_{J-1},g_0,\ldots,g_{t-1}) d ∈ span ( 0 , w ˉ 0 , … , w ˉ J − 1 , g 0 , … , g t − 1 ) , the first minimizer of ⟨ d , ⋅ ⟩ \langle d,\cdot\rangle ⟨ d , ⋅ ⟩ over the list 0 , w ˉ 0 , … , w ˉ N − 1 0,\bar{w}_0,\ldots,\bar{w}_{N-1} 0 , w ˉ 0 , … , w ˉ N − 1 is either 0 0 0 or an atom w ˉ i \bar{w}_i w ˉ i with index i ≤ J i\leq J i ≤ J . In both cases LMO ( d ) ∈ { 0 , w ˉ 0 , … , w ˉ J } \operatorname{LMO}(d)\in\{0,\bar{w}_0,\ldots,\bar{w}_J\} LMO ( d ) ∈ { 0 , w ˉ 0 , … , w ˉ J } .
Proof of prop:span
Proof of Theorem 2.4. We first show by induction over the queries that the k k k -th query of round t ≥ 1 t\geq1 t ≥ 1 returns 0 0 0 or an atom of index at most J : = m ( t − 1 ) + k − 1 J:=m(t-1)+k-1 J := m ( t − 1 ) + k − 1 . Indeed, by the induction hypothesis the atoms returned before this query have index below J J J , so d t , k ∈ span ( 0 , w ˉ 0 , … , w ˉ J − 1 , g 0 , … , g t − 1 ) d_{t,k}\in\operatorname{span}(0,\bar{w}_0,\ldots,\bar{w}_{J-1},g_0,\ldots,g_{t-1}) d t , k ∈ span ( 0 , w ˉ 0 , … , w ˉ J − 1 , g 0 , … , g t − 1 ) since the learner is assumed to be a span method, and applying Theorem 2.3 (note that J ≥ κ t − 1 = m ( t − 1 ) J\geq\kappa_{t-1}=m(t-1) J ≥ κ t − 1 = m ( t − 1 ) ) completes the induction step.
In particular, every atom returned before the play of round t t t has index at most m ( t − 1 ) + m − 1 = κ t − 1 m(t-1)+m-1=\kappa_t-1 m ( t − 1 ) + m − 1 = κ t − 1 . Consequently, x t x_t x t is a convex combination of 0 0 0 and of atoms w ˉ j \bar{w}_j w ˉ j of index j < κ t j<\kappa_t j < κ t , which all have zero inner product with the loss vector g t = g ˉ ( κ t ) g_t=\bar{g}^{(\kappa_t)} g t = g ˉ ( κ t ) by (3). Hence we have ⟨ g t , x t ⟩ = 0 \langle g_t,x_t\rangle=0 ⟨ g t , x t ⟩ = 0 for all t < T t<T t < T .
Finally, the cumulative loss u ↦ ∑ t < T ⟨ g t , u ⟩ u\mapsto\sum_{t<T}\langle g_t,u\rangle u ↦ ∑ t < T ⟨ g t , u ⟩ is linear, so its minimum over K ˉ \bar{K} K ˉ is attained at a vertex: it is 0 0 0 at the origin and − ∑ t : κ t ≤ j c ˉ κ t -\sum_{t:\kappa_t\leq j}\bar{c}_{\kappa_t} − ∑ t : κ t ≤ j c ˉ κ t at atom w ˉ j \bar{w}_j w ˉ j by (3). As every c ˉ κ \bar{c}_\kappa c ˉ κ is positive, the minimum is attained at j = N − 1 j=N-1 j = N − 1 and, since κ t ≤ N − 1 \kappa_t\leq N-1 κ t ≤ N − 1 for all t < T t<T t < T , equals − ∑ t < T c ˉ κ t -\sum_{t<T}\bar{c}_{\kappa_t} − ∑ t < T c ˉ κ t . Therefore Reg T = ∑ t < T c ˉ κ t \operatorname{Reg}_T=\sum_{t<T}\bar{c}_{\kappa_t} Reg T = ∑ t < T c ˉ κ t for all span methods.
Proof of lem:cbar
We first compute the distance from the walk position M κ M_\kappa M κ to the span of the head atoms, by eliminating the unknowns of a least-squares problem one at a time along the chain. Each elimination uses the same identity: for x , y > 0 x,y>0 x , y > 0 define the parallel sum
x ∥ y = x y x + y , equivalently 1 x ∥ y = 1 x + 1 y , x\mathbin{\|}y=\frac{xy}{x+y},\qquad\text{equivalently}\qquad\frac{1}{x\mathbin{\|}y}=\frac{1}{x}+\frac{1}{y}, x ∥ y = x + y x y , equivalently x ∥ y 1 = x 1 + y 1 ,
the formula for two electrical resistances in parallel. It is increasing in x x x and in y y y , and it is the value of a one-variable quadratic problem:
min t ∈ R ( x t 2 + y ( t − 1 ) 2 ) = x ∥ y . (10) \min_{t\in\mathbb{R}}\left(xt^2+y(t-1)^2\right)=x\mathbin{\|}y. \tag*{(10)} t ∈ R min ( x t 2 + y ( t − 1 ) 2 ) = x ∥ y . (10)
Lemma A.2 (Distance of M κ M_\kappa M κ to the head span). For 0 ≤ κ ≤ N − 1 0\leq\kappa\leq N-1 0 ≤ κ ≤ N − 1 , we have
dist ( M κ , span ( w ˉ 0 , … , w ˉ κ − 1 ) ) 2 = q κ : = min Z 0 , … , Z κ − 1 [ ∑ i = 0 κ δ i Z i 2 + ∑ s = 0 κ − 1 f s ( Z s − Z s + 1 ) 2 ] with Z κ = 1 , \operatorname{dist}\left(M_\kappa,\operatorname{span}(\bar w_0,\ldots,\bar w_{\kappa-1})\right)^2=q_\kappa:=\min_{Z_0,\ldots,Z_{\kappa-1}}\left[\sum_{i=0}^{\kappa}\delta_i Z_i^2+\sum_{s=0}^{\kappa-1}f_s(Z_s-Z_{s+1})^2\right]\quad\text{with }Z_\kappa=1, dist ( M κ , span ( w ˉ 0 , … , w ˉ κ − 1 ) ) 2 = q κ := Z 0 , … , Z κ − 1 min [ i = 0 ∑ κ δ i Z i 2 + s = 0 ∑ κ − 1 f s ( Z s − Z s + 1 ) 2 ] with Z κ = 1 , and q κ ≥ 1 / ( 2 N ) q_\kappa\geq1/(2\sqrt{N}) q κ ≥ 1/ ( 2 N ) . In particular, we have
dist ( M κ , span ( w ˉ 0 , … , w ˉ κ − 1 ) ) ≥ ( 4 N ) − 1 / 4 . \operatorname{dist}\left(M_\kappa,\operatorname{span}(\bar w_0,\ldots,\bar w_{\kappa-1})\right)\geq(4N)^{-1/4}. dist ( M κ , span ( w ˉ 0 , … , w ˉ κ − 1 ) ) ≥ ( 4 N ) − 1/4 .
Proof. Step 1: The equality. By definition, the squared distance dist ( M κ , span ( w ˉ 0 , … , w ˉ κ − 1 ) ) 2 \operatorname{dist}(M_\kappa,\operatorname{span}(\bar w_0,\ldots,\bar w_{\kappa-1}))^2 dist ( M κ , span ( w ˉ 0 , … , w ˉ κ − 1 ) ) 2 is equal to min y 0 , … , y κ − 1 ∥ M κ − ∑ s < κ y s w ˉ s ∥ 2 \min_{y_0,\ldots,y_{\kappa-1}}\left\|M_\kappa-\sum_{s<\kappa}y_s\bar w_s\right\|^2 min y 0 , … , y κ − 1 M κ − ∑ s < κ y s w ˉ s 2 . We write the vector inside the norm in the (orthonormal) basis ( u i , ν i ) i < N (u_i,\nu_i)_{i<N} ( u i , ν i ) i < N and replace the coefficients y s y_s y s by the cumulative quantities Z i = 1 − ∑ s = i κ − 1 y s Z_i=1-\sum_{s=i}^{\kappa-1}y_s Z i = 1 − ∑ s = i κ − 1 y s for 0 ≤ i ≤ κ 0\leq i\leq\kappa 0 ≤ i ≤ κ , so that Z κ = 1 Z_\kappa=1 Z κ = 1 and y s = Z s + 1 − Z s y_s=Z_{s+1}-Z_s y s = Z s + 1 − Z s . This change of variables is a bijection. The coordinate along u i u_i u i collects δ i \sqrt{\delta_i} δ i from M κ M_\kappa M κ and − δ i -\sqrt{\delta_i} − δ i from each atom w ˉ s \bar w_s w ˉ s with s ≥ i s\geq i s ≥ i , so it equals δ i Z i \sqrt{\delta_i}Z_i δ i Z i . The coordinate along ν s \nu_s ν s is − y s f s = f s ( Z s − Z s + 1 ) -y_s\sqrt{f_s}=\sqrt{f_s}(Z_s-Z_{s+1}) − y s f s = f s ( Z s − Z s + 1 ) . Summing the squares of those coordinates gives the equality of the lemma.
Step 2: The sequence ( q k ) k < N (q_k)_{k<N} ( q k ) k < N satisfies a recursion formula. The problem defining q k + 1 q_{k+1} q k + 1 consists of the problem for q k q_k q k , with Z k Z_k Z k now free, plus the two terms δ k + 1 Z k + 1 2 = δ k + 1 \delta_{k+1}Z_{k+1}^2=\delta_{k+1} δ k + 1 Z k + 1 2 = δ k + 1 and f k ( Z k − 1 ) 2 f_k(Z_k-1)^2 f k ( Z k − 1 ) 2 . If Z k Z_k Z k is fixed to a value t t t instead of 1 1 1 in the definition of q k q_k q k , the minimum becomes q k t 2 q_kt^2 q k t 2 by homogeneity. Thus, minimizing first over Z 0 , … , Z k − 1 Z_0,\ldots,Z_{k-1} Z 0 , … , Z k − 1 and then over Z k Z_k Z k in the definition of q k + 1 q_{k+1} q k + 1 gives
q k + 1 = δ k + 1 + min Z k [ f k ( Z k − 1 ) 2 + min Z 0 , … , Z k − 1 [ ∑ i = 0 k δ i Z i 2 + ∑ s = 0 k − 1 f s ( Z s − Z s + 1 ) 2 ] ] = δ k + 1 + min Z k [ f k ( Z k − 1 ) 2 + q k Z k 2 ] , \begin{aligned}
q_{k+1}=\delta_{k+1}+\min_{Z_k}\left[f_k(Z_k-1)^2+\min_{Z_0,\ldots,Z_{k-1}}\left[\sum_{i=0}^{k}\delta_iZ_i^2+\sum_{s=0}^{k-1}f_s(Z_s-Z_{s+1})^2\right]\right] \\
&=\delta_{k+1}+\min_{Z_k}\left[f_k(Z_k-1)^2+q_kZ_k^2\right],
\end{aligned} q k + 1 = δ k + 1 + Z k min [ f k ( Z k − 1 ) 2 + Z 0 , … , Z k − 1 min [ i = 0 ∑ k δ i Z i 2 + s = 0 ∑ k − 1 f s ( Z s − Z s + 1 ) 2 ] ] = δ k + 1 + Z k min [ f k ( Z k − 1 ) 2 + q k Z k 2 ] ,
and using (10) gives the recursion
q 0 = δ 0 , q k + 1 = δ k + 1 + ( q k ∥ f k ) . q_0=\delta_0,\qquad q_{k+1}=\delta_{k+1}+(q_k\mathbin{\|}f_k). q 0 = δ 0 , q k + 1 = δ k + 1 + ( q k ∥ f k ) .
Step 3: The recursion stays above 1 / ( 2 N ) 1/(2\sqrt{N}) 1/ ( 2 N ) . Let Q k = N q k Q_k=\sqrt{N}q_k Q k = N q k . We show by induction that Q k ≥ 1 2 Q_k\geq\frac12 Q k ≥ 2 1 for all k < N k<N k < N . For k = 0 k=0 k = 0 , we have Q 0 = N δ 0 = 1 2 Q_0=\sqrt{N}\delta_0=\frac12 Q 0 = N δ 0 = 2 1 . Let k > 0 k>0 k > 0 , and assume that Q k − 1 ≥ 1 2 Q_{k-1}\geq\frac12 Q k − 1 ≥ 2 1 . Multiplying the recursion by N \sqrt{N} N and using Theorem A.1 (b,c), we get
Q k ≥ 1 Ψ k + ( Q k − 1 ∥ h b 2 ) , b = b k − 1 . Q_k\geq\frac{1}{\Psi_k}+\left(Q_{k-1}\mathbin{\|}\frac{h_b}{2}\right),\qquad b=b_{k-1}. Q k ≥ Ψ k 1 + ( Q k − 1 ∥ 2 h b ) , b = b k − 1 .
Since the parallel sum ∥ \mathbin{\|} ∥ is increasing in its two arguments, and as Q k − 1 ≥ 1 2 Q_{k-1}\geq\frac12 Q k − 1 ≥ 2 1 , we get
Q k ≥ 1 Ψ k + ( 1 2 ∥ h b 2 ) = 1 Ψ k + h b 2 ( h b + 1 ) . Q_k\geq\frac{1}{\Psi_k}+\left(\frac12\mathbin{\|}\frac{h_b}{2}\right)=\frac{1}{\Psi_k}+\frac{h_b}{2(h_b+1)}. Q k ≥ Ψ k 1 + ( 2 1 ∥ 2 h b ) = Ψ k 1 + 2 ( h b + 1 ) h b .
The right side is at least 1 2 \frac12 2 1 exactly when Ψ k ≤ 2 ( h b + 1 ) \Psi_k\leq2(h_b+1) Ψ k ≤ 2 ( h b + 1 ) , and this is where the choice of profile ( e j ) j < N (e_j)_{j<N} ( e j ) j < N enters (through Theorem A.1 ). We have Ψ k = h a + h a − 1 \Psi_k=h_a+h_{a-1} Ψ k = h a + h a − 1 with a = a k ≥ 1 a=a_k\geq1 a = a k ≥ 1 , and Theorem A.1 (d) gives b ≥ a − 1 b\geq a-1 b ≥ a − 1 , so h b ≥ h a − 1 h_b\geq h_{a-1} h b ≥ h a − 1 . Theorem A.1 (e) then gives
Ψ k = h a + h a − 1 ≤ 2 h a − 1 + 1 ≤ 2 h b + 2. \Psi_k=h_a+h_{a-1}\leq2h_{a-1}+1\leq2h_b+2. Ψ k = h a + h a − 1 ≤ 2 h a − 1 + 1 ≤ 2 h b + 2.
Hence, we have Q k ≥ 1 2 Q_k\geq\frac12 Q k ≥ 2 1 for every k < N k<N k < N , that is, q k ≥ 1 / ( 2 N ) q_k\geq1/(2\sqrt{N}) q k ≥ 1/ ( 2 N ) . This concludes the proof of the lemma.
Using Theorem 2.5, we can now prove Theorem 2.5.
Proof of thm:span
Proof of Theorem 2.5. Let κ \kappa κ with 0 ≤ κ ≤ N − 1 0 \le\kappa\le N-1 0 ≤ κ ≤ N − 1 be fixed. Write g ˉ = g ˉ ( κ ) \bar{g}=\bar{g}^{(\kappa)} g ˉ = g ˉ ( κ ) and c ˉ = c ˉ κ \bar{c}=\bar{c}_{\kappa} c ˉ = c ˉ κ .
The equality. Let y = ∑ j ≥ κ λ j w ˉ j y=\sum_{j\geq\kappa}\lambda_j\bar{w}_j y = ∑ j ≥ κ λ j w ˉ j with ∑ j ≥ κ λ j = 1 \sum_{j\geq\kappa}\lambda_j=1 ∑ j ≥ κ λ j = 1 be a point of the affine hull aff ( w ˉ κ , … , w ˉ N − 1 ) \operatorname{aff}(\bar{w}_{\kappa},\ldots,\bar{w}_{N-1}) aff ( w ˉ κ , … , w ˉ N − 1 ) and let z ∈ span ( w ˉ 0 , … , w ˉ κ − 1 ) z\in\operatorname{span}(\bar{w}_0,\ldots,\bar{w}_{\kappa-1}) z ∈ span ( w ˉ 0 , … , w ˉ κ − 1 ) . By (3) , we have ⟨ g ˉ , y − z ⟩ = − c ˉ ∑ j ≥ κ λ j = − c ˉ \langle\bar{g},y-z\rangle=-\bar{c}\sum_{j\geq\kappa}\lambda_j=-\bar{c} ⟨ g ˉ , y − z ⟩ = − c ˉ ∑ j ≥ κ λ j = − c ˉ , and since ∥ g ˉ ∥ = 1 \|\bar{g}\|=1 ∥ g ˉ ∥ = 1 the Cauchy–Schwarz inequality gives ∥ y − z ∥ ≥ c ˉ \|y-z\|\geq\bar{c} ∥ y − z ∥ ≥ c ˉ . Conversely, since g ˉ ( κ ) = − c ˉ κ W ˉ A ˉ − 1 1 κ \bar{g}^{(\kappa)}=-\bar{c}_{\kappa}\bar{W}\bar{A}^{-1}\mathbf{1}_{\kappa} g ˉ ( κ ) = − c ˉ κ W ˉ A ˉ − 1 1 κ , writing g ˉ = ∑ j β j w ˉ j \bar{g}=\sum_j\beta_j\bar{w}_j g ˉ = ∑ j β j w ˉ j with β = − c ˉ A ˉ − 1 1 κ \beta=-\bar{c}\bar{A}^{-1}\mathbf{1}_{\kappa} β = − c ˉ A ˉ − 1 1 κ , and using (3) we get
1 = ⟨ g ˉ , g ˉ ⟩ = ∑ j β j ⟨ g ˉ , w ˉ j ⟩ = − c ˉ ∑ j ≥ κ β j . 1=\langle\bar{g},\bar{g}\rangle=\sum_j\beta_j\langle\bar{g},\bar{w}_j\rangle=-\bar{c}\sum_{j\geq\kappa}\beta_j. 1 = ⟨ g ˉ , g ˉ ⟩ = j ∑ β j ⟨ g ˉ , w ˉ j ⟩ = − c ˉ j ≥ κ ∑ β j .
So c ˉ = ∥ c ˉ g ˉ ∥ = ∥ y ∗ − z ∗ ∥ \bar{c}=\|\bar{c}\bar{g}\|=\|y^*-z^*\| c ˉ = ∥ c ˉ g ˉ ∥ = ∥ y ∗ − z ∗ ∥ where y ∗ = − c ˉ ∑ j ≥ κ β j w ˉ j ∈ aff ( w ˉ κ , … , w ˉ N − 1 ) y^*=-\bar{c}\sum_{j\geq\kappa}\beta_j\bar{w}_j\in\operatorname{aff}(\bar{w}_{\kappa},\ldots,\bar{w}_{N-1}) y ∗ = − c ˉ ∑ j ≥ κ β j w ˉ j ∈ aff ( w ˉ κ , … , w ˉ N − 1 ) since its weights ( − c ˉ β j ) j ≥ κ (-\bar{c}\beta_j)_{j\geq\kappa} ( − c ˉ β j ) j ≥ κ sum to 1, and z ∗ = c ˉ ∑ j < κ β j w ˉ j ∈ span ( w ˉ 0 , … , w ˉ κ − 1 ) z^*=\bar{c}\sum_{j<\kappa}\beta_j\bar{w}_j\in\operatorname{span}(\bar{w}_0,\ldots,\bar{w}_{\kappa-1}) z ∗ = c ˉ ∑ j < κ β j w ˉ j ∈ span ( w ˉ 0 , … , w ˉ κ − 1 ) . Hence we get the equality in the lemma.
The first inequality. Every tail atom w ˉ j \bar{w}_j w ˉ j with j ≥ κ j\geq\kappa j ≥ κ is w ˉ j = M κ + a j \bar{w}_j=M_{\kappa}+a_j w ˉ j = M κ + a j where a j = f j ν j + ∑ κ < i ≤ j δ i u i a_j=\sqrt{f_j}\nu_j+\sum_{\kappa<i\leq j}\sqrt{\delta_i}u_i a j = f j ν j + ∑ κ < i ≤ j δ i u i . So any y = ∑ j ≥ κ λ j w ˉ j y=\sum_{j\geq\kappa}\lambda_j\bar{w}_j y = ∑ j ≥ κ λ j w ˉ j with ∑ j ≥ κ λ j = 1 \sum_{j\geq\kappa}\lambda_j=1 ∑ j ≥ κ λ j = 1 in the affine hull aff ( w ˉ κ , … , w ˉ N − 1 ) \operatorname{aff}(\bar{w}_{\kappa},\ldots,\bar{w}_{N-1}) aff ( w ˉ κ , … , w ˉ N − 1 ) can be written as M κ + a M_{\kappa}+a M κ + a with a = ∑ j ≥ κ λ j a j a=\sum_{j\geq\kappa}\lambda_ja_j a = ∑ j ≥ κ λ j a j . The vector a a a is orthogonal to E κ = span ( u 0 , … , u κ , ν 0 , … , ν κ − 1 ) E_{\kappa}=\operatorname{span}(u_0,\ldots,u_{\kappa},\nu_0,\ldots,\nu_{\kappa-1}) E κ = span ( u 0 , … , u κ , ν 0 , … , ν κ − 1 ) , which contains M κ M_{\kappa} M κ and the head atoms. Thus for z ∈ span ( w ˉ 0 , … , w ˉ κ − 1 ) z\in\operatorname{span}(\bar{w}_0,\ldots,\bar{w}_{\kappa-1}) z ∈ span ( w ˉ 0 , … , w ˉ κ − 1 ) , we have ∥ y − z ∥ 2 = ∥ M κ − z ∥ 2 + ∥ a ∥ 2 ≥ ∥ M κ − z ∥ 2 \|y-z\|^2=\|M_{\kappa}-z\|^2+\|a\|^2\geq\|M_{\kappa}-z\|^2 ∥ y − z ∥ 2 = ∥ M κ − z ∥ 2 + ∥ a ∥ 2 ≥ ∥ M κ − z ∥ 2 . Taking the infimum over y y y and z z z gives c ˉ κ ≥ dist ( M κ , span ( w ˉ 0 , … , w ˉ κ − 1 ) ) \bar{c}_{\kappa}\geq\operatorname{dist}(M_{\kappa},\operatorname{span}(\bar{w}_0,\ldots,\bar{w}_{\kappa-1})) c ˉ κ ≥ dist ( M κ , span ( w ˉ 0 , … , w ˉ κ − 1 )) .
The second inequality. This is Theorem A.2 .
Proof of Theorem 2.2
Proof of Theorem 2.2. Consider the instance of Section 2.1. It has the stated form: ∥ g t ∥ = ∥ g ˉ ( κ t ) ∥ = 1 \|g_t\|=\|\bar{g}^{(\kappa_t)}\|=1 ∥ g t ∥ = ∥ g ˉ ( κ t ) ∥ = 1 for all t t t , K ˉ ⊂ R 2 N \bar{K}\subset\mathbb{R}^{2N} K ˉ ⊂ R 2 N , and diam K ˉ ≤ 1 \operatorname{diam}\bar{K}\leq1 diam K ˉ ≤ 1 by Theorem A.1 (f), since by the martingale identity two atoms are at squared distance ∥ w ˉ j − w ˉ i ∥ 2 = e j − e i + f i + f j ≤ 1 \|\bar{w}_j-\bar{w}_i\|^2=e_j-e_i+f_i+f_j\leq1 ∥ w ˉ j − w ˉ i ∥ 2 = e j − e i + f i + f j ≤ 1 for i < j i<j i < j , and an atom w ˉ j \bar{w}_j w ˉ j is at distance e j + f j ≤ 1 \sqrt{e_j+f_j}\leq1 e j + f j ≤ 1 from the origin x 0 x_0 x 0 .
By Theorem 2.4, every span method making at most m m m oracle calls per round has regret Reg T = ∑ t < T c ˉ κ t \operatorname{Reg}_T=\sum_{t<T}\bar{c}_{\kappa_t} Reg T = ∑ t < T c ˉ κ t , and by Theorem 2.5 each c ˉ κ t \bar{c}_{\kappa_t} c ˉ κ t is at least ( 4 N ) − 1 / 4 (4N)^{-1/4} ( 4 N ) − 1/4 . Hence we get Reg T ≥ T ( 4 N ) − 1 / 4 ≥ T 3 / 4 ( 4 m ) − 1 / 4 \operatorname{Reg}_T\geq T(4N)^{-1/4}\geq T^{3/4}(4m)^{-1/4} Reg T ≥ T ( 4 N ) − 1/4 ≥ T 3/4 ( 4 m ) − 1/4 , where the last inequality comes from N = m ( T − 1 ) + 1 ≤ m T N=m(T-1)+1\leq mT N = m ( T − 1 ) + 1 ≤ m T . This proves the theorem.
Finally, for L , D > 0 L,D>0 L , D > 0 , the scaled instance ( D K ˉ , ( L g t ) t < T ) (D\bar{K},(Lg_t)_{t<T}) ( D K ˉ , ( L g t ) t < T ) has diameter at most D D D and loss vectors of norm L L L . A span method run on it is, after the changes of variables x ↦ x / D x\mapsto x/D x ↦ x / D and d ↦ d / L d\mapsto d/L d ↦ d / L , a span method run on the original instance with the same oracle answers up to the factor D D D , so its regret is L D LD L D times the original one. This proves the statement after Theorem 2.2 for scaled instances.
Proof of Theorem 3.1: every learner
This section contains the complete proofs of all the results in Section 3, and in particular Theorem 3.1 and Theorem 3.2. We start by proving Theorem 3.2, which is a direct corollary of Theorem 3.1 by choosing a learner-dependent rotation of the instance’s random frame.
Proof of Theorem 3.2. Fix a learner making at most m m m oracle calls per round. In Theorem 3.1 the expectation is over the random frame and over the learner’s randomness ξ \xi ξ , so the average over the frame of E ξ [ Reg T ] \mathbb{E}_{\xi}[\operatorname{Reg}_T] E ξ [ Reg T ] is at least 0.23 T N − 1 / 4 0.23TN^{-1/4} 0.23 T N − 1/4 (recall the instance’s randomness is independent of the learner’s randomness ξ \xi ξ ). Hence, some realization of the frame, that is, some fixed instance depending on the learner (but not on ξ \xi ξ ), satisfies E ξ [ Reg T ] ≥ 0.23 T N − 1 / 4 \mathbb{E}_{\xi}[\operatorname{Reg}_T]\geq0.23TN^{-1/4} E ξ [ Reg T ] ≥ 0.23 T N − 1/4 . Scaling this instance by D / ( 5 / 2 ) D/(\sqrt{5}/2) D / ( 5 /2 ) in space and by L L L in the losses gives diam K ≤ D \operatorname{diam}K\leq D diam K ≤ D and ∥ g t ∥ = L \|g_t\|=L ∥ g t ∥ = L , and multiplies the regret by 2 L D / 5 2LD/\sqrt{5} 2 L D / 5 . Since 0.23 ⋅ 2 / 5 > 0.2 0.23\cdot2/\sqrt{5}>0.2 0.23 ⋅ 2/ 5 > 0.2 , the claim follows.
The rest of this section is devoted to the proof of Theorem 3.1. As discussed directly after its statement, it suffices to treat deterministic learners: the instance distribution does not depend on the learner nor on its internal randomness ξ \xi ξ , so conditionally on ξ \xi ξ a randomized learner behaves as a deterministic one, and both bounds of the theorem, once proved for every fixed value of ξ \xi ξ , therefore hold unconditionally (Yao’s principle). From now on the learner is deterministic.
Throughout this section, the instance we consider is the hidden instance of § 3.1 : the random frame ( u 0 , ν 0 , … , u N − 1 , ν N − 1 ) (u_0,\nu_0,\ldots,u_{N-1},\nu_{N-1}) ( u 0 , ν 0 , … , u N − 1 , ν N − 1 ) in R n \mathbb{R}^n R n , the shrunk atoms w j = ρ j M j + f j ν j w_j=\rho_jM_j+\sqrt{f_j}\nu_j w j = ρ j M j + f j ν j with ρ j = 1 − j / ( 2 ( N − 1 ) ) \rho_j=1-j/(2(N-1)) ρ j = 1 − j / ( 2 ( N − 1 )) and γ = ρ j − ρ j + 1 = 1 / ( 2 ( N − 1 ) ) \gamma=\rho_j-\rho_{j+1}=1/(2(N-1)) γ = ρ j − ρ j + 1 = 1/ ( 2 ( N − 1 )) , the body K = conv ( { 0 } ∪ { w j } ∪ B r ) K=\operatorname{conv}(\{0\}\cup\{w_j\}\cup B_r) K = conv ({ 0 } ∪ { w j } ∪ B r ) with r = 1 40 N − 1 / 4 r=\frac{1}{40}N^{-1/4} r = 40 1 N − 1/4 , the oracle returning the first minimizer in the list 0 , w 0 , … , w N − 1 , − r d / ∥ d ∥ 0,w_0,\ldots,w_{N-1},-r d/\lVert d\rVert 0 , w 0 , … , w N − 1 , − r d / ∥ d ∥ , and the vectors g ( κ ) = − ( M κ − Π κ M κ ) / c κ g^{(\kappa)}=-(M_\kappa-\Pi_\kappa M_\kappa)/c_\kappa g ( κ ) = − ( M κ − Π κ M κ ) / c κ and the losses g t = g ( κ t ) g_t=g^{(\kappa_t)} g t = g ( κ t ) , with κ t = m t \kappa_t=mt κ t = m t . The proof has five steps, one per subsection: build the losses and the levels (Section B.1); prove four deterministic facts about the geometry of K K K (Section B.2); introduce a clean oracle that ignores the hidden directions, and show that it agrees with the true oracle as long as the learner’s messages are nearly orthogonal to them (Section B.3); show that the messages are indeed nearly orthogonal to the hidden directions, with high probability (Section B.4); and put the pieces together (Section B.5).
The losses and the process levels
We first prove the loss vectors g ( κ ) g^{(\kappa)} g ( κ ) have a similar pattern as those of the span-method instance from § 2.1 .
Lemma B.1 (Structure of the losses). Let 0 ≤ κ ≤ N − 1 0\leq\kappa\leq N-1 0 ≤ κ ≤ N − 1 , and let E κ = span ( u 0 , … , u κ , ν 0 , … , ν κ − 1 ) E_\kappa=\operatorname{span}(u_0,\ldots,u_\kappa,\nu_0,\ldots,\nu_{\kappa-1}) E κ = span ( u 0 , … , u κ , ν 0 , … , ν κ − 1 ) . The vector g ( κ ) g^{(\kappa)} g ( κ ) is a unit vector of E κ E_\kappa E κ , and we have
⟨ g ( κ ) , w j ⟩ = { 0 , j < κ , − ρ j c κ , j ≥ κ . \langle g^{(\kappa)},w_j\rangle=
\begin{cases}
0, & j<\kappa,\\
-\rho_jc_\kappa, & j\geq\kappa.
\end{cases} ⟨ g ( κ ) , w j ⟩ = { 0 , − ρ j c κ , j < κ , j ≥ κ . In particular, we have ⟨ g ( κ ) , b ⟩ ≥ − r \langle g^{(\kappa)},b\rangle\geq-r ⟨ g ( κ ) , b ⟩ ≥ − r for every b ∈ B r b\in B_r b ∈ B r .
This means the loss g ( κ ) g^{(\kappa)} g ( κ ) cannot be exploited with atoms before κ \kappa κ , it rewards all atoms from κ \kappa κ on by c κ c_\kappa c κ times their shrinkage factor ρ j \rho_j ρ j , hence by an amount between c κ / 2 c_\kappa/2 c κ /2 and c κ c_\kappa c κ , and it rewards the comparator by exactly c κ / 2 c_\kappa/2 c κ /2 , since ρ N − 1 = 1 2 \rho_{N-1}=\frac{1}{2} ρ N − 1 = 2 1 . As discussed in § 3.1 , the loss vector g ( κ ) g^{(\kappa)} g ( κ ) is in E κ E_\kappa E κ whereas the loss vectors were in the span of all the atoms in the span-method instance of § 2.1 , which would reveal the frame.
Proof. Step 1: The earlier atoms. The walk position M κ M_\kappa M κ and the atoms w s w_s w s with s < κ s<\kappa s < κ are built from the vectors u 0 , … , u κ u_0,\ldots,u_\kappa u 0 , … , u κ and ν 0 , … , ν κ − 1 \nu_0,\ldots,\nu_{\kappa-1} ν 0 , … , ν κ − 1 only, so they lie in E κ E_\kappa E κ ; hence so does M κ − Π κ M κ M_\kappa-\Pi_\kappa M_\kappa M κ − Π κ M κ (recall Π κ \Pi_\kappa Π κ is the orthogonal projection onto span ( w 0 , … , w κ − 1 ) \operatorname{span}(w_0,\ldots,w_{\kappa-1}) span ( w 0 , … , w κ − 1 ) ). The vector M κ − Π κ M κ M_\kappa-\Pi_\kappa M_\kappa M κ − Π κ M κ is orthogonal to every w s w_s w s with s < κ s<\kappa s < κ by the definition of an orthogonal projection. This proves the first case since g ( κ ) ∝ M κ − Π κ M κ g^{(\kappa)}\propto M_\kappa-\Pi_\kappa M_\kappa g ( κ ) ∝ M κ − Π κ M κ , and g ( κ ) g^{(\kappa)} g ( κ ) has norm one by construction.
Step 2: The later atoms. Let j ≥ κ j\geq\kappa j ≥ κ . The atom w j w_j w j splits into a part in E κ E_\kappa E κ and a part built from frame vectors outside E κ E_\kappa E κ :
w j = ρ j M κ + ( ρ j ∑ κ < i ≤ j δ i u i + f j ν j ) . w_j=\rho_jM_\kappa+\left(\rho_j\sum_{\kappa<i\leq j}\sqrt{\delta_i}u_i+\sqrt{f_j}\nu_j\right). w j = ρ j M κ + ( ρ j κ < i ≤ j ∑ δ i u i + f j ν j ) .
Since g ( κ ) g^{(\kappa)} g ( κ ) lies in E κ E_\kappa E κ , only the first part contributes:
⟨ g ( κ ) , w j ⟩ = ρ j ⟨ g ( κ ) , M κ ⟩ = − ρ j c κ ⟨ M κ − Π κ M κ , M κ ⟩ = − ρ j c κ . \langle g^{(\kappa)},w_j\rangle=\rho_j\langle g^{(\kappa)},M_\kappa\rangle=-\frac{\rho_j}{c_\kappa}\langle M_\kappa-\Pi_\kappa M_\kappa,M_\kappa\rangle=-\rho_jc_\kappa. ⟨ g ( κ ) , w j ⟩ = ρ j ⟨ g ( κ ) , M κ ⟩ = − c κ ρ j ⟨ M κ − Π κ M κ , M κ ⟩ = − ρ j c κ .
The last equality holds because M κ − Π κ M κ M_\kappa-\Pi_\kappa M_\kappa M κ − Π κ M κ is orthogonal to Π κ M κ \Pi_\kappa M_\kappa Π κ M κ , so its inner product with M κ M_\kappa M κ is its squared norm, whose value is c κ 2 c_\kappa^2 c κ 2 .
Step 3: The ball. This is the Cauchy–Schwarz inequality, since g ( κ ) g^{(\kappa)} g ( κ ) has norm one and b b b has norm at most r r r . □ \square □
In the proof we keep track of how much of the frame the learner could know. We say that the process is at level L L L , for 0 ≤ L ≤ N − 1 0\leq L\leq N-1 0 ≤ L ≤ N − 1 , when the first 2 L + 1 2L+1 2 L + 1 vectors of the frame are revealed:
u 0 , ν 0 , u 1 , ν 1 , … , u L − 1 , ν L − 1 , u L . u_0,\ \nu_0,\ u_1,\ \nu_1,\ \ldots,\ u_{L-1},\ \nu_{L-1},\ u_L. u 0 , ν 0 , u 1 , ν 1 , … , u L − 1 , ν L − 1 , u L .
At level L L L the atoms w 0 , … , w L − 1 w_0,\ldots,w_{L-1} w 0 , … , w L − 1 and the walk position M L M_L M L are functions of the revealed vectors. We call these atoms revealed and the other atoms unrevealed . Let U L U_L U L be the span of the unrevealed frame vectors,
U L = span ( ν L , u L + 1 , ν L + 1 , … , u N − 1 , ν N − 1 ) , U_L=\operatorname{span}(\nu_L,u_{L+1},\nu_{L+1},\ldots,u_{N-1},\nu_{N-1}), U L = span ( ν L , u L + 1 , ν L + 1 , … , u N − 1 , ν N − 1 ) ,
and let Π U L \Pi_{U_L} Π U L be the orthogonal projection onto it. Every unrevealed atom, j ≥ L j \ge L j ≥ L , splits into a known part and a hidden part:
w j = ρ j M L + a j ( L ) , a j ( L ) = ρ j ∑ L < i ≤ j δ i u i + f j ν j ∈ U L . (11) w_j = \rho_j M_L + a_j^{(L)}, \qquad a_j^{(L)} = \rho_j \sum_{L<i\le j} \sqrt{\delta_i}\,u_i + \sqrt{f_j}\,\nu_j \in U_L. \tag*{(11)} w j = ρ j M L + a j ( L ) , a j ( L ) = ρ j L < i ≤ j ∑ δ i u i + f j ν j ∈ U L . (11)
The known parts of all unrevealed atoms are multiples of the same vector M L M_L M L , and they differ only through the shrinkage factors. This is the precise sense in which unrevealed atoms look alike to the learner. The hidden part a j ( L ) a_j^{(L)} a j ( L ) is the component of w j w_j w j orthogonal to the revealed vectors, so its norm is at most ∥ w j ∥ ≤ 1 \lVert w_j\rVert\le1 ∥ w j ∥ ≤ 1 (Theorem B.2). Finally, the loss vector g ( κ ) g^{(\kappa)} g ( κ ) is a function of the vectors revealed at level κ \kappa κ .
Four deterministic lemmas
This subsection proves four facts about the geometry of K K K . The body is small (Theorem B.2). The per-round costs c κ c_\kappa c κ are not too small (Theorem B.3). The hidden parts of unrevealed atoms cannot cancel each other (Theorem B.4). As a consequence, a point of K K K with a small hidden component earns little (Theorem B.5). No randomness is involved.
Lemma B.2 (Body). Every point of K K K has norm at most 1, and the diameter of K K K is at most 5 / 2 < 1.12 \sqrt{5}/2 < 1.12 5 /2 < 1.12 .
Proof. Step 1: Norms for all points of K K K . The frame is orthonormal, so the squared norm of an atom is the sum of its squared coordinates:
∥ w j ∥ 2 = ρ j 2 e j + f j ≤ e j + f j ≤ 1 \lVert w_j\rVert^2 = \rho_j^2 e_j + f_j \le e_j + f_j \le1 ∥ w j ∥ 2 = ρ j 2 e j + f j ≤ e j + f j ≤ 1
by Theorem A.1 (f). The ball has radius r < 1 r < 1 r < 1 and the origin has norm 0. A convex combination of points of norm at most one has norm at most one. Hence, every point of K K K has norm at most one.
Step 2: Distance between two atoms. Let i < j i < j i < j . The two atoms w i , w j w_i,w_j w i , w j share the steps 0 , … , i 0,\ldots,i 0 , … , i of the walk (with different shrinkage factors) and differ by the steps i + 1 , … , j i+1,\ldots,j i + 1 , … , j and by their private noises. Reading off coordinates, we get
∥ w i − w j ∥ 2 = ρ j 2 ( e j − e i ) + ( ρ i − ρ j ) 2 e i + f i + f j . \lVert w_i-w_j\rVert^2 = \rho_j^2(e_j-e_i) + (\rho_i-\rho_j)^2 e_i + f_i + f_j. ∥ w i − w j ∥ 2 = ρ j 2 ( e j − e i ) + ( ρ i − ρ j ) 2 e i + f i + f j .
The first and last terms add up to at most e j − e i + f i + f j e_j-e_i+f_i+f_j e j − e i + f i + f j , which is at most 1 by Theorem A.1 (f). The middle term is at most 1 / 4 1/4 1/4 , because the shrinkage factors lie in [ 1 / 2 , 1 ] [1/2,1] [ 1/2 , 1 ] . So the distance is at most 5 / 2 \sqrt{5}/2 5 /2 .
Step 3: Distances for the other pairs. An atom is at distance at most 1 + r < 5 / 2 1+r < \sqrt{5}/2 1 + r < 5 /2 from the origin and from every point of the ball, and two points of the ball are at distance at most 2 r 2r 2 r . The diameter of a convex hull is the largest distance between two of its generating points. Hence, we get diam ( K ) ≤ 5 / 2 < 1.12 \operatorname{diam}(K) \le\sqrt{5}/2 < 1.12 diam ( K ) ≤ 5 /2 < 1.12 . □
The next lemma lower bounds the per-round costs c κ c_\kappa c κ . Its proof reduces to the recursion of Theorem A.2 .
Lemma B.3 (Lower bound on per-round costs). For every 0 ≤ κ ≤ N − 1 0 \le\kappa\le N-1 0 ≤ κ ≤ N − 1 , we have c κ ≥ ( 4 N ) − 1 / 4 c_\kappa\ge(4N)^{-1/4} c κ ≥ ( 4 N ) − 1/4 .
This lemma says that each loss vector rewards the unrevealed atoms by an amount of order N − 1 / 4 N^{-1/4} N − 1/4 ; it is where the rate T ⋅ N − 1 / 4 T\cdot N^{-1/4} T ⋅ N − 1/4 comes from.
Proof. By definition c κ 2 = ∥ M κ − Π κ M κ ∥ 2 = min y 0 , … , y κ − 1 ∥ M κ − ∑ s < κ y s w s ∥ 2 c_\kappa^2 = \lVert M_\kappa-\Pi_\kappa M_\kappa\rVert^2 = \min_{y_0,\ldots,y_{\kappa-1}}\lVert M_\kappa-\sum_{s<\kappa}y_s w_s\rVert^2 c κ 2 = ∥ M κ − Π κ M κ ∥ 2 = min y 0 , … , y κ − 1 ∥ M κ − ∑ s < κ y s w s ∥ 2 (recall Π κ \Pi_\kappa Π κ is the orthogonal projection onto span ( { w 0 , … , w κ − 1 } ) \operatorname{span}(\{w_0,\ldots,w_{\kappa-1}\}) span ({ w 0 , … , w κ − 1 }) ). The computation of Step 1 in the proof of Theorem A.2 , with the shrunk atoms w s = ρ s M s + f s ν s w_s=\rho_sM_s+\sqrt{f_s}\nu_s w s = ρ s M s + f s ν s and the cumulative quantities Z i = 1 − ∑ s ≤ i − 1 ρ s y s Z_i=1-\sum_{s\le i-1}\rho_s y_s Z i = 1 − ∑ s ≤ i − 1 ρ s y s , gives the coordinate δ i Z i \sqrt{\delta_i}Z_i δ i Z i along u i u_i u i and − y s f s = ( f s / ρ s ) ( Z s − Z s + 1 ) -y_s\sqrt{f_s}= (\sqrt{f_s}/\rho_s)(Z_s-Z_{s+1}) − y s f s = ( f s / ρ s ) ( Z s − Z s + 1 ) along ν s \nu_s ν s , hence
c κ 2 = min Z 0 , … , Z κ − 1 [ ∑ i = 0 κ δ i Z i 2 + ∑ s = 0 κ − 1 f s ρ s 2 ( Z s − Z s + 1 ) 2 ] with Z κ = 1. c_\kappa^2 = \min_{Z_0,\ldots,Z_{\kappa-1}}\left[\sum_{i=0}^{\kappa}\delta_i Z_i^2+\sum_{s=0}^{\kappa-1}\frac{f_s}{\rho_s^2}(Z_s-Z_{s+1})^2\right]\quad\text{with }Z_\kappa=1. c κ 2 = Z 0 , … , Z κ − 1 min [ i = 0 ∑ κ δ i Z i 2 + s = 0 ∑ κ − 1 ρ s 2 f s ( Z s − Z s + 1 ) 2 ] with Z κ = 1.
Since ρ s ≤ 1 \rho_s\le1 ρ s ≤ 1 , the coefficients f s / ρ s 2 f_s/\rho_s^2 f s / ρ s 2 are at least f s f_s f s for all s s s , so for every Z 0 , … , Z κ − 1 Z_0,\ldots,Z_{\kappa-1} Z 0 , … , Z κ − 1 the bracket is at least the one of Theorem A.2 , and we get c κ 2 ≥ q κ ≥ 1 / ( 2 N ) c_\kappa^2\ge q_\kappa\ge1/(2\sqrt{N}) c κ 2 ≥ q κ ≥ 1/ ( 2 N ) , that is c κ ≥ ( 4 N ) − 1 / 4 c_\kappa\ge(4N)^{-1/4} c κ ≥ ( 4 N ) − 1/4 . □
A learner might try to use unrevealed atoms without knowing their hidden parts, by mixing them with a ball point that cancels those parts. The next lemma rules this out: a nonnegative combination of hidden parts of unrevealed atoms has norm proportional to its total weight. Since the ball has radius r r r , the total weight on unrevealed atoms in such a mixture is at most about 4 N 1 / 4 r = 1 10 4N^{1/4}r = \frac{1}{10} 4 N 1/4 r = 10 1 (Theorem B.5).
Lemma B.4 (Atom cancellation). For every level L L L and all nonnegative weights λ L , … , λ N − 1 \lambda_L,\ldots,\lambda_{N-1} λ L , … , λ N − 1 with sum Λ \Lambda Λ ,
∥ ∑ j ≥ L λ j a j ( L ) ∥ ≥ Λ 4 N 1 / 4 . \left\|\sum_{j \ge L}\lambda_j a_j^{(L)}\right\| \ge\frac{\Lambda}{4N^{1/4}}. j ≥ L ∑ λ j a j ( L ) ≥ 4 N 1/4 Λ .
Proof. The proof runs the recursion of Theorem A.2 from the tail of the chain, with the same parallel sum ((10) ).
Step 1: Lower bound by a quadratic problem r L r_L r L . Let Y i = ∑ j ≥ i λ j Y_i=\sum_{j\ge i}\lambda_j Y i = ∑ j ≥ i λ j be the weight on the atoms i , … , N − 1 i,\ldots,N-1 i , … , N − 1 . Then Y L = Λ Y_L=\Lambda Y L = Λ , Y N = 0 Y_N=0 Y N = 0 and λ j = Y j − Y j + 1 \lambda_j=Y_j-Y_{j+1} λ j = Y j − Y j + 1 . By ((11) ), the coordinate of ∑ j λ j a j ( L ) \sum_j\lambda_j a_j^{(L)} ∑ j λ j a j ( L ) along u i u_i u i (for i > L i>L i > L ) is δ i ∑ j ≥ i ρ j λ j \sqrt{\delta_i}\sum_{j\ge i}\rho_j\lambda_j δ i ∑ j ≥ i ρ j λ j , and its coordinate along v j v_j v j is f j λ j \sqrt{f_j}\lambda_j f j λ j . The weights λ L , … , λ N − 1 \lambda_L,\ldots,\lambda_{N-1} λ L , … , λ N − 1 are nonnegative and ρ j ≥ 1 2 \rho_j\ge\frac{1}{2} ρ j ≥ 2 1 , so the coordinate along u i u_i u i is at least δ i Y i / 2 \sqrt{\delta_i}Y_i/2 δ i Y i /2 . Hence, we have
∥ ∑ j λ j a j ( L ) ∥ 2 ≥ 1 4 [ ∑ i = L + 1 N − 1 δ i Y i 2 + ∑ j = L N − 1 f j ( Y j − Y j + 1 ) 2 ] (12) \left\|\sum_j\lambda_j a_j^{(L)}\right\|^2 \ge\frac{1}{4}\left[\sum_{i=L+1}^{N-1}\delta_iY_i^2+\sum_{j=L}^{N-1}f_j(Y_j-Y_{j+1})^2\right] \tag*{(12)} j ∑ λ j a j ( L ) 2 ≥ 4 1 i = L + 1 ∑ N − 1 δ i Y i 2 + j = L ∑ N − 1 f j ( Y j − Y j + 1 ) 2 (12)
(putting the factor 1 4 \frac{1}{4} 4 1 on the coordinates v j v_j v j , L ≤ j ≤ N − 1 L\le j\le N-1 L ≤ j ≤ N − 1 , as well only weakens the bound). The bracket is a quadratic function of Y L , … , Y N − 1 Y_L,\ldots,Y_{N-1} Y L , … , Y N − 1 . Let r L r_L r L be its minimum when Y L = 1 Y_L=1 Y L = 1 and Y N = 0 Y_N=0 Y N = 0 . That is, for every L < N L<N L < N ,
r L : = min Y L + 1 , … , Y N − 1 [ ∑ i = L + 1 N − 1 δ i Y i 2 + ∑ j = L N − 1 f j ( Y j − Y j + 1 ) 2 ] with Y L = 1 and Y N = 0. r_L:=\min_{Y_{L+1},\ldots,Y_{N-1}}\left[\sum_{i=L+1}^{N-1}\delta_iY_i^2+\sum_{j=L}^{N-1}f_j(Y_j-Y_{j+1})^2\right]\quad\text{with }Y_L=1\text{ and }Y_N=0. r L := Y L + 1 , … , Y N − 1 min i = L + 1 ∑ N − 1 δ i Y i 2 + j = L ∑ N − 1 f j ( Y j − Y j + 1 ) 2 with Y L = 1 and Y N = 0.
By homogeneity the bracket in ((12) ) is at least Λ 2 r L \Lambda^2r_L Λ 2 r L . Thus, we are left to prove that r L ≥ 1 / ( 4 N ) r_L\ge1/(4\sqrt{N}) r L ≥ 1/ ( 4 N ) .
Step 2: The sequence ( r L ) L < N (r_L)_{L<N} ( r L ) L < N satisfies a recursion formula. For L = N − 1 L=N-1 L = N − 1 there is no free variable and r N − 1 = f N − 1 r_{N-1}=f_{N-1} r N − 1 = f N − 1 . For L < N − 1 L<N-1 L < N − 1 , the problem defining r L r_L r L consists of the problem for r L + 1 r_{L+1} r L + 1 , with Y L + 1 Y_{L+1} Y L + 1 now free, plus the two terms δ L + 1 Y L + 1 2 \delta_{L+1}Y_{L+1}^2 δ L + 1 Y L + 1 2 and f L ( 1 − Y L + 1 ) 2 f_L(1-Y_{L+1})^2 f L ( 1 − Y L + 1 ) 2 . If Y L + 1 Y_{L+1} Y L + 1 is fixed to a value t t t instead of 1 1 1 in the definition of r L + 1 r_{L+1} r L + 1 , the minimum becomes r L + 1 t 2 r_{L+1}t^2 r L + 1 t 2 by homogeneity. Thus, minimizing first over Y L + 2 , … , Y N − 1 Y_{L+2},\ldots,Y_{N-1} Y L + 2 , … , Y N − 1 and then over Y L + 1 Y_{L+1} Y L + 1 in the definition of r L r_L r L gives
r L = min Y L + 1 [ f L ( 1 − Y L + 1 ) 2 + δ L + 1 Y L + 1 2 + min Y L + 2 , … , Y N − 1 [ ∑ i = L + 2 N − 1 δ i Y i 2 + ∑ j = L + 1 N − 1 f j ( Y j − Y j + 1 ) 2 ] ] = min Y L + 1 [ f L ( 1 − Y L + 1 ) 2 + ( δ L + 1 + r L + 1 ) Y L + 1 2 ] , \begin{aligned}
r_L=\min_{Y_{L+1}}\left[f_L(1-Y_{L+1})^2+\delta_{L+1}Y_{L+1}^2+\min_{Y_{L+2},\ldots,Y_{N-1}}\left[\sum_{i=L+2}^{N-1}\delta_iY_i^2+\sum_{j=L+1}^{N-1}f_j(Y_j-Y_{j+1})^2\right]\right] \\
&=\min_{Y_{L+1}}\left[f_L(1-Y_{L+1})^2+(\delta_{L+1}+r_{L+1})Y_{L+1}^2\right],
\end{aligned} r L = Y L + 1 min f L ( 1 − Y L + 1 ) 2 + δ L + 1 Y L + 1 2 + Y L + 2 , … , Y N − 1 min i = L + 2 ∑ N − 1 δ i Y i 2 + j = L + 1 ∑ N − 1 f j ( Y j − Y j + 1 ) 2 = Y L + 1 min [ f L ( 1 − Y L + 1 ) 2 + ( δ L + 1 + r L + 1 ) Y L + 1 2 ] ,
and using ((10) ) gives the recursion
r N − 1 = f N − 1 , r L = f L ∥ ( δ L + 1 + r L + 1 ) ( L < N − 1 ) . r_{N-1}=f_{N-1},\qquad r_L=f_L\mathbin{\|}(\delta_{L+1}+r_{L+1})\quad(L<N-1). r N − 1 = f N − 1 , r L = f L ∥ ( δ L + 1 + r L + 1 ) ( L < N − 1 ) .
Step 3: The recursion stays above 1 / ( 4 N ) 1/(4\sqrt{N}) 1/ ( 4 N ) . Let R L = N r L R_L=\sqrt{N}r_L R L = N r L . We prove by descending induction that R L ≥ 1 / 4 R_L\ge1/4 R L ≥ 1/4 for all L < N L<N L < N . At L = N − 1 L=N-1 L = N − 1 , we have R N − 1 = N f N − 1 = h 0 / 2 = 1 / 2 R_{N-1}=\sqrt{N}f_{N-1}=h_0/2=1/2 R N − 1 = N f N − 1 = h 0 /2 = 1/2 by Theorem A.1 (b), since b N − 1 = 0 b_{N-1}=0 b N − 1 = 0 . Suppose R L + 1 ≥ 1 / 4 R_{L+1}\ge1/4 R L + 1 ≥ 1/4 , and let a = a L + 1 ≥ 1 a=a_{L+1}\ge1 a = a L + 1 ≥ 1 and b = b L b=b_L b = b L . By Theorem A.1 (b,c) and the monotonicity of the parallel sum,
R L ≥ h b 2 ∥ ( 1 ψ a + 1 4 ) . R_L\ge\frac{h_b}{2}\mathbin{\|}\left(\frac{1}{\psi_a}+\frac{1}{4}\right). R L ≥ 2 h b ∥ ( ψ a 1 + 4 1 ) .
Using 1 / ( x ∥ y ) = 1 / x + 1 / y 1/(x\mathbin{\|}y)=1/x+1/y 1/ ( x ∥ y ) = 1/ x + 1/ y , the right side is at least 1 4 \frac{1}{4} 4 1 exactly when
2 h b + 4 ψ a 4 + ψ a ≤ 4 , that is, 4 + ψ a ≤ 8 h b . \frac{2}{h_b}+\frac{4\psi_a}{4+\psi_a}\le4,\qquad\text{that is,}\qquad4+\psi_a\le8h_b. h b 2 + 4 + ψ a 4 ψ a ≤ 4 , that is, 4 + ψ a ≤ 8 h b .
By Theorem A.1 (d,e) we have h b ≥ h a − 1 ≥ 1 h_b \ge h_{a-1} \ge1 h b ≥ h a − 1 ≥ 1 and ψ a = h a + h a − 1 ≤ 2 h a − 1 + 1 \psi_a = h_a + h_{a-1} \le2h_{a-1} + 1 ψ a = h a + h a − 1 ≤ 2 h a − 1 + 1 . Thus we have
4 + ψ a ≤ 5 + 2 h a − 1 ≤ 8 h a − 1 ≤ 8 h b , 4 + \psi_a \le5 + 2h_{a-1} \le8h_{a-1} \le8h_b, 4 + ψ a ≤ 5 + 2 h a − 1 ≤ 8 h a − 1 ≤ 8 h b ,
and R L ≥ 1 4 R_L \ge\frac{1}{4} R L ≥ 4 1 . Hence, we get r L ≥ 1 / ( 4 N ) r_L \ge1/(4\sqrt{N}) r L ≥ 1/ ( 4 N ) for all L < N L < N L < N .
Step 4: Conclusion. Steps 1 to 3 give that the squared norm is at least Λ 2 / ( 16 N ) \Lambda^2/(16\sqrt{N}) Λ 2 / ( 16 N ) , which is the claim.
Apart from the ball, only unrevealed atoms earn, that is, have a negative loss (Theorem B.1), and putting weight on those atoms requires a hidden component (Theorem B.4). The next lemma combines these two facts.
Lemma B.5 (What a point of K K K can earn). Let L L L be a level, let L ≤ κ ≤ N − 1 L \le\kappa\le N - 1 L ≤ κ ≤ N − 1 , and let x ∈ K x \in K x ∈ K . Then
⟨ g ( κ ) , x ⟩ ≥ − 4 N 1 / 4 c κ ( ∥ Π U L x ∥ + r ) − r . \langle g^{(\kappa)},x\rangle\ge-4N^{1/4}c_\kappa\left(\lVert\Pi_{U_L}x\rVert+r\right)-r. ⟨ g ( κ ) , x ⟩ ≥ − 4 N 1/4 c κ ( ∥ Π U L x ∥ + r ) − r . The bound should be compared with the loss − c κ / 2 -c_\kappa/2 − c κ /2 of the comparator. If the hidden component ∥ Π U L x ∥ \lVert\Pi_{U_L}x\rVert ∥ Π U L x ∥ is negligible, the right side is about − 1 10 c κ − r -\frac{1}{10}c_\kappa-r − 10 1 c κ − r , because 4 N 1 / 4 r = 1 10 4N^{1/4}r=\frac{1}{10} 4 N 1/4 r = 10 1 .
Proof. Step 1: Write x x x as a mixture. Every point of K K K is a convex combination of the origin, the atoms and one point of the ball:
x = ∑ j λ j w j + μ b , λ j , μ ≥ 0 , ∑ j λ j + μ ≤ 1 , b ∈ B r . x=\sum_j\lambda_j w_j+\mu b,\qquad\lambda_j,\mu\ge0,\qquad\sum_j\lambda_j+\mu\le1,\qquad b\in B_r. x = j ∑ λ j w j + μ b , λ j , μ ≥ 0 , j ∑ λ j + μ ≤ 1 , b ∈ B r . Step 2: The hidden component controls the weight on unrevealed atoms. Revealed atoms lie in the span of the revealed vectors, so they have no component in U L U_L U L , while by (11) each unrevealed atom contributes its hidden part. Projecting onto U L U_L U L , we get
Π U L x = ∑ j ≥ L λ j a j ( L ) + μ Π U L b . \Pi_{U_L}x=\sum_{j\ge L}\lambda_j a_j^{(L)}+\mu\Pi_{U_L}b. Π U L x = j ≥ L ∑ λ j a j ( L ) + μ Π U L b . The last term has norm at most r r r , so the combination ∑ j ≥ L λ j a j ( L ) \sum_{j\ge L}\lambda_j a_j^{(L)} ∑ j ≥ L λ j a j ( L ) of hidden parts has norm at most ∥ Π U L x ∥ + r \lVert\Pi_{U_L}x\rVert+r ∥ Π U L x ∥ + r . Theorem B.4 then bounds the weight Λ = ∑ j ≥ L λ j \Lambda=\sum_{j\ge L}\lambda_j Λ = ∑ j ≥ L λ j on unrevealed atoms:
Λ ≤ 4 N 1 / 4 ( ∥ Π U L x ∥ + r ) . \Lambda\le4N^{1/4}\left(\lVert\Pi_{U_L}x\rVert+r\right). Λ ≤ 4 N 1/4 ( ∥ Π U L x ∥ + r ) . Step 3: Compute the loss. By Theorem B.1, atoms before κ \kappa κ contribute nothing, each atom from κ \kappa κ on contributes − ρ j c κ ≥ − c κ -\rho_jc_\kappa\ge-c_\kappa − ρ j c κ ≥ − c κ per unit of weight, and the ball point contributes at least − r -r − r . Since κ ≥ L \kappa\ge L κ ≥ L , the atoms from κ \kappa κ on are unrevealed, so their total weight is at most Λ \Lambda Λ . Hence ⟨ g ( κ ) , x ⟩ ≥ − c κ Λ − r \langle g^{(\kappa)},x\rangle\ge-c_\kappa\Lambda-r ⟨ g ( κ ) , x ⟩ ≥ − c κ Λ − r , and Step 2 gives the claim.
Theorem B.5 applies to every point of K K K . We never assume that the learner plays combinations of oracle answers: it may play any point it can certify to lie in K K K , for instance using its knowledge of the distribution of the instance.
The clean process We want to show that each oracle call reveals at most the next atom. The difficulty is circular: the oracle’s answers depend on the unrevealed frame, so the learner’s later messages depend on it too, and then we cannot use the randomness of the unrevealed frame to control those messages. To break the circle we introduce a clean oracle that answers using only the revealed vectors. The clean process is the run of (a copy of) the learner against this oracle, as in § 3.2 . Theorem B.8 shows that it agrees with the true oracle as long as the learner’s messages are nearly orthogonal to the unrevealed frame, and Section B.4 shows that this holds with high probability. The clean process is an analysis device: the actual run is against the true oracle, and Section B.5 shows that the two runs coincide with high probability.
Definition B.6 (Clean oracle). Fix a margin parameter τ \tau τ with 0 < τ ≤ γ r / 5 0 < \tau\le\gamma r/5 0 < τ ≤ γ r /5 . The clean oracle keeps a level L L L , initially 0, and handles queries and losses as follows.
Queries. For the query d = 0 d = 0 d = 0 it answers 0. For a query d ≠ 0 d \ne0 d = 0 at level L L L , it computes three numbers from the revealed vectors:
V = min ( 0 , min s < L ⟨ d , w s ⟩ , − r ∥ d ∥ ) , A = ρ L ⟨ d , M L ⟩ , β = τ ∥ d ∥ . V = \min\left(0,\min_{s<L}\langle d,w_s\rangle,-r\lVert d\rVert\right), \qquad A = \rho_L\langle d,M_L\rangle, \qquad\beta= \tau\lVert d\rVert. V = min ( 0 , s < L min ⟨ d , w s ⟩ , − r ∥ d ∥ ) , A = ρ L ⟨ d , M L ⟩ , β = τ ∥ d ∥ . Here V V V is the best value among the candidates known at level L L L , namely 0, the revealed atoms w 0 , … , w L − 1 w_0,\ldots,w_{L-1} w 0 , … , w L − 1 (none when L = 0 L=0 L = 0 ) and the ball point − r d / ∥ d ∥ -r\,d/\lVert d\rVert − r d / ∥ d ∥ , and p V p_V p V denotes the first of them, in this order (the order of the oracle’s list in § 3.1 ), that attains V V V . The number A A A is the value the next atom w L w_L w L would have if its hidden part were zero, and β \beta β is a margin for the effect of hidden parts. The clean oracle’s answer has two cases:
(C1) If A ≥ V + β A \ge V+\beta A ≥ V + β , the answer is p V p_V p V and the level does not change. The reason: even allowing for the hidden parts, no unrevealed atom can beat the known candidates.
(C2) Otherwise, the level is raised to L + 1 L+1 L + 1 , which reveals ν L \nu_L ν L and u L + 1 u_{L+1} u L + 1 and hence the atom w L w_L w L . The answer is the first candidate among 0 , w 0 , … , w L 0,w_0,\ldots,w_L 0 , w 0 , … , w L and − r d / ∥ d ∥ -r\,d/\lVert d\rVert − r d / ∥ d ∥ that minimizes ⟨ d , ⋅ ⟩ \langle d,\cdot\rangle ⟨ d , ⋅ ⟩ . The reason: the next atom might win, so we reveal it and compare exactly.
Losses. After the play of round t t t , the loss g t = g ( κ t ) g_t=g^{(\kappa_t)} g t = g ( κ t ) is announced, and the level is raised to κ t \kappa_t κ t if it is lower. This is needed because the loss vector is a function of the vectors revealed at level κ t \kappa_t κ t .
Every answer of the clean oracle, and every change of its level, is computed from the learner’s messages and the revealed vectors only. Rule (C2) requires L ≤ N − 2 L\le N-2 L ≤ N − 2 ; the next lemma shows that this always holds.
Lemma B.7 (Level counting). In the clean process, the i i i -th query of round t ≥ 1 t\ge1 t ≥ 1 (1 ≤ i ≤ m 1\le i\le m 1 ≤ i ≤ m ) is made at level at most m ( t − 1 ) + i − 1 m(t-1)+i-1 m ( t − 1 ) + i − 1 , which is at most κ t − 1 ≤ N − 2 \kappa_t-1\le N-2 κ t − 1 ≤ N − 2 . The play of round t t t is made at level at most κ t = m t \kappa_t=mt κ t = m t .
Proof. Each query raises the level by at most one, by construction. Round 0 makes no query, so x 0 x_0 x 0 is played at level 0 = κ 0 0=\kappa_0 0 = κ 0 , and the loss g 0 = g ( 0 ) g_0=g^{(0)} g 0 = g ( 0 ) leaves the level at 0. Suppose that the level is at most κ t − 1 = m ( t − 1 ) \kappa_{t-1}=m(t-1) κ t − 1 = m ( t − 1 ) when round t ≥ 1 t\ge1 t ≥ 1 starts. Then the i i i -th query of the round is made at level at most m ( t − 1 ) + i − 1 m(t-1)+i-1 m ( t − 1 ) + i − 1 , and the play at level at most m ( t − 1 ) + m = κ t m(t-1)+m=\kappa_t m ( t − 1 ) + m = κ t . When the loss is announced the level becomes exactly κ t = m t \kappa_t=mt κ t = m t , which is the induction hypothesis for round t + 1 t+1 t + 1 . Finally κ t − 1 ≤ m ( T − 1 ) − 1 = N − 2 \kappa_t-1\le m(T-1)-1=N-2 κ t − 1 ≤ m ( T − 1 ) − 1 = N − 2 .
The next lemma shows that clean oracle answers are true oracle answers for queries with small hidden component.
Lemma B.8. Let d d d be a query made at level L ≤ N − 2 L\le N-2 L ≤ N − 2 , and suppose that d = 0 d=0 d = 0 or ∥ Π U L d ∥ < τ ∥ d ∥ \lVert\Pi_{U_L}d\rVert<\tau\lVert d\rVert ∥ Π U L d ∥ < τ ∥ d ∥ . Then the true oracle’s answer LMO ( d ) \operatorname{LMO}(d) LMO ( d ) is the clean oracle answer.
Proof. If d = 0 d=0 d = 0 both answers are 0, so let d ≠ 0 d\ne0 d = 0 . Recall the definitions of V , A , β V,A,\beta V , A , β in Theorem B.6.
Step 1: The values of the unrevealed atoms. By (11) , every unrevealed atom has value
⟨ d , w j ⟩ = ρ j A 0 + ε j , where A 0 = ⟨ d , M L ⟩ and ε j = ⟨ Π U L d , a j ( L ) ⟩ . \langle d,w_j\rangle=\rho_j A_0+\varepsilon_j,\qquad\text{where}\qquad A_0=\langle d,M_L\rangle\quad\text{and}\quad\varepsilon_j=\langle\Pi_{U_L}d,a_j^{(L)}\rangle. ⟨ d , w j ⟩ = ρ j A 0 + ε j , where A 0 = ⟨ d , M L ⟩ and ε j = ⟨ Π U L d , a j ( L ) ⟩ .
The known quantity A 0 A_0 A 0 is common to all unrevealed atoms. The hidden term ε j \varepsilon_j ε j is small: by the assumption ∥ Π U L d ∥ < τ ∥ d ∥ \lVert\Pi_{U_L}d\rVert<\tau\lVert d\rVert ∥ Π U L d ∥ < τ ∥ d ∥ , and since ∥ a j ( L ) ∥ ≤ 1 \lVert a_j^{(L)}\rVert\le1 ∥ a j ( L ) ∥ ≤ 1 , we have ∣ ε j ∣ < β \lvert\varepsilon_j\rvert<\beta ∣ ε j ∣ < β . The ball point has value − r ∥ d ∥ -r\lVert d\rVert − r ∥ d ∥ , so V ≤ − r ∥ d ∥ V\le-r\lVert d\rVert V ≤ − r ∥ d ∥ , and since τ ≤ γ r / 5 ≤ r / 10 \tau\le\gamma r/5\le r/10 τ ≤ γ r /5 ≤ r /10 this is well below − β -\beta − β :
V ≤ − r ∥ d ∥ < − β . V\le-r\lVert d\rVert<-\beta. V ≤ − r ∥ d ∥ < − β .
The true oracle returns the first candidate of the full list 0 , w 0 , … , w N − 1 , − r d / ∥ d ∥ 0,w_0,\ldots,w_{N-1},-r\,d/\lVert d\rVert 0 , w 0 , … , w N − 1 , − r d / ∥ d ∥ that attains the true minimum. We show that the unrevealed atoms never change the clean answer.
Step 2: Under rule (C1) no unrevealed atom reaches V V V . Suppose A ≥ V + β A\ge V+\beta A ≥ V + β , so the clean oracle uses rule (C1). If A 0 ≥ 0 A_0\ge0 A 0 ≥ 0 , every unrevealed atom has value greater than − β -\beta − β , hence greater than V V V . If A 0 < 0 A_0<0 A 0 < 0 , the shrinkage makes later atoms worse, not better: ρ j ≤ ρ L \rho_j\le\rho_L ρ j ≤ ρ L gives ρ j A 0 ≥ ρ L A 0 = A \rho_jA_0\ge\rho_LA_0=A ρ j A 0 ≥ ρ L A 0 = A , so the value is greater than A − β ≥ V A-\beta\ge V A − β ≥ V . In both cases every unrevealed atom is strictly above V V V . So the true minimum is V V V , and the first candidate attaining it is p V p_V p V . Step 3: Under rule (C2) no atom beyond w L w_L w L can win. Otherwise, A < V + β A < V + \beta A < V + β and the clean oracle uses rule (C2). Suppose an atom w j w_j w j with j > L j > L j > L has value at most V V V (otherwise, the true oracle answers is the clean oracle answers under rule (C2)). We show that w L w_L w L is then strictly better, so w j w_j w j is not a minimizer.
First, the known part of the query must point strongly towards the walk. Indeed ρ j A 0 + ε j ≤ V ≤ − r ∥ d ∥ \rho_j A_0 + \varepsilon_j \le V \le-r\lVert d\rVert ρ j A 0 + ε j ≤ V ≤ − r ∥ d ∥ and ε j > − β \varepsilon_j > -\beta ε j > − β , so
ρ j A 0 < − r ∥ d ∥ + β < 0 , hence ∣ A 0 ∣ > r ∥ d ∥ − β ≥ 9 10 r ∥ d ∥ . \rho_j A_0 < -r\lVert d\rVert+ \beta< 0,\qquad\text{hence}\qquad|A_0| > r\lVert d\rVert-\beta\ge\frac{9}{10}r\lVert d\rVert. ρ j A 0 < − r ∥ d ∥ + β < 0 , hence ∣ A 0 ∣ > r ∥ d ∥ − β ≥ 10 9 r ∥ d ∥ .
using ρ j ≤ 1 \rho_j \le1 ρ j ≤ 1 and β ≤ r ∥ d ∥ / 10 \beta\le r\lVert d\rVert/10 β ≤ r ∥ d ∥ /10 . This is where the ball matters: since it offers the value − r ∥ d ∥ -r\lVert d\rVert − r ∥ d ∥ , an unrevealed atom can compete only if ∣ A 0 ∣ |A_0| ∣ A 0 ∣ is of order r ∥ d ∥ r\lVert d\rVert r ∥ d ∥ .
Second, the shrinkage turns this into a strict preference for w L w_L w L . Since A 0 < 0 A_0 < 0 A 0 < 0 and ρ L − ρ j ≥ γ \rho_L-\rho_j \ge\gamma ρ L − ρ j ≥ γ ,
⟨ d , w L ⟩ − ⟨ d , w j ⟩ = − ( ρ L − ρ j ) ∣ A 0 ∣ + ε L − ε j < − 9 10 γ r ∥ d ∥ + 2 τ ∥ d ∥ ≤ − 1 2 γ r ∥ d ∥ < 0 , \langle d,w_L\rangle-\langle d,w_j\rangle=-(\rho_L-\rho_j)|A_0|+\varepsilon_L-\varepsilon_j<-\frac{9}{10}\gamma r\lVert d\rVert+2\tau\lVert d\rVert\le-\frac{1}{2}\gamma r\lVert d\rVert<0, ⟨ d , w L ⟩ − ⟨ d , w j ⟩ = − ( ρ L − ρ j ) ∣ A 0 ∣ + ε L − ε j < − 10 9 γ r ∥ d ∥ + 2 τ ∥ d ∥ ≤ − 2 1 γ r ∥ d ∥ < 0 ,
using τ ≤ γ r / 5 \tau\le\gamma r/5 τ ≤ γ r /5 . So the minimizers of ⟨ d , ⋅ ⟩ \langle d,\cdot\rangle ⟨ d , ⋅ ⟩ in the full list are among 0 , w 0 , … , w L 0,w_0,\ldots,w_L 0 , w 0 , … , w L and the ball point. These appear in the same order in the full list and in the list used by (C2), so the true answer is the clean answer. □
The proof shows how the two design choices work together. Because the ball offers − r ∥ d ∥ -r\lVert d\rVert − r ∥ d ∥ for free, an unrevealed atom can win only if ∣ A 0 ∣ |A_0| ∣ A 0 ∣ is at least about r ∥ d ∥ r\lVert d\rVert r ∥ d ∥ . When ∣ A 0 ∣ |A_0| ∣ A 0 ∣ is that large, the shrinkage gives the next atom an advantage of at least γ ∣ A 0 ∣ ≈ γ r ∥ d ∥ \gamma|A_0|\approx\gamma r\lVert d\rVert γ ∣ A 0 ∣ ≈ γ r ∥ d ∥ , which exceeds any perturbation from the hidden parts, at most 2 τ ∥ d ∥ 2\tau\lVert d\rVert 2 τ ∥ d ∥ .
Hidden directions stay hidden with high probability
It remains to show that the hypothesis of Theorem B.8 holds for every query with high probability, and that the plays have small hidden components, as needed in Theorem B.5. Let v 1 , v 2 , … , v Q v_1,v_2,\ldots,v_Q v 1 , v 2 , … , v Q be all messages of the learner in the clean process (its queries and its plays, in order), so that Q ≤ m ( T − 1 ) + T ≤ 2 N Q\le m(T-1)+T\le2N Q ≤ m ( T − 1 ) + T ≤ 2 N . Let L i L_i L i be the level when v i v_i v i is sent, ℓ i = 2 L i + 1 \ell_i=2L_i+1 ℓ i = 2 L i + 1 the number of revealed frame vectors, and U i = U L i U_i=U_{L_i} U i = U L i . In this section, we denote by ϕ 1 , … , ϕ 2 N \phi_1,\ldots,\phi_{2N} ϕ 1 , … , ϕ 2 N the random orthonormal vectors u 0 , ν 0 , … , u N − 1 , ν N − 1 u_0,\nu_0,\ldots,u_{N-1},\nu_{N-1} u 0 , ν 0 , … , u N − 1 , ν N − 1 in the same order.
The argument has two parts. First, in the clean process every message is a function of the revealed vectors (Theorem B.9). Second, given the revealed vectors, the rest of the frame is uniformly random, and a fixed vector has a tiny projection onto a random low-dimensional subspace of a high-dimensional space (Theorem B.10).
Lemma B.9 (Prefix property). For every i i i and ℓ \ell ℓ , the event { ℓ i = ℓ } \{\ell_i=\ell\} { ℓ i = ℓ } , and the message v i v_i v i on this event, are functions of ϕ 1 , … , ϕ ℓ \phi_1,\ldots,\phi_\ell ϕ 1 , … , ϕ ℓ .
Proof. The clean process uses the frame only through revealed vectors, and it reveals them one at a time in the fixed order ϕ 1 , ϕ 2 , … \phi_1,\phi_2,\ldots ϕ 1 , ϕ 2 , … . The decision to reveal the next vector depends only on the vectors already revealed and on the learner’s messages. The learner is deterministic, so its messages are functions of the answers and losses it has received, which are themselves computed from revealed vectors. By induction over the steps of the process, everything that happens before the ( ℓ + 1 ) (\ell+1) ( ℓ + 1 ) -th vector is revealed is a function of ϕ 1 , … , ϕ ℓ \phi_1,\ldots,\phi_\ell ϕ 1 , … , ϕ ℓ . Both claims follow. □
Lemma B.10 (Projection onto a random subspace). Let W W W be a Euclidean space of dimension n W n_W n W , let U U U be a uniformly random k k k -dimensional subspace of W W W , and let e ∈ W e\in W e ∈ W be a fixed unit vector. For every s ≥ 0 s\ge0 s ≥ 0 with n W > 1 + s \sqrt{n_W}>1+s n W > 1 + s ,
P [ ∥ Π U e ∥ ≥ k + s n W − 1 − s ] ≤ 2 e − s 2 / 2 . \mathbb{P}\left[\lVert\Pi_U e\rVert\ge\frac{\sqrt{k}+s}{\sqrt{n_W}-1-s}\right]\le2e^{-s^2/2}. P [ ∥ Π U e ∥ ≥ n W − 1 − s k + s ] ≤ 2 e − s 2 /2 .
The bound says that ∥ Π U e ∥ \lVert\Pi_U e\rVert ∥ Π U e ∥ is at most about k / n W \sqrt{k/n_W} k / n W with high probability: a fixed direction has a tiny projection onto a random subspace of much smaller dimension.
Proof. Step 1: Reduce to a Gaussian vector. By rotation invariance we may instead fix U U U , say as the span of the first k k k coordinates of W W W , and take e e e uniformly distributed on the unit sphere of W W W . A uniform point of the sphere can be written z / ∥ z ∥ z/\lVert z\rVert z / ∥ z ∥ with z z z a standard Gaussian vector of W W W , so ∥ Π U e ∥ \lVert\Pi_Ue\rVert ∥ Π U e ∥ has the law of ∥ z 1 : k ∥ / ∥ z ∥ \lVert z_{1:k}\rVert/\lVert z\rVert ∥ z 1 : k ∥ / ∥ z ∥ , where z 1 : k z_{1:k} z 1 : k denotes the first k k k coordinates. Step 2: The numerator and the denominator concentrate. The maps z ↦ ∥ z 1 : k ∥ z \mapsto\lVert z_{1:k}\rVert z ↦ ∥ z 1 : k ∥ and z ↦ ∥ z ∥ z \mapsto\lVert z\rVert z ↦ ∥ z ∥ are 1-Lipschitz, so by Gaussian concentration [2 Thm. 5.6] each is within s s s of its mean, on the relevant side, with probability at least 1 − e − s 2 / 2 1-e^{-s^{2}/2} 1 − e − s 2 /2 . The means satisfy E ∥ z 1 : k ∥ ≤ k \mathbb{E}\lVert z_{1:k}\rVert\le\sqrt{k} E ∥ z 1 : k ∥ ≤ k , by Jensen’s inequality, and E ∥ z ∥ ≥ n W − 1 ≥ n W − 1 \mathbb{E}\lVert z\rVert\ge\sqrt{n_W-1} \ge\sqrt{n_W}-1 E ∥ z ∥ ≥ n W − 1 ≥ n W − 1 , because the variance of ∥ z ∥ \lVert z\rVert ∥ z ∥ is at most one by the Gaussian Poincaré inequality. On the intersection of the two events, the ratio is below the stated threshold.
Proposition B.11 (The good event). Let δ ∈ ( 0 , 1 ) \delta\in(0,1) δ ∈ ( 0 , 1 ) and s = 2 log ( 2 Q / δ ) s=\sqrt{2\log(2Q/\delta)} s = 2 log ( 2 Q / δ ) , assume n − 2 N > 1 + s \sqrt{n-2N}>1+s n − 2 N > 1 + s , and let
τ n = 2 N + s n − 2 N − 1 − s . \tau_n=\frac{\sqrt{2N}+s}{\sqrt{n-2N}-1-s}. τ n = n − 2 N − 1 − s 2 N + s . If τ n ≤ τ \tau_n\le\tau τ n ≤ τ , then with probability at least 1 − δ 1-\delta 1 − δ every message satisfies v i = 0 v_i=0 v i = 0 or ∥ Π U i v i ∥ < τ ∥ v i ∥ \lVert\Pi_{U_i}v_i\rVert<\tau\lVert v_i\rVert ∥ Π U i v i ∥ < τ ∥ v i ∥ . We call this event E \mathcal{E} E .
Proof. Step 1: Condition on the revealed vectors. Fix a message index i i i and a prefix length ℓ \ell ℓ , and condition on ϕ 1 , … , ϕ ℓ \phi_1,\ldots,\phi_\ell ϕ 1 , … , ϕ ℓ . By Theorem B.9, on the event { ℓ i = ℓ } \{\ell_i=\ell\} { ℓ i = ℓ } the message v i v_i v i is then a fixed vector.
Step 2: The rest of the frame is still random. Because the frame is uniformly random, its remaining vectors ϕ ℓ + 1 , … , ϕ 2 N \phi_{\ell+1},\ldots,\phi_{2N} ϕ ℓ + 1 , … , ϕ 2 N form, conditionally, a uniformly random orthonormal family in the orthogonal complement W W W of the revealed vectors. So U i U_i U i is a uniformly random subspace of W W W , of dimension k = 2 N − ℓ ≤ 2 N k=2N-\ell\le2N k = 2 N − ℓ ≤ 2 N , while W W W has dimension n W = n − ℓ ≥ n − 2 N n_W=n-\ell\ge n-2N n W = n − ℓ ≥ n − 2 N .
Step 3: Apply Theorem B.10. There is nothing to prove if v i = 0 v_i=0 v i = 0 , or if v i v_i v i is orthogonal to W W W (then Π U i v i = 0 \Pi_{U_i}v_i=0 Π U i v i = 0 ). Otherwise let e e e be the normalized projection of v i v_i v i onto W W W . Since U i ⊂ W U_i\subset W U i ⊂ W , the projection of v i v_i v i onto U i U_i U i is its projection onto W W W followed by Π U i \Pi_{U_i} Π U i , so ∥ Π U i v i ∥ ≤ ∥ v i ∥ ∥ Π U i e ∥ \lVert\Pi_{U_i}v_i\rVert\le\lVert v_i\rVert\lVert\Pi_{U_i}e\rVert ∥ Π U i v i ∥ ≤ ∥ v i ∥ ∥ Π U i e ∥ . The threshold of Theorem B.10 with these k k k and n W n_W n W is at most τ n \tau_n τ n , so the conditional probability that ∥ Π U i v i ∥ ≥ τ n ∥ v i ∥ \lVert\Pi_{U_i}v_i\rVert\ge\tau_n\lVert v_i\rVert ∥ Π U i v i ∥ ≥ τ n ∥ v i ∥ is at most 2 e − s 2 / 2 = δ / Q 2e^{-s^{2}/2}=\delta/Q 2 e − s 2 /2 = δ / Q .
Step 4: Remove the conditioning. Averaging over ϕ 1 , … , ϕ ℓ \phi_1,\ldots,\phi_\ell ϕ 1 , … , ϕ ℓ and summing over ℓ \ell ℓ , message i i i fails with probability at most δ / Q \delta/Q δ / Q . A union bound over the at most Q Q Q messages, together with τ n ≤ τ \tau_n\le\tau τ n ≤ τ , gives the claim.
Putting the pieces together
Step 1: parameters. We take
r = 1 40 N − 1 / 4 , τ = γ r 5 = N − 1 / 4 400 ( N − 1 ) , δ = 1 100 N − 1 / 4 , n ≥ 1.1 ⋅ 10 6 N 7 / 2 . r=\frac{1}{40}N^{-1/4},\qquad\tau=\frac{\gamma r}{5}=\frac{N^{-1/4}}{400(N-1)},\qquad\delta=\frac{1}{100}N^{-1/4},\qquad n\ge1.1\cdot10^{6}N^{7/2}. r = 40 1 N − 1/4 , τ = 5 γ r = 400 ( N − 1 ) N − 1/4 , δ = 100 1 N − 1/4 , n ≥ 1.1 ⋅ 1 0 6 N 7/2 .
Each choice has a reason.
The ball radius r r r is small enough that the ball earns the learner at most r = 1 40 N − 1 / 4 r=\frac{1}{40}N^{-1/4} r = 40 1 N − 1/4 per round and lets it fake a weight of at most 4 N 1 / 4 r = 1 10 4N^{1/4}r=\frac{1}{10} 4 N 1/4 r = 10 1 on unrevealed atoms (Theorem B.5).
The margin τ \tau τ is the largest allowed by Theorem B.8.
The failure probability δ \delta δ is small enough that failures cost little (Step 5).
The dimension n n n makes a random direction’s projection onto the unrevealed frame, of order N / n \sqrt{N/n} N / n , smaller than τ ≈ N − 5 / 4 \tau\approx N^{-5/4} τ ≈ N − 5/4 . Precisely, with Q ≤ 2 N Q\le2N Q ≤ 2 N we have 2 Q / δ ≤ 400 N 5 / 4 2Q/\delta\le400N^{5/4} 2 Q / δ ≤ 400 N 5/4 , and a direct computation shows that n ≥ 1.1 ⋅ 10 6 N 7 / 2 n\ge1.1\cdot10^{6}N^{7/2} n ≥ 1.1 ⋅ 1 0 6 N 7/2 implies n − 2 N > 1 + s \sqrt{n-2N}>1+s n − 2 N > 1 + s and τ n ≤ τ \tau_n\le\tau τ n ≤ τ for every N ≥ 2 N\ge2 N ≥ 2 . For large N N N it suffices that n ≥ 3.2 ⋅ 10 5 N 7 / 2 n\ge3.2\cdot10^{5}N^{7/2} n ≥ 3.2 ⋅ 1 0 5 N 7/2 .
By Theorem B.11, the good event E \mathcal{E} E has probability at least 1 − δ 1-\delta 1 − δ .
Step 2: on the good event the learner is in the clean process. Run the learner against the true oracle and, in parallel, against the clean oracle. On E \mathcal{E} E the two runs produce the same messages and answers. We check this message by message. As long as all answers so far agree, the next message is the same in both runs, because the learner is deterministic and the losses are fixed. If this message is a query, it is made at a level at most N − 2 N-2 N − 2 (Theorem B.7) and, on E \mathcal{E} E , it satisfies the hypothesis of Theorem B.8; so the two answers agree again. Step 3: what the learner knows at each play. By Theorem B.7, the play x t x_t x t of round t t t is made at a level L t ≤ κ t L_t \le\kappa_t L t ≤ κ t . So every atom with index at least κ t \kappa_t κ t , the comparator included, is unrevealed at that time. On E \mathcal{E} E the play also has a small hidden component, ∥ Π U L t x t ∥ ≤ τ ∥ x t ∥ ≤ τ \|\Pi_{U_{L_t}}x_t\| \le\tau\|x_t\| \le\tau ∥ Π U L t x t ∥ ≤ τ ∥ x t ∥ ≤ τ , using ∥ x t ∥ ≤ 1 \|x_t\| \le1 ∥ x t ∥ ≤ 1 (Theorem B.2).
Step 4: the regret of one round. Theorem B.5, applied with κ = κ t \kappa= \kappa_t κ = κ t and L = L t L = L_t L = L t , bounds what the play earns:
⟨ g t , x t ⟩ ≥ − c κ t ⋅ 4 N 1 / 4 ( τ + r ) − r ≥ − 0.11 c κ t − r , \langle g_t,x_t\rangle\ge-c_{\kappa_t}\cdot4N^{1/4}(\tau+r)-r \ge-0.11c_{\kappa_t}-r, ⟨ g t , x t ⟩ ≥ − c κ t ⋅ 4 N 1/4 ( τ + r ) − r ≥ − 0.11 c κ t − r ,
since 4 N 1 / 4 r = 0.1 4N^{1/4}r=0.1 4 N 1/4 r = 0.1 and 4 N 1 / 4 τ = 1 / ( 100 ( N − 1 ) ) ≤ 0.01 4N^{1/4}\tau=1/(100(N-1))\le0.01 4 N 1/4 τ = 1/ ( 100 ( N − 1 )) ≤ 0.01 . The comparator earns c κ t / 2 c_{\kappa_t}/2 c κ t /2 by Theorem B.1, because its index N − 1 N-1 N − 1 is at least κ t \kappa_t κ t . With Theorem B.3 and the value of r r r ,
⟨ g t , x t − x ⋆ ⟩ ≥ 0.39 c κ t − r ≥ ( 0.39 2 − 1 40 ) N − 1 / 4 ≥ 0.25 N − 1 / 4 . \langle g_t,x_t-x^\star\rangle\ge0.39c_{\kappa_t}-r \ge\left(\frac{0.39}{\sqrt{2}}-\frac{1}{40}\right)N^{-1/4}\ge0.25N^{-1/4}. ⟨ g t , x t − x ⋆ ⟩ ≥ 0.39 c κ t − r ≥ ( 2 0.39 − 40 1 ) N − 1/4 ≥ 0.25 N − 1/4 .
Summing over the T T T rounds, the regret against x ⋆ x^\star x ⋆ , and hence the regret, is at least 0.25 T N − 1 / 4 0.25TN^{-1/4} 0.25 T N − 1/4 on E \mathcal{E} E .
Step 5: the expectation. Off the good event we use the trivial bound: the regret is at least − T -T − T times the diameter, which is at least − 1.12 T -1.12T − 1.12 T by Theorem B.2. Therefore
E [ Reg T ] ≥ 0.25 T N − 1 / 4 ( 1 − δ ) − 1.12 T δ ≥ ( 0.25 − 0.0025 − 0.0112 ) T N − 1 / 4 ≥ 0.23 T N − 1 / 4 . \mathbb{E}[\operatorname{Reg}_T]\ge0.25TN^{-1/4}(1-\delta)-1.12T\delta\ge(0.25-0.0025-0.0112)TN^{-1/4}\ge0.23TN^{-1/4}. E [ Reg T ] ≥ 0.25 T N − 1/4 ( 1 − δ ) − 1.12 T δ ≥ ( 0.25 − 0.0025 − 0.0112 ) T N − 1/4 ≥ 0.23 T N − 1/4 .
This proves Theorem 3.1. □ \square □
Proofs for Section 4: the upper bound
We use the notation of § 4 : rounds are numbered t = 1 , … , T t=1,\ldots,T t = 1 , … , T , G t = ∑ s ≤ t g s G_t=\sum_{s\le t}g_s G t = ∑ s ≤ t g s , and Reg T = ∑ t = 1 T ⟨ g t , x t − x ⋆ ⟩ \operatorname{Reg}_T=\sum_{t=1}^T\langle g_t,x_t-x^\star\rangle Reg T = ∑ t = 1 T ⟨ g t , x t − x ⋆ ⟩ for a fixed comparator x ⋆ ∈ K x^\star\in K x ⋆ ∈ K . For convex losses ℓ t \ell_t ℓ t with g t ∈ ∂ ℓ t ( x t ) g_t\in\partial\ell_t(x_t) g t ∈ ∂ ℓ t ( x t ) , convexity gives ∑ t ℓ t ( x t ) − ℓ t ( x ⋆ ) ≤ Reg T \sum_t\ell_t(x_t)-\ell_t(x^\star)\le\operatorname{Reg}_T ∑ t ℓ t ( x t ) − ℓ t ( x ⋆ ) ≤ Reg T , so it suffices to bound Reg T \operatorname{Reg}_T Reg T , and we work with the linear losses x ↦ ⟨ g t , x ⟩ x\mapsto\langle g_t,x\rangle x ↦ ⟨ g t , x ⟩ throughout. We only use ∥ g t ∥ ≤ L \|g_t\|\le L ∥ g t ∥ ≤ L and ∥ p − q ∥ ≤ D \|p-q\|\le D ∥ p − q ∥ ≤ D for all p , q ∈ K p,q\in K p , q ∈ K , and the fact that the minimizer x ⋆ x_\star x ⋆ of a differentiable convex function f f f over K K K satisfies ⟨ ∇ f ( x ⋆ ) , x ⋆ − x ⟩ ≤ 0 \langle\nabla f(x_\star),x_\star-x\rangle\le0 ⟨ ∇ f ( x ⋆ ) , x ⋆ − x ⟩ ≤ 0 for every x ∈ K x\in K x ∈ K .
The proofs follow those of Weibel et al. [20 ] for m = 1 m=1 m = 1 step by step. The only new ingredient is that a round of Algorithm 1 performs m m m Frank–Wolfe steps instead of one, so that the contraction factor 1 − 2 σ 1-2\sigma 1 − 2 σ of a single step becomes ( 1 − 2 σ ) m = 1 − u (1-2\sigma)^m=1-u ( 1 − 2 σ ) m = 1 − u , with u u u as in (7) ; for m = 1 m=1 m = 1 , u = 2 σ u=2\sigma u = 2 σ and every statement below is the corresponding statement of Weibel et al. [20 ] .
The exact minimizers and the potential. Let y 1 : = x 1 y_1:=x_1 y 1 := x 1 and, for t ≥ 1 t\ge1 t ≥ 1 ,
y t + 1 : = arg min y ∈ K η ⟨ G t , y ⟩ + 1 2 ∥ y − x 1 ∥ 2 , y_{t+1}:=\underset{y\in K}{\arg\min}\ \eta\langle G_t,y\rangle+\frac{1}{2}\|y-x_1\|^2, y t + 1 := y ∈ K arg min η ⟨ G t , y ⟩ + 2 1 ∥ y − x 1 ∥ 2 ,
the iterates of follow-the-regularized-leader run on the same gradients. The potential is
ϕ t : = ∑ s = 1 t ⟨ g s , x s − y t + 1 ⟩ − 1 2 η ∥ y t + 1 − x 1 ∥ 2 + 1 6 η ∥ x t + 1 − y t + 1 ∥ 2 , t = 0 , … , T , \phi_t:=\sum_{s=1}^{t}\langle g_s,x_s-y_{t+1}\rangle-\frac{1}{2\eta}\|y_{t+1}-x_1\|^2+\frac{1}{6\eta}\|x_{t+1}-y_{t+1}\|^2,\qquad t=0,\ldots,T, ϕ t := s = 1 ∑ t ⟨ g s , x s − y t + 1 ⟩ − 2 η 1 ∥ y t + 1 − x 1 ∥ 2 + 6 η 1 ∥ x t + 1 − y t + 1 ∥ 2 , t = 0 , … , T ,
so that ϕ 0 = 0 \phi_0=0 ϕ 0 = 0 . Its first two terms are the potential of the classical regret analysis of follow-the-regularized-leader; the third one records how far the Frank–Wolfe iterate is from the exact minimizer.
Fixed parameters
Lemma C.1 (Per-round inequality). Let σ ∈ ( 0 , 1 2 ] \sigma\in(0,\frac{1}{2}] σ ∈ ( 0 , 2 1 ] , η > 0 \eta>0 η > 0 , and u u u , λ g = 8 η / ( 3 u ) \lambda^g=8\eta/(3u) λ g = 8 η / ( 3 u ) as in (7) . For every t ∈ { 1 , … , T } t\in\{1,\ldots,T\} t ∈ { 1 , … , T } ,
ϕ t − ϕ t − 1 ≤ λ g ∥ g t ∥ 2 + σ 2 6 η ∑ k = 1 m ( 1 − 2 σ ) m − k ∥ z t , k − 1 − v t , k ∥ 2 . \phi_t-\phi_{t-1}\le\lambda^g\|g_t\|^2+\frac{\sigma^2}{6\eta}\sum_{k=1}^{m}(1-2\sigma)^{m-k}\|z_{t,k-1}-v_{t,k}\|^2. ϕ t − ϕ t − 1 ≤ λ g ∥ g t ∥ 2 + 6 η σ 2 k = 1 ∑ m ( 1 − 2 σ ) m − k ∥ z t , k − 1 − v t , k ∥ 2 .
For m = 1 m=1 m = 1 the sum is ∥ x t − v t ∥ 2 \|x_t-v_t\|^2 ∥ x t − v t ∥ 2 and λ g = 4 η / ( 3 σ ) \lambda^g=4\eta/(3\sigma) λ g = 4 η / ( 3 σ ) , which is the lemma of Weibel et al. [20 ] .
Proof. Fix t t t and write P : = x t − y t + 1 P := x_t - y_{t+1} P := x t − y t + 1 and Y : = y t − y t + 1 Y := y_t - y_{t+1} Y := y t − y t + 1 , so that x t − y t = P − Y x_t - y_t = P - Y x t − y t = P − Y .
Step 1: the regularizer terms. By definition of the potential,
ϕ t − ϕ t − 1 = ⟨ g t , P ⟩ + ⟨ G t − 1 , Y ⟩ + 1 6 η ( ∥ x t + 1 − y t + 1 ∥ 2 − ∥ x t − y t ∥ 2 ) + 1 2 η ( ∥ y t − x 1 ∥ 2 − ∥ y t + 1 − x 1 ∥ 2 ) . \phi_t - \phi_{t-1} = \langle g_t, P\rangle+ \langle G_{t-1}, Y\rangle
+ \frac{1}{6\eta}\left(\lVert x_{t+1} - y_{t+1}\rVert^2 - \lVert x_t - y_t\rVert^2\right) + \frac{1}{2\eta}\left(\lVert y_t - x_1\rVert^2 - \lVert y_{t+1} - x_1\rVert^2\right). ϕ t − ϕ t − 1 = ⟨ g t , P ⟩ + ⟨ G t − 1 , Y ⟩ + 6 η 1 ( ∥ x t + 1 − y t + 1 ∥ 2 − ∥ x t − y t ∥ 2 ) + 2 η 1 ( ∥ y t − x 1 ∥ 2 − ∥ y t + 1 − x 1 ∥ 2 ) .
The optimality of y t y_t y t for η ⟨ G t − 1 , ⋅ ⟩ + 1 2 ∥ ⋅ − x 1 ∥ 2 \eta\langle G_{t-1}, \cdot\rangle+ \frac{1}{2}\lVert\cdot- x_1\rVert^2 η ⟨ G t − 1 , ⋅ ⟩ + 2 1 ∥ ⋅ − x 1 ∥ 2 gives ⟨ G t − 1 , Y ⟩ ≤ − 1 η ⟨ y t − x 1 , Y ⟩ \langle G_{t-1}, Y\rangle\le-\frac{1}{\eta}\langle y_t - x_1, Y\rangle ⟨ G t − 1 , Y ⟩ ≤ − η 1 ⟨ y t − x 1 , Y ⟩ , and − 2 ⟨ y t − x 1 , Y ⟩ + ∥ y t − x 1 ∥ 2 − ∥ y t + 1 − x 1 ∥ 2 = − ∥ Y ∥ 2 -2\langle y_t - x_1, Y\rangle+ \lVert y_t - x_1\rVert^2 - \lVert y_{t+1} - x_1\rVert^2 = -\lVert Y\rVert^2 − 2 ⟨ y t − x 1 , Y ⟩ + ∥ y t − x 1 ∥ 2 − ∥ y t + 1 − x 1 ∥ 2 = − ∥ Y ∥ 2 . Hence
ϕ t − ϕ t − 1 ≤ ⟨ g t , P ⟩ − 1 2 η ∥ Y ∥ 2 + 1 6 η ( ∥ x t + 1 − y t + 1 ∥ 2 − ∥ P − Y ∥ 2 ) . (13) \phi_t - \phi_{t-1} \le\langle g_t, P\rangle- \frac{1}{2\eta}\lVert Y\rVert^2 + \frac{1}{6\eta}\left(\lVert x_{t+1} - y_{t+1}\rVert^2 - \lVert P - Y\rVert^2\right). \tag*{(13)} ϕ t − ϕ t − 1 ≤ ⟨ g t , P ⟩ − 2 η 1 ∥ Y ∥ 2 + 6 η 1 ( ∥ x t + 1 − y t + 1 ∥ 2 − ∥ P − Y ∥ 2 ) . (13)
Step 2: m m m Frank–Wolfe steps contract the distance to y t + 1 y_{t+1} y t + 1 . Fix k ∈ { 1 , … , m } k \in\{1,\ldots,m\} k ∈ { 1 , … , m } . The optimality of v t , k v_{t,k} v t , k for the linear function ⟨ η G t + z t , k − 1 − x 1 , ⋅ ⟩ \langle\eta G_t + z_{t,k-1} - x_1, \cdot\rangle ⟨ η G t + z t , k − 1 − x 1 , ⋅ ⟩ and that of y t + 1 y_{t+1} y t + 1 give
⟨ G t , v t , k − y t + 1 ⟩ + 1 η ⟨ z t , k − 1 − x 1 , v t , k − y t + 1 ⟩ ≤ 0 , ⟨ G t , y t + 1 − v t , k ⟩ + 1 η ⟨ y t + 1 − x 1 , y t + 1 − v t , k ⟩ ≤ 0 , \langle G_t, v_{t,k} - y_{t+1}\rangle+ \frac{1}{\eta}\langle z_{t,k-1} - x_1, v_{t,k} - y_{t+1}\rangle\le0,\qquad
\langle G_t, y_{t+1} - v_{t,k}\rangle+ \frac{1}{\eta}\langle y_{t+1} - x_1, y_{t+1} - v_{t,k}\rangle\le0, ⟨ G t , v t , k − y t + 1 ⟩ + η 1 ⟨ z t , k − 1 − x 1 , v t , k − y t + 1 ⟩ ≤ 0 , ⟨ G t , y t + 1 − v t , k ⟩ + η 1 ⟨ y t + 1 − x 1 , y t + 1 − v t , k ⟩ ≤ 0 ,
and adding them, ⟨ z t , k − 1 − y t + 1 , v t , k − y t + 1 ⟩ ≤ 0 \langle z_{t,k-1} - y_{t+1}, v_{t,k} - y_{t+1}\rangle\le0 ⟨ z t , k − 1 − y t + 1 , v t , k − y t + 1 ⟩ ≤ 0 . Since z t , k = ( 1 − σ ) z t , k − 1 + σ v t , k z_{t,k} = (1-\sigma)z_{t,k-1} + \sigma v_{t,k} z t , k = ( 1 − σ ) z t , k − 1 + σ v t , k ,
∥ z t , k − y t + 1 ∥ 2 = ( 1 − σ ) 2 ∥ z t , k − 1 − y t + 1 ∥ 2 + 2 σ ( 1 − σ ) ⟨ z t , k − 1 − y t + 1 , v t , k − y t + 1 ⟩ + σ 2 ∥ v t , k − y t + 1 ∥ 2 ≤ ( 1 − σ ) 2 ∥ z t , k − 1 − y t + 1 ∥ 2 − 2 σ 2 ⟨ z t , k − 1 − y t + 1 , v t , k − y t + 1 ⟩ + σ 2 ∥ v t , k − y t + 1 ∥ 2 = ( 1 − 2 σ ) ∥ z t , k − 1 − y t + 1 ∥ 2 + σ 2 ∥ z t , k − 1 − v t , k ∥ 2 , \begin{aligned}
\lVert z_{t,k} - y_{t+1}\rVert^2
&= (1-\sigma)^2\lVert z_{t,k-1} - y_{t+1}\rVert^2 + 2\sigma(1-\sigma)\langle z_{t,k-1} - y_{t+1}, v_{t,k} - y_{t+1}\rangle+ \sigma^2\lVert v_{t,k} - y_{t+1}\rVert^2 \\
&\le(1-\sigma)^2\lVert z_{t,k-1} - y_{t+1}\rVert^2 - 2\sigma^2\langle z_{t,k-1} - y_{t+1}, v_{t,k} - y_{t+1}\rangle+ \sigma^2\lVert v_{t,k} - y_{t+1}\rVert^2 \\
&= (1-2\sigma)\lVert z_{t,k-1} - y_{t+1}\rVert^2 + \sigma^2\lVert z_{t,k-1} - v_{t,k}\rVert^2,
\end{aligned} ∥ z t , k − y t + 1 ∥ 2 = ( 1 − σ ) 2 ∥ z t , k − 1 − y t + 1 ∥ 2 + 2 σ ( 1 − σ ) ⟨ z t , k − 1 − y t + 1 , v t , k − y t + 1 ⟩ + σ 2 ∥ v t , k − y t + 1 ∥ 2 ≤ ( 1 − σ ) 2 ∥ z t , k − 1 − y t + 1 ∥ 2 − 2 σ 2 ⟨ z t , k − 1 − y t + 1 , v t , k − y t + 1 ⟩ + σ 2 ∥ v t , k − y t + 1 ∥ 2 = ( 1 − 2 σ ) ∥ z t , k − 1 − y t + 1 ∥ 2 + σ 2 ∥ z t , k − 1 − v t , k ∥ 2 ,
where the inequality multiplies the nonpositive inner product by − 2 σ 2 -2\sigma^2 − 2 σ 2 instead of the larger coefficient 2 σ ( 1 − σ ) 2\sigma(1-\sigma) 2 σ ( 1 − σ ) .
For m = 1 m=1 m = 1 this is the computation of Weibel et al. [20 ] . Iterating it from z t , 0 = x t z_{t,0}=x_t z t , 0 = x t to z t , m = x t + 1 z_{t,m}=x_{t+1} z t , m = x t + 1 , which uses 1 − 2 σ ≥ 0 1-2\sigma\ge0 1 − 2 σ ≥ 0 ,
∥ x t + 1 − y t + 1 ∥ 2 ≤ ( 1 − u ) ∥ P ∥ 2 + σ 2 ∑ k = 1 m ( 1 − 2 σ ) m − k ∥ z t , k − 1 − v t , k ∥ 2 . (14) \lVert x_{t+1} - y_{t+1}\rVert^2 \le(1-u)\lVert P\rVert^2 + \sigma^2\sum_{k=1}^{m}(1-2\sigma)^{m-k}\lVert z_{t,k-1} - v_{t,k}\rVert^2. \tag*{(14)} ∥ x t + 1 − y t + 1 ∥ 2 ≤ ( 1 − u ) ∥ P ∥ 2 + σ 2 k = 1 ∑ m ( 1 − 2 σ ) m − k ∥ z t , k − 1 − v t , k ∥ 2 . (14)
Inserting (14) into (13) ,
ϕ t − ϕ t − 1 ≤ σ 2 6 η ∑ k = 1 m ( 1 − 2 σ ) m − k ∥ z t , k − 1 − v t , k ∥ 2 + ⟨ g t , P ⟩ − 1 2 η ∥ Y ∥ 2 + 1 6 η ( ( 1 − u ) ∥ P ∥ 2 − ∥ P − Y ∥ 2 ) . (15) \phi_t - \phi_{t-1} \le\frac{\sigma^2}{6\eta}\sum_{k=1}^{m}(1-2\sigma)^{m-k}\lVert z_{t,k-1} - v_{t,k}\rVert^2 + \langle g_t, P\rangle- \frac{1}{2\eta}\lVert Y\rVert^2 + \frac{1}{6\eta}\left((1-u)\lVert P\rVert^2 - \lVert P-Y\rVert^2\right). \tag*{(15)} ϕ t − ϕ t − 1 ≤ 6 η σ 2 k = 1 ∑ m ( 1 − 2 σ ) m − k ∥ z t , k − 1 − v t , k ∥ 2 + ⟨ g t , P ⟩ − 2 η 1 ∥ Y ∥ 2 + 6 η 1 ( ( 1 − u ) ∥ P ∥ 2 − ∥ P − Y ∥ 2 ) . (15)
Step 3: trading the gradient term for a quadratic form. Let λ ≥ 0 \lambda\ge0 λ ≥ 0 . The optimality of y t y_t y t and of y t + 1 y_{t+1} y t + 1 give ⟨ G t − 1 , Y ⟩ + 1 η ⟨ y t − x 1 , Y ⟩ ≤ 0 \langle G_{t-1},Y\rangle+ \frac{1}{\eta}\langle y_t-x_1,Y\rangle\le0 ⟨ G t − 1 , Y ⟩ + η 1 ⟨ y t − x 1 , Y ⟩ ≤ 0 and − ⟨ G t , Y ⟩ − 1 η ⟨ y t + 1 − x 1 , Y ⟩ ≤ 0 -\langle G_t,Y\rangle- \frac{1}{\eta}\langle y_{t+1}-x_1,Y\rangle\le0 − ⟨ G t , Y ⟩ − η 1 ⟨ y t + 1 − x 1 , Y ⟩ ≤ 0 ; adding them, − ⟨ g t , Y ⟩ ≤ − 1 η ∥ Y ∥ 2 -\langle g_t,Y\rangle\le-\frac{1}{\eta}\lVert Y\rVert^2 − ⟨ g t , Y ⟩ ≤ − η 1 ∥ Y ∥ 2 . Hence, by Young’s inequality ⟨ a , b ⟩ ≤ λ g ∥ a ∥ 2 + ∥ b ∥ 2 / ( 4 λ g ) \langle a,b\rangle\le\lambda^g\lVert a\rVert^2 + \lVert b\rVert^2/(4\lambda^g) ⟨ a , b ⟩ ≤ λ g ∥ a ∥ 2 + ∥ b ∥ 2 / ( 4 λ g ) ,
⟨ g t , P ⟩ = ⟨ g t , P + λ Y ⟩ − λ ⟨ g t , Y ⟩ ≤ λ g ∥ g t ∥ 2 + 1 4 λ g ∥ P + λ Y ∥ 2 − λ η ∥ Y ∥ 2 . \langle g_t,P\rangle= \langle g_t,P+\lambda Y\rangle- \lambda\langle g_t,Y\rangle\le\lambda^g\lVert g_t\rVert^2 + \frac{1}{4\lambda^g}\lVert P+\lambda Y\rVert^2 - \frac{\lambda}{\eta}\lVert Y\rVert^2. ⟨ g t , P ⟩ = ⟨ g t , P + λY ⟩ − λ ⟨ g t , Y ⟩ ≤ λ g ∥ g t ∥ 2 + 4 λ g 1 ∥ P + λY ∥ 2 − η λ ∥ Y ∥ 2 .
With (15) this gives
ϕ t − ϕ t − 1 ≤ λ g ∥ g t ∥ 2 + σ 2 6 η ∑ k = 1 m ( 1 − 2 σ ) m − k ∥ z t , k − 1 − v t , k ∥ 2 − Q ( λ ) , \phi_t - \phi_{t-1} \le\lambda^g\lVert g_t\rVert^2 + \frac{\sigma^2}{6\eta}\sum_{k=1}^{m}(1-2\sigma)^{m-k}\lVert z_{t,k-1} - v_{t,k}\rVert^2 - Q(\lambda), ϕ t − ϕ t − 1 ≤ λ g ∥ g t ∥ 2 + 6 η σ 2 k = 1 ∑ m ( 1 − 2 σ ) m − k ∥ z t , k − 1 − v t , k ∥ 2 − Q ( λ ) ,
Q ( λ ) : = 1 + 2 λ 2 η ∥ Y ∥ 2 − 1 4 λ g ∥ P + λ Y ∥ 2 − 1 6 η ( ( 1 − u ) ∥ P ∥ 2 − ∥ P − Y ∥ 2 ) , Q(\lambda) := \frac{1+2\lambda}{2\eta}\lVert Y\rVert^2 - \frac{1}{4\lambda^g}\lVert P+\lambda Y\rVert^2 - \frac{1}{6\eta}\left((1-u)\lVert P\rVert^2 - \lVert P-Y\rVert^2\right), Q ( λ ) := 2 η 1 + 2 λ ∥ Y ∥ 2 − 4 λ g 1 ∥ P + λY ∥ 2 − 6 η 1 ( ( 1 − u ) ∥ P ∥ 2 − ∥ P − Y ∥ 2 ) ,
and it remains to find λ ≥ 0 \lambda\ge0 λ ≥ 0 with Q ( λ ) ≥ 0 Q(\lambda)\ge0 Q ( λ ) ≥ 0 .
Step 4: Q ( λ ) Q(\lambda) Q ( λ ) is a sum of squares. Expanding the squares,
Q ( λ ) = ( 2 + 3 λ 3 η − λ 2 4 λ g ) ∥ Y ∥ 2 + ( u 6 η − 1 4 λ g ) ∥ P ∥ 2 − 2 ( 1 6 η + λ 4 λ g ) ⟨ P , Y ⟩ . Q(\lambda) = \left(\frac{2+3\lambda}{3\eta} - \frac{\lambda^2}{4\lambda^g}\right)\lVert Y\rVert^2 + \left(\frac{u}{6\eta} - \frac{1}{4\lambda^g}\right)\lVert P\rVert^2 - 2\left(\frac{1}{6\eta} + \frac{\lambda}{4\lambda^g}\right)\langle P,Y\rangle. Q ( λ ) = ( 3 η 2 + 3 λ − 4 λ g λ 2 ) ∥ Y ∥ 2 + ( 6 η u − 4 λ g 1 ) ∥ P ∥ 2 − 2 ( 6 η 1 + 4 λ g λ ) ⟨ P , Y ⟩ .
Take λ = 4 / ( 3 u ) \lambda= 4/(3u) λ = 4/ ( 3 u ) . With λ g = 8 η / ( 3 u ) \lambda^{\mathrm{g}} = 8\eta/(3u) λ g = 8 η / ( 3 u ) , that is 1 / ( 4 λ g ) = 3 u / ( 32 η ) 1/(4\lambda^{\mathrm{g}}) = 3u/(32\eta) 1/ ( 4 λ g ) = 3 u / ( 32 η ) , the three coefficients are 4 u + 7 6 u η \frac{4u+7}{6u\eta} 6 u η 4 u + 7 , 7 u 96 η \frac{7u}{96\eta} 96 η 7 u and 7 24 η \frac{7}{24\eta} 24 η 7 , and
Q ( λ ) = 7 96 η u ∥ u P − 4 Y ∥ 2 + 2 3 η ∥ Y ∥ 2 ≥ 0 , Q(\lambda) = \frac{7}{96\eta u}\lVert uP-4Y\rVert^{2}+\frac{2}{3\eta}\lVert Y\rVert^{2}\ge0, Q ( λ ) = 96 η u 7 ∥ u P − 4 Y ∥ 2 + 3 η 2 ∥ Y ∥ 2 ≥ 0 ,
since 7 96 η u ∥ u P − 4 Y ∥ 2 = 7 u 6 η ∥ P ∥ 2 − 7 12 η ⟨ P , Y ⟩ + 7 6 η u ∥ Y ∥ 2 \frac{7}{96\eta u}\lVert uP-4Y\rVert^{2}=\frac{7u}{6\eta}\lVert P\rVert^{2}-\frac{7}{12\eta}\langle P,Y\rangle+\frac{7}{6\eta u}\lVert Y\rVert^{2} 96 η u 7 ∥ u P − 4 Y ∥ 2 = 6 η 7 u ∥ P ∥ 2 − 12 η 7 ⟨ P , Y ⟩ + 6 η u 7 ∥ Y ∥ 2 and 4 u + 7 6 u η − 7 6 u η = 2 3 η \frac{4u+7}{6u\eta}-\frac{7}{6u\eta}=\frac{2}{3\eta} 6 u η 4 u + 7 − 6 u η 7 = 3 η 2 . This proves the lemma. For m = 1 m=1 m = 1 , Weibel et al. [20 ] show by a Schur-complement argument that Q ( λ ) ≥ 0 Q(\lambda)\ge0 Q ( λ ) ≥ 0 for a whole interval of values of λ \lambda λ ; the value λ = 4 / ( 3 u ) \lambda=4/(3u) λ = 4/ ( 3 u ) belongs to it and makes the sum of squares explicit.
Theorem C.2 (Potential-function bound). Let σ ∈ ( 0 , 1 2 ] \sigma\in(0,\frac{1}{2}] σ ∈ ( 0 , 2 1 ] , η > 0 \eta>0 η > 0 and u , λ g u,\lambda^{\mathrm{g}} u , λ g as in (7) . For every x ⋆ ∈ K x_{\star}\in K x ⋆ ∈ K ,
Reg T ≤ λ g ∑ t = 1 T ∥ g t ∥ 2 + σ 2 6 η ∑ t = 1 T ∑ k = 1 m ( 1 − 2 σ ) m − k ∥ z t , k − 1 − v t , k ∥ 2 + 1 2 η ∥ x ⋆ − x 1 ∥ 2 ≤ 8 η L 2 T 3 u + σ u D 2 T 12 η + D 2 2 η . \operatorname{Reg}_{T}\le\lambda^{\mathrm{g}}\sum_{t=1}^{T}\lVert g_{t}\rVert^{2}+\frac{\sigma^{2}}{6\eta}\sum_{t=1}^{T}\sum_{k=1}^{m}(1-2\sigma)^{m-k}\lVert z_{t,k-1}-v_{t,k}\rVert^{2}+\frac{1}{2\eta}\lVert x_{\star}-x_{1}\rVert^{2}\le\frac{8\eta L^{2}T}{3u}+\frac{\sigma uD^{2}T}{12\eta}+\frac{D^{2}}{2\eta}. Reg T ≤ λ g t = 1 ∑ T ∥ g t ∥ 2 + 6 η σ 2 t = 1 ∑ T k = 1 ∑ m ( 1 − 2 σ ) m − k ∥ z t , k − 1 − v t , k ∥ 2 + 2 η 1 ∥ x ⋆ − x 1 ∥ 2 ≤ 3 u 8 η L 2 T + 12 η σ u D 2 T + 2 η D 2 .
Proof. Summing Theorem C.1 over t = 1 , … , T t=1,\ldots,T t = 1 , … , T and using ϕ 0 = 0 \phi_{0}=0 ϕ 0 = 0 bounds ϕ T \phi_{T} ϕ T by the first two sums. Since y T + 1 y_{T+1} y T + 1 minimizes η ⟨ G T , ⋅ ⟩ + 1 2 ∥ ⋅ − x 1 ∥ 2 \eta\langle G_{T},\cdot\rangle+\frac{1}{2}\lVert\cdot-x_{1}\rVert^{2} η ⟨ G T , ⋅ ⟩ + 2 1 ∥ ⋅ − x 1 ∥ 2 over K K K , its value at y T + 1 y_{T+1} y T + 1 is at most its value at x ⋆ x_{\star} x ⋆ , which reads
∑ t = 1 T ⟨ g t , x t − x ⋆ ⟩ − ∥ x ⋆ − x 1 ∥ 2 2 η ≤ ∑ t = 1 T ⟨ g t , x t − y T + 1 ⟩ − ∥ y T + 1 − x 1 ∥ 2 2 η ≤ ϕ T , \sum_{t=1}^{T}\langle g_{t},x_{t}-x_{\star}\rangle-\frac{\lVert x_{\star}-x_{1}\rVert^{2}}{2\eta}\le\sum_{t=1}^{T}\langle g_{t},x_{t}-y_{T+1}\rangle-\frac{\lVert y_{T+1}-x_{1}\rVert^{2}}{2\eta}\le\phi_{T}, t = 1 ∑ T ⟨ g t , x t − x ⋆ ⟩ − 2 η ∥ x ⋆ − x 1 ∥ 2 ≤ t = 1 ∑ T ⟨ g t , x t − y T + 1 ⟩ − 2 η ∥ y T + 1 − x 1 ∥ 2 ≤ ϕ T ,
the last step dropping 1 6 η ∥ x T + 1 − y T + 1 ∥ 2 ≥ 0 \frac{1}{6\eta}\lVert x_{T+1}-y_{T+1}\rVert^{2}\ge0 6 η 1 ∥ x T + 1 − y T + 1 ∥ 2 ≥ 0 . This is the first bound. For the second, ∥ g t ∥ ≤ L \lVert g_{t}\rVert\le L ∥ g t ∥ ≤ L , ∥ z t , k − 1 − v t , k ∥ ≤ D \lVert z_{t,k-1}-v_{t,k}\rVert\le D ∥ z t , k − 1 − v t , k ∥ ≤ D , ∥ x ⋆ − x 1 ∥ ≤ D \lVert x_{\star}-x_{1}\rVert\le D ∥ x ⋆ − x 1 ∥ ≤ D , and
σ 2 ∑ k = 1 m ( 1 − 2 σ ) m − k = σ 2 1 − ( 1 − 2 σ ) m 2 σ = σ u 2 . \sigma^{2}\sum_{k=1}^{m}(1-2\sigma)^{m-k}=\sigma^{2}\frac{1-(1-2\sigma)^{m}}{2\sigma}=\frac{\sigma u}{2}. σ 2 k = 1 ∑ m ( 1 − 2 σ ) m − k = σ 2 2 σ 1 − ( 1 − 2 σ ) m = 2 σ u .
The tuning of Theorem 4.2 balances the three terms of Theorem C.2 . Two elementary facts about u = 1 − ( 1 − 2 σ ) m u=1-(1-2\sigma)^{m} u = 1 − ( 1 − 2 σ ) m are used: by Bernoulli’s inequality ( 1 − 2 σ ) m ≥ 1 − 2 m σ (1-2\sigma)^{m}\ge1-2m\sigma ( 1 − 2 σ ) m ≥ 1 − 2 mσ , so
u ≤ 2 m σ , (16) u\le2m\sigma, \tag*{(16)} u ≤ 2 mσ , (16)
and ( 1 − 2 σ ) m ≤ e − 2 m σ (1-2\sigma)^{m}\le e^{-2m\sigma} ( 1 − 2 σ ) m ≤ e − 2 mσ together with e ε ≥ 1 + ε e^{\varepsilon}\ge1+\varepsilon e ε ≥ 1 + ε gives, for ε : = 2 m σ \varepsilon:=2m\sigma ε := 2 mσ ,
u ≥ 1 − e − ε ≥ ε 1 + ε , that is 1 u ≤ 1 ε + 1. (17) u\ge1-e^{-\varepsilon}\ge\frac{\varepsilon}{1+\varepsilon},\qquad\text{that is}\qquad\frac{1}{u}\le\frac{1}{\varepsilon}+1. \tag*{(17)} u ≥ 1 − e − ε ≥ 1 + ε ε , that is u 1 ≤ ε 1 + 1. (17)
For m = 1 m=1 m = 1 both are equalities, u = 2 σ = ε u=2\sigma=\varepsilon u = 2 σ = ε .
Proof of Theorem 4.2. Here η = D 2 L ( 3 / T ) 3 / 4 m 1 / 4 \eta=\frac{D}{2L}(3/T)^{3/4}m^{1/4} η = 2 L D ( 3/ T ) 3/4 m 1/4 and σ = 3 / ( m T ) \sigma=\sqrt{3/(mT)} σ = 3/ ( m T ) , which is at most 1 2 \frac{1}{2} 2 1 because m T ≥ 12 mT\ge12 m T ≥ 12 ; note m σ 2 T = 3 m\sigma^{2}T=3 m σ 2 T = 3 and η L 2 T = L D 2 3 3 / 4 ( m T ) 1 / 4 \eta L^{2}T=\frac{LD}{2}3^{3/4}(mT)^{1/4} η L 2 T = 2 L D 3 3/4 ( m T ) 1/4 . We bound the three terms of Theorem C.2 . The regularizer term:
D 2 2 η = L D 3 3 / 4 T 3 / 4 m − 1 / 4 . \frac{D^{2}}{2\eta}=\frac{LD}{3^{3/4}}T^{3/4}m^{-1/4}. 2 η D 2 = 3 3/4 L D T 3/4 m − 1/4 .
The atom term: by (16) , σ u D 2 T 12 η ≤ m σ 2 D 2 T 6 η = D 2 2 η \frac{\sigma uD^{2}T}{12\eta}\le\frac{m\sigma^{2}D^{2}T}{6\eta}=\frac{D^{2}}{2\eta} 12 η σ u D 2 T ≤ 6 η m σ 2 D 2 T = 2 η D 2 . The gradient term: by (17) with ε = 2 m σ = 2 3 m / T \varepsilon=2m\sigma=2\sqrt{3m/T} ε = 2 mσ = 2 3 m / T ,
8 η L 2 T 3 u ≤ 8 η L 2 T 3 ε + 8 η L 2 T 3 = 4 η L 2 T 3 m σ + 8 η L 2 T 3 = 2 L D 3 3 / 4 T 3 / 4 m − 1 / 4 + 4 L D 3 1 / 4 ( m T ) 1 / 4 . \frac{8\eta L^{2}T}{3u}\le\frac{8\eta L^{2}T}{3\varepsilon}+\frac{8\eta L^{2}T}{3}=\frac{4\eta L^{2}T}{3m\sigma}+\frac{8\eta L^{2}T}{3}=\frac{2LD}{3^{3/4}}T^{3/4}m^{-1/4}+\frac{4LD}{3^{1/4}}(mT)^{1/4}. 3 u 8 η L 2 T ≤ 3 ε 8 η L 2 T + 3 8 η L 2 T = 3 mσ 4 η L 2 T + 3 8 η L 2 T = 3 3/4 2 L D T 3/4 m − 1/4 + 3 1/4 4 L D ( m T ) 1/4 .
Adding the three bounds gives the claim. For m = 1 m=1 m = 1 the two facts are equalities, so the atom term equals D 2 / ( 2 η ) D^{2}/(2\eta) D 2 / ( 2 η ) and the gradient term equals 2 L D 3 3 / 4 T 3 / 4 \frac{2LD}{3^{3/4}}T^{3/4} 3 3/4 2 L D T 3/4 : this is the bound of Weibel et al. [20 ] .
Anytime parameters
Let ( η t , σ t ) t ≥ 2 (\eta_{t},\sigma_{t})_{t\ge2} ( η t , σ t ) t ≥ 2 be parameters with η t \eta_{t} η t non-increasing and σ t ∈ ( 0 , 1 2 ] \sigma_{t}\in(0,\frac{1}{2}] σ t ∈ ( 0 , 2 1 ] . As in Weibel et al. [20 ] , round t t t uses the parameters of index t + 1 t+1 t + 1 :
dir t , k = η t + 1 G t + ( z t , k − 1 − x 1 ) , z t , k = ( 1 − σ t + 1 ) z t , k − 1 + σ t + 1 v t , k , y t + 1 = arg min y ∈ K η t + 1 ⟨ G t , y ⟩ + 1 2 ∥ y − x 1 ∥ 2 , \operatorname{dir}_{t,k}=\eta_{t+1}G_{t}+(z_{t,k-1}-x_{1}),\qquad z_{t,k}=(1-\sigma_{t+1})z_{t,k-1}+\sigma_{t+1}v_{t,k},\qquad y_{t+1}=\underset{y\in K}{\operatorname{arg\,min}}\ \eta_{t+1}\langle G_{t},y\rangle+\frac{1}{2}\lVert y-x_{1}\rVert^{2}, dir t , k = η t + 1 G t + ( z t , k − 1 − x 1 ) , z t , k = ( 1 − σ t + 1 ) z t , k − 1 + σ t + 1 v t , k , y t + 1 = y ∈ K arg min η t + 1 ⟨ G t , y ⟩ + 2 1 ∥ y − x 1 ∥ 2 ,
with y 1 = x 1 y_{1}=x_{1} y 1 = x 1 , and the potential is
ϕ t : = ∑ s = 1 t ⟨ g s , x s − y t + 1 ⟩ − 1 2 η t + 1 ∥ y t + 1 − x 1 ∥ 2 + 1 6 η t + 1 ∥ x t + 1 − y t + 1 ∥ 2 . \phi_{t}:=\sum_{s=1}^{t}\langle g_{s},x_{s}-y_{t+1}\rangle-\frac{1}{2\eta_{t+1}}\lVert y_{t+1}-x_{1}\rVert^{2}+\frac{1}{6\eta_{t+1}}\lVert x_{t+1}-y_{t+1}\rVert^{2}. ϕ t := s = 1 ∑ t ⟨ g s , x s − y t + 1 ⟩ − 2 η t + 1 1 ∥ y t + 1 − x 1 ∥ 2 + 6 η t + 1 1 ∥ x t + 1 − y t + 1 ∥ 2 .
For t ≥ 3 t \ge3 t ≥ 3 let
u t : = 1 − ( 1 − 2 σ t ) m , d t : = 1 − η t η t − 1 ∈ [ 0 , 1 ) , δ t : = d t u t , λ t g : = 8 η t 3 u t ⋅ 1 1 − 2 3 δ t ( when 3 2 δ t < 1 ) . u_t := 1 - (1 - 2\sigma_t)^m,\qquad d_t := 1 - \frac{\eta_t}{\eta_{t-1}} \in[0,1),\qquad\delta_t := \frac{d_t}{u_t},\qquad\lambda_t^g := \frac{8\eta_t}{3u_t}\cdot\frac{1}{1-\frac{2}{3}\delta_t}\qquad\left(\text{when }\frac{3}{2}\delta_t<1\right). u t := 1 − ( 1 − 2 σ t ) m , d t := 1 − η t − 1 η t ∈ [ 0 , 1 ) , δ t := u t d t , λ t g := 3 u t 8 η t ⋅ 1 − 3 2 δ t 1 ( when 2 3 δ t < 1 ) .
The number d t d_t d t measures the growth of the regularization weight from one round to the next, 1 η t − 1 η t − 1 = d t η t \frac{1}{\eta_t}-\frac{1}{\eta_{t-1}}=\frac{d_t}{\eta_t} η t 1 − η t − 1 1 = η t d t . With constant parameters d t = 0 d_t=0 d t = 0 and λ t g \lambda_t^g λ t g is the constant of Theorem C.1; the growing weight costs the factor ( 1 − 2 3 δ t ) − 1 (1-\frac{2}{3}\delta_t)^{-1} ( 1 − 3 2 δ t ) − 1 on the gradient term and nothing else.
Lemma C.3 (Per-round inequality, time-varying parameters). For every t ≥ 2 t\ge2 t ≥ 2 with 3 2 δ t + 1 < 1 \frac{3}{2}\delta_{t+1}<1 2 3 δ t + 1 < 1 ,
ϕ t − ϕ t − 1 ≤ λ t + 1 g ∥ g t ∥ 2 + σ t + 1 2 6 η t + 1 ∑ k = 1 m ( 1 − 2 σ t + 1 ) m − k ∥ z t , k − 1 − v t , k ∥ 2 . \phi_t-\phi_{t-1}\le\lambda_{t+1}^g\lVert g_t\rVert^2+\frac{\sigma_{t+1}^2}{6\eta_{t+1}}\sum_{k=1}^{m}(1-2\sigma_{t+1})^{m-k}\lVert z_{t,k-1}-v_{t,k}\rVert^2. ϕ t − ϕ t − 1 ≤ λ t + 1 g ∥ g t ∥ 2 + 6 η t + 1 σ t + 1 2 k = 1 ∑ m ( 1 − 2 σ t + 1 ) m − k ∥ z t , k − 1 − v t , k ∥ 2 .
Proof. Fix t ≥ 2 t\ge2 t ≥ 2 and abbreviate u : = u t + 1 u:=u_{t+1} u := u t + 1 , σ : = σ t + 1 \sigma:=\sigma_{t+1} σ := σ t + 1 , δ : = δ t + 1 \delta:=\delta_{t+1} δ := δ t + 1 , λ : = 4 / ( 3 u ) \lambda:=4/(3u) λ := 4/ ( 3 u ) and λ g : = λ t + 1 g \lambda^g:=\lambda^g_{t+1} λ g := λ t + 1 g . Let Δ : = 1 η t + 1 − 1 η t = δ u η t + 1 ≥ 0 \Delta:=\frac{1}{\eta_{t+1}}-\frac{1}{\eta_t}=\frac{\delta u}{\eta_{t+1}}\ge0 Δ := η t + 1 1 − η t 1 = η t + 1 δ u ≥ 0 , and write P : = x t − y t + 1 P:=x_t-y_{t+1} P := x t − y t + 1 , Y : = y t − y t + 1 Y:=y_t-y_{t+1} Y := y t − y t + 1 and r : = y t + 1 − x 1 r:=y_{t+1}-x_1 r := y t + 1 − x 1 , so that x t − y t = P − Y x_t-y_t=P-Y x t − y t = P − Y and y t − x 1 = Y + r y_t-x_1=Y+r y t − x 1 = Y + r . We follow the proof of Theorem C.1; the difference between η t \eta_t η t and η t + 1 \eta_{t+1} η t + 1 creates the extra terms carrying Δ \Delta Δ .
Step 1. Now
ϕ t − ϕ t − 1 = ⟨ g t , P ⟩ + ⟨ G t − 1 , Y ⟩ + ∥ x t + 1 − y t + 1 ∥ 2 6 η t + 1 − ∥ P − Y ∥ 2 6 η t + ∥ Y + r ∥ 2 2 η t − ∥ r ∥ 2 2 η t + 1 . \phi_t-\phi_{t-1}=\langle g_t,P\rangle+\langle G_{t-1},Y\rangle+\frac{\lVert x_{t+1}-y_{t+1}\rVert^2}{6\eta_{t+1}}-\frac{\lVert P-Y\rVert^2}{6\eta_t}+\frac{\lVert Y+r\rVert^2}{2\eta_t}-\frac{\lVert r\rVert^2}{2\eta_{t+1}}. ϕ t − ϕ t − 1 = ⟨ g t , P ⟩ + ⟨ G t − 1 , Y ⟩ + 6 η t + 1 ∥ x t + 1 − y t + 1 ∥ 2 − 6 η t ∥ P − Y ∥ 2 + 2 η t ∥ Y + r ∥ 2 − 2 η t + 1 ∥ r ∥ 2 .
The optimality of y t y_t y t (weight η t \eta_t η t ) gives ⟨ G t − 1 , Y ⟩ ≤ − 1 η t ⟨ Y + r , Y ⟩ \langle G_{t-1},Y\rangle\le-\frac{1}{\eta_t}\langle Y+r,Y\rangle ⟨ G t − 1 , Y ⟩ ≤ − η t 1 ⟨ Y + r , Y ⟩ , and ∥ Y + r ∥ 2 2 η t − ⟨ Y + r , Y ⟩ η t = ∥ r ∥ 2 − ∥ Y ∥ 2 2 η t \frac{\lVert Y+r\rVert^2}{2\eta_t}-\frac{\langle Y+r,Y\rangle}{\eta_t}=\frac{\lVert r\rVert^2-\lVert Y\rVert^2}{2\eta_t} 2 η t ∥ Y + r ∥ 2 − η t ⟨ Y + r , Y ⟩ = 2 η t ∥ r ∥ 2 − ∥ Y ∥ 2 . Hence
ϕ t − ϕ t − 1 ≤ ⟨ g t , P ⟩ − ∥ Y ∥ 2 2 η t − Δ 2 ∥ r ∥ 2 + ∥ x t + 1 − y t + 1 ∥ 2 6 η t + 1 − ∥ P − Y ∥ 2 6 η t ; (18) \phi_t-\phi_{t-1}\le\langle g_t,P\rangle-\frac{\lVert Y\rVert^2}{2\eta_t}-\frac{\Delta}{2}\lVert r\rVert^2+\frac{\lVert x_{t+1}-y_{t+1}\rVert^2}{6\eta_{t+1}}-\frac{\lVert P-Y\rVert^2}{6\eta_t}; \tag*{(18)} ϕ t − ϕ t − 1 ≤ ⟨ g t , P ⟩ − 2 η t ∥ Y ∥ 2 − 2 Δ ∥ r ∥ 2 + 6 η t + 1 ∥ x t + 1 − y t + 1 ∥ 2 − 6 η t ∥ P − Y ∥ 2 ; (18)
the growing weight produces the free nonpositive term − Δ 2 ∥ r ∥ 2 -\frac{\Delta}{2}\lVert r\rVert^2 − 2 Δ ∥ r ∥ 2 .
Step 2. Step 2 of Theorem C.1 applies verbatim with ( η , σ ) = ( η t + 1 , σ t + 1 ) (\eta,\sigma)=(\eta_{t+1},\sigma_{t+1}) ( η , σ ) = ( η t + 1 , σ t + 1 ) , since both optimality relations used there carry the weight η t + 1 \eta_{t+1} η t + 1 :
∥ x t + 1 − y t + 1 ∥ 2 ≤ ( 1 − u ) ∥ P ∥ 2 + σ 2 ∑ k ( 1 − 2 σ ) m − k ∥ z t , k − 1 − v t , k ∥ 2 . \lVert x_{t+1}-y_{t+1}\rVert^2\le(1-u)\lVert P\rVert^2+\sigma^2\sum_k(1-2\sigma)^{m-k}\lVert z_{t,k-1}-v_{t,k}\rVert^2. ∥ x t + 1 − y t + 1 ∥ 2 ≤ ( 1 − u ) ∥ P ∥ 2 + σ 2 k ∑ ( 1 − 2 σ ) m − k ∥ z t , k − 1 − v t , k ∥ 2 .
Step 3. The optimality of y t y_t y t (weight η t \eta_t η t ) and of y t + 1 y_{t+1} y t + 1 (weight η t + 1 \eta_{t+1} η t + 1 ) give ⟨ G t − 1 , Y ⟩ + 1 η t ⟨ Y + r , Y ⟩ ≤ 0 \langle G_{t-1},Y\rangle+\frac{1}{\eta_t}\langle Y+r,Y\rangle\le0 ⟨ G t − 1 , Y ⟩ + η t 1 ⟨ Y + r , Y ⟩ ≤ 0 and − ⟨ G t , Y ⟩ − 1 η t + 1 ⟨ r , Y ⟩ ≤ 0 -\langle G_t,Y\rangle-\frac{1}{\eta_{t+1}}\langle r,Y\rangle\le0 − ⟨ G t , Y ⟩ − η t + 1 1 ⟨ r , Y ⟩ ≤ 0 ; adding them, − ⟨ g t , Y ⟩ ≤ − 1 η t ∥ Y ∥ 2 + Δ ⟨ r , Y ⟩ -\langle g_t,Y\rangle\le-\frac{1}{\eta_t}\lVert Y\rVert^2+\Delta\langle r,Y\rangle − ⟨ g t , Y ⟩ ≤ − η t 1 ∥ Y ∥ 2 + Δ ⟨ r , Y ⟩ . Young’s inequality then gives
⟨ g t , P ⟩ = ⟨ g t , P + λ Y ⟩ − λ ⟨ g t , Y ⟩ ≤ λ g ∥ g t ∥ 2 + ∥ P + λ Y ∥ 2 4 λ g − λ η t ∥ Y ∥ 2 + λ Δ ⟨ r , Y ⟩ . \langle g_t,P\rangle=\langle g_t,P+\lambda Y\rangle-\lambda\langle g_t,Y\rangle\le\lambda^g\lVert g_t\rVert^2+\frac{\lVert P+\lambda Y\rVert^2}{4\lambda^g}-\frac{\lambda}{\eta_t}\lVert Y\rVert^2+\lambda\Delta\langle r,Y\rangle. ⟨ g t , P ⟩ = ⟨ g t , P + λY ⟩ − λ ⟨ g t , Y ⟩ ≤ λ g ∥ g t ∥ 2 + 4 λ g ∥ P + λY ∥ 2 − η t λ ∥ Y ∥ 2 + λ Δ ⟨ r , Y ⟩ .
The drift term is absorbed by the free term of Step 1, as in Weibel et al. [20 ] :
− Δ 2 ∥ r ∥ 2 + λ Δ ⟨ r , Y ⟩ = − Δ 2 ∥ r − λ Y ∥ 2 + λ 2 Δ 2 ∥ Y ∥ 2 ≤ λ 2 Δ 2 ∥ Y ∥ 2 . -\frac{\Delta}{2}\lVert r\rVert^2+\lambda\Delta\langle r,Y\rangle=-\frac{\Delta}{2}\lVert r-\lambda Y\rVert^2+\frac{\lambda^2\Delta}{2}\lVert Y\rVert^2\le\frac{\lambda^2\Delta}{2}\lVert Y\rVert^2. − 2 Δ ∥ r ∥ 2 + λ Δ ⟨ r , Y ⟩ = − 2 Δ ∥ r − λY ∥ 2 + 2 λ 2 Δ ∥ Y ∥ 2 ≤ 2 λ 2 Δ ∥ Y ∥ 2 .
Combining with Steps 1 and 2,
ϕ t − ϕ t − 1 ≤ λ g ∥ g t ∥ 2 + σ 2 6 η t + 1 ∑ k ( 1 − 2 σ ) m − k ∥ z t , k − 1 − v t , k ∥ 2 − Q ~ ( λ ) , \phi_t-\phi_{t-1}\le\lambda^g\lVert g_t\rVert^2+\frac{\sigma^2}{6\eta_{t+1}}\sum_k(1-2\sigma)^{m-k}\lVert z_{t,k-1}-v_{t,k}\rVert^2-\widetilde Q(\lambda), ϕ t − ϕ t − 1 ≤ λ g ∥ g t ∥ 2 + 6 η t + 1 σ 2 k ∑ ( 1 − 2 σ ) m − k ∥ z t , k − 1 − v t , k ∥ 2 − Q ( λ ) ,
Q ~ ( λ ) : = ( 1 + 2 λ 2 η t − λ 2 Δ 2 ) ∥ Y ∥ 2 − ∥ P + λ Y ∥ 2 4 λ g − 1 − u 6 η t + 1 ∥ P ∥ 2 + ∥ P − Y ∥ 2 6 η t , \widetilde Q(\lambda):=\left(\frac{1+2\lambda}{2\eta_t}-\frac{\lambda^2\Delta}{2}\right)\lVert Y\rVert^2-\frac{\lVert P+\lambda Y\rVert^2}{4\lambda^g}-\frac{1-u}{6\eta_{t+1}}\lVert P\rVert^2+\frac{\lVert P-Y\rVert^2}{6\eta_t}, Q ( λ ) := ( 2 η t 1 + 2 λ − 2 λ 2 Δ ) ∥ Y ∥ 2 − 4 λ g ∥ P + λY ∥ 2 − 6 η t + 1 1 − u ∥ P ∥ 2 + 6 η t ∥ P − Y ∥ 2 ,
and it remains to show that Q ~ ( λ ) ≥ 0 \widetilde Q(\lambda)\ge0 Q ( λ ) ≥ 0 .
Step 4: comparison with the constant-parameter form. For a symmetric 2 × 2 2\times2 2 × 2 matrix M M M write q M : = M 11 ∥ Y ∥ 2 + 2 M 12 ⟨ Y , P ⟩ + M 22 ∥ P ∥ 2 q_M:=M_{11}\lVert Y\rVert^2+2M_{12}\langle Y,P\rangle+M_{22}\lVert P\rVert^2 q M := M 11 ∥ Y ∥ 2 + 2 M 12 ⟨ Y , P ⟩ + M 22 ∥ P ∥ 2 ; if M M M is positive semidefinite then q M ≥ 0 q_M\ge0 q M ≥ 0 . Substituting 1 η t = 1 η t + 1 − Δ \frac{1}{\eta_t}=\frac{1}{\eta_{t+1}}-\Delta η t 1 = η t + 1 1 − Δ and expanding, Q ~ ( λ ) = q N − ∥ P + λ Y ∥ 2 4 λ g \widetilde Q(\lambda)=q_N-\frac{\lVert P+\lambda Y\rVert^2}{4\lambda^g} Q ( λ ) = q N − 4 λ g ∥ P + λY ∥ 2 with
N = 1 η t + 1 N ~ − Δ E , N ~ = ( 2 3 + λ − 1 6 − 1 6 − 1 6 u 6 ) , E = ( 2 3 + λ + λ 2 2 − 1 6 − 1 6 1 6 ) . N=\frac{1}{\eta_{t+1}}\widetilde N-\Delta E,\qquad\widetilde N=\begin{pmatrix}\frac{2}{3}+\lambda-\frac{1}{6} & -\frac{1}{6}\\ -\frac{1}{6} & \frac{u}{6}\end{pmatrix},\qquad E=\begin{pmatrix}\frac{2}{3}+\lambda+\frac{\lambda^2}{2} & -\frac{1}{6}\\ -\frac{1}{6} & \frac{1}{6}\end{pmatrix}. N = η t + 1 1 N − Δ E , N = ( 3 2 + λ − 6 1 − 6 1 − 6 1 6 u ) , E = ( 3 2 + λ + 2 λ 2 − 6 1 − 6 1 6 1 ) .
For constant parameters (Δ = 0 \Delta= 0 Δ = 0 ) and λ g = 8 η t + 1 / ( 3 u ) \lambda^{g} = 8\eta_{t+1}/(3u) λ g = 8 η t + 1 / ( 3 u ) , the expression q ~ N / η t + 1 − ∥ P + λ Y ∥ 2 / ( 4 λ g ) \widetilde{q}_{N}/\eta_{t+1} - \lVert P+\lambda Y\rVert^{2}/(4\lambda^{g}) q N / η t + 1 − ∥ P + λY ∥ 2 / ( 4 λ g ) is exactly the form Q ( λ ) Q(\lambda) Q ( λ ) of Step 4 of Theorem C.1, which is a sum of squares. We compare the two forms. Since Δ = δ u / η t + 1 \Delta= \delta u/\eta_{t+1} Δ = δ u / η t + 1 ,
N − ( 1 − 3 2 δ ) N ~ η t + 1 = δ u η t + 1 ( 3 2 u N ~ − E ) , 3 2 u N ~ − E = ( 10 − 3 u − 6 u 2 9 u 2 2 u − 3 12 u 2 u − 3 12 u 1 12 ) . N-\left(1-\frac{3}{2}\delta\right)\frac{\widetilde{N}}{\eta_{t+1}}=\frac{\delta u}{\eta_{t+1}}\left(\frac{3}{2u}\widetilde{N}-E\right),\qquad\frac{3}{2u}\widetilde{N}-E=\begin{pmatrix}\dfrac{10-3u-6u^{2}}{9u^{2}} & \dfrac{2u-3}{12u}\\[4pt]\dfrac{2u-3}{12u} & \dfrac{1}{12}\end{pmatrix}. N − ( 1 − 2 3 δ ) η t + 1 N = η t + 1 δ u ( 2 u 3 N − E ) , 2 u 3 N − E = 9 u 2 10 − 3 u − 6 u 2 12 u 2 u − 3 12 u 2 u − 3 12 1 .
for λ = 4 / ( 3 u ) \lambda= 4/(3u) λ = 4/ ( 3 u ) , and the last matrix is positive semidefinite: its lower-right entry is positive and its determinant 13 + 24 u − 36 u 2 432 u 2 \frac{13+24u-36u^{2}}{432u^{2}} 432 u 2 13 + 24 u − 36 u 2 is positive for 0 < u ≤ 1 0 < u \le1 0 < u ≤ 1 , the numerator being concave with values 13 and 1 at u = 0 u = 0 u = 0 and u = 1 u = 1 u = 1 . Hence q N ≥ ( 1 − 3 2 δ ) q ~ N / η t + 1 q_N \ge(1-\frac{3}{2}\delta)\widetilde{q}_N/\eta_{t+1} q N ≥ ( 1 − 2 3 δ ) q N / η t + 1 . Finally λ g = λ t + 1 g \lambda^{g}=\lambda^{g}_{t+1} λ g = λ t + 1 g was defined so that ( 1 − 3 2 δ ) / ( 4 λ g ) = 1 / ( 4 ⋅ 8 η t + 1 / ( 3 u ) ) (1-\frac{3}{2}\delta)/(4\lambda^{g})=1/(4\cdot8\eta_{t+1}/(3u)) ( 1 − 2 3 δ ) / ( 4 λ g ) = 1/ ( 4 ⋅ 8 η t + 1 / ( 3 u )) , and therefore
Q ~ ( λ ) ≥ ( 1 − 3 2 δ ) [ q ~ N η t + 1 − 3 u 32 η t + 1 ∥ P + λ Y ∥ 2 ] = ( 1 − 3 2 δ ) Q ( λ ) ≥ 0 , \widetilde{Q}(\lambda)\ge\left(1-\frac{3}{2}\delta\right)\left[\frac{\widetilde{q}_N}{\eta_{t+1}}-\frac{3u}{32\eta_{t+1}}\lVert P+\lambda Y\rVert^{2}\right]=\left(1-\frac{3}{2}\delta\right)Q(\lambda)\ge0, Q ( λ ) ≥ ( 1 − 2 3 δ ) [ η t + 1 q N − 32 η t + 1 3 u ∥ P + λY ∥ 2 ] = ( 1 − 2 3 δ ) Q ( λ ) ≥ 0 ,
where Q ( λ ) Q(\lambda) Q ( λ ) is the form of Theorem C.1 for the parameters ( η t + 1 , σ t + 1 ) (\eta_{t+1},\sigma_{t+1}) ( η t + 1 , σ t + 1 ) .
Theorem C.4 (Anytime potential bound). Let ( η t ) t ≥ 2 (\eta_t)_{t\ge2} ( η t ) t ≥ 2 be non-increasing, σ t ∈ ( 0 , 1 2 ] \sigma_t\in(0,\frac{1}{2}] σ t ∈ ( 0 , 2 1 ] , and 3 2 δ t < 1 \frac{3}{2}\delta_t<1 2 3 δ t < 1 for 3 ≤ t ≤ T + 1 3\le t\le T+1 3 ≤ t ≤ T + 1 . Then for every T ≥ 1 T\ge1 T ≥ 1 and x ⋆ ∈ K x_\star\in K x ⋆ ∈ K ,
Reg T ≤ ∥ x ⋆ − x 1 ∥ 2 2 η T + 1 + ϕ 1 + ∑ t = 2 T λ t + 1 g ∥ g t ∥ 2 + ∑ t = 2 T σ t + 1 2 6 η t + 1 ∑ k = 1 m ( 1 − 2 σ t + 1 ) m − k ∥ z t , k − 1 − v t , k ∥ 2 \operatorname{Reg}_T\le\frac{\lVert x_\star-x_1\rVert^2}{2\eta_{T+1}}+\phi_1+\sum_{t=2}^{T}\lambda^{g}_{t+1}\lVert g_t\rVert^2+\sum_{t=2}^{T}\frac{\sigma_{t+1}^{2}}{6\eta_{t+1}}\sum_{k=1}^{m}(1-2\sigma_{t+1})^{m-k}\lVert z_{t,k-1}-v_{t,k}\rVert^2 Reg T ≤ 2 η T + 1 ∥ x ⋆ − x 1 ∥ 2 + ϕ 1 + t = 2 ∑ T λ t + 1 g ∥ g t ∥ 2 + t = 2 ∑ T 6 η t + 1 σ t + 1 2 k = 1 ∑ m ( 1 − 2 σ t + 1 ) m − k ∥ z t , k − 1 − v t , k ∥ 2 ≤ D 2 2 η T + 1 + L D 6 η 2 + D 2 ∑ t = 2 T λ t + 1 g + L 2 ∑ t = 2 T η t + 1 g + D 2 ∑ t = 2 T σ t + 1 u t + 1 12 η t + 1 . \le\frac{D^2}{2\eta_{T+1}}+\frac{LD}{6\eta_2}+D^2\sum_{t=2}^{T}\lambda^{g}_{t+1}+L^2\sum_{t=2}^{T}\eta^{g}_{t+1}+D^2\sum_{t=2}^{T}\frac{\sigma_{t+1}u_{t+1}}{12\eta_{t+1}}. ≤ 2 η T + 1 D 2 + 6 η 2 L D + D 2 t = 2 ∑ T λ t + 1 g + L 2 t = 2 ∑ T η t + 1 g + D 2 t = 2 ∑ T 12 η t + 1 σ t + 1 u t + 1 .
Proof. As for Theorem C.2 : ϕ T = ϕ 1 + ∑ t = 2 T ( ϕ t − ϕ t − 1 ) \phi_T=\phi_1+\sum_{t=2}^{T}(\phi_t-\phi_{t-1}) ϕ T = ϕ 1 + ∑ t = 2 T ( ϕ t − ϕ t − 1 ) , each increment is bounded by Theorem C.3, and the optimality of y T + 1 y_{T+1} y T + 1 (weight η T + 1 \eta_{T+1} η T + 1 ) gives ∑ t < T ⟨ g t , x t − x ⋆ ⟩ − ∥ x ⋆ − x 1 ∥ 2 2 η T + 1 ≤ ϕ T \sum_{t<T}\langle g_t,x_t-x_\star\rangle-\frac{\lVert x_\star-x_1\rVert^2}{2\eta_{T+1}}\le\phi_T ∑ t < T ⟨ g t , x t − x ⋆ ⟩ − 2 η T + 1 ∥ x ⋆ − x 1 ∥ 2 ≤ ϕ T . For the second line, ϕ 1 = ⟨ g 1 , x 1 − y 2 ⟩ − ∥ y 2 − x 1 ∥ 2 2 η 2 + ∥ x 2 − y 2 ∥ 2 6 η 2 ≤ L D + D 2 6 η 2 \phi_1=\langle g_1,x_1-y_2\rangle-\frac{\lVert y_2-x_1\rVert^2}{2\eta_2}+\frac{\lVert x_2-y_2\rVert^2}{6\eta_2}\le LD+\frac{D^2}{6\eta_2} ϕ 1 = ⟨ g 1 , x 1 − y 2 ⟩ − 2 η 2 ∥ y 2 − x 1 ∥ 2 + 6 η 2 ∥ x 2 − y 2 ∥ 2 ≤ L D + 6 η 2 D 2 , and the other terms are bounded as in Theorem C.2 .
Proof of Theorem 4.3 . Here η t = D 2 L ( 3 / t ) 3 / 4 m 1 / 4 \eta_t=\frac{D}{2L}(3/t)^{3/4}m^{1/4} η t = 2 L D ( 3/ t ) 3/4 m 1/4 and σ t = min { 1 2 , 3 / ( m t ) } \sigma_t=\min\{\frac{1}{2},\sqrt{3/(mt)}\} σ t = min { 2 1 , 3/ ( m t ) } , so that η t L 2 = L D 2 3 3 / 4 m 1 / 4 t − 3 / 4 \eta_tL^2=\frac{LD}{2}3^{3/4}m^{1/4}t^{-3/4} η t L 2 = 2 L D 3 3/4 m 1/4 t − 3/4 and D 2 2 t η t = L D 3 3 / 4 m − 1 / 4 t − 1 / 4 \frac{D^2}{2t\eta_t}=\frac{LD}{3^{3/4}}m^{-1/4}t^{-1/4} 2 t η t D 2 = 3 3/4 L D m − 1/4 t − 1/4 . Put ε ˉ t : = 2 3 m / t \bar{\varepsilon}_t:=2\sqrt{3m/t} ε ˉ t := 2 3 m / t , which equals 2 m σ t 2m\sigma_t 2 m σ t when σ t < 1 2 \sigma_t<\frac{1}{2} σ t < 2 1 . We first collect three facts, valid for every t ≥ 3 t\ge3 t ≥ 3 .
(c) 3 2 δ t ≤ 3 4 \frac{3}{2}\delta_t\le\frac{3}{4} 2 3 δ t ≤ 4 3 and 1 1 − 3 2 δ t ≤ 1 + 6 δ t \frac{1}{1-\frac{3}{2}\delta_t}\le1+6\delta_t 1 − 2 3 δ t 1 ≤ 1 + 6 δ t : indeed d t = 1 − ( ( t − 1 ) / t ) 3 / 4 ≤ 1 / t d_t=1-((t-1)/t)^{3/4}\le1/t d t = 1 − (( t − 1 ) / t ) 3/4 ≤ 1/ t because x 3 / 4 ≥ x x^{3/4}\ge x x 3/4 ≥ x on [ 0 , 1 ] [0,1] [ 0 , 1 ] , so by (b) δ t ≤ 1 t ( 1 + 1 ε ˉ t ) = 1 t + 1 2 3 m t − 1 / 2 ≤ 1 3 + 1 6 = 1 2 \delta_t\le\frac{1}{t}(1+\frac{1}{\bar{\varepsilon}_t})=\frac{1}{t}+\frac{1}{2\sqrt{3m}}\,t^{-1/2}\le\frac{1}{3}+\frac{1}{6}=\frac{1}{2} δ t ≤ t 1 ( 1 + ε ˉ t 1 ) = t 1 + 2 3 m 1 t − 1/2 ≤ 3 1 + 6 1 = 2 1 , and 1 1 − x ≤ 1 + 4 x \frac{1}{1-x}\le1+4x 1 − x 1 ≤ 1 + 4 x for 0 ≤ x ≤ 3 4 0\le x\le\frac{3}{4} 0 ≤ x ≤ 4 3 .
By (c), Theorem C.4 applies. We bound its terms, with all sums over the parameter index t ∈ { 3 , … , T + 1 } t\in\{3,\ldots,T+1\} t ∈ { 3 , … , T + 1 } (the round being t − 1 t-1 t − 1 ) and the elementary estimates ∑ t − 1 / 4 ≤ ∫ 2 T + 1 x − 1 / 4 d x ≤ 4 3 ( T + 1 ) 3 / 4 \sum t^{-1/4}\le\int_{2}^{T+1}x^{-1/4}dx\le\frac{4}{3}(T+1)^{3/4} ∑ t − 1/4 ≤ ∫ 2 T + 1 x − 1/4 d x ≤ 3 4 ( T + 1 ) 3/4 , ∑ t − 3 / 4 ≤ 4 ( T + 1 ) 1 / 4 \sum t^{-3/4}\le4(T+1)^{1/4} ∑ t − 3/4 ≤ 4 ( T + 1 ) 1/4 and ∑ t ≥ 3 t − 7 / 4 ≤ 3 − 7 / 4 + ∫ 3 ∞ x − 7 / 4 d x ≤ 0.74 \sum_{t\ge3}t^{-7/4}\le3^{-7/4}+\int_{3}^{\infty}x^{-7/4}dx\le0.74 ∑ t ≥ 3 t − 7/4 ≤ 3 − 7/4 + ∫ 3 ∞ x − 7/4 d x ≤ 0.74 .
The regularizer term is D 2 2 η T + 1 = L D 3 3 / 4 m − 1 / 4 ( T + 1 ) 3 / 4 \frac{D^2}{2\eta_{T+1}}=\frac{LD}{3^{3/4}}m^{-1/4}(T+1)^{3/4} 2 η T + 1 D 2 = 3 3/4 L D m − 1/4 ( T + 1 ) 3/4 , and L D + D 2 6 η 2 = L D ( 1 + 1 3 ( 2 3 ) 3 / 4 m − 1 / 4 ) ≤ 1.25 L D LD+\frac{D^2}{6\eta_2}=LD(1+\frac{1}{3}(\frac{2}{3})^{3/4}m^{-1/4})\le1.25LD L D + 6 η 2 D 2 = L D ( 1 + 3 1 ( 3 2 ) 3/4 m − 1/4 ) ≤ 1.25 L D .
By (a), the atom term of round t − 1 t-1 t − 1 is at most D 2 2 t η t = L D 3 3 / 4 m − 1 / 4 t − 1 / 4 \frac{D^2}{2t\eta_t}=\frac{LD}{3^{3/4}}m^{-1/4}t^{-1/4} 2 t η t D 2 = 3 3/4 L D m − 1/4 t − 1/4 , and the sum is at most 4 3 L D 3 3 / 4 m − 1 / 4 ( T + 1 ) 3 / 4 \frac{4}{3}\frac{LD}{3^{3/4}}m^{-1/4}(T+1)^{3/4} 3 4 3 3/4 L D m − 1/4 ( T + 1 ) 3/4 .
By (b) and (c), λ t g ≤ 8 η t 3 u t + 8 η t 3 u t ⋅ 6 δ t ≤ 8 η t 3 ( 1 ε ˉ t + 1 ) + 16 η t d t u t 2 \lambda^{g}_t\le\frac{8\eta_t}{3u_t}+\frac{8\eta_t}{3u_t}\cdot6\delta_t\le\frac{8\eta_t}{3}(\frac{1}{\bar{\varepsilon}_t}+1)+16\eta_t\frac{d_t}{u_t^2} λ t g ≤ 3 u t 8 η t + 3 u t 8 η t ⋅ 6 δ t ≤ 3 8 η t ( ε ˉ t 1 + 1 ) + 16 η t u t 2 d t , and d t u t 2 ≤ 1 t ( 1 + 1 ε ˉ t ) 2 ≤ 1 t ( 2 + 2 ε ˉ t 2 ) = 2 t + 1 6 m \frac{d_t}{u_t^2}\le\frac{1}{t}(1+\frac{1}{\bar{\varepsilon}_t})^2\le\frac{1}{t}(2+\frac{2}{\bar{\varepsilon}_t^2})=\frac{2}{t}+\frac{1}{6m} u t 2 d t ≤ t 1 ( 1 + ε ˉ t 1 ) 2 ≤ t 1 ( 2 + ε ˉ t 2 2 ) = t 2 + 6 m 1 .
The four resulting pieces of L 2 ∑ t λ t g L^2\sum_t \lambda_t^g L 2 ∑ t λ t g are
∑ 8 η t L 2 3 ε ˉ t = 2 L D 3 3 / 4 m − 1 / 4 ∑ t − 1 / 4 ≤ 8 3 L D 3 3 / 4 m − 1 / 4 ( T + 1 ) 3 / 4 , ∑ 8 η t L 2 3 = 4 L D 3 3 / 4 m 1 / 4 ∑ t − 3 / 4 ≤ 16 3 1 / 4 L D ( m ( T + 1 ) ) 1 / 4 , ∑ 16 η t L 2 2 3 = 16 ⋅ 3 3 / 4 L D m 1 / 4 ∑ t − 7 / 4 ≤ 27 L D m 1 / 4 , ∑ 16 η t L 2 6 m = 4 3 3 3 / 4 L D m − 3 / 4 ∑ t − 3 / 4 ≤ 12.2 L D m − 3 / 4 ( T + 1 ) 1 / 4 . \begin{aligned}
\sum\frac{8\eta_tL^2}{3\bar{\varepsilon}_t}
&=\frac{2LD}{3^{3/4}}m^{-1/4}\sum t^{-1/4}
\le\frac{8}{3}\frac{LD}{3^{3/4}}m^{-1/4}(T+1)^{3/4},\\
\sum\frac{8\eta_tL^2}{3}
&=\frac{4LD}{3^{3/4}}m^{1/4}\sum t^{-3/4}
\le\frac{16}{3^{1/4}}LD\left(m(T+1)\right)^{1/4},\\
\sum16\eta_tL^2\frac{2}{3}
&=16\cdot3^{3/4}LDm^{1/4}\sum t^{-7/4}
\le27LDm^{1/4},\\
\sum\frac{16\eta_tL^2}{6m}
&=\frac{4}{3}3^{3/4}LDm^{-3/4}\sum t^{-3/4}
\le12.2LDm^{-3/4}(T+1)^{1/4}.
\end{aligned} ∑ 3 ε ˉ t 8 η t L 2 ∑ 3 8 η t L 2 ∑ 16 η t L 2 3 2 ∑ 6 m 16 η t L 2 = 3 3/4 2 L D m − 1/4 ∑ t − 1/4 ≤ 3 8 3 3/4 L D m − 1/4 ( T + 1 ) 3/4 , = 3 3/4 4 L D m 1/4 ∑ t − 3/4 ≤ 3 1/4 16 L D ( m ( T + 1 ) ) 1/4 , = 16 ⋅ 3 3/4 L D m 1/4 ∑ t − 7/4 ≤ 27 L D m 1/4 , = 3 4 3 3/4 L D m − 3/4 ∑ t − 3/4 ≤ 12.2 L D m − 3/4 ( T + 1 ) 1/4 .
The three leading terms add up to ( 1 + 4 3 + 8 3 ) 3 − 3 / 4 = 5 ⋅ 3 − 3 / 4 (1+\frac{4}{3}+\frac{8}{3})3^{-3/4}=5\cdot3^{-3/4} ( 1 + 3 4 + 3 8 ) 3 − 3/4 = 5 ⋅ 3 − 3/4 times L D m − 1 / 4 ( T + 1 ) 3 / 4 LDm^{-1/4}(T+1)^{3/4} L D m − 1/4 ( T + 1 ) 3/4 . For T ≥ 2 T\ge2 T ≥ 2 we have ( T + 1 ) 3 / 4 ≤ T 3 / 4 + 3 4 T − 1 / 4 ≤ T 3 / 4 + 0.63 (T+1)^{3/4}\le T^{3/4}+\frac{3}{4}T^{-1/4}\le T^{3/4}+0.63 ( T + 1 ) 3/4 ≤ T 3/4 + 4 3 T − 1/4 ≤ T 3/4 + 0.63 and ( T + 1 ) 1 / 4 ≤ ( 3 2 ) 1 / 4 T 1 / 4 ≤ 1.11 T 1 / 4 (T+1)^{1/4}\le(\frac{3}{2})^{1/4}T^{1/4}\le1.11T^{1/4} ( T + 1 ) 1/4 ≤ ( 2 3 ) 1/4 T 1/4 ≤ 1.11 T 1/4 , while m 1 / 4 ≤ ( m T ) 1 / 4 m^{1/4}\le(mT)^{1/4} m 1/4 ≤ ( m T ) 1/4 and m − 3 / 4 ≤ m 1 / 4 m^{-3/4}\le m^{1/4} m − 3/4 ≤ m 1/4 ; hence the lower-order terms are at most ( 1.25 + 5 ⋅ 3 − 3 / 4 ⋅ 0.63 + 16 ⋅ 3 − 1 / 4 ⋅ 1.11 + 27 + 12.2 ⋅ 1.11 ) L D ( m T ) 1 / 4 ≤ 57 L D ( m T ) 1 / 4 (1.25+5\cdot3^{-3/4}\cdot0.63+16\cdot3^{-1/4}\cdot1.11+27+12.2\cdot1.11)LD(mT)^{1/4}\le57LD(mT)^{1/4} ( 1.25 + 5 ⋅ 3 − 3/4 ⋅ 0.63 + 16 ⋅ 3 − 1/4 ⋅ 1.11 + 27 + 12.2 ⋅ 1.11 ) L D ( m T ) 1/4 ≤ 57 L D ( m T ) 1/4 . For T = 1 T=1 T = 1 the claim is trivial since Reg 1 ≤ L D \operatorname{Reg}_1\le LD Reg 1 ≤ L D .
Proof for the capped tuning
Proof of Theorem 4.4 , fixed-horizon part. Now η = D 2 L ( 3 / T ) 3 / 4 κ \eta=\frac{D}{2L}(3/T)^{3/4}\kappa η = 2 L D ( 3/ T ) 3/4 κ with κ = min { m , T } 1 / 4 \kappa=\min\{m,T\}^{1/4} κ = min { m , T } 1/4 , σ = 3 / ( m T ) ≤ 1 2 \sigma=\sqrt{3/(mT)}\le\frac{1}{2} σ = 3/ ( m T ) ≤ 2 1 , and η L 2 T = L D 2 3 3 / 4 T 1 / 4 / κ \eta L^2T=\frac{LD}{2}3^{3/4}T^{1/4}/\kappa η L 2 T = 2 L D 3 3/4 T 1/4 / κ . As in the previous proof, D 2 2 η = L D 3 3 / 4 T 3 / 4 κ \frac{D^2}{2\eta}=\frac{LD}{3^{3/4}}\frac{T^{3/4}}{\kappa} 2 η D 2 = 3 3/4 L D κ T 3/4 , the atom term is at most D 2 2 η \frac{D^2}{2\eta} 2 η D 2 , and by (17)
8 η L 2 T 3 u ≤ 4 η L 2 T 3 m σ + 8 η L 2 T 3 = 2 L D 3 3 / 4 T 3 / 4 κ m + 4 L D 3 1 / 4 T 1 / 4 κ . \frac{8\eta L^2T}{3u}\le\frac{4\eta L^2T}{3m\sigma}+\frac{8\eta L^2T}{3}=\frac{2LD}{3^{3/4}}\frac{T^{3/4}\kappa}{\sqrt{m}}+\frac{4LD}{3^{1/4}}T^{1/4}\kappa. 3 u 8 η L 2 T ≤ 3 mσ 4 η L 2 T + 3 8 η L 2 T = 3 3/4 2 L D m T 3/4 κ + 3 1/4 4 L D T 1/4 κ .
If m ≤ T m\le T m ≤ T , then κ = m 1 / 4 \kappa=m^{1/4} κ = m 1/4 , the three terms add up to 4 L D 3 3 / 4 T 3 / 4 m − 1 / 4 + 4 L D 3 1 / 4 ( m T ) 1 / 4 \frac{4LD}{3^{3/4}}T^{3/4}m^{-1/4}+\frac{4LD}{3^{1/4}}(mT)^{1/4} 3 3/4 4 L D T 3/4 m − 1/4 + 3 1/4 4 L D ( m T ) 1/4 , and ( m T ) 1 / 4 ≤ T 3 / 4 m − 1 / 4 (mT)^{1/4}\le T^{3/4}m^{-1/4} ( m T ) 1/4 ≤ T 3/4 m − 1/4 . If m ≥ T m\ge T m ≥ T , then κ = T 1 / 4 \kappa=T^{1/4} κ = T 1/4 , T 3 / 4 κ / m = T / m ≤ T T^{3/4}\kappa/\sqrt{m}=T/\sqrt{m}\le\sqrt{T} T 3/4 κ / m = T / m ≤ T and T 1 / 4 κ = T T^{1/4}\kappa=\sqrt{T} T 1/4 κ = T , so the three terms add up to at most ( 4 3 3 / 4 + 4 3 1 / 4 ) L D T (\frac{4}{3^{3/4}}+\frac{4}{3^{1/4}})LD\sqrt{T} ( 3 3/4 4 + 3 1/4 4 ) L D T . In both cases Reg T ≤ ( 4 ⋅ 3 − 3 / 4 + 4 ⋅ 3 − 1 / 4 ) L D T 3 / 4 min { m , T } − 1 / 4 \operatorname{Reg}_T\le(4\cdot3^{-3/4}+4\cdot3^{-1/4})LD\,T^{3/4}\min\{m,T\}^{-1/4} Reg T ≤ ( 4 ⋅ 3 − 3/4 + 4 ⋅ 3 − 1/4 ) L D T 3/4 min { m , T } − 1/4 , and 4 ⋅ 3 − 3 / 4 + 4 ⋅ 3 − 1 / 4 < 4.8 4\cdot3^{-3/4}+4\cdot3^{-1/4}<4.8 4 ⋅ 3 − 3/4 + 4 ⋅ 3 − 1/4 < 4.8 .
Proof of Theorem 4.4 , anytime part. Write κ t : = min { m , t } 1 / 4 \kappa_t:=\min\{m,t\}^{1/4} κ t := min { m , t } 1/4 , so that η ~ t = D 2 L ( 3 / t ) 3 / 4 κ t \tilde{\eta}_t=\frac{D}{2L}(3/t)^{3/4}\kappa_t η ~ t = 2 L D ( 3/ t ) 3/4 κ t , and Λ : = max { ( T + 1 ) 3 / 4 m − 1 / 4 , T + 1 } \Lambda:=\max\{(T+1)^{3/4}m^{-1/4},\sqrt{T+1}\} Λ := max {( T + 1 ) 3/4 m − 1/4 , T + 1 } . Since κ t \kappa_t κ t is non-decreasing, η ~ t \tilde{\eta}_t η ~ t is non-increasing and η ~ t / η ~ t − 1 ≥ ( ( t − 1 ) / t ) 3 / 4 \tilde{\eta}_t/\tilde{\eta}_{t-1}\ge((t-1)/t)^{3/4} η ~ t / η ~ t − 1 ≥ (( t − 1 ) / t ) 3/4 , so the facts (a), (b), (c) of the proof of Theorem 4.3 hold, and Theorem C.4 gives, as in that proof,
Reg T ≤ D 2 2 η ~ T + 1 + 1.25 L D + ∑ t = 3 T + 1 [ D 2 2 t η ~ t + 8 η ~ t L 2 3 ( 1 ε ˉ t + 1 ) + 16 η ~ t L 2 ( 2 t + 1 6 m ) ] . (19) \operatorname{Reg}_T\le\frac{D^2}{2\tilde{\eta}_{T+1}}+1.25LD+\sum_{t=3}^{T+1}\left[\frac{D^2}{2t\tilde{\eta}_t}+\frac{8\tilde{\eta}_tL^2}{3}\left(\frac{1}{\bar{\varepsilon}_t}+1\right)+16\tilde{\eta}_tL^2\left(\frac{2}{t}+\frac{1}{6m}\right)\right]. \tag*{(19)} Reg T ≤ 2 η ~ T + 1 D 2 + 1.25 L D + t = 3 ∑ T + 1 [ 2 t η ~ t D 2 + 3 8 η ~ t L 2 ( ε ˉ t 1 + 1 ) + 16 η ~ t L 2 ( t 2 + 6 m 1 ) ] . (19)
with ε ˉ t = 2 3 m / t \bar{\varepsilon}_t=2\sqrt{3m/t} ε ˉ t = 2 3 m / t . We use η ~ t L 2 = L D 2 3 3 / 4 t − 3 / 4 κ t \tilde{\eta}_tL^2=\frac{LD}{2}3^{3/4}t^{-3/4}\kappa_t η ~ t L 2 = 2 L D 3 3/4 t − 3/4 κ t and D 2 2 t η ~ t = L D 3 3 / 4 t − 1 / 4 κ t \frac{D^2}{2t\tilde{\eta}_t}=\frac{LD}{3^{3/4}}\frac{t^{-1/4}}{\kappa_t} 2 t η ~ t D 2 = 3 3/4 L D κ t t − 1/4 , the three elementary bounds
κ t ≤ m 1 / 4 , κ t ≤ t 1 / 4 , 1 κ t ≤ m − 1 / 4 + t − 1 / 4 , \kappa_t\le m^{1/4},\qquad\kappa_t\le t^{1/4},\qquad\frac{1}{\kappa_t}\le m^{-1/4}+t^{-1/4}, κ t ≤ m 1/4 , κ t ≤ t 1/4 , κ t 1 ≤ m − 1/4 + t − 1/4 ,
and the integral estimates ∑ t = 3 T + 1 t − 1 / 4 ≤ 4 3 ( T + 1 ) 3 / 4 \sum_{t=3}^{T+1}t^{-1/4}\le\frac{4}{3}(T+1)^{3/4} ∑ t = 3 T + 1 t − 1/4 ≤ 3 4 ( T + 1 ) 3/4 , ∑ t = 3 T + 1 t − 1 / 2 ≤ 2 T + 1 \sum_{t=3}^{T+1}t^{-1/2}\le2\sqrt{T+1} ∑ t = 3 T + 1 t − 1/2 ≤ 2 T + 1 and ∑ t ≥ 3 t − 3 / 2 ≤ 1.35 \sum_{t\ge3}t^{-3/2}\le1.35 ∑ t ≥ 3 t − 3/2 ≤ 1.35 . Both ( T + 1 ) 3 / 4 m − 1 / 4 (T+1)^{3/4}m^{-1/4} ( T + 1 ) 3/4 m − 1/4 and T + 1 \sqrt{T+1} T + 1 are at most Λ \Lambda Λ . The terms of (19) are bounded one by one:
D 2 2 η ~ T + 1 = L D 3 3 / 4 ( T + 1 ) 3 / 4 κ T + 1 ≤ L D 3 3 / 4 ( ( T + 1 ) 3 / 4 m − 1 / 4 + T + 1 ) ≤ 0.9 L D Λ . \frac{D^2}{2\widetilde{\eta}_{T+1}}=\frac{LD}{3^{3/4}}\frac{(T+1)^{3/4}}{\kappa_{T+1}}\leq\frac{LD}{3^{3/4}}\left((T+1)^{3/4}m^{-1/4}+\sqrt{T+1}\right)\leq0.9LD\Lambda. 2 η T + 1 D 2 = 3 3/4 L D κ T + 1 ( T + 1 ) 3/4 ≤ 3 3/4 L D ( ( T + 1 ) 3/4 m − 1/4 + T + 1 ) ≤ 0.9 L D Λ.
∑ t = 3 T + 1 D 2 2 t η ~ t ≤ L D 3 3 / 4 ( m − 1 / 4 ∑ t = 3 T + 1 t − 1 / 4 + ∑ t = 3 T + 1 t − 1 / 2 ) ≤ L D 3 3 / 4 ( 4 3 + 2 ) Λ ≤ 1.5 L D Λ . \sum_{t=3}^{T+1}\frac{D^2}{2t\widetilde{\eta}_t}\leq\frac{LD}{3^{3/4}}\left(m^{-1/4}\sum_{t=3}^{T+1}t^{-1/4}+\sum_{t=3}^{T+1}t^{-1/2}\right)\leq\frac{LD}{3^{3/4}}\left(\frac{4}{3}+2\right)\Lambda\leq1.5LD\Lambda. t = 3 ∑ T + 1 2 t η t D 2 ≤ 3 3/4 L D ( m − 1/4 t = 3 ∑ T + 1 t − 1/4 + t = 3 ∑ T + 1 t − 1/2 ) ≤ 3 3/4 L D ( 3 4 + 2 ) Λ ≤ 1.5 L D Λ.
∑ t = 3 T + 1 8 η ~ t L 2 3 ε ~ t = 2 ⋅ 3 1 / 4 3 L D ∑ t = 3 T + 1 κ t t − 1 / 4 m ≤ 2 ⋅ 3 1 / 4 3 L D m − 1 / 4 ∑ t = 3 T + 1 t − 1 / 4 ≤ 1.2 L D Λ , \sum_{t=3}^{T+1}\frac{8\widetilde{\eta}_tL^2}{3\widetilde{\varepsilon}_t}=\frac{2\cdot3^{1/4}}{3}LD\sum_{t=3}^{T+1}\frac{\kappa_t t^{-1/4}}{\sqrt{m}}\leq\frac{2\cdot3^{1/4}}{3}LDm^{-1/4}\sum_{t=3}^{T+1}t^{-1/4}\leq1.2LD\Lambda, t = 3 ∑ T + 1 3 ε t 8 η t L 2 = 3 2 ⋅ 3 1/4 L D t = 3 ∑ T + 1 m κ t t − 1/4 ≤ 3 2 ⋅ 3 1/4 L D m − 1/4 t = 3 ∑ T + 1 t − 1/4 ≤ 1.2 L D Λ ,
∑ t = 3 T + 1 8 η ~ t L 2 3 = 4 3 3 3 / 4 L D ∑ t = 3 T + 1 t − 3 / 4 κ t ≤ 4 3 3 3 / 4 L D ∑ t = 3 T + 1 t − 1 / 2 ≤ 6.1 L D Λ , \sum_{t=3}^{T+1}\frac{8\widetilde{\eta}_tL^2}{3}=\frac{4}{3}3^{3/4}LD\sum_{t=3}^{T+1}t^{-3/4}\kappa_t\leq\frac{4}{3}3^{3/4}LD\sum_{t=3}^{T+1}t^{-1/2}\leq6.1LD\Lambda, t = 3 ∑ T + 1 3 8 η t L 2 = 3 4 3 3/4 L D t = 3 ∑ T + 1 t − 3/4 κ t ≤ 3 4 3 3/4 L D t = 3 ∑ T + 1 t − 1/2 ≤ 6.1 L D Λ ,
∑ t = 3 T + 1 32 η ~ t L 2 t = 16 ⋅ 3 3 / 4 L D ∑ t = 3 T + 1 t − 7 / 4 κ t ≤ 16 ⋅ 3 3 / 4 L D ∑ t = 3 T + 1 t − 3 / 2 ≤ 50 L D , \sum_{t=3}^{T+1}\frac{32\widetilde{\eta}_tL^2}{t}=16\cdot3^{3/4}LD\sum_{t=3}^{T+1}t^{-7/4}\kappa_t\leq16\cdot3^{3/4}LD\sum_{t=3}^{T+1}t^{-3/2}\leq50LD, t = 3 ∑ T + 1 t 32 η t L 2 = 16 ⋅ 3 3/4 L D t = 3 ∑ T + 1 t − 7/4 κ t ≤ 16 ⋅ 3 3/4 L D t = 3 ∑ T + 1 t − 3/2 ≤ 50 L D ,
∑ t = 3 T + 1 16 η ~ t L 2 6 m ≤ 4 3 3 3 / 4 L D ∑ t = 3 T + 1 t − 3 / 4 κ t ≤ 6.1 L D Λ . \sum_{t=3}^{T+1}\frac{16\widetilde{\eta}_tL^2}{6m}\leq\frac{4}{3}3^{3/4}LD\sum_{t=3}^{T+1}t^{-3/4}\kappa_t\leq6.1LD\Lambda. t = 3 ∑ T + 1 6 m 16 η t L 2 ≤ 3 4 3 3/4 L D t = 3 ∑ T + 1 t − 3/4 κ t ≤ 6.1 L D Λ.
The first two lines use 1 / κ t ≤ m − 1 / 4 + t − 1 / 4 1/\kappa_t\leq m^{-1/4}+t^{-1/4} 1/ κ t ≤ m − 1/4 + t − 1/4 , the third κ t ≤ m 1 / 4 \kappa_t\leq m^{1/4} κ t ≤ m 1/4 , the last three κ t ≤ t 1 / 4 \kappa_t\leq t^{1/4} κ t ≤ t 1/4 and m ≥ 1 m\geq1 m ≥ 1 . Adding up those bounds, we get Reg T ≤ 16 L D Λ + 52 L D \operatorname{Reg}_T\leq16LD\Lambda+52LD Reg T ≤ 16 L D Λ + 52 L D . For T ≥ 2 T\geq2 T ≥ 2 we have Λ ≤ ( 3 / 2 ) 3 / 4 max { T 3 / 4 m − 1 / 4 , T } ≤ 1.4 max { T 3 / 4 m − 1 / 4 , T } \Lambda\leq(3/2)^{3/4}\max\{T^{3/4}m^{-1/4},\sqrt{T}\}\leq1.4\max\{T^{3/4}m^{-1/4},\sqrt{T}\} Λ ≤ ( 3/2 ) 3/4 max { T 3/4 m − 1/4 , T } ≤ 1.4 max { T 3/4 m − 1/4 , T } and max { T 3 / 4 m − 1 / 4 , T } ≥ 2 \max\{T^{3/4}m^{-1/4},\sqrt{T}\}\geq\sqrt{2} max { T 3/4 m − 1/4 , T } ≥ 2 , so Reg T ≤ ( 16 ⋅ 1.4 + 52 / 2 ) L D max { T 3 / 4 m − 1 / 4 , T } ≤ 60 L D max { T 3 / 4 m − 1 / 4 , T } \operatorname{Reg}_T\leq(16\cdot1.4+52/\sqrt{2})LD\max\{T^{3/4}m^{-1/4},\sqrt{T}\}\leq60LD\max\{T^{3/4}m^{-1/4},\sqrt{T}\} Reg T ≤ ( 16 ⋅ 1.4 + 52/ 2 ) L D max { T 3/4 m − 1/4 , T } ≤ 60 L D max { T 3/4 m − 1/4 , T } . For T = 1 T=1 T = 1 the claim is trivial since Reg 1 ≤ L D \operatorname{Reg}_1\leq LD Reg 1 ≤ L D .