betaFeedback

WorksMathematics

DREAM: Delay-Resistant Acceleration with Momentum

Abstract

With exact, delay-free gradients, Nesterov's accelerated gradient (NAG) achieves optimal κ\sqrtκ dependence for LL-smooth, μμ-strongly convex objectives, where κ=L/μκ=L/μ; heavy-ball (HB) achieves the same scaling on strongly convex quadratics. In asynchronous and distributed settings, gradients can be inexact, with delays varying across workers and iterations. We isolate staleness using a fixed delay τ≥1τ\ge1, replacing ∇f(yk)\nabla f(y_k) with ∇f(yk−τ)\nabla f(y_{k-τ}) at iteration kk. Arjevani, Shamir, and Srebro establish a worst-case lower bound of Ω(τκlog⁡(1/ε))Ω(τ\sqrtκ\log(1/\varepsilon)) iterations to reach objective error at most ε\varepsilon as ε→0\varepsilon\to0. We show that for any fixed positive delay, neither NAG nor HB retains κ\sqrtκ-acceleration on strongly convex quadratics when updating every iteration, for any stable constant step size and positive momentum. We introduce delay-resistant accelerated momentum (DREAM), which decouples gradient-query extrapolation from momentum and uses explicit delay-dependent parameters. DREAM attains a linear convergence rate 1−Θ(1/(τκ))1-Θ(1/(τ\sqrtκ)) for all smooth strongly convex objectives as κ→∞κ\to\infty, yielding complexity matching the delayed first-order lower bound up to logarithmic factors. Our proof combines characteristic-polynomial stability analysis on quadratics with an analytic Zames–Falb integral quadratic constraint certificate. We extend DREAM to smooth convex objectives, obtaining near-optimal iteration complexity up to logarithmic factors for sufficiently small target accuracy. Experiments on strongly convex quadratics and regularized logistic regression corroborate DREAM's accelerated condition-number dependence under delay, unlike NAG and HB, and suggest this scaling persists when parameters use a conservative bound on the true delay.

This research was supported in part by the Office of Naval Research under award numbers N00014-24-1-2628 and N00014-24-1-2666.

Mathematics Subject Classification. 65K05, 90C25, 90C30, 93D10, 93D20.

Introduction

We consider unconstrained minimization of a smooth convex function,

f⋆:=inf⁡x∈Rdf(x),f∈S0,L,(1)f_\star:= \inf_{x\in\mathbb{R}^{d}} f(x), \qquad f \in\mathcal{S}_{0,L}, \tag*{(1)}

where S0,L\mathcal{S}_{0,L} denotes the class of continuously differentiable convex functions f:Rd→Rf : \mathbb{R}^{d} \to\mathbb{R} with LL-Lipschitz gradients for some L>0L > 0. We study both this class and its strongly convex subclass Sμ,L⊆S0,L\mathcal{S}_{\mu,L} \subseteq\mathcal{S}_{0,L}, with most of our analysis devoted to the subclass Sμ,L\mathcal{S}_{\mu,L}. More precisely, for 0<μ≤L0 < \mu\le L, Sμ,L\mathcal{S}_{\mu,L} consists of functions satisfying

μ2∥x−y∥2≤f(x)−f(y)−⟨∇f(y),x−y⟩≤L2∥x−y∥2,x,y∈Rd,(2)\frac{\mu}{2}\lVert x-y\rVert^{2} \le f(x)-f(y)-\langle\nabla f(y),x-y\rangle\le\frac{L}{2}\lVert x-y\rVert^{2}, \qquad x,y\in\mathbb{R}^{d}, \tag*{(2)}

where we use ∥⋅∥\lVert\cdot\rVert for the Euclidean norm. Setting μ=0\mu= 0 in (2) gives the smooth convex class S0,L\mathcal{S}_{0,L} introduced above. For this class, we assume that the minimum is attained, i.e., there exists x⋆∈Rdx_\star\in\mathbb{R}^{d} such that f⋆=f(x⋆)f_\star= f(x_\star). For functions in Sμ,L\mathcal{S}_{\mu,L}, the minimizer is unique, and we write κ:=L/μ\kappa:= L/\mu for the condition number of the function ff.

In the classical first-order oracle model for iteratively solving (1), where an exact gradient ∇f(yk)\nabla f(y_k) at the current query point yky_k is available without any delay at each iteration kk, the optimal worst-case iteration complexity for the class Sμ,L\mathcal{S}_{\mu,L} of LL-smooth, μ\mu-strongly convex functions scales as Θ(κlog⁡(1/ϵ))\Theta(\sqrt{\kappa}\log(1/\epsilon)) [39], where ϵ>0\epsilon> 0 is the target objective-value error. For the class S0,L\mathcal{S}_{0,L} of LL-smooth convex functions, the corresponding complexity is Θ(LR2/ϵ)\Theta(\sqrt{LR^2/\epsilon}) [39], where RR bounds the initial distance to a minimizer. These lower bounds are established through hard instances in sufficiently high dimensions. The underlying finite-iteration lower bounds hold for every κ>1\kappa> 1 in the strongly convex case; the commonly stated Θ(κlog⁡(1/ϵ))\Theta(\sqrt{\kappa}\log(1/\epsilon)) complexity scaling is understood for condition numbers bounded away from 1 and sufficiently small target accuracy ϵ\epsilon. In terms of achievability, gradient descent (GD) does not attain these bounds in general: its worst-case iteration complexity is O(κlog⁡(1/ϵ))O(\kappa\log(1/\epsilon)) in the strongly convex case and O(1/ϵ)O(1/\epsilon) in the convex case. On the other hand, Nesterov’s accelerated gradient method (NAG) [38] achieves these accelerated rates with constant step-size and momentum parameters for functions in Sμ,L\mathcal{S}_{\mu,L}, and with appropriately varying momentum parameters for functions in S0,L\mathcal{S}_{0,L} [39]. Polyak’s heavy-ball method (HB) [41] also achieves the accelerated κ\sqrt{\kappa} dependence on strongly convex quadratics, although the same guarantee does not extend to arbitrary smooth strongly convex objectives under the classical HB parameter choices [34]. These optimal rates of NAG and HB provide a natural benchmark for characterizing the performance of first-order methods under settings that deviate from the classical first-order oracle model.

In this paper, we study a fixed-delay variation of the classical first-order oracle model for solving (1), with a focus on non-idling methods. Specifically, for a fixed integer τ≥1\tau\ge1, the oracle returns the exact gradient ∇f(yk−τ)\nabla f(y_{k-\tau}) at iteration kk, where yk−τy_{k-\tau} is the point queried at iteration k−τk-\tau [3, 51 CCS24]. We call a method non-idling if it uses this arriving gradient to perform an update at every iteration. An idling (waiting) method instead postpones its next update until the gradient at its current query point is available. This non-idling, fixed-delay setting is motivated by asynchronous and distributed optimization. For example, a worker in such a system may evaluate a gradient at one iterate while updates from other workers change the shared iterate. When the gradient arrives, it therefore corresponds to an earlier iterate and is stale [7, 52, 46, 13]. Although delays in such systems can vary across workers and iterations, we consider exact gradients and a common fixed delay to isolate the effect of staleness. Analyzing this setting can help develop tools for studying more general asynchronous distributed methods, including those with time-varying delays and stochastic gradient errors. We are particularly interested in whether non-idling methods can attain the optimal iteration-complexity scaling in this fixed-delay model.

We use the worst-case lower bounds of Arjevani, Shamir, and Srebro [3 Theorem 4] as the target iteration-complexity scale for non-idling methods. These bounds hold for a deterministic fixed-delay first-order span model, which permits both waiting and non-idling methods. At iteration kk, the span model requires the new iterate to be a linear combination of gradients from queries made no later than iteration k−τk-\tau. The precise span condition is stated later in Theorem 2.1. For sufficiently large dimension and under the same accuracy and conditioning qualifications as above, their results imply the following iteration-complexity lower bounds for this span model

Ω((τ+1)κlog⁡1ϵ)for f∈Sμ,L,Ω((τ+1)LR2ϵ)for f∈S0,L.(3)\begin{aligned} \Omega\left((\tau+1)\sqrt{\kappa}\log\frac{1}{\epsilon}\right)&\quad\text{for } f\in\mathcal{S}_{\mu,L},\\ \Omega\left((\tau+1)\sqrt{\frac{LR^2}{\epsilon}}\right)&\quad\text{for } f\in\mathcal{S}_{0,L}. \tag*{(3)} \end{aligned}

The key point is that, compared with the undelayed accelerated rates, these lower bounds introduce an additional factor of τ+1\tau+1. This delay penalty occurs even for quadratic objectives.

In terms of achievability in this fixed-delay, non-idling setting, Arjevani, Shamir, and Srebro [3] analyze delayed gradient descent (GD),

xk+1=xk−α∇f(xk−τ),(4)x_{k+1}=x_k-\alpha\nabla f(x_{k-\tau}), \tag*{(4)}

which processes a τ\tau-stale gradient at every iteration. On smooth strongly convex quadratics, their guarantees for delayed GD have complexity of order (τ+1)κlog⁡(1/ϵ)(\tau+1)\kappa\log(1/\epsilon), and on smooth convex quadratics the iteration complexity is of order (τ+1)LR2/ϵ(\tau+1)LR^2/\epsilon [3]. Hence delayed GD matches the linear dependence on delay in the lower bounds, but, as in the undelayed setting, it does not achieve the accelerated dependence on κ\kappa in the strongly convex case or ϵ\epsilon in the convex case. In particular, they leave open whether an accelerated gradient method can attain these rates while updating with a stale gradient at every iteration. This raises two related questions: can the classical HB and NAG recurrences retain acceleration under delay with suitably chosen parameters while remaining non-idling, and, if not, can a more general non-idling momentum recurrence attain these lower complexity bounds?

To answer these questions, we study a generalized momentum method (GMM) with constant parameters under a fixed delay in the non-idling setting. With xkx_k denoting the main iterate and yky_k the gradient query point, the delayed GMM updates take the following form:

yk=xk+ν(xk−xk−1),xk+1=xk−α∇f(yk−τ)+β(xk−xk−1),k≥0.(5)\begin{aligned} y_k&=x_k+\nu(x_k-x_{k-1}),\\ x_{k+1}&=x_k-\alpha\nabla f(y_{k-\tau})+\beta(x_k-x_{k-1}),\qquad k\ge0. \tag*{(5)} \end{aligned}

Here α>0\alpha>0 is the stepsize, β≥0\beta\ge0 is the momentum coefficient, and ν≥0\nu\ge0 is the gradient-query extrapolation coefficient. The parameters (α,β,ν)(\alpha,\beta,\nu) remain fixed across iterations, although their values may depend on the smoothness and strong-convexity constants and the delay. The initial values x−τ−1,…,x0x_{-\tau-1},\ldots,x_0 determine the query points needed to start the iteration. We use the terms delayed HB and delayed NAG for the specializations ν=0\nu=0 and ν=β\nu=\beta, respectively, in (5). These two specializations are the classical HB and NAG updates using the arriving τ\tau-delayed gradient at every iteration. Delayed GMM allows ν\nu to be chosen independently of β\beta, rather than requiring ν=0\nu=0 or ν=β\nu=\beta. We use this freedom to adapt the extrapolation to the delay parameter τ\tau.

The main results in this paper identify a sharp qualitative difference between the delayed HB and NAG specializations and the general delayed GMM family. Under every fixed positive delay τ≥1\tau\geq1, delayed HB with constant parameters fails to retain κ\sqrt{\kappa}-acceleration on strongly convex quadratics; the same is true of delayed NAG, even when gradients are delayed by only one iteration (τ=1\tau=1), for every choice of constant parameters in the standard positive-momentum regime that ensures convergence on strongly convex quadratics. In contrast, we give an explicit choice of parameters, with ν\nu proportional to the delay, for which delayed GMM achieves κ\sqrt{\kappa}-acceleration on the entire class Sμ,L\mathcal{S}_{\mu,L} for every fixed delay τ≥1\tau\geq1 and for all sufficiently large κ\kappa. We call this method delay-resistant acceleration with momentum (DREAM). We also extend this approach to the smooth convex class S0,L\mathcal{S}_{0,L} by using DREAM as an inner solver in an accelerated proximal-point construction. For sufficiently small target error, the resulting iteration complexity matches the delay and accuracy dependence of the convex lower bound in (3) up to logarithmic factors.

Main results and contributions

We now state the main contributions of this paper more precisely. But first, we clarify their relation to the lower bounds in (3). These bounds apply to both waiting and non-idling methods, whereas our focus is on non-idling methods. Indeed, a waiting method can match the order of these lower bounds by running NAG, making an update only after the gradient at its current query point has been returned, and remaining idle for the intervening τ\tau iterations [3]. Thus one accelerated update is performed every τ+1\tau+1 iterations. This construction establishes attainability of the lower-bound scaling, but it does not exploit gradients that arrive while the algorithm is waiting. In an asynchronous distributed architecture, forcing every update to wait for a fresh gradient can sacrifice the benefits of pipelining or require waiting for the slowest worker. The question we address is therefore whether non-idling methods can achieve acceleration while using the arriving τ\tau-stale gradient at every iteration.

We now turn to our main results, presented in § 3–§ 7. In what follows, we also use the class Qμ,L⊂Sμ,L\mathcal{Q}_{\mu,L}\subset\mathcal{S}_{\mu,L} of quadratic objectives whose symmetric Hessians have spectra contained in [μ,L][\mu,L]. Throughout these results, τ\tau is a fixed positive integer.

Limitations of delayed HB and NAG. In § 3, we first characterize the convergence of delayed GMM on Qμ,L\mathcal{Q}_{\mu,L}. For fixed parameters (α,β,ν)(\alpha,\beta,\nu), the delayed GMM iterates converge to the minimizer x⋆x_\star for every f∈Qμ,Lf\in\mathcal{Q}_{\mu,L} and every initial history if and only if

ρ(α,β,ν):=sup⁡λ∈[μ,L]max⁡{∣z∣:z∈C,Pτ,αλ(z)=0}<1,\rho(\alpha,\beta,\nu):=\sup_{\lambda\in[\mu,L]}\max\{|z|:z\in\mathbb{C},P_{\tau,\alpha\lambda}(z)=0\}<1,

where the characteristic polynomial of the scalar delayed GMM error recurrence along a Hessian eigendirection with eigenvalue λ\lambda is given by

Pτ,αλ(z):=zτ(z−1)(z−β)+αλ((1+ν)z−ν).(6)P_{\tau,\alpha\lambda}(z):=z^\tau(z-1)(z-\beta)+\alpha\lambda\bigl((1+\nu)z-\nu\bigr). \tag*{(6)}

When ρ(α,β,ν)∈(0,1)\rho(\alpha,\beta,\nu)\in(0,1), the iterates satisfy

∥xk−x⋆∥≤ckρ(α,β,ν)k,k≥0,\lVert x_k-x_\star\rVert\leq c_k\rho(\alpha,\beta,\nu)^k,\qquad k\geq0,

for some non-negative sequence {ck}k≥0\{c_k\}_{k\geq0} that grows at most polynomially in kk. We call such a parameter choice Schur stable, as it implies that the polynomial Pτ,αλP_{\tau,\alpha}^{\lambda} is Schur stable, i.e., all the roots lie strictly inside the unit circle for all λ∈[μ,L]\lambda\in[\mu,L]. Thus, ρ(α,β,ν)\rho(\alpha,\beta,\nu) characterizes the worst-case asymptotic convergence factor over Qμ,LQ_{\mu,L}. In particular, for every ρ^\widehat{\rho} satisfying ρ(α,β,ν)<ρ^<1\rho(\alpha,\beta,\nu)<\widehat{\rho}<1, the iterates admit a geometric bound of the form ∥xk−x⋆∥≤Cρ^ρ^k\lVert x_k-x_\star\rVert\leq C_{\widehat{\rho}}\widehat{\rho}^{k} for some constant Cρ^>0C_{\widehat{\rho}}>0 independent of kk.

References

  1. [1][AFGO19] Necdet Serhat Aybat, Alireza Fallah, Mert Gurbuzbalaban, and Asuman Ozdaglar. A universally optimal multistage accelerated stochastic gradient method. Advances in Neural Information Processing Systems, 32, 2019.arxiv.org/abs/1901.08022
  2. [2]Thalaiyasingam Ajanthan, Sameera Ramasinghe, Yan Zuo, Gil Avraham, and Alexander Long. Nesterov method for asynchronous pipeline parallel optimization. In Aarti Singh, Maryam Fazel, Daniel Hsu, Simon Lacoste-Julien, Felix Berkenkamp, Tegan Maharaj, Kiri Wagstaff, and Jerry Zhu, editors, Proceedings of the 42nd International Conference on Machine Learning, volume 267 of Proceedings of Machine Learning Research, pages 909–924. PMLR, 13–19 Jul 2025.arxiv.org/abs/2505.01099
  3. [3]Yossi Arjevani, Ohad Shamir, and Nathan Srebro. A tight convergence analysis for stochastic gradient descent with delayed updates. In Proceedings of the 31st International Conference on Algorithmic Learning Theory, volume 117 of Proceedings of Machine Learning Research, pages 111–132. PMLR, 2020.
  4. [4]Stephen Barnett. Matrices in Control Theory: With Applications to Linear Programming. Van Nostrand Reinhold, London, 1971.
  5. [5]Joseph A. Ball, Gilbert J. Groenewald, and Sanne Ter Horst. Standard versus strict bounded real lemma with infinite-dimensional state space. i. the state-space-similarity approach. Journal of Operator Theory, 80(1):pp. 225–253, 2018.arxiv.org/abs/1804.08507
  6. [6]Ross Boczar, Laurent Lessard, and Benjamin Recht. Exponential convergence bounds using integral quadratic constraints. In 2015 54th IEEE Conference on Decision and Control (CDC), pages 7516–7521. IEEE, 2015.arxiv.org/abs/1503.07222
  7. [7]Dimitri Bertsekas and John Tsitsiklis. Parallel and distributed computation: numerical methods. Athena Scientific, 2015.
  8. [9]Bugra Can and Mert Gürbüzbalaban. Entropic risk-averse generalized momentum methods. Optimization Methods and Software, 40(6):1535–1583, 2025.arxiv.org/abs/2204.11292
  9. [10]Saman Cyrus, Bin Hu, Bryan Van Scoy, and Laurent Lessard. A robust accelerated optimization algorithm for strongly convex functions. In 2018 Annual American Control Conference (ACC), pages 1376–1381, 2018.arxiv.org/abs/1710.04753
  10. [11]Joaquin Carrasco, William P. Heath, Jingfan Zhang, Nur Syazreen Ahmad, and Shuai Wang. Convex searches for discrete-time zames–falb multipliers. IEEE Transactions on Automatic Control, 65(11):4538–4553, 2020.DOI
  11. [12]Arthur Cohn. Über die anzahl der wurzeln einer algebraischen gleichung in einem kreise. Mathematische Zeitschrift, 14(1):110–148, 1922.DOI
  12. [13]Bugra Can, Saeed Soori, Maryam Mehri Dehnavi, and Mert Gürbüzbalaban. L-DQN: An asynchronous limited-memory distributed quasi-newton method. In 2021 60th IEEE Conference on Decision and Control (CDC), pages 2386–2393. IEEE, 2021.arxiv.org/abs/2108.09365
  13. [14]Olivier Devolder, François Glineur, and Yurii Nesterov. First-order methods with inexact oracle: The strongly convex case. CORE Discussion Paper 2013/16, Center for Operations Research and Econometrics (CORE), Université catholique de Louvain, 2013.
  14. [15]Olivier Devolder, François Glineur, and Yurii Nesterov. First-order methods of smooth convex optimization with inexact oracle. Mathematical Programming, 146(1):37–75, 2014.
  15. [16]Carlos E de Souza and Lihua Xie. On the discrete-time bounded real lemma with application in the characterization of static state feedback H∞ controllers. Systems & Control Letters, 18(1):61–71, 1992.
  16. [17]Cong Fang, Yameng Huang, and Zhouchen Lin. Accelerating asynchronous algorithms for convex optimization by momentum compensation. arXiv preprint arXiv:1802.09747, 2018.
  17. [18]Randy A. Freeman. Noncausal Zames–Falb multipliers for tighter estimates of exponential convergence rates. In 2018 Annual American Control Conference (ACC), pages 2984–2989, Milwaukee, WI, 2018. IEEE.DOI
  18. [19]Ronald L. Graham, Donald E. Knuth, and Oren Patashnik. Concrete Mathematics: A Foundation for Computer Science. Addison–Wesley, Reading, MA, 2nd edition, 1994. 34th printing, January 2022.
  19. [20]Laura Gardini, Noemi Schmitt, Iryna Sushko, Fabio Tramontana, and Frank H. Westerhoff. Necessary and sufficient conditions for the roots of a cubic polynomial and bifurcations of codimension-1, -2, -3 for 3D maps. Journal of Difference Equations and Applications, 27(4):557–578, 2021.
  20. [22]Fredric J. Harris. On the use of windows for harmonic analysis with the discrete Fourier transform. Proceedings of the IEEE, 66(1):51–83, 1978.
  21. [23]Hadrien Hendrikx, Francis Bach, and Laurent Massoulie. Accelerated decentralized optimization with local updates for smooth and strongly convex objectives. In Kamalika Chaudhuri and Masashi Sugiyama, editors, Proceedings of the Twenty-Second International Conference on Artificial Intelligence and Statistics, volume 89 of Proceedings of Machine Learning Research, pages 897–906. PMLR, 16–18 Apr 2019.
  22. [24]Robert Hannah, Fei Feng, and Wotao Yin. A2BCD: Asynchronous acceleration with optimal complexity. In International Conference on Learning Representations, 2019.
  23. [25]Bin Hu and Laurent Lessard. Dissipativity theory for nesterov’s accelerated method. In International Conference on Machine Learning, pages 1549–1557. PMLR, 2017.arxiv.org/abs/1706.04381
  24. [26]Bin Hu, Márcio J. Lacerda, and Peter Seiler. Robustness analysis of uncertain discrete-time systems with dissipation inequalities and integral quadratic constraints. International Journal of Robust and Nonlinear Control, 27(11):1940–1962, 2017.
  25. [27]Diederich Hinrichsen and Anthony J Pritchard. Mathematical systems theory I: modelling, state space analysis, stability and robustness. Springer Science & Business Media, 2005.
  26. [28]Eliahu I. Jury. Theory and Application of the z-Transform Method. John Wiley & Sons, New York, 1964.
  27. [29]Tosio Kato. Perturbation Theory for Linear Operators. Classics in Mathematics. Springer, Berlin, 1995. Reprint of the 1980 edition.
  28. [30]Steven George Krantz and Harold R Parks. The implicit function theorem: history, theory, and applications, volume 202. Springer, 2002.
  29. [31]A. Liénard and M. H. Chipart. Sur le signe de la partie réelle des racines d’une équation algébrique. Journal de Mathématiques Pures et Appliquées, 10(6):291–346, 1914.
  30. [32]Hongzhou Lin, Julien Mairal, and Zaid Harchaoui. A universal catalyst for first-order optimization. In Advances in Neural Information Processing Systems, volume 28, 2015.
  31. [33]Hongzhou Lin, Julien Mairal, and Zaid Harchaoui. Catalyst acceleration for first-order convex optimization: from theory to practice. Journal of Machine Learning Research, 18(212):1–54, 2018.
  32. [34]Laurent Lessard, Benjamin Recht, and Andrew Packard. Analysis and design of optimization algorithms via integral quadratic constraints. SIAM Journal on Optimization, 26(1):57–95, 2016.DOI
  33. [35]Morris Marden. Geometry of Polynomials. Number 3 in Mathematical Surveys. American Mathematical Society, Providence, RI, 2nd edition, 1966.
  34. [36]Alexandre Megretski and Anders Rantzer. System analysis via integral quadratic constraints. IEEE Transactions on Automatic Control, 42(6):819–830, 1997.
  35. [37]Simon Michalowsky, Carsten Scherer, and Christian Ebenbauer. Robust and structure exploiting optimisation algorithms: an integral quadratic constraint approach. International Journal of Control, 94(11):2956–2979, 2021.DOI
  36. [38]Yurii Nesterov. A method for solving the convex programming problem with convergence rate O(1/k²). Soviet Mathematics Doklady, 27(2):372–376, 1983.
  37. [39]Yurii Nesterov. Lectures on Convex Optimization, volume 137 of Springer Optimization and Its Applications. Springer, Cham, 2nd edition, 2018.DOI
  38. [40]Alan V. Oppenheim, Alan S. Willsky, and S. Hamid Nawab. Signals and Systems. Prentice Hall, Upper Saddle River, NJ, 2nd edition, 1996.
  39. [41]Boris T. Polyak. Some methods of speeding up the convergence of iteration methods. USSR Computational Mathematics and Mathematical Physics, 4(5):1–17, 1964.DOI
  40. [42]Harald Pfifer and Peter Seiler. Integral quadratic constraints for delayed nonlinear and parameter-varying systems. Automatica, 56:36–43, 2015.DOI
  41. [43]Ellie Pond, Yichen Zhao, and Matthew Hale. A distributed asynchronous generalized momentum algorithm without delay bounds. arXiv preprint arXiv:2508.08218, 2025.arxiv.org/abs/2508.08218
  42. [44]Anders Rantzer. On the Kalman–Yakubovich–Popov lemma. Systems & Control Letters, 28(1):7–10, 1996.DOI
  43. [45]R. Tyrrell Rockafellar. Characterization of the subdifferentials of convex functions. Pacific Journal of Mathematics, 17(3):497–510, 1966.DOI
  44. [46]Saeed Soori, Bugra Can, Mert Gurbuzbalaban, and Maryam Mehri Dehnavi. ASYNC: A cloud engine with asynchrony and history for distributed machine learning. In 2020 IEEE International Parallel and Distributed Processing Symposium (IPDPS), pages 429–439, 2020.arxiv.org/abs/1907.08526
  45. [47]Issai Schur. Über potenzreihen, die im innern des einheitskreises beschränkt sind. Journal für die reine und angewandte Mathematik, 147:205–232, 1917.
  46. [48]Carsten Scherer and Christian Ebenbauer. Convex synthesis of accelerated gradient algorithms. SIAM Journal on Control and Optimization, 59(6):4615–4645, 2021.arxiv.org/abs/2102.06520
  47. [49]Carsten W. Scherer, Christian Ebenbauer, and Tobias Holicki. Optimization algorithm synthesis based on integral quadratic constraints: a tutorial. In the 62nd IEEE Conference on Decision and Control (CDC), pages 2995–3002. IEEE, 2023.arxiv.org/abs/2306.00565
  48. [50]Peter Seiler. Stability analysis with dissipation inequalities and integral quadratic constraints. IEEE Transactions on Automatic Control, 60(6):1704–1709, 2015.DOI
  49. [51]Sebastian U. Stich and Sai Praneeth Karimireddy. The error-feedback framework: Better rates for sgd with delayed gradients and compressed updates. Journal of Machine Learning Research, 21(237):1–36, 2020.
  50. [52]Saeed Soori, Konstantin Mishchenko, Aryan Mokhtari, Maryam Mehri Dehnavi, and Mert Gurbuzbalaban. DAve-QN: A distributed averaged quasi-newton method with local superlinear convergence rate. In International conference on artificial intelligence and statistics, pages 1965–1976. PMLR, 2020.
  51. [53]Richard S Varga. Matrix iterative analysis, volume 27. Springer Science & Business Media, 1999.
  52. [54]Bryan Van Scoy, Randy A. Freeman, and Kevin M. Lynch. The fastest known globally convergent first-order method for minimizing strongly convex functions. IEEE Control Systems Letters, 2(1):49–54, 2018.DOI
  53. [55]Bryan Van Scoy and Laurent Lessard. The fastest known first-order method for minimizing twice continuously differentiable smooth strongly convex functions. IEEE Control Systems Letters, 9:655–660, 2025.arxiv.org/abs/2506.01168
  54. [56]Jan C. Willems. Dissipative dynamical systems part I: General theory. Archive for Rational Mechanics and Analysis, 45:321–351, 1972.DOI
  55. [57]George Zames and Peter L. Falb. Stability conditions for systems with monotone and slope-restricted nonlinearities. SIAM Journal on Control, 6(1):89–108, 1968.DOI
  56. [58]Jingfan Zhang, Peter Seiler, and Joaquin Carrasco. Zames–Falb multipliers for convergence rate: motivating example and convex searches. International Journal of Control, 95(3):821–829, 2022.

Paper details

vlnc:W_e5we5at3
Version
v1 · latest
Published
Fresh — added to the Hub within the last 30 days
Added
License
CC BY 4.0
Usage

views · 0

downloads · 0

A view counts once per browser per day, a download each time the PDF is requested.

  • Mathematics

generative artificial intelligence (AI) tools

View statement
Looking for related papers…
Contents