betaFeedback

WorksMathematics

On the Nonoccurrence of Gradient Restart in Nesterov Accelerated Gradient Method

Abstract

Gradient restarting often improves the practical performance of accelerated gradient methods for unconstrained convex optimization problems, but a general theoretical guarantee of such improvement remains unclear. In this paper, we construct smooth, strongly convex objectives with Lipschitz-continuous gradients and corresponding initial points for which gradient restart never occurs, both for Nesterov's accelerated gradient method and for its ordinary differential equation model. From these examples, we conclude that strong convexity and smoothness alone do not guarantee that gradient restart improves the convergence behavior of the accelerated method or its continuous-time model. In particular, for the continuous-time example, the objective gap is asymptotic to 1/(πt3)1/(πt^3). We therefore obtain a negative answer to the open problem raised by Su, Boyd, and Cand{è}s [J. Mach. Learn. Res., 2016, 17(153), 1–43] of whether gradient restart guarantees linear convergence of the continuous dynamics for smooth strongly convex objectives.

Paper details

vlnc:W_x4m432m9
Version
v1 · latest
Published
Fresh — added to the Hub within the last 30 days
Added
License
arXiv nonexclusive license
Usage

views · 0

downloads · 0

A view counts once per browser per day, a download each time the PDF is requested.

  • Mathematics

GPT-6

View statement
Looking for related papers…