betaFeedback

WorksMathematics

Proximal Adam algorithm for nonsmooth composite optimization and linear convergence under proximal PL geometry

Abstract

We analyze Adam-type methods for nonsmooth composite optimization problems min⁡xΦ(x)=f(x)+h(x)\min_xΦ(x)=f(x)+h(x), where ff is smooth and hh is convex, proper, and lower semicontinuous. The nonsmooth term is treated by a metric-weighted proximal map, while the smooth term is preconditioned by the Adam second-moment matrix. We prove a composite descent lemma whose constants depend explicitly on the spectral bounds of the adaptive metric and a residual–step equivalence that converts proximal step decay into subgradient stationarity. These yield an O(1/N)O(1/N) bound on the minimum proximal stationarity residual for the deterministic proximal normalized gradient method without convexity of ff, and the same O(1/N)O(1/N) rate for proximal Adam under quantitative momentum-alignment and residual-coherence conditions together with a metric-compatibility condition. By estimating the momentum error directly, the same O(1/N)O(1/N) rate is also obtained without the momentum-alignment and residual-coherence assumptions. When the composite objective satisfies a proximal Polyak–Łojasiewicz inequality, both algorithms and the relaxed proximal Adam scheme converge linearly with explicit contraction factors. Numerical experiments on a diagonal Lasso problem, two-dimensional image TV denoising, and two-dimensional elastic parameter identification illustrate the predicted stationarity and linear rates.

Paper details

vlnc:W_2dtnnn3j
Version
v1 · latest
Published
Fresh — added to the Hub within the last 30 days
Added
License
arXiv nonexclusive license
Usage

views · 0

downloads · 0

A view counts once per browser per day, a download each time the PDF is requested.

  • Mathematics

Not recorded for this paper

View statement
Looking for related papers…