WorksMathematics
Proximal Adam algorithm for nonsmooth composite optimization and linear convergence under proximal PL geometry
Abstract
We analyze Adam-type methods for nonsmooth composite optimization problems , where is smooth and is convex, proper, and lower semicontinuous. The nonsmooth term is treated by a metric-weighted proximal map, while the smooth term is preconditioned by the Adam second-moment matrix. We prove a composite descent lemma whose constants depend explicitly on the spectral bounds of the adaptive metric and a residual–step equivalence that converts proximal step decay into subgradient stationarity. These yield an bound on the minimum proximal stationarity residual for the deterministic proximal normalized gradient method without convexity of , and the same rate for proximal Adam under quantitative momentum-alignment and residual-coherence conditions together with a metric-compatibility condition. By estimating the momentum error directly, the same rate is also obtained without the momentum-alignment and residual-coherence assumptions. When the composite objective satisfies a proximal Polyak–Łojasiewicz inequality, both algorithms and the relaxed proximal Adam scheme converge linearly with explicit contraction factors. Numerical experiments on a diagonal Lasso problem, two-dimensional image TV denoising, and two-dimensional elastic parameter identification illustrate the predicted stationarity and linear rates.