betaFeedback

WorksMathematics

Simultaneous analysis of approximate leave-one-out cross-validation and mean-field inference II: General convex penalties

Abstract

For smooth penalties and group Lasso, Part I (https://arxiv.org/abs/2501.02624) analyzed approximate leave-one-out (ALO) cross-validation by comparing its weights with the trace correction used in mean-field inference. For a C2C^2 penalty RR, the matrix entering these corrections is A=[∇2R(b^)+XTDX]−1A=[\nabla^2R(\widehat b)+X^TDX]^{-1}, where b^\widehat b is the full-data estimator and DD is the diagonal matrix of fitted loss curvatures; for group Lasso, the penalty and loss curvatures are first restricted to the active support before taking the pseudoinverse. The obstacle to extending this argument to general convex penalties was the lack of an explicit formula for the matrix AA entering these corrections. For every finite-valued strongly convex penalty, we construct such a matrix from ordinary derivatives of the penalty response at the observed subgradient. Its projection–pseudoinverse formula separates the response space and penalty curvature, fixed by conditioning, from the explicit design dependence. This supplies the missing ingredient to extend the probabilistic argument of Part~I. The construction recovers the inverse Hessian for smooth penalties and the active-group inverse for group Lasso, including inactive boundary groups. In the proportional regime, for Gaussian designs with general positive-definite covariance and covariance-scaled strong convexity, we obtain unconditional bounds comparing both ALO and mean-field predictions with exact leave-one-out predictions in regression and single-index models. The expected average squared prediction error is O(n−1)O(n^{-1}), and the expected absolute criterion error is O(n−1/2)O(n^{-1/2}), under bounded, 1-Lipschitz loss scores and suitable test functions. The matrix and ALO weights use only observed data; the mean-field correction additionally uses the design covariance. The proofs are formalized in Lean~4.

Paper details

vlnc:W_e94d4k9t
Version
v1 · latest
Published
Fresh — added to the Hub within the last 30 days
Added
License
arXiv nonexclusive license
Usage

views · 9

downloads · 0

A view counts once per browser per day, a download each time the PDF is requested.

  • Mathematics

Not recorded for this paper

View statement
Looking for related papers…