betaFeedback

WorksEconomics, Econometrics and Finance

Broken Adaptive Ridge in High-Dimensional Vector Autoregressions

Abstract

We study regularized estimation in high-dimensional vector autoregressions under physical dependence and row-wise approximate sparsity using the Broken Adaptive Ridge (BAR) estimator. BAR is an iteratively reweighted Ridge procedure whose fixed-point first-order conditions imply a coordinatewise thresholding rule that produces exact zeros. We show that the BAR estimation error can be analyzed on the union of the population support above the BAR threshold and the support selected in the sample. The size of this union can be controlled at the rate implied by approximate sparsity, allowing us to establish the required restricted-curvature condition and derive L1, L2, and prediction error rates when the number of regressors grows polynomially with the sample size and may exceed it. Under exact sparsity, the resulting rates recover the familiar rates for sparse high-dimensional estimation. Monte Carlo experiments across a range of high-dimensional VAR designs show that BAR delivers strong estimation and forecasting performance, and retains a high share of relevant coefficients relative to Lasso, adaptive Lasso, and SCAD. An application to U.S. inflation forecasting using FRED-MD shows particularly strong performance at the one-month horizon.

Introduction

The Lasso ([17]) solves a convex relaxation of the ideal but computationally infeasible ℓ0\ell_0-penalized least squares problem. This “convexification” yields tractability, but the linear ℓ1\ell_1 penalty also shrinks large coefficients, potentially introducing substantial bias on strong signals. Following its introduction, a large literature has developed methods aimed at reducing this shrinkage bias, including the adaptive Lasso of Zou [20] and folded-concave penalties such as SCAD ([7]). These approaches modify the uniform penalization induced by the Lasso, either through data-dependent weights or nonconvex penalty functions, so that sufficiently large coefficients are penalized less severely.

Broken Adaptive Ridge (BAR) ([6]) belongs broadly to this family of adaptive regularization methods, but with a different geometry. Instead of a weighted ℓ1\ell_1 penalty, BAR uses an adaptive quadratic penalty implemented through iteratively reweighted Ridge steps. Each reweighting step solves a strictly convex weighted-Ridge problem, and the resulting estimates determine the penalty weights used in the next iteration. The limiting estimator also inherits a Ridge-type grouping effect under strong collinearity. At the same time, the penalty curvature is highly heterogeneous: small coefficients face very steep curvature and are pushed toward zero, whereas large coefficients face almost no shrinkage. In this sense, BAR acts as a closer surrogate to the ideal ℓ0\ell_0 penalty, which primarily cares about whether a coefficient is active and leaves strong signals essentially unshrunk.

These features are particularly appealing in high-dimensional vector autoregressions (VARs), where strong collinearity across variables and across lags is the rule rather than the exception. In such settings, regularization should primarily decide which lagged effects enter the model, without unduly shrinking the dominant autoregressive coefficients. BAR is well suited to this task: strong own-lags are kept nearly unbiased, weak cross-lags are aggressively shrunk toward zero, and highly correlated lagged regressors are treated in a more grouped fashion.

By contrast, even adaptive ℓ1\ell_1 procedures remain tied to an ℓ1\ell_1 geometry, which can still induce substantial shrinkage bias on dominant autoregressive effects.

Regularization of large VARs has been developed primarily around ℓ1\ell_1-based methods. Basu & Michailidis [1] establish high-dimensional estimation bounds for sparse VARs under temporal dependence, while Kock & Callot [11] derive oracle inequalities for the Lasso and adaptive Lasso. Nicholson et al. [16] develop hierarchical group-Lasso penalties for high-dimensional VAR forecasting, allowing the lag structure itself to be regularized. Related sparsity concepts for high-dimensional VARs are discussed by Krampe & Paparoditis [12]. BAR itself was introduced by Dai et al. [6], with the scalable coordinatewise cycBAR implementation developed by Kawaguchi et al. [10]. BAR has also been considered by Huang et al. [9], specifically for model selection in increasing-order sparse VARs under full-rank design conditions, a substantially different setting compared to ours.

This paper develops the theory of BAR estimation for high-dimensional VARs, allowing for both temporal and cross-sectional dependence. Under physical dependence of the underlying process and row-wise weak sparsity of the VAR coefficient matrices, we derive high-probability prediction and ℓ1/ℓ2\ell_1/\ell_2 estimation error bounds under polynomial dimensional growth, including regimes in which the number of regressors exceeds the sample size. The analysis exploits a feature specific to BAR: its fixed-point thresholding rule allows the estimation error to be analyzed on the union of coefficients above the BAR threshold in the population and those selected by BAR in the sample. We show that the size of this union can be controlled at the relevant sparsity rate, after which restricted curvature is established on this set. This differs from standard ℓ1\ell_1-based approaches, where curvature is typically imposed over a cone of directions. Weak sparsity allows many small nonzero coefficients, shifting attention from exact support recovery to estimation and prediction.

Computationally, we adapt the “cycBAR” algorithm of Kawaguchi et al. [10] to the high-dimensional VAR setting. Its coordinatewise thresholding updates avoid repeated inversion of large Gram matrices and remain feasible when the number of regressors exceeds the sample size. Monte Carlo experiments across high-dimensional VAR designs show that BAR reduces shrinkage bias on large coefficients and performs strongly in estimation and prediction relative to Lasso, adaptive Lasso, and SCAD.

Finally, we illustrate the method in an application to U.S. inflation forecasting. The empirical exercise is consistent with the advantages suggested by the theory and simulations: BAR yields a stable sparse representation and delivers competitive out-of-sample forecasts relative to standard penalized VAR approaches, with particularly strong performance at the one-month horizon.

A few notational conventions are used throughout. Bold lowercase and uppercase letters denote vectors and matrices. For a matrix M\mathbf{M}, ∥M∥max⁡\|\mathbf{M}\|_{\max} and ∥M∥op\|\mathbf{M}\|_{\mathrm{op}} denote the elementwise sup- and operator norms, and λmin⁡(M)\lambda_{\min}(\mathbf{M}) and λmax⁡(M)\lambda_{\max}(\mathbf{M}) its extreme eigenvalues. For a vector v\mathbf{v}, ∥v∥r\|\mathbf{v}\|_{r} is the usual ℓr\ell_r norm and ∥v∥0\|\mathbf{v}\|_{0} its number of nonzero entries. For a scalar random variable ZZ, ∥Z∥Lq:=(E∣Z∣q)1/q\|Z\|_{L_q}:=(\mathbb{E}|Z|^q)^{1/q} denotes its LqL_q norm. We write aT≲bTa_T \lesssim b_T if there exists a constant C>0C > 0, independent of (K,T)(K,T), such that aT≤CbTa_T \le Cb_T uniformly in (K,T)(K,T) while aT≍bTa_T \asymp b_T if both inequalities hold. All asymptotic statements are joint in K,T→∞K,T \to\infty unless stated otherwise.

Modeling Set up

Let y1,…,yT\mathbf{y}_1,\ldots,\mathbf{y}_T be a KK-dimensional multiple time series process, where yt=(y1,t,…,yK,t)′\mathbf{y}_t=(y_{1,t},\ldots,y_{K,t})^\prime. We assume yt\mathbf{y}_t is generated by a stable VAR(pp) process as

yt=A1yt−1+⋯+Apyt−p+ut,t=p+1,…,T,(1)\mathbf{y}_t=\mathbf{A}_1\mathbf{y}_{t-1}+\cdots+\mathbf{A}_p\mathbf{y}_{t-p}+\mathbf{u}_t,\qquad t=p+1,\ldots,T, \tag*{(1)}

where A1,…,Ap\mathbf{A}_1,\ldots,\mathbf{A}_p are K×KK \times K transition matrices and ut\mathbf{u}_t is a K×1K \times1 vector of zero mean error terms with covariance Σu=E(utut′)\boldsymbol{\Sigma}_u = \mathbb{E}(\mathbf{u}_t\mathbf{u}_t').2 We omit both the intercept and any polynomial time trend from the model to ease the notation, but the results we derive easily extend to those cases as well. The following assumptions are satisfied by the VAR in (1).

Assumption 1 (Data generating process (DGP)). Let ht:=(yt′,ut′)′∈R2K\mathbf{h}_t := (\mathbf{y}_t',\mathbf{u}_t')' \in\mathbb{R}^{2K}, t∈Zt \in\mathbb{Z}, be covariance stationary. Suppose that there exist constants q>4q > 4, C>0C > 0, and α>0\alpha> 0, with α≠1/2−2/q\alpha\ne1/2 - 2/q, such that:

(i) E(ht)=0\mathbb{E}(\mathbf{h}_t) = 0; E(ut∣yt−1,…,yt−p)=0\mathbb{E}(\mathbf{u}_t \mid\mathbf{y}_{t-1},\ldots,\mathbf{y}_{t-p}) = 0; max⁡1≤j≤2KE(∣hj,t∣q)≤C\max_{1 \le j \le2K} \mathbb{E}(|h_{j,t}|^q) \le C.

(ii) The process is causally generated as ht=G(…,et−1,et)=G(Ft)\mathbf{h}_t = G(\ldots,\mathbf{e}_{t-1},\mathbf{e}_t) = G(\mathcal{F}^t), where {et}t∈Z\{\mathbf{e}_t\}_{t \in\mathbb{Z}} is an iid sequence in R2K\mathbb{R}^{2K} and G:E∞→R2KG:\mathcal{E}^{\infty} \to\mathbb{R}^{2K} is measurable.3

(iii) Let Ft,0:=(…,e−1,e0∗,e1,…,et)\mathcal{F}^{t,0} := (\ldots,\mathbf{e}_{-1},\mathbf{e}_0^*,\mathbf{e}_1,\ldots,\mathbf{e}_t), where e0∗\mathbf{e}_0^* is an independent copy of e0\mathbf{e}_0, and set ht,0:=G(Ft,0)\mathbf{h}_{t,0} := G(\mathcal{F}^{t,0}). For j=1,…,2Kj = 1,\ldots,2K, define δq,j,t:=∥hj,t−hj,t,0∥Lq\delta_{q,j,t} := \lVert h_{j,t}-h_{j,t,0}\rVert_{L_q}, Δq,j,m:=∑t=m∞δq,j,t\Delta_{q,j,m} := \sum_{t=m}^{\infty}\delta_{q,j,t}, and ∥hj,∙∥q,α:=sup⁡m≥0(m+1)αΔq,j,m\lVert h_{j,\bullet}\rVert_{q,\alpha} := \sup_{m \ge0}(m+1)^\alpha\Delta_{q,j,m}, the latter being the dependence adjusted qq-norm (DANq_q) for the process hj,∙={hj,t}t∈Zh_{j,\bullet} = \{h_{j,t}\}_{t \in\mathbb{Z}}, with Ψq,α:=max⁡j∥hj,∙∥q,α\Psi_{q,\alpha} := \max_j \lVert h_{j,\bullet}\rVert_{q,\alpha}. To control the largest coupling effect across the system, also let ωq,t:=∥ht−ht,0∥∞,Lq\omega_{q,t} := \lVert\mathbf{h}_t-\mathbf{h}_{t,0}\rVert_{\infty,L_q}, Ωq,m:=∑t=m∞ωq,t\Omega_{q,m} := \sum_{t=m}^{\infty}\omega_{q,t}, and Hq,α,T:=sup⁡m≥0(m+1)αΩq,m\mathcal{H}_{q,\alpha,T} := \sup_{m \ge0}(m+1)^\alpha\Omega_{q,m}. Set r:=q/2>2r := q/2 > 2 and let α1:=1\alpha_1 := 1 if α>1/2−1/r\alpha> 1/2 - 1/r, while α1:=r/2−rα\alpha_1 := r/2-r\alpha if 0<α<1/2−1/r0 < \alpha< 1/2 - 1/r. Assume

Ψq,0∨Ψq,α≤C,(2)\Psi_{q,0} \vee\Psi_{q,\alpha} \le C, \tag*{(2)}
Hq,0,THq,α,Tlog⁡(Kp+1)=o(T1/2−α1/r).(3)\mathcal{H}_{q,0,T}\mathcal{H}_{q,\alpha,T}\log(Kp+1)=o\left(T^{1/2-\alpha_1/r}\right). \tag*{(3)}

(iv) The number of regressors in each equation satisfies Kp=O(Tν)Kp = O(T^\nu) for some fixed ν>0\nu> 0.

Assumption 1(i) imposes conditional mean-zero innovations and uniformly bounded qqth moments. Parts (ii)–(iii) formulate short-range temporal dependence through the physical-dependence framework of Wu [18], under which the stationary causal process is also ergodic. This framework is useful because it accommodates a broad range of nonlinear and conditionally heteroskedastic time-series dynamics while retaining a tractable innovation-based measure of dependence.4 The coupling mechanism replaces one innovation by an independent copy and measures how strongly this perturbation propagates through the future of the process. Boundedness of Ψq,α\Psi_{q,\alpha} in (2) implies the remaining cumulative effect must decay at least polynomially, uniformly over the individual coordinates of the process. The quantity Hq,α,T\mathcal{H}_{q,\alpha,T}, which is allowed to grow with the dimension, strengthens this coordinatewise control by tracking the largest coupling effect across the entire system. This additional control is needed for the joint high-dimensional concentration argument used later for the empirical process and sample Gram matrix; see Lemma 2. Condition (3) restricts the combined growth of the cumulative maximal coupling effect Hq,0,T\mathcal{H}_{q,0,T}, its temporal persistence as measured by Hq,α,T\mathcal{H}_{q,\alpha,T}, and the dimension through log⁡(Kp+1)\log(Kp+1), relative to the sample size. This sufficient condition is adapted from the vector-valued maximal inequality of Zhang & Wu [19] and its second-order implementation in Margaritella & Sessinou [13]. Part (iv) permits polynomial dimensional growth and does not require Kp<TKp<T; in particular, Kp/T→∞Kp/T \to\infty is allowed when ν>1\nu>1. Because pp is fixed (see Remark 1 below), all dimensional growth is driven by KK. A fuller discussion, including a simple coupling example and the relation between coordinatewise and maximal dependence, is given in Appendix § 8.1.1.

Remark 1 (VAR order pp). We take pp to be fixed, known, and small. As usual, a finite-order VAR may be viewed as a working approximation to a more general VAR(∞\infty) representation; our theoretical results are conditional on the chosen truncation order and do not address the approximation error from omitted higher lags. Keeping pp small is nevertheless natural in

high-dimensional VARs because cross-sectional propagation already generates rich marginal dynamics: the final-equation representation implies that a VAR(pp) induces univariate ARMA orders up to (Kp,(K−1)p)(Kp,(K-1)p), see Cubadda et al. [5] and the references therein. Thus, even p=1p=1 can accommodate substantial persistence when KK is large.

Assumption 2 (Row-wise Approximate Sparsity). Each row of the concatenated coefficient matrix A:=[A1 ⋯ Ap]∈RK×Kp\mathcal{A}:=[A_1\ \cdots\ A_p]\in\mathbb{R}^{K\times Kp} is approximately sparse, that is, for i=1,…,Ki=1,\ldots,K,

max⁡i∑j=1Kp∣Aij∣qs=max⁡i∑ℓ=1p∑j=1K∣Aℓ,ij∣qs≤s0,qs∈[0,1).\max_i\sum_{j=1}^{Kp}|\mathcal{A}_{ij}|^{q_s}=\max_i\sum_{\ell=1}^{p}\sum_{j=1}^{K}|A_{\ell,ij}|^{q_s}\le s_0,\quad q_s\in[0,1).

Assumption 2 allows for approximate, rather than exact, row-wise sparsity. When qs=0q_s=0, it reduces to exact sparsity, with s0s_0 bounding the number of nonzero coefficients in each row of A\mathcal{A}. For qs>0q_s>0, coefficients need not be exactly zero, and s0s_0 instead controls the overall weak-sparsity level of each row: for fixed qsq_s, larger values of s0s_0 allow more or larger non-negligible coefficients. The parameter qsq_s determines how much of this signal may be spread over small coefficients, with smaller values corresponding to stronger sparsity restrictions. In the VAR setting, this row-wise formulation implies that each variable can be strongly influenced by only a limited number of the KpKp lagged regressors, leading to an approximately sparse interaction structure (see also Krampe & Paparoditis [12]).

References

  1. [1]Basu, S. & Michailidis, G. (2015), ‘Regularized estimation in sparse high-dimensional time series models’, The Annals of Statistics 43(4), 1535–1567.DOI
  2. [2]Blumensath, T. & Davies, M. E. (2009), ‘Iterative hard thresholding for compressed sensing’, Applied and Computational Harmonic Analysis 27(3), 265–274.arxiv.org/abs/0805.0510
  3. [3]Chen, J. & Chen, Z. (2008), ‘Extended bayesian information criteria for model selection with large model spaces’, Biometrika 95(3), 759–771.DOI
  4. [4]Chernozhukov, V., Härdle, W. K., Huang, C. & Wang, W. (2021), ‘Lasso-driven inference in time and space’, The Annals of Statistics 49(3), 1702–1735.
  5. [5]Cubadda, G., Hecq, A. & Palm, F. (2009), ‘Studying co-movements in large multivariate data prior to multivariate modelling’, Journal of Econometrics 148(1), 25–35.
  6. [6]Dai, L., Chen, K., Sun, Z., Liu, Z. & Li, G. (2018), ‘Broken adaptive ridge regression and its asymptotic properties’, Journal of Multivariate Analysis 168, 334–351.DOI
  7. [7]Fan, J. & Li, R. (2001), ‘Variable selection via nonconcave penalized likelihood and its oracle properties’, Journal of the American Statistical Association 96(456), 1348–1360.
  8. [8]Hansen, P. R., Lunde, A. & Nason, J. M. (2011), ‘The model confidence set’, Econometrica 79(2), 453–497.
  9. [9]Huang, J., Li, B. & Xu, X. (2025), ‘Model selection for vector autoregressive processes using broken adaptive ridge’, Statistica Neerlandica 79(4), e70021.DOI
  10. [10]Kawaguchi, E. S., Shen, J. I., Suchard, M. A. & Li, G. (2021), ‘Scalable algorithms for large competing risks data’, Journal of Computational and Graphical Statistics 30(3), 685–693.DOI
  11. [11]Kock, A. B. & Callot, L. (2015), ‘Oracle inequalities for high dimensional vector autoregressions’, Journal of Econometrics 186(2), 325–344.
  12. [12]Krampe, J. & Paparoditis, E. (2021), ‘Sparsity concepts and estimation procedures for high-dimensional vector autoregressive models’, Journal of Time Series Analysis 42(5-6), 554–579.DOI
  13. [13]Margaritella, L. & Sessiniou, R. (2025), ‘Precision least squares: Estimation and inference in high-dimensions’, Journal of Business & Economic Statistics 43(4), 884–896.
  14. [14]McCracken, M. W. & Ng, S. (2016), ‘Fred-md: A monthly database for macroeconomic research’, Journal of Business & Economic Statistics 34(4), 574–589.
  15. [15]Medeiros, M. C., Vasconcelos, G. F. R., Álvaro Veiga & Zilberman, E. (2021), ‘Forecasting inflation in a data-rich environment: The benefits of machine learning methods’, Journal of Business & Economic Statistics 39(1), 98–119.
  16. [16]Nicholson, W. B., Wilms, I., Bien, J. & Matteson, D. S. (2020), ‘High dimensional forecasting via interpretable vector autoregression’, Journal of Machine Learning Research 21(166), 1–52.
  17. [17]Tibshirani, R. (1996), ‘Regression shrinkage and selection via the lasso’, Journal of the Royal Statistical Society Series B: Statistical Methodology 58(1), 267–288.
  18. [18]Wu, W. B. (2005), ‘Nonlinear system theory: Another look at dependence’, Proceedings of the National Academy of Sciences 102(40), 14150–14154.pubmed.ncbi.nlm.nih.gov/16179388
  19. [19]Zhang, D. & Wu, W. B. (2017), ‘Gaussian approximation for high dimensional time series’, The Annals of Statistics 45(5), 1895–1919.arxiv.org/abs/1508.07036
  20. [20]Zou, H. (2006), ‘The adaptive lasso and its oracle properties’, Journal of the American Statistical Association 101(476), 1418–1429.DOI

Paper details

vlnc:W_pasv2mms
Version
v1 · latest
Published
Fresh — added to the Hub within the last 30 days
Added
License
CC BY 4.0
Usage

views · 0

downloads · 0

A view counts once per browser per day, a download each time the PDF is requested.

  • Economics, Econometrics and Finance

Not recorded for this paper

View statement
Looking for related papers…
Contents