Introduction
The Lasso ([17 ] ) solves a convex relaxation of the ideal but computationally infeasible ℓ 0 \ell_0 ℓ 0 -penalized least squares problem. This “convexification” yields tractability, but the linear ℓ 1 \ell_1 ℓ 1 penalty also shrinks large coefficients, potentially introducing substantial bias on strong signals. Following its introduction, a large literature has developed methods aimed at reducing this shrinkage bias, including the adaptive Lasso of Zou [20 ] and folded-concave penalties such as SCAD ([7 ] ). These approaches modify the uniform penalization induced by the Lasso, either through data-dependent weights or nonconvex penalty functions, so that sufficiently large coefficients are penalized less severely.
Broken Adaptive Ridge (BAR) ([6 ] ) belongs broadly to this family of adaptive regularization methods, but with a different geometry. Instead of a weighted ℓ 1 \ell_1 ℓ 1 penalty, BAR uses an adaptive quadratic penalty implemented through iteratively reweighted Ridge steps. Each reweighting step solves a strictly convex weighted-Ridge problem, and the resulting estimates determine the penalty weights used in the next iteration. The limiting estimator also inherits a Ridge-type grouping effect under strong collinearity. At the same time, the penalty curvature is highly heterogeneous: small coefficients face very steep curvature and are pushed toward zero, whereas large coefficients face almost no shrinkage. In this sense, BAR acts as a closer surrogate to the ideal ℓ 0 \ell_0 ℓ 0 penalty, which primarily cares about whether a coefficient is active and leaves strong signals essentially unshrunk.
These features are particularly appealing in high-dimensional vector autoregressions (VARs), where strong collinearity across variables and across lags is the rule rather than the exception. In such settings, regularization should primarily decide which lagged effects enter the model, without unduly shrinking the dominant autoregressive coefficients. BAR is well suited to this task: strong own-lags are kept nearly unbiased, weak cross-lags are aggressively shrunk toward zero, and highly correlated lagged regressors are treated in a more grouped fashion.
By contrast, even adaptive ℓ 1 \ell_1 ℓ 1 procedures remain tied to an ℓ 1 \ell_1 ℓ 1 geometry, which can still induce substantial shrinkage bias on dominant autoregressive effects.
Regularization of large VARs has been developed primarily around ℓ 1 \ell_1 ℓ 1 -based methods. Basu & Michailidis [1 ] establish high-dimensional estimation bounds for sparse VARs under temporal dependence, while Kock & Callot [11 ] derive oracle inequalities for the Lasso and adaptive Lasso. Nicholson et al. [16 ] develop hierarchical group-Lasso penalties for high-dimensional VAR forecasting, allowing the lag structure itself to be regularized. Related sparsity concepts for high-dimensional VARs are discussed by Krampe & Paparoditis [12 ] . BAR itself was introduced by Dai et al. [6 ] , with the scalable coordinatewise cycBAR implementation developed by Kawaguchi et al. [10 ] . BAR has also been considered by Huang et al. [9 ] , specifically for model selection in increasing-order sparse VARs under full-rank design conditions, a substantially different setting compared to ours.
This paper develops the theory of BAR estimation for high-dimensional VARs, allowing for both temporal and cross-sectional dependence. Under physical dependence of the underlying process and row-wise weak sparsity of the VAR coefficient matrices, we derive high-probability prediction and ℓ 1 / ℓ 2 \ell_1/\ell_2 ℓ 1 / ℓ 2 estimation error bounds under polynomial dimensional growth, including regimes in which the number of regressors exceeds the sample size. The analysis exploits a feature specific to BAR: its fixed-point thresholding rule allows the estimation error to be analyzed on the union of coefficients above the BAR threshold in the population and those selected by BAR in the sample. We show that the size of this union can be controlled at the relevant sparsity rate, after which restricted curvature is established on this set. This differs from standard ℓ 1 \ell_1 ℓ 1 -based approaches, where curvature is typically imposed over a cone of directions. Weak sparsity allows many small nonzero coefficients, shifting attention from exact support recovery to estimation and prediction.
Computationally, we adapt the “cycBAR” algorithm of Kawaguchi et al. [10 ] to the high-dimensional VAR setting. Its coordinatewise thresholding updates avoid repeated inversion of large Gram matrices and remain feasible when the number of regressors exceeds the sample size. Monte Carlo experiments across high-dimensional VAR designs show that BAR reduces shrinkage bias on large coefficients and performs strongly in estimation and prediction relative to Lasso, adaptive Lasso, and SCAD.
Finally, we illustrate the method in an application to U.S. inflation forecasting. The empirical exercise is consistent with the advantages suggested by the theory and simulations: BAR yields a stable sparse representation and delivers competitive out-of-sample forecasts relative to standard penalized VAR approaches, with particularly strong performance at the one-month horizon.
A few notational conventions are used throughout. Bold lowercase and uppercase letters denote vectors and matrices. For a matrix M \mathbf{M} M , ∥ M ∥ max \|\mathbf{M}\|_{\max} ∥ M ∥ m a x and ∥ M ∥ o p \|\mathbf{M}\|_{\mathrm{op}} ∥ M ∥ op denote the elementwise sup- and operator norms, and λ min ( M ) \lambda_{\min}(\mathbf{M}) λ m i n ( M ) and λ max ( M ) \lambda_{\max}(\mathbf{M}) λ m a x ( M ) its extreme eigenvalues. For a vector v \mathbf{v} v , ∥ v ∥ r \|\mathbf{v}\|_{r} ∥ v ∥ r is the usual ℓ r \ell_r ℓ r norm and ∥ v ∥ 0 \|\mathbf{v}\|_{0} ∥ v ∥ 0 its number of nonzero entries. For a scalar random variable Z Z Z , ∥ Z ∥ L q : = ( E ∣ Z ∣ q ) 1 / q \|Z\|_{L_q}:=(\mathbb{E}|Z|^q)^{1/q} ∥ Z ∥ L q := ( E ∣ Z ∣ q ) 1/ q denotes its L q L_q L q norm. We write a T ≲ b T a_T \lesssim b_T a T ≲ b T if there exists a constant C > 0 C > 0 C > 0 , independent of ( K , T ) (K,T) ( K , T ) , such that a T ≤ C b T a_T \le Cb_T a T ≤ C b T uniformly in ( K , T ) (K,T) ( K , T ) while a T ≍ b T a_T \asymp b_T a T ≍ b T if both inequalities hold. All asymptotic statements are joint in K , T → ∞ K,T \to\infty K , T → ∞ unless stated otherwise.
Modeling Set up
Let y 1 , … , y T \mathbf{y}_1,\ldots,\mathbf{y}_T y 1 , … , y T be a K K K -dimensional multiple time series process, where y t = ( y 1 , t , … , y K , t ) ′ \mathbf{y}_t=(y_{1,t},\ldots,y_{K,t})^\prime y t = ( y 1 , t , … , y K , t ) ′ . We assume y t \mathbf{y}_t y t is generated by a stable VAR(p p p ) process as
y t = A 1 y t − 1 + ⋯ + A p y t − p + u t , t = p + 1 , … , T , (1) \mathbf{y}_t=\mathbf{A}_1\mathbf{y}_{t-1}+\cdots+\mathbf{A}_p\mathbf{y}_{t-p}+\mathbf{u}_t,\qquad t=p+1,\ldots,T, \tag*{(1)} y t = A 1 y t − 1 + ⋯ + A p y t − p + u t , t = p + 1 , … , T , (1)
where A 1 , … , A p \mathbf{A}_1,\ldots,\mathbf{A}_p A 1 , … , A p are K × K K \times K K × K transition matrices and u t \mathbf{u}_t u t is a K × 1 K \times1 K × 1 vector of zero mean error terms with covariance Σ u = E ( u t u t ′ ) \boldsymbol{\Sigma}_u = \mathbb{E}(\mathbf{u}_t\mathbf{u}_t') Σ u = E ( u t u t ′ ) .2 We omit both the intercept and any polynomial time trend from the model to ease the notation, but the results we derive easily extend to those cases as well. The following assumptions are satisfied by the VAR in (1) .
Assumption 1 (Data generating process (DGP)). Let h t : = ( y t ′ , u t ′ ) ′ ∈ R 2 K \mathbf{h}_t := (\mathbf{y}_t',\mathbf{u}_t')' \in\mathbb{R}^{2K} h t := ( y t ′ , u t ′ ) ′ ∈ R 2 K , t ∈ Z t \in\mathbb{Z} t ∈ Z , be covariance stationary. Suppose that there exist constants q > 4 q > 4 q > 4 , C > 0 C > 0 C > 0 , and α > 0 \alpha> 0 α > 0 , with α ≠ 1 / 2 − 2 / q \alpha\ne1/2 - 2/q α = 1/2 − 2/ q , such that:
(i) E ( h t ) = 0 \mathbb{E}(\mathbf{h}_t) = 0 E ( h t ) = 0 ; E ( u t ∣ y t − 1 , … , y t − p ) = 0 \mathbb{E}(\mathbf{u}_t \mid\mathbf{y}_{t-1},\ldots,\mathbf{y}_{t-p}) = 0 E ( u t ∣ y t − 1 , … , y t − p ) = 0 ; max 1 ≤ j ≤ 2 K E ( ∣ h j , t ∣ q ) ≤ C \max_{1 \le j \le2K} \mathbb{E}(|h_{j,t}|^q) \le C max 1 ≤ j ≤ 2 K E ( ∣ h j , t ∣ q ) ≤ C .
(ii) The process is causally generated as h t = G ( … , e t − 1 , e t ) = G ( F t ) \mathbf{h}_t = G(\ldots,\mathbf{e}_{t-1},\mathbf{e}_t) = G(\mathcal{F}^t) h t = G ( … , e t − 1 , e t ) = G ( F t ) , where { e t } t ∈ Z \{\mathbf{e}_t\}_{t \in\mathbb{Z}} { e t } t ∈ Z is an iid sequence in R 2 K \mathbb{R}^{2K} R 2 K and G : E ∞ → R 2 K G:\mathcal{E}^{\infty} \to\mathbb{R}^{2K} G : E ∞ → R 2 K is measurable.3
(iii) Let F t , 0 : = ( … , e − 1 , e 0 ∗ , e 1 , … , e t ) \mathcal{F}^{t,0} := (\ldots,\mathbf{e}_{-1},\mathbf{e}_0^*,\mathbf{e}_1,\ldots,\mathbf{e}_t) F t , 0 := ( … , e − 1 , e 0 ∗ , e 1 , … , e t ) , where e 0 ∗ \mathbf{e}_0^* e 0 ∗ is an independent copy of e 0 \mathbf{e}_0 e 0 , and set h t , 0 : = G ( F t , 0 ) \mathbf{h}_{t,0} := G(\mathcal{F}^{t,0}) h t , 0 := G ( F t , 0 ) . For j = 1 , … , 2 K j = 1,\ldots,2K j = 1 , … , 2 K , define δ q , j , t : = ∥ h j , t − h j , t , 0 ∥ L q \delta_{q,j,t} := \lVert h_{j,t}-h_{j,t,0}\rVert_{L_q} δ q , j , t := ∥ h j , t − h j , t , 0 ∥ L q , Δ q , j , m : = ∑ t = m ∞ δ q , j , t \Delta_{q,j,m} := \sum_{t=m}^{\infty}\delta_{q,j,t} Δ q , j , m := ∑ t = m ∞ δ q , j , t , and ∥ h j , ∙ ∥ q , α : = sup m ≥ 0 ( m + 1 ) α Δ q , j , m \lVert h_{j,\bullet}\rVert_{q,\alpha} := \sup_{m \ge0}(m+1)^\alpha\Delta_{q,j,m} ∥ h j , ∙ ∥ q , α := sup m ≥ 0 ( m + 1 ) α Δ q , j , m , the latter being the dependence adjusted q q q -norm (DANq _q q ) for the process h j , ∙ = { h j , t } t ∈ Z h_{j,\bullet} = \{h_{j,t}\}_{t \in\mathbb{Z}} h j , ∙ = { h j , t } t ∈ Z , with Ψ q , α : = max j ∥ h j , ∙ ∥ q , α \Psi_{q,\alpha} := \max_j \lVert h_{j,\bullet}\rVert_{q,\alpha} Ψ q , α := max j ∥ h j , ∙ ∥ q , α . To control the largest coupling effect across the system, also let ω q , t : = ∥ h t − h t , 0 ∥ ∞ , L q \omega_{q,t} := \lVert\mathbf{h}_t-\mathbf{h}_{t,0}\rVert_{\infty,L_q} ω q , t := ∥ h t − h t , 0 ∥ ∞ , L q , Ω q , m : = ∑ t = m ∞ ω q , t \Omega_{q,m} := \sum_{t=m}^{\infty}\omega_{q,t} Ω q , m := ∑ t = m ∞ ω q , t , and H q , α , T : = sup m ≥ 0 ( m + 1 ) α Ω q , m \mathcal{H}_{q,\alpha,T} := \sup_{m \ge0}(m+1)^\alpha\Omega_{q,m} H q , α , T := sup m ≥ 0 ( m + 1 ) α Ω q , m . Set r : = q / 2 > 2 r := q/2 > 2 r := q /2 > 2 and let α 1 : = 1 \alpha_1 := 1 α 1 := 1 if α > 1 / 2 − 1 / r \alpha> 1/2 - 1/r α > 1/2 − 1/ r , while α 1 : = r / 2 − r α \alpha_1 := r/2-r\alpha α 1 := r /2 − r α if 0 < α < 1 / 2 − 1 / r 0 < \alpha< 1/2 - 1/r 0 < α < 1/2 − 1/ r . Assume
Ψ q , 0 ∨ Ψ q , α ≤ C , (2) \Psi_{q,0} \vee\Psi_{q,\alpha} \le C, \tag*{(2)} Ψ q , 0 ∨ Ψ q , α ≤ C , (2) H q , 0 , T H q , α , T log ( K p + 1 ) = o ( T 1 / 2 − α 1 / r ) . (3) \mathcal{H}_{q,0,T}\mathcal{H}_{q,\alpha,T}\log(Kp+1)=o\left(T^{1/2-\alpha_1/r}\right). \tag*{(3)} H q , 0 , T H q , α , T log ( K p + 1 ) = o ( T 1/2 − α 1 / r ) . (3) (iv) The number of regressors in each equation satisfies K p = O ( T ν ) Kp = O(T^\nu) K p = O ( T ν ) for some fixed ν > 0 \nu> 0 ν > 0 .
Assumption 1(i) imposes conditional mean-zero innovations and uniformly bounded q q q th moments. Parts (ii)–(iii) formulate short-range temporal dependence through the physical-dependence framework of Wu [18 ] , under which the stationary causal process is also ergodic. This framework is useful because it accommodates a broad range of nonlinear and conditionally heteroskedastic time-series dynamics while retaining a tractable innovation-based measure of dependence.4 The coupling mechanism replaces one innovation by an independent copy and measures how strongly this perturbation propagates through the future of the process. Boundedness of Ψ q , α \Psi_{q,\alpha} Ψ q , α in (2) implies the remaining cumulative effect must decay at least polynomially, uniformly over the individual coordinates of the process. The quantity H q , α , T \mathcal{H}_{q,\alpha,T} H q , α , T , which is allowed to grow with the dimension, strengthens this coordinatewise control by tracking the largest coupling effect across the entire system. This additional control is needed for the joint high-dimensional concentration argument used later for the empirical process and sample Gram matrix; see Lemma 2. Condition (3) restricts the combined growth of the cumulative maximal coupling effect H q , 0 , T \mathcal{H}_{q,0,T} H q , 0 , T , its temporal persistence as measured by H q , α , T \mathcal{H}_{q,\alpha,T} H q , α , T , and the dimension through log ( K p + 1 ) \log(Kp+1) log ( K p + 1 ) , relative to the sample size. This sufficient condition is adapted from the vector-valued maximal inequality of Zhang & Wu [19 ] and its second-order implementation in Margaritella & Sessinou [13 ] . Part (iv) permits polynomial dimensional growth and does not require K p < T Kp<T K p < T ; in particular, K p / T → ∞ Kp/T \to\infty K p / T → ∞ is allowed when ν > 1 \nu>1 ν > 1 . Because p p p is fixed (see Remark 1 below), all dimensional growth is driven by K K K . A fuller discussion, including a simple coupling example and the relation between coordinatewise and maximal dependence, is given in Appendix § 8 .1.1.
high-dimensional VARs because cross-sectional propagation already generates rich marginal dynamics: the final-equation representation implies that a VAR(p p p ) induces univariate ARMA orders up to ( K p , ( K − 1 ) p ) (Kp,(K-1)p) ( K p , ( K − 1 ) p ) , see Cubadda et al. [5 ] and the references therein. Thus, even p = 1 p=1 p = 1 can accommodate substantial persistence when K K K is large.
Assumption 2 (Row-wise Approximate Sparsity). Each row of the concatenated coefficient matrix A : = [ A 1 ⋯ A p ] ∈ R K × K p \mathcal{A}:=[A_1\ \cdots\ A_p]\in\mathbb{R}^{K\times Kp} A := [ A 1 ⋯ A p ] ∈ R K × K p is approximately sparse, that is, for i = 1 , … , K i=1,\ldots,K i = 1 , … , K ,
max i ∑ j = 1 K p ∣ A i j ∣ q s = max i ∑ ℓ = 1 p ∑ j = 1 K ∣ A ℓ , i j ∣ q s ≤ s 0 , q s ∈ [ 0 , 1 ) . \max_i\sum_{j=1}^{Kp}|\mathcal{A}_{ij}|^{q_s}=\max_i\sum_{\ell=1}^{p}\sum_{j=1}^{K}|A_{\ell,ij}|^{q_s}\le s_0,\quad q_s\in[0,1). i max j = 1 ∑ K p ∣ A ij ∣ q s = i max ℓ = 1 ∑ p j = 1 ∑ K ∣ A ℓ , ij ∣ q s ≤ s 0 , q s ∈ [ 0 , 1 ) .
Assumption 2 allows for approximate, rather than exact, row-wise sparsity. When q s = 0 q_s=0 q s = 0 , it reduces to exact sparsity, with s 0 s_0 s 0 bounding the number of nonzero coefficients in each row of A \mathcal{A} A . For q s > 0 q_s>0 q s > 0 , coefficients need not be exactly zero, and s 0 s_0 s 0 instead controls the overall weak-sparsity level of each row: for fixed q s q_s q s , larger values of s 0 s_0 s 0 allow more or larger non-negligible coefficients. The parameter q s q_s q s determines how much of this signal may be spread over small coefficients, with smaller values corresponding to stronger sparsity restrictions. In the VAR setting, this row-wise formulation implies that each variable can be strongly influenced by only a limited number of the K p Kp K p lagged regressors, leading to an approximately sparse interaction structure (see also Krampe & Paparoditis [12 ] ).
Because BAR is applied equation-by-equation, we shall without loss of generality focus on a single equation of the system in (1) , i.e., the i i i -th component of y t \boldsymbol{y}_t y t as y i , t = x t ′ β i + u i , t y_{i,t}=\boldsymbol{x}_t^{\prime}\beta_i+u_{i,t} y i , t = x t ′ β i + u i , t , for x t = ( y t − 1 ′ , … , y t − p ′ ) ′ \boldsymbol{x}_t=(\boldsymbol{y}_{t-1}^{\prime},\ldots,\boldsymbol{y}_{t-p}^{\prime})^{\prime} x t = ( y t − 1 ′ , … , y t − p ′ ) ′ and stack this along the time dimension as
y i = X β i + u i , i = 1 , … , K , (4) \boldsymbol{y}_i=X\beta_i+\boldsymbol{u}_i,\quad i=1,\ldots,K, \tag*{(4)} y i = X β i + u i , i = 1 , … , K , (4)
where y i , u i ∈ R T − p \boldsymbol{y}_i,\boldsymbol{u}_i\in\mathbb{R}^{T-p} y i , u i ∈ R T − p , X = ( x p + 1 ′ , … , x T ′ ) ′ ∈ R ( T − p ) × K p X=(\boldsymbol{x}_{p+1}^{\prime},\ldots,\boldsymbol{x}_{T}^{\prime})^{\prime}\in\mathbb{R}^{(T-p)\times Kp} X = ( x p + 1 ′ , … , x T ′ ) ′ ∈ R ( T − p ) × K p . Since each column of X X X is one lag of one variable across time, we later write X j X_j X j for the j j j th column of X X X , j = 1 , … , K p j=1,\ldots,Kp j = 1 , … , K p . Since the first p p p observations are used to construct the lagged regressors, the effective sample size is originally T − p T-p T − p . To simplify notation, from this point onward we relabel this effective sample size as T T T . Since p p p is fixed (see Remark 1 ), this relabeling has no asymptotic consequence.
The transpose of the i i i th row of A \mathcal{A} A is precisely the coefficient vector β i \boldsymbol{\beta}_i β i in (4) ; hence Assumption 2 implies ∥ β i ∥ q s ≤ s 0 \|\boldsymbol{\beta}_i\|_{q_s} \le s_0 ∥ β i ∥ q s ≤ s 0 . Moreover, because y i , t y_{i,t} y i , t , u i , t u_{i,t} u i , t , and each component of x t \boldsymbol{x}_t x t are coordinates or lags of h t \boldsymbol{h}_t h t , the single-equation representation inherits the physical-dependence conditions in Assumption 1. Henceforth, we consider an arbitrary equation i i i and suppress its index. The assumptions and subsequent bounds are stated with constants that do not depend on i i i , so the same equationwise results apply to every VAR equation.
BAR builds a sequence of weighted Ridge regressions, starting from an ordinary Ridge estimator and using each iterate to determine the weights in the next step. For τ ≥ 1 \tau\ge1 τ ≥ 1 , define
β ^ ( 0 ) = arg min β { T − 1 ∥ y − X β ∥ 2 2 + ξ ∑ j = 1 K p β j 2 } , β ^ ( τ ) = g { β ^ ( τ − 1 ) } , (5) \widehat{\boldsymbol{\beta}}^{(0)}
= \underset{\boldsymbol{\beta}}{\arg\min}
\left\{
T^{-1}\|\boldsymbol{y}-\boldsymbol{X}\boldsymbol{\beta}\|_2^2
+\xi\sum_{j=1}^{Kp}\beta_j^2
\right\},
\qquad
\widehat{\boldsymbol{\beta}}^{(\tau)}
=g\{\widehat{\boldsymbol{\beta}}^{(\tau-1)}\}, \tag*{(5)} β ( 0 ) = β arg min { T − 1 ∥ y − X β ∥ 2 2 + ξ j = 1 ∑ K p β j 2 } , β ( τ ) = g { β ( τ − 1 ) } , (5)
g ( β ~ ) = arg min β { T − 1 ∥ y − X β ∥ 2 2 + λ ∑ j = 1 K p β j 2 β ~ j 2 } , β ^ B A R = lim τ → ∞ β ^ ( τ ) . (6) g(\widetilde{\boldsymbol{\beta}})
= \underset{\boldsymbol{\beta}}{\arg\min}
\left\{
T^{-1}\|\boldsymbol{y}-\boldsymbol{X}\boldsymbol{\beta}\|_2^2
+\lambda\sum_{j=1}^{Kp}\frac{\beta_j^2}{\widetilde{\beta}_j^2}
\right\},
\qquad
\widehat{\boldsymbol{\beta}}^{BAR}
=\lim_{\tau\to\infty}\widehat{\boldsymbol{\beta}}^{(\tau)}. \tag*{(6)} g ( β ) = β arg min { T − 1 ∥ y − X β ∥ 2 2 + λ j = 1 ∑ K p β j 2 β j 2 } , β B A R = τ → ∞ lim β ( τ ) . (6)
Equations (5)–(6) give the original iteratively reweighted Ridge formulation of BAR proposed by Dai et al. [6 ] . Here ξ \xi ξ and λ \lambda λ are tuning parameters, and the closed-form Ridge expressions follow directly from the first-order conditions. The reweighting scheme, similar in spirit to the adaptive Lasso of Zou [20 ] , assigns increasingly large weights to small coefficients while leaving sufficiently large coefficients comparatively unshrunk. This breaks away from the ordinary Ridge path and induces sparsity in the limit; see Figure 1 for an illustration of the geometry.
Figure 1. Stylized two-dimensional geometry of BAR and Adaptive Lasso. Left: starting from a Ridge initialization, BAR generates a sequence of reweighted ℓ 2 \ell_2 ℓ 2 -type penalty contours. The ordinary Ridge contour is spherical, while successive BAR contours become increasingly compressed in the direction of the smaller coordinate. Right: Adaptive Lasso uses a single weighted ℓ 1 \ell_1 ℓ 1 penalty contour based on a initializing estimator. Relative to the ordinary Lasso diamond, the weighted diamond is more compressed in directions associated with larger penalty weights, but its geometry is fixed once the initializer weights are chosen.
Generically, neither the initial Ridge estimator β ^ ( 0 ) \widehat{\boldsymbol{\beta}}^{(0)} β ( 0 ) nor any finite iterate β ^ ( τ ) \widehat{\boldsymbol{\beta}}^{(\tau)} β ( τ ) in (5) contains exact zeros, since the weighted ℓ 2 \ell_2 ℓ 2 problems do not induce sparsity. Exact zeros therefore arise only at the limiting fixed point of the BAR recursion. Consequently, a numerical implementation of the original recursion requires a cutoff to classify sufficiently small finite-iterate coefficients as zero.
Direct implementation of (6) can though also be numerically unstable because the weights become very large when some elements of the previous iterate approach zero. While a small perturbation of the weights can alleviate this problem, Dai et al. [6 ] proposed a more numerically stable reformulation based on rescaling the regressors by the current coefficient estimates, thereby avoiding the explicit formation of unstable weights.5 However, a direct implementation of this formulation entails the repeated inversion of ( K p ) × ( K p ) (Kp) \times(Kp) ( K p ) × ( K p ) matrices and can therefore become computationally burdensome in high dimensions. This was not a central issue in Dai et al. [6 ] , where the number of regressors was assumed to be smaller than the sample size or was reduced through preliminary screening. In our setting, Assumption 1(iv) allows K p Kp K p to be much larger than T T T . A direct implementation would then require repeated inversion of dense ( K p ) × ( K p ) (Kp) \times(Kp) ( K p ) × ( K p ) matrices, with a computational cost of order O ( ( K p ) 3 ) O((Kp)^3) O (( K p ) 3 ) per iteration under standard matrix-inversion routines.6
To avoid these inversions, we adopt the coordinatewise fixed-point strategy underlying “cycBAR”, as designed in Kawaguchi et al. [10 ] . Rather than solving the complete weighted Ridge problem in (6) at every iteration, we update each coordinate directly through the corresponding BAR fixed-point equation. For the least-squares loss considered here, this equation admits an explicit thresholding representation with a closed-form outer-root solution. The resulting procedure produces exact zeros at each iteration and avoids the inversion of large Gram matrices. Its pseudo-code is summarized in Algorithm 1.
Algorithm 1 Pseudo-code of BAR Coordinatewise Fixed-Point Thresholding
Inputs: y , X \mathbf{y},\mathbf{X} y , X (Data); ξ , λ \xi,\lambda ξ , λ (Tuning Parameters); N max N_{\max} N m a x (Max nr iter); ϵ \epsilon ϵ (Tolerance);
Initialize β ^ ( 0 ) = ( T − 1 X ′ X + ξ I ) − 1 ( T − 1 X ′ y ) \hat{\boldsymbol{\beta}}^{(0)} = (T^{-1}\mathbf{X}^{\prime}\mathbf{X}+\xi I)^{-1}(T^{-1}\mathbf{X}^{\prime}\mathbf{y}) β ^ ( 0 ) = ( T − 1 X ′ X + ξ I ) − 1 ( T − 1 X ′ y ) ; (Ridge estimator)
for n = 1 n=1 n = 1 to N max N_{\max} N m a x do (Fixed-point iteration)
for j = 1 j=1 j = 1 to K p K_p K p do (Coordinatewise fixed-point update)
compute the partial residual: r j [ n ] = y − X β ^ ( n − 1 ) + X j β ^ j ( n − 1 ) = y − X − j β ^ − j ( n − 1 ) \mathbf{r}_{j}^{[n]}=\mathbf{y}-\mathbf{X}\hat{\boldsymbol{\beta}}^{(n-1)}+\mathbf{X}_{j}\hat{\beta}_{j}^{(n-1)}=\mathbf{y}-\mathbf{X}_{-j}\hat{\boldsymbol{\beta}}_{-j}^{(n-1)} r j [ n ] = y − X β ^ ( n − 1 ) + X j β ^ j ( n − 1 ) = y − X − j β ^ − j ( n − 1 ) ;
compute the normalized score: b j [ n ] = T − 1 X j ′ r j [ n ] b_{j}^{[n]}=T^{-1}\mathbf{X}_{j}^{\prime}\mathbf{r}_{j}^{[n]} b j [ n ] = T − 1 X j ′ r j [ n ] ;
compute the activation threshold: T j ( λ ) = 2 λ ( T − 1 X j ′ X j ) \mathcal{T}_{j}(\lambda)=2\sqrt{\lambda(T^{-1}\mathbf{X}_{j}^{\prime}\mathbf{X}_{j})} T j ( λ ) = 2 λ ( T − 1 X j ′ X j ) ;
if ∣ b j [ n ] ∣ ≥ T j ( λ ) \lvert b_{j}^{[n]}\rvert\geq\mathcal{T}_{j}(\lambda) ∣ b j [ n ] ∣ ≥ T j ( λ ) then (Thresholding update)
β ^ j ( n ) = b j [ n ] + sign ( b j [ n ] ) ( b j [ n ] ) 2 − 4 λ ( T − 1 X j ′ X j ) 2 T − 1 X j ′ X j \hat{\beta}_{j}^{(n)}=\frac{b_{j}^{[n]}+\operatorname{sign}(b_{j}^{[n]})\sqrt{(b_{j}^{[n]})^{2}-4\lambda(T^{-1}\mathbf{X}_{j}^{\prime}\mathbf{X}_{j})}}{2T^{-1}\mathbf{X}_{j}^{\prime}\mathbf{X}_{j}} β ^ j ( n ) = 2 T − 1 X j ′ X j b j [ n ] + sign ( b j [ n ] ) ( b j [ n ] ) 2 − 4 λ ( T − 1 X j ′ X j ) ;
else
β ^ j ( n ) = 0 \hat{\beta}_{j}^{(n)}=0 β ^ j ( n ) = 0 ;
end if
end for
if ∥ β ^ ( n ) − β ^ ( n − 1 ) ∥ ≤ ϵ \lVert\hat{\boldsymbol{\beta}}^{(n)}-\hat{\boldsymbol{\beta}}^{(n-1)}\rVert\leq\epsilon ∥ β ^ ( n ) − β ^ ( n − 1 ) ∥ ≤ ϵ then (Convergence check)
Break (Stop upon convergence)
end if
end for
Upon convergence, set β ^ B A R = β ^ ( n ) \hat{\boldsymbol{\beta}}_{BAR}=\hat{\boldsymbol{\beta}}^{(n)} β ^ B A R = β ^ ( n ) ; (BAR estimator)
Few more details on Algorithm 1 are in order. The key idea is to start from the weighted-Ridge recursion, assume it converges, and characterize what any nonzero limiting coefficient must satisfy. Indeed, suppose that the recursion in (5) –(6) converges to β ^ B A R \hat{\boldsymbol{\beta}}_{BAR} β ^ B A R and let j ∈ S ^ : = { j : β ^ B A R , j ≠ 0 } j\in\hat{S}:=\{j:\hat{\beta}_{BAR,j}\ne0\} j ∈ S ^ := { j : β ^ B A R , j = 0 } . Define the partial residual r j ( β ^ B A R ) : = y − X β ^ B A R + X j β ^ B A R , j \mathbf{r}_{j}(\hat{\boldsymbol{\beta}}_{BAR}):=\mathbf{y}-\mathbf{X}\hat{\boldsymbol{\beta}}_{BAR}+\mathbf{X}_{j}\hat{\beta}_{BAR,j} r j ( β ^ B A R ) := y − X β ^ B A R + X j β ^ B A R , j , the corresponding normalized score b j ( β ^ B A R ) : = T − 1 X j ′ r j ( β ^ B A R ) b_{j}(\hat{\boldsymbol{\beta}}_{BAR}):=T^{-1}\mathbf{X}_{j}^{\prime}\mathbf{r}_{j}(\hat{\boldsymbol{\beta}}_{BAR}) b j ( β ^ B A R ) := T − 1 X j ′ r j ( β ^ B A R ) , and a j : = T − 1 X j ′ X j a_{j}:=T^{-1}\mathbf{X}_{j}^{\prime}\mathbf{X}_{j} a j := T − 1 X j ′ X j . At iteration τ \tau τ of the original weighted-Ridge recursion, the weights are fixed at their values from iteration τ − 1 \tau-1 τ − 1 , and β ^ ( τ ) \hat{\beta}^{(\tau)} β ^ ( τ ) solves the corresponding weighted-Ridge problem. Hence, for coordinate j j j , its first-order condition is − T − 1 X j ′ ( y − X β ^ ( τ ) ) + λ β ^ j ( τ ) / { β ^ j ( τ − 1 ) } 2 = 0 -T^{-1}X_j^{\prime}(\mathbf{y}-X\hat{\boldsymbol{\beta}}^{(\tau)})+\lambda\hat{\beta}^{(\tau)}_j/\{\hat{\beta}^{(\tau-1)}_j\}^{2}=0 − T − 1 X j ′ ( y − X β ^ ( τ ) ) + λ β ^ j ( τ ) / { β ^ j ( τ − 1 ) } 2 = 0 .7 If β ^ ( τ ) → β ^ B A R \hat{\boldsymbol{\beta}}^{(\tau)}\to\hat{\boldsymbol{\beta}}_{BAR} β ^ ( τ ) → β ^ B A R as τ → ∞ \tau\to\infty τ → ∞ and β ^ B A R , j ≠ 0 \hat{\beta}_{BAR,j}\ne0 β ^ B A R , j = 0 , then also β ^ j ( τ − 1 ) → β ^ B A R , j \hat{\beta}^{(\tau-1)}_j\to\hat{\beta}_{BAR,j} β ^ j ( τ − 1 ) → β ^ B A R , j , so that λ β ^ j ( τ ) / { β ^ j ( τ − 1 ) } 2 → λ / β ^ B A R , j \lambda\hat{\beta}^{(\tau)}_j/\{\hat{\beta}^{(\tau-1)}_j\}^{2}\to\lambda/\hat{\beta}_{BAR,j} λ β ^ j ( τ ) / { β ^ j ( τ − 1 ) } 2 → λ / β ^ B A R , j . Thus, passing to the limit τ → ∞ \tau\to\infty τ → ∞ gives the active-coordinate fixed-point identity
T − 1 X j ′ ( y − X β ^ B A R ) = λ / β ^ B A R , j . (7) T^{-1}X_j^{\prime}(\mathbf{y}-X\hat{\boldsymbol{\beta}}_{BAR})=\lambda/\hat{\beta}_{BAR,j}. \tag*{(7)} T − 1 X j ′ ( y − X β ^ B A R ) = λ / β ^ B A R , j . (7)
Combining this with T − 1 X j ′ ( y − X β ^ B A R ) = b j ( β ^ B A R ) − a j β ^ B A R , j T^{-1}X_j^{\prime}(\mathbf{y}-X\hat{\boldsymbol{\beta}}_{BAR})=b_j(\hat{\boldsymbol{\beta}}_{BAR})-a_j\hat{\beta}_{BAR,j} T − 1 X j ′ ( y − X β ^ B A R ) = b j ( β ^ B A R ) − a j β ^ B A R , j yields a quadratic equation a j β ^ B A R , j 2 − b j ( β ^ B A R ) β ^ B A R , j + λ = 0 a_j\hat{\beta}_{BAR,j}^{2}-b_j(\hat{\boldsymbol{\beta}}_{BAR})\hat{\beta}_{BAR,j}+\lambda=0 a j β ^ B A R , j 2 − b j ( β ^ B A R ) β ^ B A R , j + λ = 0 which is solved as
β ^ B A R , j = b j ( β ^ B A R ) + sign { b j ( β ^ B A R ) } b j ( β ^ B A R ) 2 − 4 λ a j 2 a j . (8) \hat{\beta}_{BAR,j}=\frac{b_j(\hat{\boldsymbol{\beta}}_{BAR})+\operatorname{sign}\{b_j(\hat{\boldsymbol{\beta}}_{BAR})\}\sqrt{b_j(\hat{\boldsymbol{\beta}}_{BAR})^{2}-4\lambda a_j}}{2a_j}. \tag*{(8)} β ^ B A R , j = 2 a j b j ( β ^ B A R ) + sign { b j ( β ^ B A R )} b j ( β ^ B A R ) 2 − 4 λ a j . (8)
A real nonzero fixed point exists only if ∣ b j ( β ^ B A R ) ∣ ≥ 2 λ a j \lvert b_j(\hat{\boldsymbol{\beta}}_{BAR})\rvert\ge2\sqrt{\lambda a_j} ∣ b j ( β ^ B A R )∣ ≥ 2 λ a j , which is precisely the coordinate activation threshold T j ( λ ) \mathcal{T}_j(\lambda) T j ( λ ) in Algorithm 1. When ∣ b j ( β ^ B A R ) ∣ > 2 λ a j \lvert b_j(\hat{\boldsymbol{\beta}}_{BAR})\rvert>2\sqrt{\lambda a_j} ∣ b j ( β ^ B A R )∣ > 2 λ a j , the quadratic has two roots. Following Kawaguchi et al. [10 ] , we select the outer root; its stability under the scalar fixed-point iteration is shown in Appendix § 8 .2.1. Algorithm 1 applies this update coordinate by coordinate using the current normalized score; if the threshold is not met, the coefficient is set to zero. Another useful feature of the cyclic update is that it produces exact zeros directly through the activation rule. This is unlike the original weighted-Ridge BAR recursion, whose finite iterates remain dense and therefore require an arbitrary numerical cutoff to induce sparsity. Hence, the algorithm searches directly for a solution of the BAR fixed-point equations without solving a full weighted-Ridge problem at every iteration.
Further implementation details, including reduced-space computation of the Ridge initializer and the roles of N max N_{\max} N m a x and ϵ \epsilon ϵ , are reported in Appendix § 8 .2.2.
1 and the later Lemma 2, max j ∣ T − 1 X j ′ X j − 1 ∣ = o p ( 1 ) \max_j \lvert T^{-1}\mathbf{X}_j^{\prime}\mathbf{X}_j-1\rvert=o_p(1) max j ∣ T − 1 X j ′ X j − 1 ∣ = o p ( 1 ) , so replacing the sample second moments by one does not affect the asymptotic rates. In implementation, the predictors are sample centered and scaled so that T − 1 X j ′ X j = 1 T^{-1}\mathbf{X}_j^{\prime}\mathbf{X}_j=1 T − 1 X j ′ X j = 1 , or equivalently ∥ X j ∥ 2 2 = T \lVert\mathbf{X}_j\rVert_2^2=T ∥ X j ∥ 2 2 = T , exactly.
Then the activation threshold in Algorithm 1 becomes T j ( λ ) = 2 λ \mathcal{T}_j(\lambda)=2\sqrt{\lambda} T j ( λ ) = 2 λ for all j j j , and the active-coordinate update reduces to { b j [ n ] + sign ( b j [ n ] ) [ ( b j [ n ] ) 2 − 4 λ ] 1 / 2 } / 2 \{b_j^{[n]}+\operatorname{sign}(b_j^{[n]})[(b_j^{[n]})^2-4\lambda]^{1/2}\}/2 { b j [ n ] + sign ( b j [ n ] ) [( b j [ n ] ) 2 − 4 λ ] 1/2 } /2 . In particular, every active coordinate satisfies ∣ β ^ j ( n ) ∣ ≥ λ \lvert\hat{\beta}_j^{(n)}\rvert\ge\sqrt{\lambda} ∣ β ^ j ( n ) ∣ ≥ λ . Algorithm 1 is written in its unnormalized form only to make the general mechanics transparent.
Henceforth, the predictors are understood to be population standardized.
Remark 3 (Grouping effect). The BAR fixed-point identity also yields a finite-sample grouping property. Let ρ ^ j k : = T − 1 X j ′ X k \hat{\rho}_{jk}:=T^{-1}\mathbf{X}_j^{\prime}\mathbf{X}_k ρ ^ j k := T − 1 X j ′ X k denote the sample correlation between the standardized regressors X j \mathbf{X}_j X j and X k \mathbf{X}_k X k . As shown in Proposition 3 of Appendix § 8 .2.3, for any j , k ∈ S ^ : = { j : β ^ j ≠ 0 } j,k\in\hat{S}:=\{j:\hat{\beta}_j\ne0\} j , k ∈ S ^ := { j : β ^ j = 0 } ,
∣ 1 β ^ B A R , j − 1 β ^ B A R , k ∣ ≤ ∥ y ∥ 2 λ T 2 ( 1 − ρ ^ j k ) . (9) \left\lvert\frac{1}{\hat{\beta}_{\mathrm{BAR},j}}-\frac{1}{\hat{\beta}_{\mathrm{BAR},k}}\right\rvert\le\frac{\lVert y\rVert_2}{\lambda\sqrt{T}}\sqrt{2(1-\hat{\rho}_{jk})}. \tag*{(9)} β ^ BAR , j 1 − β ^ BAR , k 1 ≤ λ T ∥ y ∥ 2 2 ( 1 − ρ ^ j k ) . (9)
Thus, the inverse estimated coefficients become close as ρ ^ j k \hat{\rho}_{jk} ρ ^ j k approaches one. An analogous result holds as ρ ^ j k \hat{\rho}_{jk} ρ ^ j k approaches minus one, in which case the inverse coefficients become close after a sign reversal. This property is particularly relevant for persistent VARs, where different lags of the same variable are often strongly correlated.
Algorithm 1 described the coordinatewise fixed-point thresholding implementation of BAR for fixed tuning parameters ξ \xi ξ and λ \lambda λ . In practice, the BAR estimator -along with its initial Ridge warm start- is applied over a grid of penalty parameters, as described in Algorithm 2.
The computational grid in Algorithm 2 ranges from a lower endpoint anchored to the theoretical rate to a data-driven upper endpoint, specifically:
λ ( 1 ) = 10 − 3 log ( K p ) T , λ ( G ) = 1 4 max 1 ≤ j ≤ K p ( b j [ 1 ] ) 2 T − 1 X j ′ X j . (10) \lambda^{(1)}=10^{-3}\frac{\log(Kp)}{T},\qquad\lambda^{(G)}=\frac{1}{4}\max_{1\le j\le Kp}\frac{\left(b_j^{[1]}\right)^2}{T^{-1}\mathbf{X}_j^{\prime}\mathbf{X}_j}. \tag*{(10)} λ ( 1 ) = 1 0 − 3 T log ( K p ) , λ ( G ) = 4 1 1 ≤ j ≤ K p max T − 1 X j ′ X j ( b j [ 1 ] ) 2 . (10)
Algorithm 2 BAR Tuning Path over a Grid of Penalties
1: Inputs: y \mathbf{y} y , X \mathbf{X} X , Ridge parameter ξ \xi ξ , N max N_{\max} N m a x , tolerance ϵ \epsilon ϵ , and a grid Λ = { λ ( 1 ) , … , λ ( G ) } \Lambda= \{\lambda^{(1)}, \ldots, \lambda^{(G)}\} Λ = { λ ( 1 ) , … , λ ( G ) }
2: for g = 1 , … , G g = 1, \ldots, G g = 1 , … , G do
3: Run Algorithm 1 at λ ( g ) \lambda^{(g)} λ ( g ) and obtain β ^ B A R ( λ ( g ) ) \widehat{\boldsymbol{\beta}}_{BAR}(\lambda^{(g)}) β B A R ( λ ( g ) )
4: Set S ^ ( λ ( g ) ) = { j : β ^ B A R , j ( λ ( g ) ) ≠ 0 } \widehat{S}(\lambda^{(g)}) = \{j : \widehat{\beta}_{BAR,j}(\lambda^{(g)}) \ne0\} S ( λ ( g ) ) = { j : β B A R , j ( λ ( g ) ) = 0 } and R S S ( λ ( g ) ) = ∥ y − X β ^ B A R ( λ ( g ) ) ∥ 2 RSS(\lambda^{(g)}) = \lVert\mathbf{y} - \mathbf{X}\widehat{\boldsymbol{\beta}}_{BAR}(\lambda^{(g)}) \rVert^{2} R S S ( λ ( g ) ) = ∥ y − X β B A R ( λ ( g ) ) ∥ 2
5: Evaluate an information criterion C ( λ ( g ) ) C(\lambda^{(g)}) C ( λ ( g ) )
6: end for
7: Select λ ^ = arg min λ ∈ Λ C ( λ ) \widehat{\lambda} = \operatorname*{arg\,min}_{\lambda\in\Lambda} C(\lambda) λ = arg min λ ∈ Λ C ( λ ) and return β ^ B A R ( λ ^ ) \widehat{\boldsymbol{\beta}}_{BAR}(\widehat{\lambda}) β B A R ( λ ) and S ^ ( λ ^ ) \widehat{S}(\widehat{\lambda}) S ( λ ) .
The lower endpoint follows from the empirical-process rate derived below in Lemma 2: because the BAR threshold is of order λ \sqrt{\lambda} λ , the smallest statistically meaningful penalty is of order log ( K p ) / T \log(Kp)/T log ( K p ) / T . Here, we use a deliberately small multiplier (10 − 3 10^{-3} 1 0 − 3 ) so that the computational path also includes relatively dense models. The upper endpoint is the first-iteration activation boundary; penalties strictly above λ ( G ) \lambda^{(G)} λ ( G ) yield no active coordinate at the first iteration. Under standardization, it reduces to λ ( G ) = 1 4 max j ( b j [ 1 ] ) 2 \lambda^{(G)} = \frac{1}{4}\max_j (b_j^{[1]})^2 λ ( G ) = 4 1 max j ( b j [ 1 ] ) 2 . The Ridge parameter ξ \xi ξ determines the dense initializer and therefore influences the first-iteration normalized scores, whereas λ \lambda λ directly controls thresholding and sparsification.
The tuning parameters ξ \xi ξ , and in particular λ \lambda λ , are central to both the computation and the theory of BAR. Algorithm 2 chooses them by minimizing an information criterion C ( λ ) C(\lambda) C ( λ ) along the corresponding tuning path. In § 3.3 , and supported by the simulation evidence reported in § 4 , we advocate for the use of the Bayesian Information Criterion (BIC) for the Ridge initializer parameter ξ \xi ξ , and the Extended BIC (EBIC; [3 ] ) for the sparsity-inducing penalty λ \lambda λ . This combination works well in simulations: BIC yields a numerically stable initializing estimator, whereas EBIC tends to avoid the extreme ends of the BAR path and more often selects penalties in the empirically relevant interior region. More on this in § 3.3 .
BAR High-Dimensional Theory & Error Bounds
Given a BAR estimate β ^ B A R ∈ R K p \hat{\beta}_{BAR} \in\mathbb{R}^{Kp} β ^ B A R ∈ R K p , we assess its quality through prediction and estimation consistency. Prediction consistency ∥ X ( β ^ B A R − β ) ∥ 2 2 \lVert\mathbf{X}(\hat{\beta}_{BAR}-\beta)\rVert_2^2 ∥ X ( β ^ B A R − β ) ∥ 2 2 concerns how well the fitted model reproduces the systematic component of the data. Estimation consistency, instead, concerns how close β ^ B A R \hat{\beta}_{BAR} β ^ B A R is to the true parameter vector β \beta β as measured by ∥ β ^ B A R − β ∥ r \lVert\hat{\beta}_{BAR}-\beta\rVert_r ∥ β ^ B A R − β ∥ r , r ∈ { 1 , 2 } r \in\{1,2\} r ∈ { 1 , 2 } . In high-dimensional settings, the two need not coincide, so both criteria are considered.
Under approximate sparsity (Assumption 2), exact support recovery is not a natural main objective, since β \beta β may contain many small but nonzero coefficients. For a fixed tuning parameter λ ≥ 0 \lambda\ge0 λ ≥ 0 , we therefore write S ^ : = S ^ ( λ ) \hat{S}:=\hat{S}(\lambda) S ^ := S ^ ( λ ) for the support selected by BAR and define the corresponding population “effective support” as
S λ : = { j : ∣ β j ∣ > 2 λ } . (11) S_{\lambda}:=\left\{j:|\beta_j|>2\sqrt{\lambda}\right\}. \tag*{(11)} S λ := { j : ∣ β j ∣ > 2 λ } . (11)
We denote its cardinality by ∣ S λ ∣ |S_{\lambda}| ∣ S λ ∣ and its complement by S λ c = { 1 , … , K p } ∖ S λ S_{\lambda}^{c}=\{1,\ldots,Kp\}\setminus S_{\lambda} S λ c = { 1 , … , K p } ∖ S λ . The set S λ S_{\lambda} S λ collects the true coefficients lying above the BAR activation threshold T j ( λ ) T_j(\lambda) T j ( λ ) . For λ = 0 \lambda=0 λ = 0 , it reduces to the “exact support” of β \beta β .8 The next lemma formalizes the two roles played by S λ S_{\lambda} S λ : controlling the number of true coefficients above the BAR threshold scale and, at the same time, the aggregate mass of those below it. Both quantities will indeed enter the BAR error bounds and therefore control for them is needed.
Lemma 1 (Effective support and tail bounds). Given Assumption 2, for every q s ∈ [ 0 , 1 ) q_s \in[0,1) q s ∈ [ 0 , 1 ) and λ > 0 \lambda>0 λ > 0 ,
∣ S λ ∣ = ∑ j = 1 K p 1 { ∣ β j ∣ > 2 λ } ≲ s 0 λ − q s / 2 , ∥ β S λ c ∥ r ≲ s 0 1 / r λ 1 2 − q s 2 r , r ∈ { 1 , 2 } . (12) |S_{\lambda}|=\sum_{j=1}^{Kp}\mathbf{1}\{|\beta_j|>2\sqrt{\lambda}\}\lesssim s_0\lambda^{-q_s/2},\qquad\lVert\beta_{S_{\lambda}^{c}}\rVert_r\lesssim s_0^{1/r}\lambda^{\frac{1}{2}-\frac{q_s}{2r}},\qquad r\in\{1,2\}. \tag*{(12)} ∣ S λ ∣ = j = 1 ∑ K p 1 { ∣ β j ∣ > 2 λ } ≲ s 0 λ − q s /2 , ∥ β S λ c ∥ r ≲ s 0 1/ r λ 2 1 − 2 r q s , r ∈ { 1 , 2 } . (12)
The bounds in Lemma 1 clarify the role of the weak-sparsity parameters from Assumption 2.
The overall amount of signal is controlled by s 0 s_0 s 0 , while q s q_s q s determines how much of it may be spread over small coefficients. Accordingly, the bounds allow both the effective support and the aggregate mass below the threshold to increase with s 0 s_0 s 0 and q s q_s q s . The tuning parameter λ \lambda λ determines the trade-off between the two: a smaller λ \lambda λ lowers the cutoff 2 λ 2\sqrt{\lambda} 2 λ and therefore increases ∣ S λ ∣ |S_\lambda| ∣ S λ ∣ , but at the same time reduces ∥ β S λ c ∥ r \|\beta_{S_\lambda^c}\|_r ∥ β S λ c ∥ r , since fewer coefficients remain below the cutoff. For q s = 0 q_s=0 q s = 0 the weak-sparsity condition reduces to exact sparsity, ∥ β ∥ 0 ≤ s 0 \|\beta\|_0 \le s_0 ∥ β ∥ 0 ≤ s 0 , and the bounds of Lemma 1 continue to hold with ∣ S λ ∣ ≤ s 0 |S_\lambda| \le s_0 ∣ S λ ∣ ≤ s 0 and ∥ β S λ c ∥ r ≲ s 0 1 / r λ \|\beta_{S_\lambda^c}\|_r \lesssim s_0^{1/r}\sqrt{\lambda} ∥ β S λ c ∥ r ≲ s 0 1/ r λ . The proof is given in the Appendix § 9 .
To make these bounds operational, however, we need to determine the appropriate size of λ \lambda λ . The cutoff 2 λ 2\sqrt{\lambda} 2 λ defining S λ S_\lambda S λ in population is also the BAR activation threshold T j ( λ ) \mathcal{T}_j(\lambda) T j ( λ ) in sample. Since BAR activates coordinate j j j according to the normalized score b j [ n ] = T − 1 X j ′ r j [ n ] b_j^{[n]}=T^{-1}X_j'r_j^{[n]} b j [ n ] = T − 1 X j ′ r j [ n ] , T j ( λ ) \mathcal{T}_j(\lambda) T j ( λ ) should be large enough that a coordinate is not activated because of stochastic terms contained in b j [ n ] b_j^{[n]} b j [ n ] pushing its absolute value erroneously over T j ( λ ) \mathcal{T}_j(\lambda) T j ( λ ) . The next subsection decomposes b j [ n ] b_j^{[n]} b j [ n ] into the true signal and two additional random components generated by the empirical noise and the estimation errors in the remaining coordinates, see (13) . We first derive the stochastic order of the empirical-noise component, which determines the appropriate order of λ \lambda λ , and then turn to the cross-design component in the following subsection.
Empirical Process and Gram-Matrix Concentration
The normalized score b j [ n ] = T − 1 X j ′ r j [ n ] b_j^{[n]}=T^{-1}X_j'r_j^{[n]} b j [ n ] = T − 1 X j ′ r j [ n ] is the main character in Algorithm 1 (line 6) as its absolute value decides whether the j j j th coefficient is activated or set to zero. It also reveals the two main stochastic ingredients required for the BAR analysis. Using r j [ n ] = y − X − j β ^ − j ( n − 1 ) r_j^{[n]}=\mathbf{y}-X_{-j}\hat{\boldsymbol{\beta}}_{-j}^{(n-1)} r j [ n ] = y − X − j β ^ − j ( n − 1 ) , substituting y = X β + u \mathbf{y}=X\boldsymbol{\beta}+\mathbf{u} y = X β + u , and via standardization:
b j [ n ] = β j − T − 1 X j ′ X − j ( β ^ − j ( n − 1 ) − β − j ) + T − 1 X j ′ u . (13) b_j^{[n]}=\beta_j-T^{-1}X_j'X_{-j}\left(\hat{\boldsymbol{\beta}}_{-j}^{(n-1)}-\boldsymbol{\beta}_{-j}\right)+T^{-1}X_j'\mathbf{u}. \tag*{(13)} b j [ n ] = β j − T − 1 X j ′ X − j ( β ^ − j ( n − 1 ) − β − j ) + T − 1 X j ′ u . (13)
The three terms on the right hand side in (13) are, respectively, the true j j j th signal β j \beta_j β j , a cross-design error term, and the empirical process (the “noise”). For j ∈ S λ c j \in S_{\lambda}^{c} j ∈ S λ c , by definition ∣ β j ∣ ≤ T ( λ ) |\beta_j| \le\mathcal{T}(\lambda) ∣ β j ∣ ≤ T ( λ ) , so the true signal alone clearly does not exceed the BAR activation threshold.9
Thence, a coordinate outside the effective support can cross the activation threshold only because of the other two terms in (13) . The second term, which at fixed point becomes − T − 1 X j ′ X − j ( β ^ B A R , − j − β − j ) -T^{-1}X_j^{\prime}X_{-j}(\widehat{\beta}_{BAR,-j}-\beta_{-j}) − T − 1 X j ′ X − j ( β B A R , − j − β − j ) ,10 reflects how estimation errors in the remaining coordinates can feed into the j j j th normalized score through the off-diagonal entries of the sample Gram matrix Σ ^ x : = T − 1 X ′ X \widehat{\Sigma}_x := T^{-1}X^{\prime}X Σ x := T − 1 X ′ X . The last term, T − 1 X j ′ u T^{-1}X_j^{\prime}u T − 1 X j ′ u , is instead the direct contribution of the noise. To determine the stochastic scale at which T ( λ ) \mathcal{T}(\lambda) T ( λ ) should be set, we consider first the last term. Uniformly over the coordinates, the activation threshold should be at least of the same order as ∥ T − 1 X ′ u ∥ ∞ \lVert T^{-1}X^{\prime}u\rVert_{\infty} ∥ T − 1 X ′ u ∥ ∞ ; otherwise coordinates may be activated simply because the empirical noise term is large, rather than because of its true signal. Since T ( λ ) = 2 λ \mathcal{T}(\lambda)=2\sqrt{\lambda} T ( λ ) = 2 λ , this suggests choosing λ \lambda λ on the order of ∥ T − 1 X ′ u ∥ ∞ 2 \lVert T^{-1}X^{\prime}u\rVert_{\infty}^{2} ∥ T − 1 X ′ u ∥ ∞ 2 .
The cross-design error term requires a different type of control. It depends on the geometry of the sample Gram matrix and on the estimation error itself, and therefore cannot simply be absorbed into the penalty calibration as for the noise. Let Σ x : = E ( x t x t ′ ) \Sigma_x := \mathbb{E}(x_tx_t^{\prime}) Σ x := E ( x t x t ′ ) denote the population counterpart of Σ ^ x \widehat{\Sigma}_x Σ x . The strategy pursued below is to first localize the estimation error to a sufficiently small support and then impose restricted curvature only along the relevant directions. This requires control of the sample Gram matrix in addition to control of the empirical process. Thus, the two stochastic objects needed for the subsequent analysis are T − 1 X ′ u T^{-1}X^{\prime}u T − 1 X ′ u and Σ ^ x − Σ x \widehat{\Sigma}_x-\Sigma_x Σ x − Σ x . Under Assumption 1, both objects can be controlled jointly at the same high-dimensional rate, as done in the following lemma.
Lemma 2 (Joint Empirical Process and Gram-matrix Concentration). Under Assumption
1, there exists a sufficiently large fixed constant C χ > 0 C_{\chi} > 0 C χ > 0 such that the event
E χ : = { ∥ T − 1 X ′ u ∥ ∞ ∨ ∥ Σ ^ x − Σ x ∥ max ≤ C χ log ( K p ) T } , (14) \mathcal{E}_{\chi} := \left\{ \left\|T^{-1}X^{\prime}u\right\|_{\infty} \vee\left\|\widehat{\Sigma}_{x} - \Sigma_{x}\right\|_{\max} \le C_{\chi}\sqrt{\frac{\log(Kp)}{T}} \right\}, \tag*{(14)} E χ := { T − 1 X ′ u ∞ ∨ Σ x − Σ x m a x ≤ C χ T log ( K p ) } , (14)
satisfies P ( E χ ) → 1 \mathbb{P}(\mathcal{E}_{\chi}) \to1 P ( E χ ) → 1 as K , T → ∞ K,T \to\infty K , T → ∞ . Then, ∥ T − 1 X ′ u ∥ ∞ ∨ ∥ Σ ^ x − Σ x ∥ max = O p ( log ( K p ) T ) \left\|T^{-1}X^{\prime}u\right\|_{\infty} \vee\left\|\widehat{\Sigma}_{x} - \Sigma_{x}\right\|_{\max} = O_{p}\left(\sqrt{\frac{\log(Kp)}{T}}\right) T − 1 X ′ u ∞ ∨ Σ x − Σ x m a x = O p ( T l o g ( K p ) ) .
The two bounds in Lemma 2 play different roles. The bound on ∥ T − 1 X ′ u ∥ ∞ \left\|T^{-1}X^{\prime}u\right\|_{\infty} T − 1 X ′ u ∞ determines the stochastic scale of the BAR activation threshold, motivating λ = C λ log ( K p ) / T \lambda= C_{\lambda}\log(Kp)/T λ = C λ log ( K p ) / T , where C λ > 0 C_{\lambda} > 0 C λ > 0 is a sufficiently large fixed constant.11 The Gram-matrix deviation bound, instead, is useful for controlling the cross-design term in (13) , since this term depends on how estimation errors in the remaining coordinates are transmitted through the off-diagonal elements of Σ ^ x \widehat{\Sigma}_{x} Σ x . The proof of Lemma 2 follows the second-order process construction of Margaritella & Sessinou [13 ] , based on the vector-valued maximal inequality of Zhang & Wu [19 ] .12
Now, by Lemma 2, setting λ = C λ log ( K p ) / T \lambda= C_{\lambda}\log(Kp)/T λ = C λ log ( K p ) / T within Lemma 1 gives the following rates.
Corollary 1. If λ = C λ log ( K p ) / T \lambda= C_{\lambda}\log(Kp)/T λ = C λ log ( K p ) / T , then, for q s ∈ [ 0 , 1 ) q_s \in[0,1) q s ∈ [ 0 , 1 ) and r ∈ { 1 , 2 } r \in\{1,2\} r ∈ { 1 , 2 } ,
∣ S λ ∣ ≲ s 0 ( T log ( K p ) ) q s / 2 , ∥ β S λ c ∥ r ≲ s 0 1 / r ( log ( K p ) T ) 1 2 − q s 2 r . (15) |S_{\lambda}| \lesssim s_0\left(\frac{T}{\log(Kp)}\right)^{q_s/2}, \qquad\left\|\beta_{S_{\lambda}^{c}}\right\|_r \lesssim s_0^{1/r}\left(\frac{\log(Kp)}{T}\right)^{\frac{1}{2}-\frac{q_s}{2r}}. \tag*{(15)} ∣ S λ ∣ ≲ s 0 ( log ( K p ) T ) q s /2 , β S λ c r ≲ s 0 1/ r ( T log ( K p ) ) 2 1 − 2 r q s . (15)
Corollary 1 makes the dimensional scaling of Lemma 1 explicit. Under Assumption 1, K p = O ( T ν ) Kp = O(T^{\nu}) K p = O ( T ν ) for some fixed ν > 0 \nu> 0 ν > 0 , so log ( K p ) = O ( log T ) \log(Kp) = O(\log T) log ( K p ) = O ( log T ) and polynomial growth in the number of regressors enters these rates only through a logarithmic factor.13 As T / log ( K p ) T/\log(Kp) T / log ( K p ) increases, the effective support may grow at rate s 0 { T / log ( K p ) } q s / 2 s_0\{T/\log(Kp)\}^{q_s/2} s 0 { T / log ( K p ) } q s /2 , while the ℓ 1 \ell_1 ℓ 1 and ℓ 2 \ell_2 ℓ 2 mass below the threshold shrink. Thus, larger samples allow more coefficients to enter the effective support, whereas faster growth in K p K_p K p works in the opposite direction. These rates determine the sparsity growth allowed in the arguments that follow.
Restricted Curvature on the Support Union
We now turn to the cross-design term in (13) . At the BAR fixed point, let Δ : = β ^ B A R − β \Delta:= \hat{\beta}_{BAR} - \beta Δ := β ^ B A R − β . The term becomes − T − 1 X j ′ X − j Δ − j -T^{-1}X_j'X_{-j}\Delta_{-j} − T − 1 X j ′ X − j Δ − j and therefore cannot be controlled directly without first exploiting some structure on the estimation error Δ \Delta Δ . Unlike the Lasso, BAR does not restrict the estimation error to an ℓ 1 \ell_1 ℓ 1 cone. Its fixed-point rule instead has a hard-thresholding-type structure: coordinates below the activation threshold are set to zero, while active coordinates are mapped to a nonzero root. We exploit this structure to localize the estimation error to an affine set determined by the support union U U U , which we now define (see also Figure 2 for an illustration).
Figure 2. Illustration of BAR error support restriction against the Lasso cone in R 3 \mathbb{R}^3 R 3 . The blue plane represents the BAR support-union for U = { 1 , 2 } U = \{1, 2\} U = { 1 , 2 } : since Δ U c = − β U c \Delta_{U^c} = -\beta_{U^c} Δ U c = − β U c , the third coordinate is fixed at Δ 3 = − β 3 = − 0.35 \Delta_3 = -\beta_3 = -0.35 Δ 3 = − β 3 = − 0.35 , while Δ 1 \Delta_1 Δ 1 and Δ 2 \Delta_2 Δ 2 remain free. The orange double cone represents the usual Lasso cone restriction, here illustrated by ∣ Δ 2 ∣ + ∣ Δ 3 ∣ ≤ c ∗ ∣ Δ 1 ∣ |\Delta_2| + |\Delta_3| \le c^*|\Delta_1| ∣ Δ 2 ∣ + ∣ Δ 3 ∣ ≤ c ∗ ∣ Δ 1 ∣ with c ∗ = 1 c^* = 1 c ∗ = 1 . BAR localizes the error to a lower-dimensional affine set once U U U is fixed, whereas the Lasso restricts the error to a cone of admissible directions.
Let U : = S λ ∪ S ^ U := S_\lambda\cup\hat{S} U := S λ ∪ S ^ . Recall S λ : = { j : ∣ β j ∣ > 2 λ } S_\lambda:= \{j : |\beta_j| > 2\sqrt{\lambda}\} S λ := { j : ∣ β j ∣ > 2 λ } is the effective support under weak sparsity and S ^ : = { j : β ^ B A R , j ≠ 0 } \hat{S} := \{j : \hat{\beta}_{BAR,j} \ne0\} S ^ := { j : β ^ B A R , j = 0 } is the support selected by BAR. The set U U U then contains both the coordinates that are truly large enough to matter and the coordinates that BAR actually selects. This construction is useful because outside such union, i.e., for every j ∈ U c j \in U^c j ∈ U c , the true coefficient lies below the threshold scale, since j ∉ S λ j \notin S_\lambda j ∈ / S λ , while the estimated coefficient is exactly zero, as j ∉ S ^ j \notin\hat{S} j ∈ / S ^ . Hence, the BAR error is exactly Δ j = − β j \Delta_j = -\beta_j Δ j = − β j on U c U^c U c , making all the remaining uncertainty in the estimation error confined to the random set U U U .14 The next lemma formalizes this discussion over the support characterization of the BAR error.
Lemma 3 (BAR Error Representation on U U U ). Let U : = S λ ∪ S ^ U := S_\lambda\cup\hat{S} U := S λ ∪ S ^ , where S λ = { j : ∣ β j ∣ > 2 λ } S_\lambda= \{j : |\beta_j| > 2\sqrt{\lambda}\} S λ = { j : ∣ β j ∣ > 2 λ } and S ^ = { j : β ^ B A R , j ≠ 0 } \hat{S} = \{j : \hat{\beta}_{BAR,j} \ne0\} S ^ = { j : β ^ B A R , j = 0 } . Then Δ U c = − β U c \Delta_{U^c} = -\beta_{U^c} Δ U c = − β U c ; Δ = Δ U − β U c \Delta= \Delta_U - \beta_{U^c} Δ = Δ U − β U c . Since U c ⊆ S λ c U^c \subseteq S_\lambda^c U c ⊆ S λ c , for r ∈ { 1 , 2 } r \in\{1,2\} r ∈ { 1 , 2 } , ∥ Δ U c ∥ r ≤ ∥ β S λ c ∥ r \|\Delta_{U^c}\|_r \le\|\beta_{S_\lambda^c}\|_r ∥ Δ U c ∥ r ≤ ∥ β S λ c ∥ r ; ∥ Δ ∥ r ≤ ∥ Δ U ∥ r + ∥ β S λ c ∥ r \|\Delta\|_r \le\|\Delta_U\|_r + \|\beta_{S_\lambda^c}\|_r ∥Δ ∥ r ≤ ∥ Δ U ∥ r + ∥ β S λ c ∥ r . In particular, plugging Corollary
ℓ r \ell_r ℓ r -rates:
∥ Δ ∥ r ≤ ∥ Δ U ∥ r + O [ s 0 1 / r ( log ( K p ) T ) 1 2 − q s 2 r ] , r ∈ { 1 , 2 } . (16) \lVert\Delta\rVert_r \le\lVert\Delta_U \rVert_r + \mathcal{O}\left[s_0^{1/r}\left(\frac{\log(Kp)}{T}\right)^{\frac{1}{2}-\frac{q_s}{2r}}\right], \qquad r \in\{1,2\}. \tag*{(16)} ∥ Δ ∥ r ≤ ∥ Δ U ∥ r + O [ s 0 1/ r ( T log ( K p ) ) 2 1 − 2 r q s ] , r ∈ { 1 , 2 } . (16)
Lemma 3 shows that outside U U U the BAR estimation error is completely determined by the true coefficients that lie below the threshold, whose ℓ r \ell_r ℓ r norms are already controlled by Corollary 1. The remaining unknown part of the error is therefore Δ U \Delta_U Δ U , so the subsequent analysis can focus on controlling the exact cardinality of U U U and the size of Δ U \Delta_U Δ U .
Controlling Δ U \Delta_U Δ U requires sufficient curvature of the quadratic loss component of the BAR objective along the coordinates in U U U . A lower bound on the smallest eigenvalue of Σ ^ x , U U \widehat{\Sigma}_{x,UU} Σ x , U U would ensure that small prediction errors translate into small estimation errors, as one would wish. This is the role of the restricted eigenvalue (RE) condition, typically employed in the literature. In a high-dimensional setting such curvature cannot be required globally, since Σ ^ x \widehat{\Sigma}_x Σ x is singular whenever K p > T Kp > T K p > T , and any submatrix with more than T T T columns is necessarily singular too. It can only hold on subsets containing sufficiently few regressors. Rather than assuming such a property directly for the random sample Gram matrix, it is typically more natural to impose the corresponding RE condition on the population covariance matrix Σ x \Sigma_x Σ x , as a property of the DGP, and then show that it is inherited by Σ ^ x \widehat{\Sigma}_x Σ x via some deviation bound such as that in Lemma 2. However, the size of the subsets is important for this transfer, since for any index set J J J , ∥ Σ ^ x , J J − Σ x , J J ∥ op ≤ ∣ J ∣ ∥ Σ ^ x − Σ x ∥ max \lVert\widehat{\Sigma}_{x,JJ}-\Sigma_{x,JJ}\rVert_{\operatorname{op}} \le|J|\lVert\widehat{\Sigma}_x-\Sigma_x\rVert_{\max} ∥ Σ x , J J − Σ x , J J ∥ op ≤ ∣ J ∣ ∥ Σ x − Σ x ∥ m a x . Hence, the smaller the admissible support size, the smaller the population-to-sample perturbation and the easier it is for the sample Gram matrix to inherit the population RE.15 Hence, controlling the cardinality of U U U is essential. But another BAR-specific issue here is that U = S λ ∪ S ^ U=S_\lambda\cup\widehat{S} U = S λ ∪ S is itself random. We therefore first need to show that, with high probability, U U U belongs to a deterministic class of subsets with controlled cardinality, and impose a preliminary RE condition uniformly over that class.16
The remainder of the argument proceeds in three steps. First, we derive a preliminary deterministic upper bound on ∣ U ∣ |U| ∣ U ∣ . Second, exploiting the BAR fixed-point equations, we further reduce this bound down to the sparsity scale s 0 λ − q s / 2 s_0\lambda^{-q_s/2} s 0 λ − q s /2 (cf Corollary 1). Third, on this sufficiently smaller support scale, we transfer a population RE condition to the sample Gram matrix using Lemma 2. This yields the restricted curvature needed to control Δ U \Delta_U Δ U .
Proposition 1 (Crude support bound). Let β ^ B A R \widehat{\beta}_{BAR} β B A R be a fixed point of Algorithm 1. Then ∣ S ^ ∣ ≤ ∥ y ∥ 2 2 / ( 4 λ T ) |\widehat{S}| \le\lVert y\rVert_2^2/(4\lambda T) ∣ S ∣ ≤ ∥ y ∥ 2 2 / ( 4 λ T ) . Moreover, for any fixed C y > 0 C_y>0 C y > 0 , on the event E y : = { T − 1 ∥ y ∥ 2 2 ≤ C y } , \mathcal{E}_y:=\{T^{-1}\lVert y\rVert_2^2\le C_y\}, E y := { T − 1 ∥ y ∥ 2 2 ≤ C y } ,
∣ U ∣ ≤ m ˉ T : = ⌈ 2 − q s s 0 λ − q s / 2 + C y 4 λ ⌉ ≲ s 0 λ − q s / 2 + λ − 1 . (17) |U|\le\bar{m}_T:=\left\lceil2^{-q_s}s_0\lambda^{-q_s/2}+\frac{C_y}{4\lambda}\right\rceil\lesssim s_0\lambda^{-q_s/2}+\lambda^{-1}. \tag*{(17)} ∣ U ∣ ≤ m ˉ T := ⌈ 2 − q s s 0 λ − q s /2 + 4 λ C y ⌉ ≲ s 0 λ − q s /2 + λ − 1 . (17)
By stationarity and ergodicity under Assumption 1, T − 1 ∥ y ∥ 2 2 → p E ( y t 2 ) T^{-1}\lVert y\rVert_2^2 \overset{p}{\to}\mathbb{E}(y_t^2) T − 1 ∥ y ∥ 2 2 → p E ( y t 2 ) . Hence, choosing C y > E ( y t 2 ) C_y>\mathbb{E}(y_t^2) C y > E ( y t 2 ) gives P ( E y ) → 1 \mathbb{P}(\mathcal{E}_y)\to1 P ( E y ) → 1 . Defining J m ˉ T : = { J ⊆ { 1 , … , K p } : ∣ J ∣ ≤ m ˉ T } \mathcal{J}_{\bar{m}_T}:=\{J\subseteq\{1,\ldots,Kp\}:|J|\le\bar{m}_T\} J m ˉ T := { J ⊆ { 1 , … , K p } : ∣ J ∣ ≤ m ˉ T } , Proposition 1 therefore implies U ∈ J m ˉ T U\in\mathcal{J}_{\bar{m}_T} U ∈ J m ˉ T with probability tending to one.17 The class J m ˉ T \mathcal{J}_{\bar{m}_T} J m ˉ T is deterministic, so a restricted eigenvalue condition could now be imposed uniformly over it.
The problem is that currently the obtained support bound m ˉ T \bar{m}_T m ˉ T is too large to obtain the desired sample curvature directly from the population covariance using our Lemma 2. In fact, we get by Lemma 2
∥ Σ ^ x − Σ x ∥ max = O p ( λ ) , \left\lVert\widehat{\boldsymbol{\Sigma}}_x-\boldsymbol{\Sigma}_x\right\rVert_{\max}=\mathcal{O}_p(\sqrt{\lambda}), Σ x − Σ x m a x = O p ( λ ) ,
and over a set J J J of size m ˉ T \bar{m}_T m ˉ T ,
∥ Σ ^ x , J J − Σ x , J J ∥ o p ≲ p m ˉ T λ . \left\lVert\widehat{\boldsymbol{\Sigma}}_{x,JJ}-\boldsymbol{\Sigma}_{x,JJ}\right\rVert_{\mathrm{op}}\lesssim_p \bar{m}_T\sqrt{\lambda}. Σ x , J J − Σ x , J J op ≲ p m ˉ T λ .
So plugging in the obtained rate m ˉ T ≍ s 0 λ − q s / 2 + λ − 1 \bar{m}_T\asymp s_0\lambda^{-q_s/2}+\lambda^{-1} m ˉ T ≍ s 0 λ − q s /2 + λ − 1 one obtains s 0 λ ( 1 − q s ) / 2 + λ − 1 / 2 s_0\lambda^{(1-q_s)/2}+\lambda^{-1/2} s 0 λ ( 1 − q s ) /2 + λ − 1/2 . Since λ = C λ log ( K p ) / T → 0 \lambda=C_\lambda\log(Kp)/T\to0 λ = C λ log ( K p ) / T → 0 under Assumption 1, the second term diverges. Thus, at the crude support scale m ˉ T \bar{m}_T m ˉ T , the available max-norm concentration bound is not sufficient to transfer a population RE condition to the corresponding sample Gram submatrices. We therefore have to impose a preliminary sample RE condition uniformly over the deterministic class J m ˉ T \mathcal{J}_{\bar{m}_T} J m ˉ T .
Assumption 3 (Preliminary Sample RE). For the deterministic sequence m ˉ T \bar{m}_T m ˉ T in (17) and J m ˉ T : = { J ⊆ { 1 , … , K p } : ∣ J ∣ ≤ m ˉ T } \mathcal{J}_{\bar{m}_T}:=\{J\subseteq\{1,\ldots,Kp\}:|J|\leq\bar{m}_T\} J m ˉ T := { J ⊆ { 1 , … , K p } : ∣ J ∣ ≤ m ˉ T } , define
E c r u d e R E : = { min J ∈ J m ˉ T λ min ( Σ ^ x , J J ) ≥ κ c r u d e } , (18) \mathcal{E}_{\mathrm{crudeRE}}:=\left\{\min_{J\in\mathcal{J}_{\bar{m}_T}}\lambda_{\min}\left(\widehat{\boldsymbol{\Sigma}}_{x,JJ}\right)\geq\kappa_{\mathrm{crude}}\right\}, \tag*{(18)} E crudeRE := { J ∈ J m ˉ T min λ m i n ( Σ x , J J ) ≥ κ crude } , (18) where κ c r u d e > 0 \kappa_{\mathrm{crude}}>0 κ crude > 0 . Assume that P ( E c r u d e R E ) → 1 \mathbb{P}(\mathcal{E}_{\mathrm{crudeRE}})\to1 P ( E crudeRE ) → 1 .
The need for this high-level, preliminary sample RE arises because BAR determines its support through the activation threshold T j ( λ ) T_j(\lambda) T j ( λ ) , which determines which coordinates remain active but does not directly constrain their number.18 Assumption 3 is therefore an intermediate requirement of the present argument. It provides the preliminary curvature needed to tighten the crude bound on ∣ U ∣ |U| ∣ U ∣ using the BAR fixed-point equations. The resulting refinement below reduces in fact the realized support union from s 0 λ − q s / 2 + λ − 1 s_0\lambda^{-q_s/2}+\lambda^{-1} s 0 λ − q s /2 + λ − 1 to only the sparsity scale s 0 λ − q s / 2 s_0\lambda^{-q_s/2} s 0 λ − q s /2 . This is important for two reasons: first, as seen, at this scale the population-to-sample curvature transfer becomes feasible under the deviation bound of Lemma 2.19 Second, as will become clear later, the support size plays a fundamental role also in order to obtain the sharp estimation and prediction error rates derived below, so regardless of the RE, the refinement from s 0 λ − q s / 2 + λ − 1 s_0\lambda^{-q s/2}+\lambda^{-1} s 0 λ − q s /2 + λ − 1 to s 0 λ − q s / 2 s_0\lambda^{-q s/2} s 0 λ − q s /2 is necessary. The starting point for this refinement is the following quadratic identity on U U U .
Lemma 4 (Quadratic Identity on U U U ). Let v : = T − 1 X ′ ( y − X β ^ B A R ) v := T^{-1}X^{\prime}(\boldsymbol{y}-X\widehat{\beta}_{BAR}) v := T − 1 X ′ ( y − X β B A R ) . On U : = S λ ∪ S ^ U := S_{\lambda} \cup\widehat{S} U := S λ ∪ S , the estimation error Δ : = β ^ B A R − β \Delta:= \widehat{\beta}_{BAR}-\beta Δ := β B A R − β satisfies
Δ U ′ Σ ^ x , U U Δ U + ∑ j ∈ U Δ j v j = Δ U ′ T − 1 X U ′ u + Δ U ′ Σ ^ x , U U c β U c . (19) \Delta_U^{\prime}\widehat{\Sigma}_{x,UU}\Delta_U+\sum_{j\in U}\Delta_jv_j=\Delta_U^{\prime}T^{-1}X_U^{\prime}\boldsymbol{u}+\Delta_U^{\prime}\widehat{\Sigma}_{x,UU^c}\beta_{U^c}. \tag*{(19)} Δ U ′ Σ x , U U Δ U + j ∈ U ∑ Δ j v j = Δ U ′ T − 1 X U ′ u + Δ U ′ Σ x , U U c β U c . (19)
Equation (19) provides the device for reducing the preliminary bound on ∣ U ∣ |U| ∣ U ∣ to the desired rate. Since U = S λ ∪ S ^ U=S_{\lambda}\cup\widehat{S} U = S λ ∪ S , we have ∣ U ∣ ≤ ∣ S λ ∣ + ∣ F ∣ |U|\leq|S_{\lambda}|+|F| ∣ U ∣ ≤ ∣ S λ ∣ + ∣ F ∣ , where F : = S ^ ∖ S λ F:=\widehat{S}\setminus S_{\lambda} F := S ∖ S λ denotes the set of coordinates selected by BAR as active even though their true coefficients lie below the population threshold T ( λ ) T(\lambda) T ( λ ) , i.e., the “false positives”. The term ∣ S λ ∣ |S_{\lambda}| ∣ S λ ∣ is already controlled at the desired sparsity scale by Lemma 1. Hence, it remains to show that ∣ F ∣ |F| ∣ F ∣ is of the same order. The false positives in F F F need to be treated differently depending on the size of their true coefficients. If ∣ β j ∣ |\beta_j| ∣ β j ∣ lies just below the activation threshold, selection by BAR need not imply a large estimation error, but the number of such coefficients can itself be controlled by weak sparsity. If instead ∣ β j ∣ |\beta_j| ∣ β j ∣ is sufficiently below the threshold, then BAR selection, together with ∣ β ^ j ∣ ≥ λ |\widehat{\beta}_j|\geq\sqrt{\lambda} ∣ β j ∣ ≥ λ , forces a non-negligible discrepancy between β ^ j \widehat{\beta}_j β j and β j \beta_j β j . The fixed-point identity (19) , combined with the preliminary sample RE, can then be used to limit how many such coordinates can be selected. This motivates the following decomposition. Fix c 0 ∈ ( 0 , 1 ) c_0\in(0,1) c 0 ∈ ( 0 , 1 ) and define
B λ : = { j ∈ S λ c : c 0 λ < ∣ β j ∣ ≤ 2 λ } , D λ : = { j ∈ S λ c : ∣ β j ∣ ≤ c 0 λ } , F B : = F ∩ B λ , F D : = F ∩ D λ . (20) \begin{aligned}
B_{\lambda}&:=\{j\in S_{\lambda}^{c}:c_0\sqrt{\lambda}<|\beta_j|\leq2\sqrt{\lambda}\}, \qquad D_{\lambda}:=\{j\in S_{\lambda}^{c}:|\beta_j|\leq c_0\sqrt{\lambda}\},\\
F_B&:=F\cap B_{\lambda}, \qquad F_D:=F\cap D_{\lambda}. \tag*{(20)}
\end{aligned} B λ F B := { j ∈ S λ c : c 0 λ < ∣ β j ∣ ≤ 2 λ } , D λ := { j ∈ S λ c : ∣ β j ∣ ≤ c 0 λ } , := F ∩ B λ , F D := F ∩ D λ . (20)
Since F B F_B F B and F D F_D F D form a partition of F F F , we can first control ∣ F B ∣ |F_B| ∣ F B ∣ by weak sparsity.
Lemma 5 (False positives near the threshold). Under Assumption 2,
∣ F B ∣ ≤ ∣ B λ ∣ ≲ s 0 λ − q s / 2 . |F_B| \le|B_\lambda| \lesssim s_0\lambda^{-q_s/2}. ∣ F B ∣ ≤ ∣ B λ ∣ ≲ s 0 λ − q s /2 .
Now, since ∣ U ∣ ≤ ∣ S λ ∣ + ∣ F B ∣ + ∣ F D ∣ |U| \le|S_\lambda| + |F_B| + |F_D| ∣ U ∣ ≤ ∣ S λ ∣ + ∣ F B ∣ + ∣ F D ∣ and both ∣ S λ ∣ |S_\lambda| ∣ S λ ∣ and ∣ F B ∣ |F_B| ∣ F B ∣ are already of order s 0 λ − q s / 2 s_0\lambda^{-q_s/2} s 0 λ − q s /2 , the only component that can account for the additional λ − 1 \lambda^{-1} λ − 1 term in the crude bound m ˉ T ≲ s 0 λ − q s / 2 + λ − 1 \bar{m}_T \lesssim s_0\lambda^{-q_s/2} + \lambda^{-1} m ˉ T ≲ s 0 λ − q s /2 + λ − 1 is F D F_D F D . Therefore, removing the λ − 1 \lambda^{-1} λ − 1 term from the bound on ∣ U ∣ |U| ∣ U ∣ amounts to showing that ∣ F D ∣ = O p ( s 0 λ − q s / 2 ) |F_D| = O_p(s_0\lambda^{-q_s/2}) ∣ F D ∣ = O p ( s 0 λ − q s /2 ) . For j ∈ F D j \in F_D j ∈ F D , we have ∣ β j ∣ ≤ c 0 λ |\beta_j| \le c_0\sqrt{\lambda} ∣ β j ∣ ≤ c 0 λ , while BAR selection implies ∣ β ^ j ∣ ≥ λ |\hat{\beta}_j| \ge\sqrt{\lambda} ∣ β ^ j ∣ ≥ λ . Hence, each such coordinate inflates the term ∑ j ∈ U Δ j v j \sum_{j\in U}\Delta_jv_j ∑ j ∈ U Δ j v j on the left-hand side of (19) by at least ( 1 − c 0 ) λ (1-c_0)\lambda ( 1 − c 0 ) λ , as formalized next.
Lemma 6 (Contribution of F D F_D F D ). For every j ∈ F D j \in F_D j ∈ F D , the fixed-point identity (7) and the active-coordinate bound ∣ β ^ j ∣ ≥ λ |\hat{\beta}_j| \ge\sqrt{\lambda} ∣ β ^ j ∣ ≥ λ imply
Δ j v j = λ ( 1 − β j β ^ j ) ≥ ( 1 − c 0 ) λ , ∑ j ∈ F D Δ j v j ≥ ( 1 − c 0 ) λ ∣ F D ∣ . (21) \Delta_jv_j = \lambda\left(1-\frac{\beta_j}{\hat{\beta}_j}\right) \ge(1-c_0)\lambda,\qquad\sum_{j\in F_D}\Delta_jv_j \ge(1-c_0)\lambda|F_D|. \tag*{(21)} Δ j v j = λ ( 1 − β ^ j β j ) ≥ ( 1 − c 0 ) λ , j ∈ F D ∑ Δ j v j ≥ ( 1 − c 0 ) λ ∣ F D ∣. (21)
If d = ∣ F D ∣ d = |F_D| d = ∣ F D ∣ such coordinates are selected, their contribution to ∑ j ∈ U Δ j v j \sum_{j\in U}\Delta_jv_j ∑ j ∈ U Δ j v j is at least ( 1 − c 0 ) λ d (1-c_0)\lambda d ( 1 − c 0 ) λ d . Equation (19) then limits how large d d d can be. In particular, on U ∈ J m ˉ T U \in\mathcal{J}_{\bar{m}_T} U ∈ J m ˉ T , Assumption 3 gives the lower bound Δ U ′ Σ ^ x , U U Δ U ≥ κ c r u d e ∥ Δ U ∥ 2 2 \Delta_U^\prime\widehat{\Sigma}_{x,UU}\Delta_U \ge\kappa_{\mathrm{crude}}\|\Delta_U\|_2^2 Δ U ′ Σ x , U U Δ U ≥ κ crude ∥ Δ U ∥ 2 2 . The empirical-noise term Δ U ′ T − 1 X U ′ u \Delta_U^\prime T^{-1}X_U^\prime u Δ U ′ T − 1 X U ′ u is controlled on E χ \mathcal{E}_\chi E χ by Lemma 2, while the interaction Δ U ′ Σ ^ x , U U c β U c \Delta_U^\prime\widehat{\Sigma}_{x,UU^c}\beta_{U^c} Δ U ′ Σ x , U U c β U c is controlled using the Gram-matrix deviation in the same lemma, combined with an upper eigenvalue bound on Σ x \Sigma_x Σ x , and the bounds on ∥ β S x c ∥ r \|\beta_{S_x^c}\|_r ∥ β S x c ∥ r , r ∈ { 1 , 2 } r \in\{1,2\} r ∈ { 1 , 2 } . Finally, the remaining terms that can contribute inflating ∣ F D ∣ |F_D| ∣ F D ∣ , namely S λ S_\lambda S λ and F B F_B F B , are already controlled by their cardinalities. Balancing these terms against ( 1 − c 0 ) λ d (1-c_0)\lambda d ( 1 − c 0 ) λ d yields the following refinement.
Proposition 2 (Support-union dimension refinement). Assume λ max ( Σ x ) ≤ C max \lambda_{\max}(\Sigma_x) \le C_{\max} λ m a x ( Σ x ) ≤ C m a x and let λ = C λ log ( K p ) / T \lambda= C_\lambda\log(Kp)/T λ = C λ log ( K p ) / T , c u : = C χ / C λ c_u := C_\chi/\sqrt{C_\lambda} c u := C χ / C λ , a u : = c u 2 / κ c r u d e a_u := c_u^2/\kappa_{\mathrm{crude}} a u := c u 2 / κ crude , and G : = E y ∩ E c r u d e R E ∩ E χ \mathcal{G} := \mathcal{E}_y \cap\mathcal{E}_{\mathrm{crudeRE}} \cap\mathcal{E}_\chi G := E y ∩ E crudeRE ∩ E χ . If a u < 1 − c 0 a_u < 1-c_0 a u < 1 − c 0 and s 0 λ ( 1 − q s ) / 2 = o ( 1 ) s_0\lambda^{(1-q_s)/2} = o(1) s 0 λ ( 1 − q s ) /2 = o ( 1 ) , then on G \mathcal{G} G ,
∣ F D ∣ ≤ C 1 ( ∣ S λ ∣ + ∣ F B ∣ ) + C 2 λ − 1 { ∥ β S x c ∥ 2 2 + λ ∥ β S x c ∥ 1 2 } . (22) |F_D| \le C_1(|S_\lambda|+|F_B|)+C_2\lambda^{-1}\left\{\|\beta_{S_x^c}\|_2^2+\sqrt{\lambda}\|\beta_{S_x^c}\|_1^2\right\}. \tag*{(22)} ∣ F D ∣ ≤ C 1 ( ∣ S λ ∣ + ∣ F B ∣ ) + C 2 λ − 1 { ∥ β S x c ∥ 2 2 + λ ∥ β S x c ∥ 1 2 } . (22)
Consequently, there exists a fixed constant C U > 0 C_U>0 C U > 0 such that, for all sufficiently large T T T ,
∣ U ∣ ≤ C U s 0 λ − q s / 2 on G . (23) |U| \le C_U s_0\lambda^{-q_s/2}\qquad\text{on }\mathcal{G}. \tag*{(23)} ∣ U ∣ ≤ C U s 0 λ − q s /2 on G . (23)
Hence, if P ( G ) → 1 \mathbb{P}(\mathcal{G})\to1 P ( G ) → 1 , ∣ U ∣ = O p ( s 0 λ − q s / 2 ) |U|=\mathcal{O}_p(s_0\lambda^{-q_s/2}) ∣ U ∣ = O p ( s 0 λ − q s /2 ) .
The bound in (22) is of the desired order. Indeed, ∣ S λ ∣ + ∣ F B ∣ ≲ s 0 λ − q s / 2 |S_\lambda|+|F_B|\lesssim s_0\lambda^{-q_s/2} ∣ S λ ∣ + ∣ F B ∣ ≲ s 0 λ − q s /2 , while λ − 1 ∥ β S λ c ∥ 2 2 ≲ s 0 λ − q s / 2 \lambda^{-1}\|\beta_{S_\lambda^c}\|_2^2\lesssim s_0\lambda^{-q_s/2} λ − 1 ∥ β S λ c ∥ 2 2 ≲ s 0 λ − q s /2 . The remaining term is of smaller order under s 0 λ ( 1 − q s ) / 2 = o ( 1 ) s_0\lambda^{(1-q_s)/2}=o(1) s 0 λ ( 1 − q s ) /2 = o ( 1 ) , since s 0 λ ( 1 − q s ) / 2 = ( s 0 λ − q s / 2 ) λ = o ( 1 ) s_0\lambda^{(1-q_s)/2}=(s_0\lambda^{-q_s/2})\sqrt{\lambda}=o(1) s 0 λ ( 1 − q s ) /2 = ( s 0 λ − q s /2 ) λ = o ( 1 ) . Hence ∣ F D ∣ = O p ( s 0 λ − q s / 2 ) |F_D|=\mathcal{O}_p(s_0\lambda^{-q_s/2}) ∣ F D ∣ = O p ( s 0 λ − q s /2 ) and, together with the bounds for S λ S_\lambda S λ and F B F_B F B , ∣ U ∣ = O p ( s 0 λ − q s / 2 ) |U|=\mathcal{O}_p(s_0\lambda^{-q_s/2}) ∣ U ∣ = O p ( s 0 λ − q s /2 ) .
Having shrunk down the cardinality of U U U from m ˉ T ≲ s 0 λ − q s / 2 + λ − 1 \bar m_T\lesssim s_0\lambda^{-q_s/2}+\lambda^{-1} m ˉ T ≲ s 0 λ − q s /2 + λ − 1 to m T ♯ : = ⌈ C U s 0 λ − q s / 2 ⌉ m_T^\sharp:=\lceil C_U s_0\lambda^{-q_s/2}\rceil m T ♯ := ⌈ C U s 0 λ − q s /2 ⌉ , we can use Lemma 2 to transfer a population RE condition to the sample Gram matrix on this refined support scale. Thus, although the high-level sample condition in Assumption 3 is needed for the preliminary support-refinement step, the curvature used in the final error analysis can instead be derived from the more natural population RE condition in (24) .
Lemma 7 (RE on the BAR support union). Assume that
min J ⊆ { 1 , … , K p } ∣ J ∣ ≤ m T ♯ λ min ( Σ x , J J ) ≥ κ > 0. (24) \min_{\substack{J\subseteq\{1,\ldots,K_p\}\\ |J|\le m_T^\sharp}}\lambda_{\min}(\Sigma_{x,JJ})\ge\kappa>0. \tag*{(24)} J ⊆ { 1 , … , K p } ∣ J ∣ ≤ m T ♯ min λ m i n ( Σ x , J J ) ≥ κ > 0. (24) Under s 0 λ ( 1 − q s ) / 2 = o ( 1 ) s_0\lambda^{(1-q_s)/2}=o(1) s 0 λ ( 1 − q s ) /2 = o ( 1 ) , for all sufficiently large T T T , on { ∣ U ∣ ≤ m T ♯ } ∩ E χ \{|U|\le m_T^\sharp\}\cap\mathcal{E}_\chi { ∣ U ∣ ≤ m T ♯ } ∩ E χ , λ min ( Σ ^ x , U U ) ≥ κ / 2 \lambda_{\min}(\widehat{\Sigma}_{x,UU})\ge\kappa/2 λ m i n ( Σ x , U U ) ≥ κ /2 .
Indeed, on { ∣ U ∣ ≤ m T ♯ } ∩ E χ \{|U|\le m_T^\sharp\}\cap\mathcal{E}_\chi { ∣ U ∣ ≤ m T ♯ } ∩ E χ , Lemma 2 gives
∥ Σ ^ x , U U − Σ x , U U ∥ o p ≤ C χ m T ♯ log ( K p ) T = O ( s 0 λ ( 1 − q s ) / 2 ) = o ( 1 ) , \left\|\widehat{\Sigma}_{x,UU}-\Sigma_{x,UU}\right\|_{\mathrm{op}}\le C_\chi m_T^\sharp\sqrt{\frac{\log(K_p)}{T}}=\mathcal{O}\left(s_0\lambda^{(1-q_s)/2}\right)=o(1), Σ x , U U − Σ x , U U op ≤ C χ m T ♯ T log ( K p ) = O ( s 0 λ ( 1 − q s ) /2 ) = o ( 1 ) ,
where the last step follows from s 0 λ ( 1 − q s ) / 2 = o ( 1 ) s_0\lambda^{(1-q_s)/2}=o(1) s 0 λ ( 1 − q s ) /2 = o ( 1 ) . Weyl’s inequality then yields the claimed sample RE. By Proposition 2 and Lemma 2, the event { ∣ U ∣ ≤ m T ♯ } ∩ E χ \{|U|\le m_T^\sharp\}\cap\mathcal{E}_\chi { ∣ U ∣ ≤ m T ♯ } ∩ E χ has probability tending to one.
We can now return to the cross-design term that motivated this subsection. Its difficulty came from the fact that it depends on the unknown estimation error Δ \Delta Δ . The support-union representation separates this error into the unknown component Δ U \Delta_U Δ U and the controlled component Δ U c = − β U c \Delta_{U^c}=-\beta_{U^c} Δ U c = − β U c . In the quadratic identity (19) , the former is handled through the restricted curvature of Σ ^ x , U U \widehat{\Sigma}_{x,UU} Σ x , U U , while the latter is controlled by the weak-sparsity bounds together with the Gram-matrix deviation. This resolves the cross-design difficulty without requiring a direct coordinatewise bound on the original term.
With the RE condition verified on the refined support scale m T ♯ m_T^{\sharp} m T ♯ , we can now derive the high-probability error bounds for the BAR estimator. We return to the quadratic inequality used in Proposition 2, substitute the sample curvature bound λ min ( Σ ^ x , U U ) ≥ κ / 2 \lambda_{\min}(\widehat{\Sigma}_{x,UU})\geq\kappa/2 λ m i n ( Σ x , U U ) ≥ κ /2 , and solve for ∥ Δ U ∥ 2 \|\Delta_U\|_2 ∥ Δ U ∥ 2 . The full ℓ 2 \ell_2 ℓ 2 error then follows from Δ = Δ U − β U c \Delta=\Delta_U-\beta_{U^c} Δ = Δ U − β U c together with the bounds on ∥ β U c ∥ 2 \|\beta_{U^c}\|_2 ∥ β U c ∥ 2 . The ℓ 1 \ell_1 ℓ 1 rate is obtained similarly using ∥ Δ U ∥ 1 ≤ ∣ U ∣ ∥ Δ U ∥ 2 \|\Delta_U\|_1\leq\sqrt{|U|}\|\Delta_U\|_2 ∥ Δ U ∥ 1 ≤ ∣ U ∣ ∥ Δ U ∥ 2 , while the prediction error follows from the same quadratic inequality.
Theorem 1 (Estimation and Prediction Error Rates). Suppose the conditions of Proposition 2 and Lemma 7 hold and λ ≍ C λ log ( K p ) / T \lambda\asymp C_\lambda\log(Kp)/T λ ≍ C λ log ( K p ) / T . On the event E T : = G ∩ { ∣ U ∣ ≤ m T ♯ } \mathcal{E}_T:=\mathcal{G}\cap\{|U|\leq m_T^{\sharp}\} E T := G ∩ { ∣ U ∣ ≤ m T ♯ } , whose probability tends to one,
∥ β ^ B A R − β ∥ 2 = O p ( s 0 λ 1 − q s / 2 ) , ∥ β ^ B A R − β ∥ 1 = O p ( s 0 λ ( 1 − q s ) / 2 ) , (25) \|\widehat{\beta}_{BAR}-\beta\|_2=\mathcal{O}_p\left(s_0\lambda^{1-q_s/2}\right),\qquad\|\widehat{\beta}_{BAR}-\beta\|_1=\mathcal{O}_p\left(s_0\lambda^{(1-q_s)/2}\right), \tag*{(25)} ∥ β B A R − β ∥ 2 = O p ( s 0 λ 1 − q s /2 ) , ∥ β B A R − β ∥ 1 = O p ( s 0 λ ( 1 − q s ) /2 ) , (25) T − 1 ∥ X ( β ^ B A R − β ) ∥ 2 2 = O p ( s 0 λ 1 − q s / 2 ) . (26) T^{-1}\|X(\widehat{\beta}_{BAR}-\beta)\|_2^2=\mathcal{O}_p\left(s_0\lambda^{1-q_s/2}\right). \tag*{(26)} T − 1 ∥ X ( β B A R − β ) ∥ 2 2 = O p ( s 0 λ 1 − q s /2 ) . (26)
Corollary 2. Under the conditions of Theorem 1, the prediction, ℓ 2 \ell_2 ℓ 2 , and ℓ 1 \ell_1 ℓ 1 errors all vanish if
s 0 { log ( K p ) T } ( 1 − q s ) / 2 → 0. (27) s_0\left\{\frac{\log(Kp)}{T}\right\}^{(1-q_s)/2}\to0. \tag*{(27)} s 0 { T log ( K p ) } ( 1 − q s ) /2 → 0. (27) In particular, bounded s 0 s_0 s 0 suffices.
Condition (27) coincides with s 0 λ ( 1 − q s ) / 2 = o ( 1 ) s_0\lambda^{(1-q_s)/2}=o(1) s 0 λ ( 1 − q s ) /2 = o ( 1 ) from Proposition 2, which was already used for the support refinement and the population-to-sample RE transfer. It is also the condition required for the ℓ 1 \ell_1 ℓ 1 error in Theorem 1 to vanish, while the prediction and $\ell_2 rates alone allow weaker sparsity growth. Under Assumption 1, K p = O ( T ν ) Kp = O(T^{\nu}) K p = O ( T ν ) for some fixed ν > 0 \nu> 0 ν > 0 , so a sufficient growth condition is s 0 = o { ( T / log T ) ( 1 − q s ) / 2 } s_0 = o\{(T/\log T)^{(1-q_s)/2}\} s 0 = o {( T / log T ) ( 1 − q s ) /2 } .
Information-criterion tuning of the BAR penalty
The theory considers the deterministic penalty sequence λ = C λ log ( K p ) / T \lambda= C_{\lambda}\log(Kp)/T λ = C λ log ( K p ) / T . The multiplier C λ C_{\lambda} C λ must be sufficiently large relative to the constants entering the concentration and preliminary curvature bounds; in particular, Proposition 2 requires C λ > C χ 2 κ c r u d e ( 1 − c 0 ) C_{\lambda} > \frac{C_{\chi}^{2}}{\kappa_{\mathrm{crude}}(1-c_0)} C λ > κ crude ( 1 − c 0 ) C χ 2 . This condition is not directly feasible, however, since C χ C_{\chi} C χ and κ c r u d e \kappa_{\mathrm{crude}} κ crude are unknown without strong parametric assumptions on the innovation distribution, the dependence and design structure. Algorithm 2 therefore selects the penalty from a data-driven grid using an information criterion. The Ridge initializer parameter ξ \xi ξ is selected by BIC, whereas the sparsity-inducing penalty λ \lambda λ is selected along the BAR path by an information criterion. The two tuning parameters play distinct roles but are not independent: ξ \xi ξ determines the initial Ridge estimate β ^ ( 0 ) \hat{\beta}^{(0)} β ^ ( 0 ) and thereby affects the scale of the data-driven BAR penalty path, whereas λ \lambda λ governs thresholding and sparsification.
For each candidate λ \lambda λ , let R S S ( λ ) : = ∥ y − X β ^ B A R ( λ ) ∥ 2 2 RSS(\lambda) := \lVert y - X\hat{\beta}_{BAR}(\lambda)\rVert_2^2 R S S ( λ ) := ∥ y − X β ^ B A R ( λ ) ∥ 2 2 and S ^ ( λ ) : = supp { β ^ B A R ( λ ) } \hat{S}(\lambda) := \operatorname{supp}\{\hat{\beta}_{BAR}(\lambda)\} S ^ ( λ ) := supp { β ^ B A R ( λ )} . For the sparsity-inducing estimators considered below, we use the generic criterion
I C τ ( λ ) = log { R S S ( λ ) / T } + τ T T ∣ S ^ ( λ ) ∣ . (28) IC_{\tau}(\lambda) = \log\{RSS(\lambda)/T\} + \frac{\tau_T}{T}|\hat{S}(\lambda)|. \tag*{(28)} I C τ ( λ ) = log { R S S ( λ ) / T } + T τ T ∣ S ^ ( λ ) ∣. (28)
The usual choices are τ T = 2 \tau_T = 2 τ T = 2 for AIC, 2 log log T 2\log\log T 2 log log T for HQ, log T \log T log T for BIC, log T + 1 \log T + 1 log T + 1 for CAIC, and log T + 2 γ log ( K p ) \log T + 2\gamma\log(Kp) log T + 2 γ log ( K p ) for EBIC, with γ ∈ [ 0 , 1 ] \gamma\in[0,1] γ ∈ [ 0 , 1 ] . The BIC used to select the Ridge initializer ξ \xi ξ is computed separately using the corresponding Ridge effective degrees of freedom, rather than the support cardinality in (28) .
Under the polynomial dimensional regime in Assumption 1(iv), log ( K p ) = O ( log T ) \log(Kp) = O(\log T) log ( K p ) = O ( log T ) , so BIC and EBIC have complexity weights of the same asymptotic order. EBIC nevertheless adds the dimension-dependent term 2 γ log ( K p ) 2\gamma\log(Kp) 2 γ log ( K p ) and can therefore impose substantially stronger penalization when the number of candidate regressors is large. We use BIC for the Ridge initializer and EBIC for the sparsity-inducing BAR penalty as a practical calibration choice. Accordingly, the theoretical results above apply to the deterministic penalty sequence λ = C λ log ( K p ) / T \lambda= C_{\lambda}\log(Kp)/T λ = C λ log ( K p ) / T and are not intended as a formal justification of the EBIC-selected penalty. In particular, we do not claim that the data-driven λ ^ \hat{\lambda} λ ^ automatically satisfies the sufficiently-large-multiplier condition required by the theoretical results.
Nonetheless, the tuning behavior in the simulations provides further motivation for this choice. Table 1 reports the median position of the selected penalty in the 100-point grid, ordered from the largest λ \lambda λ at position 1 to the smallest λ \lambda λ at position 100. Hence, larger positions correspond to weaker regularization.
Table 1. Median position of the selected λ \lambda λ in the 100-point grid
When K ≤ T K \le T K ≤ T , BIC and EBIC generally select similar regions of each estimator’s tuning path. When K > T K > T K > T , however, BIC frequently selects penalties at or near the weakest-regularization boundary. For BAR, for example, the median selected position moves from 96 under BIC to 49 under EBIC, while analogous shifts toward stronger regularization are visible for the competing sparse estimators. Thus, EBIC avoids the systematic tendency of BIC to select the weakest penalties in the high-dimensional designs considered here. This evidence does not establish selection consistency or verify the sufficiently-large-multiplier condition used in the theory, but it provides a finite-sample motivation for using EBIC to tune the sparsity-inducing penalty.
Monte Carlo simulations
We assess the finite-sample performance of BAR through a simulation study based on the K K K -dimensional VAR(1)
y t = A y t − 1 + u t , A = V − 1 D V , (29) \boldsymbol{y}_t = \boldsymbol{A}\boldsymbol{y}_{t-1} + \boldsymbol{u}_t,\qquad\boldsymbol{A} = \boldsymbol{V}^{-1}\boldsymbol{D}\boldsymbol{V}, \tag*{(29)} y t = A y t − 1 + u t , A = V − 1 D V , (29)
where, for a block of size r r r , V = I K / r ⊗ ν \boldsymbol{V} = I_{K/r} \otimes\boldsymbol{\nu} V = I K / r ⊗ ν , with [ ν ] i j = cos ( 2 π i j / r ) + sin ( 2 π i j / r ) [\boldsymbol{\nu}]_{ij} = \cos(2\pi ij/r) + \sin(2\pi ij/r) [ ν ] ij = cos ( 2 π ij / r ) + sin ( 2 π ij / r ) , while
D = I K / r ⊗ diag ( s ⊙ δ ) \boldsymbol{D} = I_{K/r} \otimes\operatorname{diag}(\boldsymbol{s} \odot\boldsymbol{\delta}) D = I K / r ⊗ diag ( s ⊙ δ ) , where δ = ( δ 1 , … , δ r ) ′ \boldsymbol{\delta} = (\delta_1,\ldots,\delta_r)^{\prime} δ = ( δ 1 , … , δ r ) ′ contains equally spaced values in [ 0.25 , 0.75 ] [0.25,0.75] [ 0.25 , 0.75 ] and s = ( s 1 , … , s r ) ′ \boldsymbol{s} = (s_1,\ldots,s_r)^{\prime} s = ( s 1 , … , s r ) ′ is a deterministic alternating-sign vector (s j = ( − 1 ) j − 1 s_j = (-1)^{j-1} s j = ( − 1 ) j − 1 ).20 When r = 1 r=1 r = 1 , we set δ 1 = 0.5 \delta_1=0.5 δ 1 = 0.5 . The resulting A \boldsymbol{A} A is block diagonal, and each equation contains exactly r r r nonzero autoregressive coefficients. Thus r = 1 r=1 r = 1 yields a diagonal system, whereas larger values of r r r generate progressively denser within-block dynamics. Each replication is generated with a burn-in of 50 observations, which is discarded. The following T T T observations are used for estimation, while the subsequent observation T + 1 T+1 T + 1 is reserved for one-step-ahead forecast evaluation. The innovations are Gaussian, u t ∼ i i d N K ( 0 , Σ u ) \boldsymbol{u}_t \mathrel{\overset{\mathrm{iid}}{\sim}} \mathcal{N}_K(0,\boldsymbol{\Sigma}_u) u t ∼ iid N K ( 0 , Σ u ) , with Toeplitz covariance matrix [ Σ u ] i j = ρ ∣ i − j ∣ [\boldsymbol{\Sigma}_u]_{ij} = \rho^{|i-j|} [ Σ u ] ij = ρ ∣ i − j ∣ . We consider ρ ∈ { 0 , 0.5 } \rho\in\{0,0.5\} ρ ∈ { 0 , 0.5 } , where ρ = 0 \rho=0 ρ = 0 corresponds to cross-sectionally independent innovations and ρ = 0.5 \rho=0.5 ρ = 0.5 introduces contemporaneous dependence. The simulation design covers K ∈ { 20 , 40 , 80 , 160 , 320 } K \in\{20,40,80,160,320\} K ∈ { 20 , 40 , 80 , 160 , 320 } , T ∈ { 50 , 100 , 500 } T \in\{50,100,500\} T ∈ { 50 , 100 , 500 } , r ∈ { 1 , 2 , 5 , 10 } r \in\{1,2,5,10\} r ∈ { 1 , 2 , 5 , 10 } , and ρ ∈ { 0 , 0.5 } \rho\in\{0,0.5\} ρ ∈ { 0 , 0.5 } . We compare BAR with adaptive Lasso initialized by Ridge (aLASSO(ridge)), adaptive Lasso initialized by Lasso (aLASSO(lasso)), Lasso, and SCAD. The sparsity-inducing penalties are selected by EBIC with γ = 0.5 \gamma=0.5 γ = 0.5 . The adaptive-Lasso weights are constructed as the inverse absolute value of the corresponding initial estimates, while the SCAD shape parameter is set to 3.7. All estimators use the same centering and standardization convention.
Performance is evaluated through the five criteria summarized in Table 2 . Let S = { ( i , j ) : \mathcal{S} = \{(i,j): S = {( i , j ) : ∣ A i j ∣ > 0 \lvert A_{ij}\rvert> 0 ∣ A ij ∣ > 0 denote the true support and S ^ \widehat{S} S its estimate.
Table 2. Monte Carlo performance measures.
For each ( T , ρ ) (T,\rho) ( T , ρ ) , the performance measures are first computed over the N M C = 1000 N_{\mathrm{MC}}=1000 N MC = 1000 replications for each of the 20 combinations of ( K , r ) (K,r) ( K , r ) . A win is assigned to the estimator attaining the best value after rounding the corresponding measure to three decimal places. Ties are counted as wins for all tied estimators, except when all methods attain zero for Correct sparsity or True model included, in which case no win is assigned. The win rates are then computed across the 20 ( K , r ) (K,r) ( K , r ) designs. Figure 3 reports their simple averages over the six combinations of T ∈ { 50 , 100 , 500 } T\in\{50,100,500\} T ∈ { 50 , 100 , 500 } and ρ ∈ { 0 , 0.5 } \rho\in\{0,0.5\} ρ ∈ { 0 , 0.5 } .
Figure 3. Average Monte Carlo win rates across the six combinations of T ∈ { 50 , 100 , 500 } T \in\{50, 100, 500\} T ∈ { 50 , 100 , 500 } and ρ ∈ { 0 , 0.5 } \rho\in\{0, 0.5\} ρ ∈ { 0 , 0.5 } . Each underlying win rate is aggregated over ( K , r ) (K, r) ( K , r ) . For RMSFE and RMSE, a win corresponds to the smallest value; for the remaining criteria, it corresponds to the largest value. Since ties are counted as wins for all tied methods, the win rates need not sum to 100%.
As clear, BAR achieves the highest average win rate by a wide margin for forecasting accuracy, estimation accuracy, and the share of relevant variables selected. As expected, correct sparsity remains difficult for all methods, with SCAD attaining the highest average win rate for this criterion. True-model inclusion is more mixed, with Lasso performing best on average. The disaggregated results in the Appendix § 10 show how these patterns vary with the sample size and contemporaneous innovation dependence.
Empirical application
We examine whether the BAR improves forecasts of U.S. inflation using a large set of economic indicators. Accurate inflation forecasts are crucial for policymakers, financial institutions, firms, and households. Central banks rely on them to conduct monetary policy and anchor inflation expectations, while financial markets incorporate them into interest rates, asset prices, and investment decisions. Inflation expectations also shape firms’ and households’ employment, investment, and consumption choices. We use the FRED-MD database developed by McCracken & Ng [14 ] , which contains 126 monthly U.S. macroeconomic series spanning 1959–2025.21 To obtain the longest balanced sample, we exclude five series with substantial missing values and retain data from January 1960 to December 2025, yielding 792 monthly observations.22 The target variable is the all-items Consumer Price Index (CPIAUCSL). We evaluate forecast accuracy over three out-of-sample periods, each corresponding to a distinct economic environment:
2020–2021: the COVID-19 pandemic, a major exogenous shock;
2022–2025: the post-pandemic period, encompassing the inflation surge and subsequent disinflation during the monetary tightening cycle.
For forecasting, we apply BAR to the single-equation regression representation underlying the VAR analysis, allowing a forecasting-specific lag structure with four lags of inflation and the current value of each additional predictor:
y ^ t + h = ∑ i = 1 4 ϕ ^ i y t − i + 1 + ∑ j = 1 K − 1 θ ^ j x j , t , (30) \hat{y}_{t+h} = \sum_{i=1}^{4} \hat{\phi}_i y_{t-i+1} + \sum_{j=1}^{K-1} \hat{\theta}_j x_{j,t}, \tag*{(30)} y ^ t + h = i = 1 ∑ 4 ϕ ^ i y t − i + 1 + j = 1 ∑ K − 1 θ ^ j x j , t , (30)
where h h h denotes forecast horizon and K − 1 K-1 K − 1 is the number of additional predictors. We consider four forecast horizons, h ∈ { 1 , 3 , 6 , 12 } h \in\{1,3,6,12\} h ∈ { 1 , 3 , 6 , 12 } , covering a full year. Following Medeiros et al. [15 ] , the models are estimated over rolling windows. We use a window of W = 360 W = 360 W = 360 observations for forecasts produced over 1990–2000 and W = 492 W = 492 W = 492 observations from 2001 onward. Accordingly, the effective estimation sample is R h , p = W − h − p + 1 R_{h,p} = W - h - p + 1 R h , p = W − h − p + 1 , where p = 4 p = 4 p = 4 denotes the maximum lag order. For each horizon, all models are estimated using the direct forecasting approach, with the regularization parameter λ \lambda λ selected by minimizing the EBIC. All predictors are standardized to have zero mean and unit variance within each rolling estimation window. Forecast accuracy is evaluated using two measures:23
Root Mean Squared Forecast Error (RMSFE): 1 T ∑ t = 1 T ( e ^ t + h ∣ t ) 2 \sqrt{\frac{1}{T}\sum_{t=1}^{T}(\hat{e}_{t+h|t})^2} T 1 ∑ t = 1 T ( e ^ t + h ∣ t ) 2 ,
Mean Absolute Forecast Error (MAFE): 1 T ∑ t = 1 T ∣ e ^ t + h ∣ t ∣ \frac{1}{T}\sum_{t=1}^{T}|\hat{e}_{t+h|t}| T 1 ∑ t = 1 T ∣ e ^ t + h ∣ t ∣ ,
where e ^ t + h ∣ t = y ^ t + h ∣ t − y t + h \hat{e}_{t+h|t}=\hat{y}_{t+h|t}-y_{t+h} e ^ t + h ∣ t = y ^ t + h ∣ t − y t + h denotes forecast error, and T T T is the number of out-of-sample forecasts. Table 3 reports RMSFE and MAFE ratios relative to the AR(1) benchmark, so that values below one indicate superior forecast accuracy. The colored cells identify the Set of Superior Models (SSM) obtained from the 50% Model Confidence Set (MCS) procedure of Hansen et al. [8 ] . Gray cells identify models included in the SSM under squared-error loss, whereas blue cells indicate inclusion under absolute-error loss. During the pre-COVID period (1990–2019), BAR dominates at the one-month horizon: it achieves the lowest RMSFE and MAFE ratios and is the only model included in the SSM. At longer horizons, differences across models are relatively small. During the COVID period (2020–2021), BAR generally underperforms, except at h = 12 h=12 h = 12 , where it ranks first. In the post-COVID period (2022–2025), its performance is broadly comparable to that of the competing models.
Table 3. RMSFE and MAFE ratios relative to the AR(1). Notes: the RMSFE and the MAFE are relative to the AR(1) benchmark. Bold and underlined entries denote the best-performing model(s) for each forecast horizon and period. Grey cells indicate models belonging to the Set of Superior Models (SSM) under the 50% MCS procedure using the T max T_{\max} T m a x statistic with squared errors, whereas blue cells indicate models included in the SSM when absolute errors are used.
An insight on the evolution of forecast performance over time is provided in Figure 4 , which plots the cumulative sum of squared forecast error differentials (CSSFED) for each model relative to the AR(1) benchmark. A declining trajectory therefore indicates that the specific model is accumulating smaller squared forecast errors than the AR(1), whereas an increasing trajectory indicates a relative deterioration.24 At h = 1 h=1 h = 1 , BAR clearly outperforms the competing models: its CSSFED declines markedly between 2000 and 2010 and remains well below the other trajectories. At h = 3 h=3 h = 3 , the CSSFED paths are closely clustered, indicating broadly similar forecasting performance across models. By contrast, BAR deteriorates substantially at h = 6 h=6 h = 6 during the COVID-19 period and at h = 12 h=12 h = 12 in 2009, immediately after the 2008 financial crisis. Interestingly, these same episodes coincide with sizeable forecasting gains for BAR at the one-month horizon.
Figure 4. CSSFED relative to the AR(1).
The results show clear horizon dependence: BAR performs best at one month, while its advantage fades at longer horizons. This is consistent with short-run inflation being driven by a more concentrated set of signals, whereas longer-horizon predictability is more diffuse and less suited to aggressive thresholding.
Conclusion
This paper studies the Broken Adaptive Ridge (BAR) estimator for high-dimensional VARs. We show that BAR remains theoretically well behaved under physical dependence and row-wise approximate sparsity, allowing both the dimension of the system and the number of of relevant coefficients to grow with the sample size. The analysis exploits the distinctive fixed-point and thresholding structure of BAR to control the selected model and establish ℓ 1 \ell_1 ℓ 1 , ℓ 2 \ell_2 ℓ 2 , and prediction error rates in high-dimensional settings. Monte Carlo experiments show that BAR can achieve accurate variable selection, limited shrinkage of large coefficients, and competitive forecasting performance in high-dimensional and strongly correlated designs. In an application to U.S. inflation forecasting using FRED-MD, BAR performs particularly well at the one-month horizon, while its relative performance at longer horizons varies across economic periods. Overall, the results indicate that BAR provides a theoretically justified and practically competitive regularization method for high-dimensional time-series models.
Appendix
Additional Technical Details and Auxiliary Results
This appendix collects supporting derivations, computational details, and proofs omitted from the main text.
Additional Details on the Assumptions and VAR Setup
Further Discussion of the Physical-Dependence Assumption
Assumption 1 uses the physical-dependence framework of Wu [18 ] to quantify temporal dependence through the data-generating mechanism itself. Assumption 1(ii) represents h t \mathbf{h}_t h t as a measurable function of the innovation history F t = ( … , ϵ t − 1 , ϵ t ) \mathcal{F}^t = (\ldots, \boldsymbol{\epsilon}_{t-1}, \boldsymbol{\epsilon}_t) F t = ( … , ϵ t − 1 , ϵ t ) . The coupled history F t , 0 \mathcal{F}^{t,0} F t , 0 replaces only ϵ 0 \boldsymbol{\epsilon}_0 ϵ 0 by an independent copy, leaving every other innovation unchanged. Comparing h t = G ( F t ) \mathbf{h}_t = G(\mathcal{F}^t) h t = G ( F t ) with h t , 0 = G ( F t , 0 ) \mathbf{h}_{t,0} = G(\mathcal{F}^{t,0}) h t , 0 = G ( F t , 0 ) therefore isolates the effect of a single innovation on the process t t t periods later.
For an individual coordinate h j , t h_{j,t} h j , t , the functional-dependence measure δ q , j , t = ∥ h j , t − h j , t , 0 ∥ L q \delta_{q,j,t} = \lVert h_{j,t} - h_{j,t,0} \rVert_{L_q} δ q , j , t = ∥ h j , t − h j , t , 0 ∥ L q quantifies the magnitude of this effect at horizon t t t , while
Δ q , j , m = ∑ t = m ∞ δ q , j , t \Delta_{q,j,m} = \sum_{t=m}^{\infty} \delta_{q,j,t} Δ q , j , m = t = m ∑ ∞ δ q , j , t
measures its remaining cumulative effect from horizon m m m onward. The dependence-adjusted norm
∥ h j , ∙ ∥ q , α = sup m ≥ 0 ( m + 1 ) α Δ q , j , m \lVert h_{j,\bullet} \rVert_{q,\alpha} = \sup_{m \ge0} (m+1)^\alpha\Delta_{q,j,m} ∥ h j , ∙ ∥ q , α = m ≥ 0 sup ( m + 1 ) α Δ q , j , m
weights the cumulative effect remaining after horizon m m m by the factor ( m + 1 ) α (m+1)^\alpha ( m + 1 ) α . Requiring this norm to be bounded implies Δ q , j , m = O ( ( m + 1 ) − α ) \Delta_{q,j,m} = O((m+1)^{-\alpha}) Δ q , j , m = O (( m + 1 ) − α ) , so that the cumulative effect decays at least polynomially. The quantity Ψ q , α = max 1 ≤ j ≤ 2 K ∥ h j , ∙ ∥ q , α \Psi_{q,\alpha} = \max_{1 \le j \le2K} \lVert h_{j,\bullet} \rVert_{q,\alpha} Ψ q , α = max 1 ≤ j ≤ 2 K ∥ h j , ∙ ∥ q , α extends this control uniformly over the individual coordinates of the system. In particular, Ψ q , 0 < ∞ \Psi_{q,0} < \infty Ψ q , 0 < ∞ requires the effect of a single innovation to be summable over future horizons, whereas boundedness of Ψ q , α \Psi_{q,\alpha} Ψ q , α additionally controls the rate at which the remaining effect vanishes.
Coordinatewise control, however, does not by itself control the largest perturbation across the coordinates of a growing system. This motivates
ω q , t = ∥ h t − h t , 0 ∥ ∞ ∣ L q , Ω q , m = ∑ t = m ∞ ω q , t , \omega_{q,t} = \left\| h_t - h_{t,0} \right\|_{\infty\mid L_q}, \qquad\Omega_{q,m} = \sum_{t=m}^{\infty} \omega_{q,t}, ω q , t = ∥ h t − h t , 0 ∥ ∞ ∣ L q , Ω q , m = t = m ∑ ∞ ω q , t ,
and the associated maximal dependence-adjusted norm
H q , α , T = sup m ≥ 0 ( m + 1 ) α Ω q , m . \mathcal{H}_{q,\alpha,T} = \sup_{m \ge0} (m+1)^\alpha\Omega_{q,m}. H q , α , T = m ≥ 0 sup ( m + 1 ) α Ω q , m .
Thus, while Ψ q , α \Psi_{q,\alpha} Ψ q , α controls the effect of the innovation replacement one coordinate at a time, H q , α , T \mathcal{H}_{q,\alpha,T} H q , α , T controls the largest such perturbation over all coordinates. The distinction is purely cross-sectional: both quantities use the same moment order q q q and are based on the same single-innovation perturbation.
This additional maximal control is needed because the stochastic objects entering the BAR theory are themselves high dimensional. In fact, Lemma 2 simultaneously controls the empirical score and all entries of the sample Gram matrix. These quantities are products of coordinates of the underlying process and are collected in the proof into a single second-order process. The vector-valued maximal inequality of Zhang & Wu [19 ] , used through the second-order construction of Margaritella & Sessinou [13 ] , then requires control of the largest perturbation across the system. Condition (3) imposes the corresponding growth restriction.
The dependence-adjusted norms are high-level quantities, but they can be related to primitive restrictions for familiar time-series models. For example, Chernozhukov et al. [4 ] provides sufficient conditions for VARMA-type and multivariate-GARCH processes, including high-dimensional VARs.
The following simple example illustrates the innovation-replacement construction.
Example 1 (Physical dependence in an AR(1)). Consider the stationary AR(1)
y t = ϕ y t − 1 + e t , ∣ ϕ ∣ < 1 , y_t = \phi y_{t-1} + e_t, \qquad|\phi| < 1, y t = ϕ y t − 1 + e t , ∣ ϕ ∣ < 1 , where { e t } \{e_t\} { e t } is iid, mean zero, and E ∣ e t ∣ q < ∞ \mathbb{E}|e_t|^q < \infty E ∣ e t ∣ q < ∞ . Its causal representation is
y t = ∑ ℓ = 0 ∞ ϕ ℓ e t − ℓ . y_t = \sum_{\ell=0}^{\infty} \phi^\ell e_{t-\ell}. y t = ℓ = 0 ∑ ∞ ϕ ℓ e t − ℓ . Let e 0 ∗ e_0^* e 0 ∗ be an independent copy of e 0 e_0 e 0 and construct y t , 0 y_{t,0} y t , 0 by replacing only e 0 e_0 e 0 with e 0 ∗ e_0^* e 0 ∗ . For t ≥ 0 t \ge0 t ≥ 0 ,
y t − y t , 0 = ϕ t ( e 0 − e 0 ∗ ) , y_t-y_{t,0}=\phi^t(e_0-e_0^*), y t − y t , 0 = ϕ t ( e 0 − e 0 ∗ ) , and hence
δ q , t = ∣ ϕ t ∣ ∥ e 0 − e 0 ∗ ∥ L q , \delta_{q,t}=|\phi^t|\lVert e_0-e_0^*\rVert_{L_q}, δ q , t = ∣ ϕ t ∣ ∥ e 0 − e 0 ∗ ∥ L q , Δ q , m = ∣ ϕ ∣ m 1 − ∣ ϕ ∣ ∥ e 0 − e 0 ∗ ∥ L q , \Delta_{q,m}=\frac{|\phi|^m}{1-|\phi|}\lVert e_0-e_0^*\rVert_{L_q}, Δ q , m = 1 − ∣ ϕ ∣ ∣ ϕ ∣ m ∥ e 0 − e 0 ∗ ∥ L q , ∥ y ∙ ∥ q , α = sup m ≥ 0 ( m + 1 ) α Δ q , m < ∞ \lVert y_\bullet\rVert_{q,\alpha}=\sup_{m\ge0}(m+1)^\alpha\Delta_{q,m}<\infty ∥ y ∙ ∥ q , α = m ≥ 0 sup ( m + 1 ) α Δ q , m < ∞ for every α ≥ 0 \alpha\ge0 α ≥ 0 , since geometric decay dominates any polynomial factor.
In the univariate case the coordinatewise and maximal measures coincide. Their distinction appears in multivariate systems. For example, for K K K parallel AR(1) processes with the same autoregressive coefficient,
ω q , t = ∣ ϕ t ∣ ∥ max 1 ≤ j ≤ K ∣ e j , 0 − e j , 0 ∗ ∣ ∥ L q . \omega_{q,t}=|\phi^t|\left\lVert\max_{1\le j\le K}|e_{j,0}-e_{j,0}^*|\right\rVert_{L_q}. ω q , t = ∣ ϕ t ∣ 1 ≤ j ≤ K max ∣ e j , 0 − e j , 0 ∗ ∣ L q . The temporal decay remains geometric, while the magnitude of the largest contemporaneous perturbation may increase with K K K . The same interpretation applies to a stable VAR(p p p ), where the perturbation is propagated through the matrices A 1 , … , A p \mathbf{A}_1,\ldots,\mathbf{A}_p A 1 , … , A p .
Computational Details of the BAR Estimator
Selection of the Outer-Root Branch
In § 2 , the coordinatewise BAR update selects the outer solution of the quadratic fixed-point equation whenever the activation threshold is satisfied. We now show that this choice corresponds to the locally stable branch of the underlying scalar weighted-Ridge fixed-point map.
Fix the remaining coordinates and let b j b_j b j denote the corresponding partial score, with a j : = T − 1 X j ′ X j a_j := T^{-1}X_j^{\prime}X_j a j := T − 1 X j ′ X j . If z z z denotes the previous value of coordinate j j j , its weighted-Ridge update is
h j ( z ) = b j a j + λ / z 2 = b j z 2 a j z 2 + λ . h_j(z) = \frac{b_j}{a_j + \lambda/z^2} = \frac{b_j z^2}{a_j z^2 + \lambda}. h j ( z ) = a j + λ / z 2 b j = a j z 2 + λ b j z 2 .
A nonzero fixed point z = h j ( z ) z = h_j(z) z = h j ( z ) therefore satisfies
a j z 2 − b j z + λ = 0. a_j z^2 - b_j z + \lambda= 0. a j z 2 − b j z + λ = 0.
When ∣ b j ∣ > 2 λ a j \lvert b_j\rvert> 2\sqrt{\lambda a_j} ∣ b j ∣ > 2 λ a j , this equation has an inner and an outer root. At either nonzero fixed point,
h j ′ ( z ) = 2 b j z λ ( a j z 2 + λ ) 2 = 2 λ a j z 2 + λ , h_j^{\prime}(z) = \frac{2b_j z\lambda}{(a_j z^2+\lambda)^2} = \frac{2\lambda}{a_j z^2+\lambda}, h j ′ ( z ) = ( a j z 2 + λ ) 2 2 b j z λ = a j z 2 + λ 2 λ ,
where the second equality follows from b j z = a j z 2 + λ b_jz = a_jz^2 + \lambda b j z = a j z 2 + λ . For the outer root, a j z 2 > λ a_jz^2 > \lambda a j z 2 > λ , and hence ∣ h j ′ ( z ) ∣ < 1 \lvert h_j^{\prime}(z)\rvert< 1 ∣ h j ′ ( z )∣ < 1 , whereas for the inner root a j z 2 < λ a_jz^2 < \lambda a j z 2 < λ and ∣ h j ′ ( z ) ∣ > 1 \lvert h_j^{\prime}(z)\rvert> 1 ∣ h j ′ ( z )∣ > 1 . Thus, conditional on the remaining coordinates, the outer root is the locally stable branch of the scalar weighted-Ridge fixed-point map. Following Kawaguchi et al. [10 ] , Algorithm 1 uses this branch and sets the coordinate to zero when ∣ b j ∣ < 2 λ a j \lvert b_j\rvert< 2\sqrt{\lambda a_j} ∣ b j ∣ < 2 λ a j .
Additional Implementation Details
When K p ≫ T K_p \gg T K p ≫ T , the Ridge initializer need not be computed by inverting a ( K p ) × ( K p ) (Kp) \times(Kp) ( K p ) × ( K p ) matrix. Indeed,
β ^ ( 0 ) = ( T − 1 X ′ X + ξ I ) − 1 T − 1 X ′ y = X ′ ( X X ′ + T ξ I T ) − 1 y , \hat{\beta}^{(0)} = \left(T^{-1}X^{\prime}X + \xi I\right)^{-1}T^{-1}X^{\prime}y
= X^{\prime}\left(XX^{\prime} + T\xi I_T\right)^{-1}y, β ^ ( 0 ) = ( T − 1 X ′ X + ξ I ) − 1 T − 1 X ′ y = X ′ ( X X ′ + T ξ I T ) − 1 y ,
so the inversion can instead be performed in the T T T -dimensional sample space. Alternatively, the Ridge initializer may itself be computed by coordinate descent. For partial residual r j = y − X − j β ^ − j r_j = y - X_{-j}\hat{\beta}_{-j} r j = y − X − j β ^ − j , the corresponding update is
β ^ j = T − 1 X j ′ r j T − 1 X j ′ X j + ξ . \hat{\beta}_j = \frac{T^{-1}X_j^{\prime}r_j}{T^{-1}X_j^{\prime}X_j + \xi}. β ^ j = T − 1 X j ′ X j + ξ T − 1 X j ′ r j .
Both approaches avoid forming or inverting a large dense Gram matrix.
For the stopping rule in Algorithm 1, we use N max = 10 3 N_{\max} = 10^3 N m a x = 1 0 3 and ϵ = 10 − 4 \epsilon= 10^{-4} ϵ = 1 0 − 4 in the simulations and applications. Increasing N max N_{\max} N m a x or reducing ϵ \epsilon ϵ further produced no material improvement in coefficient accuracy in the implementations considered here, while increasing computational cost.
Grouping Property of the BAR Fixed Point
The following result provides the grouping property stated in Remark 3.
Proposition 3 (Grouping property). Suppose that the predictors are sample standardized so that ∥ X j ∥ 2 = T \lVert X_j\rVert_2 = T ∥ X j ∥ 2 = T , and define ρ ^ j k : = T − 1 X j ′ X k \hat{\rho}_{jk} := T^{-1}X_j^{\prime}X_k ρ ^ j k := T − 1 X j ′ X k . Let β ^ B A R \hat{\beta}_{BAR} β ^ B A R be a BAR fixed point and r ^ : = y − X β ^ B A R \hat{r} := y - X\hat{\beta}_{BAR} r ^ := y − X β ^ B A R . Then, for any j , k ∈ S ^ j,k \in\hat{S} j , k ∈ S ^ ,
∣ 1 β ^ B A R , j − 1 β ^ B A R , k ∣ ≤ ∥ r ^ ∥ 2 λ T 2 ( 1 − ρ ^ j k ) ≤ ∥ y ∥ 2 λ T 2 ( 1 − ρ ^ j k ) . (31) \left|\frac{1}{\hat{\beta}_{BAR,j}} - \frac{1}{\hat{\beta}_{BAR,k}}\right| \le\frac{\lVert\hat{r}\rVert_2}{\lambda\sqrt{T}}\sqrt{2(1-\hat{\rho}_{jk})} \le\frac{\lVert y\rVert_2}{\lambda\sqrt{T}}\sqrt{2(1-\hat{\rho}_{jk})}. \tag*{(31)} β ^ B A R , j 1 − β ^ B A R , k 1 ≤ λ T ∥ r ^ ∥ 2 2 ( 1 − ρ ^ j k ) ≤ λ T ∥ y ∥ 2 2 ( 1 − ρ ^ j k ) . (31)
Similarly,
∣ 1 β ^ B A R , j + 1 β ^ B A R , k ∣ ≤ ∥ r ^ ∥ 2 λ T 2 ( 1 + ρ ^ j k ) ≤ ∥ y ∥ 2 λ T 2 ( 1 + ρ ^ j k ) . (32) \left|\frac{1}{\widehat{\beta}_{BAR,j}}+\frac{1}{\widehat{\beta}_{BAR,k}}\right|
\le\frac{\lVert\hat{\mathbf r}\rVert_2}{\lambda\sqrt{T}}\sqrt{2(1+\hat{\rho}_{jk})}
\le\frac{\lVert\mathbf y\rVert_2}{\lambda\sqrt{T}}\sqrt{2(1+\hat{\rho}_{jk})}. \tag*{(32)} β B A R , j 1 + β B A R , k 1 ≤ λ T ∥ r ^ ∥ 2 2 ( 1 + ρ ^ j k ) ≤ λ T ∥ y ∥ 2 2 ( 1 + ρ ^ j k ) . (32)
Proof. For j , k ∈ S ^ j,k \in\widehat{S} j , k ∈ S , the fixed-point identity (7) gives
T − 1 X j ′ r ^ = λ β ^ B A R , j , T − 1 X k ′ r ^ = λ β ^ B A R , k . T^{-1}\mathbf X_j'\hat{\mathbf r}=\frac{\lambda}{\widehat{\beta}_{BAR,j}},\qquad
T^{-1}\mathbf X_k'\hat{\mathbf r}=\frac{\lambda}{\widehat{\beta}_{BAR,k}}. T − 1 X j ′ r ^ = β B A R , j λ , T − 1 X k ′ r ^ = β B A R , k λ .
Hence, by Cauchy–Schwarz,
λ ∣ 1 β ^ B A R , j − 1 β ^ B A R , k ∣ = T − 1 ∣ ( X j − X k ) ′ r ^ ∣ ≤ T − 1 ∥ X j − X k ∥ 2 ∥ r ^ ∥ 2 = ∥ r ^ ∥ 2 T 2 ( 1 − ρ ^ j k ) , \begin{aligned}
\lambda\left|\frac{1}{\widehat{\beta}_{BAR,j}}-\frac{1}{\widehat{\beta}_{BAR,k}}\right|
&=T^{-1}\left|(\mathbf X_j-\mathbf X_k)'\hat{\mathbf r}\right|\\
&\le T^{-1}\lVert\mathbf X_j-\mathbf X_k\rVert_2\lVert\hat{\mathbf r}\rVert_2\\
&=\frac{\lVert\hat{\mathbf r}\rVert_2}{\sqrt{T}}\sqrt{2(1-\hat{\rho}_{jk})},
\end{aligned} λ β B A R , j 1 − β B A R , k 1 = T − 1 ∣ ( X j − X k ) ′ r ^ ∣ ≤ T − 1 ∥ X j − X k ∥ 2 ∥ r ^ ∥ 2 = T ∥ r ^ ∥ 2 2 ( 1 − ρ ^ j k ) ,
where the last equality follows from ∥ X j ∥ 2 2 = ∥ X k ∥ 2 2 = T \lVert\mathbf X_j\rVert_2^2=\lVert\mathbf X_k\rVert_2^2=T ∥ X j ∥ 2 2 = ∥ X k ∥ 2 2 = T . To show ∥ r ^ ∥ 2 ≤ ∥ y ∥ 2 \lVert\hat{\mathbf r}\rVert_2\le\lVert\mathbf y\rVert_2 ∥ r ^ ∥ 2 ≤ ∥ y ∥ 2 , multiply the fixed-point identity by β ^ B A R , j \widehat{\beta}_{BAR,j} β B A R , j and sum over j ∈ S ^ j\in\widehat{S} j ∈ S . Since β ^ B A R , j = 0 \widehat{\beta}_{BAR,j}=0 β B A R , j = 0 outside S ^ \widehat{S} S ,
λ ∣ S ^ ∣ = T − 1 ( X β ^ B A R ) ′ ( y − X β ^ B A R ) . \lambda|\widehat{S}|=T^{-1}(\mathbf X\widehat{\boldsymbol{\beta}}_{BAR})'(\mathbf y-\mathbf X\widehat{\boldsymbol{\beta}}_{BAR}). λ ∣ S ∣ = T − 1 ( X β B A R ) ′ ( y − X β B A R ) .
Therefore,
T − 1 ∥ r ^ ∥ 2 2 = T − 1 ∥ y ∥ 2 2 − T − 1 ∥ X β ^ B A R ∥ 2 2 − 2 λ ∣ S ^ ∣ ≤ T − 1 ∥ y ∥ 2 2 . \begin{aligned}
T^{-1}\lVert\hat{\mathbf r}\rVert_2^2
&=T^{-1}\lVert\mathbf y\rVert_2^2-T^{-1}\lVert\mathbf X\widehat{\boldsymbol{\beta}}_{BAR}\rVert_2^2-2\lambda|\widehat{S}|\\
&\le T^{-1}\lVert\mathbf y\rVert_2^2.
\end{aligned} T − 1 ∥ r ^ ∥ 2 2 = T − 1 ∥ y ∥ 2 2 − T − 1 ∥ X β B A R ∥ 2 2 − 2 λ ∣ S ∣ ≤ T − 1 ∥ y ∥ 2 2 .
This proves (A.1). The proof of (A.2) is identical after replacing X j − X k \mathbf X_j-\mathbf X_k X j − X k by X j + X k \mathbf X_j+\mathbf X_k X j + X k , since
∥ X j + X k ∥ 2 2 = 2 T ( 1 + ρ ^ j k ) . \lVert\mathbf X_j+\mathbf X_k\rVert_2^2=2T(1+\hat{\rho}_{jk}). ∥ X j + X k ∥ 2 2 = 2 T ( 1 + ρ ^ j k ) .
Proofs
Throughout this section, Σ ^ x : = T − 1 X ′ X \widehat{\Sigma}_x := T^{-1}X'X Σ x := T − 1 X ′ X denotes the sample Gram matrix and Σ x : = E ( x t x t ′ ) \Sigma_x := \mathbb{E}(x_t x_t') Σ x := E ( x t x t ′ ) its population counterpart. For the fixed-point calculations we use the standardized representation described in Remark 2. By Lemma 2, max j ∣ T − 1 X j ′ X j − 1 ∣ = o p ( 1 ) \max_j |T^{-1}X_j'X_j - 1| = o_p(1) max j ∣ T − 1 X j ′ X j − 1∣ = o p ( 1 ) under population standardization, so this normalization does not affect the rates below.
Proof of Lemma 1. Suppose first that q s ∈ ( 0 , 1 ) q_s \in(0,1) q s ∈ ( 0 , 1 ) . We use the bound 1 { x > a } ≤ ( x / a ) q s \mathbb{1}\{x>a\} \le(x/a)^{q_s} 1 { x > a } ≤ ( x / a ) q s for x , a > 0 x,a>0 x , a > 0 , the fact that ∣ β j ∣ ≤ 2 λ |\beta_j| \le2\sqrt{\lambda} ∣ β j ∣ ≤ 2 λ for every j ∈ S λ c j \in S_\lambda^c j ∈ S λ c , and the approximate-sparsity condition ∑ j = 1 K p ∣ β j ∣ q s ≤ s 0 \sum_{j=1}^{Kp} |\beta_j|^{q_s} \le s_0 ∑ j = 1 K p ∣ β j ∣ q s ≤ s 0 from Assumption 2. Then,
∣ S λ ∣ = ∑ j = 1 K p 1 { ∣ β j ∣ > 2 λ } ≤ ( 2 λ ) − q s ∑ j = 1 K p ∣ β j ∣ q s ≤ 2 − q s s 0 λ − q s / 2 , |S_\lambda| = \sum_{j=1}^{Kp} \mathbb{1}\{|\beta_j|>2\sqrt{\lambda}\} \le(2\sqrt{\lambda})^{-q_s}\sum_{j=1}^{Kp}|\beta_j|^{q_s} \le2^{-q_s}s_0\lambda^{-q_s/2}, ∣ S λ ∣ = j = 1 ∑ K p 1 { ∣ β j ∣ > 2 λ } ≤ ( 2 λ ) − q s j = 1 ∑ K p ∣ β j ∣ q s ≤ 2 − q s s 0 λ − q s /2 ,
∥ β S λ c ∥ r r = ∑ j ∈ S λ c ∣ β j ∣ r − q s ∣ β j ∣ q s ≤ ( 2 λ ) r − q s ∑ j ∈ S λ c ∣ β j ∣ q s ≤ s 0 ( 2 λ ) r − q s , r ∈ { 1 , 2 } . \|\beta_{S_\lambda^c}\|_r^r = \sum_{j\in S_\lambda^c}|\beta_j|^{r-q_s}|\beta_j|^{q_s} \le(2\sqrt{\lambda})^{r-q_s}\sum_{j\in S_\lambda^c}|\beta_j|^{q_s} \le s_0(2\sqrt{\lambda})^{r-q_s}, \qquad r\in\{1,2\}. ∥ β S λ c ∥ r r = j ∈ S λ c ∑ ∣ β j ∣ r − q s ∣ β j ∣ q s ≤ ( 2 λ ) r − q s j ∈ S λ c ∑ ∣ β j ∣ q s ≤ s 0 ( 2 λ ) r − q s , r ∈ { 1 , 2 } .
Taking the r r r th root in the second inequality gives ∥ β S λ c ∥ r ≲ s 0 1 / r λ 1 / 2 − q s / ( 2 r ) \|\beta_{S_\lambda^c}\|_r \lesssim s_0^{1/r}\lambda^{1/2-q_s/(2r)} ∥ β S λ c ∥ r ≲ s 0 1/ r λ 1/2 − q s / ( 2 r ) .
When q s = 0 q_s=0 q s = 0 , Assumption 2 is exact sparsity and ∥ β ∥ 0 ≤ s 0 \|\beta\|_0\le s_0 ∥ β ∥ 0 ≤ s 0 . Hence,
∣ S λ ∣ ≤ s 0 , ∥ β S λ c ∥ r r ≤ s 0 ( 2 λ ) r , ∥ β S λ c ∥ r ≲ s 0 1 / r λ , |S_\lambda|\le s_0,\qquad\|\beta_{S_\lambda^c}\|_r^r\le s_0(2\sqrt{\lambda})^r,\qquad\|\beta_{S_\lambda^c}\|_r\lesssim s_0^{1/r}\sqrt{\lambda}, ∣ S λ ∣ ≤ s 0 , ∥ β S λ c ∥ r r ≤ s 0 ( 2 λ ) r , ∥ β S λ c ∥ r ≲ s 0 1/ r λ ,
which yields the same bounds at $q_s=0.
Proof of Lemma 2. Let d : = K p d:=Kp d := K p , r : = q / 2 > 2 r:=q/2>2 r := q /2 > 2 , ℓ T : = 2 log ( d + 1 ) \ell_T:=2\log(d+1) ℓ T := 2 log ( d + 1 ) , and define
g t : = ( x t ′ , u t ′ ) ′ ∈ R d + 1 , χ t : = vec { g t g t ′ − E ( g t g t ′ ) } , χ ˉ T : = T − 1 ∑ t = 1 T χ t . g_t := (x_t',u_t')' \in\mathbb{R}^{d+1},\qquad\chi_t := \operatorname{vec}\{g_tg_t' - \mathbb{E}(g_tg_t')\},\qquad\bar{\chi}_T := T^{-1}\sum_{t=1}^{T}\chi_t. g t := ( x t ′ , u t ′ ) ′ ∈ R d + 1 , χ t := vec { g t g t ′ − E ( g t g t ′ )} , χ ˉ T := T − 1 t = 1 ∑ T χ t .
Let g t , 0 g_{t,0} g t , 0 and χ t , 0 \chi_{t,0} χ t , 0 denote the corresponding coupled processes. We further define
Ψ q , α ( g ) : = max 1 ≤ j ≤ d + 1 ∥ g j , ∙ ∥ q , α , G q , α , T : = sup m ≥ 0 ( m + 1 ) α ∑ t = m ∞ ∥ g t − g t , 0 ∥ ∞ ∥ L q . \Psi_{q,\alpha}^{(g)} := \max_{1\le j\le d+1}\|g_{j,\bullet}\|_{q,\alpha},\qquad\mathcal{G}_{q,\alpha,T} := \sup_{m\ge0}(m+1)^\alpha\sum_{t=m}^{\infty}\|g_t-g_{t,0}\|_\infty\big\|_{L_q}. Ψ q , α ( g ) := 1 ≤ j ≤ d + 1 max ∥ g j , ∙ ∥ q , α , G q , α , T := m ≥ 0 sup ( m + 1 ) α t = m ∑ ∞ ∥ g t − g t , 0 ∥ ∞ L q .
Since p p p is fixed and the components of g t g_t g t are either coordinates of u t u_t u t or fixed lags of coordinates of y t \mathbf{y}_t y t , Assumption 1 implies
Ψ q , 0 ( g ) ≲ Ψ q , 0 , Ψ q , α ( g ) ≲ Ψ q , α , G q , 0 , T ≲ H q , 0 , T , G q , α , T ≲ H q , α , T . \Psi^{(g)}_{q,0} \lesssim\Psi_{q,0}, \qquad\Psi^{(g)}_{q,\alpha} \lesssim\Psi_{q,\alpha}, \qquad G_{q,0,T} \lesssim H_{q,0,T}, \qquad G_{q,\alpha,T} \lesssim H_{q,\alpha,T}. Ψ q , 0 ( g ) ≲ Ψ q , 0 , Ψ q , α ( g ) ≲ Ψ q , α , G q , 0 , T ≲ H q , 0 , T , G q , α , T ≲ H q , α , T .
Furthermore, a generic coordinate of χ t \boldsymbol{\chi}_t χ t , indexed by a = ( j , k ) a=(j,k) a = ( j , k ) , satisfies, by Minkowski’s and Hölder’s inequalities,
∥ χ a , t − χ a , t , 0 ∥ L r ≤ ∥ ( g j , t − g j , t , 0 ) g k , t ∥ L r + ∥ g j , t , 0 ( g k , t − g k , t , 0 ) ∥ L r ≤ ∥ g j , t − g j , t , 0 ∥ L q ∥ g k , t ∥ L q + ∥ g j , t , 0 ∥ L q ∥ g k , t − g k , t , 0 ∥ L q . \begin{aligned}
\|\chi_{a,t}-\chi_{a,t,0}\|_{L^r} \leq\|(g_{j,t}-g_{j,t,0})g_{k,t}\|_{L^r}+\|g_{j,t,0}(g_{k,t}-g_{k,t,0})\|_{L^r} \\
&\leq\|g_{j,t}-g_{j,t,0}\|_{L^q}\|g_{k,t}\|_{L^q}+\|g_{j,t,0}\|_{L^q}\|g_{k,t}-g_{k,t,0}\|_{L^q}.
\end{aligned} ∥ χ a , t − χ a , t , 0 ∥ L r ≤ ∥ ( g j , t − g j , t , 0 ) g k , t ∥ L r + ∥ g j , t , 0 ( g k , t − g k , t , 0 ) ∥ L r ≤ ∥ g j , t − g j , t , 0 ∥ L q ∥ g k , t ∥ L q + ∥ g j , t , 0 ∥ L q ∥ g k , t − g k , t , 0 ∥ L q .
The standard product inequality for dependence-adjusted norms (Zhang & Wu [19 ] , Section 3.1) therefore gives
A 2 , T : = max 1 ≤ a ≤ ( d + 1 ) 2 ∥ χ a , ∙ ∥ 2 , α ≲ Ψ q , 0 ( g ) Ψ q , α ( g ) ≲ Ψ q , 0 Ψ q , α , A_{2,T}:=\max_{1\leq a\leq(d+1)^2}\|\chi_{a,\bullet}\|_{2,\alpha}\lesssim\Psi^{(g)}_{q,0}\Psi^{(g)}_{q,\alpha}\lesssim\Psi_{q,0}\Psi_{q,\alpha}, A 2 , T := 1 ≤ a ≤ ( d + 1 ) 2 max ∥ χ a , ∙ ∥ 2 , α ≲ Ψ q , 0 ( g ) Ψ q , α ( g ) ≲ Ψ q , 0 Ψ q , α ,
B r , T : = sup m ≥ 0 ( m + 1 ) α ∑ t = m ∞ ∥ χ t − χ t , 0 ∥ ∞ , L r ≲ G q , 0 , T G q , α , T ≲ H q , 0 , T H q , α , T . (33) B_{r,T}:=\sup_{m\geq0}(m+1)^\alpha\sum_{t=m}^{\infty}\|\chi_t-\chi_{t,0}\|_{\infty,L^r}\lesssim G_{q,0,T}G_{q,\alpha,T}\lesssim H_{q,0,T}H_{q,\alpha,T}. \tag*{(33)} B r , T := m ≥ 0 sup ( m + 1 ) α t = m ∑ ∞ ∥ χ t − χ t , 0 ∥ ∞ , L r ≲ G q , 0 , T G q , α , T ≲ H q , 0 , T H q , α , T . (33)
By (2) , A 2 , T = O ( 1 ) A_{2,T}=O(1) A 2 , T = O ( 1 ) . Indeed, since q > 4 q>4 q > 4 , the L 4 L^4 L 4 dependence norms in the first line are controlled by their L q L^q L q counterparts. Moreover, (3) implies
B r , T ℓ T = o ( T 1 / 2 − α 1 / r ) . (34) B_{r,T}\ell_T=o\left(T^{1/2-\alpha_1/r}\right). \tag*{(34)} B r , T ℓ T = o ( T 1/2 − α 1 / r ) . (34)
Following the centered second-order construction in Margaritella & Sessinou [13 ] , we apply the Fuk-Nagaev bound of Theorem 6.2 of Zhang & Wu [19 ] with
κ T , d : = C 0 { T − 1 / 2 ℓ T 1 / 2 A 2 , T + T α 1 / r − 1 ℓ T 3 / 2 B r , T } . (35) \kappa_{T,d}:=C_0\left\{T^{-1/2}\ell_T^{1/2}A_{2,T}+T^{\alpha_1/r-1}\ell_T^{3/2}B_{r,T}\right\}. \tag*{(35)} κ T , d := C 0 { T − 1/2 ℓ T 1/2 A 2 , T + T α 1 / r − 1 ℓ T 3/2 B r , T } . (35)
For a sufficiently large fixed C 0 C_0 C 0 and for positive constants, denoted generically by C r , α C_{r,\alpha} C r , α , depending only on ( r , α ) (r,\alpha) ( r , α ) ,
P ( ∥ χ ‾ T ∥ ∞ > κ T , d ) ≤ C r , α T α 1 − r ℓ T r / 2 B r , T κ T , d r + C r , α exp { − C r , α T κ T , d 2 A 2 , T 2 } . (36) \mathbb{P}\left(\|\overline{\boldsymbol{\chi}}_T\|_{\infty}>\kappa_{T,d}\right)\leq C_{r,\alpha}\frac{T^{\alpha_1-r}\ell_T^{r/2}B_{r,T}}{\kappa_{T,d}^{r}}+C_{r,\alpha}\exp\left\{-\frac{C_{r,\alpha}T\kappa_{T,d}^{2}}{A_{2,T}^{2}}\right\}. \tag*{(36)} P ( ∥ χ T ∥ ∞ > κ T , d ) ≤ C r , α κ T , d r T α 1 − r ℓ T r /2 B r , T + C r , α exp { − A 2 , T 2 C r , α T κ T , d 2 } . (36)
By (B.4) and A 2 , T = O ( 1 ) A_{2,T}=O(1) A 2 , T = O ( 1 ) ,
\begin{aligned}
T^{\alpha_1/r-1}\ell_T^{3/2}B_{r,T} &= T^{-1/2}\ell_T^{1/2}\left\{T^{\alpha_1/r-1/2}\ell_TB_{r,T}\right\}=o\left(T^{-1/2}\ell_T^{1/2}\right),\\
\kappa_{T,d} &= O\left(\sqrt{\frac{\log(d)}{T}}\right),\\
\frac{T^{\alpha_1-r\rho_T^{r/2}B_{r,T}^r}{\kappa_{T,d}^r} &\le\ell_T^{-r}=o(1),\qquad\frac{T\kappa_{T,d}^2}{A_{2,T}^2}\gtrsim\ell_T\to\infty.
\end{aligned}}
Both terms on the right-hand side of (B.6) therefore vanish. Hence, for a sufficiently large fixed C χ C_\chi C χ ,
P { ∥ χ ˉ T ∥ ∞ ≤ C χ log ( d ) T } → 1. \mathbb{P}\left\{\|\bar{\chi}_T\|_\infty\le C_\chi\sqrt{\frac{\log(d)}{T}}\right\}\to1. P { ∥ χ ˉ T ∥ ∞ ≤ C χ T log ( d ) } → 1.
Finally, by construction, the coordinates of T − 1 X ′ u T^{-1}X^{\prime}u T − 1 X ′ u and vec ( Σ ^ x − Σ x ) \operatorname{vec}(\widehat{\Sigma}_x-\Sigma_x) vec ( Σ x − Σ x ) are contained in χ ˉ T \bar{\chi}_T χ ˉ T . In particular, E ( x t u t ) = 0 \mathbb{E}(x_tu_t)=0 E ( x t u t ) = 0 identifies the score block, while the x t x t ′ x_tx_t^{\prime} x t x t ′ block yields the centered sample Gram matrix. Therefore, with d = K p d=Kp d = K p ,
∥ T − 1 X ′ u ∥ ∞ ∨ ∥ Σ ^ x − Σ x ∥ max = O p ( log ( K p ) T ) , \left\|T^{-1}X^{\prime}u\right\|_\infty\vee\left\|\widehat{\Sigma}_x-\Sigma_x\right\|_{\max}
=O_p\left(\sqrt{\frac{\log(Kp)}{T}}\right), T − 1 X ′ u ∞ ∨ Σ x − Σ x m a x = O p ( T log ( K p ) ) ,
which proves Lemma 2. □
Proof of Lemma 3. For every j ∈ U c j\in U^c j ∈ U c , j ∉ S ^ j\notin\widehat{S} j ∈ / S and therefore β ^ B A R , j = 0 \widehat{\beta}_{\mathrm{BAR},j}=0 β BAR , j = 0 . Hence,
Δ U c = − β U c , Δ = Δ U − β U c , ∥ Δ U c ∥ r = ∥ β U c ∥ r ≤ ∥ β S λ c ∥ r , r ∈ { 1 , 2 } . \Delta_{U^c}=-\beta_{U^c},\qquad\Delta=\Delta_U-\beta_{U^c},\qquad\|\Delta_{U^c}\|_r=\|\beta_{U^c}\|_r\le\|\beta_{S_\lambda^c}\|_r,\qquad r\in\{1,2\}. Δ U c = − β U c , Δ = Δ U − β U c , ∥ Δ U c ∥ r = ∥ β U c ∥ r ≤ ∥ β S λ c ∥ r , r ∈ { 1 , 2 } .
The final bound follows by the triangle inequality and Corollary 1. □
Proof of Proposition 1. For every j ∈ S ^ j\in\widehat{S} j ∈ S , (7) gives v j = λ / β ^ B A R , j v_j=\lambda/\widehat{\beta}_{\mathrm{BAR},j} v j = λ / β BAR , j . Therefore,
λ ∣ S ^ ∣ = ∑ j ∈ S ^ β ^ B A R , j v j = T − 1 ( X β ^ B A R ) ′ ( y − X β ^ B A R ) = 1 4 T ∥ y ∥ 2 2 − 1 T ∥ X β ^ B A R − y 2 ∥ 2 2 ≤ 1 4 T ∥ y ∥ 2 2 , (37) \begin{aligned}
\lambda|\widehat{S}|=\sum_{j\in\widehat{S}}\widehat{\beta}_{\mathrm{BAR},j}v_j
=T^{-1}(X\widehat{\beta}_{\mathrm{BAR}})^{\prime}(y-X\widehat{\beta}_{\mathrm{BAR}}) \\
&=\frac{1}{4T}\|y\|_2^2-\frac{1}{T}\left\|X\widehat{\beta}_{\mathrm{BAR}}-\frac{y}{2}\right\|_2^2
\le\frac{1}{4T}\|y\|_2^2, \tag*{(37)}
\end{aligned} λ ∣ S ∣ = j ∈ S ∑ β BAR , j v j = T − 1 ( X β BAR ) ′ ( y − X β BAR ) = 4 T 1 ∥ y ∥ 2 2 − T 1 X β BAR − 2 y 2 2 ≤ 4 T 1 ∥ y ∥ 2 2 , (37)
where we used β ^ B A R , j = 0 \widehat{\beta}_{BAR,j}=0 β B A R , j = 0 for j ∉ S ^ j\notin\widehat{S} j ∈ / S . Consequently,
∣ S ^ ∣ ≤ ∥ y ∥ 2 2 4 λ T , |\widehat{S}| \le\frac{\|y\|_{2}^{2}}{4\lambda T}, ∣ S ∣ ≤ 4 λ T ∥ y ∥ 2 2 ,
∣ U ∣ ≤ ∣ S λ ∣ + ∣ S ^ ∣ ≤ 2 − q s s 0 λ − q s / 2 + ∥ y ∥ 2 2 4 λ T ≤ m ‾ T on E y , |U| \le|S_{\lambda}|+|\widehat{S}| \le2^{-q_s}s_0\lambda^{-q_s/2}+\frac{\|y\|_{2}^{2}}{4\lambda T}\le\overline{m}_T \quad\text{on }\mathcal{E}_y, ∣ U ∣ ≤ ∣ S λ ∣ + ∣ S ∣ ≤ 2 − q s s 0 λ − q s /2 + 4 λ T ∥ y ∥ 2 2 ≤ m T on E y ,
where the second line uses Lemma 1 and the definition of E y \mathcal{E}_y E y .
Proof of Lemma 4 . Using y = X β + u y=X\beta+u y = X β + u and Δ = β ^ B A R − β \Delta=\widehat{\beta}_{BAR}-\beta Δ = β B A R − β ,
v = T − 1 X ′ ( y − X β ^ B A R ) = T − 1 X ′ u − Σ ^ x Δ , v=T^{-1}X^{\prime}(y-X\widehat{\beta}_{BAR})=T^{-1}X^{\prime}u-\widehat{\Sigma}_x\Delta, v = T − 1 X ′ ( y − X β B A R ) = T − 1 X ′ u − Σ x Δ ,
v U = T − 1 X U ′ u − Σ ^ x , U U Δ U − Σ ^ x , U U c Δ U c = T − 1 X U ′ u − Σ ^ x , U U Δ U + Σ ^ x , U U c β U c , \begin{aligned}
v_U&=T^{-1}X_U^{\prime}u-\widehat{\Sigma}_{x,UU}\Delta_U-\widehat{\Sigma}_{x,UU^c}\Delta_{U^c}\\
&=T^{-1}X_U^{\prime}u-\widehat{\Sigma}_{x,UU}\Delta_U+\widehat{\Sigma}_{x,UU^c}\beta_{U^c},
\end{aligned} v U = T − 1 X U ′ u − Σ x , U U Δ U − Σ x , U U c Δ U c = T − 1 X U ′ u − Σ x , U U Δ U + Σ x , U U c β U c ,
where the last equality follows from Lemma 3. Premultiplication by Δ U ′ \Delta_U^{\prime} Δ U ′ and rearrangement give (19) .
Proof of Lemma 5 . For q s ∈ ( 0 , 1 ) q_s\in(0,1) q s ∈ ( 0 , 1 ) ,
∣ F B ∣ ≤ ∣ B λ ∣ ≤ ∑ j = 1 K p 1 { ∣ β j ∣ > c 0 λ } ≤ ( c 0 λ ) − q s ∑ j = 1 K p ∣ β j ∣ q s ≤ c 0 − q s s 0 λ − q s / 2 ≲ s 0 λ − q s / 2 . \begin{aligned}
|F_B|\le|B_{\lambda}|\le\sum_{j=1}^{Kp}\mathbb{1}\{|\beta_j|>c_0\sqrt{\lambda}\} \\
&\le(c_0\sqrt{\lambda})^{-q_s}\sum_{j=1}^{Kp}|\beta_j|^{q_s}\le c_0^{-q_s}s_0\lambda^{-q_s/2}\lesssim s_0\lambda^{-q_s/2}.
\end{aligned} ∣ F B ∣ ≤ ∣ B λ ∣ ≤ j = 1 ∑ K p 1 { ∣ β j ∣ > c 0 λ } ≤ ( c 0 λ ) − q s j = 1 ∑ K p ∣ β j ∣ q s ≤ c 0 − q s s 0 λ − q s /2 ≲ s 0 λ − q s /2 .
When q s = 0 q_s=0 q s = 0 , every coordinate in B λ B_{\lambda} B λ is nonzero and exact sparsity gives ∣ F B ∣ ≤ ∣ B λ ∣ ≤ ∥ β ∥ 0 ≤ s 0 |F_B|\le|B_{\lambda}|\le\|\beta\|_0\le s_0 ∣ F B ∣ ≤ ∣ B λ ∣ ≤ ∥ β ∥ 0 ≤ s 0 , which is the same bound at q s = 0 q_s=0 q s = 0 .
Proof of Lemma 6 . For every j ∈ F D j\in F_D j ∈ F D , j ∈ S ^ j\in\widehat{S} j ∈ S and therefore, by (7),
Δ j v j = ( β ^ B A R , j − β j ) λ β ^ B A R , j = λ ( 1 − β j β ^ B A R , j ) ≥ λ ( 1 − ∣ β j β ^ B A R , j ∣ ) ≥ ( 1 − c 0 ) λ , \begin{aligned}
\Delta_jv_j=(\widehat{\beta}_{BAR,j}-\beta_j)\frac{\lambda}{\widehat{\beta}_{BAR,j}}=\lambda\left(1-\frac{\beta_j}{\widehat{\beta}_{BAR,j}}\right) \\
&\ge\lambda\left(1-\left|\frac{\beta_j}{\widehat{\beta}_{BAR,j}}\right|\right)\ge(1-c_0)\lambda,
\end{aligned} Δ j v j = ( β B A R , j − β j ) β B A R , j λ = λ ( 1 − β B A R , j β j ) ≥ λ ( 1 − β B A R , j β j ) ≥ ( 1 − c 0 ) λ ,
where the last inequality uses ∣ β j ∣ ≤ c 0 λ |\beta_j|\le c_0\sqrt{\lambda} ∣ β j ∣ ≤ c 0 λ from equation 20 in the main text and $|\widehat{\beta}_{BAR,j}|\ge λ \sqrt{\lambda} λ . Summing over F D F_D F D yields
∑ j ∈ F D Δ j v j ≥ ( 1 − c 0 ) λ ∣ F D ∣ . \sum_{j\in F_D}\Delta_jv_j \ge(1-c_0)\lambda|F_D|. j ∈ F D ∑ Δ j v j ≥ ( 1 − c 0 ) λ ∣ F D ∣.
Proof of Proposition 2. Let
J : = U ∖ F D = S λ ∪ F B , ∣ J ∣ = ∣ S λ ∣ + ∣ F B ∣ . J := U\setminus F_D = S_\lambda\cup F_B,\qquad|J|=|S_\lambda|+|F_B|. J := U ∖ F D = S λ ∪ F B , ∣ J ∣ = ∣ S λ ∣ + ∣ F B ∣.
By Lemmas 4 and 6,
Δ U ′ Σ ^ x , U U Δ U + ( 1 − c 0 ) λ ∣ F D ∣ ≤ ∣ Δ U ′ T − 1 X U ′ u ∣ + ∣ Δ U ′ Σ ^ x , U U c β U c ∣ + ∣ ∑ j ∈ J Δ j v j ∣ . (38) \Delta_U'\widehat{\Sigma}_{x,UU}\Delta_U+(1-c_0)\lambda|F_D|\le\left|\Delta_U'T^{-1}X_U'u\right|+\left|\Delta_U'\widehat{\Sigma}_{x,UU^c}\beta_{U^c}\right|+\left|\sum_{j\in J}\Delta_jv_j\right|. \tag*{(38)} Δ U ′ Σ x , U U Δ U + ( 1 − c 0 ) λ ∣ F D ∣ ≤ Δ U ′ T − 1 X U ′ u + Δ U ′ Σ x , U U c β U c + j ∈ J ∑ Δ j v j . (38)
On E λ \mathcal{E}_\lambda E λ , Lemma 2 gives ∥ T − 1 X U ′ u ∥ ∞ ≤ c u λ \|T^{-1}X_U'u\|_\infty\le c_u\sqrt{\lambda} ∥ T − 1 X U ′ u ∥ ∞ ≤ c u λ , where c u : = C χ / C A c_u:=C_\chi/\sqrt{C_A} c u := C χ / C A . Moreover, ∣ v j ∣ ≤ 2 λ |v_j|\le2\sqrt{\lambda} ∣ v j ∣ ≤ 2 λ for all j ∈ J j\in J j ∈ J : if j ∈ J ∩ S ^ j\in J\cap\widehat{S} j ∈ J ∩ S , this follows from v j = λ / β ^ B A R , j v_j=\lambda/\widehat{\beta}_{BAR,j} v j = λ / β B A R , j and ∣ β ^ B A R , j ∣ ≥ λ |\widehat{\beta}_{BAR,j}|\ge\sqrt{\lambda} ∣ β B A R , j ∣ ≥ λ ; if j ∈ J ∩ S ^ c j\in J\cap\widehat{S}^c j ∈ J ∩ S c , then j ∈ S λ ∩ S ^ c j\in S_\lambda\cap\widehat{S}^c j ∈ S λ ∩ S c and inactivity implies ∣ v j ∣ = ∣ b j ∣ < 2 λ |v_j|=|b_j|<2\sqrt{\lambda} ∣ v j ∣ = ∣ b j ∣ < 2 λ . Hence,
∣ Δ U ′ T − 1 X U ′ u ∣ + ∣ ∑ j ∈ J Δ j v j ∣ ≤ λ [ ( c u + 2 ) ∥ Δ J ∥ 1 + c u ∥ Δ F D ∥ 1 ] ≤ ( c u + 2 ) 2 ∣ J ∣ λ + c u 2 ∣ F D ∣ λ ∥ Δ U ∥ 2 . \left|\Delta_U'T^{-1}X_U'u\right|+\left|\sum_{j\in J}\Delta_jv_j\right|
\le\sqrt{\lambda}\left[(c_u+2)\|\Delta_J\|_1+c_u\|\Delta_{F_D}\|_1\right]
\le\sqrt{(c_u+2)^2|J|\lambda+c_u^2|F_D|\lambda}\|\Delta_U\|_2. Δ U ′ T − 1 X U ′ u + j ∈ J ∑ Δ j v j ≤ λ [ ( c u + 2 ) ∥ Δ J ∥ 1 + c u ∥ Δ F D ∥ 1 ] ≤ ( c u + 2 ) 2 ∣ J ∣ λ + c u 2 ∣ F D ∣ λ ∥ Δ U ∥ 2 .
The last step follows from Cauchy–Schwarz. For the remaining cross-block term,
∣ Δ U ′ Σ ^ x , U U c β U c ∣ = ∣ T − 1 ( X U Δ U ) ′ ( X U c β U c ) ∣ ≤ ( Δ U ′ Σ ^ x , U U Δ U ) 1 / 2 ( β U c ′ Σ ^ x , U c U c β U c ) 1 / 2 ≤ 1 2 Δ U ′ Σ ^ x , U U Δ U + 1 2 β U c ′ Σ ^ x , U c U c β U c . (39) \left|\Delta_U'\widehat{\Sigma}_{x,UU^c}\beta_{U^c}\right|
=\left|T^{-1}(X_U\Delta_U)'(X_{U^c}\beta_{U^c})\right|
\le\left(\Delta_U'\widehat{\Sigma}_{x,UU}\Delta_U\right)^{1/2}\left(\beta_{U^c}'\widehat{\Sigma}_{x,U^cU^c}\beta_{U^c}\right)^{1/2}
\le\frac{1}{2}\Delta_U'\widehat{\Sigma}_{x,UU}\Delta_U+\frac{1}{2}\beta_{U^c}'\widehat{\Sigma}_{x,U^cU^c}\beta_{U^c}. \tag*{(39)} Δ U ′ Σ x , U U c β U c = T − 1 ( X U Δ U ) ′ ( X U c β U c ) ≤ ( Δ U ′ Σ x , U U Δ U ) 1/2 ( β U c ′ Σ x , U c U c β U c ) 1/2 ≤ 2 1 Δ U ′ Σ x , U U Δ U + 2 1 β U c ′ Σ x , U c U c β U c . (39)
Substituting the preceding bounds into (B.7), using Δ U ′ Σ ^ x , U U Δ U ≥ κ c r u d e ∥ Δ U ∥ 2 2 \Delta_U'\widehat{\Sigma}_{x,UU}\Delta_U\ge\kappa_{\mathrm{crude}}\|\Delta_U\|_2^2 Δ U ′ Σ x , U U Δ U ≥ κ crude ∥ Δ U ∥ 2 2 on G \mathcal{G} G , and applying Young’s inequality gives
κ c r u d e 2 ∥ Δ U ∥ 2 2 + ( 1 − c 0 ) λ ∣ F D ∣ ≤ ( c u + 2 ) 2 ∣ J ∣ λ + c u 2 ∣ F D ∣ λ ∥ Δ U ∥ 2 + 1 2 β U c ′ Σ ^ x , U c U c β U c ≤ κ c r u d e 4 ∥ Δ U ∥ 2 2 + ( c u + 2 ) 2 κ c r u d e ∣ J ∣ λ + c u 2 κ c r u d e ∣ F D ∣ λ + 1 2 β U c ′ Σ ^ x , U c U c β U c . \begin{aligned}
\frac{\kappa_{\mathrm{crude}}}{2}\|\Delta_U\|_2^2+(1-c_0)\lambda|F_D|\le\sqrt{(c_u+2)^2|J|\lambda+c_u^2|F_D|\lambda}\|\Delta_U\|_2+\frac{1}{2}\beta_{U^c}'\widehat{\Sigma}_{x,U^cU^c}\beta_{U^c} \\
&\le\frac{\kappa_{\mathrm{crude}}}{4}\lVert\Delta U\rVert_2^2+\frac{(c_u+2)^2}{\kappa_{\mathrm{crude}}}|J|\lambda+\frac{c_u^2}{\kappa_{\mathrm{crude}}}|F_D|\lambda+\frac{1}{2}\beta_{U^c}^{\prime}\widehat{\Sigma}_{x,U^cU^c}\beta_{U^c}.
\end{aligned} 2 κ crude ∥ Δ U ∥ 2 2 + ( 1 − c 0 ) λ ∣ F D ∣ ≤ ( c u + 2 ) 2 ∣ J ∣ λ + c u 2 ∣ F D ∣ λ ∥ Δ U ∥ 2 + 2 1 β U c ′ Σ x , U c U c β U c ≤ 4 κ crude ∥ Δ U ∥ 2 2 + κ crude ( c u + 2 ) 2 ∣ J ∣ λ + κ crude c u 2 ∣ F D ∣ λ + 2 1 β U c ′ Σ x , U c U c β U c .
Therefore, with a u : = c u 2 / κ c r u d e a_u := c_u^2/\kappa_{\mathrm{crude}} a u := c u 2 / κ crude ,
( 1 − c 0 − a u ) λ ∣ F D ∣ ≤ ( c u + 2 ) 2 κ c r u d e λ ( ∣ S λ ∣ + ∣ F B ∣ ) + 1 2 β U c ′ Σ ^ x , U c U c β U c . (40) (1-c_0-a_u)\lambda|F_D|\le\frac{(c_u+2)^2}{\kappa_{\mathrm{crude}}}\lambda\left(|S_\lambda|+|F_B|\right)+\frac{1}{2}\beta_{U^c}^{\prime}\widehat{\Sigma}_{x,U^cU^c}\beta_{U^c}. \tag*{(40)} ( 1 − c 0 − a u ) λ ∣ F D ∣ ≤ κ crude ( c u + 2 ) 2 λ ( ∣ S λ ∣ + ∣ F B ∣ ) + 2 1 β U c ′ Σ x , U c U c β U c . (40)
To control the last term, let δ T : = ∥ Σ ^ x − Σ x ∥ max \delta_T := \lVert\widehat{\Sigma}_x-\Sigma_x\rVert_{\max} δ T := ∥ Σ x − Σ x ∥ m a x . Since U c ⊆ S λ c U^c\subseteq S^c_\lambda U c ⊆ S λ c ,
β U c ′ Σ ^ x , U c U c β U c ≤ β U c ′ Σ x , U c U c β U c + δ T ∥ β U c ∥ 1 2 ≤ C max ∥ β S λ c ∥ 2 2 + C χ log ( K p ) T ∥ β S λ c ∥ 1 2 \begin{aligned}
\beta_{U^c}^{\prime}\widehat{\Sigma}_{x,U^cU^c}\beta_{U^c}\le\beta_{U^c}^{\prime}\Sigma_{x,U^cU^c}\beta_{U^c}+\delta_T\lVert\beta_{U^c}\rVert_1^2 \\
&\le C_{\max}\lVert\beta_{S^c_\lambda}\rVert_2^2+C_{\chi}\sqrt{\frac{\log(Kp)}{T}}\lVert\beta_{S^c_\lambda}\rVert_1^2
\end{aligned} β U c ′ Σ x , U c U c β U c ≤ β U c ′ Σ x , U c U c β U c + δ T ∥ β U c ∥ 1 2 ≤ C m a x ∥ β S λ c ∥ 2 2 + C χ T log ( K p ) ∥ β S λ c ∥ 1 2
≲ ∥ β S λ c ∥ 2 2 + λ ∥ β S λ c ∥ 1 2 , (41) \lesssim\lVert\beta_{S^c_\lambda}\rVert_2^2+\sqrt{\lambda}\lVert\beta_{S^c_\lambda}\rVert_1^2, \tag*{(41)} ≲ ∥ β S λ c ∥ 2 2 + λ ∥ β S λ c ∥ 1 2 , (41)
where the second line uses λ max ( Σ x ) ≤ C max \lambda_{\max}(\Sigma_x)\le C_{\max} λ m a x ( Σ x ) ≤ C m a x and the Gram-matrix deviation bound in Lemma 2 on E χ \mathcal{E}_{\chi} E χ , while the last line uses λ = C λ log ( K p ) / T \lambda=C_{\lambda}\log(Kp)/T λ = C λ log ( K p ) / T . Since 1 − c 0 − a u > 0 1-c_0-a_u>0 1 − c 0 − a u > 0 , (B.9) therefore yields
∣ F D ∣ ≤ C 1 ( ∣ S λ ∣ + ∣ F B ∣ ) + C 2 λ − 1 { ∥ β S λ c ∥ 2 2 + λ ∥ β S λ c ∥ 1 2 } . |F_D|\le C_1(|S_\lambda|+|F_B|)+C_2\lambda^{-1}\left\{\lVert\beta_{S^c_\lambda}\rVert_2^2+\sqrt{\lambda}\lVert\beta_{S^c_\lambda}\rVert_1^2\right\}. ∣ F D ∣ ≤ C 1 ( ∣ S λ ∣ + ∣ F B ∣ ) + C 2 λ − 1 { ∥ β S λ c ∥ 2 2 + λ ∥ β S λ c ∥ 1 2 } .
which proves (22) .
Finally, Lemmas 1 and 5 and Corollary 1 imply
∣ S λ ∣ + ∣ F B ∣ ≲ s 0 λ − q s / 2 , |S_\lambda|+|F_B|\lesssim s_0\lambda^{-q_s/2}, ∣ S λ ∣ + ∣ F B ∣ ≲ s 0 λ − q s /2 ,
λ − 1 ∥ β S λ c ∥ 2 2 ≲ s 0 λ − q s / 2 , \lambda^{-1}\lVert\beta_{S^c_\lambda}\rVert_2^2\lesssim s_0\lambda^{-q_s/2}, λ − 1 ∥ β S λ c ∥ 2 2 ≲ s 0 λ − q s /2 ,
λ − 1 / 2 ∥ β S λ c ∥ 1 2 ≲ s 0 2 λ 1 / 2 − q s = ( s 0 λ − q s / 2 ) ( s 0 λ ( 1 − q s ) / 2 ) = o ( s 0 λ − q s / 2 ) . \lambda^{-1/2}\lVert\beta_{S^c_\lambda}\rVert_1^2\lesssim s_0^2\lambda^{1/2-q_s}=\left(s_0\lambda^{-q_s/2}\right)\left(s_0\lambda^{(1-q_s)/2}\right)=o\left(s_0\lambda^{-q_s/2}\right). λ − 1/2 ∥ β S λ c ∥ 1 2 ≲ s 0 2 λ 1/2 − q s = ( s 0 λ − q s /2 ) ( s 0 λ ( 1 − q s ) /2 ) = o ( s 0 λ − q s /2 ) .
Thus, for all sufficiently large T T T , there exist fixed constants C D , C U > 0 C_D,C_U>0 C D , C U > 0 such that, on G \mathcal{G} G ,
∣ F D ∣ ≤ C D s 0 λ − q s / 2 , ∣ U ∣ ≤ C U s 0 λ − q s / 2 . |F_D|\le C_Ds_0\lambda^{-q_s/2},\qquad|U|\le C_Us_0\lambda^{-q_s/2}. ∣ F D ∣ ≤ C D s 0 λ − q s /2 , ∣ U ∣ ≤ C U s 0 λ − q s /2 .
Since P ( G ) → 1 \mathbb{P}(\mathcal{G})\to1 P ( G ) → 1 , the final O p ( s 0 λ − q s / 2 ) O_p(s_0\lambda^{-q_s/2}) O p ( s 0 λ − q s /2 ) statement follows.
Proof of Lemma 7. On { ∣ U ∣ ≤ m T } ∩ E χ \{|U| \le m_T^{\sqrt{}} \} \cap\mathcal{E}_{\chi} { ∣ U ∣ ≤ m T } ∩ E χ , Weyl’s inequality and Lemma 2 give
λ min ( Σ ^ x , U U ) ≥ λ min ( Σ x , U U ) − ∥ Σ ^ x , U U − Σ x , U U ∥ o p ≥ κ − ∣ U ∣ ∥ Σ ^ x − Σ x ∥ max ≥ κ − C χ m T log ( K p ) T = κ − O { s 0 λ ( 1 − q s ) / 2 + λ } = κ − o ( 1 ) . \begin{aligned}
\lambda_{\min}(\widehat{\Sigma}_{x,UU}) \ge\lambda_{\min}(\Sigma_{x,UU}) - \lVert\widehat{\Sigma}_{x,UU} - \Sigma_{x,UU}\rVert_{\mathrm{op}} \\
&\ge\kappa- |U|\lVert\widehat{\Sigma}_{x} - \Sigma_{x}\rVert_{\max} \\
&\ge\kappa- C_{\chi}m_T^{\sqrt{}} \sqrt{\frac{\log(Kp)}{T}} \\
&= \kappa- O\left\{s_0\lambda^{(1-q_s)/2} + \sqrt{\lambda}\right\} = \kappa- o(1).
\end{aligned} λ m i n ( Σ x , U U ) ≥ λ m i n ( Σ x , U U ) − ∥ Σ x , U U − Σ x , U U ∥ op ≥ κ − ∣ U ∣ ∥ Σ x − Σ x ∥ m a x ≥ κ − C χ m T T log ( K p ) = κ − O { s 0 λ ( 1 − q s ) /2 + λ } = κ − o ( 1 ) .
The λ \sqrt{\lambda} λ term in the penultimate line comes from the ceiling in m T = ⌈ C U s 0 λ − q s / 2 ⌉ m_T^{\sqrt{}} = \lceil C_U s_0\lambda^{-q_s/2}\rceil m T = ⌈ C U s 0 λ − q s /2 ⌉ . Since s 0 λ ( 1 − q s ) / 2 = o ( 1 ) s_0\lambda^{(1-q_s)/2} = o(1) s 0 λ ( 1 − q s ) /2 = o ( 1 ) and λ → 0 \lambda\to0 λ → 0 , the final remainder is smaller than κ / 2 \kappa/2 κ /2 for all sufficiently large T T T , which proves λ min ( Σ ^ x , U U ) ≥ κ / 2 \lambda_{\min}(\widehat{\Sigma}_{x,UU}) \ge\kappa/2 λ m i n ( Σ x , U U ) ≥ κ /2 .
Proof of Theorem 1. Let J : = U ∖ F D J := U \setminus F_D J := U ∖ F D . Starting from (19) , using Lemma 6, the bounds from the proof of Proposition 2, and (B.8), we obtain on E T \mathcal{E}_T E T
1 2 Δ U ′ Σ ^ x , U U Δ U ≤ ( c u + 2 ) ∣ U ∣ λ ∥ Δ U ∥ 2 + 1 2 β U c ′ Σ ^ x , U c U c β U c ≤ ( c u + 2 ) ∣ U ∣ λ ∥ Δ U ∥ 2 + O p ( s 0 λ 1 − q s / 2 ) . \begin{aligned}
\frac{1}{2}\Delta_U^\prime\widehat{\Sigma}_{x,UU}\Delta_U \le(c_u+2)\sqrt{|U|}\lambda\lVert\Delta_U\rVert_2 + \frac{1}{2}\beta_{U^c}^\prime\widehat{\Sigma}_{x,U^cU^c}\beta_{U^c} \\
&\le(c_u+2)\sqrt{|U|}\lambda\lVert\Delta_U\rVert_2 + O_p\left(s_0\lambda^{1-q_s/2}\right).
\end{aligned} 2 1 Δ U ′ Σ x , U U Δ U ≤ ( c u + 2 ) ∣ U ∣ λ ∥ Δ U ∥ 2 + 2 1 β U c ′ Σ x , U c U c β U c ≤ ( c u + 2 ) ∣ U ∣ λ ∥ Δ U ∥ 2 + O p ( s 0 λ 1 − q s /2 ) .
Here, the last line follows from (B.10), Corollary 1, and s 0 λ ( 1 − q s ) / 2 = o ( 1 ) s_0\lambda^{(1-q_s)/2} = o(1) s 0 λ ( 1 − q s ) /2 = o ( 1 ) . By Lemma 7 and Young’s inequality,
κ 4 ∥ Δ U ∥ 2 2 ≤ ( c u + 2 ) ∣ U ∣ λ ∥ Δ U ∥ 2 + 1 2 β U c ′ Σ ^ x , U c U c β U c ≤ κ 8 ∥ Δ U ∥ 2 2 + 2 ( c u + 2 ) 2 κ ∣ U ∣ λ + 1 2 β U c ′ Σ ^ x , U c U c β U c , \begin{aligned}
\frac{\kappa}{4}\lVert\Delta_U\rVert_2^2 \le(c_u+2)\sqrt{|U|}\lambda\lVert\Delta_U\rVert_2 + \frac{1}{2}\beta_{U^c}^\prime\widehat{\Sigma}_{x,U^cU^c}\beta_{U^c} \\
&\le\frac{\kappa}{8}\lVert\Delta_U\rVert_2^2 + \frac{2(c_u+2)^2}{\kappa}|U|\lambda+ \frac{1}{2}\beta_{U^c}^\prime\widehat{\Sigma}_{x,U^cU^c}\beta_{U^c},
\end{aligned} 4 κ ∥ Δ U ∥ 2 2 ≤ ( c u + 2 ) ∣ U ∣ λ ∥ Δ U ∥ 2 + 2 1 β U c ′ Σ x , U c U c β U c ≤ 8 κ ∥ Δ U ∥ 2 2 + κ 2 ( c u + 2 ) 2 ∣ U ∣ λ + 2 1 β U c ′ Σ x , U c U c β U c ,
∥ Δ U ∥ 2 2 ≤ 16 ( c u + 2 ) 2 κ 2 ∣ U ∣ λ + 4 κ β U c ′ Σ ^ x , U c U c β U c = O p ( s 0 λ 1 − q s / 2 ) . (42) \lVert\Delta_U\rVert_2^2 \le\frac{16(c_u+2)^2}{\kappa^2}|U|\lambda+ \frac{4}{\kappa}\beta_{U^c}^\prime\widehat{\Sigma}_{x,U^cU^c}\beta_{U^c} = O_p\left(s_0\lambda^{1-q_s/2}\right). \tag*{(42)} ∥ Δ U ∥ 2 2 ≤ κ 2 16 ( c u + 2 ) 2 ∣ U ∣ λ + κ 4 β U c ′ Σ x , U c U c β U c = O p ( s 0 λ 1 − q s /2 ) . (42)
where Proposition 2 gives ∣ U ∣ λ = O p ( s 0 λ 1 − q s / 2 ) |U|\lambda= O_p(s_0\lambda^{1-q_s/2}) ∣ U ∣ λ = O p ( s 0 λ 1 − q s /2 ) . Thus
∥ Δ U ∥ 2 = O p ( s 0 λ 1 − q s / 2 ) . \lVert\Delta_U\rVert_2 = O_p\left(\sqrt{s_0\lambda^{1-q_s/2}}\right). ∥ Δ U ∥ 2 = O p ( s 0 λ 1 − q s /2 ) .
Using Lemma 3 and Corollary 1,
∥ β ^ B A R − β ∥ 2 ≤ ∥ Δ U ∥ 2 + ∥ β S λ c ∥ 2 = O p ( s 0 λ 1 − q s / 2 ) \left\|\widehat{\beta}_{BAR}-\beta\right\|_2 \leq\left\|\Delta_U\right\|_2+\left\|\beta_{S_\lambda^c}\right\|_2=O_p\left(\sqrt{s_0\lambda^{1-q_s/2}}\right) β B A R − β 2 ≤ ∥ Δ U ∥ 2 + β S λ c 2 = O p ( s 0 λ 1 − q s /2 )
∥ β ^ B A R − β ∥ 1 ≤ ∣ U ∣ ∥ Δ U ∥ 2 + ∥ β S λ c ∥ 1 = O p ( s 0 λ − q s / 2 s 0 λ 1 − q s / 2 ) + O ( s 0 λ ( 1 − q s ) / 2 ) = O p ( s 0 λ ( 1 − q s ) / 2 ) . \begin{aligned}
\left\|\widehat{\beta}_{BAR}-\beta\right\|_1
&\leq\sqrt{|U|}\left\|\Delta_U\right\|_2+\left\|\beta_{S_\lambda^c}\right\|_1\\
&=O_p\left(\sqrt{s_0\lambda^{-q_s/2}}\sqrt{s_0\lambda^{1-q_s/2}}\right)+O\left(s_0\lambda^{(1-q_s)/2}\right)\\
&=O_p\left(s_0\lambda^{(1-q_s)/2}\right).
\end{aligned} β B A R − β 1 ≤ ∣ U ∣ ∥ Δ U ∥ 2 + β S λ c 1 = O p ( s 0 λ − q s /2 s 0 λ 1 − q s /2 ) + O ( s 0 λ ( 1 − q s ) /2 ) = O p ( s 0 λ ( 1 − q s ) /2 ) .
Finally,
1 T ∥ X Δ ∥ 2 2 = Δ ′ Σ ^ x Δ = Δ U ′ Σ ^ x , U U Δ U − 2 Δ U ′ Σ ^ x , U U c β U c + β U c ′ Σ ^ x , U c U c β U c \begin{aligned}
\frac{1}{T}\|X\Delta\|_2^2=\Delta^\prime\widehat{\Sigma}_x\Delta\\
&=\Delta_U^\prime\widehat{\Sigma}_{x,UU}\Delta_U-2\Delta_U^\prime\widehat{\Sigma}_{x,UU^c}\beta_{U^c}+\beta_{U^c}^\prime\widehat{\Sigma}_{x,U^cU^c}\beta_{U^c}
\end{aligned} T 1 ∥ X Δ ∥ 2 2 = Δ ′ Σ x Δ = Δ U ′ Σ x , U U Δ U − 2 Δ U ′ Σ x , U U c β U c + β U c ′ Σ x , U c U c β U c
≤ 2 Δ U ′ Σ ^ x , U U Δ U + 2 β U c ′ Σ ^ x , U c U c β U c . \leq2\Delta_U^\prime\widehat{\Sigma}_{x,UU}\Delta_U+2\beta_{U^c}^\prime\widehat{\Sigma}_{x,U^cU^c}\beta_{U^c}. ≤ 2 Δ U ′ Σ x , U U Δ U + 2 β U c ′ Σ x , U c U c β U c .
Moreover, on { ∣ U ∣ ≤ m T ♯ } ∩ E χ \{|U|\leq m_T^\sharp\}\cap\mathcal{E}_\chi { ∣ U ∣ ≤ m T ♯ } ∩ E χ ,
λ max ( Σ ^ x , U U ) ≤ λ max ( Σ x , U U ) + ∥ Σ ^ x , U U − Σ x , U U ∥ o p ≤ C max + o ( 1 ) , \begin{aligned}
\lambda_{\max}(\widehat{\Sigma}_{x,UU})\leq\lambda_{\max}(\Sigma_{x,UU})+\|\widehat{\Sigma}_{x,UU}-\Sigma_{x,UU}\|_{\mathrm{op}} \\
&\leq C_{\max}+o(1),
\end{aligned} λ m a x ( Σ x , U U ) ≤ λ m a x ( Σ x , U U ) + ∥ Σ x , U U − Σ x , U U ∥ op ≤ C m a x + o ( 1 ) ,
so that
Δ U ′ Σ ^ x , U U Δ U = O p ( ∥ Δ U ∥ 2 2 ) = O p ( s 0 λ 1 − q s / 2 ) , \Delta_U^\prime\widehat{\Sigma}_{x,UU}\Delta_U=O_p\left(\|\Delta_U\|_2^2\right)=O_p\left(s_0\lambda^{1-q_s/2}\right), Δ U ′ Σ x , U U Δ U = O p ( ∥ Δ U ∥ 2 2 ) = O p ( s 0 λ 1 − q s /2 ) ,
β U c ′ Σ ^ x , U c U c β U c = O p ( s 0 λ 1 − q s / 2 ) . \beta_{U^c}^\prime\widehat{\Sigma}_{x,U^cU^c}\beta_{U^c}=O_p\left(s_0\lambda^{1-q_s/2}\right). β U c ′ Σ x , U c U c β U c = O p ( s 0 λ 1 − q s /2 ) .
Therefore,
T − 1 ∥ X ( β ^ B A R − β ) ∥ 2 2 = O p ( s 0 λ 1 − q s / 2 ) , T^{-1}\|X(\widehat{\beta}_{BAR}-\beta)\|_2^2=O_p\left(s_0\lambda^{1-q_s/2}\right), T − 1 ∥ X ( β B A R − β ) ∥ 2 2 = O p ( s 0 λ 1 − q s /2 ) ,
which completes the proof.
Proof of Corollary 2. Under K p = O ( T ν ) Kp=O(T^\nu) K p = O ( T ν ) , log ( K p ) = O ( log T ) \log(Kp)=O(\log T) log ( K p ) = O ( log T ) and λ ≍ log ( K p ) / T → 0 \lambda\asymp\log(Kp)/T\to0 λ ≍ log ( K p ) / T → 0 . Hence,
s 0 λ ( 1 − q s ) / 2 → 0 , s 0 λ 1 − q s / 2 = { s 0 λ ( 1 − q s ) / 2 } λ 1 / 2 → 0. s_0\lambda^{(1-q_s)/2} \to0, \\
s_0\lambda^{1-q_s/2} = \left\{s_0\lambda^{(1-q_s)/2}\right\}\lambda^{1/2} \to0. s 0 λ ( 1 − q s ) /2 → 0 , s 0 λ 1 − q s /2 = { s 0 λ ( 1 − q s ) /2 } λ 1/2 → 0.
The first line makes the ℓ 1 \ell_1 ℓ 1 rate vanish, while the second makes the prediction error and squared ℓ 2 \ell_2 ℓ 2 rate vanish. Bounded s 0 s_0 s 0 is sufficient since q s < 1 q_s < 1 q s < 1 and λ → 0 \lambda\to0 λ → 0 .
Additional Simulation Results
The following tables report the detailed Monte Carlo results underlying the aggregate win rates presented in the main text.25 Whereas the win-rate table summarizes performance across values of K K K and r r r , the appendix tables report results for each individual simulation design, allowing a more granular comparison of the competing estimators across sample sizes, dimensionality, sparsity levels, and innovation dependence. In addition to the measures reported in the win-rate table, we include three further evaluation criteria:
False positive rate = # { j : β ^ j ≠ 0 ∧ β j = 0 } # { j : β j = 0 } , \text{False positive rate} = \frac{\#\{j : \widehat{\beta}_j \ne0 \land\beta_j = 0\}}{\#\{j : \beta_j = 0\}}, False positive rate = # { j : β j = 0 } # { j : β j = 0 ∧ β j = 0 } ,
capturing the risk of including irrelevant variables.
False negative rate = # { j : β ^ j = 0 ∧ β j ≠ 0 } # { j : β j ≠ 0 } , \text{False negative rate} = \frac{\#\{j : \widehat{\beta}_j = 0 \land\beta_j \ne0\}}{\#\{j : \beta_j \ne0\}}, False negative rate = # { j : β j = 0 } # { j : β j = 0 ∧ β j = 0 } ,
capturing the risk of excluding relevant variables.
Table C.1. MC simulations: r = 1 r=1 r = 1 , ρ = 0 \rho=0 ρ = 0 , EBIC.
Table C.2. MC simulations: r = 1 r=1 r = 1 , ρ = 0.5 \rho=0.5 ρ = 0.5 , EBIC.
Table C.3. MC simulations: r = 2 r=2 r = 2 , ρ = 0 \rho=0 ρ = 0 , EBIC.
Table C.4. MC simulations: r = 2 r=2 r = 2 , ρ = 0.5 \rho=0.5 ρ = 0.5 , EBIC.
Table C.5. MC simulations: r = 5 r=5 r = 5 , ρ = 0 \rho=0 ρ = 0 , EBIC.
Table C.6. MC simulations: r = 5 r=5 r = 5 , ρ = 0.5 \rho=0.5 ρ = 0.5 , EBIC.
Table C.7. MC simulations: r = 10 r=10 r = 10 , ρ = 0 \rho=0 ρ = 0 , EBIC.
Table C.8. MC simulations: r = 10 r=10 r = 10 , ρ = 0.5 \rho=0.5 ρ = 0.5 , EBIC.
To provide a more detailed assessment of the support recovery, we complement the Monte Carlo tables with heatmaps of the estimated coefficient matrices. Each cell reports the element-wise mean absolute estimate,
1 n ∑ m = 1 n ∣ Φ ^ i j ( m ) ∣ , \frac{1}{n}\sum_{m=1}^{n}\left|\widehat{\Phi}_{ij}^{(m)}\right|, n 1 m = 1 ∑ n Φ ij ( m ) ,
where Φ ^ i j ( m ) \widehat{\Phi}_{ij}^{(m)} Φ ij ( m ) denotes the coefficient in position ( i , j ) (i,j) ( i , j ) estimated in the m th m^{\text{th}} m th Monte Carlo simulation. Taking absolute values prevents coefficients with opposite signs from cancelling across simulations and therefore provides a graphical representation of their average estimated magnitude. Darker blue cells indicate larger average coefficient magnitudes, whereas lighter blue cells correspond to coefficients estimated to be close to zero. Because the heatmap entries are averages across MC simulations, coefficients selected only occasionally may still produce small positive entries. To avoid visually overstating such negligible estimates, mean absolute coefficients below 10 − 3 10^{-3} 1 0 − 3 are treated as zero and displayed in white; without this threshold, the heatmaps would appear almost entirely colored. For readability, we report the designs with K = 20 K=20 K = 20 , T = 500 T=500 T = 500 , ρ = 0 \rho=0 ρ = 0 , and r ∈ { 1 , 2 , 5 , 10 } r\in\{1,2,5,10\} r ∈ { 1 , 2 , 5 , 10 } , with the regularization parameter λ \lambda λ selected by the EBIC.
Figure C.1. Heatmap of ∣ Φ ^ ∣ \lvert\widehat{\Phi}\rvert ∣ Φ ∣ for r = 1 r=1 r = 1 .
Figure C.2. Heatmap of ∣ Φ ^ ∣ |\widehat{\Phi}| ∣ Φ ∣ for r = 2 r = 2 r = 2 .
Figure C.3. Heatmap of ∣ Φ ^ ∣ |\widehat{\Phi}| ∣ Φ ∣ for r = 5 r = 5 r = 5 .
Figure C.4. Heatmap of ∣ Φ ^ ∣ |\widehat{\Phi}| ∣ Φ ∣ for r = 10 r = 10 r = 10 .
Additional Empirical Results
This section assesses the robustness of the empirical findings to the estimation window scheme. We retain the same dataset, forecasting equation, forecast horizons, standardization procedure, AR(1) benchmark, and evaluation measures used in the main empirical analysis.
We then compare the baseline rolling window results with those obtained using an expanding window, thereby assessing whether the strong short-horizon performance of BAR, as well as its more heterogeneous performance at longer horizons, is sensitive to the estimation-window scheme. The results are fully confirmed for the pre-COVID period, for which the two estimation schemes yield identical conclusions, and remain broadly consistent for the COVID-19 and post-pandemic periods, despite some minor differences in relative performance.
Table D.9. RMSFE and MAFE ratios relative to the AR(1) using an expanding window estimation.
Notes: the RMSFE and the MAFE are relative to the AR(1) benchmark. Bold and underlined entries denote the best-performing model(s) for each forecast horizon and period. Grey cells indicate models belonging to the Set of Superior Models (SSM) under the 50% MCS procedure using the T max T_{\max} T m a x statistic with squared errors, whereas blue cells indicate models included in the SSM when absolute errors are used.