Introduction
High-dimensional factor models describe common variation in many observed variables through a small number of latent factors. They provide a framework for dimension reduction, forecasting, and the analysis of economic and financial panels. Principal component methods and their large-dimensional theory have played a central role in these developments [27 , 3 , 2 , 5 ] , where the primary focus lies in estimating the vector space spanned by the matrix of loadings, a.k.a. loading space . Another critical part of factor analysis is to determine the number of factors, and several approaches have been proposed which are based on information criteria or eigenvalue ratios [3 , 1 , 21 ] . While it is commonly assumed that the factors are pervasive across the cross-sections, there also exist studies for understanding how the strength of factors affects the rates of estimation [22 , 4 ] .
Two distinct stylized features of modern data complicate the use of conventional principal component methods. First, the data may have heavy tails as frequently observed in economics and finance [9 ] , which invalidates the commonly made moment conditions underpinning covariance-based approaches. Second, some observations may be anomalous and arbitrarily deviate from the underlying model, due to the events such as financial crisis. When heavy tails and anomalies co-exist, extreme observations may be the realizations from the heavy-tailed distribution or anomalies and, regardless of their origin, negatively influence the accuracy of the existing principal component-based estimators [19 ] (He et al. 2022). In this paper, we develop methods for estimating the loading space and the number of factors under elliptical factor models while permitting the presence of heavy tails, anomalies, and weak factors.
Elliptical distributions are popularly considered for high-dimensional data analysis under heavy tails. For instance, Han and Liu [15 ] develop elliptical component analysis using multivariate Kendall’s tau, and Fan et al. [12 ] study robust covariance estimation through elliptical factor models. For factor modeling, methods based on spatial Kendall’s tau have been developed for estimating the loading space and the latent factor process (He et al., 2022) and the factor number (Yu et al., 2019). Other methods address heavy tails under different distributional assumptions, e.g., by adopting the Spearman correlation for factor number estimation under a scale-mixture structure [25 ] , utilizing Huber loss [7 ] or data truncation [6 ] . These approaches offer different tradeoffs between distributional structure and moment requirements.
There also exists a closely related but distinct line of research which focuses on developing principal component methods that are robust against outliers. Such methods combine the projection pursuit with robust scatter estimation [19 ] , and can accommodate sparse loadings [18 ] , or cellwise and rowwise outliers [20 ] . It is worth pointing out that the procedures designed to control heavy-tailed fluctuations and those designed to be robust against the anomalies, address related but different objectives; see Raymaekers and Rousseau [26 ] for a comprehensive overview and further discussions.
In this paper, we propose the Divide-and-Conquer (DC) estimator for the loading space under elliptical factor models. Simply put, DC partitions the data into small blocks, computes the orthogonal projector onto the full column space of each block, and extracts the leading eigenspace of their average. When the factor number r r r is known, we take the block size to be r r r ; if it is unknown, we set the block size to be a strict upper bound on r r r . This sets the DC estimator apart from the existing tail-robust methods that often rely on selecting the tuning parameters that depend on the unknown tail index. The aggregation of local spectral information has an established role in distributed estimation [13 , 14 ] (He et al. 2025) but in contrast to these earlier works, the DC estimator aggregates the blockwise orthogonal projectors of full column rank of the block size, even when the block size exceeds the number of factors. Our investigation reveals that this proposed approach allows for establishing a signal-to-noise eigengap under possibly weak and heterogeneous factor strengths, without making any moment constraint.
For factor number estimation, we propose the Split-sample Tail- and Anomaly-Robust (STAR) estimator. splitting the data into two, the STAR projects the second half of the data onto the DC estimator from the first half, takes the ratio of each pair of successive squared scores, and locates the maximizer of the sample median of these ratios over time. By its construction, the STAR estimator utilizes the ellipticity of the data and benefits from the eigengap preserved in the DC loading space estimator.
Theoretically, without imposing any finite-moment requirements while permitting the presence of anomalies, we establish a non-asymptotic probabilistic bound for the estimation error of the DC estimator. If factors are strong and the anomalies are absent, by setting the block size b b b as the factor number (b = r b=r b = r ), the convergence rate of the DC estimator matches the rate attained by the classical principal component methods under the existence of the fourth moment in the data [3 ] . We also show the consistency of the STAR estimator with which the additional application of the DC estimator allows us to estimate the factors and common components, also without any moment constraints.
To summarize, Table 1 and Table 2 provide an overview of the methods most closely related to our study in comparison with DC and STAR estimators, respectively. For Barigozzi et al. [6 ] , the displayed rate concerns the vector factor model under their general dependence conditions. Under their bounded deterministic factors and the remaining conditions of their loading estimation result, their rate would reduce to O P ( 1 / T + 1 / p ) O_{\mathrm{P}}(1/\sqrt{T}+1/p) O P ( 1/ T + 1/ p ) .
Table 1. Comparison of loading space and principal subspace estimators. The rates are derived for the distance between true loading matrix A \mathbf{A} A and estimator A ^ \widehat{\mathbf{A}} A measured by ∥ A ( A T A ) − 1 A T − A ^ ( A ^ T A ^ ) − 1 A ^ T ∥ F \left\|\mathbf{A}(\mathbf{A}^{\mathsf{T}}\mathbf{A})^{-1}\mathbf{A}^{\mathsf{T}}-\widehat{\mathbf{A}}(\widehat{\mathbf{A}}^{\mathsf{T}}\widehat{\mathbf{A}})^{-1}\widehat{\mathbf{A}}^{\mathsf{T}}\right\|_{F} A ( A T A ) − 1 A T − A ( A T A ) − 1 A T F , under strong factors and in the absence of anomalies; the rate reported for DC is a special case of Theorem 1. Here, T T T refers to the sample size and p p p the dimension of the data.
Table 2. Comparison of factor number estimators.
Notations. Unless specified otherwise, we denote scalars, vectors and matrices by plain letters, lower-case bold letters and capital bold letters, i.e., x x x (or X X X ), x \mathbf{x} x and X \mathbf{X} X , respectively. We use x i x_i x i and X i j X_{ij} X ij to denote the i i i th element of a vector x \mathbf{x} x and the ( i , j ) (i,j) ( i , j ) th element of a matrix X \mathbf{X} X , respectively. We use a ≍ b a \asymp b a ≍ b (resp. a ≍ P b a \asymp_{\mathrm{P}} b a ≍ P b ) to denote a = O ( b ) a=O(b) a = O ( b ) and b = O ( a ) b=O(a) b = O ( a ) (resp. a = O P ( b ) a=O_{\mathrm{P}}(b) a = O P ( b ) and b = O P ( a ) b=O_{\mathrm{P}}(a) b = O P ( a ) ). Hereafter, given a positive integer m m m , define [ m ] : = { 1 , 2 , … , m } [m]:=\{1,2,\ldots,m\} [ m ] := { 1 , 2 , … , m } . The i i i th largest eigenvalue of a real symmetric matrix X \mathbf{X} X is denoted by λ i ( X ) \lambda_i(\mathbf{X}) λ i ( X ) , and SVD i ( X ) \operatorname{SVD}_i(\mathbf{X}) SVD i ( X ) represents the leading i i i eigenvectors of X \mathbf{X} X , by which we always refer to the orthonormal eigenvectors of the matrix corresponding to the i i i largest eigenvalues. For any symmetric matrices X 1 \mathbf{X}_1 X 1 and X 2 \mathbf{X}_2 X 2 , the notation X 1 ⪯ X 2 \mathbf{X}_1 \preceq\mathbf{X}_2 X 1 ⪯ X 2 means X 2 − X 1 \mathbf{X}_2-\mathbf{X}_1 X 2 − X 1 is positive semi-definite.
Methodology
Model and problem formulation
Consider a sequence of p p p -dimensional random vectors { x t 0 } t ∈ [ T ] \{x_t^0\}_{t\in[T]} { x t 0 } t ∈ [ T ] admitting the factor model representation (e.g. Bai and Ng [3 ] ; Bai [2 ] ) such that
x t 0 = A f t + ϵ t = : χ t + ϵ t , (1) x_t^0 = \mathbf{A}f_t+\epsilon_t=:\chi_t+\epsilon_t, \tag*{(1)} x t 0 = A f t + ϵ t =: χ t + ϵ t , (1)
where A ∈ R p × r \mathbf{A} \in\mathbb{R}^{p \times r} A ∈ R p × r is the factor loading matrix, f t ∈ R r \mathbf{f}_t \in\mathbb{R}^r f t ∈ R r is the latent core factor, and ϵ t ∈ R p \boldsymbol{\epsilon}_t \in\mathbb{R}^p ϵ t ∈ R p is the idiosyncratic noise. The factor-driven χ t \boldsymbol{\chi}_t χ t is referred to as the common component. The number of factors r r r is fixed but unknown. Then, we observe { x t } t ∈ [ T ] \{\mathbf{x}_t\}_{t \in[T]} { x t } t ∈ [ T ] , where
x t = { x t 0 , t ∉ T , z t , t ∈ T , (2) \mathbf{x}_t =
\begin{cases}
\mathbf{x}_t^0, & t \notin\mathcal{T},\\
\mathbf{z}_t, & t \in\mathcal{T},
\end{cases} \tag*{(2)} x t = { x t 0 , z t , t ∈ / T , t ∈ T , (2)
for an unknown set T ⊂ [ T ] \mathcal{T} \subset[T] T ⊂ [ T ] and arbitrary vectors z t ∈ R p \mathbf{z}_t \in\mathbb{R}^p z t ∈ R p representing the anomalies. The index set of anomalies may be random or deterministic, see Assumption 4 below for the conditions on T \mathcal{T} T . In this setup, it is of interest to estimate the loading space spanned by the columns of A \mathbf{A} A and the factor number r r r under (1) , from the observations { x t } t ∈ [ T ] \{\mathbf{x}_t\}_{t \in[T]} { x t } t ∈ [ T ] which are potentially contaminated by anomalies. Further, even in the absence of anomalies, we permit { x t 0 } t ∈ [ T ] \{\mathbf{x}_t^0\}_{t \in[T]} { x t 0 } t ∈ [ T ] to be heavy-tailed without any moment condition as specified below.
Following Fan et al. [12 ] and He et al. (2022), we formulate the model for x t 0 \mathbf{x}_t^0 x t 0 through a joint elliptical distribution. We say a random vector y ∈ R h \mathbf{y} \in\mathbb{R}^h y ∈ R h follows an elliptical distribution, denoted by y ∼ E D h ( μ , Σ 0 , ζ ) \mathbf{y} \sim\mathrm{ED}_h(\boldsymbol{\mu}, \mathbf{\Sigma}_0, \zeta) y ∼ ED h ( μ , Σ 0 , ζ ) , if
y = d μ + ζ A 0 u , (3) \mathbf{y} \overset{d}{=} \boldsymbol{\mu} + \zeta\mathbf{A}_0 \mathbf{u}, \tag*{(3)} y = d μ + ζ A 0 u , (3)
where μ ∈ R h \boldsymbol{\mu} \in\mathbb{R}^h μ ∈ R h is deterministic, u ∈ R q \mathbf{u} \in\mathbb{R}^q u ∈ R q is a random vector uniformly distributed on the unit sphere S q − 1 \mathbb{S}^{q-1} S q − 1 with q ≤ h q \le h q ≤ h , the radius ζ ≥ 0 \zeta\ge0 ζ ≥ 0 is a scalar random variable independent of u \mathbf{u} u , A 0 ∈ R h × q \mathbf{A}_0 \in\mathbb{R}^{h \times q} A 0 ∈ R h × q is a deterministic matrix with full column rank, and the matrix Σ 0 = A 0 A 0 T \mathbf{\Sigma}_0 = \mathbf{A}_0\mathbf{A}_0^{\mathsf{T}} Σ 0 = A 0 A 0 T is referred to as the scatter matrix of y \mathbf{y} y . Such framework contains multivariate Gaussian distribution as a special case and, more importantly, includes a large class of heavy-tailed distributions such as the multivariate elliptical t t t -distribution.
Note that the scatter matrix Σ 0 \mathbf{\Sigma}_0 Σ 0 is only defined up to a positive scalar factor. Namely, the distribution in (3) would remain unchanged if A 0 \mathbf{A}_0 A 0 and ζ \zeta ζ are replaced by c A 0 c\mathbf{A}_0 c A 0 and c − 1 ζ c^{-1}\zeta c − 1 ζ , respectively. In view of this, in formulating the elliptical distribution of x t 0 \mathbf{x}_t^0 x t 0 , we adopt a representation where the block of the joint scatter matrix corresponding to the factor is I r \mathbf{I}_r I r .
Assumption 1. For each t ∈ [ T ] t \in[T] t ∈ [ T ] , we assume that f t \mathbf{f}_t f t and ϵ t \boldsymbol{\epsilon}_t ϵ t in (1) satisfy
\begin{pmatrix}
\mathbf{f}_t\\
\boldsymbol{\epsilon}_t
\end{pmatrix}
=
\zeta_t
\begin{pmatrix}
\mathbf{I}_r & \mathbf{0}\\
\mathbf{0} & \mathbf{A}_{\epsilon}
\end{pmatrix}
\frac{\mathbf{u}_t}{\lVert\mathbf{u}_t \rVert}, \tag*{(4)}\end{pmatrix}
where ζ t \zeta_t ζ t and u t \mathbf{u}_t u t are independent copies of a scalar random variable ζ \zeta ζ and a random vector u ∼ N ( 0 , I r + p ) \mathbf{u} \sim\mathcal{N}(\mathbf{0}, \mathbf{I}_{r+p}) u ∼ N ( 0 , I r + p ) , respectively. Also, ζ \zeta ζ is independent of u \mathbf{u} u , and is finite and strictly positive almost surely.
Assumption 1 implies that ( f t ⊤ , ϵ t ⊤ ) ⊤ ∼ ED r + p ( 0 , diag ( I r , Σ ϵ ) , ζ ) (\mathbf{f}_t^\top,\boldsymbol{\epsilon}_t^\top)^\top\sim\operatorname{ED}_{r+p}(0,\operatorname{diag}(\mathbf{I}_r,\boldsymbol{\Sigma}_{\epsilon}),\zeta) ( f t ⊤ , ϵ t ⊤ ) ⊤ ∼ ED r + p ( 0 , diag ( I r , Σ ϵ ) , ζ ) , where Σ ϵ : = A ϵ A ϵ ⊤ ∈ R p × p \boldsymbol{\Sigma}_{\epsilon}:=\mathbf{A}_{\epsilon}\mathbf{A}_{\epsilon}^{\top}\in\mathbb{R}^{p\times p} Σ ϵ := A ϵ A ϵ ⊤ ∈ R p × p . Since linear transformations of elliptically distributed random vectors remain elliptical, the model in (1) further implies that x t 0 \mathbf{x}_t^0 x t 0 is elliptically distributed itself. Specifically, let us define M : = [ A , A ϵ ] ∈ R p × ( r + p ) \mathbf{M}:=[\mathbf{A},\mathbf{A}_{\epsilon}]\in\mathbb{R}^{p\times(r+p)} M := [ A , A ϵ ] ∈ R p × ( r + p ) and v t : = u t / ∥ u t ∥ \mathbf{v}_t:=\mathbf{u}_t/\lVert\mathbf{u}_t\rVert v t := u t / ∥ u t ∥ for t ∈ [ T ] t\in[T] t ∈ [ T ] . Then x t 0 \mathbf{x}_t^0 x t 0 is elliptically distributed and admits the representation
x t 0 = ζ t M v t . (5) \mathbf{x}_t^0=\zeta_t\mathbf{M}\mathbf{v}_t. \tag*{(5)} x t 0 = ζ t M v t . (5)
with its scatter matrix
Σ x : = M M ⊤ = A A ⊤ + Σ ϵ . (6) \boldsymbol{\Sigma}_x:=\mathbf{M}\mathbf{M}^{\top}=\mathbf{A}\mathbf{A}^{\top}+\boldsymbol{\Sigma}_{\epsilon}. \tag*{(6)} Σ x := M M ⊤ = A A ⊤ + Σ ϵ . (6)
Without loss of generality, we suppose that ζ \zeta ζ is nonnegative as the sign can be absorbed into the direction vector u t \mathbf{u}_t u t . Such convention is standard in the stochastic representation of elliptical distributions [12 ] (He et al. 2022). Assumption 1 particularly requires P ( ζ > 0 ) = 1 \mathbb{P}(\zeta>0)=1 P ( ζ > 0 ) = 1 , which can be seen as a nondegeneracy condition. Also, we do not place any restriction on the growth of ζ t \zeta_t ζ t with p p p , unlike in He et al. (2022) where it is required that ζ / p = O P ( 1 ) \zeta/\sqrt{p}=\mathcal{O}_{P}(1) ζ / p = O P ( 1 ) , see their Assumption A. Further, we do not impose any moment condition on x t 0 \mathbf{x}_t^0 x t 0 , viz. x t 0 \mathbf{x}_t^0 x t 0 is not required to have finite first or second moments, and the scatter matrix Σ x \boldsymbol{\Sigma}_x Σ x is a well-defined quantity that characterizes the cross-sectional dependence of the data even when the covariance matrix of x t 0 \mathbf{x}_t^0 x t 0 does not exist.
Assumption 2. The number of factors r r r is fixed. For some fixed δ > 0 \delta>0 δ > 0 and an absolute constant c λ > 0 c_{\lambda}>0 c λ > 0 , the loading matrix satisfies A ⊤ A ⪰ c λ p δ I r \mathbf{A}^{\top}\mathbf{A}\succeq c_{\lambda}p^{\delta}\mathbf{I}_r A ⊤ A ⪰ c λ p δ I r for all sufficiently large p p p .
Assumption 3. The matrix Σ ϵ \boldsymbol{\Sigma}_{\epsilon} Σ ϵ satisfies c ϵ I p ⪯ Σ ϵ ⪯ C ϵ I p c_{\epsilon}\mathbf{I}_p\preceq\boldsymbol{\Sigma}_{\epsilon}\preceq C_{\epsilon}\mathbf{I}_p c ϵ I p ⪯ Σ ϵ ⪯ C ϵ I p for some absolute constants c ϵ , C ϵ > 0 c_{\epsilon},C_{\epsilon}>0 c ϵ , C ϵ > 0 .
Assumption 2 provides a very general setup, in that it only imposes a lower bound on the weakest loading direction, and permits the factor strengths captured by the eigenvalues of A ⊤ A \mathbf{A}^{\top}\mathbf{A} A ⊤ A to be highly heterogeneous. The case when 0 < δ < 1 0<\delta<1 0 < δ < 1 allows for weak factors, while δ = 1 \delta=1 δ = 1 indicates that the factors are pervasive across the cross-sections (cf. Lam et al. [22 ] , § 2.3 ), and we allow for the case where δ > 1 \delta>1 δ > 1 . Assumption 3 bounds the noise scatter eigenvalues away from zero and infinity. From (6) , Assumptions 2 and 3, we have λ r ( Σ x ) ≥ c λ p δ + c ϵ \lambda_r(\boldsymbol{\Sigma}_x)\ge c_{\lambda}p^{\delta}+c_{\epsilon} λ r ( Σ x ) ≥ c λ p δ + c ϵ for all sufficiently large p p p , while c ϵ ≤ λ j ( Σ x ) ≤ C ϵ c_{\epsilon}\le\lambda_j(\boldsymbol{\Sigma}_x)\le C_{\epsilon} c ϵ ≤ λ j ( Σ x ) ≤ C ϵ for r < j ≤ p r<j\le p r < j ≤ p , so that the leading r r r eigenvalues of Σ x \boldsymbol{\Sigma}_x Σ x are separated from the remaining ones.
We allow a subset of the observations to be replaced by arbitrary anomalous vectors, as formalized below.
Assumption 4. There exists a constant T ‾ ∈ [ 0 , T ) \overline{T}\in[0,T) T ∈ [ 0 , T ) such that the possibly random anomaly index set T \mathcal{T} T is independent of { ( ζ t , u t ) } t ∈ [ T ] \{(\zeta_t,\mathbf{u}_t)\}_{t\in[T]} {( ζ t , u t ) } t ∈ [ T ] in Assumption 1, and satisfies
T ⊂ [ T ] and ∣ T ∣ ≤ T ‾ almost surely . (7) \mathcal{T}\subset[T]\quad\text{and}\quad|\mathcal{T}|\le\overline{T}\quad\text{almost surely}. \tag*{(7)} T ⊂ [ T ] and ∣ T ∣ ≤ T almost surely . (7)
It is worth pointing out that the bound T ‾ \overline{T} T in Assumption 4 may depend on T T T and p p p , and the anomalous vectors may depend on the latent { x t 0 } t ∈ [ T ] \{x_t^0\}_{t\in[T]} { x t 0 } t ∈ [ T ] and need not have bounded magnitudes nor belong to any specified subspace. Throughout, the set T \mathcal{T} T and the anomalous observations are unknown.
Divide-and-Conquer estimation of the loading space
We describe the proposed DC loading space estimation method, which proceeds by combining blockwise estimators of the loading space from the data blocks of size b b b . We first present the method when the factor number r r r is known, in which case we set the block size as b = r b=r b = r . More generally, the following construction is defined for any integer r ≤ b ≤ min ( T , p ) r\le b\le\min(T,p) r ≤ b ≤ min ( T , p ) .
Set m : = ⌊ T / b ⌋ m:=\lfloor T/b\rfloor m := ⌊ T / b ⌋ and partition the first m b mb mb observations into consecutive blocks M i : = { ( i − 1 ) b + 1 , … , i b } \mathcal{M}_i:=\{(i-1)b+1,\ldots,ib\} M i := {( i − 1 ) b + 1 , … , ib } , i ∈ [ m ] i\in[m] i ∈ [ m ] . For each block i ∈ [ m ] i\in[m] i ∈ [ m ] , compute the following matrix and thus its b b b leading eigenvectors as
S ^ i : = 1 b ∑ t ∈ M i x t x t ⊤ ∈ R p × p , U ^ i : = SVD b ( S ^ i ) ∈ R p × b , (8) \widehat{S}_i:=\frac{1}{b}\sum_{t\in\mathcal{M}_i}x_tx_t^{\top}\in\mathbb{R}^{p\times p},\qquad\widehat{U}_i:=\operatorname{SVD}_b(\widehat{S}_i)\in\mathbb{R}^{p\times b}, \tag*{(8)} S i := b 1 t ∈ M i ∑ x t x t ⊤ ∈ R p × p , U i := SVD b ( S i ) ∈ R p × b , (8)
respectively. Then we construct the aggregation of the projectors over all m m m blocks and obtain the DC estimator as
P ^ : = 1 m ∑ i = 1 m U ^ i U ^ i ⊤ ∈ R p × p and A ^ b : = SVD b ( P ^ ) ∈ R p × b , (9) \widehat{P}:=\frac{1}{m}\sum_{i=1}^{m}\widehat{U}_i\widehat{U}_i^{\top}\in\mathbb{R}^{p\times p}\quad\text{and}\quad\widehat{A}_b:=\operatorname{SVD}_b(\widehat{P})\in\mathbb{R}^{p\times b}, \tag*{(9)} P := m 1 i = 1 ∑ m U i U i ⊤ ∈ R p × p and A b := SVD b ( P ) ∈ R p × b , (9)
respectively.
Markedly, many DC or distributed PCA-type of methods [13 , 14 ] take the r r r leading eigenvectors in their blockwise estimation step analogous to (8) . Our investigation reveals that it is crucial to take the b b b rather than r r r leading eigenvectors, even when b > r b>r b > r , so that for an uncontaminated block where M i ∩ T = ∅ \mathcal{M}_i\cap\mathcal{T}=\varnothing M i ∩ T = ∅ , the matrix U ^ i U ^ i ⊤ \widehat{U}_i\widehat{U}_i^{\top} U i U i ⊤ projects onto the full column space of the block data matrix. This allows for the exact characterization of all the eigenvalues of E [ U ^ i U ^ i ⊤ ] \mathbb{E}[\widehat{U}_i\widehat{U}_i^{\top}] E [ U i U i ⊤ ] and for deriving the desired eigengap, which in turn is utilized for the estimation of the loading space by the column space of the first r r r columns of A ^ b \widehat{A}_b A b (see Lemma 4).
To see the tail-robust property of the DC estimator, note that retaining the full block span makes the projector U ^ i U ^ i ⊤ \widehat{U}_i\widehat{U}_i^{\top} U i U i ⊤ invariant to nonzero rescaling of individual observations within the block, and its operator norm is always bounded by one. Therefore, anomalous observations cannot increase the operator norm of a block contribution by arbitrarily large magnitudes. The presence of anomalies can still change the column spaces of the affected block data matrices, but only the blocks containing replacements are affected so their influence is controlled by the fraction of affected blocks, which is captured by the term T / m \mathcal{T}/m T / m in Theorem 1 below.
We point out that the individual block spaces need not consistently estimate the loading space, and each block estimation in (8) can be viewed as a harmless “truncation” that could preserve the span of the loading space, before the aggregation step in (9) .
Remark 1 (Blockwise projector construction). Suppose the true factor number r r r is known. Then it is natural to set b = r b=r b = r and hence A ^ r = SVD r ( P ^ ) \widehat{\mathbf A}_r=\operatorname{SVD}_r(\widehat{\mathbf P}) A r = SVD r ( P ) in (9) . However, the choice of retaining all b b b eigenvectors in (8) is deliberate and essential for the desirable properties of the DC estimator.
First, our Theorem 1 allows for b ≥ r b\ge r b ≥ r , as long as the b b b leading eigenvectors are taken for blockwise projector construction as in (8) . This is exactly the setting in which we propose the factor number estimator in § 2.3 , where we set b = r max b=r_{\max} b = r m a x with r max r_{\max} r m a x denoting a strict upper bound on the unknown factor number r r r . When r r r is known, the choice b = r b=r b = r is the smallest block size allowable and it yields the best convergence rate for the DC estimator.
On the contrary, if b > r b>r b > r but only the first r r r eigenvectors are retained, i.e., U ^ i = SVD r ( S ^ i ) \widehat{\mathbf U}_i=\operatorname{SVD}_r(\widehat{\mathbf S}_i) U i = SVD r ( S i ) in (8) , then U ^ i U ^ i T \widehat{\mathbf U}_i\widehat{\mathbf U}_i^{\mathsf T} U i U i T need not preserve the full column span of the block data matrix, and our proof does not directly apply to this construction. In this case, a considerably stronger moment condition may be needed to let each U ^ i U ^ i T \widehat{\mathbf U}_i\widehat{\mathbf U}_i^{\mathsf T} U i U i T stay sufficiently close to the true projector onto the loading space so as to apply arguments as in the proof of Theorem 2 of Fan et al. [13 ] . It may also require the block size b b b to be lower bounded by some unknown constant (cf. Theorem 4 of Fan et al. [13 ] , or Theorem 1 of Li et al. (2024)). Figure 2 numerically compares this alternative with our DC construction.
Figure 2. Sensitivity to the block size for ( T , p ) = ( 100 , 100 ) (T,p)=(100,100) ( T , p ) = ( 100 , 100 ) , r = 2 r=2 r = 2 , a = ϕ = 0 a=\phi=0 a = ϕ = 0 and ε = 0 \varepsilon=0 ε = 0 . The upper panel retains b b b eigenvectors locally as in our DC construction; the lower panel retains r r r eigenvectors locally. The horizontal axis is over the block size b b b , and the vertical axis gives the average loading space error d ( ⋅ , A ) d(\cdot,\mathcal{A}) d ( ⋅ , A ) over 100 replications. Different curves correspond to ν ∈ { 0.5 , 1.5 , 2.5 , 4.5 } \nu\in\{0.5,1.5,2.5,4.5\} ν ∈ { 0.5 , 1.5 , 2.5 , 4.5 } .
Remark 2 (Connection to spatial sign covariance matrices). We note a connection between the DC estimator and robust principal component methods based on spatial sign covariance matrices. In particular, when DC is applied with a single observation in each block such that b = 1 b=1 b = 1 , the average of all block projectors coincides with the spatial sign covariance matrix, T − 1 ∑ t = 1 T x t x t T / ∥ x t ∥ 2 T^{-1}\sum_{t=1}^{T}\mathbf{x}_t\mathbf{x}_t^{\mathsf T}/\lVert\mathbf{x}_t\rVert^2 T − 1 ∑ t = 1 T x t x t T / ∥ x t ∥ 2 . Principal component analysis based on spatial sign covariance matrices is studied by Taskinen et al. [28 ] , and their population eigenvalue structure under elliptical distributions is investigated by Dürre et al. [10 ] . The correspondence between population eigenspaces does not, by itself, ensure accurate estimation when the relevant population eigengap becomes small. For instance, under an increasingly dominant factor, the choice of b = 1 b=1 b = 1 may lead to a poor estimator in the presence of anomalies, whereas our DC construction with b = r b=r b = r remains consistent under the same model even when the anomalies are present. We show an example in the supplementary material (Section B) that the DC estimator using singleton blocks may fail to consistently estimate the loading space unless, for instance, stronger conditions on the factor strengths are imposed.
Split-sample tail- and anomaly-robust factor number estimator
We next present the STAR method for estimating the unknown factor number r r r . Let r max ∈ [ 2 , min ( p , ⌊ T / 2 ⌋ ) ] r_{\max}\in[2,\min(p,\lfloor T/2\rfloor)] r m a x ∈ [ 2 , min ( p , ⌊ T /2 ⌋)] be an integer such that r < r max r<r_{\max} r < r m a x , i.e., r max − 1 r_{\max}-1 r m a x − 1 is an upper bound on the candidate factor number. For simplicity, we only consider the regime where the loading matrix satisfies
c λ I r ⪯ A T A / p ⪯ C λ I r . (10) c_{\lambda}\mathbf I_r\preceq\mathbf A^{\mathsf T}\mathbf A/p\preceq C_{\lambda}\mathbf I_r. \tag*{(10)} c λ I r ⪯ A T A / p ⪯ C λ I r . (10)
for some absolute constants 0 < c λ ≤ C λ < ∞ 0 < c_{\lambda} \le C_{\lambda} < \infty 0 < c λ ≤ C λ < ∞ . Condition (10) is not required for the DC loading space guarantee in Theorem 1 below, and can be relaxed at the cost of more complicated arguments. Under this, Assumption 2 holds with δ = 1 \delta= 1 δ = 1 . Then, STAR proceeds as below.
Stage 1 (Sample splitting and DC). Let I 1 : = { 1 , … , ⌊ T / 2 ⌋ } \mathcal{I}_1 := \{1, \ldots, \lfloor T/2 \rfloor\} I 1 := { 1 , … , ⌊ T /2 ⌋} and I 2 : = { ⌊ T / 2 ⌋ + 1 , … , T } \mathcal{I}_2 := \{\lfloor T/2 \rfloor+ 1, \ldots, T\} I 2 := {⌊ T /2 ⌋ + 1 , … , T } be a deterministic partition of [ T ] [T] [ T ] . Apply the DC construction in (8) and (9) to { x t } t ∈ I 1 \{\boldsymbol{x}_t\}_{t\in\mathcal{I}_1} { x t } t ∈ I 1 with b = r max b = r_{\max} b = r m a x . Denote the resulting estimator by A ^ r max ( 1 ) ∈ R p × r max \widehat{\mathbf{A}}_{r_{\max}}^{(1)} \in\mathbb{R}^{p\times r_{\max}} A r m a x ( 1 ) ∈ R p × r m a x .
Stage 2 (projected coordinates). For every t ∈ I 2 t\in\mathcal{I}_2 t ∈ I 2 , we construct a low-dimensional projection score such that
ξ ^ t : = ( A ^ r max ( 1 ) ) ⊤ x t ∈ R r max . (11) \widehat{\boldsymbol{\xi}}_t := \left(\widehat{\mathbf{A}}_{r_{\max}}^{(1)}\right)^\top\boldsymbol{x}_t \in\mathbb{R}^{r_{\max}}. \tag*{(11)} ξ t := ( A r m a x ( 1 ) ) ⊤ x t ∈ R r m a x . (11)
To see why such construction can be useful, define A r max : = SVD r max ( Σ x ) \mathbf{A}_{r_{\max}} := \operatorname{SVD}_{r_{\max}}(\boldsymbol{\Sigma}_x) A r m a x := SVD r m a x ( Σ x ) , and the oracle projected vector ξ t : = A r max ⊤ x t \boldsymbol{\xi}_t := \mathbf{A}_{r_{\max}}^\top\boldsymbol{x}_t ξ t := A r m a x ⊤ x t . Then for a clean observation with t ∉ T t \notin\mathcal{T} t ∈ / T , note that ξ t = ζ t A r max ⊤ M v t \boldsymbol{\xi}_t = \zeta_t\mathbf{A}_{r_{\max}}^\top\mathbf{M}\boldsymbol{v}_t ξ t = ζ t A r m a x ⊤ M v t by (5) , and its scatter matrix is
A r max ⊤ Σ x A r max = diag ( λ 1 ( Σ x ) , … , λ r max ( Σ x ) ) : = Λ r max . \mathbf{A}_{r_{\max}}^\top\boldsymbol{\Sigma}_x\mathbf{A}_{r_{\max}} = \operatorname{diag}\left(\lambda_1(\boldsymbol{\Sigma}_x),\ldots,\lambda_{r_{\max}}(\boldsymbol{\Sigma}_x)\right) := \boldsymbol{\Lambda}_{r_{\max}}. A r m a x ⊤ Σ x A r m a x = diag ( λ 1 ( Σ x ) , … , λ r m a x ( Σ x ) ) := Λ r m a x .
In particular, by Assumptions 2, 3, and (10) , we have
λ j ( Σ x ) ≍ { p , j ∈ [ r ] , 1 , r < j ≤ r max . (12) \lambda_j(\boldsymbol{\Sigma}_x) \asymp
\begin{cases}
p, & j \in[r],\\
1, & r < j \le r_{\max}.
\end{cases} \tag*{(12)} λ j ( Σ x ) ≍ { p , 1 , j ∈ [ r ] , r < j ≤ r m a x . (12)
Markedly, such separation between the eigenvalues of the scatter matrix of ξ t \boldsymbol{\xi}_t ξ t holds even when the projected coordinates have no finite second moments, and its sample counterpart is exploited in the next stage.
Stage 3 (median-of-ratios). For each t ∈ I 2 t\in\mathcal{I}_2 t ∈ I 2 and coordinate j ∈ [ r max − 1 ] j\in[r_{\max}-1] j ∈ [ r m a x − 1 ] , define the ratio
ρ ^ t , j : = ξ ^ t , j 2 ξ ^ t , j + 1 2 . (13) \widehat{\rho}_{t,j} := \frac{\widehat{\xi}_{t,j}^{2}}{\widehat{\xi}_{t,j+1}^{2}}. \tag*{(13)} ρ t , j := ξ t , j + 1 2 ξ t , j 2 . (13)
By convention, when ξ ^ t , j + 1 2 = 0 \widehat{\xi}_{t,j+1}^{2}=0 ξ t , j + 1 2 = 0 , we set ρ ^ t , j = ∞ \widehat{\rho}_{t,j}=\infty ρ t , j = ∞ if ξ ^ t , j 2 > 0 \widehat{\xi}_{t,j}^{2}>0 ξ t , j 2 > 0 , and ρ ^ t , j = 1 \widehat{\rho}_{t,j}=1 ρ t , j = 1 otherwise. For the oracle projected coordinates, we can show that median t ( ξ t , j 2 / ξ t , j + 1 2 ) = λ j ( Σ x ) / λ j + 1 ( Σ x ) \operatorname{median}_{t}(\xi_{t,j}^{2}/\xi_{t,j+1}^{2})=\lambda_j(\boldsymbol{\Sigma}_x)/\lambda_{j+1}(\boldsymbol{\Sigma}_x) median t ( ξ t , j 2 / ξ t , j + 1 2 ) = λ j ( Σ x ) / λ j + 1 ( Σ x ) for j ∈ [ r max − 1 ] j\in[r_{\max}-1] j ∈ [ r m a x − 1 ] , which follows from the ellipticity of the data (see Lemma 7). From (12) , these population medians are of order p p p at j = r j=r j = r and of order 1 1 1 at all other candidate indices. To account for potential contamination in the data, we compute the sample median of these ratios. across the observations, as
ρ ^ j : = median t ∈ I 2 ρ ^ t , j , j ∈ [ r max − 1 ] , (14) \widehat{\rho}_j := \operatorname{median}_{t \in I_2} \widehat{\rho}_{t,j}, \qquad j \in[r_{\max}-1], \tag*{(14)} ρ j := median t ∈ I 2 ρ t , j , j ∈ [ r m a x − 1 ] , (14)
where we use the convention that this sample median is the ⌈ ∣ I 2 ∣ / 2 ⌉ \lceil|I_2|/2\rceil ⌈ ∣ I 2 ∣/2 ⌉ th smallest ratio. The sample median limits the effect of anomalous projected ratios, and the STAR estimator is set as the index at which this median ratio is maximized:
r ^ : = arg max j ∈ [ r max − 1 ] ρ ^ j . (15) \widehat{r} := \underset{j \in[r_{\max}-1]}{\arg\max}\ \widehat{\rho}_j. \tag*{(15)} r := j ∈ [ r m a x − 1 ] arg max ρ j . (15)
If the maximum is attained at more than one index, we take the smallest such index.
Remark 3 (Computational complexity). For the implementation of the DC estimator, note that the local matrices S ^ i \widehat{S}_i S i in (8) need not be formed explicitly. For a block data matrix of size p × b p \times b p × b , a thin singular value decomposition gives the required b b b left singular vectors in O ( p b 2 ) O(pb^2) O ( p b 2 ) operations. Hence the total cost of the steps in (8) over all m = ⌊ T / b ⌋ m = \lfloor T/b\rfloor m = ⌊ T / b ⌋ blocks is O ( T p b ) O(Tpb) O ( T p b ) . In (9) , constructing P ^ \widehat{P} P costs O ( T p 2 ) O(Tp^2) O ( T p 2 ) operations, and the final eigendecomposition costs O ( p 3 ) O(p^3) O ( p 3 ) operations. Altogether, the computational complexity of our DC estimation is O ( T p b + T p 2 + p 3 ) O(Tpb+Tp^2+p^3) O ( T p b + T p 2 + p 3 ) which, when b = r b=r b = r is constant, is the same as that of conventional PCA estimators which costs O ( T p 2 + p 3 ) O(Tp^2+p^3) O ( T p 2 + p 3 ) .
For STAR, the DC estimation cost is of the same order as above. Computation of the projection scores costs O ( T p r max ) O(Tpr_{\max}) O ( T p r m a x ) operations, and the subsequent ratios and medians can be computed in O ( T r max ) O(Tr_{\max}) O ( T r m a x ) operations. Hence the computational complexity of STAR is the same as that of the full-sample DC estimation.
Theoretical results
For two matrices A 1 A_1 A 1 and A 2 A_2 A 2 with full column rank and the same number of rows, we measure their distance by d ( A 1 , A 2 ) : = ∥ A 1 ( A 1 ⊤ A 1 ) − 1 A 1 ⊤ − A 2 ( A 2 ⊤ A 2 ) − 1 A 2 ⊤ ∥ F d(A_1,A_2) := \left\|A_1(A_1^\top A_1)^{-1}A_1^\top- A_2(A_2^\top A_2)^{-1}A_2^\top\right\|_F d ( A 1 , A 2 ) := A 1 ( A 1 ⊤ A 1 ) − 1 A 1 ⊤ − A 2 ( A 2 ⊤ A 2 ) − 1 A 2 ⊤ F , which can be seen as the distance between the projection spaces of A 1 A_1 A 1 and A 2 A_2 A 2 .
Theorem 1 (Statistical errors of DC). Let Assumptions 1, 2, 3, and 4 hold. Recall that m = ⌊ T / b ⌋ m = \lfloor T/b\rfloor m = ⌊ T / b ⌋ , and suppose that r ≤ b r \le b r ≤ b and b = o ( p min { δ , 1 } ) b = o(p^{\min\{\delta,1\}}) b = o ( p m i n { δ , 1 } ) . Let A ^ b , 1 : r \widehat{A}_{b,1:r} A b , 1 : r denote the first r r r columns of A ^ b \widehat{A}_b A b in (9) . Then there exists a constant C > 0 C > 0 C > 0 such that for all sufficiently large p p p and every
x ∈ ( 0 , 1 ) x \in(0,1) x ∈ ( 0 , 1 ) ,
P ( d ( A ^ b , 1 : r , A ) > C [ p max { 1 − δ , 0 } { min ( b log ( 4 / x ) m , log ( 4 p / x ) m ) + T ‾ m } + p − δ ] ) ≤ x . \mathbb{P}\left(d(\widehat{A}_{b,1:r},A)>C\left[p^{\max\{1-\delta,0\}}\left\{\min\left(\sqrt{\frac{b\log(4/x)}{m}},\sqrt{\frac{\log(4p/x)}{m}}\right)+\frac{\overline{\mathcal{T}}}{m}\right\}+p^{-\delta}\right]\right)\leq x. P ( d ( A b , 1 : r , A ) > C [ p m a x { 1 − δ , 0 } { min ( m b log ( 4/ x ) , m log ( 4 p / x ) ) + m T } + p − δ ] ) ≤ x .
In particular, for any fixed b ≥ r b \geq r b ≥ r , as min ( m , p ) → ∞ \min(m,p)\to\infty min ( m , p ) → ∞ , we have
d ( A ^ b , 1 : r , A ) = O P ( p max { 1 − δ , 0 } { 1 m + T ‾ m } + p − δ ) . d(\widehat{A}_{b,1:r},A)=O_P\left(p^{\max\{1-\delta,0\}}\left\{\frac{1}{\sqrt{m}}+\frac{\overline{\mathcal{T}}}{m}\right\}+p^{-\delta}\right). d ( A b , 1 : r , A ) = O P ( p m a x { 1 − δ , 0 } { m 1 + m T } + p − δ ) .
Note that when b = r b=r b = r , these results directly apply to A ^ r : = A ^ b , 1 : r \widehat{A}_r:=\widehat{A}_{b,1:r} A r := A b , 1 : r .
In Theorem 1, the T ‾ / m \overline{\mathcal{T}}/m T / m term accounts for the cost of anomalies, while a bias-variance type of decomposition is used for analyzing the DC estimator applied to { x t 0 } t ∈ [ T ] \{x_t^0\}_{t\in[T]} { x t 0 } t ∈ [ T ] as in Fan et al. [13 ] , although here we do not require the existence of any positive moment. Notably, if all factors are pervasive and δ = 1 \delta=1 δ = 1 , no anomalies are present, and b b b is fixed, the convergence rate of DC is comparable to that of the conventional PCA estimator attainable under the assumption that the data have four or more moments, which is O P ( 1 / T + 1 / p ) O_P(1/\sqrt{T}+1/p) O P ( 1/ T + 1/ p ) (e.g., Bai and Ng, [3 ] ).
In the scenario where b = r b=r b = r , the rate of convergence in Theorem 1 becomes O P ( p 1 − δ T ‾ / T + p 1 − δ T ‾ / T + p − δ ) O_P(p^{1-\delta}\sqrt{\overline{T}/T}+p^{1-\delta}\overline{\mathcal{T}}/T+p^{-\delta}) O P ( p 1 − δ T / T + p 1 − δ T / T + p − δ ) for 0 < δ < 1 0<\delta<1 0 < δ < 1 , and O P ( 1 / T + T ‾ / T + p − δ ) O_P(1/\sqrt{T}+\overline{\mathcal{T}}/T+p^{-\delta}) O P ( 1/ T + T / T + p − δ ) for δ ≥ 1 \delta\geq1 δ ≥ 1 . Hence for any δ > 0 \delta>0 δ > 0 , a sufficient condition for the consistency of the DC estimator is
max { 1 p , p 2 max { 1 − δ , 0 } T , p max { 1 − δ , 0 } T ‾ T } = o ( 1 ) . (16) \max\left\{\frac{1}{p},\frac{p^{2\max\{1-\delta,0\}}}{T},\frac{p^{\max\{1-\delta,0\}}\overline{\mathcal{T}}}{T}\right\}=o(1). \tag*{(16)} max { p 1 , T p 2 m a x { 1 − δ , 0 } , T p m a x { 1 − δ , 0 } T } = o ( 1 ) . (16)
Next, we make the following assumption for deriving the consistency of the STAR estimator.
Assumption 5. Recall the partition I 1 \mathcal{I}_1 I 1 and I 2 \mathcal{I}_2 I 2 in § 2.3 . We assume the observations { x t } t ∈ I 1 \{x_t\}_{t\in\mathcal{I}_1} { x t } t ∈ I 1 and { x t } t ∈ I 2 ∖ T \{x_t\}_{t\in\mathcal{I}_2\setminus\mathcal{T}} { x t } t ∈ I 2 ∖ T are independent, conditional on T \mathcal{T} T .
Together with Assumption 4, Assumption 5 ensures that the second-half clean observations retain their original distribution conditionally on the first-half data and the anomaly locations. Note that it does not rule out the case where anomalous observations in I 1 \mathcal{I}_1 I 1 depend on the clean observations in I 1 \mathcal{I}_1 I 1 , while imposing no additional restriction on the anomalous observations in I 2 \mathcal{I}_2 I 2 .
Theorem 2 (Consistency of STAR). Let Assumptions 1, 2, 3, 4, 5, and (2.10) hold. Suppose
that r < r max r < r_{\max} r < r m a x and b = r max = o ( p ) b = r_{\max} = o(p) b = r m a x = o ( p ) . Recalling that m = ⌊ T / b ⌋ m = \lfloor T/b \rfloor m = ⌊ T / b ⌋ , we further assume that
p log ( p ) + T ‾ p 1 / 4 = o ( m ) . (17) \sqrt{p}\log(p) + \overline{T}p^{1/4} = o(m). \tag*{(17)} p log ( p ) + T p 1/4 = o ( m ) . (17)
Then as min ( m , p ) → ∞ \min(m,p) \to\infty min ( m , p ) → ∞ , the STAR estimator in (15) is consistent such that P ( r ^ = r ) → 1 \mathbb{P}(\widehat{r}=r) \to1 P ( r = r ) → 1 .
Theorem 2 establishes the consistency of the proposed STAR estimator. Condition (17) ensures separation of the conditional population medians of the clean score ratios and controls the effect of sampling fluctuations and anomalies on their empirical medians. More concretely, for each j ∈ [ r max − 1 ] j \in[r_{\max}-1] j ∈ [ r m a x − 1 ] , consider the median of the conditional distribution of ρ ^ t , j \widehat{\rho}_{t,j} ρ t , j for t ∈ I 2 ∖ T t \in\mathcal{I}_2 \setminus\mathcal{T} t ∈ I 2 ∖ T , given the sigma-algebra σ ( T , { x t } t ∈ I 1 ) \sigma(\mathcal{T}, \{\mathbf{x}_t\}_{t\in\mathcal{I}_1}) σ ( T , { x t } t ∈ I 1 ) . Then with probability approaching one, these conditional medians are uniquely maximized at j = r j=r j = r .
Estimation of the factor structure
When the factor number is unknown, upon the consistent estimation of r r r by the STAR estimator r ^ \widehat{r} r (Theorem 2), we can perform the DC estimation again to all observations, now with block size b = r ^ b=\widehat{r} b = r . Denoting the resulting estimator by A ^ r ^ \widehat{\mathbf{A}}_{\widehat{r}} A r , the following guarantee follows immediately.
Corollary 1 (Convergence rate of DC combined with STAR). Let all conditions in Theorem 2 hold. Then as min ( T , p ) → ∞ \min(T,p) \to\infty min ( T , p ) → ∞ , we have
d ( A ^ r ^ , A ) = O P ( 1 T + 1 p + T ‾ T ) . d(\widehat{\mathbf{A}}_{\widehat{r}},\mathbf{A}) = O_{\mathbb{P}}\left(\frac{1}{\sqrt{T}}+\frac{1}{p}+\frac{\overline{T}}{T}\right). d ( A r , A ) = O P ( T 1 + p 1 + T T ) .
Next, we discuss the estimation of the latent core factor or the factor-driven common component under (1) . For this, we suppose that we directly observe { x t o } t ∈ [ T ] \{\mathbf{x}_t^o\}_{t\in[T]} { x t o } t ∈ [ T ] or, equivalently, that T = ∅ \mathcal{T}=\varnothing T = ∅ . Then with the DC estimator A ^ r ^ \widehat{\mathbf{A}}_{\widehat{r}} A r , for every t ∈ [ T ] t \in[T] t ∈ [ T ] , we can subsequently estimate the core factor f t \mathbf{f}_t f t and thus the common component χ t = A f t \boldsymbol{\chi}_t=\mathbf{A}\mathbf{f}_t χ t = A f t , respectively, as
f ^ t : = 1 p A ^ r ^ ⊤ x t , χ ^ t : = A ^ r ^ A ^ r ^ ⊤ x t , (18) \widehat{\mathbf{f}}_t := \frac{1}{\sqrt{p}}\widehat{\mathbf{A}}_{\widehat{r}}^{\top}\mathbf{x}_t,\qquad\widehat{\boldsymbol{\chi}}_t := \widehat{\mathbf{A}}_{\widehat{r}}\widehat{\mathbf{A}}_{\widehat{r}}^{\top}\mathbf{x}_t, \tag*{(18)} f t := p 1 A r ⊤ x t , χ t := A r A r ⊤ x t , (18)
where χ ^ t \widehat{\boldsymbol{\chi}}_t χ t is essentially the orthogonal projection of x t \mathbf{x}_t x t onto the estimated loading space.
Corollary 2 (Convergence rates for factor and common component estimation). *Suppose that all
the conditions in Theorem 2 hold. Assume T = ∅ \mathcal{T}=\varnothing T = ∅ , and ζ / p = O P ( 1 ) \zeta/\sqrt{p}=\mathcal{O}_{P}(1) ζ / p = O P ( 1 ) as min ( T , p ) → ∞ \min(T,p)\to\infty min ( T , p ) → ∞ . Define the matrix H ^ : = A ^ r ^ ⊤ A / p \widehat{\mathbf{H}}:=\widehat{\mathbf{A}}_{\widehat{r}}^{\top}\mathbf{A}/\sqrt{p} H := A r ⊤ A / p . Then as min ( T , p ) → ∞ \min(T,p)\to\infty min ( T , p ) → ∞ , the singular values of H ^ \widehat{\mathbf{H}} H are bounded away from zero and infinity with probability approaching one, and for every t ∈ [ T ] t\in[T] t ∈ [ T ] ,
∥ f ^ t − H ^ f t ∥ 2 = O P ( 1 T + 1 p ) , 1 p ∥ χ ^ t − χ t ∥ 2 = O P ( 1 T + 1 p ) . \left\|\widehat{\mathbf{f}}_{t}-\widehat{\mathbf{H}}\mathbf{f}_{t}\right\|^{2}=\mathcal{O}_{P}\left(\frac{1}{T}+\frac{1}{p}\right),\qquad\frac{1}{p}\left\|\widehat{\boldsymbol{\chi}}_{t}-\boldsymbol{\chi}_{t}\right\|^{2}=\mathcal{O}_{P}\left(\frac{1}{T}+\frac{1}{p}\right). f t − H f t 2 = O P ( T 1 + p 1 ) , p 1 ∥ χ t − χ t ∥ 2 = O P ( T 1 + p 1 ) .
The additional condition that ζ / p = O P ( 1 ) \zeta/\sqrt{p}=\mathcal{O}_{P}(1) ζ / p = O P ( 1 ) is imposed to control the stochastic scale of individual observations, see also Remark 4. A similar condition is found in Assumption 1 of He et al. (2022), and a stronger version, viz. E [ ζ 2 / p ] = 1 \mathbb{E}[\zeta^{2}/p]=1 E [ ζ 2 / p ] = 1 , is found in Assumption 4.1 of Fan et al. [12 ] .
Remark 4. The condition ζ t / p = O P ( 1 ) \zeta_{t}/\sqrt{p}=\mathcal{O}_{P}(1) ζ t / p = O P ( 1 ) does not require any finite positive moment. For example, let Z t Z_{t} Z t be independent copies of a random variable with density f Z ( z ) = z − 2 1 { z ≥ 1 } f_{Z}(z)=z^{-2}\mathbf{1}_{\{z\geq1\}} f Z ( z ) = z − 2 1 { z ≥ 1 } , and be independent of the direction vectors in Assumption 1, and set ζ t = p exp ( Z t ) \zeta_{t}=\sqrt{p}\exp(Z_{t}) ζ t = p exp ( Z t ) . Then ζ t \zeta_{t} ζ t is finite and strictly positive almost surely as required in Assumption 1. As M → ∞ M\to\infty M → ∞ , we have
sup p ≥ 1 P ( ζ t p > M ) = P ( Z t > log ( M ) ) = 1 log ( M ) → 0 , \sup_{p\geq1}\mathbb{P}\left(\frac{\zeta_{t}}{\sqrt{p}}>M\right)=\mathbb{P}(Z_{t}>\log(M))=\frac{1}{\log(M)}\to0, p ≥ 1 sup P ( p ζ t > M ) = P ( Z t > log ( M )) = log ( M ) 1 → 0 ,
which shows ζ t / p = O P ( 1 ) \zeta_{t}/\sqrt{p}=\mathcal{O}_{P}(1) ζ t / p = O P ( 1 ) . However, for every q > 0 q>0 q > 0 , we have
E [ ( ζ t p ) q ] = ∫ 1 ∞ exp ( q z ) z 2 d z = ∞ . \mathbb{E}\left[\left(\frac{\zeta_{t}}{\sqrt{p}}\right)^{q}\right]=\int_{1}^{\infty}\frac{\exp(qz)}{z^{2}}\,\mathrm{d}z=\infty. E [ ( p ζ t ) q ] = ∫ 1 ∞ z 2 exp ( q z ) d z = ∞.
Numerical experiments
Simulation
We investigate the finite-sample performance of the proposed estimators. Our data generating mechanism is motivated by that of He et al. (2022). In particular, we include a continuous parameter which controls the departure from ellipticity together with an autoregressive coefficient to both factors and noise, so as to assess the estimation performance when the underlying model departs from our assumptions.
Data generating mechanism. Throughout, we fix r = 2 r=2 r = 2 ; other specifications examined in our preliminary experiments result in similar patterns. For factor number estimation by STAR, we set r max = 5 r_{\max}=5 r m a x = 5 and hence b = r max b=r_{\max} b = r m a x . For loading space estimation by DC with the true factor number given, we set b = r = 2 b=r=2 b = r = 2 . We detail the data generation below. First, independently across t t t , generate g t ∼ N ( 0 , I r + p ) \mathbf{g}_t\sim\mathcal{N}(0,\mathbf{I}_{r+p}) g t ∼ N ( 0 , I r + p ) and s t ∈ R r + p \mathbf{s}_t\in\mathbb{R}^{r+p} s t ∈ R r + p whose coordinates are independent copies of E − 1 E-1 E − 1 , where E ∼ Exp ( 1 ) E\sim\operatorname{Exp}(1) E ∼ Exp ( 1 ) . The vectors g t \mathbf{g}_t g t and s t \mathbf{s}_t s t are independent. For some a ∈ [ 0 , 1 ] a\in[0,1] a ∈ [ 0 , 1 ] , define q t : = 1 − a g t + a s t \mathbf{q}_t:=\sqrt{1-a}\mathbf{g}_t+\sqrt{a}\mathbf{s}_t q t := 1 − a g t + a s t and w t : = R t q t \mathbf{w}_t:=R_t\mathbf{q}_t w t := R t q t , where R t R_t R t is independent of these vectors and independent across time, with R t : = ( ν − 2 ) / V t R_t:=\sqrt{(\nu-2)/V_t} R t := ( ν − 2 ) / V t for ν > 2 \nu>2 ν > 2 , R t = ν / V t R_t=\sqrt{\nu/V_t} R t = ν / V t for ν ∈ ( 0 , 2 ] \nu\in(0,2] ν ∈ ( 0 , 2 ] , and V t ∼ χ ν 2 V_t\sim\chi^2_\nu V t ∼ χ ν 2 . The variables V t V_t V t are independent across t t t and independent of { g t , s t } t ∈ [ T ] \{\mathbf{g}_t,\mathbf{s}_t\}_{t\in[T]} { g t , s t } t ∈ [ T ] .
For ν > 2 \nu>2 ν > 2 , this construction gives E [ w t ] = 0 \mathbb{E}[\mathbf{w}_t]=0 E [ w t ] = 0 and E [ w t w t T ] = I r + p \mathbb{E}[\mathbf{w}_t\mathbf{w}_t^{\mathsf{T}}]=\mathbf{I}_{r+p} E [ w t w t T ] = I r + p for every a a a . When a = 0 a=0 a = 0 , the vector w t \mathbf{w}_t w t is spherical with either a covariance-normalized multivariate t ν t_\nu t ν distribution if ν > 2 \nu>2 ν > 2 , or a standard multivariate t ν t_\nu t ν distribution if ν ∈ ( 0 , 2 ] \nu\in(0,2] ν ∈ ( 0 , 2 ] . Note that the latter does not have finite second moments, and even its mean is undefined when ν ≤ 1 \nu\leq1 ν ≤ 1 . Setting a > 0 a>0 a > 0 introduces asymmetry and violates the setup of elliptical factor models. With the above, for ∣ ϕ ∣ < 1 |\phi|<1 ∣ ϕ ∣ < 1 , we generate the AR(1) process h t = ϕ h t − 1 + 1 − ϕ 2 w t \mathbf{h}_t=\phi\mathbf{h}_{t-1}+\sqrt{1-\phi^2}\mathbf{w}_t h t = ϕ h t − 1 + 1 − ϕ 2 w t , where we set h 1 = w 1 \mathbf{h}_1=\mathbf{w}_1 h 1 = w 1 in implementation and only retain observations after a short period of initialization.
The loading matrix is set as A = p Q \mathbf{A}=\sqrt{p}\mathbf{Q} A = p Q , where Q ∈ R p × r \mathbf{Q}\in\mathbb{R}^{p\times r} Q ∈ R p × r is the half-orthogonal matrix from QR decomposition of a p × r p\times r p × r matrix with N ( 0 , 1 ) \mathcal{N}(0,1) N ( 0 , 1 ) entries independent of each other and independent of the innovations. Also, with Σ ϵ = ( 0.3 ∣ j − k ∣ ) j , k ∈ [ p ] \boldsymbol{\Sigma}_{\epsilon}=(0.3^{|j-k|})_{j,k\in[p]} Σ ϵ = ( 0. 3 ∣ j − k ∣ ) j , k ∈ [ p ] , let Σ ϵ = A ϵ A ϵ T \boldsymbol{\Sigma}_{\epsilon}=\mathbf{A}_{\epsilon}\mathbf{A}_{\epsilon}^{\mathsf{T}} Σ ϵ = A ϵ A ϵ T be its Cholesky decomposition with A ϵ \mathbf{A}_{\epsilon} A ϵ lower triangular. Then we partition h t = ( h f , t T , h ϵ , t T ) T \mathbf{h}_t=(\mathbf{h}_{f,t}^{\mathsf{T}},\mathbf{h}_{\epsilon,t}^{\mathsf{T}})^{\mathsf{T}} h t = ( h f , t T , h ϵ , t T ) T with h f , t ∈ R r \mathbf{h}_{f,t}\in\mathbb{R}^r h f , t ∈ R r , and generate the clean observations as
f t = h f , t , ϵ t = A ϵ h ϵ , t , x t 0 = A f t + ϵ t . \mathbf{f}_t=\mathbf{h}_{f,t},\qquad\boldsymbol{\epsilon}_t=\mathbf{A}_{\epsilon}\mathbf{h}_{\epsilon,t},\qquad\mathbf{x}_t^0=\mathbf{A}\mathbf{f}_t+\boldsymbol{\epsilon}_t. f t = h f , t , ϵ t = A ϵ h ϵ , t , x t 0 = A f t + ϵ t .
Note that for ν > 2 \nu>2 ν > 2 and all choices of a a a and ϕ \phi ϕ , we have E [ f t f t T ] = I r \mathbb{E}[\mathbf{f}_t\mathbf{f}_t^{\mathsf{T}}]=\mathbf{I}_r E [ f t f t T ] = I r , E [ ϵ t ϵ t T ] = Σ ϵ \mathbb{E}[\boldsymbol{\epsilon}_t\boldsymbol{\epsilon}_t^{\mathsf{T}}]=\boldsymbol{\Sigma}_{\epsilon} E [ ϵ t ϵ t T ] = Σ ϵ , and E [ f t ϵ t T ] = 0 \mathbb{E}[\mathbf{f}_t\boldsymbol{\epsilon}_t^{\mathsf{T}}]=0 E [ f t ϵ t T ] = 0 . The lag-h h h covariance of the clean observation x t 0 \mathbf{x}_t^0 x t 0 is then ϕ h ( A A T + Σ ϵ ) \phi^h(\mathbf{A}\mathbf{A}^{\mathsf{T}}+\boldsymbol{\Sigma}_{\epsilon}) ϕ h ( A A T + Σ ϵ ) for h ≥ 0 h\geq0 h ≥ 0 . For a = 0 a=0 a = 0 and ϕ = 0 \phi=0 ϕ = 0 , all values ν > 0 \nu>0 ν > 0 yield clean observations satisfying the moment-free elliptical formulation.
To introduce anomalies, we select T ⊂ [ T ] \mathcal{T}\subset[T] T ⊂ [ T ] of size ⌈ ε T ⌉ \lceil\varepsilon T\rceil ⌈ εT ⌉ uniformly and independently of the clean process. For each t ∈ T t\in\mathcal{T} t ∈ T , independently generate z t 0 ∼ N ( 0 , I p ) \mathbf{z}_t^0\sim\mathcal{N}(0,\mathbf{I}_p) z t 0 ∼ N ( 0 , I p ) , set z t = 100 p z t 0 / ∥ z t 0 ∥ \mathbf{z}_t=100\sqrt{p}\mathbf{z}_t^0/\|\mathbf{z}_t^0\| z t = 100 p z t 0 /∥ z t 0 ∥ and generate x t \mathbf{x}_t x t as in (2) , where ε ∈ [ 0 , 1 / 2 ) \varepsilon\in[0,1/2) ε ∈ [ 0 , 1/2 ) controls the fraction of anomalies.
Competing methods. For each competing method below, any tuning parameters are specified as suggested in the corresponding work. For loading space estimation, we provide the true factor number r r r to each method, and compare DC with: (i) the plain principal component estimator (e.g. [3 , 2 ] ), denoted by PCA; (ii) the robust two-step method of He et al. (2022), denoted by RTS; (iii) ROBPCA of [19 ] , which combines projection pursuit with robust scatter matrix estimation; (iv) ROSPCA of [18 ] , which incorporates sparse loading estimation into a robust principal component procedure; (v) MacroPCA of [20 ] , which accommodates cellwise outliers; (vi) ROBTFA of [6 ] , based on entrywise truncation; and (vii) RTFA of [7 ] , based on a Huber loss applied to the normalized residual norm iteratively.
For estimating the factor numbers, we compare STAR with: (i) the adjusted correlation thresholding estimator of [11 ] , denoted by ACT; (ii) the Spearman correlation matrix based estimator of [25 ] , denoted by SR; (iii)–(iv) the multivariate Kendall’s tau eigenvalue ratio and transformed contribution ratio estimators of Yu et al. (2019), denoted by MKER and
MKTCR, respectively; (v) the automatic component selection procedure associated with ROBPCA of Hubert et al. [19 ] ; (vi) the data truncation-based rank estimator of Barigozzi et al. [6 ] , denoted by ROBTFA; and (vii) the Huber loss-based eigenvalue ratio estimator of Barigozzi et al. [7 ] , denoted by RTFA.
Settings and results. To measure the performance of different estimators, we provide the true factor number r r r to each method and report the distance d ( ⋅ , ⋅ ) d(\cdot,\cdot) d ( ⋅ , ⋅ ) introduced earlier for loading space estimation, and we report the absolute error ∣ r ^ − r ∣ |\widehat{r}-r| ∣ r − r ∣ for factor number estimation. Each error is averaged over 100 Monte Carlo replications, and reported with the average computation time in seconds for the corresponding estimation task. To align with the setup in this paper, we first consider a = 0 a=0 a = 0 and ϕ = 0 \phi=0 ϕ = 0 , so that the clean observations are independent across time and follow an elliptical distribution. We vary the degrees of freedom over ν ∈ { 0.5 , 1.5 , 2.5 , 4.5 } \nu\in\{0.5,1.5,2.5,4.5\} ν ∈ { 0.5 , 1.5 , 2.5 , 4.5 } and the fraction of anomalies over ϵ ∈ { 0 , 0.05 , 0.1 , 0.2 } \epsilon\in\{0,0.05,0.1,0.2\} ϵ ∈ { 0 , 0.05 , 0.1 , 0.2 } . The former grid includes the distributions without finite first, second, and fourth moments when ν = 0.5 \nu=0.5 ν = 0.5 , 1.5 1.5 1.5 , and 2.5 2.5 2.5 , respectively.
We present the results for ( T , p ) = ( 100 , 100 ) (T,p)=(100,100) ( T , p ) = ( 100 , 100 ) in Figure 1 ; additional experiments are reported in the supplement (Section C) and exhibit similar patterns. DC maintains small errors which increase only moderately with ϵ \epsilon ϵ and vary little over ν \nu ν . Notably, its errors are either the smallest or comparable to most competing methods such as RTS, ROBPCA, ROSPCA, and MacroPCA. While it is unsurprising that PCA is sensitive to both heavy tails and anomalies, RTFA also deteriorates in the presence of anomalies and ROBTFA exhibits a gradual increase in error as the fraction of anomalies grows. For factor number estimation, STAR, MKER, and MKTCR recover the true factor number in every replication across the settings considered. SR also performs well, with small errors appearing under anomalies. The behavior of ROBTFA and RTFA are akin to their loading space estimation. The performance of ACT is as expected since it requires more than 6th moment (cf. Assumption (b) of Fan et al. [11 ] ). Method ROBPCA (in particular, its rank selection procedure here) performs poorly, as it is only proposed aside its subspace estimation.
Figure 1. Performance of loading space and factor number estimation for ( T , p ) = ( 100 , 100 ) (T,p) = (100,100) ( T , p ) = ( 100 , 100 ) . Within each heatmap, rows correspond to ν ∈ { 0.5 , 1.5 , 2.5 , 4.5 } \nu\in\{0.5,1.5,2.5,4.5\} ν ∈ { 0.5 , 1.5 , 2.5 , 4.5 } and columns correspond to ϵ ∈ { 0 , 0.05 , 0.1 , 0.2 } \epsilon\in\{0,0.05,0.1,0.2\} ϵ ∈ { 0 , 0.05 , 0.1 , 0.2 } . Darker shading indicates larger values within each column.
A salient feature of DC is its computation time, which is the lowest in almost all settings, among all competing methods excluding those designed for light-tailed data (i.e., PCA for loading space estimation and ACT for factor number estimation). Particularly for loading space estimation, while DC is slower but remains comparable to PCA, it is significantly faster than all other methods.
Sensitivity to the block size. We examine the role of the block size and the local retained rank, as discussed in Remark 1. We fix ( T , p ) = ( 100 , 100 ) (T,p)=(100,100) ( T , p ) = ( 100 , 100 ) , r = 2 r=2 r = 2 , a = ϕ = 0 a=\phi=0 a = ϕ = 0 , ϵ = 0 \epsilon=0 ϵ = 0 , and consider ν ∈ { 0.5 , 1.5 , 2.5 , 4.5 } \nu\in\{0.5,1.5,2.5,4.5\} ν ∈ { 0.5 , 1.5 , 2.5 , 4.5 } . For each configuration, we vary the block size over b ∈ { 2 , 4 , 8 , 16 , 32 , 64 , 100 } b \in\{2,4,8,16,32,64,100\} b ∈ { 2 , 4 , 8 , 16 , 32 , 64 , 100 } . We compare the proposed DC construction, which retains the leading b b b eigenvectors of each S ^ i \widehat{S}_i S i in (8) , with an alternative that retains only the leading r r r eigenvectors, irrespective of b b b . Both procedures average the resulting local projectors and retain the leading r r r eigenvectors of the aggregate. The two procedures coincide when b = r b=r b = r , and the alternative would reduce to PCA when b = T b=T b = T .
Figure 2 illustrates the distinction between the two local constructions. For our proposed DC construction, the curves nearly coincide across ν \nu ν , and the errors remain small for b ∈ { 2 , 4 , 8 , 16 , 32 } b \in\{2,4,8,16,32\} b ∈ { 2 , 4 , 8 , 16 , 32 } . Such insensitivity to the tail parameter pinpoints why we should retain the full block span. The sharp increase in the upper panel at b = 64 b=64 b = 64 and b = 100 b=100 b = 100 is due to the number of blocks m = 1 m=1 m = 1 , so that the aggregate is a single rank-b b b projector whose first b b b eigenvalues are all one. Consequently, its leading rank-r r r subspace is not uniquely determined and chosen randomly.
For the alternative construction, retaining only r r r directions when b > r b>r b > r no longer preserves the full block span. Its errors increase markedly with b b b under the heavier-tailed configurations, particularly when ν = 0.5 \nu=0.5 ν = 0.5 and 1.5 1.5 1.5 , whereas the deterioration is much milder when ν = 4.5 \nu=4.5 ν = 4.5 . At b = T b=T b = T , this alternative coincides with PCA, and thus gives performance as expected. Comparing the two panels thus supports the proposed blockwise projector construction for the DC estimator.
Model misspecification. We next investigate sensitivity to non-ellipticity and temporal dependence. We fix ν = 2.5 \nu=2.5 ν = 2.5 and ε = 0.05 \varepsilon=0.05 ε = 0.05 , and consider all combinations of a ∈ { 0 , 0.5 , 1 } a \in\{0,0.5,1\} a ∈ { 0 , 0.5 , 1 } , ϕ ∈ { 0 , 0.25 , 0.5 } \phi\in\{0,0.25,0.5\} ϕ ∈ { 0 , 0.25 , 0.5 } , and ( T , p ) ∈ { ( 50 , 50 ) , ( 100 , 100 ) } (T,p) \in\{(50,50),(100,100)\} ( T , p ) ∈ {( 50 , 50 ) , ( 100 , 100 )} , where a a a controls non-ellipticity and ϕ \phi ϕ controls temporal dependence in the generated data. All remaining parameters follow the preceding experiment. Each setting is repeated 100 times, and the results are summarized in Figure 3 .
Figure 3. Performance of loading space and factor number estimation under non-ellipticity and temporal dependence, with ν = 2.5 \nu= 2.5 ν = 2.5 and ε = 0.05 \varepsilon= 0.05 ε = 0.05 . Within each heatmap, rows correspond to a ∈ { 0 , 0.5 , 1 } a \in\{0, 0.5, 1\} a ∈ { 0 , 0.5 , 1 } and columns correspond to ϕ ∈ { 0 , 0.25 , 0.5 } \phi\in\{0, 0.25, 0.5\} ϕ ∈ { 0 , 0.25 , 0.5 } . Darker shades indicate larger values within each column.
From the figure, DC remains accurate across the parameter grids, and increasing either a a a or ϕ \phi ϕ leads to a moderate increase in its error, while increasing T T T and p p p jointly reduces the error in every corresponding setting. DC and RTS exhibit comparable performance when ϕ = 0 \phi= 0 ϕ = 0 , with RTS attaining slightly smaller errors in most of these settings. However, when ϕ ∈ { 0.25 , 0.5 } \phi\in\{0.25, 0.5\} ϕ ∈ { 0.25 , 0.5 } , DC has smaller errors than RTS throughout the grid, suggesting that the former is less sensitive to temporal dependence in this experiment. The errors of PCA and RTFA remain close to 2, indicating the failure of these methods in these scenarios.
For factor number estimation, STAR remains accurate across all combinations of a a a and ϕ \phi ϕ . At ( T , p ) = ( 50 , 50 ) (T,p) = (50,50) ( T , p ) = ( 50 , 50 ) , its mean absolute rank error is at most 0.050, and all errors are zero at ( T , p ) = ( 100 , 100 ) (T,p) = (100,100) ( T , p ) = ( 100 , 100 ) . MKER and MKTCR exhibit similarly strong performance. Compared to the good performance of SR in Figure 1 , it appears quite sensitive to the non-ellipticity and temporal dependence, which is interesting since this method does not require ellipticity [25 ] . These results provide numerical evidence that DC and STAR remain effective under the departures from ellipticity and temporal independence considered here.
Overall, the experiments numerically show the effectiveness of DC and STAR when compared to a variety of competing methods. Due to space constraints, we defer additional experiments on common component estimation in the supplementary material (Section C).
Real data analysis
In this subsection, we investigate the stability of the proposed methods using the Fama–French portfolio returns. Particularly, we analyze monthly observations from January 1996 to June 2021 for 100 portfolios formed by sorting stocks on size and book-to-market equity, and denote each observation by x t ∈ R p x_t \in\mathbb{R}^{p} x t ∈ R p with p = 100 p = 100 p = 100 and T = 306 T = 306 T = 306 . See the details of the data preparation in Section D of the supplementary material.
We assess the sensitivity of the estimation results to the three periods associated with major market disruptions: (i) October 1999–September 2000 for the dot-com bubble; (ii) April 2008–March 2009 for the collapse of Lehman Brothers; and (iii) October 2019–September 2020 for the onset of COVID-19. For each event, we remove a one-year window and compare the estimates obtained from the remaining observations with those obtained from the complete period. The three windows are removed one at a time, so each reduced sample contains 294 observations. Figure 4 displays the average absolute return and the selected windows, where plot exhibits a pronounced spike around the dot-com bubble, with smaller elevations around the other two events.
Figure 4. Average absolute return, p − 1 ∑ j = 1 p ∣ x t , j ∣ p^{-1}\sum_{j=1}^{p}|x_{t,j}| p − 1 ∑ j = 1 p ∣ x t , j ∣ , for the portfolio data from January 1996 to June 2021. Shaded regions indicate the one-year windows removed in the separate comparisons. Vertical dashed lines mark March 2000, September 2008, and March 2020.
First, we compare loading space estimates at each rank specified from r ∈ { 1 , 2 , 3 , 4 } r \in\{1, 2, 3, 4\} r ∈ { 1 , 2 , 3 , 4 } . For every method and event, we compute the loading space distance d ( ⋅ , ⋅ ) d(\cdot, \cdot) d ( ⋅ , ⋅ ) between the rank-r r r estimate from the complete and reduced samples. Figure 5 shows that DC remains stable overall, with its distances ranging from 0.021 to 0.211, and are either comparable to competing methods or smaller than e.g. those of PCA and RTS in every displayed comparison. This indicates that the loading spaces estimated by DC are less sensitive to the removal of potentially anomalous periods.
Figure 5. Loading space sensitivity to separate exclusions of the three events. Within each panel, rows correspond to r ∈ { 1 , 2 , 3 , 4 } r \in\{1, 2, 3, 4\} r ∈ { 1 , 2 , 3 , 4 } and columns correspond to the eight methods. Darker shading indicates larger values, which represent lower stability.
We next examine factor number estimation, using r max = 5 r_{\max}=5 r m a x = 5 for STAR, so that its candidate factor numbers are 1 , … , 4 1,\ldots,4 1 , … , 4 , and the same for competing methods. The detailed results are relegated to the supplementary material, which show that all the factor number estimators retain their respective estimates when excluding each of the three events. In particular, STAR, SR, MKER, ROBTFA, and RTFA select one factor, whereas ACT, MKTCR, and ROBPCA select more than one factors.
For STAR, we further show in Figure 6 the corresponding ratio criterion, akin to the scree plots for the ratio-based estimators as in e.g., Ahn and Horenstein [1 ] . The diagnostic plots display the median ratios directly, with the maximizing index giving the estimated factor number. The considerable spikes in the first index again highlights the stability of STAR.
Figure 6. Median-ratio criterion of STAR with r max = 5 r_{\max}=5 r m a x = 5 . Each panel compares the complete sample with the sample obtained by removing the indicated event window. The vertical axis shows the median-ratio criterion ρ ^ j \widehat{\rho}_j ρ j in (14) , for j ∈ { 1 , 2 , 3 , 4 } j \in\{1,2,3,4\} j ∈ { 1 , 2 , 3 , 4 } .
Concluding remarks
We have studied loading space and factor number estimation under a moment-free elliptical factor model with anomalies. The DC estimator uses the full column spaces of small blocks before aggregation, and we derive an explicit bound on loading space error that accounts for both sampling fluctuation and contamination. The STAR estimator combines DC with medians of projected coordinate ratios, and is shown to consistently estimate the factor number. The resulting loading space estimator can then be used for factor and common component estimation.
Our numerical experiments suggest that the robustness property (with respect to temporal dependence, non-ellipticity) of DC might be more ample than what we have established in this paper, which merits further investigation. Also, we note that while elliptical distribution is inevitable in the technical arguments, our results may serve as an intermediate step in controlling the rates of performance if one can characterize how far the data distribution is from the set of all elliptical factor models.
Proofs of theorems and auxiliary results
Let the half-orthogonal matrix A ¨ ∈ R p × r \ddot{A} \in\mathbb{R}^{p \times r} A ¨ ∈ R p × r represent an orthonormal basis of span ( A ) \operatorname{span}(A) span ( A ) . Hence throughout this section, for any half-orthogonal matrix A ^ \widehat{A} A , we consider
d ( A ^ , A ) = ∥ A ^ ( A ^ T A ^ ) − 1 A ^ T − A ( A T A ) − 1 A T ∥ F = ∥ A ^ A ^ T − A ¨ A ¨ T ∥ F = d ( A ^ , A ¨ ) . d(\widehat{A}, A) = \left\|\widehat{A}(\widehat{A}^{\mathsf{T}}\widehat{A})^{-1}\widehat{A}^{\mathsf{T}} - A(A^{\mathsf{T}}A)^{-1}A^{\mathsf{T}}\right\|_{F} = \left\|\widehat{A}\widehat{A}^{\mathsf{T}} - \ddot{A}\ddot{A}^{\mathsf{T}}\right\|_{F} = d(\widehat{A}, \ddot{A}). d ( A , A ) = A ( A T A ) − 1 A T − A ( A T A ) − 1 A T F = A A T − A ¨ A ¨ T F = d ( A , A ¨ ) .
Throughout, let Σ x = V x Λ x V x T \Sigma_x = V_x\Lambda_x V_x^{\mathsf{T}} Σ x = V x Λ x V x T denote the eigendecomposition of Σ x \Sigma_x Σ x , where V x V_x V x is orthogonal and Λ x = diag ( λ 1 ( Σ x ) , … , λ p ( Σ x ) ) \Lambda_x = \operatorname{diag}(\lambda_1(\Sigma_x),\ldots,\lambda_p(\Sigma_x)) Λ x = diag ( λ 1 ( Σ x ) , … , λ p ( Σ x )) has its diagonal entries in non-increasing order. Also define A r ∗ : = SVD r ( Σ x ) A_r^* := \operatorname{SVD}_r(\Sigma_x) A r ∗ := SVD r ( Σ x ) and Π r ∗ : = A r ∗ ( A r ∗ ) T \Pi_r^* := A_r^*(A_r^*)^{\mathsf{T}} Π r ∗ := A r ∗ ( A r ∗ ) T . Note that by Assumptions 2 and 3, the matrix Π r ∗ \Pi_r^* Π r ∗ is uniquely defined for all sufficiently large p p p .
Proof of theorems
Proof of Theorem 1. Recall the latent observations x t 0 x_t^0 x t 0 in (1) . For each i ∈ [ m ] i \in[m] i ∈ [ m ] , let P i 0 P_i^0 P i 0 be the orthogonal projector onto the column space of the i i i th block [ x t 0 : t ∈ M i ] [x_t^0 : t \in\mathcal{M}_i] [ x t 0 : t ∈ M i ] , and define P ^ 0 : = m − 1 ∑ i = 1 m P i 0 \widehat{P}^0 := m^{-1}\sum_{i=1}^m P_i^0 P 0 := m − 1 ∑ i = 1 m P i 0 and P ∗ : = E [ P i 0 ] P^* := \mathbb{E}[P_i^0] P ∗ := E [ P i 0 ] . Then by Assumption 1 and Lemma 1, the matrices P i 0 P_i^0 P i 0 , i ∈ [ m ] i \in[m] i ∈ [ m ] , are i.i.d. and have rank b b b almost surely. Hence by Lemma 2, with probability x / 2 x/2 x /2 in each of its two bounds, we obtain
∥ P ^ 0 − P ∗ ∥ ≤ C min { b log ( 4 / x ) m , log ( 4 p / x ) m } . (19) \left\|\widehat{P}^0-P^*\right\| \le C\min\left\{\sqrt{\frac{b\log(4/x)}{m}},\sqrt{\frac{\log(4p/x)}{m}}\right\}. \tag*{(19)} P 0 − P ∗ ≤ C min { m b log ( 4/ x ) , m log ( 4 p / x ) } . (19)
with probability at least 1 − x 1-x 1 − x , where for the second bound in Lemma 2, we used the fact that
b m + b log ( 2 / x ) m = b m { 1 + log ( 2 / x ) } ≤ 2 b log ( 4 / x ) m . \sqrt{\frac{b}{m}}+\sqrt{\frac{b\log(2/x)}{m}}=\sqrt{\frac{b}{m}}\left\{1+\sqrt{\log(2/x)}\right\}\le2\sqrt{\frac{b\log(4/x)}{m}}. m b + m b log ( 2/ x ) = m b { 1 + log ( 2/ x ) } ≤ 2 m b log ( 4/ x ) .
Denote by B : = { i ∈ [ m ] : M i ∩ T ≠ ∅ } \mathcal{B}:=\{i\in[m]:\mathcal{M}_i\cap\mathcal{T}\ne\varnothing\} B := { i ∈ [ m ] : M i ∩ T = ∅ } , and note that ∣ B ∣ ≤ ∣ T ∣ ≤ T |\mathcal{B}|\le|\mathcal{T}|\le\mathcal{T} ∣ B ∣ ≤ ∣ T ∣ ≤ T almost surely. For i ∉ B i\notin\mathcal{B} i ∈ / B , we have U ^ i U ^ i T = P i 0 \widehat{U}_i\widehat{U}_i^{\mathsf{T}}=P_i^0 U i U i T = P i 0 ; for i ∈ B i\in\mathcal{B} i ∈ B , both U ^ i U ^ i T \widehat{U}_i\widehat{U}_i^{\mathsf{T}} U i U i T and P i 0 P_i^0 P i 0 are still orthogonal projectors, so that
− I p ⪯ U ^ i U ^ i T − P i 0 ⪯ I p , ∥ U ^ i U ^ i T − P i 0 ∥ ≤ 1 , -I_p\preceq\widehat{U}_i\widehat{U}_i^{\mathsf{T}}-P_i^0\preceq I_p,\qquad\left\|\widehat{U}_i\widehat{U}_i^{\mathsf{T}}-P_i^0\right\|\le1, − I p ⪯ U i U i T − P i 0 ⪯ I p , U i U i T − P i 0 ≤ 1 ,
which then implies
∥ P ^ − P 0 ∥ ≤ 1 m ∑ i ∈ B ∥ U ^ i U ^ i ⊤ − P i 0 ∥ ≤ ∣ B ∣ m ≤ T ‾ m . (20) \lVert\widehat{\mathbf{P}}-\mathbf{P}^{0}\rVert\le\frac{1}{m}\sum_{i\in\mathcal{B}}\lVert\widehat{\mathbf{U}}_{i}\widehat{\mathbf{U}}_{i}^{\top}-\mathbf{P}_{i}^{0}\rVert\le\frac{|\mathcal{B}|}{m}\le\frac{\overline{T}}{m}. \tag*{(20)} ∥ P − P 0 ∥ ≤ m 1 i ∈ B ∑ ∥ U i U i ⊤ − P i 0 ∥ ≤ m ∣ B ∣ ≤ m T . (20)
From Lemma 4, the matrix P ∗ \mathbf{P}^{*} P ∗ has leading r r r eigenspace projector Π r ∗ \mathbf{\Pi}_{r}^{*} Π r ∗ and eigengap at least ( c ∗ / 2 ) p − max { 1 − δ , 0 } (c_{*}/2)p^{-\max\{1-\delta,0\}} ( c ∗ /2 ) p − m a x { 1 − δ , 0 } for all sufficiently large p p p . Then by the Davis–Kahan theorem (Yu et al., 2015, Theorem 2), we have
d ( A ^ b , 1 : r , A r ∗ ) ≤ 4 2 r c ∗ p max { 1 − δ , 0 } ∥ P ^ − P ∗ ∥ ≤ 4 2 r c ∗ p max { 1 − δ , 0 } { ∥ P 0 − P ∗ ∥ + T ‾ m } . (21) d(\widehat{\mathbf{A}}_{b,1:r},\mathbf{A}_{r}^{*})\le\frac{4\sqrt{2r}}{c_{*}}p^{\max\{1-\delta,0\}}\lVert\widehat{\mathbf{P}}-\mathbf{P}^{*}\rVert\le\frac{4\sqrt{2r}}{c_{*}}p^{\max\{1-\delta,0\}}\left\{\lVert\mathbf{P}^{0}-\mathbf{P}^{*}\rVert+\frac{\overline{T}}{m}\right\}. \tag*{(21)} d ( A b , 1 : r , A r ∗ ) ≤ c ∗ 4 2 r p m a x { 1 − δ , 0 } ∥ P − P ∗ ∥ ≤ c ∗ 4 2 r p m a x { 1 − δ , 0 } { ∥ P 0 − P ∗ ∥ + m T } . (21)
which, combined with ((A.1), Lemma 3, and the triangle inequality, implies the desired high-probability bound in Theorem 1. The desired asymptotic rate of convergence is direct for fixed b b b , and this concludes the proof.
Proof of Theorem 2. If r max = 2 r_{\max}=2 r m a x = 2 , then r = 1 r=1 r = 1 and the maximization in ((15) ) has only one candidate, and the conclusion trivially holds. It hence suffices to consider r max ≥ 3 r_{\max}\ge3 r m a x ≥ 3 . Throughout this proof, r + 1 ≤ r max = b r+1\le r_{\max}=b r + 1 ≤ r m a x = b and b = o ( p ) b=o(p) b = o ( p ) .
Let m 1 : = ⌊ ∣ I 1 ∣ / b ⌋ m_{1}:=\lfloor|\mathcal{I}_{1}|/b\rfloor m 1 := ⌊ ∣ I 1 ∣/ b ⌋ be the number of full blocks in I 1 \mathcal{I}_{1} I 1 , and by construction, m 1 = m / 2 + O ( 1 ) m_{1}=m/2+O(1) m 1 = m /2 + O ( 1 ) .
As ((17) ) implies T ‾ / m → 0 \overline{T}/m\to0 T / m → 0 , we thus have
m 1 − T ‾ ≍ m , m − T ‾ ≍ m , m_{1}-\overline{T}\asymp m,\qquad m-\overline{T}\asymp m, m 1 − T ≍ m , m − T ≍ m ,
and in particular, m 1 > T ‾ m_{1}>\overline{T} m 1 > T for all sufficiently large m , p m,p m , p . Assumption 4 implies ∣ T ∩ I 1 ∣ ≤ T ‾ |\mathcal{T}\cap\mathcal{I}_{1}|\le\overline{T} ∣ T ∩ I 1 ∣ ≤ T almost surely. Hence with the notation r m r_{m} r m from Lemma 5, we have
log ( p ) m 1 − T ‾ + 1 p + T ‾ m 1 = O ( r m ) . \sqrt{\frac{\log(p)}{m_{1}-\overline{T}}+\frac{1}{p}+\frac{\overline{T}}{m_{1}}}=O(r_{m}). m 1 − T log ( p ) + p 1 + m 1 T = O ( r m ) .
Separately, by the definition of r m r_m r m and (17) , we have
p r m 2 ≤ 3 { p log ( p ) m − T ˉ + 1 p 3 / 2 + p ( T ˉ m ) 2 } → 0. (22) \sqrt{p}r_m^2 \le3\left\{\frac{\sqrt{p}\log(p)}{m-\bar{T}}+\frac{1}{p^{3/2}}+\sqrt{p}\left(\frac{\bar{T}}{m}\right)^2\right\}\to0. \tag*{(22)} p r m 2 ≤ 3 { m − T ˉ p log ( p ) + p 3/2 1 + p ( m T ˉ ) 2 } → 0. (22)
For j ∈ [ r max ] j\in[r_{\max}] j ∈ [ r m a x ] , recall b j \mathbf{b}_j b j in Lemma 5, and here, we can write b j = M ⊤ A ^ r max ( 1 ) e j \mathbf{b}_j=\mathbf{M}^{\top}\widehat{\mathbf{A}}^{(1)}_{r_{\max}}\mathbf{e}_j b j = M ⊤ A r m a x ( 1 ) e j . Further define
B p : = max r < j ≤ r max ∥ b j ∥ 2 B_p:=\max_{r<j\le r_{\max}}\|\mathbf{b}_j\|^2 B p := max r < j ≤ r m a x ∥ b j ∥ 2 . As (A.4) implies r m = o ( 1 ) r_m=o(1) r m = o ( 1 ) , we may apply Lemma 5, so that there exist absolute constants c 1 , C 1 > 0 c_1,C_1>0 c 1 , C 1 > 0 such that, with probability approaching 1 1 1 ,
c 1 p ≤ min j ∈ [ r ] ∥ b j ∥ 2 ≤ max j ∈ [ r ] ∥ b j ∥ 2 ≤ C 1 p . (23) c_1p\le\min_{j\in[r]}\|\mathbf{b}_j\|^2\le\max_{j\in[r]}\|\mathbf{b}_j\|^2\le C_1p. \tag*{(23)} c 1 p ≤ j ∈ [ r ] min ∥ b j ∥ 2 ≤ j ∈ [ r ] max ∥ b j ∥ 2 ≤ C 1 p . (23)
The same lemma gives ∥ b j ∥ 2 ≥ c ϵ \|\mathbf{b}_j\|^2\ge c_{\epsilon} ∥ b j ∥ 2 ≥ c ϵ for every j ∈ [ r max ] j\in[r_{\max}] j ∈ [ r m a x ] , and c ϵ ≤ B p ≤ C ϵ + O P ( p r m 2 ) c_{\epsilon}\le B_p\le C_{\epsilon}+O_P(pr_m^2) c ϵ ≤ B p ≤ C ϵ + O P ( p r m 2 ) . Together with (A.4), this implies
B p p ≤ C ϵ p + O P ( p r m 2 ) = o P ( 1 ) . (24) \frac{B_p}{\sqrt{p}}\le\frac{C_{\epsilon}}{\sqrt{p}}+O_P(\sqrt{p}r_m^2)=o_P(1). \tag*{(24)} p B p ≤ p C ϵ + O P ( p r m 2 ) = o P ( 1 ) . (24)
Next, consider the conditional distribution of the clean ratios in I 2 \mathcal{I}_2 I 2 . For clarity, define the sigma-algebra F 1 : = σ ( T , { x t } t ∈ I 1 ) \mathcal{F}_1:=\sigma\left(\mathcal{T},\{\mathbf{x}_t\}_{t\in\mathcal{I}_1}\right) F 1 := σ ( T , { x t } t ∈ I 1 ) . The estimator A ^ r max ( 1 ) \widehat{\mathbf{A}}^{(1)}_{r_{\max}} A r m a x ( 1 ) and hence each b j \mathbf{b}_j b j , is measurable with respect to F 1 \mathcal{F}_1 F 1 . By Assumption 5 and independence of T \mathcal{T} T from the clean data, conditionally on F 1 \mathcal{F}_1 F 1 , the observations { x t } t ∈ I 2 ∖ T \{\mathbf{x}_t\}_{t\in\mathcal{I}_2\setminus\mathcal{T}} { x t } t ∈ I 2 ∖ T remain independent with their original elliptical distribution. Thus their conditional distribution admits the representation in (5) (with independent copies of the original radius and uniform direction). Then noting that P ( ζ > 0 ) = 1 \mathbb{P}(\zeta>0)=1 P ( ζ > 0 ) = 1 by Assumption 1, and that b j ≠ 0 \mathbf{b}_j\ne0 b j = 0 by ∥ b j ∥ 2 ≥ c ϵ \|\mathbf{b}_j\|^2\ge c_{\epsilon} ∥ b j ∥ 2 ≥ c ϵ , so that conditionally on F 1 \mathcal{F}_1 F 1 , the clean denominators in (13) for t ∉ T t\notin\mathcal{T} t ∈ / T are nonzero almost surely. With (11) , the conditional distribution of a clean ratio (i.e., for t ∈ I 2 ∖ T t\in\mathcal{I}_2\setminus\mathcal{T} t ∈ I 2 ∖ T ) is hence
ρ ^ t , j = ζ t 2 ( b j ⊤ v t ) 2 ζ t 2 ( b j + 1 ⊤ v t ) 2 = ( b j ⊤ v t ) 2 ( b j + 1 ⊤ v t ) 2 . \widehat{\rho}_{t,j}=\frac{\zeta_t^2(\mathbf{b}_j^\top\mathbf{v}_t)^2}{\zeta_t^2(\mathbf{b}_{j+1}^\top\mathbf{v}_t)^2}=\frac{(\mathbf{b}_j^\top\mathbf{v}_t)^2}{(\mathbf{b}_{j+1}^\top\mathbf{v}_t)^2}. ρ t , j = ζ t 2 ( b j + 1 ⊤ v t ) 2 ζ t 2 ( b j ⊤ v t ) 2 = ( b j + 1 ⊤ v t ) 2 ( b j ⊤ v t ) 2 .
Let F j F_j F j denote the conditional distribution function of ρ ^ t , j \hat{\rho}_{t,j} ρ ^ t , j for t ∈ I 2 ∖ T t \in\mathcal{I}_2 \setminus\mathcal{T} t ∈ I 2 ∖ T given F 1 \mathcal{F}_1 F 1 . By Lemma 7 , the population median of the conditional distribution F j F_j F j is m j ∗ : = ∥ b j ∥ 2 2 / ∥ b j + 1 ∥ 2 2 m_j^* := \lVert\mathbf{b}_j \rVert_2^2/\lVert\mathbf{b}_{j+1} \rVert_2^2 m j ∗ := ∥ b j ∥ 2 2 / ∥ b j + 1 ∥ 2 2 . Note that on the event that (A.5) holds, we have
m r ∗ ≥ c 1 p B p , (25) m_r^* \ge\frac{c_1p}{B_p}, \tag*{(25)} m r ∗ ≥ B p c 1 p , (25)
and similarly,
m j ∗ ≤ { C 1 / c 1 , j < r , B p / c ε , r < j ≤ r max − 1 , m_j^* \le
\begin{cases}
C_1/c_1, & j < r,\\
B_p/c_\varepsilon, & r < j \le r_{\max}-1,
\end{cases} m j ∗ ≤ { C 1 / c 1 , B p / c ε , j < r , r < j ≤ r m a x − 1 ,
which, together with B p ≥ c ε B_p \ge c_\varepsilon B p ≥ c ε , implies that there exists an absolute constant C 2 > 0 C_2 > 0 C 2 > 0 such that max j ∈ [ r max − 1 ] , j ≠ r m j ∗ ≤ C 2 B p \max_{j \in[r_{\max}-1],\,j \ne r} m_j^* \le C_2B_p max j ∈ [ r m a x − 1 ] , j = r m j ∗ ≤ C 2 B p on the same event. Combining this with (A.6) and (A.7), we obtain
m r ∗ max j ∈ [ r max − 1 ] , j ≠ r m j ∗ ≥ c 1 p C 2 B p 2 → P ∞ . (26) \frac{m_r^*}{\max_{j \in[r_{\max}-1],\,j \ne r} m_j^*}
\ge\frac{c_1p}{C_2B_p^2}
\xrightarrow{P} \infty. \tag*{(26)} max j ∈ [ r m a x − 1 ] , j = r m j ∗ m r ∗ ≥ C 2 B p 2 c 1 p P ∞. (26)
It remains to control the empirical medians, including the effect of anomalies in I 2 \mathcal{I}_2 I 2 . To this end, define ε 2 : = ∣ I 2 ∩ T ∣ / ∣ I 2 ∣ \varepsilon_2 := |\mathcal{I}_2 \cap\mathcal{T}|/|\mathcal{I}_2| ε 2 := ∣ I 2 ∩ T ∣/∣ I 2 ∣ . By Assumption 4, we have
ε 2 ≤ T ‾ ∣ I 2 ∣ ≤ 2 T ‾ T ≤ 2 T ‾ m → 0 , \varepsilon_2 \le\frac{\overline{\mathcal{T}}}{|\mathcal{I}_2|}
\le\frac{2\overline{\mathcal{T}}}{\mathcal{T}}
\le\frac{2\overline{\mathcal{T}}}{m} \to0, ε 2 ≤ ∣ I 2 ∣ T ≤ T 2 T ≤ m 2 T → 0 ,
almost surely, and hence ∣ I 2 ∖ T ∣ ≥ ∣ I 2 ∣ − T ‾ ≍ T |\mathcal{I}_2 \setminus\mathcal{T}| \ge|\mathcal{I}_2|-\overline{\mathcal{T}} \asymp\mathcal{T} ∣ I 2 ∖ T ∣ ≥ ∣ I 2 ∣ − T ≍ T . As log ( p ) = o ( m ) \log(p)=o(m) log ( p ) = o ( m ) by (17) , we also have log ( p ) = o ( ∣ I 2 ∣ − T ‾ ) \log(p)=o(|\mathcal{I}_2|-\overline{\mathcal{T}}) log ( p ) = o ( ∣ I 2 ∣ − T ) . Then for j ∈ [ r max − 1 ] j \in[r_{\max}-1] j ∈ [ r m a x − 1 ] , define the empirical distribution functions
F ^ j c l e a n ( s ) : = 1 ∣ I 2 ∖ T ∣ ∑ t ∈ I 2 ∖ T 1 { ρ ^ t , j ≤ s } , F ^ j ( s ) : = 1 ∣ I 2 ∣ ∑ t ∈ I 2 1 { ρ ^ t , j ≤ s } . \widehat{F}^{\mathrm{clean}}_j(s) := \frac{1}{|\mathcal{I}_2 \setminus\mathcal{T}|}\sum_{t \in\mathcal{I}_2 \setminus\mathcal{T}} \mathbb{1}\{\hat{\rho}_{t,j}\le s\}, \qquad
\widehat{F}_j(s) := \frac{1}{|\mathcal{I}_2|}\sum_{t \in\mathcal{I}_2} \mathbb{1}\{\hat{\rho}_{t,j}\le s\}. F j clean ( s ) := ∣ I 2 ∖ T ∣ 1 t ∈ I 2 ∖ T ∑ 1 { ρ ^ t , j ≤ s } , F j ( s ) := ∣ I 2 ∣ 1 t ∈ I 2 ∑ 1 { ρ ^ t , j ≤ s } .
Conditionally on F 1 \mathcal{F}_1 F 1 , the clean ratios ρ ^ t , j \hat{\rho}_{t,j} ρ ^ t , j for t ∈ I 2 ∖ T t \in\mathcal{I}_2 \setminus\mathcal{T} t ∈ I 2 ∖ T are i.i.d. across t t t , and m j ∗ m_j^* m j ∗ is fixed. Hence by Lemma 7 with δ = 1 / 2 \delta= 1/2 δ = 1/2 and the notation c 0 > 0 c_0 > 0 c 0 > 0 therein, we have
F j ( m j ∗ 2 ) ≤ 1 2 − c 0 2 , F j ( 3 m j ∗ 2 ) ≥ 1 2 + c 0 2 . (27) F_j\left(\frac{m_j^*}{2}\right) \leq\frac{1}{2} - \frac{c_0}{2}, \qquad F_j\left(\frac{3m_j^*}{2}\right) \geq\frac{1}{2} + \frac{c_0}{2}. \tag*{(27)} F j ( 2 m j ∗ ) ≤ 2 1 − 2 c 0 , F j ( 2 3 m j ∗ ) ≥ 2 1 + 2 c 0 . (27)
Fix j ∈ [ r max − 1 ] j \in[r_{\max}-1] j ∈ [ r m a x − 1 ] and a ∈ { 1 / 2 , 3 / 2 } a \in\{1/2,3/2\} a ∈ { 1/2 , 3/2 } . Then conditionally on F 1 \mathcal{F}_1 F 1 , the set I 2 ∖ T \mathcal{I}_2 \setminus\mathcal{T} I 2 ∖ T and the threshold a m j ∗ am_j^* a m j ∗ are fixed, together with the fact that { p ^ t , j } t ∈ I 2 ∖ T \{\widehat{p}_{t,j}\}_{t\in\mathcal{I}_2\setminus\mathcal{T}} { p t , j } t ∈ I 2 ∖ T are i.i.d. under the conditional distribution, it holds that { 1 { p ^ t , j ≤ a m j ∗ } } t ∈ I 2 ∖ T \{\mathbf{1}_{\{\widehat{p}_{t,j}\leq am_j^*\}}\}_{t\in\mathcal{I}_2\setminus\mathcal{T}} { 1 { p t , j ≤ a m j ∗ } } t ∈ I 2 ∖ T are conditionally independent, and satisfy E [ 1 { p ^ t , j ≤ a m j ∗ } ∣ F 1 ] = F j ( a m j ∗ ) \mathbb{E}[\mathbf{1}_{\{\widehat{p}_{t,j}\leq am_j^*\}}\mid\mathcal{F}_1] = F_j(am_j^*) E [ 1 { p t , j ≤ a m j ∗ } ∣ F 1 ] = F j ( a m j ∗ ) . Hence by the Hoeffding’s inequality [30 Theorem 2.2.6] , for every u > 0 u > 0 u > 0 , we have
P ( ∣ F ^ j c l e a n ( a m j ∗ ) − F j ( a m j ∗ ) ∣ > u ∣ F 1 ) = P ( ∣ 1 ∣ I 2 ∖ T ∣ ∑ t ∈ I 2 ∖ T { 1 { p ^ t , j ≤ a m j ∗ } − F j ( a m j ∗ ) } ∣ > u ∣ F 1 ) = P ( ∣ ∑ t ∈ I 2 ∖ T { 1 { p ^ t , j ≤ a m j ∗ } − E [ 1 { p ^ t , j ≤ a m j ∗ } ∣ F 1 ] } ∣ > u ⋅ ∣ I 2 ∖ T ∣ ∣ F 1 ) ≤ 2 exp ( − 2 ∣ I 2 ∖ T ∣ ⋅ u 2 ) , \begin{aligned}
\mathbb{P}\left(\left|\widehat{F}_j^{\mathrm{clean}}(am_j^*)-F_j(am_j^*)\right|>u\mid\mathcal{F}_1\right)
&=\mathbb{P}\left(\left|\frac{1}{|\mathcal{I}_2\setminus\mathcal{T}|}\sum_{t\in\mathcal{I}_2\setminus\mathcal{T}}\left\{\mathbf{1}_{\{\widehat{p}_{t,j}\leq am_j^*\}}-F_j(am_j^*)\right\}\right|>u\mid\mathcal{F}_1\right)\\
&=\mathbb{P}\left(\left|\sum_{t\in\mathcal{I}_2\setminus\mathcal{T}}\left\{\mathbf{1}_{\{\widehat{p}_{t,j}\leq am_j^*\}}-\mathbb{E}\left[\mathbf{1}_{\{\widehat{p}_{t,j}\leq am_j^*\}}\mid\mathcal{F}_1\right]\right\}\right|>u\cdot|\mathcal{I}_2\setminus\mathcal{T}|\mid\mathcal{F}_1\right)\\
&\leq2\exp\left(-2|\mathcal{I}_2\setminus\mathcal{T}|\cdot u^2\right),
\end{aligned} P ( F j clean ( a m j ∗ ) − F j ( a m j ∗ ) > u ∣ F 1 ) = P ∣ I 2 ∖ T ∣ 1 t ∈ I 2 ∖ T ∑ { 1 { p t , j ≤ a m j ∗ } − F j ( a m j ∗ ) } > u ∣ F 1 = P t ∈ I 2 ∖ T ∑ { 1 { p t , j ≤ a m j ∗ } − E [ 1 { p t , j ≤ a m j ∗ } ∣ F 1 ] } > u ⋅ ∣ I 2 ∖ T ∣ ∣ F 1 ≤ 2 exp ( − 2∣ I 2 ∖ T ∣ ⋅ u 2 ) ,
which, by taking u = c 0 / 8 u=c_0/8 u = c 0 /8 and a union bound argument, gives
P ( max j ∈ [ r max − 1 ] max a ∈ { 1 / 2 , 3 / 2 } ∣ F ^ j c l e a n ( a m j ∗ ) − F j ( a m j ∗ ) ∣ > c 0 8 ∣ F 1 ) ≤ ∑ j = 1 r max − 1 ∑ a ∈ { 1 / 2 , 3 / 2 } P ( ∣ F ^ j c l e a n ( a m j ∗ ) − F j ( a m j ∗ ) ∣ > c 0 8 ∣ F 1 ) ≤ 4 ( r max − 1 ) exp ( − ∣ I 2 ∖ T ∣ ⋅ c 0 2 32 ) ≤ 4 p exp ( − ( ∣ I 2 ∣ − T ‾ ) ⋅ c 0 2 32 ) = o ( 1 ) , (28) \begin{aligned}
\mathbb{P}\left(\max_{j\in[r_{\max}-1]}\max_{a\in\{1/2,3/2\}}\left|\widehat{F}_j^{\mathrm{clean}}(am_j^*)-F_j(am_j^*)\right|>\frac{c_0}{8}\mid\mathcal{F}_1\right)
&\leq\sum_{j=1}^{r_{\max}-1}\sum_{a\in\{1/2,3/2\}}\mathbb{P}\left(\left|\widehat{F}_j^{\mathrm{clean}}(am_j^*)-F_j(am_j^*)\right|>\frac{c_0}{8}\mid\mathcal{F}_1\right)\\
&\leq4(r_{\max}-1)\exp\left(-\frac{|\mathcal{I}_2\setminus\mathcal{T}|\cdot c_0^2}{32}\right)\leq4p\exp\left(-\frac{(|\mathcal{I}_2|-\overline{T})\cdot c_0^2}{32}\right)=o(1), \tag*{(28)}
\end{aligned} P ( j ∈ [ r m a x − 1 ] max a ∈ { 1/2 , 3/2 } max F j clean ( a m j ∗ ) − F j ( a m j ∗ ) > 8 c 0 ∣ F 1 ) ≤ j = 1 ∑ r m a x − 1 a ∈ { 1/2 , 3/2 } ∑ P ( F j clean ( a m j ∗ ) − F j ( a m j ∗ ) > 8 c 0 ∣ F 1 ) ≤ 4 ( r m a x − 1 ) exp ( − 32 ∣ I 2 ∖ T ∣ ⋅ c 0 2 ) ≤ 4 p exp ( − 32 ( ∣ I 2 ∣ − T ) ⋅ c 0 2 ) = o ( 1 ) , (28)
uniformly over the conditioning, where the last inequality used ∣ I 2 ∖ T ∣ ≥ ∣ I 2 ∣ − T ‾ |\mathcal{I}_2\setminus\mathcal{T}|\geq|\mathcal{I}_2|-\overline{T} ∣ I 2 ∖ T ∣ ≥ ∣ I 2 ∣ − T and r max = b = o ( p ) r_{\max}=b=o(p) r m a x = b = o ( p ) , and the last equality used log ( p ) = o ( ∣ I 2 ∣ − T ‾ ) \log(p)=o(|\mathcal{I}_2|-\overline{T}) log ( p ) = o ( ∣ I 2 ∣ − T ) by log ( p ) = o ( m ) \log(p)=o(m) log ( p ) = o ( m ) , m ≤ T m\leq T m ≤ T , and T ≲ ∣ I 2 ∣ − T ‾ T\lesssim|\mathcal{I}_2|-\overline{T} T ≲ ∣ I 2 ∣ − T . Note that by taking expectations of the conditional probability bound, (A.10) also holds unconditionally.
For any finite s s s , we have the deterministic inequality
∣ F ^ j ( s ) − F j ( s ) ∣ ≤ ( 1 − ε 2 ) ⋅ ∣ F ^ j c l e a n ( s ) − F j ( s ) ∣ + ε 2 ≤ ∣ F ^ j c l e a n ( s ) − F j ( s ) ∣ + ε 2 . \left|\widehat{F}_j(s)-F_j(s)\right| \le(1-\varepsilon_2)\cdot\left|\widehat{F}_j^{\mathrm{clean}}(s)-F_j(s)\right|+\varepsilon_2 \le\left|\widehat{F}_j^{\mathrm{clean}}(s)-F_j(s)\right|+\varepsilon_2. F j ( s ) − F j ( s ) ≤ ( 1 − ε 2 ) ⋅ F j clean ( s ) − F j ( s ) + ε 2 ≤ F j clean ( s ) − F j ( s ) + ε 2 .
which, noting that ε 2 → 0 \varepsilon_2 \to0 ε 2 → 0 so that eventually ε 2 ≤ c 0 / 8 \varepsilon_2 \le c_0/8 ε 2 ≤ c 0 /8 , together with (A.9) and the unconditional version of (A.10) justified above, it holds with probability approaching 1 that
F ^ j ( m j ∗ 2 ) ≤ F j ( m j ∗ 2 ) + c 0 4 ≤ 1 2 − c 0 4 , F ^ j ( 3 m j ∗ 2 ) ≥ F j ( 3 m j ∗ 2 ) − c 0 4 ≥ 1 2 + c 0 4 , \widehat{F}_j\left(\frac{m_j^*}{2}\right) \le F_j\left(\frac{m_j^*}{2}\right)+\frac{c_0}{4}\le\frac{1}{2}-\frac{c_0}{4},\qquad\widehat{F}_j\left(\frac{3m_j^*}{2}\right) \ge F_j\left(\frac{3m_j^*}{2}\right)-\frac{c_0}{4}\ge\frac{1}{2}+\frac{c_0}{4}, F j ( 2 m j ∗ ) ≤ F j ( 2 m j ∗ ) + 4 c 0 ≤ 2 1 − 4 c 0 , F j ( 2 3 m j ∗ ) ≥ F j ( 2 3 m j ∗ ) − 4 c 0 ≥ 2 1 + 4 c 0 ,
so that with the fact that 1 / 2 − c 0 / 4 < ⌈ ∣ I 2 ∣ / 2 ⌉ / ∣ I 2 ∣ ≤ 1 / 2 + c 0 / 4 1/2-c_0/4<\lceil|\mathcal{I}_2|/2\rceil/|\mathcal{I}_2|\le1/2+c_0/4 1/2 − c 0 /4 < ⌈ ∣ I 2 ∣/2 ⌉ /∣ I 2 ∣ ≤ 1/2 + c 0 /4 for sufficiently large ∣ I 2 ∣ |\mathcal{I}_2| ∣ I 2 ∣ , we obtain
F ^ j ( m j ∗ 2 ) < ⌈ ∣ I 2 ∣ / 2 ⌉ ∣ I 2 ∣ ≤ F ^ j ( 3 m j ∗ 2 ) , (29) \widehat{F}_j\left(\frac{m_j^*}{2}\right)<\frac{\lceil|\mathcal{I}_2|/2\rceil}{|\mathcal{I}_2|}\le\widehat{F}_j\left(\frac{3m_j^*}{2}\right), \tag*{(29)} F j ( 2 m j ∗ ) < ∣ I 2 ∣ ⌈ ∣ I 2 ∣/2 ⌉ ≤ F j ( 2 3 m j ∗ ) , (29)
uniformly for all j ∈ [ r max − 1 ] j\in[r_{\max}-1] j ∈ [ r m a x − 1 ] .
Recall that ρ ^ j \widehat{\rho}_j ρ j in (14) is the ⌈ ∣ I 2 ∣ / 2 ⌉ \lceil|\mathcal{I}_2|/2\rceil ⌈ ∣ I 2 ∣/2 ⌉ th smallest ratio, and also recall the definition that F ^ j ( s ) = ∣ I 2 ∣ − 1 ∑ t ∈ I 2 1 { ρ ^ t , j ≤ s } \widehat{F}_j(s)=|\mathcal{I}_2|^{-1}\sum_{t\in\mathcal{I}_2}\mathbb{1}\{\widehat{\rho}_{t,j}\le s\} F j ( s ) = ∣ I 2 ∣ − 1 ∑ t ∈ I 2 1 { ρ t , j ≤ s } . Together with (A.11), we finally conclude
m j ∗ 2 < ρ ^ j ≤ 3 m j ∗ 2 , j ∈ [ r max − 1 ] , \frac{m_j^*}{2}<\widehat{\rho}_j\le\frac{3m_j^*}{2},\qquad j\in[r_{\max}-1], 2 m j ∗ < ρ j ≤ 2 3 m j ∗ , j ∈ [ r m a x − 1 ] ,
simultaneously with probability approaching 1, which, combined with (A.8), implies that on an event with probability approaching 1,
ρ ^ r max j ∈ [ r max − 1 ] j ≠ r ρ ^ j ≥ 1 3 ⋅ m r ∗ max j ∈ [ r max − 1 ] j ≠ r m j ∗ → P ∞ . \frac{\widehat{\rho}_r}{\max_{\substack{j\in[r_{\max}-1]\\j\ne r}}\widehat{\rho}_j}
\ge\frac{1}{3}\cdot\frac{m_r^*}{\max_{\substack{j\in[r_{\max}-1]\\j\ne r}}m_j^*}
\xrightarrow{P}\infty. max j ∈ [ r m a x − 1 ] j = r ρ j ρ r ≥ 3 1 ⋅ max j ∈ [ r m a x − 1 ] j = r m j ∗ m r ∗ P ∞.
By the definition in (15) , this concludes P ( r ^ = r ) → 1 \mathbb{P}(\widehat{r}=r)\to1 P ( r = r ) → 1 and thus completes this proof.
Auxiliary results with proofs
Lemmas for DC
Lemma 1 (Uniqueness of U ^ i U ^ i T \widehat{\mathbf{U}}_i\widehat{\mathbf{U}}_i^{\mathsf{T}} U i U i T ). Let Assumptions 1, 3, and 4 hold, and suppose 1 ≤ b ≤ p 1 \le b \le p 1 ≤ b ≤ p . Conditional on T \mathcal{T} T , for any clean block i ∈ [ m ] i \in[m] i ∈ [ m ] with M i ∩ T = ∅ \mathcal{M}_i \cap\mathcal{T} = \varnothing M i ∩ T = ∅ (i.e., there are no anomalies in the i i i th interval), the matrix S ^ i \widehat{\mathbf{S}}_i S i in (8) has rank b b b almost surely. Hence its leading b b b eigenspace has a uniquely defined orthogonal projector U ^ i U ^ i T \widehat{\mathbf{U}}_i\widehat{\mathbf{U}}_i^{\mathsf{T}} U i U i T , also almost surely.
Proof of Lemma 1. Condition on T \mathcal{T} T and fix any clean block i ∈ [ m ] i \in[m] i ∈ [ m ] . For simplicity, let M i \mathcal{M}_i M i be explicitly M i = { t 1 , … , t b } \mathcal{M}_i = \{t_1, \ldots, t_b\} M i = { t 1 , … , t b } , and define X i : = [ x t 1 , … , x t b ] ∈ R p × b \mathbf{X}_i := [\mathbf{x}_{t_1}, \ldots, \mathbf{x}_{t_b}] \in\mathbb{R}^{p \times b} X i := [ x t 1 , … , x t b ] ∈ R p × b , Y i : = [ M u t 1 , … , M u t b ] \mathbf{Y}_i := [\mathbf{M}\mathbf{u}_{t_1}, \ldots, \mathbf{M}\mathbf{u}_{t_b}] Y i := [ M u t 1 , … , M u t b ] , and D i : = diag ( ζ t 1 / ∥ u t 1 ∥ , … , ζ t b / ∥ u t b ∥ ) \mathbf{D}_i := \operatorname{diag}(\zeta_{t_1}/\lVert\mathbf{u}_{t_1}\rVert, \ldots, \zeta_{t_b}/\lVert\mathbf{u}_{t_b}\rVert) D i := diag ( ζ t 1 / ∥ u t 1 ∥ , … , ζ t b / ∥ u t b ∥) . Then X i = Y i D i \mathbf{X}_i = \mathbf{Y}_i\mathbf{D}_i X i = Y i D i by (5) , and D i \mathbf{D}_i D i is invertible almost surely by Assumption 1.
As we also have Σ x ⪰ c ϵ I p \boldsymbol{\Sigma}_x \succeq c_\epsilon\mathbf{I}_p Σ x ⪰ c ϵ I p by Assumption 3 and (6) , we may conclude that the columns of Y i \mathbf{Y}_i Y i are independent Gaussian vectors with covariance matrix Σ x \boldsymbol{\Sigma}_x Σ x , and hence these columns are linearly independent almost surely as b ≤ p b \le p b ≤ p . Thus, it holds almost surely that the rank of Y i \mathbf{Y}_i Y i is b b b , and so is the rank of X i \mathbf{X}_i X i and hence that of S ^ i \widehat{\mathbf{S}}_i S i by S ^ i = b − 1 X i X i T \widehat{\mathbf{S}}_i = b^{-1}\mathbf{X}_i\mathbf{X}_i^{\mathsf{T}} S i = b − 1 X i X i T . Namely, it holds almost surely that S ^ i \widehat{\mathbf{S}}_i S i has b b b positive eigenvalues and p − b p-b p − b zero eigenvalues. Hence its leading b b b eigenspace, i.e., U ^ i U ^ i T \widehat{\mathbf{U}}_i\widehat{\mathbf{U}}_i^{\mathsf{T}} U i U i T , is uniquely defined almost surely, which concludes this proof.
Lemma 2 (Concentration of the aggregated projector). Let Assumptions 1 and 3 hold, and suppose there are no anomalies, i.e., T = ∅ \mathcal{T} = \varnothing T = ∅ in (7) . Suppose 1 ≤ b ≤ p 1 \le b \le p 1 ≤ b ≤ p . Then the aggregated projector in (9) satisfies that, for every x ∈ ( 0 , 1 ) x \in(0,1) x ∈ ( 0 , 1 ) ,
P ( ∥ P ^ − E [ P ^ ] ∥ > 4 2 log ( 2 p / x ) m ) ≤ x , \mathbb{P}\left(\left\lVert\widehat{\mathbf{P}}-\mathbb{E}[\widehat{\mathbf{P}}]\right\rVert> 4\sqrt{\frac{2\log(2p/x)}{m}}\right) \le x, P ( P − E [ P ] > 4 m 2 log ( 2 p / x ) ) ≤ x , P ( ∥ P ^ − E [ P ^ ] ∥ > b m + log ( 1 / x ) 2 m ) ≤ x . \mathbb{P}\left(\left\lVert\widehat{\mathbf{P}}-\mathbb{E}[\widehat{\mathbf{P}}]\right\rVert> \sqrt{\frac{b}{m}}+\sqrt{\frac{\log(1/x)}{2m}}\right) \le x. P ( P − E [ P ] > m b + 2 m log ( 1/ x ) ) ≤ x .
Proof of Lemma 2. By Lemma 1, each U ^ i U ^ i ⊤ \widehat{U}_i\widehat{U}_i^{\top} U i U i ⊤ is uniquely defined with probability 1. Hence using notations in (8) , for each block i ∈ [ m ] i \in[m] i ∈ [ m ] , define Π ^ i : = U ^ i U ^ i ⊤ − E [ U ^ i U ^ i ⊤ ] \widehat{\Pi}_i:=\widehat{U}_i\widehat{U}_i^{\top}-\mathbb{E}[\widehat{U}_i\widehat{U}_i^{\top}] Π i := U i U i ⊤ − E [ U i U i ⊤ ] . By Assumption 1, { Π ^ i } i = 1 m \{\widehat{\Pi}_i\}_{i=1}^{m} { Π i } i = 1 m are i.i.d. symmetric random matrices in R p × p \mathbb{R}^{p\times p} R p × p with E [ Π ^ i ] = 0 \mathbb{E}[\widehat{\Pi}_i]=0 E [ Π i ] = 0 . Also, note that by construction and by the triangle inequality, for every i ∈ [ m ] i \in[m] i ∈ [ m ] we have
∥ Π ^ i ∥ ≤ ∥ U ^ i U ^ i ⊤ ∥ + ∥ E [ U ^ i U ^ i ⊤ ] ∥ ≤ 2. \|\widehat{\Pi}_i\| \leq\|\widehat{U}_i\widehat{U}_i^{\top}\|+\|\mathbb{E}[\widehat{U}_i\widehat{U}_i^{\top}]\| \leq2. ∥ Π i ∥ ≤ ∥ U i U i ⊤ ∥ + ∥ E [ U i U i ⊤ ] ∥ ≤ 2.
Hence the matrix Azuma’s inequality (e.g. [29 Theorem 7.1] ) is applicable, which gives that, every u > 0 u>0 u > 0 ,
P ( ∥ 1 m ∑ i = 1 m Π ^ i ∥ ≥ u ) ≤ 2 p exp ( − m u 2 32 ) . (30) \mathbb{P}\left(\left\|\frac{1}{m}\sum_{i=1}^{m}\widehat{\Pi}_i\right\|\geq u\right)\leq2p\exp\left(-\frac{mu^2}{32}\right). \tag*{(30)} P ( m 1 i = 1 ∑ m Π i ≥ u ) ≤ 2 p exp ( − 32 m u 2 ) . (30)
Setting u = 4 2 log ( 2 p / x ) / m u=4\sqrt{2\log(2p/x)/m} u = 4 2 log ( 2 p / x ) / m gives m u 2 / 32 = log ( 2 p / x ) mu^2/32=\log(2p/x) m u 2 /32 = log ( 2 p / x ) , so that the right-hand side of (A.12) is x x x . The proof of the first bound is then complete by noting that by (9) ,
∥ P ^ − E [ P ^ ] ∥ = ∥ 1 m ∑ i = 1 m ( U ^ i U ^ i ⊤ − E [ U ^ i U ^ i ⊤ ] ) ∥ = ∥ 1 m ∑ i = 1 m Π ^ i ∥ . \left\|\widehat{P}-\mathbb{E}[\widehat{P}]\right\|=\left\|\frac{1}{m}\sum_{i=1}^{m}\left(\widehat{U}_i\widehat{U}_i^{\top}-\mathbb{E}[\widehat{U}_i\widehat{U}_i^{\top}]\right)\right\|=\left\|\frac{1}{m}\sum_{i=1}^{m}\widehat{\Pi}_i\right\|. P − E [ P ] = m 1 i = 1 ∑ m ( U i U i ⊤ − E [ U i U i ⊤ ] ) = m 1 i = 1 ∑ m Π i .
For the second bound, similarly by Lemma 1, the matrices U ^ i U ^ i ⊤ \widehat{U}_i\widehat{U}_i^{\top} U i U i ⊤ are independent rank-b b b orthogonal projectors almost surely, with ∥ U ^ i U ^ i ⊤ ∥ F 2 = tr ( U ^ i U ^ i ⊤ ) = b \|\widehat{U}_i\widehat{U}_i^{\top}\|_{F}^{2}=\operatorname{tr}(\widehat{U}_i\widehat{U}_i^{\top})=b ∥ U i U i ⊤ ∥ F 2 = tr ( U i U i ⊤ ) = b . Hence by Jensen’s inequality, we have
E [ ∥ P ^ − E [ P ^ ] ∥ ] ≤ E [ ∥ P ^ − E [ P ^ ] ∥ F ] = E [ ∥ 1 m ∑ i = 1 m ( U ^ i U ^ i ⊤ − E [ U ^ i U ^ i ⊤ ] ) ∥ F ] ≤ { E [ ∥ 1 m ∑ i = 1 m ( U ^ i U ^ i ⊤ − E [ U ^ i U ^ i ⊤ ] ) ∥ F 2 ] } 1 / 2 = { 1 m 2 ∑ i = 1 m E [ ∥ U ^ i U ^ i ⊤ − E [ U ^ i U ^ i ⊤ ] ∥ F 2 ] } 1 / 2 = { 1 m { E [ ∥ U ^ i U ^ i ⊤ ∥ F 2 ] − ∥ E [ U ^ i U ^ i ⊤ ] ∥ F 2 } } 1 / 2 ≤ b m . (31) \begin{aligned}
\mathbb{E}\left[\|\widehat{P}-\mathbb{E}[\widehat{P}]\|\right]\leq\mathbb{E}\left[\|\widehat{P}-\mathbb{E}[\widehat{P}]\|_{F}\right]=\mathbb{E}\left[\left\|\frac{1}{m}\sum_{i=1}^{m}\left(\widehat{U}_i\widehat{U}_i^{\top}-\mathbb{E}[\widehat{U}_i\widehat{U}_i^{\top}]\right)\right\|_{F}\right] \\
&\leq\left\{\mathbb{E}\left[\left\|\frac{1}{m}\sum_{i=1}^{m}\left(\widehat{U}_i\widehat{U}_i^{\top}-\mathbb{E}[\widehat{U}_i\widehat{U}_i^{\top}]\right)\right\|_{F}^{2}\right]\right\}^{1/2}=\left\{\frac{1}{m^{2}}\sum_{i=1}^{m}\mathbb{E}\left[\|\widehat{U}_i\widehat{U}_i^{\top}-\mathbb{E}[\widehat{U}_i\widehat{U}_i^{\top}]\|_{F}^{2}\right]\right\}^{1/2} \\
&=\left\{\frac{1}{m}\left\{\mathbb{E}\left[\|\widehat{U}_i\widehat{U}_i^{\top}\|_{F}^{2}\right]-\|\mathbb{E}[\widehat{U}_i\widehat{U}_i^{\top}]\|_{F}^{2}\right\}\right\}^{1/2}\leq\sqrt{\frac{b}{m}}. \tag*{(31)}
\end{aligned} E [ ∥ P − E [ P ] ∥ ] ≤ E [ ∥ P − E [ P ] ∥ F ] = E [ m 1 i = 1 ∑ m ( U i U i ⊤ − E [ U i U i ⊤ ] ) F ] ≤ ⎩ ⎨ ⎧ E m 1 i = 1 ∑ m ( U i U i ⊤ − E [ U i U i ⊤ ] ) F 2 ⎭ ⎬ ⎫ 1/2 = { m 2 1 i = 1 ∑ m E [ ∥ U i U i ⊤ − E [ U i U i ⊤ ] ∥ F 2 ] } 1/2 = { m 1 { E [ ∥ U i U i ⊤ ∥ F 2 ] − ∥ E [ U i U i ⊤ ] ∥ F 2 } } 1/2 ≤ m b . (31)
Next, consider replacing U ^ i U ^ i T \widehat{U}_i\widehat{U}_i^{\mathsf{T}} U i U i T by another rank-b b b orthogonal projector U ^ i ′ ( U ^ i ′ ) T \widehat{U}'_i(\widehat{U}'_i)^{\mathsf{T}} U i ′ ( U i ′ ) T , and note that
− I p ⪯ U ^ i U ^ i T − U ^ i ′ ( U ^ i ′ ) T ⪯ I p . -I_p \preceq\widehat{U}_i\widehat{U}_i^{\mathsf{T}}-\widehat{U}'_i(\widehat{U}'_i)^{\mathsf{T}} \preceq I_p. − I p ⪯ U i U i T − U i ′ ( U i ′ ) T ⪯ I p .
which implies ∥ U ^ i U ^ i T − U ^ i ′ ( U ^ i ′ ) T ∥ ≤ 1 \left\|\widehat{U}_i\widehat{U}_i^{\mathsf{T}}-\widehat{U}'_i(\widehat{U}'_i)^{\mathsf{T}}\right\|\leq1 U i U i T − U i ′ ( U i ′ ) T ≤ 1 . By the reverse triangle inequality, replacing this block changes the value of ∥ P ^ − E [ P ^ ] ∥ \left\|\widehat{P}-\mathbb{E}[\widehat{P}]\right\| P − E [ P ] by at most 1 / m 1/m 1/ m . Together with the fact that U ^ i U ^ i T \widehat{U}_i\widehat{U}_i^{\mathsf{T}} U i U i T are independent with each other, we may apply McDiarmid’s inequality (Vershynin, [30 ] ), with bounded differences 1 / m 1/m 1/ m for each of the m m m blocks, so that for every u > 0 u>0 u > 0 ,
P ( ∥ P ^ − E [ P ^ ] ∥ − E [ ∥ P ^ − E [ P ^ ] ∥ ] > u ) ≤ exp ( − 2 u 2 ∑ i = 1 m ( 1 / m ) 2 ) = exp ( − 2 m u 2 ) . \mathbb{P}\left(\left\|\widehat{P}-\mathbb{E}[\widehat{P}]\right\|-\mathbb{E}\left[\left\|\widehat{P}-\mathbb{E}[\widehat{P}]\right\|\right]>u\right)\leq\exp\left(-\frac{2u^2}{\sum_{i=1}^{m}(1/m)^2}\right)=\exp(-2mu^2). P ( P − E [ P ] − E [ P − E [ P ] ] > u ) ≤ exp ( − ∑ i = 1 m ( 1/ m ) 2 2 u 2 ) = exp ( − 2 m u 2 ) .
which, combined with (A.13), gives
P ( ∥ P ^ − E [ P ^ ] ∥ > b m + u ) ≤ exp ( − 2 m u 2 ) . \mathbb{P}\left(\left\|\widehat{P}-\mathbb{E}[\widehat{P}]\right\|>\sqrt{\frac{b}{m}}+u\right)\leq\exp(-2mu^2). P ( P − E [ P ] > m b + u ) ≤ exp ( − 2 m u 2 ) .
Setting u = log ( 1 / x ) / ( 2 m ) u=\sqrt{\log(1/x)/(2m)} u = log ( 1/ x ) / ( 2 m ) proves the second bound and thus completes the proof.
Lemma 3. Let Assumptions 1 1 1 , 2 2 2 , and 3 3 3 hold. Recall that A ¨ ∈ R p × r \ddot{A}\in\mathbb{R}^{p\times r} A ¨ ∈ R p × r consists of an orthonormal basis of span ( A ) \operatorname{span}(\mathbf{A}) span ( A ) , and A r ∗ = SVD r ( Σ x ) \mathbf{A}^{*}_{r}=\operatorname{SVD}_{r}(\boldsymbol{\Sigma}_{x}) A r ∗ = SVD r ( Σ x ) . Then for all sufficiently large p p p , we have
d ( A r ∗ , A ¨ ) ≤ 2 2 r C ϵ c λ p δ . d(\mathbf{A}^{*}_{r},\ddot{A})\leq\frac{2\sqrt{2r}C_{\epsilon}}{c\lambda p^{\delta}}. d ( A r ∗ , A ¨ ) ≤ c λ p δ 2 2 r C ϵ .
Proof of Lemma 3. By (6) and Assumption 3 3 3 , we have ∥ Σ x − A A T ∥ = ∥ Σ ϵ ∥ ≤ C ϵ \left\|\boldsymbol{\Sigma}_{x}-\mathbf{A}\mathbf{A}^{\mathsf{T}}\right\|=\left\|\boldsymbol{\Sigma}_{\epsilon}\right\|\leq C_{\epsilon} Σ x − A A T = ∥ Σ ϵ ∥ ≤ C ϵ . Also, the gap between the r r r th and ( r + 1 ) (r+1) ( r + 1 ) th eigenvalues of A A T \mathbf{A}\mathbf{A}^{\mathsf{T}} A A T is λ r ( A T A ) ≥ c λ p δ \lambda_{r}(\mathbf{A}^{\mathsf{T}}\mathbf{A})\geq c\lambda p^{\delta} λ r ( A T A ) ≥ c λ p δ by Assumption 2 2 2 . Hence by the Davis–Kahan theorem (Yu et al., 2015, Theorem 2), we have
d ( A r ∗ , A ¨ ) ≤ 2 2 r ∥ Σ x − A A T ∥ λ r ( A T A ) ≤ 2 2 r C ϵ c λ p δ . d(\mathbf{A}^{*}_{r},\ddot{A})\leq\frac{2\sqrt{2r}\left\|\boldsymbol{\Sigma}_{x}-\mathbf{A}\mathbf{A}^{\mathsf{T}}\right\|}{\lambda_{r}(\mathbf{A}^{\mathsf{T}}\mathbf{A})}\leq\frac{2\sqrt{2r}C_{\epsilon}}{c\lambda p^{\delta}}. d ( A r ∗ , A ¨ ) ≤ λ r ( A T A ) 2 2 r Σ x − A A T ≤ c λ p δ 2 2 r C ϵ .
as desired, which completes this proof.
Lemma 4. Let Assumptions 1, 2, 3, and 4 hold. Suppose r ≤ b r \le b r ≤ b and b = o ( p min { δ , 1 } ) b = o\left(p^{\min\{\delta,1\}}\right) b = o ( p m i n { δ , 1 } ) . Recall V x \mathbf{V}_x V x , Λ x \boldsymbol{\Lambda}_x Λ x , A r ∗ \mathbf{A}_r^* A r ∗ , and Π r ∗ \mathbf{\Pi}_r^* Π r ∗ defined at the beginning of this supplement. Then for every clean block i ∈ [ m ] i \in[m] i ∈ [ m ] (i.e., there are no anomalies in the i i i th block), there exist γ 1 , … , γ p \gamma_1,\ldots,\gamma_p γ 1 , … , γ p such that
E [ U ^ i U ^ i T ∣ T ] = V x diag ( γ 1 , … , γ p ) V x T . \mathbb{E}\left[\widehat{\mathbf{U}}_i\widehat{\mathbf{U}}_i^{\mathsf{T}} \mid\mathcal{T}\right] = \mathbf{V}_x\operatorname{diag}(\gamma_1,\ldots,\gamma_p)\mathbf{V}_x^{\mathsf{T}}. E [ U i U i T ∣ T ] = V x diag ( γ 1 , … , γ p ) V x T . In particular, with d : = b − r + 1 d := b-r+1 d := b − r + 1 , for all sufficiently large p p p , it holds that
γ j ≥ d λ j ( Σ x ) ∑ k = r + 1 p λ k ( Σ x ) + ( d + 2 ) λ j ( Σ x ) , j ∈ [ r ] , \gamma_j \ge\frac{d\lambda_j(\mathbf{\Sigma}_x)}{\sum_{k=r+1}^{p}\lambda_k(\mathbf{\Sigma}_x)+(d+2)\lambda_j(\mathbf{\Sigma}_x)}, \qquad j \in[r], γ j ≥ ∑ k = r + 1 p λ k ( Σ x ) + ( d + 2 ) λ j ( Σ x ) d λ j ( Σ x ) , j ∈ [ r ] , γ j ≤ b λ j ( Σ x ) c ϵ ( p − r ) , r < j ≤ p . (32) \gamma_j \le\frac{b\lambda_j(\mathbf{\Sigma}_x)}{c_{\epsilon}(p-r)}, \qquad r<j\le p. \tag*{(32)} γ j ≤ c ϵ ( p − r ) b λ j ( Σ x ) , r < j ≤ p . (32) Also, with c ∗ : = c λ / ( C ϵ + 3 c λ ) > 0 c_* := c_{\lambda}/(C_{\epsilon}+3c_{\lambda})>0 c ∗ := c λ / ( C ϵ + 3 c λ ) > 0 , it holds that
min j ∈ [ r ] γ j ≥ c ∗ p − max { 1 − δ , 0 } , max r < j ≤ p γ j ≤ b C ϵ c ϵ ( p − r ) , \min_{j\in[r]}\gamma_j \ge c_*p^{-\max\{1-\delta,0\}}, \qquad\max_{r<j\le p}\gamma_j \le\frac{bC_{\epsilon}}{c_{\epsilon}(p-r)}, j ∈ [ r ] min γ j ≥ c ∗ p − m a x { 1 − δ , 0 } , r < j ≤ p max γ j ≤ c ϵ ( p − r ) b C ϵ , and the leading r r r eigenspace of E [ U ^ i U ^ i T ∣ T ] \mathbb{E}\left[\widehat{\mathbf{U}}_i\widehat{\mathbf{U}}_i^{\mathsf{T}}\mid\mathcal{T}\right] E [ U i U i T ∣ T ] has orthogonal projector Π r ∗ \mathbf{\Pi}_r^* Π r ∗ , with the corresponding eigengap λ r ( E [ U ^ i U ^ i T ∣ T ] ) − λ r + 1 ( E [ U ^ i U ^ i T ∣ T ] ) ≥ ( c ∗ / 2 ) p − max { 1 − δ , 0 } \lambda_r\left(\mathbb{E}\left[\widehat{\mathbf{U}}_i\widehat{\mathbf{U}}_i^{\mathsf{T}}\mid\mathcal{T}\right]\right)-\lambda_{r+1}\left(\mathbb{E}\left[\widehat{\mathbf{U}}_i\widehat{\mathbf{U}}_i^{\mathsf{T}}\mid\mathcal{T}\right]\right)\ge(c_*/2)p^{-\max\{1-\delta,0\}} λ r ( E [ U i U i T ∣ T ] ) − λ r + 1 ( E [ U i U i T ∣ T ] ) ≥ ( c ∗ /2 ) p − m a x { 1 − δ , 0 } for all sufficiently large p p p .
Proof of Lemma 4. Condition on T \mathcal{T} T and fix a clean block i ∈ [ m ] i\in[m] i ∈ [ m ] . For t ∈ M i t\in\mathcal{M}_i t ∈ M i , define g t : = Λ x − 1 / 2 V x T M u t \mathbf{g}_t := \boldsymbol{\Lambda}_x^{-1/2}\mathbf{V}_x^{\mathsf{T}}\mathbf{M}\mathbf{u}_t g t := Λ x − 1/2 V x T M u t . Then by (2.6), we can write
E [ g t g t T ] = Λ x − 1 / 2 V x T M M T V x Λ x − 1 / 2 = I p . \mathbb{E}[\mathbf{g}_t\mathbf{g}_t^{\mathsf{T}}] = \boldsymbol{\Lambda}_x^{-1/2}\mathbf{V}_x^{\mathsf{T}}\mathbf{M}\mathbf{M}^{\mathsf{T}}\mathbf{V}_x\boldsymbol{\Lambda}_x^{-1/2} = \mathbf{I}_p. E [ g t g t T ] = Λ x − 1/2 V x T M M T V x Λ x − 1/2 = I p .
Hence the vectors g t \mathbf{g}_t g t are independent standard Gaussian random vectors in R p \mathbb{R}^p R p . Recall the notations M i = { t 1 , … , t b } \mathcal{M}_i=\{t_1,\ldots,t_b\} M i = { t 1 , … , t b } , X i \mathbf{X}_i X i , and D i \mathbf{D}_i D i in the proof of Lemma 1. Also define $\mathbf{G}_i := [\mathbf{g}_{t_1},\ldots,\mathbf{g}_{t_b}] \in R p × b \mathbb{R}^{p\times b} R p × b , and denote its j j j th row as g i , j ⊤ \mathbf{g}_{i,j}^{\top} g i , j ⊤ , where g i , j ∈ R b \mathbf{g}_{i,j} \in\mathbb{R}^{b} g i , j ∈ R b . Then by construction, V x ⊤ X i = Λ x 1 / 2 G i D i \mathbf{V}_{x}^{\top}\mathbf{X}_{i} = \boldsymbol{\Lambda}_{x}^{1/2}\mathbf{G}_{i}\mathbf{D}_{i} V x ⊤ X i = Λ x 1/2 G i D i and since D i \mathbf{D}_{i} D i is invertible almost surely under Assumption 1, by Lemma 1 we have
V x ⊤ U ^ i U ^ i ⊤ V x = Λ x 1 / 2 G i ( G i ⊤ Λ x G i ) − 1 G i ⊤ Λ x 1 / 2 , (33) \mathbf{V}_{x}^{\top}\widehat{\mathbf{U}}_{i}\widehat{\mathbf{U}}_{i}^{\top}\mathbf{V}_{x} = \boldsymbol{\Lambda}_{x}^{1/2}\mathbf{G}_{i}(\mathbf{G}_{i}^{\top}\boldsymbol{\Lambda}_{x}\mathbf{G}_{i})^{-1}\mathbf{G}_{i}^{\top}\boldsymbol{\Lambda}_{x}^{1/2}, \tag*{(33)} V x ⊤ U i U i ⊤ V x = Λ x 1/2 G i ( G i ⊤ Λ x G i ) − 1 G i ⊤ Λ x 1/2 , (33)
where the inverse exists almost surely as G i \mathbf{G}_{i} G i has full column rank and Λ x ⪰ c ϵ I p \boldsymbol{\Lambda}_{x} \succeq c_{\epsilon}\mathbf{I}_{p} Λ x ⪰ c ϵ I p .
For any j ∈ [ p ] j \in[p] j ∈ [ p ] , let D e , j : = I p − 2 e j e j ⊤ \mathbf{D}_{e,j} := \mathbf{I}_{p}-2\mathbf{e}_{j}\mathbf{e}_{j}^{\top} D e , j := I p − 2 e j e j ⊤ , and the matrices D e , j G i \mathbf{D}_{e,j}\mathbf{G}_{i} D e , j G i and G i \mathbf{G}_{i} G i have the same distribution. As D e , j Λ x D e , j = Λ x \mathbf{D}_{e,j}\boldsymbol{\Lambda}_{x}\mathbf{D}_{e,j} = \boldsymbol{\Lambda}_{x} D e , j Λ x D e , j = Λ x , following the proof of Theorem 2 of Fan et al. [13 ] , by (A.15) we have
V x ⊤ E [ U ^ i U ^ i ⊤ ∣ T ] V x = D e , j V x ⊤ E [ U ^ i U ^ i ⊤ ∣ T ] V x D e , j , \mathbf{V}_{x}^{\top}\mathbb{E}[\widehat{\mathbf{U}}_{i}\widehat{\mathbf{U}}_{i}^{\top}\mid\mathcal{T}]\mathbf{V}_{x} = \mathbf{D}_{e,j}\mathbf{V}_{x}^{\top}\mathbb{E}[\widehat{\mathbf{U}}_{i}\widehat{\mathbf{U}}_{i}^{\top}\mid\mathcal{T}]\mathbf{V}_{x}\mathbf{D}_{e,j}, V x ⊤ E [ U i U i ⊤ ∣ T ] V x = D e , j V x ⊤ E [ U i U i ⊤ ∣ T ] V x D e , j ,
which holds for any j ∈ [ p ] j \in[p] j ∈ [ p ] , and hence V x ⊤ E [ U ^ i U ^ i ⊤ ∣ T ] V x \mathbf{V}_{x}^{\top}\mathbb{E}[\widehat{\mathbf{U}}_{i}\widehat{\mathbf{U}}_{i}^{\top}\mid\mathcal{T}]\mathbf{V}_{x} V x ⊤ E [ U i U i ⊤ ∣ T ] V x must be diagonal with E [ U ^ i U ^ i ⊤ ∣ T ] = V x diag ( γ 1 , … , γ p ) V x ⊤ \mathbb{E}[\widehat{\mathbf{U}}_{i}\widehat{\mathbf{U}}_{i}^{\top}\mid\mathcal{T}] = \mathbf{V}_{x}\operatorname{diag}(\gamma_{1},\ldots,\gamma_{p})\mathbf{V}_{x}^{\top} E [ U i U i ⊤ ∣ T ] = V x diag ( γ 1 , … , γ p ) V x ⊤ , where γ j = λ j ( Σ x ) E [ g i , j ⊤ ( G i ⊤ Λ x G i ) − 1 g i , j ] \gamma_{j} = \lambda_{j}(\boldsymbol{\Sigma}_{x})\mathbb{E}[\mathbf{g}_{i,j}^{\top}(\mathbf{G}_{i}^{\top}\boldsymbol{\Lambda}_{x}\mathbf{G}_{i})^{-1}\mathbf{g}_{i,j}] γ j = λ j ( Σ x ) E [ g i , j ⊤ ( G i ⊤ Λ x G i ) − 1 g i , j ] .
Fix j ∈ [ r ] j \in[r] j ∈ [ r ] . Let R j ∈ R b × b \mathbf{R}_{j} \in\mathbb{R}^{b\times b} R j ∈ R b × b be the orthogonal projection matrix onto the orthogonal complement of span { g i , k : k ∈ [ r ] , k ≠ j } \operatorname{span}\{\mathbf{g}_{i,k}: k \in[r], k \ne j\} span { g i , k : k ∈ [ r ] , k = j } , where we set R j = I b \mathbf{R}_{j} = \mathbf{I}_{b} R j = I b in the case of r = 1 r = 1 r = 1 . Then note that R j \mathbf{R}_{j} R j has rank d = b − r + 1 d = b-r+1 d = b − r + 1 , and is independent of g i , j \mathbf{g}_{i,j} g i , j . Define v : = R j g i , j \mathbf{v} := \mathbf{R}_{j}\mathbf{g}_{i,j} v := R j g i , j , s : = g i , j ⊤ R j g i , j s := \mathbf{g}_{i,j}^{\top}\mathbf{R}_{j}\mathbf{g}_{i,j} s := g i , j ⊤ R j g i , j , and H 0 : = ∑ k = r + 1 p λ k ( Σ x ) g i , k g i , k ⊤ \mathbf{H}_{0} := \sum_{k=r+1}^{p}\lambda_{k}(\boldsymbol{\Sigma}_{x})\mathbf{g}_{i,k}\mathbf{g}_{i,k}^{\top} H 0 := ∑ k = r + 1 p λ k ( Σ x ) g i , k g i , k ⊤ . Then conditional on R j \mathbf{R}_{j} R j , the variable s ∼ χ d 2 s \sim\chi^{2}_{d} s ∼ χ d 2 , so that
E [ s ] = d , E [ s 2 ] = d ( d + 2 ) , ∥ v ∥ 2 = s . (34) \mathbb{E}[s] = d,\qquad\mathbb{E}[s^{2}] = d(d+2),\qquad\lVert\mathbf{v}\rVert^{2} = s. \tag*{(34)} E [ s ] = d , E [ s 2 ] = d ( d + 2 ) , ∥ v ∥ 2 = s . (34)
We also have
E [ v ⊤ H 0 v ] = ∑ k = r + 1 p λ k ( Σ x ) ⋅ E [ E { ( g i , k ⊤ v ) 2 ∣ v } ] = ∑ k = r + 1 p λ k ( Σ x ) E [ ∥ v ∥ 2 ] = d ∑ k = r + 1 p λ k ( Σ x ) . (35) \mathbb{E}[\mathbf{v}^{\top}\mathbf{H}_{0}\mathbf{v}] = \sum_{k=r+1}^{p}\lambda_{k}(\boldsymbol{\Sigma}_{x})\cdot\mathbb{E}\left[\mathbb{E}\left\{(\mathbf{g}_{i,k}^{\top}\mathbf{v})^{2}\mid\mathbf{v}\right\}\right] = \sum_{k=r+1}^{p}\lambda_{k}(\boldsymbol{\Sigma}_{x})\mathbb{E}[\lVert\mathbf{v}\rVert^{2}] = d\sum_{k=r+1}^{p}\lambda_{k}(\boldsymbol{\Sigma}_{x}). \tag*{(35)} E [ v ⊤ H 0 v ] = k = r + 1 ∑ p λ k ( Σ x ) ⋅ E [ E { ( g i , k ⊤ v ) 2 ∣ v } ] = k = r + 1 ∑ p λ k ( Σ x ) E [∥ v ∥ 2 ] = d k = r + 1 ∑ p λ k ( Σ x ) . (35)
As G i T Λ x G i \mathbf{G}_i^{\mathsf{T}}\boldsymbol{\Lambda}_x\mathbf{G}_i G i T Λ x G i is positive definite and v \mathbf{v} v is nonzero, by the Cauchy–Schwarz inequality, we have
s 2 = ( g i , j T v ) 2 = { ( ( G i T Λ x G i ) − 1 / 2 g i , j ) T ( ( G i T Λ x G i ) 1 / 2 v ) } 2 ≤ ( g i , j T ( G i T Λ x G i ) − 1 g i , j ) ( v T G i T Λ x G i v ) , \begin{aligned}
s^2 = (\mathbf{g}_{i,j}^{\mathsf{T}}\mathbf{v})^2 = \left\{ \left((\mathbf{G}_i^{\mathsf{T}}\boldsymbol{\Lambda}_x\mathbf{G}_i)^{-1/2}\mathbf{g}_{i,j}\right)^{\mathsf{T}} \left((\mathbf{G}_i^{\mathsf{T}}\boldsymbol{\Lambda}_x\mathbf{G}_i)^{1/2}\mathbf{v}\right) \right\}^2 \\
&\leq\left(\mathbf{g}_{i,j}^{\mathsf{T}}(\mathbf{G}_i^{\mathsf{T}}\boldsymbol{\Lambda}_x\mathbf{G}_i)^{-1}\mathbf{g}_{i,j}\right)\left(\mathbf{v}^{\mathsf{T}}\mathbf{G}_i^{\mathsf{T}}\boldsymbol{\Lambda}_x\mathbf{G}_i\mathbf{v}\right),
\end{aligned} s 2 = ( g i , j T v ) 2 = { ( ( G i T Λ x G i ) − 1/2 g i , j ) T ( ( G i T Λ x G i ) 1/2 v ) } 2 ≤ ( g i , j T ( G i T Λ x G i ) − 1 g i , j ) ( v T G i T Λ x G i v ) ,
which, together with the fact that v T G i T Λ x G i v = λ j ( Σ x ) s 2 + 0 + v T H 0 v \mathbf{v}^{\mathsf{T}}\mathbf{G}_i^{\mathsf{T}}\boldsymbol{\Lambda}_x\mathbf{G}_i\mathbf{v}=\lambda_j(\boldsymbol{\Sigma}_x)s^2+0+\mathbf{v}^{\mathsf{T}}\mathbf{H}_0\mathbf{v} v T G i T Λ x G i v = λ j ( Σ x ) s 2 + 0 + v T H 0 v , implies
λ j ( Σ x ) ⋅ g i , j T ( G i T Λ x G i ) − 1 g i , j ≥ λ j ( Σ x ) ⋅ s 2 λ j ( Σ x ) s 2 + v T H 0 v . \lambda_j(\boldsymbol{\Sigma}_x)\cdot\mathbf{g}_{i,j}^{\mathsf{T}}(\mathbf{G}_i^{\mathsf{T}}\boldsymbol{\Lambda}_x\mathbf{G}_i)^{-1}\mathbf{g}_{i,j}\geq\frac{\lambda_j(\boldsymbol{\Sigma}_x)\cdot s^2}{\lambda_j(\boldsymbol{\Sigma}_x)s^2+\mathbf{v}^{\mathsf{T}}\mathbf{H}_0\mathbf{v}}. λ j ( Σ x ) ⋅ g i , j T ( G i T Λ x G i ) − 1 g i , j ≥ λ j ( Σ x ) s 2 + v T H 0 v λ j ( Σ x ) ⋅ s 2 .
Combining this with (A.16) and (A.17), we obtain
γ j = E [ λ j ( Σ x ) ⋅ g i , j T ( G i T Λ x G i ) − 1 g i , j ] ≥ E [ s 2 s 2 + v T H 0 v / λ j ( Σ x ) ] ≥ { E [ s ] } 2 E [ s 2 + v T H 0 v / λ j ( Σ x ) ] = { E [ s ] } 2 E [ s 2 ] + E [ v T H 0 v ] / λ j ( Σ x ) = d 2 d ( d + 2 ) + d ∑ k = r + 1 p λ k ( Σ x ) / λ j ( Σ x ) = d λ j ( Σ x ) ∑ k = r + 1 p λ k ( Σ x ) + ( d + 2 ) λ j ( Σ x ) . \begin{aligned}
\gamma_j=\mathbb{E}\left[\lambda_j(\boldsymbol{\Sigma}_x)\cdot\mathbf{g}_{i,j}^{\mathsf{T}}(\mathbf{G}_i^{\mathsf{T}}\boldsymbol{\Lambda}_x\mathbf{G}_i)^{-1}\mathbf{g}_{i,j}\right]\geq\mathbb{E}\left[\frac{s^2}{s^2+\mathbf{v}^{\mathsf{T}}\mathbf{H}_0\mathbf{v}/\lambda_j(\boldsymbol{\Sigma}_x)}\right] \\
&\geq\frac{\{\mathbb{E}[s]\}^2}{\mathbb{E}[s^2+\mathbf{v}^{\mathsf{T}}\mathbf{H}_0\mathbf{v}/\lambda_j(\boldsymbol{\Sigma}_x)]}=\frac{\{\mathbb{E}[s]\}^2}{\mathbb{E}[s^2]+\mathbb{E}[\mathbf{v}^{\mathsf{T}}\mathbf{H}_0\mathbf{v}]/\lambda_j(\boldsymbol{\Sigma}_x)} \\
&=\frac{d^2}{d(d+2)+d\sum_{k=r+1}^{p}\lambda_k(\boldsymbol{\Sigma}_x)/\lambda_j(\boldsymbol{\Sigma}_x)}=\frac{d\lambda_j(\boldsymbol{\Sigma}_x)}{\sum_{k=r+1}^{p}\lambda_k(\boldsymbol{\Sigma}_x)+(d+2)\lambda_j(\boldsymbol{\Sigma}_x)}.
\end{aligned} γ j = E [ λ j ( Σ x ) ⋅ g i , j T ( G i T Λ x G i ) − 1 g i , j ] ≥ E [ s 2 + v T H 0 v / λ j ( Σ x ) s 2 ] ≥ E [ s 2 + v T H 0 v / λ j ( Σ x )] { E [ s ] } 2 = E [ s 2 ] + E [ v T H 0 v ] / λ j ( Σ x ) { E [ s ] } 2 = d ( d + 2 ) + d ∑ k = r + 1 p λ k ( Σ x ) / λ j ( Σ x ) d 2 = ∑ k = r + 1 p λ k ( Σ x ) + ( d + 2 ) λ j ( Σ x ) d λ j ( Σ x ) .
which is the desired lower bound in (A.14), where the second inequality used the Cauchy–Schwarz inequality on s = ( s / s 2 + v T H 0 v / λ j ( Σ x ) ) s 2 + v T H 0 v / λ j ( Σ x ) s=(s/\sqrt{s^2+\mathbf{v}^{\mathsf{T}}\mathbf{H}_0\mathbf{v}/\lambda_j(\boldsymbol{\Sigma}_x)})\sqrt{s^2+\mathbf{v}^{\mathsf{T}}\mathbf{H}_0\mathbf{v}/\lambda_j(\boldsymbol{\Sigma}_x)} s = ( s / s 2 + v T H 0 v / λ j ( Σ x ) ) s 2 + v T H 0 v / λ j ( Σ x ) .
For the upper bound in (A.14), let G i , 0 ∈ R ( p − r ) × b \mathbf{G}_{i,0}\in\mathbb{R}^{(p-r)\times b} G i , 0 ∈ R ( p − r ) × b denote the sub-matrix containing rows r + 1 , … , p r+1,\ldots,p r + 1 , … , p of G i \mathbf{G}_i G i . As b = o ( p min { δ , 1 } ) b=o(p^{\min\{\delta,1\}}) b = o ( p m i n { δ , 1 } ) , we have b ≤ p − r b\leq p-r b ≤ p − r for all sufficiently large p p p . Hence G i , 0 T G i , 0 \mathbf{G}_{i,0}^{\mathsf{T}}\mathbf{G}_{i,0} G i , 0 T G i , 0 is invertible almost surely, and
G i T Λ x G i ⪰ c ϵ G i , 0 T G i , 0 , ( G i T Λ x G i ) − 1 ⪯ c ϵ − 1 ( G i , 0 T G i , 0 ) − 1 . \mathbf{G}_i^{\mathsf{T}}\boldsymbol{\Lambda}_x\mathbf{G}_i\succeq c_{\epsilon}\mathbf{G}_{i,0}^{\mathsf{T}}\mathbf{G}_{i,0},\qquad
(\mathbf{G}_i^{\mathsf{T}}\boldsymbol{\Lambda}_x\mathbf{G}_i)^{-1}\preceq c_{\epsilon}^{-1}(\mathbf{G}_{i,0}^{\mathsf{T}}\mathbf{G}_{i,0})^{-1}. G i T Λ x G i ⪰ c ϵ G i , 0 T G i , 0 , ( G i T Λ x G i ) − 1 ⪯ c ϵ − 1 ( G i , 0 T G i , 0 ) − 1 .
Note that the matrix G i , 0 ( G i , 0 T G i , 0 ) − 1 G i , 0 T \mathbf{G}_{i,0}(\mathbf{G}_{i,0}^{\mathsf{T}}\mathbf{G}_{i,0})^{-1}\mathbf{G}_{i,0}^{\mathsf{T}} G i , 0 ( G i , 0 T G i , 0 ) − 1 G i , 0 T is a rank-b b b projection matrix with exchangeable diagonal entries and trace exactly b b b , so that each of its diagonal entries has expectation b / ( p − r ) b/(p-r) b / ( p − r ) .
Hence for j > r j>r j > r , we have
γ j ≤ λ j ( Σ x ) c ϵ E [ g i , j T ( G i , 0 T G i , 0 ) − 1 g i , j ] = b λ j ( Σ x ) c ϵ ( p − r ) . \gamma_j \le\frac{\lambda_j(\boldsymbol{\Sigma}_x)}{c_\epsilon}\mathbb{E}\left[\mathbf{g}_{i,j}^{\mathsf T}(\mathbf{G}_{i,0}^{\mathsf T}\mathbf{G}_{i,0})^{-1}\mathbf{g}_{i,j}\right] = \frac{b\lambda_j(\boldsymbol{\Sigma}_x)}{c_\epsilon(p-r)}. γ j ≤ c ϵ λ j ( Σ x ) E [ g i , j T ( G i , 0 T G i , 0 ) − 1 g i , j ] = c ϵ ( p − r ) b λ j ( Σ x ) .
which is the desired result in (A.14).
Finally, for j ≤ r j\le r j ≤ r , since d ≥ 1 d\ge1 d ≥ 1 and ∑ k = r + 1 p λ k ( Σ x ) ≥ 0 \sum_{k=r+1}^{p}\lambda_k(\boldsymbol{\Sigma}_x)\ge0 ∑ k = r + 1 p λ k ( Σ x ) ≥ 0 , we have
γ j ≥ λ j ( Σ x ) ∑ k = r + 1 p λ k ( Σ x ) + 3 λ j ( Σ x ) ≥ c λ p δ C ϵ p + 3 c λ p δ ≥ c λ C ϵ + 3 c λ min { 1 , p δ − 1 } = c ∗ p − max { 1 − δ , 0 } . \gamma_j \ge\frac{\lambda_j(\boldsymbol{\Sigma}_x)}{\sum_{k=r+1}^{p}\lambda_k(\boldsymbol{\Sigma}_x)+3\lambda_j(\boldsymbol{\Sigma}_x)} \ge\frac{c_\lambda p^\delta}{C_\epsilon p+3c_\lambda p^\delta} \ge\frac{c_\lambda}{C_\epsilon+3c_\lambda}\min\{1,p^{\delta-1}\}=c_*p^{-\max\{1-\delta,0\}}. γ j ≥ ∑ k = r + 1 p λ k ( Σ x ) + 3 λ j ( Σ x ) λ j ( Σ x ) ≥ C ϵ p + 3 c λ p δ c λ p δ ≥ C ϵ + 3 c λ c λ min { 1 , p δ − 1 } = c ∗ p − m a x { 1 − δ , 0 } .
whereas for j > r j>r j > r , the upper bound in (A.14) and λ j ( Σ x ) ≤ C ϵ \lambda_j(\boldsymbol{\Sigma}_x)\le C_\epsilon λ j ( Σ x ) ≤ C ϵ apply. Moreover, we have
b / ( p − r ) p − max { 1 − δ , 0 } ≲ b p min { δ , 1 } → 0 , \frac{b/(p-r)}{p^{-\max\{1-\delta,0\}}}\lesssim\frac{b}{p^{\min\{\delta,1\}}}\to0, p − m a x { 1 − δ , 0 } b / ( p − r ) ≲ p m i n { δ , 1 } b → 0 ,
so that min j ∈ [ r ] γ j − max r < j ≤ p γ j ≥ ( c ∗ / 2 ) p − max { 1 − δ , 0 } \min_{j\in[r]}\gamma_j-\max_{r<j\le p}\gamma_j\ge(c_*/2)p^{-\max\{1-\delta,0\}} min j ∈ [ r ] γ j − max r < j ≤ p γ j ≥ ( c ∗ /2 ) p − m a x { 1 − δ , 0 } for all sufficiently large p p p , which thus concludes the proof. □
Lemmas for STAR
Lemma 5. Let Assumptions 1, 2, 3, 4, and {(2.10)} hold. Suppose r < r max = b r<r_{\max}=b r < r m a x = b and b = o ( p ) b=o(p) b = o ( p ) . Recalling A ^ b \widehat{\mathbf{A}}_b A b from {(2.9)}, denote by A ^ r max : = A ^ b \widehat{\mathbf{A}}_{r_{\max}}:=\widehat{\mathbf{A}}_b A r m a x := A b for the choice of b = r max b=r_{\max} b = r m a x , and let A ^ r max , 1 : r \widehat{\mathbf{A}}_{r_{\max},1:r} A r m a x , 1 : r denote its first r r r columns. For each j ∈ [ r max ] j\in[r_{\max}] j ∈ [ r m a x ] , let e j \mathbf{e}_j e j denote the j j j th standard basis vector in R r max \mathbb{R}^{r_{\max}} R r m a x , and define
b j : = M T A ^ r max e j ∈ R r + p , r m : = log ( p ) m − τ ‾ + 1 p + τ ‾ m . \mathbf{b}_j:=\mathbf{M}^{\mathsf T}\widehat{\mathbf{A}}_{r_{\max}}\mathbf{e}_j\in\mathbb{R}^{r+p},\qquad r_m:=\sqrt{\frac{\log(p)}{m-\overline{\tau}}+\frac{1}{p}+\frac{\overline{\tau}}{m}}. b j := M T A r m a x e j ∈ R r + p , r m := m − τ log ( p ) + p 1 + m τ .
Also suppose min ( m , p ) → ∞ \min(m,p) \to\infty min ( m , p ) → ∞ , m > T ‾ m > \overline{T} m > T , and r m = o ( 1 ) r_m = o(1) r m = o ( 1 ) . Then there exist absolute constants c 1 , C 1 > 0 c_1,C_1 > 0 c 1 , C 1 > 0 such that
P ( c 1 p ≤ min j ∈ [ r ] ∥ b j ∥ 2 ≤ max j ∈ [ r ] ∥ b j ∥ 2 ≤ C 1 p ) → 1. \mathbb{P}\left(c_1p \le\min_{j \in[r]}\|\mathbf{b}_j\|^2 \le\max_{j \in[r]}\|\mathbf{b}_j\|^2 \le C_1p\right) \to1. P ( c 1 p ≤ j ∈ [ r ] min ∥ b j ∥ 2 ≤ j ∈ [ r ] max ∥ b j ∥ 2 ≤ C 1 p ) → 1.
Moreover, with the notations in Assumption 3, ∥ b j ∥ 2 ≥ c ϵ \|\mathbf{b}_j\|^2 \ge c_{\epsilon} ∥ b j ∥ 2 ≥ c ϵ for every j ∈ [ r max ] j \in[r_{\max}] j ∈ [ r m a x ] , and if r max > r r_{\max} > r r m a x > r ,
max r < j ≤ r max ∥ b j ∥ 2 ≤ C ϵ + O P ( p r m 2 ) . \max_{r<j\le r_{\max}}\|\mathbf{b}_j\|^2 \le C_{\epsilon} + O_P(pr_m^2). r < j ≤ r m a x max ∥ b j ∥ 2 ≤ C ϵ + O P ( p r m 2 ) .
Proof of Lemma 5. Under (10) , Assumption 2 holds with δ = 1 \delta= 1 δ = 1 . Using (A.3) with b = r max b = r_{\max} b = r m a x and Lemma 2, we have
∥ A ^ r max , 1 : r A ^ r max , 1 : r T − Π r ∗ ∥ F = O P ( log ( p ) m + T ‾ m ) = O P ( r m ) , \left\|\widehat{\mathbf{A}}_{r_{\max},1:r}\widehat{\mathbf{A}}_{r_{\max},1:r}^{\mathsf{T}}-\mathbf{\Pi}^{*}_{r}\right\|_{F}
= O_P\left(\sqrt{\frac{\log(p)}{m}}+\frac{\overline{T}}{m}\right)
= O_P(r_m), A r m a x , 1 : r A r m a x , 1 : r T − Π r ∗ F = O P ( m log ( p ) + m T ) = O P ( r m ) ,
where the last equality used m > T ‾ m > \overline{T} m > T . Hence
∥ A ^ r max , 1 : r A ^ r max , 1 : r T − Π r ∗ ∥ F = O P ( r m ) . (36) \left\|\widehat{\mathbf{A}}_{r_{\max},1:r}\widehat{\mathbf{A}}_{r_{\max},1:r}^{\mathsf{T}}-\mathbf{\Pi}^{*}_{r}\right\|_{F}=O_P(r_m). \tag*{(36)} A r m a x , 1 : r A r m a x , 1 : r T − Π r ∗ F = O P ( r m ) . (36)
Note that by (6) and the definition of b j \mathbf{b}_j b j , we can write ∥ b j ∥ 2 = ( A ^ r max e j ) T Σ x ( A ^ r max e j ) \|\mathbf{b}_j\|^2 = (\widehat{\mathbf{A}}_{r_{\max}}\mathbf{e}_j)^{\mathsf{T}}\boldsymbol{\Sigma}_x(\widehat{\mathbf{A}}_{r_{\max}}\mathbf{e}_j) ∥ b j ∥ 2 = ( A r m a x e j ) T Σ x ( A r m a x e j ) . Since ∥ A ^ r max e j ∥ = 1 \|\widehat{\mathbf{A}}_{r_{\max}}\mathbf{e}_j\|=1 ∥ A r m a x e j ∥ = 1 and Σ x ⪰ c ϵ I p \boldsymbol{\Sigma}_x \succeq c_{\epsilon}\mathbf{I}_p Σ x ⪰ c ϵ I p by Assumption 3, we arrive at ∥ b j ∥ 2 ≥ c ϵ \|\mathbf{b}_j\|^2 \ge c_{\epsilon} ∥ b j ∥ 2 ≥ c ϵ for every j ∈ [ r max ] j \in[r_{\max}] j ∈ [ r m a x ] .
Consider j ≤ r j \le r j ≤ r . Since A ^ r max e j \widehat{\mathbf{A}}_{r_{\max}}\mathbf{e}_j A r m a x e j is in the column space of A ^ r max , 1 : r \widehat{\mathbf{A}}_{r_{\max},1:r} A r m a x , 1 : r , we have
∥ ( I p − Π r ∗ ) A ^ r max e j ∥ = ∥ ( A ^ r max , 1 : r A ^ r max , 1 : r T − Π r ∗ ) A ^ r max e j ∥ ≤ ∥ A ^ r max , 1 : r A ^ r max , 1 : r T − Π r ∗ ∥ F = O P ( r m ) , \begin{aligned}
\left\|(\mathbf{I}_p-\mathbf{\Pi}^{*}_r)\widehat{\mathbf{A}}_{r_{\max}}\mathbf{e}_j\right\|
= \left\|\left(\widehat{\mathbf{A}}_{r_{\max},1:r}\widehat{\mathbf{A}}_{r_{\max},1:r}^{\mathsf{T}}-\mathbf{\Pi}^{*}_r\right)\widehat{\mathbf{A}}_{r_{\max}}\mathbf{e}_j\right\| \\
&\le\left\|\widehat{\mathbf{A}}_{r_{\max},1:r}\widehat{\mathbf{A}}_{r_{\max},1:r}^{\mathsf{T}}-\mathbf{\Pi}^{*}_r\right\|_{F}=O_P(r_m),
\end{aligned} ( I p − Π r ∗ ) A r m a x e j = ( A r m a x , 1 : r A r m a x , 1 : r T − Π r ∗ ) A r m a x e j ≤ A r m a x , 1 : r A r m a x , 1 : r T − Π r ∗ F = O P ( r m ) ,
uniformly over j ∈ [ r ] j \in[r] j ∈ [ r ] , where the last equality used (A.18). Hence with the fact that $\mathbf{\Pi}^{*}_r\widehat{\mathbf{A}}_{r_{\max}}\mathbf{e}_j is orthogonal to ( I p − Π r ∗ ) A ^ r max e j (I_p-\Pi_r^*)\widehat{A}_{r_{\max}}e_j ( I p − Π r ∗ ) A r m a x e j , we obtain
min j ∈ [ r ] ∥ Π r ∗ A ^ r max e j ∥ 2 = min j ∈ [ r ] { ∥ A ^ r max e j ∥ 2 − ∥ ( I p − Π r ∗ ) A ^ r max e j ∥ 2 } ≥ 1 − O P ( r m 2 ) . (37) \min_{j\in[r]}\left\|\Pi_r^*\widehat{A}_{r_{\max}}e_j\right\|^2=\min_{j\in[r]}\left\{\left\|\widehat{A}_{r_{\max}}e_j\right\|^2-\left\|(I_p-\Pi_r^*)\widehat{A}_{r_{\max}}e_j\right\|^2\right\}\geq1-O_P(r_m^2). \tag*{(37)} j ∈ [ r ] min Π r ∗ A r m a x e j 2 = j ∈ [ r ] min { A r m a x e j 2 − ( I p − Π r ∗ ) A r m a x e j 2 } ≥ 1 − O P ( r m 2 ) . (37)
Note that it is direct that max j ∈ [ r ] ∥ Π r ∗ A ^ r max e j ∥ 2 ≤ 1 \max_{j\in[r]}\left\|\Pi_r^*\widehat{A}_{r_{\max}}e_j\right\|^2\leq1 max j ∈ [ r ] Π r ∗ A r m a x e j 2 ≤ 1 .
By (10) and Assumption 3, we have λ r ( Σ x ) ≥ c λ p \lambda_r(\Sigma_x)\geq c\lambda p λ r ( Σ x ) ≥ c λ p and λ 1 ( Σ x ) ≤ C λ p + C ϵ \lambda_1(\Sigma_x)\leq C\lambda p+C\epsilon λ 1 ( Σ x ) ≤ C λ p + C ϵ . Then by (A.19), we have
min j ∈ [ r ] ∥ b j ∥ 2 ≥ λ r ( Σ x ) min j ∈ [ r ] ∥ Π r ∗ A ^ r max e j ∥ 2 ≥ c λ p { 1 − O P ( r m 2 ) } , \min_{j\in[r]}\|b_j\|^2\geq\lambda_r(\Sigma_x)\min_{j\in[r]}\left\|\Pi_r^*\widehat{A}_{r_{\max}}e_j\right\|^2\geq c\lambda p\{1-O_P(r_m^2)\}, j ∈ [ r ] min ∥ b j ∥ 2 ≥ λ r ( Σ x ) j ∈ [ r ] min Π r ∗ A r m a x e j 2 ≥ c λ p { 1 − O P ( r m 2 )} ,
and also max j ∈ [ r ] ∥ b j ∥ 2 ≤ λ 1 ( Σ x ) ≤ C λ p + C ϵ \max_{j\in[r]}\|b_j\|^2\leq\lambda_1(\Sigma_x)\leq C\lambda p+C\epsilon max j ∈ [ r ] ∥ b j ∥ 2 ≤ λ 1 ( Σ x ) ≤ C λ p + C ϵ . Together with r m = o ( 1 ) r_m=o(1) r m = o ( 1 ) , these conclude the proof for the results for j ≤ r j\leq r j ≤ r .
It remains to show the upper bound for j > r j>r j > r (the lower bound has been established at the beginning of this proof). First, we decompose
A ^ r max e j = Π r ∗ A ^ r max e j + ( I p − Π r ∗ ) A ^ r max e j = : s j + n j , \widehat{A}_{r_{\max}}e_j=\Pi_r^*\widehat{A}_{r_{\max}}e_j+(I_p-\Pi_r^*)\widehat{A}_{r_{\max}}e_j=:s_j+n_j, A r m a x e j = Π r ∗ A r m a x e j + ( I p − Π r ∗ ) A r m a x e j =: s j + n j ,
where, note that immediately, s j s_j s j is orthogonal to n j n_j n j , and hence 1 = ∥ A ^ r max e j ∥ 2 = ∥ s j ∥ 2 + ∥ n j ∥ 2 1=\|\widehat{A}_{r_{\max}}e_j\|^2=\|s_j\|^2+\|n_j\|^2 1 = ∥ A r m a x e j ∥ 2 = ∥ s j ∥ 2 + ∥ n j ∥ 2 .
For s j s_j s j , noting that the vector A ^ r max e j \widehat{A}_{r_{\max}}e_j A r m a x e j is orthogonal to A ^ r max , 1 : r \widehat{A}_{r_{\max},1:r} A r m a x , 1 : r by construction, we have
∥ s j ∥ = ∥ Π r ∗ A ^ r max e j ∥ = ∥ ( Π r ∗ − A ^ r max , 1 : r A ^ r max , 1 : r T ) A ^ r max e j ∥ = O P ( r m ) , (38) \|s_j\|=\|\Pi_r^*\widehat{A}_{r_{\max}}e_j\|=\left\|\left(\Pi_r^*-\widehat{A}_{r_{\max},1:r}\widehat{A}_{r_{\max},1:r}^{\mathsf{T}}\right)\widehat{A}_{r_{\max}}e_j\right\|=O_P(r_m), \tag*{(38)} ∥ s j ∥ = ∥ Π r ∗ A r m a x e j ∥ = ( Π r ∗ − A r m a x , 1 : r A r m a x , 1 : r T ) A r m a x e j = O P ( r m ) , (38)
uniformly over r < j ≤ r max r<j\leq r_{\max} r < j ≤ r m a x , where the second equality used A ^ r max , 1 : r T A ^ r max e j = 0 \widehat{A}_{r_{\max},1:r}^{\mathsf{T}}\widehat{A}_{r_{\max}}e_j=0 A r m a x , 1 : r T A r m a x e j = 0 , and the last used ∥ A ^ r max e j ∥ = 1 \|\widehat{A}_{r_{\max}}e_j\|=1 ∥ A r m a x e j ∥ = 1 and (A.18).
Note that by construction, Σ x Π r ∗ = Π r ∗ Σ x \Sigma_x\Pi_r^*=\Pi_r^*\Sigma_x Σ x Π r ∗ = Π r ∗ Σ x and hence s j T Σ x n j = 0 s_j^{\mathsf{T}}\Sigma_xn_j=0 s j T Σ x n j = 0 . Also, by construction, the largest eigenvalue of ( I p − Π ∗ ) Σ x ( I p − Π ∗ ) (I_p-\Pi^*)\Sigma_x(I_p-\Pi^*) ( I p − Π ∗ ) Σ x ( I p − Π ∗ ) is exactly λ r + 1 ( Σ x ) \lambda_{r+1}(\Sigma_x) λ r + 1 ( Σ x ) . Hence we have the Rayleigh quotient n j ⊤ Σ x n j / ∥ n j ∥ 2 ≤ λ r + 1 ( Σ x ) ≤ ∥ Σ ϵ ∥ ≤ C ϵ \mathbf{n}_j^\top\Sigma_x\mathbf{n}_j/\|\mathbf{n}_j\|^2 \le\lambda_{r+1}(\Sigma_x) \le\|\Sigma_\epsilon\| \le C_\epsilon n j ⊤ Σ x n j /∥ n j ∥ 2 ≤ λ r + 1 ( Σ x ) ≤ ∥ Σ ϵ ∥ ≤ C ϵ by (6) , Assumptions 2 and 3 (note that when n j = 0 \mathbf{n}_j=0 n j = 0 , we have trivially n j ⊤ Σ x n j ≤ C ϵ ⋅ ∥ n j ∥ 2 \mathbf{n}_j^\top\Sigma_x\mathbf{n}_j \le C_\epsilon\cdot\|\mathbf{n}_j\|^2 n j ⊤ Σ x n j ≤ C ϵ ⋅ ∥ n j ∥ 2 , which is sufficient for the below). Combining these with (A.20), we finally have
∥ b j ∥ 2 = ∥ M ⊤ s j + M ⊤ n j ∥ 2 = s j ⊤ Σ x s j + 2 s j ⊤ Σ x n j + n j ⊤ Σ x n j ≤ ∥ Σ x ∥ ⋅ ∥ s j ∥ 2 + n j ⊤ Σ x n j ≤ O P ( p r m 2 ) + C ϵ ⋅ ∥ n j ∥ 2 ≤ O P ( p r m 2 ) + C ϵ , \begin{aligned}
\|\mathbf{b}_j\|^2=\|M^\top\mathbf{s}_j+M^\top\mathbf{n}_j\|^2=\mathbf{s}_j^\top\Sigma_x\mathbf{s}_j+2\mathbf{s}_j^\top\Sigma_x\mathbf{n}_j+\mathbf{n}_j^\top\Sigma_x\mathbf{n}_j \\
&\le\|\Sigma_x\|\cdot\|\mathbf{s}_j\|^2+\mathbf{n}_j^\top\Sigma_x\mathbf{n}_j\le O_P(pr_m^2)+C_\epsilon\cdot\|\mathbf{n}_j\|^2\le O_P(pr_m^2)+C_\epsilon,
\end{aligned} ∥ b j ∥ 2 = ∥ M ⊤ s j + M ⊤ n j ∥ 2 = s j ⊤ Σ x s j + 2 s j ⊤ Σ x n j + n j ⊤ Σ x n j ≤ ∥ Σ x ∥ ⋅ ∥ s j ∥ 2 + n j ⊤ Σ x n j ≤ O P ( p r m 2 ) + C ϵ ⋅ ∥ n j ∥ 2 ≤ O P ( p r m 2 ) + C ϵ ,
uniformly over r < j ≤ r max r<j\le r_{\max} r < j ≤ r m a x , where the last inequality used the fact that ∥ n j ∥ 2 = 1 − ∥ s j ∥ 2 ≤ 1 \|\mathbf{n}_j\|^2=1-\|\mathbf{s}_j\|^2\le1 ∥ n j ∥ 2 = 1 − ∥ s j ∥ 2 ≤ 1 from 1 = ∥ s j ∥ 2 + ∥ n j ∥ 2 1=\|\mathbf{s}_j\|^2+\|\mathbf{n}_j\|^2 1 = ∥ s j ∥ 2 + ∥ n j ∥ 2 . This thus concludes the proof of this lemma.
Lemma 6 ([24 ] ). For any two jointly normal variables z z z and w w w with means μ z \mu_z μ z , μ w \mu_w μ w , positive variances σ z 2 \sigma_z^2 σ z 2 , σ w 2 \sigma_w^2 σ w 2 , and correlation ρ ∈ ( − 1 , 1 ) \rho\in(-1,1) ρ ∈ ( − 1 , 1 ) , the ratio z / w z/w z / w is distributed as
1 r ( a + x b + y ) + s , \frac{1}{r}\left(\frac{a+x}{b+y}\right)+s, r 1 ( b + y a + x ) + s , where x , y x,y x , y are independent standard normal, and
r : = σ w σ z 1 − ρ 2 , s : = ρ σ z σ w , b : = μ w σ w , a : = μ z / σ z − ρ μ w / σ w 1 − ρ 2 . r:=\frac{\sigma_w}{\sigma_z\sqrt{1-\rho^2}},\qquad s:=\rho\frac{\sigma_z}{\sigma_w},\qquad b:=\frac{\mu_w}{\sigma_w},\qquad a:=\frac{\mu_z/\sigma_z-\rho\mu_w/\sigma_w}{\sqrt{1-\rho^2}}. r := σ z 1 − ρ 2 σ w , s := ρ σ w σ z , b := σ w μ w , a := 1 − ρ 2 μ z / σ z − ρ μ w / σ w . In particular, if μ z = μ w = 0 \mu_z=\mu_w=0 μ z = μ w = 0 , then z / w z/w z / w is distributed as
σ z 1 − ρ 2 σ w c ∗ + ρ σ z σ w , \frac{\sigma_z\sqrt{1-\rho^2}}{\sigma_w}c^*+\rho\frac{\sigma_z}{\sigma_w}, σ w σ z 1 − ρ 2 c ∗ + ρ σ w σ z , where c ∗ ∼ Cauchy ( 0 , 1 ) c^*\sim\operatorname{Cauchy}(0,1) c ∗ ∼ Cauchy ( 0 , 1 ) .
Proof of Lemma 6. This is a direct quote from proposition in Section 2 of Marsaglia [24 ] , up to rewording and explicitly stating the constants according to the discussion therein.
Lemma 7. Let u \mathbf{u} u be uniformly distributed on S h − 1 \mathbb{S}^{h-1} S h − 1 with h ≥ 1 h \ge1 h ≥ 1 , and let a , b ∈ R h \mathbf{a}, \mathbf{b} \in\mathbb{R}^{h} a , b ∈ R h be deterministic nonzero vectors. Then it holds that
median u ∈ S h − 1 ( ( a T u ) 2 ( b T u ) 2 ) = ∥ a ∥ 2 ∥ b ∥ 2 . \operatorname{median}_{\mathbf{u}\in\mathbb{S}^{h-1}}\left(\frac{(\mathbf{a}^{\mathsf{T}}\mathbf{u})^{2}}{(\mathbf{b}^{\mathsf{T}}\mathbf{u})^{2}}\right)=\frac{\lVert\mathbf{a}\rVert^{2}}{\lVert\mathbf{b}\rVert^{2}}. median u ∈ S h − 1 ( ( b T u ) 2 ( a T u ) 2 ) = ∥ b ∥ 2 ∥ a ∥ 2 . Moreover, there exists an absolute constant c 0 > 0 c_{0} > 0 c 0 > 0 such that, for every δ ∈ ( 0 , 1 ) \delta\in(0,1) δ ∈ ( 0 , 1 ) ,
P ( ( a T u ) 2 ( b T u ) 2 ≤ ( 1 − δ ) ∥ a ∥ 2 ∥ b ∥ 2 ) ≤ 1 2 − c 0 δ , \mathbb{P}\left(\frac{(\mathbf{a}^{\mathsf{T}}\mathbf{u})^{2}}{(\mathbf{b}^{\mathsf{T}}\mathbf{u})^{2}}\le(1-\delta)\frac{\lVert\mathbf{a}\rVert^{2}}{\lVert\mathbf{b}\rVert^{2}}\right)\le\frac{1}{2}-c_{0}\delta, P ( ( b T u ) 2 ( a T u ) 2 ≤ ( 1 − δ ) ∥ b ∥ 2 ∥ a ∥ 2 ) ≤ 2 1 − c 0 δ , P ( ( a T u ) 2 ( b T u ) 2 ≤ ( 1 + δ ) ∥ a ∥ 2 ∥ b ∥ 2 ) ≥ 1 2 + c 0 δ , \mathbb{P}\left(\frac{(\mathbf{a}^{\mathsf{T}}\mathbf{u})^{2}}{(\mathbf{b}^{\mathsf{T}}\mathbf{u})^{2}}\le(1+\delta)\frac{\lVert\mathbf{a}\rVert^{2}}{\lVert\mathbf{b}\rVert^{2}}\right)\ge\frac{1}{2}+c_{0}\delta, P ( ( b T u ) 2 ( a T u ) 2 ≤ ( 1 + δ ) ∥ b ∥ 2 ∥ a ∥ 2 ) ≥ 2 1 + c 0 δ , and these bounds hold uniformly over h h h , a \mathbf{a} a , and b \mathbf{b} b .
Proof of Lemma 7 . As u \mathbf{u} u is uniformly distributed on the unit sphere S h − 1 \mathbb{S}^{h-1} S h − 1 , it can be represented as u = z / ∥ z ∥ \mathbf{u}=\mathbf{z}/\lVert\mathbf{z}\rVert u = z / ∥ z ∥ where z ∼ N ( 0 , I h ) \mathbf{z}\sim\mathcal{N}(0,I_{h}) z ∼ N ( 0 , I h ) . Hence we have
( a T u ) 2 ( b T u ) 2 = ( a T z / ∥ z ∥ ) 2 ( b T z / ∥ z ∥ ) 2 = ( a T z ) 2 ( b T z ) 2 = ∥ a ∥ 2 ∥ b ∥ 2 ⋅ x 2 y 2 , \frac{(\mathbf{a}^{\mathsf{T}}\mathbf{u})^{2}}{(\mathbf{b}^{\mathsf{T}}\mathbf{u})^{2}}=\frac{(\mathbf{a}^{\mathsf{T}}\mathbf{z}/\lVert\mathbf{z}\rVert)^{2}}{(\mathbf{b}^{\mathsf{T}}\mathbf{z}/\lVert\mathbf{z}\rVert)^{2}}=\frac{(\mathbf{a}^{\mathsf{T}}\mathbf{z})^{2}}{(\mathbf{b}^{\mathsf{T}}\mathbf{z})^{2}}=\frac{\lVert\mathbf{a}\rVert^{2}}{\lVert\mathbf{b}\rVert^{2}}\cdot\frac{x^{2}}{y^{2}}, ( b T u ) 2 ( a T u ) 2 = ( b T z / ∥ z ∥ ) 2 ( a T z / ∥ z ∥ ) 2 = ( b T z ) 2 ( a T z ) 2 = ∥ b ∥ 2 ∥ a ∥ 2 ⋅ y 2 x 2 ,
where x : = a T z / ∥ a ∥ ∼ N ( 0 , 1 ) x:=\mathbf{a}^{\mathsf{T}}\mathbf{z}/\lVert\mathbf{a}\rVert\sim\mathcal{N}(0,1) x := a T z / ∥ a ∥ ∼ N ( 0 , 1 ) and y : = b T z / ∥ b ∥ ∼ N ( 0 , 1 ) y:=\mathbf{b}^{\mathsf{T}}\mathbf{z}/\lVert\mathbf{b}\rVert\sim\mathcal{N}(0,1) y := b T z / ∥ b ∥ ∼ N ( 0 , 1 ) . Denote the correlation between x x x and y y y by ρ : = ( a T b ) / ( ∥ a ∥ ∥ b ∥ ) \rho:=(\mathbf{a}^{\mathsf{T}}\mathbf{b})/(\lVert\mathbf{a}\rVert\lVert\mathbf{b}\rVert) ρ := ( a T b ) / (∥ a ∥ ∥ b ∥) . Note that ∣ ρ ∣ = 1 \lvert\rho\rvert=1 ∣ ρ ∣ = 1 if and only if a = c ⋅ b \mathbf{a}=c\cdot\mathbf{b} a = c ⋅ b for some constant c ≠ 0 c\ne0 c = 0 . The desired result holds trivially if a = c ⋅ b \mathbf{a}=c\cdot\mathbf{b} a = c ⋅ b , and hence it suffices to consider the case where ∣ ρ ∣ ≠ 1 \lvert\rho\rvert\ne1 ∣ ρ ∣ = 1 .
Then by Lemma 6, let c ∗ c^{*} c ∗ be a random sample from standard Cauchy, and we have
P ( ( a T u ) 2 ( b T u ) 2 ≥ ∥ a ∥ 2 ∥ b ∥ 2 ) = P ( x 2 y 2 ≥ 1 ) = P ( ( 1 − ρ 2 ⋅ c ∗ + ρ ) 2 ≥ 1 ) = P ( 1 − ρ 2 ⋅ c ∗ + ρ ≥ 1 ) + P ( 1 − ρ 2 ⋅ c ∗ + ρ ≤ − 1 ) = P ( c ∗ ≥ 1 − ρ 1 + ρ ) + P ( c ∗ ≤ − 1 + ρ 1 − ρ ) = P ( c ∗ ≥ 1 − ρ 1 + ρ ) + P ( c ∗ ≥ 1 + ρ 1 − ρ ) = 1 2 − 1 π arctan ( 1 − ρ 1 + ρ ) + 1 2 − 1 π arctan ( 1 + ρ 1 − ρ ) = 1 − 1 π ⋅ π 2 = 1 2 , \begin{aligned}
\mathbb{P}\left(\frac{(\mathbf{a}^{\mathsf{T}}\mathbf{u})^{2}}{(\mathbf{b}^{\mathsf{T}}\mathbf{u})^{2}}\ge\frac{\lVert\mathbf{a}\rVert^{2}}{\lVert\mathbf{b}\rVert^{2}}\right)=\mathbb{P}\left(\frac{x^{2}}{y^{2}}\ge1\right)=\mathbb{P}\left(\left(\sqrt{1-\rho^{2}}\cdot c^{*}+\rho\right)^{2}\ge1\right) \\
&= \mathbb{P}\left(\sqrt{1-\rho^{2}}\cdot c^{*}+\rho\ge1\right)+\mathbb{P}\left(\sqrt{1-\rho^{2}}\cdot c^{*}+\rho\le-1\right) \\
&= \mathbb{P}\left(c^{*}\ge\sqrt{\frac{1-\rho}{1+\rho}}\right)+\mathbb{P}\left(c^{*}\le-\sqrt{\frac{1+\rho}{1-\rho}}\right)=\mathbb{P}\left(c^{*}\ge\sqrt{\frac{1-\rho}{1+\rho}}\right)+\mathbb{P}\left(c^{*}\ge\sqrt{\frac{1+\rho}{1-\rho}}\right) \\
&=\frac{1}{2}-\frac{1}{\pi}\arctan\left(\sqrt{\frac{1-\rho}{1+\rho}}\right)+\frac{1}{2}-\frac{1}{\pi}\arctan\left(\sqrt{\frac{1+\rho}{1-\rho}}\right)=1-\frac{1}{\pi}\cdot\frac{\pi}{2}=\frac{1}{2},
\end{aligned} P ( ( b T u ) 2 ( a T u ) 2 ≥ ∥ b ∥ 2 ∥ a ∥ 2 ) = P ( y 2 x 2 ≥ 1 ) = P ( ( 1 − ρ 2 ⋅ c ∗ + ρ ) 2 ≥ 1 ) = P ( 1 − ρ 2 ⋅ c ∗ + ρ ≥ 1 ) + P ( 1 − ρ 2 ⋅ c ∗ + ρ ≤ − 1 ) = P ( c ∗ ≥ 1 + ρ 1 − ρ ) + P ( c ∗ ≤ − 1 − ρ 1 + ρ ) = P ( c ∗ ≥ 1 + ρ 1 − ρ ) + P ( c ∗ ≥ 1 − ρ 1 + ρ ) = 2 1 − π 1 arctan ( 1 + ρ 1 − ρ ) + 2 1 − π 1 arctan ( 1 − ρ 1 + ρ ) = 1 − π 1 ⋅ 2 π = 2 1 ,
as desired, where the last line used the fact that arctan ( x ) + arctan ( 1 / x ) = π / 2 \arctan(x)+\arctan(1/x)=\pi/2 arctan ( x ) + arctan ( 1/ x ) = π /2 for any x > 0 x>0 x > 0 .
This shows the desired median.
For every s > 0 s>0 s > 0 , by Lemma 6 as above again, and by direct computation, we obtain
P ( x 2 y 2 ≤ s ) = 1 π { arctan ( s − ρ 1 − ρ 2 ) − arctan ( − s − ρ 1 − ρ 2 ) } = 1 2 + 1 π arctan ( s − 1 2 s 1 − ρ 2 ) , \mathbb{P}\left(\frac{x^{2}}{y^{2}}\le s\right)=\frac{1}{\pi}\left\{\arctan\left(\frac{\sqrt{s}-\rho}{\sqrt{1-\rho^{2}}}\right)-\arctan\left(\frac{-\sqrt{s}-\rho}{\sqrt{1-\rho^{2}}}\right)\right\}
=\frac{1}{2}+\frac{1}{\pi}\arctan\left(\frac{s-1}{2\sqrt{s}\sqrt{1-\rho^{2}}}\right), P ( y 2 x 2 ≤ s ) = π 1 { arctan ( 1 − ρ 2 s − ρ ) − arctan ( 1 − ρ 2 − s − ρ ) } = 2 1 + π 1 arctan ( 2 s 1 − ρ 2 s − 1 ) ,
so that for δ ∈ ( 0 , 1 ) \delta\in(0,1) δ ∈ ( 0 , 1 ) , using 1 − ρ 2 ≤ 1 \sqrt{1-\rho^{2}}\le1 1 − ρ 2 ≤ 1 , we have
P ( x 2 y 2 ≤ 1 + δ ) − 1 2 = 1 π arctan ( δ 2 1 + δ 1 − ρ 2 ) ≥ 1 π arctan ( δ 2 2 ) ≥ δ 4 π 2 , \mathbb{P}\left(\frac{x^{2}}{y^{2}}\le1+\delta\right)-\frac{1}{2}
=\frac{1}{\pi}\arctan\left(\frac{\delta}{2\sqrt{1+\delta}\sqrt{1-\rho^{2}}}\right)
\ge\frac{1}{\pi}\arctan\left(\frac{\delta}{2\sqrt{2}}\right)\ge\frac{\delta}{4\pi\sqrt{2}}, P ( y 2 x 2 ≤ 1 + δ ) − 2 1 = π 1 arctan ( 2 1 + δ 1 − ρ 2 δ ) ≥ π 1 arctan ( 2 2 δ ) ≥ 4 π 2 δ ,
and similarly,
1 2 − P ( x 2 y 2 ≤ 1 − δ ) = 1 π arctan ( δ 2 1 − δ 1 − ρ 2 ) ≥ 1 π arctan ( δ 2 2 ) ≥ δ 4 π 2 , \frac{1}{2}-\mathbb{P}\left(\frac{x^{2}}{y^{2}}\le1-\delta\right)
=\frac{1}{\pi}\arctan\left(\frac{\delta}{2\sqrt{1-\delta}\sqrt{1-\rho^{2}}}\right)
\ge\frac{1}{\pi}\arctan\left(\frac{\delta}{2\sqrt{2}}\right)\ge\frac{\delta}{4\pi\sqrt{2}}, 2 1 − P ( y 2 x 2 ≤ 1 − δ ) = π 1 arctan ( 2 1 − δ 1 − ρ 2 δ ) ≥ π 1 arctan ( 2 2 δ ) ≥ 4 π 2 δ ,
where the last line used arctan ( v ) ≥ v / 2 \arctan(v)\ge v/2 arctan ( v ) ≥ v /2 for v ∈ [ 0 , 1 ] v\in[0,1] v ∈ [ 0 , 1 ] . Thus we may take c 0 = 1 / ( 4 π 2 ) c_{0}=1/(4\pi\sqrt{2}) c 0 = 1/ ( 4 π 2 ) , which completes the proof of this lemma.
Proofs of corollaries and propositions
Proof of Corollary 1. Let A ^ r \widehat{\mathbf A}_r A r denote the DC estimator computed from all observations with the deterministic block size b = r b=r b = r . Note that the resulting number of blocks satisfies ⌊ T / r ⌋ ≍ T \lfloor T/r\rfloor\asymp T ⌊ T / r ⌋ ≍ T as r r r is fixed, T ‾ = o ( T ) \overline{T}=o(T) T = o ( T ) by (17) , and hence by Theorem 1 we have
d ( A ^ r , A ) = O P ( 1 T + 1 p + T ‾ T ) . (39) d(\widehat{\mathbf A}_r,\mathbf A)=\mathcal{O}_P\left(\frac{1}{\sqrt{T}}+\frac{1}{p}+\frac{\overline{T}}{T}\right). \tag*{(39)} d ( A r , A ) = O P ( T 1 + p 1 + T T ) . (39)
On the event { r ^ = r } \{\widehat r=r\} { r = r } , the estimator A ^ r ^ = A ^ r \widehat{\mathbf A}_{\widehat r}=\widehat{\mathbf A}_r A r = A r . Hence for every deterministic u > 0 u>0 u > 0 ,
P ( d ( A ^ r ^ , A ) > u ) ≤ P ( r ^ ≠ r ) + P ( d ( A ^ r , A ) > u ) , \mathbb{P}\left(d(\widehat{\mathbf A}_{\widehat r},\mathbf A)>u\right)\leq\mathbb{P}(\widehat r\ne r)+\mathbb{P}\left(d(\widehat{\mathbf A}_r,\mathbf A)>u\right), P ( d ( A r , A ) > u ) ≤ P ( r = r ) + P ( d ( A r , A ) > u ) ,
which, together with Theorem 2 and (A.21), concludes this proof. □
Proof of Corollary 2. Fix any t ∈ [ T ] t\in[T] t ∈ [ T ] , recall that T = ∅ \mathcal T=\varnothing T = ∅ here, and recall v t = u t / ∥ u t ∥ \mathbf v_t=\mathbf u_t/\|\mathbf u_t\| v t = u t /∥ u t ∥ . Since v t \mathbf v_t v t is uniform on S r + p − 1 \mathbb S^{r+p-1} S r + p − 1 , we have E [ v t v t T ] = ( r + p ) − 1 I r + p \mathbb E[\mathbf v_t\mathbf v_t^\mathsf{T}]=(r+p)^{-1}\mathbf I_{r+p} E [ v t v t T ] = ( r + p ) − 1 I r + p . Then by (4) , (5) , and ζ t > 0 \zeta_t>0 ζ t > 0 almost surely, we obtain
E [ r + p p ζ t 2 ∥ x t ∥ 2 ] = r + p p E [ ∥ M v t ∥ 2 ] = 1 p tr ( M M T ) = 1 p tr ( Σ x ) ≤ r C λ + C ϵ , E [ r + p p ζ t 2 ∥ ϵ t ∥ 2 ] = 1 p tr ( Σ ϵ ) ≤ C ϵ , E [ r + p ζ t 2 ∥ A ¨ A ¨ T ϵ t ∥ 2 ] = tr ( A ¨ A ¨ T Σ ϵ ) ≤ r C ϵ , \begin{aligned}
\mathbb E\left[\frac{r+p}{p\zeta_t^2}\|\mathbf x_t\|^2\right]
&=\frac{r+p}{p}\mathbb E[\|\mathbf M\mathbf v_t\|^2]
=\frac{1}{p}\operatorname{tr}(\mathbf M\mathbf M^\mathsf{T})
=\frac{1}{p}\operatorname{tr}(\boldsymbol\Sigma_x)
\leq rC_\lambda+C_\epsilon,\\
\mathbb E\left[\frac{r+p}{p\zeta_t^2}\|\boldsymbol\epsilon_t\|^2\right]
&=\frac{1}{p}\operatorname{tr}(\boldsymbol\Sigma_\epsilon)\leq C_\epsilon,\\
\mathbb E\left[\frac{r+p}{\zeta_t^2}\|\ddot{\mathbf A}\ddot{\mathbf A}^\mathsf{T}\boldsymbol\epsilon_t\|^2\right]
&=\operatorname{tr}\left(\ddot{\mathbf A}\ddot{\mathbf A}^\mathsf{T}\boldsymbol\Sigma_\epsilon\right)\leq rC_\epsilon,
\end{aligned} E [ p ζ t 2 r + p ∥ x t ∥ 2 ] E [ p ζ t 2 r + p ∥ ϵ t ∥ 2 ] E [ ζ t 2 r + p ∥ A ¨ A ¨ T ϵ t ∥ 2 ] = p r + p E [ ∥ M v t ∥ 2 ] = p 1 tr ( M M T ) = p 1 tr ( Σ x ) ≤ r C λ + C ϵ , = p 1 tr ( Σ ϵ ) ≤ C ϵ , = tr ( A ¨ A ¨ T Σ ϵ ) ≤ r C ϵ ,
where, particularly, note that these expectations are finite without any moment condition on ζ t \zeta_t ζ t , since each ζ t 2 \zeta_t^2 ζ t 2 is canceled out by itself within ∥ x t ∥ 2 \|\mathbf x_t\|^2 ∥ x t ∥ 2 , ∥ ϵ t ∥ 2 \|\boldsymbol\epsilon_t\|^2 ∥ ϵ t ∥ 2 , and ∥ A ¨ A ¨ T ϵ t ∥ 2 \|\ddot{\mathbf A}\ddot{\mathbf A}^\mathsf{T}\boldsymbol\epsilon_t\|^2 ∥ A ¨ A ¨ T ϵ t ∥ 2 , respectively. Hence by ξ t 2 / p = O P ( 1 ) \xi_t^2/p = \mathcal{O}_P(1) ξ t 2 / p = O P ( 1 ) , it holds that
∥ x t ∥ 2 p = O P ( 1 ) , ∥ ϵ t ∥ 2 p = O P ( 1 ) , ∥ A ¨ A ¨ T ϵ t ∥ 2 = O P ( 1 ) . (40) \frac{\lVert x_t\rVert^2}{p}=\mathcal{O}_P(1),\qquad\frac{\lVert\epsilon_t\rVert^2}{p}=\mathcal{O}_P(1),\qquad\lVert\ddot{A}\ddot{A}^{\mathsf{T}}\epsilon_t\rVert^2=\mathcal{O}_P(1). \tag*{(40)} p ∥ x t ∥ 2 = O P ( 1 ) , p ∥ ϵ t ∥ 2 = O P ( 1 ) , ∥ A ¨ A ¨ T ϵ t ∥ 2 = O P ( 1 ) . (40)
Hence by model (1) and definitions of the estimators and H ^ \widehat{H} H , we obtain
∥ f ^ t − H ^ f t ∥ 2 = ∥ 1 p A ^ r ^ T ϵ t ∥ 2 = 1 p ∥ A ^ r ^ A ^ r ^ T ϵ t ∥ 2 ≤ 2 p ∥ A ¨ A ¨ T ϵ t ∥ 2 + 2 p ∥ A ^ r ^ A ^ r ^ T − A ¨ A ¨ T ∥ 2 ⋅ ∥ ϵ t ∥ 2 = O P ( 1 p + ∥ A ^ r ^ A ^ r ^ T − A ¨ A ¨ T ∥ 2 ) = O P ( 1 T + 1 p ) , \begin{aligned}
\lVert\widehat{f}_t-\widehat{H}f_t\rVert^2
=\left\lVert\frac{1}{\sqrt{p}}\widehat{A}_{\widehat{r}}^{\mathsf{T}}\epsilon_t\right\rVert^2
=\frac{1}{p}\lVert\widehat{A}_{\widehat{r}}\widehat{A}_{\widehat{r}}^{\mathsf{T}}\epsilon_t\rVert^2 \\
&\leq\frac{2}{p}\lVert\ddot{A}\ddot{A}^{\mathsf{T}}\epsilon_t\rVert^2
+\frac{2}{p}\lVert\widehat{A}_{\widehat{r}}\widehat{A}_{\widehat{r}}^{\mathsf{T}}-\ddot{A}\ddot{A}^{\mathsf{T}}\rVert^2\cdot\lVert\epsilon_t\rVert^2 \\
&=\mathcal{O}_P\left(\frac{1}{p}+\lVert\widehat{A}_{\widehat{r}}\widehat{A}_{\widehat{r}}^{\mathsf{T}}-\ddot{A}\ddot{A}^{\mathsf{T}}\rVert^2\right)
=\mathcal{O}_P\left(\frac{1}{T}+\frac{1}{p}\right),
\end{aligned} ∥ f t − H f t ∥ 2 = p 1 A r T ϵ t 2 = p 1 ∥ A r A r T ϵ t ∥ 2 ≤ p 2 ∥ A ¨ A ¨ T ϵ t ∥ 2 + p 2 ∥ A r A r T − A ¨ A ¨ T ∥ 2 ⋅ ∥ ϵ t ∥ 2 = O P ( p 1 + ∥ A r A r T − A ¨ A ¨ T ∥ 2 ) = O P ( T 1 + p 1 ) ,
as desired, where the last line used (A.22) and Corollary 1.
Similarly for the common component, as χ ^ t − χ t = ( A ^ r ^ A ^ r ^ T − A ¨ A ¨ T ) x t + A ¨ A ¨ T ϵ t \widehat{\chi}_t-\chi_t=(\widehat{A}_{\widehat{r}}\widehat{A}_{\widehat{r}}^{\mathsf{T}}-\ddot{A}\ddot{A}^{\mathsf{T}})x_t+\ddot{A}\ddot{A}^{\mathsf{T}}\epsilon_t χ t − χ t = ( A r A r T − A ¨ A ¨ T ) x t + A ¨ A ¨ T ϵ t , by (A.22) and Corollary 1 again, we have
1 p ∥ χ ^ t − χ t ∥ 2 ≤ 2 ∥ A ^ r ^ A ^ r ^ T − A ¨ A ¨ T ∥ 2 ∥ x t ∥ 2 p + 2 p ∥ A ¨ A ¨ T ϵ t ∥ 2 = O P ( 1 T + 1 p ) , \frac{1}{p}\lVert\widehat{\chi}_t-\chi_t\rVert^2
\leq2\lVert\widehat{A}_{\widehat{r}}\widehat{A}_{\widehat{r}}^{\mathsf{T}}-\ddot{A}\ddot{A}^{\mathsf{T}}\rVert^2\frac{\lVert x_t\rVert^2}{p}
+\frac{2}{p}\lVert\ddot{A}\ddot{A}^{\mathsf{T}}\epsilon_t\rVert^2
=\mathcal{O}_P\left(\frac{1}{T}+\frac{1}{p}\right), p 1 ∥ χ t − χ t ∥ 2 ≤ 2 ∥ A r A r T − A ¨ A ¨ T ∥ 2 p ∥ x t ∥ 2 + p 2 ∥ A ¨ A ¨ T ϵ t ∥ 2 = O P ( T 1 + p 1 ) ,
as desired. It remains to show H ^ \widehat{H} H is well conditioned. As P ( r ^ = r ) → 1 \mathbb{P}(\widehat{r}=r)\to1 P ( r = r ) → 1 by Theorem 2, it suffices to work under the event { r ^ = r } \{\widehat{r}=r\} { r = r } . Then H ^ \widehat{H} H being well conditioned is straightforward by (10) and noting that
∥ 1 p A T A − H ^ T H ^ ∥ = ∥ 1 p A T A − 1 p A T A ^ r ^ A ^ r ^ T A ∥ = 1 p ∥ A T ( I p − A ^ r ^ A ^ r ^ T ) A ∥ = 1 p ∥ ( I p − A ^ r ^ A ^ r ^ T ) A ∥ 2 ≤ 1 p ∥ A ∥ 2 ⋅ ∥ A ^ r ^ A ^ r ^ T − A ¨ A ¨ T ∥ 2 = o P ( 1 ) , \begin{aligned}
\left\lVert\frac{1}{p}A^{\mathsf{T}}A-\widehat{H}^{\mathsf{T}}\widehat{H}\right\rVert
=\left\lVert\frac{1}{p}A^{\mathsf{T}}A-\frac{1}{p}A^{\mathsf{T}}\widehat{A}_{\widehat{r}}\widehat{A}_{\widehat{r}}^{\mathsf{T}}A\right\rVert
=\frac{1}{p}\left\lVert A^{\mathsf{T}}(I_p-\widehat{A}_{\widehat{r}}\widehat{A}_{\widehat{r}}^{\mathsf{T}})A\right\rVert\\
&=\frac{1}{p}\lVert(I_p-\widehat{A}_{\widehat{r}}\widehat{A}_{\widehat{r}}^{\mathsf{T}})A\rVert^2
\leq\frac{1}{p}\lVert A\rVert^2\cdot\lVert\widehat{A}_{\widehat{r}}\widehat{A}_{\widehat{r}}^{\mathsf{T}}-\ddot{A}\ddot{A}^{\mathsf{T}}\rVert^2=o_P(1),
\end{aligned} p 1 A T A − H T H = p 1 A T A − p 1 A T A r A r T A = p 1 A T ( I p − A r A r T ) A = p 1 ∥( I p − A r A r T ) A ∥ 2 ≤ p 1 ∥ A ∥ 2 ⋅ ∥ A r A r T − A ¨ A ¨ T ∥ 2 = o P ( 1 ) ,
where the third equality used the fact that ( I p − A ^ r ^ A ^ r ^ T ) (I_p-\widehat{A}_{\widehat{r}}\widehat{A}_{\widehat{r}}^{\mathsf{T}}) ( I p − A r A r T ) is an orthogonal projection matrix, the inequality used A ¨ A ¨ T A = A \ddot{A}\ddot{A}^{\mathsf{T}}A=A A ¨ A ¨ T A = A by construction, and the last equality used Corollary 1 again. This thus concludes this proof.
Proof of Proposition 1. As in the proof of Theorem 1, recall the latent observation x t 0 \mathbf{x}^{0}_{t} x t 0 . It is direct to check Assumption 1 is satisfied. By construction in the statement, the eigenvalues of A ⊤ A \mathbf{A}^{\top}\mathbf{A} A ⊤ A are κ p ( p + p δ ) \kappa_{p}(p+p^{\delta}) κ p ( p + p δ ) and p δ p^{\delta} p δ , respectively, so Assumption 2 holds with c λ = 1 c_{\lambda}=1 c λ = 1 . Assumption 3 holds with c ϵ = C ϵ = 1 c_{\epsilon}=C_{\epsilon}=1 c ϵ = C ϵ = 1 . Also, ∣ T ∣ = T ‾ = 1 |\mathcal{T}|=\overline{\mathcal{T}}=1 ∣ T ∣ = T = 1 and satisfies Assumption 4.
Next, note that we can represent
x t 0 = a p G t e 1 + y t , a p 2 : = κ p ( p + p δ ) + 1 , e 1 ⊤ y t = 0 , \mathbf{x}^{0}_{t}=a_{p}G_{t}\mathbf{e}_{1}+\mathbf{y}_{t},\qquad a_{p}^{2}:=\kappa_{p}(p+p^{\delta})+1,\qquad\mathbf{e}_{1}^{\top}\mathbf{y}_{t}=0, x t 0 = a p G t e 1 + y t , a p 2 := κ p ( p + p δ ) + 1 , e 1 ⊤ y t = 0 ,
where G t ∼ N ( 0 , 1 ) G_{t}\sim\mathcal{N}(0,1) G t ∼ N ( 0 , 1 ) independent of y t \mathbf{y}_{t} y t , and E [ ∥ y t ∥ 2 ] = p δ + p − 1 ≤ p + p δ \mathbb{E}[\|\mathbf{y}_{t}\|^{2}]=p^{\delta}+p-1\leq p+p^{\delta} E [ ∥ y t ∥ 2 ] = p δ + p − 1 ≤ p + p δ . Hence with Q t 0 : = x t 0 ( x t 0 ) ⊤ / ∥ x t 0 ∥ 2 \mathbf{Q}^{0}_{t}:=\mathbf{x}^{0}_{t}(\mathbf{x}^{0}_{t})^{\top}/\|\mathbf{x}^{0}_{t}\|^{2} Q t 0 := x t 0 ( x t 0 ) ⊤ /∥ x t 0 ∥ 2 , we have
∥ Q t 0 − e 1 e 1 ⊤ ∥ F = 2 ⋅ ∥ y t ∥ a p 2 G t 2 + ∥ y t ∥ 2 ≤ 2 ⋅ min { 1 , ∥ y t ∥ a p ⋅ ∣ G t ∣ } . (41) \left\|\mathbf{Q}^{0}_{t}-\mathbf{e}_{1}\mathbf{e}_{1}^{\top}\right\|_{F}=\sqrt{2}\cdot\frac{\|\mathbf{y}_{t}\|}{\sqrt{a_{p}^{2}G_{t}^{2}+\|\mathbf{y}_{t}\|^{2}}}\leq\sqrt{2}\cdot\min\left\{1,\frac{\|\mathbf{y}_{t}\|}{a_{p}\cdot|G_{t}|}\right\}. \tag*{(41)} Q t 0 − e 1 e 1 ⊤ F = 2 ⋅ a p 2 G t 2 + ∥ y t ∥ 2 ∥ y t ∥ ≤ 2 ⋅ min { 1 , a p ⋅ ∣ G t ∣ ∥ y t ∥ } . (41)
Note that conditional on G t G_{t} G t , by the Jensen’s inequality, we have
E [ min { 1 , ∥ y t ∥ a p ⋅ ∣ G t ∣ } ∣ G t ] ≤ min { 1 , E [ ∥ y t ∥ 2 ] a p ⋅ ∣ G t ∣ } ≤ min { 1 , κ p − 1 / 2 ∣ G t ∣ } , \mathbb{E}\left[\left.\min\left\{1,\frac{\|\mathbf{y}_{t}\|}{a_{p}\cdot|G_{t}|}\right\}\right|G_{t}\right]\leq\min\left\{1,\frac{\sqrt{\mathbb{E}[\|\mathbf{y}_{t}\|^{2}]}}{a_{p}\cdot|G_{t}|}\right\}\leq\min\left\{1,\frac{\kappa_{p}^{-1/2}}{|G_{t}|}\right\}, E [ min { 1 , a p ⋅ ∣ G t ∣ ∥ y t ∥ } G t ] ≤ min { 1 , a p ⋅ ∣ G t ∣ E [ ∥ y t ∥ 2 ] } ≤ min { 1 , ∣ G t ∣ κ p − 1/2 } ,
which, together with the notation ϕ ( ⋅ ) \phi(\cdot) ϕ ( ⋅ ) representing the density function of N ( 0 , 1 ) \mathcal{N}(0,1) N ( 0 , 1 ) , using κ p − 1 / 2 < 1 \kappa_{p}^{-1/2}<1 κ p − 1/2 < 1 , implies
E [ min { 1 , κ p − 1 / 2 ∣ G t ∣ } ] = P ( ∣ G t ∣ ≤ κ p − 1 / 2 ) + 2 κ p − 1 / 2 ∫ κ p − 1 / 2 ∞ ϕ ( v ) v d v \mathbb{E}\left[\min\left\{1,\frac{\kappa_{p}^{-1/2}}{|G_{t}|}\right\}\right]=\mathbb{P}\left(|G_{t}|\leq\kappa_{p}^{-1/2}\right)+2\kappa_{p}^{-1/2}\int_{\kappa_{p}^{-1/2}}^{\infty}\frac{\phi(v)}{v}\,\mathrm{d}v E [ min { 1 , ∣ G t ∣ κ p − 1/2 } ] = P ( ∣ G t ∣ ≤ κ p − 1/2 ) + 2 κ p − 1/2 ∫ κ p − 1/2 ∞ v ϕ ( v ) d v
≤ 2 ϕ ( 0 ) κ p − 1 / 2 + 2 ϕ ( 0 ) κ p − 1 / 2 ∫ κ p − 1 / 2 1 1 v d v + 2 κ p − 1 / 2 ∫ 1 ∞ ϕ ( v ) d v ≤ C κ p − 1 / 2 { 1 + log ( 1 / κ p − 1 / 2 ) } ≤ C log ( κ p ) κ p , \begin{aligned}
\leq2\phi(0)\kappa_{p}^{-1/2}+2\phi(0)\kappa_{p}^{-1/2}\int_{\kappa_{p}^{-1/2}}^{1}\frac{1}{v}\,\mathrm{d}v+2\kappa_{p}^{-1/2}\int_{1}^{\infty}\phi(v)\,\mathrm{d}v \\
&\le C\kappa_p^{-1/2}\left\{1+\log\left(1/\kappa_p^{-1/2}\right)\right\}\le C\frac{\log(\kappa_p)}{\sqrt{\kappa_p}},
\end{aligned} ≤ 2 ϕ ( 0 ) κ p − 1/2 + 2 ϕ ( 0 ) κ p − 1/2 ∫ κ p − 1/2 1 v 1 d v + 2 κ p − 1/2 ∫ 1 ∞ ϕ ( v ) d v ≤ C κ p − 1/2 { 1 + log ( 1/ κ p − 1/2 ) } ≤ C κ p log ( κ p ) ,
which, combined with (A.23), gives
E [ ∥ Q t 0 − e 1 e 1 T ∥ F ] ≤ C log ( κ p ) κ p . (42) \mathbb{E}\left[\left\|\mathbf{Q}^{0}_{t}-\mathbf{e}_{1}\mathbf{e}_{1}^{\mathsf{T}}\right\|_{F}\right]\le C\frac{\log(\kappa_p)}{\sqrt{\kappa_p}}. \tag*{(42)} E [ Q t 0 − e 1 e 1 T F ] ≤ C κ p log ( κ p ) . (42)
Let J J J be the anomaly location, and note that w : = z / ∥ z ∥ \mathbf{w}:=\mathbf{z}/\|\mathbf{z}\| w := z /∥ z ∥ is independent and uniform on the unit sphere in R p \mathbb{R}^{p} R p . Then define the matrix
L : = T − 1 T e 1 e 1 T + 1 T w w T . \mathbf{L}:=\frac{T-1}{T}\mathbf{e}_{1}\mathbf{e}_{1}^{\mathsf{T}}+\frac{1}{T}\mathbf{w}\mathbf{w}^{\mathsf{T}}. L := T T − 1 e 1 e 1 T + T 1 w w T .
By the triangle inequality, (A.24), and the independence of J J J from the clean data, we have
E [ ∥ P ^ ( 1 ) − L ∥ ] ≤ E [ 1 T ∑ t ∈ [ T ] ∖ { J } ∥ Q t 0 − e 1 e 1 T ∥ F ] = T − 1 T ⋅ E [ ∥ Q 1 0 − e 1 e 1 T ∥ F ] ≤ C log ( κ p ) κ p , \begin{aligned}
\mathbb{E}\left[\left\|\widehat{\mathbf{P}}^{(1)}-\mathbf{L}\right\|\right]
&\le\mathbb{E}\left[\frac{1}{T}\sum_{t\in[T]\setminus\{J\}}\left\|\mathbf{Q}^{0}_{t}-\mathbf{e}_{1}\mathbf{e}_{1}^{\mathsf{T}}\right\|_{F}\right]\\
&=\frac{T-1}{T}\cdot\mathbb{E}\left[\left\|\mathbf{Q}^{0}_{1}-\mathbf{e}_{1}\mathbf{e}_{1}^{\mathsf{T}}\right\|_{F}\right]\le C\frac{\log(\kappa_p)}{\sqrt{\kappa_p}},
\end{aligned} E [ P ( 1 ) − L ] ≤ E T 1 t ∈ [ T ] ∖ { J } ∑ Q t 0 − e 1 e 1 T F = T T − 1 ⋅ E [ Q 1 0 − e 1 e 1 T F ] ≤ C κ p log ( κ p ) ,
which, under (B.1), implies ∥ P ^ ( 1 ) − L ∥ = o P ( 1 / T ) \|\widehat{\mathbf{P}}^{(1)}-\mathbf{L}\|=o_{P}(1/T) ∥ P ( 1 ) − L ∥ = o P ( 1/ T ) .
Since E [ ( e 1 T w ) 2 ] = 1 / p \mathbb{E}[(\mathbf{e}_{1}^{\mathsf{T}}\mathbf{w})^{2}]=1/p E [( e 1 T w ) 2 ] = 1/ p , we have e 1 T w = o P ( 1 ) \mathbf{e}_{1}^{\mathsf{T}}\mathbf{w}=o_{P}(1) e 1 T w = o P ( 1 ) . Note that the range of L \mathbf{L} L is span ( e 1 , w ) \operatorname{span}(\mathbf{e}_{1},\mathbf{w}) span ( e 1 , w ) , and its two positive eigenvalues have sum 1 1 1 and product { ( T − 1 ) / T 2 } ( 1 − ( e 1 T w ) 2 ) \{(T-1)/T^{2}\}(1-(\mathbf{e}_{1}^{\mathsf{T}}\mathbf{w})^{2}) {( T − 1 ) / T 2 } ( 1 − ( e 1 T w ) 2 ) . As λ 1 ( L ) ≤ 1 \lambda_{1}(\mathbf{L})\le1 λ 1 ( L ) ≤ 1 , we hence have
λ 2 ( L ) = ( T − 1 ) ( 1 − ( e 1 T w ) 2 ) T 2 λ 1 ( L ) ≥ ( T − 1 ) ( 1 − ( e 1 T w ) 2 ) T 2 ≥ 1 4 T , \lambda_{2}(\mathbf{L})=\frac{(T-1)(1-(\mathbf{e}_{1}^{\mathsf{T}}\mathbf{w})^{2})}{T^{2}\lambda_{1}(\mathbf{L})}\ge\frac{(T-1)(1-(\mathbf{e}_{1}^{\mathsf{T}}\mathbf{w})^{2})}{T^{2}}\ge\frac{1}{4T}, λ 2 ( L ) = T 2 λ 1 ( L ) ( T − 1 ) ( 1 − ( e 1 T w ) 2 ) ≥ T 2 ( T − 1 ) ( 1 − ( e 1 T w ) 2 ) ≥ 4 T 1 ,
with probability approaching one, while λ 3 ( L ) = 0 \lambda_{3}(\mathbf{L}) = 0 λ 3 ( L ) = 0 . Hence with the definition
v : = w − e 1 w e 1 1 − ( e 1 T w ) 2 , Q : = e 1 e 1 T + v v T , \mathbf{v} := \frac{\mathbf{w} - \mathbf{e}_{1}\mathbf{w}\mathbf{e}_{1}}{\sqrt{1 - (\mathbf{e}_{1}^{\mathsf{T}}\mathbf{w})^{2}}}, \qquad\mathbf{Q} := \mathbf{e}_{1}\mathbf{e}_{1}^{\mathsf{T}} + \mathbf{v}\mathbf{v}^{\mathsf{T}}, v := 1 − ( e 1 T w ) 2 w − e 1 w e 1 , Q := e 1 e 1 T + v v T ,
by the Davis–Kahan theorem (Yu et al., 2015, Theorem 2), we have
∥ A ^ ( 1 ) ( A ^ ( 1 ) ) T − Q ∥ F ≤ C T ∥ P ^ ( 1 ) − L ∥ = o P ( 1 ) . \left\|\widehat{\mathbf{A}}^{(1)}\left(\widehat{\mathbf{A}}^{(1)}\right)^{\mathsf{T}} - \mathbf{Q}\right\|_{F} \le CT\left\|\widehat{\mathbf{P}}^{(1)} - \mathbf{L}\right\| = o_{P}(1). A ( 1 ) ( A ( 1 ) ) T − Q F ≤ C T P ( 1 ) − L = o P ( 1 ) .
Finally, from E [ ( e 2 T w ) 2 ] = 1 / p \mathbb{E}[(\mathbf{e}_{2}^{\mathsf{T}}\mathbf{w})^{2}] = 1/p E [( e 2 T w ) 2 ] = 1/ p and ( e 1 T w ) 2 = o P ( 1 ) (\mathbf{e}_{1}^{\mathsf{T}}\mathbf{w})^{2} = o_{P}(1) ( e 1 T w ) 2 = o P ( 1 ) , we have
∥ Q − ( e 1 e 1 T + e 2 e 2 T ) ∥ F 2 = 2 { 1 − ( e 2 T v ) 2 } , ( e 2 T v ) 2 = ( e 2 T w ) 2 1 − ( e 1 T w ) 2 = o P ( 1 ) , \left\|\mathbf{Q} - (\mathbf{e}_{1}\mathbf{e}_{1}^{\mathsf{T}} + \mathbf{e}_{2}\mathbf{e}_{2}^{\mathsf{T}})\right\|_{F}^{2} = 2\{1 - (\mathbf{e}_{2}^{\mathsf{T}}\mathbf{v})^{2}\}, \qquad(\mathbf{e}_{2}^{\mathsf{T}}\mathbf{v})^{2} = \frac{(\mathbf{e}_{2}^{\mathsf{T}}\mathbf{w})^{2}}{1 - (\mathbf{e}_{1}^{\mathsf{T}}\mathbf{w})^{2}} = o_{P}(1), Q − ( e 1 e 1 T + e 2 e 2 T ) F 2 = 2 { 1 − ( e 2 T v ) 2 } , ( e 2 T v ) 2 = 1 − ( e 1 T w ) 2 ( e 2 T w ) 2 = o P ( 1 ) ,
so the reverse triangle inequality now gives d ( A ^ ( 1 ) , A ) − 2 = o P ( 1 ) d(\widehat{\mathbf{A}}^{(1)}, \mathbf{A}) - \sqrt{2} = o_{P}(1) d ( A ( 1 ) , A ) − 2 = o P ( 1 ) as desired, which concludes this proof.
Additional discussion of Remark 2
We discuss here the performance of the DC estimator applied with the block size b = 1 b=1 b = 1 in (8) , which fails under our general setup. In particular, we present the following proposition, with its proof in Section A.3.
Proposition 1. Fix any δ > 0 \delta>0 δ > 0 , let r = 2 r=2 r = 2 , and let e 1 , e 2 \mathbf{e}_1,\mathbf{e}_2 e 1 , e 2 be the first two standard basis vectors in R p \mathbb{R}^p R p . Then with κ p ≥ 2 \kappa_p\ge2 κ p ≥ 2 , construct the factor loading matrix
A = [ κ p ( p + p δ ) e 1 , p δ / 2 e 2 ] . \mathbf{A}=\left[\sqrt{\kappa_p(p+p^\delta)}\mathbf{e}_1,p^{\delta/2}\mathbf{e}_2\right]. A = [ κ p ( p + p δ ) e 1 , p δ /2 e 2 ] . Let f t \mathbf{f}_t f t and ϵ t \boldsymbol{\epsilon}_t ϵ t be independent standard Gaussian vectors that are independently across observations.
Also, replace one uniformly selected observation x t \mathbf{x}_t x t by an anomaly vector z ∼ N ( 0 , I p ) \mathbf{z}\sim\mathcal{N}(\mathbf{0},\mathbf{I}_p) z ∼ N ( 0 , I p ) , where z \mathbf{z} z is independent of all latent clean observations, and its location is independent of all latent clean observations and of z \mathbf{z} z . Suppose
1 p , p 2 max { 1 − δ , 0 } T , T log ( κ p ) κ p = o ( 1 ) . (43) \frac{1}{p},\qquad\frac{p^{2\max\{1-\delta,0\}}}{T},\qquad\frac{T\log(\kappa_p)}{\sqrt{\kappa_p}}=o(1). \tag*{(43)} p 1 , T p 2 m a x { 1 − δ , 0 } , κ p T log ( κ p ) = o ( 1 ) . (43) Then the above setup satisfies Assumptions 1 1 1 , 2 2 2 , 3 3 3 , and 4 4 4 , with T ‾ = 1 \overline{T}=1 T = 1 . Then with the notations
P ^ ( 1 ) : = 1 T ∑ t = 1 T x t x t T ∥ x t ∥ 2 , A ^ ( 1 ) : = SVD 2 ( P ^ ( 1 ) ) , (44) \widehat{\mathbf{P}}^{(1)}:=\frac{1}{T}\sum_{t=1}^{T}\frac{\mathbf{x}_t\mathbf{x}_t^{\mathsf{T}}}{\lVert\mathbf{x}_t\rVert^2},\qquad\widehat{\mathbf{A}}^{(1)}:=\operatorname{SVD}_2(\widehat{\mathbf{P}}^{(1)}), \tag*{(44)} P ( 1 ) := T 1 t = 1 ∑ T ∥ x t ∥ 2 x t x t T , A ( 1 ) := SVD 2 ( P ( 1 ) ) , (44) we have d ( A ^ ( 1 ) , A ) − 2 = o P ( 1 ) d(\widehat{\mathbf{A}}^{(1)},\mathbf{A})-\sqrt{2}=o_P(1) d ( A ( 1 ) , A ) − 2 = o P ( 1 ) .
The conditions in (B.1) are compatible for every fixed δ > 0 \delta>0 δ > 0 , such as T ≍ p a T\asymp p^a T ≍ p a for any fixed a > 2 max { 1 − δ , 0 } a>2\max\{1-\delta,0\} a > 2 max { 1 − δ , 0 } , and κ p ≍ T 3 \kappa_p\asymp T^3 κ p ≍ T 3 . Proposition 1 establishes a failure of using block size b = 1 b=1 b = 1 in (8) . In contrast, by our construction with b = r = 2 b=r=2 b = r = 2 , Theorem 1 gives
d ( A ^ 2 , A ) = O P ( p max { 1 − δ , 0 } T + p − δ ) = o P ( 1 ) . d(\widehat{\mathbf{A}}_2,\mathbf{A})=O_P\left(\frac{p^{\max\{1-\delta,0\}}}{\sqrt{T}}+p^{-\delta}\right)=o_P(1). d ( A 2 , A ) = O P ( T p m a x { 1 − δ , 0 } + p − δ ) = o P ( 1 ) .
Additional numerical results
Main numerical experiments under various dimensions
We report here the results as in the experiments corresponding to Figure 1 , but under dimensions ( T , p ) ∈ { ( 50 , 50 ) , ( 200 , 200 ) } (T,p) \in\{(50,50),(200,200)\} ( T , p ) ∈ {( 50 , 50 ) , ( 200 , 200 )} . The results are shown in Figure 7 and Figure 8 , respectively. From these figures, the discussions to Figure 1 remain valid.
Figure 7. Performance of loading space and factor number estimation for ( T , p ) = ( 50 , 50 ) (T,p)=(50,50) ( T , p ) = ( 50 , 50 ) . Within each heatmap, rows correspond to ν ∈ { 0.5 , 1.5 , 2.5 , 4.5 } \nu\in\{0.5,1.5,2.5,4.5\} ν ∈ { 0.5 , 1.5 , 2.5 , 4.5 } and columns correspond to ε ∈ { 0 , 0.05 , 0.1 , 0.2 } \varepsilon\in\{0,0.05,0.1,0.2\} ε ∈ { 0 , 0.05 , 0.1 , 0.2 } . Darker shading indicates larger values within each column.
Figure 8. Performance of loading space and factor number estimation for ( T , p ) = ( 200 , 200 ) (T,p) = (200,200) ( T , p ) = ( 200 , 200 ) . Within each heatmap, rows correspond to ν ∈ { 0.5 , 1.5 , 2.5 , 4.5 } \nu\in\{0.5,1.5,2.5,4.5\} ν ∈ { 0.5 , 1.5 , 2.5 , 4.5 } and columns correspond to ϵ ∈ { 0 , 0.05 , 0.1 , 0.2 } \epsilon\in\{0,0.05,0.1,0.2\} ϵ ∈ { 0 , 0.05 , 0.1 , 0.2 } . Darker shading indicates larger values within each column.
Common component estimation
With Corollary 2, we further investigate common component estimation under heavy tails and anomalies, using the data-generating process in § 5.1 . We fix r = 2 r = 2 r = 2 and a = ϕ = 0 a = \phi= 0 a = ϕ = 0 , and consider ( T , p ) ∈ { ( 50 , 50 ) , ( 100 , 100 ) , ( 200 , 200 ) , ( 400 , 400 ) } (T,p) \in\{(50,50),(100,100),(200,200),(400,400)\} ( T , p ) ∈ {( 50 , 50 ) , ( 100 , 100 ) , ( 200 , 200 ) , ( 400 , 400 )} . For each dimension pair, we vary the degrees of freedom over ν ∈ { 0.5 , 1.5 , 2.5 , 4.5 } \nu\in\{0.5,1.5,2.5,4.5\} ν ∈ { 0.5 , 1.5 , 2.5 , 4.5 } and the fraction of anomalies over ϵ ∈ { 0 , 0.05 , 0.1 , 0.2 } \epsilon\in\{0,0.05,0.1,0.2\} ϵ ∈ { 0 , 0.05 , 0.1 , 0.2 } . The remaining specifications follow the main experiment in § 5.1 . Each configuration is repeated 100 times, with all methods applied to the same observations within each replication.
We compare DC with the seven loading space estimators considered in § 5.1 , namely PCA, RTS, ROBPCA, ROSPCA, MacroPCA, ROBTFA, and RTFA. The true factor number r r r is provided to every method, and hence DC uses b = r = 2 b = r = 2 b = r = 2 . For each estimated loading space, we form its orthogonal projector and estimate the common component by projecting the observed x t x_t x t onto that space, as in (18) . All observations, including anomalies, are used to estimate the loading space. For evaluation, we retain the latent common components generated before any observations are replaced and measure reconstruction error at the clean time points. Specifically, for each replication we compute
( ∑ t ∈ [ T ] ∖ T ∥ χ ^ t − χ t ∥ 2 ∑ t ∈ [ T ] ∖ T ∥ χ t ∥ 2 ) 1 / 2 . (45) \left(\frac{\sum_{t\in[T]\setminus\mathcal{T}}\lVert\widehat{\chi}_t-\chi_t\rVert^2}{\sum_{t\in[T]\setminus\mathcal{T}}\lVert\chi_t\rVert^2}\right)^{1/2}. \tag*{(45)} ( ∑ t ∈ [ T ] ∖ T ∥ χ t ∥ 2 ∑ t ∈ [ T ] ∖ T ∥ χ t − χ t ∥ 2 ) 1/2 . (45)
It is noteworthy that the anomaly indices are only used in evaluation, rather than provided to any estimator. The relative normalization in (C.1) facilitates comparisons across dimensions and tail parameters, and similar metrics are used in e.g. Cen and Lam [8 ] .
Figure 9. Common component estimation under heavy tails and anomalies. Within each heatmap, rows correspond to ν ∈ { 0.5 , 1.5 , 2.5 , 4.5 } \nu\in\{0.5,1.5,2.5,4.5\} ν ∈ { 0.5 , 1.5 , 2.5 , 4.5 } and columns correspond to ε ∈ { 0 , 0.05 , 0.1 , 0.2 } \varepsilon\in\{0,0.05,0.1,0.2\} ε ∈ { 0 , 0.05 , 0.1 , 0.2 } . Cell values report the average relative common component error in (C.1) over 100 replications. Darker shading indicates larger values within each column.
Additional information for the real data analysis
The raw observation at each timestamp consists of 10 × 10 10 \times10 10 × 10 categories with each forming one portfolio, where the stocks are weighted by their market equity. If no stocks are allocated to a category at a given timestamp, the corresponding return is unavailable. The missing pattern is irregular across categories, and dropping four categories to obtain a complete data set, or restricting attention to the period after July 2009, would discard a substantial portion of the information. We therefore adopt the imputed panels of Cen and Lam [8 ] , with the market factor also removed from the dataset following their procedure. Each monthly observation as a 10 × 10 10 \times10 10 × 10 array is then vectorized, and we analyze the resulting vector x t ∈ R 100 \mathbf{x}_t \in\mathbb{R}^{100} x t ∈ R 100 .
Figure 10 reports the estimated factor numbers. Every method gives the same estimate before and after each exclusion, although the selected factor numbers differ across methods.
Figure 10. Estimated factor numbers for the complete sample and the samples obtained by separately removing the windows corresponding to the three events. Blue circles represent complete-sample estimates and orange triangles represent estimates after deletion.