Introduction
We propose a class of factor models designed for analyzing high-dimensional datasets of time series that display nonlinear dynamics and potentially heavy tails. TTo this end, we assume that the latent-factor dynamics are nonlinear and generated by the multivariate score-driven framework introduced by Creal et al. [17 ] and Harvey [34 ] . Our model framework is designed to flexibly accommodate both high-dimensional conditional mean ([31 , 20 ] ) and conditional covariance models ([16 , 13 ] ).
The factor model we consider has three main features. First, it is approximate in the sense that it is characterized by the assumption of weakly cross-sectionally correlated idiosyncratic components, a realistic feature of high-dimensional panels of time series. Second, since the factors are serially dependent, we explicitly model their possibly nonlinear time evolution. Third, although we allow for serially correlated idiosyncratic components, we leave their dynamics unspecified. Our modeling strategy is consistent with most existing literature on factor models for high-dimensional time series (see Stock and Watson, [50 ] , and references therein). The last aspect is what distinguishes our work from existing score-driven factor models, which, however, are studied only under very restrictive assumptions [1 ] .
Estimation is as follows. First, we extract latent factors using Principal Component (PC) analysis [2 ] . Then, we apply the score-driven methodology to model the time-varying parameters of the latent factors, estimating the relevant parameters via Quasi Maximum Likelihood (QMLE) [10 ] . This two-step strategy yields flexible high-dimensional nonlinear time series models that are both statistically tractable and computationally efficient.
It is well known that, in an approximate factor model, the factor-driven common component can be disentangled from the idiosyncratic component only as the cross-sectional dimension N N N diverges [6 ] . Hence our asymptotic theory is developed under the double asymptotic framework N , T → ∞ N,T \to\infty N , T → ∞ , where T T T is the sample size. Furthermore, since the factors are in general not identified, we must develop our theory by explicitly mapping the effect of this indeterminacy on all considered estimators. This task is far from trivial because the estimators of the parameters governing the dynamics of the factors have not a closed form expression.
We make three theoretical contributions. First, we formally derive conditions for strict stationarity, ergodicity, and filter invertibility of our general nonlinear score-driven high-dimensional approximate dynamic factor model in the case of Gaussian and heavy-tailed conditional location (mean) and conditional covariance models. Second, to address the aforementioned factor-identification issue, we develop an asymptotic theory by adapting the framework of Gallant and White [30 ] and Pötscher and Prucha [47 ] for set-identified nonlinear dynamic models, allowing us to establish low-level conditions based on the model’s primitive structure, such as contraction, smoothness and domination conditions. This allows us to derive appropriate convergence rates, ensuring the consistency and asymptotic normality of the proposed QMLE of the parameters driving the score-driven model for the latent factors. Third, we construct asymptotically valid prediction intervals for the filtered process which do not depend on any specific identification condition one might impose on the factors.
Our approach aligns with the recent work of Blasques et al. [9 ] , who introduced the class of quasi score-driven models where the dynamic equation is decoupled from the log-likelihood function while preserving the robustness properties of traditional score-driven models. In this sense, our model can be viewed as a quasi score-driven model, as PC can be formulated within a Gaussian latent variable framework, though the driving force of the score-driven model remains non-Gaussian. Furthermore, PC factor estimation has the same consistency rate as QML estimation [3 , 7 ] . Alternatives to PCs requiring fewer moment constraints are briefly discussed and studied numerically [25 , 5 ] (He et al. 2020).
Related literature. So far, to the best of our knowledge, only two studies develop a complete QML theory for score-driven factor models. While Artemova [1 ] focuses on estimation, Buccheri et al. [14 ] also addresses the factor-identification problem. Unlike our approach, however, both studies keep the cross-sectional dimension fixed and therefore consider exact factor models with cross-sectionally uncorrelated idiosyncratic components. Moreover, their asymptotic theories rely on high-level conditions concerning the convergence of the log-likelihood function and its derivatives. Extending the approach of Artemova [1 ] to a high-dimensional setting is far from trivial, because high-dimensional maximization creates difficulties concerning both the theoretical existence of the maximizers and the numerical convergence of the optimization algorithms employed.
Our work is the first to provide a joint N , T N,T N , T asymptotic theory for high-dimensional factor generalized autoregressive conditional heteroskedasticity (GARCH) models These models, dating back to Diebold and Nerlove [19 ] , Engle et al. [24 ] , and Harvey et al. [36 ] , were developed to address two major challenges: the exponential growth in the number of parameters as dimensionality increases and the algebraic constraints required to ensure the positive definiteness of the conditional covariance matrix. To the best of our knowledge no theory has ever been developed for high-dimensional approximate factor GARCH models. While only a few works address the related estimation theory in a fixed dimensional exact factor model setting [49 , 54 , 55 , 28 , 48 ] , other works extend the model to a high-dimensional setting, but provide either partial theoretical results, or just numerical results [39 , 27 , 57 , 53 ] (Li et al. 2016).
Finally, our theory generalizes to the nonlinear dynamics setting the works on factor augmented regressions, where the dynamics of the factors is assumed to be generated by a linear multivariate autoregressive model. Two steps estimators of such models, having PCA as first step, are studied by Bai and Ng [4 ] , Forni et al. [29 ] , and Doz et al. [23 ] .
The rest of the paper is organized as follows. § 2 outlines the model and its main features. § 3 describes estimation. § 4 contains all the asymptotic properties of the estimators and the required assumptions to derive them. § 5 studies the extension to the joint mean-variance model. § 6 contains simulation results. Additional details on the model, the relation between the score-driven parameters and the factors, all proofs, and more numerical results are in the Appendix.
Basic notation. For a vector v \boldsymbol{v} v , we denote its Euclidean norm by ∥ v ∥ \lVert\boldsymbol{v}\rVert ∥ v ∥ . For a matrix A \boldsymbol{A} A , we denote its spectral norm by ∥ A ∥ \lVert\boldsymbol{A}\rVert ∥ A ∥ and its determinant by ∣ A ∣ \lvert\boldsymbol{A}\rvert ∣ A ∣ . For a random vector x \boldsymbol{x} x , we denote its L p L_p L p -norm by ∥ x ∥ p = ( E [ ∥ x ∥ p ] ) 1 / p \lVert\boldsymbol{x}\rVert_p = \left(\mathbb{E}\left[\lVert\boldsymbol{x}\rVert^p\right]\right)^{1/p} ∥ x ∥ p = ( E [ ∥ x ∥ p ] ) 1/ p , for 1 ≤ p < ∞ 1 \le p < \infty 1 ≤ p < ∞ . Convergence in probability is denoted by ⟶ P \overset{\mathbb{P}}{\longrightarrow} ⟶ P , almost-sure convergence by ⟶ a . s . \overset{a.s.}{\longrightarrow} ⟶ a . s . , exponentially fast almost-sure convergence by ⟶ e . a . s . \overset{e.a.s.}{\longrightarrow} ⟶ e . a . s . , and convergence in distribution by ⇒ \Rightarrow ⇒ . Boundedness in probability and almost surely are denoted by O P O_{\mathbb{P}} O P and O a . s . O_{a.s.} O a . s . , respectively.
Model
We consider a real N N N -valued weakly stationary time series { x t } t ∈ Z \{\boldsymbol{x}_t\}_{t\in\mathbb{Z}} { x t } t ∈ Z , such that x t = ( x 1 t ⋯ x N t ) ⊤ ∈ R N \boldsymbol{x}_t = (x_{1t}\ \cdots\ x_{Nt})^{\top} \in\mathbb{R}^{N} x t = ( x 1 t ⋯ x N t ) ⊤ ∈ R N , for all t ∈ Z t \in\mathbb{Z} t ∈ Z , and, without loss of generality, let x t \boldsymbol{x}_t x t be zero-mean. We consider the following model
x t = Λ F t + e t , t ∈ Z , (1) \boldsymbol{x}_t = \boldsymbol{\Lambda}\boldsymbol{F}_t + \boldsymbol{e}_t, \qquad t \in\mathbb{Z}, \tag*{(1)} x t = Λ F t + e t , t ∈ Z , (1)
F t = q ( ψ t ∣ t − 1 , v t ) , (2) \boldsymbol{F}_t = \boldsymbol{q}\left(\boldsymbol{\psi}_{t\mid t-1}, \boldsymbol{v}_t\right), \tag*{(2)} F t = q ( ψ t ∣ t − 1 , v t ) , (2)
where Λ ∈ R N × r \boldsymbol{\Lambda} \in\mathbb{R}^{N \times r} Λ ∈ R N × r is the matrix of factor loadings, { F t } t ∈ Z \{\boldsymbol{F}_t\}_{t\in\mathbb{Z}} { F t } t ∈ Z is the r r r -dimensional process of latent factors with F t ∈ R r \boldsymbol{F}_t \in\mathbb{R}^{r} F t ∈ R r , for all t ∈ Z t \in\mathbb{Z} t ∈ Z , and r ≪ N r \ll N r ≪ N being the number of factors, { e t } t ∈ Z \{\boldsymbol{e}_t\}_{t\in\mathbb{Z}} { e t } t ∈ Z is the N N N -dimensional process of idiosyncratic components with e t ∈ R N \boldsymbol{e}_t \in\mathbb{R}^{N} e t ∈ R N , for all t ∈ Z t \in\mathbb{Z} t ∈ Z , which are allowed to be weakly serially and cross-correlated, q ( ⋅ , ⋅ ) : R r × R r → R r \boldsymbol{q}(\cdot,\cdot): \mathbb{R}^{r} \times\mathbb{R}^{r} \to\mathbb{R}^{r} q ( ⋅ , ⋅ ) : R r × R r → R r is a link function which determines the (possibly) nonlinear relation between a time-varying parameter { ψ t ∣ t − 1 } t ∈ Z \{\boldsymbol{\psi}_{t\mid t-1}\}_{t\in\mathbb{Z}} { ψ t ∣ t − 1 } t ∈ Z , and a noise process { v t } t ∈ Z \{\boldsymbol{v}_t\}_{t\in\mathbb{Z}} { v t } t ∈ Z , which jointly generate the factor process.
In particular, ψ t ∣ t − 1 \boldsymbol{\psi}_{t\mid t-1} ψ t ∣ t − 1 is a measurable function of the sigma-field F t − 1 = σ { F t − 1 , F t − 2 , … } \mathcal{F}_{t-1} = \sigma\{\boldsymbol{F}_{t-1},\boldsymbol{F}_{t-2},\ldots\} F t − 1 = σ { F t − 1 , F t − 2 , … } generated by { F t } t ∈ Z \{\boldsymbol{F}_t\}_{t\in\mathbb{Z}} { F t } t ∈ Z , hence the subscript t ∣ t − 1 t\mid t-1 t ∣ t − 1 , following a first-order auto-regressive equation of the form
ψ t + 1 ∣ t = δ + Φ ψ t ∣ t − 1 + K u t , t ∈ Z , (3) \boldsymbol{\psi}_{t+1\mid t} = \boldsymbol{\delta} + \boldsymbol{\Phi}\boldsymbol{\psi}_{t\mid t-1} + \boldsymbol{K}\boldsymbol{u}_t, \qquad t \in\mathbb{Z}, \tag*{(3)} ψ t + 1 ∣ t = δ + Φ ψ t ∣ t − 1 + K u t , t ∈ Z , (3)
where u t \boldsymbol{u}_t u t is the score vector with respect to ψ t ∣ t − 1 \boldsymbol{\psi}_{t\mid t-1} ψ t ∣ t − 1 of the quasi log-likelihood contribution of F t \boldsymbol{F}_t F t , conditional on F t − 1 \mathcal{F}_{t-1} F t − 1 , possibly scaled by a proper matrix, e.g., the conditional Information matrix. The scaled-score process, { u t } t ∈ Z \{\boldsymbol{u}_t\}_{t\in\mathbb{Z}} { u t } t ∈ Z , then plays the role of an innovation process for { ψ t ∣ t − 1 } t ∈ Z \{\boldsymbol{\psi}_{t\mid t-1}\}_{t\in\mathbb{Z}} { ψ t ∣ t − 1 } t ∈ Z . In (3) , δ \boldsymbol{\delta} δ is an r r r -dimensional vector of intercepts, whereas Φ \mathbf{\Phi} Φ and K \mathbf{K} K are r × r r \times r r × r matrices of static parameters which drive the dynamics of the unobservable process { ψ t ∣ t − 1 } t ∈ Z \{\boldsymbol{\psi}_{t|t-1}\}_{t\in\mathbb{Z}} { ψ t ∣ t − 1 } t ∈ Z . We denote as θ \boldsymbol{\theta} θ the vector containing the values of the parameters governing (3) , which contains ( δ ′ , vec Φ ′ , vec K ′ ) ′ (\boldsymbol{\delta}^{\prime}, \operatorname{vec}\boldsymbol{\Phi}^{\prime}, \operatorname{vec}\boldsymbol{K}^{\prime})^{\prime} ( δ ′ , vec Φ ′ , vec K ′ ) ′ plus any parameter specific to the chosen distribution. More precisely, let ℓ F ( F t ; θ ) \ell_F(\boldsymbol{F}_t;\boldsymbol{\theta}) ℓ F ( F t ; θ ) be the conditional quasi log-likelihood of F t \boldsymbol{F}_t F t , where, for simplicity of notation, we leave implicit the conditioning on F t − 1 \mathcal{F}_{t-1} F t − 1 . Then, the rescaled score vector is given by
u t = S t − 1 ∂ ℓ F ( F t ; θ ) ∂ ψ t ∣ t − 1 = S t − 1 ∂ ℓ F ( q ( ψ t ∣ t − 1 , v t ) ; θ ) ∂ ψ t ∣ t − 1 , t ∈ Z , \boldsymbol{u}_t = \boldsymbol{S}_t^{-1}\frac{\partial\ell_F(\boldsymbol{F}_t;\boldsymbol{\theta})}{\partial\boldsymbol{\psi}_{t|t-1}} = \boldsymbol{S}_t^{-1}\frac{\partial\ell_F(\boldsymbol{q}(\boldsymbol{\psi}_{t|t-1},\boldsymbol{v}_t);\boldsymbol{\theta})}{\partial\boldsymbol{\psi}_{t|t-1}}, \qquad t \in\mathbb{Z}, u t = S t − 1 ∂ ψ t ∣ t − 1 ∂ ℓ F ( F t ; θ ) = S t − 1 ∂ ψ t ∣ t − 1 ∂ ℓ F ( q ( ψ t ∣ t − 1 , v t ) ; θ ) , t ∈ Z ,
where S t \boldsymbol{S}_t S t is a scaling r × r r \times r r × r matrix and in the second step we used (2) to highlight the dependence on the time-varying parameter, while, for simplicity of notation, we leave implicit the dependence of ψ t ∣ t − 1 \boldsymbol{\psi}_{t|t-1} ψ t ∣ t − 1 and u t \boldsymbol{u}_t u t on θ \boldsymbol{\theta} θ .
Clearly, depending on the considered conditional distribution we obtain a different recurrence equation for the unobservable process { ψ t ∣ t − 1 } t ∈ Z \{\boldsymbol{\psi}_{t|t-1}\}_{t\in\mathbb{Z}} { ψ t ∣ t − 1 } t ∈ Z and a different parameter vector θ \boldsymbol{\theta} θ . Popular examples that satisfy the relation in (2) –(3) are the classical VAR and multivariate GARCH models and score-driven models for multivariate time-varying locations or covariance matrices. In this paper, we consider in detail the four cases listed below.
Time-varying locations. In this case, we suppose that the non-stochastic link function, q ( ⋅ , v t ) \boldsymbol{q}(\cdot,\boldsymbol{v}_t) q ( ⋅ , v t ) , generates the process { F t } t ∈ Z \{\boldsymbol{F}_t\}_{t\in\mathbb{Z}} { F t } t ∈ Z with a linear relation of the following form
F t = q ( ψ t ∣ t − 1 , v t ) = μ t ∣ t − 1 + v t , (4) \boldsymbol{F}_t = \boldsymbol{q}(\boldsymbol{\psi}_{t|t-1},\boldsymbol{v}_t) = \boldsymbol{\mu}_{t|t-1} + \boldsymbol{v}_t, \tag*{(4)} F t = q ( ψ t ∣ t − 1 , v t ) = μ t ∣ t − 1 + v t , (4)
so that in this case ψ t ∣ t − 1 ≡ μ t ∣ t − 1 ∈ R r \boldsymbol{\psi}_{t|t-1} \equiv\boldsymbol{\mu}_{t|t-1} \in\mathbb{R}^{r} ψ t ∣ t − 1 ≡ μ t ∣ t − 1 ∈ R r is the vector of time-varying conditional locations. In particular, (3) assumes the following form (recall that we assumed F t \boldsymbol{F}_t F t to have zero mean)
μ t + 1 ∣ t = Φ μ t ∣ t − 1 + K u t . (5) \boldsymbol{\mu}_{t+1|t} = \boldsymbol{\Phi}\boldsymbol{\mu}_{t|t-1} + \boldsymbol{K}\boldsymbol{u}_t. \tag*{(5)} μ t + 1∣ t = Φ μ t ∣ t − 1 + K u t . (5)
Gaussian updates. We assume that v t ∼ N ( 0 r , Σ ) \boldsymbol{v}_t \sim\mathcal{N}(\boldsymbol{0}_r,\boldsymbol{\Sigma}) v t ∼ N ( 0 r , Σ ) , i.e., { v t } t ∈ Z \{\boldsymbol{v}_t\}_{t\in\mathbb{Z}} { v t } t ∈ Z is Gaussian with r r r -dimensional zero location vector 0 r \boldsymbol{0}_r 0 r and Σ ∈ R r × r \boldsymbol{\Sigma} \in\mathbb{R}^{r \times r} Σ ∈ R r × r positive definite covariance matrix. The corresponding conditional quasi log-likelihood contribution is
ℓ F ( F t ; θ ) = − r 2 log ( 2 π ) − 1 2 log ∣ Σ ∣ − 1 2 ( F t − μ t ∣ t − 1 ) T Σ − 1 ( F t − μ t ∣ t − 1 ) , (6) \ell_F(\boldsymbol{F}_t;\boldsymbol{\theta}) = -\frac{r}{2}\log(2\pi) - \frac{1}{2}\log\lvert\boldsymbol{\Sigma}\rvert- \frac{1}{2}(\boldsymbol{F}_t-\boldsymbol{\mu}_{t|t-1})^{\mathsf{T}}\boldsymbol{\Sigma}^{-1}(\boldsymbol{F}_t-\boldsymbol{\mu}_{t|t-1}), \tag*{(6)} ℓ F ( F t ; θ ) = − 2 r log ( 2 π ) − 2 1 log ∣ Σ ∣ − 2 1 ( F t − μ t ∣ t − 1 ) T Σ − 1 ( F t − μ t ∣ t − 1 ) , (6)
and it is straightforward to show that, when we set S t = Σ − 1 \boldsymbol{S}_t=\boldsymbol{\Sigma}^{-1} S t = Σ − 1 , u t = F t − μ t ∣ t − 1 \boldsymbol{u}_t=\boldsymbol{F}_t-\boldsymbol{\mu}_{t|t-1} u t = F t − μ t ∣ t − 1 . From (5) and (6) we see that the unknown parameter vector is given by θ = ( vech Σ ⊤ , vec Φ ⊤ , vec K ⊤ ) ⊤ \boldsymbol{\theta} = (\operatorname{vech}\boldsymbol{\Sigma}^{\top}, \operatorname{vec}\boldsymbol{\Phi}^{\top}, \operatorname{vec}\boldsymbol{K}^{\top})^{\top} θ = ( vech Σ ⊤ , vec Φ ⊤ , vec K ⊤ ) ⊤ , which has dimension d = r ( r + 1 ) / 2 + 2 r 2 d = r(r+1)/2 + 2r^{2} d = r ( r + 1 ) /2 + 2 r 2 .
Student’s t t t updates. Following D’Innocenzo et al. [20 ] we assume that v t ∼ t ν ( 0 r , Σ ) \boldsymbol{v}_{t} \sim t_{\nu}(\boldsymbol{0}_{r}, \boldsymbol{\Sigma}) v t ∼ t ν ( 0 r , Σ ) , i.e., { v t } t ∈ Z \{\boldsymbol{v}_{t}\}_{t\in\mathbb{Z}} { v t } t ∈ Z follows a multivariate Student’s t t t distribution with ν > 0 \nu> 0 ν > 0 degrees of freedom, an r r r -dimensional zero location vector 0 r \boldsymbol{0}_{r} 0 r and Σ ∈ R r × r \boldsymbol{\Sigma} \in\mathbb{R}^{r\times r} Σ ∈ R r × r positive definite scale matrix. The conditional log-likelihood of the factors is:
ℓ F ( F t ; θ ) = log Γ ( ν + r 2 ) − log Γ ( ν 2 ) − r 2 log ( π ν ) − 1 2 log ∣ Σ ∣ − ν + r 2 log ( 1 + ( F t − μ t ∣ t − 1 ) ⊤ Σ − 1 ( F t − μ t ∣ t − 1 ) ν ) . (7) \ell_{F}(\boldsymbol{F}_{t};\boldsymbol{\theta}) = \log\Gamma\left(\frac{\nu+r}{2}\right) - \log\Gamma\left(\frac{\nu}{2}\right) - \frac{r}{2}\log(\pi\nu) - \frac{1}{2}\log|\boldsymbol{\Sigma}| - \frac{\nu+r}{2}\log\left(1+\frac{(\boldsymbol{F}_{t}-\boldsymbol{\mu}_{t|t-1})^{\top}\boldsymbol{\Sigma}^{-1}(\boldsymbol{F}_{t}-\boldsymbol{\mu}_{t|t-1})}{\nu}\right). \tag*{(7)} ℓ F ( F t ; θ ) = log Γ ( 2 ν + r ) − log Γ ( 2 ν ) − 2 r log ( π ν ) − 2 1 log ∣ Σ ∣ − 2 ν + r log ( 1 + ν ( F t − μ t ∣ t − 1 ) ⊤ Σ − 1 ( F t − μ t ∣ t − 1 ) ) . (7)
Thus, the scaled-score vector assumes the following form u t = w t ( F t − μ t ∣ t − 1 ) \boldsymbol{u}_{t} = w_{t}(\boldsymbol{F}_{t}-\boldsymbol{\mu}_{t|t-1}) u t = w t ( F t − μ t ∣ t − 1 ) , with w t = { 1 + ( F t − μ t ∣ t − 1 ) ⊤ Σ − 1 ( F t − μ t ∣ t − 1 ) / ν } − 1 w_{t} = \left\{1+(\boldsymbol{F}_{t}-\boldsymbol{\mu}_{t|t-1})^{\top}\boldsymbol{\Sigma}^{-1}(\boldsymbol{F}_{t}-\boldsymbol{\mu}_{t|t-1})/\nu\right\}^{-1} w t = { 1 + ( F t − μ t ∣ t − 1 ) ⊤ Σ − 1 ( F t − μ t ∣ t − 1 ) / ν } − 1 . From (5) and (7) we see that the unknown parameter vector is given by θ = ( ν , vech Σ ⊤ , vec Φ ⊤ , vec K ⊤ ) ⊤ \boldsymbol{\theta} = (\nu,\operatorname{vech}\boldsymbol{\Sigma}^{\top},\operatorname{vec}\boldsymbol{\Phi}^{\top},\operatorname{vec}\boldsymbol{K}^{\top})^{\top} θ = ( ν , vech Σ ⊤ , vec Φ ⊤ , vec K ⊤ ) ⊤ , which has dimension d = 1 + r ( r + 1 ) / 2 + 2 r 2 d = 1+r(r+1)/2+2r^{2} d = 1 + r ( r + 1 ) /2 + 2 r 2 . When ν → ∞ \nu\to\infty ν → ∞ , the model with Student’s t t t score updates reduces to the Gaussian ones.
Time-varying variances. In this case, we suppose that the non-stochastic link function, q ( ⋅ , v t ) \boldsymbol{q}(\cdot,\boldsymbol{v}_{t}) q ( ⋅ , v t ) , generates the process { F t } t ∈ Z \{\boldsymbol{F}_{t}\}_{t\in\mathbb{Z}} { F t } t ∈ Z with a multiplicative relation of the following form
F t = q ( ψ t ∣ t − 1 , v t ) = Σ t ∣ t − 1 1 / 2 v t , (8) \boldsymbol{F}_{t} = \boldsymbol{q}\left(\boldsymbol{\psi}_{t|t-1},\boldsymbol{v}_{t}\right) = \boldsymbol{\Sigma}_{t|t-1}^{1/2}\boldsymbol{v}_{t}, \tag*{(8)} F t = q ( ψ t ∣ t − 1 , v t ) = Σ t ∣ t − 1 1/2 v t , (8)
such that Σ t ∣ t − 1 ∈ R r × r \boldsymbol{\Sigma}_{t|t-1} \in\mathbb{R}^{r\times r} Σ t ∣ t − 1 ∈ R r × r is its associated time-varying conditional covariance matrix assumed to be positive definite, and with Σ t ∣ t − 1 1 / 2 \boldsymbol{\Sigma}_{t|t-1}^{1/2} Σ t ∣ t − 1 1/2 we denote its square root. To model the full covariance matrix, we assume that the time-varying conditional covariance matrix has a diagonal structure, such that Σ t ∣ t − 1 = diag ( σ t ∣ t − 1 2 ) \boldsymbol{\Sigma}_{t|t-1} = \operatorname{diag}(\boldsymbol{\sigma}^{2}_{t|t-1}) Σ t ∣ t − 1 = diag ( σ t ∣ t − 1 2 ) , with σ t ∣ t − 1 2 = ( σ 1 , t ∣ t − 1 2 , … , σ r , t ∣ t − 1 2 ) ⊤ \boldsymbol{\sigma}^{2}_{t|t-1} = (\sigma^{2}_{1,t|t-1},\ldots,\sigma^{2}_{r,t|t-1})^{\top} σ t ∣ t − 1 2 = ( σ 1 , t ∣ t − 1 2 , … , σ r , t ∣ t − 1 2 ) ⊤ ([49 ] ). Thus, in this case ψ t ∣ t − 1 ≡ σ t ∣ t − 1 2 ∈ R r \boldsymbol{\psi}_{t|t-1} \equiv\boldsymbol{\sigma}^{2}_{t|t-1} \in\mathbb{R}^{r} ψ t ∣ t − 1 ≡ σ t ∣ t − 1 2 ∈ R r is the time-varying vector of conditional variances. Then, (3) specified for σ t ∣ t − 1 2 \boldsymbol{\sigma}^{2}_{t|t-1} σ t ∣ t − 1 2 becomes
σ t + 1 ∣ t 2 = δ + Φ σ t ∣ t − 1 2 + K u t . (9) \boldsymbol{\sigma}^{2}_{t+1|t} = \boldsymbol{\delta} + \boldsymbol{\Phi}\boldsymbol{\sigma}^{2}_{t|t-1} + \boldsymbol{K}\boldsymbol{u}_{t}. \tag*{(9)} σ t + 1∣ t 2 = δ + Φ σ t ∣ t − 1 2 + K u t . (9)
Furthermore, consider the r 2 × r r^{2}\times r r 2 × r matrix W r = ∑ i = 1 r ( e i ⊗ E i ) \boldsymbol{W}_{r} = \sum_{i=1}^{r}(\boldsymbol{e}_{i}\otimes\boldsymbol{E}_{i}) W r = ∑ i = 1 r ( e i ⊗ E i ) , where, for i = 1 , … , r i=1,\ldots,r i = 1 , … , r , E i ∈ R r × r \boldsymbol{E}_{i}\in\mathbb{R}^{r\times r} E i ∈ R r × r with all zero entries except for the identity in ( i , i ) (i,i) ( i , i ) , and e i ∈ R r \boldsymbol{e}_{i}\in\mathbb{R}^{r} e i ∈ R r is a vector with all zeros except for the i i i -th position. Notice also that W r ⊤ W r = I r \boldsymbol{W}_{r}^{\top}\boldsymbol{W}_{r}=\boldsymbol{I}_{r} W r ⊤ W r = I r . Then, vec Σ t ∣ t − 1 = W r σ t ∣ t − 1 2 \operatorname{vec}\boldsymbol{\Sigma}_{t|t-1}=\boldsymbol{W}_{r}\boldsymbol{\sigma}^{2}_{t|t-1} vec Σ t ∣ t − 1 = W r σ t ∣ t − 1 2 and σ t ∣ t − 1 2 = W r ⊤ vec Σ t ∣ t − 1 \boldsymbol{\sigma}^{2}_{t|t-1}=\boldsymbol{W}_{r}^{\top}\operatorname{vec}\boldsymbol{\Sigma}_{t|t-1} σ t ∣ t − 1 2 = W r ⊤ vec Σ t ∣ t − 1 . Notice that while it is clear that we can always derive σ t ∣ t − 1 2 \boldsymbol{\sigma}^{2}_{t|t-1} σ t ∣ t − 1 2 from the full matrix Σ t ∣ t − 1 \boldsymbol{\Sigma}_{t|t-1} Σ t ∣ t − 1 , in our setting the vice versa is also possible since we assume the matrix Σ t ∣ t − 1 \boldsymbol{\Sigma}_{t|t-1} Σ t ∣ t − 1 to be diagonal for each t t t . The recursion in (9) can also be extended to incorporate leverage effects as detailed in Appendix § 8 .
Gaussian updates. Analogously to the previous dynamic location, if we assume that v t ∼ N ( 0 r , I r ) \boldsymbol{v}_t \sim\mathcal{N}(\boldsymbol{0}_r,\boldsymbol{I}_r) v t ∼ N ( 0 r , I r ) , i.e., { v t } t ∈ Z \{\boldsymbol{v}_t\}_{t\in\mathbb{Z}} { v t } t ∈ Z follows a multivariate standard Gaussian distribution, the conditional log-likelihood of the factors is:
ℓ F ( F t ; θ ) = − r 2 log ( 2 π ) − 1 2 ( log ∣ Σ t ∣ t − 1 ∣ + F t T Σ t ∣ t − 1 − 1 F t ) . (10) \ell_F(F_t;\boldsymbol{\theta})=-\frac{r}{2}\log(2\pi)-\frac{1}{2}\left(\log\left|\boldsymbol{\Sigma}_{t|t-1}\right|+F_t^{\mathsf{T}}\boldsymbol{\Sigma}_{t|t-1}^{-1}F_t\right). \tag*{(10)} ℓ F ( F t ; θ ) = − 2 r log ( 2 π ) − 2 1 ( log Σ t ∣ t − 1 + F t T Σ t ∣ t − 1 − 1 F t ) . (10)
and the scaled-score vector assumes the following form u t = W r T ( ( F t ⊗ F t ) − vec Σ t ∣ t − 1 ) \boldsymbol{u}_t=\boldsymbol{W}_r^{\mathsf{T}}\left((F_t\otimes F_t)-\operatorname{vec}\boldsymbol{\Sigma}_{t|t-1}\right) u t = W r T ( ( F t ⊗ F t ) − vec Σ t ∣ t − 1 ) . From (9) and (10) we see that the unknown parameter vector is given by θ = ( δ T , vec Φ T , vec K T ) T \boldsymbol{\theta}=(\boldsymbol{\delta}^{\mathsf{T}},\operatorname{vec}\boldsymbol{\Phi}^{\mathsf{T}},\operatorname{vec}\boldsymbol{K}^{\mathsf{T}})^{\mathsf{T}} θ = ( δ T , vec Φ T , vec K T ) T , which has dimension d = r + 2 r 2 d=r+2r^2 d = r + 2 r 2 .
Student’s t updates. Here, the time-varying conditional covariance matrix is driven by the score of the multivariate Student’s t t t density, see Creal et al. [16 ] . In particular, we assume that v t ∼ t ν ( 0 r , I r ) \boldsymbol{v}_t\sim t_\nu(\boldsymbol{0}_r,\boldsymbol{I}_r) v t ∼ t ν ( 0 r , I r ) , i.e., { v t } t ∈ Z \{\boldsymbol{v}_t\}_{t\in\mathbb{Z}} { v t } t ∈ Z follows a multivariate standard Student’s t t t distribution with ν > 2 \nu>2 ν > 2 degrees of freedom, an r r r -dimensional zero mean vector 0 r \boldsymbol{0}_r 0 r , and I r ∈ R r × r \boldsymbol{I}_r\in\mathbb{R}^{r\times r} I r ∈ R r × r as scale matrix. The conditional log-likelihood of the factors is:
ℓ F ( F t ; θ ) = log Γ ( ν + r 2 ) − log Γ ( ν 2 ) − r 2 log ( π ( ν − 2 ) ) − 1 2 ( log ∣ Σ t ∣ t − 1 ∣ + ( ν + r ) log ( 1 + F t T Σ t ∣ t − 1 − 1 F t ν − 2 ) ) . (11) \ell_F(F_t;\boldsymbol{\theta})=\log\Gamma\left(\frac{\nu+r}{2}\right)-\log\Gamma\left(\frac{\nu}{2}\right)-\frac{r}{2}\log\left(\pi(\nu-2)\right)-\frac{1}{2}\left(\log\left|\boldsymbol{\Sigma}_{t|t-1}\right|+(\nu+r)\log\left(1+\frac{F_t^{\mathsf{T}}\boldsymbol{\Sigma}_{t|t-1}^{-1}F_t}{\nu-2}\right)\right). \tag*{(11)} ℓ F ( F t ; θ ) = log Γ ( 2 ν + r ) − log Γ ( 2 ν ) − 2 r log ( π ( ν − 2 ) ) − 2 1 ( log Σ t ∣ t − 1 + ( ν + r ) log ( 1 + ν − 2 F t T Σ t ∣ t − 1 − 1 F t ) ) . (11)
Following the proof of Creal et al. [16 ] we set (see Appendix A for more details)
u t = W r T ( ν + r ν − 2 + F t T Σ t ∣ t − 1 − 1 F t ( F t ⊗ F t ) − vec Σ t ∣ t − 1 ) . \boldsymbol{u}_t=\boldsymbol{W}_r^{\mathsf{T}}\left(\frac{\nu+r}{\nu-2+F_t^{\mathsf{T}}\boldsymbol{\Sigma}_{t|t-1}^{-1}F_t}(F_t\otimes F_t)-\operatorname{vec}\boldsymbol{\Sigma}_{t|t-1}\right). u t = W r T ( ν − 2 + F t T Σ t ∣ t − 1 − 1 F t ν + r ( F t ⊗ F t ) − vec Σ t ∣ t − 1 ) .
This choice greatly simplifies calculations and is in line with the discussion by Opschoor et al. [45 ] . From (9) and (11) the unknown parameter vector is given by θ = ( ν , δ T , vec Φ T , vec K T ) T \boldsymbol{\theta}=(\nu,\boldsymbol{\delta}^{\mathsf{T}},\operatorname{vec}\boldsymbol{\Phi}^{\mathsf{T}},\operatorname{vec}\boldsymbol{K}^{\mathsf{T}})^{\mathsf{T}} θ = ( ν , δ T , vec Φ T , vec K T ) T , which has dimension d = 1 + r + 2 r 2 d=1+r+2r^2 d = 1 + r + 2 r 2 . When ν → ∞ \nu\to\infty ν → ∞ , the model with Student’s t t t score updates reduces to the Gaussian ones.
Estimation
Factors. Consider the N × N N \times N N × N sample covariance matrix Γ ^ x = T − 1 ∑ t = 1 T x t x t T \widehat{\Gamma}_x = T^{-1}\sum_{t=1}^{T} x_t x_t^{\mathsf{T}} Γ x = T − 1 ∑ t = 1 T x t x t T , with normalized eigenvectors corresponding to its r r r largest eigenvalues collected in the columns of the N × r N \times r N × r matrix P ^ \widehat{P} P . Then, the PC estimators are given by
Λ ^ = N P ^ , (12) \widehat{\Lambda} = \sqrt{N}\widehat{P}, \tag*{(12)} Λ = N P , (12)
F ^ t = P ^ T x t N . (13) \widehat{F}_t = \frac{\widehat{P}^{\mathsf{T}}x_t}{\sqrt{N}}. \tag*{(13)} F t = N P T x t . (13)
Notice that in this case N − 1 Λ ^ T Λ ^ = I r N^{-1}\widehat{\Lambda}^{\mathsf{T}}\widehat{\Lambda} = I_r N − 1 Λ T Λ = I r and T − 1 ∑ t = 1 T F ^ t F ^ t T T^{-1}\sum_{t=1}^{T}\widehat{F}_t\widehat{F}_t^{\mathsf{T}} T − 1 ∑ t = 1 T F t F t T is an r × r r \times r r × r diagonal matrix containing the eigenvalues of Γ ^ x \widehat{\Gamma}_x Γ x divided by N N N . This is a standard formulation of PC estimators used, e.g., by Stock and Watson [51 ] and differs from the one in Bai [2 ] just by a different rescaling of the estimated loadings. Alternatives to PCs are discussed in the next section.
Remark 1. We notice that a Student’s t t t with ν = 4 + δ \nu= 4 + \delta ν = 4 + δ , for some δ > 0 \delta> 0 δ > 0 , degrees of freedom would satisfy our assumptions for PC to be consistent but still allow for quite heavy tails. In fact, as shown in Gasperoni et al. [31 ] , for a standard r r r -variate Student’s t t t the convergence rate to the Gaussian slows with the dimension r r r , and a conservative bound for Gaussian-like integration constant is ν ≃ 30 r 2 \nu\simeq30r^2 ν ≃ 30 r 2 . Therefore, a moderate r r r and a ν \nu ν slightly larger than 4 make the Student’s t t t behave closer to Gaussian for covariance/eigenspace accuracy, while preserving tail-robustness. Hence, the PC approach is likely to allow us to still capture heavy-tailed reality. This is confirmed by the numerical results in § 6 .
Score-driven model parameters. For observed factors F = ( F 1 ⋯ F T ) T F = (F_1 \cdots F_T)^{\mathsf{T}} F = ( F 1 ⋯ F T ) T it is clear that we can estimate the unknown vector of parameters in θ \boldsymbol{\theta} θ by maximizing the quasi log-likelihood ∑ t = 1 T ℓ F ( F t ; θ ) \sum_{t=1}^{T}\ell_F(F_t;\boldsymbol{\theta}) ∑ t = 1 T ℓ F ( F t ; θ ) with respect to θ \boldsymbol{\theta} θ . However, F t F_t F t , μ t ∣ t − 1 \boldsymbol{\mu}_{t\mid t-1} μ t ∣ t − 1 , and σ t ∣ t − 1 2 \boldsymbol{\sigma}_{t\mid t-1}^{2} σ t ∣ t − 1 2 are not observable. Still, given the first estimation step, we can replace F t F_t F t with F ^ t \widehat{F}_t F t for t = 1 , … , T t = 1,\ldots,T t = 1 , … , T , which, in turn, can be employed to obtain the score-driven filtered paths of μ ^ t ∣ t − 1 ( θ ) \widehat{\boldsymbol{\mu}}_{t\mid t-1}(\boldsymbol{\theta}) μ t ∣ t − 1 ( θ ) and Σ ^ t ∣ t − 1 ( θ ) \widehat{\boldsymbol{\Sigma}}_{t\mid t-1}(\boldsymbol{\theta}) Σ t ∣ t − 1 ( θ ) or σ ^ t ∣ t − 1 2 ( θ ) \widehat{\boldsymbol{\sigma}}_{t\mid t-1}^{2}(\boldsymbol{\theta}) σ t ∣ t − 1 2 ( θ ) for any given θ \boldsymbol{\theta} θ . This will give rise to the approximate and feasible quasi log-likelihood function ∑ t = 1 T ℓ ^ F ( F ^ t ; θ ) \sum_{t=1}^{T}\widehat{\ell}_F(\widehat{F}_t;\boldsymbol{\theta}) ∑ t = 1 T ℓ F ( F t ; θ ) , which is a function of the estimated factors F ^ t \widehat{F}_t F t , of the score-driven filters μ ^ t ∣ t − 1 ( θ ) \widehat{\boldsymbol{\mu}}_{t\mid t-1}(\boldsymbol{\theta}) μ t ∣ t − 1 ( θ ) and σ ^ t ∣ t − 1 2 ( θ ) \widehat{\boldsymbol{\sigma}}_{t\mid t-1}^{2}(\boldsymbol{\theta}) σ t ∣ t − 1 2 ( θ ) , for t = 1 , … , T t = 1,\ldots,T t = 1 , … , T , and of θ \boldsymbol{\theta} θ . Thus, we consider the QMLE obtained by solving the following problem:
θ ^ = arg max θ ∈ Θ ⊂ R d ∑ t = 1 T ℓ ^ F ( F ^ t ; θ ) . (14) \widehat{\boldsymbol{\theta}} = \underset{\boldsymbol{\theta}\in\Theta\subset\mathbb{R}^{d}}{\operatorname{arg\,max}}\sum_{t=1}^{T}\widehat{\ell}_F(\widehat{F}_t;\boldsymbol{\theta}). \tag*{(14)} θ = θ ∈ Θ ⊂ R d arg max t = 1 ∑ T ℓ F ( F t ; θ ) . (14)
In general the true factors F t \mathbf{F}_t F t are identified only up to left-multiplication by an invertible r × r r \times r r × r matrix. Therefore, any derived asymptotic property must take the effect of such indeterminacy fully into consideration. This discussed in detail below. Here we just note that, while QMLE is classical in both the correctly specified or misspecified score-driven literature (see, for instance, Blasques et al. [9 ] ), the presence of estimated factors makes our overall model only set-identified at the first stage, according to the taxonomy of Gallant and White [30 ] and Pötscher and Prucha [47 ] .
Remark 2. In principle, the loadings could be estimated via QML jointly with the score-driven parameters. Besides the need to avoid computational issues discussed in the Introduction, our choice has also two theoretical motivations, which hold at least when considering a Gaussian likelihood. First, Barigozzi and Luciani [7 ] prove that, for large N N N , the QMLE in case of serially correlated factors is asymptotically equivalent to the QMLE of the case in which factors follow a dynamic parametric model. Second, Bai and Li [3 ] prove that, for large N N N , the QMLE of the loadings has the same convergence rate as the PCA estimator.
Asymptotic theory
Hereafter, the factor model in (1) is defined for an infinite dimensional process { x i t , i ∈ N , t ∈ Z } \{x_{it}, i \in\mathbb{N}, t \in\mathbb{Z}\} { x i t , i ∈ N , t ∈ Z } , and the properties and related assumptions are defined for the N N N -dimensional sub-process { x i t , i = 1 … , N , t ∈ Z } \{x_{it}, i = 1 \ldots, N, t \in\mathbb{Z}\} { x i t , i = 1 … , N , t ∈ Z } either holding for all N ∈ N N \in\mathbb{N} N ∈ N or in the limit N → ∞ N \to\infty N → ∞ . The properties of the estimators are then derived for a given N T NT N T -dimensional realization { x i t , i = 1 … , N , t = 1 , … , T } \{x_{it}, i = 1 \ldots, N, t = 1, \ldots, T\} { x i t , i = 1 … , N , t = 1 , … , T } and hold in the limit N , T → ∞ N, T \to\infty N , T → ∞ .
Throughout this section, for any i ∈ N i \in\mathbb{N} i ∈ N , we denote with λ 0 i \boldsymbol{\lambda}_{0i} λ 0 i the r r r -dimensional vector of true factor loadings of { x i t } t ∈ Z \{x_{it}\}_{t\in\mathbb{Z}} { x i t } t ∈ Z , with λ 0 i ∈ R r \boldsymbol{\lambda}_{0i} \in\mathbb{R}^r λ 0 i ∈ R r . Then, for any N ∈ N N \in\mathbb{N} N ∈ N , we denote with Λ 0 = ( λ 01 ⋯ λ 0 N ) ⊤ \boldsymbol{\Lambda}_0 = (\boldsymbol{\lambda}_{01} \cdots\boldsymbol{\lambda}_{0N})^\top Λ 0 = ( λ 01 ⋯ λ 0 N ) ⊤ the true N × r N \times r N × r matrix of loadings, with Λ 0 ∈ R N × r \boldsymbol{\Lambda}_0 \in\mathbb{R}^{N\times r} Λ 0 ∈ R N × r . We also denote with θ 0 \boldsymbol{\theta}_0 θ 0 the d d d -dimensional vector of true score-driven parameters, with θ 0 ∈ Θ ⊂ R d \boldsymbol{\theta}_0 \in\Theta\subset\mathbb{R}^d θ 0 ∈ Θ ⊂ R d . The true conditional mean or variance processes are then generically indicated as { ψ t ∣ t − 1 ( θ 0 ) } t ∈ Z \{\psi_{t|t-1}(\boldsymbol{\theta}_0)\}_{t\in\mathbb{Z}} { ψ t ∣ t − 1 ( θ 0 ) } t ∈ Z , such that ψ t ∣ t − 1 ( θ 0 ) ∈ P ⊂ R r \psi_{t|t-1}(\boldsymbol{\theta}_0) \in\mathcal{P} \subset\mathbb{R}^r ψ t ∣ t − 1 ( θ 0 ) ∈ P ⊂ R r for all t ∈ Z t \in\mathbb{Z} t ∈ Z , and with either ψ t ∣ t − 1 ( θ 0 ) = μ t ∣ t − 1 ( θ 0 ) \psi_{t|t-1}(\boldsymbol{\theta}_0) = \boldsymbol{\mu}_{t|t-1}(\boldsymbol{\theta}_0) ψ t ∣ t − 1 ( θ 0 ) = μ t ∣ t − 1 ( θ 0 ) or ψ t ∣ t − 1 ( θ 0 ) = σ t ∣ t − 1 2 ( θ 0 ) \psi_{t|t-1}(\boldsymbol{\theta}_0) = \boldsymbol{\sigma}^2_{t|t-1}(\boldsymbol{\theta}_0) ψ t ∣ t − 1 ( θ 0 ) = σ t ∣ t − 1 2 ( θ 0 ) , as defined in § 2 . The process of true factors is denoted as { F t } t ∈ Z \{\mathbf{F}_t\}_{t\in\mathbb{Z}} { F t } t ∈ Z , such that F t ∈ R r \mathbf{F}_t \in\mathbb{R}^r F t ∈ R r for all t ∈ Z t \in\mathbb{Z} t ∈ Z , and the true rescaled score, i.e., computed when using the true factors and parameters θ 0 \boldsymbol{\theta}_0 θ 0 , is denoted as { u t } t ∈ Z \{\mathbf{u}_t\}_{t\in\mathbb{Z}} { u t } t ∈ Z , such that u t ∈ R r \mathbf{u}_t \in\mathbb{R}^r u t ∈ R r for all t ∈ Z t \in\mathbb{Z} t ∈ Z .
Hence, the model in (1) –(3) now reads:
x i t = λ 0 i ⊤ F t + e i t , i ∈ N , t ∈ Z , (15) x_{it} = \boldsymbol{\lambda}_{0i}^{\top}\mathbf{F}_t + e_{it}, \qquad i \in\mathbb{N}, \quad t \in\mathbb{Z}, \tag*{(15)} x i t = λ 0 i ⊤ F t + e i t , i ∈ N , t ∈ Z , (15)
F t = q ( ψ t ∣ t − 1 ( θ 0 ) , v t ) , (16) F_t = q\left(\psi_{t\mid t-1}(\theta_0),v_t\right), \tag*{(16)} F t = q ( ψ t ∣ t − 1 ( θ 0 ) , v t ) , (16)
ψ t + 1 ∣ t ( θ 0 ) = δ 0 + Φ 0 ψ t ∣ t − 1 ( θ 0 ) + K 0 u t ( θ 0 ) . (17) \psi_{t+1\mid t}(\theta_0) = \delta_0 + \Phi_0\psi_{t\mid t-1}(\theta_0) + K_0u_t(\theta_0). \tag*{(17)} ψ t + 1 ∣ t ( θ 0 ) = δ 0 + Φ 0 ψ t ∣ t − 1 ( θ 0 ) + K 0 u t ( θ 0 ) . (17)
Furthermore, a generic trajectory of the conditional mean or variance is denoted as { ψ t ∣ t − 1 } t ∈ Z \{\psi_{t\mid t-1}\}_{t\in\mathbb{Z}} { ψ t ∣ t − 1 } t ∈ Z such that ψ t ∣ t − 1 ∈ P ⊂ R r \psi_{t\mid t-1}\in\mathcal{P}\subset\mathbb{R}^r ψ t ∣ t − 1 ∈ P ⊂ R r for all t ∈ Z t\in\mathbb{Z} t ∈ Z . Although this distinction implies that we should introduce a new notation for generic factors generated as F t = q ( ψ t ∣ t − 1 , v t ) F_t=q(\psi_{t\mid t-1},v_t) F t = q ( ψ t ∣ t − 1 , v t ) , and the corresponding rescaled score, we refrain from doing so, since, in the following, the only case in which { F t } t ∈ Z \{F_t\}_{t\in\mathbb{Z}} { F t } t ∈ Z indicates a generic factor process is when we consider derivatives of the log-likelihood or of the rescaled score with respect to F t F_t F t .
Asymptotic properties of the estimated factors and their loadings
We start by making the following assumptions on the noise process { v t } t ∈ Z \{v_t\}_{t\in\mathbb{Z}} { v t } t ∈ Z and the link function q ( ⋅ , ⋅ ) q(\cdot,\cdot) q ( ⋅ , ⋅ ) .
Assumption 1. *(i) The process { v t } t ∈ Z \{v_t\}_{t\in\mathbb{Z}} { v t } t ∈ Z is geometrically β \beta β -mixing. In particular, letting β v ( h ) \beta_v(h) β v ( h ) denote its β \beta β -mixing coefficients, there exist a finite positive real C β C_\beta C β and 0 < ρ β < 1 0<\rho_\beta<1 0 < ρ β < 1 such that β v ( h ) ≤ C β ρ β h \beta_v(h)\le C_\beta\rho_\beta^h β v ( h ) ≤ C β ρ β h for every h ≥ 1 h\ge1 h ≥ 1 . Moreover, E [ v t ] = 0 r \mathbb{E}[v_t]=\mathbf{0}_r E [ v t ] = 0 r and E [ v t v t T ] \mathbb{E}[v_tv_t^\mathsf{T}] E [ v t v t T ] is finite and positive definite for all t ∈ Z t\in\mathbb{Z} t ∈ Z . (ii) The distribution of { v t } t ∈ Z \{v_t\}_{t\in\mathbb{Z}} { v t } t ∈ Z has a density strictly positive in a neighbourhood of 0 with respect to the Lebesgue measure on R r \mathbb{R}^r R r . (iii) The link function q : R r × R r → R r q:\mathbb{R}^r\times\mathbb{R}^r\to\mathbb{R}^r q : R r × R r → R r is non-stochastic, Borel measurable and continuously differentiable in its first argument. Moreover, for the same p > 2 p>2 p > 2 as in Assumptions 2 and 7, there exist Borel measurable functions B t : R r × R r → [ 0 , ∞ ) B_t:\mathbb{R}^r\times\mathbb{R}^r\to[0,\infty) B t : R r × R r → [ 0 , ∞ ) such that, for all z , z ⋆ ∈ R r z,z^\star\in\mathbb{R}^r z , z ⋆ ∈ R r , ∥ q ( z , v t ) − q ( z ⋆ , v t ) ∥ ≤ B t ( z , z ⋆ ) ∥ z − z ⋆ ∥ \lVert q(z,v_t)-q(z^\star,v_t)\rVert\le B_t(z,z^\star)\lVert z-z^\star\rVert ∥ q ( z , v t ) − q ( z ⋆ , v t )∥ ≤ B t ( z , z ⋆ ) ∥ z − z ⋆ ∥ , and sup t ∈ Z sup z , z ⋆ ∈ R r ∥ B t ( z , z ⋆ ) ∥ 2 p p − 2 ≤ B \sup_{t\in\mathbb{Z}}\sup_{z,z^\star\in\mathbb{R}^r}\lVert B_t(z,z^\star)\rVert_{\frac{2p}{p-2}}\le B sup t ∈ Z sup z , z ⋆ ∈ R r ∥ B t ( z , z ⋆ ) ∥ p − 2 2 p ≤ B , for some finite positive real B B B independent of t t t .
Part (i) imposes centering and non-degeneracy of the innovations and controls their temporal dependence. Geometric β \beta β -mixing is used together with the near-epoch dependence results below in the finite-window empirical-process argument that yields the uniform root-T T T rate in Proposition 3. Since every β \beta β -mixing process is also α \alpha α -mixing, it also provides the mixing condition required by the near-epoch dependent central limit theorem underlying Theorem 2. Part (ii) rules out degenerate innovation distributions and guarantees sufficient local support for the score-driven process. Finally, part (iii) ensures that perturbations of the filtered parameter process translate into controlled perturbations of the factor process. This Lipschitz property will be used in Proposition 2, in Lemma 6, and consequently in the likelihood approximation of Proposition 3.
For the factor process { F t } t ∈ Z \{F_t\}_{t\in\mathbb{Z}} { F t } t ∈ Z and the true stochastic parameter process { ψ t ∣ t − 1 ( θ 0 ) } t ∈ Z \{\psi_{t\mid t-1}(\theta_0)\}_{t\in\mathbb{Z}} { ψ t ∣ t − 1 ( θ 0 ) } t ∈ Z we assume the following.
Assumption 2. (i) The process { F t } t ∈ Z \{\mathbf{F}_t\}_{t\in\mathbb{Z}} { F t } t ∈ Z with F t = 0 r \mathbf{F}_t=\mathbf{0}_r F t = 0 r for t ≤ 0 t\leq0 t ≤ 0 , converges as t → ∞ t\to\infty t → ∞ to a unique strictly stationary and ergodic process { F ˉ t } t ∈ Z \{\bar{\mathbf{F}}_t\}_{t\in\mathbb{Z}} { F ˉ t } t ∈ Z , and is such that E [ F t ] = 0 r \mathbb{E}[\mathbf{F}_t]=\mathbf{0}_r E [ F t ] = 0 r and E [ F t F t ⊤ ] = Γ F \mathbb{E}[\mathbf{F}_t\mathbf{F}_t^\top]=\boldsymbol{\Gamma}_{F} E [ F t F t ⊤ ] = Γ F finite and positive definite. (ii) The process { ψ t ∣ t − 1 ( θ 0 ) } t ∈ Z \{\psi_{t|t-1}(\boldsymbol{\theta}_0)\}_{t\in\mathbb{Z}} { ψ t ∣ t − 1 ( θ 0 ) } t ∈ Z solving (17) when using { F t } t ∈ Z \{\mathbf{F}_t\}_{t\in\mathbb{Z}} { F t } t ∈ Z and initialized with some random variable ψ 1 ∣ 0 ∈ P \psi_{1|0}\in\mathcal{P} ψ 1∣0 ∈ P with ∥ ψ 1 ∣ 0 ∥ p ≤ C 0 \|\psi_{1|0}\|_p\leq C_0 ∥ ψ 1∣0 ∥ p ≤ C 0 for p > 2 p>2 p > 2 and some finite positive real C 0 C_0 C 0 , converges as t → ∞ t\to\infty t → ∞ to a unique strictly stationary and ergodic { ψ ˉ t ∣ t − 1 ( θ 0 ) } t ∈ Z \{\bar{\psi}_{t|t-1}(\boldsymbol{\theta}_0)\}_{t\in\mathbb{Z}} { ψ ˉ t ∣ t − 1 ( θ 0 ) } t ∈ Z solution of (17) computed when using { F ˉ t } t ∈ Z \{\bar{\mathbf{F}}_t\}_{t\in\mathbb{Z}} { F ˉ t } t ∈ Z . (iii) For all t ∈ Z t\in\mathbb{Z} t ∈ Z , E [ ∥ F t ∥ 4 ] ≤ Q F \mathbb{E}[\|\mathbf{F}_t\|^4]\leq Q_F E [ ∥ F t ∥ 4 ] ≤ Q F for some finite positive real Q F Q_F Q F independent of t t t .
Parts (i) and (ii) are standard in score-driven models and can be substituted with appropriate conditions on the parameters of the score-driven recursion in (3) ([20 ] ). Part (iii) is standard in PC literature ([2 ] ).
Remark 3. Under Assumptions 1-2 we obtain that ∥ T − 1 ∑ t = 1 T F t F t ⊤ − Γ F ∥ = O P ( T − 1 / 2 ) \|T^{-1}\sum_{t=1}^{T}\mathbf{F}_t\mathbf{F}_t^\top-\boldsymbol{\Gamma}_F\|=O_{\mathbb{P}}(T^{-1/2}) ∥ T − 1 ∑ t = 1 T F t F t ⊤ − Γ F ∥ = O P ( T − 1/2 ) as T → ∞ T\to\infty T → ∞ , which directly follows from an application of the ergodic theorem for real-valued sequences of random variables of Birkhoff [8 ] .
For the loadings, we make the following assumption.
Assumption 3. As N → ∞ N\to\infty N → ∞ , ∥ N − 1 Λ 0 ⊤ Λ 0 − Σ Λ ∥ = O P ( N − 1 / 2 ) \|N^{-1}\boldsymbol{\Lambda}_0^\top\boldsymbol{\Lambda}_0-\boldsymbol{\Sigma}_{\Lambda}\|=O_{\mathbb{P}}(N^{-1/2}) ∥ N − 1 Λ 0 ⊤ Λ 0 − Σ Λ ∥ = O P ( N − 1/2 ) for some r × r r\times r r × r finite and positive definite matrix Σ Λ \boldsymbol{\Sigma}_{\Lambda} Σ Λ , and, for all i ∈ N i\in\mathbb{N} i ∈ N , ∥ λ 0 i ∥ ≤ M Λ \|\boldsymbol{\lambda}_{0i}\|\leq M_{\Lambda} ∥ λ 0 i ∥ ≤ M Λ for some finite positive real M Λ M_{\Lambda} M Λ independent of i i i .
Note that the rate N \sqrt{N} N is often not given in the literature but it is reasonable to assume it in absence of any specific ordering of the cross-sectional items ([6 ] ). This setting corresponds to considering only strongly pervasive factors.
For the idiosyncratic process { e t } t ∈ Z \{e_t\}_{t\in\mathbb{Z}} { e t } t ∈ Z , we make the following assumptions.
Assumption 4. (i) For all i ∈ N i\in\mathbb{N} i ∈ N and t ∈ Z t\in\mathbb{Z} t ∈ Z , E [ e i t ] = 0 \mathbb{E}[e_{it}]=0 E [ e i t ] = 0 and σ i 2 = E [ e i t 2 ] \sigma_i^2=\mathbb{E}[e_{it}^2] σ i 2 = E [ e i t 2 ] such that σ i 2 ∈ [ C e , C e ′ ] \sigma_i^2\in[C_e,C'_e] σ i 2 ∈ [ C e , C e ′ ] for some finite positive reals C e C_e C e and C e ′ C'_e C e ′ independent of i i i and t t t . (ii) For all i , j ∈ N i,j\in\mathbb{N} i , j ∈ N , t ∈ Z t\in\mathbb{Z} t ∈ Z , and k ∈ Z k\in\mathbb{Z} k ∈ Z , ∣ E [ e i t e j , t − k ] ∣ ≤ ρ ∣ k ∣ M i j |\mathbb{E}[e_{it}e_{j,t-k}]|\leq\rho^{|k|}M_{ij} ∣ E [ e i t e j , t − k ] ∣ ≤ ρ ∣ k ∣ M ij , where ρ \rho ρ and M i j M_{ij} M ij are finite positive reals independent of t t t such that 0 ≤ ρ < 1 0\leq\rho<1 0 ≤ ρ < 1 , M i i = σ i 2 M_{ii}=\sigma_i^2 M ii = σ i 2 , ∑ j = 1 , i ≠ j N M i j ≤ M e \sum_{j=1,i\ne j}^{N}M_{ij}\leq M_e ∑ j = 1 , i = j N M ij ≤ M e , and ∑ i = 1 , j ≠ i N M i j ≤ M e \sum_{i=1,j\ne i}^{N}M_{ij}\leq M_e ∑ i = 1 , j = i N M ij ≤ M e for some finite positive real M e M_e M e independent of i i i , j j j , and N N N . (iii) For any fixed t ∈ Z t\in\mathbb{Z} t ∈ Z , as N → ∞ N\to\infty N → ∞ , N − 1 / 2 ∑ i = 1 N λ 0 i e i t ⇒ N ( 0 , Π t , t ) N^{-1/2}\sum_{i=1}^{N}\boldsymbol{\lambda}_{0i}e_{it}\Rightarrow\mathcal{N}(\mathbf{0},\boldsymbol{\Pi}_{t,t}) N − 1/2 ∑ i = 1 N λ 0 i e i t ⇒ N ( 0 , Π t , t ) , with Π s , u = lim N → ∞ N − 1 ∑ i = 1 N ∑ j = 1 N λ 0 i λ 0 j ⊤ E [ e i s e j u ] \boldsymbol{\Pi}_{s,u}=\lim_{N\to\infty}N^{-1}\sum_{i=1}^{N}\sum_{j=1}^{N}\boldsymbol{\lambda}_{0i}\boldsymbol{\lambda}_{0j}^{\top}\mathbb{E}[e_{is}e_{ju}] Π s , u = lim N → ∞ N − 1 ∑ i = 1 N ∑ j = 1 N λ 0 i λ 0 j ⊤ E [ e i s e j u ] . (iv) For all i = 1 , … , N i=1,\ldots,N i = 1 , … , N , t = 1 , … , T t=1,\ldots,T t = 1 , … , T , and N , T ∈ N N,T\in\mathbb{N} N , T ∈ N , E [ e i t 4 ] ≤ Q e \mathbb{E}[e_{it}^4]\leq Q_e E [ e i t 4 ] ≤ Q e for some finite positive real Q e Q_e Q e independent of i i i and t t t . (v) For all s , u = 1 , … , T s,u=1,\ldots,T s , u = 1 , … , T and N , T ∈ N N,T\in\mathbb{N} N , T ∈ N , E [ ∣ N − 1 / 2 ∑ i = 1 N { e i u e i s − E [ e i u e i s ] } ∣ 2 ] ≤ K e \mathbb{E}\left[\left|N^{-1/2}\sum_{i=1}^{N}\{e_{iu}e_{is}-\mathbb{E}[e_{iu}e_{is}]\}\right|^2\right]\leq K_e E [ N − 1/2 ∑ i = 1 N { e i u e i s − E [ e i u e i s ]} 2 ] ≤ K e , for some finite positive real K e K_e K e independent of s s s , u u u , N N N , and T T T .
Regarding the relation between idiosyncratic components and factors, we assume the following.
Assumption 5. (i) The processes { F t } t ∈ Z \{\mathbf{F}_t\}_{t\in\mathbb{Z}} { F t } t ∈ Z and { e i t } t ∈ Z \{e_{it}\}_{t\in\mathbb{Z}} { e i t } t ∈ Z are independent for all i ∈ N i \in\mathbb{N} i ∈ N . (ii) For all t = 1 , … , T t = 1,\ldots,T t = 1 , … , T and N , T ∈ N N,T \in\mathbb{N} N , T ∈ N , E [ ∥ ( N T ) − 1 / 2 ∑ s = 1 T ∑ k = 1 N F s ( e k s e k t − E [ e k s e k t ] ) ∥ 2 ] ≤ K e , 1 \mathbb{E}\left[\left\|(NT)^{-1/2}\sum_{s=1}^{T}\sum_{k=1}^{N}\mathbf{F}_s(e_{ks}e_{kt}-\mathbb{E}[e_{ks}e_{kt}])\right\|^2\right] \le K_{e,1} E [ ( N T ) − 1/2 ∑ s = 1 T ∑ k = 1 N F s ( e k s e k t − E [ e k s e k t ]) 2 ] ≤ K e , 1 , for some finite positive real K e , 1 K_{e,1} K e , 1 independent of t t t , N N N , and T T T . (iii) For all N , T ∈ N N,T \in\mathbb{N} N , T ∈ N , E [ ∥ ( N T ) − 1 / 2 ∑ t = 1 T ∑ k = 1 N F t λ 0 k ⊤ e k t ∥ 2 ] ≤ K e , 2 \mathbb{E}\left[\left\|(NT)^{-1/2}\sum_{t=1}^{T}\sum_{k=1}^{N}\mathbf{F}_t\lambda_{0k}^{\top}e_{kt}\right\|^2\right] \le K_{e,2} E [ ( N T ) − 1/2 ∑ t = 1 T ∑ k = 1 N F t λ 0 k ⊤ e k t 2 ] ≤ K e , 2 , for some finite positive real K e , 2 K_{e,2} K e , 2 independent of N N N and T T T . (iv) For any i ∈ N i \in\mathbb{N} i ∈ N , as T → ∞ T \to\infty T → ∞ , T − 1 / 2 ∑ t = 1 T F t e i t ⇒ N ( 0 r , Ξ i ) T^{-1/2}\sum_{t=1}^{T}\mathbf{F}_te_{it} \Rightarrow\mathcal{N}(\mathbf{0}_r,\Xi_i) T − 1/2 ∑ t = 1 T F t e i t ⇒ N ( 0 r , Ξ i ) , with Ξ i = lim T → ∞ T − 1 ∑ t = 1 T ∑ s = 1 T E [ F t F s ⊤ ] E [ e i t e i s ] \Xi_i = \lim_{T\to\infty}T^{-1}\sum_{t=1}^{T}\sum_{s=1}^{T}\mathbb{E}[\mathbf{F}_t\mathbf{F}_s^{\top}]\mathbb{E}[e_{it}e_{is}] Ξ i = lim T → ∞ T − 1 ∑ t = 1 T ∑ s = 1 T E [ F t F s ⊤ ] E [ e i t e i s ] . (v) The number of factors r r r is independent of N N N and the r r r eigenvalues of Γ F Σ Λ \Gamma_F\Sigma_\Lambda Γ F Σ Λ are distinct, where Γ F \Gamma_F Γ F and Σ Λ \Sigma_\Lambda Σ Λ are defined in Assumptions 2(i) and 3, respectively.
As shown in Appendix § 10 , the Assumptions 1–5 nest all assumptions in Bai [2 ] , hence we can easily adapt the proofs therein despite using here the definition of PC estimators by Stock and Watson [51 ] . The estimated loadings and factors are then consistently retrieving the true ones up to an invertible linear transformation.
Proposition 1. Under Assumptions 1–5, for any i = 1 , … , N i = 1,\ldots,N i = 1 , … , N and t = 1 , … , T t = 1,\ldots,T t = 1 , … , T , there exists an r × r r \times r r × r matrix H ^ \widehat{H} H , depending on N N N and T T T , such that, H 0 = P - lim N , T → ∞ H ^ \mathbf{H}_0 = \mathbb{P}\text{-}\lim_{N,T\to\infty}\widehat{\mathbf{H}} H 0 = P - lim N , T → ∞ H is r × r r \times r r × r finite and positive definite and, as N , T → ∞ N,T \to\infty N , T → ∞ : (a) ∥ F ^ t − H ^ F t ∥ = O P ( N − 1 / 2 + T − 1 ) \|\widehat{\mathbf{F}}_t-\widehat{\mathbf{H}}\mathbf{F}_t\| = O_{\mathbb{P}}(N^{-1/2}+T^{-1}) ∥ F t − H F t ∥ = O P ( N − 1/2 + T − 1 ) ; (b) if N / T → 0 \sqrt{N}/T \to0 N / T → 0 , N ( F ^ t − H ^ F t ) ⇒ N ( 0 r , H 0 Σ Λ − 1 Π t , t Σ Λ − 1 H 0 ⊤ ) \sqrt{N}(\widehat{\mathbf{F}}_t-\widehat{\mathbf{H}}\mathbf{F}_t) \Rightarrow\mathcal{N}(\mathbf{0}_r,\mathbf{H}_0\Sigma_\Lambda^{-1}\Pi_{t,t}\Sigma_\Lambda^{-1}\mathbf{H}_0^\top) N ( F t − H F t ) ⇒ N ( 0 r , H 0 Σ Λ − 1 Π t , t Σ Λ − 1 H 0 ⊤ ) , where Π t , t \Pi_{t,t} Π t , t is defined in Assumption 4(iii); (c) ∥ λ ^ i − H ^ − ⊤ λ 0 i ∥ = O P ( N − 1 + T − 1 / 2 ) \|\widehat{\lambda}_i-\widehat{\mathbf{H}}^{-\top}\lambda_{0i}\| = O_{\mathbb{P}}(N^{-1}+T^{-1/2}) ∥ λ i − H − ⊤ λ 0 i ∥ = O P ( N − 1 + T − 1/2 ) ; (d) if T / N → 0 \sqrt{T}/N \to0 T / N → 0 , T ( λ ^ i − H ^ − ⊤ λ 0 i ) ⇒ N ( 0 r , H 0 − ⊤ Γ F − 1 Ξ i Γ F − 1 H 0 − 1 ) \sqrt{T}(\widehat{\lambda}_i-\widehat{\mathbf{H}}^{-\top}\lambda_{0i}) \Rightarrow\mathcal{N}(\mathbf{0}_r,\mathbf{H}_0^{-\top}\Gamma_F^{-1}\Xi_i\Gamma_F^{-1}\mathbf{H}_0^{-1}) T ( λ i − H − ⊤ λ 0 i ) ⇒ N ( 0 r , H 0 − ⊤ Γ F − 1 Ξ i Γ F − 1 H 0 − 1 ) , where Ξ i \Xi_i Ξ i is defined in Assumption 5(iv); (e) ∥ F ^ t − H 0 F t ∥ = O P ( N − 1 / 2 + T − 1 / 2 ) \|\widehat{\mathbf{F}}_t-\mathbf{H}_0\mathbf{F}_t\| = O_{\mathbb{P}}(N^{-1/2}+T^{-1/2}) ∥ F t − H 0 F t ∥ = O P ( N − 1/2 + T − 1/2 ) and ∥ λ ^ i − H 0 − ⊤ λ 0 i ∥ = O P ( N − 1 / 2 + T − 1 / 2 ) \|\widehat{\lambda}_i-\mathbf{H}_0^{-\top}\lambda_{0i}\| = O_{\mathbb{P}}(N^{-1/2}+T^{-1/2}) ∥ λ i − H 0 − ⊤ λ 0 i ∥ = O P ( N − 1/2 + T − 1/2 ) .
While parts (a)-(d) are standard, part (e) simply reflects the fact that by Remark 3 and Assumption 3, ∥ H ^ − H 0 ∥ = O P ( N − 1 / 2 + T − 1 / 2 ) \|\widehat{\mathbf{H}}-\mathbf{H}_0\| = O_{\mathbb{P}}(N^{-1/2}+T^{-1/2}) ∥ H − H 0 ∥ = O P ( N − 1/2 + T − 1/2 ) , a result needed for proving the following results.
Remark 4. At least three other alternatives to PCs are possible which do not require finite 4th moments. First, we could use PCs based on the spatial Kendall’s tau matrix ([26 ] (He et al. 2020)), which require no moment conditions and have the same consistency rates as in Proposition 1. However, this approach is computationally very intensive and its consistency is proved only in the case of serially independent data following a multivariate elliptical distribution. In Appendix § 17 .§ 2 we show numerical results when using this methodology. Second, we could use diversified projections ([25 ] ), which are N \sqrt{N} N -consistent for the factors and require moment conditions only for the idiosyncratic components. However, their efficacy is based on considering more projections than the number of factors and it is not clear how to choose the related weighted schemes. Last, we could use robust PCs, which are based on thresholding the data and require the existence of at least 2nd moments ([5 ] ). For a fixed threshold this approach has the same consistency rates as in Proposition 1. However, a threshold pre-selection step can be highly computationally demanding.
Asymptotic properties of estimated time-varying mean and variance
We introduce the following contraction and moment conditions for the scaled-score process { u t } t ∈ Z \{u_t\}_{t\in\mathbb{Z}} { u t } t ∈ Z and its derivatives. Recall that, at any given t ∈ Z t\in\mathbb{Z} t ∈ Z , u t u_t u t is a function of F t F_t F t and therefore also of ψ t ∣ t − 1 \psi_{t|t-1} ψ t ∣ t − 1 .
Assumption 6. (i) Θ \Theta Θ is a compact subset of R d \mathbb{R}^d R d . (ii) The process { u t ( θ ) } t ∈ Z \{u_t(\theta)\}_{t\in\mathbb{Z}} { u t ( θ ) } t ∈ Z is such that E [ log + sup θ ∈ Θ ∥ u t ( θ ) ∥ ] ≤ K u \mathbb{E}[\log^+\sup_{\theta\in\Theta}\|u_t(\theta)\|]\leq K_u E [ log + sup θ ∈ Θ ∥ u t ( θ ) ∥ ] ≤ K u , for some finite positive real K u K_u K u independent of t t t , and where log + ( x ) = max { 0 , log ( x ) } \log^+(x)=\max\{0,\log(x)\} log + ( x ) = max { 0 , log ( x )} . (iii) Let X t , k ( θ ) = sup ψ ∈ P ∥ ∏ j = 1 k X t − j + 1 ( θ ) ∥ X_{t,k}(\theta)=\sup_{\psi\in\mathcal{P}}\left\|\prod_{j=1}^{k}X_{t-j+1}(\theta)\right\| X t , k ( θ ) = sup ψ ∈ P ∏ j = 1 k X t − j + 1 ( θ ) with X t ( θ ) = Φ + K ∂ u t ( θ ) ∂ ψ t ∣ t − 1 ∣ ψ t ∣ t − 1 = ψ t ∣ t − 1 ( θ ) X_t(\theta)=\Phi+K\left.\frac{\partial u_t(\theta)}{\partial\psi_{t|t-1}}\right|_{\psi_{t|t-1}=\psi_{t|t-1}(\theta)} X t ( θ ) = Φ + K ∂ ψ t ∣ t − 1 ∂ u t ( θ ) ψ t ∣ t − 1 = ψ t ∣ t − 1 ( θ ) then, for some k ≥ 1 k\geq1 k ≥ 1 , E [ log sup θ ∈ Θ ∣ X t , k ( θ ) ∣ ] < 0 \mathbb{E}[\log\sup_{\theta\in\Theta}|X_{t,k}(\theta)|]<0 E [ log sup θ ∈ Θ ∣ X t , k ( θ ) ∣ ] < 0 .
Assumption 7. (i) For some k ≥ 1 k\geq1 k ≥ 1 , sup θ ∈ Θ ∣ X t , k ( θ ) ∣ ≤ X k \sup_{\theta\in\Theta}|X_{t,k}(\theta)|\leq\mathcal{X}_k sup θ ∈ Θ ∣ X t , k ( θ ) ∣ ≤ X k , almost surely, for finite positive real X k < 1 \mathcal{X}_k<1 X k < 1 independent of t t t , with X t , k ( θ ) X_{t,k}(\theta) X t , k ( θ ) defined in Assumption 6. (ii) Let, Y t ( θ ) = sup ψ ∈ P ∥ Y t ( θ ) ∥ Y_t(\theta)=\sup_{\psi\in\mathcal{P}}\|Y_t(\theta)\| Y t ( θ ) = sup ψ ∈ P ∥ Y t ( θ ) ∥ with Y t ( θ ) = K ∂ u t ( θ ) ∂ F t ∣ ψ t ∣ t − 1 = ψ t ∣ t − 1 ( θ ) Y_t(\theta)=\left.K\frac{\partial u_t(\theta)}{\partial F_t}\right|_{\psi_{t|t-1}=\psi_{t|t-1}(\theta)} Y t ( θ ) = K ∂ F t ∂ u t ( θ ) ψ t ∣ t − 1 = ψ t ∣ t − 1 ( θ ) , then, for some k ≥ 1 k\geq1 k ≥ 1 ,
max { ∥ ∑ j = 1 k sup θ ∈ Θ ∣ Y t − j + 1 ( θ ) ∣ ∥ F ˉ t − j + 1 − F t − j + 1 ∥ ∥ p , ∥ ∑ j = 1 k sup θ ∈ Θ ∣ Y t − j + 1 ( θ ) ∣ ∥ p } ≤ Y k , \max\left\{\left\|\sum_{j=1}^{k}\sup_{\theta\in\Theta}|Y_{t-j+1}(\theta)|\|\bar{F}_{t-j+1}-F_{t-j+1}\|\right\|_{p},\left\|\sum_{j=1}^{k}\sup_{\theta\in\Theta}|Y_{t-j+1}(\theta)|\right\|_{p}\right\}\leq\mathcal{Y}_k, max ⎩ ⎨ ⎧ j = 1 ∑ k θ ∈ Θ sup ∣ Y t − j + 1 ( θ ) ∣∥ F ˉ t − j + 1 − F t − j + 1 ∥ p , j = 1 ∑ k θ ∈ Θ sup ∣ Y t − j + 1 ( θ ) ∣ p ⎭ ⎬ ⎫ ≤ Y k , for finite positive real Y k < ∞ \mathcal{Y}_k<\infty Y k < ∞ independent of t t t , and p > 2 p>2 p > 2 with F ˉ t \bar{F}_t F ˉ t defined in Assumption 2.
We then establish a preliminary result concerning the approximation of the unobserved stochastic parameter process. Let { ψ ˉ t ∣ t − 1 ( θ ) } t ∈ Z \{\bar{\psi}_{t|t-1}(\theta)\}_{t\in\mathbb{Z}} { ψ ˉ t ∣ t − 1 ( θ ) } t ∈ Z denote the unique stationary and ergodic solution of (17) , evaluated at some generic θ ∈ Θ \theta\in\Theta θ ∈ Θ and driven by the transformed stationary and ergodic factors { H 0 F ˉ t } t ∈ Z \{H_0\bar{F}_t\}_{t\in\mathbb{Z}} { H 0 F ˉ t } t ∈ Z . Let { ψ ~ t ∣ t − 1 ( θ ) } t ∈ N \{\tilde{\psi}_{t|t-1}(\theta)\}_{t\in\mathbb{N}} { ψ ~ t ∣ t − 1 ( θ ) } t ∈ N denote the filtered sequence driven by { H 0 F ˉ t } t ∈ Z \{H_0\bar{F}_t\}_{t\in\mathbb{Z}} { H 0 F ˉ t } t ∈ Z initialized at the fixed value ψ 1 ∣ 0 ∗ \psi^*_{1|0} ψ 1∣0 ∗ . Moreover, let { ψ ^ t ∣ t − 1 ( θ ) } t = 1 T \{\hat{\psi}_{t|t-1}(\theta)\}_{t=1}^{T} { ψ ^ t ∣ t − 1 ( θ ) } t = 1 T be the filter driven by { H ^ F t } t = 1 T \{\widehat{H}F_t\}_{t=1}^{T} { H F t } t = 1 T and initialized at ψ 1 ∣ 0 ∗ \psi^*_{1|0} ψ 1∣0 ∗ , and let { ψ ^ ^ t ∣ t − 1 ( θ ) } t = 1 T \{\widehat{\hat{\psi}}_{t|t-1}(\theta)\}_{t=1}^{T} { ψ ^ t ∣ t − 1 ( θ ) } t = 1 T be the feasible filter driven by { F ^ t } t = 1 T \{\widehat{F}_t\}_{t=1}^{T} { F t } t = 1 T and initialized at ψ 1 ∣ 0 \psi_{1|0} ψ 1∣0 .
We consider the decomposition
ψ ^ ^ t ∣ t − 1 ( θ ) − ψ ˉ t ∣ t − 1 ( θ ) = ( ψ ^ ^ t ∣ t − 1 ( θ ) − ψ ^ t ∣ t − 1 ( θ ) ) + ( ψ ^ t ∣ t − 1 ( θ ) − ψ ~ t ∣ t − 1 ( θ ) ) + ( ψ ~ t ∣ t − 1 ( θ ) − ψ ˉ t ∣ t − 1 ( θ ) ) . (18) \widehat{\hat{\psi}}_{t|t-1}(\theta)-\bar{\psi}_{t|t-1}(\theta)
=
\left(\widehat{\hat{\psi}}_{t|t-1}(\theta)-\hat{\psi}_{t|t-1}(\theta)\right)
+\left(\hat{\psi}_{t|t-1}(\theta)-\tilde{\psi}_{t|t-1}(\theta)\right)
+\left(\tilde{\psi}_{t|t-1}(\theta)-\bar{\psi}_{t|t-1}(\theta)\right). \tag*{(18)} ψ ^ t ∣ t − 1 ( θ ) − ψ ˉ t ∣ t − 1 ( θ ) = ( ψ ^ t ∣ t − 1 ( θ ) − ψ ^ t ∣ t − 1 ( θ ) ) + ( ψ ^ t ∣ t − 1 ( θ ) − ψ ~ t ∣ t − 1 ( θ ) ) + ( ψ ~ t ∣ t − 1 ( θ ) − ψ ˉ t ∣ t − 1 ( θ ) ) . (18)
The first term reflects the factor-estimation error F ^ t − H ^ F t \widehat{F}_t-\widehat{H}F_t F t − H F t together with the possibly different filter initializations. The second term is generated by H ^ F t − H 0 F ˉ t = ( H ^ − H 0 ) F t + H 0 ( F t − F ˉ t ) \widehat{H}F_t-H_0\bar{F}_t=(\widehat{H}-H_0)F_t+H_0(F_t-\bar{F}_t) H F t − H 0 F ˉ t = ( H − H 0 ) F t + H 0 ( F t − F ˉ t ) , and therefore captures both the transformation-estimation error and the convergence of the factor process { F t } t ∈ N \{F_t\}_{t\in\mathbb{N}} { F t } t ∈ N to its unique stationary solution { F ˉ t } t ∈ Z \{\bar{F}_t\}_{t\in\mathbb{Z}} { F ˉ t } t ∈ Z . The third term contains only the effect of the filter initialization, since both processes are driven by the same stationary factor sequence. The asymptotic behavior of these terms is given in the following proposition.
Proposition 2. Under Assumptions 1–7, let k ≥ 1 k \ge1 k ≥ 1 and 0 < X k < 1 0 < \mathcal{X}_k < 1 0 < X k < 1 be as in Assumption 7. Then, for some ϱ k \varrho_k ϱ k such that X k < ϱ k ≤ 1 \mathcal{X}_k < \varrho_k \le1 X k < ϱ k ≤ 1 and any fixed initial conditions ψ 1 ∣ 0 , ψ 1 ∣ 0 ∗ ∈ P \psi_{1|0}, \psi^*_{1|0} \in\mathcal{P} ψ 1∣0 , ψ 1∣0 ∗ ∈ P , for any t ≥ 1 t \ge1 t ≥ 1 , as N , T → ∞ N,T \to\infty N , T → ∞ : (a) sup θ ∈ Θ ∥ ψ ^ t + 1 ∣ t ( θ ) − ψ ^ t + 1 ∣ t ∥ = O P ( N − 1 / 2 + T − 1 ) + O a . s . ( X k ⌊ t / k ⌋ ) \sup_{\theta\in\Theta}\left\|\widehat{\psi}_{t+1|t}(\theta)-\widehat{\psi}_{t+1|t}\right\|=O_P(N^{-1/2}+T^{-1})+O_{a.s.}(\mathcal{X}_k^{\lfloor t/k\rfloor}) sup θ ∈ Θ ψ t + 1∣ t ( θ ) − ψ t + 1∣ t = O P ( N − 1/2 + T − 1 ) + O a . s . ( X k ⌊ t / k ⌋ ) ; (b) sup θ ∈ Θ ∥ ψ ^ t ∣ t − 1 ( θ ) − ψ ~ t ∣ t − 1 ( θ ) ∥ = O P ( N − 1 / 2 + T − 1 / 2 ) + O a . s . ( ϱ k ⌊ t / k ⌋ ) \sup_{\theta\in\Theta}\left\|\widehat{\psi}_{t|t-1}(\theta)-\widetilde{\psi}_{t|t-1}(\theta)\right\|=O_P(N^{-1/2}+T^{-1/2})+O_{a.s.}(\varrho_k^{\lfloor t/k\rfloor}) sup θ ∈ Θ ψ t ∣ t − 1 ( θ ) − ψ t ∣ t − 1 ( θ ) = O P ( N − 1/2 + T − 1/2 ) + O a . s . ( ϱ k ⌊ t / k ⌋ ) ; (c) sup θ ∈ Θ ∥ ψ ~ t ∣ t − 1 ( θ ) − ψ ˉ t ∣ t − 1 ( θ ) ∥ = O a . s . ( ϱ k ⌊ t / k ⌋ ) \sup_{\theta\in\Theta}\left\|\widetilde{\psi}_{t|t-1}(\theta)-\bar{\psi}_{t|t-1}(\theta)\right\|=O_{a.s.}(\varrho_k^{\lfloor t/k\rfloor}) sup θ ∈ Θ ψ t ∣ t − 1 ( θ ) − ψ ˉ t ∣ t − 1 ( θ ) = O a . s . ( ϱ k ⌊ t / k ⌋ ) ; (d) the process { ψ ˉ t ∣ t − 1 ( θ 0 ) } t ∈ N \{\bar{\psi}_{t|t-1}(\theta_0)\}_{t\in\mathbb{N}} { ψ ˉ t ∣ t − 1 ( θ 0 ) } t ∈ N is near-epoch dependent of size − 1 -1 − 1 on the β \beta β -mixing process { v t } t ∈ Z \{v_t\}_{t\in\mathbb{Z}} { v t } t ∈ Z ; (e) sup θ ∈ Θ ∥ ψ ^ t + 1 ∣ t ( θ ) − ψ ˉ t + 1 ∣ t ( θ ) ∥ = O P ( N − 1 / 2 + T − 1 / 2 ) + O a . s . ( X k ⌊ t / k ⌋ + ϱ k ⌊ t / k ⌋ ) \sup_{\theta\in\Theta}\left\|\widehat{\psi}_{t+1|t}(\theta)-\bar{\psi}_{t+1|t}(\theta)\right\|=O_P(N^{-1/2}+T^{-1/2})+O_{a.s.}(\mathcal{X}_k^{\lfloor t/k\rfloor}+\varrho_k^{\lfloor t/k\rfloor}) sup θ ∈ Θ ψ t + 1∣ t ( θ ) − ψ ˉ t + 1∣ t ( θ ) = O P ( N − 1/2 + T − 1/2 ) + O a . s . ( X k ⌊ t / k ⌋ + ϱ k ⌊ t / k ⌋ ) .
In the proof, the first two terms are controlled by combining the factor convergence rates in Proposition 1 with the contraction and moment conditions in Assumptions 6 and 7. The last term is controlled through the invertibility of the filter by an application of Theorem 3.1 of Bougerol [12 ] .
Remark 5. An important implication of Proposition 2 is that the feasible score-driven filter inherits the full sample dependence generated by the first-step PC estimation. Indeed, although its dependence on T T T is suppressed in the notation, each estimated factor F ^ t \widehat{F}_t F t depends on Λ ^ \widehat{\Lambda} Λ , which is computed using the entire sample { x t } t = 1 T \{x_t\}_{t=1}^{T} { x t } t = 1 T . Consequently, each element of { ψ ^ t ∣ t − 1 ( θ ) } t = 1 T \{\widehat{\psi}_{t|t-1}(\theta)\}_{t=1}^{T} { ψ t ∣ t − 1 ( θ ) } t = 1 T depends, through the recursive filter and the PC plug-in estimator, on the whole sample. The feasible filter must therefore be treated as a triangular array rather than as an ordinary stationary time series process. The rates in Proposition 2 reflect this structure. The terms depending on N N N and T T T account for the estimation of the factors and of their transformation, whereas the geometrically decreasing terms account for the different filter initializations. Filter contraction controls the propagation of these errors through the recursion, but it does not remove the common full sample dependence induced by the PC estimator.
Set-consistency of estimated score-driven parameters
The main difference between classical score-driven models treated in the literature and our model is that, in the former, the process entering the score recursion is observed without error, whereas in the latter the factor process { F t } t ∈ Z \{F_t\}_{t\in\mathbb{Z}} { F t } t ∈ Z is latent. Consequently, while in classical score-driven models the parameter vector θ ∈ Θ \theta\in\Theta θ ∈ Θ governing the score update can be point identified, in the present setting, since the latent factor process is replaced by the estimated factors { F ^ t } t = 1 T \{\widehat{F}_t\}_{t=1}^{T} { F t } t = 1 T , the factors are identified only up to an r × r r\times r r × r invertible linear transformation, therefore, the population criterion therefore identifies an orbit rather than a single parameter vector. This means that the parameter vector is set identified, with the required separation outside the identified set imposed below through the regularity of the relevant level sets.
The relevant consistency concept is therefore set-consistency. Formally, consider the general linear group of degree r r r , G L ( r ) = { H ∈ R r × r : ∣ H ∣ ≠ 0 } GL(r) = \{H \in\mathbb{R}^{r \times r}: |H| \ne0\} G L ( r ) = { H ∈ R r × r : ∣ H ∣ = 0 } , and, for every admissible H ∈ G L ( r ) H \in GL(r) H ∈ G L ( r ) , let r H r_H r H denote the invertible reparameterization such that, whenever θ \boldsymbol{\theta} θ governs the dynamics of F t \boldsymbol{F}_t F t , r H ( θ ) r_H(\boldsymbol{\theta}) r H ( θ ) governs the dynamics of H F t H\boldsymbol{F}_t H F t . We refer to H H H as the alignment matrix and to r H r_H r H as the alignment mapping. For every fixed H H H , the mapping and its inverse are linear in θ \boldsymbol{\theta} θ , although generally nonlinear in H H H (see Appendix C for their explicit expressions in the time-varying location and variance specifications). We say that an estimator θ ^ \hat{\boldsymbol{\theta}} θ ^ is set-consistent for θ 0 \boldsymbol{\theta}_0 θ 0 if inf ϑ ∈ Θ H ∥ θ ^ − ϑ ∥ = o P ( 1 ) \inf_{\boldsymbol{\vartheta}\in\Theta_H}\lVert\hat{\boldsymbol{\theta}}-\boldsymbol{\vartheta}\rVert=o_P(1) inf ϑ ∈ Θ H ∥ θ ^ − ϑ ∥ = o P ( 1 ) , where Θ H = { r H ( θ 0 ) ∈ Θ : H ∈ G L ( r ) } \Theta_H=\{r_H(\boldsymbol{\theta}_0)\in\Theta:H\in GL(r)\} Θ H = { r H ( θ 0 ) ∈ Θ : H ∈ G L ( r )} .
To establish consistency, we require standard smoothness and domination conditions on the infeasible conditional quasi log-likelihood ℓ F ( F t ; θ ) \ell_F(\boldsymbol{F}_t;\boldsymbol{\theta}) ℓ F ( F t ; θ ) and its derivatives.
Assumption 8. (i) The process { ℓ F ( F t ; θ ) } t ∈ N \{\ell_F(\boldsymbol{F}_t;\boldsymbol{\theta})\}_{t\in\mathbb{N}} { ℓ F ( F t ; θ ) } t ∈ N is almost surely Lipschitz-L p L_p L p on Θ \Theta Θ . (ii) For ℓ F ( F t ; θ ) \ell_F(\boldsymbol{F}_t;\boldsymbol{\theta}) ℓ F ( F t ; θ ) it holds that sup θ ∈ Θ ∣ ℓ F ( F t ; θ ) ∣ ≤ D 1 , t \sup_{\boldsymbol{\theta}\in\Theta}|\ell_F(\boldsymbol{F}_t;\boldsymbol{\theta})|\le D_{1,t} sup θ ∈ Θ ∣ ℓ F ( F t ; θ ) ∣ ≤ D 1 , t where sup t ≥ 1 E [ D 1 , t p ] < ∞ \sup_{t\ge1}\mathbb{E}[D_{1,t}^p]<\infty sup t ≥ 1 E [ D 1 , t p ] < ∞ for p > 2 p>2 p > 2 .
Assumption 9. (i) For any t ≥ 1 t\ge1 t ≥ 1 , ℓ F ( F t ; θ ) \ell_F(\boldsymbol{F}_t;\boldsymbol{\theta}) ℓ F ( F t ; θ ) is continuously differentiable with respect to ψ t ∣ t − 1 ∈ P \boldsymbol{\psi}_{t|t-1}\in\mathcal{P} ψ t ∣ t − 1 ∈ P and F t ∈ R r \boldsymbol{F}_t\in\mathbb{R}^r F t ∈ R r . (ii) Let ∇ ψ ℓ F ( F t ; θ ) = ∂ ℓ F ( F t ; θ ) ∂ ψ t ∣ t − 1 ∣ ψ t ∣ t − 1 = ψ t ∣ t − 1 ( θ ) \nabla^\psi\ell_F(\boldsymbol{F}_t;\boldsymbol{\theta})=\left.\frac{\partial\ell_F(\boldsymbol{F}_t;\boldsymbol{\theta})}{\partial\boldsymbol{\psi}_{t|t-1}}\right|_{\boldsymbol{\psi}_{t|t-1}=\boldsymbol{\psi}_{t|t-1}(\boldsymbol{\theta})} ∇ ψ ℓ F ( F t ; θ ) = ∂ ψ t ∣ t − 1 ∂ ℓ F ( F t ; θ ) ψ t ∣ t − 1 = ψ t ∣ t − 1 ( θ ) and ∇ F ℓ F ( F t ; θ ) = ∂ ℓ F ( F t ; θ ) ∂ F t ∣ ψ t ∣ t − 1 = ψ t ∣ t − 1 ( θ ) \nabla^F\ell_F(\boldsymbol{F}_t;\boldsymbol{\theta})=\left.\frac{\partial\ell_F(\boldsymbol{F}_t;\boldsymbol{\theta})}{\partial\boldsymbol{F}_t}\right|_{\boldsymbol{\psi}_{t|t-1}=\boldsymbol{\psi}_{t|t-1}(\boldsymbol{\theta})} ∇ F ℓ F ( F t ; θ ) = ∂ F t ∂ ℓ F ( F t ; θ ) ψ t ∣ t − 1 = ψ t ∣ t − 1 ( θ ) , then, sup θ ∈ Θ ∥ ∇ ψ ℓ F ( F t ; θ ) ∥ ≤ D 2 , ψ t ∣ t − 1 \sup_{\boldsymbol{\theta}\in\Theta}\lVert\nabla^\psi\ell_F(\boldsymbol{F}_t;\boldsymbol{\theta})\rVert\le D_{2,\psi_{t|t-1}} sup θ ∈ Θ ∥ ∇ ψ ℓ F ( F t ; θ )∥ ≤ D 2 , ψ t ∣ t − 1 and sup θ ∈ Θ ∥ ∇ F ℓ F ( F t ; θ ) ∥ ≤ D 2 , F t \sup_{\boldsymbol{\theta}\in\Theta}\lVert\nabla^F\ell_F(\boldsymbol{F}_t;\boldsymbol{\theta})\rVert\le D_{2,F_t} sup θ ∈ Θ ∥ ∇ F ℓ F ( F t ; θ )∥ ≤ D 2 , F t where sup t ≥ 1 E [ D 2 , ψ t ∣ t − 1 q ] < ∞ \sup_{t\ge1}\mathbb{E}[D_{2,\psi_{t|t-1}}^q]<\infty sup t ≥ 1 E [ D 2 , ψ t ∣ t − 1 q ] < ∞ and sup t ≥ 1 E [ D 2 , F t q ] < ∞ \sup_{t\ge1}\mathbb{E}[D_{2,F_t}^q]<\infty sup t ≥ 1 E [ D 2 , F t q ] < ∞ for q = 2 p / ( p − 2 ) q=2p/(p-2) q = 2 p / ( p − 2 ) . (iii) The processes { ∇ ψ ℓ F ( F t ; θ ) } t ∈ N \{\nabla^\psi\ell_F(\boldsymbol{F}_t;\boldsymbol{\theta})\}_{t\in\mathbb{N}} { ∇ ψ ℓ F ( F t ; θ ) } t ∈ N and { ∇ F ℓ F ( F t ; θ ) } t ∈ N \{\nabla^F\ell_F(\boldsymbol{F}_t;\boldsymbol{\theta})\}_{t\in\mathbb{N}} { ∇ F ℓ F ( F t ; θ ) } t ∈ N are almost surely locally Lipschitz with respect to ψ t ∣ t − 1 ∈ P \boldsymbol{\psi}_{t|t-1}\in\mathcal{P} ψ t ∣ t − 1 ∈ P and F t ∈ R r \boldsymbol{F}_t\in\mathbb{R}^r F t ∈ R r , uniformly over θ ∈ Θ \boldsymbol{\theta}\in\Theta θ ∈ Θ , and their Lipschitz coefficients have uniformly bounded moments of order q = 2 p / ( p − 2 ) q=2p/(p-2) q = 2 p / ( p − 2 ) .
Hereafter, we refer to T − 1 ∑ t = 1 T ℓ ^ F ( F ^ t ; r H ^ ( θ ) ) + log ∣ H ^ ∣ T^{-1}\sum_{t=1}^{T}\hat{\ell}_F(\hat{\boldsymbol{F}}_t;r_{\hat{H}}(\boldsymbol{\theta}))+\log|\hat{H}| T − 1 ∑ t = 1 T ℓ ^ F ( F ^ t ; r H ^ ( θ )) + log ∣ H ^ ∣ as the aligned feasible criterion. Assumptions 8 and 9 allow us to compare, uniformly over Θ \Theta Θ , this criterion with the stationary infeasible criterion T − 1 ∑ t = 1 T ℓ F ( F ˉ t ; θ ) T^{-1}\sum_{t=1}^{T}\ell_F(\bar{\boldsymbol{F}}_t;\boldsymbol{\theta}) T − 1 ∑ t = 1 T ℓ F ( F ˉ t ; θ ) and its population counterpart. These assumptions rely on the almost sure Lipschitz continuity and domination concepts of Gallant and White [30 ] .
The triangular array structure discussed in Remark 5 implies that the feasible quasi log-likelihood cannot be treated directly as the sample average of a stationary time-series process. In particular, the pointwise filter rates in Proposition 2 are not, by themselves, sufficient to obtain the rate of convergence of the time averaged criterion. The dependence generated by the PC plug-in estimator must instead be retained throughout the likelihood expansion.
Accordingly, we decompose the difference between the aligned feasible criterion and its population counterpart by separating the effect of factor estimation and its propagation through the score-driven filter from the effects of filter initialization, convergence to stationarity, and sampling variation. The factor estimation component is studied directly at the level of the time averaged likelihood, thereby preserving the averaging structure of the PC estimation error. The remaining components are controlled using filter contraction and the near-epoch dependence of the stationary factor and filter processes (see also Remark 7). This delivers the uniform convergence rate stated in the following proposition.
Proposition 3. Under Assumptions 1–9, with p > 2 p>2 p > 2 , as N , T → ∞ N,T\to\infty N , T → ∞ ,
sup θ ∈ Θ ∣ 1 T ∑ t = 1 T ℓ ^ F ( F ^ t ; r H ^ ( θ ) ) + log ∣ H ^ ∣ − E [ ℓ F ( F ‾ t ; θ ) ] ∣ = O P ( 1 N + 1 T ) , \sup_{\theta\in\Theta}\left\lvert\frac{1}{T}\sum_{t=1}^{T}\widehat{\ell}_{F}\left(\widehat{\mathbf{F}}_{t};\mathbf{r}_{\widehat{H}}(\theta)\right)+\log\left\lvert\widehat{H}\right\rvert-\mathbb{E}\left[\ell_{F}\left(\overline{\mathbf{F}}_{t};\theta\right)\right]\right\rvert=O_{\mathbb{P}}\left(\frac{1}{N}+\frac{1}{\sqrt{T}}\right), θ ∈ Θ sup T 1 t = 1 ∑ T ℓ F ( F t ; r H ( θ ) ) + log H − E [ ℓ F ( F t ; θ ) ] = O P ( N 1 + T 1 ) , where F ‾ \overline{\mathbf{F}} F is defined in Assumption 2(i).
Remark 6. The term log ∣ H ^ ∣ \log\left\lvert\widehat{H}\right\rvert log H that appears in the statement of Proposition 3 is independent of θ \theta θ and therefore has no effect on the maximizer of the feasible criterion. Its inclusion serves only to account for the change of variables induced by the mapping r H ^ ( θ ) \mathbf{r}_{\widehat{H}}(\theta) r H ( θ ) .
Remark 7. Although the pointwise PC estimation error is of order O P ( N − 1 / 2 + T − 1 ) O_{\mathbb{P}}(N^{-1/2}+T^{-1}) O P ( N − 1/2 + T − 1 ) , the feasible criterion in Proposition 3 does not inherit the pointwise cross-sectional rate N − 1 / 2 N^{-1/2} N − 1/2 . As discussed in Remark 5, the filter { ψ ^ t ∣ t − 1 ( θ ) } t = 1 T \{\widehat{\psi}_{t|t-1}(\theta)\}_{t=1}^{T} { ψ t ∣ t − 1 ( θ ) } t = 1 T has a triangular array structure because each estimated factor depends on the entire sample through the PC estimator. The proof of Proposition 3 therefore studies the factor estimation error directly within the time averaged likelihood, without bounding each likelihood contribution separately. This retains the cancellations in the weighted time average of the PC estimation errors and gives O P ( ( N T ) − 1 / 2 ) O_{\mathbb{P}}((NT)^{-1/2}) O P (( N T ) − 1/2 ) for the first order component. The remaining factor estimation error enters the likelihood quadratically and is therefore of order O P ( N − 1 + T − 1 ) O_{\mathbb{P}}(N^{-1}+T^{-1}) O P ( N − 1 + T − 1 ) . Combining these terms with the T − 1 / 2 T^{-1/2} T − 1/2 sampling variation of the stationary criterion gives O P ( T − 1 / 2 + ( N T ) − 1 / 2 + N − 1 + T − 1 ) = O P ( N − 1 + T − 1 / 2 ) O_{\mathbb{P}}(T^{-1/2}+(NT)^{-1/2}+N^{-1}+T^{-1})=O_{\mathbb{P}}(N^{-1}+T^{-1/2}) O P ( T − 1/2 + ( N T ) − 1/2 + N − 1 + T − 1 ) = O P ( N − 1 + T − 1/2 ) . Hence, the term N − 1 N^{-1} N − 1 in Proposition 3 arises from the time averaging of the first order factor estimation error and from the quadratic order of the corresponding remainder. It does not represent a faster pointwise convergence rate for either the estimated factors or the feasible filter.
We then assume a regularity condition of the level-sets (Pötscher and Prucha, [47 ] , Definition 4.1).
Assumption 10. (i) The compact parameter space Θ ⊂ R d \Theta\subset\mathbb{R}^{d} Θ ⊂ R d satisfies ∣ K ∣ ≠ 0 \lvert K\rvert\ne0 ∣ K ∣ = 0 . (ii) Let K T = sup θ ∈ Θ E [ ℓ F ( F ‾ t ; θ ) ] K_{T}=\sup_{\theta\in\Theta}\mathbb{E}\left[\ell_{F}\left(\overline{\mathbf{F}}_{t};\theta\right)\right] K T = sup θ ∈ Θ E [ ℓ F ( F t ; θ ) ] , then K T < ∞ K_{T}<\infty K T < ∞ for every T ∈ N T\in\mathbb{N} T ∈ N . (iii) let ϑ ∈ Θ H \vartheta\in\Theta_{H} ϑ ∈ Θ H so that, by definition, there exists a matrix H ∈ G L ( r ) H\in GL(r) H ∈ G L ( r ) such that ϑ = r H ( θ 0 ) \vartheta=\mathbf{r}_{H}(\theta_{0}) ϑ = r H ( θ 0 ) , then, E [ ℓ F ( F ‾ t ; ϑ ) ] ≥ K T \mathbb{E}\left[\ell_{F}\left(\overline{\mathbf{F}}_{t};\vartheta\right)\right]\ge K_{T} E [ ℓ F ( F t ; ϑ ) ] ≥ K T ; (iv) The sequence
of level sets is regular, that is, for every ε > 0 \varepsilon> 0 ε > 0 ,
lim inf T → ∞ ( inf θ ∈ Θ H E [ ℓ F ( F t ; θ ) ] − sup θ ∈ Θ : inf ϑ ∈ Θ H ∥ θ − ϑ ∥ ≥ ε E [ ℓ F ( F t ; θ ) ] ) > 0. \liminf_{T\to\infty}\left(\inf_{\theta\in\Theta_H}\mathbb{E}\left[\ell_F(F_t;\theta)\right]-\sup_{\theta\in\Theta:\inf_{\vartheta\in\Theta_H}\lVert\theta-\vartheta\rVert\ge\varepsilon}\mathbb{E}\left[\ell_F(F_t;\theta)\right]\right)>0. T → ∞ lim inf ( θ ∈ Θ H inf E [ ℓ F ( F t ; θ ) ] − θ ∈ Θ : i n f ϑ ∈ Θ H ∥ θ − ϑ ∥ ≥ ε sup E [ ℓ F ( F t ; θ ) ] ) > 0.
Given Proposition 3 and the separation condition in Assumption 10, we can establish the set consistency of the QMLE θ ^ \widehat{\theta} θ .
Theorem 1. Under Assumptions 1–10, with p > 2 p>2 p > 2 , for every fixed initialization ψ 1 ∣ 0 ∈ P \psi_{1\mid0}\in\mathcal{P} ψ 1 ∣ 0 ∈ P , as N , T → ∞ N,T\to\infty N , T → ∞ , inf ϑ ∈ Θ H ∥ θ ^ − ϑ ∥ = o P ( 1 ) \inf_{\vartheta\in\Theta_H}\lVert\widehat{\theta}-\vartheta\rVert=o_{\mathbb{P}}(1) inf ϑ ∈ Θ H ∥ θ − ϑ ∥ = o P ( 1 ) . Moreover, for H = H 0 \mathbf{H}=\mathbf{H}_0 H = H 0 , with H 0 \mathbf{H}_0 H 0 defined in Proposition 1, as N , T → ∞ N,T\to\infty N , T → ∞ , ∥ θ ^ − r H 0 ( θ 0 ) ∥ = o P ( 1 ) \lVert\widehat{\theta}-r_{\mathbf{H}_0}(\theta_0)\rVert=o_{\mathbb{P}}(1) ∥ θ − r H 0 ( θ 0 )∥ = o P ( 1 ) , and ∥ r H ^ − 1 ( θ ^ ) − θ 0 ∥ = o P ( 1 ) \lVert r_{\widehat{\mathbf{H}}^{-1}}(\widehat{\theta})-\theta_0\rVert=o_{\mathbb{P}}(1) ∥ r H − 1 ( θ ) − θ 0 ∥ = o P ( 1 ) .
Theorem 1 establishes both the set-consistency of θ ^ \widehat{\theta} θ with respect to the orbit Θ H \Theta_H Θ H and, after fixing the limiting factor orientation H 0 \mathbf{H}_0 H 0 , its point consistency for the corresponding representative r H 0 ( θ 0 ) r_{\mathbf{H}_0}(\theta_0) r H 0 ( θ 0 ) . The last statement, which holds since H ^ → P H 0 \widehat{\mathbf{H}}\xrightarrow{\mathbb{P}}\mathbf{H}_0 H P H 0 as N , T → ∞ N,T\to\infty N , T → ∞ , is useful for proving asymptotic normality.
Remark 8. The assumptions of Theorem 1 are sufficient for consistency, but not for deriving an exact convergence rate, which requires the local score and Hessian conditions introduced below. Nevertheless, Proposition 3 suggests the conjectured rate ∥ θ ^ − r H ^ ( θ 0 ) ∥ = O P ( N − 1 + T − 1 / 2 ) \lVert\widehat{\theta}-r_{\widehat{\mathbf{H}}}(\theta_0)\rVert=O_{\mathbb{P}}(N^{-1}+T^{-1/2}) ∥ θ − r H ( θ 0 )∥ = O P ( N − 1 + T − 1/2 ) . Theorem 2 confirms this rate when the required additional assumptions are introduced.
Asymptotic normality of score-driven parameters
Because ∥ H ^ − H 0 ∥ = O ( N − 1 / 2 + T − 1 / 2 ) \lVert\widehat{\mathbf{H}}-\mathbf{H}_0\rVert=O(N^{-1/2}+T^{-1/2}) ∥ H − H 0 ∥ = O ( N − 1/2 + T − 1/2 ) , we cannot use the first statement of Theorem 1 to prove asymptotic normality. To this end, it is more useful the last statement of Theorem 1 which provides point-consistency of the aligned estimator which depends explicitly on H ^ \widehat{\mathbf{H}} H . In particular, for obtaining asymptotic normality of r H ^ − 1 ( θ ^ ) r_{\widehat{\mathbf{H}}^{-1}}(\widehat{\theta}) r H − 1 ( θ ) , both the score vector ∇ θ ℓ F ( F t ; θ ) \nabla^\theta\ell_F(F_t;\theta) ∇ θ ℓ F ( F t ; θ ) and the Hessian matrix ∇ θ θ ℓ F ( F t ; θ ) \nabla^{\theta\theta}\ell_F(F_t;\theta) ∇ θ θ ℓ F ( F t ; θ ) need to satisfy appropriate smoothness and domination conditions. These conditions are required to establish the uniform convergence of the score and Hessian processes to their stationary and ergodic counterparts over Θ \Theta Θ (see Lemmas 7 and 8 for their explicit expressions). They rely on the same concepts employed in Assumptions 8 and 9. The required conditions are stated below ([47 Lemma 8.1] ).
Assumption 11. (i) For any t ≥ 1 t\ge1 t ≥ 1 , ℓ F ( F t ; θ ) \ell_F(F_t;\theta) ℓ F ( F t ; θ ) is twice continuously differentiable with respect to θ \theta θ , and the processes { ∇ θ ℓ F ( F t ; θ ) } t ∈ N \{\nabla^\theta\ell_F(F_t;\theta)\}_{t\in\mathbb{N}} { ∇ θ ℓ F ( F t ; θ ) } t ∈ N and { ∇ θ θ ℓ F ( F t ; θ ) } t ∈ N \{\nabla^{\theta\theta}\ell_F(F_t;\theta)\}_{t\in\mathbb{N}} { ∇ θ θ ℓ F ( F t ; θ ) } t ∈ N are almost surely Lipschitz-L p L_p L p on Θ \Theta Θ , for p > 2 p>2 p > 2 . (ii) For ∇ θ ℓ F ( F t ; θ ) \nabla^\theta\ell_F(F_t;\theta) ∇ θ ℓ F ( F t ; θ ) and ∇ θ θ ℓ F ( F t ; θ ) \nabla^{\theta\theta}\ell_F(F_t;\theta) ∇ θ θ ℓ F ( F t ; θ ) it holds that sup θ ∈ Θ ∥ ∇ θ ℓ F ( F t ; θ ) ∥ ≤ D 2 , θ , t \sup_{\theta\in\Theta}\lVert\nabla^\theta\ell_F(F_t;\theta)\rVert\le D_{2,\theta,t} sup θ ∈ Θ ∥ ∇ θ ℓ F ( F t ; θ )∥ ≤ D 2 , θ , t and
sup θ ∈ Θ ∥ ∇ θ θ ℓ F ( F t ; θ ) ∥ ≤ D 3 , t \sup_{\theta\in\Theta}\left\|\nabla^{\theta\theta}\ell_F(F_t;\theta)\right\| \le D_{3,t} sup θ ∈ Θ ∇ θ θ ℓ F ( F t ; θ ) ≤ D 3 , t where sup t ≥ 1 E [ D 2 , θ , t p ] < ∞ \sup_{t\ge1}\mathbb{E}\left[D_{2,\theta,t}^{p}\right] < \infty sup t ≥ 1 E [ D 2 , θ , t p ] < ∞ and sup t ≥ 1 E [ D 3 , t q ] < ∞ \sup_{t\ge1}\mathbb{E}\left[D_{3,t}^{q}\right] < \infty sup t ≥ 1 E [ D 3 , t q ] < ∞ for q = 2 p / ( p − 2 ) q = 2p/(p - 2) q = 2 p / ( p − 2 ) . (iii) The processes { ∇ θ ℓ F ( F t ; θ ) } t ∈ N \{\nabla^\theta\ell_F(F_t;\theta)\}_{t\in\mathbb{N}} { ∇ θ ℓ F ( F t ; θ ) } t ∈ N and { ∇ θ θ ℓ F ( F t ; θ ) } t ∈ N \{\nabla^{\theta\theta}\ell_F(F_t;\theta)\}_{t\in\mathbb{N}} { ∇ θ θ ℓ F ( F t ; θ ) } t ∈ N are almost surely locally Lipschitz with respect to ψ t ∣ t − 1 ∈ P \psi_{t|t-1} \in\mathcal{P} ψ t ∣ t − 1 ∈ P and F t ∈ R r F_t \in\mathbb{R}^r F t ∈ R r , uniformly over θ ∈ Θ \theta\in\Theta θ ∈ Θ , and their Lipschitz coefficients have uniformly bounded moments of order q = 2 p / ( p − 2 ) q = 2p/(p - 2) q = 2 p / ( p − 2 ) . (iv) For any p > 2 p > 2 p > 2 , ∥ sup θ ∈ Θ sup ψ t ∣ t − 1 ∈ P ∥ ∂ vec X t ( θ ) ∂ ψ t ∣ t − 1 ∥ ∥ p ≤ K X ′ \left\|\sup_{\theta\in\Theta}\sup_{\psi_{t|t-1}\in\mathcal{P}}\left\|\frac{\partial\operatorname{vec}X_t(\theta)}{\partial\psi_{t|t-1}}\right\|\right\|_p \le K'_X sup θ ∈ Θ sup ψ t ∣ t − 1 ∈ P ∂ ψ t ∣ t − 1 ∂ vec X t ( θ ) p ≤ K X ′ , and ∥ sup θ ∈ Θ ∥ ∂ vec X t ( θ ) ∂ F t ∥ ∥ p ≤ K X ′ ′ \left\|\sup_{\theta\in\Theta}\left\|\frac{\partial\operatorname{vec}X_t(\theta)}{\partial F_t}\right\|\right\|_p \le K''_X sup θ ∈ Θ ∂ F t ∂ vec X t ( θ ) p ≤ K X ′′ , for some finite positive reals K X ′ K'_X K X ′ and K X ′ ′ K''_X K X ′′ independent of t t t , with X t ( θ ) X_t(\theta) X t ( θ ) defined in Assumption 6. (v) Let D t ( θ ) = ∂ ψ t + 1 ∣ t ( θ ) ∂ θ ⊤ D_t(\theta) = \frac{\partial\psi_{t+1|t}(\theta)}{\partial\theta^\top} D t ( θ ) = ∂ θ ⊤ ∂ ψ t + 1∣ t ( θ ) , then, for any p > 2 p > 2 p > 2 , ∥ sup θ ∈ Θ ∥ D t ( θ ) ∥ ∥ p ≤ K D ′ \left\|\sup_{\theta\in\Theta}\|D_t(\theta)\|\right\|_p \le K'_D ∥ sup θ ∈ Θ ∥ D t ( θ ) ∥ ∥ p ≤ K D ′ , ∥ sup θ ∈ Θ sup ψ t ∣ t − 1 ∈ P ∥ ∂ vec D t ( θ ) ∂ ψ t ∣ t − 1 ∥ ∥ p ≤ K D ′ ′ \left\|\sup_{\theta\in\Theta}\sup_{\psi_{t|t-1}\in\mathcal{P}}\left\|\frac{\partial\operatorname{vec}D_t(\theta)}{\partial\psi_{t|t-1}}\right\|\right\|_p \le K''_D sup θ ∈ Θ sup ψ t ∣ t − 1 ∈ P ∂ ψ t ∣ t − 1 ∂ vec D t ( θ ) p ≤ K D ′′ , and ∥ sup θ ∈ Θ ∥ ∂ vec D t ( θ ) ∂ F t ∥ ∥ p ≤ K D ′ ′ ′ \left\|\sup_{\theta\in\Theta}\left\|\frac{\partial\operatorname{vec}D_t(\theta)}{\partial F_t}\right\|\right\|_p \le K'''_D sup θ ∈ Θ ∂ F t ∂ vec D t ( θ ) p ≤ K D ′′′ , for some finite positive reals K D ′ K'_D K D ′ , K D ′ ′ K''_D K D ′′ , and K D ′ ′ ′ K'''_D K D ′′′ independent of t t t . (vi) For any p > 2 p > 2 p > 2 , ∥ sup θ ∈ Θ ∥ ∂ vec ∂ ψ ⊤ ( ∂ vec X t ( θ ) ∂ θ ⊤ ) ∥ ∥ p ≤ C X ′ \left\|\sup_{\theta\in\Theta}\left\|\frac{\partial\operatorname{vec}}{\partial\psi^\top}\left(\frac{\partial\operatorname{vec}X_t(\theta)}{\partial\theta^\top}\right)\right\|\right\|_p \le C'_X sup θ ∈ Θ ∂ ψ ⊤ ∂ vec ( ∂ θ ⊤ ∂ vec X t ( θ ) ) p ≤ C X ′ and ∥ sup θ ∈ Θ ∥ ∂ vec ∂ F ⊤ ( ∂ vec X t ( θ ) ∂ θ ⊤ ) ∥ ∥ p ≤ C X ′ ′ \left\|\sup_{\theta\in\Theta}\left\|\frac{\partial\operatorname{vec}}{\partial F^\top}\left(\frac{\partial\operatorname{vec}X_t(\theta)}{\partial\theta^\top}\right)\right\|\right\|_p \le C''_X sup θ ∈ Θ ∂ F ⊤ ∂ vec ( ∂ θ ⊤ ∂ vec X t ( θ ) ) p ≤ C X ′′ , for some finite positive reals C X ′ C'_X C X ′ and C X ′ ′ C''_X C X ′′ independent of t t t . (vii) For any p > 2 p > 2 p > 2 , ∥ sup θ ∈ Θ ∥ ∂ vec D t ( θ ) ∂ θ ⊤ ∥ ∥ p ≤ C D ′ \left\|\sup_{\theta\in\Theta}\left\|\frac{\partial\operatorname{vec}D_t(\theta)}{\partial\theta^\top}\right\|\right\|_p \le C'_D sup θ ∈ Θ ∂ θ ⊤ ∂ vec D t ( θ ) p ≤ C D ′ , ∥ sup θ ∈ Θ ∥ ∂ vec ∂ ψ ⊤ ( ∂ vec D t ( θ ) ∂ θ ⊤ ) ∥ ∥ p ≤ C D ′ ′ \left\|\sup_{\theta\in\Theta}\left\|\frac{\partial\operatorname{vec}}{\partial\psi^\top}\left(\frac{\partial\operatorname{vec}D_t(\theta)}{\partial\theta^\top}\right)\right\|\right\|_p \le C''_D sup θ ∈ Θ ∂ ψ ⊤ ∂ vec ( ∂ θ ⊤ ∂ vec D t ( θ ) ) p ≤ C D ′′ , and ∥ sup θ ∈ Θ ∥ ∂ vec ∂ F ⊤ ( ∂ vec D t ( θ ) ∂ θ ⊤ ) ∥ ∥ p ≤ C D ′ ′ ′ \left\|\sup_{\theta\in\Theta}\left\|\frac{\partial\operatorname{vec}}{\partial F^\top}\left(\frac{\partial\operatorname{vec}D_t(\theta)}{\partial\theta^\top}\right)\right\|\right\|_p \le C'''_D sup θ ∈ Θ ∂ F ⊤ ∂ vec ( ∂ θ ⊤ ∂ vec D t ( θ ) ) p ≤ C D ′′′ , for some finite positive reals C D ′ C'_D C D ′ , C D ′ ′ C''_D C D ′′ , and C D ′ ′ ′ C'''_D C D ′′′ independent of t t t .
The next assumptions are standard in the QML estimation literature.
Assumption 12. The d × d d \times d d × d matrix I ( θ 0 ) = − E [ ∇ θ θ ℓ F ( F ˉ t ; θ 0 ) ] \mathcal{I}(\theta_0) = -\mathbb{E}\left[\nabla^{\theta\theta}\ell_F(\bar{F}_t;\theta_0)\right] I ( θ 0 ) = − E [ ∇ θ θ ℓ F ( F ˉ t ; θ 0 ) ] is positive definite.
Assumption 13. θ 0 \theta_0 θ 0 lies in the interior of the (compact) parameter space Θ \Theta Θ .
Asymptotic normality then holds but the QMLE is not centered around the true value but around a nonlinear transformation thereof, which depends on H ^ \widehat{H} H accounting for the fact that the estimated factors do not identify the true ones. We therefore first derive the asymptotic distribution of the aligned estimator r H ^ − 1 ( θ ^ ) r_{\widehat{H}}^{-1}(\widehat{\theta}) r H − 1 ( θ ) and subsequently map the result back to θ ^ \widehat{\theta} θ .
Theorem 2. Under Assumptions 1–13, with p > 2 p > 2 p > 2 , for every fixed initial condition ψ 1 ∣ 0 ∈ P \psi_{1|0} \in\mathcal{P} ψ 1∣0 ∈ P , and if T / N → 0 \sqrt{T}/N \to0 T / N → 0 , as N , T → ∞ N,T \to\infty N , T → ∞ , T ( θ ^ − r H ^ ( θ 0 ) ) ⇒ N ( 0 d , I H 0 − 1 ( θ 0 ) V H 0 ( θ 0 ) I H 0 − 1 ( θ 0 ) ) \sqrt{T}\left(\widehat{\theta} - r_{\widehat{H}}(\theta_0)\right) \Rightarrow\mathcal{N}\left(0_d,\mathcal{I}_{H_0}^{-1}(\theta_0)V_{H_0}(\theta_0)\mathcal{I}_{H_0}^{-1}(\theta_0)\right) T ( θ − r H ( θ 0 ) ) ⇒ N ( 0 d , I H 0 − 1 ( θ 0 ) V H 0 ( θ 0 ) I H 0 − 1 ( θ 0 ) ) , with I H 0 ( θ 0 ) = R H 0 ⊤ I ( θ 0 ) R H 0 \mathcal{I}_{H_0}(\theta_0) = R_{H_0}^{\top}\mathcal{I}(\theta_0)R_{H_0} I H 0 ( θ 0 ) = R H 0 ⊤ I ( θ 0 ) R H 0 , V H 0 ( θ 0 ) = R H 0 ⊤ V ( θ 0 ) R H 0 V_{H_0}(\theta_0) = R_{H_0}^{\top}V(\theta_0)R_{H_0} V H 0 ( θ 0 ) = R H 0 ⊤ V ( θ 0 ) R H 0 , and R H 0 = ∂ r H ( θ ) ∂ θ ⊤ ∣ θ = θ 0 R_{H_0} = \left.\frac{\partial r_H(\theta)}{\partial\theta^\top}\right|_{\theta=\theta_0} R H 0 = ∂ θ ⊤ ∂ r H ( θ ) θ = θ 0 , and where I ( θ 0 ) = lim T → ∞ − T − 1 E [ ∇ θ θ ℓ F ( F ˉ ; θ 0 ) ] \mathcal{I}(\theta_0) = \lim_{T\to\infty}-T^{-1}\mathbb{E}\left[\nabla^{\theta\theta}\ell_F(\bar{F};\theta_0)\right] I ( θ 0 ) = lim T → ∞ − T − 1 E [ ∇ θ θ ℓ F ( F ˉ ; θ 0 ) ] , V ( θ 0 ) = lim T → ∞ T − 1 E [ ∇ θ ℓ F ( F ˉ ; θ 0 ) ∇ θ ℓ F ( F ˉ ; θ 0 ) ⊤ ] V(\theta_0) = \lim_{T\to\infty}T^{-1}\mathbb{E}\left[\nabla^\theta\ell_F(\bar{F};\theta_0)\nabla^\theta\ell_F(\bar{F};\theta_0)^\top\right] V ( θ 0 ) = lim T → ∞ T − 1 E [ ∇ θ ℓ F ( F ˉ ; θ 0 ) ∇ θ ℓ F ( F ˉ ; θ 0 ) ⊤ ] .
This result implies the consistency rate O P ( max ( N − 1 , T − 1 / 2 ) ) O_{\mathbb{P}}(\max(N^{-1},T^{-1/2})) O P ( max ( N − 1 , T − 1/2 )) for the QMLE suggested by Proposition 3. The constraint T / N → 0 \sqrt{T}/N \to0 T / N → 0 is standard in factor models literature [2 ] and it allows use to neglect the error made when estimating the factors, thus treating them as observed, hence, the classical T \sqrt{T} T consistency rate for θ ^ \widehat{\theta} θ . Nevertheless, the above result still depend on H ^ \widehat{H} H , hence, in general, it cannot be directly used for making inference.
Asymptotic prediction intervals
We derive the approximation error between the feasible filtered predictor of the conditional location C ^ i , t ∣ t − 1 ( θ ^ ) = λ ^ i T μ ^ t ∣ t − 1 ( θ ^ ) \widehat{C}_{i,t|t-1}(\widehat{\theta}) = \widehat{\lambda}_{i}^{\mathsf{T}}\widehat{\boldsymbol{\mu}}_{t|t-1}(\widehat{\theta}) C i , t ∣ t − 1 ( θ ) = λ i T μ t ∣ t − 1 ( θ ) and its unobserved infeasible counterpart C i , t ∣ t − 1 ( θ 0 ) = λ 0 i T μ t ∣ t − 1 ( θ 0 ) C_{i,t|t-1}(\theta_{0}) = \lambda_{0i}^{\mathsf{T}}\boldsymbol{\mu}_{t|t-1}(\theta_{0}) C i , t ∣ t − 1 ( θ 0 ) = λ 0 i T μ t ∣ t − 1 ( θ 0 ) . Similarly, we derive the approximation error between the feasible filtered predictor of the conditional variance P ^ i , t ∣ t − 1 ( θ ^ ) = λ ^ i T Σ ^ t ∣ t − 1 ( θ ^ ) λ ^ i \widehat{P}_{i,t|t-1}(\widehat{\theta}) = \widehat{\lambda}_{i}^{\mathsf{T}}\widehat{\boldsymbol{\Sigma}}_{t|t-1}(\widehat{\theta})\widehat{\lambda}_{i} P i , t ∣ t − 1 ( θ ) = λ i T Σ t ∣ t − 1 ( θ ) λ i and its unobserved infeasible counterpart P i , t ∣ t − 1 ( θ 0 ) = λ 0 i T Σ t ∣ t − 1 ( θ 0 ) λ 0 i P_{i,t|t-1}(\theta_{0}) = \lambda_{0i}^{\mathsf{T}}\boldsymbol{\Sigma}_{t|t-1}(\theta_{0})\lambda_{0i} P i , t ∣ t − 1 ( θ 0 ) = λ 0 i T Σ t ∣ t − 1 ( θ 0 ) λ 0 i . We start with a consistency result.
Theorem 3. Under Assumptions 1–13, with p > 2 p > 2 p > 2 , for every fixed initial condition ψ 1 ∣ 0 ∈ P \psi_{1|0} \in\mathcal{P} ψ 1∣0 ∈ P , for any i = 1 , … , N i = 1,\ldots,N i = 1 , … , N and t = 1 , … , T t = 1,\ldots,T t = 1 , … , T , as N , T → ∞ N,T \to\infty N , T → ∞ : (a) ∣ C ^ i , t ∣ t − 1 ( θ ^ ) − C i , t ∣ t − 1 ( θ 0 ) ∣ = O P ( N − 1 / 2 + T − 1 / 2 ) + O a . s . ( X k ⌊ t / k ⌋ ) \left|\widehat{C}_{i,t|t-1}(\widehat{\theta}) - C_{i,t|t-1}(\theta_{0})\right| = O_{\mathbb{P}}\left(N^{-1/2} + T^{-1/2}\right) + O_{\mathrm{a.s.}}\left(\mathcal{X}_{k}^{\lfloor t/k\rfloor}\right) C i , t ∣ t − 1 ( θ ) − C i , t ∣ t − 1 ( θ 0 ) = O P ( N − 1/2 + T − 1/2 ) + O a.s. ( X k ⌊ t / k ⌋ ) ; (b) ∣ P ^ i , t ∣ t − 1 ( θ ^ ) − P i , t ∣ t − 1 ( θ 0 ) ∣ = O P ( N − 1 / 2 + T − 1 / 2 ) + O a . s . ( X k ⌊ t / k ⌋ ) \left|\widehat{P}_{i,t|t-1}(\widehat{\theta}) - P_{i,t|t-1}(\theta_{0})\right| = O_{\mathbb{P}}\left(N^{-1/2} + T^{-1/2}\right) + O_{\mathrm{a.s.}}\left(\mathcal{X}_{k}^{\lfloor t/k\rfloor}\right) P i , t ∣ t − 1 ( θ ) − P i , t ∣ t − 1 ( θ 0 ) = O P ( N − 1/2 + T − 1/2 ) + O a.s. ( X k ⌊ t / k ⌋ ) .
The rate coincides with the PCA rate of the estimated common component λ ^ i T F ^ t \widehat{\lambda}_{i}^{\mathsf{T}}\widehat{F}_{t} λ i T F t which follows directly from Proposition 1 (Bai, [2 ] , Theorem 3), but for an additional term which is due to initial conditions. To control for this term we introduce the following assumption.
Assumption 14. The initial condition μ 1 ∣ 0 \boldsymbol{\mu}_{1|0} μ 1∣0 and F 1 \boldsymbol{F}_{1} F 1 are fixed and known, with μ 1 ∣ 0 = F 1 = 0 r \boldsymbol{\mu}_{1|0} = \boldsymbol{F}_{1} = \boldsymbol{0}_{r} μ 1∣0 = F 1 = 0 r . The initial condition σ 1 ∣ 0 2 \boldsymbol{\sigma}^{2}_{1|0} σ 1∣0 2 and F 1 \boldsymbol{F}_{1} F 1 are fixed and known, with σ 1 ∣ 0 2 = W r T ( F 1 ⊗ F 1 ) = ( I r − Φ 0 ) − 1 δ 0 \boldsymbol{\sigma}^{2}_{1|0} = \boldsymbol{W}_{r}^{\mathsf{T}}(\boldsymbol{F}_{1} \otimes\boldsymbol{F}_{1}) = (\boldsymbol{I}_{r} - \boldsymbol{\Phi}_{0})^{-1}\boldsymbol{\delta}_{0} σ 1∣0 2 = W r T ( F 1 ⊗ F 1 ) = ( I r − Φ 0 ) − 1 δ 0 .
Clearly, Assumption 14 is stronger than the asymptotic negligibility of initial conditions established in Proposition 2. Similar assumptions have been considered in the early univariate score-driven literature, such as Harvey [34 ] and Harvey and Luati [35 ] , as well as by Patton et al. [46 ] and Catania and Luati [15 ] for their non-Lipschitz score-driven filters. More recently, Hafner and Wang [33 ] and D’Innocenzo et al. [21 ] have also adopted similar assumptions for their multivariate score-driven specifications.
In this context, Proposition 2 can be further refined. Specifically, the estimation error of the factors determines the rate of convergence for the time-varying mean and variance.
Proposition 4. Under Assumptions 1–14, for any t = 1 , … , T t = 1,\ldots,T t = 1 , … , T , as N , T → ∞ N,T \to\infty N , T → ∞ : (a) sup θ ∈ Θ ∥ H ^ − 1 μ ^ t ∣ t − 1 ( r H ^ ( θ ) ) − μ t ∣ t − 1 ( θ ) ∥ = O P ( N − 1 / 2 + T − 1 ) \sup_{\theta\in\Theta}\left\|\widehat{\boldsymbol{H}}^{-1}\widehat{\boldsymbol{\mu}}_{t|t-1}\left(r_{\widehat{H}}(\theta)\right) - \boldsymbol{\mu}_{t|t-1}(\theta)\right\| = O_{\mathbb{P}}\left(N^{-1/2} + T^{-1}\right) sup θ ∈ Θ H − 1 μ t ∣ t − 1 ( r H ( θ ) ) − μ t ∣ t − 1 ( θ ) = O P ( N − 1/2 + T − 1 ) ; (b) sup θ ∈ Θ ∥ ( H ^ − 1 ⊗ H ^ − 1 ) W r σ ^ t ∣ t − 1 2 ( r H ^ ( θ ) ) − W r σ t ∣ t − 1 2 ( θ ) ∥ = O P ( N − 1 / 2 + T − 1 ) \sup_{\theta\in\Theta}\left\|\left(\widehat{\boldsymbol{H}}^{-1} \otimes\widehat{\boldsymbol{H}}^{-1}\right)\boldsymbol{W}_{r}\widehat{\boldsymbol{\sigma}}^{2}_{t|t-1}\left(r_{\widehat{H}}(\theta)\right) - \boldsymbol{W}_{r}\boldsymbol{\sigma}^{2}_{t|t-1}(\theta)\right\| = O_{\mathbb{P}}\left(N^{-1/2} + T^{-1}\right) sup θ ∈ Θ ( H − 1 ⊗ H − 1 ) W r σ t ∣ t − 1 2 ( r H ( θ ) ) − W r σ t ∣ t − 1 2 ( θ ) = O P ( N − 1/2 + T − 1 ) .
Remark 9. The different rates in Propositions 2 and 4 are due to their different comparison targets. Proposition 2 compares the feasible filter with its stationary counterpart and therefore includes the T − 1 / 2 T^{-1/2} T − 1/2 error arising from H ^ − H 0 \widehat{\boldsymbol{H}} - \boldsymbol{H}_{0} H − H 0 , as well as the initialization effect. Proposition 4, instead, compares filters expressed under the same estimated transformation H ^ \widehat{\boldsymbol{H}} H , while Assumption 14 removes the initialization effect. The remaining perturbation is therefore generated only by F ^ t − H ^ F t \widehat{F}_t-\widehat{H}F_t F t − H F t , whose rate is O P ( N − 1 / 2 + T − 1 ) O_{\mathrm{P}}(N^{-1/2}+T^{-1}) O P ( N − 1/2 + T − 1 ) . Filter contraction preserves this rate, explaining the absence of a T − 1 / 2 T^{-1/2} T − 1/2 term in Proposition 4.
We then prove asymptotic normality of the common-component predictors.
Theorem 4. Under Assumptions 1–14, with p > 2 p>2 p > 2 , for any i = 1 , … , N i=1,\ldots,N i = 1 , … , N and t = 1 , … , T t=1,\ldots,T t = 1 , … , T , and if T / N → 0 T/N\to0 T / N → 0 , as N , T → ∞ N,T\to\infty N , T → ∞ ,
T C ^ i , t ∣ t − 1 ( θ ^ ) − C i , t ∣ t − 1 ( θ 0 ) a i , t ∣ t − 1 T ( θ 0 ) Ω ( θ 0 ) a i , t ∣ t − 1 ( θ 0 ) ∣ F t − 1 ⇒ N ( 0 , 1 ) , T P ^ i , t ∣ t − 1 ( θ ^ ) − P i , t ∣ t − 1 ( θ 0 ) b i , t ∣ t − 1 T ( θ 0 ) Ω ( θ 0 ) b i , t ∣ t − 1 ( θ 0 ) ∣ F t − 1 ⇒ N ( 0 , 1 ) , \sqrt{T}\frac{\widehat{C}_{i,t|t-1}(\widehat{\theta})-C_{i,t|t-1}(\theta_0)}{\sqrt{a_{i,t|t-1}^{\mathsf{T}}(\theta_0)\Omega(\theta_0)a_{i,t|t-1}(\theta_0)}}\mathrel{\big|}\mathcal{F}_{t-1}\Rightarrow\mathcal{N}(0,1),\qquad
\sqrt{T}\frac{\widehat{P}_{i,t|t-1}(\widehat{\theta})-P_{i,t|t-1}(\theta_0)}{\sqrt{b_{i,t|t-1}^{\mathsf{T}}(\theta_0)\Omega(\theta_0)b_{i,t|t-1}(\theta_0)}}\mathrel{\big|}\mathcal{F}_{t-1}\Rightarrow\mathcal{N}(0,1), T a i , t ∣ t − 1 T ( θ 0 ) Ω ( θ 0 ) a i , t ∣ t − 1 ( θ 0 ) C i , t ∣ t − 1 ( θ ) − C i , t ∣ t − 1 ( θ 0 ) F t − 1 ⇒ N ( 0 , 1 ) , T b i , t ∣ t − 1 T ( θ 0 ) Ω ( θ 0 ) b i , t ∣ t − 1 ( θ 0 ) P i , t ∣ t − 1 ( θ ) − P i , t ∣ t − 1 ( θ 0 ) F t − 1 ⇒ N ( 0 , 1 ) , where
a i , t ∣ t − 1 T ( θ 0 ) = [ μ t ∣ t − 1 ( θ 0 ) T λ 0 i T ∇ θ μ t ∣ t − 1 ( θ 0 ) ] a_{i,t|t-1}^{\mathsf{T}}(\theta_0)=\left[\mu_{t|t-1}(\theta_0)^{\mathsf{T}}\ \ \lambda_{0i}^{\mathsf{T}}\nabla^\theta\mu_{t|t-1}(\theta_0)\right] a i , t ∣ t − 1 T ( θ 0 ) = [ μ t ∣ t − 1 ( θ 0 ) T λ 0 i T ∇ θ μ t ∣ t − 1 ( θ 0 ) ] and
b i , t ∣ t − 1 T ( θ 0 ) = [ σ t ∣ t − 1 2 ( θ 0 ) W r T ( I r ⊗ λ 0 i + λ 0 i ⊗ I r ) | ( λ 0 i ⊗ λ 0 i ) T W r ∇ θ σ t ∣ t − 1 2 ( θ 0 ) ] b_{i,t|t-1}^{\mathsf{T}}(\theta_0)=\left[\sigma_{t|t-1}^{2}(\theta_0)\mathbf{W}_{r}^{\mathsf{T}}\left(I_r\otimes\lambda_{0i}+\lambda_{0i}\otimes I_r\right)\ \middle|\ (\lambda_{0i}\otimes\lambda_{0i})^{\mathsf{T}}\mathbf{W}_{r}\nabla^\theta\sigma_{t|t-1}^{2}(\theta_0)\right] b i , t ∣ t − 1 T ( θ 0 ) = [ σ t ∣ t − 1 2 ( θ 0 ) W r T ( I r ⊗ λ 0 i + λ 0 i ⊗ I r ) ( λ 0 i ⊗ λ 0 i ) T W r ∇ θ σ t ∣ t − 1 2 ( θ 0 ) ] are ( r + d ) (r+d) ( r + d ) -dimensional row vectors and
Ω ( θ 0 ) = [ Γ F − 1 Ξ i Γ F − 1 0 r × d 0 d × r I − 1 ( θ 0 ) V ( θ 0 ) I − 1 ( θ 0 ) ] , \Omega(\theta_0)=
\begin{bmatrix}
\Gamma_F^{-1}\Xi_i\Gamma_F^{-1} & \mathbf{0}_{r\times d}\\
\mathbf{0}_{d\times r} & \mathcal{I}^{-1}(\theta_0)V(\theta_0)\mathcal{I}^{-1}(\theta_0)
\end{bmatrix}, Ω ( θ 0 ) = [ Γ F − 1 Ξ i Γ F − 1 0 d × r 0 r × d I − 1 ( θ 0 ) V ( θ 0 ) I − 1 ( θ 0 ) ] , is an ( r + d ) × ( r + d ) (r+d)\times(r+d) ( r + d ) × ( r + d ) matrix with Γ F \Gamma_F Γ F defined in Assumption 2, Ξ i \Xi_i Ξ i defined in Assumption 5, I ( θ 0 ) \mathcal{I}(\theta_0) I ( θ 0 ) and V ( θ 0 ) V(\theta_0) V ( θ 0 ) defined in Theorem 2.
Although the matrix H ^ \widehat{H} H appears in Proposition 4, it does not play any role in the last result. Indeed, the estimated loading vectors transform contravariantly to the filtered factor moments, so that the H ^ \widehat{H} H cancels from the common component predictors. Consequently, the limiting distributions and their feasible estimators can be expressed entirely in terms of common-component quantities which do not depend on H ^ \widehat{H} H .
Remark 10. The condition T / N → 0 T/N\to0 T / N → 0 in Theorem 4 ensures that the first-stage factor-estimation error and its propagation through the filter are negligible under the root-T T T normalization. Indeed, Proposition 4 gives this component the rate O P ( N − 1 / 2 + T − 1 ) O_{\mathrm{P}}(N^{-1/2}+T^{-1}) O P ( N − 1/2 + T − 1 ) . The restriction T / N → 0 T/N\to0 T / N → 0 , therefore, allows the limiting distribution to be determined only by the loading estimation and QMLE score components. For the Gaussian specifications considered below, the affine structure of the score updates makes it possible to retain the first-stage factor estimation component explicitly. For the Student’s t t t specifications, instead, the score updates contain nonlinear random weights depending jointly on the estimated factors and filtered parameters. The corresponding first-stage component does not admit the same explicit linear representation.
For the Gaussian score updates, we can improve on Theorem 4 and remove the restriction on the relative rates of N N N and T T T . Indeed, the recursively propagated effect of the score updates can be written as a linear function of the PC estimation errors, so that its cross-sectional Gaussian limit can be combined with the time series limits of the loading estimator and the QMLE, yielding a variance containing both N − 1 N^{-1} N − 1 and T − 1 T^{-1} T − 1 components and resulting in the following Central Limit Theorem.
Theorem 5. For the conditional location and variance models, suppose that the filters are based on the Gaussian quasi log-likelihoods in (6) and (10) , respectively. Then under Assumptions 1–14, with p > 2 p > 2 p > 2 , for any i = 1 , … , N i = 1,\ldots,N i = 1 , … , N and t = 1 , … , T t = 1,\ldots,T t = 1 , … , T , as N , T → ∞ N,T \to\infty N , T → ∞ ,
C ^ i , t ∣ t − 1 ( θ ^ ) − C i , t ∣ t − 1 ( θ 0 ) 1 N λ 0 i ⊤ V t ( θ 0 ) λ 0 i + 1 T a i , t ∣ t − 1 ⊤ ( θ 0 ) Ω ( θ 0 ) a i , t ∣ t − 1 ( θ 0 ) ∣ F t − 1 ⇒ N ( 0 , 1 ) , \left.\frac{\widehat{C}_{i,t|t-1}(\widehat{\theta})-C_{i,t|t-1}(\theta_0)}{\sqrt{\frac{1}{N}\lambda_{0i}^{\top}\mathcal{V}_t(\theta_0)\lambda_{0i}+\frac{1}{T}a_{i,t|t-1}^{\top}(\theta_0)\Omega(\theta_0)a_{i,t|t-1}(\theta_0)}}\right|\mathcal{F}_{t-1}\Rightarrow\mathcal{N}(0,1), N 1 λ 0 i ⊤ V t ( θ 0 ) λ 0 i + T 1 a i , t ∣ t − 1 ⊤ ( θ 0 ) Ω ( θ 0 ) a i , t ∣ t − 1 ( θ 0 ) C i , t ∣ t − 1 ( θ ) − C i , t ∣ t − 1 ( θ 0 ) F t − 1 ⇒ N ( 0 , 1 ) , P ^ i , t ∣ t − 1 ( θ ^ ) − P i , t ∣ t − 1 ( θ 0 ) 1 N ( λ 0 i ⊗ λ 0 i ) ⊤ W t ( θ 0 ) ( λ 0 i ⊗ λ 0 i ) + 1 T b i , t ∣ t − 1 ⊤ ( θ 0 ) Ω ( θ 0 ) b i , t ∣ t − 1 ( θ 0 ) ∣ F t − 1 ⇒ N ( 0 , 1 ) . \left.\frac{\widehat{P}_{i,t|t-1}(\widehat{\theta})-P_{i,t|t-1}(\theta_0)}{\sqrt{\frac{1}{N}(\lambda_{0i}\otimes\lambda_{0i})^{\top}\mathcal{W}_t(\theta_0)(\lambda_{0i}\otimes\lambda_{0i})+\frac{1}{T}b_{i,t|t-1}^{\top}(\theta_0)\Omega(\theta_0)b_{i,t|t-1}(\theta_0)}}\right|\mathcal{F}_{t-1}\Rightarrow\mathcal{N}(0,1). N 1 ( λ 0 i ⊗ λ 0 i ) ⊤ W t ( θ 0 ) ( λ 0 i ⊗ λ 0 i ) + T 1 b i , t ∣ t − 1 ⊤ ( θ 0 ) Ω ( θ 0 ) b i , t ∣ t − 1 ( θ 0 ) P i , t ∣ t − 1 ( θ ) − P i , t ∣ t − 1 ( θ 0 ) F t − 1 ⇒ N ( 0 , 1 ) . where
V t ( θ 0 ) = ∑ k = 0 t − 1 ∑ h = 0 t − 1 ( Φ 0 − K 0 ) k K 0 Σ Λ − 1 Π t − k − 1 , t − h − 1 Σ Λ − 1 K 0 ⊤ ( Φ 0 − K 0 ) h ⊤ . \mathcal{V}_t(\theta_0)=\sum_{k=0}^{t-1}\sum_{h=0}^{t-1}(\Phi_0-K_0)^kK_0\Sigma_{\Lambda}^{-1}\Pi_{t-k-1,t-h-1}\Sigma_{\Lambda}^{-1}K_0^{\top}(\Phi_0-K_0)^{h\top}. V t ( θ 0 ) = k = 0 ∑ t − 1 h = 0 ∑ t − 1 ( Φ 0 − K 0 ) k K 0 Σ Λ − 1 Π t − k − 1 , t − h − 1 Σ Λ − 1 K 0 ⊤ ( Φ 0 − K 0 ) h ⊤ . W t ( θ 0 ) = ∑ k = 0 t − 1 ∑ h = 0 t − 1 ( W r ( Φ 0 − K 0 ) W r ⊤ ) k W r K 0 W r ⊤ × ( I r ⊗ F t − k − 1 + F t − k − 1 ⊗ I r ) Σ Λ − 1 Π t − k − 1 , t − h − 1 Σ Λ − 1 × ( I r ⊗ F t − h − 1 + F t − h − 1 ⊗ I r ) ⊤ W r K 0 ⊤ W r ⊤ ( W r ( Φ 0 − K 0 ) W r ⊤ ) h ⊤ . \begin{aligned}
\mathcal{W}_t(\theta_0)
={}&\sum_{k=0}^{t-1}\sum_{h=0}^{t-1}\left(W_r(\Phi_0-K_0)W_r^{\top}\right)^kW_rK_0W_r^{\top}\\
&\quad\times(I_r\otimes F_{t-k-1}+F_{t-k-1}\otimes I_r)\Sigma_{\Lambda}^{-1}\Pi_{t-k-1,t-h-1}\Sigma_{\Lambda}^{-1}\\
&\quad\times(I_r\otimes F_{t-h-1}+F_{t-h-1}\otimes I_r)^{\top}W_rK_0^{\top}W_r^{\top}\left(W_r(\Phi_0-K_0)W_r^{\top}\right)^{h\top}.
\end{aligned} W t ( θ 0 ) = k = 0 ∑ t − 1 h = 0 ∑ t − 1 ( W r ( Φ 0 − K 0 ) W r ⊤ ) k W r K 0 W r ⊤ × ( I r ⊗ F t − k − 1 + F t − k − 1 ⊗ I r ) Σ Λ − 1 Π t − k − 1 , t − h − 1 Σ Λ − 1 × ( I r ⊗ F t − h − 1 + F t − h − 1 ⊗ I r ) ⊤ W r K 0 ⊤ W r ⊤ ( W r ( Φ 0 − K 0 ) W r ⊤ ) h ⊤ . The matrices V t ( θ 0 ) \mathcal{V}_t(\theta_0) V t ( θ 0 ) and W t ( θ 0 ) \mathcal{W}_t(\theta_0) W t ( θ 0 ) have dimensions r × r r\times r r × r and r 2 × r 2 r^2\times r^2 r 2 × r 2 , respectively. Moreover, Σ Λ \Sigma_{\Lambda} Σ Λ and Π s , u \Pi_{s,u} Π s , u are defined in Assumptions 3 and 4, respectively, and a i , t ∣ t − 1 ( θ 0 ) a_{i,t|t-1}(\theta_0) a i , t ∣ t − 1 ( θ 0 ) , b i , t ∣ t − 1 ( θ 0 ) b_{i,t|t-1}(\theta_0) b i , t ∣ t − 1 ( θ 0 ) , and Ω ( θ 0 ) \Omega(\theta_0) Ω ( θ 0 ) are defined as in Theorem 4.
Feasible covariance estimation. All the asymptotic variances in Theorems 4 and 5 are feasible. Let e ^ i t = x i t − λ ^ i ⊤ F ^ t \widehat{e}_{it}=x_{it}-\widehat{\lambda}_{i}^{\top}\widehat{F}_t e i t = x i t − λ i ⊤ F t and, for a generic zero-mean vector process { g t } t = 1 T \{g_t\}_{t=1}^{T} { g t } t = 1 T , define the Bartlett HAC estimator HAC M T ( { g t } t = 1 T ) = 1 T ∑ t = 1 T g t g t ⊤ + ∑ h = 1 M T ( 1 − h M T + 1 ) 1 T ∑ t = h + 1 T ( g t g t − h ⊤ + g t − h g t ⊤ ) , \operatorname{HAC}_{M_T}\left(\{g_t\}_{t=1}^{T}\right)=\frac{1}{T}\sum_{t=1}^{T}g_tg_t^{\top}+\sum_{h=1}^{M_T}\left(1-\frac{h}{M_T+1}\right)\frac{1}{T}\sum_{t=h+1}^{T}(g_tg_{t-h}^{\top}+g_{t-h}g_t^{\top}), HAC M T ( { g t } t = 1 T ) = T 1 ∑ t = 1 T g t g t ⊤ + ∑ h = 1 M T ( 1 − M T + 1 h ) T 1 ∑ t = h + 1 T ( g t g t − h ⊤ + g t − h g t ⊤ ) , where we set M T = ⌊ T 1 / 4 ⌋ M_T=\lfloor T^{1/4}\rfloor M T = ⌊ T 1/4 ⌋ . Then, let Γ ^ F = T − 1 ∑ t = 1 T F ^ t F ^ t ⊤ \widehat{\Gamma}_F=T^{-1}\sum_{t=1}^{T}\widehat{F}_t\widehat{F}_t^{\top} Γ F = T − 1 ∑ t = 1 T F t F t ⊤ , Ξ ^ i = HAC M T ( { F ^ t e ^ i t } t = 1 T ) \widehat{\Xi}_i=\operatorname{HAC}_{M_T}\left(\{\widehat{F}_t\widehat{e}_{it}\}_{t=1}^{T}\right) Ξ i = HAC M T ( { F t e i t } t = 1 T ) , I ^ = − T − 1 ∇ θ θ ⊤ ℓ ^ F ( F ^ ; θ ^ ) \widehat{I}=-T^{-1}\nabla_{\theta\theta^{\top}}\widehat{\ell}_F\left(\widehat{F};\widehat{\theta}\right) I = − T − 1 ∇ θ θ ⊤ ℓ F ( F ; θ ) , and
V ^ = HAC M T ( { ∇ ⊤ ℓ ^ F ( F ^ t ; θ ^ ) } t = 1 T ) \widehat{V}=\operatorname{HAC}_{M_T}\left(\left\{\nabla^\top\widehat{\ell}_{F}(\widehat{F}_t;\widehat{\theta})\right\}_{t=1}^{T}\right) V = HAC M T ( { ∇ ⊤ ℓ F ( F t ; θ ) } t = 1 T )
[44 ] . These quantities provide the estimator
Ω ^ = [ Γ ^ F − 1 Ξ ^ i Γ ^ F − 1 0 r × d 0 d × r I ^ − 1 V ^ I ^ − 1 ] . \widehat{\Omega}=
\begin{bmatrix}
\widehat{\Gamma}_{F}^{-1}\widehat{\Xi}_{i}\widehat{\Gamma}_{F}^{-1} & \mathbf{0}_{r\times d}\\
\mathbf{0}_{d\times r} & \widehat{\mathcal{I}}^{-1}\widehat{V}\widehat{\mathcal{I}}^{-1}
\end{bmatrix}. Ω = [ Γ F − 1 Ξ i Γ F − 1 0 d × r 0 r × d I − 1 V I − 1 ] .
For Theorem 5, we additionally estimate Σ Λ \Sigma_{\Lambda} Σ Λ by Σ ^ Λ = N − 1 Λ ^ ⊤ Λ ^ \widehat{\Sigma}_{\Lambda}=N^{-1}\widehat{\Lambda}^{\top}\widehat{\Lambda} Σ Λ = N − 1 Λ ⊤ Λ . Then, to accommodate weak cross-sectional dependence, let η ^ i , t = λ ^ i e ^ i t \widehat{\eta}_{i,t}=\widehat{\lambda}_{i}\widehat{e}_{it} η i , t = λ i e i t and define d ^ i j = ∣ 1 / ρ ^ i j ∣ − 1 \widehat{d}_{ij}=\left|1/\widehat{\rho}_{ij}\right|-1 d ij = ∣ 1/ ρ ij ∣ − 1 , with ρ ^ i j \widehat{\rho}_{ij} ρ ij denoting the sample correlation between { e ^ i t } t = 1 T \{\widehat{e}_{it}\}_{t=1}^{T} { e i t } t = 1 T and { e ^ j t } t = 1 T \{\widehat{e}_{jt}\}_{t=1}^{T} { e j t } t = 1 T . Following the AV-SHAC approach of Kim [38 ] , we use
Π ^ s , u = 1 N ∑ i = 1 N ∑ j = 1 N κ ( d ^ i j b N ) η ^ i , s η ^ j , u ⊤ , κ ( x ) = ( 1 − ∣ x ∣ ) 1 { ∣ x ∣ ≤ 1 } , \widehat{\Pi}_{s,u}=\frac{1}{N}\sum_{i=1}^{N}\sum_{j=1}^{N}\kappa\left(\frac{\widehat{d}_{ij}}{b_N}\right)\widehat{\eta}_{i,s}\widehat{\eta}_{j,u}^{\top},\qquad\kappa(x)=(1-|x|)\mathbf{1}_{\{|x|\leq1\}}, Π s , u = N 1 i = 1 ∑ N j = 1 ∑ N κ ( b N d ij ) η i , s η j , u ⊤ , κ ( x ) = ( 1 − ∣ x ∣ ) 1 { ∣ x ∣ ≤ 1 } ,
where the cross-sectional bandwidth b N b_N b N may be selected using the cluster wild-bootstrap procedure proposed by Kim [38 ] . Under covariance stationarity, estimation precision can be improved by averaging Π ^ s , u \widehat{\Pi}_{s,u} Π s , u over all time pairs sharing the same lag s − u s-u s − u . At lag zero, this yields the original contemporaneous AV-SHAC estimator. The matrices V ^ t \widehat{\mathcal{V}}_t V t and W ^ t \widehat{\mathcal{W}}_t W t are obtained by substituting the feasible quantities into their definitions in Theorem 5. Importantly, both summations are retained, so that all matrices Π ^ t − k − 1 , t − h − 1 \widehat{\Pi}_{t-k-1,t-h-1} Π t − k − 1 , t − h − 1 , including those associated with k ≠ h k\ne h k = h , enter the estimated asymptotic variance. The vectors a ^ i , t ∣ t − 1 \widehat{a}_{i,t|t-1} a i , t ∣ t − 1 and b ^ i , t ∣ t − 1 \widehat{b}_{i,t|t-1} b i , t ∣ t − 1 are constructed analogously by replacing the unknown parameters, loadings, filters, and filter derivatives with their feasible estimated counterparts. Although the intermediate matrices are expressed in the factor coordinates selected by the PC estimator, the resulting scalar quadratic forms are invariant to the alignment of factors and loadings. Hence, neither H ^ \widehat{H} H nor H 0 H_0 H 0 plays any role, and all the asymptotic variances are directly computable in empirical applications.
Joint time-varying conditional location-variance models
The time-varying location and variance models defined in § 2 can be combined, so that the full multivariate dynamic model generated by the non-stochastic link function q ( ⋅ , v t ) \mathbf{q}(\cdot,\mathbf{v}_t) q ( ⋅ , v t ) in (2) assumes the form
F t = q ( ψ t ∣ t − 1 , v t ) = μ t ∣ t − 1 + Σ t ∣ t − 1 1 / 2 v t , t ∈ Z , (19) \mathbf{F}_t=\mathbf{q}\left(\boldsymbol{\psi}_{t|t-1},\mathbf{v}_t\right)=\boldsymbol{\mu}_{t|t-1}+\boldsymbol{\Sigma}_{t|t-1}^{1/2}\mathbf{v}_t,\qquad t\in\mathbb{Z}, \tag*{(19)} F t = q ( ψ t ∣ t − 1 , v t ) = μ t ∣ t − 1 + Σ t ∣ t − 1 1/2 v t , t ∈ Z , (19)
where ψ t ∣ t − 1 ≡ ( μ t ∣ t − 1 ⊤ , σ t ∣ t − 1 2 ⊤ ) ⊤ \psi_{t|t-1} \equiv\left(\mu_{t|t-1}^{\top},\sigma_{t|t-1}^{2\top}\right)^{\top} ψ t ∣ t − 1 ≡ ( μ t ∣ t − 1 ⊤ , σ t ∣ t − 1 2⊤ ) ⊤ . The noise process { v t } t ∈ Z \{v_t\}_{t\in\mathbb{Z}} { v t } t ∈ Z satisfies Assumption 1, with E [ v t v t ⊤ ] = I r \mathbb{E}\left[v_t v_t^{\top}\right]=I_r E [ v t v t ⊤ ] = I r . The conditional location and variance processes in (19) are defined by
μ t + 1 ∣ t = Φ μ μ t ∣ t − 1 + K μ ( F t − μ t ∣ t − 1 ) , (20) \mu_{t+1|t}=\Phi^\mu\mu_{t|t-1}+K^\mu\left(F_t-\mu_{t|t-1}\right), \tag*{(20)} μ t + 1∣ t = Φ μ μ t ∣ t − 1 + K μ ( F t − μ t ∣ t − 1 ) , (20)
ϵ t = F t − μ t ∣ t − 1 , (21) \epsilon_t=F_t-\mu_{t|t-1}, \tag*{(21)} ϵ t = F t − μ t ∣ t − 1 , (21)
vec Σ t ∣ t − 1 = W r σ t ∣ t − 1 2 , (22) \operatorname{vec}\Sigma_{t|t-1}=W_r\sigma_{t|t-1}^{2}, \tag*{(22)} vec Σ t ∣ t − 1 = W r σ t ∣ t − 1 2 , (22)
σ t + 1 ∣ t 2 = δ σ + Φ σ σ t ∣ t − 1 2 + ( K σ + Υ σ I ( F t ) ) W r ⊤ ( ϵ t ⊗ ϵ t − vec Σ t ∣ t − 1 ) , (23) \sigma_{t+1|t}^{2}=\delta^\sigma+\Phi^\sigma\sigma_{t|t-1}^{2}+\left(K^\sigma+\Upsilon^\sigma I(F_t)\right)W_r^{\top}\left(\epsilon_t\otimes\epsilon_t-\operatorname{vec}\Sigma_{t|t-1}\right), \tag*{(23)} σ t + 1∣ t 2 = δ σ + Φ σ σ t ∣ t − 1 2 + ( K σ + Υ σ I ( F t ) ) W r ⊤ ( ϵ t ⊗ ϵ t − vec Σ t ∣ t − 1 ) , (23)
where I ( F t ) = diag ( 1 { F 1 t ≤ 0 } , … , 1 { F r t ≤ 0 } ) I(F_t)=\operatorname{diag}\left(\mathbb{1}_{\{F_{1t}\leq0\}},\ldots,\mathbb{1}_{\{F_{rt}\leq0\}}\right) I ( F t ) = diag ( 1 { F 1 t ≤ 0 } , … , 1 { F r t ≤ 0 } ) . The main difference between the joint model in (19) –(23) and the time-varying conditional variance model in (9) is that the conditional variance process { σ t ∣ t − 1 2 } t ∈ Z \{\sigma_{t|t-1}^{2}\}_{t\in\mathbb{Z}} { σ t ∣ t − 1 2 } t ∈ Z now depends on the unobserved conditional location process { μ t ∣ t − 1 } t ∈ Z \{\mu_{t|t-1}\}_{t\in\mathbb{Z}} { μ t ∣ t − 1 } t ∈ Z through the residuals { ϵ t } t ∈ Z \{\epsilon_t\}_{t\in\mathbb{Z}} { ϵ t } t ∈ Z . Consequently, even after replacing the latent factors { F t } t ∈ Z \{F_t\}_{t\in\mathbb{Z}} { F t } t ∈ Z with their first-step estimators { F ^ t } t = 1 T \{\widehat{F}_t\}_{t=1}^{T} { F t } t = 1 T , the variance filter cannot be evaluated without first obtaining a filtered path for the conditional location process. While the first-step PC estimator and its asymptotic properties remain unchanged, the score-driven estimation stage and its associated asymptotic theory must therefore be adapted to account for the additional estimation error generated by the filtered conditional location process.
Remark 11. The leverage indicator in (23) is defined in terms of F t F_t F t , rather than the centered residual ϵ t ( θ μ ) \epsilon_t(\theta^\mu) ϵ t ( θ μ ) , so that, conditional on the factor path, it depends neither on θ μ \theta^\mu θ μ and θ σ \theta^\sigma θ σ nor on the filtered location process μ t ∣ t − 1 ( θ μ ) \mu_{t|t-1}(\theta^\mu) μ t ∣ t − 1 ( θ μ ) . It therefore enters the conditional variance recursion as an exogenous measurable input and does not introduce an additional discontinuity in the stochastic recurrence equation. This preserves the differentiability and contraction structure required to apply the contraction and near-epoch-dependence arguments developed in § 4 .
Sequential QML estimation. To overcome the issue discussed above, we propose a sequential QML estimation strategy. After retrieving the factors { F ^ t } t = 1 T \{\widehat{F}_t\}_{t=1}^{T} { F t } t = 1 T from the first-step, we first estimate the parameters governing the conditional location dynamics in (20) by means of a quasi-likelihood (QL) criterion constructed under a working homoskedastic assumption. More precisely, we use the classical quasi-Gaussian likelihood for the factors given in (6) , but we treat the conditional covariance matrix as fixed and equal to the identity matrix, such that Σ \Sigma Σ is replaced by I r I_r I r .
Hence, at the first-stage, for a generic parameter vector θ μ = ( vec Φ μ ⊤ , vec K μ ⊤ ) ⊤ \theta^\mu=\left(\operatorname{vec}\Phi^{\mu\top},\operatorname{vec}K^{\mu\top}\right)^\top θ μ = ( vec Φ μ ⊤ , vec K μ ⊤ ) ⊤ such that θ μ ∈ Θ μ ⊂ R 2 r 2 \theta^\mu\in\Theta^\mu\subset\mathbb{R}^{2r^2} θ μ ∈ Θ μ ⊂ R 2 r 2 , we define
θ ^ μ = arg max θ μ ∈ Θ μ ⊂ R 2 r 2 ∑ t = 1 T ℓ ^ μ , F ( F ^ t ; θ μ ) , (24) \widehat{\theta}^\mu= \underset{\theta^\mu\in\Theta^\mu\subset\mathbb{R}^{2r^2}}{\operatorname{arg\,max}} \sum_{t=1}^{T} \widehat{\ell}_{\mu,F}\left(\widehat{F}_t;\theta^\mu\right), \tag*{(24)} θ μ = θ μ ∈ Θ μ ⊂ R 2 r 2 arg max t = 1 ∑ T ℓ μ , F ( F t ; θ μ ) , (24)
ℓ ^ μ , F ( F ^ t ; θ μ ) = − 1 2 ( F ^ t − μ ^ t ∣ t − 1 ( θ μ ) ) T ( F ^ t − μ ^ t ∣ t − 1 ( θ μ ) ) , (25) \widehat{\ell}_{\mu,F}\left(\widehat{F}_t;\theta^\mu\right)=-\frac{1}{2}\left(\widehat{F}_t-\widehat{\mu}_{t|t-1}\left(\theta^\mu\right)\right)^{\mathsf{T}}\left(\widehat{F}_t-\widehat{\mu}_{t|t-1}\left(\theta^\mu\right)\right), \tag*{(25)} ℓ μ , F ( F t ; θ μ ) = − 2 1 ( F t − μ t ∣ t − 1 ( θ μ ) ) T ( F t − μ t ∣ t − 1 ( θ μ ) ) , (25)
up to an irrelevant additive constant, and where { μ ^ t ∣ t − 1 ( θ μ ) } t = 1 T \left\{\widehat{\mu}_{t|t-1}\left(\theta^\mu\right)\right\}_{t=1}^{T} { μ t ∣ t − 1 ( θ μ ) } t = 1 T is generated recursively from the conditional location equation in (20) initialized at some arbitrary μ 1 ∣ 0 \boldsymbol{\mu}_{1|0} μ 1∣0 and substituting the latent factors { F t } t ∈ Z \left\{F_t\right\}_{t\in\mathbb{Z}} { F t } t ∈ Z with their first-step PC estimators { F ^ t } t = 1 T \left\{\widehat{F}_t\right\}_{t=1}^{T} { F t } t = 1 T . This first-stage criterion is not intended to provide a full specification of the joint conditional density of the factors, but rather to deliver a feasible estimator of the conditional location dynamics and, consequently, a filtered path for the unobservable process { μ t ∣ t − 1 } t ∈ Z \left\{\boldsymbol{\mu}_{t|t-1}\right\}_{t\in\mathbb{Z}} { μ t ∣ t − 1 } t ∈ Z .
Given the estimator θ ^ μ \widehat{\theta}^\mu θ μ , we then construct the filtered conditional location path { μ ^ t ∣ t − 1 ( θ ^ μ ) } t = 1 T \left\{\widehat{\boldsymbol{\mu}}_{t|t-1}\left(\widehat{\theta}^\mu\right)\right\}_{t=1}^{T} { μ t ∣ t − 1 ( θ μ ) } t = 1 T and define the corresponding centered residuals
ϵ ^ t ( θ ^ μ ) = F ^ t − μ ^ t ∣ t − 1 ( θ ^ μ ) , t = 1 , … , T , (26) \widehat{\boldsymbol{\epsilon}}_t\left(\widehat{\theta}^\mu\right)=\widehat{F}_t-\widehat{\boldsymbol{\mu}}_{t|t-1}\left(\widehat{\theta}^\mu\right),\qquad t=1,\ldots,T, \tag*{(26)} ϵ t ( θ μ ) = F t − μ t ∣ t − 1 ( θ μ ) , t = 1 , … , T , (26)
which are the approximated version of the unobserved stationary residuals ϵ ‾ t ( θ 0 μ ) = F ‾ t − μ ‾ t ∣ t − 1 ( θ 0 μ ) \overline{\boldsymbol{\epsilon}}_t\left(\theta^\mu_0\right)=\overline{F}_t-\overline{\boldsymbol{\mu}}_{t|t-1}\left(\theta^\mu_0\right) ϵ t ( θ 0 μ ) = F t − μ t ∣ t − 1 ( θ 0 μ ) , with t ∈ Z t\in\mathbb{Z} t ∈ Z . Because the first-stage QLE in (24) and (25) ignores the time-variation in the conditional covariance matrix, the residuals { ϵ ^ t ( θ ^ μ ) } t = 1 T \left\{\widehat{\boldsymbol{\epsilon}}_t\left(\widehat{\theta}^\mu\right)\right\}_{t=1}^{T} { ϵ t ( θ μ ) } t = 1 T still retain conditional heteroskedasticity. This is precisely the feature that makes them suitable for the second-stage estimation of the conditional variance process. Indeed, once the time-varying location component has been filtered out, the residuals can be treated as the relevant inputs for the variance recursion, so that the latter becomes feasible after replacing the unobserved process { μ t ∣ t − 1 } t ∈ Z \left\{\boldsymbol{\mu}_{t|t-1}\right\}_{t\in\mathbb{Z}} { μ t ∣ t − 1 } t ∈ Z with its first-stage filtered counterpart { μ ^ t ∣ t − 1 ( θ ^ μ ) } t = 1 T \left\{\widehat{\boldsymbol{\mu}}_{t|t-1}\left(\widehat{\theta}^\mu\right)\right\}_{t=1}^{T} { μ t ∣ t − 1 ( θ μ ) } t = 1 T .
In the second stage, we therefore estimate the parameters driving the conditional variance dynamics by maximizing the full quasi-Gaussian likelihood, conditional on the first-stage estimate of the location process. Let θ σ = ( δ σ T , vec Φ σ T , vec K σ T , vec Υ σ T ) T ∈ Θ σ ⊂ R r + 3 r 2 \theta^\sigma=\left(\delta^{\sigma\mathsf{T}},\operatorname{vec}\Phi^{\sigma\mathsf{T}},\operatorname{vec}K^{\sigma\mathsf{T}},\operatorname{vec}\Upsilon^{\sigma\mathsf{T}}\right)^{\mathsf{T}}\in\Theta^\sigma\subset\mathbb{R}^{r+3r^2} θ σ = ( δ σ T , vec Φ σ T , vec K σ T , vec Υ σ T ) T ∈ Θ σ ⊂ R r + 3 r 2 denote the vector of parameters governing the conditional variance recursion. Then, we define
θ ^ σ = arg max θ σ ∈ Θ σ ⊂ R r + 3 r 2 ∑ t = 1 T ℓ ^ σ , ϵ ( ϵ ^ t ( θ ^ μ ) ; θ σ ) , (27) \widehat{\theta}^\sigma=\underset{\theta^\sigma\in\Theta^\sigma\subset\mathbb{R}^{r+3r^2}}{\operatorname{arg\,max}}\sum_{t=1}^{T}\widehat{\ell}_{\sigma,\epsilon}\left(\widehat{\boldsymbol{\epsilon}}_t\left(\widehat{\theta}^\mu\right);\theta^\sigma\right), \tag*{(27)} θ σ = θ σ ∈ Θ σ ⊂ R r + 3 r 2 arg max t = 1 ∑ T ℓ σ , ϵ ( ϵ t ( θ μ ) ; θ σ ) , (27)
ℓ ^ σ , ϵ ( ϵ ^ t ( θ ^ μ ) ; θ σ ) = − 1 2 ( log ∣ Σ ^ t ∣ t − 1 ( θ ^ μ , θ σ ) ∣ + ϵ ^ t ( θ ^ μ ) T Σ ^ t ∣ t − 1 ( θ ^ μ , θ σ ) − 1 ϵ ^ t ( θ ^ μ ) ) . (28) \widehat{\ell}_{\sigma,\epsilon}\left(\widehat{\boldsymbol{\epsilon}}_t\left(\widehat{\theta}^\mu\right);\theta^\sigma\right)=-\frac{1}{2}\left(\log\left|\widehat{\Sigma}_{t|t-1}\left(\widehat{\theta}^\mu,\theta^\sigma\right)\right|+\widehat{\boldsymbol{\epsilon}}_t\left(\widehat{\theta}^\mu\right)^{\mathsf{T}}\widehat{\Sigma}_{t|t-1}\left(\widehat{\theta}^\mu,\theta^\sigma\right)^{-1}\widehat{\boldsymbol{\epsilon}}_t\left(\widehat{\theta}^\mu\right)\right). \tag*{(28)} ℓ σ , ϵ ( ϵ t ( θ μ ) ; θ σ ) = − 2 1 ( log Σ t ∣ t − 1 ( θ μ , θ σ ) + ϵ t ( θ μ ) T Σ t ∣ t − 1 ( θ μ , θ σ ) − 1 ϵ t ( θ μ ) ) . (28)
The sequence { Σ ^ t ∣ t − 1 ( θ ^ μ , θ σ ) } t = 1 T \{\widehat{\Sigma}_{t|t-1}(\widehat{\theta}^{\mu},\theta^{\sigma})\}_{t=1}^{T} { Σ t ∣ t − 1 ( θ μ , θ σ ) } t = 1 T is generated recursively from (22) –(23) using the estimated residuals { ϵ ^ t ( θ ^ μ ) } t = 1 T \{\widehat{\epsilon}_{t}(\widehat{\theta}^{\mu})\}_{t=1}^{T} { ϵ t ( θ μ ) } t = 1 T defined in (26) . Equivalently, the second-stage filter is obtained by injecting into the variance recursion both the pre-estimated factors and the filtered conditional location process obtained in the first-stage. In this way, after retrieving the factors { F ^ t } t = 1 T \{\widehat{F}_{t}\}_{t=1}^{T} { F t } t = 1 T from the first-step PC estimator, the original joint estimation problem is decomposed into two additional feasible and interpretable steps. First, filtering and estimating the conditional location dynamics, and second, filtering and estimating the conditional variance dynamics of the resulting mean-corrected factors.
As a result, we obtain the sequential-QMLE: θ ^ J = ( θ ^ μ T , θ ^ σ T ) T \widehat{\theta}^{J}=(\widehat{\theta}^{\mu\mathsf{T}},\widehat{\theta}^{\sigma\mathsf{T}})^{\mathsf{T}} θ J = ( θ μ T , θ σ T ) T , and, accordingly, we denote its true counterpart as θ 0 J = ( θ 0 μ T , θ 0 σ T ) T ∈ Θ μ × Θ σ ⊂ R d \theta_{0}^{J}=(\theta_{0}^{\mu\mathsf{T}},\theta_{0}^{\sigma\mathsf{T}})^{\mathsf{T}}\in\Theta^{\mu}\times\Theta^{\sigma}\subset\mathbb{R}^{d} θ 0 J = ( θ 0 μ T , θ 0 σ T ) T ∈ Θ μ × Θ σ ⊂ R d with d = r + 5 r 2 d=r+5r^{2} d = r + 5 r 2 . This sequential approach is analogous in spirit to a pseudo-likelihood procedure of Ling and McAleer [41 ] for their vector ARMA-GARCH model. The first stage uses a misspecified working likelihood to recover the dynamic conditional mean, while the second stage uses the full quasi-Gaussian likelihood to model the remaining conditional heteroskedasticity. Therefore, although the two components are jointly present in the data-generating mechanism, estimation remains feasible by proceeding sequentially, with the first-stage filtered conditional location process acting as an input into the second-stage conditional variance filter.
The first-stage criterion in (25) is a quasi-likelihood for the conditional location process. We therefore impose Assumptions 8–13 on this criterion, with ℓ F \ell_{F} ℓ F , θ \theta θ , and Θ \Theta Θ replaced by ℓ μ , F \ell_{\mu,F} ℓ μ , F , θ μ \theta^{\mu} θ μ , and Θ μ \Theta^{\mu} Θ μ , respectively. Theorem 1 can therefore be applied to the first-stage estimator θ ^ μ \widehat{\theta}^{\mu} θ μ . Under the additional conditions required in Theorem 2, including T / N → 0 \sqrt{T}/N\to0 T / N → 0 , the corresponding asymptotic normality result also applies.
Relative to the framework developed in § 3 , the sequential procedure introduces an additional source of estimation error, because the second-stage variance filter is driven by the estimated residuals in (26) . The following assumption, which extends Assumptions 6 and 7 to the joint conditional location-variance process, provides sufficient conditions for controlling the propagation of this error through the conditional location and variance recursions.
Assumption 15. (i) The parameter spaces Θ μ \Theta^{\mu} Θ μ and Θ σ \Theta^{\sigma} Θ σ are compact subsets of R 2 r 2 \mathbb{R}^{2r^{2}} R 2 r 2 and R r + 3 r 2 \mathbb{R}^{r+3r^{2}} R r + 3 r 2 , respectively. (ii) For positive reals X 1 μ < 1 X_{1}^{\mu}<1 X 1 μ < 1 and Y 1 μ < ∞ Y_{1}^{\mu}<\infty Y 1 μ < ∞ , sup θ μ ∈ Θ μ ∥ Φ μ − K μ ∥ ≤ X 1 μ \sup_{\theta^{\mu}\in\Theta^{\mu}}\|\Phi^{\mu}-K^{\mu}\|\leq X_{1}^{\mu} sup θ μ ∈ Θ μ ∥ Φ μ − K μ ∥ ≤ X 1 μ and sup θ μ ∈ Θ μ ∥ K μ ∥ ≤ Y 1 μ \sup_{\theta^{\mu}\in\Theta^{\mu}}\|K^{\mu}\|\leq Y_{1}^{\mu} sup θ μ ∈ Θ μ ∥ K μ ∥ ≤ Y 1 μ . (iii) Let G r = { G = diag ( g 1 , … , g r ) : g j ∈ { 0 , 1 } , j = 1 , … , r } \mathcal{G}_{r}=\{G=\operatorname{diag}(g_{1},\ldots,g_{r}):g_{j}\in\{0,1\},\ j=1,\ldots,r\} G r = { G = diag ( g 1 , … , g r ) : g j ∈ { 0 , 1 } , j = 1 , … , r } . Then, for every θ σ ∈ Θ σ \theta^{\sigma}\in\Theta^{\sigma} θ σ ∈ Θ σ and G ∈ G r G\in\mathcal{G}_{r} G ∈ G r , K σ + Υ σ G ≥ 0 r × r K^{\sigma}+\Upsilon^{\sigma}G\geq0_{r\times r} K σ + Υ σ G ≥ 0 r × r , and Φ σ − K σ − Υ σ G ≥ 0 r × r \Phi^{\sigma}-K^{\sigma}-\Upsilon^{\sigma}G\geq0_{r\times r} Φ σ − K σ − Υ σ G ≥ 0 r × r , where the matrix inequalities are understood
componentwise. (iv) For some k ≥ 1 k \ge1 k ≥ 1 and some finite positive real χ k σ < 1 \chi_k^\sigma< 1 χ k σ < 1 ,
sup θ σ ∈ Θ σ max G 1 , … , G k ∈ G r ∥ ∏ j = 1 k ( Φ σ − K σ − Υ σ G j ) ∥ ≤ χ k σ . \sup_{\theta^\sigma\in\Theta^\sigma}\max_{G_1,\ldots,G_k \in\mathcal{G}_r}\left\|\prod_{j=1}^{k}\left(\Phi^\sigma-K^\sigma-\Upsilon^\sigma G_j\right)\right\| \le\chi_k^\sigma. θ σ ∈ Θ σ sup G 1 , … , G k ∈ G r max j = 1 ∏ k ( Φ σ − K σ − Υ σ G j ) ≤ χ k σ .
(v) Let Z t ( θ μ , θ σ ) = ( K σ + Υ σ I ( F t ) ) W r T ( I r ⊗ ϵ t ( θ μ ) + ϵ t ( θ μ ) ⊗ I r ) Z_t(\theta^\mu,\theta^\sigma)=(K^\sigma+\Upsilon^\sigma I(F_t))W_r^\mathsf{T}\left(I_r\otimes\epsilon_t(\theta^\mu)+\epsilon_t(\theta^\mu)\otimes I_r\right) Z t ( θ μ , θ σ ) = ( K σ + Υ σ I ( F t )) W r T ( I r ⊗ ϵ t ( θ μ ) + ϵ t ( θ μ ) ⊗ I r ) . For the same p > 2 p>2 p > 2 as in Assumption 7, there exists a finite positive real Z k σ Z_k^\sigma Z k σ , independent of t t t , such that
∥ ∑ j = 1 k sup θ μ ∈ Θ μ θ σ ∈ Θ σ ∥ Z t − j + 1 ( θ μ , θ σ ) ∥ ∥ ρ ≤ Z k σ . \left\|\sum_{j=1}^{k}\sup_{\theta^\mu\in\Theta^\mu\atop\theta^\sigma\in\Theta^\sigma}\left\|Z_{t-j+1}(\theta^\mu,\theta^\sigma)\right\|\right\|_\rho\le Z_k^\sigma. j = 1 ∑ k θ σ ∈ Θ σ θ μ ∈ Θ μ sup ∥ Z t − j + 1 ( θ μ , θ σ ) ∥ ρ ≤ Z k σ .
(vi) There exist finite positive reals C I C_I C I and c H c_H c H such that, for every sufficiently small η > 0 \eta>0 η > 0 ,
sup H ∈ G L ( r ) : ∥ H − H 0 ∥ ≤ c H η sup t ∈ N max 1 ≤ i ≤ r P ( ∣ e i T H F ‾ t ∣ ≤ η ) ≤ C I η . \sup_{\substack{H\in GL(r):\\ \|H-H_0\|\le c_H\eta}}\sup_{t\in\mathbb{N}}\max_{1\le i\le r}\mathbb{P}\left(\left|e_i^\mathsf{T}H\overline{F}_t\right|\le\eta\right)\le C_I\eta. H ∈ G L ( r ) : ∥ H − H 0 ∥ ≤ c H η sup t ∈ N sup 1 ≤ i ≤ r max P ( e i T H F t ≤ η ) ≤ C I η .
Part (i) imposes compactness of the parameter spaces, while part (ii) ensures contraction of the conditional location filter. Parts (iii) and (iv) guarantee positivity and contraction of the conditional variance filter for every possible sequence of indicator realizations, and part (v) controls the propagation of the estimated-location error through the quadratic residuals. Part (vi) is a uniform 1-continuity condition for the leverage indicator; see Borovkova et al. [11 ] , Definition 2.10, Example 2.1(ii), and Proposition 2.11. It controls the indicator under factor approximation and preserves geometrically decreasing approximation coefficients under truncation, as required for part (d) of Proposition 2.
With the next proposition, we show that the filter approximation results established in Proposition 2 extend to the joint conditional location-variance model after accounting for the additional perturbation generated by the estimated conditional location process.
Proposition 5. Under Assumptions 1–7 and 15 , with p > 2 p>2 p > 2 , for any fixed initial conditions of the conditional location and variance filters, as N , T → ∞ N,T\to\infty N , T → ∞ , statements (a)–(e) of Proposition 2 continue to hold for the joint process { ψ t ∣ t − 1 } t ∈ Z ≡ { ( μ t ∣ t − 1 T , σ t ∣ t − 1 2 T ) T } t ∈ Z \{\psi_{t\mid t-1}\}_{t\in\mathbb{Z}}\equiv\{(\mu_{t\mid t-1}^{\mathsf{T}},\sigma_{t\mid t-1}^{2\mathsf{T}})^{\mathsf{T}}\}_{t\in\mathbb{Z}} { ψ t ∣ t − 1 } t ∈ Z ≡ {( μ t ∣ t − 1 T , σ t ∣ t − 1 2 T ) T } t ∈ Z , uniformly over ( θ μ , θ σ ) ∈ Θ μ × Θ σ (\theta^\mu,\theta^\sigma)\in\Theta^\mu\times\Theta^\sigma ( θ μ , θ σ ) ∈ Θ μ × Θ σ .
Proposition 5 extends the filter approximation rates and the near-epoch-dependence result of Proposition 2 to the joint conditional location-variance process. Proposition 3 and Theorem 1 can therefore be applied sequentially to (25) and (28) , yielding the set-consistency of the sequential-QMLE. More formally, as N , T → ∞ N,T\to\infty N , T → ∞ , it holds that ∥ r H ^ − 1 ( θ ^ J ) − θ 0 J ∥ = o P ( 1 ) \left\|\mathfrak{r}_{\widehat{H}}^{-1}(\widehat{\theta}^{J})-\theta_0^{J}\right\|=o_P(1) r H − 1 ( θ J ) − θ 0 J = o P ( 1 ) , where the joint realignment mapping is defined blockwise as r H J ( θ J ) = ( r H μ ( θ μ ) T , r H σ ( θ σ ) T ) T r_H^J(\theta^J) = \left(r_H^\mu(\theta^\mu)^\mathsf{T}, r_H^\sigma(\theta^\sigma)^\mathsf{T}\right)^\mathsf{T} r H J ( θ J ) = ( r H μ ( θ μ ) T , r H σ ( θ σ ) T ) T , with inverse defined analogously, see Appendix C for further details.
We therefore focus directly on asymptotic normality. At the root-T T T scale, the first-stage estimation error enters the second-stage score through the estimated residuals and the variance filter. Since the second-stage criterion is quasi-Gaussian and the leverage indicator is independent of θ μ \theta^\mu θ μ and θ σ \theta^\sigma θ σ , the required mixed-score and Hessian expansions can be established directly. The following proposition provides these additional results.
Proposition 6. Under Assumptions 1–15, with p > 2 p > 2 p > 2 , for every fixed initialization of the conditional location and variance filters, as N , T → ∞ N,T \to\infty N , T → ∞ , if T / N → 0 \sqrt{T}/N \to0 T / N → 0 , then
T ( θ ^ J − r H J ( θ 0 J ) ) ⇒ N ( 0 , I J , H 0 − 1 ( θ 0 J ) V J , H 0 ( θ 0 J ) I J , H 0 − 1 ( θ 0 J ) T ) . \sqrt{T}\left(\widehat{\theta}^J-r_H^J(\theta_0^J)\right) \Rightarrow\mathcal{N}\left(0,\mathcal{I}_{J,H_0}^{-1}(\theta_0^J)\mathcal{V}_{J,H_0}(\theta_0^J)\mathcal{I}_{J,H_0}^{-1}(\theta_0^J)^\mathsf{T}\right). T ( θ J − r H J ( θ 0 J ) ) ⇒ N ( 0 , I J , H 0 − 1 ( θ 0 J ) V J , H 0 ( θ 0 J ) I J , H 0 − 1 ( θ 0 J ) T ) . with I J , H 0 ( θ 0 J ) = R H 0 J T I J ( θ 0 J ) R H 0 J \mathcal{I}_{J,H_0}(\theta_0^J)=R_{H_0}^{J\mathsf{T}}\mathcal{I}_J(\theta_0^J)R_{H_0}^J I J , H 0 ( θ 0 J ) = R H 0 J T I J ( θ 0 J ) R H 0 J , V J , H 0 ( θ 0 J ) = R H 0 J T V J ( θ 0 J ) R H 0 J \mathcal{V}_{J,H_0}(\theta_0^J)=R_{H_0}^{J\mathsf{T}}\mathcal{V}_J(\theta_0^J)R_{H_0}^J V J , H 0 ( θ 0 J ) = R H 0 J T V J ( θ 0 J ) R H 0 J ,
R H J = ( R H μ 0 0 R H σ ) , R H μ = ∂ ∂ θ μ T r H μ , − 1 ( θ 0 μ ) , R H σ = ∂ ∂ θ σ T r H σ , − 1 ( θ 0 σ ) . R_H^J=\begin{pmatrix} R_H^\mu& 0 \\ 0 & R_H^\sigma\end{pmatrix}, \qquad R_H^\mu=\frac{\partial}{\partial\theta^{\mu\mathsf{T}}}r_H^{\mu,-1}(\theta_0^\mu), \qquad R_H^\sigma=\frac{\partial}{\partial\theta^{\sigma\mathsf{T}}}r_H^{\sigma,-1}(\theta_0^\sigma). R H J = ( R H μ 0 0 R H σ ) , R H μ = ∂ θ μ T ∂ r H μ , − 1 ( θ 0 μ ) , R H σ = ∂ θ σ T ∂ r H σ , − 1 ( θ 0 σ ) . and where
I J ( θ 0 J ) = ( I μ μ ( θ 0 μ ) 0 I σ μ ( θ 0 J ) I σ σ ( θ 0 σ ) ) , \mathcal{I}_J(\theta_0^J)=\begin{pmatrix} \mathcal{I}_{\mu\mu}(\theta_0^\mu) & 0 \\ \mathcal{I}_{\sigma\mu}(\theta_0^J) & \mathcal{I}_{\sigma\sigma}(\theta_0^\sigma) \end{pmatrix}, I J ( θ 0 J ) = ( I μμ ( θ 0 μ ) I σ μ ( θ 0 J ) 0 I σ σ ( θ 0 σ ) ) , with I μ μ ( θ 0 μ ) = lim T → ∞ − 1 T E [ ∇ θ μ ℓ μ , F ( F ‾ ; θ 0 μ ) ] \mathcal{I}_{\mu\mu}(\theta_0^\mu)=\lim_{T\to\infty}-\frac{1}{T}\mathbb{E}\left[\nabla^{\theta^\mu}\ell_{\mu,F}\left(\overline{F};\theta_0^\mu\right)\right] I μμ ( θ 0 μ ) = lim T → ∞ − T 1 E [ ∇ θ μ ℓ μ , F ( F ; θ 0 μ ) ] , I σ σ ( θ 0 σ ) = lim T → ∞ − 1 T E [ ∇ θ σ ℓ σ , ϵ ( ϵ ‾ ( θ 0 μ ) ; θ 0 σ ) ] \mathcal{I}_{\sigma\sigma}(\theta_0^\sigma)=\lim_{T\to\infty}-\frac{1}{T}\mathbb{E}\left[\nabla^{\theta^\sigma}\ell_{\sigma,\epsilon}\left(\overline{\epsilon}(\theta_0^\mu);\theta_0^\sigma\right)\right] I σ σ ( θ 0 σ ) = lim T → ∞ − T 1 E [ ∇ θ σ ℓ σ , ϵ ( ϵ ( θ 0 μ ) ; θ 0 σ ) ] ,
I σ μ ( θ 0 J ) = lim T → ∞ − 1 T E [ ∇ θ σ ∇ θ μ ℓ σ , ϵ ( ϵ ‾ ( θ 0 μ ) ; θ 0 σ ) ] \mathcal{I}_{\sigma\mu}(\theta_0^J)=\lim_{T\to\infty}-\frac{1}{T}\mathbb{E}\left[\nabla^{\theta^\sigma}\nabla^{\theta^\mu}\ell_{\sigma,\epsilon}\left(\overline{\epsilon}(\theta_0^\mu);\theta_0^\sigma\right)\right] I σ μ ( θ 0 J ) = lim T → ∞ − T 1 E [ ∇ θ σ ∇ θ μ ℓ σ , ϵ ( ϵ ( θ 0 μ ) ; θ 0 σ ) ] ,
V J ( θ 0 J ) = lim T → ∞ 1 T E [ ( ∇ θ μ ℓ μ , F ( F ‾ ; θ 0 μ ) ∇ θ σ ℓ σ , ϵ ( ϵ ‾ ( θ 0 μ ) ; θ 0 σ ) ) ( ∇ θ μ ℓ μ , F ( F ‾ ; θ 0 μ ) ∇ θ σ ℓ σ , ϵ ( ϵ ‾ ( θ 0 μ ) ; θ 0 σ ) ) T ] . \mathcal{V}_J(\theta_0^J)=\lim_{T\to\infty}\frac{1}{T}\mathbb{E}\left[
\begin{pmatrix}
\nabla^{\theta^\mu}\ell_{\mu,F}\left(\overline{F};\theta_0^\mu\right)\\
\nabla^{\theta^\sigma}\ell_{\sigma,\epsilon}\left(\overline{\epsilon}(\theta_0^\mu);\theta_0^\sigma\right)
\end{pmatrix}
\begin{pmatrix}
\nabla^{\theta^\mu}\ell_{\mu,F}\left(\overline{F};\theta_0^\mu\right)\\
\nabla^{\theta^\sigma}\ell_{\sigma,\epsilon}\left(\overline{\epsilon}(\theta_0^\mu);\theta_0^\sigma\right)
\end{pmatrix}^{\mathsf{T}}
\right]. V J ( θ 0 J ) = T → ∞ lim T 1 E [ ( ∇ θ μ ℓ μ , F ( F ; θ 0 μ ) ∇ θ σ ℓ σ , ϵ ( ϵ ( θ 0 μ ) ; θ 0 σ ) ) ( ∇ θ μ ℓ μ , F ( F ; θ 0 μ ) ∇ θ σ ℓ σ , ϵ ( ϵ ( θ 0 μ ) ; θ 0 σ ) ) T ] .
This result shows that the implied consistency rate of the joint QMLE is still O P ( max ( N − 1 , T − 1 / 2 ) ) O_{\mathrm{P}}\left(\max(N^{-1},T^{-1/2})\right) O P ( max ( N − 1 , T − 1/2 ) ) . Furthermore, it is clear that Theorems 4 and 5 extend directly to the joint model by replacing the QMLE covariance block in the bottom d × d d\times d d × d diagonal block of Ω ( θ 0 ) \boldsymbol{\Omega}(\theta_0) Ω ( θ 0 ) with the asymptotic covariance I J − 1 ( θ 0 J ) V J ( θ 0 J ) I J − 1 ( θ 0 J ) T \mathcal{I}_J^{-1}(\theta_0^J)\mathcal{V}_J(\theta_0^J)\mathcal{I}_J^{-1}(\theta_0^J)^\mathsf{T} I J − 1 ( θ 0 J ) V J ( θ 0 J ) I J − 1 ( θ 0 J ) T as defined in Proposition 6. For the conditional location, only its upper-left block enters, because μ t ∣ t − 1 \mu_{t|t-1} μ t ∣ t − 1 depends only on θ μ \theta^\mu θ μ . For the conditional variance, the full matrix, including its cross-blocks, is required because σ t ∣ t − 1 2 \sigma_{t|t-1}^2 σ t ∣ t − 1 2 depends on both θ μ \theta^\mu θ μ and θ σ \theta^\sigma θ σ . In Theorem 5, the matrix V t ( θ 0 μ ) \mathcal{V}_t(\theta_0^\mu) V t ( θ 0 μ ) remains unchanged, whereas W t ( θ 0 ) \mathcal{W}_t(\theta_0) W t ( θ 0 ) is replaced by
W t J ( θ 0 J ) = ∑ k = 0 t − 1 ∑ h = 0 t − 1 ( W r ( Φ 0 σ − K 0 σ ) W r ⊤ ) k W r K 0 σ W r ⊤ ( I r ⊗ ϵ t − k − 1 ( θ 0 μ ) + ϵ t − k − 1 ( θ 0 μ ) ⊗ I r ) Π t − k − 1 , t − h − 1 ϵ ( I r ⊗ ϵ t − h − 1 ( θ 0 μ ) + ϵ t − h − 1 ( θ 0 μ ) ⊗ I r ) ⊤ W r K 0 σ ⊤ W r ⊤ ( W r ( Φ 0 σ − K 0 σ ) W r ⊤ ) h ⊤ , \mathcal{W}^{J}_t(\theta^J_0)=\sum_{k=0}^{t-1}\sum_{h=0}^{t-1}\left(\mathbf{W}_r\left(\mathbf{\Phi}^{\sigma}_0-\mathbf{K}^{\sigma}_0\right)\mathbf{W}_r^{\top}\right)^k\mathbf{W}_r\mathbf{K}^{\sigma}_0\mathbf{W}_r^{\top}
\left(\mathbf{I}_r\otimes\boldsymbol{\epsilon}_{t-k-1}(\theta^\mu_0)+\boldsymbol{\epsilon}_{t-k-1}(\theta^\mu_0)\otimes\mathbf{I}_r\right)\mathbf{\Pi}^{\epsilon}_{t-k-1,t-h-1}
\left(\mathbf{I}_r\otimes\boldsymbol{\epsilon}_{t-h-1}(\theta^\mu_0)+\boldsymbol{\epsilon}_{t-h-1}(\theta^\mu_0)\otimes\mathbf{I}_r\right)^{\top}
\mathbf{W}_r\mathbf{K}^{\sigma\top}_0\mathbf{W}_r^{\top}
\left(\mathbf{W}_r\left(\mathbf{\Phi}^{\sigma}_0-\mathbf{K}^{\sigma}_0\right)\mathbf{W}_r^{\top}\right)^{h\top}, W t J ( θ 0 J ) = k = 0 ∑ t − 1 h = 0 ∑ t − 1 ( W r ( Φ 0 σ − K 0 σ ) W r ⊤ ) k W r K 0 σ W r ⊤ ( I r ⊗ ϵ t − k − 1 ( θ 0 μ ) + ϵ t − k − 1 ( θ 0 μ ) ⊗ I r ) Π t − k − 1 , t − h − 1 ϵ ( I r ⊗ ϵ t − h − 1 ( θ 0 μ ) + ϵ t − h − 1 ( θ 0 μ ) ⊗ I r ) ⊤ W r K 0 σ ⊤ W r ⊤ ( W r ( Φ 0 σ − K 0 σ ) W r ⊤ ) h ⊤ ,
where
Π s , u ϵ = lim N → ∞ 1 N ∑ i = 1 N ∑ j = 1 N E [ ( Σ Λ − 1 λ 0 i ϵ i s − ∑ a = 0 s − 1 ( Φ 0 μ − K 0 μ ) a K 0 μ Σ Λ − 1 λ 0 i ϵ i , s − a − 1 ) ( Σ Λ − 1 λ 0 j ϵ j u − ∑ b = 0 u − 1 ( Φ 0 μ − K 0 μ ) b K 0 μ Σ Λ − 1 λ 0 j ϵ j , u − b − 1 ) ⊤ ] , \mathbf{\Pi}^{\epsilon}_{s,u}=\lim_{N\to\infty}\frac{1}{N}\sum_{i=1}^{N}\sum_{j=1}^{N}\mathbb{E}\left[\left(\mathbf{\Sigma}_{\Lambda}^{-1}\boldsymbol{\lambda}_{0i}\epsilon_{is}-\sum_{a=0}^{s-1}\left(\mathbf{\Phi}^{\mu}_0-\mathbf{K}^{\mu}_0\right)^a\mathbf{K}^{\mu}_0\mathbf{\Sigma}_{\Lambda}^{-1}\boldsymbol{\lambda}_{0i}\epsilon_{i,s-a-1}\right)
\left(\mathbf{\Sigma}_{\Lambda}^{-1}\boldsymbol{\lambda}_{0j}\epsilon_{ju}-\sum_{b=0}^{u-1}\left(\mathbf{\Phi}^{\mu}_0-\mathbf{K}^{\mu}_0\right)^b\mathbf{K}^{\mu}_0\mathbf{\Sigma}_{\Lambda}^{-1}\boldsymbol{\lambda}_{0j}\epsilon_{j,u-b-1}\right)^{\top}\right], Π s , u ϵ = N → ∞ lim N 1 i = 1 ∑ N j = 1 ∑ N E ( Σ Λ − 1 λ 0 i ϵ i s − a = 0 ∑ s − 1 ( Φ 0 μ − K 0 μ ) a K 0 μ Σ Λ − 1 λ 0 i ϵ i , s − a − 1 ) ( Σ Λ − 1 λ 0 j ϵ j u − b = 0 ∑ u − 1 ( Φ 0 μ − K 0 μ ) b K 0 μ Σ Λ − 1 λ 0 j ϵ j , u − b − 1 ) ⊤ ,
is the limiting covariance of the cross-sectional PC error after subtracting the part propagated through the conditional-location filter.
Monte Carlo study
Data generating process. For each replication and for t = 1 , … , T t=1,\ldots,T t = 1 , … , T , the observed N N N -dimensional time series { x t } t ∈ Z \{x_t\}_{t\in\mathbb{Z}} { x t } t ∈ Z is generated by a static factor model as in equation (1) . The matrix of factor loadings is generated according to a Gaussian random vector, such that λ i ∼ N ( 0 r , Σ Λ ) \boldsymbol{\lambda}_i\sim\mathcal{N}(\mathbf{0}_r,\mathbf{\Sigma}_{\Lambda}) λ i ∼ N ( 0 r , Σ Λ ) , for each i = 1 , … , N i=1,\ldots,N i = 1 , … , N , independently across i i i and where Σ Λ \mathbf{\Sigma}_{\Lambda} Σ Λ is a diagonal r × r r\times r r × r matrix with distinct entries in [ 0.6 , 1.6 ] [0.6,1.6] [ 0.6 , 1.6 ] . We consider the cases with r ∈ { 1 , 3 } r\in\{1,3\} r ∈ { 1 , 3 } factors. The idiosyncratic term is generated purely as cross-sectionally correlated Gaussian noise with no time dynamics, and then de-meaned. In particular, we set ϵ t ∼ N ( 0 N , Σ e ) \epsilon_t\sim\mathcal{N}(\mathbf{0}_N,\mathbf{\Sigma}_e) ϵ t ∼ N ( 0 N , Σ e ) , where Σ e = Γ 1 1 / 2 Γ 2 Γ 1 1 / 2 \mathbf{\Sigma}_e=\mathbf{\Gamma}_1^{1/2}\mathbf{\Gamma}_2\mathbf{\Gamma}_1^{1/2} Σ e = Γ 1 1/2 Γ 2 Γ 1 1/2 , with Γ 1 = diag ( s 1 , … , s N ) \mathbf{\Gamma}_1=\operatorname{diag}(s_1,\ldots,s_N) Γ 1 = diag ( s 1 , … , s N ) with s i ∼ Unif ( 0.5 , 1.5 ) s_i\sim\operatorname{Unif}(0.5,1.5) s i ∼ Unif ( 0.5 , 1.5 ) , and Γ 2 \mathbf{\Gamma}_2 Γ 2 is Toeplitz with entries [ Γ 2 ] i j = 0.5 ∣ i − j ∣ 1 { ∣ i − j ∣ ≤ 10 } [\mathbf{\Gamma}_2]_{ij}=0.5^{|i-j|}\mathbb{1}\{|i-j|\leq10\} [ Γ 2 ] ij = 0. 5 ∣ i − j ∣ 1 { ∣ i − j ∣ ≤ 10 } .
In the conditional location case, the latent factors satisfy equation (4) , and the conditional location process follows the score-driven recursion in equation (5) . We consider alternative distributions for the innovation term. Specifically, we set v t ∼ N ( 0 r , Σ ) \boldsymbol{v}_t\sim\mathcal{N}(\mathbf{0}_r,\mathbf{\Sigma}) v t ∼ N ( 0 r , Σ ) in the Gaussian case, and v t ∼ t ν ( 0 r , Σ ) \boldsymbol{v}_t\sim t_{\nu}(\mathbf{0}_r,\mathbf{\Sigma}) v t ∼ t ν ( 0 r , Σ ) in the heavy-tailed case, with ν ∈ { 4 , 5 } \nu\in\{4,5\} ν ∈ { 4 , 5 } , so that the score-driven recursion satisfies the Gaussian updates case, in the former, or Student’s t t t updates in the latter. We fix the score-recursion parameters ex ante by setting Φ = 0.8 ⋅ I r \mathbf{\Phi}=0.8\cdot\mathbf{I}_r Φ = 0.8 ⋅ I r , K = 0.15 ⋅ I r \mathbf{K}=0.15\cdot\mathbf{I}_r K = 0.15 ⋅ I r , and drawing Σ = diag ( 10 ⋅ ∣ z 1 ∣ , … , 10 ⋅ ∣ z r ∣ ) \mathbf{\Sigma}=\operatorname{diag}(10\cdot|z_1|,\ldots,10\cdot|z_r|) Σ = diag ( 10 ⋅ ∣ z 1 ∣ , … , 10 ⋅ ∣ z r ∣ ) with z l ∼ N ( 0 , 1 ) z_l\sim\mathcal{N}(0,1) z l ∼ N ( 0 , 1 ) for l = 1 , … , r l = 1, \ldots, r l = 1 , … , r . In the conditional variance case, the latent factors satisfy equation (8) , and the conditional variance process follows the score-driven recursion in equation (9) . Analogously to the previous case, we consider two distributions for the innovation term, namely v t ∼ N ( 0 r , I r ) \boldsymbol{v}_t \sim\mathcal{N}(\boldsymbol{0}_r, \boldsymbol{I}_r) v t ∼ N ( 0 r , I r ) in the Gaussian case, and v t ∼ t ν ( 0 r , I r ) \boldsymbol{v}_t \sim t_{\nu}(\boldsymbol{0}_r, \boldsymbol{I}_r) v t ∼ t ν ( 0 r , I r ) in the Student’s t t t case, with ν ∈ { 4 , 5 } \nu\in\{4,5\} ν ∈ { 4 , 5 } , and the score-driven recursion modifies accordingly with the Gaussian updates, in the former, and Student’s t t t updates in the latter. For the score-driven parameters, we set Φ = 0.8 ⋅ I r \boldsymbol{\Phi}=0.8\cdot\boldsymbol{I}_r Φ = 0.8 ⋅ I r , K = 0.15 ⋅ I r \boldsymbol{K}=0.15\cdot\boldsymbol{I}_r K = 0.15 ⋅ I r and fix δ = diag ( I r − Φ ) \boldsymbol{\delta}=\operatorname{diag}(\boldsymbol{I}_r-\boldsymbol{\Phi}) δ = diag ( I r − Φ ) .
Results. In the following, we report results averaged over the Monte Carlo replications. Factors and loadings are always estimated via PCA. Since the numerical performance of PCA has been thoroughly investigated in many works, we do not report results on estimation of the factors but we directly focus on the QMLE of the score driven parameters and the implied filtered processes and prediction intervals. Results obtained using the spatial Kendall’s tau PCs approach by He et al. (2020) are in Appendix § 17 .§ 2 .
First, we verify consistency of QMLE. According to Theorem 1 the QMLE estimator θ ^ \hat{\boldsymbol{\theta}} θ ^ of the score-driven parameters is a consistent estimator of r H 0 ( θ 0 ) \boldsymbol{r}_{H_0}(\boldsymbol{\theta}_0) r H 0 ( θ 0 ) . Table 1 report the squared bias and variance for θ ^ \hat{\boldsymbol{\theta}} θ ^ averaged over all replications, where, in agreement with Proposition 1, H 0 \boldsymbol{H}_0 H 0 is estimated as the coefficient H ^ \hat{\boldsymbol{H}} H ^ of the linear projection of F ^ \hat{\boldsymbol{F}} F ^ onto F \boldsymbol{F} F and the mapping r H 0 ( ⋅ ) \boldsymbol{r}_{H_0}(\cdot) r H 0 ( ⋅ ) is given in Appendix C.
Table 1. Monte Carlo simulation results for the QMLE of the conditional location and variance models. Monte Carlo averages of the squared bias and variance of the QMLE. For the conditional location models, entries are arithmetic means of the coordinate-specific values for the common dynamic parameters vec ( Φ ) \operatorname{vec}(\boldsymbol{\Phi}) vec ( Φ ) and vec ( K ) \operatorname{vec}(\boldsymbol{K}) vec ( K ) . For the conditional variance models, the averages additionally include the intercept parameters δ \boldsymbol{\delta} δ . Results are based on 100 replications.
The columns under conditional locations report PICPC _C C , the empirical coverage probability of the prediction interval for the common conditional location C i , t ∣ t − 1 C_{i,t|t-1} C i , t ∣ t − 1 . The columns under conditional variances report PICPP _P P , the empirical coverage probability of the prediction interval for the common conditional variance P i , t ∣ t − 1 P_{i,t|t-1} P i , t ∣ t − 1 . The nominal coverage probability is 0.95. Results are based on 300 replications.
Second, we verify Theorems 4 and 5 by computing the empirical coverage of the corresponding 95% asymptotic prediction intervals. For the Gaussian models, the intervals are constructed using Theorem 5, whereas for the Student’s t t t models they are constructed using Theorem 4. The asymptotic covariance matrices are estimated as described at the end of § 4.5 . Table 2 reports the coverage probabilities for both the common conditional locations and the common conditional variances.
Table 2. Monte Carlo coverage probabilities for the common conditional locations and variances
Third, we compute the asymptotic prediction intervals implied by Proposition 6 when using the joint conditional location–variance estimator. Table 3 reports the coverage probabilities for the joint common conditional locations and the common conditional variances. In Appendix § 17 .§ 1 we report additional results on the goodness of the recovery of the time-varying parameters { ψ ^ t ∣ t − 1 ( θ ) } t ∈ N \{\widehat{\psi}_{t|t-1}(\theta)\}_{t\in\mathbb{N}} { ψ t ∣ t − 1 ( θ ) } t ∈ N .
Table 3. Monte Carlo coverage probabilities for the joint conditional location and variance models
PICPC is the empirical coverage probability of the prediction interval for the common conditional location C i , t ∣ t − 1 C_{i,t|t-1} C i , t ∣ t − 1 , whereas PICPP is the empirical coverage probability of the prediction interval for the common conditional variance P i , t ∣ t − 1 P_{i,t|t-1} P i , t ∣ t − 1 . The nominal coverage probability is 0.95. Results are based on 300 replications.
Score for the conditional variance with Student's t t t updates
Following the proof of Creal et al. [16 ]
u t = 1 2 S t − 1 W r T ( J t ⊗ J t T ) ( 1 + ν − 1 r 1 + ν − 1 ( F t T J t T J t F t − 2 ) ( J t F t ⊗ J t F t ) − vec I r ) , \boldsymbol{u}_{t} = \frac{1}{2}S_{t}^{-1}W_{r}^{\mathsf{T}}\left(\mathbf{J}_{t} \otimes\mathbf{J}_{t}^{\mathsf{T}}\right)\left(\frac{1+\nu^{-1}r}{1+\nu^{-1}\left(F_{t}^{\mathsf{T}}\mathbf{J}_{t}^{\mathsf{T}}\mathbf{J}_{t}F_{t}-2\right)}\left(\mathbf{J}_{t}F_{t} \otimes\mathbf{J}_{t}F_{t}\right)-\operatorname{vec}I_{r}\right), u t = 2 1 S t − 1 W r T ( J t ⊗ J t T ) ( 1 + ν − 1 ( F t T J t T J t F t − 2 ) 1 + ν − 1 r ( J t F t ⊗ J t F t ) − vec I r ) ,
where J t \mathbf{J}_{t} J t is such that Σ t ∣ t − 1 − 1 = J t T J t \boldsymbol{\Sigma}_{t|t-1}^{-1}=\mathbf{J}_{t}^{\mathsf{T}}\mathbf{J}_{t} Σ t ∣ t − 1 − 1 = J t T J t and J t \mathbf{J}_{t} J t can be obtained from any convenient matrix decomposition procedure. Since Σ t ∣ t − 1 \boldsymbol{\Sigma}_{t|t-1} Σ t ∣ t − 1 is diagonal and positive definite we can set J t = Σ t ∣ t − 1 − 1 / 2 \mathbf{J}_{t}=\boldsymbol{\Sigma}_{t|t-1}^{-1/2} J t = Σ t ∣ t − 1 − 1/2 and J t T = Σ t ∣ t − 1 − 1 / 2 \mathbf{J}_{t}^{\mathsf{T}}=\boldsymbol{\Sigma}_{t|t-1}^{-1/2} J t T = Σ t ∣ t − 1 − 1/2 , hence,
u t = 1 2 S t − 1 W r T ( Σ t ∣ t − 1 − 1 / 2 ⊗ Σ t ∣ t − 1 − 1 / 2 ) ( 1 + ν − 1 r 1 + ν − 1 ( F t T Σ t ∣ t − 1 − 1 F t − 2 ) ( Σ t ∣ t − 1 − 1 / 2 F t ⊗ Σ t ∣ t − 1 − 1 / 2 F t ) − vec I r ) , \boldsymbol{u}_{t} = \frac{1}{2}S_{t}^{-1}W_{r}^{\mathsf{T}}\left(\boldsymbol{\Sigma}_{t|t-1}^{-1/2} \otimes\boldsymbol{\Sigma}_{t|t-1}^{-1/2}\right)\left(\frac{1+\nu^{-1}r}{1+\nu^{-1}\left(F_{t}^{\mathsf{T}}\boldsymbol{\Sigma}_{t|t-1}^{-1}F_{t}-2\right)}\left(\boldsymbol{\Sigma}_{t|t-1}^{-1/2}F_{t} \otimes\boldsymbol{\Sigma}_{t|t-1}^{-1/2}F_{t}\right)-\operatorname{vec}I_{r}\right), u t = 2 1 S t − 1 W r T ( Σ t ∣ t − 1 − 1/2 ⊗ Σ t ∣ t − 1 − 1/2 ) 1 + ν − 1 ( F t T Σ t ∣ t − 1 − 1 F t − 2 ) 1 + ν − 1 r ( Σ t ∣ t − 1 − 1/2 F t ⊗ Σ t ∣ t − 1 − 1/2 F t ) − vec I r ,
or, equivalently,
u t = 1 2 S t − 1 W r T ( Σ t ∣ t − 1 − 1 ⊗ Σ t ∣ t − 1 − 1 ) ( ν + r ν − 2 + F t T Σ t ∣ t − 1 − 1 F t ( F t ⊗ F t ) − vec Σ t ∣ t − 1 ) , \boldsymbol{u}_{t} = \frac{1}{2}S_{t}^{-1}W_{r}^{\mathsf{T}}\left(\boldsymbol{\Sigma}_{t|t-1}^{-1} \otimes\boldsymbol{\Sigma}_{t|t-1}^{-1}\right)\left(\frac{\nu+r}{\nu-2+F_{t}^{\mathsf{T}}\boldsymbol{\Sigma}_{t|t-1}^{-1}F_{t}}\left(F_{t} \otimes F_{t}\right)-\operatorname{vec}\boldsymbol{\Sigma}_{t|t-1}\right), u t = 2 1 S t − 1 W r T ( Σ t ∣ t − 1 − 1 ⊗ Σ t ∣ t − 1 − 1 ) ( ν − 2 + F t T Σ t ∣ t − 1 − 1 F t ν + r ( F t ⊗ F t ) − vec Σ t ∣ t − 1 ) ,
Furthermore, by adopting the rescaling
S t − 1 = 2 W N T ( Σ t ∣ t − 1 − 1 ⊗ Σ t ∣ t − 1 − 1 ) W N , S_{t}^{-1}=2W_{N}^{\mathsf{T}}\left(\boldsymbol{\Sigma}_{t|t-1}^{-1} \otimes\boldsymbol{\Sigma}_{t|t-1}^{-1}\right)W_{N}, S t − 1 = 2 W N T ( Σ t ∣ t − 1 − 1 ⊗ Σ t ∣ t − 1 − 1 ) W N ,
we have
u t = W r T ( ν + r ν − 2 + F t T Σ t ∣ t − 1 − 1 F t ( F t ⊗ F t ) − vec Σ t ∣ t − 1 ) . \boldsymbol{u}_{t}=W_{r}^{\mathsf{T}}\left(\frac{\nu+r}{\nu-2+F_{t}^{\mathsf{T}}\boldsymbol{\Sigma}_{t|t-1}^{-1}F_{t}}\left(F_{t}\otimes F_{t}\right)-\operatorname{vec}\boldsymbol{\Sigma}_{t|t-1}\right). u t = W r T ( ν − 2 + F t T Σ t ∣ t − 1 − 1 F t ν + r ( F t ⊗ F t ) − vec Σ t ∣ t − 1 ) .
And further calculations show that
S t = 1 2 W r T ( Σ t ∣ t − 1 − 1 ⊗ Σ t ∣ t − 1 − 1 ) W r , S_{t}=\frac{1}{2}W_{r}^{\mathsf{T}}\left(\boldsymbol{\Sigma}_{t|t-1}^{-1}\otimes\boldsymbol{\Sigma}_{t|t-1}^{-1}\right)W_{r}, S t = 2 1 W r T ( Σ t ∣ t − 1 − 1 ⊗ Σ t ∣ t − 1 − 1 ) W r ,
which is proportional to the conditional Fisher Information matrix used by Creal et al. [16 ] who show that:
S t = 1 4 W r T ( Σ t ∣ t − 1 − 1 / 2 ⊗ Σ t ∣ t − 1 − 1 / 2 ) ( g G − vec I r vec I r T ) ( Σ t ∣ t − 1 − 1 / 2 ⊗ Σ t ∣ t − 1 − 1 / 2 ) W r , S_{t}=\frac{1}{4}W_{r}^{\mathsf{T}}\left(\boldsymbol{\Sigma}_{t|t-1}^{-1/2}\otimes\boldsymbol{\Sigma}_{t|t-1}^{-1/2}\right)\left(gG-\operatorname{vec}I_{r}\operatorname{vec}I_{r}^{\mathsf{T}}\right)\left(\boldsymbol{\Sigma}_{t|t-1}^{-1/2}\otimes\boldsymbol{\Sigma}_{t|t-1}^{-1/2}\right)W_{r}, S t = 4 1 W r T ( Σ t ∣ t − 1 − 1/2 ⊗ Σ t ∣ t − 1 − 1/2 ) ( g G − vec I r vec I r T ) ( Σ t ∣ t − 1 − 1/2 ⊗ Σ t ∣ t − 1 − 1/2 ) W r ,
with g = ν + r ν + r + 2 g=\frac{\nu+r}{\nu+r+2} g = ν + r + 2 ν + r and: G [ ( i − 1 ) N + ℓ , ( j − 1 ) r + m ] = δ i j δ ℓ m + δ i ℓ δ j m + δ i m δ j ℓ G\left[(i-1)N+\ell,(j-1)r+m\right]=\delta_{ij}\delta_{\ell m}+\delta_{i\ell}\delta_{jm}+\delta_{im}\delta_{j\ell} G [ ( i − 1 ) N + ℓ , ( j − 1 ) r + m ] = δ ij δ ℓ m + δ i ℓ δ j m + δ im δ j ℓ , where δ i j \delta_{ij} δ ij is the Kronecker delta.
Conditional variance models with leverage effects
The time-varying variance models introduced in § 2 can be extended to incorporate leverage effects, in the spirit of the GJR-GARCH model of Glosten et al. [32 ] and the asymmetric vector GARCH model of McAleer et al. [42 ] . In particular, for the multivariate multiplicative model in (8) , we augment the updating equation in (9) to allow for asymmetric effects across equations, so that the model becomes
F t = q ( ψ t ∣ t − 1 , v t ) = Σ t ∣ t − 1 1 / 2 v t , vec Σ t ∣ t − 1 = W r σ t ∣ t − 1 2 , (29) F_t=q\left(\psi_{t|t-1},v_t\right)=\Sigma_{t|t-1}^{1/2}v_t,\qquad\operatorname{vec}\Sigma_{t|t-1}=W_r\sigma_{t|t-1}^{2}, \tag*{(29)} F t = q ( ψ t ∣ t − 1 , v t ) = Σ t ∣ t − 1 1/2 v t , vec Σ t ∣ t − 1 = W r σ t ∣ t − 1 2 , (29)
σ t + 1 ∣ t 2 = δ + Φ σ t ∣ t − 1 2 + ( K + Υ I ( F t ) ) u t , (30) \sigma_{t+1|t}^{2}=\delta+\Phi\sigma_{t|t-1}^{2}+\left(K+\Upsilon I\left(F_t\right)\right)u_t, \tag*{(30)} σ t + 1∣ t 2 = δ + Φ σ t ∣ t − 1 2 + ( K + Υ I ( F t ) ) u t , (30)
where Υ ∈ R r × r \Upsilon\in\mathbb{R}^{r\times r} Υ ∈ R r × r is the additional matrix of static parameters and I ( F t ) = diag ( 1 { F 1 t ≤ 0 } , … , 1 { F r t ≤ 0 } ) I(F_t)=\operatorname{diag}\left(1_{\{F_{1t}\leq0\}},\ldots,1_{\{F_{rt}\leq0\}}\right) I ( F t ) = diag ( 1 { F 1 t ≤ 0 } , … , 1 { F r t ≤ 0 } ) is an r × r r\times r r × r diagonal matrix with indicator functions on the main diagonal.
The new specification for the time-varying conditional variances in (30) is valid for both the Gaussian or Student’s t t t updates, that is, by either assuming that noise process { v t } t ∈ Z \{v_t\}_{t\in\mathbb{Z}} { v t } t ∈ Z satisfies v t ∼ N ( 0 r , I r ) v_t\sim\mathcal{N}(0_r,I_r) v t ∼ N ( 0 r , I r ) or v t ∼ t ν ( 0 r , I r ) v_t\sim t_\nu(0_r,I_r) v t ∼ t ν ( 0 r , I r ) , respectively. Consequently, once the factors has been retrieved via PCA, the unknown parameter vector, which is given by θ = ( δ T , vec Φ T , vec K T , vec Υ T ) T \theta=(\delta^\mathsf{T},\operatorname{vec}\Phi^\mathsf{T},\operatorname{vec}K^\mathsf{T},\operatorname{vec}\Upsilon^\mathsf{T})^\mathsf{T} θ = ( δ T , vec Φ T , vec K T , vec Υ T ) T for Gaussian updates or is defined as θ = ( ν , δ T , vec Φ T , vec K T , vec Υ T ) T \theta=(\nu,\delta^\mathsf{T},\operatorname{vec}\Phi^\mathsf{T},\operatorname{vec}K^\mathsf{T},\operatorname{vec}\Upsilon^\mathsf{T})^\mathsf{T} θ = ( ν , δ T , vec Φ T , vec K T , vec Υ T ) T for Student’s t t t updates, can be easily estimated using the QMLE in (14) , by employing either the log-likelihood in (10) or in (11) , respectively. Therefore, the two-stage estimation strategy outlined in § 3 can be readily employed, and the asymptotic theory developed in § 4 applies straightforwardly.
Parameter identifications and the mapping r H ( ⋅ ) r_H(\cdot) r H ( ⋅ )
For a generic r × r r\times r r × r invertible matrix H H H consider the mapping θ H = r H ( θ 0 ) \theta_H=r_H(\theta_0) θ H = r H ( θ 0 ) , such that θ H \theta_H θ H are the parameters governing the dynamics of the transformed factors H F t HF_t H F t . We give below the explicit forms of the time-varying means or variances when computed using the transformed parameters, i.e., ψ t ∣ t − 1 ( θ H ) \psi_{t|t-1}(\theta_H) ψ t ∣ t − 1 ( θ H ) which is the solution of (17) when using as parameters θ H \theta_H θ H instead of θ 0 \theta_0 θ 0 .
When using H F t HF_t H F t , then (16) becomes
H F t = H q ( ψ t ∣ t − 1 ( θ 0 ) , v t ) = { H μ t ∣ t − 1 ( θ 0 ) + H v t , v t ∼ ( 0 r , Σ 0 ) , H Σ t ∣ t − 1 1 / 2 ( θ 0 ) v t , v t ∼ ( 0 r , I r ) . (31) HF_t=Hq\left(\psi_{t|t-1}(\theta_0),v_t\right)=
\begin{cases}
H\mu_{t|t-1}(\theta_0)+Hv_t, & v_t\sim(0_r,\Sigma_0),\\
H\Sigma_{t|t-1}^{1/2}(\theta_0)v_t, & v_t\sim(0_r,I_r).
\end{cases} \tag*{(31)} H F t = H q ( ψ t ∣ t − 1 ( θ 0 ) , v t ) = { H μ t ∣ t − 1 ( θ 0 ) + H v t , H Σ t ∣ t − 1 1/2 ( θ 0 ) v t , v t ∼ ( 0 r , Σ 0 ) , v t ∼ ( 0 r , I r ) . (31)
Time-varying locations
From (31) ,
μ t ∣ t − 1 ( θ H ) = H μ t ∣ t − 1 ( θ 0 ) . (32) \mu_{t|t-1}(\theta_H)=H\mu_{t|t-1}(\theta_0). \tag*{(32)} μ t ∣ t − 1 ( θ H ) = H μ t ∣ t − 1 ( θ 0 ) . (32)
Therefore, for Gaussian updates
μ t + 1 ∣ t ( θ H ) = ( H Φ 0 H − 1 ) H μ t ∣ t − 1 ( θ 0 ) + ( H K 0 H − 1 ) H ( F t − μ t ∣ t − 1 ( θ 0 ) ) = Φ H μ t ∣ t − 1 ( θ H ) + K H ( H F t − μ t ∣ t − 1 ( θ H ) ) , \begin{aligned}
\mu_{t+1|t}(\theta_H) &= \left(H\Phi_0H^{-1}\right)H\mu_{t|t-1}(\theta_0)+\left(HK_0H^{-1}\right)H\left(F_t-\mu_{t|t-1}(\theta_0)\right) \\
&= \Phi_H\mu_{t|t-1}(\theta_H)+K_H\left(HF_t-\mu_{t|t-1}(\theta_H)\right),
\end{aligned} μ t + 1∣ t ( θ H ) = ( H Φ 0 H − 1 ) H μ t ∣ t − 1 ( θ 0 ) + ( H K 0 H − 1 ) H ( F t − μ t ∣ t − 1 ( θ 0 ) ) = Φ H μ t ∣ t − 1 ( θ H ) + K H ( H F t − μ t ∣ t − 1 ( θ H ) ) ,
and Σ H = H Σ 0 H T \Sigma_H=H\Sigma_0H^\mathsf{T} Σ H = H Σ 0 H T . While, for Student’s t t t updates
μ t + 1 ∣ t ( θ H ) = ( H Φ 0 H − 1 ) H μ t ∣ t − 1 ( θ 0 ) + ( H K 0 H − 1 ) H F t − μ t ∣ t − 1 ( θ 0 ) 1 + ( F t − μ t ∣ t − 1 ( θ 0 ) ) T H T ( H Σ 0 H T ) − 1 H ( F t − μ t ∣ t − 1 ( θ 0 ) ) / ν 0 = Φ H μ t ∣ t − 1 ( θ H ) + K H H F t − μ t ∣ t − 1 ( θ H ) 1 + ( H F t − μ t ∣ t − 1 ( θ H ) ) T Σ H − 1 ( H F t − μ t ∣ t − 1 ( θ H ) ) / ν H , \begin{aligned}
\mu_{t+1|t}(\theta_H) &= \left(H\Phi_0H^{-1}\right)H\mu_{t|t-1}(\theta_0) \\
&\quad+\left(HK_0H^{-1}\right)H\frac{F_t-\mu_{t|t-1}(\theta_0)}{1+\left(F_t-\mu_{t|t-1}(\theta_0)\right)^\mathsf{T}H^\mathsf{T}\left(H\Sigma_0H^\mathsf{T}\right)^{-1}H\left(F_t-\mu_{t|t-1}(\theta_0)\right)/\nu_0} \\
&= \Phi_H\mu_{t|t-1}(\theta_H)+K_H\frac{HF_t-\mu_{t|t-1}(\theta_H)}{1+\left(HF_t-\mu_{t|t-1}(\theta_H)\right)^\mathsf{T}\Sigma_H^{-1}\left(HF_t-\mu_{t|t-1}(\theta_H)\right)/\nu_H},
\end{aligned} μ t + 1∣ t ( θ H ) = ( H Φ 0 H − 1 ) H μ t ∣ t − 1 ( θ 0 ) + ( H K 0 H − 1 ) H 1 + ( F t − μ t ∣ t − 1 ( θ 0 ) ) T H T ( H Σ 0 H T ) − 1 H ( F t − μ t ∣ t − 1 ( θ 0 ) ) / ν 0 F t − μ t ∣ t − 1 ( θ 0 ) = Φ H μ t ∣ t − 1 ( θ H ) + K H 1 + ( H F t − μ t ∣ t − 1 ( θ H ) ) T Σ H − 1 ( H F t − μ t ∣ t − 1 ( θ H ) ) / ν H H F t − μ t ∣ t − 1 ( θ H ) ,
and ν H = ν 0 \nu_H=\nu_0 ν H = ν 0 .
Time-varying covariances
In this case, by taking the expectation of the outer product of (31) , we have Σ t ∣ t − 1 ( θ H ) = H Σ t ∣ t − 1 ( θ 0 ) H T \Sigma_{t|t-1}(\theta_H)=H\Sigma_{t|t-1}(\theta_0)H^\mathsf{T} Σ t ∣ t − 1 ( θ H ) = H Σ t ∣ t − 1 ( θ 0 ) H T , so
vec Σ t ∣ t − 1 ( θ H ) = W r σ t ∣ t − 1 2 ( θ H ) = ( H ⊗ H ) W r σ t ∣ t − 1 2 ( θ 0 ) , \operatorname{vec}\Sigma_{t|t-1}(\theta_H)=W_r\sigma^2_{t|t-1}(\theta_H)=(H\otimes H)W_r\sigma^2_{t|t-1}(\theta_0), vec Σ t ∣ t − 1 ( θ H ) = W r σ t ∣ t − 1 2 ( θ H ) = ( H ⊗ H ) W r σ t ∣ t − 1 2 ( θ 0 ) ,
which implies
σ t ∣ t − 1 2 ( θ H ) = W r T ( H ⊗ H ) W r σ t ∣ t − 1 2 ( θ 0 ) . (33) \sigma^2_{t|t-1}(\theta_H)=W_r^\mathsf{T}(H\otimes H)W_r\sigma^2_{t|t-1}(\theta_0). \tag*{(33)} σ t ∣ t − 1 2 ( θ H ) = W r T ( H ⊗ H ) W r σ t ∣ t − 1 2 ( θ 0 ) . (33)
Therefore, for Gaussian updates
σ t + 1 ∣ t 2 ( θ H ) = W r T ( H ⊗ H ) W r δ 0 + W r T ( H ⊗ H ) W r Φ 0 { W r T ( H ⊗ H ) W r } − 1 W r T ( H ⊗ H ) W r σ t ∣ t − 1 2 ( θ 0 ) + W r T ( H ⊗ H ) W r K 0 { W r T ( H ⊗ H ) W r } − 1 × W r T ( H ⊗ H ) W r W r T ( ( F t ⊗ F t ) − W r σ t ∣ t − 1 2 ( θ 0 ) ) = δ H + Φ H σ t ∣ t − 1 2 ( θ H ) + K H W r T ( ( H F t ⊗ H F t ) − vec Σ t ∣ t − 1 ( θ H ) ) . \begin{aligned}
\sigma^2_{t+1|t}(\theta_H) &= W_r^\mathsf{T}(H\otimes H)W_r\delta_0 \\
&\quad+W_r^\mathsf{T}(H\otimes H)W_r\Phi_0\left\{W_r^\mathsf{T}(H\otimes H)W_r\right\}^{-1}W_r^\mathsf{T}(H\otimes H)W_r\sigma^2_{t|t-1}(\theta_0) \\
&\quad+W_r^\mathsf{T}(H\otimes H)W_rK_0\left\{W_r^\mathsf{T}(H\otimes H)W_r\right\}^{-1} \\
&\qquad\qquad\times W_r^\mathsf{T}(H\otimes H)W_rW_r^\mathsf{T}\left((F_t\otimes F_t)-W_r\sigma^2_{t|t-1}(\theta_0)\right) \\
&= \delta_H + \Phi_H \sigma^2_{t|t-1}(\theta_H) + K_H W_r^\mathsf{T}\left((HF_t \otimes HF_t) - \operatorname{vec}\Sigma_{t|t-1}(\theta_H)\right).
\end{aligned} σ t + 1∣ t 2 ( θ H ) = W r T ( H ⊗ H ) W r δ 0 + W r T ( H ⊗ H ) W r Φ 0 { W r T ( H ⊗ H ) W r } − 1 W r T ( H ⊗ H ) W r σ t ∣ t − 1 2 ( θ 0 ) + W r T ( H ⊗ H ) W r K 0 { W r T ( H ⊗ H ) W r } − 1 × W r T ( H ⊗ H ) W r W r T ( ( F t ⊗ F t ) − W r σ t ∣ t − 1 2 ( θ 0 ) ) = δ H + Φ H σ t ∣ t − 1 2 ( θ H ) + K H W r T ( ( H F t ⊗ H F t ) − vec Σ t ∣ t − 1 ( θ H ) ) .
While, for Student’s t t t updates
σ t + 1 ∣ t 2 ( θ H ) = W r T ( H ⊗ H ) W r δ 0 + W r T ( H ⊗ H ) W r Φ 0 { W r T ( H ⊗ H ) W r } − 1 W r T ( H ⊗ H ) W r σ t ∣ t − 1 2 ( θ 0 ) + W r T ( H ⊗ H ) W r K 0 { W r T ( H ⊗ H ) W r } − 1 W r T ( H ⊗ H ) W r × W r T ( ν + N ν − 2 + F t T H T ( H Σ t ∣ t − 1 ( θ 0 ) H T ) − 1 H F t ( F t ⊗ F t ) − W r σ t ∣ t − 1 2 ( θ 0 ) ) = δ H + Φ H σ t ∣ t − 1 2 ( θ H ) + K H W r T ( ν + N ν − 2 + F t T H T ( Σ t ∣ t − 1 ( θ H ) ) − 1 H F t ( H F t ⊗ H F t ) − vec Σ t ∣ t − 1 ( θ H ) ) . \begin{aligned}
\sigma^2_{t+1|t}(\theta_H) ={}& W_r^\mathsf{T}(H \otimes H)W_r\delta_0 \\
&+ W_r^\mathsf{T}(H \otimes H)W_r\Phi_0\left\{W_r^\mathsf{T}(H \otimes H)W_r\right\}^{-1}W_r^\mathsf{T}(H \otimes H)W_r\sigma^2_{t|t-1}(\theta_0) \\
&+ W_r^\mathsf{T}(H \otimes H)W_rK_0\left\{W_r^\mathsf{T}(H \otimes H)W_r\right\}^{-1}W_r^\mathsf{T}(H \otimes H)W_r \\
&\quad\times W_r^\mathsf{T}\left(\frac{\nu+N}{\nu-2+F_t^\mathsf{T}H^\mathsf{T}\left(H\Sigma_{t|t-1}(\theta_0)H^\mathsf{T}\right)^{-1}HF_t}(F_t \otimes F_t)-W_r\sigma^2_{t|t-1}(\theta_0)\right) \\
={}& \delta_H + \Phi_H\sigma^2_{t|t-1}(\theta_H) \\
&+ K_HW_r^\mathsf{T}\left(\frac{\nu+N}{\nu-2+F_t^\mathsf{T}H^\mathsf{T}\left(\Sigma_{t|t-1}(\theta_H)\right)^{-1}HF_t}(HF_t \otimes HF_t)-\operatorname{vec}\Sigma_{t|t-1}(\theta_H)\right).
\end{aligned} σ t + 1∣ t 2 ( θ H ) = = W r T ( H ⊗ H ) W r δ 0 + W r T ( H ⊗ H ) W r Φ 0 { W r T ( H ⊗ H ) W r } − 1 W r T ( H ⊗ H ) W r σ t ∣ t − 1 2 ( θ 0 ) + W r T ( H ⊗ H ) W r K 0 { W r T ( H ⊗ H ) W r } − 1 W r T ( H ⊗ H ) W r × W r T ( ν − 2 + F t T H T ( H Σ t ∣ t − 1 ( θ 0 ) H T ) − 1 H F t ν + N ( F t ⊗ F t ) − W r σ t ∣ t − 1 2 ( θ 0 ) ) δ H + Φ H σ t ∣ t − 1 2 ( θ H ) + K H W r T ( ν − 2 + F t T H T ( Σ t ∣ t − 1 ( θ H ) ) − 1 H F t ν + N ( H F t ⊗ H F t ) − vec Σ t ∣ t − 1 ( θ H ) ) .
Proofs
Proof of Proposition 1
First, let us notice that all assumptions in Bai [2 ] hold also in our setting. Indeed, 1. Assumption A in Bai [2 ] is equivalent to our Assumption 2 ( i i i ) 2(iii) 2 ( iii ) plus Remark 3; 2. Assumption B in Bai [2 ] is equivalent to our Assumption 3; 3. Assumption C.1 in Bai [2 ] is not necessary since, from closer inspection of the original proofs, we see that we just need E [ e i t 4 ] < ∞ \mathbb{E}[e_{it}^{4}] < \infty E [ e i t 4 ] < ∞ and not finite 8th moments and this follows from our Assumption 4 ( i v ) 4(iv) 4 ( i v ) ; 4. Assumptions C.2-C.4 in Bai [2 ] are implied by our Assumptions 4 ( i ) 4(i) 4 ( i ) -4 ( i i ) 4(ii) 4 ( ii ) ; 5. Assumption C.5 in Bai [2 ] is not necessary since, from closer inspection of the original proofs, we see that we just need it to hold with power 2 and not 4 and this follows from our Assumption 4 ( v ) 4(v) 4 ( v ) ; 6. Assumption D in Bai [2 ] is implied by our Assumption 5 ( i ) 5(i) 5 ( i ) ; 7. Assumptions E.1-E2 in Bai [2 ] are implied by our Assumptions 4 ( i ) 4(i) 4 ( i ) -4 ( i i ) 4(ii) 4 ( ii ) ; 8. Assumption F.1, F.2, and F.4 in Bai [2 ] are equivalent to our Assumptions 5 ( i i ) 5(ii) 5 ( ii ) -5 ( i v ) 5(iv) 5 ( i v ) ; 9. Assumption F.3 in Bai [2 ] is equivalent to our Assumption 4 ( i i i ) 4(iii) 4 ( iii ) ; 10. Assumption G in Bai [2 ] is equivalent to our Assumption 4 ( v ) 4(v) 4 ( v ) .
Consider the T × N T \times N T × N data matrix X = ( x 1 ⋯ x T ) T \mathbf{X}=(\mathbf{x}_1 \cdots\mathbf{x}_T)^\mathsf{T} X = ( x 1 ⋯ x T ) T so that in matrix notation the factor model (15) is
X = F Λ 0 T + E , (34) \mathbf{X} = \mathbf{F}\Lambda_0^\mathsf{T} + \mathbf{E}, \tag*{(34)} X = F Λ 0 T + E , (34)
where F = ( F 1 ⋯ F T ) T \mathbf{F}=(F_1\ \cdots\ F_T)^{\mathsf{T}} F = ( F 1 ⋯ F T ) T is T × r T \times r T × r and E = ( e 1 ⋯ e T ) T \mathbf{E}=(e_1\ \cdots\ e_T)^{\mathsf{T}} E = ( e 1 ⋯ e T ) T is T × N T \times N T × N .
By definition the PC estimators are such that they solve the following minimization:
min L ∈ R N × r G ∈ R T × r 1 N T ∥ X − G L T ∥ 2 , s.t. L T L N = I r and G T G T is diagonal , (35) \min_{\substack{L\in\mathbb{R}^{N\times r}\\G\in\mathbb{R}^{T\times r}}}\frac{1}{NT}\left\lVert\mathbf{X}-G L^{\mathsf{T}}\right\rVert^2,\quad\text{s.t.}\quad\frac{L^{\mathsf{T}}L}{N}=I_r\ \text{and}\ \frac{G^{\mathsf{T}}G}{T}\ \text{is diagonal}, \tag*{(35)} L ∈ R N × r G ∈ R T × r min N T 1 X − G L T 2 , s.t. N L T L = I r and T G T G is diagonal , (35)
where the constraints are imposed in order to identify the minimum. The solution is obtained by noticing that for given loadings the factors are just the linear projection of the loadings onto the data. Hence, the problem in (35) can be solved by first solving
Λ ^ = arg min L ∈ R N × r 1 N T ∥ X − X L ( L T L ) − 1 L T ∥ 2 , s.t. L T L N = I r , \widehat{\Lambda}=\underset{L\in\mathbb{R}^{N\times r}}{\arg\min}\frac{1}{NT}\left\lVert\mathbf{X}-\mathbf{X}L\left(L^{\mathsf{T}}L\right)^{-1}L^{\mathsf{T}}\right\rVert^2,\quad\text{s.t.}\quad\frac{L^{\mathsf{T}}L}{N}=I_r, Λ = L ∈ R N × r arg min N T 1 X − X L ( L T L ) − 1 L T 2 , s.t. N L T L = I r ,
which is solved by the estimated loadings given in (12) , i.e., Λ ^ = N P ^ \widehat{\Lambda}=\sqrt{N}\widehat{P} Λ = N P , where P ^ \widehat{P} P are the eigenvectors of T − 1 X T X T^{-1}\mathbf{X}^{\mathsf{T}}\mathbf{X} T − 1 X T X corresponding to its r r r largest eigenvalues. The estimated factors are obtained by linear projection as in (13) , i.e., F ^ = X Λ ^ ( Λ ^ T Λ ^ ) − 1 = N − 1 / 2 X P ^ \widehat{\mathbf{F}}=\mathbf{X}\widehat{\Lambda}\left(\widehat{\Lambda}^{\mathsf{T}}\widehat{\Lambda}\right)^{-1}=N^{-1/2}\mathbf{X}\widehat{P} F = X Λ ( Λ T Λ ) − 1 = N − 1/2 X P . Note that, as required, Λ ^ T Λ ^ = N I r \widehat{\Lambda}^{\mathsf{T}}\widehat{\Lambda}=NI_r Λ T Λ = N I r and T − 1 F ^ T F ^ = N − 1 M ^ T^{-1}\widehat{\mathbf{F}}^{\mathsf{T}}\widehat{\mathbf{F}}=N^{-1}\widehat{M} T − 1 F T F = N − 1 M , where M ^ \widehat{M} M is the r × r r \times r r × r diagonal matrix of the largest r r r eigenvalues of T − 1 X T X T^{-1}\mathbf{X}^{\mathsf{T}}\mathbf{X} T − 1 X T X .
An equivalent solution of (35) same solution is however obtained also by noticing that for given factors the loadings are just the linear projection of the factors onto the data. Hence, the problem in (35) can be also solved by first solving
F ^ = arg min G ∈ R T × r 1 N T ∥ X − G ( G T G ) − 1 G T X ∥ 2 , s.t. G T G T is diagonal , \widehat{\mathbf{F}}=\underset{G\in\mathbb{R}^{T\times r}}{\arg\min}\frac{1}{NT}\left\lVert\mathbf{X}-G\left(G^{\mathsf{T}}G\right)^{-1}G^{\mathsf{T}}\mathbf{X}\right\rVert^2,\quad\text{s.t.}\quad\frac{G^{\mathsf{T}}G}{T}\ \text{is diagonal}, F = G ∈ R T × r arg min N T 1 X − G ( G T G ) − 1 G T X 2 , s.t. T G T G is diagonal ,
which gives
F ^ = P ^ M ^ 1 / 2 , (36) \widehat{\mathbf{F}}=\widehat{P}\widehat{M}^{1/2}, \tag*{(36)} F = P M 1/2 , (36)
where P ^ \widehat{P} P are the eigenvectors of N − 1 X X T N^{-1}\mathbf{X}\mathbf{X}^{\mathsf{T}} N − 1 X X T corresponding to its r r r largest eigenvalues which are collected in the r × r r\times r r × r diagonal matrix M ^ \widehat{M} M . Notice that, since ( N T ) − 1 X X T (NT)^{-1}\mathbf{X}\mathbf{X}^{\mathsf{T}} ( N T ) − 1 X X T and ( N T ) − 1 X T X (NT)^{-1}\mathbf{X}^{\mathsf{T}}\mathbf{X} ( N T ) − 1 X T X have the same eigenvalues, then
F ^ T F ^ T = M ^ 1 / 2 P ^ T P ^ M ^ 1 / 2 T = M ^ T = M ~ N , (37) \frac{\widehat{\mathbf{F}}^{\mathsf{T}}\widehat{\mathbf{F}}}{T}=\frac{\widehat{M}^{1/2}\widehat{P}^{\mathsf{T}}\widehat{P}\widehat{M}^{1/2}}{T}=\frac{\widehat{M}}{T}=\frac{\widetilde{M}}{N}, \tag*{(37)} T F T F = T M 1/2 P T P M 1/2 = T M = N M , (37)
which is diagonal as required. The estimated loadings are then obtained by linear projection, i.e.,
Λ ^ = X T F ^ ( F ^ T F ^ ) − 1 = X T F ^ T ( M ^ N ) − 1 , (38) \widehat{\Lambda}=\mathbf{X}^{\mathsf{T}}\widehat{\mathbf{F}}\left(\widehat{\mathbf{F}}^{\mathsf{T}}\widehat{\mathbf{F}}\right)^{-1}=\frac{\mathbf{X}^{\mathsf{T}}\widehat{\mathbf{F}}}{T}\left(\frac{\widehat{M}}{N}\right)^{-1}, \tag*{(38)} Λ = X T F ( F T F ) − 1 = T X T F ( N M ) − 1 , (38)
where we used the last equality in (37) . It follows, that, as requested
A ^ T A ^ N = ( M ^ N ) − 1 F ^ T X X T F ^ N T 2 ( M ^ N ) − 1 = ( M ~ T ) − 1 M ~ 1 / 2 P ~ T X X T P ~ M ~ 1 / 2 N T 2 ( M ~ T ) − 1 = ( M ~ T ) − 1 M ~ 1 / 2 P ~ T P ~ M ~ M ~ 1 / 2 T 2 ( M ~ T ) − 1 = I N . \begin{aligned}
\frac{\widehat{A}^{\mathsf{T}}\widehat{A}}{N}=\left(\frac{\widehat{M}}{N}\right)^{-1}\frac{\widehat{F}^{\mathsf{T}}XX^{\mathsf{T}}\widehat{F}}{NT^{2}}\left(\frac{\widehat{M}}{N}\right)^{-1}=\left(\frac{\widetilde{M}}{T}\right)^{-1}\frac{\widetilde{M}^{1/2}\widetilde{P}^{\mathsf{T}}XX^{\mathsf{T}}\widetilde{P}\widetilde{M}^{1/2}}{NT^{2}}\left(\frac{\widetilde{M}}{T}\right)^{-1} \\
&=\left(\frac{\widetilde{M}}{T}\right)^{-1}\frac{\widetilde{M}^{1/2}\widetilde{P}^{\mathsf{T}}\widetilde{P}\widetilde{M}\widetilde{M}^{1/2}}{T^{2}}\left(\frac{\widetilde{M}}{T}\right)^{-1}=I_N.
\end{aligned} N A T A = ( N M ) − 1 N T 2 F T X X T F ( N M ) − 1 = ( T M ) − 1 N T 2 M 1/2 P T X X T P M 1/2 ( T M ) − 1 = ( T M ) − 1 T 2 M 1/2 P T P M M 1/2 ( T M ) − 1 = I N .
Given the above, hereafter, we can use the formulations of the PC estimators in (36) and (38) , instead of (13) and (12) , to study their asymptotic properties.
(a) Starting from the factors, as formulated in (36) , by definition of eigenvectors we have
X X T N T F ^ = F ^ M ~ T . (39) \frac{XX^{\mathsf{T}}}{NT}\widehat{F}=\widehat{F}\frac{\widetilde{M}}{T}. \tag*{(39)} N T X X T F = F T M . (39)
By substituting (34) in (39) , using the last equality in (37) , and rearranging we have
F ^ − F [ Λ 0 Λ 0 T F T F ^ N 1 T ( M ~ N ) − 1 ] = { E Λ 0 F T F ^ N T + F Λ 0 T E T F ^ N T + E E T F ^ N T } ( M ~ N ) − 1 . (40) \widehat{F}-F\left[\frac{\Lambda_0\Lambda_0^{\mathsf{T}}F^{\mathsf{T}}\widehat{F}}{N}\frac{1}{T}\left(\frac{\widetilde{M}}{N}\right)^{-1}\right]=\left\{\frac{E\Lambda_0F^{\mathsf{T}}\widehat{F}}{NT}+\frac{F\Lambda_0^{\mathsf{T}}E^{\mathsf{T}}\widehat{F}}{NT}+\frac{EE^{\mathsf{T}}\widehat{F}}{NT}\right\}\left(\frac{\widetilde{M}}{N}\right)^{-1}. \tag*{(40)} F − F N Λ 0 Λ 0 T F T F T 1 ( N M ) − 1 = { N T E Λ 0 F T F + N T F Λ 0 T E T F + N T E E T F } ( N M ) − 1 . (40)
By letting
H ^ = ( M ~ N ) − 1 F ^ T F T Λ 0 T Λ 0 N , (41) \widehat{H}=\left(\frac{\widetilde{M}}{N}\right)^{-1}\frac{\widehat{F}^{\mathsf{T}}F}{T}\frac{\Lambda_0^{\mathsf{T}}\Lambda_0}{N}, \tag*{(41)} H = ( N M ) − 1 T F T F N Λ 0 T Λ 0 , (41)
and by taking the t t t -th row of (40) , we have
F ^ t − H ^ F t = ( M ~ N ) − 1 { 1 N T ∑ s = 1 T F ^ s F s T Λ 0 T e t ⏟ A + 1 N T ∑ s = 1 T F ^ s F t T Λ 0 T e s ⏟ B \widehat{F}_t-\widehat{H}F_t=\left(\frac{\widetilde{M}}{N}\right)^{-1}\left\{\underbrace{\frac{1}{NT}\sum_{s=1}^{T}\widehat{F}_sF_s^{\mathsf{T}}\Lambda_0^{\mathsf{T}}e_t}_{A}+\underbrace{\frac{1}{NT}\sum_{s=1}^{T}\widehat{F}_sF_t^{\mathsf{T}}\Lambda_0^{\mathsf{T}}e_s}_{B}
\right. F t − H F t = ( N M ) − 1 ⎩ ⎨ ⎧ A N T 1 s = 1 ∑ T F s F s T Λ 0 T e t + B N T 1 s = 1 ∑ T F s F t T Λ 0 T e s
+ 1 N T ∑ s = 1 T F ^ s E [ e s e t ] ⏟ C + 1 N T ∑ s = 1 T F ^ s ( e s T e t − E [ e s T e t ] ) ⏟ D } . (42) \left.
+\underbrace{\frac{1}{NT}\sum_{s=1}^{T}\widehat{F}_s\mathbb{E}[e_s e_t]}_{C}+\underbrace{\frac{1}{NT}\sum_{s=1}^{T}\widehat{F}_s\left(e_s^{\mathsf{T}}e_t-\mathbb{E}[e_s^{\mathsf{T}}e_t]\right)}_{D}\right\}. \tag*{(42)} + C N T 1 s = 1 ∑ T F s E [ e s e t ] + D N T 1 s = 1 ∑ T F s ( e s T e t − E [ e s T e t ] ) ⎭ ⎬ ⎫ . (42)
which is the same expression as in Bai [2 ] . Since all assumptions in Bai [2 ] hold also in this paper, then we can apply Lemma A.2 therein to (42) and obtain
∥ A ∥ = O P ( 1 N ) , ∥ B ∥ = O P ( 1 N min ( N , T ) ) . (43) \lVert A\rVert=O_{\mathrm{P}}\left(\frac{1}{\sqrt{N}}\right),\qquad\lVert B\rVert=O_{\mathrm{P}}\left(\frac{1}{\sqrt{N}\min\left(\sqrt{N},\sqrt{T}\right)}\right). \tag*{(43)} ∥ A ∥ = O P ( N 1 ) , ∥ B ∥ = O P N min ( N , T ) 1 . (43)
∥ C ∥ = O P ( 1 T min ( N , T ) ) , ∥ D ∥ = O P ( 1 N min ( N , T ) ) , \lVert C\rVert= O_{\mathrm{P}}\left(\frac{1}{\sqrt{T}\min\left(\sqrt{N},\sqrt{T}\right)}\right), \qquad\lVert D\rVert= O_{\mathrm{P}}\left(\frac{1}{\sqrt{N}\min\left(\sqrt{N},\sqrt{T}\right)}\right), ∥ C ∥ = O P T min ( N , T ) 1 , ∥ D ∥ = O P N min ( N , T ) 1 ,
Moreover, from Lemma A.3 in Bai [2 ] ∥ ( N − 1 M ^ ) − 1 ∥ = O P ( 1 ) \lVert(N^{-1}\widehat{M})^{-1}\rVert= O_{\mathrm{P}}(1) ∥( N − 1 M ) − 1 ∥ = O P ( 1 ) . By using (43) into (42) we prove part (a).
(b) Now, consider the following spectral decomposition: Σ Λ 1 / 2 Γ F Σ Λ 1 / 2 = Υ V Υ ⊤ \Sigma_{\Lambda}^{1/2}\Gamma_{F}\Sigma_{\Lambda}^{1/2} = \Upsilon V\Upsilon^{\top} Σ Λ 1/2 Γ F Σ Λ 1/2 = Υ V Υ ⊤ , where V V V is the N × N N \times N N × N diagonal matrix of eigenvalues and the columns of Υ \Upsilon Υ are the corresponding eigenvectors. Denote as V N V_{N} V N the eigenvalues of N − 1 Λ Γ F Λ ′ N^{-1}\Lambda\Gamma_{F}\Lambda^{\prime} N − 1 Λ Γ F Λ ′ . Then, because of Assumption 3 and Weyl’s inequality (Merikoski and Kumar, [43 ] )
∥ V N − V ∥ ≤ ∥ Λ 0 ⊤ Λ 0 N Γ F − Σ Λ Γ F ∥ = O P ( 1 N ) , (44) \lVert V_{N}-V\rVert\le\left\lVert\frac{\Lambda_{0}^{\top}\Lambda_{0}}{N}\Gamma_{F}-\Sigma_{\Lambda}\Gamma_{F}\right\rVert= O_{\mathrm{P}}\left(\frac{1}{\sqrt{N}}\right), \tag*{(44)} ∥ V N − V ∥ ≤ N Λ 0 ⊤ Λ 0 Γ F − Σ Λ Γ F = O P ( N 1 ) , (44)
since V V V are also eigenvalues of Σ Λ Γ F \Sigma_{\Lambda}\Gamma_{F} Σ Λ Γ F . Moreover, since (see e.g. Birkhoff, [8 ] ),
∥ F ⊤ F T − Γ F ∥ = O P ( 1 T ) , (45) \left\lVert\frac{F^{\top}F}{T}-\Gamma_{F}\right\rVert= O_{\mathrm{P}}\left(\frac{1}{\sqrt{T}}\right), \tag*{(45)} T F ⊤ F − Γ F = O P ( T 1 ) , (45)
then, for all N ∈ N N \in\mathbb{N} N ∈ N , we have
∥ M ^ N − V N ∥ ≤ ∥ Λ 0 ⊤ Λ 0 N F ⊤ F T − Λ 0 ⊤ Λ 0 N Γ F ∥ = O P ( 1 T ) , (46) \left\lVert\frac{\widehat{M}}{N}-V_{N}\right\rVert\le\left\lVert\frac{\Lambda_{0}^{\top}\Lambda_{0}}{N}\frac{F^{\top}F}{T}-\frac{\Lambda_{0}^{\top}\Lambda_{0}}{N}\Gamma_{F}\right\rVert= O_{\mathrm{P}}\left(\frac{1}{\sqrt{T}}\right), \tag*{(46)} N M − V N ≤ N Λ 0 ⊤ Λ 0 T F ⊤ F − N Λ 0 ⊤ Λ 0 Γ F = O P ( T 1 ) , (46)
where we used again Weyl’s inequality. Then, from (44) and (46) ,
∥ M ^ N − V ∥ = O P ( max ( 1 N , 1 T ) ) . (47) \left\lVert\frac{\widehat{M}}{N}-V\right\rVert= O_{\mathrm{P}}\left(\max\left(\frac{1}{\sqrt{N}},\frac{1}{\sqrt{T}}\right)\right). \tag*{(47)} N M − V = O P ( max ( N 1 , T 1 ) ) . (47)
Denote as Υ N T \Upsilon_{NT} Υ N T the matrix of normalized eigenvectors of ( N − 1 Λ 0 ⊤ Λ 0 ) 1 / 2 ( T − 1 F ⊤ F ) ( N − 1 Λ 0 ⊤ Λ 0 ) 1 / 2 (N^{-1}\Lambda_{0}^{\top}\Lambda_{0})^{1/2}(T^{-1}F^{\top}F)(N^{-1}\Lambda_{0}^{\top}\Lambda_{0})^{1/2} ( N − 1 Λ 0 ⊤ Λ 0 ) 1/2 ( T − 1 F ⊤ F ) ( N − 1 Λ 0 ⊤ Λ 0 ) 1/2 , then by Davis Kahan Theorem (Yu et al., 2015, Corollary 1), Assumption 3, and (45) ,
∥ Υ N T − Υ J N T ∥ ≤ C ∥ ( Λ 0 ⊤ Λ 0 N ) 1 / 2 ( F ⊤ F T ) ( Λ 0 ⊤ Λ 0 N ) 1 / 2 − Σ Λ 1 / 2 Γ F Σ Λ 1 / 2 ∥ (48) \lVert\Upsilon_{NT}-\Upsilon J_{NT}\rVert\le C\left\lVert\left(\frac{\Lambda_{0}^{\top}\Lambda_{0}}{N}\right)^{1/2}\left(\frac{F^{\top}F}{T}\right)\left(\frac{\Lambda_{0}^{\top}\Lambda_{0}}{N}\right)^{1/2}-\Sigma_{\Lambda}^{1/2}\Gamma_{F}\Sigma_{\Lambda}^{1/2}\right\rVert \tag*{(48)} ∥ Υ N T − Υ J N T ∥ ≤ C ( N Λ 0 ⊤ Λ 0 ) 1/2 ( T F ⊤ F ) ( N Λ 0 ⊤ Λ 0 ) 1/2 − Σ Λ 1/2 Γ F Σ Λ 1/2 (48)
for some finite constant C > 0 C>0 C > 0 independent of N N N and T T T and where J N T J_{NT} J N T is a diagonal matrix with entries ± 1 \pm1 ± 1 which depends on N N N and T T T accounting for the possible sign discordance between sample and population eigenvectors.
Now, from Proposition 1 in Bai [2 ] and using (37) , we have
∥ ( M ^ N ) − 1 / 2 F ^ ⊤ F T − V 1 / 2 J Γ F Σ Λ − 1 / 2 ∥ = ∥ ( M ^ T ) − 1 / 2 F ^ ⊤ F T − V 1 / 2 J Γ F Σ Λ − 1 / 2 ∥ = o P ( 1 ) , (49) \left\lVert\left(\frac{\widehat{M}}{N}\right)^{-1/2}\frac{\widehat{F}^{\top}F}{T}-V^{1/2}J\Gamma_{F}\Sigma_{\Lambda}^{-1/2}\right\rVert= \left\lVert\left(\frac{\widehat{M}}{T}\right)^{-1/2}\frac{\widehat{F}^{\top}F}{T}-V^{1/2}J\Gamma_{F}\Sigma_{\Lambda}^{-1/2}\right\rVert= o_{\mathrm{P}}(1), \tag*{(49)} ( N M ) − 1/2 T F ⊤ F − V 1/2 J Γ F Σ Λ − 1/2 = ( T M ) − 1/2 T F ⊤ F − V 1/2 J Γ F Σ Λ − 1/2 = o P ( 1 ) , (49)
where J = P - lim N , T → ∞ J N T \mathcal{J}=\mathbb{P}\text{-}\lim_{N,T\to\infty}\mathbf{J}_{NT} J = P - lim N , T → ∞ J N T is a diagonal matrix with entries ± 1 \pm1 ± 1 independent of N N N and T T T , which in the proof of Bai [2 ] is set to J = I r \mathcal{J}=I_r J = I r for convention. Inspection of the proof of Proposition 1 in Bai [2 ] shows that the leading rate in (49) is determined by (47) and (48) . Hence, from (47) and (49) ,
∥ F ^ T F T − V J T T Σ Λ − 1 / 2 ∥ = O P ( max ( 1 N , 1 T ) ) . (50) \left\|\frac{\widehat{\mathbf{F}}^{\mathsf{T}}\mathbf{F}}{T}-\mathbf{V}\mathcal{J}\mathcal{T}^{\mathsf{T}}\Sigma_{\Lambda}^{-1/2}\right\|=O_{\mathbb{P}}\left(\max\left(\frac{1}{\sqrt{N}},\frac{1}{\sqrt{T}}\right)\right). \tag*{(50)} T F T F − V J T T Σ Λ − 1/2 = O P ( max ( N 1 , T 1 ) ) . (50)
Let H 0 = J T T Σ Λ 1 / 2 \mathbf{H}_0=\mathcal{J}\mathcal{T}^{\mathsf{T}}\Sigma_{\Lambda}^{1/2} H 0 = J T T Σ Λ 1/2 and notice that it is finite and positive definite because of Assumptions 2(i) and 3, which imply that Σ Λ 1 / 2 Γ F Σ Λ 1 / 2 \Sigma_{\Lambda}^{1/2}\Gamma_{\mathbf{F}}\Sigma_{\Lambda}^{1/2} Σ Λ 1/2 Γ F Σ Λ 1/2 has finite and positive eigenvalues. Then, by using Assumption 3, (50) , and (47) , into (41) ,
∥ H ^ − H 0 ∥ = ∥ H ^ − J T T Σ Λ 1 / 2 ∥ = O P ( max ( 1 N , 1 T ) ) . (51) \left\|\widehat{\mathbf{H}}-\mathbf{H}_0\right\|=\left\|\widehat{\mathbf{H}}-\mathcal{J}\mathcal{T}^{\mathsf{T}}\Sigma_{\Lambda}^{1/2}\right\|=O_{\mathbb{P}}\left(\max\left(\frac{1}{\sqrt{N}},\frac{1}{\sqrt{T}}\right)\right). \tag*{(51)} H − H 0 = H − J T T Σ Λ 1/2 = O P ( max ( N 1 , T 1 ) ) . (51)
Therefore, from (42) and (43) , for any t = 1 , … , T t=1,\ldots,T t = 1 , … , T , if N / T → 0 \sqrt{N}/T\to0 N / T → 0 , as N , T → ∞ N,T\to\infty N , T → ∞ ,
N ( F ^ t − H ^ F t ) = ( M ^ N ) − 1 ( F ^ T F T ) Λ 0 T e t N + o P ( 1 ) = H ^ ( Λ 0 T Λ 0 N ) − 1 ( 1 N ∑ i = 1 N λ 0 i e i t ) + o P ( 1 ) = H 0 Σ Λ − 1 ( 1 N ∑ i = 1 N λ 0 i e i t ) + o P ( 1 ) ⇒ N ( 0 N , H 0 Σ Λ − 1 Π t , t Σ Λ − 1 H 0 T ) , \begin{aligned}
\sqrt{N}\left(\widehat{\mathbf{F}}_t-\widehat{\mathbf{H}}\mathbf{F}_t\right)=\left(\frac{\widehat{\mathbf{M}}}{N}\right)^{-1}\left(\frac{\widehat{\mathbf{F}}^{\mathsf{T}}\mathbf{F}}{T}\right)\frac{\Lambda_0^{\mathsf{T}}\mathbf{e}_t}{\sqrt{N}}+o_{\mathbb{P}}(1)=\widehat{\mathbf{H}}\left(\frac{\Lambda_0^{\mathsf{T}}\Lambda_0}{N}\right)^{-1}\left(\frac{1}{\sqrt{N}}\sum_{i=1}^{N}\lambda_{0i}\mathbf{e}_{it}\right)+o_{\mathbb{P}}(1) \\
&=\mathbf{H}_0\Sigma_{\Lambda}^{-1}\left(\frac{1}{\sqrt{N}}\sum_{i=1}^{N}\lambda_{0i}\mathbf{e}_{it}\right)+o_{\mathbb{P}}(1)\Rightarrow\mathcal{N}\left(\mathbf{0}_N,\mathbf{H}_0\Sigma_{\Lambda}^{-1}\boldsymbol{\Pi}_{t,t}\Sigma_{\Lambda}^{-1}\mathbf{H}_0^{\mathsf{T}}\right),
\end{aligned} N ( F t − H F t ) = ( N M ) − 1 ( T F T F ) N Λ 0 T e t + o P ( 1 ) = H ( N Λ 0 T Λ 0 ) − 1 ( N 1 i = 1 ∑ N λ 0 i e i t ) + o P ( 1 ) = H 0 Σ Λ − 1 ( N 1 i = 1 ∑ N λ 0 i e i t ) + o P ( 1 ) ⇒ N ( 0 N , H 0 Σ Λ − 1 Π t , t Σ Λ − 1 H 0 T ) ,
where we used also (41) , (51) , Assumptions 3 and 4(iii), and Slutsky’s theorem. This proves part (b).
(c) Consider the estimated loadings, as formulated in (38) . Then, using (34) and (37)
Λ ^ = X T F ^ T ( M ^ N ) − 1 = { Λ 0 F T F ^ T + E T F ^ T } ( M ^ N ) − 1 = Λ 0 H ^ − 1 + { Λ 0 ( F − F ^ H ^ − T ) T F ^ T + E T ( F ^ − F H ^ T ) T + E T F H ^ T T } ( M ^ N ) − 1 , (52) \begin{aligned}
\widehat{\Lambda}=\frac{\mathbf{X}^{\mathsf{T}}\widehat{\mathbf{F}}}{T}\left(\frac{\widehat{\mathbf{M}}}{N}\right)^{-1}=\left\{\frac{\Lambda_0\mathbf{F}^{\mathsf{T}}\widehat{\mathbf{F}}}{T}+\frac{\mathbf{E}^{\mathsf{T}}\widehat{\mathbf{F}}}{T}\right\}\left(\frac{\widehat{\mathbf{M}}}{N}\right)^{-1} \\
&=\Lambda_0\widehat{\mathbf{H}}^{-1}+\left\{\frac{\Lambda_0\left(\mathbf{F}-\widehat{\mathbf{F}}\widehat{\mathbf{H}}^{-\mathsf{T}}\right)^{\mathsf{T}}\widehat{\mathbf{F}}}{T}+\frac{\mathbf{E}^{\mathsf{T}}\left(\widehat{\mathbf{F}}-\mathbf{F}\widehat{\mathbf{H}}^{\mathsf{T}}\right)}{T}+\frac{\mathbf{E}^{\mathsf{T}}\mathbf{F}\widehat{\mathbf{H}}^{\mathsf{T}}}{T}\right\}\left(\frac{\widehat{\mathbf{M}}}{N}\right)^{-1}, \tag*{(52)}
\end{aligned} Λ = T X T F ( N M ) − 1 = { T Λ 0 F T F + T E T F } ( N M ) − 1 = Λ 0 H − 1 + ⎩ ⎨ ⎧ T Λ 0 ( F − F H − T ) T F + T E T ( F − F H T ) + T E T F H T ⎭ ⎬ ⎫ ( N M ) − 1 , (52)
where, from (41) , we have
H ^ − 1 = ( Λ 0 T Λ 0 N ) − 1 ( F ^ T F T ) − 1 M ^ N . (53) \widehat{\mathbf{H}}^{-1}=\left(\frac{\Lambda_0^{\mathsf{T}}\Lambda_0}{N}\right)^{-1}\left(\frac{\widehat{\mathbf{F}}^{\mathsf{T}}\mathbf{F}}{T}\right)^{-1}\frac{\widehat{\mathbf{M}}}{N}. \tag*{(53)} H − 1 = ( N Λ 0 T Λ 0 ) − 1 ( T F T F ) − 1 N M . (53)
By taking the i i i th row of (52) and rearranging, we have
λ ^ i − H ^ − 1 λ 0 i = ( M ^ N ) − 1 { H ^ F T e i T ⏟ E + ( F ^ − F H ^ T ) T e i T ⏟ F + F ^ ( F − F ^ H ^ − 1 T ) λ 0 i T ⏟ G } , (54) \hat{\lambda}_i-\hat{H}^{-1}\lambda_{0i}=\left(\frac{\hat{M}}{N}\right)^{-1}\left\{\underbrace{\frac{\hat{H}F^{\mathsf{T}}e_i}{T}}_{E}+\underbrace{\frac{\left(\hat{F}-F\hat{H}^{\mathsf{T}}\right)^{\mathsf{T}}e_i}{T}}_{F}+\underbrace{\frac{\hat{F}\left(F-\hat{F}\hat{H}^{-1\mathsf{T}}\right)\lambda_{0i}}{T}}_{G}\right\}, \tag*{(54)} λ ^ i − H ^ − 1 λ 0 i = ( N M ^ ) − 1 ⎩ ⎨ ⎧ E T H ^ F T e i + F T ( F ^ − F H ^ T ) T e i + G T F ^ ( F − F ^ H ^ − 1 T ) λ 0 i ⎭ ⎬ ⎫ , (54)
where e i = ( e i 1 ⋯ e i T ) T e_i=(e_{i1}\cdots e_{iT})^{\mathsf{T}} e i = ( e i 1 ⋯ e i T ) T . The expression in (54) coincides with the one in Bai [2 ] , equation just before equation (B.2) but for the first term. By Assumption 5 5 5 (iv)
∥ E ∥ = O P ( 1 T ) , (55) \lVert E\rVert=O_{\mathbb{P}}\left(\frac{1}{\sqrt{T}}\right), \tag*{(55)} ∥ E ∥ = O P ( T 1 ) , (55)
while Lemmas B.1 and B.3 in Bai [2 ] imply that
∥ F ∥ = O P ( 1 min ( N , T ) ) and ∥ G ∥ = O P ( 1 min ( N , T ) ) . (56) \lVert F\rVert=O_{\mathbb{P}}\left(\frac{1}{\min(N,T)}\right)\quad\text{and}\quad\lVert G\rVert=O_{\mathbb{P}}\left(\frac{1}{\min(N,T)}\right). \tag*{(56)} ∥ F ∥ = O P ( min ( N , T ) 1 ) and ∥ G ∥ = O P ( min ( N , T ) 1 ) . (56)
By using (55) and (56) into (54) and since ∥ ( N − 1 M ^ ) − 1 ∥ = O P ( 1 ) \lVert(N^{-1}\hat{M})^{-1}\rVert=O_{\mathbb{P}}(1) ∥( N − 1 M ^ ) − 1 ∥ = O P ( 1 ) by Bai [2 ] , Lemma A.3, we prove part (c).
(d) From (54) , (55) , and (56) , for any i = 1 , … , N i=1,\ldots,N i = 1 , … , N , if T / N → 0 \sqrt{T}/N\to0 T / N → 0 , as N , T → ∞ N,T\to\infty N , T → ∞ ,
T ( λ ^ i − H ^ − 1 T λ 0 i ) = ( M ^ N ) − 1 H ^ F T e i T + o P ( 1 ) = ( F ^ T F ^ T ) − 1 H ^ ( 1 T ∑ t = 1 T F t e i t ) + o P ( 1 ) = ( H ^ F T F H ^ T T ) − 1 H ^ ( 1 T ∑ t = 1 T F t e i t ) + o P ( 1 ) = H ^ − 1 T ( F T F T ) − 1 ( 1 T ∑ t = 1 T F t e i t ) + o P ( 1 ) = H 0 − 1 T Γ F − 1 ( 1 T ∑ t = 1 T F t e i t ) + o P ( 1 ) ⇒ N ( 0 N , H 0 − 1 T Γ F − 1 Ξ i Γ F − 1 H 0 − 1 ) , (57) \begin{aligned}
\sqrt{T}\left(\hat{\lambda}_i-\hat{H}^{-1\mathsf{T}}\lambda_{0i}\right)
&=\left(\frac{\hat{M}}{N}\right)^{-1}\frac{\hat{H}F^{\mathsf{T}}e_i}{\sqrt{T}}+o_{\mathbb{P}}(1)
=\left(\frac{\hat{F}^{\mathsf{T}}\hat{F}}{T}\right)^{-1}\hat{H}\left(\frac{1}{\sqrt{T}}\sum_{t=1}^{T}F_t e_{it}\right)+o_{\mathbb{P}}(1)\\
&=\left(\frac{\hat{H}F^{\mathsf{T}}F\hat{H}^{\mathsf{T}}}{T}\right)^{-1}\hat{H}\left(\frac{1}{\sqrt{T}}\sum_{t=1}^{T}F_t e_{it}\right)+o_{\mathbb{P}}(1)\\
&=\hat{H}^{-1\mathsf{T}}\left(\frac{F^{\mathsf{T}}F}{T}\right)^{-1}\left(\frac{1}{\sqrt{T}}\sum_{t=1}^{T}F_t e_{it}\right)+o_{\mathbb{P}}(1)\\
&=H_0^{-1\mathsf{T}}\Gamma_F^{-1}\left(\frac{1}{\sqrt{T}}\sum_{t=1}^{T}F_t e_{it}\right)+o_{\mathbb{P}}(1)
\Rightarrow\mathcal{N}\left(0_N,H_0^{-1\mathsf{T}}\Gamma_F^{-1}\Xi_i\Gamma_F^{-1}H_0^{-1}\right), \tag*{(57)}
\end{aligned} T ( λ ^ i − H ^ − 1 T λ 0 i ) = ( N M ^ ) − 1 T H ^ F T e i + o P ( 1 ) = ( T F ^ T F ^ ) − 1 H ^ ( T 1 t = 1 ∑ T F t e i t ) + o P ( 1 ) = ( T H ^ F T F H ^ T ) − 1 H ^ ( T 1 t = 1 ∑ T F t e i t ) + o P ( 1 ) = H ^ − 1 T ( T F T F ) − 1 ( T 1 t = 1 ∑ T F t e i t ) + o P ( 1 ) = H 0 − 1 T Γ F − 1 ( T 1 t = 1 ∑ T F t e i t ) + o P ( 1 ) ⇒ N ( 0 N , H 0 − 1 T Γ F − 1 Ξ i Γ F − 1 H 0 − 1 ) , (57)
where we used also (37) and part (a) which, together with Lemma B.3 in Bai [2 ] , implies ∥ T − 1 F ^ T F ^ − T − 1 H ^ F T F H ^ T ∥ = o P ( 1 ) \lVert T^{-1}\hat{F}^{\mathsf{T}}\hat{F}-T^{-1}\hat{H}F^{\mathsf{T}}F\hat{H}^{\mathsf{T}}\rVert=o_{\mathbb{P}}(1) ∥ T − 1 F ^ T F ^ − T − 1 H ^ F T F H ^ T ∥ = o P ( 1 ) , (51) together with the continuous mapping theorem, Assumptions 2 2 2 (i) and 5 5 5 (iv), and Slutsky’s theorem. This proves part (d).
(e) From parts (a) and (c) combined with (51) we prove part (e) and complete the proof. ▪
Proof of Proposition 2
(a) To prove the first claim, let us embed the nonstationary filtered sequence { ψ ^ t ∣ t − 1 ( θ ) } t ∈ N \left\{\widehat{\psi}_{t|t-1}(\theta)\right\}_{t\in\mathbb{N}} { ψ t ∣ t − 1 ( θ ) } t ∈ N in the following dynamical system
ψ ^ t + 1 ∣ t ( θ ) = g t ( ψ ^ t ∣ t − 1 ( θ ) , F ^ t , θ ) , (58) \widehat{\psi}_{t+1|t}(\theta)=g_t\left(\widehat{\psi}_{t|t-1}(\theta),\widehat{F}_t,\theta\right), \tag*{(58)} ψ t + 1∣ t ( θ ) = g t ( ψ t ∣ t − 1 ( θ ) , F t , θ ) , (58)
where by using (3) , the stochastic function g t : P × R r × Θ → P g_t:\mathcal{P}\times\mathbb{R}^r\times\Theta\to\mathcal{P} g t : P × R r × Θ → P is given by
g t ( ψ ^ t ∣ t − 1 ( θ ) , F ^ t , θ ) = δ + Φ ψ ^ t ∣ t − 1 ( θ ) + K u ^ t ( θ ) , (59) g_t\left(\widehat{\psi}_{t|t-1}(\theta),\widehat{F}_t,\theta\right)=\delta+\Phi\widehat{\psi}_{t|t-1}(\theta)+K\widehat{u}_t(\theta), \tag*{(59)} g t ( ψ t ∣ t − 1 ( θ ) , F t , θ ) = δ + Φ ψ t ∣ t − 1 ( θ ) + K u t ( θ ) , (59)
where u ^ t ( θ ) \widehat{u}_t(\theta) u t ( θ ) is the scaled-score function evaluated at ψ ^ t ∣ t − 1 ( θ ) \widehat{\psi}_{t|t-1}(\theta) ψ t ∣ t − 1 ( θ ) which is, in fact, a nonlinear function of F ^ t \widehat{F}_t F t . Hence, for any fixed and non-stochastic starting point ψ 1 ∣ 0 \psi_{1|0} ψ 1∣0 , define the functions g t ( k ) g_t^{(k)} g t ( k ) , representing the k k k -iterations of the dynamic system, by the following recursions
g t ( k + 1 ) ( ψ 1 ∣ 0 , F ^ 1 , … , F ^ k + 1 , θ ) = g t + k ( g t ( k ) ( ψ 1 ∣ 0 , F ^ 1 , … , F ^ k , θ ) , F ^ k + 1 , θ ) , g_t^{(k+1)}\left(\psi_{1|0},\widehat{F}_1,\ldots,\widehat{F}_{k+1},\theta\right)=g_{t+k}\left(g_t^{(k)}\left(\psi_{1|0},\widehat{F}_1,\ldots,\widehat{F}_k,\theta\right),\widehat{F}_{k+1},\theta\right), g t ( k + 1 ) ( ψ 1∣0 , F 1 , … , F k + 1 , θ ) = g t + k ( g t ( k ) ( ψ 1∣0 , F 1 , … , F k , θ ) , F k + 1 , θ ) ,
such that g t = g t ( 1 ) g_t=g_t^{(1)} g t = g t ( 1 ) . Note that, substituting F ^ t \widehat{F}_t F t with H ^ F t \widehat{H}F_t H F t in equations (58) and (59) , one obtains the analogous dynamical system for the sequence { ψ ^ t ∣ t − 1 ( θ ) } t ∈ N \left\{\widehat{\psi}_{t|t-1}(\theta)\right\}_{t\in\mathbb{N}} { ψ t ∣ t − 1 ( θ ) } t ∈ N . Thus, by repeated applications of the mean value theorem, Hölder’s and Minkowski’s inequalities, we can show that
∥ g t ( k ) ( ψ 1 ∣ 0 , F ^ 1 , … , F ^ k , θ ) − g t ( k ) ( ψ 1 ∣ 0 ∗ , H ^ F 1 , … , H ^ F k , θ ) ∥ ≤ sup ψ ∈ P ∥ ∏ j = 1 k X k − j + 1 ( θ ) ∥ ∥ ψ 1 ∣ 0 − ψ 1 ∣ 0 ∗ ∥ + ∣ ∑ j = 1 k sup ψ ∈ P ∥ Y k − j + 1 ( θ ) ∥ ∥ F ^ k − j + 1 − H ^ F k − j + 1 ∥ ∣ . \left\|g_t^{(k)}\left(\psi_{1|0},\widehat{F}_1,\ldots,\widehat{F}_k,\theta\right)-g_t^{(k)}\left(\psi^*_{1|0},\widehat{H}F_1,\ldots,\widehat{H}F_k,\theta\right)\right\|
\le\sup_{\psi\in\mathcal{P}}\left\|\prod_{j=1}^{k}X_{k-j+1}(\theta)\right\|\left\|\psi_{1|0}-\psi^*_{1|0}\right\|+\left|\sum_{j=1}^{k}\sup_{\psi\in\mathcal{P}}\left\|Y_{k-j+1}(\theta)\right\|\left\|\widehat{F}_{k-j+1}-\widehat{H}F_{k-j+1}\right\|\right|. g t ( k ) ( ψ 1∣0 , F 1 , … , F k , θ ) − g t ( k ) ( ψ 1∣0 ∗ , H F 1 , … , H F k , θ ) ≤ ψ ∈ P sup j = 1 ∏ k X k − j + 1 ( θ ) ψ 1∣0 − ψ 1∣0 ∗ + j = 1 ∑ k ψ ∈ P sup ∥ Y k − j + 1 ( θ ) ∥ F k − j + 1 − H F k − j + 1 .
Now, proceeding recursively for t ∈ N t\in\mathbb{N} t ∈ N , and using Assumption 7 together with Proposition 1, we obtain
sup θ ∈ Θ ∥ ψ ^ t + 1 ∣ t ( θ ) − ψ ^ t + 1 ∣ t ( θ ) ∥ ≤ sup θ ∈ Θ ∥ X t , k ( θ ) ∥ sup θ ∈ Θ ∥ ψ ^ t − k + 1 ∣ t − k ( θ ) − ψ ^ t − k + 1 ∣ t − k ( θ ) ∥ + ∣ ∑ j = 1 k sup θ ∈ Θ ∥ Y t − j + 1 ( θ ) ∥ ∥ F ^ t − j + 1 − H ^ F t − j + 1 ∥ ∣ . \sup_{\theta\in\Theta}\left\|\widehat{\psi}_{t+1|t}(\theta)-\widehat{\psi}_{t+1|t}(\theta)\right\|
\le\sup_{\theta\in\Theta}\left\|X_{t,k}(\theta)\right\|\sup_{\theta\in\Theta}\left\|\widehat{\psi}_{t-k+1|t-k}(\theta)-\widehat{\psi}_{t-k+1|t-k}(\theta)\right\|+\left|\sum_{j=1}^{k}\sup_{\theta\in\Theta}\left\|Y_{t-j+1}(\theta)\right\|\left\|\widehat{F}_{t-j+1}-\widehat{H}F_{t-j+1}\right\|\right|. θ ∈ Θ sup ψ t + 1∣ t ( θ ) − ψ t + 1∣ t ( θ ) ≤ θ ∈ Θ sup ∥ X t , k ( θ ) ∥ θ ∈ Θ sup ψ t − k + 1∣ t − k ( θ ) − ψ t − k + 1∣ t − k ( θ ) + j = 1 ∑ k θ ∈ Θ sup ∥ Y t − j + 1 ( θ ) ∥ F t − j + 1 − H F t − j + 1 .
Thus, by recursively unfolding the bound back to t = 1 t=1 t = 1 , and applying Assumption 7, we obtain,
sup θ ∈ Θ ∥ ψ ^ t + 1 ∣ t ( θ ) − ψ ^ t + 1 ∣ t ( θ ) ∥ ≤ X k ⌊ t / k ⌋ ∥ ψ 1 ∣ 0 − ψ 1 ∣ 0 ∗ ∥ + ∑ j = 1 t X k ⌊ ( j − 1 ) / k ⌋ sup θ ∈ Θ ∥ Y t − j + 1 ( θ ) ∥ ∥ F ^ t − j + 1 − H ^ F t − j + 1 ∥ . ≤ X k [ t / k ] Δ 1 + k Δ 2 / ( 1 − X k ) , (60) \begin{aligned}
\sup_{\theta\in\Theta}\left\|\widehat{\psi}_{t+1|t}(\theta)-\widehat{\psi}_{t+1|t}(\theta)\right\|
\le\mathcal{X}_k^{\lfloor t/k\rfloor}\left\|\psi_{1|0}-\psi^*_{1|0}\right\|+\sum_{j=1}^{t}\mathcal{X}_k^{\lfloor(j-1)/k\rfloor}\sup_{\theta\in\Theta}\left\|Y_{t-j+1}(\theta)\right\|\left\|\widehat{F}_{t-j+1}-\widehat{H}F_{t-j+1}\right\|. \\
&\le\mathcal{X}_k^{[t/k]}\Delta_1 + k\Delta_2/(1-\mathcal{X}_k), \tag*{(60)}
\end{aligned} θ ∈ Θ sup ψ t + 1∣ t ( θ ) − ψ t + 1∣ t ( θ ) ≤ X k ⌊ t / k ⌋ ψ 1∣0 − ψ 1∣0 ∗ + j = 1 ∑ t X k ⌊( j − 1 ) / k ⌋ θ ∈ Θ sup ∥ Y t − j + 1 ( θ ) ∥ F t − j + 1 − H F t − j + 1 . ≤ X k [ t / k ] Δ 1 + k Δ 2 / ( 1 − X k ) , (60)
where 0 < Δ 1 < ∞ 0<\Delta_1<\infty 0 < Δ 1 < ∞ is defined as Δ 1 = ∥ ψ 1 ∣ 0 − ψ 1 ⋆ ∣ 0 ∥ \Delta_1=\left\|\psi_1|_0-\psi_1^\star|_0\right\| Δ 1 = ∥ ψ 1 ∣ 0 − ψ 1 ⋆ ∣ 0 ∥ . At each recursive step of (60) , factor-approximation errors inherited from previous iterations are multiplied by an additional contraction coefficient. Consequently, an error introduced j − 1 j-1 j − 1 recursive steps earlier is weighted by X k [ ( j − 1 ) / k ] \mathcal{X}_k^{[(j-1)/k]} X k [( j − 1 ) / k ] for j = 1 , … , t j=1,\ldots,t j = 1 , … , t .
An inspection of (42) and (43) in the proof of Proposition 1(a) shows that, for every fixed s ≥ 1 s\geq1 s ≥ 1 ,
∥ F ^ s − H ^ F s ∥ = O P ( 1 min ( N , T ) ) . \left\|\widehat{F}_s-\widehat{H}F_s\right\|=O_{\mathbb{P}}\left(\frac{1}{\min\left(\sqrt{N},T\right)}\right). F s − H F s = O P min ( N , T ) 1 .
Since t t t is fixed, an application of the union bound shows that
lim M → ∞ lim sup N , T → ∞ P ( min ( N , T ) max 1 ≤ s ≤ t ∥ F ^ s − H ^ F s ∥ > M ) ≤ ∑ s = 1 t lim M → ∞ lim sup N , T → ∞ P ( min ( N , T ) ∥ F ^ s − H ^ F s ∥ > M ) = 0 , \begin{aligned}
\lim_{M\to\infty}\limsup_{N,T\to\infty}\mathbb{P}\left(\min\left(\sqrt{N},T\right)\max_{1\leq s\leq t}\left\|\widehat{F}_s-\widehat{H}F_s\right\|>M\right) \\
&\leq\sum_{s=1}^{t}\lim_{M\to\infty}\limsup_{N,T\to\infty}\mathbb{P}\left(\min\left(\sqrt{N},T\right)\left\|\widehat{F}_s-\widehat{H}F_s\right\|>M\right)=0,
\end{aligned} M → ∞ lim N , T → ∞ lim sup P ( min ( N , T ) 1 ≤ s ≤ t max F s − H F s > M ) ≤ s = 1 ∑ t M → ∞ lim N , T → ∞ lim sup P ( min ( N , T ) F s − H F s > M ) = 0 ,
and, consequently,
max 1 ≤ s ≤ t ∥ F ^ s − H ^ F s ∥ = O P ( 1 min ( N , T ) ) . (61) \max_{1\leq s\leq t}\left\|\widehat{F}_s-\widehat{H}F_s\right\|=O_{\mathbb{P}}\left(\frac{1}{\min\left(\sqrt{N},T\right)}\right). \tag*{(61)} 1 ≤ s ≤ t max F s − H F s = O P min ( N , T ) 1 . (61)
Moreover, Assumption 7(ii) and Markov’s inequality imply that, for every fixed t t t ,
max 1 ≤ s ≤ t sup θ ∈ Θ ∣ Y s ( θ ) ∣ = O P ( 1 ) . \max_{1\leq s\leq t}\sup_{\theta\in\Theta}|Y_s(\theta)|=O_{\mathbb{P}}(1). 1 ≤ s ≤ t max θ ∈ Θ sup ∣ Y s ( θ ) ∣ = O P ( 1 ) .
It therefore follows from (61) that
Δ 2 = max 1 ≤ s ≤ t ( sup θ ∈ Θ ∣ Y s ( θ ) ∣ ∥ F ^ s − H ^ F s ∥ ) ≤ max 1 ≤ s ≤ t sup θ ∈ Θ ∣ Y s ( θ ) ∣ max 1 ≤ s ≤ t ∥ F ^ s − H ^ F s ∥ = O P ( 1 min ( N , T ) ) . (62) \begin{aligned}
\Delta_2=\max_{1\leq s\leq t}\left(\sup_{\theta\in\Theta}|Y_s(\theta)|\left\|\widehat{F}_s-\widehat{H}F_s\right\|\right) \\
&\leq\max_{1\leq s\leq t}\sup_{\theta\in\Theta}|Y_s(\theta)|\max_{1\leq s\leq t}\left\|\widehat{F}_s-\widehat{H}F_s\right\|=O_{\mathbb{P}}\left(\frac{1}{\min\left(\sqrt{N},T\right)}\right). \tag*{(62)}
\end{aligned} Δ 2 = 1 ≤ s ≤ t max ( θ ∈ Θ sup ∣ Y s ( θ ) ∣ F s − H F s ) ≤ 1 ≤ s ≤ t max θ ∈ Θ sup ∣ Y s ( θ ) ∣ 1 ≤ s ≤ t max F s − H F s = O P min ( N , T ) 1 . (62)
Finally, since Δ 1 < ∞ \Delta_1<\infty Δ 1 < ∞ , k k k is fixed, and 0 < X k < 1 0<\mathcal{X}_k<1 0 < X k < 1 , substituting (62) into (60) gives
sup θ ∈ Θ ∥ ψ ^ t + 1 ∣ t ( θ ) − ψ ^ t + 1 ∣ t ( θ ) ∥ = O P ( 1 min ( N , T ) ) + O a . s . ( X k [ t / k ] ) . (63) \sup_{\theta\in\Theta}\left\|\widehat{\psi}_{t+1|t}(\theta)-\widehat{\psi}_{t+1|t}(\theta)\right\|=O_{\mathbb{P}}\left(\frac{1}{\min\left(\sqrt{N},T\right)}\right)+O_{\mathrm{a.s.}}\left(\mathcal{X}_k^{[t/k]}\right). \tag*{(63)} θ ∈ Θ sup ψ t + 1∣ t ( θ ) − ψ t + 1∣ t ( θ ) = O P min ( N , T ) 1 + O a.s. ( X k [ t / k ] ) . (63)
as N , T → ∞ N,T \to\infty N , T → ∞ .
(b) For the second claim, we note that the sequence { ψ ~ t ∣ t − 1 ( θ ) } t ∈ N \left\{\widetilde{\psi}_{t|t-1}(\theta)\right\}_{t\in\mathbb{N}} { ψ t ∣ t − 1 ( θ ) } t ∈ N satisfies (58) and (59) , but replacing F ^ t \widehat{F}_t F t with H 0 F ‾ t H_0\overline{F}_t H 0 F t . Therefore, by the same arguments discussed in point (a), we immediately obtain
sup θ ∈ Θ ∥ g t ( k ) ( ψ 1 ∣ 0 ∗ , H ^ F 1 , … , H ^ F k , θ ) − g t ( k ) ( ψ 1 ∣ 0 ∗ , H 0 F ‾ 1 , … , H 0 F ‾ k , θ ) ∥ ≤ ∥ ∑ j = 1 k sup θ ∈ Θ ∣ Y k − j + 1 ( θ ) ∣ ∥ H ^ F k − j + 1 − H 0 F ‾ k − j + 1 ∥ ∥ . \sup_{\theta\in\Theta}\left\|g_t^{(k)}\left(\psi_{1|0}^*,\widehat{H}F_1,\ldots,\widehat{H}F_k,\theta\right)-g_t^{(k)}\left(\psi_{1|0}^*,H_0\overline{F}_1,\ldots,H_0\overline{F}_k,\theta\right)\right\|
\leq
\left\|\sum_{j=1}^{k}\sup_{\theta\in\Theta}\left|Y_{k-j+1}(\theta)\right|\left\|\widehat{H}F_{k-j+1}-H_0\overline{F}_{k-j+1}\right\|\right\|. θ ∈ Θ sup g t ( k ) ( ψ 1∣0 ∗ , H F 1 , … , H F k , θ ) − g t ( k ) ( ψ 1∣0 ∗ , H 0 F 1 , … , H 0 F k , θ ) ≤ j = 1 ∑ k θ ∈ Θ sup ∣ Y k − j + 1 ( θ ) ∣ H F k − j + 1 − H 0 F k − j + 1 .
Thus, using the decomposition H ^ F t − H 0 F ‾ t = ( H ^ − H 0 ) F t + H 0 ( F t − F ‾ t ) \widehat{H}F_t-H_0\overline{F}_t=(\widehat{H}-H_0)F_t+H_0(F_t-\overline{F}_t) H F t − H 0 F t = ( H − H 0 ) F t + H 0 ( F t − F t ) , (60) becomes
sup θ ∈ Θ ∥ ψ ~ t + 1 ∣ t ( θ ) − ψ ~ t + 1 ∣ t ( θ ) ∥ ≤ ∑ j = 1 t X k ⌊ ( j − 1 ) / k ⌋ sup θ ∈ Θ ∣ Y t − j + 1 ( θ ) ∣ ∥ H ^ F t − j + 1 − H 0 F ‾ t − j + 1 ∥ ≤ ∑ j = 1 t X k ⌊ ( j − 1 ) / k ⌋ sup θ ∈ Θ ∣ Y t − j + 1 ( θ ) ∣ ∥ ( H ^ − H 0 ) F t − j + 1 ∥ + ∑ j = 1 t X k ⌊ ( j − 1 ) / k ⌋ sup θ ∈ Θ ∣ Y t − j + 1 ( θ ) ∣ ∥ H 0 ( F t − j + 1 − F ‾ t − j + 1 ) ∥ . (64) \begin{aligned}
\sup_{\theta\in\Theta}\left\|\widetilde{\psi}_{t+1|t}(\theta)-\widetilde{\psi}_{t+1|t}(\theta)\right\|
&\leq\sum_{j=1}^{t}\mathcal{X}_k^{\lfloor(j-1)/k\rfloor}\sup_{\theta\in\Theta}\left|Y_{t-j+1}(\theta)\right|\left\|\widehat{H}F_{t-j+1}-H_0\overline{F}_{t-j+1}\right\|\\
&\leq\sum_{j=1}^{t}\mathcal{X}_k^{\lfloor(j-1)/k\rfloor}\sup_{\theta\in\Theta}\left|Y_{t-j+1}(\theta)\right|\left\|(\widehat{H}-H_0)F_{t-j+1}\right\|\\
&\quad+\sum_{j=1}^{t}\mathcal{X}_k^{\lfloor(j-1)/k\rfloor}\sup_{\theta\in\Theta}\left|Y_{t-j+1}(\theta)\right|\left\|H_0(F_{t-j+1}-\overline{F}_{t-j+1})\right\|. \tag*{(64)}
\end{aligned} θ ∈ Θ sup ψ t + 1∣ t ( θ ) − ψ t + 1∣ t ( θ ) ≤ j = 1 ∑ t X k ⌊( j − 1 ) / k ⌋ θ ∈ Θ sup ∣ Y t − j + 1 ( θ ) ∣ H F t − j + 1 − H 0 F t − j + 1 ≤ j = 1 ∑ t X k ⌊( j − 1 ) / k ⌋ θ ∈ Θ sup ∣ Y t − j + 1 ( θ ) ∣ ( H − H 0 ) F t − j + 1 + j = 1 ∑ t X k ⌊( j − 1 ) / k ⌋ θ ∈ Θ sup ∣ Y t − j + 1 ( θ ) ∣ H 0 ( F t − j + 1 − F t − j + 1 ) . (64)
For every fixed t t t , the first term on the right-hand side of (64) can be bounded by
∑ j = 1 t X k ⌊ ( j − 1 ) / k ⌋ sup θ ∈ Θ ∣ Y t − j + 1 ( θ ) ∣ ∥ ( H ^ − H 0 ) F t − j + 1 ∥ ≤ Δ 3 ∑ j = 1 t X k ⌊ ( j − 1 ) / k ⌋ ≤ k Δ 3 1 − X k , \sum_{j=1}^{t}\mathcal{X}_k^{\lfloor(j-1)/k\rfloor}\sup_{\theta\in\Theta}\left|Y_{t-j+1}(\theta)\right|\left\|(\widehat{H}-H_0)F_{t-j+1}\right\|
\leq\Delta_3\sum_{j=1}^{t}\mathcal{X}_k^{\lfloor(j-1)/k\rfloor}
\leq\frac{k\Delta_3}{1-\mathcal{X}_k}, j = 1 ∑ t X k ⌊( j − 1 ) / k ⌋ θ ∈ Θ sup ∣ Y t − j + 1 ( θ ) ∣ ( H − H 0 ) F t − j + 1 ≤ Δ 3 j = 1 ∑ t X k ⌊( j − 1 ) / k ⌋ ≤ 1 − X k k Δ 3 ,
where using (51) , Assumptions 1, 2 and 7,
Δ 3 = max 1 ≤ s ≤ t ∣ sup θ ∈ Θ ∥ Y s ( θ ) ∥ ∥ ( H ^ − H 0 ) F s ∥ ∣ = O ( 1 min ( N , T ) ) . (65) \Delta_3=\max_{1\leq s\leq t}\left|\sup_{\theta\in\Theta}\left\|Y_s(\theta)\right\|\left\|(\widehat{H}-H_0)F_s\right\|\right|=O\left(\frac{1}{\min\left(\sqrt{N},\sqrt{T}\right)}\right). \tag*{(65)} Δ 3 = 1 ≤ s ≤ t max θ ∈ Θ sup ∥ Y s ( θ ) ∥ ( H − H 0 ) F s = O min ( N , T ) 1 . (65)
To control the second term on the right-hand side of (64) , we use the fact that F t F_t F t and F ‾ t \overline{F}_t F t admit the model representation in (16) . Assumptions 1 and 2 therefore allow us to exploit the geometric contraction governing the difference between the stationary and nonstationary factor processes.
In particular, we have
sup θ ∈ Θ ∥ Y s ( θ ) ∥ ∥ H 0 ∥ ∥ F s − F ˉ s ∥ ≤ sup θ ∈ Θ ∥ Y s ( θ ) ∥ ∥ H 0 ∥ ∥ q ( ψ s ∣ s − 1 ( θ 0 ) , v s ) − q ( ψ ˉ s ∣ s − 1 ( θ 0 ) , v s ) ∥ = O a . s . ( X k [ s / k ] ) , \begin{aligned}
\sup_{\theta\in\Theta}\left\|Y_s(\theta)\right\|\left\|H_0\right\|\left\|F_s-\bar{F}_s\right\| \\
&\leq\sup_{\theta\in\Theta}\left\|Y_s(\theta)\right\|\left\|H_0\right\|\left\|q\left(\psi_{s|s-1}(\theta_0),v_s\right)-q\left(\bar{\psi}_{s|s-1}(\theta_0),v_s\right)\right\|=O_{a.s.}\left(\mathcal{X}_k^{[s/k]}\right),
\end{aligned} θ ∈ Θ sup ∥ Y s ( θ ) ∥ ∥ H 0 ∥ F s − F ˉ s ≤ θ ∈ Θ sup ∥ Y s ( θ ) ∥ ∥ H 0 ∥ q ( ψ s ∣ s − 1 ( θ 0 ) , v s ) − q ( ψ ˉ s ∣ s − 1 ( θ 0 ) , v s ) = O a . s . ( X k [ s / k ] ) ,
by Assumptions 1 1 1 , 2 2 2 and 7 7 7 . Substituting s = t − j + 1 s=t-j+1 s = t − j + 1 , we obtain
∑ j = 1 t X k ⌊ ( j − 1 ) / k ⌋ ∥ sup θ ∈ Θ ∥ Y t − j + 1 ( θ ) ∥ ∥ H 0 ∥ ( F t − j + 1 − F ˉ t − j + 1 ) ∥ = O ( ∑ j = 1 t X k ⌊ ( j − 1 ) / k ⌋ + [ ( t − j + 1 ) / k ] ) . (66) \sum_{j=1}^{t}\mathcal{X}_k^{\lfloor(j-1)/k\rfloor}\left\|\sup_{\theta\in\Theta}\left\|Y_{t-j+1}(\theta)\right\|\left\|H_0\right\|\left(F_{t-j+1}-\bar{F}_{t-j+1}\right)\right\|=O\left(\sum_{j=1}^{t}\mathcal{X}_k^{\lfloor(j-1)/k\rfloor+[(t-j+1)/k]}\right). \tag*{(66)} j = 1 ∑ t X k ⌊( j − 1 ) / k ⌋ θ ∈ Θ sup ∥ Y t − j + 1 ( θ ) ∥ ∥ H 0 ∥ ( F t − j + 1 − F ˉ t − j + 1 ) = O ( j = 1 ∑ t X k ⌊( j − 1 ) / k ⌋ + [( t − j + 1 ) / k ] ) . (66)
Since ⌊ j − 1 k ⌋ + ⌊ t − j + 1 k ⌋ ∈ { ⌊ t k ⌋ − 1 , ⌊ t k ⌋ } \left\lfloor\frac{j-1}{k}\right\rfloor+\left\lfloor\frac{t-j+1}{k}\right\rfloor\in\left\{\left\lfloor\frac{t}{k}\right\rfloor-1,\left\lfloor\frac{t}{k}\right\rfloor\right\} ⌊ k j − 1 ⌋ + ⌊ k t − j + 1 ⌋ ∈ { ⌊ k t ⌋ − 1 , ⌊ k t ⌋ } and 0 < X k < 1 0<\mathcal{X}_k<1 0 < X k < 1 , we can write X k ⌊ ( j − 1 ) / k ⌋ + ⌊ ( t − j + 1 ) / k ⌋ ≤ X k ⌊ t / k ⌋ − 1 \mathcal{X}_k^{\lfloor(j-1)/k\rfloor+\lfloor(t-j+1)/k\rfloor}\leq\mathcal{X}_k^{\lfloor t/k\rfloor-1} X k ⌊( j − 1 ) / k ⌋ + ⌊( t − j + 1 ) / k ⌋ ≤ X k ⌊ t / k ⌋ − 1 , such that ∑ j = 1 t X k ⌊ ( j − 1 ) / k ⌋ + ⌊ ( t − j + 1 ) / k ⌋ ≤ t ⋅ X k ⌊ t / k ⌋ − 1 \sum_{j=1}^{t}\mathcal{X}_k^{\lfloor(j-1)/k\rfloor+\lfloor(t-j+1)/k\rfloor}\leq t\cdot\mathcal{X}_k^{\lfloor t/k\rfloor-1} ∑ j = 1 t X k ⌊( j − 1 ) / k ⌋ + ⌊( t − j + 1 ) / k ⌋ ≤ t ⋅ X k ⌊ t / k ⌋ − 1 . By inserting this result into (66) we obtain
∑ j = 1 t X k ⌊ ( j − 1 ) / k ⌋ ∥ sup θ ∈ Θ ∥ Y t − j + 1 ( θ ) ∥ ∥ H 0 ∥ ( F t − j + 1 − F ˉ t − j + 1 ) ∥ = O ( ϱ k ⌊ t / k ⌋ ) , (67) \sum_{j=1}^{t}\mathcal{X}_k^{\lfloor(j-1)/k\rfloor}\left\|\sup_{\theta\in\Theta}\left\|Y_{t-j+1}(\theta)\right\|\left\|H_0\right\|\left(F_{t-j+1}-\bar{F}_{t-j+1}\right)\right\|=O\left(\varrho_k^{\lfloor t/k\rfloor}\right), \tag*{(67)} j = 1 ∑ t X k ⌊( j − 1 ) / k ⌋ θ ∈ Θ sup ∥ Y t − j + 1 ( θ ) ∥ ∥ H 0 ∥ ( F t − j + 1 − F ˉ t − j + 1 ) = O ( ϱ k ⌊ t / k ⌋ ) , (67)
for some 0 < X k < ϱ k < 1 0<\mathcal{X}_k<\varrho_k<1 0 < X k < ϱ k < 1 .
Therefore, substituting (67) and (65) into (64) , reveals that
sup θ ∈ Θ ∥ ψ ^ t + 1 ∣ t ( θ ) − ψ ~ t + 1 ∣ t ( θ ) ∥ = O P ( 1 min ( N , T ) ) + O a . s . ( ϱ k ⌊ t / k ⌋ ) . (68) \sup_{\theta\in\Theta}\left\|\hat{\psi}_{t+1|t}(\theta)-\tilde{\psi}_{t+1|t}(\theta)\right\|=O_{\mathrm{P}}\left(\frac{1}{\min(\sqrt{N},\sqrt{T})}\right)+O_{a.s.}\left(\varrho_k^{\lfloor t/k\rfloor}\right). \tag*{(68)} θ ∈ Θ sup ψ ^ t + 1∣ t ( θ ) − ψ ~ t + 1∣ t ( θ ) = O P ( min ( N , T ) 1 ) + O a . s . ( ϱ k ⌊ t / k ⌋ ) . (68)
(c) For proving the third claim, we verify the conditions given in Theorem 3.1 of Bougerol [12 ] . First, Assumptions 1 1 1 and 2 2 2 imply that the sequence of stochastic functions defining the filtering recursion, driven by the stationary and ergodic process { H 0 F ˉ t } t ∈ Z \left\{H_0\bar{F}_t\right\}_{t\in\mathbb{Z}} { H 0 F ˉ t } t ∈ Z , is itself stationary and ergodic. Therefore, to establish the existence of a unique stationary and ergodic solution { ψ ˉ t ∣ t − 1 ( θ ) } t ∈ Z \left\{\bar{\psi}_{t|t-1}(\theta)\right\}_{t\in\mathbb{Z}} { ψ ˉ t ∣ t − 1 ( θ ) } t ∈ Z and the convergence of the initialized filter toward this solution, it remains to verify conditions (C.1) and (C.2) of Theorem 3.1 of Bougerol [12 ] .
We start by verifying (C.1), that requires the existence of a log-moment for the displacement of the stochastic recurrence { ψ ~ t ∣ t − 1 ( θ ) } t ∈ Z \left\{\tilde{\psi}_{t|t-1}(\theta)\right\}_{t\in\mathbb{Z}} { ψ ~ t ∣ t − 1 ( θ ) } t ∈ Z at one fixed point. To prove this, we note that Assumption 6 6 6 entails that
E [ log + sup θ ∈ Θ ∥ ψ ~ 2 ∣ 1 ( θ ) − ψ 1 ∣ 0 ∥ ] ≤ 2 log 2 + log + sup θ ∈ Θ ∥ δ ∥ + log + sup θ ∈ Θ ∥ Φ − I r ∥ + log + sup θ ∈ Θ ∥ ψ 1 ∣ 0 ∥ \mathbb{E}\left[\log^{+}\sup_{\theta\in\Theta}\left\|\tilde{\psi}_{2|1}(\theta)-\psi_{1|0}\right\|\right]\leq2\log2+\log^{+}\sup_{\theta\in\Theta}\|\delta\|+\log^{+}\sup_{\theta\in\Theta}\left\|\Phi-I_r\right\|+\log^{+}\sup_{\theta\in\Theta}\left\|\psi_{1|0}\right\| E [ log + θ ∈ Θ sup ψ ~ 2∣1 ( θ ) − ψ 1∣0 ] ≤ 2 log 2 + log + θ ∈ Θ sup ∥ δ ∥ + log + θ ∈ Θ sup ∥ Φ − I r ∥ + log + θ ∈ Θ sup ψ 1∣0
+ log + sup θ ∈ Θ ∥ K ∥ + E [ log + sup θ ∈ Θ ∥ u 1 ( θ ) ∥ ] < ∞ . (69) \qquad+\log^{+}\sup_{\theta\in\Theta}\|K\|+\mathbb{E}\left[\log^{+}\sup_{\theta\in\Theta}\left\|u_1(\theta)\right\|\right]<\infty. \tag*{(69)} + log + θ ∈ Θ sup ∥ K ∥ + E [ log + θ ∈ Θ sup ∥ u 1 ( θ ) ∥ ] < ∞. (69)
Furthermore, for condition (C.2) in Theorem 3.1 of Bougerol [12 ] , it is enough to note that, by repeated applications of the mean value theorem, we get
sup θ ∈ Θ ∥ ψ ~ t + 1 ∣ t ( θ ) − ψ ‾ t + 1 ∣ t ( θ ) ∥ = sup θ ∈ Θ ∥ Φ + K ∂ u t ∂ ψ T ∣ ψ = ψ t − 1 ∣ t − 1 ( θ ) ( ψ ~ t ∣ t − 1 ( θ ) − ψ ‾ t ∣ t − 1 ( θ ) ) ∥ = sup θ ∈ Θ ∥ ∏ j = 1 k ( Φ + K ∂ u t − j + 1 ∂ ψ T ∣ ψ = ψ t − j + 1 ∣ t − j ( θ ) ) ( ψ ~ t − k + 1 ∣ t − k ( θ ) − ψ ‾ t − k + 1 ∣ t − k ( θ ) ) ∥ ≤ sup θ ∈ Θ sup ψ ∈ P ∥ ∏ j = 1 k X t − j + 1 ( θ ) ∥ sup θ ∈ Θ ∥ ψ ~ t − k + 1 ∣ t − k ( θ ) − ψ ‾ t − k + 1 ∣ t − k ( θ ) ∥ ≤ sup θ ∈ Θ ∣ X t , k ( θ ) ∣ sup θ ∈ Θ ∥ ψ ~ t − k + 1 ∣ t − k ( θ ) − ψ ‾ t − k + 1 ∣ t − k ( θ ) ∥ , \sup_{\theta\in\Theta}\left\|\widetilde{\psi}_{t+1|t}(\theta)-\overline{\psi}_{t+1|t}(\theta)\right\|
= \sup_{\theta\in\Theta}\left\|\Phi+K\left.\frac{\partial u_t}{\partial\psi^\mathsf{T}}\right|_{\psi=\psi_{t-1|t-1}(\theta)}\left(\widetilde{\psi}_{t|t-1}(\theta)-\overline{\psi}_{t|t-1}(\theta)\right)\right\|
= \sup_{\theta\in\Theta}\left\|\prod_{j=1}^{k}\left(\Phi+K\left.\frac{\partial u_{t-j+1}}{\partial\psi^\mathsf{T}}\right|_{\psi=\psi_{t-j+1|t-j}(\theta)}\right)\left(\widetilde{\psi}_{t-k+1|t-k}(\theta)-\overline{\psi}_{t-k+1|t-k}(\theta)\right)\right\|
\leq\sup_{\theta\in\Theta}\sup_{\psi\in\mathcal{P}}\left\|\prod_{j=1}^{k}X_{t-j+1}(\theta)\right\|\sup_{\theta\in\Theta}\left\|\widetilde{\psi}_{t-k+1|t-k}(\theta)-\overline{\psi}_{t-k+1|t-k}(\theta)\right\|
\leq\sup_{\theta\in\Theta}\left|X_{t,k}(\theta)\right|\sup_{\theta\in\Theta}\left\|\widetilde{\psi}_{t-k+1|t-k}(\theta)-\overline{\psi}_{t-k+1|t-k}(\theta)\right\|, θ ∈ Θ sup ψ t + 1∣ t ( θ ) − ψ t + 1∣ t ( θ ) = θ ∈ Θ sup Φ + K ∂ ψ T ∂ u t ψ = ψ t − 1∣ t − 1 ( θ ) ( ψ t ∣ t − 1 ( θ ) − ψ t ∣ t − 1 ( θ ) ) = θ ∈ Θ sup j = 1 ∏ k ( Φ + K ∂ ψ T ∂ u t − j + 1 ψ = ψ t − j + 1∣ t − j ( θ ) ) ( ψ t − k + 1∣ t − k ( θ ) − ψ t − k + 1∣ t − k ( θ ) ) ≤ θ ∈ Θ sup ψ ∈ P sup j = 1 ∏ k X t − j + 1 ( θ ) θ ∈ Θ sup ψ t − k + 1∣ t − k ( θ ) − ψ t − k + 1∣ t − k ( θ ) ≤ θ ∈ Θ sup ∣ X t , k ( θ ) ∣ θ ∈ Θ sup ψ t − k + 1∣ t − k ( θ ) − ψ t − k + 1∣ t − k ( θ ) ,
with ψ t − j + 1 ∣ t − j ( θ ) ∈ ( ψ ~ t − j + 1 ∣ t − j ( θ ) , ψ ‾ t − j + 1 ∣ t − j ( θ ) ) \psi_{t-j+1|t-j}(\theta)\in\left(\widetilde{\psi}_{t-j+1|t-j}(\theta),\overline{\psi}_{t-j+1|t-j}(\theta)\right) ψ t − j + 1∣ t − j ( θ ) ∈ ( ψ t − j + 1∣ t − j ( θ ) , ψ t − j + 1∣ t − j ( θ ) ) for j = 1 , … , k j=1,\ldots,k j = 1 , … , k and k ≥ 1 k\geq1 k ≥ 1 . Now, since under Assumption 2 both { F ‾ t } t ∈ Z \{\overline{F}_t\}_{t\in\mathbb{Z}} { F t } t ∈ Z and { X t , k ( θ ) } t ∈ N \{X_{t,k}(\theta)\}_{t\in\mathbb{N}} { X t , k ( θ ) } t ∈ N are strictly stationary and ergodic, it follows by Assumption 6 that also C.2 of Theorem 3.1 of Bougerol [12 ] is satisfied.
Thus, without loss of generality, assuming that t t t is a multiple of k k k and proceeding recursively,
sup θ ∈ Θ ∥ ψ ~ t + 1 ∣ t ( θ ) − ψ ‾ t + 1 ∣ t ( θ ) ∥ ≤ sup θ ∈ Θ ∣ X t , k ( θ ) ∣ sup θ ∈ Θ ∥ ψ ~ t − k + 1 ∣ t − k ( θ ) − ψ ‾ t − k + 1 ∣ t − k ( θ ) ∥ ≤ sup θ ∈ Θ ∣ X t , k ( θ ) ∣ ∣ X t − k , k ( θ ) ∣ sup θ ∈ Θ ∥ ψ ~ t − 2 k + 1 ∣ t − 2 k ( θ ) − ψ ‾ t − 2 k + 1 ∣ t − 2 k ( θ ) ∥ ≤ sup θ ∈ Θ ∣ X t , k ( θ ) ∣ ∣ X t − k , k ( θ ) ∣ ∣ X t − 2 k , k ( θ ) ∣ ⋯ ∣ X k , k ( θ ) ∣ sup θ ∈ Θ ∥ ψ 1 ∣ 0 ∗ − ψ 1 ∣ 0 ( θ ) ∥ . \sup_{\theta\in\Theta}\left\|\widetilde{\psi}_{t+1|t}(\theta)-\overline{\psi}_{t+1|t}(\theta)\right\|
\leq\sup_{\theta\in\Theta}\left|X_{t,k}(\theta)\right|\sup_{\theta\in\Theta}\left\|\widetilde{\psi}_{t-k+1|t-k}(\theta)-\overline{\psi}_{t-k+1|t-k}(\theta)\right\|
\leq\sup_{\theta\in\Theta}\left|X_{t,k}(\theta)\right|\left|X_{t-k,k}(\theta)\right|\sup_{\theta\in\Theta}\left\|\widetilde{\psi}_{t-2k+1|t-2k}(\theta)-\overline{\psi}_{t-2k+1|t-2k}(\theta)\right\|
\leq\sup_{\theta\in\Theta}\left|X_{t,k}(\theta)\right|\left|X_{t-k,k}(\theta)\right|\left|X_{t-2k,k}(\theta)\right|\cdots\left|X_{k,k}(\theta)\right|\sup_{\theta\in\Theta}\left\|\psi^*_{1|0}-\psi_{1|0}(\theta)\right\|. θ ∈ Θ sup ψ t + 1∣ t ( θ ) − ψ t + 1∣ t ( θ ) ≤ θ ∈ Θ sup ∣ X t , k ( θ ) ∣ θ ∈ Θ sup ψ t − k + 1∣ t − k ( θ ) − ψ t − k + 1∣ t − k ( θ ) ≤ θ ∈ Θ sup ∣ X t , k ( θ ) ∣ ∣ X t − k , k ( θ ) ∣ θ ∈ Θ sup ψ t − 2 k + 1∣ t − 2 k ( θ ) − ψ t − 2 k + 1∣ t − 2 k ( θ ) ≤ θ ∈ Θ sup ∣ X t , k ( θ ) ∣ ∣ X t − k , k ( θ ) ∣ ∣ X t − 2 k , k ( θ ) ∣ ⋯ ∣ X k , k ( θ ) ∣ θ ∈ Θ sup ψ 1∣0 ∗ − ψ 1∣0 ( θ ) .
Clearly, the process { sup θ ∈ Θ ∣ X t − i k , k ( θ ) ∣ } i ≥ 0 \{\sup_{\theta\in\Theta}|X_{t-ik,k}(\theta)|\}_{i\geq0} { sup θ ∈ Θ ∣ X t − ik , k ( θ ) ∣ } i ≥ 0 in the last inequality is stationary and ergodic, and from Assumption 6 we have that E [ log sup θ ∈ Θ ∣ X t , k ( θ ) ∣ ] < 0 \mathbb{E}\left[\log\sup_{\theta\in\Theta}|X_{t,k}(\theta)|\right]<0 E [ log sup θ ∈ Θ ∣ X t , k ( θ ) ∣ ] < 0 . Then, by Lemma 2.4 of Straumann and Mikosch [52 ] it holds that ∏ i = 0 ⌊ t / k ⌋ − 1 sup θ ∈ Θ ∣ X t − i k , k ( θ ) ∣ → a.s. 0 \prod_{i=0}^{\lfloor t/k\rfloor-1}\sup_{\theta\in\Theta}|X_{t-ik,k}(\theta)|\xrightarrow{\text{a.s.}}0 ∏ i = 0 ⌊ t / k ⌋ − 1 sup θ ∈ Θ ∣ X t − ik , k ( θ ) ∣ a.s. 0 as t → ∞ t\to\infty t → ∞ and therefore, this result in combination with the bound obtained in (69) implies that
sup θ ∈ Θ ∥ ψ ~ t + 1 ∣ t ( θ ) − ψ ‾ t + 1 ∣ t ( θ ) ∥ ≤ ( ∏ i = 0 ⌊ t / k ⌋ − 1 sup θ ∈ Θ ∣ X t − i k , k ( θ ) ∣ ) sup θ ∈ Θ ∥ ψ 1 ∣ 0 − ψ ‾ 1 ∣ 0 ( θ ) ∥ → a.s. 0 , t → ∞ . \sup_{\theta\in\Theta}\left\|\widetilde{\psi}_{t+1|t}(\theta)-\overline{\psi}_{t+1|t}(\theta)\right\|
\leq\left(\prod_{i=0}^{\lfloor t/k\rfloor-1}\sup_{\theta\in\Theta}|X_{t-ik,k}(\theta)|\right)\sup_{\theta\in\Theta}\left\|\psi_{1|0}-\overline{\psi}_{1|0}(\theta)\right\|\xrightarrow{\text{a.s.}}0,\qquad t\to\infty. θ ∈ Θ sup ψ t + 1∣ t ( θ ) − ψ t + 1∣ t ( θ ) ≤ i = 0 ∏ ⌊ t / k ⌋ − 1 θ ∈ Θ sup ∣ X t − ik , k ( θ ) ∣ θ ∈ Θ sup ψ 1∣0 − ψ 1∣0 ( θ ) a.s. 0 , t → ∞.
This proves that the filtered vector process { ψ ~ t ∣ t − 1 ( θ ) } t ∈ N \{\widetilde{\psi}_{t|t-1}(\theta)\}_{t\in\mathbb{N}} { ψ t ∣ t − 1 ( θ ) } t ∈ N is invertible, and converges exponentially fast almost surely to the unique stationary ergodic sequence { ψ ‾ t ∣ t − 1 ( θ ) } t ∈ Z \{\overline{\psi}_{t|t-1}(\theta)\}_{t\in\mathbb{Z}} { ψ t ∣ t − 1 ( θ ) } t ∈ Z for any initialization of the filtering recursion, ψ 1 ∣ 0 ∈ P \psi_{1|0}\in\mathcal{P} ψ 1∣0 ∈ P uniformly over Θ \Theta Θ and so
sup θ ∈ Θ ∥ ψ ~ t + 1 ∣ t ( θ ) − ψ ‾ t + 1 ∣ t ( θ ) ∥ = O a . s . ( ϱ k ⌊ t / k ⌋ ) , (70) \sup_{\theta\in\Theta}\left\|\widetilde{\psi}_{t+1|t}(\theta)-\overline{\psi}_{t+1|t}(\theta)\right\|=O_{a.s.}\left(\varrho_k^{\lfloor t/k\rfloor}\right), \tag*{(70)} θ ∈ Θ sup ψ t + 1∣ t ( θ ) − ψ t + 1∣ t ( θ ) = O a . s . ( ϱ k ⌊ t / k ⌋ ) , (70)
with ∏ i = 0 ⌊ t / k ⌋ − 1 sup θ ∈ Θ ∣ X t − i k , k ( θ ) ∣ = O a . s . ( ϱ k ⌊ t / k ⌋ ) \prod_{i=0}^{\lfloor t/k\rfloor-1}\sup_{\theta\in\Theta}|X_{t-ik,k}(\theta)|=O_{a.s.}\left(\varrho_k^{\lfloor t/k\rfloor}\right) ∏ i = 0 ⌊ t / k ⌋ − 1 sup θ ∈ Θ ∣ X t − ik , k ( θ ) ∣ = O a . s . ( ϱ k ⌊ t / k ⌋ ) for some 0 < ϱ k < 1 0<\varrho_k<1 0 < ϱ k < 1 .
(d) The near-epoch dependence property of { ψ ‾ t ∣ t − 1 ( θ 0 ) } t ∈ N \left\{\overline{\psi}_{t|t-1}(\theta_0)\right\}_{t\in\mathbb{N}} { ψ t ∣ t − 1 ( θ 0 ) } t ∈ N follows by a direct application of Lemma 6.
(e) The proof of part (e) follows directly from (63) , (68) and (70) . This completes the proof. ▪
Proof of Proposition 3
By the triangle inequality,
sup θ ∈ Θ ∣ 1 T ∑ t = 1 T ℓ ^ F ( F ^ t ; r H ^ ( θ ) ) + log ∣ H ^ ∣ − E [ ℓ F ( F ‾ t ; θ ) ] ∣ ≤ sup θ ∈ Θ ∣ 1 T ∑ t = 1 T ℓ ^ F ( F ^ t ; r H ^ ( θ ) ) − 1 T ∑ t = 1 T ℓ ^ F ( F t H ^ T ; r H ^ ( θ ) ) ∣ + sup θ ∈ Θ ∣ 1 T ∑ t = 1 T ℓ ^ F ( F t H ^ T ; r H ^ ( θ ) ) + log ∣ H ^ ∣ − 1 T ∑ t = 1 T ℓ F ( F t ; θ ) ∣ + sup θ ∈ Θ ∣ 1 T ∑ t = 1 T ℓ F ( F t ; θ ) − 1 T ∑ t = 1 T ℓ F ( F ‾ t ; θ ) ∣ + sup θ ∈ Θ ∣ 1 T ∑ t = 1 T ℓ F ( F ‾ t ; θ ) − E [ ℓ F ( F ‾ t ; θ ) ] ∣ . (71) \begin{aligned}
\sup_{\theta\in\Theta}\left|\frac{1}{T}\sum_{t=1}^{T}\widehat{\ell}_{F}\left(\widehat{F}_{t};r_{\widehat{H}}(\theta)\right)+\log\left|\widehat{H}\right|-\mathbb{E}\left[\ell_{F}\left(\overline{F}_{t};\theta\right)\right]\right|
&\leq\sup_{\theta\in\Theta}\left|\frac{1}{T}\sum_{t=1}^{T}\widehat{\ell}_{F}\left(\widehat{F}_{t};r_{\widehat{H}}(\theta)\right)-\frac{1}{T}\sum_{t=1}^{T}\widehat{\ell}_{F}\left(F_{t}\widehat{H}^{\mathsf{T}};r_{\widehat{H}}(\theta)\right)\right|\\
&\quad+\sup_{\theta\in\Theta}\left|\frac{1}{T}\sum_{t=1}^{T}\widehat{\ell}_{F}\left(F_{t}\widehat{H}^{\mathsf{T}};r_{\widehat{H}}(\theta)\right)+\log\left|\widehat{H}\right|-\frac{1}{T}\sum_{t=1}^{T}\ell_{F}\left(F_{t};\theta\right)\right|\\
&\quad+\sup_{\theta\in\Theta}\left|\frac{1}{T}\sum_{t=1}^{T}\ell_{F}\left(F_{t};\theta\right)-\frac{1}{T}\sum_{t=1}^{T}\ell_{F}\left(\overline{F}_{t};\theta\right)\right|\\
&\quad+\sup_{\theta\in\Theta}\left|\frac{1}{T}\sum_{t=1}^{T}\ell_{F}\left(\overline{F}_{t};\theta\right)-\mathbb{E}\left[\ell_{F}\left(\overline{F}_{t};\theta\right)\right]\right|. \tag*{(71)}
\end{aligned} θ ∈ Θ sup T 1 t = 1 ∑ T ℓ F ( F t ; r H ( θ ) ) + log H − E [ ℓ F ( F t ; θ ) ] ≤ θ ∈ Θ sup T 1 t = 1 ∑ T ℓ F ( F t ; r H ( θ ) ) − T 1 t = 1 ∑ T ℓ F ( F t H T ; r H ( θ ) ) + θ ∈ Θ sup T 1 t = 1 ∑ T ℓ F ( F t H T ; r H ( θ ) ) + log H − T 1 t = 1 ∑ T ℓ F ( F t ; θ ) + θ ∈ Θ sup T 1 t = 1 ∑ T ℓ F ( F t ; θ ) − T 1 t = 1 ∑ T ℓ F ( F t ; θ ) + θ ∈ Θ sup T 1 t = 1 ∑ T ℓ F ( F t ; θ ) − E [ ℓ F ( F t ; θ ) ] . (71)
We now prove that each term on the right-hand side of (71) converges in probability to zero uniformly over θ ∈ Θ \theta\in\Theta θ ∈ Θ .
For the first term, we retain the time-averaging structure and write
sup θ ∈ Θ ∣ 1 T ∑ t = 1 T ℓ ^ F ( F ^ t ; r H ^ ( θ ) ) − 1 T ∑ t = 1 T ℓ ^ F ( F t H ^ T ; r H ^ ( θ ) ) ∣ = sup θ ∈ Θ ∣ 1 T ∑ t = 1 T ( ℓ ^ F ( F ^ t ; r H ^ ( θ ) ) − ℓ ^ F ( H ^ F t ; r H ^ ( θ ) ) ) ∣ . (72) \begin{aligned}
\sup_{\theta\in\Theta}\left|\frac{1}{T}\sum_{t=1}^{T}\widehat{\ell}_{F}\left(\widehat{F}_{t};r_{\widehat{H}}(\theta)\right)-\frac{1}{T}\sum_{t=1}^{T}\widehat{\ell}_{F}\left(F_{t}\widehat{H}^{\mathsf{T}};r_{\widehat{H}}(\theta)\right)\right|\\
=\sup_{\theta\in\Theta}\left|\frac{1}{T}\sum_{t=1}^{T}\left(\widehat{\ell}_{F}\left(\widehat{F}_{t};r_{\widehat{H}}(\theta)\right)-\widehat{\ell}_{F}\left(\widehat{H}F_{t};r_{\widehat{H}}(\theta)\right)\right)\right|. \tag*{(72)}
\end{aligned} θ ∈ Θ sup T 1 t = 1 ∑ T ℓ F ( F t ; r H ( θ ) ) − T 1 t = 1 ∑ T ℓ F ( F t H T ; r H ( θ ) ) = θ ∈ Θ sup T 1 t = 1 ∑ T ( ℓ F ( F t ; r H ( θ ) ) − ℓ F ( H F t ; r H ( θ ) ) ) . (72)
Notice that we do not apply the triangle inequality to the sum in (72) , since doing so would discard the time-averaging structure of the factor-estimation error.
For the second term, by the triangle inequality
sup θ ∈ Θ ∣ 1 T ∑ t = 1 T ℓ ^ F ( F t H ^ T ; r H ^ ( θ ) ) + log ∣ H ^ ∣ − 1 T ∑ t = 1 T ℓ F ( F t ; θ ) ∣ ≤ 1 T ∑ t = 1 T sup θ ∈ Θ ∣ ℓ F ( H ^ F t ; r H ^ ( θ ) ) + log ∣ H ^ ∣ − ℓ F ( F t ; θ ) ∣ . (73) \begin{aligned}
\sup_{\theta\in\Theta}\left|\frac{1}{T}\sum_{t=1}^{T}\widehat{\ell}_{F}\left(F_{t}\widehat{H}^{\mathsf{T}};r_{\widehat{H}}(\theta)\right)+\log\left|\widehat{H}\right|-\frac{1}{T}\sum_{t=1}^{T}\ell_{F}\left(F_{t};\theta\right)\right| \\
&\le\frac{1}{T}\sum_{t=1}^{T}\sup_{\theta\in\Theta}\left|\ell_F\left(\widehat{H}F_t;r_{\widehat{H}}(\theta)\right)+\log\left|\widehat{H}\right|-\ell_F(F_t;\theta)\right|. \tag*{(73)}
\end{aligned} θ ∈ Θ sup T 1 t = 1 ∑ T ℓ F ( F t H T ; r H ( θ ) ) + log H − T 1 t = 1 ∑ T ℓ F ( F t ; θ ) ≤ T 1 t = 1 ∑ T θ ∈ Θ sup ℓ F ( H F t ; r H ( θ ) ) + log H − ℓ F ( F t ; θ ) . (73)
By equivariance of the conditional likelihood under the alignment mapping (see equations (112) –(114) below), the summands on the right-hand side of (73) would be zero if the two filters were initialized consistently under the transformation. Consequently, these summands contain only the effect of the possibly different filter initializations.
For the third term, we similarly obtain
sup θ ∈ Θ ∣ 1 T ∑ t = 1 T ℓ F ( F t ; θ ) − 1 T ∑ t = 1 T ℓ F ( F ‾ t ; θ ) ∣ ≤ 1 T ∑ t = 1 T sup θ ∈ Θ ∣ ℓ F ( F t ; θ ) − ℓ F ( F ‾ t ; θ ) ∣ . (74) \begin{aligned}
\sup_{\theta\in\Theta}\left|\frac{1}{T}\sum_{t=1}^{T}\ell_F(F_t;\theta)-\frac{1}{T}\sum_{t=1}^{T}\ell_F(\overline{F}_t;\theta)\right| \\
&\le\frac{1}{T}\sum_{t=1}^{T}\sup_{\theta\in\Theta}\left|\ell_F(F_t;\theta)-\ell_F(\overline{F}_t;\theta)\right|. \tag*{(74)}
\end{aligned} θ ∈ Θ sup T 1 t = 1 ∑ T ℓ F ( F t ; θ ) − T 1 t = 1 ∑ T ℓ F ( F t ; θ ) ≤ T 1 t = 1 ∑ T θ ∈ Θ sup ℓ F ( F t ; θ ) − ℓ F ( F t ; θ ) . (74)
Finally, the fourth term is
sup θ ∈ Θ ∣ 1 T ∑ t = 1 T ( ℓ F ( F ‾ t ; θ ) − E [ ℓ F ( F ‾ t ; θ ) ] ) ∣ . (75) \sup_{\theta\in\Theta}\left|\frac{1}{T}\sum_{t=1}^{T}\left(\ell_F(\overline{F}_t;\theta)-\mathbb{E}\left[\ell_F(\overline{F}_t;\theta)\right]\right)\right|. \tag*{(75)} θ ∈ Θ sup T 1 t = 1 ∑ T ( ℓ F ( F t ; θ ) − E [ ℓ F ( F t ; θ ) ] ) . (75)
We now examine the first term on the right-hand side of (71) .
Controlling the term in (72)
A first-order Taylor expansion around ( ψ ^ t ∣ t − 1 ( r H ^ ( θ ) ) , H ^ F t ) \left(\widehat{\psi}_{t|t-1}\left(r_{\widehat{H}}(\theta)\right),\widehat{H}F_t\right) ( ψ t ∣ t − 1 ( r H ( θ ) ) , H F t ) with a second-order remainder gives, uniformly over θ ∈ Θ \theta\in\Theta θ ∈ Θ ,
1 T ∑ t = 1 T ( ℓ ^ F ( F ^ t ; r H ^ ( θ ) ) − ℓ ^ F ( H ^ F t ; r H ^ ( θ ) ) ) = 1 T ∑ t = 1 T ∇ ψ ℓ ^ F ( H ^ F t ; r H ^ ( θ ) ) T ( ψ ^ t ∣ t − 1 ( r H ^ ( θ ) ) − ψ ^ t ∣ t − 1 ( r H ^ ( θ ) ) ) + 1 T ∑ t = 1 T ∇ F ℓ ^ F ( H ^ F t ; r H ^ ( θ ) ) T ( F ^ t − H ^ F t ) + 1 T ∑ t = 1 T R t , N T ( θ ) . (76) \begin{aligned}
\frac{1}{T}\sum_{t=1}^{T}\left(\widehat{\ell}_F\left(\widehat{F}_t;r_{\widehat{H}}(\theta)\right)-\widehat{\ell}_F\left(\widehat{H}F_t;r_{\widehat{H}}(\theta)\right)\right)
&= \frac{1}{T}\sum_{t=1}^{T}\nabla^\psi\widehat{\ell}_F\left(\widehat{H}F_t;r_{\widehat{H}}(\theta)\right)^{\mathsf{T}}\left(\widehat{\psi}_{t|t-1}\left(r_{\widehat{H}}(\theta)\right)-\widehat{\psi}_{t|t-1}\left(r_{\widehat{H}}(\theta)\right)\right) \\
&\quad+ \frac{1}{T}\sum_{t=1}^{T}\nabla^F\widehat{\ell}_F\left(\widehat{H}F_t;r_{\widehat{H}}(\theta)\right)^{\mathsf{T}}\left(\widehat{F}_t-\widehat{H}F_t\right) \\
&\quad+ \frac{1}{T}\sum_{t=1}^{T}R_{t,NT}(\theta). \tag*{(76)}
\end{aligned} T 1 t = 1 ∑ T ( ℓ F ( F t ; r H ( θ ) ) − ℓ F ( H F t ; r H ( θ ) ) ) = T 1 t = 1 ∑ T ∇ ψ ℓ F ( H F t ; r H ( θ ) ) T ( ψ t ∣ t − 1 ( r H ( θ ) ) − ψ t ∣ t − 1 ( r H ( θ ) ) ) + T 1 t = 1 ∑ T ∇ F ℓ F ( H F t ; r H ( θ ) ) T ( F t − H F t ) + T 1 t = 1 ∑ T R t , N T ( θ ) . (76)
Under Assumptions 8 and 9, there exists a non-negative process { C t } t ∈ N \{C_t\}_{t\in\mathbb{N}} { C t } t ∈ N , with uniformly bounded moments of the required order, such that
sup θ ∈ Θ ∣ R t , N T ( θ ) ∣ ≤ C t ( ∥ F ^ t − H ^ F t ∥ 2 + sup θ ∈ Θ ∥ ψ ^ t ∣ t − 1 ( r H ^ ( θ ) ) − ψ ^ t ∣ t − 1 ( r H ^ ( θ ) ) ∥ 2 ) . (77) \sup_{\theta\in\Theta}\left|R_{t,NT}(\theta)\right|\le C_t\left(\left\|\widehat{F}_t-\widehat{H}F_t\right\|^2+\sup_{\theta\in\Theta}\left\|\widehat{\psi}_{t|t-1}\left(r_{\widehat{H}}(\theta)\right)-\widehat{\psi}_{t|t-1}\left(r_{\widehat{H}}(\theta)\right)\right\|^2\right). \tag*{(77)} θ ∈ Θ sup ∣ R t , N T ( θ ) ∣ ≤ C t ( F t − H F t 2 + θ ∈ Θ sup ψ t ∣ t − 1 ( r H ( θ ) ) − ψ t ∣ t − 1 ( r H ( θ ) ) 2 ) . (77)
The remainder also contains a cross-product between the factor and filter errors, which is absorbed into (77) using 2 ∥ a ∥ ∥ b ∥ ≤ ∥ a ∥ 2 + ∥ b ∥ 2 2\lVert a\rVert\lVert b\rVert\le\lVert a\rVert^2+\lVert b\rVert^2 2 ∥ a ∥ ∥ b ∥ ≤ ∥ a ∥ 2 + ∥ b ∥ 2 , for any a , b ∈ R r a,b\in\mathbb{R}^r a , b ∈ R r .
To control the two average squared errors appearing on the right-hand side of (77) , we can apply Lemma A.1 of Bai [2 ] which implies that
1 T ∑ t = 1 T ∥ F ^ t − H ^ F t ∥ 2 = O P ( 1 N + 1 T ) . (78) \frac{1}{T}\sum_{t=1}^{T}\left\lVert\widehat{F}_t-\widehat{H}F_t\right\rVert^2=O_{\mathbb{P}}\left(\frac{1}{N}+\frac{1}{T}\right). \tag*{(78)} T 1 t = 1 ∑ T F t − H F t 2 = O P ( N 1 + T 1 ) . (78)
Equivalently, the root-mean-square factor-estimation error satisfies
( 1 T ∑ t = 1 T ∥ F ^ t − H ^ F t ∥ 2 ) 1 / 2 = O P ( 1 N + 1 T ) . \left(\frac{1}{T}\sum_{t=1}^{T}\left\lVert\widehat{F}_t-\widehat{H}F_t\right\rVert^2\right)^{1/2}=O_{\mathbb{P}}\left(\frac{1}{\sqrt{N}}+\frac{1}{\sqrt{T}}\right). ( T 1 t = 1 ∑ T F t − H F t 2 ) 1/2 = O P ( N 1 + T 1 ) .
The filter inherits the same root-mean-square rate. In fact, in a similar way as done in the proof of Proposition 2(a), iterating the one-step filter difference and using Assumption 7 we get
sup θ ∈ Θ ∥ ψ ^ t ∣ t − 1 ( r H ^ ( θ ) ) − ψ ^ t ∣ t − 1 ( r H ^ ( θ ) ) ∥ ≤ X k ( t − 1 ) / k ∥ ψ 1 ∣ 0 − ψ 1 ∣ 0 ∗ ∥ + ∑ s = 1 t − 1 X k ( t − s − 1 ) / k sup θ ∈ Θ ∥ Y s ( r H ^ ( θ ) ) ∥ ∥ F ^ s − H ^ F s ∥ . \sup_{\theta\in\Theta}\left\lVert\widehat{\psi}_{t\mid t-1}\left(r_{\widehat{H}}(\theta)\right)-\widehat{\psi}_{t\mid t-1}\left(r_{\widehat{H}}(\theta)\right)\right\rVert
\leq\mathcal{X}_k^{(t-1)/k}\left\lVert\psi_{1\mid0}-\psi^*_{1\mid0}\right\rVert+\sum_{s=1}^{t-1}\mathcal{X}_k^{(t-s-1)/k}\sup_{\theta\in\Theta}\left\lVert Y_s\left(r_{\widehat{H}}(\theta)\right)\right\rVert\left\lVert\widehat{F}_s-\widehat{H}F_s\right\rVert. θ ∈ Θ sup ψ t ∣ t − 1 ( r H ( θ ) ) − ψ t ∣ t − 1 ( r H ( θ ) ) ≤ X k ( t − 1 ) / k ψ 1 ∣ 0 − ψ 1 ∣ 0 ∗ + s = 1 ∑ t − 1 X k ( t − s − 1 ) / k θ ∈ Θ sup ∥ Y s ( r H ( θ ) ) ∥ F s − H F s .
Hence, by the discrete convolution inequality and the geometric summability of the powers of X k \mathcal{X}_k X k ,
( 1 T ∑ t = 1 T sup θ ∈ Θ ∥ ψ ^ t ∣ t − 1 ( r H ^ ( θ ) ) − ψ ^ t ∣ t − 1 ( r H ^ ( θ ) ) ∥ 2 ) 1 / 2 = O P ( ( 1 T ∑ t = 1 T ∥ F ^ t − H ^ F t ∥ 2 ) 1 / 2 + ( 1 T ∑ t = 1 T X k 2 ( t − 1 ) / k ) 1 / 2 ) = O P ( 1 N + 1 T ) . \begin{aligned}
\left(\frac{1}{T}\sum_{t=1}^{T}\sup_{\theta\in\Theta}\left\lVert\widehat{\psi}_{t\mid t-1}\left(r_{\widehat{H}}(\theta)\right)-\widehat{\psi}_{t\mid t-1}\left(r_{\widehat{H}}(\theta)\right)\right\rVert^2\right)^{1/2}
&=O_{\mathbb{P}}\left(\left(\frac{1}{T}\sum_{t=1}^{T}\left\lVert\widehat{F}_t-\widehat{H}F_t\right\rVert^2\right)^{1/2}+\left(\frac{1}{T}\sum_{t=1}^{T}\mathcal{X}_k^{2(t-1)/k}\right)^{1/2}\right)\\
&=O_{\mathbb{P}}\left(\frac{1}{\sqrt{N}}+\frac{1}{\sqrt{T}}\right).
\end{aligned} ( T 1 t = 1 ∑ T θ ∈ Θ sup ψ t ∣ t − 1 ( r H ( θ ) ) − ψ t ∣ t − 1 ( r H ( θ ) ) 2 ) 1/2 = O P ( T 1 t = 1 ∑ T F t − H F t 2 ) 1/2 + ( T 1 t = 1 ∑ T X k 2 ( t − 1 ) / k ) 1/2 = O P ( N 1 + T 1 ) .
Here,
1 T ∑ t = 1 T X k 2 ( t − 1 ) / k ≤ k T ( 1 − X k 2 ) = O ( 1 T ) , \frac{1}{T}\sum_{t=1}^{T}\mathcal{X}_k^{2(t-1)/k}\leq\frac{k}{T(1-\mathcal{X}_k^2)}=O\left(\frac{1}{T}\right), T 1 t = 1 ∑ T X k 2 ( t − 1 ) / k ≤ T ( 1 − X k 2 ) k = O ( T 1 ) ,
so the second term accounts for the geometrically decreasing effect of the different filter initializations. By squaring the preceding root-mean-square bound we therefore get
1 T ∑ t = 1 T sup θ ∈ Θ ∥ ψ ^ t ∣ t − 1 ( r H ^ ( θ ) ) − ψ ^ t ∣ t − 1 ( r H ^ ( θ ) ) ∥ 2 = O P ( 1 N + 1 T ) . (79) \frac{1}{T}\sum_{t=1}^{T}\sup_{\theta\in\Theta}\left\lVert\widehat{\psi}_{t\mid t-1}\left(r_{\widehat{H}}(\theta)\right)-\widehat{\psi}_{t\mid t-1}\left(r_{\widehat{H}}(\theta)\right)\right\rVert^2=O_{\mathbb{P}}\left(\frac{1}{N}+\frac{1}{T}\right). \tag*{(79)} T 1 t = 1 ∑ T θ ∈ Θ sup ψ t ∣ t − 1 ( r H ( θ ) ) − ψ t ∣ t − 1 ( r H ( θ ) ) 2 = O P ( N 1 + T 1 ) . (79)
Indeed, ( 1 N + 1 T ) 2 = 1 N + 1 T + 2 N T = O ( 1 N + 1 T ) \left(\frac{1}{\sqrt{N}}+\frac{1}{\sqrt{T}}\right)^2=\frac{1}{N}+\frac{1}{T}+\frac{2}{\sqrt{NT}}=O\left(\frac{1}{N}+\frac{1}{T}\right) ( N 1 + T 1 ) 2 = N 1 + T 1 + N T 2 = O ( N 1 + T 1 ) , where the last equality follows from the fact that 2 N T ≤ 1 N + 1 T \frac{2}{\sqrt{NT}}\leq\frac{1}{N}+\frac{1}{T} N T 2 ≤ N 1 + T 1 . Consequently, under Assumptions 8 and 9, Hölder’s inequality, and (78) –(79) imply
1 T ∑ t = 1 T sup θ ∈ Θ ∣ R t , N T ( θ ) ∣ = O P ( 1 N + 1 T ) . (80) \frac{1}{T}\sum_{t=1}^{T}\sup_{\theta\in\Theta}\left|R_{t,NT}(\theta)\right|=O_{\mathbb{P}}\left(\frac{1}{N}+\frac{1}{T}\right). \tag*{(80)} T 1 t = 1 ∑ T θ ∈ Θ sup ∣ R t , N T ( θ ) ∣ = O P ( N 1 + T 1 ) . (80)
Thus, (76) can equivalently be written as
1 T ∑ t = 1 T ( ℓ ^ F ( F ^ t ; r H ^ ( θ ) ) − ℓ ^ F ( H ^ F t ; r H ^ ( θ ) ) ) = 1 T ∑ t = 1 T ∇ ψ ℓ ^ F ( H ^ F t ; r H ^ ( θ ) ) T ( ψ ^ t ∣ t − 1 ( r H ^ ( θ ) ) − ψ ^ t ∣ t − 1 ( r H ^ ( θ ) ) ) + 1 T ∑ t = 1 T ∇ F ℓ ^ F ( H ^ F t ; r H ^ ( θ ) ) T ( F ^ t − H ^ F t ) + O P ( 1 N + 1 T ) , (81) \begin{aligned}
\frac{1}{T}\sum_{t=1}^{T}\left(\widehat{\ell}_{F}\left(\widehat{F}_{t};r_{\widehat{H}}(\theta)\right)-\widehat{\ell}_{F}\left(\widehat{H}F_{t};r_{\widehat{H}}(\theta)\right)\right)
&=\frac{1}{T}\sum_{t=1}^{T}\nabla^{\psi}\widehat{\ell}_{F}\left(\widehat{H}F_{t};r_{\widehat{H}}(\theta)\right)^{\mathsf{T}}\left(\widehat{\psi}_{t|t-1}\left(r_{\widehat{H}}(\theta)\right)-\widehat{\psi}_{t|t-1}\left(r_{\widehat{H}}(\theta)\right)\right)\\
&\quad+\frac{1}{T}\sum_{t=1}^{T}\nabla_{F}\widehat{\ell}_{F}\left(\widehat{H}F_{t};r_{\widehat{H}}(\theta)\right)^{\mathsf{T}}\left(\widehat{F}_{t}-\widehat{H}F_{t}\right)+O_{\mathbb{P}}\left(\frac{1}{N}+\frac{1}{T}\right), \tag*{(81)}
\end{aligned} T 1 t = 1 ∑ T ( ℓ F ( F t ; r H ( θ ) ) − ℓ F ( H F t ; r H ( θ ) ) ) = T 1 t = 1 ∑ T ∇ ψ ℓ F ( H F t ; r H ( θ ) ) T ( ψ t ∣ t − 1 ( r H ( θ ) ) − ψ t ∣ t − 1 ( r H ( θ ) ) ) + T 1 t = 1 ∑ T ∇ F ℓ F ( H F t ; r H ( θ ) ) T ( F t − H F t ) + O P ( N 1 + T 1 ) , (81)
uniformly over θ ∈ Θ \theta\in\Theta θ ∈ Θ .
Direct from a direct differentiation of the update maps, together with Assumptions 1–7, and a first-order Taylor expansion with a second-order remainder we therefore obtain
ψ ^ t + 1 ∣ t ( r H ^ ( θ ) ) − ψ ^ t + 1 ∣ t ( r H ^ ( θ ) ) = X t ( r H ^ ( θ ) ) ( ψ ^ t ∣ t − 1 ( r H ^ ( θ ) ) − ψ ^ t ∣ t − 1 ( r H ^ ( θ ) ) ) + Y t ( r H ^ ( θ ) ) ( F ^ t − H ^ F t ) + O P ( ∥ ψ ^ t ∣ t − 1 ( r H ^ ( θ ) ) − ψ ^ t ∣ t − 1 ( r H ^ ( θ ) ) ∥ 2 + ∥ F ^ t − H ^ F t ∥ 2 ) , (82) \begin{aligned}
\widehat{\psi}_{t+1|t}\left(r_{\widehat{H}}(\theta)\right)-\widehat{\psi}_{t+1|t}\left(r_{\widehat{H}}(\theta)\right)
&=X_{t}\left(r_{\widehat{H}}(\theta)\right)\left(\widehat{\psi}_{t|t-1}\left(r_{\widehat{H}}(\theta)\right)-\widehat{\psi}_{t|t-1}\left(r_{\widehat{H}}(\theta)\right)\right)\\
&\quad+Y_{t}\left(r_{\widehat{H}}(\theta)\right)\left(\widehat{F}_{t}-\widehat{H}F_{t}\right)+O_{\mathbb{P}}\left(\left\|\widehat{\psi}_{t|t-1}\left(r_{\widehat{H}}(\theta)\right)-\widehat{\psi}_{t|t-1}\left(r_{\widehat{H}}(\theta)\right)\right\|^{2}+\left\|\widehat{F}_{t}-\widehat{H}F_{t}\right\|^{2}\right), \tag*{(82)}
\end{aligned} ψ t + 1∣ t ( r H ( θ ) ) − ψ t + 1∣ t ( r H ( θ ) ) = X t ( r H ( θ ) ) ( ψ t ∣ t − 1 ( r H ( θ ) ) − ψ t ∣ t − 1 ( r H ( θ ) ) ) + Y t ( r H ( θ ) ) ( F t − H F t ) + O P ( ψ t ∣ t − 1 ( r H ( θ ) ) − ψ t ∣ t − 1 ( r H ( θ ) ) 2 + F t − H F t 2 ) , (82)
uniformly over θ ∈ Θ \theta\in\Theta θ ∈ Θ . Then, iterating (82) yields
ψ ^ t + 1 ∣ t ( r H ^ ( θ ) ) − ψ ^ t + 1 ∣ t ( r H ^ ( θ ) ) = ( ∏ h = 1 t X t − h + 1 ( r H ^ ( θ ) ) ) ( ψ 1 ∣ 0 − ψ 1 ∣ 0 ⋆ ) + ∑ s = 1 t ( ∏ h = s + 1 t X h ( r H ^ ( θ ) ) ) Y s ( r H ^ ( θ ) ) ( F ^ s − H ^ F s ) + second-order terms , (83) \begin{aligned}
\widehat{\psi}_{t+1|t}\left(r_{\widehat{H}}(\theta)\right)-\widehat{\psi}_{t+1|t}\left(r_{\widehat{H}}(\theta)\right)
&=\left(\prod_{h=1}^{t}X_{t-h+1}\left(r_{\widehat{H}}(\theta)\right)\right)\left(\psi_{1|0}-\psi^{\star}_{1|0}\right)\\
&\quad+\sum_{s=1}^{t}\left(\prod_{h=s+1}^{t}X_{h}\left(r_{\widehat{H}}(\theta)\right)\right)Y_{s}\left(r_{\widehat{H}}(\theta)\right)\left(\widehat{F}_{s}-\widehat{H}F_{s}\right)\\
&\quad+\text{second-order terms}, \tag*{(83)}
\end{aligned} ψ t + 1∣ t ( r H ( θ ) ) − ψ t + 1∣ t ( r H ( θ ) ) = ( h = 1 ∏ t X t − h + 1 ( r H ( θ ) ) ) ( ψ 1∣0 − ψ 1∣0 ⋆ ) + s = 1 ∑ t ( h = s + 1 ∏ t X h ( r H ( θ ) ) ) Y s ( r H ( θ ) ) ( F s − H F s ) + second-order terms , (83)
where all products are ordered with the largest time index on the left, and any product whose lower limit exceeds its upper limit is equal to I r \mathbf{I}_{r} I r . By Assumptions 7 and 9, and the geometric summability of the products of X t X_{t} X t , the contribution of the second-order terms in (83) to the average likelihood is O P ( N − 1 + T − 1 ) O_{\mathbb{P}}\left(N^{-1}+T^{-1}\right) O P ( N − 1 + T − 1 ) . Moreover, the contribution of the first term on the right-hand side of (83) is O P ( T − 1 ) O_{\mathbb{P}}\left(T^{-1}\right) O P ( T − 1 ) . Consequently, by substituting (83) into the first sum in (76) , we obtain
1 T ∑ t = 1 T ( ℓ ^ F ( F ^ t ; r H ^ ( θ ) ) − ℓ ^ F ( H ^ F t ; r H ^ ( θ ) ) ) = 1 T ∑ t = 1 T ∇ ψ ℓ ^ F ( H ^ F t ; r H ^ ( θ ) ) T ∑ s = 1 t − 1 ( ∏ h = s + 1 t − 1 X h ( r H ^ ( θ ) ) ) Y s ( r H ^ ( θ ) ) ( F ^ s − H ^ F s ) + 1 T ∑ t = 1 T ∇ F ℓ ^ F ( H ^ F t ; r H ^ ( θ ) ) T ( F ^ t − H ^ F t ) + O P ( 1 N + 1 T ) . (84) \begin{aligned}
\frac{1}{T}\sum_{t=1}^{T}\left(\widehat{\ell}_{F}\left(\widehat{F}_{t};r_{\widehat{H}}(\theta)\right)-\widehat{\ell}_{F}\left(\widehat{H}F_{t};r_{\widehat{H}}(\theta)\right)\right)
={}&\frac{1}{T}\sum_{t=1}^{T}\nabla^{\psi}\widehat{\ell}_{F}\left(\widehat{H}F_{t};r_{\widehat{H}}(\theta)\right)^{\mathsf{T}}\sum_{s=1}^{t-1}\left(\prod_{h=s+1}^{t-1}X_{h}\left(r_{\widehat{H}}(\theta)\right)\right)Y_{s}\left(r_{\widehat{H}}(\theta)\right)\left(\widehat{F}_{s}-\widehat{H}F_{s}\right)\\
&+\frac{1}{T}\sum_{t=1}^{T}\nabla^{F}\widehat{\ell}_{F}\left(\widehat{H}F_{t};r_{\widehat{H}}(\theta)\right)^{\mathsf{T}}\left(\widehat{F}_{t}-\widehat{H}F_{t}\right)+O_{\mathbb{P}}\left(\frac{1}{N}+\frac{1}{T}\right). \tag*{(84)}
\end{aligned} T 1 t = 1 ∑ T ( ℓ F ( F t ; r H ( θ ) ) − ℓ F ( H F t ; r H ( θ ) ) ) = T 1 t = 1 ∑ T ∇ ψ ℓ F ( H F t ; r H ( θ ) ) T s = 1 ∑ t − 1 ( h = s + 1 ∏ t − 1 X h ( r H ( θ ) ) ) Y s ( r H ( θ ) ) ( F s − H F s ) + T 1 t = 1 ∑ T ∇ F ℓ F ( H F t ; r H ( θ ) ) T ( F t − H F t ) + O P ( N 1 + T 1 ) . (84)
Since ∑ t = 1 T ∑ s = 1 t − 1 = ∑ s = 1 T − 1 ∑ t = s + 1 T \sum_{t=1}^{T}\sum_{s=1}^{t-1}=\sum_{s=1}^{T-1}\sum_{t=s+1}^{T} ∑ t = 1 T ∑ s = 1 t − 1 = ∑ s = 1 T − 1 ∑ t = s + 1 T , equation (84) can be rewritten as
1 T ∑ t = 1 T ( ℓ ^ F ( F ^ t ; r H ^ ( θ ) ) − ℓ ^ F ( H ^ F t ; r H ^ ( θ ) ) ) = 1 T ∑ s = 1 T − 1 ( ∇ F ℓ F ( H ^ F s ; r H ^ ( θ ) ) + Y s ( r H ^ ( θ ) ) T ∑ t = s + 1 T ( ∏ h = s + 1 t − 1 X h ( r H ^ ( θ ) ) ) T ∇ ψ ℓ ^ F ( H ^ F t ; r H ^ ( θ ) ) ) T × ( F ^ s − H ^ F s ) + O P ( 1 N + 1 T ) . (85) \begin{aligned}
\frac{1}{T}\sum_{t=1}^{T}\left(\widehat{\ell}_{F}\left(\widehat{F}_{t};r_{\widehat{H}}(\theta)\right)-\widehat{\ell}_{F}\left(\widehat{H}F_{t};r_{\widehat{H}}(\theta)\right)\right)
={}&\frac{1}{T}\sum_{s=1}^{T-1}\left(\nabla^{F}\ell_{F}\left(\widehat{H}F_{s};r_{\widehat{H}}(\theta)\right)+Y_{s}\left(r_{\widehat{H}}(\theta)\right)^{\mathsf{T}}\sum_{t=s+1}^{T}\left(\prod_{h=s+1}^{t-1}X_{h}\left(r_{\widehat{H}}(\theta)\right)\right)^{\mathsf{T}}\nabla^{\psi}\widehat{\ell}_{F}\left(\widehat{H}F_{t};r_{\widehat{H}}(\theta)\right)\right)^{\mathsf{T}}\\
&\qquad\times\left(\widehat{F}_{s}-\widehat{H}F_{s}\right)+O_{\mathbb{P}}\left(\frac{1}{N}+\frac{1}{T}\right). \tag*{(85)}
\end{aligned} T 1 t = 1 ∑ T ( ℓ F ( F t ; r H ( θ ) ) − ℓ F ( H F t ; r H ( θ ) ) ) = T 1 s = 1 ∑ T − 1 ∇ F ℓ F ( H F s ; r H ( θ ) ) + Y s ( r H ( θ ) ) T t = s + 1 ∑ T ( h = s + 1 ∏ t − 1 X h ( r H ( θ ) ) ) T ∇ ψ ℓ F ( H F t ; r H ( θ ) ) T × ( F s − H F s ) + O P ( N 1 + T 1 ) . (85)
Now, let H ⊂ G L ( r ) \mathcal{H}\subset GL(r) H ⊂ G L ( r ) be a fixed compact neighbourhood of H 0 \boldsymbol{H}_{0} H 0 containing only invertible matrices, with G L ( r ) = { H ∈ R r × r : ∣ H ∣ ≠ 0 } GL(r)=\{\boldsymbol{H}\in\mathbb{R}^{r\times r}:|\boldsymbol{H}|\ne0\} G L ( r ) = { H ∈ R r × r : ∣ H ∣ = 0 } . For each H ∈ H \boldsymbol{H}\in\mathcal{H} H ∈ H , let
G s , T ( θ , H ) = ∇ F ℓ ^ F ( H F s ; r H ( θ ) ) + Y s ( r H ( θ ) ) T ∑ t = s + 1 T ( ∏ h = s + 1 t − 1 X h ( r H ( θ ) ) ) T ∇ ψ ℓ ^ F ( H F t ; r H ( θ ) ) , (86) \mathcal{G}_{s,T}(\theta,\boldsymbol{H})=\nabla^{F}\widehat{\ell}_{F}\left(\boldsymbol{H}F_{s};r_{\boldsymbol{H}}(\theta)\right)
+Y_{s}\left(r_{\boldsymbol{H}}(\theta)\right)^{\mathsf{T}}\sum_{t=s+1}^{T}\left(\prod_{h=s+1}^{t-1}X_{h}\left(r_{\boldsymbol{H}}(\theta)\right)\right)^{\mathsf{T}}\nabla^{\psi}\widehat{\ell}_{F}\left(\boldsymbol{H}F_{t};r_{\boldsymbol{H}}(\theta)\right), \tag*{(86)} G s , T ( θ , H ) = ∇ F ℓ F ( H F s ; r H ( θ ) ) + Y s ( r H ( θ ) ) T t = s + 1 ∑ T ( h = s + 1 ∏ t − 1 X h ( r H ( θ ) ) ) T ∇ ψ ℓ F ( H F t ; r H ( θ ) ) , (86)
and note that, by Assumptions 7 and 9, together with the geometric summability of the products of X t X_{t} X t , it holds that
sup T ≥ 1 sup 1 ≤ s ≤ T E [ sup θ ∈ Θ sup H ∈ H ∥ G s , T ( θ , H ) ∥ 2 ] < ∞ . (87) \sup_{T\ge1}\sup_{1\le s\le T}\mathbb{E}\left[\sup_{\theta\in\Theta}\sup_{\boldsymbol{H}\in\mathcal{H}}\left\|\mathcal{G}_{s,T}(\theta,\boldsymbol{H})\right\|^{2}\right]<\infty. \tag*{(87)} T ≥ 1 sup 1 ≤ s ≤ T sup E [ θ ∈ Θ sup H ∈ H sup ∥ G s , T ( θ , H ) ∥ 2 ] < ∞. (87)
The same smoothness conditions and the compactness of Θ \Theta Θ and H \mathcal{H} H imply that G s , T ( θ , H ) \mathcal{G}_{s,T}(\theta,\boldsymbol{H}) G s , T ( θ , H ) is locally Lipschitz in θ \theta θ and H \boldsymbol{H} H , uniformly over Θ × H \Theta\times\mathcal{H} Θ × H , with a random Lipschitz coefficient having a bounded second moment. Moreover, (87) , the fact that P ( H ^ ∈ H ) ⟶ 1 \mathbb{P}\left(\widehat{H}\in\mathcal{H}\right)\longrightarrow1 P ( H ∈ H ) ⟶ 1 , and Markov's inequality imply
1 T ∑ s = 1 T sup θ ∈ Θ ∥ G s , T ( θ , H ^ ) ∥ 2 = O P ( 1 ) . (88) \frac{1}{T}\sum_{s=1}^{T}\sup_{\theta\in\Theta}\left\|\mathcal{G}_{s,T}\left(\theta,\widehat{H}\right)\right\|^{2}=O_{\mathbb{P}}(1). \tag*{(88)} T 1 s = 1 ∑ T θ ∈ Θ sup G s , T ( θ , H ) 2 = O P ( 1 ) . (88)
By substituting (86) into (85) and using the decomposition in (42) we get
1 T ∑ t = 1 T ( ℓ ^ F ( F ^ t ; r H ^ ( θ ) ) − ℓ ^ F ( H ^ F ^ t ; r H ^ ( θ ) ) ) = 1 T ∑ s = 1 T G s , T ( θ , H ^ ) T ( M ^ N ) − 1 ( A s + B s + C s + D s ) + O P ( 1 N + 1 T ) , (89) \frac{1}{T}\sum_{t=1}^{T}\left(\widehat{\ell}_{F}\left(\widehat{F}_{t};r_{\widehat{H}}(\theta)\right)-\widehat{\ell}_{F}\left(\widehat{H}\widehat{F}_{t};r_{\widehat{H}}(\theta)\right)\right)
= \frac{1}{T}\sum_{s=1}^{T}\mathcal{G}_{s,T}(\theta,\widehat{H})^{\mathsf{T}}\left(\frac{\widehat{M}}{N}\right)^{-1}(A_s+B_s+C_s+D_s)+O_{\mathbb{P}}\left(\frac{1}{N}+\frac{1}{T}\right), \tag*{(89)} T 1 t = 1 ∑ T ( ℓ F ( F t ; r H ( θ ) ) − ℓ F ( H F t ; r H ( θ ) ) ) = T 1 s = 1 ∑ T G s , T ( θ , H ) T ( N M ) − 1 ( A s + B s + C s + D s ) + O P ( N 1 + T 1 ) , (89)
uniformly over θ ∈ Θ \theta\in\Theta θ ∈ Θ , where
A s = ( 1 T ∑ u = 1 T F ^ u F ^ u T ) ( Λ ^ 0 T e s N ) , (90) A_s=\left(\frac{1}{T}\sum_{u=1}^{T}\widehat{F}_u\widehat{F}_u^{\mathsf{T}}\right)\left(\frac{\widehat{\Lambda}_0^{\mathsf{T}}e_s}{N}\right), \tag*{(90)} A s = ( T 1 u = 1 ∑ T F u F u T ) ( N Λ 0 T e s ) , (90)
B s = ( 1 N T ∑ u = 1 T F ^ u e u T Λ ^ 0 ) F s , (91) B_s=\left(\frac{1}{NT}\sum_{u=1}^{T}\widehat{F}_u e_u^{\mathsf{T}}\widehat{\Lambda}_0\right)F_s, \tag*{(91)} B s = ( N T 1 u = 1 ∑ T F u e u T Λ 0 ) F s , (91)
C s = 1 N T ∑ u = 1 T F ^ u E [ e u T e s ] , (92) C_s=\frac{1}{NT}\sum_{u=1}^{T}\widehat{F}_u\mathbb{E}\left[e_u^{\mathsf{T}}e_s\right], \tag*{(92)} C s = N T 1 u = 1 ∑ T F u E [ e u T e s ] , (92)
D s = 1 N T ∑ u = 1 T F ^ u ( e u T e s − E [ e u T e s ] ) . (93) D_s=\frac{1}{NT}\sum_{u=1}^{T}\widehat{F}_u\left(e_u^{\mathsf{T}}e_s-\mathbb{E}\left[e_u^{\mathsf{T}}e_s\right]\right). \tag*{(93)} D s = N T 1 u = 1 ∑ T F u ( e u T e s − E [ e u T e s ] ) . (93)
By Lemma A.3 of Bai [2 ] ,
∥ ( M ^ N ) − 1 ∥ = O P ( 1 ) . (94) \left\|\left(\frac{\widehat{M}}{N}\right)^{-1}\right\|=O_{\mathbb{P}}(1). \tag*{(94)} ( N M ) − 1 = O P ( 1 ) . (94)
We now control separately the four terms on the right-hand side of (89) .
Contribution of A s A_s A s . For every nonrandom H ∈ H H \in\mathcal{H} H ∈ H , the process G s , T ( θ , H ) \mathcal{G}_{s,T}(\theta,H) G s , T ( θ , H ) is measurable with respect to the factor process and is therefore independent of the idiosyncratic process. Moreover, Assumptions 3 and 4 imply that, for every s , u ∈ Z s,u \in\mathbb{Z} s , u ∈ Z ,
∥ E [ Λ ^ 0 T e s e u T Λ ^ 0 N ] ∥ ≤ C N ρ ∣ s − u ∣ , (95) \left\|\mathbb{E}\left[\frac{\widehat{\Lambda}_0^{\mathsf{T}}e_s e_u^{\mathsf{T}}\widehat{\Lambda}_0}{N}\right]\right\|\leq\frac{C}{N}\rho^{|s-u|}, \tag*{(95)} E [ N Λ 0 T e s e u T Λ 0 ] ≤ N C ρ ∣ s − u ∣ , (95)
for some finite positive real C C C independent of N N N , s s s , and u u u . Since Θ \Theta Θ is compact and G s , T ( θ , H 0 ) \mathcal{G}_{s,T}(\theta,H_0) G s , T ( θ , H 0 ) is uniformly Lipschitz in θ \theta θ with a square-integrable envelope, the standard L 2 L_2 L 2 maximal inequality for finite-dimensional Lipschitz classes, applied conditionally on the factor process, together with (87) , (95) , and Assumption 5 ( i ) 5(i) 5 ( i ) implies
E [ sup θ ∈ Θ ∥ 1 T ∑ s = 1 T Λ ^ 0 T e s N G s , T ( θ , H 0 ) ∥ 2 ] ≤ C N T 2 ∑ s = 1 T ∑ u = 1 T ρ ∣ s − u ∣ ≤ C N T . \mathbb{E}\left[\sup_{\theta\in\Theta}\left\|\frac{1}{T}\sum_{s=1}^{T}\frac{\widehat{\Lambda}_0^{\mathsf{T}}e_s}{N}\mathcal{G}_{s,T}(\theta,H_0)\right\|^2\right]\leq\frac{C}{NT^2}\sum_{s=1}^{T}\sum_{u=1}^{T}\rho^{|s-u|}\leq\frac{C}{NT}. E θ ∈ Θ sup T 1 s = 1 ∑ T N Λ 0 T e s G s , T ( θ , H 0 ) 2 ≤ N T 2 C s = 1 ∑ T u = 1 ∑ T ρ ∣ s − u ∣ ≤ N T C .
and therefore, by Markov’s inequality,
sup θ ∈ Θ ∥ 1 T ∑ s = 1 T Λ 0 T N G s , T ( θ , H 0 ) T ∥ = O P ( 1 N T ) . (96) \sup_{\theta\in\Theta}\left\Vert\frac{1}{T}\sum_{s=1}^{T}\frac{\Lambda_0^{\mathsf{T}}}{N}\mathcal{G}_{s,T}(\theta,\mathbf{H}_0)^{\mathsf{T}}\right\Vert= O_{\mathbb{P}}\left(\frac{1}{\sqrt{NT}}\right). \tag*{(96)} θ ∈ Θ sup T 1 s = 1 ∑ T N Λ 0 T G s , T ( θ , H 0 ) T = O P ( N T 1 ) . (96)
To replace H 0 \mathbf{H}_0 H 0 with H ^ \widehat{\mathbf{H}} H , the Cauchy–Schwarz inequality and the uniform local Lipschitz property stated above yield
sup θ ∈ Θ ∥ 1 T ∑ s = 1 T Λ 0 T e s N ( G s , T ( θ , H ^ ) − G s , T ( θ , H 0 ) ) T ∥ ≤ ( 1 T ∑ s = 1 T ∥ Λ 0 T e s N ∥ 2 ) 1 / 2 ( 1 T ∑ s = 1 T sup θ ∈ Θ ∥ G s , T ( θ , H ^ ) − G s , T ( θ , H 0 ) ∥ 2 ) 1 / 2 = O P ( 1 N ) O P ( ∥ H ^ − H 0 ∥ ) = O P ( 1 N + 1 N T ) . \begin{aligned}
\sup_{\theta\in\Theta}\left\Vert\frac{1}{T}\sum_{s=1}^{T}\frac{\Lambda_0^{\mathsf{T}}e_s}{N}\left(\mathcal{G}_{s,T}(\theta,\widehat{\mathbf{H}})-\mathcal{G}_{s,T}(\theta,\mathbf{H}_0)\right)^{\mathsf{T}}\right\Vert
&\leq\left(\frac{1}{T}\sum_{s=1}^{T}\left\Vert\frac{\Lambda_0^{\mathsf{T}}e_s}{N}\right\Vert^2\right)^{1/2}\left(\frac{1}{T}\sum_{s=1}^{T}\sup_{\theta\in\Theta}\left\Vert\mathcal{G}_{s,T}(\theta,\widehat{\mathbf{H}})-\mathcal{G}_{s,T}(\theta,\mathbf{H}_0)\right\Vert^2\right)^{1/2}\\
&=O_{\mathbb{P}}\left(\frac{1}{\sqrt{N}}\right)O_{\mathbb{P}}\left(\left\Vert\widehat{\mathbf{H}}-\mathbf{H}_0\right\Vert\right)=O_{\mathbb{P}}\left(\frac{1}{N}+\frac{1}{\sqrt{NT}}\right).
\end{aligned} θ ∈ Θ sup T 1 s = 1 ∑ T N Λ 0 T e s ( G s , T ( θ , H ) − G s , T ( θ , H 0 ) ) T ≤ ( T 1 s = 1 ∑ T N Λ 0 T e s 2 ) 1/2 ( T 1 s = 1 ∑ T θ ∈ Θ sup G s , T ( θ , H ) − G s , T ( θ , H 0 ) 2 ) 1/2 = O P ( N 1 ) O P ( H − H 0 ) = O P ( N 1 + N T 1 ) .
where the last equality follows from (51) . Combining this bound with (96) we get
sup θ ∈ Θ ∥ 1 T ∑ s = 1 T Λ 0 T e s N G s , T ( θ , H ^ ) T ∥ = O P ( 1 N T + 1 N ) . (97) \sup_{\theta\in\Theta}\left\Vert\frac{1}{T}\sum_{s=1}^{T}\frac{\Lambda_0^{\mathsf{T}}e_s}{N}\mathcal{G}_{s,T}(\theta,\widehat{\mathbf{H}})^{\mathsf{T}}\right\Vert=O_{\mathbb{P}}\left(\frac{1}{\sqrt{NT}}+\frac{1}{N}\right). \tag*{(97)} θ ∈ Θ sup T 1 s = 1 ∑ T N Λ 0 T e s G s , T ( θ , H ) T = O P ( N T 1 + N 1 ) . (97)
Additionally, by the matrix Cauchy–Schwarz inequality,
∥ 1 T ∑ u = 1 T F ^ u F u T ∥ ≤ ( ∥ 1 T ∑ u = 1 T F ^ u F ^ u T ∥ ∥ 1 T ∑ u = 1 T F u F u T ∥ ) 1 / 2 , \left\Vert\frac{1}{T}\sum_{u=1}^{T}\widehat{F}_uF_u^{\mathsf{T}}\right\Vert\leq\left(\left\Vert\frac{1}{T}\sum_{u=1}^{T}\widehat{F}_u\widehat{F}_u^{\mathsf{T}}\right\Vert\left\Vert\frac{1}{T}\sum_{u=1}^{T}F_uF_u^{\mathsf{T}}\right\Vert\right)^{1/2}, T 1 u = 1 ∑ T F u F u T ≤ ( T 1 u = 1 ∑ T F u F u T T 1 u = 1 ∑ T F u F u T ) 1/2 ,
and using the equivalence in (37) , 1 T ∑ s = 1 T F ^ s F ^ s T = M ^ N \frac{1}{T}\sum_{s=1}^{T}\widehat{F}_s\widehat{F}_s^{\mathsf{T}}=\frac{\widehat{M}}{N} T 1 ∑ s = 1 T F s F s T = N M , where ∥ M ^ N ∥ = O P ( 1 ) \left\Vert\frac{\widehat{M}}{N}\right\Vert=O_{\mathbb{P}}(1) N M = O P ( 1 ) . Under Assumption 2, we also know that ∥ 1 T ∑ s = 1 T F s F s T ∥ = O P ( 1 ) \left\Vert\frac{1}{T}\sum_{s=1}^{T}F_sF_s^{\mathsf{T}}\right\Vert=O_{\mathbb{P}}(1) T 1 ∑ s = 1 T F s F s T = O P ( 1 ) , and hence,
∥ 1 T ∑ u = 1 T F ^ u F u T ∥ = O P ( 1 ) . (98) \left\Vert\frac{1}{T}\sum_{u=1}^{T}\widehat{F}_uF_u^{\mathsf{T}}\right\Vert=O_{\mathbb{P}}(1). \tag*{(98)} T 1 u = 1 ∑ T F u F u T = O P ( 1 ) . (98)
Consequently, using the trace inequality, (98) , and (97) , the contribution of A s A_s A s to (89) is
sup θ ∈ Θ ∣ 1 T ∑ s = 1 T G s , T ( θ , H ^ ) T ( M ^ N ) − 1 A s ∣ = O P ( 1 N T + 1 N ) . (99) \sup_{\theta\in\Theta}\left\vert\frac{1}{T}\sum_{s=1}^{T}\mathcal{G}_{s,T}(\theta,\widehat{\mathbf{H}})^{\mathsf{T}}\left(\frac{\widehat{M}}{N}\right)^{-1}A_s\right\vert=O_{\mathbb{P}}\left(\frac{1}{\sqrt{NT}}+\frac{1}{N}\right). \tag*{(99)} θ ∈ Θ sup T 1 s = 1 ∑ T G s , T ( θ , H ) T ( N M ) − 1 A s = O P ( N T 1 + N 1 ) . (99)
Contribution of B s B_s B s . We have
1 N T ∑ u = 1 T F u e u ⊤ Λ 0 = H ^ ( 1 N T ∑ u = 1 T F u e u ⊤ Λ 0 ) + 1 T ∑ u = 1 T ( F ^ u − H ^ F u ) e u ⊤ Λ 0 N . (100) \frac{1}{NT}\sum_{u=1}^{T}F_u e_u^{\top}\Lambda_0=\widehat{H}\left(\frac{1}{NT}\sum_{u=1}^{T}F_u e_u^{\top}\Lambda_0\right)+\frac{1}{T}\sum_{u=1}^{T}\left(\widehat{F}_u-\widehat{H}F_u\right)\frac{e_u^{\top}\Lambda_0}{N}. \tag*{(100)} N T 1 u = 1 ∑ T F u e u ⊤ Λ 0 = H ( N T 1 u = 1 ∑ T F u e u ⊤ Λ 0 ) + T 1 u = 1 ∑ T ( F u − H F u ) N e u ⊤ Λ 0 . (100)
Assumption 5(iii) implies that
∥ 1 N T ∑ u = 1 T F u e u ⊤ Λ 0 ∥ = O P ( 1 N T ) . (101) \left\Vert\frac{1}{NT}\sum_{u=1}^{T}F_u e_u^{\top}\Lambda_0\right\Vert=O_{\mathrm{P}}\left(\frac{1}{\sqrt{NT}}\right). \tag*{(101)} N T 1 u = 1 ∑ T F u e u ⊤ Λ 0 = O P ( N T 1 ) . (101)
For the second term in (100) , using the matrix Cauchy–Schwarz inequality we get
∥ 1 T ∑ u = 1 T ( F ^ u − H ^ F u ) e u ⊤ Λ 0 N ∥ ≤ ( 1 T ∑ u = 1 T ∥ F ^ u − H ^ F u ∥ 2 ) 1 / 2 ( 1 T ∑ u = 1 T ∥ Λ 0 ⊤ e u N ∥ 2 ) 1 / 2 . \left\Vert\frac{1}{T}\sum_{u=1}^{T}\left(\widehat{F}_u-\widehat{H}F_u\right)\frac{e_u^{\top}\Lambda_0}{N}\right\Vert\leq\left(\frac{1}{T}\sum_{u=1}^{T}\left\Vert\widehat{F}_u-\widehat{H}F_u\right\Vert^2\right)^{1/2}\left(\frac{1}{T}\sum_{u=1}^{T}\left\Vert\frac{\Lambda_0^{\top}e_u}{N}\right\Vert^2\right)^{1/2}. T 1 u = 1 ∑ T ( F u − H F u ) N e u ⊤ Λ 0 ≤ ( T 1 u = 1 ∑ T F u − H F u 2 ) 1/2 ( T 1 u = 1 ∑ T N Λ 0 ⊤ e u 2 ) 1/2 .
Lemma A.1 of Bai [2 ] and (95) imply, respectively,
( 1 T ∑ u = 1 T ∥ F ^ u − H ^ F u ∥ 2 ) 1 / 2 = O P ( 1 N + 1 T ) , \left(\frac{1}{T}\sum_{u=1}^{T}\left\Vert\widehat{F}_u-\widehat{H}F_u\right\Vert^2\right)^{1/2}=O_{\mathrm{P}}\left(\frac{1}{\sqrt{N}}+\frac{1}{\sqrt{T}}\right), ( T 1 u = 1 ∑ T F u − H F u 2 ) 1/2 = O P ( N 1 + T 1 ) ,
( 1 T ∑ u = 1 T ∥ Λ 0 ⊤ e u N ∥ 2 ) 1 / 2 = O P ( 1 N ) . \left(\frac{1}{T}\sum_{u=1}^{T}\left\Vert\frac{\Lambda_0^{\top}e_u}{N}\right\Vert^2\right)^{1/2}=O_{\mathrm{P}}\left(\frac{1}{\sqrt{N}}\right). ( T 1 u = 1 ∑ T N Λ 0 ⊤ e u 2 ) 1/2 = O P ( N 1 ) .
It follows that
∥ 1 N T ∑ u = 1 T F ^ u e u ⊤ Λ 0 ∥ = O P ( 1 N T + 1 N ) . (102) \left\Vert\frac{1}{NT}\sum_{u=1}^{T}\widehat{F}_u e_u^{\top}\Lambda_0\right\Vert=O_{\mathrm{P}}\left(\frac{1}{\sqrt{NT}}+\frac{1}{N}\right). \tag*{(102)} N T 1 u = 1 ∑ T F u e u ⊤ Λ 0 = O P ( N T 1 + N 1 ) . (102)
Furthermore, by (88) and the fact that, under Assumption 2, 1 T ∑ s = 1 T ∥ F s ∥ 2 = O P ( 1 ) \frac{1}{T}\sum_{s=1}^{T}\Vert F_s\Vert^2=O_{\mathrm{P}}(1) T 1 ∑ s = 1 T ∥ F s ∥ 2 = O P ( 1 ) , we obtain
1 T ∑ s = 1 T sup θ ∈ Θ ∥ g s , T ( θ , H ^ ) ∥ ∥ F s ∥ ≤ ( 1 T ∑ s = 1 T sup θ ∈ Θ ∥ g s , T ( θ , H ^ ) ∥ 2 ) 1 / 2 ( 1 T ∑ s = 1 T ∥ F s ∥ 2 ) 1 / 2 = O P ( 1 ) . \begin{aligned}
\frac{1}{T}\sum_{s=1}^{T}\sup_{\theta\in\Theta}\left\Vert g_{s,T}\left(\theta,\widehat{H}\right)\right\Vert\Vert F_s\Vert\\
&\leq\left(\frac{1}{T}\sum_{s=1}^{T}\sup_{\theta\in\Theta}\left\Vert g_{s,T}\left(\theta,\widehat{H}\right)\right\Vert^2\right)^{1/2}\left(\frac{1}{T}\sum_{s=1}^{T}\Vert F_s\Vert^2\right)^{1/2}=O_{\mathrm{P}}(1).
\end{aligned} T 1 s = 1 ∑ T θ ∈ Θ sup g s , T ( θ , H ) ∥ F s ∥ ≤ ( T 1 s = 1 ∑ T θ ∈ Θ sup g s , T ( θ , H ) 2 ) 1/2 ( T 1 s = 1 ∑ T ∥ F s ∥ 2 ) 1/2 = O P ( 1 ) .
Consequently, the contribution of B s B_s B s is
sup θ ∈ Θ ∣ 1 T ∑ s = 1 T g s , T ( θ , H ^ ) ⊤ ( M ^ N ) − 1 B s ∣ = O P ( 1 N T + 1 N ) . (103) \sup_{\theta\in\Theta}\left\vert\frac{1}{T}\sum_{s=1}^{T}g_{s,T}\left(\theta,\widehat{H}\right)^{\top}\left(\frac{\widehat{M}}{N}\right)^{-1}B_s\right\vert=O_{\mathrm{P}}\left(\frac{1}{\sqrt{NT}}+\frac{1}{N}\right). \tag*{(103)} θ ∈ Θ sup T 1 s = 1 ∑ T g s , T ( θ , H ) ⊤ ( N M ) − 1 B s = O P ( N T 1 + N 1 ) . (103)
Contribution of C s C_s C s . By Assumption 4(ii),
∣ E [ e u T e s ] ∣ ≤ N C e ′ ρ ∣ u − s ∣ . \left\lvert\mathbb{E}\left[e_u^{\mathsf{T}}e_s\right]\right\rvert\le NC'_e\rho^{\lvert u-s\rvert}. E [ e u T e s ] ≤ N C e ′ ρ ∣ u − s ∣ .
Therefore,
∥ C s ∥ ≤ C e ′ T ∑ u = 1 T ρ ∣ u − s ∣ ∥ F ^ u ∥ . \left\lVert C_s\right\rVert\le\frac{C'_e}{T}\sum_{u=1}^{T}\rho^{\lvert u-s\rvert}\left\lVert\widehat{F}_u\right\rVert. ∥ C s ∥ ≤ T C e ′ u = 1 ∑ T ρ ∣ u − s ∣ F u .
By the discrete convolution inequality and the fact that T − 1 ∑ u = 1 T ∥ F ^ u ∥ 2 = O P ( 1 ) T^{-1}\sum_{u=1}^{T}\left\lVert\widehat{F}_u\right\rVert^2=O_{\mathrm{P}}(1) T − 1 ∑ u = 1 T F u 2 = O P ( 1 ) , we obtain
( 1 T ∑ s = 1 T ∥ C s ∥ 2 ) 1 / 2 ≤ C e ′ T ∑ h = − ∞ ∞ ρ ∣ h ∣ ( 1 T ∑ u = 1 T ∥ F ^ u ∥ 2 ) 1 / 2 = O P ( 1 T ) . (104) \left(\frac{1}{T}\sum_{s=1}^{T}\left\lVert C_s\right\rVert^2\right)^{1/2}\le\frac{C'_e}{T}\sum_{h=-\infty}^{\infty}\rho^{\lvert h\rvert}\left(\frac{1}{T}\sum_{u=1}^{T}\left\lVert\widehat{F}_u\right\rVert^2\right)^{1/2}=O_{\mathrm{P}}\left(\frac{1}{T}\right). \tag*{(104)} ( T 1 s = 1 ∑ T ∥ C s ∥ 2 ) 1/2 ≤ T C e ′ h = − ∞ ∑ ∞ ρ ∣ h ∣ ( T 1 u = 1 ∑ T F u 2 ) 1/2 = O P ( T 1 ) . (104)
Combining (88) and (104) with the Cauchy–Schwarz inequality shows that the contribution of C s C_s C s is
sup θ ∈ Θ ∣ 1 T ∑ s = 1 T g s , T ( θ , H ^ ) T ( M ^ N ^ ) − 1 C s ∣ = O P ( 1 T ) . (105) \sup_{\theta\in\Theta}\left\lvert\frac{1}{T}\sum_{s=1}^{T}g_{s,T}(\theta,\widehat{H})^{\mathsf{T}}\left(\frac{\widehat{M}}{\widehat{N}}\right)^{-1}C_s\right\rvert=O_{\mathrm{P}}\left(\frac{1}{T}\right). \tag*{(105)} θ ∈ Θ sup T 1 s = 1 ∑ T g s , T ( θ , H ) T ( N M ) − 1 C s = O P ( T 1 ) . (105)
Contribution of D s D_s D s . From (93) ,
D s = H ^ N T ∑ u = 1 T F u ( e u T e s − E [ e u T e s ] ) + 1 T ∑ u = 1 T ( F ^ u − H ^ F u ) e u T e s − E [ e u T e s ] N . (106) D_s=\frac{\widehat{H}}{NT}\sum_{u=1}^{T}F_u\left(e_u^{\mathsf{T}}e_s-\mathbb{E}\left[e_u^{\mathsf{T}}e_s\right]\right)+\frac{1}{T}\sum_{u=1}^{T}\left(\widehat{F}_u-\widehat{H}F_u\right)\frac{e_u^{\mathsf{T}}e_s-\mathbb{E}\left[e_u^{\mathsf{T}}e_s\right]}{N}. \tag*{(106)} D s = N T H u = 1 ∑ T F u ( e u T e s − E [ e u T e s ] ) + T 1 u = 1 ∑ T ( F u − H F u ) N e u T e s − E [ e u T e s ] . (106)
Assumption 5(i) and ∥ H ^ ∥ = O P ( 1 ) \left\lVert\widehat{H}\right\rVert=O_{\mathrm{P}}(1) H = O P ( 1 ) yield
( 1 T ∑ s = 1 T ∥ H ^ N T ∑ u = 1 T F u ( e u T e s − E [ e u T e s ] ) ∥ 2 ) 1 / 2 = O P ( 1 N T ) . (107) \left(\frac{1}{T}\sum_{s=1}^{T}\left\lVert\frac{\widehat{H}}{NT}\sum_{u=1}^{T}F_u\left(e_u^{\mathsf{T}}e_s-\mathbb{E}\left[e_u^{\mathsf{T}}e_s\right]\right)\right\rVert^2\right)^{1/2}=O_{\mathrm{P}}\left(\frac{1}{\sqrt{NT}}\right). \tag*{(107)} T 1 s = 1 ∑ T N T H u = 1 ∑ T F u ( e u T e s − E [ e u T e s ] ) 2 1/2 = O P ( N T 1 ) . (107)
For the second term in (106) , Cauchy–Schwarz gives, for every s s s ,
∥ 1 T ∑ u = 1 T ( F ^ u − H ^ F u ) e u T e s − E [ e u T e s ] N ∥ 2 ≤ ( 1 T ∑ u = 1 T ∥ F ^ u − H ^ F u ∥ 2 ) ( 1 T ∑ u = 1 T ∣ e u T e s − E [ e u T e s ] ∣ 2 N 2 ) . \begin{aligned}
\left\lVert\frac{1}{T}\sum_{u=1}^{T}\left(\widehat{F}_u-\widehat{H}F_u\right)\frac{e_u^{\mathsf{T}}e_s-\mathbb{E}\left[e_u^{\mathsf{T}}e_s\right]}{N}\right\rVert^2 \\
&\le\left(\frac{1}{T}\sum_{u=1}^{T}\left\lVert\widehat{F}_u-\widehat{H}F_u\right\rVert^2\right)\left(\frac{1}{T}\sum_{u=1}^{T}\frac{\left\lvert e_u^{\mathsf{T}}e_s-\mathbb{E}\left[e_u^{\mathsf{T}}e_s\right]\right\rvert^2}{N^2}\right).
\end{aligned} T 1 u = 1 ∑ T ( F u − H F u ) N e u T e s − E [ e u T e s ] 2 ≤ ( T 1 u = 1 ∑ T F u − H F u 2 ) ( T 1 u = 1 ∑ T N 2 e u T e s − E [ e u T e s ] 2 ) .
By Assumption 4(v) and Markov’s inequality,
1 T 2 ∑ s = 1 T ∑ u = 1 T [ e u T e s − E [ e u T e s ] ] 2 N 2 = O P ( 1 N ) . (108) \frac{1}{T^{2}}\sum_{s=1}^{T}\sum_{u=1}^{T}\frac{\left[e_{u}^{\mathsf{T}}e_{s}-\mathbb{E}\left[e_{u}^{\mathsf{T}}e_{s}\right]\right]^{2}}{N^{2}}=O_{\mathbb{P}}\left(\frac{1}{N}\right). \tag*{(108)} T 2 1 s = 1 ∑ T u = 1 ∑ T N 2 [ e u T e s − E [ e u T e s ] ] 2 = O P ( N 1 ) . (108)
It follows that
( 1 T ∑ s = 1 T ∥ 1 T ∑ u = 1 T ( F ^ u − H ^ F u ) e u T e s − E [ e u T e s ] N ∥ 2 ) 1 / 2 = O P ( ( 1 N + 1 T ) 1 N ) = O P ( 1 N + 1 N T ) . (109) \begin{aligned}
\left(\frac{1}{T}\sum_{s=1}^{T}\left\Vert\frac{1}{T}\sum_{u=1}^{T}\left(\widehat{F}_{u}-\widehat{H}F_{u}\right)\frac{e_{u}^{\mathsf{T}}e_{s}-\mathbb{E}\left[e_{u}^{\mathsf{T}}e_{s}\right]}{N}\right\Vert^{2}\right)^{1/2} \\
&=O_{\mathbb{P}}\left(\left(\frac{1}{\sqrt{N}}+\frac{1}{\sqrt{T}}\right)\frac{1}{\sqrt{N}}\right)=O_{\mathbb{P}}\left(\frac{1}{N}+\frac{1}{\sqrt{NT}}\right). \tag*{(109)}
\end{aligned} T 1 s = 1 ∑ T T 1 u = 1 ∑ T ( F u − H F u ) N e u T e s − E [ e u T e s ] 2 1/2 = O P ( ( N 1 + T 1 ) N 1 ) = O P ( N 1 + N T 1 ) . (109)
Combining (88) , (107) , and (109) with the Cauchy–Schwarz inequality shows that the contribution of D s D_{s} D s is
sup θ ∈ Θ ∣ 1 T ∑ s = 1 T g s , T ( θ , H ^ ) T ( M ^ N ) − 1 D s ∣ = O P ( 1 N T + 1 N ) . (110) \sup_{\theta\in\Theta}\left\vert\frac{1}{T}\sum_{s=1}^{T}\boldsymbol{g}_{s,T}(\theta,\widehat{H})^{\mathsf{T}}\left(\frac{\widehat{M}}{N}\right)^{-1}D_{s}\right\vert=O_{\mathbb{P}}\left(\frac{1}{\sqrt{NT}}+\frac{1}{N}\right). \tag*{(110)} θ ∈ Θ sup T 1 s = 1 ∑ T g s , T ( θ , H ) T ( N M ) − 1 D s = O P ( N T 1 + N 1 ) . (110)
Therefore, by combining (99) , (103) , (105) , and (110) with (89) we finally obtain
sup θ ∈ Θ ∣ 1 T ∑ t = 1 T ( ℓ ^ F ( F ^ t ; r H ^ ( θ ) ) − ℓ ^ F ( H ^ F t ; r H ^ ( θ ) ) ) ∣ = O P ( 1 N T + 1 N + 1 T ) . (111) \sup_{\theta\in\Theta}\left\vert\frac{1}{T}\sum_{t=1}^{T}\left(\widehat{\ell}_{F}\left(\widehat{F}_{t};r_{\widehat{H}}(\theta)\right)-\widehat{\ell}_{F}\left(\widehat{H}F_{t};r_{\widehat{H}}(\theta)\right)\right)\right\vert=O_{\mathbb{P}}\left(\frac{1}{\sqrt{NT}}+\frac{1}{N}+\frac{1}{T}\right). \tag*{(111)} θ ∈ Θ sup T 1 t = 1 ∑ T ( ℓ F ( F t ; r H ( θ ) ) − ℓ F ( H F t ; r H ( θ ) ) ) = O P ( N T 1 + N 1 + T 1 ) . (111)
Controlling the term in (73)
Fix an invertible r × r r\times r r × r matrix H H H . By the definition of r H r_{H} r H , the conditional density of the transformed factors satisfies
f H F ( H F t ∣ F t − 1 ; r H ( θ ) ) = 1 ∣ H ∣ f F ( F t ∣ F t − 1 ; θ ) . (112) f_{HF}\left(HF_{t}\mid F_{t-1};r_{H}(\theta)\right)=\frac{1}{\lvert H\rvert}f_{F}\left(F_{t}\mid F_{t-1};\theta\right). \tag*{(112)} f H F ( H F t ∣ F t − 1 ; r H ( θ ) ) = ∣ H ∣ 1 f F ( F t ∣ F t − 1 ; θ ) . (112)
For the time-varying location model, this follows from (32) ; for the time-varying covariance model, it follows from (33) . Thus, if the two filters are initialized equivariantly, then, for every t ≥ 1 t\geq1 t ≥ 1 ,
ℓ F ( H F t ; r H ( θ ) ) + log ∣ H ∣ = ℓ F ( F t ; θ ) . (113) \ell_{F}\left(HF_{t};r_{H}(\theta)\right)+\log\lvert H\rvert=\ell_{F}\left(F_{t};\theta\right). \tag*{(113)} ℓ F ( H F t ; r H ( θ ) ) + log ∣ H ∣ = ℓ F ( F t ; θ ) . (113)
Summing (113) over t = 1 , … , T t=1,\ldots,T t = 1 , … , T
ℓ F ( H F ; r H ( θ ) ) + T log ∣ H ∣ = ℓ F ( F ; θ ) . (114) \ell_{F}\left(HF;r_{H}(\theta)\right)+T\log\lvert H\rvert=\ell_{F}\left(F;\theta\right). \tag*{(114)} ℓ F ( H F ; r H ( θ ) ) + T log ∣ H ∣ = ℓ F ( F ; θ ) . (114)
It remains to control the difference between ℓ ^ F \widehat{\ell}_{F} ℓ F and ℓ F \ell_{F} ℓ F generated by their possibly different filter initializations. Let { ψ ^ t ∣ t − 1 ( r H ( θ ) ) } t ∈ N \left\{\widehat{\psi}_{t|t-1}\left(r_{H}(\theta)\right)\right\}_{t\in\mathbb{N}} { ψ t ∣ t − 1 ( r H ( θ ) ) } t ∈ N denote the filter used to construct ℓ ^ F ( H F ; r H ( θ ) ) \widehat{\ell}_{F}\left(HF;r_{H}(\theta)\right) ℓ F ( H F ; r H ( θ ) ) , initialized at ψ 1 ∣ 0 ⋆ \psi^{\star}_{1|0} ψ 1∣0 ⋆ , and let { ψ t ∣ t − 1 ( r H ( θ ) ) } t ∈ N \left\{\psi_{t|t-1}\left(r_{H}(\theta)\right)\right\}_{t\in\mathbb{N}} { ψ t ∣ t − 1 ( r H ( θ ) ) } t ∈ N denote the filter driven by the same transformed factors and initialized at the value obtained by applying the transformation associated with H H H to the initial condition of the untransformed filter. By construction, the latter filter is the one entering (113).
For each H ∈ H H\in\mathcal{H} H ∈ H , with H ⊂ G L ( r ) \mathcal{H}\subset GL(r) H ⊂ G L ( r ) as defined above equation (85), the continuity of the transformation associated with r H \mathbf{r}_{H} r H and the compactness of Θ \Theta Θ and H \mathcal{H} H imply
sup H ∈ H sup θ ∈ Θ ∥ ψ 1 ∣ 0 ⋆ − ψ 1 ∣ 0 ( r H ( θ ) ) ∥ ≤ Δ 1 , (115) \sup_{H\in\mathcal{H}}\sup_{\theta\in\Theta}\left\|\psi^{\star}_{1|0}-\psi_{1|0}\left(r_{H}(\theta)\right)\right\|\leq\Delta_{1}, \tag*{(115)} H ∈ H sup θ ∈ Θ sup ψ 1∣0 ⋆ − ψ 1∣0 ( r H ( θ ) ) ≤ Δ 1 , (115)
for some finite positive real Δ 1 \Delta_{1} Δ 1 independent of N N N , T T T , and t t t . Now, following the same approach of the Proof of Proposition 2, by repeated applications of the mean value theorem to the two filters and Assumption 7, there exists a finite positive real C C C , independent of N N N , T T T , t t t , θ \theta θ , and H H H , such that
sup H ∈ H sup θ ∈ Θ ∥ ψ ^ t ∣ t − 1 ( r H ( θ ) ) − ψ t ∣ t − 1 ( r H ( θ ) ) ∥ ≤ C χ k ( t − 1 ) / k Δ 1 , (116) \sup_{H\in\mathcal{H}}\sup_{\theta\in\Theta}\left\|\widehat{\psi}_{t|t-1}\left(r_{H}(\theta)\right)-\psi_{t|t-1}\left(r_{H}(\theta)\right)\right\|\leq C\chi_{k}^{(t-1)/k}\Delta_{1}, \tag*{(116)} H ∈ H sup θ ∈ Θ sup ψ t ∣ t − 1 ( r H ( θ ) ) − ψ t ∣ t − 1 ( r H ( θ ) ) ≤ C χ k ( t − 1 ) / k Δ 1 , (116)
almost surely. An application of the mean value theorem to each likelihood contribution, together with Assumption 9, now implies that
sup H ∈ H sup θ ∈ Θ ∣ 1 T ∑ t = 1 T ℓ ^ F ( H F t ; r H ( θ ) ) − 1 T ∑ t = 1 T ℓ F ( H F t ; r H ( θ ) ) ∣ ≤ C Δ 1 T ∑ t = 1 T D 2 , ψ t ∣ t − 1 χ k ( t − 1 ) / k , (117) \sup_{H\in\mathcal{H}}\sup_{\theta\in\Theta}\left|\frac{1}{T}\sum_{t=1}^{T}\widehat{\ell}_{F}\left(HF_{t};r_{H}(\theta)\right)-\frac{1}{T}\sum_{t=1}^{T}\ell_{F}\left(HF_{t};r_{H}(\theta)\right)\right|\leq\frac{C\Delta_{1}}{T}\sum_{t=1}^{T}D_{2,\psi_{t|t-1}}\chi_{k}^{(t-1)/k}, \tag*{(117)} H ∈ H sup θ ∈ Θ sup T 1 t = 1 ∑ T ℓ F ( H F t ; r H ( θ ) ) − T 1 t = 1 ∑ T ℓ F ( H F t ; r H ( θ ) ) ≤ T C Δ 1 t = 1 ∑ T D 2 , ψ t ∣ t − 1 χ k ( t − 1 ) / k , (117)
almost surely.
Taking expectations in (117) and using the geometric-sum bound yields
E [ sup H ∈ H sup θ ∈ Θ ∣ 1 T ∑ t = 1 T ℓ ^ F ( H F t ; r H ( θ ) ) − 1 T ∑ t = 1 T ℓ F ( H F t ; r H ( θ ) ) ∣ ] ≤ C Δ 1 T sup t ≥ 1 E [ D 2 , ψ t ∣ t − 1 ] ∑ t = 1 T χ k ( t − 1 ) / k ≤ C Δ 1 T sup t ≥ 1 E [ D 2 , ψ t ∣ t − 1 ] k 1 − χ k = O ( 1 T ) . \begin{aligned}
\mathbb{E}\left[\sup_{H\in\mathcal{H}}\sup_{\theta\in\Theta}\left|\frac{1}{T}\sum_{t=1}^{T}\widehat{\ell}_{F}\left(HF_{t};r_{H}(\theta)\right)-\frac{1}{T}\sum_{t=1}^{T}\ell_{F}\left(HF_{t};r_{H}(\theta)\right)\right|\right]
&\leq\frac{C\Delta_{1}}{T}\sup_{t\geq1}\mathbb{E}\left[D_{2,\psi_{t|t-1}}\right]\sum_{t=1}^{T}\chi_{k}^{(t-1)/k}\\
&\leq\frac{C\Delta_{1}}{T}\sup_{t\geq1}\mathbb{E}\left[D_{2,\psi_{t|t-1}}\right]\frac{k}{1-\chi_{k}}=O\left(\frac{1}{T}\right).
\end{aligned} E [ H ∈ H sup θ ∈ Θ sup T 1 t = 1 ∑ T ℓ F ( H F t ; r H ( θ ) ) − T 1 t = 1 ∑ T ℓ F ( H F t ; r H ( θ ) ) ] ≤ T C Δ 1 t ≥ 1 sup E [ D 2 , ψ t ∣ t − 1 ] t = 1 ∑ T χ k ( t − 1 ) / k ≤ T C Δ 1 t ≥ 1 sup E [ D 2 , ψ t ∣ t − 1 ] 1 − χ k k = O ( T 1 ) .
Markov’s inequality therefore implies
sup H ∈ H sup θ ∈ Θ ∣ 1 T ∑ t = 1 T ℓ ^ F ( H F t ; r H ( θ ) ) − 1 T ∑ t = 1 T ℓ F ( H F t ; r H ( θ ) ) ∣ = O P ( 1 T ) . (118) \sup_{H\in\mathcal{H}}\sup_{\theta\in\Theta}\left|\frac{1}{T}\sum_{t=1}^{T}\widehat{\ell}_{F}\left(HF_{t};r_{H}(\theta)\right)-\frac{1}{T}\sum_{t=1}^{T}\ell_{F}\left(HF_{t};r_{H}(\theta)\right)\right|=O_{\mathbb{P}}\left(\frac{1}{T}\right). \tag*{(118)} H ∈ H sup θ ∈ Θ sup T 1 t = 1 ∑ T ℓ F ( H F t ; r H ( θ ) ) − T 1 t = 1 ∑ T ℓ F ( H F t ; r H ( θ ) ) = O P ( T 1 ) . (118)
Since, H ^ − H 0 = o P ( 1 ) \widehat{H}-H_{0}=o_{\mathbb{P}}(1) H − H 0 = o P ( 1 ) by Proposition 1, and H 0 H_{0} H 0 belongs to the interior of H \mathcal{H} H , we have P ( H ^ ∈ H ) → 1 \mathbb{P}\left(\widehat{H}\in\mathcal{H}\right)\to1 P ( H ∈ H ) → 1 . Hence, (118) also holds at H = H ^ H = \widehat{H} H = H . Combining this result with (114) proves
sup θ ∈ Θ ∣ 1 T ∑ t = 1 T ℓ F ( F t H ^ ⊤ ; r H ^ ( θ ) ) + log ∣ H ^ ∣ − 1 T ∑ t = 1 T ℓ F ( F t ; θ ) ∣ = O P ( 1 T ) . (119) \sup_{\theta\in\Theta}\left|\frac{1}{T}\sum_{t=1}^{T}\ell_F\left(F_t\widehat{H}^{\top};r_{\widehat{H}}(\theta)\right)+\log\left|\widehat{H}\right|-\frac{1}{T}\sum_{t=1}^{T}\ell_F(F_t;\theta)\right|=O_{\mathbb{P}}\left(\frac{1}{T}\right). \tag*{(119)} θ ∈ Θ sup T 1 t = 1 ∑ T ℓ F ( F t H ⊤ ; r H ( θ ) ) + log H − T 1 t = 1 ∑ T ℓ F ( F t ; θ ) = O P ( T 1 ) . (119)
Controlling the term in (74)
We first collect the two exponential-convergence results needed below. By Assumptions 1, 2, and 7, and using Proposition 2(c), for some 0 < ϱ k < ϱ k < 1 0 < \varrho_k < \varrho_k < 1 0 < ϱ k < ϱ k < 1 ,
∥ F t − F ‾ t ∥ = O a . s . ( ϱ k ⌊ ( t − 1 ) / k ⌋ ) , (120) \left\|F_t-\overline{F}_t\right\|=O_{\mathrm{a.s.}}\left(\varrho_k^{\lfloor(t-1)/k\rfloor}\right), \tag*{(120)} F t − F t = O a.s. ( ϱ k ⌊( t − 1 ) / k ⌋ ) , (120)
sup θ ∈ Θ ∥ ψ t ∣ t − 1 ( θ ) − ψ ‾ t ∣ t − 1 ( θ ) ∥ = O a . s . ( ϱ k ⌊ ( t − 1 ) / k ⌋ ) . (121) \sup_{\theta\in\Theta}\left\|\psi_{t|t-1}(\theta)-\overline{\psi}_{t|t-1}(\theta)\right\|=O_{\mathrm{a.s.}}\left(\varrho_k^{\lfloor(t-1)/k\rfloor}\right). \tag*{(121)} θ ∈ Θ sup ψ t ∣ t − 1 ( θ ) − ψ t ∣ t − 1 ( θ ) = O a.s. ( ϱ k ⌊( t − 1 ) / k ⌋ ) . (121)
For completeness, (121) follows by inserting between the two filters the filter driven by { F ‾ t } t ∈ Z \left\{\overline{F}_t\right\}_{t\in\mathbb{Z}} { F t } t ∈ Z and initialized at ψ 1 ∣ 0 \psi_{1|0} ψ 1∣0 . The difference between the first two filters is controlled by the same iteration used in (67), whereas the difference between the latter filter and { ψ ‾ t ∣ t − 1 ( θ ) } t ∈ Z \left\{\overline{\psi}_{t|t-1}(\theta)\right\}_{t\in\mathbb{Z}} { ψ t ∣ t − 1 ( θ ) } t ∈ Z is controlled by Proposition 2(c), or equivalently by (70). Enlarging ϱ k \varrho_k ϱ k , if necessary, allows the two bounds to be written with the same geometric rate.
The mean value theorem and Assumption 9 then imply
sup θ ∈ Θ ∣ ℓ F ( F t ; θ ) − ℓ F ( F ‾ t ; θ ) ∣ ≤ D 2 , ψ t ∣ t − 1 sup θ ∈ Θ ∥ ψ t ∣ t − 1 ( θ ) − ψ ‾ t ∣ t − 1 ( θ ) ∥ + D 2 , F t ∥ F t − F ‾ t ∥ , (122) \sup_{\theta\in\Theta}\left|\ell_F(F_t;\theta)-\ell_F(\overline{F}_t;\theta)\right|\leq D_{2,\psi_{t|t-1}}\sup_{\theta\in\Theta}\left\|\psi_{t|t-1}(\theta)-\overline{\psi}_{t|t-1}(\theta)\right\|+D_{2,F_t}\left\|F_t-\overline{F}_t\right\|, \tag*{(122)} θ ∈ Θ sup ℓ F ( F t ; θ ) − ℓ F ( F t ; θ ) ≤ D 2 , ψ t ∣ t − 1 θ ∈ Θ sup ψ t ∣ t − 1 ( θ ) − ψ t ∣ t − 1 ( θ ) + D 2 , F t F t − F t , (122)
almost surely. Here, the two derivative envelopes are evaluated along the corresponding line segments in P \mathcal{P} P and R r \mathbb{R}^{r} R r , as allowed by Assumption 9. It remains to verify that the derivative envelopes do not alter the geometric rate. Let q = 2 p / ( p − 2 ) > 1 q=2p/(p-2)>1 q = 2 p / ( p − 2 ) > 1 , as in Assumption 9, and choose a real number a a a such that 1 < a < 1 ϱ k 1<a<\frac{1}{\varrho_k} 1 < a < ϱ k 1 . By Markov’s inequality and Assumption 9,
∑ t = 1 ∞ P ( D 2 , ψ t ∣ t − 1 > a ⌊ ( t − 1 ) / k ⌋ ) + ∑ t = 1 ∞ P ( D 2 , F t > a ⌊ ( t − 1 ) / k ⌋ ) ≤ sup t ≥ 1 E [ D 2 , ψ t ∣ t − 1 q + D 2 , F t q ] ∑ t = 1 ∞ a − q ⌊ ( t − 1 ) / k ⌋ < ∞ . (123) \sum_{t=1}^{\infty}\mathbb{P}\left(D_{2,\psi_{t|t-1}}>a^{\lfloor(t-1)/k\rfloor}\right)+\sum_{t=1}^{\infty}\mathbb{P}\left(D_{2,F_t}>a^{\lfloor(t-1)/k\rfloor}\right)
\leq\sup_{t\geq1}\mathbb{E}\left[D_{2,\psi_{t|t-1}}^{q}+D_{2,F_t}^{q}\right]\sum_{t=1}^{\infty}a^{-q\lfloor(t-1)/k\rfloor}<\infty. \tag*{(123)} t = 1 ∑ ∞ P ( D 2 , ψ t ∣ t − 1 > a ⌊( t − 1 ) / k ⌋ ) + t = 1 ∑ ∞ P ( D 2 , F t > a ⌊( t − 1 ) / k ⌋ ) ≤ t ≥ 1 sup E [ D 2 , ψ t ∣ t − 1 q + D 2 , F t q ] t = 1 ∑ ∞ a − q ⌊( t − 1 ) / k ⌋ < ∞. (123)
By the Borel-Cantelli lemma we therefore get
D 2 , ψ t ∣ t − 1 + D 2 , F t = O a . s . ( a ⌊ ( t − 1 ) / k ⌋ ) . (124) D_{2,\psi_{t|t-1}}+D_{2,F_t}=O_{\mathrm{a.s.}}\left(a^{\lfloor(t-1)/k\rfloor}\right). \tag*{(124)} D 2 , ψ t ∣ t − 1 + D 2 , F t = O a.s. ( a ⌊( t − 1 ) / k ⌋ ) . (124)
Combining (120) , (121) , (122) , and (124) , we obtain
sup θ ∈ Θ ∣ ℓ F ( F t ; θ ) − ℓ F ( F ˉ t ; θ ) ∣ = O a . s . ( ( a ϱ k ) ⌊ ( t − 1 ) / k ⌋ ) . (125) \sup_{\theta\in\Theta}\left|\ell_F(F_t;\theta)-\ell_F(\bar{F}_t;\theta)\right|=O_{\mathrm{a.s.}}\left((a\varrho_k)^{\lfloor(t-1)/k\rfloor}\right). \tag*{(125)} θ ∈ Θ sup ℓ F ( F t ; θ ) − ℓ F ( F ˉ t ; θ ) = O a.s. ( ( a ϱ k ) ⌊( t − 1 ) / k ⌋ ) . (125)
Since a ϱ k < 1 a\varrho_k<1 a ϱ k < 1 , the right-hand side is summable, and hence,
sup θ ∈ Θ ∣ 1 T ∑ t = 1 T ℓ F ( F t ; θ ) − 1 T ∑ t = 1 T ℓ F ( F ˉ t ; θ ) ∣ ≤ 1 T ∑ t = 1 T sup θ ∈ Θ ∣ ℓ F ( F t ; θ ) − ℓ F ( F ˉ t ; θ ) ∣ = O a . s . ( 1 T ∑ t = 1 ∞ ( a ϱ k ) ⌊ ( t − 1 ) / k ⌋ ) = O a . s . ( 1 T ) , (126) \sup_{\theta\in\Theta}\left|\frac{1}{T}\sum_{t=1}^{T}\ell_F(F_t;\theta)-\frac{1}{T}\sum_{t=1}^{T}\ell_F(\bar{F}_t;\theta)\right|
\leq\frac{1}{T}\sum_{t=1}^{T}\sup_{\theta\in\Theta}\left|\ell_F(F_t;\theta)-\ell_F(\bar{F}_t;\theta)\right|
=O_{\mathrm{a.s.}}\left(\frac{1}{T}\sum_{t=1}^{\infty}(a\varrho_k)^{\lfloor(t-1)/k\rfloor}\right)=O_{\mathrm{a.s.}}\left(\frac{1}{T}\right), \tag*{(126)} θ ∈ Θ sup T 1 t = 1 ∑ T ℓ F ( F t ; θ ) − T 1 t = 1 ∑ T ℓ F ( F ˉ t ; θ ) ≤ T 1 t = 1 ∑ T θ ∈ Θ sup ℓ F ( F t ; θ ) − ℓ F ( F ˉ t ; θ ) = O a.s. ( T 1 t = 1 ∑ ∞ ( a ϱ k ) ⌊( t − 1 ) / k ⌋ ) = O a.s. ( T 1 ) , (126)
as ∑ t = 1 ∞ ( a ϱ k ) ⌊ ( t − 1 ) / k ⌋ = k 1 − a ϱ k < ∞ \sum_{t=1}^{\infty}(a\varrho_k)^{\lfloor(t-1)/k\rfloor}=\frac{k}{1-a\varrho_k}<\infty ∑ t = 1 ∞ ( a ϱ k ) ⌊( t − 1 ) / k ⌋ = 1 − a ϱ k k < ∞ . Equation (126) implies
sup θ ∈ Θ ∣ 1 T ∑ t = 1 T ℓ F ( F t ; θ ) − 1 T ∑ t = 1 T ℓ F ( F ˉ t ; θ ) ∣ = O P ( 1 T ) . (127) \sup_{\theta\in\Theta}\left|\frac{1}{T}\sum_{t=1}^{T}\ell_F(F_t;\theta)-\frac{1}{T}\sum_{t=1}^{T}\ell_F(\bar{F}_t;\theta)\right|=O_{\mathbb{P}}\left(\frac{1}{T}\right). \tag*{(127)} θ ∈ Θ sup T 1 t = 1 ∑ T ℓ F ( F t ; θ ) − T 1 t = 1 ∑ T ℓ F ( F ˉ t ; θ ) = O P ( T 1 ) . (127)
Controlling the term in (75)
For every θ ∈ Θ \theta\in\Theta θ ∈ Θ , define
Z t ( θ ) = ℓ F ( F ˉ t ; θ ) − E [ ℓ F ( F ˉ t ; θ ) ] . (128) Z_t(\theta)=\ell_F(\bar{F}_t;\theta)-\mathbb{E}\left[\ell_F(\bar{F}_t;\theta)\right]. \tag*{(128)} Z t ( θ ) = ℓ F ( F ˉ t ; θ ) − E [ ℓ F ( F ˉ t ; θ ) ] . (128)
By construction, E [ Z t ( θ ) ] = 0 \mathbb{E}\left[Z_t(\theta)\right]=0 E [ Z t ( θ ) ] = 0 for every θ ∈ Θ \theta\in\Theta θ ∈ Θ .
We first verify the dependence condition for the entire likelihood family. For any τ ≥ 1 \tau\geq1 τ ≥ 1 , consider the jointly truncated factor and filter processes, obtained by initializing the score-driven recursion at t − τ t-\tau t − τ and subsequently using v t , … , v t − τ v_t,\ldots,v_{t-\tau} v t , … , v t − τ . Thus, F t ( τ ) F_t^{(\tau)} F t ( τ ) and ψ t ∣ t − 1 ( τ ) ( θ ) \psi_{t|t-1}^{(\tau)}(\theta) ψ t ∣ t − 1 ( τ ) ( θ ) are measurable functions of v t , … , v t − τ v_t,\ldots,v_{t-\tau} v t , … , v t − τ . The proof of Lemma 6, together with the uniform contraction in Assumption 7, applies uniformly over θ ∈ Θ \theta\in\Theta θ ∈ Θ and yields
sup t ∈ N { ∥ F ˉ t − F t ( τ ) ∥ 2 + sup θ ∈ Θ ∥ ψ ˉ t ∣ t − 1 ( θ ) − ψ t ∣ t − 1 ( τ ) ( θ ) ∥ 2 } = O ( χ k ⌊ τ / k ⌋ ) . (129) \sup_{t\in\mathbb{N}}\left\{\left\|\bar{F}_t-F_t^{(\tau)}\right\|_2+\sup_{\theta\in\Theta}\left\|\bar{\psi}_{t|t-1}(\theta)-\psi_{t|t-1}^{(\tau)}(\theta)\right\|_2\right\}=O\left(\chi_k^{\lfloor\tau/k\rfloor}\right). \tag*{(129)} t ∈ N sup { F ˉ t − F t ( τ ) 2 + θ ∈ Θ sup ψ ˉ t ∣ t − 1 ( θ ) − ψ t ∣ t − 1 ( τ ) ( θ ) 2 } = O ( χ k ⌊ τ / k ⌋ ) . (129)
Let Z t ( τ ) ( θ ) Z_t^{(\tau)}(\theta) Z t ( τ ) ( θ ) denote the likelihood contribution computed from F t ( τ ) F_t^{(\tau)} F t ( τ ) and ψ t ∣ t − 1 ( τ ) ( θ ) \psi_{t|t-1}^{(\tau)}(\theta) ψ t ∣ t − 1 ( τ ) ( θ ) , after subtracting the same unconditional mean as in (128) . Therefore, Z t ( τ ) ( θ ) Z_t^{(\tau)}(\theta) Z t ( τ ) ( θ ) is measurable with respect to σ ( v t , … , v t − τ ) \sigma(v_t,\ldots,v_{t-\tau}) σ ( v t , … , v t − τ ) . Since the same unconditional mean is subtracted from Z t ( θ ) Z_t(\theta) Z t ( θ ) and Z t ( τ ) ( θ ) Z_t^{(\tau)}(\theta) Z t ( τ ) ( θ ) , their difference is equal to the difference between the corresponding likelihood contributions. Therefore, by the mean value theorem,
sup θ ∈ Θ ∣ Z t ( θ ) − Z t ( τ ) ( θ ) ∣ ≤ D 2 , ψ t ∣ t − 1 sup θ ∈ Θ ∥ ψ ˉ t ∣ t − 1 ( θ ) − ψ t ∣ t − 1 ( τ ) ( θ ) ∥ + D 2 , F t ∥ F ˉ t − F t ( τ ) ∥ , (130) \begin{aligned}
\sup_{\theta\in\Theta}\left|Z_t(\theta)-Z_t^{(\tau)}(\theta)\right| \\
&\le D_{2,\psi_{t|t-1}}\sup_{\theta\in\Theta}\left\|\bar{\psi}_{t|t-1}(\theta)-\psi^{(\tau)}_{t|t-1}(\theta)\right\|+D_{2,F_t}\left\|\bar{F}_t-F_t^{(\tau)}\right\|, \tag*{(130)}
\end{aligned} θ ∈ Θ sup Z t ( θ ) − Z t ( τ ) ( θ ) ≤ D 2 , ψ t ∣ t − 1 θ ∈ Θ sup ψ ˉ t ∣ t − 1 ( θ ) − ψ t ∣ t − 1 ( τ ) ( θ ) + D 2 , F t F ˉ t − F t ( τ ) , (130)
almost surely, where the derivative envelopes are evaluated along the corresponding line segments in R r \mathbb{R}^r R r and P \mathcal{P} P , as allowed by Assumption 9.
Taking the L 1 L_1 L 1 -norm on both sides of (130) and applying the Cauchy–Schwarz inequality we get
∥ sup θ ∈ Θ ∣ Z t ( θ ) − Z t ( τ ) ( θ ) ∣ ∥ 1 ≤ ∥ D 2 , ψ t ∣ t − 1 ∥ 2 ∥ sup θ ∈ Θ ∣ ψ ˉ t ∣ t − 1 ( θ ) − ψ t ∣ t − 1 ( τ ) ( θ ) ∣ ∥ 2 + ∥ D 2 , F t ∥ 2 ∥ F ˉ t − F t ( τ ) ∥ 2 . (131) \left\|\sup_{\theta\in\Theta}\left|Z_t(\theta)-Z_t^{(\tau)}(\theta)\right|\right\|_1
\le\left\|D_{2,\psi_{t|t-1}}\right\|_2\left\|\sup_{\theta\in\Theta}\left|\bar{\psi}_{t|t-1}(\theta)-\psi_{t|t-1}^{(\tau)}(\theta)\right|\right\|_2+\left\|D_{2,F_t}\right\|_2\left\|\bar{F}_t-F_t^{(\tau)}\right\|_2. \tag*{(131)} θ ∈ Θ sup Z t ( θ ) − Z t ( τ ) ( θ ) 1 ≤ D 2 , ψ t ∣ t − 1 2 θ ∈ Θ sup ψ ˉ t ∣ t − 1 ( θ ) − ψ t ∣ t − 1 ( τ ) ( θ ) 2 + ∥ D 2 , F t ∥ 2 F ˉ t − F t ( τ ) 2 . (131)
Since q = 2 p / ( p − 2 ) > 2 q=2p/(p-2)>2 q = 2 p / ( p − 2 ) > 2 , Assumption 9 implies that the two derivative envelopes have uniformly bounded second moments. Hence, combining (131) with (129) , we obtain
sup t ∈ N ∥ sup θ ∈ Θ ∣ Z t ( θ ) − Z t ( τ ) ( θ ) ∣ ∥ 1 = O ( χ k τ / k ) . (132) \sup_{t\in\mathbb{N}}\left\|\sup_{\theta\in\Theta}\left|Z_t(\theta)-Z_t^{(\tau)}(\theta)\right|\right\|_1=O\left(\chi_k^{\tau/k}\right). \tag*{(132)} t ∈ N sup θ ∈ Θ sup Z t ( θ ) − Z t ( τ ) ( θ ) 1 = O ( χ k τ / k ) . (132)
In addition, Assumption 8, applied to both the stationary and the truncated likelihood contributions, and Minkowski’s inequality imply that
sup τ ≥ 1 sup t ∈ N ∥ sup θ ∈ Θ ∣ Z t ( θ ) − Z t ( τ ) ( θ ) ∣ ∥ p < ∞ . (133) \sup_{\tau\ge1}\sup_{t\in\mathbb{N}}\left\|\sup_{\theta\in\Theta}\left|Z_t(\theta)-Z_t^{(\tau)}(\theta)\right|\right\|_p<\infty. \tag*{(133)} τ ≥ 1 sup t ∈ N sup θ ∈ Θ sup Z t ( θ ) − Z t ( τ ) ( θ ) p < ∞. (133)
We can now interpolate between (132) and (133) . In particular, applying Hölder’s inequality to the underlying expectation yields
E [ ( sup θ ∈ Θ ∣ Z t ( θ ) − Z t ( τ ) ( θ ) ∣ ) 2 ] = E [ ( sup θ ∈ Θ ∣ Z t ( θ ) − Z t ( τ ) ( θ ) ∣ ) p − 2 p − 1 ( sup θ ∈ Θ ∣ Z t ( θ ) − Z t ( τ ) ( θ ) ∣ ) p p − 1 ] ≤ ( E [ sup θ ∈ Θ ∣ Z t ( θ ) − Z t ( τ ) ( θ ) ∣ ] ) p − 2 p − 1 ( E [ ( sup θ ∈ Θ ∣ Z t ( θ ) − Z t ( τ ) ( θ ) ∣ ) p ] ) 1 p − 1 . \begin{aligned}
\mathbb{E}\left[\left(\sup_{\theta\in\Theta}\left|Z_t(\theta)-Z_t^{(\tau)}(\theta)\right|\right)^2\right]
&=\mathbb{E}\left[\left(\sup_{\theta\in\Theta}\left|Z_t(\theta)-Z_t^{(\tau)}(\theta)\right|\right)^{\frac{p-2}{p-1}}\left(\sup_{\theta\in\Theta}\left|Z_t(\theta)-Z_t^{(\tau)}(\theta)\right|\right)^{\frac{p}{p-1}}\right]\\
&\le\left(\mathbb{E}\left[\sup_{\theta\in\Theta}\left|Z_t(\theta)-Z_t^{(\tau)}(\theta)\right|\right]\right)^{\frac{p-2}{p-1}}\left(\mathbb{E}\left[\left(\sup_{\theta\in\Theta}\left|Z_t(\theta)-Z_t^{(\tau)}(\theta)\right|\right)^p\right]\right)^{\frac{1}{p-1}}.
\end{aligned} E [ ( θ ∈ Θ sup Z t ( θ ) − Z t ( τ ) ( θ ) ) 2 ] = E [ ( θ ∈ Θ sup Z t ( θ ) − Z t ( τ ) ( θ ) ) p − 1 p − 2 ( θ ∈ Θ sup Z t ( θ ) − Z t ( τ ) ( θ ) ) p − 1 p ] ≤ ( E [ θ ∈ Θ sup Z t ( θ ) − Z t ( τ ) ( θ ) ] ) p − 1 p − 2 ( E [ ( θ ∈ Θ sup Z t ( θ ) − Z t ( τ ) ( θ ) ) p ] ) p − 1 1 .
Rewriting these expectations back into their respective L 1 L^1 L 1 and L p L^p L p norm representations
∥ sup θ ∈ Θ ∣ Z t ( θ ) − Z t ( τ ) ( θ ) ∣ ∥ 2 2 ≤ ∥ sup θ ∈ Θ ∣ Z t ( θ ) − Z t ( τ ) ( θ ) ∣ ∥ 1 p − 2 p − 1 ∥ sup θ ∈ Θ ∣ Z t ( θ ) − Z t ( τ ) ( θ ) ∣ ∥ p p p − 1 . \begin{aligned}
\left\|\sup_{\theta\in\Theta}\left|Z_t(\theta)-Z_t^{(\tau)}(\theta)\right|\right\|_2^2 \\
&\le\left\| \sup_{\theta\in\Theta} \left| Z_t(\theta) - Z_t^{(\tau)}(\theta) \right| \right\|_1^{\frac{p-2}{p-1}} \left\| \sup_{\theta\in\Theta} \left| Z_t(\theta) - Z_t^{(\tau)}(\theta) \right| \right\|_p^{\frac{p}{p-1}} .
\end{aligned} θ ∈ Θ sup Z t ( θ ) − Z t ( τ ) ( θ ) 2 2 ≤ θ ∈ Θ sup Z t ( θ ) − Z t ( τ ) ( θ ) 1 p − 1 p − 2 θ ∈ Θ sup Z t ( θ ) − Z t ( τ ) ( θ ) p p − 1 p .
Taking the square root of both sides then yields the final statement
∥ sup θ ∈ Θ ∣ Z t ( θ ) − Z t ( τ ) ( θ ) ∣ ∥ 2 ≤ ∥ sup θ ∈ Θ ∣ Z t ( θ ) − Z t ( τ ) ( θ ) ∣ ∥ 1 p − 2 2 ( p − 1 ) ∥ sup θ ∈ Θ ∣ Z t ( θ ) − Z t ( τ ) ( θ ) ∣ ∥ p p 2 ( p − 1 ) , (134) \left\| \sup_{\theta\in\Theta} \left| Z_t(\theta) - Z_t^{(\tau)}(\theta) \right| \right\|_2
\le\left\| \sup_{\theta\in\Theta} \left| Z_t(\theta) - Z_t^{(\tau)}(\theta) \right| \right\|_1^{\frac{p-2}{2(p-1)}} \left\| \sup_{\theta\in\Theta} \left| Z_t(\theta) - Z_t^{(\tau)}(\theta) \right| \right\|_p^{\frac{p}{2(p-1)}} , \tag*{(134)} θ ∈ Θ sup Z t ( θ ) − Z t ( τ ) ( θ ) 2 ≤ θ ∈ Θ sup Z t ( θ ) − Z t ( τ ) ( θ ) 1 2 ( p − 1 ) p − 2 θ ∈ Θ sup Z t ( θ ) − Z t ( τ ) ( θ ) p 2 ( p − 1 ) p , (134)
where 1 2 = p − 2 2 ( p − 1 ) + p 2 ( p − 1 ) 1 p \frac{1}{2} = \frac{p-2}{2(p-1)} + \frac{p}{2(p-1)}\frac{1}{p} 2 1 = 2 ( p − 1 ) p − 2 + 2 ( p − 1 ) p p 1 .
Consequently,
sup t ∈ N ∥ sup θ ∈ Θ ∣ Z t ( θ ) − Z t ( τ ) ( θ ) ∣ ∥ 2 = O ( χ k p − 2 2 ( p − 1 ) [ τ / k ] ) . (135) \sup_{t \in\mathbb{N}} \left\| \sup_{\theta\in\Theta} \left| Z_t(\theta) - Z_t^{(\tau)}(\theta) \right| \right\|_2 = O\left( \chi_k^{\frac{p-2}{2(p-1)}} [\tau/k] \right). \tag*{(135)} t ∈ N sup θ ∈ Θ sup Z t ( θ ) − Z t ( τ ) ( θ ) 2 = O ( χ k 2 ( p − 1 ) p − 2 [ τ / k ] ) . (135)
Since Z t ( τ ) ( θ ) Z_t^{(\tau)}(\theta) Z t ( τ ) ( θ ) is measurable with respect to σ ( v t , … , v t − τ ) \sigma(v_t,\ldots,v_{t-\tau}) σ ( v t , … , v t − τ ) , we have
Z t ( θ ) − E [ Z t ( θ ) ∣ v t , … , v t − τ ] = Z t ( θ ) − Z t ( τ ) ( θ ) − E [ Z t ( θ ) − Z t ( τ ) ( θ ) ∣ v t , … , v t − τ ] . Z_t(\theta) - \mathbb{E}\left[Z_t(\theta) \mid v_t,\ldots,v_{t-\tau}\right]
= Z_t(\theta) - Z_t^{(\tau)}(\theta) - \mathbb{E}\left[ Z_t(\theta) - Z_t^{(\tau)}(\theta) \mid v_t,\ldots,v_{t-\tau} \right]. Z t ( θ ) − E [ Z t ( θ ) ∣ v t , … , v t − τ ] = Z t ( θ ) − Z t ( τ ) ( θ ) − E [ Z t ( θ ) − Z t ( τ ) ( θ ) ∣ v t , … , v t − τ ] .
Therefore, Minkowski’s inequality and the contraction property of conditional expectations
∥ sup θ ∈ Θ ∣ Z t ( θ ) − E [ Z t ( θ ) ∣ v t , … , v t − τ ] ∣ ∥ 2 ≤ 2 ∥ sup θ ∈ Θ ∣ Z t ( θ ) − Z t ( τ ) ( θ ) ∣ ∥ 2 . \left\| \sup_{\theta\in\Theta} \left| Z_t(\theta) - \mathbb{E}\left[ Z_t(\theta) \mid v_t,\ldots,v_{t-\tau} \right] \right| \right\|_2
\le2 \left\| \sup_{\theta\in\Theta} \left| Z_t(\theta) - Z_t^{(\tau)}(\theta) \right| \right\|_2. θ ∈ Θ sup ∣ Z t ( θ ) − E [ Z t ( θ ) ∣ v t , … , v t − τ ] ∣ 2 ≤ 2 θ ∈ Θ sup Z t ( θ ) − Z t ( τ ) ( θ ) 2 .
Combining this inequality with (135) yields
sup t ∈ N ∥ sup θ ∈ Θ ∣ Z t ( θ ) − E [ Z t ( θ ) ∣ v t , … , v t − τ ] ∣ ∥ 2 = O ( χ k p − 2 2 ( p − 1 ) [ τ / k ] ) . (136) \sup_{t \in\mathbb{N}} \left\| \sup_{\theta\in\Theta} \left| Z_t(\theta) - \mathbb{E}\left[ Z_t(\theta) \mid v_t,\ldots,v_{t-\tau} \right] \right| \right\|_2 = O\left( \chi_k^{\frac{p-2}{2(p-1)}} [\tau/k] \right). \tag*{(136)} t ∈ N sup θ ∈ Θ sup ∣ Z t ( θ ) − E [ Z t ( θ ) ∣ v t , … , v t − τ ] ∣ 2 = O ( χ k 2 ( p − 1 ) p − 2 [ τ / k ] ) . (136)
Hence, { Z t ( θ ) } t ∈ Z \{Z_t(\theta)\}_{t \in\mathbb{Z}} { Z t ( θ ) } t ∈ Z is uniformly near-epoch dependent of size − 1 -1 − 1 on { v t } t ∈ Z \{v_t\}_{t \in\mathbb{Z}} { v t } t ∈ Z , uniformly over θ ∈ Θ \theta\in\Theta θ ∈ Θ . Moreover, by Assumption 8
sup t ∈ Z E [ sup θ ∈ Θ ∣ Z t ( θ ) ∣ p ] < ∞ for some p > 2. (137) \sup_{t \in\mathbb{Z}} \mathbb{E}\left[ \sup_{\theta\in\Theta} |Z_t(\theta)|^p \right] < \infty\qquad\text{for some } p > 2. \tag*{(137)} t ∈ Z sup E [ θ ∈ Θ sup ∣ Z t ( θ ) ∣ p ] < ∞ for some p > 2. (137)
To obtain the root-T T T bound uniformly over Θ \Theta Θ , we must control the complexity of the class of centered likelihood contributions. To this end, let us fix 2 < s < p 2 < s < p 2 < s < p , so that by Assumption 8, there exists a nonnegative process { L t } t ∈ Z \{L_t\}_{t\in\mathbb{Z}} { L t } t ∈ Z such that, almost surely,
∣ ℓ F ( F ˉ t ; θ 1 ) − ℓ F ( F ˉ t ; θ 2 ) ∣ ≤ L t ∥ θ 1 − θ 2 ∥ , θ 1 , θ 2 ∈ Θ , (138) \left|\ell_F(\bar{F}_t;\theta_1)-\ell_F(\bar{F}_t;\theta_2)\right| \le L_t\|\theta_1-\theta_2\|,\qquad\theta_1,\theta_2\in\Theta, \tag*{(138)} ℓ F ( F ˉ t ; θ 1 ) − ℓ F ( F ˉ t ; θ 2 ) ≤ L t ∥ θ 1 − θ 2 ∥ , θ 1 , θ 2 ∈ Θ , (138)
and sup t ∈ Z E [ L t p ] < ∞ \sup_{t\in\mathbb{Z}}\mathbb{E}\left[L_t^p\right]<\infty sup t ∈ Z E [ L t p ] < ∞ . It follows from the definition of Z t ( θ ) Z_t(\theta) Z t ( θ ) and (138) that, almost surely,
∣ Z t ( θ 1 ) − Z t ( θ 2 ) ∣ ≤ ( L t + E [ L t ] ) ∥ θ 1 − θ 2 ∥ . (139) |Z_t(\theta_1)-Z_t(\theta_2)|\leq(L_t+\mathbb{E}[L_t])\|\theta_1-\theta_2\|. \tag*{(139)} ∣ Z t ( θ 1 ) − Z t ( θ 2 ) ∣ ≤ ( L t + E [ L t ]) ∥ θ 1 − θ 2 ∥. (139)
Moreover, since 2 < s < p 2<s<p 2 < s < p ,
sup t ∈ Z ∥ L t + E [ L t ] ∥ s ≤ 2 sup t ∈ Z ∥ L t ∥ s < ∞ . (140) \sup_{t\in\mathbb{Z}}\|L_t+\mathbb{E}[L_t]\|_s\leq2\sup_{t\in\mathbb{Z}}\|L_t\|_s<\infty. \tag*{(140)} t ∈ Z sup ∥ L t + E [ L t ] ∥ s ≤ 2 t ∈ Z sup ∥ L t ∥ s < ∞. (140)
Recall that, following Doukhan et al. [22 ] , N [ ] ( ε , { Z t ( θ ) : θ ∈ Θ } , ∥ ⋅ ∥ s ) N_{[]}(\varepsilon,\{Z_t(\theta):\theta\in\Theta\},\|\cdot\|_s) N [ ] ( ε , { Z t ( θ ) : θ ∈ Θ } , ∥ ⋅ ∥ s ) denotes the minimum number of brackets [ Z ‾ t , Z ‾ t ] \left[\underline{Z}_t,\overline{Z}_t\right] [ Z t , Z t ] satisfying Z ‾ t ≤ Z ‾ t \underline{Z}_t\leq\overline{Z}_t Z t ≤ Z t almost surely and ∥ Z ‾ t − Z ‾ t ∥ s ≤ ε \|\overline{Z}_t-\underline{Z}_t\|_s\leq\varepsilon ∥ Z t − Z t ∥ s ≤ ε required to cover { Z t ( θ ) : θ ∈ Θ } \{Z_t(\theta):\theta\in\Theta\} { Z t ( θ ) : θ ∈ Θ } . If θ 1 , … , θ J \theta_1,\ldots,\theta_J θ 1 , … , θ J form a Euclidean δ \delta δ -cover of Θ \Theta Θ , the inequality in (139) shows that the brackets of the form [ Z t ( θ j ) − δ ( L t + E [ L t ] ) , Z t ( θ j ) + δ ( L t + E [ L t ] ) ] \left[Z_t(\theta_j)-\delta(L_t+\mathbb{E}[L_t]),Z_t(\theta_j)+\delta(L_t+\mathbb{E}[L_t])\right] [ Z t ( θ j ) − δ ( L t + E [ L t ]) , Z t ( θ j ) + δ ( L t + E [ L t ]) ] , for j = 1 , … , J j=1,\ldots,J j = 1 , … , J , cover the centered likelihood class and have L s L_s L s -size bounded by 2 δ sup u ∈ Z ∥ L u + E [ L u ] ∥ s 2\delta\sup_{u\in\mathbb{Z}}\|L_u+\mathbb{E}[L_u]\|_s 2 δ sup u ∈ Z ∥ L u + E [ L u ] ∥ s . Since Θ \Theta Θ is a compact subset of R d \mathbb{R}^d R d , choosing δ = ε 2 sup u ∈ Z ∥ L u + E [ L u ] ∥ s \delta=\frac{\varepsilon}{2\sup_{u\in\mathbb{Z}}\|L_u+\mathbb{E}[L_u]\|_s} δ = 2 s u p u ∈ Z ∥ L u + E [ L u ] ∥ s ε gives, for some finite positive real C Θ C_\Theta C Θ and every 0 < ε ≤ 1 0<\varepsilon\leq1 0 < ε ≤ 1 ,
N [ ] ( ε , { Z t ( θ ) : θ ∈ Θ } , ∥ ⋅ ∥ s ) ≤ C Θ ε − d . (141) N_{[]}(\varepsilon,\{Z_t(\theta):\theta\in\Theta\},\|\cdot\|_s)\leq C_\Theta\varepsilon^{-d}. \tag*{(141)} N [ ] ( ε , { Z t ( θ ) : θ ∈ Θ } , ∥ ⋅ ∥ s ) ≤ C Θ ε − d . (141)
Consequently,
∫ 0 1 log N [ ] ( ε , { Z t ( θ ) : θ ∈ Θ } , ∥ ⋅ ∥ s ) d ε ≤ ∫ 0 1 log C Θ + d log ( 1 ε ) d ε < ∞ . (142) \begin{aligned}
\int_0^1\sqrt{\log N_{[]}(\varepsilon,\{Z_t(\theta):\theta\in\Theta\},\|\cdot\|_s)}\,\mathrm{d}\varepsilon
&\leq\int_0^1\sqrt{\log C_\Theta+d\log\left(\frac{1}{\varepsilon}\right)}\,\mathrm{d}\varepsilon<\infty. \tag*{(142)}
\end{aligned} ∫ 0 1 log N [ ] ( ε , { Z t ( θ ) : θ ∈ Θ } , ∥ ⋅ ∥ s ) d ε ≤ ∫ 0 1 log C Θ + d log ( ε 1 ) d ε < ∞. (142)
This is the bracketing entropy condition used in Theorem 1 and Application 1 of Doukhan et al. [22 ] . Next, to control the approximation of the stationary likelihood by its finite-window counterpart in the same L s L_s L s norm, we note that since 1 s = 2 ( p − s ) s ( p − 2 ) 1 2 + p ( s − 2 ) s ( p − 2 ) 1 p \frac{1}{s}=\frac{2(p-s)}{s(p-2)}\frac{1}{2}+\frac{p(s-2)}{s(p-2)}\frac{1}{p} s 1 = s ( p − 2 ) 2 ( p − s ) 2 1 + s ( p − 2 ) p ( s − 2 ) p 1 , by Hölder’s inequality
∥ sup θ ∈ Θ ∣ Z t ( θ ) − Z t ( τ ) ( θ ) ∣ ∥ s ≤ ∥ sup θ ∈ Θ ∣ Z t ( θ ) − Z t ( τ ) ( θ ) ∣ ∥ 2 2 ( p − s ) s ( p − 2 ) ∥ sup θ ∈ Θ ∣ Z t ( θ ) − Z t ( τ ) ( θ ) ∣ ∥ p p ( s − 2 ) s ( p − 2 ) . \left\|\sup_{\theta\in\Theta}|Z_t(\theta)-Z_t^{(\tau)}(\theta)|\right\|_s
\leq
\left\|\sup_{\theta\in\Theta}|Z_t(\theta)-Z_t^{(\tau)}(\theta)|\right\|_2^{\frac{2(p-s)}{s(p-2)}}
\left\|\sup_{\theta\in\Theta}|Z_t(\theta)-Z_t^{(\tau)}(\theta)|\right\|_p^{\frac{p(s-2)}{s(p-2)}}. θ ∈ Θ sup ∣ Z t ( θ ) − Z t ( τ ) ( θ ) ∣ s ≤ θ ∈ Θ sup ∣ Z t ( θ ) − Z t ( τ ) ( θ ) ∣ 2 s ( p − 2 ) 2 ( p − s ) θ ∈ Θ sup ∣ Z t ( θ ) − Z t ( τ ) ( θ ) ∣ p s ( p − 2 ) p ( s − 2 ) .
Combining (135) and (133) therefore yields
sup t ∈ Z sup θ ∈ Θ ∥ Z t ( θ ) − Z t ( τ ) ( θ ) ∥ s = O ( χ k p − s p − 1 [ τ k ] ) . (143) \sup_{t\in\mathbb{Z}}\sup_{\theta\in\Theta}\left\|Z_t(\theta)-Z_t^{(\tau)}(\theta)\right\|_s=O\left(\chi_k^{\frac{p-s}{p-1}}\left[\frac{\tau}{k}\right]\right). \tag*{(143)} t ∈ Z sup θ ∈ Θ sup Z t ( θ ) − Z t ( τ ) ( θ ) s = O ( χ k p − 1 p − s [ k τ ] ) . (143)
For every fixed τ \tau τ , the process { Z t ( τ ) ( θ ) } t ∈ Z \left\{Z_t^{(\tau)}(\theta)\right\}_{t\in\mathbb{Z}} { Z t ( τ ) ( θ ) } t ∈ Z is a measurable function of the finite blocks { v t , … , v t − τ } t ∈ Z \{v_t,\ldots,v_{t-\tau}\}_{t\in\mathbb{Z}} { v t , … , v t − τ } t ∈ Z . Hence, for every h > τ h>\tau h > τ , its β \beta β -mixing coefficients satisfy β Z ( τ ) ( h ) ≤ β v ( h − τ ) \beta_{Z^{(\tau)}}(h)\leq\beta_v(h-\tau) β Z ( τ ) ( h ) ≤ β v ( h − τ ) . It follows from Assumption 1(i) that, for every fixed τ \tau τ , the finite-window process is geometrically β \beta β -mixing.
Consider now the centered finite-window decomposition
Z t ( θ ) = Z t ( 1 ) ( θ ) − E [ Z t ( 1 ) ( θ ) ] + ∑ j = 1 ∞ ( Z t ( 2 j ) ( θ ) − Z t ( 2 j − 1 ) ( θ ) − E [ Z t ( 2 j ) ( θ ) − Z t ( 2 j − 1 ) ( θ ) ] ) , \begin{aligned}
Z_t(\theta)&=Z_t^{(1)}(\theta)-\mathbb{E}\left[Z_t^{(1)}(\theta)\right]\\
&\quad+\sum_{j=1}^{\infty}\left(Z_t^{(2^j)}(\theta)-Z_t^{(2^{j-1})}(\theta)-\mathbb{E}\left[Z_t^{(2^j)}(\theta)-Z_t^{(2^{j-1})}(\theta)\right]\right),
\end{aligned} Z t ( θ ) = Z t ( 1 ) ( θ ) − E [ Z t ( 1 ) ( θ ) ] + j = 1 ∑ ∞ ( Z t ( 2 j ) ( θ ) − Z t ( 2 j − 1 ) ( θ ) − E [ Z t ( 2 j ) ( θ ) − Z t ( 2 j − 1 ) ( θ ) ] ) ,
which holds in L s L_s L s by (143) . For every j ≥ 1 j\geq1 j ≥ 1 , Minkowski’s inequality and (143) give
sup t ∈ Z ∥ sup θ ∈ Θ ∣ Z t ( 2 j ) ( θ ) − Z t ( 2 j − 1 ) ( θ ) − E [ Z t ( 2 j ) ( θ ) − Z t ( 2 j − 1 ) ( θ ) ] ∣ ∥ s ≤ 2 sup t ∈ Z ∥ sup θ ∈ Θ ∣ Z t ( θ ) − Z t ( 2 j ) ( θ ) ∣ ∥ s + 2 sup t ∈ Z ∥ sup θ ∈ Θ ∣ Z t ( θ ) − Z t ( 2 j − 1 ) ( θ ) ∣ ∥ s = O ( χ k p − s p − 1 [ 2 j − 1 k ] ) . (144) \begin{aligned}
\sup_{t\in\mathbb{Z}}\left\|\sup_{\theta\in\Theta}\left|Z_t^{(2^j)}(\theta)-Z_t^{(2^{j-1})}(\theta)-\mathbb{E}\left[Z_t^{(2^j)}(\theta)-Z_t^{(2^{j-1})}(\theta)\right]\right|\right\|_s
&\leq2\sup_{t\in\mathbb{Z}}\left\|\sup_{\theta\in\Theta}\left|Z_t(\theta)-Z_t^{(2^j)}(\theta)\right|\right\|_s\\
&\quad+2\sup_{t\in\mathbb{Z}}\left\|\sup_{\theta\in\Theta}\left|Z_t(\theta)-Z_t^{(2^{j-1})}(\theta)\right|\right\|_s\\
&=O\left(\chi_k^{\frac{p-s}{p-1}}\left[\frac{2^{j-1}}{k}\right]\right). \tag*{(144)}
\end{aligned} t ∈ Z sup θ ∈ Θ sup Z t ( 2 j ) ( θ ) − Z t ( 2 j − 1 ) ( θ ) − E [ Z t ( 2 j ) ( θ ) − Z t ( 2 j − 1 ) ( θ ) ] s ≤ 2 t ∈ Z sup θ ∈ Θ sup Z t ( θ ) − Z t ( 2 j ) ( θ ) s + 2 t ∈ Z sup θ ∈ Θ sup Z t ( θ ) − Z t ( 2 j − 1 ) ( θ ) s = O ( χ k p − 1 p − s [ k 2 j − 1 ] ) . (144)
The centered difference in (144) is measurable with respect to σ ( v t , … , v t − 2 j ) \sigma\left(v_t,\ldots,v_{t-2^j}\right) σ ( v t , … , v t − 2 j ) . Hence, for h > 2 j h>2^j h > 2 j , its β \beta β -mixing coefficients are bounded by β v ( h − 2 j ) \beta_v\left(h-2^j\right) β v ( h − 2 j ) .
The same Lipschitz argument used in (139) –(142) gives the corresponding polynomial L s L_s L s -bracketing bound for these finite-window differences. Therefore, Theorem 1 of Doukhan et al. [22 ] , together with (144) , imply that, for some finite positive real C C C independent of j j j ,
lim sup T → ∞ E [ sup θ ∈ Θ ∣ 1 T ∑ t = 1 T ( Z t ( 2 j ) ( θ ) − Z t ( 2 j − 1 ) ( θ ) − E [ Z t ( 2 j ) ( θ ) − Z t ( 2 j − 1 ) ( θ ) ] ) ∣ ] ≤ C 2 j χ k p − s p − 1 [ 2 j − 1 k ] . \limsup_{T\to\infty}\mathbb{E}\left[\sup_{\theta\in\Theta}\left|\frac{1}{\sqrt{T}}\sum_{t=1}^{T}\left(Z_t^{(2^j)}(\theta)-Z_t^{(2^{j-1})}(\theta)-\mathbb{E}\left[Z_t^{(2^j)}(\theta)-Z_t^{(2^{j-1})}(\theta)\right]\right)\right|\right]\leq C2^j\chi_k^{\frac{p-s}{p-1}}\left[\frac{2^{j-1}}{k}\right]. T → ∞ lim sup E [ θ ∈ Θ sup T 1 t = 1 ∑ T ( Z t ( 2 j ) ( θ ) − Z t ( 2 j − 1 ) ( θ ) − E [ Z t ( 2 j ) ( θ ) − Z t ( 2 j − 1 ) ( θ ) ] ) ] ≤ C 2 j χ k p − 1 p − s [ k 2 j − 1 ] .
Since $0<\chi_k<1$, ∑ j = 1 ∞ 2 j χ k p − s p − 1 [ 2 j − 1 / k ] < ∞ \sum_{j=1}^{\infty}2^j\chi_k^{\frac{p-s}{p-1}}\left[2^{j-1}/k\right]<\infty ∑ j = 1 ∞ 2 j χ k p − 1 p − s [ 2 j − 1 / k ] < ∞ , and therefore, applying the same result to $Z_t^{(1)}(\theta)-\mathbb{E}\left[Z_t^{(1)}(\theta)\right] and using the centered finite-window decomposition therefore yields
sup θ ∈ Θ ∣ 1 T ∑ t = 1 T Z t ( θ ) ∣ = O P ( 1 ) (145) \sup_{\theta\in\Theta} \left| \frac{1}{\sqrt{T}} \sum_{t=1}^{T} Z_t(\theta) \right| = O_{\mathbb{P}}(1) \tag*{(145)} θ ∈ Θ sup T 1 t = 1 ∑ T Z t ( θ ) = O P ( 1 ) (145)
as T → ∞ T \to\infty T → ∞ . It follows that
sup θ ∈ Θ ∣ 1 T ∑ t = 1 T ℓ F ( F ‾ t ; θ ) − E [ ℓ F ( F ‾ t ; θ ) ] ∣ = O P ( 1 T ) . (146) \sup_{\theta\in\Theta} \left| \frac{1}{T} \sum_{t=1}^{T} \ell_F\left(\overline{F}_t;\theta\right) - \mathbb{E}\left[\ell_F\left(\overline{F}_t;\theta\right)\right] \right| = O_{\mathbb{P}}\left(\frac{1}{\sqrt{T}}\right). \tag*{(146)} θ ∈ Θ sup T 1 t = 1 ∑ T ℓ F ( F t ; θ ) − E [ ℓ F ( F t ; θ ) ] = O P ( T 1 ) . (146)
Concluding the Proof of Proposition 3
By combining the rates in (111) , (119) , (127) , and (146) , and taking into account the triangle inequality in (71) , we finally obtain
sup θ ∈ Θ ∣ 1 T ∑ t = 1 T ℓ ^ F ( F ‾ t ; r H ^ ( θ ) ) + log ∣ H ^ ∣ − E [ ℓ F ( F ‾ t ; θ ) ] ∣ = O P ( 1 T + 1 N T + 1 N + 1 T ) = O P ( 1 N + 1 T ) . \begin{aligned}
\sup_{\theta\in\Theta} \left| \frac{1}{T} \sum_{t=1}^{T} \widehat{\ell}_F\left(\overline{F}_t;r_{\widehat{H}}(\theta)\right) + \log\left|\widehat{H}\right| - \mathbb{E}\left[\ell_F\left(\overline{F}_t;\theta\right)\right] \right| = O_{\mathbb{P}}\left(\frac{1}{\sqrt{T}}+\frac{1}{\sqrt{NT}}+\frac{1}{N}+\frac{1}{T}\right) \\
&= O_{\mathbb{P}}\left(\frac{1}{N}+\frac{1}{\sqrt{T}}\right).
\end{aligned} θ ∈ Θ sup T 1 t = 1 ∑ T ℓ F ( F t ; r H ( θ ) ) + log H − E [ ℓ F ( F t ; θ ) ] = O P ( T 1 + N T 1 + N 1 + T 1 ) = O P ( N 1 + T 1 ) .
This completes the proof.
Proof of Theorem 1
Since H 0 ∈ G L ( r ) H_0 \in GL(r) H 0 ∈ G L ( r ) and H ^ → P H 0 \widehat{H} \xrightarrow{\mathbb{P}} H_0 H P H 0 as N , T → ∞ N,T \to\infty N , T → ∞ by Proposition 1, we have
P ( H ^ ∈ G L ( r ) ) ⟶ 1 , N , T → ∞ . \mathbb{P}\left(\widehat{H} \in GL(r)\right) \longrightarrow1, \qquad N,T \to\infty. P ( H ∈ G L ( r ) ) ⟶ 1 , N , T → ∞.
The following arguments can therefore be restricted to this event, whose complement is asymptotically negligible. Since r H ^ r_{\widehat{H}} r H is an invertible reparameterization of Θ \Theta Θ onto itself, the definition of θ ^ \widehat{\theta} θ in (14) implies, on the event H ^ ∈ G L ( r ) \widehat{H} \in GL(r) H ∈ G L ( r ) , that
r H ^ − 1 ( θ ^ ) ∈ arg max θ ∈ Θ { 1 T ∑ t = 1 T ℓ ^ F ( F ^ t ; r H ^ ( θ ) ) + log ∣ H ^ ∣ } , (147) r_{\widehat{H}}^{-1}(\widehat{\theta}) \in\mathop{\arg\max}_{\theta\in\Theta} \left\{ \frac{1}{T} \sum_{t=1}^{T} \widehat{\ell}_F\left(\widehat{F}_t;r_{\widehat{H}}(\theta)\right) + \log\left|\widehat{H}\right| \right\}, \tag*{(147)} r H − 1 ( θ ) ∈ arg max θ ∈ Θ { T 1 t = 1 ∑ T ℓ F ( F t ; r H ( θ ) ) + log H } , (147)
almost surely, where log ∣ H ^ ∣ \log\left|\widehat{H}\right| log H does not affect the maximizer because it does not depend on θ \theta θ .
Define the uniform approximation error
δ N T = sup θ ∈ Θ ∣ 1 T ∑ t = 1 T ℓ ^ F ( F ^ t ; r H ^ ( θ ) ) + log ∣ H ^ ∣ − E [ ℓ F ( F ‾ t ; θ ) ] ∣ . (148) \delta_{NT} = \sup_{\theta\in\Theta} \left| \frac{1}{T} \sum_{t=1}^{T} \widehat{\ell}_F\left(\widehat{F}_t;r_{\widehat{H}}(\theta)\right) + \log\left|\widehat{H}\right| - \mathbb{E}\left[\ell_F\left(\overline{F}_t;\theta\right)\right] \right|. \tag*{(148)} δ N T = θ ∈ Θ sup T 1 t = 1 ∑ T ℓ F ( F t ; r H ( θ ) ) + log H − E [ ℓ F ( F t ; θ ) ] . (148)
Thus, for every θ ∈ Θ \theta\in\Theta θ ∈ Θ ,
− δ N T ≤ 1 T ∑ t = 1 T ℓ ^ F ( F ^ t ; r H ^ ( θ ) ) + log ∣ H ^ ∣ − E [ ℓ F ( F ‾ t ; θ ) ] ≤ δ N T , (149) -\delta_{NT} \le\frac{1}{T}\sum_{t=1}^{T}\widehat{\ell}_{F}\left(\widehat{F}_{t};r_{\widehat{H}}(\theta)\right)+\log\left|\widehat{H}\right|-\mathbb{E}\left[\ell_{F}\left(\overline{F}_{t};\theta\right)\right]\le\delta_{NT}, \tag*{(149)} − δ N T ≤ T 1 t = 1 ∑ T ℓ F ( F t ; r H ( θ ) ) + log H − E [ ℓ F ( F t ; θ ) ] ≤ δ N T , (149)
almost surely. By Proposition 3, as N , T → ∞ N,T\to\infty N , T → ∞ , min ( N , T ) δ N T = O P ( 1 ) \min\left(N,\sqrt{T}\right)\delta_{NT}=O_{\mathbb{P}}(1) min ( N , T ) δ N T = O P ( 1 ) , and since min ( N , T ) → ∞ \min\left(N,\sqrt{T}\right)\to\infty min ( N , T ) → ∞ as N , T → ∞ N,T\to\infty N , T → ∞ , it follows that
δ N T = o P ( 1 ) , N , T → ∞ . (150) \delta_{NT}=o_{\mathbb{P}}(1),\qquad N,T\to\infty. \tag*{(150)} δ N T = o P ( 1 ) , N , T → ∞. (150)
For every ϑ ∈ Θ H \vartheta\in\Theta_{H} ϑ ∈ Θ H , (147) and (149) imply, almost surely on the event H ^ ∈ G L ( r ) \widehat{H}\in GL(r) H ∈ G L ( r ) , that
E [ ℓ F ( F ‾ t ; r H ^ − 1 ( θ ^ ) ) ] ≥ 1 T ∑ t = 1 T ℓ ^ F ( F ^ t ; r H ^ ( r H ^ − 1 ( θ ^ ) ) ) + log ∣ H ^ ∣ − δ N T ≥ 1 T ∑ t = 1 T ℓ ^ F ( F ^ t ; r H ^ ( ϑ ) ) + log ∣ H ^ ∣ − δ N T ≥ E [ ℓ F ( F ‾ t ; ϑ ) ] − 2 δ N T , (151) \begin{aligned}
\mathbb{E}\left[\ell_{F}\left(\overline{F}_{t};r_{\widehat{H}}^{-1}(\widehat{\theta})\right)\right]
&\ge\frac{1}{T}\sum_{t=1}^{T}\widehat{\ell}_{F}\left(\widehat{F}_{t};r_{\widehat{H}}\left(r_{\widehat{H}}^{-1}(\widehat{\theta})\right)\right)+\log\left|\widehat{H}\right|-\delta_{NT}\\
&\ge\frac{1}{T}\sum_{t=1}^{T}\widehat{\ell}_{F}\left(\widehat{F}_{t};r_{\widehat{H}}(\vartheta)\right)+\log\left|\widehat{H}\right|-\delta_{NT}\\
&\ge\mathbb{E}\left[\ell_{F}\left(\overline{F}_{t};\vartheta\right)\right]-2\delta_{NT}, \tag*{(151)}
\end{aligned} E [ ℓ F ( F t ; r H − 1 ( θ ) ) ] ≥ T 1 t = 1 ∑ T ℓ F ( F t ; r H ( r H − 1 ( θ ) ) ) + log H − δ N T ≥ T 1 t = 1 ∑ T ℓ F ( F t ; r H ( ϑ ) ) + log H − δ N T ≥ E [ ℓ F ( F t ; ϑ ) ] − 2 δ N T , (151)
where the first and third inequalities follow from (149) , whereas the second follows from the maximizing property in (147) .
By Assumption 10(iii) and the definition of K T K_{T} K T , every ϑ ∈ Θ H \vartheta\in\Theta_{H} ϑ ∈ Θ H satisfies E [ ℓ F ( F ‾ t ; ϑ ) ] = K T \mathbb{E}\left[\ell_{F}\left(\overline{F}_{t};\vartheta\right)\right]=K_{T} E [ ℓ F ( F t ; ϑ ) ] = K T . In fact, Assumption 10(iii) gives
Θ H = { θ ∈ Θ : E [ ℓ F ( F ‾ t ; θ ) ] ≥ K T } , \Theta_{H}=\left\{\theta\in\Theta:\mathbb{E}\left[\ell_{F}\left(\overline{F}_{t};\theta\right)\right]\ge K_{T}\right\}, Θ H = { θ ∈ Θ : E [ ℓ F ( F t ; θ ) ] ≥ K T } ,
whereas K T = sup θ ∈ Θ E [ ℓ F ( F ‾ t ; θ ) ] K_{T}=\sup_{\theta\in\Theta}\mathbb{E}\left[\ell_{F}\left(\overline{F}_{t};\theta\right)\right] K T = sup θ ∈ Θ E [ ℓ F ( F t ; θ ) ] . Therefore, the inequality defining the upper level set must hold with equality for every θ ∈ Θ H \theta\in\Theta_{H} θ ∈ Θ H . In particular, it holds that inf ϑ ∈ Θ H E [ ℓ F ( F ‾ t ; ϑ ) ] = K T \inf_{\vartheta\in\Theta_{H}}\mathbb{E}\left[\ell_{F}\left(\overline{F}_{t};\vartheta\right)\right]=K_{T} inf ϑ ∈ Θ H E [ ℓ F ( F t ; ϑ ) ] = K T . Consequently, (151) implies, on the event H ^ ∈ G L ( r ) \widehat{H}\in GL(r) H ∈ G L ( r ) , that
E [ ℓ F ( F ‾ t ; r H ^ − 1 ( θ ^ ) ) ] ≥ K T − 2 δ N T , (152) \mathbb{E}\left[\ell_{F}\left(\overline{F}_{t};r_{\widehat{H}}^{-1}(\widehat{\theta})\right)\right]\ge K_{T}-2\delta_{NT}, \tag*{(152)} E [ ℓ F ( F t ; r H − 1 ( θ ) ) ] ≥ K T − 2 δ N T , (152)
almost surely.
Hereafter, let m Θ ( θ ^ , Θ H ) = inf ϑ ∈ Θ H ∥ θ ^ − ϑ ∥ m_{\Theta}\left(\widehat{\theta},\Theta_{H}\right)=\inf_{\vartheta\in\Theta_{H}}\left\|\widehat{\theta}-\vartheta\right\| m Θ ( θ , Θ H ) = inf ϑ ∈ Θ H θ − ϑ . Now, fix ε > 0 \varepsilon>0 ε > 0 , then, by Assumption 10(iv), there exist η ε > 0 \eta_{\varepsilon}>0 η ε > 0 and T ε < ∞ T_{\varepsilon}<\infty T ε < ∞ such that, for every T ≥ T ε T\ge T_{\varepsilon} T ≥ T ε ,
K T − sup θ ∈ Θ : m Θ ( θ , Θ H ) ≥ ε E [ ℓ F ( F ‾ t ; θ ) ] ≥ η ε . (153) K_{T}-\sup_{\theta\in\Theta:m_{\Theta}(\theta,\Theta_{H})\ge\varepsilon}\mathbb{E}\left[\ell_{F}\left(\overline{F}_{t};\theta\right)\right]\ge\eta_{\varepsilon}. \tag*{(153)} K T − θ ∈ Θ : m Θ ( θ , Θ H ) ≥ ε sup E [ ℓ F ( F t ; θ ) ] ≥ η ε . (153)
Hence, the population criterion equals K T K_T K T on Θ H \boldsymbol{\Theta}_H Θ H and is uniformly smaller than K T K_T K T outside every fixed neighbourhood of Θ H \boldsymbol{\Theta}_H Θ H . Then, on the event { m Θ ( r H ^ − 1 ( θ ^ ) , Θ H ) ≥ ε } \left\{m_{\boldsymbol{\Theta}}\left(r_{\widehat{H}}^{-1}(\widehat{\theta}),\boldsymbol{\Theta}_H\right)\ge\varepsilon\right\} { m Θ ( r H − 1 ( θ ) , Θ H ) ≥ ε } , the definition of the supremum in (153) implies that
E [ ℓ F ( F ‾ t ; r H ^ − 1 ( θ ^ ) ) ] ≤ sup θ ∈ Θ : m Θ ( θ , Θ H ) ≥ ε E [ ℓ F ( F ‾ t ; θ ) ] ≤ K T − η ε , (154) \mathbb{E}\left[\ell_F\left(\overline{F}_t;r_{\widehat{H}}^{-1}(\widehat{\theta})\right)\right]
\le\sup_{\theta\in\boldsymbol{\Theta}:m_{\boldsymbol{\Theta}}(\theta,\boldsymbol{\Theta}_H)\ge\varepsilon}\mathbb{E}\left[\ell_F\left(\overline{F}_t;\theta\right)\right]
\le K_T-\eta_\varepsilon, \tag*{(154)} E [ ℓ F ( F t ; r H − 1 ( θ ) ) ] ≤ θ ∈ Θ : m Θ ( θ , Θ H ) ≥ ε sup E [ ℓ F ( F t ; θ ) ] ≤ K T − η ε , (154)
almost surely. Combining (152) and (154) gives, on the event m Θ ( r H ^ − 1 ( θ ^ ) , Θ H ) ≥ ε m_{\boldsymbol{\Theta}}\left(r_{\widehat{H}}^{-1}(\widehat{\theta}),\boldsymbol{\Theta}_H\right)\ge\varepsilon m Θ ( r H − 1 ( θ ) , Θ H ) ≥ ε , that K T − 2 δ N T ≤ K T − η ε K_T-2\delta_{NT}\le K_T-\eta_\varepsilon K T − 2 δ N T ≤ K T − η ε , and hence 2 δ N T ≥ η ε 2\delta_{NT}\ge\eta_\varepsilon 2 δ N T ≥ η ε almost surely. Consequently,
P ( m Θ ( r H ^ − 1 ( θ ^ ) , Θ H ) ≥ ε ) ≤ P ( 2 δ N T ≥ η ε ) + P ( H ^ ∉ G L ( r ) ) ⟶ 0 \mathbb{P}\left(m_{\boldsymbol{\Theta}}\left(r_{\widehat{H}}^{-1}(\widehat{\theta}),\boldsymbol{\Theta}_H\right)\ge\varepsilon\right)
\le\mathbb{P}(2\delta_{NT}\ge\eta_\varepsilon)+\mathbb{P}\left(\widehat{H}\notin GL(r)\right)\longrightarrow0 P ( m Θ ( r H − 1 ( θ ) , Θ H ) ≥ ε ) ≤ P ( 2 δ N T ≥ η ε ) + P ( H ∈ / G L ( r ) ) ⟶ 0
as N , T → ∞ N,T\to\infty N , T → ∞ , where the convergence follows from (150) and the fact that
P ( H ^ ∉ G L ( r ) ) ⟶ 0 , N , T → ∞ . \mathbb{P}\left(\widehat{H}\notin GL(r)\right)\longrightarrow0,\qquad N,T\to\infty. P ( H ∈ / G L ( r ) ) ⟶ 0 , N , T → ∞.
Therefore,
m Θ ( r H ^ − 1 ( θ ^ ) , Θ H ) = o P ( 1 ) , N , T → ∞ . (155) m_{\boldsymbol{\Theta}}\left(r_{\widehat{H}}^{-1}(\widehat{\theta}),\boldsymbol{\Theta}_H\right)=o_P(1),\qquad N,T\to\infty. \tag*{(155)} m Θ ( r H − 1 ( θ ) , Θ H ) = o P ( 1 ) , N , T → ∞. (155)
We now transfer (155) to θ ^ \widehat{\theta} θ . Since the mapping r H ( ⋅ ) \mathbf{r}_H(\cdot) r H ( ⋅ ) satisfies
r H 1 ∘ r H 2 = r H 1 H 2 , and r H − 1 = r H − 1 , \mathbf{r}_{H_1}\circ\mathbf{r}_{H_2}=\mathbf{r}_{H_1H_2},\qquad\text{and}\qquad\mathbf{r}_H^{-1}=\mathbf{r}_{H^{-1}}, r H 1 ∘ r H 2 = r H 1 H 2 , and r H − 1 = r H − 1 ,
for any H 1 , H 2 ∈ G L ( r ) H_1,H_2\in GL(r) H 1 , H 2 ∈ G L ( r ) they preserve the orbit. In particular, on the event H ^ ∈ G L ( r ) \widehat{H}\in GL(r) H ∈ G L ( r ) , we have that
r H ^ ( Θ H ) = Θ H , (156) \mathbf{r}_{\widehat{H}}(\boldsymbol{\Theta}_H)=\boldsymbol{\Theta}_H, \tag*{(156)} r H ( Θ H ) = Θ H , (156)
almost surely. Indeed, if ϑ = r G ( θ 0 ) ∈ Θ H \vartheta=\mathbf{r}_G(\theta_0)\in\boldsymbol{\Theta}_H ϑ = r G ( θ 0 ) ∈ Θ H for some G ∈ G L ( r ) G\in GL(r) G ∈ G L ( r ) , then
r H ^ ( ϑ ) = r H ^ ( r G ( θ 0 ) ) = r H ^ G ( θ 0 ) ∈ Θ H , \mathbf{r}_{\widehat{H}}(\vartheta)=\mathbf{r}_{\widehat{H}}\left(\mathbf{r}_G(\theta_0)\right)=\mathbf{r}_{\widehat{H}G}(\theta_0)\in\boldsymbol{\Theta}_H, r H ( ϑ ) = r H ( r G ( θ 0 ) ) = r H G ( θ 0 ) ∈ Θ H ,
and the reverse inclusion follows by applying the same argument to r H ^ − 1 = r H ^ − 1 \mathbf{r}_{\widehat{H}}^{-1}=\mathbf{r}_{\widehat{H}^{-1}} r H − 1 = r H − 1 . Moreover, H ^ → P H 0 \widehat{H}\xrightarrow{\mathbb{P}}H_0 H P H 0 as N , T → ∞ N,T\to\infty N , T → ∞ by Proposition 1, and the mapping r H ( θ ) \mathbf{r}_H(\theta) r H ( θ ) is jointly continuous in H H H and θ \theta θ , uniformly over the compact parameter space Θ \boldsymbol{\Theta} Θ for H H H in a fixed neighbourhood of H 0 H_0 H 0 . Hence, uniformly for H ^ \widehat{H} H in that neighbourhood, parameters that are arbitrarily close to Θ H \Theta_H Θ H are mapped into parameters that are arbitrarily close to r H ^ ( Θ H ) = Θ H r_{\widehat{H}}(\Theta_H)=\Theta_H r H ( Θ H ) = Θ H .
Since, on the event H ^ ∈ G L ( r ) \widehat{H}\in GL(r) H ∈ G L ( r ) , θ ^ = r H ^ ( r H ^ − 1 ( θ ) ) \widehat{\theta}=r_{\widehat{H}}\left(r_{\widehat{H}}^{-1}(\theta)\right) θ = r H ( r H − 1 ( θ ) ) almost surely, equations (155) and (156) , together with the uniform continuity of the mapping r H ( ⋅ ) r_H(\cdot) r H ( ⋅ ) , imply
m Θ ( θ ^ , Θ H ) = o P ( 1 ) as N , T → ∞ , m_{\Theta}\left(\widehat{\theta},\Theta_H\right)=o_P(1) \qquad\text{as } N,T\to\infty, m Θ ( θ , Θ H ) = o P ( 1 ) as N , T → ∞ ,
as claimed.
This proves the first conclusion of the theorem. To establish the second conclusion, fix the limiting factor orientation H = H 0 H=H_0 H = H 0 , with H 0 H_0 H 0 defined in Proposition 1. By the equivariance of the likelihood, the corresponding population criterion is E [ ℓ F ( H 0 F ‾ t ; θ ) ] \mathbb{E}\left[\ell_F\left(H_0\overline{F}_t;\theta\right)\right] E [ ℓ F ( H 0 F t ; θ ) ] . The nondegeneracy of the score-driven process, together with ∣ K ∣ ≠ 0 \lvert K\rvert\ne0 ∣ K ∣ = 0 in Assumption 10 ( i ) 10(i) 10 ( i ) , implies that this criterion is uniquely maximized at r H 0 ( θ 0 ) r_{H_0}(\theta_0) r H 0 ( θ 0 ) . Indeed, equality between the population criteria can hold only if the corresponding conditional densities coincide almost surely. The nondegeneracy of the score-driven process then implies equality of the associated filtered processes and, since ∣ K ∣ ≠ 0 \lvert K\rvert\ne0 ∣ K ∣ = 0 , equality of their static parameters. Hence, θ = r H 0 ( θ 0 ) \theta=r_{H_0}(\theta_0) θ = r H 0 ( θ 0 ) . Moreover, Proposition 3, the equivariance of the likelihood, H ^ → P H 0 \widehat{H}\mathrel{\mathop{\to}\limits^{P}}H_0 H → P H 0 as N , T → ∞ N,T\to\infty N , T → ∞ from (51) in the Proof of Proposition 1, and the uniform continuity of the mapping r H ( ⋅ ) r_H(\cdot) r H ( ⋅ ) over the compact parameter space imply that
sup θ ∈ Θ ∣ 1 T ∑ t = 1 T ℓ ^ F ( F ^ t ; θ ) + log ∣ H ^ ∣ − E [ ℓ F ( H 0 F ‾ t ; θ ) ] − log ∣ H 0 ∣ ∣ = o P ( 1 ) \sup_{\theta\in\Theta}\left\lvert\frac{1}{T}\sum_{t=1}^{T}\widehat{\ell}_F\left(\widehat{F}_t;\theta\right)+\log\lvert\widehat{H}\rvert-\mathbb{E}\left[\ell_F\left(H_0\overline{F}_t;\theta\right)\right]-\log\lvert H_0\rvert\right\rvert=o_P(1) θ ∈ Θ sup T 1 t = 1 ∑ T ℓ F ( F t ; θ ) + log ∣ H ∣ − E [ ℓ F ( H 0 F t ; θ ) ] − log ∣ H 0 ∣ = o P ( 1 )
as N , T → ∞ N,T\to\infty N , T → ∞ . Since the two logarithmic terms do not depend on θ \theta θ , they do not affect the corresponding maximizers. Therefore, by the standard argmax argument, together with the unique maximization of the population criterion at r H 0 ( θ 0 ) r_{H_0}(\theta_0) r H 0 ( θ 0 ) , we get
θ ^ − r H 0 ( θ 0 ) = o P ( 1 ) \widehat{\theta}-r_{H_0}(\theta_0)=o_P(1) θ − r H 0 ( θ 0 ) = o P ( 1 )
as N , T → ∞ N,T\to\infty N , T → ∞ . This proves the second conclusion.
Finally, since H ^ → P H 0 \widehat{H}\mathrel{\mathop{\to}\limits^{P}}H_0 H → P H 0 as N , T → ∞ N,T\to\infty N , T → ∞ by Proposition 1, and the inverse mapping r H − 1 ( ⋅ ) r_H^{-1}(\cdot) r H − 1 ( ⋅ ) is continuous, the above also implies that
r H ^ − 1 ( θ ^ ) − θ 0 = o P ( 1 ) r_{\widehat{H}}^{-1}(\widehat{\theta})-\theta_0=o_P(1) r H − 1 ( θ ) − θ 0 = o P ( 1 )
as N , T → ∞ N,T\to\infty N , T → ∞ . This completes the proof.
Proof of Theorem 2.
Since θ 0 \theta_0 θ 0 lies in the interior of Θ \boldsymbol{\Theta} Θ by Assumption 13, Theorem 1 implies that the first-order condition holds with probability approaching one as N , T → ∞ N,T \to\infty N , T → ∞ . By Lemma 9 , uniformly over θ ∈ Θ \theta\in\boldsymbol{\Theta} θ ∈ Θ ,
∥ 1 T ∑ t = 1 T ∇ θ ℓ ^ F ( F ^ t ; θ ) − 1 T ∑ t = 1 T R H ^ T ∇ θ ℓ F ( F t ; r H ^ − 1 ( θ ) ) ∥ = O P ( 1 N + T N + 1 T ) = o P ( 1 ) \left\Vert\frac{1}{\sqrt{T}}\sum_{t=1}^{T}\nabla_{\theta}\widehat{\ell}_{F}\left(\widehat{F}_{t};\theta\right)-\frac{1}{\sqrt{T}}\sum_{t=1}^{T}R_{\widehat{H}}^{\mathsf{T}}\nabla^{\theta}\ell_{F}\left(F_t;r_{\widehat{H}}^{-1}(\theta)\right)\right\Vert=O_{\mathrm{P}}\left(\frac{1}{\sqrt{N}}+\frac{\sqrt{T}}{N}+\frac{1}{\sqrt{T}}\right)=o_{\mathrm{P}}(1) T 1 t = 1 ∑ T ∇ θ ℓ F ( F t ; θ ) − T 1 t = 1 ∑ T R H T ∇ θ ℓ F ( F t ; r H − 1 ( θ ) ) = O P ( N 1 + N T + T 1 ) = o P ( 1 )
as N , T → ∞ N,T \to\infty N , T → ∞ , where the last equality follows from T / N → 0 \sqrt{T}/N \to0 T / N → 0 . Hence, using the first-order condition, and by the change of variables induced by r H ^ − 1 r_{\widehat{H}}^{-1} r H − 1 , we have
0 d = 1 T ∑ t = 1 T ∇ θ ℓ ^ F ( F ^ t ; θ ^ ) = 1 T ∑ t = 1 T R H ^ T ∇ θ ℓ F ( F t ; r H ^ − 1 ( θ ^ ) ) + o P ( 1 ) \mathbf{0}_d=\frac{1}{\sqrt{T}}\sum_{t=1}^{T}\nabla_{\theta}\widehat{\ell}_{F}\left(\widehat{F}_{t};\widehat{\theta}\right)=\frac{1}{\sqrt{T}}\sum_{t=1}^{T}R_{\widehat{H}}^{\mathsf{T}}\nabla^{\theta}\ell_{F}\left(F_t;r_{\widehat{H}}^{-1}(\widehat{\theta})\right)+o_{\mathrm{P}}(1) 0 d = T 1 t = 1 ∑ T ∇ θ ℓ F ( F t ; θ ) = T 1 t = 1 ∑ T R H T ∇ θ ℓ F ( F t ; r H − 1 ( θ ) ) + o P ( 1 )
as N , T → ∞ N,T \to\infty N , T → ∞ , where R H = ∂ ∂ θ r H − 1 ( θ ) R_H=\frac{\partial}{\partial\theta}r_H^{-1}(\theta) R H = ∂ θ ∂ r H − 1 ( θ ) denotes the Jacobian of the inverse transformation mapping. For every fixed H \boldsymbol{H} H such that r H r_H r H defines an invertible reparameterization of the maintained model, the explicit transformations reported in Appendix C are linear in θ \theta θ . Therefore, R H R_H R H does not depend on θ \theta θ .
A Taylor expansion around r H ^ ( θ 0 ) r_{\widehat{H}}(\theta_0) r H ( θ 0 ) , followed by a rescaling by T / T \sqrt{T}/T T / T , yields
o P ( 1 ) = 1 T ∑ t = 1 T R H ^ T ∇ θ ℓ F ( F t ; θ 0 ) + 1 T ∑ t = 1 T R H ^ T ∇ θ θ ℓ F ( F t ; r H ^ − 1 ( θ ^ ∗ ) ) R H ^ T ( θ ^ − r H ^ ( θ 0 ) ) , o_{\mathrm{P}}(1)=\frac{1}{\sqrt{T}}\sum_{t=1}^{T}R_{\widehat{H}}^{\mathsf{T}}\nabla^{\theta}\ell_F(F_t;\theta_0)+\frac{1}{T}\sum_{t=1}^{T}R_{\widehat{H}}^{\mathsf{T}}\nabla^{\theta\theta}\ell_F\left(F_t;r_{\widehat{H}}^{-1}(\widehat{\theta}^{*})\right)R_{\widehat{H}}\sqrt{T}\left(\widehat{\theta}-r_{\widehat{H}}(\theta_0)\right), o P ( 1 ) = T 1 t = 1 ∑ T R H T ∇ θ ℓ F ( F t ; θ 0 ) + T 1 t = 1 ∑ T R H T ∇ θ θ ℓ F ( F t ; r H − 1 ( θ ∗ ) ) R H T ( θ − r H ( θ 0 ) ) ,
where θ ^ ∗ \widehat{\theta}^{*} θ ∗ lies between θ ^ \widehat{\theta} θ and r H ^ ( θ 0 ) r_{\widehat{H}}(\theta_0) r H ( θ 0 ) , so that ∥ r H ^ − 1 ( θ ^ ∗ ) − θ 0 ∥ ≤ ∥ r H ^ − 1 ( θ ^ ) − θ 0 ∥ \left\Vert r_{\widehat{H}}^{-1}(\widehat{\theta}^{*})-\theta_0\right\Vert\leq\left\Vert r_{\widehat{H}}^{-1}(\widehat{\theta})-\theta_0\right\Vert r H − 1 ( θ ∗ ) − θ 0 ≤ r H − 1 ( θ ) − θ 0 . Since r H ^ − 1 ( θ ^ ) − θ 0 = o P ( 1 ) r_{\widehat{H}}^{-1}(\widehat{\theta})-\theta_0=o_{\mathrm{P}}(1) r H − 1 ( θ ) − θ 0 = o P ( 1 ) as N , T → ∞ N,T\to\infty N , T → ∞ by Theorem 1 and H ^ → P H 0 \widehat{H}\xrightarrow{\mathrm{P}}H_0 H P H 0 as N , T → ∞ N,T\to\infty N , T → ∞ by Proposition 1, it also follows that r H ^ − 1 ( θ ^ ∗ ) → P θ 0 r_{\widehat{H}}^{-1}(\widehat{\theta}^{*})\xrightarrow{\mathrm{P}}\theta_0 r H − 1 ( θ ∗ ) P θ 0 , as N , T → ∞ N,T\to\infty N , T → ∞ .
Moreover, since R H ^ → P R H 0 R_{\widehat{H}}\xrightarrow{\mathrm{P}}R_{H_0} R H P R H 0 as N , T → ∞ N,T\to\infty N , T → ∞ , and 1 T ∑ t = 1 T ∇ θ ℓ F ( F t ; θ 0 ) = O P ( 1 ) \frac{1}{\sqrt{T}}\sum_{t=1}^{T}\nabla^{\theta}\ell_F(F_t;\theta_0)=O_{\mathrm{P}}(1) T 1 ∑ t = 1 T ∇ θ ℓ F ( F t ; θ 0 ) = O P ( 1 ) by Lemma 11, we obtain
1 T ∑ t = 1 T R H 0 T ∇ θ ℓ F ( F t ; θ 0 ) + o P ( 1 ) = − 1 T ∑ t = 1 T R H ^ T ∇ θ θ ℓ F ( F t ; r H ^ − 1 ( θ ^ ∗ ) ) R H ^ T ( θ ^ − r H ^ ( θ 0 ) ) . (157) \begin{aligned}
\frac{1}{\sqrt{T}}\sum_{t=1}^{T}R_{H_0}^{\mathsf{T}}\nabla^{\theta}\ell_F(F_t;\theta_0)+o_{\mathrm{P}}(1) \\
&=-\frac{1}{T}\sum_{t=1}^{T}R_{\widehat{H}}^{\mathsf{T}}\nabla^{\theta\theta}\ell_F\left(F_t;r_{\widehat{H}}^{-1}(\widehat{\theta}^{*})\right)R_{\widehat{H}}\sqrt{T}\left(\widehat{\theta}-r_{\widehat{H}}(\theta_0)\right). \tag*{(157)}
\end{aligned} T 1 t = 1 ∑ T R H 0 T ∇ θ ℓ F ( F t ; θ 0 ) + o P ( 1 ) = − T 1 t = 1 ∑ T R H T ∇ θ θ ℓ F ( F t ; r H − 1 ( θ ∗ ) ) R H T ( θ − r H ( θ 0 ) ) . (157)
By Lemma 10,
sup θ ∈ Θ ∥ 1 T ∑ t = 1 T ∇ θ θ ℓ F ( F t ; θ ) − E [ ∇ θ θ ℓ F ( F ‾ t ; θ ) ] ∥ → a . s . 0 (158) \sup_{\theta\in\boldsymbol{\Theta}}\left\Vert\frac{1}{T}\sum_{t=1}^{T}\nabla^{\theta\theta}\ell_F(F_t;\theta)-\mathbb{E}\left[\nabla^{\theta\theta}\ell_F(\overline{F}_t;\theta)\right]\right\Vert\xrightarrow{\mathrm{a.s.}}0 \tag*{(158)} θ ∈ Θ sup T 1 t = 1 ∑ T ∇ θ θ ℓ F ( F t ; θ ) − E [ ∇ θ θ ℓ F ( F t ; θ ) ] a.s. 0 (158)
as T → ∞ T\to\infty T → ∞ . Moreover, since r H ^ − 1 ( θ ^ ∗ ) → P θ 0 r_{\widehat{H}}^{-1}(\widehat{\theta}^{*})\xrightarrow{\mathrm{P}}\theta_0 r H − 1 ( θ ∗ ) P θ 0 as N , T → ∞ N,T\to\infty N , T → ∞ , stationarity, continuity, and the domination condition in Assumption 11 imply that
− E [ ∇ θ ℓ F ( F ˉ t ; r H ^ − 1 ( θ ^ ) ) ] = − E [ ∇ θ ℓ F ( F ˉ t ; r H ^ − 1 ( θ ^ ) ) ] → P I ( θ 0 ) (159) -\mathbb{E}\left[\nabla^{\theta}\ell_F\left(\bar{F}_t;r_{\widehat{H}}^{-1}\left(\hat{\theta}\right)\right)\right]=-\mathbb{E}\left[\nabla^{\theta}\ell_F\left(\bar{F}_t;r_{\widehat{H}}^{-1}\left(\hat{\theta}\right)\right)\right]\xrightarrow{\mathbb{P}}\mathcal{I}\left(\theta_0\right) \tag*{(159)} − E [ ∇ θ ℓ F ( F ˉ t ; r H − 1 ( θ ^ ) ) ] = − E [ ∇ θ ℓ F ( F ˉ t ; r H − 1 ( θ ^ ) ) ] P I ( θ 0 ) (159)
as N , T → ∞ N,T \to\infty N , T → ∞ .
Combining (158) and (159) , and using R H ^ → P R H 0 R_{\widehat{H}}\xrightarrow{\mathbb{P}}R_{H_0} R H P R H 0 as N , T → ∞ N,T\to\infty N , T → ∞ ,
− 1 T ∑ t = 1 T R H ^ T ∇ θ ℓ F ( F t ; r H ^ − 1 ( θ ^ ) ) R H ^ → P R H 0 T I ( θ 0 ) R H 0 = I H 0 ( θ 0 ) (160) -\frac{1}{T}\sum_{t=1}^{T}R_{\widehat{H}}^{\mathsf{T}}\nabla^{\theta}\ell_F\left(F_t;r_{\widehat{H}}^{-1}\left(\hat{\theta}\right)\right)R_{\widehat{H}}\xrightarrow{\mathbb{P}}R_{H_0}^{\mathsf{T}}\mathcal{I}\left(\theta_0\right)R_{H_0}=\mathcal{I}_{H_0}\left(\theta_0\right) \tag*{(160)} − T 1 t = 1 ∑ T R H T ∇ θ ℓ F ( F t ; r H − 1 ( θ ^ ) ) R H P R H 0 T I ( θ 0 ) R H 0 = I H 0 ( θ 0 ) (160)
as N , T → ∞ N,T\to\infty N , T → ∞ .
Since r H 0 − 1 r_{H_0}^{-1} r H 0 − 1 is continuously differentiable and invertible, R H 0 R_{H_0} R H 0 has full rank. Hence, the positive definiteness of I ( θ 0 ) \mathcal{I}(\theta_0) I ( θ 0 ) implies that I H 0 ( θ 0 ) \mathcal{I}_{H_0}(\theta_0) I H 0 ( θ 0 ) is positive definite and invertible. We can therefore solve (157) to obtain
T ( θ ^ − r H ^ ( θ 0 ) ) = I H 0 − 1 ( θ 0 ) ( 1 T ∑ t = 1 T R H 0 T ∇ θ ℓ F ( F t ; θ 0 ) + o P ( 1 ) ) . (161) \sqrt{T}\left(\hat{\theta}-r_{\widehat{H}}(\theta_0)\right)=\mathcal{I}_{H_0}^{-1}\left(\theta_0\right)\left(\frac{1}{\sqrt{T}}\sum_{t=1}^{T}R_{H_0}^{\mathsf{T}}\nabla^{\theta}\ell_F\left(F_t;\theta_0\right)+o_{\mathbb{P}}(1)\right). \tag*{(161)} T ( θ ^ − r H ( θ 0 ) ) = I H 0 − 1 ( θ 0 ) ( T 1 t = 1 ∑ T R H 0 T ∇ θ ℓ F ( F t ; θ 0 ) + o P ( 1 ) ) . (161)
By Lemma 11,
1 T ∑ t = 1 T ∇ θ ℓ F ( F t ; θ 0 ) ⇒ N ( 0 d , V ( θ 0 ) ) \frac{1}{\sqrt{T}}\sum_{t=1}^{T}\nabla^{\theta}\ell_F\left(F_t;\theta_0\right)\Rightarrow\mathcal{N}\left(0_d,V\left(\theta_0\right)\right) T 1 t = 1 ∑ T ∇ θ ℓ F ( F t ; θ 0 ) ⇒ N ( 0 d , V ( θ 0 ) )
as T → ∞ T\to\infty T → ∞ , and hence,
1 T ∑ t = 1 T R H 0 T ∇ θ ℓ F ( F t ; θ 0 ) ⇒ N ( 0 d , V H 0 ( θ 0 ) ) \frac{1}{\sqrt{T}}\sum_{t=1}^{T}R_{H_0}^{\mathsf{T}}\nabla^{\theta}\ell_F\left(F_t;\theta_0\right)\Rightarrow\mathcal{N}\left(0_d,V_{H_0}\left(\theta_0\right)\right) T 1 t = 1 ∑ T R H 0 T ∇ θ ℓ F ( F t ; θ 0 ) ⇒ N ( 0 d , V H 0 ( θ 0 ) )
as T → ∞ T\to\infty T → ∞ , where
V H 0 ( θ 0 ) = R H 0 T V ( θ 0 ) R H 0 . V_{H_0}\left(\theta_0\right)=R_{H_0}^{\mathsf{T}}V\left(\theta_0\right)R_{H_0}. V H 0 ( θ 0 ) = R H 0 T V ( θ 0 ) R H 0 .
Therefore, by Slutsky’s theorem,
T ( θ ^ − r H ^ ( θ 0 ) ) ⇒ N ( 0 d , I H 0 − 1 ( θ 0 ) V H 0 ( θ 0 ) I H 0 − 1 ( θ 0 ) ) \sqrt{T}\left(\hat{\theta}-r_{\widehat{H}}\left(\theta_0\right)\right)\Rightarrow\mathcal{N}\left(0_d,\mathcal{I}_{H_0}^{-1}\left(\theta_0\right)V_{H_0}\left(\theta_0\right)\mathcal{I}_{H_0}^{-1}\left(\theta_0\right)\right) T ( θ ^ − r H ( θ 0 ) ) ⇒ N ( 0 d , I H 0 − 1 ( θ 0 ) V H 0 ( θ 0 ) I H 0 − 1 ( θ 0 ) )
as N , T → ∞ N,T\to\infty N , T → ∞ . This completes the proof. ■
Proof of Theorem 3
The first-order condition and a mean-value expansion, together with Lemmas 9 and 10, imply, as N , T → ∞ N,T\to\infty N , T → ∞ , that
r H ^ − 1 ( θ ^ ) − θ 0 = O P ( 1 N + 1 T ) . r_{\widehat{H}}^{-1}\left(\hat{\theta}\right)-\theta_0=O_{\mathbb{P}}\left(\frac{1}{N}+\frac{1}{\sqrt{T}}\right). r H − 1 ( θ ^ ) − θ 0 = O P ( N 1 + T 1 ) .
Since the alignment mapping is continuously differentiable with a uniformly bounded Jacobian for H ^ \widehat{H} H in a neighbourhood of H 0 H_0 H 0 , it follows that
θ ^ − r H ^ ( θ 0 ) = O P ( 1 N + 1 T ) . (162) \widehat{\theta}-r_{\widehat{H}}(\theta_0)=O_{\mathbb{P}}\left(\frac{1}{N}+\frac{1}{\sqrt{T}}\right). \tag*{(162)} θ − r H ( θ 0 ) = O P ( N 1 + T 1 ) . (162)
For the conditional mean, consider the difference
C ^ i , t ∣ t − 1 ( θ ^ ) − C ^ i , t ∣ t − 1 ( θ 0 ) = λ ^ i T μ ^ t ∣ t − 1 ( θ ^ ) − λ 0 i T μ t ∣ t − 1 ( θ 0 ) = ( λ ^ i T − λ 0 i T H ^ − 1 ) μ ^ t ∣ t − 1 ( θ ^ ) + λ 0 i T ( H ^ − 1 μ ^ t ∣ t − 1 ( θ ^ ) − μ t ∣ t − 1 ( θ 0 ) ) . (163) \begin{aligned}
\widehat{C}_{i,t|t-1}(\widehat{\theta})-\widehat{C}_{i,t|t-1}(\theta_0)=\widehat{\lambda}_i^{\mathsf{T}}\widehat{\mu}_{t|t-1}(\widehat{\theta})-\lambda_{0i}^{\mathsf{T}}\mu_{t|t-1}(\theta_0) \\
&=\left(\widehat{\lambda}_i^{\mathsf{T}}-\lambda_{0i}^{\mathsf{T}}\widehat{H}^{-1}\right)\widehat{\mu}_{t|t-1}(\widehat{\theta})+\lambda_{0i}^{\mathsf{T}}\left(\widehat{H}^{-1}\widehat{\mu}_{t|t-1}(\widehat{\theta})-\mu_{t|t-1}(\theta_0)\right). \tag*{(163)}
\end{aligned} C i , t ∣ t − 1 ( θ ) − C i , t ∣ t − 1 ( θ 0 ) = λ i T μ t ∣ t − 1 ( θ ) − λ 0 i T μ t ∣ t − 1 ( θ 0 ) = ( λ i T − λ 0 i T H − 1 ) μ t ∣ t − 1 ( θ ) + λ 0 i T ( H − 1 μ t ∣ t − 1 ( θ ) − μ t ∣ t − 1 ( θ 0 ) ) . (163)
By adding and removing the aligned filters, the first term on the right-hand side of (163) can be rewritten as
( λ ^ i T − λ 0 i T H ^ − 1 ) μ ^ t ∣ t − 1 ( θ ^ ) = ( λ ^ i T − λ 0 i T H ^ − 1 ) H ^ μ t ∣ t − 1 ( θ 0 ) + ( λ ^ i T − λ 0 i T H ^ − 1 ) ( μ ^ t ∣ t − 1 ( θ ^ ) − μ ^ t ∣ t − 1 ( r H ^ ( θ 0 ) ) ) + ( λ ^ i T − λ 0 i T H ^ − 1 ) ( μ ^ t ∣ t − 1 ( r H ^ ( θ 0 ) ) − μ ^ t ∣ t − 1 ( r H ^ ( θ 0 ) ) ) + ( λ ^ i T − λ 0 i T H ^ − 1 ) ( μ ^ t ∣ t − 1 ( r H ^ ( θ 0 ) ) − H ^ μ t ∣ t − 1 ( θ 0 ) ) = O P ( max ( 1 N , 1 T ) ) + O P ( max ( 1 N , 1 T ) ) O P ( max ( 1 N , 1 T ) + K [ t / k ] ) . (164) \begin{aligned}
\left(\widehat{\lambda}_i^{\mathsf{T}}-\lambda_{0i}^{\mathsf{T}}\widehat{H}^{-1}\right)\widehat{\mu}_{t|t-1}(\widehat{\theta})
={}&\left(\widehat{\lambda}_i^{\mathsf{T}}-\lambda_{0i}^{\mathsf{T}}\widehat{H}^{-1}\right)\widehat{H}\mu_{t|t-1}(\theta_0)\\
&+\left(\widehat{\lambda}_i^{\mathsf{T}}-\lambda_{0i}^{\mathsf{T}}\widehat{H}^{-1}\right)\left(\widehat{\mu}_{t|t-1}(\widehat{\theta})-\widehat{\mu}_{t|t-1}\left(r_{\widehat{H}}(\theta_0)\right)\right)\\
&+\left(\widehat{\lambda}_i^{\mathsf{T}}-\lambda_{0i}^{\mathsf{T}}\widehat{H}^{-1}\right)\left(\widehat{\mu}_{t|t-1}\left(r_{\widehat{H}}(\theta_0)\right)-\widehat{\mu}_{t|t-1}\left(r_{\widehat{H}}(\theta_0)\right)\right)\\
&+\left(\widehat{\lambda}_i^{\mathsf{T}}-\lambda_{0i}^{\mathsf{T}}\widehat{H}^{-1}\right)\left(\widehat{\mu}_{t|t-1}\left(r_{\widehat{H}}(\theta_0)\right)-\widehat{H}\mu_{t|t-1}(\theta_0)\right)\\
={}&O_{\mathbb{P}}\left(\max\left(\frac{1}{N},\frac{1}{\sqrt{T}}\right)\right)+O_{\mathbb{P}}\left(\max\left(\frac{1}{N},\frac{1}{\sqrt{T}}\right)\right)O_{\mathbb{P}}\left(\max\left(\frac{1}{\sqrt{N}},\frac{1}{\sqrt{T}}\right)+\mathcal{K}^{[t/k]}\right). \tag*{(164)}
\end{aligned} ( λ i T − λ 0 i T H − 1 ) μ t ∣ t − 1 ( θ ) = = ( λ i T − λ 0 i T H − 1 ) H μ t ∣ t − 1 ( θ 0 ) + ( λ i T − λ 0 i T H − 1 ) ( μ t ∣ t − 1 ( θ ) − μ t ∣ t − 1 ( r H ( θ 0 ) ) ) + ( λ i T − λ 0 i T H − 1 ) ( μ t ∣ t − 1 ( r H ( θ 0 ) ) − μ t ∣ t − 1 ( r H ( θ 0 ) ) ) + ( λ i T − λ 0 i T H − 1 ) ( μ t ∣ t − 1 ( r H ( θ 0 ) ) − H μ t ∣ t − 1 ( θ 0 ) ) O P ( max ( N 1 , T 1 ) ) + O P ( max ( N 1 , T 1 ) ) O P ( max ( N 1 , T 1 ) + K [ t / k ] ) . (164)
The last equality follows from Proposition 1(c), (162) , Lemma 7, Proposition 2(a), and Assumption 7. In particular, the four terms on the right-hand side account, respectively, for the loading-estimation error, the parameter-estimation error, the pointwise factor-filter error, and the effect of the possibly different initial conditions.
Similarly, the second term on the right-hand side of (163) is
λ 0 i T ( H ^ − 1 μ ^ t ∣ t − 1 ( θ ^ ) − μ t ∣ t − 1 ( θ 0 ) ) = λ 0 i T H ^ − 1 ( μ ^ t ∣ t − 1 ( θ ^ ) − μ ^ t ∣ t − 1 ( r H ^ ( θ 0 ) ) ) + λ 0 i T H ^ − 1 ( μ ^ t ∣ t − 1 ( r H ^ ( θ 0 ) ) − μ ^ t ∣ t − 1 ( r H ^ ( θ 0 ) ) ) + λ 0 i T ( H ^ − 1 μ ^ t ∣ t − 1 ( r H ^ ( θ 0 ) ) − μ t ∣ t − 1 ( θ 0 ) ) = O P ( max ( 1 N , 1 T ) ) + O a . s . ( χ k ⌊ t / k ⌋ ) . (165) \begin{aligned}
\lambda_{0i}^{\mathsf{T}}\left(\widehat{H}^{-1}\widehat{\mu}_{t|t-1}(\widehat{\theta})-\mu_{t|t-1}(\theta_0)\right)
={}&\lambda_{0i}^{\mathsf{T}}\widehat{H}^{-1}\left(\widehat{\mu}_{t|t-1}(\widehat{\theta})-\widehat{\mu}_{t|t-1}\left(r_{\widehat{H}}(\theta_0)\right)\right)\\
&+\lambda_{0i}^{\mathsf{T}}\widehat{H}^{-1}\left(\widehat{\mu}_{t|t-1}\left(r_{\widehat{H}}(\theta_0)\right)-\widehat{\mu}_{t|t-1}\left(r_{\widehat{H}}(\theta_0)\right)\right)\\
&+\lambda_{0i}^{\mathsf{T}}\left(\widehat{H}^{-1}\widehat{\mu}_{t|t-1}\left(r_{\widehat{H}}(\theta_0)\right)-\mu_{t|t-1}(\theta_0)\right) \\
&= O_{\mathbb{P}}\left(\max\left(\frac{1}{\sqrt{N}},\frac{1}{\sqrt{T}}\right)\right) + O_{a.s.}\left(\chi_k^{\lfloor t/k\rfloor}\right). \tag*{(165)}
\end{aligned} λ 0 i T ( H − 1 μ t ∣ t − 1 ( θ ) − μ t ∣ t − 1 ( θ 0 ) ) = λ 0 i T H − 1 ( μ t ∣ t − 1 ( θ ) − μ t ∣ t − 1 ( r H ( θ 0 ) ) ) + λ 0 i T H − 1 ( μ t ∣ t − 1 ( r H ( θ 0 ) ) − μ t ∣ t − 1 ( r H ( θ 0 ) ) ) + λ 0 i T ( H − 1 μ t ∣ t − 1 ( r H ( θ 0 ) ) − μ t ∣ t − 1 ( θ 0 ) ) = O P ( max ( N 1 , T 1 ) ) + O a . s . ( χ k ⌊ t / k ⌋ ) . (165)
The last equality follows from (162) , Lemma 7, Proposition 2(a), and Assumption 7. In particular, the three terms on the right-hand side account, respectively, for the parameter-estimation error, the pointwise factor-filter error, and the initialization effect after alignment. The last term decreases geometrically because the two filters are driven by the same sequence { H ^ F s } s = 1 t − 1 \left\{\widehat{H}F_s\right\}_{s=1}^{t-1} { H F s } s = 1 t − 1 , and by the mapping in (32) we get H ^ − 1 μ ^ t ∣ t − 1 ( r H ^ ( θ 0 ) ) − μ t ∣ t − 1 ( θ 0 ) = O a . s . ( χ k ⌊ t / k ⌋ ) \widehat{H}^{-1}\widehat{\mu}_{t|t-1}\left(r_{\widehat{H}}(\theta_0)\right)-\mu_{t|t-1}(\theta_0)=O_{a.s.}\left(\chi_k^{\lfloor t/k\rfloor}\right) H − 1 μ t ∣ t − 1 ( r H ( θ 0 ) ) − μ t ∣ t − 1 ( θ 0 ) = O a . s . ( χ k ⌊ t / k ⌋ ) .
Therefore, substituting (164) and (165) into (163) , as N , T → ∞ N,T\to\infty N , T → ∞ ,
∣ C ^ i , t ∣ t − 1 ( θ ^ ) − C i , t ∣ t − 1 ( θ 0 ) ∣ = O P ( 1 min ( N , T ) + 1 min ( N , T ) ) + O a . s . ( χ k ⌊ t / k ⌋ ) . \left|\widehat{C}_{i,t|t-1}(\widehat{\theta})-C_{i,t|t-1}(\theta_0)\right|=O_{\mathbb{P}}\left(\frac{1}{\min\left(N,\sqrt{T}\right)}+\frac{1}{\min\left(\sqrt{N},T\right)}\right)+O_{a.s.}\left(\chi_k^{\lfloor t/k\rfloor}\right). C i , t ∣ t − 1 ( θ ) − C i , t ∣ t − 1 ( θ 0 ) = O P min ( N , T ) 1 + min ( N , T ) 1 + O a . s . ( χ k ⌊ t / k ⌋ ) .
Analogously, for the conditional variance, we consider the difference
P ^ i , t ∣ t − 1 ( θ ^ ) − P i , t ∣ t − 1 ( θ 0 ) = λ ^ i T Σ ^ t ∣ t − 1 ( θ ^ ) λ ^ i − λ 0 i T Σ t ∣ t − 1 ( θ 0 ) λ 0 i = ( λ ^ i ⊗ λ ^ i ) T vec Σ ^ t ∣ t − 1 ( θ ^ ) − ( λ 0 i ⊗ λ 0 i ) T vec Σ t ∣ t − 1 ( θ 0 ) = ( ( λ ^ i T ⊗ λ ^ i T ) − ( λ 0 i T H ^ − 1 ⊗ λ 0 i T H ^ − 1 ) ) W r σ ^ t ∣ t − 1 2 ( θ ^ ) + ( λ 0 i T ⊗ λ 0 i T ) ( ( H ^ − 1 ⊗ H ^ − 1 ) W r σ ^ t ∣ t − 1 2 ( θ ^ ) − W r σ t ∣ t − 1 2 ( θ 0 ) ) . (166) \begin{aligned}
\widehat{P}_{i,t|t-1}(\widehat{\theta})-P_{i,t|t-1}(\theta_0)
&=\widehat{\lambda}_i^{\mathsf{T}}\widehat{\Sigma}_{t|t-1}(\widehat{\theta})\widehat{\lambda}_i-\lambda_{0i}^{\mathsf{T}}\Sigma_{t|t-1}(\theta_0)\lambda_{0i}\\
&=\left(\widehat{\lambda}_i\otimes\widehat{\lambda}_i\right)^{\mathsf{T}}\operatorname{vec}\widehat{\Sigma}_{t|t-1}(\widehat{\theta})-\left(\lambda_{0i}\otimes\lambda_{0i}\right)^{\mathsf{T}}\operatorname{vec}\Sigma_{t|t-1}(\theta_0)\\
&=\left(\left(\widehat{\lambda}_i^{\mathsf{T}}\otimes\widehat{\lambda}_i^{\mathsf{T}}\right)-\left(\lambda_{0i}^{\mathsf{T}}\widehat{H}^{-1}\otimes\lambda_{0i}^{\mathsf{T}}\widehat{H}^{-1}\right)\right)W_r\widehat{\sigma}_{t|t-1}^{2}(\widehat{\theta})\\
&\quad+\left(\lambda_{0i}^{\mathsf{T}}\otimes\lambda_{0i}^{\mathsf{T}}\right)\left(\left(\widehat{H}^{-1}\otimes\widehat{H}^{-1}\right)W_r\widehat{\sigma}_{t|t-1}^{2}(\widehat{\theta})-W_r\sigma_{t|t-1}^{2}(\theta_0)\right). \tag*{(166)}
\end{aligned} P i , t ∣ t − 1 ( θ ) − P i , t ∣ t − 1 ( θ 0 ) = λ i T Σ t ∣ t − 1 ( θ ) λ i − λ 0 i T Σ t ∣ t − 1 ( θ 0 ) λ 0 i = ( λ i ⊗ λ i ) T vec Σ t ∣ t − 1 ( θ ) − ( λ 0 i ⊗ λ 0 i ) T vec Σ t ∣ t − 1 ( θ 0 ) = ( ( λ i T ⊗ λ i T ) − ( λ 0 i T H − 1 ⊗ λ 0 i T H − 1 ) ) W r σ t ∣ t − 1 2 ( θ ) + ( λ 0 i T ⊗ λ 0 i T ) ( ( H − 1 ⊗ H − 1 ) W r σ t ∣ t − 1 2 ( θ ) − W r σ t ∣ t − 1 2 ( θ 0 ) ) . (166)
Using the same decomposition as for the conditional location, the first term on the right-hand side of (166) can be rewritten as
( ( λ ^ i T ⊗ λ ^ i T ) − ( λ 0 i T H ^ − 1 ⊗ λ 0 i T H ^ − 1 ) ) W r σ ^ t ∣ t − 1 2 ( θ ^ ) = ( ( λ ^ i T ⊗ λ ^ i T ) − ( λ 0 i T H ^ − 1 ⊗ λ 0 i T H ^ − 1 ) ) ( H ^ ⊗ H ^ ) W r σ t ∣ t − 1 2 ( θ 0 ) + ( ( λ ^ i T ⊗ λ ^ i T ) − ( λ 0 i T H ^ − 1 ⊗ λ 0 i T H ^ − 1 ) ) ( W r σ ^ t ∣ t − 1 2 ( θ ^ ) − W r σ ^ t ∣ t − 1 2 ( r H ^ ( θ 0 ) ) ) + ( ( λ ^ i T ⊗ λ ^ i T ) − ( λ 0 i T H ^ − 1 ⊗ λ 0 i T H ^ − 1 ) ) ( W r σ ^ t ∣ t − 1 2 ( r H ^ ( θ 0 ) ) − W r σ t ∣ t − 1 2 ( r H ^ ( θ 0 ) ) ) + ( ( λ ^ i T ⊗ λ ^ i T ) − ( λ 0 i T H ^ − 1 ⊗ λ 0 i T H ^ − 1 ) ) ( W r σ t ∣ t − 1 2 ( r H ^ ( θ 0 ) ) − ( H ^ ⊗ H ^ ) W r σ t ∣ t − 1 2 ( θ 0 ) ) = O P ( max ( 1 N , 1 T ) ) + O P ( max ( 1 N , 1 T ) ) O P ( max ( 1 N , 1 T ) + χ k ⌊ t / k ⌋ ) . (167) \begin{aligned}
&\left(\left(\widehat{\lambda}_i^{\mathsf{T}}\otimes\widehat{\lambda}_i^{\mathsf{T}}\right)-\left(\lambda_{0i}^{\mathsf{T}}\widehat{H}^{-1}\otimes\lambda_{0i}^{\mathsf{T}}\widehat{H}^{-1}\right)\right)W_r\widehat{\sigma}_{t|t-1}^{2}(\widehat{\theta})\\
&=\left(\left(\widehat{\lambda}_i^{\mathsf{T}}\otimes\widehat{\lambda}_i^{\mathsf{T}}\right)-\left(\lambda_{0i}^{\mathsf{T}}\widehat{H}^{-1}\otimes\lambda_{0i}^{\mathsf{T}}\widehat{H}^{-1}\right)\right)\left(\widehat{H}\otimes\widehat{H}\right)W_r\sigma_{t|t-1}^{2}(\theta_0)\\
&\quad+\left(\left(\widehat{\lambda}_i^{\mathsf{T}}\otimes\widehat{\lambda}_i^{\mathsf{T}}\right)-\left(\lambda_{0i}^{\mathsf{T}}\widehat{H}^{-1}\otimes\lambda_{0i}^{\mathsf{T}}\widehat{H}^{-1}\right)\right)\left(W_r\widehat{\sigma}_{t|t-1}^{2}(\widehat{\theta})-W_r\widehat{\sigma}_{t|t-1}^{2}\left(r_{\widehat{H}}(\theta_0)\right)\right)\\
&\quad+\left(\left(\widehat{\lambda}_i^{\mathsf{T}}\otimes\widehat{\lambda}_i^{\mathsf{T}}\right)-\left(\lambda_{0i}^{\mathsf{T}}\widehat{H}^{-1}\otimes\lambda_{0i}^{\mathsf{T}}\widehat{H}^{-1}\right)\right)\left(W_r\widehat{\sigma}_{t|t-1}^{2}\left(r_{\widehat{H}}(\theta_0)\right)-W_r\sigma_{t|t-1}^{2}\left(r_{\widehat{H}}(\theta_0)\right)\right)\\
&\quad+\left(\left(\widehat{\lambda}_i^{\mathsf{T}}\otimes\widehat{\lambda}_i^{\mathsf{T}}\right)-\left(\lambda_{0i}^{\mathsf{T}}\widehat{H}^{-1}\otimes\lambda_{0i}^{\mathsf{T}}\widehat{H}^{-1}\right)\right)\left(W_r\sigma_{t|t-1}^{2}\left(r_{\widehat{H}}(\theta_0)\right)-\left(\widehat{H}\otimes\widehat{H}\right)W_r\sigma_{t|t-1}^{2}(\theta_0)\right)\\
&=O_{\mathbb{P}}\left(\max\left(\frac{1}{N},\frac{1}{\sqrt{T}}\right)\right)+O_{\mathbb{P}}\left(\max\left(\frac{1}{N},\frac{1}{\sqrt{T}}\right)\right)O_{\mathbb{P}}\left(\max\left(\frac{1}{\sqrt{N}},\frac{1}{\sqrt{T}}\right)+\chi_k^{\lfloor t/k\rfloor}\right). \tag*{(167)}
\end{aligned} ( ( λ i T ⊗ λ i T ) − ( λ 0 i T H − 1 ⊗ λ 0 i T H − 1 ) ) W r σ t ∣ t − 1 2 ( θ ) = ( ( λ i T ⊗ λ i T ) − ( λ 0 i T H − 1 ⊗ λ 0 i T H − 1 ) ) ( H ⊗ H ) W r σ t ∣ t − 1 2 ( θ 0 ) + ( ( λ i T ⊗ λ i T ) − ( λ 0 i T H − 1 ⊗ λ 0 i T H − 1 ) ) ( W r σ t ∣ t − 1 2 ( θ ) − W r σ t ∣ t − 1 2 ( r H ( θ 0 ) ) ) + ( ( λ i T ⊗ λ i T ) − ( λ 0 i T H − 1 ⊗ λ 0 i T H − 1 ) ) ( W r σ t ∣ t − 1 2 ( r H ( θ 0 ) ) − W r σ t ∣ t − 1 2 ( r H ( θ 0 ) ) ) + ( ( λ i T ⊗ λ i T ) − ( λ 0 i T H − 1 ⊗ λ 0 i T H − 1 ) ) ( W r σ t ∣ t − 1 2 ( r H ( θ 0 ) ) − ( H ⊗ H ) W r σ t ∣ t − 1 2 ( θ 0 ) ) = O P ( max ( N 1 , T 1 ) ) + O P ( max ( N 1 , T 1 ) ) O P ( max ( N 1 , T 1 ) + χ k ⌊ t / k ⌋ ) . (167)
The last equality follows from Proposition 1(c), (162) , Lemma 7, Proposition 2(a), and Assumption 7. Moreover, for any conformable matrices A \mathbf{A} A and B \mathbf{B} B , ∥ A ⊗ B ∥ s p = ∥ A ∥ s p ∥ B ∥ s p \lVert\mathbf{A} \otimes\mathbf{B} \rVert_{\mathrm{sp}} = \lVert\mathbf{A} \rVert_{\mathrm{sp}} \lVert\mathbf{B} \rVert_{\mathrm{sp}} ∥ A ⊗ B ∥ sp = ∥ A ∥ sp ∥ B ∥ sp , so that Proposition 1(c) also gives the same rate for the corresponding difference between the Kronecker products of the estimated and rotated loadings.
Furthermore, the second term on the right-hand side of (166) is
( λ 0 i T ⊗ λ 0 i T ) ( ( H ^ − 1 ⊗ H ^ − 1 ) W r σ ^ t ∣ t − 1 2 ( θ ^ ) − W r σ t ∣ t − 1 2 ( θ 0 ) ) = ( λ 0 i T H ^ − 1 ⊗ λ 0 i T H ^ − 1 ) W r ( σ ^ t ∣ t − 1 2 ( θ ^ ) − σ ^ t ∣ t − 1 2 ( r H ^ ( θ 0 ) ) ) + ( λ 0 i T H ^ − 1 ⊗ λ 0 i T H ^ − 1 ) W r ( σ ^ t ∣ t − 1 2 ( r H ^ ( θ 0 ) ) − σ ^ t ∣ t − 1 2 ( r H ^ ( θ 0 ) ) ) + ( λ 0 i T ⊗ λ 0 i T ) ( ( H ^ − 1 ⊗ H ^ − 1 ) W r σ ^ t ∣ t − 1 2 ( r H ^ ( θ 0 ) ) − W r σ t ∣ t − 1 2 ( θ 0 ) ) = O P ( max ( 1 N , 1 T ) ) + O a . s . ( χ k ⌈ t / k ⌉ ) . (168) \begin{aligned}
&\left(\lambda_{0i}^{\mathsf{T}} \otimes\lambda_{0i}^{\mathsf{T}}\right)\left(\left(\widehat{H}^{-1} \otimes\widehat{H}^{-1}\right)W_r\widehat{\sigma}_{t|t-1}^{2}(\widehat{\theta})-W_r\sigma_{t|t-1}^{2}(\theta_0)\right) \\
={}&\left(\lambda_{0i}^{\mathsf{T}}\widehat{H}^{-1} \otimes\lambda_{0i}^{\mathsf{T}}\widehat{H}^{-1}\right)W_r\left(\widehat{\sigma}_{t|t-1}^{2}(\widehat{\theta})-\widehat{\sigma}_{t|t-1}^{2}\left(r_{\widehat{H}}(\theta_0)\right)\right) \\
&+\left(\lambda_{0i}^{\mathsf{T}}\widehat{H}^{-1} \otimes\lambda_{0i}^{\mathsf{T}}\widehat{H}^{-1}\right)W_r\left(\widehat{\sigma}_{t|t-1}^{2}\left(r_{\widehat{H}}(\theta_0)\right)-\widehat{\sigma}_{t|t-1}^{2}\left(r_{\widehat{H}}(\theta_0)\right)\right) \\
&+\left(\lambda_{0i}^{\mathsf{T}}\otimes\lambda_{0i}^{\mathsf{T}}\right)\left(\left(\widehat{H}^{-1}\otimes\widehat{H}^{-1}\right)W_r\widehat{\sigma}_{t|t-1}^{2}\left(r_{\widehat{H}}(\theta_0)\right)-W_r\sigma_{t|t-1}^{2}(\theta_0)\right) \\
={}&O_{\mathbb{P}}\left(\max\left(\frac{1}{\sqrt{N}},\frac{1}{\sqrt{T}}\right)\right)+O_{\mathrm{a.s.}}\left(\chi_k^{\lceil t/k\rceil}\right). \tag*{(168)}
\end{aligned} = = ( λ 0 i T ⊗ λ 0 i T ) ( ( H − 1 ⊗ H − 1 ) W r σ t ∣ t − 1 2 ( θ ) − W r σ t ∣ t − 1 2 ( θ 0 ) ) ( λ 0 i T H − 1 ⊗ λ 0 i T H − 1 ) W r ( σ t ∣ t − 1 2 ( θ ) − σ t ∣ t − 1 2 ( r H ( θ 0 ) ) ) + ( λ 0 i T H − 1 ⊗ λ 0 i T H − 1 ) W r ( σ t ∣ t − 1 2 ( r H ( θ 0 ) ) − σ t ∣ t − 1 2 ( r H ( θ 0 ) ) ) + ( λ 0 i T ⊗ λ 0 i T ) ( ( H − 1 ⊗ H − 1 ) W r σ t ∣ t − 1 2 ( r H ( θ 0 ) ) − W r σ t ∣ t − 1 2 ( θ 0 ) ) O P ( max ( N 1 , T 1 ) ) + O a.s. ( χ k ⌈ t / k ⌉ ) . (168)
The last equality follows from (162) , Lemma 7, Proposition 2(a), and Assumption 7. The three terms on the right-hand side account, respectively, for the parameter-estimation error, the pointwise factor-filter error, and the initialization effect after alignment. The last term decreases geometrically because the two filters are driven by the same sequence { H ^ F s } s = 1 t − 1 \left\{\widehat{H}F_s\right\}_{s=1}^{t-1} { H F s } s = 1 t − 1 , and the mapping in (33) gives ( H ^ − 1 ⊗ H ^ − 1 ) W r σ ^ t ∣ t − 1 2 ( r H ^ ( θ 0 ) ) − W r σ t ∣ t − 1 2 ( θ 0 ) = O a . s . ( χ k ⌈ t / k ⌉ ) \left(\widehat{H}^{-1}\otimes\widehat{H}^{-1}\right)W_r\widehat{\sigma}_{t|t-1}^{2}\left(r_{\widehat{H}}(\theta_0)\right)-W_r\sigma_{t|t-1}^{2}(\theta_0)=O_{\mathrm{a.s.}}\left(\chi_k^{\lceil t/k\rceil}\right) ( H − 1 ⊗ H − 1 ) W r σ t ∣ t − 1 2 ( r H ( θ 0 ) ) − W r σ t ∣ t − 1 2 ( θ 0 ) = O a.s. ( χ k ⌈ t / k ⌉ ) .
Therefore, substituting (167) and (168) into (166) , as N , T → ∞ N,T\to\infty N , T → ∞ ,
∣ P ^ i , t ∣ t − 1 ( θ ^ ) − P i , t ∣ t − 1 ( θ 0 ) ∣ = O P ( 1 min ( N , T ) + 1 min ( N , T ) ) + O a . s . ( χ k ⌈ t / k ⌉ ) . \left|\widehat{P}_{i,t|t-1}(\widehat{\theta})-P_{i,t|t-1}(\theta_0)\right|=O_{\mathbb{P}}\left(\frac{1}{\min\left(N,\sqrt{T}\right)}+\frac{1}{\min\left(\sqrt{N},T\right)}\right)+O_{\mathrm{a.s.}}\left(\chi_k^{\lceil t/k\rceil}\right). P i , t ∣ t − 1 ( θ ) − P i , t ∣ t − 1 ( θ 0 ) = O P min ( N , T ) 1 + min ( N , T ) 1 + O a.s. ( χ k ⌈ t / k ⌉ ) .
Since both stochastic terms are O ( 1 / min ( N , T ) ) O\left(1/\min\left(\sqrt{N},\sqrt{T}\right)\right) O ( 1/ min ( N , T ) ) , the rates stated in Theorem 3 follow. This completes the proof.
Proof of Proposition 4
We first consider the conditional location process. By adding and subtracting H ^ − 1 μ ^ t ∣ t − 1 ( r H ^ ( θ ) ) \widehat{H}^{-1}\widehat{\mu}_{t|t-1}\left(r_{\widehat{H}}(\theta)\right) H − 1 μ t ∣ t − 1 ( r H ( θ ) ) , we obtain
sup θ ∈ Θ ∥ H ^ − 1 μ ^ t ∣ t − 1 ( r H ^ ( θ ) ) − μ t ∣ t − 1 ( θ ) ∥ ≤ ∥ H ^ − 1 ∥ sup θ ∈ Θ ∥ μ ^ t ∣ t − 1 ( r H ^ ( θ ) ) − μ t ∣ t − 1 ( r H ^ ( θ ) ) ∥ + sup θ ∈ Θ ∥ H ^ − 1 μ ^ t ∣ t − 1 ( r H ^ ( θ ) ) − μ t ∣ t − 1 ( θ ) ∥ . (169) \begin{aligned}
\sup_{\theta\in\Theta}\left\lVert\widehat{H}^{-1}\widehat{\mu}_{t|t-1}\left(r_{\widehat{H}}(\theta)\right)-\mu_{t|t-1}(\theta)\right\rVert\\
&\leq\left\lVert\widehat{H}^{-1}\right\rVert\sup_{\theta\in\Theta}\left\lVert\widehat{\mu}_{t|t-1}\left(r_{\widehat{H}}(\theta)\right)-\mu_{t|t-1}\left(r_{\widehat{H}}(\theta)\right)\right\rVert+\sup_{\theta\in\Theta}\left\lVert\widehat{H}^{-1}\widehat{\mu}_{t|t-1}\left(r_{\widehat{H}}(\theta)\right)-\mu_{t|t-1}(\theta)\right\rVert. \tag*{(169)}
\end{aligned} θ ∈ Θ sup H − 1 μ t ∣ t − 1 ( r H ( θ ) ) − μ t ∣ t − 1 ( θ ) ≤ H − 1 θ ∈ Θ sup μ t ∣ t − 1 ( r H ( θ ) ) − μ t ∣ t − 1 ( r H ( θ ) ) + θ ∈ Θ sup H − 1 μ t ∣ t − 1 ( r H ( θ ) ) − μ t ∣ t − 1 ( θ ) . (169)
The second term on the right-hand side of (169) is equal to zero. Indeed, the filter μ ^ t ∣ t − 1 ( r H ^ ( θ ) ) \widehat{\boldsymbol{\mu}}_{t|t-1}\left(r_{\widehat{\mathbf{H}}}(\theta)\right) μ t ∣ t − 1 ( r H ( θ ) ) is driven by H ^ F t \widehat{\mathbf{H}}\mathbf{F}_t H F t , and the mapping in (32) , together with Assumption 14, implies that
μ ^ t ∣ t − 1 ( r H ^ ( θ ) ) = H ^ μ t ∣ t − 1 ( θ ) \widehat{\boldsymbol{\mu}}_{t|t-1}\left(r_{\widehat{\mathbf{H}}}(\theta)\right)=\widehat{\mathbf{H}}\boldsymbol{\mu}_{t|t-1}(\theta) μ t ∣ t − 1 ( r H ( θ ) ) = H μ t ∣ t − 1 ( θ )
for every θ ∈ Θ \theta\in\Theta θ ∈ Θ and every t ≥ 1 t\ge1 t ≥ 1 .
Since r H ^ r_{\widehat{\mathbf{H}}} r H is an invertible reparameterization of Θ \Theta Θ , by Proposition 2(a)
sup θ ∈ Θ ∥ μ ^ t ∣ t − 1 ( r H ^ ( θ ) ) − μ ^ t ∣ t − 1 ( r H ^ ( θ ) ) ∥ = O P ( 1 min ( N , T ) ) . \sup_{\theta\in\Theta}\left\|\widehat{\boldsymbol{\mu}}_{t|t-1}\left(r_{\widehat{\mathbf{H}}}(\theta)\right)-\widehat{\boldsymbol{\mu}}_{t|t-1}\left(r_{\widehat{\mathbf{H}}}(\theta)\right)\right\|=O_{\mathbb{P}}\left(\frac{1}{\min\left(\sqrt{N},T\right)}\right). θ ∈ Θ sup μ t ∣ t − 1 ( r H ( θ ) ) − μ t ∣ t − 1 ( r H ( θ ) ) = O P min ( N , T ) 1 .
The geometrically decreasing initialization term appearing in Proposition 2(a) is equal to zero because the two filters have the same initial condition under Assumption 14. Moreover, ∥ H ^ − 1 ∥ = O P ( 1 ) \left\|\widehat{\mathbf{H}}^{-1}\right\|=O_{\mathbb{P}}(1) H − 1 = O P ( 1 ) by Proposition 1. Substitution into (169) yields
sup θ ∈ Θ ∥ H ^ − 1 μ ^ t ∣ t − 1 ( r H ^ ( θ ) ) − μ t ∣ t − 1 ( θ ) ∥ = O P ( 1 min ( N , T ) ) , \sup_{\theta\in\Theta}\left\|\widehat{\mathbf{H}}^{-1}\widehat{\boldsymbol{\mu}}_{t|t-1}\left(r_{\widehat{\mathbf{H}}}(\theta)\right)-\boldsymbol{\mu}_{t|t-1}(\theta)\right\|=O_{\mathbb{P}}\left(\frac{1}{\min\left(\sqrt{N},T\right)}\right), θ ∈ Θ sup H − 1 μ t ∣ t − 1 ( r H ( θ ) ) − μ t ∣ t − 1 ( θ ) = O P min ( N , T ) 1 ,
which proves the first statement.
For the conditional-variance process, by adding and subtracting ( H ^ − 1 ⊗ H ^ − 1 ) W r Σ ^ t ∣ t − 1 ( r H ^ ( θ ) ) \left(\widehat{\mathbf{H}}^{-1}\otimes\widehat{\mathbf{H}}^{-1}\right)\mathbf{W}_r\widehat{\boldsymbol{\Sigma}}_{t|t-1}\left(r_{\widehat{\mathbf{H}}}(\theta)\right) ( H − 1 ⊗ H − 1 ) W r Σ t ∣ t − 1 ( r H ( θ ) ) we get
sup θ ∈ Θ ∥ ( H ^ − 1 ⊗ H ^ − 1 ) W r σ ^ t ∣ t − 1 2 ( r H ^ ( θ ) ) − W r σ t ∣ t − 1 2 ( θ ) ∥ ≤ ∥ H ^ − 1 ⊗ H ^ − 1 ∥ sup θ ∈ Θ ∥ W r σ ^ t ∣ t − 1 2 ( r H ^ ( θ ) ) − W r Σ ^ t ∣ t − 1 ( r H ^ ( θ ) ) ∥ + sup θ ∈ Θ ∥ ( H ^ − 1 ⊗ H ^ − 1 ) W r Σ ^ t ∣ t − 1 ( r H ^ ( θ ) ) − W r σ t ∣ t − 1 2 ( θ ) ∥ . (170) \begin{aligned}
\sup_{\theta\in\Theta}\left\|\left(\widehat{\mathbf{H}}^{-1}\otimes\widehat{\mathbf{H}}^{-1}\right)\mathbf{W}_r\widehat{\boldsymbol{\sigma}}^2_{t|t-1}\left(r_{\widehat{\mathbf{H}}}(\theta)\right)-\mathbf{W}_r\boldsymbol{\sigma}^2_{t|t-1}(\theta)\right\|
&\leq\left\|\widehat{\mathbf{H}}^{-1}\otimes\widehat{\mathbf{H}}^{-1}\right\|\sup_{\theta\in\Theta}\left\|\mathbf{W}_r\widehat{\boldsymbol{\sigma}}^2_{t|t-1}\left(r_{\widehat{\mathbf{H}}}(\theta)\right)-\mathbf{W}_r\widehat{\boldsymbol{\Sigma}}_{t|t-1}\left(r_{\widehat{\mathbf{H}}}(\theta)\right)\right\|\\
&\quad+\sup_{\theta\in\Theta}\left\|\left(\widehat{\mathbf{H}}^{-1}\otimes\widehat{\mathbf{H}}^{-1}\right)\mathbf{W}_r\widehat{\boldsymbol{\Sigma}}_{t|t-1}\left(r_{\widehat{\mathbf{H}}}(\theta)\right)-\mathbf{W}_r\boldsymbol{\sigma}^2_{t|t-1}(\theta)\right\|. \tag*{(170)}
\end{aligned} θ ∈ Θ sup ( H − 1 ⊗ H − 1 ) W r σ t ∣ t − 1 2 ( r H ( θ ) ) − W r σ t ∣ t − 1 2 ( θ ) ≤ H − 1 ⊗ H − 1 θ ∈ Θ sup W r σ t ∣ t − 1 2 ( r H ( θ ) ) − W r Σ t ∣ t − 1 ( r H ( θ ) ) + θ ∈ Θ sup ( H − 1 ⊗ H − 1 ) W r Σ t ∣ t − 1 ( r H ( θ ) ) − W r σ t ∣ t − 1 2 ( θ ) . (170)
The second term on the right-hand side of (170) is equal to zero. Indeed, by (33) and Assumption 14,
W r Σ ^ t ∣ t − 1 ( r H ^ ( θ ) ) = ( H ^ ⊗ H ^ ) W r σ t ∣ t − 1 2 ( θ ) \mathbf{W}_r\widehat{\boldsymbol{\Sigma}}_{t|t-1}\left(r_{\widehat{\mathbf{H}}}(\theta)\right)=\left(\widehat{\mathbf{H}}\otimes\widehat{\mathbf{H}}\right)\mathbf{W}_r\boldsymbol{\sigma}^2_{t|t-1}(\theta) W r Σ t ∣ t − 1 ( r H ( θ ) ) = ( H ⊗ H ) W r σ t ∣ t − 1 2 ( θ )
for every θ ∈ Θ \theta\in\Theta θ ∈ Θ and every t ≥ 1 t\ge1 t ≥ 1 .
Proposition 2(a) therefore implies that
sup θ ∈ Θ ∥ W r σ ^ t ∣ t − 1 2 ( r H ^ ( θ ) ) − W r Σ ^ t ∣ t − 1 ( r H ^ ( θ ) ) ∥ = O P ( 1 min ( N , T ) ) . \sup_{\theta\in\Theta}\left\|\mathbf{W}_r\widehat{\boldsymbol{\sigma}}^2_{t|t-1}\left(r_{\widehat{\mathbf{H}}}(\theta)\right)-\mathbf{W}_r\widehat{\boldsymbol{\Sigma}}_{t|t-1}\left(r_{\widehat{\mathbf{H}}}(\theta)\right)\right\|=O_{\mathbb{P}}\left(\frac{1}{\min\left(\sqrt{N},T\right)}\right). θ ∈ Θ sup W r σ t ∣ t − 1 2 ( r H ( θ ) ) − W r Σ t ∣ t − 1 ( r H ( θ ) ) = O P min ( N , T ) 1 .
Moreover, Proposition 1 implies that H ^ → P H 0 \widehat{H} \xrightarrow{\mathbb{P}} H_0 H P H 0 , where H 0 H_0 H 0 is finite and nonsingular. Hence, by the continuous mapping theorem, H ^ − 1 → P H 0 − 1 \widehat{H}^{-1} \xrightarrow{\mathbb{P}} H_0^{-1} H − 1 P H 0 − 1 and ∥ H ^ − 1 ⊗ H ^ − 1 ∥ s p = ∥ H ^ − 1 ∥ s p 2 = O P ( 1 ) \left\lVert\widehat{H}^{-1} \otimes\widehat{H}^{-1} \right\rVert_{\mathrm{sp}} = \left\lVert\widehat{H}^{-1} \right\rVert_{\mathrm{sp}}^2 = O_{\mathbb{P}}(1) H − 1 ⊗ H − 1 sp = H − 1 sp 2 = O P ( 1 ) . Substitution into (170) yields
sup θ ∈ Θ ∥ ( H ^ − 1 ⊗ H ^ − 1 ) W r σ ^ t ∣ t − 1 2 ( r H ^ ( θ ) ) − W r σ t ∣ t − 1 2 ( θ ) ∥ = O P ( 1 min ( N , T ) ) , \sup_{\theta\in\Theta} \left\lVert\left(\widehat{H}^{-1} \otimes\widehat{H}^{-1}\right) W_{r}\widehat{\sigma}_{t\mid t-1}^{2}\left(r_{\widehat{H}}(\theta)\right) - W_{r}\sigma_{t\mid t-1}^{2}\left(\theta\right) \right\rVert= O_{\mathbb{P}}\left(\frac{1}{\min\left(\sqrt{N},T\right)}\right), θ ∈ Θ sup ( H − 1 ⊗ H − 1 ) W r σ t ∣ t − 1 2 ( r H ( θ ) ) − W r σ t ∣ t − 1 2 ( θ ) = O P min ( N , T ) 1 ,
which proves the second statement. ■
Proof of Theorem 4
For the conditional mean, by Proposition 1(c)–(d), Proposition 4, Theorem 2, and Lemma 7, and by using the same steps as in (164) and (165) , we obtain
T ( C ^ i , t ∣ t − 1 ( θ ^ ) − C i , t ∣ t − 1 ( θ 0 ) ) = T ( λ ^ i T − λ 0 i T H ^ − 1 ) μ ^ t ∣ t − 1 ( θ ^ ) + T λ 0 i T ( H ^ − 1 μ ^ t ∣ t − 1 ( θ ^ ) − μ t ∣ t − 1 ( θ 0 ) ) = T ( λ ^ i T − λ 0 i T H ^ − 1 ) μ t ∣ t − 1 ( r H ^ ( θ 0 ) ) + T ( λ ^ i T − λ 0 i T H ^ − 1 ) ( μ ^ t ∣ t − 1 ( θ ^ ) − μ ^ t ∣ t − 1 ( r H ^ ( θ 0 ) ) ) + T λ 0 i T H ^ − 1 ( μ ^ t ∣ t − 1 ( θ ^ ) − μ ^ t ∣ t − 1 ( r H ^ ( θ 0 ) ) ) + T λ 0 i T H ^ − 1 ( μ ^ t ∣ t − 1 ( r H ^ ( θ 0 ) ) − μ t ∣ t − 1 ( r H ^ ( θ 0 ) ) ) = T ( λ ^ i O L S − λ 0 i ) T H ^ − 1 μ t ∣ t − 1 ( r H ^ ( θ 0 ) ) + T O P ( 1 N ) + T O P ( max ( 1 N , 1 T ) ) { O P ( max ( 1 N , 1 T ) ) + O P ( 1 T ) } + T λ 0 i T ( μ t ∣ t − 1 ( r H ^ − 1 ( θ ^ ) ) − μ t ∣ t − 1 ( θ 0 ) ) + T λ 0 i T H ^ − 1 ( μ ^ t ∣ t − 1 ( θ ^ ) − H ^ μ t ∣ t − 1 ( r H ^ − 1 ( θ ^ ) ) ) = T ( λ ^ i O L S − λ 0 i ) T μ t ∣ t − 1 ( θ 0 ) + T λ 0 i T ∇ θ μ t ∣ t − 1 ( θ 0 ) ( r H ^ − 1 ( θ ^ ) − θ 0 + O P ( 1 T ) ) + T λ 0 i T H ^ − 1 ( μ ^ t ∣ t − 1 ( θ ^ ) − H ^ μ t ∣ t − 1 ( r H ^ − 1 ( θ ^ ) ) ) = T ( λ ^ i O L S − λ 0 i ) T μ t ∣ t − 1 ( θ 0 ) + T λ 0 i T ∇ θ μ t ∣ t − 1 ( θ 0 ) ( r H ^ − 1 ( θ ^ ) − θ 0 ) + T λ 0 i T H ^ − 1 ( μ ^ t ∣ t − 1 ( θ ^ ) − H ^ μ t ∣ t − 1 ( r H ^ − 1 ( θ ^ ) ) ) , (171) \begin{aligned}
\sqrt{T}\left(\widehat{C}_{i,t\mid t-1}(\widehat{\theta})-C_{i,t\mid t-1}(\theta_0)\right)
={}&\sqrt{T}\left(\widehat{\lambda}_{i}^{\mathsf{T}}-\lambda_{0i}^{\mathsf{T}}\widehat{H}^{-1}\right)\widehat{\mu}_{t\mid t-1}(\widehat{\theta})+\sqrt{T}\lambda_{0i}^{\mathsf{T}}\left(\widehat{H}^{-1}\widehat{\mu}_{t\mid t-1}(\widehat{\theta})-\mu_{t\mid t-1}(\theta_0)\right)\\
={}&\sqrt{T}\left(\widehat{\lambda}_{i}^{\mathsf{T}}-\lambda_{0i}^{\mathsf{T}}\widehat{H}^{-1}\right)\mu_{t\mid t-1}\left(r_{\widehat{H}}(\theta_0)\right)\\
&+\sqrt{T}\left(\widehat{\lambda}_{i}^{\mathsf{T}}-\lambda_{0i}^{\mathsf{T}}\widehat{H}^{-1}\right)\left(\widehat{\mu}_{t\mid t-1}(\widehat{\theta})-\widehat{\mu}_{t\mid t-1}\left(r_{\widehat{H}}(\theta_0)\right)\right)\\
&+\sqrt{T}\lambda_{0i}^{\mathsf{T}}\widehat{H}^{-1}\left(\widehat{\mu}_{t\mid t-1}(\widehat{\theta})-\widehat{\mu}_{t\mid t-1}\left(r_{\widehat{H}}(\theta_0)\right)\right)\\
&+\sqrt{T}\lambda_{0i}^{\mathsf{T}}\widehat{H}^{-1}\left(\widehat{\mu}_{t\mid t-1}\left(r_{\widehat{H}}(\theta_0)\right)-\mu_{t\mid t-1}\left(r_{\widehat{H}}(\theta_0)\right)\right)\\
={}&\sqrt{T}\left(\widehat{\lambda}_{i}^{\mathrm{OLS}}-\lambda_{0i}\right)^{\mathsf{T}}\widehat{H}^{-1}\mu_{t\mid t-1}\left(r_{\widehat{H}}(\theta_0)\right)+\sqrt{T}O_{\mathbb{P}}\left(\frac{1}{N}\right)\\
&+\sqrt{T}O_{\mathbb{P}}\left(\max\left(\frac{1}{N},\frac{1}{\sqrt{T}}\right)\right)\left\{O_{\mathbb{P}}\left(\max\left(\frac{1}{\sqrt{N}},\frac{1}{T}\right)\right)+O_{\mathbb{P}}\left(\frac{1}{\sqrt{T}}\right)\right\}\\
&+\sqrt{T}\lambda_{0i}^{\mathsf{T}}\left(\mu_{t\mid t-1}\left(r_{\widehat{H}}^{-1}(\widehat{\theta})\right)-\mu_{t\mid t-1}(\theta_0)\right)\\
&+\sqrt{T}\lambda_{0i}^{\mathsf{T}}\widehat{H}^{-1}\left(\widehat{\mu}_{t\mid t-1}(\widehat{\theta})-\widehat{H}\mu_{t\mid t-1}\left(r_{\widehat{H}}^{-1}(\widehat{\theta})\right)\right)\\
={}&\sqrt{T}\left(\widehat{\lambda}_{i}^{\mathrm{OLS}}-\lambda_{0i}\right)^{\mathsf{T}}\mu_{t\mid t-1}(\theta_0)+\sqrt{T}\lambda_{0i}^{\mathsf{T}}\nabla^\theta\mu_{t\mid t-1}(\theta_0)\left(r_{\widehat{H}}^{-1}(\widehat{\theta})-\theta_0+O_{\mathbb{P}}\left(\frac{1}{T}\right)\right)\\
&+\sqrt{T}\lambda_{0i}^{\mathsf{T}}\widehat{H}^{-1}\left(\widehat{\mu}_{t\mid t-1}(\widehat{\theta})-\widehat{H}\mu_{t\mid t-1}\left(r_{\widehat{H}}^{-1}(\widehat{\theta})\right)\right)\\
={}&\sqrt{T}\left(\widehat{\lambda}_{i}^{\mathrm{OLS}}-\lambda_{0i}\right)^{\mathsf{T}}\mu_{t\mid t-1}(\theta_0)+\sqrt{T}\lambda_{0i}^{\mathsf{T}}\nabla^\theta\mu_{t\mid t-1}(\theta_0)\left(r_{\widehat{H}}^{-1}(\widehat{\theta})-\theta_0\right)\\
&+\sqrt{T}\lambda_{0i}^{\mathsf{T}}\widehat{H}^{-1}\left(\widehat{\mu}_{t\mid t-1}(\widehat{\theta})-\widehat{H}\mu_{t\mid t-1}\left(r_{\widehat{H}}^{-1}(\widehat{\theta})\right)\right), \tag*{(171)}
\end{aligned} T ( C i , t ∣ t − 1 ( θ ) − C i , t ∣ t − 1 ( θ 0 ) ) = = = = = T ( λ i T − λ 0 i T H − 1 ) μ t ∣ t − 1 ( θ ) + T λ 0 i T ( H − 1 μ t ∣ t − 1 ( θ ) − μ t ∣ t − 1 ( θ 0 ) ) T ( λ i T − λ 0 i T H − 1 ) μ t ∣ t − 1 ( r H ( θ 0 ) ) + T ( λ i T − λ 0 i T H − 1 ) ( μ t ∣ t − 1 ( θ ) − μ t ∣ t − 1 ( r H ( θ 0 ) ) ) + T λ 0 i T H − 1 ( μ t ∣ t − 1 ( θ ) − μ t ∣ t − 1 ( r H ( θ 0 ) ) ) + T λ 0 i T H − 1 ( μ t ∣ t − 1 ( r H ( θ 0 ) ) − μ t ∣ t − 1 ( r H ( θ 0 ) ) ) T ( λ i OLS − λ 0 i ) T H − 1 μ t ∣ t − 1 ( r H ( θ 0 ) ) + T O P ( N 1 ) + T O P ( max ( N 1 , T 1 ) ) { O P ( max ( N 1 , T 1 ) ) + O P ( T 1 ) } + T λ 0 i T ( μ t ∣ t − 1 ( r H − 1 ( θ ) ) − μ t ∣ t − 1 ( θ 0 ) ) + T λ 0 i T H − 1 ( μ t ∣ t − 1 ( θ ) − H μ t ∣ t − 1 ( r H − 1 ( θ ) ) ) T ( λ i OLS − λ 0 i ) T μ t ∣ t − 1 ( θ 0 ) + T λ 0 i T ∇ θ μ t ∣ t − 1 ( θ 0 ) ( r H − 1 ( θ ) − θ 0 + O P ( T 1 ) ) + T λ 0 i T H − 1 ( μ t ∣ t − 1 ( θ ) − H μ t ∣ t − 1 ( r H − 1 ( θ ) ) ) T ( λ i OLS − λ 0 i ) T μ t ∣ t − 1 ( θ 0 ) + T λ 0 i T ∇ θ μ t ∣ t − 1 ( θ 0 ) ( r H − 1 ( θ ) − θ 0 ) + T λ 0 i T H − 1 ( μ t ∣ t − 1 ( θ ) − H μ t ∣ t − 1 ( r H − 1 ( θ ) ) ) , (171)
where we used also the fourth equality in (57) in the proof of Proposition 1 with λ ^ i O L S \hat{\lambda}^{\mathrm{OLS}}_i λ ^ i OLS being the OLS estimator of λ i \lambda_i λ i were the factors known, and a Taylor expansion of μ t ∣ t − 1 \boldsymbol{\mu}_{t|t-1} μ t ∣ t − 1 in the neighborhood of θ 0 \theta_0 θ 0 .
Moreover, by Proposition 4, we also have
T X 0 T H ^ − 1 ( μ ^ t ∣ t − 1 ( θ ^ ) − H ^ μ t ∣ t − 1 ( r H ^ − 1 ( θ ^ ) ) ) = O P ( T N + 1 T ) = O P ( 1 ) . (172) \sqrt{T}\boldsymbol{X}_0^{\mathsf{T}}\hat{H}^{-1}\left(\widehat{\boldsymbol{\mu}}_{t|t-1}(\hat{\theta})-\hat{H}\boldsymbol{\mu}_{t|t-1}\left(r_{\hat{H}}^{-1}(\hat{\theta})\right)\right)=O_{\mathbb{P}}\left(\sqrt{\frac{T}{N}}+\frac{1}{\sqrt{T}}\right)=O_{\mathbb{P}}(1). \tag*{(172)} T X 0 T H ^ − 1 ( μ t ∣ t − 1 ( θ ^ ) − H ^ μ t ∣ t − 1 ( r H ^ − 1 ( θ ^ ) ) ) = O P ( N T + T 1 ) = O P ( 1 ) . (172)
This follows from Proposition 4, Assumption 3, and the fact that ∥ H ^ − 1 ∥ = O P ( 1 ) \left\Vert\hat{H}^{-1}\right\Vert=O_{\mathbb{P}}(1) H ^ − 1 = O P ( 1 ) by Proposition 1 and the continuous mapping theorem. Then, if T / N → 0 T/N\to0 T / N → 0 as N , T → ∞ N,T\to\infty N , T → ∞ , from (171) and (172) we get
T ( C ^ i , t ∣ t − 1 ( θ ^ ) − C i , t ∣ t − 1 ( θ 0 ) ) = [ μ t ∣ t − 1 T ( θ 0 ) ∣ λ 0 i T ∇ θ μ t ∣ t − 1 ( θ 0 ) ] T [ λ ^ i O L S − λ 0 i r H ^ − 1 ( θ ^ ) − θ 0 ] + o P ( 1 ) . (173) \sqrt{T}\left(\widehat{C}_{i,t|t-1}(\hat{\theta})-C_{i,t|t-1}(\theta_0)\right)=\left[\boldsymbol{\mu}_{t|t-1}^{\mathsf{T}}(\theta_0)\mid\lambda_{0i}^{\mathsf{T}}\nabla^{\theta}\boldsymbol{\mu}_{t|t-1}(\theta_0)\right]\sqrt{T}\begin{bmatrix}\hat{\lambda}^{\mathrm{OLS}}_i-\lambda_{0i}\\ r_{\hat{H}}^{-1}(\hat{\theta})-\theta_0\end{bmatrix}+o_{\mathbb{P}}(1). \tag*{(173)} T ( C i , t ∣ t − 1 ( θ ^ ) − C i , t ∣ t − 1 ( θ 0 ) ) = [ μ t ∣ t − 1 T ( θ 0 ) ∣ λ 0 i T ∇ θ μ t ∣ t − 1 ( θ 0 ) ] T [ λ ^ i OLS − λ 0 i r H ^ − 1 ( θ ^ ) − θ 0 ] + o P ( 1 ) . (173)
By recalling that, by definition of OLS and since P - lim T → ∞ T − 1 ∑ t = 1 T F t F t T = Γ F \mathbb{P}\text{-}\lim_{T\to\infty}T^{-1}\sum_{t=1}^{T}F_tF_t^{\mathsf{T}}=\Gamma_F P - lim T → ∞ T − 1 ∑ t = 1 T F t F t T = Γ F because of the ergodic theorem [8 ] ,
T ( λ ^ i O L S − λ 0 i ) = Γ F − 1 ( 1 T ∑ s = 1 T F s ε i s ) + o P ( 1 ) , (174) \sqrt{T}\left(\hat{\lambda}^{\mathrm{OLS}}_i-\lambda_{0i}\right)=\Gamma_F^{-1}\left(\frac{1}{\sqrt{T}}\sum_{s=1}^{T}F_s\varepsilon_{is}\right)+o_{\mathbb{P}}(1), \tag*{(174)} T ( λ ^ i OLS − λ 0 i ) = Γ F − 1 ( T 1 s = 1 ∑ T F s ε i s ) + o P ( 1 ) , (174)
and since, conditional on H ^ \hat{H} H ^ , the inverse alignment mapping is linear in θ \theta θ (see Appendix C),
T ( r H ^ − 1 ( θ ^ ) − θ 0 ) = R H ^ T ( θ ^ − r H ^ ( θ 0 ) ) . \sqrt{T}\left(r_{\hat{H}}^{-1}(\hat{\theta})-\theta_0\right)=R_{\hat{H}}\sqrt{T}\left(\hat{\theta}-r_{\hat{H}}(\theta_0)\right). T ( r H ^ − 1 ( θ ^ ) − θ 0 ) = R H ^ T ( θ ^ − r H ^ ( θ 0 ) ) .
Since R H ^ → P R H 0 R_{\hat{H}}\xrightarrow{\mathbb{P}}R_{H_0} R H ^ P R H 0 as N , T → ∞ N,T\to\infty N , T → ∞ , equation (161) implies that
T ( r H ^ − 1 ( θ ^ ) − θ 0 ) = I − 1 ( θ 0 ) 1 T ∑ t = 1 T ∇ θ ℓ F ( F t ; θ 0 ) + o P ( 1 ) , (175) \sqrt{T}\left(r_{\hat{H}}^{-1}(\hat{\theta})-\theta_0\right)=\mathcal{I}^{-1}(\theta_0)\frac{1}{\sqrt{T}}\sum_{t=1}^{T}\nabla^{\theta}\ell_F(F_t;\theta_0)+o_{\mathbb{P}}(1), \tag*{(175)} T ( r H ^ − 1 ( θ ^ ) − θ 0 ) = I − 1 ( θ 0 ) T 1 t = 1 ∑ T ∇ θ ℓ F ( F t ; θ 0 ) + o P ( 1 ) , (175)
where we used the fact that R H 0 I H 0 − 1 ( θ 0 ) R H 0 T = I − 1 ( θ 0 ) R_{H_0}\mathcal{I}_{H_0}^{-1}(\theta_0)R_{H_0}^{\mathsf{T}}=\mathcal{I}^{-1}(\theta_0) R H 0 I H 0 − 1 ( θ 0 ) R H 0 T = I − 1 ( θ 0 ) .
Therefore by (174) and Assumption 5(iv), and by (175) , Theorem 2 together with an application of Slutsky’s theorem, and Lemma 11,
T ( λ ^ i O L S − λ 0 i ) ∣ F t − 1 ⇒ N ( 0 r , Γ F − 1 Ξ i Γ F − 1 ) , (176) \sqrt{T}\left(\hat{\lambda}^{\mathrm{OLS}}_i-\lambda_{0i}\right)\mid\mathcal{F}_{t-1}\Rightarrow\mathcal{N}\left(\mathbf{0}_r,\Gamma_F^{-1}\Xi_i\Gamma_F^{-1}\right), \tag*{(176)} T ( λ ^ i OLS − λ 0 i ) ∣ F t − 1 ⇒ N ( 0 r , Γ F − 1 Ξ i Γ F − 1 ) , (176)
T ( r H ^ − 1 ( θ ^ ) − θ 0 ) ∣ F t − 1 ⇒ N ( 0 d , I − 1 ( θ 0 ) V ( θ 0 ) I − 1 ( θ 0 ) ) . (177) \sqrt{T}\left(r_{\hat{H}}^{-1}(\hat{\theta})-\theta_0\right)\mid\mathcal{F}_{t-1}\Rightarrow\mathcal{N}\left(\mathbf{0}_d,\mathcal{I}^{-1}(\theta_0)V(\theta_0)\mathcal{I}^{-1}(\theta_0)\right). \tag*{(177)} T ( r H ^ − 1 ( θ ^ ) − θ 0 ) ∣ F t − 1 ⇒ N ( 0 d , I − 1 ( θ 0 ) V ( θ 0 ) I − 1 ( θ 0 ) ) . (177)
Notice that in fact these results are independent of t t t , because of stationarity as guaranteed by Assumptions 2 and 4. Likewise the conditional asymptotic covariance of these two terms is in fact independent of t t t and is given by
lim T → ∞ E [ ( 1 T ∑ s = 1 T F s e i s ) ( 1 T ∑ s = 1 T ∇ θ ℓ F ( F s ; θ 0 ) ) T ] = lim T → ∞ 1 T ∑ s 1 , s 2 = 1 T E [ F s 1 ( ∇ θ ℓ F ( F s 2 ; θ 0 ) ) T e i s 1 ] = lim T → ∞ 1 T ∑ s 1 , s 2 = 1 T E [ F s 1 ( ∇ θ ℓ F ( F s 2 ; θ 0 ) ) T ] E [ e i s 1 ] = 0 r × d , (178) \begin{aligned}
\lim_{T\to\infty}\mathbb{E}\left[\left(\frac{1}{\sqrt{T}}\sum_{s=1}^{T}F_s e_{is}\right)\left(\frac{1}{\sqrt{T}}\sum_{s=1}^{T}\nabla_{\theta}\ell_F(F_s;\theta_0)\right)^{\mathsf{T}}\right]
&= \lim_{T\to\infty}\frac{1}{T}\sum_{s_1,s_2=1}^{T}\mathbb{E}\left[F_{s_1}\left(\nabla_{\theta}\ell_F(F_{s_2};\theta_0)\right)^{\mathsf{T}}e_{is_1}\right]\\
&= \lim_{T\to\infty}\frac{1}{T}\sum_{s_1,s_2=1}^{T}\mathbb{E}\left[F_{s_1}\left(\nabla_{\theta}\ell_F(F_{s_2};\theta_0)\right)^{\mathsf{T}}\right]\mathbb{E}[e_{is_1}]
=\mathbf{0}_{r\times d}, \tag*{(178)}
\end{aligned} T → ∞ lim E ( T 1 s = 1 ∑ T F s e i s ) ( T 1 s = 1 ∑ T ∇ θ ℓ F ( F s ; θ 0 ) ) T = T → ∞ lim T 1 s 1 , s 2 = 1 ∑ T E [ F s 1 ( ∇ θ ℓ F ( F s 2 ; θ 0 ) ) T e i s 1 ] = T → ∞ lim T 1 s 1 , s 2 = 1 ∑ T E [ F s 1 ( ∇ θ ℓ F ( F s 2 ; θ 0 ) ) T ] E [ e i s 1 ] = 0 r × d , (178)
since the score is just function of the process { F t } t ∈ Z \{F_t\}_{t\in\mathbb{Z}} { F t } t ∈ Z which is independent of { e i t } t ∈ Z \{e_{it}\}_{t\in\mathbb{Z}} { e i t } t ∈ Z because of Assumption 5(i) and E [ e i t ] = 0 \mathbb{E}[e_{it}]=0 E [ e i t ] = 0 because of Assumption 4(i).
By using (176) , (177) , and (178) into (173) , by the standard Cramér-Wold device we get
T ( C ^ i , t ∣ t − 1 ( θ ^ ) − C i , t ∣ t − 1 ( θ 0 ) ) ∣ F t − 1 ⇒ N ( 0 , a i , t ∣ t − 1 ( θ 0 ) T [ Γ F − 1 Ξ i Γ F − 1 0 r × d 0 d × r I − 1 ( θ 0 ) V ( θ 0 ) I − 1 ( θ 0 ) ] a i , t ∣ t − 1 ( θ 0 ) ) , \sqrt{T}\left(\widehat{C}_{i,t|t-1}(\widehat{\theta})-C_{i,t|t-1}(\theta_0)\right)\mathrel{\Big|}\mathcal{F}_{t-1}
\Rightarrow\mathcal{N}\left(0,a_{i,t|t-1}(\theta_0)^{\mathsf{T}}
\begin{bmatrix}
\Gamma_F^{-1}\Xi_i\Gamma_F^{-1} & \mathbf{0}_{r\times d}\\
\mathbf{0}_{d\times r} & \mathcal{I}^{-1}(\theta_0)\mathcal{V}(\theta_0)\mathcal{I}^{-1}(\theta_0)
\end{bmatrix}
a_{i,t|t-1}(\theta_0)\right), T ( C i , t ∣ t − 1 ( θ ) − C i , t ∣ t − 1 ( θ 0 ) ) F t − 1 ⇒ N ( 0 , a i , t ∣ t − 1 ( θ 0 ) T [ Γ F − 1 Ξ i Γ F − 1 0 d × r 0 r × d I − 1 ( θ 0 ) V ( θ 0 ) I − 1 ( θ 0 ) ] a i , t ∣ t − 1 ( θ 0 ) ) ,
where a i , t ∣ t − 1 ( θ 0 ) T = [ μ t ∣ t − 1 ( θ 0 ) T ∣ λ 0 T ∇ θ μ t ∣ t − 1 ( θ 0 ) ] a_{i,t|t-1}(\theta_0)^{\mathsf{T}}=\left[\boldsymbol{\mu}_{t|t-1}(\theta_0)^{\mathsf{T}}\mid\lambda_{0}^{\mathsf{T}}\nabla^{\theta}\boldsymbol{\mu}_{t|t-1}(\theta_0)\right] a i , t ∣ t − 1 ( θ 0 ) T = [ μ t ∣ t − 1 ( θ 0 ) T ∣ λ 0 T ∇ θ μ t ∣ t − 1 ( θ 0 ) ] . This proves the statement for the conditional mean.
For the conditional variance, we first note that, because the Kronecker product is a bilinear and continuous operator, it respects the standard rules for combining asymptotic expansions. Hence as N , T → ∞ N,T\to\infty N , T → ∞ , we can use the fact that
( λ ^ i ⊤ ⊗ λ ^ i ⊤ ) − ( λ 0 i ⊤ H ^ − 1 ⊗ λ 0 i ⊤ H ^ − 1 ) = ( λ ^ i ⊤ − λ 0 i ⊤ H ^ − 1 ) ⊗ λ 0 i ⊤ H ^ − 1 + λ 0 i ⊤ H ^ − 1 ⊗ ( λ ^ i ⊤ − λ 0 i ⊤ H ^ − 1 ) + ( λ ^ i ⊤ − λ 0 i ⊤ H ^ − 1 ) ⊗ ( λ ^ i ⊤ − λ 0 i ⊤ H ^ − 1 ) = ( λ ^ i ⊤ − λ 0 i ⊤ H ^ − 1 ) ⊗ λ 0 i ⊤ H ^ − 1 + λ 0 i ⊤ H ^ − 1 ⊗ ( λ ^ i ⊤ − λ 0 i ⊤ H ^ − 1 ) + O P ( max ( 1 T , 1 N ) ) = ( ( λ ^ i ⊤ − λ 0 i ⊤ H ^ − 1 ) ⊗ λ 0 i ⊤ H ^ − 1 ) ( I r 2 + K r 2 × r 2 ) + O P ( max ( 1 T , 1 N ) ) , (179) \begin{aligned}
\left(\widehat{\lambda}_i^{\top}\otimes\widehat{\lambda}_i^{\top}\right)-\left(\lambda_{0i}^{\top}\widehat{H}^{-1}\otimes\lambda_{0i}^{\top}\widehat{H}^{-1}\right)
&=\left(\widehat{\lambda}_i^{\top}-\lambda_{0i}^{\top}\widehat{H}^{-1}\right)\otimes\lambda_{0i}^{\top}\widehat{H}^{-1}
+\lambda_{0i}^{\top}\widehat{H}^{-1}\otimes\left(\widehat{\lambda}_i^{\top}-\lambda_{0i}^{\top}\widehat{H}^{-1}\right)\\
&\quad+\left(\widehat{\lambda}_i^{\top}-\lambda_{0i}^{\top}\widehat{H}^{-1}\right)\otimes\left(\widehat{\lambda}_i^{\top}-\lambda_{0i}^{\top}\widehat{H}^{-1}\right)\\
&=\left(\widehat{\lambda}_i^{\top}-\lambda_{0i}^{\top}\widehat{H}^{-1}\right)\otimes\lambda_{0i}^{\top}\widehat{H}^{-1}
+\lambda_{0i}^{\top}\widehat{H}^{-1}\otimes\left(\widehat{\lambda}_i^{\top}-\lambda_{0i}^{\top}\widehat{H}^{-1}\right)
+O_{\mathbb{P}}\left(\max\left(\frac{1}{T},\frac{1}{N}\right)\right)\\
&=\left(\left(\widehat{\lambda}_i^{\top}-\lambda_{0i}^{\top}\widehat{H}^{-1}\right)\otimes\lambda_{0i}^{\top}\widehat{H}^{-1}\right)\left(I_{r^2}+\mathcal{K}_{r^2\times r^2}\right)
+O_{\mathbb{P}}\left(\max\left(\frac{1}{T},\frac{1}{N}\right)\right), \tag*{(179)}
\end{aligned} ( λ i ⊤ ⊗ λ i ⊤ ) − ( λ 0 i ⊤ H − 1 ⊗ λ 0 i ⊤ H − 1 ) = ( λ i ⊤ − λ 0 i ⊤ H − 1 ) ⊗ λ 0 i ⊤ H − 1 + λ 0 i ⊤ H − 1 ⊗ ( λ i ⊤ − λ 0 i ⊤ H − 1 ) + ( λ i ⊤ − λ 0 i ⊤ H − 1 ) ⊗ ( λ i ⊤ − λ 0 i ⊤ H − 1 ) = ( λ i ⊤ − λ 0 i ⊤ H − 1 ) ⊗ λ 0 i ⊤ H − 1 + λ 0 i ⊤ H − 1 ⊗ ( λ i ⊤ − λ 0 i ⊤ H − 1 ) + O P ( max ( T 1 , N 1 ) ) = ( ( λ i ⊤ − λ 0 i ⊤ H − 1 ) ⊗ λ 0 i ⊤ H − 1 ) ( I r 2 + K r 2 × r 2 ) + O P ( max ( T 1 , N 1 ) ) , (179)
where in the last equality we used Proposition 1(d). Therefore, using the identity
( H ^ − 1 ⊗ H ^ − 1 ) ( I r 2 + K r 2 × r 2 ) ( H ^ ⊗ H ^ ) = ( H ^ − 1 ⊗ H ^ − 1 ) ( H ^ ⊗ H ^ ) + ( H ^ − 1 ⊗ H ^ − 1 ) K r 2 × r 2 ( H ^ ⊗ H ^ ) = I r 2 + ( H ^ − 1 ⊗ H ^ − 1 ) ( H ^ ⊗ H ^ ) K r 2 × r 2 = I r 2 + K r 2 × r 2 , \begin{aligned}
\left(\widehat{H}^{-1}\otimes\widehat{H}^{-1}\right)\left(I_{r^2}+\mathcal{K}_{r^2\times r^2}\right)\left(\widehat{H}\otimes\widehat{H}\right) \\
&= \left(\widehat{H}^{-1} \otimes\widehat{H}^{-1}\right)\left(\widehat{H} \otimes\widehat{H}\right) + \left(\widehat{H}^{-1} \otimes\widehat{H}^{-1}\right)\mathcal{K}_{r_2 \times r_2}\left(\widehat{H} \otimes\widehat{H}\right)
\\
= I_{r_2} + \left(\widehat{H}^{-1} \otimes\widehat{H}^{-1}\right)\left(\widehat{H} \otimes\widehat{H}\right)\mathcal{K}_{r_2 \times r_2}
\\
= I_{r_2} + \mathcal{K}_{r_2 \times r_2},
\end{aligned} ( H − 1 ⊗ H − 1 ) ( I r 2 + K r 2 × r 2 ) ( H ⊗ H ) = I r 2 + ( H − 1 ⊗ H − 1 ) ( H ⊗ H ) K r 2 × r 2 = I r 2 + K r 2 × r 2 , = ( H − 1 ⊗ H − 1 ) ( H ⊗ H ) + ( H − 1 ⊗ H − 1 ) K r 2 × r 2 ( H ⊗ H )
and since σ t ∣ t − 1 2 ( r H ^ ( θ 0 ) ) = W r T ( H ^ ⊗ H ^ ) W r σ t ∣ t − 1 2 ( θ 0 ) \sigma^2_{t|t-1}(r_{\widehat{H}}(\theta_0)) = \mathbf{W}_r^{\mathsf{T}}(\widehat{H} \otimes\widehat{H})\mathbf{W}_r\sigma^2_{t|t-1}(\theta_0) σ t ∣ t − 1 2 ( r H ( θ 0 )) = W r T ( H ⊗ H ) W r σ t ∣ t − 1 2 ( θ 0 ) by definition (see (33) ), by Proposition 1(c)–(d), Proposition 4, Theorem 2, and Lemma 7, by using the same steps as in (167) and (168) , we obtain
T ( P ^ i , t ∣ t − 1 ( θ ^ ) − P i , t ∣ t − 1 ( θ 0 ) ) = T ( ( λ ^ i T ⊗ λ ^ i T ) − ( λ 0 i T H ^ − 1 ⊗ λ 0 i T H ^ − 1 ) ) W r σ ^ t ∣ t − 1 2 ( θ ^ ) + T ( λ 0 i T ⊗ λ 0 i T ) ( ( H ^ − 1 ⊗ H ^ − 1 ) W r σ ^ t ∣ t − 1 2 ( θ ^ ) − W r σ t ∣ t − 1 2 ( θ 0 ) ) = T ( ( λ ^ i T ⊗ λ ^ i T ) − ( λ 0 i T H ^ − 1 ⊗ λ 0 i T H ^ − 1 ) ) W r σ t ∣ t − 1 2 ( r H ^ ( θ 0 ) ) + T ( ( λ ^ i T ⊗ λ ^ i T ) − ( λ 0 i T H ^ − 1 ⊗ λ 0 i T H ^ − 1 ) ) ( W r σ ^ t ∣ t − 1 2 ( θ ^ ) − W r σ t ∣ t − 1 2 ( r H ^ ( θ 0 ) ) ) + T ( λ 0 i T H ^ − 1 ⊗ λ 0 i T H ^ − 1 ) ( W r σ ^ t ∣ t − 1 2 ( θ ^ ) − W r σ ^ t ∣ t − 1 2 ( r H ^ ( θ 0 ) ) ) = T ( ( λ ^ i O L S ⊗ λ ^ i O L S ) T − ( λ 0 i ⊗ λ 0 i ) T ) W r σ t ∣ t − 1 2 ( θ 0 ) + T O P ( max ( 1 N , 1 T ) ) + T O P ( max ( 1 N , 1 T ) ) { O P ( max ( 1 N , 1 T ) ) + O P ( 1 T ) } + T ( λ 0 i T H ^ − 1 ⊗ λ 0 i T H ^ − 1 ) ( H ^ ⊗ H ^ ) W r ( σ t ∣ t − 1 2 ( r H ^ ( θ ^ ) ) − σ t ∣ t − 1 2 ( θ 0 ) ) + T ( λ 0 i T H ^ − 1 ⊗ λ 0 i T H ^ − 1 ) ( W r σ ^ t ∣ t − 1 2 ( θ ^ ) − ( H ^ ⊗ H ^ ) W r σ t ∣ t − 1 2 ( r H ^ − 1 ( θ ^ ) ) ) = T ( ( λ ^ i O L S ⊗ λ ^ i O L S ) T − ( λ 0 i ⊗ λ 0 i ) T ) W r σ t ∣ t − 1 2 ( θ 0 ) + T ( λ 0 i ⊗ λ 0 i ) T W r ∇ θ σ t ∣ t − 1 2 ( θ 0 ) ( r H ^ − 1 ( θ ^ ) − θ 0 + O P ( 1 T ) ) + T ( λ 0 i T H ^ − 1 ⊗ λ 0 i T H ^ − 1 ) ( W r σ ^ t ∣ t − 1 2 ( θ ^ ) − ( H ^ ⊗ H ^ ) W r σ t ∣ t − 1 2 ( r H ^ − 1 ( θ ^ ) ) ) , = T ( ( λ ^ i O L S ⊗ λ ^ i O L S ) T − ( λ 0 i ⊗ λ 0 i ) T ) W r σ t ∣ t − 1 2 ( θ 0 ) + T ( λ 0 i ⊗ λ 0 i ) T W r ∇ θ σ t ∣ t − 1 2 ( θ 0 ) ( r H ^ − 1 ( θ ^ ) − θ 0 ) + T ( λ 0 i T H ^ − 1 ⊗ λ 0 i T H ^ − 1 ) ( W r σ ^ t ∣ t − 1 2 ( θ ^ ) − ( H ^ ⊗ H ^ ) W r σ t ∣ t − 1 2 ( r H ^ − 1 ( θ ^ ) ) ) , (180) \begin{aligned}
\sqrt{T}\left(\widehat{P}_{i,t|t-1}(\widehat{\theta})-P_{i,t|t-1}(\theta_0)\right)
&=\sqrt{T}\left(\left(\widehat{\lambda}_i^{\mathsf{T}}\otimes\widehat{\lambda}_i^{\mathsf{T}}\right)-\left(\lambda_{0i}^{\mathsf{T}}\widehat{H}^{-1}\otimes\lambda_{0i}^{\mathsf{T}}\widehat{H}^{-1}\right)\right)\mathbf{W}_r\widehat{\sigma}^2_{t|t-1}(\widehat{\theta})\\
&\quad+\sqrt{T}\left(\lambda_{0i}^{\mathsf{T}}\otimes\lambda_{0i}^{\mathsf{T}}\right)\left(\left(\widehat{H}^{-1}\otimes\widehat{H}^{-1}\right)\mathbf{W}_r\widehat{\sigma}^2_{t|t-1}(\widehat{\theta})-\mathbf{W}_r\sigma^2_{t|t-1}(\theta_0)\right)\\
&=\sqrt{T}\left(\left(\widehat{\lambda}_i^{\mathsf{T}}\otimes\widehat{\lambda}_i^{\mathsf{T}}\right)-\left(\lambda_{0i}^{\mathsf{T}}\widehat{H}^{-1}\otimes\lambda_{0i}^{\mathsf{T}}\widehat{H}^{-1}\right)\right)\mathbf{W}_r\sigma^2_{t|t-1}(r_{\widehat{H}}(\theta_0))\\
&\quad+\sqrt{T}\left(\left(\widehat{\lambda}_i^{\mathsf{T}}\otimes\widehat{\lambda}_i^{\mathsf{T}}\right)-\left(\lambda_{0i}^{\mathsf{T}}\widehat{H}^{-1}\otimes\lambda_{0i}^{\mathsf{T}}\widehat{H}^{-1}\right)\right)\left(\mathbf{W}_r\widehat{\sigma}^2_{t|t-1}(\widehat{\theta})-\mathbf{W}_r\sigma^2_{t|t-1}(r_{\widehat{H}}(\theta_0))\right)\\
&\quad+\sqrt{T}\left(\lambda_{0i}^{\mathsf{T}}\widehat{H}^{-1}\otimes\lambda_{0i}^{\mathsf{T}}\widehat{H}^{-1}\right)\left(\mathbf{W}_r\widehat{\sigma}^2_{t|t-1}(\widehat{\theta})-\mathbf{W}_r\widehat{\sigma}^2_{t|t-1}(r_{\widehat{H}}(\theta_0))\right)\\
&=\sqrt{T}\left(\left(\widehat{\lambda}_i^{\mathrm{OLS}}\otimes\widehat{\lambda}_i^{\mathrm{OLS}}\right)^{\mathsf{T}}-\left(\lambda_{0i}\otimes\lambda_{0i}\right)^{\mathsf{T}}\right)\mathbf{W}_r\sigma^2_{t|t-1}(\theta_0)\\
&\quad+\sqrt{T}O_{\mathbb{P}}\left(\max\left(\frac{1}{N},\frac{1}{T}\right)\right)+\sqrt{T}O_{\mathbb{P}}\left(\max\left(\frac{1}{N},\frac{1}{\sqrt{T}}\right)\right)\left\{O_{\mathbb{P}}\left(\max\left(\frac{1}{\sqrt{N}},\frac{1}{T}\right)\right)+O_{\mathbb{P}}\left(\frac{1}{\sqrt{T}}\right)\right\}\\
&\quad+\sqrt{T}\left(\lambda_{0i}^{\mathsf{T}}\widehat{H}^{-1}\otimes\lambda_{0i}^{\mathsf{T}}\widehat{H}^{-1}\right)(\widehat{H}\otimes\widehat{H})\mathbf{W}_r\left(\sigma^2_{t|t-1}(r_{\widehat{H}}(\widehat{\theta}))-\sigma^2_{t|t-1}(\theta_0)\right)\\
&\quad+\sqrt{T}\left(\lambda_{0i}^{\mathsf{T}}\widehat{H}^{-1}\otimes\lambda_{0i}^{\mathsf{T}}\widehat{H}^{-1}\right)\left(\mathbf{W}_r\widehat{\sigma}^2_{t|t-1}(\widehat{\theta})-(\widehat{H}\otimes\widehat{H})\mathbf{W}_r\sigma^2_{t|t-1}\left(r_{\widehat{H}}^{-1}(\widehat{\theta})\right)\right)\\
&=\sqrt{T}\left(\left(\widehat{\lambda}_i^{\mathrm{OLS}}\otimes\widehat{\lambda}_i^{\mathrm{OLS}}\right)^{\mathsf{T}}-\left(\lambda_{0i}\otimes\lambda_{0i}\right)^{\mathsf{T}}\right)\mathbf{W}_r\sigma^2_{t|t-1}(\theta_0)\\
&\quad+\sqrt{T}\left(\lambda_{0i}\otimes\lambda_{0i}\right)^{\mathsf{T}}\mathbf{W}_r\nabla_{\theta}\sigma^2_{t|t-1}(\theta_0)\left(r_{\widehat{H}}^{-1}(\widehat{\theta})-\theta_0+O_{\mathbb{P}}\left(\frac{1}{T}\right)\right)\\
&\quad+\sqrt{T}\left(\lambda_{0i}^{\mathsf{T}}\widehat{H}^{-1}\otimes\lambda_{0i}^{\mathsf{T}}\widehat{H}^{-1}\right)\left(\mathbf{W}_r\widehat{\sigma}^2_{t|t-1}(\widehat{\theta})-(\widehat{H}\otimes\widehat{H})\mathbf{W}_r\sigma^2_{t|t-1}\left(r_{\widehat{H}}^{-1}(\widehat{\theta})\right)\right),\\
&=\sqrt{T}\left(\left(\widehat{\lambda}_i^{\mathrm{OLS}}\otimes\widehat{\lambda}_i^{\mathrm{OLS}}\right)^{\mathsf{T}}-\left(\lambda_{0i}\otimes\lambda_{0i}\right)^{\mathsf{T}}\right)\mathbf{W}_r\sigma^2_{t|t-1}(\theta_0) \\
&+ \sqrt{T}(\lambda_{0i}\otimes\lambda_{0i})^{\mathsf T}W_r\nabla^\theta\sigma_{t|t-1}^{2}(\theta_0)\left(r_{\widehat H}^{-1}(\widehat\theta)-\theta_0\right) \\
&+ \sqrt{T}\left(\lambda_{0i}^{\mathsf T}\widehat H^{-1}\otimes\lambda_{0i}^{\mathsf T}\widehat H^{-1}\right)\left(W_r\widehat\sigma_{t|t-1}^{2}(\widehat\theta)-(\widehat H\otimes\widehat H)W_r\sigma_{t|t-1}^{2}\left(r_{\widehat H}^{-1}(\widehat\theta)\right)\right), \tag*{(180)}
\end{aligned} T ( P i , t ∣ t − 1 ( θ ) − P i , t ∣ t − 1 ( θ 0 ) ) = T ( ( λ i T ⊗ λ i T ) − ( λ 0 i T H − 1 ⊗ λ 0 i T H − 1 ) ) W r σ t ∣ t − 1 2 ( θ ) + T ( λ 0 i T ⊗ λ 0 i T ) ( ( H − 1 ⊗ H − 1 ) W r σ t ∣ t − 1 2 ( θ ) − W r σ t ∣ t − 1 2 ( θ 0 ) ) = T ( ( λ i T ⊗ λ i T ) − ( λ 0 i T H − 1 ⊗ λ 0 i T H − 1 ) ) W r σ t ∣ t − 1 2 ( r H ( θ 0 )) + T ( ( λ i T ⊗ λ i T ) − ( λ 0 i T H − 1 ⊗ λ 0 i T H − 1 ) ) ( W r σ t ∣ t − 1 2 ( θ ) − W r σ t ∣ t − 1 2 ( r H ( θ 0 )) ) + T ( λ 0 i T H − 1 ⊗ λ 0 i T H − 1 ) ( W r σ t ∣ t − 1 2 ( θ ) − W r σ t ∣ t − 1 2 ( r H ( θ 0 )) ) = T ( ( λ i OLS ⊗ λ i OLS ) T − ( λ 0 i ⊗ λ 0 i ) T ) W r σ t ∣ t − 1 2 ( θ 0 ) + T O P ( max ( N 1 , T 1 ) ) + T O P ( max ( N 1 , T 1 ) ) { O P ( max ( N 1 , T 1 ) ) + O P ( T 1 ) } + T ( λ 0 i T H − 1 ⊗ λ 0 i T H − 1 ) ( H ⊗ H ) W r ( σ t ∣ t − 1 2 ( r H ( θ )) − σ t ∣ t − 1 2 ( θ 0 ) ) + T ( λ 0 i T H − 1 ⊗ λ 0 i T H − 1 ) ( W r σ t ∣ t − 1 2 ( θ ) − ( H ⊗ H ) W r σ t ∣ t − 1 2 ( r H − 1 ( θ ) ) ) = T ( ( λ i OLS ⊗ λ i OLS ) T − ( λ 0 i ⊗ λ 0 i ) T ) W r σ t ∣ t − 1 2 ( θ 0 ) + T ( λ 0 i ⊗ λ 0 i ) T W r ∇ θ σ t ∣ t − 1 2 ( θ 0 ) ( r H − 1 ( θ ) − θ 0 + O P ( T 1 ) ) + T ( λ 0 i T H − 1 ⊗ λ 0 i T H − 1 ) ( W r σ t ∣ t − 1 2 ( θ ) − ( H ⊗ H ) W r σ t ∣ t − 1 2 ( r H − 1 ( θ ) ) ) , = T ( ( λ i OLS ⊗ λ i OLS ) T − ( λ 0 i ⊗ λ 0 i ) T ) W r σ t ∣ t − 1 2 ( θ 0 ) + T ( λ 0 i ⊗ λ 0 i ) T W r ∇ θ σ t ∣ t − 1 2 ( θ 0 ) ( r H − 1 ( θ ) − θ 0 ) + T ( λ 0 i T H − 1 ⊗ λ 0 i T H − 1 ) ( W r σ t ∣ t − 1 2 ( θ ) − ( H ⊗ H ) W r σ t ∣ t − 1 2 ( r H − 1 ( θ ) ) ) , (180)
where we used also (179) , the fourth equality in (57) in the proof of Proposition 1, and a Taylor expansion of σ t ∣ t − 1 2 \sigma_{t|t-1}^{2} σ t ∣ t − 1 2 in the neighborhood of θ 0 \theta_0 θ 0 .
Moreover, by Proposition 4, we also have
T ( λ 0 i T H ^ − 1 ⊗ λ 0 i T H ^ − 1 ) ( W r σ ^ t ∣ t − 1 2 ( θ ^ ) − ( H ^ ⊗ H ^ ) W r σ t ∣ t − 1 2 ( r H ^ − 1 ( θ ^ ) ) ) = O P ( T N + 1 T ) = o P ( 1 ) . (181) \begin{aligned}
\sqrt{T}\left(\lambda_{0i}^{\mathsf T}\widehat H^{-1}\otimes\lambda_{0i}^{\mathsf T}\widehat H^{-1}\right)\left(W_r\widehat\sigma_{t|t-1}^{2}(\widehat\theta)-(\widehat H\otimes\widehat H)W_r\sigma_{t|t-1}^{2}\left(r_{\widehat H}^{-1}(\widehat\theta)\right)\right) \\
&=O_{\mathbb P}\left(\sqrt{\frac{T}{N}}+\frac{1}{\sqrt{T}}\right)=o_{\mathbb P}(1). \tag*{(181)}
\end{aligned} T ( λ 0 i T H − 1 ⊗ λ 0 i T H − 1 ) ( W r σ t ∣ t − 1 2 ( θ ) − ( H ⊗ H ) W r σ t ∣ t − 1 2 ( r H − 1 ( θ ) ) ) = O P ( N T + T 1 ) = o P ( 1 ) . (181)
This follows from Proposition 4, Assumption 3, and the fact that ∥ H ^ − 1 ⊗ H ^ − 1 ∥ = O P ( 1 ) \left\|\widehat H^{-1}\otimes\widehat H^{-1}\right\|=O_{\mathbb P}(1) H − 1 ⊗ H − 1 = O P ( 1 ) by Proposition 1 and the continuous mapping theorem.
Hence, if T / N → 0 T/N\to0 T / N → 0 as N , T → ∞ N,T\to\infty N , T → ∞ , from (180) and (181) , we get
T ( P ^ i , t ∣ t − 1 ( θ ^ ) − P i , t ∣ t − 1 ( θ 0 ) ) = [ σ t ∣ t − 1 2 ( θ 0 ) T W r T ∣ ( λ 0 i ⊗ λ 0 i ) T W r ∇ θ σ t ∣ t − 1 2 ( θ 0 ) ] × T [ ( λ ^ i O L S ⊗ λ ^ i O L S r H ^ − 1 ( θ ^ ) − θ 0 ) − ( λ 0 i ⊗ λ 0 i 0 ) ] + o P ( 1 ) . = [ σ t ∣ t − 1 2 ( θ 0 ) T W r T ∣ ( λ 0 i ⊗ λ 0 i ) T W r ∇ θ σ t ∣ t − 1 2 ( θ 0 ) ] [ I r ⊗ λ 0 i + λ 0 i ⊗ I r 0 r 2 × d 0 d × r I d ] × T [ λ ^ i O L S − λ 0 i r H ^ − 1 ( θ ^ ) − θ 0 ] + o P ( 1 ) , (182) \begin{aligned}
\sqrt{T}\left(\widehat P_{i,t|t-1}(\widehat\theta)-P_{i,t|t-1}(\theta_0)\right) \\
&=\left[\sigma_{t|t-1}^{2}(\theta_0)^{\mathsf T}W_r^{\mathsf T}\mid(\lambda_{0i}\otimes\lambda_{0i})^{\mathsf T}W_r\nabla^\theta\sigma_{t|t-1}^{2}(\theta_0)\right]
\times\sqrt{T}\left[
\begin{pmatrix}
\widehat\lambda_i^{\mathrm{OLS}}\otimes\widehat\lambda_i^{\mathrm{OLS}}\\
r_{\widehat H}^{-1}(\widehat\theta)-\theta_0
\end{pmatrix}
-
\begin{pmatrix}
\lambda_{0i}\otimes\lambda_{0i}\\
0
\end{pmatrix}
\right]+o_{\mathbb P}(1). \\
&=\left[\sigma_{t|t-1}^{2}(\theta_0)^{\mathsf T}W_r^{\mathsf T}\mid(\lambda_{0i}\otimes\lambda_{0i})^{\mathsf T}W_r\nabla^\theta\sigma_{t|t-1}^{2}(\theta_0)\right]
\begin{bmatrix}
I_r\otimes\lambda_{0i}+\lambda_{0i}\otimes I_r & 0_{r^2\times d}\\
0_{d\times r} & I_d
\end{bmatrix}
\times\sqrt{T}
\begin{bmatrix}
\widehat\lambda_i^{\mathrm{OLS}}-\lambda_{0i}\\
r_{\widehat H}^{-1}(\widehat\theta)-\theta_0
\end{bmatrix}
+o_{\mathbb P}(1), \tag*{(182)}
\end{aligned} T ( P i , t ∣ t − 1 ( θ ) − P i , t ∣ t − 1 ( θ 0 ) ) = [ σ t ∣ t − 1 2 ( θ 0 ) T W r T ∣ ( λ 0 i ⊗ λ 0 i ) T W r ∇ θ σ t ∣ t − 1 2 ( θ 0 ) ] × T [ ( λ i OLS ⊗ λ i OLS r H − 1 ( θ ) − θ 0 ) − ( λ 0 i ⊗ λ 0 i 0 ) ] + o P ( 1 ) . = [ σ t ∣ t − 1 2 ( θ 0 ) T W r T ∣ ( λ 0 i ⊗ λ 0 i ) T W r ∇ θ σ t ∣ t − 1 2 ( θ 0 ) ] [ I r ⊗ λ 0 i + λ 0 i ⊗ I r 0 d × r 0 r 2 × d I d ] × T [ λ i OLS − λ 0 i r H − 1 ( θ ) − θ 0 ] + o P ( 1 ) , (182)
where the second equality follows by an application of the mean value theorem to the nonlinear map ϕ ( x ) = x ⊗ x = vec ( x x T ) \phi(x)=x\otimes x=\operatorname{vec}(xx^{\mathsf T}) ϕ ( x ) = x ⊗ x = vec ( x x T ) , for any generic r r r -dimensional vector x ∈ R r x\in\mathbb R^r x ∈ R r .
Therefore, it is now clear that the same arguments used in the discussion to obtain (176) , (177) , and (178) apply here, and hence, using these results into (182) , by the standard Cramér-Wold device we get
T ( P ^ i , t ∣ t − 1 ( θ ^ ) − P i , t ∣ t − 1 ( θ 0 ) ) ∣ F t − 1 ⇒ N ( 0 , b i , t ∣ t − 1 ⊤ ( θ 0 ) [ Γ F − 1 Ξ i Γ F − 1 0 r × d 0 d × r I − 1 ( θ 0 ) V ( θ 0 ) I − 1 ( θ 0 ) ] b i , t ∣ t − 1 ( θ 0 ) ) , \begin{aligned}
\sqrt{T}\left(\widehat P_{i,t|t-1}(\widehat\theta)-P_{i,t|t-1}(\theta_0)\right)\mid\mathcal F_{t-1} \\
&\Rightarrow\mathcal{N}\left(0,b_{i,t|t-1}^{\top}(\theta_0)\begin{bmatrix}\Gamma_F^{-1}\Xi_i\Gamma_F^{-1} & \mathbf{0}_{r\times d}\\ \mathbf{0}_{d\times r} & \mathcal{I}^{-1}(\theta_0)\mathbf{V}(\theta_0)\mathcal{I}^{-1}(\theta_0)\end{bmatrix}b_{i,t|t-1}(\theta_0)\right),
\end{aligned} T ( P i , t ∣ t − 1 ( θ ) − P i , t ∣ t − 1 ( θ 0 ) ) ∣ F t − 1 ⇒ N ( 0 , b i , t ∣ t − 1 ⊤ ( θ 0 ) [ Γ F − 1 Ξ i Γ F − 1 0 d × r 0 r × d I − 1 ( θ 0 ) V ( θ 0 ) I − 1 ( θ 0 ) ] b i , t ∣ t − 1 ( θ 0 ) ) ,
where
b i , t ∣ t − 1 ( θ 0 ) = [ σ t ∣ t − 1 2 ( θ 0 ) W r ⊤ ( I r ⊗ λ 0 i + λ 0 i ⊗ I r ) ∣ ( λ 0 i ⊗ λ 0 i ) ⊤ W r ∇ θ σ t ∣ t − 1 2 ( θ 0 ) ] . b_{i,t|t-1}(\theta_0)=\left[\sigma^2_{t|t-1}(\theta_0)\mathbf{W}_r^{\top}(\mathbf{I}_r\otimes\lambda_{0i}+\lambda_{0i}\otimes\mathbf{I}_r)\mid(\lambda_{0i}\otimes\lambda_{0i})^{\top}\mathbf{W}_r\nabla_{\theta}\sigma^2_{t|t-1}(\theta_0)\right]. b i , t ∣ t − 1 ( θ 0 ) = [ σ t ∣ t − 1 2 ( θ 0 ) W r ⊤ ( I r ⊗ λ 0 i + λ 0 i ⊗ I r ) ∣ ( λ 0 i ⊗ λ 0 i ) ⊤ W r ∇ θ σ t ∣ t − 1 2 ( θ 0 ) ] .
This completes the proof.
Proof of Theorem 5
From the result obtained in (171) we know that
min ( N , T ) ( C ^ i , t ∣ t − 1 ( θ ^ ) − C i , t ∣ t − 1 ( θ 0 ) ) = min ( N , T ) ( λ i O L S − λ 0 i ) ⊤ μ t ∣ t − 1 ( θ 0 ) + min ( N , T ) λ 0 i ⊤ ∇ θ μ t ∣ t − 1 ( θ 0 ) ( r H ^ − 1 ( θ ^ ) − θ 0 ) + min ( N , T ) λ 0 i ⊤ H ^ − 1 ( μ ^ t ∣ t − 1 ( θ ^ ) − H ^ μ t ∣ t − 1 ( r H ^ − 1 ( θ ^ ) ) ) + o P ( 1 ) . (183) \begin{aligned}
\min\left(\sqrt{N},\sqrt{T}\right)\left(\widehat{C}_{i,t|t-1}(\widehat{\theta})-C_{i,t|t-1}(\theta_0)\right)
={}&\min\left(\sqrt{N},\sqrt{T}\right)\left(\lambda_i^{\mathrm{OLS}}-\lambda_{0i}\right)^{\top}\mu_{t|t-1}(\theta_0)\\
&+\min\left(\sqrt{N},\sqrt{T}\right)\lambda_{0i}^{\top}\nabla_{\theta}\mu_{t|t-1}(\theta_0)\left(r_{\widehat{H}}^{-1}(\widehat{\theta})-\theta_0\right)\\
&+\min\left(\sqrt{N},\sqrt{T}\right)\lambda_{0i}^{\top}\widehat{H}^{-1}\left(\widehat{\mu}_{t|t-1}(\widehat{\theta})-\widehat{H}\mu_{t|t-1}\left(r_{\widehat{H}}^{-1}(\widehat{\theta})\right)\right)+o_{\mathbb{P}}(1). \tag*{(183)}
\end{aligned} min ( N , T ) ( C i , t ∣ t − 1 ( θ ) − C i , t ∣ t − 1 ( θ 0 ) ) = min ( N , T ) ( λ i OLS − λ 0 i ) ⊤ μ t ∣ t − 1 ( θ 0 ) + min ( N , T ) λ 0 i ⊤ ∇ θ μ t ∣ t − 1 ( θ 0 ) ( r H − 1 ( θ ) − θ 0 ) + min ( N , T ) λ 0 i ⊤ H − 1 ( μ t ∣ t − 1 ( θ ) − H μ t ∣ t − 1 ( r H − 1 ( θ ) ) ) + o P ( 1 ) . (183)
Now, from (174) , and using the decomposition in equation (54) in the proof of Proposition 1(c), we get
min ( N , T ) ( λ i O L S − λ 0 i ) ⊤ μ t ∣ t − 1 ( θ 0 ) = min ( N , T ) T ( 1 T ∑ s = 1 T F s e i s ) ⊤ Γ F − 1 μ t ∣ t − 1 ( θ 0 ) + O P ( min ( N , T ) N ) . (184) \begin{aligned}
\min\left(\sqrt{N},\sqrt{T}\right)\left(\lambda_i^{\mathrm{OLS}}-\lambda_{0i}\right)^{\top}\mu_{t|t-1}(\theta_0)
={}&\frac{\min\left(\sqrt{N},\sqrt{T}\right)}{\sqrt{T}}\left(\frac{1}{\sqrt{T}}\sum_{s=1}^{T}\mathbf{F}_s e_{is}\right)^{\top}\Gamma_F^{-1}\mu_{t|t-1}(\theta_0)\\
&+O_{\mathbb{P}}\left(\frac{\min\left(\sqrt{N},\sqrt{T}\right)}{N}\right). \tag*{(184)}
\end{aligned} min ( N , T ) ( λ i OLS − λ 0 i ) ⊤ μ t ∣ t − 1 ( θ 0 ) = T min ( N , T ) ( T 1 s = 1 ∑ T F s e i s ) ⊤ Γ F − 1 μ t ∣ t − 1 ( θ 0 ) + O P N min ( N , T ) . (184)
Then by Proposition 1(d)
( 1 T ∑ s = 1 T F s e i s ) ⊤ Γ F − 1 μ t ∣ t − 1 ( θ 0 ) ∣ F t − 1 ⇒ N ( 0 , μ t ∣ t − 1 ( θ 0 ) ⊤ Γ F − 1 Ξ i Γ F − 1 μ t ∣ t − 1 ( θ 0 ) ) , \left(\frac{1}{\sqrt{T}}\sum_{s=1}^{T}\mathbf{F}_s e_{is}\right)^{\top}\Gamma_F^{-1}\mu_{t|t-1}(\theta_0)\mid\mathcal{F}_{t-1}\Rightarrow\mathcal{N}\left(0,\mu_{t|t-1}(\theta_0)^{\top}\Gamma_F^{-1}\Xi_i\Gamma_F^{-1}\mu_{t|t-1}(\theta_0)\right), ( T 1 s = 1 ∑ T F s e i s ) ⊤ Γ F − 1 μ t ∣ t − 1 ( θ 0 ) ∣ F t − 1 ⇒ N ( 0 , μ t ∣ t − 1 ( θ 0 ) ⊤ Γ F − 1 Ξ i Γ F − 1 μ t ∣ t − 1 ( θ 0 ) ) ,
where Ξ i \Xi_i Ξ i is defined in Assumption 5(iv).
Moreover, applying the first-order condition and the Hessian expansion used in the proof of Theorem 2, together with Lemmas 9 and 10, we obtain, as N , T → ∞ N,T\to\infty N , T → ∞ ,
r H ^ − 1 ( θ ^ ) − θ 0 = I − 1 ( θ 0 ) 1 T ∇ θ ℓ F ( F ; θ 0 ) + O P ( 1 N + 1 N T + 1 T ) . (185) r_{\widehat{H}}^{-1}(\widehat{\theta})-\theta_0=\mathcal{I}^{-1}(\theta_0)\frac{1}{T}\nabla_{\theta}\ell_F(\mathbf{F};\theta_0)+O_{\mathbb{P}}\left(\frac{1}{N}+\frac{1}{\sqrt{NT}}+\frac{1}{T}\right). \tag*{(185)} r H − 1 ( θ ) − θ 0 = I − 1 ( θ 0 ) T 1 ∇ θ ℓ F ( F ; θ 0 ) + O P ( N 1 + N T 1 + T 1 ) . (185)
Consequently,
min ( N , T ) λ 0 i ⊤ ∇ θ μ t ∣ t − 1 ( θ 0 ) ( r H ^ − 1 ( θ ^ ) − θ 0 ) = min ( N , T ) T λ 0 i ⊤ ∇ θ μ t ∣ t − 1 ( θ 0 ) I ( θ 0 ) − 1 ( 1 T ∑ s = 1 T ∇ θ ℓ F ( F s ; θ 0 ) ) + O P ( min ( N , T ) N + min ( N , T ) T ) , (186) \begin{aligned}
\min\left(\sqrt{N},\sqrt{T}\right)\lambda_{0i}^{\top}\nabla^\theta\mu_{t|t-1}(\theta_0)\left(r_{\widehat{H}}^{-1}\left(\widehat{\theta}\right)-\theta_0\right) \\
&=\frac{\min\left(\sqrt{N},\sqrt{T}\right)}{\sqrt{T}}\lambda_{0i}^{\top}\nabla^\theta\mu_{t|t-1}(\theta_0)\mathcal{I}(\theta_0)^{-1}\left(\frac{1}{\sqrt{T}}\sum_{s=1}^{T}\nabla^\theta\ell_F(F_s;\theta_0)\right) \\
&+O_{\mathbb{P}}\left(\frac{\min\left(\sqrt{N},\sqrt{T}\right)}{N}+\frac{\min\left(\sqrt{N},\sqrt{T}\right)}{T}\right), \tag*{(186)}
\end{aligned} min ( N , T ) λ 0 i ⊤ ∇ θ μ t ∣ t − 1 ( θ 0 ) ( r H − 1 ( θ ) − θ 0 ) = T min ( N , T ) λ 0 i ⊤ ∇ θ μ t ∣ t − 1 ( θ 0 ) I ( θ 0 ) − 1 ( T 1 s = 1 ∑ T ∇ θ ℓ F ( F s ; θ 0 ) ) + O P N min ( N , T ) + T min ( N , T ) , (186)
Then by Lemma 11 (see also the proof of Theorem 2)
λ 0 i ⊤ ∇ θ μ t ∣ t − 1 ( θ 0 ) I ( θ 0 ) − 1 ( 1 T ∑ s = 1 T ∇ θ ℓ F ( F s ; θ 0 ) ) ∣ F t − 1 ⇒ N ( 0 , λ 0 i ⊤ ∇ θ μ t ∣ t − 1 ( θ 0 ) I ( θ 0 ) − 1 V ( θ 0 ) I ( θ 0 ) − 1 ∇ θ μ t ∣ t − 1 ( θ 0 ) ⊤ λ 0 i ) . \begin{aligned}
\lambda_{0i}^{\top}\nabla^\theta\mu_{t|t-1}(\theta_0)\mathcal{I}(\theta_0)^{-1}\left(\frac{1}{\sqrt{T}}\sum_{s=1}^{T}\nabla^\theta\ell_F(F_s;\theta_0)\right)\bigg|_{\mathcal{F}_{t-1}} \\
&\Rightarrow\mathcal{N}\left(0,\lambda_{0i}^{\top}\nabla^\theta\mu_{t|t-1}(\theta_0)\mathcal{I}(\theta_0)^{-1}V(\theta_0)\mathcal{I}(\theta_0)^{-1}\nabla^\theta\mu_{t|t-1}(\theta_0)^{\top}\lambda_{0i}\right).
\end{aligned} λ 0 i ⊤ ∇ θ μ t ∣ t − 1 ( θ 0 ) I ( θ 0 ) − 1 ( T 1 s = 1 ∑ T ∇ θ ℓ F ( F s ; θ 0 ) ) F t − 1 ⇒ N ( 0 , λ 0 i ⊤ ∇ θ μ t ∣ t − 1 ( θ 0 ) I ( θ 0 ) − 1 V ( θ 0 ) I ( θ 0 ) − 1 ∇ θ μ t ∣ t − 1 ( θ 0 ) ⊤ λ 0 i ) .
Finally, for the third term on the right-hand side of equation (183) , we have
H ^ − 1 ( μ ^ t ∣ t − 1 ( θ ^ ) − H ^ μ t ∣ t − 1 ( r H ^ − 1 ( θ ^ ) ) ) = H ^ − 1 ( Φ ^ μ ^ t − 1 ∣ t − 2 ( θ ^ ) + K ^ ( F ^ t − 1 − μ ^ t − 1 ∣ t − 2 ( θ ^ ) ) − Φ ^ H ^ μ t − 1 ∣ t − 2 ( r H ^ − 1 ( θ ^ ) ) \begin{aligned}
\widehat{H}^{-1}\left(\widehat{\mu}_{t|t-1}(\widehat{\theta})-\widehat{H}\mu_{t|t-1}\left(r_{\widehat{H}}^{-1}\left(\widehat{\theta}\right)\right)\right) \\
&=\widehat{H}^{-1}\left(\widehat{\Phi}\widehat{\mu}_{t-1|t-2}(\widehat{\theta})+\widehat{K}\left(\widehat{F}_{t-1}-\widehat{\mu}_{t-1|t-2}(\widehat{\theta})\right)-\widehat{\Phi}\widehat{H}\mu_{t-1|t-2}\left(r_{\widehat{H}}^{-1}\left(\widehat{\theta}\right)\right)
\right.
\end{aligned} H − 1 ( μ t ∣ t − 1 ( θ ) − H μ t ∣ t − 1 ( r H − 1 ( θ ) ) ) = H − 1 ( Φ μ t − 1∣ t − 2 ( θ ) + K ( F t − 1 − μ t − 1∣ t − 2 ( θ ) ) − Φ H μ t − 1∣ t − 2 ( r H − 1 ( θ ) )
− K ^ ( H ^ F t − 1 − H ^ μ t − 1 ∣ t − 2 ( r H ^ − 1 ( θ ^ ) ) ) ) = H ^ − 1 ( Φ ^ − K ^ ) H ^ H ^ − 1 ( μ ^ t − 1 ∣ t − 2 ( θ ^ ) − H ^ μ t − 1 ∣ t − 2 ( r H ^ − 1 ( θ ^ ) ) ) + H ^ − 1 K ^ ( F ^ t − 1 − H ^ F t − 1 ) = H ^ − 1 ( Φ ^ − K ^ ) H ^ H ^ − 1 ( μ ^ t − 1 ∣ t − 2 ( θ ^ ) − H ^ μ t − 1 ∣ t − 2 ( r H ^ − 1 ( θ ^ ) ) ) + H ^ − 1 K ^ H ^ ( Λ 0 ⊤ Λ 0 N ) − 1 Λ 0 ⊤ ε t − 1 N + O P ( 1 T ) = ( H ^ − 1 ( Φ ^ − K ^ ) H ^ − ( Φ 0 − K 0 ) + ( Φ 0 − K 0 ) ) H ^ − 1 ( μ ^ t − 1 ∣ t − 2 ( θ ^ ) − H ^ μ t − 1 ∣ t − 2 ( r H ^ − 1 ( θ ^ ) ) ) + ( H ^ − 1 K ^ H ^ − K 0 + K 0 ) ( Λ 0 ⊤ Λ 0 N ) − 1 Λ 0 ⊤ ε t − 1 N + O P ( 1 T ) = ( Φ 0 − K 0 ) H ^ − 1 ( μ ^ t − 1 ∣ t − 2 ( θ ^ ) − H ^ μ t − 1 ∣ t − 2 ( r H ^ − 1 ( θ ^ ) ) ) + K 0 ( Λ 0 ⊤ Λ 0 N ) − 1 Λ 0 ⊤ ε t − 1 N + O P ( 1 T ) + O P ( 1 N T ) = ∑ k = 0 t − 1 ( Φ 0 − K 0 ) k K 0 ( Λ 0 T Λ 0 N ) − 1 Λ 0 T e t − k − 1 N + O P ( max ( 1 T , 1 N T ) ) , \begin{aligned}
\left.
-\widehat{K}\left(\widehat{H}F_{t-1}-\widehat{H}\mu_{t-1|t-2}\left(r_{\widehat{H}}^{-1}\left(\widehat{\theta}\right)\right)\right)\right) \\
&=\widehat{H}^{-1}\left(\widehat{\Phi}-\widehat{K}\right)\widehat{H}\widehat{H}^{-1}\left(\widehat{\mu}_{t-1|t-2}(\widehat{\theta})-\widehat{H}\mu_{t-1|t-2}\left(r_{\widehat{H}}^{-1}\left(\widehat{\theta}\right)\right)\right)+\widehat{H}^{-1}\widehat{K}\left(\widehat{F}_{t-1}-\widehat{H}F_{t-1}\right) \\
&=\widehat{H}^{-1}\left(\widehat{\Phi}-\widehat{K}\right)\widehat{H}\widehat{H}^{-1}\left(\widehat{\mu}_{t-1|t-2}(\widehat{\theta})-\widehat{H}\mu_{t-1|t-2}\left(r_{\widehat{H}}^{-1}\left(\widehat{\theta}\right)\right)\right) \\
&+\widehat{H}^{-1}\widehat{K}\widehat{H}\left(\frac{\Lambda_0^{\top}\Lambda_0}{N}\right)^{-1}\frac{\Lambda_0^{\top}\varepsilon_{t-1}}{N}+O_{\mathbb{P}}\left(\frac{1}{T}\right) \\
&=\left(\widehat{H}^{-1}\left(\widehat{\Phi}-\widehat{K}\right)\widehat{H}-(\Phi_0-K_0)+(\Phi_0-K_0)\right)\widehat{H}^{-1}\left(\widehat{\mu}_{t-1|t-2}(\widehat{\theta})-\widehat{H}\mu_{t-1|t-2}\left(r_{\widehat{H}}^{-1}\left(\widehat{\theta}\right)\right)\right) \\
&+\left(\widehat{H}^{-1}\widehat{K}\widehat{H}-K_0+K_0\right)\left(\frac{\Lambda_0^{\top}\Lambda_0}{N}\right)^{-1}\frac{\Lambda_0^{\top}\varepsilon_{t-1}}{N}+O_{\mathbb{P}}\left(\frac{1}{T}\right) \\
&=(\Phi_0-K_0)\widehat{H}^{-1}\left(\widehat{\mu}_{t-1|t-2}(\widehat{\theta})-\widehat{H}\mu_{t-1|t-2}\left(r_{\widehat{H}}^{-1}\left(\widehat{\theta}\right)\right)\right) \\
&+K_0\left(\frac{\Lambda_0^{\top}\Lambda_0}{N}\right)^{-1}\frac{\Lambda_0^{\top}\varepsilon_{t-1}}{N}+O_{\mathbb{P}}\left(\frac{1}{T}\right)+O_{\mathbb{P}}\left(\frac{1}{\sqrt{NT}}\right) \\
&= \sum_{k=0}^{t-1} (\Phi_0-K_0)^k K_0 \left(\frac{\Lambda_0^{\mathsf T}\Lambda_0}{N}\right)^{-1}\frac{\Lambda_0^{\mathsf T}e_{t-k-1}}{N}+O_{\mathbb{P}}\left(\max\left(\frac{1}{T},\frac{1}{\sqrt{NT}}\right)\right),
\end{aligned} − K ( H F t − 1 − H μ t − 1∣ t − 2 ( r H − 1 ( θ ) ) ) ) = H − 1 ( Φ − K ) H H − 1 ( μ t − 1∣ t − 2 ( θ ) − H μ t − 1∣ t − 2 ( r H − 1 ( θ ) ) ) + H − 1 K ( F t − 1 − H F t − 1 ) = H − 1 ( Φ − K ) H H − 1 ( μ t − 1∣ t − 2 ( θ ) − H μ t − 1∣ t − 2 ( r H − 1 ( θ ) ) ) + H − 1 K H ( N Λ 0 ⊤ Λ 0 ) − 1 N Λ 0 ⊤ ε t − 1 + O P ( T 1 ) = ( H − 1 ( Φ − K ) H − ( Φ 0 − K 0 ) + ( Φ 0 − K 0 ) ) H − 1 ( μ t − 1∣ t − 2 ( θ ) − H μ t − 1∣ t − 2 ( r H − 1 ( θ ) ) ) + ( H − 1 K H − K 0 + K 0 ) ( N Λ 0 ⊤ Λ 0 ) − 1 N Λ 0 ⊤ ε t − 1 + O P ( T 1 ) = ( Φ 0 − K 0 ) H − 1 ( μ t − 1∣ t − 2 ( θ ) − H μ t − 1∣ t − 2 ( r H − 1 ( θ ) ) ) + K 0 ( N Λ 0 ⊤ Λ 0 ) − 1 N Λ 0 ⊤ ε t − 1 + O P ( T 1 ) + O P ( N T 1 ) = k = 0 ∑ t − 1 ( Φ 0 − K 0 ) k K 0 ( N Λ 0 T Λ 0 ) − 1 N Λ 0 T e t − k − 1 + O P ( max ( T 1 , N T 1 ) ) ,
by Theorem 1 and Proposition 4. Therefore, as N , T → ∞ N,T \to\infty N , T → ∞ ,
min ( N , T ) λ 0 i T H ^ − 1 ( μ ^ t ∣ t − 1 ( θ ^ ) − H ^ μ t ∣ t − 1 ( r H ^ − 1 ( θ ^ ) ) ) = min ( N , T ) N λ 0 i T ∑ k = 0 t − 1 ( Φ 0 − K 0 ) k K 0 ( Λ 0 T Λ 0 N ) − 1 Λ 0 T e t − k − 1 N + O P ( min ( N , T ) T + min ( N , T ) N T ) . (187) \begin{aligned}
\min(\sqrt{N},\sqrt{T})\lambda_{0i}^{\mathsf T}\widehat{H}^{-1}\left(\widehat{\mu}_{t|t-1}(\widehat{\theta})-\widehat{H}\mu_{t|t-1}\left(r_{\widehat{H}}^{-1}(\widehat{\theta})\right)\right)
&=\frac{\min(\sqrt{N},\sqrt{T})}{\sqrt{N}}\lambda_{0i}^{\mathsf T}\sum_{k=0}^{t-1}(\Phi_0-K_0)^kK_0\left(\frac{\Lambda_0^{\mathsf T}\Lambda_0}{N}\right)^{-1}\frac{\Lambda_0^{\mathsf T}e_{t-k-1}}{\sqrt{N}}\\
&\quad+O_{\mathbb{P}}\left(\frac{\min(\sqrt{N},\sqrt{T})}{T}+\frac{\min(\sqrt{N},\sqrt{T})}{\sqrt{NT}}\right). \tag*{(187)}
\end{aligned} min ( N , T ) λ 0 i T H − 1 ( μ t ∣ t − 1 ( θ ) − H μ t ∣ t − 1 ( r H − 1 ( θ ) ) ) = N min ( N , T ) λ 0 i T k = 0 ∑ t − 1 ( Φ 0 − K 0 ) k K 0 ( N Λ 0 T Λ 0 ) − 1 N Λ 0 T e t − k − 1 + O P ( T min ( N , T ) + N T min ( N , T ) ) . (187)
Then, by the finite-dimensional result in Proposition 1(b) and the Cramér–Wold device,
λ 0 i T ∑ k = 0 t − 1 ( Φ 0 − K 0 ) k K 0 ( Λ 0 T Λ 0 N ) − 1 Λ 0 T e t − k − 1 N ∣ F t − 1 ⇒ N ( 0 , λ 0 i T ( ∑ k = 0 t − 1 ∑ h = 0 t − 1 ( Φ 0 − K 0 ) k K 0 Σ Λ − 1 Π t − k − 1 , t − h − 1 Σ Λ − 1 K 0 T ( Φ 0 − K 0 ) h T ) λ 0 i ) . \lambda_{0i}^{\mathsf T}\sum_{k=0}^{t-1}(\Phi_0-K_0)^kK_0\left(\frac{\Lambda_0^{\mathsf T}\Lambda_0}{N}\right)^{-1}\frac{\Lambda_0^{\mathsf T}e_{t-k-1}}{\sqrt{N}}\mid\mathcal{F}_{t-1}
\Rightarrow\mathcal{N}\left(0,\lambda_{0i}^{\mathsf T}\left(\sum_{k=0}^{t-1}\sum_{h=0}^{t-1}(\Phi_0-K_0)^kK_0\Sigma_{\Lambda}^{-1}\Pi_{t-k-1,t-h-1}\Sigma_{\Lambda}^{-1}K_0^{\mathsf T}(\Phi_0-K_0)^{h\mathsf T}\right)\lambda_{0i}\right). λ 0 i T k = 0 ∑ t − 1 ( Φ 0 − K 0 ) k K 0 ( N Λ 0 T Λ 0 ) − 1 N Λ 0 T e t − k − 1 ∣ F t − 1 ⇒ N ( 0 , λ 0 i T ( k = 0 ∑ t − 1 h = 0 ∑ t − 1 ( Φ 0 − K 0 ) k K 0 Σ Λ − 1 Π t − k − 1 , t − h − 1 Σ Λ − 1 K 0 T ( Φ 0 − K 0 ) h T ) λ 0 i ) .
The leading term in (184) is the time-series loading term, the leading term in (186) is the QMLE-score term, and the leading term in (187) is the cross-sectional factor-estimation term. Assumptions 4–5, Proposition 1(b) and (d), Lemma 11, and the Cramér–Wold device imply their joint Gaussian convergence. The asymptotic covariance between the first two terms is zero by (178) . The covariance between the QMLE-score term and the factor-estimation term is zero by Assumption 5(i) and E [ e i t ] = 0 \mathbb{E}[e_{it}]=0 E [ e i t ] = 0 , whereas the covariance between the loading term and the factor-estimation term converges to zero by Assumption 4(ii). Thus, from (183) ,
min ( N , T ) ( C ^ i , t ∣ t − 1 ( θ ^ ) − C i , t ∣ t − 1 ( θ 0 ) ) = min ( N , T ) T ( 1 T ∑ s = 1 T F s e i s ) T Γ F − 1 μ t ∣ t − 1 ( θ 0 ) + min ( N , T ) T λ 0 i T ∇ θ μ t ∣ t − 1 ( θ 0 ) I ( θ 0 ) − 1 ( 1 T ∑ s = 1 T ∇ θ ℓ F ( F s ; θ 0 ) ) + min ( N , T ) N λ 0 i T ∑ k = 0 t − 1 ( Φ 0 − K 0 ) k K 0 ( Λ 0 T Λ 0 N ) − 1 Λ 0 T e t − k − 1 N + O P ( 1 min ( N , T ) ) . \begin{aligned}
\min(\sqrt{N},\sqrt{T})\left(\widehat{C}_{i,t|t-1}(\widehat{\theta})-C_{i,t|t-1}(\theta_0)\right)
&=\frac{\min(\sqrt{N},\sqrt{T})}{\sqrt{T}}\left(\frac{1}{\sqrt{T}}\sum_{s=1}^{T}F_se_{is}\right)^{\mathsf T}\Gamma_F^{-1}\mu_{t|t-1}(\theta_0)\\
&\quad+\frac{\min(\sqrt{N},\sqrt{T})}{\sqrt{T}}\lambda_{0i}^{\mathsf T}\nabla^\theta\mu_{t|t-1}(\theta_0)\mathcal{I}(\theta_0)^{-1}\left(\frac{1}{\sqrt{T}}\sum_{s=1}^{T}\nabla^\theta\ell_F(F_s;\theta_0)\right)\\
&\quad+\frac{\min(\sqrt{N},\sqrt{T})}{\sqrt{N}}\lambda_{0i}^{\mathsf T}\sum_{k=0}^{t-1}(\Phi_0-K_0)^kK_0\left(\frac{\Lambda_0^{\mathsf T}\Lambda_0}{N}\right)^{-1}\frac{\Lambda_0^{\mathsf T}e_{t-k-1}}{\sqrt{N}} \\
&+ O_{\mathsf{P}}\left(\frac{1}{\min\left(\sqrt{N},\sqrt{T}\right)}\right).
\end{aligned} min ( N , T ) ( C i , t ∣ t − 1 ( θ ) − C i , t ∣ t − 1 ( θ 0 ) ) = T min ( N , T ) ( T 1 s = 1 ∑ T F s e i s ) T Γ F − 1 μ t ∣ t − 1 ( θ 0 ) + T min ( N , T ) λ 0 i T ∇ θ μ t ∣ t − 1 ( θ 0 ) I ( θ 0 ) − 1 ( T 1 s = 1 ∑ T ∇ θ ℓ F ( F s ; θ 0 ) ) + N min ( N , T ) λ 0 i T k = 0 ∑ t − 1 ( Φ 0 − K 0 ) k K 0 ( N Λ 0 T Λ 0 ) − 1 N Λ 0 T e t − k − 1 + O P min ( N , T ) 1 .
The preceding decomposition is the analogue of equation (C.1) in the proof of Theorem 3 of Bai [2 ] , augmented by the QMLE-score term. We therefore adapt the two-rate argument used therein. To conclude without imposing a restriction on the relative rate of N N N and T T T , consider an arbitrary subsequence. It admits a further subsequence along which N / T N/T N / T converges to an element of [ 0 , ∞ ) [0,\infty) [ 0 , ∞ ) . If N / T → 0 N/T \to0 N / T → 0 , the N − 1 / 2 N^{-1/2} N − 1/2 factor-estimation term is the only non-negligible component under the normalization in the statement, and N / T → 0 \sqrt{N}/T \to0 N / T → 0 . If N / T → ∞ N/T \to\infty N / T → ∞ , the T − 1 / 2 T^{-1/2} T − 1/2 loading and QMLE-score terms are the only non-negligible components, and T / N → 0 \sqrt{T}/N \to0 T / N → 0 . If N / T N/T N / T converges to a finite positive real, all three components may contribute and both rate conditions hold. Hence, along every such subsequence, division by the conditional standard deviation in the statement gives a standard Gaussian limit. The subsequence principle proves the first result of the theorem as N , T → ∞ N,T \to\infty N , T → ∞ .
For the conditional variance, we proceed in a similar way. We first use (180) , and write
min ( N , T ) ( P ^ i , t ∣ t − 1 ( θ ^ ) − P i , t ∣ t − 1 ( θ 0 ) ) = min ( N , T ) ( ( λ ^ i O L S ⊗ λ ^ i O L S ) T − ( λ 0 i ⊗ λ 0 i ) T ) W r σ t ∣ t − 1 2 ( θ 0 ) + min ( N , T ) ( λ 0 i ⊗ λ 0 i ) T W r ∇ θ σ t ∣ t − 1 2 ( θ 0 ) ( r H − 1 ( θ ^ ) − θ 0 ) + min ( N , T ) ( λ 0 i T H ^ − 1 ⊗ λ 0 i T H ^ − 1 ) ( W r σ ^ t ∣ t − 1 2 ( θ ^ ) − ( H ^ ⊗ H ^ ) W r σ t ∣ t − 1 2 ( r H ^ − 1 ( θ ^ ) ) ) + o P ( 1 ) , (188) \begin{aligned}
&\min\left(\sqrt{N},\sqrt{T}\right)\left(\widehat{P}_{i,t\mid t-1}(\widehat{\theta})-P_{i,t\mid t-1}(\theta_0)\right) \\
={}&\min\left(\sqrt{N},\sqrt{T}\right)\left(\left(\widehat{\lambda}^{\mathrm{OLS}}_i\otimes\widehat{\lambda}^{\mathrm{OLS}}_i\right)^{\mathrm{T}}-\left(\lambda_{0i}\otimes\lambda_{0i}\right)^{\mathrm{T}}\right)W_r\sigma^2_{t\mid t-1}(\theta_0) \\
&+\min\left(\sqrt{N},\sqrt{T}\right)\left(\lambda_{0i}\otimes\lambda_{0i}\right)^{\mathrm{T}}W_r\nabla^{\theta}\sigma^2_{t\mid t-1}(\theta_0)\left(r_H^{-1}(\widehat{\theta})-\theta_0\right) \\
&+\min\left(\sqrt{N},\sqrt{T}\right)\left(\lambda_{0i}^{\mathrm{T}}\widehat{H}^{-1}\otimes\lambda_{0i}^{\mathrm{T}}\widehat{H}^{-1}\right)\left(W_r\widehat{\sigma}^{2}_{t\mid t-1}(\widehat{\theta})-\left(\widehat{H}\otimes\widehat{H}\right)W_r\sigma^2_{t\mid t-1}\left(r_{\widehat{H}}^{-1}(\widehat{\theta})\right)\right)+o_{\mathsf{P}}(1), \tag*{(188)}
\end{aligned} = min ( N , T ) ( P i , t ∣ t − 1 ( θ ) − P i , t ∣ t − 1 ( θ 0 ) ) min ( N , T ) ( ( λ i OLS ⊗ λ i OLS ) T − ( λ 0 i ⊗ λ 0 i ) T ) W r σ t ∣ t − 1 2 ( θ 0 ) + min ( N , T ) ( λ 0 i ⊗ λ 0 i ) T W r ∇ θ σ t ∣ t − 1 2 ( θ 0 ) ( r H − 1 ( θ ) − θ 0 ) + min ( N , T ) ( λ 0 i T H − 1 ⊗ λ 0 i T H − 1 ) ( W r σ t ∣ t − 1 2 ( θ ) − ( H ⊗ H ) W r σ t ∣ t − 1 2 ( r H − 1 ( θ ) ) ) + o P ( 1 ) , (188)
and then, for the first term, on the right-hand side of the latter expression we can directly use the same arguments as those applied for the derivation of (182) , to obtain
min ( N , T ) ( ( λ ^ i O L S ⊗ λ ^ i O L S ) T − ( λ 0 i ⊗ λ 0 i ) T ) W r σ t ∣ t − 1 2 ( θ 0 ) = min ( N , T ) ( λ ^ i O L S − λ 0 i ) T ( I r ⊗ λ 0 i + λ 0 i ⊗ I r ) T W r σ t ∣ t − 1 2 ( θ 0 ) = min ( N , T ) T ( 1 T ∑ s = 1 T F s e i s ) T Γ F − 1 ( I r ⊗ λ 0 i + λ 0 i ⊗ I r ) T W r σ t ∣ t − 1 2 ( θ 0 ) + O P ( min ( N , T ) N ) . (189) \begin{aligned}
&\min\left(\sqrt{N},\sqrt{T}\right)\left(\left(\widehat{\lambda}^{\mathrm{OLS}}_i\otimes\widehat{\lambda}^{\mathrm{OLS}}_i\right)^{\mathrm{T}}-\left(\lambda_{0i}\otimes\lambda_{0i}\right)^{\mathrm{T}}\right)W_r\sigma^2_{t\mid t-1}(\theta_0) \\
={}&\min\left(\sqrt{N},\sqrt{T}\right)\left(\widehat{\lambda}^{\mathrm{OLS}}_i-\lambda_{0i}\right)^{\mathrm{T}}\left(I_r\otimes\lambda_{0i}+\lambda_{0i}\otimes I_r\right)^{\mathrm{T}}W_r\sigma^2_{t\mid t-1}(\theta_0) \\
={}&\frac{\min\left(\sqrt{N},\sqrt{T}\right)}{\sqrt{T}}\left(\frac{1}{\sqrt{T}}\sum_{s=1}^{T}F_s e_{is}\right)^{\mathrm{T}}\Gamma_F^{-1}\left(I_r\otimes\lambda_{0i}+\lambda_{0i}\otimes I_r\right)^{\mathrm{T}}W_r\sigma^2_{t\mid t-1}(\theta_0) \\
&+O_{\mathsf{P}}\left(\frac{\min\left(\sqrt{N},\sqrt{T}\right)}{N}\right). \tag*{(189)}
\end{aligned} = = min ( N , T ) ( ( λ i OLS ⊗ λ i OLS ) T − ( λ 0 i ⊗ λ 0 i ) T ) W r σ t ∣ t − 1 2 ( θ 0 ) min ( N , T ) ( λ i OLS − λ 0 i ) T ( I r ⊗ λ 0 i + λ 0 i ⊗ I r ) T W r σ t ∣ t − 1 2 ( θ 0 ) T min ( N , T ) ( T 1 s = 1 ∑ T F s e i s ) T Γ F − 1 ( I r ⊗ λ 0 i + λ 0 i ⊗ I r ) T W r σ t ∣ t − 1 2 ( θ 0 ) + O P N min ( N , T ) . (189)
Then by Proposition 1(d)
( 1 T ∑ s = 1 T F s ε i s ) T Γ F − 1 ( I r ⊗ λ 0 i + λ 0 i ⊗ I r ) T W r σ t ∣ t − 1 2 ( θ 0 ) ∣ F t − 1 ⇒ N ( 0 , σ t ∣ t − 1 2 ( θ 0 ) W r T ( I r ⊗ λ 0 i + λ 0 i ⊗ I r ) Γ F − 1 Ξ i Γ F − 1 \begin{aligned}
\left(\frac{1}{\sqrt{T}}\sum_{s=1}^{T}F_s\varepsilon_{is}\right)^{\mathsf{T}}\Gamma_F^{-1}\left(I_r\otimes\lambda_{0i}+\lambda_{0i}\otimes I_r\right)^{\mathsf{T}}W_r\sigma_{t|t-1}^{2}(\theta_0)\mid\mathcal{F}_{t-1} \\
&\Rightarrow\mathcal{N}\left(0,\sigma_{t|t-1}^{2}(\theta_0)W_r^{\mathsf{T}}\left(I_r\otimes\lambda_{0i}+\lambda_{0i}\otimes I_r\right)\Gamma_F^{-1}\Xi_i\Gamma_F^{-1}
\right.
\end{aligned} ( T 1 s = 1 ∑ T F s ε i s ) T Γ F − 1 ( I r ⊗ λ 0 i + λ 0 i ⊗ I r ) T W r σ t ∣ t − 1 2 ( θ 0 ) ∣ F t − 1 ⇒ N ( 0 , σ t ∣ t − 1 2 ( θ 0 ) W r T ( I r ⊗ λ 0 i + λ 0 i ⊗ I r ) Γ F − 1 Ξ i Γ F − 1
× ( I r ⊗ λ 0 i + λ 0 i ⊗ I r ) T W r σ t ∣ t − 1 2 ( θ 0 ) ) , \left.
\times\left(I_r\otimes\lambda_{0i}+\lambda_{0i}\otimes I_r\right)^{\mathsf{T}}W_r\sigma_{t|t-1}^{2}(\theta_0)\right), × ( I r ⊗ λ 0 i + λ 0 i ⊗ I r ) T W r σ t ∣ t − 1 2 ( θ 0 ) ) ,
where Ξ i \Xi_i Ξ i is defined in Assumption 5(iv).
Moreover, by (185) , we obtain
min ( N , T ) ( λ 0 i ⊗ λ 0 i ) T W r ∇ θ σ t ∣ t − 1 2 ( θ 0 ) ( r ^ H ^ − 1 ( θ ^ ) − θ 0 ) = min ( N , T ) T ( λ 0 i ⊗ λ 0 i ) T W r ∇ θ σ t ∣ t − 1 2 ( θ 0 ) I ( θ 0 ) − 1 ( 1 T ∑ s = 1 T ∇ θ ℓ F ( F s ; θ 0 ) ) + O P ( min ( N , T ) N + min ( N , T ) T ) , (190) \begin{aligned}
\min\left(\sqrt{N},\sqrt{T}\right)(\lambda_{0i}\otimes\lambda_{0i})^{\mathsf{T}}W_r\nabla^\theta\sigma_{t|t-1}^{2}(\theta_0)\left(\hat{r}_{\widehat{H}}^{-1}(\hat{\theta})-\theta_0\right) \\
&=\frac{\min\left(\sqrt{N},\sqrt{T}\right)}{\sqrt{T}}(\lambda_{0i}\otimes\lambda_{0i})^{\mathsf{T}}W_r\nabla^\theta\sigma_{t|t-1}^{2}(\theta_0)\mathcal{I}(\theta_0)^{-1}
\left(\frac{1}{\sqrt{T}}\sum_{s=1}^{T}\nabla^\theta\ell_F(F_s;\theta_0)\right) \\
&+O_{\mathbb{P}}\left(\frac{\min\left(\sqrt{N},\sqrt{T}\right)}{N}+\frac{\min\left(\sqrt{N},\sqrt{T}\right)}{T}\right), \tag*{(190)}
\end{aligned} min ( N , T ) ( λ 0 i ⊗ λ 0 i ) T W r ∇ θ σ t ∣ t − 1 2 ( θ 0 ) ( r ^ H − 1 ( θ ^ ) − θ 0 ) = T min ( N , T ) ( λ 0 i ⊗ λ 0 i ) T W r ∇ θ σ t ∣ t − 1 2 ( θ 0 ) I ( θ 0 ) − 1 ( T 1 s = 1 ∑ T ∇ θ ℓ F ( F s ; θ 0 ) ) + O P N min ( N , T ) + T min ( N , T ) , (190)
Then by Lemma 11 (see also the proof of Theorem 2)
( λ 0 i ⊗ λ 0 i ) T W r ∇ θ σ t ∣ t − 1 2 ( θ 0 ) I ( θ 0 ) − 1 ( 1 T ∑ s = 1 T ∇ θ ℓ F ( F s ; θ 0 ) ) ∣ F t − 1 ⇒ N ( 0 , ( λ 0 i ⊗ λ 0 i ) T W r ∇ θ σ t ∣ t − 1 2 ( θ 0 ) I ( θ 0 ) − 1 V ( θ 0 ) I ( θ 0 ) − 1 ∇ θ σ t ∣ t − 1 2 ( θ 0 ) T W r T ( λ 0 i ⊗ λ 0 i ) ) . \begin{aligned}
(\lambda_{0i}\otimes\lambda_{0i})^{\mathsf{T}}W_r\nabla^\theta\sigma_{t|t-1}^{2}(\theta_0)\mathcal{I}(\theta_0)^{-1}
\left(\frac{1}{\sqrt{T}}\sum_{s=1}^{T}\nabla^\theta\ell_F(F_s;\theta_0)\right)\Bigg|\mathcal{F}_{t-1} \\
&\Rightarrow\mathcal{N}\left(0,(\lambda_{0i}\otimes\lambda_{0i})^{\mathsf{T}}W_r\nabla^\theta\sigma_{t|t-1}^{2}(\theta_0)\mathcal{I}(\theta_0)^{-1}\mathcal{V}(\theta_0)\mathcal{I}(\theta_0)^{-1}\nabla^\theta\sigma_{t|t-1}^{2}(\theta_0)^{\mathsf{T}}W_r^{\mathsf{T}}(\lambda_{0i}\otimes\lambda_{0i})\right).
\end{aligned} ( λ 0 i ⊗ λ 0 i ) T W r ∇ θ σ t ∣ t − 1 2 ( θ 0 ) I ( θ 0 ) − 1 ( T 1 s = 1 ∑ T ∇ θ ℓ F ( F s ; θ 0 ) ) F t − 1 ⇒ N ( 0 , ( λ 0 i ⊗ λ 0 i ) T W r ∇ θ σ t ∣ t − 1 2 ( θ 0 ) I ( θ 0 ) − 1 V ( θ 0 ) I ( θ 0 ) − 1 ∇ θ σ t ∣ t − 1 2 ( θ 0 ) T W r T ( λ 0 i ⊗ λ 0 i ) ) .
In conclusion, for the third term on the right-hand side of equation (188) , by performing some algebra, we can use the fact that
( ( H ^ − 1 ⊗ H ^ − 1 ) W r σ ^ t ∣ t − 1 2 ( θ ^ ) − W r σ t ∣ t − 1 2 ( r ^ H ^ − 1 ( θ ^ ) ) ) = ( H ^ − 1 ⊗ H ^ − 1 ) W r ( Φ ^ − K ^ ) W r T ( H ^ ⊗ H ^ ) × ( ( H ^ − 1 ⊗ H ^ − 1 ) W r σ ^ t − 1 ∣ t − 2 2 ( θ ^ ) − W r σ t − 1 ∣ t − 2 2 ( r ^ H ^ − 1 ( θ ^ ) ) ) + ( H ^ − 1 ⊗ H ^ − 1 ) W r K ^ W r T ( F ^ t − 1 ⊗ F ^ t − 1 − H ^ F t − 1 ⊗ H ^ F t − 1 ) = ( H ^ − 1 ⊗ H ^ − 1 ) W r ( Φ ^ − K ^ ) W r T ( H ^ ⊗ H ^ ) \begin{aligned}
\left(\left(\widehat{H}^{-1}\otimes\widehat{H}^{-1}\right)W_r\widehat{\sigma}_{t|t-1}^{2}(\hat{\theta})-W_r\sigma_{t|t-1}^{2}\left(\hat{r}_{\widehat{H}}^{-1}(\hat{\theta})\right)\right) \\
&=\left(\widehat{H}^{-1}\otimes\widehat{H}^{-1}\right)W_r(\widehat{\Phi}-\widehat{K})W_r^{\mathsf{T}}(\widehat{H}\otimes\widehat{H}) \\
&\times\left(\left(\widehat{H}^{-1}\otimes\widehat{H}^{-1}\right)W_r\widehat{\sigma}_{t-1|t-2}^{2}(\hat{\theta})-W_r\sigma_{t-1|t-2}^{2}\left(\hat{r}_{\widehat{H}}^{-1}(\hat{\theta})\right)\right) \\
&+\left(\widehat{H}^{-1}\otimes\widehat{H}^{-1}\right)W_r\widehat{K}W_r^{\mathsf{T}}\left(\widehat{F}_{t-1}\otimes\widehat{F}_{t-1}-\widehat{H}F_{t-1}\otimes\widehat{H}F_{t-1}\right) \\
&=\left(\widehat{H}^{-1}\otimes\widehat{H}^{-1}\right)W_r(\widehat{\Phi}-\widehat{K})W_r^{\mathsf{T}}(\widehat{H}\otimes\widehat{H})
\end{aligned} ( ( H − 1 ⊗ H − 1 ) W r σ t ∣ t − 1 2 ( θ ^ ) − W r σ t ∣ t − 1 2 ( r ^ H − 1 ( θ ^ ) ) ) = ( H − 1 ⊗ H − 1 ) W r ( Φ − K ) W r T ( H ⊗ H ) × ( ( H − 1 ⊗ H − 1 ) W r σ t − 1∣ t − 2 2 ( θ ^ ) − W r σ t − 1∣ t − 2 2 ( r ^ H − 1 ( θ ^ ) ) ) + ( H − 1 ⊗ H − 1 ) W r K W r T ( F t − 1 ⊗ F t − 1 − H F t − 1 ⊗ H F t − 1 ) = ( H − 1 ⊗ H − 1 ) W r ( Φ − K ) W r T ( H ⊗ H )
× ( ( H ^ − 1 ⊗ H ^ − 1 ) W r σ ^ t − 1 ∣ t − 2 2 ( θ ^ ) − W r σ t − 1 ∣ t − 2 2 ( r H ^ − 1 ( θ ^ ) ) ) + ( H ^ − 1 ⊗ H ^ − 1 ) W r K ^ W r T ( H ^ ⊗ H ^ ) ( ( H ^ − 1 ⊗ H ^ − 1 ) ( F ^ t − 1 ⊗ F ^ t − 1 ) − ( F t − 1 ⊗ F t − 1 ) ) = ( ( H ^ − 1 ⊗ H ^ − 1 ) W r ( Φ ^ − K ^ ) W r T ( H ^ ⊗ H ^ ) − W r ( Φ 0 − K 0 ) W r T + W r ( Φ 0 − K 0 ) W r T ) × ( ( H ^ − 1 ⊗ H ^ − 1 ) W r σ ^ t − 1 ∣ t − 2 2 ( θ ^ ) − W r σ t − 1 ∣ t − 2 2 ( r H ^ − 1 ( θ ^ ) ) ) + ( ( H ^ − 1 ⊗ H ^ − 1 ) W r K ^ W r T ( H ^ ⊗ H ^ ) − W r K 0 W r T + W r K 0 W r T ) × ( H ^ − 1 ⊗ H ^ − 1 ) ( F ^ t − 1 ⊗ F ^ t − 1 − H ^ F t − 1 ⊗ H ^ F t − 1 ) = W r ( Φ 0 − K 0 ) W r T ( ( H ^ − 1 ⊗ H ^ − 1 ) W r σ ^ t − 1 ∣ t − 2 2 ( θ ^ ) − W r σ t − 1 ∣ t − 2 2 ( r H ^ − 1 ( θ ^ ) ) ) + W r K 0 W r T ( H ^ − 1 ⊗ H ^ − 1 ) ( F ^ t − 1 ⊗ F ^ t − 1 − H ^ F t − 1 ⊗ H ^ F t − 1 ) + O P ( 1 N + 1 T + 1 N T ) = W r ( Φ 0 − K 0 ) W r T ( ( H ^ − 1 ⊗ H ^ − 1 ) W r σ ^ t − 1 ∣ t − 2 2 ( θ ^ ) − W r σ t − 1 ∣ t − 2 2 ( r H ^ − 1 ( θ ^ ) ) ) + W r K 0 W r T ( H ^ − 1 ⊗ H ^ − 1 ) ( I r ⊗ H ^ F t − 1 + H ^ F t − 1 ⊗ I r ) ( F ^ t − 1 − H ^ F t − 1 ) + O P ( 1 N + 1 T + 1 N T ) = W r ( Φ 0 − K 0 ) W r T ( ( H ^ − 1 ⊗ H ^ − 1 ) W r σ ^ t − 1 ∣ t − 2 2 ( θ ^ ) − W r σ t − 1 ∣ t − 2 2 ( r H ^ − 1 ( θ ^ ) ) ) + W r K 0 W r T ( I r ⊗ F t − 1 + F t − 1 ⊗ I r ) ( Λ 0 T Λ 0 N ) − 1 Λ 0 T e t − 1 N + O P ( 1 N + 1 T + 1 N T ) = ∑ k = 0 t − 1 ( W r ( Φ 0 − K 0 ) W r T ) k W r K 0 W r T ( I r ⊗ F t − k − 1 + F t − k − 1 ⊗ I r ) ( Λ 0 T Λ 0 N ) − 1 Λ 0 T e t − k − 1 N + O P ( 1 N + 1 T + 1 N T ) , \begin{aligned}
&\times\left(\left(\widehat{H}^{-1}\otimes\widehat{H}^{-1}\right)W_r\widehat{\sigma}^{2}_{t-1|t-2}(\widehat{\theta})-W_r\sigma^{2}_{t-1|t-2}\left(r_{\widehat{H}}^{-1}(\widehat{\theta})\right)\right)\\
&+\left(\widehat{H}^{-1}\otimes\widehat{H}^{-1}\right)W_r\widehat{K}W_r^{\mathsf{T}}\left(\widehat{H}\otimes\widehat{H}\right)\left(\left(\widehat{H}^{-1}\otimes\widehat{H}^{-1}\right)\left(\widehat{F}_{t-1}\otimes\widehat{F}_{t-1}\right)-\left(F_{t-1}\otimes F_{t-1}\right)\right)\\
&=\left(\left(\widehat{H}^{-1}\otimes\widehat{H}^{-1}\right)W_r\left(\widehat{\Phi}-\widehat{K}\right)W_r^{\mathsf{T}}\left(\widehat{H}\otimes\widehat{H}\right)-W_r\left(\Phi_0-K_0\right)W_r^{\mathsf{T}}+W_r\left(\Phi_0-K_0\right)W_r^{\mathsf{T}}\right)\\
&\times\left(\left(\widehat{H}^{-1}\otimes\widehat{H}^{-1}\right)W_r\widehat{\sigma}^{2}_{t-1|t-2}(\widehat{\theta})-W_r\sigma^{2}_{t-1|t-2}\left(r_{\widehat{H}}^{-1}(\widehat{\theta})\right)\right)\\
&+\left(\left(\widehat{H}^{-1}\otimes\widehat{H}^{-1}\right)W_r\widehat{K}W_r^{\mathsf{T}}\left(\widehat{H}\otimes\widehat{H}\right)-W_rK_0W_r^{\mathsf{T}}+W_rK_0W_r^{\mathsf{T}}\right)\\
&\times\left(\widehat{H}^{-1}\otimes\widehat{H}^{-1}\right)\left(\widehat{F}_{t-1}\otimes\widehat{F}_{t-1}-\widehat{H}F_{t-1}\otimes\widehat{H}F_{t-1}\right)\\
&=W_r\left(\Phi_0-K_0\right)W_r^{\mathsf{T}}\left(\left(\widehat{H}^{-1}\otimes\widehat{H}^{-1}\right)W_r\widehat{\sigma}^{2}_{t-1|t-2}(\widehat{\theta})-W_r\sigma^{2}_{t-1|t-2}\left(r_{\widehat{H}}^{-1}(\widehat{\theta})\right)\right)\\
&\quad+W_rK_0W_r^{\mathsf{T}}\left(\widehat{H}^{-1}\otimes\widehat{H}^{-1}\right)\left(\widehat{F}_{t-1}\otimes\widehat{F}_{t-1}-\widehat{H}F_{t-1}\otimes\widehat{H}F_{t-1}\right)+O_{\mathbb{P}}\left(\frac{1}{N}+\frac{1}{T}+\frac{1}{\sqrt{NT}}\right)\\
&=W_r\left(\Phi_0-K_0\right)W_r^{\mathsf{T}}\left(\left(\widehat{H}^{-1}\otimes\widehat{H}^{-1}\right)W_r\widehat{\sigma}^{2}_{t-1|t-2}(\widehat{\theta})-W_r\sigma^{2}_{t-1|t-2}\left(r_{\widehat{H}}^{-1}(\widehat{\theta})\right)\right)\\
&\quad+W_rK_0W_r^{\mathsf{T}}\left(\widehat{H}^{-1}\otimes\widehat{H}^{-1}\right)\left(I_r\otimes\widehat{H}F_{t-1}+\widehat{H}F_{t-1}\otimes I_r\right)\left(\widehat{F}_{t-1}-\widehat{H}F_{t-1}\right)\\
&\quad+O_{\mathbb{P}}\left(\frac{1}{N}+\frac{1}{T}+\frac{1}{\sqrt{NT}}\right)\\
&=W_r\left(\Phi_0-K_0\right)W_r^{\mathsf{T}}\left(\left(\widehat{H}^{-1}\otimes\widehat{H}^{-1}\right)W_r\widehat{\sigma}^{2}_{t-1|t-2}(\widehat{\theta})-W_r\sigma^{2}_{t-1|t-2}\left(r_{\widehat{H}}^{-1}(\widehat{\theta})\right)\right)\\
&\quad+W_rK_0W_r^{\mathsf{T}}\left(I_r\otimes F_{t-1}+F_{t-1}\otimes I_r\right)\left(\frac{\Lambda_0^{\mathsf{T}}\Lambda_0}{N}\right)^{-1}\frac{\Lambda_0^{\mathsf{T}}e_{t-1}}{N}+O_{\mathbb{P}}\left(\frac{1}{N}+\frac{1}{T}+\frac{1}{\sqrt{NT}}\right)\\
&=\sum_{k=0}^{t-1}\left(W_r\left(\Phi_0-K_0\right)W_r^{\mathsf{T}}\right)^kW_rK_0W_r^{\mathsf{T}}\left(I_r\otimes F_{t-k-1}+F_{t-k-1}\otimes I_r\right)\left(\frac{\Lambda_0^{\mathsf{T}}\Lambda_0}{N}\right)^{-1}\frac{\Lambda_0^{\mathsf{T}}e_{t-k-1}}{N}\\
&\quad+O_{\mathbb{P}}\left(\frac{1}{N}+\frac{1}{T}+\frac{1}{\sqrt{NT}}\right),
\end{aligned} × ( ( H − 1 ⊗ H − 1 ) W r σ t − 1∣ t − 2 2 ( θ ) − W r σ t − 1∣ t − 2 2 ( r H − 1 ( θ ) ) ) + ( H − 1 ⊗ H − 1 ) W r K W r T ( H ⊗ H ) ( ( H − 1 ⊗ H − 1 ) ( F t − 1 ⊗ F t − 1 ) − ( F t − 1 ⊗ F t − 1 ) ) = ( ( H − 1 ⊗ H − 1 ) W r ( Φ − K ) W r T ( H ⊗ H ) − W r ( Φ 0 − K 0 ) W r T + W r ( Φ 0 − K 0 ) W r T ) × ( ( H − 1 ⊗ H − 1 ) W r σ t − 1∣ t − 2 2 ( θ ) − W r σ t − 1∣ t − 2 2 ( r H − 1 ( θ ) ) ) + ( ( H − 1 ⊗ H − 1 ) W r K W r T ( H ⊗ H ) − W r K 0 W r T + W r K 0 W r T ) × ( H − 1 ⊗ H − 1 ) ( F t − 1 ⊗ F t − 1 − H F t − 1 ⊗ H F t − 1 ) = W r ( Φ 0 − K 0 ) W r T ( ( H − 1 ⊗ H − 1 ) W r σ t − 1∣ t − 2 2 ( θ ) − W r σ t − 1∣ t − 2 2 ( r H − 1 ( θ ) ) ) + W r K 0 W r T ( H − 1 ⊗ H − 1 ) ( F t − 1 ⊗ F t − 1 − H F t − 1 ⊗ H F t − 1 ) + O P ( N 1 + T 1 + N T 1 ) = W r ( Φ 0 − K 0 ) W r T ( ( H − 1 ⊗ H − 1 ) W r σ t − 1∣ t − 2 2 ( θ ) − W r σ t − 1∣ t − 2 2 ( r H − 1 ( θ ) ) ) + W r K 0 W r T ( H − 1 ⊗ H − 1 ) ( I r ⊗ H F t − 1 + H F t − 1 ⊗ I r ) ( F t − 1 − H F t − 1 ) + O P ( N 1 + T 1 + N T 1 ) = W r ( Φ 0 − K 0 ) W r T ( ( H − 1 ⊗ H − 1 ) W r σ t − 1∣ t − 2 2 ( θ ) − W r σ t − 1∣ t − 2 2 ( r H − 1 ( θ ) ) ) + W r K 0 W r T ( I r ⊗ F t − 1 + F t − 1 ⊗ I r ) ( N Λ 0 T Λ 0 ) − 1 N Λ 0 T e t − 1 + O P ( N 1 + T 1 + N T 1 ) = k = 0 ∑ t − 1 ( W r ( Φ 0 − K 0 ) W r T ) k W r K 0 W r T ( I r ⊗ F t − k − 1 + F t − k − 1 ⊗ I r ) ( N Λ 0 T Λ 0 ) − 1 N Λ 0 T e t − k − 1 + O P ( N 1 + T 1 + N T 1 ) ,
by Theorem 1 and Proposition 4, where we have used the fact that
( H ^ − 1 ⊗ H ^ − 1 ) ( I r ⊗ H ^ F t − 1 ) ( F ^ t − 1 − H ^ F t − 1 ) = H ^ − 1 ( F ^ t − 1 − H ^ F t − 1 ) ⊗ F t − 1 = ( I r ⊗ F t − 1 ) H ^ − 1 ( F ^ t − 1 − H ^ F t − 1 ) . \begin{aligned}
\left(\widehat{H}^{-1}\otimes\widehat{H}^{-1}\right)\left(I_r\otimes\widehat{H}F_{t-1}\right)\left(\widehat{F}_{t-1}-\widehat{H}F_{t-1}\right)=\widehat{H}^{-1}\left(\widehat{F}_{t-1}-\widehat{H}F_{t-1}\right)\otimes F_{t-1} \\
&= (I_r \otimes F_{t-1}) \widehat{H}^{-1}\left(\widehat{F}_{t-1}-\widehat{H}F_{t-1}\right).
\end{aligned} ( H − 1 ⊗ H − 1 ) ( I r ⊗ H F t − 1 ) ( F t − 1 − H F t − 1 ) = H − 1 ( F t − 1 − H F t − 1 ) ⊗ F t − 1 = ( I r ⊗ F t − 1 ) H − 1 ( F t − 1 − H F t − 1 ) .
Therefore, as N , T → ∞ N,T \to\infty N , T → ∞ ,
min ( N , T ) ( λ 0 i ⊤ ⊗ λ 0 i ⊤ ) ( ( H ^ − 1 ⊗ H ^ − 1 ) W r σ ~ t ∣ t − 1 2 ( θ ^ ) − W r σ t ∣ t − 1 2 ( r H ^ − 1 ( θ ^ ) ) ) = min ( N , T ) N ( λ 0 i ⊤ ⊗ λ 0 i ⊤ ) ∑ k = 0 t − 1 ( W r ( Φ 0 − K 0 ) W r ⊤ ) k W r K 0 W r ⊤ × ( I r ⊗ F t − k − 1 + F t − k − 1 ⊗ I r ) ( Λ 0 ⊤ Λ 0 N ) − 1 Λ 0 ⊤ e t − k − 1 N + O P ( min ( N , T ) N + min ( N , T ) T + min ( N , T ) N T ) . (191) \begin{aligned}
&\min\left(\sqrt{N},\sqrt{T}\right)\left(\lambda_{0i}^{\top}\otimes\lambda_{0i}^{\top}\right)\left(\left(\widehat{H}^{-1}\otimes\widehat{H}^{-1}\right)W_r\widetilde{\sigma}^{2}_{t|t-1}(\widehat{\theta})-W_r\sigma^{2}_{t|t-1}\left(r_{\widehat{H}}^{-1}(\widehat{\theta})\right)\right)\\
&=\frac{\min\left(\sqrt{N},\sqrt{T}\right)}{\sqrt{N}}\left(\lambda_{0i}^{\top}\otimes\lambda_{0i}^{\top}\right)\sum_{k=0}^{t-1}\left(W_r\left(\Phi_0-K_0\right)W_r^{\top}\right)^kW_rK_0W_r^{\top}\\
&\quad\times\left(I_r\otimes F_{t-k-1}+F_{t-k-1}\otimes I_r\right)\left(\frac{\Lambda_0^{\top}\Lambda_0}{N}\right)^{-1}\frac{\Lambda_0^{\top}e_{t-k-1}}{\sqrt{N}}\\
&\quad+O_{\mathbb{P}}\left(\frac{\min\left(\sqrt{N},\sqrt{T}\right)}{N}+\frac{\min\left(\sqrt{N},\sqrt{T}\right)}{T}+\frac{\min\left(\sqrt{N},\sqrt{T}\right)}{\sqrt{NT}}\right). \tag*{(191)}
\end{aligned} min ( N , T ) ( λ 0 i ⊤ ⊗ λ 0 i ⊤ ) ( ( H − 1 ⊗ H − 1 ) W r σ t ∣ t − 1 2 ( θ ) − W r σ t ∣ t − 1 2 ( r H − 1 ( θ ) ) ) = N min ( N , T ) ( λ 0 i ⊤ ⊗ λ 0 i ⊤ ) k = 0 ∑ t − 1 ( W r ( Φ 0 − K 0 ) W r ⊤ ) k W r K 0 W r ⊤ × ( I r ⊗ F t − k − 1 + F t − k − 1 ⊗ I r ) ( N Λ 0 ⊤ Λ 0 ) − 1 N Λ 0 ⊤ e t − k − 1 + O P N min ( N , T ) + T min ( N , T ) + N T min ( N , T ) . (191)
Then, by the finite-dimensional result in Proposition 1(b) and the Cramér–Wold device,
\begin{aligned}
&\left(\lambda_{0i}^{\top}\otimes\lambda_{0i}^{\top}\right)\sum_{k=0}^{t-1}\left(W_r\left(\Phi_0-K_0\right)W_r^{\top}\right)^kW_rK_0W_r^{\top}\\
&\quad\times\left(I_r\otimes F_{t-k-1}+F_{t-k-1}\otimes I_r\right)\left(\frac{\Lambda_0^{\top}\Lambda_0}{N}\right)^{-1}\frac{\Lambda_0^{\top}e_{t-k-1}}{\sqrt{N}}\middle|\mathcal{F}_{t-1}\\
&\Rightarrow\mathcal{N}\left(0,\left(\lambda_{0i}^{\top}\otimes\lambda_{0i}^{\top}\right)\left(\sum_{k=0}^{t-1}\sum_{h=0}^{t-1}\left(W_r\left(\Phi_0-K_0\right)W_r^{\top}\right)^kW_rK_0W_r^{\top}\right.\right.\\
&\quad\left.\left.\times\left(I_r\otimes F_{t-k-1}+F_{t-k-1}\otimes I_r\right)\Sigma_{\Lambda}^{-1}\Pi_{t-k-1,t-h-1}\Sigma_{\Lambda}^{-1}\left(I_r\otimes F_{t-h-1}+F_{t-h-1}\otimes I_r\right)^{\top}\right.\right.\\
&\quad\left.\left.\times W_rK_0W_r^{\top}\left(W_r\left(\Phi_0-K_0\right)W_r^{\top}\right)^{h\top}\right)\left(\lambda_{0i}\otimes\lambda_{0i}\right)\right).
\end{aligned}
The leading term in (189) is the time-series loading term, the leading term in (190) is the QMLE-score term, and the leading term in (191) is the cross-sectional factor-estimation term. The same Cramér–Wold argument used for the conditional mean gives their joint Gaussian convergence. Their pairwise asymptotic covariances are zero by (178) and Assumptions 4–5. Thus, from (188) ,
min ( N , T ) ( P ^ i , t ∣ t − 1 ( θ ^ ) − P i , t ∣ t − 1 ( θ 0 ) ) = min ( N , T ) T ( 1 T ∑ s = 1 T F s e i s ) ⊤ Γ F − 1 ( I r ⊗ λ 0 i + λ 0 i ⊗ I r ) ⊤ W r σ t ∣ t − 1 2 ( θ 0 ) + min ( N , T ) T ( λ 0 i ⊗ λ 0 i ) ⊤ W r ∇ θ σ t ∣ t − 1 2 ( θ 0 ) I ( θ 0 ) − 1 ( 1 T ∑ s = 1 T ∇ θ ℓ F ( F s ; θ 0 ) + min ( N , T ) N ( Λ 0 i ⊤ ⊗ Λ 0 i ⊤ ) ∑ k = 0 t − 1 ( W r ( Φ 0 − K 0 ) W r ⊤ ) k W r K 0 W r ⊤ × ( I r ⊗ F t − k − 1 + F t − k − 1 ⊗ I r ) ( Λ 0 ⊤ Λ 0 N ) − 1 Λ 0 ⊤ e t − k − 1 N + O P ( 1 min ( N , T ) ) . \begin{aligned}
&\min(\sqrt{N},\sqrt{T})(\widehat{P}_{i,t|t-1}(\widehat{\theta})-P_{i,t|t-1}(\theta_0))\\
&=\frac{\min(\sqrt{N},\sqrt{T})}{\sqrt{T}}(\frac{1}{\sqrt{T}}\sum_{s=1}^{T}F_se_{is})^{\top}\Gamma_F^{-1}(I_r\otimes\lambda_{0i}+\lambda_{0i}\otimes I_r)^{\top}W_r\sigma^{2}_{t|t-1}(\theta_0)\\
&\quad+\frac{\min(\sqrt{N},\sqrt{T})}{\sqrt{T}}(\lambda_{0i}\otimes\lambda_{0i})^{\top}W_r\nabla^{\theta}\sigma^{2}_{t|t-1}(\theta_0)\mathcal{I}(\theta_0)^{-1}(\frac{1}{\sqrt{T}}\sum_{s=1}^{T}\nabla^{\theta}\ell_F(F_s;\theta_0) \\
&+\frac{\min\left(\sqrt{N},\sqrt{T}\right)}{\sqrt{N}}\left(\Lambda_{0i}^{\top}\otimes\Lambda_{0i}^{\top}\right)\sum_{k=0}^{t-1}\left(\mathbf{W}_{r}\left(\Phi_{0}-K_{0}\right)\mathbf{W}_{r}^{\top}\right)^{k}\mathbf{W}_{r}K_{0}\mathbf{W}_{r}^{\top}
\\
\times\left(I_{r}\otimes F_{t-k-1}+F_{t-k-1}\otimes I_{r}\right)\left(\frac{\Lambda_{0}^{\top}\Lambda_{0}}{N}\right)^{-1}\frac{\Lambda_{0}^{\top}e_{t-k-1}}{\sqrt{N}}
\\
+O_{\mathrm{P}}\left(\frac{1}{\min\left(\sqrt{N},\sqrt{T}\right)}\right).
\end{aligned} × ( I r ⊗ F t − k − 1 + F t − k − 1 ⊗ I r ) ( N Λ 0 ⊤ Λ 0 ) − 1 N Λ 0 ⊤ e t − k − 1 + O P min ( N , T ) 1 . min ( N , T ) ( P i , t ∣ t − 1 ( θ ) − P i , t ∣ t − 1 ( θ 0 )) = T min ( N , T ) ( T 1 s = 1 ∑ T F s e i s ) ⊤ Γ F − 1 ( I r ⊗ λ 0 i + λ 0 i ⊗ I r ) ⊤ W r σ t ∣ t − 1 2 ( θ 0 ) + T min ( N , T ) ( λ 0 i ⊗ λ 0 i ) ⊤ W r ∇ θ σ t ∣ t − 1 2 ( θ 0 ) I ( θ 0 ) − 1 ( T 1 s = 1 ∑ T ∇ θ ℓ F ( F s ; θ 0 ) + N min ( N , T ) ( Λ 0 i ⊤ ⊗ Λ 0 i ⊤ ) k = 0 ∑ t − 1 ( W r ( Φ 0 − K 0 ) W r ⊤ ) k W r K 0 W r ⊤
This decomposition is likewise the variance counterpart of equation (C.1) in the proof of Theorem 3 of Bai [2 ] , augmented by the QMLE-score term. Applying the same subsequence argument used above and dividing by the conditional standard deviation in the statement gives a standard Gaussian limit along every subsequence. The subsequence principle therefore proves the second result of the theorem as N , T → ∞ N,T\to\infty N , T → ∞ .
Proof of Proposition 5
The only additional term relative to the proof of Proposition 2 is generated by the dependence of the conditional variance recursion on the filtered conditional location process. We show that this term does not change any of the rates in Proposition 2.
(a) For every θ μ ∈ Θ μ \theta^{\mu}\in\Theta^{\mu} θ μ ∈ Θ μ , define
ϵ ^ t ( θ μ ) = F ^ t − μ ^ t ∣ t − 1 ( θ μ ) , \widehat{\epsilon}_{t}(\theta^{\mu})=\widehat{F}_{t}-\widehat{\mu}_{t|t-1}(\theta^{\mu}), ϵ t ( θ μ ) = F t − μ t ∣ t − 1 ( θ μ ) ,
e ^ t ( θ μ ) = H ^ F ^ t − μ ^ t ∣ t − 1 ( θ μ ) , \widehat{e}_{t}(\theta^{\mu})=\widehat{H}\widehat{F}_{t}-\widehat{\mu}_{t|t-1}(\theta^{\mu}), e t ( θ μ ) = H F t − μ t ∣ t − 1 ( θ μ ) ,
and note that
ϵ ^ t ( θ μ ) − e ^ t ( θ μ ) = F ^ t − H ^ F ^ t − ( μ ^ t ∣ t − 1 ( θ μ ) − μ ^ t ∣ t − 1 ( θ μ ) ) . (192) \widehat{\epsilon}_{t}(\theta^{\mu})-\widehat{e}_{t}(\theta^{\mu})=\widehat{F}_{t}-\widehat{H}\widehat{F}_{t}-\left(\widehat{\mu}_{t|t-1}(\theta^{\mu})-\widehat{\mu}_{t|t-1}(\theta^{\mu})\right). \tag*{(192)} ϵ t ( θ μ ) − e t ( θ μ ) = F t − H F t − ( μ t ∣ t − 1 ( θ μ ) − μ t ∣ t − 1 ( θ μ ) ) . (192)
where the corresponding conditional location filters satisfy
μ ^ t + 1 ∣ t ( θ μ ) = ( Φ μ − K μ ) μ ^ t ∣ t − 1 ( θ μ ) + K μ F ^ t , \widehat{\mu}_{t+1|t}(\theta^{\mu})=(\Phi^{\mu}-K^{\mu})\widehat{\mu}_{t|t-1}(\theta^{\mu})+K^{\mu}\widehat{F}_{t}, μ t + 1∣ t ( θ μ ) = ( Φ μ − K μ ) μ t ∣ t − 1 ( θ μ ) + K μ F t ,
μ ^ t + 1 ∣ t ( θ μ ) = ( Φ μ − K μ ) μ ^ t ∣ t − 1 ( θ μ ) + K μ H ^ F ^ t . \widehat{\mu}_{t+1|t}(\theta^{\mu})=(\Phi^{\mu}-K^{\mu})\widehat{\mu}_{t|t-1}(\theta^{\mu})+K^{\mu}\widehat{H}\widehat{F}_{t}. μ t + 1∣ t ( θ μ ) = ( Φ μ − K μ ) μ t ∣ t − 1 ( θ μ ) + K μ H F t .
Therefore, it is straightforward to obtain
μ ^ t + 1 ∣ t ( θ μ ) − μ ^ t + 1 ∣ t ( θ μ ) = ( Φ μ − K μ ) ( μ ^ t ∣ t − 1 ( θ μ ) − μ ^ t ∣ t − 1 ( θ μ ) ) + K μ ( F ^ t − H ^ F ^ t ) . (193) \begin{aligned}
\widehat{\mu}_{t+1|t}(\theta^{\mu})-\widehat{\mu}_{t+1|t}(\theta^{\mu})
&=(\Phi^{\mu}-K^{\mu})\left(\widehat{\mu}_{t|t-1}(\theta^{\mu})-\widehat{\mu}_{t|t-1}(\theta^{\mu})\right)\\
&\quad+K^{\mu}\left(\widehat{F}_{t}-\widehat{H}\widehat{F}_{t}\right). \tag*{(193)}
\end{aligned} μ t + 1∣ t ( θ μ ) − μ t + 1∣ t ( θ μ ) = ( Φ μ − K μ ) ( μ t ∣ t − 1 ( θ μ ) − μ t ∣ t − 1 ( θ μ ) ) + K μ ( F t − H F t ) . (193)
Iterating (193) and using Assumption 15 (ii) and Proposition 1(a) we obtain, for any t ≥ 1 t \ge1 t ≥ 1 , as N , T → ∞ N,T \to\infty N , T → ∞ ,
sup θ μ ∈ Θ μ ∥ μ ^ t + 1 ∣ t ( θ μ ) − μ ^ t + 1 ∣ t ( θ μ ) ∥ = O P ( 1 min ( N , T ) ) + O a . s . ( ( X 1 μ ) t ) . (194) \sup_{\theta^\mu\in\Theta^\mu} \left\lVert\widehat{\boldsymbol{\mu}}_{t+1|t}(\theta^\mu)-\widehat{\boldsymbol{\mu}}_{t+1|t}(\theta^\mu)\right\rVert= O_{\mathrm{P}}\left(\frac{1}{\min\left(\sqrt{N},T\right)}\right) + O_{\mathrm{a.s.}}\left(\left(\mathcal{X}_1^\mu\right)^t\right). \tag*{(194)} θ μ ∈ Θ μ sup μ t + 1∣ t ( θ μ ) − μ t + 1∣ t ( θ μ ) = O P min ( N , T ) 1 + O a.s. ( ( X 1 μ ) t ) . (194)
Now, for every θ σ ∈ Θ σ \theta^\sigma\in\Theta^\sigma θ σ ∈ Θ σ , we have that
σ ^ t + 1 ∣ t 2 ( θ μ , θ σ ) = δ σ + Φ σ σ ^ t ∣ t − 1 2 ( θ μ , θ σ ) + ( K σ + Υ σ I ( F ^ t ) ) W r T ( ϵ ^ t ( θ μ ) ⊗ ϵ ^ t ( θ μ ) − vec Σ ^ t ∣ t − 1 ( θ μ , θ σ ) ) (195) \widehat{\sigma}_{t+1|t}^{2}(\theta^\mu,\theta^\sigma)=\delta^\sigma+\Phi^\sigma\widehat{\sigma}_{t|t-1}^{2}(\theta^\mu,\theta^\sigma)
+\left(K^\sigma+\Upsilon^\sigma I\left(\widehat{F}_t\right)\right)W_r^{\mathsf{T}}\left(\widehat{\epsilon}_t(\theta^\mu)\otimes\widehat{\epsilon}_t(\theta^\mu)-\operatorname{vec}\widehat{\Sigma}_{t|t-1}(\theta^\mu,\theta^\sigma)\right) \tag*{(195)} σ t + 1∣ t 2 ( θ μ , θ σ ) = δ σ + Φ σ σ t ∣ t − 1 2 ( θ μ , θ σ ) + ( K σ + Υ σ I ( F t ) ) W r T ( ϵ t ( θ μ ) ⊗ ϵ t ( θ μ ) − vec Σ t ∣ t − 1 ( θ μ , θ σ ) ) (195)
σ ^ t + 1 ∣ t 2 ( θ μ , θ σ ) = δ σ + Φ σ σ ^ t ∣ t − 1 2 ( θ μ , θ σ ) + ( K σ + Υ σ I ( H ^ F t ) ) W r T ( ϵ ^ t ( θ μ ) ⊗ ϵ ^ t ( θ μ ) − vec Σ ^ t ∣ t − 1 ( θ μ , θ σ ) ) , (196) \widehat{\sigma}_{t+1|t}^{2}(\theta^\mu,\theta^\sigma)=\delta^\sigma+\Phi^\sigma\widehat{\sigma}_{t|t-1}^{2}(\theta^\mu,\theta^\sigma)
+\left(K^\sigma+\Upsilon^\sigma I\left(\widehat{H}F_t\right)\right)W_r^{\mathsf{T}}\left(\widehat{\epsilon}_t(\theta^\mu)\otimes\widehat{\epsilon}_t(\theta^\mu)-\operatorname{vec}\widehat{\Sigma}_{t|t-1}(\theta^\mu,\theta^\sigma)\right), \tag*{(196)} σ t + 1∣ t 2 ( θ μ , θ σ ) = δ σ + Φ σ σ t ∣ t − 1 2 ( θ μ , θ σ ) + ( K σ + Υ σ I ( H F t ) ) W r T ( ϵ t ( θ μ ) ⊗ ϵ t ( θ μ ) − vec Σ t ∣ t − 1 ( θ μ , θ σ ) ) , (196)
and we let
A ^ t σ = Φ σ − K σ − Υ σ I ( F ^ t ) , B ^ t σ = K σ + Υ σ I ( F ^ t ) . \widehat{A}_t^\sigma=\Phi^\sigma-K^\sigma-\Upsilon^\sigma I\left(\widehat{F}_t\right),\qquad
\widehat{B}_t^\sigma=K^\sigma+\Upsilon^\sigma I\left(\widehat{F}_t\right). A t σ = Φ σ − K σ − Υ σ I ( F t ) , B t σ = K σ + Υ σ I ( F t ) .
Then, subtracting the variance recursion in (196) from the feasible recursion in (195) driven by { F ^ t , ϵ ^ t ( θ μ ) } \{\widehat{F}_t,\widehat{\epsilon}_t(\theta^\mu)\} { F t , ϵ t ( θ μ )} yields
σ ^ t + 1 ∣ t 2 ( θ μ , θ σ ) − σ ^ t + 1 ∣ t 2 ( θ μ , θ σ ) = A ^ t σ ( σ ^ t ∣ t − 1 2 ( θ μ , θ σ ) − σ ^ t ∣ t − 1 2 ( θ μ , θ σ ) ) + B ^ t σ W r T ( ϵ ^ t ( θ μ ) ⊗ ϵ ^ t ( θ μ ) − ϵ ^ t ( θ μ ) ⊗ ϵ ^ t ( θ μ ) ) + Υ σ ( I ( F ^ t ) − I ( H ^ F t ) ) ( W r T ( ϵ ^ t ( θ μ ) ⊗ ϵ ^ t ( θ μ ) ) − σ ^ t ∣ t − 1 2 ( θ μ , θ σ ) ) . (197) \begin{aligned}
\widehat{\sigma}_{t+1|t}^{2}(\theta^\mu,\theta^\sigma)-\widehat{\sigma}_{t+1|t}^{2}(\theta^\mu,\theta^\sigma)
={}&\widehat{A}_t^\sigma\left(\widehat{\sigma}_{t|t-1}^{2}(\theta^\mu,\theta^\sigma)-\widehat{\sigma}_{t|t-1}^{2}(\theta^\mu,\theta^\sigma)\right)\\
&+\widehat{B}_t^\sigma W_r^{\mathsf{T}}\left(\widehat{\epsilon}_t(\theta^\mu)\otimes\widehat{\epsilon}_t(\theta^\mu)-\widehat{\epsilon}_t(\theta^\mu)\otimes\widehat{\epsilon}_t(\theta^\mu)\right)\\
&+\Upsilon^\sigma\left(I\left(\widehat{F}_t\right)-I\left(\widehat{H}F_t\right)\right)\left(W_r^{\mathsf{T}}\left(\widehat{\epsilon}_t(\theta^\mu)\otimes\widehat{\epsilon}_t(\theta^\mu)\right)-\widehat{\sigma}_{t|t-1}^{2}(\theta^\mu,\theta^\sigma)\right). \tag*{(197)}
\end{aligned} σ t + 1∣ t 2 ( θ μ , θ σ ) − σ t + 1∣ t 2 ( θ μ , θ σ ) = A t σ ( σ t ∣ t − 1 2 ( θ μ , θ σ ) − σ t ∣ t − 1 2 ( θ μ , θ σ ) ) + B t σ W r T ( ϵ t ( θ μ ) ⊗ ϵ t ( θ μ ) − ϵ t ( θ μ ) ⊗ ϵ t ( θ μ ) ) + Υ σ ( I ( F t ) − I ( H F t ) ) ( W r T ( ϵ t ( θ μ ) ⊗ ϵ t ( θ μ ) ) − σ t ∣ t − 1 2 ( θ μ , θ σ ) ) . (197)
The difference between the two quadratic residual terms in (197) satisfies
ϵ ^ t ( θ μ ) ⊗ ϵ ^ t ( θ μ ) − ϵ ^ t ( θ μ ) ⊗ ϵ ^ t ( θ μ ) = 1 2 ( I r ⊗ ( ϵ ^ t ( θ μ ) + ϵ ^ t ( θ μ ) ) + ( ϵ ^ t ( θ μ ) + ϵ ^ t ( θ μ ) ) ⊗ I r ) ( ϵ ^ t ( θ μ ) − ϵ ^ t ( θ μ ) ) . (198) \widehat{\epsilon}_t(\theta^\mu)\otimes\widehat{\epsilon}_t(\theta^\mu)-\widehat{\epsilon}_t(\theta^\mu)\otimes\widehat{\epsilon}_t(\theta^\mu)
=\frac{1}{2}\left(I_r\otimes\left(\widehat{\epsilon}_t(\theta^\mu)+\widehat{\epsilon}_t(\theta^\mu)\right)+\left(\widehat{\epsilon}_t(\theta^\mu)+\widehat{\epsilon}_t(\theta^\mu)\right)\otimes I_r\right)\left(\widehat{\epsilon}_t(\theta^\mu)-\widehat{\epsilon}_t(\theta^\mu)\right). \tag*{(198)} ϵ t ( θ μ ) ⊗ ϵ t ( θ μ ) − ϵ t ( θ μ ) ⊗ ϵ t ( θ μ ) = 2 1 ( I r ⊗ ( ϵ t ( θ μ ) + ϵ t ( θ μ ) ) + ( ϵ t ( θ μ ) + ϵ t ( θ μ ) ) ⊗ I r ) ( ϵ t ( θ μ ) − ϵ t ( θ μ ) ) . (198)
Substituting (192) and (198) into (197) and iterating over k k k periods yields
\begin{aligned}
\widehat{\sigma}_{t+1|t}^{2}(\theta^\mu,\theta^\sigma)-\widehat{\sigma}_{t+1|t}^{2}(\theta^\mu,\theta^\sigma)
={}&\left(\prod_{h=0}^{k-1}\widehat{A}_{t-h}^\sigma\right)\left(\widehat{\sigma}_{t-k+1|t-k}^{2}(\theta^\mu,\theta^\sigma)-\widehat{\sigma}_{t-k+1|t-k}^{2}(\theta^\mu,\theta^\sigma)\right) \\
&+ \frac{1}{2}\sum_{j=1}^{k}\left(\prod_{h=0}^{j-2}\widehat{A}_{t-h}^{\sigma}\right)\widehat{B}_{t-j+1}^{\sigma}W_{r}^{\mathsf{T}} \\
&\times\left(I_r\otimes\left(\widehat{\epsilon}_{t-j+1}(\theta^\mu)+\widehat{\epsilon}_{t-j+1}(\theta^\mu)\right)+\left(\widehat{\epsilon}_{t-j+1}(\theta^\mu)+\widehat{\epsilon}_{t-j+1}(\theta^\mu)\right)\otimes I_r\right) \\
&\times\left(\widehat{F}_{t-j+1}-\widehat{H}F_{t-j+1}-\left(\widehat{\mu}_{t-j+1|t-j}(\theta^\mu)-\widehat{\mu}_{t-j+1|t-j}(\theta^\mu)\right)\right) \\
&+\sum_{j=1}^{k}\left(\prod_{h=0}^{j-2}\widehat{A}_{t-h}^{\sigma}\right)Y^\sigma\left(I\left(\widehat{F}_{t-j+1}\right)-I\left(\widehat{H}F_{t-j+1}\right)\right) \\
&\times\left(W_r^{\mathsf{T}}\left(\widehat{\epsilon}_{t-j+1}(\theta^\mu)\otimes\widehat{\epsilon}_{t-j+1}(\theta^\mu)\right)-\widehat{\sigma}_{t-j+1|t-j}^{2}(\theta^\mu,\theta^\sigma)\right),
\qquad{#eq:199} \tag*{(199)}
\end{aligned}
where the product equals I r I_r I r when j = 1 j=1 j = 1 .
Note that Assumption 15 (iv) implies that
\sup_{\theta^\sigma\in\Theta^\sigma}\left\|\prod_{h=0}^{k-1}\widehat{A}_{t-h}^{\sigma}\right\|\leq\chi_k^\sigma<1
\qquad{#eq:200}
almost surely, whereas Assumption 15 (v), together with (194) and (192) , implies that the coefficient multiplying the residual difference in (199) is O P ( 1 ) O_{\mathbb{P}}(1) O P ( 1 ) uniformly over Θ μ × Θ σ \Theta^\mu\times\Theta^\sigma Θ μ × Θ σ .
To control the indicator term in (199) , fix t ≥ 1 t\geq1 t ≥ 1 and let η > 0 \eta>0 η > 0 be sufficiently small. Componentwise sign changes can occur only if at least one component of H 0 F t H_0F_t H 0 F t is within η \eta η of zero, or if either of the two approximating vectors is more than η \eta η away from H 0 F t H_0F_t H 0 F t . Therefore,
{ I ( F ^ t ) ≠ I ( H ^ F t ) } ⊆ ⋃ i = 1 r { ∣ e i T H 0 F t ∣ ≤ η } ∪ { ∥ F ^ t − H 0 F t ∥ > η } ∪ { ∥ H ^ F t − H 0 F t ∥ > η } . \left\{I\left(\widehat{F}_t\right)\ne I\left(\widehat{H}F_t\right)\right\}
\subseteq\bigcup_{i=1}^{r}\left\{\left|e_i^{\mathsf{T}}H_0F_t\right|\leq\eta\right\}\cup\left\{\left\|\widehat{F}_t-H_0F_t\right\|>\eta\right\}\cup\left\{\left\|\widehat{H}F_t-H_0F_t\right\|>\eta\right\}. { I ( F t ) = I ( H F t ) } ⊆ i = 1 ⋃ r { e i T H 0 F t ≤ η } ∪ { F t − H 0 F t > η } ∪ { H F t − H 0 F t > η } .
Indeed, on the complement of the event on the right-hand side, F ^ t \widehat{F}_t F t and H ^ F t \widehat{H}F_t H F t are both within η \eta η of H 0 F t H_0F_t H 0 F t , while every component of H 0 F t H_0F_t H 0 F t has absolute value larger than η \eta η . Hence, both vectors have the same componentwise signs as H 0 F t H_0F_t H 0 F t .
Taking probabilities and applying the union bound
\begin{aligned}
\mathbb{P}\left(I\left(\widehat{F}_t\right)\ne I\left(\widehat{H}F_t\right)\right)
\leq\sum_{i=1}^{r}\mathbb{P}\left(\left|e_i^{\mathsf{T}}H_0F_t\right|\leq\eta\right)+\mathbb{P}\left(\left\|\widehat{F}_t-H_0F_t\right\|>\eta\right)+\mathbb{P}\left(\left\|\left(\widehat{H}-H_0\right)F_t\right\|>\eta\right) \\
&\leq rC_I\eta+\mathbb{P}\left(\left\|\widehat{F}_t-H_0F_t\right\|>\eta\right)+\mathbb{P}\left(\left\|\left(\widehat{H}-H_0\right)F_t\right\|>\eta\right),
\qquad{#eq:201}
\end{aligned}
where the last inequality follows from Assumption 15(vi) evaluated at H = H 0 H = H_0 H = H 0 .
Recall that, for every fixed η > 0 \eta> 0 η > 0 , Proposition 1(e) implies that P ( ∥ F ^ t − H 0 F t ∥ > η ) → 0 \mathbb{P}\left(\left\|\widehat{F}_t - H_0F_t\right\| > \eta\right) \to0 P ( F t − H 0 F t > η ) → 0 as N , T → ∞ N,T \to\infty N , T → ∞ .
Moreover,
∥ ( H ^ − H 0 ) F t ∥ ≤ ∥ H ^ − H 0 ∥ ∥ F t ∥ = O P ( 1 ) , \left\|\left(\widehat{H}-H_0\right)F_t\right\| \leq\left\|\widehat{H}-H_0\right\|\left\|F_t\right\| = O_{\mathbb{P}}(1), ( H − H 0 ) F t ≤ H − H 0 ∥ F t ∥ = O P ( 1 ) ,
because H ^ → P H 0 \widehat{H} \overset{\mathbb{P}}{\to} H_0 H → P H 0 and F t = O P ( 1 ) F_t = O_{\mathbb{P}}(1) F t = O P ( 1 ) by Assumption 2. Consequently, (201) yields
lim N , T → ∞ P ( I ( F ^ t ) ≠ I ( H F t ) ) ≤ r C I η . \lim_{N,T\to\infty}\mathbb{P}\left(I\left(\widehat{F}_t\right)\ne I\left(HF_t\right)\right)\leq rC_I\eta. N , T → ∞ lim P ( I ( F t ) = I ( H F t ) ) ≤ r C I η .
Letting η ↓ 0 \eta\downarrow0 η ↓ 0 , we obtain P ( I ( F ^ t ) ≠ I ( H ^ F t ) ) → 0 \mathbb{P}\left(I\left(\widehat{F}_t\right)\ne I\left(\widehat{H}F_t\right)\right)\to0 P ( I ( F t ) = I ( H F t ) ) → 0 as N , T → ∞ N,T\to\infty N , T → ∞ , and hence I ( F ^ t ) − I ( H ^ F t ) = o P ( 1 ) I\left(\widehat{F}_t\right)-I\left(\widehat{H}F_t\right)=o_{\mathbb{P}}(1) I ( F t ) − I ( H F t ) = o P ( 1 ) .
Since k k k is fixed and Θ σ \boldsymbol{\Theta}^{\sigma} Θ σ is compact, the coefficient of the indicator difference satisfies
max 1 ≤ j ≤ k sup θ μ ∈ Θ μ θ σ ∈ Θ σ ∥ ( ∏ h = 0 j − 2 A ^ t − h σ ) Υ σ ∥ × ∥ W r T ( ϵ ^ t − j + 1 ( θ μ ) ⊗ ϵ ^ t − j + 1 ( θ μ ) ) − σ ^ t − j + 1 ∣ t − j 2 ( θ μ , θ σ ) ∥ ≤ C max 1 ≤ j ≤ k sup θ μ ∈ Θ μ θ σ ∈ Θ σ ( 1 + ∥ ϵ ^ t − j + 1 ( θ μ ) ∥ 2 + ∥ σ ^ t − j + 1 ∣ t − j 2 ( θ μ , θ σ ) ∥ ) = O P ( 1 ) , (200) \begin{aligned}
&\max_{1\leq j\leq k}\sup_{\theta^{\mu}\in\boldsymbol{\Theta}^{\mu}\atop\theta^{\sigma}\in\boldsymbol{\Theta}^{\sigma}}
\left\|\left(\prod_{h=0}^{j-2}\widehat{A}_{t-h}^{\sigma}\right)\boldsymbol{\Upsilon}^{\sigma}\right\| \\
&\quad\times\left\|\mathbf{W}_{r}^{\mathsf{T}}\left(\widehat{\epsilon}_{t-j+1}(\theta^{\mu})\otimes\widehat{\epsilon}_{t-j+1}(\theta^{\mu})\right)-\widehat{\sigma}_{t-j+1|t-j}^2(\theta^{\mu},\theta^{\sigma})\right\| \\
&\leq C\max_{1\leq j\leq k}\sup_{\theta^{\mu}\in\boldsymbol{\Theta}^{\mu}\atop\theta^{\sigma}\in\boldsymbol{\Theta}^{\sigma}}\left(1+\left\|\widehat{\epsilon}_{t-j+1}(\theta^{\mu})\right\|^2+\left\|\widehat{\sigma}_{t-j+1|t-j}^2(\theta^{\mu},\theta^{\sigma})\right\|\right)=O_{\mathbb{P}}(1), \tag*{(200)}
\end{aligned} 1 ≤ j ≤ k max θ σ ∈ Θ σ θ μ ∈ Θ μ sup ( h = 0 ∏ j − 2 A t − h σ ) Υ σ × W r T ( ϵ t − j + 1 ( θ μ ) ⊗ ϵ t − j + 1 ( θ μ ) ) − σ t − j + 1∣ t − j 2 ( θ μ , θ σ ) ≤ C 1 ≤ j ≤ k max θ σ ∈ Θ σ θ μ ∈ Θ μ sup ( 1 + ∥ ϵ t − j + 1 ( θ μ ) ∥ 2 + σ t − j + 1∣ t − j 2 ( θ μ , θ σ ) ) = O P ( 1 ) , (200)
where the last equality follows from Assumption 15(ii)–(v). Therefore the last sum in (199) is
∑ j = 1 k ( ∏ h = 0 j − 2 A ^ t − h σ ) Υ σ ( I ( F ^ t − j + 1 ) − I ( H ^ F t − j + 1 ) ) × ( W r T ( ϵ ^ t − j + 1 ( θ μ ) ⊗ ϵ ^ t − j + 1 ( θ μ ) ) − σ ^ t − j + 1 ∣ t − j 2 ( θ μ , θ σ ) ) = O P ( 1 min ( N , T ) ) . (201) \begin{aligned}
\sum_{j=1}^{k}\left(\prod_{h=0}^{j-2}\widehat{A}_{t-h}^{\sigma}\right)\boldsymbol{\Upsilon}^{\sigma}\left(I\left(\widehat{F}_{t-j+1}\right)-I\left(\widehat{H}F_{t-j+1}\right)\right) \\
\times\left(\mathbf{W}_{r}^{\mathsf{T}}\left(\widehat{\epsilon}_{t-j+1}(\theta^{\mu})\otimes\widehat{\epsilon}_{t-j+1}(\theta^{\mu})\right)-\widehat{\sigma}_{t-j+1|t-j}^{2}(\theta^{\mu},\theta^{\sigma})\right) \\
=O_{\mathbb{P}}\left(\frac{1}{\min\left(\sqrt{N},T\right)}\right). \tag*{(201)}
\end{aligned} j = 1 ∑ k ( h = 0 ∏ j − 2 A t − h σ ) Υ σ ( I ( F t − j + 1 ) − I ( H F t − j + 1 ) ) × ( W r T ( ϵ t − j + 1 ( θ μ ) ⊗ ϵ t − j + 1 ( θ μ ) ) − σ t − j + 1∣ t − j 2 ( θ μ , θ σ ) ) = O P min ( N , T ) 1 . (201)
Taking the supremum over Θ μ × Θ σ \boldsymbol{\Theta}^{\mu}\times\boldsymbol{\Theta}^{\sigma} Θ μ × Θ σ in (199), and using (194), (200), (202), and (203) we obtain
sup θ μ ∈ Θ μ θ σ ∈ Θ σ ∥ σ ^ t + 1 ∣ t 2 ( θ μ , θ σ ) − σ ^ t + 1 ∣ t 2 ( θ μ , θ σ ) ∥ ≤ χ k σ sup θ μ ∈ Θ μ θ σ ∈ Θ σ ∥ σ ^ t − k + 1 ∣ t − k 2 ( θ μ , θ σ ) − σ ^ t − k + 1 ∣ t − k 2 ( θ μ , θ σ ) ∥ + O P ( 1 min ( N , T ) ) + O a . s . ( ( χ 1 μ ) t ) . \begin{aligned}
\sup_{\theta^{\mu}\in\boldsymbol{\Theta}^{\mu}\atop\theta^{\sigma}\in\boldsymbol{\Theta}^{\sigma}}
\left\|\widehat{\sigma}_{t+1|t}^{2}(\theta^{\mu},\theta^{\sigma})-\widehat{\sigma}_{t+1|t}^{2}(\theta^{\mu},\theta^{\sigma})\right\| \\
\leq\chi_k^{\sigma}\sup_{\theta^{\mu}\in\boldsymbol{\Theta}^{\mu}\atop\theta^{\sigma}\in\boldsymbol{\Theta}^{\sigma}}
\left\|\widehat{\sigma}_{t-k+1|t-k}^{2}(\theta^{\mu},\theta^{\sigma})-\widehat{\sigma}_{t-k+1|t-k}^{2}(\theta^{\mu},\theta^{\sigma})\right\| \\
\quad+O_{\mathbb{P}}\left(\frac{1}{\min\left(\sqrt{N},T\right)}\right)+O_{a.s.}\left((\chi_1^{\mu})^t\right).
\end{aligned} θ σ ∈ Θ σ θ μ ∈ Θ μ sup σ t + 1∣ t 2 ( θ μ , θ σ ) − σ t + 1∣ t 2 ( θ μ , θ σ ) ≤ χ k σ θ σ ∈ Θ σ θ μ ∈ Θ μ sup σ t − k + 1∣ t − k 2 ( θ μ , θ σ ) − σ t − k + 1∣ t − k 2 ( θ μ , θ σ ) + O P min ( N , T ) 1 + O a . s . ( ( χ 1 μ ) t ) .
Choose ρ k \rho_k ρ k such that max { X k σ , ( X 1 μ ) k } < ρ k < 1 \max\left\{\mathcal{X}_k^\sigma,(\mathcal{X}_1^\mu)^k\right\}<\rho_k<1 max { X k σ , ( X 1 μ ) k } < ρ k < 1 , then, iterating the preceding inequality yields
sup θ μ ∈ Θ μ θ σ ∈ Θ σ ∥ σ ^ t + 1 ∣ t 2 ( θ μ , θ σ ) − σ ^ t + 1 ∣ t 2 ( θ μ , θ σ ) ∥ = O P ( 1 min ( N , T ) ) + O a . s . ( ρ k ⌊ t / k ⌋ ) , (202) \sup_{\theta^\mu\in\Theta^\mu\atop\theta^\sigma\in\Theta^\sigma}\left\|\widehat{\sigma}_{t+1|t}^{2}(\theta^\mu,\theta^\sigma)-\widehat{\sigma}_{t+1|t}^{2}(\theta^\mu,\theta^\sigma)\right\|=O_{\mathbb{P}}\left(\frac{1}{\min\left(\sqrt{N},T\right)}\right)+O_{a.s.}\left(\rho_k^{\lfloor t/k\rfloor}\right), \tag*{(202)} θ σ ∈ Θ σ θ μ ∈ Θ μ sup σ t + 1∣ t 2 ( θ μ , θ σ ) − σ t + 1∣ t 2 ( θ μ , θ σ ) = O P min ( N , T ) 1 + O a . s . ( ρ k ⌊ t / k ⌋ ) , (202)
thus, equations (194) and (202) prove part ( a ) (a) ( a ) .
For part ( b ) (b) ( b ) , the smooth terms are controlled exactly as in the proof of Proposition 2(b). For the indicator term, we first write
I ( H ^ F t ) − I ( H 0 F ‾ t ) = I ( H ^ F t ) − I ( H 0 F t ) + I ( H 0 F t ) − I ( H 0 F ‾ t ) , \begin{aligned}
I\left(\widehat{H}F_t\right)-I\left(H_0\overline{F}_t\right) \\
&=I\left(\widehat{H}F_t\right)-I\left(H_0F_t\right)+I\left(H_0F_t\right)-I\left(H_0\overline{F}_t\right),
\end{aligned} I ( H F t ) − I ( H 0 F t ) = I ( H F t ) − I ( H 0 F t ) + I ( H 0 F t ) − I ( H 0 F t ) ,
and hence, since by Proposition 1(a) and ( e ) (e) ( e ) , it holds that
∥ ( H ^ − H 0 ) F t ∥ ≤ ∥ H ^ F t − F ^ t ∥ + ∥ F ^ t − H 0 F t ∥ = O P ( 1 min ( N , T ) ) , (203) \left\|\left(\widehat{H}-H_0\right)F_t\right\|\leq\left\|\widehat{H}F_t-\widehat{F}_t\right\|+\left\|\widehat{F}_t-H_0F_t\right\|=O_{\mathbb{P}}\left(\frac{1}{\min\left(\sqrt{N},\sqrt{T}\right)}\right), \tag*{(203)} ( H − H 0 ) F t ≤ H F t − F t + F t − H 0 F t = O P min ( N , T ) 1 , (203)
we get, by using Assumption 15(vi) and the same argument used to obtain (201) ,
∥ I ( H ^ F t ) − I ( H 0 F t ) ∥ = O P ( 1 min ( N , T ) ) . (204) \left\|I\left(\widehat{H}F_t\right)-I\left(H_0F_t\right)\right\|=O_{\mathbb{P}}\left(\frac{1}{\min\left(\sqrt{N},\sqrt{T}\right)}\right). \tag*{(204)} I ( H F t ) − I ( H 0 F t ) = O P min ( N , T ) 1 . (204)
Moreover, Assumptions 1(iii) and 7 imply that
∥ H 0 ( F t − F ‾ t ) ∥ 2 ≤ C X k ⌊ t / k ⌋ \left\|H_0\left(F_t-\overline{F}_t\right)\right\|_2\leq C\mathcal{X}_k^{\lfloor t/k\rfloor} H 0 ( F t − F t ) 2 ≤ C X k ⌊ t / k ⌋
for some finite positive real C C C independent of t t t . For every sufficiently small η > 0 \eta>0 η > 0 , we have
{ I ( H 0 F t ) ≠ I ( H 0 F ‾ t ) } ⊆ ⋃ i = 1 r { ∣ e i T H 0 F ‾ t ∣ ≤ η } ∪ { ∥ H 0 ( F t − F ‾ t ) ∥ > η } , \left\{I\left(H_0F_t\right)\ne I\left(H_0\overline{F}_t\right)\right\}\subseteq\bigcup_{i=1}^{r}\left\{\left|e_i^\mathsf{T}H_0\overline{F}_t\right|\leq\eta\right\}\cup\left\{\left\|H_0\left(F_t-\overline{F}_t\right)\right\|>\eta\right\}, { I ( H 0 F t ) = I ( H 0 F t ) } ⊆ i = 1 ⋃ r { e i T H 0 F t ≤ η } ∪ { H 0 ( F t − F t ) > η } ,
because, outside the event on the right-hand side, every component of H 0 F ‾ t H_0\overline{F}_t H 0 F t is farther than η \eta η from zero and the perturbation H 0 ( F t − F ‾ t ) H_0(F_t-\overline{F}_t) H 0 ( F t − F t ) has norm at most η \eta η . Hence, H 0 F t H_0F_t H 0 F t and H 0 F ‾ t H_0\overline{F}_t H 0 F t have the same componentwise signs. Therefore, by the union bound, Assumption 15(vi), and Markov’s inequality,
\begin{aligned}
\mathbb{P}\left(I\left(H_0F_t\right)\ne I\left(H_0\overline{F}_t\right)\right\\
&\le\sum_{i=1}^{r} \mathbb{P}\left(\left|e_i^{\mathsf{T}}H_0\overline{F}_t\right|\le\eta\right)+\mathbb{P}\left(\left\|H_0\left(F_t-\overline{F}_t\right)\right\|>\eta\right) \\
&\le rC_I\eta+\frac{C\chi_k^{2\lceil t/k\rceil}}{\eta^2}.
\end{aligned}
By setting η = χ k 2 ⌈ t / k ⌉ / 3 \eta=\chi_k^{2\lceil t/k\rceil/3} η = χ k 2 ⌈ t / k ⌉ /3 we get
P ( I ( H 0 F t ) ≠ I ( H 0 F ‾ t ) ) ≤ C ′ χ k 2 ⌈ t / k ⌉ / 3 , \mathbb{P}\left(I\left(H_0F_t\right)\ne I\left(H_0\overline{F}_t\right)\right)\le C'\chi_k^{2\lceil t/k\rceil/3}, P ( I ( H 0 F t ) = I ( H 0 F t ) ) ≤ C ′ χ k 2 ⌈ t / k ⌉ /3 ,
for some finite positive real C ′ C' C ′ independent of t t t . The right-hand side is summable in t t t and hence, by the Borel–Cantelli lemma, I ( H 0 F t ) − I ( H 0 F ‾ t ) = 0 r × r I\left(H_0F_t\right)-I\left(H_0\overline{F}_t\right)=0_{r\times r} I ( H 0 F t ) − I ( H 0 F t ) = 0 r × r almost surely for all sufficiently large t t t . Consequently, the effect of this indicator difference on the variance recursion decreases at the geometric contraction rate. By combining this result with the bound for I ( H ^ F t ) − I ( H 0 F t ) I\left(\widehat{H}F_t\right)-I\left(H_0F_t\right) I ( H F t ) − I ( H 0 F t ) in (206), (205) and the other smooth terms, we obtain
sup θ μ ∈ Θ μ θ σ ∈ Θ σ ∥ ( μ ^ t ∣ t − 1 ( θ μ ) T , σ ^ t ∣ t − 1 2 ( θ μ , θ σ ) T ) T − ( μ ^ t ∣ t − 1 ( θ μ ) T , σ ^ t ∣ t − 1 2 ( θ μ , θ σ ) T ) T ∥ = O P ( 1 min ( N , T ) ) + O a . s . ( ϱ k ⌈ t / k ⌉ ) , \begin{aligned}
\sup_{\substack{\theta^\mu\in\Theta^\mu\\ \theta^\sigma\in\Theta^\sigma}}\left\|\left(\widehat{\mu}_{t|t-1}\left(\theta^\mu\right)^{\mathsf{T}},\widehat{\sigma}^{2}_{t|t-1}\left(\theta^\mu,\theta^\sigma\right)^{\mathsf{T}}\right)^{\mathsf{T}}-\left(\widehat{\mu}_{t|t-1}\left(\theta^\mu\right)^{\mathsf{T}},\widehat{\sigma}^{2}_{t|t-1}\left(\theta^\mu,\theta^\sigma\right)^{\mathsf{T}}\right)^{\mathsf{T}}\right\| \\
&=O_{\mathbb{P}}\left(\frac{1}{\min\left(\sqrt{N},\sqrt{T}\right)}\right)+O_{a.s.}\left(\varrho_k^{\lceil t/k\rceil}\right),
\end{aligned} θ μ ∈ Θ μ θ σ ∈ Θ σ sup ( μ t ∣ t − 1 ( θ μ ) T , σ t ∣ t − 1 2 ( θ μ , θ σ ) T ) T − ( μ t ∣ t − 1 ( θ μ ) T , σ t ∣ t − 1 2 ( θ μ , θ σ ) T ) T = O P min ( N , T ) 1 + O a . s . ( ϱ k ⌈ t / k ⌉ ) ,
which proves part ( b ) (b) ( b ) .
For part ( c ) (c) ( c ) , both filters are driven by H 0 F ‾ t H_0\overline{F}_t H 0 F t , and hence, the factor and indicator differences in (199) are identically zero. The contraction argument used above also implies
sup θ μ ∈ Θ μ θ σ ∈ Θ σ ∥ ( μ ^ t ∣ t − 1 ( θ μ ) T , σ ^ t ∣ t − 1 2 ( θ μ , θ σ ) T ) T − ( μ ^ t ∣ t − 1 ( θ μ ) T , σ ^ t ∣ t − 1 2 ( θ μ , θ σ ) T ) T ∥ = O a . s . ( ϱ k ⌈ t / k ⌉ ) , \begin{aligned}
\sup_{\substack{\theta^\mu\in\Theta^\mu\\ \theta^\sigma\in\Theta^\sigma}}\left\|\left(\widehat{\mu}_{t|t-1}\left(\theta^\mu\right)^{\mathsf{T}},\widehat{\sigma}^{2}_{t|t-1}\left(\theta^\mu,\theta^\sigma\right)^{\mathsf{T}}\right)^{\mathsf{T}}-\left(\widehat{\mu}_{t|t-1}\left(\theta^\mu\right)^{\mathsf{T}},\widehat{\sigma}^{2}_{t|t-1}\left(\theta^\mu,\theta^\sigma\right)^{\mathsf{T}}\right)^{\mathsf{T}}\right\| \\
&=O_{a.s.}\left(\varrho_k^{\lceil t/k\rceil}\right),
\end{aligned} θ μ ∈ Θ μ θ σ ∈ Θ σ sup ( μ t ∣ t − 1 ( θ μ ) T , σ t ∣ t − 1 2 ( θ μ , θ σ ) T ) T − ( μ t ∣ t − 1 ( θ μ ) T , σ t ∣ t − 1 2 ( θ μ , θ σ ) T ) T = O a . s . ( ϱ k ⌈ t / k ⌉ ) ,
which proves part ( c ) (c) ( c ) .
For part ( d ) (d) ( d ) , let F t ( τ ) F_t^{(\tau)} F t ( τ ) be the truncated factor process used in Lemma 6. Then
sup t ∈ N ∥ H 0 ( F ‾ t − F t ( τ ) ) ∥ 2 = O ( χ k ⌈ t / k ⌉ ) . (205) \sup_{t\in\mathbb{N}}\left\|H_0\left(\overline{F}_t-F_t^{(\tau)}\right)\right\|_2=O\left(\chi_k^{\lceil t/k\rceil}\right). \tag*{(205)} t ∈ N sup H 0 ( F t − F t ( τ ) ) 2 = O ( χ k ⌈ t / k ⌉ ) . (205)
For every i = 1 , … , r i=1,\ldots,r i = 1 , … , r and every sufficiently small η > 0 \eta>0 η > 0 , Assumption 15 (vi), (207), and Markov’s inequality we get
P ( I ( e i T H 0 F ‾ t ) ≠ I ( e i T H 0 F t ( τ ) ) ) ≤ C I η + C χ k 2 ⌈ τ / k ⌉ η 2 . \begin{aligned}
\mathbb{P}\left(I\left(e_i^{\mathsf{T}}H_0\overline{F}_t\right)\ne I\left(e_i^{\mathsf{T}}H_0F_t^{(\tau)}\right)\right) \\
&\le C_I\eta+\frac{C\chi_k^{2\lceil\tau/k\rceil}}{\eta^2}.
\end{aligned} P ( I ( e i T H 0 F t ) = I ( e i T H 0 F t ( τ ) ) ) ≤ C I η + η 2 C χ k 2 ⌈ τ / k ⌉ .
Setting η = χ k 2 ⌈ τ / k ⌉ / 3 \eta=\chi_k^{2\lceil\tau/k\rceil/3} η = χ k 2 ⌈ τ / k ⌉ /3 and summing over the fixed number of components yields
sup t ∈ N ∥ I ( H 0 F ‾ t ) − I ( H 0 F t ( τ ) ) ∥ 2 = O ( χ k 1 3 ⌈ τ / k ⌉ ) . (206) \sup_{t\in\mathbb{N}}\left\|I\left(H_0\overline{F}_t\right)-I\left(H_0F_t^{(\tau)}\right)\right\|_2=O\left(\chi_k^{\frac{1}{3}\lceil\tau/k\rceil}\right). \tag*{(206)} t ∈ N sup I ( H 0 F t ) − I ( H 0 F t ( τ ) ) 2 = O ( χ k 3 1 ⌈ τ / k ⌉ ) . (206)
The conditional location truncation error has the same geometric form by Assumption 15(ii) and (205) . Therefore, enlarging ϱ k \varrho_k ϱ k if necessary so that χ k 1 / 3 < ϱ k < 1 \chi_k^{1/3}<\varrho_k<1 χ k 1/3 < ϱ k < 1 , the k k k -step variance recursion and (200) imply that
sup t ∈ N ∥ ( μ ‾ t ∣ t − 1 ( θ μ ) T , σ ‾ t ∣ t − 1 2 ( θ μ , θ σ ) T ) T − ( μ t ∣ t − 1 ( τ ) ( θ μ ) T , σ t ∣ t − 1 2 ( τ ) ( θ μ , θ σ ) T ) T ∥ 2 = O ( ϱ k ⌈ τ / k ⌉ ) . (207) \sup_{t\in\mathbb{N}}\left\|\left(\overline{\boldsymbol{\mu}}_{t|t-1}(\theta^\mu)^\mathsf{T},\overline{\boldsymbol{\sigma}}^2_{t|t-1}(\theta^\mu,\theta^\sigma)^\mathsf{T}\right)^\mathsf{T}-\left(\boldsymbol{\mu}_{t|t-1}^{(\tau)}(\theta^\mu)^\mathsf{T},\boldsymbol{\sigma}_{t|t-1}^{2(\tau)}(\theta^\mu,\theta^\sigma)^\mathsf{T}\right)^\mathsf{T}\right\|_2=O\left(\varrho_k^{\lceil\tau/k\rceil}\right). \tag*{(207)} t ∈ N sup ( μ t ∣ t − 1 ( θ μ ) T , σ t ∣ t − 1 2 ( θ μ , θ σ ) T ) T − ( μ t ∣ t − 1 ( τ ) ( θ μ ) T , σ t ∣ t − 1 2 ( τ ) ( θ μ , θ σ ) T ) T 2 = O ( ϱ k ⌈ τ / k ⌉ ) . (207)
Since the truncated filter is measurable with respect to v t , … , v t − τ \boldsymbol{v}_t,\ldots,\boldsymbol{v}_{t-\tau} v t , … , v t − τ , from the projection property of conditional expectations and (207) we get
sup t ∈ N ∥ ( μ ‾ t ∣ t − 1 ( θ μ ) T , σ ‾ t ∣ t − 1 2 ( θ μ , θ σ ) T ) T − E [ ( μ ‾ t ∣ t − 1 ( θ μ ) T , σ ‾ t ∣ t − 1 2 ( θ μ , θ σ ) T ) T ∣ v t , … , v t − τ ] ∥ 2 ≤ sup t ∈ N ∥ ( μ ‾ t ∣ t − 1 ( θ μ ) T , σ ‾ t ∣ t − 1 2 ( θ μ , θ σ ) T ) T − ( μ t ∣ t − 1 ( τ ) ( θ μ ) T , σ t ∣ t − 1 2 ( τ ) ( θ μ , θ σ ) T ) T ∥ 2 = O ( ϱ k ⌈ τ / k ⌉ ) . \begin{aligned}
\sup_{t\in\mathbb{N}}\left\|\left(\overline{\boldsymbol{\mu}}_{t|t-1}(\theta^\mu)^\mathsf{T},\overline{\boldsymbol{\sigma}}^2_{t|t-1}(\theta^\mu,\theta^\sigma)^\mathsf{T}\right)^\mathsf{T}-\mathbb{E}\left[\left(\overline{\boldsymbol{\mu}}_{t|t-1}(\theta^\mu)^\mathsf{T},\overline{\boldsymbol{\sigma}}^2_{t|t-1}(\theta^\mu,\theta^\sigma)^\mathsf{T}\right)^\mathsf{T}\mid\boldsymbol{v}_t,\ldots,\boldsymbol{v}_{t-\tau}\right]\right\|_2 \\
&\le\sup_{t\in\mathbb{N}}\left\|\left(\overline{\boldsymbol{\mu}}_{t|t-1}(\theta^\mu)^\mathsf{T},\overline{\boldsymbol{\sigma}}^2_{t|t-1}(\theta^\mu,\theta^\sigma)^\mathsf{T}\right)^\mathsf{T}-\left(\boldsymbol{\mu}_{t|t-1}^{(\tau)}(\theta^\mu)^\mathsf{T},\boldsymbol{\sigma}_{t|t-1}^{2(\tau)}(\theta^\mu,\theta^\sigma)^\mathsf{T}\right)^\mathsf{T}\right\|_2=O\left(\varrho_k^{\lceil\tau/k\rceil}\right).
\end{aligned} t ∈ N sup ( μ t ∣ t − 1 ( θ μ ) T , σ t ∣ t − 1 2 ( θ μ , θ σ ) T ) T − E [ ( μ t ∣ t − 1 ( θ μ ) T , σ t ∣ t − 1 2 ( θ μ , θ σ ) T ) T ∣ v t , … , v t − τ ] 2 ≤ t ∈ N sup ( μ t ∣ t − 1 ( θ μ ) T , σ t ∣ t − 1 2 ( θ μ , θ σ ) T ) T − ( μ t ∣ t − 1 ( τ ) ( θ μ ) T , σ t ∣ t − 1 2 ( τ ) ( θ μ , θ σ ) T ) T 2 = O ( ϱ k ⌈ τ / k ⌉ ) .
Thus, the stationary joint filter is near-epoch dependent of size − 1 -1 − 1 on { v t } t ∈ Z \{\boldsymbol{v}_t\}_{t\in\mathbb{Z}} { v t } t ∈ Z , proving part ( d ) (d) ( d ) . Finally, part ( e ) (e) ( e ) follows from parts ( a ) (a) ( a ) –( c ) (c) ( c ) and the triangle inequality.
Proof of Proposition 6
Let
S ^ J , T ( θ μ , θ σ ) = ( ∑ t = 1 T ∇ θ μ ℓ ^ μ , F ( F ^ t ; θ μ ) ∑ t = 1 T ∇ θ σ ℓ ^ σ , ϵ ( ϵ ^ t ( θ μ ) ; θ σ ) ) \widehat{\boldsymbol{S}}_{J,T}(\theta^\mu,\theta^\sigma)=
\begin{pmatrix}
\sum_{t=1}^{T}\nabla^{\theta^\mu}\widehat{\ell}_{\mu,F}\left(\widehat{F}_t;\theta^\mu\right)\\
\sum_{t=1}^{T}\nabla^{\theta^\sigma}\widehat{\ell}_{\sigma,\epsilon}\left(\widehat{\epsilon}_t\left(\theta^\mu\right);\theta^\sigma\right)
\end{pmatrix} S J , T ( θ μ , θ σ ) = ( ∑ t = 1 T ∇ θ μ ℓ μ , F ( F t ; θ μ ) ∑ t = 1 T ∇ θ σ ℓ σ , ϵ ( ϵ t ( θ μ ) ; θ σ ) )
denote the stacked feasible score. Since the two estimators are consistent for interior points of their respective parameter spaces, their first-order conditions imply that, with probability approaching one,
S ^ J , T ( θ ^ μ , θ ^ σ ) = 0 (208) \widehat{\boldsymbol{S}}_{J,T}\left(\widehat{\theta}^\mu,\widehat{\theta}^\sigma\right)=0 \tag*{(208)} S J , T ( θ μ , θ σ ) = 0 (208)
as N , T → ∞ N,T\to\infty N , T → ∞ . For a generic θ J = ( θ μ T , θ σ T ) T \theta^{J}=\left(\theta^{\mu\mathsf{T}},\theta^{\sigma\mathsf{T}}\right)^{\mathsf{T}} θ J = ( θ μ T , θ σ T ) T , define the stacked stationary score contribution as
s t J ( θ J ) = ( ∇ θ μ ℓ μ , F ( F ‾ t ; θ μ ) ∇ θ σ ℓ σ , ϵ ( ϵ ‾ t ( θ μ ) ; θ σ ) ) . s_{t}^{J}\left(\theta^{J}\right)=\begin{pmatrix}
\nabla^{\theta^{\mu}}\ell_{\mu,F}\left(\overline{F}_{t};\theta^{\mu}\right)\\
\nabla^{\theta^{\sigma}}\ell_{\sigma,\epsilon}\left(\overline{\epsilon}_{t}\left(\theta^{\mu}\right);\theta^{\sigma}\right)
\end{pmatrix}. s t J ( θ J ) = ( ∇ θ μ ℓ μ , F ( F t ; θ μ ) ∇ θ σ ℓ σ , ϵ ( ϵ t ( θ μ ) ; θ σ ) ) .
Proposition 5(a)–(c) provides the filter approximation rates required in the proof of Lemma 9, while Lemma 7(a)–(c) provides the corresponding rates for the first-derivative filters. Moreover, the leverage indicator does not depend on θ μ \theta^{\mu} θ μ or θ σ \theta^{\sigma} θ σ , and its approximation is controlled by Assumption 15(vi) as in the proof of Proposition 5. Therefore, applying componentwise the score expansion and the weighted PC argument used in the proof of Lemma 9 we directly obtain
sup θ J ∈ Θ μ ⊕ Θ σ ∥ 1 T S ^ J , T ( r H ^ J ( θ J ) ) − 1 T R H ^ J T ∑ t = 1 T s t J ( θ J ) ∥ = O P ( 1 N + T N + 1 T ) = o P ( 1 ) (209) \sup_{\theta^{J}\in\Theta^{\mu}\oplus\Theta^{\sigma}}\left\Vert\frac{1}{\sqrt{T}}\widehat{S}_{J,T}\left(r_{\widehat{H}}^{J}\left(\theta^{J}\right)\right)-\frac{1}{\sqrt{T}}R_{\widehat{H}}^{J\mathsf{T}}\sum_{t=1}^{T}s_{t}^{J}\left(\theta^{J}\right)\right\Vert
=O_{\mathbb{P}}\left(\frac{1}{\sqrt{N}}+\frac{\sqrt{T}}{N}+\frac{1}{\sqrt{T}}\right)=o_{\mathbb{P}}(1) \tag*{(209)} θ J ∈ Θ μ ⊕ Θ σ sup T 1 S J , T ( r H J ( θ J ) ) − T 1 R H J T t = 1 ∑ T s t J ( θ J ) = O P ( N 1 + N T + T 1 ) = o P ( 1 ) (209)
as N , T → ∞ N,T\to\infty N , T → ∞ , where the last equality follows from T / N → 0 \sqrt{T}/N\to0 T / N → 0 . In particular, evaluating (209) at θ J = θ 0 J \theta^{J}=\theta_{0}^{J} θ J = θ 0 J yields
1 T S ^ J , T ( r H ^ J ( θ 0 J ) ) = R H ^ J T 1 T ∑ t = 1 T s t J ( θ 0 J ) + o P ( 1 ) (210) \frac{1}{\sqrt{T}}\widehat{S}_{J,T}\left(r_{\widehat{H}}^{J}\left(\theta_{0}^{J}\right)\right)=R_{\widehat{H}}^{J\mathsf{T}}\frac{1}{\sqrt{T}}\sum_{t=1}^{T}s_{t}^{J}\left(\theta_{0}^{J}\right)+o_{\mathbb{P}}(1) \tag*{(210)} T 1 S J , T ( r H J ( θ 0 J ) ) = R H J T T 1 t = 1 ∑ T s t J ( θ 0 J ) + o P ( 1 ) (210)
as N , T → ∞ N,T\to\infty N , T → ∞ .
An application of the mean-value expansion to (208) around r H ^ J ( θ 0 J ) r_{\widehat{H}}^{J}\left(\theta_{0}^{J}\right) r H J ( θ 0 J ) yields
0 = 1 T S ^ J , T ( r H ^ J ( θ 0 J ) ) − I ^ J , T T ( θ ^ J − r H ^ J ( θ 0 J ) ) , (211) \mathbf{0}=\frac{1}{\sqrt{T}}\widehat{S}_{J,T}\left(r_{\widehat{H}}^{J}\left(\theta_{0}^{J}\right)\right)-\widehat{\mathcal{I}}_{J,T}\sqrt{T}\left(\widehat{\theta}^{J}-r_{\widehat{H}}^{J}\left(\theta_{0}^{J}\right)\right), \tag*{(211)} 0 = T 1 S J , T ( r H J ( θ 0 J ) ) − I J , T T ( θ J − r H J ( θ 0 J ) ) , (211)
where
I ^ J , T = − 1 T ∂ S ^ J , T ( θ ^ J ⋆ ) ∂ θ J T , (212) \widehat{\mathcal{I}}_{J,T}=-\frac{1}{T}\frac{\partial\widehat{S}_{J,T}\left(\widehat{\theta}^{J\star}\right)}{\partial\theta^{J\mathsf{T}}}, \tag*{(212)} I J , T = − T 1 ∂ θ J T ∂ S J , T ( θ J ⋆ ) , (212)
and the Jacobian evaluated at θ ^ J ⋆ \widehat{\theta}^{J\star} θ J ⋆ denotes the Jacobian averaged along the line segment joining θ ^ J \widehat{\theta}^{J} θ J and r H ^ J ( θ 0 J ) r_{\widehat{H}}^{J}\left(\theta_{0}^{J}\right) r H J ( θ 0 J ) .
The Jacobian in (212) contains the joint location and variance filters together with their first and second derivatives. From proposition 5, Lemmas 7 and 8, and the uniform Hessian argument of Lemma 10 we therefore obtain
I ^ J , T → P R H 0 J ⊤ I J ( θ 0 J ) R H 0 J = I J , H 0 ( θ 0 J ) (213) \widehat{\mathcal I}_{J,T} \xrightarrow{\mathbb{P}} R_{H_0}^{J\top}\mathcal I_J\left(\theta_0^J\right)R_{H_0}^J=\mathcal I_{J,H_0}\left(\theta_0^J\right) \tag*{(213)} I J , T P R H 0 J ⊤ I J ( θ 0 J ) R H 0 J = I J , H 0 ( θ 0 J ) (213)
as N , T → ∞ N,T\to\infty N , T → ∞ , where
I J ( θ 0 J ) = − E [ ∂ s t J ( θ 0 J ) ∂ θ J ⊤ ] = ( I μ μ ( θ 0 μ ) 0 I σ μ ( θ 0 J ) I σ σ ( θ 0 J ) ) . \mathcal I_J\left(\theta_0^J\right)=-\mathbb{E}\left[\frac{\partial s_t^J\left(\theta_0^J\right)}{\partial\theta^J{}^\top}\right]=\begin{pmatrix}\mathcal I_{\mu\mu}\left(\theta_0^\mu\right)&\mathbf{0}\\ \mathcal I_{\sigma\mu}\left(\theta_0^J\right)&\mathcal I_{\sigma\sigma}\left(\theta_0^J\right)\end{pmatrix}. I J ( θ 0 J ) = − E [ ∂ θ J ⊤ ∂ s t J ( θ 0 J ) ] = ( I μμ ( θ 0 μ ) I σ μ ( θ 0 J ) 0 I σ σ ( θ 0 J ) ) .
The upper-right block is zero because the first-stage criterion does not depend on θ σ \theta^\sigma θ σ , whereas the lower-left block accounts for the dependence of the second-stage score on θ μ \theta^\mu θ μ through the residuals and the conditional variance filter. The diagonal blocks are nonsingular under Assumption 13, hence, I J ( θ 0 J ) \mathcal I_J\left(\theta_0^J\right) I J ( θ 0 J ) and I J , H 0 ( θ 0 J ) \mathcal I_{J,H_0}\left(\theta_0^J\right) I J , H 0 ( θ 0 J ) are nonsingular.
By Proposition 5(d), Lemma 7(d), and the moment conditions in Assumption 11, the process { s t J ( θ 0 J ) } t ∈ Z \left\{s_t^J\left(\theta_0^J\right)\right\}_{t\in\mathbb{Z}} { s t J ( θ 0 J ) } t ∈ Z is near-epoch dependent of size − 1 -1 − 1 on { v t } t ∈ Z \left\{v_t\right\}_{t\in\mathbb{Z}} { v t } t ∈ Z . Therefore, the same Crámer–Wold argument used in Lemma 11 implies that
1 T ∑ t = 1 T s t J ( θ 0 J ) ⇒ N ( 0 , V J ( θ 0 J ) ) (214) \frac{1}{\sqrt{T}}\sum_{t=1}^{T}s_t^J\left(\theta_0^J\right)\Rightarrow\mathcal N\left(\mathbf{0},V_J\left(\theta_0^J\right)\right) \tag*{(214)} T 1 t = 1 ∑ T s t J ( θ 0 J ) ⇒ N ( 0 , V J ( θ 0 J ) ) (214)
as T → ∞ T\to\infty T → ∞ .
Then, by combining (210) and (211) we can write
T ( θ ^ J − r H J ( θ 0 J ) ) = I ^ J , T − 1 R H J 1 T ∑ t = 1 T s t J ( θ 0 J ) + o P ( 1 ) (215) \sqrt{T}\left(\widehat{\theta}^J-r_H^J\left(\theta_0^J\right)\right)=\widehat{\mathcal I}_{J,T}^{-1}R_H^J\frac{1}{\sqrt{T}}\sum_{t=1}^{T}s_t^J\left(\theta_0^J\right)+o_{\mathbb P}(1) \tag*{(215)} T ( θ J − r H J ( θ 0 J ) ) = I J , T − 1 R H J T 1 t = 1 ∑ T s t J ( θ 0 J ) + o P ( 1 ) (215)
as N , T → ∞ N,T\to\infty N , T → ∞ . Finally, R H J → P R H 0 J R_H^J\xrightarrow{\mathbb P}R_{H_0}^J R H J P R H 0 J as N , T → ∞ N,T\to\infty N , T → ∞ . Hence, (213) , (214) , and Slutsky’s theorem imply that
T ( θ ^ J − r H J ( θ 0 J ) ) ⇒ N ( 0 , I J , H 0 − 1 ( θ 0 J ) V J , H 0 ( θ 0 J ) ( I J , H 0 ( θ 0 J ) ⊤ ) − 1 ) , \sqrt{T}\left(\widehat{\theta}^J-r_H^J\left(\theta_0^J\right)\right)\Rightarrow\mathcal N\left(\mathbf{0},\mathcal I_{J,H_0}^{-1}\left(\theta_0^J\right)V_{J,H_0}\left(\theta_0^J\right)\left(\mathcal I_{J,H_0}\left(\theta_0^J\right)^\top\right)^{-1}\right), T ( θ J − r H J ( θ 0 J ) ) ⇒ N ( 0 , I J , H 0 − 1 ( θ 0 J ) V J , H 0 ( θ 0 J ) ( I J , H 0 ( θ 0 J ) ⊤ ) − 1 ) ,
which proves the result. ■
Auxiliary lemmas
Lemma 6. *Under Assumptions 1–7, the process { ( F ‾ t ⊤ , ψ ‾ t ∣ t − 1 ( θ 0 ) ⊤ ) ⊤ } t ∈ N \left\{\left(\overline{F}_t^\top,\overline{\psi}_{t|t-1}\left(\theta_0\right)^\top\right)^\top\right\}_{t\in\mathbb{N}} { ( F t ⊤ , ψ t ∣ t − 1 ( θ 0 ) ⊤ ) ⊤ } t ∈ N is near-epoch dependent of size − 1 -1 − 1 on the geometrically β \beta β -mixing process { v t } t ∈ Z \left\{v_t\right\}_{t\in\mathbb{Z}} { v t } t ∈ Z . Consequently, both { F ‾ t } t ∈ N \left\{\overline{F}_t\right\}_{t\in\mathbb{N}} { F t } t ∈ N and { ψ ‾ t ∣ t − 1 ( θ 0 ) } t ∈ N \left\{\overline{\psi}_{t|t-1}\left(\theta_0\right)\right\}_{t\in\mathbb{N}} { ψ t ∣ t − 1 ( θ 0 ) } t ∈ N are
near-epoch dependent of size − 1 -1 − 1 on { v t } t ∈ Z \{v_t\}_{t\in\mathbb{Z}} { v t } t ∈ Z .
Proof. Let k ≥ 1 k \ge1 k ≥ 1 be as in Assumption 7. For any τ ≥ 1 \tau\ge1 τ ≥ 1 , consider the τ \tau τ -truncated processes { F t ( τ ) } t ∈ N \left\{\mathbf{F}^{(\tau)}_t\right\}_{t\in\mathbb{N}} { F t ( τ ) } t ∈ N and { ψ t ∣ t − 1 ( τ ) ( θ 0 ) } t ∈ N \left\{\psi^{(\tau)}_{t|t-1}(\theta_0)\right\}_{t\in\mathbb{N}} { ψ t ∣ t − 1 ( τ ) ( θ 0 ) } t ∈ N , obtained by initializing the score-driven recursion at t − τ t-\tau t − τ with ψ 1 ∣ 0 \psi_{1|0} ψ 1∣0 and subsequently using v t , … , v t − τ v_t,\ldots,v_{t-\tau} v t , … , v t − τ . Therefore, ( F t ( τ ) T , ψ t ∣ t − 1 ( τ ) T ( θ 0 ) ) T \left(\mathbf{F}^{(\tau)\mathsf{T}}_t,\psi^{(\tau)\mathsf{T}}_{t|t-1}(\theta_0)\right)^{\mathsf{T}} ( F t ( τ ) T , ψ t ∣ t − 1 ( τ ) T ( θ 0 ) ) T is a measurable function of v t , … , v t − τ v_t,\ldots,v_{t-\tau} v t , … , v t − τ .
By repeated applications of the mean value theorem to the score-driven recursion for t ≤ τ t \le\tau t ≤ τ , Assumption 7 implies that
∥ ψ ˉ t + 1 ∣ t ( θ 0 ) − ψ t + 1 ∣ t ( τ ) ( θ 0 ) ∥ p ≤ ∥ X t , k ( θ 0 ) ∥ ∥ ψ ˉ t − k + 1 ∣ t − k ( θ 0 ) − ψ t − k + 1 ∣ t − k ( τ ) ( θ 0 ) ∥ p + ∑ j = 1 k sup ψ ∈ P ∥ Y t − j + 1 ( θ 0 ) ∥ ∥ F ˉ t − j + 1 − F t − j + 1 ( τ ) ∥ p ≤ X k ∥ ψ ˉ t − k + 1 ∣ t − k ( θ 0 ) − ψ t − k + 1 ∣ t − k ( τ ) ( θ 0 ) ∥ p + ∑ j = 1 k sup ψ ∈ P ∥ Y t − j + 1 ( θ 0 ) ∥ ∥ F ˉ t − j + 1 − F t − j + 1 ( τ ) ∥ p . (216) \begin{aligned}
\left\|\bar{\psi}_{t+1|t}(\theta_0)-\psi^{(\tau)}_{t+1|t}(\theta_0)\right\|_p
&\leq\left\|X_{t,k}(\theta_0)\right\|\left\|\bar{\psi}_{t-k+1|t-k}(\theta_0)-\psi^{(\tau)}_{t-k+1|t-k}(\theta_0)\right\|_p \\
&\quad+\sum_{j=1}^{k}\sup_{\psi\in\mathcal{P}}\left\|Y_{t-j+1}(\theta_0)\right\|\left\|\bar{\mathbf{F}}_{t-j+1}-\mathbf{F}^{(\tau)}_{t-j+1}\right\|_p \\
&\leq\mathcal{X}_k\left\|\bar{\psi}_{t-k+1|t-k}(\theta_0)-\psi^{(\tau)}_{t-k+1|t-k}(\theta_0)\right\|_p \\
&\quad+\sum_{j=1}^{k}\sup_{\psi\in\mathcal{P}}\left\|Y_{t-j+1}(\theta_0)\right\|\left\|\bar{\mathbf{F}}_{t-j+1}-\mathbf{F}^{(\tau)}_{t-j+1}\right\|_p . \tag*{(216)}
\end{aligned} ψ ˉ t + 1∣ t ( θ 0 ) − ψ t + 1∣ t ( τ ) ( θ 0 ) p ≤ ∥ X t , k ( θ 0 ) ∥ ψ ˉ t − k + 1∣ t − k ( θ 0 ) − ψ t − k + 1∣ t − k ( τ ) ( θ 0 ) p + j = 1 ∑ k ψ ∈ P sup ∥ Y t − j + 1 ( θ 0 ) ∥ F ˉ t − j + 1 − F t − j + 1 ( τ ) p ≤ X k ψ ˉ t − k + 1∣ t − k ( θ 0 ) − ψ t − k + 1∣ t − k ( τ ) ( θ 0 ) p + j = 1 ∑ k ψ ∈ P sup ∥ Y t − j + 1 ( θ 0 ) ∥ F ˉ t − j + 1 − F t − j + 1 ( τ ) p . (216)
Thus, without loss of generality, assuming that t ≥ τ + 1 t \geq\tau+1 t ≥ τ + 1 and proceeding recursively, we get
∥ ψ ˉ t + 1 ∣ t ( θ 0 ) − ψ t + 1 ∣ t ( τ ) ( θ 0 ) ∥ p ≤ X k ⌊ t / k ⌋ ∥ ψ ˉ 1 ∣ 0 ( θ 0 ) − ψ 1 ∣ 0 ∥ p + X k ⌊ t / k ⌋ ∑ j = 1 t − k X k ⌊ ( j − 1 ) / k ⌋ sup ψ ∈ P ∥ Y t − k − j + 1 ( θ 0 ) ∥ p ∥ F ˉ t − k − j + 1 − F t − k − j + 1 ( τ ) ∥ p , (217) \begin{aligned}
\left\|\bar{\psi}_{t+1|t}(\theta_0)-\psi^{(\tau)}_{t+1|t}(\theta_0)\right\|_p \\
&\leq\mathcal{X}_k^{\lfloor t/k\rfloor}\left\|\bar{\psi}_{1|0}(\theta_0)-\psi_{1|0}\right\|_p+\mathcal{X}_k^{\lfloor t/k\rfloor}\sum_{j=1}^{t-k}\mathcal{X}_k^{\lfloor(j-1)/k\rfloor}\sup_{\psi\in\mathcal{P}}\left\|Y_{t-k-j+1}(\theta_0)\right\|_p\left\|\bar{\mathbf{F}}_{t-k-j+1}-\mathbf{F}^{(\tau)}_{t-k-j+1}\right\|_p , \tag*{(217)}
\end{aligned} ψ ˉ t + 1∣ t ( θ 0 ) − ψ t + 1∣ t ( τ ) ( θ 0 ) p ≤ X k ⌊ t / k ⌋ ψ ˉ 1∣0 ( θ 0 ) − ψ 1∣0 p + X k ⌊ t / k ⌋ j = 1 ∑ t − k X k ⌊( j − 1 ) / k ⌋ ψ ∈ P sup ∥ Y t − k − j + 1 ( θ 0 ) ∥ p F ˉ t − k − j + 1 − F t − k − j + 1 ( τ ) p , (217)
and note that ∥ ψ ˉ 1 ∣ 0 ( θ 0 ) − ψ 1 ∣ 0 ∥ p ≤ Δ 1 < ∞ \left\|\bar{\psi}_{1|0}(\theta_0)-\psi_{1|0}\right\|_p\leq\Delta_1<\infty ψ ˉ 1∣0 ( θ 0 ) − ψ 1∣0 p ≤ Δ 1 < ∞ , with ψ ˉ 1 ∣ 0 ( θ 0 ) = ( I r − Φ 0 ) − 1 δ 0 + ∑ k = 0 ∞ Φ 0 k K 0 u − k \bar{\psi}_{1|0}(\theta_0)=(I_r-\Phi_0)^{-1}\delta_0+\sum_{k=0}^{\infty}\Phi_0^kK_0u_{-k} ψ ˉ 1∣0 ( θ 0 ) = ( I r − Φ 0 ) − 1 δ 0 + ∑ k = 0 ∞ Φ 0 k K 0 u − k which converges almost surely, by Assumption 1, whereas
∥ sup ψ ∈ P ∥ Y t − k − j + 1 ( θ 0 ) ∥ ∥ F ˉ t − k − j + 1 − F t − k − j + 1 ( τ ) ∥ ∥ p ≤ Δ 2 < ∞ , \left\|\sup_{\psi\in\mathcal{P}}\left\|Y_{t-k-j+1}(\theta_0)\right\|\left\|\bar{\mathbf{F}}_{t-k-j+1}-\mathbf{F}^{(\tau)}_{t-k-j+1}\right\|\right\|_p\leq\Delta_2<\infty, ψ ∈ P sup ∥ Y t − k − j + 1 ( θ 0 ) ∥ F ˉ t − k − j + 1 − F t − k − j + 1 ( τ ) p ≤ Δ 2 < ∞ ,
for some finite positive real Δ 2 \Delta_2 Δ 2 independent of t t t and τ \tau τ , and X k < 1 \mathcal{X}_k<1 X k < 1 , are ensured by Assumption 7.
Since t ≥ τ + 1 t\geq\tau+1 t ≥ τ + 1 and 0 < X k < 1 0<\mathcal{X}_k<1 0 < X k < 1 , it follows that X k ⌊ t / k ⌋ ≤ X k ⌊ τ / k ⌋ \mathcal{X}_k^{\lfloor t/k\rfloor}\leq\mathcal{X}_k^{\lfloor\tau/k\rfloor} X k ⌊ t / k ⌋ ≤ X k ⌊ τ / k ⌋ . Moreover, Assumption 7(ii), Minkowski’s inequality, and the geometric-sum bound imply that
sup t ∈ N ∥ ∑ j = 1 t − k X k ⌊ ( j − 1 ) / k ⌋ sup ψ ∈ P ∥ Y t − k − j + 1 ( θ 0 ) ∥ ∥ F ˉ t − k − j + 1 − F t − k − j + 1 ( τ ) ∥ ∥ p ≤ Δ 2 ∑ j = 1 ∞ X k ⌊ ( j − 1 ) / k ⌋ = k Δ 2 1 − X k < ∞ , \begin{aligned}
\sup_{t\in\mathbb{N}}\left\|\sum_{j=1}^{t-k}\mathcal{X}_k^{\lfloor(j-1)/k\rfloor}\sup_{\psi\in\mathcal{P}}\left\|Y_{t-k-j+1}(\theta_0)\right\|\left\|\bar{\mathbf{F}}_{t-k-j+1}-\mathbf{F}^{(\tau)}_{t-k-j+1}\right\|\right\|_p
&\leq\Delta_2\sum_{j=1}^{\infty}\mathcal{X}_k^{\lfloor(j-1)/k\rfloor} \\
&= \frac{k\Delta_2}{1-\mathcal{X}_k}<\infty,
\end{aligned} t ∈ N sup j = 1 ∑ t − k X k ⌊( j − 1 ) / k ⌋ ψ ∈ P sup ∥ Y t − k − j + 1 ( θ 0 ) ∥ F ˉ t − k − j + 1 − F t − k − j + 1 ( τ ) p ≤ Δ 2 j = 1 ∑ ∞ X k ⌊( j − 1 ) / k ⌋ = 1 − X k k Δ 2 < ∞ ,
Consequently, combining the cases t ≤ τ t \le\tau t ≤ τ and t ≥ τ + 1 t \ge\tau+ 1 t ≥ τ + 1 , we obtain
sup t ∈ N ∥ ψ ˉ t ∣ t − 1 ( θ 0 ) − ψ t ∣ t − 1 ( τ ) ( θ 0 ) ∥ p ≤ X k ⌊ t / k ⌋ ⋅ Δ 1 + X k ⌊ τ / k ⌋ ⋅ k Δ 2 / ( 1 − X k ) = O ( X k ⌊ τ / k ⌋ ) . (218) \sup_{t \in\mathbb{N}} \left\lVert\bar{\psi}_{t|t-1}(\theta_0) - \psi^{(\tau)}_{t|t-1}(\theta_0) \right\rVert_p \le\mathcal{X}_k^{\lfloor t/k \rfloor} \cdot\Delta_1 + \mathcal{X}_k^{\lfloor\tau/k \rfloor} \cdot k\Delta_2/(1-\mathcal{X}_k) = O\left(\mathcal{X}_k^{\lfloor\tau/k \rfloor}\right). \tag*{(218)} t ∈ N sup ψ ˉ t ∣ t − 1 ( θ 0 ) − ψ t ∣ t − 1 ( τ ) ( θ 0 ) p ≤ X k ⌊ t / k ⌋ ⋅ Δ 1 + X k ⌊ τ / k ⌋ ⋅ k Δ 2 / ( 1 − X k ) = O ( X k ⌊ τ / k ⌋ ) . (218)
For p > 2 p > 2 p > 2 and using Hölder’s inequality, we get
∥ F ˉ t − F t ( τ ) ∥ 2 = ∥ q ( ψ ˉ t ∣ t − 1 ( θ 0 ) , v t ) − q ( ψ t ∣ t − 1 ( τ ) ( θ 0 ) , v t ) ∥ 2 ≤ ∥ B t ( ψ ˉ t ∣ t − 1 ( θ 0 ) , ψ t ∣ t − 1 ( τ ) ( θ 0 ) ) ∥ 2 p p − 2 ∥ ψ ˉ t ∣ t − 1 ( θ 0 ) − ψ t ∣ t − 1 ( τ ) ( θ 0 ) ∥ p ≤ ∥ B t ( ψ ˉ t ∣ t − 1 ( θ 0 ) , ψ t ∣ t − 1 ( τ ) ( θ 0 ) ) ∥ 2 p p − 2 ∥ ψ ˉ t ∣ t − 1 ( θ 0 ) − ψ t ∣ t − 1 ( τ ) ( θ 0 ) ∥ p ≤ B ∥ ψ ˉ t ∣ t − 1 ( θ 0 ) − ψ t ∣ t − 1 ( τ ) ( θ 0 ) ∥ p . \begin{aligned}
\left\lVert\bar{F}_t - F_t^{(\tau)} \right\rVert_2
&= \left\lVert q\left(\bar{\psi}_{t|t-1}(\theta_0),v_t\right)-q\left(\psi^{(\tau)}_{t|t-1}(\theta_0),v_t\right) \right\rVert_2 \\
&\le\left\lVert B_t\left(\bar{\psi}_{t|t-1}(\theta_0),\psi^{(\tau)}_{t|t-1}(\theta_0)\right) \right\rVert_{\frac{2p}{p-2}}\left\lVert\bar{\psi}_{t|t-1}(\theta_0)-\psi^{(\tau)}_{t|t-1}(\theta_0) \right\rVert_p \\
&\le\left\lVert B_t\left(\bar{\psi}_{t|t-1}(\theta_0),\psi^{(\tau)}_{t|t-1}(\theta_0)\right) \right\rVert_{\frac{2p}{p-2}}\left\lVert\bar{\psi}_{t|t-1}(\theta_0)-\psi^{(\tau)}_{t|t-1}(\theta_0) \right\rVert_p \\
&\le B\left\lVert\bar{\psi}_{t|t-1}(\theta_0)-\psi^{(\tau)}_{t|t-1}(\theta_0) \right\rVert_p.
\end{aligned} F ˉ t − F t ( τ ) 2 = q ( ψ ˉ t ∣ t − 1 ( θ 0 ) , v t ) − q ( ψ t ∣ t − 1 ( τ ) ( θ 0 ) , v t ) 2 ≤ B t ( ψ ˉ t ∣ t − 1 ( θ 0 ) , ψ t ∣ t − 1 ( τ ) ( θ 0 ) ) p − 2 2 p ψ ˉ t ∣ t − 1 ( θ 0 ) − ψ t ∣ t − 1 ( τ ) ( θ 0 ) p ≤ B t ( ψ ˉ t ∣ t − 1 ( θ 0 ) , ψ t ∣ t − 1 ( τ ) ( θ 0 ) ) p − 2 2 p ψ ˉ t ∣ t − 1 ( θ 0 ) − ψ t ∣ t − 1 ( τ ) ( θ 0 ) p ≤ B ψ ˉ t ∣ t − 1 ( θ 0 ) − ψ t ∣ t − 1 ( τ ) ( θ 0 ) p .
and hence, Assumptions 1–7, and (218) ,
sup t ∈ N ∥ F ˉ t − F t ( τ ) ∥ 2 ≤ B ⋅ ( X k ⌊ t / k ⌋ ⋅ Δ 1 + X k ⌊ τ / k ⌋ ⋅ k Δ 2 / ( 1 − X k ) ) . (219) \sup_{t \in\mathbb{N}}\left\lVert\bar{F}_t-F_t^{(\tau)}\right\rVert_2 \le B\cdot\left(\mathcal{X}_k^{\lfloor t/k\rfloor}\cdot\Delta_1+\mathcal{X}_k^{\lfloor\tau/k\rfloor}\cdot k\Delta_2/(1-\mathcal{X}_k)\right). \tag*{(219)} t ∈ N sup F ˉ t − F t ( τ ) 2 ≤ B ⋅ ( X k ⌊ t / k ⌋ ⋅ Δ 1 + X k ⌊ τ / k ⌋ ⋅ k Δ 2 / ( 1 − X k ) ) . (219)
Equations (218) and (219) imply that
sup t ∈ N ∥ ( F ˉ t ⊤ , ψ ˉ t ∣ t − 1 ( θ 0 ) ⊤ ) ⊤ − ( F t ( τ ) ⊤ , ψ t ∣ t − 1 ( τ ) ( θ 0 ) ⊤ ) ⊤ ∥ 2 ≤ B ′ ⋅ ( X k ⌊ t / k ⌋ ⋅ Δ 1 + X k ⌊ τ / k ⌋ ⋅ k Δ 2 / ( 1 − X k ) ) = O ( X k ⌊ τ / k ⌋ ) , (220) \sup_{t \in\mathbb{N}}\left\lVert\left(\bar{F}_t^{\top},\bar{\psi}_{t|t-1}(\theta_0)^{\top}\right)^{\top}-\left(F_t^{(\tau)\top},\psi_{t|t-1}^{(\tau)}(\theta_0)^{\top}\right)^{\top}\right\rVert_2
\le B'\cdot\left(\mathcal{X}_k^{\lfloor t/k\rfloor}\cdot\Delta_1+\mathcal{X}_k^{\lfloor\tau/k\rfloor}\cdot k\Delta_2/(1-\mathcal{X}_k)\right)
=O\left(\mathcal{X}_k^{\lfloor\tau/k\rfloor}\right), \tag*{(220)} t ∈ N sup ( F ˉ t ⊤ , ψ ˉ t ∣ t − 1 ( θ 0 ) ⊤ ) ⊤ − ( F t ( τ ) ⊤ , ψ t ∣ t − 1 ( τ ) ( θ 0 ) ⊤ ) ⊤ 2 ≤ B ′ ⋅ ( X k ⌊ t / k ⌋ ⋅ Δ 1 + X k ⌊ τ / k ⌋ ⋅ k Δ 2 / ( 1 − X k ) ) = O ( X k ⌊ τ / k ⌋ ) , (220)
for some B ′ ≥ B > 0 B' \ge B > 0 B ′ ≥ B > 0 . Since the truncated pair is a measurable function of v t , … , v t − τ v_t,\ldots,v_{t-\tau} v t , … , v t − τ , it follows from (220) that
0 ≤ sup t ∈ N ∥ ( F ˉ t ⊤ , ψ ˉ t ∣ t − 1 ( θ 0 ) ⊤ ) ⊤ − E [ ( F ˉ t ⊤ , ψ ˉ t ∣ t − 1 ( θ 0 ) ⊤ ) ⊤ ∣ v t , … , v t − τ ] ∥ 2 ≤ sup t ∈ N ∥ ( F ˉ t ⊤ , ψ ˉ t ∣ t − 1 ( θ 0 ) ⊤ ) ⊤ − ( F t ( τ ) ⊤ , ψ t ∣ t − 1 ( τ ) ( θ 0 ) ⊤ ) ⊤ ∥ 2 = O ( X k ⌊ τ / k ⌋ ) . \begin{aligned}
0 &\le\sup_{t\in\mathbb{N}}\left\lVert\left(\bar{F}_t^{\top},\bar{\psi}_{t|t-1}(\theta_0)^{\top}\right)^{\top}-\mathbb{E}\left[\left(\bar{F}_t^{\top},\bar{\psi}_{t|t-1}(\theta_0)^{\top}\right)^{\top}\mid v_t,\ldots,v_{t-\tau}\right]\right\rVert_2 \\
&\le\sup_{t\in\mathbb{N}}\left\lVert\left(\bar{F}_t^{\top},\bar{\psi}_{t|t-1}(\theta_0)^{\top}\right)^{\top}-\left(F_t^{(\tau)\top},\psi_{t|t-1}^{(\tau)}(\theta_0)^{\top}\right)^{\top}\right\rVert_2 \\
&=O\left(\mathcal{X}_k^{\lfloor\tau/k\rfloor}\right).
\end{aligned} 0 ≤ t ∈ N sup ( F ˉ t ⊤ , ψ ˉ t ∣ t − 1 ( θ 0 ) ⊤ ) ⊤ − E [ ( F ˉ t ⊤ , ψ ˉ t ∣ t − 1 ( θ 0 ) ⊤ ) ⊤ ∣ v t , … , v t − τ ] 2 ≤ t ∈ N sup ( F ˉ t ⊤ , ψ ˉ t ∣ t − 1 ( θ 0 ) ⊤ ) ⊤ − ( F t ( τ ) ⊤ , ψ t ∣ t − 1 ( τ ) ( θ 0 ) ⊤ ) ⊤ 2 = O ( X k ⌊ τ / k ⌋ ) .
Because 0 < X k < 1 0 < \mathcal{X}_k < 1 0 < X k < 1 , the approximation coefficients decrease geometrically and are therefore of size − 1 -1 − 1 . Hence, by an application of Theorem 6.12 (or, equivalently, Lemma D5 in Appendix D) of Pötscher and Prucha [47 ] , we obtain that the joint process { ( F ˉ t ⊤ , ψ ˉ t ∣ t − 1 ( θ 0 ) ⊤ ) ⊤ } t ∈ N \left\{\left(\bar{F}_t^{\top},\bar{\psi}_{t|t-1}(\theta_0)^{\top}\right)^{\top}\right\}_{t\in\mathbb{N}} { ( F ˉ t ⊤ , ψ ˉ t ∣ t − 1 ( θ 0 ) ⊤ ) ⊤ } t ∈ N is near-epoch dependent of size − 1 -1 − 1 on { v t } t ∈ Z \{v_t\}_{t\in\mathbb{Z}} { v t } t ∈ Z when generated by the nonlinear system given by (15) –(17) . The two marginal processes inherit the same property as coordinate projections of the joint process.
Lemma 7. Consider the first derivative process associated with the score-driven recursion,
∇ θ ψ t + 1 ∣ t ( θ ) = X t ( θ ) ∇ θ ψ t ∣ t − 1 ( θ ) + D t ( θ ) , (221) \nabla^\theta\psi_{t+1|t}(\theta)=X_t(\theta)\nabla^\theta\psi_{t|t-1}(\theta)+D_t(\theta), \tag*{(221)} ∇ θ ψ t + 1∣ t ( θ ) = X t ( θ ) ∇ θ ψ t ∣ t − 1 ( θ ) + D t ( θ ) , (221) where
X t ( θ ) = Φ + K ∂ u t ( θ ) ∂ ψ t ∣ t − 1 ∣ ψ t ∣ t − 1 = ψ t ∣ t − 1 ( θ ) , D t ( θ ) = ∂ ψ t + 1 ∣ t ( θ ) ∂ θ T , X_t(\theta)=\Phi+K\left.\frac{\partial u_t(\theta)}{\partial\psi_{t|t-1}}\right|_{\psi_{t|t-1}=\psi_{t|t-1}(\theta)},\qquad D_t(\theta)=\frac{\partial\psi_{t+1|t}(\theta)}{\partial\theta^\mathsf{T}}, X t ( θ ) = Φ + K ∂ ψ t ∣ t − 1 ∂ u t ( θ ) ψ t ∣ t − 1 = ψ t ∣ t − 1 ( θ ) , D t ( θ ) = ∂ θ T ∂ ψ t + 1∣ t ( θ ) , and the partial derivative defining D t ( θ ) D_t(\theta) D t ( θ ) is taken holding ψ t ∣ t − 1 \psi_{t|t-1} ψ t ∣ t − 1 fixed.
By differentiating (18) with respect to θ \theta θ we get
∇ θ ψ ^ t ∣ t − 1 ( θ ) − ∇ θ ψ ~ t ∣ t − 1 ( θ ) = ( ∇ θ ψ ^ t ∣ t − 1 ( θ ) − ∇ θ ψ ^ t ∣ t − 1 ( θ ) ) + ( ∇ θ ψ ^ t ∣ t − 1 ( θ ) − ∇ θ ψ ~ t ∣ t − 1 ( θ ) ) + ( ∇ θ ψ ~ t ∣ t − 1 ( θ ) − ∇ θ ψ ˉ t ∣ t − 1 ( θ ) ) . (222) \begin{aligned}
\nabla^\theta\widehat{\psi}_{t|t-1}(\theta)-\nabla^\theta\widetilde{\psi}_{t|t-1}(\theta)
={}&\left(\nabla^\theta\widehat{\psi}_{t|t-1}(\theta)-\nabla^\theta\widehat{\psi}_{t|t-1}(\theta)\right)\\
&+\left(\nabla^\theta\widehat{\psi}_{t|t-1}(\theta)-\nabla^\theta\widetilde{\psi}_{t|t-1}(\theta)\right)
+\left(\nabla^\theta\widetilde{\psi}_{t|t-1}(\theta)-\nabla^\theta\bar{\psi}_{t|t-1}(\theta)\right). \tag*{(222)}
\end{aligned} ∇ θ ψ t ∣ t − 1 ( θ ) − ∇ θ ψ t ∣ t − 1 ( θ ) = ( ∇ θ ψ t ∣ t − 1 ( θ ) − ∇ θ ψ t ∣ t − 1 ( θ ) ) + ( ∇ θ ψ t ∣ t − 1 ( θ ) − ∇ θ ψ t ∣ t − 1 ( θ ) ) + ( ∇ θ ψ t ∣ t − 1 ( θ ) − ∇ θ ψ ˉ t ∣ t − 1 ( θ ) ) . (222) Then, under Assumptions 1 1 1 –7 7 7 and 11 11 11 , with p > 2 p>2 p > 2 , the following results hold:
(a) For any t ≥ 1 t\ge1 t ≥ 1 , as N , T → ∞ N,T\to\infty N , T → ∞ ,
sup θ ∈ Θ ∥ ∇ θ ψ ^ t + 1 ∣ t ( θ ) − ∇ θ ψ t + 1 ∣ t ( θ ) ∥ = O P ( 1 min ( N , T ) ) + O a . s . ( X k ⌊ t / ( 2 k ) ⌋ ) . \sup_{\theta\in\Theta}\left\|\nabla^\theta\widehat{\psi}_{t+1|t}(\theta)-\nabla^\theta\psi_{t+1|t}(\theta)\right\|=O_{\mathbb{P}}\left(\frac{1}{\min\left(\sqrt{N},T\right)}\right)+O_{\mathrm{a.s.}}\left(\mathcal{X}_k^{\lfloor t/(2k)\rfloor}\right). θ ∈ Θ sup ∇ θ ψ t + 1∣ t ( θ ) − ∇ θ ψ t + 1∣ t ( θ ) = O P min ( N , T ) 1 + O a.s. ( X k ⌊ t / ( 2 k )⌋ ) . (b) For any t ≥ 1 t\ge1 t ≥ 1 , as N , T → ∞ N,T\to\infty N , T → ∞ ,
sup θ ∈ Θ ∥ ∇ θ ψ ^ t ∣ t − 1 ( θ ) − ∇ θ ψ ~ t ∣ t − 1 ( θ ) ∥ = O P ( 1 min ( N , T ) ) + O a . s . ( X k ⌊ t / ( 2 k ) ⌋ + ϱ k ⌊ t / ( 2 k ) ⌋ ) . \sup_{\theta\in\Theta}\left\|\nabla^\theta\widehat{\psi}_{t|t-1}(\theta)-\nabla^\theta\widetilde{\psi}_{t|t-1}(\theta)\right\|=O_{\mathbb{P}}\left(\frac{1}{\min\left(\sqrt{N},\sqrt{T}\right)}\right)+O_{\mathrm{a.s.}}\left(\mathcal{X}_k^{\lfloor t/(2k)\rfloor}+\varrho_k^{\lfloor t/(2k)\rfloor}\right). θ ∈ Θ sup ∇ θ ψ t ∣ t − 1 ( θ ) − ∇ θ ψ t ∣ t − 1 ( θ ) = O P min ( N , T ) 1 + O a.s. ( X k ⌊ t / ( 2 k )⌋ + ϱ k ⌊ t / ( 2 k )⌋ ) . (c) For any fixed initialization ψ 1 ∣ 0 ∈ P \psi_{1|0}\in\mathcal{P} ψ 1∣0 ∈ P , as t → ∞ t\to\infty t → ∞ ,
sup θ ∈ Θ ∥ ∇ θ ψ ~ t ∣ t − 1 ( θ ) − ∇ θ ψ ˉ t ∣ t − 1 ( θ ) ∥ = O a . s . ( X k ⌊ t / ( 2 k ) ⌋ + ϱ k ⌊ t / ( 2 k ) ⌋ ) . \sup_{\theta\in\Theta}\left\|\nabla^\theta\widetilde{\psi}_{t|t-1}(\theta)-\nabla^\theta\bar{\psi}_{t|t-1}(\theta)\right\|=O_{\mathrm{a.s.}}\left(\mathcal{X}_k^{\lfloor t/(2k)\rfloor}+\varrho_k^{\lfloor t/(2k)\rfloor}\right). θ ∈ Θ sup ∇ θ ψ t ∣ t − 1 ( θ ) − ∇ θ ψ ˉ t ∣ t − 1 ( θ ) = O a.s. ( X k ⌊ t / ( 2 k )⌋ + ϱ k ⌊ t / ( 2 k )⌋ ) . (d) The process { ∇ θ ψ ˉ t ∣ t − 1 ( θ ) } t ∈ Z \left\{\nabla^\theta\bar{\psi}_{t|t-1}(\theta)\right\}_{t\in\mathbb{Z}} { ∇ θ ψ ˉ t ∣ t − 1 ( θ ) } t ∈ Z is strictly stationary and ergodic and satisfies
E [ sup θ ∈ Θ ∥ ∇ θ ψ ˉ t ∣ t − 1 ( θ ) ∥ p ] < ∞ . \mathbb{E}\left[\sup_{\theta\in\Theta}\left\|\nabla^\theta\bar{\psi}_{t|t-1}(\theta)\right\|^p\right]<\infty. E [ θ ∈ Θ sup ∇ θ ψ ˉ t ∣ t − 1 ( θ ) p ] < ∞. Moreover, { ∇ θ ψ ˉ t ∣ t − 1 ( θ 0 ) } t ∈ Z \left\{\nabla^\theta\bar{\psi}_{t|t-1}(\theta_0)\right\}_{t\in\mathbb{Z}} { ∇ θ ψ ˉ t ∣ t − 1 ( θ 0 ) } t ∈ Z is near-epoch dependent of size − 1 -1 − 1 on the geometrically β \beta β -mixing process { v t } t ∈ Z \{v_t\}_{t\in\mathbb{Z}} { v t } t ∈ Z .
(e) For any fixed initialization ψ ∣ 1 ∣ 0 ∈ P \psi|_{1|0} \in\mathcal{P} ψ ∣ 1∣0 ∈ P and any t ≥ 1 t \ge1 t ≥ 1 , as N , T → ∞ N,T \to\infty N , T → ∞ ,
sup θ ∈ Θ ∥ ∇ θ ψ ^ t + 1 ∣ t ( θ ) − ∇ θ ψ ‾ t + 1 ∣ t ( θ ) ∥ = O P ( 1 min ( N , T ) ) + O P ( 1 min ( N , T ) ) + O a . s . ( X k [ t / ( 2 k ) ] + ϱ k [ t / ( 2 k ) ] ) . \sup_{\theta\in\Theta}\left\|\nabla^\theta\widehat{\psi}_{t+1|t}(\theta)-\nabla^\theta\overline{\psi}_{t+1|t}(\theta)\right\|=O_{\mathbb{P}}\left(\frac{1}{\min\left(\sqrt{N},T\right)}\right)+O_{\mathbb{P}}\left(\frac{1}{\min\left(\sqrt{N},\sqrt{T}\right)}\right)+O_{\mathrm{a.s.}}\left(\mathcal{X}_k^{[t/(2k)]}+\varrho_k^{[t/(2k)]}\right). θ ∈ Θ sup ∇ θ ψ t + 1∣ t ( θ ) − ∇ θ ψ t + 1∣ t ( θ ) = O P min ( N , T ) 1 + O P min ( N , T ) 1 + O a.s. ( X k [ t / ( 2 k )] + ϱ k [ t / ( 2 k )] ) .
Proof. We first establish two preliminary results that will be used throughout the proof. Iterating (221) in blocks of length k k k yields
∇ θ ψ ‾ t + 1 ∣ t ( θ ) = ( ∏ j = 1 k X t − j + 1 ( θ ) ) ∇ θ ψ ‾ t − k + 1 ∣ t − k ( θ ) + ∑ j = 1 k ( ∏ h = 1 j − 1 X t − h + 1 ( θ ) ) D t − j + 1 ( θ ) , \nabla^\theta\overline{\psi}_{t+1|t}(\theta)=\left(\prod_{j=1}^{k}X_{t-j+1}(\theta)\right)\nabla^\theta\overline{\psi}_{t-k+1|t-k}(\theta)+\sum_{j=1}^{k}\left(\prod_{h=1}^{j-1}X_{t-h+1}(\theta)\right)D_{t-j+1}(\theta), ∇ θ ψ t + 1∣ t ( θ ) = ( j = 1 ∏ k X t − j + 1 ( θ ) ) ∇ θ ψ t − k + 1∣ t − k ( θ ) + j = 1 ∑ k ( h = 1 ∏ j − 1 X t − h + 1 ( θ ) ) D t − j + 1 ( θ ) ,
where the product corresponding to j = 1 j=1 j = 1 is understood to be I r I_r I r . Therefore,
∥ sup θ ∈ Θ ∥ ∇ θ ψ ‾ t + 1 ∣ t ( θ ) ∥ ∥ p ≤ X k ∥ sup θ ∈ Θ ∥ ∇ θ ψ ‾ t − k + 1 ∣ t − k ( θ ) ∥ ∥ p + ∥ sup θ ∈ Θ ∥ ∑ j = 1 k ( ∏ h = 1 j − 1 X t − h + 1 ( θ ) ) D t − j + 1 ( θ ) ∥ ∥ p . (223) \left\|\sup_{\theta\in\Theta}\left\|\nabla^\theta\overline{\psi}_{t+1|t}(\theta)\right\|\right\|_p
\leq\mathcal{X}_k\left\|\sup_{\theta\in\Theta}\left\|\nabla^\theta\overline{\psi}_{t-k+1|t-k}(\theta)\right\|\right\|_p+\left\|\sup_{\theta\in\Theta}\left\|\sum_{j=1}^{k}\left(\prod_{h=1}^{j-1}X_{t-h+1}(\theta)\right)D_{t-j+1}(\theta)\right\|\right\|_p . \tag*{(223)} θ ∈ Θ sup ∇ θ ψ t + 1∣ t ( θ ) p ≤ X k θ ∈ Θ sup ∇ θ ψ t − k + 1∣ t − k ( θ ) p + θ ∈ Θ sup j = 1 ∑ k ( h = 1 ∏ j − 1 X t − h + 1 ( θ ) ) D t − j + 1 ( θ ) p . (223)
By the definition of D t ( θ ) D_t(\theta) D t ( θ ) , the matrix ∂ vec D t ( θ ) ∂ ψ t ∣ t − 1 \frac{\partial\operatorname{vec}D_t(\theta)}{\partial\psi_{t|t-1}} ∂ ψ t ∣ t − 1 ∂ vec D t ( θ ) contains the derivative of the score entering K ∂ u t ( θ ) ∂ ψ t ∣ t − 1 K\frac{\partial u_t(\theta)}{\partial\psi_{t|t-1}} K ∂ ψ t ∣ t − 1 ∂ u t ( θ ) . Hence, the compactness of Θ \Theta Θ and Assumption 11( i v ) (iv) ( i v ) imply that, for every q > 2 q>2 q > 2 ,
sup t ∈ Z ∥ sup θ ∈ Θ sup ψ ∈ P ∥ X t ( θ ) ∥ ∥ q < ∞ . (224) \sup_{t\in\mathbb{Z}}\left\|\sup_{\theta\in\Theta}\sup_{\psi\in\mathcal{P}}\left\|X_t(\theta)\right\|\right\|_q<\infty. \tag*{(224)} t ∈ Z sup θ ∈ Θ sup ψ ∈ P sup ∥ X t ( θ ) ∥ q < ∞. (224)
Consequently, since k k k is fixed, repeated applications of Hölder’s inequality, together with (224) and Assumption 11( i v ) (iv) ( i v ) , give, for every j = 1 , … , k j=1,\ldots,k j = 1 , … , k ,
sup t ∈ Z ∥ sup θ ∈ Θ ∥ ( ∏ h = 1 j − 1 X t − h + 1 ( θ ) ) D t − j + 1 ( θ ) ∥ ∥ p ≤ ( sup s ∈ Z ∥ sup θ ∈ Θ sup ψ ∈ P ∥ X s ( θ ) ∥ ∥ j p ) j − 1 sup s ∈ Z ∥ sup θ ∈ Θ ∥ D s ( θ ) ∥ ∥ j p < ∞ . \sup_{t\in\mathbb{Z}}\left\|\sup_{\theta\in\Theta}\left\|\left(\prod_{h=1}^{j-1}X_{t-h+1}(\theta)\right)D_{t-j+1}(\theta)\right\|\right\|_p
\leq\left(\sup_{s\in\mathbb{Z}}\left\|\sup_{\theta\in\Theta}\sup_{\psi\in\mathcal{P}}\left\|X_s(\theta)\right\|\right\|_{jp}\right)^{j-1}\sup_{s\in\mathbb{Z}}\left\|\sup_{\theta\in\Theta}\left\|D_s(\theta)\right\|\right\|_{jp}<\infty. t ∈ Z sup θ ∈ Θ sup ( h = 1 ∏ j − 1 X t − h + 1 ( θ ) ) D t − j + 1 ( θ ) p ≤ s ∈ Z sup θ ∈ Θ sup ψ ∈ P sup ∥ X s ( θ ) ∥ j p j − 1 s ∈ Z sup θ ∈ Θ sup ∥ D s ( θ ) ∥ j p < ∞.
Since the sum in the second term on the right-hand side of (223) contains only k k k terms, it follows that, for some finite positive real K K K independent of t t t ,
sup t ∈ Z sup θ ∈ Θ ∥ ∥ ∑ j = 1 k ( ∏ h = 1 j − 1 X t − h + 1 ( θ ) ) D t − j + 1 ( θ ) ∥ ∥ p ≤ K . (225) \sup_{t\in\mathbb{Z}}\sup_{\theta\in\Theta}\left\|\left\|\sum_{j=1}^{k}\left(\prod_{h=1}^{j-1}X_{t-h+1}(\theta)\right)D_{t-j+1}(\theta)\right\|\right\|_{p}\le K. \tag*{(225)} t ∈ Z sup θ ∈ Θ sup j = 1 ∑ k ( h = 1 ∏ j − 1 X t − h + 1 ( θ ) ) D t − j + 1 ( θ ) p ≤ K . (225)
Now, for every n ≥ 1 n\ge1 n ≥ 1 , let { ∇ θ ψ ˉ s ∣ s − 1 [ n ] ( θ ) } s ∈ Z \left\{\nabla^\theta\bar{\psi}^{[n]}_{s|s-1}(\theta)\right\}_{s\in\mathbb{Z}} { ∇ θ ψ ˉ s ∣ s − 1 [ n ] ( θ ) } s ∈ Z denote the derivative recursion driven by the stationary processes { H 0 F ˉ s } s ∈ Z \left\{H_0\bar{F}_s\right\}_{s\in\mathbb{Z}} { H 0 F ˉ s } s ∈ Z and { ψ ˉ s ∣ s − 1 ( θ ) } s ∈ Z \left\{\bar{\psi}_{s|s-1}(\theta)\right\}_{s\in\mathbb{Z}} { ψ ˉ s ∣ s − 1 ( θ ) } s ∈ Z and initialized at time t − n k t-nk t − nk according to ∇ θ ψ ˉ t − n k + 1 ∣ t − n k [ n ] ( θ ) = 0 r × d \nabla^\theta\bar{\psi}^{[n]}_{t-nk+1|t-nk}(\theta)=0_{r\times d} ∇ θ ψ ˉ t − nk + 1∣ t − nk [ n ] ( θ ) = 0 r × d . By an application of the k k k -step inequality in (223) recursively over the n n n blocks between t − n k t-nk t − nk and t t t , and using (225) , we get
sup θ ∈ Θ ∥ ∇ θ ψ ˉ t + 1 ∣ t [ n ] ( θ ) ∥ p ≤ K ∑ j = 0 n − 1 X k j ≤ K 1 − X k . (226) \sup_{\theta\in\Theta}\left\|\nabla^\theta\bar{\psi}^{[n]}_{t+1|t}(\theta)\right\|_{p}\le K\sum_{j=0}^{n-1}\mathcal{X}_{k}^{j}\le\frac{K}{1-\mathcal{X}_{k}}. \tag*{(226)} θ ∈ Θ sup ∇ θ ψ ˉ t + 1∣ t [ n ] ( θ ) p ≤ K j = 0 ∑ n − 1 X k j ≤ 1 − X k K . (226)
The first inequality follows because the contribution generated within each block has L p L_p L p -norm bounded by K K K , while each preceding complete block contributes an additional factor X k \mathcal{X}_k X k .
By the uniform k k k -step contraction,
sup θ ∈ Θ ∥ ∇ θ ψ ˉ t + 1 ∣ t [ n ] ( θ ) − ∇ θ ψ ˉ t + 1 ∣ t ( θ ) ∥ ⟶ e . a . s . 0 , n → ∞ . \sup_{\theta\in\Theta}\left\|\nabla^\theta\bar{\psi}^{[n]}_{t+1|t}(\theta)-\nabla^\theta\bar{\psi}_{t+1|t}(\theta)\right\|\mathrel{\mathop{\longrightarrow}^{\mathrm{e.a.s.}}}0,\qquad n\to\infty. θ ∈ Θ sup ∇ θ ψ ˉ t + 1∣ t [ n ] ( θ ) − ∇ θ ψ ˉ t + 1∣ t ( θ ) ⟶ e.a.s. 0 , n → ∞.
Therefore, Fatou’s lemma and (226) imply that
E [ sup θ ∈ Θ ∥ ∇ θ ψ ˉ t + 1 ∣ t ( θ ) ∥ p ] ≤ lim inf n → ∞ E [ sup θ ∈ Θ ∥ ∇ θ ψ ˉ t + 1 ∣ t [ n ] ( θ ) ∥ p ] ≤ ( K 1 − X k ) p . \mathbb{E}\left[\sup_{\theta\in\Theta}\left\|\nabla^\theta\bar{\psi}_{t+1|t}(\theta)\right\|^{p}\right]
\le\liminf_{n\to\infty}\mathbb{E}\left[\sup_{\theta\in\Theta}\left\|\nabla^\theta\bar{\psi}^{[n]}_{t+1|t}(\theta)\right\|^{p}\right]
\le\left(\frac{K}{1-\mathcal{X}_{k}}\right)^{p}. E [ θ ∈ Θ sup ∇ θ ψ ˉ t + 1∣ t ( θ ) p ] ≤ n → ∞ lim inf E [ θ ∈ Θ sup ∇ θ ψ ˉ t + 1∣ t [ n ] ( θ ) p ] ≤ ( 1 − X k K ) p .
Since the bound is independent of t t t , we take the p p p -th root in order to get
sup t ∈ Z ∥ sup θ ∈ Θ ∥ ∇ θ ψ ˉ t + 1 ∣ t ( θ ) ∥ ∥ p ≤ K 1 − X k < ∞ . (227) \sup_{t\in\mathbb{Z}}\left\|\sup_{\theta\in\Theta}\left\|\nabla^\theta\bar{\psi}_{t+1|t}(\theta)\right\|\right\|_{p}\le\frac{K}{1-\mathcal{X}_{k}}<\infty. \tag*{(227)} t ∈ Z sup θ ∈ Θ sup ∇ θ ψ ˉ t + 1∣ t ( θ ) p ≤ 1 − X k K < ∞. (227)
Moreover, the mean value theorem and Assumption 11(iii)–11(iv) imply that, for any two filters and their respective factor inputs, the differences between the corresponding matrices X t ( θ ) X_t(\theta) X t ( θ ) and D t ( θ ) D_t(\theta) D t ( θ ) are bounded by the differences between the underlying filters and factor inputs. More precisely, let us denote with ψ t ∣ t − 1 ( 1 ) \psi^{(1)}_{t|t-1} ψ t ∣ t − 1 ( 1 ) and ψ t ∣ t − 1 ( 2 ) \psi^{(2)}_{t|t-1} ψ t ∣ t − 1 ( 2 ) two recursions being compared. Then, repeated applications of the mean value theorem, together with Assumption 11(iii)–(iv),
sup θ ∈ Θ ∥ X t ( 1 ) ( θ ) − X t ( 2 ) ( θ ) ∥ ≤ sup θ ∈ Θ sup ψ ∈ P ∥ ∂ vec X t ( θ ) ∂ ψ T ∥ sup θ ∈ Θ ∥ ψ t ∣ t − 1 ( 1 ) ( θ ) − ψ t ∣ t − 1 ( 2 ) ( θ ) ∥ + sup θ ∈ Θ ∥ ∂ vec X t ( θ ) ∂ F t T ∥ ∥ F t ( 1 ) − F t ( 2 ) ∥ , (228) \sup_{\theta\in\Theta}\left\|\mathbf{X}^{(1)}_t(\theta)-\mathbf{X}^{(2)}_t(\theta)\right\|
\leq\sup_{\theta\in\Theta}\sup_{\psi\in\mathcal{P}}\left\|\frac{\partial\operatorname{vec}X_t(\theta)}{\partial\psi^{\mathsf{T}}}\right\|\sup_{\theta\in\Theta}\left\|\psi^{(1)}_{t|t-1}(\theta)-\psi^{(2)}_{t|t-1}(\theta)\right\|+\sup_{\theta\in\Theta}\left\|\frac{\partial\operatorname{vec}X_t(\theta)}{\partial F_t^{\mathsf{T}}}\right\|\left\|F_t^{(1)}-F_t^{(2)}\right\|, \tag*{(228)} θ ∈ Θ sup X t ( 1 ) ( θ ) − X t ( 2 ) ( θ ) ≤ θ ∈ Θ sup ψ ∈ P sup ∂ ψ T ∂ vec X t ( θ ) θ ∈ Θ sup ψ t ∣ t − 1 ( 1 ) ( θ ) − ψ t ∣ t − 1 ( 2 ) ( θ ) + θ ∈ Θ sup ∂ F t T ∂ vec X t ( θ ) F t ( 1 ) − F t ( 2 ) , (228)
almost surely, and
sup θ ∈ Θ ∥ D t ( 1 ) ( θ ) − D t ( 2 ) ( θ ) ∥ ≤ sup θ ∈ Θ sup ψ ∈ P ∥ ∂ vec D t ( θ ) ∂ ψ T ∥ sup θ ∈ Θ ∥ ψ t ∣ t − 1 ( 1 ) ( θ ) − ψ t ∣ t − 1 ( 2 ) ( θ ) ∥ + sup θ ∈ Θ ∥ ∂ vec D t ( θ ) ∂ F t T ∥ ∥ F t ( 1 ) − F t ( 2 ) ∥ , (229) \sup_{\theta\in\Theta}\left\|\mathbf{D}^{(1)}_t(\theta)-\mathbf{D}^{(2)}_t(\theta)\right\|
\leq\sup_{\theta\in\Theta}\sup_{\psi\in\mathcal{P}}\left\|\frac{\partial\operatorname{vec}D_t(\theta)}{\partial\psi^{\mathsf{T}}}\right\|\sup_{\theta\in\Theta}\left\|\psi^{(1)}_{t|t-1}(\theta)-\psi^{(2)}_{t|t-1}(\theta)\right\|+\sup_{\theta\in\Theta}\left\|\frac{\partial\operatorname{vec}D_t(\theta)}{\partial F_t^{\mathsf{T}}}\right\|\left\|F_t^{(1)}-F_t^{(2)}\right\|, \tag*{(229)} θ ∈ Θ sup D t ( 1 ) ( θ ) − D t ( 2 ) ( θ ) ≤ θ ∈ Θ sup ψ ∈ P sup ∂ ψ T ∂ vec D t ( θ ) θ ∈ Θ sup ψ t ∣ t − 1 ( 1 ) ( θ ) − ψ t ∣ t − 1 ( 2 ) ( θ ) + θ ∈ Θ sup ∂ F t T ∂ vec D t ( θ ) F t ( 1 ) − F t ( 2 ) , (229)
almost surely. Taking the L p / 2 L_{p/2} L p /2 -norm on both sides and applying Minkowski’s and Hölder’s inequalities, Assumption 11(iii)–(iv) yields
∥ sup θ ∈ Θ ∥ X t ( 1 ) ( θ ) − X t ( 2 ) ( θ ) ∥ + sup θ ∈ Θ ∥ D t ( 1 ) ( θ ) − D t ( 2 ) ( θ ) ∥ ∥ p / 2 ≤ ( K X ′ + K D ′ ) ∥ sup θ ∈ Θ ∥ ψ t ∣ t − 1 ( 1 ) ( θ ) − ψ t ∣ t − 1 ( 2 ) ( θ ) ∥ ∥ p + ( K X ′ ′ + K D ′ ′ ) ∥ F t ( 1 ) − F t ( 2 ) ∥ p . (230) \left\|\sup_{\theta\in\Theta}\left\|\mathbf{X}^{(1)}_t(\theta)-\mathbf{X}^{(2)}_t(\theta)\right\|+\sup_{\theta\in\Theta}\left\|\mathbf{D}^{(1)}_t(\theta)-\mathbf{D}^{(2)}_t(\theta)\right\|\right\|_{p/2}
\leq(K'_X+K'_D)\left\|\sup_{\theta\in\Theta}\left\|\psi^{(1)}_{t|t-1}(\theta)-\psi^{(2)}_{t|t-1}(\theta)\right\|\right\|_p+(K''_X+K''_D)\left\|F_t^{(1)}-F_t^{(2)}\right\|_p. \tag*{(230)} θ ∈ Θ sup X t ( 1 ) ( θ ) − X t ( 2 ) ( θ ) + θ ∈ Θ sup D t ( 1 ) ( θ ) − D t ( 2 ) ( θ ) p /2 ≤ ( K X ′ + K D ′ ) θ ∈ Θ sup ψ t ∣ t − 1 ( 1 ) ( θ ) − ψ t ∣ t − 1 ( 2 ) ( θ ) p + ( K X ′′ + K D ′′ ) F t ( 1 ) − F t ( 2 ) p . (230)
We now apply this bound separately to the three terms on the right-hand side of (222) .
(a) By subtracting the derivative recursions associated with ψ ^ t + 1 ∣ t ( θ ) \widehat{\psi}_{t+1|t}(\theta) ψ t + 1∣ t ( θ ) and ψ ^ t + 1 ∣ t ( θ ) \widehat{\psi}_{t+1|t}(\theta) ψ t + 1∣ t ( θ ) we have
∇ θ ψ ^ t + 1 ∣ t ( θ ) − ∇ θ ψ ^ t + 1 ∣ t ( θ ) = X ^ t ( θ ) ( ∇ θ ψ ^ t ∣ t − 1 ( θ ) − ∇ θ ψ ^ t ∣ t − 1 ( θ ) ) + ( X ^ t ( θ ) − X t ( θ ) ) ∇ θ ψ ^ t ∣ t − 1 ( θ ) + D ^ t ( θ ) − D t ( θ ) . (231) \nabla^\theta\widehat{\psi}_{t+1|t}(\theta)-\nabla^\theta\widehat{\psi}_{t+1|t}(\theta)
=\widehat{X}_t(\theta)\left(\nabla^\theta\widehat{\psi}_{t|t-1}(\theta)-\nabla^\theta\widehat{\psi}_{t|t-1}(\theta)\right)+\left(\widehat{X}_t(\theta)-X_t(\theta)\right)\nabla^\theta\widehat{\psi}_{t|t-1}(\theta)+\widehat{D}_t(\theta)-D_t(\theta). \tag*{(231)} ∇ θ ψ t + 1∣ t ( θ ) − ∇ θ ψ t + 1∣ t ( θ ) = X t ( θ ) ( ∇ θ ψ t ∣ t − 1 ( θ ) − ∇ θ ψ t ∣ t − 1 ( θ ) ) + ( X t ( θ ) − X t ( θ ) ) ∇ θ ψ t ∣ t − 1 ( θ ) + D t ( θ ) − D t ( θ ) . (231)
Iterating (231) in blocks of length k k k , and using the expansions in (228) and (229) , together with Assumption 7, we obtain
sup θ ∈ Θ ∥ ∇ θ ψ ^ t + 1 ∣ t ( θ ) − ∇ θ ψ ^ t + 1 ∣ t ( θ ) ∥ ≤ χ k ⌊ t / k ⌋ sup θ ∈ Θ ∥ ∇ θ ψ 1 ∣ 0 ( θ ) − ∇ θ ψ 1 ∣ 0 ⋆ ( θ ) ∥ + ∑ s = 1 t χ k ⌊ ( t − s ) / k ⌋ C s , N T ψ sup θ ∈ Θ ∥ ψ ^ s ∣ s − 1 ( θ ) − ψ ^ s ∣ s − 1 ( θ ) ∥ + ∑ s = 1 t χ k ⌊ ( t − s ) / k ⌋ C s , N T F ∥ F ^ s − H ^ F s ∥ , (232) \sup_{\theta\in\Theta}\left\|\nabla^\theta\widehat{\psi}_{t+1|t}(\theta)-\nabla^\theta\widehat{\psi}_{t+1|t}(\theta)\right\|
\leq\chi_k^{\lfloor t/k\rfloor}\sup_{\theta\in\Theta}\left\|\nabla^\theta\psi_{1|0}(\theta)-\nabla^\theta\psi^\star_{1|0}(\theta)\right\|
+\sum_{s=1}^{t}\chi_k^{\lfloor(t-s)/k\rfloor}C^\psi_{s,NT}\sup_{\theta\in\Theta}\left\|\widehat{\psi}_{s|s-1}(\theta)-\widehat{\psi}_{s|s-1}(\theta)\right\|
+\sum_{s=1}^{t}\chi_k^{\lfloor(t-s)/k\rfloor}C^F_{s,NT}\left\|\widehat{F}_s-\widehat{H}F_s\right\|, \tag*{(232)} θ ∈ Θ sup ∇ θ ψ t + 1∣ t ( θ ) − ∇ θ ψ t + 1∣ t ( θ ) ≤ χ k ⌊ t / k ⌋ θ ∈ Θ sup ∇ θ ψ 1∣0 ( θ ) − ∇ θ ψ 1∣0 ⋆ ( θ ) + s = 1 ∑ t χ k ⌊( t − s ) / k ⌋ C s , N T ψ θ ∈ Θ sup ψ s ∣ s − 1 ( θ ) − ψ s ∣ s − 1 ( θ ) + s = 1 ∑ t χ k ⌊( t − s ) / k ⌋ C s , N T F F s − H F s , (232)
where
C s , N T ψ = sup θ ∈ Θ sup ψ ∈ Ψ ∥ ∂ vec X s ( θ ) ∂ ψ T ∥ sup θ ∈ Θ ∥ ∇ θ ψ ^ s ∣ s − 1 ( θ ) ∥ + sup θ ∈ Θ sup ψ ∈ Ψ ∥ ∂ vec D s ( θ ) ∂ ψ T ∥ , \mathcal{C}^{\psi}_{s,NT}=\sup_{\theta\in\Theta}\sup_{\psi\in\Psi}\left\lVert\frac{\partial\operatorname{vec}X_s(\theta)}{\partial\psi^{\mathsf{T}}}\right\rVert\sup_{\theta\in\Theta}\left\lVert\nabla_{\theta}\widehat{\psi}_{s\mid s-1}(\theta)\right\rVert+\sup_{\theta\in\Theta}\sup_{\psi\in\Psi}\left\lVert\frac{\partial\operatorname{vec}D_s(\theta)}{\partial\psi^{\mathsf{T}}}\right\rVert, C s , N T ψ = θ ∈ Θ sup ψ ∈ Ψ sup ∂ ψ T ∂ vec X s ( θ ) θ ∈ Θ sup ∇ θ ψ s ∣ s − 1 ( θ ) + θ ∈ Θ sup ψ ∈ Ψ sup ∂ ψ T ∂ vec D s ( θ ) ,
and
C s , N T F = sup θ ∈ Θ ∥ ∂ vec X s ( θ ) ∂ F s T ∥ sup θ ∈ Θ ∥ ∇ θ ψ ^ s ∣ s − 1 ( θ ) ∥ + sup θ ∈ Θ ∥ ∂ vec D s ( θ ) ∂ F s T ∥ . \mathcal{C}^{F}_{s,NT}=\sup_{\theta\in\Theta}\left\lVert\frac{\partial\operatorname{vec}X_s(\theta)}{\partial F_s^{\mathsf{T}}}\right\rVert\sup_{\theta\in\Theta}\left\lVert\nabla_{\theta}\widehat{\psi}_{s\mid s-1}(\theta)\right\rVert+\sup_{\theta\in\Theta}\left\lVert\frac{\partial\operatorname{vec}D_s(\theta)}{\partial F_s^{\mathsf{T}}}\right\rVert. C s , N T F = θ ∈ Θ sup ∂ F s T ∂ vec X s ( θ ) θ ∈ Θ sup ∇ θ ψ s ∣ s − 1 ( θ ) + θ ∈ Θ sup ∂ F s T ∂ vec D s ( θ ) .
Since both ψ 1 ∣ 0 \psi_{1\mid0} ψ 1 ∣ 0 and ψ 1 ∣ 0 ∗ \psi^*_{1\mid0} ψ 1 ∣ 0 ∗ are fixed and do not depend on θ \theta θ , ∇ θ ψ 1 ∣ 0 = ∇ θ ψ 1 ∣ 0 ∗ = 0 r × d \nabla^\theta\psi_{1\mid0}=\nabla^\theta\psi^*_{1\mid0}=\mathbf{0}_{r\times d} ∇ θ ψ 1 ∣ 0 = ∇ θ ψ 1 ∣ 0 ∗ = 0 r × d . Hence, the first term on the right-hand side of (232) is identically zero. The initialization effect remaining in the derivative process is instead inherited from the difference between the initializations of the underlying filters through ψ ^ s ∣ s − 1 ( θ ) − ψ ^ s ∣ s − 1 ∗ ( θ ) \widehat{\psi}_{s\mid s-1}(\theta)-\widehat{\psi}^*_{s\mid s-1}(\theta) ψ s ∣ s − 1 ( θ ) − ψ s ∣ s − 1 ∗ ( θ ) .
By the same k k k -step moment argument used to establish (227) , applied to ∇ θ ψ ^ s ∣ s − 1 ( θ ) \nabla^\theta\widehat{\psi}_{s\mid s-1}(\theta) ∇ θ ψ s ∣ s − 1 ( θ ) , together with Assumption 11( i i i ) (iii) ( iii ) –( i v ) (iv) ( i v ) , it holds that, for every fixed t t t and every M > 0 M>0 M > 0 ,
lim M → ∞ lim sup N , T → ∞ P ( max 1 ≤ s ≤ t C s , N T ψ > M ) = 0 , (233) \lim_{M\to\infty}\limsup_{N,T\to\infty}\mathbb{P}\left(\max_{1\le s\le t}\mathcal{C}^{\psi}_{s,NT}>M\right)=0, \tag*{(233)} M → ∞ lim N , T → ∞ lim sup P ( 1 ≤ s ≤ t max C s , N T ψ > M ) = 0 , (233)
lim M → ∞ lim sup N , T → ∞ P ( max 1 ≤ s ≤ t C s , N T F > M ) = 0. (234) \lim_{M\to\infty}\limsup_{N,T\to\infty}\mathbb{P}\left(\max_{1\le s\le t}\mathcal{C}^{F}_{s,NT}>M\right)=0. \tag*{(234)} M → ∞ lim N , T → ∞ lim sup P ( 1 ≤ s ≤ t max C s , N T F > M ) = 0. (234)
To control the contribution of ψ ^ s ∣ s − 1 ( θ ) − ψ ^ s ∣ s − 1 ∗ ( θ ) \widehat{\psi}_{s\mid s-1}(\theta)-\widehat{\psi}^*_{s\mid s-1}(\theta) ψ s ∣ s − 1 ( θ ) − ψ s ∣ s − 1 ∗ ( θ ) , recall from (60) that, almost surely,
sup θ ∈ Θ ∥ ψ ^ s ∣ s − 1 ( θ ) − ψ ^ s ∣ s − 1 ∗ ( θ ) ∥ ≤ χ k ⌊ ( s − 1 ) / k ⌋ Δ 1 + ∑ j = 1 s − 1 χ k ⌊ ( j − 1 ) / k ⌋ sup θ ∈ Θ ∥ Y s − j ( θ ) ∥ ∥ F ^ s − j − H ^ F s − j ∥ , (235) \sup_{\theta\in\Theta}\left\lVert\widehat{\psi}_{s\mid s-1}(\theta)-\widehat{\psi}^*_{s\mid s-1}(\theta)\right\rVert
\le\chi_k^{\lfloor(s-1)/k\rfloor}\Delta_1+\sum_{j=1}^{s-1}\chi_k^{\lfloor(j-1)/k\rfloor}\sup_{\theta\in\Theta}\left\lVert Y_{s-j}(\theta)\right\rVert\left\lVert\widehat{F}_{s-j}-\widehat{H}F_{s-j}\right\rVert, \tag*{(235)} θ ∈ Θ sup ψ s ∣ s − 1 ( θ ) − ψ s ∣ s − 1 ∗ ( θ ) ≤ χ k ⌊( s − 1 ) / k ⌋ Δ 1 + j = 1 ∑ s − 1 χ k ⌊( j − 1 ) / k ⌋ θ ∈ Θ sup ∥ Y s − j ( θ ) ∥ F s − j − H F s − j , (235)
where Δ 1 = ∥ ψ 1 ∣ 0 − ψ 1 ∣ 0 ∗ ∥ < ∞ \Delta_1=\left\lVert\psi_{1\mid0}-\psi^*_{1\mid0}\right\rVert<\infty Δ 1 = ψ 1 ∣ 0 − ψ 1 ∣ 0 ∗ < ∞ . Substituting (235) into the corresponding term of (232)
∑ s = 1 t χ k ⌊ ( t − s ) / k ⌋ C s , N T ψ sup θ ∈ Θ ∥ ψ ^ s ∣ s − 1 ( θ ) − ψ ^ s ∣ s − 1 ∗ ( θ ) ∥ ≤ Δ 1 ∑ s = 1 t χ k ⌊ ( t − s ) / k ⌋ + ⌊ ( s − 1 ) / k ⌋ C s , N T ψ + ∑ s = 1 t ∑ j = 1 s − 1 χ k ⌊ ( t − s ) / k ⌋ + ⌊ ( j − 1 ) / k ⌋ C s , N T ψ sup θ ∈ Θ ∥ Y s − j ( θ ) ∥ ∥ F ^ s − j − H ^ F s − j ∥ . (236) \begin{aligned}
\sum_{s=1}^{t}\chi_k^{\lfloor(t-s)/k\rfloor}\mathcal{C}^{\psi}_{s,NT}\sup_{\theta\in\Theta}\left\lVert\widehat{\psi}_{s\mid s-1}(\theta)-\widehat{\psi}^*_{s\mid s-1}(\theta)\right\rVert
&\le\Delta_1\sum_{s=1}^{t}\chi_k^{\lfloor(t-s)/k\rfloor+\lfloor(s-1)/k\rfloor}\mathcal{C}^{\psi}_{s,NT}\\
&\quad+\sum_{s=1}^{t}\sum_{j=1}^{s-1}\chi_k^{\lfloor(t-s)/k\rfloor+\lfloor(j-1)/k\rfloor}\mathcal{C}^{\psi}_{s,NT}\sup_{\theta\in\Theta}\left\lVert Y_{s-j}(\theta)\right\rVert\left\lVert\widehat{F}_{s-j}-\widehat{H}F_{s-j}\right\rVert. \tag*{(236)}
\end{aligned} s = 1 ∑ t χ k ⌊( t − s ) / k ⌋ C s , N T ψ θ ∈ Θ sup ψ s ∣ s − 1 ( θ ) − ψ s ∣ s − 1 ∗ ( θ ) ≤ Δ 1 s = 1 ∑ t χ k ⌊( t − s ) / k ⌋ + ⌊( s − 1 ) / k ⌋ C s , N T ψ + s = 1 ∑ t j = 1 ∑ s − 1 χ k ⌊( t − s ) / k ⌋ + ⌊( j − 1 ) / k ⌋ C s , N T ψ θ ∈ Θ sup ∥ Y s − j ( θ ) ∥ F s − j − H F s − j . (236)
For the first term on the right-hand side of (236) , the same k k k -step argument used to obtain (227) , together with
Assumption 11(iii)–(iv), implies
sup N , T sup 1 ≤ s ≤ T ∥ sup θ ∈ Θ ∥ ∇ θ ψ ^ s ∣ s − 1 ( θ ) ∥ ∥ p ≤ K 1 − χ k < ∞ . \sup_{N,T}\sup_{1\le s\le T}\left\|\sup_{\theta\in\Theta}\left\|\nabla_\theta\widehat{\psi}_{s\mid s-1}(\theta)\right\|\right\|_p \le\frac{K}{1-\chi_k}<\infty. N , T sup 1 ≤ s ≤ T sup θ ∈ Θ sup ∇ θ ψ s ∣ s − 1 ( θ ) p ≤ 1 − χ k K < ∞.
Consequently, Hölder’s inequality and Assumption 11(iii)–(iv) imply that, with q = p / 2 > 1 q=p/2>1 q = p /2 > 1 ,
max { sup N , T sup 1 ≤ s ≤ T ∥ C s , N T ψ ∥ q , sup N , T sup 1 ≤ s ≤ T ∥ C s , N T F ∥ q } ≤ K 1 − χ k max { K X ′ , K X ′ ′ } + max { K D ′ ′ , K D ′ ′ ′ } < ∞ . (237) \max\left\{\sup_{N,T}\sup_{1\le s\le T}\left\|\mathcal{C}^{\psi}_{s,NT}\right\|_q,\sup_{N,T}\sup_{1\le s\le T}\left\|\mathcal{C}^{F}_{s,NT}\right\|_q\right\}
\le\frac{K}{1-\chi_k}\max\left\{K'_X,K''_X\right\}+\max\left\{K''_D,K'''_D\right\}<\infty. \tag*{(237)} max { N , T sup 1 ≤ s ≤ T sup C s , N T ψ q , N , T sup 1 ≤ s ≤ T sup C s , N T F q } ≤ 1 − χ k K max { K X ′ , K X ′′ } + max { K D ′′ , K D ′′′ } < ∞. (237)
Thus, the required uniform moment bound holds with q = p / 2 > 1 q=p/2>1 q = p /2 > 1 . Therefore, for every ε > 0 \varepsilon>0 ε > 0 , the Markov’s inequality implies
∑ s = 1 ∞ P ( C s , N T ψ > s 1 / q + ε ) ≤ sup N , T sup 1 ≤ s ≤ T E [ ( C s , N T ψ ) q ] ∑ s = 1 ∞ s − 1 − q ε < ∞ , \sum_{s=1}^{\infty}\mathbb{P}\left(\mathcal{C}^{\psi}_{s,NT}>s^{1/q+\varepsilon}\right)\le\sup_{N,T}\sup_{1\le s\le T}\mathbb{E}\left[\left(\mathcal{C}^{\psi}_{s,NT}\right)^q\right]\sum_{s=1}^{\infty}s^{-1-q\varepsilon}<\infty, s = 1 ∑ ∞ P ( C s , N T ψ > s 1/ q + ε ) ≤ N , T sup 1 ≤ s ≤ T sup E [ ( C s , N T ψ ) q ] s = 1 ∑ ∞ s − 1 − q ε < ∞ ,
and thus, the Borel–Cantelli lemma gives C s , N T ψ = O a . s . ( s 1 / q + ε ) \mathcal{C}^{\psi}_{s,NT}=O_{a.s.}\left(s^{1/q+\varepsilon}\right) C s , N T ψ = O a . s . ( s 1/ q + ε ) . Moreover, by using similar arguments as in (66) in the proof of Proposition 2(a),
Δ 1 ∑ s = 1 t χ k ⌊ ( t − s ) / k ⌋ + ⌊ ( s − 1 ) / k ⌋ C s , N T ψ = O a . s . ( t 1 + 1 / q + ε χ k ⌊ ( t − 1 ) / k ⌋ − 1 ) = O a . s . ( χ k ⌊ t / ( 2 k ) ⌋ ) . (238) \Delta_1\sum_{s=1}^{t}\chi_k^{\lfloor(t-s)/k\rfloor+\lfloor(s-1)/k\rfloor}\mathcal{C}^{\psi}_{s,NT}
=O_{a.s.}\left(t^{1+1/q+\varepsilon}\chi_k^{\lfloor(t-1)/k\rfloor-1}\right)=O_{a.s.}\left(\chi_k^{\lfloor t/(2k)\rfloor}\right). \tag*{(238)} Δ 1 s = 1 ∑ t χ k ⌊( t − s ) / k ⌋ + ⌊( s − 1 ) / k ⌋ C s , N T ψ = O a . s . ( t 1 + 1/ q + ε χ k ⌊( t − 1 ) / k ⌋ − 1 ) = O a . s . ( χ k ⌊ t / ( 2 k )⌋ ) . (238)
Now, for the second term on the right-hand side of (236) , we have
∑ s = 1 t ∑ j = 1 s − 1 χ k ⌊ ( t − s ) / k ⌋ + ⌊ ( j − 1 ) / k ⌋ C s , N T ψ sup θ ∈ Θ ∥ Y s − j ( θ ) ∥ ∥ F ^ s − j − H ^ F ^ s − j ∥ ≤ ( k 1 − χ k ) 2 max 1 ≤ s ≤ t C s , N T ψ max 1 ≤ u ≤ t sup θ ∈ Θ ∣ Y u ( θ ) ∣ max 1 ≤ u ≤ t ∥ F ^ u − H ^ F ^ u ∥ . \sum_{s=1}^{t}\sum_{j=1}^{s-1}\chi_k^{\lfloor(t-s)/k\rfloor+\lfloor(j-1)/k\rfloor}\mathcal{C}^{\psi}_{s,NT}\sup_{\theta\in\Theta}\left\|Y_{s-j}(\theta)\right\|\left\|\widehat{F}_{s-j}-\widehat{H}\widehat{F}_{s-j}\right\|
\le\left(\frac{k}{1-\chi_k}\right)^2\max_{1\le s\le t}\mathcal{C}^{\psi}_{s,NT}\max_{1\le u\le t}\sup_{\theta\in\Theta}\left|Y_u(\theta)\right|\max_{1\le u\le t}\left\|\widehat{F}_u-\widehat{H}\widehat{F}_u\right\|. s = 1 ∑ t j = 1 ∑ s − 1 χ k ⌊( t − s ) / k ⌋ + ⌊( j − 1 ) / k ⌋ C s , N T ψ θ ∈ Θ sup ∥ Y s − j ( θ ) ∥ F s − j − H F s − j ≤ ( 1 − χ k k ) 2 1 ≤ s ≤ t max C s , N T ψ 1 ≤ u ≤ t max θ ∈ Θ sup ∣ Y u ( θ ) ∣ 1 ≤ u ≤ t max F u − H F u .
The first two maxima are tight by Assumptions 7 and 11, while Proposition 1(a) and the finite-maximum argument give max 1 ≤ u ≤ t ∥ F ^ u − H ^ F ^ u ∥ = O P ( 1 min ( N , T ) ) \max_{1\le u\le t}\left\|\widehat{F}_u-\widehat{H}\widehat{F}_u\right\|=O_P\left(\frac{1}{\min(\sqrt{N},T)}\right) max 1 ≤ u ≤ t F u − H F u = O P ( m i n ( N , T ) 1 ) . Consequently,
∑ s = 1 t ∑ j = 1 s − 1 χ k ⌊ ( t − s ) / k ⌋ + ⌊ ( j − 1 ) / k ⌋ C s , N T ψ sup θ ∈ Θ ∥ Y s − j ( θ ) ∥ ∥ F ^ s − j − H ^ F ^ s − j ∥ = O P ( 1 min ( N , T ) ) . (239) \begin{aligned}
\sum_{s=1}^{t}\sum_{j=1}^{s-1}\chi_k^{\lfloor(t-s)/k\rfloor+\lfloor(j-1)/k\rfloor}\mathcal{C}^{\psi}_{s,NT}\sup_{\theta\in\Theta}\left\|Y_{s-j}(\theta)\right\|\left\|\widehat{F}_{s-j}-\widehat{H}\widehat{F}_{s-j}\right\| \\
&= O_{\mathbb{P}}\left(\frac{1}{\min\left(\sqrt{N},T\right)}\right). \tag*{(239)}
\end{aligned} s = 1 ∑ t j = 1 ∑ s − 1 χ k ⌊( t − s ) / k ⌋ + ⌊( j − 1 ) / k ⌋ C s , N T ψ θ ∈ Θ sup ∥ Y s − j ( θ ) ∥ F s − j − H F s − j = O P min ( N , T ) 1 . (239)
It remains to control the third term in the right-hand side of (232) . The finite-maximum argument used in the proof of Proposition 2(a) gives, for every L > 0 L>0 L > 0 ,
lim L → ∞ lim sup N , T → ∞ P ( min ( N , T ) max 1 ≤ s ≤ t ∥ F ^ s − H ^ F s ∥ > L ) = 0. (240) \lim_{L\to\infty}\limsup_{N,T\to\infty}\mathbb{P}\left(\min\left(\sqrt{N},T\right)\max_{1\leq s\leq t}\left\|\widehat{F}_s-\widehat{H}F_s\right\|>L\right)=0. \tag*{(240)} L → ∞ lim N , T → ∞ lim sup P ( min ( N , T ) 1 ≤ s ≤ t max F s − H F s > L ) = 0. (240)
Moreover, almost surely,
∑ s = 1 t χ k ⌊ ( t − s ) / k ⌋ c s , N T F ∥ F ^ s − H ^ F s ∥ ≤ k 1 − χ k max 1 ≤ s ≤ t c s , N T F max 1 ≤ s ≤ t ∥ F ^ s − H ^ F s ∥ . \sum_{s=1}^{t}\chi_k^{\lfloor(t-s)/k\rfloor}c_{s,NT}^{F}\left\|\widehat{F}_s-\widehat{H}F_s\right\|\leq\frac{k}{1-\chi_k}\max_{1\leq s\leq t}c_{s,NT}^{F}\max_{1\leq s\leq t}\left\|\widehat{F}_s-\widehat{H}F_s\right\|. s = 1 ∑ t χ k ⌊( t − s ) / k ⌋ c s , N T F F s − H F s ≤ 1 − χ k k 1 ≤ s ≤ t max c s , N T F 1 ≤ s ≤ t max F s − H F s .
Therefore, for every M , L > 0 M,L>0 M , L > 0 ,
P ( ∑ s = 1 t χ k ⌊ ( t − s ) / k ⌋ c s , N T F ∥ F ^ s − H ^ F s ∥ > k M L ( 1 − χ k ) min ( N , T ) ) ≤ P ( max 1 ≤ s ≤ t c s , N T F > M ) + P ( min ( N , T ) max 1 ≤ s ≤ t ∥ F ^ s − H ^ F s ∥ > L ) . \begin{aligned}
\mathbb{P}\left(\sum_{s=1}^{t}\chi_k^{\lfloor(t-s)/k\rfloor}c_{s,NT}^{F}\left\|\widehat{F}_s-\widehat{H}F_s\right\|>\frac{kML}{(1-\chi_k)\min\left(\sqrt{N},T\right)}\right) \\
&\leq\mathbb{P}\left(\max_{1\leq s\leq t}c_{s,NT}^{F}>M\right)+\mathbb{P}\left(\min\left(\sqrt{N},T\right)\max_{1\leq s\leq t}\left\|\widehat{F}_s-\widehat{H}F_s\right\|>L\right).
\end{aligned} P s = 1 ∑ t χ k ⌊( t − s ) / k ⌋ c s , N T F F s − H F s > ( 1 − χ k ) min ( N , T ) k M L ≤ P ( 1 ≤ s ≤ t max c s , N T F > M ) + P ( min ( N , T ) 1 ≤ s ≤ t max F s − H F s > L ) .
Letting N , T → ∞ N,T\to\infty N , T → ∞ , followed by M , L → ∞ M,L\to\infty M , L → ∞ , and using (234) –(240) ,
∑ s = 1 t χ k ⌊ ( t − s ) / k ⌋ c s , N T F ∥ F ^ s − H ^ F s ∥ = O P ( 1 min ( N , T ) ) . (241) \sum_{s=1}^{t}\chi_k^{\lfloor(t-s)/k\rfloor}c_{s,NT}^{F}\left\|\widehat{F}_s-\widehat{H}F_s\right\|=O_{\mathbb{P}}\left(\frac{1}{\min\left(\sqrt{N},T\right)}\right). \tag*{(241)} s = 1 ∑ t χ k ⌊( t − s ) / k ⌋ c s , N T F F s − H F s = O P min ( N , T ) 1 . (241)
Combining (238) , (239) , and (241) , and substituting these bounds into (232) ,
sup θ ∈ Θ ∥ ∇ θ ψ ^ t + 1 ∣ t ( θ ) − ∇ θ ψ ~ t + 1 ∣ t ( θ ) ∥ = O P ( 1 min ( N , T ) ) + O a . s . ( χ k ⌊ t / ( 2 k ) ⌋ ) , \sup_{\theta\in\Theta}\left\|\nabla^\theta\widehat{\psi}_{t+1|t}(\theta)-\nabla^\theta\widetilde{\psi}_{t+1|t}(\theta)\right\|=O_{\mathbb{P}}\left(\frac{1}{\min\left(\sqrt{N},T\right)}\right)+O_{\mathrm{a.s.}}\left(\chi_k^{\lfloor t/(2k)\rfloor}\right), θ ∈ Θ sup ∇ θ ψ t + 1∣ t ( θ ) − ∇ θ ψ t + 1∣ t ( θ ) = O P min ( N , T ) 1 + O a.s. ( χ k ⌊ t / ( 2 k )⌋ ) ,
which proves part ( a ) (a) ( a ) .
( b ) (b) ( b ) The filters ψ ^ t ∣ t − 1 ( θ ) \widehat{\psi}_{t|t-1}(\theta) ψ t ∣ t − 1 ( θ ) and ψ ~ t ∣ t − 1 ( θ ) \widetilde{\psi}_{t|t-1}(\theta) ψ t ∣ t − 1 ( θ ) have the same fixed initialization ψ 1 ∣ 0 ⋆ \psi^\star_{1|0} ψ 1∣0 ⋆ and are driven, respectively, by H ^ F t \widehat{H}F_t H F t and H 0 F ~ t H_0\widetilde{F}_t H 0 F t . Subtracting their derivative recursions
∇ θ ψ ^ t + 1 ∣ t ( θ ) − ∇ θ ψ ~ t + 1 ∣ t ( θ ) = X ^ t ( θ ) ( ∇ θ ψ ^ t ∣ t − 1 ( θ ) − ∇ θ ψ ~ t ∣ t − 1 ( θ ) ) + ( X ^ t ( θ ) − X ~ t ( θ ) ) ∇ θ ψ ~ t ∣ t − 1 ( θ ) + D ^ t ( θ ) − D ~ t ( θ ) . (242) \nabla^\theta\widehat{\psi}_{t+1|t}(\theta)-\nabla^\theta\widetilde{\psi}_{t+1|t}(\theta)
=\widehat{X}_t(\theta)\left(\nabla^\theta\widehat{\psi}_{t|t-1}(\theta)-\nabla^\theta\widetilde{\psi}_{t|t-1}(\theta)\right)+\left(\widehat{X}_t(\theta)-\widetilde{X}_t(\theta)\right)\nabla^\theta\widetilde{\psi}_{t|t-1}(\theta)+\widehat{D}_t(\theta)-\widetilde{D}_t(\theta). \tag*{(242)} ∇ θ ψ t + 1∣ t ( θ ) − ∇ θ ψ t + 1∣ t ( θ ) = X t ( θ ) ( ∇ θ ψ t ∣ t − 1 ( θ ) − ∇ θ ψ t ∣ t − 1 ( θ ) ) + ( X t ( θ ) − X t ( θ ) ) ∇ θ ψ t ∣ t − 1 ( θ ) + D t ( θ ) − D t ( θ ) . (242)
Therefore, as in point (a), iterating (242) in blocks of length k k k , and using the expansions in (228) and (229) , together with Assumption 7,
sup θ ∈ Θ ∥ ∇ θ ψ ^ t + 1 ∣ t ( θ ) − ∇ θ ψ ~ t + 1 ∣ t ( θ ) ∥ ≤ ∑ s = 1 t χ k ⌊ ( t − s ) / k ⌋ c s , N T ψ sup θ ∈ Θ ∥ ψ ^ s ∣ s − 1 ( θ ) − ψ ~ s ∣ s − 1 ( θ ) ∥ + ∑ s = 1 t χ k ⌊ ( t − s ) / k ⌋ c s , N T F ∥ H ^ F s − H 0 F ‾ s ∥ , (243) \sup_{\theta\in\Theta}\left\|\nabla^\theta\widehat{\psi}_{t+1|t}(\theta)-\nabla^\theta\widetilde{\psi}_{t+1|t}(\theta)\right\|
\le\sum_{s=1}^{t}\chi_k^{\lfloor(t-s)/k\rfloor}c^\psi_{s,NT}\sup_{\theta\in\Theta}\left\|\widehat{\psi}_{s|s-1}(\theta)-\widetilde{\psi}_{s|s-1}(\theta)\right\|+\sum_{s=1}^{t}\chi_k^{\lfloor(t-s)/k\rfloor}c^F_{s,NT}\left\|\widehat{H}F_s-H_0\overline{F}_s\right\|, \tag*{(243)} θ ∈ Θ sup ∇ θ ψ t + 1∣ t ( θ ) − ∇ θ ψ t + 1∣ t ( θ ) ≤ s = 1 ∑ t χ k ⌊( t − s ) / k ⌋ c s , N T ψ θ ∈ Θ sup ψ s ∣ s − 1 ( θ ) − ψ s ∣ s − 1 ( θ ) + s = 1 ∑ t χ k ⌊( t − s ) / k ⌋ c s , N T F H F s − H 0 F s , (243)
where c s , N T ψ c^\psi_{s,NT} c s , N T ψ and c s , N T F c^F_{s,NT} c s , N T F are defined as in part (a), with ∇ θ ψ ^ s ∣ s − 1 ( θ ) \nabla^\theta\widehat{\psi}_{s|s-1}(\theta) ∇ θ ψ s ∣ s − 1 ( θ ) replaced by ∇ θ ψ ~ s ∣ s − 1 ( θ ) \nabla^\theta\widetilde{\psi}_{s|s-1}(\theta) ∇ θ ψ s ∣ s − 1 ( θ ) . By the same argument used in (237) , with q = p / 2 > 1 q=p/2>1 q = p /2 > 1 ,
max { sup N , T 1 ≤ s ≤ T ∥ c s , N T ψ ∥ q , sup N , T 1 ≤ s ≤ T ∥ c s , N T F ∥ q } < ∞ . (244) \max\left\{\sup_{N,T\ 1\le s\le T}\left\|c^\psi_{s,NT}\right\|_q,\sup_{N,T\ 1\le s\le T}\left\|c^F_{s,NT}\right\|_q\right\}<\infty. \tag*{(244)} max { N , T 1 ≤ s ≤ T sup c s , N T ψ q , N , T 1 ≤ s ≤ T sup c s , N T F q } < ∞. (244)
Consequently, for every ε > 0 \varepsilon>0 ε > 0 , Markov’s inequality and the Borel–Cantelli lemma imply that
max { c s , N T ψ , c s , N T F } = O a . s . ( s 1 / q + ε ) . (245) \max\left\{c^\psi_{s,NT},c^F_{s,NT}\right\}=O_{a.s.}\left(s^{1/q+\varepsilon}\right). \tag*{(245)} max { c s , N T ψ , c s , N T F } = O a . s . ( s 1/ q + ε ) . (245)
We first consider the first term on the right-hand side of (243) . From the recursive bound used in the proof of Proposition 2(b) we get, almost surely,
sup θ ∈ Θ ∥ ψ ^ s ∣ s − 1 ( θ ) − ψ ~ s ∣ s − 1 ( θ ) ∥ ≤ ∑ j = 1 s − 1 χ k ⌊ ( j − 1 ) / k ⌋ sup θ ∈ Θ ∣ Y s − j ( θ ) ∣ ∥ ( H ^ − H 0 ) F s − j ∥ + ∑ j = 1 s − 1 χ k ⌊ ( j − 1 ) / k ⌋ sup θ ∈ Θ ∣ Y s − j ( θ ) ∣ ∥ H 0 ( F s − j − F ‾ s − j ) ∥ . (246) \sup_{\theta\in\Theta}\left\|\widehat{\psi}_{s|s-1}(\theta)-\widetilde{\psi}_{s|s-1}(\theta)\right\|
\le\sum_{j=1}^{s-1}\chi_k^{\lfloor(j-1)/k\rfloor}\sup_{\theta\in\Theta}\left|Y_{s-j}(\theta)\right|\left\|(\widehat{H}-H_0)F_{s-j}\right\|
+\sum_{j=1}^{s-1}\chi_k^{\lfloor(j-1)/k\rfloor}\sup_{\theta\in\Theta}\left|Y_{s-j}(\theta)\right|\left\|H_0(F_{s-j}-\overline{F}_{s-j})\right\|. \tag*{(246)} θ ∈ Θ sup ψ s ∣ s − 1 ( θ ) − ψ s ∣ s − 1 ( θ ) ≤ j = 1 ∑ s − 1 χ k ⌊( j − 1 ) / k ⌋ θ ∈ Θ sup ∣ Y s − j ( θ ) ∣ ( H − H 0 ) F s − j + j = 1 ∑ s − 1 χ k ⌊( j − 1 ) / k ⌋ θ ∈ Θ sup ∣ Y s − j ( θ ) ∣ H 0 ( F s − j − F s − j ) . (246)
Substituting (246) into the first term on the right-hand side of (243) gives the sum of the following two terms:
I t = ∑ s = 1 t ∑ j = 1 s − 1 χ k ⌊ ( t − s ) / k ⌋ + ⌊ ( j − 1 ) / k ⌋ c s , N T ψ sup θ ∈ Θ ∣ Y s − j ( θ ) ∣ ∥ ( H ^ − H 0 ) F s − j ∥ , (247) I_t=\sum_{s=1}^{t}\sum_{j=1}^{s-1}\chi_k^{\lfloor(t-s)/k\rfloor+\lfloor(j-1)/k\rfloor}c^\psi_{s,NT}\sup_{\theta\in\Theta}\left|Y_{s-j}(\theta)\right|\left\|(\widehat{H}-H_0)F_{s-j}\right\|, \tag*{(247)} I t = s = 1 ∑ t j = 1 ∑ s − 1 χ k ⌊( t − s ) / k ⌋ + ⌊( j − 1 ) / k ⌋ c s , N T ψ θ ∈ Θ sup ∣ Y s − j ( θ ) ∣ ( H − H 0 ) F s − j , (247)
I I t = ∑ s = 1 t χ k ⌊ ( t − s ) / k ⌋ c s , N T ψ ∑ j = 1 s − 1 χ k ⌊ ( j − 1 ) / k ⌋ sup θ ∈ Θ ∣ Y s − j ( θ ) ∣ ∥ H 0 ( F s − j − F ‾ s − j ) ∥ . (248) II_t=\sum_{s=1}^{t}\chi_k^{\lfloor(t-s)/k\rfloor}c^\psi_{s,NT}\sum_{j=1}^{s-1}\chi_k^{\lfloor(j-1)/k\rfloor}\sup_{\theta\in\Theta}\left|Y_{s-j}(\theta)\right|\left\|H_0(F_{s-j}-\overline{F}_{s-j})\right\|. \tag*{(248)} I I t = s = 1 ∑ t χ k ⌊( t − s ) / k ⌋ c s , N T ψ j = 1 ∑ s − 1 χ k ⌊( j − 1 ) / k ⌋ θ ∈ Θ sup ∣ Y s − j ( θ ) ∣ H 0 ( F s − j − F s − j ) . (248)
For (247) , geometric summability implies, almost surely,
I t ≤ ( k 1 − χ k ) 2 max 1 ≤ s ≤ t c s , N T ψ ∣ max 1 ≤ u ≤ t sup θ ∈ Θ ∣ Y u ( θ ) ∣ ∣ max 1 ≤ u ≤ t ∥ ( H ^ − H 0 ) F u ∥ . I_t\le\left(\frac{k}{1-\chi_k}\right)^2\max_{1\le s\le t}c^\psi_{s,NT}\left|\max_{1\le u\le t}\sup_{\theta\in\Theta}|Y_u(\theta)|\right|\max_{1\le u\le t}\left\|(\widehat{H}-H_0)F_u\right\|. I t ≤ ( 1 − χ k k ) 2 1 ≤ s ≤ t max c s , N T ψ 1 ≤ u ≤ t max θ ∈ Θ sup ∣ Y u ( θ ) ∣ 1 ≤ u ≤ t max ( H − H 0 ) F u .
Proposition 1 and the finite-maximum argument imply
max 1 ≤ u ≤ t ∥ ( H ^ − H 0 ) F u ∥ = O P ( 1 min ( N , T ) ) . (249) \max_{1\leq u\leq t}\left\lVert\left(\widehat{H}-H_0\right)F_u\right\rVert=O_{\mathbb{P}}\left(\frac{1}{\min\left(\sqrt{N},\sqrt{T}\right)}\right). \tag*{(249)} 1 ≤ u ≤ t max ( H − H 0 ) F u = O P min ( N , T ) 1 . (249)
The other two maxima are tight by (244) and Assumption 7. Hence, by applying the probability argument used in (234) –(241)
I t = O P ( 1 min ( N , T ) ) . (250) I_t=O_{\mathbb{P}}\left(\frac{1}{\min\left(\sqrt{N},\sqrt{T}\right)}\right). \tag*{(250)} I t = O P min ( N , T ) 1 . (250)
For (248) , the bound established in (67) , applied with t = s − 1 t=s-1 t = s − 1 , implies that
∑ j = 1 s − 1 χ k ⌊ ( j − 1 ) / k ⌋ sup θ ∈ Θ ∣ Y s − j ( θ ) ∣ ∥ H 0 ( F s − j − F ‾ s − j ) ∥ = O a . s . ( ϱ k ⌊ ( s − 1 ) / k ⌋ ) . (251) \sum_{j=1}^{s-1}\chi_k^{\lfloor(j-1)/k\rfloor}\sup_{\theta\in\Theta}\left\lvert Y_{s-j}(\theta)\right\rvert\left\lVert H_0\left(F_{s-j}-\overline{F}_{s-j}\right)\right\rVert=O_{\mathrm{a.s.}}\left(\varrho_k^{\lfloor(s-1)/k\rfloor}\right). \tag*{(251)} j = 1 ∑ s − 1 χ k ⌊( j − 1 ) / k ⌋ θ ∈ Θ sup ∣ Y s − j ( θ ) ∣ H 0 ( F s − j − F s − j ) = O a.s. ( ϱ k ⌊( s − 1 ) / k ⌋ ) . (251)
Combining (245) and (251) in (248)
I I t = O a . s . ( ∑ s = 1 t s 1 / q + ε χ k ⌊ ( t − s ) / k ⌋ ϱ k ⌊ ( s − 1 ) / k ⌋ ) = O a . s . ( χ k ⌊ t / ( 2 k ) ⌋ + ϱ k ⌊ t / ( 2 k ) ⌋ ) . (252) I I_t=O_{\mathrm{a.s.}}\left(\sum_{s=1}^{t}s^{1/q+\varepsilon}\chi_k^{\lfloor(t-s)/k\rfloor}\varrho_k^{\lfloor(s-1)/k\rfloor}\right)=O_{\mathrm{a.s.}}\left(\chi_k^{\lfloor t/(2k)\rfloor}+\varrho_k^{\lfloor t/(2k)\rfloor}\right). \tag*{(252)} I I t = O a.s. ( s = 1 ∑ t s 1/ q + ε χ k ⌊( t − s ) / k ⌋ ϱ k ⌊( s − 1 ) / k ⌋ ) = O a.s. ( χ k ⌊ t / ( 2 k )⌋ + ϱ k ⌊ t / ( 2 k )⌋ ) . (252)
It follows from (250) and (252) that the first term on the right-hand side of (243) is
∑ s = 1 t χ k ⌊ ( t − s ) / k ⌋ c s , N T ψ sup θ ∈ Θ ∥ ψ ^ s ∣ s − 1 ( θ ) − ψ ~ s ∣ s − 1 ( θ ) ∥ = O P ( 1 min ( N , T ) ) + O a . s . ( χ k ⌊ t / ( 2 k ) ⌋ + ϱ k ⌊ t / ( 2 k ) ⌋ ) . (253) \begin{aligned}
\sum_{s=1}^{t}\chi_k^{\lfloor(t-s)/k\rfloor}c_{s,NT}^{\psi}\sup_{\theta\in\Theta}\left\lVert\widehat{\psi}_{s|s-1}(\theta)-\widetilde{\psi}_{s|s-1}(\theta)\right\rVert\\
&=O_{\mathbb{P}}\left(\frac{1}{\min\left(\sqrt{N},\sqrt{T}\right)}\right)+O_{\mathrm{a.s.}}\left(\chi_k^{\lfloor t/(2k)\rfloor}+\varrho_k^{\lfloor t/(2k)\rfloor}\right). \tag*{(253)}
\end{aligned} s = 1 ∑ t χ k ⌊( t − s ) / k ⌋ c s , N T ψ θ ∈ Θ sup ψ s ∣ s − 1 ( θ ) − ψ s ∣ s − 1 ( θ ) = O P min ( N , T ) 1 + O a.s. ( χ k ⌊ t / ( 2 k )⌋ + ϱ k ⌊ t / ( 2 k )⌋ ) . (253)
We now consider the second term on the right-hand side of (243) . Since
H ^ F ‾ s − H 0 F ‾ s = ( H ^ − H 0 ) F s + H 0 ( F s − F ‾ s ) , \widehat{H}\overline{F}_s-H_0\overline{F}_s=\left(\widehat{H}-H_0\right)F_s+H_0\left(F_s-\overline{F}_s\right), H F s − H 0 F s = ( H − H 0 ) F s + H 0 ( F s − F s ) ,
we have, almost surely,
∑ s = 1 t χ k ⌊ ( t − s ) / k ⌋ c s , N T F ∥ H ^ F ‾ s − H 0 F ‾ s ∥ ≤ ∑ s = 1 t χ k ⌊ ( t − s ) / k ⌋ c s , N T F ∥ ( H ^ − H 0 ) F s ∥ + ∑ s = 1 t χ k ⌊ ( t − s ) / k ⌋ c s , N T F ∥ H 0 ( F s − F ‾ s ) ∥ . (254) \sum_{s=1}^{t}\chi_k^{\lfloor(t-s)/k\rfloor}c_{s,NT}^{F}\left\lVert\widehat{H}\overline{F}_s-H_0\overline{F}_s\right\rVert
\leq\sum_{s=1}^{t}\chi_k^{\lfloor(t-s)/k\rfloor}c_{s,NT}^{F}\left\lVert\left(\widehat{H}-H_0\right)F_s\right\rVert+\sum_{s=1}^{t}\chi_k^{\lfloor(t-s)/k\rfloor}c_{s,NT}^{F}\left\lVert H_0\left(F_s-\overline{F}_s\right)\right\rVert. \tag*{(254)} s = 1 ∑ t χ k ⌊( t − s ) / k ⌋ c s , N T F H F s − H 0 F s ≤ s = 1 ∑ t χ k ⌊( t − s ) / k ⌋ c s , N T F ( H − H 0 ) F s + s = 1 ∑ t χ k ⌊( t − s ) / k ⌋ c s , N T F H 0 ( F s − F s ) . (254)
For the first term on the right-hand side of (254) , by the geometric summability, we obtain
∑ s = 1 t X k ⌊ ( t − s ) / k ⌋ C s , N T F ∥ ( H ^ − H 0 ) F s ∥ ≤ k 1 − X k max 1 ≤ s ≤ t C s , N T F max 1 ≤ s < t ∥ ( H ^ − H 0 ) F s ∥ . \sum_{s=1}^{t} \mathcal{X}_k^{\lfloor(t-s)/k\rfloor} \mathcal{C}^{F}_{s,NT} \left\| \left(\widehat{H}-H_0\right) F_s \right\| \leq\frac{k}{1-\mathcal{X}_k} \max_{1\leq s\leq t}\mathcal{C}^{F}_{s,NT}\max_{1\leq s<t}\left\| \left(\widehat{H}-H_0\right) F_s \right\|. s = 1 ∑ t X k ⌊( t − s ) / k ⌋ C s , N T F ( H − H 0 ) F s ≤ 1 − X k k 1 ≤ s ≤ t max C s , N T F 1 ≤ s < t max ( H − H 0 ) F s .
Therefore, by (244) and (249) ,
∑ s = 1 t X k ⌊ ( t − s ) / k ⌋ C s , N T F ∥ ( H ^ − H 0 ) F s ∥ = O P ( 1 min ( N , T ) ) . (255) \sum_{s=1}^{t} \mathcal{X}_k^{\lfloor(t-s)/k\rfloor} \mathcal{C}^{F}_{s,NT}\left\| \left(\widehat{H}-H_0\right) F_s \right\| = O_{\mathbb{P}}\left(\frac{1}{\min\left(\sqrt{N},\sqrt{T}\right)}\right). \tag*{(255)} s = 1 ∑ t X k ⌊( t − s ) / k ⌋ C s , N T F ( H − H 0 ) F s = O P min ( N , T ) 1 . (255)
For the second term on the right-hand side of (254) , using the geometric convergence established in the proof of Proposition 2(b) we get
∥ H 0 ( F s − F ‾ s ) ∥ = O a . s . ( X k ⌊ s / k ⌋ ) . \left\| H_0\left(F_s-\overline{F}_s\right)\right\|=O_{a.s.}\left(\mathcal{X}_k^{\lfloor s/k\rfloor}\right). H 0 ( F s − F s ) = O a . s . ( X k ⌊ s / k ⌋ ) .
Hence, by (245) ,
∑ s = 1 t X k ⌊ ( t − s ) / k ⌋ C s , N T F ∥ H 0 ( F s − F ‾ s ) ∥ = O a . s . ( ∑ s = 1 t s 1 / η + ε X k ⌊ ( t − s ) / k ⌋ + ⌊ s / k ⌋ ) = O a . s . ( X k ⌊ t / ( 2 k ) ⌋ ) . (256) \sum_{s=1}^{t}\mathcal{X}_k^{\lfloor(t-s)/k\rfloor}\mathcal{C}^{F}_{s,NT}\left\|H_0\left(F_s-\overline{F}_s\right)\right\|=O_{a.s.}\left(\sum_{s=1}^{t}s^{1/\eta+\varepsilon}\mathcal{X}_k^{\lfloor(t-s)/k\rfloor+\lfloor s/k\rfloor}\right)=O_{a.s.}\left(\mathcal{X}_k^{\lfloor t/(2k)\rfloor}\right). \tag*{(256)} s = 1 ∑ t X k ⌊( t − s ) / k ⌋ C s , N T F H 0 ( F s − F s ) = O a . s . ( s = 1 ∑ t s 1/ η + ε X k ⌊( t − s ) / k ⌋ + ⌊ s / k ⌋ ) = O a . s . ( X k ⌊ t / ( 2 k )⌋ ) . (256)
Combining (255) and (256) in (254)
∑ s = 1 t X k ⌊ ( t − s ) / k ⌋ C s , N T F ∥ H ^ F s − H 0 F ‾ s ∥ = O P ( 1 min ( N , T ) ) + O a . s . ( X k ⌊ t / ( 2 k ) ⌋ ) . (257) \sum_{s=1}^{t}\mathcal{X}_k^{\lfloor(t-s)/k\rfloor}\mathcal{C}^{F}_{s,NT}\left\|\widehat{H}F_s-H_0\overline{F}_s\right\|=O_{\mathbb{P}}\left(\frac{1}{\min\left(\sqrt{N},\sqrt{T}\right)}\right)+O_{a.s.}\left(\mathcal{X}_k^{\lfloor t/(2k)\rfloor}\right). \tag*{(257)} s = 1 ∑ t X k ⌊( t − s ) / k ⌋ C s , N T F H F s − H 0 F s = O P min ( N , T ) 1 + O a . s . ( X k ⌊ t / ( 2 k )⌋ ) . (257)
Finally, substituting (253) and (257) into (243) yields
sup θ ∈ Θ ∥ ∇ θ ψ ~ t + 1 ∣ t ( θ ) − ∇ θ ψ ~ t + 1 ∣ t ( θ ) ∥ = O P ( 1 min ( N , T ) ) + O a . s . ( X k ⌊ t / ( 2 k ) ⌋ + ρ k ⌊ t / ( 2 k ) ⌋ ) , \sup_{\theta\in\Theta}\left\|\nabla^\theta\widetilde{\psi}_{t+1|t}(\theta)-\nabla^\theta\widetilde{\psi}_{t+1|t}(\theta)\right\|=O_{\mathbb{P}}\left(\frac{1}{\min\left(\sqrt{N},\sqrt{T}\right)}\right)+O_{a.s.}\left(\mathcal{X}_k^{\lfloor t/(2k)\rfloor}+\rho_k^{\lfloor t/(2k)\rfloor}\right), θ ∈ Θ sup ∇ θ ψ t + 1∣ t ( θ ) − ∇ θ ψ t + 1∣ t ( θ ) = O P min ( N , T ) 1 + O a . s . ( X k ⌊ t / ( 2 k )⌋ + ρ k ⌊ t / ( 2 k )⌋ ) ,
which proves part ( b ) (b) ( b ) .
(c) The processes ψ ~ t ∣ t − 1 ( θ ) \widetilde{\psi}_{t|t-1}(\theta) ψ t ∣ t − 1 ( θ ) and ψ ‾ t ∣ t − 1 ( θ ) \overline{\psi}_{t|t-1}(\theta) ψ t ∣ t − 1 ( θ ) are driven by the same stationary factor process H 0 F ‾ t H_0\overline{F}_t H 0 F t . By subtracting their derivative recursions we get
∇ θ ψ ~ t + 1 ∣ t ( θ ) − ∇ θ ψ ‾ t + 1 ∣ t ( θ ) = X ‾ t ( θ ) ( ∇ θ ψ ~ t ∣ t − 1 ( θ ) − ∇ θ ψ ‾ t ∣ t − 1 ( θ ) ) + ( X ~ t ( θ ) − X ‾ t ( θ ) ) ∇ θ ψ ‾ t ∣ t − 1 ( θ ) + D ~ t ( θ ) − D ‾ t ( θ ) . (258) \nabla^\theta\widetilde{\psi}_{t+1|t}(\theta)-\nabla^\theta\overline{\psi}_{t+1|t}(\theta)
=\overline{X}_t(\theta)\left(\nabla^\theta\widetilde{\psi}_{t|t-1}(\theta)-\nabla^\theta\overline{\psi}_{t|t-1}(\theta)\right)+\left(\widetilde{X}_t(\theta)-\overline{X}_t(\theta)\right)\nabla^\theta\overline{\psi}_{t|t-1}(\theta)+\widetilde{D}_t(\theta)-\overline{D}_t(\theta). \tag*{(258)} ∇ θ ψ t + 1∣ t ( θ ) − ∇ θ ψ t + 1∣ t ( θ ) = X t ( θ ) ( ∇ θ ψ t ∣ t − 1 ( θ ) − ∇ θ ψ t ∣ t − 1 ( θ ) ) + ( X t ( θ ) − X t ( θ ) ) ∇ θ ψ t ∣ t − 1 ( θ ) + D t ( θ ) − D t ( θ ) . (258)
Since the factor input is the same in the two recursions, a straightforward application of the mean-value theorem shows that
sup θ ∈ Θ ∥ ∇ θ ψ ~ t + 1 ∣ t ( θ ) − ∇ θ ψ ˉ t + 1 ∣ t ( θ ) ∥ ≤ X k ⌊ t / k ⌋ sup θ ∈ Θ ∥ ∇ θ ψ 1 ∣ 0 ⋆ − ∇ θ ψ ˉ 1 ∣ 0 ( θ ) ∥ + ∑ s = 1 t X k ⌊ ( t − s ) / k ⌋ C s ψ sup θ ∈ Θ ∥ ψ ~ s ∣ s − 1 ( θ ) − ψ ˉ s ∣ s − 1 ( θ ) ∥ , (259) \sup_{\theta\in\Theta}\left\|\nabla^\theta\tilde{\psi}_{t+1|t}(\theta)-\nabla^\theta\bar{\psi}_{t+1|t}(\theta)\right\|
\leq\mathcal{X}_k^{\lfloor t/k\rfloor}\sup_{\theta\in\Theta}\left\|\nabla^\theta\psi^\star_{1|0}-\nabla^\theta\bar{\psi}_{1|0}(\theta)\right\|+\sum_{s=1}^{t}\mathcal{X}_k^{\lfloor(t-s)/k\rfloor}C_s^\psi\sup_{\theta\in\Theta}\left\|\tilde{\psi}_{s|s-1}(\theta)-\bar{\psi}_{s|s-1}(\theta)\right\|, \tag*{(259)} θ ∈ Θ sup ∇ θ ψ ~ t + 1∣ t ( θ ) − ∇ θ ψ ˉ t + 1∣ t ( θ ) ≤ X k ⌊ t / k ⌋ θ ∈ Θ sup ∇ θ ψ 1∣0 ⋆ − ∇ θ ψ ˉ 1∣0 ( θ ) + s = 1 ∑ t X k ⌊( t − s ) / k ⌋ C s ψ θ ∈ Θ sup ψ ~ s ∣ s − 1 ( θ ) − ψ ˉ s ∣ s − 1 ( θ ) , (259)
where C s ψ C_s^\psi C s ψ is defined as in part (a) with ∇ θ ψ ^ s ∣ s − 1 ( θ ) \nabla^\theta\hat{\psi}_{s|s-1}(\theta) ∇ θ ψ ^ s ∣ s − 1 ( θ ) replaced by ∇ θ ψ ˉ s ∣ s − 1 ( θ ) \nabla^\theta\bar{\psi}_{s|s-1}(\theta) ∇ θ ψ ˉ s ∣ s − 1 ( θ ) . By (227) , Hölder’s inequality, and Assumption 11(iii)–(iv), with q = p / 2 > 1 q=p/2>1 q = p /2 > 1 ,
sup s ∈ Z ∥ C s ψ ∥ q ≤ K X ′ sup s ∈ Z ∥ sup θ ∈ Θ ∥ ∇ θ ψ ˉ s ∣ s − 1 ( θ ) ∥ ∥ p + K D ′ ′ < ∞ . (260) \sup_{s\in\mathbb{Z}}\|C_s^\psi\|_q\leq K'_X\sup_{s\in\mathbb{Z}}\left\|\sup_{\theta\in\Theta}\|\nabla^\theta\bar{\psi}_{s|s-1}(\theta)\|\right\|_p+K''_D<\infty. \tag*{(260)} s ∈ Z sup ∥ C s ψ ∥ q ≤ K X ′ s ∈ Z sup θ ∈ Θ sup ∥ ∇ θ ψ ˉ s ∣ s − 1 ( θ ) ∥ p + K D ′′ < ∞. (260)
Consequently, for every ε > 0 \varepsilon>0 ε > 0 ,
∑ s = 1 ∞ P ( C s ψ > s 1 / q + ε ) ≤ sup s ∈ Z E [ ( C s ψ ) q ] ∑ s = 1 ∞ s − 1 − q ε < ∞ . \sum_{s=1}^{\infty}\mathbb{P}\left(C_s^\psi>s^{1/q+\varepsilon}\right)\leq\sup_{s\in\mathbb{Z}}\mathbb{E}\left[\left(C_s^\psi\right)^q\right]\sum_{s=1}^{\infty}s^{-1-q\varepsilon}<\infty. s = 1 ∑ ∞ P ( C s ψ > s 1/ q + ε ) ≤ s ∈ Z sup E [ ( C s ψ ) q ] s = 1 ∑ ∞ s − 1 − q ε < ∞.
The Borel–Cantelli lemma therefore implies
C s ψ = O a . s . ( s 1 / q + ε ) . (261) C_s^\psi=O_{\mathrm{a.s.}}\left(s^{1/q+\varepsilon}\right). \tag*{(261)} C s ψ = O a.s. ( s 1/ q + ε ) . (261)
Again, since ψ 1 ∣ 0 ⋆ \psi^\star_{1|0} ψ 1∣0 ⋆ is fixed, ∇ θ ψ 1 ∣ 0 ⋆ = 0 r × d \nabla^\theta\psi^\star_{1|0}=0_{r\times d} ∇ θ ψ 1∣0 ⋆ = 0 r × d , and (227) implies that
sup θ ∈ Θ ∥ ∇ θ ψ ˉ 1 ∣ 0 ( θ ) ∥ < ∞ almost surely . \sup_{\theta\in\Theta}\left\|\nabla^\theta\bar{\psi}_{1|0}(\theta)\right\|<\infty\quad\text{almost surely}. θ ∈ Θ sup ∇ θ ψ ˉ 1∣0 ( θ ) < ∞ almost surely .
Hence, the first term on the right-hand side of (259) satisfies
X k ⌊ t / k ⌋ sup θ ∈ Θ ∥ ∇ θ ψ 1 ∣ 0 ⋆ − ∇ θ ψ ˉ 1 ∣ 0 ( θ ) ∥ = O a . s . ( X k ⌊ t / k ⌋ ) . \mathcal{X}_k^{\lfloor t/k\rfloor}\sup_{\theta\in\Theta}\left\|\nabla^\theta\psi^\star_{1|0}-\nabla^\theta\bar{\psi}_{1|0}(\theta)\right\|=O_{\mathrm{a.s.}}\left(\mathcal{X}_k^{\lfloor t/k\rfloor}\right). X k ⌊ t / k ⌋ θ ∈ Θ sup ∇ θ ψ 1∣0 ⋆ − ∇ θ ψ ˉ 1∣0 ( θ ) = O a.s. ( X k ⌊ t / k ⌋ ) .
Moreover, Proposition 2(c) further implies that
sup θ ∈ Θ ∥ ψ ~ s ∣ s − 1 ( θ ) − ψ ˉ s ∣ s − 1 ( θ ) ∥ = O a . s . ( L k ⌊ ( s − 1 ) / k ⌋ ) . \sup_{\theta\in\Theta}\left\|\tilde{\psi}_{s|s-1}(\theta)-\bar{\psi}_{s|s-1}(\theta)\right\|=O_{\mathrm{a.s.}}\left(\mathcal{L}_k^{\lfloor(s-1)/k\rfloor}\right). θ ∈ Θ sup ψ ~ s ∣ s − 1 ( θ ) − ψ ˉ s ∣ s − 1 ( θ ) = O a.s. ( L k ⌊( s − 1 ) / k ⌋ ) .
Combining this result with (261) , the second term on the right-hand side of (259) satisfies
∑ s = 1 t X k ⌊ ( t − s ) / k ⌋ C s ψ sup θ ∈ Θ ∥ ψ ~ s ∣ s − 1 ( θ ) − ψ ˉ s ∣ s − 1 ( θ ) ∥ = O a . s . ( ∑ s = 1 t X k 1 / q + ε X k ⌊ ( t − s ) / k ⌋ ϱ k ⌊ ( s − 1 ) / k ⌋ ) = O a . s . ( X k ⌊ t / ( 2 k ) ⌋ + ϱ k ⌊ t / ( 2 k ) ⌋ ) , \begin{aligned}
\sum_{s=1}^{t}\mathcal{X}_k^{\lfloor(t-s)/k\rfloor}C_s^\psi\sup_{\theta\in\Theta}\left\|\tilde{\psi}_{s|s-1}(\theta)-\bar{\psi}_{s|s-1}(\theta)\right\| \\
&= O_{a.s.}\left(\sum_{s=1}^{t} \mathcal{X}_k^{1/q+\varepsilon}\mathcal{X}_k^{\lfloor(t-s)/k\rfloor}\varrho_k^{\lfloor(s-1)/k\rfloor}\right)=O_{a.s.}\left(\mathcal{X}_k^{\lfloor t/(2k)\rfloor}+\varrho_k^{\lfloor t/(2k)\rfloor}\right),
\end{aligned} s = 1 ∑ t X k ⌊( t − s ) / k ⌋ C s ψ θ ∈ Θ sup ψ ~ s ∣ s − 1 ( θ ) − ψ ˉ s ∣ s − 1 ( θ ) = O a . s . ( s = 1 ∑ t X k 1/ q + ε X k ⌊( t − s ) / k ⌋ ϱ k ⌊( s − 1 ) / k ⌋ ) = O a . s . ( X k ⌊ t / ( 2 k )⌋ + ϱ k ⌊ t / ( 2 k )⌋ ) ,
so that, by substituting the preceding two bounds into (259) we clearly get
sup θ ∈ Θ ∥ ∇ θ ψ ~ t + 1 ∣ t ( θ ) − ∇ θ ψ ‾ t + 1 ∣ t ( θ ) ∥ = O a . s . ( X k ⌊ t / ( 2 k ) ⌋ + ϱ k ⌊ t / ( 2 k ) ⌋ ) , \sup_{\theta\in\Theta}\left\|\nabla^\theta\widetilde{\psi}_{t+1|t}(\theta)-\nabla^\theta\overline{\psi}_{t+1|t}(\theta)\right\|=O_{a.s.}\left(\mathcal{X}_k^{\lfloor t/(2k)\rfloor}+\varrho_k^{\lfloor t/(2k)\rfloor}\right), θ ∈ Θ sup ∇ θ ψ t + 1∣ t ( θ ) − ∇ θ ψ t + 1∣ t ( θ ) = O a . s . ( X k ⌊ t / ( 2 k )⌋ + ϱ k ⌊ t / ( 2 k )⌋ ) ,
which proves part (c).
(d) When evaluated using the stationary and ergodic processes { H 0 F ‾ t } t ∈ Z \left\{H_0\overline{F}_t\right\}_{t\in\mathbb{Z}} { H 0 F t } t ∈ Z and { ψ ‾ t ∣ t − 1 ( θ ) } t ∈ Z \left\{\overline{\psi}_{t|t-1}(\theta)\right\}_{t\in\mathbb{Z}} { ψ t ∣ t − 1 ( θ ) } t ∈ Z , the sequences { X t ( θ ) } t ∈ Z \left\{X_t(\theta)\right\}_{t\in\mathbb{Z}} { X t ( θ ) } t ∈ Z and { D t ( θ ) } t ∈ Z \left\{D_t(\theta)\right\}_{t\in\mathbb{Z}} { D t ( θ ) } t ∈ Z are themselves strictly stationary and ergodic. Moreover, Assumption 11(iv) implies that
E [ log + sup θ ∈ Θ ∥ D t ( θ ) ∥ ] < ∞ , \mathbb{E}\left[\log^+\sup_{\theta\in\Theta}\left\|D_t(\theta)\right\|\right]<\infty, E [ log + θ ∈ Θ sup ∥ D t ( θ ) ∥ ] < ∞ ,
whereas Assumption 7(i) implies that the k k k -step Lipschitz coefficient of the derivative recursion is bounded almost surely by X k < 1 \mathcal{X}_k<1 X k < 1 . Therefore, Theorem 3.1 of Bougerol [12 ] , applied to (221) , implies that the derivative recursion admits a unique causal strictly stationary and ergodic solution given by
∇ θ ψ ‾ t + 1 ∣ t ( θ ) = D t ( θ ) + ∑ j = 1 ∞ ( ∏ h = 0 j − 1 X t − h ( θ ) ) D t − j ( θ ) . (262) \nabla^\theta\overline{\psi}_{t+1|t}(\theta)=D_t(\theta)+\sum_{j=1}^{\infty}\left(\prod_{h=0}^{j-1}X_{t-h}(\theta)\right)D_{t-j}(\theta). \tag*{(262)} ∇ θ ψ t + 1∣ t ( θ ) = D t ( θ ) + j = 1 ∑ ∞ ( h = 0 ∏ j − 1 X t − h ( θ ) ) D t − j ( θ ) . (262)
The series in (262) converges almost surely. Furthermore, by (227)
sup t ∈ Z E [ sup θ ∈ Θ ∥ ∇ θ ψ ‾ t ∣ t − 1 ( θ ) ∥ p ] < ∞ . \sup_{t\in\mathbb{Z}}\mathbb{E}\left[\sup_{\theta\in\Theta}\left\|\nabla^\theta\overline{\psi}_{t|t-1}(\theta)\right\|^p\right]<\infty. t ∈ Z sup E [ θ ∈ Θ sup ∇ θ ψ t ∣ t − 1 ( θ ) p ] < ∞.
It remains to establish near-epoch dependence. For any τ ≥ 1 \tau\geq1 τ ≥ 1 , consider the τ \tau τ -truncated factor and filter processes used in the proof of Lemma 6, and construct the corresponding derivative recursion by initializing it at t − τ t-\tau t − τ with the fixed value 0 r × d \mathbf{0}_{r\times d} 0 r × d . The truncated derivative is a measurable function of v t , … , v t − τ \mathbf{v}_t,\ldots,\mathbf{v}_{t-\tau} v t , … , v t − τ . An application of the derivative perturbation bound (230) , the contraction condition, and the geometric approximation obtained in Lemma 6, reveals that
∥ ∇ θ ψ ‾ t ∣ t − 1 ( θ 0 ) − ∇ θ ψ ‾ t ∣ t − 1 ( τ ) ( θ 0 ) ∥ 2 = O ( X k ⌊ τ / ( 2 k ) ⌋ ) . \left\|\nabla^\theta\overline{\psi}_{t|t-1}(\theta_0)-\nabla^\theta\overline{\psi}^{(\tau)}_{t|t-1}(\theta_0)\right\|_2=O\left(\mathcal{X}_k^{\lfloor\tau/(2k)\rfloor}\right). ∇ θ ψ t ∣ t − 1 ( θ 0 ) − ∇ θ ψ t ∣ t − 1 ( τ ) ( θ 0 ) 2 = O ( X k ⌊ τ / ( 2 k )⌋ ) .
Consequently, by the contraction property of conditional expectations,
∥ ∇ θ ψ ‾ t ∣ t − 1 ( θ 0 ) − E [ ∇ θ ψ ‾ t ∣ t − 1 ( θ 0 ) ∣ v t , … , v t − τ ] ∥ 2 ≤ 2 ∥ ∇ θ ψ ‾ t ∣ t − 1 ( θ 0 ) − ∇ θ ψ ‾ t ∣ t − 1 ( τ ) ( θ 0 ) ∥ 2 = O ( X k ⌊ τ / ( 2 k ) ⌋ ) . \left\|\nabla^\theta\overline{\psi}_{t|t-1}(\theta_0)-\mathbb{E}\left[\nabla^\theta\overline{\psi}_{t|t-1}(\theta_0)\mid\mathbf{v}_t,\ldots,\mathbf{v}_{t-\tau}\right]\right\|_2
\leq2\left\|\nabla^\theta\overline{\psi}_{t|t-1}(\theta_0)-\nabla^\theta\overline{\psi}^{(\tau)}_{t|t-1}(\theta_0)\right\|_2
=O\left(\mathcal{X}_k^{\lfloor\tau/(2k)\rfloor}\right). ∇ θ ψ t ∣ t − 1 ( θ 0 ) − E [ ∇ θ ψ t ∣ t − 1 ( θ 0 ) ∣ v t , … , v t − τ ] 2 ≤ 2 ∇ θ ψ t ∣ t − 1 ( θ 0 ) − ∇ θ ψ t ∣ t − 1 ( τ ) ( θ 0 ) 2 = O ( X k ⌊ τ / ( 2 k )⌋ ) .
The approximation coefficients therefore decrease geometrically and are of size − 1 -1 − 1 . Hence, the process { ∇ θ ψ ˉ t ∣ t − 1 ( θ ) } t ∈ Z \left\{\nabla^\theta\bar{\psi}_{t|t-1}(\theta)\right\}_{t\in\mathbb{Z}} { ∇ θ ψ ˉ t ∣ t − 1 ( θ ) } t ∈ Z is near-epoch dependent of size − 1 -1 − 1 on { v t } t ∈ Z \{v_t\}_{t\in\mathbb{Z}} { v t } t ∈ Z , which is geometrically β \beta β -mixing by Assumption 1 ( i ) 1(i) 1 ( i ) .
(e) Finally, using the decomposition in (222) it is straightforward to see that the result follows immediately by combining parts (a)–(c) and the triangle inequality.
Lemma 8. Consider the second derivative process associated with the score-driven recursion,
∇ θ θ ψ t + 1 ∣ t ( θ ) = ( I d ⊗ X t ( θ ) ) ∇ θ θ ψ t ∣ t − 1 ( θ ) + ( ∇ θ ψ t ∣ t − 1 ( θ ) T ⊗ I r ) ∂ vec X t ( θ ) ∂ θ T + ∂ vec D t ( θ ) ∂ θ T . (263) \nabla^{\theta\theta}\psi_{t+1|t}(\theta) = (I_d \otimes X_t(\theta))\nabla^{\theta\theta}\psi_{t|t-1}(\theta)
+ \left(\nabla^\theta\psi_{t|t-1}(\theta)^\mathsf{T}\otimes I_r\right)\frac{\partial\operatorname{vec}X_t(\theta)}{\partial\theta^\mathsf{T}}+\frac{\partial\operatorname{vec}D_t(\theta)}{\partial\theta^\mathsf{T}}. \tag*{(263)} ∇ θ θ ψ t + 1∣ t ( θ ) = ( I d ⊗ X t ( θ )) ∇ θ θ ψ t ∣ t − 1 ( θ ) + ( ∇ θ ψ t ∣ t − 1 ( θ ) T ⊗ I r ) ∂ θ T ∂ vec X t ( θ ) + ∂ θ T ∂ vec D t ( θ ) . (263) By differentiating (222) with respect to θ \theta θ we get
∇ θ θ ψ ^ t ∣ t − 1 ( θ ) − ∇ θ θ ψ ˉ t ∣ t − 1 ( θ ) = ( ∇ θ θ ψ ^ t ∣ t − 1 ( θ ) − ∇ θ θ ψ ^ t ∣ t − 1 ( θ ) ) + ( ∇ θ θ ψ ^ t ∣ t − 1 ( θ ) − ∇ θ θ ψ ~ t ∣ t − 1 ( θ ) ) + ( ∇ θ θ ψ ~ t ∣ t − 1 ( θ ) − ∇ θ θ ψ ˉ t ∣ t − 1 ( θ ) ) . (264) \begin{aligned}
\nabla^{\theta\theta}\widehat{\psi}_{t|t-1}(\theta)-\nabla^{\theta\theta}\bar{\psi}_{t|t-1}(\theta)
={}&\left(\nabla^{\theta\theta}\widehat{\psi}_{t|t-1}(\theta)-\nabla^{\theta\theta}\widehat{\psi}_{t|t-1}(\theta)\right)\\
&+\left(\nabla^{\theta\theta}\widehat{\psi}_{t|t-1}(\theta)-\nabla^{\theta\theta}\widetilde{\psi}_{t|t-1}(\theta)\right)\\
&+\left(\nabla^{\theta\theta}\widetilde{\psi}_{t|t-1}(\theta)-\nabla^{\theta\theta}\bar{\psi}_{t|t-1}(\theta)\right). \tag*{(264)}
\end{aligned} ∇ θ θ ψ t ∣ t − 1 ( θ ) − ∇ θ θ ψ ˉ t ∣ t − 1 ( θ ) = ( ∇ θ θ ψ t ∣ t − 1 ( θ ) − ∇ θ θ ψ t ∣ t − 1 ( θ ) ) + ( ∇ θ θ ψ t ∣ t − 1 ( θ ) − ∇ θ θ ψ t ∣ t − 1 ( θ ) ) + ( ∇ θ θ ψ t ∣ t − 1 ( θ ) − ∇ θ θ ψ ˉ t ∣ t − 1 ( θ ) ) . (264) Then, under Assumptions 1 1 1 –7 7 7 and 11 11 11 , with p > 2 p>2 p > 2 , the following results hold:
(a) For any t ≥ 1 t\ge1 t ≥ 1 , as N , T → ∞ N,T\to\infty N , T → ∞ ,
sup θ ∈ Θ ∥ ∇ θ θ ψ ^ t + 1 ∣ t ( θ ) − ∇ θ θ ψ ^ t + 1 ∣ t ( θ ) ∥ = O P ( 1 min ( N , T ) ) + O a . s . ( χ k ⌊ t / ( 4 k ) ⌋ ) . \sup_{\theta\in\Theta}\left\|\nabla^{\theta\theta}\widehat{\psi}_{t+1|t}(\theta)-\nabla^{\theta\theta}\widehat{\psi}_{t+1|t}(\theta)\right\|=O_{\mathbb{P}}\left(\frac{1}{\min\left(\sqrt{N},T\right)}\right)+O_{a.s.}\left(\chi_k^{\lfloor t/(4k)\rfloor}\right). θ ∈ Θ sup ∇ θ θ ψ t + 1∣ t ( θ ) − ∇ θ θ ψ t + 1∣ t ( θ ) = O P min ( N , T ) 1 + O a . s . ( χ k ⌊ t / ( 4 k )⌋ ) . (b) For any t ≥ 1 t\ge1 t ≥ 1 , as N , T → ∞ N,T\to\infty N , T → ∞ ,
sup θ ∈ Θ ∥ ∇ θ θ ψ ^ t ∣ t − 1 ( θ ) − ∇ θ θ ψ ~ t ∣ t − 1 ( θ ) ∥ = O P ( 1 min ( N , T ) ) + O a . s . ( χ k ⌊ t / ( 4 k ) ⌋ + ϱ k ⌊ t / ( 4 k ) ⌋ ) . \sup_{\theta\in\Theta}\left\|\nabla^{\theta\theta}\widehat{\psi}_{t|t-1}(\theta)-\nabla^{\theta\theta}\widetilde{\psi}_{t|t-1}(\theta)\right\|=O_{\mathbb{P}}\left(\frac{1}{\min\left(\sqrt{N},\sqrt{T}\right)}\right)+O_{a.s.}\left(\chi_k^{\lfloor t/(4k)\rfloor}+\varrho_k^{\lfloor t/(4k)\rfloor}\right). θ ∈ Θ sup ∇ θ θ ψ t ∣ t − 1 ( θ ) − ∇ θ θ ψ t ∣ t − 1 ( θ ) = O P min ( N , T ) 1 + O a . s . ( χ k ⌊ t / ( 4 k )⌋ + ϱ k ⌊ t / ( 4 k )⌋ ) . (c) For any fixed initialization ψ 1 ∣ 0 ∗ ∈ P \psi^*_{1|0}\in\mathcal{P} ψ 1∣0 ∗ ∈ P , as t → ∞ t\to\infty t → ∞ ,
sup θ ∈ Θ ∥ ∇ θ θ ψ ~ t ∣ t − 1 ( θ ) − ∇ θ θ ψ ˉ t ∣ t − 1 ( θ ) ∥ = O a . s . ( χ k ⌊ t / ( 4 k ) ⌋ + ϱ k ⌊ t / ( 4 k ) ⌋ ) . \sup_{\theta\in\Theta}\left\|\nabla^{\theta\theta}\widetilde{\psi}_{t|t-1}(\theta)-\nabla^{\theta\theta}\bar{\psi}_{t|t-1}(\theta)\right\|=O_{a.s.}\left(\chi_k^{\lfloor t/(4k)\rfloor}+\varrho_k^{\lfloor t/(4k)\rfloor}\right). θ ∈ Θ sup ∇ θ θ ψ t ∣ t − 1 ( θ ) − ∇ θ θ ψ ˉ t ∣ t − 1 ( θ ) = O a . s . ( χ k ⌊ t / ( 4 k )⌋ + ϱ k ⌊ t / ( 4 k )⌋ ) .
(d) The process { ∇ θ θ ψ ˉ t ∣ t − 1 ( θ ) } t ∈ Z \left\{\nabla^{\theta\theta}\bar{\psi}_{t|t-1}(\theta)\right\}_{t\in\mathbb{Z}} { ∇ θ θ ψ ˉ t ∣ t − 1 ( θ ) } t ∈ Z is strictly stationary and ergodic and satisfies
E [ sup θ ∈ Θ ∥ ∇ θ θ ψ ˉ t ∣ t − 1 ( θ ) ∥ p ] < ∞ . \mathbb{E}\left[\sup_{\theta\in\Theta}\left\|\nabla^{\theta\theta}\bar{\psi}_{t|t-1}(\theta)\right\|^{p}\right]<\infty. E [ θ ∈ Θ sup ∇ θ θ ψ ˉ t ∣ t − 1 ( θ ) p ] < ∞.
Moreover, { ∇ θ θ ψ ˉ t ∣ t − 1 ( θ 0 ) } t ∈ Z \left\{\nabla^{\theta\theta}\bar{\psi}_{t|t-1}(\theta_{0})\right\}_{t\in\mathbb{Z}} { ∇ θ θ ψ ˉ t ∣ t − 1 ( θ 0 ) } t ∈ Z is near-epoch dependent of size − 1 -1 − 1 on the geometrically β \beta β -mixing process { v t } t ∈ Z \{v_t\}_{t\in\mathbb{Z}} { v t } t ∈ Z .
(e) For any fixed initialization ψ 1 ∣ 0 ∈ P \psi_{1|0}\in\mathcal{P} ψ 1∣0 ∈ P and any t ≥ 1 t\ge1 t ≥ 1 , as N , T → ∞ N,T\to\infty N , T → ∞ ,
sup θ ∈ Θ ∥ ∇ θ θ ψ ^ t + 1 ∣ t ( θ ) − ∇ θ θ ψ ˉ t + 1 ∣ t ( θ ) ∥ = O P ( 1 min ( N , T ) ) + O P ( 1 min ( N , T ) ) + O a . s . ( χ k ⌊ t / ( 4 k ) ⌋ + ϱ k ⌊ t / ( 4 k ) ⌋ ) . \sup_{\theta\in\Theta}\left\|\nabla^{\theta\theta}\widehat{\psi}_{t+1|t}(\theta)-\nabla^{\theta\theta}\bar{\psi}_{t+1|t}(\theta)\right\|=O_{\mathrm{P}}\left(\frac{1}{\min\left(\sqrt{N},T\right)}\right)+O_{\mathrm{P}}\left(\frac{1}{\min\left(\sqrt{N},\sqrt{T}\right)}\right)+O_{\mathrm{a.s.}}\left(\chi_k^{\lfloor t/(4k)\rfloor}+\varrho_k^{\lfloor t/(4k)\rfloor}\right). θ ∈ Θ sup ∇ θ θ ψ t + 1∣ t ( θ ) − ∇ θ θ ψ ˉ t + 1∣ t ( θ ) = O P min ( N , T ) 1 + O P min ( N , T ) 1 + O a.s. ( χ k ⌊ t / ( 4 k )⌋ + ϱ k ⌊ t / ( 4 k )⌋ ) .
Proof. The coefficient multiplying the lagged second derivative in (263) is I d ⊗ X t ( θ ) I_d\otimes X_t(\theta) I d ⊗ X t ( θ ) . Therefore,
sup θ ∈ Θ ∥ ∏ j = 1 k ( I d ⊗ X t − j + 1 ( θ ) ) ∥ = sup θ ∈ Θ ∥ I d ⊗ ∏ j = 1 k X t − j + 1 ( θ ) ∥ ≤ χ k \sup_{\theta\in\Theta}\left\|\prod_{j=1}^{k}\left(I_d\otimes X_{t-j+1}(\theta)\right)\right\|=\sup_{\theta\in\Theta}\left\|I_d\otimes\prod_{j=1}^{k}X_{t-j+1}(\theta)\right\|\le\chi_k θ ∈ Θ sup j = 1 ∏ k ( I d ⊗ X t − j + 1 ( θ ) ) = θ ∈ Θ sup I d ⊗ j = 1 ∏ k X t − j + 1 ( θ ) ≤ χ k
almost surely. Hence, the second derivative recursion inherits the same k k k -step contraction as the first derivative recursion.
We first establish the required moment bound. By the equality of mixed partial derivatives, the matrices
∂ vec X t ( θ ) ∂ θ T and ∂ vec D t ( θ ) ∂ ψ t ∣ t − 1 T \frac{\partial\operatorname{vec}X_t(\theta)}{\partial\theta^{\mathsf{T}}}
\qquad\text{and}\qquad
\frac{\partial\operatorname{vec}D_t(\theta)}{\partial\psi_{t|t-1}^{\mathsf{T}}} ∂ θ T ∂ vec X t ( θ ) and ∂ ψ t ∣ t − 1 T ∂ vec D t ( θ )
contain the same mixed derivatives, up to a fixed rearrangement of their elements. The additional terms arising from evaluation along the filtered process are controlled by ∇ θ ψ t ∣ t − 1 ( θ ) \nabla^\theta\psi_{t|t-1}(\theta) ∇ θ ψ t ∣ t − 1 ( θ ) and the derivative envelopes in Assumption 11( i i i ) (iii) ( iii ) –( v i ) (vi) ( v i ) . Consequently, Lemma 7( d ) (d) ( d ) , Hölder’s inequality, and Assumption 11( i i i ) (iii) ( iii ) –( v i ) (vi) ( v i ) imply that
∥ sup θ ∈ Θ ∥ ( ∇ θ ψ ˉ t ∣ t − 1 ( θ ) T ⊗ I r ) ∂ vec X t ( θ ) ∂ θ T ∥ ∥ p ≤ ∥ sup θ ∈ Θ ∥ ∇ θ ψ ˉ t ∣ t − 1 ( θ ) ∥ ∥ 2 p ∥ sup θ ∈ Θ ∥ ∂ vec X t ( θ ) ∂ θ T ∥ ∥ 2 p < ∞ . \left\|\sup_{\theta\in\Theta}\left\|\left(\nabla^\theta\bar{\psi}_{t|t-1}(\theta)^{\mathsf{T}}\otimes I_r\right)\frac{\partial\operatorname{vec}X_t(\theta)}{\partial\theta^{\mathsf{T}}}\right\|\right\|_p
\le
\left\|\sup_{\theta\in\Theta}\left\|\nabla^\theta\bar{\psi}_{t|t-1}(\theta)\right\|\right\|_{2p}
\left\|\sup_{\theta\in\Theta}\left\|\frac{\partial\operatorname{vec}X_t(\theta)}{\partial\theta^{\mathsf{T}}}\right\|\right\|_{2p}
<\infty. θ ∈ Θ sup ( ∇ θ ψ ˉ t ∣ t − 1 ( θ ) T ⊗ I r ) ∂ θ T ∂ vec X t ( θ ) p ≤ θ ∈ Θ sup ∇ θ ψ ˉ t ∣ t − 1 ( θ ) 2 p θ ∈ Θ sup ∂ θ T ∂ vec X t ( θ ) 2 p < ∞.
The same bound holds uniformly for the corresponding nonstationary and feasible recursions.
Applying the k k k -step moment argument used in (224) –(227) to (263) , and using the fact that the finite partial products of X t ( θ ) \mathbf{X}_t(\theta) X t ( θ ) have bounded moments, yields
sup t ∈ Z sup θ ∈ Θ ∥ ∇ θ θ ψ ˉ t ∣ t − 1 ( θ ) ∥ p < ∞ . (265) \sup_{t\in\mathbb{Z}}\sup_{\theta\in\Theta}\left\lVert\nabla^{\theta\theta}\bar{\psi}_{t|t-1}(\theta)\right\rVert_p<\infty. \tag*{(265)} t ∈ Z sup θ ∈ Θ sup ∇ θ θ ψ ˉ t ∣ t − 1 ( θ ) p < ∞. (265)
The same argument gives the corresponding uniform moment bounds for the second derivative processes associated with ψ ^ t ∣ t − 1 ( θ ) \widehat{\psi}_{t|t-1}(\theta) ψ t ∣ t − 1 ( θ ) , ψ ~ t ∣ t − 1 ( θ ) \widetilde{\psi}_{t|t-1}(\theta) ψ t ∣ t − 1 ( θ ) , and ψ ~ t ∣ t − 1 ( θ ) \widetilde{\psi}_{t|t-1}(\theta) ψ t ∣ t − 1 ( θ ) .
We now compare any two of these derivative recursions. Subtracting their respective versions of (263) , applying the mean value theorem, and iterating the resulting inequality in blocks of length k k k
sup θ ∈ Θ ∥ ∇ θ θ ψ t + 1 ∣ t ( 1 ) ( θ ) − ∇ θ θ ψ t + 1 ∣ t ( 2 ) ( θ ) ∥ ≤ χ k ⌊ t / k ⌋ sup θ ∈ Θ ∥ ∇ θ θ ψ 1 ∣ 0 ( 1 ) ( θ ) − ∇ θ θ ψ 1 ∣ 0 ( 2 ) ( θ ) ∥ + ∑ s = 1 t χ k ⌊ ( t − s ) / k ⌋ C s , N T ( 2 ) ( sup θ ∈ Θ ∥ ψ s ∣ s − 1 ( 1 ) ( θ ) − ψ s ∣ s − 1 ( 2 ) ( θ ) ∥ + ∥ F s ( 1 ) − F s ( 2 ) ∥ + sup θ ∈ Θ ∥ ∇ θ ψ s ∣ s − 1 ( 1 ) ( θ ) − ∇ θ ψ s ∣ s − 1 ( 2 ) ( θ ) ∥ ) . (266) \begin{aligned}
\sup_{\theta\in\Theta}\left\lVert\nabla^{\theta\theta}\psi^{(1)}_{t+1|t}(\theta)-\nabla^{\theta\theta}\psi^{(2)}_{t+1|t}(\theta)\right\rVert
&\leq\chi_k^{\lfloor t/k\rfloor}\sup_{\theta\in\Theta}\left\lVert\nabla^{\theta\theta}\psi^{(1)}_{1|0}(\theta)-\nabla^{\theta\theta}\psi^{(2)}_{1|0}(\theta)\right\rVert\\
&\quad+\sum_{s=1}^{t}\chi_k^{\lfloor(t-s)/k\rfloor}\mathcal{C}^{(2)}_{s,NT}\left(\sup_{\theta\in\Theta}\left\lVert\psi^{(1)}_{s|s-1}(\theta)-\psi^{(2)}_{s|s-1}(\theta)\right\rVert\right.\\
&\qquad\left.+\left\lVert F_s^{(1)}-F_s^{(2)}\right\rVert+\sup_{\theta\in\Theta}\left\lVert\nabla^\theta\psi^{(1)}_{s|s-1}(\theta)-\nabla^\theta\psi^{(2)}_{s|s-1}(\theta)\right\rVert\right). \tag*{(266)}
\end{aligned} θ ∈ Θ sup ∇ θ θ ψ t + 1∣ t ( 1 ) ( θ ) − ∇ θ θ ψ t + 1∣ t ( 2 ) ( θ ) ≤ χ k ⌊ t / k ⌋ θ ∈ Θ sup ∇ θ θ ψ 1∣0 ( 1 ) ( θ ) − ∇ θ θ ψ 1∣0 ( 2 ) ( θ ) + s = 1 ∑ t χ k ⌊( t − s ) / k ⌋ C s , N T ( 2 ) ( θ ∈ Θ sup ψ s ∣ s − 1 ( 1 ) ( θ ) − ψ s ∣ s − 1 ( 2 ) ( θ ) + F s ( 1 ) − F s ( 2 ) + θ ∈ Θ sup ∇ θ ψ s ∣ s − 1 ( 1 ) ( θ ) − ∇ θ ψ s ∣ s − 1 ( 2 ) ( θ ) ) . (266)
Here, C s , N T ( 2 ) \mathcal{C}^{(2)}_{s,NT} C s , N T ( 2 ) collects the derivative envelopes and the norms of the first- and second-derivative processes arising from the mean-value expansion. By (227) , (265) , Hölder’s inequality, and Assumption 11( i i i ) (iii) ( iii ) –( v i ) (vi) ( v i ) , applied at sufficiently high moment orders, there exists q > 1 q>1 q > 1 such that
sup N , T sup 1 ≤ s ≤ T ∥ C s , N T ( 2 ) ∥ q < ∞ . \sup_{N,T}\sup_{1\leq s\leq T}\left\lVert\mathcal{C}^{(2)}_{s,NT}\right\rVert_q<\infty. N , T sup 1 ≤ s ≤ T sup C s , N T ( 2 ) q < ∞.
Thus, the coefficients are tight for every fixed t t t , while the same Markov and Borel–Cantelli argument used in the proof of Lemma 7 controls the almost-sure components.
For part ( a ) (a) ( a ) , apply (266) using Proposition 1( a ) (a) ( a ) , Proposition 2( a ) (a) ( a ) , and Lemma 7( a ) (a) ( a ) . For part ( b ) (b) ( b ) , use the corresponding results in Proposition 1, Proposition 2( b ) (b) ( b ) , and Lemma 7( b ) (b) ( b ) . Finally, for part ( c ) (c) ( c ) , the factor inputs coincide, and therefore Proposition 2( c ) (c) ( c ) and Lemma 7( c ) (c) ( c ) apply directly.
The stochastic components retain the rates stated in parts ( a ) (a) ( a ) and ( b ) (b) ( b ) because the corresponding coefficients are tight for every fixed t t t . For the almost-sure components, the geometric-convolution bounds
∑ s = 1 t s 1 / q + ε χ k ⌊ ( t − s ) / k ⌋ χ k ⌊ s / ( 2 k ) ⌋ = O ( χ k ⌊ t / ( 4 k ) ⌋ ) , ∑ s = 1 t s 1 / q + ε χ k ⌊ ( t − s ) / k ⌋ ϱ k ⌊ s / ( 2 k ) ⌋ = O ( χ k ⌊ t / ( 4 k ) ⌋ + ϱ k ⌊ t / ( 4 k ) ⌋ ) \begin{aligned}
\sum_{s=1}^{t}s^{1/q+\varepsilon}\chi_k^{\lfloor(t-s)/k\rfloor}\chi_k^{\lfloor s/(2k)\rfloor}
&=O\left(\chi_k^{\lfloor t/(4k)\rfloor}\right),\\
\sum_{s=1}^{t}s^{1/q+\varepsilon}\chi_k^{\lfloor(t-s)/k\rfloor}\varrho_k^{\lfloor s/(2k)\rfloor}
&=O\left(\chi_k^{\lfloor t/(4k)\rfloor}+\varrho_k^{\lfloor t/(4k)\rfloor}\right)
\end{aligned} s = 1 ∑ t s 1/ q + ε χ k ⌊( t − s ) / k ⌋ χ k ⌊ s / ( 2 k )⌋ s = 1 ∑ t s 1/ q + ε χ k ⌊( t − s ) / k ⌋ ϱ k ⌊ s / ( 2 k )⌋ = O ( χ k ⌊ t / ( 4 k )⌋ ) , = O ( χ k ⌊ t / ( 4 k )⌋ + ϱ k ⌊ t / ( 4 k )⌋ )
give the rates stated in parts ( a ) (a) ( a ) –( c ) (c) ( c ) .
When evaluated using the stationary processes { H 0 F ˉ t } t ∈ Z \{H_0\bar F_t\}_{t\in\mathbb{Z}} { H 0 F ˉ t } t ∈ Z and { ψ ˉ t ∣ t − 1 ( θ ) } t ∈ Z \{\bar\psi_{t|t-1}(\theta)\}_{t\in\mathbb{Z}} { ψ ˉ t ∣ t − 1 ( θ ) } t ∈ Z , the second derivative recursion has stationary and ergodic coefficients. The k k k -step contraction established above and the moment bound in (265) therefore imply that it admits a unique causal strictly stationary and ergodic solution.
For any τ ≥ 1 \tau\ge1 τ ≥ 1 , construct the corresponding truncated second derivative recursion using the truncated factor, filter, and first derivative processes. Applying (266) , Lemma 6, and Lemma 7(d) shows that its L 2 L_2 L 2 -approximation error decreases geometrically in τ \tau τ . By construction, the truncated second derivative is a measurable function of v t , … , v t − τ \boldsymbol{v}_t,\ldots,\boldsymbol{v}_{t-\tau} v t , … , v t − τ . Consequently,
∥ ∇ θ θ ψ ˉ t ∣ t − 1 ( θ 0 ) − E [ ∇ θ θ ψ ˉ t ∣ t − 1 ( θ 0 ) ∣ v t , … , v t − τ ] ∥ 2 ≤ 2 ∥ ∇ θ θ ψ ˉ t ∣ t − 1 ( θ 0 ) − ∇ θ θ ψ ˉ t ∣ t − 1 ( τ ) ( θ 0 ) ∥ 2 , \left\| \nabla^{\theta\theta}\bar\psi_{t|t-1}(\theta_0)-\mathbb{E}\left[\nabla^{\theta\theta}\bar\psi_{t|t-1}(\theta_0)\mid\boldsymbol{v}_t,\ldots,\boldsymbol{v}_{t-\tau}\right]\right\|_2
\leq2\left\|\nabla^{\theta\theta}\bar\psi_{t|t-1}(\theta_0)-\nabla^{\theta\theta}\bar\psi_{t|t-1}^{(\tau)}(\theta_0)\right\|_2, ∇ θ θ ψ ˉ t ∣ t − 1 ( θ 0 ) − E [ ∇ θ θ ψ ˉ t ∣ t − 1 ( θ 0 ) ∣ v t , … , v t − τ ] 2 ≤ 2 ∇ θ θ ψ ˉ t ∣ t − 1 ( θ 0 ) − ∇ θ θ ψ ˉ t ∣ t − 1 ( τ ) ( θ 0 ) 2 ,
which also decreases geometrically in τ \tau τ . Hence, { ∇ θ θ ψ ˉ t ∣ t − 1 ( θ 0 ) } t ∈ Z \{\nabla^{\theta\theta}\bar\psi_{t|t-1}(\theta_0)\}_{t\in\mathbb{Z}} { ∇ θ θ ψ ˉ t ∣ t − 1 ( θ 0 ) } t ∈ Z is near-epoch dependent of size − 1 -1 − 1 on { v t } t ∈ Z \{\boldsymbol{v}_t\}_{t\in\mathbb{Z}} { v t } t ∈ Z .
Finally, part (e) follows from (264) , parts (a)--(c), and the triangle inequality.
Lemma 9. Consider the score vector
∇ θ ℓ F ( F t ; θ ) = ∇ θ ψ t ∣ t − 1 ( θ ) ⊤ ∇ ψ ℓ F ( F t ; θ ) + ∂ ℓ F ( F t ; θ ) ∂ θ , (267) \nabla^\theta\ell_F(F_t;\theta)=\nabla^\theta\psi_{t|t-1}(\theta)^\top\nabla^\psi\ell_F(F_t;\theta)+\frac{\partial\ell_F(F_t;\theta)}{\partial\theta}, \tag*{(267)} ∇ θ ℓ F ( F t ; θ ) = ∇ θ ψ t ∣ t − 1 ( θ ) ⊤ ∇ ψ ℓ F ( F t ; θ ) + ∂ θ ∂ ℓ F ( F t ; θ ) , (267) where the partial derivative in the second term is computed holding ψ t ∣ t − 1 \psi_{t|t-1} ψ t ∣ t − 1 and F t F_t F t fixed and is subsequently evaluated at ψ t ∣ t − 1 = ψ t ∣ t − 1 ( θ ) . \psi_{t|t-1}=\psi_{t|t-1}(\theta). ψ t ∣ t − 1 = ψ t ∣ t − 1 ( θ ) .
Under Assumptions 1--13, with p > 2 p>2 p > 2 , for every fixed initial condition ψ 1 ∣ 0 ∈ P \psi_{1|0}\in\mathcal{P} ψ 1∣0 ∈ P , as N , T → ∞ , N,T\to\infty, N , T → ∞ ,
sup θ ∈ Θ ∥ 1 T ∑ t = 1 T ∇ θ ℓ ^ F ( F ^ t ; r H ^ ( θ ) ) − E [ ∇ θ ℓ F ( F t ; θ ) ] ∥ = O P ( 1 min ( N , T ) ) , (268) \sup_{\theta\in\Theta}\left\|\frac{1}{T}\sum_{t=1}^{T}\nabla^\theta\widehat{\ell}_F\left(\widehat F_t;r_{\widehat H}(\theta)\right)-\mathbb{E}\left[\nabla^\theta\ell_F(F_t;\theta)\right]\right\|
=O_P\left(\frac{1}{\min(N,\sqrt{T})}\right), \tag*{(268)} θ ∈ Θ sup T 1 t = 1 ∑ T ∇ θ ℓ F ( F t ; r H ( θ ) ) − E [ ∇ θ ℓ F ( F t ; θ ) ] = O P ( min ( N , T ) 1 ) , (268) where the derivative of the first term is taken with respect to θ \theta θ through the mapping r H ^ ( θ ) r_{\widehat H}(\theta) r H ( θ ) .
Moreover,
1 T sup θ ∈ Θ ∥ ∑ t = 1 T ∇ θ ℓ ^ F ( F ^ t ; r H ^ ( θ ) ) − ∑ t = 1 T ∇ θ ℓ F ( F ˉ t ; θ ) ∥ = O P ( 1 N + T N + 1 T ) . (269) \frac{1}{\sqrt{T}}\sup_{\theta\in\Theta}\left\|\sum_{t=1}^{T}\nabla^\theta\widehat{\ell}_F\left(\widehat F_t;r_{\widehat H}(\theta)\right)-\sum_{t=1}^{T}\nabla^\theta\ell_F\left(\bar F_t;\theta\right)\right\|
=O_P\left(\frac{1}{\sqrt{N}}+\frac{\sqrt{T}}{N}+\frac{1}{\sqrt{T}}\right). \tag*{(269)} T 1 θ ∈ Θ sup t = 1 ∑ T ∇ θ ℓ F ( F t ; r H ( θ ) ) − t = 1 ∑ T ∇ θ ℓ F ( F ˉ t ; θ ) = O P ( N 1 + N T + T 1 ) . (269) Consequently, if T / N → 0 \sqrt{T}/N\to0 T / N → 0 , then, as N , T → ∞ , N,T\to\infty, N , T → ∞ ,
1 T sup θ ∈ Θ ∥ ∑ t = 1 T ∇ θ ℓ ^ F ( F ^ t ; r H ^ ( θ ) ) − ∑ t = 1 T ∇ θ ℓ F ( F ˉ t ; θ ) ∥ = o P ( 1 ) . (270) \frac{1}{\sqrt{T}}\sup_{\theta\in\Theta}\left\|\sum_{t=1}^{T}\nabla^\theta\widehat{\ell}_F\left(\widehat F_t;r_{\widehat H}(\theta)\right)-\sum_{t=1}^{T}\nabla^\theta\ell_F\left(\bar F_t;\theta\right)\right\|=o_P(1). \tag*{(270)} T 1 θ ∈ Θ sup t = 1 ∑ T ∇ θ ℓ F ( F t ; r H ( θ ) ) − t = 1 ∑ T ∇ θ ℓ F ( F ˉ t ; θ ) = o P ( 1 ) . (270)
Finally, the stationary score process { ∇ θ ℓ F ( F ‾ t ; θ 0 ) } t ∈ Z \left\{\nabla^\theta\ell_F\left(\overline{F}_t;\theta_0\right)\right\}_{t\in\mathbb{Z}} { ∇ θ ℓ F ( F t ; θ 0 ) } t ∈ Z has mean zero and is near-epoch dependent of size − 1 -1 − 1 on the geometrically β \beta β -mixing process { v t } t ∈ Z \{v_t\}_{t\in\mathbb{Z}} { v t } t ∈ Z .
Proof. Similarly to the proof of Proposition 3, an application of the triangle inequality shows that
sup θ ∈ Θ ∥ 1 T ∑ t = 1 T ∇ θ ℓ ^ F ( F ^ t ; r H ^ ( θ ) ) − E [ ∇ θ ℓ F ( F ‾ t ; θ ) ] ∥ ≤ sup θ ∈ Θ ∥ 1 T ∑ t = 1 T ∇ θ ℓ ^ F ( F ^ t ; r H ^ ( θ ) ) − 1 T ∑ t = 1 T ∇ θ ℓ ^ F ( F t H ^ T ; r H ^ ( θ ) ) ∥ + sup θ ∈ Θ ∥ 1 T ∑ t = 1 T ∇ θ ℓ ^ F ( F t H ^ T ; r H ^ ( θ ) ) − 1 T ∑ t = 1 T ∇ θ ℓ F ( F t ; θ ) ∥ + sup θ ∈ Θ ∥ 1 T ∑ t = 1 T ∇ θ ℓ F ( F t ; θ ) − 1 T ∑ t = 1 T ∇ θ ℓ F ( F ‾ t ; θ ) ∥ + sup θ ∈ Θ ∥ 1 T ∑ t = 1 T ∇ θ ℓ F ( F ‾ t ; θ ) − E [ ∇ θ ℓ F ( F ‾ t ; θ ) ] ∥ . (271) \begin{aligned}
&\sup_{\theta\in\Theta}\left\Vert\frac{1}{T}\sum_{t=1}^{T}\nabla^{\theta}\widehat{\ell}_F\left(\widehat{F}_t;r_{\widehat{H}}(\theta)\right)-\mathbb{E}\left[\nabla^{\theta}\ell_F\left(\overline{F}_t;\theta\right)\right]\right\Vert\\
&\leq\sup_{\theta\in\Theta}\left\Vert\frac{1}{T}\sum_{t=1}^{T}\nabla^{\theta}\widehat{\ell}_F\left(\widehat{F}_t;r_{\widehat{H}}(\theta)\right)-\frac{1}{T}\sum_{t=1}^{T}\nabla^{\theta}\widehat{\ell}_F\left(F_t\widehat{H}^{\mathsf{T}};r_{\widehat{H}}(\theta)\right)\right\Vert\\
&\quad+\sup_{\theta\in\Theta}\left\Vert\frac{1}{T}\sum_{t=1}^{T}\nabla^{\theta}\widehat{\ell}_F\left(F_t\widehat{H}^{\mathsf{T}};r_{\widehat{H}}(\theta)\right)-\frac{1}{T}\sum_{t=1}^{T}\nabla^{\theta}\ell_F\left(F_t;\theta\right)\right\Vert\\
&\quad+\sup_{\theta\in\Theta}\left\Vert\frac{1}{T}\sum_{t=1}^{T}\nabla^{\theta}\ell_F\left(F_t;\theta\right)-\frac{1}{T}\sum_{t=1}^{T}\nabla^{\theta}\ell_F\left(\overline{F}_t;\theta\right)\right\Vert\\
&\quad+\sup_{\theta\in\Theta}\left\Vert\frac{1}{T}\sum_{t=1}^{T}\nabla^{\theta}\ell_F\left(\overline{F}_t;\theta\right)-\mathbb{E}\left[\nabla^{\theta}\ell_F\left(\overline{F}_t;\theta\right)\right]\right\Vert. \tag*{(271)}
\end{aligned} θ ∈ Θ sup T 1 t = 1 ∑ T ∇ θ ℓ F ( F t ; r H ( θ ) ) − E [ ∇ θ ℓ F ( F t ; θ ) ] ≤ θ ∈ Θ sup T 1 t = 1 ∑ T ∇ θ ℓ F ( F t ; r H ( θ ) ) − T 1 t = 1 ∑ T ∇ θ ℓ F ( F t H T ; r H ( θ ) ) + θ ∈ Θ sup T 1 t = 1 ∑ T ∇ θ ℓ F ( F t H T ; r H ( θ ) ) − T 1 t = 1 ∑ T ∇ θ ℓ F ( F t ; θ ) + θ ∈ Θ sup T 1 t = 1 ∑ T ∇ θ ℓ F ( F t ; θ ) − T 1 t = 1 ∑ T ∇ θ ℓ F ( F t ; θ ) + θ ∈ Θ sup T 1 t = 1 ∑ T ∇ θ ℓ F ( F t ; θ ) − E [ ∇ θ ℓ F ( F t ; θ ) ] . (271)
For the likelihoods evaluated at r H ^ ( θ ) r_{\widehat{H}}(\theta) r H ( θ ) , the derivatives in (271) are taken with respect to θ \theta θ through the mapping r H ( ⋅ ) r_H(\cdot) r H ( ⋅ ) . The four terms on the right-hand side of (271) account, respectively, for factor estimation and its recursive propagation through the filter, the effect of the possibly different filter initializations under the alignment mapping, convergence to the stationary factor and filter processes, and the sampling variation of the stationary score.
Controlling the first term in (271) .
In the following, we use the notation
∇ θ ψ ′ ℓ F ( F t ; θ ) = ∂ ∇ θ ℓ F ( F t ; θ ) ∂ vec ∇ θ ψ t ∣ t − 1 ( θ ) T , ∇ θ ψ ℓ F ( F t ; θ ) = ∂ ∇ θ ℓ F ( F t ; θ ) ∂ ψ t ∣ t − 1 ( θ ) T , ∇ θ F ℓ F ( F t ; θ ) = ∂ ∇ θ ℓ F ( F t ; θ ) ∂ F t T . \begin{aligned}
\nabla^{\theta\psi'}\ell_F(F_t;\theta)&=\frac{\partial\nabla^\theta\ell_F(F_t;\theta)}{\partial\operatorname{vec}\nabla^\theta\psi_{t|t-1}(\theta)^{\mathsf{T}}},\\
\nabla^{\theta\psi}\ell_F(F_t;\theta)&=\frac{\partial\nabla^\theta\ell_F(F_t;\theta)}{\partial\psi_{t|t-1}(\theta)^{\mathsf{T}}},\\
\nabla^{\theta F}\ell_F(F_t;\theta)&=\frac{\partial\nabla^\theta\ell_F(F_t;\theta)}{\partial F_t^{\mathsf{T}}}.
\end{aligned} ∇ θ ψ ′ ℓ F ( F t ; θ ) ∇ θ ψ ℓ F ( F t ; θ ) ∇ θ F ℓ F ( F t ; θ ) = ∂ vec ∇ θ ψ t ∣ t − 1 ( θ ) T ∂ ∇ θ ℓ F ( F t ; θ ) , = ∂ ψ t ∣ t − 1 ( θ ) T ∂ ∇ θ ℓ F ( F t ; θ ) , = ∂ F t T ∂ ∇ θ ℓ F ( F t ; θ ) .
Regarding the full score contribution as a function of ∇ θ ψ t ∣ t − 1 \nabla^\theta\psi_{t|t-1} ∇ θ ψ t ∣ t − 1 , ψ t ∣ t − 1 \psi_{t|t-1} ψ t ∣ t − 1 , and F t F_t F t , a first-order expansion around ( ∇ θ ψ ^ t ∣ t − 1 ( r H ^ ( θ ) ) , ψ ^ t ∣ t − 1 ( r H ^ ( θ ) ) , H ^ F t ) \left(\nabla^\theta\widehat{\psi}_{t|t-1}\left(r_{\widehat{H}}(\theta)\right),\widehat{\psi}_{t|t-1}\left(r_{\widehat{H}}(\theta)\right),\widehat{H}F_t\right) ( ∇ θ ψ t ∣ t − 1 ( r H ( θ ) ) , ψ t ∣ t − 1 ( r H ( θ ) ) , H F t ) shows that, uniformly over θ ∈ Θ \theta\in\Theta θ ∈ Θ ,
1 T ( ∑ t = 1 T ∇ θ ℓ ^ F ( F ^ t ; r H ^ ( θ ) ) − ∑ t = 1 T ∇ θ ℓ ^ F ( F t H ^ T ; r H ^ ( θ ) ) ) = 1 T ∑ t = 1 T ( ∇ θ ℓ ^ F ( F ^ t ; r H ^ ( θ ) ) − ∇ θ ℓ ^ F ( H ^ F t ; r H ^ ( θ ) ) ) = 1 T ∑ t = 1 T ∇ θ ψ ℓ ^ F ′ ( H ^ F t ; r H ^ ( θ ) ) vec ( ∇ θ ψ ^ t ∣ t − 1 ( r H ^ ( θ ) ) − ∇ θ ψ t ∣ t − 1 ( r H ^ ( θ ) ) ) + 1 T ∑ t = 1 T ∇ θ ϕ ℓ ^ F ( H ^ F t ; r H ^ ( θ ) ) ( ψ ^ t ∣ t − 1 ( r H ^ ( θ ) ) − ψ ^ t ∣ t − 1 ( r H ^ ( θ ) ) ) + 1 T ∑ t = 1 T ∇ θ F ℓ ^ F ( H ^ F t ; r H ^ ( θ ) ) ( F ^ t − H ^ F t ) + 1 T ∑ t = 1 T R t , N T ′ ( θ ) . (272) \begin{aligned}
\frac{1}{T}\left(\sum_{t=1}^{T}\nabla^\theta\widehat{\ell}_F\left(\widehat{F}_t;r_{\widehat{H}}(\theta)\right)-\sum_{t=1}^{T}\nabla^\theta\widehat{\ell}_F\left(F_t\widehat{H}^{\mathsf{T}};r_{\widehat{H}}(\theta)\right)\right) \\
&= \frac{1}{T}\sum_{t=1}^{T}\left(\nabla^\theta\widehat{\ell}_{F}\left(\widehat{F}_{t};r_{\widehat{H}}(\theta)\right)-\nabla^\theta\widehat{\ell}_{F}\left(\widehat{H}F_{t};r_{\widehat{H}}(\theta)\right)\right)
\\
= \frac{1}{T}\sum_{t=1}^{T}\nabla^\theta\psi'_{\widehat{\ell}_{F}}\left(\widehat{H}F_{t};r_{\widehat{H}}(\theta)\right)\operatorname{vec}\left(\nabla^\theta\widehat{\psi}_{t|t-1}\left(r_{\widehat{H}}(\theta)\right)-\nabla^\theta\psi_{t|t-1}\left(r_{\widehat{H}}(\theta)\right)\right)
\\
+ \frac{1}{T}\sum_{t=1}^{T}\nabla^{\theta\phi}\widehat{\ell}_{F}\left(\widehat{H}F_{t};r_{\widehat{H}}(\theta)\right)\left(\widehat{\psi}_{t|t-1}\left(r_{\widehat{H}}(\theta)\right)-\widehat{\psi}_{t|t-1}\left(r_{\widehat{H}}(\theta)\right)\right)
\\
+ \frac{1}{T}\sum_{t=1}^{T}\nabla^{\theta F}\widehat{\ell}_{F}\left(\widehat{H}F_{t};r_{\widehat{H}}(\theta)\right)\left(\widehat{F}_{t}-\widehat{H}F_{t}\right)
\\
+ \frac{1}{T}\sum_{t=1}^{T}R'_{t,NT}(\theta). \tag*{(272)}
\end{aligned} T 1 ( t = 1 ∑ T ∇ θ ℓ F ( F t ; r H ( θ ) ) − t = 1 ∑ T ∇ θ ℓ F ( F t H T ; r H ( θ ) ) ) = T 1 t = 1 ∑ T ∇ θ ψ ℓ F ′ ( H F t ; r H ( θ ) ) vec ( ∇ θ ψ t ∣ t − 1 ( r H ( θ ) ) − ∇ θ ψ t ∣ t − 1 ( r H ( θ ) ) ) + T 1 t = 1 ∑ T ∇ θ ϕ ℓ F ( H F t ; r H ( θ ) ) ( ψ t ∣ t − 1 ( r H ( θ ) ) − ψ t ∣ t − 1 ( r H ( θ ) ) ) + T 1 t = 1 ∑ T ∇ θ F ℓ F ( H F t ; r H ( θ ) ) ( F t − H F t ) + T 1 t = 1 ∑ T R t , N T ′ ( θ ) . = T 1 t = 1 ∑ T ( ∇ θ ℓ F ( F t ; r H ( θ ) ) − ∇ θ ℓ F ( H F t ; r H ( θ ) ) ) (272)
By the uniform second-order smoothness in Assumption 11, the multivariate remainder R t , N T ′ ( θ ) R'_{t,NT}(\theta) R t , N T ′ ( θ ) is quadratic in the factor, filter, and first-derivative-filter perturbations. Moreover, the argument used in the proof of Lemma 7(a) shows that the first derivative filter perturbation inherits the average squared rate of the factor and filter perturbations. Therefore,
1 T ∑ t = 1 T sup θ ∈ Θ ∥ ∇ θ ψ ^ t ∣ t − 1 ( r H ^ ( θ ) ) − ∇ θ ψ t ∣ t − 1 ( r H ^ ( θ ) ) ∥ 2 = O P ( 1 N + 1 T ) . (273) \frac{1}{T}\sum_{t=1}^{T}\sup_{\theta\in\Theta}\left\|\nabla^\theta\widehat{\psi}_{t|t-1}\left(r_{\widehat{H}}(\theta)\right)-\nabla^\theta\psi_{t|t-1}\left(r_{\widehat{H}}(\theta)\right)\right\|^{2}=O_{\mathbb{P}}\left(\frac{1}{N}+\frac{1}{T}\right). \tag*{(273)} T 1 t = 1 ∑ T θ ∈ Θ sup ∇ θ ψ t ∣ t − 1 ( r H ( θ ) ) − ∇ θ ψ t ∣ t − 1 ( r H ( θ ) ) 2 = O P ( N 1 + T 1 ) . (273)
Indeed, the stochastic component in Lemma 7(a) is generated by the factor and filter perturbations, whose average squared rates are given in (78) and (79) , whereas its geometrically decreasing initialization component has average squared order O ( T − 1 ) O\left(T^{-1}\right) O ( T − 1 ) . Hence, from the Cauchy–Schwarz inequality, (78) , (79) , and (273) we get
sup θ ∈ Θ ∥ 1 T ∑ t = 1 T R t , N T ′ ( θ ) ∥ ≤ 1 T ∑ t = 1 T sup θ ∈ Θ ∥ R t , N T ′ ( θ ) ∥ = O P ( 1 N + 1 T ) . \sup_{\theta\in\Theta}\left\|\frac{1}{T}\sum_{t=1}^{T}R'_{t,NT}(\theta)\right\|\leq\frac{1}{T}\sum_{t=1}^{T}\sup_{\theta\in\Theta}\left\|R'_{t,NT}(\theta)\right\|=O_{\mathbb{P}}\left(\frac{1}{N}+\frac{1}{T}\right). θ ∈ Θ sup T 1 t = 1 ∑ T R t , N T ′ ( θ ) ≤ T 1 t = 1 ∑ T θ ∈ Θ sup R t , N T ′ ( θ ) = O P ( N 1 + T 1 ) .
Given this result, we first expand the two recursive differences appearing in the first two terms of (272) . From (83) , we know that for every t ≥ 2 t\geq2 t ≥ 2 ,
ψ ^ t ∣ t − 1 ( r H ^ ( θ ) ) − ψ t ∣ t − 1 ( r H ^ ( θ ) ) = ( ∏ h = 1 t − 1 X t − h ( r H ^ ( θ ) ) ) ( ψ 1 ∣ 0 − ψ 1 ∣ 0 ⋆ ) + ∑ s = 1 t − 1 ( ∏ h = s + 1 t − 1 X h ( r H ^ ( θ ) ) ) Y s ( r H ^ ( θ ) ) ( F ^ s − H ^ F s ) + second-order terms . (274) \widehat{\psi}_{t|t-1}\left(r_{\widehat{H}}(\theta)\right)-\psi_{t|t-1}\left(r_{\widehat{H}}(\theta)\right)
\\
=\left(\prod_{h=1}^{t-1}X_{t-h}\left(r_{\widehat{H}}(\theta)\right)\right)\left(\psi_{1|0}-\psi^{\star}_{1|0}\right)+\sum_{s=1}^{t-1}\left(\prod_{h=s+1}^{t-1}X_{h}\left(r_{\widehat{H}}(\theta)\right)\right)Y_{s}\left(r_{\widehat{H}}(\theta)\right)\left(\widehat{F}_{s}-\widehat{H}F_{s}\right)
\\
+\text{ second-order terms}. \tag*{(274)} ψ t ∣ t − 1 ( r H ( θ ) ) − ψ t ∣ t − 1 ( r H ( θ ) ) = ( h = 1 ∏ t − 1 X t − h ( r H ( θ ) ) ) ( ψ 1∣0 − ψ 1∣0 ⋆ ) + s = 1 ∑ t − 1 ( h = s + 1 ∏ t − 1 X h ( r H ( θ ) ) ) Y s ( r H ( θ ) ) ( F s − H F s ) + second-order terms . (274)
Analogously, from (221) , another first-order expansion around ( ∇ θ ψ ^ t ∣ t − 1 ( r H ^ ( θ ) ) , ψ ^ t ∣ t − 1 ( r H ^ ( θ ) ) , H ^ F t ) \left(\nabla^\theta\widehat{\psi}_{t|t-1}\left(r_{\widehat{H}}(\theta)\right),\widehat{\psi}_{t|t-1}\left(r_{\widehat{H}}(\theta)\right),\widehat{H}F_t\right) ( ∇ θ ψ t ∣ t − 1 ( r H ( θ ) ) , ψ t ∣ t − 1 ( r H ( θ ) ) , H F t ) reveals that
vec ( ∇ θ ψ ^ t + 1 ∣ t ( r ^ H ( θ ) ) − ∇ θ ψ ^ t + 1 ∣ t ( r ^ H ( θ ) ) ) = ( I d ⊗ X t ( r ^ H ( θ ) ) ) vec ( ∇ θ ψ ^ t ∣ t − 1 ( r ^ H ( θ ) ) − ∇ θ ψ ^ t ∣ t − 1 ( r ^ H ( θ ) ) ) + { ( ∇ θ ψ ^ t ∣ t − 1 ( r ^ H ( θ ) ) ) T ⊗ I r } ∂ vec X t ( r ^ H ( θ ) ) ∂ ψ ^ t ∣ t − 1 ( r ^ H ( θ ) ) T + ∂ vec D t ( r ^ H ( θ ) ) ∂ ψ ^ t ∣ t − 1 ( r ^ H ( θ ) ) T { ψ ^ t ∣ t − 1 ( r ^ H ( θ ) ) − ψ ^ t ∣ t − 1 ( r ^ H ( θ ) ) } + { ( ∇ θ ψ ^ t ∣ t − 1 ( r ^ H ( θ ) ) ) T ⊗ I r } ∂ vec X t ( r ^ H ( θ ) ) ∂ ( H ^ F t ) T + ∂ vec D t ( r ^ H ( θ ) ) ∂ ( H ^ F t ) T ( F ^ t − H ^ F ^ t ) + second-order terms . (275) \begin{aligned}
\operatorname{vec}\left(\nabla^\theta\widehat{\psi}_{t+1|t}\left(\widehat{r}_{H}(\theta)\right)-\nabla^\theta\widehat{\psi}_{t+1|t}\left(\widehat{r}_{H}(\theta)\right)\right)
={}&\left(I_d\otimes X_t\left(\widehat{r}_{H}(\theta)\right)\right)\operatorname{vec}\left(\nabla^\theta\widehat{\psi}_{t|t-1}\left(\widehat{r}_{H}(\theta)\right)-\nabla^\theta\widehat{\psi}_{t|t-1}\left(\widehat{r}_{H}(\theta)\right)\right)\\
&+\left\{\left(\nabla^\theta\widehat{\psi}_{t|t-1}\left(\widehat{r}_{H}(\theta)\right)\right)^\mathsf{T}\otimes I_r\right\}\frac{\partial\operatorname{vec}X_t\left(\widehat{r}_{H}(\theta)\right)}{\partial\widehat{\psi}_{t|t-1}\left(\widehat{r}_{H}(\theta)\right)^\mathsf{T}}\\
&\qquad+\frac{\partial\operatorname{vec}D_t\left(\widehat{r}_{H}(\theta)\right)}{\partial\widehat{\psi}_{t|t-1}\left(\widehat{r}_{H}(\theta)\right)^\mathsf{T}}\left\{\widehat{\psi}_{t|t-1}\left(\widehat{r}_{H}(\theta)\right)-\widehat{\psi}_{t|t-1}\left(\widehat{r}_{H}(\theta)\right)\right\}\\
&+\left\{\left(\nabla^\theta\widehat{\psi}_{t|t-1}\left(\widehat{r}_{H}(\theta)\right)\right)^\mathsf{T}\otimes I_r\right\}\frac{\partial\operatorname{vec}X_t\left(\widehat{r}_{H}(\theta)\right)}{\partial\left(\widehat{H}F_t\right)^\mathsf{T}}\\
&\qquad+\frac{\partial\operatorname{vec}D_t\left(\widehat{r}_{H}(\theta)\right)}{\partial\left(\widehat{H}F_t\right)^\mathsf{T}}\left(\widehat{F}_t-\widehat{H}\widehat{F}_t\right)\\
&+\text{second-order terms}. \tag*{(275)}
\end{aligned} vec ( ∇ θ ψ t + 1∣ t ( r H ( θ ) ) − ∇ θ ψ t + 1∣ t ( r H ( θ ) ) ) = ( I d ⊗ X t ( r H ( θ ) ) ) vec ( ∇ θ ψ t ∣ t − 1 ( r H ( θ ) ) − ∇ θ ψ t ∣ t − 1 ( r H ( θ ) ) ) + { ( ∇ θ ψ t ∣ t − 1 ( r H ( θ ) ) ) T ⊗ I r } ∂ ψ t ∣ t − 1 ( r H ( θ ) ) T ∂ vec X t ( r H ( θ ) ) + ∂ ψ t ∣ t − 1 ( r H ( θ ) ) T ∂ vec D t ( r H ( θ ) ) { ψ t ∣ t − 1 ( r H ( θ ) ) − ψ t ∣ t − 1 ( r H ( θ ) ) } + { ( ∇ θ ψ t ∣ t − 1 ( r H ( θ ) ) ) T ⊗ I r } ∂ ( H F t ) T ∂ vec X t ( r H ( θ ) ) + ∂ ( H F t ) T ∂ vec D t ( r H ( θ ) ) ( F t − H F t ) + second-order terms . (275)
Since the two initial conditions are fixed and do not depend on θ \theta θ , ∇ θ ψ 1 ∣ 0 = ∇ θ ψ 1 ∣ 0 ⋆ = 0 r × d \nabla^\theta\psi_{1|0}=\nabla^\theta\psi^\star_{1|0}=\mathbf{0}_{r\times d} ∇ θ ψ 1∣0 = ∇ θ ψ 1∣0 ⋆ = 0 r × d . Consequently, iterating (275) gives, for every t ≥ 2 t\ge2 t ≥ 2 ,
vec ∇ θ ψ ^ t ∣ t − 1 ( r ^ H ( θ ) ) − vec ∇ θ ψ ^ t ∣ t − 1 ( r ^ H ( θ ) ) = ∑ s = 1 t − 1 ( ∏ h = s + 1 t − 1 ( I d ⊗ X h ( r ^ H ( θ ) ) ) ) × { ( ∇ θ ψ ^ s ∣ s − 1 ( r ^ H ( θ ) ) ) T ⊗ I r } ∂ vec X s ( r ^ H ( θ ) ) ∂ ψ ^ s ∣ s − 1 ( r ^ H ( θ ) ) T + ∂ vec D s ( r ^ H ( θ ) ) ∂ ψ ^ s ∣ s − 1 ( r ^ H ( θ ) ) T { ψ ^ s ∣ s − 1 ( r ^ H ( θ ) ) − ψ ^ s ∣ s − 1 ( r ^ H ( θ ) ) } + ∑ s = 1 t − 1 ( ∏ h = s + 1 t − 1 ( I d ⊗ X h ( r ^ H ( θ ) ) ) ) × { ( ∇ θ ψ ^ s ∣ s − 1 ( r ^ H ( θ ) ) ) T ⊗ I r } ∂ vec X s ( r ^ H ( θ ) ) ∂ ( H ^ F s ) T + ∂ D ⃗ s ( r H ^ ( θ ) ) ∂ ( H ^ F s ) ( F ^ s − H ^ F s ) \begin{aligned}
\operatorname{vec}\nabla^\theta\widehat{\psi}_{t|t-1}\left(\widehat{r}_{H}(\theta)\right)-\operatorname{vec}\nabla^\theta\widehat{\psi}_{t|t-1}\left(\widehat{r}_{H}(\theta)\right)
={}&\sum_{s=1}^{t-1}\left(\prod_{h=s+1}^{t-1}\left(I_d\otimes X_h\left(\widehat{r}_{H}(\theta)\right)\right)\right)\\
&\times\left\{\left(\nabla^\theta\widehat{\psi}_{s|s-1}\left(\widehat{r}_{H}(\theta)\right)\right)^\mathsf{T}\otimes I_r\right\}\frac{\partial\operatorname{vec}X_s\left(\widehat{r}_{H}(\theta)\right)}{\partial\widehat{\psi}_{s|s-1}\left(\widehat{r}_{H}(\theta)\right)^\mathsf{T}}\\
&\qquad+\frac{\partial\operatorname{vec}D_s\left(\widehat{r}_{H}(\theta)\right)}{\partial\widehat{\psi}_{s|s-1}\left(\widehat{r}_{H}(\theta)\right)^\mathsf{T}}\left\{\widehat{\psi}_{s|s-1}\left(\widehat{r}_{H}(\theta)\right)-\widehat{\psi}_{s|s-1}\left(\widehat{r}_{H}(\theta)\right)\right\}\\
&+\sum_{s=1}^{t-1}\left(\prod_{h=s+1}^{t-1}\left(I_d\otimes X_h\left(\widehat{r}_{H}(\theta)\right)\right)\right)\\
&\times\left\{\left(\nabla^\theta\widehat{\psi}_{s|s-1}\left(\widehat{r}_{H}(\theta)\right)\right)^\mathsf{T}\otimes I_r\right\}\frac{\partial\operatorname{vec}X_s\left(\widehat{r}_{H}(\theta)\right)}{\partial\left(\widehat{H}F_s\right)^\mathsf{T}} \\
&+ \frac{\partial\vec{D}_s\left(r_{\widehat{H}}(\theta)\right)}{\partial\left(\widehat{H}F_s\right)}\left(\widehat{F}_s-\widehat{H}F_s\right)
\end{aligned} vec ∇ θ ψ t ∣ t − 1 ( r H ( θ ) ) − vec ∇ θ ψ t ∣ t − 1 ( r H ( θ ) ) = s = 1 ∑ t − 1 ( h = s + 1 ∏ t − 1 ( I d ⊗ X h ( r H ( θ ) ) ) ) × { ( ∇ θ ψ s ∣ s − 1 ( r H ( θ ) ) ) T ⊗ I r } ∂ ψ s ∣ s − 1 ( r H ( θ ) ) T ∂ vec X s ( r H ( θ ) ) + ∂ ψ s ∣ s − 1 ( r H ( θ ) ) T ∂ vec D s ( r H ( θ ) ) { ψ s ∣ s − 1 ( r H ( θ ) ) − ψ s ∣ s − 1 ( r H ( θ ) ) } + s = 1 ∑ t − 1 ( h = s + 1 ∏ t − 1 ( I d ⊗ X h ( r H ( θ ) ) ) ) × { ( ∇ θ ψ s ∣ s − 1 ( r H ( θ ) ) ) T ⊗ I r } ∂ ( H F s ) T ∂ vec X s ( r H ( θ ) ) + ∂ ( H F s ) ∂ D s ( r H ( θ ) ) ( F s − H F s )
All products in (274) and (278) are ordered with the largest time index on the left. An empty product in (274) is equal to I r I_r I r , whereas an empty product in (278) is equal to I r d I_{rd} I r d .
Since F ^ s − H ^ F s \widehat{F}_s-\widehat{H}F_s F s − H F s does not depend on θ \theta θ , the linear terms obtained by substituting (274) and (278) into (272) , and then interchanging the resulting time sums, coincide with the derivative with respect to θ \theta θ of the effective likelihood weight defined in (86) . Accordingly, for every H ∈ H \mathbf{H}\in\mathcal{H} H ∈ H , define
S s , T ( θ , H ) : = ∇ θ L s , T ( θ , H ) = ∇ θ [ ∇ F ℓ ^ F ( H F s ; r H ( θ ) ) + Y s ( r H ( θ ) ) ⊤ ∑ t = s + 1 T ( ∏ h = s + 1 t − 1 X h ( r H ( θ ) ) ) ⊤ ∇ ψ ℓ ^ F ( H F t ; r H ( θ ) ) ] . (276) \begin{aligned}
\mathbf{S}_{s,T}(\theta,\mathbf{H}) := \nabla^\theta\mathcal{L}_{s,T}(\theta,\mathbf{H}) \\
&= \nabla^\theta\left[\nabla^F\widehat{\ell}_F\left(\mathbf{H}F_s;r_{\mathbf{H}}(\theta)\right)
+Y_s\left(r_{\mathbf{H}}(\theta)\right)^\top
\sum_{t=s+1}^{T}
\left(\prod_{h=s+1}^{t-1}X_h\left(r_{\mathbf{H}}(\theta)\right)\right)^\top
\nabla^\psi\widehat{\ell}_F\left(\mathbf{H}F_t;r_{\mathbf{H}}(\theta)\right)\right]. \tag*{(276)}
\end{aligned} S s , T ( θ , H ) := ∇ θ L s , T ( θ , H ) = ∇ θ ∇ F ℓ F ( H F s ; r H ( θ ) ) + Y s ( r H ( θ ) ) ⊤ t = s + 1 ∑ T ( h = s + 1 ∏ t − 1 X h ( r H ( θ ) ) ) ⊤ ∇ ψ ℓ F ( H F t ; r H ( θ ) ) . (276)
Here, the derivative in (276) acts on every occurrence of r H ( θ ) r_{\mathbf{H}}(\theta) r H ( θ ) , including its occurrence through the filtered process, X h \mathbf{X}_h X h , Y s \mathbf{Y}_s Y s , and the derivatives of the likelihood. Thus, the product-rule expansion of S s , T ( θ , H ) \mathbf{S}_{s,T}(\theta,\mathbf{H}) S s , T ( θ , H ) contains exactly the linear terms generated by the filter and first-derivative-filter expansions above.
It follows that
1 T ( ∑ t = 1 T ∇ θ ℓ ^ F ( F ^ t ; r H ^ ( θ ) ) − ∑ t = 1 T ∇ θ ℓ ^ F ( F t H ^ ⊤ ; r H ^ ( θ ) ) ) = 1 T ∑ s = 1 T S s , T ( θ , H ^ ) ⊤ ( F ^ s − H ^ F s ) + O P ( 1 N + 1 T ) , (277) \begin{aligned}
\frac{1}{T}\left(\sum_{t=1}^{T}\nabla^\theta\widehat{\ell}_F\left(\widehat{F}_t;r_{\widehat{H}}(\theta)\right)-\sum_{t=1}^{T}\nabla^\theta\widehat{\ell}_F\left(F_t\widehat{H}^{\top};r_{\widehat{H}}(\theta)\right)\right) \\
&=\frac{1}{T}\sum_{s=1}^{T}\mathbf{S}_{s,T}(\theta,\widehat{\mathbf{H}})^\top\left(\widehat{F}_s-\widehat{H}F_s\right)+O_{\mathbb{P}}\left(\frac{1}{N}+\frac{1}{T}\right), \tag*{(277)}
\end{aligned} T 1 ( t = 1 ∑ T ∇ θ ℓ F ( F t ; r H ( θ ) ) − t = 1 ∑ T ∇ θ ℓ F ( F t H ⊤ ; r H ( θ ) ) ) = T 1 s = 1 ∑ T S s , T ( θ , H ) ⊤ ( F s − H F s ) + O P ( N 1 + T 1 ) , (277)
uniformly over θ ∈ Θ \theta\in\Theta θ ∈ Θ . By substituting the PC representation in (42) into (277) we therefore obtain
1 T ( ∑ t = 1 T ∇ θ ℓ ^ F ( F ^ t ; r H ^ ( θ ) ) − ∑ t = 1 T ∇ θ ℓ ^ F ( F t H ^ ⊤ ; r H ^ ( θ ) ) ) = 1 T ∑ s = 1 T S s , T ( θ , H ^ ) ⊤ ( M ^ N ) − 1 ( A s + B s + C s + D s ) + O P ( 1 N + 1 T ) . (278) \begin{aligned}
\frac{1}{T}\left(\sum_{t=1}^{T}\nabla^\theta\widehat{\ell}_F\left(\widehat{F}_t;r_{\widehat{H}}(\theta)\right)-\sum_{t=1}^{T}\nabla^\theta\widehat{\ell}_F\left(F_t\widehat{H}^{\top};r_{\widehat{H}}(\theta)\right)\right) \\
&=\frac{1}{T}\sum_{s=1}^{T}\mathbf{S}_{s,T}(\theta,\widehat{\mathbf{H}})^\top\left(\frac{\widehat{M}}{N}\right)^{-1}(A_s+B_s+C_s+D_s)+O_{\mathbb{P}}\left(\frac{1}{N}+\frac{1}{T}\right). \tag*{(278)}
\end{aligned} T 1 ( t = 1 ∑ T ∇ θ ℓ F ( F t ; r H ( θ ) ) − t = 1 ∑ T ∇ θ ℓ F ( F t H ⊤ ; r H ( θ ) ) ) = T 1 s = 1 ∑ T S s , T ( θ , H ) ⊤ ( N M ) − 1 ( A s + B s + C s + D s ) + O P ( N 1 + T 1 ) . (278)
Now, By Assumption 11, Lemmas 7(d) and 8(d), and the geometric summability of the products of X t \mathbf{X}_t X t , it holds that
sup T ≥ 1 sup 1 ≤ s ≤ T E [ sup H ∈ H sup θ ∈ Θ ∥ S s , T ( θ , H ) ∥ 2 ] < ∞ . (279) \sup_{T\geq1}\sup_{1\leq s\leq T}\mathbb{E}\left[\sup_{\mathbf{H}\in\mathcal{H}}\sup_{\theta\in\Theta}\left\|\mathbf{S}_{s,T}(\theta,\mathbf{H})\right\|^2\right]<\infty. \tag*{(279)} T ≥ 1 sup 1 ≤ s ≤ T sup E [ H ∈ H sup θ ∈ Θ sup ∥ S s , T ( θ , H ) ∥ 2 ] < ∞. (279)
The same conditions, together with the compactness of Θ \Theta Θ and H \mathcal{H} H , imply that S s , T ( θ , H ) \mathbf{S}_{s,T}(\theta,\mathbf{H}) S s , T ( θ , H ) is locally Lipschitz in θ \theta θ and H \mathbf{H} H , uniformly over Θ × H \Theta\times\mathcal{H} Θ × H , with a random Lipschitz coefficient having a bounded second moment. Consequently, the arguments used in (99) , (103) , (105) , and (110) apply with G s , T \mathbf{G}_{s,T} G s , T replaced by S s , T \mathbf{S}_{s,T} S s , T , since d d d is fixed. By combining those bounds with (278) we get
sup θ ∈ Θ ∥ 1 T ∑ t = 1 T ∇ θ ℓ ^ F ( F ^ t ; r H ^ ( θ ) ) − 1 T ∑ t = 1 T ∇ θ ℓ ^ F ( F t H ^ T ; r H ^ ( θ ) ) ∥ = O P ( 1 N T + 1 N + 1 T ) . (280) \sup_{\theta\in\Theta}\left\|\frac{1}{T}\sum_{t=1}^{T}\nabla^\theta\widehat{\ell}_{F}\left(\widehat{\mathbf{F}}_t;r_{\widehat{\mathbf{H}}}(\theta)\right)-\frac{1}{T}\sum_{t=1}^{T}\nabla^\theta\widehat{\ell}_{F}\left(\mathbf{F}_t\widehat{\mathbf{H}}^{\mathsf{T}};r_{\widehat{\mathbf{H}}}(\theta)\right)\right\|
=O_{\mathrm{P}}\left(\frac{1}{\sqrt{NT}}+\frac{1}{N}+\frac{1}{T}\right). \tag*{(280)} θ ∈ Θ sup T 1 t = 1 ∑ T ∇ θ ℓ F ( F t ; r H ( θ ) ) − T 1 t = 1 ∑ T ∇ θ ℓ F ( F t H T ; r H ( θ ) ) = O P ( N T 1 + N 1 + T 1 ) . (280)
Controlling the second term in (271) . Since log ∣ H ∣ \log|\mathbf{H}| log ∣ H ∣ does not depend on θ \theta θ , differentiating (114) with respect to θ \theta θ yields
∇ θ ℓ F ( F t H T ; r H ( θ ) ) = ∇ θ ℓ F ( F t ; θ ) , (281) \nabla^\theta\ell_{F}\left(\mathbf{F}_t\mathbf{H}^{\mathsf{T}};r_{\mathbf{H}}(\theta)\right)=\nabla^\theta\ell_{F}(\mathbf{F}_t;\theta), \tag*{(281)} ∇ θ ℓ F ( F t H T ; r H ( θ ) ) = ∇ θ ℓ F ( F t ; θ ) , (281)
whenever the two filters are initialized equivariantly.
It therefore remains only to control the possibly different filter initializations. Using the same block iteration used in (116) , applied also to (221) , we obtain, for some 0 < ϱ k < 1 0<\varrho_k<1 0 < ϱ k < 1 ,
sup H ∈ H sup θ ∈ Θ { ∥ ψ ^ t ∣ t − 1 ( r H ( θ ) ) − ψ t ∣ t − 1 ( r H ( θ ) ) ∥ + ∥ ∇ θ ψ ^ t ∣ t − 1 ( r H ( θ ) ) − ∇ θ ψ t ∣ t − 1 ( r H ( θ ) ) ∥ } = O a . s . ( ϱ k ⌊ ( t − 1 ) / ( 2 k ) ⌋ ) . (282) \sup_{\mathbf{H}\in\mathcal{H}}\sup_{\theta\in\Theta}\left\{\left\|\widehat{\psi}_{t|t-1}(r_{\mathbf{H}}(\theta))-\psi_{t|t-1}(r_{\mathbf{H}}(\theta))\right\|+\left\|\nabla^\theta\widehat{\psi}_{t|t-1}(r_{\mathbf{H}}(\theta))-\nabla^\theta\psi_{t|t-1}(r_{\mathbf{H}}(\theta))\right\|\right\}
=O_{\mathrm{a.s.}}\left(\varrho_k^{\lfloor(t-1)/(2k)\rfloor}\right). \tag*{(282)} H ∈ H sup θ ∈ Θ sup { ψ t ∣ t − 1 ( r H ( θ )) − ψ t ∣ t − 1 ( r H ( θ )) + ∇ θ ψ t ∣ t − 1 ( r H ( θ )) − ∇ θ ψ t ∣ t − 1 ( r H ( θ )) } = O a.s. ( ϱ k ⌊( t − 1 ) / ( 2 k )⌋ ) . (282)
Using the score representation in (267) , the mean value theorem, Assumptions 9 and 11, and (282) , we obtain
sup H ∈ H sup θ ∈ Θ ∥ ∇ θ ℓ ^ F ( H F t ; r H ( θ ) ) − ∇ θ ℓ F ( H F t ; r H ( θ ) ) ∥ = O a . s . ( ϱ k ⌊ ( t − 1 ) / ( 2 k ) ⌋ ) , \sup_{\mathbf{H}\in\mathcal{H}}\sup_{\theta\in\Theta}\left\|\nabla^\theta\widehat{\ell}_{F}\left(\mathbf{H}\mathbf{F}_t;r_{\mathbf{H}}(\theta)\right)-\nabla^\theta\ell_{F}\left(\mathbf{H}\mathbf{F}_t;r_{\mathbf{H}}(\theta)\right)\right\|
=O_{\mathrm{a.s.}}\left(\varrho_k^{\lfloor(t-1)/(2k)\rfloor}\right), H ∈ H sup θ ∈ Θ sup ∇ θ ℓ F ( H F t ; r H ( θ ) ) − ∇ θ ℓ F ( H F t ; r H ( θ ) ) = O a.s. ( ϱ k ⌊( t − 1 ) / ( 2 k )⌋ ) ,
after enlarging ϱ k \varrho_k ϱ k , if necessary, exactly as in (125) . Consequently,
sup H ∈ H sup θ ∈ Θ ∥ 1 T ∑ t = 1 T ∇ θ ℓ ^ F ( F t H T ; r H ( θ ) ) − 1 T ∑ t = 1 T ∇ θ ℓ F ( F t H T ; r H ( θ ) ) ∥ = O P ( 1 T ) . (283) \sup_{\mathbf{H}\in\mathcal{H}}\sup_{\theta\in\Theta}\left\|\frac{1}{T}\sum_{t=1}^{T}\nabla^\theta\widehat{\ell}_{F}\left(\mathbf{F}_t\mathbf{H}^{\mathsf{T}};r_{\mathbf{H}}(\theta)\right)-\frac{1}{T}\sum_{t=1}^{T}\nabla^\theta\ell_{F}\left(\mathbf{F}_t\mathbf{H}^{\mathsf{T}};r_{\mathbf{H}}(\theta)\right)\right\|=O_{\mathrm{P}}\left(\frac{1}{T}\right). \tag*{(283)} H ∈ H sup θ ∈ Θ sup T 1 t = 1 ∑ T ∇ θ ℓ F ( F t H T ; r H ( θ ) ) − T 1 t = 1 ∑ T ∇ θ ℓ F ( F t H T ; r H ( θ ) ) = O P ( T 1 ) . (283)
Since P ( H ^ ∈ H ) → 1 \mathbb{P}\left(\widehat{H} \in\mathcal{H}\right) \to1 P ( H ∈ H ) → 1 as N , T → ∞ N,T \to\infty N , T → ∞ , combining (281) and (283) , evaluated at H = H ^ H=\widehat{H} H = H , yields
sup θ ∈ Θ ∥ 1 T ∑ t = 1 T ∇ θ ℓ ^ F ( F t H ^ ⊤ ; r H ^ ( θ ) ) − 1 T ∑ t = 1 T ∇ θ ℓ F ( F t ; θ ) ∥ = O P ( 1 T ) . (284) \sup_{\theta\in\Theta}\left\Vert\frac{1}{T}\sum_{t=1}^{T}\nabla^{\theta}\widehat{\ell}_{F}\left(F_{t}\widehat{H}^{\top};r_{\widehat{H}}(\theta)\right)-\frac{1}{T}\sum_{t=1}^{T}\nabla^{\theta}\ell_{F}(F_{t};\theta)\right\Vert=O_{\mathbb{P}}\left(\frac{1}{T}\right). \tag*{(284)} θ ∈ Θ sup T 1 t = 1 ∑ T ∇ θ ℓ F ( F t H ⊤ ; r H ( θ ) ) − T 1 t = 1 ∑ T ∇ θ ℓ F ( F t ; θ ) = O P ( T 1 ) . (284)
Controlling the third term in (271) .
By (120) , (121) , and Lemma 7(c), after enlarging ϱ k \varrho_{k} ϱ k , if necessary,
∥ F t − F ‾ t ∥ + sup θ ∈ Θ ∥ ψ t ∣ t − 1 ( θ ) − ψ ‾ t ∣ t − 1 ( θ ) ∥ + sup θ ∈ Θ ∥ ∇ θ ψ t ∣ t − 1 ( θ ) − ∇ θ ψ ‾ t ∣ t − 1 ( θ ) ∥ = O a . s . ( ϱ k ⌊ ( t − 1 ) / ( 2 k ) ⌋ ) . (285) \left\Vert F_{t}-\overline{F}_{t}\right\Vert+\sup_{\theta\in\Theta}\left\Vert\psi_{t|t-1}(\theta)-\overline{\psi}_{t|t-1}(\theta)\right\Vert
+\sup_{\theta\in\Theta}\left\Vert\nabla^{\theta}\psi_{t|t-1}(\theta)-\nabla^{\theta}\overline{\psi}_{t|t-1}(\theta)\right\Vert
=O_{\mathrm{a.s.}}\left(\varrho_{k}^{\lfloor(t-1)/(2k)\rfloor}\right). \tag*{(285)} F t − F t + θ ∈ Θ sup ψ t ∣ t − 1 ( θ ) − ψ t ∣ t − 1 ( θ ) + θ ∈ Θ sup ∇ θ ψ t ∣ t − 1 ( θ ) − ∇ θ ψ t ∣ t − 1 ( θ ) = O a.s. ( ϱ k ⌊( t − 1 ) / ( 2 k )⌋ ) . (285)
The score representation in (267) , the mean value theorem, and the derivative envelopes in Assumptions 9 and 11 therefore imply
sup θ ∈ Θ ∥ ∇ θ ℓ F ( F t ; θ ) − ∇ θ ℓ F ( F ‾ t ; θ ) ∥ = O a . s . ( ϱ k ⌊ ( t − 1 ) / ( 2 k ) ⌋ ) , (286) \sup_{\theta\in\Theta}\left\Vert\nabla^{\theta}\ell_{F}(F_{t};\theta)-\nabla^{\theta}\ell_{F}(\overline{F}_{t};\theta)\right\Vert
=O_{\mathrm{a.s.}}\left(\varrho_{k}^{\lfloor(t-1)/(2k)\rfloor}\right), \tag*{(286)} θ ∈ Θ sup ∇ θ ℓ F ( F t ; θ ) − ∇ θ ℓ F ( F t ; θ ) = O a.s. ( ϱ k ⌊( t − 1 ) / ( 2 k )⌋ ) , (286)
after one further enlargement of ϱ k \varrho_{k} ϱ k , using the same Markov and Borel–Cantelli arguments as in (123) –(124) . Since the right-hand side of (286) is summable,
sup θ ∈ Θ ∥ 1 T ∑ t = 1 T ∇ θ ℓ F ( F t ; θ ) − 1 T ∑ t = 1 T ∇ θ ℓ F ( F ‾ t ; θ ) ∥ = O P ( 1 T ) . (287) \sup_{\theta\in\Theta}\left\Vert\frac{1}{T}\sum_{t=1}^{T}\nabla^{\theta}\ell_{F}(F_{t};\theta)-\frac{1}{T}\sum_{t=1}^{T}\nabla^{\theta}\ell_{F}(\overline{F}_{t};\theta)\right\Vert=O_{\mathbb{P}}\left(\frac{1}{T}\right). \tag*{(287)} θ ∈ Θ sup T 1 t = 1 ∑ T ∇ θ ℓ F ( F t ; θ ) − T 1 t = 1 ∑ T ∇ θ ℓ F ( F t ; θ ) = O P ( T 1 ) . (287)
Controlling the fourth term in (271) .
Consider the centered stationary score family ( ∇ θ ℓ F ( F ‾ t ; θ ) − E [ ∇ θ ℓ F ( F ‾ t ; θ ) ] ) \left(\nabla^{\theta}\ell_{F}(\overline{F}_{t};\theta)-\mathbb{E}\left[\nabla^{\theta}\ell_{F}(\overline{F}_{t};\theta)\right]\right) ( ∇ θ ℓ F ( F t ; θ ) − E [ ∇ θ ℓ F ( F t ; θ ) ] ) . The uniform contraction argument in Lemma 6, together with the corresponding argument for the first derivative recursion in Lemma 7(d), gives geometrically decreasing approximation coefficients for the jointly truncated process
{ ( F ‾ t ( τ ) ⊤ , ψ ‾ t ∣ t − 1 ( τ ) ⊤ ( θ ) , vec ∇ θ ψ ‾ t ∣ t − 1 ( τ ) ( θ ) ⊤ ) ⊤ } t ∈ Z , \left\{\left(\overline{F}_{t}^{(\tau)\top},\overline{\psi}_{t|t-1}^{(\tau)\top}(\theta),\operatorname{vec}\nabla^{\theta}\overline{\psi}_{t|t-1}^{(\tau)}(\theta)^{\top}\right)^{\top}\right\}_{t\in\mathbb{Z}}, { ( F t ( τ ) ⊤ , ψ t ∣ t − 1 ( τ ) ⊤ ( θ ) , vec ∇ θ ψ t ∣ t − 1 ( τ ) ( θ ) ⊤ ) ⊤ } t ∈ Z ,
uniformly over θ ∈ Θ \theta\in\Theta θ ∈ Θ . The score representation in (267) , the mean value theorem, and the moment conditions in Assumptions 9 and 11 therefore imply that the centered stationary score family is uniformly near-epoch dependent with geometrically decreasing approximation coefficients.
Moreover, Assumption 11 and Lemma 7(d) give the required uniform moment and Lipschitz bounds. Hence, the finite-window decomposition and the β \beta β -mixing empirical-process argument used in (139) –(145) applies componentwise to the centered stationary score family. Consequently,
sup θ ∈ Θ ∥ 1 T ∑ t = 1 T ∇ θ ℓ F ( F ˉ t ; θ ) − E [ ∇ θ ℓ F ( F ˉ t ; θ ) ] ∥ = O P ( 1 T ) . (288) \sup_{\theta\in\Theta}\left\Vert\frac{1}{T}\sum_{t=1}^{T}\nabla^\theta\ell_F\left(\bar{F}_t;\theta\right)-\mathbb{E}\left[\nabla^\theta\ell_F\left(\bar{F}_t;\theta\right)\right]\right\Vert=O_{\mathrm{P}}\left(\frac{1}{\sqrt{T}}\right). \tag*{(288)} θ ∈ Θ sup T 1 t = 1 ∑ T ∇ θ ℓ F ( F ˉ t ; θ ) − E [ ∇ θ ℓ F ( F ˉ t ; θ ) ] = O P ( T 1 ) . (288)
The combination of (280) , (284) , (287) , and (288) shows that
sup θ ∈ Θ ∥ 1 T ∑ t = 1 T ∇ θ ℓ ^ F ( F ^ t ; r H ( θ ) ) − E [ ∇ θ ℓ F ( F ˉ t ; θ ) ] ∥ = O P ( 1 T + 1 N T + 1 N + 1 T ) = O P ( 1 T + 1 N ) . \begin{aligned}
\sup_{\theta\in\Theta}\left\Vert\frac{1}{T}\sum_{t=1}^{T}\nabla^\theta\widehat{\ell}_F\left(\widehat{F}_t;r_H(\theta)\right)-\mathbb{E}\left[\nabla^\theta\ell_F\left(\bar{F}_t;\theta\right)\right]\right\Vert\\
&=O_{\mathrm{P}}\left(\frac{1}{\sqrt{T}}+\frac{1}{\sqrt{NT}}+\frac{1}{N}+\frac{1}{T}\right)=O_{\mathrm{P}}\left(\frac{1}{\sqrt{T}}+\frac{1}{N}\right).
\end{aligned} θ ∈ Θ sup T 1 t = 1 ∑ T ∇ θ ℓ F ( F t ; r H ( θ ) ) − E [ ∇ θ ℓ F ( F ˉ t ; θ ) ] = O P ( T 1 + N T 1 + N 1 + T 1 ) = O P ( T 1 + N 1 ) .
This proves (268) .
Combining only (280) , (284) , and (287) we get
1 T sup θ ∈ Θ ∥ ∑ t = 1 T ∇ θ ℓ ^ F ( F ^ t ; r H ( θ ) ) − ∑ t = 1 T ∇ θ ℓ F ( F ˉ t ; θ ) ∥ = O P ( 1 N + T N + 1 T ) , \begin{aligned}
\frac{1}{\sqrt{T}}\sup_{\theta\in\Theta}\left\Vert\sum_{t=1}^{T}\nabla^\theta\widehat{\ell}_F\left(\widehat{F}_t;r_H(\theta)\right)-\sum_{t=1}^{T}\nabla^\theta\ell_F\left(\bar{F}_t;\theta\right)\right\Vert\\
&=O_{\mathrm{P}}\left(\frac{1}{\sqrt{N}}+\frac{\sqrt{T}}{N}+\frac{1}{\sqrt{T}}\right),
\end{aligned} T 1 θ ∈ Θ sup t = 1 ∑ T ∇ θ ℓ F ( F t ; r H ( θ ) ) − t = 1 ∑ T ∇ θ ℓ F ( F ˉ t ; θ ) = O P ( N 1 + N T + T 1 ) ,
which proves (269) . If T / N → 0 \sqrt{T}/N\to0 T / N → 0 , the right-hand side is o P ( 1 ) o_{\mathrm{P}}(1) o P ( 1 ) as N , T → ∞ N,T\to\infty N , T → ∞ , proving (270) .
Finally, evaluating the preceding truncated process argument at θ = θ 0 \theta=\theta_0 θ = θ 0 shows that the true score process { ∇ θ ℓ F ( F ˉ t ; θ 0 ) } t ∈ Z \left\{\nabla^\theta\ell_F\left(\bar{F}_t;\theta_0\right)\right\}_{t\in\mathbb{Z}} { ∇ θ ℓ F ( F ˉ t ; θ 0 ) } t ∈ Z is near-epoch dependent of size − 1 -1 − 1 on the β \beta β -mixing process { v t } t ∈ Z \{v_t\}_{t\in\mathbb{Z}} { v t } t ∈ Z . Since θ 0 \theta_0 θ 0 is an interior maximizer of the stationary population criterion, differentiation under the expectation implies that E [ ∇ θ ℓ F ( F ˉ t ; θ 0 ) ] = 0 d \mathbb{E}\left[\nabla^\theta\ell_F\left(\bar{F}_t;\theta_0\right)\right]=0_d E [ ∇ θ ℓ F ( F ˉ t ; θ 0 ) ] = 0 d . Thus, the stationary score process is zero mean and near-epoch dependent of size − 1 -1 − 1 on the geometrically β \beta β -mixing process { v t } t ∈ Z \{v_t\}_{t\in\mathbb{Z}} { v t } t ∈ Z .
Lemma 10. Consider the Hessian matrix
∇ θ θ ℓ F ( F t ; θ ) = ∇ θ ψ t ∣ t − 1 ( θ ) ⊤ ∇ ψ ψ ℓ F ( F t ; θ ) ∇ θ ψ t ∣ t − 1 ( θ ) + ( I d ⊗ ∇ ψ ℓ F ( F t ; θ ) ) ⊤ ∇ θ θ ψ t ∣ t − 1 ( θ ) \nabla^{\theta\theta}\ell_F(F_t;\theta)=\nabla^\theta\psi_{t|t-1}(\theta)^\top\nabla^{\psi\psi}\ell_F(F_t;\theta)\nabla^\theta\psi_{t|t-1}(\theta)+\left(I_d\otimes\nabla^\psi\ell_F(F_t;\theta)\right)^\top\nabla^{\theta\theta}\psi_{t|t-1}(\theta) ∇ θ θ ℓ F ( F t ; θ ) = ∇ θ ψ t ∣ t − 1 ( θ ) ⊤ ∇ ψ ψ ℓ F ( F t ; θ ) ∇ θ ψ t ∣ t − 1 ( θ ) + ( I d ⊗ ∇ ψ ℓ F ( F t ; θ ) ) ⊤ ∇ θ θ ψ t ∣ t − 1 ( θ ) + ∇ θ ψ t ∣ t − 1 ( θ ) ⊤ ∂ 2 ℓ F ( F t ; θ ) ∂ ψ t ∣ t − 1 ∂ θ + ∂ 2 ℓ F ( F t ; θ ) ∂ θ ∂ ψ t ∣ t − 1 ∇ θ ψ t ∣ t − 1 ( θ ) + ∂ 2 ℓ F ( F t ; θ ) ∂ θ ∂ θ , (289) +\nabla^\theta\psi_{t|t-1}(\theta)^\top\frac{\partial^2\ell_F(F_t;\theta)}{\partial\psi_{t|t-1}\partial\theta}+\frac{\partial^2\ell_F(F_t;\theta)}{\partial\theta\partial\psi_{t|t-1}}\nabla^\theta\psi_{t|t-1}(\theta)+\frac{\partial^2\ell_F(F_t;\theta)}{\partial\theta\partial\theta}, \tag*{(289)} + ∇ θ ψ t ∣ t − 1 ( θ ) ⊤ ∂ ψ t ∣ t − 1 ∂ θ ∂ 2 ℓ F ( F t ; θ ) + ∂ θ ∂ ψ t ∣ t − 1 ∂ 2 ℓ F ( F t ; θ ) ∇ θ ψ t ∣ t − 1 ( θ ) + ∂ θ ∂ θ ∂ 2 ℓ F ( F t ; θ ) , (289) where all partial derivatives in (289) are computed holding ψ t ∣ t − 1 \psi_{t|t-1} ψ t ∣ t − 1 and F t F_t F t fixed and is subsequently evaluated at ψ t ∣ t − 1 = ψ t ∣ t − 1 ( θ ) \psi_{t|t-1}=\psi_{t|t-1}(\theta) ψ t ∣ t − 1 = ψ t ∣ t − 1 ( θ ) .
Under Assumptions 1–13, as T → ∞ T\to\infty T → ∞ ,
sup θ ∈ Θ ∥ 1 T ∑ t = 1 T ∇ θ θ ℓ F ( F t ; θ ) − E [ ∇ θ θ ℓ F ( F ˉ t ; θ ) ] ∥ → a . s . 0. (290) \sup_{\theta\in\Theta}\left\Vert\frac{1}{T}\sum_{t=1}^{T}\nabla^{\theta\theta}\ell_F(F_t;\theta)-\mathbb{E}\left[\nabla^{\theta\theta}\ell_F(\bar{F}_t;\theta)\right]\right\Vert\xrightarrow{\mathrm{a.s.}}0. \tag*{(290)} θ ∈ Θ sup T 1 t = 1 ∑ T ∇ θ θ ℓ F ( F t ; θ ) − E [ ∇ θ θ ℓ F ( F ˉ t ; θ ) ] a.s. 0. (290)
Moreover, as T → ∞ T \to\infty T → ∞ ,
− E [ ∇ θ θ ℓ F ( F t ; θ 0 ) ] ⟶ I ( θ 0 ) = − E [ ∇ θ θ ℓ F ( F ˉ t ; θ 0 ) ] , (291) -\mathbb{E}\left[\nabla^{\theta\theta}\ell_F(F_t;\theta_0)\right] \longrightarrow\mathcal{I}(\theta_0) = -\mathbb{E}\left[\nabla^{\theta\theta}\ell_F(\bar{F}_t;\theta_0)\right], \tag*{(291)} − E [ ∇ θ θ ℓ F ( F t ; θ 0 ) ] ⟶ I ( θ 0 ) = − E [ ∇ θ θ ℓ F ( F ˉ t ; θ 0 ) ] , (291)
where I ( θ 0 ) \mathcal{I}(\theta_0) I ( θ 0 ) is positive definite and therefore invertible.
Proof. By (120) , (121) , Lemma 7(c), and Lemma 8 (c), after enlarging ϱ k \varrho_k ϱ k , if necessary,
∥ F t − F ˉ t ∥ + sup θ ∈ Θ ∥ ψ t ∣ t − 1 ( θ ) − ψ ˉ t ∣ t − 1 ( θ ) ∥ + sup θ ∈ Θ ∥ ∇ θ ψ t ∣ t − 1 ( θ ) − ∇ θ ψ ˉ t ∣ t − 1 ( θ ) ∥ + sup θ ∈ Θ ∥ ∇ θ θ ψ t ∣ t − 1 ( θ ) − ∇ θ θ ψ ˉ t ∣ t − 1 ( θ ) ∥ = O a . s . ( ϱ k ⌊ ( t − 1 ) / ( 4 k ) ⌋ ) . (292) \begin{aligned}
\lVert F_t-\bar{F}_t\rVert
&+\sup_{\theta\in\Theta}\left\lVert\psi_{t|t-1}(\theta)-\bar{\psi}_{t|t-1}(\theta)\right\rVert\\
&+\sup_{\theta\in\Theta}\left\lVert\nabla^\theta\psi_{t|t-1}(\theta)-\nabla^\theta\bar{\psi}_{t|t-1}(\theta)\right\rVert\\
&+\sup_{\theta\in\Theta}\left\lVert\nabla^{\theta\theta}\psi_{t|t-1}(\theta)-\nabla^{\theta\theta}\bar{\psi}_{t|t-1}(\theta)\right\rVert
= O_{a.s.}\left(\varrho_k^{\lfloor(t-1)/(4k)\rfloor}\right). \tag*{(292)}
\end{aligned} ∥ F t − F ˉ t ∥ + θ ∈ Θ sup ψ t ∣ t − 1 ( θ ) − ψ ˉ t ∣ t − 1 ( θ ) + θ ∈ Θ sup ∇ θ ψ t ∣ t − 1 ( θ ) − ∇ θ ψ ˉ t ∣ t − 1 ( θ ) + θ ∈ Θ sup ∇ θ θ ψ t ∣ t − 1 ( θ ) − ∇ θ θ ψ ˉ t ∣ t − 1 ( θ ) = O a . s . ( ϱ k ⌊( t − 1 ) / ( 4 k )⌋ ) . (292)
By Assumption 11, the mean value theorem, and (292) we therefore get
sup θ ∈ Θ ∥ ∇ θ θ ℓ F ( F t ; θ ) − ∇ θ θ ℓ F ( F ˉ t ; θ ) ∥ = O a . s . ( ϱ k ⌊ ( t − 1 ) / ( 4 k ) ⌋ ) , (293) \sup_{\theta\in\Theta}\left\lVert\nabla^{\theta\theta}\ell_F(F_t;\theta)-\nabla^{\theta\theta}\ell_F(\bar{F}_t;\theta)\right\rVert
= O_{a.s.}\left(\varrho_k^{\lfloor(t-1)/(4k)\rfloor}\right), \tag*{(293)} θ ∈ Θ sup ∇ θ θ ℓ F ( F t ; θ ) − ∇ θ θ ℓ F ( F ˉ t ; θ ) = O a . s . ( ϱ k ⌊( t − 1 ) / ( 4 k )⌋ ) , (293)
after one further enlargement of ϱ k \varrho_k ϱ k , if necessary, using the same Markov and Borel–Cantelli arguments employed in (123) –(124) .
Since the right-hand side of (293) is summable, we obtain
sup θ ∈ Θ ∥ 1 T ∑ t = 1 T ∇ θ θ ℓ F ( F t ; θ ) − 1 T ∑ t = 1 T ∇ θ θ ℓ F ( F ˉ t ; θ ) ∥ = O a . s . ( 1 T ) . (294) \sup_{\theta\in\Theta}\left\lVert\frac{1}{T}\sum_{t=1}^{T}\nabla^{\theta\theta}\ell_F(F_t;\theta)-\frac{1}{T}\sum_{t=1}^{T}\nabla^{\theta\theta}\ell_F(\bar{F}_t;\theta)\right\rVert
= O_{a.s.}\left(\frac{1}{T}\right). \tag*{(294)} θ ∈ Θ sup T 1 t = 1 ∑ T ∇ θ θ ℓ F ( F t ; θ ) − T 1 t = 1 ∑ T ∇ θ θ ℓ F ( F ˉ t ; θ ) = O a . s . ( T 1 ) . (294)
By Lemmas 6, 7(d), and 8 (d), the process { ∇ θ θ ℓ F ( F ˉ t ; θ ) } t ∈ Z \left\{\nabla^{\theta\theta}\ell_F(\bar{F}_t;\theta)\right\}_{t\in\mathbb{Z}} { ∇ θ θ ℓ F ( F ˉ t ; θ ) } t ∈ Z is strictly stationary and ergodic for every θ ∈ Θ \theta\in\Theta θ ∈ Θ . Moreover, Assumption 11 implies that it is almost surely continuous in θ \theta θ and satisfies
E [ sup θ ∈ Θ ∥ ∇ θ θ ℓ F ( F ˉ t ; θ ) ∥ ] < ∞ . \mathbb{E}\left[\sup_{\theta\in\Theta}\left\lVert\nabla^{\theta\theta}\ell_F(\bar{F}_t;\theta)\right\rVert\right] < \infty. E [ θ ∈ Θ sup ∇ θ θ ℓ F ( F ˉ t ; θ ) ] < ∞.
Since Θ \Theta Θ is compact, the uniform ergodic law of large numbers implies that
sup θ ∈ Θ ∥ 1 T ∑ t = 1 T ∇ θ θ ℓ F ( F ˉ t ; θ ) − E [ ∇ θ θ ℓ F ( F ˉ t ; θ ) ] ∥ → a . s . 0 (295) \sup_{\theta\in\Theta}\left\lVert\frac{1}{T}\sum_{t=1}^{T}\nabla^{\theta\theta}\ell_F(\bar{F}_t;\theta)-\mathbb{E}\left[\nabla^{\theta\theta}\ell_F(\bar{F}_t;\theta)\right]\right\rVert\xrightarrow{\ a.s.\ } 0 \tag*{(295)} θ ∈ Θ sup T 1 t = 1 ∑ T ∇ θ θ ℓ F ( F ˉ t ; θ ) − E [ ∇ θ θ ℓ F ( F ˉ t ; θ ) ] a . s . 0 (295)
as T → ∞ T \to\infty T → ∞ .
An application of the triangle inequality then shows
sup θ ∈ Θ ∥ 1 T ∑ t = 1 T ∇ θ θ ℓ F ( F ˉ t ; θ ) − E [ ∇ θ θ ℓ F ( F ˉ t ; θ ) ] ∥ ≤ sup θ ∈ Θ ∥ 1 T ∑ t = 1 T ∇ θ θ ℓ F ( F ˉ t ; θ ) − 1 T ∑ t = 1 T ∇ θ θ ℓ F ( F ˉ t ; θ ) ∥ + sup θ ∈ Θ ∥ 1 T ∑ t = 1 T ∇ θ θ ℓ F ( F ˉ t ; θ ) − E [ ∇ θ θ ℓ F ( F ˉ t ; θ ) ] ∥ . \sup_{\theta\in\Theta}\left\lVert\frac{1}{T}\sum_{t=1}^{T}\nabla^{\theta\theta}\ell_F(\bar{F}_t;\theta)-\mathbb{E}\left[\nabla^{\theta\theta}\ell_F(\bar{F}_t;\theta)\right]\right\rVert
\leq\sup_{\theta\in\Theta}\left\lVert\frac{1}{T}\sum_{t=1}^{T}\nabla^{\theta\theta}\ell_F(\bar{F}_t;\theta)-\frac{1}{T}\sum_{t=1}^{T}\nabla^{\theta\theta}\ell_F(\bar{F}_t;\theta)\right\rVert
+\sup_{\theta\in\Theta}\left\lVert\frac{1}{T}\sum_{t=1}^{T}\nabla^{\theta\theta}\ell_F(\bar{F}_t;\theta)-\mathbb{E}\left[\nabla^{\theta\theta}\ell_F(\bar{F}_t;\theta)\right]\right\rVert. θ ∈ Θ sup T 1 t = 1 ∑ T ∇ θ θ ℓ F ( F ˉ t ; θ ) − E [ ∇ θ θ ℓ F ( F ˉ t ; θ ) ] ≤ θ ∈ Θ sup T 1 t = 1 ∑ T ∇ θ θ ℓ F ( F ˉ t ; θ ) − T 1 t = 1 ∑ T ∇ θ θ ℓ F ( F ˉ t ; θ ) + θ ∈ Θ sup T 1 t = 1 ∑ T ∇ θ θ ℓ F ( F ˉ t ; θ ) − E [ ∇ θ θ ℓ F ( F ˉ t ; θ ) ] .
Equations (294) and (295) prove (290) as T → ∞ T\to\infty T → ∞ .
Finally, the corresponding L 1 L_1 L 1 version of (293) is given by
∥ E [ ∇ θ θ ℓ F ( F ˉ t ; θ 0 ) − ∇ θ θ ℓ F ( F ˉ t ; θ 0 ) ] ∥ = O ( 1 T ) . \left\lVert\mathbb{E}\left[\nabla^{\theta\theta}\ell_F(\bar{F}_t;\theta_0)-\nabla^{\theta\theta}\ell_F(\bar{F}_t;\theta_0)\right]\right\rVert=O\left(\frac{1}{T}\right). E [ ∇ θ θ ℓ F ( F ˉ t ; θ 0 ) − ∇ θ θ ℓ F ( F ˉ t ; θ 0 ) ] = O ( T 1 ) .
By stationarity,
− E [ ∇ θ θ ℓ F ( F ˉ t ; θ 0 ) ] = − E [ ∇ θ θ ℓ F ( F ˉ t ; θ 0 ) ] = I ( θ 0 ) . -\mathbb{E}\left[\nabla^{\theta\theta}\ell_F(\bar{F}_t;\theta_0)\right]=-\mathbb{E}\left[\nabla^{\theta\theta}\ell_F(\bar{F}_t;\theta_0)\right]=\mathcal{I}(\theta_0). − E [ ∇ θ θ ℓ F ( F ˉ t ; θ 0 ) ] = − E [ ∇ θ θ ℓ F ( F ˉ t ; θ 0 ) ] = I ( θ 0 ) .
These two results prove (291) . Finally, I ( θ 0 ) \mathcal{I}(\theta_0) I ( θ 0 ) is positive definite by Assumption 12 and is therefore invertible.
Lemma 11. Under Assumptions 1–13, with p > 2 p>2 p > 2 , for every fixed initial condition ψ 1 ∣ 0 ∈ P \psi_{1|0}\in\mathcal{P} ψ 1∣0 ∈ P , define
V T ( θ 0 ) = ∑ t = 1 T ∑ s = 1 T E [ ∇ θ ℓ F ( F ˉ t ; θ 0 ) ∇ θ ℓ F ( F ˉ s ; θ 0 ) T ] . (296) V_T(\theta_0)=\sum_{t=1}^{T}\sum_{s=1}^{T}\mathbb{E}\left[\nabla^\theta\ell_F(\bar{F}_t;\theta_0)\nabla^\theta\ell_F(\bar{F}_s;\theta_0)^\mathsf{T}\right]. \tag*{(296)} V T ( θ 0 ) = t = 1 ∑ T s = 1 ∑ T E [ ∇ θ ℓ F ( F ˉ t ; θ 0 ) ∇ θ ℓ F ( F ˉ s ; θ 0 ) T ] . (296) Then, as T → ∞ T\to\infty T → ∞ ,
1 T V T ( θ 0 ) ⟶ V ( θ 0 ) , (297) \frac{1}{T}V_T(\theta_0)\longrightarrow V(\theta_0), \tag*{(297)} T 1 V T ( θ 0 ) ⟶ V ( θ 0 ) , (297) where
V ( θ 0 ) = ∑ h = − ∞ ∞ E [ ∇ θ ℓ F ( F ˉ t ; θ 0 ) ∇ θ ℓ F ( F ˉ t − h ; θ 0 ) T ] (298) V(\theta_0)=\sum_{h=-\infty}^{\infty}\mathbb{E}\left[\nabla^\theta\ell_F(\bar{F}_t;\theta_0)\nabla^\theta\ell_F(\bar{F}_{t-h};\theta_0)^\mathsf{T}\right] \tag*{(298)} V ( θ 0 ) = h = − ∞ ∑ ∞ E [ ∇ θ ℓ F ( F ˉ t ; θ 0 ) ∇ θ ℓ F ( F ˉ t − h ; θ 0 ) T ] (298) exists and is finite. Moreover, as T → ∞ T\to\infty T → ∞ ,
1 T ∑ t = 1 T ∇ θ ℓ F ( F ˉ t ; θ 0 ) ⇒ N ( 0 d , V ( θ 0 ) ) . (299) \frac{1}{\sqrt{T}}\sum_{t=1}^{T}\nabla^\theta\ell_F(\bar{F}_t;\theta_0)\Rightarrow\mathcal{N}(0_d,V(\theta_0)). \tag*{(299)} T 1 t = 1 ∑ T ∇ θ ℓ F ( F ˉ t ; θ 0 ) ⇒ N ( 0 d , V ( θ 0 )) . (299)
Proof. By Lemma 9 , the stationary score process { ∇ θ ℓ F ( F ˉ t ; θ 0 ) } t ∈ Z \left\{\nabla^\theta\ell_F\left(\bar{F}_t;\theta_0\right)\right\}_{t\in\mathbb{Z}} { ∇ θ ℓ F ( F ˉ t ; θ 0 ) } t ∈ Z has mean zero and is near-epoch dependent of size − 1 -1 − 1 on the geometrically β \beta β -mixing process { v t } t ∈ Z \{v_t\}_{t\in\mathbb{Z}} { v t } t ∈ Z . Moreover, Assumption 11 and Lemma 7(d) imply that
E [ ∥ ∇ θ ℓ F ( F ˉ t ; θ 0 ) ∥ p ] < ∞ . (300) \mathbb{E}\left[\left\|\nabla^\theta\ell_F\left(\bar{F}_t;\theta_0\right)\right\|^p\right] < \infty. \tag*{(300)} E [ ∇ θ ℓ F ( F ˉ t ; θ 0 ) p ] < ∞. (300)
To establish the absolute summability of the score autocovariances, let
s t = ∇ θ ℓ F ( F ˉ t ; θ 0 ) , s_t=\nabla^\theta\ell_F\left(\bar{F}_t;\theta_0\right), s t = ∇ θ ℓ F ( F ˉ t ; θ 0 ) ,
and let s t ( τ ) s_t^{(\tau)} s t ( τ ) denote the corresponding score contribution computed from the τ \tau τ -truncated factor, filter, and first-derivative filter processes. By construction, s t ( τ ) s_t^{(\tau)} s t ( τ ) is measurable with respect to σ ( v t , … , v t − τ ) \sigma(v_t,\ldots,v_{t-\tau}) σ ( v t , … , v t − τ ) . The truncated-process argument used in the proof of Lemma 9 , together with Assumption 11, gives, for some finite positive real C C C and some 0 < ϱ k < 1 0<\varrho_k<1 0 < ϱ k < 1 ,
sup t ∈ Z ∥ s t − s t ( τ ) ∥ 2 ≤ C ϱ k ⌊ τ / k ⌋ , (301) \sup_{t\in\mathbb{Z}}\left\|s_t-s_t^{(\tau)}\right\|_2\le C\varrho_k^{\lfloor\tau/k\rfloor}, \tag*{(301)} t ∈ Z sup s t − s t ( τ ) 2 ≤ C ϱ k ⌊ τ / k ⌋ , (301)
sup τ ≥ 1 sup t ∈ Z ∥ s t ( τ ) ∥ p ≤ C . (302) \sup_{\tau\ge1}\sup_{t\in\mathbb{Z}}\left\|s_t^{(\tau)}\right\|_p\le C. \tag*{(302)} τ ≥ 1 sup t ∈ Z sup s t ( τ ) p ≤ C . (302)
For all sufficiently large h h h , set τ = ⌊ h / 3 ⌋ \tau=\lfloor h/3\rfloor τ = ⌊ h /3 ⌋ . The truncated variables s t ( τ ) s_t^{(\tau)} s t ( τ ) and s t − h ( τ ) s_{t-h}^{(\tau)} s t − h ( τ ) are measurable with respect to σ ( v t , … , v t − τ ) \sigma(v_t,\ldots,v_{t-\tau}) σ ( v t , … , v t − τ ) and σ ( v t − h , … , v t − h − τ ) \sigma(v_{t-h},\ldots,v_{t-h-\tau}) σ ( v t − h , … , v t − h − τ ) , respectively. These two innovation blocks are separated by h − τ h-\tau h − τ periods, and h − τ = h − ⌊ h / 3 ⌋ ≥ ⌊ h / 3 ⌋ h-\tau=h-\lfloor h/3\rfloor\ge\lfloor h/3\rfloor h − τ = h − ⌊ h /3 ⌋ ≥ ⌊ h /3 ⌋ . Thus, the choice τ = ⌊ h / 3 ⌋ \tau=\lfloor h/3\rfloor τ = ⌊ h /3 ⌋ ensures that both the truncation depth and the separation between the resulting innovation blocks increase proportionally with h h h .
Using the covariance decomposition
Cov ( s t , s t − h ) = Cov ( s t − s t ( τ ) , s t − h ) + Cov ( s t ( τ ) , s t − h − s t − h ( τ ) ) + Cov ( s t ( τ ) , s t − h ( τ ) ) , \operatorname{Cov}(s_t,s_{t-h})=\operatorname{Cov}\left(s_t-s_t^{(\tau)},s_{t-h}\right)+\operatorname{Cov}\left(s_t^{(\tau)},s_{t-h}-s_{t-h}^{(\tau)}\right)+\operatorname{Cov}\left(s_t^{(\tau)},s_{t-h}^{(\tau)}\right), Cov ( s t , s t − h ) = Cov ( s t − s t ( τ ) , s t − h ) + Cov ( s t ( τ ) , s t − h − s t − h ( τ ) ) + Cov ( s t ( τ ) , s t − h ( τ ) ) ,
the Cauchy–Schwarz inequality, (300) , and (301) give
∥ Cov ( s t − s t ( τ ) , s t − h ) ∥ + ∥ Cov ( s t ( τ ) , s t − h − s t − h ( τ ) ) ∥ ≤ C ϱ k ⌊ h / ( 3 k ) ⌋ . (303) \left\|\operatorname{Cov}\left(s_t-s_t^{(\tau)},s_{t-h}\right)\right\|+\left\|\operatorname{Cov}\left(s_t^{(\tau)},s_{t-h}-s_{t-h}^{(\tau)}\right)\right\|\le C\varrho_k^{\lfloor h/(3k)\rfloor}. \tag*{(303)} Cov ( s t − s t ( τ ) , s t − h ) + Cov ( s t ( τ ) , s t − h − s t − h ( τ ) ) ≤ C ϱ k ⌊ h / ( 3 k )⌋ . (303)
Moreover, every β \beta β -mixing process is α \alpha α -mixing, with α v ( h ) ≤ β v ( h ) \alpha_v(h)\le\beta_v(h) α v ( h ) ≤ β v ( h ) . Hence, by using the covariance inequality for α \alpha α -mixing random variables of Davydov [18 ] , with q = p q=p q = p and $1/r=(p-2)/p$, coordinatewise to the truncated score vectors, whose underlying innovation blocks are separated by h − τ h-\tau h − τ periods, and using $\alpha_v(h)\le β v ( h ) \beta_v(h) β v ( h ) together with (302) , we obtain
∥ Cov ( s t ( τ ) , s t − h ( τ ) ) ∥ ≤ C β v ( h − τ ) p − 2 p ∥ s t ( τ ) ∥ p ∥ s t − h ( τ ) ∥ p . \left\|\operatorname{Cov}\left(s_t^{(\tau)},s_{t-h}^{(\tau)}\right)\right\| \le C\beta_v(h-\tau)^{\frac{p-2}{p}}\left\|s_t^{(\tau)}\right\|_p\left\|s_{t-h}^{(\tau)}\right\|_p. Cov ( s t ( τ ) , s t − h ( τ ) ) ≤ C β v ( h − τ ) p p − 2 s t ( τ ) p s t − h ( τ ) p .
By (302) , the two L p L_p L p norms are uniformly bounded. Since τ = ⌊ h / 3 ⌋ \tau=\lfloor h/3\rfloor τ = ⌊ h /3 ⌋ and the mixing coefficients are nonincreasing, β v ( h − τ ) ≤ β v ( ⌊ h / 3 ⌋ ) \beta_v(h-\tau)\le\beta_v(\lfloor h/3\rfloor) β v ( h − τ ) ≤ β v (⌊ h /3 ⌋) , and hence
∥ Cov ( s t ( τ ) , s t − h ( τ ) ) ∥ ≤ C β v ( ⌊ h / 3 ⌋ ) p − 2 p . (304) \left\|\operatorname{Cov}\left(s_t^{(\tau)},s_{t-h}^{(\tau)}\right)\right\| \le C\beta_v(\lfloor h/3\rfloor)^{\frac{p-2}{p}}. \tag*{(304)} Cov ( s t ( τ ) , s t − h ( τ ) ) ≤ C β v (⌊ h /3 ⌋ ) p p − 2 . (304)
Since E [ s t ] = 0 d \mathbb{E}[s_t]=\mathbf{0}_d E [ s t ] = 0 d , Cov ( s t , s t − h ) = E [ s t s t − h T ] \operatorname{Cov}(s_t,s_{t-h})=\mathbb{E}\left[s_ts_{t-h}^{\mathsf T}\right] Cov ( s t , s t − h ) = E [ s t s t − h T ] and therefore, combining (303) and (304) yields
∥ E [ ∇ θ ℓ F ( F ˉ t ; θ 0 ) ∇ θ ℓ F ( F ˉ t − h ; θ 0 ) T ] ∥ ≤ C ( ϱ k ⌊ h / ( 3 k ) ⌋ + β v ( ⌊ h / 3 ⌋ ) p − 2 p ) . (305) \left\|\mathbb{E}\left[\nabla^\theta\ell_F\left(\bar F_t;\theta_0\right)\nabla^\theta\ell_F\left(\bar F_{t-h};\theta_0\right)^{\mathsf T}\right]\right\|
\le C\left(\varrho_k^{\lfloor h/(3k)\rfloor}+\beta_v(\lfloor h/3\rfloor)^{\frac{p-2}{p}}\right). \tag*{(305)} E [ ∇ θ ℓ F ( F ˉ t ; θ 0 ) ∇ θ ℓ F ( F ˉ t − h ; θ 0 ) T ] ≤ C ( ϱ k ⌊ h / ( 3 k )⌋ + β v (⌊ h /3 ⌋ ) p p − 2 ) . (305)
By Assumption 1(i), β v ( h ) \beta_v(h) β v ( h ) decreases geometrically, and thus, it clearly holds that ∑ h = 1 ∞ ϱ k ⌊ h / ( 3 k ) ⌋ < ∞ \sum_{h=1}^{\infty}\varrho_k^{\lfloor h/(3k)\rfloor}<\infty ∑ h = 1 ∞ ϱ k ⌊ h / ( 3 k )⌋ < ∞ , and ∑ h = 1 ∞ β v ( ⌊ h / 3 ⌋ ) p − 2 p < ∞ \sum_{h=1}^{\infty}\beta_v(\lfloor h/3\rfloor)^{\frac{p-2}{p}}<\infty ∑ h = 1 ∞ β v (⌊ h /3 ⌋ ) p p − 2 < ∞ . The covariance at h = 0 h=0 h = 0 is finite by (300) , while stationarity gives the same bound for negative lags. Consequently,
∑ h = − ∞ ∞ ∥ E [ ∇ θ ℓ F ( F ˉ t ; θ 0 ) ∇ θ ℓ F ( F ˉ t − h ; θ 0 ) T ] ∥ < ∞ . (306) \sum_{h=-\infty}^{\infty}\left\|\mathbb{E}\left[\nabla^\theta\ell_F\left(\bar F_t;\theta_0\right)\nabla^\theta\ell_F\left(\bar F_{t-h};\theta_0\right)^{\mathsf T}\right]\right\|<\infty. \tag*{(306)} h = − ∞ ∑ ∞ E [ ∇ θ ℓ F ( F ˉ t ; θ 0 ) ∇ θ ℓ F ( F ˉ t − h ; θ 0 ) T ] < ∞. (306)
By stationarity and the zero-mean property of the stationary score we further obtain
1 T V T ( θ 0 ) = ∑ h = − ( T − 1 ) T − 1 ( 1 − ∣ h ∣ T ) E [ ∇ θ ℓ F ( F ˉ t ; θ 0 ) ∇ θ ℓ F ( F ˉ t − h ; θ 0 ) T ] . \frac{1}{T}V_T(\theta_0)=\sum_{h=-(T-1)}^{T-1}\left(1-\frac{|h|}{T}\right)\mathbb{E}\left[\nabla^\theta\ell_F\left(\bar F_t;\theta_0\right)\nabla^\theta\ell_F\left(\bar F_{t-h};\theta_0\right)^{\mathsf T}\right]. T 1 V T ( θ 0 ) = h = − ( T − 1 ) ∑ T − 1 ( 1 − T ∣ h ∣ ) E [ ∇ θ ℓ F ( F ˉ t ; θ 0 ) ∇ θ ℓ F ( F ˉ t − h ; θ 0 ) T ] .
Therefore, (306) and the dominated convergence theorem imply (297) as T → ∞ T\to\infty T → ∞ , with V ( θ 0 ) V(\theta_0) V ( θ 0 ) given by (298) . Now fix any a ∈ R d \mathbf{a}\in\mathbb{R}^d a ∈ R d . The scalar process { a T ∇ θ ℓ F ( F ˉ t ; θ 0 ) } t ∈ Z \left\{\mathbf{a}^{\mathsf T}\nabla^\theta\ell_F\left(\bar F_t;\theta_0\right)\right\}_{t\in\mathbb{Z}} { a T ∇ θ ℓ F ( F ˉ t ; θ 0 ) } t ∈ Z has mean zero, has a bounded p p p -th moment, and is near-epoch dependent of size − 1 -1 − 1 on the geometrically β \beta β -mixing process { v t } t ∈ Z \{v_t\}_{t\in\mathbb{Z}} { v t } t ∈ Z . Since every β \beta β -mixing process is also α \alpha α -mixing and its α \alpha α -mixing coefficients are bounded by its β \beta β -mixing coefficients, Assumption 1(i) implies that { v t } t ∈ Z \{v_t\}_{t\in\mathbb{Z}} { v t } t ∈ Z is α \alpha α -mixing of size − 2 p / ( p − 2 ) -2p/(p-2) − 2 p / ( p − 2 ) . Hence, by Theorem 10.2 of Pötscher and Prucha [47 ] we obtain
1 T ∑ t = 1 T a T ∇ θ ℓ F ( F ˉ t ; θ 0 ) ⇒ N ( 0 , a T V ( θ 0 ) a ) \frac{1}{\sqrt{T}}\sum_{t=1}^{T}\mathbf{a}^{\mathsf T}\nabla^\theta\ell_F\left(\bar F_t;\theta_0\right)\Rightarrow\mathcal{N}\left(0,\mathbf{a}^{\mathsf T}V(\theta_0)\mathbf{a}\right) T 1 t = 1 ∑ T a T ∇ θ ℓ F ( F ˉ t ; θ 0 ) ⇒ N ( 0 , a T V ( θ 0 ) a )
as T → ∞ T \to\infty T → ∞ . If a T V ( θ 0 ) a = 0 a^{\mathsf{T}} \mathbf{V}(\theta_0) a = 0 a T V ( θ 0 ) a = 0 , then, since the stationary score has mean zero,
∇ [ 1 T ∑ t = 1 T a T ∇ θ ℓ F ( F ˉ t ; θ 0 ) ] = a T V T ( θ 0 ) T a ⟶ a T V ( θ 0 ) a = 0 \nabla\left[\frac{1}{\sqrt{T}}\sum_{t=1}^{T} a^{\mathsf{T}}\nabla^{\theta}\ell_F\left(\bar{F}_t;\theta_0\right)\right] = a^{\mathsf{T}}\frac{\mathbf{V}_T(\theta_0)}{T}a \longrightarrow a^{\mathsf{T}}\mathbf{V}(\theta_0)a = 0 ∇ [ T 1 t = 1 ∑ T a T ∇ θ ℓ F ( F ˉ t ; θ 0 ) ] = a T T V T ( θ 0 ) a ⟶ a T V ( θ 0 ) a = 0
as T → ∞ T \to\infty T → ∞ , by (297) . Therefore, for every ε > 0 \varepsilon> 0 ε > 0 , the Chebyshev’s inequality implies
P ( ∣ 1 T ∑ t = 1 T a T ∇ θ ℓ F ( F ˉ t ; θ 0 ) ∣ > ε ) ≤ 1 ε 2 a T V T ( θ 0 ) T a ⟶ 0 \mathbb{P}\left(\left|\frac{1}{\sqrt{T}}\sum_{t=1}^{T}a^{\mathsf{T}}\nabla^{\theta}\ell_F\left(\bar{F}_t;\theta_0\right)\right|>\varepsilon\right) \le\frac{1}{\varepsilon^2}a^{\mathsf{T}}\frac{\mathbf{V}_T(\theta_0)}{T}a \longrightarrow0 P ( T 1 t = 1 ∑ T a T ∇ θ ℓ F ( F ˉ t ; θ 0 ) > ε ) ≤ ε 2 1 a T T V T ( θ 0 ) a ⟶ 0
as T → ∞ T \to\infty T → ∞ . Hence, 1 T ∑ t = 1 T a T ∇ θ ℓ F ( F ˉ t ; θ 0 ) → P 0 \frac{1}{\sqrt{T}}\sum_{t=1}^{T}a^{\mathsf{T}}\nabla^{\theta}\ell_F\left(\bar{F}_t;\theta_0\right)\xrightarrow{\mathbb{P}}0 T 1 ∑ t = 1 T a T ∇ θ ℓ F ( F ˉ t ; θ 0 ) P 0 , and therefore it converges in distribution to the degenerate normal distribution N ( 0 , 0 ) \mathcal{N}(0,0) N ( 0 , 0 ) . The Cramér–Wold device consequently yields
1 T ∑ t = 1 T ∇ θ ℓ F ( F ˉ t ; θ 0 ) ⇒ N ( 0 d , V ( θ 0 ) ) (307) \frac{1}{\sqrt{T}}\sum_{t=1}^{T}\nabla^{\theta}\ell_F\left(\bar{F}_t;\theta_0\right) \Rightarrow\mathcal{N}\left(\mathbf{0}_d,\mathbf{V}(\theta_0)\right) \tag*{(307)} T 1 t = 1 ∑ T ∇ θ ℓ F ( F ˉ t ; θ 0 ) ⇒ N ( 0 d , V ( θ 0 ) ) (307)
as T → ∞ T \to\infty T → ∞ .
Finally, by evaluating (287) at θ = θ 0 \theta= \theta_0 θ = θ 0 we get
1 T ∥ ∑ t = 1 T ∇ θ ℓ F ( F t ; θ 0 ) − ∑ t = 1 T ∇ θ ℓ F ( F ˉ t ; θ 0 ) ∥ = T ∥ 1 T ∑ t = 1 T ∇ θ ℓ F ( F t ; θ 0 ) − 1 T ∑ t = 1 T ∇ θ ℓ F ( F ˉ t ; θ 0 ) ∥ = O P ( 1 T ) , \begin{aligned}
\frac{1}{\sqrt{T}}\left\|\sum_{t=1}^{T}\nabla^{\theta}\ell_F\left(F_t;\theta_0\right)-\sum_{t=1}^{T}\nabla^{\theta}\ell_F\left(\bar{F}_t;\theta_0\right)\right\| \\
= \sqrt{T}\left\|\frac{1}{T}\sum_{t=1}^{T}\nabla^{\theta}\ell_F\left(F_t;\theta_0\right)-\frac{1}{T}\sum_{t=1}^{T}\nabla^{\theta}\ell_F\left(\bar{F}_t;\theta_0\right)\right\| = O_{\mathbb{P}}\left(\frac{1}{\sqrt{T}}\right),
\end{aligned} T 1 t = 1 ∑ T ∇ θ ℓ F ( F t ; θ 0 ) − t = 1 ∑ T ∇ θ ℓ F ( F ˉ t ; θ 0 ) = T T 1 t = 1 ∑ T ∇ θ ℓ F ( F t ; θ 0 ) − T 1 t = 1 ∑ T ∇ θ ℓ F ( F ˉ t ; θ 0 ) = O P ( T 1 ) ,
and thus, combining this result with (307) and applying Slutsky’s theorem proves (299) . ▪
Additional Monte Carlo studies
Trace statistics
We measure the performance of the estimated unobservable process { ψ ^ t ∣ t − 1 ( θ ) } t ∈ N \{\widehat{\psi}_{t|t-1}(\theta)\}_{t\in\mathbb{N}} { ψ t ∣ t − 1 ( θ ) } t ∈ N by means of the following trace statistics, which is not affected by the identification indeterminacy of the factors:
T R M L E = tr ( ψ T ψ ^ ( θ ^ ) ( ψ ^ ( θ ^ ) T ψ ^ ( θ ^ ) ) − 1 ψ ^ ( θ ^ ) T ψ ) tr ( ψ T ψ ) TR_{MLE}=\frac{\operatorname{tr}\left(\psi^{\mathsf{T}}\widehat{\psi}(\widehat{\theta})\left(\widehat{\psi}(\widehat{\theta})^{\mathsf{T}}\widehat{\psi}(\widehat{\theta})\right)^{-1}\widehat{\psi}(\widehat{\theta})^{\mathsf{T}}\psi\right)}{\operatorname{tr}\left(\psi^{\mathsf{T}}\psi\right)} T R M L E = tr ( ψ T ψ ) tr ( ψ T ψ ( θ ) ( ψ ( θ ) T ψ ( θ ) ) − 1 ψ ( θ ) T ψ )
where ψ = ( ψ 1 ∣ 0 , … , ψ T ∣ T − 1 ) T \psi=(\psi_{1|0},\ldots,\psi_{T|T-1})^{\mathsf{T}} ψ = ( ψ 1∣0 , … , ψ T ∣ T − 1 ) T and ψ ^ ( θ ^ ) = ( ψ ^ 1 ∣ 0 ( θ ^ ) , … , ψ ^ T ∣ T − 1 ( θ ^ ) ) T \widehat{\psi}(\widehat{\theta})=(\widehat{\psi}_{1|0}(\widehat{\theta}),\ldots,\widehat{\psi}_{T|T-1}(\widehat{\theta}))^{\mathsf{T}} ψ ( θ ) = ( ψ 1∣0 ( θ ) , … , ψ T ∣ T − 1 ( θ ) ) T denote the true and filtered (which is also a function of the MLE) time-varying parameter processes, respectively. The closer T R M L E TR_{MLE} T R M L E is to one, the closer the space spanned by ψ ^ ( θ ^ ) \widehat{\psi}(\widehat{\theta}) ψ ( θ ) is to the space spanned by ψ \psi ψ . Results are in Table 4 .
Table 4. Monte Carlo simulation results for the filtered conditional locations and variances
Monte Carlo averages of T R M L E TR_{MLE} T R M L E for the filtered vectors of conditional locations and conditional variances. Results are based on 100 replications.
Results when using spatial Kendall's PCs as in He et al. (2020)
Table 5. Monte Carlo simulation results for the QMLE of the conditional location and variance models using spatial Kendall’s tau factor estimates
Monte Carlo averages of the squared bias and variance of the QMLE under Student’s t 4 t_4 t 4 innovations, using spatial Kendall’s tau factor estimates in the first stage. For the conditional location models, entries are arithmetic means of the coordinate-specific values for vec ( Φ ) \operatorname{vec}(\boldsymbol{\Phi}) vec ( Φ ) and vec ( K ) \operatorname{vec}(\boldsymbol{K}) vec ( K ) . For the conditional variance models, the averages additionally include the intercept parameters δ \boldsymbol{\delta} δ . Results are based on 100 replications.
Table 6. Monte Carlo simulation results for the filtered conditional locations and variances using spatial Kendall’s tau factor estimates
Monte Carlo averages of T R M L E T_{\mathrm{RMLE}} T RMLE for the filtered vectors of conditional locations and conditional variances under Student’s t 4 t_4 t 4 innovations, using spatial Kendall’s tau factor estimates in the first stage. Results are based on 100 replications.