JEL classification: C12; C14; C58
Introduction
Financial market volatility fluctuates over multiple horizons. Daily return innovations contain immediate information about current volatility, while structural changes in macroeconomic and market conditions can generate slow-moving variation in unconditional variance Hong et al., [21 ] . Capturing both sources of variation is important for volatility measurement, forecasting, and risk management. In the seminal generalized autoregressive conditional heteroskedasticity (GARCH) frameworks of Engle [12 ] and Bollerslev [4 ] , short-run volatility clustering is modeled through a conditional variance driven by lagged returns and past volatility. As emphasized by Politis [31 ] , however, conditioning exclusively on past information may overlook the contemporaneous signal contained in the current return. A related insight underlies the realized-volatility literature, which demonstrates the value of within-period information for tracking rapid changes in volatility [3 , 19 ] .
To incorporate contemporaneous information directly into daily return models, Smetanina [33 ] introduced the real-time GARCH (RT-GARCH) model, in which the current innovation enters both the return equation and the volatility process. Unlike stochastic volatility models, which introduce a separate latent volatility shock, RT-GARCH allows the volatility scale to respond immediately to the return innovation while retaining a tractable closed-form likelihood. Smetanina and Wu [34 ] subsequently established the large-sample theory for maximum likelihood estimation of the stationary RT-GARCH model. More recent extensions have incorporated joint nowcasting and forecasting, low-frequency macroeconomic covariates, heavy-tailed conditional distributions, realized volatility measures, and intraday range information; see, for example, Ding [11 ] ; Wu et al. (2023); Kruse-Becher and Liu [25 ] ; Wu and Cai (2025); Xu and Wu [42 ] . Despite their flexibility in describing short-run volatility dynamics, these models maintain a constant unconditional variance and may therefore be misspecified over long spans of financial data.
A complementary literature addresses nonstationarity in unconditional volatility. Mikosch and Stărică [28 ] show that unmodeled changes in baseline volatility can generate spurious integrated-GARCH behavior and distort estimates of volatility persistence when constant unconditional variance is imposed. This observation has motivated models with smooth deterministic trends or structural breaks [35 , 13 , 6 , 36 ] , as well as locally stationary ARCH processes with time-varying parameters [9 ] . Within this literature, multiplicative component models provide an especially convenient framework by decomposing returns into a smooth deterministic scale and a stationary GARCH process [14 , 18 , 23 , 8 ] . Related work studies adaptive estimation, specification testing, outlier robustness, and simultaneous inference in time-varying volatility models [24 , 5 , 1 , 2 ] . Existing multiplicative GARCH specifications, however, continue to rely on lagged return shocks and therefore do not exploit contemporaneous return information in the short-run volatility recursion.
In this paper, we bridge these two strands of the literature by proposing a semiparametric real-time GARCH (SRT-GARCH) model. Returns are decomposed into a smooth deterministic scale function τ ( t / T ) \tau(t/T) τ ( t / T ) and a stationary RT-GARCH process. A second-moment normalization of the short-run component identifies τ ( t / T ) \tau(t/T) τ ( t / T ) directly as the time-varying unconditional variance of returns. The proposed specification nests the semiparametric GARCH (S-GARCH) model of Jiang et al. [23 ] when the contemporaneous feedback parameter is zero and a variance-targeted version of the RT-GARCH model of Smetanina [33 ] when the long-run scale is constant. We characterize the conditional moments and conditional density of the model. The long-run component determines the overall volatility, whereas the real-time feedback parameter introduces additional short-run distributional flexibility, including time-dependent conditional kurtosis and, under Gaussian innovations, exponential-type conditional heavy tails.
Combining an unknown long-run volatility component with contemporaneous short-run feedback creates nontrivial challenges for estimation and inference. The asymptotic theory cannot be obtained by simply combining existing results for S-GARCH and RT-GARCH models. First, because short-run volatility depends contemporaneously on the current innovation, recovering the latent volatility scale from observed returns requires inversion of a nonlinear quadratic equation at each time point. The resulting observed-data volatility recursion and its parameter derivatives therefore follow coupled nonlinear dynamics rather than the linear recursions familiar from conventional GARCH models. To control temporal dependence without imposing high-level α \alpha α - or β \beta β -mixing assumptions, we exploit a random-coefficient representation of the true short-run state to establish the geometric moment contraction property [37 , 32 ] . The resulting coupling bounds are then extended to the nonlinear observed-data volatility process and its higher-order parameter derivatives.
Second, because the deterministic variance function τ ( ⋅ ) \tau(\cdot) τ ( ⋅ ) is unknown, we estimate the model in two steps. We first estimate τ ( ⋅ ) \tau(\cdot) τ ( ⋅ ) nonparametrically from squared returns using a boundary-reflected Nadaraya–Watson kernel estimator τ ^ t \widehat{\tau}_t τ t for τ t = τ ( t / T ) \tau_t=\tau(t/T) τ t = τ ( t / T ) . We then estimate the short-run parameters by Gaussian conditional maximum likelihood based on the standardized returns u ^ t = r t / τ ^ t \widehat{u}_t=r_t/\sqrt{\widehat{\tau}_t} u t = r t / τ t , where { r t } \{r_t\} { r t } is the observed return process, and u t u_t u t denotes the corresponding oracle RT-GARCH process. Since u ^ t 2 \widehat{u}_t^2 u t 2 enters both the contemporaneous quadratic root and all subsequent volatility states, the first-step estimation error propagates nonlinearly through the likelihood recursion and has a non-vanishing first-order effect on the limiting distribution of the second-step estimator. By projecting this infinite-horizon perturbation onto the first-step innovations, we establish consistency and asymptotic normality of the two-step estimator. Under the variance normalization, we further show that the second-step score is contemporaneously orthogonal to the martingale innovation driving the first-step error and that the infinite-series sensitivity vector admits a closed-form contemporaneous representation. These two structural properties reduce the contribution of first-step estimation to a positive semidefinite rank-one adjustment of the oracle covariance matrix and yield a tuning-free plug-in asymptotic covariance estimator that requires neither lag truncation nor additional tuning parameters.
We also derive closed-form expressions for multi-step-ahead volatility forecasts and one-step-ahead Value-at-Risk (VaR) forecasts. Monte Carlo experiments indicate that the finite-sample behavior of the estimator and the associated inference procedure is broadly consistent with the asymptotic theory. We apply the model to daily FTSE 100 returns and U.S. natural gas futures. In both applications, covariance stationarity is rejected, providing evidence consistent with a time-varying long-run volatility component. The forecasting results show that incorporating a time-varying long-run component improves substantially on the RT-GARCH benchmark, while the additional real-time feedback mechanism delivers more pronounced gains in the natural gas application than in the FTSE application. The VaR results are also broadly supportive of the proposed specification.
The remainder of the paper is organized as follows. § 2 introduces the SRT-GARCH model and characterizes its moment and distributional properties. § 3 develops the two-step estimation procedure, establishes the large-sample theory, and derives the plug-in covariance estimator. § 4 presents the volatility forecasting framework and its multi-step extension. § 5 and § 6 report the Monte Carlo experiments and empirical applications, respectively. § 7 concludes. All technical proofs are provided in the Appendix.
The SRT-GARCH(1,1) Model
For an observed return series { r t } t = 1 T \{r_t\}_{t=1}^{T} { r t } t = 1 T , we consider the following semiparametric real-time GARCH (SRT-GARCH) model:
{ r t = τ t u t , τ t = τ ( t / T ) , u t = h t ε t , t = 1 , … , T , h t = ω + α u t − 1 2 + β h t − 1 + φ ε t 2 , (1) \begin{cases}
r_t = \sqrt{\tau_t}u_t, & \tau_t=\tau(t/T),\\
u_t = \sqrt{h_t}\,\varepsilon_t, & t=1,\ldots,T,\\
h_t = \omega+\alpha u_{t-1}^{2}+\beta h_{t-1}+\varphi\varepsilon_t^{2}, \tag*{(1)}
\end{cases} ⎩ ⎨ ⎧ r t = τ t u t , u t = h t ε t , h t = ω + α u t − 1 2 + β h t − 1 + φ ε t 2 , τ t = τ ( t / T ) , t = 1 , … , T , (1)
where τ : [ 0 , 1 ] → ( 0 , ∞ ) \tau:[0,1]\to(0,\infty) τ : [ 0 , 1 ] → ( 0 , ∞ ) is an unknown, smoothly time-varying deterministic function.
In model (1) , τ t \tau_t τ t captures slow-moving changes in the unconditional variance, while u t u_t u t governs stationary short-run volatility dynamics. The innovations { ϵ t } \{\epsilon_t\} { ϵ t } are assumed to be independent and identically distributed (i.i.d.) with zero mean and unit variance. The short-run parameters satisfy ω > 0 \omega> 0 ω > 0 and α , β , φ ≥ 0 \alpha,\beta,\varphi\ge0 α , β , φ ≥ 0 . Here, the coefficient φ \varphi φ measures the immediate feedback of the contemporaneous return shock on the short-run scale: setting φ = 0 \varphi= 0 φ = 0 recovers the semiparametric GARCH (S-GARCH) model of Jiang et al. [23 ] , whereas restricting τ t \tau_t τ t to be constant yields a variance-normalized version of the real-time GARCH (RT-GARCH) model of Smetanina [33 ] .
Unlike stochastic volatility models, model (1) does not introduce a separate latent volatility innovation. Because the same shock ϵ t \epsilon_t ϵ t drives both the return equation and the short-run volatility update, h t h_t h t and ϵ t \epsilon_t ϵ t can be uniquely recovered from the observed return r t r_t r t given past information via a nonlinear transformation. This property yields a closed-form conditional density and makes conditional maximum likelihood estimation feasible.
We begin with the regularity conditions that ensure the short-run process is well defined and the long-run component is identifiable. Let θ = ( α , β , φ ) T \theta= (\alpha,\beta,\varphi)^\mathsf{T} θ = ( α , β , φ ) T and denote the true parameter as θ 0 = ( α 0 , β 0 , φ 0 ) T \theta_0 = (\alpha_0,\beta_0,\varphi_0)^\mathsf{T} θ 0 = ( α 0 , β 0 , φ 0 ) T .
Assumption 1. (i) The parameter space Θ \Theta Θ is a compact subset of
{ ( α , β , φ ) T ∈ [ 0 , ∞ ) 3 : α + β ≤ ρ < 1 , ω ( θ ) > 0 } \left\{(\alpha,\beta,\varphi)^\mathsf{T} \in[0,\infty)^3 : \alpha+ \beta\le\rho< 1,\ \omega(\theta) > 0\right\} { ( α , β , φ ) T ∈ [ 0 , ∞ ) 3 : α + β ≤ ρ < 1 , ω ( θ ) > 0 } where ρ ∈ ( 0 , 1 ) \rho\in(0,1) ρ ∈ ( 0 , 1 ) is a fixed constant, and
ω ( θ ) = 1 − α − β − β φ + φ ( β − 1 ) μ 4 , μ 4 = E ( ϵ t 4 ) . (2) \omega(\theta) = 1 - \alpha- \beta- \beta\varphi+ \varphi(\beta- 1)\mu_4,\qquad\mu_4 = \mathbb{E}(\epsilon_t^4). \tag*{(2)} ω ( θ ) = 1 − α − β − β φ + φ ( β − 1 ) μ 4 , μ 4 = E ( ϵ t 4 ) . (2) (ii) The true parameter satisfies θ 0 ∈ Θ \theta_0 \in\Theta θ 0 ∈ Θ .
(iii) The innovations { ϵ t } \{\epsilon_t\} { ϵ t } are independently and identically distributed, symmetric about zero, and satisfy E ( ϵ t ) = 0 \mathbb{E}(\epsilon_t) = 0 E ( ϵ t ) = 0 , E ( ϵ t 2 ) = 1 \mathbb{E}(\epsilon_t^2) = 1 E ( ϵ t 2 ) = 1 , and μ 4 = E ( ϵ t 4 ) < ∞ \mu_4 = \mathbb{E}(\epsilon_t^4) < \infty μ 4 = E ( ϵ t 4 ) < ∞ .
Normalization and Second-Order Properties
Assumption 1(i) defines a compact parameter space. The nonnegativity restrictions together with ω ( θ ) > 0 \omega(\theta) > 0 ω ( θ ) > 0 , ensure the positivity of the real-time scale h t h_t h t . The persistence condition α + β ≤ ρ < 1 \alpha+\beta\le\rho< 1 α + β ≤ ρ < 1 guarantees that the short-run dynamic admits a strictly stationary and ergodic solution with finite second-order moments, as formalized in the following proposition.
Proposition 1. Under Assumption 1, the short-run recursion has a unique causal, strictly stationary, and ergodic solution satisfying E ( h t ) < ∞ \mathbb{E}(h_t) < \infty E ( h t ) < ∞ and E ( u t 2 ) < ∞ \mathbb{E}(u_t^2) < \infty E ( u t 2 ) < ∞ .
To separately identify the nonparametric scale τ t \tau_t τ t and the short-run process u t u_t u t in (1) , Assumption 1 also helps normalize the unconditional variance of u t u_t u t to one so that τ t \tau_t τ t can be interpreted as the unconditional variance.
Proposition 2. Under Assumption 1, we have E ( u t 2 ) = 1 \mathbb{E}(u_t^2) = 1 E ( u t 2 ) = 1 , and E ( h t ) = 1 − φ ( μ 4 − 1 ) \mathbb{E}(h_t) = 1 - \varphi(\mu_4 - 1) E ( h t ) = 1 − φ ( μ 4 − 1 ) . Moreover, E ( u t ∣ F t − 1 ) = 0 \mathbb{E}(u_t \mid\mathcal{F}_{t-1}) = 0 E ( u t ∣ F t − 1 ) = 0 and E ( u t u t − j ) = 0 \mathbb{E}(u_tu_{t-j}) = 0 E ( u t u t − j ) = 0 for every j ≥ 1 j \ge1 j ≥ 1 , i.e. { u t } \{u_t\} { u t } is a martingale difference sequence with unit unconditional variance.
Unlike standard GARCH models, where E ( u t 2 ) = E ( h t ) \mathbb{E}(u_t^2) = \mathbb{E}(h_t) E ( u t 2 ) = E ( h t ) , the normalization in (2) must account for the contemporaneous covariance between h t h_t h t and ϵ t 2 \epsilon_t^2 ϵ t 2 . Writing m u = E ( u t 2 ) m_u = \mathbb{E}(u_t^2) m u = E ( u t 2 ) and m h = E ( h t ) m_h = \mathbb{E}(h_t) m h = E ( h t ) and taking unconditional expectations in the short-run equations h t = ω + α u t − 1 2 + β h t − 1 + φ ϵ t 2 h_t = \omega+ \alpha u_{t-1}^2 + \beta h_{t-1} + \varphi\epsilon_t^2 h t = ω + α u t − 1 2 + β h t − 1 + φ ϵ t 2 and u t 2 = ( ω + α u t − 1 2 + β h t − 1 ) ϵ t 2 + φ ϵ t 4 u_t^2 = (\omega+ \alpha u_{t-1}^2 + \beta h_{t-1})\epsilon_t^2 + \varphi\epsilon_t^4 u t 2 = ( ω + α u t − 1 2 + β h t − 1 ) ϵ t 2 + φ ϵ t 4 yields that
m h = ω + α m u + β m h + φ , m u = ω + α m u + β m h + φ μ 4 . \begin{aligned}
m_h &= \omega+ \alpha m_u + \beta m_h + \varphi,\\
m_u &= \omega+ \alpha m_u + \beta m_h + \varphi\mu_4.
\end{aligned} m h m u = ω + α m u + β m h + φ , = ω + α m u + β m h + φ μ 4 .
Together with (2) and α + β < 1 \alpha+ \beta< 1 α + β < 1 , we obtain that m u = 1 m_u = 1 m u = 1 and m h = 1 − φ ( μ 4 − 1 ) m_h = 1 - \varphi(\mu_4 - 1) m h = 1 − φ ( μ 4 − 1 ) . Consequently, E ( r t 2 ) = τ t \mathbb{E}(r_t^2) = \tau_t E ( r t 2 ) = τ t , so the long-run component has the direct interpretation of unconditional return variance. When φ = 0 \varphi= 0 φ = 0 , the normalization reduces to ω = 1 − α − β \omega= 1 - \alpha- \beta ω = 1 − α − β as in Francq et al. [15 ] and Jiang et al. [23 ] ; under standard Gaussian innovations, it becomes ω = 1 − α − β − 3 φ + 2 β φ \omega= 1 - \alpha- \beta- 3\varphi+ 2\beta\varphi ω = 1 − α − β − 3 φ + 2 β φ .
Conditional Moments
While the deterministic function τ t \tau_t τ t represents the overall level of return variance, the real-time feedback parameter φ 0 \varphi_0 φ 0 can change the shape of the conditional return distribution. Following Smetanina [33 ] , we characterize their different roles through the second and fourth conditional moments of r t r_t r t .
Let b t − 1 = ω 0 + α 0 u t − 1 2 + β 0 h t − 1 b_{t-1} = \omega_0 + \alpha_0 u_{t-1}^2 + \beta_0 h_{t-1} b t − 1 = ω 0 + α 0 u t − 1 2 + β 0 h t − 1 denote the short-run scale, so that h t = b t − 1 + φ 0 ϵ t 2 h_t = b_{t-1} + \varphi_0\epsilon_t^2 h t = b t − 1 + φ 0 ϵ t 2 and u t 2 = b t − 1 ϵ t 2 + φ 0 ϵ t 4 u_t^2 = b_{t-1}\epsilon_t^2 + \varphi_0\epsilon_t^4 u t 2 = b t − 1 ϵ t 2 + φ 0 ϵ t 4 . Write μ j = E ( ϵ t j ) \mu_j = \mathbb{E}(\epsilon_t^j) μ j = E ( ϵ t j ) for j = 2 , 4 , 6 , 8 j = 2,4,6,8 j = 2 , 4 , 6 , 8 , with μ 2 = 1 \mu_2 = 1 μ 2 = 1 .
Proposition 3. Suppose that Assumption 1 holds and E ( ϵ t 8 ) < ∞ \mathbb{E}(\epsilon_t^8) < \infty E ( ϵ t 8 ) < ∞ . Then
E ( r t 2 ∣ F t − 1 ) = τ t ( b t − 1 + φ 0 μ 4 ) , (3) \mathbb{E}(r_t^2 \mid\mathcal{F}_{t-1}) = \tau_t\left(b_{t-1} + \varphi_0\mu_4\right), \tag*{(3)} E ( r t 2 ∣ F t − 1 ) = τ t ( b t − 1 + φ 0 μ 4 ) , (3) E ( r t 4 ∣ F t − 1 ) = τ t 2 ( μ 4 b t − 1 2 + 2 φ 0 μ 6 b t − 1 + φ 0 2 μ 8 ) . \mathbb{E}(r_t^4 \mid\mathcal{F}_{t-1}) = \tau_t^2\left(\mu_4b_{t-1}^2 + 2\varphi_0\mu_6b_{t-1} + \varphi_0^2\mu_8\right). E ( r t 4 ∣ F t − 1 ) = τ t 2 ( μ 4 b t − 1 2 + 2 φ 0 μ 6 b t − 1 + φ 0 2 μ 8 ) . Consequently,
E ( r t 4 ∣ F t − 1 ) [ E ( r t 2 ∣ F t − 1 ) ] 2 = μ 4 b t − 1 2 + 2 φ 0 μ 6 b t − 1 + φ 0 2 μ 8 ( b t − 1 + φ 0 μ 4 ) 2 . (4) \frac{\mathbb{E}(r_t^4 \mid\mathcal{F}_{t-1})}{\left[\mathbb{E}(r_t^2 \mid\mathcal{F}_{t-1})\right]^2}
=
\frac{\mu_4b_{t-1}^2 + 2\varphi_0\mu_6b_{t-1} + \varphi_0^2\mu_8}{\left(b_{t-1} + \varphi_0\mu_4\right)^2}. \tag*{(4)} [ E ( r t 2 ∣ F t − 1 ) ] 2 E ( r t 4 ∣ F t − 1 ) = ( b t − 1 + φ 0 μ 4 ) 2 μ 4 b t − 1 2 + 2 φ 0 μ 6 b t − 1 + φ 0 2 μ 8 . (4)
Proposition 3 clarifies the different roles of the long-run and short-run components. The deterministic part τ t \tau_t τ t proportionally scales the conditional variance (3) and fourth moment, but cancels out of the standardized fourth moment (4) . By contrast, when φ 0 > 0 \varphi_0 > 0 φ 0 > 0 , the conditional kurtosis in (4) varies endogenously with the short-run component b t − 1 b_{t-1} b t − 1 . Thus, τ t \tau_t τ t governs low-frequency shifts in the variance level, whereas contemporaneous feedback generates time-varying conditional tail heaviness.
Conditional Density
Conditional on F t − 1 \mathcal{F}_{t-1} F t − 1 , the map ϵ t ↦ u t = ϵ t b t − 1 + φ 0 ϵ t 2 \epsilon_t \mapsto u_t = \epsilon_t\sqrt{b_{t-1} + \varphi_0\epsilon_t^2} ϵ t ↦ u t = ϵ t b t − 1 + φ 0 ϵ t 2 is strictly increasing and hence invertible for every φ 0 ≥ 0 \varphi_0 \ge0 φ 0 ≥ 0 . Squaring both sides gives the quadratic equation h t 2 − b t − 1 h t − φ 0 u t 2 = 0 h_t^2 - b_{t-1}h_t - \varphi_0u_t^2 = 0 h t 2 − b t − 1 h t − φ 0 u t 2 = 0 in h t = b t − 1 + φ 0 ϵ t 2 > 0 h_t = b_{t-1} + \varphi_0\epsilon_t^2 > 0 h t = b t − 1 + φ 0 ϵ t 2 > 0 . Therefore, solving for the positive root at u = r / τ t u = r/\sqrt{\tau_t} u = r / τ t yields the implied short-run scale
h t ( r ) = b t − 1 + b t − 1 2 + 4 φ 0 r 2 / τ t 2 . h_t(r) = \frac{b_{t-1} + \sqrt{b_{t-1}^2 + 4\varphi_0r^2/\tau_t}}{2}. h t ( r ) = 2 b t − 1 + b t − 1 2 + 4 φ 0 r 2 / τ t .
By the change of variables, we obtain the following conditional return density.
Proposition 4. Suppose that Assumption 1 holds and that the innovation ϵ t \epsilon_t ϵ t has density f ϵ f_\epsilon f ϵ . Then the conditional density of r t r_t r t given F t − 1 \mathcal{F}_{t-1} F t − 1 is
f t ( r ∣ F t − 1 ) = { h t ( r ) τ t b t − 1 2 + 4 φ 0 r 2 / τ t f ϵ ( r τ t h t ( r ) ) , φ 0 > 0 , 1 τ t b t − 1 f ϵ ( r τ t b t − 1 ) , φ 0 = 0. (5) f_t(r \mid\mathcal{F}_{t-1}) =
\begin{cases}
\frac{\sqrt{h_t(r)}}{\sqrt{\tau_t}\sqrt{b_{t-1}^2+4\varphi_0r^2/\tau_t}}f_\epsilon\left(\frac{r}{\sqrt{\tau_t h_t(r)}}\right), & \varphi_0>0,\\[1.2em]
\frac{1}{\sqrt{\tau_t b_{t-1}}}f_\epsilon\left(\frac{r}{\sqrt{\tau_t b_{t-1}}}\right), & \varphi_0=0.
\end{cases} \tag*{(5)} f t ( r ∣ F t − 1 ) = ⎩ ⎨ ⎧ τ t b t − 1 2 + 4 φ 0 r 2 / τ t h t ( r ) f ϵ ( τ t h t ( r ) r ) , τ t b t − 1 1 f ϵ ( τ t b t − 1 r ) , φ 0 > 0 , φ 0 = 0. (5)
Comparing the two cases in (5) showcases how real-time feedback can change the conditional return distribution. When φ 0 = 0 \varphi_0=0 φ 0 = 0 , the Jacobian factor reduces to ( τ t b t − 1 ) − 1 / 2 (\tau_t b_{t-1})^{-1/2} ( τ t b t − 1 ) − 1/2 , recovering the S-GARCH ([23 ] ) conditional density with constant conditional kurtosis μ 4 \mu_4 μ 4 . When φ 0 > 0 \varphi_0>0 φ 0 > 0 , the nonlinear transformation from ϵ t \epsilon_t ϵ t to r t r_t r t introduces a state-dependent Jacobian term that depends on both the predetermined state b t − 1 b_{t-1} b t − 1 and the current return r t r_t r t and stretches large realizations of ϵ t \epsilon_t ϵ t . Indeed, the proof in the Appendix shows that when f ϵ f_\epsilon f ϵ is standard Gaussian, any φ 0 > 0 \varphi_0>0 φ 0 > 0 transforms the conditional tails of r t r_t r t from Gaussian-type ( log f t ( r ∣ F t − 1 ) ≍ − r 2 ) (\log f_t(r\mid\mathcal{F}_{t-1})\asymp-r^2) ( log f t ( r ∣ F t − 1 ) ≍ − r 2 ) to exponential-type ( log f t ( r ∣ F t − 1 ) ≍ − ∣ r ∣ ) (\log f_t(r\mid\mathcal{F}_{t-1})\asymp-|r|) ( log f t ( r ∣ F t − 1 ) ≍ − ∣ r ∣ ) . This helps the model capture the heavy-tailed phenomenon commonly observed in financial returns.
Distributional Implications of the Real-Time Component
To illustrate how the contemporaneous feedback term φ ϵ t 2 \varphi\epsilon_t^2 φ ϵ t 2 affects the shape of the return distribution, we compare the SRT-GARCH model with its nested S-GARCH counterpart under three different long-run volatility patterns. Specifically,
D G P 1 ( S R T - G A R C H 1 ) : τ 1 ( x ) = 1 + 0.5 sin ( 2 π x ) , ( α , β , φ ) = ( 0.20 , 0.60 , 0.05 ) , D G P 2 ( S R T - G A R C H 2 ) : τ 2 ( x ) = 1 + 4 ( x − x 2 ) , ( α , β , φ ) = ( 0.20 , 0.50 , 0.03 ) , D G P 3 ( S R T - G A R C H 3 ) : τ 3 ( x ) = 1 + 4 x 2 , ( α , β , φ ) = ( 0.15 , 0.65 , 0.04 ) . \begin{aligned}
\mathrm{DGP\ 1\ (SRT\text{-}GARCH\ 1)}:\quad&\tau_1(x)=1+0.5\sin(2\pi x), \quad(\alpha,\beta,\varphi)=(0.20,0.60,0.05),\\
\mathrm{DGP\ 2\ (SRT\text{-}GARCH\ 2)}:\quad&\tau_2(x)=1+4(x-x^2), \quad(\alpha,\beta,\varphi)=(0.20,0.50,0.03),\\
\mathrm{DGP\ 3\ (SRT\text{-}GARCH\ 3)}:\quad&\tau_3(x)=1+4x^2, \quad(\alpha,\beta,\varphi)=(0.15,0.65,0.04).
\end{aligned} DGP 1 ( SRT - GARCH 1 ) : DGP 2 ( SRT - GARCH 2 ) : DGP 3 ( SRT - GARCH 3 ) : τ 1 ( x ) = 1 + 0.5 sin ( 2 π x ) , ( α , β , φ ) = ( 0.20 , 0.60 , 0.05 ) , τ 2 ( x ) = 1 + 4 ( x − x 2 ) , ( α , β , φ ) = ( 0.20 , 0.50 , 0.03 ) , τ 3 ( x ) = 1 + 4 x 2 , ( α , β , φ ) = ( 0.15 , 0.65 , 0.04 ) .
DGPs 1–3 capture periodic variation, a temporary rise followed by a decline, and a monotone nonlinear increase in unconditional variance, respectively. For each j = 1 , 2 , 3 j=1,2,3 j = 1 , 2 , 3 , S-GARCH j j j shares the same long-run component τ j ( x ) \tau_j(x) τ j ( x ) and parameters ( α , β ) (\alpha,\beta) ( α , β ) as DGP j j j , but sets φ = 0 \varphi=0 φ = 0 . In all specifications, the intercept ω \omega ω is determined by the normalization in (2) under standard Gaussian innovations.
Figure 1 plots the kernel density estimates of the simulated return series with T = 10,000 T=10{,}000 T = 10 , 000 observations alongside the standard normal reference density. Across the three long-run specifications, the SRT-GARCH return distributions exhibit a sharper peak around zero and noticeably heavier tails than their nested S-GARCH counterparts. Thus, incorporating real-time innovation feedback with a time-varying unconditional variance generates more pronounced leptokurtosis without requiring a heavy-tailed innovation distribution.
Figure 1. Kernel density estimates for the sample-standardized simulated returns with T = 10,000 T = 10{,}000 T = 10 , 000 .
Two-Step Estimation Procedure
The normalization in Section 2 separates long-run scale from short-run dynamics and suggests a natural two-step estimator. In the first step, we recover the smooth unconditional variance function τ ( ⋅ ) \tau(\cdot) τ ( ⋅ ) nonparametrically from the squared returns r t 2 r_t^2 r t 2 . In the second step, we standardize the observed returns as u ^ t = r t / τ ^ t \hat{u}_t=r_t/\sqrt{\hat{\tau}_t} u ^ t = r t / τ ^ t by the estimated scale τ ^ t \sqrt{\hat{\tau}_t} τ ^ t and estimate the short-run parameter θ = ( α , β , φ ) T ∈ Θ \theta=(\alpha,\beta,\varphi)^\mathsf{T}\in\Theta θ = ( α , β , φ ) T ∈ Θ , whose true value is denoted by θ 0 = ( α 0 , β 0 , φ 0 ) T \theta_0=(\alpha_0,\beta_0,\varphi_0)^\mathsf{T} θ 0 = ( α 0 , β 0 , φ 0 ) T , via Gaussian conditional maximum likelihood estimation. The parameter space allows zero values so that the nested S-GARCH specification ( φ = 0 ) (\varphi= 0) ( φ = 0 ) remains admissible. While our consistency result in Theorem 2 permits θ 0 \theta_0 θ 0 to lie on the boundary when the parameters are identified, deriving the asymptotic distribution in Theorem 3 requires θ 0 \theta_0 θ 0 to be an interior point of Θ \Theta Θ . The boundary case leads to nonstandard asymptotics and is highly non-trivial. We refer to Francq and Zakoïan [16 ] for a detailed treatment of boundary inference and testing in GARCH type models.
The two-step procedure follows the broad semiparametric strategy of Jiang et al. [23 ] , but the presence of contemporaneous feedback φ ϵ t 2 \varphi\epsilon_t^2 φ ϵ t 2 changes both the second-step likelihood recursion and its first-order asymptotic behavior in two fundamental ways. First, given the observations, the implied short-run scale h t ( θ ) h_t(\theta) h t ( θ ) is defined through a nonlinear quadratic root rather than a linear recursion as in standard GARCH models, so the observed-data volatility process and its parameter derivatives follow coupled nonlinear functions. Second, because the generated standardized returns u ^ t = r t / τ ^ t \widehat{u}_t = r_t/\sqrt{\widehat{\tau}_t} u t = r t / τ t enter both the current scale and the recursive state at every period, the first-step kernel estimation error can propagate through the entire volatility process and makes a non-vanishing first-order contribution to the limiting distribution. Developing the large-sample theory therefore requires tracking this nonlinear propagation explicitly and characterizing the resulting two-step adjustment, which is highly non-trivial.
Estimation of the Long-Run Component
Under the normalization E ( u t 2 ) = 1 \mathbb{E}(u_t^2) = 1 E ( u t 2 ) = 1 , we have E ( r t 2 ) = τ t = τ ( t / T ) \mathbb{E}(r_t^2) = \tau_t = \tau(t/T) E ( r t 2 ) = τ t = τ ( t / T ) . The squared return can therefore be written as
r t 2 = τ ( t / T ) + v t , T , v t , T = τ ( t / T ) ( u t 2 − 1 ) , (6) r_t^2 = \tau(t/T) + v_{t,T}, \qquad v_{t,T} = \tau(t/T)(u_t^2 - 1), \tag*{(6)} r t 2 = τ ( t / T ) + v t , T , v t , T = τ ( t / T ) ( u t 2 − 1 ) , (6)
where E ( v t , T ) = 0 \mathbb{E}(v_{t,T}) = 0 E ( v t , T ) = 0 . Equation (6) defines a fixed-design nonparametric regression with serially dependent and heteroskedastic errors.
Following Jiang et al. [23 ] , we estimate τ ( x ) \tau(x) τ ( x ) using the simplified Nadaraya–Watson kernel estimator
τ ^ ( x ) = 1 T ∑ s = 1 T K h ( x − s T ) r s 2 , x ∈ [ 0 , 1 ] , (7) \widehat{\tau}(x) = \frac{1}{T}\sum_{s=1}^{T} K_h\left(x-\frac{s}{T}\right)r_s^2, \qquad x \in[0,1], \tag*{(7)} τ ( x ) = T 1 s = 1 ∑ T K h ( x − T s ) r s 2 , x ∈ [ 0 , 1 ] , (7)
where h > 0 h > 0 h > 0 is the smoothing bandwidth and K h ( v ) = h − 1 K ( v / h ) K_h(v) = h^{-1}K(v/h) K h ( v ) = h − 1 K ( v / h ) . In the practical analysis, we use the Epanechnikov kernel K ( v ) = 3 4 ( 1 − v 2 ) I ( ∣ v ∣ ≤ 1 ) K(v) = \frac{3}{4}(1-v^2)I(|v| \le1) K ( v ) = 4 3 ( 1 − v 2 ) I ( ∣ v ∣ ≤ 1 ) .
To derive the asymptotic properties of τ ^ ( x ) \widehat{\tau}(x) τ ( x ) , we impose the following assumptions.
Assumption 2. The function τ : [ 0 , 1 ] → ( 0 , ∞ ) \tau: [0,1] \to(0,\infty) τ : [ 0 , 1 ] → ( 0 , ∞ ) is twice continuously differentiable, and there exist finite constants 0 < τ ‾ < τ ‾ < ∞ 0 < \underline{\tau} < \overline{\tau} < \infty 0 < τ < τ < ∞ such that τ ‾ ≤ τ ( x ) ≤ τ ‾ \underline{\tau} \le\tau(x) \le\overline{\tau} τ ≤ τ ( x ) ≤ τ for all x ∈ [ 0 , 1 ] x \in[0,1] x ∈ [ 0 , 1 ] .
Assumption 3. The kernel K K K is symmetric, bounded, and Lipschitz continuous, has compact support [ − 1 , 1 ] [-1,1] [ − 1 , 1 ] , and satisfies ∫ − 1 1 K ( v ) d v = 1 \int_{-1}^{1} K(v)\,\mathrm{d}v = 1 ∫ − 1 1 K ( v ) d v = 1 . Let C j = ∫ − 1 1 v j K ( v ) d v C_j = \int_{-1}^{1} v^j K(v)\,\mathrm{d}v C j = ∫ − 1 1 v j K ( v ) d v for j = 1 , 2 j = 1,2 j = 1 , 2 . As T → ∞ T \to\infty T → ∞ , the bandwidth satisfies h → 0 h \to0 h → 0 , T h → ∞ Th \to\infty T h → ∞ , and T h 5 = O ( 1 ) Th^5 = O(1) T h 5 = O ( 1 ) .
Assumption 4. For z t = u t 2 − 1 z_t = u_t^2 - 1 z t = u t 2 − 1 , E ( z t 2 ) < ∞ \mathbb{E}(z_t^2) < \infty E ( z t 2 ) < ∞ .
To analyze the kernel estimator τ ^ ( x ) \widehat{\tau}(x) τ ( x ) and the subsequent second-step likelihood, we must control the serial dependence of the short-run process { u t 2 } \{u_t^2\} { u t 2 } as well as the nonlinear observed-data volatility recursion and its parameter derivatives. Instead of imposing high-level α \alpha α - or β \beta β -mixing conditions on the unobservable process, we follow Smetanina and Wu [34 ] and use the physical dependence measure introduced by Wu [37 ] to establish geometric moment contraction (GMC) property in Shao and Wu [32 ] for these quantities.
Definition 1 (Geometric moment contraction). Let X t = F ( ϵ t , ϵ t − 1 , … ) X_t = F(\epsilon_t,\epsilon_{t-1},\ldots) X t = F ( ϵ t , ϵ t − 1 , … ) be a causal process, where F F F is measurable and { ϵ t } \{\epsilon_t\} { ϵ t } is i.i.d. Let { ϵ t ∘ } \{\epsilon_t^\circ\} { ϵ t ∘ } be an independent copy of the innovation sequence, and define
X t ∘ = F ( ϵ t , … , ϵ 1 , ϵ 0 ∘ , ϵ − 1 ∘ , … ) , t ≥ 1. X_t^\circ= F(\epsilon_t,\ldots,\epsilon_1,\epsilon_0^\circ,\epsilon_{-1}^\circ,\ldots), \qquad t \ge1. X t ∘ = F ( ϵ t , … , ϵ 1 , ϵ 0 ∘ , ϵ − 1 ∘ , … ) , t ≥ 1. For p ≥ 1 p \ge1 p ≥ 1 , the process is L p L^p L p geometrically moment contracting (GMC) if there exist C < ∞ C < \infty C < ∞ and ρ p ∈ ( 0 , 1 ) \rho_p \in(0,1) ρ p ∈ ( 0 , 1 ) such that
∥ X t − X t ∘ ∥ p ≤ C ρ p t , t ≥ 1 , \lVert X_t - X_t^\circ\rVert_p \le C\rho_p^t, \qquad t \ge1, ∥ X t − X t ∘ ∥ p ≤ C ρ p t , t ≥ 1 , where ∥ X ∥ p = ( E ∥ X ∥ p ) 1 / p \lVert X\rVert_p = (\mathbb{E}\lVert X\rVert^p)^{1/p} ∥ X ∥ p = ( E ∥ X ∥ p ) 1/ p .
Moreover, let X t ∗ = F ( ϵ t , … , ϵ 1 , ϵ 0 ∘ , ϵ − 1 , … ) X_t^* = F(\epsilon_t,\ldots,\epsilon_1,\epsilon_0^\circ,\epsilon_{-1},\ldots) X t ∗ = F ( ϵ t , … , ϵ 1 , ϵ 0 ∘ , ϵ − 1 , … ) denote the single-innovation coupled version obtained by replacing only ϵ 0 \epsilon_0 ϵ 0 with an independent copy. By Minkowski’s inequality and the i.i.d. structure of the innovations,
∥ X t − X t ∗ ∥ p ≤ ∥ X t − X t ∘ ∥ p + ∥ X t ∘ − X t ∗ ∥ p = ∥ X t − X t ∘ ∥ p + ∥ X t + 1 − X t + 1 ∘ ∥ p ≤ C ( 1 + ρ p ) ρ p t ≤ 2 C ρ p t . \begin{aligned}
\lVert X_t - X_t^* \rVert_p
&\le\lVert X_t - X_t^\circ\rVert_p + \lVert X_t^\circ- X_t^* \rVert_p \\
&= \lVert X_t - X_t^\circ\rVert_p + \lVert X_{t+1} - X_{t+1}^\circ\rVert_p \\
&\le C(1+\rho_p)\rho_p^t \le2C\rho_p^t.
\end{aligned} ∥ X t − X t ∗ ∥ p ≤ ∥ X t − X t ∘ ∥ p + ∥ X t ∘ − X t ∗ ∥ p = ∥ X t − X t ∘ ∥ p + ∥ X t + 1 − X t + 1 ∘ ∥ p ≤ C ( 1 + ρ p ) ρ p t ≤ 2 C ρ p t .
Hence, GMC also implies geometric decay of the single-innovation physical dependence measure. Furthermore, GMC can be extended to higher moment orders through interpolation. Specifically, if { X t } \{X_t\} { X t } is L r L^r L r GMC for some r > 1 r > 1 r > 1 and E ∥ X t ∥ a < ∞ \mathbb{E}\lVert X_t\rVert^a < \infty E ∥ X t ∥ a < ∞ for some a > r a > r a > r , then Hölder’s inequality implies L p L^p L p GMC for every p ∈ ( r , a ) p \in(r,a) p ∈ ( r , a ) , possibly with a different geometric decay rate; see Remark 3.1 of Shao and Wu [32 ] .
Lemma 1 in the Appendix collects this moment interpolation and the resulting recursive contraction and covariance bounds, which we apply to obtain the GMC properties for the short-run model process and score functions in Lemma 4. The following proposition establishes GMC for the short-run process.
Proposition 5 (Moment contraction of the short-run process). Under Assumption 1, suppose E ∣ u t ∣ 2 p < ∞ \mathbb{E}|u_t|^{2p} < \infty E ∣ u t ∣ 2 p < ∞ for some p ≥ 2 p \ge2 p ≥ 2 . Then b t = ω 0 + α 0 u t 2 + β 0 h t b_t = \omega_0 + \alpha_0 u_t^2 + \beta_0 h_t b t = ω 0 + α 0 u t 2 + β 0 h t , h t h_t h t , and z t = u t 2 − 1 z_t = u_t^2 - 1 z t = u t 2 − 1 are L p L^p L p GMC.
Assumptions 2 and 3 impose the usual smoothness, positivity, kernel, and bandwidth conditions. The restrictions h → 0 h \to0 h → 0 and T h → ∞ Th \to\infty T h → ∞ balance local bias and variance, while T h 5 = O ( 1 ) Th^5 = O(1) T h 5 = O ( 1 ) controls the remainder in the pointwise bias expansion. Assumption 4 only requires a finite variance for squared short-run returns; by Proposition 2, it is equivalent to E ( u t 4 ) < ∞ \mathbb{E}(u_t^4)<\infty E ( u t 4 ) < ∞ . For the RT-GARCH recursion, Proposition 5 shows that this moment condition and Assumption 1 imply that { z t } \{z_t\} { z t } is L 2 L^2 L 2 GMC; see, e.g., Wu and Shao (2004). The argument uses the random-coefficient representation of the true short-run state and the coupling approach of Wu [37 ] and Smetanina and Wu [34 ] . It is worth noting that the established GMC property implies absolute summability of the autocovariances and the functional central limit theorem
1 T ∑ t = 1 ⌊ T r ⌋ z t ⇒ Ω z 1 / 2 B ( r ) , 0 ≤ r ≤ 1 , \frac{1}{\sqrt{T}}\sum_{t=1}^{\lfloor Tr\rfloor}z_t \Rightarrow\Omega_z^{1/2}B(r), \qquad0\le r\le1, T 1 t = 1 ∑ ⌊ T r ⌋ z t ⇒ Ω z 1/2 B ( r ) , 0 ≤ r ≤ 1 ,
where Ω z = ∑ j = − ∞ ∞ Cov ( z 0 , z j ) \Omega_z=\sum_{j=-\infty}^{\infty}\operatorname{Cov}(z_0,z_j) Ω z = ∑ j = − ∞ ∞ Cov ( z 0 , z j ) and B ( ⋅ ) B(\cdot) B ( ⋅ ) is a standard Brownian motion. In contrast to Jiang et al. [23 ] , who rely on β \beta β -mixing conditions for standard GARCH processes, our functional-dependence derivation requires no external mixing assumptions. Because K K K is Lipschitz continuous and has compact support, summation by parts then gives
1 T h ∑ t = 1 T K ( x − t / T h ) z t → d N ( 0 , Ω z ∫ − 1 1 K 2 ( v ) d v ) , \frac{1}{\sqrt{Th}}\sum_{t=1}^{T}K\left(\frac{x-t/T}{h}\right)z_t \to_d N\left(0,\Omega_z\int_{-1}^{1}K^2(v)\,dv\right), T h 1 t = 1 ∑ T K ( h x − t / T ) z t → d N ( 0 , Ω z ∫ − 1 1 K 2 ( v ) d v ) ,
for every fixed x ∈ ( 0 , 1 ) x\in(0,1) x ∈ ( 0 , 1 ) . Hence neither covariance summability nor a kernel-specific central limit theorem is needed to be imposed separately.
The preceding result, together with the usual second-order bias expansion for the kernel estimator, leads directly to the pointwise asymptotic normality of τ ^ ( x ) \widehat{\tau}(x) τ ( x ) as below.
Theorem 1. Suppose Assumptions 1–4 hold. Then, for each fixed x ∈ ( 0 , 1 ) x\in(0,1) x ∈ ( 0 , 1 ) ,
T h { τ ^ ( x ) − τ ( x ) − h 2 B τ ( x ) } → d N ( 0 , V τ ( x ) ) , \sqrt{Th}\{\widehat{\tau}(x)-\tau(x)-h^2B_\tau(x)\}\to_d N(0,V_\tau(x)), T h { τ ( x ) − τ ( x ) − h 2 B τ ( x )} → d N ( 0 , V τ ( x )) , where B τ ( x ) = C 2 τ ′ ′ ( x ) / 2 B_\tau(x)=C_2\tau^{\prime\prime}(x)/2 B τ ( x ) = C 2 τ ′′ ( x ) /2 and V τ ( x ) = τ 2 ( x ) { ∫ − 1 1 K 2 ( v ) d v } Ω z V_\tau(x)=\tau^2(x)\{\int_{-1}^{1}K^2(v)\,dv\}\Omega_z V τ ( x ) = τ 2 ( x ) { ∫ − 1 1 K 2 ( v ) d v } Ω z , with Ω z = ∑ j = − ∞ ∞ Cov ( z 0 , z j ) \Omega_z=\sum_{j=-\infty}^{\infty}\operatorname{Cov}(z_0,z_j) Ω z = ∑ j = − ∞ ∞ Cov ( z 0 , z j ) .
The proof derives the kernel central limit theorem for { z t } \{z_t\} { z t } under the GMC condition. This parallels Theorem 2.1 of Jiang et al. [23 ] using mixing type conditions.
For t = 1 , … , T t=1,\ldots,T t = 1 , … , T , define τ ^ t = τ ^ ( t / T ) \widehat{\tau}_t=\widehat{\tau}(t/T) τ t = τ ( t / T ) . While the unadjusted estimator in (7) achieves an O ( h 2 ) O(h^2) O ( h 2 ) smoothing bias at interior points t / T ∈ [ h , 1 − h ] t/T\in[h,1-h] t / T ∈ [ h , 1 − h ] , its kernel weights lose a nonvanishing fraction of their mass near the boundaries x = 0 x=0 x = 0 and x = 1 x=1 x = 1 , causing an O ( 1 ) O(1) O ( 1 ) boundary bias. To correct this loss, we adopt the reflection method used by Chen and Hong [6 ] and Jiang et al. [23 ] .
Specifically, let m T = ⌊ T h ⌋ m_T=\lfloor Th\rfloor m T = ⌊ T h ⌋ , define the interior index set I T = { m T + 1 , … , T − m T } \mathcal{I}_T=\{m_T+1,\ldots,T-m_T\} I T = { m T + 1 , … , T − m T } and the boundary index set B T = { 1 , … , T } ∖ I T \mathcal{B}_T=\{1,\ldots,T\}\setminus\mathcal{I}_T B T = { 1 , … , T } ∖ I T , and reflect the squared returns across the endpoints via r 1 − j 2 = r 1 + j 2 r_{1-j}^2=r_{1+j}^2 r 1 − j 2 = r 1 + j 2 and r T + j 2 = r T + 1 − j 2 r_{T+j}^2=r_{T+1-j}^2 r T + j 2 = r T + 1 − j 2 for j = 1 , … , m T j=1,\ldots,m_T j = 1 , … , m T . The boundary-adjusted estimator is then defined as
τ ^ t r e f = 1 T ∑ s = t − m T t + m T K h ( t − s T ) r s 2 , t = 1 , … , T . \widehat{\tau}_t^{\mathrm{ref}}=\frac{1}{T}\sum_{s=t-m_T}^{t+m_T}K_h\left(\frac{t-s}{T}\right)r_s^2,\qquad t=1,\ldots,T. τ t ref = T 1 s = t − m T ∑ t + m T K h ( T t − s ) r s 2 , t = 1 , … , T .
For notational simplicity, τ ^ t \widehat{\tau}_t τ t denotes τ ^ t r e f \widehat{\tau}_t^{\mathrm{ref}} τ t ref throughout the remainder of the paper.
Remark 2. Note that unless τ ′ ( 0 ) = τ ′ ( 1 ) = 0 \tau'(0)=\tau'(1)=0 τ ′ ( 0 ) = τ ′ ( 1 ) = 0 , simple reflection eliminates the O ( 1 ) O(1) O ( 1 ) loss of kernel mass but leaves a first-order O ( h ) O(h) O ( h ) bias within the boundary band B T \mathcal{B}_T B T , while preserving the O ( h 2 ) O(h^2) O ( h 2 ) bias on the interior I T \mathcal{I}_T I T . Importantly, we do not trim the boundary observations when constructing the second-step likelihood: the volatility recursion and log-likelihood are evaluated over the full sample. In our asymptotic analysis, we partition the first-step estimation errors into B T \mathcal{B}_T B T and I T \mathcal{I}_T I T and show that because ∣ B T ∣ = 2 m T = O ( T h ) |\mathcal{B}_T|=2m_T=O(Th) ∣ B T ∣ = 2 m T = O ( T h ) , the cumulative impact of the O ( h ) O(h) O ( h ) boundary errors—including their recursive propagation into subsequent interior scores—is of order O p ( h log T + T h 2 ) = o p ( 1 ) O_p(\sqrt{h}\log T+\sqrt{T}h^2)=o_p(1) O p ( h log T + T h 2 ) = o p ( 1 ) under Assumption 5. Consequently, the O ( h ) O(h) O ( h ) endpoint bias does not affect the first-order asymptotic distribution of the short-run parameter estimator.
Estimation of the Short-Run Parameters
Let θ = ( α , β , φ ) T ∈ Θ \theta= (\alpha, \beta, \varphi)^{\mathsf{T}} \in\Theta θ = ( α , β , φ ) T ∈ Θ denote the finite-dimensional parameter vector. Under Gaussian innovations, the normalization condition in (2) becomes ω ( θ ) = 1 − α − β − 3 φ + 2 β φ \omega(\theta) = 1 - \alpha- \beta- 3\varphi+ 2\beta\varphi ω ( θ ) = 1 − α − β − 3 φ + 2 β φ . Thus, ω \omega ω is not estimated as an additional free parameter.
We first define an infeasible estimator based on the unobservable short-run process { u t } \{u_t\} { u t } . Conditional on F t − 1 \mathcal{F}_{t-1} F t − 1 , its nonlinear scale recursion is given by
b t − 1 ( θ ) = ω ( θ ) + α u t − 1 2 + β h t − 1 ( θ ) , b_{t-1}(\theta) = \omega(\theta) + \alpha u_{t-1}^{2} + \beta h_{t-1}(\theta), b t − 1 ( θ ) = ω ( θ ) + α u t − 1 2 + β h t − 1 ( θ ) ,
h t ( θ ) = b t − 1 ( θ ) + b t − 1 2 ( θ ) + 4 φ u t 2 2 . h_t(\theta) = \frac{b_{t-1}(\theta) + \sqrt{b_{t-1}^{2}(\theta) + 4\varphi u_t^{2}}}{2}. h t ( θ ) = 2 b t − 1 ( θ ) + b t − 1 2 ( θ ) + 4 φ u t 2 .
Under the standard Gaussian innovation, the conditional negative log-likelihood contribution, up to a constant independent of θ \theta θ , is
ℓ t ( θ ) = 1 2 u t 2 h t ( θ ) − 1 2 log h t ( θ ) + log { h t ( θ ) + φ u t 2 h t ( θ ) } . (8) \ell_t(\theta) = \frac{1}{2}\frac{u_t^{2}}{h_t(\theta)} - \frac{1}{2}\log h_t(\theta) + \log\left\{h_t(\theta) + \varphi\frac{u_t^{2}}{h_t(\theta)}\right\}. \tag*{(8)} ℓ t ( θ ) = 2 1 h t ( θ ) u t 2 − 2 1 log h t ( θ ) + log { h t ( θ ) + φ h t ( θ ) u t 2 } . (8)
The corresponding infeasible objective function is L T ( θ ) = ∑ t = 1 T ℓ t ( θ ) L_T(\theta) = \sum_{t=1}^{T} \ell_t(\theta) L T ( θ ) = ∑ t = 1 T ℓ t ( θ ) .
For finite-sample computation, let h ~ t ( θ ) \widetilde{h}_t(\theta) h t ( θ ) denote the initialized infeasible process based on u 1 2 , … , u T 2 u_1^2,\ldots,u_T^2 u 1 2 , … , u T 2 with the initial values h ~ 0 ( θ ) = u 0 2 = 1 \widetilde{h}_0(\theta) = u_0^2 = 1 h 0 ( θ ) = u 0 2 = 1 . Let ℓ ~ t ( θ ) \widetilde{\ell}_t(\theta) ℓ t ( θ ) denote (8) evaluated at ( u t , h ~ t ( θ ) ) (u_t,\widetilde{h}_t(\theta)) ( u t , h t ( θ )) . Moreover, because τ t \tau_t τ t is unobservable, we replace u t u_t u t by u ^ t = r t / τ ^ t \widehat{u}_t = r_t/\sqrt{\widehat{\tau}_t} u t = r t / τ t for t = 1 , … , T t=1,\ldots,T t = 1 , … , T . Using the same initialization convention, we then obtain h ^ t ( θ ) \widehat{h}_t(\theta) h t ( θ ) from the preceding recursion by replacing the squared inputs u t 2 u_t^2 u t 2 with u ^ t 2 \widehat{u}_t^2 u t 2 and the previous state h t − 1 ( θ ) h_{t-1}(\theta) h t − 1 ( θ ) with h ^ t − 1 ( θ ) \widehat{h}_{t-1}(\theta) h t − 1 ( θ ) . Let ℓ ^ t ( θ ) \widehat{\ell}_t(\theta) ℓ t ( θ ) be (8) evaluated at ( u ^ t , h ^ t ( θ ) ) (\widehat{u}_t,\widehat{h}_t(\theta)) ( u t , h t ( θ )) . The feasible Gaussian conditional maximum likelihood estimator is
θ ^ T = arg min θ ∈ Θ L ^ T ( θ ) , L ^ T ( θ ) = ∑ t = 1 T ℓ ^ t ( θ ) . (9) \widehat{\theta}_T = \mathop{\arg\min}_{\theta\in\Theta}\widehat{L}_T(\theta), \qquad\widehat{L}_T(\theta) = \sum_{t=1}^{T}\widehat{\ell}_t(\theta). \tag*{(9)} θ T = arg min θ ∈ Θ L T ( θ ) , L T ( θ ) = t = 1 ∑ T ℓ t ( θ ) . (9)
Assumption 5. The bandwidth satisfies h = c h T − λ h h = c_hT^{-\lambda_h} h = c h T − λ h , where 1 / 4 < λ h < 1 / 2 1/4 < \lambda_h < 1/2 1/4 < λ h < 1/2 and 0 < c h < ∞ 0 < c_h < \infty 0 < c h < ∞ .
Assumption 5 specifies an undersmoothing bandwidth range for the two-step procedure. On the one hand, the lower bound λ h > 1 / 4 \lambda_h > 1/4 λ h > 1/4 implies T h 2 → 0 \sqrt{T}h^2 \to0 T h 2 → 0 , which simultaneously eliminates the O ( h 2 ) O(h^2) O ( h 2 ) interior smoothing bias and the aggregate O ( h ) O(h) O ( h ) boundary bias over the O ( T h ) O(Th) O ( T h ) observations in B T B_T B T . On the other hand, the upper bound λ h < 1 / 2 \lambda_h < 1/2 λ h < 1/2 guarantees T h → ∞ \sqrt{T}h \to\infty T h → ∞ and log T / ( T h ) → 0 \log T/(\sqrt{T}h) \to0 log T / ( T h ) → 0 . Together with T h 2 → 0 \sqrt{T}h^2 \to0 T h 2 → 0 , this can be used to control the second-order Taylor remainder of the feasible score and to ensure that the cross-term in the interior kernel projection vanishes at rate O p ( ( T h ) − 1 ) = o p ( 1 ) O_p((\sqrt{T}h)^{-1}) = o_p(1) O p (( T h ) − 1 ) = o p ( 1 ) .
Assumption 6. For some q > 4 q > 4 q > 4 , E ∣ z t ∣ q < ∞ \mathbb{E}|z_t|^q < \infty E ∣ z t ∣ q < ∞ , where z t = u t 2 − 1 z_t = u_t^2 - 1 z t = u t 2 − 1 .
Assumption 6 provides the moment order needed to establish uniform consistency of τ ^ t \widehat{\tau}_t τ t over 1 ≤ t ≤ T 1 \le t \le T 1 ≤ t ≤ T and to control the higher-order derivatives of the log-likelihood. By Proposition 5 , E ∣ z t ∣ q < ∞ \mathbb{E}|z_t|^q < \infty E ∣ z t ∣ q < ∞ implies that { z t } \{z_t\} { z t } is also L q L^q L q GMC.
Proposition 6 (Uniform first-step rate). Under Assumptions 1–6, the boundary-adjusted first-step estimator satisfies that, with probability approaching one, (i). max 1 ≤ t ≤ T ∣ τ ^ t − τ t ∣ = O p ( κ T ) \max_{1\le t\le T}|\widehat{\tau}_t-\tau_t| = O_p(\kappa_T) max 1 ≤ t ≤ T ∣ τ t − τ t ∣ = O p ( κ T ) , where κ T = log T / T h + h \kappa_T = \sqrt{\log T/Th} + h κ T = log T / T h + h ; (ii). for some c τ > 0 c_\tau> 0 c τ > 0 , inf 1 ≤ t ≤ T τ ^ t ≥ c τ \inf_{1\le t\le T}\widehat{\tau}_t \ge c_\tau inf 1 ≤ t ≤ T τ t ≥ c τ ; and (iii). for the interior block, max t ∈ I T ∣ τ ^ t − τ t ∣ = O p ( κ I , T ) \max_{t\in I_T}|\widehat{\tau}_t-\tau_t| = O_p(\kappa_{I,T}) max t ∈ I T ∣ τ t − τ t ∣ = O p ( κ I , T ) , where κ I , T = log T / T h + h 2 \kappa_{I,T} = \sqrt{\log T/Th} + h^2 κ I , T = log T / T h + h 2 .
The proof of Proposition 6 relies on a maximal inequality for L q L^q L q GMC processes (Liu et al., [27 ] ). The difference between the full-sample rate κ T \kappa_T κ T and the interior rate κ I , T \kappa_{I,T} κ I , T reflects the O ( h ) O(h) O ( h ) boundary bias on B T B_T B T . Equipped with the uniform first-step convergence rate in Proposition 6, we first establish the consistency of θ ^ T \widehat{\theta}_T θ T .
Theorem 2. Suppose the innovations are standard Gaussian and Assumptions 1–6 hold, and α 0 + φ 0 > 0 \alpha_0+\varphi_0>0 α 0 + φ 0 > 0 . Then θ ^ T → p θ 0 \widehat{\theta}_T \to_p \theta_0 θ T → p θ 0 as T → ∞ T \to\infty T → ∞ .
When α 0 + φ 0 > 0 \alpha_0+\varphi_0>0 α 0 + φ 0 > 0 , the short-run dynamic parameters are uniquely identified from the Gaussian conditional likelihood (Smetanina and Wu, [34 ] ). Therefore, by the uniform convergence of the feasible criterion T − 1 L ^ T ( θ ) T^{-1}\widehat{L}_T(\theta) T − 1 L T ( θ ) to the population likelihood E { ℓ t ( θ ) } \mathbb{E}\{\ell_t(\theta)\} E { ℓ t ( θ )} , we thus obtain the consistency from the classical argmax continuous mapping theorem (Newey and McFadden, [29 ] ).
We now turn to the asymptotic distribution of θ ^ T \widehat{\theta}_T θ T . Let s t ( θ ) = ∂ ℓ t ( θ ) / ∂ θ s_t(\theta)=\partial\ell_t(\theta)/\partial\theta s t ( θ ) = ∂ ℓ t ( θ ) / ∂ θ denote the infeasible score and denote the population Hessian matrix as
H = E [ ∂ 2 ℓ t ( θ 0 ) / ( ∂ θ ∂ θ ⊤ ) ] . (10) H=\mathbb{E}[\partial^2\ell_t(\theta_0)/(\partial\theta\,\partial\theta^\top)]. \tag*{(10)} H = E [ ∂ 2 ℓ t ( θ 0 ) / ( ∂ θ ∂ θ ⊤ )] . (10)
Because the feasible score ∂ L ^ T ( θ 0 ) / ∂ θ \partial\widehat{L}_T(\theta_0)/\partial\theta ∂ L T ( θ 0 ) / ∂ θ evaluates the model at the standardized returns u ^ i 2 = u i 2 τ i / τ ^ i \widehat{u}_i^2=u_i^2\tau_i/\widehat{\tau}_i u i 2 = u i 2 τ i / τ i , a perturbation in τ ^ i \widehat{\tau}_i τ i at time i i i affects not only the time-i i i score s i ( θ 0 ) s_i(\theta_0) s i ( θ 0 ) directly, but also every subsequent score s t ( θ 0 ) s_t(\theta_0) s t ( θ 0 ) (t > i t>i t > i ) through the recursive volatility process h t ( θ 0 ) h_t(\theta_0) h t ( θ 0 ) . The key idea of the proof is to show that the cumulative sensitivity of future scores to a relative scale perturbation at time t t t is governed by the mean loading vector
C τ = E { ∑ j = 0 ∞ u t − j 2 ∂ s t ( θ 0 ) ∂ ( u t − j 2 ) } . (11) C_\tau=\mathbb{E}\left\{\sum_{j=0}^{\infty}u_{t-j}^2\frac{\partial s_t(\theta_0)}{\partial(u_{t-j}^2)}\right\}. \tag*{(11)} C τ = E { j = 0 ∑ ∞ u t − j 2 ∂ ( u t − j 2 ) ∂ s t ( θ 0 ) } . (11)
where ∂ s t ( θ 0 ) / ∂ ( u t − j 2 ) \partial s_t(\theta_0)/\partial(u_{t-j}^2) ∂ s t ( θ 0 ) / ∂ ( u t − j 2 ) measures the partial derivative of the time-t t t score with respect to the squared observation u t − j 2 u_{t-j}^2 u t − j 2 , holding all other observations fixed. Projecting the first-step kernel estimation error τ ^ i − τ i \widehat{\tau}_i-\tau_i τ i − τ i onto the score expansion yields the first-order score adjustment
ψ t = − C τ z t . \psi_t=-C_\tau z_t. ψ t = − C τ z t .
so that the effective influence vector at time t t t is
ξ t = s t ( θ 0 ) + ψ t = s t ( θ 0 ) − C τ z t . \xi_t=s_t(\theta_0)+\psi_t=s_t(\theta_0)-C_\tau z_t. ξ t = s t ( θ 0 ) + ψ t = s t ( θ 0 ) − C τ z t .
Theorem 3. Suppose the innovations are standard Gaussian, Assumptions 1–6 hold, θ 0 \theta_0 θ 0 is interior to Θ \Theta Θ , and H H H is nonsingular. Then
T ( θ ^ T − θ 0 ) → d N ( 0 , Σ ) , Σ = H − 1 V H − ⊤ , \sqrt{T}(\widehat{\theta}_T-\theta_0)\to_d N(0,\Sigma),\qquad\Sigma=H^{-1}VH^{-\top}, T ( θ T − θ 0 ) → d N ( 0 , Σ ) , Σ = H − 1 V H − ⊤ , where H H H is defined in (10) , and V = ∑ j = − ∞ ∞ Cov ( ξ t , ξ t − j ) V=\sum_{j=-\infty}^{\infty}\operatorname{Cov}(\xi_t,\xi_{t-j}) V = ∑ j = − ∞ ∞ Cov ( ξ t , ξ t − j ) is the long-run covariance matrix of { ξ t } \{\xi_t\} { ξ t } .
In general two-step semiparametric time-series models, conducting inference based on Σ = H − 1 V H − 1 \Sigma=H^{-1}VH^{-1} Σ = H − 1 V H − 1 is complicated by two practical difficulties. First, although the exact conditional score { s t ( θ 0 ) } \{s_t(\theta_0)\} { s t ( θ 0 )} is a martingale difference sequence with E { s t ( θ 0 ) s t ( θ 0 ) ⊤ } = H \mathbb{E}\{s_t(\theta_0)s_t(\theta_0)^\top\}=H E { s t ( θ 0 ) s t ( θ 0 ) ⊤ } = H , the first-step error sequence { z t } \{z_t\} { z t } is serially correlated across all lags j ≠ 0 j\ne0 j = 0 , which typically induces non-vanishing autocovariances Cov ( z t , z t − j ) \operatorname{Cov}(z_t,z_{t-j}) Cov ( z t , z t − j ) as well as cross-covariances Cov ( s t ( θ 0 ) , z t + j ) \operatorname{Cov}(s_t(\theta_0),z_{t+j}) Cov ( s t ( θ 0 ) , z t + j ) for j > 0 j>0 j > 0 . Estimating V = ∑ j = − ∞ ∞ Cov ( ξ t , ξ t − j ) V = \sum_{j=-\infty}^{\infty} \operatorname{Cov}(\xi_t,\xi_{t-j}) V = ∑ j = − ∞ ∞ Cov ( ξ t , ξ t − j ) directly from sample residuals would therefore require a heteroskedasticity and autocorrelation consistent (HAC) kernel estimator (e.g., Newey and McFadden, [29 ] ) with a user-selected lag-truncation bandwidth, which can suffer from substantial finite-sample distortion when volatility persistence α 0 + β 0 \alpha_0 + \beta_0 α 0 + β 0 is high. Second, the definition of the loading vector C τ C_\tau C τ in (11) involves an infinite series of recursive observation derivatives ∂ s t ( θ 0 ) / ∂ ( u t − j 2 ) \partial s_t(\theta_0)/\partial(u_{t-j}^2) ∂ s t ( θ 0 ) / ∂ ( u t − j 2 ) , which would ordinarily require an additional truncation lag to approximate, see such treatment in Jiang et al. [22 ] in the multivariate semiparametric models.
Under the normalized Gaussian SRT-GARCH specification, however, two structural properties of the model completely eliminate the need for HAC estimation or lag truncation. This mirrors the adaptive inference phenomenon documented for the nested S-GARCH model in Jiang et al. [23 ] , where the two-step estimator is shown to be adaptive to the unknown functional form of τ ( ⋅ ) \tau(\cdot) τ ( ⋅ ) and to share the exact same asymptotic efficiency as the parametric variance-targeting estimator (Francq et al., [15 ] ) when the process is stationary. Specifically, the limiting covariance matrix Σ \Sigma Σ depends only on θ 0 \theta_0 θ 0 and is invariant to τ ( ⋅ ) \tau(\cdot) τ ( ⋅ ) ; moreover, because the kernel projection of τ ^ i − τ i \widehat{\tau}_i - \tau_i τ i − τ i aggregates to the same influence term − C τ T − 1 / 2 ∑ s = 1 T z s -C_\tau T^{-1/2}\sum_{s=1}^{T} z_s − C τ T − 1/2 ∑ s = 1 T z s as the parametric sample-variance error ( τ ‾ T − τ 0 ) / τ 0 = T − 1 ∑ s = 1 T z s (\overline{\tau}_T-\tau_0)/\tau_0 = T^{-1}\sum_{s=1}^{T} z_s ( τ T − τ 0 ) / τ 0 = T − 1 ∑ s = 1 T z s , estimating a time-varying scale τ ( ⋅ ) \tau(\cdot) τ ( ⋅ ) nonparametrically incurs no first-order asymptotic efficiency loss relative to parametric variance targeting.
Nevertheless, establishing this simplification for SRT-GARCH differs fundamentally from the S-GARCH analysis in Jiang et al. [23 ] . When φ 0 > 0 \varphi_0 > 0 φ 0 > 0 , contemporaneous feedback makes u t 2 = b t − 1 ϵ t 2 + φ 0 ϵ t 4 u_t^2 = b_{t-1}\epsilon_t^2 + \varphi_0\epsilon_t^4 u t 2 = b t − 1 ϵ t 2 + φ 0 ϵ t 4 a quadratic polynomial in ϵ t 2 \epsilon_t^2 ϵ t 2 , renders the score s t ( θ 0 ) s_t(\theta_0) s t ( θ 0 ) a nonlinear function of the innovation, and couples the scale of u t 2 u_t^2 u t 2 with both the intercept ω ( θ ) \omega(\theta) ω ( θ ) and the feedback parameter φ \varphi φ inside the square-root recursion h t ( θ ) h_t(\theta) h t ( θ ) . Consequently, neither the score–error orthogonality nor the truncation-free representation of C τ C_\tau C τ follows from standard GARCH models; instead, we establish them through the following two results:
Rank-one covariance decomposition and score–error orthogonality. In Lemma 11 of the Appendix, we show that z t = u t 2 − 1 z_t = u_t^2 - 1 z t = u t 2 − 1 admits the ARMA-type representation
z t = ( α 0 + β 0 ) z t − 1 + ζ t − β 0 ζ t − 1 + β 0 φ 0 ( ϵ t − 1 2 − 1 ) , z_t = (\alpha_0 + \beta_0)z_{t-1} + \zeta_t - \beta_0\zeta_{t-1} + \beta_0\varphi_0(\epsilon_{t-1}^2 - 1), z t = ( α 0 + β 0 ) z t − 1 + ζ t − β 0 ζ t − 1 + β 0 φ 0 ( ϵ t − 1 2 − 1 ) ,
where ζ t = u t 2 − b t − 1 − 3 φ 0 \zeta_t = u_t^2 - b_{t-1} - 3\varphi_0 ζ t = u t 2 − b t − 1 − 3 φ 0 and ϵ t 2 − 1 \epsilon_t^2 - 1 ϵ t 2 − 1 are martingale differences with respect to F t \mathcal{F}_t F t . By a Beveridge–Nelson telescoping decomposition, the partial sum T − 1 / 2 ∑ t = 1 T z t T^{-1/2}\sum_{t=1}^{T} z_t T − 1/2 ∑ t = 1 T z t is L 2 L^2 L 2 -equivalent to the partial sum of the martingale difference sequence
χ t = ( 1 − β 0 ) ζ t + β 0 φ 0 ( ϵ t 2 − 1 ) 1 − α 0 − β 0 . \chi_t = \frac{(1-\beta_0)\zeta_t + \beta_0\varphi_0(\epsilon_t^2 - 1)}{1-\alpha_0-\beta_0}. χ t = 1 − α 0 − β 0 ( 1 − β 0 ) ζ t + β 0 φ 0 ( ϵ t 2 − 1 ) .
Because both s t ( θ 0 ) s_t(\theta_0) s t ( θ 0 ) and χ t \chi_t χ t are martingale difference sequences with respect to F t \mathcal{F}_t F t , all of their serial autocovariances and cross-covariances at nonzero lags (j ≠ 0 j \ne0 j = 0 ) vanish identically. Even more importantly, at lag zero, the conditional score identities implied by the variance normalization ω ( θ ) = 1 − α − β − 3 φ + 2 β φ \omega(\theta) = 1-\alpha-\beta-3\varphi+2\beta\varphi ω ( θ ) = 1 − α − β − 3 φ + 2 β φ ensure exact contemporaneous orthogonality between the second-step score and the first-step martingale innovation:
E { s t ( θ 0 ) χ t } = 0. \mathbb{E}\{s_t(\theta_0)\chi_t\} = 0. E { s t ( θ 0 ) χ t } = 0.
Consequently, the cross-terms between s t ( θ 0 ) s_t(\theta_0) s t ( θ 0 ) and ψ t = − C τ z t \psi_t = -C_\tau z_t ψ t = − C τ z t cancel completely in the long-run covariance matrix V V V , yielding the exact rank-one decomposition
V = H + Ω z C τ C τ T , Σ = H − 1 + Ω z ( H − 1 C τ ) ( H − 1 C τ ) T , (12) V = H + \Omega_z C_\tau C_\tau^{\mathsf{T}}, \qquad\Sigma= H^{-1} + \Omega_z(H^{-1}C_\tau)(H^{-1}C_\tau)^{\mathsf{T}}, \tag*{(12)} V = H + Ω z C τ C τ T , Σ = H − 1 + Ω z ( H − 1 C τ ) ( H − 1 C τ ) T , (12)
where Ω z = ∑ j = − ∞ ∞ Cov ( z 0 , z j ) = E ( χ t 2 ) \Omega_z = \sum_{j=-\infty}^{\infty}\operatorname{Cov}(z_0,z_j) = \mathbb{E}(\chi_t^2) Ω z = ∑ j = − ∞ ∞ Cov ( z 0 , z j ) = E ( χ t 2 ) . Thus, nonparametric estimation of the unknown long-run scale τ ( ⋅ ) \tau(\cdot) τ ( ⋅ ) inflates the oracle asymptotic covariance H − 1 H^{-1} H − 1 by a positive semidefinite matrix of rank at most one.
Lemma 11 further shows that under Gaussian innovations, Ω z = E ( χ t 2 ) \Omega_z = \mathbb{E}(\chi_t^2) Ω z = E ( χ t 2 ) admits the explicit closed-form rational expression in terms of ( α 0 , β 0 , φ 0 ) (\alpha_0,\beta_0,\varphi_0) ( α 0 , β 0 , φ 0 ) :
Ω z = 2 ( 1 − β 0 ) 2 Var ( b t ) + 2 { 1 − β 0 + ( 3 − 2 β 0 ) φ 0 } 2 + 24 ( 1 − β 0 ) 2 φ 0 2 ( 1 − α 0 − β 0 ) 2 , (13) \Omega_z = \frac{2(1-\beta_0)^2\operatorname{Var}(b_t)+2\{1-\beta_0+(3-2\beta_0)\varphi_0\}^2+24(1-\beta_0)^2\varphi_0^2}{(1-\alpha_0-\beta_0)^2}, \tag*{(13)} Ω z = ( 1 − α 0 − β 0 ) 2 2 ( 1 − β 0 ) 2 Var ( b t ) + 2 { 1 − β 0 + ( 3 − 2 β 0 ) φ 0 } 2 + 24 ( 1 − β 0 ) 2 φ 0 2 , (13)
with Var ( b t ) = [ 2 { α 0 + ( 3 α 0 + β 0 ) φ 0 } 2 + 24 α 0 2 φ 0 2 ] / ( 1 − 3 α 0 2 − 2 α 0 β 0 − β 0 2 ) \operatorname{Var}(b_t)=[2\{\alpha_0+(3\alpha_0+\beta_0)\varphi_0\}^2+24\alpha_0^2\varphi_0^2]/(1-3\alpha_0^2-2\alpha_0\beta_0-\beta_0^2) Var ( b t ) = [ 2 { α 0 + ( 3 α 0 + β 0 ) φ 0 } 2 + 24 α 0 2 φ 0 2 ] / ( 1 − 3 α 0 2 − 2 α 0 β 0 − β 0 2 ) . Moreover, Lemma 10 of the Appendix re-expresses the infinite series C τ C_{\tau} C τ in (11) as a single expectation:
C τ = − E [ ∂ ℓ t ( θ 0 ) ∂ θ { φ 0 ∂ ℓ t ( θ 0 ) ∂ φ + ( 1 − α 0 − β 0 ) ∂ ℓ t ( θ 0 ) ∂ ω } ] , (14) C_{\tau}=-\mathbb{E}\left[\frac{\partial\ell_t(\theta_0)}{\partial\theta}\left\{\varphi_0\frac{\partial\ell_t(\theta_0)}{\partial\varphi}+(1-\alpha_0-\beta_0)\frac{\partial\ell_t(\theta_0)}{\partial\omega}\right\}\right], \tag*{(14)} C τ = − E [ ∂ θ ∂ ℓ t ( θ 0 ) { φ 0 ∂ φ ∂ ℓ t ( θ 0 ) + ( 1 − α 0 − β 0 ) ∂ ω ∂ ℓ t ( θ 0 ) } ] , (14)
where ∂ ℓ t ( θ ) / ∂ ω \partial\ell_t(\theta)/\partial\omega ∂ ℓ t ( θ ) / ∂ ω denotes the partial derivative of ℓ t ( θ ) \ell_t(\theta) ℓ t ( θ ) with respect to an unnormalized intercept ω \omega ω (holding ( α , β , φ ) (\alpha,\beta,\varphi) ( α , β , φ ) and the observations fixed) and is computed via one scalar recursion; see (A.74)–(A.75) in the Appendix.
Together, (12)–(14) yield a plug-in covariance estimator requiring neither lag truncation nor additional tuning parameters
Σ ^ = H ^ − 1 + Ω ^ z ( H ^ − 1 C ^ τ ) ( H ^ − 1 C ^ τ ) T , H ^ = 1 T ∑ t = 1 T ∂ 2 ℓ t ( θ ^ T ) ∂ θ ∂ θ T , \widehat{\Sigma}=\widehat{H}^{-1}+\widehat{\Omega}_z(\widehat{H}^{-1}\widehat{C}_{\tau})(\widehat{H}^{-1}\widehat{C}_{\tau})^{\mathsf{T}},\qquad\widehat{H}=\frac{1}{T}\sum_{t=1}^{T}\frac{\partial^2\ell_t(\widehat{\theta}_T)}{\partial\theta\,\partial\theta^{\mathsf{T}}}, Σ = H − 1 + Ω z ( H − 1 C τ ) ( H − 1 C τ ) T , H = T 1 t = 1 ∑ T ∂ θ ∂ θ T ∂ 2 ℓ t ( θ T ) ,
where Ω ^ z \widehat{\Omega}_z Ω z is obtained by substituting θ ^ T \widehat{\theta}_T θ T into (13) and C ^ τ \widehat{C}_{\tau} C τ is the sample average counterpart of (14) evaluated at θ ^ T \widehat{\theta}_T θ T .
Volatility Forecasting
Having established the estimation and asymptotic properties of the model, we now turn to volatility forecasting. The SRT-GARCH specification naturally separates the forecast into two components: a short-run conditional-moment recursion and a forecast of the smooth long-run volatility component.
For forecasts beyond the observed sample, suppose that the long-run component τ ( ⋅ ) \tau(\cdot) τ ( ⋅ ) admits a positive deterministic continuation over the required forecast horizon. Conditional on this continuation, symmetry implies
E ( r t ∣ F t − 1 ) = 0 , \mathbb{E}(r_t\mid\mathcal{F}_{t-1})=0, E ( r t ∣ F t − 1 ) = 0 ,
so forecasting the conditional variance is equivalent to forecasting
E ( r T + k 2 ∣ F T ) = τ T + k E ( u T + k 2 ∣ F T ) . \mathbb{E}(r_{T+k}^2\mid\mathcal{F}_T)=\tau_{T+k}\mathbb{E}(u_{T+k}^2\mid\mathcal{F}_T). E ( r T + k 2 ∣ F T ) = τ T + k E ( u T + k 2 ∣ F T ) .
The short-run expectation is determined by the model recursion, whereas forecasting τ T + k \tau_{T+k} τ T + k requires a separate extrapolation of the long-run component.
Under Gaussian innovations and the normalization (2) , the feasible one-step-ahead volatility forecast is
E ^ ( r T + 1 2 ∣ F T ) = τ ^ T + 1 ∣ T [ 1 − α ^ − β ^ + 2 β ^ φ ^ + α ^ u T 2 + β ^ h T ] . \widehat{\mathbb{E}}(r_{T+1}^2\mid\mathcal{F}_T)=\widehat{\tau}_{T+1\mid T}\left[1-\widehat{\alpha}-\widehat{\beta}+2\widehat{\beta}\widehat{\varphi}+\widehat{\alpha}u_T^2+\widehat{\beta}h_T\right]. E ( r T + 1 2 ∣ F T ) = τ T + 1 ∣ T [ 1 − α − β + 2 β φ + α u T 2 + β h T ] .
where τ ^ T + 1 ∣ T \widehat{\tau}_{T+1\mid T} τ T + 1 ∣ T denotes the forecast of the long-run volatility component. The corresponding population recursion and its multi-step extension are derived in Appendix A.§ 2 .
In the empirical rolling-window forecasts, the long-run component is extrapolated using
τ ^ s + k ∣ s = max { ( k + 1 ) τ ^ s − k τ ^ s − 1 , c τ } . \widehat{\tau}_{s+k\mid s}=\max\{(k+1)\widehat{\tau}_s-k\widehat{\tau}_{s-1},c_{\tau}\}. τ s + k ∣ s = max {( k + 1 ) τ s − k τ s − 1 , c τ } .
where c τ > 0 c_{\tau}>0 c τ > 0 is a small numerical floor that preserves positivity. This two-point extrapolation is an empirical forecasting convention rather than a consequence of the population moment recursion.
Simulations
We use pilot-based cross-validation to select the smoothing bandwidth and examine the finite-sample behavior of the two-step estimator under the three DGPs described in § 2.4 . They represent, respectively, periodic variation, a temporary rise followed by a decline, and a monotone nonlinear increase in the unconditional variance. For each DGP, we consider sample sizes T ∈ { 1000 , 2000 } T \in\{1000,2000\} T ∈ { 1000 , 2000 } . The innovations are independently generated from N ( 0 , 1 ) \mathcal{N}(0,1) N ( 0 , 1 ) . The intercept ω 0 \omega_0 ω 0 is determined by the Gaussian normalization ω 0 = 1 − α 0 − β 0 − 3 φ 0 + 2 β 0 φ 0 \omega_0 = 1-\alpha_0-\beta_0-3\varphi_0+2\beta_0\varphi_0 ω 0 = 1 − α 0 − β 0 − 3 φ 0 + 2 β 0 φ 0 . The number of Monte Carlo replications is M = 1000 M=1000 M = 1000 . For each replication, we first generate a stationary short-run RT-GARCH process after discarding the first 500 observations as burn-in, and then multiply the short-run process by τ ( t / T ) \sqrt{\tau(t/T)} τ ( t / T ) to obtain the observed return series { r t } \{r_t\} { r t } .
Bandwidth Selection
For both the simulations and empirical applications, we follow the pilot-based cross-validation approach of Jiang et al. [23 ] . The bandwidth for estimating τ ( ⋅ ) \tau(\cdot) τ ( ⋅ ) is selected as follows.
Algorithm 1 (Cross-validation bandwidth selection).
Choose a pilot bandwidth h 0 = T − λ 0 h_0=T^{-\lambda_0} h 0 = T − λ 0 with λ 0 ∈ ( 1 / 4 , 1 / 2 ) \lambda_0\in(1/4,1/2) λ 0 ∈ ( 1/4 , 1/2 ) . Using the kernel estimator in (7) , compute the pilot estimates τ ^ t , 0 = τ ^ 0 ( t / T ) \widehat{\tau}_{t,0}=\widehat{\tau}_0(t/T) τ t , 0 = τ 0 ( t / T ) and u ^ t , 0 = r t / τ ^ t , 0 \widehat{u}_{t,0}=r_t/\sqrt{\widehat{\tau}_{t,0}} u t , 0 = r t / τ t , 0 .
Estimate the short-run SRT-GARCH model from { u ^ t , 0 } t = 1 T \{\widehat{u}_{t,0}\}_{t=1}^{T} { u t , 0 } t = 1 T by Gaussian conditional maximum likelihood, and obtain the pilot scale estimates { h ^ t , 0 } t = 1 T \{\widehat{h}_{t,0}\}_{t=1}^{T} { h t , 0 } t = 1 T .
For each candidate bandwidth h ∈ H T h\in\mathcal{H}_T h ∈ H T , compute the leave-one-out kernel estimator
τ ^ − t , h = 1 T ∑ s = 1 s ≠ t T K h ( t − s T ) r s 2 . \widehat{\tau}_{-t,h}=\frac{1}{T}\sum_{\substack{s=1\\s\ne t}}^{T}K_h\left(\frac{t-s}{T}\right)r_s^2. τ − t , h = T 1 s = 1 s = t ∑ T K h ( T t − s ) r s 2 . Define the cross-validation criterion
C V ( h ) = 1 T ∑ t = 1 T ( r t 2 τ ^ − t , h h ^ t , 0 − 1 ) 2 . (15) CV(h)=\frac{1}{T}\sum_{t=1}^{T}\left(\frac{r_t^2}{\widehat{\tau}_{-t,h}\widehat{h}_{t,0}}-1\right)^2. \tag*{(15)} C V ( h ) = T 1 t = 1 ∑ T ( τ − t , h h t , 0 r t 2 − 1 ) 2 . (15) Select h ^ C V = arg min h ∈ H T C V ( h ) \widehat{h}_{CV}=\mathop{\operatorname{arg\,min}}_{h\in\mathcal{H}_T}CV(h) h C V = arg min h ∈ H T C V ( h ) , where H T = [ c min T − λ 0 , c max T − λ 0 ] \mathcal{H}_T=[c_{\min}T^{-\lambda_0},c_{\max}T^{-\lambda_0}] H T = [ c m i n T − λ 0 , c m a x T − λ 0 ] .
Finite-Sample Performance of the Parameter Estimator
For the m m m th replication, we select the bandwidth using Algorithm 1, estimate τ ^ t ( m ) \widehat{\tau}_t^{(m)} τ t ( m ) , construct u ^ t ( m ) = r t ( m ) / τ ^ t ( m ) \widehat{u}_t^{(m)}=r_t^{(m)}/\sqrt{\widehat{\tau}_t^{(m)}} u t ( m ) = r t ( m ) / τ t ( m ) , and obtain the feasible estimator θ ^ ( m ) \widehat{\theta}^{(m)} θ ( m ) by Gaussian conditional maximum likelihood in (9) . For each component of $\theta=(\alpha,\beta,\varphi)^\mathsf{T}, {xref}Table 1` reports the empirical bias and empirical standard deviation (ESD) of the parameter estimates across the M M M Monte Carlo replications, together with the average estimated asymptotic standard deviation (ASD) and the empirical coverage probability of the corresponding 95% confidence intervals (CP0.95 _{0.95} 0.95 ).
Table 1 shows a common pattern across the three designs. The absolute bias and empirical dispersion generally decline as the sample size T T T increases, broadly consistent with root-T T T convergence. The empirical coverage probabilities are generally close to the nominal 95% level, supporting the finite-sample accuracy of the proposed asymptotic inference procedure.
Table 1. Finite-sample performance of the parameter estimates.
Figure 2 provides a complementary distributional check for the parameter estimates under DGP 3. The QQ plots move closer to the reference line as the sample size increases, with the closest agreement observed at T = 2000 T = 2000 T = 2000 . Similar patterns are observed for DGPs 1 and 2, and are therefore omitted for brevity. Together with Table 1 , these results confirm that the asymptotic normal approximation provides reliable finite-sample inference.
Figure 2. QQ plots of parameter estimates under DGP 3.
Applications
The empirical analysis considers two financial return series: the FTSE 100 index and U.S. natural gas futures. For each dataset, we first test covariance stationarity and examine the fitted long-run and short-run volatility components. We then compare residual diagnostics, out-of-sample volatility forecasts, and Value-at-Risk (VaR) forecasts against the nested RT-GARCH and S-GARCH models.
For reference, we also report randomly weighted bootstrap standard deviation (BSD) alongside the asymptotic standard errors implied by Theorem 3 as a robustness check. The implementation details are provided in Appendix A.§ 3 .
To assess the forecasting performance of the proposed model, we consider both volatility and tail-risk forecasts. While the multi-step volatility forecasts follow directly from the forecasting formulas developed above, the monotone observation equation provides a convenient way to construct one-step-ahead VaR forecasts. Let q p q_p q p denote the p p p th quantile of the innovation distribution. Under the Gaussian specification, q p = Φ − 1 ( p ) q_p = \Phi^{-1}(p) q p = Φ − 1 ( p ) . Since r t + 1 r_{t+1} r t + 1 is a strictly increasing function of ϵ t + 1 \epsilon_{t+1} ϵ t + 1 conditional on F t \mathcal{F}_t F t , its conditional p p p th quantile is obtained by evaluating the observation equation at ϵ t + 1 = q p \epsilon_{t+1} = q_p ϵ t + 1 = q p . Under the positive-loss convention, the resulting one-step-ahead SRT-GARCH VaR is
VaR t + 1 ∣ t ( p ) = − τ t + 1 q p ω + α u t 2 + β h t + φ q p 2 . \operatorname{VaR}_{t+1\mid t}(p) = -\sqrt{\tau_{t+1}}q_p\sqrt{\omega+ \alpha u_t^2 + \beta h_t + \varphi q_p^2}. VaR t + 1 ∣ t ( p ) = − τ t + 1 q p ω + α u t 2 + β h t + φ q p 2 .
In implementation, τ t + 1 \tau_{t+1} τ t + 1 is replaced by its forecast τ ^ t + 1 ∣ t \widehat{\tau}_{t+1\mid t} τ t + 1 ∣ t , and the unknown parameters and state variables are replaced by their estimated counterparts. The same procedure is applied to the benchmark models. In particular, the RT-GARCH model imposes a constant long-run component, whereas the S-GARCH model is obtained by setting φ = 0 \varphi= 0 φ = 0 . The empirical analysis below therefore compares the three models in terms of both volatility forecasting accuracy and one-step-ahead VaR performance.
FTSE 100 Index
The first application uses T = 2700 T = 2700 T = 2700 daily FTSE 100 returns from January 2, 2004, to September 10, 2014. The covariance-stationarity test of Xiao and Lima [41 ] , implemented with the Bartlett kernel, produces C T = 2.8374 C_T = 2.8374 C T = 2.8374 and rejects stationarity at the 1% level. This provides evidence consistent with a time-varying long-run volatility component and motivates the semiparametric specification before comparing the alternative short-run dynamics.
The parameter estimates in Table 2 are based on the first 2050 observations, covering January 2, 2004, through February 13, 2012. Algorithm 1 selects the same bandwidth, h ^ C V = 0.0981 \hat{h}_{\mathrm{CV}} = 0.0981 h ^ CV = 0.0981 , for both SRT-GARCH and S-GARCH. Table 2 reports the SRT-GARCH parameter estimates together with their asymptotic standard errors. Randomly weighted bootstrap standard deviations based on 500 replications are also reported as a finite-sample robustness check.
Table 2. SRT-GARCH(1,1) parameter estimates for FTSE 100 returns.
The large estimate of β \beta β indicates persistent short-run volatility dynamics. The positive estimate φ ^ = 0.0262 \hat{\varphi} = 0.0262 φ ^ = 0.0262 is consistent with contemporaneous volatility feedback. Relative to its estimated uncertainty, the conventional t t t -statistic based on the asymptotic and bootstrap standard errors are 2.7579 and 2.9438, respectively, providing clear evidence of real-time return feedback. Although these statistics do not account for the boundary restriction φ ≥ 0 \varphi\ge0 φ ≥ 0 , they still indicate that the estimated contemporaneous feedback effect is substantial.
Figure 3 illustrates the resulting volatility decomposition. Panel (b) shows gradual changes in the estimated long-run scale. The SRT-GARCH and S-GARCH long-run estimates coincide because their cross-validation procedures select the same bandwidth and both models consequently use the same first-step kernel estimate of τ t \tau_t τ t . Panel (c) compares the fitted conditional standard deviations [ E ^ ( r t 2 ∣ F t − 1 ) ] 1 / 2 [\widehat{E}(r_t^2 \mid\mathcal{F}_{t-1})]^{1/2} [ E ( r t 2 ∣ F t − 1 ) ] 1/2 across the three models.
Figure 3. FTSE 100 returns and estimated volatility components over the period from January 2, 2004, to February 13, 2012.
The residual diagnostics provide further evidence on the adequacy of the alternative short-run specifications. Figure 4 places the SRT-GARCH residuals closer to the normal reference line. The corresponding Kolmogorov–Smirnov normality tests yield p p p -values of 0.0550, 0.0340, and 0.0143 for SRT-GARCH, RT-GARCH, and S-GARCH, respectively. At the 5% significance level, the standard-normal innovation specification is not rejected for SRT-GARCH, whereas it is rejected for both RT-GARCH and S-GARCH.
Figure 4. QQ plots of FTSE 100 standardized residuals.
We next evaluate out-of-sample volatility forecasting performance over the final N = 650 N = 650 N = 650 trading days of the sample using a rolling window of fixed length W = T − N W = T - N W = T − N . In particular, at each forecast date s s s , all three models are re-estimated using the most recent W W W observations { r s − W + 1 , … , r s } \{r_{s-W+1},\ldots,r_s\} { r s − W + 1 , … , r s } , and k k k -step-ahead conditional variance forecasts v ^ s + k ∣ s = E ^ ( r s + k 2 ∣ F s ) \hat{v}_{s+k\mid s} = \widehat{E}(r_{s+k}^2 \mid\mathcal{F}_s) v ^ s + k ∣ s = E ( r s + k 2 ∣ F s ) are generated for horizons k ∈ { 1 , 5 , 10 , 15 } k \in\{1, 5, 10, 15\} k ∈ { 1 , 5 , 10 , 15 } .
Following Patton [30 ] , forecast accuracy is evaluated using the mean squared error (MSE) and QLIKE loss. For a forecast horizon k k k , the average losses are defined as
MSE ( k ) = 1 N − k + 1 ∑ s = T − N T − k [ r s + k 2 − v ^ s + k ∣ s ] 2 , \operatorname{MSE}(k) = \frac{1}{N-k+1}\sum_{s=T-N}^{T-k}\left[r_{s+k}^{2}-\widehat{v}_{s+k|s}\right]^{2}, MSE ( k ) = N − k + 1 1 s = T − N ∑ T − k [ r s + k 2 − v s + k ∣ s ] 2 ,
QLIKE ( k ) = 1 N − k + 1 ∑ s = T − N T − k [ r s + k 2 v ^ s + k ∣ s + log ( v ^ s + k ∣ s ) ] . \operatorname{QLIKE}(k) = \frac{1}{N-k+1}\sum_{s=T-N}^{T-k}\left[\frac{r_{s+k}^{2}}{\widehat{v}_{s+k|s}}+\log\left(\widehat{v}_{s+k|s}\right)\right]. QLIKE ( k ) = N − k + 1 1 s = T − N ∑ T − k [ v s + k ∣ s r s + k 2 + log ( v s + k ∣ s ) ] .
Lower values of MSE and QLIKE indicate more accurate volatility forecasts.
To further assess whether differences in average losses are statistically distinguishable, we use the Diebold–Mariano test with the Harvey–Leybourne–Newbold finite-sample correction (DM–HLN) [10 , 20 ] . The null hypothesis is equal unconditional predictive accuracy. All reported tests are two-sided and use Bartlett–Newey–West estimates of the long-run variance.
Table 3 reports the out-of-sample volatility forecasting performance. SRT-GARCH yields the smallest MSE at all four forecast horizons. The SRT-GARCH model achieves the lowest MSE across all four forecast horizons (0.9492, 0.9689, 0.9855, and 0.9866 at k = 1 , 5 , 10 , 15 k = 1,5,10,15 k = 1 , 5 , 10 , 15 ) and the lowest QLIKE at the 5-, 10-, and 15-step horizons (0.3601, 0.3717, and 0.3696). Relative to the stationary RT-GARCH model, SRT-GARCH delivers uniformly lower losses at every horizon, and the gains grow increasingly pronounced when the forecast horizon is large. More specifically, the DM–HLN test rejects equal predictive accuracy in favor of SRT-GARCH at the 5% level for 10- and 15-step MSE and for 5-, 10-, and 15-step QLIKE (and at the 10% level for 1-step QLIKE). This pattern demonstrates the substantial improvement of accounting for smooth changes in the unconditional variance when forming multi-step volatility forecasts. Relative to the S-GARCH model, which shares the nonparametric long-run component, SRT-GARCH still yields slightly lower average losses in most cases, although the pairwise differences between the two semiparametric specifications are not statistically significant.
Table 3. Out-of-sample volatility forecasting performance for FTSE 100 returns.
Notes: Lower values indicate better forecasting performance, and bold entries denote the smallest loss at each horizon. Values in parentheses are DM–HLN p p p -values, denoted by p D M p_{\mathrm{DM}} p DM , for testing equal unconditional predictive accuracy between SRT-GARCH and the model in the corresponding row. An asterisk indicates p D M < 0.10 p_{\mathrm{DM}} < 0.10 p DM < 0.10 .
We next evaluate one-step-ahead Value-at-Risk forecasts at the left-tail probability level p p p by the conditional coverage test of Christoffersen [7 ] . Let I t ( p ) = I { r t < − V a R ^ t ∣ t − 1 ( p ) } I_t(p) = \mathbb{I}\{r_t < -\widehat{\mathrm{VaR}}_{t|t-1}(p)\} I t ( p ) = I { r t < − VaR t ∣ t − 1 ( p )} denote the VaR violation indicator, where VaR is reported as a positive loss. Under the null hypothesis, the sequence { I t ( p ) } \{I_t(p)\} { I t ( p )} is independently and identically distributed as Bernoulli(p p p ). Thus, the test jointly examines whether the empirical violation frequency is correctly calibrated and whether VaR violations are serially independent.
We also compare the VaR forecasts using the quantile loss function considered by Francq and Zakoian [17 ] . For an out-of-sample evaluation period containing N N N observations, the average loss is defined as
QLoss ( p ) = 1 N ∑ t = T − N + 1 T [ r t + V a R ^ t ∣ t − 1 ( p ) ] [ p − I { r t < − V a R ^ t ∣ t − 1 ( p ) } ] . \operatorname{QLoss}(p) = \frac{1}{N}\sum_{t=T-N+1}^{T}\left[r_t+\widehat{\mathrm{VaR}}_{t|t-1}(p)\right]\left[p-\mathbb{I}\left\{r_t<-\widehat{\mathrm{VaR}}_{t|t-1}(p)\right\}\right]. QLoss ( p ) = N 1 t = T − N + 1 ∑ T [ r t + VaR t ∣ t − 1 ( p ) ] [ p − I { r t < − VaR t ∣ t − 1 ( p ) } ] .
Lower values of QLoss ( p ) \operatorname{QLoss}(p) QLoss ( p ) indicate more accurate VaR forecasts.
Table 4 reports the 1%, 5%, and 10% tail probabilities together with the corresponding quantile losses. For all three models and at all three tail probabilities, the conditional coverage null is not rejected at conventional significance levels. SRT-GARCH achieves the smallest quantile loss at each tail probability. The differences in quantile loss are modest, but the results show that the SRT-GARCH VaR forecasts are at least as well calibrated as those of the benchmark models while delivering slightly lower tail-loss values across all three probability levels.
Table 4. Evaluation of one-step-ahead VaR forecasts for FTSE 100 returns.
U.S. Natural Gas Futures
Natural gas futures provide a second application in a market exposed to changes in weather, inventories, and supply–demand fundamentals. We use T = 3000 T = 3000 T = 3000 daily returns from May 14, 2013, to October 4, 2024. The Xiao and Lima [41 ] statistic is C T = 4.7216 C_T = 4.7216 C T = 4.7216 , again rejecting covariance stationarity at the 1% level and motivating a time-varying long-run volatility component and the semiparametric specification.
Before turning to the forecasting exercise, we estimate the models using the first 2350 observations from May 14, 2013, to April 21, 2022. Algorithm 1 selects bandwidths of 0.0843 and 0.0734 for SRT-GARCH and S-GARCH, respectively, and Table 5 reports the estimation results. The estimate φ ^ = 0.0345 \hat{\varphi} = 0.0345 φ ^ = 0.0345 is positive and consistent with contemporaneous volatility feedback. The conventional t t t -statistics based on the asymptotic and bootstrap standard errors are 4.0588 and 4.3125, respectively.
Table 5. SRT-GARCH(1,1) parameter estimates for U.S. Natural Gas Futures.
Figure 5 plots the return series and the estimated volatility components. In Panel (b), the nonparametric scale τ ^ t \sqrt{\hat{\tau}_t} τ ^ t captures the sharp structural rise in energy market volatility around 2021–2022. In Panel (c), the fitted conditional standard deviations diverge markedly during extreme return episodes: the SRT-GARCH specification responds immediately to large contemporaneous shocks while adapting smoothly to the elevated long-run volatility regime.
Figure 5. U.S. natural gas futures returns and estimated volatility components over the period from May 14, 2013, to April 21, 2022.
The residual diagnostics provide further evidence on the adequacy of the alternative short-run specifications. Figure 6 shows that the SRT-GARCH standardized residuals exhibit substantially closer agreement with the normal reference line than those of the benchmark models.
Figure 6. Normal QQ plots of U.S. natural gas futures standardized residuals.
Using the same rolling-window design as before with a fixed estimation-window length of W = 2350 W = 2350 W = 2350 and an out-of-sample evaluation period of N = 650 N = 650 N = 650 , we next compare multi-step-ahead volatility forecasts across horizons k ∈ { 1 , 5 , 10 , 15 } k \in\{1, 5, 10, 15\} k ∈ { 1 , 5 , 10 , 15 } using MSE and QLIKE loss.
Table 6 shows that SRT-GARCH yields the smallest sample-average MSE and QLIKE loss at every forecast horizon. The pairwise DM–HLN tests confirm that SRT-GARCH significantly improves upon RT-GARCH under QLIKE at the 5% level for k = 1 k = 1 k = 1 and k = 10 k = 10 k = 10 (p D M = 0.0296 p_{\mathrm{DM}} = 0.0296 p DM = 0.0296 and 0.0415) and at the 10% level for k = 5 k = 5 k = 5 and k = 15 k = 15 k = 15 (p D M = 0.0575 p_{\mathrm{DM}} = 0.0575 p DM = 0.0575 and 0.0641). At the same time, SRT-GARCH improves over S-GARCH under MSE at the 5% level for k = 1 k = 1 k = 1 (p D M = 0.0234 p_{\mathrm{DM}} = 0.0234 p DM = 0.0234 ) and at the 10% level for k = 5 k = 5 k = 5 and k = 15 k = 15 k = 15 (p D M = 0.0595 p_{\mathrm{DM}} = 0.0595 p DM = 0.0595 and 0.0735). Thus, in this example, both the nonparametric long-run scale and the real-time short-run feedback contribute meaningfully to out-of-sample forecast accuracy.
Table 6. Out-of-sample volatility forecast losses for U.S. natural gas futures.
Notes: Lower values indicate better forecasting performance, and bold entries denote the smallest loss at each horizon. Values in parentheses are DM–HLN p p p -values, denoted by p D M p_{\mathrm{DM}} p DM , for testing equal unconditional predictive accuracy between SRT-GARCH and the model in the corresponding row. An asterisk indicates p D M < 0.10 p_{\mathrm{DM}} < 0.10 p DM < 0.10 .
Finally, we evaluate one-step-ahead VaR forecasts at the p = 1 % p = 1\% p = 1% , 5 % 5\% 5% , and 10 % 10\% 10% left-tail probability levels. Table 7 reports the p p p -values from the Christoffersen conditional coverage test together with the corresponding quantile losses.
Table 7. Evaluation of one-step-ahead VaR forecasts for U.S. natural gas futures.
As reported in Table 7 , none of the three specifications is rejected by the Christoffersen conditional coverage test at any of the three tail probability levels, with SRT-GARCH exhibiting the highest p p p -value (0.9091) at the extreme 1% tail. In terms of quantile loss, all three models perform nearly identically at p = 1 % p = 1\% p = 1% (0.0666–0.0670), while SRT-GARCH achieves the lowest quantile losses at both the 5% and 10% levels (0.2138 and 0.3446).
Notes: p C C p_{\mathrm{CC}} p CC denotes the p p p -value of the Christoffersen conditional coverage test. Bold entries indicate the smallest quantile loss at each tail probability.
Conclusion
This paper develops a semiparametric real-time GARCH model that combines a smooth deterministic long-run volatility component with stationary short-run dynamics that respond to the contemporaneous innovation. The model nests both the semiparametric GARCH and real-time GARCH specifications as special cases.
We propose a two-step estimation procedure in which the long-run component is estimated nonparametrically by kernel smoothing and the short-run parameters are subsequently estimated by conditional Gaussian maximum likelihood. Under the stated conditions, we establish consistency and asymptotic normality of the resulting estimator. A key feature of the asymptotic theory is that estimation of the long-run component has a non-vanishing first-order effect on the short-run parameter estimates. By combining the kernel projection with a martingale approximation, we show that this contribution enters the asymptotic covariance matrix through a positive semidefinite rank-one adjustment to the oracle covariance. Under Gaussian innovations, the resulting covariance matrix admits a tuning-free plug-in estimator that requires neither HAC estimation nor lag-truncation parameters. The dependence conditions needed for these results are derived from model-specific geometric moment contraction rather than imposed through generic mixing assumptions.
Monte Carlo experiments show finite-sample behavior broadly consistent with the asymptotic theory. The empirical applications to FTSE 100 returns and U.S. natural gas futures indicate that allowing for a time-varying long-run component improves substantially on the RT-GARCH benchmark, while the additional real-time feedback mechanism provides more pronounced forecasting gains in the natural gas market than in the FTSE application. The VaR results are also broadly supportive of the proposed specification.
Two extensions are particularly worth pursuing. First, it would be useful to extend the boundary inference developed for the parametric GARCH/RT-GARCH model to the present two-step semiparametric setting, where nonparametric estimation of the long-run component contributes to the first-order distribution of the short-run parameter estimator. Second, extending the exact-likelihood analysis beyond Gaussian innovations would broaden the applicability of the model and allow more flexible treatment of heavy-tailed and asymmetric returns.
Data and Code Availability
The data and code required to reproduce the simulation and empirical results will be made available to the editors for replicability review. Subject to any licensing restrictions imposed by the data providers, the replication files will be deposited in a public repository upon acceptance.
Proofs
Throughout the proofs, let β ˉ = sup θ ∈ Θ β < 1 \bar{\beta}=\sup_{\theta\in\Theta}\beta<1 β ˉ = sup θ ∈ Θ β < 1 and ω ‾ = inf θ ∈ Θ ω ( θ ) > 0 \underline{\omega}=\inf_{\theta\in\Theta}\omega(\theta)>0 ω = inf θ ∈ Θ ω ( θ ) > 0 , which follow from Assumption 1. The constant C C C below may change from line to line and does not depend on t t t , T T T , or θ ∈ Θ \theta\in\Theta θ ∈ Θ .
Write
b t − 1 ( θ ) = ω ( θ ) + α u t − 1 2 + β h t − 1 ( θ ) , b_{t-1}(\theta)=\omega(\theta)+\alpha u_{t-1}^{2}+\beta h_{t-1}(\theta), b t − 1 ( θ ) = ω ( θ ) + α u t − 1 2 + β h t − 1 ( θ ) ,
and define, for b > 0 b>0 b > 0 , v ≥ 0 v\geq0 v ≥ 0 , and φ ≥ 0 \varphi\geq0 φ ≥ 0 ,
g ( b , v ; φ ) : = b + b 2 + 4 φ v 2 , f ( v , h , φ ) : = v 2 h + log ( h 2 + φ v ) − 3 2 log h . (16) g(b,v;\varphi):=\frac{b+\sqrt{b^{2}+4\varphi v}}{2},\qquad f(v,h,\varphi):=\frac{v}{2h}+\log(h^{2}+\varphi v)-\frac{3}{2}\log h. \tag*{(16)} g ( b , v ; φ ) := 2 b + b 2 + 4 φ v , f ( v , h , φ ) := 2 h v + log ( h 2 + φ v ) − 2 3 log h . (16)
Thus, h t ( θ ) = g ( b t − 1 ( θ ) , u t 2 ; φ ) h_t(\theta)=g(b_{t-1}(\theta),u_t^{2};\varphi) h t ( θ ) = g ( b t − 1 ( θ ) , u t 2 ; φ ) and ℓ t ( θ ) = f ( u t 2 , h t ( θ ) , φ ) \ell_t(\theta)=f(u_t^{2},h_t(\theta),\varphi) ℓ t ( θ ) = f ( u t 2 , h t ( θ ) , φ ) . Throughout the proof, we will use the fact that
∂ g ( b , v ; φ ) ∂ b = 1 2 ( 1 + b b 2 + 4 φ v ) ∈ [ 1 / 2 , 1 ] , (17) \frac{\partial g(b,v;\varphi)}{\partial b}=\frac{1}{2}\left(1+\frac{b}{\sqrt{b^{2}+4\varphi v}}\right)\in[1/2,1], \tag*{(17)} ∂ b ∂ g ( b , v ; φ ) = 2 1 ( 1 + b 2 + 4 φ v b ) ∈ [ 1/2 , 1 ] , (17)
and, whenever b 1 , b 2 ≥ ω > 0 b_1,b_2\geq\omega>0 b 1 , b 2 ≥ ω > 0 and v 1 , v 2 ≥ 0 v_1,v_2\geq0 v 1 , v 2 ≥ 0 ,
∣ g ( b 1 , v 1 ; φ ) − g ( b 2 , v 2 ; φ ) ∣ ≤ ∣ b 1 − b 2 ∣ + φ ω ∣ v 1 − v 2 ∣ . (18) \left|g(b_1,v_1;\varphi)-g(b_2,v_2;\varphi)\right|\leq|b_1-b_2|+\frac{\varphi}{\omega}|v_1-v_2|. \tag*{(18)} ∣ g ( b 1 , v 1 ; φ ) − g ( b 2 , v 2 ; φ ) ∣ ≤ ∣ b 1 − b 2 ∣ + ω φ ∣ v 1 − v 2 ∣. (18)
We first outline some useful results on the derivatives. More generally, direct differentiation on { ( b , v , φ ) : b ≥ ω > 0 , v ≥ 0 , θ ∈ Θ } \{(b,v,\varphi):b\geq\omega>0,\ v\geq0,\ \theta\in\Theta\} {( b , v , φ ) : b ≥ ω > 0 , v ≥ 0 , θ ∈ Θ } yields that
∣ ∂ g ∂ v ∣ = φ b 2 + 4 φ v ≤ φ ω ≤ C , ∣ ∂ g ∂ φ ∣ = v b 2 + 4 φ v ≤ v ω ≤ C v , (19) \left|\frac{\partial g}{\partial v}\right|=\frac{\varphi}{\sqrt{b^{2}+4\varphi v}}\leq\frac{\varphi}{\omega}\leq C,\qquad\left|\frac{\partial g}{\partial\varphi}\right|=\frac{v}{\sqrt{b^{2}+4\varphi v}}\leq\frac{v}{\omega}\leq Cv, \tag*{(19)} ∂ v ∂ g = b 2 + 4 φ v φ ≤ ω φ ≤ C , ∂ φ ∂ g = b 2 + 4 φ v v ≤ ω v ≤ C v , (19)
∣ ∂ 2 g ∂ b 2 ∣ = 2 φ v ( b 2 + 4 φ v ) 3 / 2 ≤ 1 2 ω ≤ C , ∣ ∂ 2 g ∂ b ∂ v ∣ = φ b ( b 2 + 4 φ v ) 3 / 2 ≤ φ ω 2 ≤ C , \left|\frac{\partial^{2}g}{\partial b^{2}}\right|=\frac{2\varphi v}{(b^{2}+4\varphi v)^{3/2}}\leq\frac{1}{2\omega}\leq C,\qquad\left|\frac{\partial^{2}g}{\partial b\,\partial v}\right|=\frac{\varphi b}{(b^{2}+4\varphi v)^{3/2}}\leq\frac{\varphi}{\omega^{2}}\leq C, ∂ b 2 ∂ 2 g = ( b 2 + 4 φ v ) 3/2 2 φ v ≤ 2 ω 1 ≤ C , ∂ b ∂ v ∂ 2 g = ( b 2 + 4 φ v ) 3/2 φ b ≤ ω 2 φ ≤ C ,
∣ ∂ 2 g ∂ v 2 ∣ = 2 φ 2 ( b 2 + 4 φ v ) 3 / 2 ≤ 2 φ 2 ω 3 ≤ C , ∣ ∂ 2 g ∂ v ∂ φ ∣ = b 2 + 2 φ v ( b 2 + 4 φ v ) 3 / 2 ≤ 1 ω ≤ C , \left|\frac{\partial^{2}g}{\partial v^{2}}\right|=\frac{2\varphi^{2}}{(b^{2}+4\varphi v)^{3/2}}\leq\frac{2\varphi^{2}}{\omega^{3}}\leq C,\qquad\left|\frac{\partial^{2}g}{\partial v\,\partial\varphi}\right|=\frac{b^{2}+2\varphi v}{(b^{2}+4\varphi v)^{3/2}}\leq\frac{1}{\omega}\leq C, ∂ v 2 ∂ 2 g = ( b 2 + 4 φ v ) 3/2 2 φ 2 ≤ ω 3 2 φ 2 ≤ C , ∂ v ∂ φ ∂ 2 g = ( b 2 + 4 φ v ) 3/2 b 2 + 2 φ v ≤ ω 1 ≤ C ,
∣ ∂ 2 g ∂ b ∂ φ ∣ = b v ( b 2 + 4 φ v ) 3 / 2 ≤ v ω 2 ≤ C v , ∣ ∂ 2 g ∂ φ 2 ∣ = 2 v 2 ( b 2 + 4 φ v ) 3 / 2 ≤ 2 v 2 ω 3 ≤ C v 2 , (20) \left|\frac{\partial^{2}g}{\partial b\,\partial\varphi}\right|=\frac{bv}{(b^{2}+4\varphi v)^{3/2}}\leq\frac{v}{\omega^{2}}\leq Cv,\qquad\left|\frac{\partial^{2}g}{\partial\varphi^{2}}\right|=\frac{2v^{2}}{(b^{2}+4\varphi v)^{3/2}}\leq\frac{2v^{2}}{\omega^{3}}\leq Cv^{2}, \tag*{(20)} ∂ b ∂ φ ∂ 2 g = ( b 2 + 4 φ v ) 3/2 b v ≤ ω 2 v ≤ C v , ∂ φ 2 ∂ 2 g = ( b 2 + 4 φ v ) 3/2 2 v 2 ≤ ω 3 2 v 2 ≤ C v 2 , (20)
where we use 2 φ v ≤ ( b 2 + 4 φ v ) / 2 2\varphi v\leq(b^{2}+4\varphi v)/2 2 φ v ≤ ( b 2 + 4 φ v ) /2 and b 2 + 4 φ v ≥ ω \sqrt{b^{2}+4\varphi v}\geq\omega b 2 + 4 φ v ≥ ω . Similarly, the third-order partial derivatives of g g g satisfy
∑ i + j = 3 ∣ ∂ 3 g ∂ b i ∂ v j ∣ + ∣ ∂ 3 g ∂ b ∂ v ∂ φ ∣ + ∣ ∂ 3 g ∂ v 2 ∂ φ ∣ ≤ C , ∣ ∂ 3 g ∂ b 2 ∂ φ ∣ ≤ C ( 1 + v ) , ∣ ∂ 3 g ∂ b ∂ φ 2 ∣ ≤ C v 2 . (21) \sum_{i+j=3}\left|\frac{\partial^{3}g}{\partial b^{i}\partial v^{j}}\right|+\left|\frac{\partial^{3}g}{\partial b\,\partial v\,\partial\varphi}\right|+\left|\frac{\partial^{3}g}{\partial v^{2}\partial\varphi}\right|\leq C,\qquad\left|\frac{\partial^{3}g}{\partial b^{2}\partial\varphi}\right|\leq C(1+v),\qquad\left|\frac{\partial^{3}g}{\partial b\,\partial\varphi^{2}}\right|\leq Cv^{2}. \tag*{(21)} i + j = 3 ∑ ∂ b i ∂ v j ∂ 3 g + ∂ b ∂ v ∂ φ ∂ 3 g + ∂ v 2 ∂ φ ∂ 3 g ≤ C , ∂ b 2 ∂ φ ∂ 3 g ≤ C ( 1 + v ) , ∂ b ∂ φ 2 ∂ 3 g ≤ C v 2 . (21)
Consequently, by the mean value theorem, for any b 1 , b 2 ≥ ω b_1,b_2\geq\omega b 1 , b 2 ≥ ω , v 1 , v 2 ≥ 0 v_1,v_2\geq0 v 1 , v 2 ≥ 0 , and θ ∈ Θ \theta\in\Theta θ ∈ Θ , the partial derivatives of g g g satisfy the explicit Lipschitz bounds
∣ ∂ g ( 1 ) ∂ b − ∂ g ( 2 ) ∂ b ∣ + ∣ ∂ g ( 1 ) ∂ v − ∂ g ( 2 ) ∂ v ∣ + ∣ ∂ 2 g ( 1 ) ∂ b 2 − ∂ 2 g ( 2 ) ∂ b 2 ∣ ≤ C ( ∣ b 1 − b 2 ∣ + ∣ v 1 − v 2 ∣ ) , \left|\frac{\partial g^{(1)}}{\partial b}-\frac{\partial g^{(2)}}{\partial b}\right|+\left|\frac{\partial g^{(1)}}{\partial v}-\frac{\partial g^{(2)}}{\partial v}\right|+\left|\frac{\partial^{2}g^{(1)}}{\partial b^{2}}-\frac{\partial^{2}g^{(2)}}{\partial b^{2}}\right|\leq C(|b_1-b_2|+|v_1-v_2|), ∂ b ∂ g ( 1 ) − ∂ b ∂ g ( 2 ) + ∂ v ∂ g ( 1 ) − ∂ v ∂ g ( 2 ) + ∂ b 2 ∂ 2 g ( 1 ) − ∂ b 2 ∂ 2 g ( 2 ) ≤ C ( ∣ b 1 − b 2 ∣ + ∣ v 1 − v 2 ∣ ) ,
∣ ∂ g ( 1 ) ∂ φ − ∂ g ( 2 ) ∂ φ ∣ + ∣ ∂ 2 g ( 1 ) ∂ b ∂ φ − ∂ 2 g ( 2 ) ∂ b ∂ φ ∣ ≤ C ( 1 + v 1 + v 2 ) ( ∣ b 1 − b 2 ∣ + ∣ v 1 − v 2 ∣ ) , (22) \left|\frac{\partial g^{(1)}}{\partial\varphi}-\frac{\partial g^{(2)}}{\partial\varphi}\right|+\left|\frac{\partial^{2}g^{(1)}}{\partial b\,\partial\varphi}-\frac{\partial^{2}g^{(2)}}{\partial b\,\partial\varphi}\right|\leq C(1+v_1+v_2)(|b_1-b_2|+|v_1-v_2|), \tag*{(22)} ∂ φ ∂ g ( 1 ) − ∂ φ ∂ g ( 2 ) + ∂ b ∂ φ ∂ 2 g ( 1 ) − ∂ b ∂ φ ∂ 2 g ( 2 ) ≤ C ( 1 + v 1 + v 2 ) ( ∣ b 1 − b 2 ∣ + ∣ v 1 − v 2 ∣ ) , (22)
∣ ∂ 2 g ( 1 ) ∂ φ 2 − ∂ 2 g ( 2 ) ∂ φ 2 ∣ ≤ C ( 1 + v 1 + v 2 ) 2 ( ∣ b 1 − b 2 ∣ + ∣ v 1 − v 2 ∣ ) . \left|\frac{\partial^{2}g^{(1)}}{\partial\varphi^{2}}-\frac{\partial^{2}g^{(2)}}{\partial\varphi^{2}}\right|\leq C(1+v_1+v_2)^{2}(|b_1-b_2|+|v_1-v_2|). ∂ φ 2 ∂ 2 g ( 1 ) − ∂ φ 2 ∂ 2 g ( 2 ) ≤ C ( 1 + v 1 + v 2 ) 2 ( ∣ b 1 − b 2 ∣ + ∣ v 1 − v 2 ∣ ) .
where g ( m ) : = g ( b m , v m ; φ ) g^{(m)} := g(b_m,v_m;\varphi) g ( m ) := g ( b m , v m ; φ ) (m = 1 , 2 m = 1,2 m = 1 , 2 ).
Next, for f ( v , h , φ ) f(v,h,\varphi) f ( v , h , φ ) defined in (A.1), direct differentiation on { ( v , h , φ ) : v ≥ 0 , h ≥ ω > 0 , θ ∈ Θ } \{(v,h,\varphi): v \ge0,\ h \ge\omega> 0,\ \theta\in\Theta\} {( v , h , φ ) : v ≥ 0 , h ≥ ω > 0 , θ ∈ Θ } gives
∂ f ∂ h = − v 2 h 2 + 2 h h 2 + φ v − 3 2 h , ∂ f ∂ φ = v h 2 + φ v , ∂ f ∂ v = 1 2 h + φ h 2 + φ v , \frac{\partial f}{\partial h}=-\frac{v}{2h^2}+\frac{2h}{h^2+\varphi v}-\frac{3}{2h},\qquad
\frac{\partial f}{\partial\varphi}=\frac{v}{h^2+\varphi v},\qquad
\frac{\partial f}{\partial v}=\frac{1}{2h}+\frac{\varphi}{h^2+\varphi v}, ∂ h ∂ f = − 2 h 2 v + h 2 + φ v 2 h − 2 h 3 , ∂ φ ∂ f = h 2 + φ v v , ∂ v ∂ f = 2 h 1 + h 2 + φ v φ ,
∂ 2 f ∂ h 2 = v h 3 + 2 ( φ v − h 2 ) ( h 2 + φ v ) 2 + 3 2 h 2 , ∂ 2 f ∂ h ∂ φ = − 2 h v ( h 2 + φ v ) 2 , ∂ 2 f ∂ φ 2 = − v 2 ( h 2 + φ v ) 2 , \frac{\partial^2 f}{\partial h^2}=\frac{v}{h^3}+\frac{2(\varphi v-h^2)}{(h^2+\varphi v)^2}+\frac{3}{2h^2},\qquad
\frac{\partial^2 f}{\partial h\partial\varphi}=-\frac{2hv}{(h^2+\varphi v)^2},\qquad
\frac{\partial^2 f}{\partial\varphi^2}=-\frac{v^2}{(h^2+\varphi v)^2}, ∂ h 2 ∂ 2 f = h 3 v + ( h 2 + φ v ) 2 2 ( φ v − h 2 ) + 2 h 2 3 , ∂ h ∂ φ ∂ 2 f = − ( h 2 + φ v ) 2 2 h v , ∂ φ 2 ∂ 2 f = − ( h 2 + φ v ) 2 v 2 ,
∂ 2 f ∂ h ∂ v = − 1 2 h 2 − 2 h φ ( h 2 + φ v ) 2 , ∂ 2 f ∂ φ ∂ v = h 2 ( h 2 + φ v ) 2 . (23) \frac{\partial^2 f}{\partial h\partial v}=-\frac{1}{2h^2}-\frac{2h\varphi}{(h^2+\varphi v)^2},\qquad
\frac{\partial^2 f}{\partial\varphi\partial v}=\frac{h^2}{(h^2+\varphi v)^2}. \tag*{(23)} ∂ h ∂ v ∂ 2 f = − 2 h 2 1 − ( h 2 + φ v ) 2 2 h φ , ∂ φ ∂ v ∂ 2 f = ( h 2 + φ v ) 2 h 2 . (23)
Since h 2 + φ v ≥ h 2 ≥ ω 2 > 0 h^2+\varphi v \ge h^2 \ge\omega^2 > 0 h 2 + φ v ≥ h 2 ≥ ω 2 > 0 , the partial derivatives of f f f up to order three satisfy that uniformly on Θ \Theta Θ
∑ j = 1 3 ∣ ∂ j f ∂ h j ∣ ≤ C ( 1 + v ) , ∣ ∂ f ∂ φ ∣ + ∣ ∂ 2 f ∂ h ∂ φ ∣ + ∣ ∂ 3 f ∂ h 2 ∂ φ ∣ ≤ C v , ∣ ∂ 2 f ∂ φ 2 ∣ + ∣ ∂ 3 f ∂ h ∂ φ 2 ∣ ≤ C v 2 , \sum_{j=1}^{3}\left|\frac{\partial^j f}{\partial h^j}\right|\le C(1+v),\qquad
\left|\frac{\partial f}{\partial\varphi}\right|+\left|\frac{\partial^2 f}{\partial h\partial\varphi}\right|+\left|\frac{\partial^3 f}{\partial h^2\partial\varphi}\right|\le Cv,\qquad
\left|\frac{\partial^2 f}{\partial\varphi^2}\right|+\left|\frac{\partial^3 f}{\partial h\partial\varphi^2}\right|\le Cv^2, j = 1 ∑ 3 ∂ h j ∂ j f ≤ C ( 1 + v ) , ∂ φ ∂ f + ∂ h ∂ φ ∂ 2 f + ∂ h 2 ∂ φ ∂ 3 f ≤ C v , ∂ φ 2 ∂ 2 f + ∂ h ∂ φ 2 ∂ 3 f ≤ C v 2 ,
∣ ∂ f ∂ v ∣ + ∣ ∂ 2 f ∂ h ∂ v ∣ + ∣ ∂ 2 f ∂ φ ∂ v ∣ + ∣ ∂ 3 f ∂ h 2 ∂ v ∣ + ∣ ∂ 3 f ∂ h ∂ φ ∂ v ∣ ≤ C , ∣ ∂ 2 f ∂ φ 2 ∂ v ∣ ≤ C v . (24) \left|\frac{\partial f}{\partial v}\right|+\left|\frac{\partial^2 f}{\partial h\partial v}\right|+\left|\frac{\partial^2 f}{\partial\varphi\partial v}\right|+\left|\frac{\partial^3 f}{\partial h^2\partial v}\right|+\left|\frac{\partial^3 f}{\partial h\partial\varphi\partial v}\right|\le C,\qquad
\left|\frac{\partial^2 f}{\partial\varphi^2\partial v}\right|\le Cv. \tag*{(24)} ∂ v ∂ f + ∂ h ∂ v ∂ 2 f + ∂ φ ∂ v ∂ 2 f + ∂ h 2 ∂ v ∂ 3 f + ∂ h ∂ φ ∂ v ∂ 3 f ≤ C , ∂ φ 2 ∂ v ∂ 2 f ≤ C v . (24)
By the mean value theorem and (A.9), for any v 1 , v 2 ≥ 0 v_1,v_2 \ge0 v 1 , v 2 ≥ 0 , h 1 , h 2 ≥ ω h_1,h_2 \ge\omega h 1 , h 2 ≥ ω , and θ ∈ Θ \theta\in\Theta θ ∈ Θ , we have the explicit Lipschitz inequalities
∣ f ( 1 ) − f ( 2 ) ∣ + ∣ ∂ f ( 1 ) ∂ h − ∂ f ( 2 ) ∂ h ∣ + ∣ ∂ f ( 1 ) ∂ φ − ∂ f ( 2 ) ∂ φ ∣ ≤ C ∣ v 1 − v 2 ∣ + C ( 1 + v 1 + v 2 ) ∣ h 1 − h 2 ∣ , |f^{(1)}-f^{(2)}|+\left|\frac{\partial f^{(1)}}{\partial h}-\frac{\partial f^{(2)}}{\partial h}\right|+\left|\frac{\partial f^{(1)}}{\partial\varphi}-\frac{\partial f^{(2)}}{\partial\varphi}\right|
\le C|v_1-v_2|+C(1+v_1+v_2)|h_1-h_2|, ∣ f ( 1 ) − f ( 2 ) ∣ + ∂ h ∂ f ( 1 ) − ∂ h ∂ f ( 2 ) + ∂ φ ∂ f ( 1 ) − ∂ φ ∂ f ( 2 ) ≤ C ∣ v 1 − v 2 ∣ + C ( 1 + v 1 + v 2 ) ∣ h 1 − h 2 ∣ ,
∣ ∂ 2 f ( 1 ) ∂ h 2 − ∂ 2 f ( 2 ) ∂ h 2 ∣ + ∣ ∂ 2 f ( 1 ) ∂ h ∂ φ − ∂ 2 f ( 2 ) ∂ h ∂ φ ∣ ≤ C ∣ v 1 − v 2 ∣ + C ( 1 + v 1 + v 2 ) ∣ h 1 − h 2 ∣ , \left|\frac{\partial^2 f^{(1)}}{\partial h^2}-\frac{\partial^2 f^{(2)}}{\partial h^2}\right|+
\left|\frac{\partial^2 f^{(1)}}{\partial h\partial\varphi}-\frac{\partial^2 f^{(2)}}{\partial h\partial\varphi}\right|
\le C|v_1-v_2|+C(1+v_1+v_2)|h_1-h_2|, ∂ h 2 ∂ 2 f ( 1 ) − ∂ h 2 ∂ 2 f ( 2 ) + ∂ h ∂ φ ∂ 2 f ( 1 ) − ∂ h ∂ φ ∂ 2 f ( 2 ) ≤ C ∣ v 1 − v 2 ∣ + C ( 1 + v 1 + v 2 ) ∣ h 1 − h 2 ∣ ,
∣ ∂ 2 f ( 1 ) ∂ φ 2 − ∂ 2 f ( 2 ) ∂ φ 2 ∣ ≤ C ( 1 + v 1 + v 2 ) ∣ v 1 − v 2 ∣ + C ( 1 + v 1 + v 2 ) 2 ∣ h 1 − h 2 ∣ . (25) \left|\frac{\partial^2 f^{(1)}}{\partial\varphi^2}-\frac{\partial^2 f^{(2)}}{\partial\varphi^2}\right|
\le C(1+v_1+v_2)|v_1-v_2|+C(1+v_1+v_2)^2|h_1-h_2|. \tag*{(25)} ∂ φ 2 ∂ 2 f ( 1 ) − ∂ φ 2 ∂ 2 f ( 2 ) ≤ C ( 1 + v 1 + v 2 ) ∣ v 1 − v 2 ∣ + C ( 1 + v 1 + v 2 ) 2 ∣ h 1 − h 2 ∣. (25)
where f ( m ) : = f ( v m , h m , φ ) f^{(m)} := f(v_m,h_m,\varphi) f ( m ) := f ( v m , h m , φ ) (m = 1 , 2 m = 1,2 m = 1 , 2 ).
Furthermore, on any compact subset N \mathcal{N} N of the interior of Θ \Theta Θ , we have φ ‾ : = inf θ ∈ N φ > 0 \underline{\varphi}:=\inf_{\theta\in\mathcal{N}}\varphi>0 φ := inf θ ∈ N φ > 0 . At any state h = g ( b , v ; φ ) h=g(b,v;\varphi) h = g ( b , v ; φ ) with b ≥ ω b\ge\omega b ≥ ω , the quadratic relation h 2 = b h + φ v ≥ max ( ω 2 , φ v ) h^2=bh+\varphi v\ge\max(\omega^2,\varphi v) h 2 = bh + φ v ≥ max ( ω 2 , φ v ) implies v / h 2 ≤ 1 / φ v/h^2\le1/\varphi v / h 2 ≤ 1/ φ and h 2 + φ v ≥ max ( h 2 , φ v ) h^2+\varphi v\ge\max(h^2,\varphi v) h 2 + φ v ≥ max ( h 2 , φ v ) . Consequently, at any state on N \mathcal{N} N , all partial derivatives of f f f up to order three satisfy
∑ 1 ≤ j + k ≤ 3 ∣ ∂ j + k f ( v , h , φ ) ∂ h j ∂ φ k ∣ + ∑ i = 1 2 ∑ 1 ≤ j + k ≤ 3 − i v i − 1 ∣ ∂ i + j + k f ( v , h , φ ) ∂ v i ∂ h j ∂ φ k ∣ ≤ C . (26) \sum_{1\le j+k\le3}\left|\frac{\partial^{j+k}f(v,h,\varphi)}{\partial h^j\partial\varphi^k}\right|
+\sum_{i=1}^{2}\sum_{1\le j+k\le3-i}v^{i-1}\left|\frac{\partial^{i+j+k}f(v,h,\varphi)}{\partial v^i\partial h^j\partial\varphi^k}\right|
\le C. \tag*{(26)} 1 ≤ j + k ≤ 3 ∑ ∂ h j ∂ φ k ∂ j + k f ( v , h , φ ) + i = 1 ∑ 2 1 ≤ j + k ≤ 3 − i ∑ v i − 1 ∂ v i ∂ h j ∂ φ k ∂ i + j + k f ( v , h , φ ) ≤ C . (26)
Proof of Proposition 1. Since h t = b t − 1 + φ ϵ t 2 h_t=b_{t-1}+\varphi\epsilon_t^2 h t = b t − 1 + φ ϵ t 2 and u t 2 = b t − 1 ϵ t 2 + φ ϵ t 4 u_t^2=b_{t-1}\epsilon_t^2+\varphi\epsilon_t^4 u t 2 = b t − 1 ϵ t 2 + φ ϵ t 4 , we have that
b t = C t + A t b t − 1 , b_t=C_t+A_tb_{t-1}, b t = C t + A t b t − 1 ,
where A t = α ϵ t 2 + β A_t=\alpha\epsilon_t^2+\beta A t = α ϵ t 2 + β , and C t = ω + φ ( α ϵ t 4 + β ϵ t 2 ) C_t=\omega+\varphi(\alpha\epsilon_t^4+\beta\epsilon_t^2) C t = ω + φ ( α ϵ t 4 + β ϵ t 2 ) , with ( A t , C t ) (A_t,C_t) ( A t , C t ) being independent of b t − 1 b_{t-1} b t − 1 . Note that { ( A t , C t ) } \{(A_t,C_t)\} {( A t , C t )} are i.i.d., E ( A t ) = α + β < 1 E(A_t)=\alpha+\beta<1 E ( A t ) = α + β < 1 , and E ( C t ) < ∞ E(C_t)<\infty E ( C t ) < ∞ by that E ( ϵ t 4 ) < ∞ E(\epsilon_t^4)<\infty E ( ϵ t 4 ) < ∞ . By iteration, we obtain that
b t = C t + ∑ k = 1 ∞ ( ∏ j = 0 k − 1 A t − j ) C t − k b_t=C_t+\sum_{k=1}^{\infty}\left(\prod_{j=0}^{k-1}A_{t-j}\right)C_{t-k} b t = C t + k = 1 ∑ ∞ ( j = 0 ∏ k − 1 A t − j ) C t − k
which converges in L 1 L^1 L 1 and almost surely because ∑ k ≥ 0 E ( C t ) { E ( A t ) } k < ∞ \sum_{k\geq0} E(C_t)\{E(A_t)\}^k < \infty ∑ k ≥ 0 E ( C t ) { E ( A t ) } k < ∞ . The rest of the proof follows directly from Theorem 1 in Smetanina and Wu [34 ] .
Proof of Proposition 2. Let m u = E ( u t 2 ) m_u = E(u_t^2) m u = E ( u t 2 ) and m h = E ( h t ) m_h = E(h_t) m h = E ( h t ) . Since b t − 1 b_{t-1} b t − 1 is F t − 1 \mathcal{F}_{t-1} F t − 1 -measurable and ϵ t \epsilon_t ϵ t is independent of F t − 1 \mathcal{F}_{t-1} F t − 1 , taking expectations in h t = b t − 1 + φ ϵ t 2 h_t = b_{t-1} + \varphi\epsilon_t^2 h t = b t − 1 + φ ϵ t 2 and u t 2 = b t − 1 ϵ t 2 + φ ϵ t 4 u_t^2 = b_{t-1}\epsilon_t^2 + \varphi\epsilon_t^4 u t 2 = b t − 1 ϵ t 2 + φ ϵ t 4 gives
m h = ω + α m u + β m h + φ , (27) m_h = \omega+ \alpha m_u + \beta m_h + \varphi, \tag*{(27)} m h = ω + α m u + β m h + φ , (27)
m u = ω + α m u + β m h + φ μ 4 . (28) m_u = \omega+ \alpha m_u + \beta m_h + \varphi\mu_4. \tag*{(28)} m u = ω + α m u + β m h + φ μ 4 . (28)
Hence, m u − m h = φ ( μ 4 − 1 ) m_u - m_h = \varphi(\mu_4 - 1) m u − m h = φ ( μ 4 − 1 ) . Substituting this and (2) into (A.12) gives ( 1 − α − β ) ( m u − 1 ) = 0 (1-\alpha-\beta)(m_u-1)=0 ( 1 − α − β ) ( m u − 1 ) = 0 . Thus m u = 1 m_u=1 m u = 1 and m h = 1 − φ ( μ 4 − 1 ) m_h=1-\varphi(\mu_4-1) m h = 1 − φ ( μ 4 − 1 ) .
For fixed b t − 1 > 0 b_{t-1}>0 b t − 1 > 0 , the mapping e ↦ e b t − 1 + φ e 2 e \mapsto e\sqrt{b_{t-1}+\varphi e^2} e ↦ e b t − 1 + φ e 2 is odd. Using symmetry and integrability, we then obtain that E ( u t ∣ F t − 1 ) = 0 E(u_t\mid\mathcal{F}_{t-1})=0 E ( u t ∣ F t − 1 ) = 0 . For j ≥ 1 j\geq1 j ≥ 1 , u t − j u_{t-j} u t − j is F t − 1 \mathcal{F}_{t-1} F t − 1 -measurable and E ∣ u t u t − j ∣ < ∞ E|u_tu_{t-j}|<\infty E ∣ u t u t − j ∣ < ∞ by Cauchy-Schwarz inequality. Hence E ( u t u t − j ) = E { u t − j E ( u t ∣ F t − 1 ) } = 0 E(u_tu_{t-j})=E\{u_{t-j}E(u_t\mid\mathcal{F}_{t-1})\}=0 E ( u t u t − j ) = E { u t − j E ( u t ∣ F t − 1 )} = 0 , the result follows.
Proof of Proposition 3. Conditional on F t − 1 \mathcal{F}_{t-1} F t − 1 , b t − 1 b_{t-1} b t − 1 is fixed and
r t 2 = τ t ( b t − 1 ϵ t 2 + φ 0 ϵ t 4 ) . r_t^2=\tau_t\left(b_{t-1}\epsilon_t^2+\varphi_0\epsilon_t^4\right). r t 2 = τ t ( b t − 1 ϵ t 2 + φ 0 ϵ t 4 ) .
Using the independence of ϵ t \epsilon_t ϵ t from F t − 1 \mathcal{F}_{t-1} F t − 1 , we have
E ( r t 2 ∣ F t − 1 ) = τ t ( b t − 1 + φ 0 μ 4 ) . E(r_t^2\mid\mathcal{F}_{t-1})=\tau_t(b_{t-1}+\varphi_0\mu_4). E ( r t 2 ∣ F t − 1 ) = τ t ( b t − 1 + φ 0 μ 4 ) .
Similarly,
r t 4 = τ t 2 ( b t − 1 2 ϵ t 4 + 2 φ 0 b t − 1 ϵ t 6 + φ 0 2 ϵ t 8 ) . r_t^4=\tau_t^2\left(b_{t-1}^2\epsilon_t^4+2\varphi_0b_{t-1}\epsilon_t^6+\varphi_0^2\epsilon_t^8\right). r t 4 = τ t 2 ( b t − 1 2 ϵ t 4 + 2 φ 0 b t − 1 ϵ t 6 + φ 0 2 ϵ t 8 ) .
Dividing the latter expression by the square of the conditional second moment proves (4) .
Proof of Proposition 5. Recall in the proof of Proposition 1, we have b t = C t + A t b t − 1 b_t=C_t+A_tb_{t-1} b t = C t + A t b t − 1 , A t = α ϵ t 2 + β A_t=\alpha\epsilon_t^2+\beta A t = α ϵ t 2 + β , C t = ω + φ { α ϵ t 4 + β ϵ t 2 } C_t=\omega+\varphi\{\alpha\epsilon_t^4+\beta\epsilon_t^2\} C t = ω + φ { α ϵ t 4 + β ϵ t 2 } , where { ( A t , C t ) } \{(A_t,C_t)\} {( A t , C t )} is i.i.d. and independent of b t − 1 b_{t-1} b t − 1 . Moreover,
u t 2 = b t − 1 ϵ t 2 + φ ϵ t 4 . u_t^2=b_{t-1}\epsilon_t^2+\varphi\epsilon_t^4. u t 2 = b t − 1 ϵ t 2 + φ ϵ t 4 .
Since b t − 1 ≥ ω > 0 b_{t-1}\geq\omega>0 b t − 1 ≥ ω > 0 is independent of ϵ t \epsilon_t ϵ t , the moment E ∣ u t ∣ 2 p < ∞ E|u_t|^{2p}<\infty E ∣ u t ∣ 2 p < ∞ implies E ( b t p ) < ∞ E(b_t^p)<\infty E ( b t p ) < ∞ and E ∣ ϵ t ∣ 2 p < ∞ E|\epsilon_t|^{2p}<\infty E ∣ ϵ t ∣ 2 p < ∞ . Also, b t ≥ C t ≥ ω b_t\geq C_t\geq\omega b t ≥ C t ≥ ω gives 0 < E ( C t p ) < ∞ 0<E(C_t^p)<\infty 0 < E ( C t p ) < ∞ . By the fact that ( a + c ) p ≥ a p + c p (a+c)^p\geq a^p+c^p ( a + c ) p ≥ a p + c p for a , c ≥ 0 a,c\geq0 a , c ≥ 0 , we then obtain that
E ( b t p ) ≥ E ( C t p ) + E ( A t p ) E ( b t − 1 p ) . E(b_t^p)\geq E(C_t^p)+E(A_t^p)E(b_{t-1}^p). E ( b t p ) ≥ E ( C t p ) + E ( A t p ) E ( b t − 1 p ) .
Consequently, E ( A t p ) < 1 E(A_t^p)<1 E ( A t p ) < 1 using the stationarity of b t b_t b t and that 0 < E ( C t p ) < ∞ 0<E(C_t^p)<\infty 0 < E ( C t p ) < ∞ . Without loss of generality, we choose ρ p ∈ ( ( E ( A t p ) ) 1 / p , 1 ) \rho_p\in((E(A_t^p))^{1/p},1) ρ p ∈ (( E ( A t p ) ) 1/ p , 1 ) .
Let b t ∘ b_t^\circ b t ∘ be the coupled stationary solution by Definition 1, then iterating the recursion gives
∥ b t − b t ∘ ∥ p = { E ( A t p ) } t / p ∥ b 0 − b 0 ∘ ∥ p . \|b_t-b_t^\circ\|_p=\{E(A_t^p)\}^{t/p}\|b_0-b_0^\circ\|_p. ∥ b t − b t ∘ ∥ p = { E ( A t p ) } t / p ∥ b 0 − b 0 ∘ ∥ p .
For t ≥ 1 t\geq1 t ≥ 1 , we have h t − h t ∘ = b t − 1 − b t − 1 ∘ h_t-h_t^\circ=b_{t-1}-b_{t-1}^\circ h t − h t ∘ = b t − 1 − b t − 1 ∘ and z t − z t ∘ = ϵ t 2 ( b t − 1 − b t − 1 ∘ ) z_t-z_t^\circ=\epsilon_t^2(b_{t-1}-b_{t-1}^\circ) z t − z t ∘ = ϵ t 2 ( b t − 1 − b t − 1 ∘ ) . Therefore, replacing only ϵ 0 \epsilon_0 ϵ 0 changes the initial difference in b 0 b_0 b 0 and b 0 ∘ b_0^\circ b 0 ∘ but leaves the same product recursion. This proves the result.
Proof of Theorem 1. Applying Proposition 5 with p = 2 p=2 p = 2 and Lemma 1(ii) to the stationary sequence { z t } \{z_t\} { z t } yields that ∣ Cov ( z 0 , z j ) ∣ ≤ C ρ ∣ j ∣ |\operatorname{Cov}(z_0,z_j)| \le C\rho^{|j|} ∣ Cov ( z 0 , z j ) ∣ ≤ C ρ ∣ j ∣ for all j ∈ Z j \in\mathbb{Z} j ∈ Z . Hence, the absolute convergence of Ω z = ∑ j ∈ Z Cov ( z 0 , z j ) \Omega_z=\sum_{j\in\mathbb{Z}}\operatorname{Cov}(z_0,z_j) Ω z = ∑ j ∈ Z Cov ( z 0 , z j ) , and the functional central limit theorem Wu [37 ] is satisfied, and therefore
1 n ∑ t = 1 ⌊ n r ⌋ z t ⇒ Ω z 1 / 2 B ( r ) , 0 ≤ r ≤ 1. (29) \frac{1}{\sqrt{n}}\sum_{t=1}^{\lfloor nr\rfloor}z_t \Rightarrow\Omega_z^{1/2}B(r),\qquad0\le r\le1. \tag*{(29)} n 1 t = 1 ∑ ⌊ n r ⌋ z t ⇒ Ω z 1/2 B ( r ) , 0 ≤ r ≤ 1. (29)
For fixed x ∈ ( 0 , 1 ) x\in(0,1) x ∈ ( 0 , 1 ) , the kernel uses only a consecutive block of about 2 T h 2Th 2 T h observations. Let M = T h M=Th M = T h and l T = ⌊ T x − M ⌋ l_T=\lfloor Tx-M\rfloor l T = ⌊ T x − M ⌋ . By stationarity, the partial sums of z l T + 1 , z l T + 2 , … z_{l_T+1},z_{l_T+2},\ldots z l T + 1 , z l T + 2 , … have the same distribution as those starting at t = 1 t=1 t = 1 . (A.14) therefore gives
M − 1 / 2 ∑ j = 1 ⌊ M r ⌋ z l T + j ⇒ Ω z 1 / 2 B ( r ) , 0 ≤ r ≤ 2. M^{-1/2}\sum_{j=1}^{\lfloor Mr\rfloor}z_{l_T+j}\Rightarrow\Omega_z^{1/2}B(r),\qquad0\le r\le2. M − 1/2 j = 1 ∑ ⌊ M r ⌋ z l T + j ⇒ Ω z 1/2 B ( r ) , 0 ≤ r ≤ 2.
Since K K K is Lipschitz and has bounded variation, summation by parts gives the limit Ω z 1 / 2 ∫ 0 2 K ( 1 − r ) d B ( r ) \Omega_z^{1/2}\int_0^2K(1-r)dB(r) Ω z 1/2 ∫ 0 2 K ( 1 − r ) d B ( r ) . Integer endpoints change the weighted sum by o p ( 1 ) o_p(1) o p ( 1 ) . Consequently,
1 T h ∑ s = 1 T K ( x − s / T h ) z s → d N ( 0 , Ω z ∫ − 1 1 K 2 ( v ) d v ) . \frac{1}{\sqrt{Th}}\sum_{s=1}^{T}K\left(\frac{x-s/T}{h}\right)z_s\to_d N\left(0,\Omega_z\int_{-1}^{1}K^2(v)\,dv\right). T h 1 s = 1 ∑ T K ( h x − s / T ) z s → d N ( 0 , Ω z ∫ − 1 1 K 2 ( v ) d v ) .
For a fixed interior point x x x , write
τ ^ ( x ) − E { τ ^ ( x ) } = 1 T h ∑ s = 1 T K ( x − s / T h ) τ ( s / T ) z s . \widehat{\tau}(x)-E\{\widehat{\tau}(x)\}=\frac{1}{Th}\sum_{s=1}^{T}K\left(\frac{x-s/T}{h}\right)\tau(s/T)z_s. τ ( x ) − E { τ ( x )} = T h 1 s = 1 ∑ T K ( h x − s / T ) τ ( s / T ) z s .
A second-order Taylor expansion of τ ( s / T ) \tau(s/T) τ ( s / T ) around x x x , symmetry of K K K , and a Riemann-sum approximation give
E { τ ^ ( x ) } − τ ( x ) = h 2 C 2 2 τ ′ ′ ( x ) + o ( h 2 ) + O ( ( T h ) − 1 ) . E\{\widehat{\tau}(x)\}-\tau(x)=\frac{h^2C_2}{2}\tau^{\prime\prime}(x)+o(h^2)+O((Th)^{-1}). E { τ ( x )} − τ ( x ) = 2 h 2 C 2 τ ′′ ( x ) + o ( h 2 ) + O (( T h ) − 1 ) .
Because T h 5 = O ( 1 ) Th^5=O(1) T h 5 = O ( 1 ) and T h → ∞ Th\to\infty T h → ∞ , this implies that
T h ( E { τ ^ ( x ) } − τ ( x ) ) = T h h 2 C 2 2 τ ′ ′ ( x ) + o ( 1 ) . \sqrt{Th}\bigl(E\{\widehat{\tau}(x)\}-\tau(x)\bigr)=\sqrt{Th}\frac{h^2C_2}{2}\tau^{\prime\prime}(x)+o(1). T h ( E { τ ( x )} − τ ( x ) ) = T h 2 h 2 C 2 τ ′′ ( x ) + o ( 1 ) .
Furthermore, using the boundedness of τ \tau τ , and Assumption 3, we can see that
∥ 1 T h ∑ s = 1 T K ( x − s / T h ) { τ ( s / T ) − τ ( x ) } z s ∥ 2 2 ≤ C h 2 ∑ j ∈ Z ∣ Cov ( z 0 , z j ) ∣ = O ( h 2 ) . \left\|\frac{1}{\sqrt{Th}}\sum_{s=1}^{T}K\left(\frac{x-s/T}{h}\right)\{\tau(s/T)-\tau(x)\}z_s\right\|_2^2\le Ch^2\sum_{j\in\mathbb{Z}}|\operatorname{Cov}(z_0,z_j)|=O(h^2). T h 1 s = 1 ∑ T K ( h x − s / T ) { τ ( s / T ) − τ ( x )} z s 2 2 ≤ C h 2 j ∈ Z ∑ ∣ Cov ( z 0 , z j ) ∣ = O ( h 2 ) .
Here the weights vanish outside O ( T h ) O(Th) O ( T h ) indices and ∣ τ ( s / T ) − τ ( x ) ∣ ≤ C h |\tau(s/T)-\tau(x)|\le Ch ∣ τ ( s / T ) − τ ( x ) ∣ ≤ C h on that block. The kernel central limit theorem therefore yields
1 T h ∑ s = 1 T K ( x − s / T h ) τ ( s / T ) z s → d N ( 0 , τ 2 ( x ) ∫ − 1 1 K 2 ( v ) d v Ω z ) . \frac{1}{\sqrt{Th}}\sum_{s=1}^{T}K\left(\frac{x-s/T}{h}\right)\tau(s/T)z_s\to_d N\left(0,\tau^2(x)\int_{-1}^{1}K^2(v)\,dv\,\Omega_z\right). T h 1 s = 1 ∑ T K ( h x − s / T ) τ ( s / T ) z s → d N ( 0 , τ 2 ( x ) ∫ − 1 1 K 2 ( v ) d v Ω z ) .
The result thus follows. ▪
Proof of Proposition 4. Write b = b t − 1 > 0 b=b_{t-1}>0 b = b t − 1 > 0 and r = τ t e b + φ 0 e 2 r=\sqrt{\tau_t}e\sqrt{b+\varphi_0e^2} r = τ t e b + φ 0 e 2 . This transformation is strictly increasing, with derivative τ t ( b + 2 φ 0 e 2 ) / b + φ 0 e 2 \sqrt{\tau_t}(b+2\varphi_0e^2)/\sqrt{b+\varphi_0e^2} τ t ( b + 2 φ 0 e 2 ) / b + φ 0 e 2 . Its inverse satisfies e ( r ) = r / τ t h t ( r ) e(r)=r/\sqrt{\tau_t h_t(r)} e ( r ) = r / τ t h t ( r ) , where h t ( r ) = { b + b 2 + 4 φ 0 r 2 / τ t } / 2 h_t(r)=\{b+\sqrt{b^2+4\varphi_0r^2/\tau_t}\}/2 h t ( r ) = { b + b 2 + 4 φ 0 r 2 / τ t } /2 . The change-of-variables therefore gives
f t ( r ∣ F t − 1 ) = h t ( r ) τ t b 2 + 4 φ 0 r 2 / τ t f ε ( r τ t h t ( r ) ) . (30) f_t(r\mid\mathcal{F}_{t-1})=\frac{\sqrt{h_t(r)}}{\sqrt{\tau_t}\sqrt{b^2+4\varphi_0r^2/\tau_t}}f_\varepsilon\left(\frac{r}{\sqrt{\tau_t h_t(r)}}\right). \tag*{(30)} f t ( r ∣ F t − 1 ) = τ t b 2 + 4 φ 0 r 2 / τ t h t ( r ) f ε ( τ t h t ( r ) r ) . (30)
At φ 0 = 0 \varphi_0=0 φ 0 = 0 this reduces to
f t ( r ∣ F t − 1 ) = 1 τ t b f ε ( r τ t b ) , (31) f_t(r\mid\mathcal{F}_{t-1})=\frac{1}{\sqrt{\tau_t b}}f_\varepsilon\left(\frac{r}{\sqrt{\tau_t b}}\right), \tag*{(31)} f t ( r ∣ F t − 1 ) = τ t b 1 f ε ( τ t b r ) , (31)
the result follows. ■
For φ 0 > 0 \varphi_0>0 φ 0 > 0 , the inverse transformation above satisfies
e ( r ) 2 = − b + b 2 + 4 φ 0 r 2 / τ t 2 φ 0 = ∣ r ∣ τ t φ 0 + O ( 1 ) . e(r)^2=\frac{-b+\sqrt{b^2+4\varphi_0r^2/\tau_t}}{2\varphi_0}=\frac{|r|}{\sqrt{\tau_t}\varphi_0}+O(1). e ( r ) 2 = 2 φ 0 − b + b 2 + 4 φ 0 r 2 / τ t = τ t φ 0 ∣ r ∣ + O ( 1 ) .
The logarithm of the Jacobian factor in (A.15) is O ( log ∣ r ∣ ) O(\log|r|) O ( log ∣ r ∣ ) . Therefore, under Gaussian innovations,
lim ∣ r ∣ → ∞ log f t ( r ∣ F t − 1 ) ∣ r ∣ = − 1 2 τ t φ 0 , φ 0 > 0. \lim_{|r|\to\infty}\frac{\log f_t(r\mid\mathcal{F}_{t-1})}{|r|}=-\frac{1}{2\sqrt{\tau_t\varphi_0}},\qquad\varphi_0>0. ∣ r ∣ → ∞ lim ∣ r ∣ log f t ( r ∣ F t − 1 ) = − 2 τ t φ 0 1 , φ 0 > 0.
When φ 0 = 0 \varphi_0=0 φ 0 = 0 , (A.16) instead gives
lim ∣ r ∣ → ∞ log f t ( r ∣ F t − 1 ) r 2 = − 1 2 τ t b t − 1 . \lim_{|r|\to\infty}\frac{\log f_t(r\mid\mathcal{F}_{t-1})}{r^2}=-\frac{1}{2\tau_tb_{t-1}}. ∣ r ∣ → ∞ lim r 2 log f t ( r ∣ F t − 1 ) = − 2 τ t b t − 1 1 .
Thus the conditional tails are exponential-type for φ 0 > 0 \varphi_0>0 φ 0 > 0 and Gaussian-type for φ 0 = 0 \varphi_0=0 φ 0 = 0 .
Auxiliary Lemmas
We first establish three key lemmas: Lemma 1 handles contractive triangular recursions and GMC moment interpolation; Lemma 2 establishes the stationary moment envelopes; and Lemma 3 provides explicit perturbation inequalities for the volatility process and the composite log-likelihood with initial values.
Lemma 1. (i) Let β ˉ ∈ ( 0 , 1 ) \bar{\beta}\in(0,1) β ˉ ∈ ( 0 , 1 ) and let { x 0 , t , x 1 , t , x 2 , t } t ≥ 0 \{x_{0,t},x_{1,t},x_{2,t}\}_{t\geq0} { x 0 , t , x 1 , t , x 2 , t } t ≥ 0 be nonnegative random sequences satisfying, for t ≥ 1 t\geq1 t ≥ 1 ,
x 0 , t ≤ β ˉ x 0 , t − 1 + ξ 0 , t , x 1 , t ≤ β ˉ x 1 , t − 1 + B t ( x 0 , t − 1 + ξ 0 , t ) + ξ 1 , t , x 2 , t ≤ β ˉ x 2 , t − 1 + B t x 1 , t − 1 + C t ( x 0 , t − 1 + ξ 0 , t ) + ξ 2 , t , (32) \begin{aligned}
x_{0,t}&\leq\bar{\beta}x_{0,t-1}+\xi_{0,t},\\
x_{1,t}&\leq\bar{\beta}x_{1,t-1}+B_t(x_{0,t-1}+\xi_{0,t})+\xi_{1,t},\\
x_{2,t}&\leq\bar{\beta}x_{2,t-1}+B_tx_{1,t-1}+C_t(x_{0,t-1}+\xi_{0,t})+\xi_{2,t}, \tag*{(32)}
\end{aligned} x 0 , t x 1 , t x 2 , t ≤ β ˉ x 0 , t − 1 + ξ 0 , t , ≤ β ˉ x 1 , t − 1 + B t ( x 0 , t − 1 + ξ 0 , t ) + ξ 1 , t , ≤ β ˉ x 2 , t − 1 + B t x 1 , t − 1 + C t ( x 0 , t − 1 + ξ 0 , t ) + ξ 2 , t , (32) where B t B_t B t , C t C_t C t , ξ r , t ≥ 0 \xi_{r,t}\geq0 ξ r , t ≥ 0 are random sequences. Fix any ρ ∈ ( β ˉ , 1 ) \rho\in(\bar{\beta},1) ρ ∈ ( β ˉ , 1 ) .
(a) Suppose sup t ∥ B t ∥ q < ∞ \sup_t\|B_t\|_q<\infty sup t ∥ B t ∥ q < ∞ , sup t ∥ C t ∥ q / 2 < ∞ \sup_t\|C_t\|_{q/2}<\infty sup t ∥ C t ∥ q /2 < ∞ , ∥ x r , 0 ∥ q / ( r + 1 ) < ∞ \|x_{r,0}\|_{q/(r+1)}<\infty ∥ x r , 0 ∥ q / ( r + 1 ) < ∞ , and ∥ ξ r , t ∥ q / ( r + 1 ) ≤ C 0 ρ t \|\xi_{r,t}\|_{q/(r+1)}\leq C_0\rho^t ∥ ξ r , t ∥ q / ( r + 1 ) ≤ C 0 ρ t for r = 0 , 1 , 2 r=0,1,2 r = 0 , 1 , 2 and t ≥ 1 t\geq1 t ≥ 1 . Then there exists C < ∞ C<\infty C < ∞ depending only on ( β ˉ , ρ , C 0 ) (\bar{\beta},\rho,C_0) ( β ˉ , ρ , C 0 ) and the moment bounds of ( B t , C t , x r , 0 ) (B_t,C_t,x_{r,0}) ( B t , C t , x r , 0 ) such that
∥ x 0 , t ∥ q ≤ C ρ t , ∥ x 1 , t ∥ q / 2 ≤ C ρ t , ∥ x 2 , t ∥ q / 3 ≤ C ρ t , t ≥ 1. (33) \|x_{0,t}\|_q\leq C\rho^t,\qquad\|x_{1,t}\|_{q/2}\leq C\rho^t,\qquad\|x_{2,t}\|_{q/3}\leq C\rho^t,\qquad t\geq1. \tag*{(33)} ∥ x 0 , t ∥ q ≤ C ρ t , ∥ x 1 , t ∥ q /2 ≤ C ρ t , ∥ x 2 , t ∥ q /3 ≤ C ρ t , t ≥ 1. (33)
(b) For any stationary sequence u t 2 ∈ L q u_t^2 \in L^q u t 2 ∈ L q (q > 4 q > 4 q > 4 ), define the stationary ergodic envelopes
U t : = C ∑ j = 0 ∞ ρ j ( 1 + u t − j 2 ) ∈ L q , G 1 , t : = C ∑ j = 0 ∞ ρ j U t − j 2 ∈ L q / 2 , U_t := C \sum_{j=0}^{\infty} \rho^j(1+u_{t-j}^2) \in L^q,\qquad G_{1,t} := C \sum_{j=0}^{\infty} \rho^j U_{t-j}^2 \in L^{q/2}, U t := C j = 0 ∑ ∞ ρ j ( 1 + u t − j 2 ) ∈ L q , G 1 , t := C j = 0 ∑ ∞ ρ j U t − j 2 ∈ L q /2 ,
G 2 , t : = C ∑ j = 0 ∞ ρ j { U t − j 3 + U t − j G 1 , t − j − 1 } ∈ L q / 3 . (34) G_{2,t} := C \sum_{j=0}^{\infty} \rho^j\{U_{t-j}^3+U_{t-j}G_{1,t-j-1}\} \in L^{q/3}. \tag*{(34)} G 2 , t := C j = 0 ∑ ∞ ρ j { U t − j 3 + U t − j G 1 , t − j − 1 } ∈ L q /3 . (34)
If x r , 0 = 0 x_{r,0}=0 x r , 0 = 0 , B t ≤ C U t B_t \le CU_t B t ≤ C U t , C t ≤ C ( U t 2 + G 1 , t − 1 ) C_t \le C(U_t^2+G_{1,t-1}) C t ≤ C ( U t 2 + G 1 , t − 1 ) , and for some nonnegative random variable Δ T \Delta_T Δ T , ξ 0 , t ≤ C Δ T ( 1 + u t 2 + u t − 1 2 ) \xi_{0,t} \le C\Delta_T(1+u_t^2+u_{t-1}^2) ξ 0 , t ≤ C Δ T ( 1 + u t 2 + u t − 1 2 ) , ξ 1 , t ≤ C Δ T U t 2 \xi_{1,t} \le C\Delta_TU_t^2 ξ 1 , t ≤ C Δ T U t 2 , and ξ 2 , t ≤ C Δ T ( U t 3 + U t G 1 , t − 1 ) \xi_{2,t} \le C\Delta_T(U_t^3+U_tG_{1,t-1}) ξ 2 , t ≤ C Δ T ( U t 3 + U t G 1 , t − 1 ) , then for t ≥ 1 t \ge1 t ≥ 1 ,
x 0 , t ≤ C Δ T U t , x 1 , t ≤ C Δ T G 1 , t , x 2 , t ≤ C Δ T G 2 , t . (35) x_{0,t} \le C\Delta_TU_t,\qquad x_{1,t} \le C\Delta_TG_{1,t},\qquad x_{2,t} \le C\Delta_TG_{2,t}. \tag*{(35)} x 0 , t ≤ C Δ T U t , x 1 , t ≤ C Δ T G 1 , t , x 2 , t ≤ C Δ T G 2 , t . (35)
(ii) Let X t = F ( ϵ t , ϵ t − 1 , … ) X_t=F(\epsilon_t,\epsilon_{t-1},\ldots) X t = F ( ϵ t , ϵ t − 1 , … ) be a stationary causal process with coupled version X t ∘ X_t^\circ X t ∘ (from Definition 1) and single-innovation coupled version X t ∗ = F ( ϵ t , … , ϵ 1 , ϵ 0 ∘ , ϵ − 1 , … ) X_t^*=F(\epsilon_t,\ldots,\epsilon_1,\epsilon_0^\circ,\epsilon_{-1},\ldots) X t ∗ = F ( ϵ t , … , ϵ 1 , ϵ 0 ∘ , ϵ − 1 , … ) .
(a) If ∥ X t ∥ a < ∞ \lVert X_t\rVert_a<\infty ∥ X t ∥ a < ∞ and ∥ X t − X t ∘ ∥ r ≤ C ρ t \lVert X_t-X_t^\circ\rVert_r\le C\rho^t ∥ X t − X t ∘ ∥ r ≤ C ρ t for some 1 < r < a 1<r<a 1 < r < a and ρ ∈ ( 0 , 1 ) \rho\in(0,1) ρ ∈ ( 0 , 1 ) , then for every p ∈ [ r , a ) p\in[r,a) p ∈ [ r , a ) ,
∥ X t − X t ∘ ∥ p ≤ C ρ p t , ∥ X t − X t ∗ ∥ p ≤ 2 ∥ X t − X t ∘ ∥ p ≤ 2 C ρ p t , (36) \lVert X_t-X_t^\circ\rVert_p\le C\rho_p^t,\qquad\lVert X_t-X_t^*\rVert_p\le2\lVert X_t-X_t^\circ\rVert_p\le2C\rho_p^t, \tag*{(36)} ∥ X t − X t ∘ ∥ p ≤ C ρ p t , ∥ X t − X t ∗ ∥ p ≤ 2 ∥ X t − X t ∘ ∥ p ≤ 2 C ρ p t , (36)
where ρ p : = ρ r ( a − p ) p ( a − r ) ∈ ( 0 , 1 ) \rho_p:=\rho^{\frac{r(a-p)}{p(a-r)}}\in(0,1) ρ p := ρ p ( a − r ) r ( a − p ) ∈ ( 0 , 1 ) .
(b) If E ( X t ) = 0 E(X_t)=0 E ( X t ) = 0 and { X t } \{X_t\} { X t } is L 2 L^2 L 2 GMC with decay rate ρ ∈ ( 0 , 1 ) \rho\in(0,1) ρ ∈ ( 0 , 1 ) , then ∥ Cov ( X 0 , X j ) ∥ ≤ ∥ X 0 ∥ 2 ∥ X j − X j ∘ ∥ 2 ≤ C ρ ∣ j ∣ \lVert\operatorname{Cov}(X_0,X_j)\rVert\le\lVert X_0\rVert_2\lVert X_j-X_j^\circ\rVert_2\le C\rho^{|j|} ∥ Cov ( X 0 , X j )∥ ≤ ∥ X 0 ∥ 2 ∥ X j − X j ∘ ∥ 2 ≤ C ρ ∣ j ∣ for all j ∈ Z j\in\mathbb{Z} j ∈ Z , the series Ω X : = ∑ j ∈ Z Cov ( X 0 , X j ) \Omega_X:=\sum_{j\in\mathbb{Z}}\operatorname{Cov}(X_0,X_j) Ω X := ∑ j ∈ Z Cov ( X 0 , X j ) converges absolutely, Var ( T − 1 / 2 ∑ t = 1 T X t ) → Ω X \operatorname{Var}(T^{-1/2}\sum_{t=1}^T X_t)\to\Omega_X Var ( T − 1/2 ∑ t = 1 T X t ) → Ω X , and n − 1 / 2 ∑ t = 1 ⌊ n r ⌋ X t ⇒ Ω X 1 / 2 B ( r ) n^{-1/2}\sum_{t=1}^{\lfloor nr\rfloor}X_t\Rightarrow\Omega_X^{1/2}B(r) n − 1/2 ∑ t = 1 ⌊ n r ⌋ X t ⇒ Ω X 1/2 B ( r ) on [ 0 , 1 ] [0,1] [ 0 , 1 ] .
Proof. For (i), iterating x 0 , t ≤ β ˉ x 0 , t − 1 + ξ 0 , t x_{0,t}\le\bar{\beta}x_{0,t-1}+\xi_{0,t} x 0 , t ≤ β ˉ x 0 , t − 1 + ξ 0 , t gives x 0 , t ≤ β ˉ t x 0 , 0 + ∑ j = 0 t − 1 β ˉ j ξ 0 , t − j x_{0,t}\le\bar{\beta}^t x_{0,0}+\sum_{j=0}^{t-1}\bar{\beta}^j\xi_{0,t-j} x 0 , t ≤ β ˉ t x 0 , 0 + ∑ j = 0 t − 1 β ˉ j ξ 0 , t − j .
Under (a), Minkowski’s inequality and β ˉ < ρ \bar{\beta}<\rho β ˉ < ρ give
∥ x 0 , t ∥ q ≤ β ˉ t ∥ x 0 , 0 ∥ q + C 0 ∑ j = 0 t − 1 β ˉ j ρ t − j ≤ C ρ t . \lVert x_{0,t}\rVert_q\le\bar{\beta}^t\lVert x_{0,0}\rVert_q+C_0\sum_{j=0}^{t-1}\bar{\beta}^j\rho^{t-j}\le C\rho^t. ∥ x 0 , t ∥ q ≤ β ˉ t ∥ x 0 , 0 ∥ q + C 0 j = 0 ∑ t − 1 β ˉ j ρ t − j ≤ C ρ t .
By Hölder’s inequality,
∥ B t ( x 0 , t − 1 + ξ 0 , t ) + ξ 1 , t ∥ q / 2 ≤ ∥ B t ∥ q ( ∥ x 0 , t − 1 ∥ q + ∥ ξ 0 , t ∥ q ) + ∥ ξ 1 , t ∥ q / 2 ≤ C ′ ρ t , \lVert B_t(x_{0,t-1}+\xi_{0,t})+\xi_{1,t}\rVert_{q/2}\le\lVert B_t\rVert_q(\lVert x_{0,t-1}\rVert_q+\lVert\xi_{0,t}\rVert_q)+\lVert\xi_{1,t}\rVert_{q/2}\le C'\rho^t, ∥ B t ( x 0 , t − 1 + ξ 0 , t ) + ξ 1 , t ∥ q /2 ≤ ∥ B t ∥ q (∥ x 0 , t − 1 ∥ q + ∥ ξ 0 , t ∥ q ) + ∥ ξ 1 , t ∥ q /2 ≤ C ′ ρ t ,
iterating x 1 , t x_{1,t} x 1 , t and using the fact that ∑ j = 0 t − 1 ( β ˉ / ρ ) j ≤ ( 1 − β ˉ / ρ ) − 1 \sum_{j=0}^{t-1}(\bar{\beta}/\rho)^j\le(1-\bar{\beta}/\rho)^{-1} ∑ j = 0 t − 1 ( β ˉ / ρ ) j ≤ ( 1 − β ˉ / ρ ) − 1 gives ∥ x 1 , t ∥ q / 2 ≤ C ρ t \lVert x_{1,t}\rVert_{q/2}\le C\rho^t ∥ x 1 , t ∥ q /2 ≤ C ρ t . The bound ∥ x 2 , t ∥ q / 3 ≤ C ρ t \lVert x_{2,t}\rVert_{q/3}\le C\rho^t ∥ x 2 , t ∥ q /3 ≤ C ρ t follows identically using ∥ B t x 1 , t − 1 ∥ q / 3 ≤ ∥ B t ∥ q ∥ x 1 , t − 1 ∥ q / 2 \lVert B_tx_{1,t-1}\rVert_{q/3}\le\lVert B_t\rVert_q\lVert x_{1,t-1}\rVert_{q/2} ∥ B t x 1 , t − 1 ∥ q /3 ≤ ∥ B t ∥ q ∥ x 1 , t − 1 ∥ q /2 and ∥ C t ( x 0 , t − 1 + ξ 0 , t ) ∥ q / 3 ≤ ∥ C t ∥ q / 2 ∥ x 0 , t − 1 + ξ 0 , t ∥ q \lVert C_t(x_{0,t-1}+\xi_{0,t})\rVert_{q/3}\le\lVert C_t\rVert_{q/2}\lVert x_{0,t-1}+\xi_{0,t}\rVert_q ∥ C t ( x 0 , t − 1 + ξ 0 , t ) ∥ q /3 ≤ ∥ C t ∥ q /2 ∥ x 0 , t − 1 + ξ 0 , t ∥ q .
Under (b), Minkowski’s and Hölder’s inequalities give U t ∈ L q U_t\in L^q U t ∈ L q , G 1 , t ∈ L q / 2 G_{1,t}\in L^{q/2} G 1 , t ∈ L q /2 , and G 2 , t ∈ L q / 3 G_{2,t}\in L^{q/3} G 2 , t ∈ L q /3 , and direct substitution of β ˉ ≤ ρ \bar{\beta}\le\rho β ˉ ≤ ρ and U t − 1 ≤ ρ − 1 U t U_{t-1}\le\rho^{-1}U_t U t − 1 ≤ ρ − 1 U t into the iterated sums yields x 0 , t ≤ C Δ T U t x_{0,t}\le C\Delta_TU_t x 0 , t ≤ C Δ T U t , x 1 , t ≤ C Δ T G 1 , t x_{1,t}\le C\Delta_TG_{1,t} x 1 , t ≤ C Δ T G 1 , t , and x 2 , t ≤ C Δ T G 2 , t x_{2,t}\le C\Delta_TG_{2,t} x 2 , t ≤ C Δ T G 2 , t .
For (ii)(a), let 1 / p = θ / r + ( 1 − θ ) / a 1/p=\theta/r+(1-\theta)/a 1/ p = θ / r + ( 1 − θ ) / a with θ = r ( a − p ) / { p ( a − r ) } ∈ ( 0 , 1 ] \theta=r(a-p)/\{p(a-r)\}\in(0,1] θ = r ( a − p ) / { p ( a − r )} ∈ ( 0 , 1 ] . Then, Hölder’s inequality gives ∥ X t − X t ∘ ∥ p ≤ ∥ X t − X t ∘ ∥ r θ ( ∥ X t ∥ a + ∥ X t ∘ ∥ a ) 1 − θ ≤ C ρ θ t \lVert X_t-X_t^\circ\rVert_p\le\lVert X_t-X_t^\circ\rVert_r^\theta(\lVert X_t\rVert_a+\lVert X_t^\circ\rVert_a)^{1-\theta}\le C\rho^{\theta t} ∥ X t − X t ∘ ∥ p ≤ ∥ X t − X t ∘ ∥ r θ (∥ X t ∥ a + ∥ X t ∘ ∥ a ) 1 − θ ≤ C ρ θ t ; see Remark 3.1 of Shao and Wu [32 ] . Since X t ∗ X_t^* X t ∗ and X t X_t X t share ϵ s \epsilon_s ϵ s for s ≠ 0 s\ne0 s = 0 , then X t − X t ∗ = ( X t − X t ∘ ) + ( X t ∘ − X t ∗ ) X_t-X_t^*=(X_t-X_t^\circ)+(X_t^\circ-X_t^*) X t − X t ∗ = ( X t − X t ∘ ) + ( X t ∘ − X t ∗ ) with ∥ X t − X t ∗ ∥ p ≤ ∥ X t − X t ∘ ∥ p + ∥ X t ∘ − X t ∗ ∥ p = ∥ X t − X t ∘ ∥ p + ∥ X t + 1 − X t + 1 ∘ ∥ p ≤ C ρ p t + C ρ p t + 1 \lVert X_t-X_t^*\rVert_p\le\lVert X_t-X_t^\circ\rVert_p+\lVert X_t^\circ-X_t^*\rVert_p=\lVert X_t-X_t^\circ\rVert_p+\lVert X_{t+1}-X_{t+1}^\circ\rVert_p\le C\rho_p^t+C\rho_p^{t+1} ∥ X t − X t ∗ ∥ p ≤ ∥ X t − X t ∘ ∥ p + ∥ X t ∘ − X t ∗ ∥ p = ∥ X t − X t ∘ ∥ p + ∥ X t + 1 − X t + 1 ∘ ∥ p ≤ C ρ p t + C ρ p t + 1 .
For (ii)(b), independence of X j ∘ X_j^\circ X j ∘ from X 0 X_0 X 0 for j ≥ 1 j\ge1 j ≥ 1 gives ∥ Cov ( X 0 , X j ) ∥ = ∥ E { X 0 ( X j − X j ∘ ) ⊤ } ∥ ≤ ∥ X 0 ∥ 2 ∥ X j − X j ∘ ∥ 2 ≤ C ρ j \lVert\operatorname{Cov}(X_0,X_j)\rVert=\lVert E\{X_0(X_j-X_j^\circ)^\top\}\rVert\le\lVert X_0\rVert_2\lVert X_j-X_j^\circ\rVert_2\le C\rho^j ∥ Cov ( X 0 , X j )∥ = ∥ E { X 0 ( X j − X j ∘ ) ⊤ }∥ ≤ ∥ X 0 ∥ 2 ∥ X j − X j ∘ ∥ 2 ≤ C ρ j , and stationarity covers j < 0 j<0 j < 0 . Moreover, the physical dependence coefficient of Wu [37 ] satisfies δ 2 ( j ) = ∥ X j − X j ∗ ∥ 2 ≤ 2 C ρ j \delta_2(j)=\lVert X_j-X_j^*\rVert_2\le2C\rho^j δ 2 ( j ) = ∥ X j − X j ∗ ∥ 2 ≤ 2 C ρ j , so Theorems 1(i) and 3(i) of Wu [37 ] yield the FCLT. ■
Lemma 2. Under Assumptions 1 and 6, there exist C < ∞ C < \infty C < ∞ and ρ ∈ ( β ˉ , 1 ) \rho\in(\bar{\beta},1) ρ ∈ ( β ˉ , 1 ) such that
1 + u t 2 + sup θ ∈ Θ { h t ( θ ) + ∥ ∂ h t ( θ ) ∂ θ ∥ } ≤ U t : = C ∑ j = 0 ∞ ρ j ( 1 + u t − j 2 ) ∈ L q , (37) 1+u_t^2+\sup_{\theta\in\Theta}\left\{h_t(\theta)+\left\|\frac{\partial h_t(\theta)}{\partial\theta}\right\|\right\}\le U_t:=C\sum_{j=0}^{\infty}\rho^j(1+u_{t-j}^2)\in L^q, \tag*{(37)} 1 + u t 2 + θ ∈ Θ sup { h t ( θ ) + ∂ θ ∂ h t ( θ ) } ≤ U t := C j = 0 ∑ ∞ ρ j ( 1 + u t − j 2 ) ∈ L q , (37) and
sup θ ∈ Θ ∥ ∂ 2 h t ( θ ) ∂ θ ∂ θ ⊤ ∥ ≤ G 1 , t : = C ∑ j = 0 ∞ ρ ˉ j U t − j 2 ∈ L q / 2 . (38) \sup_{\theta\in\Theta}\left\|\frac{\partial^2 h_t(\theta)}{\partial\theta\partial\theta^\top}\right\|\le G_{1,t}:=C\sum_{j=0}^{\infty}\bar{\rho}^jU_{t-j}^2\in L^{q/2}. \tag*{(38)} θ ∈ Θ sup ∂ θ ∂ θ ⊤ ∂ 2 h t ( θ ) ≤ G 1 , t := C j = 0 ∑ ∞ ρ ˉ j U t − j 2 ∈ L q /2 . (38) The score function satisfies that sup θ ∈ Θ ∥ s t ( θ ) ∥ ≤ C U t 2 ∈ L q / 2 \sup_{\theta\in\Theta}\|s_t(\theta)\|\le CU_t^2\in L^{q/2} sup θ ∈ Θ ∥ s t ( θ ) ∥ ≤ C U t 2 ∈ L q /2 .
Proof. Let e α = ( 1 , 0 , 0 ) ⊤ e_\alpha=(1,0,0)^\top e α = ( 1 , 0 , 0 ) ⊤ , e β = ( 0 , 1 , 0 ) ⊤ e_\beta=(0,1,0)^\top e β = ( 0 , 1 , 0 ) ⊤ , and e φ = ( 0 , 0 , 1 ) ⊤ e_\varphi=(0,0,1)^\top e φ = ( 0 , 0 , 1 ) ⊤ be the standard basis vectors. All derivatives with respect to θ \theta θ hold the observations { u t 2 } \{u_t^2\} { u t 2 } fixed. Since b t − 1 = ω + α u t − 1 2 + β h t − 1 b_{t-1}=\omega+\alpha u_{t-1}^2+\beta h_{t-1} b t − 1 = ω + α u t − 1 2 + β h t − 1 , we have
∂ b t − 1 ∂ θ = ∂ ω ∂ θ + e α u t − 1 2 + e β h t − 1 + β ∂ h t − 1 ∂ θ , (39) \frac{\partial b_{t-1}}{\partial\theta}=\frac{\partial\omega}{\partial\theta}+e_\alpha u_{t-1}^2+e_\beta h_{t-1}+\beta\frac{\partial h_{t-1}}{\partial\theta}, \tag*{(39)} ∂ θ ∂ b t − 1 = ∂ θ ∂ ω + e α u t − 1 2 + e β h t − 1 + β ∂ θ ∂ h t − 1 , (39)
∂ 2 b t − 1 ∂ θ ∂ θ ⊤ = ∂ 2 ω ∂ θ ∂ θ ⊤ + e β ∂ h t − 1 ∂ θ ⊤ + ∂ h t − 1 ∂ θ e β ⊤ + β ∂ 2 h t − 1 ∂ θ ∂ θ ⊤ . (40) \frac{\partial^2b_{t-1}}{\partial\theta\partial\theta^\top}=\frac{\partial^2\omega}{\partial\theta\partial\theta^\top}+e_\beta\frac{\partial h_{t-1}}{\partial\theta^\top}+\frac{\partial h_{t-1}}{\partial\theta}e_\beta^\top+\beta\frac{\partial^2h_{t-1}}{\partial\theta\partial\theta^\top}. \tag*{(40)} ∂ θ ∂ θ ⊤ ∂ 2 b t − 1 = ∂ θ ∂ θ ⊤ ∂ 2 ω + e β ∂ θ ⊤ ∂ h t − 1 + ∂ θ ∂ h t − 1 e β ⊤ + β ∂ θ ∂ θ ⊤ ∂ 2 h t − 1 . (40)
Applying the chain rule to h t = g ( b t − 1 , u t 2 ; φ ) h_t=g(b_{t-1},u_t^2;\varphi) h t = g ( b t − 1 , u t 2 ; φ ) gives
∂ h t ∂ θ = ∂ g ∂ b ∂ b t − 1 ∂ θ + ∂ g ∂ φ e φ , (41) \frac{\partial h_t}{\partial\theta}=\frac{\partial g}{\partial b}\frac{\partial b_{t-1}}{\partial\theta}+\frac{\partial g}{\partial\varphi}e_\varphi, \tag*{(41)} ∂ θ ∂ h t = ∂ b ∂ g ∂ θ ∂ b t − 1 + ∂ φ ∂ g e φ , (41)
∂ 2 h t ∂ θ ∂ θ ⊤ = ∂ g ∂ b ∂ 2 b t − 1 ∂ θ ∂ θ ⊤ + ∂ 2 g ∂ b 2 ∂ b t − 1 ∂ θ ∂ b t − 1 ∂ θ ⊤ + ∂ 2 g ∂ b ∂ φ { ∂ b t − 1 ∂ θ e φ ⊤ + e φ ∂ b t − 1 ∂ θ ⊤ } + ∂ 2 g ∂ φ 2 e φ e φ ⊤ . (42) \frac{\partial^2h_t}{\partial\theta\partial\theta^\top}=\frac{\partial g}{\partial b}\frac{\partial^2b_{t-1}}{\partial\theta\partial\theta^\top}+\frac{\partial^2g}{\partial b^2}\frac{\partial b_{t-1}}{\partial\theta}\frac{\partial b_{t-1}}{\partial\theta^\top}+\frac{\partial^2g}{\partial b\partial\varphi}\left\{\frac{\partial b_{t-1}}{\partial\theta}e_\varphi^\top+e_\varphi\frac{\partial b_{t-1}}{\partial\theta^\top}\right\}+\frac{\partial^2g}{\partial\varphi^2}e_\varphi e_\varphi^\top. \tag*{(42)} ∂ θ ∂ θ ⊤ ∂ 2 h t = ∂ b ∂ g ∂ θ ∂ θ ⊤ ∂ 2 b t − 1 + ∂ b 2 ∂ 2 g ∂ θ ∂ b t − 1 ∂ θ ⊤ ∂ b t − 1 + ∂ b ∂ φ ∂ 2 g { ∂ θ ∂ b t − 1 e φ ⊤ + e φ ∂ θ ⊤ ∂ b t − 1 } + ∂ φ 2 ∂ 2 g e φ e φ ⊤ . (42)
Substituting (A.24) into (A.26) and using β ( ∂ g / ∂ b ) ≤ β ˉ \beta(\partial g/\partial b)\le\bar{\beta} β ( ∂ g / ∂ b ) ≤ β ˉ from (A.2) together with (A.3) and (A.4), we obtain
sup θ ∈ Θ h t ( θ ) ≤ β ˉ sup θ ∈ Θ h t − 1 ( θ ) + C ( 1 + u t 2 + u t − 1 2 ) , \sup_{\theta\in\Theta}h_t(\theta)\le\bar{\beta}\sup_{\theta\in\Theta}h_{t-1}(\theta)+C(1+u_t^2+u_{t-1}^2), θ ∈ Θ sup h t ( θ ) ≤ β ˉ θ ∈ Θ sup h t − 1 ( θ ) + C ( 1 + u t 2 + u t − 1 2 ) ,
sup θ ∈ Θ ∥ ∂ h t ( θ ) ∂ θ ∥ ≤ β ˉ sup θ ∈ Θ ∥ ∂ h t − 1 ( θ ) ∂ θ ∥ + C { 1 + u t 2 + u t − 1 2 + sup θ ∈ Θ h t − 1 ( θ ) } . \sup_{\theta\in\Theta}\left\|\frac{\partial h_t(\theta)}{\partial\theta}\right\|\le\bar{\beta}\sup_{\theta\in\Theta}\left\|\frac{\partial h_{t-1}(\theta)}{\partial\theta}\right\|+C\left\{1+u_t^2+u_{t-1}^2+\sup_{\theta\in\Theta}h_{t-1}(\theta)\right\}. θ ∈ Θ sup ∂ θ ∂ h t ( θ ) ≤ β ˉ θ ∈ Θ sup ∂ θ ∂ h t − 1 ( θ ) + C { 1 + u t 2 + u t − 1 2 + θ ∈ Θ sup h t − 1 ( θ ) } .
Therefore, applying Lemma 1(i)(b) with Δ T = 1 \Delta_T=1 Δ T = 1 proves (A.22), which in turn implies that
sup θ ∈ Θ ∥ ∂ b t − 1 ( θ ) ∂ θ ∥ ≤ C ( 1 + u t − 1 2 + sup θ ∈ Θ { h t − 1 ( θ ) + ∥ ∂ θ h t − 1 ( θ ) ∥ } ) ≤ C U t . \sup_{\theta\in\Theta}\left\|\frac{\partial b_{t-1}(\theta)}{\partial\theta}\right\|\le C\left(1+u_{t-1}^2+\sup_{\theta\in\Theta}\{h_{t-1}(\theta)+\|\partial_\theta h_{t-1}(\theta)\|\}\right)\le CU_t. θ ∈ Θ sup ∂ θ ∂ b t − 1 ( θ ) ≤ C ( 1 + u t − 1 2 + θ ∈ Θ sup { h t − 1 ( θ ) + ∥ ∂ θ h t − 1 ( θ ) ∥ } ) ≤ C U t .
Substituting (A.25) into (A.27) and bounding the second-order derivatives of g g g via (A.5) yields
sup θ ∈ Θ ∥ ∂ 2 h t ( θ ) ∂ θ ∂ θ ⊤ ∥ ≤ β ˉ sup θ ∈ Θ ∥ ∂ 2 h t − 1 ( θ ) ∂ θ ∂ θ ⊤ ∥ + C U t 2 , \sup_{\theta\in\Theta}\left\|\frac{\partial^2h_t(\theta)}{\partial\theta\partial\theta^\top}\right\|\le\bar{\beta}\sup_{\theta\in\Theta}\left\|\frac{\partial^2h_{t-1}(\theta)}{\partial\theta\partial\theta^\top}\right\|+CU_t^2, θ ∈ Θ sup ∂ θ ∂ θ ⊤ ∂ 2 h t ( θ ) ≤ β ˉ θ ∈ Θ sup ∂ θ ∂ θ ⊤ ∂ 2 h t − 1 ( θ ) + C U t 2 ,
and Lemma 1(i)(b) gives (A.23).
Finally, differentiating ℓ t ( θ ) = f ( u t 2 , h t ( θ ) , φ ) \ell_t(\theta)=f(u_t^2,h_t(\theta),\varphi) ℓ t ( θ ) = f ( u t 2 , h t ( θ ) , φ ) gives the score and Hessian matrix as
s t ( θ ) = ∂ ℓ t ( θ ) ∂ θ = ∂ f ∂ h ∂ h t ∂ θ + ∂ f ∂ φ e φ . (43) s_t(\theta)=\frac{\partial\ell_t(\theta)}{\partial\theta}=\frac{\partial f}{\partial h}\frac{\partial h_t}{\partial\theta}+\frac{\partial f}{\partial\varphi}e_\varphi. \tag*{(43)} s t ( θ ) = ∂ θ ∂ ℓ t ( θ ) = ∂ h ∂ f ∂ θ ∂ h t + ∂ φ ∂ f e φ . (43)
∂ 2 ℓ t ( θ ) ∂ θ ∂ θ ⊤ = ∂ f ∂ h ∂ 2 h t ∂ θ ∂ θ ⊤ + ∂ 2 f ∂ h 2 ∂ h t ∂ θ ∂ h t ∂ θ ⊤ + ∂ 2 f ∂ h ∂ φ { ∂ h t ∂ θ e φ ⊤ + e φ ∂ h t ∂ θ ⊤ } + ∂ 2 f ∂ φ 2 e φ e φ ⊤ . (44) \frac{\partial^2 \ell_t(\theta)}{\partial\theta\partial\theta^\top}
=
\frac{\partial f}{\partial h}\frac{\partial^2 h_t}{\partial\theta\partial\theta^\top}
+
\frac{\partial^2 f}{\partial h^2}\frac{\partial h_t}{\partial\theta}\frac{\partial h_t}{\partial\theta^\top}
+
\frac{\partial^2 f}{\partial h\partial\varphi}
\left\{
\frac{\partial h_t}{\partial\theta}e_\varphi^\top
+
e_\varphi\frac{\partial h_t}{\partial\theta^\top}
\right\}
+
\frac{\partial^2 f}{\partial\varphi^2}e_\varphi e_\varphi^\top. \tag*{(44)} ∂ θ ∂ θ ⊤ ∂ 2 ℓ t ( θ ) = ∂ h ∂ f ∂ θ ∂ θ ⊤ ∂ 2 h t + ∂ h 2 ∂ 2 f ∂ θ ∂ h t ∂ θ ⊤ ∂ h t + ∂ h ∂ φ ∂ 2 f { ∂ θ ∂ h t e φ ⊤ + e φ ∂ θ ⊤ ∂ h t } + ∂ φ 2 ∂ 2 f e φ e φ ⊤ . (44)
Combining (A.28) with (A.9) and (A.22), we obtain
sup θ ∈ Θ ∥ s t ( θ ) ∥ ≤ C ( 1 + u t 2 ) U t ≤ C U t 2 ∈ L q / 2 . \sup_{\theta\in\Theta}\lVert s_t(\theta)\rVert\le C(1+u_t^2)U_t\le CU_t^2\in L^{q/2}. θ ∈ Θ sup ∥ s t ( θ )∥ ≤ C ( 1 + u t 2 ) U t ≤ C U t 2 ∈ L q /2 .
Lemma 3. For any nonnegative input sequence v = { v t } t ≥ 0 \mathbf{v}=\{v_t\}_{t\ge0} v = { v t } t ≥ 0 and initial state h 0 ( θ ) > 0 h_0(\theta)>0 h 0 ( θ ) > 0 , let h t ( θ ; v , h 0 ) h_t(\theta;\mathbf{v},h_0) h t ( θ ; v , h 0 ) be generated by b t − 1 ( θ ) = ω ( θ ) + α v t − 1 + β h t − 1 ( θ ) b_{t-1}(\theta)=\omega(\theta)+\alpha v_{t-1}+\beta h_{t-1}(\theta) b t − 1 ( θ ) = ω ( θ ) + α v t − 1 + β h t − 1 ( θ ) and h t ( θ ) = g ( b t − 1 ( θ ) , v t ; φ ) h_t(\theta)=g(b_{t-1}(\theta),v_t;\varphi) h t ( θ ) = g ( b t − 1 ( θ ) , v t ; φ ) , and let ℓ t ( θ ; v , h 0 ) : = f ( v t , h t ( θ ; v , h 0 ) , φ ) \ell_t(\theta;\mathbf{v},h_0):=f(v_t,h_t(\theta;\mathbf{v},h_0),\varphi) ℓ t ( θ ; v , h 0 ) := f ( v t , h t ( θ ; v , h 0 ) , φ ) . Given two input sequences v ( 1 ) , v ( 2 ) \mathbf{v}^{(1)},\mathbf{v}^{(2)} v ( 1 ) , v ( 2 ) satisfying 0 ≤ v t ( m ) ≤ 2 ( u t ( m ) ) 2 0\le v_t^{(m)}\le2(u_t^{(m)})^2 0 ≤ v t ( m ) ≤ 2 ( u t ( m ) ) 2 (where u t ( m ) u_t^{(m)} u t ( m ) has envelope U t ( m ) U_t^{(m)} U t ( m ) and G 1 , t ( m ) G_{1,t}^{(m)} G 1 , t ( m ) as in (A.19)) and initial states h 0 ( m ) h_0^{(m)} h 0 ( m ) for m = 1 , 2 m=1,2 m = 1 , 2 . Write h t ( m ) ( θ ) : = h t ( θ ; v ( m ) , h 0 ( m ) ) h_t^{(m)}(\theta):=h_t(\theta;\mathbf{v}^{(m)},h_0^{(m)}) h t ( m ) ( θ ) := h t ( θ ; v ( m ) , h 0 ( m ) ) and ℓ t ( m ) ( θ ) : = ℓ t ( θ ; v ( m ) , h 0 ( m ) ) \ell_t^{(m)}(\theta):=\ell_t(\theta;\mathbf{v}^{(m)},h_0^{(m)}) ℓ t ( m ) ( θ ) := ℓ t ( θ ; v ( m ) , h 0 ( m ) ) for m = 1 , 2 m=1,2 m = 1 , 2 , and define
Δ v t : = ∣ v t ( 1 ) − v t ( 2 ) ∣ , Δ 0 h t : = sup θ ∈ Θ ∣ h t ( 1 ) ( θ ) − h t ( 2 ) ( θ ) ∣ , \Delta v_t:=|v_t^{(1)}-v_t^{(2)}|,\qquad
\Delta_0h_t:=\sup_{\theta\in\Theta}|h_t^{(1)}(\theta)-h_t^{(2)}(\theta)|, Δ v t := ∣ v t ( 1 ) − v t ( 2 ) ∣ , Δ 0 h t := θ ∈ Θ sup ∣ h t ( 1 ) ( θ ) − h t ( 2 ) ( θ ) ∣ , Δ 1 h t : = sup θ ∈ Θ ∥ ∂ h t ( 1 ) ( θ ) ∂ θ − ∂ h t ( 2 ) ( θ ) ∂ θ ∥ , Δ 2 h t : = sup θ ∈ Θ ∥ ∂ 2 h t ( 1 ) ( θ ) ∂ θ ∂ θ ⊤ − ∂ 2 h t ( 2 ) ( θ ) ∂ θ ∂ θ ⊤ ∥ . (45) \Delta_1h_t:=\sup_{\theta\in\Theta}
\left\|
\frac{\partial h_t^{(1)}(\theta)}{\partial\theta}
-
\frac{\partial h_t^{(2)}(\theta)}{\partial\theta}
\right\|,
\qquad
\Delta_2h_t:=\sup_{\theta\in\Theta}
\left\|
\frac{\partial^2 h_t^{(1)}(\theta)}{\partial\theta\partial\theta^\top}
-
\frac{\partial^2 h_t^{(2)}(\theta)}{\partial\theta\partial\theta^\top}
\right\|. \tag*{(45)} Δ 1 h t := θ ∈ Θ sup ∂ θ ∂ h t ( 1 ) ( θ ) − ∂ θ ∂ h t ( 2 ) ( θ ) , Δ 2 h t := θ ∈ Θ sup ∂ θ ∂ θ ⊤ ∂ 2 h t ( 1 ) ( θ ) − ∂ θ ∂ θ ⊤ ∂ 2 h t ( 2 ) ( θ ) . (45) Let B t : = C ( U t ( 1 ) + U t ( 2 ) ) B_t:=C(U_t^{(1)}+U_t^{(2)}) B t := C ( U t ( 1 ) + U t ( 2 ) ) and C t : = C { B t 2 + G 1 , t − 1 ( 1 ) + G 1 , t − 1 ( 2 ) } C_t:=C\{B_t^2+G_{1,t-1}^{(1)}+G_{1,t-1}^{(2)}\} C t := C { B t 2 + G 1 , t − 1 ( 1 ) + G 1 , t − 1 ( 2 ) } . Then for all t ≥ 1 t\ge1 t ≥ 1 :
(i) The process differences { Δ 0 h t , Δ 1 h t , Δ 2 h t } \{\Delta_0h_t,\Delta_1h_t,\Delta_2h_t\} { Δ 0 h t , Δ 1 h t , Δ 2 h t } satisfy the contractive triangular recursions
Δ 0 h t ≤ β ˉ Δ 0 h t − 1 + C ( Δ v t + Δ v t − 1 ) , \Delta_0h_t\le\bar{\beta}\Delta_0h_{t-1}+C(\Delta v_t+\Delta v_{t-1}), Δ 0 h t ≤ β ˉ Δ 0 h t − 1 + C ( Δ v t + Δ v t − 1 ) , Δ 1 h t ≤ β ˉ Δ 1 h t − 1 + C B t ( Δ 0 h t − 1 + Δ v t + Δ v t − 1 ) , \Delta_1h_t\le\bar{\beta}\Delta_1h_{t-1}+CB_t(\Delta_0h_{t-1}+\Delta v_t+\Delta v_{t-1}), Δ 1 h t ≤ β ˉ Δ 1 h t − 1 + C B t ( Δ 0 h t − 1 + Δ v t + Δ v t − 1 ) , Δ 2 h t ≤ β ˉ Δ 2 h t − 1 + C B t Δ 1 h t − 1 + C C t ( Δ 0 h t − 1 + Δ v t + Δ v t − 1 ) . (46) \Delta_2h_t\le\bar{\beta}\Delta_2h_{t-1}+CB_t\Delta_1h_{t-1}+CC_t(\Delta_0h_{t-1}+\Delta v_t+\Delta v_{t-1}). \tag*{(46)} Δ 2 h t ≤ β ˉ Δ 2 h t − 1 + C B t Δ 1 h t − 1 + C C t ( Δ 0 h t − 1 + Δ v t + Δ v t − 1 ) . (46) (ii) The log-likelihood, score, and Hessian differences satisfy
sup θ ∈ Θ ∣ ℓ t ( 1 ) ( θ ) − ℓ t ( 2 ) ( θ ) ∣ ≤ C Δ v t + C B t Δ 0 h t , \sup_{\theta\in\Theta}|\ell_t^{(1)}(\theta)-\ell_t^{(2)}(\theta)|
\le C\Delta v_t+CB_t\Delta_0h_t, θ ∈ Θ sup ∣ ℓ t ( 1 ) ( θ ) − ℓ t ( 2 ) ( θ ) ∣ ≤ C Δ v t + C B t Δ 0 h t , sup θ ∈ Θ ∥ ∂ ℓ t ( 1 ) ( θ ) ∂ θ − ∂ ℓ t ( 2 ) ( θ ) ∂ θ ∥ ≤ C B t 2 ( Δ v t + Δ 0 h t ) + C B t Δ 1 h t , \sup_{\theta\in\Theta}
\left\|
\frac{\partial\ell_t^{(1)}(\theta)}{\partial\theta}
-
\frac{\partial\ell_t^{(2)}(\theta)}{\partial\theta}
\right\|
\le CB_t^2(\Delta v_t+\Delta_0h_t)+CB_t\Delta_1h_t, θ ∈ Θ sup ∂ θ ∂ ℓ t ( 1 ) ( θ ) − ∂ θ ∂ ℓ t ( 2 ) ( θ ) ≤ C B t 2 ( Δ v t + Δ 0 h t ) + C B t Δ 1 h t , sup θ ∈ Θ ∥ ∂ 2 ℓ t ( 1 ) ( θ ) ∂ θ ∂ θ ⊤ − ∂ 2 ℓ t ( 2 ) ( θ ) ∂ θ ∂ θ ⊤ ∥ ≤ C ( B t C t + B t 3 ) ( Δ v t + Δ 0 h t ) + C B t 2 Δ 1 h t + C B t Δ 2 h t . (47) \sup_{\theta\in\Theta}
\left\|
\frac{\partial^2\ell_t^{(1)}(\theta)}{\partial\theta\partial\theta^\top}
-
\frac{\partial^2\ell_t^{(2)}(\theta)}{\partial\theta\partial\theta^\top}
\right\|
\le C(B_tC_t+B_t^3)(\Delta v_t+\Delta_0h_t)+CB_t^2\Delta_1h_t+CB_t\Delta_2h_t. \tag*{(47)} θ ∈ Θ sup ∂ θ ∂ θ ⊤ ∂ 2 ℓ t ( 1 ) ( θ ) − ∂ θ ∂ θ ⊤ ∂ 2 ℓ t ( 2 ) ( θ ) ≤ C ( B t C t + B t 3 ) ( Δ v t + Δ 0 h t ) + C B t 2 Δ 1 h t + C B t Δ 2 h t . (47)
Proof. Write g ( m ) g^{(m)} g ( m ) and f ( m ) f^{(m)} f ( m ) for g g g and f f f evaluated at ( b t − 1 ( m ) ( θ ) , v t ( m ) ; φ ) (b_{t-1}^{(m)}(\theta),v_t^{(m)};\varphi) ( b t − 1 ( m ) ( θ ) , v t ( m ) ; φ ) and ( v t ( m ) , h t ( m ) ( θ ) , φ ) (v_t^{(m)},h_t^{(m)}(\theta),\varphi) ( v t ( m ) , h t ( m ) ( θ ) , φ ) , respectively. By (A.24)–(A.25), the differences of b t − 1 ( 1 ) ( θ ) b_{t-1}^{(1)}(\theta) b t − 1 ( 1 ) ( θ ) and b t − 1 ( 2 ) ( θ ) b_{t-1}^{(2)}(\theta) b t − 1 ( 2 ) ( θ ) satisfy
sup θ ∈ Θ ∣ b t − 1 ( 1 ) ( θ ) − b t − 1 ( 2 ) ( θ ) ∣ ≤ α Δ v t − 1 + β ˉ Δ 0 h t − 1 , \sup_{\theta\in\Theta}|b_{t-1}^{(1)}(\theta)-b_{t-1}^{(2)}(\theta)|
\le\alpha\Delta v_{t-1}+\bar{\beta}\Delta_0h_{t-1}, θ ∈ Θ sup ∣ b t − 1 ( 1 ) ( θ ) − b t − 1 ( 2 ) ( θ ) ∣ ≤ α Δ v t − 1 + β ˉ Δ 0 h t − 1 ,
sup θ ∈ Θ ∥ ∂ b t − 1 ( 1 ) ( θ ) ∂ θ − ∂ b t − 1 ( 2 ) ( θ ) ∂ θ ∥ ≤ Δ v t − 1 + Δ 0 h t − 1 + β ˉ Δ 1 h t − 1 , \sup_{\theta\in\Theta}
\left\|
\frac{\partial b_{t-1}^{(1)}(\theta)}{\partial\theta}
-
\frac{\partial b_{t-1}^{(2)}(\theta)}{\partial\theta}
\right\|
\le\Delta v_{t-1}+\Delta_0h_{t-1}+\bar{\beta}\Delta_1h_{t-1}, θ ∈ Θ sup ∂ θ ∂ b t − 1 ( 1 ) ( θ ) − ∂ θ ∂ b t − 1 ( 2 ) ( θ ) ≤ Δ v t − 1 + Δ 0 h t − 1 + β ˉ Δ 1 h t − 1 ,
sup θ ∈ Θ ∥ ∂ 2 b t − 1 ( 1 ) ( θ ) ∂ θ ∂ θ ⊤ − ∂ 2 b t − 1 ( 2 ) ( θ ) ∂ θ ∂ θ ⊤ ∥ ≤ 2 Δ 1 h t − 1 + β ˉ Δ 2 h t − 1 . (48) \sup_{\theta\in\Theta}
\left\|
\frac{\partial^2 b_{t-1}^{(1)}(\theta)}{\partial\theta\partial\theta^\top}
-
\frac{\partial^2 b_{t-1}^{(2)}(\theta)}{\partial\theta\partial\theta^\top}
\right\|
\le2\Delta_1h_{t-1}+\bar{\beta}\Delta_2h_{t-1}. \tag*{(48)} θ ∈ Θ sup ∂ θ ∂ θ ⊤ ∂ 2 b t − 1 ( 1 ) ( θ ) − ∂ θ ∂ θ ⊤ ∂ 2 b t − 1 ( 2 ) ( θ ) ≤ 2 Δ 1 h t − 1 + β ˉ Δ 2 h t − 1 . (48)
Applying (A.2)–(A.3) to h t ( 1 ) ( θ ) − h t ( 2 ) ( θ ) = g ( b t − 1 ( 1 ) , v t ( 1 ) ; φ ) − g ( b t − 1 ( 2 ) , v t ( 2 ) ; φ ) h_t^{(1)}(\theta)-h_t^{(2)}(\theta)=g(b_{t-1}^{(1)},v_t^{(1)};\varphi)-g(b_{t-1}^{(2)},v_t^{(2)};\varphi) h t ( 1 ) ( θ ) − h t ( 2 ) ( θ ) = g ( b t − 1 ( 1 ) , v t ( 1 ) ; φ ) − g ( b t − 1 ( 2 ) , v t ( 2 ) ; φ ) gives the first inequality in (A.31). Next, by (A.24) and (A.26), subtracting the first-derivative recursions gives
∂ h t ( 1 ) ∂ θ − ∂ h t ( 2 ) ∂ θ = β ∂ g ( 1 ) ∂ b ( ∂ h t − 1 ( 1 ) ∂ θ − ∂ h t − 1 ( 2 ) ∂ θ ) + ∂ g ( 1 ) ∂ b { e α ( v t − 1 ( 1 ) − v t − 1 ( 2 ) ) + e β ( h t − 1 ( 1 ) − h t − 1 ( 2 ) ) } + ( ∂ g ( 1 ) ∂ b − ∂ g ( 2 ) ∂ b ) ∂ b t − 1 ( 2 ) ∂ θ + ( ∂ g ( 1 ) ∂ φ − ∂ g ( 2 ) ∂ φ ) e φ . \begin{aligned}
\frac{\partial h_t^{(1)}}{\partial\theta}-\frac{\partial h_t^{(2)}}{\partial\theta}=\beta\frac{\partial g^{(1)}}{\partial b}\left(\frac{\partial h_{t-1}^{(1)}}{\partial\theta}-\frac{\partial h_{t-1}^{(2)}}{\partial\theta}\right)+\frac{\partial g^{(1)}}{\partial b}\left\{e_\alpha\left(v_{t-1}^{(1)}-v_{t-1}^{(2)}\right)+e_\beta\left(h_{t-1}^{(1)}-h_{t-1}^{(2)}\right)\right\} \\
&+\left(\frac{\partial g^{(1)}}{\partial b}-\frac{\partial g^{(2)}}{\partial b}\right)\frac{\partial b_{t-1}^{(2)}}{\partial\theta}+\left(\frac{\partial g^{(1)}}{\partial\varphi}-\frac{\partial g^{(2)}}{\partial\varphi}\right)e_\varphi.
\end{aligned} ∂ θ ∂ h t ( 1 ) − ∂ θ ∂ h t ( 2 ) = β ∂ b ∂ g ( 1 ) ( ∂ θ ∂ h t − 1 ( 1 ) − ∂ θ ∂ h t − 1 ( 2 ) ) + ∂ b ∂ g ( 1 ) { e α ( v t − 1 ( 1 ) − v t − 1 ( 2 ) ) + e β ( h t − 1 ( 1 ) − h t − 1 ( 2 ) ) } + ( ∂ b ∂ g ( 1 ) − ∂ b ∂ g ( 2 ) ) ∂ θ ∂ b t − 1 ( 2 ) + ( ∂ φ ∂ g ( 1 ) − ∂ φ ∂ g ( 2 ) ) e φ .
Bounding the partial derivatives of g g g and their differences via (A.2) and (A.7), and using sup θ ∈ Θ ∥ ∂ θ b t − 1 ( 2 ) ( θ ) ∥ ≤ C B t \sup_{\theta\in\Theta}\|\partial_\theta b_{t-1}^{(2)}(\theta)\|\le CB_t sup θ ∈ Θ ∥ ∂ θ b t − 1 ( 2 ) ( θ ) ∥ ≤ C B t and 1 + v t − 1 ( 1 ) + v t − 1 ( 2 ) ≤ C B t 1+v_{t-1}^{(1)}+v_{t-1}^{(2)}\le CB_t 1 + v t − 1 ( 1 ) + v t − 1 ( 2 ) ≤ C B t , yields the second inequality in (A.31). Similarly, subtracting (A.27) for m = 1 , 2 m=1,2 m = 1 , 2 gives
∂ 2 h t ( 1 ) ∂ θ ∂ θ ⊤ − ∂ 2 h t ( 2 ) ∂ θ ∂ θ ⊤ = ∂ g ( 1 ) ∂ b ( ∂ 2 b t − 1 ( 1 ) ∂ θ ∂ θ ⊤ − ∂ 2 b t − 1 ( 2 ) ∂ θ ∂ θ ⊤ ) + ( ∂ g ( 1 ) ∂ b − ∂ g ( 2 ) ∂ b ) ∂ 2 b t − 1 ( 2 ) ∂ θ ∂ θ ⊤ + ∂ 2 g ( 1 ) ∂ b 2 ( ∂ b t − 1 ( 1 ) ∂ θ ∂ b t − 1 ( 1 ) ∂ θ ⊤ − ∂ b t − 1 ( 2 ) ∂ θ ∂ b t − 1 ( 2 ) ∂ θ ⊤ ) + ( ∂ 2 g ( 1 ) ∂ b 2 − ∂ 2 g ( 2 ) ∂ b 2 ) ∂ b t − 1 ( 2 ) ∂ θ ∂ b t − 1 ( 2 ) ∂ θ ⊤ + ∂ 2 g ( 1 ) ∂ b ∂ φ { ( ∂ b t − 1 ( 1 ) ∂ θ − ∂ b t − 1 ( 2 ) ∂ θ ) e φ ⊤ + e φ ( ∂ b t − 1 ( 1 ) ∂ θ ⊤ − ∂ b t − 1 ( 2 ) ∂ θ ⊤ ) } + ( ∂ 2 g ( 1 ) ∂ b ∂ φ − ∂ 2 g ( 2 ) ∂ b ∂ φ ) ( ∂ b t − 1 ( 2 ) ∂ θ e φ ⊤ + e φ ∂ b t − 1 ( 2 ) ∂ θ ⊤ ) + ( ∂ 2 g ( 1 ) ∂ φ 2 − ∂ 2 g ( 2 ) ∂ φ 2 ) e φ e φ ⊤ . \begin{aligned}
\frac{\partial^2h_t^{(1)}}{\partial\theta\partial\theta^\top}-\frac{\partial^2h_t^{(2)}}{\partial\theta\partial\theta^\top}=\frac{\partial g^{(1)}}{\partial b}\left(\frac{\partial^2b_{t-1}^{(1)}}{\partial\theta\partial\theta^\top}-\frac{\partial^2b_{t-1}^{(2)}}{\partial\theta\partial\theta^\top}\right)+\left(\frac{\partial g^{(1)}}{\partial b}-\frac{\partial g^{(2)}}{\partial b}\right)\frac{\partial^2b_{t-1}^{(2)}}{\partial\theta\partial\theta^\top} \\
&+\frac{\partial^2g^{(1)}}{\partial b^2}\left(\frac{\partial b_{t-1}^{(1)}}{\partial\theta}\frac{\partial b_{t-1}^{(1)}}{\partial\theta^\top}-\frac{\partial b_{t-1}^{(2)}}{\partial\theta}\frac{\partial b_{t-1}^{(2)}}{\partial\theta^\top}\right)+\left(\frac{\partial^2g^{(1)}}{\partial b^2}-\frac{\partial^2g^{(2)}}{\partial b^2}\right)\frac{\partial b_{t-1}^{(2)}}{\partial\theta}\frac{\partial b_{t-1}^{(2)}}{\partial\theta^\top} \\
&+\frac{\partial^2g^{(1)}}{\partial b\partial\varphi}\left\{\left(\frac{\partial b_{t-1}^{(1)}}{\partial\theta}-\frac{\partial b_{t-1}^{(2)}}{\partial\theta}\right)e_\varphi^\top+e_\varphi\left(\frac{\partial b_{t-1}^{(1)}}{\partial\theta^\top}-\frac{\partial b_{t-1}^{(2)}}{\partial\theta^\top}\right)\right\} \\
&+\left(\frac{\partial^2g^{(1)}}{\partial b\partial\varphi}-\frac{\partial^2g^{(2)}}{\partial b\partial\varphi}\right)\left(\frac{\partial b_{t-1}^{(2)}}{\partial\theta}e_\varphi^\top+e_\varphi\frac{\partial b_{t-1}^{(2)}}{\partial\theta^\top}\right)+\left(\frac{\partial^2g^{(1)}}{\partial\varphi^2}-\frac{\partial^2g^{(2)}}{\partial\varphi^2}\right)e_\varphi e_\varphi^\top.
\end{aligned} ∂ θ ∂ θ ⊤ ∂ 2 h t ( 1 ) − ∂ θ ∂ θ ⊤ ∂ 2 h t ( 2 ) = ∂ b ∂ g ( 1 ) ( ∂ θ ∂ θ ⊤ ∂ 2 b t − 1 ( 1 ) − ∂ θ ∂ θ ⊤ ∂ 2 b t − 1 ( 2 ) ) + ( ∂ b ∂ g ( 1 ) − ∂ b ∂ g ( 2 ) ) ∂ θ ∂ θ ⊤ ∂ 2 b t − 1 ( 2 ) + ∂ b 2 ∂ 2 g ( 1 ) ( ∂ θ ∂ b t − 1 ( 1 ) ∂ θ ⊤ ∂ b t − 1 ( 1 ) − ∂ θ ∂ b t − 1 ( 2 ) ∂ θ ⊤ ∂ b t − 1 ( 2 ) ) + ( ∂ b 2 ∂ 2 g ( 1 ) − ∂ b 2 ∂ 2 g ( 2 ) ) ∂ θ ∂ b t − 1 ( 2 ) ∂ θ ⊤ ∂ b t − 1 ( 2 ) + ∂ b ∂ φ ∂ 2 g ( 1 ) { ( ∂ θ ∂ b t − 1 ( 1 ) − ∂ θ ∂ b t − 1 ( 2 ) ) e φ ⊤ + e φ ( ∂ θ ⊤ ∂ b t − 1 ( 1 ) − ∂ θ ⊤ ∂ b t − 1 ( 2 ) ) } + ( ∂ b ∂ φ ∂ 2 g ( 1 ) − ∂ b ∂ φ ∂ 2 g ( 2 ) ) ( ∂ θ ∂ b t − 1 ( 2 ) e φ ⊤ + e φ ∂ θ ⊤ ∂ b t − 1 ( 2 ) ) + ( ∂ φ 2 ∂ 2 g ( 1 ) − ∂ φ 2 ∂ 2 g ( 2 ) ) e φ e φ ⊤ .
Applying (A.5), (A.7), and (A.33) together with sup θ ∈ Θ ∥ ∂ θ θ ⊤ 2 b t − 1 ( 2 ) ( θ ) ∥ ≤ C C t \sup_{\theta\in\Theta}\|\partial_{\theta\theta^\top}^2b_{t-1}^{(2)}(\theta)\|\le CC_t sup θ ∈ Θ ∥ ∂ θ θ ⊤ 2 b t − 1 ( 2 ) ( θ ) ∥ ≤ C C t gives the third inequality in (A.31). This proves (i).
To prove (ii), the first inequality in (A.32) follows directly from the first line of (A.10). For the score difference, subtracting (A.28) for m = 1 , 2 m=1,2 m = 1 , 2 gives
∂ ℓ t ( 1 ) ∂ θ − ∂ ℓ t ( 2 ) ∂ θ = ∂ f ( 1 ) ∂ h ( ∂ h t ( 1 ) ∂ θ − ∂ h t ( 2 ) ∂ θ ) + ( ∂ f ( 1 ) ∂ h − ∂ f ( 2 ) ∂ h ) ∂ h t ( 2 ) ∂ θ + ( ∂ f ( 1 ) ∂ φ − ∂ f ( 2 ) ∂ φ ) e φ . (49) \frac{\partial\ell_t^{(1)}}{\partial\theta}-\frac{\partial\ell_t^{(2)}}{\partial\theta}=\frac{\partial f^{(1)}}{\partial h}\left(\frac{\partial h_t^{(1)}}{\partial\theta}-\frac{\partial h_t^{(2)}}{\partial\theta}\right)+\left(\frac{\partial f^{(1)}}{\partial h}-\frac{\partial f^{(2)}}{\partial h}\right)\frac{\partial h_t^{(2)}}{\partial\theta}+\left(\frac{\partial f^{(1)}}{\partial\varphi}-\frac{\partial f^{(2)}}{\partial\varphi}\right)e_\varphi. \tag*{(49)} ∂ θ ∂ ℓ t ( 1 ) − ∂ θ ∂ ℓ t ( 2 ) = ∂ h ∂ f ( 1 ) ( ∂ θ ∂ h t ( 1 ) − ∂ θ ∂ h t ( 2 ) ) + ( ∂ h ∂ f ( 1 ) − ∂ h ∂ f ( 2 ) ) ∂ θ ∂ h t ( 2 ) + ( ∂ φ ∂ f ( 1 ) − ∂ φ ∂ f ( 2 ) ) e φ . (49)
and applying (A.9) and (A.10) with sup θ ∈ Θ ∥ ∂ θ h t ( 2 ) ( θ ) ∥ ≤ C B t \sup_{\theta\in\Theta}\|\partial_\theta h_t^{(2)}(\theta)\|\le CB_t sup θ ∈ Θ ∥ ∂ θ h t ( 2 ) ( θ ) ∥ ≤ C B t yields the second inequality in (A.32). Finally, subtracting the Hessian (A.29) for m = 1 , 2 m=1,2 m = 1 , 2 gives
∂ 2 ℓ t ( 1 ) ∂ θ ∂ θ ⊤ − ∂ 2 ℓ t ( 2 ) ∂ θ ∂ θ ⊤ = ∂ f ( 1 ) ∂ h ( ∂ 2 h t ( 1 ) ∂ θ ∂ θ ⊤ − ∂ 2 h t ( 2 ) ∂ θ ∂ θ ⊤ ) + ( ∂ f ( 1 ) ∂ h − ∂ f ( 2 ) ∂ h ) ∂ 2 h t ( 2 ) ∂ θ ∂ θ ⊤ + ∂ 2 f ( 1 ) ∂ h 2 ( ∂ h t ( 1 ) ∂ θ ∂ h t ( 1 ) ∂ θ ⊤ − ∂ h t ( 2 ) ∂ θ ∂ h t ( 2 ) ∂ θ ⊤ ) + ( ∂ 2 f ( 1 ) ∂ h 2 − ∂ 2 f ( 2 ) ∂ h 2 ) ∂ h t ( 2 ) ∂ θ ∂ h t ( 2 ) ∂ θ ⊤ + ∂ 2 f ( 1 ) ∂ h ∂ φ { ( ∂ h t ( 1 ) ∂ θ − ∂ h t ( 2 ) ∂ θ ) e φ ⊤ + e φ ( ∂ h t ( 1 ) ∂ θ ⊤ − ∂ h t ( 2 ) ∂ θ ⊤ ) } + ( ∂ 2 f ( 1 ) ∂ h ∂ φ − ∂ 2 f ( 2 ) ∂ h ∂ φ ) ( ∂ h t ( 2 ) ∂ θ e φ ⊤ + e φ ∂ h t ( 2 ) ∂ θ ⊤ ) + ( ∂ 2 f ( 1 ) ∂ φ 2 − ∂ 2 f ( 2 ) ∂ φ 2 ) e φ e φ ⊤ . \begin{aligned}
\frac{\partial^2\ell_t^{(1)}}{\partial\theta\partial\theta^\top}-\frac{\partial^2\ell_t^{(2)}}{\partial\theta\partial\theta^\top}=\frac{\partial f^{(1)}}{\partial h}\left(\frac{\partial^2h_t^{(1)}}{\partial\theta\partial\theta^\top}-\frac{\partial^2h_t^{(2)}}{\partial\theta\partial\theta^\top}\right)+\left(\frac{\partial f^{(1)}}{\partial h}-\frac{\partial f^{(2)}}{\partial h}\right)\frac{\partial^2h_t^{(2)}}{\partial\theta\partial\theta^\top} \\
&+\frac{\partial^2f^{(1)}}{\partial h^2}\left(\frac{\partial h_t^{(1)}}{\partial\theta}\frac{\partial h_t^{(1)}}{\partial\theta^\top}-\frac{\partial h_t^{(2)}}{\partial\theta}\frac{\partial h_t^{(2)}}{\partial\theta^\top}\right)+\left(\frac{\partial^2f^{(1)}}{\partial h^2}-\frac{\partial^2f^{(2)}}{\partial h^2}\right)\frac{\partial h_t^{(2)}}{\partial\theta}\frac{\partial h_t^{(2)}}{\partial\theta^\top} \\
&+\frac{\partial^2f^{(1)}}{\partial h\partial\varphi}\left\{\left(\frac{\partial h_t^{(1)}}{\partial\theta}-\frac{\partial h_t^{(2)}}{\partial\theta}\right)e_\varphi^\top+e_\varphi\left(\frac{\partial h_t^{(1)}}{\partial\theta^\top}-\frac{\partial h_t^{(2)}}{\partial\theta^\top}\right)\right\} \\
&+\left(\frac{\partial^2f^{(1)}}{\partial h\partial\varphi}-\frac{\partial^2f^{(2)}}{\partial h\partial\varphi}\right)\left(\frac{\partial h_t^{(2)}}{\partial\theta}e_\varphi^\top+e_\varphi\frac{\partial h_t^{(2)}}{\partial\theta^\top}\right)+\left(\frac{\partial^2f^{(1)}}{\partial\varphi^2}-\frac{\partial^2f^{(2)}}{\partial\varphi^2}\right)e_\varphi e_\varphi^\top.
\end{aligned} ∂ θ ∂ θ ⊤ ∂ 2 ℓ t ( 1 ) − ∂ θ ∂ θ ⊤ ∂ 2 ℓ t ( 2 ) = ∂ h ∂ f ( 1 ) ( ∂ θ ∂ θ ⊤ ∂ 2 h t ( 1 ) − ∂ θ ∂ θ ⊤ ∂ 2 h t ( 2 ) ) + ( ∂ h ∂ f ( 1 ) − ∂ h ∂ f ( 2 ) ) ∂ θ ∂ θ ⊤ ∂ 2 h t ( 2 ) + ∂ h 2 ∂ 2 f ( 1 ) ( ∂ θ ∂ h t ( 1 ) ∂ θ ⊤ ∂ h t ( 1 ) − ∂ θ ∂ h t ( 2 ) ∂ θ ⊤ ∂ h t ( 2 ) ) + ( ∂ h 2 ∂ 2 f ( 1 ) − ∂ h 2 ∂ 2 f ( 2 ) ) ∂ θ ∂ h t ( 2 ) ∂ θ ⊤ ∂ h t ( 2 ) + ∂ h ∂ φ ∂ 2 f ( 1 ) { ( ∂ θ ∂ h t ( 1 ) − ∂ θ ∂ h t ( 2 ) ) e φ ⊤ + e φ ( ∂ θ ⊤ ∂ h t ( 1 ) − ∂ θ ⊤ ∂ h t ( 2 ) ) } + ( ∂ h ∂ φ ∂ 2 f ( 1 ) − ∂ h ∂ φ ∂ 2 f ( 2 ) ) ( ∂ θ ∂ h t ( 2 ) e φ ⊤ + e φ ∂ θ ⊤ ∂ h t ( 2 ) ) + ( ∂ φ 2 ∂ 2 f ( 1 ) − ∂ φ 2 ∂ 2 f ( 2 ) ) e φ e φ ⊤ .
Bounding each term by (A.9), (A.10), sup θ ∈ Θ ∥ ∂ θ h t ( m ) ( θ ) ∥ ≤ C B t \sup_{\theta\in\Theta}\|\partial_\theta h_t^{(m)}(\theta)\|\le CB_t sup θ ∈ Θ ∥ ∂ θ h t ( m ) ( θ ) ∥ ≤ C B t , and sup θ ∈ Θ ∥ ∂ θ θ ⊤ 2 h t ( m ) ( θ ) ∥ ≤ C C t \sup_{\theta\in\Theta}\|\partial_{\theta\theta^\top}^2h_t^{(m)}(\theta)\|\le CC_t sup θ ∈ Θ ∥ ∂ θ θ ⊤ 2 h t ( m ) ( θ ) ∥ ≤ C C t gives the third inequality in (A.32). ■
Lemma 4. Under Assumptions 1 and 6, we have that uniformly for θ ∈ Θ \theta\in\Theta θ ∈ Θ ,
(i) z t z_t z t and h t ( θ ) h_t(\theta) h t ( θ ) are L q L^q L q GMC;
(ii) ∂ h t ( θ ) / ∂ θ \partial h_t(\theta)/\partial\theta ∂ h t ( θ ) / ∂ θ is L p L^p L p GMC for 1 ≤ p < q 1 \le p < q 1 ≤ p < q ;
(iii) ∂ 2 h t ( θ ) / ( ∂ θ ∂ θ ⊤ ) \partial^2 h_t(\theta)/(\partial\theta\partial\theta^{\top}) ∂ 2 h t ( θ ) / ( ∂ θ ∂ θ ⊤ ) and s t ( θ 0 ) s_t(\theta_0) s t ( θ 0 ) are L p L^p L p GMC for 1 ≤ p < q / 2 1 \le p < q/2 1 ≤ p < q /2 .
Proof. Apply Lemma 3 with v ( 1 ) = { u t 2 } \mathbf{v}^{(1)} = \{u_t^2\} v ( 1 ) = { u t 2 } and v ( 2 ) = { ( u t ∘ ) 2 } \mathbf{v}^{(2)} = \{(u_t^\circ)^2\} v ( 2 ) = {( u t ∘ ) 2 } . We retain its differences Δ r h t \Delta_r h_t Δ r h t and Δ v t = ∣ u t 2 − ( u t ∘ ) 2 ∣ \Delta v_t = |u_t^2 - (u_t^\circ)^2| Δ v t = ∣ u t 2 − ( u t ∘ ) 2 ∣ . Proposition 5 with p = q p=q p = q gives ∥ Δ v t + Δ v t − 1 ∥ q ≤ C ρ t \|\Delta v_t+\Delta v_{t-1}\|_q \le C\rho^t ∥Δ v t + Δ v t − 1 ∥ q ≤ C ρ t , after increasing ρ < 1 \rho<1 ρ < 1 if necessary. The initial process differences have the moments required by Lemma 1(i)(a), since each version satisfies Lemma 2. Hence, Lemma 3(i) yields
∥ Δ 0 h t ∥ q ≤ C ρ t , ∥ Δ 1 h t ∥ q / 2 ≤ C ρ t , ∥ Δ 2 h t ∥ q / 3 ≤ C ρ t . \|\Delta_0 h_t\|_q \le C\rho^t,\qquad\|\Delta_1 h_t\|_{q/2} \le C\rho^t,\qquad\|\Delta_2 h_t\|_{q/3} \le C\rho^t. ∥ Δ 0 h t ∥ q ≤ C ρ t , ∥ Δ 1 h t ∥ q /2 ≤ C ρ t , ∥ Δ 2 h t ∥ q /3 ≤ C ρ t .
Moreover, the second inequality in (A.32) of Lemma 3(ii) gives
sup θ ∈ Θ ∥ s t ( θ ) − s t ∘ ( θ ) ∥ = Δ 1 ℓ t ≤ C { B t 2 ( Δ 0 h t + Δ v t ) + B t Δ 1 h t } . (50) \sup_{\theta\in\Theta}\|s_t(\theta)-s_t^\circ(\theta)\|=\Delta_1\ell_t\le C\{B_t^2(\Delta_0h_t+\Delta v_t)+B_t\Delta_1h_t\}. \tag*{(50)} θ ∈ Θ sup ∥ s t ( θ ) − s t ∘ ( θ ) ∥ = Δ 1 ℓ t ≤ C { B t 2 ( Δ 0 h t + Δ v t ) + B t Δ 1 h t } . (50)
where ∥ B t ∥ q ≤ C \|B_t\|_q\le C ∥ B t ∥ q ≤ C . Hölder’s inequality therefore yields
∥ sup θ ∈ Θ ∥ s t ( θ ) − s t ∘ ( θ ) ∥ ∥ q / 3 ≤ C ∥ B t ∥ q 2 ( ∥ Δ 0 h t ∥ q + ∥ Δ v t ∥ q ) + C ∥ B t ∥ q ∥ Δ 1 h t ∥ q / 2 ≤ C ρ t . \left\|\sup_{\theta\in\Theta}\|s_t(\theta)-s_t^\circ(\theta)\|\right\|_{q/3}
\le C\|B_t\|_q^2(\|\Delta_0h_t\|_q+\|\Delta v_t\|_q)+C\|B_t\|_q\|\Delta_1h_t\|_{q/2}\le C\rho^t. θ ∈ Θ sup ∥ s t ( θ ) − s t ∘ ( θ ) ∥ q /3 ≤ C ∥ B t ∥ q 2 ( ∥ Δ 0 h t ∥ q + ∥Δ v t ∥ q ) + C ∥ B t ∥ q ∥ Δ 1 h t ∥ q /2 ≤ C ρ t .
The first bound proves L q L^q L q GMC for h t h_t h t , and Proposition 5 already gives it for z t z_t z t . For the first parameter derivative, combine its L q / 2 L^{q/2} L q /2 coupling bound with its finite q q q th moment. For the second parameter derivative and score, combine their L q / 3 L^{q/3} L q /3 coupling bounds with their finite ( q / 2 ) (q/2) ( q /2 ) th moments. The interpolation in Lemma 1(ii)(a) then gives (ii) and (iii). ■
Proof of Proposition 6 . The proof adapts the bias and stochastic error decomposition in Lemma A.1 of Jiang et al. [23 ] , and the key idea is to use a GMC maximal inequality for the stochastic term. In particular, we can write
τ ^ t = 1 T h ∑ s = 1 T w t , s r s 2 . \widehat{\tau}_t=\frac{1}{Th}\sum_{s=1}^{T}w_{t,s}r_s^2. τ t = T h 1 s = 1 ∑ T w t , s r s 2 .
where w t , s w_{t,s} w t , s is the sum of the kernel weights at r s 2 r_s^2 r s 2 and its reflected copies. For t ∈ I T t\in\mathcal{I}_T t ∈ I T , no reflection is involved and w t , s = K ( ( t − s ) / ( T h ) ) w_{t,s}=K((t-s)/(Th)) w t , s = K (( t − s ) / ( T h )) . Since r s 2 = τ s ( 1 + z s ) r_s^2=\tau_s(1+z_s) r s 2 = τ s ( 1 + z s ) and E z s = 0 Ez_s=0 E z s = 0 , we have
τ ^ t − τ t = ( E τ ^ t − τ t ) + ( τ ^ t − E τ ^ t ) = { 1 T h ∑ s = 1 T w t , s τ s − τ t } + 1 T h ∑ s = 1 T w t , s τ s z s = : D t , T + S t , T . \begin{aligned}
\widehat{\tau}_t-\tau_t=(E\widehat{\tau}_t-\tau_t)+(\widehat{\tau}_t-E\widehat{\tau}_t) \\
&=\left\{\frac{1}{Th}\sum_{s=1}^{T}w_{t,s}\tau_s-\tau_t\right\}+\frac{1}{Th}\sum_{s=1}^{T}w_{t,s}\tau_sz_s=:D_{t,T}+S_{t,T}.
\end{aligned} τ t − τ t = ( E τ t − τ t ) + ( τ t − E τ t ) = { T h 1 s = 1 ∑ T w t , s τ s − τ t } + T h 1 s = 1 ∑ T w t , s τ s z s =: D t , T + S t , T .
The first term is the deterministic bias, while the second is the centered stochastic error. We bound these two terms separately.
Since the weights are uniformly bounded, and are nonzero for at most C T h CTh C T h observations, then by standard calculation,
max 1 ≤ t ≤ T ∣ D t , T ∣ = O ( h + 1 T h ) , \max_{1\le t\le T}|D_{t,T}|=O\left(h+\frac{1}{Th}\right), 1 ≤ t ≤ T max ∣ D t , T ∣ = O ( h + T h 1 ) ,
max t ∈ I T ∣ D t , T ∣ = O ( h 2 + 1 T h ) . \max_{t\in\mathcal{I}_T}|D_{t,T}|=O\left(h^2+\frac{1}{Th}\right). t ∈ I T max ∣ D t , T ∣ = O ( h 2 + T h 1 ) .
The first bound uses Lipschitz continuity of τ \tau τ at the reflected boundaries. In the interior, kernel symmetry removes the first-order Taylor term.
Let m = ⌈ T h ⌉ m=\lceil T h\rceil m = ⌈ T h ⌉ and partition { 1 , … , T } \{1,\ldots,T\} { 1 , … , T } into ⌈ T / m ⌉ \lceil T/m\rceil ⌈ T / m ⌉ consecutive blocks, each containing m m m or m − 1 m-1 m − 1 observations. For the j j j th block, write a = ( j − 1 ) m + 1 a=(j-1)m+1 a = ( j − 1 ) m + 1 and b = min ( j m , T ) b=\min(jm,T) b = min ( j m , T ) for its endpoints. Then, we have
∑ s = a b ω t , s τ s z s = ω t , b τ b ∑ v = a b z v + ∑ s = a b − 1 ( ω t , s τ s − ω t , s + 1 τ s + 1 ) ∑ v = a s z v \sum_{s=a}^{b}\omega_{t,s}\tau_s z_s=\omega_{t,b}\tau_b\sum_{v=a}^{b}z_v+\sum_{s=a}^{b-1}(\omega_{t,s}\tau_s-\omega_{t,s+1}\tau_{s+1})\sum_{v=a}^{s}z_v s = a ∑ b ω t , s τ s z s = ω t , b τ b v = a ∑ b z v + s = a ∑ b − 1 ( ω t , s τ s − ω t , s + 1 τ s + 1 ) v = a ∑ s z v
By Assumption 3, the kernel is bounded and has bounded variation. In addition, τ \tau τ and its first derivative are bounded. Consequently, uniformly over t t t and the blocks,
∣ ω t , b τ b ∣ + ∑ s = a b − 1 ∣ ω t , s τ s − ω t , s + 1 τ s + 1 ∣ ≤ C . |\omega_{t,b}\tau_b|+\sum_{s=a}^{b-1}|\omega_{t,s}\tau_s-\omega_{t,s+1}\tau_{s+1}|\le C. ∣ ω t , b τ b ∣ + s = a ∑ b − 1 ∣ ω t , s τ s − ω t , s + 1 τ s + 1 ∣ ≤ C .
It follows that
∣ ∑ s = a b ω t , s τ s z s ∣ ≤ C max a ≤ k ≤ b ∣ ∑ s = a k z s ∣ . \left|\sum_{s=a}^{b}\omega_{t,s}\tau_s z_s\right|\le C\max_{a\le k\le b}\left|\sum_{s=a}^{k}z_s\right|. s = a ∑ b ω t , s τ s z s ≤ C a ≤ k ≤ b max s = a ∑ k z s .
For each t t t , the nonzero weights arise from a fixed number of original or reflected kernel windows, each of length O ( m ) O(m) O ( m ) . Hence only a bounded number of blocks contribute to the weighted sum. Since T h ≍ m Th\asymp m T h ≍ m , we obtain
max 1 ≤ t ≤ T ∣ τ ^ t − E τ ^ t ∣ ≤ C m max 1 ≤ j ≤ ⌈ T / m ⌉ max ( j − 1 ) m < k ≤ min ( j m , T ) ∣ ∑ s = ( j − 1 ) m + 1 k z s ∣ . \max_{1\le t\le T}|\widehat{\tau}_t-E\widehat{\tau}_t|\le\frac{C}{m}\max_{1\le j\le\lceil T/m\rceil}\max_{(j-1)m<k\le\min(jm,T)}\left|\sum_{s=(j-1)m+1}^{k}z_s\right|. 1 ≤ t ≤ T max ∣ τ t − E τ t ∣ ≤ m C 1 ≤ j ≤ ⌈ T / m ⌉ max ( j − 1 ) m < k ≤ m i n ( j m , T ) max s = ( j − 1 ) m + 1 ∑ k z s .
By Proposition 5 , we have that z t z_t z t is L q L^q L q GMC. Therefore, by Theorem 2(ii) of Liu et al. [27 ] , we have that
P { max 1 ≤ k ≤ m ∣ ∑ s = 1 k z s ∣ > x } ≤ C m x q + C exp ( − c x 2 m ) P\left\{\max_{1\le k\le m}\left|\sum_{s=1}^{k}z_s\right|>x\right\}\le\frac{Cm}{x^q}+C\exp\left(-\frac{cx^2}{m}\right) P { 1 ≤ k ≤ m max s = 1 ∑ k z s > x } ≤ x q C m + C exp ( − m c x 2 )
whenever x ≥ C 0 m x\ge C_0\sqrt{m} x ≥ C 0 m , for a sufficiently large fixed constant C 0 C_0 C 0 .
By stationarity, this bound applies to every block. Hence, a union bound over the ⌈ T / m ⌉ \lceil T/m\rceil ⌈ T / m ⌉ blocks yields
P { max 1 ≤ t ≤ T ∣ τ ^ t − E τ ^ t ∣ > C x m } ≤ ⌈ T m ⌉ P { max 1 ≤ k ≤ m ∣ ∑ s = 1 k z s ∣ > x } ≤ C T x q + C T m exp ( − c x 2 m ) . \begin{aligned}
P\left\{\max_{1\le t\le T}|\widehat{\tau}_t-E\widehat{\tau}_t|>\frac{Cx}{m}\right\}\le\left\lceil\frac{T}{m}\right\rceil P\left\{\max_{1\le k\le m}\left|\sum_{s=1}^{k}z_s\right|>x\right\} \\
&\le\frac{CT}{x^q}+\frac{CT}{m}\exp\left(-\frac{cx^2}{m}\right).
\end{aligned} P { 1 ≤ t ≤ T max ∣ τ t − E τ t ∣ > m C x } ≤ ⌈ m T ⌉ P { 1 ≤ k ≤ m max s = 1 ∑ k z s > x } ≤ x q C T + m C T exp ( − m c x 2 ) .
Taking x = C 1 m log T x=C_1\sqrt{m\log T} x = C 1 m log T for a sufficiently large C 1 C_1 C 1 , we thus obtain that
P { max 1 ≤ t ≤ T ∣ τ ^ t − E τ ^ t ∣ > C 1 log T T h } ≤ C T m − q / 2 ( log T ) − q / 2 + C ( T / m ) T − c C 1 2 = o ( 1 ) . P\left\{\max_{1\le t\le T}|\widehat{\tau}_t-E\widehat{\tau}_t|>C_1\sqrt{\frac{\log T}{Th}}\right\}
\le CTm^{-q/2}(\log T)^{-q/2}+C(T/m)T^{-cC_1^2}=o(1). P { 1 ≤ t ≤ T max ∣ τ t − E τ t ∣ > C 1 T h log T } ≤ C T m − q /2 ( log T ) − q /2 + C ( T / m ) T − c C 1 2 = o ( 1 ) .
By Assumptions 5 and 6, m ≍ T 1 − λ h m\asymp T^{1-\lambda_h} m ≍ T 1 − λ h , q > 4 q>4 q > 4 , and λ h < 1 / 2 \lambda_h<1/2 λ h < 1/2 . These imply that 1 − ( 1 − λ h ) q / 2 < 0 1-(1-\lambda_h)q/2<0 1 − ( 1 − λ h ) q /2 < 0 , so the first term tends to zero. The second term tends to zero for a sufficiently large C 1 C_1 C 1 .
In other words, we have that
max 1 ≤ t ≤ T ∣ τ ^ t − E τ ^ t ∣ = O p ( log T T h ) . \max_{1\le t\le T}|\widehat{\tau}_t-E\widehat{\tau}_t|=O_p\left(\sqrt{\frac{\log T}{Th}}\right). 1 ≤ t ≤ T max ∣ τ t − E τ t ∣ = O p ( T h log T ) .
Combining this bound with the two bias bounds gives the convergence rate. Moreover, κ T → 0 \kappa_T\to0 κ T → 0 and inf t τ t ≥ τ > 0 \inf_t\tau_t\ge\tau>0 inf t τ t ≥ τ > 0 imply inf t τ ^ t ≥ τ / 2 \inf_t\widehat{\tau}_t\ge\tau/2 inf t τ t ≥ τ /2 with probability approaching one.
In the following, we distinguish three processes. The stationary infeasible one is h t ( θ ) h_t(\theta) h t ( θ ) . The infeasible one with initial values based on the true { u t } \{u_t\} { u t } is denoted by h ~ t ( θ ) \widetilde{h}_t(\theta) h t ( θ ) , while the feasible one based on u ^ t = r t / τ ^ t \widehat{u}_t=r_t/\sqrt{\widehat{\tau}_t} u t = r t / τ t is h ^ t ( θ ) \widehat{h}_t(\theta) h t ( θ ) . Both initialized processes use fixed positive initial values at the beginning of the sample. The corresponding negative log-likelihood functions are defined as ℓ t ( θ ) \ell_t(\theta) ℓ t ( θ ) , ℓ ~ t ( θ ) \widetilde{\ell}_t(\theta) ℓ t ( θ ) , and ℓ ^ t ( θ ) \widehat{\ell}_t(\theta) ℓ t ( θ ) , respectively.
Lemma 5. Under Assumptions 1–6, let κ T \kappa_T κ T be defined in Proposition 6 . Then
u ^ t 2 − u t 2 = − u t 2 τ ^ t − τ t τ t + u t 2 R t T , max 1 ≤ t ≤ T ∣ R t T ∣ = O p ( κ T 2 ) . (51) \widehat{u}_t^2-u_t^2=-u_t^2\frac{\widehat{\tau}_t-\tau_t}{\tau_t}+u_t^2R_{tT},\qquad\max_{1\le t\le T}|R_{tT}|=O_p(\kappa_T^2). \tag*{(51)} u t 2 − u t 2 = − u t 2 τ t τ t − τ t + u t 2 R tT , 1 ≤ t ≤ T max ∣ R tT ∣ = O p ( κ T 2 ) . (51) In addition, max t ∣ u ^ t 2 − u t 2 ∣ / ( 1 + u t 2 ) = O p ( κ T ) \max_t|\widehat{u}_t^2-u_t^2|/(1+u_t^2)=O_p(\kappa_T) max t ∣ u t 2 − u t 2 ∣/ ( 1 + u t 2 ) = O p ( κ T ) . For every p ≥ 1 p\ge1 p ≥ 1 such that E ∣ u t ∣ 2 p < ∞ E|u_t|^{2p}<\infty E ∣ u t ∣ 2 p < ∞ ,
1 T ∑ t = 1 T ∣ u ^ t 2 − u t 2 ∣ p = O p ( κ T p ) . \frac{1}{T}\sum_{t=1}^{T}|\widehat{u}_t^2-u_t^2|^p=O_p(\kappa_T^p). T 1 t = 1 ∑ T ∣ u t 2 − u t 2 ∣ p = O p ( κ T p ) .
Proof. Let δ t , T = ( τ ^ t − τ t ) / τ t \delta_{t,T}=(\widehat{\tau}_t-\tau_t)/\tau_t δ t , T = ( τ t − τ t ) / τ t . Proposition 6 gives max t ∣ δ t , T ∣ = O p ( κ T ) \max_t|\delta_{t,T}|=O_p(\kappa_T) max t ∣ δ t , T ∣ = O p ( κ T ) . Since κ T → 0 \kappa_T\to0 κ T → 0 , the event max t ∣ δ t , T ∣ ≤ 1 / 2 \max_t|\delta_{t,T}|\le1/2 max t ∣ δ t , T ∣ ≤ 1/2 has probability approaching one. The positivity truncation is also inactive with probability approaching one. On these events, 1 + δ t , T ≥ 1 / 2 1+\delta_{t,T}\ge1/2 1 + δ t , T ≥ 1/2 and u ^ t 2 = u t 2 / ( 1 + δ t , T ) \widehat{u}_t^2=u_t^2/(1+\delta_{t,T}) u t 2 = u t 2 / ( 1 + δ t , T ) . Hence,
u ^ t 2 − u t 2 = − u t 2 δ t , T + u t 2 δ t , T 2 1 + δ t , T , \widehat{u}_t^2-u_t^2=-u_t^2\delta_{t,T}+u_t^2\frac{\delta_{t,T}^2}{1+\delta_{t,T}}, u t 2 − u t 2 = − u t 2 δ t , T + u t 2 1 + δ t , T δ t , T 2 ,
which gives (A.36) with R t T = δ t , T 2 / ( 1 + δ t , T ) R_{tT}=\delta_{t,T}^2/(1+\delta_{t,T}) R tT = δ t , T 2 / ( 1 + δ t , T ) , because
max t ∣ R t T ∣ ≤ 2 ( max t ∣ δ t , T ∣ ) 2 = O p ( κ T 2 ) . \max_t|R_{tT}|\le2\left(\max_t|\delta_{t,T}|\right)^2=O_p(\kappa_T^2). t max ∣ R tT ∣ ≤ 2 ( t max ∣ δ t , T ∣ ) 2 = O p ( κ T 2 ) .
Moreover, ∣ u ^ t 2 − u t 2 ∣ ≤ 2 u t 2 ∣ δ t , T ∣ |\widehat{u}_t^2-u_t^2|\le2u_t^2|\delta_{t,T}| ∣ u t 2 − u t 2 ∣ ≤ 2 u t 2 ∣ δ t , T ∣ . Dividing by 1 + u t 2 1+u_t^2 1 + u t 2 and taking the maximum over t t t gives the stated ratio bound. For every p ≥ 1 p\ge1 p ≥ 1 with E ∣ u t ∣ 2 p < ∞ E|u_t|^{2p}<\infty E ∣ u t ∣ 2 p < ∞ , the same inequality yields
1 T ∑ t = 1 T ∣ u ^ t 2 − u t 2 ∣ p ≤ 2 p ( max t ∣ δ t , T ∣ ) p 1 T ∑ t = 1 T ∣ u t ∣ 2 p = O p ( κ T p ) , \frac{1}{T}\sum_{t=1}^{T}|\widehat{u}_t^2-u_t^2|^p\le2^p\left(\max_t|\delta_{t,T}|\right)^p\frac{1}{T}\sum_{t=1}^{T}|u_t|^{2p}=O_p(\kappa_T^p), T 1 t = 1 ∑ T ∣ u t 2 − u t 2 ∣ p ≤ 2 p ( t max ∣ δ t , T ∣ ) p T 1 t = 1 ∑ T ∣ u t ∣ 2 p = O p ( κ T p ) ,
where the last equality follows by the ergodic theorem T − 1 ∑ t = 1 T ∣ u t ∣ 2 p → E ∣ u t ∣ 2 p < ∞ T^{-1}\sum_{t=1}^{T}|u_t|^{2p}\to E|u_t|^{2p}<\infty T − 1 ∑ t = 1 T ∣ u t ∣ 2 p → E ∣ u t ∣ 2 p < ∞ a.s.
By combining Lemma 1 and Lemma 3, we now establish the initialization error bounds and the first-step estimation error bounds as follows.
Lemma 6. Suppose Assumptions 1–6 hold with some fixed positive initial values. Then, for every ρ ∈ ( β ˉ , 1 ) \rho\in(\bar{\beta},1) ρ ∈ ( β ˉ , 1 ) ,
∥ sup θ ∈ Θ ∣ h ~ t ( θ ) − h t ( θ ) ∣ ∥ q ≤ C ρ t . (52) \left\|\sup_{\theta\in\Theta}|\widetilde{h}_t(\theta)-h_t(\theta)|\right\|_q\le C\rho^t. \tag*{(52)} θ ∈ Θ sup ∣ h t ( θ ) − h t ( θ ) ∣ q ≤ C ρ t . (52)
∥ sup θ ∈ Θ ∥ ∂ h ~ t ( θ ) ∂ θ − ∂ h t ( θ ) ∂ θ ∥ ∥ q / 2 ≤ C ρ t , (53) \left\|\sup_{\theta\in\Theta}\left\|\frac{\partial\widetilde{h}_t(\theta)}{\partial\theta}-\frac{\partial h_t(\theta)}{\partial\theta}\right\|\right\|_{q/2}\le C\rho^t, \tag*{(53)} θ ∈ Θ sup ∂ θ ∂ h t ( θ ) − ∂ θ ∂ h t ( θ ) q /2 ≤ C ρ t , (53)
∥ sup θ ∈ Θ ∥ ∂ 2 h ~ t ( θ ) ∂ θ ∂ θ ⊤ − ∂ 2 h t ( θ ) ∂ θ ∂ θ ⊤ ∥ ∥ q / 3 ≤ C ρ t . (54) \left\|\sup_{\theta\in\Theta}\left\|\frac{\partial^2\widetilde{h}_t(\theta)}{\partial\theta\partial\theta^\top}-\frac{\partial^2h_t(\theta)}{\partial\theta\partial\theta^\top}\right\|\right\|_{q/3}\le C\rho^t. \tag*{(54)} θ ∈ Θ sup ∂ θ ∂ θ ⊤ ∂ 2 h t ( θ ) − ∂ θ ∂ θ ⊤ ∂ 2 h t ( θ ) q /3 ≤ C ρ t . (54)
Furthermore, the log-likelihood, score, and Hessian initialization errors satisfy
E [ sup θ ∈ Θ ∣ ℓ ~ t ( θ ) − ℓ t ( θ ) ∣ ] + E [ sup θ ∈ Θ ∥ ∂ ℓ ~ t ( θ ) ∂ θ − ∂ ℓ t ( θ ) ∂ θ ∥ ] + E [ sup θ ∈ Θ ∥ ∂ 2 ℓ ~ t ( θ ) ∂ θ ∂ θ ⊤ − ∂ 2 ℓ t ( θ ) ∂ θ ∂ θ ⊤ ∥ ] ≤ C ρ t , (55) E\left[\sup_{\theta\in\Theta}\left|\widetilde{\ell}_t(\theta)-\ell_t(\theta)\right|\right]+E\left[\sup_{\theta\in\Theta}\left\|\frac{\partial\widetilde{\ell}_t(\theta)}{\partial\theta}-\frac{\partial\ell_t(\theta)}{\partial\theta}\right\|\right]+E\left[\sup_{\theta\in\Theta}\left\|\frac{\partial^2\widetilde{\ell}_t(\theta)}{\partial\theta\partial\theta^\top}-\frac{\partial^2\ell_t(\theta)}{\partial\theta\partial\theta^\top}\right\|\right]\le C\rho^t, \tag*{(55)} E [ θ ∈ Θ sup ℓ t ( θ ) − ℓ t ( θ ) ] + E [ θ ∈ Θ sup ∂ θ ∂ ℓ t ( θ ) − ∂ θ ∂ ℓ t ( θ ) ] + E [ θ ∈ Θ sup ∂ θ ∂ θ ⊤ ∂ 2 ℓ t ( θ ) − ∂ θ ∂ θ ⊤ ∂ 2 ℓ t ( θ ) ] ≤ C ρ t , (55)
and consequently,
sup θ ∈ Θ ∣ 1 T ∑ t = 1 T { ℓ ~ t ( θ ) − ℓ t ( θ ) } ∣ = O p ( T − 1 ) = o p ( 1 ) , (56) \sup_{\theta\in\Theta}\left|\frac{1}{T}\sum_{t=1}^{T}\{\widetilde{\ell}_t(\theta)-\ell_t(\theta)\}\right|=O_p(T^{-1})=o_p(1), \tag*{(56)} θ ∈ Θ sup T 1 t = 1 ∑ T { ℓ t ( θ ) − ℓ t ( θ )} = O p ( T − 1 ) = o p ( 1 ) , (56)
sup θ ∈ Θ ∥ 1 T ∑ t = 1 T { ∂ ℓ ~ t ( θ ) ∂ θ − ∂ ℓ t ( θ ) ∂ θ } ∥ = O p ( T − 1 ) = o p ( 1 ) , (57) \sup_{\theta\in\Theta}\left\|\frac{1}{T}\sum_{t=1}^{T}\left\{\frac{\partial\widetilde{\ell}_t(\theta)}{\partial\theta}-\frac{\partial\ell_t(\theta)}{\partial\theta}\right\}\right\|=O_p(T^{-1})=o_p(1), \tag*{(57)} θ ∈ Θ sup T 1 t = 1 ∑ T { ∂ θ ∂ ℓ t ( θ ) − ∂ θ ∂ ℓ t ( θ ) } = O p ( T − 1 ) = o p ( 1 ) , (57)
sup θ ∈ Θ ∥ 1 T ∑ t = 1 T { ∂ 2 ℓ ~ t ( θ ) ∂ θ ∂ θ ⊤ − ∂ 2 ℓ t ( θ ) ∂ θ ∂ θ ⊤ } ∥ = O p ( T − 1 ) = o p ( 1 ) . (58) \sup_{\theta\in\Theta}\left\|\frac{1}{T}\sum_{t=1}^{T}\left\{\frac{\partial^2\widetilde{\ell}_t(\theta)}{\partial\theta\partial\theta^\top}-\frac{\partial^2\ell_t(\theta)}{\partial\theta\partial\theta^\top}\right\}\right\|=O_p(T^{-1})=o_p(1). \tag*{(58)} θ ∈ Θ sup T 1 t = 1 ∑ T { ∂ θ ∂ θ ⊤ ∂ 2 ℓ t ( θ ) − ∂ θ ∂ θ ⊤ ∂ 2 ℓ t ( θ ) } = O p ( T − 1 ) = o p ( 1 ) . (58)
Moreover, the same bounds hold uniformly when { u t 2 } \{u_t^2\} { u t 2 } is replaced by the interpolated inputs u t 2 ( a ) = u t 2 + a ( u ^ t 2 − u t 2 ) u_t^2(a)=u_t^2+a(\widehat{u}_t^2-u_t^2) u t 2 ( a ) = u t 2 + a ( u t 2 − u t 2 ) , a ∈ [ 0 , 1 ] a\in[0,1] a ∈ [ 0 , 1 ] , on the event max 1 ≤ s ≤ T ∣ τ s / τ ^ s − 1 ∣ ≤ 1 / 2 \max_{1\le s\le T}|\tau_s/\widehat{\tau}_s-1|\le1/2 max 1 ≤ s ≤ T ∣ τ s / τ s − 1∣ ≤ 1/2 .
Proof. Apply Lemma 3 with identical inputs v t ( 1 ) = v t ( 2 ) = u t 2 v_t^{(1)}=v_t^{(2)}=u_t^2 v t ( 1 ) = v t ( 2 ) = u t 2 (or u t 2 ( a ) ≤ 2 u t 2 u_t^2(a)\le2u_t^2 u t 2 ( a ) ≤ 2 u t 2 ) for t ≥ 1 t\ge1 t ≥ 1 , so that Δ v t = ∣ v t ( 1 ) − v t ( 2 ) ∣ = 0 \Delta v_t=|v_t^{(1)}-v_t^{(2)}|=0 Δ v t = ∣ v t ( 1 ) − v t ( 2 ) ∣ = 0 for t ≥ 1 t\ge1 t ≥ 1 , while h 0 ( 1 ) h_0^{(1)} h 0 ( 1 ) is the fixed initial value and h 0 ( 2 ) = h 0 ( θ ) h_0^{(2)}=h_0(\theta) h 0 ( 2 ) = h 0 ( θ ) is the stationary initial state. Then (A.31) in Lemma 3(i) and (A.18) in Lemma 1(i)(a) immediately yield (A.37)–(A.39). Next, setting Δ v t = 0 \Delta v_t=0 Δ v t = 0 in (A.32) of Lemma 3(ii) gives
sup θ ∈ Θ ∣ ℓ ~ t ( θ ) − ℓ t ( θ ) ∣ ≤ C B t Δ 0 h t , \sup_{\theta\in\Theta}|\widetilde{\ell}_t(\theta)-\ell_t(\theta)|\le CB_t\Delta_0h_t, θ ∈ Θ sup ∣ ℓ t ( θ ) − ℓ t ( θ ) ∣ ≤ C B t Δ 0 h t ,
sup θ ∈ Θ ∥ ∂ ℓ ~ t ( θ ) ∂ θ − ∂ ℓ t ( θ ) ∂ θ ∥ ≤ C B t 2 Δ 0 h t + C B t Δ 1 h t , \sup_{\theta\in\Theta}\left\|\frac{\partial\widetilde{\ell}_t(\theta)}{\partial\theta}-\frac{\partial\ell_t(\theta)}{\partial\theta}\right\|\le CB_t^2\Delta_0h_t+CB_t\Delta_1h_t, θ ∈ Θ sup ∂ θ ∂ ℓ t ( θ ) − ∂ θ ∂ ℓ t ( θ ) ≤ C B t 2 Δ 0 h t + C B t Δ 1 h t ,
sup θ ∈ Θ ∥ ∂ 2 ℓ ~ t ( θ ) ∂ θ ∂ θ ⊤ − ∂ 2 ℓ t ( θ ) ∂ θ ∂ θ ⊤ ∥ ≤ C ( B t C t + B t 3 ) Δ 0 h t + C B t 2 Δ 1 h t + C B t Δ 2 h t . \sup_{\theta\in\Theta}\left\|\frac{\partial^2\widetilde{\ell}_t(\theta)}{\partial\theta\partial\theta^\top}-\frac{\partial^2\ell_t(\theta)}{\partial\theta\partial\theta^\top}\right\|\le C(B_tC_t+B_t^3)\Delta_0h_t+CB_t^2\Delta_1h_t+CB_t\Delta_2h_t. θ ∈ Θ sup ∂ θ ∂ θ ⊤ ∂ 2 ℓ t ( θ ) − ∂ θ ∂ θ ⊤ ∂ 2 ℓ t ( θ ) ≤ C ( B t C t + B t 3 ) Δ 0 h t + C B t 2 Δ 1 h t + C B t Δ 2 h t .
Since sup t ∥ B t ∥ q < ∞ \sup_t\|B_t\|_q<\infty sup t ∥ B t ∥ q < ∞ , sup t ∥ C t ∥ q / 2 < ∞ \sup_t\|C_t\|_{q/2}<\infty sup t ∥ C t ∥ q /2 < ∞ , and q > 4 q>4 q > 4 (so q / 4 > 1 q/4>1 q /4 > 1 ), Hölder’s inequality implies that the L q / 4 L^{q/4} L q /4 norms (and hence the expectations) of the right-hand sides are bounded by C ρ t C\rho^t C ρ t , proving (A.40). Summing (A.40) over t = 1 , … , T t=1,\ldots,T t = 1 , … , T and applying Markov’s inequality proves (A.41)–(A.43). ■
Lemma 7. Define u t 2 ( a ) = u t 2 + a ( u ^ t 2 − u t 2 ) u_t^2(a)=u_t^2+a(\widehat{u}_t^2-u_t^2) u t 2 ( a ) = u t 2 + a ( u t 2 − u t 2 ) , a ∈ [ 0 , 1 ] a\in[0,1] a ∈ [ 0 , 1 ] . Suppose Assumptions 1–6 hold. Then with probability approaching one, for all 1 ≤ t ≤ T 1\le t\le T 1 ≤ t ≤ T and any a , a ′ ∈ [ 0 , 1 ] a,a'\in[0,1] a , a ′ ∈ [ 0 , 1 ] , the process h t ( a , θ ) h_t(a,\theta) h t ( a , θ ) driven by { u s 2 ( a ) } \{u_s^2(a)\} { u s 2 ( a )} with fixed initial values satisfies
sup θ ∈ Θ ∣ h t ( a ′ , θ ) − h t ( a , θ ) ∣ ≤ C ∣ a ′ − a ∣ U t max 1 ≤ s ≤ T ∣ τ s / τ ^ s − 1 ∣ , \sup_{\theta\in\Theta}|h_t(a',\theta)-h_t(a,\theta)|\le C|a'-a|U_t\max_{1\le s\le T}|\tau_s/\widehat{\tau}_s-1|, θ ∈ Θ sup ∣ h t ( a ′ , θ ) − h t ( a , θ ) ∣ ≤ C ∣ a ′ − a ∣ U t 1 ≤ s ≤ T max ∣ τ s / τ s − 1∣ , sup θ ∈ Θ ∥ ∂ h t ( a ′ , θ ) ∂ θ − ∂ h t ( a , θ ) ∂ θ ∥ ≤ C ∣ a ′ − a ∣ G 1 , t max 1 ≤ s ≤ T ∣ τ s / τ ^ s − 1 ∣ , \sup_{\theta\in\Theta}\left\|\frac{\partial h_t(a',\theta)}{\partial\theta}-\frac{\partial h_t(a,\theta)}{\partial\theta}\right\|\le C|a'-a|G_{1,t}\max_{1\le s\le T}|\tau_s/\widehat{\tau}_s-1|, θ ∈ Θ sup ∂ θ ∂ h t ( a ′ , θ ) − ∂ θ ∂ h t ( a , θ ) ≤ C ∣ a ′ − a ∣ G 1 , t 1 ≤ s ≤ T max ∣ τ s / τ s − 1∣ , sup θ ∈ Θ ∥ ∂ 2 h t ( a ′ , θ ) ∂ θ ∂ θ ⊤ − ∂ 2 h t ( a , θ ) ∂ θ ∂ θ ⊤ ∥ ≤ C ∣ a ′ − a ∣ G 2 , t max 1 ≤ s ≤ T ∣ τ s / τ ^ s − 1 ∣ . (59) \sup_{\theta\in\Theta}\left\|\frac{\partial^2h_t(a',\theta)}{\partial\theta\partial\theta^\top}-\frac{\partial^2h_t(a,\theta)}{\partial\theta\partial\theta^\top}\right\|\le C|a'-a|G_{2,t}\max_{1\le s\le T}|\tau_s/\widehat{\tau}_s-1|. \tag*{(59)} θ ∈ Θ sup ∂ θ ∂ θ ⊤ ∂ 2 h t ( a ′ , θ ) − ∂ θ ∂ θ ⊤ ∂ 2 h t ( a , θ ) ≤ C ∣ a ′ − a ∣ G 2 , t 1 ≤ s ≤ T max ∣ τ s / τ s − 1∣. (59)
In particular, at ( a ′ , a ) = ( 1 , 0 ) (a',a)=(1,0) ( a ′ , a ) = ( 1 , 0 ) (so h t ( 1 , θ ) = h ^ t ( θ ) h_t(1,\theta)=\widehat{h}_t(\theta) h t ( 1 , θ ) = h t ( θ ) and h t ( 0 , θ ) = h ~ t ( θ ) h_t(0,\theta)=\widetilde{h}_t(\theta) h t ( 0 , θ ) = h t ( θ ) ). The pointwise likelihood, score, and Hessian differences satisfy that uniformly for 1 ≤ t ≤ T 1\le t\le T 1 ≤ t ≤ T with probability approaching one:
sup θ ∈ Θ ∣ ℓ ^ t ( θ ) − ℓ ~ t ( θ ) ∣ ≤ C U t 2 max 1 ≤ s ≤ T ∣ τ s / τ ^ s − 1 ∣ , sup θ ∈ Θ ∥ ∂ ℓ ^ t ( θ ) ∂ θ − ∂ ℓ ~ t ( θ ) ∂ θ ∥ ≤ C { U t G 1 , t + U t 3 } max 1 ≤ s ≤ T ∣ τ s / τ ^ s − 1 ∣ , sup θ ∈ Θ ∥ ∂ 2 ℓ ^ t ( θ ) ∂ θ ∂ θ ⊤ − ∂ 2 ℓ ~ t ( θ ) ∂ θ ∂ θ ⊤ ∥ ≤ C J t max 1 ≤ s ≤ T ∣ τ s / τ ^ s − 1 ∣ . (60) \begin{aligned}
\sup_{\theta\in\Theta}\left|\widehat{\ell}_t(\theta)-\widetilde{\ell}_t(\theta)\right|&\le CU_t^2\max_{1\le s\le T}\left|\tau_s/\widehat{\tau}_s-1\right|,\\
\sup_{\theta\in\Theta}\left\|\frac{\partial\widehat{\ell}_t(\theta)}{\partial\theta}-\frac{\partial\widetilde{\ell}_t(\theta)}{\partial\theta}\right\|&\le C\{U_tG_{1,t}+U_t^3\}\max_{1\le s\le T}\left|\tau_s/\widehat{\tau}_s-1\right|,\\
\sup_{\theta\in\Theta}\left\|\frac{\partial^2\widehat{\ell}_t(\theta)}{\partial\theta\partial\theta^\top}-\frac{\partial^2\widetilde{\ell}_t(\theta)}{\partial\theta\partial\theta^\top}\right\|&\le CJ_t\max_{1\le s\le T}\left|\tau_s/\widehat{\tau}_s-1\right|. \tag*{(60)}
\end{aligned} θ ∈ Θ sup ℓ t ( θ ) − ℓ t ( θ ) θ ∈ Θ sup ∂ θ ∂ ℓ t ( θ ) − ∂ θ ∂ ℓ t ( θ ) θ ∈ Θ sup ∂ θ ∂ θ ⊤ ∂ 2 ℓ t ( θ ) − ∂ θ ∂ θ ⊤ ∂ 2 ℓ t ( θ ) ≤ C U t 2 1 ≤ s ≤ T max ∣ τ s / τ s − 1 ∣ , ≤ C { U t G 1 , t + U t 3 } 1 ≤ s ≤ T max ∣ τ s / τ s − 1 ∣ , ≤ C J t 1 ≤ s ≤ T max ∣ τ s / τ s − 1 ∣ . (60)
where J t : = C { U t G 2 , t + U t 2 G 1 , t + U t 4 } ∈ L q / 4 ⊂ L 1 J_t:=C\{U_tG_{2,t}+U_t^2G_{1,t}+U_t^4\}\in L^{q/4}\subset L^1 J t := C { U t G 2 , t + U t 2 G 1 , t + U t 4 } ∈ L q /4 ⊂ L 1 , and the sample averages satisfy
1 T ∑ t = 1 T sup θ ∈ Θ ∣ h ^ t ( θ ) − h ~ t ( θ ) ∣ = O p ( κ T ) , (61) \frac{1}{T}\sum_{t=1}^{T}\sup_{\theta\in\Theta}\left|\widehat{h}_t(\theta)-\widetilde{h}_t(\theta)\right|=O_p(\kappa_T), \tag*{(61)} T 1 t = 1 ∑ T θ ∈ Θ sup h t ( θ ) − h t ( θ ) = O p ( κ T ) , (61)
1 T ∑ t = 1 T sup θ ∈ Θ ∥ ∂ h ^ t ( θ ) ∂ θ − ∂ h ~ t ( θ ) ∂ θ ∥ = O p ( κ T ) , (62) \frac{1}{T}\sum_{t=1}^{T}\sup_{\theta\in\Theta}\left\|\frac{\partial\widehat{h}_t(\theta)}{\partial\theta}-\frac{\partial\widetilde{h}_t(\theta)}{\partial\theta}\right\|=O_p(\kappa_T), \tag*{(62)} T 1 t = 1 ∑ T θ ∈ Θ sup ∂ θ ∂ h t ( θ ) − ∂ θ ∂ h t ( θ ) = O p ( κ T ) , (62)
1 T ∑ t = 1 T sup θ ∈ Θ ∥ ∂ 2 h ^ t ( θ ) ∂ θ ∂ θ ⊤ − ∂ 2 h ~ t ( θ ) ∂ θ ∂ θ ⊤ ∥ = O p ( κ T ) , (63) \frac{1}{T}\sum_{t=1}^{T}\sup_{\theta\in\Theta}\left\|\frac{\partial^2\widehat{h}_t(\theta)}{\partial\theta\partial\theta^\top}-\frac{\partial^2\widetilde{h}_t(\theta)}{\partial\theta\partial\theta^\top}\right\|=O_p(\kappa_T), \tag*{(63)} T 1 t = 1 ∑ T θ ∈ Θ sup ∂ θ ∂ θ ⊤ ∂ 2 h t ( θ ) − ∂ θ ∂ θ ⊤ ∂ 2 h t ( θ ) = O p ( κ T ) , (63)
1 T ∑ t = 1 T sup θ ∈ Θ ∣ ℓ ^ t ( θ ) − ℓ ~ t ( θ ) ∣ = O p ( κ T ) = o p ( 1 ) , (64) \frac{1}{T}\sum_{t=1}^{T}\sup_{\theta\in\Theta}\left|\widehat{\ell}_t(\theta)-\widetilde{\ell}_t(\theta)\right|=O_p(\kappa_T)=o_p(1), \tag*{(64)} T 1 t = 1 ∑ T θ ∈ Θ sup ℓ t ( θ ) − ℓ t ( θ ) = O p ( κ T ) = o p ( 1 ) , (64)
1 T ∑ t = 1 T sup θ ∈ Θ ∥ ∂ ℓ ^ t ( θ ) ∂ θ − ∂ ℓ ~ t ( θ ) ∂ θ ∥ = O p ( κ T ) = o p ( 1 ) , (65) \frac{1}{T}\sum_{t=1}^{T}\sup_{\theta\in\Theta}\left\|\frac{\partial\widehat{\ell}_t(\theta)}{\partial\theta}-\frac{\partial\widetilde{\ell}_t(\theta)}{\partial\theta}\right\|=O_p(\kappa_T)=o_p(1), \tag*{(65)} T 1 t = 1 ∑ T θ ∈ Θ sup ∂ θ ∂ ℓ t ( θ ) − ∂ θ ∂ ℓ t ( θ ) = O p ( κ T ) = o p ( 1 ) , (65)
1 T ∑ t = 1 T sup θ ∈ Θ ∥ ∂ 2 ℓ ^ t ( θ ) ∂ θ ∂ θ ⊤ − ∂ 2 ℓ ~ t ( θ ) ∂ θ ∂ θ ⊤ ∥ = O p ( κ T ) = o p ( 1 ) . (66) \frac{1}{T}\sum_{t=1}^{T}\sup_{\theta\in\Theta}\left\|\frac{\partial^2\widehat{\ell}_t(\theta)}{\partial\theta\partial\theta^\top}-\frac{\partial^2\widetilde{\ell}_t(\theta)}{\partial\theta\partial\theta^\top}\right\|=O_p(\kappa_T)=o_p(1). \tag*{(66)} T 1 t = 1 ∑ T θ ∈ Θ sup ∂ θ ∂ θ ⊤ ∂ 2 ℓ t ( θ ) − ∂ θ ∂ θ ⊤ ∂ 2 ℓ t ( θ ) = O p ( κ T ) = o p ( 1 ) . (66)
Proof. The proof is similar to that of Lemma A.3 in Jiang et al. [23 ] . By Lemma 5, the event δ T : = max 1 ≤ s ≤ T ∣ τ s / τ ^ s − 1 ∣ ≤ 1 / 2 \delta_T:=\max_{1\le s\le T}|\tau_s/\widehat{\tau}_s-1|\le1/2 δ T := max 1 ≤ s ≤ T ∣ τ s / τ s − 1∣ ≤ 1/2 has probability approaching one, and on this event u t 2 ( a ) ≤ 2 u t 2 u_t^2(a)\le2u_t^2 u t 2 ( a ) ≤ 2 u t 2 and
Δ v t = ∣ u t 2 ( a ′ ) − u t 2 ( a ) ∣ = ∣ a ′ − a ∣ ∣ u ^ t 2 − u t 2 ∣ ≤ ∣ a ′ − a ∣ δ T u t 2 . \Delta v_t=\left|u_t^2(a')-u_t^2(a)\right|=\left|a'-a\right|\left|\widehat{u}_t^2-u_t^2\right|\le\left|a'-a\right|\delta_Tu_t^2. Δ v t = u t 2 ( a ′ ) − u t 2 ( a ) = ∣ a ′ − a ∣ u t 2 − u t 2 ≤ ∣ a ′ − a ∣ δ T u t 2 .
Applying Lemma 3(i) with identical initial values h 0 ( 1 ) = h 0 ( 2 ) h_0^{(1)}=h_0^{(2)} h 0 ( 1 ) = h 0 ( 2 ) and (A.20) of Lemma 1(i)(b) with Δ T = ∣ a ′ − a ∣ δ T \Delta_T=|a'-a|\delta_T Δ T = ∣ a ′ − a ∣ δ T immediately yields (A.44). Setting ( a ′ , a ) = ( 1 , 0 ) (a',a)=(1,0) ( a ′ , a ) = ( 1 , 0 ) and substituting Δ v t ≤ δ T U t \Delta v_t\le\delta_TU_t Δ v t ≤ δ T U t , Δ 0 h t ≤ C δ T U t \Delta_0h_t\le C\delta_TU_t Δ 0 h t ≤ C δ T U t , Δ 1 h t ≤ C δ T G 1 , t \Delta_1h_t\le C\delta_TG_{1,t} Δ 1 h t ≤ C δ T G 1 , t , and Δ 2 h t ≤ C δ T G 2 , t \Delta_2h_t\le C\delta_TG_{2,t} Δ 2 h t ≤ C δ T G 2 , t (with B t ≤ C U t B_t\le CU_t B t ≤ C U t and C t ≤ C ( U t 2 + G 1 , t − 1 ) ≤ C ( U t 2 + G 1 , t ) ) C_t\le C(U_t^2+G_{1,t-1})\le C(U_t^2+G_{1,t})) C t ≤ C ( U t 2 + G 1 , t − 1 ) ≤ C ( U t 2 + G 1 , t )) into (A.32) of Lemma 3(ii) gives (A.45). Since U t ∈ L q U_t\in L^q U t ∈ L q , G 1 , t ∈ L q / 2 G_{1,t}\in L^{q/2} G 1 , t ∈ L q /2 , G 2 , t ∈ L q / 3 G_{2,t}\in L^{q/3} G 2 , t ∈ L q /3 , and J t ∈ L q / 4 ⊂ L 1 J_t\in L^{q/4}\subset L^1 J t ∈ L q /4 ⊂ L 1 are stationary and ergodic, their sample averages are O p ( 1 ) O_p(1) O p ( 1 ) by the ergodic theorem, which together with δ T = O p ( κ T ) \delta_T=O_p(\kappa_T) δ T = O p ( κ T ) proves (A.46)–(A.51). ■
Proof of Theorem 2. Let Q T ( θ ) = T − 1 ∑ t = 1 T ℓ t ( θ ) Q_T(\theta)=T^{-1}\sum_{t=1}^{T}\ell_t(\theta) Q T ( θ ) = T − 1 ∑ t = 1 T ℓ t ( θ ) , Q ~ T ( θ ) = T − 1 ∑ t = 1 T ℓ ~ t ( θ ) \widetilde{Q}_T(\theta)=T^{-1}\sum_{t=1}^{T}\widetilde{\ell}_t(\theta) Q T ( θ ) = T − 1 ∑ t = 1 T ℓ t ( θ ) , Q ^ T ( θ ) = T − 1 ∑ t = 1 T ℓ ^ t ( θ ) \widehat{Q}_T(\theta)=T^{-1}\sum_{t=1}^{T}\widehat{\ell}_t(\theta) Q T ( θ ) = T − 1 ∑ t = 1 T ℓ t ( θ ) , and define the population criterion Q ( θ ) = E { ℓ t ( θ ) } Q(\theta)=E\{\ell_t(\theta)\} Q ( θ ) = E { ℓ t ( θ )} .
First, θ ↦ ℓ t ( θ ) \theta\mapsto\ell_t(\theta) θ ↦ ℓ t ( θ ) is almost surely continuous. Indeed, ω ( θ ) \omega(\theta) ω ( θ ) is continuous, h t ( θ ) h_t(\theta) h t ( θ ) is obtained from a continuous stable recursion, and h t ( θ ) ≥ ω > 0 h_t(\theta)\ge\omega>0 h t ( θ ) ≥ ω > 0 uniformly on Θ \Theta Θ . Under the moment conditions in Assumption 6,
E [ sup θ ∈ Θ ∣ ℓ t ( θ ) ∣ ] < ∞ . E\left[\sup_{\theta\in\Theta}\left|\ell_t(\theta)\right|\right]<\infty. E [ θ ∈ Θ sup ∣ ℓ t ( θ ) ∣ ] < ∞.
Since the stationary augmented process is ergodic, a uniform ergodic theorem applies. More precisely, compactness of Θ \Theta Θ , almost-sure continuity, the displayed integrable envelope, and the pointwise ergodic theorem verify the conditions of Lemma 2.4 of Newey and McFadden [29 ] , which gives
sup θ ∈ Θ ∣ Q T ( θ ) − Q ( θ ) ∣ = o p ( 1 ) . (67) \sup_{\theta\in\Theta}\left|Q_T(\theta)-Q(\theta)\right|=o_p(1). \tag*{(67)} θ ∈ Θ sup ∣ Q T ( θ ) − Q ( θ ) ∣ = o p ( 1 ) . (67)
Second, combining (A.41) in Lemma 6 and (A.49) in Lemma 7 implies
sup θ ∈ Θ ∣ Q ^ T ( θ ) − Q T ( θ ) ∣ = o p ( 1 ) . (68) \sup_{\theta\in\Theta}\left|\widehat{Q}_T(\theta)-Q_T(\theta)\right|=o_p(1). \tag*{(68)} θ ∈ Θ sup Q T ( θ ) − Q T ( θ ) = o p ( 1 ) . (68)
Combining (A.52) and (A.53) gives
sup θ ∈ Θ ∣ Q ^ T ( θ ) − Q ( θ ) ∣ = o p ( 1 ) . \sup_{\theta\in\Theta}\left|\widehat{Q}_T(\theta)-Q(\theta)\right|=o_p(1). θ ∈ Θ sup Q T ( θ ) − Q ( θ ) = o p ( 1 ) .
Note that under Gaussian innovations, the identification argument for condition (iv) in the proof of Theorem 5 of Smetanina and Wu [34 ] implies that there is a unique minimizer θ 0 \theta_0 θ 0 on Θ \Theta Θ . Theorem 2.1 of Newey and McFadden [29 ] then yields
θ ^ T → p θ 0 . \widehat{\theta}_T \xrightarrow{p} \theta_0. θ T p θ 0 .
The next lemma gives the expansion needed when the true squared returns are replaced by their estimates.
Lemma 8. Suppose Assumptions 1–6 hold and θ 0 \theta_0 θ 0 is an interior point of Θ \Theta Θ . Let N \mathcal{N} N be a compact neighborhood of θ 0 \theta_0 θ 0 contained in the interior of Θ \Theta Θ , and let p = q / 2 > 2 p=q/2>2 p = q /2 > 2 . Then, for some ρ ∈ ( 0 , 1 ) \rho\in(0,1) ρ ∈ ( 0 , 1 ) ,
∥ u t − j 2 ∂ s t ( θ 0 ) ∂ ( u t − j 2 ) ∥ p ≤ C ρ j , j ≥ 0. (69) \left\|u_{t-j}^{2}\frac{\partial s_t(\theta_0)}{\partial(u_{t-j}^{2})}\right\|_p\le C\rho^j,\qquad j\ge0. \tag*{(69)} u t − j 2 ∂ ( u t − j 2 ) ∂ s t ( θ 0 ) p ≤ C ρ j , j ≥ 0. (69) For 1 ≤ i ≤ T 1\le i\le T 1 ≤ i ≤ T , consider the squared observations u i 2 ( a ) = u i 2 + a ( u ^ i 2 − u i 2 ) u_i^2(a)=u_i^2+a(\widehat{u}_i^2-u_i^2) u i 2 ( a ) = u i 2 + a ( u i 2 − u i 2 ) , where 0 ≤ a ≤ 1 0\le a\le1 0 ≤ a ≤ 1 . Let s t ( a , θ ) s_t(a,\theta) s t ( a , θ ) denote the resulting score function. Then, with probability approaching one,
sup θ ∈ N sup 0 ≤ a ≤ 1 ∥ ∂ 2 s t ( a , θ ) ∂ a 2 ∥ ≤ C J t ( max 1 ≤ i ≤ T ∣ τ i τ ^ i − 1 ∣ ) 2 , 1 ≤ t ≤ T , (70) \sup_{\theta\in\mathcal{N}}\sup_{0\le a\le1}\left\|\frac{\partial^2s_t(a,\theta)}{\partial a^2}\right\|\le CJ_t\left(\max_{1\le i\le T}\left|\frac{\tau_i}{\widehat{\tau}_i}-1\right|\right)^2,\qquad1\le t\le T, \tag*{(70)} θ ∈ N sup 0 ≤ a ≤ 1 sup ∂ a 2 ∂ 2 s t ( a , θ ) ≤ C J t ( 1 ≤ i ≤ T max τ i τ i − 1 ) 2 , 1 ≤ t ≤ T , (70) where J t J_t J t is defined in (A.45). Moreover, the Hessian matrix for oracle process is almost surely continuous on N \mathcal{N} N and satisfies
E sup θ ∈ N ∥ ∂ 2 ℓ t ( θ ) ∂ θ ∂ θ ⊤ ∥ < ∞ . (71) E\sup_{\theta\in\mathcal{N}}\left\|\frac{\partial^2\ell_t(\theta)}{\partial\theta\partial\theta^\top}\right\|<\infty. \tag*{(71)} E θ ∈ N sup ∂ θ ∂ θ ⊤ ∂ 2 ℓ t ( θ ) < ∞. (71)
Proof. We use the partial-derivative bounds of g g g and f f f in (A.4)–(A.6) and (A.11), together with Lemma 1. First, taking first order derivative with respect to the squared return u i 2 u_i^2 u i 2 gives
∂ h t ∂ ( u i 2 ) = ∂ g 1 ∂ v 1 { i = t } + ∂ g ∂ b { α 1 { i = t − 1 } + β ∂ h t − 1 ∂ ( u i 2 ) } . (72) \frac{\partial h_t}{\partial(u_i^2)}=\frac{\partial g_1}{\partial v}\mathbb{1}\{i=t\}+\frac{\partial g}{\partial b}\left\{\alpha\mathbb{1}\{i=t-1\}+\beta\frac{\partial h_{t-1}}{\partial(u_i^2)}\right\}. \tag*{(72)} ∂ ( u i 2 ) ∂ h t = ∂ v ∂ g 1 1 { i = t } + ∂ b ∂ g { α 1 { i = t − 1 } + β ∂ ( u i 2 ) ∂ h t − 1 } . (72)
A change in u i 2 u_i^2 u i 2 affects h i h_i h i directly and h i + 1 h_{i+1} h i + 1 through the terms α u i 2 \alpha u_i^2 α u i 2 and β h i \beta h_i β h i . For t ≥ i + 2 t\ge i+2 t ≥ i + 2 , its effect is multiplied at each step by β ( ∂ g / ∂ b ) ≤ β ˉ \beta(\partial g/\partial b)\le\bar{\beta} β ( ∂ g / ∂ b ) ≤ β ˉ . By (A.2) and (A.4), ∣ ∂ g / ∂ v ∣ ≤ C |\partial g/\partial v|\le C ∣ ∂ g / ∂ v ∣ ≤ C and 0 ≤ β ( ∂ g / ∂ b ) ≤ β ˉ 0\le\beta(\partial g/\partial b)\le\bar{\beta} 0 ≤ β ( ∂ g / ∂ b ) ≤ β ˉ , so iterating (A.57) gives
sup θ ∈ Θ ∣ ∂ h t ( θ ) ∂ ( u i 2 ) ∣ ≤ C β ˉ t − i , t ≥ i . (73) \sup_{\theta\in\Theta}\left|\frac{\partial h_t(\theta)}{\partial(u_i^2)}\right|\le C\bar{\beta}^{t-i},\qquad t\ge i. \tag*{(73)} θ ∈ Θ sup ∂ ( u i 2 ) ∂ h t ( θ ) ≤ C β ˉ t − i , t ≥ i . (73)
and hence ∥ sup θ ∈ N ∣ u i 2 { ∂ h t ( θ ) / ∂ ( u i 2 ) } ∥ q ≤ C ∥ u i 2 ∥ q β ˉ t − i ≤ C ρ t − i \lVert\sup_{\theta\in\mathcal{N}}\lvert u_i^2\{\partial h_t(\theta)/\partial(u_i^2)\}\rVert_q\le C\lVert u_i^2\rVert_q\bar{\beta}^{t-i}\le C\rho^{t-i} ∥ sup θ ∈ N ∣ u i 2 { ∂ h t ( θ ) / ∂ ( u i 2 )} ∥ q ≤ C ∥ u i 2 ∥ q β ˉ t − i ≤ C ρ t − i . For t ≥ i + 2 t\ge i+2 t ≥ i + 2 ,
∂ 2 h t ∂ θ ∂ ( u i 2 ) = β ∂ g ∂ b ∂ 2 h t − 1 ∂ θ ∂ ( u i 2 ) + ∂ ∂ θ ( β ∂ g ∂ b ) ∂ h t − 1 ∂ ( u i 2 ) , (74) \frac{\partial^2 h_t}{\partial\theta\partial(u_i^2)}=\beta\frac{\partial g}{\partial b}\frac{\partial^2h_{t-1}}{\partial\theta\partial(u_i^2)}+\frac{\partial}{\partial\theta}\left(\beta\frac{\partial g}{\partial b}\right)\frac{\partial h_{t-1}}{\partial(u_i^2)}, \tag*{(74)} ∂ θ ∂ ( u i 2 ) ∂ 2 h t = β ∂ b ∂ g ∂ θ ∂ ( u i 2 ) ∂ 2 h t − 1 + ∂ θ ∂ ( β ∂ b ∂ g ) ∂ ( u i 2 ) ∂ h t − 1 , (74)
where by the chain rule and (A.5),
∂ ∂ θ ( β ∂ g ∂ b ) = ∂ g ∂ b e β + β ( ∂ 2 g ∂ b 2 ∂ b t − 1 ∂ θ + ∂ 2 g ∂ b ∂ φ e φ ) , sup θ ∈ Θ ∥ ∂ ∂ θ ( β ∂ g ∂ b ) ∥ ≤ C U t . (75) \frac{\partial}{\partial\theta}\left(\beta\frac{\partial g}{\partial b}\right)=\frac{\partial g}{\partial b}e_{\beta}+\beta\left(\frac{\partial^2g}{\partial b^2}\frac{\partial b_{t-1}}{\partial\theta}+\frac{\partial^2g}{\partial b\partial\varphi}e_{\varphi}\right),\qquad\sup_{\theta\in\Theta}\left\lVert\frac{\partial}{\partial\theta}\left(\beta\frac{\partial g}{\partial b}\right)\right\rVert\le CU_t. \tag*{(75)} ∂ θ ∂ ( β ∂ b ∂ g ) = ∂ b ∂ g e β + β ( ∂ b 2 ∂ 2 g ∂ θ ∂ b t − 1 + ∂ b ∂ φ ∂ 2 g e φ ) , θ ∈ Θ sup ∂ θ ∂ ( β ∂ b ∂ g ) ≤ C U t . (75)
Combining (A.58)–(A.60) shows that for t ≥ i + 2 t\ge i+2 t ≥ i + 2 ,
sup θ ∈ N ∥ u i 2 ∂ 2 h t ( θ ) ∂ θ ∂ ( u i 2 ) ∥ ≤ β ˉ sup θ ∈ N ∥ u i 2 ∂ 2 h t − 1 ( θ ) ∂ θ ∂ ( u i 2 ) ∥ + C U t β ˉ t − i , \sup_{\theta\in\mathcal{N}}\left\lVert u_i^2\frac{\partial^2h_t(\theta)}{\partial\theta\partial(u_i^2)}\right\rVert\le\bar{\beta}\sup_{\theta\in\mathcal{N}}\left\lVert u_i^2\frac{\partial^2h_{t-1}(\theta)}{\partial\theta\partial(u_i^2)}\right\rVert+CU_t\bar{\beta}^{t-i}, θ ∈ N sup u i 2 ∂ θ ∂ ( u i 2 ) ∂ 2 h t ( θ ) ≤ β ˉ θ ∈ N sup u i 2 ∂ θ ∂ ( u i 2 ) ∂ 2 h t − 1 ( θ ) + C U t β ˉ t − i ,
and applying Lemma 1(i)(a) (with Hölder’s inequality ∥ U t u i 2 ∥ q / 2 ≤ ∥ U t ∥ q ∥ u i 2 ∥ q ≤ C \lVert U_tu_i^2\rVert_{q/2}\le\lVert U_t\rVert_q\lVert u_i^2\rVert_q\le C ∥ U t u i 2 ∥ q /2 ≤ ∥ U t ∥ q ∥ u i 2 ∥ q ≤ C ) yields
∥ sup θ ∈ N ∥ u i 2 ∂ 2 h t ( θ ) ∂ θ ∂ ( u i 2 ) ∥ ∥ q / 2 ≤ C ρ t − i , t ≥ i . (76) \left\lVert\sup_{\theta\in\mathcal{N}}\left\lVert u_i^2\frac{\partial^2h_t(\theta)}{\partial\theta\partial(u_i^2)}\right\rVert\right\rVert_{q/2}\le C\rho^{t-i},\qquad t\ge i. \tag*{(76)} θ ∈ N sup u i 2 ∂ θ ∂ ( u i 2 ) ∂ 2 h t ( θ ) q /2 ≤ C ρ t − i , t ≥ i . (76)
Differentiating the score (A.28) with respect to u i 2 u_i^2 u i 2 gives
∂ s t ( θ ) ∂ ( u i 2 ) = ∂ f ∂ h ∂ 2 h t ∂ θ ∂ ( u i 2 ) + ( ∂ 2 f ∂ h 2 ∂ h t ∂ θ + ∂ 2 f ∂ φ ∂ h e φ ) ∂ h t ∂ ( u i 2 ) + ( ∂ 2 f ∂ h ∂ v ∂ h t ∂ θ + ∂ 2 f ∂ φ ∂ v e φ ) 1 { i = t } . (77) \frac{\partial s_t(\theta)}{\partial(u_i^2)}=\frac{\partial f}{\partial h}\frac{\partial^2h_t}{\partial\theta\partial(u_i^2)}+\left(\frac{\partial^2f}{\partial h^2}\frac{\partial h_t}{\partial\theta}+\frac{\partial^2f}{\partial\varphi\partial h}e_{\varphi}\right)\frac{\partial h_t}{\partial(u_i^2)}+\left(\frac{\partial^2f}{\partial h\partial v}\frac{\partial h_t}{\partial\theta}+\frac{\partial^2f}{\partial\varphi\partial v}e_{\varphi}\right)\mathbf{1}\{i=t\}. \tag*{(77)} ∂ ( u i 2 ) ∂ s t ( θ ) = ∂ h ∂ f ∂ θ ∂ ( u i 2 ) ∂ 2 h t + ( ∂ h 2 ∂ 2 f ∂ θ ∂ h t + ∂ φ ∂ h ∂ 2 f e φ ) ∂ ( u i 2 ) ∂ h t + ( ∂ h ∂ v ∂ 2 f ∂ θ ∂ h t + ∂ φ ∂ v ∂ 2 f e φ ) 1 { i = t } . (77)
Applying the bounds for the partial derivatives of f f f on N \mathcal{N} N in (A.11) to (A.62) gives
∥ u i 2 ∂ s t ( θ 0 ) ∂ ( u i 2 ) ∥ ≤ C [ ∥ u i 2 ∂ 2 h t ( θ 0 ) ∂ θ ∂ ( u i 2 ) ∥ + ( 1 + ∥ ∂ h t ( θ 0 ) ∂ θ ∥ ) { ∣ u i 2 ∂ h t ( θ 0 ) ∂ ( u i 2 ) ∣ + u i 2 1 { i = t } } ] . \left\lVert u_i^2\frac{\partial s_t(\theta_0)}{\partial(u_i^2)}\right\rVert\le C\left[\left\lVert u_i^2\frac{\partial^2h_t(\theta_0)}{\partial\theta\partial(u_i^2)}\right\rVert+\left(1+\left\lVert\frac{\partial h_t(\theta_0)}{\partial\theta}\right\rVert\right)\left\{\left\lvert u_i^2\frac{\partial h_t(\theta_0)}{\partial(u_i^2)}\right\rvert+u_i^2\mathbf{1}\{i=t\}\right\}\right]. u i 2 ∂ ( u i 2 ) ∂ s t ( θ 0 ) ≤ C [ u i 2 ∂ θ ∂ ( u i 2 ) ∂ 2 h t ( θ 0 ) + ( 1 + ∂ θ ∂ h t ( θ 0 ) ) { u i 2 ∂ ( u i 2 ) ∂ h t ( θ 0 ) + u i 2 1 { i = t } } ] .
Taking the L q / 2 L^{q/2} L q /2 norm and using (A.58), (A.61), Lemma 2, and Hölder’s inequality proves (A.54). Next consider the interpolated observations u i 2 ( a ) = u i 2 + a ( u ~ i 2 − u i 2 ) u_i^2(a)=u_i^2+a(\widetilde{u}_i^2-u_i^2) u i 2 ( a ) = u i 2 + a ( u i 2 − u i 2 ) on the event δ T : = max 1 ≤ i ≤ T ∣ τ i / τ ^ i − 1 ∣ ≤ 1 / 2 \delta_T:=\max_{1\le i\le T}\lvert\tau_i/\widehat{\tau}_i-1\rvert\le1/2 δ T := max 1 ≤ i ≤ T ∣ τ i / τ i − 1 ∣ ≤ 1/2 , and let h t ( a , θ ) h_t(a,\theta) h t ( a , θ ) and b t − 1 ( a , θ ) b_{t-1}(a,\theta) b t − 1 ( a , θ ) denote the resulting processes evaluated at ( a , θ ) (a,\theta) ( a , θ ) . Dividing the Lipschitz bounds (A.44) in Lemma 7 by ∣ a ′ − a ∣ \lvert a'-a\rvert ∣ a ′ − a ∣ and letting a ′ → a a'\to a a ′ → a gives the uniform first-order path bounds
sup θ ∈ Θ ∣ ∂ h t ( a , θ ) ∂ a ∣ ≤ C δ T U t , sup θ ∈ Θ ∥ ∂ 2 h t ( a , θ ) ∂ a ∂ θ ∥ ≤ C δ T G 1 , t , (78) \sup_{\theta\in\Theta}\left\lvert\frac{\partial h_t(a,\theta)}{\partial a}\right\rvert\le C\delta_TU_t,\qquad\sup_{\theta\in\Theta}\left\lVert\frac{\partial^2h_t(a,\theta)}{\partial a\partial\theta}\right\rVert\le C\delta_TG_{1,t}, \tag*{(78)} θ ∈ Θ sup ∂ a ∂ h t ( a , θ ) ≤ C δ T U t , θ ∈ Θ sup ∂ a ∂ θ ∂ 2 h t ( a , θ ) ≤ C δ T G 1 , t , (78)
which in turn imply
sup θ ∈ Θ ∣ ∂ b t − 1 ( a , θ ) ∂ a ∣ + ∣ ∂ u t 2 ( a ) ∂ a ∣ ≤ C δ T U t , sup θ ∈ Θ ∥ ∂ 2 b t − 1 ( a , θ ) ∂ a ∂ θ ∥ ≤ C δ T ( U t + G 1 , t − 1 ) . (79) \sup_{\theta\in\Theta}\left\lvert\frac{\partial b_{t-1}(a,\theta)}{\partial a}\right\rvert+\left\lvert\frac{\partial u_t^2(a)}{\partial a}\right\rvert\le C\delta_TU_t,\qquad\sup_{\theta\in\Theta}\left\lVert\frac{\partial^2b_{t-1}(a,\theta)}{\partial a\partial\theta}\right\rVert\le C\delta_T(U_t+G_{1,t-1}). \tag*{(79)} θ ∈ Θ sup ∂ a ∂ b t − 1 ( a , θ ) + ∂ a ∂ u t 2 ( a ) ≤ C δ T U t , θ ∈ Θ sup ∂ a ∂ θ ∂ 2 b t − 1 ( a , θ ) ≤ C δ T ( U t + G 1 , t − 1 ) . (79)
Since u t 2 ( a ) u_t^2(a) u t 2 ( a ) is linear in a a a , we can obtain ∂ 2 u t 2 ( a ) / ∂ a 2 = 0 \partial^2u_t^2(a)/\partial a^2=0 ∂ 2 u t 2 ( a ) / ∂ a 2 = 0 , ∂ 2 b t − 1 / ∂ a 2 = β ( ∂ 2 h t − 1 / ∂ a 2 ) \partial^2b_{t-1}/\partial a^2=\beta(\partial^2h_{t-1}/\partial a^2) ∂ 2 b t − 1 / ∂ a 2 = β ( ∂ 2 h t − 1 / ∂ a 2 ) and ∂ 3 b t − 1 / ( ∂ a 2 ∂ θ ) = e β ( ∂ 2 h t − 1 / ∂ a 2 ) + β { ∂ 3 h t − 1 / ( ∂ a 2 ∂ θ ) } \partial^3b_{t-1}/(\partial a^2\partial\theta)=e_{\beta}(\partial^2h_{t-1}/\partial a^2)+\beta\{\partial^3h_{t-1}/(\partial a^2\partial\theta)\} ∂ 3 b t − 1 / ( ∂ a 2 ∂ θ ) = e β ( ∂ 2 h t − 1 / ∂ a 2 ) + β { ∂ 3 h t − 1 / ( ∂ a 2 ∂ θ )} . Differentiating h t ( a , θ ) = g ( b t − 1 ( a , θ ) , u t 2 ( a ) ; φ ) h_t(a,\theta)=g(b_{t-1}(a,\theta),u_t^2(a);\varphi) h t ( a , θ ) = g ( b t − 1 ( a , θ ) , u t 2 ( a ) ; φ ) twice with respect to a a a gives
∂ 2 h t ∂ a 2 = β ∂ g ∂ b ∂ 2 h t − 1 ∂ a 2 + ∂ 2 g ∂ b 2 ( ∂ b t − 1 ∂ a ) 2 + 2 ∂ 2 g ∂ b ∂ v ∂ b t − 1 ∂ a ∂ u t 2 ( a ) ∂ a + ∂ 2 g ∂ v 2 ( ∂ u t 2 ( a ) ∂ a ) 2 . \frac{\partial^2h_t}{\partial a^2}=\beta\frac{\partial g}{\partial b}\frac{\partial^2h_{t-1}}{\partial a^2}+\frac{\partial^2g}{\partial b^2}\left(\frac{\partial b_{t-1}}{\partial a}\right)^2+2\frac{\partial^2g}{\partial b\partial v}\frac{\partial b_{t-1}}{\partial a}\frac{\partial u_t^2(a)}{\partial a}+\frac{\partial^2g}{\partial v^2}\left(\frac{\partial u_t^2(a)}{\partial a}\right)^2. ∂ a 2 ∂ 2 h t = β ∂ b ∂ g ∂ a 2 ∂ 2 h t − 1 + ∂ b 2 ∂ 2 g ( ∂ a ∂ b t − 1 ) 2 + 2 ∂ b ∂ v ∂ 2 g ∂ a ∂ b t − 1 ∂ a ∂ u t 2 ( a ) + ∂ v 2 ∂ 2 g ( ∂ a ∂ u t 2 ( a ) ) 2 .
Bounding the derivatives of g g g via (A.2) and (A.5) and applying (A.64) yields
sup θ ∈ N ∣ ∂ 2 h t ( a , θ ) ∂ a 2 ∣ ≤ β ˉ sup θ ∈ N ∣ ∂ 2 h t − 1 ( a , θ ) ∂ a 2 ∣ + C δ T 2 U t 2 , \sup_{\theta\in\mathcal{N}}\left|\frac{\partial^2 h_t(a,\theta)}{\partial a^2}\right|\le\bar{\beta}\sup_{\theta\in\mathcal{N}}\left|\frac{\partial^2 h_{t-1}(a,\theta)}{\partial a^2}\right|+C\delta_T^2U_t^2, θ ∈ N sup ∂ a 2 ∂ 2 h t ( a , θ ) ≤ β ˉ θ ∈ N sup ∂ a 2 ∂ 2 h t − 1 ( a , θ ) + C δ T 2 U t 2 ,
so Lemma 1(i)(b) gives
sup θ ∈ N ∣ ∂ 2 h t ( a , θ ) ∂ a 2 ∣ ≤ C δ T 2 G 1 , t , sup θ ∈ N ∣ ∂ 2 b t − 1 ( a , θ ) ∂ a 2 ∣ ≤ C δ T 2 G 1 , t − 1 . (80) \sup_{\theta\in\mathcal{N}}\left|\frac{\partial^2 h_t(a,\theta)}{\partial a^2}\right|\le C\delta_T^2G_{1,t},\qquad\sup_{\theta\in\mathcal{N}}\left|\frac{\partial^2 b_{t-1}(a,\theta)}{\partial a^2}\right|\le C\delta_T^2G_{1,t-1}. \tag*{(80)} θ ∈ N sup ∂ a 2 ∂ 2 h t ( a , θ ) ≤ C δ T 2 G 1 , t , θ ∈ N sup ∂ a 2 ∂ 2 b t − 1 ( a , θ ) ≤ C δ T 2 G 1 , t − 1 . (80)
Similarly, differentiating ∂ h t / ∂ θ = ( ∂ g / ∂ b ) ( ∂ b t − 1 / ∂ θ ) + ( ∂ g / ∂ φ ) e φ \partial h_t/\partial\theta=(\partial g/\partial b)(\partial b_{t-1}/\partial\theta)+(\partial g/\partial\varphi)e_\varphi ∂ h t / ∂ θ = ( ∂ g / ∂ b ) ( ∂ b t − 1 / ∂ θ ) + ( ∂ g / ∂ φ ) e φ twice with respect to a a a gives
∂ 3 h t ∂ a 2 ∂ θ = β ∂ g ∂ b ∂ 3 h t − 1 ∂ a 2 ∂ θ + ∂ g ∂ b ∂ 2 h t − 1 ∂ a 2 e φ + 2 ( ∂ 2 g ∂ b 2 ∂ b t − 1 ∂ a + ∂ 2 g ∂ b ∂ v ∂ u t 2 ( a ) ∂ a ) ∂ b t − 1 ∂ a ∂ θ + { ∂ 3 g ∂ b 3 ( ∂ b t − 1 ∂ a ) 2 + 2 ∂ 3 g ∂ b 2 ∂ v ∂ b t − 1 ∂ a ∂ u t 2 ( a ) ∂ a + ∂ 3 g ∂ b ∂ v 2 ( ∂ u t 2 ( a ) ∂ a ) 2 + ∂ 2 g ∂ b 2 ∂ 2 b t − 1 ∂ a 2 } ∂ b t − 1 ∂ θ + { ∂ 3 g ∂ b 2 ∂ φ ( ∂ b t − 1 ∂ a ) 2 + 2 ∂ 3 g ∂ b ∂ v ∂ φ ∂ b t − 1 ∂ a ∂ u t 2 ( a ) ∂ a + ∂ 3 g ∂ v 2 ∂ φ ( ∂ u t 2 ( a ) ∂ a ) 2 + ∂ 2 g ∂ b ∂ φ ∂ 2 b t − 1 ∂ a 2 } e φ . \begin{aligned}
\frac{\partial^3h_t}{\partial a^2\partial\theta}
={}&\beta\frac{\partial g}{\partial b}\frac{\partial^3h_{t-1}}{\partial a^2\partial\theta}
+\frac{\partial g}{\partial b}\frac{\partial^2h_{t-1}}{\partial a^2}e_\varphi
+2\left(\frac{\partial^2g}{\partial b^2}\frac{\partial b_{t-1}}{\partial a}
+\frac{\partial^2g}{\partial b\partial v}\frac{\partial u_t^2(a)}{\partial a}\right)\frac{\partial b_{t-1}}{\partial a\partial\theta}\\
&+\left\{\frac{\partial^3g}{\partial b^3}\left(\frac{\partial b_{t-1}}{\partial a}\right)^2
+2\frac{\partial^3g}{\partial b^2\partial v}\frac{\partial b_{t-1}}{\partial a}\frac{\partial u_t^2(a)}{\partial a}
+\frac{\partial^3g}{\partial b\partial v^2}\left(\frac{\partial u_t^2(a)}{\partial a}\right)^2
+\frac{\partial^2g}{\partial b^2}\frac{\partial^2b_{t-1}}{\partial a^2}\right\}\frac{\partial b_{t-1}}{\partial\theta}\\
&+\left\{\frac{\partial^3g}{\partial b^2\partial\varphi}\left(\frac{\partial b_{t-1}}{\partial a}\right)^2
+2\frac{\partial^3g}{\partial b\partial v\partial\varphi}\frac{\partial b_{t-1}}{\partial a}\frac{\partial u_t^2(a)}{\partial a}
+\frac{\partial^3g}{\partial v^2\partial\varphi}\left(\frac{\partial u_t^2(a)}{\partial a}\right)^2
+\frac{\partial^2g}{\partial b\partial\varphi}\frac{\partial^2b_{t-1}}{\partial a^2}\right\}e_\varphi.
\end{aligned} ∂ a 2 ∂ θ ∂ 3 h t = β ∂ b ∂ g ∂ a 2 ∂ θ ∂ 3 h t − 1 + ∂ b ∂ g ∂ a 2 ∂ 2 h t − 1 e φ + 2 ( ∂ b 2 ∂ 2 g ∂ a ∂ b t − 1 + ∂ b ∂ v ∂ 2 g ∂ a ∂ u t 2 ( a ) ) ∂ a ∂ θ ∂ b t − 1 + { ∂ b 3 ∂ 3 g ( ∂ a ∂ b t − 1 ) 2 + 2 ∂ b 2 ∂ v ∂ 3 g ∂ a ∂ b t − 1 ∂ a ∂ u t 2 ( a ) + ∂ b ∂ v 2 ∂ 3 g ( ∂ a ∂ u t 2 ( a ) ) 2 + ∂ b 2 ∂ 2 g ∂ a 2 ∂ 2 b t − 1 } ∂ θ ∂ b t − 1 + { ∂ b 2 ∂ φ ∂ 3 g ( ∂ a ∂ b t − 1 ) 2 + 2 ∂ b ∂ v ∂ φ ∂ 3 g ∂ a ∂ b t − 1 ∂ a ∂ u t 2 ( a ) + ∂ v 2 ∂ φ ∂ 3 g ( ∂ a ∂ u t 2 ( a ) ) 2 + ∂ b ∂ φ ∂ 2 g ∂ a 2 ∂ 2 b t − 1 } e φ .
Applying (A.5)–(A.6), sup θ ∈ N ∥ ∂ θ b t − 1 ( a , θ ) ∥ ≤ C U t \sup_{\theta\in\mathcal{N}}\|\partial_\theta b_{t-1}(a,\theta)\|\le CU_t sup θ ∈ N ∥ ∂ θ b t − 1 ( a , θ ) ∥ ≤ C U t , (A.64), and (A.65) yields
sup θ ∈ N ∥ ∂ 3 h t ( a , θ ) ∂ a 2 ∂ θ ∥ ≤ β ˉ sup θ ∈ N ∥ ∂ 3 h t − 1 ( a , θ ) ∂ a 2 ∂ θ ∥ + C δ T 2 { U t 3 + U t G 1 , t − 1 } , \sup_{\theta\in\mathcal{N}}\left\|\frac{\partial^3h_t(a,\theta)}{\partial a^2\partial\theta}\right\|
\le\bar{\beta}\sup_{\theta\in\mathcal{N}}\left\|\frac{\partial^3h_{t-1}(a,\theta)}{\partial a^2\partial\theta}\right\|
+C\delta_T^2\{U_t^3+U_tG_{1,t-1}\}, θ ∈ N sup ∂ a 2 ∂ θ ∂ 3 h t ( a , θ ) ≤ β ˉ θ ∈ N sup ∂ a 2 ∂ θ ∂ 3 h t − 1 ( a , θ ) + C δ T 2 { U t 3 + U t G 1 , t − 1 } ,
and Lemma 1(i)(b) gives
sup θ ∈ N ∥ ∂ 3 h t ( a , θ ) ∂ a 2 ∂ θ ∥ ≤ C δ T 2 G 2 , t . (81) \sup_{\theta\in\mathcal{N}}\left\|\frac{\partial^3h_t(a,\theta)}{\partial a^2\partial\theta}\right\|\le C\delta_T^2G_{2,t}. \tag*{(81)} θ ∈ N sup ∂ a 2 ∂ θ ∂ 3 h t ( a , θ ) ≤ C δ T 2 G 2 , t . (81)
Twice differentiating the score s t ( a , θ ) = ( ∂ f / ∂ h ) ( ∂ h t / ∂ θ ) + ( ∂ f / ∂ φ ) e φ s_t(a,\theta)=(\partial f/\partial h)(\partial h_t/\partial\theta)+(\partial f/\partial\varphi)e_\varphi s t ( a , θ ) = ( ∂ f / ∂ h ) ( ∂ h t / ∂ θ ) + ( ∂ f / ∂ φ ) e φ with respect to a a a and bounding the partial derivatives of f f f at ( u t 2 ( a ) , h t ( a , θ ) , φ ) (u_t^2(a),h_t(a,\theta),\varphi) ( u t 2 ( a ) , h t ( a , θ ) , φ ) by the bound in (A.11) gives
∥ ∂ 2 s t ( a , θ ) ∂ a 2 ∥ ≤ C [ ∥ ∂ 3 h t ( a , θ ) ∂ a 2 ∂ θ ∥ + ( ∣ ∂ h t ( a , θ ) ∂ a ∣ + ∣ u t 2 − u t 2 ∣ ) ∥ ∂ 2 h t ( a , θ ) ∂ a ∂ θ ∥ + { ∣ ∂ 2 h t ( a , θ ) ∂ a 2 ∣ + ( ∣ ∂ h t ( a , θ ) ∂ a ∣ + ∣ u t 2 − u t 2 ∣ ) 2 } ( 1 + ∥ ∂ h t ( a , θ ) ∂ θ ∥ ) ] . \begin{aligned}
\left\|\frac{\partial^2s_t(a,\theta)}{\partial a^2}\right\|
\le C\Bigg[&\left\|\frac{\partial^3h_t(a,\theta)}{\partial a^2\partial\theta}\right\|
+\left(\left|\frac{\partial h_t(a,\theta)}{\partial a}\right|+|u_t^2-u_t^2|\right)\left\|\frac{\partial^2h_t(a,\theta)}{\partial a\partial\theta}\right\|\\
&+\left\{\left|\frac{\partial^2h_t(a,\theta)}{\partial a^2}\right|
+\left(\left|\frac{\partial h_t(a,\theta)}{\partial a}\right|+|u_t^2-u_t^2|\right)^2\right\}
\left(1+\left\|\frac{\partial h_t(a,\theta)}{\partial\theta}\right\|\right)\Bigg].
\end{aligned} ∂ a 2 ∂ 2 s t ( a , θ ) ≤ C [ ∂ a 2 ∂ θ ∂ 3 h t ( a , θ ) + ( ∂ a ∂ h t ( a , θ ) + ∣ u t 2 − u t 2 ∣ ) ∂ a ∂ θ ∂ 2 h t ( a , θ ) + { ∂ a 2 ∂ 2 h t ( a , θ ) + ( ∂ a ∂ h t ( a , θ ) + ∣ u t 2 − u t 2 ∣ ) 2 } ( 1 + ∂ θ ∂ h t ( a , θ ) ) ] .
Substituting (A.63), (A.65), and (A.66) into the right-hand side yields
sup θ ∈ N sup 0 ≤ a ≤ 1 ∥ ∂ 2 s t ( a , θ ) ∂ a 2 ∥ ≤ C δ T 2 { G 2 , t + U t G 1 , t + U t 3 } ≤ C δ T 2 J t , \sup_{\theta\in\mathcal{N}}\sup_{0\le a\le1}\left\|\frac{\partial^2s_t(a,\theta)}{\partial a^2}\right\|
\le C\delta_T^2\{G_{2,t}+U_tG_{1,t}+U_t^3\}\le C\delta_T^2J_t, θ ∈ N sup 0 ≤ a ≤ 1 sup ∂ a 2 ∂ 2 s t ( a , θ ) ≤ C δ T 2 { G 2 , t + U t G 1 , t + U t 3 } ≤ C δ T 2 J t ,
which proves (A.55).
Finally, to prove (A.56) and the almost-sure continuity of the stationary Hessian on N \mathcal{N} N , we apply the local constant bounds for the partial derivatives of f f f in (A.11) to the explicit Hessian (A.29): for all θ ∈ N \theta\in\mathcal{N} θ ∈ N ,
∥ ∂ 2 ℓ t ( θ ) ∂ θ ∂ θ ′ ∥ ≤ C ( ∥ ∂ 2 h t ( θ ) ∂ θ ∂ θ ′ ∥ + ∥ ∂ h t ( θ ) ∂ θ ∥ 2 + ∥ ∂ h t ( θ ) ∂ θ ∥ + 1 ) ≤ C { G 1 , t + U t 2 } . \left\|\frac{\partial^2\ell_t(\theta)}{\partial\theta\partial\theta^\prime}\right\|
\le C\left(\left\|\frac{\partial^2h_t(\theta)}{\partial\theta\partial\theta^\prime}\right\|
+\left\|\frac{\partial h_t(\theta)}{\partial\theta}\right\|^2
+\left\|\frac{\partial h_t(\theta)}{\partial\theta}\right\|+1\right)
\le C\{G_{1,t}+U_t^2\}. ∂ θ ∂ θ ′ ∂ 2 ℓ t ( θ ) ≤ C ( ∂ θ ∂ θ ′ ∂ 2 h t ( θ ) + ∂ θ ∂ h t ( θ ) 2 + ∂ θ ∂ h t ( θ ) + 1 ) ≤ C { G 1 , t + U t 2 } .
Since G 1 , t , U t 2 ∈ L q / 2 ⊂ L 1 G_{1,t}, U_t^2 \in L^{q/2} \subset L^1 G 1 , t , U t 2 ∈ L q /2 ⊂ L 1 (q > 4 q > 4 q > 4 ), taking the supremum over θ ∈ N \theta\in\mathcal{N} θ ∈ N and taking expectations immediately yields (A.56). Moreover, for each integer m ≥ 1 m \ge1 m ≥ 1 , let h t , m ( θ ) h_{t,m}(\theta) h t , m ( θ ) denote the truncated volatility process at time t t t obtained by starting the recursion at time t − m t-m t − m from the fixed constant h t − m , 0 = ω > 0 h_{t-m,0} = \omega> 0 h t − m , 0 = ω > 0 . Because h t , m ( θ ) h_{t,m}(\theta) h t , m ( θ ) is a finite m m m -fold composition of the smooth function g g g , the maps θ ↦ h t , m ( θ ) \theta\mapsto h_{t,m}(\theta) θ ↦ h t , m ( θ ) , θ ↦ ∂ θ h t , m ( θ ) \theta\mapsto\partial_\theta h_{t,m}(\theta) θ ↦ ∂ θ h t , m ( θ ) , and θ ↦ ∂ θ θ T 2 h t , m ( θ ) \theta\mapsto\partial_{\theta\theta^\mathsf{T}}^2 h_{t,m}(\theta) θ ↦ ∂ θ θ T 2 h t , m ( θ ) are continuous on Θ \Theta Θ . By stationarity and (A.37)–(A.39) in Lemma 6,
∑ m = 1 ∞ E [ sup θ ∈ Θ { ∣ h t , m ( θ ) − h t ( θ ) ∣ + ∥ ∂ h t , m ( θ ) ∂ θ − ∂ h t ( θ ) ∂ θ ∥ + ∥ ∂ 2 h t , m ( θ ) ∂ θ ∂ θ T − ∂ 2 h t ( θ ) ∂ θ ∂ θ T ∥ } ] ≤ C ∑ m = 1 ∞ ρ m < ∞ . \sum_{m=1}^{\infty} E\left[\sup_{\theta\in\Theta}\left\{\left|h_{t,m}(\theta)-h_t(\theta)\right|+\left\|\frac{\partial h_{t,m}(\theta)}{\partial\theta}-\frac{\partial h_t(\theta)}{\partial\theta}\right\|+\left\|\frac{\partial^2 h_{t,m}(\theta)}{\partial\theta\partial\theta^\mathsf{T}}-\frac{\partial^2 h_t(\theta)}{\partial\theta\partial\theta^\mathsf{T}}\right\|\right\}\right]\le C\sum_{m=1}^{\infty}\rho^m<\infty. m = 1 ∑ ∞ E [ θ ∈ Θ sup { ∣ h t , m ( θ ) − h t ( θ ) ∣ + ∂ θ ∂ h t , m ( θ ) − ∂ θ ∂ h t ( θ ) + ∂ θ ∂ θ T ∂ 2 h t , m ( θ ) − ∂ θ ∂ θ T ∂ 2 h t ( θ ) } ] ≤ C m = 1 ∑ ∞ ρ m < ∞.
By the monotone convergence theorem, the nonnegative series inside the expectation converges almost surely, which implies that h t , m ( θ ) h_{t,m}(\theta) h t , m ( θ ) and its first and second parameter derivatives converge uniformly on Θ \Theta Θ almost surely as m → ∞ m \to\infty m → ∞ to h t ( θ ) h_t(\theta) h t ( θ ) and its parameter derivatives. Being the uniform limits of continuous functions, θ ↦ h t ( θ ) \theta\mapsto h_t(\theta) θ ↦ h t ( θ ) , θ ↦ ∂ θ h t ( θ ) \theta\mapsto\partial_{\theta}h_t(\theta) θ ↦ ∂ θ h t ( θ ) , and θ ↦ ∂ θ θ T 2 h t ( θ ) \theta\mapsto\partial_{\theta\theta^\mathsf{T}}^2h_t(\theta) θ ↦ ∂ θ θ T 2 h t ( θ ) are almost surely continuous on Θ \Theta Θ . Substituting these continuous processes into (A.29) establishes that θ ↦ ∂ 2 ℓ t ( θ ) / ( ∂ θ ∂ θ T ) \theta\mapsto\partial^2\ell_t(\theta)/(\partial\theta\partial\theta^\mathsf{T}) θ ↦ ∂ 2 ℓ t ( θ ) / ( ∂ θ ∂ θ T ) is almost surely continuous on N \mathcal{N} N . ■
Recall that ∂ s t ( θ 0 ) / ∂ ( u i 2 ) \partial s_t(\theta_0)/\partial(u_i^2) ∂ s t ( θ 0 ) / ∂ ( u i 2 ) from Lemma 8 measures how changing u i 2 u_i^2 u i 2 changes the score at time t t t . The effect decreases geometrically as t − i t-i t − i increases. Define the mean coefficient of the first-step error by
C τ = E { ∑ j = 0 ∞ ∂ s t ( θ 0 ) ∂ ( u t − j 2 ) u t − j 2 } . (82) C_\tau=E\left\{\sum_{j=0}^{\infty}\frac{\partial s_t(\theta_0)}{\partial(u_{t-j}^2)}u_{t-j}^2\right\}. \tag*{(82)} C τ = E { j = 0 ∑ ∞ ∂ ( u t − j 2 ) ∂ s t ( θ 0 ) u t − j 2 } . (82)
The series converges in L p L^p L p , with p = q / 2 > 2 p=q/2>2 p = q /2 > 2 , by Lemma 8 . The next lemma controls the effect of u i 2 u_i^2 u i 2 to the score at t = i , i + 1 , … t=i,i+1,\ldots t = i , i + 1 , … . Define A i = ∑ j ≥ 0 u i 2 ( ∂ s i + j ( θ 0 ) / ∂ ( u i 2 ) ) A_i=\sum_{j\ge0}u_i^2(\partial s_{i+j}(\theta_0)/\partial(u_i^2)) A i = ∑ j ≥ 0 u i 2 ( ∂ s i + j ( θ 0 ) / ∂ ( u i 2 )) . Although A i A_i A i depends on future innovations, the effect of a distant innovation decreases geometrically.
Lemma 9. Under the conditions of Lemma 8 , A i ∈ L p A_i\in L^p A i ∈ L p , where p = q / 2 > 2 p=q/2>2 p = q /2 > 2 , E A i = C τ EA_i=C_\tau E A i = C τ , and
∥ A k − A k ∗ ∥ 2 ≤ C ρ ∣ k ∣ , k ∈ Z , \|A_k-A_k^*\|_2\le C\rho^{|k|},\qquad k\in\mathbb{Z}, ∥ A k − A k ∗ ∥ 2 ≤ C ρ ∣ k ∣ , k ∈ Z , for some C < ∞ C<\infty C < ∞ and ρ ∈ ( 0 , 1 ) \rho\in(0,1) ρ ∈ ( 0 , 1 ) . Here A k ∗ A_k^* A k ∗ is obtained by replacing only ϵ 0 \epsilon_0 ϵ 0 by an independent copy.
Proof. Let W i , j = u i 2 ( ∂ s i + j ( θ 0 ) / ∂ ( u i 2 ) ) W_{i,j}=u_i^2(\partial s_{i+j}(\theta_0)/\partial(u_i^2)) W i , j = u i 2 ( ∂ s i + j ( θ 0 ) / ∂ ( u i 2 )) . Lemma 8 gives ∥ W i , j ∥ p ≤ C ϱ j \|W_{i,j}\|_p\le C\varrho^j ∥ W i , j ∥ p ≤ C ϱ j , so its summation is also L p L^p L p . By stationarity, E W i , j = E ( u t − j 2 ( ∂ s t ( θ 0 ) / ∂ ( u t − j 2 ) ) ) EW_{i,j}=E(u_{t-j}^2(\partial s_t(\theta_0)/\partial(u_{t-j}^2))) E W i , j = E ( u t − j 2 ( ∂ s t ( θ 0 ) / ∂ ( u t − j 2 ))) , and summing over j j j gives E A i = C τ EA_i=C_\tau E A i = C τ . If k < 0 k<0 k < 0 , replacing ϵ 0 \epsilon_0 ϵ 0 leaves W k , j W_{k,j} W k , j unchanged for j < − k j<-k j < − k . Thus
∥ A k − A k ∗ ∥ 2 ≤ 2 ∑ j ≥ − k ∥ W k , j ∥ 2 ≤ C ϱ ∣ k ∣ . \|A_k-A_k^*\|_2\le2\sum_{j\ge-k}\|W_{k,j}\|_2\le C\varrho^{|k|}. ∥ A k − A k ∗ ∥ 2 ≤ 2 j ≥ − k ∑ ∥ W k , j ∥ 2 ≤ C ϱ ∣ k ∣ .
For k ≥ 0 k\ge0 k ≥ 0 , we use the observation-derivative recursions (A.57) and (A.59) in the proof of Lemma 8 , evaluated at θ = θ 0 \theta=\theta_0 θ = θ 0 . By (A.58) and (A.61), for some γ ∈ ( β ˉ , 1 ) \gamma\in(\bar{\beta},1) γ ∈ ( β ˉ , 1 ) ,
∥ ∂ h k + j ( θ 0 ) ∂ ( u k 2 ) ∥ ≤ C γ j , ∥ ∂ 2 h k + j ( θ 0 ) ∂ θ ∂ ( u k 2 ) ∥ q ≤ C γ j , j ≥ 0. (83) \left\|\frac{\partial h_{k+j}(\theta_0)}{\partial(u_k^2)}\right\|\le C\gamma^j,\qquad\left\|\frac{\partial^2 h_{k+j}(\theta_0)}{\partial\theta\,\partial(u_k^2)}\right\|_q\le C\gamma^j,\qquad j\ge0. \tag*{(83)} ∂ ( u k 2 ) ∂ h k + j ( θ 0 ) ≤ C γ j , ∂ θ ∂ ( u k 2 ) ∂ 2 h k + j ( θ 0 ) q ≤ C γ j , j ≥ 0. (83)
Here the second bound follows directly from iterating (A.59) in L q L^q L q using ∥ ∂ θ { β ( ∂ g / ∂ b ) } ∥ q ≤ C ∥ U k + j ∥ q ≤ C \|\partial_\theta\{\beta(\partial g/\partial b)\}\|_q\le C\|U_{k+j}\|_q\le C ∥ ∂ θ { β ( ∂ g / ∂ b )} ∥ q ≤ C ∥ U k + j ∥ q ≤ C . Let a superscript ∗ * ∗ denote a quantity evaluated using the single-shock coupled process (where only ϵ 0 \epsilon_0 ϵ 0 is replaced by ϵ 0 ∗ \epsilon_0^* ϵ 0 ∗ ). By Lemma 4 and (A.21) in Lemma 1(ii)(a), for some η ∈ ( 0 , 1 ) \eta\in(0,1) η ∈ ( 0 , 1 ) and all t ≥ 1 t \ge1 t ≥ 1 ,
∥ u t 2 − ( u t ∗ ) 2 ∥ q 2 + ∥ h t ( θ 0 ) − h t ∗ ( θ 0 ) ∥ q ≤ C η t , ∥ ∂ h t ( θ 0 ) ∂ θ − ∂ h t ∗ ( θ 0 ) ∂ θ ∥ q / 2 ≤ C η t . (84) \lVert u_t^2-(u_t^*)^2\rVert_q^2+\lVert h_t(\theta_0)-h_t^*(\theta_0)\rVert_q \le C\eta^t,\qquad\left\lVert\frac{\partial h_t(\theta_0)}{\partial\theta}-\frac{\partial h_t^*(\theta_0)}{\partial\theta}\right\rVert_{q/2}\le C\eta^t. \tag*{(84)} ∥ u t 2 − ( u t ∗ ) 2 ∥ q 2 + ∥ h t ( θ 0 ) − h t ∗ ( θ 0 ) ∥ q ≤ C η t , ∂ θ ∂ h t ( θ 0 ) − ∂ θ ∂ h t ∗ ( θ 0 ) q /2 ≤ C η t . (84)
Applying the Lipschitz bounds (A.7) to (A.60) and using (A.69) with Hölder’s inequality yields, for all t ≥ 1 t \ge1 t ≥ 1 ,
∥ β 0 ∂ g ∂ b − β 0 ( ∂ g ∂ b ) ∗ ∥ q ≤ C η t , ∥ ∂ ∂ θ ( β ∂ g ∂ b ) − [ ∂ ∂ θ ( β ∂ g ∂ b ) ] ∗ ∥ q / 2 ≤ C η t . (85) \left\lVert\beta_0\frac{\partial g}{\partial b}-\beta_0\left(\frac{\partial g}{\partial b}\right)^*\right\rVert_q\le C\eta^t,\qquad\left\lVert\frac{\partial}{\partial\theta}\left(\beta\frac{\partial g}{\partial b}\right)-\left[\frac{\partial}{\partial\theta}\left(\beta\frac{\partial g}{\partial b}\right)\right]^*\right\rVert_{q/2}\le C\eta^t. \tag*{(85)} β 0 ∂ b ∂ g − β 0 ( ∂ b ∂ g ) ∗ q ≤ C η t , ∂ θ ∂ ( β ∂ b ∂ g ) − [ ∂ θ ∂ ( β ∂ b ∂ g ) ] ∗ q /2 ≤ C η t . (85)
For t = k + j ≥ k + 2 t=k+j\ge k+2 t = k + j ≥ k + 2 , subtracting the coupled versions from (A.57) and (A.59) gives
∂ h t ∂ ( u k 2 ) − ( ∂ h t ∂ ( u k 2 ) ) ∗ = β 0 ∂ g ∂ b { ∂ h t − 1 ∂ ( u k 2 ) − ( ∂ h t − 1 ∂ ( u k 2 ) ) ∗ } + β 0 { ∂ g ∂ b − ( ∂ g ∂ b ) ∗ } ( ∂ h t − 1 ∂ ( u k 2 ) ) ∗ , \frac{\partial h_t}{\partial(u_k^2)}-\left(\frac{\partial h_t}{\partial(u_k^2)}\right)^*=\beta_0\frac{\partial g}{\partial b}\left\{\frac{\partial h_{t-1}}{\partial(u_k^2)}-\left(\frac{\partial h_{t-1}}{\partial(u_k^2)}\right)^*\right\}+\beta_0\left\{\frac{\partial g}{\partial b}-\left(\frac{\partial g}{\partial b}\right)^*\right\}\left(\frac{\partial h_{t-1}}{\partial(u_k^2)}\right)^*, ∂ ( u k 2 ) ∂ h t − ( ∂ ( u k 2 ) ∂ h t ) ∗ = β 0 ∂ b ∂ g { ∂ ( u k 2 ) ∂ h t − 1 − ( ∂ ( u k 2 ) ∂ h t − 1 ) ∗ } + β 0 { ∂ b ∂ g − ( ∂ b ∂ g ) ∗ } ( ∂ ( u k 2 ) ∂ h t − 1 ) ∗ ,
∂ 2 h t ∂ θ ∂ ( u k 2 ) − ( ∂ 2 h t ∂ θ ∂ ( u k 2 ) ) ∗ = β 0 ∂ g ∂ b { ∂ 2 h t − 1 ∂ θ ∂ ( u k 2 ) − ( ∂ 2 h t − 1 ∂ θ ∂ ( u k 2 ) ) ∗ } + β 0 { ∂ g ∂ b − ( ∂ g ∂ b ) ∗ } ( ∂ 2 h t − 1 ∂ θ ∂ ( u k 2 ) ) ∗ + ∂ ∂ θ ( β ∂ g ∂ b ) { ∂ h t − 1 ∂ ( u k 2 ) − ( ∂ h t − 1 ∂ ( u k 2 ) ) ∗ } + { ∂ ∂ θ ( β ∂ g ∂ b ) − [ ∂ ∂ θ ( β ∂ g ∂ b ) ] ∗ } ( ∂ h t − 1 ∂ ( u k 2 ) ) ∗ . \begin{aligned}
\frac{\partial^2h_t}{\partial\theta\partial(u_k^2)}-\left(\frac{\partial^2h_t}{\partial\theta\partial(u_k^2)}\right)^*
={}&\beta_0\frac{\partial g}{\partial b}\left\{\frac{\partial^2h_{t-1}}{\partial\theta\partial(u_k^2)}-\left(\frac{\partial^2h_{t-1}}{\partial\theta\partial(u_k^2)}\right)^*\right\}\\
&+\beta_0\left\{\frac{\partial g}{\partial b}-\left(\frac{\partial g}{\partial b}\right)^*\right\}\left(\frac{\partial^2h_{t-1}}{\partial\theta\partial(u_k^2)}\right)^*\\
&+\frac{\partial}{\partial\theta}\left(\beta\frac{\partial g}{\partial b}\right)\left\{\frac{\partial h_{t-1}}{\partial(u_k^2)}-\left(\frac{\partial h_{t-1}}{\partial(u_k^2)}\right)^*\right\}\\
&+\left\{\frac{\partial}{\partial\theta}\left(\beta\frac{\partial g}{\partial b}\right)-\left[\frac{\partial}{\partial\theta}\left(\beta\frac{\partial g}{\partial b}\right)\right]^*\right\}\left(\frac{\partial h_{t-1}}{\partial(u_k^2)}\right)^*.
\end{aligned} ∂ θ ∂ ( u k 2 ) ∂ 2 h t − ( ∂ θ ∂ ( u k 2 ) ∂ 2 h t ) ∗ = β 0 ∂ b ∂ g { ∂ θ ∂ ( u k 2 ) ∂ 2 h t − 1 − ( ∂ θ ∂ ( u k 2 ) ∂ 2 h t − 1 ) ∗ } + β 0 { ∂ b ∂ g − ( ∂ b ∂ g ) ∗ } ( ∂ θ ∂ ( u k 2 ) ∂ 2 h t − 1 ) ∗ + ∂ θ ∂ ( β ∂ b ∂ g ) { ∂ ( u k 2 ) ∂ h t − 1 − ( ∂ ( u k 2 ) ∂ h t − 1 ) ∗ } + { ∂ θ ∂ ( β ∂ b ∂ g ) − [ ∂ θ ∂ ( β ∂ b ∂ g ) ] ∗ } ( ∂ ( u k 2 ) ∂ h t − 1 ) ∗ .
Taking the L q L^q L q and L q / 2 L^{q/2} L q /2 norms of these two identities and applying (A.68) and (A.70) with Hölder’s inequality gives
∥ ∂ h k + j ∂ ( u k 2 ) − ( ∂ h k + j ∂ ( u k 2 ) ) ∗ ∥ q ≤ β ˉ ∥ ∂ h k + j − 1 ∂ ( u k 2 ) − ( ∂ h k + j − 1 ∂ ( u k 2 ) ) ∗ ∥ q + C γ j η k + j , \left\lVert\frac{\partial h_{k+j}}{\partial(u_k^2)}-\left(\frac{\partial h_{k+j}}{\partial(u_k^2)}\right)^*\right\rVert_q\le\bar\beta\left\lVert\frac{\partial h_{k+j-1}}{\partial(u_k^2)}-\left(\frac{\partial h_{k+j-1}}{\partial(u_k^2)}\right)^*\right\rVert_q+C\gamma^j\eta^{k+j}, ∂ ( u k 2 ) ∂ h k + j − ( ∂ ( u k 2 ) ∂ h k + j ) ∗ q ≤ β ˉ ∂ ( u k 2 ) ∂ h k + j − 1 − ( ∂ ( u k 2 ) ∂ h k + j − 1 ) ∗ q + C γ j η k + j ,
∥ ∂ 2 h k + j ∂ θ ∂ ( u k 2 ) − ( ∂ 2 h k + j ∂ θ ∂ ( u k 2 ) ) ∗ ∥ q / 2 ≤ β ˉ ∥ ∂ 2 h k + j − 1 ∂ θ ∂ ( u k 2 ) − ( ∂ 2 h k + j − 1 ∂ θ ∂ ( u k 2 ) ) ∗ ∥ q / 2 + C ∥ U k + j ∥ q ∥ ∂ h k + j − 1 ∂ ( u k 2 ) − ( ∂ h k + j − 1 ∂ ( u k 2 ) ) ∗ ∥ q + C γ j η k + j . \begin{aligned}
\left\lVert\frac{\partial^2h_{k+j}}{\partial\theta\partial(u_k^2)}-\left(\frac{\partial^2h_{k+j}}{\partial\theta\partial(u_k^2)}\right)^*\right\rVert_{q/2}
\le{}&\bar\beta\left\lVert\frac{\partial^2h_{k+j-1}}{\partial\theta\partial(u_k^2)}-\left(\frac{\partial^2h_{k+j-1}}{\partial\theta\partial(u_k^2)}\right)^*\right\rVert_{q/2}\\
&+C\lVert U_{k+j}\rVert_q\left\lVert\frac{\partial h_{k+j-1}}{\partial(u_k^2)}-\left(\frac{\partial h_{k+j-1}}{\partial(u_k^2)}\right)^*\right\rVert_q+C\gamma^j\eta^{k+j}.
\end{aligned} ∂ θ ∂ ( u k 2 ) ∂ 2 h k + j − ( ∂ θ ∂ ( u k 2 ) ∂ 2 h k + j ) ∗ q /2 ≤ β ˉ ∂ θ ∂ ( u k 2 ) ∂ 2 h k + j − 1 − ( ∂ θ ∂ ( u k 2 ) ∂ 2 h k + j − 1 ) ∗ q /2 + C ∥ U k + j ∥ q ∂ ( u k 2 ) ∂ h k + j − 1 − ( ∂ ( u k 2 ) ∂ h k + j − 1 ) ∗ q + C γ j η k + j .
Checking the initial values at j = 0 , 1 j=0,1 j = 0 , 1 directly from (A.57) and (A.7), and iterating in j ≥ 0 j\ge0 j ≥ 0 via Lemma 1(i)(a) (after enlarging γ ∈ ( β ˉ , 1 ) \gamma\in(\bar\beta,1) γ ∈ ( β ˉ , 1 ) if necessary), yields
∥ ∂ h k + j ∂ ( u k 2 ) − ( ∂ h k + j ∂ ( u k 2 ) ) ∗ ∥ q + ∥ ∂ 2 h k + j ∂ θ ∂ ( u k 2 ) − ( ∂ 2 h k + j ∂ θ ∂ ( u k 2 ) ) ∗ ∥ q / 2 ≤ C γ j η k , k , j ≥ 0. (86) \left\lVert\frac{\partial h_{k+j}}{\partial(u_k^2)}-\left(\frac{\partial h_{k+j}}{\partial(u_k^2)}\right)^*\right\rVert_q+\left\lVert\frac{\partial^2h_{k+j}}{\partial\theta\partial(u_k^2)}-\left(\frac{\partial^2h_{k+j}}{\partial\theta\partial(u_k^2)}\right)^*\right\rVert_{q/2}\le C\gamma^j\eta^k,\qquad k,j\ge0. \tag*{(86)} ∂ ( u k 2 ) ∂ h k + j − ( ∂ ( u k 2 ) ∂ h k + j ) ∗ q + ∂ θ ∂ ( u k 2 ) ∂ 2 h k + j − ( ∂ θ ∂ ( u k 2 ) ∂ 2 h k + j ) ∗ q /2 ≤ C γ j η k , k , j ≥ 0. (86)
Next, by (A.62), W k , j = u k 2 { ∂ s k + j ( θ 0 ) / ∂ ( u k 2 ) } W_{k,j}=u_k^2\{\partial s_{k+j}(\theta_0)/\partial(u_k^2)\} W k , j = u k 2 { ∂ s k + j ( θ 0 ) / ∂ ( u k 2 )} takes the explicit form
W k , j = u k 2 [ ∂ f ∂ h ∂ 2 h k + j ∂ θ ∂ ( u k 2 ) + ( ∂ 2 f ∂ h 2 ∂ h k + j ∂ θ + ∂ 2 f ∂ φ ∂ h e φ ) ∂ h k + j ∂ ( u k 2 ) + ( ∂ 2 f ∂ h ∂ v ∂ h k + j ∂ θ + ∂ 2 f ∂ φ ∂ v e φ ) 1 { j = 0 } ] , (87) \begin{aligned}
W_{k,j}=u_k^2\bigg[&\frac{\partial f}{\partial h}\frac{\partial^2h_{k+j}}{\partial\theta\partial(u_k^2)}+\left(\frac{\partial^2f}{\partial h^2}\frac{\partial h_{k+j}}{\partial\theta}+\frac{\partial^2f}{\partial\varphi\partial h}e_\varphi\right)\frac{\partial h_{k+j}}{\partial(u_k^2)}\\
&+\left(\frac{\partial^2f}{\partial h\partial v}\frac{\partial h_{k+j}}{\partial\theta}+\frac{\partial^2f}{\partial\varphi\partial v}e_\varphi\right)\mathbf{1}\{j=0\}\bigg], \tag*{(87)}
\end{aligned} W k , j = u k 2 [ ∂ h ∂ f ∂ θ ∂ ( u k 2 ) ∂ 2 h k + j + ( ∂ h 2 ∂ 2 f ∂ θ ∂ h k + j + ∂ φ ∂ h ∂ 2 f e φ ) ∂ ( u k 2 ) ∂ h k + j + ( ∂ h ∂ v ∂ 2 f ∂ θ ∂ h k + j + ∂ φ ∂ v ∂ 2 f e φ ) 1 { j = 0 } ] , (87)
where the partial derivatives of f f f are evaluated at ( u k + j 2 , h k + j ( θ 0 ) , φ 0 ) (u_{k+j}^{2}, h_{k+j}(\theta_{0}), \varphi_{0}) ( u k + j 2 , h k + j ( θ 0 ) , φ 0 ) . Subtract the coupled version W k , j ∗ W_{k,j}^{*} W k , j ∗ from (A.72) by changing one factor at a time. By (A.10) and (A.69), the differences of the partial derivatives of f f f at time k + j k+j k + j satisfy
∥ ∂ f ∂ h − ( ∂ f ∂ h ) ∗ ∥ q / 2 + ∥ ∂ 2 f ∂ h 2 − ( ∂ 2 f ∂ h 2 ) ∗ ∥ q / 2 + ∥ ∂ 2 f ∂ φ ∂ h − ( ∂ 2 f ∂ φ ∂ h ) ∗ ∥ q / 2 ≤ C η k + j . \left\lVert\frac{\partial f}{\partial h}-\left(\frac{\partial f}{\partial h}\right)^{*}\right\rVert_{q/2}+\left\lVert\frac{\partial^{2}f}{\partial h^{2}}-\left(\frac{\partial^{2}f}{\partial h^{2}}\right)^{*}\right\rVert_{q/2}+\left\lVert\frac{\partial^{2}f}{\partial\varphi\,\partial h}-\left(\frac{\partial^{2}f}{\partial\varphi\,\partial h}\right)^{*}\right\rVert_{q/2}\le C\eta^{k+j}. ∂ h ∂ f − ( ∂ h ∂ f ) ∗ q /2 + ∂ h 2 ∂ 2 f − ( ∂ h 2 ∂ 2 f ) ∗ q /2 + ∂ φ ∂ h ∂ 2 f − ( ∂ φ ∂ h ∂ 2 f ) ∗ q /2 ≤ C η k + j .
Combining this with (A.68), (A.69), and (A.71) via Hölder’s inequality yields
∥ W k , j − W k , j ∗ ∥ q / 4 ≤ C γ j η k . \left\lVert W_{k,j}-W_{k,j}^{*}\right\rVert_{q/4}\le C\gamma^{j}\eta^{k}. W k , j − W k , j ∗ q /4 ≤ C γ j η k .
Finally, since ∥ W k , j − W k , j ∗ ∥ q / 2 ≤ ∥ W k , j ∥ q / 2 + ∥ W k , j ∗ ∥ q / 2 ≤ C γ j \left\lVert W_{k,j}-W_{k,j}^{*}\right\rVert_{q/2}\le\left\lVert W_{k,j}\right\rVert_{q/2}+\left\lVert W_{k,j}^{*}\right\rVert_{q/2}\le C\gamma^{j} W k , j − W k , j ∗ q /2 ≤ ∥ W k , j ∥ q /2 + W k , j ∗ q /2 ≤ C γ j by (A.54), applying the moment interpolation (A.21) in Lemma 1(ii)(a) with ( r , p , a ) = ( q / 4 , 2 , q / 2 ) (r,p,a)=(q/4,2,q/2) ( r , p , a ) = ( q /4 , 2 , q /2 ) when 4 < q < 8 4<q<8 4 < q < 8 (and norm monotonicity ∥ W k , j − W k , j ∗ ∥ 2 ≤ ∥ W k , j − W k , j ∗ ∥ q / 4 \left\lVert W_{k,j}-W_{k,j}^{*}\right\rVert_{2}\le\left\lVert W_{k,j}-W_{k,j}^{*}\right\rVert_{q/4} W k , j − W k , j ∗ 2 ≤ W k , j − W k , j ∗ q /4 when q ≥ 8 q\ge8 q ≥ 8 ) gives ∥ W k , j − W k , j ∗ ∥ 2 ≤ C γ j η ~ k \left\lVert W_{k,j}-W_{k,j}^{*}\right\rVert_{2}\le C\gamma^{j}\widetilde{\eta}^{k} W k , j − W k , j ∗ 2 ≤ C γ j η k for some η ~ ∈ ( 0 , 1 ) \widetilde{\eta}\in(0,1) η ∈ ( 0 , 1 ) . Minkowski’s inequality then yields ∥ A k − A k ∗ ∥ 2 ≤ ∑ j = 0 ∞ ∥ W k , j − W k , j ∗ ∥ 2 ≤ C η ~ k ∑ j = 0 ∞ γ j ≤ C η ~ k \left\lVert A_{k}-A_{k}^{*}\right\rVert_{2}\le\sum_{j=0}^{\infty}\left\lVert W_{k,j}-W_{k,j}^{*}\right\rVert_{2}\le C\widetilde{\eta}^{k}\sum_{j=0}^{\infty}\gamma^{j}\le C\widetilde{\eta}^{k} ∥ A k − A k ∗ ∥ 2 ≤ ∑ j = 0 ∞ W k , j − W k , j ∗ 2 ≤ C η k ∑ j = 0 ∞ γ j ≤ C η k for all k ≥ 0 k\ge0 k ≥ 0 . ■
The coefficient C τ C_{\tau} C τ can also be written as an expectation of likelihood derivatives. This representation avoids the classical HAC estimator when estimating C τ C_{\tau} C τ .
Lemma 10. Suppose the innovations are standard Gaussian, Assumptions 1–6 hold, and θ 0 \theta_{0} θ 0 is interior to Θ \Theta Θ . In derivatives with respect to ω \omega ω below, treat the intercept as independent of ( α , β , φ ) (\alpha,\beta,\varphi) ( α , β , φ ) , hold the observations fixed, and evaluate at ω = ω ( θ ) \omega=\omega(\theta) ω = ω ( θ ) afterwards. Derivatives with respect to θ \theta θ and φ \varphi φ retain the normalization ω ( θ ) = 1 − α − β − 3 φ + 2 β φ \omega(\theta)=1-\alpha-\beta-3\varphi+2\beta\varphi ω ( θ ) = 1 − α − β − 3 φ + 2 β φ . Then
C τ = − E [ ∂ ℓ t ( θ 0 ) ∂ θ { φ 0 ∂ ℓ t ( θ 0 ) ∂ φ + ( 1 − α 0 − β 0 ) ∂ ℓ t ( θ 0 ) ∂ ω } ] . (88) C_{\tau}=-E\left[\frac{\partial\ell_{t}(\theta_{0})}{\partial\theta}\left\{\varphi_{0}\frac{\partial\ell_{t}(\theta_{0})}{\partial\varphi}+(1-\alpha_{0}-\beta_{0})\frac{\partial\ell_{t}(\theta_{0})}{\partial\omega}\right\}\right]. \tag*{(88)} C τ = − E [ ∂ θ ∂ ℓ t ( θ 0 ) { φ 0 ∂ φ ∂ ℓ t ( θ 0 ) + ( 1 − α 0 − β 0 ) ∂ ω ∂ ℓ t ( θ 0 ) } ] . (88) The additional derivative is obtained from
∂ h t ( θ ) ∂ ω = ∂ g ∂ b { 1 + β ∂ h t − 1 ( θ ) ∂ ω } , (89) \frac{\partial h_{t}(\theta)}{\partial\omega}=\frac{\partial g}{\partial b}\left\{1+\beta\frac{\partial h_{t-1}(\theta)}{\partial\omega}\right\}, \tag*{(89)} ∂ ω ∂ h t ( θ ) = ∂ b ∂ g { 1 + β ∂ ω ∂ h t − 1 ( θ ) } , (89) ∂ ℓ t ( θ ) ∂ ω = { − u t 2 2 h t 2 ( θ ) + 2 h t ( θ ) h t 2 ( θ ) + φ u t 2 − 3 2 h t ( θ ) } ∂ h t ( θ ) ∂ ω . (90) \frac{\partial\ell_{t}(\theta)}{\partial\omega}=\left\{-\frac{u_{t}^{2}}{2h_{t}^{2}(\theta)}+\frac{2h_{t}(\theta)}{h_{t}^{2}(\theta)+\varphi u_{t}^{2}}-\frac{3}{2h_{t}(\theta)}\right\}\frac{\partial h_{t}(\theta)}{\partial\omega}. \tag*{(90)} ∂ ω ∂ ℓ t ( θ ) = { − 2 h t 2 ( θ ) u t 2 + h t 2 ( θ ) + φ u t 2 2 h t ( θ ) − 2 h t ( θ ) 3 } ∂ ω ∂ h t ( θ ) . (90) In (A.74), ∂ g / ∂ b \partial g/\partial b ∂ g / ∂ b is evaluated at ( b t − 1 ( θ ) , u t 2 ; φ ) (b_{t-1}(\theta),u_{t}^{2};\varphi) ( b t − 1 ( θ ) , u t 2 ; φ ) .
Proof. We first derive (A.73). For c > 0 c>0 c > 0 near one, replace every squared input u i 2 u_{i}^{2} u i 2 , i ≤ t i\le t i ≤ t , by u i 2 / c u_{i}^{2}/c u i 2 / c , and denote the resulting stationary volatility process by h t ( c , θ ) h_{t}(c,\theta) h t ( c , θ ) . Multiplying the long-run scale by c c c gives the conditional negative log-likelihood
ℓ t ( c , θ ) = 1 2 log c + f { u t 2 / c , h t ( c , θ ) , φ } , \ell_{t}(c,\theta)=\frac{1}{2}\log c+f\{u_{t}^{2}/c,h_{t}(c,\theta),\varphi\}, ℓ t ( c , θ ) = 2 1 log c + f { u t 2 / c , h t ( c , θ ) , φ } ,
where f f f is defined in (A.1). At c = 1 c=1 c = 1 , this is ℓ t ( θ ) \ell_{t}(\theta) ℓ t ( θ ) . The chain rule gives
∂ 2 ℓ t ( c , θ 0 ) ∂ θ ∂ log c ∣ c = 1 = − ∑ j = 0 ∞ u t − j 2 ∂ s t ( θ 0 ) ∂ ( u t − j 2 ) . (91) \left.\frac{\partial^{2}\ell_{t}(c,\theta_{0})}{\partial\theta\,\partial\log c}\right|_{c=1}
=-\sum_{j=0}^{\infty}u_{t-j}^{2}\frac{\partial s_{t}(\theta_{0})}{\partial(u_{t-j}^{2})}. \tag*{(91)} ∂ θ ∂ log c ∂ 2 ℓ t ( c , θ 0 ) c = 1 = − j = 0 ∑ ∞ u t − j 2 ∂ ( u t − j 2 ) ∂ s t ( θ 0 ) . (91)
The geometric bound in Lemma 8 makes this series summable in L q / 2 L^{q/2} L q /2 and justifies taking expectations. Hence the expectation in (A.76) is − C τ -C_{\tau} − C τ .
To express the scale derivative using the parameter derivatives, observe that
g ( c b , c u 2 ; c φ ) = c g ( b , u 2 ; φ ) , f ( c u 2 , c h , c φ ) = f ( u 2 , h , φ ) + 1 2 log c . g(cb,cu^2;c\varphi)=c\,g(b,u^2;\varphi),\qquad f(cu^2,ch,c\varphi)=f(u^2,h,\varphi)+\frac{1}{2}\log c. g ( c b , c u 2 ; c φ ) = c g ( b , u 2 ; φ ) , f ( c u 2 , c h , c φ ) = f ( u 2 , h , φ ) + 2 1 log c .
It follows that ℓ t ( c , θ ) \ell_t(c,\theta) ℓ t ( c , θ ) is also the likelihood evaluated at the original observations with intercept c ω ( θ ) c\omega(\theta) c ω ( θ ) and contemporaneous coefficient c φ c\varphi c φ , while α \alpha α and β \beta β remain fixed. Since the normalized derivative with respect to φ \varphi φ already includes the change in ω ( θ ) \omega(\theta) ω ( θ ) , the chain rule yields
∂ ℓ t ( c , θ ) ∂ log c ∣ c = 1 = φ ∂ ℓ t ( θ ) ∂ φ + { ω ( θ ) − φ ∂ ω ( θ ) ∂ φ } ∂ ℓ t ( θ ) ∂ ω = φ ∂ ℓ t ( θ ) ∂ φ + ( 1 − α − β ) ∂ ℓ t ( θ ) ∂ ω . \begin{aligned}
\left.\frac{\partial\ell_t(c,\theta)}{\partial\log c}\right|_{c=1}
=\varphi\frac{\partial\ell_t(\theta)}{\partial\varphi}
+\left\{\omega(\theta)-\varphi\frac{\partial\omega(\theta)}{\partial\varphi}\right\}\frac{\partial\ell_t(\theta)}{\partial\omega} \\
&=\varphi\frac{\partial\ell_t(\theta)}{\partial\varphi}
+(1-\alpha-\beta)\frac{\partial\ell_t(\theta)}{\partial\omega}.
\end{aligned} ∂ log c ∂ ℓ t ( c , θ ) c = 1 = φ ∂ φ ∂ ℓ t ( θ ) + { ω ( θ ) − φ ∂ φ ∂ ω ( θ ) } ∂ ω ∂ ℓ t ( θ ) = φ ∂ φ ∂ ℓ t ( θ ) + ( 1 − α − β ) ∂ ω ∂ ℓ t ( θ ) .
where ∂ ω ( θ ) / ∂ φ = − 3 + 2 β \partial\omega(\theta)/\partial\varphi=-3+2\beta ∂ ω ( θ ) / ∂ φ = − 3 + 2 β . Under the correctly specified Gaussian conditional density, differentiating its integral, which equals one, gives the mixed information identity
E [ ∂ 2 ℓ t ( c , θ 0 ) ∂ θ ∂ log c ∣ c = 1 ] = E [ ∂ ℓ t ( θ 0 ) ∂ θ ∂ ℓ t ( c , θ 0 ) ∂ log c ∣ c = 1 ] . E\left[\left.\frac{\partial^2\ell_t(c,\theta_0)}{\partial\theta\,\partial\log c}\right|_{c=1}\right]
=
E\left[\left.\frac{\partial\ell_t(\theta_0)}{\partial\theta}\frac{\partial\ell_t(c,\theta_0)}{\partial\log c}\right|_{c=1}\right]. E [ ∂ θ ∂ log c ∂ 2 ℓ t ( c , θ 0 ) c = 1 ] = E [ ∂ θ ∂ ℓ t ( θ 0 ) ∂ log c ∂ ℓ t ( c , θ 0 ) c = 1 ] .
The moment bounds in Lemmas 2 and 8 justify differentiation under the conditional integral. Combining the last identity with (A.76) proves (A.73).
Finally, define
ψ t = − C τ z t , ξ t = s t ( θ 0 ) + ψ t = s t ( θ 0 ) − C τ z t , (92) \psi_t=-C_\tau z_t,\qquad\xi_t=s_t(\theta_0)+\psi_t=s_t(\theta_0)-C_\tau z_t, \tag*{(92)} ψ t = − C τ z t , ξ t = s t ( θ 0 ) + ψ t = s t ( θ 0 ) − C τ z t , (92)
and
V = ∑ j = − ∞ ∞ Cov ( ξ t , ξ t − j ) . V=\sum_{j=-\infty}^{\infty}\operatorname{Cov}(\xi_t,\xi_{t-j}). V = j = − ∞ ∑ ∞ Cov ( ξ t , ξ t − j ) .
We then give a simplified expression for V V V .
Lemma 11. Suppose the conditions of Theorem 3 hold. Define
ζ t = u t 2 − b t − 1 − 3 φ , χ t = ( 1 − β ) ζ t + β φ ( ϵ t 2 − 1 ) 1 − α − β . \zeta_t=u_t^2-b_{t-1}-3\varphi,\qquad
\chi_t=\frac{(1-\beta)\zeta_t+\beta\varphi(\epsilon_t^2-1)}{1-\alpha-\beta}. ζ t = u t 2 − b t − 1 − 3 φ , χ t = 1 − α − β ( 1 − β ) ζ t + β φ ( ϵ t 2 − 1 ) . Then { ζ t } \{\zeta_t\} { ζ t } and { χ t } \{\chi_t\} { χ t } are martingale differences with respect to { F t } \{F_t\} { F t } , and
z t = ( α + β ) z t − 1 + ζ t − β ζ t − 1 + β φ ( ϵ t − 1 2 − 1 ) . (93) z_t=(\alpha+\beta)z_{t-1}+\zeta_t-\beta\zeta_{t-1}+\beta\varphi(\epsilon_{t-1}^2-1). \tag*{(93)} z t = ( α + β ) z t − 1 + ζ t − β ζ t − 1 + β φ ( ϵ t − 1 2 − 1 ) . (93) Moreover,
1 T ∑ t = 1 T ( z t − χ t ) → 0 in L 2 , E { s t ( θ 0 ) χ t } = 0. (94) \frac{1}{\sqrt{T}}\sum_{t=1}^{T}(z_t-\chi_t)\to0\quad\text{in }L^2,\qquad E\{s_t(\theta_0)\chi_t\}=0. \tag*{(94)} T 1 t = 1 ∑ T ( z t − χ t ) → 0 in L 2 , E { s t ( θ 0 ) χ t } = 0. (94) Consequently,
V = H + Ω z C τ C τ T , Σ = H − 1 + Ω z H − 1 C τ C τ T H − 1 , (95) V=H+\Omega_zC_\tau C_\tau^{\mathsf T},\qquad
\Sigma=H^{-1}+\Omega_zH^{-1}C_\tau C_\tau^{\mathsf T}H^{-1}, \tag*{(95)} V = H + Ω z C τ C τ T , Σ = H − 1 + Ω z H − 1 C τ C τ T H − 1 , (95) where Ω z = E ( χ t 2 ) \Omega_z=E(\chi_t^2) Ω z = E ( χ t 2 ) . In the Gaussian case, we further have that
Var ( b t ) = 2 { α + ( 3 α + β ) φ } 2 + 24 α 2 φ 2 1 − 3 α 2 − 2 α β − β 2 , (96) \operatorname{Var}(b_t)=\frac{2\{\alpha+(3\alpha+\beta)\varphi\}^2+24\alpha^2\varphi^2}{1-3\alpha^2-2\alpha\beta-\beta^2}, \tag*{(96)} Var ( b t ) = 1 − 3 α 2 − 2 α β − β 2 2 { α + ( 3 α + β ) φ } 2 + 24 α 2 φ 2 , (96) Ω z = 2 ( 1 − β ) 2 Var ( b t ) + 2 { 1 − β + ( 3 − 2 β ) φ } 2 + 24 ( 1 − β ) 2 φ 2 ( 1 − α − β ) 2 . (97) \Omega_z=\frac{2(1-\beta)^2\operatorname{Var}(b_t)+2\{1-\beta+(3-2\beta)\varphi\}^2+24(1-\beta)^2\varphi^2}{(1-\alpha-\beta)^2}. \tag*{(97)} Ω z = ( 1 − α − β ) 2 2 ( 1 − β ) 2 Var ( b t ) + 2 { 1 − β + ( 3 − 2 β ) φ } 2 + 24 ( 1 − β ) 2 φ 2 . (97)
Proof. We divide the proof into four steps: (1) deriving the ARMA representation (A.78) and the decomposition for z t − χ t z_t-\chi_t z t − χ t ; (2) establishing the orthogonality E { s t ( θ 0 ) χ t } = 0 E\{s_t(\theta_0)\chi_t\}=0 E { s t ( θ 0 ) χ t } = 0 via a general conditional score identity; (3) deriving the rank-one covariance update (A.80); and (4) computing the closed-form Gaussian variances (A.81)–(A.82) via orthogonal polynomial innovations.
Step 1: ARMA representation Since u t 2 = b t − 1 ϵ t 2 + φ ϵ t 4 u_t^2=b_{t-1}\epsilon_t^2+\varphi\epsilon_t^4 u t 2 = b t − 1 ϵ t 2 + φ ϵ t 4 and ϵ t ∼ N ( 0 , 1 ) \epsilon_t\sim\mathcal{N}(0,1) ϵ t ∼ N ( 0 , 1 ) is independent of F t − 1 \mathcal{F}_{t-1} F t − 1 , we have E ( u t 2 ∣ F t − 1 ) = b t − 1 + 3 φ E(u_t^2\mid\mathcal{F}_{t-1})=b_{t-1}+3\varphi E ( u t 2 ∣ F t − 1 ) = b t − 1 + 3 φ and E ( ϵ t 2 − 1 ∣ F t − 1 ) = 0 E(\epsilon_t^2-1\mid\mathcal{F}_{t-1})=0 E ( ϵ t 2 − 1 ∣ F t − 1 ) = 0 . Thus ζ t = u t 2 − E ( u t 2 ∣ F t − 1 ) \zeta_t=u_t^2-E(u_t^2\mid\mathcal{F}_{t-1}) ζ t = u t 2 − E ( u t 2 ∣ F t − 1 ) and χ t \chi_t χ t are martingale differences with respect to { F t } \{\mathcal{F}_t\} { F t } .
From z t = u t 2 − 1 z_t=u_t^2-1 z t = u t 2 − 1 and ζ t = u t 2 − b t − 1 − 3 φ \zeta_t=u_t^2-b_{t-1}-3\varphi ζ t = u t 2 − b t − 1 − 3 φ , we have z t − ζ t = b t − 1 + 3 φ − 1 z_t-\zeta_t=b_{t-1}+3\varphi-1 z t − ζ t = b t − 1 + 3 φ − 1 . Moreover, eliminating b t − 2 b_{t-2} b t − 2 between h t − 1 = b t − 2 + φ ϵ t − 1 2 h_{t-1}=b_{t-2}+\varphi\epsilon_{t-1}^2 h t − 1 = b t − 2 + φ ϵ t − 1 2 and u t − 1 2 = b t − 2 + 3 φ + ζ t − 1 u_{t-1}^2=b_{t-2}+3\varphi+\zeta_{t-1} u t − 1 2 = b t − 2 + 3 φ + ζ t − 1 gives
h t − 1 = u t − 1 2 − ζ t − 1 + φ ( ϵ t − 1 2 − 1 ) − 2 φ . h_{t-1}=u_{t-1}^2-\zeta_{t-1}+\varphi(\epsilon_{t-1}^2-1)-2\varphi. h t − 1 = u t − 1 2 − ζ t − 1 + φ ( ϵ t − 1 2 − 1 ) − 2 φ .
Substituting this expression and the normalization ω = 1 − α − β − 3 φ + 2 β φ \omega=1-\alpha-\beta-3\varphi+2\beta\varphi ω = 1 − α − β − 3 φ + 2 β φ into b t − 1 = ω + α u t − 1 2 + β h t − 1 b_{t-1}=\omega+\alpha u_{t-1}^2+\beta h_{t-1} b t − 1 = ω + α u t − 1 2 + β h t − 1 yields
z t − ζ t = b t − 1 + 3 φ − 1 = ( α + β ) z t − 1 − β ζ t − 1 + β φ ( ϵ t − 1 2 − 1 ) . z_t-\zeta_t=b_{t-1}+3\varphi-1=(\alpha+\beta)z_{t-1}-\beta\zeta_{t-1}+\beta\varphi(\epsilon_{t-1}^2-1). z t − ζ t = b t − 1 + 3 φ − 1 = ( α + β ) z t − 1 − β ζ t − 1 + β φ ( ϵ t − 1 2 − 1 ) .
which is (A.78). Subtracting ( α + β ) z t − β ζ t + β φ ( ϵ t 2 − 1 ) (\alpha+\beta)z_t-\beta\zeta_t+\beta\varphi(\epsilon_t^2-1) ( α + β ) z t − β ζ t + β φ ( ϵ t 2 − 1 ) from both sides of (A.78) and dividing by 1 − α − β > 0 1-\alpha-\beta>0 1 − α − β > 0 gives
z t − χ t = − 1 1 − α − β Δ { ( α + β ) z t − β ζ t + β φ ( ϵ t 2 − 1 ) } , Δ x t : = x t − x t − 1 . z_t-\chi_t=-\frac{1}{1-\alpha-\beta}\Delta\{(\alpha+\beta)z_t-\beta\zeta_t+\beta\varphi(\epsilon_t^2-1)\},\qquad\Delta x_t:=x_t-x_{t-1}. z t − χ t = − 1 − α − β 1 Δ {( α + β ) z t − β ζ t + β φ ( ϵ t 2 − 1 )} , Δ x t := x t − x t − 1 .
Summing over t = 1 , … , T t=1,\ldots,T t = 1 , … , T yields
( 1 − α − β ) ∑ t = 1 T ( z t − χ t ) = ( α + β ) ( z 0 − z T ) − β ( ζ 0 − ζ T ) + β φ ( ϵ 0 2 − ϵ T 2 ) . (1-\alpha-\beta)\sum_{t=1}^{T}(z_t-\chi_t)=(\alpha+\beta)(z_0-z_T)-\beta(\zeta_0-\zeta_T)+\beta\varphi(\epsilon_0^2-\epsilon_T^2). ( 1 − α − β ) t = 1 ∑ T ( z t − χ t ) = ( α + β ) ( z 0 − z T ) − β ( ζ 0 − ζ T ) + β φ ( ϵ 0 2 − ϵ T 2 ) .
Since z t , ζ t , ϵ t 2 ∈ L 2 z_t,\zeta_t,\epsilon_t^2\in L^2 z t , ζ t , ϵ t 2 ∈ L 2 by stationarity and Assumption 6, dividing by T \sqrt{T} T proves T − 1 / 2 ∑ t = 1 T ( z t − χ t ) → 0 T^{-1/2}\sum_{t=1}^{T}(z_t-\chi_t)\to0 T − 1/2 ∑ t = 1 T ( z t − χ t ) → 0 in L 2 L^2 L 2 , and the martingale-difference property of χ t \chi_t χ t implies Ω z = E ( χ t 2 ) \Omega_z=E(\chi_t^2) Ω z = E ( χ t 2 ) .
Step 2: Orthogonality E { s t ( θ 0 ) χ t } = 0 E\{s_t(\theta_0)\chi_t\}=0 E { s t ( θ 0 ) χ t } = 0 via the conditional score identity. Let f t ( r ∣ F t − 1 ; θ ) f_t(r\mid\mathcal{F}_{t-1};\theta) f t ( r ∣ F t − 1 ; θ ) be the conditional density of r t r_t r t , so that s t ( θ ) = − ∂ θ log f t ( r t ∣ F t − 1 ; θ ) s_t(\theta)=-\partial_\theta\log f_t(r_t\mid\mathcal{F}_{t-1};\theta) s t ( θ ) = − ∂ θ log f t ( r t ∣ F t − 1 ; θ ) . For any smooth function Ψ ( u t , θ ) \Psi(u_t,\theta) Ψ ( u t , θ ) whose value and θ \theta θ -derivative (holding u t u_t u t fixed) are dominated on N \mathcal{N} N by an integrable envelope, differentiating ∫ Ψ ( r / τ t , θ ) f t ( r ∣ F t − 1 ; θ ) d r = E θ { Ψ ( u t , θ ) ∣ F t − 1 } \int\Psi(r/\sqrt{\tau_t},\theta)f_t(r\mid\mathcal{F}_{t-1};\theta)\,dr=E_\theta\{\Psi(u_t,\theta)\mid\mathcal{F}_{t-1}\} ∫ Ψ ( r / τ t , θ ) f t ( r ∣ F t − 1 ; θ ) d r = E θ { Ψ ( u t , θ ) ∣ F t − 1 } at θ = θ 0 \theta=\theta_0 θ = θ 0 gives the general conditional score identity
E { s t ( θ 0 ) Ψ ( u t , θ 0 ) ∣ F t − 1 } = E { ∂ Ψ ( u t , θ 0 ) ∂ θ ∣ F t − 1 } − ∂ ∂ θ E θ { Ψ ( u t , θ ) ∣ F t − 1 } ∣ θ = θ 0 . (98) E\{s_t(\theta_0)\Psi(u_t,\theta_0)\mid\mathcal{F}_{t-1}\}
=E\left\{\left.\frac{\partial\Psi(u_t,\theta_0)}{\partial\theta}\right|\mathcal{F}_{t-1}\right\}
-\left.\frac{\partial}{\partial\theta}E_\theta\{\Psi(u_t,\theta)\mid\mathcal{F}_{t-1}\}\right|_{\theta=\theta_0}. \tag*{(98)} E { s t ( θ 0 ) Ψ ( u t , θ 0 ) ∣ F t − 1 } = E { ∂ θ ∂ Ψ ( u t , θ 0 ) F t − 1 } − ∂ θ ∂ E θ { Ψ ( u t , θ ) ∣ F t − 1 } θ = θ 0 . (98)
where differentiation under the integral is justified by Lemmas 2 and 8. Applying (A.83) to three choices of Ψ ( u t , θ ) \Psi(u_t,\theta) Ψ ( u t , θ ) yields:
Taking Ψ ( u t , θ ) ≡ 1 \Psi(u_t,\theta)\equiv1 Ψ ( u t , θ ) ≡ 1 gives E { s t ( θ 0 ) ∣ F t − 1 } = 0 E\{s_t(\theta_0)\mid\mathcal{F}_{t-1}\}=0 E { s t ( θ 0 ) ∣ F t − 1 } = 0 , so { s t ( θ 0 ) } \{s_t(\theta_0)\} { s t ( θ 0 )} is a martingale difference sequence.
Taking Ψ ( u t , θ ) = u t 2 − b t − 1 ( θ ) − 3 φ \Psi(u_t,\theta)=u_t^2-b_{t-1}(\theta)-3\varphi Ψ ( u t , θ ) = u t 2 − b t − 1 ( θ ) − 3 φ (which equals ζ t \zeta_t ζ t at θ 0 \theta_0 θ 0 and satisfies E θ { Ψ ( u t , θ ) ∣ F t − 1 } ≡ 0 E_\theta\{\Psi(u_t,\theta)\mid\mathcal{F}_{t-1}\}\equiv0 E θ { Ψ ( u t , θ ) ∣ F t − 1 } ≡ 0 for all θ \theta θ ) gives
E { s t ( θ 0 ) ζ t ∣ F t − 1 } = − ∂ b t − 1 ( θ 0 ) ∂ θ − 3 e φ . E\{s_t(\theta_0)\zeta_t\mid\mathcal{F}_{t-1}\}
=-\frac{\partial b_{t-1}(\theta_0)}{\partial\theta}-3e_\varphi. E { s t ( θ 0 ) ζ t ∣ F t − 1 } = − ∂ θ ∂ b t − 1 ( θ 0 ) − 3 e φ .
Taking Ψ ( u t , θ ) = u t 2 / h t ( θ ) − 1 \Psi(u_t,\theta)=u_t^2/h_t(\theta)-1 Ψ ( u t , θ ) = u t 2 / h t ( θ ) − 1 (which equals ϵ t 2 − 1 \epsilon_t^2-1 ϵ t 2 − 1 at θ 0 \theta_0 θ 0 and satisfies E θ { Ψ ( u t , θ ) ∣ F t − 1 } ≡ 0 E_\theta\{\Psi(u_t,\theta)\mid\mathcal{F}_{t-1}\}\equiv0 E θ { Ψ ( u t , θ ) ∣ F t − 1 } ≡ 0 for all θ \theta θ ) gives
E { s t ( θ 0 ) ( ϵ t 2 − 1 ) ∣ F t − 1 } = − E { ϵ t 2 h t ( θ 0 ) ∂ h t ( θ 0 ) ∂ θ ∣ F t − 1 } . E\{s_t(\theta_0)(\epsilon_t^2-1)\mid\mathcal{F}_{t-1}\}
=-E\left\{\left.\frac{\epsilon_t^2}{h_t(\theta_0)}\frac{\partial h_t(\theta_0)}{\partial\theta}\right|\mathcal{F}_{t-1}\right\}. E { s t ( θ 0 ) ( ϵ t 2 − 1 ) ∣ F t − 1 } = − E { h t ( θ 0 ) ϵ t 2 ∂ θ ∂ h t ( θ 0 ) F t − 1 } .
To relate these two expectations, differentiate the quadratic identity h t ( θ ) = b t − 1 ( θ ) + φ u t 2 / h t ( θ ) h_t(\theta)=b_{t-1}(\theta)+\varphi u_t^2/h_t(\theta) h t ( θ ) = b t − 1 ( θ ) + φ u t 2 / h t ( θ ) with respect to θ \theta θ at θ 0 \theta_0 θ 0 (holding u t 2 u_t^2 u t 2 fixed) to obtain
− φ ϵ t 2 h t ( θ 0 ) ∂ h t ( θ 0 ) ∂ θ = ∂ h t ( θ 0 ) ∂ θ − ∂ b t − 1 ( θ 0 ) ∂ θ − ϵ t 2 e φ . -\varphi\frac{\epsilon_t^2}{h_t(\theta_0)}\frac{\partial h_t(\theta_0)}{\partial\theta}
=
\frac{\partial h_t(\theta_0)}{\partial\theta}
-
\frac{\partial b_{t-1}(\theta_0)}{\partial\theta}
-
\epsilon_t^2 e_\varphi. − φ h t ( θ 0 ) ϵ t 2 ∂ θ ∂ h t ( θ 0 ) = ∂ θ ∂ h t ( θ 0 ) − ∂ θ ∂ b t − 1 ( θ 0 ) − ϵ t 2 e φ .
Meanwhile, taking unconditional expectations in ∂ θ b t − 1 ( θ 0 ) = ( − 1 , − 1 , − 3 + 2 β ) T + e α u t − 1 2 + e β h t − 1 + β ∂ θ h t − 1 ( θ 0 ) \partial_\theta b_{t-1}(\theta_0)=(-1,-1,-3+2\beta)^\mathsf{T}+e_\alpha u_{t-1}^2+e_\beta h_{t-1}+\beta\partial_\theta h_{t-1}(\theta_0) ∂ θ b t − 1 ( θ 0 ) = ( − 1 , − 1 , − 3 + 2 β ) T + e α u t − 1 2 + e β h t − 1 + β ∂ θ h t − 1 ( θ 0 ) and using E ( u t − 1 2 ) = 1 E(u_{t-1}^2)=1 E ( u t − 1 2 ) = 1 and E ( h t − 1 ) = 1 − 2 φ E(h_{t-1})=1-2\varphi E ( h t − 1 ) = 1 − 2 φ gives
E { ∂ b t − 1 ( θ 0 ) ∂ θ } = ( − 3 + 2 β ) e φ + β E { ∂ h t ( θ 0 ) ∂ θ } . E\left\{\frac{\partial b_{t-1}(\theta_0)}{\partial\theta}\right\}
=
(-3+2\beta)e_\varphi+\beta E\left\{\frac{\partial h_t(\theta_0)}{\partial\theta}\right\}. E { ∂ θ ∂ b t − 1 ( θ 0 ) } = ( − 3 + 2 β ) e φ + β E { ∂ θ ∂ h t ( θ 0 ) } .
Substituting this into the unconditional expectations of items 2 and 3 above shows that both covariances are collinear with D h : = E { ∂ θ h t ( θ 0 ) } + 2 e φ D_h:=E\{\partial_\theta h_t(\theta_0)\}+2e_\varphi D h := E { ∂ θ h t ( θ 0 )} + 2 e φ :
E { s t ( θ 0 ) ζ t } = − β D h , φ E { s t ( θ 0 ) ( ϵ t 2 − 1 ) } = ( 1 − β ) D h . E\{s_t(\theta_0)\zeta_t\}=-\beta D_h,\qquad
\varphi E\{s_t(\theta_0)(\epsilon_t^2-1)\}=(1-\beta)D_h. E { s t ( θ 0 ) ζ t } = − β D h , φE { s t ( θ 0 ) ( ϵ t 2 − 1 )} = ( 1 − β ) D h .
Therefore, ( 1 − β ) E { s t ( θ 0 ) ζ t } + β φ E { s t ( θ 0 ) ( ϵ t 2 − 1 ) } = 0 (1-\beta)E\{s_t(\theta_0)\zeta_t\}+\beta\varphi E\{s_t(\theta_0)(\epsilon_t^2-1)\}=0 ( 1 − β ) E { s t ( θ 0 ) ζ t } + β φE { s t ( θ 0 ) ( ϵ t 2 − 1 )} = 0 , which proves E { s t ( θ 0 ) χ t } = 0 E\{s_t(\theta_0)\chi_t\}=0 E { s t ( θ 0 ) χ t } = 0 .
Step 3: Long-run covariance simplification. Since both { s t ( θ 0 ) } \{s_t(\theta_0)\} { s t ( θ 0 )} and { χ t } \{\chi_t\} { χ t } are stationary martingale difference sequences with respect to { F t } \{\mathcal{F}_t\} { F t } and are mutually orthogonal at lag zero ( E { s t ( θ 0 ) χ t } = 0 ) (E\{s_t(\theta_0)\chi_t\}=0) ( E { s t ( θ 0 ) χ t } = 0 ) , all auto- and cross-covariances of s t ( θ 0 ) s_t(\theta_0) s t ( θ 0 ) and χ t \chi_t χ t at nonzero lags vanish. By Step 1, T − 1 / 2 ∑ t = 1 T ξ t = T − 1 / 2 ∑ t = 1 T { s t ( θ 0 ) − C τ χ t } + o L 2 ( 1 ) T^{-1/2}\sum_{t=1}^T\xi_t=T^{-1/2}\sum_{t=1}^T\{s_t(\theta_0)-C_\tau\chi_t\}+o_{L^2}(1) T − 1/2 ∑ t = 1 T ξ t = T − 1/2 ∑ t = 1 T { s t ( θ 0 ) − C τ χ t } + o L 2 ( 1 ) . Moreover, since q > 4 q>4 q > 4 , Proposition 5 and Lemma 4(iii) imply that { ξ t } \{\xi_t\} { ξ t } is L 2 L^2 L 2 GMC, so Lemma 1(ii)(b) gives
V = ∑ j = − ∞ ∞ Cov ( ξ t , ξ t − j ) = lim T → ∞ Var ( 1 T ∑ t = 1 T ξ t ) = E { { s t ( θ 0 ) − C τ χ t } { s t ( θ 0 ) − C τ χ t } T } = E { s t ( θ 0 ) s t ( θ 0 ) T } + Ω z C τ C τ T . \begin{aligned}
V=\sum_{j=-\infty}^{\infty}\operatorname{Cov}(\xi_t,\xi_{t-j})
=\lim_{T\to\infty}\operatorname{Var}\left(\frac{1}{\sqrt{T}}\sum_{t=1}^T\xi_t\right) \\
&=E\left\{\{s_t(\theta_0)-C_\tau\chi_t\}\{s_t(\theta_0)-C_\tau\chi_t\}^{\mathsf{T}}\right\}
=E\{s_t(\theta_0)s_t(\theta_0)^{\mathsf{T}}\}+\Omega_zC_\tau C_\tau^{\mathsf{T}}.
\end{aligned} V = j = − ∞ ∑ ∞ Cov ( ξ t , ξ t − j ) = T → ∞ lim Var ( T 1 t = 1 ∑ T ξ t ) = E { { s t ( θ 0 ) − C τ χ t } { s t ( θ 0 ) − C τ χ t } T } = E { s t ( θ 0 ) s t ( θ 0 ) T } + Ω z C τ C τ T .
Differentiating ∫ f t ( r ∣ F t − 1 ; θ ) d r = 1 \int f_t(r\mid\mathcal{F}_{t-1};\theta)\,dr=1 ∫ f t ( r ∣ F t − 1 ; θ ) d r = 1 twice at θ = θ 0 \theta=\theta_0 θ = θ 0 gives the conditional information equality
E { s t ( θ 0 ) s t ( θ 0 ) T ∣ F t − 1 } = E { ∂ 2 ℓ t ( θ 0 ) ∂ θ ∂ θ T ∣ F t − 1 } , E\{s_t(\theta_0)s_t(\theta_0)^{\mathsf{T}}\mid\mathcal{F}_{t-1}\}
=
E\left\{\left.\frac{\partial^2\ell_t(\theta_0)}{\partial\theta\partial\theta^{\mathsf{T}}}\right|\mathcal{F}_{t-1}\right\}, E { s t ( θ 0 ) s t ( θ 0 ) T ∣ F t − 1 } = E { ∂ θ ∂ θ T ∂ 2 ℓ t ( θ 0 ) F t − 1 } ,
and taking expectations yields E { s t ( θ 0 ) s t ( θ 0 ) T } = H E\{s_t(\theta_0)s_t(\theta_0)^{\mathsf{T}}\}=H E { s t ( θ 0 ) s t ( θ 0 ) T } = H . Substituting this into Σ = H − 1 V H − T \Sigma=H^{-1}VH^{-\mathsf{T}} Σ = H − 1 V H − T proves (A.80).
Step 4: Explicit Gaussian variances via orthogonal polynomial innovations. For ϵ t ∼ N ( 0 , 1 ) \epsilon_t\sim\mathcal{N}(0,1) ϵ t ∼ N ( 0 , 1 ) , define the centered Hermite-type polynomials
H 1 , t : = ϵ t 2 − 1 , H 2 , t : = ϵ t 4 − 6 ϵ t 2 + 3. H_{1,t}:=\epsilon_t^2-1,\qquad H_{2,t}:=\epsilon_t^4-6\epsilon_t^2+3. H 1 , t := ϵ t 2 − 1 , H 2 , t := ϵ t 4 − 6 ϵ t 2 + 3.
Direct Gaussian moment calculation gives E ( H 1 , t ) = E ( H 2 , t ) = E ( H 1 , t H 2 , t ) = 0 E(H_{1,t})=E(H_{2,t})=E(H_{1,t}H_{2,t})=0 E ( H 1 , t ) = E ( H 2 , t ) = E ( H 1 , t H 2 , t ) = 0 , E ( H 1 , t 2 ) = 2 E(H_{1,t}^2)=2 E ( H 1 , t 2 ) = 2 , and E ( H 2 , t 2 ) = 24 E(H_{2,t}^2)=24 E ( H 2 , t 2 ) = 24 . Using b ˉ : = E ( b t ) = 1 − 3 φ \bar b:=E(b_t)=1-3\varphi b ˉ := E ( b t ) = 1 − 3 φ and ϵ t 4 = H 2 , t + 6 H 1 , t + 3 \epsilon_t^4=H_{2,t}+6H_{1,t}+3 ϵ t 4 = H 2 , t + 6 H 1 , t + 3 , we can expand both the centered state b t − b ˉ = ( α ϵ t 2 + β ) b t − 1 + ω + φ ( α ϵ t 4 + β ϵ t 2 ) − b ˉ b_t-\bar b=(\alpha\epsilon_t^2+\beta)b_{t-1}+\omega+\varphi(\alpha\epsilon_t^4+\beta\epsilon_t^2)-\bar b b t − b ˉ = ( α ϵ t 2 + β ) b t − 1 + ω + φ ( α ϵ t 4 + β ϵ t 2 ) − b ˉ and the martingale difference ( 1 − α − β ) χ t = ( 1 − β ) { ( b t − 1 + φ ϵ t 2 ) ϵ t 2 − b t − 1 − 3 φ } + β φ ( ϵ t 2 − 1 ) (1-\alpha-\beta)\chi_t=(1-\beta)\{(b_{t-1}+\varphi\epsilon_t^2)\epsilon_t^2-b_{t-1}-3\varphi\}+\beta\varphi(\epsilon_t^2-1) ( 1 − α − β ) χ t = ( 1 − β ) {( b t − 1 + φ ϵ t 2 ) ϵ t 2 − b t − 1 − 3 φ } + β φ ( ϵ t 2 − 1 ) in the mutually orthogonal basis { b t − 1 − b ˉ , H 1 , t , H 2 , t } \{b_{t-1}-\bar b,H_{1,t},H_{2,t}\} { b t − 1 − b ˉ , H 1 , t , H 2 , t } :
b t − b ˉ = ( α ϵ t 2 + β ) ( b t − 1 − b ˉ ) + { α + ( 3 α + β ) φ } H 1 , t + α φ H 2 , t , b_t-\bar b=(\alpha\epsilon_t^2+\beta)(b_{t-1}-\bar b)+\{\alpha+(3\alpha+\beta)\varphi\}H_{1,t}+\alpha\varphi H_{2,t}, b t − b ˉ = ( α ϵ t 2 + β ) ( b t − 1 − b ˉ ) + { α + ( 3 α + β ) φ } H 1 , t + α φ H 2 , t ,
( 1 − α − β ) χ t = ( 1 − β ) ( b t − 1 − b ˉ ) H 1 , t + { 1 − β + ( 3 − 2 β ) φ } H 1 , t + ( 1 − β ) φ H 2 , t . (1-\alpha-\beta)\chi_t=(1-\beta)(b_{t-1}-\bar b)H_{1,t}+\{1-\beta+(3-2\beta)\varphi\}H_{1,t}+(1-\beta)\varphi H_{2,t}. ( 1 − α − β ) χ t = ( 1 − β ) ( b t − 1 − b ˉ ) H 1 , t + { 1 − β + ( 3 − 2 β ) φ } H 1 , t + ( 1 − β ) φ H 2 , t .
Since b t − 1 − b ˉ b_{t-1}-\bar{b} b t − 1 − b ˉ has mean zero and is independent of ϵ t \epsilon_t ϵ t , while E ( α ϵ t 2 + β ) 2 = 3 α 2 + 2 α β + β 2 < 1 E(\alpha\epsilon_t^2+\beta)^2=3\alpha^2+2\alpha\beta+\beta^2<1 E ( α ϵ t 2 + β ) 2 = 3 α 2 + 2 α β + β 2 < 1 (by Proposition 5 with p = 2 p=2 p = 2 ), taking variances in the first equation yields
Var ( b t ) = ( 3 α 2 + 2 α β + β 2 ) Var ( b t ) + 2 { α + ( 3 α + β ) φ } 2 + 24 α 2 φ 2 , \operatorname{Var}(b_t)=(3\alpha^2+2\alpha\beta+\beta^2)\operatorname{Var}(b_t)+2\{\alpha+(3\alpha+\beta)\varphi\}^2+24\alpha^2\varphi^2, Var ( b t ) = ( 3 α 2 + 2 α β + β 2 ) Var ( b t ) + 2 { α + ( 3 α + β ) φ } 2 + 24 α 2 φ 2 ,
which solves to (A.81). Similarly, since ( b t − 1 − b ˉ ) H 1 , t (b_{t-1}-\bar{b})H_{1,t} ( b t − 1 − b ˉ ) H 1 , t , H 1 , t H_{1,t} H 1 , t , and H 2 , t H_{2,t} H 2 , t are mutually orthogonal with variances 2 Var ( b t ) 2\operatorname{Var}(b_t) 2 Var ( b t ) , 2 2 2 , and 24 24 24 , taking the second moment of the second equation gives
( 1 − α − β ) 2 Ω z = ( 1 − α − β ) 2 E ( χ t 2 ) = 2 ( 1 − β ) 2 Var ( b t ) + 2 { 1 − β + ( 3 − 2 β ) φ } 2 + 24 ( 1 − β ) 2 φ 2 , (1-\alpha-\beta)^2\Omega_z=(1-\alpha-\beta)^2E(\chi_t^2)=2(1-\beta)^2\operatorname{Var}(b_t)+2\{1-\beta+(3-2\beta)\varphi\}^2+24(1-\beta)^2\varphi^2, ( 1 − α − β ) 2 Ω z = ( 1 − α − β ) 2 E ( χ t 2 ) = 2 ( 1 − β ) 2 Var ( b t ) + 2 { 1 − β + ( 3 − 2 β ) φ } 2 + 24 ( 1 − β ) 2 φ 2 ,
proving (A.82). This generalizes the ARMA reduction in equations (A.8)–(A.9) of Jiang et al. [23 ] by accounting for the contemporaneous feedback term β φ ( ϵ t − 1 2 − 1 ) \beta\varphi(\epsilon_{t-1}^2-1) β φ ( ϵ t − 1 2 − 1 ) in (A.78). ■
Proof of Theorem 3. The proof consists of three steps. We first show that the sample Hessian matrix converges to H H H . We then derive the asymptotic expansion of the score function by replacing the long-run component with its estimates in the first step. Finally, we apply the central limit theorem for the martingale difference sequence.
Step 1.. Because θ 0 \theta_0 θ 0 is an interior point, by the first order condition, we have that
0 = ∂ L ^ T ( θ ^ T ) ∂ θ . 0=\frac{\partial\widehat{L}_T(\widehat{\theta}_T)}{\partial\theta}. 0 = ∂ θ ∂ L T ( θ T ) .
By the Taylor expansion, we have that
0 = 1 T ∂ L ^ T ( θ 0 ) ∂ θ + H T ∗ T ( θ ^ T − θ 0 ) , 0=\frac{1}{\sqrt{T}}\frac{\partial\widehat{L}_T(\theta_0)}{\partial\theta}+H_T^*\sqrt{T}(\widehat{\theta}_T-\theta_0), 0 = T 1 ∂ θ ∂ L T ( θ 0 ) + H T ∗ T ( θ T − θ 0 ) ,
where
H T ∗ = ∫ 0 1 1 T ∂ 2 L ^ T { θ 0 + s ( θ ^ T − θ 0 ) } ∂ θ ∂ θ ⊤ d s . H_T^*=\int_0^1\frac{1}{T}\frac{\partial^2\widehat{L}_T\{\theta_0+s(\widehat{\theta}_T-\theta_0)\}}{\partial\theta\partial\theta^\top}\,ds. H T ∗ = ∫ 0 1 T 1 ∂ θ ∂ θ ⊤ ∂ 2 L T { θ 0 + s ( θ T − θ 0 )} d s .
Let N ( θ 0 ) ⊂ N \mathcal{N}(\theta_0)\subset\mathcal{N} N ( θ 0 ) ⊂ N be a compact neighborhood of θ 0 \theta_0 θ 0 contained in the interior of Θ \Theta Θ . By Lemma 6 and Lemma 7, we can show that
sup θ ∈ N ( θ 0 ) ∥ 1 T ∑ t = 1 T { ∂ 2 ℓ ^ t ( θ ) ∂ θ ∂ θ ⊤ − ∂ 2 ℓ t ( θ ) ∂ θ ∂ θ ⊤ } ∥ = o p ( 1 ) \sup_{\theta\in\mathcal{N}(\theta_0)}
\left\|
\frac{1}{T}\sum_{t=1}^{T}
\left\{
\frac{\partial^2\widehat{\ell}_t(\theta)}{\partial\theta\partial\theta^\top}
-
\frac{\partial^2\ell_t(\theta)}{\partial\theta\partial\theta^\top}
\right\}
\right\|=o_p(1) θ ∈ N ( θ 0 ) sup T 1 t = 1 ∑ T { ∂ θ ∂ θ ⊤ ∂ 2 ℓ t ( θ ) − ∂ θ ∂ θ ⊤ ∂ 2 ℓ t ( θ ) } = o p ( 1 )
for a neighborhood N ( θ 0 ) \mathcal{N}(\theta_0) N ( θ 0 ) of θ 0 \theta_0 θ 0 . Moreover, Lemma 8 and Lemma 2.4 of Newey and McFadden [29 ] , we can show that
sup θ ∈ N ( θ 0 ) ∥ 1 T ∑ t = 1 T ∂ 2 ℓ t ( θ ) ∂ θ ∂ θ ⊤ − E ∂ 2 ℓ t ( θ ) ∂ θ ∂ θ ⊤ ∥ = o p ( 1 ) . \sup_{\theta\in\mathcal{N}(\theta_0)}
\left\|
\frac{1}{T}\sum_{t=1}^{T}\frac{\partial^2\ell_t(\theta)}{\partial\theta\partial\theta^\top}
-
E\frac{\partial^2\ell_t(\theta)}{\partial\theta\partial\theta^\top}
\right\|=o_p(1). θ ∈ N ( θ 0 ) sup T 1 t = 1 ∑ T ∂ θ ∂ θ ⊤ ∂ 2 ℓ t ( θ ) − E ∂ θ ∂ θ ⊤ ∂ 2 ℓ t ( θ ) = o p ( 1 ) .
Since E [ ∂ 2 ℓ t ( θ ) / ( ∂ θ ∂ θ ⊤ ) ] E[\partial^2\ell_t(\theta)/(\partial\theta\partial\theta^\top)] E [ ∂ 2 ℓ t ( θ ) / ( ∂ θ ∂ θ ⊤ )] is continuous on N ( θ 0 ) \mathcal{N}(\theta_0) N ( θ 0 ) by dominated convergence theorem, and that θ ^ T → p θ 0 \widehat{\theta}_T\to_p\theta_0 θ T → p θ 0 , the line segment { θ 0 + s ( θ ^ T − θ 0 ) : s ∈ [ 0 , 1 ] } \{\theta_0+s(\widehat{\theta}_T-\theta_0):s\in[0,1]\} { θ 0 + s ( θ T − θ 0 ) : s ∈ [ 0 , 1 ]} lies in N ( θ 0 ) \mathcal{N}(\theta_0) N ( θ 0 ) with probability approaching one, which yields
H T ∗ → p E [ ∂ 2 ℓ t ( θ 0 ) ∂ θ ∂ θ ⊤ ] = H . (99) H_T^*\to_p E\left[\frac{\partial^2\ell_t(\theta_0)}{\partial\theta\partial\theta^\top}\right]=H. \tag*{(99)} H T ∗ → p E [ ∂ θ ∂ θ ⊤ ∂ 2 ℓ t ( θ 0 ) ] = H . (99)
Since H H H is nonsingular, H T ∗ H_T^* H T ∗ is invertible with probability approaching one. Hence, we can write
T ( θ ^ T − θ 0 ) = − H T ∗ − 1 1 T ∂ L ^ T ( θ 0 ) ∂ θ = − [ H − 1 + o p ( 1 ) ] 1 T ∂ L ^ T ( θ 0 ) ∂ θ . (100) \sqrt{T}(\widehat{\theta}_T-\theta_0)
=-H_T^{*-1}\frac{1}{\sqrt{T}}\frac{\partial\widehat{L}_T(\theta_0)}{\partial\theta}
=-[H^{-1}+o_p(1)]\frac{1}{\sqrt{T}}\frac{\partial\widehat{L}_T(\theta_0)}{\partial\theta}. \tag*{(100)} T ( θ T − θ 0 ) = − H T ∗− 1 T 1 ∂ θ ∂ L T ( θ 0 ) = − [ H − 1 + o p ( 1 )] T 1 ∂ θ ∂ L T ( θ 0 ) . (100)
Step 2. Throughout Step 2, we work on the high-probability event δ T : = max 1 ≤ i ≤ T ∣ τ i / τ ˉ i − 1 ∣ ≤ 1 / 2 \delta_T := \max_{1\le i\le T}|\tau_i/\bar{\tau}_i-1|\le1/2 δ T := max 1 ≤ i ≤ T ∣ τ i / τ ˉ i − 1∣ ≤ 1/2 . Recall the linear interpolation of squared returns u i 2 ( a ) = u i 2 + a ( u ˉ i 2 − u i 2 ) u_i^2(a)=u_i^2+a(\bar{u}_i^2-u_i^2) u i 2 ( a ) = u i 2 + a ( u ˉ i 2 − u i 2 ) for a ∈ [ 0 , 1 ] a\in[0,1] a ∈ [ 0 , 1 ] and 1 ≤ i ≤ T 1\le i\le T 1 ≤ i ≤ T , and let s t ( a , θ 0 ) s_t(a,\theta_0) s t ( a , θ 0 ) denote the score at θ 0 \theta_0 θ 0 evaluated along the sample inputs { u i 2 ( a ) } i = 1 T \{u_i^2(a)\}_{i=1}^T { u i 2 ( a ) } i = 1 T while holding the stationary presample state ( u 0 2 , h 0 ( θ 0 ) ) (u_0^2,h_0(\theta_0)) ( u 0 2 , h 0 ( θ 0 )) fixed. We decompose T − 1 / 2 ∂ θ L T ( θ 0 ) T^{-1/2}\partial_\theta L_T(\theta_0) T − 1/2 ∂ θ L T ( θ 0 ) in four sub-steps.
Step 2(a): Taylor expansion. At a = 1 a=1 a = 1 , the feasible score ∂ θ ℓ ^ t ( θ 0 ) \partial_\theta\hat{\ell}_t(\theta_0) ∂ θ ℓ ^ t ( θ 0 ) and the path endpoint s t ( 1 , θ 0 ) s_t(1,\theta_0) s t ( 1 , θ 0 ) share identical sample inputs { u ˉ i 2 } i = 1 T \{\bar{u}_i^2\}_{i=1}^T { u ˉ i 2 } i = 1 T and differ only in their initial values at t = 0 t=0 t = 0 . By Lemma 6 (r = 1 r=1 r = 1 at a = 1 a=1 a = 1 ),
R 1 , T : = 1 T ∑ t = 1 T { ∂ ℓ ^ t ( θ 0 ) ∂ θ − s t ( 1 , θ 0 ) } = O p ( T − 1 / 2 ) = o p ( 1 ) . R_{1,T}:=\frac{1}{\sqrt{T}}\sum_{t=1}^T\left\{\frac{\partial\hat{\ell}_t(\theta_0)}{\partial\theta}-s_t(1,\theta_0)\right\}=O_p(T^{-1/2})=o_p(1). R 1 , T := T 1 t = 1 ∑ T { ∂ θ ∂ ℓ ^ t ( θ 0 ) − s t ( 1 , θ 0 ) } = O p ( T − 1/2 ) = o p ( 1 ) .
At a = 0 a=0 a = 0 , s t ( 0 , θ 0 ) = s t ( θ 0 ) s_t(0,\theta_0)=s_t(\theta_0) s t ( 0 , θ 0 ) = s t ( θ 0 ) is the true stationary infeasible score. Applying the second-order Taylor to a ↦ ∑ t = 1 T s t ( a , θ 0 ) a\mapsto\sum_{t=1}^T s_t(a,\theta_0) a ↦ ∑ t = 1 T s t ( a , θ 0 ) around a = 0 a=0 a = 0 gives
1 T ∑ t = 1 T s t ( 1 , θ 0 ) = 1 T ∑ t = 1 T s t ( θ 0 ) + 1 T ∑ t = 1 T ∂ s t ( a , θ 0 ) ∂ a ∣ a = 0 + R 2 , T , \frac{1}{\sqrt{T}}\sum_{t=1}^T s_t(1,\theta_0)=\frac{1}{\sqrt{T}}\sum_{t=1}^T s_t(\theta_0)+\frac{1}{\sqrt{T}}\sum_{t=1}^T\left.\frac{\partial s_t(a,\theta_0)}{\partial a}\right|_{a=0}+R_{2,T}, T 1 t = 1 ∑ T s t ( 1 , θ 0 ) = T 1 t = 1 ∑ T s t ( θ 0 ) + T 1 t = 1 ∑ T ∂ a ∂ s t ( a , θ 0 ) a = 0 + R 2 , T ,
where R 2 , T : = T − 1 / 2 ∫ 0 1 ( 1 − a ) ∑ t = 1 T { ∂ 2 s t ( a , θ 0 ) / ∂ a 2 } d a R_{2,T}:=T^{-1/2}\int_0^1(1-a)\sum_{t=1}^T\{\partial^2s_t(a,\theta_0)/\partial a^2\}\,da R 2 , T := T − 1/2 ∫ 0 1 ( 1 − a ) ∑ t = 1 T { ∂ 2 s t ( a , θ 0 ) / ∂ a 2 } d a . By (A.55) in Lemma 8 , the ergodic theorem for J t ∈ L 1 J_t\in L^1 J t ∈ L 1 , and T κ T 2 ≤ 2 log T / ( T h ) + 2 T h 2 → 0 \sqrt{T}\kappa_T^2\le2\log T/(\sqrt{T}h)+2\sqrt{T}h^2\to0 T κ T 2 ≤ 2 log T / ( T h ) + 2 T h 2 → 0 under Assumption 5,
∥ R 2 , T ∥ ≤ 1 T ∑ t = 1 T sup 0 ≤ a ≤ 1 ∥ ∂ 2 s t ( a , θ 0 ) ∂ a 2 ∥ ≤ C T δ T 2 ( 1 T ∑ t = 1 T J t ) = O p ( T κ T 2 ) = o p ( 1 ) . \lVert R_{2,T}\rVert\le\frac{1}{\sqrt{T}}\sum_{t=1}^T\sup_{0\le a\le1}\left\lVert\frac{\partial^2s_t(a,\theta_0)}{\partial a^2}\right\rVert\le C\sqrt{T}\delta_T^2\left(\frac{1}{T}\sum_{t=1}^T J_t\right)=O_p(\sqrt{T}\kappa_T^2)=o_p(1). ∥ R 2 , T ∥ ≤ T 1 t = 1 ∑ T 0 ≤ a ≤ 1 sup ∂ a 2 ∂ 2 s t ( a , θ 0 ) ≤ C T δ T 2 ( T 1 t = 1 ∑ T J t ) = O p ( T κ T 2 ) = o p ( 1 ) .
Step 2(b): Linearization in the first-step relative error. By the chain rule, ∂ s t ( a , θ 0 ) / ∂ a ∣ a = 0 = ∑ i = 1 t { ∂ s t ( θ 0 ) / ∂ ( u i 2 ) } ( u ˉ i 2 − u i 2 ) \left.\partial s_t(a,\theta_0)/\partial a\right|_{a=0}=\sum_{i=1}^t\{\partial s_t(\theta_0)/\partial(u_i^2)\}(\bar{u}_i^2-u_i^2) ∂ s t ( a , θ 0 ) / ∂ a ∣ a = 0 = ∑ i = 1 t { ∂ s t ( θ 0 ) / ∂ ( u i 2 )} ( u ˉ i 2 − u i 2 ) . Interchanging the order of summation ∑ t = 1 T ∑ i = 1 t = ∑ i = 1 T ∑ t = i T \sum_{t=1}^T\sum_{i=1}^t=\sum_{i=1}^T\sum_{t=i}^T ∑ t = 1 T ∑ i = 1 t = ∑ i = 1 T ∑ t = i T and substituting u ˉ i 2 − u i 2 = − u i 2 δ i , T + u i 2 R i T \bar{u}_i^2-u_i^2=-u_i^2\delta_{i,T}+u_i^2R_{iT} u ˉ i 2 − u i 2 = − u i 2 δ i , T + u i 2 R i T from (A.36) in Lemma 5 (where δ i , T = ( τ ˉ i − τ i ) / τ i \delta_{i,T}=(\bar{\tau}_i-\tau_i)/\tau_i δ i , T = ( τ ˉ i − τ i ) / τ i and max 1 ≤ i ≤ T ∣ R i T ∣ = O p ( κ T 2 ) \max_{1\le i\le T}|R_{iT}|=O_p(\kappa_T^2) max 1 ≤ i ≤ T ∣ R i T ∣ = O p ( κ T 2 ) ) gives
1 T ∑ t = 1 T ∂ s t ( a , θ 0 ) ∂ a ∣ a = 0 = − 1 T ∑ i = 1 T A i , T δ i , T + R 3 , T , A i , T : = u i 2 ∑ t = i T ∂ s t ( θ 0 ) ∂ ( u i 2 ) , \frac{1}{\sqrt{T}}\sum_{t=1}^T\left.\frac{\partial s_t(a,\theta_0)}{\partial a}\right|_{a=0}=-\frac{1}{\sqrt{T}}\sum_{i=1}^T A_{i,T}\delta_{i,T}+R_{3,T},\qquad A_{i,T}:=u_i^2\sum_{t=i}^T\frac{\partial s_t(\theta_0)}{\partial(u_i^2)}, T 1 t = 1 ∑ T ∂ a ∂ s t ( a , θ 0 ) a = 0 = − T 1 i = 1 ∑ T A i , T δ i , T + R 3 , T , A i , T := u i 2 t = i ∑ T ∂ ( u i 2 ) ∂ s t ( θ 0 ) ,
where R 3 , T : = T − 1 / 2 ∑ i = 1 T A i , T R i T R_{3,T}:=T^{-1/2}\sum_{i=1}^T A_{i,T}R_{iT} R 3 , T := T − 1/2 ∑ i = 1 T A i , T R i T . By (A.54) in Lemma 8 , sup 1 ≤ i ≤ T ∥ A i , T ∥ p ≤ ∑ j = 0 ∞ ∥ W i , j ∥ p ≤ C < ∞ \sup_{1\le i\le T}\lVert A_{i,T}\rVert_p\le\sum_{j=0}^{\infty}\lVert W_{i,j}\rVert_p\le C<\infty sup 1 ≤ i ≤ T ∥ A i , T ∥ p ≤ ∑ j = 0 ∞ ∥ W i , j ∥ p ≤ C < ∞ (p = q / 2 > 2 p=q/2>2 p = q /2 > 2 ), so Markov’s inequality implies T − 1 ∑ i = 1 T ∥ A i , T ∥ = O p ( 1 ) T^{-1}\sum_{i=1}^T\lVert A_{i,T}\rVert=O_p(1) T − 1 ∑ i = 1 T ∥ A i , T ∥ = O p ( 1 ) and hence
∥ R 3 , T ∥ ≤ T ( max 1 ≤ i ≤ T ∣ R i T ∣ ) 1 T ∑ i = 1 T ∥ A i , T ∥ = O p ( T κ T 2 ) = o p ( 1 ) . \lVert R_{3,T}\rVert\le\sqrt{T}\left(\max_{1\le i\le T}|R_{iT}|\right)\frac{1}{T}\sum_{i=1}^T\lVert A_{i,T}\rVert=O_p(\sqrt{T}\kappa_T^2)=o_p(1). ∥ R 3 , T ∥ ≤ T ( 1 ≤ i ≤ T max ∣ R i T ∣ ) T 1 i = 1 ∑ T ∥ A i , T ∥ = O p ( T κ T 2 ) = o p ( 1 ) .
Combining Steps 2(a) and 2(b) yields the intermediate score expansion
1 T ∂ L ^ T ( θ 0 ) ∂ θ = 1 T ∑ t = 1 T s t ( θ 0 ) − 1 T ∑ i = 1 T A i , T δ i , T + o p ( 1 ) . (101) \frac{1}{\sqrt{T}}\frac{\partial\hat{L}_T(\theta_0)}{\partial\theta}=\frac{1}{\sqrt{T}}\sum_{t=1}^T s_t(\theta_0)-\frac{1}{\sqrt{T}}\sum_{i=1}^T A_{i,T}\delta_{i,T}+o_p(1). \tag*{(101)} T 1 ∂ θ ∂ L ^ T ( θ 0 ) = T 1 t = 1 ∑ T s t ( θ 0 ) − T 1 i = 1 ∑ T A i , T δ i , T + o p ( 1 ) . (101)
Step 2(c): Split boundary and interior observations. We partition the input index i ∈ { 1 , … , T } i\in\{1,\ldots,T\} i ∈ { 1 , … , T } into the boundary set B T B_T B T (∣ B T ∣ = 2 m T = O ( T h ) |B_T|=2m_T=O(Th) ∣ B T ∣ = 2 m T = O ( T h ) ) and the interior set I T = { m T + 1 , … , T − m T } I_T=\{m_T+1,\ldots,T-m_T\} I T = { m T + 1 , … , T − m T } .
Boundary terms (i ∈ B T i \in B_T i ∈ B T ): Since sup i , T E ∥ A i , T ∥ ≤ C \sup_{i,T} E\lVert A_{i,T}\rVert\le C sup i , T E ∥ A i , T ∥ ≤ C , Markov’s inequality gives ∑ i ∈ B T ∥ A i , T ∥ = O p ( T h ) \sum_{i\in B_T}\lVert A_{i,T}\rVert= O_p(Th) ∑ i ∈ B T ∥ A i , T ∥ = O p ( T h ) . Combining this with max 1 ≤ i ≤ T ∣ δ i , T ∣ = O p ( κ T ) \max_{1\le i\le T}|\delta_{i,T}| = O_p(\kappa_T) max 1 ≤ i ≤ T ∣ δ i , T ∣ = O p ( κ T ) from Proposition 6 and Assumption 5 yields
1 T ∥ ∑ i ∈ B T A i , T δ i , T ∥ ≤ max 1 ≤ i ≤ T ∣ δ i , T ∣ T ∑ i ∈ B T ∥ A i , T ∥ = O p ( T h κ T ) = o p ( 1 ) . (102) \frac{1}{\sqrt{T}}\left\lVert\sum_{i\in B_T} A_{i,T}\delta_{i,T}\right\rVert
\le\frac{\max_{1\le i\le T}|\delta_{i,T}|}{\sqrt{T}}\sum_{i\in B_T}\lVert A_{i,T}\rVert
= O_p(\sqrt{T}h\kappa_T)=o_p(1). \tag*{(102)} T 1 i ∈ B T ∑ A i , T δ i , T ≤ T max 1 ≤ i ≤ T ∣ δ i , T ∣ i ∈ B T ∑ ∥ A i , T ∥ = O p ( T h κ T ) = o p ( 1 ) . (102)
Replacement of A i , T A_{i,T} A i , T by A i A_i A i on I T \mathcal{I}_T I T : Recall A i = u i 2 { ∂ s t ( θ 0 ) / ∂ ( u i 2 ) } A_i=u_i^2\{\partial s_t(\theta_0)/\partial(u_i^2)\} A i = u i 2 { ∂ s t ( θ 0 ) / ∂ ( u i 2 )} from Lemma 9 . By (A.54), ∥ A i − A i , T ∥ p ≤ ∑ j = T − i + 1 ∞ ∥ W i , j ∥ p ≤ C ϱ T − i \lVert A_i-A_{i,T}\rVert_p\le\sum_{j=T-i+1}^{\infty}\lVert W_{i,j}\rVert_p\le C\varrho^{T-i} ∥ A i − A i , T ∥ p ≤ ∑ j = T − i + 1 ∞ ∥ W i , j ∥ p ≤ C ϱ T − i for some ϱ ∈ ( 0 , 1 ) \varrho\in(0,1) ϱ ∈ ( 0 , 1 ) . Since T − i ≥ m T = ⌊ T h ⌋ T-i\ge m_T=\lfloor Th\rfloor T − i ≥ m T = ⌊ T h ⌋ for i ∈ I T i\in\mathcal{I}_T i ∈ I T , we have E ∑ i ∈ I T ∥ A i , T − A i ∥ ≤ C ∑ k = m T ∞ ϱ k = O ( ϱ T h ) E\sum_{i\in\mathcal{I}_T}\lVert A_{i,T}-A_i\rVert\le C\sum_{k=m_T}^{\infty}\varrho^k=O(\varrho^{Th}) E ∑ i ∈ I T ∥ A i , T − A i ∥ ≤ C ∑ k = m T ∞ ϱ k = O ( ϱ T h ) , and therefore
1 T ∥ ∑ i ∈ I T ( A i , T − A i ) δ i , T ∥ ≤ max 1 ≤ i ≤ T ∣ δ i , T ∣ T ∑ i ∈ I T ∥ A i , T − A i ∥ = O p ( T − 1 / 2 κ T ϱ T h ) = o p ( 1 ) . \frac{1}{\sqrt{T}}\left\lVert\sum_{i\in\mathcal{I}_T}(A_{i,T}-A_i)\delta_{i,T}\right\rVert
\le\frac{\max_{1\le i\le T}|\delta_{i,T}|}{\sqrt{T}}\sum_{i\in\mathcal{I}_T}\lVert A_{i,T}-A_i\rVert
=O_p(T^{-1/2}\kappa_T\varrho^{Th})=o_p(1). T 1 i ∈ I T ∑ ( A i , T − A i ) δ i , T ≤ T max 1 ≤ i ≤ T ∣ δ i , T ∣ i ∈ I T ∑ ∥ A i , T − A i ∥ = O p ( T − 1/2 κ T ϱ T h ) = o p ( 1 ) .
Substitution of the interior kernel decomposition: Note that for t ∈ I T t\in\mathcal{I}_T t ∈ I T , we have
δ t , T = 1 T h ∑ s = 1 T K ( t − s T h ) z s + d t , T + r t , T , (103) \delta_{t,T}=\frac{1}{Th}\sum_{s=1}^{T}K\left(\frac{t-s}{Th}\right)z_s+d_{t,T}+r_{t,T}, \tag*{(103)} δ t , T = T h 1 s = 1 ∑ T K ( T h t − s ) z s + d t , T + r t , T , (103)
where d t , T = h 2 C 2 τ ′ ′ ( t / T ) / ( 2 τ t ) d_{t,T}=h^2C_2\tau^{\prime\prime}(t/T)/(2\tau_t) d t , T = h 2 C 2 τ ′′ ( t / T ) / ( 2 τ t ) , and max t ∈ I T ∣ d t , T ∣ = O ( h 2 ) \max_{t\in\mathcal{I}_T}|d_{t,T}|=O(h^2) max t ∈ I T ∣ d t , T ∣ = O ( h 2 ) , T max t ∈ I T ∣ r t , T ∣ = O ( T − 1 / 2 h − 1 ) + o ( T h 2 ) + O p ( h log T ) = o p ( 1 ) \sqrt{T}\max_{t\in\mathcal{I}_T}|r_{t,T}|=O(T^{-1/2}h^{-1})+o(\sqrt{T}h^2)+O_p(h\sqrt{\log T})=o_p(1) T max t ∈ I T ∣ r t , T ∣ = O ( T − 1/2 h − 1 ) + o ( T h 2 ) + O p ( h log T ) = o p ( 1 ) . Then let M : = T h M:=Th M := T h , we obtain that
1 T ∑ i ∈ I T A i δ i , T = 1 T M ∑ i ∈ I T ∑ s = 1 T K ( i − s M ) A i z s + R 4 , T , \frac{1}{\sqrt{T}}\sum_{i\in\mathcal{I}_T}A_i\delta_{i,T}
=\frac{1}{\sqrt{TM}}\sum_{i\in\mathcal{I}_T}\sum_{s=1}^{T}K\left(\frac{i-s}{M}\right)A_i z_s+R_{4,T}, T 1 i ∈ I T ∑ A i δ i , T = T M 1 i ∈ I T ∑ s = 1 ∑ T K ( M i − s ) A i z s + R 4 , T ,
where
∥ R 4 , T ∥ ≤ T ( max i ∈ I T ∣ d i , T ∣ + max i ∈ I T ∣ r i , T ∣ ) 1 T ∑ i = 1 T ∥ A i ∥ = O p ( T h 2 ) + o p ( 1 ) = o p ( 1 ) . \lVert R_{4,T}\rVert\le\sqrt{T}\left(\max_{i\in\mathcal{I}_T}|d_{i,T}|+\max_{i\in\mathcal{I}_T}|r_{i,T}|\right)\frac{1}{T}\sum_{i=1}^{T}\lVert A_i\rVert
=O_p(\sqrt{T}h^2)+o_p(1)=o_p(1). ∥ R 4 , T ∥ ≤ T ( i ∈ I T max ∣ d i , T ∣ + i ∈ I T max ∣ r i , T ∣ ) T 1 i = 1 ∑ T ∥ A i ∥ = O p ( T h 2 ) + o p ( 1 ) = o p ( 1 ) .
Step 2(d): Evaluation of the kernel projection via centering A i = C τ + ( A i − C τ ) A_i=C_\tau+(A_i-C_\tau) A i = C τ + ( A i − C τ ) . Define the local kernel average z ˉ i , M : = M − 1 ∑ s = 1 T K { ( i − s ) / M } z s \bar z_{i,M}:=M^{-1}\sum_{s=1}^{T}K\{(i-s)/M\}z_s z ˉ i , M := M − 1 ∑ s = 1 T K {( i − s ) / M } z s . Since { z s } \{z_s\} { z s } is stationary with ∑ k ∈ Z ∣ Cov ( z 0 , z k ) ∣ < ∞ \sum_{k\in\mathbb{Z}}|\operatorname{Cov}(z_0,z_k)|<\infty ∑ k ∈ Z ∣ Cov ( z 0 , z k ) ∣ < ∞ and K K K is bounded and supported on [ − 1 , 1 ] [-1,1] [ − 1 , 1 ] ,
sup 1 ≤ i ≤ T ∥ z ˉ i , M ∥ 2 2 = sup 1 ≤ i ≤ T 1 M 2 ∑ s , s ′ = 1 T K ( i − s M ) K ( i − s ′ M ) Cov ( z s , z s ′ ) ≤ C M ∑ k ∈ Z ∣ Cov ( z 0 , z k ) ∣ = O ( M − 1 ) , \begin{aligned}
\sup_{1\le i\le T}\lVert\bar z_{i,M}\rVert_2^2
=\sup_{1\le i\le T}\frac{1}{M^2}\sum_{s,s'=1}^{T}K\left(\frac{i-s}{M}\right)K\left(\frac{i-s'}{M}\right)\operatorname{Cov}(z_s,z_{s'}) \\
&\le\frac{C}{M}\sum_{k\in\mathbb{Z}}|\operatorname{Cov}(z_0,z_k)|=O(M^{-1}),
\end{aligned} 1 ≤ i ≤ T sup ∥ z ˉ i , M ∥ 2 2 = 1 ≤ i ≤ T sup M 2 1 s , s ′ = 1 ∑ T K ( M i − s ) K ( M i − s ′ ) Cov ( z s , z s ′ ) ≤ M C k ∈ Z ∑ ∣ Cov ( z 0 , z k ) ∣ = O ( M − 1 ) ,
so sup 1 ≤ i ≤ T ∥ z ˉ i , M ∥ 2 ≤ C M − 1 / 2 \sup_{1\le i\le T}\lVert\bar z_{i,M}\rVert_2\le CM^{-1/2} sup 1 ≤ i ≤ T ∥ z ˉ i , M ∥ 2 ≤ C M − 1/2 . We now evaluate T − 1 / 2 ∑ i ∈ I T A i z ˉ i , M T^{-1/2}\sum_{i\in\mathcal{I}_T}A_i\bar z_{i,M} T − 1/2 ∑ i ∈ I T A i z ˉ i , M in three steps:
Extending the outer sum from I T \mathcal{I}_T I T to { 1 , … , T } \{1,\ldots,T\} { 1 , … , T } : By Cauchy–Schwarz inequality, E ∥ A i z ˉ i , M ∥ ≤ ∥ A i ∥ 2 ∥ z ˉ i , M ∥ 2 ≤ C M − 1 / 2 E\lVert A_i\bar z_{i,M}\rVert\le\lVert A_i\rVert_2\lVert\bar z_{i,M}\rVert_2\le CM^{-1/2} E ∥ A i z ˉ i , M ∥ ≤ ∥ A i ∥ 2 ∥ z ˉ i , M ∥ 2 ≤ C M − 1/2 . Since ∣ B T ∣ = O ( M ) |B_T|=O(M) ∣ B T ∣ = O ( M ) ,
E ∥ 1 T ∑ i ∈ B T A i z ˉ i , M ∥ ≤ 1 T ∑ i ∈ B T ∥ A i ∥ 2 ∥ z ˉ i , M ∥ 2 ≤ C M T M − 1 / 2 = O ( h ) = o ( 1 ) . E\left\lVert\frac{1}{\sqrt{T}}\sum_{i\in B_T}A_i\bar z_{i,M}\right\rVert
\le\frac{1}{\sqrt{T}}\sum_{i\in B_T}\lVert A_i\rVert_2\lVert\bar z_{i,M}\rVert_2
\le C\frac{M}{\sqrt{T}}M^{-1/2}=O(\sqrt{h})=o(1). E T 1 i ∈ B T ∑ A i z ˉ i , M ≤ T 1 i ∈ B T ∑ ∥ A i ∥ 2 ∥ z ˉ i , M ∥ 2 ≤ C T M M − 1/2 = O ( h ) = o ( 1 ) .
Thus we may replace ∑ i ∈ I T \sum_{i\in\mathcal{I}_T} ∑ i ∈ I T by ∑ i = 1 T \sum_{i=1}^{T} ∑ i = 1 T at cost o p ( 1 ) o_p(1) o p ( 1 ) , and decompose A i = C τ + A ~ i A_i=C_\tau+\widetilde A_i A i = C τ + A i with A ~ i : = A i − C τ \widetilde A_i:=A_i-C_\tau A i := A i − C τ .
The mean-loading term C τ C_\tau C τ : Put ω s , T : = M − 1 ∑ i = 1 T K { ( i − s ) / M } \omega_{s,T}:=M^{-1}\sum_{i=1}^{T}K\{(i-s)/M\} ω s , T := M − 1 ∑ i = 1 T K {( i − s ) / M } . Since K K K is Lipschitz and ∫ − 1 1 K ( v ) d v = 1 \int_{-1}^{1}K(v)\,\mathrm{d}v=1 ∫ − 1 1 K ( v ) d v = 1 , Riemann-sum approximation gives max s ∈ I T ∣ ω s , T − 1 ∣ = O ( M − 1 ) \max_{s\in\mathcal{I}_T}|\omega_{s,T}-1|=O(M^{-1}) max s ∈ I T ∣ ω s , T − 1∣ = O ( M − 1 ) , while max s ∈ B T ∣ ω s , T − 1 ∣ ≤ C \max_{s\in\mathcal{B}_T}|\omega_{s,T}-1|\le C max s ∈ B T ∣ ω s , T − 1∣ ≤ C on the O ( M ) O(M) O ( M ) boundary indices. By covariance summability of { z s } \{z_s\} { z s } ,
∥ 1 T ∑ s = 1 T ( ω s , T − 1 ) z s ∥ 2 2 ≤ C T ∑ s = 1 T ∣ ω s , T − 1 ∣ 2 ∑ k ∈ Z ∣ Cov ( z 0 , z k ) ∣ = O ( h + M − 2 ) = o ( 1 ) . \left\|\frac{1}{\sqrt{T}}\sum_{s=1}^{T}(\omega_{s,T}-1)z_s\right\|_2^2\le\frac{C}{T}\sum_{s=1}^{T}|\omega_{s,T}-1|^2\sum_{k\in\mathbb{Z}}|\operatorname{Cov}(z_0,z_k)|=O(h+M^{-2})=o(1). T 1 s = 1 ∑ T ( ω s , T − 1 ) z s 2 2 ≤ T C s = 1 ∑ T ∣ ω s , T − 1 ∣ 2 k ∈ Z ∑ ∣ Cov ( z 0 , z k ) ∣ = O ( h + M − 2 ) = o ( 1 ) .
Consequently,
C τ T − 1 / 2 ∑ i = 1 T z ˉ i , M = C τ T − 1 / 2 ∑ s = 1 T ω s , T z s = C τ T − 1 / 2 ∑ s = 1 T z s + o p ( 1 ) . C_\tau T^{-1/2}\sum_{i=1}^{T}\bar{z}_{i,M}=C_\tau T^{-1/2}\sum_{s=1}^{T}\omega_{s,T}z_s=C_\tau T^{-1/2}\sum_{s=1}^{T}z_s+o_p(1). C τ T − 1/2 i = 1 ∑ T z ˉ i , M = C τ T − 1/2 s = 1 ∑ T ω s , T z s = C τ T − 1/2 s = 1 ∑ T z s + o p ( 1 ) .
The centered cross-term A ~ i = A i − C τ \widetilde{A}_i=A_i-C_\tau A i = A i − C τ : It remains to show that
Δ A , T : = 1 T M ∑ i , s = 1 T K ( i − s M ) A ~ i z s = 1 T h { 1 T ∑ i , s = 1 T K ( i − s M ) A ~ i z s } = o p ( 1 ) . \Delta_{A,T}:=\frac{1}{\sqrt{TM}}\sum_{i,s=1}^{T}K\left(\frac{i-s}{M}\right)\widetilde{A}_iz_s=\frac{1}{\sqrt{Th}}\left\{\frac{1}{T}\sum_{i,s=1}^{T}K\left(\frac{i-s}{M}\right)\widetilde{A}_iz_s\right\}=o_p(1). Δ A , T := T M 1 i , s = 1 ∑ T K ( M i − s ) A i z s = T h 1 { T 1 i , s = 1 ∑ T K ( M i − s ) A i z s } = o p ( 1 ) .
For each coordinate a i a_i a i of A ~ i \widetilde{A}_i A i , Lemma 9 and Proposition 5 give summable L 2 L^2 L 2 physical dependence coefficients ∑ k ∈ Z ∥ Y k − Y k ∗ ∥ 2 < ∞ \sum_{k\in\mathbb{Z}}\|Y_k-Y_k^*\|_2<\infty ∑ k ∈ Z ∥ Y k − Y k ∗ ∥ 2 < ∞ for the two-sided centered stationary scalar processes Y i = a i + z i Y_i=a_i+z_i Y i = a i + z i and Y i = a i − z i Y_i=a_i-z_i Y i = a i − z i . Under Assumption 3, the compactly supported Lipschitz kernel K K K has an absolutely integrable Fourier transform; if K ( 0 ) ≠ 0 K(0)\ne0 K ( 0 ) = 0 we rescale by K ( 0 ) K(0) K ( 0 ) , and if K ( 0 ) = 0 K(0)=0 K ( 0 ) = 0 we write K = ( K + K 0 ) − K 0 K=(K+K_0)-K_0 K = ( K + K 0 ) − K 0 with K 0 ( x ) = ( 1 − ∣ x ∣ ) 1 { ∣ x ∣ ≤ 1 } K_0(x)=(1-|x|)1_{\{|x|\le1\}} K 0 ( x ) = ( 1 − ∣ x ∣ ) 1 { ∣ x ∣ ≤ 1 } . Applying Theorem 1 and Section 2 of Liu and Wu [26 ] at frequency zero (with n = T n=T n = T , B n = M = T h → ∞ B_n=M=Th\to\infty B n = M = T h → ∞ , B n / n = h → 0 B_n/n=h\to0 B n / n = h → 0 , and p = 2 p=2 p = 2 ) yields T − 1 ∑ i , s = 1 T K { ( i − s ) / M } Y i Y s = O p ( 1 ) T^{-1}\sum_{i,s=1}^{T}K\{(i-s)/M\}Y_iY_s=O_p(1) T − 1 ∑ i , s = 1 T K {( i − s ) / M } Y i Y s = O p ( 1 ) for both Y i = a i + z i Y_i=a_i+z_i Y i = a i + z i and Y i = a i − z i Y_i=a_i-z_i Y i = a i − z i . Using the symmetry of K K K , we then obtain that
1 T ∑ i , s = 1 T K ( i − s M ) A ~ i z s = O p ( 1 ) → Δ A , T = O p ( ( T h ) − 1 ) = o p ( 1 ) , \frac{1}{T}\sum_{i,s=1}^{T}K\left(\frac{i-s}{M}\right)\widetilde{A}_iz_s=O_p(1)\quad\to\quad\Delta_{A,T}=O_p\left((\sqrt{Th})^{-1}\right)=o_p(1), T 1 i , s = 1 ∑ T K ( M i − s ) A i z s = O p ( 1 ) → Δ A , T = O p ( ( T h ) − 1 ) = o p ( 1 ) ,
since T h = c h T 1 / 2 − λ h → ∞ \sqrt{Th}=c_hT^{1/2-\lambda_h}\to\infty T h = c h T 1/2 − λ h → ∞ under Assumption 5.
Combining Steps 2(c) and 2(d) establishes that
1 T ∑ i = 1 T A i , T δ i , T = C τ 1 T ∑ s = 1 T z s + o p ( 1 ) . (104) \frac{1}{\sqrt{T}}\sum_{i=1}^{T}A_{i,T}\delta_{i,T}=C_\tau\frac{1}{\sqrt{T}}\sum_{s=1}^{T}z_s+o_p(1). \tag*{(104)} T 1 i = 1 ∑ T A i , T δ i , T = C τ T 1 s = 1 ∑ T z s + o p ( 1 ) . (104)
Substituting (A.89) into (A.86) completes Step 2:
1 T ∂ L ^ T ( θ 0 ) ∂ θ = 1 T ∑ t = 1 T { s t ( θ 0 ) − C τ z t } + o p ( 1 ) = 1 T ∑ t = 1 T ξ t + o p ( 1 ) . \frac{1}{\sqrt{T}}\frac{\partial\widehat{L}_T(\theta_0)}{\partial\theta}=\frac{1}{\sqrt{T}}\sum_{t=1}^{T}\{s_t(\theta_0)-C_\tau z_t\}+o_p(1)=\frac{1}{\sqrt{T}}\sum_{t=1}^{T}\xi_t+o_p(1). T 1 ∂ θ ∂ L T ( θ 0 ) = T 1 t = 1 ∑ T { s t ( θ 0 ) − C τ z t } + o p ( 1 ) = T 1 t = 1 ∑ T ξ t + o p ( 1 ) .
Step 3. By (A.79) in Lemma 11 , replacing z t z_t z t by the martingale difference χ t \chi_t χ t costs o L 2 ( 1 ) o_{L^2}(1) o L 2 ( 1 ) :
1 T ∑ t = 1 T ξ t = 1 T ∑ t = 1 T { s t ( θ 0 ) − C τ χ t } + o L 2 ( 1 ) . \frac{1}{\sqrt{T}}\sum_{t=1}^{T}\xi_t=\frac{1}{\sqrt{T}}\sum_{t=1}^{T}\{s_t(\theta_0)-C_\tau\chi_t\}+o_{L^2}(1). T 1 t = 1 ∑ T ξ t = T 1 t = 1 ∑ T { s t ( θ 0 ) − C τ χ t } + o L 2 ( 1 ) .
The summands ξ ~ t : = s t ( θ 0 ) − C τ χ t \widetilde{\xi}_t := s_t(\theta_0) - C_\tau\chi_t ξ t := s t ( θ 0 ) − C τ χ t form a stationary, ergodic martingale difference sequence with respect to { F t } \{\mathcal{F}_t\} { F t } , with covariance matrix V = E ( ξ ~ t ξ ~ t ⊤ ) = H + Ω z C τ C τ ⊤ V = E(\widetilde{\xi}_t\widetilde{\xi}_t^\top) = H + \Omega_z C_\tau C_\tau^\top V = E ( ξ t ξ t ⊤ ) = H + Ω z C τ C τ ⊤ and finite p p p th moment (p = q / 2 > 2 p = q/2 > 2 p = q /2 > 2 ). By the ergodic theorem,
1 T ∑ t = 1 T E ( ξ ~ t ξ ~ t ⊤ ∣ F t − 1 ) → V a.s. , \frac{1}{T}\sum_{t=1}^{T} E(\widetilde{\xi}_t\widetilde{\xi}_t^\top\mid\mathcal{F}_{t-1}) \to V \quad\text{a.s.}, T 1 t = 1 ∑ T E ( ξ t ξ t ⊤ ∣ F t − 1 ) → V a.s. ,
and for any ε > 0 \varepsilon> 0 ε > 0 , the conditional Lindeberg term satisfies
( ε T ) − ( p − 2 ) T ∑ t = 1 T E ( ∥ ξ ~ t ∥ p ∣ F t − 1 ) = O p ( T − ( p − 2 ) / 2 ) = o p ( 1 ) . \frac{(\varepsilon\sqrt{T})^{-(p-2)}}{T}\sum_{t=1}^{T} E(\|\widetilde{\xi}_t\|^p \mid\mathcal{F}_{t-1}) = O_p(T^{-(p-2)/2}) = o_p(1). T ( ε T ) − ( p − 2 ) t = 1 ∑ T E ( ∥ ξ t ∥ p ∣ F t − 1 ) = O p ( T − ( p − 2 ) /2 ) = o p ( 1 ) .
By Theorem 3(i) of Wu [37 ] , we have that T − 1 / 2 ∑ t = 1 T ξ t → d N ( 0 , V ) T^{-1/2}\sum_{t=1}^{T}\xi_t \to_d N(0,V) T − 1/2 ∑ t = 1 T ξ t → d N ( 0 , V ) . Combining this with (A.84), (A.85), and Slutsky’s theorem gives
T ( θ ^ T − θ 0 ) → d N ( 0 , H − 1 V H − ⊤ ) , \sqrt{T}(\widehat{\theta}_T-\theta_0) \to_d N(0,H^{-1}VH^{-\top}), T ( θ T − θ 0 ) → d N ( 0 , H − 1 V H − ⊤ ) ,
with V V V and Σ = H − 1 V H − ⊤ \Sigma= H^{-1}VH^{-\top} Σ = H − 1 V H − ⊤ given by (A.80). ■
Multi-Step Volatility Forecasts
Write a = α + β a = \alpha+\beta a = α + β and b t = ω + α u t 2 + β h t b_t = \omega+\alpha u_t^2+\beta h_t b t = ω + α u t 2 + β h t . The true-state representation used in the stationarity proof gives
b t = ω + ( α ϵ t 2 + β ) b t − 1 + φ ( α ϵ t 4 + β ϵ t 2 ) . b_t = \omega+ (\alpha\epsilon_t^2+\beta)b_{t-1}+\varphi(\alpha\epsilon_t^4+\beta\epsilon_t^2). b t = ω + ( α ϵ t 2 + β ) b t − 1 + φ ( α ϵ t 4 + β ϵ t 2 ) .
Consequently, E ( b t ∣ F t − 1 ) = a b t − 1 + ω + φ ( α μ 4 + β ) E(b_t \mid\mathcal{F}_{t-1}) = ab_{t-1}+\omega+\varphi(\alpha\mu_4+\beta) E ( b t ∣ F t − 1 ) = a b t − 1 + ω + φ ( α μ 4 + β ) . By normalization, b ˉ = E ( b t ) = 1 − φ μ 4 \bar{b}=E(b_t)=1-\varphi\mu_4 b ˉ = E ( b t ) = 1 − φ μ 4 , so the constant term equals ( 1 − a ) b ˉ (1-a)\bar{b} ( 1 − a ) b ˉ . Iteration from the current state b T b_T b T yields
E ( b T + k − 1 ∣ F T ) = b ˉ + a k − 1 ( b T − b ˉ ) , k ≥ 1. E(b_{T+k-1} \mid\mathcal{F}_T) = \bar{b}+a^{k-1}(b_T-\bar{b}), \qquad k\geq1. E ( b T + k − 1 ∣ F T ) = b ˉ + a k − 1 ( b T − b ˉ ) , k ≥ 1.
Since E ( u T + k 2 ∣ F T + k − 1 ) = b T + k − 1 + φ μ 4 E(u_{T+k}^2 \mid\mathcal{F}_{T+k-1})=b_{T+k-1}+\varphi\mu_4 E ( u T + k 2 ∣ F T + k − 1 ) = b T + k − 1 + φ μ 4 , the tower property then gives
E ( r T + k 2 ∣ F T ) = τ T + k [ 1 + a k − 1 { b T + φ μ 4 − 1 } ] . (105) E(r_{T+k}^2 \mid\mathcal{F}_T) = \tau_{T+k}\left[1+a^{k-1}\{b_T+\varphi\mu_4-1\}\right]. \tag*{(105)} E ( r T + k 2 ∣ F T ) = τ T + k [ 1 + a k − 1 { b T + φ μ 4 − 1 } ] . (105)
This identity requires only the innovation moments in Assumption 1, not Gaussianity. The short-run forecast converges to one as k → ∞ k\to\infty k → ∞ because a < 1 a<1 a < 1 .
Under Gaussian innovations, μ 4 = 3 \mu_4=3 μ 4 = 3 and b T + 3 φ = 1 − α − β + 2 β φ + α u T 2 + β h T b_T+3\varphi=1-\alpha-\beta+2\beta\varphi+\alpha u_T^2+\beta h_T b T + 3 φ = 1 − α − β + 2 β φ + α u T 2 + β h T . Thus the feasible multi-step forecast is
v ^ T + k ∣ T = τ ^ T + k ∣ T [ 1 + ( α ^ + β ^ ) k − 1 { b ^ T + 3 φ ^ − 1 } ] , \widehat{v}_{T+k\mid T}=\widehat{\tau}_{T+k\mid T}\left[1+(\widehat{\alpha}+\widehat{\beta})^{k-1}\{\widehat{b}_T+3\widehat{\varphi}-1\}\right], v T + k ∣ T = τ T + k ∣ T [ 1 + ( α + β ) k − 1 { b T + 3 φ − 1 } ] ,
where b ^ T = ω ( θ ^ ) + α ^ u T 2 + β ^ h T \widehat{b}_T=\omega(\widehat{\theta})+\widehat{\alpha}u_T^2+\widehat{\beta}h_T b T = ω ( θ ) + α u T 2 + β h T . For k = 1 k=1 k = 1 , this is the expression in § 4 .
Randomly Weighted Bootstrap
As a finite-sample robustness check, we also consider a randomly weighted bootstrap that perturbs both stages of the estimation procedure. The procedure is not used for formal inference, since its validity for reproducing the long-run covariance matrix in Theorem 3 is not established here.
Let { r t } t = 1 T \{r_t\}_{t=1}^{T} { r t } t = 1 T denote the observed return series and let θ ^ T \widehat{\theta}_T θ T be the corresponding two-step estimator. For bootstrap replication b b b , proceed as follows.
Algorithm 2 (Randomly weighted bootstrap).
Generate i.i.d. weights
w t ( b ) ∼ Exp ( 1 ) , t = 1 , … , T , w_t^{(b)} \sim\operatorname{Exp}(1), \qquad t=1,\ldots,T, w t ( b ) ∼ Exp ( 1 ) , t = 1 , … , T , so that
E ( w t ( b ) ) = 1 , Var ( w t ( b ) ) = 1. E(w_t^{(b)})=1, \qquad\operatorname{Var}(w_t^{(b)})=1. E ( w t ( b ) ) = 1 , Var ( w t ( b ) ) = 1. Using the bandwidth selected from the original sample, compute the weighted kernel estimator
τ ^ ( b ) ( x ) = ∑ s = 1 T w s ( b ) K h ( x − s T ) r s 2 ∑ s = 1 T w s ( b ) K h ( x − s T ) . \widehat{\tau}^{(b)}(x)=\frac{\sum_{s=1}^{T}w_s^{(b)}K_h\left(x-\frac{s}{T}\right)r_s^2}{\sum_{s=1}^{T}w_s^{(b)}K_h\left(x-\frac{s}{T}\right)}. τ ( b ) ( x ) = ∑ s = 1 T w s ( b ) K h ( x − T s ) ∑ s = 1 T w s ( b ) K h ( x − T s ) r s 2 . At boundary points, we apply the same reflection scheme as in the original first-step estimator. Set
τ ^ t ( b ) = τ ^ ( b ) ( t / T ) , u ^ t ( b ) = r t τ ^ t ( b ) . \widehat{\tau}_t^{(b)}=\widehat{\tau}^{(b)}(t/T), \qquad\widehat{u}_t^{(b)}=\frac{r_t}{\sqrt{\widehat{\tau}_t^{(b)}}}. τ t ( b ) = τ ( b ) ( t / T ) , u t ( b ) = τ t ( b ) r t . For each θ ∈ Θ \theta\in\Theta θ ∈ Θ , construct h ^ t ( b ) ( θ ) \widehat{h}_t^{(b)}(\theta) h t ( b ) ( θ ) recursively from { u ^ t ( b ) } t = 1 T \{\widehat{u}_t^{(b)}\}_{t=1}^{T} { u t ( b ) } t = 1 T using the same initialization rule as in the original estimation, and define
L ^ T ( b ) ( θ ) = ∑ t = 1 T w t ( b ) ℓ ^ t ( b ) ( θ ) , \widehat{L}_T^{(b)}(\theta)=\sum_{t=1}^{T}w_t^{(b)}\widehat{\ell}_t^{(b)}(\theta), L T ( b ) ( θ ) = t = 1 ∑ T w t ( b ) ℓ t ( b ) ( θ ) , where ℓ ^ t ( b ) ( θ ) \widehat{\ell}_t^{(b)}(\theta) ℓ t ( b ) ( θ ) is the negative log-likelihood contribution in (8) evaluated at u ^ t ( b ) \widehat{u}_t^{(b)} u t ( b ) and h ^ t ( b ) ( θ ) \widehat{h}_t^{(b)}(\theta) h t ( b ) ( θ ) .
Compute
θ ^ T ( b ) = arg min θ ∈ Θ L ^ T ( b ) ( θ ) . \widehat{\theta}_T^{(b)}=\underset{\theta\in\Theta}{\arg\min}\ \widehat{L}_T^{(b)}(\theta). θ T ( b ) = θ ∈ Θ arg min L T ( b ) ( θ ) . Repeat Steps 1–4 for b = 1 , … , B b=1,\ldots,B b = 1 , … , B .
The bootstrap standard deviation for parameter component j j j is computed as
se ^ b o o t ( θ ^ j , T ) = [ 1 B − 1 ∑ b = 1 B ( θ ^ j , T ( b ) − θ ‾ j , T ∗ ) 2 ] 1 / 2 , \widehat{\operatorname{se}}_{\mathrm{boot}}\left(\widehat{\theta}_{j,T}\right)=\left[\frac{1}{B-1}\sum_{b=1}^{B}\left(\widehat{\theta}_{j,T}^{(b)}-\overline{\theta}_{j,T}^{*}\right)^2\right]^{1/2}, se boot ( θ j , T ) = [ B − 1 1 b = 1 ∑ B ( θ j , T ( b ) − θ j , T ∗ ) 2 ] 1/2 , where
θ ‾ j , T ∗ = 1 B ∑ b = 1 B θ ^ j , T ( b ) . \overline{\theta}_{j,T}^{*}=\frac{1}{B}\sum_{b=1}^{B}\widehat{\theta}_{j,T}^{(b)}. θ j , T ∗ = B 1 b = 1 ∑ B θ j , T ( b ) .