Introduction
Many statistical estimators are defined implicitly as roots of data-dependent estimating equations. Given a parameter space Θ ⊆ R d \Theta\subseteq\mathbb{R}^{d} Θ ⊆ R d q n : Θ → R d \boldsymbol{q}_{n}: \Theta\to\mathbb{R}^{d} q n : Θ → R d
q n ( θ ∣ X n ) = 0 , (1) \boldsymbol{q}_{n}(\theta\mid\boldsymbol{X}_{n}) = 0, \tag*{(1)} q n ( θ ∣ X n ) = 0 , (1) a root — when it exists — is known as a Z-estimator. When q n \boldsymbol{q}_{n} q n Θ \Theta Θ q n \boldsymbol{q}_{n} q n [e.g. 17 ] also fit into this framework. The study of such estimators dates back to Huber [ [23 ] ; modern treatments are given by van der Vaart [ [38 ] and van der Vaart and Wellner [ [39 ] who develop asymptotic theory of these estimators.
In the simplest models, (1) can be solved explicitly, yielding a closed-form1 X n \boldsymbol{X}_{n} X n Θ \Theta Θ [6 , 29 ] , the Student’s t t t [25 , 19 ] , the von Mises distribution [34 , 21 ] , and families used in extreme value analysis [32 , 16 ] . Although modern root-finding methods and software often make solving such equations routine, an explicit approximation avoids the need for an initial value, a stopping criterion, and repeated computations; it can also be useful in regimes where numerical root-finding is unstable, and can facilitate theoretical analysis by making the estimator’s dependence on the data more transparent. Work on specific models has therefore pursued closed-form or rapidly computable alternatives when exact likelihood estimation requires iteration; examples include Ye and Chen [2017] for the gamma distribution, Clauset et al. [ [9 ] for the zeta distribution, and Ho Chang et al. [ [20 ] for the Dirichlet distribution.
In this paper, we present an approach to construct closed-form approximations to roots of intractable estimating equations of the form (1) by replacing the original estimating equation with a tractable surrogate that admits a closed-form solution, and we develop a general framework for determining when such approximations are accurate and for quantifying the resulting error. The framework considers sequences of data-generating parameters that approach a limiting value; importantly, this limiting value may lie on the boundary of the parameter space or be a formal value such as infinity.
The gamma model provides a simple illustration. Suppose that X 1 , X 2 , … , X n X_{1}, X_{2}, \ldots, X_{n} X 1 , X 2 , … , X n ( α , β ) (\alpha, \beta) ( α , β ) β \beta β α ^ n \hat{\alpha}_n α ^ n
ψ ( α ^ n ) = log ( β G n ) , (2) \psi(\hat{\alpha}_n) = \log(\beta G_n), \tag*{(2)} ψ ( α ^ n ) = log ( β G n ) , (2) where ψ : = ( log ( Γ ) ) ′ \psi:= (\log(\Gamma))' ψ := ( log ( Γ ) ) ′ G n : = ( ∏ i = 1 n X i ) 1 / n G_n := \left(\prod_{i=1}^{n} X_i\right)^{1/n} G n := ( ∏ i = 1 n X i ) 1/ n ψ \psi ψ α ^ n \hat{\alpha}_n α ^ n [15 ] ; Choi and Wette [ [8 ] ; Minka [ [30 ] for computational treatments. However, (2) is equivalent to
exp ( − ψ ( α ^ n ) ) = 1 β G n , (3) \exp(-\psi(\hat{\alpha}_n)) = \frac{1}{\beta G_n}, \tag*{(3)} exp ( − ψ ( α ^ n )) = β G n 1 , (3) and, as α → ∞ \alpha\to\infty α → ∞
exp ( − ψ ( α ) ) = 1 α − 1 / 2 + O ( 1 α 3 ) . \exp(-\psi(\alpha)) = \frac{1}{\alpha- 1/2} + O\left(\frac{1}{\alpha^3}\right). exp ( − ψ ( α )) = α − 1/2 1 + O ( α 3 1 ) . Replacing the left side of (3) by its leading approximation and solving yields the explicit estimator
α ~ n : = 1 2 + β ( ∏ i = 1 n X i ) 1 / n . (4) \tilde{\alpha}_n := \frac{1}{2} + \beta\left(\prod_{i=1}^{n} X_i\right)^{1/n}. \tag*{(4)} α ~ n := 2 1 + β ( i = 1 ∏ n X i ) 1/ n . (4) We show in § 3 that, for data-generating sequences { α n } n ≥ 1 \{\alpha_n\}_{n\geq1} { α n } n ≥ 1 α n → ∞ \alpha_n \to\infty α n → ∞
∣ α ~ n − α ^ n ∣ = o P α n ( 1 ) , \left|\tilde{\alpha}_n-\hat{\alpha}_n\right|=o_{P_{\alpha_n}}(1), ∣ α ~ n − α ^ n ∣ = o P α n ( 1 ) , so the two estimators are equivalent on the original parameter scale. If, in addition, α n 3 / n → ∞ \alpha_n^3/n \to\infty α n 3 / n → ∞
n α n ∣ α ~ n − α ^ n ∣ = o P α n ( 1 ) , \sqrt{\frac{n}{\alpha_n}}\left|\tilde{\alpha}_n-\hat{\alpha}_n\right|=o_{P_{\alpha_n}}(1), α n n ∣ α ~ n − α ^ n ∣ = o P α n ( 1 ) , and under the same condition, we further show that
n α n ( α ~ n − α n ) → d N ( 0 , 1 ) and n α n ( α ^ n − α n ) → d N ( 0 , 1 ) . \sqrt{\frac{n}{\alpha_n}}(\tilde{\alpha}_n-\alpha_n)\xrightarrow{d}\mathcal{N}(0,1)
\quad\text{and}\quad
\sqrt{\frac{n}{\alpha_n}}(\hat{\alpha}_n-\alpha_n)\xrightarrow{d}\mathcal{N}(0,1). α n n ( α ~ n − α n ) d N ( 0 , 1 ) and α n n ( α ^ n − α n ) d N ( 0 , 1 ) . Thus, the approximate estimator and the MLE have the same first-order asymptotic distribution.
The preceding calculation suggests a general strategy. First, rewrite the exact estimating equation q n ( θ ∣ X n ) = 0 \boldsymbol{q}_n(\theta\mid X_n)=\boldsymbol{0} q n ( θ ∣ X n ) = 0 q ~ n ( θ ∣ X n ) = 0 \widetilde{\boldsymbol{q}}_n(\theta\mid X_n)=\boldsymbol{0} q n ( θ ∣ X n ) = 0 θ ~ n \tilde{\theta}_n θ ~ n θ ^ n \hat{\theta}_n θ ^ n θ ^ n \hat{\theta}_n θ ^ n
Although deriving θ ~ n \tilde{\theta}_n θ ~ n θ ^ n \hat{\theta}_n θ ^ n ∥ q n ( θ ∣ X n ) − q ~ n ( θ ∣ X n ) ∥ → p 0 \|\boldsymbol{q}_n(\theta\mid X_n)-\widetilde{\boldsymbol{q}}_n(\theta\mid X_n)\|\xrightarrow{p}0 ∥ q n ( θ ∣ X n ) − q n ( θ ∣ X n ) ∥ p 0 θ ∈ Θ \theta\in\Theta θ ∈ Θ ∥ θ ^ n − θ ~ n ∥ → p 0 \|\hat{\theta}_n-\tilde{\theta}_n\|\xrightarrow{p}0 ∥ θ ^ n − θ ~ n ∥ p 0 θ ^ n \hat{\theta}_n θ ^ n θ ~ n \tilde{\theta}_n θ ~ n [33 ] ; a deterministic antecedent appears in de Bruijn [ [12 ] .
To address these difficulties, we introduce one-to-one transformations of the parameter, which we call vanishing charts , under which the transformed data-generating parameter converges to zero along the regime under consideration. Such regimes can differ substantially from standard fixed-parameter asymptotics, exhibiting altered convergence rates, non-Gaussian limits, or even failure of MLE existence [37 , 36 , 14 ] . After localizing the exact and surrogate roots on a common deterministic set in vanishing chart coordinates, our analysis separates three factors governing their discrepancy: the error incurred when q n q_n q n q ~ n \tilde{q}_n q ~ n θ ~ n \tilde{\theta}_n θ ~ n θ ^ n \hat{\theta}_n θ ^ n
We study the gamma family in detail: the known-rate case is scalar, joint estimation of the shape and rate introduces different componentwise rates, and gamma regression shows that the observations need not be identically distributed (a real-world data example compares the exact and approximate gamma regression fits for measurements of black cherry trees). Our remaining examples highlight other features of the construction, with mathematical details left to the appendix. The zeta distribution is a discrete example where the MLE can fail to be finite. For the von Mises model, we preserve the explicit location estimator while approximating a ratio of Bessel functions for the concentration parameter. The generalized Pareto model has nonregular, parameter-dependent support, requiring the likelihood to be profiled prior to truncation. For all of these models, we use R simulations to assess the finite-sample accuracy of the approximate estimators and to illustrate the distinct features of each model.
The remainder of the paper develops the general theory in § 2 , presents the worked examples in § 3 and § 4 , and concludes with a discussion in § 5 ; technical details are deferred to the appendices.
Theory and Practical Guidance
Setup
For each n ≥ 1 n \ge1 n ≥ 1 X n : = ( X 1 , … , X n ) X_n := (X_1,\ldots,X_n) X n := ( X 1 , … , X n ) { P n , θ : θ ∈ Θ ⊆ R d } n ≥ 1 \{\mathbb{P}_{n,\theta}:\theta\in\Theta\subseteq\mathbb{R}^d\}_{n\ge1} { P n , θ : θ ∈ Θ ⊆ R d } n ≥ 1 P n , θ \mathbb{P}_{n,\theta} P n , θ X n X_n X n X n = ( X 1 , … , X n ) X_n = (X_1,\ldots,X_n) X n = ( X 1 , … , X n ) P n , θ = P θ ⊗ n \mathbb{P}_{n,\theta} = \mathbb{P}_{\theta}^{\otimes n} P n , θ = P θ ⊗ n p n , θ p_{n,\theta} p n , θ log ( p n , θ ( X n ) ) \log(p_{n,\theta}(X_n)) log ( p n , θ ( X n )) ∇ log ( p n , θ ( X n ) ) \nabla\log(p_{n,\theta}(X_n)) ∇ log ( p n , θ ( X n )) F : R d → R m F:\mathbb{R}^d \to\mathbb{R}^m F : R d → R m J F ( x ) JF(x) J F ( x ) x x x A A A σ min ( A ) \sigma_{\min}(A) σ m i n ( A ) ∥ ⋅ ∥ \|\cdot\| ∥ ⋅ ∥ z ∈ R m z \in\mathbb{R}^m z ∈ R m r > 0 r > 0 r > 0 B r ( z ) : = { x ∈ R m : ∥ x − z ∥ ≤ r } B_r(z) := \{x \in\mathbb{R}^m:\|x-z\| \le r\} B r ( z ) := { x ∈ R m : ∥ x − z ∥ ≤ r } r r r z z z
Z n → P n , θ n z Z_n \xrightarrow{\mathbb{P}_{n,\theta_n}} z Z n P n , θ n z to mean that P n , θ n ( ∥ Z n − z ∥ > ϵ ) → 0 \mathbb{P}_{n,\theta_n}(\|Z_n-z\|>\epsilon)\to0 P n , θ n ( ∥ Z n − z ∥ > ϵ ) → 0 ϵ > 0 \epsilon>0 ϵ > 0 Z n → p z Z_n \xrightarrow{p} z Z n p z { θ n } n ≥ 1 \{\theta_n\}_{n\ge1} { θ n } n ≥ 1 r n > 0 r_n>0 r n > 0 Z n = O P n , θ n ( r n ) Z_n=O_{\mathbb{P}_{n,\theta_n}}(r_n) Z n = O P n , θ n ( r n ) Z n / r n → 0 Z_n/r_n\to0 Z n / r n → 0 { P n , θ n } n ≥ 1 \{\mathbb{P}_{n,\theta_n}\}_{n\ge1} { P n , θ n } n ≥ 1 Z n = O P n , θ n ( r n ) Z_n=O_{\mathbb{P}_{n,\theta_n}}(r_n) Z n = O P n , θ n ( r n ) Z n / r n Z_n/r_n Z n / r n O P ( r n ) O_P(r_n) O P ( r n ) O P ( r n ) O_P(r_n) O P ( r n )
Our framework involves parameter sequences that may diverge or approach the boundary of the parameter space. We formalize this behavior through a limiting regime and then map it to zero using a vanishing chart:
Definition 1. A limiting regime ( R , θ ∗ ) (\mathcal{R},\theta^*) ( R , θ ∗ ) R \mathcal{R} R { θ n } n ≥ 1 ⊆ Θ \{\theta_n\}_{n\ge1} \subseteq\Theta { θ n } n ≥ 1 ⊆ Θ limiting value θ ∗ \theta^* θ ∗ θ n → θ ∗ \theta_n \to\theta^* θ n → θ ∗ { θ n } n ≥ 1 ∈ R \{\theta_n\}_{n\ge1} \in\mathcal{R} { θ n } n ≥ 1 ∈ R R \mathcal{R} R Θ ‾ \overline{\Theta} Θ
Definition 2. A vanishing chart for a limiting regime R \mathcal{R} R ϕ : N → U \phi:N \to U ϕ : N → U U : = ϕ ( N ) ⊆ R d U:=\phi(N)\subseteq\mathbb{R}^d U := ϕ ( N ) ⊆ R d N ⊆ Θ N\subseteq\Theta N ⊆ Θ { θ n } n ≥ 1 ∈ R \{\theta_n\}_{n\ge1}\in\mathcal{R} { θ n } n ≥ 1 ∈ R θ n ∈ N \theta_n\in N θ n ∈ N ϕ ( θ n ) → 0 \phi(\theta_n)\to0 ϕ ( θ n ) → 0
write ϕ − 1 : U → N \phi^{-1}: U \to N ϕ − 1 : U → N N N N U U U ϕ \phi ϕ ϕ − 1 \phi^{-1} ϕ − 1
An estimator may be naturally defined only on an event E n E_n E n { θ n } n ≥ 1 ∈ R \{\theta_n\}_{n\geq1} \in\mathcal{R} { θ n } n ≥ 1 ∈ R Θ \Theta Θ Θ \Theta Θ E n c E_n^c E n c exceptional event ); this value is chosen in advance and does not depend on the unknown parameter or the regime sequence. Similarly, if an estimator falls outside the domain N N N ϕ → U \phi\to U ϕ → U ϕ \phi ϕ U U U
Definition 3. Along a fixed regime sequence { θ n } n ≥ 1 ∈ R \{\theta_n\}_{n\geq1} \in\mathcal{R} { θ n } n ≥ 1 ∈ R { Z n } n ≥ 1 \{Z_n\}_{n\geq1} { Z n } n ≥ 1 localized in deterministic sets A n A_n A n P n , θ n ( Z n ∈ A n ) ⟶ 1 \mathbb{P}_{n,\theta_n}(Z_n \in A_n) \longrightarrow1 P n , θ n ( Z n ∈ A n ) ⟶ 1 sup z ∈ A n ∥ z − z 0 ∥ ⟶ 0 \sup_{z\in A_n}\|z-z_0\| \longrightarrow0 sup z ∈ A n ∥ z − z 0 ∥ ⟶ 0 Z n Z_n Z n localized at z 0 z_0 z 0
Definition 4. Let ϕ : N → U \phi:N\to U ϕ : N → U R \mathcal{R} R { θ ^ n } n ≥ 1 \{\widehat{\theta}_n\}_{n\geq1} { θ n } n ≥ 1 { θ ~ n } n ≥ 1 \{\widetilde{\theta}_n\}_{n\geq1} { θ n } n ≥ 1 Θ \Theta Θ { θ n } n ≥ 1 ∈ R \{\theta_n\}_{n\geq1}\in\mathcal{R} { θ n } n ≥ 1 ∈ R ρ n ≥ 1 \rho_n\geq1 ρ n ≥ 1 equivalent at scale ρ n \rho_n ρ n ϕ \phi ϕ { θ n } n ≥ 1 \{\theta_n\}_{n\geq1} { θ n } n ≥ 1
P n , θ n ( θ ^ n , θ ~ n ∈ N ) ⟶ 1 and ϕ ( θ ^ n ) = o P n , θ n ( 1 ) , \mathbb{P}_{n,\theta_n}\left(\widehat{\theta}_n,\widetilde{\theta}_n\in N\right)\longrightarrow1 \quad\text{and}\quad\phi(\widehat{\theta}_n)=o_{\mathbb{P}_{n,\theta_n}}(1), P n , θ n ( θ n , θ n ∈ N ) ⟶ 1 and ϕ ( θ n ) = o P n , θ n ( 1 ) , and
ρ n ∥ ϕ ( θ ~ n ) − ϕ ( θ ^ n ) ∥ = o P n , θ n ( 1 ) . \rho_n\left\|\phi(\widetilde{\theta}_n)-\phi(\widehat{\theta}_n)\right\|=o_{\mathbb{P}_{n,\theta_n}}(1). ρ n ϕ ( θ n ) − ϕ ( θ n ) = o P n , θ n ( 1 ) . These conditions imply ϕ ( θ ~ n ) = o P n , θ n ( 1 ) \phi(\widetilde{\theta}_n)=o_{\mathbb{P}_{n,\theta_n}}(1) ϕ ( θ n ) = o P n , θ n ( 1 ) ρ n ′ > 0 \rho'_n>0 ρ n ′ > 0 equivalent at scale ρ n ′ \rho'_n ρ n ′ { θ n } n ≥ 1 \{\theta_n\}_{n\geq1} { θ n } n ≥ 1
ρ n ′ ∥ θ ~ n − θ ^ n ∥ = o P n , θ n ( 1 ) . \rho'_n\left\|\widetilde{\theta}_n-\widehat{\theta}_n\right\|=o_{\mathbb{P}_{n,\theta_n}}(1). ρ n ′ θ n − θ n = o P n , θ n ( 1 ) . When either scale sequence is identically one, we simply say that the estimators are equivalent on the corresponding scale. When the scale sequence tends to infinity, we say that the estimators are asymptotically equivalent under the corresponding normalization. Each property holds in the limiting regime R \mathcal{R} R R \mathcal{R} R R \mathcal{R} R θ ^ n \widehat{\theta}_n θ n θ ~ n \widetilde{\theta}_n θ n approximate maximum likelihood estimator , or approximate MLE , at that scale.
Chart equivalence at scale one records simultaneous localization of the two roots near the limiting chart value, but need not quantify surrogate accuracy beyond that localization. However, if
R n ( ϕ ( θ ^ n ) − ϕ ( θ n ) ) → d Z and R n ( ϕ ( θ ~ n ) − ϕ ( θ ^ n ) ) = o P n , θ n ( 1 ) , R_n\left(\phi(\widehat{\theta}_n)-\phi(\theta_n)\right)\xrightarrow{d} Z \quad\text{and}\quad R_n\left(\phi(\widetilde{\theta}_n)-\phi(\widehat{\theta}_n)\right)=o_{\mathbb{P}_{n,\theta_n}}(1), R n ( ϕ ( θ n ) − ϕ ( θ n ) ) d Z and R n ( ϕ ( θ n ) − ϕ ( θ n ) ) = o P n , θ n ( 1 ) , then by Slutsky’s theorem, the corresponding normalized error of θ ~ n \widetilde{\theta}_n θ n Z Z Z
One may naturally ask whether the asymptotic equivalence in Definition 4 depends on the specific choice of vanishing chart. Our first proposition shows that both asymptotic equivalence and the corresponding normalization are preserved under a smooth mapping whose Jacobian at zero is nonsingular:
Proposition 1. Let ϕ : N 1 → U 1 \phi:N_1\to U_1 ϕ : N 1 → U 1 ψ : N 2 → U 2 \psi:N_2\to U_2 ψ : N 2 → U 2 R \mathcal{R} R T : = ψ ∘ ϕ − 1 T:=\psi\circ\phi^{-1} T := ψ ∘ ϕ − 1 ϕ ( N 1 ∩ N 2 ) \phi(N_1\cap N_2) ϕ ( N 1 ∩ N 2 ) C 1 C^1 C 1 T ~ \widetilde{T} T 0 0 0 T ~ ( 0 ) = 0 \widetilde{T}(0)=0 T ( 0 ) = 0 J T ~ ( 0 ) J_{\widetilde{T}}(0) J T ( 0 ) { θ n } n ≥ 1 ∈ R \{\theta_n\}_{n\geq1}\in\mathcal{R} { θ n } n ≥ 1 ∈ R { θ ^ n ( 1 ) } n ≥ 1 \{\widehat{\theta}^{(1)}_n\}_{n\geq1} { θ n ( 1 ) } n ≥ 1 { θ ^ n ( 2 ) } n ≥ 1 \{\widehat{\theta}^{(2)}_n\}_{n\geq1} { θ n ( 2 ) } n ≥ 1
P n , θ n ( θ ^ n ( 1 ) , θ ^ n ( 2 ) ∈ N 1 ∩ N 2 ) ⟶ 1 , \mathbb{P}_{n,\theta_n}\left(\widehat{\theta}^{(1)}_n,\widehat{\theta}^{(2)}_n\in N_1\cap N_2\right)\longrightarrow1, P n , θ n ( θ n ( 1 ) , θ n ( 2 ) ∈ N 1 ∩ N 2 ) ⟶ 1 , and ϕ ( θ ^ n ( i ) ) = o P n , θ n ( 1 ) \phi(\hat{\theta}_n^{(i)}) = o_{\mathbb{P}_{n,\theta_n}}(1) ϕ ( θ ^ n ( i ) ) = o P n , θ n ( 1 ) i = 1 , 2 i = 1, 2 i = 1 , 2 ψ ( θ ^ n ( i ) ) = o P n , θ n ( 1 ) \psi(\hat{\theta}_n^{(i)}) = o_{\mathbb{P}_{n,\theta_n}}(1) ψ ( θ ^ n ( i ) ) = o P n , θ n ( 1 ) i = 1 , 2 i = 1, 2 i = 1 , 2 ρ n ≥ 1 \rho_n \ge1 ρ n ≥ 1
ρ n ∥ ϕ ( θ ^ n ( 1 ) ) − ϕ ( θ ^ n ( 2 ) ) ∥ = o P n , θ n ( 1 ) ⟺ ρ n ∥ ψ ( θ ^ n ( 1 ) ) − ψ ( θ ^ n ( 2 ) ) ∥ = o P n , θ n ( 1 ) . (5) \rho_n \left\|\phi(\hat{\theta}_n^{(1)}) - \phi(\hat{\theta}_n^{(2)})\right\| = o_{\mathbb{P}_{n,\theta_n}}(1) \quad\Longleftrightarrow\quad\rho_n \left\|\psi(\hat{\theta}_n^{(1)}) - \psi(\hat{\theta}_n^{(2)})\right\| = o_{\mathbb{P}_{n,\theta_n}}(1). \tag*{(5)} ρ n ϕ ( θ ^ n ( 1 ) ) − ϕ ( θ ^ n ( 2 ) ) = o P n , θ n ( 1 ) ⟺ ρ n ψ ( θ ^ n ( 1 ) ) − ψ ( θ ^ n ( 2 ) ) = o P n , θ n ( 1 ) . (5) Proof. Fix an arbitrary { θ n } n ≥ 1 ∈ R \{\theta_n\}_{n\ge1} \in\mathcal{R} { θ n } n ≥ 1 ∈ R { P n , θ n } n ≥ 1 \{\mathbb{P}_{n,\theta_n}\}_{n\ge1} { P n , θ n } n ≥ 1 T ~ : W → R d \widetilde{T}: W \to\mathbb{R}^d T : W → R d C 1 C^1 C 1 T = ψ ∘ ϕ − 1 T = \psi\circ\phi^{-1} T = ψ ∘ ϕ − 1 W W W 0 0 0 T ~ \widetilde{T} T 0 0 0 r > 0 r > 0 r > 0 B ‾ r ( 0 ) ⊆ W \overline{B}_r(0) \subseteq W B r ( 0 ) ⊆ W
sup u ∈ B ‾ r ( 0 ) ∥ J T ~ ( u ) − J T ~ ( 0 ) ∥ ≤ σ min ( J T ~ ( 0 ) ) 2 . \sup_{u\in\overline{B}_r(0)} \left\|J_{\widetilde{T}}(u) - J_{\widetilde{T}}(0)\right\| \le\frac{\sigma_{\min}(J_{\widetilde{T}}(0))}{2}. u ∈ B r ( 0 ) sup J T ( u ) − J T ( 0 ) ≤ 2 σ m i n ( J T ( 0 )) . Because B ‾ r ( 0 ) \overline{B}_r(0) B r ( 0 ) u u u v v v B ‾ r ( 0 ) \overline{B}_r(0) B r ( 0 ) t ↦ T ~ ( u + t ( v − u ) ) t \mapsto\widetilde{T}(u+t(v-u)) t ↦ T ( u + t ( v − u )) u , v ∈ B ‾ r ( 0 ) u,v \in\overline{B}_r(0) u , v ∈ B r ( 0 )
T ~ ( v ) − T ~ ( u ) = ∫ 0 1 J T ~ ( u + t ( v − u ) ) ( v − u ) d t \widetilde{T}(v)-\widetilde{T}(u)=\int_0^1 J_{\widetilde{T}}(u+t(v-u))(v-u)\,\mathrm{d}t T ( v ) − T ( u ) = ∫ 0 1 J T ( u + t ( v − u )) ( v − u ) d t [e.g. 31 Theorem 3.2.7] . Consequently,
σ min ( J T ~ ( 0 ) ) 2 ∥ v − u ∥ ≤ ∥ T ~ ( v ) − T ~ ( u ) ∥ ≤ ( ∥ J T ~ ( 0 ) ∥ + σ min ( J T ~ ( 0 ) ) 2 ) ∥ v − u ∥ . (6) \frac{\sigma_{\min}(J_{\widetilde{T}}(0))}{2}\|v-u\| \le\|\widetilde{T}(v)-\widetilde{T}(u)\| \le\left(\|J_{\widetilde{T}}(0)\|+\frac{\sigma_{\min}(J_{\widetilde{T}}(0))}{2}\right)\|v-u\|. \tag*{(6)} 2 σ m i n ( J T ( 0 )) ∥ v − u ∥ ≤ ∥ T ( v ) − T ( u ) ∥ ≤ ( ∥ J T ( 0 ) ∥ + 2 σ m i n ( J T ( 0 )) ) ∥ v − u ∥. (6) where the upper bound follows directly from the integral formula and the lower bound follows from subtracting J T ~ ( 0 ) ( v − u ) J_{\widetilde{T}}(0)(v-u) J T ( 0 ) ( v − u ) ∥ J T ~ ( 0 ) ( v − u ) ∥ ≥ σ min ( J T ~ ( 0 ) ) ∥ v − u ∥ \|J_{\widetilde{T}}(0)(v-u)\| \ge\sigma_{\min}(J_{\widetilde{T}}(0))\|v-u\| ∥ J T ( 0 ) ( v − u ) ∥ ≥ σ m i n ( J T ( 0 )) ∥ v − u ∥
For i = 1 , 2 i = 1, 2 i = 1 , 2 U i , n : = ϕ ( θ ^ n ( i ) ) U_{i,n} := \phi(\hat{\theta}_n^{(i)}) U i , n := ϕ ( θ ^ n ( i ) ) N 1 N_1 N 1
E n : = { θ ^ n ( 1 ) , θ ^ n ( 2 ) ∈ N 1 ∩ N 2 } ∩ { ϕ ( θ ^ n ( 1 ) ) , ϕ ( θ ^ n ( 2 ) ) ∈ B r ( 0 ) } . E_n := \{\hat{\theta}_n^{(1)},\hat{\theta}_n^{(2)} \in N_1 \cap N_2\} \cap\{\phi(\hat{\theta}_n^{(1)}),\phi(\hat{\theta}_n^{(2)}) \in B_r(0)\}. E n := { θ ^ n ( 1 ) , θ ^ n ( 2 ) ∈ N 1 ∩ N 2 } ∩ { ϕ ( θ ^ n ( 1 ) ) , ϕ ( θ ^ n ( 2 ) ) ∈ B r ( 0 )} . Our assumptions, together with ϕ ( θ ^ n ( 1 ) ) , ϕ ( θ ^ n ( 2 ) ) = o P n , θ n ( 1 ) \phi(\hat{\theta}_n^{(1)}),\phi(\hat{\theta}_n^{(2)}) = o_{\mathbb{P}_{n,\theta_n}}(1) ϕ ( θ ^ n ( 1 ) ) , ϕ ( θ ^ n ( 2 ) ) = o P n , θ n ( 1 ) P n , θ n ( E n ) → 1 \mathbb{P}_{n,\theta_n}(E_n) \to1 P n , θ n ( E n ) → 1 E n E_n E n
ψ ( θ ^ n ( 1 ) ) = T ~ ( ϕ ( θ ^ n ( 1 ) ) ) and ψ ( θ ^ n ( 2 ) ) = T ~ ( ϕ ( θ ^ n ( 2 ) ) ) . \psi(\hat{\theta}_n^{(1)})=\widetilde{T}(\phi(\hat{\theta}_n^{(1)})) \quad\text{and}\quad\psi(\hat{\theta}_n^{(2)})=\widetilde{T}(\phi(\hat{\theta}_n^{(2)})). ψ ( θ ^ n ( 1 ) ) = T ( ϕ ( θ ^ n ( 1 ) )) and ψ ( θ ^ n ( 2 ) ) = T ( ϕ ( θ ^ n ( 2 ) )) . Since T ~ ( 0 ) = 0 \widetilde{T}(0)=0 T ( 0 ) = 0 (6) , applied with u = 0 u=0 u = 0 ψ ( θ ^ n ( i ) ) = o P n , θ n ( 1 ) \psi(\hat{\theta}_n^{(i)})=o_{\mathbb{P}_{n,\theta_n}}(1) ψ ( θ ^ n ( i ) ) = o P n , θ n ( 1 ) (6) to ϕ ( θ ^ n ( 1 ) ) \phi(\hat{\theta}_n^{(1)}) ϕ ( θ ^ n ( 1 ) ) ϕ ( θ ^ n ( 2 ) ) \phi(\hat{\theta}_n^{(2)}) ϕ ( θ ^ n ( 2 ) ) E n E_n E n ρ n ≥ 1 \rho_n \ge1 ρ n ≥ 1
σ min ( J T ~ ( 0 ) ) 2 ρ n ∥ ϕ ( θ ^ n ( 1 ) ) − ϕ ( θ ^ n ( 2 ) ) ∥ ≤ ρ n ∥ ψ ( θ ^ n ( 1 ) ) − ψ ( θ ^ n ( 2 ) ) ∥ ≤ ( ∥ J T ~ ( 0 ) ∥ + σ min ( J T ~ ( 0 ) ) 2 ) ρ n ∥ ϕ ( θ ^ n ( 1 ) ) − ϕ ( θ ^ n ( 2 ) ) ∥ . \frac{\sigma_{\min}(J_{\widetilde{T}}(0))}{2}\rho_n\left\|\phi(\hat{\theta}_n^{(1)})-\phi(\hat{\theta}_n^{(2)})\right\|
\le\rho_n\left\|\psi(\hat{\theta}_n^{(1)})-\psi(\hat{\theta}_n^{(2)})\right\|
\le\left(\|J_{\widetilde{T}}(0)\|+\frac{\sigma_{\min}(J_{\widetilde{T}}(0))}{2}\right)\rho_n\left\|\phi(\hat{\theta}_n^{(1)})-\phi(\hat{\theta}_n^{(2)})\right\|. 2 σ m i n ( J T ( 0 )) ρ n ϕ ( θ ^ n ( 1 ) ) − ϕ ( θ ^ n ( 2 ) ) ≤ ρ n ψ ( θ ^ n ( 1 ) ) − ψ ( θ ^ n ( 2 ) ) ≤ ( ∥ J T ( 0 ) ∥ + 2 σ m i n ( J T ( 0 )) ) ρ n ϕ ( θ ^ n ( 1 ) ) − ϕ ( θ ^ n ( 2 ) ) . Since P n , θ n ( E n c ) → 0 \mathbb{P}_{n,\theta_n}(E_n^c)\to0 P n , θ n ( E n c ) → 0 (5) . Since the choice of regime sequence was arbitrary, the conclusion holds throughout R \mathcal{R} R
Uniform Root Stability
We now develop results that establish the existence and localization of an exact root of the estimating equation, and bound its distance from the surrogate root. To this end, fix an arbitrary regime sequence { θ n } n ≥ 1 ∈ R \{\theta_n\}_{n\ge1}\in\mathcal{R} { θ n } n ≥ 1 ∈ R P n , θ n \mathbb{P}_{n,\theta_n} P n , θ n ϕ : N → U \phi:N\to U ϕ : N → U q n , q ~ n : Θ → R d q_n,\widetilde{q}_n:\Theta\to\mathbb{R}^d q n , q n : Θ → R d u ∈ U u\in U u ∈ U
Q n ( u ) : = q n ( ϕ − 1 ( u ) ) and Q ~ n ( u ) : = q ~ n ( ϕ − 1 ( u ) ) . Q_n(u):=q_n(\phi^{-1}(u)) \quad\text{and}\quad\widetilde{Q}_n(u):=\widetilde{q}_n(\phi^{-1}(u)). Q n ( u ) := q n ( ϕ − 1 ( u )) and Q n ( u ) := q n ( ϕ − 1 ( u )) . The corresponding roots in these coordinates are
u ^ n : = ϕ ( θ ^ n ) and u ~ n : = ϕ ( θ ~ n ) , \hat{u}_n := \phi(\hat{\theta}_n) \quad\text{and}\quad\tilde{u}_n := \phi(\tilde{\theta}_n), u ^ n := ϕ ( θ ^ n ) and u ~ n := ϕ ( θ ~ n ) , using the convention for exceptional events from § 2.1 .
Our first theorem is useful when only the surrogate root has been localized. It both produces a nearby exact root and bounds every exact root in the same neighborhood:
Theorem 1. Let U ⊆ R d U \subseteq\mathbb{R}^{d} U ⊆ R d Q : U → R d \mathbf{Q}: U \to\mathbb{R}^{d} Q : U → R d Q ~ ( u ~ ) = 0 \widetilde{\mathbf{Q}}(\widetilde{\mathbf{u}}) = 0 Q ( u ) = 0 Q ~ : U → R d \widetilde{\mathbf{Q}}: U \to\mathbb{R}^{d} Q : U → R d u ~ ∈ U \widetilde{\mathbf{u}} \in U u ∈ U R , J ∈ R d × d \mathbf{R}, \mathbf{J} \in\mathbb{R}^{d \times d} R , J ∈ R d × d r > 0 r > 0 r > 0
D : = { u : ∥ R ( u − u ~ ) ∥ ≤ r } ⊆ U , D := \left\{u : \left\|\mathbf{R}(u-\widetilde{u})\right\| \leq r\right\} \subseteq U, D := { u : ∥ R ( u − u ) ∥ ≤ r } ⊆ U , and suppose that Q ~ \widetilde{\mathbf{Q}} Q D D D
κ : = sup u ∈ D ∥ R J − 1 J Q ~ ( u ) R − 1 − I d ∥ and η : = sup u ∈ D ∥ R J − 1 [ Q ( u ) − Q ~ ( u ) ] ∥ . \kappa:= \sup_{u \in D}\left\|\mathbf{R}\mathbf{J}^{-1}\mathbf{J}_{\widetilde{\mathbf{Q}}}(u)\mathbf{R}^{-1}-\mathbf{I}_{d}\right\| \quad\text{and}\quad\eta:= \sup_{u \in D}\left\|\mathbf{R}\mathbf{J}^{-1}\left[\mathbf{Q}(u)-\widetilde{\mathbf{Q}}(u)\right]\right\|. κ := u ∈ D sup R J − 1 J Q ( u ) R − 1 − I d and η := u ∈ D sup R J − 1 [ Q ( u ) − Q ( u ) ] . If κ < 1 \kappa< 1 κ < 1 η < ( 1 − κ ) r \eta< (1-\kappa)r η < ( 1 − κ ) r Q \mathbf{Q} Q D D D u ^ ∈ D \widehat{u} \in D u ∈ D Q \mathbf{Q} Q
∥ R ( u ^ − u ~ ) ∥ ≤ η 1 − κ . (7) \left\|\mathbf{R}(\widehat{u}-\widetilde{u})\right\| \leq\frac{\eta}{1-\kappa}. \tag*{(7)} ∥ R ( u − u ) ∥ ≤ 1 − κ η . (7) Proof. For x ∈ B ‾ r ( 0 ) x \in\overline{B}_{r}(0) x ∈ B r ( 0 ) u ( x ) : = u ~ + R − 1 x u(x) := \widetilde{u}+\mathbf{R}^{-1}x u ( x ) := u + R − 1 x
F ( x ) : = R J − 1 Q ~ ( u ( x ) ) and G ( x ) : = R J − 1 Q ( u ( x ) ) . F(x) := \mathbf{R}\mathbf{J}^{-1}\widetilde{\mathbf{Q}}(u(x)) \quad\text{and}\quad G(x) := \mathbf{R}\mathbf{J}^{-1}\mathbf{Q}(u(x)). F ( x ) := R J − 1 Q ( u ( x )) and G ( x ) := R J − 1 Q ( u ( x )) . Since F ( 0 ) = 0 F(0) = 0 F ( 0 ) = 0
F ( x ) = ∫ 0 1 R J − 1 J Q ~ ( u ( t x ) ) R − 1 x d t . (8) F(x) = \int_{0}^{1}\mathbf{R}\mathbf{J}^{-1}\mathbf{J}_{\widetilde{\mathbf{Q}}}(u(tx))\mathbf{R}^{-1}x\,\mathrm{d}t. \tag*{(8)} F ( x ) = ∫ 0 1 R J − 1 J Q ( u ( t x )) R − 1 x d t . (8) Therefore
F ( x ) T x ≥ ( 1 − κ ) ∥ x ∥ 2 and ∥ G ( x ) − F ( x ) ∥ ≤ η . F(x)^{\mathsf{T}}x \geq(1-\kappa)\lVert x\rVert^{2} \quad\text{and}\quad\lVert G(x)-F(x)\rVert\leq\eta. F ( x ) T x ≥ ( 1 − κ ) ∥ x ∥ 2 and ∥ G ( x ) − F ( x )∥ ≤ η . On ∥ x ∥ = r \lVert x\rVert= r ∥ x ∥ = r
G ( x ) T x ≥ ( 1 − κ ) r 2 − η r > 0. G(x)^{\mathsf{T}}x \geq(1-\kappa)r^{2}-\eta r > 0. G ( x ) T x ≥ ( 1 − κ ) r 2 − η r > 0. This condition implies that G G G B ‾ r ( 0 ) \overline{B}_{r}(0) B r ( 0 ) x ↦ − r G ( x ) / ∥ G ( x ) ∥ x \mapsto-rG(x)/\lVert G(x)\rVert x ↦ − r G ( x ) / ∥ G ( x )∥ B ‾ r ( 0 ) \overline{B}_{r}(0) B r ( 0 ) B ‾ r ( 0 ) \overline{B}_{r}(0) B r ( 0 ) G ( x ) T x < 0 G(x)^{\mathsf{T}}x < 0 G ( x ) T x < 0 Q \mathbf{Q} Q D D D
If u ^ ∈ D \widehat{u} \in D u ∈ D x ^ = R ( u ^ − u ~ ) \widehat{x}=\mathbf{R}(\widehat{u}-\widetilde{u}) x = R ( u − u )
∥ F ( x ^ ) ∥ ≥ ( 1 − κ ) ∥ x ^ ∥ . \lVert F(\widehat{x})\rVert\geq(1-\kappa)\lVert\widehat{x}\rVert. ∥ F ( x )∥ ≥ ( 1 − κ ) ∥ x ∥ . Since G ( x ^ ) = 0 G(\widehat{x})=0 G ( x ) = 0 ∥ F ( x ^ ) − G ( x ^ ) ∥ ≤ η \lVert F(\widehat{x})-G(\widehat{x})\rVert\leq\eta ∥ F ( x ) − G ( x )∥ ≤ η
In statistical applications, the estimating equations are random, so the conditions of Theorem 1 must be verified on events whose probabilities tend to one. The following corollary casts this verification into stochastic localization and an error bound.
Corollary 1. For each n n n Q n , Q ~ n , u ~ n , R n , J n , r n \mathbf{Q}_n,\widetilde{\mathbf{Q}}_n,\widetilde{u}_n,\mathbf{R}_n,\mathbf{J}_n,r_n Q n , Q n , u n , R n , J n , r n D n D_n D n κ n \kappa_n κ n η n \eta_n η n κ n = o P ( 1 ) \kappa_n=o_{\mathrm{P}}(1) κ n = o P ( 1 ) η n / r n = o P ( 1 ) \eta_n/r_n=o_{\mathrm{P}}(1) η n / r n = o P ( 1 ) Q n \mathbf{Q}_n Q n D n D_n D n u ∈ D n u \in D_n u ∈ D n
∥ R n ( u − u ~ n ) ∥ ≤ η n 1 − κ n . (9) \left\|\mathbf{R}_n(u-\widetilde{u}_n)\right\| \leq\frac{\eta_n}{1-\kappa_n}. \tag*{(9)} ∥ R n ( u − u n ) ∥ ≤ 1 − κ n η n . (9) Let u ^ n l o c \widehat{u}_n^{\mathrm{loc}} u n loc D n D_n D n § 2.1 . For any deterministic s n > 0 s_n>0 s n > 0 η n = O P ( s n ) \eta_n=O_{\mathrm{P}}(s_n) η n = O P ( s n ) R n ( u ^ n l o c − u ~ n ) = O P ( s n ) \mathbf{R}_n(\widehat{u}_n^{\mathrm{loc}}-\widetilde{u}_n)=O_{\mathrm{P}}(s_n) R n ( u n loc − u n ) = O P ( s n ) η n = o P ( s n ) \eta_n=o_{\mathrm{P}}(s_n) η n = o P ( s n ) R n ( u ^ n l o c − u ~ n ) = o P ( s n ) \mathbf{R}_n(\widehat{u}_n^{\mathrm{loc}}-\widetilde{u}_n)=o_{\mathrm{P}}(s_n) R n ( u n loc − u n ) = o P ( s n ) Q n \mathbf{Q}_n Q n u ^ n \widehat{u}_n u n u ^ n l o c = u ^ n \widehat{u}_n^{\mathrm{loc}}=\widehat{u}_n u n loc = u n
Proof. Let A n A_n A n P n , θ n ( A n ) → 1 \mathbb{P}_{n,\theta_n}(A_n) \to1 P n , θ n ( A n ) → 1
B n : = A n ∩ { κ n < 1 2 , η n < r n 2 } . B_n := A_n \cap\left\{\kappa_n < \frac{1}{2},\ \eta_n < \frac{r_n}{2}\right\}. B n := A n ∩ { κ n < 2 1 , η n < 2 r n } . The assumptions κ n = o P ( 1 ) \kappa_n = o_{\mathbb{P}}(1) κ n = o P ( 1 ) η n / r n = o P ( 1 ) \eta_n/r_n = o_{\mathbb{P}}(1) η n / r n = o P ( 1 ) P n , θ n ( B n ) → 1 \mathbb{P}_{n,\theta_n}(B_n) \to1 P n , θ n ( B n ) → 1 B n B_n B n D n D_n D n (9) for every zero in D n D_n D n 1 / ( 1 − κ n ) = 1 + o P ( 1 ) 1/(1-\kappa_n) = 1 + o_{\mathbb{P}}(1) 1/ ( 1 − κ n ) = 1 + o P ( 1 ) u ^ n l o c \hat{u}_n^{\mathrm{loc}} u ^ n loc u ^ n \hat{u}_n u ^ n
An error in an estimating equation translates into an error in its root only after we account for the sensitivity of that root to perturbations. We make that comparison in coordinates of a vanishing chart, where our main result, Theorem 2 below, bounds the difference between the exact and surrogate roots by the uniform equation error divided by a scale set by the smallest singular value:
Theorem 2. Suppose that there are deterministic convex sets D n ⊆ U D_n \subseteq U D n ⊆ U E n E_n E n P n , θ n ( E n ) → 1 \mathbb{P}_{n,\theta_n}(E_n) \to1 P n , θ n ( E n ) → 1 E n E_n E n
θ ^ n , θ ~ n ∈ N and q n ( θ ^ n ) = 0 and q ~ n ( θ ~ n ) = 0 and u ^ n , u ~ n ∈ D n , (10) \hat{\theta}_n,\widetilde{\theta}_n \in N \quad\text{and}\quad q_n(\hat{\theta}_n)=0 \quad\text{and}\quad\widetilde{q}_n(\widetilde{\theta}_n)=0 \quad\text{and}\quad\hat{u}_n,\widetilde{u}_n \in D_n, \tag*{(10)} θ ^ n , θ n ∈ N and q n ( θ ^ n ) = 0 and q n ( θ n ) = 0 and u ^ n , u n ∈ D n , (10) and Q ~ n ∣ D n \widetilde{Q}_n|_{D_n} Q n ∣ D n C 1 C^1 C 1 D n D_n D n J Q ~ n J_{\widetilde{Q}_n} J Q n J n ∈ R d × d J_n \in\mathbb{R}^{d \times d} J n ∈ R d × d a n > 0 a_n > 0 a n > 0
P n , θ n ( σ min ( J n ) ≥ a n ) ⟶ 1 and sup u ∈ D n ∥ J Q ~ n ( u ) − J n ∥ = o P n , θ n ( a n ) on E n . \mathbb{P}_{n,\theta_n}\left(\sigma_{\min}(J_n) \geq a_n\right) \longrightarrow1
\quad\text{and}\quad
\sup_{u\in D_n}\left\|J_{\widetilde{Q}_n}(u)-J_n\right\| = o_{\mathbb{P}_{n,\theta_n}}(a_n)\ \text{on }E_n. P n , θ n ( σ m i n ( J n ) ≥ a n ) ⟶ 1 and u ∈ D n sup J Q n ( u ) − J n = o P n , θ n ( a n ) on E n . Define
Δ n : = sup u ∈ D n ∥ Q ~ n ( u ) − Q n ( u ) ∥ . \Delta_n := \sup_{u\in D_n}\left\|\widetilde{Q}_n(u)-Q_n(u)\right\|. Δ n := u ∈ D n sup Q n ( u ) − Q n ( u ) . Then
P n , θ n ( ∥ u ~ n − u ^ n ∥ ≤ 2 Δ n a n ) ⟶ 1. (11) \mathbb{P}_{n,\theta_n}\left(\|\widetilde{u}_n-\hat{u}_n\| \leq\frac{2\Delta_n}{a_n}\right) \longrightarrow1. \tag*{(11)} P n , θ n ( ∥ u n − u ^ n ∥ ≤ a n 2 Δ n ) ⟶ 1. (11) Proof. Let
H n : = E n ∩ { σ min ( J n ) ≥ a n } ∩ { sup u ∈ D n ∥ J Q ~ n ( u ) − J n ∥ ≤ a n 2 } . H_n := E_n \cap\{\sigma_{\min}(J_n) \geq a_n\} \cap\left\{\sup_{u\in D_n}\left\|J_{\widetilde{Q}_n}(u)-J_n\right\| \leq\frac{a_n}{2}\right\}. H n := E n ∩ { σ m i n ( J n ) ≥ a n } ∩ { u ∈ D n sup J Q n ( u ) − J n ≤ 2 a n } . Our assumptions imply P n , θ n ( H n ) → 1 \mathbb{P}_{n,\theta_n}(H_n) \to1 P n , θ n ( H n ) → 1 H n H_n H n { u ^ n + t ( u ~ n − u ^ n ) : t ∈ [ 0 , 1 ] } \{\hat{u}_n+t(\widetilde{u}_n-\hat{u}_n):t\in[0,1]\} { u ^ n + t ( u n − u ^ n ) : t ∈ [ 0 , 1 ]} D n D_n D n (10) gives Q ~ n ( u ~ n ) = 0 \widetilde{Q}_n(\widetilde{u}_n)=0 Q n ( u n ) = 0 Q ~ n ( u ~ n ) = 0 \widetilde{Q}_n(\widetilde{u}_n)=0 Q n ( u n ) = 0 Q ~ n ∣ D n \widetilde{Q}_n|_{D_n} Q n ∣ D n
A n ( u ~ n − u ^ n ) = − ( Q ~ n ( u ^ n ) − Q n ( u ^ n ) ) , where A n : = ∫ 0 1 J Q ~ n ( u ^ n + t ( u ~ n − u ^ n ) ) d t . (12) A_n(\widetilde{u}_n-\hat{u}_n)=-\left(\widetilde{Q}_n(\hat{u}_n)-Q_n(\hat{u}_n)\right),
\quad\text{where}\quad
A_n:=\int_0^1 J_{\widetilde{Q}_n}\left(\hat{u}_n+t(\widetilde{u}_n-\hat{u}_n)\right)\,\mathrm{d}t. \tag*{(12)} A n ( u n − u ^ n ) = − ( Q n ( u ^ n ) − Q n ( u ^ n ) ) , where A n := ∫ 0 1 J Q n ( u ^ n + t ( u n − u ^ n ) ) d t . (12) We also have that
∥ A n − J n ∥ ≤ sup u ∈ D n ∥ J Q ~ n ( u ) − J n ∥ ≤ a n 2 . \|A_n-J_n\|\leq\sup_{u\in D_n}\left\|J_{\widetilde{Q}_n}(u)-J_n\right\|\leq\frac{a_n}{2}. ∥ A n − J n ∥ ≤ u ∈ D n sup J Q n ( u ) − J n ≤ 2 a n . Weyl’s inequality now yields σ min ( A n ) ≥ σ min ( J n ) − ∥ A n − J n ∥ ≥ a n / 2 \sigma_{\min}(A_n)\geq\sigma_{\min}(J_n)-\|A_n-J_n\|\geq a_n/2 σ m i n ( A n ) ≥ σ m i n ( J n ) − ∥ A n − J n ∥ ≥ a n /2 A n A_n A n H n H_n H n ∥ A n − 1 ∥ ≤ 2 / a n \|A_n^{-1}\|\leq2/a_n ∥ A n − 1 ∥ ≤ 2/ a n
∥ u ~ n − u ^ n ∥ ≤ ∥ A n − 1 ∥ ∥ Q ~ n ( u ^ n ) − Q n ( u ^ n ) ∥ ≤ 2 Δ n a n . \|\widetilde{u}_n-\hat{u}_n\|
\leq\|A_n^{-1}\|\left\|\widetilde{Q}_n(\hat{u}_n)-Q_n(\hat{u}_n)\right\|
\leq\frac{2\Delta_n}{a_n}. ∥ u n − u ^ n ∥ ≤ ∥ A n − 1 ∥ Q n ( u ^ n ) − Q n ( u ^ n ) ≤ a n 2 Δ n . Since this bound holds on H n H_n H n P n , θ n ( H n ) → 1 \mathbb{P}_{n,\theta_n}(H_n)\to1 P n , θ n ( H n ) → 1 (11) follows.
To connect the bound in Theorem 2 with Definition 4 , we must localize the transformed roots near zero and compare the root error Δ n / a n \Delta_n/a_n Δ n / a n
Corollary 2. Under the assumptions of Theorem 2, define
r n : = sup u ∈ D n ∥ u ∥ . r_n := \sup_{u \in D_n} \lVert u \rVert. r n := u ∈ D n sup ∥ u ∥ . If r n → 0 r_n \to0 r n → 0
u ^ n = o P n , θ n ( 1 ) and u ~ n = o P n , θ n ( 1 ) and ∥ u ~ n − u ^ n ∥ = o P n , θ n ( 1 ) . (13) \widehat{u}_n = o_{\mathbb{P}_n,\theta_n}(1) \quad\text{and} \quad\widetilde{u}_n = o_{\mathbb{P}_n,\theta_n}(1) \quad\text{and} \quad\lVert\widetilde{u}_n - \widehat{u}_n \rVert= o_{\mathbb{P}_n,\theta_n}(1). \tag*{(13)} u n = o P n , θ n ( 1 ) and u n = o P n , θ n ( 1 ) and ∥ u n − u n ∥ = o P n , θ n ( 1 ) . (13) Therefore, the roots are equivalent at scale one relative to ϕ \phi ϕ { θ n } n ≥ 1 \{\theta_n\}_{n \ge1} { θ n } n ≥ 1
If, in addition, there exists a deterministic sequence ρ n ≥ 1 \rho_n \ge1 ρ n ≥ 1
ρ n Δ n a n = o P n , θ n ( 1 ) , (14) \frac{\rho_n \Delta_n}{a_n} = o_{\mathbb{P}_n,\theta_n}(1), \tag*{(14)} a n ρ n Δ n = o P n , θ n ( 1 ) , (14) then
ρ n ∥ u ~ n − u ^ n ∥ = o P n , θ n ( 1 ) . \rho_n \lVert\widetilde{u}_n - \widehat{u}_n \rVert= o_{\mathbb{P}_n,\theta_n}(1). ρ n ∥ u n − u n ∥ = o P n , θ n ( 1 ) . Thus the estimators are equivalent at scale ρ n \rho_n ρ n ϕ \phi ϕ { θ n } n ≥ 1 \{\theta_n\}_{n \ge1} { θ n } n ≥ 1 ρ n → ∞ \rho_n \to\infty ρ n → ∞ ρ n \rho_n ρ n r n → 0 r_n \to0 r n → 0 { θ n } n ≥ 1 ∈ R \{\theta_n\}_{n \ge1} \in\mathcal{R} { θ n } n ≥ 1 ∈ R ρ n \rho_n ρ n R \mathcal{R} R
Proof. On E n E_n E n u ^ n , u ~ n ∈ D n \widehat{u}_n,\widetilde{u}_n \in D_n u n , u n ∈ D n
∥ u ^ n ∥ ≤ r n and ∥ u ~ n ∥ ≤ r n and ∥ u ~ n − u ^ n ∥ ≤ 2 r n . \lVert\widehat{u}_n \rVert\le r_n \quad\text{and} \quad\lVert\widetilde{u}_n \rVert\le r_n \quad\text{and} \quad\lVert\widetilde{u}_n - \widehat{u}_n \rVert\le2r_n. ∥ u n ∥ ≤ r n and ∥ u n ∥ ≤ r n and ∥ u n − u n ∥ ≤ 2 r n . Because P n , θ n ( E n ) → 1 \mathbb{P}_{n,\theta_n}(E_n) \to1 P n , θ n ( E n ) → 1 r n → 0 r_n \to0 r n → 0 (13) . Moreover, E n ⊆ { θ ^ n , θ ~ n ∈ N } E_n \subseteq\{\widehat{\theta}_n,\widetilde{\theta}_n \in N\} E n ⊆ { θ n , θ n ∈ N } (13) , this proves equivalence at chart scale one.
Now, let
G n : = { ∥ u ~ n − u ^ n ∥ ≤ 2 Δ n a n } . G_n := \left\{\lVert\widetilde{u}_n - \widehat{u}_n \rVert\le\frac{2\Delta_n}{a_n}\right\}. G n := { ∥ u n − u n ∥ ≤ a n 2 Δ n } . By Theorem 2, P n , θ n ( G n ) → 1 \mathbb{P}_{n,\theta_n}(G_n) \to1 P n , θ n ( G n ) → 1 ε > 0 \varepsilon> 0 ε > 0
P n , θ n ( ρ n ∥ u ~ n − u ^ n ∥ > ε ) ≤ P n , θ n ( G n c ) + P n , θ n ( 2 ρ n Δ n a n > ε ) ⟶ 0 , \mathbb{P}_{n,\theta_n}\left(\rho_n\lVert\widetilde{u}_n - \widehat{u}_n \rVert> \varepsilon\right) \le\mathbb{P}_{n,\theta_n}(G_n^c) + \mathbb{P}_{n,\theta_n}\left(\frac{2\rho_n\Delta_n}{a_n} > \varepsilon\right) \longrightarrow0, P n , θ n ( ρ n ∥ u n − u n ∥ > ε ) ≤ P n , θ n ( G n c ) + P n , θ n ( a n 2 ρ n Δ n > ε ) ⟶ 0 , where the convergence follows from (14); the remaining conclusions follow from Definition 4. Since the regime sequence fixed at the beginning of the subsection was arbitrary, the same argument applies throughout R \mathcal{R} R ρ n \rho_n ρ n
The following matrix-scaled version of Theorem 2 controls the root discrepancy after a general normalization R n \mathbf{R}_n R n
Proposition 2. Suppose that there are deterministic convex sets D n ⊆ U D_n \subseteq U D n ⊆ U E n E_n E n P n , θ n ( E n ) → 1 \mathbb{P}_{n,\theta_n}(E_n) \to1 P n , θ n ( E n ) → 1 E n E_n E n
θ ^ n , θ ~ n ∈ N and q n ( θ ^ n ) = 0 and q ~ n ( θ ~ n ) = 0 and u ^ n , u ~ n ∈ D n , \widehat{\theta}_n,\widetilde{\theta}_n \in N \quad\text{and} \quad q_n(\widehat{\theta}_n) = 0 \quad\text{and} \quad\widetilde{q}_n(\widetilde{\theta}_n) = 0 \quad\text{and} \quad\widehat{u}_n,\widetilde{u}_n \in D_n, θ n , θ n ∈ N and q n ( θ n ) = 0 and q n ( θ n ) = 0 and u n , u n ∈ D n , and Q ~ n ∣ D n \widetilde{Q}_n|_{D_n} Q n ∣ D n C 1 C^1 C 1 D n D_n D n J Q ~ n J_{\widetilde{Q}_n} J Q n R n ∈ R d × d \mathbf{R}_n \in\mathbb{R}^{d \times d} R n ∈ R d × d J n ∈ R d × d \mathbf{J}_n \in\mathbb{R}^{d \times d} J n ∈ R d × d B n B_n B n J n \mathbf{J}_n J n
Γ n : = sup u ∈ D n ∥ R n J n − 1 ( Q ~ n ( u ) − Q n ( u ) ) ∥ . \Gamma_n := \sup_{u \in D_n} \left\lVert\mathbf{R}_n \mathbf{J}_n^{-1}\left(\widetilde{Q}_n(u) - Q_n(u)\right) \right\rVert. Γ n := u ∈ D n sup R n J n − 1 ( Q n ( u ) − Q n ( u ) ) . Extend Γ n \Gamma_n Γ n B n c B_n^c B n c E n ∩ B n E_n \cap B_n E n ∩ B n
sup u ∈ D n ∥ R n J n − 1 ( J ~ Q n ( u ) − J n ) R n − 1 ∥ = o P n , θ n ( 1 ) and Γ n = o P n , θ n ( 1 ) . (15) \sup_{u\in D_n}\left\|\mathbf{R}_n\mathbf{J}_n^{-1}\left(\widetilde{\mathbf{J}}_{Q_n}(u)-\mathbf{J}_n\right)\mathbf{R}_n^{-1}\right\|=o_{\mathbb{P}_{n,\theta_n}}(1) \quad\text{and}\quad\Gamma_n=o_{\mathbb{P}_{n,\theta_n}}(1). \tag*{(15)} u ∈ D n sup R n J n − 1 ( J Q n ( u ) − J n ) R n − 1 = o P n , θ n ( 1 ) and Γ n = o P n , θ n ( 1 ) . (15) Then
R n ( u ~ n − u ^ n ) = o P n , θ n ( 1 ) . (16) \mathbf{R}_n(\widetilde{u}_n-\widehat{u}_n)=o_{\mathbb{P}_{n,\theta_n}}(1). \tag*{(16)} R n ( u n − u n ) = o P n , θ n ( 1 ) . (16) More precisely,
P n , θ n ( ∥ R n ( u ~ n − u ^ n ) ∥ ≤ 2 Γ n ) ⟶ 1. (17) \mathbb{P}_{n,\theta_n}\left(\left\|\mathbf{R}_n(\widetilde{u}_n-\widehat{u}_n)\right\|\leq2\Gamma_n\right)\longrightarrow1. \tag*{(17)} P n , θ n ( ∥ R n ( u n − u n ) ∥ ≤ 2 Γ n ) ⟶ 1. (17) Proof. All quantities below are considered on F n : = E n ∩ B n F_n:=E_n\cap B_n F n := E n ∩ B n P n , θ n ( F n ) → 1 \mathbb{P}_{n,\theta_n}(F_n)\to1 P n , θ n ( F n ) → 1 Q n ( u ^ n ) = 0 Q_n(\widehat{u}_n)=0 Q n ( u n ) = 0 Q ~ n ( u ~ n ) = 0 \widetilde{Q}_n(\widetilde{u}_n)=0 Q n ( u n ) = 0 u ^ n \widehat{u}_n u n u ~ n \widetilde{u}_n u n D n D_n D n Q ~ n \widetilde{Q}_n Q n
A n ( u ~ n − u ^ n ) = − ( Q ~ n ( u ^ n ) − Q n ( u ^ n ) ) and A n : = ∫ 0 1 J Q ~ n ( u ^ n + t ( u ~ n − u ^ n ) ) d t . (18) \mathbf{A}_n(\widetilde{u}_n-\widehat{u}_n)=-\left(\widetilde{Q}_n(\widehat{u}_n)-Q_n(\widehat{u}_n)\right)
\quad\text{and}\quad
\mathbf{A}_n:=\int_0^1\mathbf{J}_{\widetilde{Q}_n}\left(\widehat{u}_n+t(\widetilde{u}_n-\widehat{u}_n)\right)\,dt. \tag*{(18)} A n ( u n − u n ) = − ( Q n ( u n ) − Q n ( u n ) ) and A n := ∫ 0 1 J Q n ( u n + t ( u n − u n ) ) d t . (18) Define
B n : = R n J n − 1 ( A n − J n ) R n − 1 . \mathbf{B}_n:=\mathbf{R}_n\mathbf{J}_n^{-1}(\mathbf{A}_n-\mathbf{J}_n)\mathbf{R}_n^{-1}. B n := R n J n − 1 ( A n − J n ) R n − 1 . Again using the fact that the segment joining u ^ n \widehat{u}_n u n u ~ n \widetilde{u}_n u n D n D_n D n
∥ B n ∥ ≤ ∫ 0 1 ∥ R n J n − 1 [ J Q ~ n ( u ^ n + t ( u ~ n − u ^ n ) ) − J n ] R n − 1 ∥ d t ≤ sup u ∈ D n ∥ R n J n − 1 ( J Q ~ n ( u ) − J n ) R n − 1 ∥ = o P n , θ n ( 1 ) , \begin{aligned}
\|\mathbf{B}_n\|
&\leq\int_0^1\left\|\mathbf{R}_n\mathbf{J}_n^{-1}\left[\mathbf{J}_{\widetilde{Q}_n}\left(\widehat{u}_n+t(\widetilde{u}_n-\widehat{u}_n)\right)-\mathbf{J}_n\right]\mathbf{R}_n^{-1}\right\|\,dt\\
&\leq\sup_{u\in D_n}\left\|\mathbf{R}_n\mathbf{J}_n^{-1}\left(\mathbf{J}_{\widetilde{Q}_n}(u)-\mathbf{J}_n\right)\mathbf{R}_n^{-1}\right\|\\
&=o_{\mathbb{P}_{n,\theta_n}}(1),
\end{aligned} ∥ B n ∥ ≤ ∫ 0 1 R n J n − 1 [ J Q n ( u n + t ( u n − u n ) ) − J n ] R n − 1 d t ≤ u ∈ D n sup R n J n − 1 ( J Q n ( u ) − J n ) R n − 1 = o P n , θ n ( 1 ) , so B n = o P n , θ n ( 1 ) \mathbf{B}_n=o_{\mathbb{P}_{n,\theta_n}}(1) B n = o P n , θ n ( 1 ) G n : = F n ∩ { ∥ B n ∥ < 1 / 2 } G_n:=F_n\cap\{\|\mathbf{B}_n\|<1/2\} G n := F n ∩ { ∥ B n ∥ < 1/2 }
I d + B n = R n J n − 1 A n R n − 1 , \mathbf{I}_d+\mathbf{B}_n=\mathbf{R}_n\mathbf{J}_n^{-1}\mathbf{A}_n\mathbf{R}_n^{-1}, I d + B n = R n J n − 1 A n R n − 1 , so A n \mathbf{A}_n A n G n G_n G n
( I d + B n ) − 1 = R n A n − 1 J n R n − 1 , (\mathbf{I}_d+\mathbf{B}_n)^{-1}=\mathbf{R}_n\mathbf{A}_n^{-1}\mathbf{J}_n\mathbf{R}_n^{-1}, ( I d + B n ) − 1 = R n A n − 1 J n R n − 1 , or equivalently,
R n = ( I d + B n ) − 1 R n J n − 1 A n . \mathbf{R}_n=(\mathbf{I}_d+\mathbf{B}_n)^{-1}\mathbf{R}_n\mathbf{J}_n^{-1}\mathbf{A}_n. R n = ( I d + B n ) − 1 R n J n − 1 A n . On G n G_n G n (18) gives
R n ( u ~ n − u ^ n ) = − ( I d + B n ) − 1 R n J n − 1 ( Q ~ n ( u ^ n ) − Q n ( u ^ n ) ) . \mathbf{R}_n(\widetilde{u}_n-\widehat{u}_n)=-(\mathbf{I}_d+\mathbf{B}_n)^{-1}\mathbf{R}_n\mathbf{J}_n^{-1}\left(\widetilde{Q}_n(\widehat{u}_n)-Q_n(\widehat{u}_n)\right). R n ( u n − u n ) = − ( I d + B n ) − 1 R n J n − 1 ( Q n ( u n ) − Q n ( u n ) ) . With Γ n \Gamma_n Γ n
∥ ( I d + B n ) − 1 ∥ = ∥ ∑ k = 0 ∞ ( − B n ) k ∥ ≤ 1 1 − ∥ B n ∥ ≤ 2 \left\|(\mathbf{I}_d+\mathbf{B}_n)^{-1}\right\|=\left\|\sum_{k=0}^{\infty}(-\mathbf{B}_n)^k\right\|\leq\frac{1}{1-\|\mathbf{B}_n\|}\leq2 ( I d + B n ) − 1 = k = 0 ∑ ∞ ( − B n ) k ≤ 1 − ∥ B n ∥ 1 ≤ 2 on G n G_n G n u ^ n ∈ D n \widehat{u}_n\in D_n u n ∈ D n F n F_n F n
∥ R n ( u ~ n − u ^ n ) ∥ ≤ ∥ ( I d + B n ) − 1 ∥ Γ n ≤ 2 Γ n \left\|\mathbf{R}_n(\widetilde{u}_n-\widehat{u}_n)\right\|\leq\left\|(\mathbf{I}_d+\mathbf{B}_n)^{-1}\right\|\Gamma_n\leq2\Gamma_n ∥ R n ( u n − u n ) ∥ ≤ ( I d + B n ) − 1 Γ n ≤ 2 Γ n on G n G_n G n P ( G n ) → 1 \mathbb{P}(G_n)\to1 P ( G n ) → 1 (17) . Moreover, for every ε > 0 \varepsilon>0 ε > 0
P n , θ n ( ∥ R n ( u ~ n − u ^ n ) ∥ > ε ) ≤ P n , θ n ( G n c ) + P n , θ n ( 2 Γ n > ε ) ⟶ 0 , \mathbb{P}_{n,\theta_n}\left(\left\|\mathbf{R}_n(\widetilde{u}_n-\widehat{u}_n)\right\|>\varepsilon\right)\leq\mathbb{P}_{n,\theta_n}(G_n^c)+\mathbb{P}_{n,\theta_n}(2\Gamma_n>\varepsilon)\longrightarrow0, P n , θ n ( ∥ R n ( u n − u n ) ∥ > ε ) ≤ P n , θ n ( G n c ) + P n , θ n ( 2 Γ n > ε ) ⟶ 0 , which proves (16) .
Remark. Let u n : = ϕ ( θ n ) u_n := \phi(\theta_n) u n := ϕ ( θ n ) n n n R n ( u ^ n − u n ) ⟶ d Z \mathbf{R}_n(\hat{u}_n-u_n) \overset{d}{\longrightarrow} Z R n ( u ^ n − u n ) ⟶ d Z Z Z Z R n ( u ~ n − u n ) ⟶ d Z \mathbf{R}_n(\tilde{u}_n-u_n) \overset{d}{\longrightarrow} Z R n ( u ~ n − u n ) ⟶ d Z R n ( u ~ n − u ^ n ) = o P n , θ n ( 1 ) \mathbf{R}_n(\tilde{u}_n-\hat{u}_n)=o_{\mathbb{P}_{n,\theta_n}}(1) R n ( u ~ n − u ^ n ) = o P n , θ n ( 1 ) R n : = ρ n I d \mathbf{R}_n:=\rho_n\mathbf{I}_d R n := ρ n I d § 3 and § 4 to transfer asymptotic distributions from exact to approximate estimators.
Theorem 2, Corollary 2, and Proposition 2 control the discrepancy between the roots in the coordinates of the vanishing chart. However, this control need not directly translate to the original parameter scale, since ϕ − 1 \phi^{-1} ϕ − 1 ϕ − 1 \phi^{-1} ϕ − 1
Corollary 3. Suppose that there are deterministic convex sets K n ⊆ U K_n\subseteq U K n ⊆ U F n F_n F n P n , θ n ( F n ) → 1 \mathbb{P}_{n,\theta_n}(F_n)\to1 P n , θ n ( F n ) → 1
θ ^ n , θ ~ n ∈ N and q n ( θ ^ n ) = 0 and q ~ n ( θ ~ n ) = 0 and u ^ n , u ~ n ∈ K n \hat{\theta}_n,\tilde{\theta}_n\in N \quad\text{and}\quad q_n(\hat{\theta}_n)=0 \quad\text{and}\quad\tilde{q}_n(\tilde{\theta}_n)=0 \quad\text{and}\quad\hat{u}_n,\tilde{u}_n\in K_n θ ^ n , θ ~ n ∈ N and q n ( θ ^ n ) = 0 and q ~ n ( θ ~ n ) = 0 and u ^ n , u ~ n ∈ K n on F n F_n F n Q ~ n ∣ K n \tilde{Q}_n|_{K_n} Q ~ n ∣ K n C 1 C^1 C 1 K n K_n K n J Q ~ n J_{\tilde{Q}_n} J Q ~ n J n ∈ R d × d J_n\in\mathbb{R}^{d\times d} J n ∈ R d × d a n , b n > 0 a_n,b_n>0 a n , b n > 0
P n , θ n ( σ min ( J n ) ≥ a n ) ⟶ 1 and sup u ∈ K n ∥ J Q ~ n ( u ) − J n ∥ = o P n , θ n ( a n ) \mathbb{P}_{n,\theta_n}\bigl(\sigma_{\min}(J_n)\geq a_n\bigr)\longrightarrow1
\quad\text{and}\quad
\sup_{u\in K_n}\left\|J_{\tilde{Q}_n}(u)-J_n\right\|=o_{\mathbb{P}_{n,\theta_n}}(a_n) P n , θ n ( σ m i n ( J n ) ≥ a n ) ⟶ 1 and u ∈ K n sup J Q ~ n ( u ) − J n = o P n , θ n ( a n ) on F n F_n F n ϕ − 1 \phi^{-1} ϕ − 1 b n b_n b n K n K_n K n
Δ K , n : = sup u ∈ K n ∥ Q ~ n ( u ) − Q n ( u ) ∥ . \Delta_{K,n}:=\sup_{u\in K_n}\left\|\tilde{Q}_n(u)-Q_n(u)\right\|. Δ K , n := u ∈ K n sup Q ~ n ( u ) − Q n ( u ) . Let { ρ n ′ } n ≥ 1 \{\rho'_n\}_{n\geq1} { ρ n ′ } n ≥ 1 ρ n ′ > 0 \rho'_n>0 ρ n ′ > 0 n n n
ρ n ′ b n Δ K , n a n = o P n , θ n ( 1 ) , \frac{\rho'_n b_n\Delta_{K,n}}{a_n}=o_{\mathbb{P}_{n,\theta_n}}(1), a n ρ n ′ b n Δ K , n = o P n , θ n ( 1 ) , then
ρ n ′ ∥ θ ~ n − θ ^ n ∥ = o P n , θ n ( 1 ) . \rho'_n\left\|\tilde{\theta}_n-\hat{\theta}_n\right\|=o_{\mathbb{P}_{n,\theta_n}}(1). ρ n ′ θ ~ n − θ ^ n = o P n , θ n ( 1 ) . Proof. The assumptions of Theorem 2 hold with K n K_n K n D n D_n D n Δ K , n \Delta_{K,n} Δ K , n Δ n \Delta_n Δ n
G n : = { ∥ u ~ n − u ^ n ∥ ≤ 2 Δ K , n a n } . G_n:=\left\{\left\|\tilde{u}_n-\hat{u}_n\right\|\leq\frac{2\Delta_{K,n}}{a_n}\right\}. G n := { ∥ u ~ n − u ^ n ∥ ≤ a n 2 Δ K , n } . By Theorem 2, P n , θ n ( G n ) → 1 \mathbb{P}_{n,\theta_n}(G_n)\to1 P n , θ n ( G n ) → 1 F n ∩ G n F_n\cap G_n F n ∩ G n K n K_n K n ϕ − 1 \phi^{-1} ϕ − 1 b n b_n b n
ρ n ′ ∥ θ ~ n − θ ^ n ∥ ≤ ρ n ′ b n ∥ u ~ n − u ^ n ∥ ≤ 2 ρ n ′ b n Δ K , n a n . \rho'_n\left\|\tilde{\theta}_n-\hat{\theta}_n\right\|
\leq\rho'_n b_n\left\|\tilde{u}_n-\hat{u}_n\right\|
\leq\frac{2\rho'_n b_n\Delta_{K,n}}{a_n}. ρ n ′ θ ~ n − θ ^ n ≤ ρ n ′ b n ∥ u ~ n − u ^ n ∥ ≤ a n 2 ρ n ′ b n Δ K , n . For every ε > 0 \varepsilon>0 ε > 0
P n , θ n ( ρ n ′ ∥ θ ~ n − θ ^ n ∥ > ε ) ≤ P n , θ n ( F n c ∪ G n c ) + P n , θ n ( 2 ρ n ′ b n Δ K , n a n > ε ) ⟶ 0 , \mathbb{P}_{n,\theta_n}\left(\rho'_n\left\|\tilde{\theta}_n-\hat{\theta}_n\right\|>\varepsilon\right)
\leq\mathbb{P}_{n,\theta_n}(F_n^c\cup G_n^c)
+\mathbb{P}_{n,\theta_n}\left(\frac{2\rho'_n b_n\Delta_{K,n}}{a_n}>\varepsilon\right)
\longrightarrow0, P n , θ n ( ρ n ′ θ ~ n − θ ^ n > ε ) ≤ P n , θ n ( F n c ∪ G n c ) + P n , θ n ( a n 2 ρ n ′ b n Δ K , n > ε ) ⟶ 0 , because ρ n ′ b n Δ K , n / a n = o P n , θ n ( 1 ) \rho'_n b_n\Delta_{K,n}/a_n=o_{\mathbb{P}_{n,\theta_n}}(1) ρ n ′ b n Δ K , n / a n = o P n , θ n ( 1 ) F n ∩ G n F_n\cap G_n F n ∩ G n
Remark. The effective approximation scale is Δ n / a n \Delta_n/a_n Δ n / a n ρ n \rho_n ρ n θ n \theta_n θ n θ ∗ \theta^* θ ∗ D n D_n D n a n a_n a n D n D_n D n Δ n \Delta_n Δ n
A Construction and Verification Procedure
Our procedure for constructing and verifying an approximate estimator can be organized into the following three broad stages:
Derive the approximate estimator.
Begin by specifying the scale on which the roots are to be compared — a scalar or matrix normalization in chart coordinates, or the original parameter scale — since this choice can determine the appropriate chart and the number of retained terms. Identify the selected exact root θ ^ n \hat{\theta}_n θ ^ n q n q_n q n { θ n } n ≥ 1 ∈ R \{\theta_n\}_{n\geq1} \in\mathbb{R} { θ n } n ≥ 1 ∈ R q n q_n q n ϕ : N → U \phi: N \to U ϕ : N → U u n : = ϕ ( θ n ) u_n := \phi(\theta_n) u n := ϕ ( θ n ) Q ~ n ( u ) = 0 \widetilde{Q}_n(u) = 0 Q n ( u ) = 0 u ~ n \widetilde{u}_n u n u ~ n ∈ U \widetilde{u}_n \in U u n ∈ U θ ~ n : = ϕ − 1 ( u ~ n ) \widetilde{\theta}_n := \phi^{-1}(\widetilde{u}_n) θ n := ϕ − 1 ( u n )
Verify the approximation.
Begin by localizing the explicit root u ~ n \widetilde{u}_n u n u ~ n \widetilde{u}_n u n U U U u ^ n \hat{u}_n u ^ n D n ⊆ U D_n \subseteq U D n ⊆ U
sup u ∈ D n ∥ u ∥ ⟶ 0. \sup_{u\in D_n} \lVert u\rVert\longrightarrow0. u ∈ D n sup ∥ u ∥ ⟶ 0. Next, ensure that Q ~ n \widetilde{Q}_n Q n C 1 C^1 C 1 D n D_n D n J n J_n J n a n > 0 a_n > 0 a n > 0
P n , θ n ( σ min ( J n ) ≥ a n ) ⟶ 1 and sup u ∈ D n ∥ J Q ~ n ( u ) − J n ∥ = o P ( a n ) . \mathbb{P}_{n,\theta_n}\left(\sigma_{\min}(J_n) \geq a_n\right) \longrightarrow1
\quad\text{and}\quad
\sup_{u\in D_n}\left\lVert J\widetilde{Q}_n(u)-J_n\right\rVert= o_{\mathbb{P}}(a_n). P n , θ n ( σ m i n ( J n ) ≥ a n ) ⟶ 1 and u ∈ D n sup J Q n ( u ) − J n = o P ( a n ) . Define the error
Δ n : = sup u ∈ D n ∥ Q ~ n ( u ) − Q n ( u ) ∥ . \Delta_n := \sup_{u\in D_n}\left\lVert\widetilde{Q}_n(u)-Q_n(u)\right\rVert. Δ n := u ∈ D n sup Q n ( u ) − Q n ( u ) . Any random or diverging factor multiplying a truncated term must be included in this error. By Theorem 2 and Corollary 2, equivalence at a scalar chart scale ρ n \rho_n ρ n
ρ n Δ n a n = o P ( 1 ) . \frac{\rho_n\Delta_n}{a_n}=o_{\mathbb{P}}(1). a n ρ n Δ n = o P ( 1 ) . For a general matrix normalization, use Proposition 2. If comparison is required on the original parameter scale and ϕ − 1 \phi^{-1} ϕ − 1 b n b_n b n b n Δ n / a n = o P ( 1 ) b_n\Delta_n/a_n=o_{\mathbb{P}}(1) b n Δ n / a n = o P ( 1 )
Obtain the asymptotic distribution, when required.
Derive the limiting distribution of whichever estimator is more tractable (usually, the explicit approximate estimator). Because the data-generating parameter θ n \theta_n θ n n n n W n , i W_{n,i} W n , i
∑ i E n , θ n [ ∣ W n , i ∣ 2 + δ ] ( ∑ i Var n , θ n ( W n , i ) ) 1 + δ / 2 ⟶ 0. \frac{\sum_i \mathbb{E}_{n,\theta_n}\left[|W_{n,i}|^{2+\delta}\right]}
{\left(\sum_i \operatorname{Var}_{n,\theta_n}(W_{n,i})\right)^{1+\delta/2}}
\longrightarrow0. ( ∑ i Var n , θ n ( W n , i ) ) 1 + δ /2 ∑ i E n , θ n [ ∣ W n , i ∣ 2 + δ ] ⟶ 0. for some δ > 0 \delta> 0 δ > 0 E n , θ n \mathbb{E}_{n,\theta_n} E n , θ n Var n , θ n \operatorname{Var}_{n,\theta_n} Var n , θ n P n , θ n \mathbb{P}_{n,\theta_n} P n , θ n ( 2 + δ ) (2+\delta) ( 2 + δ ) [38 Theorem 2.27] or by the Cramér–Wold device. A Taylor expansion or the delta method can then transfer the limit from the underlying sample statistics to the explicit estimator.
Once the limiting distribution of one root is established, verify that a deterministic nonsingular normalization R n \mathbf{R}_n R n
R n ( u ~ n − u ^ n ) = o P ( 1 ) . \mathbf{R}_n(\tilde{\mathbf{u}}_n-\hat{\mathbf{u}}_n)=o_{\mathbb{P}(1).} R n ( u ~ n − u ^ n ) = o P ( 1 ) . Slutsky’s theorem then transfers the weak limit between the two transformed roots; see the remark following Proposition 2. A limiting distribution on the original parameter scale may additionally require a separate expansion or the delta method for ϕ − 1 \phi^{-1} ϕ − 1 u n u_n u n
Worked Examples: the Gamma Distribution
We begin our examples with the gamma distribution, which serves as a useful test case for our general framework. The distribution is simple enough that the likelihood and score functions can be written out explicitly, but the score function for the shape parameter involves the digamma function and therefore does not admit a root in closed form. The first two examples consider the regime where the gamma shape tends to infinity, while gamma regression considers the equivalent regime in terms of the precision. Section 3.1 treats a known rate and gives a one-dimensional version of the construction. Section 3.2 estimates shape and rate jointly, producing a two-dimensional chart and different componentwise rates. Section 3.3 extends the construction to independent, non-identically distributed responses. Section 3.4 reports simulation studies for all three gamma settings, and Section 3.5 compares gamma regression fits based on the exact MLE and our proposed closed-form estimator in a real-world data setting.
Fixed Rate Parameter
To begin with, let X 1 , … , X n ∼ i i d Gamma ( α , β ) X_1,\ldots,X_n \mathrel{\overset{\mathrm{iid}}{\sim}} \operatorname{Gamma}(\alpha,\beta) X 1 , … , X n ∼ iid Gamma ( α , β ) β > 0 \beta> 0 β > 0 α → ∞ \alpha\to\infty α → ∞ β \beta β x > 0 x > 0 x > 0
f α , β ( x ) : = β α Γ ( α ) x α − 1 e − β x . f_{\alpha,\beta}(x) := \frac{\beta^\alpha}{\Gamma(\alpha)}x^{\alpha-1}e^{-\beta x}. f α , β ( x ) := Γ ( α ) β α x α − 1 e − β x . Write G n : = ( ∏ i = 1 n X i ) 1 / n G_n := \left(\prod_{i=1}^{n}X_i\right)^{1/n} G n := ( ∏ i = 1 n X i ) 1/ n X i X_i X i
Derivation of the Approximate MLE
The score equation for the shape parameter is
ψ ( α ) = log ( β G n ) , \psi(\alpha)=\log(\beta G_n), ψ ( α ) = log ( β G n ) , where ψ : = Γ ′ / Γ \psi:= \Gamma'/\Gamma ψ := Γ ′ /Γ ψ \psi ψ − ∞ -\infty − ∞ + ∞ +\infty + ∞ α ^ n \hat{\alpha}_n α ^ n ℓ n \ell_n ℓ n α \alpha α ℓ n ′ ′ ( α ) = − n ψ 1 ( α ) < 0 \ell_n''(\alpha)=-n\psi_1(\alpha)<0 ℓ n ′′ ( α ) = − n ψ 1 ( α ) < 0 α ^ n \hat{\alpha}_n α ^ n ℓ n \ell_n ℓ n α ^ n \hat{\alpha}_n α ^ n
q n ( α ) : = exp ( − ψ ( α ) ) − 1 β G n , q_n(\alpha):=\exp(-\psi(\alpha))-\frac{1}{\beta G_n}, q n ( α ) := exp ( − ψ ( α )) − β G n 1 , which is convenient to use as α → ∞ \alpha\to\infty α → ∞
ψ ( x ) = log ( x ) − 1 2 x − 1 12 x 2 + O ( 1 x 4 ) and exp ( − ψ ( x ) ) = 1 x − 1 / 2 + O ( 1 x 3 ) (19) \psi(x)=\log(x)-\frac{1}{2x}-\frac{1}{12x^2}+O\left(\frac{1}{x^4}\right)
\quad\text{and}\quad
\exp(-\psi(x))=\frac{1}{x-1/2}+O\left(\frac{1}{x^3}\right) \tag*{(19)} ψ ( x ) = log ( x ) − 2 x 1 − 12 x 2 1 + O ( x 4 1 ) and exp ( − ψ ( x )) = x − 1/2 1 + O ( x 3 1 ) (19) as x → ∞ x \to\infty x → ∞ [2 Equation 6.3.18] . This suggests the vanishing chart ϕ : N → U \phi: N \to U ϕ : N → U N : = ( 1 / 2 , ∞ ) N := (1/2,\infty) N := ( 1/2 , ∞ ) U : = ( 0 , ∞ ) U := (0,\infty) U := ( 0 , ∞ )
ϕ ( α ) : = 1 α − 1 / 2 and u : = ϕ ( α ) and ϕ − 1 ( u ) = 1 2 + 1 u , \phi(\alpha) := \frac{1}{\alpha- 1/2} \quad\text{and}\quad u := \phi(\alpha) \quad\text{and}\quad\phi^{-1}(u) = \frac{1}{2} + \frac{1}{u}, ϕ ( α ) := α − 1/2 1 and u := ϕ ( α ) and ϕ − 1 ( u ) = 2 1 + u 1 , under which the exact and surrogate equations on the chart scale are
Q n ( u ) : = exp ( − ψ ( 1 2 + 1 u ) ) − 1 β G n and Q ~ n ( u ) : = u − 1 β G n . Q_n(u) := \exp\left(-\psi\left(\frac{1}{2}+\frac{1}{u}\right)\right)-\frac{1}{\beta G_n}
\quad\text{and}\quad
\widetilde{Q}_n(u) := u-\frac{1}{\beta G_n}. Q n ( u ) := exp ( − ψ ( 2 1 + u 1 ) ) − β G n 1 and Q n ( u ) := u − β G n 1 . The latter function has the explicit root u ~ n : = 1 / ( β G n ) \widetilde{u}_n := 1/(\beta G_n) u n := 1/ ( β G n )
α ~ n : = ϕ − 1 ( u ~ n ) = 1 2 + β G n . \widetilde{\alpha}_n := \phi^{-1}(\widetilde{u}_n)=\frac{1}{2}+\beta G_n. α n := ϕ − 1 ( u n ) = 2 1 + β G n . Remark. In practice, of course, α \alpha α α \alpha α a > 0 a>0 a > 0 β \beta β H 0 : α ≤ a H_0:\alpha\leq a H 0 : α ≤ a H A : α > a H_A:\alpha>a H A : α > a δ ∈ ( 0 , 1 ) \delta\in(0,1) δ ∈ ( 0 , 1 ) H 0 H_0 H 0
β X ‾ n > a + a ( 1 − δ ) n δ \beta\overline{X}_n>a+\sqrt{\frac{a(1-\delta)}{n\delta}} β X n > a + n δ a ( 1 − δ ) defines a level-δ \delta δ α \alpha α β X ‾ n ∼ Gamma ( n α , n ) \beta\overline{X}_n\sim\operatorname{Gamma}(n\alpha,n) β X n ∼ Gamma ( n α , n ) β \beta β Gamma ( α , β ) \operatorname{Gamma}(\alpha,\beta) Gamma ( α , β )
Verification of Approximation
Fix a deterministic sequence { α n } n ≥ 1 \{\alpha_n\}_{n\geq1} { α n } n ≥ 1 α n → ∞ \alpha_n\to\infty α n → ∞
u n : = ϕ ( α n ) = 1 α n − 1 / 2 and u ^ n : = ϕ ( α ^ n ) . (20) u_n:=\phi(\alpha_n)=\frac{1}{\alpha_n-1/2}
\quad\text{and}\quad
\widehat{u}_n:=\phi(\widehat{\alpha}_n). \tag*{(20)} u n := ϕ ( α n ) = α n − 1/2 1 and u n := ϕ ( α n ) . (20) By Lemma A.3 in Section A,
u ~ n − u n = O P ( u n 3 n ) + O ( u n 2 n ) + O ( u n 3 ) = o P ( u n ) . \widetilde{u}_n-u_n=O_{\mathbb{P}}\left(\sqrt{\frac{u_n^3}{n}}\right)+O\left(\frac{u_n^2}{n}\right)+O(u_n^3)=o_{\mathbb{P}}(u_n). u n − u n = O P ( n u n 3 ) + O ( n u n 2 ) + O ( u n 3 ) = o P ( u n ) . On the event ∣ u ~ n / u n − 1 ∣ ≤ 1 / 4 |\widetilde{u}_n/u_n-1|\leq1/4 ∣ u n / u n − 1∣ ≤ 1/4
D n ∗ : = { u : ∣ u − u ~ n ∣ ≤ u n 4 } . D_n^*:=\left\{u:|u-\widetilde{u}_n|\leq\frac{u_n}{4}\right\}. D n ∗ := { u : ∣ u − u n ∣ ≤ 4 u n } . This event has probability tending to one, and D n ∗ ⊂ [ u n / 2 , 3 u n / 2 ] D_n^*\subset[u_n/2,3u_n/2] D n ∗ ⊂ [ u n /2 , 3 u n /2 ] R n : = u n − 1 R_n:=u_n^{-1} R n := u n − 1 J n : = 1 J_n:=1 J n := 1 r n : = 1 / 4 r_n:=1/4 r n := 1/4 J Q ~ n ( u ) = 1 J\widetilde{Q}_n(u)=1 J Q n ( u ) = 1 κ n = 0 \kappa_n=0 κ n = 0
Q n ( u ) − Q ~ n ( u ) = exp ( − ψ ( 1 2 + 1 u ) ) − u = O ( u 3 ) Q_n(u)-\widetilde{Q}_n(u)=\exp\left(-\psi\left(\frac{1}{2}+\frac{1}{u}\right)\right)-u=O(u^3) Q n ( u ) − Q n ( u ) = exp ( − ψ ( 2 1 + u 1 ) ) − u = O ( u 3 ) as u ↓ 0 u\downarrow0 u ↓ 0
η n = sup u ∈ D n ∗ u n − 1 ∣ Q n ( u ) − Q ~ n ( u ) ∣ = O ( u n 2 ) = o ( 1 ) . \eta_n=\sup_{u\in D_n^*}u_n^{-1}|Q_n(u)-\widetilde{Q}_n(u)|=O(u_n^2)=o(1). η n = u ∈ D n ∗ sup u n − 1 ∣ Q n ( u ) − Q n ( u ) ∣ = O ( u n 2 ) = o ( 1 ) . The exact equation has a unique root, so Corollary 1 gives
∣ u ~ n − u ^ n ∣ = O P ( u n 3 ) and u ^ n − u n = o P ( u n ) . (21) |\widetilde{u}_n-\widehat{u}_n|=O_{\mathbb{P}}(u_n^3)
\quad\text{and}\quad
\widehat{u}_n-u_n=o_{\mathbb{P}}(u_n). \tag*{(21)} ∣ u n − u n ∣ = O P ( u n 3 ) and u n − u n = o P ( u n ) . (21) Consequently, u ~ n / u n , u ^ n / u n → p 1 \widetilde{u}_n/u_n,\widehat{u}_n/u_n \xrightarrow{p} 1 u n / u n , u n / u n p 1
D n : = [ u n 2 , 2 u n ] D_n := \left[\frac{u_n}{2},2u_n\right] D n := [ 2 u n , 2 u n ] therefore contain both roots with probability tending to one, and sup u ∈ D n ∣ u ∣ = 2 u n → 0 \sup_{u\in D_n}|u|=2u_n\to0 sup u ∈ D n ∣ u ∣ = 2 u n → 0 J n : = 1 J_n:=1 J n := 1 a n : = 1 a_n:=1 a n := 1
Δ n : = sup u ∈ D n ∣ Q n ( u ) − Q ~ n ( u ) ∣ = O ( u n 3 ) . \Delta_n := \sup_{u\in D_n}|Q_n(u)-\widetilde{Q}_n(u)|=O(u_n^3). Δ n := u ∈ D n sup ∣ Q n ( u ) − Q n ( u ) ∣ = O ( u n 3 ) . Thus the roots are equivalent at scale 1 relative to the ϕ \phi ϕ Definition 4 ). More generally, if ρ n → ∞ \rho_n\to\infty ρ n → ∞ ρ n u n 3 → 0 \rho_nu_n^3\to0 ρ n u n 3 → 0
ρ n ∣ u ~ n − u ^ n ∣ = o P ( 1 ) , \rho_n|\widetilde{u}_n-\widehat{u}_n|=o_P(1), ρ n ∣ u n − u n ∣ = o P ( 1 ) , and the estimators are asymptotically equivalent under the normalization ρ n \rho_n ρ n u 1 , u 2 ∈ D n u_1,u_2\in D_n u 1 , u 2 ∈ D n
∣ ϕ − 1 ( u 1 ) − ϕ − 1 ( u 2 ) ∣ = ∣ u 1 − u 2 ∣ u 1 u 2 ≤ 4 u n 2 ∣ u 1 − u 2 ∣ . |\phi^{-1}(u_1)-\phi^{-1}(u_2)|=\frac{|u_1-u_2|}{u_1u_2}\leq\frac{4}{u_n^2}|u_1-u_2|. ∣ ϕ − 1 ( u 1 ) − ϕ − 1 ( u 2 ) ∣ = u 1 u 2 ∣ u 1 − u 2 ∣ ≤ u n 2 4 ∣ u 1 − u 2 ∣. Thus ϕ − 1 \phi^{-1} ϕ − 1 b n b_n b n D n D_n D n b n : = 4 / u n 2 b_n:=4/u_n^2 b n := 4/ u n 2
b n Δ n a n = O ( u n ) ⟶ 0. \frac{b_n\Delta_n}{a_n}=O(u_n)\longrightarrow0. a n b n Δ n = O ( u n ) ⟶ 0. It follows from Corollary 3 that
∣ α ~ n − α ^ n ∣ = O P ( b n Δ n a n ) = O P ( u n ) = O P ( 1 α n ) = o P ( 1 ) , |\widetilde{\alpha}_n-\widehat{\alpha}_n|=O_P\left(\frac{b_n\Delta_n}{a_n}\right)=O_P(u_n)=O_P\left(\frac{1}{\alpha_n}\right)=o_P(1), ∣ α n − α n ∣ = O P ( a n b n Δ n ) = O P ( u n ) = O P ( α n 1 ) = o P ( 1 ) , which confirms that α ~ n \widetilde{\alpha}_n α n
Asymptotic Distribution
Suppose now that n u n 3 ⟶ 0 nu_n^3\longrightarrow0 n u n 3 ⟶ 0 α n 3 / n → ∞ \alpha_n^3/n\to\infty α n 3 / n → ∞ n n n X n , 1 , … , X n , n ∼ i i d Gamma ( α n , β ) X_{n,1},\ldots,X_{n,n}\mathrel{\mathop{\sim}^{\mathrm{iid}}}\operatorname{Gamma}(\alpha_n,\beta) X n , 1 , … , X n , n ∼ iid Gamma ( α n , β )
Y n , i : = log ( X n , i ) and Y ‾ n : = 1 n ∑ i = 1 n Y n , i . Y_{n,i}:=\log(X_{n,i}) \quad\text{and}\quad\overline{Y}_n:=\frac{1}{n}\sum_{i=1}^nY_{n,i}. Y n , i := log ( X n , i ) and Y n := n 1 i = 1 ∑ n Y n , i . The third moment bound in Lemma A.4 of Section A gives the Lyapunov triangular array CLT
n [ Y ‾ n − ( ψ ( α n ) − log ( β ) ) ] ψ 1 ( α n ) → d N ( 0 , 1 ) . \frac{\sqrt{n}\left[\overline{Y}_n-\left(\psi(\alpha_n)-\log(\beta)\right)\right]}{\sqrt{\psi_1(\alpha_n)}}\xrightarrow{d}\mathcal{N}(0,1). ψ 1 ( α n ) n [ Y n − ( ψ ( α n ) − log ( β ) ) ] d N ( 0 , 1 ) . For ease of notation, define
Z n : = Y ‾ n − ( ψ ( α n ) − log ( β ) ) and c n : = e − ψ ( α n ) . Z_n:=\overline{Y}_n-\left(\psi(\alpha_n)-\log(\beta)\right)\quad\text{and}\quad c_n:=e^{-\psi(\alpha_n)}. Z n := Y n − ( ψ ( α n ) − log ( β ) ) and c n := e − ψ ( α n ) . Since
ψ 1 ( α n ) = u n + O ( u n 3 ) , \psi_1(\alpha_n)=u_n+O(u_n^3), ψ 1 ( α n ) = u n + O ( u n 3 ) , the preceding CLT implies
n u n Z n → d N ( 0 , 1 ) . \sqrt{\frac{n}{u_n}}Z_n\xrightarrow{d}\mathcal{N}(0,1). u n n Z n d N ( 0 , 1 ) . Moreover, u ~ n = c n e − Z n \widetilde{u}_n=c_ne^{-Z_n} u n = c n e − Z n c n = u n + O ( u n 3 ) c_n=u_n+O(u_n^3) c n = u n + O ( u n 3 ) Z n = O P ( u n / n ) Z_n=O_P\left(\sqrt{u_n/n}\right) Z n = O P ( u n / n ) e − Z n e^{-Z_n} e − Z n
u ~ n − u n = − c n Z n + ( c n − u n ) + O P ( c n Z n 2 ) . \widetilde{u}_n-u_n=-c_nZ_n+(c_n-u_n)+O_P(c_nZ_n^2). u n − u n = − c n Z n + ( c n − u n ) + O P ( c n Z n 2 ) . Under the given regime,
n u n 3 ( c n − u n ) = O ( n u n 3 ) → 0 \sqrt{\frac{n}{u_n^3}}(c_n-u_n)=O(\sqrt{nu_n^3})\to0 u n 3 n ( c n − u n ) = O ( n u n 3 ) → 0 and
n u n 3 c n Z n 2 = O P ( u n n ) = o P ( 1 ) . \sqrt{\frac{n}{u_n^3}}c_n Z_n^2=O_{\mathrm{P}}\left(\sqrt{\frac{u_n}{n}}\right)=o_{\mathrm{P}}(1). u n 3 n c n Z n 2 = O P ( n u n ) = o P ( 1 ) . Since c n / u n → 1 c_n/u_n\to1 c n / u n → 1
n u n 3 ( u ~ n − u n ) → d N ( 0 , 1 ) . \sqrt{\frac{n}{u_n^3}}(\tilde{u}_n-u_n)\xrightarrow{d}\mathcal{N}(0,1). u n 3 n ( u ~ n − u n ) d N ( 0 , 1 ) . We have ∣ u ~ n − u ^ n ∣ = O P ( u n 3 ) |\tilde{u}_n-\hat{u}_n|=O_{\mathrm{P}}(u_n^3) ∣ u ~ n − u ^ n ∣ = O P ( u n 3 ) (21) . Hence
n u n 3 ( u ~ n − u ^ n ) = O P ( n u n 3 ) = o P ( 1 ) , \sqrt{\frac{n}{u_n^3}}(\tilde{u}_n-\hat{u}_n)=O_{\mathrm{P}}(\sqrt{nu_n^3})=o_{\mathrm{P}}(1), u n 3 n ( u ~ n − u ^ n ) = O P ( n u n 3 ) = o P ( 1 ) , and Slutsky’s theorem yields
n u n 3 ( u ^ n − u n ) → d N ( 0 , 1 ) . \sqrt{\frac{n}{u_n^3}}(\hat{u}_n-u_n)\xrightarrow{d}\mathcal{N}(0,1). u n 3 n ( u ^ n − u n ) d N ( 0 , 1 ) . Finally, Lemma A.5 gives
n u n ( ϕ − 1 ( u ~ n ) − α n ) = n u n 3 ( u ~ n − u n ) + o P ( 1 ) \sqrt{nu_n}\left(\phi^{-1}(\tilde{u}_n)-\alpha_n\right)=\sqrt{\frac{n}{u_n^3}}(\tilde{u}_n-u_n)+o_{\mathrm{P}}(1) n u n ( ϕ − 1 ( u ~ n ) − α n ) = u n 3 n ( u ~ n − u n ) + o P ( 1 ) and
n u n ( ϕ − 1 ( u ^ n ) − α n ) = n u n 3 ( u ^ n − u n ) + o P ( 1 ) . \sqrt{nu_n}\left(\phi^{-1}(\hat{u}_n)-\alpha_n\right)=\sqrt{\frac{n}{u_n^3}}(\hat{u}_n-u_n)+o_{\mathrm{P}}(1). n u n ( ϕ − 1 ( u ^ n ) − α n ) = u n 3 n ( u ^ n − u n ) + o P ( 1 ) . Since u n = 1 / α n + O ( α n − 2 ) u_n=1/\alpha_n+O(\alpha_n^{-2}) u n = 1/ α n + O ( α n − 2 )
n α n ( α ~ n − α n ) → d N ( 0 , 1 ) \sqrt{\frac{n}{\alpha_n}}(\tilde{\alpha}_n-\alpha_n)\xrightarrow{d}\mathcal{N}(0,1) α n n ( α ~ n − α n ) d N ( 0 , 1 ) and
n α n ( α ^ n − α n ) → d N ( 0 , 1 ) . \sqrt{\frac{n}{\alpha_n}}(\hat{\alpha}_n-\alpha_n)\xrightarrow{d}\mathcal{N}(0,1). α n n ( α ^ n − α n ) d N ( 0 , 1 ) . Unknown Rate Parameter
We now suppose that X 1 , … , X n ∼ i i d Gamma ( α , β ) X_1,\ldots,X_n\mathrel{\overset{\mathrm{iid}}{\sim}}\operatorname{Gamma}(\alpha,\beta) X 1 , … , X n ∼ iid Gamma ( α , β ) α \alpha α β \beta β α → ∞ \alpha\to\infty α → ∞ β > 0 \beta>0 β > 0
X ˉ n : = 1 n ∑ i = 1 n X i and G n : = ( ∏ i = 1 n X i ) 1 n . \bar{X}_n:=\frac{1}{n}\sum_{i=1}^{n}X_i
\qquad\text{and}\qquad
G_n:=\left(\prod_{i=1}^{n}X_i\right)^{\frac{1}{n}}. X ˉ n := n 1 i = 1 ∑ n X i and G n := ( i = 1 ∏ n X i ) n 1 . Derivation of the Approximate MLE
The score vector is
S n ( α , β ) : = ( ∂ ∂ α ℓ n ( α , β ) ∂ ∂ β ℓ n ( α , β ) ) = ( n ( log ( β ) + log ( G n ) − ψ ( α ) ) n ( α β − X ˉ n ) ) . S_n(\alpha,\beta):=
\begin{pmatrix}
\frac{\partial}{\partial\alpha}\ell_n(\alpha,\beta)\\
\frac{\partial}{\partial\beta}\ell_n(\alpha,\beta)
\end{pmatrix}
=
\begin{pmatrix}
n(\log(\beta)+\log(G_n)-\psi(\alpha))\\
n\left(\frac{\alpha}{\beta}-\bar{X}_n\right)
\end{pmatrix}. S n ( α , β ) := ( ∂ α ∂ ℓ n ( α , β ) ∂ β ∂ ℓ n ( α , β ) ) = ( n ( log ( β ) + log ( G n ) − ψ ( α )) n ( β α − X ˉ n ) ) . For each α > 0 \alpha>0 α > 0 β \beta β
∂ ∂ β ℓ n ( α , β ) = n ( α β − X ˉ n ) = 0 , \frac{\partial}{\partial\beta}\ell_n(\alpha,\beta)
=n\left(\frac{\alpha}{\beta}-\bar{X}_n\right)=0, ∂ β ∂ ℓ n ( α , β ) = n ( β α − X ˉ n ) = 0 , has the unique solution β = α / X ˉ n \beta=\alpha/\bar{X}_n β = α / X ˉ n β \beta β α \alpha α
log ( α ^ n ) − ψ ( α ^ n ) = log ( X ˉ n G n ) , (22) \log(\hat{\alpha}_n)-\psi(\hat{\alpha}_n)=\log\left(\frac{\bar{X}_n}{G_n}\right), \tag*{(22)} log ( α ^ n ) − ψ ( α ^ n ) = log ( G n X ˉ n ) , (22) with β ^ n = α ^ n / X ˉ n \hat{\beta}_n = \hat{\alpha}_n/\bar{X}_n β ^ n = α ^ n / X ˉ n n ≥ 2 n \ge2 n ≥ 2 log ( X ˉ n / G ˉ n ) > 0 \log(\bar{X}_n/\bar{G}_n) > 0 log ( X ˉ n / G ˉ n ) > 0
g ( α ) : = log ( α ) − ψ ( α ) g(\alpha) := \log(\alpha) - \psi(\alpha) g ( α ) := log ( α ) − ψ ( α ) is strictly decreasing from ∞ \infty ∞ (22) changes sign exactly once. Its unique root α ^ n \hat{\alpha}_n α ^ n β ^ n = α ^ n / X ˉ n \hat{\beta}_n = \hat{\alpha}_n/\bar{X}_n β ^ n = α ^ n / X ˉ n
An equivalent system defining the exact root is
q n ( α , β ) : = ( log ( α ) − ψ ( α ) − log ( X ˉ n G ˉ n ) β α − 1 X ˉ n ) = 0. \mathbf{q}_n(\alpha,\beta) := \begin{pmatrix} \log(\alpha) - \psi(\alpha) - \log\left(\dfrac{\bar{X}_n}{\bar{G}_n}\right) \\ \dfrac{\beta}{\alpha} - \dfrac{1}{\bar{X}_n} \end{pmatrix} = \mathbf{0}. q n ( α , β ) := log ( α ) − ψ ( α ) − log ( G ˉ n X ˉ n ) α β − X ˉ n 1 = 0 . By (19) , we have
log ( α ) − ψ ( α ) = 1 2 α + 1 12 α 2 + O ( 1 α 4 ) . \log(\alpha) - \psi(\alpha) = \frac{1}{2\alpha} + \frac{1}{12\alpha^2} + O\left(\frac{1}{\alpha^4}\right). log ( α ) − ψ ( α ) = 2 α 1 + 12 α 2 1 + O ( α 4 1 ) . We retain the first two terms of the expansion and use the surrogate
q ~ n ( α , β ) : = ( 1 2 α + 1 12 α 2 − log ( X ˉ n G ˉ n ) β α − 1 X ˉ n ) , \widetilde{\mathbf{q}}_n(\alpha,\beta) := \begin{pmatrix} \dfrac{1}{2\alpha} + \dfrac{1}{12\alpha^2} - \log\left(\dfrac{\bar{X}_n}{\bar{G}_n}\right) \\ \dfrac{\beta}{\alpha} - \dfrac{1}{\bar{X}_n} \end{pmatrix}, q n ( α , β ) := 2 α 1 + 12 α 2 1 − log ( G ˉ n X ˉ n ) α β − X ˉ n 1 , which suggests the chart
ϕ ( α , β ) : = ( 1 2 α + 1 12 α 2 β α ) and u : = ϕ ( α , β ) = ( u 1 u 2 ) . (23) \phi(\alpha,\beta) := \begin{pmatrix} \dfrac{1}{2\alpha} + \dfrac{1}{12\alpha^2} \\ \dfrac{\beta}{\alpha} \end{pmatrix}
\quad\text{and}\quad
\mathbf{u} := \phi(\alpha,\beta) = \begin{pmatrix} u_1 \\ u_2 \end{pmatrix}. \tag*{(23)} ϕ ( α , β ) := 2 α 1 + 12 α 2 1 α β and u := ϕ ( α , β ) = ( u 1 u 2 ) . (23) This is a bijection from ( 0 , ∞ ) 2 (0,\infty)^2 ( 0 , ∞ ) 2 U : = ( 0 , ∞ ) 2 U := (0,\infty)^2 U := ( 0 , ∞ ) 2
ϕ − 1 ( u ) = ( 1 − 3 + 9 + 12 u 1 u 2 − 3 + 9 + 12 u 1 ) . (24) \phi^{-1}(\mathbf{u}) = \begin{pmatrix} \dfrac{1}{-3 + \sqrt{9 + 12u_1}} \\ \dfrac{u_2}{-3 + \sqrt{9 + 12u_1}} \end{pmatrix}. \tag*{(24)} ϕ − 1 ( u ) = − 3 + 9 + 12 u 1 1 − 3 + 9 + 12 u 1 u 2 . (24) For fixed β > 0 \beta> 0 β > 0 α → ∞ \alpha\to\infty α → ∞ ϕ \phi ϕ
h ( u 1 ) : = log ( 1 − 3 + 9 + 12 u 1 ) − ψ ( 1 − 3 + 9 + 12 u 1 ) . h(u_1) := \log\left(\frac{1}{-3 + \sqrt{9 + 12u_1}}\right) - \psi\left(\frac{1}{-3 + \sqrt{9 + 12u_1}}\right). h ( u 1 ) := log ( − 3 + 9 + 12 u 1 1 ) − ψ ( − 3 + 9 + 12 u 1 1 ) . The exact and surrogate equations in chart coordinates are, respectively,
Q n ( u ) : = ( h ( u 1 ) − log ( X ˉ n G ˉ n ) u 2 − 1 X ˉ n ) and Q ~ n ( u ) : = ( u 1 − log ( X ˉ n G ˉ n ) u 2 − 1 X ˉ n ) , \mathbf{Q}_n(\mathbf{u}) := \begin{pmatrix} h(u_1) - \log\left(\dfrac{\bar{X}_n}{\bar{G}_n}\right) \\ u_2 - \dfrac{1}{\bar{X}_n} \end{pmatrix}
\quad\text{and}\quad
\widetilde{\mathbf{Q}}_n(\mathbf{u}) := \begin{pmatrix} u_1 - \log\left(\dfrac{\bar{X}_n}{\bar{G}_n}\right) \\ u_2 - \dfrac{1}{\bar{X}_n} \end{pmatrix}, Q n ( u ) := h ( u 1 ) − log ( G ˉ n X ˉ n ) u 2 − X ˉ n 1 and Q n ( u ) := u 1 − log ( G ˉ n X ˉ n ) u 2 − X ˉ n 1 , and the surrogate has the explicit root
u ~ n : = ( log ( X ˉ n G ˉ n ) 1 X ˉ n ) . \widetilde{\mathbf{u}}_n := \begin{pmatrix} \log\left(\dfrac{\bar{X}_n}{\bar{G}_n}\right) \\ \dfrac{1}{\bar{X}_n} \end{pmatrix}. u n := log ( G ˉ n X ˉ n ) X ˉ n 1 . Because X ˉ n > G n > 0 \bar{X}_n > G_n > 0 X ˉ n > G n > 0 n ≥ 2 n \ge2 n ≥ 2 u ~ n ∈ U \tilde{u}_n \in U u ~ n ∈ U
α ~ n : = 1 − 3 + 9 + 12 log ( X ˉ n / G n ) and β ~ n : = α ~ n X ˉ n . \tilde{\alpha}_n := \frac{1}{-3+\sqrt{9+12\log(\bar{X}_n/G_n)}} \quad\text{and} \quad\tilde{\beta}_n := \frac{\tilde{\alpha}_n}{\bar{X}_n}. α ~ n := − 3 + 9 + 12 log ( X ˉ n / G n ) 1 and β ~ n := X ˉ n α ~ n . Remark. 1. It is essential to retain the term 1 / ( 12 α 2 ) 1/(12\alpha^2) 1/ ( 12 α 2 ) (23) for approximation on the original parameter scale. If only 1 / ( 2 α ) 1/(2\alpha) 1/ ( 2 α ) u 1 2 u_1^2 u 1 2 u 1 − 2 u_1^{-2} u 1 − 2 O ( 1 ) O(1) O ( 1 ) O ( u 1 4 ) O(u_1^4) O ( u 1 4 ) O ( u 1 2 ) O(u_1^2) O ( u 1 2 )
The formula 1 / ( − 3 + 9 + 12 u ) 1/(-3+\sqrt{9+12u}) 1/ ( − 3 + 9 + 12 u ) u → 0 u \to0 u → 0 ( 3 + 9 + 12 u ) / ( 12 u ) (3+\sqrt{9+12u})/(12u) ( 3 + 9 + 12 u ) / ( 12 u )
Verification of Approximation
For notational convenience, write θ : = ( α , β ) T \theta:= (\alpha,\beta)^\mathsf{T} θ := ( α , β ) T θ ^ n : = ( α ^ n , β ^ n ) T \hat{\theta}_n := (\hat{\alpha}_n,\hat{\beta}_n)^\mathsf{T} θ ^ n := ( α ^ n , β ^ n ) T θ ~ n : = ( α ~ n , β ~ n ) T \tilde{\theta}_n := (\tilde{\alpha}_n,\tilde{\beta}_n)^\mathsf{T} θ ~ n := ( α ~ n , β ~ n ) T β > 0 \beta> 0 β > 0 { α n } n ≥ 1 \{\alpha_n\}_{n\ge1} { α n } n ≥ 1 α n → ∞ \alpha_n \to\infty α n → ∞ n n n X n , 1 , … , X n , n ∼ i i d Gamma ( α n , β ) X_{n,1},\ldots,X_{n,n} \overset{\mathrm{iid}}{\sim} \operatorname{Gamma}(\alpha_n,\beta) X n , 1 , … , X n , n ∼ iid Gamma ( α n , β ) X ˉ n \bar{X}_n X ˉ n G n G_n G n
u n : = ϕ ( α n , β ) = ( u 1 , n u 2 , n ) = ( 1 2 α n + 1 12 α n 2 β α n ) . u_n := \phi(\alpha_n,\beta) =
\begin{pmatrix}
u_{1,n}\\
u_{2,n}
\end{pmatrix}
=
\begin{pmatrix}
\dfrac{1}{2\alpha_n}+\dfrac{1}{12\alpha_n^2}\\
\dfrac{\beta}{\alpha_n}
\end{pmatrix}. u n := ϕ ( α n , β ) = ( u 1 , n u 2 , n ) = 2 α n 1 + 12 α n 2 1 α n β . In particular, u 2 , n ∼ 2 β u 1 , n u_{2,n} \sim2\beta u_{1,n} u 2 , n ∼ 2 β u 1 , n Lemma A.8 of Section A give the coordinatewise bounds
u ~ 1 , n − u 1 , n = O P ( u 1 , n n ) + O ( u 1 , n n ) + O ( u 1 , n 4 ) , \tilde{u}_{1,n}-u_{1,n}
=O_{\mathbb{P}}\left(\frac{u_{1,n}}{\sqrt{n}}\right)
+O\left(\frac{u_{1,n}}{n}\right)
+O(u_{1,n}^4), u ~ 1 , n − u 1 , n = O P ( n u 1 , n ) + O ( n u 1 , n ) + O ( u 1 , n 4 ) , u ~ 2 , n − u 2 , n = O P ( u 1 , n 3 / 2 n ) + O ( u 1 , n 2 n ) , \tilde{u}_{2,n}-u_{2,n}
=O_{\mathbb{P}}\left(\frac{u_{1,n}^{3/2}}{\sqrt{n}}\right)
+O\left(\frac{u_{1,n}^2}{n}\right), u ~ 2 , n − u 2 , n = O P ( n u 1 , n 3/2 ) + O ( n u 1 , n 2 ) , and so u ~ n = u n + o P ( u 1 , n ) \tilde{u}_n=u_n+o_{\mathbb{P}}(u_{1,n}) u ~ n = u n + o P ( u 1 , n ) r 0 > 0 r_0>0 r 0 > 0 B ‾ r 0 ( ( 1 , 2 β ) T ) ⊆ ( 0 , ∞ ) 2 \overline{B}_{r_0}((1,2\beta)^\mathsf{T})\subseteq(0,\infty)^2 B r 0 (( 1 , 2 β ) T ) ⊆ ( 0 , ∞ ) 2 u 1 , n − 1 u ~ n ∈ B ‾ r 0 / 2 ( ( 1 , 2 β ) T ) u_{1,n}^{-1}\tilde{u}_n\in\overline{B}_{r_0/2}((1,2\beta)^\mathsf{T}) u 1 , n − 1 u ~ n ∈ B r 0 /2 (( 1 , 2 β ) T )
D n ∗ : = { u : ∥ 1 u 1 , n ( u − u ~ n ) ∥ ≤ r 0 2 } . D_n^*:=\left\{u:\left\|\frac{1}{u_{1,n}}(u-\tilde{u}_n)\right\|\le\frac{r_0}{2}\right\}. D n ∗ := { u : u 1 , n 1 ( u − u ~ n ) ≤ 2 r 0 } . This event has probability tending to one and D n ∗ ⊂ U D_n^*\subset U D n ∗ ⊂ U R n : = u 1 , n − 1 I 2 R_n:=u_{1,n}^{-1}I_2 R n := u 1 , n − 1 I 2 J n : = I 2 J_n:=I_2 J n := I 2 r n : = r 0 / 2 r_n:=r_0/2 r n := r 0 /2 J ~ Q ~ n = I 2 \tilde{J}\tilde{Q}_n=I_2 J ~ Q ~ n = I 2 κ n = 0 \kappa_n=0 κ n = 0 h ( u 1 ) − u 1 = O ( u 1 4 ) h(u_1)-u_1=O(u_1^4) h ( u 1 ) − u 1 = O ( u 1 4 ) u 1 ↓ 0 u_1\downarrow0 u 1 ↓ 0
η n = sup u ∈ D n ∗ ∥ R n [ Q n ( u ) − Q ~ n ( u ) ] ∥ = O ( u 1 , n 3 ) = o ( 1 ) . \eta_n=\sup_{u\in D_n^*}\left\|R_n[Q_n(u)-\tilde{Q}_n(u)]\right\|=O(u_{1,n}^3)=o(1). η n = u ∈ D n ∗ sup R n [ Q n ( u ) − Q ~ n ( u )] = O ( u 1 , n 3 ) = o ( 1 ) . Because the joint MLE is unique, the equation Q n ( u ) = 0 Q_n(u)=0 Q n ( u ) = 0 u ^ n : = ϕ ( α ^ n , β ^ n ) \hat{u}_n:=\phi(\hat{\alpha}_n,\hat{\beta}_n) u ^ n := ϕ ( α ^ n , β ^ n )
∥ u ~ n − u ^ n ∥ = O P ( u 1 , n 4 ) and u ^ n = u n + o P ( u 1 , n ) . (25) \|\tilde{u}_n-\hat{u}_n\|=O_{\mathbb{P}}(u_{1,n}^4)
\quad\text{and}\quad
\hat{u}_n=u_n+o_{\mathbb{P}}(u_{1,n}). \tag*{(25)} ∥ u ~ n − u ^ n ∥ = O P ( u 1 , n 4 ) and u ^ n = u n + o P ( u 1 , n ) . (25) Since u 2 , n ≍ u 1 , n u_{2,n}\asymp u_{1,n} u 2 , n ≍ u 1 , n j = 1 , 2 j=1,2 j = 1 , 2
u ^ j , n u j , n → P 1 and u ~ j , n u j , n → P 1. \frac{\hat{u}_{j,n}}{u_{j,n}}\xrightarrow{\mathbb{P}}1
\quad\text{and}\quad
\frac{\tilde{u}_{j,n}}{u_{j,n}}\xrightarrow{\mathbb{P}}1. u j , n u ^ j , n P 1 and u j , n u ~ j , n P 1. It follows that the deterministic convex sets
D n : = [ u 1 , n 2 , 2 u 1 , n ] × [ u 2 , n 2 , 2 u 2 , n ] D_n:=\left[\frac{u_{1,n}}{2},2u_{1,n}\right]\times\left[\frac{u_{2,n}}{2},2u_{2,n}\right] D n := [ 2 u 1 , n , 2 u 1 , n ] × [ 2 u 2 , n , 2 u 2 , n ] contain both transformed roots with probability tending to one. Moreover,
sup u ∈ D n ∥ u ∥ = O ( u 1 , n ) ⟶ 0. \sup_{u\in D_n}\lVert u\rVert= O(u_{1,n}) \longrightarrow0. u ∈ D n sup ∥ u ∥ = O ( u 1 , n ) ⟶ 0. Because J Q ~ n = I 2 \mathbf{J}_{\widetilde{Q}_n} = \mathbf{I}_2 J Q n = I 2 J n : = I 2 \mathbf{J}_n := \mathbf{I}_2 J n := I 2 a n : = 1 a_n := 1 a n := 1 h ( u 1 ) − u 1 = O ( u 1 4 ) h(u_1)-u_1 = O(u_1^4) h ( u 1 ) − u 1 = O ( u 1 4 ) u 1 ↓ 0 u_1 \downarrow0 u 1 ↓ 0
Δ n : = sup u ∈ D n ∥ Q n ( u ) − Q ~ n ( u ) ∥ = O ( u 1 , n 4 ) . \Delta_n := \sup_{u\in D_n}\lVert Q_n(u)-\widetilde{Q}_n(u)\rVert= O(u_{1,n}^4). Δ n := u ∈ D n sup ∥ Q n ( u ) − Q n ( u )∥ = O ( u 1 , n 4 ) . Now, (25) already establishes approximation relative to the vanishing chart. On the original parameter scale, Lemma A.9 gives
sup u ∈ D n ∥ J ϕ − 1 ( u ) ∥ = O ( 1 u 1 , n 2 ) , \sup_{u\in D_n}\lVert\mathbf{J}_{\phi^{-1}}(u)\rVert= O\left(\frac{1}{u_{1,n}^2}\right), u ∈ D n sup ∥ J ϕ − 1 ( u )∥ = O ( u 1 , n 2 1 ) , so that ϕ − 1 \phi^{-1} ϕ − 1 b n b_n b n D n D_n D n b n = O ( u 1 , n − 2 ) b_n = O(u_{1,n}^{-2}) b n = O ( u 1 , n − 2 )
b n Δ n a n = O ( u 1 , n 2 ) ⟶ 0. \frac{b_n\Delta_n}{a_n} = O(u_{1,n}^2) \longrightarrow0. a n b n Δ n = O ( u 1 , n 2 ) ⟶ 0. It follows from Corollary 3 that
∥ θ ^ n − θ ~ n ∥ = o P ( 1 ) . \lVert\widehat{\theta}_n-\widetilde{\theta}_n\rVert= o_P(1). ∥ θ n − θ n ∥ = o P ( 1 ) . Hence ( α ~ n , β ~ n ) (\widetilde{\alpha}_n,\widetilde{\beta}_n) ( α n , β n )
Joint Asymptotic Distribution
We assume henceforth that n u 1 , n 3 ⟶ 0 \sqrt{n}\,u_{1,n}^3 \longrightarrow0 n u 1 , n 3 ⟶ 0 α n 3 / n ⟶ ∞ \alpha_n^3/\sqrt{n}\longrightarrow\infty α n 3 / n ⟶ ∞ O ( u 1 , n 4 ) O(u_{1,n}^4) O ( u 1 , n 4 )
R n : = ( n u 1 , n 0 0 n u 1 , n 3 / 2 ) . \mathbf{R}_n :=
\begin{pmatrix}
\frac{\sqrt{n}}{u_{1,n}} & 0 \\
0 & \frac{\sqrt{n}}{u_{1,n}^{3/2}}
\end{pmatrix}. R n := u 1 , n n 0 0 u 1 , n 3/2 n . By Lemma A.8, we have
n u 1 , n ( u ~ 1 , n − u 1 , n ) ⟶ d N ( 0 , 2 ) \frac{\sqrt{n}}{u_{1,n}}(\widetilde{u}_{1,n}-u_{1,n}) \mathrel{\mathop{\longrightarrow}^{d}} \mathcal{N}(0,2) u 1 , n n ( u 1 , n − u 1 , n ) ⟶ d N ( 0 , 2 ) and
n u 1 , n 3 / 2 ( u ~ 2 , n − u 2 , n ) ⟶ d N ( 0 , 8 β 2 ) . \frac{\sqrt{n}}{u_{1,n}^{3/2}}(\widetilde{u}_{2,n}-u_{2,n}) \mathrel{\mathop{\longrightarrow}^{d}} \mathcal{N}(0,8\beta^2). u 1 , n 3/2 n ( u 2 , n − u 2 , n ) ⟶ d N ( 0 , 8 β 2 ) . To identify their joint limit, let
W n : = ∑ i = 1 n X n , i and P n , i : = X n , i W n . W_n := \sum_{i=1}^{n}X_{n,i}
\qquad\text{and}\qquad
P_{n,i} := \frac{X_{n,i}}{W_n}. W n := i = 1 ∑ n X n , i and P n , i := W n X n , i . By the standard gamma–Dirichlet decomposition [e.g. 24 Chapter 49] , W n W_n W n P n : = ( P n , 1 , … , P n , n ) P_n := (P_{n,1},\ldots,P_{n,n}) P n := ( P n , 1 , … , P n , n )
u ~ 1 , n = − log ( n ) − 1 n ∑ i = 1 n log ( P n , i ) and u ~ 2 , n = n W n . \widetilde{u}_{1,n}=-\log(n)-\frac{1}{n}\sum_{i=1}^{n}\log(P_{n,i})
\qquad\text{and}\qquad
\widetilde{u}_{2,n}=\frac{n}{W_n}. u 1 , n = − log ( n ) − n 1 i = 1 ∑ n log ( P n , i ) and u 2 , n = W n n . Thus u ~ 1 , n \widetilde{u}_{1,n} u 1 , n u ~ 2 , n \widetilde{u}_{2,n} u 2 , n n n n
R n ( u ~ n − u n ) ⟶ d N ( ( 0 0 ) , ( 2 0 0 8 β 2 ) ) . (26) \mathbf{R}_n(\widetilde{u}_n-u_n)\mathrel{\mathop{\longrightarrow}^{d}}
\mathcal{N}\left(
\begin{pmatrix}0\\0\end{pmatrix},
\begin{pmatrix}2&0\\0&8\beta^2\end{pmatrix}
\right). \tag*{(26)} R n ( u n − u n ) ⟶ d N ( ( 0 0 ) , ( 2 0 0 8 β 2 ) ) . (26) We transfer this limit to the exact root. Taking J n = I 2 J_n = I_2 J n = I 2
Q ~ n ( u ) − Q n ( u ) = ( u 1 − h ( u 1 ) 0 ) . \widetilde{Q}_n(u)-Q_n(u)=\begin{pmatrix}u_1-h(u_1)\\0\end{pmatrix}. Q n ( u ) − Q n ( u ) = ( u 1 − h ( u 1 ) 0 ) . On the sets D n D_n D n
sup u ∈ D n ∥ R n ( Q ~ n ( u ) − Q n ( u ) ) ∥ = O ( n u 1 , n 3 ) = o ( 1 ) . \sup_{u\in D_n}\left\|\mathbb{R}_n\left(\widetilde{Q}_n(u)-Q_n(u)\right)\right\|=O\left(\sqrt{n}u_{1,n}^{3}\right)=o(1). u ∈ D n sup R n ( Q n ( u ) − Q n ( u ) ) = O ( n u 1 , n 3 ) = o ( 1 ) . Thus, Proposition 2 gives
R n ( u ~ n − u ^ n ) = o P ( 1 ) . \mathbb{R}_n(\widetilde{u}_n-\widehat{u}_n)=o_{\mathbb{P}}(1). R n ( u n − u n ) = o P ( 1 ) . By Slutsky’s theorem, we therefore have
R n ( u ^ n − u n ) → d N ( 0 , ( 2 0 0 8 β 2 ) ) . \mathbb{R}_n(\widehat{u}_n-u_n)\xrightarrow{d}\mathcal{N}\left(0,\begin{pmatrix}2&0\\0&8\beta^2\end{pmatrix}\right). R n ( u n − u n ) d N ( 0 , ( 2 0 0 8 β 2 ) ) . It remains to transform these limits back to the original parameter scale. Let F : = ϕ − 1 F:=\phi^{-1} F := ϕ − 1
T n : = ( n u 1 , n 0 0 n ) . T_n:=\begin{pmatrix}\sqrt{n}u_{1,n}&0\\0&\sqrt{n}\end{pmatrix}. T n := ( n u 1 , n 0 0 n ) . By Lemma A.9(ii),
T n J F ( u n ) R n − 1 ⟶ A : = ( − 1 2 0 − β 0 ) . T_n\mathbb{J}_F(u_n)\mathbb{R}_n^{-1}\longrightarrow A:=\begin{pmatrix}-\frac{1}{2}&0\\-\beta&0\end{pmatrix}. T n J F ( u n ) R n − 1 ⟶ A := ( − 2 1 − β 0 0 ) . Moreover, Lemma A.9(iii) gives
T n [ F ( u ~ n ) − F ( u n ) − J F ( u n ) ( u ~ n − u n ) ] = o P ( 1 ) T_n[F(\widetilde{u}_n)-F(u_n)-\mathbb{J}_F(u_n)(\widetilde{u}_n-u_n)]=o_{\mathbb{P}}(1) T n [ F ( u n ) − F ( u n ) − J F ( u n ) ( u n − u n )] = o P ( 1 ) and
T n [ F ( u ^ n ) − F ( u n ) − J F ( u n ) ( u ^ n − u n ) ] = o P ( 1 ) . T_n[F(\widehat{u}_n)-F(u_n)-\mathbb{J}_F(u_n)(\widehat{u}_n-u_n)]=o_{\mathbb{P}}(1). T n [ F ( u n ) − F ( u n ) − J F ( u n ) ( u n − u n )] = o P ( 1 ) . Writing
Σ u : = ( 2 0 0 8 β 2 ) , \Sigma_u:=\begin{pmatrix}2&0\\0&8\beta^2\end{pmatrix}, Σ u := ( 2 0 0 8 β 2 ) , the limiting covariance under the normalization T n T_n T n
A Σ u A T = ( 1 2 β β 2 β 2 ) . A\Sigma_u A^{\mathsf{T}}=\begin{pmatrix}\frac{1}{2}&\beta\\\beta&2\beta^2\end{pmatrix}. A Σ u A T = ( 2 1 β β 2 β 2 ) . Since 2 u 1 , n ∼ 1 / α n 2u_{1,n}\sim1/\alpha_n 2 u 1 , n ∼ 1/ α n
\begin{pmatrix}\sqrt{n}\dfrac{\widetilde{\alpha}_n-\alpha_n}{\alpha_n}\\\sqrt{n}(\widetilde{\beta}_n-\beta)\end{pmatrix}\xrightarrow{d}\mathcal{N}\left(0,\begin{pmatrix}2&2\beta\\2\beta&2\beta^2\end{pmatrix}\right)
\quad\text{and}\quad
\begin{pmatrix}\sqrt{n}\dfrac{\widehat{\alpha}_n-\alpha_n}{\alpha_n}\\\sqrt{n}(\widehat{\beta}_n-\beta)\end{pmatrix}\xrightarrow{d}\mathcal{N}\left(0,\begin{pmatrix}2&2\beta\\2\beta&2\beta^2\end{pmatrix}\right). \tag*{(27)}\end{pmatrix}
Remark. The common covariance matrix in the asymptotic distributions in (27) has rank one because
n ( β ~ n − β ) = β n α n ( α ~ n − α n ) + o P ( 1 ) . \sqrt{n}(\widetilde{\beta}_n-\beta)=\beta\frac{\sqrt{n}}{\alpha_n}(\widetilde{\alpha}_n-\alpha_n)+o_{\mathbb{P}}(1). n ( β n − β ) = β α n n ( α n − α n ) + o P ( 1 ) . This singular limit does not indicate nonidentifiability. It merely arises because under the normalization T n T_n T n
Gamma Regression
Gamma regression extends the preceding example to non-identically distributed observations. For convenience, we turn to the mean–dispersion parameterization of the gamma distribution, whose density for y > 0 y > 0 y > 0
f μ , φ ( y ) : = y 1 / φ − 1 exp ( − y μ φ ) Γ ( 1 φ ) ( μ φ ) 1 / φ . (28) f_{\mu,\varphi}(y) := \frac{y^{1/\varphi-1}\exp\left(-\frac{y}{\mu\varphi}\right)}{\Gamma\left(\frac{1}{\varphi}\right)(\mu\varphi)^{1/\varphi}}. \tag*{(28)} f μ , φ ( y ) := Γ ( φ 1 ) ( μ φ ) 1/ φ y 1/ φ − 1 exp ( − μ φ y ) . (28) Let k : = p + 1 k := p + 1 k := p + 1 X n ∈ R n × k \mathbf{X}_n \in\mathbb{R}^{n \times k} X n ∈ R n × k x n , i T \mathbf{x}_{n,i}^{\mathsf{T}} x n , i T 1 n \mathbf{1}_n 1 n
sup n ≥ 1 max 1 ≤ i ≤ n ∥ x n , i ∥ < ∞ and 1 n X n T X n ⟶ S , (29) \sup_{n \ge1}\max_{1 \le i \le n}\lVert\mathbf{x}_{n,i}\rVert< \infty
\quad\text{and}\quad
\frac{1}{n}\mathbf{X}_n^{\mathsf{T}}\mathbf{X}_n \longrightarrow\mathbf{S}, \tag*{(29)} n ≥ 1 sup 1 ≤ i ≤ n max ∥ x n , i ∥ < ∞ and n 1 X n T X n ⟶ S , (29) where S \mathbf{S} S β 0 ∈ R k \boldsymbol{\beta}_0 \in\mathbb{R}^k β 0 ∈ R k Y n : = ( Y n , 1 , … , Y n , n ) T \mathbf{Y}_n := (Y_{n,1},\ldots,Y_{n,n})^{\mathsf{T}} Y n := ( Y n , 1 , … , Y n , n ) T Y n , 1 , … , Y n , n Y_{n,1},\ldots,Y_{n,n} Y n , 1 , … , Y n , n
μ n , i : = exp ( x n , i T β 0 ) \mu_{n,i} := \exp(\mathbf{x}_{n,i}^{\mathsf{T}}\boldsymbol{\beta}_0) μ n , i := exp ( x n , i T β 0 ) and common dispersion φ n ↓ 0 \varphi_n \downarrow0 φ n ↓ 0
Z n , i : = Y n , i exp ( − x n , i T β 0 ) ∼ i i d Gamma ( 1 , φ n ) , Z_{n,i} := Y_{n,i}\exp(-\mathbf{x}_{n,i}^{\mathsf{T}}\boldsymbol{\beta}_0) \mathrel{\overset{\mathrm{iid}}{\sim}} \operatorname{Gamma}(1,\varphi_n), Z n , i := Y n , i exp ( − x n , i T β 0 ) ∼ iid Gamma ( 1 , φ n ) , where the notation on the right refers to the parameterization in (28) .
The limiting regime φ n ↓ 0 \varphi_n \downarrow0 φ n ↓ 0 [27 ] , ν : = φ − 1 \nu:= \varphi^{-1} ν := φ − 1 φ \varphi φ φ \varphi φ β \boldsymbol{\beta} β [27 Sections 8.2 and 8.3.4] . Below, we show that the least squares estimator based on the log responses, paired with an explicit estimator of φ \varphi φ β \boldsymbol{\beta} β
Derivation of the Approximate MLE
For a candidate β ∈ R k \boldsymbol{\beta} \in\mathbb{R}^k β ∈ R k
Z n , i ( β ) : = Y n , i exp ( − x n , i T β ) and Z n ( β ) : = ( Z n , 1 ( β ) , … , Z n , n ( β ) ) T and Z n : = Z n ( β 0 ) , Z_{n,i}(\boldsymbol{\beta}) := Y_{n,i}\exp(-\mathbf{x}_{n,i}^{\mathsf{T}}\boldsymbol{\beta})
\quad\text{and}\quad
\mathbf{Z}_n(\boldsymbol{\beta}) := (Z_{n,1}(\boldsymbol{\beta}),\ldots,Z_{n,n}(\boldsymbol{\beta}))^{\mathsf{T}}
\quad\text{and}\quad
\mathbf{Z}_n := \mathbf{Z}_n(\boldsymbol{\beta}_0), Z n , i ( β ) := Y n , i exp ( − x n , i T β ) and Z n ( β ) := ( Z n , 1 ( β ) , … , Z n , n ( β ) ) T and Z n := Z n ( β 0 ) , and let
h ( z ) : = z − 1 − log ( z ) and g ( φ ) : = − log ( φ ) − ψ ( 1 φ ) . h(z) := z - 1 - \log(z)
\quad\text{and}\quad
g(\varphi) := -\log(\varphi) - \psi\left(\frac{1}{\varphi}\right). h ( z ) := z − 1 − log ( z ) and g ( φ ) := − log ( φ ) − ψ ( φ 1 ) . Differentiating the log likelihood and multiplying the two score equations by nonzero constant factors gives the equivalent system
q n ( φ , β ) : = ( g ( φ ) − 1 n ∑ i = 1 n h ( Z n , i ( β ) ) φ 2 n X n T ( Z n ( β ) − 1 n ) ) = 0. (30) \mathbf{q}_n(\varphi,\boldsymbol{\beta}) :=
\begin{pmatrix}
g(\varphi) - \dfrac{1}{n}\sum_{i=1}^{n} h(Z_{n,i}(\boldsymbol{\beta})) \\
\dfrac{\varphi}{2n}\mathbf{X}_n^{\mathsf{T}}\left(\mathbf{Z}_n(\boldsymbol{\beta})-\mathbf{1}_n\right)
\end{pmatrix}
= \mathbf{0}. \tag*{(30)} q n ( φ , β ) := g ( φ ) − n 1 ∑ i = 1 n h ( Z n , i ( β )) 2 n φ X n T ( Z n ( β ) − 1 n ) = 0 . (30) Define the log-response vector and the log residuals by
L n : = ( log ( Y n , 1 ) , … , log ( Y n , n ) ) T and r n , i ( β ) : = log ( Y n , i ) − x n , i T β . \mathbf{L}_n := (\log(Y_{n,1}),\ldots,\log(Y_{n,n}))^{\mathsf{T}}
\quad\text{and}\quad
r_{n,i}(\boldsymbol{\beta}) := \log(Y_{n,i})-\mathbf{x}_{n,i}^{\mathsf{T}}\boldsymbol{\beta}. L n := ( log ( Y n , 1 ) , … , log ( Y n , n ) ) T and r n , i ( β ) := log ( Y n , i ) − x n , i T β . For each fixed i i i
r n , i ( β 0 ) = log ( Z n , i ) ⟶ p 0 r_{n,i}(\beta_0)=\log(Z_{n,i}) \mathrel{\overset{p}{\longrightarrow}} 0 r n , i ( β 0 ) = log ( Z n , i ) ⟶ p 0 under the limiting regime. The expansions
g ( φ ) = φ 2 + O ( φ 2 ) and h ( e r ) = r 2 2 + O ( ∣ r ∣ 3 ) and e r − 1 = r + O ( r 2 ) g(\varphi)=\frac{\varphi}{2}+O(\varphi^2) \quad\text{and} \quad h(e^r)=\frac{r^2}{2}+O(\lvert r\rvert^3) \quad\text{and} \quad e^r-1=r+O(r^2) g ( φ ) = 2 φ + O ( φ 2 ) and h ( e r ) = 2 r 2 + O (∣ r ∣ 3 ) and e r − 1 = r + O ( r 2 ) therefore suggest the surrogate equations
X n T ( L n − X n β ) = 0 and φ = 1 n ∥ L n − X n β ∥ 2 . X_n^{\mathsf{T}}(L_n-X_n\beta)=0 \quad\text{and} \quad\varphi=\frac{1}{n}\lVert L_n-X_n\beta\rVert^2. X n T ( L n − X n β ) = 0 and φ = n 1 ∥ L n − X n β ∥ 2 . For sufficiently large n n n X n T X n X_n^{\mathsf{T}}X_n X n T X n
H n : = X n ( X n T X n ) − 1 X n T and M n : = I n − H n . H_n:=X_n(X_n^{\mathsf{T}}X_n)^{-1}X_n^{\mathsf{T}} \quad\text{and} \quad M_n:=I_n-H_n. H n := X n ( X n T X n ) − 1 X n T and M n := I n − H n . Solving the surrogate equations gives the candidate estimators
β ~ n : = ( X n T X n ) − 1 X n T L n and φ ~ n : = 1 n ∥ M n L n ∥ 2 . (31) \widetilde{\beta}_n:=(X_n^{\mathsf{T}}X_n)^{-1}X_n^{\mathsf{T}}L_n \quad\text{and} \quad\widetilde{\varphi}_n:=\frac{1}{n}\lVert M_nL_n\rVert^2. \tag*{(31)} β n := ( X n T X n ) − 1 X n T L n and φ n := n 1 ∥ M n L n ∥ 2 . (31) Thus β ~ n \widetilde{\beta}_n β n φ ~ n \widetilde{\varphi}_n φ n n n n n > k n>k n > k L n L_n L n φ ~ n > 0 \widetilde{\varphi}_n>0 φ n > 0
Remark. The estimator φ ~ n \widetilde{\varphi}_n φ n [27 ] . Define
D n ( β ) : = 2 ∑ i = 1 n h ( Z n , i ( β ) ) = − 2 ∑ i = 1 n [ log ( Y n , i μ n , i ( β ) ) − Y n , i − μ n , i ( β ) μ n , i ( β ) ] . D_n(\beta):=2\sum_{i=1}^{n}h\left(Z_{n,i}(\beta)\right)=-2\sum_{i=1}^{n}\left[\log\left(\frac{Y_{n,i}}{\mu_{n,i}(\beta)}\right)-\frac{Y_{n,i}-\mu_{n,i}(\beta)}{\mu_{n,i}(\beta)}\right]. D n ( β ) := 2 i = 1 ∑ n h ( Z n , i ( β ) ) = − 2 i = 1 ∑ n [ log ( μ n , i ( β ) Y n , i ) − μ n , i ( β ) Y n , i − μ n , i ( β ) ] . If ν ^ n = 1 / φ ^ n \widehat{\nu}_n=1/\widehat{\varphi}_n ν n = 1/ φ n (30) , evaluated at the MLE, becomes
2 n ( log ( ν ^ n ) − ψ ( ν ^ n ) ) = D n ( β ^ n ) , 2n\left(\log(\widehat{\nu}_n)-\psi(\widehat{\nu}_n)\right)=D_n(\widehat{\beta}_n), 2 n ( log ( ν n ) − ψ ( ν n ) ) = D n ( β n ) , which is the likelihood equation given by McCullagh and Nelder [ [27 ] , Equation (8.1). Their large precision approximation has leading term φ ^ n ≈ ( 1 / n ) D n ( β ^ n ) \widehat{\varphi}_n\approx(1/n)D_n(\widehat{\beta}_n) φ n ≈ ( 1/ n ) D n ( β n ) Z n , i ( β ) = exp ( r n , i ( β ) ) Z_{n,i}(\beta)=\exp(r_{n,i}(\beta)) Z n , i ( β ) = exp ( r n , i ( β )) 2 h ( e r ) = r 2 + O ( ∣ r ∣ 3 ) 2h(e^r)=r^2+O(\lvert r\rvert^3) 2 h ( e r ) = r 2 + O (∣ r ∣ 3 ) ( 1 / n ) D n ( β ) (1/n)D_n(\beta) ( 1/ n ) D n ( β ) β = β ~ n \beta=\widetilde{\beta}_n β = β n
1 n ∑ i = 1 n r n , i ( β ~ n ) 2 = φ ~ n . \frac{1}{n}\sum_{i=1}^{n}r_{n,i}(\widetilde{\beta}_n)^2=\widetilde{\varphi}_n. n 1 i = 1 ∑ n r n , i ( β n ) 2 = φ n . Thus we obtain φ ~ n \widetilde{\varphi}_n φ n β \beta β β ~ n \widetilde{\beta}_n β n
We use the vanishing chart
ϕ ( φ , β ) : = ( φ / 2 ( φ / 2 ) β ) and u : = ϕ ( φ , β ) = ( u 1 u 2 ) , \phi(\varphi,\beta):=\begin{pmatrix}\varphi/2\\(\varphi/2)\beta\end{pmatrix}
\quad\text{and} \quad
u:=\phi(\varphi,\beta)=\begin{pmatrix}u_1\\u_2\end{pmatrix}, ϕ ( φ , β ) := ( φ /2 ( φ /2 ) β ) and u := ϕ ( φ , β ) = ( u 1 u 2 ) , which is a bijection from N : = ( 0 , ∞ ) × R k N:=(0,\infty)\times\mathbb{R}^k N := ( 0 , ∞ ) × R k U : = ( 0 , ∞ ) × R k U:=(0,\infty)\times\mathbb{R}^k U := ( 0 , ∞ ) × R k
ϕ − 1 ( u ) = ( 2 u 1 u 2 / u 1 ) . (32) \phi^{-1}(u)=\begin{pmatrix}2u_1\\u_2/u_1\end{pmatrix}. \tag*{(32)} ϕ − 1 ( u ) = ( 2 u 1 u 2 / u 1 ) . (32) For fixed β 0 \beta_0 β 0 ϕ ( φ n , β 0 ) ⟶ 0 \phi(\varphi_n,\beta_0)\longrightarrow\mathbf{0} ϕ ( φ n , β 0 ) ⟶ 0 u = ( u 1 , u 2 ) T \mathbf{u}=(u_1,u_2)^{\mathsf T} u = ( u 1 , u 2 ) T
r n ( u ) : = L n − X n u 2 u 1 , r_n(\mathbf{u}):=L_n-X_n\frac{u_2}{u_1}, r n ( u ) := L n − X n u 1 u 2 , and let r n , i ( u ) r_{n,i}(\mathbf{u}) r n , i ( u ) i i i (30) in chart coordinates is
Q n ( u ) : = ( g ( 2 u 1 ) − 1 n ∑ i = 1 n h ( exp ( r n , i ( u ) ) ) u 1 n X n T [ Z n ( u 2 u 1 ) − 1 n ] ) . (33) Q_n(\mathbf{u}):=\begin{pmatrix}
g(2u_1)-\frac{1}{n}\sum_{i=1}^{n}h\bigl(\exp(r_{n,i}(\mathbf{u}))\bigr)\\
\frac{u_1}{n}X_n^{\mathsf T}\left[Z_n\left(\frac{u_2}{u_1}\right)-\mathbf{1}_n\right]
\end{pmatrix}. \tag*{(33)} Q n ( u ) := ( g ( 2 u 1 ) − n 1 ∑ i = 1 n h ( exp ( r n , i ( u )) ) n u 1 X n T [ Z n ( u 1 u 2 ) − 1 n ] ) . (33) The following surrogate for Q n Q_n Q n
Q ~ n ( u ) : = ( u 1 − u ~ 1 , n 1 n X n T ( u 1 L n − X n u 2 ) ) and u ~ 1 , n : = φ ~ n 2 . (34) \widetilde{Q}_n(\mathbf{u}):=\begin{pmatrix}
u_1-\widetilde{u}_{1,n}\\
\frac{1}{n}X_n^{\mathsf T}(u_1L_n-X_nu_2)
\end{pmatrix}
\quad\text{and}\quad
\widetilde{u}_{1,n}:=\frac{\widetilde{\varphi}_n}{2}. \tag*{(34)} Q n ( u ) := ( u 1 − u 1 , n n 1 X n T ( u 1 L n − X n u 2 ) ) and u 1 , n := 2 φ n . (34) whose unique root is
u ~ n : = ( u ~ 1 , n u ~ 1 , n β ~ n ) , \widetilde{\mathbf{u}}_n:=\begin{pmatrix}\widetilde{u}_{1,n}\\ \widetilde{u}_{1,n}\widetilde{\beta}_n\end{pmatrix}, u n := ( u 1 , n u 1 , n β n ) , which belongs to U U U n n n
Verification of Approximation
For notational convenience, write
θ n : = ( φ n , β 0 T ) T and θ ^ n : = ( φ ^ n , β ^ n T ) T and θ ~ n : = ( φ ~ n , β ~ n T ) T , \boldsymbol{\theta}_n:=(\varphi_n,\beta_0^{\mathsf T})^{\mathsf T}
\quad\text{and}\quad
\widehat{\boldsymbol{\theta}}_n:=(\widehat{\varphi}_n,\widehat{\beta}_n^{\mathsf T})^{\mathsf T}
\quad\text{and}\quad
\widetilde{\boldsymbol{\theta}}_n:=(\widetilde{\varphi}_n,\widetilde{\beta}_n^{\mathsf T})^{\mathsf T}, θ n := ( φ n , β 0 T ) T and θ n := ( φ n , β n T ) T and θ n := ( φ n , β n T ) T , and let
u n : = ϕ ( θ n ) and u ^ n : = ϕ ( θ ^ n ) and u ~ n : = ϕ ( θ ~ n ) . \mathbf{u}_n:=\phi(\boldsymbol{\theta}_n)
\quad\text{and}\quad
\widehat{\mathbf{u}}_n:=\phi(\widehat{\boldsymbol{\theta}}_n)
\quad\text{and}\quad
\widetilde{\mathbf{u}}_n:=\phi(\widetilde{\boldsymbol{\theta}}_n). u n := ϕ ( θ n ) and u n := ϕ ( θ n ) and u n := ϕ ( θ n ) . Write P n : = P n , θ n \mathbb{P}_n:=\mathbb{P}_{n,\boldsymbol{\theta}_n} P n := P n , θ n
u n : = ( u 1 , n u 2 , n ) = ( φ n 2 φ n 2 β 0 ) . \mathbf{u}_n:=\begin{pmatrix}u_{1,n}\\u_{2,n}\end{pmatrix}
=\begin{pmatrix}\frac{\varphi_n}{2}\\\frac{\varphi_n}{2}\beta_0\end{pmatrix}. u n := ( u 1 , n u 2 , n ) = ( 2 φ n 2 φ n β 0 ) . By Lemma A.10 , the likelihood has a unique maximizer with probability tending to one; the required localization and equation error bounds are established in Lemma A.11. In particular, there exist deterministic convex sets D n D_n D n
P n ( u ^ n , u ~ n ∈ D n ) ⟶ 1 and sup u ∈ D n ∥ u ∥ = O ( φ n ) ⟶ 0 , (35) \mathbb{P}_n(\widehat{\mathbf{u}}_n,\widetilde{\mathbf{u}}_n\in D_n)\longrightarrow1
\quad\text{and}\quad
\sup_{\mathbf{u}\in D_n}\|\mathbf{u}\|=O(\varphi_n)\longrightarrow0, \tag*{(35)} P n ( u n , u n ∈ D n ) ⟶ 1 and u ∈ D n sup ∥ u ∥ = O ( φ n ) ⟶ 0 , (35) and
Δ D , n : = sup u ∈ D n ∥ Q ~ n ( u ) − Q n ( u ) ∥ = o P ( φ n ) . (36) \Delta_{D,n}:=\sup_{\mathbf{u}\in D_n}\|\widetilde{Q}_n(\mathbf{u})-Q_n(\mathbf{u})\|=o_{\mathbb{P}}(\varphi_n). \tag*{(36)} Δ D , n := u ∈ D n sup ∥ Q n ( u ) − Q n ( u ) ∥ = o P ( φ n ) . (36) The surrogate has constant Jacobian
J n : = J Q ~ n = ( 1 0 T 1 n X n T L n − 1 n X n T X n ) , J_n:=J\widetilde{Q}_n=\begin{pmatrix}
1 & \mathbf{0}^{\mathsf T}\\
\frac{1}{n}X_n^{\mathsf T}L_n & -\frac{1}{n}X_n^{\mathsf T}X_n
\end{pmatrix}, J n := J Q n = ( 1 n 1 X n T L n 0 T − n 1 X n T X n ) , whose inverse is computed via block matrix inversion as
J n − 1 = ( 1 0 T β ~ n − n ( X n T X n ) − 1 ) . \mathbf{J}_n^{-1} =
\begin{pmatrix}
1 & \mathbf{0}^{\mathsf{T}} \\
\widetilde{\boldsymbol{\beta}}_n & -n(\mathbf{X}_n^{\mathsf{T}}\mathbf{X}_n)^{-1}
\end{pmatrix}. J n − 1 = ( 1 β n 0 T − n ( X n T X n ) − 1 ) . Since β ~ n ⟶ p β 0 \widetilde{\boldsymbol{\beta}}_n \overset{p}{\longrightarrow} \boldsymbol{\beta}_0 β n ⟶ p β 0 n − 1 X n T X n ⟶ S n^{-1}\mathbf{X}_n^{\mathsf{T}}\mathbf{X}_n \longrightarrow\mathbf{S} n − 1 X n T X n ⟶ S ∥ J n − 1 ∥ = O P ( 1 ) \lVert\mathbf{J}_n^{-1}\rVert= O_{\mathsf{P}}(1) ∥ J n − 1 ∥ = O P ( 1 ) a > 0 a > 0 a > 0
P n ( σ min ( J n ) ≥ a ) ⟶ 1. \mathbb{P}_n\bigl(\sigma_{\min}(\mathbf{J}_n) \ge a\bigr) \longrightarrow1. P n ( σ m i n ( J n ) ≥ a ) ⟶ 1. Moreover, because the surrogate Jacobian is identically equal to J n \mathbf{J}_n J n ρ n = 1 / φ n \rho_n = 1/\varphi_n ρ n = 1/ φ n (35) and (36) give
1 φ n ∥ u ~ n − u ^ n ∥ = o P ( 1 ) . (37) \frac{1}{\varphi_n}\lVert\widetilde{\mathbf{u}}_n-\widehat{\mathbf{u}}_n\rVert=o_{\mathsf{P}}(1). \tag*{(37)} φ n 1 ∥ u n − u n ∥ = o P ( 1 ) . (37) Thus u ~ n \widetilde{\mathbf{u}}_n u n u ^ n \widehat{\mathbf{u}}_n u n 1 / φ n 1/\varphi_n 1/ φ n
It remains to translate this comparison back to the original parameter scale. On D n D_n D n
J ϕ − 1 ( u ) = ( 2 0 T − u 2 u 1 2 I k u 1 ) . \mathbf{J}_{\phi^{-1}}(\mathbf{u}) =
\begin{pmatrix}
2 & \mathbf{0}^{\mathsf{T}} \\
-\dfrac{u_2}{u_1^2} & \dfrac{\mathbf{I}_k}{u_1}
\end{pmatrix}. J ϕ − 1 ( u ) = 2 − u 1 2 u 2 0 T u 1 I k . and the construction of D n D_n D n u 1 ≍ φ n u_1 \asymp\varphi_n u 1 ≍ φ n u 2 / u 1 u_2/u_1 u 2 / u 1 D n D_n D n
sup u ∈ D n ∥ J ϕ − 1 ( u ) ∥ = O ( 1 φ n ) , \sup_{\mathbf{u}\in D_n}\lVert\mathbf{J}_{\phi^{-1}}(\mathbf{u})\rVert
=O\left(\frac{1}{\varphi_n}\right), u ∈ D n sup ∥ J ϕ − 1 ( u )∥ = O ( φ n 1 ) , so ϕ − 1 \phi^{-1} ϕ − 1 b n b_n b n D n D_n D n b n : = C / φ n b_n := C/\varphi_n b n := C / φ n
Δ D , n = o P ( φ n ) = o P ( a b n ) , \Delta_{D,n}=o_{\mathsf{P}}(\varphi_n)=o_{\mathsf{P}}\left(\frac{a}{b_n}\right), Δ D , n = o P ( φ n ) = o P ( b n a ) , Corollary 3 yields
∥ θ ~ n − θ ^ n ∥ = o P ( 1 ) . \lVert\widetilde{\boldsymbol{\theta}}_n-\widehat{\boldsymbol{\theta}}_n\rVert=o_{\mathsf{P}}(1). ∥ θ n − θ n ∥ = o P ( 1 ) . Together with (37) , we obtain the explicit componentwise differences
∣ φ ~ n − φ ^ n ∣ = o P ( φ n ) and ∥ β ~ n − β ^ n ∥ = o P ( 1 ) . |\widetilde{\varphi}_n-\widehat{\varphi}_n|=o_{\mathsf{P}}(\varphi_n)
\quad\text{and}\quad
\lVert\widetilde{\boldsymbol{\beta}}_n-\widehat{\boldsymbol{\beta}}_n\rVert=o_{\mathsf{P}}(1). ∣ φ n − φ n ∣ = o P ( φ n ) and ∥ β n − β n ∥ = o P ( 1 ) . The estimators in (31) are therefore approximate MLEs both relative to the chart and on the original parameter scale.
Joint Asymptotic Distribution
The approximate estimator of β \boldsymbol{\beta} β (31) can have a first-order bias in its first component, which corresponds to the intercept. To see this, define e 0 : = ( 1 , 0 , … , 0 ) T ∈ R k \mathbf{e}_0 := (1,0,\ldots,0)^{\mathsf{T}}\in\mathbb{R}^k e 0 := ( 1 , 0 , … , 0 ) T ∈ R k
m n : = E [ log ( Z n , i ) ] = ψ ( 1 φ n ) + log ( φ n ) and v n : = Var ( log ( Z n , i ) ) = ψ 1 ( 1 φ n ) . m_n:=\mathbb{E}[\log(Z_{n,i})]=\psi\left(\frac{1}{\varphi_n}\right)+\log(\varphi_n)
\quad\text{and}\quad
v_n:=\operatorname{Var}(\log(Z_{n,i}))=\psi_1\left(\frac{1}{\varphi_n}\right). m n := E [ log ( Z n , i )] = ψ ( φ n 1 ) + log ( φ n ) and v n := Var ( log ( Z n , i )) = ψ 1 ( φ n 1 ) . and set
c n : = ( 1 − k n ) v n . c_n:=\left(1-\frac{k}{n}\right)v_n. c n := ( 1 − n k ) v n . Since the first column of X n \mathbf{X}_n X n 1 n \mathbf{1}_n 1 n M n \mathbf{M}_n M n n − k n-k n − k
E [ β ~ n ] = β 0 + m n e 0 and E [ φ ~ n ] = c n , \mathbb{E}[\widetilde{\boldsymbol{\beta}}_n]=\boldsymbol{\beta}_0+m_n\mathbf{e}_0
\quad\text{and}\quad
\mathbb{E}[\widetilde{\varphi}_n]=c_n, E [ β n ] = β 0 + m n e 0 and E [ φ n ] = c n , so m n e 0 m_n e_0 m n e 0 c n − φ n c_n-\varphi_n c n − φ n β ~ n \widetilde{\beta}_n β n φ ~ n \widetilde{\varphi}_n φ n
The joint limits proved in Lemma A.12 are
\begin{pmatrix}
\frac{\sqrt{n}}{\varphi_n}(\widetilde{\varphi}_n-c_n) \\
\sqrt{\frac{n}{\varphi_n}}(\widetilde{\beta}_n-\beta_0-m_n e_0)
\end{pmatrix}
\xrightarrow{d}
\mathcal{N}_{k+1}\left(0,
\begin{pmatrix}
2 & \mathbf{0}^{\mathsf{T}} \\
\mathbf{0} & \mathbf{S}^{-1}
\end{pmatrix}
\right) \tag*{(38)}\end{pmatrix}
and
\begin{pmatrix}
\frac{\sqrt{n}}{\varphi_n}(\widehat{\varphi}_n-\varphi_n) \\
\sqrt{\frac{n}{\varphi_n}}(\widehat{\beta}_n-\beta_0)
\end{pmatrix}
\xrightarrow{d}
\mathcal{N}_{k+1}\left(0,
\begin{pmatrix}
2 & \mathbf{0}^{\mathsf{T}} \\
\mathbf{0} & \mathbf{S}^{-1}
\end{pmatrix}
\right). \tag*{(39)}\end{pmatrix}
Thus the dispersion error has order φ n / n \varphi_n/\sqrt{n} φ n / n φ n / n \sqrt{\varphi_n/n} φ n / n
Cov ( Z n , i − 1 , h ( Z n , i ) ) = 0. \operatorname{Cov}(Z_{n,i}-1,h(Z_{n,i}))=0. Cov ( Z n , i − 1 , h ( Z n , i )) = 0. Note that the deterministic centering terms c n c_n c n β 0 + m n e 0 \beta_0+m_n e_0 β 0 + m n e 0 (38) cannot always be replaced by the true parameters. Indeed,
m n = − φ n 2 + O ( φ n 2 ) and c n − φ n = φ n 2 2 + O ( φ n 3 + φ n n ) . m_n=-\frac{\varphi_n}{2}+O(\varphi_n^2)
\quad\text{and}\quad
c_n-\varphi_n=\frac{\varphi_n^2}{2}+O\left(\varphi_n^3+\frac{\varphi_n}{n}\right). m n = − 2 φ n + O ( φ n 2 ) and c n − φ n = 2 φ n 2 + O ( φ n 3 + n φ n ) . If n φ n → λ ∈ [ 0 , ∞ ) n\varphi_n\to\lambda\in[0,\infty) n φ n → λ ∈ [ 0 , ∞ ) (38) becomes
\begin{pmatrix}
\frac{\sqrt{n}}{\varphi_n}(\widetilde{\varphi}_n-\varphi_n) \\
\sqrt{\frac{n}{\varphi_n}}(\widetilde{\beta}_n-\beta_0)
\end{pmatrix}
\xrightarrow{d}
\mathcal{N}_{k+1}\left(
\begin{pmatrix}
0 \\
-\sqrt{\lambda}\,e_0/2
\end{pmatrix},
\begin{pmatrix}
2 & \mathbf{0}^{\mathsf{T}} \\
\mathbf{0} & \mathbf{S}^{-1}
\end{pmatrix}
\right). \tag*{(40)}\end{pmatrix}
When n φ n → 0 n\varphi_n\to0 n φ n → 0 n φ n → λ > 0 n\varphi_n\to\lambda>0 n φ n → λ > 0 (40) . If n φ n → ∞ n\varphi_n\to\infty n φ n → ∞
Removing the leading bias is useful because it relaxes the sufficient condition for the explicit and exact estimators to have the same first-order distribution from n φ n → 0 n\varphi_n\to0 n φ n → 0 n φ n 4 → 0 n\varphi_n^4\to0 n φ n 4 → 0
E [ φ ~ n ] = ( 1 − k n ) v n , \mathbb{E}[\widetilde{\varphi}_n]=\left(1-\frac{k}{n}\right)v_n, E [ φ n ] = ( 1 − n k ) v n , first define
v ˇ n : = φ ~ n 1 − k / n , \check{v}_n:=\frac{\widetilde{\varphi}_n}{1-k/n}, v ˇ n := 1 − k / n φ n , which corrects φ ~ n \widetilde{\varphi}_n φ n
φ n = v n − 1 2 v n 2 + O ( v n 3 ) and − m n = 1 2 v n − 1 6 v n 2 + O ( v n 3 ) , \varphi_n=v_n-\frac{1}{2}v_n^2+O(v_n^3)
\quad\text{and}\quad
-m_n=\frac{1}{2}v_n-\frac{1}{6}v_n^2+O(v_n^3), φ n = v n − 2 1 v n 2 + O ( v n 3 ) and − m n = 2 1 v n − 6 1 v n 2 + O ( v n 3 ) , which lead to the corrections
φ ~ n b c : = v ˇ n − 1 2 v ˇ n 2 and β ~ n b c : = β ~ n + ( 1 2 v ˇ n − 1 6 v ˇ n 2 ) e 0 . (41) \widetilde{\varphi}_n^{\mathrm{bc}}:=\check{v}_n-\frac{1}{2}\check{v}_n^2
\quad\text{and}\quad
\widetilde{\beta}_n^{\mathrm{bc}}:=\widetilde{\beta}_n+
\left(\frac{1}{2}\check{v}_n-\frac{1}{6}\check{v}_n^2\right)e_0. \tag*{(41)} φ n bc := v ˇ n − 2 1 v ˇ n 2 and β n bc := β n + ( 2 1 v ˇ n − 6 1 v ˇ n 2 ) e 0 . (41) Only the intercept is corrected because the common mean m n m_n m n v ˇ n → p 0 \check{v}_n \xrightarrow{p} 0 v ˇ n p 0 § 2.1 . For the joint limit of these bias-corrected estimators, if n φ n 4 ⟶ 0 n\varphi_n^4 \longrightarrow0 n φ n 4 ⟶ 0 n φ n 2 ⟶ 0 \sqrt{n}\varphi_n^2 \longrightarrow0 n φ n 2 ⟶ 0
\begin{pmatrix}
\frac{\sqrt{n}}{\varphi_n}(\widetilde{\varphi}^{\mathrm{bc}}_n-\varphi_n) \\
\sqrt{\frac{n}{\varphi_n}}(\widetilde{\boldsymbol{\beta}}^{\mathrm{bc}}_n-\boldsymbol{\beta}_0)
\end{pmatrix}
\xrightarrow{d}
\mathcal{N}_{k+1}\left(
\boldsymbol{0},
\begin{pmatrix}
2 & \boldsymbol{0}^{\mathsf{T}} \\
\boldsymbol{0} & S^{-1}
\end{pmatrix}
\right). \tag*{(42)}\end{pmatrix}
In fact, the corrected explicit estimator and the exact MLE are jointly asymptotically equivalent:
\begin{pmatrix}
\frac{\sqrt{n}}{\varphi_n}(\widetilde{\varphi}^{\mathrm{bc}}_n-\widehat{\varphi}_n) \\
\sqrt{\frac{n}{\varphi_n}}(\widetilde{\boldsymbol{\beta}}^{\mathrm{bc}}_n-\widehat{\boldsymbol{\beta}}_n)
\end{pmatrix}
=o_P(1). \tag*{(43)}\end{pmatrix}
Since v ˇ n = O P ( φ n ) \check{v}_n=O_P(\varphi_n) v ˇ n = O P ( φ n ) o P ( 1 ) o_P(1) o P ( 1 ) o P ( φ n ) o_P(\varphi_n) o P ( φ n ) φ ~ n b c \widetilde{\varphi}^{\mathrm{bc}}_n φ n bc β ~ n b c \widetilde{\boldsymbol{\beta}}^{\mathrm{bc}}_n β n bc n φ n 4 ⟶ 0 n\varphi_n^4 \longrightarrow0 n φ n 4 ⟶ 0 (39) .
Simulation Study
We compared the three gamma models under the same simulation design. Let a n : = n γ a_n:=n^\gamma a n := n γ
γ ∈ { 0.1 , 0.4 , 0.7 } and n ∈ { 250 , 500 , 1000 , 2500 , 5000 , 10000 , 25000 } . \gamma\in\{0.1,0.4,0.7\}\quad\text{and}\quad n\in\{250,500,1000,2500,5000,10000,25000\}. γ ∈ { 0.1 , 0.4 , 0.7 } and n ∈ { 250 , 500 , 1000 , 2500 , 5000 , 10000 , 25000 } . For the two iid models of § 3.1 and § 3.2 , we set α n : = a n \alpha_n:=a_n α n := a n β : = 2 \beta:=2 β := 2 § 3.3 , we set φ n : = a n − 1 \varphi_n:=a_n^{-1} φ n := a n − 1 x n , i : = ( 1 , x n , i ) T \boldsymbol{x}_{n,i}:=(1,x_{n,i})^{\mathsf{T}} x n , i := ( 1 , x n , i ) T x n , i x_{n,i} x n , i [ − 1 , 1 ] [-1,1] [ − 1 , 1 ] β 0 : = ( 0.5 , − 0.75 ) T \boldsymbol{\beta}_0:=(0.5,-0.75)^{\mathsf{T}} β 0 := ( 0.5 , − 0.75 ) T γ > 1 / 3 \gamma>1/3 γ > 1/3 γ > 1 / 6 \gamma>1/6 γ > 1/6 γ > 1 / 4 \gamma>1/4 γ > 1/4
In each replication, we computed the exact MLE using Newton’s method and the corresponding approximate estimators from § 3.1 to § 3.3 . We divided the absolute difference between an approximate estimator and the exact MLE by the asymptotic standard error of the corresponding MLE coordinate, which compared the approximation error with sampling variation. We also divided the RMSE of the approximate estimator by that of the exact MLE to measure any finite-sample loss in accuracy.
Figure 1 shows a clear transition across the rate conditions. In the known-rate model, the normalized discrepancy increases with n n n γ = 0.1 \gamma=0.1 γ = 0.1 γ = 0.4 \gamma=0.4 γ = 0.4 0.7 0.7 0.7 n = 25000 n=25000 n = 25000 γ = 0.1 \gamma=0.1 γ = 0.1 γ = 0.4 \gamma=0.4 γ = 0.4 1.59 × 10 − 4 1.59\times10^{-4} 1.59 × 1 0 − 4 γ = 0.7 \gamma=0.7 γ = 0.7 γ = 0.1 \gamma=0.1 γ = 0.1 1 / 6 1/6 1/6 n = 25000 n=25000 n = 25000 γ = 0.7 \gamma=0.7 γ = 0.7 γ > 1 / 4 \gamma>1/4 γ > 1/4 n n n
Figure 1. Median absolute difference between each explicit estimator and the exact MLE, divided by the asymptotic standard error of the corresponding MLE coordinate. The horizontal dashed line marks one asymptotic standard error. The vertical scales differ among panels. Each point is based on 2000 replications.
For the unknown-rate model, we also included the bias-corrected mixed log moment estimators of Ye and Chen [2017] as a closed-form benchmark, which were derived from two likelihood equations for the generalized gamma family as a noniterative alternative to the gamma MLE. In our shape-rate parameterization, these estimators are given by
α ~ n : = ( n − 1 n + 2 ) X ‾ n τ ~ n and β ~ n : = ( n − 1 n + 2 ) 1 τ ~ n and τ ~ n : = 1 n ∑ i = 1 n ( X i − X ‾ n ) log ( X i ) . \tilde{\alpha}_n := \left(\frac{n-1}{n+2}\right)\frac{\overline{X}_n}{\tilde{\tau}_n}
\quad\text{and}\quad
\tilde{\beta}_n := \left(\frac{n-1}{n+2}\right)\frac{1}{\tilde{\tau}_n}
\quad\text{and}\quad
\tilde{\tau}_n := \frac{1}{n}\sum_{i=1}^{n}(X_i-\overline{X}_n)\log(X_i). α ~ n := ( n + 2 n − 1 ) τ ~ n X n and β ~ n := ( n + 2 n − 1 ) τ ~ n 1 and τ ~ n := n 1 i = 1 ∑ n ( X i − X n ) log ( X i ) . The RMSE comparisons in Figure 2 lead to the same conclusions. For known rate and n = 25000 n = 25000 n = 25000 γ = 0.10 \gamma= 0.10 γ = 0.10 γ = 0.40 \gamma= 0.40 γ = 0.40 γ = 0.40 \gamma= 0.40 γ = 0.40 φ n \varphi_n φ n
Figure 2. RMSEs of each explicit estimator relative to that of the exact MLE. The curves for the estimators of Ye and Chen [2017] appear only for the unknown-rate model, and the raw and bias-corrected curves appear only for gamma regression; note that the vertical scales differ across panels. Each point is based on 2000 replications.
Finally, Figure 3 examines the limiting distributions at ( n , γ ) = ( 25000 , 0.4 ) (n,\gamma)=(25000,0.4) ( n , γ ) = ( 25000 , 0.4 ) n ( α ~ n − α n ) / α n \sqrt{n}(\widetilde{\alpha}_n-\alpha_n)/\alpha_n n ( α n − α n ) / α n n ( β ~ n − β ) \sqrt{n}(\widetilde{\beta}_n-\beta) n ( β n − β ) (39) and (42) with unit variance. The empirical variances of the corrected intercept, slope, and dispersion errors are 0.994, 0.977, and 1.016, respectively. Thus the finite-sample results reproduce both the marginal limits and the singular joint behavior derived above.
Figure 3. Distributional diagnostics at n = 25000 n = 25000 n = 25000 γ = 0.4 \gamma= 0.4 γ = 0.4 z β = 2 z α z_\beta= 2z_\alpha z β = 2 z α
Application: Black Cherry Tree Data
To illustrate the practical use of our framework, we conclude this section with a brief application to the trees data distributed with R, which record the diameter, height, and timber volume of 31 felled black cherry trees [28 ] . These data have been used to illustrate gamma regression with a log link by Wood [ [41 ] . For i = 1 , … , 31 i=1,\ldots,31 i = 1 , … , 31
Volume i ∣ d i , h i ∼ Gamma ( exp ( β 0 + β 1 d i + β 2 h i ) , ϕ ) , \operatorname{Volume}_i \mid d_i,h_i \sim\operatorname{Gamma}\left(\exp(\beta_0+\beta_1d_i+\beta_2h_i),\phi\right), Volume i ∣ d i , h i ∼ Gamma ( exp ( β 0 + β 1 d i + β 2 h i ) , ϕ ) , in the mean-dispersion parameterization given by (28) , where d i d_i d i h i h_i h i i i i
The raw explicit estimator provides a direct way to assess the plausibility of the small-ϕ \phi ϕ ϕ ~ n = 0.008456 \widetilde{\phi}_n=0.008456 ϕ n = 0.008456 ϕ ~ n = 0.0920 \sqrt{\widetilde{\phi}_n}=0.0920 ϕ n = 0.0920 β ~ n \widetilde{\boldsymbol{\beta}}_n β n
log ( Y i ) − x i T β ~ n \log(Y_i) - x_i^{\mathsf{T}}\widetilde{\boldsymbol{\beta}}_n log ( Y i ) − x i T β n max i ∣ r ~ i ∣ = 0.1773 \max_i|\widetilde{r}_i| = 0.1773 max i ∣ r i ∣ = 0.1773 h ( e r ) ≈ r 2 / 2 h(e^r) \approx r^2/2 h ( e r ) ≈ r 2 /2
2 n ∑ i h ( e r ~ i ) = 0.008486 , \frac{2}{n}\sum_i h(e^{\widetilde{r}_i}) = 0.008486, n 2 i ∑ h ( e r i ) = 0.008486 , whereas its quadratic approximation is
1 n ∑ i r ~ i 2 = ϕ ~ n = 0.008456 , \frac{1}{n}\sum_i \widetilde{r}_i^2 = \widetilde{\phi}_n = 0.008456, n 1 i ∑ r i 2 = ϕ n = 0.008456 , a relative difference of only 0.36%.
The usual gamma GLM, fitted numerically using the glm function in R with the log link, converged in six iterations. Table 1 compares it with both the raw estimators in (31) and the bias-corrected estimators in (41) . For the regression coefficients, the usual GLM uses ϕ ^ n P ( X n T X n ) − 1 \widehat{\phi}^{\mathrm{P}}_n(\mathbf{X}_n^{\mathsf{T}}\mathbf{X}_n)^{-1} ϕ n P ( X n T X n ) − 1 ϕ ^ n P \widehat{\phi}^{\mathrm{P}}_n ϕ n P ϕ ^ n P \widehat{\phi}^{\mathrm{P}}_n ϕ n P ϕ ~ n \widetilde{\phi}_n ϕ n ϕ ~ n b c \widetilde{\phi}^{\mathrm{bc}}_n ϕ n bc 2 ϕ 2 / n 2\phi^2/n 2 ϕ 2 / n ϕ \phi ϕ
Table 1. Gamma regression estimates for the black cherry tree data, with 95% Wald intervals in parentheses. The glm dispersion is the Pearson estimate divided by the residual degrees of freedom; the full likelihood estimate of dispersion is 0.008455.
Both explicit coefficient vectors differ from the usual GLM estimate by less than 0.07 of the GLM standard error, and all three sets of intervals lead to the same conclusions. The raw coefficient standard errors are
5.2% smaller than the GLM values, whereas the corrected standard errors differ by only 0.49%. The raw dispersion estimate 0.008456 is nearly identical to the full likelihood estimate 0.008455, while the corrected estimate 0.009318 is closer to the Pearson estimate 0.009410 reported by glm. Relative to the GLM fitted means, the mean and maximum absolute relative differences decrease from 0.42% and 0.63% for the raw fit to 0.10% and 0.29% after correction.
Thus the correction improves overall agreement with the conventional GLM fit, although it does not necessarily reduce the realized distance from the exact MLE in every coordinate or every sample. Its purpose is to remove deterministic bias relative to the true parameter. The correction is negligible for the unscaled comparison with the MLE, so the corrected and raw estimators retain the same unscaled equivalence properties; however, it removes terms that may be non-negligible under the first-order normalization, thereby weakening the sufficient condition for sharing the MLE’s first-order distribution from n φ n → 0 n\varphi_n \to0 n φ n → 0 n φ n 4 → 0 n\varphi_n^4 \to0 n φ n 4 → 0
Additional Examples
In this section, we illustrate the generality of our approach by applying it to several additional parametric families, focusing here on the main constructions and resulting approximate MLEs. Complete proofs and calculations, including verification of the assumptions of Theorem 2, the derivation of corresponding asymptotic distributions, and additional simulation results, are deferred to the appendices.
Zeta Distribution
For s > 1 s > 1 s > 1 k ∈ { 1 , 2 , … } k \in\{1, 2, \ldots\} k ∈ { 1 , 2 , … } s s s
P ( X = k ) = k − s ζ ( s ) . \mathbb{P}(X = k) = \frac{k^{-s}}{\zeta(s)}. P ( X = k ) = ζ ( s ) k − s . As s → ∞ s \to\infty s → ∞ k = 1 k = 1 k = 1
P s ( X > 1 ) = 1 − 1 ζ ( s ) = 1 2 s ( 1 + o ( 1 ) ) . \mathbb{P}_s(X > 1) = 1 - \frac{1}{\zeta(s)} = \frac{1}{2^s}(1 + o(1)). P s ( X > 1 ) = 1 − ζ ( s ) 1 = 2 s 1 ( 1 + o ( 1 )) . For X 1 , … , X n ∼ i i d Zeta ( s ) X_1, \ldots, X_n \overset{\mathrm{iid}}{\sim} \operatorname{Zeta}(s) X 1 , … , X n ∼ iid Zeta ( s ) G n : = ( ∏ i = 1 n X i ) 1 / n G_n := \left(\prod_{i=1}^{n} X_i\right)^{1/n} G n := ( ∏ i = 1 n X i ) 1/ n
q n ( s ) : = − n ( ζ ′ ( s ) ζ ( s ) + log ( G n ) ) and q n ′ ( s ) = − n Var s ( log ( X ) ) < 0. (44) q_n(s) := -n\left(\frac{\zeta'(s)}{\zeta(s)} + \log(G_n)\right)
\quad\text{and}\quad
q_n'(s) = -n\operatorname{Var}_s(\log(X)) < 0. \tag*{(44)} q n ( s ) := − n ( ζ ( s ) ζ ′ ( s ) + log ( G n ) ) and q n ′ ( s ) = − n Var s ( log ( X )) < 0. (44) As s ↓ 1 s \downarrow1 s ↓ 1 q n ( s ) → ∞ q_n(s) \to\infty q n ( s ) → ∞ s → ∞ s \to\infty s → ∞ q n ( s ) → − n log ( G n ) q_n(s) \to-n\log(G_n) q n ( s ) → − n log ( G n ) G n > 1 G_n > 1 G n > 1 G n = 1 G_n = 1 G n = 1 X 1 = ⋯ = X n = 1 X_1 = \cdots= X_n = 1 X 1 = ⋯ = X n = 1 s → ∞ s \to\infty s → ∞ § 2.1 .
The function ζ ′ / ζ \zeta'/\zeta ζ ′ / ζ
− ζ ′ ( s ) ζ ( s ) = ∑ m = 2 ∞ Λ ( m ) m s , (45) -\frac{\zeta'(s)}{\zeta(s)} = \sum_{m=2}^{\infty} \frac{\Lambda(m)}{m^s}, \tag*{(45)} − ζ ( s ) ζ ′ ( s ) = m = 2 ∑ ∞ m s Λ ( m ) , (45) valid for all s > 1 s > 1 s > 1 [18 Theorem 294] , where Λ \Lambda Λ 2 s → ∞ s \to\infty s → ∞
− ζ ′ ( s ) ζ ( s ) = log ( 2 ) 2 s + O ( 1 3 s ) . (46) -\frac{\zeta'(s)}{\zeta(s)} = \frac{\log(2)}{2^s} + O\left(\frac{1}{3^s}\right). \tag*{(46)} − ζ ( s ) ζ ′ ( s ) = 2 s log ( 2 ) + O ( 3 s 1 ) . (46) Thus, in contrast to the gamma examples of § 3 , this approximation arises from a Dirichlet series rather than an expansion in integer powers of the parameter. Define the vanishing chart by ϕ ( s ) : = 2 − s \phi(s) := 2^{-s} ϕ ( s ) := 2 − s u : = ϕ ( s ) u := \phi(s) u := ϕ ( s ) U : = ( 0 , 1 / 2 ) U := (0, 1/2) U := ( 0 , 1/2 ) ϕ − 1 ( u ) = − log 2 ( u ) \phi^{-1}(u) = -\log_2(u) ϕ − 1 ( u ) = − log 2 ( u )
Q n ( u ) : = n ( m ( u ) − log ( G n ) ) and Q ~ n ( u ) : = n ( log ( 2 ) u − log ( G n ) ) , where m ( u ) : = ∑ m = 2 ∞ Λ ( m ) u log 2 ( m ) . Q_n(u) := n(m(u) - \log(G_n)) \quad\text{and}\quad\widetilde{Q}_n(u) := n(\log(2)u - \log(G_n)), \qquad\text{where}\quad m(u) := \sum_{m=2}^{\infty} \Lambda(m)u^{\log_2(m)}. Q n ( u ) := n ( m ( u ) − log ( G n )) and Q n ( u ) := n ( log ( 2 ) u − log ( G n )) , where m ( u ) := m = 2 ∑ ∞ Λ ( m ) u l o g 2 ( m ) . The surrogate root and corresponding approximate MLE are, respectively,
u ~ n : = log ( G n ) log ( 2 ) and s ~ n : = − log 2 ( log ( G n ) log ( 2 ) ) = − log 2 ( log 2 ( G n ) ) . \widetilde{u}_n := \frac{\log(G_n)}{\log(2)} \quad\text{and}\quad\widetilde{s}_n := -\log_2\left(\frac{\log(G_n)}{\log(2)}\right) = -\log_2(\log_2(G_n)). u n := log ( 2 ) log ( G n ) and s n := − log 2 ( log ( 2 ) log ( G n ) ) = − log 2 ( log 2 ( G n )) . Fix s n → ∞ s_n \to\infty s n → ∞ u n : = 2 − s n u_n := 2^{-s_n} u n := 2 − s n p : = log 2 ( 3 ) p := \log_2(3) p := log 2 ( 3 ) P n : = P s n ⊗ n \mathbb{P}_n := \mathbb{P}_{s_n}^{\otimes n} P n := P s n ⊗ n n u n → ∞ nu_n \to\infty n u n → ∞
u ^ n = u n + o P n ( u n ) and u ~ n = u n + o P n ( u n ) . \widehat{u}_n = u_n + o_{\mathbb{P}_n}(u_n) \quad\text{and}\quad\widetilde{u}_n = u_n + o_{\mathbb{P}_n}(u_n). u n = u n + o P n ( u n ) and u n = u n + o P n ( u n ) . Moreover,
P n ( G n = 1 ) = 1 ζ ( s n ) n = exp ( − n u n ( 1 + o ( 1 ) ) ) ⟶ 0 , \mathbb{P}_n(G_n = 1) = \frac{1}{\zeta(s_n)^n} = \exp(-nu_n(1 + o(1))) \longrightarrow0, P n ( G n = 1 ) = ζ ( s n ) n 1 = exp ( − n u n ( 1 + o ( 1 ))) ⟶ 0 , so both u ^ n \widehat{u}_n u n u ~ n \widetilde{u}_n u n U U U
The verification in Section B.2 gives
∣ u ~ n − u ^ n ∣ = O P n ( u n p ) and ∣ s ~ n − s ^ n ∣ = O P n ( u n p − 1 ) = O P n ( 2 − s n ( p − 1 ) ) = o P n ( 1 ) . \lvert\widetilde{u}_n - \widehat{u}_n \rvert= O_{\mathbb{P}_n}(u_n^p) \quad\text{and}\quad\lvert\widetilde{s}_n - \widehat{s}_n \rvert= O_{\mathbb{P}_n}(u_n^{p-1}) = O_{\mathbb{P}_n}\left(2^{-s_n(p-1)}\right) = o_{\mathbb{P}_n}(1). ∣ u n − u n ∣ = O P n ( u n p ) and ∣ s n − s n ∣ = O P n ( u n p − 1 ) = O P n ( 2 − s n ( p − 1 ) ) = o P n ( 1 ) . The exact MLE also satisfies
n 2 s n ( s ^ n − s n ) → d N ( 0 , 1 log ( 2 ) 2 ) . \sqrt{\frac{n}{2^{s_n}}}(\widehat{s}_n - s_n) \xrightarrow{d} \mathcal{N}\left(0, \frac{1}{\log(2)^2}\right). 2 s n n ( s n − s n ) d N ( 0 , log ( 2 ) 2 1 ) . If, in addition, n u n p − 1 / 2 → 0 \sqrt{n}u_n^{p-1/2} \to0 n u n p − 1/2 → 0 s ^ n \widehat{s}_n s n s ~ n \widetilde{s}_n s n
n 2 s n ( s ~ n − s n ) → d N ( 0 , 1 log ( 2 ) 2 ) . \sqrt{\frac{n}{2^{s_n}}}(\widetilde{s}_n - s_n) \xrightarrow{d} \mathcal{N}\left(0, \frac{1}{\log(2)^2}\right). 2 s n n ( s n − s n ) d N ( 0 , log ( 2 ) 2 1 ) . Other closed-form estimators of s s s [9 ] obtain one by approximating a lower-truncated discrete power law by its continuous analogue and applying a continuity correction. The accuracy of this approximation depends on a sufficiently large lower truncation x min x_{\min} x m i n x min = 1 x_{\min} = 1 x m i n = 1
The full verification and simulation study are provided in Section B, along with supporting moment calculations and a verification of the relevant Lyapunov condition.
von Mises Distribution
For κ > 0 \kappa> 0 κ > 0 ( μ , κ ) (\mu,\kappa) ( μ , κ ) [ 0 , 2 π ) [0,2\pi) [ 0 , 2 π )
f μ , κ ( x ) : = exp ( κ cos ( x − μ ) ) 2 π I 0 ( κ ) . f_{\mu,\kappa}(x) := \frac{\exp(\kappa\cos(x-\mu))}{2\pi I_0(\kappa)}. f μ , κ ( x ) := 2 π I 0 ( κ ) exp ( κ cos ( x − μ )) . where μ \mu μ 2 π 2\pi 2 π I m I_m I m m m m μ \mu μ κ \kappa κ μ \mu μ μ \mu μ κ \kappa κ
For X 1 , … , X n ∼ i i d von Mises ( μ , κ ) X_1,\ldots,X_n \mathrel{\overset{\mathrm{iid}}{\sim}} \operatorname{von\,Mises}(\mu,\kappa) X 1 , … , X n ∼ iid von Mises ( μ , κ )
C ‾ n : = 1 n ∑ i = 1 n cos ( X i ) and S ‾ n : = 1 n ∑ i = 1 n sin ( X i ) and R n : = C ‾ n 2 + S ‾ n 2 , \overline{C}_n := \frac{1}{n}\sum_{i=1}^{n}\cos(X_i)
\quad\text{and}\quad
\overline{S}_n := \frac{1}{n}\sum_{i=1}^{n}\sin(X_i)
\quad\text{and}\quad
R_n := \sqrt{\overline{C}_n^2+\overline{S}_n^2}, C n := n 1 i = 1 ∑ n cos ( X i ) and S n := n 1 i = 1 ∑ n sin ( X i ) and R n := C n 2 + S n 2 , and let μ ^ n : = atan2 ( S ‾ n , C ‾ n ) \widehat{\mu}_n := \operatorname{atan2}(\overline{S}_n,\overline{C}_n) μ n := atan2 ( S n , C n )
Writing A ( κ ) : = I 1 ( κ ) / I 0 ( κ ) A(\kappa) := I_1(\kappa)/I_0(\kappa) A ( κ ) := I 1 ( κ ) / I 0 ( κ ) ( μ ^ n , κ ^ n ) (\widehat{\mu}_n,\widehat{\kappa}_n) ( μ n , κ n )
μ ^ n = atan2 ( S ‾ n , C ‾ n ) and A ( κ ^ n ) = R n . \widehat{\mu}_n = \operatorname{atan2}(\overline{S}_n,\overline{C}_n)
\quad\text{and}\quad
A(\widehat{\kappa}_n) = R_n. μ n = atan2 ( S n , C n ) and A ( κ n ) = R n . The function A A A 0 0 0 1 1 1 ( 0 , ∞ ) (0,\infty) ( 0 , ∞ ) n ≥ 2 n \ge2 n ≥ 2 R n = 0 R_n = 0 R n = 0 R n = 1 R_n = 1 R n = 1 2 π 2\pi 2 π § 2.1 on exceptional events.
To work with an ordinary coordinate for μ \mu μ M ⊂ R M \subset\mathbb{R} M ⊂ R 2 π 2\pi 2 π μ \mu μ μ ^ n \widehat{\mu}_n μ n M M M κ → ∞ \kappa\to\infty κ → ∞
To preserve the explicit location estimator while placing both parameters in a vanishing chart, we combine the location and concentration equations as follows:
q n ( μ , κ ) : = ( μ ( A ( κ ) − 1 ) − μ ^ n ( R n − 1 ) A ( κ ) − R n ) = 0. (47) q_n(\mu,\kappa) :=
\begin{pmatrix}
\dfrac{\mu(A(\kappa)-1)-\widehat{\mu}_n(R_n-1)}{A(\kappa)-R_n}
\end{pmatrix}
= \mathbf{0}. \tag*{(47)} q n ( μ , κ ) := ( A ( κ ) − R n μ ( A ( κ ) − 1 ) − μ n ( R n − 1 ) ) = 0 . (47) This modification preserves the selected root. Indeed, if q 2 , n = 0 q_{2,n}=0 q 2 , n = 0
q 1 , n = ( R n − 1 ) ( μ − μ ^ n ) , q_{1,n}=(R_n-1)(\mu-\widehat{\mu}_n), q 1 , n = ( R n − 1 ) ( μ − μ n ) , which forces μ = μ ^ n \mu=\widehat{\mu}_n μ = μ n R n < 1 R_n<1 R n < 1 A ( κ ) − 1 A(\kappa)-1 A ( κ ) − 1 κ → ∞ \kappa\to\infty κ → ∞
The Bessel ratio has the expansion
A ( κ ) = 1 − 1 2 κ − 1 8 κ 2 − 1 8 κ 3 + O ( 1 κ 4 ) . A(\kappa)=1-\frac{1}{2\kappa}-\frac{1}{8\kappa^2}-\frac{1}{8\kappa^3}+O\left(\frac{1}{\kappa^4}\right). A ( κ ) = 1 − 2 κ 1 − 8 κ 2 1 − 8 κ 3 1 + O ( κ 4 1 ) . [2 Formulas 9.7.1 and 9.7.3] Retaining the first two non-constant terms gives g ( κ ) : = − 1 / ( 2 κ ) − 1 / ( 8 κ 2 ) g(\kappa):=-1/(2\kappa)-1/(8\kappa^2) g ( κ ) := − 1/ ( 2 κ ) − 1/ ( 8 κ 2 ) A ( κ ) − 1 A(\kappa)-1 A ( κ ) − 1 g ( κ ) g(\kappa) g ( κ ) (47) leaves the location root unchanged and gives the candidate estimators
μ ~ n : = μ ^ n and κ ~ n : = 1 + 3 − 2 R n 4 ( 1 − R n ) . \widetilde{\mu}_n := \widehat{\mu}_n
\quad\text{and}\quad
\widetilde{\kappa}_n := \frac{1+\sqrt{3-2R_n}}{4(1-R_n)}. μ n := μ n and κ n := 4 ( 1 − R n ) 1 + 3 − 2 R n . Thus only the concentration equation is approximated. The leading term of κ ~ n \widetilde{\kappa}_n κ n p p p
κ ^ ≈ p − 1 2 ( 1 − R ‾ ) . \widehat{\kappa}\approx\frac{p-1}{2(1-\overline{R})}. κ ≈ 2 ( 1 − R ) p − 1 . [26 Equation (10.3.7] , which is also used by Banerjee et al. [ [4 ] . In the present setting, set p : = 2 p := 2 p := 2 R : = R n R := R_n R := R n δ n : = 1 − R n \delta_n := 1 - R_n δ n := 1 − R n
κ ~ n = 1 2 δ n + 1 4 + δ n 8 + O ( δ n 2 ) . \widetilde{\kappa}_n = \frac{1}{2\delta_n} + \frac{1}{4} + \frac{\delta_n}{8} + O(\delta_n^2). κ n = 2 δ n 1 + 4 1 + 8 δ n + O ( δ n 2 ) . Thus the classical approximation is the leading term of κ ~ n \widetilde{\kappa}_n κ n A A A
κ ^ n = 1 2 δ n + 1 4 + 3 δ n 8 + O ( δ n 2 ) , \widehat{\kappa}_n = \frac{1}{2\delta_n} + \frac{1}{4} + \frac{3\delta_n}{8} + O(\delta_n^2), κ n = 2 δ n 1 + 4 1 + 8 3 δ n + O ( δ n 2 ) , so the two estimators agree through their constant terms. The following analysis makes this formal comparison rigorous under the coupled asymptotic regime.
Set N : = M × ( 0 , ∞ ) N := M \times(0,\infty) N := M × ( 0 , ∞ )
ϕ ( μ , κ ) : = ( μ g ( κ ) , g ( κ ) ) T and u : = ϕ ( μ , κ ) = ( u 1 , u 2 ) T and U : = { ( u 1 , u 2 ) : u 2 < 0 , u 1 u 2 ∈ M } , \phi(\mu,\kappa) := (\mu g(\kappa),g(\kappa))^{\mathsf T}\quad\text{and}\quad\boldsymbol{u} := \phi(\mu,\kappa) = (u_1,u_2)^{\mathsf T}\quad\text{and}\quad U := \{(u_1,u_2): u_2 < 0,\ \frac{u_1}{u_2} \in M\}, ϕ ( μ , κ ) := ( μg ( κ ) , g ( κ ) ) T and u := ϕ ( μ , κ ) = ( u 1 , u 2 ) T and U := {( u 1 , u 2 ) : u 2 < 0 , u 2 u 1 ∈ M } , has inverse
ϕ − 1 ( u 1 , u 2 ) = ( u 1 u 2 , − 1 + 1 − 2 u 2 4 u 2 ) T . \phi^{-1}(u_1,u_2) = \left(\frac{u_1}{u_2},-\frac{1+\sqrt{1-2u_2}}{4u_2}\right)^{\mathsf T}. ϕ − 1 ( u 1 , u 2 ) = ( u 2 u 1 , − 4 u 2 1 + 1 − 2 u 2 ) T . For fixed μ \mu μ κ → ∞ \kappa\to\infty κ → ∞ u 1 / u 2 u_1/u_2 u 1 / u 2
Fix κ n → ∞ \kappa_n \to\infty κ n → ∞ P n : = P μ , κ n P_n := P_{\mu,\kappa_n} P n := P μ , κ n K n ⊂ U K_n \subset U K n ⊂ U sup u ∈ K n ∥ u ∥ → 0 \sup_{u\in K_n}\|u\| \to0 sup u ∈ K n ∥ u ∥ → 0
J Q ~ n = I 2 and sup u ∈ K n ∥ Q n ( u ) − Q ~ n ( u ) ∥ = O ( 1 κ n 3 ) , J\widetilde{Q}_n = I_2 \quad\text{and}\quad\sup_{u\in K_n}\|Q_n(u)-\widetilde{Q}_n(u)\| = O\left(\frac{1}{\kappa_n^3}\right), J Q n = I 2 and u ∈ K n sup ∥ Q n ( u ) − Q n ( u ) ∥ = O ( κ n 3 1 ) , and ϕ − 1 \phi^{-1} ϕ − 1 O ( κ n 2 ) O(\kappa_n^2) O ( κ n 2 )
∥ u ~ n − u ^ n ∥ = O P n ( 1 κ n 3 ) , \|\widetilde{\boldsymbol{u}}_n-\widehat{\boldsymbol{u}}_n\| = O_{P_n}\left(\frac{1}{\kappa_n^3}\right), ∥ u n − u n ∥ = O P n ( κ n 3 1 ) , while Corollary 3 gives
μ ~ n = μ ^ n and κ ~ n − κ ^ n = O P n ( 1 κ n ) = o P n ( 1 ) . \widetilde{\mu}_n = \widehat{\mu}_n \quad\text{and}\quad\widetilde{\kappa}_n-\widehat{\kappa}_n = O_{P_n}\left(\frac{1}{\kappa_n}\right) = o_{P_n}(1). μ n = μ n and κ n − κ n = O P n ( κ n 1 ) = o P n ( 1 ) . The Fisher information for a one observation is
I 1 ( μ , κ ) = ( κ A ( κ ) 0 0 A ′ ( κ ) ) , \mathcal{I}_1(\mu,\kappa) =
\begin{pmatrix}
\kappa A(\kappa) & 0\\
0 & A'(\kappa)
\end{pmatrix}, I 1 ( μ , κ ) = ( κ A ( κ ) 0 0 A ′ ( κ ) ) , where A ′ ( κ ) ∼ 1 / ( 2 κ 2 ) A'(\kappa) \sim1/(2\kappa^2) A ′ ( κ ) ∼ 1/ ( 2 κ 2 )
( n κ n ( μ ^ n − μ ) n κ ^ n − κ n κ n ) → d N ( ( 0 0 ) , ( 1 0 0 2 ) ) . \begin{pmatrix}
\sqrt{n\kappa_n}(\widehat{\mu}_n-\mu)\\
\sqrt{n}\dfrac{\widehat{\kappa}_n-\kappa_n}{\kappa_n}
\end{pmatrix}
\xrightarrow{d}
\mathcal{N}\left(
\begin{pmatrix}0\\0\end{pmatrix},
\begin{pmatrix}1&0\\0&2\end{pmatrix}
\right). n κ n ( μ n − μ ) n κ n κ n − κ n d N ( ( 0 0 ) , ( 1 0 0 2 ) ) . If in addition n / κ n 2 → 0 \sqrt{n}/\kappa_n^2 \to0 n / κ n 2 → 0 κ n / n 1 / 4 → ∞ \kappa_n/n^{1/4} \to\infty κ n / n 1/4 → ∞
n κ n ( κ ~ n − κ ^ n ) = o P n ( 1 ) . \frac{\sqrt{n}}{\kappa_n}(\widetilde{\kappa}_n-\widehat{\kappa}_n) = o_{P_n}(1). κ n n ( κ n − κ n ) = o P n ( 1 ) . Since the exact and approximate location MLEs coincide, Slutsky’s theorem gives the same joint limit for ( μ ~ n , κ ~ n ) (\widetilde{\mu}_n,\widetilde{\kappa}_n) ( μ n , κ n )
For comparison, Abeyasekera and Collett [ [1 ] use a normal approximation to a highly concentrated von Mises distribution and, for n > 3 n > 3 n > 3
κ ^ l i n : = 1 1 n − 3 ∑ i = 1 n ( X i − X ‾ n ) 2 . \widehat{\kappa}_{\mathrm{lin}} := \frac{1}{\frac{1}{n-3}\sum_{i=1}^{n}(X_i-\overline{X}_n)^2}. κ lin := n − 3 1 ∑ i = 1 n ( X i − X n ) 2 1 . Because this estimator treats the angles as ordinary real observations, an unwrapping convention is needed when the sample straddles the circular discontinuity. Their construction approximates the distribution by a normal law, whereas ours retains the exact von Mises likelihood and approximates only the Bessel ratio equation. The detailed verification and a simulation study that includes a comparison with the estimator of Abeyasekera and Collett [ [1 ] are given in Section C.
Generalized Pareto Distribution
Finally, we consider the generalized Pareto distribution, which presents two additional complications: its support depends on one parameter, and the score function for another parameter has a singularity at the boundary of the parameter space.
Fix 0 < ξ ˉ < 1 / 6 0 < \bar{\xi} < 1/6 0 < ξ ˉ < 1/6
Θ ξ ˉ : = { ( μ , σ , ξ ) : μ ∈ R , σ > 0 , 0 ≤ ξ ≤ ξ ˉ } . \Theta_{\bar{\xi}} := \{(\mu,\sigma,\xi): \mu\in\mathbb{R},\ \sigma> 0,\ 0 \le\xi\le\bar{\xi}\}. Θ ξ ˉ := {( μ , σ , ξ ) : μ ∈ R , σ > 0 , 0 ≤ ξ ≤ ξ ˉ } . For ξ > 0 \xi> 0 ξ > 0 x ≥ μ x \ge\mu x ≥ μ
f μ , σ , ξ ( x ) : = 1 σ ( 1 + ξ x − μ σ ) − 1 − 1 ξ . f_{\mu,\sigma,\xi}(x) := \frac{1}{\sigma}\left(1+\xi\frac{x-\mu}{\sigma}\right)^{-1-\frac{1}{\xi}}. f μ , σ , ξ ( x ) := σ 1 ( 1 + ξ σ x − μ ) − 1 − ξ 1 . At ξ = 0 \xi= 0 ξ = 0 x ≥ μ x \ge\mu x ≥ μ
f μ , σ , 0 ( x ) : = 1 σ exp ( − x − μ σ ) . f_{\mu,\sigma,0}(x) := \frac{1}{\sigma}\exp\left(-\frac{x-\mu}{\sigma}\right). f μ , σ , 0 ( x ) := σ 1 exp ( − σ x − μ ) . We observe X n , 1 , … , X n , n ∼ i i d GPD ( μ , σ , ξ n ) X_{n,1},\ldots,X_{n,n} \mathrel{\overset{\mathrm{iid}}{\sim}} \operatorname{GPD}(\mu,\sigma,\xi_n) X n , 1 , … , X n , n ∼ iid GPD ( μ , σ , ξ n ) μ ∈ R \mu\in\mathbb{R} μ ∈ R σ > 0 \sigma> 0 σ > 0 ξ n ↓ 0 \xi_n \downarrow0 ξ n ↓ 0 P n : = P μ , σ , ξ n ⊗ n P_n := P_{\mu,\sigma,\xi_n}^{\otimes n} P n := P μ , σ , ξ n ⊗ n ξ ≤ ξ ˉ \xi\le\bar{\xi} ξ ≤ ξ ˉ ξ ˉ < 1 / 6 \bar{\xi} < 1/6 ξ ˉ < 1/6 [16 ] .
Writing X ( 1 ) X_{(1)} X ( 1 ) X i X_i X i μ \mu μ μ ^ n = X ( 1 ) \widehat{\mu}_n = X_{(1)} μ n = X ( 1 )
Y i : = X n , i − X ( 1 ) and M k , n : = 1 n ∑ i = 1 n Y i k . Y_i := X_{n,i}-X_{(1)} \quad\text{and}\quad M_{k,n} := \frac{1}{n}\sum_{i=1}^{n}Y_i^k. Y i := X n , i − X ( 1 ) and M k , n := n 1 i = 1 ∑ n Y i k . We profile μ \mu μ X ( 1 ) X_{(1)} X ( 1 )
S σ ( σ , ξ ) : = 1 σ ( ∑ i = 1 n ( 1 + ξ ) t i ( σ ) 1 + ξ t i ( σ ) − n ) and S ξ ( σ , ξ ) : = 1 ξ ( ∑ i = 1 n log ( 1 + ξ t i ( σ ) ) ξ − ∑ i = 1 n ( 1 + ξ ) t i ( σ ) 1 + ξ t i ( σ ) ) S_{\sigma}(\sigma,\xi) := \frac{1}{\sigma}\left(\sum_{i=1}^{n}\frac{(1+\xi)t_i(\sigma)}{1+\xi t_i(\sigma)}-n\right)
\quad\text{and}\quad
S_{\xi}(\sigma,\xi) := \frac{1}{\xi}\left(\sum_{i=1}^{n}\frac{\log(1+\xi t_i(\sigma))}{\xi}-\sum_{i=1}^{n}\frac{(1+\xi)t_i(\sigma)}{1+\xi t_i(\sigma)}\right) S σ ( σ , ξ ) := σ 1 ( i = 1 ∑ n 1 + ξ t i ( σ ) ( 1 + ξ ) t i ( σ ) − n ) and S ξ ( σ , ξ ) := ξ 1 ( i = 1 ∑ n ξ log ( 1 + ξ t i ( σ )) − i = 1 ∑ n 1 + ξ t i ( σ ) ( 1 + ξ ) t i ( σ ) ) where t i ( s ) : = Y i / s t_i(s) := Y_i/s t i ( s ) := Y i / s
B n ( σ , ξ ) : = 1 n ∑ i = 1 n log ( 1 + ξ t i ( σ ) ) ξ and A n ( σ , ξ ) : = 1 n ∑ i = 1 n ( 1 + ξ ) t i ( σ ) 1 + ξ t i ( σ ) , B_n(\sigma,\xi) := \frac{1}{n}\sum_{i=1}^{n}\frac{\log(1+\xi t_i(\sigma))}{\xi}
\quad\text{and}\quad
A_n(\sigma,\xi) := \frac{1}{n}\sum_{i=1}^{n}\frac{(1+\xi)t_i(\sigma)}{1+\xi t_i(\sigma)}, B n ( σ , ξ ) := n 1 i = 1 ∑ n ξ log ( 1 + ξ t i ( σ )) and A n ( σ , ξ ) := n 1 i = 1 ∑ n 1 + ξ t i ( σ ) ( 1 + ξ ) t i ( σ ) , the equivalent system
q n ( σ , ξ ) : = ( B n ( σ , ξ ) − 1 σ B n ( σ , ξ ) − A n ( σ , ξ ) ξ ) = 1 n ( σ ξ 0 1 ) ( S σ ( σ , ξ ) S ξ ( σ , ξ ) ) \mathbf{q}_n(\sigma,\xi):=\begin{pmatrix}\dfrac{B_n(\sigma,\xi)-1}{\sigma}\\[4pt]\dfrac{B_n(\sigma,\xi)-A_n(\sigma,\xi)}{\xi}\end{pmatrix}=\frac{1}{n}\begin{pmatrix}\sigma& \xi\\0 & 1\end{pmatrix}\begin{pmatrix}S_\sigma(\sigma,\xi)\\S_\xi(\sigma,\xi)\end{pmatrix} q n ( σ , ξ ) := σ B n ( σ , ξ ) − 1 ξ B n ( σ , ξ ) − A n ( σ , ξ ) = n 1 ( σ 0 ξ 1 ) ( S σ ( σ , ξ ) S ξ ( σ , ξ ) ) has the same interior roots because the matrix is nonsingular for σ > 0 \sigma>0 σ > 0 ξ = 0 \xi=0 ξ = 0 ξ = 0 \xi=0 ξ = 0
q ~ n ( σ , ξ ) : = ( M 1 , n σ − 1 − ξ M 2 , n 2 σ 2 − M 1 , n σ + M 2 , n 2 σ 2 + ξ ( M 2 , n σ 2 − 2 M 3 , n 3 σ 3 ) ) . \widetilde{\mathbf{q}}_n(\sigma,\xi):=\begin{pmatrix}\dfrac{M_{1,n}}{\sigma}-1-\dfrac{\xi M_{2,n}}{2\sigma^2}\\[8pt]-\dfrac{M_{1,n}}{\sigma}+\dfrac{M_{2,n}}{2\sigma^2}+\xi\left(\dfrac{M_{2,n}}{\sigma^2}-\dfrac{2M_{3,n}}{3\sigma^3}\right)\end{pmatrix}. q n ( σ , ξ ) := σ M 1 , n − 1 − 2 σ 2 ξ M 2 , n − σ M 1 , n + 2 σ 2 M 2 , n + ξ ( σ 2 M 2 , n − 3 σ 3 2 M 3 , n ) . Setting
C n : = 6 M 1 , n M 2 , n + 8 M 3 , n and E n : = 8 M 1 , n M 3 , n − 3 M 2 , n 2 , C_n:=6M_{1,n}M_{2,n}+8M_{3,n}\quad\text{and}\quad E_n:=8M_{1,n}M_{3,n}-3M_{2,n}^{2}, C n := 6 M 1 , n M 2 , n + 8 M 3 , n and E n := 8 M 1 , n M 3 , n − 3 M 2 , n 2 , we select the root ( σ ~ n , ξ ~ n ) (\widetilde{\sigma}_n,\widetilde{\xi}_n) ( σ n , ξ n )
σ ~ n : = C n − C n 2 − 48 M 2 , n E n 24 M 2 , n and ξ ~ n : = 2 σ ~ n ( M 1 , n − σ ~ n ) M 2 , n . \widetilde{\sigma}_n:=\frac{C_n-\sqrt{C_n^2-48M_{2,n}E_n}}{24M_{2,n}}\quad\text{and}\quad\widetilde{\xi}_n:=\frac{2\widetilde{\sigma}_n(M_{1,n}-\widetilde{\sigma}_n)}{M_{2,n}}. σ n := 24 M 2 , n C n − C n 2 − 48 M 2 , n E n and ξ n := M 2 , n 2 σ n ( M 1 , n − σ n ) . At the population moments of the shifted exponential distribution, the two scale roots are σ \sigma σ 3 σ / 2 3\sigma/2 3 σ /2 μ ~ n : = X ( 1 ) \widetilde{\mu}_n:=X_{(1)} μ n := X ( 1 ) Lemma D.3 . If the selected raw branch is not real or does not belong to ( 0 , ∞ ) × [ 0 , ξ ‾ ] (0,\infty)\times[0,\overline{\xi}] ( 0 , ∞ ) × [ 0 , ξ ] ( σ ~ n , ξ ~ n ) : = ( M 1 , n , 0 ) (\widetilde{\sigma}_n,\widetilde{\xi}_n):=(M_{1,n},0) ( σ n , ξ n ) := ( M 1 , n , 0 )
The first surrogate equation also gives a useful interpretation of the scale estimator. On the event where the selected root is well defined, we have
σ ~ n = M 1 , n + M 1 , n 2 − 2 M 2 , n ξ ~ n 2 . \widetilde{\sigma}_n=\frac{M_{1,n}+\sqrt{M_{1,n}^{2}-2M_{2,n}\widetilde{\xi}_n}}{2}. σ n = 2 M 1 , n + M 1 , n 2 − 2 M 2 , n ξ n . Under the regime considered below, ξ ~ n = o P n ( 1 ) \widetilde{\xi}_n=o_{P_n}(1) ξ n = o P n ( 1 ) ξ ~ n = 0 \widetilde{\xi}_n=0 ξ n = 0
σ ~ n = M 1 , n − M 2 , n 2 M 1 , n ξ ~ n + O P n ( ξ ~ n 2 ) . \widetilde{\sigma}_n=M_{1,n}-\frac{M_{2,n}}{2M_{1,n}}\widetilde{\xi}_n+O_{P_n}\left(\widetilde{\xi}_n^{2}\right). σ n = M 1 , n − 2 M 1 , n M 2 , n ξ n + O P n ( ξ n 2 ) . At ξ = 0 \xi=0 ξ = 0 μ ~ n = X ( 1 ) \widetilde{\mu}_n=X_{(1)} μ n = X ( 1 )
M 1 , n = 1 n ∑ i = 1 n ( X n , i − X ( 1 ) ) . M_{1,n}=\frac{1}{n}\sum_{i=1}^{n}\left(X_{n,i}-X_{(1)}\right). M 1 , n = n 1 i = 1 ∑ n ( X n , i − X ( 1 ) ) . Thus σ ~ n \widetilde{\sigma}_n σ n
The population target of the selected branch satisfies
σ ∗ ( ξ ) = σ ( 1 + 8 ξ 2 − 68 ξ 3 + O ( ξ 4 ) ) and ξ ∗ ( ξ ) = ξ − 10 ξ 2 + 100 ξ 3 + O ( ξ 4 ) , \sigma_*(\xi)=\sigma\left(1+8\xi^2-68\xi^3+O(\xi^4)\right)\quad\text{and}\quad\xi_*(\xi)=\xi-10\xi^2+100\xi^3+O(\xi^4), σ ∗ ( ξ ) = σ ( 1 + 8 ξ 2 − 68 ξ 3 + O ( ξ 4 ) ) and ξ ∗ ( ξ ) = ξ − 10 ξ 2 + 100 ξ 3 + O ( ξ 4 ) , so the population discrepancy begins at order ξ 2 \xi^2 ξ 2
ϕ ( σ , ξ ) : = ( ξ σ ξ σ 2 ) and F ( u 2 , u 3 ) : = ϕ − 1 ( u 2 , u 3 ) = ( u 2 u 3 u 2 2 u 3 ) . \phi(\sigma,\xi):=\begin{pmatrix}\dfrac{\xi}{\sigma}\\[4pt]\dfrac{\xi}{\sigma^2}\end{pmatrix}\quad\text{and}\quad F(u_2,u_3):=\phi^{-1}(u_2,u_3)=\begin{pmatrix}\dfrac{u_2}{u_3}\\[6pt]\dfrac{u_2^2}{u_3}\end{pmatrix}. ϕ ( σ , ξ ) := σ ξ σ 2 ξ and F ( u 2 , u 3 ) := ϕ − 1 ( u 2 , u 3 ) = u 3 u 2 u 3 u 2 2 . on 0 < ξ < ξ ˉ 0 < \xi< \bar{\xi} 0 < ξ < ξ ˉ
U : = { ( u 2 , u 3 ) : u 2 , u 3 > 0 , u 2 2 u 3 < ξ ˉ } . U := \left\{(u_2,u_3): u_2,u_3 > 0,\ \frac{u_2^2}{u_3} < \bar{\xi}\right\}. U := { ( u 2 , u 3 ) : u 2 , u 3 > 0 , u 3 u 2 2 < ξ ˉ } . Suppose first that we are in the regime where n ξ n → ∞ \sqrt{n}\xi_n \to\infty n ξ n → ∞ n \sqrt{n} n
∥ ( σ ~ n ξ ~ n ) − ( σ ^ n ξ ^ n ) ∥ = O P n ( ξ n 2 ) = o P n ( 1 ) . \left\Vert
\begin{pmatrix}
\tilde{\sigma}_n\\
\tilde{\xi}_n
\end{pmatrix}
-
\begin{pmatrix}
\hat{\sigma}_n\\
\hat{\xi}_n
\end{pmatrix}
\right\Vert
= O_{\mathbb{P}_n}(\xi_n^2)=o_{\mathbb{P}_n}(1). ( σ ~ n ξ ~ n ) − ( σ ^ n ξ ^ n ) = O P n ( ξ n 2 ) = o P n ( 1 ) . Because the two estimators have the same location coordinate, θ ~ n \tilde{\theta}_n θ ~ n
If in addition n ξ n 2 → 0 \sqrt{n}\xi_n^2 \to0 n ξ n 2 → 0 n \sqrt{n} n
n ( σ ~ n − σ ^ n ξ ~ n − ξ ^ n ) = o P n ( 1 ) . \sqrt{n}
\begin{pmatrix}
\tilde{\sigma}_n-\hat{\sigma}_n\\
\tilde{\xi}_n-\hat{\xi}_n
\end{pmatrix}
= o_{\mathbb{P}_n}(1). n ( σ ~ n − σ ^ n ξ ~ n − ξ ^ n ) = o P n ( 1 ) . Together, the conditions n ξ n → ∞ \sqrt{n}\xi_n \to\infty n ξ n → ∞ n ξ n 2 → 0 \sqrt{n}\xi_n^2 \to0 n ξ n 2 → 0 n − 1 / 2 ≪ ξ n ≪ n − 1 / 4 n^{-1/2} \ll\xi_n \ll n^{-1/4} n − 1/2 ≪ ξ n ≪ n − 1/4
n ( σ ^ n − σ ξ ^ n − ξ n ) → d N ( ( 0 0 ) , ( 2 σ 2 − σ − σ 1 ) ) \sqrt{n}
\begin{pmatrix}
\hat{\sigma}_n-\sigma\\
\hat{\xi}_n-\xi_n
\end{pmatrix}
\xrightarrow{d}
\mathcal{N}\left(
\begin{pmatrix}
0\\
0
\end{pmatrix},
\begin{pmatrix}
2\sigma^2 & -\sigma\\
-\sigma& 1
\end{pmatrix}
\right) n ( σ ^ n − σ ξ ^ n − ξ n ) d N ( ( 0 0 ) , ( 2 σ 2 − σ − σ 1 ) ) and
n ( σ ~ n − σ ξ ~ n − ξ n ) → d N ( ( 0 0 ) , ( 2 σ 2 − σ − σ 1 ) ) . \sqrt{n}
\begin{pmatrix}
\tilde{\sigma}_n-\sigma\\
\tilde{\xi}_n-\xi_n
\end{pmatrix}
\xrightarrow{d}
\mathcal{N}\left(
\begin{pmatrix}
0\\
0
\end{pmatrix},
\begin{pmatrix}
2\sigma^2 & -\sigma\\
-\sigma& 1
\end{pmatrix}
\right). n ( σ ~ n − σ ξ ~ n − ξ n ) d N ( ( 0 0 ) , ( 2 σ 2 − σ − σ 1 ) ) . The standard location estimator μ ^ n = μ ~ n = X ( 1 ) \hat{\mu}_n=\tilde{\mu}_n=X_{(1)} μ ^ n = μ ~ n = X ( 1 )
n σ ( μ ^ n − μ ) = n σ ( μ ~ n − μ ) → d Exp ( 1 ) . \frac{n}{\sigma}(\hat{\mu}_n-\mu)=\frac{n}{\sigma}(\tilde{\mu}_n-\mu)\xrightarrow{d}\operatorname{Exp}(1). σ n ( μ ^ n − μ ) = σ n ( μ ~ n − μ ) d Exp ( 1 ) . For comparison, Hosking and Wallis [ [22 ] give an explicit estimator based on probability weighted moments for the two parameter GPD with a known threshold. In the present three-parameter setting, we adapt their formulas to the excesses over X ( 1 ) X_{(1)} X ( 1 )
Discussion
It is often desirable to obtain closed-form estimators of statistical parameters, and it can therefore be worthwhile to replace an implicitly defined root of an estimating equation by an explicit root of a suitable surrogate equation. Such a replacement immediately prompts the question of when such a root remains close to the selected root of the original equation. Our framework answers this question by distinguishing three sources of error: the approximation of the surrogate equation itself, the sensitivity of the surrogate root to perturbations, and the amplification of errors through the inverse chart. While we focus on MLEs, our results apply to general Z Z Z
Our examples show that no single approximation rule works in all cases. The gamma models of § 3 illustrate how increasing the complexity of the underlying statistical model leads to an increase in the complexity of the approximate estimator. The examples in § 4 require different constructions: the zeta example approximates a Dirichlet series and requires us to consider a non-null event on which a finite MLE does not exist, the von Mises example preserves the exact MLE in one coordinate while approximating the other, and the GPD example requires that we carefully rewrite the profiled score equation prior to truncation, and has a separate non-Gaussian limit in one component. Our simulations and real-world data application support our theoretical developments and illustrate the importance of the rate condition stated in each example.
Our framework has some limitations. The construction of an approximate estimator is inherently model dependent: choosing a useful chart, reexpressing the estimating equation in an appropriate form, and selecting the truncation order can each require substantial work. Our framework requires a prespecified limiting regime, so in applications we require some basis for assuming that the data-generating parameter is sufficiently close to the corresponding limiting value. The plausibility of the required rate is model- and application-dependent; our simulations and data example illustrate both successful and visibly inadequate regimes. Although Theorem 1 can establish the existence of a nearby exact root, global existence, uniqueness, and identification of that root as the relevant MLE can remain model-specific, as in the zeta and GPD examples. Finally, while our handling of exceptional events is harmless as n → ∞ n \to\infty n → ∞
There are a number of avenues for future work. Instead of focusing solely on fixed limiting regimes, allowing for uniform control over broad families of parameter sequences would be useful when the rate of approach to the limiting value is unknown. Because our results are first order, a natural question is whether the exact and surrogate roots agree at higher orders. Edgeworth and saddlepoint approximations for roots of estimating equations may provide suitable frameworks for investigating this question [5 , 13 , 11 ] . Showing that the exact and approximate estimators have the same distributional expansions beyond first order would require sharper control of the distance between them.
The bias correction in the gamma regression example of § 3.3 suggests another direction. By removing the leading deterministic centering terms, the correction relaxes the rate condition required for first-order equivalence while preserving both the estimator’s closed form and the properties of an approximate MLE. It would be useful to develop general methods for constructing such corrections and to determine how far the rate restrictions can be relaxed without sacrificing equivalence to the exact estimator.
One may also create a limiting regime by embedding the model under consideration within a larger family, similar to exponential tilting. The GPD example serves as a prototype, since at ξ = 0 \xi= 0 ξ = 0 P θ P_{\theta} P θ P θ , γ P_{\theta,\gamma} P θ , γ P θ , γ P_{\theta,\gamma} P θ , γ P θ P_{\theta} P θ γ → γ ∗ \gamma\to\gamma^{*} γ → γ ∗ θ \theta θ θ \theta θ
R code is available at https://github.com/rob-zimmerman/closed-form-z-estimation.
Acknowledgments
The authors thank Nancy Reid for helpful discussions.
The authors acknowledge the use of ChatGPT to audit a number of mathematical arguments and derivations in the preparation of this paper and to assist in the setup and presentation of our simulation studies, and the use of Mathematica to verify several algebraic manipulations in the appendices.
Technical Results for the Gamma Models
Gamma Moment Identities
Lemma A.1. Let X ∼ Gamma ( α , β ) X \sim\operatorname{Gamma}(\alpha,\beta) X ∼ Gamma ( α , β ) s > − α s > -\alpha s > − α
E [ X s ] = β − s Γ ( α + s ) Γ ( α ) . (48) \mathbb{E}[X^s] = \beta^{-s}\frac{\Gamma(\alpha+s)}{\Gamma(\alpha)}. \tag*{(48)} E [ X s ] = β − s Γ ( α ) Γ ( α + s ) . (48) Moreover, for k ∈ N k \in\mathbb{N} k ∈ N
E [ log ( X ) k ] = ∑ r = 0 k ( k r ) ( − log ( β ) ) r Γ ( k − r ) ( α ) Γ ( α ) . \mathbb{E}\left[\log(X)^k\right] = \sum_{r=0}^{k} \binom{k}{r}(-\log(\beta))^r\frac{\Gamma^{(k-r)}(\alpha)}{\Gamma(\alpha)}. E [ log ( X ) k ] = r = 0 ∑ k ( r k ) ( − log ( β ) ) r Γ ( α ) Γ ( k − r ) ( α ) . Equivalently,
E [ log ( X ) k ] = ∑ r = 0 k ( k r ) ( − log ( β ) ) r Y k − r ( ψ ( α ) , ψ 1 ( α ) , … , ψ k − r − 1 ( α ) ) , (49) \mathbb{E}\left[\log(X)^k\right] = \sum_{r=0}^{k} \binom{k}{r}(-\log(\beta))^rY_{k-r}(\psi(\alpha),\psi_1(\alpha),\ldots,\psi_{k-r-1}(\alpha)), \tag*{(49)} E [ log ( X ) k ] = r = 0 ∑ k ( r k ) ( − log ( β ) ) r Y k − r ( ψ ( α ) , ψ 1 ( α ) , … , ψ k − r − 1 ( α )) , (49) where Y m Y_m Y m m m m Y 0 : = 1 Y_0 := 1 Y 0 := 1
Proof. The Gamma integral gives (A.1) directly:
E [ X s ] = β α Γ ( α ) ∫ 0 ∞ x α + s − 1 e − β x d x = β − s Γ ( α + s ) Γ ( α ) . \mathbb{E}[X^s] = \frac{\beta^\alpha}{\Gamma(\alpha)}\int_0^\infty x^{\alpha+s-1}e^{-\beta x}\,\mathrm{d}x = \beta^{-s}\frac{\Gamma(\alpha+s)}{\Gamma(\alpha)}. E [ X s ] = Γ ( α ) β α ∫ 0 ∞ x α + s − 1 e − β x d x = β − s Γ ( α ) Γ ( α + s ) . For the logarithmic moments, let y : = β x y := \beta x y := β x
E [ log ( X ) k ] = 1 Γ ( α ) ∫ 0 ∞ ( log ( y ) − log ( β ) ) k y α − 1 e − y d y \mathbb{E}\left[\log(X)^k\right] = \frac{1}{\Gamma(\alpha)}\int_0^\infty(\log(y)-\log(\beta))^k y^{\alpha-1}e^{-y}\,\mathrm{d}y E [ log ( X ) k ] = Γ ( α ) 1 ∫ 0 ∞ ( log ( y ) − log ( β ) ) k y α − 1 e − y d y = 1 Γ ( α ) ∑ r = 0 k ( k r ) ( − log ( β ) ) r ∫ 0 ∞ ( log ( y ) ) k − r y α − 1 e − y d y = \frac{1}{\Gamma(\alpha)}\sum_{r=0}^{k}\binom{k}{r}(-\log(\beta))^r\int_0^\infty(\log(y))^{k-r}y^{\alpha-1}e^{-y}\,\mathrm{d}y = Γ ( α ) 1 r = 0 ∑ k ( r k ) ( − log ( β ) ) r ∫ 0 ∞ ( log ( y ) ) k − r y α − 1 e − y d y = ∑ r = 0 k ( k r ) ( − log ( β ) ) r Γ ( k − r ) ( α ) Γ ( α ) . = \sum_{r=0}^{k}\binom{k}{r}(-\log(\beta))^r\frac{\Gamma^{(k-r)}(\alpha)}{\Gamma(\alpha)}. = r = 0 ∑ k ( r k ) ( − log ( β ) ) r Γ ( α ) Γ ( k − r ) ( α ) . Finally, applying the Faà di Bruno formula [10 Theorem 3.4 C] to Γ ( α ) = exp ( log ( Γ ( α ) ) ) \Gamma(\alpha) = \exp(\log(\Gamma(\alpha))) Γ ( α ) = exp ( log ( Γ ( α ))) m ≥ 1 m \ge1 m ≥ 1
Γ ( m ) ( α ) = Γ ( α ) Y m ( ψ ( α ) , ψ 1 ( α ) , … , ψ m − 1 ( α ) ) , \Gamma^{(m)}(\alpha) = \Gamma(\alpha)Y_m(\psi(\alpha),\psi_1(\alpha),\ldots,\psi_{m-1}(\alpha)), Γ ( m ) ( α ) = Γ ( α ) Y m ( ψ ( α ) , ψ 1 ( α ) , … , ψ m − 1 ( α )) , which yields (A.2).
Known Rate
The following results establish the localization, inverse chart expansion, and triangular array central limit theorem used in § 3.1 .
Lemma A.2. Let X 1 , X 2 , … , X n ∼ i i d Gamma ( α , β ) X_1,X_2,\ldots,X_n \overset{\mathrm{iid}}{\sim} \operatorname{Gamma}(\alpha,\beta) X 1 , X 2 , … , X n ∼ iid Gamma ( α , β ) G n : = ( ∏ i = 1 n X i ) 1 / n G_n := \left(\prod_{i=1}^{n}X_i\right)^{1/n} G n := ( ∏ i = 1 n X i ) 1/ n k k k k / n < α k/n < \alpha k / n < α
E [ ( β G n ) − k ] = ( Γ ( α − k / n ) Γ ( α ) ) n . \mathbb{E}\left[(\beta G_n)^{-k}\right] = \left(\frac{\Gamma(\alpha-k/n)}{\Gamma(\alpha)}\right)^n. E [ ( β G n ) − k ] = ( Γ ( α ) Γ ( α − k / n ) ) n . In particular, whenever 2 / n < α 2/n < \alpha 2/ n < α
E [ 1 β G n ] = ( Γ ( α − 1 / n ) Γ ( α ) ) n , \mathbb{E}\left[\frac{1}{\beta G_n}\right] = \left(\frac{\Gamma(\alpha-1/n)}{\Gamma(\alpha)}\right)^n, E [ β G n 1 ] = ( Γ ( α ) Γ ( α − 1/ n ) ) n , and
Var ( 1 β G n ) = ( Γ ( α − 2 / n ) Γ ( α ) ) n − ( Γ ( α − 1 / n ) Γ ( α ) ) 2 n . \operatorname{Var}\left(\frac{1}{\beta G_n}\right) = \left(\frac{\Gamma(\alpha-2/n)}{\Gamma(\alpha)}\right)^n-\left(\frac{\Gamma(\alpha-1/n)}{\Gamma(\alpha)}\right)^{2n}. Var ( β G n 1 ) = ( Γ ( α ) Γ ( α − 2/ n ) ) n − ( Γ ( α ) Γ ( α − 1/ n ) ) 2 n . Proof. Since G n − k = ∏ i = 1 n X i − k / n G_n^{-k}=\prod_{i=1}^{n}X_i^{-k/n} G n − k = ∏ i = 1 n X i − k / n k / n < α k/n<\alpha k / n < α
E [ ( β G n ) − k ] = β − k ∏ i = 1 n E [ X i − k / n ] = β − k ( β k / n Γ ( α − k / n ) Γ ( α ) ) n = ( Γ ( α − k / n ) Γ ( α ) ) n , \mathbb{E}\left[(\beta G_n)^{-k}\right]=\beta^{-k}\prod_{i=1}^{n}\mathbb{E}\left[X_i^{-k/n}\right]=\beta^{-k}\left(\beta^{k/n}\frac{\Gamma(\alpha-k/n)}{\Gamma(\alpha)}\right)^n=\left(\frac{\Gamma(\alpha-k/n)}{\Gamma(\alpha)}\right)^n, E [ ( β G n ) − k ] = β − k i = 1 ∏ n E [ X i − k / n ] = β − k ( β k / n Γ ( α ) Γ ( α − k / n ) ) n = ( Γ ( α ) Γ ( α − k / n ) ) n , from which both the first moment and variance identities follow by taking k = 1 k=1 k = 1 k = 2 k=2 k = 2
Lemma A.3. Let X n , 1 , … , X n , n ∼ i i d Gamma ( α n , β ) X_{n,1},\ldots,X_{n,n}\mathrel{\overset{\mathrm{iid}}{\sim}}\operatorname{Gamma}(\alpha_n,\beta) X n , 1 , … , X n , n ∼ iid Gamma ( α n , β ) β > 0 \beta>0 β > 0 α n → ∞ \alpha_n\to\infty α n → ∞
u n : = 1 α n − 1 / 2 and u ~ n : = 1 β G n . u_n:=\frac{1}{\alpha_n-1/2}\qquad\text{and}\qquad\widetilde{u}_n:=\frac{1}{\beta G_n}. u n := α n − 1/2 1 and u n := β G n 1 . Then
E α n [ u ~ n ] = u n + O ( u n 2 n ) + O ( u n 3 ) and Var α n ( u ~ n ) = O ( u n 3 n ) , (50) \mathbb{E}_{\alpha_n}[\widetilde{u}_n]=u_n+O\left(\frac{u_n^2}{n}\right)+O(u_n^3)\qquad\text{and}\qquad\operatorname{Var}_{\alpha_n}(\widetilde{u}_n)=O\left(\frac{u_n^3}{n}\right), \tag*{(50)} E α n [ u n ] = u n + O ( n u n 2 ) + O ( u n 3 ) and Var α n ( u n ) = O ( n u n 3 ) , (50) and hence
u ~ n − u n = O P ( u n 3 n ) + O ( u n 2 n ) + O ( u n 3 ) = o P ( u n ) . (51) \widetilde{u}_n-u_n=O_{\mathbb{P}}\left(\sqrt{\frac{u_n^3}{n}}\right)+O\left(\frac{u_n^2}{n}\right)+O(u_n^3)=o_{\mathbb{P}}(u_n). \tag*{(51)} u n − u n = O P ( n u n 3 ) + O ( n u n 2 ) + O ( u n 3 ) = o P ( u n ) . (51) Proof. For fixed k ∈ { 1 , 2 } k\in\{1,2\} k ∈ { 1 , 2 }
L k , n : = n ( log ( Γ ( α n − k n ) ) − log ( Γ ( α n ) ) ) . L_{k,n}:=n\left(\log\left(\Gamma\left(\alpha_n-\frac{k}{n}\right)\right)-\log(\Gamma(\alpha_n))\right). L k , n := n ( log ( Γ ( α n − n k ) ) − log ( Γ ( α n )) ) . Applying Taylor’s theorem to the function x ↦ log ( Γ ( α n − x ) ) x\mapsto\log(\Gamma(\alpha_n-x)) x ↦ log ( Γ ( α n − x )) x = 0 x=0 x = 0 x = k / n x=k/n x = k / n
L k , n = − k ψ ( α n ) + k 2 2 n ψ 1 ( α n ) + O ( 1 n 2 α n 2 ) . (52) L_{k,n}=-k\psi(\alpha_n)+\frac{k^2}{2n}\psi_1(\alpha_n)+O\left(\frac{1}{n^2\alpha_n^2}\right). \tag*{(52)} L k , n = − k ψ ( α n ) + 2 n k 2 ψ 1 ( α n ) + O ( n 2 α n 2 1 ) . (52) Before multiplication by n n n
C ( k n ) 3 sup x ∈ [ α n − k / n , α n ] ∣ ψ 2 ( x ) ∣ . C\left(\frac{k}{n}\right)^3\sup_{x\in[\alpha_n-k/n,\alpha_n]}|\psi_2(x)|. C ( n k ) 3 x ∈ [ α n − k / n , α n ] sup ∣ ψ 2 ( x ) ∣. Now,
ψ 2 ( x ) = − 2 ∑ j = 0 ∞ 1 ( x + j ) 3 \psi_2(x)=-2\sum_{j=0}^{\infty}\frac{1}{(x+j)^3} ψ 2 ( x ) = − 2 j = 0 ∑ ∞ ( x + j ) 3 1 and so
∣ ψ 2 ( x ) ∣ = 2 ∑ j = 0 ∞ 1 ( x + j ) 3 ≤ 2 x 3 + 2 ∫ 0 ∞ d t ( x + t ) 3 = 2 x 3 + 1 x 2 = O ( 1 x 2 ) , |\psi_2(x)|=2\sum_{j=0}^{\infty}\frac{1}{(x+j)^3}\leq\frac{2}{x^3}+2\int_0^\infty\frac{\mathrm{d}t}{(x+t)^3}=\frac{2}{x^3}+\frac{1}{x^2}=O\left(\frac{1}{x^2}\right), ∣ ψ 2 ( x ) ∣ = 2 j = 0 ∑ ∞ ( x + j ) 3 1 ≤ x 3 2 + 2 ∫ 0 ∞ ( x + t ) 3 d t = x 3 2 + x 2 1 = O ( x 2 1 ) , where the first equality is Equation 6.4.10 of Abramowitz and Stegun [1965]``. Thus ψ 2 ( x ) = O ( x − 2 ) \psi_2(x)=O(x^{-2}) ψ 2 ( x ) = O ( x − 2 ) [ α n − k / n , α n ] [\alpha_n-k/n,\alpha_n] [ α n − k / n , α n ] n n n α n \alpha_n α n [2 Equations 6.3.18 and 6.4.12] , together with u n = ( α n − 1 / 2 ) − 1 u_n=(\alpha_n-1/2)^{-1} u n = ( α n − 1/2 ) − 1
e − ψ ( α n ) = u n + O ( u n 3 ) and ψ 1 ( α n ) = u n + O ( u n 3 ) . e^{-\psi(\alpha_n)}=u_n+O(u_n^3)\qquad\text{and}\qquad\psi_1(\alpha_n)=u_n+O(u_n^3). e − ψ ( α n ) = u n + O ( u n 3 ) and ψ 1 ( α n ) = u n + O ( u n 3 ) . Now, by Lemma A.2, E α n [ u ~ n k ] = e L k , n \mathbb{E}_{\alpha_n}[\widetilde{u}_n^k]=e^{L_{k,n}} E α n [ u n k ] = e L k , n k = 1 k=1 k = 1
E α n [ u ~ n ] = e − ψ ( α n ) ( 1 + O ( u n n ) ) = u n + O ( u n 2 n ) + O ( u n 3 ) . (53) \mathbb{E}_{\alpha_n}[\widetilde{u}_n]=e^{-\psi(\alpha_n)}\left(1+O\left(\frac{u_n}{n}\right)\right)=u_n+O\left(\frac{u_n^2}{n}\right)+O(u_n^3). \tag*{(53)} E α n [ u n ] = e − ψ ( α n ) ( 1 + O ( n u n ) ) = u n + O ( n u n 2 ) + O ( u n 3 ) . (53) Moreover,
L 2 , n − 2 L 1 , n = ψ 1 ( α n ) n + O ( 1 n 2 α n 2 ) = O ( u n n ) , L_{2,n}-2L_{1,n}=\frac{\psi_1(\alpha_n)}{n}+O\left(\frac{1}{n^2\alpha_n^2}\right)=O\left(\frac{u_n}{n}\right), L 2 , n − 2 L 1 , n = n ψ 1 ( α n ) + O ( n 2 α n 2 1 ) = O ( n u n ) , and e 2 L 1 , n = O ( u n 2 ) e^{2L_{1,n}}=O(u_n^2) e 2 L 1 , n = O ( u n 2 )
Var α n ( u ~ n ) = e 2 L 1 , n ( e L 2 , n − 2 L 1 , n − 1 ) = O ( u n 3 n ) . \operatorname{Var}_{\alpha_n}(\widetilde{u}_n)=e^{2L_{1,n}}\left(e^{L_{2,n}-2L_{1,n}}-1\right)=O\left(\frac{u_n^3}{n}\right). Var α n ( u n ) = e 2 L 1 , n ( e L 2 , n − 2 L 1 , n − 1 ) = O ( n u n 3 ) . Chebyshev’s inequality then gives, for sufficiently large n n n M > 0 M>0 M > 0
P α n ( ∣ u ~ n − E α n [ u ~ n ] ∣ > M u n 3 n ) ≤ C M 2 , \mathbb{P}_{\alpha_n}\left(\left|\widetilde{u}_n-\mathbb{E}_{\alpha_n}[\widetilde{u}_n]\right|>M\sqrt{\frac{u_n^3}{n}}\right)\leq\frac{C}{M^2}, P α n ( ∣ u n − E α n [ u n ] ∣ > M n u n 3 ) ≤ M 2 C , which, in combination with (A.6), yields the first equality in (A.4). Since
u n 3 n = o ( u n ) and u n 2 n = o ( u n ) and u n 3 = o ( u n ) \sqrt{\frac{u_n^3}{n}}=o(u_n)\quad\text{and}\quad\frac{u_n^2}{n}=o(u_n)\quad\text{and}\quad u_n^3=o(u_n) n u n 3 = o ( u n ) and n u n 2 = o ( u n ) and u n 3 = o ( u n ) the second equality follows.
Lemma A.4. Let X α ∼ Gamma ( α , β ) X_\alpha\sim\operatorname{Gamma}(\alpha,\beta) X α ∼ Gamma ( α , β ) β > 0 \beta>0 β > 0
Y α : = log ( X α ) , μ α : = ψ ( α ) − log ( β ) and σ α 2 : = ψ 1 ( α ) . Y_\alpha:=\log(X_\alpha),\qquad\mu_\alpha:=\psi(\alpha)-\log(\beta)\quad\text{and}\quad\sigma_\alpha^2:=\psi_1(\alpha). Y α := log ( X α ) , μ α := ψ ( α ) − log ( β ) and σ α 2 := ψ 1 ( α ) . There exist α 0 > 0 \alpha_0>0 α 0 > 0 C < ∞ C<\infty C < ∞
sup α ≥ α 0 E [ ∣ Y α − μ α σ α ∣ 3 ] ≤ C . (54) \sup_{\alpha\geq\alpha_0}\mathbb{E}\left[\left|\frac{Y_\alpha-\mu_\alpha}{\sigma_\alpha}\right|^3\right]\leq C. \tag*{(54)} α ≥ α 0 sup E [ σ α Y α − μ α 3 ] ≤ C . (54) Consequently, if X n , 1 , … , X n , n ∼ i i d Gamma ( α n , β ) X_{n,1},\ldots,X_{n,n}\mathrel{\overset{\mathrm{iid}}{\sim}}\operatorname{Gamma}(\alpha_n,\beta) X n , 1 , … , X n , n ∼ iid Gamma ( α n , β ) α n → ∞ \alpha_n\to\infty α n → ∞ Y n , i : = log ( X n , i ) Y_{n,i}:=\log(X_{n,i}) Y n , i := log ( X n , i )
Proof. Without loss of generality, we may assume that β = 1 \beta=1 β = 1 β \beta β log ( X α ) \log(X_\alpha) log ( X α )
Z α : = Y α − ψ ( α ) ψ 1 ( α ) . Z_\alpha:=\frac{Y_\alpha-\psi(\alpha)}{\sqrt{\psi_1(\alpha)}}. Z α := ψ 1 ( α ) Y α − ψ ( α ) . For fixed t t t
h α ( t ) : = t ψ 1 ( α ) , h_\alpha(t):=\frac{t}{\sqrt{\psi_1(\alpha)}}, h α ( t ) := ψ 1 ( α ) t , Lemma A.1 gives
E [ e t Z α ] = Γ ( α + h α ( t ) ) Γ ( α ) exp ( − h α ( t ) ψ ( α ) ) , \mathbb{E}\left[e^{tZ_\alpha}\right]=\frac{\Gamma(\alpha+h_\alpha(t))}{\Gamma(\alpha)}\exp\left(-h_\alpha(t)\psi(\alpha)\right), E [ e t Z α ] = Γ ( α ) Γ ( α + h α ( t )) exp ( − h α ( t ) ψ ( α ) ) , so
log ( E [ e t Z α ] ) = log ( Γ ( α + h α ( t ) ) ) − log ( Γ ( α ) ) − h α ( t ) ψ ( α ) . \log\left(\mathbb{E}\left[e^{tZ_\alpha}\right]\right)=\log\left(\Gamma(\alpha+h_\alpha(t))\right)-\log(\Gamma(\alpha))-h_\alpha(t)\psi(\alpha). log ( E [ e t Z α ] ) = log ( Γ ( α + h α ( t )) ) − log ( Γ ( α )) − h α ( t ) ψ ( α ) . Since ψ 1 ( α ) ∼ α − 1 \psi_1(\alpha)\sim\alpha^{-1} ψ 1 ( α ) ∼ α − 1 α → ∞ \alpha\to\infty α → ∞ h α ( t ) = O ( α ) h_\alpha(t)=O(\sqrt{\alpha}) h α ( t ) = O ( α ) t t t
log ( E [ e t Z α ] ) = h α ( t ) 2 2 ψ 1 ( α ) + h α ( t ) 3 6 ψ 2 ( ξ α , t ) = t 2 2 + O ( α − 1 / 2 ) , (55) \log\left(\mathbb{E}\left[e^{tZ_\alpha}\right]\right)=\frac{h_\alpha(t)^2}{2}\psi_1(\alpha)+\frac{h_\alpha(t)^3}{6}\psi_2(\xi_{\alpha,t})=\frac{t^2}{2}+O\left(\alpha^{-1/2}\right), \tag*{(55)} log ( E [ e t Z α ] ) = 2 h α ( t ) 2 ψ 1 ( α ) + 6 h α ( t ) 3 ψ 2 ( ξ α , t ) = 2 t 2 + O ( α − 1/2 ) , (55) where ξ α , t \xi_{\alpha,t} ξ α , t α \alpha α α + h α ( t ) \alpha+h_\alpha(t) α + h α ( t ) ψ 2 ( x ) = O ( x − 2 ) \psi_2(x)=O(x^{-2}) ψ 2 ( x ) = O ( x − 2 ) α \alpha α ξ α , t ≥ α / 2 \xi_{\alpha,t}\geq\alpha/2 ξ α , t ≥ α /2 η > 0 \eta>0 η > 0
sup α ≥ α 0 ( E [ e η Z α ] + E [ e − η Z α ] ) < ∞ . \sup_{\alpha\geq\alpha_0}\left(\mathbb{E}\left[e^{\eta Z_\alpha}\right]+\mathbb{E}\left[e^{-\eta Z_\alpha}\right]\right)<\infty. α ≥ α 0 sup ( E [ e η Z α ] + E [ e − η Z α ] ) < ∞. Since
e η ∣ x ∣ ≤ e η x + e − η x , e^{\eta|x|}\leq e^{\eta x}+e^{-\eta x}, e η ∣ x ∣ ≤ e η x + e − η x , we thus have
sup α ≥ α 0 E [ e η ∣ Z α ∣ ] < ∞ . \sup_{\alpha\ge\alpha_0} \mathbb{E}\left[e^{\eta|Z_\alpha|}\right] < \infty. α ≥ α 0 sup E [ e η ∣ Z α ∣ ] < ∞. Using the elementary bound
x 3 ≤ 6 η 3 e η x , (56) x^3 \le\frac{6}{\eta^3}e^{\eta x}, \tag*{(56)} x 3 ≤ η 3 6 e η x , (56) valid for all x ≥ 0 x \ge0 x ≥ 0
sup α ≥ α 0 E [ ∣ Z α ∣ 3 ] ≤ 6 η 3 sup α ≥ α 0 E [ e η ∣ Z α ∣ ] < ∞ , \sup_{\alpha\ge\alpha_0}\mathbb{E}\left[|Z_\alpha|^3\right] \le\frac{6}{\eta^3}\sup_{\alpha\ge\alpha_0}\mathbb{E}\left[e^{\eta|Z_\alpha|}\right] < \infty, α ≥ α 0 sup E [ ∣ Z α ∣ 3 ] ≤ η 3 6 α ≥ α 0 sup E [ e η ∣ Z α ∣ ] < ∞ , which proves (A.7). Finally, writing
Z n , i : = Y n , i − μ α n ψ 1 ( α n ) , Z_{n,i}:=\frac{Y_{n,i}-\mu_{\alpha_n}}{\sqrt{\psi_1(\alpha_n)}}, Z n , i := ψ 1 ( α n ) Y n , i − μ α n , we have that
n E [ ∣ Y n , 1 − μ α n ∣ 3 ] ( n ψ 1 ( α n ) ) 3 / 2 = E [ ∣ Z n , 1 ∣ 3 ] n ⟶ 0 , \frac{n\mathbb{E}\left[|Y_{n,1}-\mu_{\alpha_n}|^3\right]}{(n\psi_1(\alpha_n))^{3/2}}=\frac{\mathbb{E}\left[|Z_{n,1}|^3\right]}{\sqrt{n}}\longrightarrow0, ( n ψ 1 ( α n ) ) 3/2 n E [ ∣ Y n , 1 − μ α n ∣ 3 ] = n E [ ∣ Z n , 1 ∣ 3 ] ⟶ 0 , so the Lyapunov condition holds.
Lemma A.5. Let u n ↓ 0 u_n \downarrow0 u n ↓ 0 V n V_n V n
V n − u n = O P ( u n 3 n ) V_n-u_n=O_P\left(\sqrt{\frac{u_n^3}{n}}\right) V n − u n = O P ( n u n 3 ) and
P ( V n ∈ [ u n 2 , 2 u n ] ) ⟶ 1. \mathbb{P}\left(V_n\in\left[\frac{u_n}{2},2u_n\right]\right)\longrightarrow1. P ( V n ∈ [ 2 u n , 2 u n ] ) ⟶ 1. Then for ϕ − 1 ( u ) = 1 / 2 + 1 / u \phi^{-1}(u)=1/2+1/u ϕ − 1 ( u ) = 1/2 + 1/ u
n u n ( ϕ − 1 ( V n ) − ϕ − 1 ( u n ) ) = − n u n 3 ( V n − u n ) + o P ( 1 ) . \sqrt{nu_n}\left(\phi^{-1}(V_n)-\phi^{-1}(u_n)\right)=-\sqrt{\frac{n}{u_n^3}}(V_n-u_n)+o_P(1). n u n ( ϕ − 1 ( V n ) − ϕ − 1 ( u n ) ) = − u n 3 n ( V n − u n ) + o P ( 1 ) . Proof. The exact identity
1 V n − 1 u n = − V n − u n u n 2 + ( V n − u n ) 2 u n 2 V n \frac{1}{V_n}-\frac{1}{u_n}=-\frac{V_n-u_n}{u_n^2}+\frac{(V_n-u_n)^2}{u_n^2V_n} V n 1 − u n 1 = − u n 2 V n − u n + u n 2 V n ( V n − u n ) 2 shows that, on { V n ∈ [ u n / 2 , 2 u n ] } \{V_n\in[u_n/2,2u_n]\} { V n ∈ [ u n /2 , 2 u n ]} 2 ( V n − u n ) 2 / u n 3 = O P ( 1 / n ) 2(V_n-u_n)^2/u_n^3=O_P(1/n) 2 ( V n − u n ) 2 / u n 3 = O P ( 1/ n ) n u n \sqrt{nu_n} n u n
O P ( u n n ) = o P ( 1 ) . O_P\left(\sqrt{\frac{u_n}{n}}\right)=o_P(1). O P ( n u n ) = o P ( 1 ) . Since P ( V n ∈ [ u n / 2 , 2 u n ] ) ⟶ 1 \mathbb{P}(V_n\in[u_n/2,2u_n])\longrightarrow1 P ( V n ∈ [ u n /2 , 2 u n ]) ⟶ 1 o P ( 1 ) o_P(1) o P ( 1 )
Unknown Rate
Lemma A.6. Let X 1 , … , X n ∼ i i d Gamma ( α , β ) X_1,\ldots,X_n \mathrel{\mathop{\sim}^{\mathrm{iid}}} \operatorname{Gamma}(\alpha,\beta) X 1 , … , X n ∼ iid Gamma ( α , β ) n α > 2 n\alpha>2 n α > 2
E [ 1 X ‾ n ] = n β n α − 1 . \mathbb{E}\left[\frac{1}{\overline{X}_n}\right]=\frac{n\beta}{n\alpha-1}. E [ X n 1 ] = n α − 1 n β .
Var ( 1 X ‾ n ) = n 2 β 2 ( n α − 1 ) 2 ( n α − 2 ) . \operatorname{Var}\left(\frac{1}{\overline{X}_n}\right)=\frac{n^2\beta^2}{(n\alpha-1)^2(n\alpha-2)}. Var ( X n 1 ) = ( n α − 1 ) 2 ( n α − 2 ) n 2 β 2 .
E [ log ( X ‾ n G n ) ] = ψ ( n α ) − ψ ( α ) − log ( n ) . \mathbb{E}\left[\log\left(\frac{\overline{X}_n}{G_n}\right)\right]=\psi(n\alpha)-\psi(\alpha)-\log(n). E [ log ( G n X n ) ] = ψ ( n α ) − ψ ( α ) − log ( n ) . (iv)
Var ( log ( X ‾ n G n ) ) = ψ 1 ( α ) n − ψ 1 ( n α ) . \operatorname{Var}\left(\log\left(\frac{\overline{X}_n}{G_n}\right)\right)=\frac{\psi_1(\alpha)}{n}-\psi_1(n\alpha). Var ( log ( G n X n ) ) = n ψ 1 ( α ) − ψ 1 ( n α ) . Proof. Let
S n : = ∑ i = 1 n X i . S_n:=\sum_{i=1}^{n}X_i. S n := i = 1 ∑ n X i . Since S n ∼ Gamma ( n α , β ) S_n\sim\operatorname{Gamma}(n\alpha,\beta) S n ∼ Gamma ( n α , β )
X ‾ n = S n n ∼ Gamma ( n α , n β ) . \overline{X}_n=\frac{S_n}{n}\sim\operatorname{Gamma}(n\alpha,n\beta). X n = n S n ∼ Gamma ( n α , n β ) . The negative gamma moments in Lemma A.1 give
E [ 1 X ‾ n ] = n β n α − 1 and E [ 1 X ‾ n 2 ] = n 2 β 2 ( n α − 1 ) ( n α − 2 ) . \mathbb{E}\left[\frac{1}{\overline{X}_n}\right]=\frac{n\beta}{n\alpha-1}
\quad\text{and}\quad
\mathbb{E}\left[\frac{1}{\overline{X}_n^2}\right]=\frac{n^2\beta^2}{(n\alpha-1)(n\alpha-2)}. E [ X n 1 ] = n α − 1 n β and E [ X n 2 1 ] = ( n α − 1 ) ( n α − 2 ) n 2 β 2 . Subtracting the square of the first expectation from the second proves parts (i) and (ii).
For the remaining two claims, (A.2) in Lemma A.1 with k = 1 , 2 k=1,2 k = 1 , 2
log ( X ‾ n G n ) = log ( S n ) − log ( n ) − 1 n ∑ i = 1 n log ( X i ) . \log\left(\frac{\overline{X}_n}{G_n}\right)=\log(S_n)-\log(n)-\frac{1}{n}\sum_{i=1}^{n}\log(X_i). log ( G n X n ) = log ( S n ) − log ( n ) − n 1 i = 1 ∑ n log ( X i ) . It follows that
E [ log ( X ‾ n G n ) ] = ( ψ ( n α ) − log ( β ) ) − log ( n ) − ( ψ ( α ) − log ( β ) ) = ψ ( n α ) − ψ ( α ) − log ( n ) , \mathbb{E}\left[\log\left(\frac{\overline{X}_n}{G_n}\right)\right]=(\psi(n\alpha)-\log(\beta))-\log(n)-(\psi(\alpha)-\log(\beta))=\psi(n\alpha)-\psi(\alpha)-\log(n), E [ log ( G n X n ) ] = ( ψ ( n α ) − log ( β )) − log ( n ) − ( ψ ( α ) − log ( β )) = ψ ( n α ) − ψ ( α ) − log ( n ) , which is (iii). For (iv), write P i : = X i / S n P_i:=X_i/S_n P i := X i / S n [24 Chapter 49] , S n S_n S n ( P 1 , … , P n ) (P_1,\ldots,P_n) ( P 1 , … , P n )
log ( X i ) = log ( S n ) + log ( P i ) . \log(X_i)=\log(S_n)+\log(P_i). log ( X i ) = log ( S n ) + log ( P i ) . It follows that
Cov ( log ( S n ) , log ( X i ) ) = Var ( log ( S n ) ) = ψ 1 ( n α ) . \operatorname{Cov}(\log(S_n),\log(X_i))=\operatorname{Var}(\log(S_n))=\psi_1(n\alpha). Cov ( log ( S n ) , log ( X i )) = Var ( log ( S n )) = ψ 1 ( n α ) . Since the X i X_i X i
Var ( 1 n ∑ i = 1 n log ( X i ) ) = ψ 1 ( α ) n . \operatorname{Var}\left(\frac{1}{n}\sum_{i=1}^{n}\log(X_i)\right)=\frac{\psi_1(\alpha)}{n}. Var ( n 1 i = 1 ∑ n log ( X i ) ) = n ψ 1 ( α ) . Therefore
Var ( log ( X ‾ n G n ) ) = ψ 1 ( n α ) + ψ 1 ( α ) n − 2 ψ 1 ( n α ) = ψ 1 ( α ) n − ψ 1 ( n α ) . \operatorname{Var}\left(\log\left(\frac{\overline{X}_n}{G_n}\right)\right)=\psi_1(n\alpha)+\frac{\psi_1(\alpha)}{n}-2\psi_1(n\alpha)=\frac{\psi_1(\alpha)}{n}-\psi_1(n\alpha). Var ( log ( G n X n ) ) = ψ 1 ( n α ) + n ψ 1 ( α ) − 2 ψ 1 ( n α ) = n ψ 1 ( α ) − ψ 1 ( n α ) . Lemma A.7. Let
X n , 1 , … , X n , n ∼ i i d Gamma ( α n , β ) X_{n,1},\ldots,X_{n,n}\mathrel{\overset{\mathrm{iid}}{\sim}}\operatorname{Gamma}(\alpha_n,\beta) X n , 1 , … , X n , n ∼ iid Gamma ( α n , β ) and α n → ∞ \alpha_n\to\infty α n → ∞ β > 0 \beta>0 β > 0 ( α n , β ) (\alpha_n,\beta) ( α n , β )
S n , i : = ( log ( β ) + log ( X n , i ) − ψ ( α n ) α n β − X n , i ) . S_{n,i}:=
\begin{pmatrix}
\log(\beta)+\log(X_{n,i})-\psi(\alpha_n)\\
\frac{\alpha_n}{\beta}-X_{n,i}
\end{pmatrix}. S n , i := ( log ( β ) + log ( X n , i ) − ψ ( α n ) β α n − X n , i ) . Then its covariance matrix is given by
I n : = Var ( S n , i ) = ( ψ 1 ( α n ) − 1 β − 1 β − 1 β α n β 2 ) , (57) \mathcal{I}_n := \operatorname{Var}(S_{n,i}) =
\begin{pmatrix}
\psi_1(\alpha_n) - \frac{1}{\beta} & -\frac{1}{\beta} \\
-\frac{1}{\beta} & \frac{\alpha_n}{\beta^2}
\end{pmatrix}, \tag*{(57)} I n := Var ( S n , i ) = ( ψ 1 ( α n ) − β 1 − β 1 − β 1 β 2 α n ) , (57) and there exist α 0 > 0 \alpha_0 > 0 α 0 > 0 C < ∞ C < \infty C < ∞
sup α ≥ α 0 sup ∥ t ∥ = 1 E α , β [ ∣ t T I α − 1 2 S α ∣ 3 ] ≤ C , (58) \sup_{\alpha\ge\alpha_0}\sup_{\lVert t\rVert=1}\mathbb{E}_{\alpha,\beta}\left[\left\lvert t^{\mathsf{T}}\mathcal{I}_{\alpha}^{-\frac{1}{2}}S_\alpha\right\rvert^3\right] \le C, \tag*{(58)} α ≥ α 0 sup ∥ t ∥ = 1 sup E α , β [ t T I α − 2 1 S α 3 ] ≤ C , (58) where S α S_\alpha S α I α \mathcal{I}_\alpha I α α n \alpha_n α n α \alpha α
1 n I n − 1 2 ∑ i = 1 n S n , i → d N ( 0 , I 2 ) . \frac{1}{\sqrt{n}}\mathcal{I}_n^{-\frac{1}{2}}\sum_{i=1}^{n}S_{n,i}\xrightarrow{d}\mathcal{N}(0,\mathbb{I}_2). n 1 I n − 2 1 i = 1 ∑ n S n , i d N ( 0 , I 2 ) . Proof. Let
W α : = β X α ∼ Gamma ( α , 1 ) W_\alpha:= \beta X_\alpha\sim\operatorname{Gamma}(\alpha,1) W α := β X α ∼ Gamma ( α , 1 ) and define
v α : = ψ 1 ( α ) − 1 α . v_\alpha:= \psi_1(\alpha) - \frac{1}{\alpha}. v α := ψ 1 ( α ) − α 1 . Since
ψ 1 ( α ) = ∑ k = 0 ∞ 1 ( α + k ) 2 > ∫ 0 ∞ d x ( α + x ) 2 = 1 α , \psi_1(\alpha)=\sum_{k=0}^{\infty}\frac{1}{(\alpha+k)^2}>\int_0^{\infty}\frac{\mathrm{d}x}{(\alpha+x)^2}=\frac{1}{\alpha}, ψ 1 ( α ) = k = 0 ∑ ∞ ( α + k ) 2 1 > ∫ 0 ∞ ( α + x ) 2 d x = α 1 , we have v α > 0 v_\alpha> 0 v α > 0
v α = 1 2 α 2 + 1 6 α 3 + O ( 1 α 5 ) . v_\alpha=\frac{1}{2\alpha^2}+\frac{1}{6\alpha^3}+O\left(\frac{1}{\alpha^5}\right). v α = 2 α 2 1 + 6 α 3 1 + O ( α 5 1 ) . Set
T α : = log ( W α ) − ψ ( α ) − ( W α − α ) / α v α and Z α : = W α − α α . T_\alpha:=\frac{\log(W_\alpha)-\psi(\alpha)-(W_\alpha-\alpha)/\alpha}{\sqrt{v_\alpha}}
\qquad\text{and}\qquad
Z_\alpha:=\frac{W_\alpha-\alpha}{\sqrt{\alpha}}. T α := v α log ( W α ) − ψ ( α ) − ( W α − α ) / α and Z α := α W α − α . To verify the covariance identity (A.10), let W ~ α ∼ Gamma ( α + 1 , 1 ) \widetilde{W}_\alpha\sim\operatorname{Gamma}(\alpha+1,1) W α ∼ Gamma ( α + 1 , 1 ) Gamma ( α , 1 ) \operatorname{Gamma}(\alpha,1) Gamma ( α , 1 ) w w w α \alpha α α + 1 \alpha+1 α + 1 α \alpha α
E [ W α log ( W α ) ] = α E [ log ( W ~ α ) ] = α ψ ( α + 1 ) . \mathbb{E}[W_\alpha\log(W_\alpha)]=\alpha\mathbb{E}\left[\log\left(\widetilde{W}_\alpha\right)\right]=\alpha\psi(\alpha+1). E [ W α log ( W α )] = α E [ log ( W α ) ] = α ψ ( α + 1 ) . Since E [ W α ] = α \mathbb{E}[W_\alpha]=\alpha E [ W α ] = α E [ log ( W α ) ] = ψ ( α ) \mathbb{E}[\log(W_\alpha)]=\psi(\alpha) E [ log ( W α )] = ψ ( α ) ψ ( α + 1 ) = ψ ( α ) + 1 / α \psi(\alpha+1)=\psi(\alpha)+1/\alpha ψ ( α + 1 ) = ψ ( α ) + 1/ α
Cov ( log ( W α ) , W α ) = 1. \operatorname{Cov}(\log(W_\alpha),W_\alpha)=1. Cov ( log ( W α ) , W α ) = 1. Together with
Var ( log ( W α ) ) = ψ 1 ( α ) and Var ( W α ) = α , \operatorname{Var}(\log(W_\alpha))=\psi_1(\alpha)\quad\text{and}\quad\operatorname{Var}(W_\alpha)=\alpha, Var ( log ( W α )) = ψ 1 ( α ) and Var ( W α ) = α , which follow from Lemma A.1, this proves (A.10). The same identities also give
Var ( T α ) = 1 and Var ( Z α ) = 1 and Cov ( T α , Z α ) = 0. \operatorname{Var}(T_\alpha)=1\quad\text{and}\quad\operatorname{Var}(Z_\alpha)=1\quad\text{and}\quad\operatorname{Cov}(T_\alpha,Z_\alpha)=0. Var ( T α ) = 1 and Var ( Z α ) = 1 and Cov ( T α , Z α ) = 0. Equivalently, if
D α : = ( v α − 1 2 β α v α 0 − β α ) , D_\alpha:=
\begin{pmatrix}
v_\alpha^{-\frac{1}{2}} & \frac{\beta}{\alpha\sqrt{v_\alpha}} \\
0 & -\frac{\beta}{\sqrt{\alpha}}
\end{pmatrix}, D α := ( v α − 2 1 0 α v α β − α β ) , then
D α S α = ( T α Z α ) and D α I α D α T = I 2 . \mathbf{D}_{\alpha}\mathbf{S}_{\alpha} =
\begin{pmatrix}
T_{\alpha}\\
Z_{\alpha}
\end{pmatrix}
\quad\text{and}\quad
\mathbf{D}_{\alpha}\mathcal{I}_{\alpha}\mathbf{D}_{\alpha}^{\mathsf{T}}=\mathbf{I}_2. D α S α = ( T α Z α ) and D α I α D α T = I 2 . Hence
O α : = D α I α 1 2 \mathbf{O}_{\alpha}:=\mathbf{D}_{\alpha}\mathcal{I}_{\alpha}^{\frac{1}{2}} O α := D α I α 2 1 is orthogonal, and therefore
I α − 1 2 S α = O α T ( T α Z α ) . \mathcal{I}_{\alpha}^{-\frac{1}{2}}\mathbf{S}_{\alpha}
=\mathbf{O}_{\alpha}^{\mathsf{T}}
\begin{pmatrix}
T_{\alpha}\\
Z_{\alpha}
\end{pmatrix}. I α − 2 1 S α = O α T ( T α Z α ) . Next, we establish uniform bounds for the third moments of T α T_{\alpha} T α Z α Z_{\alpha} Z α ( α , 1 ) (\alpha,1) ( α , 1 ) [40 ] and Hölder’s inequality give
sup α ≥ α 0 E [ ∣ Z α ∣ 3 ] ≤ sup α ≥ α 0 ( E [ Z α 4 ] ) 3 4 = ( 3 + 6 α 0 ) 3 4 < ∞ . \sup_{\alpha\geq\alpha_0}\mathbb{E}\left[|Z_{\alpha}|^3\right]
\leq
\sup_{\alpha\geq\alpha_0}\left(\mathbb{E}\left[Z_{\alpha}^4\right]\right)^{\frac{3}{4}}
=
\left(3+\frac{6}{\alpha_0}\right)^{\frac{3}{4}}<\infty. α ≥ α 0 sup E [ ∣ Z α ∣ 3 ] ≤ α ≥ α 0 sup ( E [ Z α 4 ] ) 4 3 = ( 3 + α 0 6 ) 4 3 < ∞. For T α T_{\alpha} T α
ξ α : = log ( W α ) − ψ ( α ) − W α − α α and T α = ξ α v α . \xi_{\alpha}:=\log(W_{\alpha})-\psi(\alpha)-\frac{W_{\alpha}-\alpha}{\alpha}
\quad\text{and}\quad
T_{\alpha}=\frac{\xi_{\alpha}}{\sqrt{v_{\alpha}}}. ξ α := log ( W α ) − ψ ( α ) − α W α − α and T α = v α ξ α . For r > − α r>-\alpha r > − α
E [ e r ξ α ] = e r ( 1 − ψ ( α ) ) Γ ( α + r ) Γ ( α ) ( 1 + r α ) − ( α + r ) . \mathbb{E}\left[e^{r\xi_{\alpha}}\right]
=e^{r(1-\psi(\alpha))}
\frac{\Gamma(\alpha+r)}{\Gamma(\alpha)}
\left(1+\frac{r}{\alpha}\right)^{-(\alpha+r)}. E [ e r ξ α ] = e r ( 1 − ψ ( α )) Γ ( α ) Γ ( α + r ) ( 1 + α r ) − ( α + r ) . For s s s
r α ( s ) : = s v α and c α ( s ) : = r α ( s ) α = s α v α . r_{\alpha}(s):=\frac{s}{\sqrt{v_{\alpha}}}
\quad\text{and}\quad
c_{\alpha}(s):=\frac{r_{\alpha}(s)}{\alpha}
=\frac{s}{\alpha\sqrt{v_{\alpha}}}. r α ( s ) := v α s and c α ( s ) := α r α ( s ) = α v α s . Since
α v α ⟶ 1 2 , \alpha\sqrt{v_{\alpha}}\longrightarrow\frac{1}{\sqrt{2}}, α v α ⟶ 2 1 , we may choose η > 0 \eta>0 η > 0 α 0 \alpha_0 α 0
∣ c α ( s ) ∣ ≤ 1 2 for all α ≥ α 0 and ∣ s ∣ ≤ η . |c_{\alpha}(s)|\leq\frac{1}{2}
\quad\text{for all }\alpha\geq\alpha_0\text{ and }|s|\leq\eta. ∣ c α ( s ) ∣ ≤ 2 1 for all α ≥ α 0 and ∣ s ∣ ≤ η . Applying Stirling’s formula to α ( 1 + c ) \alpha(1+c) α ( 1 + c ) α \alpha α α ( 1 + c ) ≥ α / 2 \alpha(1+c)\geq\alpha/2 α ( 1 + c ) ≥ α /2 ∣ c ∣ ≤ 1 / 2 |c|\leq1/2 ∣ c ∣ ≤ 1/2
log ( Γ ( α ( 1 + c ) ) ) − log ( Γ ( α ) ) = α c log ( α ) + α ( 1 + c ) log ( 1 + c ) − α c − 1 2 log ( 1 + c ) + O ( 1 α ) . \log(\Gamma(\alpha(1+c)))-\log(\Gamma(\alpha))
=\alpha c\log(\alpha)+\alpha(1+c)\log(1+c)-\alpha c-\frac{1}{2}\log(1+c)+O\left(\frac{1}{\alpha}\right). log ( Γ ( α ( 1 + c ))) − log ( Γ ( α )) = α c log ( α ) + α ( 1 + c ) log ( 1 + c ) − α c − 2 1 log ( 1 + c ) + O ( α 1 ) . Substituting c = c α ( s ) c=c_{\alpha}(s) c = c α ( s ) ∣ s ∣ ≤ η |s|\leq\eta ∣ s ∣ ≤ η
log ( E [ e s T α ] ) = α c α ( s ) ( log ( α ) − ψ ( α ) ) − 1 2 log ( 1 + c α ( s ) ) + O ( 1 α ) . \log\left(\mathbb{E}\left[e^{sT_{\alpha}}\right]\right)
=\alpha c_{\alpha}(s)(\log(\alpha)-\psi(\alpha))
-\frac{1}{2}\log(1+c_{\alpha}(s))
+O\left(\frac{1}{\alpha}\right). log ( E [ e s T α ] ) = α c α ( s ) ( log ( α ) − ψ ( α )) − 2 1 log ( 1 + c α ( s )) + O ( α 1 ) . Since
α ( log ( α ) − ψ ( α ) ) = 1 2 + O ( 1 α ) , \alpha(\log(\alpha)-\psi(\alpha))
=\frac{1}{2}+O\left(\frac{1}{\alpha}\right), α ( log ( α ) − ψ ( α )) = 2 1 + O ( α 1 ) , it follows that
sup α ≥ α 0 ( E [ e η T α ] + E [ e − η T α ] ) < ∞ . \sup_{\alpha\geq\alpha_0}
\left(\mathbb{E}\left[e^{\eta T_{\alpha}}\right]
+\mathbb{E}\left[e^{-\eta T_{\alpha}}\right]\right)<\infty. α ≥ α 0 sup ( E [ e η T α ] + E [ e − η T α ] ) < ∞. Since
e η ∣ x ∣ ≤ e η x + e − η x , e^{\eta|x|}\leq e^{\eta x}+e^{-\eta x}, e η ∣ x ∣ ≤ e η x + e − η x , the preceding bound implies
sup α ≥ α 0 E [ e η ∣ T α ∣ ] < ∞ . \sup_{\alpha\ge\alpha_0}\mathbb{E}\left[e^{\eta|T_\alpha|}\right]<\infty. α ≥ α 0 sup E [ e η ∣ T α ∣ ] < ∞. The inequality (A.9) then gives
sup α ≥ α 0 E [ ∣ T α ∣ 3 ] ≤ 6 η 3 sup α ≥ α 0 E [ e η ∣ T α ∣ ] < ∞ , \sup_{\alpha\ge\alpha_0}\mathbb{E}\left[|T_\alpha|^3\right]\leq\frac{6}{\eta^3}\sup_{\alpha\ge\alpha_0}\mathbb{E}\left[e^{\eta|T_\alpha|}\right]<\infty, α ≥ α 0 sup E [ ∣ T α ∣ 3 ] ≤ η 3 6 α ≥ α 0 sup E [ e η ∣ T α ∣ ] < ∞ , proving the uniform third moment bound for T α T_\alpha T α
Let ∥ t ∥ = 1 \lVert t\rVert=1 ∥ t ∥ = 1 O α \mathbf{O}_\alpha O α q α : = O α t \mathbf{q}_\alpha:=\mathbf{O}_\alpha t q α := O α t
t T I α − 1 / 2 S α = q α , 1 T α + q α , 2 Z α . t^{\mathsf T}\mathcal{I}_\alpha^{-1/2}S_\alpha=q_{\alpha,1}T_\alpha+q_{\alpha,2}Z_\alpha. t T I α − 1/2 S α = q α , 1 T α + q α , 2 Z α . Therefore
∣ t T I α − 1 / 2 S α ∣ 3 ≤ 4 ( ∣ T α ∣ 3 + ∣ Z α ∣ 3 ) , \left|t^{\mathsf T}\mathcal{I}_\alpha^{-1/2}S_\alpha\right|^3\leq4\left(|T_\alpha|^3+|Z_\alpha|^3\right), t T I α − 1/2 S α 3 ≤ 4 ( ∣ T α ∣ 3 + ∣ Z α ∣ 3 ) , which proves (A.11). Moreover, for each fixed nonzero r ∈ R 2 r\in\mathbb{R}^2 r ∈ R 2
n E [ ∣ r T I n − 1 / 2 S n , 1 ∣ 3 ] ( n ∥ r ∥ 2 ) 3 / 2 ≤ C n ⟶ 0. \frac{n\mathbb{E}\left[\left|r^{\mathsf T}\mathcal{I}_n^{-1/2}S_{n,1}\right|^3\right]}{\left(n\lVert r\rVert^2\right)^{3/2}}\leq\frac{C}{\sqrt{n}}\longrightarrow0. ( n ∥ r ∥ 2 ) 3/2 n E [ r T I n − 1/2 S n , 1 3 ] ≤ n C ⟶ 0. Therefore,
1 n r T I n − 1 / 2 ∑ i = 1 n S n , i → d N ( 0 , ∥ r ∥ 2 ) . \frac{1}{\sqrt{n}}r^{\mathsf T}\mathcal{I}_n^{-1/2}\sum_{i=1}^{n}S_{n,i}\xrightarrow{d}\mathcal{N}\left(0,\lVert r\rVert^2\right). n 1 r T I n − 1/2 i = 1 ∑ n S n , i d N ( 0 , ∥ r ∥ 2 ) . The Cramér–Wold device completes the proof.
Lemma A.8. Let X n , 1 , … , X n , n ∼ i i d Gamma ( α n , β ) X_{n,1},\ldots,X_{n,n}\mathrel{\overset{\mathrm{iid}}{\sim}}\operatorname{Gamma}(\alpha_n,\beta) X n , 1 , … , X n , n ∼ iid Gamma ( α n , β ) β > 0 \beta>0 β > 0 α n → ∞ \alpha_n\to\infty α n → ∞
X ‾ n : = 1 n ∑ i = 1 n X n , i and G n : = ( ∏ i = 1 n X n , i ) 1 / n . \overline{X}_n:=\frac{1}{n}\sum_{i=1}^{n}X_{n,i}\qquad\text{and}\qquad G_n:=\left(\prod_{i=1}^{n}X_{n,i}\right)^{1/n}. X n := n 1 i = 1 ∑ n X n , i and G n := ( i = 1 ∏ n X n , i ) 1/ n . Define
u 1 , n : = 1 2 α n + 1 12 α n 2 , u 2 , n : = β α n , u ~ 1 , n : = log ( X ‾ n G n ) and u ~ 2 , n : = 1 X ‾ n . u_{1,n}:=\frac{1}{2\alpha_n}+\frac{1}{12\alpha_n^2},\qquad
u_{2,n}:=\frac{\beta}{\alpha_n},\qquad
\widetilde{u}_{1,n}:=\log\left(\frac{\overline{X}_n}{G_n}\right)
\qquad\text{and}\qquad
\widetilde{u}_{2,n}:=\frac{1}{\overline{X}_n}. u 1 , n := 2 α n 1 + 12 α n 2 1 , u 2 , n := α n β , u 1 , n := log ( G n X n ) and u 2 , n := X n 1 . Then the following hold:
The first coordinate satisfies
u ~ 1 , n − u 1 , n = O P ( u 1 , n n ) + O ( u 1 , n 4 n ) . \widetilde{u}_{1,n}-u_{1,n}
=O_{\mathbb{P}}\left(\frac{u_{1,n}}{\sqrt{n}}\right)
+O\left(\frac{u_{1,n}^4}{n}\right). u 1 , n − u 1 , n = O P ( n u 1 , n ) + O ( n u 1 , n 4 ) . The second coordinate satisfies
u ~ 2 , n − u 2 , n = O P ( u 1 , n 3 / 2 n ) + O ( u 1 , n 2 n ) . \widetilde{u}_{2,n}-u_{2,n}
=O_{\mathbb{P}}\left(\frac{u_{1,n}^{3/2}}{\sqrt{n}}\right)
+O\left(\frac{u_{1,n}^2}{n}\right). u 2 , n − u 2 , n = O P ( n u 1 , n 3/2 ) + O ( n u 1 , n 2 ) . If n u 1 , n 3 → 0 \sqrt{n}u_{1,n}^3\to0 n u 1 , n 3 → 0
n u 1 , n ( u ~ 1 , n − u 1 , n ) → d N ( 0 , 2 ) . \frac{\sqrt{n}}{u_{1,n}}\left(\widetilde{u}_{1,n}-u_{1,n}\right)\xrightarrow{d}\mathcal{N}(0,2). u 1 , n n ( u 1 , n − u 1 , n ) d N ( 0 , 2 ) . (iv) The second coordinate also satisfies
n u 1 , n 3 / 2 ( u ~ 2 , n − u 2 , n ) → d N ( 0 , 8 β 2 ) . \frac{\sqrt{n}}{u_{1,n}^{3/2}}(\widetilde{u}_{2,n}-u_{2,n}) \xrightarrow{d} \mathcal{N}(0,8\beta^2). u 1 , n 3/2 n ( u 2 , n − u 2 , n ) d N ( 0 , 8 β 2 ) . Proof. First note that u 1 , n ∼ ( 2 α n ) − 1 u_{1,n}\sim(2\alpha_n)^{-1} u 1 , n ∼ ( 2 α n ) − 1 u 1 , n ≍ α n − 1 u_{1,n}\asymp\alpha_n^{-1} u 1 , n ≍ α n − 1 n α n > 2 n\alpha_n>2 n α n > 2 n n n Lemma A.6 applies.
For (i), parts (iii) and (iv) of Lemma A.6 give
E [ u ~ 1 , n ] = ψ ( n α n ) − ψ ( α n ) − log ( n ) and Var ( u ~ 1 , n ) = ψ 1 ( α n ) n − ψ 1 ( n α n ) . \mathbb{E}[\widetilde{u}_{1,n}]=\psi(n\alpha_n)-\psi(\alpha_n)-\log(n)
\quad\text{and}\quad
\operatorname{Var}(\widetilde{u}_{1,n})=\frac{\psi_1(\alpha_n)}{n}-\psi_1(n\alpha_n). E [ u 1 , n ] = ψ ( n α n ) − ψ ( α n ) − log ( n ) and Var ( u 1 , n ) = n ψ 1 ( α n ) − ψ 1 ( n α n ) . Substituting in the standard digamma expansion in (19) yields
ψ ( n α n ) − ψ ( α n ) − log ( n ) = u 1 , n − 1 2 n α n − 1 12 n 2 α n 2 + O ( 1 α n 4 ) , \psi(n\alpha_n)-\psi(\alpha_n)-\log(n)
=u_{1,n}-\frac{1}{2n\alpha_n}-\frac{1}{12n^2\alpha_n^2}+O\left(\frac{1}{\alpha_n^4}\right), ψ ( n α n ) − ψ ( α n ) − log ( n ) = u 1 , n − 2 n α n 1 − 12 n 2 α n 2 1 + O ( α n 4 1 ) , where the remainder is uniform because ( n α n ) − 4 ≤ α n − 4 (n\alpha_n)^{-4}\leq\alpha_n^{-4} ( n α n ) − 4 ≤ α n − 4
E [ u ~ 1 , n ] − u 1 , n = O ( u 1 , n n ) + O ( u 1 , n 4 ) . (59) \mathbb{E}[\widetilde{u}_{1,n}]-u_{1,n}
=O\left(\frac{u_{1,n}}{n}\right)+O(u_{1,n}^4). \tag*{(59)} E [ u 1 , n ] − u 1 , n = O ( n u 1 , n ) + O ( u 1 , n 4 ) . (59) The trigamma expansion [2 Equation 6.4.12]
ψ 1 ( x ) = 1 x + 1 2 x 2 + 1 6 x 3 + O ( 1 x 5 ) \psi_1(x)=\frac{1}{x}+\frac{1}{2x^2}+\frac{1}{6x^3}+O\left(\frac{1}{x^5}\right) ψ 1 ( x ) = x 1 + 2 x 2 1 + 6 x 3 1 + O ( x 5 1 ) gives
Var ( u ~ 1 , n ) = 1 − 1 / n 2 n α n 2 + 1 − 1 / n 2 6 n α n 3 + O ( 1 n α n 5 ) = O ( u 1 , n 2 n ) . \operatorname{Var}(\widetilde{u}_{1,n})
=\frac{1-1/n}{2n\alpha_n^2}+\frac{1-1/n^2}{6n\alpha_n^3}
+O\left(\frac{1}{n\alpha_n^5}\right)
=O\left(\frac{u_{1,n}^2}{n}\right). Var ( u 1 , n ) = 2 n α n 2 1 − 1/ n + 6 n α n 3 1 − 1/ n 2 + O ( n α n 5 1 ) = O ( n u 1 , n 2 ) . Chebyshev’s inequality therefore implies that
u ~ 1 , n − E [ u ~ 1 , n ] = O P ( u 1 , n n ) . \widetilde{u}_{1,n}-\mathbb{E}[\widetilde{u}_{1,n}]
=O_{\mathbb{P}}\left(\frac{u_{1,n}}{\sqrt{n}}\right). u 1 , n − E [ u 1 , n ] = O P ( n u 1 , n ) . Combining this with (A.12) proves (i).
For (ii), parts (i) and (ii) of Lemma A.6 give
E [ u ~ 2 , n ] − u 2 , n = n β n α n − 1 − β α n = β α n ( n α n − 1 ) = O ( u 1 , n 2 n ) . (60) \mathbb{E}[\widetilde{u}_{2,n}]-u_{2,n}
=\frac{n\beta}{n\alpha_n-1}-\frac{\beta}{\alpha_n}
=\frac{\beta}{\alpha_n(n\alpha_n-1)}
=O\left(\frac{u_{1,n}^2}{n}\right). \tag*{(60)} E [ u 2 , n ] − u 2 , n = n α n − 1 n β − α n β = α n ( n α n − 1 ) β = O ( n u 1 , n 2 ) . (60) and
Var ( u ~ 2 , n ) = n 2 β 2 ( n α n − 1 ) 2 ( n α n − 2 ) = O ( 1 n α n 3 ) = O ( u 1 , n 3 n ) . \operatorname{Var}(\widetilde{u}_{2,n})
=\frac{n^2\beta^2}{(n\alpha_n-1)^2(n\alpha_n-2)}
=O\left(\frac{1}{n\alpha_n^3}\right)
=O\left(\frac{u_{1,n}^3}{n}\right). Var ( u 2 , n ) = ( n α n − 1 ) 2 ( n α n − 2 ) n 2 β 2 = O ( n α n 3 1 ) = O ( n u 1 , n 3 ) . Another application of Chebyshev’s inequality yields
u ~ 2 , n − E [ u ~ 2 , n ] = O P ( u 1 , n 3 / 2 n ) , \widetilde{u}_{2,n}-\mathbb{E}[\widetilde{u}_{2,n}]
=O_{\mathbb{P}}\left(\frac{u_{1,n}^{3/2}}{\sqrt{n}}\right), u 2 , n − E [ u 2 , n ] = O P ( n u 1 , n 3/2 ) , and combining this with (A.13) proves (ii).
For (iii), let
ξ n , i : = log ( β X n , i ) − ψ ( α n ) + 1 − β X n , i α n and v n : = Var ( ξ n , i ) = ψ 1 ( α n ) − 1 α n . \xi_{n,i}:=\log(\beta X_{n,i})-\psi(\alpha_n)+1-\frac{\beta X_{n,i}}{\alpha_n}
\quad\text{and}\quad
v_n:=\operatorname{Var}(\xi_{n,i})=\psi_1(\alpha_n)-\frac{1}{\alpha_n}. ξ n , i := log ( β X n , i ) − ψ ( α n ) + 1 − α n β X n , i and v n := Var ( ξ n , i ) = ψ 1 ( α n ) − α n 1 . where the variance identity follows from
Var ( log ( β X n , i ) ) = ψ 1 ( α n ) and Var ( β X n , i / α n ) = Cov ( log ( β X n , i ) , β X n , i / α n ) = 1 α n . \operatorname{Var}(\log(\beta X_{n,i}))=\psi_1(\alpha_n)
\quad\text{and}\quad
\operatorname{Var}(\beta X_{n,i}/\alpha_n)=\operatorname{Cov}(\log(\beta X_{n,i}),\beta X_{n,i}/\alpha_n)=\frac{1}{\alpha_n}. Var ( log ( β X n , i )) = ψ 1 ( α n ) and Var ( β X n , i / α n ) = Cov ( log ( β X n , i ) , β X n , i / α n ) = α n 1 . Note that E [ ξ n , i ] = 0 \mathbb{E}[\xi_{n,i}]=0 E [ ξ n , i ] = 0 ξ n , i / v n \xi_{n,i}/\sqrt{v_n} ξ n , i / v n
1 n v n ∑ i = 1 n ξ n , i → d N ( 0 , 1 ) . \frac{1}{\sqrt{n v_n}}\sum_{i=1}^{n}\xi_{n,i}\xrightarrow{d}\mathcal{N}(0,1). n v n 1 i = 1 ∑ n ξ n , i d N ( 0 , 1 ) . Now, set
Z n = β X ‾ n α n − 1. Z_n=\frac{\beta\overline{X}_n}{\alpha_n}-1. Z n = α n β X n − 1. Since E [ Z n ] = 0 \mathbb{E}[Z_n]=0 E [ Z n ] = 0 Var ( Z n ) = 1 / ( n α n ) \operatorname{Var}(Z_n)=1/(n\alpha_n) Var ( Z n ) = 1/ ( n α n ) Z n = O P ( ( n α n ) − 1 / 2 ) = o P ( 1 ) Z_n=O_{\mathrm{P}}((n\alpha_n)^{-1/2})=o_{\mathrm{P}}(1) Z n = O P (( n α n ) − 1/2 ) = o P ( 1 )
log ( 1 + Z n ) − 1 n ∑ i = 1 n ( log ( β X n , i ) − ψ ( α n ) ) = u ~ 1 , n − ( log ( α n ) − ψ ( α n ) ) \log(1+Z_n)-\frac{1}{n}\sum_{i=1}^{n}\left(\log(\beta X_{n,i})-\psi(\alpha_n)\right)=\widetilde{u}_{1,n}-(\log(\alpha_n)-\psi(\alpha_n)) log ( 1 + Z n ) − n 1 i = 1 ∑ n ( log ( β X n , i ) − ψ ( α n ) ) = u 1 , n − ( log ( α n ) − ψ ( α n )) and
Z n − 1 n ∑ i = 1 n ( log ( β X n , i ) − ψ ( α n ) ) = − 1 n ∑ i = 1 n ξ n , i Z_n-\frac{1}{n}\sum_{i=1}^{n}\left(\log(\beta X_{n,i})-\psi(\alpha_n)\right)=-\frac{1}{n}\sum_{i=1}^{n}\xi_{n,i} Z n − n 1 i = 1 ∑ n ( log ( β X n , i ) − ψ ( α n ) ) = − n 1 i = 1 ∑ n ξ n , i yield that
u ~ 1 , n − ( log ( α n ) − ψ ( α n ) ) = − 1 n ∑ i = 1 n ξ n , i + log ( 1 + Z n ) − Z n . \widetilde{u}_{1,n}-(\log(\alpha_n)-\psi(\alpha_n))=-\frac{1}{n}\sum_{i=1}^{n}\xi_{n,i}+\log(1+Z_n)-Z_n. u 1 , n − ( log ( α n ) − ψ ( α n )) = − n 1 i = 1 ∑ n ξ n , i + log ( 1 + Z n ) − Z n . On the event ∣ Z n ∣ ≤ 1 / 2 |Z_n|\leq1/2 ∣ Z n ∣ ≤ 1/2 ∣ log ( 1 + Z n ) − Z n ∣ ≤ C Z n 2 |\log(1+Z_n)-Z_n|\leq CZ_n^2 ∣ log ( 1 + Z n ) − Z n ∣ ≤ C Z n 2
log ( 1 + Z n ) − Z n = O P ( 1 n α n ) . \log(1+Z_n)-Z_n=O_{\mathrm{P}}\left(\frac{1}{n\alpha_n}\right). log ( 1 + Z n ) − Z n = O P ( n α n 1 ) . The expansions of ψ \psi ψ ψ 1 \psi_1 ψ 1
v n u 1 , n 2 ⟶ 2 and log ( α n ) − ψ ( α n ) − u 1 , n = O ( u 1 , n 4 ) . \frac{v_n}{u_{1,n}^2}\longrightarrow2
\quad\text{and}\quad
\log(\alpha_n)-\psi(\alpha_n)-u_{1,n}=O(u_{1,n}^4). u 1 , n 2 v n ⟶ 2 and log ( α n ) − ψ ( α n ) − u 1 , n = O ( u 1 , n 4 ) . Therefore,
n u 1 , n ( u ~ 1 , n − u 1 , n ) = − v n u 1 , n 1 n v n ∑ i = 1 n ξ n , i + n u 1 , n ( log ( α n ) − ψ ( α n ) − u 1 , n ) + n u 1 , n ( log ( 1 + Z n ) − Z n ) . \frac{\sqrt{n}}{u_{1,n}}(\widetilde{u}_{1,n}-u_{1,n})=
-\frac{\sqrt{v_n}}{u_{1,n}}\frac{1}{\sqrt{n v_n}}\sum_{i=1}^{n}\xi_{n,i}
+\frac{\sqrt{n}}{u_{1,n}}(\log(\alpha_n)-\psi(\alpha_n)-u_{1,n})
+\frac{\sqrt{n}}{u_{1,n}}(\log(1+Z_n)-Z_n). u 1 , n n ( u 1 , n − u 1 , n ) = − u 1 , n v n n v n 1 i = 1 ∑ n ξ n , i + u 1 , n n ( log ( α n ) − ψ ( α n ) − u 1 , n ) + u 1 , n n ( log ( 1 + Z n ) − Z n ) . Since n u 1 , n 3 → 0 \sqrt{n}u_{1,n}^3\to0 n u 1 , n 3 → 0 o ( 1 ) o(1) o ( 1 ) α n u 1 , n → 1 / 2 \alpha_nu_{1,n}\to1/2 α n u 1 , n → 1/2
O P ( 1 n α n u 1 , n ) = o P ( 1 ) . O_{\mathrm{P}}\left(\frac{1}{\sqrt{n}\alpha_nu_{1,n}}\right)=o_{\mathrm{P}}(1). O P ( n α n u 1 , n 1 ) = o P ( 1 ) . Slutsky’s theorem then proves (iii).
For (iv), let μ n : = α n / β \mu_n:=\alpha_n/\beta μ n := α n / β Δ n : = X ‾ n − μ n \Delta_n:=\overline{X}_n-\mu_n Δ n := X n − μ n ( β X n , i − α n ) / α n (\beta X_{n,i}-\alpha_n)/\sqrt{\alpha_n} ( β X n , i − α n ) / α n
n β α n Δ n → d N ( 0 , 1 ) . \frac{\sqrt{n}\beta}{\sqrt{\alpha_n}}\Delta_n\xrightarrow{d}\mathcal{N}(0,1). α n n β Δ n d N ( 0 , 1 ) . The central limit theorem implies
Δ n = O P ( α n β n ) . \Delta_n=O_{\mathrm{P}}\left(\frac{\sqrt{\alpha_n}}{\beta\sqrt{n}}\right). Δ n = O P ( β n α n ) . Since μ n = α n / β \mu_n = \alpha_n/\beta μ n = α n / β
Δ n μ n = O P ( 1 n α n ) = o P ( 1 ) , \frac{\Delta_n}{\mu_n} = O_{\mathbb{P}}\left(\frac{1}{\sqrt{n\alpha_n}}\right) = o_{\mathbb{P}}(1), μ n Δ n = O P ( n α n 1 ) = o P ( 1 ) , and hence P ( ∣ Δ n ∣ ≤ μ n / 2 ) → 1 \mathbb{P}(|\Delta_n| \le\mu_n/2) \to1 P ( ∣ Δ n ∣ ≤ μ n /2 ) → 1
1 X ˉ n − 1 μ n = − Δ n μ n 2 + Δ n 2 μ n 2 X ˉ n \frac{1}{\bar{X}_n} - \frac{1}{\mu_n} = -\frac{\Delta_n}{\mu_n^2} + \frac{\Delta_n^2}{\mu_n^2\bar{X}_n} X ˉ n 1 − μ n 1 = − μ n 2 Δ n + μ n 2 X ˉ n Δ n 2 shows that, on { ∣ Δ n ∣ ≤ μ n / 2 } \{|\Delta_n| \le\mu_n/2\} { ∣ Δ n ∣ ≤ μ n /2 } X ˉ n = μ n + Δ n ≥ μ n / 2 \bar{X}_n = \mu_n + \Delta_n \ge\mu_n/2 X ˉ n = μ n + Δ n ≥ μ n /2
∣ Δ n 2 μ n 2 X ˉ n ∣ ≤ 2 Δ n 2 μ n 3 . \left|\frac{\Delta_n^2}{\mu_n^2\bar{X}_n}\right| \le\frac{2\Delta_n^2}{\mu_n^3}. μ n 2 X ˉ n Δ n 2 ≤ μ n 3 2 Δ n 2 . Therefore
Δ n 2 μ n 2 X ˉ n = O P ( 1 n α n 2 ) . \frac{\Delta_n^2}{\mu_n^2\bar{X}_n} = O_{\mathbb{P}}\left(\frac{1}{n\alpha_n^2}\right). μ n 2 X ˉ n Δ n 2 = O P ( n α n 2 1 ) . Since α n 3 / 2 u 1 , n 3 / 2 → 2 − 3 / 2 \alpha_n^{3/2}u_{1,n}^{3/2} \to2^{-3/2} α n 3/2 u 1 , n 3/2 → 2 − 3/2
n u 1 , n 3 / 2 O P ( 1 n α n 2 ) = O P ( 1 n α n ) = o P ( 1 ) . \frac{\sqrt{n}}{u_{1,n}^{3/2}}O_{\mathbb{P}}\left(\frac{1}{n\alpha_n^2}\right) = O_{\mathbb{P}}\left(\frac{1}{\sqrt{n\alpha_n}}\right) = o_{\mathbb{P}}(1). u 1 , n 3/2 n O P ( n α n 2 1 ) = O P ( n α n 1 ) = o P ( 1 ) . It follows that
n u 1 , n 3 / 2 ( u ~ 2 , n − u 2 , n ) = − β α n 3 / 2 u 1 , n 3 / 2 n β α n Δ n + o P ( 1 ) . \frac{\sqrt{n}}{u_{1,n}^{3/2}}(\tilde{u}_{2,n}-u_{2,n}) = -\frac{\beta}{\alpha_n^{3/2}u_{1,n}^{3/2}}\frac{\sqrt{n}\beta}{\sqrt{\alpha_n}}\Delta_n + o_{\mathbb{P}}(1). u 1 , n 3/2 n ( u ~ 2 , n − u 2 , n ) = − α n 3/2 u 1 , n 3/2 β α n n β Δ n + o P ( 1 ) . The deterministic term converges to − 2 3 / 2 β -2^{3/2}\beta − 2 3/2 β 8 β 2 8\beta^2 8 β 2
Lemma A.9. Let β > 0 \beta> 0 β > 0 α n → ∞ \alpha_n \to\infty α n → ∞
u n = ( 1 2 α n + 1 12 α n 2 β α n ) = ( u 1 , n u 2 , n ) , u_n = \begin{pmatrix} \dfrac{1}{2\alpha_n}+\dfrac{1}{12\alpha_n^2} \\[4pt] \dfrac{\beta}{\alpha_n} \end{pmatrix} = \begin{pmatrix} u_{1,n} \\ u_{2,n} \end{pmatrix}, u n = 2 α n 1 + 12 α n 2 1 α n β = ( u 1 , n u 2 , n ) , and let
K n : = [ u 1 , n 2 , 2 u 1 , n ] × [ u 2 , n 2 , 2 u 2 , n ] . K_n := \left[\frac{u_{1,n}}{2},2u_{1,n}\right]\times\left[\frac{u_{2,n}}{2},2u_{2,n}\right]. K n := [ 2 u 1 , n , 2 u 1 , n ] × [ 2 u 2 , n , 2 u 2 , n ] . For F : = ϕ − 1 F := \phi^{-1} F := ϕ − 1 (24) , the following hold:
The Jacobian satisfies
sup u ∈ K n ∥ J F ( u ) ∥ = O ( 1 u 1 , n 2 ) , (61) \sup_{u\in K_n}\left\lVert\mathbf{J}_F(u)\right\rVert= O\left(\frac{1}{u_{1,n}^2}\right), \tag*{(61)} u ∈ K n sup ∥ J F ( u ) ∥ = O ( u 1 , n 2 1 ) , (61) and therefore F F F b n b_n b n K n K_n K n b n = O ( u 1 , n − 2 ) b_n = O(u_{1,n}^{-2}) b n = O ( u 1 , n − 2 )
Define
R n : = diag ( n u 1 , n , n u 1 , n 3 / 2 ) and T n : = diag ( n u 1 , n , n ) . R_n := \operatorname{diag}\left(\frac{\sqrt{n}}{u_{1,n}},\frac{\sqrt{n}}{u_{1,n}^{3/2}}\right) \quad\text{and} \quad T_n := \operatorname{diag}(\sqrt{n}u_{1,n},\sqrt{n}). R n := diag ( u 1 , n n , u 1 , n 3/2 n ) and T n := diag ( n u 1 , n , n ) . Then
T n J F ( u n ) R n − 1 ⟶ ( − 1 2 0 − β 0 ) . T_n\mathbf{J}_F(u_n)R_n^{-1} \longrightarrow\begin{pmatrix} -\dfrac{1}{2} & 0 \\ -\beta& 0 \end{pmatrix}. T n J F ( u n ) R n − 1 ⟶ ( − 2 1 − β 0 0 ) . (iii) If a random sequence V n V_n V n ( 0 , ∞ ) 2 (0,\infty)^2 ( 0 , ∞ ) 2 P ( V n ∈ K n ) → 1 \mathbb{P}(V_n \in K_n) \to1 P ( V n ∈ K n ) → 1 R n ( V n − u n ) = O P ( 1 ) R_n(V_n-u_n)=O_{\mathbb{P}}(1) R n ( V n − u n ) = O P ( 1 )
T n [ F ( V n ) − F ( u n ) − J F ( u n ) ( V n − u n ) ] = o P ( 1 ) . T_n[F(V_n)-F(u_n)-J_F(u_n)(V_n-u_n)]=o_{\mathbb{P}}(1). T n [ F ( V n ) − F ( u n ) − J F ( u n ) ( V n − u n )] = o P ( 1 ) . Proof. Write
d ( u 1 ) : = − 3 + 9 + 12 u 1 d(u_1):=-3+\sqrt{9+12u_1} d ( u 1 ) := − 3 + 9 + 12 u 1 and
f ( u 1 ) : = 1 d ( u 1 ) , f(u_1):=\frac{1}{d(u_1)}, f ( u 1 ) := d ( u 1 ) 1 , so F ( u ) = ( f ( u 1 ) , u 2 f ( u 1 ) ) T F(u)=(f(u_1),u_2f(u_1))^{\mathsf{T}} F ( u ) = ( f ( u 1 ) , u 2 f ( u 1 ) ) T d d d
d ( u 1 ) = 12 u 1 3 + 9 + 12 u 1 . d(u_1)=\frac{12u_1}{3+\sqrt{9+12u_1}}. d ( u 1 ) = 3 + 9 + 12 u 1 12 u 1 . Thus we have d ( u 1 ) ≍ u 1 , n d(u_1)\asymp u_{1,n} d ( u 1 ) ≍ u 1 , n 9 + 12 u 1 ≍ 1 \sqrt{9+12u_1}\asymp1 9 + 12 u 1 ≍ 1 K n K_n K n
f ′ ( u 1 ) = − 6 9 + 12 u 1 d ( u 1 ) 2 f'(u_1)=-\frac{6}{\sqrt{9+12u_1}d(u_1)^2} f ′ ( u 1 ) = − 9 + 12 u 1 d ( u 1 ) 2 6 and
f ′ ′ ( u 1 ) = 36 ( 9 + 12 u 1 ) 3 / 2 d ( u 1 ) 2 + 72 ( 9 + 12 u 1 ) d ( u 1 ) 3 . f''(u_1)=\frac{36}{(9+12u_1)^{3/2}d(u_1)^2}+\frac{72}{(9+12u_1)d(u_1)^3}. f ′′ ( u 1 ) = ( 9 + 12 u 1 ) 3/2 d ( u 1 ) 2 36 + ( 9 + 12 u 1 ) d ( u 1 ) 3 72 . It follows that, uniformly on K n K_n K n
f ( u 1 ) = O ( 1 u 1 , n ) f(u_1)=O\left(\frac{1}{u_{1,n}}\right) f ( u 1 ) = O ( u 1 , n 1 ) and
f ′ ( u 1 ) = O ( 1 u 1 , n 2 ) f'(u_1)=O\left(\frac{1}{u_{1,n}^2}\right) f ′ ( u 1 ) = O ( u 1 , n 2 1 ) and
f ′ ′ ( u 1 ) = O ( 1 u 1 , n 3 ) . f''(u_1)=O\left(\frac{1}{u_{1,n}^3}\right). f ′′ ( u 1 ) = O ( u 1 , n 3 1 ) . Also, u 2 = O ( u 1 , n ) u_2=O(u_{1,n}) u 2 = O ( u 1 , n ) u 2 , n ≍ u 1 , n u_{2,n}\asymp u_{1,n} u 2 , n ≍ u 1 , n
J F ( u ) = ( f ′ ( u 1 ) 0 u 2 f ′ ( u 1 ) f ( u 1 ) ) = ( O ( 1 u 1 , n 2 ) 0 O ( 1 u 1 , n ) O ( 1 u 1 , n ) ) (62) J_F(u)=
\begin{pmatrix}
f'(u_1) & 0\\
u_2f'(u_1) & f(u_1)
\end{pmatrix}
=
\begin{pmatrix}
O\left(\frac{1}{u_{1,n}^2}\right) & 0\\
O\left(\frac{1}{u_{1,n}}\right) & O\left(\frac{1}{u_{1,n}}\right)
\end{pmatrix} \tag*{(62)} J F ( u ) = ( f ′ ( u 1 ) u 2 f ′ ( u 1 ) 0 f ( u 1 ) ) = O ( u 1 , n 2 1 ) O ( u 1 , n 1 ) 0 O ( u 1 , n 1 ) (62) uniformly on K n K_n K n K n K_n K n v , w ∈ K n v,w\in K_n v , w ∈ K n
F ( v ) − F ( w ) = ∫ 0 1 J F [ w + t ( v − w ) ] ( v − w ) d t , F(v)-F(w)=\int_0^1 J_F[w+t(v-w)](v-w)\,\mathrm{d}t, F ( v ) − F ( w ) = ∫ 0 1 J F [ w + t ( v − w )] ( v − w ) d t , and therefore the Lipschitz constant for F F F K n K_n K n u 1 , n − 2 u_{1,n}^{-2} u 1 , n − 2
At u n u_n u n d ( u 1 , n ) = 1 / α n d(u_{1,n})=1/\alpha_n d ( u 1 , n ) = 1/ α n u 2 , n = β d ( u 1 , n ) u_{2,n}=\beta d(u_{1,n}) u 2 , n = β d ( u 1 , n ) 9 + 12 u 1 , n = 3 + 1 / α n \sqrt{9+12u_{1,n}}=3+1/\alpha_n 9 + 12 u 1 , n = 3 + 1/ α n
T n J F ( u n ) R n − 1 = ( − 6 α n 2 u 1 , n 2 3 + 1 / α n 0 − 6 β α n u 1 , n 3 + 1 / α n α n u 1 , n 3 / 2 ) T_nJ_F(u_n)R_n^{-1}
=
\begin{pmatrix}
-\dfrac{6\alpha_n^2u_{1,n}^2}{3+1/\alpha_n} & 0\\
-\dfrac{6\beta\alpha_nu_{1,n}}{3+1/\alpha_n} & \alpha_nu_{1,n}^{3/2}
\end{pmatrix} T n J F ( u n ) R n − 1 = − 3 + 1/ α n 6 α n 2 u 1 , n 2 − 3 + 1/ α n 6 β α n u 1 , n 0 α n u 1 , n 3/2 and
α n u 1 , n = 1 2 + 1 12 α n \alpha_nu_{1,n}=\frac{1}{2}+\frac{1}{12\alpha_n} α n u 1 , n = 2 1 + 12 α n 1 and
α n u 1 , n 3 / 2 = ( α n u 1 , n ) u 1 , n 1 / 2 ⟶ 0 , \alpha_nu_{1,n}^{3/2}=(\alpha_nu_{1,n})u_{1,n}^{1/2}\longrightarrow0, α n u 1 , n 3/2 = ( α n u 1 , n ) u 1 , n 1/2 ⟶ 0 , which together prove (ii).
For (iii), write δ j , n : = V j , n − u j , n \delta_{j,n}:=V_{j,n}-u_{j,n} δ j , n := V j , n − u j , n A n : = { V n ∈ K n } A_n:=\{V_n\in K_n\} A n := { V n ∈ K n } R n ( V n − u n ) = O P ( 1 ) R_n(V_n-u_n)=O_{\mathbb{P}}(1) R n ( V n − u n ) = O P ( 1 )
δ 1 , n = O P ( u 1 , n n ) \delta_{1,n}=O_{\mathbb{P}}\left(\frac{u_{1,n}}{\sqrt{n}}\right) δ 1 , n = O P ( n u 1 , n ) and
δ 2 , n = O P ( u 1 , n 3 / 2 n ) . \delta_{2,n}=O_{\mathbb{P}}\left(\frac{u_{1,n}^{3/2}}{\sqrt{n}}\right). δ 2 , n = O P ( n u 1 , n 3/2 ) . On A n A_n A n u n u_n u n V n V_n V n K n K_n K n
δ 1 , n 2 ∫ 0 1 ( 1 − t ) f ′ ′ ( u 1 , n + t δ 1 , n ) d t = O P ( δ 1 , n 2 u 1 , n 3 ) = O P ( 1 n u 1 , n ) . \delta_{1,n}^{2}\int_{0}^{1}(1-t)f^{\prime\prime}(u_{1,n}+t\delta_{1,n})\,\mathrm{d}t=O_{\mathbb{P}}\left(\frac{\delta_{1,n}^{2}}{u_{1,n}^{3}}\right)=O_{\mathbb{P}}\left(\frac{1}{nu_{1,n}}\right). δ 1 , n 2 ∫ 0 1 ( 1 − t ) f ′′ ( u 1 , n + t δ 1 , n ) d t = O P ( u 1 , n 3 δ 1 , n 2 ) = O P ( n u 1 , n 1 ) . After multiplying by n u 1 , n \sqrt{n}u_{1,n} n u 1 , n O P ( n − 1 / 2 ) = o P ( 1 ) O_{\mathbb{P}}(n^{-1/2})=o_{\mathbb{P}}(1) O P ( n − 1/2 ) = o P ( 1 ) F 2 ( u ) = u 2 f ( u 1 ) F_2(u)=u_2f(u_1) F 2 ( u ) = u 2 f ( u 1 ) K n K_n K n
∂ 11 F 2 ( u ) = u 2 f ′ ′ ( u 1 ) = O ( 1 u 1 , n 2 ) and ∂ 12 F 2 ( u ) = f ′ ( u 1 ) = O ( 1 u 1 , n 2 ) and ∂ 22 F 2 ( u ) = 0. \partial_{11}F_2(u)=u_2f^{\prime\prime}(u_1)=O\left(\frac{1}{u_{1,n}^{2}}\right)\quad\text{and}\quad\partial_{12}F_2(u)=f^{\prime}(u_1)=O\left(\frac{1}{u_{1,n}^{2}}\right)\quad\text{and}\quad\partial_{22}F_2(u)=0. ∂ 11 F 2 ( u ) = u 2 f ′′ ( u 1 ) = O ( u 1 , n 2 1 ) and ∂ 12 F 2 ( u ) = f ′ ( u 1 ) = O ( u 1 , n 2 1 ) and ∂ 22 F 2 ( u ) = 0. Applying the same integral expansion, its remainder is
O P ( δ 1 , n 2 u 1 , n 2 ) + O P ( ∣ δ 1 , n δ 2 , n ∣ u 1 , n 2 ) = O P ( 1 n ) + O P ( u 1 , n n ) , O_{\mathbb{P}}\left(\frac{\delta_{1,n}^{2}}{u_{1,n}^{2}}\right)+O_{\mathbb{P}}\left(\frac{|\delta_{1,n}\delta_{2,n}|}{u_{1,n}^{2}}\right)=O_{\mathbb{P}}\left(\frac{1}{n}\right)+O_{\mathbb{P}}\left(\frac{\sqrt{u_{1,n}}}{n}\right), O P ( u 1 , n 2 δ 1 , n 2 ) + O P ( u 1 , n 2 ∣ δ 1 , n δ 2 , n ∣ ) = O P ( n 1 ) + O P ( n u 1 , n ) , which, after multiplying by n \sqrt{n} n
O P ( n − 1 2 ) + O P ( u 1 , n n ) = o P ( 1 ) . O_{\mathbb{P}}\left(n^{-\frac{1}{2}}\right)+O_{\mathbb{P}}\left(\sqrt{\frac{u_{1,n}}{n}}\right)=o_{\mathbb{P}}(1). O P ( n − 2 1 ) + O P ( n u 1 , n ) = o P ( 1 ) . Finally, P ( A n c ) → 0 \mathbb{P}(A_n^c)\to0 P ( A n c ) → 0 A n A_n A n
Gamma Regression
Lemma A.10. Assume the gamma regression model and design conditions of § 3.3 , where φ n ↓ 0 \varphi_n\downarrow0 φ n ↓ 0 β 0 ∈ R k \beta_0\in\mathbb{R}^k β 0 ∈ R k
(i) The explicit estimators in (31) satisfy
β ~ n − β 0 = O ( φ n ) + O P ( φ n n ) , (63) \widetilde{\beta}_n-\beta_0=O(\varphi_n)+O_{\mathbb{P}}\left(\sqrt{\frac{\varphi_n}{n}}\right), \tag*{(63)} β n − β 0 = O ( φ n ) + O P ( n φ n ) , (63) and
φ ~ n − φ n = O ( φ n 2 + φ n n ) + O P ( φ n n ) . (64) \widetilde{\varphi}_n-\varphi_n=O\left(\varphi_n^2+\frac{\varphi_n}{n}\right)+O_{\mathbb{P}}\left(\frac{\varphi_n}{\sqrt{n}}\right). \tag*{(64)} φ n − φ n = O ( φ n 2 + n φ n ) + O P ( n φ n ) . (64) (ii) The exact MLE exists uniquely with probability tending to one and satisfies
β ^ n − β 0 = O P ( φ n n ) (65) \widehat{\beta}_n-\beta_0=O_{\mathbb{P}}\left(\sqrt{\frac{\varphi_n}{n}}\right) \tag*{(65)} β n − β 0 = O P ( n φ n ) (65) and
φ ^ n − φ n = O P ( φ n n ) . (66) \widehat{\varphi}_n-\varphi_n=O_{\mathbb{P}}\left(\frac{\varphi_n}{\sqrt{n}}\right). \tag*{(66)} φ n − φ n = O P ( n φ n ) . (66) Proof. Write
W n , i : = log ( Z n , i ) and W n : = ( W n , 1 , … , W n , n ) ⊤ , W_{n,i}:=\log(Z_{n,i})\quad\text{and}\quad W_n:=(W_{n,1},\ldots,W_{n,n})^\top, W n , i := log ( Z n , i ) and W n := ( W n , 1 , … , W n , n ) ⊤ , and let
m n : = E [ W n , i ] = ψ ( 1 φ n ) + log ( φ n ) . m_n:=\mathbb{E}[W_{n,i}]=\psi\left(\frac{1}{\varphi_n}\right)+\log(\varphi_n). m n := E [ W n , i ] = ψ ( φ n 1 ) + log ( φ n ) . The standard digamma and trigamma expansions [2 Formulas 6.3.18 and 6.4.12] give
m n = − φ n 2 + O ( φ n 2 ) and Var ( W n , i ) = ψ 1 ( 1 φ n ) = φ n + O ( φ n 2 ) . (67) m_n=-\frac{\varphi_n}{2}+O(\varphi_n^2)\quad\text{and}\quad\operatorname{Var}(W_{n,i})=\psi_1\left(\frac{1}{\varphi_n}\right)=\varphi_n+O(\varphi_n^2). \tag*{(67)} m n = − 2 φ n + O ( φ n 2 ) and Var ( W n , i ) = ψ 1 ( φ n 1 ) = φ n + O ( φ n 2 ) . (67) By Lemma A.1, the cumulants of W n , i W_{n,i} W n , i r ≥ 2 r\ge2 r ≥ 2 ψ r − 1 ( 1 / φ n ) \psi_{r-1}(1/\varphi_n) ψ r − 1 ( 1/ φ n ) μ 4 = κ 4 + 3 κ 2 2 \mu_4=\kappa_4+3\kappa_2^2 μ 4 = κ 4 + 3 κ 2 2 [40 ] therefore gives
E [ ( W n , i − m n ) 4 ] = ψ 3 ( 1 φ n ) + 3 ψ 1 ( 1 φ n ) 2 = O ( φ n 2 ) . \mathbb{E}\left[(W_{n,i}-m_n)^4\right]=\psi_3\left(\frac{1}{\varphi_n}\right)+3\psi_1\left(\frac{1}{\varphi_n}\right)^2=O(\varphi_n^2). E [ ( W n , i − m n ) 4 ] = ψ 3 ( φ n 1 ) + 3 ψ 1 ( φ n 1 ) 2 = O ( φ n 2 ) . Hence, Hölder’s inequality gives
E [ ∣ W n , i − m n ∣ 3 ] ≤ ( E [ ( W n , i − m n ) 4 ] ) 3 4 = O ( φ n 3 2 ) . \mathbb{E}\left[|W_{n,i}-m_n|^3\right]\leq\left(\mathbb{E}\left[(W_{n,i}-m_n)^4\right]\right)^{\frac{3}{4}}=O(\varphi_n^{\frac{3}{2}}). E [ ∣ W n , i − m n ∣ 3 ] ≤ ( E [ ( W n , i − m n ) 4 ] ) 4 3 = O ( φ n 2 3 ) . Since m n = O ( φ n ) m_n=O(\varphi_n) m n = O ( φ n ) ∣ x + y ∣ 3 ≤ 4 ( ∣ x ∣ 3 + ∣ y ∣ 3 ) |x+y|^3\leq4(|x|^3+|y|^3) ∣ x + y ∣ 3 ≤ 4 ( ∣ x ∣ 3 + ∣ y ∣ 3 )
E [ ∣ W n , i ∣ 3 ] = O ( φ n 3 2 ) . \mathbb{E}\left[|W_{n,i}|^3\right]=O(\varphi_n^{\frac{3}{2}}). E [ ∣ W n , i ∣ 3 ] = O ( φ n 2 3 ) . It remains to control the corresponding moment weighted by Z n , i Z_{n,i} Z n , i a n : = 1 / φ n a_n:=1/\varphi_n a n := 1/ φ n Z n , i ∼ Gamma ( a n , a n ) Z_{n,i}\sim\operatorname{Gamma}(a_n,a_n) Z n , i ∼ Gamma ( a n , a n ) E [ Z n , i ] = 1 \mathbb{E}[Z_{n,i}]=1 E [ Z n , i ] = 1 z z z
z a n a n Γ ( a n ) z a n − 1 exp ( − a n z ) = a n a n + 1 Γ ( a n + 1 ) z a n exp ( − a n z ) , z\frac{a_n^{a_n}}{\Gamma(a_n)}z^{a_n-1}\exp(-a_nz)=\frac{a_n^{a_n+1}}{\Gamma(a_n+1)}z^{a_n}\exp(-a_nz), z Γ ( a n ) a n a n z a n − 1 exp ( − a n z ) = Γ ( a n + 1 ) a n a n + 1 z a n exp ( − a n z ) , which is the density of a Gamma ( a n + 1 , a n ) \operatorname{Gamma}(a_n+1,a_n) Gamma ( a n + 1 , a n ) Z n , i ∗ ∼ Gamma ( a n + 1 , a n ) Z_{n,i}^*\sim\operatorname{Gamma}(a_n+1,a_n) Z n , i ∗ ∼ Gamma ( a n + 1 , a n )
E [ Z n , i ∣ W n , i ∣ 3 ] = E [ ∣ log ( Z n , i ∗ ) ∣ 3 ] . \mathbb{E}\left[Z_{n,i}|W_{n,i}|^3\right]=\mathbb{E}\left[|\log(Z_{n,i}^*)|^3\right]. E [ Z n , i ∣ W n , i ∣ 3 ] = E [ ∣ log ( Z n , i ∗ ) ∣ 3 ] . The mean of log ( Z n , i ∗ ) \log(Z_{n,i}^*) log ( Z n , i ∗ )
ψ ( a n + 1 ) − log ( a n ) = m n + φ n = O ( φ n ) , \psi(a_n+1)-\log(a_n)=m_n+\varphi_n=O(\varphi_n), ψ ( a n + 1 ) − log ( a n ) = m n + φ n = O ( φ n ) , and its fourth centered moment is
ψ 3 ( a n + 1 ) + 3 ψ 1 ( a n + 1 ) 2 = O ( φ n 2 ) . \psi_3(a_n+1)+3\psi_1(a_n+1)^2=O(\varphi_n^2). ψ 3 ( a n + 1 ) + 3 ψ 1 ( a n + 1 ) 2 = O ( φ n 2 ) . Applying the same Lyapunov bound to the centered logarithm and then using its O ( φ n ) O(\varphi_n) O ( φ n )
E [ ∣ log ( Z n , i ∗ ) ∣ 3 ] = O ( φ n 3 2 ) . \mathbb{E}\left[|\log(Z_{n,i}^*)|^3\right]=O(\varphi_n^{\frac{3}{2}}). E [ ∣ log ( Z n , i ∗ ) ∣ 3 ] = O ( φ n 2 3 ) . Combining the weighted and unweighted bounds gives
E [ ( 1 + Z n , i ) ∣ W n , i ∣ 3 ] = O ( φ n 3 2 ) . (68) \mathbb{E}\left[(1+Z_{n,i})|W_{n,i}|^3\right]=O(\varphi_n^{\frac{3}{2}}). \tag*{(68)} E [ ( 1 + Z n , i ) ∣ W n , i ∣ 3 ] = O ( φ n 2 3 ) . (68) Next, we establish the rates of the explicit estimators β ~ n \widetilde{\beta}_n β n φ ~ n \widetilde{\varphi}_n φ n
β ~ n − β 0 = ( X n T X n ) − 1 X n T W n = m n e 0 + ( X n T X n ) − 1 X n T ( W n − m n 1 n ) . \widetilde{\beta}_n-\beta_0=(X_n^{\mathsf T}X_n)^{-1}X_n^{\mathsf T}W_n=m_n e_0+(X_n^{\mathsf T}X_n)^{-1}X_n^{\mathsf T}(W_n-m_n\boldsymbol{1}_n). β n − β 0 = ( X n T X n ) − 1 X n T W n = m n e 0 + ( X n T X n ) − 1 X n T ( W n − m n 1 n ) . Since the components of W n − m n 1 n W_n-m_n\boldsymbol{1}_n W n − m n 1 n ψ 1 ( 1 / φ n ) \psi_1(1/\varphi_n) ψ 1 ( 1/ φ n ) n − 1 X n T X n n^{-1}X_n^{\mathsf T}X_n n − 1 X n T X n
( X n T X n ) − 1 X n T ( ψ 1 ( 1 φ n ) I n ) X n ( X n T X n ) − 1 = ψ 1 ( 1 / φ n ) ( X n T X n ) − 1 . (X_n^{\mathsf T}X_n)^{-1}X_n^{\mathsf T}\left(\psi_1\left(\frac{1}{\varphi_n}\right)I_n\right)X_n(X_n^{\mathsf T}X_n)^{-1}=\psi_1(1/\varphi_n)(X_n^{\mathsf T}X_n)^{-1}. ( X n T X n ) − 1 X n T ( ψ 1 ( φ n 1 ) I n ) X n ( X n T X n ) − 1 = ψ 1 ( 1/ φ n ) ( X n T X n ) − 1 . The design assumptions and (A.20) therefore give
( X n T X n ) − 1 X n T ( W n − m n 1 ⃗ n ) = O P ( φ n n ) . (\mathbf{X}_n^{\mathsf{T}}\mathbf{X}_n)^{-1}\mathbf{X}_n^{\mathsf{T}}(\mathbf{W}_n-m_n\vec{\mathbf{1}}_n)=O_{\mathbb{P}}\left(\sqrt{\frac{\varphi_n}{n}}\right). ( X n T X n ) − 1 X n T ( W n − m n 1 n ) = O P ( n φ n ) . Together with m n = O ( φ n ) m_n=O(\varphi_n) m n = O ( φ n )
φ ~ n = 1 n ( W n − m n 1 ⃗ n ) T M n ( W n − m n 1 ⃗ n ) . \widetilde{\varphi}_n=\frac{1}{n}(\mathbf{W}_n-m_n\vec{\mathbf{1}}_n)^{\mathsf{T}}\mathbf{M}_n(\mathbf{W}_n-m_n\vec{\mathbf{1}}_n). φ n = n 1 ( W n − m n 1 n ) T M n ( W n − m n 1 n ) . Because M n \mathbf{M}_n M n n − k n-k n − k Tr ( M n ) = n − k \operatorname{Tr}(\mathbf{M}_n)=n-k Tr ( M n ) = n − k [35 ] gives
E [ φ ~ n ] = ψ 1 ( 1 φ n ) ( 1 − k n ) = φ n + O ( φ n 2 + φ n n ) . \mathbb{E}[\widetilde{\varphi}_n]=\psi_1\left(\frac{1}{\varphi_n}\right)\left(1-\frac{k}{n}\right)=\varphi_n+O\left(\varphi_n^2+\frac{\varphi_n}{n}\right). E [ φ n ] = ψ 1 ( φ n 1 ) ( 1 − n k ) = φ n + O ( φ n 2 + n φ n ) . Now, if ε \boldsymbol{\varepsilon} ε σ 2 \sigma^2 σ 2 κ 4 = E [ ε i 4 ] − 3 σ 4 \kappa_4=\mathbb{E}[\varepsilon_i^4]-3\sigma^4 κ 4 = E [ ε i 4 ] − 3 σ 4 A \mathbf{A} A [35 ] gives
Var ( ε T A ε ) = 2 σ 4 Tr ( A 2 ) + κ 4 ∑ i = 1 n A i i 2 . (69) \operatorname{Var}(\boldsymbol{\varepsilon}^{\mathsf{T}}\mathbf{A}\boldsymbol{\varepsilon})=2\sigma^4\operatorname{Tr}(\mathbf{A}^2)+\kappa_4\sum_{i=1}^{n}A_{ii}^2. \tag*{(69)} Var ( ε T A ε ) = 2 σ 4 Tr ( A 2 ) + κ 4 i = 1 ∑ n A ii 2 . (69) where σ 2 : = ψ 1 ( 1 / φ n ) \sigma^2:=\psi_1(1/\varphi_n) σ 2 := ψ 1 ( 1/ φ n ) κ 4 : = ψ 3 ( 1 / φ n ) \kappa_4:=\psi_3(1/\varphi_n) κ 4 := ψ 3 ( 1/ φ n ) A : = n − 1 M n \mathbf{A}:=n^{-1}\mathbf{M}_n A := n − 1 M n
Var ( φ ~ n ) = 2 ψ 1 ( 1 / φ n ) 2 n 2 Tr ( M n 2 ) + ψ 3 ( 1 / φ n ) n 2 ∑ i = 1 n ( M n ) i i 2 . \operatorname{Var}(\widetilde{\varphi}_n)=\frac{2\psi_1(1/\varphi_n)^2}{n^2}\operatorname{Tr}(\mathbf{M}_n^2)+\frac{\psi_3(1/\varphi_n)}{n^2}\sum_{i=1}^{n}(\mathbf{M}_n)_{ii}^2. Var ( φ n ) = n 2 2 ψ 1 ( 1/ φ n ) 2 Tr ( M n 2 ) + n 2 ψ 3 ( 1/ φ n ) i = 1 ∑ n ( M n ) ii 2 . Since M n 2 = M n \mathbf{M}_n^2=\mathbf{M}_n M n 2 = M n Tr ( M n 2 ) = n − k \operatorname{Tr}(\mathbf{M}_n^2)=n-k Tr ( M n 2 ) = n − k 0 ≤ ( M n ) i i ≤ 1 0\leq(\mathbf{M}_n)_{ii}\leq1 0 ≤ ( M n ) ii ≤ 1
∑ i = 1 n ( M n ) i i 2 ≤ n . \sum_{i=1}^{n}(\mathbf{M}_n)_{ii}^2\leq n. i = 1 ∑ n ( M n ) ii 2 ≤ n . Together with the polygamma expansions, these bounds give
Var ( φ ~ n ) = O ( φ n 2 n ) , \operatorname{Var}(\widetilde{\varphi}_n)=O\left(\frac{\varphi_n^2}{n}\right), Var ( φ n ) = O ( n φ n 2 ) , which proves (A.17).
Next, recall from § 3.3 that h ( z ) = z − 1 − log ( z ) h(z)=z-1-\log(z) h ( z ) = z − 1 − log ( z ) δ ∈ R k \boldsymbol{\delta}\in\mathbb{R}^k δ ∈ R k
A n ( δ ) : = 1 n ∑ i = 1 n h ( Y n , i exp ( − x n , i T ( β 0 + δ ) ) ) = 1 n ∑ i = 1 n h ( Z n , i exp ( − x n , i T δ ) ) . A_n(\boldsymbol{\delta}):=\frac{1}{n}\sum_{i=1}^{n}h\left(Y_{n,i}\exp(-\mathbf{x}_{n,i}^{\mathsf{T}}(\boldsymbol{\beta}_0+\boldsymbol{\delta}))\right)=\frac{1}{n}\sum_{i=1}^{n}h\left(Z_{n,i}\exp(-\mathbf{x}_{n,i}^{\mathsf{T}}\boldsymbol{\delta})\right). A n ( δ ) := n 1 i = 1 ∑ n h ( Y n , i exp ( − x n , i T ( β 0 + δ )) ) = n 1 i = 1 ∑ n h ( Z n , i exp ( − x n , i T δ ) ) . For each fixed φ > 0 \varphi>0 φ > 0 δ \boldsymbol{\delta} δ ( n / φ ) A n ( δ ) (n/\varphi)A_n(\boldsymbol{\delta}) ( n / φ ) A n ( δ ) β = β 0 + δ \boldsymbol{\beta}=\boldsymbol{\beta}_0+\boldsymbol{\delta} β = β 0 + δ ∇ A n ( 0 ) \nabla A_n(0) ∇ A n ( 0 ) β 0 \boldsymbol{\beta}_0 β 0 φ / n \varphi/n φ / n
∇ A n ( δ ) = − 1 n X n T ( Z n ( β 0 + δ ) − 1 ⃗ n ) and ∇ 2 A n ( δ ) = 1 n X n T diag ( Z n ( β 0 + δ ) ) X n . \nabla A_n(\boldsymbol{\delta})=-\frac{1}{n}\mathbf{X}_n^{\mathsf{T}}\left(\mathbf{Z}_n(\boldsymbol{\beta}_0+\boldsymbol{\delta})-\vec{\mathbf{1}}_n\right)
\quad\text{and}\quad
\nabla^2 A_n(\boldsymbol{\delta})=\frac{1}{n}\mathbf{X}_n^{\mathsf{T}}\operatorname{diag}(\mathbf{Z}_n(\boldsymbol{\beta}_0+\boldsymbol{\delta}))\mathbf{X}_n. ∇ A n ( δ ) = − n 1 X n T ( Z n ( β 0 + δ ) − 1 n ) and ∇ 2 A n ( δ ) = n 1 X n T diag ( Z n ( β 0 + δ )) X n . Since n − 1 X n T X n → S n^{-1}\mathbf{X}_n^{\mathsf{T}}\mathbf{X}_n\to\mathbf{S} n − 1 X n T X n → S S \mathbf{S} S X n \mathbf{X}_n X n n n n v ∈ R k \mathbf{v}\in\mathbb{R}^k v ∈ R k
v T ∇ 2 A n ( δ ) v = 1 n ∑ i = 1 n Z n , i exp ( − x n , i T δ ) ( x n , i T v ) 2 > 0 \mathbf{v}^{\mathsf{T}}\nabla^2 A_n(\boldsymbol{\delta})\mathbf{v}=\frac{1}{n}\sum_{i=1}^{n}Z_{n,i}\exp(-\mathbf{x}_{n,i}^{\mathsf{T}}\boldsymbol{\delta})(\mathbf{x}_{n,i}^{\mathsf{T}}\mathbf{v})^2>0 v T ∇ 2 A n ( δ ) v = n 1 i = 1 ∑ n Z n , i exp ( − x n , i T δ ) ( x n , i T v ) 2 > 0 whenever X n \mathbf{X}_n X n A n A_n A n coercive , meaning that A n ( δ ) ⟶ ∞ A_n(\boldsymbol{\delta})\longrightarrow\infty A n ( δ ) ⟶ ∞ ∥ δ ∥ ⟶ ∞ \lVert\boldsymbol{\delta}\rVert\longrightarrow\infty ∥ δ ∥ ⟶ ∞ { δ j } \{\boldsymbol{\delta}_j\} { δ j } ∥ δ j ∥ → ∞ \lVert\boldsymbol{\delta}_j\rVert\to\infty ∥ δ j ∥ → ∞ A n A_n A n
δ j ∥ δ j ∥ ⟶ v \frac{\boldsymbol{\delta}_j}{\lVert\boldsymbol{\delta}_j\rVert}\longrightarrow\mathbf{v} ∥ δ j ∥ δ j ⟶ v for some unit vector v v v x n , i T v ≠ 0 x_{n,i}^{\mathsf{T}}v \ne0 x n , i T v = 0 i i i ∣ x n , i T δ j ∣ → ∞ \lvert x_{n,i}^{\mathsf{T}}\delta_j\rvert\to\infty ∣ x n , i T δ j ∣ → ∞ h ( t ) → ∞ h(t) \to\infty h ( t ) → ∞ t ↓ 0 t \downarrow0 t ↓ 0 t → ∞ t \to\infty t → ∞ A n ( δ j ) A_n(\delta_j) A n ( δ j ) h h h A n ( δ j ) → ∞ A_n(\delta_j) \to\infty A n ( δ j ) → ∞ A n A_n A n R k \mathbb{R}^k R k [7 Proposition 3.2.1] , and strict convexity makes the minimizer δ ^ n \widehat{\delta}_n δ n
At zero,
∇ A n ( 0 ) = − 1 n X n T ( Z n − 1 ⃗ n ) = O P ( φ n n ) . (70) \nabla A_n(0)=-\frac{1}{n}X_n^{\mathsf{T}}(Z_n-\vec{1}_n)=O_{\mathbb{P}}\left(\sqrt{\frac{\varphi_n}{n}}\right). \tag*{(70)} ∇ A n ( 0 ) = − n 1 X n T ( Z n − 1 n ) = O P ( n φ n ) . (70) Moreover,
∇ 2 A n ( 0 ) − 1 n X n T X n = 1 n ∑ i = 1 n ( Z n , i − 1 ) x n , i x n , i T . \nabla^2 A_n(0)-\frac{1}{n}X_n^{\mathsf{T}}X_n=\frac{1}{n}\sum_{i=1}^{n}(Z_{n,i}-1)x_{n,i}x_{n,i}^{\mathsf{T}}. ∇ 2 A n ( 0 ) − n 1 X n T X n = n 1 i = 1 ∑ n ( Z n , i − 1 ) x n , i x n , i T . By independence, E [ Z n , i − 1 ] = 0 \mathbb{E}[Z_{n,i}-1]=0 E [ Z n , i − 1 ] = 0 Var ( Z n , i ) = φ n \operatorname{Var}(Z_{n,i})=\varphi_n Var ( Z n , i ) = φ n
E [ ∥ ∇ 2 A n ( 0 ) − 1 n X n T X n ∥ F 2 ] = φ n n 2 ∑ i = 1 n ∥ x n , i x n , i T ∥ F 2 = φ n n 2 ∑ i = 1 n ∥ x n , i ∥ 4 ≤ C 4 φ n n , \mathbb{E}\left[\left\|\nabla^2 A_n(0)-\frac{1}{n}X_n^{\mathsf{T}}X_n\right\|_{\mathrm{F}}^2\right]=\frac{\varphi_n}{n^2}\sum_{i=1}^{n}\left\|x_{n,i}x_{n,i}^{\mathsf{T}}\right\|_{\mathrm{F}}^2=\frac{\varphi_n}{n^2}\sum_{i=1}^{n}\|x_{n,i}\|^4\leq\frac{C^4\varphi_n}{n}, E [ ∇ 2 A n ( 0 ) − n 1 X n T X n F 2 ] = n 2 φ n i = 1 ∑ n x n , i x n , i T F 2 = n 2 φ n i = 1 ∑ n ∥ x n , i ∥ 4 ≤ n C 4 φ n , where the constant C C C max 1 ≤ i ≤ n ∥ x n , i ∥ ≤ C \max_{1\leq i\leq n}\|x_{n,i}\|\leq C max 1 ≤ i ≤ n ∥ x n , i ∥ ≤ C n n n (29) . It follows that
∇ 2 A n ( 0 ) = 1 n X n T X n + O P ( φ n n ) = 1 n X n T X n + o P ( 1 ) . \nabla^2 A_n(0)=\frac{1}{n}X_n^{\mathsf{T}}X_n+O_{\mathbb{P}}\left(\sqrt{\frac{\varphi_n}{n}}\right)=\frac{1}{n}X_n^{\mathsf{T}}X_n+o_{\mathbb{P}}(1). ∇ 2 A n ( 0 ) = n 1 X n T X n + O P ( n φ n ) = n 1 X n T X n + o P ( 1 ) . Now suppose that ∥ δ ∥ ≤ ε \|\delta\|\leq\varepsilon ∥ δ ∥ ≤ ε
∣ x n , i T δ ∣ ≤ ∥ x n , i ∥ ∥ δ ∥ ≤ C ε , |x_{n,i}^{\mathsf{T}}\delta|\leq\|x_{n,i}\|\|\delta\|\leq C\varepsilon, ∣ x n , i T δ ∣ ≤ ∥ x n , i ∥∥ δ ∥ ≤ C ε , and hence
exp ( − x n , i T δ ) ≥ exp ( − C ε ) . \exp(-x_{n,i}^{\mathsf{T}}\delta)\geq\exp(-C\varepsilon). exp ( − x n , i T δ ) ≥ exp ( − C ε ) . Consequently, for every v ∈ R k v\in\mathbb{R}^k v ∈ R k
v T ∇ 2 A n ( δ ) v = 1 n ∑ i = 1 n Z n , i exp ( − x n , i T δ ) ( x n , i T v ) 2 ≥ exp ( − C ε ) 1 n ∑ i = 1 n Z n , i ( x n , i T v ) 2 = exp ( − C ε ) v T ∇ 2 A n ( 0 ) v . \begin{aligned}
v^{\mathsf{T}}\nabla^2 A_n(\delta)v
&=\frac{1}{n}\sum_{i=1}^{n}Z_{n,i}\exp(-x_{n,i}^{\mathsf{T}}\delta)(x_{n,i}^{\mathsf{T}}v)^2\\
&\geq\exp(-C\varepsilon)\frac{1}{n}\sum_{i=1}^{n}Z_{n,i}(x_{n,i}^{\mathsf{T}}v)^2\\
&=\exp(-C\varepsilon)v^{\mathsf{T}}\nabla^2 A_n(0)v.
\end{aligned} v T ∇ 2 A n ( δ ) v = n 1 i = 1 ∑ n Z n , i exp ( − x n , i T δ ) ( x n , i T v ) 2 ≥ exp ( − C ε ) n 1 i = 1 ∑ n Z n , i ( x n , i T v ) 2 = exp ( − C ε ) v T ∇ 2 A n ( 0 ) v . Therefore,
∇ 2 A n ( δ ) ⪰ exp ( − C ε ) ∇ 2 A n ( 0 ) \nabla^2 A_n(\delta)\succeq\exp(-C\varepsilon)\nabla^2 A_n(0) ∇ 2 A n ( δ ) ⪰ exp ( − C ε ) ∇ 2 A n ( 0 ) uniformly over ∥ δ ∥ ≤ ε \|\delta\|\leq\varepsilon ∥ δ ∥ ≤ ε n − 1 X n T X n → S n^{-1}X_n^{\mathsf{T}}X_n\to S n − 1 X n T X n → S S S S c > 0 c>0 c > 0
inf ∥ δ ∥ ≤ ε λ min ( ∇ 2 A n ( δ ) ) ≥ c . \inf_{\|\delta\|\leq\varepsilon}\lambda_{\min}\left(\nabla^2 A_n(\delta)\right)\geq c. ∥ δ ∥ ≤ ε inf λ m i n ( ∇ 2 A n ( δ ) ) ≥ c . On this event, for ∥ δ ∥ = ε \|\delta\|=\varepsilon ∥ δ ∥ = ε
A n ( δ ) − A n ( 0 ) = ∇ A n ( 0 ) T δ + ∫ 0 1 ( 1 − t ) δ T ∇ 2 A n ( t δ ) δ d t ≥ − ε ∥ ∇ A n ( 0 ) ∥ + c ε 2 2 . A_n(\delta)-A_n(0)=\nabla A_n(0)^{\mathsf{T}}\delta+\int_{0}^{1}(1-t)\delta^{\mathsf{T}}\nabla^2 A_n(t\delta)\delta\,\mathrm{d}t\geq-\varepsilon\|\nabla A_n(0)\|+\frac{c\varepsilon^2}{2}. A n ( δ ) − A n ( 0 ) = ∇ A n ( 0 ) T δ + ∫ 0 1 ( 1 − t ) δ T ∇ 2 A n ( t δ ) δ d t ≥ − ε ∥∇ A n ( 0 ) ∥ + 2 c ε 2 . By (A.23), the right side is positive with probability tending to one. Convexity therefore implies that ∥ δ ^ n ∥ < ε \|\widehat{\delta}_n\|<\varepsilon ∥ δ n ∥ < ε
0 = ∇ A n ( 0 ) + ( ∫ 0 1 ∇ 2 A n ( t δ ^ n ) d t ) δ ^ n . 0=\nabla A_n(0)+\left(\int_{0}^{1}\nabla^2 A_n(t\widehat{\delta}_n)\,\mathrm{d}t\right)\widehat{\delta}_n. 0 = ∇ A n ( 0 ) + ( ∫ 0 1 ∇ 2 A n ( t δ n ) d t ) δ n . The matrix in braces has smallest eigenvalue at least c c c
∥ δ ^ n ∥ ≤ 1 c ∥ ∇ A n ( 0 ) ∥ = O P ( φ n n ) . \lVert\hat{\delta}_n\rVert\le\frac{1}{c}\lVert\nabla A_n(0)\rVert= O_{\mathrm{P}}\left(\sqrt{\frac{\varphi_n}{n}}\right). ∥ δ ^ n ∥ ≤ c 1 ∥ ∇ A n ( 0 )∥ = O P ( n φ n ) . Consequently,
β ^ n − β 0 = δ ^ n = O P ( φ n n ) , (71) \hat{\beta}_n-\beta_0=\hat{\delta}_n=O_{\mathrm{P}}\left(\sqrt{\frac{\varphi_n}{n}}\right), \tag*{(71)} β ^ n − β 0 = δ ^ n = O P ( n φ n ) , (71) which is (A.18).
Finally, since h ( t ) ≥ 0 h(t)\geq0 h ( t ) ≥ 0 t = 1 t=1 t = 1 A n ( δ ^ n ) = 0 A_n(\hat{\delta}_n)=0 A n ( δ ^ n ) = 0 i = 1 , … , n i=1,\ldots,n i = 1 , … , n
Z n , i exp ( − x n , i T δ ^ n ) = 1. Z_{n,i}\exp\left(-x_{n,i}^{\mathsf{T}}\hat{\delta}_n\right)=1. Z n , i exp ( − x n , i T δ ^ n ) = 1. Equivalently, L n L_n L n X n X_n X n n > k n>k n > k L n L_n L n col ( X n ) \operatorname{col}(X_n) col ( X n ) R n \mathbb{R}^n R n A n ( δ ^ n ) > 0 A_n(\hat{\delta}_n)>0 A n ( δ ^ n ) > 0 n n n
g ( φ ^ n ) = A n ( δ ^ n ) where g ( φ ) = − log ( φ ) − ψ ( 1 φ ) . (72) g(\hat{\varphi}_n)=A_n(\hat{\delta}_n)\quad\text{where}\quad g(\varphi)=-\log(\varphi)-\psi\left(\frac{1}{\varphi}\right). \tag*{(72)} g ( φ ^ n ) = A n ( δ ^ n ) where g ( φ ) = − log ( φ ) − ψ ( φ 1 ) . (72) The function g g g ( 0 , ∞ ) (0,\infty) ( 0 , ∞ ) x > 0 x>0 x > 0
ψ 1 ( x ) = ∑ j = 0 ∞ 1 ( x + j ) 2 > ∫ 0 ∞ d t ( x + t ) 2 = 1 x , \psi_1(x)=\sum_{j=0}^{\infty}\frac{1}{(x+j)^2}>\int_0^{\infty}\frac{dt}{(x+t)^2}=\frac{1}{x}, ψ 1 ( x ) = j = 0 ∑ ∞ ( x + j ) 2 1 > ∫ 0 ∞ ( x + t ) 2 d t = x 1 , so
g ′ ( φ ) = ψ 1 ( 1 / φ ) − φ φ 2 > 0. g'(\varphi)=\frac{\psi_1(1/\varphi)-\varphi}{\varphi^2}>0. g ′ ( φ ) = φ 2 ψ 1 ( 1/ φ ) − φ > 0. Moreover, g ( φ ) → 0 g(\varphi)\to0 g ( φ ) → 0 φ ↓ 0 \varphi\downarrow0 φ ↓ 0 g ( φ ) = φ / 2 + O ( φ 2 ) g(\varphi)=\varphi/2+O(\varphi^2) g ( φ ) = φ /2 + O ( φ 2 )
ψ ( 1 φ ) = ψ ( 1 + 1 φ ) − φ \psi\left(\frac{1}{\varphi}\right)=\psi\left(1+\frac{1}{\varphi}\right)-\varphi ψ ( φ 1 ) = ψ ( 1 + φ 1 ) − φ [2 Equation 6.3.5] , together with ψ ( 1 ) = − γ \psi(1)=-\gamma ψ ( 1 ) = − γ [2 Equation 6.3.2] , gives
g ( φ ) = φ − log ( φ ) − ψ ( 1 + 1 φ ) = φ − log ( φ ) + γ + o ( 1 ) g(\varphi)=\varphi-\log(\varphi)-\psi\left(1+\frac{1}{\varphi}\right)=\varphi-\log(\varphi)+\gamma+o(1) g ( φ ) = φ − log ( φ ) − ψ ( 1 + φ 1 ) = φ − log ( φ ) + γ + o ( 1 ) as φ → ∞ \varphi\to\infty φ → ∞ g ( φ ) → ∞ g(\varphi)\to\infty g ( φ ) → ∞ a : = 1 / φ a:=1/\varphi a := 1/ φ a a a n ( g ( 1 / a ) − A n ( δ ^ n ) ) n\left(g(1/a)-A_n(\hat{\delta}_n)\right) n ( g ( 1/ a ) − A n ( δ ^ n ) )
E [ A n ( 0 ) ] = g ( φ n ) . \mathbb{E}[A_n(0)]=g(\varphi_n). E [ A n ( 0 )] = g ( φ n ) . Using Cov ( Z n , i , log ( Z n , i ) ) = φ n \operatorname{Cov}(Z_{n,i},\log(Z_{n,i}))=\varphi_n Cov ( Z n , i , log ( Z n , i )) = φ n
Var ( h ( Z n , i ) ) = ψ 1 ( 1 φ n ) − φ n = φ n 2 2 + O ( φ n 3 ) . \operatorname{Var}(h(Z_{n,i}))=\psi_1\left(\frac{1}{\varphi_n}\right)-\varphi_n=\frac{\varphi_n^2}{2}+O(\varphi_n^3). Var ( h ( Z n , i )) = ψ 1 ( φ n 1 ) − φ n = 2 φ n 2 + O ( φ n 3 ) . It follows that
A n ( 0 ) = g ( φ n ) + O P ( φ n n ) . A_n(0)=g(\varphi_n)+O_{\mathrm{P}}\left(\frac{\varphi_n}{\sqrt{n}}\right). A n ( 0 ) = g ( φ n ) + O P ( n φ n ) . A Taylor expansion of A n A_n A n δ ^ n \hat{\delta}_n δ ^ n
A n ( δ ^ n ) − A n ( 0 ) = ∇ A n ( 0 ) T δ ^ n + ∫ 0 1 ( 1 − t ) δ ^ n T ∇ 2 A n ( t δ ^ n ) δ ^ n d t = O P ( φ n n ) , A_n(\hat{\delta}_n)-A_n(0)=\nabla A_n(0)^{\mathsf{T}}\hat{\delta}_n+\int_0^1(1-t)\hat{\delta}_n^{\mathsf{T}}\nabla^2 A_n(t\hat{\delta}_n)\hat{\delta}_n\,dt=O_{\mathrm{P}}\left(\frac{\varphi_n}{n}\right), A n ( δ ^ n ) − A n ( 0 ) = ∇ A n ( 0 ) T δ ^ n + ∫ 0 1 ( 1 − t ) δ ^ n T ∇ 2 A n ( t δ ^ n ) δ ^ n d t = O P ( n φ n ) , where we used the fact that the Hessian is uniformly O P ( 1 ) O_{\mathrm{P}}(1) O P ( 1 ) δ ^ n \widehat{\delta}_n δ n
g ( φ ^ n ) − g ( φ n ) = O P ( φ n n ) . g(\widehat{\varphi}_n)-g(\varphi_n)=O_{\mathrm{P}}\left(\frac{\varphi_n}{\sqrt{n}}\right). g ( φ n ) − g ( φ n ) = O P ( n φ n ) . The preceding bounds also imply g ( φ ^ n ) = O P ( φ n ) g(\widehat{\varphi}_n)=O_{\mathrm{P}}(\varphi_n) g ( φ n ) = O P ( φ n ) φ ^ n → p 0 \widehat{\varphi}_n \xrightarrow{\mathrm{p}} 0 φ n p 0 g ′ ( φ ) = 1 / 2 + O ( φ ) g'(\varphi)=1/2+O(\varphi) g ′ ( φ ) = 1/2 + O ( φ ) φ ↓ 0 \varphi\downarrow0 φ ↓ 0
φ ^ n − φ n = O P ( φ n n ) , \widehat{\varphi}_n-\varphi_n=O_{\mathrm{P}}\left(\frac{\varphi_n}{\sqrt{n}}\right), φ n − φ n = O P ( n φ n ) , which is (A.19).
Lemma A.11. Under the assumptions of Lemma A.10 and § 3.3 , let
θ n = ( φ n , β 0 T ) T and u n = ϕ ( θ n ) . \theta_n=(\varphi_n,\boldsymbol{\beta}_0^{\mathsf T})^{\mathsf T}
\quad\text{and}\quad
\boldsymbol{u}_n=\phi(\theta_n). θ n = ( φ n , β 0 T ) T and u n = ϕ ( θ n ) . Write P n : = P n , θ n \mathbb{P}_n:=\mathbb{P}_{n,\theta_n} P n := P n , θ n D n D_n D n u n \boldsymbol{u}_n u n
P n ( u ^ n , u ~ n ∈ D n ) ⟶ 1 and sup u ∈ D n ∥ u ∥ = O ( φ n ) . \mathbb{P}_n(\widehat{\boldsymbol{u}}_n,\widetilde{\boldsymbol{u}}_n\in D_n)\longrightarrow1
\quad\text{and}\quad
\sup_{\boldsymbol{u}\in D_n}\|\boldsymbol{u}\|=O(\varphi_n). P n ( u n , u n ∈ D n ) ⟶ 1 and u ∈ D n sup ∥ u ∥ = O ( φ n ) . Moreover, for Q n Q_n Q n Q ~ n \widetilde{Q}_n Q n (33) and (34) ,
sup u ∈ D n ∥ Q ~ n ( u ) − Q n ( u ) ∥ = o P ( φ n ) . (73) \sup_{\boldsymbol{u}\in D_n}\left\|\widetilde{Q}_n(\boldsymbol{u})-Q_n(\boldsymbol{u})\right\|=o_{\mathrm{P}}(\varphi_n). \tag*{(73)} u ∈ D n sup Q n ( u ) − Q n ( u ) = o P ( φ n ) . (73) Proof. Put
η φ , n : = φ n + n − 1 / 2 and η β , n : = φ n + φ n n . \eta_{\varphi,n}:=\varphi_n+n^{-1/2}
\quad\text{and}\quad
\eta_{\beta,n}:=\varphi_n+\sqrt{\frac{\varphi_n}{n}}. η φ , n := φ n + n − 1/2 and η β , n := φ n + n φ n . Because
η β , n 2 φ n = O ( φ n + φ n n + 1 n ) ⟶ 0 , \frac{\eta_{\beta,n}^{2}}{\varphi_n}
=O\left(\varphi_n+\sqrt{\frac{\varphi_n}{n}}+\frac{1}{n}\right)\longrightarrow0, φ n η β , n 2 = O ( φ n + n φ n + n 1 ) ⟶ 0 , we may choose a deterministic t n → ∞ t_n\to\infty t n → ∞
r φ , n : = t n η φ , n ⟶ 0 and r β , n : = t n η β , n ⟶ 0 and r β , n 2 = o ( φ n ) . r_{\varphi,n}:=t_n\eta_{\varphi,n}\longrightarrow0
\quad\text{and}\quad
r_{\beta,n}:=t_n\eta_{\beta,n}\longrightarrow0
\quad\text{and}\quad
r_{\beta,n}^{2}=o(\varphi_n). r φ , n := t n η φ , n ⟶ 0 and r β , n := t n η β , n ⟶ 0 and r β , n 2 = o ( φ n ) . With u 1 , n : = φ n / 2 u_{1,n}:=\varphi_n/2 u 1 , n := φ n /2
D n : = { u = ( u 1 , u 2 ) T : ∣ u 1 u 1 , n − 1 ∣ ≤ r φ , n , ∥ u 2 − u 1 β 0 ∥ ≤ r β , n u 1 } . (74) D_n:=\left\{\boldsymbol{u}=(u_1,\boldsymbol{u}_2)^{\mathsf T}:
\left|\frac{u_1}{u_{1,n}}-1\right|\le r_{\varphi,n},\
\|\boldsymbol{u}_2-u_1\boldsymbol{\beta}_0\|\le r_{\beta,n}u_1\right\}. \tag*{(74)} D n := { u = ( u 1 , u 2 ) T : u 1 , n u 1 − 1 ≤ r φ , n , ∥ u 2 − u 1 β 0 ∥ ≤ r β , n u 1 } . (74) For all sufficiently large n n n r φ , n < 1 r_{\varphi,n}<1 r φ , n < 1 D n ⊂ U = ( 0 , ∞ ) × R k D_n\subset U=(0,\infty)\times\mathbb{R}^k D n ⊂ U = ( 0 , ∞ ) × R k u n ∈ D n \boldsymbol{u}_n\in D_n u n ∈ D n D n D_n D n v , w ∈ D n \boldsymbol{v},\boldsymbol{w}\in D_n v , w ∈ D n v = ( v 1 , v 2 ) T \boldsymbol{v}=(v_1,\boldsymbol{v}_2)^{\mathsf T} v = ( v 1 , v 2 ) T w = ( w 1 , w 2 ) T \boldsymbol{w}=(w_1,\boldsymbol{w}_2)^{\mathsf T} w = ( w 1 , w 2 ) T λ ∈ [ 0 , 1 ] \lambda\in[0,1] λ ∈ [ 0 , 1 ] λ v + ( 1 − λ ) w \lambda\boldsymbol{v}+(1-\lambda)\boldsymbol{w} λ v + ( 1 − λ ) w
∥ λ v 2 + ( 1 − λ ) w 2 − ( λ v 1 + ( 1 − λ ) w 1 ) β 0 ∥ ≤ λ r β , n v 1 + ( 1 − λ ) r β , n w 1 = r β , n ( λ v 1 + ( 1 − λ ) w 1 ) . \left\|\lambda\boldsymbol{v}_2+(1-\lambda)\boldsymbol{w}_2-(\lambda v_1+(1-\lambda)w_1)\boldsymbol{\beta}_0\right\|
\le\lambda r_{\beta,n}v_1+(1-\lambda)r_{\beta,n}w_1
=r_{\beta,n}(\lambda v_1+(1-\lambda)w_1). ∥ λ v 2 + ( 1 − λ ) w 2 − ( λ v 1 + ( 1 − λ ) w 1 ) β 0 ∥ ≤ λ r β , n v 1 + ( 1 − λ ) r β , n w 1 = r β , n ( λ v 1 + ( 1 − λ ) w 1 ) . Thus λ v + ( 1 − λ ) w ∈ D n \lambda\boldsymbol{v}+(1-\lambda)\boldsymbol{w}\in D_n λ v + ( 1 − λ ) w ∈ D n
For either estimator, denote its original coordinates by ( φ n ∗ , β n ∗ ) (\varphi_n^*,\boldsymbol{\beta}_n^*) ( φ n ∗ , β n ∗ )
u n ∗ : = ϕ ( φ n ∗ , β n ∗ ) = ( u 1 , n ∗ u 2 , n ∗ ) . \boldsymbol{u}_n^*:=\phi(\varphi_n^*,\boldsymbol{\beta}_n^*)
=\begin{pmatrix}u_{1,n}^*\\u_{2,n}^*\end{pmatrix}. u n ∗ := ϕ ( φ n ∗ , β n ∗ ) = ( u 1 , n ∗ u 2 , n ∗ ) . These chart coordinates satisfy
∣ u 1 , n ∗ u 1 , n − 1 ∣ = ∣ φ n ∗ φ n − 1 ∣ and ∥ u 2 , n ∗ − u 1 , n ∗ β 0 ∥ u 1 , n ∗ = ∥ β n ∗ − β 0 ∥ . \left|\frac{u_{1,n}^*}{u_{1,n}}-1\right|
=\left|\frac{\varphi_n^*}{\varphi_n}-1\right|
\quad\text{and}\quad
\frac{\left\|u_{2,n}^*-u_{1,n}^*\boldsymbol{\beta}_0\right\|}{u_{1,n}^*}
=\left\|\boldsymbol{\beta}_n^*-\boldsymbol{\beta}_0\right\|. u 1 , n u 1 , n ∗ − 1 = φ n φ n ∗ − 1 and u 1 , n ∗ u 2 , n ∗ − u 1 , n ∗ β 0 = ∥ β n ∗ − β 0 ∥ . The rates in Lemma A.10 show that the two quantities on the right are O P ( η φ , n ) O_{\mathbb{P}}(\eta_{\varphi,n}) O P ( η φ , n ) O P ( η β , n ) O_{\mathbb{P}}(\eta_{\beta,n}) O P ( η β , n ) η φ , n = o ( r φ , n ) \eta_{\varphi,n}=o(r_{\varphi,n}) η φ , n = o ( r φ , n ) η β , n = o ( r β , n ) \eta_{\beta,n}=o(r_{\beta,n}) η β , n = o ( r β , n ) D n D_n D n D n D_n D n
sup u ∈ D n ∥ u ∥ = O ( φ n ) . \sup_{\mathbf{u}\in D_n}\|\mathbf{u}\|=O(\varphi_n). u ∈ D n sup ∥ u ∥ = O ( φ n ) . Finally, for u ∈ D n \mathbf{u}\in D_n u ∈ D n
β ( u ) : = u 2 u 1 and δ ( u ) : = β ( u ) − β 0 , \beta(\mathbf{u}):=\frac{u_2}{u_1}
\qquad\text{and}\qquad
\delta(\mathbf{u}):=\beta(\mathbf{u})-\beta_0, β ( u ) := u 1 u 2 and δ ( u ) := β ( u ) − β 0 , so that sup u ∈ D n ∥ δ ( u ) ∥ ≤ r β , n \sup_{\mathbf{u}\in D_n}\|\delta(\mathbf{u})\|\le r_{\beta,n} sup u ∈ D n ∥ δ ( u ) ∥ ≤ r β , n
r n ( u ) = L n − X n u 2 u 1 . r_n(\mathbf{u})=L_n-X_n\frac{u_2}{u_1}. r n ( u ) = L n − X n u 1 u 2 . is the log-residual vector. The least-squares decomposition gives
1 2 n ∥ r n ( u ) ∥ 2 = u ~ 1 , n + 1 2 ( β ( u ) − β ~ n ) T 1 n X n T X n ( β ( u ) − β ~ n ) . \frac{1}{2n}\|r_n(\mathbf{u})\|^2=\widetilde{u}_{1,n}+\frac{1}{2}\bigl(\beta(\mathbf{u})-\widetilde{\beta}_n\bigr)^\mathsf{T}\frac{1}{n}X_n^\mathsf{T}X_n\bigl(\beta(\mathbf{u})-\widetilde{\beta}_n\bigr). 2 n 1 ∥ r n ( u ) ∥ 2 = u 1 , n + 2 1 ( β ( u ) − β n ) T n 1 X n T X n ( β ( u ) − β n ) . Since β ~ n − β 0 = O P ( η β , n ) = o P ( r β , n ) \widetilde{\beta}_n-\beta_0=O_{\mathbb{P}}(\eta_{\beta,n})=o_{\mathbb{P}}(r_{\beta,n}) β n − β 0 = O P ( η β , n ) = o P ( r β , n ) n − 1 X n T X n → S n^{-1}X_n^\mathsf{T}X_n\to S n − 1 X n T X n → S O P ( r β , n 2 ) O_{\mathbb{P}}(r_{\beta,n}^2) O P ( r β , n 2 ) D n D_n D n
Recalling that h ( z ) = z − 1 − log ( z ) h(z)=z-1-\log(z) h ( z ) = z − 1 − log ( z )
h ( e x ) − x 2 2 = e x − 1 − x − x 2 2 . h(e^x)-\frac{x^2}{2}=e^x-1-x-\frac{x^2}{2}. h ( e x ) − 2 x 2 = e x − 1 − x − 2 x 2 . Taylor’s theorem gives, for some fixed C > 1 C>1 C > 1 x ∈ R x\in\mathbb{R} x ∈ R
∣ h ( e x ) − x 2 2 ∣ ≤ C ( 1 + e x ) ∣ x ∣ 3 . \left|h(e^x)-\frac{x^2}{2}\right|\le C(1+e^x)|x|^3. h ( e x ) − 2 x 2 ≤ C ( 1 + e x ) ∣ x ∣ 3 . Writing W n , i : = log ( Z n , i ) W_{n,i}:=\log(Z_{n,i}) W n , i := log ( Z n , i ) C C C r β , n → 0 r_{\beta,n}\to0 r β , n → 0 n n n u ∈ D n \mathbf{u}\in D_n u ∈ D n
∣ r n , i ( u ) − W n , i ∣ = ∣ x n , i T δ ( u ) ∣ ≤ C r β , n and exp ( r n , i ( u ) ) = Z n , i exp ( − x n , i T δ ( u ) ) ≤ C Z n , i . |r_{n,i}(\mathbf{u})-W_{n,i}|=|x_{n,i}^{\mathsf{T}}\delta(\mathbf{u})|\le Cr_{\beta,n}
\qquad\text{and}\qquad
\exp(r_{n,i}(\mathbf{u}))=Z_{n,i}\exp(-x_{n,i}^{\mathsf{T}}\delta(\mathbf{u}))\le CZ_{n,i}. ∣ r n , i ( u ) − W n , i ∣ = ∣ x n , i T δ ( u ) ∣ ≤ C r β , n and exp ( r n , i ( u )) = Z n , i exp ( − x n , i T δ ( u )) ≤ C Z n , i . Since ( a + b ) 3 ≤ 4 ( a 3 + b 3 ) (a+b)^3\le4(a^3+b^3) ( a + b ) 3 ≤ 4 ( a 3 + b 3 ) a , b ≥ 0 a,b\ge0 a , b ≥ 0
sup u ∈ D n 1 n ∑ i = 1 n ( 1 + exp ( r n , i ( u ) ) ) ∣ r n , i ( u ) ∣ 3 ≤ 4 C 4 ( 1 n ∑ i = 1 n ( 1 + Z n , i ) ∣ W n , i ∣ 3 + r β , n 3 1 n ∑ i = 1 n ( 1 + Z n , i ) ) . \sup_{\mathbf{u}\in D_n}\frac{1}{n}\sum_{i=1}^n(1+\exp(r_{n,i}(\mathbf{u})))|r_{n,i}(\mathbf{u})|^3
\le4C^4\left(\frac{1}{n}\sum_{i=1}^n(1+Z_{n,i})|W_{n,i}|^3+r_{\beta,n}^3\frac{1}{n}\sum_{i=1}^n(1+Z_{n,i})\right). u ∈ D n sup n 1 i = 1 ∑ n ( 1 + exp ( r n , i ( u ))) ∣ r n , i ( u ) ∣ 3 ≤ 4 C 4 ( n 1 i = 1 ∑ n ( 1 + Z n , i ) ∣ W n , i ∣ 3 + r β , n 3 n 1 i = 1 ∑ n ( 1 + Z n , i ) ) . By (A.21) and Markov’s inequality, the first average on the right is O P ( φ n 3 / 2 ) O_{\mathbb{P}}(\varphi_n^{3/2}) O P ( φ n 3/2 ) O P ( 1 ) O_{\mathbb{P}}(1) O P ( 1 )
sup u ∈ D n 1 n ∑ i = 1 n ∣ h ( exp ( r n , i ( u ) ) ) − 1 2 r n , i ( u ) 2 ∣ = O P ( φ n 3 / 2 + r β , n 3 ) . (75) \sup_{\mathbf{u}\in D_n}\frac{1}{n}\sum_{i=1}^n\left|h(\exp(r_{n,i}(\mathbf{u})))-\frac{1}{2}r_{n,i}(\mathbf{u})^2\right|
=O_{\mathbb{P}}\left(\varphi_n^{3/2}+r_{\beta,n}^3\right). \tag*{(75)} u ∈ D n sup n 1 i = 1 ∑ n h ( exp ( r n , i ( u ))) − 2 1 r n , i ( u ) 2 = O P ( φ n 3/2 + r β , n 3 ) . (75) The first coordinate difference can be written as
Q ~ n , 1 ( u ) − Q n , 1 ( u ) = ( u 1 − g ( 2 u 1 ) ) + [ 1 n ∑ i = 1 n h ( exp ( r n , i ( u ) ) ) − 1 2 n ∥ r n ( u ) ∥ 2 ] + [ 1 2 n ∥ r n ( u ) ∥ 2 − u ~ 1 , n ] . \widetilde{Q}_{n,1}(\mathbf{u})-Q_{n,1}(\mathbf{u})
=(u_1-g(2u_1))+\left[\frac{1}{n}\sum_{i=1}^n h(\exp(r_{n,i}(\mathbf{u})))-\frac{1}{2n}\|r_n(\mathbf{u})\|^2\right]
+\left[\frac{1}{2n}\|r_n(\mathbf{u})\|^2-\widetilde{u}_{1,n}\right]. Q n , 1 ( u ) − Q n , 1 ( u ) = ( u 1 − g ( 2 u 1 )) + [ n 1 i = 1 ∑ n h ( exp ( r n , i ( u ))) − 2 n 1 ∥ r n ( u ) ∥ 2 ] + [ 2 n 1 ∥ r n ( u ) ∥ 2 − u 1 , n ] . Because u 1 = O ( φ n ) u_1=O(\varphi_n) u 1 = O ( φ n ) D n D_n D n g ( 2 u 1 ) = u 1 + O ( u 1 2 ) g(2u_1)=u_1+O(u_1^2) g ( 2 u 1 ) = u 1 + O ( u 1 2 ) O ( φ n 2 ) O(\varphi_n^2) O ( φ n 2 ) O P ( φ n 3 / 2 + r β , n 3 ) O_{\mathbb{P}}(\varphi_n^{3/2}+r_{\beta,n}^3) O P ( φ n 3/2 + r β , n 3 ) O P ( r β , n 2 ) O_{\mathbb{P}}(r_{\beta,n}^2) O P ( r β , n 2 ) D n D_n D n r β , n → 0 r_{\beta,n}\to0 r β , n → 0
sup u ∈ D n ∣ Q ~ n , 1 ( u ) − Q n , 1 ( u ) ∣ = O P ( φ n 3 / 2 + r β , n 2 ) . \sup_{\mathbf{u}\in D_n}|\widetilde{Q}_{n,1}(\mathbf{u})-Q_{n,1}(\mathbf{u})|
=O_{\mathbb{P}}\left(\varphi_n^{3/2}+r_{\beta,n}^2\right). u ∈ D n sup ∣ Q n , 1 ( u ) − Q n , 1 ( u ) ∣ = O P ( φ n 3/2 + r β , n 2 ) . For the remaining coordinates, the identity e x − 1 − x = h ( e x ) e^x-1-x=h(e^x) e x − 1 − x = h ( e x )
Q ~ n , 2 ( u ) − Q n , 2 ( u ) = − u 1 n X n T [ Z n ( β ( u ) ) − 1 ⃗ n − r n ( u ) ] . \widetilde{Q}_{n,2}(u)-Q_{n,2}(u)=-\frac{u_1}{n}X_n^{\mathsf{T}}\left[Z_n(\beta(u))-\vec{1}_n-r_n(u)\right]. Q n , 2 ( u ) − Q n , 2 ( u ) = − n u 1 X n T [ Z n ( β ( u )) − 1 n − r n ( u ) ] . where the ith component of the bracketed vector is h ( exp ( r n , i ( u ) ) ) h(\exp(r_{n,i}(u))) h ( exp ( r n , i ( u )))
sup u ∈ D n 1 n ∑ i = 1 n h ( exp ( r n , i ( u ) ) ) = O P ( φ n + r β , n 2 ) . \sup_{u\in D_n}\frac{1}{n}\sum_{i=1}^{n}h(\exp(r_{n,i}(u)))=O_{\mathbb{P}}\left(\varphi_n+r_{\beta,n}^{2}\right). u ∈ D n sup n 1 i = 1 ∑ n h ( exp ( r n , i ( u ))) = O P ( φ n + r β , n 2 ) . Since u 1 = O ( φ n ) u_1=O(\varphi_n) u 1 = O ( φ n ) D n D_n D n
sup u ∈ D n ∥ Q ~ n , 2 ( u ) − Q n , 2 ( u ) ∥ = O P ( φ n ( φ n + r β , n 2 ) ) . \sup_{u\in D_n}\left\|\widetilde{Q}_{n,2}(u)-Q_{n,2}(u)\right\|=O_{\mathbb{P}}\left(\varphi_n(\varphi_n+r_{\beta,n}^{2})\right). u ∈ D n sup Q n , 2 ( u ) − Q n , 2 ( u ) = O P ( φ n ( φ n + r β , n 2 ) ) . Therefore
sup u ∈ D n ∥ Q ~ n ( u ) − Q n ( u ) ∥ = O P ( φ n 3 2 + r β , n 2 ) = o P ( φ n ) , \sup_{u\in D_n}\left\|\widetilde{Q}_n(u)-Q_n(u)\right\|=O_{\mathbb{P}}\left(\varphi_n^{\frac{3}{2}}+r_{\beta,n}^{2}\right)=o_{\mathbb{P}}(\varphi_n), u ∈ D n sup Q n ( u ) − Q n ( u ) = O P ( φ n 2 3 + r β , n 2 ) = o P ( φ n ) , which is (A.26).
Lemma A.12. Under the assumptions of § 3.3 , the approximate and exact estimators satisfy (38) and (39) , respectively. Moreover, if n φ n 4 → 0 n\varphi_n^4\to0 n φ n 4 → 0 (41) satisfy (42) and are asymptotically equivalent to the exact MLE as in (43) .
Proof. Continue to write
W n , i = log ( Z n , i ) and m n = E [ W n , i ] and ϵ n , i : = W n , i − m n and v n : = Var ( W n , i ) . W_{n,i}=\log(Z_{n,i}) \quad\text{and} \quad m_n=\mathbb{E}[W_{n,i}] \quad\text{and} \quad\epsilon_{n,i}:=W_{n,i}-m_n \quad\text{and} \quad v_n:=\operatorname{Var}(W_{n,i}). W n , i = log ( Z n , i ) and m n = E [ W n , i ] and ϵ n , i := W n , i − m n and v n := Var ( W n , i ) . We first collect the moments needed to compute the limiting covariance matrices and verify the Lyapunov conditions. By Lemma A.1, the cumulant generating function of W n , i W_{n,i} W n , i
K n ( t ) : = log ( E [ e t W n , i ] ) = log ( Γ ( 1 φ n + t ) ) − log ( Γ ( 1 φ n ) ) + t log ( φ n ) . K_n(t):=\log\left(\mathbb{E}\left[e^{tW_{n,i}}\right]\right)=\log\left(\Gamma\left(\frac{1}{\varphi_n}+t\right)\right)-\log\left(\Gamma\left(\frac{1}{\varphi_n}\right)\right)+t\log(\varphi_n). K n ( t ) := log ( E [ e t W n , i ] ) = log ( Γ ( φ n 1 + t ) ) − log ( Γ ( φ n 1 ) ) + t log ( φ n ) . Thus, its cumulants of orders r ≥ 2 r\ge2 r ≥ 2 ψ r − 1 ( 1 / φ n ) \psi_{r-1}(1/\varphi_n) ψ r − 1 ( 1/ φ n )
m n = ψ ( 1 φ n ) + log ( φ n ) = − φ n 2 − φ n 2 12 + O ( φ n 4 ) , v n = ψ 1 ( 1 φ n ) = φ n + φ n 2 2 + φ n 3 6 + O ( φ n 5 ) , E [ ϵ n , i 3 ] = ψ 2 ( 1 φ n ) = − φ n 2 + O ( φ n 3 ) , E [ ϵ n , i 4 ] = ψ 3 ( 1 φ n ) + 3 v n 2 = 3 φ n 2 + O ( φ n 3 ) . (76) \begin{aligned}
m_n&=\psi\left(\frac{1}{\varphi_n}\right)+\log(\varphi_n)=-\frac{\varphi_n}{2}-\frac{\varphi_n^2}{12}+O(\varphi_n^4),\\
v_n&=\psi_1\left(\frac{1}{\varphi_n}\right)=\varphi_n+\frac{\varphi_n^2}{2}+\frac{\varphi_n^3}{6}+O(\varphi_n^5),\\
\mathbb{E}[\epsilon_{n,i}^3]&=\psi_2\left(\frac{1}{\varphi_n}\right)=-\varphi_n^2+O(\varphi_n^3),\\
\mathbb{E}[\epsilon_{n,i}^4]&=\psi_3\left(\frac{1}{\varphi_n}\right)+3v_n^2=3\varphi_n^2+O(\varphi_n^3). \tag*{(76)}
\end{aligned} m n v n E [ ϵ n , i 3 ] E [ ϵ n , i 4 ] = ψ ( φ n 1 ) + log ( φ n ) = − 2 φ n − 12 φ n 2 + O ( φ n 4 ) , = ψ 1 ( φ n 1 ) = φ n + 2 φ n 2 + 6 φ n 3 + O ( φ n 5 ) , = ψ 2 ( φ n 1 ) = − φ n 2 + O ( φ n 3 ) , = ψ 3 ( φ n 1 ) + 3 v n 2 = 3 φ n 2 + O ( φ n 3 ) . (76) These expansions will determine the covariance matrices below. For the Lyapunov conditions, we will use the three bounds
E [ ∣ ϵ n , i ∣ 6 ] = O ( φ n 3 ) , (77) \mathbb{E}[|\epsilon_{n,i}|^6]=O(\varphi_n^3), \tag*{(77)} E [ ∣ ϵ n , i ∣ 6 ] = O ( φ n 3 ) , (77) E [ ∣ Z n , i − 1 ∣ 3 ] = O ( φ n 3 2 ) , (78) \mathbb{E}[|Z_{n,i}-1|^3]=O(\varphi_n^{\frac{3}{2}}), \tag*{(78)} E [ ∣ Z n , i − 1 ∣ 3 ] = O ( φ n 2 3 ) , (78) and
E [ ∣ h ( Z n , i ) − g ( φ n ) ∣ 3 ] = O ( φ n 3 ) . (79) \mathbb{E}[|h(Z_{n,i})-g(\varphi_n)|^3]=O(\varphi_n^3). \tag*{(79)} E [ ∣ h ( Z n , i ) − g ( φ n ) ∣ 3 ] = O ( φ n 3 ) . (79) The identity [40 ]
μ 6 = κ 6 + 15 κ 4 κ 2 + 10 κ 3 2 + 15 κ 2 3 (80) \mu_6=\kappa_6+15\kappa_4\kappa_2+10\kappa_3^2+15\kappa_2^3 \tag*{(80)} μ 6 = κ 6 + 15 κ 4 κ 2 + 10 κ 3 2 + 15 κ 2 3 (80) expresses the sixth centered moment of W n , i W_{n,i} W n , i
E [ ϵ n , i 6 ] = ψ 5 ( 1 φ n ) + 15 ψ 3 ( 1 φ n ) v n + 10 ψ 2 ( 1 φ n ) 2 + 15 v n 3 . (81) \mathbb{E}[\epsilon_{n,i}^{6}] = \psi_{5}\left(\frac{1}{\varphi_{n}}\right) + 15\psi_{3}\left(\frac{1}{\varphi_{n}}\right)v_{n} + 10\psi_{2}\left(\frac{1}{\varphi_{n}}\right)^{2} + 15v_{n}^{3}. \tag*{(81)} E [ ϵ n , i 6 ] = ψ 5 ( φ n 1 ) + 15 ψ 3 ( φ n 1 ) v n + 10 ψ 2 ( φ n 1 ) 2 + 15 v n 3 . (81) The polygamma expansions (A.29) therefore give E [ ∣ ϵ n , i ∣ 6 ] = O ( φ n 3 ) \mathbb{E}[|\epsilon_{n,i}|^{6}] = O(\varphi_{n}^{3}) E [ ∣ ϵ n , i ∣ 6 ] = O ( φ n 3 ) Z n , i Z_{n,i} Z n , i
E [ ( Z n , i − 1 ) 4 ] = 3 φ n 2 + 6 φ n 3 . \mathbb{E}\left[(Z_{n,i}-1)^{4}\right] = 3\varphi_{n}^{2} + 6\varphi_{n}^{3}. E [ ( Z n , i − 1 ) 4 ] = 3 φ n 2 + 6 φ n 3 . Hölder’s inequality gives
E [ ∣ Z n , i − 1 ∣ 3 ] ≤ ( E [ ( Z n , i − 1 ) 4 ] ) 3 4 = O ( φ n 3 2 ) , \mathbb{E}\left[|Z_{n,i}-1|^{3}\right] \leq\left(\mathbb{E}\left[(Z_{n,i}-1)^{4}\right]\right)^{\frac{3}{4}} = O(\varphi_{n}^{\frac{3}{2}}), E [ ∣ Z n , i − 1 ∣ 3 ] ≤ ( E [ ( Z n , i − 1 ) 4 ] ) 4 3 = O ( φ n 2 3 ) , which proves (A.31).
It remains to control the third absolute moment involving h h h
∣ h ( e w ) ∣ 3 ≤ C ( 1 + e 3 w ) ∣ w ∣ 6 , |h(e^{w})|^{3} \leq C(1+e^{3w})|w|^{6}, ∣ h ( e w ) ∣ 3 ≤ C ( 1 + e 3 w ) ∣ w ∣ 6 , we have
E [ ∣ h ( Z n , i ) ∣ 3 ] ≤ C ( E [ ∣ W n , i ∣ 6 ] + E [ Z n , i 3 ∣ W n , i ∣ 6 ] ) . (82) \mathbb{E}[|h(Z_{n,i})|^{3}] \leq C\left(\mathbb{E}[|W_{n,i}|^{6}] + \mathbb{E}[Z_{n,i}^{3}|W_{n,i}|^{6}]\right). \tag*{(82)} E [ ∣ h ( Z n , i ) ∣ 3 ] ≤ C ( E [ ∣ W n , i ∣ 6 ] + E [ Z n , i 3 ∣ W n , i ∣ 6 ] ) . (82) The bound in (A.30), together with m n = O ( φ n ) m_{n}=O(\varphi_{n}) m n = O ( φ n ) E [ ∣ W n , i ∣ 6 ] = O ( φ n 3 ) \mathbb{E}[|W_{n,i}|^{6}]=O(\varphi_{n}^{3}) E [ ∣ W n , i ∣ 6 ] = O ( φ n 3 ) a n : = 1 / φ n a_{n}:=1/\varphi_{n} a n := 1/ φ n Z n , i ∗ ∼ Gamma ( a n + 3 , a n ) Z_{n,i}^{*}\sim\operatorname{Gamma}(a_{n}+3,a_{n}) Z n , i ∗ ∼ Gamma ( a n + 3 , a n ) Gamma ( a n , a n ) \operatorname{Gamma}(a_{n},a_{n}) Gamma ( a n , a n ) z 3 z^{3} z 3 E [ Z n , i 3 ] \mathbb{E}[Z_{n,i}^{3}] E [ Z n , i 3 ] Gamma ( a n + 3 , a n ) \operatorname{Gamma}(a_{n}+3,a_{n}) Gamma ( a n + 3 , a n )
E [ Z n , i 3 ∣ W n , i ∣ 6 ] = E [ Z n , i 3 ] E [ ∣ log ( Z n , i ∗ ) ∣ 6 ] . \mathbb{E}[Z_{n,i}^{3}|W_{n,i}|^{6}] = \mathbb{E}[Z_{n,i}^{3}]\mathbb{E}[|\log(Z_{n,i}^{*})|^{6}]. E [ Z n , i 3 ∣ W n , i ∣ 6 ] = E [ Z n , i 3 ] E [ ∣ log ( Z n , i ∗ ) ∣ 6 ] . Here
E [ Z n , i 3 ] = ( 1 + φ n ) ( 1 + 2 φ n ) = O ( 1 ) , \mathbb{E}[Z_{n,i}^{3}] = (1+\varphi_{n})(1+2\varphi_{n}) = O(1), E [ Z n , i 3 ] = ( 1 + φ n ) ( 1 + 2 φ n ) = O ( 1 ) , while the mean of log ( Z n , i ∗ ) \log(Z_{n,i}^{*}) log ( Z n , i ∗ ) ψ ( a n + 3 ) − log ( a n ) = O ( φ n ) \psi(a_{n}+3)-\log(a_{n})=O(\varphi_{n}) ψ ( a n + 3 ) − log ( a n ) = O ( φ n ) r ≥ 2 r\geq2 r ≥ 2 ψ r − 1 ( a n + 3 ) \psi_{r-1}(a_{n}+3) ψ r − 1 ( a n + 3 )
E [ ∣ log ( Z n , i ∗ ) − E [ log ( Z n , i ∗ ) ] ∣ 6 ] = O ( φ n 3 ) . \mathbb{E}\left[\left|\log(Z_{n,i}^{*})-\mathbb{E}[\log(Z_{n,i}^{*})]\right|^{6}\right] = O(\varphi_{n}^{3}). E [ log ( Z n , i ∗ ) − E [ log ( Z n , i ∗ )] 6 ] = O ( φ n 3 ) . Together with the O ( φ n ) O(\varphi_{n}) O ( φ n ) E [ ∣ log ( Z n , i ∗ ) ∣ 6 ] = O ( φ n 3 ) \mathbb{E}[|\log(Z_{n,i}^{*})|^{6}]=O(\varphi_{n}^{3}) E [ ∣ log ( Z n , i ∗ ) ∣ 6 ] = O ( φ n 3 ) E [ ∣ h ( Z n , i ) ∣ 3 ] = O ( φ n 3 ) \mathbb{E}[|h(Z_{n,i})|^{3}]=O(\varphi_{n}^{3}) E [ ∣ h ( Z n , i ) ∣ 3 ] = O ( φ n 3 ) g ( φ n ) = E [ h ( Z n , i ) ] = O ( φ n ) g(\varphi_{n})=\mathbb{E}[h(Z_{n,i})]=O(\varphi_{n}) g ( φ n ) = E [ h ( Z n , i )] = O ( φ n )
E [ ∣ h ( Z n , i ) − g ( φ n ) ∣ 3 ] = O ( φ n 3 ) , \mathbb{E}[|h(Z_{n,i})-g(\varphi_{n})|^{3}] = O(\varphi_{n}^{3}), E [ ∣ h ( Z n , i ) − g ( φ n ) ∣ 3 ] = O ( φ n 3 ) , which proves (A.32). To connect these bounds with the arrays below, note that (A.30) also implies
E [ ∣ ϵ n , i ∣ 3 ] = O ( φ n 3 2 ) and E [ ∣ ϵ n , i 2 − v n ∣ 3 ] = O ( φ n 3 ) . (83) \mathbb{E}[|\epsilon_{n,i}|^{3}] = O(\varphi_{n}^{\frac{3}{2}}) \quad\text{and}\quad\mathbb{E}[|\epsilon_{n,i}^{2}-v_{n}|^{3}] = O(\varphi_{n}^{3}). \tag*{(83)} E [ ∣ ϵ n , i ∣ 3 ] = O ( φ n 2 3 ) and E [ ∣ ϵ n , i 2 − v n ∣ 3 ] = O ( φ n 3 ) . (83) The bounds in (A.36) apply to the approximate estimator array, while (A.31) and (A.32) apply to the exact score array.
We first prove the approximate estimator limit. Since the first column of X n \mathbf{X}_{n} X n 1 n \mathbf{1}_{n} 1 n
β ~ n − β 0 − m n e 0 = ( X n T X n ) − 1 X n T ϵ n and ϵ n = ( ϵ n , 1 , … , ϵ n , n ) T (84) \widetilde{\boldsymbol{\beta}}_{n}-\boldsymbol{\beta}_{0}-m_{n}\mathbf{e}_{0}=(\mathbf{X}_{n}^{\mathsf{T}}\mathbf{X}_{n})^{-1}\mathbf{X}_{n}^{\mathsf{T}}\boldsymbol{\epsilon}_{n}\quad\text{and}\quad\boldsymbol{\epsilon}_{n}=(\epsilon_{n,1},\ldots,\epsilon_{n,n})^{\mathsf{T}} \tag*{(84)} β n − β 0 − m n e 0 = ( X n T X n ) − 1 X n T ϵ n and ϵ n = ( ϵ n , 1 , … , ϵ n , n ) T (84) and
φ ~ n = 1 n ϵ n T M n ϵ n . \widetilde{\varphi}_{n}=\frac{1}{n}\boldsymbol{\epsilon}_{n}^{\mathsf{T}}\mathbf{M}_{n}\boldsymbol{\epsilon}_{n}. φ n = n 1 ϵ n T M n ϵ n . Since E [ ϵ n ] = 0 \mathbb{E}[\boldsymbol{\epsilon}_{n}]=0 E [ ϵ n ] = 0 Var ( ϵ n ) = v n I n \operatorname{Var}(\boldsymbol{\epsilon}_{n})=v_{n}\mathbf{I}_{n} Var ( ϵ n ) = v n I n Tr ( M n ) = n − k \operatorname{Tr}(\mathbf{M}_{n})=n-k Tr ( M n ) = n − k [35 ] gives
E [ φ ~ n ] = c n = ( 1 − k n ) v n . \mathbb{E}[\widetilde{\varphi}_{n}]=c_{n}=\left(1-\frac{k}{n}\right)v_{n}. E [ φ n ] = c n = ( 1 − n k ) v n . The design assumptions imply
max 1 ≤ i ≤ n ( H n ) i i = O ( 1 n ) and Tr ( H n ) = Tr ( H n 2 ) = k . \max_{1\le i\le n}(\mathbf{H}_n)_{ii}=O\left(\frac{1}{n}\right)\quad\text{and}\quad\operatorname{Tr}(\mathbf{H}_n)=\operatorname{Tr}(\mathbf{H}_n^2)=k. 1 ≤ i ≤ n max ( H n ) ii = O ( n 1 ) and Tr ( H n ) = Tr ( H n 2 ) = k . The variance formula for quadratic forms (A.22) and (A.29) give
Var ( ε n T H n ε n ) = 2 ν n 2 Tr ( H n 2 ) + ψ 3 ( 1 φ n ) ∑ i = 1 n ( H n ) i i 2 = O ( φ n 2 ) + O ( φ n 3 n ) , \operatorname{Var}(\varepsilon_n^{\mathsf{T}}\mathbf{H}_n\varepsilon_n)=2\nu_n^2\operatorname{Tr}(\mathbf{H}_n^2)+\psi_3\left(\frac{1}{\varphi_n}\right)\sum_{i=1}^{n}(\mathbf{H}_n)_{ii}^2=O(\varphi_n^2)+O\left(\frac{\varphi_n^3}{n}\right), Var ( ε n T H n ε n ) = 2 ν n 2 Tr ( H n 2 ) + ψ 3 ( φ n 1 ) i = 1 ∑ n ( H n ) ii 2 = O ( φ n 2 ) + O ( n φ n 3 ) , where ∑ i ( H n ) i i 2 ≤ k max i ( H n ) i i = O ( n − 1 ) \sum_i(\mathbf{H}_n)_{ii}^2\le k\max_i(\mathbf{H}_n)_{ii}=O(n^{-1}) ∑ i ( H n ) ii 2 ≤ k max i ( H n ) ii = O ( n − 1 )
Var ( ε n T H n ε n − k ν n n φ n ) = O ( 1 n ) + O ( φ n n 2 ) = o ( 1 ) , \operatorname{Var}\left(\frac{\varepsilon_n^{\mathsf{T}}\mathbf{H}_n\varepsilon_n-k\nu_n}{\sqrt{n}\varphi_n}\right)=O\left(\frac{1}{n}\right)+O\left(\frac{\varphi_n}{n^2}\right)=o(1), Var ( n φ n ε n T H n ε n − k ν n ) = O ( n 1 ) + O ( n 2 φ n ) = o ( 1 ) , and Chebyshev’s inequality yields
1 n φ n ( ε n T H n ε n − k ν n ) = O P ( 1 ) . (85) \frac{1}{\sqrt{n}\varphi_n}(\varepsilon_n^{\mathsf{T}}\mathbf{H}_n\varepsilon_n-k\nu_n)=O_{\mathbb{P}}(1). \tag*{(85)} n φ n 1 ( ε n T H n ε n − k ν n ) = O P ( 1 ) . (85) Now, consider the independent triangular array
1 n ∑ i = 1 n ( ε n , i 2 − ν n φ n x n , i ε n , i φ n ) . \frac{1}{\sqrt{n}}\sum_{i=1}^{n}\begin{pmatrix}\dfrac{\varepsilon_{n,i}^2-\nu_n}{\varphi_n}\\[4pt]\dfrac{x_{n,i}\varepsilon_{n,i}}{\sqrt{\varphi_n}}\end{pmatrix}. n 1 i = 1 ∑ n φ n ε n , i 2 − ν n φ n x n , i ε n , i . Write x ˉ n : = n − 1 ∑ i = 1 n x n , i \bar{x}_n:=n^{-1}\sum_{i=1}^{n}x_{n,i} x ˉ n := n − 1 ∑ i = 1 n x n , i
( ψ 3 ( 1 / φ n ) + 2 ν n 2 φ n 2 ψ 2 ( 1 / φ n ) φ n 3 / 2 x ˉ n T ψ 2 ( 1 / φ n ) φ n 3 / 2 x ˉ n ν n n φ n X n T X n ) . \begin{pmatrix}
\dfrac{\psi_3(1/\varphi_n)+2\nu_n^2}{\varphi_n^2} &
\dfrac{\psi_2(1/\varphi_n)}{\varphi_n^{3/2}}\bar{x}_n^{\mathsf{T}}\\[8pt]
\dfrac{\psi_2(1/\varphi_n)}{\varphi_n^{3/2}}\bar{x}_n &
\dfrac{\nu_n}{n\varphi_n}\mathbf{X}_n^{\mathsf{T}}\mathbf{X}_n
\end{pmatrix}. φ n 2 ψ 3 ( 1/ φ n ) + 2 ν n 2 φ n 3/2 ψ 2 ( 1/ φ n ) x ˉ n φ n 3/2 ψ 2 ( 1/ φ n ) x ˉ n T n φ n ν n X n T X n . By (A.29), the upper-left entry converges to 2 2 2
ψ 2 ( 1 / φ n ) φ n 3 / 2 = O ( φ n ) . \frac{\psi_2(1/\varphi_n)}{\varphi_n^{3/2}}=O(\sqrt{\varphi_n}). φ n 3/2 ψ 2 ( 1/ φ n ) = O ( φ n ) . The bounded design condition gives ∥ x ˉ n ∥ = O ( 1 ) \lVert\bar{x}_n\rVert=O(1) ∥ x ˉ n ∥ = O ( 1 ) ν n n φ n X n T X n → S \dfrac{\nu_n}{n\varphi_n}\mathbf{X}_n^{\mathsf{T}}\mathbf{X}_n\to\mathbf{S} n φ n ν n X n T X n → S
( 2 0 T 0 S ) . \begin{pmatrix}
2 & \mathbf{0}^{\mathsf{T}}\\
\mathbf{0} & \mathbf{S}
\end{pmatrix}. ( 2 0 0 T S ) . In addition, the bounded design condition and (A.36) give
1 n 3 / 2 ∑ i = 1 n E [ ∥ ( ε n , i 2 − ν n φ n x n , i ε n , i φ n ) ∥ 3 ] = O ( n − 1 / 2 ) . \frac{1}{n^{3/2}}\sum_{i=1}^{n}\mathbb{E}\left[\left\lVert\begin{pmatrix}\dfrac{\varepsilon_{n,i}^2-\nu_n}{\varphi_n}\\[4pt]\dfrac{x_{n,i}\varepsilon_{n,i}}{\sqrt{\varphi_n}}\end{pmatrix}\right\rVert^3\right]=O(n^{-1/2}). n 3/2 1 i = 1 ∑ n E φ n ε n , i 2 − ν n φ n x n , i ε n , i 3 = O ( n − 1/2 ) . Thus the Lyapunov condition holds. The multivariate Lindeberg–Feller theorem [38 Theorem 2.27] , (A.37), (A.38), and n − 1 X n T X n → S n^{-1}\mathbf{X}_n^{\mathsf{T}}\mathbf{X}_n\to\mathbf{S} n − 1 X n T X n → S
We next treat the exact MLE. Consider the normalized sum of independent vectors
1 n ∑ i = 1 n ( 2 h ( Z n , i ) − g ( φ n ) φ n Z n , i − 1 φ n x n , i ) \frac{1}{\sqrt{n}}\sum_{i=1}^{n}\begin{pmatrix} 2\frac{h(Z_{n,i})-g(\varphi_n)}{\varphi_n} \\ \frac{Z_{n,i}-1}{\sqrt{\varphi_n}}x_{n,i} \end{pmatrix} n 1 i = 1 ∑ n ( 2 φ n h ( Z n , i ) − g ( φ n ) φ n Z n , i − 1 x n , i ) whose summands have mean zero. The moment identities
Var ( h ( Z n , i ) ) = ψ 1 ( 1 φ n ) − φ n = φ n 2 2 + O ( φ n 3 ) \operatorname{Var}(h(Z_{n,i}))=\psi_1\left(\frac{1}{\varphi_n}\right)-\varphi_n=\frac{\varphi_n^2}{2}+O(\varphi_n^3) Var ( h ( Z n , i )) = ψ 1 ( φ n 1 ) − φ n = 2 φ n 2 + O ( φ n 3 ) and
Cov ( Z n , i − 1 , h ( Z n , i ) ) = Var ( Z n , i ) − Cov ( Z n , i , log ( Z n , i ) ) = 0 (86) \operatorname{Cov}(Z_{n,i}-1,h(Z_{n,i}))=\operatorname{Var}(Z_{n,i})-\operatorname{Cov}(Z_{n,i},\log(Z_{n,i}))=0 \tag*{(86)} Cov ( Z n , i − 1 , h ( Z n , i )) = Var ( Z n , i ) − Cov ( Z n , i , log ( Z n , i )) = 0 (86) show that its covariance matrix is
( 4 ( ψ 1 ( 1 / φ n ) − φ n ) φ n 2 0 T 0 1 n X n T X n ) , \begin{pmatrix}
\frac{4(\psi_1(1/\varphi_n)-\varphi_n)}{\varphi_n^2} & \mathbf{0}^{\mathsf{T}} \\
\mathbf{0} & \frac{1}{n}\mathbf{X}_n^{\mathsf{T}}\mathbf{X}_n
\end{pmatrix}, ( φ n 2 4 ( ψ 1 ( 1/ φ n ) − φ n ) 0 0 T n 1 X n T X n ) , which converges to
( 2 0 T 0 S ) . \begin{pmatrix}
2 & \mathbf{0}^{\mathsf{T}} \\
\mathbf{0} & \mathbf{S}
\end{pmatrix}. ( 2 0 0 T S ) . The bounded design condition, (A.31), and (A.32) give
1 n 3 / 2 ∑ i = 1 n E [ ∥ ( 2 h ( Z n , i ) − g ( φ n ) φ n Z n , i − 1 φ n x n , i ) ∥ 3 ] = O ( n − 1 / 2 ) ⟶ 0. \frac{1}{n^{3/2}}\sum_{i=1}^{n}\mathbb{E}\left[\left\lVert\begin{pmatrix} 2\frac{h(Z_{n,i})-g(\varphi_n)}{\varphi_n} \\ \frac{Z_{n,i}-1}{\sqrt{\varphi_n}}x_{n,i} \end{pmatrix}\right\rVert^3\right]=O(n^{-1/2})\longrightarrow0. n 3/2 1 i = 1 ∑ n E ( 2 φ n h ( Z n , i ) − g ( φ n ) φ n Z n , i − 1 x n , i ) 3 = O ( n − 1/2 ) ⟶ 0. Thus the Lyapunov condition holds, and the normalized sum converges to the corresponding normal law.
It remains to relate this limit to the exact roots. The coefficient score function and (A.24) give
n φ n ( β ^ n − β 0 ) = n φ n ( X n T X n ) − 1 X n T ( Z n − 1 n ) + o P ( 1 ) . (87) \sqrt{\frac{n}{\varphi_n}}(\hat{\beta}_n-\beta_0)=\sqrt{\frac{n}{\varphi_n}}(\mathbf{X}_n^{\mathsf{T}}\mathbf{X}_n)^{-1}\mathbf{X}_n^{\mathsf{T}}(\mathbf{Z}_n-\mathbf{1}_n)+o_{\mathsf{P}}(1). \tag*{(87)} φ n n ( β ^ n − β 0 ) = φ n n ( X n T X n ) − 1 X n T ( Z n − 1 n ) + o P ( 1 ) . (87) To see this, define the integrated Hessian
S ‾ n : = ∫ 0 1 1 n X n T diag ( Z n ( β 0 + t δ ^ n ) ) X n d t . \overline{\mathbf{S}}_n:=\int_0^1\frac{1}{n}\mathbf{X}_n^{\mathsf{T}}\operatorname{diag}\left(\mathbf{Z}_n(\beta_0+t\hat{\delta}_n)\right)\mathbf{X}_n\,\mathrm{d}t. S n := ∫ 0 1 n 1 X n T diag ( Z n ( β 0 + t δ ^ n ) ) X n d t . The coefficient score equation gives
δ ^ n = S ‾ n − 1 1 n X n T ( Z n − 1 n ) . \hat{\delta}_n=\overline{\mathbf{S}}_n^{-1}\frac{1}{n}\mathbf{X}_n^{\mathsf{T}}(\mathbf{Z}_n-\mathbf{1}_n). δ ^ n = S n − 1 n 1 X n T ( Z n − 1 n ) . The calculation for ∇ 2 A n ( 0 ) \nabla^2 A_n(0) ∇ 2 A n ( 0 )
1 n X n T diag ( Z n ) X n − 1 n X n T X n = o P ( 1 ) . \frac{1}{n}\mathbf{X}_n^{\mathsf{T}}\operatorname{diag}(\mathbf{Z}_n)\mathbf{X}_n-\frac{1}{n}\mathbf{X}_n^{\mathsf{T}}\mathbf{X}_n=o_{\mathsf{P}}(1). n 1 X n T diag ( Z n ) X n − n 1 X n T X n = o P ( 1 ) . Moreover, the bounded design condition and δ ^ n = o P ( 1 ) \hat{\delta}_n=o_{\mathsf{P}}(1) δ ^ n = o P ( 1 ) t ∈ [ 0 , 1 ] t\in[0,1] t ∈ [ 0 , 1 ]
max 1 ≤ i ≤ n ∣ exp ( − t x n , i T δ ^ n ) − 1 ∣ = o P ( 1 ) . \max_{1\leq i\leq n}\left|\exp\left(-t\mathbf{x}_{n,i}^{\mathsf{T}}\hat{\delta}_n\right)-1\right|=o_{\mathsf{P}}(1). 1 ≤ i ≤ n max exp ( − t x n , i T δ ^ n ) − 1 = o P ( 1 ) . Since n − 1 ∑ i Z n , i = O P ( 1 ) n^{-1}\sum_i Z_{n,i}=O_{\mathsf{P}}(1) n − 1 ∑ i Z n , i = O P ( 1 ) S ‾ n − 1 n X n T X n = o P ( 1 ) \overline{\mathbf{S}}_n-\frac{1}{n}\mathbf{X}_n^{\mathsf{T}}\mathbf{X}_n=o_{\mathsf{P}}(1) S n − n 1 X n T X n = o P ( 1 ) S ‾ n − 1 − n ( X n T X n ) − 1 = o P ( 1 ) \overline{\mathbf{S}}_n^{-1}-n(\mathbf{X}_n^{\mathsf{T}}\mathbf{X}_n)^{-1}=o_{\mathsf{P}}(1) S n − 1 − n ( X n T X n ) − 1 = o P ( 1 ) n − 1 / 2 φ n − 1 / 2 X n T ( Z n − 1 n ) = O P ( 1 ) n^{-1/2}\varphi_n^{-1/2}\mathbf{X}_n^{\mathsf{T}}(\mathbf{Z}_n-\mathbf{1}_n)=O_{\mathsf{P}}(1) n − 1/2 φ n − 1/2 X n T ( Z n − 1 n ) = O P ( 1 )
For the dispersion coordinate, the proof of Lemma A.10 showed that
A n ( δ ^ n ) − A n ( 0 ) = O P ( φ n n ) . A_n(\hat{\delta}_n)-A_n(0)=O_{\mathbb{P}}\left(\frac{\varphi_n}{n}\right). A n ( δ ^ n ) − A n ( 0 ) = O P ( n φ n ) . Therefore,
n φ n ( A n ( δ ^ n ) − A n ( 0 ) ) = O P ( n − 1 / 2 ) = o P ( 1 ) , \frac{\sqrt{n}}{\varphi_n}\left(A_n(\hat{\delta}_n)-A_n(0)\right)=O_{\mathbb{P}}\left(n^{-1/2}\right)=o_{\mathbb{P}}(1), φ n n ( A n ( δ ^ n ) − A n ( 0 ) ) = O P ( n − 1/2 ) = o P ( 1 ) , so estimating the coefficient does not contribute to the first-order dispersion limit. Since g ′ ( φ n ) → 1 / 2 g'(\varphi_n)\to1/2 g ′ ( φ n ) → 1/2
n φ n ( φ ^ n − φ n ) = 2 n φ n ∑ i = 1 n ( h ( Z n , i ) − g ( φ n ) ) + o P ( 1 ) . (88) \frac{\sqrt{n}}{\varphi_n}(\hat{\varphi}_n-\varphi_n)=\frac{2}{\sqrt{n}\varphi_n}\sum_{i=1}^{n}\left(h(Z_{n,i})-g(\varphi_n)\right)+o_{\mathbb{P}}(1). \tag*{(88)} φ n n ( φ ^ n − φ n ) = n φ n 2 i = 1 ∑ n ( h ( Z n , i ) − g ( φ n ) ) + o P ( 1 ) . (88) Combining the joint score CLT with (A.40) and (A.41) proves (39) .
Finally, consider the bias correction. From (38) ,
v ~ n − v n = O P ( φ n n ) . \tilde{v}_n-v_n=O_{\mathbb{P}}\left(\frac{\varphi_n}{\sqrt{n}}\right). v ~ n − v n = O P ( n φ n ) . The expansions in (A.29) imply
v n − 1 2 v n 2 = φ n + O ( φ n 3 ) and 1 2 v n − 1 6 v n 2 = g ( φ n ) + O ( φ n 3 ) . (89) v_n-\frac{1}{2}v_n^2=\varphi_n+O(\varphi_n^3)
\quad\text{and}\quad
\frac{1}{2}v_n-\frac{1}{6}v_n^2=g(\varphi_n)+O(\varphi_n^3). \tag*{(89)} v n − 2 1 v n 2 = φ n + O ( φ n 3 ) and 2 1 v n − 6 1 v n 2 = g ( φ n ) + O ( φ n 3 ) . (89) Set
q φ ( v ) : = v − 1 2 v 2 and q β ( v ) : = 1 2 v − 1 6 v 2 . q_{\varphi}(v):=v-\frac{1}{2}v^2
\quad\text{and}\quad
q_{\beta}(v):=\frac{1}{2}v-\frac{1}{6}v^2. q φ ( v ) := v − 2 1 v 2 and q β ( v ) := 2 1 v − 6 1 v 2 . Since q φ ′ ( v n ) → 1 q_{\varphi}'(v_n)\to1 q φ ′ ( v n ) → 1
q φ ( v ~ n ) − q φ ( v n ) = ( 1 + o P ( 1 ) ) ( v ~ n − v n ) . q_{\varphi}(\tilde{v}_n)-q_{\varphi}(v_n)=(1+o_{\mathbb{P}}(1))(\tilde{v}_n-v_n). q φ ( v ~ n ) − q φ ( v n ) = ( 1 + o P ( 1 )) ( v ~ n − v n ) . The deterministic remainder in the first expansion of (A.42) is negligible on the dispersion scale exactly when
n φ n O ( φ n 3 ) = O ( n φ n 2 ) = o ( 1 ) . \frac{\sqrt{n}}{\varphi_n}O(\varphi_n^3)=O(\sqrt{n}\varphi_n^2)=o(1). φ n n O ( φ n 3 ) = O ( n φ n 2 ) = o ( 1 ) . This follows from n φ n 4 → 0 n\varphi_n^4\to0 n φ n 4 → 0 m n = − g ( φ n ) m_n=-g(\varphi_n) m n = − g ( φ n )
m n + q β ( v n ) = O ( φ n 3 ) , m_n+q_{\beta}(v_n)=O(\varphi_n^3), m n + q β ( v n ) = O ( φ n 3 ) , while
q β ( v ~ n ) − q β ( v n ) = O P ( φ n n ) . q_{\beta}(\tilde{v}_n)-q_{\beta}(v_n)=O_{\mathbb{P}}\left(\frac{\varphi_n}{\sqrt{n}}\right). q β ( v ~ n ) − q β ( v n ) = O P ( n φ n ) . The latter term is o P ( φ n / n ) o_{\mathbb{P}}(\sqrt{\varphi_n/n}) o P ( φ n / n )
n φ n O ( φ n 3 ) = O ( n φ n 5 / 2 ) = O ( n φ n 2 ) φ n = o ( 1 ) . \sqrt{\frac{n}{\varphi_n}}O(\varphi_n^3)=O(\sqrt{n}\varphi_n^{5/2})=O(\sqrt{n}\varphi_n^2)\sqrt{\varphi_n}=o(1). φ n n O ( φ n 3 ) = O ( n φ n 5/2 ) = O ( n φ n 2 ) φ n = o ( 1 ) . Thus the coefficient bias correction removes the intercept bias without altering its random first-order term. Combining these conclusions with (38) proves (42) .
It remains to compare the corrected and exact estimators on the same probability space. Set
r n , i : = ϵ n , i − ( Z n , i − 1 ) . r_{n,i}:=\epsilon_{n,i}-(Z_{n,i}-1). r n , i := ϵ n , i − ( Z n , i − 1 ) . The identities E [ ϵ n , i ] = E [ Z n , i − 1 ] = 0 \mathbb{E}[\epsilon_{n,i}]=\mathbb{E}[Z_{n,i}-1]=0 E [ ϵ n , i ] = E [ Z n , i − 1 ] = 0 E [ Z n , i ϵ n , i ] = φ n \mathbb{E}[Z_{n,i}\epsilon_{n,i}]=\varphi_n E [ Z n , i ϵ n , i ] = φ n
E [ r n , i ] = 0 and E [ r n , i 2 ] = v n − φ n = O ( φ n 2 ) . \mathbb{E}[r_{n,i}]=0
\quad\text{and}\quad
\mathbb{E}[r_{n,i}^2]=v_n-\varphi_n=O(\varphi_n^2). E [ r n , i ] = 0 and E [ r n , i 2 ] = v n − φ n = O ( φ n 2 ) . Writing S n : = n − 1 X n T X n \mathbf{S}_n := n^{-1}\mathbf{X}_n^{\mathsf{T}}\mathbf{X}_n S n := n − 1 X n T X n
E [ ∥ 1 n φ n X n T r n ∥ 2 ] = v n − φ n n φ n Tr ( X n T X n ) = O ( φ n ) , \mathbb{E}\left[\left\|\frac{1}{\sqrt{n\varphi_n}}\mathbf{X}_n^{\mathsf{T}}\mathbf{r}_n\right\|^2\right] = \frac{v_n-\varphi_n}{n\varphi_n}\operatorname{Tr}(\mathbf{X}_n^{\mathsf{T}}\mathbf{X}_n)=O(\varphi_n), E [ n φ n 1 X n T r n 2 ] = n φ n v n − φ n Tr ( X n T X n ) = O ( φ n ) , where r n : = ( r n , 1 , … , r n , n ) T \mathbf{r}_n := (r_{n,1},\ldots,r_{n,n})^{\mathsf{T}} r n := ( r n , 1 , … , r n , n ) T S n − 1 ( n φ n ) − 1 / 2 X n T r n = o P ( 1 ) \mathbf{S}_n^{-1}(n\varphi_n)^{-1/2}\mathbf{X}_n^{\mathsf{T}}\mathbf{r}_n=o_P(1) S n − 1 ( n φ n ) − 1/2 X n T r n = o P ( 1 )
n φ n ( β ~ n b c − β 0 ) = n φ n ( X n T X n ) − 1 X n T ϵ n + o P ( 1 ) . \sqrt{\frac{n}{\varphi_n}}(\widetilde{\boldsymbol{\beta}}_n^{\mathrm{bc}}-\boldsymbol{\beta}_0)=\sqrt{\frac{n}{\varphi_n}}(\mathbf{X}_n^{\mathsf{T}}\mathbf{X}_n)^{-1}\mathbf{X}_n^{\mathsf{T}}\boldsymbol{\epsilon}_n+o_P(1). φ n n ( β n bc − β 0 ) = φ n n ( X n T X n ) − 1 X n T ϵ n + o P ( 1 ) . Subtracting (A.40) now implies
n φ n ( β ~ n b c − β ^ n ) = o P ( 1 ) . (90) \sqrt{\frac{n}{\varphi_n}}(\widetilde{\boldsymbol{\beta}}_n^{\mathrm{bc}}-\widehat{\boldsymbol{\beta}}_n)=o_P(1). \tag*{(90)} φ n n ( β n bc − β n ) = o P ( 1 ) . (90) For the dispersion coordinate, (A.38) and k k k
n φ n ( v ˇ n − v n ) = 1 n φ n ∑ i = 1 n ( ϵ n , i 2 − v n ) + o P ( 1 ) . \frac{\sqrt{n}}{\varphi_n}(\check{v}_n-v_n)=\frac{1}{\sqrt{n}\varphi_n}\sum_{i=1}^{n}(\epsilon_{n,i}^{2}-v_n)+o_P(1). φ n n ( v ˇ n − v n ) = n φ n 1 i = 1 ∑ n ( ϵ n , i 2 − v n ) + o P ( 1 ) . The Taylor expansion of q φ ( v ˇ n ) q_{\varphi}(\check{v}_n) q φ ( v ˇ n )
n φ n ( φ ~ n b c − φ n ) = 1 n φ n ∑ i = 1 n ( ϵ n , i 2 − v n ) + o P ( 1 ) . (91) \frac{\sqrt{n}}{\varphi_n}(\widetilde{\varphi}_n^{\mathrm{bc}}-\varphi_n)=\frac{1}{\sqrt{n}\varphi_n}\sum_{i=1}^{n}(\epsilon_{n,i}^{2}-v_n)+o_P(1). \tag*{(91)} φ n n ( φ n bc − φ n ) = n φ n 1 i = 1 ∑ n ( ϵ n , i 2 − v n ) + o P ( 1 ) . (91) Define
d n , i : = ( ϵ n , i 2 − v n ) − 2 { h ( Z n , i ) − g ( φ n ) } . d_{n,i}:=(\epsilon_{n,i}^{2}-v_n)-2\{h(Z_{n,i})-g(\varphi_n)\}. d n , i := ( ϵ n , i 2 − v n ) − 2 { h ( Z n , i ) − g ( φ n )} . Since m n = − g ( φ n ) m_n=-g(\varphi_n) m n = − g ( φ n ) h ( Z n , i ) − g ( φ n ) = Z n , i − 1 − ϵ n , i h(Z_{n,i})-g(\varphi_n)=Z_{n,i}-1-\epsilon_{n,i} h ( Z n , i ) − g ( φ n ) = Z n , i − 1 − ϵ n , i E [ d n , i ] = 0 \mathbb{E}[d_{n,i}]=0 E [ d n , i ] = 0
E [ Z n , i ϵ n , i ] = φ n and E [ Z n , i ϵ n , i 2 ] = v n . \mathbb{E}[Z_{n,i}\epsilon_{n,i}]=\varphi_n \quad\text{and}\quad\mathbb{E}[Z_{n,i}\epsilon_{n,i}^{2}]=v_n. E [ Z n , i ϵ n , i ] = φ n and E [ Z n , i ϵ n , i 2 ] = v n . Using these identities and the centered log-gamma moments collected above,
E [ d n , i 2 ] = ψ 3 ( 1 / φ n ) + 2 v n 2 + 4 ( v n − φ n ) + 4 ψ 2 ( 1 / φ n ) = O ( φ n 3 ) . (92) \mathbb{E}[d_{n,i}^{2}]=\psi_3(1/\varphi_n)+2v_n^2+4(v_n-\varphi_n)+4\psi_2(1/\varphi_n)=O(\varphi_n^3). \tag*{(92)} E [ d n , i 2 ] = ψ 3 ( 1/ φ n ) + 2 v n 2 + 4 ( v n − φ n ) + 4 ψ 2 ( 1/ φ n ) = O ( φ n 3 ) . (92) It follows by Chebyshev’s inequality that
1 n φ n ∑ i = 1 n d n , i = o P ( 1 ) . \frac{1}{\sqrt{n}\varphi_n}\sum_{i=1}^{n}d_{n,i}=o_P(1). n φ n 1 i = 1 ∑ n d n , i = o P ( 1 ) . Subtracting (A.41) from (A.44) proves
n φ n ( φ ~ n b c − φ ^ n ) = o P ( 1 ) . \frac{\sqrt{n}}{\varphi_n}(\widetilde{\varphi}_n^{\mathrm{bc}}-\widehat{\varphi}_n)=o_P(1). φ n n ( φ n bc − φ n ) = o P ( 1 ) . Together with (A.43), this is (43).
Zeta Distribution
Preliminary Results
The first lemma gives the moment expansions for log ( X ) \log(X) log ( X )
Lemma B.1. If X ∼ Zeta ( s ) X \sim\operatorname{Zeta}(s) X ∼ Zeta ( s ) r = 1 , 2 , 3 r = 1,2,3 r = 1 , 2 , 3
E [ ( log ( X ) ) r ] = ( − 1 ) r ζ ( r ) ( s ) ζ ( s ) . \mathbb{E}[(\log(X))^r] = \frac{(-1)^r\zeta^{(r)}(s)}{\zeta(s)}. E [( log ( X ) ) r ] = ζ ( s ) ( − 1 ) r ζ ( r ) ( s ) . In particular,
E [ log ( X ) ] = − ζ ′ ( s ) ζ ( s ) and Var ( log ( X ) ) = ζ ′ ′ ( s ) ζ ( s ) − ( ζ ′ ( s ) ζ ( s ) ) 2 . \mathbb{E}[\log(X)] = -\frac{\zeta'(s)}{\zeta(s)}
\quad\text{and}\quad
\operatorname{Var}(\log(X)) = \frac{\zeta''(s)}{\zeta(s)}-\left(\frac{\zeta'(s)}{\zeta(s)}\right)^2. E [ log ( X )] = − ζ ( s ) ζ ′ ( s ) and Var ( log ( X )) = ζ ( s ) ζ ′′ ( s ) − ( ζ ( s ) ζ ′ ( s ) ) 2 . Moreover, as s → ∞ s \to\infty s → ∞ r = 1 , 2 , 3 r = 1,2,3 r = 1 , 2 , 3
E [ ( log ( X ) ) r ] = ( log ( 2 ) ) r 2 s + O ( 1 3 s ) , \mathbb{E}[(\log(X))^r] = \frac{(\log(2))^r}{2^s}+O\left(\frac{1}{3^s}\right), E [( log ( X ) ) r ] = 2 s ( log ( 2 ) ) r + O ( 3 s 1 ) , Var ( log ( X ) ) = ( log ( 2 ) ) 2 2 s + O ( 1 3 s ) . (93) \operatorname{Var}(\log(X)) = \frac{(\log(2))^2}{2^s}+O\left(\frac{1}{3^s}\right). \tag*{(93)} Var ( log ( X )) = 2 s ( log ( 2 ) ) 2 + O ( 3 s 1 ) . (93) Proof. The zeta function is represented on s > 1 s > 1 s > 1
ζ ( s ) = ∑ k = 1 ∞ k − s . \zeta(s) = \sum_{k=1}^{\infty} k^{-s}. ζ ( s ) = k = 1 ∑ ∞ k − s . A Dirichlet series converges uniformly on compact subsets of its half-plane of convergence and may be differentiated termwise there [3 Theorems 11.11–11.12] ; therefore the differentiated series through order three converge absolutely and uniformly on every such interval. Since
d r d s r k − s = ( − log ( k ) ) r k − s , \frac{\mathrm{d}^r}{\mathrm{d}s^r}k^{-s}=(-\log(k))^r k^{-s}, d s r d r k − s = ( − log ( k ) ) r k − s , termwise differentiation gives, for r = 1 , 2 , 3 r = 1,2,3 r = 1 , 2 , 3
ζ ( r ) ( s ) = ( − 1 ) r ∑ k = 1 ∞ ( log ( k ) ) r k s . (94) \zeta^{(r)}(s)=(-1)^r\sum_{k=1}^{\infty}\frac{(\log(k))^r}{k^s}. \tag*{(94)} ζ ( r ) ( s ) = ( − 1 ) r k = 1 ∑ ∞ k s ( log ( k ) ) r . (94) On the other hand, the mass function of X X X
E [ ( log ( X ) ) r ] = ∑ k = 1 ∞ ( log ( k ) ) r k − s ζ ( s ) = 1 ζ ( s ) ∑ k = 1 ∞ ( log ( k ) ) r k s . (95) \mathbb{E}[(\log(X))^r]=\sum_{k=1}^{\infty}(\log(k))^r\frac{k^{-s}}{\zeta(s)}
=\frac{1}{\zeta(s)}\sum_{k=1}^{\infty}\frac{(\log(k))^r}{k^s}. \tag*{(95)} E [( log ( X ) ) r ] = k = 1 ∑ ∞ ( log ( k ) ) r ζ ( s ) k − s = ζ ( s ) 1 k = 1 ∑ ∞ k s ( log ( k ) ) r . (95) Combining (B.2) and (B.3) proves the first identity. Because the k = 1 k = 1 k = 1 k = 2 k = 2 k = 2
The stated mean and variance identities follow immediately. For every fixed r ∈ N r \in\mathbb{N} r ∈ N k = 2 k = 2 k = 2 s → ∞ s \to\infty s → ∞
∑ k = 2 ∞ ( log ( k ) ) r k s = ( log ( 2 ) ) r 2 s + O ( 1 3 s ) . \sum_{k=2}^{\infty}\frac{(\log(k))^r}{k^s}=\frac{(\log(2))^r}{2^s}+O\left(\frac{1}{3^s}\right). k = 2 ∑ ∞ k s ( log ( k ) ) r = 2 s ( log ( 2 ) ) r + O ( 3 s 1 ) . where the implied constant may depend on r r r ζ ( s ) = 1 + 2 − s + O ( 3 − s ) \zeta(s)=1+2^{-s}+O(3^{-s}) ζ ( s ) = 1 + 2 − s + O ( 3 − s ) ζ ( s ) \zeta(s) ζ ( s ) r = 1 , 2 r = 1,2 r = 1 , 2 ( E [ log ( X ) ] ) 2 = O ( 4 − s ) (\mathbb{E}[\log(X)])^2=O(4^{-s}) ( E [ log ( X )] ) 2 = O ( 4 − s )
Lemma B.2. Let X s ∼ Zeta ( s ) X_s \sim\operatorname{Zeta}(s) X s ∼ Zeta ( s ) Y s : = log ( X s ) Y_s := \log(X_s) Y s := log ( X s ) μ s : = E [ Y s ] \mu_s := \mathbb{E}[Y_s] μ s := E [ Y s ] σ s 2 : = Var ( Y s ) \sigma_s^2 := \operatorname{Var}(Y_s) σ s 2 := Var ( Y s ) C < ∞ C < \infty C < ∞ s s s
E [ ∣ Y s − μ s σ s ∣ 3 ] ≤ C 2 s 2 . \mathbb{E}\left[\left|\frac{Y_s-\mu_s}{\sigma_s}\right|^3\right]\le C2^{\frac{s}{2}}. E [ σ s Y s − μ s 3 ] ≤ C 2 2 s . Consequently, if X n , 1 , … , X n , n ∼ i i d Zeta ( s n ) X_{n,1},\ldots,X_{n,n}\mathrel{\overset{\mathrm{iid}}{\sim}}\operatorname{Zeta}(s_n) X n , 1 , … , X n , n ∼ iid Zeta ( s n ) s n → ∞ s_n\to\infty s n → ∞ n 2 − s n → ∞ n2^{-s_n}\to\infty n 2 − s n → ∞ Y n , i : = log ( X n , i ) Y_{n,i}:=\log(X_{n,i}) Y n , i := log ( X n , i ) Y ‾ n : = n − 1 ∑ i = 1 n Y n , i \overline{Y}_n:=n^{-1}\sum_{i=1}^nY_{n,i} Y n := n − 1 ∑ i = 1 n Y n , i
∑ i = 1 n E [ ∣ Y n , i − μ s n ∣ 3 ] ( ∑ i = 1 n Var ( Y n , i ) ) 3 / 2 ⟶ 0 \frac{\sum_{i=1}^n\mathbb{E}\left[\lvert Y_{n,i}-\mu_{s_n}\rvert^3\right]}{\left(\sum_{i=1}^n\operatorname{Var}(Y_{n,i})\right)^{3/2}}\longrightarrow0 ( ∑ i = 1 n Var ( Y n , i ) ) 3/2 ∑ i = 1 n E [ ∣ Y n , i − μ s n ∣ 3 ] ⟶ 0 holds, and so
n ( Y ‾ n − μ s n ) σ s n ⟶ d N ( 0 , 1 ) . \frac{\sqrt{n}(\overline{Y}_n-\mu_{s_n})}{\sigma_{s_n}}\mathrel{\overset{d}{\longrightarrow}}\mathcal{N}(0,1). σ s n n ( Y n − μ s n ) ⟶ d N ( 0 , 1 ) . Proof. By Lemma B.1, E [ Y s 3 ] = O ( 2 − s ) \mathbb{E}[Y_s^3]=O(2^{-s}) E [ Y s 3 ] = O ( 2 − s ) μ s = O ( 2 − s ) \mu_s=O(2^{-s}) μ s = O ( 2 − s ) σ s 2 = log ( 2 ) 2 2 − s + O ( 3 − s ) \sigma_s^2=\log(2)^2 2^{-s}+O(3^{-s}) σ s 2 = log ( 2 ) 2 2 − s + O ( 3 − s ) Y s ≥ 0 Y_s\geq0 Y s ≥ 0
∣ Y s − μ s ∣ 3 ≤ 4 ( Y s 3 + μ s 3 ) , \lvert Y_s-\mu_s\rvert^3\leq4(Y_s^3+\mu_s^3), ∣ Y s − μ s ∣ 3 ≤ 4 ( Y s 3 + μ s 3 ) , and therefore E [ ∣ Y s − μ s ∣ 3 ] = O ( 2 − s ) \mathbb{E}[\lvert Y_s-\mu_s\rvert^3]=O(2^{-s}) E [∣ Y s − μ s ∣ 3 ] = O ( 2 − s ) σ s 2 \sigma_s^2 σ s 2
E [ ∣ Y s − μ s σ s ∣ 3 ] = O ( 2 s / 2 ) . \mathbb{E}\left[\left\lvert\frac{Y_s-\mu_s}{\sigma_s}\right\rvert^3\right]=O\left(2^{s/2}\right). E [ σ s Y s − μ s 3 ] = O ( 2 s /2 ) . Within each row, the observations are identically distributed. The Lyapunov ratio is therefore
n E [ ∣ Y n , 1 − μ s n ∣ 3 ] ( n σ s n 2 ) 3 / 2 = 1 n E [ ∣ Y s n − μ s n σ s n ∣ 3 ] = O ( 2 s n n ) ⟶ 0 , \frac{n\mathbb{E}[\lvert Y_{n,1}-\mu_{s_n}\rvert^3]}{(n\sigma_{s_n}^2)^{3/2}}=\frac{1}{\sqrt{n}}\mathbb{E}\left[\left\lvert\frac{Y_{s_n}-\mu_{s_n}}{\sigma_{s_n}}\right\rvert^3\right]=O\left(\sqrt{\frac{2^{s_n}}{n}}\right)\longrightarrow0, ( n σ s n 2 ) 3/2 n E [∣ Y n , 1 − μ s n ∣ 3 ] = n 1 E [ σ s n Y s n − μ s n 3 ] = O ( n 2 s n ) ⟶ 0 , as required. The Lyapunov condition implies the Lindeberg condition, so the Lindeberg–Feller theorem [38 Theorem 2.27] gives the stated central limit theorem.
Root Enclosure and Approximation on the Original Scale
Fix s n → ∞ s_n\to\infty s n → ∞ u n : = 2 − s n u_n:=2^{-s_n} u n := 2 − s n p : = log 2 ( 3 ) p:=\log_2(3) p := log 2 ( 3 ) X n , 1 , … , X n , n ∼ i i d Zeta ( s n ) X_{n,1},\ldots,X_{n,n}\mathrel{\overset{\mathrm{iid}}{\sim}}\operatorname{Zeta}(s_n) X n , 1 , … , X n , n ∼ iid Zeta ( s n ) n u n → ∞ nu_n\to\infty n u n → ∞ P n : = P s n ⊗ n \mathbb{P}_n:=\mathbb{P}_{s_n}^{\otimes n} P n := P s n ⊗ n
m ( u ) : = ∑ k = 2 ∞ Λ ( k ) u log 2 ( k ) = E − log 2 ( u ) [ log ( X ) ] and v ( u ) : = Var − log 2 ( u ) ( log ( X ) ) , m(u):=\sum_{k=2}^{\infty}\Lambda(k)u^{\log_2(k)}=\mathbb{E}_{-\log_2(u)}[\log(X)]\quad\text{and}\quad v(u):=\operatorname{Var}_{-\log_2(u)}(\log(X)), m ( u ) := k = 2 ∑ ∞ Λ ( k ) u l o g 2 ( k ) = E − l o g 2 ( u ) [ log ( X )] and v ( u ) := Var − l o g 2 ( u ) ( log ( X )) , where Λ \Lambda Λ m ( u ) m(u) m ( u ) v ( u ) v(u) v ( u ) m ′ ( u ) m'(u) m ′ ( u ) s ( u ) : = − log 2 ( u ) s(u):=-\log_2(u) s ( u ) := − log 2 ( u ) m m m
m ′ ( u ) = log ( 2 ) + 1 u log ( 2 ) ∑ k = 3 ∞ Λ ( k ) log ( k ) k s ( u ) . m'(u)=\log(2)+\frac{1}{u\log(2)}\sum_{k=3}^{\infty}\frac{\Lambda(k)\log(k)}{k^{s(u)}}. m ′ ( u ) = log ( 2 ) + u log ( 2 ) 1 k = 3 ∑ ∞ k s ( u ) Λ ( k ) log ( k ) . Since s ( u ) = − log 2 ( u ) s(u)=-\log_2(u) s ( u ) = − log 2 ( u ) p = log 2 ( 3 ) p=\log_2(3) p = log 2 ( 3 ) 3 − s ( u ) = u p 3^{-s(u)}=u^p 3 − s ( u ) = u p 0 ≤ Λ ( k ) ≤ log ( k ) 0\leq\Lambda(k)\leq\log(k) 0 ≤ Λ ( k ) ≤ log ( k )
0 ≤ ∑ k = 3 ∞ Λ ( k ) log ( k ) k s ( u ) ≤ ∑ k = 3 ∞ ( log ( k ) ) 2 k s ( u ) = O ( 3 − s ( u ) ) = O ( u p ) . 0\leq\sum_{k=3}^{\infty}\frac{\Lambda(k)\log(k)}{k^{s(u)}}\leq\sum_{k=3}^{\infty}\frac{(\log(k))^2}{k^{s(u)}}=O\left(3^{-s(u)}\right)=O\left(u^p\right). 0 ≤ k = 3 ∑ ∞ k s ( u ) Λ ( k ) log ( k ) ≤ k = 3 ∑ ∞ k s ( u ) ( log ( k ) ) 2 = O ( 3 − s ( u ) ) = O ( u p ) . The prefactor 1 / ( u log ( 2 ) ) 1/(u\log(2)) 1/ ( u log ( 2 )) m ′ ( u ) m'(u) m ′ ( u ) O ( u p − 1 ) O(u^{p-1}) O ( u p − 1 ) u ↘ 0 u\searrow0 u ↘ 0
m ( u ) = ( log ( 2 ) ) u + O ( u p ) and m ′ ( u ) = log ( 2 ) + O ( u p − 1 ) and v ( u ) = ( log ( 2 ) ) 2 u + O ( u p ) . (96) m(u)=(\log(2))u+O(u^p)\quad\text{and}\quad m'(u)=\log(2)+O(u^{p-1})\quad\text{and}\quad v(u)=(\log(2))^2u+O(u^p). \tag*{(96)} m ( u ) = ( log ( 2 )) u + O ( u p ) and m ′ ( u ) = log ( 2 ) + O ( u p − 1 ) and v ( u ) = ( log ( 2 ) ) 2 u + O ( u p ) . (96) Because log ( G n ) = n − 1 ∑ i log ( X n , i ) \log(G_n)=n^{-1}\sum_i\log(X_{n,i}) log ( G n ) = n − 1 ∑ i log ( X n , i )
Var s n ( log ( G n ) ) = v ( u n ) n = O ( u n n ) . \operatorname{Var}_{s_n}(\log(G_n))=\frac{v(u_n)}{n}=O\left(\frac{u_n}{n}\right). Var s n ( log ( G n )) = n v ( u n ) = O ( n u n ) . Chebyshev’s inequality therefore gives
log ( G n ) = m ( u n ) + O P n ( u n n ) . (97) \log(G_n)=m(u_n)+O_{\mathbb{P}_n}\left(\sqrt{\frac{u_n}{n}}\right). \tag*{(97)} log ( G n ) = m ( u n ) + O P n ( n u n ) . (97) Since p > 1 p>1 p > 1 n u n → ∞ nu_n\to\infty n u n → ∞ u n p u_n^p u n p u n / n \sqrt{u_n/n} u n / n o ( u n ) o(u_n) o ( u n )
log ( G n ) = log ( 2 ) u n + o P n ( u n ) and u ~ n = u n + o P n ( u n ) . (98) \log(G_n)=\log(2)u_n+o_{\mathbb{P}_n}(u_n)\quad\text{and}\quad\widetilde{u}_n=u_n+o_{\mathbb{P}_n}(u_n). \tag*{(98)} log ( G n ) = log ( 2 ) u n + o P n ( u n ) and u n = u n + o P n ( u n ) . (98) The coefficients in the series for m m m k = 2 k=2 k = 2 log ( 2 ) u \log(2)u log ( 2 ) u m m m G n = 1 G_n=1 G n = 1
P n ( G n = 1 ) = ζ ( s n ) − n = exp ( − n u n ( 1 + o ( 1 ) ) ) ⟶ 0. \mathbb{P}_n(G_n=1)=\zeta(s_n)^{-n}=\exp\left(-nu_n(1+o(1))\right)\longrightarrow0. P n ( G n = 1 ) = ζ ( s n ) − n = exp ( − n u n ( 1 + o ( 1 )) ) ⟶ 0. On the event ∣ u ~ n / u n − 1 ∣ ≤ 1 / 4 |\widetilde{u}_n/u_n-1|\le1/4 ∣ u n / u n − 1∣ ≤ 1/4
D n ∗ : = { u : ∣ u − u ~ n ∣ ≤ u n 4 } . D_n^*:=\left\{u:|u-\widetilde{u}_n|\le\frac{u_n}{4}\right\}. D n ∗ := { u : ∣ u − u n ∣ ≤ 4 u n } . This event has probability tending to one and D n ∗ ⊂ [ u n / 2 , 3 u n / 2 ] ⊂ U D_n^*\subset[u_n/2,3u_n/2]\subset U D n ∗ ⊂ [ u n /2 , 3 u n /2 ] ⊂ U n n n R n : = u n − 1 R_n:=u_n^{-1} R n := u n − 1 J n : = n log ( 2 ) J_n:=n\log(2) J n := n log ( 2 ) r n : = 1 / 4 r_n:=1/4 r n := 1/4 κ n = 0 \kappa_n=0 κ n = 0
η n : = sup u ∈ D n ∗ ∣ u n − 1 ( n log ( 2 ) ) − 1 [ Q n ( u ) − Q ~ n ( u ) ] ∣ = O ( u n p − 1 ) = o ( 1 ) . \eta_n:=\sup_{u\in D_n^*}\left|u_n^{-1}(n\log(2))^{-1}[Q_n(u)-\widetilde{Q}_n(u)]\right|=O(u_n^{p-1})=o(1). η n := u ∈ D n ∗ sup u n − 1 ( n log ( 2 ) ) − 1 [ Q n ( u ) − Q n ( u )] = O ( u n p − 1 ) = o ( 1 ) . On G n > 1 G_n>1 G n > 1 m m m
∣ u ~ n − u ^ n ∣ = O P n ( u n p ) and u ^ n u n ⟶ p 1. (99) |\widetilde{u}_n-\widehat{u}_n|=O_{\mathbb{P}_n}(u_n^p)\quad\text{and}\quad\frac{\widehat{u}_n}{u_n}\mathrel{\mathop{\longrightarrow}^{p}}1. \tag*{(99)} ∣ u n − u n ∣ = O P n ( u n p ) and u n u n ⟶ p 1. (99) Consequently, after applying the completion convention, the deterministic interval K n : = [ u n / 2 , 2 u n ] K_n:=[u_n/2,2u_n] K n := [ u n /2 , 2 u n ] a n : = n log ( 2 ) a_n:=n\log(2) a n := n log ( 2 )
Returning to the original scale, the same calculation on K n K_n K n Δ K , n = O ( n u n p ) \Delta_{K,n}=O(nu_n^p) Δ K , n = O ( n u n p ) u ∈ K n u\in K_n u ∈ K n
∣ ( ϕ − 1 ) ′ ( u ) ∣ = 1 u log ( 2 ) ≤ 2 u n log ( 2 ) . |(\phi^{-1})'(u)|=\frac{1}{u\log(2)}\le\frac{2}{u_n\log(2)}. ∣ ( ϕ − 1 ) ′ ( u ) ∣ = u log ( 2 ) 1 ≤ u n log ( 2 ) 2 . Thus ϕ − 1 ( u ) = − log 2 ( u ) \phi^{-1}(u)=-\log_2(u) ϕ − 1 ( u ) = − log 2 ( u ) b n : = 2 / ( u n log ( 2 ) ) b_n:=2/(u_n\log(2)) b n := 2/ ( u n log ( 2 )) K n K_n K n
b n Δ K , n a n = O ( u n p − 1 ) = o ( 1 ) . \frac{b_n\Delta_{K,n}}{a_n}=O(u_n^{p-1})=o(1). a n b n Δ K , n = O ( u n p − 1 ) = o ( 1 ) . Therefore, the quantitative bound in the proof of Corollary 3 yields
∣ s ~ n − s ^ n ∣ = O P n ( u n p − 1 ) = O P n ( 2 − s n ( p − 1 ) ) = o P n ( 1 ) . (100) |\widetilde{s}_n-\widehat{s}_n|=O_{\mathbb{P}_n}(u_n^{p-1})=O_{\mathbb{P}_n}\left(2^{-s_n(p-1)}\right)=o_{\mathbb{P}_n}(1). \tag*{(100)} ∣ s n − s n ∣ = O P n ( u n p − 1 ) = O P n ( 2 − s n ( p − 1 ) ) = o P n ( 1 ) . (100) Limiting Distributions
Suppose that n u n → ∞ nu_n\to\infty n u n → ∞ Lemma B.2 , the triangular array Y n , i = log ( X n , i ) Y_{n,i}=\log(X_{n,i}) Y n , i = log ( X n , i )
n ( log ( G n ) − m ( u n ) ) v ( u n ) ⟶ d N ( 0 , 1 ) . \frac{\sqrt{n}(\log(G_n)-m(u_n))}{\sqrt{v(u_n)}}\mathrel{\mathop{\longrightarrow}^{d}}\mathcal{N}(0,1). v ( u n ) n ( log ( G n ) − m ( u n )) ⟶ d N ( 0 , 1 ) . On the event G n > 1 G_n > 1 G n > 1 m ( u ~ n ) = log ( G n ) m(\tilde{u}_n) = \log(G_n) m ( u ~ n ) = log ( G n ) u n ∗ u_n^* u n ∗ u n u_n u n u ~ n \tilde{u}_n u ~ n
n u n ( u ~ n − u n ) = n ( log ( G n ) − m ( u n ) ) v ( u n ) v ( u n ) / u n m ′ ( u n ∗ ) . \sqrt{\frac{n}{u_n}}(\tilde{u}_n-u_n)=\frac{\sqrt{n}(\log(G_n)-m(u_n))}{\sqrt{v(u_n)}}\frac{\sqrt{v(u_n)/u_n}}{m'(u_n^*)}. u n n ( u ~ n − u n ) = v ( u n ) n ( log ( G n ) − m ( u n )) m ′ ( u n ∗ ) v ( u n ) / u n . The localization already proved gives u n ∗ / u n → p 1 u_n^*/u_n \xrightarrow{p} 1 u n ∗ / u n p 1 m ′ ( u n ∗ ) → p log ( 2 ) m'(u_n^*) \xrightarrow{p} \log(2) m ′ ( u n ∗ ) p log ( 2 ) v ( u n ) / u n → log ( 2 ) 2 v(u_n)/u_n \to\log(2)^2 v ( u n ) / u n → log ( 2 ) 2
n u n ( u ~ n − u n ) → d N ( 0 , 1 ) . \sqrt{\frac{n}{u_n}}(\tilde{u}_n-u_n)\xrightarrow{d}\mathcal{N}(0,1). u n n ( u ~ n − u n ) d N ( 0 , 1 ) . To make the discrepancy between the roots negligible under the normalization n / u n \sqrt{n/u_n} n / u n
n u n p − 1 2 ⟶ 0. (101) \sqrt{n}u_n^{p-\frac{1}{2}}\longrightarrow0. \tag*{(101)} n u n p − 2 1 ⟶ 0. (101) Then (B.7) gives directly
n u n ( u ~ n − u ^ n ) = O P n ( n u n p − 1 2 ) = o P n ( 1 ) . (102) \sqrt{\frac{n}{u_n}}(\tilde{u}_n-\hat{u}_n)=O_{\mathbb{P}_n}\left(\sqrt{n}u_n^{p-\frac{1}{2}}\right)=o_{\mathbb{P}_n}(1). \tag*{(102)} u n n ( u ~ n − u ^ n ) = O P n ( n u n p − 2 1 ) = o P n ( 1 ) . (102) Combining this result with the limit for u ~ n \tilde{u}_n u ~ n
n u n ( u ^ n − u n ) → d N ( 0 , 1 ) . \sqrt{\frac{n}{u_n}}(\hat{u}_n-u_n)\xrightarrow{d}\mathcal{N}(0,1). u n n ( u ^ n − u n ) d N ( 0 , 1 ) . It remains to translate these limits back to the original parameter scale. Let g ( u ) : = − log 2 ( u ) g(u):=-\log_2(u) g ( u ) := − log 2 ( u )
g ′ ( u ) = − 1 u log ( 2 ) and g ′ ′ ( u ) = 1 u 2 log ( 2 ) . g'(u)=-\frac{1}{u\log(2)} \qquad\text{and}\qquad g''(u)=\frac{1}{u^2\log(2)}. g ′ ( u ) = − u log ( 2 ) 1 and g ′′ ( u ) = u 2 log ( 2 ) 1 . For w n ∈ { u ^ n , u ~ n } w_n\in\{\hat{u}_n,\tilde{u}_n\} w n ∈ { u ^ n , u ~ n }
w n − u n = O P n ( u n n ) . w_n-u_n=O_{\mathbb{P}_n}\left(\sqrt{\frac{u_n}{n}}\right). w n − u n = O P n ( n u n ) . Both u n u_n u n w n w_n w n K n = [ u n / 2 , 2 u n ] K_n=[u_n/2,2u_n] K n = [ u n /2 , 2 u n ] ξ n \xi_n ξ n u n u_n u n w n w_n w n
g ( w n ) − g ( u n ) = − w n − u n u n log ( 2 ) + ( w n − u n ) 2 2 ξ n 2 log ( 2 ) . (103) g(w_n)-g(u_n)=-\frac{w_n-u_n}{u_n\log(2)}+\frac{(w_n-u_n)^2}{2\xi_n^2\log(2)}. \tag*{(103)} g ( w n ) − g ( u n ) = − u n log ( 2 ) w n − u n + 2 ξ n 2 log ( 2 ) ( w n − u n ) 2 . (103) Since ξ n ∈ K n \xi_n\in K_n ξ n ∈ K n O P n ( 1 / ( n u n ) ) O_{\mathbb{P}_n}(1/(nu_n)) O P n ( 1/ ( n u n ))
n u n ( g ( w n ) − g ( u n ) ) = − 1 log ( 2 ) n u n ( w n − u n ) + O P n ( 1 n u n ) . \sqrt{nu_n}\bigl(g(w_n)-g(u_n)\bigr)=-\frac{1}{\log(2)}\sqrt{\frac{n}{u_n}}(w_n-u_n)+O_{\mathbb{P}_n}\left(\frac{1}{\sqrt{nu_n}}\right). n u n ( g ( w n ) − g ( u n ) ) = − log ( 2 ) 1 u n n ( w n − u n ) + O P n ( n u n 1 ) . Because n u n → ∞ nu_n\to\infty n u n → ∞ o P n ( 1 ) o_{\mathbb{P}_n}(1) o P n ( 1 ) w n = u ^ n w_n=\hat{u}_n w n = u ^ n
n u n ( s ^ n − s n ) → d N ( 0 , 1 log ( 2 ) 2 ) , \sqrt{nu_n}(\hat{s}_n-s_n)\xrightarrow{d}\mathcal{N}\left(0,\frac{1}{\log(2)^2}\right), n u n ( s ^ n − s n ) d N ( 0 , log ( 2 ) 2 1 ) , while taking w n = u ~ n w_n=\tilde{u}_n w n = u ~ n
n u n ( s ~ n − s n ) = n 2 s n ( s ~ n − s n ) → d N ( 0 , 1 log ( 2 ) 2 ) . \sqrt{nu_n}(\tilde{s}_n-s_n)=\sqrt{\frac{n}{2s_n}}(\tilde{s}_n-s_n)\xrightarrow{d}\mathcal{N}\left(0,\frac{1}{\log(2)^2}\right). n u n ( s ~ n − s n ) = 2 s n n ( s ~ n − s n ) d N ( 0 , log ( 2 ) 2 1 ) . Simulation Study
We used the same sample size grid and number of replications as in § 3.4 . We set
u n : = n − γ and s n : = − log 2 ( u n ) = γ log 2 ( n ) , u_n := n^{-\gamma} \quad\text{and} \quad s_n := -\log_2(u_n) = \gamma\log_2(n), u n := n − γ and s n := − log 2 ( u n ) = γ log 2 ( n ) , where
γ ∈ { 0.4 , 0.5 , 0.7 } and n ∈ { 250 , 500 , 1000 , 2500 , 5000 , 10000 , 25000 } . \gamma\in\{0.4, 0.5, 0.7\} \quad\text{and} \quad n \in\{250, 500, 1000, 2500, 5000, 10000, 25000\}. γ ∈ { 0.4 , 0.5 , 0.7 } and n ∈ { 250 , 500 , 1000 , 2500 , 5000 , 10000 , 25000 } . For each design point, we generated 2000 samples from the Zeta(s n s_n s n u n = n − γ u_n=n^{-\gamma} u n = n − γ p = log 2 ( 3 ) p=\log_2(3) p = log 2 ( 3 ) n u n p − 1 / 2 → 0 \sqrt{n}u_n^{p-1/2}\to0 n u n p − 1/2 → 0
n u n p − 1 / 2 = n 1 / 2 − γ ( p − 1 / 2 ) ⟶ 0 ⟺ γ > 1 2 log 2 ( 3 ) − 1 ≈ 0.461. \sqrt{n}u_n^{p-1/2}=n^{1/2-\gamma(p-1/2)}\longrightarrow0 \quad\Longleftrightarrow\quad\gamma>\frac{1}{2\log_2(3)-1}\approx0.461. n u n p − 1/2 = n 1/2 − γ ( p − 1/2 ) ⟶ 0 ⟺ γ > 2 log 2 ( 3 ) − 1 1 ≈ 0.461. Thus, the grid straddled the sufficient threshold for the approximate estimator to share the first-order limiting distribution of the exact MLE. All three choices also satisfied n u n → ∞ nu_n\to\infty n u n → ∞
For each sample with G n > 1 G_n>1 G n > 1 s ~ n \tilde{s}_n s ~ n 1 / ( log ( 2 ) n u n ) 1/(\log(2)\sqrt{nu_n}) 1/ ( log ( 2 ) n u n )
Figure 4 shows the transition predicted by the rate condition. When γ = 0.4 \gamma=0.4 γ = 0.4 γ = 0.5 \gamma=0.5 γ = 0.5 n = 25000 n=25000 n = 25000 s s s γ = 0.5 \gamma=0.5 γ = 0.5 γ = 0.7 \gamma=0.7 γ = 0.7
Figure 4. Median absolute difference between the approximate and exact MLEs on the original parameter scale, divided by the asymptotic standard error of the exact MLE. The horizontal dashed line marks one asymptotic standard error. Each point is based on 2000 replications.
Figure 5 leads to the same conclusion. At n = 25000 n=25000 n = 25000 γ = 0.4 \gamma=0.4 γ = 0.4 γ = 0.5 \gamma=0.5 γ = 0.5 γ = 0.7 \gamma=0.7 γ = 0.7 γ = 0.7 \gamma=0.7 γ = 0.7 γ = 0.5 \gamma= 0.5 γ = 0.5
Figure 5. RMSE of the approximate estimator relative to that of the exact MLE on the original parameter scale. The dashed horizontal line marks equal RMSE. Each point is based on 2000 replications with a finite MLE.
The distributional diagnostics in Figure 6 use ( n , γ ) = ( 25000 , 0.7 ) (n, \gamma) = (25000, 0.7) ( n , γ ) = ( 25000 , 0.7 ) n u n = n 0.3 n u_n = n^{0.3} n u n = n 0.3
Figure 6. Distributional and boundary diagnostics. Panel (a) compares the exact and approximate standardized errors on the original parameter scale with standard normal quantiles at n = 25000 n = 25000 n = 25000 γ = 0.7 \gamma= 0.7 γ = 0.7 ζ ( s n ) − n \zeta(s_n)^{-n} ζ ( s n ) − n
When G n = 1 G_n = 1 G n = 1 γ = 0.7 \gamma= 0.7 γ = 0.7 n ≤ 1000 n \le1000 n ≤ 1000 ζ ( s n ) − n \zeta(s_n)^{-n} ζ ( s n ) − n Figure 6 . We excluded these samples only from summaries requiring a finite MLE. Among the remaining samples, the approximate estimator always belongs to the parameter space. The numerical score inversion is also stable: its largest absolute residual over all 42000 samples is less than 3 × 10 − 12 3 \times10^{-12} 3 × 1 0 − 12
von Mises Distribution
Preliminary Results
Bessel Expansions and Moment Identities
Lemma C.1. As κ → ∞ \kappa\to\infty κ → ∞
A ( κ ) : = I 1 ( κ ) I 0 ( κ ) = 1 − 1 2 κ − 1 8 κ 2 − 1 8 κ 3 + O ( 1 κ 4 ) A(\kappa) := \frac{I_1(\kappa)}{I_0(\kappa)} = 1 - \frac{1}{2\kappa} - \frac{1}{8\kappa^2} - \frac{1}{8\kappa^3} + O\left(\frac{1}{\kappa^4}\right) A ( κ ) := I 0 ( κ ) I 1 ( κ ) = 1 − 2 κ 1 − 8 κ 2 1 − 8 κ 3 1 + O ( κ 4 1 ) and
I 2 ( κ ) I 0 ( κ ) = 1 − 2 κ + 1 κ 2 + 1 4 κ 3 + O ( 1 κ 4 ) . \frac{I_2(\kappa)}{I_0(\kappa)} = 1 - \frac{2}{\kappa} + \frac{1}{\kappa^2} + \frac{1}{4\kappa^3} + O\left(\frac{1}{\kappa^4}\right). I 0 ( κ ) I 2 ( κ ) = 1 − κ 2 + κ 2 1 + 4 κ 3 1 + O ( κ 4 1 ) . Moreover,
A ′ ( κ ) = 1 2 κ 2 + 1 4 κ 3 + O ( 1 κ 4 ) and A ′ ′ ( κ ) = O ( 1 κ 3 ) . A'(\kappa) = \frac{1}{2\kappa^2} + \frac{1}{4\kappa^3} + O\left(\frac{1}{\kappa^4}\right) \quad\text{and} \quad A''(\kappa) = O\left(\frac{1}{\kappa^3}\right). A ′ ( κ ) = 2 κ 2 1 + 4 κ 3 1 + O ( κ 4 1 ) and A ′′ ( κ ) = O ( κ 3 1 ) . Proof. For fixed ν \nu ν
I ν ( κ ) = e κ 2 π κ [ 1 − m ν − 1 8 κ + ( m ν − 1 ) ( m ν − 9 ) 2 ! ( 8 κ ) 2 − ( m ν − 1 ) ( m ν − 9 ) ( m ν − 25 ) 3 ! ( 8 κ ) 3 + O ( 1 κ 4 ) ] , I_{\nu}(\kappa) = \frac{e^{\kappa}}{\sqrt{2\pi\kappa}}\left[1-\frac{m_{\nu}-1}{8\kappa}+\frac{(m_{\nu}-1)(m_{\nu}-9)}{2!(8\kappa)^2}-\frac{(m_{\nu}-1)(m_{\nu}-9)(m_{\nu}-25)}{3!(8\kappa)^3}+O\left(\frac{1}{\kappa^4}\right)\right], I ν ( κ ) = 2 π κ e κ [ 1 − 8 κ m ν − 1 + 2 ! ( 8 κ ) 2 ( m ν − 1 ) ( m ν − 9 ) − 3 ! ( 8 κ ) 3 ( m ν − 1 ) ( m ν − 9 ) ( m ν − 25 ) + O ( κ 4 1 ) ] , where m ν : = 4 ν 2 m_{\nu}:=4\nu^2 m ν := 4 ν 2 [2 Formula 9.7.1] . For ν = 0 , 1 , 2 \nu=0,1,2 ν = 0 , 1 , 2
1 + 1 8 κ + 9 128 κ 2 + 75 1024 κ 3 + O ( 1 κ 4 ) , 1+\frac{1}{8\kappa}+\frac{9}{128\kappa^2}+\frac{75}{1024\kappa^3}+O\left(\frac{1}{\kappa^4}\right), 1 + 8 κ 1 + 128 κ 2 9 + 1024 κ 3 75 + O ( κ 4 1 ) , 1 − 3 8 κ − 15 128 κ 2 − 105 1024 κ 3 + O ( 1 κ 4 ) , 1-\frac{3}{8\kappa}-\frac{15}{128\kappa^2}-\frac{105}{1024\kappa^3}+O\left(\frac{1}{\kappa^4}\right), 1 − 8 κ 3 − 128 κ 2 15 − 1024 κ 3 105 + O ( κ 4 1 ) , 1 − 15 8 κ + 105 128 κ 2 + 315 1024 κ 3 + O ( 1 κ 4 ) . 1-\frac{15}{8\kappa}+\frac{105}{128\kappa^2}+\frac{315}{1024\kappa^3}+O\left(\frac{1}{\kappa^4}\right). 1 − 8 κ 15 + 128 κ 2 105 + 1024 κ 3 315 + O ( κ 4 1 ) . Dividing the second and third series by the first gives the expansions of A A A I 2 / I 0 I_2/I_0 I 2 / I 0 I 0 ( κ ) − I 2 ( κ ) = 2 I 1 ( κ ) / κ I_0(\kappa)-I_2(\kappa)=2I_1(\kappa)/\kappa I 0 ( κ ) − I 2 ( κ ) = 2 I 1 ( κ ) / κ
Differentiating the Bessel ratio gives
A ′ ( κ ) = 1 + I 2 ( κ ) / I 0 ( κ ) 2 − A ( κ ) 2 . A'(\kappa)=\frac{1+I_2(\kappa)/I_0(\kappa)}{2}-A(\kappa)^2. A ′ ( κ ) = 2 1 + I 2 ( κ ) / I 0 ( κ ) − A ( κ ) 2 . and substituting the two ratio expansions gives the stated formula for A ′ A' A ′ I 2 ( κ ) / I 0 ( κ ) = 1 − 2 A ( κ ) / κ I_2(\kappa)/I_0(\kappa)=1-2A(\kappa)/\kappa I 2 ( κ ) / I 0 ( κ ) = 1 − 2 A ( κ ) / κ
A ′ ( κ ) = 1 − A ( κ ) κ − A ( κ ) 2 . A'(\kappa)=1-\frac{A(\kappa)}{\kappa}-A(\kappa)^2. A ′ ( κ ) = 1 − κ A ( κ ) − A ( κ ) 2 . Differentiating this identity gives
A ′ ′ ( κ ) = − A ′ ( κ ) κ + A ( κ ) κ 2 − 2 A ( κ ) A ′ ( κ ) . A''(\kappa)=-\frac{A'(\kappa)}{\kappa}+\frac{A(\kappa)}{\kappa^2}-2A(\kappa)A'(\kappa). A ′′ ( κ ) = − κ A ′ ( κ ) + κ 2 A ( κ ) − 2 A ( κ ) A ′ ( κ ) . Substitution of the established expansions of A A A A ′ A' A ′
A ′ ( κ ) κ = O ( 1 κ 3 ) \frac{A'(\kappa)}{\kappa}=O\left(\frac{1}{\kappa^3}\right) κ A ′ ( κ ) = O ( κ 3 1 ) and
A ( κ ) κ 2 − 2 A ( κ ) A ′ ( κ ) = O ( 1 κ 3 ) , \frac{A(\kappa)}{\kappa^2}-2A(\kappa)A'(\kappa)=O\left(\frac{1}{\kappa^3}\right), κ 2 A ( κ ) − 2 A ( κ ) A ′ ( κ ) = O ( κ 3 1 ) , which proves A ′ ′ ( κ ) = O ( 1 κ 3 ) A''(\kappa)=O\left(\frac{1}{\kappa^3}\right) A ′′ ( κ ) = O ( κ 3 1 )
Lemma C.2. If W = X − μ ∼ von Mises ( 0 , κ ) W=X-\mu\sim\operatorname{von\ Mises}(0,\kappa) W = X − μ ∼ von Mises ( 0 , κ ) B ( κ ) = I 2 ( κ ) / I 0 ( κ ) B(\kappa)=I_2(\kappa)/I_0(\kappa) B ( κ ) = I 2 ( κ ) / I 0 ( κ )
E [ sin ( W ) ] = 0 and E [ cos ( W ) ] = A ( κ ) and E [ sin ( W ) cos ( W ) ] = 0 , \mathbb{E}[\sin(W)]=0 \quad\text{and}\quad\mathbb{E}[\cos(W)]=A(\kappa) \quad\text{and}\quad\mathbb{E}[\sin(W)\cos(W)]=0, E [ sin ( W )] = 0 and E [ cos ( W )] = A ( κ ) and E [ sin ( W ) cos ( W )] = 0 , while the variances satisfy
Var ( sin ( W ) ) = 1 − B ( κ ) 2 = A ( κ ) κ and Var ( cos ( W ) ) = 1 + B ( κ ) 2 − A ( κ ) 2 = A ′ ( κ ) . \operatorname{Var}(\sin(W))=\frac{1-B(\kappa)}{2}=\frac{A(\kappa)}{\kappa} \quad\text{and}\quad\operatorname{Var}(\cos(W))=\frac{1+B(\kappa)}{2}-A(\kappa)^2=A'(\kappa). Var ( sin ( W )) = 2 1 − B ( κ ) = κ A ( κ ) and Var ( cos ( W )) = 2 1 + B ( κ ) − A ( κ ) 2 = A ′ ( κ ) . Proof. For every integer k ≥ 1 k\geq1 k ≥ 1 I k I_k I k
E [ cos ( k W ) ] = 1 2 π I 0 ( κ ) ∫ − π π exp ( κ cos ( w ) ) cos ( k w ) d w = I k ( κ ) I 0 ( κ ) \mathbb{E}[\cos(kW)]=\frac{1}{2\pi I_0(\kappa)}\int_{-\pi}^{\pi}\exp(\kappa\cos(w))\cos(kw)\,\mathrm{d}w=\frac{I_k(\kappa)}{I_0(\kappa)} E [ cos ( k W )] = 2 π I 0 ( κ ) 1 ∫ − π π exp ( κ cos ( w )) cos ( k w ) d w = I 0 ( κ ) I k ( κ ) and E [ sin ( k W ) ] = 0 \mathbb{E}[\sin(kW)]=0 E [ sin ( k W )] = 0 k = 1 k=1 k = 1 E [ sin ( W ) cos ( W ) ] = 0 \mathbb{E}[\sin(W)\cos(W)]=0 E [ sin ( W ) cos ( W )] = 0
sin 2 ( w ) = 1 − cos ( 2 w ) 2 and cos 2 ( w ) = 1 + cos ( 2 w ) 2 \sin^2(w)=\frac{1-\cos(2w)}{2} \quad\text{and}\quad\cos^2(w)=\frac{1+\cos(2w)}{2} sin 2 ( w ) = 2 1 − cos ( 2 w ) and cos 2 ( w ) = 2 1 + cos ( 2 w ) then yield
Var ( sin ( W ) ) = 1 − B ( κ ) 2 and Var ( cos ( W ) ) = 1 + B ( κ ) 2 − A ( κ ) 2 . \operatorname{Var}(\sin(W))=\frac{1-B(\kappa)}{2} \quad\text{and}\quad\operatorname{Var}(\cos(W))=\frac{1+B(\kappa)}{2}-A(\kappa)^2. Var ( sin ( W )) = 2 1 − B ( κ ) and Var ( cos ( W )) = 2 1 + B ( κ ) − A ( κ ) 2 . The recurrence 1 − B ( κ ) = 2 A ( κ ) / κ 1-B(\kappa)=2A(\kappa)/\kappa 1 − B ( κ ) = 2 A ( κ ) / κ I 1 / I 0 I_1/I_0 I 1 / I 0 A ′ ( κ ) A'(\kappa) A ′ ( κ )
MLE Characterization and Root Preservation
For X 1 , … , X n ∼ i i d von Mises ( μ , κ ) X_1,\ldots,X_n \mathrel{\overset{\mathrm{iid}}{\sim}} \operatorname{von\ Mises}(\mu,\kappa) X 1 , … , X n ∼ iid von Mises ( μ , κ )
ℓ n ( μ , κ ) : = − n log ( I 0 ( κ ) ) + κ ∑ i = 1 n cos ( X i − μ ) , \ell_n(\mu,\kappa):=-n\log(I_0(\kappa))+\kappa\sum_{i=1}^{n}\cos(X_i-\mu), ℓ n ( μ , κ ) := − n log ( I 0 ( κ )) + κ i = 1 ∑ n cos ( X i − μ ) , with score
∂ μ ℓ n = κ ∑ i = 1 n sin ( X i − μ ) \partial_\mu\ell_n=\kappa\sum_{i=1}^{n}\sin(X_i-\mu) ∂ μ ℓ n = κ i = 1 ∑ n sin ( X i − μ ) and
∂ κ ℓ n = ∑ i = 1 n cos ( X i − μ ) − n A ( κ ) \partial_\kappa\ell_n=\sum_{i=1}^{n}\cos(X_i-\mu)-nA(\kappa) ∂ κ ℓ n = i = 1 ∑ n cos ( X i − μ ) − n A ( κ ) and
A ( κ ) : = I 1 ( κ ) I 0 ( κ ) . A(\kappa):=\frac{I_1(\kappa)}{I_0(\kappa)}. A ( κ ) := I 0 ( κ ) I 1 ( κ ) . Let C ‾ n \overline{C}_n C n S ‾ n \overline{S}_n S n R n R_n R n μ ^ n \widehat{\mu}_n μ n § 4.2 . The identity
∑ i = 1 n cos ( X i − μ ) = n R n cos ( μ − μ ^ n ) \sum_{i=1}^{n}\cos(X_i-\mu)=nR_n\cos(\mu-\widehat{\mu}_n) i = 1 ∑ n cos ( X i − μ ) = n R n cos ( μ − μ n ) shows that, whenever R n > 0 R_n>0 R n > 0 μ ^ n \widehat{\mu}_n μ n 2 π 2\pi 2 π κ \kappa κ n ( R n − A ( κ ) ) n(R_n-A(\kappa)) n ( R n − A ( κ ))
Since A ( κ ) = ( log ( I 0 ( κ ) ) ) ′ A(\kappa)=(\log(I_0(\kappa)))' A ( κ ) = ( log ( I 0 ( κ )) ) ′
A ′ ( κ ) = Var μ , κ ( cos ( X − μ ) ) > 0. A'(\kappa)=\operatorname{Var}_{\mu,\kappa}(\cos(X-\mu))>0. A ′ ( κ ) = Var μ , κ ( cos ( X − μ )) > 0. Moreover, A ( κ ) ↓ 0 A(\kappa)\downarrow0 A ( κ ) ↓ 0 κ ↓ 0 \kappa\downarrow0 κ ↓ 0 A ( κ ) ↑ 1 A(\kappa)\uparrow1 A ( κ ) ↑ 1 κ → ∞ \kappa\to\infty κ → ∞ A ( κ ) = R n A(\kappa)=R_n A ( κ ) = R n 0 < R n < 1 0<R_n<1 0 < R n < 1 n ≥ 2 n\ge2 n ≥ 2 R n = 1 R_n=1 R n = 1 R n = 0 R_n=0 R n = 0 n ≥ 2 n\ge2 n ≥ 2 0 < R n < 1 0<R_n<1 0 < R n < 1 R n = 1 R_n=1 R n = 1 R n = 0 R_n=0 R n = 0 κ ↓ 0 \kappa\downarrow0 κ ↓ 0 § 2.1 on these exceptional events.
The modified system in § 4.2 preserves this MLE root exactly. To see this, write
q n ( μ , κ ) : = ( μ ( A ( κ ) − 1 ) − μ ^ n ( R n − 1 ) A ( κ ) − R n ) . q_n(\mu,\kappa):=\left(\frac{\mu(A(\kappa)-1)-\widehat{\mu}_n(R_n-1)}{A(\kappa)-R_n}\right). q n ( μ , κ ) := ( A ( κ ) − R n μ ( A ( κ ) − 1 ) − μ n ( R n − 1 ) ) . If q 2 , n = 0 q_{2,n}=0 q 2 , n = 0
q 1 , n = ( R n − 1 ) ( μ − μ ^ n ) . q_{1,n}=(R_n-1)(\mu-\widehat{\mu}_n). q 1 , n = ( R n − 1 ) ( μ − μ n ) . Since R n < 1 R_n<1 R n < 1 ( μ ^ n , κ ^ n ) (\widehat{\mu}_n,\widehat{\kappa}_n) ( μ n , κ n )
The chart is built from the approximation
g ( κ ) : = − 1 2 κ − 1 8 κ 2 . g(\kappa):=-\frac{1}{2\kappa}-\frac{1}{8\kappa^2}. g ( κ ) := − 2 κ 1 − 8 κ 2 1 . The function g g g − ∞ -\infty − ∞ 0 0 0 ( 0 , ∞ ) (0,\infty) ( 0 , ∞ ) M M M 2 π 2\pi 2 π
ϕ ( μ , κ ) : = ( μ g ( κ ) , g ( κ ) ) T \phi(\mu,\kappa):=(\mu g(\kappa),g(\kappa))^{\mathsf T} ϕ ( μ , κ ) := ( μg ( κ ) , g ( κ ) ) T is a bijection from N : = M × ( 0 , ∞ ) N:=M\times(0,\infty) N := M × ( 0 , ∞ )
U : = { ( u 1 , u 2 ) : u 2 < 0 , u 1 u 2 ∈ M } , U:=\{(u_1,u_2):u_2<0,\ \frac{u_1}{u_2}\in M\}, U := {( u 1 , u 2 ) : u 2 < 0 , u 2 u 1 ∈ M } , with inverse
ϕ − 1 ( u 1 , u 2 ) = ( u 1 u 2 , k ( u 2 ) ) T and k ( u 2 ) : = − 1 + 1 − 2 u 2 4 u 2 . \phi^{-1}(u_1,u_2)=\left(\frac{u_1}{u_2},k(u_2)\right)^{\mathrm T}\quad\text{and}\quad k(u_2):=\frac{-1+\sqrt{1-2u_2}}{4u_2}. ϕ − 1 ( u 1 , u 2 ) = ( u 2 u 1 , k ( u 2 ) ) T and k ( u 2 ) := 4 u 2 − 1 + 1 − 2 u 2 . The remaining discrepancy between the exact Bessel ratio and the chart approximation is captured by h ( u 2 ) : = A ( k ( u 2 ) ) − 1 h(u_2):=A(k(u_2))-1 h ( u 2 ) := A ( k ( u 2 )) − 1 u 2 ↗ 0 u_2 \nearrow0 u 2 ↗ 0 Lemma C.1 gives
h ( u 2 ) = u 2 + O ( ∣ u 2 ∣ 3 ) and h ′ ( u 2 ) = 1 + O ( u 2 2 ) . h(u_2)=u_2+O(|u_2|^3)\quad\text{and}\quad h'(u_2)=1+O(u_2^2). h ( u 2 ) = u 2 + O ( ∣ u 2 ∣ 3 ) and h ′ ( u 2 ) = 1 + O ( u 2 2 ) . Because k = g − 1 k=g^{-1} k = g − 1 k k k A A A h h h ( − ∞ , 0 ) (-\infty,0) ( − ∞ , 0 ) ( − 1 , 0 ) (-1,0) ( − 1 , 0 )
Q n ( u ) : = ( ( u 1 u 2 ) h ( u 2 ) − μ ^ n ( R n − 1 ) h ( u 2 ) − ( R n − 1 ) ) and Q ~ n ( u ) : = ( u 1 − μ ^ n ( R n − 1 ) u 2 − ( R n − 1 ) ) . Q_n(u):=\begin{pmatrix}\left(\frac{u_1}{u_2}\right)h(u_2)-\widehat{\mu}_n(R_n-1)\\ h(u_2)-(R_n-1)\end{pmatrix}
\quad\text{and}\quad
\widetilde{Q}_n(u):=\begin{pmatrix}u_1-\widehat{\mu}_n(R_n-1)\\ u_2-(R_n-1)\end{pmatrix}. Q n ( u ) := ( ( u 2 u 1 ) h ( u 2 ) − μ n ( R n − 1 ) h ( u 2 ) − ( R n − 1 ) ) and Q n ( u ) := ( u 1 − μ n ( R n − 1 ) u 2 − ( R n − 1 ) ) . In particular, J Q ~ n = I 2 J_{\widetilde{Q}_n}=I_2 J Q n = I 2
u ~ n : = ( μ ^ n ( R n − 1 ) , R n − 1 ) T . \widetilde{u}_n:=\left(\widehat{\mu}_n(R_n-1),R_n-1\right)^{\mathrm T}. u n := ( μ n ( R n − 1 ) , R n − 1 ) T . Fix κ n → ∞ \kappa_n\to\infty κ n → ∞ μ \mu μ
u 2 , n : = g ( κ n ) and u n : = ( μ u 2 , n , u 2 , n ) T and ∣ u 2 , n ∣ ≍ 1 κ n . u_{2,n}:=g(\kappa_n)\quad\text{and}\quad\boldsymbol{u}_n:=(\mu u_{2,n},u_{2,n})^{\mathrm T}\quad\text{and}\quad|u_{2,n}|\asymp\frac{1}{\kappa_n}. u 2 , n := g ( κ n ) and u n := ( μ u 2 , n , u 2 , n ) T and ∣ u 2 , n ∣ ≍ κ n 1 . All stochastic orders in this subsection are under P n : = P μ , κ n \mathbb{P}_n:=\mathbb{P}_{\mu,\kappa_n} P n := P μ , κ n W n , i : = X n , i − μ W_{n,i}:=X_{n,i}-\mu W n , i := X n , i − μ [ − π , π ) [-\pi,\pi) [ − π , π )
C n : = 1 n ∑ i = 1 n cos ( W n , i ) and S n : = 1 n ∑ i = 1 n sin ( W n , i ) . C_n:=\frac{1}{n}\sum_{i=1}^{n}\cos(W_{n,i})\quad\text{and}\quad S_n:=\frac{1}{n}\sum_{i=1}^{n}\sin(W_{n,i}). C n := n 1 i = 1 ∑ n cos ( W n , i ) and S n := n 1 i = 1 ∑ n sin ( W n , i ) . The moment identities in Lemma C.2, together with the expansions in Lemma C.1 and Chebyshev’s inequality, give
C n = A ( κ n ) + O P ( 1 κ n n ) and S n = O P ( 1 n κ n ) . C_n=A(\kappa_n)+O_{\mathbb P}\left(\frac{1}{\kappa_n\sqrt{n}}\right)\quad\text{and}\quad S_n=O_{\mathbb P}\left(\frac{1}{\sqrt{n\kappa_n}}\right). C n = A ( κ n ) + O P ( κ n n 1 ) and S n = O P ( n κ n 1 ) . Rotating all observations by − μ -\mu − μ
R n 2 = ( 1 n ∑ i = 1 n cos ( X n , i ) ) 2 + ( 1 n ∑ i = 1 n sin ( X n , i ) ) 2 = C n 2 + S n 2 . R_n^2=\left(\frac{1}{n}\sum_{i=1}^{n}\cos(X_{n,i})\right)^2+\left(\frac{1}{n}\sum_{i=1}^{n}\sin(X_{n,i})\right)^2=C_n^2+S_n^2. R n 2 = ( n 1 i = 1 ∑ n cos ( X n , i ) ) 2 + ( n 1 i = 1 ∑ n sin ( X n , i ) ) 2 = C n 2 + S n 2 . On the event C n > 1 / 2 C_n>1/2 C n > 1/2 M M M
R n − C n = S n 2 R n + C n = O P ( 1 n κ n ) and μ ^ n − μ = atan2 ( S n , C n ) = O P ( 1 n κ n ) . R_n-C_n=\frac{S_n^2}{R_n+C_n}=O_{\mathbb P}\left(\frac{1}{n\kappa_n}\right)\quad\text{and}\quad\widehat{\mu}_n-\mu=\operatorname{atan2}(S_n,C_n)=O_{\mathbb P}\left(\frac{1}{\sqrt{n\kappa_n}}\right). R n − C n = R n + C n S n 2 = O P ( n κ n 1 ) and μ n − μ = atan2 ( S n , C n ) = O P ( n κ n 1 ) . Consequently,
R n − A ( κ n ) = O P ( 1 κ n n + 1 n κ n ) . (104) R_n-A(\kappa_n)=O_{\mathbb P}\left(\frac{1}{\kappa_n\sqrt{n}}+\frac{1}{n\kappa_n}\right). \tag*{(104)} R n − A ( κ n ) = O P ( κ n n 1 + n κ n 1 ) . (104) Combining this bound with A ( κ n ) − 1 = u 2 , n + O ( ∣ u 2 , n ∣ 3 ) A(\kappa_n)-1=u_{2,n}+O(|u_{2,n}|^3) A ( κ n ) − 1 = u 2 , n + O ( ∣ u 2 , n ∣ 3 )
u ~ 2 , n = R n − 1 = u 2 , n + o P ( ∣ u 2 , n ∣ ) . \widetilde{u}_{2,n}=R_n-1=u_{2,n}+o_{\mathbb P}(|u_{2,n}|). u 2 , n = R n − 1 = u 2 , n + o P ( ∣ u 2 , n ∣ ) . Moreover,
u ~ 1 , n − μ u 2 , n = ( μ ^ n − μ ) ( R n − 1 ) + μ ( ( R n − 1 ) − u 2 , n ) = o P ( ∣ u 2 , n ∣ ) , \widetilde{u}_{1,n}-\mu u_{2,n}=(\widehat{\mu}_n-\mu)(R_n-1)+\mu((R_n-1)-u_{2,n})=o_{\mathbb P}(|u_{2,n}|), u 1 , n − μ u 2 , n = ( μ n − μ ) ( R n − 1 ) + μ (( R n − 1 ) − u 2 , n ) = o P ( ∣ u 2 , n ∣ ) , so u ~ 1 , n = μ u 2 , n + o P ( ∣ u 2 , n ∣ ) \widetilde{u}_{1,n}=\mu u_{2,n}+o_{\mathrm{P}}(|u_{2,n}|) u 1 , n = μ u 2 , n + o P ( ∣ u 2 , n ∣ ) r 0 > 0 r_0>0 r 0 > 0 B ‾ r 0 ( ( − μ , − 1 ) T ) ⊆ { ( v 1 , v 2 ) : v 2 < 0 , v 1 / v 2 ∈ M } \overline{B}_{r_0}((-\mu,-1)^{\mathsf{T}})\subseteq\{(v_1,v_2):v_2<0,\ v_1/v_2\in M\} B r 0 (( − μ , − 1 ) T ) ⊆ {( v 1 , v 2 ) : v 2 < 0 , v 1 / v 2 ∈ M } ∣ u 2 , n ∣ − 1 u ~ n ∈ B ‾ r 0 / 2 ( ( − μ , − 1 ) T ) |u_{2,n}|^{-1}\widetilde{u}_n\in\overline{B}_{r_0/2}((-\mu,-1)^{\mathsf{T}}) ∣ u 2 , n ∣ − 1 u n ∈ B r 0 /2 (( − μ , − 1 ) T )
D n ∗ : = { u : ∥ ∣ u 2 , n ∣ − 1 ( u − u ~ n ) ∥ ≤ r 0 2 } . D_n^*:=\left\{u:\left\||u_{2,n}|^{-1}(u-\widetilde{u}_n)\right\|\leq\frac{r_0}{2}\right\}. D n ∗ := { u : ∣ u 2 , n ∣ − 1 ( u − u n ) ≤ 2 r 0 } . This event has probability tending to one and D n ∗ ⊂ U D_n^*\subset U D n ∗ ⊂ U R n : = ∣ u 2 , n ∣ − 1 I 2 R_n:=|u_{2,n}|^{-1}I_2 R n := ∣ u 2 , n ∣ − 1 I 2 J n : = I 2 J_n:=I_2 J n := I 2 r n : = r 0 / 2 r_n:=r_0/2 r n := r 0 /2 κ n = 0 \kappa_n=0 κ n = 0
Q n ( u ) − Q ~ n ( u ) = ( h ( u 2 ) − u 2 ) ( u 1 u 2 1 ) . Q_n(u)-\widetilde{Q}_n(u)=(h(u_2)-u_2)\begin{pmatrix}\dfrac{u_1}{u_2}\\1\end{pmatrix}. Q n ( u ) − Q n ( u ) = ( h ( u 2 ) − u 2 ) ( u 2 u 1 1 ) . On D n ∗ D_n^* D n ∗ ∣ u 2 ∣ ≍ ∣ u 2 , n ∣ |u_2|\asymp|u_{2,n}| ∣ u 2 ∣ ≍ ∣ u 2 , n ∣ u 1 / u 2 u_1/u_2 u 1 / u 2 h ( u 2 ) − u 2 = O ( ∣ u 2 ∣ 3 ) h(u_2)-u_2=O(|u_2|^3) h ( u 2 ) − u 2 = O ( ∣ u 2 ∣ 3 )
η n : = sup u ∈ D n ∗ ∥ R n [ Q n ( u ) − Q ~ n ( u ) ] ∥ = O ( ∣ u 2 , n ∣ 2 ) = o ( 1 ) . \eta_n:=\sup_{u\in D_n^*}\left\|R_n[Q_n(u)-\widetilde{Q}_n(u)]\right\|=O(|u_{2,n}|^2)=o(1). η n := u ∈ D n ∗ sup R n [ Q n ( u ) − Q n ( u )] = O ( ∣ u 2 , n ∣ 2 ) = o ( 1 ) . The exact system has a unique root in U U U
∥ u ~ n − u ^ n ∥ = O P ( ∣ u 2 , n ∣ 3 ) = O P ( κ n − 3 ) . (105) \|\widetilde{u}_n-\widehat{u}_n\|=O_{\mathrm{P}}(|u_{2,n}|^3)=O_{\mathrm{P}}(\kappa_n^{-3}). \tag*{(105)} ∥ u n − u n ∥ = O P ( ∣ u 2 , n ∣ 3 ) = O P ( κ n − 3 ) . (105) In particular, ∥ u ^ n − u n ∥ = o P ( ∣ u 2 , n ∣ ) \|\widehat{u}_n-u_n\|=o_{\mathrm{P}}(|u_{2,n}|) ∥ u n − u n ∥ = o P ( ∣ u 2 , n ∣ )
Choose δ > 0 \delta>0 δ > 0 [ μ − δ , μ + δ ] ⊂ M [\mu-\delta,\mu+\delta]\subset M [ μ − δ , μ + δ ] ⊂ M
K n : = { u ∈ U : 3 2 u 2 , n ≤ u 2 ≤ 1 2 u 2 , n , ∣ u 1 u 2 − μ ∣ ≤ δ } . K_n:=\left\{u\in U:\frac{3}{2}u_{2,n}\leq u_2\leq\frac{1}{2}u_{2,n},\ \left|\frac{u_1}{u_2}-\mu\right|\leq\delta\right\}. K n := { u ∈ U : 2 3 u 2 , n ≤ u 2 ≤ 2 1 u 2 , n , u 2 u 1 − μ ≤ δ } . Since u 2 < 0 u_2<0 u 2 < 0
( μ + δ ) u 2 ≤ u 1 ≤ ( μ − δ ) u 2 , (\mu+\delta)u_2\leq u_1\leq(\mu-\delta)u_2, ( μ + δ ) u 2 ≤ u 1 ≤ ( μ − δ ) u 2 , so K n K_n K n E n E_n E n N N N K n K_n K n P n ( E n ) → 1 \mathbb{P}_n(E_n)\to1 P n ( E n ) → 1 sup u ∈ K n ∥ u ∥ → 0 \sup_{u\in K_n}\|u\|\to0 sup u ∈ K n ∥ u ∥ → 0 K n K_n K n
Q n ( u ) − Q ~ n ( u ) = ( h ( u 2 ) − u 2 ) ( u 1 u 2 1 ) , Q_n(u)-\widetilde{Q}_n(u)=(h(u_2)-u_2)\begin{pmatrix}\dfrac{u_1}{u_2}\\1\end{pmatrix}, Q n ( u ) − Q n ( u ) = ( h ( u 2 ) − u 2 ) ( u 2 u 1 1 ) , and therefore
Δ K , n : = sup u ∈ K n ∥ Q n ( u ) − Q ~ n ( u ) ∥ = O ( ∣ u 2 , n ∣ 3 ) = O ( 1 κ n 3 ) . \Delta_{K,n}:=\sup_{u\in K_n}\|Q_n(u)-\widetilde{Q}_n(u)\|=O(|u_{2,n}|^3)=O\left(\frac{1}{\kappa_n^3}\right). Δ K , n := u ∈ K n sup ∥ Q n ( u ) − Q n ( u ) ∥ = O ( ∣ u 2 , n ∣ 3 ) = O ( κ n 3 1 ) . For the inverse chart,
J ϕ − 1 ( u ) = ( 1 u 2 − u 1 u 2 2 0 1 g ′ ( k ( u 2 ) ) ) . J_{\phi^{-1}}(u)=\begin{pmatrix}
\dfrac{1}{u_2} & -\dfrac{u_1}{u_2^2}\\
0 & \dfrac{1}{g'(k(u_2))}
\end{pmatrix}. J ϕ − 1 ( u ) = u 2 1 0 − u 2 2 u 1 g ′ ( k ( u 2 )) 1 . On K n K_n K n ∣ u 2 ∣ ≍ κ n − 1 |u_2|\asymp\kappa_n^{-1} ∣ u 2 ∣ ≍ κ n − 1 ∣ u 1 / u 2 ∣ = O ( 1 ) |u_1/u_2|=O(1) ∣ u 1 / u 2 ∣ = O ( 1 ) g ′ ( k ( u 2 ) ) ≍ κ n − 2 g'(k(u_2))\asymp\kappa_n^{-2} g ′ ( k ( u 2 )) ≍ κ n − 2 ϕ − 1 \phi^{-1} ϕ − 1 b n b_n b n b n b_n b n b n = O ( κ n 2 ) b_n=O(\kappa_n^2) b n = O ( κ n 2 ) b n Δ K , n = O ( 1 κ n ) b_n\Delta_{K,n}=O\left(\frac{1}{\kappa_n}\right) b n Δ K , n = O ( κ n 1 )
μ ~ n = μ ^ n and κ ~ n − κ ^ n = O P ( 1 κ n ) = o P ( 1 ) . \widetilde{\mu}_n=\widehat{\mu}_n\quad\text{and}\quad\widetilde{\kappa}_n-\widehat{\kappa}_n=O_{\mathrm{P}}\left(\frac{1}{\kappa_n}\right)=o_{\mathrm{P}}(1). μ n = μ n and κ n − κ n = O P ( κ n 1 ) = o P ( 1 ) . This approximation on the original parameter scale requires no additional relation between n n n κ n \kappa_n κ n
Limiting Distributions
Lemma C.3. Let X n , 1 , … , X n , n ∼ i i d von Mises ( μ , κ n ) X_{n,1},\ldots,X_{n,n}\mathrel{\overset{\mathrm{iid}}{\sim}}\operatorname{von\ Mises}(\mu,\kappa_n) X n , 1 , … , X n , n ∼ iid von Mises ( μ , κ n ) κ n → ∞ \kappa_n \to\infty κ n → ∞ W n , i : = X n , i − μ W_{n,i}:=X_{n,i}-\mu W n , i := X n , i − μ [ − π , π ) [-\pi,\pi) [ − π , π )
1 n ∑ i = 1 n ( κ n A ( κ n ) sin ( W n , i ) cos ( W n , i ) − A ( κ n ) A ′ ( κ n ) ) → d N ( 0 , I 2 ) . \frac{1}{\sqrt{n}}\sum_{i=1}^{n}\begin{pmatrix}
\sqrt{\frac{\kappa_n}{A(\kappa_n)}}\sin(W_{n,i})\\
\frac{\cos(W_{n,i})-A(\kappa_n)}{\sqrt{A'(\kappa_n)}}
\end{pmatrix}
\xrightarrow{d}\mathcal{N}(0,I_2). n 1 i = 1 ∑ n A ( κ n ) κ n sin ( W n , i ) A ′ ( κ n ) c o s ( W n , i ) − A ( κ n ) d N ( 0 , I 2 ) . Proof. We first establish a uniform moment bound. Let W κ ∼ von Mises ( 0 , κ ) W_\kappa\sim\operatorname{von\ Mises}(0,\kappa) W κ ∼ von Mises ( 0 , κ ) V κ : = κ W κ V_\kappa:=\sqrt{\kappa}W_\kappa V κ := κ W κ
1 − cos ( w ) ≥ 2 w 2 π 2 1-\cos(w)\ge\frac{2w^2}{\pi^2} 1 − cos ( w ) ≥ π 2 2 w 2 holds for ∣ w ∣ ≤ π |w|\le\pi ∣ w ∣ ≤ π e − κ I 0 ( κ ) ≍ κ − 1 / 2 e^{-\kappa}I_0(\kappa)\asymp\kappa^{-1/2} e − κ I 0 ( κ ) ≍ κ − 1/2 ∣ v ∣ < π κ |v|<\pi\sqrt{\kappa} ∣ v ∣ < π κ V κ V_\kappa V κ
f V κ ( v ) = exp ( − κ ( 1 − cos ( v / κ ) ) ) 2 π κ e − κ I 0 ( κ ) , f_{V_\kappa}(v)=\frac{\exp\left(-\kappa\left(1-\cos\left(v/\sqrt{\kappa}\right)\right)\right)}{2\pi\sqrt{\kappa}e^{-\kappa}I_0(\kappa)}, f V κ ( v ) = 2 π κ e − κ I 0 ( κ ) exp ( − κ ( 1 − cos ( v / κ ) ) ) , and it is zero otherwise. It follows that f V κ ( v ) ≤ C exp ( − 2 v 2 / π 2 ) f_{V_\kappa}(v)\le C\exp(-2v^2/\pi^2) f V κ ( v ) ≤ C exp ( − 2 v 2 / π 2 ) κ \kappa κ r > 0 r>0 r > 0
sup κ ≥ κ 0 E [ ∣ V κ ∣ r ] < ∞ . \sup_{\kappa\ge\kappa_0}\mathbb{E}[|V_\kappa|^r]<\infty. κ ≥ κ 0 sup E [ ∣ V κ ∣ r ] < ∞. Define
Z κ = ( κ A ( κ ) sin ( W κ ) cos ( W κ ) − A ( κ ) A ′ ( κ ) ) . Z_\kappa=\begin{pmatrix}
\sqrt{\frac{\kappa}{A(\kappa)}}\sin(W_\kappa)\\
\frac{\cos(W_\kappa)-A(\kappa)}{\sqrt{A'(\kappa)}}
\end{pmatrix}. Z κ = A ( κ ) κ sin ( W κ ) A ′ ( κ ) c o s ( W κ ) − A ( κ ) . By Lemmas C.1 and C.2, E [ Z κ ] = 0 \mathbb{E}[Z_\kappa]=0 E [ Z κ ] = 0 Var ( Z κ ) = I 2 \operatorname{Var}(Z_\kappa)=I_2 Var ( Z κ ) = I 2 A ( κ ) → 1 A(\kappa)\to1 A ( κ ) → 1 A ( κ ) A(\kappa) A ( κ ) κ \kappa κ
κ A ( κ ) ∣ sin ( W κ ) ∣ ≤ C ∣ V κ ∣ . \sqrt{\frac{\kappa}{A(\kappa)}}|\sin(W_\kappa)|\le C|V_\kappa|. A ( κ ) κ ∣ sin ( W κ ) ∣ ≤ C ∣ V κ ∣. For the second component,
κ ∣ cos ( W κ ) − A ( κ ) ∣ ≤ 1 2 V κ 2 + κ ( 1 − A ( κ ) ) . \kappa|\cos(W_\kappa)-A(\kappa)|\le\frac{1}{2}V_\kappa^2+\kappa(1-A(\kappa)). κ ∣ cos ( W κ ) − A ( κ ) ∣ ≤ 2 1 V κ 2 + κ ( 1 − A ( κ )) . Since κ ( 1 − A ( κ ) ) \kappa(1-A(\kappa)) κ ( 1 − A ( κ )) κ A ′ ( κ ) → 1 / 2 \kappa\sqrt{A'(\kappa)}\to1/\sqrt{2} κ A ′ ( κ ) → 1/ 2
sup κ ≥ κ 0 E [ ∥ Z κ ∥ 3 ] < ∞ . (106) \sup_{\kappa\ge\kappa_0}\mathbb{E}\left[\|Z_\kappa\|^3\right]<\infty. \tag*{(106)} κ ≥ κ 0 sup E [ ∥ Z κ ∥ 3 ] < ∞. (106) Let Z κ n , i Z_{\kappa_n,i} Z κ n , i W n , i W_{n,i} W n , i t ∈ R 2 t\in\mathbb{R}^2 t ∈ R 2
∑ i = 1 n E [ ∣ t T Z κ n , i ∣ 3 ] ( ∑ i = 1 n Var ( t T Z κ n , i ) ) 3 / 2 = E [ ∣ t T Z κ n ∣ 3 ] n ∥ t ∥ 3 = O ( n − 1 / 2 ) , \frac{\sum_{i=1}^{n}\mathbb{E}[|t^{\mathsf T}Z_{\kappa_n,i}|^3]}{\left(\sum_{i=1}^{n}\operatorname{Var}(t^{\mathsf T}Z_{\kappa_n,i})\right)^{3/2}}
=\frac{\mathbb{E}[|t^{\mathsf T}Z_{\kappa_n}|^3]}{\sqrt{n}\|t\|^3}
=O\left(n^{-1/2}\right), ( ∑ i = 1 n Var ( t T Z κ n , i ) ) 3/2 ∑ i = 1 n E [ ∣ t T Z κ n , i ∣ 3 ] = n ∥ t ∥ 3 E [ ∣ t T Z κ n ∣ 3 ] = O ( n − 1/2 ) , where the last equality follows from (C.3). The Lyapunov central limit theorem and the Cramér–Wold device complete the proof. ■
For one observation, the score is
s μ , κ ( X ) : = ( κ sin ( X − μ ) , cos ( X − μ ) − A ( κ ) ) T . s_{\mu,\kappa}(X):=\left(\kappa\sin(X-\mu),\ \cos(X-\mu)-A(\kappa)\right)^{\mathsf T}. s μ , κ ( X ) := ( κ sin ( X − μ ) , cos ( X − μ ) − A ( κ ) ) T . Writing W : = X − μ W := X - \mu W := X − μ W W W sin ( W ) \sin(W) sin ( W ) sin ( W ) ( cos ( W ) − A ( κ ) ) \sin(W)(\cos(W) - A(\kappa)) sin ( W ) ( cos ( W ) − A ( κ ))
E [ sin ( W ) ] = 0 and E [ sin ( W ) ( cos ( W ) − A ( κ ) ) ] = 0 , \mathbb{E}[\sin(W)] = 0 \quad\text{and} \quad\mathbb{E}[\sin(W)(\cos(W) - A(\kappa))] = 0, E [ sin ( W )] = 0 and E [ sin ( W ) ( cos ( W ) − A ( κ ))] = 0 , so the off-diagonal covariance vanishes. The variance identities in Lemma C.2 then give
I 1 ( μ , κ ) : = Var ( s μ , κ ( X ) ) = ( κ A ( κ ) 0 0 A ′ ( κ ) ) . I_1(\mu,\kappa) := \operatorname{Var}(s_{\mu,\kappa}(X)) =
\begin{pmatrix}
\kappa A(\kappa) & 0 \\
0 & A'(\kappa)
\end{pmatrix}. I 1 ( μ , κ ) := Var ( s μ , κ ( X )) = ( κ A ( κ ) 0 0 A ′ ( κ ) ) . Here the information function for the concentration coordinate can also be written through the exact identity
A ′ ( κ ) = 1 + I 2 ( κ ) / I 0 ( κ ) 2 − A ( κ ) 2 = Var ( cos ( X − μ ) ) . A'(\kappa) = \frac{1 + I_2(\kappa)/I_0(\kappa)}{2} - A(\kappa)^2 = \operatorname{Var}(\cos(X-\mu)). A ′ ( κ ) = 2 1 + I 2 ( κ ) / I 0 ( κ ) − A ( κ ) 2 = Var ( cos ( X − μ )) . The expansions in Lemma C.1 imply
κ A ( κ ) ∼ κ and A ′ ( κ ) ∼ 1 2 κ 2 . \kappa A(\kappa) \sim\kappa\quad\text{and} \quad A'(\kappa) \sim\frac{1}{2\kappa^2}. κ A ( κ ) ∼ κ and A ′ ( κ ) ∼ 2 κ 2 1 . Thus the concentration entry of the inverse Fisher information for n n n
1 n A ′ ( κ n ) ∼ 2 κ n 2 n . \frac{1}{nA'(\kappa_n)} \sim\frac{2\kappa_n^2}{n}. n A ′ ( κ n ) 1 ∼ n 2 κ n 2 . We derive the joint MLE limit directly from the empirical sine and cosine, retaining the definitions of C n C_n C n S n S_n S n
( n κ n A ( κ n ) S n n C n − A ( κ n ) A ′ ( κ n ) ) → d N ( 0 , I 2 ) . \begin{pmatrix}
\sqrt{\frac{n\kappa_n}{A(\kappa_n)}}S_n \\
\sqrt{n}\frac{C_n-A(\kappa_n)}{\sqrt{A'(\kappa_n)}}
\end{pmatrix}
\xrightarrow{d} \mathcal{N}(0,I_2). A ( κ n ) n κ n S n n A ′ ( κ n ) C n − A ( κ n ) d N ( 0 , I 2 ) . The identity
μ ^ n − μ = arctan S n C n , \widehat{\mu}_n-\mu= \arctan\frac{S_n}{C_n}, μ n − μ = arctan C n S n , together with R n − C n = S n 2 / ( R n + C n ) R_n-C_n=S_n^2/(R_n+C_n) R n − C n = S n 2 / ( R n + C n )
μ ^ n − μ = S n A ( κ n ) + o P ( ( n κ n ) − 1 / 2 ) and R n − A ( κ n ) = C n − A ( κ n ) + o P ( 1 κ n n ) . \widehat{\mu}_n-\mu=\frac{S_n}{A(\kappa_n)}+o_{\mathrm{P}}\left((n\kappa_n)^{-1/2}\right)
\quad\text{and}\quad
R_n-A(\kappa_n)=C_n-A(\kappa_n)+o_{\mathrm{P}}\left(\frac{1}{\kappa_n\sqrt{n}}\right). μ n − μ = A ( κ n ) S n + o P ( ( n κ n ) − 1/2 ) and R n − A ( κ n ) = C n − A ( κ n ) + o P ( κ n n 1 ) . Since A ( κ n ) → 1 A(\kappa_n)\to1 A ( κ n ) → 1 κ n 2 A ′ ( κ n ) → 1 / 2 \kappa_n^2A'(\kappa_n)\to1/2 κ n 2 A ′ ( κ n ) → 1/2
( n κ n ( μ ^ n − μ ) n κ n ( R n − A ( κ n ) ) ) → d N ( 0 , ( 1 0 0 1 2 ) ) . \begin{pmatrix}
\sqrt{n\kappa_n}(\widehat{\mu}_n-\mu) \\
\sqrt{n}\kappa_n(R_n-A(\kappa_n))
\end{pmatrix}
\xrightarrow{d}
\mathcal{N}\left(0,
\begin{pmatrix}
1 & 0 \\
0 & \frac{1}{2}
\end{pmatrix}\right). ( n κ n ( μ n − μ ) n κ n ( R n − A ( κ n )) ) d N ( 0 , ( 1 0 0 2 1 ) ) . The bound (C.1) and 1 − A ( κ n ) ∼ 1 / ( 2 κ n ) 1-A(\kappa_n)\sim1/(2\kappa_n) 1 − A ( κ n ) ∼ 1/ ( 2 κ n )
1 − R n 1 − A ( κ n ) = 1 − R n − A ( κ n ) 1 − A ( κ n ) = 1 + o P ( 1 ) . \frac{1-R_n}{1-A(\kappa_n)}
=1-\frac{R_n-A(\kappa_n)}{1-A(\kappa_n)}
=1+o_{\mathrm{P}}(1). 1 − A ( κ n ) 1 − R n = 1 − 1 − A ( κ n ) R n − A ( κ n ) = 1 + o P ( 1 ) . Since 1 − A ( κ ^ n ) = 1 − R n 1-A(\widehat{\kappa}_n)=1-R_n 1 − A ( κ n ) = 1 − R n
1 − A ( κ ^ n ) 1 − A ( κ n ) → P 1. \frac{1-A(\widehat{\kappa}_n)}{1-A(\kappa_n)}
\xrightarrow{\mathrm{P}}1. 1 − A ( κ n ) 1 − A ( κ n ) P 1. Because κ n → ∞ \kappa_n \to\infty κ n → ∞ 1 − A ( κ n ) → 0 1 - A(\kappa_n) \to0 1 − A ( κ n ) → 0 1 − A ( κ ^ n ) → p 0 1 - A(\widehat{\kappa}_n) \overset{p}{\to} 0 1 − A ( κ n ) → p 0 A A A κ ^ n → ∞ \widehat{\kappa}_n \to\infty κ n → ∞ 1 − A ( x ) ∼ ( 2 x ) − 1 1 - A(x) \sim(2x)^{-1} 1 − A ( x ) ∼ ( 2 x ) − 1 κ n \kappa_n κ n κ ^ n \widehat{\kappa}_n κ n
1 − A ( κ ^ n ) 1 − A ( κ n ) = κ n κ ^ n ( 1 + o P ( 1 ) ) , \frac{1-A(\widehat{\kappa}_n)}{1-A(\kappa_n)}=\frac{\kappa_n}{\widehat{\kappa}_n}(1+o_P(1)), 1 − A ( κ n ) 1 − A ( κ n ) = κ n κ n ( 1 + o P ( 1 )) , and hence
κ ^ n κ n → p 1. \frac{\widehat{\kappa}_n}{\kappa_n}\overset{p}{\to}1. κ n κ n → p 1. Let κ ‾ n \overline{\kappa}_n κ n κ n \kappa_n κ n κ ^ n \widehat{\kappa}_n κ n
R n − A ( κ n ) = A ′ ( κ ‾ n ) ( κ ^ n − κ n ) . R_n-A(\kappa_n)=A'(\overline{\kappa}_n)(\widehat{\kappa}_n-\kappa_n). R n − A ( κ n ) = A ′ ( κ n ) ( κ n − κ n ) . Because κ ^ n / κ n → p 1 \widehat{\kappa}_n/\kappa_n \overset{p}{\to} 1 κ n / κ n → p 1 κ ‾ n / κ n → p 1 \overline{\kappa}_n/\kappa_n \overset{p}{\to} 1 κ n / κ n → p 1
κ n 2 A ′ ( κ ‾ n ) → p 1 2 . \kappa_n^2 A'(\overline{\kappa}_n)\overset{p}{\to}\frac{1}{2}. κ n 2 A ′ ( κ n ) → p 2 1 . Combining this with the previous bound on R n − A ( κ n ) R_n-A(\kappa_n) R n − A ( κ n )
κ ^ n − κ n = O P ( κ n n ) . \widehat{\kappa}_n-\kappa_n=O_P\left(\frac{\kappa_n}{\sqrt{n}}\right). κ n − κ n = O P ( n κ n ) . Taylor’s theorem then gives
R n − A ( κ n ) = A ′ ( κ n ) ( κ ^ n − κ n ) + 1 2 A ′ ′ ( κ n † ) ( κ ^ n − κ n ) 2 , R_n-A(\kappa_n)=A'(\kappa_n)(\widehat{\kappa}_n-\kappa_n)+\frac{1}{2}A''(\kappa_n^\dagger)(\widehat{\kappa}_n-\kappa_n)^2, R n − A ( κ n ) = A ′ ( κ n ) ( κ n − κ n ) + 2 1 A ′′ ( κ n † ) ( κ n − κ n ) 2 , where κ n † \kappa_n^\dagger κ n † κ n \kappa_n κ n κ ^ n \widehat{\kappa}_n κ n κ ^ n / κ n → p 1 \widehat{\kappa}_n/\kappa_n \overset{p}{\to} 1 κ n / κ n → p 1 κ n † / κ n → p 1 \kappa_n^\dagger/\kappa_n \overset{p}{\to} 1 κ n † / κ n → p 1 A ′ ′ ( κ ) = O ( κ − 3 ) A''(\kappa)=O(\kappa^{-3}) A ′′ ( κ ) = O ( κ − 3 ) O P ( 1 / ( n κ n ) ) O_P(1/(n\kappa_n)) O P ( 1/ ( n κ n )) A ′ ( κ n ) ∼ 1 / ( 2 κ n 2 ) A'(\kappa_n)\sim1/(2\kappa_n^2) A ′ ( κ n ) ∼ 1/ ( 2 κ n 2 )
κ ^ n − κ n = R n − A ( κ n ) A ′ ( κ n ) + O P ( κ n n ) . \widehat{\kappa}_n-\kappa_n=\frac{R_n-A(\kappa_n)}{A'(\kappa_n)}+O_P\left(\frac{\kappa_n}{n}\right). κ n − κ n = A ′ ( κ n ) R n − A ( κ n ) + O P ( n κ n ) . It follows that
( n κ n ( μ ^ n − μ ) n κ ^ n − κ n κ n ) ⟶ d N ( ( 0 0 ) , ( 1 0 0 2 ) ) . \begin{pmatrix}
\sqrt{n\kappa_n}(\widehat{\mu}_n-\mu)\\
\sqrt{n}\dfrac{\widehat{\kappa}_n-\kappa_n}{\kappa_n}
\end{pmatrix}
\overset{d}{\longrightarrow}
\mathcal{N}\left(\begin{pmatrix}0\\0\end{pmatrix},\begin{pmatrix}1&0\\0&2\end{pmatrix}\right). n κ n ( μ n − μ ) n κ n κ n − κ n ⟶ d N ( ( 0 0 ) , ( 1 0 0 2 ) ) . The location coordinates of the exact and approximate MLEs agree exactly; for the concentration coordinates, root stability gives
n κ n ( κ ~ n − κ ^ n ) = O P ( n κ n 2 ) . \frac{\sqrt{n}}{\kappa_n}(\widetilde{\kappa}_n-\widehat{\kappa}_n)=O_P\left(\frac{\sqrt{n}}{\kappa_n^2}\right). κ n n ( κ n − κ n ) = O P ( κ n 2 n ) . If n / κ n 2 → 0 \sqrt{n}/\kappa_n^2 \to0 n / κ n 2 → 0 κ n / n 1 / 4 → ∞ \kappa_n/n^{1/4}\to\infty κ n / n 1/4 → ∞ o P ( 1 ) o_P(1) o P ( 1 )
Simulation Study
We used the same sample size grid and number of replications as in the gamma and zeta studies. We set μ : = 0 \mu:= 0 μ := 0 κ n : = n γ \kappa_n := n^\gamma κ n := n γ
γ ∈ { 0.15 , 0.25 , 0.40 } and n ∈ { 250 , 500 , 1000 , 2500 , 5000 , 10000 , 25000 } , \gamma\in\{0.15,0.25,0.40\}\quad\text{and}\quad n\in\{250,500,1000,2500,5000,10000,25000\}, γ ∈ { 0.15 , 0.25 , 0.40 } and n ∈ { 250 , 500 , 1000 , 2500 , 5000 , 10000 , 25000 } , and generated 2000 samples at each design point. These regimes lay below, at, and above the sufficient condition γ > 1 / 4 \gamma>1/4 γ > 1/4
In each replication, we obtained the exact MLE by numerically solving A ( κ ^ n ) = R n A(\widehat{\kappa}_n)=R_n A ( κ n ) = R n § 4.2 . We also included the estimator of Abeyasekera and Collett [ [1 ] as an explicit benchmark because, like our estimator, it is designed for the high-concentration, large-κ \kappa κ 2 π 2\pi 2 π μ ^ n \widehat{\mu}_n μ n X n , i ∘ X^\circ_{n,i} X n , i ∘ X ‾ n ∘ \overline{X}^\circ_n X n ∘
κ ^ n l i n : = 1 ( n − 3 ) − 1 ∑ i = 1 n ( X n , i ∘ − X ‾ n ∘ ) 2 . \widehat{\kappa}^{\mathrm{lin}}_n := \frac{1}{(n-3)^{-1}\sum_{i=1}^{n}(X^\circ_{n,i}-\overline{X}^\circ_n)^2}. κ n lin := ( n − 3 ) − 1 ∑ i = 1 n ( X n , i ∘ − X n ∘ ) 2 1 . For the comparison in Figure 7 , we divided ∣ κ ~ n − κ ^ n ∣ |\widetilde{\kappa}_n-\widehat{\kappa}_n| ∣ κ n − κ n ∣ 2 κ n / n \sqrt{2\kappa_n/n} 2 κ n / n n n n γ = 0.15 \gamma=0.15 γ = 0.15 γ = 0.25 \gamma=0.25 γ = 0.25 γ = 0.40 \gamma=0.40 γ = 0.40 n = 25000 n=25000 n = 25000 n / κ n 2 = n 1 / 2 − 2 γ \sqrt{n}/\kappa_n^2=n^{1/2-2\gamma} n / κ n 2 = n 1/2 − 2 γ
Figure 7. Median absolute difference between the approximate and exact concentration estimators, divided by the asymptotic standard error of the exact MLE. The horizontal dashed line marks one asymptotic standard error. Each point is based on 2000 replications.
The RMSE comparison in Figure 8 gives a similar picture. At n = 25000 n=25000 n = 25000 γ = 0.15 \gamma=0.15 γ = 0.15 γ = 0.25 \gamma=0.25 γ = 0.25 γ = 0.40 \gamma=0.40 γ = 0.40
Figure 8. RMSE of each explicit concentration estimator relative to that of the exact MLE. The dashed horizontal line marks equal RMSE. Each point is based on 2000 replications.
Finally, Figure 9 examines the joint limit at ( n , γ ) = ( 25000 , 0.40 ) (n,\gamma)=(25000,0.40) ( n , γ ) = ( 25000 , 0.40 ) − 0.001 -0.001 − 0.001 − 0.047 -0.047 − 0.047
Figure 9. Distributional diagnostics at n = 25000 n=25000 n = 25000 γ = 0.40 \gamma=0.40 γ = 0.40
Generalized Pareto Distribution
Throughout this section, fix 0 < ξ ˉ < 1 / 6 0 < \bar{\xi} < 1/6 0 < ξ ˉ < 1/6 μ ∈ R \mu\in\mathbb{R} μ ∈ R σ > 0 \sigma> 0 σ > 0 ξ n ↓ 0 \xi_n \downarrow0 ξ n ↓ 0
X n , 1 , … , X n , n ∼ i i d GPD ( μ , σ , ξ n ) . X_{n,1},\ldots,X_{n,n} \mathrel{\mathop{\sim}^{\mathrm{iid}}} \operatorname{GPD}(\mu,\sigma,\xi_n). X n , 1 , … , X n , n ∼ iid GPD ( μ , σ , ξ n ) . Throughout the GPD appendix, stochastic orders are understood under P n : = P μ , σ , ξ n ⊗ n \mathbb{P}_n := \mathbb{P}_{\mu,\sigma,\xi_n}^{\otimes n} P n := P μ , σ , ξ n ⊗ n
Preliminary Results
Moments and the Sample Minimum
Lemma D.1. For any integer k ≥ 1 k \ge1 k ≥ 1 n n n k ξ n < 1 k\xi_n < 1 k ξ n < 1
m k ( ξ n ) : = E P n [ ( X n , 1 − μ ) k ] = k ! σ k ∏ j = 1 k ( 1 − j ξ n ) . (107) m_k(\xi_n) := \mathbb{E}_{\mathbb{P}_n}\left[(X_{n,1}-\mu)^k\right] = \frac{k!\sigma^k}{\prod_{j=1}^{k}(1-j\xi_n)}. \tag*{(107)} m k ( ξ n ) := E P n [ ( X n , 1 − μ ) k ] = ∏ j = 1 k ( 1 − j ξ n ) k ! σ k . (107) Moreover,
n σ ( X ( 1 ) − μ ) → d Exp ( 1 ) and X ( 1 ) − μ = O P n ( 1 n ) . (108) \frac{n}{\sigma}(X_{(1)}-\mu) \xrightarrow{d} \operatorname{Exp}(1) \quad\text{and} \quad X_{(1)}-\mu= O_{\mathbb{P}_n}\left(\frac{1}{n}\right). \tag*{(108)} σ n ( X ( 1 ) − μ ) d Exp ( 1 ) and X ( 1 ) − μ = O P n ( n 1 ) . (108) For every fixed k ∈ N k \in\mathbb{N} k ∈ N
M k , n : = 1 n ∑ i = 1 n ( X n , i − X ( 1 ) ) k . M_{k,n} := \frac{1}{n}\sum_{i=1}^{n}(X_{n,i}-X_{(1)})^k. M k , n := n 1 i = 1 ∑ n ( X n , i − X ( 1 ) ) k . Then
M k , n = m k ( ξ n ) + O P n ( n − 1 / 2 ) . (109) M_{k,n}=m_k(\xi_n)+O_{\mathbb{P}_n}\left(n^{-1/2}\right). \tag*{(109)} M k , n = m k ( ξ n ) + O P n ( n − 1/2 ) . (109) Proof. Let Z : = X n , 1 − μ Z := X_{n,1}-\mu Z := X n , 1 − μ k ξ n < 1 k\xi_n < 1 k ξ n < 1
E P n [ Z k ] = k ∫ 0 ∞ y k − 1 ( 1 + ξ n y σ ) − 1 / ξ n d y = k σ k ξ n k B ( k , 1 ξ n − k ) = k ! σ k ∏ j = 1 k ( 1 − j ξ n ) , \mathbb{E}_{\mathbb{P}_n}\left[Z^k\right]
=k\int_{0}^{\infty}y^{k-1}\left(1+\frac{\xi_n y}{\sigma}\right)^{-1/\xi_n}\,\mathrm{d}y
=\frac{k\sigma^k}{\xi_n^k}B\left(k,\frac{1}{\xi_n}-k\right)
=\frac{k!\sigma^k}{\prod_{j=1}^{k}(1-j\xi_n)}, E P n [ Z k ] = k ∫ 0 ∞ y k − 1 ( 1 + σ ξ n y ) − 1/ ξ n d y = ξ n k k σ k B ( k , ξ n 1 − k ) = ∏ j = 1 k ( 1 − j ξ n ) k ! σ k , where the formula at ξ n = 0 \xi_n=0 ξ n = 0 ξ ˉ < 1 / 6 \bar{\xi}<1/6 ξ ˉ < 1/6 ξ n → 0 \xi_n \to0 ξ n → 0 q ∈ N q \in\mathbb{N} q ∈ N q q q n n n
P n ( n σ ( X ( 1 ) − μ ) > x ) = ( 1 + ξ n x n ) − n / ξ n ⟶ e − x , \mathbb{P}_n\left(\frac{n}{\sigma}(X_{(1)}-\mu)>x\right)
=\left(1+\frac{\xi_n x}{n}\right)^{-n/\xi_n}
\longrightarrow e^{-x}, P n ( σ n ( X ( 1 ) − μ ) > x ) = ( 1 + n ξ n x ) − n / ξ n ⟶ e − x , with the exponential interpretation when ξ n = 0 \xi_n=0 ξ n = 0
Put δ n : = X ( 1 ) − μ \delta_n := X_{(1)}-\mu δ n := X ( 1 ) − μ Z n , i : = X n , i − μ Z_{n,i}:=X_{n,i}-\mu Z n , i := X n , i − μ k ∈ N k \in\mathbb{N} k ∈ N
∣ M k , n − 1 n ∑ i = 1 n Z n , i k ∣ ≤ C k δ n ( 1 n ∑ i = 1 n ( 1 + Z n , i k − 1 ) + δ n k − 1 ) = O P n ( 1 n ) . \left|M_{k,n}-\frac{1}{n}\sum_{i=1}^{n}Z_{n,i}^{k}\right|
\le C_k\delta_n\left(\frac{1}{n}\sum_{i=1}^{n}(1+Z_{n,i}^{k-1})+\delta_n^{k-1}\right)
=O_{\mathbb{P}_n}\left(\frac{1}{n}\right). M k , n − n 1 i = 1 ∑ n Z n , i k ≤ C k δ n ( n 1 i = 1 ∑ n ( 1 + Z n , i k − 1 ) + δ n k − 1 ) = O P n ( n 1 ) . The eventual ( 2 k ) (2k) ( 2 k ) n − 1 ∑ i = 1 n Z n , i k n^{-1}\sum_{i=1}^{n}Z_{n,i}^{k} n − 1 ∑ i = 1 n Z n , i k O ( n − 1 ) O(n^{-1}) O ( n − 1 ) m k ( ξ n ) m_k(\xi_n) m k ( ξ n )
1 n ∑ i = 1 n Z n , i k = m k ( ξ n ) + O P n ( n − 1 / 2 ) . \frac{1}{n}\sum_{i=1}^{n}Z_{n,i}^{k}
=m_k(\xi_n)+O_{\mathbb{P}_n}\left(n^{-1/2}\right). n 1 i = 1 ∑ n Z n , i k = m k ( ξ n ) + O P n ( n − 1/2 ) . Combining this estimate with the O P n ( n − 1 ) O_{\mathbb{P}_n}(n^{-1}) O P n ( n − 1 ) μ \mu μ X ( 1 ) X_{(1)} X ( 1 )
Restricted Global MLE
Proposition D.1. For every n ≥ 2 n \ge2 n ≥ 2
Θ ξ ˉ = { ( μ ′ , s , η ) : μ ′ ∈ R , s > 0 , 0 ≤ η ≤ ξ ˉ } \Theta_{\bar{\xi}} = \{(\mu',s,\eta): \mu' \in\mathbb{R},\ s > 0,\ 0 \le\eta\le\bar{\xi}\} Θ ξ ˉ = {( μ ′ , s , η ) : μ ′ ∈ R , s > 0 , 0 ≤ η ≤ ξ ˉ } has a global maximizer almost surely. If n ξ n → ∞ \sqrt{n}\,\xi_n \to\infty n ξ n → ∞
μ ^ n = X ( 1 ) and σ ^ n − σ = O P n ( n − 1 / 2 ) and ξ ^ n − ξ n = O P n ( n − 1 / 2 ) , \widehat{\mu}_n = X_{(1)} \quad\text{and} \quad\widehat{\sigma}_n - \sigma= O_{\mathbb{P}_n}(n^{-1/2}) \quad\text{and} \quad\widehat{\xi}_n - \xi_n = O_{\mathbb{P}_n}(n^{-1/2}), μ n = X ( 1 ) and σ n − σ = O P n ( n − 1/2 ) and ξ n − ξ n = O P n ( n − 1/2 ) , and ( σ ^ n , ξ ^ n ) ∈ ( 0 , ∞ ) × ( 0 , ξ ˉ ) (\widehat{\sigma}_n,\widehat{\xi}_n) \in(0,\infty) \times(0,\bar{\xi}) ( σ n , ξ n ) ∈ ( 0 , ∞ ) × ( 0 , ξ ˉ ) ( σ , 0 ) (\sigma,0) ( σ , 0 ) ( 0 , ∞ ) × [ 0 , ξ ˉ ] (0,\infty) \times[0,\bar{\xi}] ( 0 , ∞ ) × [ 0 , ξ ˉ ]
Proof. For η > 0 \eta> 0 η > 0 μ ′ ≤ X ( 1 ) \mu' \le X_{(1)} μ ′ ≤ X ( 1 )
∂ ℓ n ∂ μ ′ = 1 + η s ∑ i = 1 n 1 1 + η ( X n , i − μ ′ ) / s > 0. \frac{\partial\ell_n}{\partial\mu'} = \frac{1+\eta}{s}\sum_{i=1}^{n}\frac{1}{1+\eta(X_{n,i}-\mu')/s} > 0. ∂ μ ′ ∂ ℓ n = s 1 + η i = 1 ∑ n 1 + η ( X n , i − μ ′ ) / s 1 > 0. At η = 0 \eta= 0 η = 0 n / s > 0 n/s > 0 n / s > 0 μ ′ = m n : = X ( 1 ) \mu' = m_n := X_{(1)} μ ′ = m n := X ( 1 ) Y i : = X n , i − m n Y_i := X_{n,i}-m_n Y i := X n , i − m n
ℓ n p ( s , η ) : = − n log ( s ) − ( 1 + 1 η ) ∑ i = 1 n log ( 1 + η Y i s ) , (110) \ell_n^p(s,\eta) := -n\log(s) - \left(1+\frac{1}{\eta}\right)\sum_{i=1}^{n}\log\left(1+\frac{\eta Y_i}{s}\right), \tag*{(110)} ℓ n p ( s , η ) := − n log ( s ) − ( 1 + η 1 ) i = 1 ∑ n log ( 1 + s η Y i ) , (110) with the continuous extension
ℓ n p ( s , 0 ) = − n log ( s ) − 1 s ∑ i = 1 n Y i . \ell_n^p(s,0) = -n\log(s) - \frac{1}{s}\sum_{i=1}^{n}Y_i. ℓ n p ( s , 0 ) = − n log ( s ) − s 1 i = 1 ∑ n Y i . The bound ℓ n p ( s , η ) ≤ − n log ( s ) \ell_n^p(s,\eta) \le-n\log(s) ℓ n p ( s , η ) ≤ − n log ( s ) η ∈ [ 0 , ξ ˉ ] \eta\in[0,\bar{\xi}] η ∈ [ 0 , ξ ˉ ] − ∞ -\infty − ∞ s → ∞ s \to\infty s → ∞ r ≥ 0 r \ge0 r ≥ 0 η ↦ log ( 1 + η r ) / η \eta\mapsto\log(1+\eta r)/\eta η ↦ log ( 1 + η r ) / η Y i Y_i Y i
ℓ n p ( s , η ) ≤ − n log ( s ) − 1 ξ ˉ ∑ i = 1 n log ( 1 + ξ ˉ Y i s ) . (111) \ell_n^p(s,\eta) \le-n\log(s) - \frac{1}{\bar{\xi}}\sum_{i=1}^{n}\log\left(1+\frac{\bar{\xi}Y_i}{s}\right). \tag*{(111)} ℓ n p ( s , η ) ≤ − n log ( s ) − ξ ˉ 1 i = 1 ∑ n log ( 1 + s ξ ˉ Y i ) . (111) Fix n n n Y i Y_i Y i s ↓ 0 s \downarrow0 s ↓ 0
log ( 1 + ξ ˉ Y i s ) = − log ( s ) + log ( ξ ˉ Y i ) + o ( 1 ) . \log\left(1+\frac{\bar{\xi}Y_i}{s}\right) = -\log(s) + \log(\bar{\xi}Y_i) + o(1). log ( 1 + s ξ ˉ Y i ) = − log ( s ) + log ( ξ ˉ Y i ) + o ( 1 ) . Thus, pathwise, the right side of (D.5) is
( n − 1 ξ ˉ − n ) log ( s ) + O ( 1 ) , \left(\frac{n-1}{\bar{\xi}}-n\right)\log(s)+O(1), ( ξ ˉ n − 1 − n ) log ( s ) + O ( 1 ) , which tends to − ∞ -\infty − ∞ ξ ˉ < 1 / 6 \bar{\xi}<1/6 ξ ˉ < 1/6 n ≥ 2 n \ge2 n ≥ 2 Z n , i : = X n , i − μ Z_{n,i} := X_{n,i}-\mu Z n , i := X n , i − μ m n − μ = O P n ( n − 1 ) m_n-\mu= O_{\mathbb{P}_n}(n^{-1}) m n − μ = O P n ( n − 1 ) z ≥ 0 z \ge0 z ≥ 0
λ ( z ; s , η ) : = − log ( s ) − ( 1 + 1 η ) log ( 1 + η z s ) , \lambda(z;s,\eta) := -\log(s)-\left(1+\frac{1}{\eta}\right)\log\left(1+\frac{\eta z}{s}\right), λ ( z ; s , η ) := − log ( s ) − ( 1 + η 1 ) log ( 1 + s η z ) , with λ ( z ; s , 0 ) : = − log ( s ) − z / s \lambda(z;s,0):=-\log(s)-z/s λ ( z ; s , 0 ) := − log ( s ) − z / s Z 0 ∼ GPD ( 0 , σ , 0 ) Z_0 \sim\operatorname{GPD}(0,\sigma,0) Z 0 ∼ GPD ( 0 , σ , 0 )
L ( s , η ) : = E [ λ ( Z 0 ; s , η ) ] . L(s,\eta):=\mathbb{E}[\lambda(Z_0;s,\eta)]. L ( s , η ) := E [ λ ( Z 0 ; s , η )] . Fix K ε , M : = [ ε , M ] × [ 0 , ξ ˉ ] \mathcal{K}_{\varepsilon,M}:=[\varepsilon,M]\times[0,\bar{\xi}] K ε , M := [ ε , M ] × [ 0 , ξ ˉ ] λ θ ( z ) : = λ ( z ; s , η ) \lambda_\theta(z):=\lambda(z;s,\eta) λ θ ( z ) := λ ( z ; s , η ) θ : = ( s , η ) \theta:=(s,\eta) θ := ( s , η ) η \eta η
sup θ ∈ K ε , M ( ∣ λ θ ( z ) ∣ + ∥ ∇ λ θ ( z ) ∥ ) ≤ G ( z ) : = C ε , M ( 1 + z 2 ) . \sup_{\theta\in\mathcal{K}_{\varepsilon,M}}\left(\lvert\lambda_\theta(z)\rvert+\lVert\nabla\lambda_\theta(z)\rVert\right)\leq G(z):=C_{\varepsilon,M}(1+z^2). θ ∈ K ε , M sup ( ∣ λ θ ( z )∣ + ∥ ∇ λ θ ( z )∥ ) ≤ G ( z ) := C ε , M ( 1 + z 2 ) . By Lemma D.1, sup n ≥ n 0 E P n [ G ( Z n , 1 ) 2 ] < ∞ \sup_{n\geq n_0}\mathbb{E}_{\mathbb{P}_n}[G(Z_{n,1})^2]<\infty sup n ≥ n 0 E P n [ G ( Z n , 1 ) 2 ] < ∞ n 0 n_0 n 0 W ∼ Exp ( 1 ) W\sim\operatorname{Exp}(1) W ∼ Exp ( 1 )
Z n ∗ : = σ exp ( ξ n W ) − 1 ξ n , Z_n^*:=\sigma\frac{\exp(\xi_nW)-1}{\xi_n}, Z n ∗ := σ ξ n exp ( ξ n W ) − 1 , with Z n ∗ : = σ W Z_n^*:=\sigma W Z n ∗ := σ W ξ n = 0 \xi_n=0 ξ n = 0 z ≥ 0 z\geq0 z ≥ 0 ξ n > 0 \xi_n>0 ξ n > 0
P ( Z n ∗ > z ) = P ( W > 1 ξ n log ( 1 + ξ n z σ ) ) = ( 1 + ξ n z σ ) − 1 / ξ n , \mathbb{P}(Z_n^*>z)=\mathbb{P}\left(W>\frac{1}{\xi_n}\log\left(1+\frac{\xi_n z}{\sigma}\right)\right)=\left(1+\frac{\xi_n z}{\sigma}\right)^{-1/\xi_n}, P ( Z n ∗ > z ) = P ( W > ξ n 1 log ( 1 + σ ξ n z ) ) = ( 1 + σ ξ n z ) − 1/ ξ n , which is the survival function of Z n , 1 = X n , 1 − μ Z_{n,1}=X_{n,1}-\mu Z n , 1 = X n , 1 − μ Z n ∗ = d Z n , 1 Z_n^*\mathrel{\overset{d}{=}}Z_{n,1} Z n ∗ = d Z n , 1
Z n ∗ ⟶ σ W . Z_n^*\longrightarrow\sigma W. Z n ∗ ⟶ σ W . Uniform integrability therefore gives, for each θ ∈ K ε , M \theta\in\mathcal{K}_{\varepsilon,M} θ ∈ K ε , M
E P n [ λ θ ( Z n , 1 ) ] ⟶ L ( θ ) . \mathbb{E}_{\mathbb{P}_n}[\lambda_\theta(Z_{n,1})]\longrightarrow L(\theta). E P n [ λ θ ( Z n , 1 )] ⟶ L ( θ ) . To obtain uniform convergence, fix δ > 0 \delta>0 δ > 0 δ \delta δ θ 1 , … , θ N \theta_1,\ldots,\theta_N θ 1 , … , θ N K ε , M \mathcal{K}_{\varepsilon,M} K ε , M a > 0 a>0 a > 0
P n ( max 1 ≤ j ≤ N ∣ 1 n ∑ i = 1 n ( λ θ j ( Z n , i ) − E P n [ λ θ j ( Z n , 1 ) ] ) ∣ > a ) ≤ N C n a 2 \mathbb{P}_n\left(\max_{1\leq j\leq N}\left\lvert\frac{1}{n}\sum_{i=1}^{n}\left(\lambda_{\theta_j}(Z_{n,i})-\mathbb{E}_{\mathbb{P}_n}[\lambda_{\theta_j}(Z_{n,1})]\right)\right\rvert>a\right)\leq\frac{NC}{na^2} P n ( 1 ≤ j ≤ N max n 1 i = 1 ∑ n ( λ θ j ( Z n , i ) − E P n [ λ θ j ( Z n , 1 )] ) > a ) ≤ n a 2 N C and
1 n ∑ i = 1 n G ( Z n , i ) = E P n [ G ( Z n , 1 ) ] + O P n ( n − 1 / 2 ) = O P n ( 1 ) . \frac{1}{n}\sum_{i=1}^{n}G(Z_{n,i})=\mathbb{E}_{\mathbb{P}_n}[G(Z_{n,1})]+O_{\mathbb{P}_n}\left(n^{-1/2}\right)=O_{\mathbb{P}_n}(1). n 1 i = 1 ∑ n G ( Z n , i ) = E P n [ G ( Z n , 1 )] + O P n ( n − 1/2 ) = O P n ( 1 ) . For every θ \theta θ θ j \theta_j θ j ∥ θ − θ j ∥ ≤ δ \lVert\theta-\theta_j\rVert\leq\delta ∥ θ − θ j ∥ ≤ δ
sup θ ∈ K ε , M ∣ 1 n ∑ i = 1 n λ θ ( Z n , i ) − L ( θ ) ∣ ≤ max 1 ≤ j ≤ N ∣ 1 n ∑ i = 1 n λ θ j ( Z n , i ) − L ( θ j ) ∣ + δ ( 1 n ∑ i = 1 n G ( Z n , i ) + E [ G ( Z 0 ) ] ) . \sup_{\theta\in\mathcal{K}_{\varepsilon,M}}\left\lvert\frac{1}{n}\sum_{i=1}^{n}\lambda_\theta(Z_{n,i})-L(\theta)\right\rvert\leq\max_{1\leq j\leq N}\left\lvert\frac{1}{n}\sum_{i=1}^{n}\lambda_{\theta_j}(Z_{n,i})-L(\theta_j)\right\rvert+\delta\left(\frac{1}{n}\sum_{i=1}^{n}G(Z_{n,i})+\mathbb{E}[G(Z_0)]\right). θ ∈ K ε , M sup n 1 i = 1 ∑ n λ θ ( Z n , i ) − L ( θ ) ≤ 1 ≤ j ≤ N max n 1 i = 1 ∑ n λ θ j ( Z n , i ) − L ( θ j ) + δ ( n 1 i = 1 ∑ n G ( Z n , i ) + E [ G ( Z 0 )] ) . Letting n → ∞ n\to\infty n → ∞ δ ↓ 0 \delta\downarrow0 δ ↓ 0
sup ( s , η ) ∈ [ ε , M ] × [ 0 , ξ ˉ ] ∣ 1 n ℓ n , μ ( s , η ) − L ( s , η ) ∣ → P n 0 , (112) \sup_{(s,\eta)\in[\varepsilon,M]\times[0,\bar{\xi}]}\left\lvert\frac{1}{n}\ell_{n,\mu}(s,\eta)-L(s,\eta)\right\rvert\xrightarrow{\mathbb{P}_n}0, \tag*{(112)} ( s , η ) ∈ [ ε , M ] × [ 0 , ξ ˉ ] sup n 1 ℓ n , μ ( s , η ) − L ( s , η ) P n 0 , (112) where ℓ n , μ \ell_{n,\mu} ℓ n , μ μ \mu μ
sup K ε , M ∣ ∂ z λ ( z ; s , η ) ∣ ≤ 1 + ξ ˉ ε , \sup_{\mathcal{K}_{\varepsilon,M}}\lvert\partial_z\lambda(z;s,\eta)\rvert\leq\frac{1+\bar{\xi}}{\varepsilon}, K ε , M sup ∣ ∂ z λ ( z ; s , η )∣ ≤ ε 1 + ξ ˉ , replacing Z n , i Z_{n,i} Z n , i Y i = Z n , i − ( m n − μ ) Y_i=Z_{n,i}-(m_n-\mu) Y i = Z n , i − ( m n − μ ) C ( m n − μ ) = O P n ( n − 1 ) C(m_n-\mu)=O_{\mathbb{P}_n}(n^{-1}) C ( m n − μ ) = O P n ( n − 1 ) ℓ n , μ \ell_{n,\mu} ℓ n , μ ℓ n p \ell_n^p ℓ n p
Let f s , η f_{s,\eta} f s , η μ \mu μ s s s η \eta η
L ( s , η ) − L ( σ , 0 ) = E [ log ( f s , η ( Z 0 + μ ) f σ , 0 ( Z 0 + μ ) ) ] ≤ log ( E [ f s , η ( Z 0 + μ ) f σ , 0 ( Z 0 + μ ) ] ) = 0. L(s,\eta)-L(\sigma,0)=\mathbb{E}\left[\log\left(\frac{f_{s,\eta}(Z_0+\mu)}{f_{\sigma,0}(Z_0+\mu)}\right)\right]\leq\log\left(\mathbb{E}\left[\frac{f_{s,\eta}(Z_0+\mu)}{f_{\sigma,0}(Z_0+\mu)}\right]\right)=0. L ( s , η ) − L ( σ , 0 ) = E [ log ( f σ , 0 ( Z 0 + μ ) f s , η ( Z 0 + μ ) ) ] ≤ log ( E [ f σ , 0 ( Z 0 + μ ) f s , η ( Z 0 + μ ) ] ) = 0. Equality requires the two densities to coincide. When η > 0 \eta>0 η > 0
∂ 2 ∂ z 2 log ( f s , η ( z + μ ) ) = η ( 1 + η ) ( s + η z ) 2 > 0 , \frac{\partial^2}{\partial z^2}\log(f_{s,\eta}(z+\mu))=\frac{\eta(1+\eta)}{(s+\eta z)^2}>0, ∂ z 2 ∂ 2 log ( f s , η ( z + μ )) = ( s + η z ) 2 η ( 1 + η ) > 0 , whereas log ( f σ , 0 ( z + μ ) ) \log(f_{\sigma,0}(z+\mu)) log ( f σ , 0 ( z + μ )) z z z η = 0 \eta=0 η = 0 s = σ s=\sigma s = σ L L L ( σ , 0 ) (\sigma,0) ( σ , 0 )
It remains to control the tails of the scale space. Choose c 0 > 0 c_0>0 c 0 > 0 q : = P ( Z 0 > 2 c 0 ) > ξ ˉ q:=\mathbb{P}(Z_0>2c_0)>\bar{\xi} q := P ( Z 0 > 2 c 0 ) > ξ ˉ q ′ ∈ ( ξ ˉ , q ) q'\in(\bar{\xi},q) q ′ ∈ ( ξ ˉ , q )
p n : = P n ( Z n , 1 > 2 c 0 ) , p_n:=\mathbb{P}_n(Z_{n,1}>2c_0), p n := P n ( Z n , 1 > 2 c 0 ) , then p n → q p_n\to q p n → q
1 n ∑ i = 1 n 1 ( Z n , i > 2 c 0 ) = p n + O P n ( n − 1 / 2 ) ⟶ P n q , \frac{1}{n}\sum_{i=1}^{n}\mathbb{1}(Z_{n,i}>2c_0)=p_n+O_{\mathbb{P}_n}(n^{-1/2})\mathrel{\mathop{\longrightarrow}^{\mathbb{P}_n}}q, n 1 i = 1 ∑ n 1 ( Z n , i > 2 c 0 ) = p n + O P n ( n − 1/2 ) ⟶ P n q , because the variance of the average is at most 1 / ( 4 n ) 1/(4n) 1/ ( 4 n ) m n − μ = o P n ( 1 ) m_n-\mu=o_{\mathbb{P}_n}(1) m n − μ = o P n ( 1 ) q ′ n q'n q ′ n Y i Y_i Y i c 0 c_0 c 0
sup η ∈ [ 0 , ξ ˉ ] 1 n ℓ n p ( s , η ) ≤ − log ( s ) − q ′ ξ ˉ log ( 1 + ξ ˉ c 0 s ) , \sup_{\eta\in[0,\bar{\xi}]}\frac{1}{n}\ell_n^p(s,\eta)\leq-\log(s)-\frac{q'}{\bar{\xi}}\log\left(1+\frac{\bar{\xi}c_0}{s}\right), η ∈ [ 0 , ξ ˉ ] sup n 1 ℓ n p ( s , η ) ≤ − log ( s ) − ξ ˉ q ′ log ( 1 + s ξ ˉ c 0 ) , which tends to − ∞ -\infty − ∞ s ↓ 0 s\downarrow0 s ↓ 0 q ′ / ξ ˉ > 1 q'/\bar{\xi}>1 q ′ / ξ ˉ > 1 ℓ n p / n ≤ − log ( s ) \ell_n^p/n\leq-\log(s) ℓ n p / n ≤ − log ( s ) s → ∞ s\to\infty s → ∞ ( σ , 0 ) (\sigma,0) ( σ , 0 )
1 n ℓ n p ( σ , 0 ) ⟶ P n L ( σ , 0 ) . \frac{1}{n}\ell_n^p(\sigma,0)\mathrel{\mathop{\longrightarrow}^{\mathbb{P}_n}}L(\sigma,0). n 1 ℓ n p ( σ , 0 ) ⟶ P n L ( σ , 0 ) . The two tail bounds therefore allow fixed 0 < ϵ < M < ∞ 0<\epsilon<M<\infty 0 < ϵ < M < ∞ s ∉ [ ϵ , M ] s\notin[\epsilon,M] s ∈ / [ ϵ , M ] L L L ( σ , 0 ) (\sigma,0) ( σ , 0 )
σ ^ n ⟶ P n σ and ξ ^ n ⟶ P n 0. (113) \widehat{\sigma}_n\mathrel{\mathop{\longrightarrow}^{\mathbb{P}_n}}\sigma\quad\text{and}\quad\widehat{\xi}_n\mathrel{\mathop{\longrightarrow}^{\mathbb{P}_n}}0. \tag*{(113)} σ n ⟶ P n σ and ξ n ⟶ P n 0. (113) For Z 0 ∼ GPD ( 0 , σ , 0 ) Z_0\sim\operatorname{GPD}(0,\sigma,0) Z 0 ∼ GPD ( 0 , σ , 0 ) λ ( z ; s , η ) \lambda(z;s,\eta) λ ( z ; s , η ) ( s , η ) (s,\eta) ( s , η ) ( σ , 0 ) (\sigma,0) ( σ , 0 )
∇ s , η λ ( Z 0 ; σ , 0 ) = ( Z 0 − σ σ 2 Z 0 2 2 σ 2 − Z 0 σ ) . \nabla_{s,\eta}\lambda(Z_0;\sigma,0)=
\begin{pmatrix}
\dfrac{Z_0-\sigma}{\sigma^2}\\
\dfrac{Z_0^2}{2\sigma^2}-\dfrac{Z_0}{\sigma}
\end{pmatrix}. ∇ s , η λ ( Z 0 ; σ , 0 ) = σ 2 Z 0 − σ 2 σ 2 Z 0 2 − σ Z 0 . Differentiating once more and using E [ Z 0 k ] = k ! σ k \mathbb{E}[Z_0^k]=k!\sigma^k E [ Z 0 k ] = k ! σ k
I 0 : = − E [ ∇ s , η 2 λ ( Z 0 ; σ , 0 ) ] = ( 1 σ 2 1 σ 1 σ 2 ) . (114) \mathcal{I}_0:=-\mathbb{E}\left[\nabla^2_{s,\eta}\lambda(Z_0;\sigma,0)\right]=
\begin{pmatrix}
\dfrac{1}{\sigma^2} & \dfrac{1}{\sigma}\\
\dfrac{1}{\sigma} & 2
\end{pmatrix}. \tag*{(114)} I 0 := − E [ ∇ s , η 2 λ ( Z 0 ; σ , 0 ) ] = σ 2 1 σ 1 σ 1 2 . (114) whose determinant is 1 σ 2 > 0 \frac{1}{\sigma^2}>0 σ 2 1 > 0 K ⊂ ( 0 , ∞ ) × [ 0 , ξ ˉ ] \mathcal{K}\subset(0,\infty)\times[0,\bar{\xi}] K ⊂ ( 0 , ∞ ) × [ 0 , ξ ˉ ]
sup ( s , η ) ∈ K ( ∥ ∇ s , η 2 λ ( z ; s , η ) ∥ + ∥ ∇ s , η 3 λ ( z ; s , η ) ∥ ) ≤ C ( 1 + z 4 ) . \sup_{(s,\eta)\in\mathcal{K}}\left(\left\|\nabla^2_{s,\eta}\lambda(z;s,\eta)\right\|+\left\|\nabla^3_{s,\eta}\lambda(z;s,\eta)\right\|\right)\leq C(1+z^4). ( s , η ) ∈ K sup ( ∇ s , η 2 λ ( z ; s , η ) + ∇ s , η 3 λ ( z ; s , η ) ) ≤ C ( 1 + z 4 ) . Applying the argument leading to (D.6) componentwise to the Hessian therefore gives, on a sufficiently small fixed neighborhood V \mathcal{V} V ( σ , 0 ) (\sigma,0) ( σ , 0 )
sup ( s , η ) ∈ V ∥ 1 n ∇ 2 ℓ n , μ ( s , η ) − E [ ∇ s , η 2 λ ( Z 0 ; s , η ) ] ∥ ⟶ P n 0. (115) \sup_{(s,\eta)\in\mathcal{V}}\left\|\frac{1}{n}\nabla^2\ell_{n,\mu}(s,\eta)-\mathbb{E}\left[\nabla^2_{s,\eta}\lambda(Z_0;s,\eta)\right]\right\|\mathrel{\mathop{\longrightarrow}^{\mathbb{P}_n}}0. \tag*{(115)} ( s , η ) ∈ V sup n 1 ∇ 2 ℓ n , μ ( s , η ) − E [ ∇ s , η 2 λ ( Z 0 ; s , η ) ] ⟶ P n 0. (115) To compare the fixed-location and profile likelihoods, set
s ( z ; s , η ) : = ∇ s , η λ ( z ; s , η ) and H ( z ; s , η ) : = ∇ s , η 2 λ ( z ; s , η ) . s(z;s,\eta):=\nabla_{s,\eta}\lambda(z;s,\eta)\quad\text{and}\quad\mathbf{H}(z;s,\eta):=\nabla^2_{s,\eta}\lambda(z;s,\eta). s ( z ; s , η ) := ∇ s , η λ ( z ; s , η ) and H ( z ; s , η ) := ∇ s , η 2 λ ( z ; s , η ) . On V \mathcal{V} V
sup ( s , η ) ∈ V ( ∥ ∂ z s ( z ; s , η ) ∥ + ∥ ∂ z H ( z ; s , η ) ∥ ) ≤ C ( 1 + z 3 ) . (116) \sup_{(s,\eta)\in\mathcal{V}}\left(\left\|\partial_zs(z;s,\eta)\right\|+\left\|\partial_z\mathbf{H}(z;s,\eta)\right\|\right)\leq C(1+z^3). \tag*{(116)} ( s , η ) ∈ V sup ( ∥ ∂ z s ( z ; s , η ) ∥ + ∥ ∂ z H ( z ; s , η ) ∥ ) ≤ C ( 1 + z 3 ) . (116) Put δ n : = m n − μ \delta_n := m_n-\mu δ n := m n − μ Y i : = Z n , i − δ n Y_i := Z_{n,i}-\delta_n Y i := Z n , i − δ n 0 ≤ Y i ≤ Z n , i 0 \le Y_i \le Z_{n,i} 0 ≤ Y i ≤ Z n , i ( s , η ) ∈ V (s,\eta) \in\mathcal{V} ( s , η ) ∈ V
∥ s ( Y i ; s , η ) − s ( Z n , i ; s , η ) ∥ + ∥ H ( Y i ; s , η ) − H ( Z n , i ; s , η ) ∥ ≤ C δ n ( 1 + Z n , i 3 ) , \lVert\mathbf{s}(Y_i;s,\eta)-\mathbf{s}(Z_{n,i};s,\eta)\rVert+\lVert\mathbf{H}(Y_i;s,\eta)-\mathbf{H}(Z_{n,i};s,\eta)\rVert\le C\delta_n(1+Z_{n,i}^{3}), ∥ s ( Y i ; s , η ) − s ( Z n , i ; s , η )∥ + ∥ H ( Y i ; s , η ) − H ( Z n , i ; s , η )∥ ≤ C δ n ( 1 + Z n , i 3 ) , because each intermediate point lies between Y i Y_i Y i Z n , i Z_{n,i} Z n , i δ n = O P n ( n − 1 ) \delta_n=O_{\mathbb{P}_n}(n^{-1}) δ n = O P n ( n − 1 )
1 n ∑ i = 1 n ( 1 + Z n , i 3 ) = O P n ( 1 ) . \frac{1}{n}\sum_{i=1}^{n}(1+Z_{n,i}^{3})=O_{\mathbb{P}_n}(1). n 1 i = 1 ∑ n ( 1 + Z n , i 3 ) = O P n ( 1 ) . Consequently,
sup V ∥ ∇ ℓ n p − ∇ ℓ n , μ ∥ = O P n ( 1 ) and 1 n sup V ∥ ∇ 2 ℓ n p − ∇ 2 ℓ n , μ ∥ = O P n ( 1 n ) . (117) \sup_{\mathcal{V}}\lVert\nabla\ell_n^p-\nabla\ell_{n,\mu}\rVert=O_{\mathbb{P}_n}(1)
\quad\text{and}\quad
\frac{1}{n}\sup_{\mathcal{V}}\lVert\nabla^2\ell_n^p-\nabla^2\ell_{n,\mu}\rVert=O_{\mathbb{P}_n}\left(\frac{1}{n}\right). \tag*{(117)} V sup ∥ ∇ ℓ n p − ∇ ℓ n , μ ∥ = O P n ( 1 ) and n 1 V sup ∥ ∇ 2 ℓ n p − ∇ 2 ℓ n , μ ∥ = O P n ( n 1 ) . (117) By (D.8) and continuity, there is a fixed convex neighborhood V 0 ⊆ V \mathcal{V}_0\subseteq\mathcal{V} V 0 ⊆ V ( 0 , ∞ ) × [ 0 , ξ ˉ ] (0,\infty)\times[0,\bar{\xi}] ( 0 , ∞ ) × [ 0 , ξ ˉ ] ℓ n p \ell_n^p ℓ n p ( σ , ξ n ) (\sigma,\xi_n) ( σ , ξ n )
E P n [ ∥ ∇ ℓ n , μ ( σ , ξ n ) ∥ 2 ] = n E P n [ ∥ s ( Z n , 1 ; σ , ξ n ) ∥ 2 ] = O ( n ) . \mathbb{E}_{\mathbb{P}_n}\left[\lVert\nabla\ell_{n,\mu}(\sigma,\xi_n)\rVert^2\right]
=n\mathbb{E}_{\mathbb{P}_n}\left[\lVert\mathbf{s}(Z_{n,1};\sigma,\xi_n)\rVert^2\right]
=O(n). E P n [ ∥ ∇ ℓ n , μ ( σ , ξ n ) ∥ 2 ] = n E P n [ ∥ s ( Z n , 1 ; σ , ξ n ) ∥ 2 ] = O ( n ) . Consequently, the first comparison above gives
∇ ℓ n p ( σ , ξ n ) = O P n ( n ) . \nabla\ell_n^p(\sigma,\xi_n)=O_{\mathbb{P}_n}(\sqrt{n}). ∇ ℓ n p ( σ , ξ n ) = O P n ( n ) . For some λ 0 > 0 \lambda_0>0 λ 0 > 0 − ∇ 2 ℓ n p -\nabla^2\ell_n^p − ∇ 2 ℓ n p λ 0 n \lambda_0n λ 0 n V 0 \mathcal{V}_0 V 0 ∥ h ∥ = M / n \lVert h\rVert=M/\sqrt{n} ∥ h ∥ = M / n ( σ , ξ n ) (\sigma,\xi_n) ( σ , ξ n ) ( σ , ξ n ) + h (\sigma,\xi_n)+h ( σ , ξ n ) + h V 0 \mathcal{V}_0 V 0
ℓ n p ( ( σ , ξ n ) + h ) − ℓ n p ( σ , ξ n ) ≤ ∥ ∇ ℓ n p ( σ , ξ n ) ∥ ∥ h ∥ − λ 0 n 2 ∥ h ∥ 2 = M O P n ( 1 ) − λ 0 M 2 2 . \ell_n^p((\sigma,\xi_n)+h)-\ell_n^p(\sigma,\xi_n)
\le\lVert\nabla\ell_n^p(\sigma,\xi_n)\rVert\lVert h\rVert-\frac{\lambda_0n}{2}\lVert h\rVert^2
=MO_{\mathbb{P}_n}(1)-\frac{\lambda_0M^2}{2}. ℓ n p (( σ , ξ n ) + h ) − ℓ n p ( σ , ξ n ) ≤ ∥ ∇ ℓ n p ( σ , ξ n )∥ ∥ h ∥ − 2 λ 0 n ∥ h ∥ 2 = M O P n ( 1 ) − 2 λ 0 M 2 . Choose a deterministic sequence M n → ∞ M_n\to\infty M n → ∞
M n n ξ n ⟶ 0. \frac{M_n}{\sqrt{n}\xi_n}\longrightarrow0. n ξ n M n ⟶ 0. Then M n / n → 0 M_n/\sqrt{n}\to0 M n / n → 0 B ‾ M n / n ( ( σ , ξ n ) ) ⊆ V 0 ∩ ( ( 0 , ∞ ) × ( 0 , ξ ˉ ) ) \overline{B}_{M_n/\sqrt{n}}((\sigma,\xi_n))\subseteq\mathcal{V}_0\cap((0,\infty)\times(0,\bar{\xi})) B M n / n (( σ , ξ n )) ⊆ V 0 ∩ (( 0 , ∞ ) × ( 0 , ξ ˉ )) n n n M = M n M=M_n M = M n ∥ ∇ ℓ n p ( σ , ξ n ) ∥ / n = O P n ( 1 ) \lVert\nabla\ell_n^p(\sigma,\xi_n)\rVert/\sqrt{n}=O_{\mathbb{P}_n}(1) ∥ ∇ ℓ n p ( σ , ξ n )∥ / n = O P n ( 1 ) B ‾ M n / n ( ( σ , ξ n ) ) \overline{B}_{M_n/\sqrt{n}}((\sigma,\xi_n)) B M n / n (( σ , ξ n )) V 0 \mathcal{V}_0 V 0 V 0 \mathcal{V}_0 V 0
To obtain the rate, fix ϵ > 0 \epsilon>0 ϵ > 0 M M M 1 − ϵ / 2 1-\epsilon/2 1 − ϵ /2 n n n V 0 \mathcal{V}_0 V 0 M / n M/\sqrt{n} M / n ( σ , ξ n ) (\sigma,\xi_n) ( σ , ξ n ) 1 − ϵ 1-\epsilon 1 − ϵ n n n
σ ^ n − σ = O P n ( 1 n ) and ξ ^ n − ξ n = O P n ( 1 n ) . (118) \widehat{\sigma}_n-\sigma=O_{\mathbb{P}_n}\left(\frac{1}{\sqrt{n}}\right)
\quad\text{and}\quad
\widehat{\xi}_n-\xi_n=O_{\mathbb{P}_n}\left(\frac{1}{\sqrt{n}}\right). \tag*{(118)} σ n − σ = O P n ( n 1 ) and ξ n − ξ n = O P n ( n 1 ) . (118) Regularized Score and Truncation Error
For t i ( s ) : = Y i / s t_i(s):=Y_i/s t i ( s ) := Y i / s
A n ( s , η ) : = 1 n ∑ i = 1 n ( 1 + η ) t i ( s ) 1 + η t i ( s ) and B n ( s , η ) : = 1 n ∑ i = 1 n log ( 1 + η t i ( s ) ) η . A_n(s,\eta):=\frac{1}{n}\sum_{i=1}^{n}\frac{(1+\eta)t_i(s)}{1+\eta t_i(s)}
\quad\text{and}\quad
B_n(s,\eta):=\frac{1}{n}\sum_{i=1}^{n}\frac{\log(1+\eta t_i(s))}{\eta}. A n ( s , η ) := n 1 i = 1 ∑ n 1 + η t i ( s ) ( 1 + η ) t i ( s ) and B n ( s , η ) := n 1 i = 1 ∑ n η log ( 1 + η t i ( s )) . Let S s ( s , η ) : = ∂ s ℓ n p ( s , η ) S_s(s,\eta):=\partial_s\ell_n^p(s,\eta) S s ( s , η ) := ∂ s ℓ n p ( s , η ) S η ( s , η ) : = ∂ η ℓ n p ( s , η ) S_\eta(s,\eta):=\partial_\eta\ell_n^p(s,\eta) S η ( s , η ) := ∂ η ℓ n p ( s , η )
S s ( s , η ) n = A n ( s , η ) − 1 s and S η ( s , η ) n = B n ( s , η ) − A n ( s , η ) η . (119) \frac{S_s(s,\eta)}{n}=\frac{A_n(s,\eta)-1}{s}
\quad\text{and}\quad
\frac{S_\eta(s,\eta)}{n}=\frac{B_n(s,\eta)-A_n(s,\eta)}{\eta}. \tag*{(119)} n S s ( s , η ) = s A n ( s , η ) − 1 and n S η ( s , η ) = η B n ( s , η ) − A n ( s , η ) . (119) Consequently,
q n ( s , η ) : = ( B n ( s , η ) − 1 B n ( s , η ) − A n ( s , η ) η ) = ( s η 0 1 ) 1 n ( S s ( s , η ) S η ( s , η ) ) . (120) q_n(s,\eta):=
\begin{pmatrix}
B_n(s,\eta)-1\\
\frac{B_n(s,\eta)-A_n(s,\eta)}{\eta}
\end{pmatrix}
=
\begin{pmatrix}
s & \eta\\
0 & 1
\end{pmatrix}
\frac{1}{n}
\begin{pmatrix}
S_s(s,\eta)\\
S_\eta(s,\eta)
\end{pmatrix}. \tag*{(120)} q n ( s , η ) := ( B n ( s , η ) − 1 η B n ( s , η ) − A n ( s , η ) ) = ( s 0 η 1 ) n 1 ( S s ( s , η ) S η ( s , η ) ) . (120) The matrix in (D.14) has determinant s > 0 s>0 s > 0 q n = 0 q_n=0 q n = 0 η = 0 \eta=0 η = 0
The surrogate used in Section 4.3 is the first-order Taylor truncation in the shape coordinate: in each component of q n q_n q n η 0 \eta^0 η 0 η 1 \eta^1 η 1 η 2 \eta^2 η 2
Lemma D.2. Let q ~ n = ( q ~ 1 , n , q ~ 2 , n ) T \widetilde q_n=(\widetilde q_{1,n},\widetilde q_{2,n})^{\mathsf T} q n = ( q 1 , n , q 2 , n ) T
q ~ 1 , n ( s , η ) : = M 1 , n s − 1 − η M 2 , n 2 s 2 , \widetilde q_{1,n}(s,\eta):=\frac{M_{1,n}}{s}-1-\frac{\eta M_{2,n}}{2s^2}, q 1 , n ( s , η ) := s M 1 , n − 1 − 2 s 2 η M 2 , n , q ~ 2 , n ( s , η ) : = − M 1 , n s + M 2 , n 2 s 2 + η ( M 2 , n s 2 − 2 M 3 , n 3 s 3 ) . (121) \widetilde q_{2,n}(s,\eta):=-\frac{M_{1,n}}{s}+\frac{M_{2,n}}{2s^2}+\eta\left(\frac{M_{2,n}}{s^2}-\frac{2M_{3,n}}{3s^3}\right). \tag*{(121)} q 2 , n ( s , η ) := − s M 1 , n + 2 s 2 M 2 , n + η ( s 2 M 2 , n − 3 s 3 2 M 3 , n ) . (121) For all s > 0 s>0 s > 0 η ≥ 0 \eta\geq0 η ≥ 0
∣ q 1 , n ( s , η ) − q ~ 1 , n ( s , η ) ∣ ≤ η 2 M 3 , n 3 s 3 , (122) \left|q_{1,n}(s,\eta)-\widetilde q_{1,n}(s,\eta)\right|\leq\frac{\eta^2M_{3,n}}{3s^3}, \tag*{(122)} ∣ q 1 , n ( s , η ) − q 1 , n ( s , η ) ∣ ≤ 3 s 3 η 2 M 3 , n , (122) and
∣ q 2 , n ( s , η ) − q ~ 2 , n ( s , η ) ∣ ≤ η 2 ( M 3 , n s 3 + 3 M 4 , n 4 s 4 ) . (123) \left|q_{2,n}(s,\eta)-\widetilde q_{2,n}(s,\eta)\right|\leq\eta^2\left(\frac{M_{3,n}}{s^3}+\frac{3M_{4,n}}{4s^4}\right). \tag*{(123)} ∣ q 2 , n ( s , η ) − q 2 , n ( s , η ) ∣ ≤ η 2 ( s 3 M 3 , n + 4 s 4 3 M 4 , n ) . (123) Thus, uniformly for s s s σ \sigma σ η = O ( ξ n ) \eta=O(\xi_n) η = O ( ξ n )
∥ q ~ n ( s , η ) − q n ( s , η ) ∥ = O P n ( ξ n 2 ) . (124) \|\widetilde q_n(s,\eta)-q_n(s,\eta)\|=O_{\mathbb P_n}(\xi_n^2). \tag*{(124)} ∥ q n ( s , η ) − q n ( s , η ) ∥ = O P n ( ξ n 2 ) . (124) Proof. It suffices to consider η > 0 \eta>0 η > 0 η = 0 \eta=0 η = 0 x ≥ 0 x\geq0 x ≥ 0 r 1 ( x ) : = log ( 1 + x ) − x + x 2 / 2 r_1(x):=\log(1+x)-x+x^2/2 r 1 ( x ) := log ( 1 + x ) − x + x 2 /2
r 1 ( 0 ) = 0 and r 1 ′ ( x ) = x 2 1 + x and ( x 3 3 − r 1 ( x ) ) ′ = x 3 1 + x , (125) r_1(0)=0
\quad\text{and}\quad
r_1'(x)=\frac{x^2}{1+x}
\quad\text{and}\quad
\left(\frac{x^3}{3}-r_1(x)\right)'=\frac{x^3}{1+x}, \tag*{(125)} r 1 ( 0 ) = 0 and r 1 ′ ( x ) = 1 + x x 2 and ( 3 x 3 − r 1 ( x ) ) ′ = 1 + x x 3 , (125) both derivatives in (D.19) are nonnegative. Since both r 1 ( x ) r_1(x) r 1 ( x ) x 3 / 3 − r 1 ( x ) x^3/3-r_1(x) x 3 /3 − r 1 ( x ) x = 0 x=0 x = 0
0 ≤ log ( 1 + x ) − x + x 2 2 ≤ x 3 3 . (126) 0\leq\log(1+x)-x+\frac{x^2}{2}\leq\frac{x^3}{3}. \tag*{(126)} 0 ≤ log ( 1 + x ) − x + 2 x 2 ≤ 3 x 3 . (126) Applying (D.20) with x = η t i ( s ) x=\eta t_i(s) x = η t i ( s ) η \eta η
0 ≤ q 1 , n ( s , η ) − q ~ 1 , n ( s , η ) ≤ η 2 3 n ∑ i = 1 n t i ( s ) 3 = η 2 M 3 , n 3 s 3 , 0\leq q_{1,n}(s,\eta)-\widetilde q_{1,n}(s,\eta)\leq\frac{\eta^2}{3n}\sum_{i=1}^{n}t_i(s)^3=\frac{\eta^2M_{3,n}}{3s^3}, 0 ≤ q 1 , n ( s , η ) − q 1 , n ( s , η ) ≤ 3 n η 2 i = 1 ∑ n t i ( s ) 3 = 3 s 3 η 2 M 3 , n , which is (D.16).
For the second component, let
h ( x ) : = log ( 1 + x ) − x 1 + x and r 2 ( x ) : = h ( x ) − x 2 2 + 2 x 3 3 . h(x):=\log(1+x)-\frac{x}{1+x}
\quad\text{and}\quad
r_2(x):=h(x)-\frac{x^2}{2}+\frac{2x^3}{3}. h ( x ) := log ( 1 + x ) − 1 + x x and r 2 ( x ) := h ( x ) − 2 x 2 + 3 2 x 3 . The identities
r 2 ( 0 ) = 0 and r 2 ′ ( x ) = x 3 ( 3 + 2 x ) ( 1 + x ) 2 and ( 3 x 4 4 − r 2 ( x ) ) ′ = x 4 ( 4 + 3 x ) ( 1 + x ) 2 (127) r_2(0)=0 \quad\text{and} \quad r_2'(x)=\frac{x^3(3+2x)}{(1+x)^2} \quad\text{and} \quad\left(\frac{3x^4}{4}-r_2(x)\right)'=\frac{x^4(4+3x)}{(1+x)^2} \tag*{(127)} r 2 ( 0 ) = 0 and r 2 ′ ( x ) = ( 1 + x ) 2 x 3 ( 3 + 2 x ) and ( 4 3 x 4 − r 2 ( x ) ) ′ = ( 1 + x ) 2 x 4 ( 4 + 3 x ) (127) show that
h ( x ) = x 2 2 − 2 x 3 3 + r 2 ( x ) and 0 ≤ r 2 ( x ) ≤ 3 x 4 4 . (128) h(x)=\frac{x^2}{2}-\frac{2x^3}{3}+r_2(x) \quad\text{and} \quad0\leq r_2(x)\leq\frac{3x^4}{4}. \tag*{(128)} h ( x ) = 2 x 2 − 3 2 x 3 + r 2 ( x ) and 0 ≤ r 2 ( x ) ≤ 4 3 x 4 . (128) Moreover,
B n − A n η = 1 n ∑ i = 1 n [ h ( η t i ( s ) ) η 2 − t i ( s ) 1 + η t i ( s ) ] (129) \frac{B_n-A_n}{\eta}=\frac{1}{n}\sum_{i=1}^{n}\left[\frac{h(\eta t_i(s))}{\eta^2}-\frac{t_i(s)}{1+\eta t_i(s)}\right] \tag*{(129)} η B n − A n = n 1 i = 1 ∑ n [ η 2 h ( η t i ( s )) − 1 + η t i ( s ) t i ( s ) ] (129) and
− t 1 + η t = − t + η t 2 − η 2 t 3 1 + η t (130) -\frac{t}{1+\eta t}=-t+\eta t^2-\frac{\eta^2t^3}{1+\eta t} \tag*{(130)} − 1 + η t t = − t + η t 2 − 1 + η t η 2 t 3 (130) hold exactly. Substituting (D.22) and (D.24) into (D.23) gives
q 2 , n ( s , η ) − q ~ 2 , n ( s , η ) = 1 n ∑ i = 1 n [ r 2 ( η t i ( s ) ) η 2 − η 2 t i ( s ) 3 1 + η t i ( s ) ] . (131) q_{2,n}(s,\eta)-\widetilde{q}_{2,n}(s,\eta)=\frac{1}{n}\sum_{i=1}^{n}\left[\frac{r_2(\eta t_i(s))}{\eta^2}-\frac{\eta^2t_i(s)^3}{1+\eta t_i(s)}\right]. \tag*{(131)} q 2 , n ( s , η ) − q 2 , n ( s , η ) = n 1 i = 1 ∑ n [ η 2 r 2 ( η t i ( s )) − 1 + η t i ( s ) η 2 t i ( s ) 3 ] . (131) Using (D.22) in (D.25), together with 1 + η t i ( s ) ≥ 1 1+\eta t_i(s)\geq1 1 + η t i ( s ) ≥ 1
∣ q 2 , n ( s , η ) − q ~ 2 , n ( s , η ) ∣ ≤ η 2 ( M 3 , n s 3 + 3 M 4 , n 4 s 4 ) , \left|q_{2,n}(s,\eta)-\widetilde{q}_{2,n}(s,\eta)\right|\leq\eta^2\left(\frac{M_{3,n}}{s^3}+\frac{3M_{4,n}}{4s^4}\right), ∣ q 2 , n ( s , η ) − q 2 , n ( s , η ) ∣ ≤ η 2 ( s 3 M 3 , n + 4 s 4 3 M 4 , n ) , which proves (D.17). Finally, Lemma D.1 shows that M 3 , n M_{3,n} M 3 , n M 4 , n M_{4,n} M 4 , n 0 < ε < σ < M < ∞ 0<\varepsilon<\sigma<M<\infty 0 < ε < σ < M < ∞ [ ε , M ] [\varepsilon,M] [ ε , M ] s − 1 s^{-1} s − 1 η = O ( ξ n ) \eta=O(\xi_n) η = O ( ξ n )
Explicit Surrogate Root and Population Discrepancy
Lemma D.3. Let
C n : = 6 M 1 , n M 2 , n + 8 M 3 , n and E n : = 8 M 1 , n M 3 , n − 3 M 2 , n 2 . C_n:=6M_{1,n}M_{2,n}+8M_{3,n} \quad\text{and} \quad E_n:=8M_{1,n}M_{3,n}-3M_{2,n}^2. C n := 6 M 1 , n M 2 , n + 8 M 3 , n and E n := 8 M 1 , n M 3 , n − 3 M 2 , n 2 . With probability tending to one, the root of q ~ n ( s , η ) = 0 \widetilde{q}_n(s,\eta)=0 q n ( s , η ) = 0 σ \sigma σ
σ ~ n : = C n − C n 2 − 48 M 2 , n E n 24 M 2 , n and ξ ~ n : = 2 σ ~ n ( M 1 , n − σ ~ n ) M 2 , n . (132) \widetilde{\sigma}_n:=\frac{C_n-\sqrt{C_n^2-48M_{2,n}E_n}}{24M_{2,n}}
\quad\text{and} \quad
\widetilde{\xi}_n:=\frac{2\widetilde{\sigma}_n(M_{1,n}-\widetilde{\sigma}_n)}{M_{2,n}}. \tag*{(132)} σ n := 24 M 2 , n C n − C n 2 − 48 M 2 , n E n and ξ n := M 2 , n 2 σ n ( M 1 , n − σ n ) . (132) If n ξ n → ∞ \sqrt{n}\xi_n\to\infty n ξ n → ∞ ( 0 , ∞ ) × ( 0 , ξ ‾ ) (0,\infty)\times(0,\overline{\xi}) ( 0 , ∞ ) × ( 0 , ξ )
σ ~ n − σ = O P n ( n − 1 / 2 + ξ n 2 ) and ξ ~ n − ξ n = O P n ( n − 1 / 2 + ξ n 2 ) . (133) \widetilde{\sigma}_n-\sigma=O_{\mathbb{P}_n}(n^{-1/2}+\xi_n^2)
\quad\text{and} \quad
\widetilde{\xi}_n-\xi_n=O_{\mathbb{P}_n}(n^{-1/2}+\xi_n^2). \tag*{(133)} σ n − σ = O P n ( n − 1/2 + ξ n 2 ) and ξ n − ξ n = O P n ( n − 1/2 + ξ n 2 ) . (133) Replacing M k , n M_{k,n} M k , n m k ( ξ ) m_k(\xi) m k ( ξ ) ( σ ∗ ( ξ ) , ξ ∗ ( ξ ) ) (\sigma_*(\xi),\xi_*(\xi)) ( σ ∗ ( ξ ) , ξ ∗ ( ξ ))
σ ∗ ( ξ ) = σ ( 1 + 8 ξ 2 − 68 ξ 3 + O ( ξ 4 ) ) and ξ ∗ ( ξ ) = ξ − 10 ξ 2 + 100 ξ 3 + O ( ξ 4 ) . (134) \sigma_*(\xi)=\sigma(1+8\xi^2-68\xi^3+O(\xi^4))
\quad\text{and} \quad
\xi_*(\xi)=\xi-10\xi^2+100\xi^3+O(\xi^4). \tag*{(134)} σ ∗ ( ξ ) = σ ( 1 + 8 ξ 2 − 68 ξ 3 + O ( ξ 4 )) and ξ ∗ ( ξ ) = ξ − 10 ξ 2 + 100 ξ 3 + O ( ξ 4 ) . (134) Proof. Setting q ~ 1 , n ( s , η ) = 0 \widetilde{q}_{1,n}(s,\eta)=0 q 1 , n ( s , η ) = 0
η = 2 s ( M 1 , n − s ) M 2 , n . (135) \eta=\frac{2s(M_{1,n}-s)}{M_{2,n}}. \tag*{(135)} η = M 2 , n 2 s ( M 1 , n − s ) . (135) Substituting (D.29) into q ~ 2 , n ( s , η ) = 0 \widetilde{q}_{2,n}(s,\eta)=0 q 2 , n ( s , η ) = 0 6 M 2 , n s 2 6M_{2,n}s^2 6 M 2 , n s 2
12 M 2 , n s 2 − ( 6 M 1 , n M 2 , n + 8 M 3 , n ) s + ( 8 M 1 , n M 3 , n − 3 M 2 , n 2 ) = 0 , (136) 12M_{2,n}s^2-(6M_{1,n}M_{2,n}+8M_{3,n})s+(8M_{1,n}M_{3,n}-3M_{2,n}^2)=0, \tag*{(136)} 12 M 2 , n s 2 − ( 6 M 1 , n M 2 , n + 8 M 3 , n ) s + ( 8 M 1 , n M 3 , n − 3 M 2 , n 2 ) = 0 , (136) so the quadratic formula gives two candidate scale coordinates, where the corresponding shape coordinates are determined by (D.29). At ( M 1 , M 2 , M 3 ) = ( σ , 2 σ 2 , 6 σ 3 ) (M_1,M_2,M_3)=(\sigma,2\sigma^2,6\sigma^3) ( M 1 , M 2 , M 3 ) = ( σ , 2 σ 2 , 6 σ 3 ) σ \sigma σ 3 σ / 2 3\sigma/2 3 σ /2 144 σ 6 > 0 144\sigma^6>0 144 σ 6 > 0 ( σ , 0 ) (\sigma,0) ( σ , 0 )
For the population branch, substitute
m k ( ξ ) = k ! σ k ∏ j = 1 k ( 1 − j ξ ) m_k(\xi)=\frac{k!\sigma^k}{\prod_{j=1}^{k}(1-j\xi)} m k ( ξ ) = ∏ j = 1 k ( 1 − j ξ ) k ! σ k into (D.26). The resulting series are
C ( ξ ) σ 3 = 60 + 336 ξ + 1332 ξ 2 + 4632 ξ 3 + O ( ξ 4 ) , E ( ξ ) σ 4 = 36 + 264 ξ + 1260 ξ 2 + 4992 ξ 3 + O ( ξ 4 ) , ( C ( ξ ) 2 − 48 m 2 ( ξ ) E ( ξ ) ) 1 / 2 σ 3 = 12 + 192 ξ + 612 ξ 2 + 6024 ξ 3 + O ( ξ 4 ) . (137) \begin{aligned}
\frac{C(\xi)}{\sigma^3} &= 60+336\xi+1332\xi^2+4632\xi^3+O(\xi^4),\\
\frac{E(\xi)}{\sigma^4} &= 36+264\xi+1260\xi^2+4992\xi^3+O(\xi^4),\\
\frac{\left(C(\xi)^2-48m_2(\xi)E(\xi)\right)^{1/2}}{\sigma^3} &= 12+192\xi+612\xi^2+6024\xi^3+O(\xi^4). \tag*{(137)}
\end{aligned} σ 3 C ( ξ ) σ 4 E ( ξ ) σ 3 ( C ( ξ ) 2 − 48 m 2 ( ξ ) E ( ξ ) ) 1/2 = 60 + 336 ξ + 1332 ξ 2 + 4632 ξ 3 + O ( ξ 4 ) , = 36 + 264 ξ + 1260 ξ 2 + 4992 ξ 3 + O ( ξ 4 ) , = 12 + 192 ξ + 612 ξ 2 + 6024 ξ 3 + O ( ξ 4 ) . (137) Substitution of (D.31) into (D.26) gives (D.28).
Let m n : = ( M 1 , n , M 2 , n , M 3 , n ) T m_n:=(M_{1,n},M_{2,n},M_{3,n})^{\mathsf T} m n := ( M 1 , n , M 2 , n , M 3 , n ) T m ( ξ n ) : = ( m 1 ( ξ n ) , m 2 ( ξ n ) , m 3 ( ξ n ) ) T m(\xi_n):=(m_1(\xi_n),m_2(\xi_n),m_3(\xi_n))^{\mathsf T} m ( ξ n ) := ( m 1 ( ξ n ) , m 2 ( ξ n ) , m 3 ( ξ n ) ) T
m n − m ( ξ n ) = O P n ( 1 n ) . m_n-m(\xi_n)=O_{\mathbb{P}_n}\left(\frac{1}{\sqrt{n}}\right). m n − m ( ξ n ) = O P n ( n 1 ) . The discriminant in (D.26) converges in probability to its value 144 σ 6 > 0 144\sigma^6>0 144 σ 6 > 0
( σ ~ n ξ ~ n ) − ( σ ∗ ( ξ n ) ξ ∗ ( ξ n ) ) = O P n ( 1 n ) . \begin{pmatrix}
\widetilde{\sigma}_n\\
\widetilde{\xi}_n
\end{pmatrix}
-
\begin{pmatrix}
\sigma_*(\xi_n)\\
\xi_*(\xi_n)
\end{pmatrix}
=O_{\mathbb{P}_n}\left(\frac{1}{\sqrt{n}}\right). ( σ n ξ n ) − ( σ ∗ ( ξ n ) ξ ∗ ( ξ n ) ) = O P n ( n 1 ) . Together with (D.28), this proves (D.27). Under n ξ n → ∞ \sqrt{n}\xi_n\to\infty n ξ n → ∞
ξ ~ n ξ n = ξ ∗ ( ξ n ) ξ n + O P n ( 1 n ξ n ) → P n 1 , \frac{\widetilde{\xi}_n}{\xi_n}
=
\frac{\xi_*(\xi_n)}{\xi_n}
+
O_{\mathbb{P}_n}\left(\frac{1}{\sqrt{n}\xi_n}\right)
\xrightarrow{\mathbb{P}_n}1, ξ n ξ n = ξ n ξ ∗ ( ξ n ) + O P n ( n ξ n 1 ) P n 1 , where the deterministic term converges to one by (D.28). Because ξ n > 0 \xi_n>0 ξ n > 0 ξ n → 0 \xi_n\to0 ξ n → 0 ξ ˉ > 0 \bar{\xi}>0 ξ ˉ > 0 ξ ~ n ∈ ( 0 , ξ ˉ ) \widetilde{\xi}_n\in(0,\bar{\xi}) ξ n ∈ ( 0 , ξ ˉ )
σ ~ n = σ ∗ ( ξ n ) + O P n ( n − 1 / 2 ) = σ + o P n ( 1 ) , \widetilde{\sigma}_n=\sigma_*(\xi_n)+O_{\mathbb{P}_n}(n^{-1/2})=\sigma+o_{\mathbb{P}_n}(1), σ n = σ ∗ ( ξ n ) + O P n ( n − 1/2 ) = σ + o P n ( 1 ) , so σ ~ n > 0 \widetilde{\sigma}_n>0 σ n > 0
Uniform Root Stability
For the root comparison, use the chart
ϕ ( s , η ) : = ( η / s s η / s 2 ) and U : = { ( u 2 , u 3 ) : u 2 , u 3 > 0 , u 2 2 u 3 < ξ ˉ } . \phi(s,\eta):=
\begin{pmatrix}
\eta/s\\
s\\
\eta/s^2
\end{pmatrix}
\qquad\text{and}\qquad
U:=\left\{(u_2,u_3):u_2,u_3>0,\ \frac{u_2^2}{u_3}<\bar{\xi}\right\}. ϕ ( s , η ) := η / s s η / s 2 and U := { ( u 2 , u 3 ) : u 2 , u 3 > 0 , u 3 u 2 2 < ξ ˉ } . This map is one-to-one from ( 0 , ∞ ) × ( 0 , ξ ˉ ) (0,\infty)\times(0,\bar{\xi}) ( 0 , ∞ ) × ( 0 , ξ ˉ ) U U U
F ( u 2 , u 3 ) : = ϕ − 1 ( u 2 , u 3 ) = ( u 2 / u 3 u 2 2 / u 3 ) . (138) F(u_2,u_3):=\phi^{-1}(u_2,u_3)=
\begin{pmatrix}
u_2/u_3\\
u_2^2/u_3
\end{pmatrix}. \tag*{(138)} F ( u 2 , u 3 ) := ϕ − 1 ( u 2 , u 3 ) = ( u 2 / u 3 u 2 2 / u 3 ) . (138) At ( σ , ξ n ) (\sigma,\xi_n) ( σ , ξ n )
u n : = ( ξ n σ ξ n ξ n σ 2 ) and J F ( u n ) = ( σ 2 ξ n − σ 3 ξ n 2 σ − σ 2 ) . (139) \boldsymbol{u}_n := \begin{pmatrix} \frac{\xi_n}{\sigma} \\ \xi_n \\ \frac{\xi_n}{\sigma^2} \end{pmatrix}
\quad\text{and}\quad
\mathbf{J}_{F}(\boldsymbol{u}_n) = \begin{pmatrix} \frac{\sigma^2}{\xi_n} - \frac{\sigma^3}{\xi_n} \\ 2\sigma- \sigma^2 \end{pmatrix}. \tag*{(139)} u n := σ ξ n ξ n σ 2 ξ n and J F ( u n ) = ( ξ n σ 2 − ξ n σ 3 2 σ − σ 2 ) . (139) Proposition D.2. Assume n ξ n → ∞ \sqrt{n}\xi_n \to\infty n ξ n → ∞ ( σ ^ n , ξ ^ n ) (\widehat{\sigma}_n,\widehat{\xi}_n) ( σ n , ξ n )
∥ ( σ ~ n ξ ~ n ) − ( σ ^ n ξ ^ n ) ∥ = O P n ( ξ n 2 ) = o P n ( 1 ) . (140) \left\Vert
\begin{pmatrix}
\widetilde{\sigma}_n \\
\widetilde{\xi}_n
\end{pmatrix}
-
\begin{pmatrix}
\widehat{\sigma}_n \\
\widehat{\xi}_n
\end{pmatrix}
\right\Vert
= O_{\mathbb{P}_n}(\xi_n^2) = o_{\mathbb{P}_n}(1). \tag*{(140)} ( σ n ξ n ) − ( σ n ξ n ) = O P n ( ξ n 2 ) = o P n ( 1 ) . (140) If additionally n ξ n 2 → 0 \sqrt{n}\xi_n^2 \to0 n ξ n 2 → 0
n ( σ ~ n − σ ^ n ξ ~ n − ξ ^ n ) = O P n ( 1 ) . (141) \sqrt{n}
\begin{pmatrix}
\widetilde{\sigma}_n-\widehat{\sigma}_n \\
\widetilde{\xi}_n-\widehat{\xi}_n
\end{pmatrix}
= O_{\mathbb{P}_n}(1). \tag*{(141)} n ( σ n − σ n ξ n − ξ n ) = O P n ( 1 ) . (141) Proof. Set c n : = n − 1 / 2 + ξ n 2 c_n := n^{-1/2}+\xi_n^2 c n := n − 1/2 + ξ n 2 ( σ , ξ n ) T (\sigma,\xi_n)^\mathsf{T} ( σ , ξ n ) T O P n ( c n ) O_{\mathbb{P}_n}(c_n) O P n ( c n ) n ξ n → ∞ \sqrt{n}\xi_n \to\infty n ξ n → ∞ ξ n → 0 \xi_n \to0 ξ n → 0
c n ξ n = 1 n ξ n + ξ n → 0. \frac{c_n}{\xi_n} = \frac{1}{\sqrt{n}\xi_n}+\xi_n \to0. ξ n c n = n ξ n 1 + ξ n → 0. Consequently, uniformly along either line segment joining these vectors to ( σ , ξ n ) T (\sigma,\xi_n)^\mathsf{T} ( σ , ξ n ) T
s − σ = O P n ( c n ) and η − ξ n = O P n ( c n ) and s σ = 1 + O P n ( c n ) and η ξ n = 1 + O P n ( c n ξ n ) . s-\sigma= O_{\mathbb{P}_n}(c_n)
\quad\text{and}\quad
\eta-\xi_n = O_{\mathbb{P}_n}(c_n)
\quad\text{and}\quad
\frac{s}{\sigma}=1+O_{\mathbb{P}_n}(c_n)
\quad\text{and}\quad
\frac{\eta}{\xi_n}=1+O_{\mathbb{P}_n}\left(\frac{c_n}{\xi_n}\right). s − σ = O P n ( c n ) and η − ξ n = O P n ( c n ) and σ s = 1 + O P n ( c n ) and ξ n η = 1 + O P n ( ξ n c n ) . In particular, s s s
J F ( u n ) J ϕ ( s , η ) = ( σ 2 η ( 2 σ − s ) ξ n s 3 σ 2 ( s − σ ) ξ n s 2 2 σ η ( σ − s ) s 3 2 σ s − σ 2 s 2 ) = I 2 + O P n ( 1 ) . \mathbf{J}_{F}(\boldsymbol{u}_n)\mathbf{J}_{\phi}(s,\eta)
=
\begin{pmatrix}
\frac{\sigma^2\eta(2\sigma-s)}{\xi_n s^3} & \frac{\sigma^2(s-\sigma)}{\xi_n s^2} \\
\frac{2\sigma\eta(\sigma-s)}{s^3} & \frac{2\sigma}{s}-\frac{\sigma^2}{s^2}
\end{pmatrix}
= \mathbf{I}_2 + O_{\mathbb{P}_n}(1). J F ( u n ) J ϕ ( s , η ) = ( ξ n s 3 σ 2 η ( 2 σ − s ) s 3 2 σ η ( σ − s ) ξ n s 2 σ 2 ( s − σ ) s 2 σ − s 2 σ 2 ) = I 2 + O P n ( 1 ) . Indeed, the four entries on the right are, respectively,
1 + O P n ( c n ξ n ) , O P n ( c n ξ n ) , O P n ( ξ n c n ) , 1 + O P n ( c n 2 ) , 1+O_{\mathbb{P}_n}\left(\frac{c_n}{\xi_n}\right),
\quad
O_{\mathbb{P}_n}\left(\frac{c_n}{\xi_n}\right),
\quad
O_{\mathbb{P}_n}(\xi_n c_n),
\quad
1+O_{\mathbb{P}_n}(c_n^2), 1 + O P n ( ξ n c n ) , O P n ( ξ n c n ) , O P n ( ξ n c n ) , 1 + O P n ( c n 2 ) , uniformly along either segment. For
θ n † ∈ { ( σ ^ n ξ ^ n ) , ( σ ~ n ξ ~ n ) } and u n † : = ϕ ( θ n † ) , \theta_n^\dagger\in\left\{
\begin{pmatrix}\widehat{\sigma}_n\\\widehat{\xi}_n\end{pmatrix},
\begin{pmatrix}\widetilde{\sigma}_n\\\widetilde{\xi}_n\end{pmatrix}
\right\}
\quad\text{and}\quad
\boldsymbol{u}_n^\dagger:= \phi(\theta_n^\dagger), θ n † ∈ { ( σ n ξ n ) , ( σ n ξ n ) } and u n † := ϕ ( θ n † ) , the fundamental theorem of calculus gives
J F ( u n ) ( u n † − u n ) = ( ∫ 0 1 J F ( u n ) J ϕ ( ( σ , ξ n ) T + t [ θ n † − ( σ , ξ n ) T ] ) d t ) ( θ n † − ( σ , ξ n ) T ) = { I 2 + O P n ( 1 ) } O P n ( c n ) = O P n ( c n ) . \begin{aligned}
\mathbf{J}_{F}(\boldsymbol{u}_n)(\boldsymbol{u}_n^\dagger-\boldsymbol{u}_n)
=
\left(
\int_0^1 \mathbf{J}_{F}(\boldsymbol{u}_n)\mathbf{J}_{\phi}\left((\sigma,\xi_n)^\mathsf{T}
+t[\theta_n^\dagger-(\sigma,\xi_n)^\mathsf{T}]\right)\,\mathrm{d}t
\right)
(\theta_n^\dagger-(\sigma,\xi_n)^\mathsf{T}) \\
&= \{\mathbf{I}_2+O_{\mathbb{P}_n}(1)\}O_{\mathbb{P}_n}(c_n)
= O_{\mathbb{P}_n}(c_n).
\end{aligned} J F ( u n ) ( u n † − u n ) = ( ∫ 0 1 J F ( u n ) J ϕ ( ( σ , ξ n ) T + t [ θ n † − ( σ , ξ n ) T ] ) d t ) ( θ n † − ( σ , ξ n ) T ) = { I 2 + O P n ( 1 )} O P n ( c n ) = O P n ( c n ) . Thus
J F ( u n ) ( u ^ n − u n ) = O P n ( c n ) and J F ( u n ) ( u ~ n − u n ) = O P n ( c n ) . (142) \mathbf{J}_{F}(\boldsymbol{u}_n)(\widehat{\boldsymbol{u}}_n-\boldsymbol{u}_n)
=O_{\mathbb{P}_n}(c_n)
\quad\text{and}\quad
\mathbf{J}_{F}(\boldsymbol{u}_n)(\widetilde{\boldsymbol{u}}_n-\boldsymbol{u}_n)
=O_{\mathbb{P}_n}(c_n). \tag*{(142)} J F ( u n ) ( u n − u n ) = O P n ( c n ) and J F ( u n ) ( u n − u n ) = O P n ( c n ) . (142) Choose a deterministic sequence t n → ∞ t_n\to\infty t n → ∞ t n c n = o ( ξ n ) t_nc_n=o(\xi_n) t n c n = o ( ξ n )
D n : = { u : ∥ J F ( u n ) ( u − u n ) ∥ ≤ t n c n } (143) D_n := \{u:\|\mathbf{J}_{F}(\boldsymbol{u}_n)(u-\boldsymbol{u}_n)\|\le t_nc_n\} \tag*{(143)} D n := { u : ∥ J F ( u n ) ( u − u n ) ∥ ≤ t n c n } (143) then contain both transformed roots with probability tending to one by (D.36).
We next verify that D n ⊆ U D_n \subseteq U D n ⊆ U u ∈ D n u \in D_n u ∈ D n
v : = J F ( u n ) ( u − u n ) = ( v 1 , v 2 ) T and r : = u 2 u 2 , n and q : = u 3 u 3 , n . v := \mathbf{J}_{F}(u_n)(u-u_n)=(v_1,v_2)^{\mathsf{T}} \quad\text{and}\quad r:=\frac{u_2}{u_{2,n}} \quad\text{and}\quad q:=\frac{u_3}{u_{3,n}}. v := J F ( u n ) ( u − u n ) = ( v 1 , v 2 ) T and r := u 2 , n u 2 and q := u 3 , n u 3 . Since J F ( u n ) − 1 = J ϕ ( σ , ξ n ) \mathbf{J}_{F}(u_n)^{-1}=\mathbf{J}_{\phi}(\sigma,\xi_n) J F ( u n ) − 1 = J ϕ ( σ , ξ n )
r = 1 − v 1 σ + v 2 ξ n and q = 1 − 2 v 1 σ + v 2 ξ n and q − r = − v 1 σ . (144) r=1-\frac{v_1}{\sigma}+\frac{v_2}{\xi_n}\quad\text{and}\quad q=1-\frac{2v_1}{\sigma}+\frac{v_2}{\xi_n}\quad\text{and}\quad q-r=-\frac{v_1}{\sigma}. \tag*{(144)} r = 1 − σ v 1 + ξ n v 2 and q = 1 − σ 2 v 1 + ξ n v 2 and q − r = − σ v 1 . (144) Thus r = 1 + o ( 1 ) r=1+o(1) r = 1 + o ( 1 ) q = 1 + o ( 1 ) q=1+o(1) q = 1 + o ( 1 ) q − r = o ( ξ n ) q-r=o(\xi_n) q − r = o ( ξ n ) D n D_n D n
F ( u ) = ( σ r q ξ n r 2 q ) = ( σ + o ( 1 ) ξ n ( 1 + o ( 1 ) ) ) . (145) F(u)=
\begin{pmatrix}
\frac{\sigma r}{q}\\
\xi_n\frac{r^2}{q}
\end{pmatrix}
=
\begin{pmatrix}
\sigma+o(1)\\
\xi_n(1+o(1))
\end{pmatrix}. \tag*{(145)} F ( u ) = ( q σ r ξ n q r 2 ) = ( σ + o ( 1 ) ξ n ( 1 + o ( 1 )) ) . (145) uniformly on D n D_n D n D n ⊆ U D_n\subseteq U D n ⊆ U
J F ( u ) J F ( u n ) − 1 = ( − 1 q + 2 r q 2 q − r u 2 , n q 2 2 u 2 , n r r − q q 2 2 r q − r 2 q 2 ) = I 2 + o ( 1 ) . (146) \mathbf{J}_{F}(u)\mathbf{J}_{F}(u_n)^{-1}
=
\begin{pmatrix}
-\frac{1}{q}+\frac{2r}{q^2} & \frac{q-r}{u_{2,n}q^2}\\
2u_{2,n}r\frac{r-q}{q^2} & \frac{2r}{q}-\frac{r^2}{q^2}
\end{pmatrix}
=\mathbf{I}_2+o(1). \tag*{(146)} J F ( u ) J F ( u n ) − 1 = ( − q 1 + q 2 2 r 2 u 2 , n r q 2 r − q u 2 , n q 2 q − r q 2 r − q 2 r 2 ) = I 2 + o ( 1 ) . (146) uniformly on D n D_n D n Q n : = q n ∘ F Q_n:=q_n\circ F Q n := q n ∘ F Q ~ n : = q ~ n ∘ F \widetilde{Q}_n:=\widetilde{q}_n\circ F Q n := q n ∘ F
H n ( s , η ) : = J q ~ n ( s , η ) = ( − M 1 , n s 2 + η M 2 , n s 3 − M 2 , n 2 s 2 M 1 , n s 2 − M 2 , n s 3 + η ( − 2 M 2 , n s 3 + 2 M 3 , n s 4 ) M 2 , n s 2 − 2 M 3 , n 3 s 3 ) . \mathbf{H}_n(s,\eta):=\mathbf{J}_{\widetilde{q}_n}(s,\eta)=
\begin{pmatrix}
-\frac{M_{1,n}}{s^2}+\eta\frac{M_{2,n}}{s^3} & -\frac{M_{2,n}}{2s^2}\\
\frac{M_{1,n}}{s^2}-\frac{M_{2,n}}{s^3}+\eta\left(-\frac{2M_{2,n}}{s^3}+\frac{2M_{3,n}}{s^4}\right) & \frac{M_{2,n}}{s^2}-\frac{2M_{3,n}}{3s^3}
\end{pmatrix}. H n ( s , η ) := J q n ( s , η ) = ( − s 2 M 1 , n + η s 3 M 2 , n s 2 M 1 , n − s 3 M 2 , n + η ( − s 3 2 M 2 , n + s 4 2 M 3 , n ) − 2 s 2 M 2 , n s 2 M 2 , n − 3 s 3 2 M 3 , n ) . The sample moment limits in (D.3) give
H n ( σ , ξ n ) → P n G 0 : = ( − 1 σ − 1 1 σ − 2 ) , (147) \mathbf{H}_n(\sigma,\xi_n)\xrightarrow{\mathbb{P}_n}\mathbf{G}_0:=
\begin{pmatrix}
-\frac{1}{\sigma} & -1\\
\frac{1}{\sigma} & -2
\end{pmatrix}, \tag*{(147)} H n ( σ , ξ n ) P n G 0 := ( − σ 1 σ 1 − 1 − 2 ) , (147) whose determinant is 1 / σ 1/\sigma 1/ σ
J n : = H n ( σ , ξ n ) J F ( u n ) . \mathbf{J}_n:=\mathbf{H}_n(\sigma,\xi_n)\mathbf{J}_{F}(u_n). J n := H n ( σ , ξ n ) J F ( u n ) . Equations (D.3) and (D.39), together with the displayed formula for H n \mathbf{H}_n H n
sup u ∈ D n ∥ H n ( F ( u ) ) − H n ( σ , ξ n ) ∥ = o P n ( 1 ) and sup u ∈ D n ∥ J F ( u ) J F ( u n ) − 1 − I 2 ∥ = o ( 1 ) . (148) \sup_{u\in D_n}\left\|\mathbf{H}_n(F(u))-\mathbf{H}_n(\sigma,\xi_n)\right\|=o_{\mathbb{P}_n}(1)
\quad\text{and}\quad
\sup_{u\in D_n}\left\|\mathbf{J}_{F}(u)\mathbf{J}_{F}(u_n)^{-1}-\mathbf{I}_2\right\|=o(1). \tag*{(148)} u ∈ D n sup ∥ H n ( F ( u )) − H n ( σ , ξ n ) ∥ = o P n ( 1 ) and u ∈ D n sup J F ( u ) J F ( u n ) − 1 − I 2 = o ( 1 ) . (148) In particular,
J Q ~ n ( u ) = H n ( F ( u ) ) J F ( u ) and J F ( u n ) J n − 1 = H n ( σ , ξ n ) − 1 . (149) \mathbf{J}_{\widetilde{Q}_n}(u)=\mathbf{H}_n(F(u))\mathbf{J}_{F}(u)
\quad\text{and}\quad
\mathbf{J}_{F}(u_n)\mathbf{J}_n^{-1}=\mathbf{H}_n(\sigma,\xi_n)^{-1}. \tag*{(149)} J Q n ( u ) = H n ( F ( u )) J F ( u ) and J F ( u n ) J n − 1 = H n ( σ , ξ n ) − 1 . (149) The normalized Jacobian difference required by Proposition 2 is therefore
H n ( σ , ξ n ) − 1 H n ( F ( u ) ) J F ( u ) J F ( u n ) − 1 − I 2 . \mathbf{H}_n(\sigma,\xi_n)^{-1}\mathbf{H}_n(F(u))\mathbf{J}_{F}(u)\mathbf{J}_{F}(u_n)^{-1}-\mathbf{I}_2. H n ( σ , ξ n ) − 1 H n ( F ( u )) J F ( u ) J F ( u n ) − 1 − I 2 . Combining the two uniform approximations in (D.42) with H n ( σ , ξ n ) → P n G 0 \mathbf{H}_n(\sigma,\xi_n)\xrightarrow{\mathbb{P}_n}\mathbf{G}_0 H n ( σ , ξ n ) P n G 0
sup u ∈ D n ∥ J F ( u n ) J n − 1 ( J Q ~ n ( u ) − J n ) J F ( u n ) − 1 ∥ = o P n ( 1 ) . (150) \sup_{u\in D_n}\left\|\mathbf{J}_{F}(u_n)\mathbf{J}_n^{-1}\left(\mathbf{J}_{\widetilde{Q}_n}(u)-\mathbf{J}_n\right)\mathbf{J}_{F}(u_n)^{-1}\right\|=o_{\mathbb{P}_n}(1). \tag*{(150)} u ∈ D n sup J F ( u n ) J n − 1 ( J Q n ( u ) − J n ) J F ( u n ) − 1 = o P n ( 1 ) . (150) By (D.39), every point in F ( D n ) F(D_n) F ( D n ) s s s σ \sigma σ η = O ( ξ n ) \eta=O(\xi_n) η = O ( ξ n )
sup u ∈ D n ∥ J F ( u n ) J n − 1 ( Q ~ n ( u ) − Q n ( u ) ) ∥ = sup u ∈ D n ∥ H n ( σ , ξ n ) − 1 [ q ~ n ( F ( u ) ) − q n ( F ( u ) ) ] ∥ = O P n ( ξ n 2 ) . (151) \sup_{u\in D_n}\left\|\mathbf{J}_{F}(u_n)\mathbf{J}_n^{-1}\left(\widetilde{Q}_n(u)-Q_n(u)\right)\right\|
=
\sup_{u\in D_n}\left\|\mathbf{H}_n(\sigma,\xi_n)^{-1}\left[\widetilde{q}_n(F(u))-q_n(F(u))\right]\right\|
=O_{\mathbb{P}_n}(\xi_n^2). \tag*{(151)} u ∈ D n sup J F ( u n ) J n − 1 ( Q n ( u ) − Q n ( u ) ) = u ∈ D n sup H n ( σ , ξ n ) − 1 [ q n ( F ( u )) − q n ( F ( u )) ] = O P n ( ξ n 2 ) . (151) To retain the order in (D.45), define
A n : = ∫ 0 1 J Q n [ u ^ n + t ( u ~ n − u ^ n ) ] d t \mathbf{A}_n := \int_0^1 \mathbf{J}_{Q_n}\left[\hat{u}_n+t(\tilde{u}_n-\hat{u}_n)\right]\,dt A n := ∫ 0 1 J Q n [ u ^ n + t ( u ~ n − u ^ n ) ] d t on the event that both roots belong to D n D_n D n
B n : = J F ( u n ) J n − 1 ( A n − J n ) J F ( u n ) − 1 . \mathbf{B}_n := \mathbf{J}_F(u_n)\mathbf{J}_n^{-1}(\mathbf{A}_n-\mathbf{J}_n)\mathbf{J}_F(u_n)^{-1}. B n := J F ( u n ) J n − 1 ( A n − J n ) J F ( u n ) − 1 . Equation (D.44) gives B n = o P n ( 1 ) \mathbf{B}_n=o_{P_n}(1) B n = o P n ( 1 )
J F ( u n ) ( u ~ n − u ^ n ) = − ( I 2 + B n ) − 1 J F ( u n ) J n − 1 ( Q ~ n ( u ~ n ) − Q n ( u ^ n ) ) . \mathbf{J}_F(u_n)(\tilde{u}_n-\hat{u}_n)=-(\mathbf{I}_2+\mathbf{B}_n)^{-1}\mathbf{J}_F(u_n)\mathbf{J}_n^{-1}\left(\widetilde{\mathbf{Q}}_n(\tilde{u}_n)-\mathbf{Q}_n(\hat{u}_n)\right). J F ( u n ) ( u ~ n − u ^ n ) = − ( I 2 + B n ) − 1 J F ( u n ) J n − 1 ( Q n ( u ~ n ) − Q n ( u ^ n ) ) . Together with (D.45), this proves
J F ( u n ) ( u ~ n − u ^ n ) = O P n ( ξ n 2 ) . (152) \mathbf{J}_F(u_n)(\tilde{u}_n-\hat{u}_n)=O_{P_n}(\xi_n^2). \tag*{(152)} J F ( u n ) ( u ~ n − u ^ n ) = O P n ( ξ n 2 ) . (152) To translate this bound through the inverse chart, apply the fundamental theorem of calculus once more. Equations (D.40) and (D.46) give
F ( u ~ n ) − F ( u ^ n ) = ( ∫ 0 1 J F ( u ^ n + t ( u ~ n − u ^ n ) ) J F ( u n ) − 1 d t ) J F ( u n ) ( u ~ n − u ^ n ) = ( I 2 + o P n ( 1 ) ) J F ( u n ) ( u ~ n − u ^ n ) = O P n ( ξ n 2 ) , (153) \begin{aligned}
F(\tilde{u}_n)-F(\hat{u}_n)=\left(\int_0^1\mathbf{J}_F\left(\hat{u}_n+t(\tilde{u}_n-\hat{u}_n)\right)\mathbf{J}_F(u_n)^{-1}\,dt\right)\mathbf{J}_F(u_n)(\tilde{u}_n-\hat{u}_n) \\
&=(\mathbf{I}_2+o_{P_n}(1))\mathbf{J}_F(u_n)(\tilde{u}_n-\hat{u}_n)=O_{P_n}(\xi_n^2), \tag*{(153)}
\end{aligned} F ( u ~ n ) − F ( u ^ n ) = ( ∫ 0 1 J F ( u ^ n + t ( u ~ n − u ^ n ) ) J F ( u n ) − 1 d t ) J F ( u n ) ( u ~ n − u ^ n ) = ( I 2 + o P n ( 1 )) J F ( u n ) ( u ~ n − u ^ n ) = O P n ( ξ n 2 ) , (153) which is (D.34). If n ξ n 2 → 0 \sqrt{n}\xi_n^2\to0 n ξ n 2 → 0 n \sqrt{n} n
Limiting Distributions
Lemma D.4. Let
V n , i : = X n , i − μ σ . V_{n,i}:=\frac{X_{n,i}-\mu}{\sigma}. V n , i := σ X n , i − μ . and define the score contribution for ( σ , ξ n ) (\sigma,\xi_n) ( σ , ξ n ) μ \mu μ
s ξ n ( V n , i ) : = ( V n , i − 1 σ ( 1 + ξ n V n , i ) , log ( 1 + ξ n V n , i ) ξ n 2 − ( 1 + ξ n ) V n , i ξ n ( 1 + ξ n V n , i ) ) . (154) s_{\xi_n}(V_{n,i}):=\left(\frac{V_{n,i}-1}{\sigma(1+\xi_nV_{n,i})},\frac{\log(1+\xi_nV_{n,i})}{\xi_n^2}-\frac{(1+\xi_n)V_{n,i}}{\xi_n(1+\xi_nV_{n,i})}\right). \tag*{(154)} s ξ n ( V n , i ) := ( σ ( 1 + ξ n V n , i ) V n , i − 1 , ξ n 2 log ( 1 + ξ n V n , i ) − ξ n ( 1 + ξ n V n , i ) ( 1 + ξ n ) V n , i ) . (154) Then E P n [ s ξ n ( V n , i ) ] = 0 \mathbb{E}_{P_n}[s_{\xi_n}(V_{n,i})]=0 E P n [ s ξ n ( V n , i )] = 0
Var P n ( s ξ n ( V n , i ) ) = I ( σ , ξ n ) = 1 1 + 2 ξ n ( 1 σ 2 − 1 σ ( 1 + ξ n ) − 1 σ ( 1 + ξ n ) 2 ( 1 + ξ n ) 2 ) . (155) \operatorname{Var}_{P_n}(s_{\xi_n}(V_{n,i}))=\mathcal{I}(\sigma,\xi_n)=\frac{1}{1+2\xi_n}\begin{pmatrix}\dfrac{1}{\sigma^2}&-\dfrac{1}{\sigma(1+\xi_n)}\\-\dfrac{1}{\sigma(1+\xi_n)}&\dfrac{2}{(1+\xi_n)^2}\end{pmatrix}. \tag*{(155)} Var P n ( s ξ n ( V n , i )) = I ( σ , ξ n ) = 1 + 2 ξ n 1 σ 2 1 − σ ( 1 + ξ n ) 1 − σ ( 1 + ξ n ) 1 ( 1 + ξ n ) 2 2 . (155) Moreover,
1 n I ( σ , ξ n ) − 1 / 2 ∑ i = 1 n s ξ n ( V n , i ) → d N ( 0 , I 2 ) . (156) \frac{1}{\sqrt{n}}\mathcal{I}(\sigma,\xi_n)^{-1/2}\sum_{i=1}^n s_{\xi_n}(V_{n,i})\xrightarrow{d}N(0,\mathbf{I}_2). \tag*{(156)} n 1 I ( σ , ξ n ) − 1/2 i = 1 ∑ n s ξ n ( V n , i ) d N ( 0 , I 2 ) . (156) Proof. For V = ( X − μ ) / σ V=(X-\mu)/\sigma V = ( X − μ ) / σ
W = 1 ξ log ( 1 + ξ V ) W=\frac{1}{\xi}\log(1+\xi V) W = ξ 1 log ( 1 + ξ V ) has the Exp ( 1 ) \operatorname{Exp}(1) Exp ( 1 ) s σ s_\sigma s σ s ξ s_\xi s ξ
s σ = 1 − ( 1 + ξ ) e − ξ W σ ξ and s ξ = W ξ − ( 1 + ξ ) ( 1 − e − ξ W ) ξ 2 . s_\sigma=\frac{1-(1+\xi)e^{-\xi W}}{\sigma\xi}
\quad\text{and}\quad
s_\xi=\frac{W}{\xi}-\frac{(1+\xi)(1-e^{-\xi W})}{\xi^2}. s σ = σ ξ 1 − ( 1 + ξ ) e − ξ W and s ξ = ξ W − ξ 2 ( 1 + ξ ) ( 1 − e − ξ W ) . For a > − 1 a>-1 a > − 1 k > − 1 k>-1 k > − 1
E [ e − a W ] = 1 1 + a and E [ W k e − a W ] = Γ ( k + 1 ) ( 1 + a ) k + 1 , \mathbb{E}[e^{-aW}]=\frac{1}{1+a}\quad\text{and}\quad\mathbb{E}[W^k e^{-aW}]=\frac{\Gamma(k+1)}{(1+a)^{k+1}}, E [ e − aW ] = 1 + a 1 and E [ W k e − aW ] = ( 1 + a ) k + 1 Γ ( k + 1 ) , show that both score components have mean zero and give
E [ s σ 2 ] = 1 σ 2 ( 1 + 2 ξ ) and E [ s σ s ξ ] = 1 σ ( 1 + ξ ) ( 1 + 2 ξ ) and E [ s ξ 2 ] = 2 ( 1 + ξ ) ( 1 + 2 ξ ) . \mathbb{E}[s_\sigma^2]=\frac{1}{\sigma^2(1+2\xi)}\quad\text{and}\quad\mathbb{E}[s_\sigma s_\xi]=\frac{1}{\sigma(1+\xi)(1+2\xi)}\quad\text{and}\quad\mathbb{E}[s_\xi^2]=\frac{2}{(1+\xi)(1+2\xi)}. E [ s σ 2 ] = σ 2 ( 1 + 2 ξ ) 1 and E [ s σ s ξ ] = σ ( 1 + ξ ) ( 1 + 2 ξ ) 1 and E [ s ξ 2 ] = ( 1 + ξ ) ( 1 + 2 ξ ) 2 . This proves (D.49). Direct inversion gives
I ( σ , ξ ) − 1 = ( 2 σ 2 ( 1 + ξ ) − σ ( 1 + ξ ) − σ ( 1 + ξ ) ( 1 + ξ ) 2 ) → ξ → 0 ( 2 σ 2 − σ − σ 1 ) . (157) \mathcal{I}(\sigma,\xi)^{-1}=
\begin{pmatrix}
2\sigma^2(1+\xi) & -\sigma(1+\xi) \\
-\sigma(1+\xi) & (1+\xi)^2
\end{pmatrix}
\xrightarrow{\xi\to0}
\begin{pmatrix}
2\sigma^2 & -\sigma\\
-\sigma& 1
\end{pmatrix}. \tag*{(157)} I ( σ , ξ ) − 1 = ( 2 σ 2 ( 1 + ξ ) − σ ( 1 + ξ ) − σ ( 1 + ξ ) ( 1 + ξ ) 2 ) ξ → 0 ( 2 σ 2 − σ − σ 1 ) . (157) It remains to verify the central limit theorem. For x ≥ 0 x\geq0 x ≥ 0
1 − e − x = x − x 2 ∫ 0 1 ( 1 − t ) e − t x d t . 1-e^{-x}=x-x^2\int_0^1(1-t)e^{-tx}\,dt. 1 − e − x = x − x 2 ∫ 0 1 ( 1 − t ) e − t x d t . Consequently, the two score components satisfy
∣ s σ ∣ = ∣ 1 − e − ξ W σ ξ − e − ξ W σ ∣ ≤ W + 1 σ , |s_\sigma|=\left|\frac{1-e^{-\xi W}}{\sigma\xi}-\frac{e^{-\xi W}}{\sigma}\right|\leq\frac{W+1}{\sigma}, ∣ s σ ∣ = σ ξ 1 − e − ξ W − σ e − ξ W ≤ σ W + 1 , ∣ s ξ ∣ = ∣ − W + ( 1 + ξ ) W 2 ∫ 0 1 ( 1 − t ) e − t ξ W d t ∣ ≤ W + 1 + ξ ˉ 2 W 2 . (158) |s_\xi|=\left|-W+(1+\xi)W^2\int_0^1(1-t)e^{-t\xi W}\,dt\right|\leq W+\frac{1+\bar{\xi}}{2}W^2. \tag*{(158)} ∣ s ξ ∣ = − W + ( 1 + ξ ) W 2 ∫ 0 1 ( 1 − t ) e − t ξ W d t ≤ W + 2 1 + ξ ˉ W 2 . (158) The same inequalities hold at ξ = 0 \xi=0 ξ = 0 W ∼ Exp ( 1 ) W\sim\operatorname{Exp}(1) W ∼ Exp ( 1 )
sup n E P n [ ∣ s ξ n ( V n , 1 ) ∣ 3 ] < ∞ . (159) \sup_n\mathbb{E}_{P_n}\left[\lvert s_{\xi_n}(V_{n,1})\rvert^3\right]<\infty. \tag*{(159)} n sup E P n [ ∣ s ξ n ( V n , 1 ) ∣ 3 ] < ∞. (159) The matrices in (D.49) are uniformly nonsingular for all sufficiently large n n n a ∈ R 2 a\in\mathbb{R}^2 a ∈ R 2
T n , i : = a T I ( σ , ξ n ) − 1 / 2 s ξ n ( V n , i ) T_{n,i}:=a^{\mathsf T}\mathcal{I}(\sigma,\xi_n)^{-1/2}s_{\xi_n}(V_{n,i}) T n , i := a T I ( σ , ξ n ) − 1/2 s ξ n ( V n , i ) have mean zero, variance one, and uniformly bounded third absolute moments. Therefore,
∑ i = 1 n E P n [ ∣ T n , i ∣ 3 ] ( ∑ i = 1 n Var P n ( T n , i ) ) 3 / 2 = E P n [ ∣ T n , 1 ∣ 3 ] n = O ( n − 1 / 2 ) . (160) \frac{\sum_{i=1}^n\mathbb{E}_{P_n}[|T_{n,i}|^3]}{\left(\sum_{i=1}^n\operatorname{Var}_{P_n}(T_{n,i})\right)^{3/2}}
=\frac{\mathbb{E}_{P_n}[|T_{n,1}|^3]}{\sqrt{n}}=O(n^{-1/2}). \tag*{(160)} ( ∑ i = 1 n Var P n ( T n , i ) ) 3/2 ∑ i = 1 n E P n [ ∣ T n , i ∣ 3 ] = n E P n [ ∣ T n , 1 ∣ 3 ] = O ( n − 1/2 ) . (160) Lyapunov’s theorem and the Cramér–Wold device now give (D.50).
Theorem D.1. Under n ξ n → ∞ \sqrt{n}\,\xi_n\to\infty n ξ n → ∞
n ( σ ^ n − σ ξ ^ n − ξ n ) → d N ( 0 , ( 2 σ 2 − σ − σ 1 ) ) . (161) \sqrt{n}
\begin{pmatrix}
\widehat{\sigma}_n-\sigma\\
\widehat{\xi}_n-\xi_n
\end{pmatrix}
\xrightarrow{d}
\mathcal{N}\left(0,
\begin{pmatrix}
2\sigma^2 & -\sigma\\
-\sigma& 1
\end{pmatrix}\right). \tag*{(161)} n ( σ n − σ ξ n − ξ n ) d N ( 0 , ( 2 σ 2 − σ − σ 1 ) ) . (161) If also n ξ n 2 → 0 \sqrt{n}\,\xi_n^2\to0 n ξ n 2 → 0 ( σ ^ n , ξ ^ n ) (\widehat{\sigma}_n,\widehat{\xi}_n) ( σ n , ξ n ) ( σ ~ n , ξ ~ n ) (\widetilde{\sigma}_n,\widetilde{\xi}_n) ( σ n , ξ n )
μ ^ n = μ ~ n = X ( 1 ) and n σ ( μ ^ n − μ ) = n σ ( μ ~ n − μ ) → d Exp ( 1 ) . (162) \widehat{\mu}_n=\widetilde{\mu}_n=X_{(1)}\quad\text{and}\quad\frac{n}{\sigma}(\widehat{\mu}_n-\mu)=\frac{n}{\sigma}(\widetilde{\mu}_n-\mu)\xrightarrow{d}\operatorname{Exp}(1). \tag*{(162)} μ n = μ n = X ( 1 ) and σ n ( μ n − μ ) = σ n ( μ n − μ ) d Exp ( 1 ) . (162) Proof. Put
θ n : = ( σ , ξ n ) T and θ ^ n : = ( σ ^ n , ξ ^ n ) T . \theta_n:=(\sigma,\xi_n)^{\mathsf T}\quad\text{and}\quad\widehat{\theta}_n:=(\widehat{\sigma}_n,\widehat{\xi}_n)^{\mathsf T}. θ n := ( σ , ξ n ) T and θ n := ( σ n , ξ n ) T . Let S n p \mathbf{S}_n^p S n p H n p \mathbf{H}_n^p H n p
1 n S n p ( θ n ) = 1 n ∑ i = 1 n s ξ n ( V n , i ) + o P n ( 1 ) . (163) \frac{1}{\sqrt{n}}\mathbf{S}_n^p(\theta_n)=\frac{1}{\sqrt{n}}\sum_{i=1}^n s_{\xi_n}(V_{n,i})+o_{P_n}(1). \tag*{(163)} n 1 S n p ( θ n ) = n 1 i = 1 ∑ n s ξ n ( V n , i ) + o P n ( 1 ) . (163) Equations (D.9), (D.11), and (D.12), together with continuity of (D.49), imply
sup 0 ≤ t ≤ 1 ∥ − 1 n H n p ( θ n + t ( θ ^ n − θ n ) ) − I ( σ , ξ n ) ∥ = o P n ( 1 ) . (164) \sup_{0\le t\le1}\left\lVert-\frac{1}{n}H_n^{p}\left(\theta_n+t\left(\widehat{\theta}_n-\theta_n\right)\right)-\mathcal{I}(\sigma,\xi_n)\right\rVert=o_{P_n}(1). \tag*{(164)} 0 ≤ t ≤ 1 sup − n 1 H n p ( θ n + t ( θ n − θ n ) ) − I ( σ , ξ n ) = o P n ( 1 ) . (164) By Proposition D.1, θ ^ n \widehat{\theta}_n θ n
n ( θ ^ n − θ n ) = I ( σ , ξ n ) − 1 1 n ∑ i = 1 n S ξ n ( V n , i ) + o P n ( 1 ) . (165) \sqrt{n}\left(\widehat{\theta}_n-\theta_n\right)=\mathcal{I}(\sigma,\xi_n)^{-1}\frac{1}{\sqrt{n}}\sum_{i=1}^{n}\mathcal{S}_{\xi_n}(V_{n,i})+o_{P_n}(1). \tag*{(165)} n ( θ n − θ n ) = I ( σ , ξ n ) − 1 n 1 i = 1 ∑ n S ξ n ( V n , i ) + o P n ( 1 ) . (165) Equations (D.50), (D.51), and (D.59) prove (D.55). Under n ξ n 2 → 0 \sqrt{n}\xi_n^2\to0 n ξ n 2 → 0
Simulation Study
We used the same sample size grid and number of replications as in the gamma and zeta studies. We simulated from the restricted GPD with μ : = 0 \mu:=0 μ := 0 σ : = 1 \sigma:=1 σ := 1 ξ n : = 0.3 n − γ \xi_n:=0.3n^{-\gamma} ξ n := 0.3 n − γ ξ ˉ : = 0.16 \bar{\xi}:=0.16 ξ ˉ := 0.16 γ ∈ { 0.20 , 0.30 , 0.55 } \gamma\in\{0.20,0.30,0.55\} γ ∈ { 0.20 , 0.30 , 0.55 } n ∈ { 250 , 500 , 1000 , 2500 , 5000 , 10000 , 25000 } n\in\{250,500,1000,2500,5000,10000,25000\} n ∈ { 250 , 500 , 1000 , 2500 , 5000 , 10000 , 25000 } γ = 0.20 \gamma=0.20 γ = 0.20 n ξ n 2 → 0 \sqrt{n}\xi_n^2\to0 n ξ n 2 → 0 γ = 0.30 \gamma=0.30 γ = 0.30 n − 1 / 2 ≪ ξ n ≪ n − 1 / 4 n^{-1/2}\ll\xi_n\ll n^{-1/4} n − 1/2 ≪ ξ n ≪ n − 1/4 γ = 0.55 \gamma=0.55 γ = 0.55 n ξ n → ∞ \sqrt{n}\xi_n\to\infty n ξ n → ∞
As a closed-form benchmark, we used the probability weighted moment estimator of Hosking and Wallis [ [22 ] , Section 3.3. Their shape parameter k k k − ξ -\xi − ξ Y Y Y V : = 1 − F ( Y ) V:=1-F(Y) V := 1 − F ( Y ) V ∼ Unif ( 0 , 1 ) V\sim\operatorname{Unif}(0,1) V ∼ Unif ( 0 , 1 ) Y = σ ( V − ξ − 1 ) / ξ Y=\sigma(V^{-\xi}-1)/\xi Y = σ ( V − ξ − 1 ) / ξ
α r : = E [ Y ( 1 − F ( Y ) ) r ] = σ ξ ∫ 0 1 ( v − ξ − v r ) d v = σ ( r + 1 ) ( r + 1 − ξ ) . \alpha_r:=\mathbb{E}\left[Y\left(1-F(Y)\right)^r\right]=\frac{\sigma}{\xi}\int_0^1\left(v^{-\xi}-v^r\right)\,dv=\frac{\sigma}{(r+1)(r+1-\xi)}. α r := E [ Y ( 1 − F ( Y ) ) r ] = ξ σ ∫ 0 1 ( v − ξ − v r ) d v = ( r + 1 ) ( r + 1 − ξ ) σ . In particular,
α 0 = σ 1 − ξ and α 1 = σ 2 ( 2 − ξ ) . \alpha_0=\frac{\sigma}{1-\xi}\qquad\text{and}\qquad\alpha_1=\frac{\sigma}{2(2-\xi)}. α 0 = 1 − ξ σ and α 1 = 2 ( 2 − ξ ) σ . Writing Y ( 1 ) ≤ ⋯ ≤ Y ( n ) Y_{(1)}\le\cdots\le Y_{(n)} Y ( 1 ) ≤ ⋯ ≤ Y ( n ) X ( 1 ) X_{(1)} X ( 1 )
a 0 : = 1 n ∑ j = 1 n Y ( j ) and a 1 : = 1 n ∑ j = 1 n n − j n − 1 Y ( j ) . a_0:=\frac{1}{n}\sum_{j=1}^{n}Y_{(j)}\qquad\text{and}\qquad a_1:=\frac{1}{n}\sum_{j=1}^{n}\frac{n-j}{n-1}Y_{(j)}. a 0 := n 1 j = 1 ∑ n Y ( j ) and a 1 := n 1 j = 1 ∑ n n − 1 n − j Y ( j ) . Replacing ( α 0 , α 1 ) (\alpha_0,\alpha_1) ( α 0 , α 1 ) ( a 0 , a 1 ) (a_0,a_1) ( a 0 , a 1 )
ξ ^ P W M : = 2 − a 0 a 0 − 2 a 1 and σ ^ P W M : = 2 a 0 a 1 a 0 − 2 a 1 . \widehat{\xi}_{\mathrm{PWM}}:=2-\frac{a_0}{a_0-2a_1}\qquad\text{and}\qquad\widehat{\sigma}_{\mathrm{PWM}}:=\frac{2a_0a_1}{a_0-2a_1}. ξ PWM := 2 − a 0 − 2 a 1 a 0 and σ PWM := a 0 − 2 a 1 2 a 0 a 1 . Hosking and Wallis [ [22 ] derived these formulas for a two-parameter GPD with a known threshold. Applying them to excesses over X ( 1 ) X_{(1)} X ( 1 )
The exact restricted MLE was computed by comparing the interior likelihood root with the two shape boundary solutions; this calculation had no numerical failures. For our estimator, we used the selected raw surrogate root whenever it belonged to the restricted parameter space, and the shifted exponential boundary estimator ( M 1 , n , 0 ) (M_{1,n},0) ( M 1 , n , 0 )
The discrepancy patterns in Figure 10 must be interpreted together with the boundary replacement. When γ = 0.20 \gamma= 0.20 γ = 0.20 γ = 0.30 \gamma= 0.30 γ = 0.30 n = 2500 n = 2500 n = 2500 n = 250 n = 250 n = 250 n = 25000 n = 25000 n = 25000 γ = 0.55 \gamma= 0.55 γ = 0.55
Figure 10. Median absolute discrepancy between the constrained approximate and exact GPD estimators, measured in units of the limiting standard errors 2 / n \sqrt{2/n} 2/ n 1 / n 1/\sqrt{n} 1/ n
Figure 11 compares the two explicit estimators with the exact MLE. For γ = 0.20 \gamma= 0.20 γ = 0.20 γ = 0.30 \gamma= 0.30 γ = 0.30 γ = 0.55 \gamma= 0.55 γ = 0.55 n = 250 n = 250 n = 250 n = 25000 n = 25000 n = 25000 γ = 0.30 \gamma= 0.30 γ = 0.30
Figure 11. RMSE of each explicit GPD estimator relative to the exact restricted MLE. The constrained approximate estimator ratio uses all 2000 replications. The PWM ratio uses only replications in which the PWM estimator belongs to the restricted parameter space, with the denominator calculated from the same replications.
At the representative setting γ = 0.30 \gamma= 0.30 γ = 0.30 n = 25000 n = 25000 n = 25000 − 1.049 -1.049 − 1.049 − 1 -1 − 1 ( 0.251 , − 0.317 ) T (0.251, -0.317)^{\mathsf{T}} ( 0.251 , − 0.317 ) T ( 0.261 , − 0.327 ) T (0.261, -0.327)^{\mathsf{T}} ( 0.261 , − 0.327 ) T n ( X ( 1 ) − μ ) / σ n(X_{(1)} - \mu)/\sigma n ( X ( 1 ) − μ ) / σ Exp ( 1 ) \operatorname{Exp}(1) Exp ( 1 ) Figure 12 . Panel (c) shows that, when γ = 0.55 \gamma= 0.55 γ = 0.55 ξ = 0 \xi= 0 ξ = 0 γ = 0.30 \gamma= 0.30 γ = 0.30
Figure 12. GPD diagnostics. Panels (a) and (b) use γ = 0.30 \gamma= 0.30 γ = 0.30 n = 25000 n = 25000 n = 25000 Exp ( 1 ) \operatorname{Exp}(1) Exp ( 1 )
