betaFeedback

WorksMathematics

Second-Order Least Squares Method for Statistical Inference in Linear Models with Asymmetric Errors

Abstract

In a linear regression model with normal random errors, the ordinary least squares estimator is the most efficient estimator. However, when the random errors have an asymmetric distribution, a more efficient estimator can be constructed using the higher moments of the data. In this paper, we use the second-order least squares method to obtain more accurate asymptotic confidence intervals and more powerful hypothesis tests in linear models. We also develop an efficient and stable numerical algorithm to calculate the proposed estimator. Our extensive simulation studies demonstrate the advantages of the proposed method. Two real data examples are presented.

Introduction

Asymmetrically distributed data arise in many fields, including environmental science, finance and economics, engineering, life-time and extreme data analysis, see e.g., Beguería and Vicente-Serrano [2]; Castro-Camilo et al. [4]; Green [8]; Wright et al. [20]; Yousof et al. [23]; Nandy et al. [12]. It is well-known that, in linear regression models, the ordinary least squares estimator (OLSE) is the best unbiased estimator and is most efficient if the model error has a normal distribution. However, if the model error has an asymmetric distribution, then the higher moments of the data can be used to construct more efficient estimators. One such example is the second-order least squares (SLS) estimator (SLSE) that uses the first two conditional moments of the response variable given the predictors. It was shown that this estimator is asymptotically more efficient than the OLSE if the model error has a non-zero third moment and that both estimators are asymptotically equivalent otherwise [19]. Additionally, the SLSE attains the optimal semiparametric efficiency bound [11]. In this paper, we demonstrate, among other things, that the efficiency gain of the SLSE may lead to conclusions different from when the OLSE is used in real data analysis (§ 6).

While the asymptotic theory of the SLSE has been established for general nonlinear regression models [19], its more in-depth properties for statistical inferences in linear models have not been explored. In this paper, we fill in these gaps by investigating more properties of the SLSE in linear models and studying its performance in asymptotic confidence intervals and hypothesis testing problems. Moreover, the theoretical results of Wang and Leblanc [19] were derived for general nonlinear models under some restrictive regularity conditions. Here we show that most of their conditions can be relaxed or removed in linear models, thus making the method more widely applicable. Another challenge with the SLSE is numerical computation, which has not been systematically explored in the literature. In § 4, we develop an iteratively reweighted quasi-Newton method that is relatively easy to implement and efficient in practice, as well as numerically stable (§ 5.2).

The SLS method has been used for parameter estimation problems, including in measurement error models [16, 17], mixed effects models [18], censored linear models [1], and dynamic models [13 b]. More recently, this method has also been used in optimal design problems [7, 3, 22, 21] (He and Yue 2022; He et al. 2024).

The SLS Estimator

Consider the linear model Y=X′β+ϵY = X^{\prime}\beta+ \epsilon, where E(ϵ∣X)=0E(\epsilon\mid X) = 0 and E(ϵ2∣X)=σ2E(\epsilon^{2} \mid X) = \sigma^{2}. Given the i.i.d. random sample (Yi,Xi)(Y_i, X_i), i=1,2,…,ni = 1, 2, \ldots, n, the OLSE β^OLS\hat{\beta}_{OLS} minimizes the “first-order” distance

Sn(β)=∑i=1n(Yi−Xi′β)2.S_n(\beta) = \sum_{i=1}^{n}(Y_i - X_i^{\prime}\beta)^2.

The OLSE for σ2\sigma^{2} is defined as

σ^OLS2=1n−pSn(β^OLS),\hat{\sigma}^{2}_{OLS} = \frac{1}{n-p}S_n(\hat{\beta}_{OLS}),

which is not obtained through an optimization process.

The SLSE is based on the first two (conditional) moments of YiY_i given XiX_i: E(Yi∣Xi)=Xi′β0E(Y_i \mid X_i) = X_i^{\prime}\beta_0 and E(Yi2∣Xi)=(Xi′β0)2+σ02E(Y_i^2 \mid X_i) = (X_i^{\prime}\beta_0)^2 + \sigma_0^2, where β0\beta_0 and σ02\sigma_0^2 are the true parameter values. Specifically, the SLSE for the vector of the unknown parameters γ=(β′,σ2)′\gamma= (\beta', \sigma^2)' is defined as

γ^SLS=argmin⁡Qn(γ):=∑i=1nρi(γ)′Wiρi(γ),(1)\hat{\gamma}_{SLS} = \operatorname*{argmin} Q_n(\gamma) := \sum_{i=1}^{n} \rho_i(\gamma)'W_i\rho_i(\gamma), \tag*{(1)}

where ρi(γ)=(Yi−Xi′β,Yi2−(Xi′β)2−σ2)′\rho_i(\gamma) = (Y_i - X_i'\beta, Y_i^2 - (X_i'\beta)^2 - \sigma^2)' and Wi=W(Xi)W_i = W(X_i) is a nonnegative definite weight matrix that may depend on XiX_i. Wang and Leblanc [19] showed that the most efficient SLSE is obtained by using the optimal weight matrix Wi=E[ρi(γ0)ρi′(γ0)∣Xi]−1W_i = E[\rho_i(\gamma_0)\rho_i'(\gamma_0)|X_i]^{-1}, with γ0=(β0′,σ02)′\gamma_0 = (\beta_0', \sigma_0^2)', which has the following form for linear models:

Wi=η(μ4−σ04+4μ3(Xi′β0)+4σ02(Xi′β0)2−μ3−2σ02(Xi′β0)−μ3−2σ02(Xi′β0)σ02),(2)W_i = \eta\begin{pmatrix} \mu_4-\sigma_0^4+4\mu_3(X_i'\beta_0)+4\sigma_0^2(X_i'\beta_0)^2 & -\mu_3-2\sigma_0^2(X_i'\beta_0) \\ -\mu_3-2\sigma_0^2(X_i'\beta_0) & \sigma_0^2 \end{pmatrix}, \tag*{(2)}

where μj=E(εj∣X)\mu_j = E(\varepsilon^j|X), j=3,4j=3,4 and η=[σ02(μ4−σ04)−μ32]−1\eta= [\sigma_0^2(\mu_4-\sigma_0^4)-\mu_3^2]^{-1}. In this paper, we will use the term SLSE to refer to the most efficient version of the estimator.

Further, Wang and Leblanc [19] established consistency and asymptotic normality of the SLSE in nonlinear models under general conditions. Their conditions can be substantially simplified for linear models as follows.

A1 The error ε\varepsilon has finite constant moments μj=E(εj∣X)\mu_j = E(\varepsilon^j|X), j=3,4j=3,4, and σ02(μ4−σ04)−μ32≠0\sigma_0^2(\mu_4-\sigma_0^4)-\mu_3^2 \ne0.

A2 The predictor XX has a finite fourth moment E∥X∥4<∞E\|X\|^4 < \infty.

A3 The true parameter value γ0=(β0′,σ02)′∈int(Γ)\gamma_0 = (\beta_0',\sigma_0^2)' \in int(\Gamma) and the parameter space Γ⊂Rp+1\Gamma\subset\mathbb{R}^{p+1} is compact.

Let μx=E(Xi)\mu_x = E(X_i) and Mxx=E(XiXi′)M_{xx} = E(X_iX_i'). Then we have the following results, the proofs of which are given in Appendix.

Theorem 1. Under assumptions A1-A3, as n→∞n \to\infty, the SLSE satisfies

  1. γ^SLS→a.s.γ\hat{\gamma}_{SLS} \xrightarrow{\mathrm{a.s.}} \gamma;

  1. n(γ^SLS−γ)→dN(0,C)\sqrt{n}(\hat{\gamma}_{SLS}-\gamma) \xrightarrow{\mathrm{d}} N(0,C), where

C=(V(β^SLS)ξV(σ^SLS2)Mxx−1μxξV(σ^SLS2)μx′Mxx−1V(σ^SLS2)),with ξ=μ32σ02(μ4−σ04),(3)C = \begin{pmatrix} V(\hat{\beta}_{SLS}) & \xi V(\hat{\sigma}_{SLS}^2)M_{xx}^{-1}\mu_x \\ \xi V(\hat{\sigma}_{SLS}^2)\mu_x'M_{xx}^{-1} & V(\hat{\sigma}_{SLS}^2) \end{pmatrix}, \qquad\text{with } \xi= \frac{\mu_3^2}{\sigma_0^2(\mu_4-\sigma_0^4)}, \tag*{(3)}
V(β^SLS)=σ02(1−ξ)(Mxx−ξμxμx′)−1,(4)V(\hat{\beta}_{SLS}) = \sigma_0^2(1-\xi)(M_{xx}-\xi\mu_x\mu_x')^{-1}, \tag*{(4)}

and

V(σ^SLS2)=(μ4−σ04)(1−ξ)1−ξμx′Mxx−1μx.(5)V(\hat{\sigma}_{SLS}^2) = \frac{(\mu_4-\sigma_0^4)(1-\xi)}{1-\xi\mu_x'M_{xx}^{-1}\mu_x}. \tag*{(5)}

From Theorem 1 we see that when μ3=0\mu_3 = 0 and hence ξ=0\xi= 0, the SLSEs β^SLS\hat{\beta}_{SLS} and σ^SLS2\hat{\sigma}^2_{SLS} have the same asymptotic variances as the respective OLSEs: V(β^SLS)=V(β^OLS)=σ02Mxx−1V(\hat{\beta}_{SLS}) = V(\hat{\beta}_{OLS}) = \sigma_0^2 M_{xx}^{-1} and V(σ^SLS2)=V(σ^OLS2)=μ4−σ04V(\hat{\sigma}^2_{SLS}) = V(\hat{\sigma}^2_{OLS}) = \mu_4 - \sigma_0^4. However, the following theorem shows that in general the SLSE is asymptotically more efficient than the OLSE when μ3≠0\mu_3 \ne0.

Theorem 2. Suppose μ3≠0\mu_3 \ne0. Then, under the conditions of Theorem 1,

  1. The matrix V(β^OLS)−V(β^SLS)V(\hat{\beta}_{OLS}) - V(\hat{\beta}_{SLS}) is positive definite if and only if μx′Mxx−1μx≠1\mu_x^{\prime}M_{xx}^{-1}\mu_x \ne1; and is nonnegative definite with rank dim⁡(β)−1\dim(\beta) - 1 if μx′Mxx−1μx=1\mu_x^{\prime}M_{xx}^{-1}\mu_x = 1.

  1. V(σ^OLS2)−V(σ^SLS2)≥0V(\hat{\sigma}^2_{OLS}) - V(\hat{\sigma}^2_{SLS}) \ge0 with equality holding if and only if μx′Mxx−1μx=1\mu_x^{\prime}M_{xx}^{-1}\mu_x = 1.

From Theorem 2 we can see that the efficiency gain of the SLSE depends on the value of μx′Mxx−1μx∈[0,1]\mu_x^{\prime}M_{xx}^{-1}\mu_x \in[0,1] (see Appendix § 11). For example, the maximum gain is achieved when the data on XX are centered, so that μx=0\mu_x = 0, in which case V(β^SLS)=(1−ξ)V(β^OLS)V(\hat{\beta}_{SLS}) = (1 - \xi)V(\hat{\beta}_{OLS}) and V(σ^SLS2)=(1−ξ)V(σ^OLS2)V(\hat{\sigma}^2_{SLS}) = (1 - \xi)V(\hat{\sigma}^2_{OLS}).

On the other hand, if the model has an intercept term, i.e., for the first predictor variable P(X1=1)=1P(X_1 = 1) = 1, then Mxxe1=μxM_{xx}e_1 = \mu_x and therefore μx′Mxx−1μx=1\mu_x^{\prime}M_{xx}^{-1}\mu_x = 1, where e1e_1 is the unit vector with the first element one and others zero. In this case, the matrix V(β^OLS)−V(β^SLS)V(\hat{\beta}_{OLS}) - V(\hat{\beta}_{SLS}) have all positive but one zero eigenvalues. In particular, using the block matrix inversion formula we obtain an alternative expression

V(β^SLS)=σ02[(1−ξ)Mxx−1+ξe1e1′].(6)V(\hat{\beta}_{SLS}) = \sigma_0^2\left[(1-\xi)M_{xx}^{-1}+\xi e_1e_1^{\prime}\right]. \tag*{(6)}

However, in the case where the model does not have intercept, then μx′Mxx−1μx<1\mu_x^{\prime}M_{xx}^{-1}\mu_x < 1 and, therefore, all elements of β^SLS\hat{\beta}_{SLS} are asymptotically more efficient than the corresponding those of β^OLS\hat{\beta}_{OLS}.

Statistical Inference

Although the SLS method has been used in point estimation and experimental design problems in the literature, it has not been used in confidence intervals and hypothesis testing problems. In this section, we use the asymptotic theory of the previous section to construct Wald statistics for statistical inferences. Specifically, by Theorem 1, for each j=1,2,…,pj = 1, 2, \ldots, p, the jj-th element β^SLS,j\hat{\beta}_{SLS,j} of β^SLS\hat{\beta}_{SLS} satisfies s^j−1(β^SLS,j−βj)→dN(0,1)\hat{s}_{j}^{-1}(\hat{\beta}_{SLS,j} - \beta_j) \xrightarrow{d} N(0,1), where s^j2\hat{s}_j^2 is the jj-th diagonal element of the estimated version of the matrix V(β^SLS)V(\hat{\beta}_{SLS}) of (4), where all moments and true parameter values are replaced with the sample moments and SLSEs, respectively.

Therefore, for any 0<α<10 < \alpha< 1, a level 1−α1 - \alpha asymptotic confidence interval for βj\beta_j is

[β^SLS,j−zα/2s^j, β^SLS,j+zα/2s^j],(7)\left[\hat{\beta}_{SLS,j} - z_{\alpha/2}\hat{s}_j,\ \hat{\beta}_{SLS,j} + z_{\alpha/2}\hat{s}_j\right], \tag*{(7)}

where zα/2z_{\alpha/2} is the upper α/2\alpha/2 quantile of the standard normal distribution.

Similarly, for testing the hypotheses

H0: βj=βj0vs.H1: βj≠βj0,(8)H_0:\ \beta_j=\beta_{j0}\quad\text{vs.}\quad H_1:\ \beta_j\ne\beta_{j0}, \tag*{(8)}

where βj0\beta_{j0} is known, the Wald test statistic is

Zj=β^SLS,j−βj0s^j.Z_j=\frac{\hat{\beta}_{SLS,j}-\beta_{j0}}{\hat{s}_j}.

Hence, for a given significance level 0<α<10<\alpha<1, H0H_0 will be rejected if ∣Zj∣≥zα/2|Z_j|\ge z_{\alpha/2}.

Reweighted Quasi-Newton Method

Although the SLSE has desirable statistical properties under skewed error distributions, its numerical computation is not straightforward, as it does not have a closed-form formula as the OLSE. In this section, we develop a numerical algorithm to calculate the SLSE in general linear models, based on the Newton method for minimization of the objective function Qn(γ)Q_n(\gamma) in (1).

First, the standard Newton procedure is based on the iterative equation

γ(t+1)=γ(t)−H(γ(t))−1G(γ(t)),t=0,1,2,…(9)\gamma^{(t+1)}=\gamma^{(t)}-H(\gamma^{(t)})^{-1}G(\gamma^{(t)}),\quad t=0,1,2,\ldots \tag*{(9)}

where G(γ)G(\gamma) and H(γ)H(\gamma) are the gradient and Hessian matrix of Qn(γ)Q_n(\gamma), respectively:

G(γ)=∇γQn(γ)=2∑i=1n∇γρi′(γ)Wiρi(γ)(10)G(\gamma)=\nabla_{\gamma}Q_n(\gamma)=2\sum_{i=1}^{n}\nabla_{\gamma}\rho_i'(\gamma)W_i\rho_i(\gamma) \tag*{(10)}

and

H(γ)=∇γγ′2Qn(γ)=2∑i=1n∇γρi′(γ)Wi∇γ′ρi(γ)+(ρi′(γ)Wi⊗Ip+1)∇γ′vec⁡(∇γρi′(γ)).(11)H(\gamma)=\nabla^2_{\gamma\gamma'}Q_n(\gamma)=2\sum_{i=1}^{n}\nabla_{\gamma}\rho_i'(\gamma)W_i\nabla_{\gamma'}\rho_i(\gamma)+\left(\rho_i'(\gamma)W_i\otimes I_{p+1}\right)\nabla_{\gamma'}\operatorname{vec}\left(\nabla_{\gamma}\rho_i'(\gamma)\right). \tag*{(11)}

However, the weight matrix WiW_i of (2) in the gradient and Hessian depends on the unknown parameters γ\gamma, μ3\mu_3 and μ4\mu_4. To overcome this complication, we propose a numerically simpler and more efficient quasi-Newton method, where these parameters are updated in each iteration by using the current value γ^(t)\hat{\gamma}^{(t)} and the corresponding residuals ei(t)=yi−xi′β^(t)e_i^{(t)}=y_i-x_i'\hat{\beta}^{(t)}. This so-called reweighted Newton update equation is given by (9) with

G(γ)=−η∑i=1n((μ4−σ4)(yi−xi′β)−μ3(yi−xi′β)2+μ3σ2σ2(yi−xi′β)2−σ4−μ3(yi−xi′β)xi).(12)G(\gamma)=-\eta\sum_{i=1}^{n}\left(\frac{(\mu_4-\sigma^4)(y_i-x_i'\beta)-\mu_3(y_i-x_i'\beta)^2+\mu_3\sigma^2}{\sigma^2(y_i-x_i'\beta)^2-\sigma^4-\mu_3(y_i-x_i'\beta)}x_i\right). \tag*{(12)}
H(γ)=η∑i=1n(μ4+σ4−2σ2(yi−xi′β)2+2μ3(yi−xi′β)−μ3xi−μ3xi′σ2)(13)H(\gamma)=\eta\sum_{i=1}^{n}\begin{pmatrix} \mu_{4}+\sigma^{4}-2\sigma^{2}(y_{i}-x_{i}^{\prime}\beta)^{2}+2\mu_{3}(y_{i}-x_{i}^{\prime}\beta) & -\mu_{3}x_{i} \\ -\mu_{3}x_{i}^{\prime} & \sigma^{2} \end{pmatrix} \tag*{(13)}

and μj=n−1∑i=1n(yi−xi′β(t))j\mu_{j}=n^{-1}\sum_{i=1}^{n}(y_{i}-x_{i}^{\prime}\beta^{(t)})^{j}, j=3,4j=3,4. The derivation of (12) and (13) are given in Appendix § 12.

Another challenge is that in the case of small samples, the so-called “bad samples” may cause the Hessian matrix to be near singular. To address this issue, we implemented adaptive ridge regularization and Armijo backtracking line search, which assure the stability of the Hessian and prevent divergence of the algorithm.

First, we initialize λ=10−8\lambda=10^{-8} and increase it over iteration until H~(γ(t))=H(γ(t))+λIp\widetilde{H}(\gamma^{(t)})=H(\gamma^{(t)})+\lambda I_{p} becomes positive definite. This way, the descent direction is computed as d(t)=−H~(γ(t))−1G(γ(t))d^{(t)}=-\widetilde{H}(\gamma^{(t)})^{-1}G(\gamma^{(t)}). Further, to introduce stronger stability and to increase convergence speed, Armijo backtracking line search is utilized as

γ(t+1)=γ(t)+αkd(t),\gamma^{(t+1)}=\gamma^{(t)}+\alpha_{k}d^{(t)},

where αk∈(0,1]\alpha_{k}\in(0,1] is the step size. We start with α0=1\alpha_{0}=1 and decrease it until two conditions (Armijo Conditions) are met:

  1. Boundary constraint: γp(t)>0\gamma_{p}^{(t)}>0;

  1. Sufficient decrease: Qn(γ(t)+αkd(t))≤Qn(γ(t))+cαkG(γ(t))′d(t)Q_{n}(\gamma^{(t)}+\alpha_{k}d^{(t)})\leq Q_{n}(\gamma^{(t)})+c\alpha_{k}G(\gamma^{(t)})^{\prime}d^{(t)}, where c is a small constant ensuring a sufficient reduction.

Finally, we use the OLSE as the starting point of the iteration process. The algorithm of this reweighted quasi-Newton procedure is given below. The numerical stability of the algorithm will be illustrated for two simulation models in § 5.2.

Algorithm: Reweighted Quasi-Newton Method with Line Search

  1. Input: Data (Y,X)(Y, X), Initial estimate γ(0)\gamma^{(0)}, Convergence tolerance δ\delta, Max iterations Kmax⁡K_{\max}.

  2. Initialize: t←0t \leftarrow0, γ(t)←γ(0)\gamma^{(t)} \leftarrow\gamma^{(0)}.

  3. while ∥γ(t)−γ(t−1)∥>δ\lVert\gamma^{(t)}-\gamma^{(t-1)}\rVert> \delta and t<Kmax⁡t < K_{\max} do

  4. Compute Gradient G(γ(t))G(\gamma^{(t)}) and Hessian H(γ(t))H(\gamma^{(t)}) using estimated moments μ^j\hat{\mu}_j.

  5. ▷ Adaptive Ridge Regularization

  6. Set λ←10−8\lambda\leftarrow10^{-8}.

  7. Repeat

  8. H~←H(t)+λIp\tilde{H} \leftarrow H^{(t)}+\lambda I_p.

  9. If H~\tilde{H} is Positive Definite (via Cholesky): Break

  10. λ←10×λ\lambda\leftarrow10 \times\lambda.

  11. Compute Descent Direction d(t)←−H~−1G(γ(t))d^{(t)} \leftarrow-\tilde{H}^{-1}G(\gamma^{(t)}).

  12. ▷ Armijo Backtracking Line Search

  13. Set α←1\alpha\leftarrow1.

  14. Repeat

  15. Candidate γnew←γ(t)+αd(t)\gamma_{\mathrm{new}} \leftarrow\gamma^{(t)}+\alpha d^{(t)}.

  16. If γnew[p]≤0\gamma_{\mathrm{new}}[p] \le0 (variance constraint violated): α←0.5α\alpha\leftarrow0.5\alpha; Continue

  17. If Qn(γnew)≤Qn(γ(t))+c1αG(γ(t))Td(t)Q_n(\gamma_{\mathrm{new}}) \le Q_n(\gamma^{(t)})+c_1\alpha G(\gamma^{(t)})^\mathsf{T}d^{(t)}: Break (Sufficient Decrease met)

  18. α←0.5×α\alpha\leftarrow0.5 \times\alpha.

  19. Until α\alpha reaches minimum tolerance.

  20. ▷ Update and Iterate

  21. γ(t+1)←γnew\gamma^{(t+1)} \leftarrow\gamma_{\mathrm{new}}.

  22. t←t+1t \leftarrow t+1.

  23. end while

  24. Output: γ^←γ(t)\hat{\gamma} \leftarrow\gamma^{(t)}.

Simulation Studies

In this section, we investigate numerically the finite-sample performance of the SLSE β^SLS,j\hat{\beta}_{SLS,j} vs. the OLSE β^OLS,j\hat{\beta}_{OLS,j} for each of the components βj\beta_j of β\beta, the asymptotic CIs for βj\beta_j based on the asymptotic normality of β^SLS,j\hat{\beta}_{SLS,j} from Theorem 1 vs. the well-known CIs based on β^OLS,j\hat{\beta}_{OLS,j}, as well as the respective asymptotic tests of the hypotheses in (8) using the corresponding Wald test statistics. In view of that σ^SLS2\hat{\sigma}_{SLS}^{2} converges to σ2\sigma^{2} much slower compared to the convergence of β^SLS\hat{\beta}_{SLS} to β\beta, comparison of the σ^SLS2\hat{\sigma}_{SLS}^{2} vs. σ^OLS2\hat{\sigma}_{OLS}^{2} based inference results is omitted in this study.

Simulation models

For our simulation studies, we consider the following two models:

Model 1: Y=β0X1+β1X2+β2X3+εY = \beta_0 X_1 + \beta_1 X_2 + \beta_2 X_3 + \varepsilon with β=(0.5,1,−1.5)′\beta= (0.5, 1, -1.5)^{\prime}, where X1=1X_1 = 1, X2∼Unif⁡(1,20)X_2 \sim\operatorname{Unif}(1, 20), X3∼N(5,4)X_3 \sim\mathcal{N}(5, 4), and X⊥ ⁣ ⁣ ⁣⊥εX \perp\!\!\!\perp\varepsilon.

Model 2: Y=β0X1+β1X2+β2X3+εY = \beta_0 X_1 + \beta_1 X_2 + \beta_2 X_3 + \varepsilon with β=(0.5,1,−1.5)′\beta= (0.5, 1, -1.5)^{\prime}, where X1∼Beta⁡(2,5)X_1 \sim\operatorname{Beta}(2, 5), (X2,X3)′∼N((2,0)′,[1,0.5;0.5,1])(X_2, X_3)^{\prime} \sim\mathcal{N}((2, 0)^{\prime}, [1, 0.5; 0.5, 1]), and X⊥ ⁣ ⁣ ⁣⊥εX \perp\!\!\!\perp\varepsilon.

For each of the models, we consider the following three error distributions with the means of zero and the variances of one: ε∼N(0,1)\varepsilon\sim\mathcal{N}(0, 1); ε∼Exp⁡(1)−1\varepsilon\sim\operatorname{Exp}(1) - 1, called std-Exp(1) hereafter, with the skewness of 2; and ε∼Weibull⁡(0.6,1)−Γ(1+1/0.6)Γ(1+2/0.6)−(Γ(1+1/0.6))2\varepsilon\sim\frac{\operatorname{Weibull}(0.6, 1) - \Gamma(1 + 1/0.6)}{\sqrt{\Gamma(1 + 2/0.6) - (\Gamma(1 + 1/0.6))^2}}, abbreviated as std-Weibull(0.6, 1), with the skewness of 4.59.

In our simulation studies, we use the sample sizes n=100n = 100, 300, 500, 1000 and, for some tables, n=50n = 50, 5000, 10000, 20000, 50000, and m=5000m = 5000 Monte Carlo replications.

We use the large-sample standard errors (ASE) and empirical standard errors (SE) to compare β^SLS,j\hat{\beta}_{SLS,j} and β^OLS,j\hat{\beta}_{OLS,j} for each component βj\beta_j of β\beta. The performance of the respective CIs is studied based on their empirical finite-sample coverage probabilities (CP) and the ratio of their empirical expected lengths, namely,

Rn=average length of m β^OLS,j based CIs for a sample size naverage length of m β^SLS,j based CIs for a sample size n.(14)R_n = \frac{\text{average length of }m\ \hat{\beta}_{OLS,j}\text{ based CIs for a sample size }n}{\text{average length of }m\ \hat{\beta}_{SLS,j}\text{ based CIs for a sample size }n}. \tag*{(14)}

The performance of the β^OLS,j\hat{\beta}_{OLS,j} and β^SLS,j\hat{\beta}_{SLS,j} based Wald tests for the hypotheses in (8) is evaluated through their empirical power functions.

Numerical stability of the SLSE

Before studying the performance of the SLSE for β\beta in statistical inference, we address numerical stability of our proposed reweighted quasi-Newton algorithm for computing β^SLS\hat{\beta}_{SLS}.

Under the standard reweighted Newton–Raphson algorithm (without regularization or line search), a large proportion of simulation runs resulted in failure due to instability in Hessian inversion. This issue was particularly severe in Model 2 for smaller sample sizes, where the rate of such “bad samples” reached as high as 37.14% under the std-Weibull(0.6, 1) error distribution for n=50n = 50 and was at 5.96% for this skewed distribution even for n=1000n = 1000, as shown in Table 1.

Reweighted NewtonReweighted quasi–Newton
ModelError distributionn=50n = 50n=100n = 100n=300n = 300n=500n = 500n=1000n = 1000n=50n = 50n=100n = 100n=300n = 300n=500n = 500n=1000n = 1000
Model 1N(0,1)13.76%3.47%< 0.01%0%0%0%0%0%0%0%
Model 1std-Exp(1)8.22%0.73%< 0.01%0%0%0%0%0%0%0%
Model 1std-Weibull(0.6, 1)2.81%< 0.01%< 0.01%0%0%< 0.01%0%0%0%0%
Model 2N(0,1)16.54%7.15%0.02%0%0%0%0%0%0%0%
Model 2std-Exp(1)28.41%13.22%2.64%0.52%0.02%0%0%0%0%0%
Model 2std-Weibull(0.6, 1)37.14%30.37%20.43%14.43%5.96%< 0.01%< 0.01%0%0%0%

Table 1. Bad sample rates when computing the SLSE using the standard reweighted Newton method and § 4 reweighted quasi–Newton method.

By introducing the reweighted quasi-Newton method with an Armijo-type line search, these failures were effectively eliminated. As seen from Table 1, across all simulation settings for both Models 1 and 2, the bad sample rate was reduced dramatically, dropping to essentially 0%. This highlights the importance of regularization and adaptive step-size control in ensuring reliable implementation of the SLSE, particularly in smaller samples and under asymmetric error distributions.

Comparison of the OLS and SLS based statistical inference

Table 2 and Table 3 below for Models 1 and 2, respectively, with three different error distributions, contain the values of the OLSE and SLSE, their respective ASEs and SEs, the CPs of the corresponding 95% CIs, and the ratio RnR_n of (14), for each βj\beta_j.

Error distributionnnParameterOLSSLSRnR_n
OLSEASESECPSLSEASESECP
N(0,1)100β0\beta_00.50410.32380.32360.94320.50440.31470.32870.93201.0292
β1\beta_11.00010.01750.01780.94001.00010.01700.01820.92941.0307
β2\beta_2-1.50060.05080.05050.9430-1.50070.04930.05130.93181.0307
300β0\beta_00.49990.18540.18520.94810.49980.18360.18570.94481.0095
β1\beta_10.99990.01000.01010.94670.99990.00990.01010.94301.0100
β2\beta_2-1.49990.02900.02900.9515-1.49990.02870.02910.94711.0100
500β0\beta_00.50050.14350.14560.94570.50050.14270.14600.94351.0057
β1\beta_11.00000.00780.00770.95051.00000.00770.00770.94771.0060
β2\beta_2-1.49990.02240.02280.9447-1.49990.02230.02290.94281.0060
1000β0\beta_00.50140.10140.10190.95030.50140.10110.10200.94991.0028
β1\beta_10.99990.00550.00550.95290.99990.00550.00550.95191.0030
β2\beta_2-1.50010.01580.01590.9488-1.50010.01580.01590.94741.0030
std-Exp(1)100β0\beta_00.49570.32160.32780.94830.49720.22450.22820.94481.4329
β1\beta_11.00000.01740.01750.94970.99990.01150.01170.94801.5165
β2\beta_2-1.49940.05040.05180.9432-1.49950.03320.03420.94301.5165
300β0\beta_00.50120.18510.18580.94670.49940.13270.13360.94951.3945
β1\beta_10.99990.01000.01000.94861.00000.00680.00690.94891.4702
β2\beta_2-1.50000.02900.02920.9466-1.49990.01970.01990.95041.4701
500β0\beta_00.49760.14320.14500.94730.49920.10360.10380.94841.3822
β1\beta_11.00010.00770.00780.94871.00000.00530.00540.94751.4548
β2\beta_2-1.49980.02240.02250.9479-1.50000.01540.01540.95021.4548
1000β0\beta_00.49920.10130.10040.95010.49950.07390.07420.95081.3710
β1\beta_11.00000.00550.00550.94881.00000.00380.00380.94761.4408
β2\beta_2-1.49990.01580.01580.9473-1.50000.01100.01100.95121.4408
std-Weibull(0.6,1)100β0\beta_00.50140.31300.32450.94350.49940.18010.18690.93571.7374
β1\beta_10.99980.01690.01770.94541.00000.00860.00890.94991.9546
β2\beta_2-1.49990.04900.05100.9478-1.49990.02510.02640.94251.9548
300β0\beta_00.49650.18270.18710.94480.49790.11260.11490.94291.6217
β1\beta_11.00020.00990.00990.95051.00010.00550.00550.95481.7843
β2\beta_2-1.49990.02850.02920.9478-1.50000.01600.01630.94961.7843
500β0\beta_00.50100.14240.14430.94780.50110.09040.09060.95341.5752
β1\beta_11.00000.00770.00780.94641.00000.00450.00460.94931.7178
β2\beta_2-1.50000.02230.02250.9500-1.50010.01300.01300.95211.7178
1000β0\beta_00.50000.10100.10100.94690.50080.06620.06660.94891.5269
β1\beta_11.00000.00550.00550.95151.00000.00330.00330.95341.6501
β2\beta_2-1.50000.01580.01590.9495-1.50010.00960.00970.94831.6502

Table 2. OLSE vs. SLSE, and 95% OLS CI vs. 95% SLS CI for Model 1.

Error distributionnnParameterOLS OLSEOLS ASEOLS SEOLS CPSLS SLSESLS ASESLS SESLS CPRnR_n
100β0\beta_00.49160.51850.52710.94420.49020.50230.53290.93281.0321
β1\beta_11.00080.07780.07850.94641.00100.07550.07940.93561.0307
N(0,1)β2\beta_2-1.49720.10880.11070.9402-1.49740.10530.11200.93041.0328
300β0\beta_00.49960.29610.29600.94620.49910.29280.29800.94241.0112
β1\beta_10.99990.04440.04440.94891.00000.04390.04460.94451.0107
β2\beta_2-1.49990.06210.06280.9477-1.49990.06140.06310.94441.0114
500β0\beta_00.49920.22900.23000.94610.49890.22750.23090.94491.0068
β1\beta_11.00020.03440.03430.95221.00020.03420.03440.94981.0065
β2\beta_2-1.49960.04800.04810.9511-1.49960.04770.04830.94771.0069
1000β0\beta_00.49910.16170.16330.94950.49910.16120.16370.94771.0033
β1\beta_11.00000.02430.02430.94961.00000.02420.02430.94811.0032
β2\beta_2-1.50010.03390.03400.9503-1.50010.03380.03400.94881.0034
std-Exp(1)100β0\beta_00.49380.51490.51800.94790.49890.34890.35910.94241.4760
β1\beta_11.00010.07710.07790.94380.99910.05440.05600.93991.4180
β2\beta_2-1.49960.10780.11130.9436-1.49970.07170.07510.93911.5040
300β0\beta_00.49800.29400.29660.94560.50070.20440.20950.94161.4390
β1\beta_10.99970.04410.04450.94490.99890.03190.03280.94281.3860
β2\beta_2-1.50060.06170.06220.9517-1.49970.04220.04260.94721.4650
500β0\beta_00.49870.22860.22910.95120.49900.16020.16220.94601.4270
β1\beta_11.00030.03430.03430.94690.99970.02490.02500.95001.3760
β2\beta_2-1.49980.04800.04810.9481-1.49970.03300.03330.94821.4520
1000β0\beta_00.50000.16150.16190.94900.49970.11460.11640.94401.4090
β1\beta_11.00010.02430.02410.95130.99980.01780.01770.95041.3600
β2\beta_2-1.49990.03390.03360.9471-1.49980.02360.02370.95001.4330
std-Weibull(0.6,1)100β0\beta_00.50470.50160.51910.94910.51120.27280.32220.92511.8390
β1\beta_10.99920.07520.07760.94081.00000.04380.05060.92281.7160
β2\beta_2-1.50040.10510.11040.9435-1.50010.05530.06550.92601.9020
300β0\beta_00.49460.29180.29730.94560.49810.17150.17900.94381.7010
β1\beta_11.00040.04380.04420.94961.00010.02740.02830.94121.6020
β2\beta_2-1.49930.06120.06100.9517-1.50010.03500.03610.94791.7520
500β0\beta_00.49920.22620.22710.95020.49810.13720.13950.94991.6490
β1\beta_10.99970.03400.03410.94950.99960.02180.02200.95181.5600
β2\beta_2-1.49990.04750.04850.9473-1.49970.02800.02880.94571.6950
1000β0\beta_00.50190.16110.16340.94610.50100.10140.10220.94801.5880
β1\beta_10.99970.02420.02440.94680.99980.01600.01620.94961.5090
β2\beta_2-1.49970.03380.03410.9468-1.49960.02080.02120.94891.6280

Table 3. OLSE vs. SLSE, and 95% OLS CI vs. 95% SLS CI for Model 2.

Notably, across all simulation settings for both models, RnR_n is greater than 1, indicating that the SLSE based CI is shorter on average than the OLSE based CI. For the N(0, 1) distributed errors, RnR_n is very close to one, which agrees with the fact that the OLSE and SLSE estimators for βj\beta_j are equally efficient for symmetrically distributed errors, as noted right after Theorem 1. However, for the std-Exp(1) and more skewed std-Weibull(0.6, 1) errors, the SLSE based CI is significantly shorter than the OLSE CI, more so for the std-Weibull(0.6, 1) errors. Moreover, as illustrated in Table 4, due to the very slow rate of convergence of RnR_n for these two error distributions, this advantage of the SLSE based CI continues to hold at least up to the sample size of 50000. Furthermore, we can also see that the efficiency gain of the SLSE is greater in Model 2 without the intercept, which is consistent with our earlier discussions after Theorem 2 in § 2.

Model, error distributionParameter5010030050010005000100002000050000
Model 1, Exp(1)β0\beta_{0}1.4561.4241.4011.3841.3691.3521.3541.3531.351
β1\beta_{1}1.5421.5051.4771.4571.4381.4171.4191.4181.415
β2\beta_{2}1.5421.5051.4771.4571.4381.4171.4191.4181.415
Model 1, Weibull(0.6,1)β0\beta_{0}1.8261.7421.6291.5731.5291.4531.4291.4211.397
β1\beta_{1}2.0881.9591.7961.7151.6531.5501.5171.5071.475
β2\beta_{2}2.0891.9581.7951.7151.6531.5501.5171.5071.475
Model 2, Exp(1)β0\beta_{0}1.5031.4751.4341.4141.4021.3901.3871.3821.382
β1\beta_{1}1.4421.4151.3821.3651.3551.3431.3411.3371.337
β2\beta_{2}1.5321.5011.4601.4391.4261.4121.4091.4041.404
Model 2, Weibull(0.6,1)β0\beta_{0}1.9471.8291.7081.6541.5901.4971.4801.4541.443
β1\beta_{1}1.8081.7111.6081.5631.5111.4331.4191.3971.388
β2\beta_{2}2.0161.8951.7571.7001.6301.5281.5091.4821.470

Table 4. Empirical expected length ratio RnR_n of (14) of the β^OLS,j\hat{\beta}_{OLS,j} and β^SLS,j\hat{\beta}_{SLS,j} based CIs across sample sizes.

As for the empirical coverages of the OLSE and SLSE based CIs, they appear to converge fast for both models. Moreover, they are essentially on par and are both close enough to 95% for n≥300n \ge300 in Table 2 and Table 3. For n=100n = 100, the coverage of the SLSE based CI is somewhat lower than that of the OLSE based CI, by at most 1.12% in Table 2 and 2.4% in Table 3, while not getting less than 92.94% for Model 1 and 92.28% for Model 2.

The superior asymptotic efficiency of the SLSE for skewed error distributions can also be seen through its lower ASEs for the std-Exp(1) and std-Weibull(0.6,1) errors in Table 2 and Table 3. The empirical SEs of the SLSE, which align closely with its ASEs, are also lower than the SEs of the OLSE for these error distributions, which indicates a higher precision of the SLSE in finite samples.

Table 5 and Table 6 contain the empirical power function comparison for the OLSE and SLSE based tests of the hypotheses in (8) for Models 1 and 2, respectively. Both tests have well-controlled empirical type I errors that are close enough to the nominal 5% for n≥300n \ge300, except for a few cases for the std-Weibull(0.6, 1) errors and n=300n = 300 and 500 for Model 2, where one or both errors are between 6.2% and 6.8%. For n=100n = 100 and all three parameters β0\beta_0, β1\beta_1, and β2\beta_2 for both models, the type I errors of the SLSE based test are somewhat higher than those of the OLSE based test, by at most 1.6% in Table 5 and 2.04% in Table 6, where the type I errors of the SLSE based test do not exceed 6.9% and 7.86%, respectively.

ParameterTest value βj0\beta_{j0}n=100n=100 OLSn=100n=100 SLSn=300n=300 OLSn=300n=300 SLSn=500n=500 OLSn=500n=500 SLSn=1000n=1000 OLSn=1000n=1000 SLS
ε∼\varepsilon\sim std-Exp(1)
β0\beta_{0}0.500.05380.06280.04880.04720.04830.05090.05540.0530
0.720.11940.20000.23850.39810.35790.58520.58840.8366
0.940.31040.51620.66650.89750.86680.98270.98800.9999
β1\beta_{1}1.000.05440.05640.05150.04530.04990.04990.05280.0507
1.020.22380.42720.51600.83960.73810.96370.95760.9993
1.040.64340.92640.97520.99980.99901.00001.00001.0000
β2\beta_{2}-1.500.05300.06900.05210.05180.05260.05050.05460.0510
-1.450.17040.33760.41650.72390.60110.89710.88170.9950
-1.400.51880.83280.92700.99780.99240.99991.00001.0000
ε∼\varepsilon\sim std-Weibull(0.6, 1)
β0\beta_{0}0.500.05760.06040.05440.05000.05300.05000.04920.0512
0.720.15440.30660.29000.50400.40400.69200.59400.8940
0.940.36360.67120.66800.94400.88000.98400.99001.0000
β1\beta_{1}1.000.04800.05140.05100.05060.05700.04960.04840.0504
1.020.25800.65160.54000.93500.76100.98900.95500.9990
1.040.68620.98200.97400.99800.99601.00001.00001.0000
β2\beta_{2}-1.500.05040.05880.04520.04860.04960.04980.05400.0554
-1.450.21500.55160.46000.89200.60000.95200.90201.0000
-1.400.56600.95100.92600.99800.99201.00001.00001.0000

Table 5. Power function comparison for the OLSE and SLSE based tests of the hypotheses in (8) for Model 1.

ParameterTest value βj0\beta_{j0}n=100n = 100n=300n = 300n=500n = 500n=1000n = 1000
OLSSLSOLSSLSOLSSLSOLSSLS
ε∼std-Exp(1)\varepsilon\sim\text{std-Exp}(1)
β0\beta_00.500.05800.06400.04780.05290.05080.05490.04900.0512
0.720.08540.11820.12390.20780.16510.29600.28730.5025
0.940.15860.27420.34070.59110.49440.78800.78230.9662
β1\beta_11.000.05040.06120.05120.05640.04970.05540.04940.0523
1.020.06420.08460.08010.11240.09160.13990.13450.2167
1.040.09780.14460.15860.26140.22340.37710.38320.6170
β2\beta_2-1.500.05080.05640.05310.05430.04850.05460.05380.0483
-1.450.07680.11480.12510.21960.17670.32260.31550.5648
-1.400.15720.28900.36370.65920.54370.85590.84290.9893
ε∼std-Weibull(0.6,1)\varepsilon\sim\text{std-Weibull}(0.6, 1)
β0\beta_00.500.05620.07280.05800.05200.04800.06600.05200.0460
0.720.09620.18500.16400.32600.18800.41600.32000.5880
0.940.18600.43700.41400.75600.53000.86000.79200.9920
β1\beta_11.000.05820.07860.06800.06800.05400.05200.05800.0400
1.020.08220.12720.08200.14700.10800.17200.15300.2690
1.040.12380.21980.19400.37800.24000.47200.36400.7020
β2\beta_2-1.500.05080.06840.04800.05400.06400.06200.04600.0540
-1.450.07600.16160.13000.30800.16600.45200.34800.6940
-1.400.17120.46800.39600.81200.53400.93400.85800.9920

Table 6. Power function comparison for the OLSE and SLSE based tests of the hypotheses in (8) for Model 2.

In order to compare the powers of the OLSE and SLSE based tests under the two-sided alternative hypotheses in (8), we consider two values of βj0\beta_{j0} for each of βj\beta_j. Notably, the SLSE based test is significantly more powerful than the OLSE based one across all settings in Table 5 and Table 6. Both test powers increase with sample size and are higher for the values of βj0\beta_{j0} that are further away from the respective true parameter values. For n=1000n = 1000 for Model 1, both powers are essentially at 100% for βj0\beta_{j0} that are further away from the true βj\beta_j, while for the other value of βj0\beta_{j0}, Table 5 shows that the power of the OLSE based test is still behind that of the SLSE based one, by as much as 30% for the std-Weibull(0.6,1) errors and β00=0.72\beta_{00} = 0.72. Compared to Model 1, both the OLSE and SLSE based tests are less powerful across all settings for Model 2, where the power convergence for both tests appears to be slower and only the SLSE based test power reaches almost 100% at n=1000n = 1000 and only for β00=0.94\beta_{00} = 0.94 and β20=−1.40\beta_{20} = -1.40. Just like for Model 1, the power superiority of the SLSE based test is still considerable across all sample sizes, where the latter test is as much as 34.6% more powerful for the std-Weibull(0.6, 1) errors and β20=−1.45\beta_{20} = -1.45.

Real data application

In this section, we apply the SLS method to two real data examples that have been used in the literature.

Australian Institute of Sports (AIS) – Biomedicine

The Australian Institute of Sports (AIS) dataset contains biomedical measurements of 706 athletes from 12 sports, collected in 1991 [15, 6]. A subset of 202 observations, available in the R sn package, has been widely used for modeling with skewed error distributions. Following the approach of Nandy et al. [12], we model plasma ferritin concentration (Fe) using body mass index (BMI) and lean body mass (LBM) as covariates:

Fei=β0+β1⋅BMIi+β2⋅LBMi+εi,i=1,2,3,…,202.\mathrm{Fe}_i = \beta_0 + \beta_1 \cdot\mathrm{BMI}_i + \beta_2 \cdot\mathrm{LBM}_i + \varepsilon_i,\quad i = 1, 2, 3, \ldots, 202.

We first fitted the model using the OLS method and obtained the residuals. The histogram and Q-Q plot of the residuals in Figure 1 clearly indicate the right-skewness of the error distribution. Subsequently, we fitted the model again using the SLS method.

Figure 1. The OLS residual distribution of the AIS dataset.

Tables 7 and 8 present the estimates, standard errors, and 95% asymptotic confidence intervals for the intercept β0\beta_0 and coefficients β1\beta_1 and β2\beta_2 using the OLSEs and SLSEs. In all cases, the SLSE produces shorter CIs, indicating improved estimation precision. For instance, the interval length for the intercept decreases from 101.34 (OLSE) to 74.35 (SLSE), for β1\beta_{1} (BMI) from 6.20 to 4.52, and for β2\beta_{2} (LBM) from 1.36 to 0.99. These reductions reflect the variance-reducing nature of the SLSE. Although the SLSE point estimates differ from their OLSE counterparts, all estimates are within the corresponding CIs. However, a striking point is that the OLSE of the coefficient β1\beta_{1} of BMI is insignificant at 5% level because the corresponding 95% CI covers zero, while the coefficient is significant by the SLSE. This means that based on the OLSE one would conclude that BMI has no impact on plasma ferritin concentration, while it does have impact based on the SLSE.

ParameterEstimateSE95% CICI Length
β0\beta_{0} (Intercept)-30.857125.8511[-81.5253, 19.8110]101.3363
β1\beta_{1} (BMI)2.56531.5816[-0.5347, 5.6652]6.1999
β2\beta_{2} (LBM)0.75290.3466[0.0737, 1.4322]1.3585

Table 7. The OLSE estimates, standard errors, 95% asymptotic CIs, and CI lengths for the ASI dataset.

ParameterEstimateSE95% CICI Length
β0\beta_{0} (Intercept)-12.649018.9672[-49.8247, 24.5267]74.3514
β1\beta_{1} (BMI)2.37431.1530[0.1144, 4.6342]4.5198
β2\beta_{2} (LBM)0.53980.2526[0.0447, 1.0350]0.9904

Table 8. The SLSE estimates, standard errors, 95% asymptotic CIs, and CI lengths for the ASI dataset.

Health insurance

A publicly available dataset containing information on 1,338 individuals from the USA was used to compare the performance of the OLSE and SLSE for statistical inference [5]. The dataset includes variables such as age, sex, body mass index (BMI), number of children, smoking status, region of residence, and medical charges. In this analysis, medical charges were treated as the response variable. All predictor variables were standardized to improve computational efficiency. The main goal was to estimate the regression coefficients (β\beta) and their variances using both OLS and SLS, and to compare the resulting confidence intervals.

Since the dataset includes categorical variables, appropriate encoding was applied before fitting the models. Specifically, sex was encoded as 0 for female and 1 for male; smoker was coded as 0 for yes and 1 for no; and region was numerically encoded as 0 for northwest, 1 for northeast, 2 for southwest, and 3 for southeast.

VariableDescriptionTypeValues
ageAge of primary beneficiaryquantitative–
sexGender of insurance contractorqualitativefemale, male
bmiBody Mass Index (BMI)quantitative–
childrenNumber of children covered by insurancequalitative0, 1, 2, ...
smokerSmoking statusqualitativeyes, no
regionResidential area in the U.S.qualitativenorth/south-east/west
chargesIndividual medical costs billed by insurancequantitative–

Table 9. Description of the variables in the health insurance dataset.

Similarly to the previous example, a linear model was first fitted using the OLS method. Again, the OLS residual plots in Figure 2 clearly indicate the right-skewness of the error distribution. Hence, the model was fitted again using the SLS method.

Figure 2. The OLS residual distribution for the health insurance data.

Table 10 and Table 11 show the OLSE and SLSE based estimates of the regression coefficients, standard errors, 95% asymptotic confidence intervals, and interval lengths across the six covariates. In all cases, the SLSE produced narrower CIs than the OLSE, indicating greater precision. For instance, from Table 10, the CI for the age coefficient under the OLSE is [0.2711, 0.3252] with a length of 0.0541, while, as shown in Table 11, the SLSE yields a tighter interval of [0.2878, 0.3190], with a length of just 0.0312.

CovariateCoef. EstimateSE95% CICI Length
age0.29810.0138[0.2711, 0.3252]0.0541
sex-0.00530.0137[-0.0323, 0.0216]0.0539
bmi0.16970.0143[0.1417, 0.1977]0.0560
children0.04660.0137[0.0198, 0.0735]0.0537
smoker-0.79560.0138[-0.8226, -0.7687]0.0539
region-0.02780.0142[-0.0557, 0.0000]0.0557

Table 10. The OLSE estimates, standard errors, 95% asymptotic CIs, and CI lengths for the regression coefficients for the health insurance dataset.

CovariateCoef. EstimateSE95% CICI Length
age0.30340.0080[0.2878, 0.3190]0.0312
sex-0.00410.0079[-0.0196, 0.0115]0.0311
bmi0.07220.0082[0.0561, 0.0884]0.0323
children0.04250.0079[0.0270, 0.0580]0.0310
smoker-0.59500.0079[-0.6105, -0.5794]0.0311
region-0.01770.0082[-0.0337, -0.0016]0.0321

Table 11. The SLSE estimates, standard errors, 95% asymptotic CIs, and CI lengths for the regression coefficients for the health insurance dataset.

This trend is consistent across other predictors, including sex, BMI, number of children, smoker status, and region. Notably, the estimated effect of BMI (correlation of 0.2 with charges) under the SLSE is substantially smaller (0.0722) compared to the OLSE (0.1697), yet the SLSE based CI remains narrower. This suggests that the SLSE may be less influenced by multicollinearity. A similar pattern is observed with the smoker variable (correlation of 0.79 with charges), where both methods detect a strong negative effect, but the SLSE estimate is more moderate (-0.5950 vs. -0.7956). Similarly to the previous example, the OLSE indicates that regional variation has weak or no impact on medical charges, while it has a clearly negative impact by the SLSE.

Overall, these results support the use of the SLSE for inference in this context. The tighter confidence intervals and more stable estimates suggest that the SLSE may offer improved robustness, particularly when the classical OLS assumptions, such as normality or homoscedasticity, are potentially violated.

Acknowledgments

The authors gratefully acknowledge financial support by the Natural Sciences and Engineering Research Council of Canada (NSERC) and the University of Manitoba Graduate Fellowship (UMGF).

Disclosure Statement

No potential conflict of interest was reported by the author(s).

Preliminary

Wang and Leblanc [19] proved the consistency and asymptotic normality of the SLSE in general nonlinear models. In the case of linear models, many of their conditions and derivations are not required and, therefore, can be substantially simplified. The following lemmas are from Amemiya (1973) or Wang and Leblanc [19]. Let Qn(Z,ψ)Q_n(Z,\psi) be a measurable function of ZZ with respect to Lebesgue measure and continuous in ψ\psi, where ψ∈Ψ\psi\in\Psi and Ψ⊂Rp\Psi\subset\mathbb{R}^{p} is compact.

Lemma 1. If, as n→∞n \to\infty, sup⁡ψ∈Ψ∣Qn(Z,ψ)−Q(ψ)∣→a.s.0\sup_{\psi\in\Psi} \lvert Q_n(Z,\psi)-Q(\psi)\rvert\overset{\mathrm{a.s.}}{\to} 0, where Q(ψ)Q(\psi) is a non-stochastic function and has a unique minimum at ψ0∈Ψ\psi_0 \in\Psi, then ψ^n=argmin⁡ψ∈ΨQn(Z,ψ)→a.s.ψ0\widehat{\psi}_n=\operatorname*{argmin}_{\psi\in\Psi}Q_n(Z,\psi) \overset{\mathrm{a.s.}}{\to} \psi_0

Lemma 2. If, as n→∞n \to\infty, sup⁡ψ∈Ψ∣Qn(Z,ψ)−Q(ψ)∣→a.s.0\sup_{\psi\in\Psi}\lvert Q_n(Z,\psi)-Q(\psi)\rvert\overset{\mathrm{a.s.}}{\to} 0 and ψ0\psi_0 is an interior point of Ψ\Psi, then for any sequence of estimators ψ^n→a.s.ψ0\widehat{\psi}_n \overset{\mathrm{a.s.}}{\to} \psi_0 it holds Qn(Z,ψ^n)→a.s.Qn(ψ0)Q_n(Z,\widehat{\psi}_n) \overset{\mathrm{a.s.}}{\to} Q_n(\psi_0).

Proof of Theorem 1

(1) First, denote qi(γ)=ρi(γ)′Wiρi(γ)q_i(\gamma)=\rho_i(\gamma)^{\prime}W_i\rho_i(\gamma) and

Wi=(wi,11wi,12wi,21wi,22),W_i=\begin{pmatrix} w_{i,11} & w_{i,12} \\ w_{i,21} & w_{i,22} \end{pmatrix},

where wi,11=μ4−σ04+4μ3(Xi′β0)+4σ02(Xi′β0)2w_{i,11}=\mu_4-\sigma_0^4+4\mu_3(X_i^{\prime}\beta_0)+4\sigma_0^2(X_i^{\prime}\beta_0)^2, wi,12=wi,21=−μ3−2σ02(Xi′β0)w_{i,12}=w_{i,21}=-\mu_3-2\sigma_0^2(X_i^{\prime}\beta_0), wi,22=w22=σ02w_{i,22}=w_{22}=\sigma_0^2. Then it is easy to see that the quadratic form

qi(γ)=wi,11(Yi−Xi′β)2+2wi,12(Yi−Xi′β)(Yi2−(Xi′β)2−σ2)+w22(Yi2−(Xi′β)2−σ2)2.q_i(\gamma)=w_{i,11}(Y_i-X_i^{\prime}\beta)^2+2w_{i,12}(Y_i-X_i^{\prime}\beta)(Y_i^2-(X_i^{\prime}\beta)^2-\sigma^2)+w_{22}(Y_i^2-(X_i^{\prime}\beta)^2-\sigma^2)^2.

can be expanded in terms of the powers of YiY_i and ∥Xi∥i\lVert X_i\rVert_i and the highest orders in this expansion are Yi4Y_i^4, Yi3∥Xi∥iY_i^3\lVert X_i\rVert_i, Yi2∥Xi∥i2Y_i^2\lVert X_i\rVert_i^2 and ∥Xi∥i4\lVert X_i\rVert_i^4. Further, since γ\gamma appears only in the coefficients of these terms and the parameter space Γ\Gamma is compact, by assumption A2 it holds E[max⁡Γqi(γ)]<∞E[\max_{\Gamma}q_i(\gamma)]<\infty. It follows from the uniform law of large numbers (ULLN) that sup⁡Γ∣Qn(γ)/n−Q(γ)∣→a.s.0\sup_{\Gamma}\lvert Q_n(\gamma)/n-Q(\gamma)\rvert\overset{\mathrm{a.s.}}{\to}0, where Q(γ)=E[qi(γ)]Q(\gamma)=E[q_i(\gamma)] (Amemiya, 1985; Jennrich, 1969). In order to show that Q(γ)Q(\gamma) has a unique minimizer at the true value γ0\gamma_0, note that, for any γ∈Γ\gamma\in\Gamma,

Q(γ)=Q(γ0)+E[(ρi(γ)−ρi(γ0))′Wi(ρi(γ)−ρi(γ0))].Q(\gamma)=Q(\gamma_0)+E[(\rho_i(\gamma)-\rho_i(\gamma_0))^{\prime}W_i(\rho_i(\gamma)-\rho_i(\gamma_0))].

It follows that Q(γ)≥Q(γ0)Q(\gamma)\geq Q(\gamma_0), with equality holding if and only if

E[(ρi(γ)−ρi(γ0))′Wi(ρi(γ)−ρi(γ0))]=0.(15)E[(\rho_i(\gamma)-\rho_i(\gamma_0))^{\prime}W_i(\rho_i(\gamma)-\rho_i(\gamma_0))]=0. \tag*{(15)}

Since WiW_i is positive definite with probability one, the above equality holds if and only if

ρi(γ)−ρi(γ0)=[Xi′(β−β0),  (β−β0)′XiXi′(β+β0)+σ2−σ02]′=0\rho_i(\gamma)-\rho_i(\gamma_0)=\left[X_i^{\prime}(\beta-\beta_0),\;(\beta-\beta_0)^{\prime}X_iX_i^{\prime}(\beta+\beta_0)+\sigma^2-\sigma_0^2\right]^{\prime}=0

with probability one, implying β=β0\beta= \beta_0 and σ2=σ02\sigma^2 = \sigma_0^2. Finally, it follows from Lemma 1 that γ^→a.s.γ0\hat{\gamma} \xrightarrow{\mathrm{a.s.}} \gamma_0.

(2) First, by Taylor expansion of the score function, we have

Gn(γ^)=Gn(γ0)+Hn(γˉ)(γ^−γ0),(16)G_n(\hat{\gamma}) = G_n(\gamma_0) + H_n(\bar{\gamma})(\hat{\gamma} - \gamma_0), \tag*{(16)}

where ∥γˉ−γ0∥≤∥γ^−γ0∥\lVert\bar{\gamma} - \gamma_0\rVert\leq\lVert\hat{\gamma} - \gamma_0\rVert,

Gn(γ)=12∇γQn(γ)=∑i=1n∇γρi(γ)′Wiρi(γ),\begin{aligned} G_n(\gamma) &= \frac{1}{2}\nabla_{\gamma} Q_n(\gamma) \\ &= \sum_{i=1}^{n}\nabla_{\gamma}\rho_i(\gamma)^{\prime}W_i\rho_i(\gamma), \end{aligned}

and

Hn(γ)=12∇γγ′2Qn(γ)=∑i=1n∇γρi(γ)′Wi∇γ′ρi(γ)+(ρi(γ)′Wi⊗Ip+1)∇γ′vec⁡(∇γρi(γ)′).\begin{aligned} H_n(\gamma) &= \frac{1}{2}\nabla_{\gamma\gamma^{\prime}}^2 Q_n(\gamma) \\ &= \sum_{i=1}^{n}\nabla_{\gamma}\rho_i(\gamma)^{\prime}W_i\nabla_{\gamma^{\prime}}\rho_i(\gamma) + \left(\rho_i(\gamma)^{\prime}W_i \otimes I_{p+1}\right)\nabla_{\gamma^{\prime}}\operatorname{vec}\left(\nabla_{\gamma}\rho_i(\gamma)^{\prime}\right). \end{aligned}

Similar to the proof (1), we can show that sup⁡γ∣Hn(γ)/n−H(γ)∣→p0\sup_{\gamma}\lvert H_n(\gamma)/n - H(\gamma)\rvert\xrightarrow{p} 0, where H(γ)=E[∇γγ′2Q(γ)]H(\gamma) = E[\nabla_{\gamma\gamma^{\prime}}^2 Q(\gamma)], and hence, by the ULLN and Lemma 2, it holds

1nHn(γˉ)→pH(γ0)=E[∇γρi(γ0)′Wi∇γ′ρi(γ0)+(ρi(γ0)′Wi⊗Ip+1)∇γ′vec⁡(∇γρi(γ0)′)]=A,\begin{aligned} \frac{1}{n}H_n(\bar{\gamma}) &\xrightarrow{p} H(\gamma_0) \\ &= E\left[\nabla_{\gamma}\rho_i(\gamma_0)^{\prime}W_i\nabla_{\gamma^{\prime}}\rho_i(\gamma_0) + \left(\rho_i(\gamma_0)^{\prime}W_i \otimes I_{p+1}\right)\nabla_{\gamma^{\prime}}\operatorname{vec}\left(\nabla_{\gamma}\rho_i(\gamma_0)^{\prime}\right)\right] \\ &= A, \end{aligned}

where the last equality holds because the second order derivative ∇γ′vec⁡(∇γρi(γ)′)\nabla_{\gamma^{\prime}}\operatorname{vec}(\nabla_{\gamma}\rho_i(\gamma)^{\prime}) does not depend on YiY_i and hence the second term has expectation zero. It follows from (16) that

n(γ^−γ0)=A−11nGn(γ0)+op(1),(17)\sqrt{n}(\hat{\gamma} - \gamma_0) = A^{-1}\frac{1}{\sqrt{n}}G_n(\gamma_0) + o_p(1), \tag*{(17)}

Further, since E[∇γρi′(γ0)Wiρi(γ0)]=0E[\nabla_{\gamma}\rho_i^{\prime}(\gamma_0)W_i\rho_i(\gamma_0)] = 0, by the Central Limit Theorem, 1nGn(γ0)→dN(0,B)\frac{1}{\sqrt{n}}G_n(\gamma_0) \xrightarrow{d} N(0,B), where

B=E[∇γρi(γ0)′Wiρi(γ0)ρi(γ0)′Wi∇γ′ρi(γ0)]=E[∇γρi(γ0)′WiE[ρi(γ0)ρi(γ0)′∣Xi]Wi∇γ′ρi(γ0)]=E[∇γρi(γ0)′WiWi−1Wi∇γ′ρi(γ0)]=A.\begin{aligned} B &= E\left[\nabla_{\gamma}\rho_i(\gamma_0)^{\prime}W_i\rho_i(\gamma_0)\rho_i(\gamma_0)^{\prime}W_i\nabla_{\gamma^{\prime}}\rho_i(\gamma_0)\right] \\ &= E\left[\nabla_{\gamma}\rho_i(\gamma_0)^{\prime}W_iE[\rho_i(\gamma_0)\rho_i(\gamma_0)^{\prime}\mid X_i]W_i\nabla_{\gamma^{\prime}}\rho_i(\gamma_0)\right] \\ &= E\left[\nabla_{\gamma}\rho_i(\gamma_0)^{\prime}W_iW_i^{-1}W_i\nabla_{\gamma^{\prime}}\rho_i(\gamma_0)\right] \\ &= A. \end{aligned}

Thus, it follows from (17) that n(γ^−γ0)→dN(0,A−1)\sqrt{n}(\hat{\gamma}-\gamma_0)\mathrel{\mathop{\to}^{d}}N(0,A^{-1}). Finally, using the expression

∇ρi′(γ0)=−(Xi2(Xi′β0)Xi01),\nabla\rho_i'(\gamma_0)=-\begin{pmatrix}X_i & 2(X_i'\beta_0)X_i\\0 & 1\end{pmatrix},

we have

A=η((μ4−σ04)Mxx−μ3μx−μ3μx′−μ3μx′σ02).A=\eta\begin{pmatrix}(\mu_4-\sigma_0^4)M_{xx}-\mu_3\mu_x & -\mu_3\mu_x'\\-\mu_3\mu_x' & \sigma_0^2\end{pmatrix}.

Therefore the asymptotic covariance matrix C=A−1C=A^{-1} can be obtained by using the block matrix inversion formula.

Proof of Theorem 2

(1). First, since μ3≠0\mu_3\ne0, by A1 we have 0<ξ<10<\xi<1. Further, since

V(β^SLS)−1=1σ02(1−ξ)(Mxx−ξμxμx′),V(\hat{\beta}_{SLS})^{-1}=\frac{1}{\sigma_0^2(1-\xi)}(M_{xx}-\xi\mu_x\mu_x'),

and V(β^OLS)−1=1σ02MxxV(\hat{\beta}_{OLS})^{-1}=\frac{1}{\sigma_0^2}M_{xx}, we have

σ02[V(β^SLS)−1−V(β^OLS)−1]=ξ1−ξ(Mxx−μxμx′)=ξ1−ξΣxx\sigma_0^2[V(\hat{\beta}_{SLS})^{-1}-V(\hat{\beta}_{OLS})^{-1}]=\frac{\xi}{1-\xi}(M_{xx}-\mu_x\mu_x') =\frac{\xi}{1-\xi}\Sigma_{xx}

which is nonnegative definite, where Σxx\Sigma_{xx} is the covariance matrix of XX. It follows that V(β^OLS)−V(β^SLS)V(\hat{\beta}_{OLS})-V(\hat{\beta}_{SLS}) is nonnegative definite. Further, since

Σxx=Mxx1/2(I−Mxx−1/2μxμx′Mxx−1/2)Mxx1/2\Sigma_{xx}=M_{xx}^{1/2}(I-M_{xx}^{-1/2}\mu_x\mu_x'M_{xx}^{-1/2})M_{xx}^{1/2}

and Mxx−1/2μxμx′Mxx−1/2M_{xx}^{-1/2}\mu_x\mu_x'M_{xx}^{-1/2} has the same nonzero eigenvalue as μx′Mxx−1μx\mu_x'M_{xx}^{-1}\mu_x, Σxx\Sigma_{xx}, and therefore V(β^OLS)−V(β^SLS)V(\hat{\beta}_{OLS})-V(\hat{\beta}_{SLS}), is positive definite if and only if μx′Mxx−1μx≠1\mu_x'M_{xx}^{-1}\mu_x\ne1.

(2). First, since

1−μx′Mxx−1μx=E[(1−μx′Mxx−1X)2]≥0,1-\mu_x'M_{xx}^{-1}\mu_x=E[(1-\mu_x'M_{xx}^{-1}X)^2]\ge0,

we have μx′Mxx−1μx≤1\mu_x'M_{xx}^{-1}\mu_x\le1. It follows that

V(σ^OLS2)−V(σ^SLS2)=(μ4−σ04)(1−1−ξ1−ξμx′Mxx−1μx)≥0V(\hat{\sigma}_{OLS}^{2})-V(\hat{\sigma}_{SLS}^{2})=(\mu_4-\sigma_0^4)\left(1-\frac{1-\xi}{1-\xi\mu_x'M_{xx}^{-1}\mu_x}\right)\ge0

with equality holding if and only if $\mu_x'M_{xx}^{-1}\mu_x=1.

Derivation of the gradient G(γ)G(\gamma) and Hessian matrix H(γ)H(\gamma) of Qn(γ)Q_n(\gamma)

We derive the gradient and Hessian for the given data yi,xi,i=1,2,…,ny_i, x_i, i = 1, 2, \ldots, n. To evaluate the gradient, we first have

∇γρi′(γ)=−(xi2(xi′β)xi01),\nabla_{\gamma}\rho_i'(\gamma)=-\begin{pmatrix}x_i & 2(x_i'\beta)x_i \\ 0 & 1\end{pmatrix},
∇γρi′(γ)Wi=−η([μ4−σ4+2μ3(xi′β)]xi−μ3xi−2σ2(xi′β)−μ3σ2).\nabla_{\gamma}\rho_i'(\gamma)W_i=-\eta\begin{pmatrix}[\mu_4-\sigma^4+2\mu_3(x_i'\beta)]x_i & -\mu_3x_i \\ -2\sigma^2(x_i'\beta)-\mu_3 & \sigma^2\end{pmatrix}.

and therefore

∇γρi′(γ)Wiρi(γ)=−η([(μ4−σ4)(yi−xi′β)−μ3(yi−xi′β)2+μ3σ2]xiσ2(yi−xi′β)2−σ4−μ3(yi−xi′β)).\nabla_{\gamma}\rho_i'(\gamma)W_i\rho_i(\gamma)=-\eta\begin{pmatrix}[(\mu_4-\sigma^4)(y_i-x_i'\beta)-\mu_3(y_i-x_i'\beta)^2+\mu_3\sigma^2]x_i \\ \sigma^2(y_i-x_i'\beta)^2-\sigma^4-\mu_3(y_i-x_i'\beta)\end{pmatrix}.

Further, to evaluate the Hessian, we have

∇γρi′(γ)Wi∇γ′ρi(γ)=η((μ4−σ4)xixi′−μ3xi−μ3xi′σ2).\nabla_{\gamma}\rho_i'(\gamma)W_i\nabla_{\gamma}'\rho_i(\gamma)=\eta\begin{pmatrix}(\mu_4-\sigma^4)x_ix_i' & -\mu_3x_i \\ -\mu_3x_i' & \sigma^2\end{pmatrix}.

Since

∇γvec⁡(∇γρi′(γ))=−(00002xixi′000),\nabla_{\gamma}\operatorname{vec}(\nabla_{\gamma}\rho_i'(\gamma))=-\begin{pmatrix}0 & 0 \\ 0 & 0 \\ 2x_ix_i' & 0 \\ 0 & 0\end{pmatrix},

we have

(ρi′(γ)Wi⊗Ip+1)∇γ′vec⁡(∇γρi′(γ))=−2η([σ2(yi−xi′β)2−σ4−μ3(yi−xi′β)]xixi′000)(\rho_i'(\gamma)W_i\otimes I_{p+1})\nabla_{\gamma}'\operatorname{vec}(\nabla_{\gamma}\rho_i'(\gamma))=-2\eta\begin{pmatrix}[\sigma^2(y_i-x_i'\beta)^2-\sigma^4-\mu_3(y_i-x_i'\beta)]x_ix_i' & 0 \\ 0 & 0\end{pmatrix}

and therefore

Hi=η([μ4+σ4−2σ2(yi−xi′β)2+2μ3(yi−xi′β)]xixi′−μ3xi−μ3xi′σ2).H_i=\eta\begin{pmatrix}[\mu_4+\sigma^4-2\sigma^2(y_i-x_i'\beta)^2+2\mu_3(y_i-x_i'\beta)]x_ix_i' & -\mu_3x_i \\ -\mu_3x_i' & \sigma^2\end{pmatrix}.

It follows that

G(γ)=∑i=1n∇γρi′(γ)Wiρi(γ)=−η∑i=1n([(μ4−σ4)(yi−xi′β)−μ3(yi−xi′β)2+μ3σ2]xiσ2(yi−xi′β)2−σ4−μ3(yi−xi′β))\begin{aligned} G(\gamma)=\sum_{i=1}^{n}\nabla_{\gamma}\rho_i'(\gamma)W_i\rho_i(\gamma) \\ &=-\eta\sum_{i=1}^{n}\begin{pmatrix}[(\mu_4-\sigma^4)(y_i-x_i'\beta)-\mu_3(y_i-x_i'\beta)^2+\mu_3\sigma^2]x_i \\ \sigma^2(y_i-x_i'\beta)^2-\sigma^4-\mu_3(y_i-x_i'\beta)\end{pmatrix} \end{aligned}

and

H(γ)=∑i=1n∇γρi′(γ)Wi∇γ′ρi(γ)+(ρi′(γ)Wi⊗Ip+1)∇γ′vec⁡(∇γρi′(γ))=η∑i=1n([μ4+σ4−2σ2(yi−xi′β)2+2μ3(yi−xi′β)]xixi′−μ3xi−μ3xi′σ2).\begin{aligned} H(\gamma) &= \sum_{i=1}^{n} \nabla_{\gamma}\rho_i'(\gamma)W_i\nabla_{\gamma'}\rho_i(\gamma) + \left(\rho_i'(\gamma)W_i \otimes I_{p+1}\right)\nabla_{\gamma'}\operatorname{vec}\left(\nabla_{\gamma}\rho_i'(\gamma)\right) \\ &= \eta\sum_{i=1}^{n} \left( \begin{array}{cc} \left[\mu_4+\sigma^4-2\sigma^2\left(y_i-x_i'\beta\right)^2+2\mu_3\left(y_i-x_i'\beta\right)\right]x_ix_i' & -\mu_3x_i \\ -\mu_3x_i' & \sigma^2 \end{array} \right). \end{aligned}

References

  1. [1]Abarin, T. and L. Wang (2009). Second-order least squares estimation of censored regression models. Journal of Statistical Planning and Inference 139(2), 125–135.DOI
  2. [2]Beguería, S. and S. M. Vicente-Serrano (2006). Mapping the hazard of extreme rainfall by peaks over threshold extreme value analysis and spatial regression techniques. Journal of applied meteorology and climatology 45(1), 108–124.DOI
  3. [3]Bose, M. and R. Mukerjee (2015). Optimal design measures under asymmetric errors, with application to binary design points. Journal of Statistical Planning and Inference 159, 28–36.arxiv.org/abs/1405.3126
  4. [4]Castro-Camilo, D., R. Huser, and H. Rue (2022). Practical strategies for generalized extreme value-based regression models for extremes. Environmetrics 33(6), e2742.
  5. [5]Choi, M. (2018). Medical cost personal datasets. Kaggle dataset. Accessed 2026-06-16.
  6. [6]Cook, R. D. and S. Weisberg (1994). An Introduction to Regression Graphics. New York: John Wiley & Sons.
  7. [7]Gao, L. L. and J. Zhou (2014). New optimal design criteria for regression models with asymmetric errors. Journal of Statistical Planning and Inference 149, 140–151.DOI
  8. [8]Green, P. J. (1984). Iteratively reweighted least squares for maximum likelihood estimation, and some robust and resistant alternatives. Journal of the Royal Statistical Society: Series B (Methodological) 46(2), 149–170.
  9. [11]Kim, M. and Y. Ma (2012). The efficiency of the second-order nonlinear least squares estimator and its extension. Annals of the Institute of Statistical Mathematics 64(4), 751–764.pubmed.ncbi.nlm.nih.gov/23794748
  10. [12]Nandy, A., A. Basu, and A. Ghosh (2025). Robust inference for linear regression models with possibly skewed error distribution. Journal of Computational and Applied Mathematics 463, 116502.arxiv.org/abs/2404.03404
  11. [13]Salamh, M. and L. Wang (2021a). Second-order least squares estimation in nonlinear time series models with arch errors. Econometrics 9(4), 41.DOI
  12. [15]Telford, R. D. and R. B. Cunningham (1991). Sex, sport, and body-size dependency of hematology in highly trained athletes. Medicine and Science in Sports and Exercise 23(7), 788–794.pubmed.ncbi.nlm.nih.gov/1921671
  13. [16]Wang, L. (2003). Estimation of nonlinear berkson-type measurement error models. Statistica Sinica 13(4), 1201–1210.
  14. [17]Wang, L. (2004). Estimation of nonlinear models with berkson measurement errors. The Annals of Statistics 32(6), 2559–2579.
  15. [18]Wang, L. (2007). A unified approach to estimation of nonlinear mixed effects and berkson measurement error models. Canadian Journal of Statistics 35(2), 233–248.DOI
  16. [19]Wang, L. and A. Leblanc (2008). Second-order nonlinear least squares estimation. Annals of the Institute of Statistical Mathematics 60(4), 883–900.DOI
  17. [20]Wright, B., P. Green, and P. Braiden (1982). Quantitative analysis of delayed fracture observed in stress rate tests on brittle materials. Journal of Materials Science 17(11), 3227–3234.
  18. [21]Yeh, C.-K. and J. Zhou (2021). Properties of optimal regression designs under the second-order least squares estimator. Statistical Papers 62(1), 75–92.
  19. [22]Yin, Y. and J. Zhou (2017). Optimal designs for regression models using the second-order least squares estimator. Statistica Sinica, 1841–1856.DOI
  20. [23]Yousof, H., S. Jahanshahi, T. Ramires, G. Aryal, and G. Hamedani (2018). A new distribution for extreme values: regression model, characterizations and applications. Journal of Data Science 16(4), 677–706.

Paper details

vlnc:W_yzakgkt5
Version
v1 · latest
Published
Fresh — added to the Hub within the last 30 days
Added
License
CC BY 4.0
Usage

views · 9

downloads · 0

A view counts once per browser per day, a download each time the PDF is requested.

  • Mathematics

Not recorded for this paper

View statement
Looking for related papers…
Contents