Introduction
Independence testing under uncertainty about confounding
Many scientific questions reduce to testing whether a variable of interest is associated with an outcome. A recurring complication is uncertainty about confounding: factors that influence the outcome may or may not also be associated with the variable of interest, and their role can differ across comparisons. Single-cell genomics provides a prominent example.
Example 1 (Potential confounding of perturbation effects in CRISPR screens). Perturbation effects are assessed through associations between detected guide indicators
and gene expression. Batch (including biological replicate) and sequencing depth can affect both guide detection and measured expression ([2 , 3 ] ). In particular, Barry et al. [3 ] show that shared variation across replicates can create an apparent association even for a non-targeting guide, although technical factors need not confound every perturbation–gene pair.
Similar uncertainty arises in other large-scale association studies. In pooled microbiome studies, study origin can confound associations between a tested taxon’s abundance and disease to varying degrees across taxa ([15 ] ). In genetic association scans, ancestry can likewise confound genotype–phenotype associations to varying degrees across markers ([29 ] ).
To formalize this problem, let X \mathbf{X} X Y \mathbf{Y} Y Z \mathbf{Z} Z X \mathbf{X} X Z \mathbf{Z} Z Y \mathbf{Y} Y Z \mathbf{Z} Z H 0 M I : X ⊥ ⊥ ( Y , Z ) H_0^{\mathrm{MI}}: \mathbf{X} \perp\!\!\!\perp(\mathbf{Y},\mathbf{Z}) H 0 MI : X ⊥ ⊥ ( Y , Z ) Z \mathbf{Z} Z H 0 C I : X ⊥ ⊥ Y ∣ Z H_0^{\mathrm{CI}}: \mathbf{X} \perp\!\!\!\perp\mathbf{Y} \mid\mathbf{Z} H 0 CI : X ⊥ ⊥ Y ∣ Z
Marginal permutation shuffles X \mathbf{X} X ( Y , Z ) (\mathbf{Y},\mathbf{Z}) ( Y , Z ) [23 ] ). Under CI, however, X \mathbf{X} X Z \mathbf{Z} Z [31 ] ). This motivates the following question:
Can marginal permutation tests retain finite-sample validity under MI while achieving (asymptotic) validity under a suitably structured CI null?
In this paper, we set out to answer this question in the affirmative by establishing a novel general validity guarantee: for appropriately constructed and empirically studentized generalized scores, marginal permutation is asymptotically valid under CI—despite disrupting the X \mathbf{X} X Z \mathbf{Z} Z
Contributions of this work
Concretely, we make three main contributions.
A validity framework for marginally permuted score statistics. We formulate a framework based on generalized scores in which marginal permutation recomputes the weight for the exposure while holding the residual for the outcome fixed. Under the stronger null H 0 M I H_0^{\mathrm{MI}} H 0 MI X \mathbf{X} X Z \mathbf{Z} Z
framework accommodates generalized linear model (GLM) scores, generalized covariance measures (GCM, Shah and Peters [31 ] ), and projected covariance measures (PCM, Lundborg et al. [26 ] ).
Permuted GLM score tests robust to variance misspecification. By separately studentizing the observed GLM score and each marginal permutation using their respective empirical score variances, we obtain a new score test that is asymptotically valid under a correct null conditional mean model and the stated regularity conditions, without requiring CI null. Moreover, our procedure allows arbitrary misspecification of the working conditional variance. Under marginal independence, our proposed test is valid in finite samples even when the working likelihood is completely misspecified.
Doubly robust tests calibrated by permutation. We develop versions of the GCM and PCM calibrated by permutation with nuisance functions estimated from the data. Under conditional independence, our theory establishes asymptotic validity, subject to the double robustness assumptions of the original GCM and PCM procedures and an additional condition on the stability of refitting after permutation. We give sufficient conditions for this stability for ordinary least squares, linear ridge regression, kernel ridge regression, and, separately, the lasso. Under marginal independence, both procedures are valid in finite samples without any accuracy requirements on nuisance estimates.
For the permuted GLM, GCM, and PCM testing procedures, we further show that permutation calibration preserves first-order local power in the local alternative regimes. Simulations, a semisynthetic experiment, and a single-cell CRISPR screen analysis demonstrate substantial calibration gains when the effective sample size is small. Figure 1 previews this behavior in the semisynthetic Cancer Cell Line Encyclopedia (CCLE) analysis and demonstrates the advantage of our proposed test.
Figure 1. Semisynthetic CCLE calibration and power for EIF5A, a mutation target with high sparsity, at s = 10 s = 10 s = 10
Related work
Our work lies at the intersection of several strands of the literature, yet differs substantially from each in its resampling mechanism and validity guarantees.
Robust permutation with studentization Studentized permutation tests are precedents for our result giving both validity guarantees. This literature uses studentization to combine finite-sample exactness under a strong null invariant under permutation with asymptotic validity for a weaker hypothesis [20 , 10 , 12 , 38 ] . Toulis [32 ] provides a recent general treatment of approximate randomization tests, including asymptotic and nonasymptotic guarantees when unobserved invariant variables are replaced by estimated proxies such as regression residuals. Barry et al. [3 ] empirically investigated this robustness phenomenon in single-cell CRISPR screens. Barry et al. [4 ] established the corresponding validity theory for a score test with information studentization in a specialized negative binomial model motivated by bulk RNA-seq differential expression analyses. Our contribution is a general theory for establishing the asymptotic validity of marginal permutation tests based on empirically studentized generalized scores under explicit oracle conditions (Theorem 1). We instantiate this theory for GLM, GCM, and PCM statistics, leading to a new class of testing procedures (§ 3 and § 4 ). § 3.3 compares our approach with the score test using information studentization proposed by Barry et al. [4 ] .
Conditional resampling and permutation methods for CI testing Conditional resampling methods instead seek to preserve the exposure–covariate relationship. The conditional randomization tests (CRTs, [9 , 25 ] ) are exact with a known conditional exposure law, but guarantees with an estimated exposure law depend on the model or hold asymptotically [27 , 24 ] . Permutation methods instead change how values are shuffled across observations to account for Z Z Z Z Z Z [5 , 21 ] . Other methods use an assumed increasing or decreasing relationship with Z Z Z [19 ] . By contrast, our marginal permutations are unrestricted and deliberately break that relationship: validity under the CI null rests on the structure of the generalized scores rather than on a simulated conditional null, so no fully specified generative model for X ∣ Z X \mid Z X ∣ Z X ∣ Z X \mid Z X ∣ Z
Robust GLM inference Robust GLM inference provides a close point of comparison. Quasi-likelihood and sandwich procedures seek validity under variance or likelihood misspecification [35 , 36 ] . Empirical studentization of the observed score considered in this paper has important precursors in robust score tests based on empirical covariance estimates [8 , 7 ] . earlier tests use regular parametric or quasi-likelihood nuisance estimation and asymptotic calibration. Recent sign-flip procedures provide related robustness results using resampling [17 , 11 ] . Our GLM tests permit general higher-order moment misspecification while preserving the desirable dual validity: asymptotic validity under the specified conditional mean null together with finite-sample validity under the MI null, a novel property not shared by the earlier robust score tests.
Organization and notation
The remainder of the paper is organized as follows. § 2 introduces the generalized score framework, its finite-sample validity under MI, and a theorem based on approximation by oracle scores for asymptotic validity. § 3 and § 4 develop the GLM score test robust to variance misspecification and the GCM and PCM tests with permutation calibration, respectively. § 5 and § 6 evaluate these methods through simulations and genomic applications. The supplementary materials provide implementation details, additional experiments, regularity conditions, proofs, and analyses of local power.
Notation We observe n n n ( X i , Y i , Z i ) i = 1 n (X_i,Y_i,Z_i)_{i=1}^{n} ( X i , Y i , Z i ) i = 1 n ( X , Y , Z ) (\mathbf{X},\mathbf{Y},\mathbf{Z}) ( X , Y , Z ) X , Y ∈ R \mathbf{X},\mathbf{Y}\in\mathbb{R} X , Y ∈ R Z ∈ R p \mathbf{Z}\in\mathbb{R}^{p} Z ∈ R p P X P_X P X P Z P_Z P Z X \mathbf{X} X Z \mathbf{Z} Z P X Z P_{XZ} P X Z X = ( X 1 , … , X n ) T \mathbf{X}=(X_1,\ldots,X_n)^{\mathsf{T}} X = ( X 1 , … , X n ) T Y = ( Y 1 , … , Y n ) T \mathbf{Y}=(Y_1,\ldots,Y_n)^{\mathsf{T}} Y = ( Y 1 , … , Y n ) T Z ∈ R n × p \mathbf{Z}\in\mathbb{R}^{n\times p} Z ∈ R n × p Z i T Z_i^{\mathsf{T}} Z i T G \mathcal{G} G { 1 , … , n } \{1,\ldots,n\} { 1 , … , n } π ∈ G \pi\in\mathcal{G} π ∈ G Π \Pi Π Π b \Pi_b Π b G \mathcal{G} G b b b → d \xrightarrow{d} d → p \xrightarrow{p} p → a . s . \xrightarrow{\mathrm{a.s.}} a.s. F n \mathcal{F}_n F n T n T_n T n E Π \mathbb{E}_{\Pi} E Π Π \Pi Π F n \mathcal{F}_n F n
W n ∣ F n → d , p W ⟺ P ( W n ≤ t ∣ F n ) → p P ( W ≤ t ) , W_n\mid\mathcal{F}_n\xrightarrow{d,p}W\quad\Longleftrightarrow\quad\mathbb{P}(W_n\le t\mid\mathcal{F}_n)\xrightarrow{p}\mathbb{P}(W\le t), W n ∣ F n d , p W ⟺ P ( W n ≤ t ∣ F n ) p P ( W ≤ t ) , where the convergence on the right holds at every continuity point t t t W W W α ∈ ( 0 , 1 ) \alpha\in(0,1) α ∈ ( 0 , 1 ) Q 1 − α ( ⋅ ∣ F n ) Q_{1-\alpha}(\cdot\mid\mathcal{F}_n) Q 1 − α ( ⋅ ∣ F n ) ( 1 − α ) (1-\alpha) ( 1 − α ) φ n α ∈ { 0 , 1 } \varphi_n^\alpha\in\{0,1\} φ n α ∈ { 0 , 1 }
Permutation for generalized score statistics
Our general testing framework is based on generalized score statistics , encompassing a broad class of test statistics for conditional independence. Concretely, given independent observations ( X i , Y i , Z i ) (X_i,Y_i,Z_i) ( X i , Y i , Z i ) i = 1 , … , n i=1,\ldots,n i = 1 , … , n weight v ^ i \hat{v}_i v ^ i X i X_i X i response residual ϵ ^ i \hat{\epsilon}_i ϵ ^ i Y i Y_i Y i R ^ i ≡ ν ^ i ϵ ^ i \hat{R}_i \equiv\hat{\nu}_i\hat{\epsilon}_i R ^ i ≡ ν ^ i ϵ ^ i
T n ≡ 1 n ∑ i = 1 n R ^ i V ^ n , where V ^ n ≡ 1 n ∑ i = 1 n R ^ i 2 − ( 1 n ∑ i = 1 n R ^ i ) 2 . (1) T_n \equiv\frac{\frac{1}{\sqrt{n}}\sum_{i=1}^{n}\hat{R}_i}{\sqrt{\hat{V}_n}}, \qquad\text{where}\quad\hat{V}_n \equiv\frac{1}{n}\sum_{i=1}^{n}\hat{R}_i^2-\left(\frac{1}{n}\sum_{i=1}^{n}\hat{R}_i\right)^2. \tag*{(1)} T n ≡ V ^ n n 1 ∑ i = 1 n R ^ i , where V ^ n ≡ n 1 i = 1 ∑ n R ^ i 2 − ( n 1 i = 1 ∑ n R ^ i ) 2 . (1) We use the empirical variance operator S n : R n → R S_n:\mathbb{R}^n\to\mathbb{R} S n : R n → R S n ( a ) = n − 1 ∑ i = 1 n a i 2 − ( n − 1 ∑ i = 1 n a i ) 2 S_n(a)=n^{-1}\sum_{i=1}^{n}a_i^2-(n^{-1}\sum_{i=1}^{n}a_i)^2 S n ( a ) = n − 1 ∑ i = 1 n a i 2 − ( n − 1 ∑ i = 1 n a i ) 2 S n ( R ^ ) = V ^ n S_n(\hat{R})=\hat{V}_n S n ( R ^ ) = V ^ n R ^ ≡ ( R ^ 1 , … , R ^ n ) \hat{R}\equiv(\hat{R}_1,\ldots,\hat{R}_n) R ^ ≡ ( R ^ 1 , … , R ^ n ) ( ν ^ i , ϵ ^ i , V ^ n ) (\hat{\nu}_i,\hat{\epsilon}_i,\hat{V}_n) ( ν ^ i , ϵ ^ i , V ^ n )
Permutation calibration for generalized score test statistic
Permutation calibration holds the fitted response residuals ϵ ^ i \hat{\epsilon}_i ϵ ^ i ν ^ i \hat{\nu}_i ν ^ i X X X π ∈ G \pi\in G π ∈ G ν ^ i ( π ) \hat{\nu}_i^{(\pi)} ν ^ i ( π ) X i X_i X i X π ( i ) X_{\pi(i)} X π ( i ) X X X
R ^ i ( π ) ≡ ν ^ i ( π ) ϵ ^ i , V ^ n ( π ) ≡ S n ( R ^ ( π ) ) , T n ( π ) ≡ n − 1 / 2 ∑ i = 1 n R ^ i ( π ) V ^ n ( π ) . (2) \hat{R}_i^{(\pi)}\equiv\hat{\nu}_i^{(\pi)}\hat{\epsilon}_i,\qquad\hat{V}_n^{(\pi)}\equiv S_n(\hat{R}^{(\pi)}),\qquad T_n^{(\pi)}\equiv\frac{n^{-1/2}\sum_{i=1}^{n}\hat{R}_i^{(\pi)}}{\sqrt{\hat{V}_n^{(\pi)}}}. \tag*{(2)} R ^ i ( π ) ≡ ν ^ i ( π ) ϵ ^ i , V ^ n ( π ) ≡ S n ( R ^ ( π ) ) , T n ( π ) ≡ V ^ n ( π ) n − 1/2 ∑ i = 1 n R ^ i ( π ) . (2) With Π \Pi Π Q 1 − α ( T n ( Π ) ∣ F n ) Q_{1-\alpha}(T_n^{(\Pi)}\mid\mathcal{F}_n) Q 1 − α ( T n ( Π ) ∣ F n ) ( 1 − α ) (1-\alpha) ( 1 − α ) T n ( Π ) T_n^{(\Pi)} T n ( Π ) α \alpha α
φ n α ≡ 1 { T n > Q 1 − α ( T n ( Π ) ∣ F n ) } = 1. (3) \varphi_n^\alpha\equiv\mathbb{1}\left\{T_n>Q_{1-\alpha}(T_n^{(\Pi)}\mid\mathcal{F}_n)\right\}=1. \tag*{(3)} φ n α ≡ 1 { T n > Q 1 − α ( T n ( Π ) ∣ F n ) } = 1. (3) Algorithm 1 gives the generic implementation.
Algorithm 1: Generalized score test calibrated by permutation
Input: Data ( X i , Y i , Z i ) i = 1 n (X_i,Y_i,Z_i)_{i=1}^{n} ( X i , Y i , Z i ) i = 1 n α ∈ ( 0 , 1 ) \alpha\in(0,1) α ∈ ( 0 , 1 ) B ∈ N B\in\mathbb{N} B ∈ N
Fit any nuisance estimators required by ν ^ i \hat{\nu}_i ν ^ i ϵ ^ i \hat{\epsilon}_i ϵ ^ i R ^ i = ν ^ i ϵ ^ i \hat{R}_i=\hat{\nu}_i\hat{\epsilon}_i R ^ i = ν ^ i ϵ ^ i T n T_n T n (1) ;
for b = 1 , … , B b=1,\ldots,B b = 1 , … , B do
Draw a permutation Π b \Pi_b Π b { 1 , … , n } \{1,\ldots,n\} { 1 , … , n }
Refit X Π b ( i ) ∣ Z i X_{\Pi_b(i)}\mid Z_i X Π b ( i ) ∣ Z i ( X Π b ( i ) , Z i ) i = 1 n (X_{\Pi_b(i)},Z_i)_{i=1}^{n} ( X Π b ( i ) , Z i ) i = 1 n ν ^ i ( Π b ) \hat{\nu}_i^{(\Pi_b)} ν ^ i ( Π b )
Form R ^ i ( Π b ) = ν ^ i ( Π b ) ϵ ^ i \hat{R}_i^{(\Pi_b)}=\hat{\nu}_i^{(\Pi_b)}\hat{\epsilon}_i R ^ i ( Π b ) = ν ^ i ( Π b ) ϵ ^ i T n ( Π b ) T_n^{(\Pi_b)} T n ( Π b ) T n T_n T n
end
Compute the (right-sided) p p p p = 1 B + 1 ( 1 + ∑ b = 1 B 1 { T n ( Π b ) ≥ T n } ) p=\frac{1}{B+1}\left(1+\sum_{b=1}^{B}\mathbb{1}\left\{T_n^{(\Pi_b)}\geq T_n\right\}\right) p = B + 1 1 ( 1 + ∑ b = 1 B 1 { T n ( Π b ) ≥ T n } )
Reject the null hypothesis at level α \alpha α p ≤ α p\leq\alpha p ≤ α
Finite-sample validity under H 0 M I H_0^{\mathrm{MI}} H 0 MI
The first validity result for the permutation calibration holds at any sample size and regardless of the choice of weight, response residual, or nuisance estimator.
Proposition 1 (Finite-sample validity under a null invariant under permutation). Let T n T_n T n G \mathcal{G} G (3) calibrated by permutation is valid in finite samples: for every n n n α ∈ ( 0 , 1 ) \alpha\in(0,1) α ∈ ( 0 , 1 )
P ( φ n α = 1 ) ≤ α , P ( p ≤ α ) ≤ α . \mathbb{P}(\varphi_n^\alpha=1) \leq\alpha,\qquad\mathbb{P}(p\leq\alpha) \leq\alpha. P ( φ n α = 1 ) ≤ α , P ( p ≤ α ) ≤ α . In particular, this applies under H 0 M I H_0^{\mathrm{MI}} H 0 MI X ⊥ ⊥ ( Y , Z ) \mathbf{X}\perp\!\!\!\perp(\mathbf{Y},\mathbf{Z}) X ⊥ ⊥ ( Y , Z ) G \mathcal{G} G { 1 , … , n } \{1,\ldots,n\} { 1 , … , n }
A proof of Proposition 1 can be found in Hemerik and Goeman [16 ] . As long as the joint distribution of the data is invariant under the permutation group G \mathcal{G} G T n ( Π b ) T_n^{(\Pi_b)} T n ( Π b ) T n T_n T n p p p
Asymptotic validity via oracle approximation
Marginal permutation can change both the nuisance targets and the variance of the score. The following theorem gives a general criterion under which empirical studentization nevertheless yields a common standard normal limit for the observed and permuted statistics, ensuring asymptotically valid permutation calibration. These conditions are verified below under the conditional mean null for the GLM score and under conditional independence for the GCM and PCM.
Theorem 1 (Asymptotic validity of permuted product scores). Let T n T_n T n T n ( Π ) T_n^{(\Pi)} T n ( Π ) (1) and (2) , respectively. Suppose there exist an auxiliary sigma-field H n ⊆ F n \mathcal{H}_n\subseteq\mathcal{F}_n H n ⊆ F n U n 1 , … , U n n U_{n1},\ldots,U_{nn} U n 1 , … , U nn ( C n , i k ) 1 ≤ i , k ≤ n (C_{n,ik})_{1\leq i,k\leq n} ( C n , ik ) 1 ≤ i , k ≤ n F n \mathcal{F}_n F n ∑ k = 1 n C n , i k = 0 \sum_{k=1}^n C_{n,ik}=0 ∑ k = 1 n C n , ik = 0 i = 1 , … , n i=1,\ldots,n i = 1 , … , n
(i) Observed oracle regularity. Conditional on H n \mathcal{H}_n H n U n 1 , … , U n n U_{n1},\ldots,U_{nn} U n 1 , … , U nn E [ U n 1 ∣ H n ] = 0 \mathbb{E}[U_{n1}\mid\mathcal{H}_n]=0 E [ U n 1 ∣ H n ] = 0 σ s , n 2 ≡ E [ U n 1 2 ∣ H n ] \sigma_{s,n}^2\equiv\mathbb{E}[U_{n1}^2\mid\mathcal{H}_n] σ s , n 2 ≡ E [ U n 1 2 ∣ H n ] P ( σ s , n 2 > 0 ) → 1 \mathbb{P}(\sigma_{s,n}^2>0)\to1 P ( σ s , n 2 > 0 ) → 1
E [ U n 1 4 ∣ H n ] n σ s , n 4 = o P ( 1 ) . \frac{\mathbb{E}[U_{n1}^4\mid\mathcal{H}_n]}{n\sigma_{s,n}^4}=o_{\mathbb{P}}(1). n σ s , n 4 E [ U n 1 4 ∣ H n ] = o P ( 1 ) . (ii) Permutation oracle regularity. Define σ p , n 2 ≡ 1 n 2 ∑ i = 1 n ∑ k = 1 n C n , i k 2 \sigma_{p,n}^2\equiv\frac{1}{n^2}\sum_{i=1}^n\sum_{k=1}^n C_{n,ik}^2 σ p , n 2 ≡ n 2 1 ∑ i = 1 n ∑ k = 1 n C n , ik 2 P ( σ p , n 2 > 0 ) → 1 \mathbb{P}(\sigma_{p,n}^2>0)\to1 P ( σ p , n 2 > 0 ) → 1 C n , ⋅ k ≡ 1 n ∑ i = 1 n C n , i k C_{n,\cdot k}\equiv\frac{1}{n}\sum_{i=1}^n C_{n,ik} C n , ⋅ k ≡ n 1 ∑ i = 1 n C n , ik
∑ k = 1 n C n , ⋅ k 2 ( n − 1 ) σ p , n 2 = o P ( 1 ) , max 1 ≤ i , k ≤ n C n , i k 2 n σ p , n 2 = o P ( 1 ) . \frac{\sum_{k=1}^n C_{n,\cdot k}^2}{(n-1)\sigma_{p,n}^2}=o_{\mathbb{P}}(1),\qquad\frac{\max_{1\leq i,k\leq n}C_{n,ik}^2}{n\sigma_{p,n}^2}=o_{\mathbb{P}}(1). ( n − 1 ) σ p , n 2 ∑ k = 1 n C n , ⋅ k 2 = o P ( 1 ) , n σ p , n 2 max 1 ≤ i , k ≤ n C n , ik 2 = o P ( 1 ) . (iii) Approximation of fitted scores by oracle scores. Writing Δ s , n i ≡ R ^ i − U n i \Delta_{s,ni}\equiv\widehat{R}_i-U_{ni} Δ s , ni ≡ R i − U ni
and Δ p , n i ≡ R ^ i ( Π ) − C n , i Π ( i ) \Delta_{p,ni} \equiv\widehat{R}^{(\Pi)}_i - C_{n,i}\Pi(i) Δ p , ni ≡ R i ( Π ) − C n , i Π ( i ) r ∈ { s , p } r \in\{s,p\} r ∈ { s , p }
1 n σ r , n ∑ i = 1 n Δ r , n i = o P ( 1 ) , 1 n σ r , n 2 ∑ i = 1 n Δ r , n i 2 = o P ( 1 ) . \frac{1}{\sqrt{n}\sigma_{r,n}}\sum_{i=1}^{n}\Delta_{r,ni}=o_P(1), \qquad\frac{1}{n\sigma_{r,n}^{2}}\sum_{i=1}^{n}\Delta_{r,ni}^{2}=o_P(1). n σ r , n 1 i = 1 ∑ n Δ r , ni = o P ( 1 ) , n σ r , n 2 1 i = 1 ∑ n Δ r , ni 2 = o P ( 1 ) . For r = p r=p r = p o P ( 1 ) o_P(1) o P ( 1 ) Π \Pi Π
Then, T n → d N ( 0 , 1 ) T_n \xrightarrow{d} N(0,1) T n d N ( 0 , 1 ) T n ( Π ) ∣ F n → d P N ( 0 , 1 ) T_n^{(\Pi)} \mid\mathcal{F}_n \xrightarrow{d_P} N(0,1) T n ( Π ) ∣ F n d P N ( 0 , 1 ) Q 1 − α ( T n ( Π ) ∣ F n ) → p z 1 − α Q_{1-\alpha}(T_n^{(\Pi)} \mid\mathcal{F}_n) \xrightarrow{p} z_{1-\alpha} Q 1 − α ( T n ( Π ) ∣ F n ) p z 1 − α P { T n > Q 1 − α ( T n ( Π ) ∣ F n ) } → α \mathbb{P}\{T_n > Q_{1-\alpha}(T_n^{(\Pi)} \mid\mathcal{F}_n)\} \to\alpha P { T n > Q 1 − α ( T n ( Π ) ∣ F n )} → α α ∈ ( 0 , 1 ) \alpha\in(0,1) α ∈ ( 0 , 1 )
Theorem 1 builds on the studentized permutation theory of Chung and Romano [10 ] and DiCiccio and Romano [12 ] . The proof is given in Appendix § 15 .§ 2.1 . We now present the following discussion of the conditions in Theorem 1.
Remark 1 (Conditions in Theorem 1). Theorem 1 gives conditions under which oracle scores approximate marginally permuted product scores with estimated nuisance functions. The two array conditions in part (ii) ensure that column centering has a negligible effect on the permutation variance and that no individual array entry dominates. For both the observed (r = s r=s r = s r = p r=p r = p σ r , n \sigma_{r,n} σ r , n σ r , n 2 \sigma_{r,n}^{2} σ r , n 2 [20 , 10 , 12 , 38 ] ) and related resampling procedures ([17 , 11 , 27 ] ).
Remark 2 (Auxiliary sigma-field H n \mathcal{H}_n H n H n \mathcal{H}_n H n
GLM score test calibrated by permutation
We first instantiate Theorem 1 using a working generalized linear model from an exponential family. Let ϑ \vartheta ϑ λ \lambda λ
p n a t ( y ; ω , ϑ ) ≡ h ϑ ( y ) e ω y Z ϑ ( ω ) = h ϑ ( y ) exp { ω y − A ϑ ( ω ) } , p_{\mathrm{nat}}(y;\omega,\vartheta) \equiv\frac{h_{\vartheta}(y)e^{\omega y}}{Z_{\vartheta}(\omega)} = h_{\vartheta}(y)\exp\{\omega y-A_{\vartheta}(\omega)\}, p nat ( y ; ω , ϑ ) ≡ Z ϑ ( ω ) h ϑ ( y ) e ω y = h ϑ ( y ) exp { ω y − A ϑ ( ω )} , where Z ϑ ( ω ) ≡ ∫ e ω u h ϑ ( u ) d λ ( u ) Z_{\vartheta}(\omega) \equiv\int e^{\omega u}h_{\vartheta}(u)\,d\lambda(u) Z ϑ ( ω ) ≡ ∫ e ω u h ϑ ( u ) d λ ( u ) A ϑ ( ω ) ≡ log Z ϑ ( ω ) A_{\vartheta}(\omega) \equiv\log Z_{\vartheta}(\omega) A ϑ ( ω ) ≡ log Z ϑ ( ω ) ω \omega ω 0 < Z ϑ ( ω ) < ∞ 0 < Z_{\vartheta}(\omega) < \infty 0 < Z ϑ ( ω ) < ∞ A ϑ ′ ′ ( ω ) > 0 A_{\vartheta}^{\prime\prime}(\omega) > 0 A ϑ ′′ ( ω ) > 0 M \mathcal{M} M A ϑ ′ A_{\vartheta}^{\prime} A ϑ ′ p ( y ; μ , ϑ ) ≡ p n a t ( y ; ( A ϑ ′ ) − 1 ( μ ) , ϑ ) p(y;\mu,\vartheta) \equiv p_{\mathrm{nat}}(y;(A_{\vartheta}^{\prime})^{-1}(\mu),\vartheta) p ( y ; μ , ϑ ) ≡ p nat ( y ; ( A ϑ ′ ) − 1 ( μ ) , ϑ ) ( A ϑ ′ ) − 1 (A_{\vartheta}^{\prime})^{-1} ( A ϑ ′ ) − 1 A ϑ ′ A_{\vartheta}^{\prime} A ϑ ′ g : M → R g:\mathcal{M}\to\mathbb{R} g : M → R γ ∈ R \gamma\in\mathbb{R} γ ∈ R β ∈ R p \beta\in\mathbb{R}^{p} β ∈ R p
Y ∣ X = x , Z = z ∼ p ( ⋅ ; g − 1 ( x γ + z ⊤ β ) , ϑ ) . Y \mid X=x, Z=z \sim p\left(\cdot; g^{-1}(x\gamma+z^{\top}\beta),\vartheta\right). Y ∣ X = x , Z = z ∼ p ( ⋅ ; g − 1 ( x γ + z ⊤ β ) , ϑ ) . The variance function induced by the working exponential family is V ϑ ( μ ) ≡ A ϑ ′ ′ ( ( A ϑ ′ ) − 1 ( μ ) ) V_{\vartheta}(\mu) \equiv A_{\vartheta}^{\prime\prime}((A_{\vartheta}^{\prime})^{-1}(\mu)) V ϑ ( μ ) ≡ A ϑ ′′ (( A ϑ ′ ) − 1 ( μ )) γ \gamma γ β \beta β Z Z Z H 0 C I : γ = 0 H_{0}^{\mathrm{CI}}:\gamma=0 H 0 CI : γ = 0 [9 ] . Differentiating the exponential family density parameterized by its mean gives, for b ∈ R p b \in\mathbb{R}^{p} b ∈ R p
s i ( b ; ϑ ) ≡ Y i − μ i ( b ) V ϑ ( μ i ( b ) ) g ′ ( μ i ( b ) ) where μ i ( b ) ≡ g − 1 ( Z i ⊤ b ) . s_i(b;\vartheta) \equiv\frac{Y_i-\mu_i(b)}{V_{\vartheta}(\mu_i(b))g^{\prime}(\mu_i(b))} \quad\text{where} \quad\mu_i(b) \equiv g^{-1}(Z_i^{\top}b). s i ( b ; ϑ ) ≡ V ϑ ( μ i ( b )) g ′ ( μ i ( b )) Y i − μ i ( b ) where μ i ( b ) ≡ g − 1 ( Z i ⊤ b ) . The null estimator β ^ \widehat{\beta} β
U n ( b ; ϑ ^ ) ≡ 1 n ∑ i = 1 n Z i s i ( b ; ϑ ^ ) , U n ( β ^ ; ϑ ^ ) = 0 (4) U_n(b;\widehat{\vartheta}) \equiv\frac{1}{n}\sum_{i=1}^{n} Z_i s_i(b;\widehat{\vartheta}), \quad U_n(\widehat{\beta};\widehat{\vartheta})=0 \tag*{(4)} U n ( b ; ϑ ) ≡ n 1 i = 1 ∑ n Z i s i ( b ; ϑ ) , U n ( β ; ϑ ) = 0 (4) whenever a root exists.
Define μ ^ i ≡ μ i ( β ^ ) \widehat{\mu}_i \equiv\mu_i(\widehat{\beta}) μ i ≡ μ i ( β ) ϵ ^ i ≡ s i ( β ^ ; ϑ ^ ) \widehat{\epsilon}_i \equiv s_i(\widehat{\beta};\widehat{\vartheta}) ϵ i ≡ s i ( β ; ϑ ) W ^ i ≡ 1 / ( V ϑ ^ ( μ ^ i ) g ′ ( μ ^ i ) 2 ) \widehat{W}_i \equiv1/(V_{\widehat{\vartheta}}(\widehat{\mu}_i)g^{\prime}(\widehat{\mu}_i)^2) W i ≡ 1/ ( V ϑ ( μ i ) g ′ ( μ i ) 2 ) W ^ = diag ( W ^ 1 , … , W ^ n ) \widehat{W}=\operatorname{diag}(\widehat{W}_1,\ldots,\widehat{W}_n) W = diag ( W 1 , … , W n ) W ^ i \widehat{W}_i W i
Permuted score test with working GLM likelihood
To display the GLM fit in the generalized score template (1) , we consider the residuals of the treatment against the nuisance design using the same working information weights: for i = 1 , … , n i=1,\ldots,n i = 1 , … , n
θ ^ ≡ ( Z ⊤ W ^ Z ) − 1 Z ⊤ W ^ X and ν ^ i G L M ≡ X i − Z i ⊤ θ ^ . (5) \widehat{\theta} \equiv(Z^{\top}\widehat{W}Z)^{-1}Z^{\top}\widehat{W}X \quad\text{and} \quad\widehat{\nu}^{\mathrm{GLM}}_i \equiv X_i-Z_i^{\top}\widehat{\theta}. \tag*{(5)} θ ≡ ( Z ⊤ W Z ) − 1 Z ⊤ W X and ν i GLM ≡ X i − Z i ⊤ θ . (5) Define R ^ G L M ≡ ( R ^ 1 G L M , … , R ^ n G L M ) \widehat{R}^{\mathrm{GLM}}\equiv(\widehat{R}^{\mathrm{GLM}}_1,\ldots,\widehat{R}^{\mathrm{GLM}}_n) R GLM ≡ ( R 1 GLM , … , R n GLM ) R ^ i G L M ≡ ν ^ i G L M ϵ ^ i \widehat{R}^{\mathrm{GLM}}_i\equiv\widehat{\nu}^{\mathrm{GLM}}_i\widehat{\epsilon}_i R i GLM ≡ ν i GLM ϵ i (1) :
T n G L M ≡ 1 n ∑ i = 1 n R ^ i G L M V ^ n G L M where V ^ n G L M ≡ S n ( R ^ G L M ) . (6) T_n^{\mathrm{GLM}} \equiv\frac{\frac{1}{\sqrt{n}}\sum_{i=1}^{n}\widehat{R}^{\mathrm{GLM}}_i}{\sqrt{\widehat{V}^{\mathrm{GLM}}_n}} \quad\text{where} \quad\widehat{V}^{\mathrm{GLM}}_n \equiv S_n(\widehat{R}^{\mathrm{GLM}}). \tag*{(6)} T n GLM ≡ V n GLM n 1 ∑ i = 1 n R i GLM where V n GLM ≡ S n ( R GLM ) . (6) ( T n G L M , V ^ n G L M ) (T_n^{\mathrm{GLM}},\widehat{V}^{\mathrm{GLM}}_n) ( T n GLM , V n GLM ) ( T n , V ^ n ) (T_n,\widehat{V}_n) ( T n , V n ) (1) . Algorithm 2 presents the resulting GLM score test calibrated by permutation.
Remark 3 (Computational shortcut in Algorithm 2). Algorithm 2 treats the realized null fit ( ϑ ^ , β ^ ) (\widehat{\vartheta},\widehat{\beta}) ( ϑ , β ) ϵ ^ i \widehat{\epsilon}_i ϵ i W ^ i \widehat{W}_i W i ( Z ⊤ W ^ Z ) − 1 Z ⊤ W ^ (Z^{\top}\widehat{W}Z)^{-1}Z^{\top}\widehat{W} ( Z ⊤ W Z ) − 1 Z ⊤ W (7) . We discuss the computational and theoretical consequences when X X X ϑ ^ \widehat{\vartheta} ϑ
Algorithm 2: The GLM score permutation test
Input: Data ( X i , Y i , Z i ) i = 1 n (X_i,Y_i,Z_i)_{i=1}^n ( X i , Y i , Z i ) i = 1 n Z i ∈ R p Z_i \in\mathbb{R}^p Z i ∈ R p α ∈ ( 0 , 1 ) \alpha\in(0,1) α ∈ ( 0 , 1 ) B ∈ N B \in\mathbb{N} B ∈ N p p p g g g ϑ ^ \hat{\vartheta} ϑ ^ ( Y , Z ) (Y,Z) ( Y , Z )
Fit the null model by solving (4) ; compute μ ^ i \hat{\mu}_i μ ^ i ϵ ^ i \hat{\epsilon}_i ϵ ^ i W ^ i \hat{W}_i W ^ i
Compute ν ^ i G L M \hat{\nu}^{\mathrm{GLM}}_i ν ^ i GLM (5) , then compute the observed statistic T n G L M T_n^{\mathrm{GLM}} T n GLM (6) ;
for b = 1 , … , B b=1,\ldots,B b = 1 , … , B
Draw Π b \Pi_b Π b G \mathcal{G} G
Form X ( π ) ≡ ( X π ( 1 ) , … , X π ( n ) ) ⊤ X^{(\pi)} \equiv(X_{\pi(1)},\ldots,X_{\pi(n)})^\top X ( π ) ≡ ( X π ( 1 ) , … , X π ( n ) ) ⊤ θ ^ ( π ) ≡ ( Z ⊤ W ^ Z ) − 1 Z ⊤ W ^ X ( π ) \hat{\theta}^{(\pi)} \equiv(Z^\top\hat{W}Z)^{-1}Z^\top\hat{W}X^{(\pi)} θ ^ ( π ) ≡ ( Z ⊤ W ^ Z ) − 1 Z ⊤ W ^ X ( π )
Form R ^ G L M , ( π ) ≡ ( R ^ 1 G L M , ( π ) , … , R ^ n G L M , ( π ) ) \hat{R}^{\mathrm{GLM},(\pi)} \equiv(\hat{R}^{\mathrm{GLM},(\pi)}_1,\ldots,\hat{R}^{\mathrm{GLM},(\pi)}_n) R ^ GLM , ( π ) ≡ ( R ^ 1 GLM , ( π ) , … , R ^ n GLM , ( π ) ) R ^ i G L M , ( π ) ≡ ν ^ i G L M , ( π ) ϵ ^ i \hat{R}^{\mathrm{GLM},(\pi)}_i \equiv\hat{\nu}^{\mathrm{GLM},(\pi)}_i\hat{\epsilon}_i R ^ i GLM , ( π ) ≡ ν ^ i GLM , ( π ) ϵ ^ i ν ^ i G L M , ( π ) ≡ X π ( i ) − Z i ⊤ θ ^ ( π ) \hat{\nu}^{\mathrm{GLM},(\pi)}_i \equiv X_{\pi(i)}-Z_i^\top\hat{\theta}^{(\pi)} ν ^ i GLM , ( π ) ≡ X π ( i ) − Z i ⊤ θ ^ ( π )
Compute V ^ n G L M , ( Π b ) \hat{V}_n^{\mathrm{GLM},(\Pi_b)} V ^ n GLM , ( Π b ) T n G L M , ( Π b ) T_n^{\mathrm{GLM},(\Pi_b)} T n GLM , ( Π b )
T n G L M , ( π ) ≡ 1 n ∑ i = 1 n R ^ i G L M , ( π ) V ^ n G L M , ( π ) where V ^ n G L M , ( π ) ≡ S n ( R ^ G L M , ( π ) ) ; (7) T_n^{\mathrm{GLM},(\pi)} \equiv\frac{\frac{1}{\sqrt{n}}\sum_{i=1}^n \hat{R}^{\mathrm{GLM},(\pi)}_i}{\sqrt{\hat{V}_n^{\mathrm{GLM},(\pi)}}}
\quad\text{where}\quad
\hat{V}_n^{\mathrm{GLM},(\pi)} \equiv S_n\left(\hat{R}^{\mathrm{GLM},(\pi)}\right); \tag*{(7)} T n GLM , ( π ) ≡ V ^ n GLM , ( π ) n 1 ∑ i = 1 n R ^ i GLM , ( π ) where V ^ n GLM , ( π ) ≡ S n ( R ^ GLM , ( π ) ) ; (7) end
Compute the p p p p ≡ 1 B + 1 ( 1 + ∑ b = 1 B 1 { T n G L M , ( Π b ) ≥ T n G L M } ) p \equiv\frac{1}{B+1}\left(1+\sum_{b=1}^B \mathbf{1}\left\{T_n^{\mathrm{GLM},(\Pi_b)} \ge T_n^{\mathrm{GLM}}\right\}\right) p ≡ B + 1 1 ( 1 + ∑ b = 1 B 1 { T n GLM , ( Π b ) ≥ T n GLM } )
Reject H 0 C M H_0^{\mathrm{CM}} H 0 CM α \alpha α p ≤ α p \le\alpha p ≤ α
Remark 4 (Connection to empirical score tests). The empirical studentizer in (6) follows the empirical score approach of Breslow [8 ] ; see also Boos [7 ] . The fitted nuisance-score equation gives ∑ i Z i ϵ ^ i = 0 \sum_i Z_i\hat{\epsilon}_i=0 ∑ i Z i ϵ ^ i = 0 ∑ i R ^ i G L M = ∑ i X i ϵ ^ i \sum_i \hat{R}^{\mathrm{GLM}}_i=\sum_i X_i\hat{\epsilon}_i ∑ i R ^ i GLM = ∑ i X i ϵ ^ i X X X γ \gamma γ
Validity and robustness under the conditional mean null
We assume that, for some β \beta β
H 0 C M : E [ Y ∣ X , Z ] = μ β ( Z ) ≡ g − 1 ( Z ⊤ β ) . (8) H_0^{\mathrm{CM}}:\quad\mathbb{E}[Y\mid X,Z]=\mu_\beta(Z)\equiv g^{-1}(Z^\top\beta). \tag*{(8)} H 0 CM : E [ Y ∣ X , Z ] = μ β ( Z ) ≡ g − 1 ( Z ⊤ β ) . (8) The true conditional variance Var ( Y ∣ X , Z ) \operatorname{Var}(Y\mid X,Z) Var ( Y ∣ X , Z ) X X X V ϑ ⋆ ( μ β ( Z ) ) V_{\vartheta_\star}(\mu_\beta(Z)) V ϑ ⋆ ( μ β ( Z )) ϑ ⋆ \vartheta_\star ϑ ⋆ ϑ ^ \hat{\vartheta} ϑ ^ V Y ( Z ) ≡ Var ( Y ∣ Z ) V_Y(Z)\equiv\operatorname{Var}(Y\mid Z) V Y ( Z ) ≡ Var ( Y ∣ Z ) (8) , set μ i ≡ μ i ( β ) \mu_i\equiv\mu_i(\beta) μ i ≡ μ i ( β ) v i ≡ V ϑ ⋆ ( μ i ) v_i\equiv V_{\vartheta_\star}(\mu_i) v i ≡ V ϑ ⋆ ( μ i ) g i ′ ≡ g ′ ( μ i ) g'_i\equiv g'(\mu_i) g i ′ ≡ g ′ ( μ i ) ξ i ≡ Y i − μ i v i g i ′ \xi_i\equiv\frac{Y_i-\mu_i}{v_i g'_i} ξ i ≡ v i g i ′ Y i − μ i W i ≡ 1 v i ( g i ′ ) 2 W_i\equiv\frac{1}{v_i(g'_i)^2} W i ≡ v i ( g i ′ ) 2 1 S i ≡ Var ( ξ i ∣ Z i ) = V Y ( Z i ) / ( v i 2 ( g i ′ ) 2 ) S_i\equiv\operatorname{Var}(\xi_i\mid Z_i)=V_Y(Z_i)/(v_i^2(g'_i)^2) S i ≡ Var ( ξ i ∣ Z i ) = V Y ( Z i ) / ( v i 2 ( g i ′ ) 2 ) S i / W i = V Y ( Z i ) / v i S_i/W_i=V_Y(Z_i)/v_i S i / W i = V Y ( Z i ) / v i Z i Z_i Z i M ≡ E [ W i Z i Z i ⊤ ] M\equiv\mathbb{E}[W_iZ_iZ_i^\top] M ≡ E [ W i Z i Z i ⊤ ] r ≡ E [ W i Z i X i ] r\equiv\mathbb{E}[W_iZ_iX_i] r ≡ E [ W i Z i X i ] θ ≡ M − 1 r \theta\equiv M^{-1}r θ ≡ M − 1 r
δ i ≡ X i − θ ⊤ Z i , R i ≡ W i δ i , τ s 2 ≡ E [ δ i 2 ξ i 2 ] , τ p 2 ≡ E [ ξ i 2 ] = E [ S i ] . \delta_i\equiv X_i-\theta^\top Z_i,\qquad R_i\equiv\sqrt{W_i}\,\delta_i,\qquad\tau_s^2\equiv\mathbb{E}[\delta_i^2\xi_i^2],\qquad\tau_p^2\equiv\mathbb{E}[\xi_i^2]=\mathbb{E}[S_i]. δ i ≡ X i − θ ⊤ Z i , R i ≡ W i δ i , τ s 2 ≡ E [ δ i 2 ξ i 2 ] , τ p 2 ≡ E [ ξ i 2 ] = E [ S i ] . Here R i R_i R i (5) , while τ s 2 \tau_s^2 τ s 2 τ p 2 \tau_p^2 τ p 2 T n G L M T_n^{\mathrm{GLM}} T n GLM
Assumption 1 (Conditions on GLM nuisance estimates). The observations ( X i , Y i , Z i ) (X_i,Y_i,Z_i) ( X i , Y i , Z i ) (8) , and the nuisance dimension p p p n → ∞ n \to\infty n → ∞ ϑ ^ → ϑ ⋆ \hat{\vartheta} \to\vartheta_\star ϑ ^ → ϑ ⋆ ϑ ⋆ \vartheta_\star ϑ ⋆ β ^ \hat{\beta} β ^ (4) and β ^ → β \hat{\beta} \to\beta β ^ → β E [ X i ] = 0 \mathbb{E}[X_i] = 0 E [ X i ] = 0 E [ X i 2 ] = 1 \mathbb{E}[X_i^2] = 1 E [ X i 2 ] = 1 E [ X i 4 ] < ∞ \mathbb{E}[X_i^4] < \infty E [ X i 4 ] < ∞
Proposition 2 (Sampling distribution of the empirically studentized GLM score). Suppose Assumptions 1, 6, and 7 hold, and suppose in addition that τ s 2 > 0 \tau_s^2 > 0 τ s 2 > 0 T n G L M → d N ( 0 , 1 ) T_n^{\mathrm{GLM}} \xrightarrow{d} N(0,1) T n GLM d N ( 0 , 1 )
Theorem 2 (Validity of empirically studentized GLM score test under the conditional mean null). Under the assumptions of Proposition 2, conditions (i)–(iii) of Theorem 1 hold with the following oracle choices:
U n i ≡ ( X i − Z i T θ ) ξ i , C n , i k ≡ ( X k − X ˉ ) ξ i , X ˉ ≡ 1 n ∑ k = 1 n X k . U_{ni} \equiv(X_i-Z_i^{\mathsf{T}}\theta)\xi_i,\qquad C_{n,ik} \equiv(X_k-\bar{X})\xi_i,\qquad\bar{X} \equiv\frac{1}{n}\sum_{k=1}^{n}X_k. U ni ≡ ( X i − Z i T θ ) ξ i , C n , ik ≡ ( X k − X ˉ ) ξ i , X ˉ ≡ n 1 k = 1 ∑ n X k . Consequently, Theorem 1 gives T n G L M → d N ( 0 , 1 ) T_n^{\mathrm{GLM}} \xrightarrow{d} N(0,1) T n GLM d N ( 0 , 1 ) T n G L M , ( Π ) ∣ F n → d , p N ( 0 , 1 ) T_n^{\mathrm{GLM},(\Pi)} \mid\mathcal{F}_n \xrightarrow{d,p} N(0,1) T n GLM , ( Π ) ∣ F n d , p N ( 0 , 1 )
Q 1 − α ( T n G L M , ( Π ) ∣ F n ) → p z 1 − α , P { T n G L M ≥ Q 1 − α ( T n G L M , ( Π ) ∣ F n ) } → α , Q_{1-\alpha}\left(T_n^{\mathrm{GLM},(\Pi)} \mid\mathcal{F}_n\right) \xrightarrow{p} z_{1-\alpha},\qquad\mathbb{P}\left\{T_n^{\mathrm{GLM}} \ge Q_{1-\alpha}\left(T_n^{\mathrm{GLM},(\Pi)} \mid\mathcal{F}_n\right)\right\} \to\alpha, Q 1 − α ( T n GLM , ( Π ) ∣ F n ) p z 1 − α , P { T n GLM ≥ Q 1 − α ( T n GLM , ( Π ) ∣ F n ) } → α , for every fixed α ∈ ( 0 , 1 ) \alpha\in(0,1) α ∈ ( 0 , 1 )
Theorem 2 establishes asymptotic validity of the GLM score permutation test under the conditional mean null, even when the working variance function is misspecified. The proof is given in Appendix § 15 .3.4.
Figure 3. Score calibration under conditional independence. Rows show high SNR, low SNR, and misspecified negative binomial variance; columns show null p p p
holds the realized value of ϑ ^ \hat{\vartheta} ϑ ^ X X X ( Y , Z ) (Y,Z) ( Y , Z ) X X X ϑ ^ \hat{\vartheta} ϑ ^ ( X ( π ) , Y , Z ) (X^{(\pi)},Y,Z) ( X ( π ) , Y , Z )
A less robust route: studentization using working Fisher information
The empirical studentizer V ^ n G L M \hat{V}^{\mathrm{GLM}}_n V ^ n GLM
T ~ n G L M ≡ 1 n ∑ i = 1 n ν ^ i G L M ϵ ^ i V ~ n G L M and V ~ n G L M ≡ 1 n ∑ i = 1 n W ^ i ( ν ^ i G L M ) 2 . (9) \widetilde{T}^{\mathrm{GLM}}_n \equiv\frac{\frac{1}{\sqrt{n}}\sum_{i=1}^{n}\hat{\nu}_i^{\mathrm{GLM}}\hat{\epsilon}_i}{\sqrt{\widetilde{V}^{\mathrm{GLM}}_n}}
\quad\text{and}\quad
\widetilde{V}^{\mathrm{GLM}}_n \equiv\frac{1}{n}\sum_{i=1}^{n}\widehat{W}_i(\hat{\nu}^{\mathrm{GLM}}_i)^2. \tag*{(9)} T n GLM ≡ V n GLM n 1 ∑ i = 1 n ν ^ i GLM ϵ ^ i and V n GLM ≡ n 1 i = 1 ∑ n W i ( ν ^ i GLM ) 2 . (9) It is not hard to check that n V ~ n G L M = X T W ^ X − X T W ^ Z ( Z T W ^ Z ) − 1 Z T W ^ X n\widetilde{V}^{\mathrm{GLM}}_n = X^{\mathsf{T}}\widehat{W}X - X^{\mathsf{T}}\widehat{W}Z(Z^{\mathsf{T}}\widehat{W}Z)^{-1}Z^{\mathsf{T}}\widehat{W}X n V n GLM = X T W X − X T W Z ( Z T W Z ) − 1 Z T W X γ \gamma γ π ∈ G \pi\in\mathcal{G} π ∈ G
T ~ n G L M , ( π ) ≡ 1 n ∑ i = 1 n ν ^ i G L M , ( π ) ϵ ^ i V ~ n G L M , ( π ) and V ~ n G L M , ( π ) ≡ 1 n ∑ i = 1 n W ^ i ( ν ^ i G L M , ( π ) ) 2 . \widetilde{T}^{\mathrm{GLM},(\pi)}_n \equiv\frac{\frac{1}{\sqrt{n}}\sum_{i=1}^{n}\hat{\nu}^{\mathrm{GLM},(\pi)}_i\hat{\epsilon}_i}{\sqrt{\widetilde{V}^{\mathrm{GLM},(\pi)}_n}}
\quad\text{and}\quad
\widetilde{V}^{\mathrm{GLM},(\pi)}_n \equiv\frac{1}{n}\sum_{i=1}^{n}\widehat{W}_i(\hat{\nu}^{\mathrm{GLM},(\pi)}_i)^2. T n GLM , ( π ) ≡ V n GLM , ( π ) n 1 ∑ i = 1 n ν ^ i GLM , ( π ) ϵ ^ i and V n GLM , ( π ) ≡ n 1 i = 1 ∑ n W i ( ν ^ i GLM , ( π ) ) 2 . Now we present the sampling and permutation distributions of the GLM score with information studentization.
Proposition 3 (Sampling distribution of the GLM score with information studentization). Suppose Assumptions 1, 6, and 7 hold. Then
T ~ n G L M → d N ( 0 , σ s 2 ) and σ s 2 ≡ τ s 2 E [ R i 2 ] . \widetilde{T}^{\mathrm{GLM}}_n \xrightarrow{d} N(0,\sigma_s^2)
\quad\text{and}\quad
\sigma_s^2 \equiv\frac{\tau_s^2}{\mathbb{E}[R_i^2]}. T n GLM d N ( 0 , σ s 2 ) and σ s 2 ≡ E [ R i 2 ] τ s 2 . If Var ( Y i ∣ X i , Z i ) = v i = V ϑ ⋆ ( μ i ) \operatorname{Var}(Y_i\mid X_i,Z_i)=v_i=V_{\vartheta_\star}(\mu_i) Var ( Y i ∣ X i , Z i ) = v i = V ϑ ⋆ ( μ i ) σ s 2 = 1 \sigma_s^2=1 σ s 2 = 1
Theorem 3 (Permutation distribution of the GLM score with information studentization). Under the assumptions of Proposition 3, let F n = σ ( ( X i , Y i , Z i ) i = 1 n ) \mathcal{F}_n=\sigma((X_i,Y_i,Z_i)_{i=1}^n) F n = σ (( X i , Y i , Z i ) i = 1 n ) Π \Pi Π
an independent uniform permutation. Then
T ~ n G L M , ( Π ) ∣ F n → d , p N ( 0 , σ p 2 ) and σ p 2 ≔ τ p 2 E [ W i ] . \widetilde{T}^{\mathrm{GLM},(\Pi)}_n \mid\mathcal{F}_n \xrightarrow{d,p} N(0,\sigma_p^2)
\quad\text{and}\quad
\sigma_p^2 \coloneqq\frac{\tau_p^2}{\mathbb{E}[W_i]}. T n GLM , ( Π ) ∣ F n d , p N ( 0 , σ p 2 ) and σ p 2 : = E [ W i ] τ p 2 . Consequently, Q 1 − α ( T ~ n G L M , ( Π ) ∣ F n ) → p σ p z 1 − α Q_{1-\alpha}\left(\widetilde{T}^{\mathrm{GLM},(\Pi)}_n \mid\mathcal{F}_n\right) \xrightarrow{p} \sigma_p z_{1-\alpha} Q 1 − α ( T n GLM , ( Π ) ∣ F n ) p σ p z 1 − α Var ( Y i ∣ X i , Z i ) = c v i = c V ϑ ⋆ ( μ i ) \operatorname{Var}(Y_i \mid X_i,Z_i)=cv_i=cV_{\vartheta^\star}(\mu_i) Var ( Y i ∣ X i , Z i ) = c v i = c V ϑ ⋆ ( μ i ) c > 0 c>0 c > 0 1 { T ~ n G L M ≥ Q 1 − α ( T ~ n G L M , ( Π ) ∣ F n ) } 1\{\widetilde{T}^{\mathrm{GLM}}_n \ge Q_{1-\alpha}(\widetilde{T}^{\mathrm{GLM},(\Pi)}_n \mid\mathcal{F}_n)\} 1 { T n GLM ≥ Q 1 − α ( T n GLM , ( Π ) ∣ F n )} α \alpha α H 0 C M H_0^{\mathrm{CM}} H 0 CM
Theorem 3 extends the results in Barry et al. [4 ] to general GLMs. Its proof is given in Appendix § 15 .3.5. We now discuss the robustness of the GLM score test with information studentization under variance misspecification.
We compare two CI designs with a binary covariate taking values 0 and 1. Both use the same covariate and exposure distributions, the same conditional outcome mean, and the same correctly specified Poisson mean model. In the first design, the conditional outcome variance is five times the mean for either covariate value. In the second, it is five times the mean when the covariate is 0 and equals the mean when it is 1. The variance multiplier is therefore constant in the first design and depends on the covariate in the second. Figure 2 reports simulations with n = 500 n=500 n = 500 B = 2 , 000 B=2,000 B = 2 , 000 § 11 .§ 1 gives the complete data-generating procedures.
Doubly robust tests calibrated by permutation
The generalized score statistic (1) provides a common template for several existing doubly robust CI tests. Section 4.1 and 4.2 develop permuted versions of the GCM of Shah and Peters [31 ] and the PCM of Lundborg et al. [26 ] , respectively. Throughout this section, F n \mathcal{F}_n F n σ \sigma σ
GCM test calibrated by permutation
Let μ x ( z ) ≡ E [ X ∣ Z = z ] \mu_x(z)\equiv\mathbb{E}[X\mid Z=z] μ x ( z ) ≡ E [ X ∣ Z = z ] μ y ( z ) ≡ E [ Y ∣ Z = z ] \mu_y(z)\equiv\mathbb{E}[Y\mid Z=z] μ y ( z ) ≡ E [ Y ∣ Z = z ] ν i G C M ≡ X i − μ x ( Z i ) \nu_i^{\mathrm{GCM}}\equiv X_i-\mu_x(Z_i) ν i GCM ≡ X i − μ x ( Z i ) ϵ i ≡ Y i − μ y ( Z i ) \epsilon_i\equiv Y_i-\mu_y(Z_i) ϵ i ≡ Y i − μ y ( Z i ) A x \mathcal{A}_x A x A y \mathcal{A}_y A y μ ^ x ( z ) ≈ E [ X ∣ Z = z ] \widehat{\mu}_x(z)\approx\mathbb{E}[X\mid Z=z] μ x ( z ) ≈ E [ X ∣ Z = z ]
caption: fig:2 : Variance robustness under the CI null at α = 0.01 \alpha= 0.01 α = 0.01
:::
μ ^ y ( z ) ≈ E [ Y ∣ Z = z ] \hat{\mu}_y(z) \approx\mathbb{E}[Y \mid Z = z] μ ^ y ( z ) ≈ E [ Y ∣ Z = z ] H 0 C I H_0^{\mathrm{CI}} H 0 CI
ν ^ i G C M ≡ X i − μ ^ x ( Z i ) and ϵ ^ i G C M ≡ Y i − μ ^ y ( Z i ) \hat{\nu}_i^{\mathrm{GCM}} \equiv X_i - \hat{\mu}_x(Z_i) \qquad\text{and} \qquad\hat{\epsilon}_i^{\mathrm{GCM}} \equiv Y_i - \hat{\mu}_y(Z_i) ν ^ i GCM ≡ X i − μ ^ x ( Z i ) and ϵ ^ i GCM ≡ Y i − μ ^ y ( Z i ) are approximately uncorrelated. The GCM test statistic is constructed based on the fitted residuals and the variance of their product: V ^ n G C M ≡ S n ( R ^ G C M ) \widehat{V}_n^{\mathrm{GCM}} \equiv S_n(\widehat{R}^{\mathrm{GCM}}) V n GCM ≡ S n ( R GCM ) R ^ G C M ≡ ( R ^ 1 G C M , … , R ^ n G C M ) \widehat{R}^{\mathrm{GCM}} \equiv(\widehat{R}_1^{\mathrm{GCM}}, \ldots, \widehat{R}_n^{\mathrm{GCM}}) R GCM ≡ ( R 1 GCM , … , R n GCM ) R ^ i G C M ≡ ν ^ i G C M ϵ ^ i G C M \widehat{R}_i^{\mathrm{GCM}} \equiv\hat{\nu}_i^{\mathrm{GCM}}\hat{\epsilon}_i^{\mathrm{GCM}} R i GCM ≡ ν ^ i GCM ϵ ^ i GCM
T n G C M ≡ 1 n ∑ i = 1 n R ^ i G C M V ^ n G C M . (10) T_n^{\mathrm{GCM}} \equiv\frac{\frac{1}{\sqrt{n}}\sum_{i=1}^{n}\widehat{R}_i^{\mathrm{GCM}}}{\sqrt{\widehat{V}_n^{\mathrm{GCM}}}}. \tag*{(10)} T n GCM ≡ V n GCM n 1 ∑ i = 1 n R i GCM . (10) For the GCM, permutation changes the weight in two ways: it relabels X i X_i X i X π ( i ) X_{\pi(i)} X π ( i ) A x \mathcal{A}_x A x ( X π ( i ) , Z i ) i = 1 n (X_{\pi(i)}, Z_i)_{i=1}^n ( X π ( i ) , Z i ) i = 1 n μ ^ y \hat{\mu}_y μ ^ y Y Y Y Z Z Z ϵ ^ i G C M \hat{\epsilon}_i^{\mathrm{GCM}} ϵ ^ i GCM
ν ^ i G C M , ( π ) ≡ X π ( i ) − μ ^ x ( π ) ( Z i ) and R ^ i G C M , ( π ) ≡ ν ^ i G C M , ( π ) ϵ ^ i G C M . \hat{\nu}_i^{\mathrm{GCM},(\pi)} \equiv X_{\pi(i)} - \hat{\mu}_x^{(\pi)}(Z_i) \qquad\text{and} \qquad\widehat{R}_i^{\mathrm{GCM},(\pi)} \equiv\hat{\nu}_i^{\mathrm{GCM},(\pi)}\hat{\epsilon}_i^{\mathrm{GCM}}. ν ^ i GCM , ( π ) ≡ X π ( i ) − μ ^ x ( π ) ( Z i ) and R i GCM , ( π ) ≡ ν ^ i GCM , ( π ) ϵ ^ i GCM . Then, T n G C M , ( π ) T_n^{\mathrm{GCM},(\pi)} T n GCM , ( π ) T n ( π ) T_n^{(\pi)} T n ( π ) (2) with these pieces. Algorithm 3 summarises the procedure.
Remark 9 (Permutation target of the refit μ ^ x ( Π b ) \hat{\mu}_x^{(\Pi_b)} μ ^ x ( Π b ) The only difference from Shah and Peters [31 ] ’s original GCM procedure is that the asymptotic cutoff is replaced by permutation calibration in Algorithm 3. The permutation refit μ ^ x ( Π b ) \hat{\mu}_x^{(\Pi_b)} μ ^ x ( Π b ) targets the constant function E [ X ] \mathbb{E}[X] E [ X ] Conditional on the observed sample, its finite-sample permutation analogue is ∑ i = 1 n X i / n \sum_{i=1}^{n} X_i/n ∑ i = 1 n X i / n
Validity under CI null For the permuted GCM test, define the empirical mean squared errors for the outcome and exposure regressions $E_{n,y} \equiv \frac{1}{n}\sum_{i=1}^{n}(\mu_y(Z_i) Algorithm 3: The GCM test calibrated by permutation
Input: Data ( X i , Y i , Z i ) i = 1 n (X_i,Y_i,Z_i)_{i=1}^n ( X i , Y i , Z i ) i = 1 n α ∈ ( 0 , 1 ) \alpha\in(0,1) α ∈ ( 0 , 1 ) B ∈ N B \in\mathbb{N} B ∈ N A x , A y \mathcal{A}_x,\mathcal{A}_y A x , A y
Apply A x \mathcal{A}_x A x ( X i , Z i ) i = 1 n (X_i,Z_i)_{i=1}^n ( X i , Z i ) i = 1 n A y \mathcal{A}_y A y ( Y i , Z i ) i = 1 n (Y_i,Z_i)_{i=1}^n ( Y i , Z i ) i = 1 n T n G C M T_n^{\mathrm{GCM}} T n GCM (10) ;
for b = 1 , … , B b=1,\ldots,B b = 1 , … , B do
Draw a permutation Π b \Pi_b Π b { 1 , … , n } \{1,\ldots,n\} { 1 , … , n }
Apply A x \mathcal{A}_x A x ( X Π b ( i ) , Z i ) i = 1 n (X_{\Pi_b(i)},Z_i)_{i=1}^n ( X Π b ( i ) , Z i ) i = 1 n μ ^ x ( Π b ) \hat{\mu}_x^{(\Pi_b)} μ ^ x ( Π b ) R ^ i G C M , ( Π b ) \hat{R}_i^{\mathrm{GCM},(\Pi_b)} R ^ i GCM , ( Π b )
Compute the permuted statistic T n G C M , ( Π b ) T_n^{\mathrm{GCM},(\Pi_b)} T n GCM , ( Π b ) ( R ^ i G C M , ( Π b ) ) i = 1 n (\hat{R}_i^{\mathrm{GCM},(\Pi_b)})_{i=1}^n ( R ^ i GCM , ( Π b ) ) i = 1 n (10) ;
end
Compute the right-tailed p p p p ≡ 1 B + 1 ( 1 + ∑ b = 1 B 1 { T n G C M , ( Π b ) ≥ T n G C M } ) p \equiv\frac{1}{B+1}\left(1+\sum_{b=1}^B 1\left\{T_n^{\mathrm{GCM},(\Pi_b)} \geq T_n^{\mathrm{GCM}}\right\}\right) p ≡ B + 1 1 ( 1 + ∑ b = 1 B 1 { T n GCM , ( Π b ) ≥ T n GCM } )
Reject H 0 C I H_0^{\mathrm{CI}} H 0 CI α \alpha α p ≤ α p \leq\alpha p ≤ α
μ ^ y ( Z i ) 2 \hat{\mu}_y(Z_i)^2 μ ^ y ( Z i ) 2 E n , x ≡ 1 n ∑ i = 1 n ( μ x ( Z i ) − μ ^ x ( Z i ) ) 2 E_{n,x} \equiv\frac{1}{n}\sum_{i=1}^n\left(\mu_x(Z_i)-\hat{\mu}_x(Z_i)\right)^2 E n , x ≡ n 1 ∑ i = 1 n ( μ x ( Z i ) − μ ^ x ( Z i ) ) 2
E ~ n , x ≡ 1 n ∑ i = 1 n ( μ ^ x ( Π ) ( Z i ) − X ‾ ) 2 . \widetilde{E}_{n,x} \equiv\frac{1}{n}\sum_{i=1}^n\left(\hat{\mu}_x^{(\Pi)}(Z_i)-\overline{X}\right)^2. E n , x ≡ n 1 i = 1 ∑ n ( μ ^ x ( Π ) ( Z i ) − X ) 2 . Assumption 2 (GCM nuisance consistency). Under the GCM conditional independence null, assume E n , y = o P ( 1 ) E_{n,y}=o_{\mathrm{P}}(1) E n , y = o P ( 1 ) E n , x = o P ( 1 ) E_{n,x}=o_{\mathrm{P}}(1) E n , x = o P ( 1 ) and n E n , y E n , x = o P ( 1 ) nE_{n,y}E_{n,x}=o_{\mathrm{P}}(1) n E n , y E n , x = o P ( 1 )
Assumption 3 (GCM permutation stability). Under the GCM conditional independence null, assume E ~ n , x = o P ( 1 ) \widetilde{E}_{n,x}=o_{\mathrm{P}}(1) E n , x = o P ( 1 ) and n E n , y E ~ n , x = o P ( 1 ) nE_{n,y}\widetilde{E}_{n,x}=o_{\mathrm{P}}(1) n E n , y E n , x = o P ( 1 )
Theorem 4 (Asymptotic validity of the permuted GCM test). Fix α ∈ ( 0 , 1 ) \alpha\in(0,1) α ∈ ( 0 , 1 ) and consider the GCM test of Algorithm 3, under H 0 C I H_0^{\mathrm{CI}} H 0 CI Assumptions 2 2 2 and 3 3 3 the shared outcome condition Assumption 6 , and Assumption 8 . Take
U n i ≡ { X i − μ x ( Z i ) } { Y i − μ y ( Z i ) } , C n , i k ≡ ( X k − X ‾ ) { Y i − μ y ( Z i ) } . U_{ni} \equiv\{X_i-\mu_x(Z_i)\}\{Y_i-\mu_y(Z_i)\}, \qquad C_{n,ik} \equiv(X_k-\overline{X})\{Y_i-\mu_y(Z_i)\}. U ni ≡ { X i − μ x ( Z i )} { Y i − μ y ( Z i )} , C n , ik ≡ ( X k − X ) { Y i − μ y ( Z i )} . Let T n = T n G C M T_n=T_n^{\mathrm{GCM}} T n = T n GCM and let T n ( Π ) T_n^{(\Pi)} T n ( Π ) be its permuted version under an independent uniform permutation, computed as in Algorithm 3. The stated assumptions verify conditions (i)–(iii) of Theorem 1 for the displayed oracle products and array. Thus
T n → d N ( 0 , 1 ) and T n ( Π ) ∣ F n → d , P N ( 0 , 1 ) . T_n \xrightarrow{d} N(0,1) \quad\text{and} \quad T_n^{(\Pi)} \mid\mathcal{F}_n \xrightarrow{d,\mathrm{P}} N(0,1). T n d N ( 0 , 1 ) and T n ( Π ) ∣ F n d , P N ( 0 , 1 ) . Consequently, P ( φ α n = 1 ) → α \mathbb{P}(\varphi_\alpha^n=1)\to\alpha P ( φ α n = 1 ) → α under the CI null.
The proof is given in Appendix § 15 .3.6. Shah and Peters [31 ] establish asymptotic normality of the original GCM under a condition on the product of nuisance error rates and conditions on nuisance errors weighted by variance. Assumption 2 retains the same requirement on the product of error rates; its consistency conditions and our bounded conditional moment assumptions imply the original conditions on weighted errors. Theorem 4 additionally establishes conditional permutation normality under Assumption 3 and the stated auxiliary regularity, thereby justifying permutation calibration under CI. This high-level condition is verified in a range of estimators including ordinary least squares, linear ridge regression, kernel ridge regression, and the lasso under their respective conditions, in Appendix § 16 .
PCM test calibrated by permutation
The projection covariance measure (PCM) of Lundborg et al. [26 ] was originally proposed to test whether a variable X \mathbf{X} X Y \mathbf{Y} Y Z \mathbf{Z} Z H 0 C I : Y ⊥ ⊥ X ∣ Z H_0^{\mathrm{CI}}: \mathbf{Y} \perp\!\!\!\perp\mathbf{X} \mid\mathbf{Z} H 0 CI : Y ⊥ ⊥ X ∣ Z ( X i , Y i , Z i ) i = 1 n (X_i,Y_i,Z_i)_{i=1}^{n} ( X i , Y i , Z i ) i = 1 n D 1 ∪ D 2 \mathcal{D}_1 \cup\mathcal{D}_2 D 1 ∪ D 2 n / 2 n/2 n /2
Training with D 2 \mathcal{D}_2 D 2 On the training fold D 2 \mathcal{D}_2 D 2 m ^ p ( x , z ) ≈ E [ Y ∣ X = x , Z = z ] \widehat{m}_p(x,z) \approx\mathbb{E}[Y \mid X=x,Z=z] m p ( x , z ) ≈ E [ Y ∣ X = x , Z = z ] A p \mathcal{A}_p A p D 2 \mathcal{D}_2 D 2 q ( z ) ≡ E [ m ^ p ( X , z ) ∣ Z = z , H n ] q(z) \equiv\mathbb{E}[\widehat{m}_p(X,z) \mid Z=z,\mathcal{H}_n] q ( z ) ≡ E [ m p ( X , z ) ∣ Z = z , H n ] m ^ p ( X i , Z i ) \widehat{m}_p(X_i,Z_i) m p ( X i , Z i ) Z i Z_i Z i D 2 \mathcal{D}_2 D 2 m ^ q \widehat{m}_q m q A q \mathcal{A}_q A q q q q f ^ ( x , z ) ≡ m ^ p ( x , z ) − m ^ q ( z ) \widehat{f}(x,z) \equiv\widehat{m}_p(x,z)-\widehat{m}_q(z) f ( x , z ) ≡ m p ( x , z ) − m q ( z ) E [ Y ∣ X , Z ] \mathbb{E}[Y \mid X,Z] E [ Y ∣ X , Z ] X X X Z Z Z
Testing with D 1 \mathcal{D}_1 D 1 Treating m ^ p \widehat{m}_p m p m ^ q \widehat{m}_q m q D 1 \mathcal{D}_1 D 1 Y Y Y Z Z Z μ ^ y ( z ) ≈ μ y ( z ) ≡ E [ Y ∣ Z = z ] \widehat{\mu}_y(z) \approx\mu_y(z) \equiv\mathbb{E}[Y \mid Z=z] μ y ( z ) ≈ μ y ( z ) ≡ E [ Y ∣ Z = z ] A y \mathcal{A}_y A y f ^ ( X i , Z i ) \widehat{f}(X_i,Z_i) f ( X i , Z i ) Z i Z_i Z i μ ^ f ( z ) ≈ μ f ( z ) ≡ E [ f ^ ( X , Z ) ∣ Z = z , H n ] \widehat{\mu}_f(z) \approx\mu_f(z) \equiv\mathbb{E}[\widehat{f}(X,Z) \mid Z=z,\mathcal{H}_n] μ f ( z ) ≈ μ f ( z ) ≡ E [ f ( X , Z ) ∣ Z = z , H n ] A f \mathcal{A}_f A f μ f ( z ) = q ( z ) − m ^ q ( z ) \mu_f(z)=q(z)-\widehat{m}_q(z) μ f ( z ) = q ( z ) − m q ( z ) ν i P C M ≡ f ^ ( X i , Z i ) − μ f ( Z i ) \nu_i^{\mathrm{PCM}} \equiv\widehat{f}(X_i,Z_i)-\mu_f(Z_i) ν i PCM ≡ f ( X i , Z i ) − μ f ( Z i ) ϵ i P C M ≡ Y i − μ y ( Z i ) \epsilon_i^{\mathrm{PCM}} \equiv Y_i-\mu_y(Z_i) ϵ i PCM ≡ Y i − μ y ( Z i ) D 1 \mathcal{D}_1 D 1
ν ^ i P C M ≡ f ^ ( X i , Z i ) − μ ^ f ( Z i ) and ϵ ^ i P C M ≡ Y i − μ ^ y ( Z i ) . \widehat{\nu}_i^{\mathrm{PCM}} \equiv\widehat{f}(X_i,Z_i)-\widehat{\mu}_f(Z_i) \quad\text{and} \quad\widehat{\epsilon}_i^{\mathrm{PCM}} \equiv Y_i-\widehat{\mu}_y(Z_i). ν i PCM ≡ f ( X i , Z i ) − μ f ( Z i ) and ϵ i PCM ≡ Y i − μ y ( Z i ) . The PCM instantiates the template (1) with these fitted quantities and V ^ n P C M ≡ S n / 2 ( R ^ P C M ) \widehat{V}_n^{\mathrm{PCM}} \equiv S_{n/2}(\widehat{R}^{\mathrm{PCM}}) V n PCM ≡ S n /2 ( R PCM ) R ^ P C M ≡ ( R ^ i P C M ) i ∈ D 1 \widehat{R}^{\mathrm{PCM}} \equiv(\widehat{R}_i^{\mathrm{PCM}})_{i\in\mathcal{D}_1} R PCM ≡ ( R i PCM ) i ∈ D 1 R ^ i P C M ≡ ν ^ i P C M ϵ ^ i P C M \widehat{R}_i^{\mathrm{PCM}} \equiv\widehat{\nu}_i^{\mathrm{PCM}}\widehat{\epsilon}_i^{\mathrm{PCM}} R i PCM ≡ ν i PCM ϵ i PCM
T n P C M ≡ 1 n / 2 ∑ i ∈ D 1 R ^ i P C M V ^ n P C M . (11) T_n^{\mathrm{PCM}} \equiv\frac{\frac{1}{\sqrt{n/2}}\sum_{i\in\mathcal{D}_1}\widehat{R}_i^{\mathrm{PCM}}}{\sqrt{\widehat{V}_n^{\mathrm{PCM}}}}. \tag*{(11)} T n PCM ≡ V n PCM n /2 1 ∑ i ∈ D 1 R i PCM . (11) Let G \mathcal{G} G D 1 \mathcal{D}_1 D 1 Π ∈ G \Pi\in\mathcal{G} Π ∈ G m ^ p , m ^ q \widehat{m}_p,\widehat{m}_q m p , m q D 2 \mathcal{D}_2 D 2 G \mathcal{G} G ϵ ^ i P C M \widehat{\epsilon}_i^{\mathrm{PCM}} ϵ i PCM μ ^ f \widehat{\mu}_f μ f f ^ ( X π ( i ) , Z i ) = m ^ p ( X π ( i ) , Z i ) − m ^ q ( Z i ) \widehat{f}(X_{\pi(i)},Z_i)=\widehat{m}_p(X_{\pi(i)},Z_i)-\widehat{m}_q(Z_i) f ( X π ( i ) , Z i ) = m p ( X π ( i ) , Z i ) − m q ( Z i ) i ∈ D 1 i\in\mathcal{D}_1 i ∈ D 1 μ ^ f ( π ) \widehat{\mu}_f^{(\pi)} μ f ( π ) f ^ ( X π ( i ) , Z i ) \widehat{f}(X_{\pi(i)},Z_i) f ( X π ( i ) , Z i ) Z i Z_i Z i D 1 \mathcal{D}_1 D 1 A f \mathcal{A}_f A f
ν ^ i P C M , ( π ) ≡ f ^ ( X π ( i ) , Z i ) − μ ^ f ( π ) ( Z i ) , R ^ i P C M , ( π ) ≡ ν ^ i P C M , ( π ) ϵ ^ i P C M . \widehat{\nu}_i^{\mathrm{PCM},(\pi)} \equiv\widehat{f}(X_{\pi(i)},Z_i)-\widehat{\mu}_f^{(\pi)}(Z_i), \qquad\widehat{R}_i^{\mathrm{PCM},(\pi)} \equiv\widehat{\nu}_i^{\mathrm{PCM},(\pi)}\widehat{\epsilon}_i^{\mathrm{PCM}}. ν i PCM , ( π ) ≡ f ( X π ( i ) , Z i ) − μ f ( π ) ( Z i ) , R i PCM , ( π ) ≡ ν i PCM , ( π ) ϵ i PCM . The statistic T n P C M , ( π ) T_n^{\mathrm{PCM},(\pi)} T n PCM , ( π ) T n ( π ) T_n^{(\pi)} T n ( π ) (2) formed from ν ^ i P C M , ( π ) ϵ ^ i P C M \widehat{\nu}_i^{\mathrm{PCM},(\pi)}\widehat{\epsilon}_i^{\mathrm{PCM}} ν i PCM , ( π ) ϵ i PCM D 1 \mathcal{D}_1 D 1
Algorithm 4: The PCM test calibrated by permutation
Input: Data ( X i , Y i , Z i ) i = 1 n (X_i,Y_i,Z_i)_{i=1}^n ( X i , Y i , Z i ) i = 1 n α ∈ ( 0 , 1 ) \alpha\in(0,1) α ∈ ( 0 , 1 ) B ∈ N B \in\mathbb{N} B ∈ N A p , A q , A y , A f \mathcal{A}_p,\mathcal{A}_q,\mathcal{A}_y,\mathcal{A}_f A p , A q , A y , A f m ^ p , m ^ q , μ ^ y , μ ^ f \hat{m}_p,\hat{m}_q,\hat{\mu}_y,\hat{\mu}_f m ^ p , m ^ q , μ ^ y , μ ^ f
Partition the sample into D 1 ∪ D 2 D_1 \cup D_2 D 1 ∪ D 2 ∣ D 1 ∣ = ∣ D 2 ∣ = n / 2 |D_1|=|D_2|=n/2 ∣ D 1 ∣ = ∣ D 2 ∣ = n /2
On D 2 D_2 D 2 Y Y Y ( X , Z ) (X,Z) ( X , Z ) m ^ p \hat{m}_p m ^ p m ^ p ( X i , Z i ) \hat{m}_p(X_i,Z_i) m ^ p ( X i , Z i ) Z i Z_i Z i m ^ q \hat{m}_q m ^ q
On D 1 D_1 D 1 f ^ ( x , z ) = m ^ p ( x , z ) − m ^ q ( z ) \hat{f}(x,z)=\hat{m}_p(x,z)-\hat{m}_q(z) f ^ ( x , z ) = m ^ p ( x , z ) − m ^ q ( z ) Y Y Y Z Z Z μ ^ y \hat{\mu}_y μ ^ y f ^ ( X i , Z i ) \hat{f}(X_i,Z_i) f ^ ( X i , Z i ) Z i Z_i Z i μ ^ f \hat{\mu}_f μ ^ f T n P C M T_n^{\mathrm{PCM}} T n PCM (11) ;
for b = 1 , … , B b=1,\ldots,B b = 1 , … , B do
Draw an independent uniform permutation Π b \Pi_b Π b D 1 D_1 D 1
Apply learner A f \mathcal{A}_f A f f ^ ( X Π b ( i ) , Z i ) \hat{f}(X_{\Pi_b(i)},Z_i) f ^ ( X Π b ( i ) , Z i ) Z i Z_i Z i μ ^ f ( Π b ) \hat{\mu}_f^{(\Pi_b)} μ ^ f ( Π b ) f ^ ( X Π b ( i ) , Z i ) = m ^ p ( X Π b ( i ) , Z i ) − m ^ q ( Z i ) \hat{f}(X_{\Pi_b(i)},Z_i)=\hat{m}_p(X_{\Pi_b(i)},Z_i)-\hat{m}_q(Z_i) f ^ ( X Π b ( i ) , Z i ) = m ^ p ( X Π b ( i ) , Z i ) − m ^ q ( Z i ) i ∈ D 1 i\in D_1 i ∈ D 1
Compute the permuted statistic T n P C M , ( Π b ) T_n^{\mathrm{PCM},(\Pi_b)} T n PCM , ( Π b ) ( R ^ i P C M , ( Π b ) ) i ∈ D 1 (\hat{R}_i^{\mathrm{PCM},(\Pi_b)})_{i\in D_1} ( R ^ i PCM , ( Π b ) ) i ∈ D 1 (11) ;
end
Compute the p p p p = 1 B + 1 ( 1 + ∑ b = 1 B 1 { T n P C M , ( Π b ) ≥ T n P C M } ) p=\frac{1}{B+1}\left(1+\sum_{b=1}^{B}\mathbb{1}\left\{T_n^{\mathrm{PCM},(\Pi_b)}\geq T_n^{\mathrm{PCM}}\right\}\right) p = B + 1 1 ( 1 + ∑ b = 1 B 1 { T n PCM , ( Π b ) ≥ T n PCM } )
Reject H 0 C I H_0^{\mathrm{CI}} H 0 CI α \alpha α p ≤ α p\leq\alpha p ≤ α
Thus the population permutation target is obtained by evaluating the learned direction under the product law P X ⊗ P Z P_X\otimes P_Z P X ⊗ P Z
Validity
Let n ⋆ = ∣ D 1 ∣ = n / 2 n_\star=|D_1|=n/2 n ⋆ = ∣ D 1 ∣ = n /2 m ^ p \hat{m}_p m ^ p m ^ q \hat{m}_q m ^ q H n \mathcal{H}_n H n D 2 D_2 D 2
μ f ( z ) ≡ E [ f ^ ( X , Z ) ∣ Z = z , H n ] , σ f , n 2 ≡ E [ { f ^ ( X , Z ) − μ f ( Z ) } 2 ∣ H n ] . \mu_f(z)\equiv\mathbb{E}[\hat{f}(X,Z)\mid Z=z,\mathcal{H}_n],\qquad\sigma_{f,n}^2\equiv\mathbb{E}\left[\{\hat{f}(X,Z)-\mu_f(Z)\}^2\mid\mathcal{H}_n\right]. μ f ( z ) ≡ E [ f ^ ( X , Z ) ∣ Z = z , H n ] , σ f , n 2 ≡ E [ { f ^ ( X , Z ) − μ f ( Z ) } 2 ∣ H n ] . This is the same variance of the learned direction used to normalize the error in the projection regression in the original PCM analysis ([26 ] , Eq. (11) ). Importantly, σ f , n 2 \sigma_{f,n}^2 σ f , n 2
E n , y P C M ≡ 1 n ⋆ ∑ i ∈ D 1 { μ y ( Z i ) − μ ^ y ( Z i ) } 2 , E n , f P C M ≡ 1 n ⋆ ∑ i ∈ D 1 { μ f ( Z i ) − μ ^ f ( Z i ) } 2 . E_{n,y}^{\mathrm{PCM}}\equiv\frac{1}{n_\star}\sum_{i\in D_1}\{\mu_y(Z_i)-\hat{\mu}_y(Z_i)\}^2,\qquad E_{n,f}^{\mathrm{PCM}}\equiv\frac{1}{n_\star}\sum_{i\in D_1}\{\mu_f(Z_i)-\hat{\mu}_f(Z_i)\}^2. E n , y PCM ≡ n ⋆ 1 i ∈ D 1 ∑ { μ y ( Z i ) − μ ^ y ( Z i ) } 2 , E n , f PCM ≡ n ⋆ 1 i ∈ D 1 ∑ { μ f ( Z i ) − μ ^ f ( Z i ) } 2 . Recall from Remark 10 that, if X ~ \widetilde{X} X X X X Z Z Z b p e r m ( z ) = E [ f ^ ( X ~ , z ) ∣ H n ] b^{\mathrm{perm}}(z)=\mathbb{E}[\hat{f}(\widetilde{X},z)\mid\mathcal{H}_n] b perm ( z ) = E [ f ^ ( X , z ) ∣ H n ]
σ f , n , p e r m 2 ≡ E [ { f ^ ( X ~ , Z ) − b p e r m ( Z ) } 2 ∣ H n ] , \sigma_{f,n,\mathrm{perm}}^2\equiv\mathbb{E}\left[\{\hat{f}(\widetilde{X},Z)-b^{\mathrm{perm}}(Z)\}^2\mid\mathcal{H}_n\right], σ f , n , perm 2 ≡ E [ { f ^ ( X , Z ) − b perm ( Z ) } 2 ∣ H n ] , where, conditional on H n \mathcal{H}_n H n X ~ ∼ P X \widetilde{X}\sim P_X X ∼ P X Z ∼ P Z Z\sim P_Z Z ∼ P Z σ f , n 2 \sigma_{f,n}^2 σ f , n 2 σ f , n , p e r m 2 \sigma_{f,n,\mathrm{perm}}^2 σ f , n , perm 2 P X Z P_{XZ} P X Z P X ⊗ P Z P_X\otimes P_Z P X ⊗ P Z
For the conditional permutation argument we also retain the exact finite-sample permutation target. For i , k ∈ D 1 i,k\in D_1 i , k ∈ D 1 B i k ≡ f ^ ( X k , Z i ) B_{ik}\equiv\hat{f}(X_k,Z_i) B ik ≡ f ^ ( X k , Z i ) B i ⋅ ≡ 1 n ⋆ ∑ k ∈ D 1 B i k B_{i\cdot}\equiv\frac{1}{n_\star}\sum_{k\in D_1}B_{ik} B i ⋅ ≡ n ⋆ 1 ∑ k ∈ D 1 B ik B i = E Π [ f ^ ( X Π ( i ) , Z i ) ∣ F n ] B_i = \mathbb{E}_{\Pi}[\hat{f}(X_{\Pi(i)}, Z_i) \mid\mathcal{F}_n] B i = E Π [ f ^ ( X Π ( i ) , Z i ) ∣ F n ]
E ~ n , f P C M ≡ 1 n ⋆ ∑ i ∈ D 1 { μ ^ f ( Π ) ( Z i ) − B i } 2 . \widetilde{E}^{\mathrm{PCM}}_{n,f} \equiv\frac{1}{n_{\star}} \sum_{i\in D_1} \{\hat{\mu}^{(\Pi)}_f(Z_i)-B_i\}^2. E n , f PCM ≡ n ⋆ 1 i ∈ D 1 ∑ { μ ^ f ( Π ) ( Z i ) − B i } 2 . Assumption 4 (PCM nuisance conditions). Under the PCM conditional independence null, conditional on H n \mathcal{H}_n H n
E n , y P C M = o P ( 1 ) , E n , f P C M σ f , n 2 = o P ( 1 ) , n ⋆ E n , y P C M E n , f P C M σ f , n 2 = o P ( 1 ) . E^{\mathrm{PCM}}_{n,y}=o_{\mathrm{P}}(1), \qquad\frac{E^{\mathrm{PCM}}_{n,f}}{\sigma^2_{f,n}}=o_{\mathrm{P}}(1), \qquad\frac{n_{\star}E^{\mathrm{PCM}}_{n,y}E^{\mathrm{PCM}}_{n,f}}{\sigma^2_{f,n}}=o_{\mathrm{P}}(1). E n , y PCM = o P ( 1 ) , σ f , n 2 E n , f PCM = o P ( 1 ) , σ f , n 2 n ⋆ E n , y PCM E n , f PCM = o P ( 1 ) . The ratios are considered on { σ f , n 2 > 0 } \{\sigma^2_{f,n}>0\} { σ f , n 2 > 0 }
Assumption 5 (Stability of PCM refitting after permutation). Under the PCM conditional independence null, assume
E ~ n , f P C M σ f , n , p e r m 2 = o P ( 1 ) , n ⋆ E n , y P C M E ~ n , f P C M σ f , n , p e r m 2 = o P ( 1 ) . \frac{\widetilde{E}^{\mathrm{PCM}}_{n,f}}{\sigma^2_{f,n,\mathrm{perm}}}=o_{\mathrm{P}}(1), \qquad\frac{n_{\star}E^{\mathrm{PCM}}_{n,y}\widetilde{E}^{\mathrm{PCM}}_{n,f}}{\sigma^2_{f,n,\mathrm{perm}}}=o_{\mathrm{P}}(1). σ f , n , perm 2 E n , f PCM = o P ( 1 ) , σ f , n , perm 2 n ⋆ E n , y PCM E n , f PCM = o P ( 1 ) . The ratios are considered on { σ f , n , p e r m 2 > 0 } \{\sigma^2_{f,n,\mathrm{perm}}>0\} { σ f , n , perm 2 > 0 }
Theorem 5 (Asymptotic validity of the permuted PCM test). Fix α ∈ ( 0 , 1 ) \alpha\in(0,1) α ∈ ( 0 , 1 ) H 0 C I H^{\mathrm{CI}}_0 H 0 CI H n \mathcal{H}_n H n U n i ≡ { f ^ ( X i , Z i ) − μ f ( Z i ) } { Y i − μ y ( Z i ) } U_{ni} \equiv\{\hat{f}(X_i,Z_i)-\mu_f(Z_i)\}\{Y_i-\mu_y(Z_i)\} U ni ≡ { f ^ ( X i , Z i ) − μ f ( Z i )} { Y i − μ y ( Z i )}
C n , i k ≡ { f ^ ( X k , Z i ) − 1 n ⋆ ∑ ℓ ∈ D 1 f ^ ( X ℓ , Z i ) } { Y i − μ y ( Z i ) } , C_{n,ik} \equiv\left\{\hat{f}(X_k,Z_i)-\frac{1}{n_{\star}}\sum_{\ell\in D_1}\hat{f}(X_\ell,Z_i)\right\}\{Y_i-\mu_y(Z_i)\}, C n , ik ≡ { f ^ ( X k , Z i ) − n ⋆ 1 ℓ ∈ D 1 ∑ f ^ ( X ℓ , Z i ) } { Y i − μ y ( Z i )} , for i , k ∈ D 1 i,k\in D_1 i , k ∈ D 1 n ⋆ = ∣ D 1 ∣ n_{\star}=|D_1| n ⋆ = ∣ D 1 ∣ μ f ( z ) = E [ f ^ ( X , Z ) ∣ Z = z , H n ] \mu_f(z)=\mathbb{E}[\hat{f}(X,Z)\mid Z=z,\mathcal{H}_n] μ f ( z ) = E [ f ^ ( X , Z ) ∣ Z = z , H n ] T n = T n P C M T_n=T^{\mathrm{PCM}}_n T n = T n PCM T n ( Π ) T^{(\Pi)}_n T n ( Π ) n ⋆ n_{\star} n ⋆
T n → d N ( 0 , 1 ) and T n ( Π ) ∣ F n → d , P N ( 0 , 1 ) . T_n \xrightarrow{d} N(0,1) \quad\text{and} \quad T^{(\Pi)}_n \mid\mathcal{F}_n \xrightarrow{d,\mathrm{P}} N(0,1). T n d N ( 0 , 1 ) and T n ( Π ) ∣ F n d , P N ( 0 , 1 ) . Consequently, P ( φ n α = 1 ) → α \mathbb{P}(\varphi^\alpha_n=1)\to\alpha P ( φ n α = 1 ) → α
The proof is given in Appendix § 15 .3.7. Assumption 4 retains the conditions on normalized nuisance errors and their products from Lundborg et al. [26 ] , allowing σ f , n 2 \sigma^2_{f,n} σ f , n 2 σ f , n , p e r m 2 \sigma^2_{f,n,\mathrm{perm}} σ f , n , perm 2 § 16 .
Further discussion
We present several further remarks on the permuted GCM and PCM tests.
Simulations
We present two simulation studies. Section 5.1 examines score tests calibrated by permutation for negative binomial outcomes, while Section 5.2 considers doubly robust tests calibrated by permutation with flexible nuisance regression.
CI tests for negative binomial outcomes
This simulation is motivated by pooled single-cell CRISPR screens, where negative binomial regression is commonly used to test whether a CRISPR perturbation indicator X \mathbf{X} X Y \mathbf{Y} Y Z Z Z [2 , 3 , 28 ] . The corresponding null hypothesis can be expressed as conditional independence, X ⊥ ⊥ Y ∣ Z \mathbf{X} \perp\!\!\perp\mathbf{Y} \mid Z X ⊥ ⊥ Y ∣ Z
Data-generating process. Suppose Z ∼ N ( 0 , 1 ) Z \sim N(0, 1) Z ∼ N ( 0 , 1 ) π ( Z ) = expit ( a + δ Z ) \pi(Z) = \operatorname{expit}(a + \delta Z) π ( Z ) = expit ( a + δ Z ) μ ( X , Z ) = 10 exp ( 0.5 Z + γ X ) \mu(X, Z) = 10\exp(0.5Z + \gamma X) μ ( X , Z ) = 10 exp ( 0.5 Z + γ X )
X ∣ Z ∼ Ber ( π ( Z ) ) , Y ∣ X , Z ∼ NB { μ ( X , Z ) , θ ( Z ) } . X \mid Z \sim\operatorname{Ber}(\pi(Z)), \qquad Y \mid X, Z \sim\operatorname{NB}\{\mu(X, Z), \theta(Z)\}. X ∣ Z ∼ Ber ( π ( Z )) , Y ∣ X , Z ∼ NB { μ ( X , Z ) , θ ( Z )} . The size function θ ( ⋅ ) \theta(\cdot) θ ( ⋅ ) Var ( Y ∣ X , Z ) = μ + μ 2 θ ( Z ) \operatorname{Var}(Y \mid X, Z) = \mu+ \frac{\mu^2}{\theta(Z)} Var ( Y ∣ X , Z ) = μ + θ ( Z ) μ 2 δ = 0 \delta= 0 δ = 0 δ = 0.5 \delta= 0.5 δ = 0.5 a a a X X X p ( a ) ≡ E [ expit ( a + 0.5 Z ) ] p(a) \equiv\mathbb{E}[\operatorname{expit}(a + 0.5Z)] p ( a ) ≡ E [ expit ( a + 0.5 Z )] ( a , θ ( ⋅ ) ) (a,\theta(\cdot)) ( a , θ ( ⋅ )) High SNR , with a = p − 1 ( 0.5 ) a=p^{-1}(0.5) a = p − 1 ( 0.5 ) θ ( Z ) = 0.5 \theta(Z)=0.5 θ ( Z ) = 0.5 Low SNR , with a = p − 1 ( 0.025 ) a=p^{-1}(0.025) a = p − 1 ( 0.025 ) θ ( Z ) = 0.5 \theta(Z)=0.5 θ ( Z ) = 0.5 Variance misspecification , with a = p − 1 ( 0.14 ) a=p^{-1}(0.14) a = p − 1 ( 0.14 ) θ ( Z ) = 46 exp ( − 2.1 Z ) \theta(Z)=46\exp(-2.1Z) θ ( Z ) = 46 exp ( − 2.1 Z )
Each setting uses n = 400 n=400 n = 400 § 11 .§ 2 reports the signal grids for γ \gamma γ
Methods compared All five procedures fit the working model by maximum likelihood using the R package MASS (Venables and Ripley, [34 ] ). In the designs with high and low SNR, the null mean and size are estimated by negative binomial regression of Y Y Y Z Z Z glm.nb function. We compare the standard parametric score test, permutation with information studentization (Barry et al., [4 ] ), empirically studentized permutation (ours), the robust sign-flipping test (Hemerik et al., [17 ] ), and its standardized version (De Santis et al., [11 ] ).
Results interpretation Figure 3 reports null calibration and rejection rates under CI. The corresponding MI results appear in Appendix § 11 .§ 2 . Under the CI null with high SNR, all five methods behave similarly and control Type I error reasonably well, with indistinguishable power. Exposure sparsity (i.e., the degree of sparsity in X \mathbf{X} X
Overall, our method shows the highest power among methods which control Type I error.
CI tests with complex nuisance structures
This simulation compares GCM and PCM procedures with permutation or asymptotic calibration in continuous regression, with dCRT (Liu et al., [25 ] ; Niu et al. [27 ] ) included as a conditional resampling benchmark.
Data-generating process The data-generating process separates marginal independence from conditional independence and include alternatives chosen to emphasize the complementary sensitivities of the GCM and PCM statistics. Let s ( z ) = sin ( 2 π z ) s(z)=\sin(2\pi z) s ( z ) = sin ( 2 π z ) Z 1 , ε ∼ N ( 0 , 1 ) \mathbf{Z}_{1},\varepsilon\sim N(0,1) Z 1 , ε ∼ N ( 0 , 1 ) Marginal independence : ξ ∼ N ( 0 , 1 ) \xi\sim N(0,1) ξ ∼ N ( 0 , 1 ) X = ξ \mathbf{X}=\xi X = ξ Y = s ( Z 1 ) + γ X 2 + ε Y=s(Z_{1})+\gamma X^{2}+\varepsilon Y = s ( Z 1 ) + γ X 2 + ε Additive Gaussian : ξ ∼ N ( 0 , 1 ) \xi\sim N(0,1) ξ ∼ N ( 0 , 1 ) X = s ( Z 1 ) + ξ X=s(Z_1)+\xi X = s ( Z 1 ) + ξ Y = s ( Z 1 ) + γ X 2 + ε Y=s(Z_1)+\gamma X^2+\varepsilon Y = s ( Z 1 ) + γ X 2 + ε Heteroscedastic: ξ + 1 ∼ Exp ( 1 ) \xi+1\sim\operatorname{Exp}(1) ξ + 1 ∼ Exp ( 1 ) X = s ( Z 1 ) ( 1 − ξ ) X=s(Z_1)(1-\xi) X = s ( Z 1 ) ( 1 − ξ ) Y = s ( Z 1 ) + γ X + ε Y=s(Z_1)+\gamma X+\varepsilon Y = s ( Z 1 ) + γ X + ε X X X Z Z Z § 11 .§ 3 .
Methods compared We consider PCM and GCM procedures with permutation or asymptotic calibration. These four procedures use random forest nuisance regression and the adaptive permutation procedure in Algorithm 5, with B max = 499 B_{\max}=499 B m a x = 499 h = 15 h=15 h = 15 [27 ] . The resampling law is set to be N { μ ^ x ( Z ) , σ ^ x 2 } N\{\hat{\mu}_x(Z),\hat{\sigma}_x^2\} N { μ ^ x ( Z ) , σ ^ x 2 } μ ^ x ( ⋅ ) \hat{\mu}_x(\cdot) μ ^ x ( ⋅ ) σ ^ x 2 \hat{\sigma}_x^2 σ ^ x 2 ∑ i = 1 n ( X i − μ ^ x ) 2 / ( n − 1 ) \sum_{i=1}^n(X_i-\hat{\mu}_x)^2/(n-1) ∑ i = 1 n ( X i − μ ^ x ) 2 / ( n − 1 ) p p p p p p
Results interpretation Figure 4 shows that permutation calibration brings both GCM and PCM substantially closer to nominal Type I error in all three designs, although permutation PCM remains mildly liberal under heteroscedasticity. In-sample dCRT is materially liberal at three nulls, with rejection rates 0.108, 0.117, and 0.125. The quadratic alternatives in the first two rows favor PCM. Under the linear Heteroscedastic alternative, GCM is more powerful than PCM. After empirical size adjustment, the tests calibrated by permutation have power broadly similar to their asymptotic counterparts. These adjusted curves are descriptive rather than comparisons with exact correction for size, because the adaptive permutation p p p
Figure 4. GCM and PCM using random forests with marginal permutation or asymptotic calibration, together with in-sample dCRT using conditional resampling calibration. Colors denote calibration and line or border types denote the test. Black dashed lines mark the nominal level 0.05.
Overall, permutation calibration improves null calibration while preserving the complementary sensitivity of PCM and GCM.
Permutation benefit under low effective sample size
Asymptotic calibration is least reliable when the effective sample size is low, which is precisely the regime where marginal permutation is most useful. We demonstrate this benefit in two complementary genomic settings: a semisynthetic Type I error study on Cancer Cell Line Encyclopedia (CCLE) mutation data [1 ] , and an FDR and power comparison on Perturb-seq data [14 ] .
A revisit on CCLE experiment: a semisynthetic analysis
The Cancer Cell Line Encyclopedia pairs genomic characterization of human cancer cell lines with their pharmacological response profiles, and a primary use of the resource is to screen which mutations carry information about drug sensitivity beyond the remaining genomic background [1 ] . Such a screening question is precisely a conditional independence hypothesis, which has been studied in Kook and Lundborg [22 ] using the asymptotic GCM and PCM tests. The mutation panel is nevertheless extremely sparse: a somatic alteration in any given gene is present in only a small fraction of the cell lines, so each binary indicator is zero for nearly every line. Among our screened targets the mutation frequencies range from 0.002 to 0.055, that is, from roughly one to twenty-six nonzero indicators out of 474. The asymptotic calibration adopted in Kook and Lundborg [22 ] does not account for this sparsity, yet it is exactly the regime in which a normal reference is least trustworthy.
Semisynthetic design The experiment uses the n = 474 n = 474 n = 474 p = 466 p = 466 p = 466 M \mathcal{M} M j ∈ M j \in\mathcal{M} j ∈ M X = X j \mathbf{X} = X_j X = X j Y Y Y Z = X − j \mathbf{Z} = X_{-j} Z = X − j ∣ M ∣ = 20 |\mathcal{M}| = 20 ∣ M ∣ = 20 Y Y Y Z Z Z s = 10 s = 10 s = 10 Y ⊥ ⊥ X j ∣ Z Y \perp\!\!\perp X_j \mid Z Y ⊥ ⊥ X j ∣ Z X j X_j X j Z Z Z Figure 5 . The detailed data-generating procedure is given in Appendix § 12 .§ 1 .
Figure 5. CCLE semisynthetic null calibration and power. Rows are the two groups defined by mutation sparsity, high sparsity (representative EIF5A, mutation frequency 0.008) on top and low sparsity (representative ATP11A, frequency 0.040) below. Column (a) summarizes the 10 conditional null targets in each sparsity group; column (b) shows the representative target used to illustrate null calibration; column (c) shows power of two selected mutations in column (b). Dashed black lines mark 0.05. The rejection rates are averaged over 500 independent draws of the synthetic outcome.
Methods compared We compare GCM tests using normal or permutation calibration and the same random forest nuisance estimators (Wright and Ziegler, [37 ] ) for E [ Y ∣ X − j ] \mathbb{E}[Y \mid X_{-j}] E [ Y ∣ X − j ] E [ X j ∣ X − j ] \mathbb{E}[X_j \mid X_{-j}] E [ X j ∣ X − j ] h = 15 h = 15 h = 15 B max = 499 B_{\max} = 499 B m a x = 499
Results interpretation Columns (a)–(b) of Figure 5 extend the EIF5A illustration of Figure 1 to the groups with high and low sparsity, one per row. Averaged over the 20 displayed targets across both groups, GCM with permutation calibration rejects at 0.050 and asymptotic GCM at 0.031, and the gap is confined to rare mutations: in the group with high sparsity asymptotic GCM rejects at 0.013 against 0.049 for permutation, whereas the two agree at 0.049 and 0.051 in the group with low sparsity. The intuitive reason is that the large amount of zeros precludes fast CLT convergence while permutation traces the sampling distribution much better. Pearson test rejects at 0.246, confirming that the constructed CI null retains substantial marginal confounding. Column (c) reports rejection at the nominal cutoff over the chosen effect grids. The cost of this conservatism appears under the alternative: in the row with high sparsity asymptotic GCM rejects less often than permutation at every effect size, 0.42 against 0.72 at the largest shift, whereas the two coincide in the row with low sparsity.
FDR and power on Frangieh Perturb-seq data
Genomic background Single-cell CRISPR screens probe gene function by combining genetic perturbations with measurements of gene expression in individual cells. Guide RNAs (gRNAs) direct perturbations to selected genomic targets, allowing researchers to assess how disrupting one gene changes the expression of that gene or others. We analyze the control condition of the melanoma Perturb-CITE-seq study of Frangieh et al. [14 ] , which combined pooled CRISPR–Cas9 perturbations with single-cell RNA and surface protein measurements to investigate mechanisms of cancer immune evasion. Our analysis uses the RNA measurements.
For each perturbation–gene pair, X X X Y Y Y Z Z Z X ⊥ ⊥ Y ∣ Z X \perp\!\!\perp Y \mid Z X ⊥ ⊥ Y ∣ Z
Evaluation with positive and negative controls We construct 20 replicates of an analysis containing 50 positive control pairs and 950 negative control pairs. The positive controls pair each of the 50 most highly expressed targeted genes with the gRNAs targeting that same gene. Across replicates, we hold the observed data and these positive control pairs fixed and regenerate the 950 negative control pairs by sampling response genes with replacement from a fixed eligible pool of non-targeted genes and pairing each with three distinct, randomly sampled non-targeting gRNAs. Appendix § 12 .§ 2 gives further details of the construction of control pairs.
Methods compared We compare the five score calibrations from § 5.1 , using the same fitted negative binomial null model and efficient score contributions for every method. All tests adjust for log ( 1 + M ) \log(1+M) log ( 1 + M ) M M M p p p q = 0.05 q = 0.05 q = 0.05 h = 15 h = 15 h = 15 B max = 25,000 B_{\max} = 25{,}000 B m a x = 25 , 000 Figure 6 reports averages across replicates.
Figure 6. Empirical false discovery rate and power for five calibrations of the same negative binomial efficient score in the Frangieh Perturb-seq analysis, at target BH level q = 0.05 q = 0.05 q = 0.05 ± 1 \pm1 ± 1
Results interpretation Both permutation calibrations achieve empirical FDR close to the 5% target in this benchmark. The permutation tests using information and empirical studentization yield FDRs of 4.97% and 4.56%, respectively, with corresponding power of 74.2% and 72.2%. The parametric score detects 78.0% of positive controls, but its empirical FDR of 6.56% exceeds the target. The two sign-flip tests exhibit much greater FDR inflation, reaching 58.91% and 58.89%. Their common positive control detection rate of 75.1% therefore does not represent power at the target FDR.
The nearly identical results for the two sign-flip procedures show that standardizing each sign flip separately does not remedy their miscalibration in this sparse setting. This pattern mirrors the simulation with low SNR in Figure 3 : permutation calibration remains accurate when sign-flip calibration fails. Figure 8 in Appendix § 12 .§ 2 shows the null and positive-control p p p
Figure 8. Pooled Frangieh p p p Figure 3 .
Conclusion
This paper develops a framework using marginal permutation for conditional independence testing based on empirically studentized generalized score statistics. Although marginal permutation disrupts the dependence between X X X Z Z Z E [ Y ∣ X , Z ] = μ β ( Z ) \mathbb{E}[Y \mid X, Z] = \mu_{\beta}(Z) E [ Y ∣ X , Z ] = μ β ( Z )
Several directions merit further study. First, it would be valuable to extend permutation calibration to other statistics in the class of generalized scores and to establish conditions under which it delivers superior finite-sample calibration relative to conventional asymptotic references. Second, although adaptive sampling and computational shortcuts reduce the cost of permutation, repeated permutation refitting can remain burdensome in large-scale applications. Efficient analytical approximations to the conditional permutation distribution, building on classical saddlepoint approximations for permutation tests and more recent conditional saddlepoint methodology [30 , 28 ] , could further accelerate these procedures. Finally, the theory developed here is first order: it establishes asymptotic validity and the absence of a first-order loss of local power, but does not explain the finite-sample gains produced by permutation calibration. Developing a higher-order theory that quantifies when and by how much permutation improves calibration would therefore be especially informative, although it is likely to be technically challenging because the permutation distribution depends on nuisance functions estimated and refitted from the data.
Supplementary materials
Supplementary conventions. In the PCM proofs of size validity and permutation stability, n ≡ ∣ D 1 ∣ = n ⋆ n \equiv|D_1| = n_\star n ≡ ∣ D 1 ∣ = n ⋆ n n n n 1 , n 2 n_1, n_2 n 1 , n 2
Practical implementation
The permutation calibration of § 2 is valid, but in its naïve form it can be expensive. For the GCM and PCM, each permutation in principle requires refitting a nuisance regression: the regression of X X X Z Z Z μ ^ x ( π ) \hat{\mu}_x^{(\pi)} μ ^ x ( π ) μ ^ f ( π ) \hat{\mu}_f^{(\pi)} μ ^ f ( π ) B B B B × B \times B ×
Two distinct inefficiencies are at play: the refit required for each permutation, and the fixed Monte Carlo budget, which draws far more permutations than a decision typically requires. We address them in turn. The first vanishes outright whenever the regression is a linear smoother: response linearity reduces every refit to a cached matrix–vector product, which we elaborate in Appendix § 9 .§ 1 . The second is mitigated by an adaptive stopping rule that draws only as many permutations as the data demand, which we describe in Appendix § 9 .§ 2 .
Suppose the learner for regressing X X X Z Z Z linear smoother (Definition 2): its fitted values are μ ^ x = S X \hat{\mu}_x = SX μ ^ x = S X X ≡ ( X 1 , … , X n ) T X \equiv(X_1, \ldots, X_n)^\mathsf{T} X ≡ ( X 1 , … , X n ) T S ∈ R n × n S \in\mathbb{R}^{n \times n} S ∈ R n × n Z Z Z π ∈ G \pi\in\mathcal{G} π ∈ G X ( π ) ≡ ( X π ( 1 ) , … , X π ( n ) ) T X^{(\pi)} \equiv(X_{\pi(1)}, \ldots, X_{\pi(n)})^\mathsf{T} X ( π ) ≡ ( X π ( 1 ) , … , X π ( n ) ) T X X X S S S ( X π ( i ) , Z i ) i = 1 n (X_{\pi(i)}, Z_i)_{i=1}^n ( X π ( i ) , Z i ) i = 1 n μ ^ x ( π ) = S X ( π ) \hat{\mu}_x^{(\pi)} = SX^{(\pi)} μ ^ x ( π ) = S X ( π )
ν G C M , ( π ) = X ( π ) − S X ( π ) = ( I − S ) X ( π ) . \nu^{\mathrm{GCM},(\pi)} = X^{(\pi)} - SX^{(\pi)} = (I - S)X^{(\pi)}. ν GCM , ( π ) = X ( π ) − S X ( π ) = ( I − S ) X ( π ) . The smoother is therefore fit once ; every subsequent permutation reuses the cached residual operator I − S I - S I − S B B B B B B § 16 separately characterizes when the resulting error measuring permutation stability is negligible. The same reduction applies to the projection step of the PCM whenever the projection learner is a linear smoother.
For learners that are genuinely nonlinear in the response—random forests, gradient boosting, neural networks—no such shortcut exists, and the adaptive scheme below is aimed at exactly that regime.
Adaptive permutation calibration
Even when each permutation is cheap, the scheme with a fixed budget of B B B B B B T n T_n T n T n T_n T n [6 ] , we replace the fixed budget by a sequential rule that draws permutations one at a time and stops as soon as the data permit a definitive decision.
Fix an integer h ≥ 1 h \ge1 h ≥ 1 B max ∈ N B_{\max} \in\mathbb{N} B m a x ∈ N ρ \rho ρ ρ ( u ) = u \rho(u) = u ρ ( u ) = u ρ ( u ) = ∣ u ∣ \rho(u) = |u| ρ ( u ) = ∣ u ∣ T n T_n T n T n ( Π t ) T_n^{(\Pi_t)} T n ( Π t ) t t t W 0 ≡ ρ ( T n ) W_0 \equiv\rho(T_n) W 0 ≡ ρ ( T n ) W t ≡ ρ ( T n ( Π t ) ) W_t \equiv\rho(T_n^{(\Pi_t)}) W t ≡ ρ ( T n ( Π t ) ) t ≥ 1 t \ge1 t ≥ 1 loss ,
K t ≡ ∑ s = 1 t 1 { W s ≥ W 0 } , t h ≡ inf { t ≥ 1 : K t = h } , K_t \equiv\sum_{s=1}^{t} \mathbf{1}\{W_s \ge W_0\}, \qquad t_h \equiv\inf\{t \ge1 : K_t = h\}, K t ≡ s = 1 ∑ t 1 { W s ≥ W 0 } , t h ≡ inf { t ≥ 1 : K t = h } , the anytime-valid generalization of the Besag–Clifford Monte Carlo p-value described by Fischer et al. [13 ] is
p t A V ≡ { h t + h − K t , K t < h , h t h , K t ≥ h . (12) p_t^{\mathrm{AV}} \equiv
\begin{cases}
\frac{h}{t+h-K_t}, & K_t < h,\\[6pt]
\frac{h}{t_h}, & K_t \ge h.
\end{cases} \tag*{(12)} p t AV ≡ ⎩ ⎨ ⎧ t + h − K t h , t h h , K t < h , K t ≥ h . (12) The procedure rejects as soon as p t A V ≤ α p_t^{\mathrm{AV}} \le\alpha p t AV ≤ α h h h p t A V > α p_t^{\mathrm{AV}} > \alpha p t AV > α p p p h / t h h/t_h h / t h G t ≡ σ ( W 0 , … , W t ) \mathcal{G}_t \equiv\sigma(W_0, \ldots, W_t) G t ≡ σ ( W 0 , … , W t ) τ ≤ B max \tau\le B_{\max} τ ≤ B m a x G t \mathcal{G}_t G t B max B_{\max} B m a x
Under the alternative the loss count grows slowly and the test rejects after a few refits. Under the null the h h h B B B § 6 , where the test is repeated thousands of times to assess differential expression across many genes or screen for significant variables. The full procedure is stated in Algorithm 5.
Validity of the adaptive permutation calibration
The adaptive rule recomputes the prescribed weight component at every permutation, by either a nuisance refit or an exact cached update, yet it inherits the validity of the procedures with a fixed budget. Under the strong marginal independence null, the entire score stream is conditionally exchangeable and the process counting losses is finite-sample anytime valid. Under the conditions of Theorem 1, the observed statistic and each fresh permutation statistic converge to the same continuous limit. The asymptotic guarantee below keeps h h h B max B_{\max} B m a x n → ∞ n \to\infty n → ∞
Algorithm 5: Adaptive permutation calibration for a generalized score statistic
Input: Data, level α ∈ ( 0 , 1 ) \alpha\in(0,1) α ∈ ( 0 , 1 ) h ≥ 1 h \ge1 h ≥ 1 B max ≥ 1 B_{\max} \ge1 B m a x ≥ 1 ρ \rho ρ T n T_n T n
Use randomness independent of the data, with independent randomization for any sample split, learner seeds, and permutation draws;
If splitting is used, fix the split and reserve the training fold; permute only exposure labels in the inference fold below;
Fix the learner seeds and use them for every fit; compute T n T_n T n W 0 = ρ ( T n ) W_0=\rho(T_n) W 0 = ρ ( T n )
Initialise t ← 0 t \leftarrow0 t ← 0 K ← 0 K \leftarrow0 K ← 0 t h ← ∞ t_h \leftarrow\infty t h ← ∞
while t < B max t < B_{\max} t < B m a x do
t ← t + 1 t \leftarrow t+1 t ← t + 1
Draw Π t \Pi_t Π t
Compute T n ( Π t ) T_n^{(\Pi_t)} T n ( Π t ) ( Y , Z ) (Y,Z) ( Y , Z )
Set W t = ρ ( T n ( Π t ) ) W_t=\rho(T_n^{(\Pi_t)}) W t = ρ ( T n ( Π t ) )
if W t ≥ W 0 W_t \ge W_0 W t ≥ W 0 then
K ← K + 1 K \leftarrow K+1 K ← K + 1
end
if K = h K=h K = h for the first time then
t h ← t t_h \leftarrow t t h ← t
end
Compute p t A V p_t^{\mathrm{AV}} p t AV (12) ;
if p t A V ≤ α p_t^{\mathrm{AV}} \le\alpha p t AV ≤ α then reject and stop;
if K = h K=h K = h and p t A V > α p_t^{\mathrm{AV}}>\alpha p t AV > α then stop for futility;
end
if the loop ends without an earlier stop then
return p B max A V p_{B_{\max}}^{\mathrm{AV}} p B m a x AV
end
Theorem 6 (Finite-sample validity of adaptive calibration under MI). Let p τ A V p_{\tau}^{\mathrm{AV}} p τ AV p p p ( X i , Y i , Z i ) i = 1 n (X_i,Y_i,Z_i)_{i=1}^n ( X i , Y i , Z i ) i = 1 n H 0 M I : X ⊥ ⊥ ( Y , Z ) H_0^{\mathrm{MI}}: X \perp\!\!\!\perp(Y,Z) H 0 MI : X ⊥ ⊥ ( Y , Z ) n n n α ∈ ( 0 , 1 ) \alpha\in(0,1) α ∈ ( 0 , 1 ) P ( p τ A V ≤ α ) ≤ α \mathbb{P}(p_{\tau}^{\mathrm{AV}}\le\alpha)\le\alpha P ( p τ AV ≤ α ) ≤ α
Theorem 7 (Asymptotic validity of adaptive calibration). Let p τ A V p_{\tau}^{\mathrm{AV}} p τ AV p p p h h h B max B_{\max} B m a x n → ∞ n\to\infty n → ∞ ρ \rho ρ ρ ( G ) \rho(G) ρ ( G ) G ∼ N ( 0 , 1 ) G\sim N(0,1) G ∼ N ( 0 , 1 ) α ∈ ( 0 , 1 ) \alpha\in(0,1) α ∈ ( 0 , 1 ) P ( p τ A V ≤ α ) ≤ α + o ( 1 ) \mathbb{P}(p_{\tau}^{\mathrm{AV}}\le\alpha)\le\alpha+o(1) P ( p τ AV ≤ α ) ≤ α + o ( 1 )
Theorems 2, 4, and 5 verify the conditions of Theorem 1 for the empirically studentized GLM score, GCM, and PCM, respectively. Both score maps used here, ρ ( u ) = u \rho(u)=u ρ ( u ) = u ρ ( u ) = ∣ u ∣ \rho(u)=|u| ρ ( u ) = ∣ u ∣
Proofs of Theorem 6 and Theorem 7
We present auxiliary results in Appendix § 9 .§ 4.1 and prove Theorems 6 and 7 in Appendix § 9 .§ 4.2 and Appendix § 9 .§ 4.4 , respectively.
Auxiliary results and their proofs
We first prove an exchangeability result for an infinite score stream. An infinite formulation is convenient because the anytime-valid property is independent of any deterministic computational cap. Algorithm 5 simply stops this stream no later than B max B_{\max} B m a x
Lemma 1 (Anytime-valid bound from conditional exchangeability). Fix an integer h ≥ 1 h \ge1 h ≥ 1 H \mathcal{H} H σ \sigma σ ( W t ) t ≥ 0 (W_t)_{t\ge0} ( W t ) t ≥ 0 H \mathcal{H} H G t ≡ σ ( W 0 , … , W t ) \mathcal{G}_t \equiv\sigma(W_0,\ldots,W_t) G t ≡ σ ( W 0 , … , W t ) K t ≡ ∑ s = 1 t 1 { W s ≥ W 0 } K_t \equiv\sum_{s=1}^{t}\mathbf{1}\{W_s \ge W_0\} K t ≡ ∑ s = 1 t 1 { W s ≥ W 0 } t h ≡ inf { t ≥ 1 : K t = h } t_h \equiv\inf\{t \ge1: K_t=h\} t h ≡ inf { t ≥ 1 : K t = h } p t A V p_t^{\mathrm{AV}} p t AV (12) , with the convention h / ∞ = 0 h/\infty=0 h /∞ = 0 ( G t ) (\mathcal{G}_t) ( G t ) τ ≥ 1 \tau\ge1 τ ≥ 1 α ∈ ( 0 , 1 ) \alpha\in(0,1) α ∈ ( 0 , 1 )
P ( p τ A V ≤ α ∣ H ) ≤ α almost surely . \mathbb{P}(p_{\tau}^{\mathrm{AV}} \le\alpha\mid\mathcal{H}) \le\alpha\quad\text{almost surely}. P ( p τ AV ≤ α ∣ H ) ≤ α almost surely . Consequently, P ( p τ A V ≤ α ) ≤ α \mathbb{P}(p_{\tau}^{\mathrm{AV}} \le\alpha) \le\alpha P ( p τ AV ≤ α ) ≤ α
Proof. Fix an integer k ≥ h k \ge h k ≥ h j ∈ { 0 , … , k − 1 } j \in\{0,\ldots,k-1\} j ∈ { 0 , … , k − 1 }
R k , j ≡ ∑ s = 0 k − 1 1 { W s ≥ W j } , A k ≡ { j ∈ { 0 , … , k − 1 } : R k , j ≤ h } . R_{k,j} \equiv\sum_{s=0}^{k-1}\mathbf{1}\{W_s \ge W_j\}, \qquad A_k \equiv\{j \in\{0,\ldots,k-1\}: R_{k,j} \le h\}. R k , j ≡ s = 0 ∑ k − 1 1 { W s ≥ W j } , A k ≡ { j ∈ { 0 , … , k − 1 } : R k , j ≤ h } . We have ∣ A k ∣ ≤ h |A_k| \le h ∣ A k ∣ ≤ h A k A_k A k j ⋆ ∈ A k j^\star\in A_k j ⋆ ∈ A k W j ⋆ = min j ∈ A k W j W_{j^\star}=\min_{j\in A_k}W_j W j ⋆ = min j ∈ A k W j s ∈ A k s \in A_k s ∈ A k W s ≥ W j ⋆ W_s \ge W_{j^\star} W s ≥ W j ⋆
∣ A k ∣ ≤ R k , j ⋆ ≤ h . |A_k| \le R_{k,j^\star} \le h. ∣ A k ∣ ≤ R k , j ⋆ ≤ h . Conditional exchangeability implies that P ( R k , j ≤ h ∣ H ) \mathbb{P}(R_{k,j} \le h \mid\mathcal{H}) P ( R k , j ≤ h ∣ H ) j j j R k , j R_{k,j} R k , j R k , 0 = 1 + K k − 1 R_{k,0}=1+K_{k-1} R k , 0 = 1 + K k − 1
P ( K k − 1 ≤ h − 1 ∣ H ) = P ( R k , 0 ≤ h ∣ H ) = 1 k ∑ j = 0 k − 1 P ( R k , j ≤ h ∣ H ) = 1 k E [ ∣ A k ∣ ∣ H ] ≤ h k . (13) \mathbb{P}(K_{k-1} \le h-1 \mid\mathcal{H}) = \mathbb{P}(R_{k,0} \le h \mid\mathcal{H})
= \frac{1}{k}\sum_{j=0}^{k-1}\mathbb{P}(R_{k,j} \le h \mid\mathcal{H})
= \frac{1}{k}\mathbb{E}[|A_k| \mid\mathcal{H}] \le\frac{h}{k}. \tag*{(13)} P ( K k − 1 ≤ h − 1 ∣ H ) = P ( R k , 0 ≤ h ∣ H ) = k 1 j = 0 ∑ k − 1 P ( R k , j ≤ h ∣ H ) = k 1 E [ ∣ A k ∣ ∣ H ] ≤ k h . (13) If t < t h < ∞ t<t_h<\infty t < t h < ∞
h − K t = K t h − K t = ∑ s = t + 1 t h 1 { W s ≥ W 0 } ≤ t h − t . h-K_t=K_{t_h}-K_t=\sum_{s=t+1}^{t_h}\mathbf{1}\{W_s\ge W_0\}\le t_h-t. h − K t = K t h − K t = s = t + 1 ∑ t h 1 { W s ≥ W 0 } ≤ t h − t . Thus t h ≥ t + h − K t t_h \ge t+h-K_t t h ≥ t + h − K t t h = ∞ t_h=\infty t h = ∞ (12) ,
p t A V = h t + h − K t ≥ h t h if t < t h , p t A V = h t h if t ≥ t h . p_t^{\mathrm{AV}}=\frac{h}{t+h-K_t}\ge\frac{h}{t_h}\quad\text{if }t<t_h,\qquad p_t^{\mathrm{AV}}=\frac{h}{t_h}\quad\text{if }t\ge t_h. p t AV = t + h − K t h ≥ t h h if t < t h , p t AV = t h h if t ≥ t h . Hence, for every almost surely finite stopping time τ ≥ 1 \tau\ge1 τ ≥ 1
{ p τ A V ≤ α } ⊆ { h t h ≤ α } = { t h ≥ ⌈ h α ⌉ } . (14) \{p_\tau^{\mathrm{AV}}\le\alpha\}\subseteq\left\{\frac{h}{t_h}\le\alpha\right\}=\left\{t_h\ge\left\lceil\frac{h}{\alpha}\right\rceil\right\}. \tag*{(14)} { p τ AV ≤ α } ⊆ { t h h ≤ α } = { t h ≥ ⌈ α h ⌉ } . (14) Let k = ⌈ h / α ⌉ k=\lceil h/\alpha\rceil k = ⌈ h / α ⌉ K t K_t K t t h ≥ k t_h\ge k t h ≥ k K k − 1 ≤ h − 1 K_{k-1}\le h-1 K k − 1 ≤ h − 1 (13) and (14) give
P ( p τ A V ≤ α ∣ H ) ≤ P ( K k − 1 ≤ h − 1 ∣ H ) ≤ h k ≤ α . \mathbb{P}(p_\tau^{\mathrm{AV}}\le\alpha\mid\mathcal{H})\le\mathbb{P}(K_{k-1}\le h-1\mid\mathcal{H})\le\frac{h}{k}\le\alpha. P ( p τ AV ≤ α ∣ H ) ≤ P ( K k − 1 ≤ h − 1 ∣ H ) ≤ k h ≤ α . Taking expectations proves the unconditional statement.
Lemma 2 (Joint convergence from a conditional deterministic limit). Let X n → d X \mathbf{X}_n\xrightarrow{d}\mathbf{X} X n d X F n ⊇ σ ( X n ) \mathcal{F}_n\supseteq\sigma(\mathbf{X}_n) F n ⊇ σ ( X n ) L ≥ 1 L\ge1 L ≥ 1 F n \mathcal{F}_n F n Y n , 1 , … , Y n , L \mathbf{Y}_{n,1},\ldots,\mathbf{Y}_{n,L} Y n , 1 , … , Y n , L
Y n , 1 ∣ F n → d , p Y , \mathbf{Y}_{n,1}\mid\mathcal{F}_n\xrightarrow{d,p}\mathbf{Y}, Y n , 1 ∣ F n d , p Y , where the law of Y \mathbf{Y} Y
( X n , Y n , 1 , … , Y n , L ) → d ( X , Y 1 , … , Y L ) , (\mathbf{X}_n,\mathbf{Y}_{n,1},\ldots,\mathbf{Y}_{n,L})\xrightarrow{d}(\mathbf{X},\mathbf{Y}_1,\ldots,\mathbf{Y}_L), ( X n , Y n , 1 , … , Y n , L ) d ( X , Y 1 , … , Y L ) , where Y 1 , … , Y L \mathbf{Y}_1,\ldots,\mathbf{Y}_L Y 1 , … , Y L Y \mathbf{Y} Y X \mathbf{X} X
Proof. Let ψ n ( u ) ≡ E [ e i u Y n , 1 ∣ F n ] \psi_n(u)\equiv\mathbb{E}[e^{iu\mathbf{Y}_{n,1}}\mid\mathcal{F}_n] ψ n ( u ) ≡ E [ e i u Y n , 1 ∣ F n ] ψ ( u ) ≡ E [ e i u Y ] \psi(u)\equiv\mathbb{E}[e^{iu\mathbf{Y}}] ψ ( u ) ≡ E [ e i u Y ] ψ n ( u ) → ψ ( u ) \psi_n(u)\to\psi(u) ψ n ( u ) → ψ ( u ) ∣ ψ n ∣ ≤ 1 |\psi_n|\le1 ∣ ψ n ∣ ≤ 1 L 1 L^1 L 1 ( u 0 , … , u L ) (u_0,\ldots,u_L) ( u 0 , … , u L )
E e i u 0 X n + i ∑ t = 1 L u t Y n , t = E [ e i u 0 X n ∏ t = 1 L ψ n ( u t ) ] ⟶ E e i u 0 X ∏ t = 1 L ψ ( u t ) . \mathbb{E}e^{iu_0\mathbf{X}_n+i\sum_{t=1}^{L}u_t\mathbf{Y}_{n,t}}
=\mathbb{E}\left[e^{iu_0\mathbf{X}_n}\prod_{t=1}^{L}\psi_n(u_t)\right]\longrightarrow\mathbb{E}e^{iu_0\mathbf{X}}\prod_{t=1}^{L}\psi(u_t). E e i u 0 X n + i ∑ t = 1 L u t Y n , t = E [ e i u 0 X n t = 1 ∏ L ψ n ( u t ) ] ⟶ E e i u 0 X t = 1 ∏ L ψ ( u t ) . Lévy’s continuity theorem proves the stated independent joint limit.
Proposition 4 (Transfer to adaptive calibration over a bounded horizon). Fix L ≥ 1 L\ge1 L ≥ 1 W n , 0 W_{n,0} W n , 0 F n \mathcal{F}_n F n F n \mathcal{F}_n F n W n , 1 , … , W n , L W_{n,1},\ldots,W_{n,L} W n , 1 , … , W n , L W W W
W n , 0 → d W , W n , 1 ∣ F n → d , p W . W_{n,0}\xrightarrow{d}W,\qquad W_{n,1}\mid\mathcal{F}_n\xrightarrow{d,p}W. W n , 0 d W , W n , 1 ∣ F n d , p W . Then, for every stopping time τ n ≤ L \tau_n\le L τ n ≤ L σ ( W n , 0 , … , W n , t ) \sigma(W_{n,0},\ldots,W_{n,t}) σ ( W n , 0 , … , W n , t )
P ( p τ n A V ≤ α ) ≤ α + o ( 1 ) . \mathbb{P}(p_{\tau_n}^{\mathrm{AV}}\le\alpha)\le\alpha+o(1). P ( p τ n AV ≤ α ) ≤ α + o ( 1 ) . Proof. Lemma 2 gives
( W n , 0 , W n , 1 , … , W n , L ) → d ( W 0 ⋆ , W 1 ⋆ , … , W L ⋆ ) , (W_{n,0},W_{n,1},\ldots,W_{n,L}) \xrightarrow{d} (W_0^{\star},W_1^{\star},\ldots,W_L^{\star}), ( W n , 0 , W n , 1 , … , W n , L ) d ( W 0 ⋆ , W 1 ⋆ , … , W L ⋆ ) , where W 0 ⋆ , … , W L ⋆ W_0^{\star},\ldots,W_L^{\star} W 0 ⋆ , … , W L ⋆ W W W p L ⋆ , A V p_L^{\star,\mathrm{AV}} p L ⋆ , AV t ↦ p t A V t \mapsto p_t^{\mathrm{AV}} t ↦ p t AV h h h t + h − K t t+h-K_t t + h − K t h h h
{ p τ n A V ≤ α } ⊆ { p L A V ≤ α } . \{p_{\tau_n}^{\mathrm{AV}}\leq\alpha\}\subseteq\{p_L^{\mathrm{AV}}\leq\alpha\}. { p τ n AV ≤ α } ⊆ { p L AV ≤ α } . The indicator of the event { p L A V ≤ α } \{p_L^{\mathrm{AV}}\leq\alpha\} { p L AV ≤ α } s t s_t s t t ≥ 1 t\geq1 t ≥ 1 s 0 s_0 s 0 1 { s t ≥ s 0 } \mathbf{1}\{s_t\geq s_0\} 1 { s t ≥ s 0 } W W W
P ( ∃ t ∈ { 1 , … , L } : W t ⋆ = W 0 ⋆ ) = 0. \mathbb{P}\left(\exists t\in\{1,\ldots,L\}:W_t^{\star}=W_0^{\star}\right)=0. P ( ∃ t ∈ { 1 , … , L } : W t ⋆ = W 0 ⋆ ) = 0. Thus { p L A V ≤ α } \{p_L^{\mathrm{AV}}\leq\alpha\} { p L AV ≤ α }
P ( p L A V ≤ α ) ⟶ P ( p L ⋆ , A V ≤ α ) . \mathbb{P}(p_L^{\mathrm{AV}}\leq\alpha)\longrightarrow\mathbb{P}(p_L^{\star,\mathrm{AV}}\leq\alpha). P ( p L AV ≤ α ) ⟶ P ( p L ⋆ , AV ≤ α ) . The infinite i.i.d. extension of ( W 0 ⋆ , … , W L ⋆ ) (W_0^{\star},\ldots,W_L^{\star}) ( W 0 ⋆ , … , W L ⋆ ) P ( p L ⋆ , A V ≤ α ) ≤ α \mathbb{P}(p_L^{\star,\mathrm{AV}}\leq\alpha)\leq\alpha P ( p L ⋆ , AV ≤ α ) ≤ α
Proof of Theorem 6
Proof. Let I I I X ( 0 ) = ( X i ) i ∈ I X^{(0)}=(X_i)_{i\in I} X ( 0 ) = ( X i ) i ∈ I X ( t ) X^{(t)} X ( t ) t t t H \mathcal{H} H ( Y i , Z i ) i ∈ I (Y_i,Z_i)_{i\in I} ( Y i , Z i ) i ∈ I ( X i ) i ∈ I (X_i)_{i\in I} ( X i ) i ∈ I
By i.i.d. sampling under MI and the algorithm’s independent randomization, X ( 0 ) ∣ H X^{(0)}\mid\mathcal{H} X ( 0 ) ∣ H t ≥ 1 t\geq1 t ≥ 1
L ( X ( t ) ∣ H , X ( 0 ) , … , X ( t − 1 ) ) = L ( X ( 0 ) ∣ H ) . \mathcal{L}(X^{(t)}\mid\mathcal{H},X^{(0)},\ldots,X^{(t-1)})=\mathcal{L}(X^{(0)}\mid\mathcal{H}). L ( X ( t ) ∣ H , X ( 0 ) , … , X ( t − 1 ) ) = L ( X ( 0 ) ∣ H ) . Hence X ( 0 ) , X ( 1 ) , … X^{(0)},X^{(1)},\ldots X ( 0 ) , X ( 1 ) , … H \mathcal{H} H H \mathcal{H} H W 0 , W 1 , … W_0,W_1,\ldots W 0 , W 1 , … τ ≤ B max \tau\leq B_{\max} τ ≤ B m a x P ( p τ A V ≤ α ∣ H ) ≤ α \mathbb{P}(p_\tau^{\mathrm{AV}}\leq\alpha\mid\mathcal{H})\leq\alpha P ( p τ AV ≤ α ∣ H ) ≤ α
Proof of Theorem 7
Proof. Set L = B max L=B_{\max} L = B m a x F n \mathcal{F}_n F n W n , 0 = ρ ( T n ) W_{n,0}=\rho(T_n) W n , 0 = ρ ( T n ) W n , t = ρ ( T n ( Π t ) ) W_{n,t}=\rho(T_n^{(\Pi_t)}) W n , t = ρ ( T n ( Π t ) ) 1 ≤ t ≤ L 1\leq t\leq L 1 ≤ t ≤ L F n \mathcal{F}_n F n F n \mathcal{F}_n F n ρ \rho ρ
W n , 0 → d ρ ( G ) , W n , 1 ∣ F n → d , p ρ ( G ) , G ∼ N ( 0 , 1 ) . W_{n,0} \xrightarrow{d} \rho(G), \qquad W_{n,1} \mid\mathcal{F}_n \xrightarrow{d,p} \rho(G), \qquad G \sim N(0,1). W n , 0 d ρ ( G ) , W n , 1 ∣ F n d , p ρ ( G ) , G ∼ N ( 0 , 1 ) . Since ρ ( G ) \rho(G) ρ ( G ) τ ≤ L \tau\le L τ ≤ L P ( p τ A V ≤ α ) ≤ α + o ( 1 ) \mathbb{P}(p_{\tau}^{\mathrm{AV}} \le\alpha) \le\alpha+ o(1) P ( p τ AV ≤ α ) ≤ α + o ( 1 )
Additional simulation results and details
This section provides additional simulation results and details that complement the main text, including details for the experiment generating Figure 2, details for negative binomial simulations and results under marginal independence.
Data generation details in fig:2
This appendix records the complete procedure used to generate the data for Figure 2. Independently across observations, generate
Z i ∘ ∼ Ber ( 3 / 4 ) , D i = ( 1 , Z i ∘ ) T , X i ∣ Z i ∘ = z ∼ N ( 0 , s z 2 ) , Var ( Y i ∣ Z i ∘ = z ) = c ( z ) μ y ( z ) , Z_i^{\circ} \sim\operatorname{Ber}(3/4), \quad D_i = (1,Z_i^{\circ})^{\mathsf{T}}, \quad X_i \mid Z_i^{\circ}=z \sim N(0,s_z^2), \quad\operatorname{Var}(Y_i \mid Z_i^{\circ}=z)=c(z)\mu_y(z), Z i ∘ ∼ Ber ( 3/4 ) , D i = ( 1 , Z i ∘ ) T , X i ∣ Z i ∘ = z ∼ N ( 0 , s z 2 ) , Var ( Y i ∣ Z i ∘ = z ) = c ( z ) μ y ( z ) , where s 0 = 1 s_0=1 s 0 = 1 s 1 = 1 / 3 s_1=1/3 s 1 = 1/3 μ y ( 0 ) = 20 \mu_y(0)=20 μ y ( 0 ) = 20 μ y ( 1 ) = 30 \mu_y(1)=30 μ y ( 1 ) = 30 c ( z ) > 1 c(z)>1 c ( z ) > 1 Y i Y_i Y i μ y ( z ) \mu_y(z) μ y ( z ) μ y ( z ) / { c ( z ) − 1 } \mu_y(z)/\{c(z)-1\} μ y ( z ) / { c ( z ) − 1 } c ( z ) μ y ( z ) c(z)\mu_y(z) c ( z ) μ y ( z ) c ( z ) = 1 c(z)=1 c ( z ) = 1 Y i Y_i Y i μ y ( z ) \mu_y(z) μ y ( z ) D i D_i D i Z i ∘ Z_i^{\circ} Z i ∘ X X X
The multiplicative design sets c ( 0 ) = c ( 1 ) = 5 c(0)=c(1)=5 c ( 0 ) = c ( 1 ) = 5 c ( 0 ) = 5 c(0)=5 c ( 0 ) = 5 c ( 1 ) = 1 c(1)=1 c ( 1 ) = 1 ( Z i ∘ , X i ) (Z_i^{\circ},X_i) ( Z i ∘ , X i )
Negative binomial simulation details
Simulation parameters All designs use n = 400 n=400 n = 400
Γ H i g h = { 0 , − 0.1125 , − 0.225 , − 0.3375 , − 0.45 } , \Gamma_{\mathrm{High}}=\{0,-0.1125,-0.225,-0.3375,-0.45\}, Γ High = { 0 , − 0.1125 , − 0.225 , − 0.3375 , − 0.45 } , Γ L o w = { 0 , − 0.2125 , − 0.425 , − 0.6375 , − 0.85 } , \Gamma_{\mathrm{Low}}=\{0,-0.2125,-0.425,-0.6375,-0.85\}, Γ Low = { 0 , − 0.2125 , − 0.425 , − 0.6375 , − 0.85 } , Γ V a r , M I = { 0 , − 0.055 , − 0.11 , − 0.165 , − 0.22 } , \Gamma_{\mathrm{Var,MI}}=\{0,-0.055,-0.11,-0.165,-0.22\}, Γ Var , MI = { 0 , − 0.055 , − 0.11 , − 0.165 , − 0.22 } , Γ V a r , C I = { 0 , − 0.0675 , − 0.135 , − 0.2025 , − 0.27 } . \Gamma_{\mathrm{Var,CI}}=\{0,-0.0675,-0.135,-0.2025,-0.27\}. Γ Var , CI = { 0 , − 0.0675 , − 0.135 , − 0.2025 , − 0.27 } . The grids for high and low SNR are used under both MI and CI.
Results under marginal independence We give the MI counterpart to the CI experiment in § 5.1 . Under MI, X ⊥ ⊥ ( Y , Z ) X \perp\!\!\perp(Y,Z) X ⊥ ⊥ ( Y , Z ) § 5.1 , the null rejection rates under high SNR are (5.05, 5.55, 5.50, 5.35, 5.30)%. All five methods are therefore close to the nominal level when the exposure is common. When the exposure prevalence is 0.025, the null rejection rates under low SNR are (3.80, 4.25, 5.25, 12.25, 12.10)%. Our method remains near nominal, whereas both sign-flip procedures reject far too often. Thus, the failure of the sign-flip reference when scores are sparse persists even without association between X X X Z Z Z
Together with the CI results, these findings show that empirically studentized permutation is the procedure whose rejection rate is most consistently close to nominal across all six null settings.
Parameters for the simulations in Section 5.2
At γ = 0 \gamma= 0 γ = 0 X ⊥ ⊥ Z X \perp\!\!\perp Z X ⊥ ⊥ Z n ∈ { 20 , 40 , 60 , 80 , 100 } n \in\{20, 40, 60, 80, 100\} n ∈ { 20 , 40 , 60 , 80 , 100 } n ∈ { 200 , 300 , 400 , 500 , 600 } n \in\{200, 300, 400, 500, 600\} n ∈ { 200 , 300 , 400 , 500 , 600 } Figure 4 reports the results when n = 100 n = 100 n = 100 n = 600 n = 600 n = 600 γ ∈ { 0 , 0.2 , 0.4 , 0.6 , 0.8 } \gamma\in\{0, 0.2, 0.4, 0.6, 0.8\} γ ∈ { 0 , 0.2 , 0.4 , 0.6 , 0.8 } X ⊥ ⊥ Z X \perp\!\!\perp Z X ⊥ ⊥ Z γ ∈ { 0 , 0.075 , 0.15 , 0.225 , 0.30 } \gamma\in\{0, 0.075, 0.15, 0.225, 0.30\} γ ∈ { 0 , 0.075 , 0.15 , 0.225 , 0.30 } γ ∈ { 0 , 0.05 , 0.10 , 0.15 , 0.20 } \gamma\in\{0, 0.05, 0.10, 0.15, 0.20\} γ ∈ { 0 , 0.05 , 0.10 , 0.15 , 0.20 }
Additional details for analyses in Section 6
CCLE semisynthetic analysis
Data and targets We use the Cancer Cell Line Encyclopedia panels of drug responses and mutations distributed with Kook and Lundborg [22 ] . After restricting to cell lines observed in both panels, the data contain the PLX4720 area under the dose-response curve and 1638 binary mutation indicators on n = 474 n = 474 n = 474 ∣ cor ^ ( Y r e a l , X k ) ∣ > 0.05 |\widehat{\operatorname{cor}}(Y_{\mathrm{real}}, X_k)| > 0.05 ∣ cor ( Y real , X k ) ∣ > 0.05 p = 466 p = 466 p = 466 set.seed(2026)). We report the 20 targets from the lowest- and highest-frequency groups. Their frequency ranges are [0.002, 0.015] and [0.034, 0.055] for high and low sparsity, respectively. The representatives used to assess null calibration are EIF5A_MUT and ATP11A_MUT from high and low sparsity groups, respectively.
Synthetic response For target j j j Z = X − j Z = X_{-j} Z = X − j j j j s = 10 s = 10 s = 10 Z Z Z S S S s s s β k ∼ i . i . d . N ( 0 , 1 ) \beta_k \overset{\mathrm{i.i.d.}}{\sim} N(0,1) β k ∼ i.i.d. N ( 0 , 1 ) k ∈ S k \in S k ∈ S ∣ I ∣ = min { 2 s , ( s 2 ) } |I| = \min\{2s, \binom{s}{2}\} ∣ I ∣ = min { 2 s , ( 2 s ) } γ k ℓ ∼ i . i . d . N ( 0 , 1 ) \gamma_{k\ell} \overset{\mathrm{i.i.d.}}{\sim} N(0,1) γ k ℓ ∼ i.i.d. N ( 0 , 1 )
μ i = ∑ k ∈ S β k Z i k + ∑ ( k , ℓ ) ∈ I γ k ℓ Z i k Z i ℓ , Y i = μ i + ε i . \mu_i = \sum_{k\in S} \beta_k Z_{ik} + \sum_{(k,\ell)\in I} \gamma_{k\ell} Z_{ik}Z_{i\ell}, \qquad Y_i = \mu_i + \varepsilon_i. μ i = k ∈ S ∑ β k Z ik + ( k , ℓ ) ∈ I ∑ γ k ℓ Z ik Z i ℓ , Y i = μ i + ε i . For each target and replicate, we calculate the empirical standard deviation of the signal across the n n n
μ ˉ ≡ 1 n ∑ i = 1 n μ i , σ μ ≡ { 1 n − 1 ∑ i = 1 n ( μ i − μ ˉ ) 2 } 1 / 2 , σ ≡ { σ μ , 0 < σ μ < ∞ , 1 , otherwise . \bar{\mu} \equiv\frac{1}{n}\sum_{i=1}^{n}\mu_i, \qquad\sigma_\mu\equiv\left\{\frac{1}{n-1}\sum_{i=1}^{n}(\mu_i-\bar{\mu})^2\right\}^{1/2}, \qquad\sigma\equiv\begin{cases} \sigma_\mu, & 0<\sigma_\mu<\infty, \\ 1, & \text{otherwise}. \end{cases} μ ˉ ≡ n 1 i = 1 ∑ n μ i , σ μ ≡ { n − 1 1 i = 1 ∑ n ( μ i − μ ˉ ) 2 } 1/2 , σ ≡ { σ μ , 1 , 0 < σ μ < ∞ , otherwise . Conditional on Z Z Z ε i ∼ N ( 0 , σ 2 ) \varepsilon_i \sim N(0,\sigma^2) ε i ∼ N ( 0 , σ 2 ) σ μ \sigma_\mu σ μ
Figure 7. Score calibration under marginal independence. Rows show high SNR, low SNR, and misspecified negative binomial variance; columns show null p p p n = 400 n = 400 n = 400
divided by the specified noise variance is exactly one; the realized variance of the noise varies across replicates. The response is generated from Z Z Z Y ⊥ ⊥ X j ∣ Z Y \perp\!\!\!\perp X_j \mid Z Y ⊥ ⊥ X j ∣ Z
Representative example construction. For the paired extension with conditional effects, we retain the 20 reported δ = 0 \delta= 0 δ = 0
Y i ( δ ) = Y i ( 0 ) + δ σ X i j . Y_i^{(\delta)} = Y_i^{(0)} + \delta\sigma X_{ij}. Y i ( δ ) = Y i ( 0 ) + δ σ X ij . The EIF5A target with four carriers uses δ ∈ { 0 , 1 , 2 , 3 , 4 } \delta\in\{0, 1, 2, 3, 4\} δ ∈ { 0 , 1 , 2 , 3 , 4 } δ ∈ { 0 , 0.5 , 1 , 1.5 , 2 } \delta\in\{0, 0.5, 1, 1.5, 2\} δ ∈ { 0 , 0.5 , 1 , 1.5 , 2 }
Methods and replication GCM uses random forest regressions of Y Y Y Z Z Z X j X_j X j Z Z Z p p p h = 15 h = 15 h = 15 B max = 499 B_{\max} = 499 B m a x = 499 R = 500 R = 500 R = 500 α = 0.05 \alpha= 0.05 α = 0.05
Frangieh Perturb-seq analysis
Data and pair construction We use the control condition of the Perturb-seq experiment [14 ] . It includes 57,624 cells with nonzero gene UMIs, 23,712 genes, 717 gene-targeting gRNAs spanning 239 targets, and 74 non-targeting (NT) gRNAs. The common control population consists of 4,148 cells carrying only NT gRNAs. We consider 20 replicates: the positive pairs are fixed and the 950 null pairs are resampled from the total pool of negative control pairs. The positive controls are the 50 most highly expressed genes among the available targets. Their treatment indicator identifies cells carrying any gRNA targeting the gene, with 238 treated cells on average. For each null, we pair a random gene outside the positive control set, with mean expression at least 0.005 with three randomly sampled NT gRNAs.
Adaptive resampling hyperparameters. Each of the four randomization methods uses two-sided adaptive BH calibration at q = 0.05 q = 0.05 q = 0.05 h = 15 h = 15 h = 15 B max = 25,000 B_{\max} = 25{,}000 B m a x = 25 , 000
Additional diagnostic plot. Figure 8 separates the p p p p p p p p p
General lemmas
Definition 1 (Conditional convergence in distribution; [27 Definition 1] ). *Let W n W_n W n F n \mathcal{F}_n F n σ \sigma σ
that W n W_n W n W W W F n \mathcal{F}_n F n t ∈ R t \in\mathbb{R} t ∈ R W W W P ( W n ≤ t ∣ F n ) → P P ( W ≤ t ) \mathbb{P}(W_n \le t \mid\mathcal{F}_n) \xrightarrow{\mathbb{P}} \mathbb{P}(W \le t) P ( W n ≤ t ∣ F n ) P P ( W ≤ t ) W n ∣ F n → d , p W W_n \mid\mathcal{F}_n \xrightarrow{d,p} W W n ∣ F n d , p W
Lemma 3 (Conditional Slutsky; [27 Supplement, Theorem 3] ). Let W n W_n W n F n \mathcal{F}_n F n σ \sigma σ a n → P 1 a_n \xrightarrow{\mathbb{P}} 1 a n P 1 b n → P 0 b_n \xrightarrow{\mathbb{P}} 0 b n P 0 W n ∣ F n → d , p W W_n \mid\mathcal{F}_n \xrightarrow{d,p} W W n ∣ F n d , p W W W W a n W n + b n ∣ F n → d , p W a_nW_n+b_n \mid\mathcal{F}_n \xrightarrow{d,p} W a n W n + b n ∣ F n d , p W
Lemma 4 (Conditional weak convergence implies marginal convergence). Let W n W_n W n F n \mathcal{F}_n F n W n ∣ F n → d , p W W_n \mid\mathcal{F}_n \xrightarrow{d,p} W W n ∣ F n d , p W W W W W n → d W W_n \xrightarrow{d} W W n d W
Proof. Fix a continuity point t t t W W W
P n ( t ) ≡ P ( W n ≤ t ∣ F n ) → P P ( W ≤ t ) . P_n(t) \equiv\mathbb{P}(W_n \le t \mid\mathcal{F}_n) \xrightarrow{\mathbb{P}} \mathbb{P}(W \le t). P n ( t ) ≡ P ( W n ≤ t ∣ F n ) P P ( W ≤ t ) . Since 0 ≤ P n ( t ) ≤ 1 0 \le P_n(t) \le1 0 ≤ P n ( t ) ≤ 1
P ( W n ≤ t ) = E [ P n ( t ) ] ⟶ P ( W ≤ t ) . \mathbb{P}(W_n \le t) = \mathbb{E}[P_n(t)] \longrightarrow\mathbb{P}(W \le t). P ( W n ≤ t ) = E [ P n ( t )] ⟶ P ( W ≤ t ) . As t t t W n → d W W_n \xrightarrow{d} W W n d W
Lemma 5 (Conditional quantile convergence; [27 Supplement, Lemma 1] ). Let W n W_n W n F n \mathcal{F}_n F n σ \sigma σ α ∈ ( 0 , 1 ) \alpha\in(0,1) α ∈ ( 0 , 1 ) W n ∣ F n → d , p W W_n \mid\mathcal{F}_n \xrightarrow{d,p} W W n ∣ F n d , p W W W W Q α [ W ] Q_\alpha[W] Q α [ W ] Q α [ W n ∣ F n ] → P Q α [ W ] Q_\alpha[W_n \mid\mathcal{F}_n] \xrightarrow{\mathbb{P}} Q_\alpha[W] Q α [ W n ∣ F n ] P Q α [ W ]
Lemma 6 (Conditional Lyapunov central limit theorem). For each n n n G n \mathcal{G}_n G n W n 1 , … , W n k n W_{n1},\ldots,W_{nk_n} W n 1 , … , W n k n G n \mathcal{G}_n G n
E [ W n i ∣ G n ] = 0 , i = 1 , … , k n . \mathbb{E}[W_{ni}\mid\mathcal{G}_n]=0,\qquad i=1,\ldots,k_n. E [ W ni ∣ G n ] = 0 , i = 1 , … , k n . Write s n 2 = ∑ i = 1 k n E [ W n i 2 ∣ G n ] s_n^2=\sum_{i=1}^{k_n}\mathbb{E}[W_{ni}^2\mid\mathcal{G}_n] s n 2 = ∑ i = 1 k n E [ W ni 2 ∣ G n ] 0 < s n 2 < ∞ 0<s_n^2<\infty 0 < s n 2 < ∞ δ > 0 \delta>0 δ > 0
L n ≡ ∑ i = 1 k n E [ ∣ W n i ∣ 2 + δ ∣ G n ] s n 2 + δ → P 0. L_n\equiv\frac{\sum_{i=1}^{k_n}\mathbb{E}[|W_{ni}|^{2+\delta}\mid\mathcal{G}_n]}{s_n^{2+\delta}}\xrightarrow{\mathrm{P}}0. L n ≡ s n 2 + δ ∑ i = 1 k n E [ ∣ W ni ∣ 2 + δ ∣ G n ] P 0. Then
∑ i = 1 k n W n i s n ∣ G n → d , P N ( 0 , 1 ) . \left.\frac{\sum_{i=1}^{k_n}W_{ni}}{s_n}\right|\mathcal{G}_n\xrightarrow{d,\mathrm{P}}N(0,1). s n ∑ i = 1 k n W ni G n d , P N ( 0 , 1 ) . Both ratios are defined as zero on { s n 2 ∉ ( 0 , ∞ ) } \{s_n^2\notin(0,\infty)\} { s n 2 ∈ / ( 0 , ∞ )}
Proof. Fix t ∈ R t\in\mathbb{R} t ∈ R T n = ∑ i W n i / s n T_n=\sum_iW_{ni}/s_n T n = ∑ i W ni / s n K n ( ω , ⋅ ) K_n(\omega,\cdot) K n ( ω , ⋅ ) ( W n i ) i = 1 k n (W_{ni})_{i=1}^{k_n} ( W ni ) i = 1 k n G n \mathcal{G}_n G n K n , i K_{n,i} K n , i i i i R k n \mathbb{R}^{k_n} R k n n n n Ω n \Omega_n Ω n P ( Ω n ) = 1 \mathbb{P}(\Omega_n)=1 P ( Ω n ) = 1 i i i
K n ( ω , ⋅ ) = ⨂ i = 1 k n K n , i ( ω , ⋅ ) , ∫ f ( x ) K n , i ( ω , d x ) = E [ f ( W n i ) ∣ G n ] ( ω ) , K_n(\omega,\cdot)=\bigotimes_{i=1}^{k_n}K_{n,i}(\omega,\cdot),\qquad
\int f(x)K_{n,i}(\omega,dx)=\mathbb{E}[f(W_{ni})\mid\mathcal{G}_n](\omega), K n ( ω , ⋅ ) = i = 1 ⨂ k n K n , i ( ω , ⋅ ) , ∫ f ( x ) K n , i ( ω , d x ) = E [ f ( W ni ) ∣ G n ] ( ω ) , for f ( x ) = x , x 2 , ∣ x ∣ 2 + δ f(x)=x,x^2,|x|^{2+\delta} f ( x ) = x , x 2 , ∣ x ∣ 2 + δ ( n j ) (n_j) ( n j )
∑ j = 1 ∞ { P ( L n j > j − 1 ) + P ( s n j 2 ∉ ( 0 , ∞ ) ) } < ∞ . \sum_{j=1}^{\infty}\left\{\mathbb{P}(L_{n_j}>j^{-1})+\mathbb{P}(s_{n_j}^2\notin(0,\infty))\right\}<\infty. j = 1 ∑ ∞ { P ( L n j > j − 1 ) + P ( s n j 2 ∈ / ( 0 , ∞ )) } < ∞. By Borel–Cantelli, there is an event Ω B C \Omega_{\mathrm{BC}} Ω BC
L n j ( ω ) ⟶ 0 , 0 < s n j 2 ( ω ) < ∞ eventually . L_{n_j}(\omega)\longrightarrow0,\qquad0<s_{n_j}^2(\omega)<\infty\quad\text{eventually}. L n j ( ω ) ⟶ 0 , 0 < s n j 2 ( ω ) < ∞ eventually . Set
Ω ⋆ = Ω B C ∩ ⋂ n = 1 ∞ Ω n , P ( Ω ⋆ ) = 1. \Omega_\star=\Omega_{\mathrm{BC}}\cap\bigcap_{n=1}^{\infty}\Omega_n,\qquad\mathbb{P}(\Omega_\star)=1. Ω ⋆ = Ω BC ∩ n = 1 ⋂ ∞ Ω n , P ( Ω ⋆ ) = 1. Fix ω ∈ Ω ⋆ \omega\in\Omega_\star ω ∈ Ω ⋆ P ω \mathbb{P}_\omega P ω E ω \mathbb{E}_\omega E ω ( V j , i ) (V_{j,i}) ( V j , i ) B ⊆ R k n j B\subseteq\mathbb{R}^{k_{n_j}} B ⊆ R k n j
P ω ( ( V j , i ) i = 1 k n j ∈ B ) = K n j ( ω , B ) . \mathbb{P}_\omega\left((V_{j,i})_{i=1}^{k_{n_j}}\in B\right)=K_{n_j}(\omega,B). P ω ( ( V j , i ) i = 1 k n j ∈ B ) = K n j ( ω , B ) . The product identity on Ω ⋆ \Omega_\star Ω ⋆ V j , 1 , … , V j , k n j V_{j,1},\ldots,V_{j,k_{n_j}} V j , 1 , … , V j , k n j P ω \mathbb{P}_\omega P ω j j j
E ω [ V j , i ] = E [ W n j i ∣ G n j ] ( ω ) = 0 , \mathbb{E}_\omega[V_{j,i}]=\mathbb{E}[W_{n_j i}\mid\mathcal{G}_{n_j}](\omega)=0, E ω [ V j , i ] = E [ W n j i ∣ G n j ] ( ω ) = 0 , ∑ i = 1 k n j E ω [ V j , i 2 ] = s n j 2 ( ω ) ∈ ( 0 , ∞ ) , \sum_{i=1}^{k_{n_j}}\mathbb{E}_\omega[V_{j,i}^2]=s_{n_j}^2(\omega)\in(0,\infty), i = 1 ∑ k n j E ω [ V j , i 2 ] = s n j 2 ( ω ) ∈ ( 0 , ∞ ) , ∑ i = 1 k n j E ω [ ∣ V j , i ∣ 2 + δ ] s n j 2 + δ ( ω ) = L n j ( ω ) ⟶ 0. \frac{\sum_{i=1}^{k_{n_j}}\mathbb{E}_\omega[|V_{j,i}|^{2+\delta}]}{s_{n_j}^{2+\delta}(\omega)}
=L_{n_j}(\omega)\longrightarrow0. s n j 2 + δ ( ω ) ∑ i = 1 k n j E ω [ ∣ V j , i ∣ 2 + δ ] = L n j ( ω ) ⟶ 0. The ordinary Lyapunov central limit theorem therefore gives, under P ω \mathbb{P}_{\omega} P ω
∑ i = 1 k n j V j , i s n j ( ω ) → d N ( 0 , 1 ) . \frac{\sum_{i=1}^{k_{n_j}} V_{j,i}}{s_{n_j}(\omega)} \xrightarrow{\mathrm{d}} N(0,1). s n j ( ω ) ∑ i = 1 k n j V j , i d N ( 0 , 1 ) . Since s n j s_{n_j} s n j G n j \mathcal{G}_{n_j} G n j K n j \mathcal{K}_{n_j} K n j
P ( T n j ≤ t ∣ G n j ) ( ω ) = P ω ( ∑ i V j , i s n j ( ω ) ≤ t ) ⟶ Φ ( t ) . \mathbb{P}(T_{n_j}\le t\mid\mathcal{G}_{n_j})(\omega)=\mathbb{P}_{\omega}\left(\frac{\sum_i V_{j,i}}{s_{n_j}(\omega)}\le t\right)\longrightarrow\Phi(t). P ( T n j ≤ t ∣ G n j ) ( ω ) = P ω ( s n j ( ω ) ∑ i V j , i ≤ t ) ⟶ Φ ( t ) . Since the initial subsequence was arbitrary, the subsequence criterion yields
P ( T n ≤ t ∣ G n ) → P Φ ( t ) , \mathbb{P}(T_n\le t\mid\mathcal{G}_n)\xrightarrow{\mathbb{P}}\Phi(t), P ( T n ≤ t ∣ G n ) P Φ ( t ) , as required.
Lemma 7 (Hájek central limit theorem; Barry et al. [4 ] ). Let X 1 , X 2 , … X_1,X_2,\ldots X 1 , X 2 , … E [ X i ] = 0 \mathbb{E}[X_i]=0 E [ X i ] = 0 E [ X i 2 ] = 1 \mathbb{E}[X_i^2]=1 E [ X i 2 ] = 1 n ≥ 2 n\ge2 n ≥ 2 ( c n , i ) i = 1 n (c_{n,i})_{i=1}^{n} ( c n , i ) i = 1 n
(i) ∑ i = 1 n c n , i = 0 \sum_{i=1}^{n}c_{n,i}=0 ∑ i = 1 n c n , i = 0
(ii) ∑ i = 1 n c n , i 2 = 1 \sum_{i=1}^{n}c_{n,i}^{2}=1 ∑ i = 1 n c n , i 2 = 1
(iii) lim d → ∞ lim sup n → ∞ ∑ i = 1 n c n , i 2 1 { ∣ n c n , i ∣ ≥ d } = 0 \lim_{d\to\infty}\limsup_{n\to\infty}\sum_{i=1}^{n}c_{n,i}^{2}\mathbf{1}\{|\sqrt{n}c_{n,i}|\ge d\}=0 lim d → ∞ lim sup n → ∞ ∑ i = 1 n c n , i 2 1 { ∣ n c n , i ∣ ≥ d } = 0
(iv) max 1 ≤ i ≤ n ∣ X i ∣ / n → P 0 \max_{1\le i\le n}|X_i|/\sqrt{n}\xrightarrow{\mathbb{P}}0 max 1 ≤ i ≤ n ∣ X i ∣/ n P 0
Let F n ≡ σ ( X 1 , … , X n , c n , 1 , … , c n , n ) \mathcal{F}_n\equiv\sigma(X_1,\ldots,X_n,c_{n,1},\ldots,c_{n,n}) F n ≡ σ ( X 1 , … , X n , c n , 1 , … , c n , n ) π \pi π { 1 , … , n } \{1,\ldots,n\} { 1 , … , n } F n \mathcal{F}_n F n ∑ i = 1 n X π ( i ) c n , i ∣ F n → d , P N ( 0 , 1 ) \sum_{i=1}^{n}X_{\pi(i)}c_{n,i}\mid\mathcal{F}_n\xrightarrow{\mathrm{d},\mathbb{P}}N(0,1) ∑ i = 1 n X π ( i ) c n , i ∣ F n d , P N ( 0 , 1 )
Lemma 8 (Conditional Markov and Chebyshev transfer). Let G n \mathcal{G}_n G n
(a) If W n ≥ 0 W_n\ge0 W n ≥ 0 E [ W n ∣ G n ] → P 0 \mathbb{E}[W_n\mid\mathcal{G}_n]\xrightarrow{\mathbb{P}}0 E [ W n ∣ G n ] P 0 W n → P 0 W_n\xrightarrow{\mathbb{P}}0 W n P 0
(b) If X n X_n X n E [ X n 2 ∣ G n ] < ∞ \mathbb{E}[X_n^2\mid\mathcal{G}_n]<\infty E [ X n 2 ∣ G n ] < ∞
E [ X n ∣ G n ] → P 0 , Var ( X n ∣ G n ) → P 0 , \mathbb{E}[X_n\mid\mathcal{G}_n]\xrightarrow{\mathbb{P}}0,\qquad\operatorname{Var}(X_n\mid\mathcal{G}_n)\xrightarrow{\mathbb{P}}0, E [ X n ∣ G n ] P 0 , Var ( X n ∣ G n ) P 0 , then X n → P 0 X_n\xrightarrow{\mathbb{P}}0 X n P 0
Proof. For part (a), fix ε , η > 0 \varepsilon,\eta>0 ε , η > 0 m n ≡ E [ W n ∣ G n ] m_n\equiv\mathbb{E}[W_n\mid\mathcal{G}_n] m n ≡ E [ W n ∣ G n ]
P ( W n > ε ) ≤ P ( m n > ε η ) + E [ 1 { m n ≤ ε η } P ( W n > ε ∣ G n ) ] ≤ P ( m n > ε η ) + η . \mathbb{P}(W_n>\varepsilon)\le\mathbb{P}(m_n>\varepsilon\eta)+\mathbb{E}\left[\mathbf{1}\{m_n\le\varepsilon\eta\}\mathbb{P}(W_n>\varepsilon\mid\mathcal{G}_n)\right]
\le\mathbb{P}(m_n>\varepsilon\eta)+\eta. P ( W n > ε ) ≤ P ( m n > ε η ) + E [ 1 { m n ≤ ε η } P ( W n > ε ∣ G n ) ] ≤ P ( m n > ε η ) + η . The first term tends to zero; then let η ↓ 0 \eta\downarrow0 η ↓ 0
For part (b), all moments are taken under the conditional law given G n \mathcal{G}_n G n m n ≡ E [ X n ∣ G n ] m_n\equiv\mathbb{E}[X_n\mid\mathcal{G}_n] m n ≡ E [ X n ∣ G n ]
E [ ( X n − m n ) 2 ∣ G n ] = Var ( X n ∣ G n ) → P 0 , \mathbb{E}[(X_n-m_n)^2\mid\mathcal{G}_n]=\operatorname{Var}(X_n\mid\mathcal{G}_n)\xrightarrow{\mathbb{P}}0, E [( X n − m n ) 2 ∣ G n ] = Var ( X n ∣ G n ) P 0 , part (a), applied to ( X n − m n ) 2 (X_n-m_n)^2 ( X n − m n ) 2 X n − m n → P 0 X_n-m_n\xrightarrow{\mathbb{P}}0 X n − m n P 0 m n → P 0 m_n\xrightarrow{\mathbb{P}}0 m n P 0
Lemma 9. (Maximal bound from a moment condition). Let G n \mathcal{G}_n G n Z n 1 , … , Z n n Z_{n1},\ldots,Z_{nn} Z n 1 , … , Z nn G n \mathcal{G}_n G n E [ ∣ Z n 1 ∣ q ∣ G n ] = O P ( 1 ) \mathbb{E}[|Z_{n1}|^q\mid\mathcal{G}_n]=O_{\mathbb{P}}(1) E [ ∣ Z n 1 ∣ q ∣ G n ] = O P ( 1 ) q > 0 q>0 q > 0 max 1 ≤ i ≤ n ∣ Z n i ∣ = O P ( n 1 / q ) \max_{1\le i\le n}|Z_{ni}|=O_{\mathbb{P}}(n^{1/q}) max 1 ≤ i ≤ n ∣ Z ni ∣ = O P ( n 1/ q ) E ∣ Z n 1 ∣ q = O ( 1 ) \mathbb{E}|Z_{n1}|^q=O(1) E ∣ Z n 1 ∣ q = O ( 1 )
Proof. For M > 0 M>0 M > 0
P ( max i ∣ Z n i ∣ > M n 1 / q | G n ) ≤ 1 M q E [ ∣ Z n 1 ∣ q ∣ G n ] . \mathbb{P}\left(\max_i |Z_{ni}|>Mn^{1/q}\,\middle|\,\mathcal{G}_n\right)\le\frac{1}{M^q}\mathbb{E}[|Z_{n1}|^q\mid\mathcal{G}_n]. P ( i max ∣ Z ni ∣ > M n 1/ q G n ) ≤ M q 1 E [ ∣ Z n 1 ∣ q ∣ G n ] . Given ε > 0 \varepsilon>0 ε > 0 K < ∞ K<\infty K < ∞ q q q K K K ε \varepsilon ε n n n K / M q K/M^q K / M q M M M K / M q ≤ ε K/M^q\le\varepsilon K / M q ≤ ε G n \mathcal{G}_n G n □ \square □
Lemma 10. (Hoeffding identity; Barry et al. [4 ] ). Let n ≥ 2 n\ge2 n ≥ 2 ( a i ) i = 1 n (a_i)_{i=1}^n ( a i ) i = 1 n ( b i ) i = 1 n (b_i)_{i=1}^n ( b i ) i = 1 n Π \Pi Π { 1 , … , n } \{1,\ldots,n\} { 1 , … , n } ( a i , b i ) i = 1 n (a_i,b_i)_{i=1}^n ( a i , b i ) i = 1 n
E [ 1 n ∑ i = 1 n a Π ( i ) b i ∣ ( a i , b i ) i = 1 n ] = ( 1 n ∑ i = 1 n a i ) ( 1 n ∑ i = 1 n b i ) , \mathbb{E}\left[\left.\frac{1}{n}\sum_{i=1}^n a_{\Pi(i)}b_i\right|(a_i,b_i)_{i=1}^n\right]=\left(\frac{1}{n}\sum_{i=1}^n a_i\right)\left(\frac{1}{n}\sum_{i=1}^n b_i\right), E [ n 1 i = 1 ∑ n a Π ( i ) b i ( a i , b i ) i = 1 n ] = ( n 1 i = 1 ∑ n a i ) ( n 1 i = 1 ∑ n b i ) , and
Var ( 1 n ∑ i = 1 n a Π ( i ) b i ∣ ( a i , b i ) i = 1 n ) ≤ 1 n − 1 ( 1 n ∑ i = 1 n a i 2 ) ( 1 n ∑ i = 1 n b i 2 ) . \operatorname{Var}\left(\left.\frac{1}{n}\sum_{i=1}^n a_{\Pi(i)}b_i\right|(a_i,b_i)_{i=1}^n\right)\le\frac{1}{n-1}\left(\frac{1}{n}\sum_{i=1}^n a_i^2\right)\left(\frac{1}{n}\sum_{i=1}^n b_i^2\right). Var ( n 1 i = 1 ∑ n a Π ( i ) b i ( a i , b i ) i = 1 n ) ≤ n − 1 1 ( n 1 i = 1 ∑ n a i 2 ) ( n 1 i = 1 ∑ n b i 2 ) . Lemma 11. (Combinatorial central limit theorem; Hoeffding [18 ] ). For each n ≥ 2 n\ge2 n ≥ 2 ( c i k ( n ) ) 1 ≤ i , k ≤ n (c_{ik}^{(n)})_{1\le i,k\le n} ( c ik ( n ) ) 1 ≤ i , k ≤ n
∑ k = 1 n c i k ( n ) = 0 \sum_{k=1}^n c_{ik}^{(n)}=0 k = 1 ∑ n c ik ( n ) = 0 for each i i i c ⋅ k ( n ) ≡ n − 1 ∑ i c i k ( n ) c_{\cdot k}^{(n)}\equiv n^{-1}\sum_i c_{ik}^{(n)} c ⋅ k ( n ) ≡ n − 1 ∑ i c ik ( n ) d i k ( n ) ≡ c i k ( n ) − c ⋅ k ( n ) d_{ik}^{(n)}\equiv c_{ik}^{(n)}-c_{\cdot k}^{(n)} d ik ( n ) ≡ c ik ( n ) − c ⋅ k ( n ) Π \Pi Π { 1 , … , n } \{1,\ldots,n\} { 1 , … , n }
(a) E [ ∑ i = 1 n c i , Π ( i ) ( n ) ] = 0 \mathbb{E}\left[\sum_{i=1}^n c_{i,\Pi(i)}^{(n)}\right]=0 E [ ∑ i = 1 n c i , Π ( i ) ( n ) ] = 0
(b) Var ( ∑ i = 1 n c i , Π ( i ) ( n ) ) = 1 n − 1 ∑ i , k ( d i k ( n ) ) 2 \operatorname{Var}\left(\sum_{i=1}^n c_{i,\Pi(i)}^{(n)}\right)=\frac{1}{n-1}\sum_{i,k}(d_{ik}^{(n)})^2 Var ( ∑ i = 1 n c i , Π ( i ) ( n ) ) = n − 1 1 ∑ i , k ( d ik ( n ) ) 2
(c) Suppose ∑ i , k ( d i k ( n ) ) 2 > 0 \sum_{i,k}(d_{ik}^{(n)})^2>0 ∑ i , k ( d ik ( n ) ) 2 > 0 n n n n → ∞ n\to\infty n → ∞
max i , k ( d i k ( n ) ) 2 1 n ∑ i , k ( d i k ( n ) ) 2 → 0 , \frac{\max_{i,k}(d_{ik}^{(n)})^2}{\frac{1}{n}\sum_{i,k}(d_{ik}^{(n)})^2}\to0, n 1 ∑ i , k ( d ik ( n ) ) 2 max i , k ( d ik ( n ) ) 2 → 0 , then
∑ i = 1 n c i , Π ( i ) ( n ) 1 n − 1 ∑ i , k ( d i k ( n ) ) 2 → d N ( 0 , 1 ) . \frac{\sum_{i=1}^n c_{i,\Pi(i)}^{(n)}}{\sqrt{\frac{1}{n-1}\sum_{i,k}(d_{ik}^{(n)})^2}}\xrightarrow{d}N(0,1). n − 1 1 ∑ i , k ( d ik ( n ) ) 2 ∑ i = 1 n c i , Π ( i ) ( n ) d N ( 0 , 1 ) . Lemma 12. (Combinatorial CLT, conditional form (Lemma 11′)). Let ( F n ) n ≥ 1 (\mathcal{F}_n)_{n\ge1} ( F n ) n ≥ 1 σ \sigma σ n ≥ 2 n\ge2 n ≥ 2 ( c i k ( n ) ) 1 ≤ i , k ≤ n (c_{ik}^{(n)})_{1\le i,k\le n} ( c ik ( n ) ) 1 ≤ i , k ≤ n F n \mathcal{F}_n F n
such that ∑ k = 1 n c i k ( n ) = 0 \sum_{k=1}^{n} c_{ik}^{(n)} = 0 ∑ k = 1 n c ik ( n ) = 0 i i i c ⋅ k ( n ) ≡ n − 1 ∑ i c i k ( n ) c_{\cdot k}^{(n)} \equiv n^{-1}\sum_i c_{ik}^{(n)} c ⋅ k ( n ) ≡ n − 1 ∑ i c ik ( n ) d i k ( n ) ≡ c i k ( n ) − c ⋅ k ( n ) d_{ik}^{(n)} \equiv c_{ik}^{(n)} - c_{\cdot k}^{(n)} d ik ( n ) ≡ c ik ( n ) − c ⋅ k ( n ) Π \Pi Π { 1 , … , n } \{1,\ldots,n\} { 1 , … , n } F n \mathcal{F}_n F n
P ( ∑ i , k ( d i k ( n ) ) 2 > 0 ) → 1 \mathbb{P}\left(\sum_{i,k}(d_{ik}^{(n)})^2>0\right)\to1 P i , k ∑ ( d ik ( n ) ) 2 > 0 → 1 and
max i , k ( d i k ( n ) ) 2 1 n ∑ i , k ( d i k ( n ) ) 2 → P 0. \frac{\max_{i,k}(d_{ik}^{(n)})^2}{\frac{1}{n}\sum_{i,k}(d_{ik}^{(n)})^2}\xrightarrow{\mathbb{P}}0. n 1 ∑ i , k ( d ik ( n ) ) 2 max i , k ( d ik ( n ) ) 2 P 0. Then, in the sense of Definition 1,
∑ i = 1 n c i , Π ( i ) ( n ) 1 n − 1 ∑ i , k ( d i k ( n ) ) 2 ∣ F n → d N ( 0 , 1 ) . \left.\frac{\sum_{i=1}^{n}c_{i,\Pi(i)}^{(n)}}{\sqrt{\frac{1}{n-1}\sum_{i,k}(d_{ik}^{(n)})^2}}\right|\mathcal{F}_n\xrightarrow{d}N(0,1). n − 1 1 ∑ i , k ( d ik ( n ) ) 2 ∑ i = 1 n c i , Π ( i ) ( n ) F n d N ( 0 , 1 ) . Both ratios are defined as zero on { ∑ i , k ( d i k ( n ) ) 2 = 0 } \left\{\sum_{i,k}(d_{ik}^{(n)})^2=0\right\} { ∑ i , k ( d ik ( n ) ) 2 = 0 }
Proof. Fix t ∈ R t\in\mathbb{R} t ∈ R R n R_n R n
s n 2 = 1 n − 1 ∑ i , k ( d i k ( n ) ) 2 , T n ( π ) = ∑ i c i , π ( i ) ( n ) s n , s_n^2=\frac{1}{n-1}\sum_{i,k}(d_{ik}^{(n)})^2,\qquad T_n^{(\pi)}=\frac{\sum_i c_{i,\pi(i)}^{(n)}}{s_n}, s n 2 = n − 1 1 i , k ∑ ( d ik ( n ) ) 2 , T n ( π ) = s n ∑ i c i , π ( i ) ( n ) , with T n ( π ) = 0 T_n^{(\pi)}=0 T n ( π ) = 0 s n = 0 s_n=0 s n = 0 F n \mathcal{F}_n F n Π \Pi Π T n ( Π ) T_n^{(\Pi)} T n ( Π ) F n \mathcal{F}_n F n
K n ( ω , B ) = 1 n ! ∑ π 1 { T n ( π ) ( ω ) ∈ B } , B ⊆ R Borel , K_n(\omega,B)=\frac{1}{n!}\sum_{\pi}\mathbf{1}\left\{T_n^{(\pi)}(\omega)\in B\right\},\qquad B\subseteq\mathbb{R}\ \text{Borel}, K n ( ω , B ) = n ! 1 π ∑ 1 { T n ( π ) ( ω ) ∈ B } , B ⊆ R Borel , where the sum runs over all permutations of { 1 , … , n } \{1,\ldots,n\} { 1 , … , n }
From any subsequence, choose a further subsequence ( n j ) (n_j) ( n j )
∑ j = 1 ∞ { P ( R n j > j − 1 ) + P ( s n j 2 = 0 ) } < ∞ . \sum_{j=1}^{\infty}\left\{\mathbb{P}(R_{n_j}>j^{-1})+\mathbb{P}(s_{n_j}^2=0)\right\}<\infty. j = 1 ∑ ∞ { P ( R n j > j − 1 ) + P ( s n j 2 = 0 ) } < ∞. By Borel–Cantelli and the almost-sure row-sum assumptions, the event
Ω ⋆ ≡ { R n j → 0 , s n j 2 > 0 eventually } ∩ ⋂ n ≥ 2 ⋂ i = 1 n { ∑ k = 1 n c i k ( n ) = 0 } \Omega_\star\equiv\{R_{n_j}\to0,\ s_{n_j}^2>0\ \text{eventually}\}\cap\bigcap_{n\geq2}\bigcap_{i=1}^{n}\left\{\sum_{k=1}^{n}c_{ik}^{(n)}=0\right\} Ω ⋆ ≡ { R n j → 0 , s n j 2 > 0 eventually } ∩ n ≥ 2 ⋂ i = 1 ⋂ n { k = 1 ∑ n c ik ( n ) = 0 } satisfies P ( Ω ⋆ ) = 1 \mathbb{P}(\Omega_\star)=1 P ( Ω ⋆ ) = 1
Fix ω ∈ Ω ⋆ \omega\in\Omega_\star ω ∈ Ω ⋆ ( c i k ( n j ) ( ω ) ) i , k (c_{ik}^{(n_j)}(\omega))_{i,k} ( c ik ( n j ) ( ω ) ) i , k j j j
max i , k ( d i k ( n j ) ( ω ) ) 2 n j − 1 ∑ i , k ( d i k ( n j ) ( ω ) ) 2 = R n j ( ω ) ⟶ 0. \frac{\max_{i,k}(d_{ik}^{(n_j)}(\omega))^2}{n_j^{-1}\sum_{i,k}(d_{ik}^{(n_j)}(\omega))^2}=R_{n_j}(\omega)\longrightarrow0. n j − 1 ∑ i , k ( d ik ( n j ) ( ω ) ) 2 max i , k ( d ik ( n j ) ( ω ) ) 2 = R n j ( ω ) ⟶ 0. Lemma 11 therefore gives
K n j ( ω , ( − ∞ , t ] ) = 1 n j ! ∑ π 1 { ∑ i c i , π ( i ) ( n j ) ( ω ) s n j ( ω ) ≤ t } ⟶ Φ ( t ) . K_{n_j}(\omega,(-\infty,t])=\frac{1}{n_j!}\sum_{\pi}\mathbf{1}\left\{\frac{\sum_i c_{i,\pi(i)}^{(n_j)}(\omega)}{s_{n_j}(\omega)}\leq t\right\}\longrightarrow\Phi(t). K n j ( ω , ( − ∞ , t ]) = n j ! 1 π ∑ 1 ⎩ ⎨ ⎧ s n j ( ω ) ∑ i c i , π ( i ) ( n j ) ( ω ) ≤ t ⎭ ⎬ ⎫ ⟶ Φ ( t ) . Since the initial subsequence was arbitrary, the subsequence criterion and the version for the conditional distribution K n K_n K n
P ( T n ( Π ) ≤ t ∣ F n ) = K n ( ⋅ , ( − ∞ , t ] ) → P Φ ( t ) , \mathbb{P}(T_n^{(\Pi)} \le t \mid\mathcal{F}_n)=K_n(\cdot,(-\infty,t]) \xrightarrow{\mathbb{P}} \Phi(t), P ( T n ( Π ) ≤ t ∣ F n ) = K n ( ⋅ , ( − ∞ , t ]) P Φ ( t ) , as required.
Lemma 13 (Moments of permutation averages within rows). Let n ≥ 2 n \ge2 n ≥ 2 F n \mathcal{F}_n F n ( h i k ) 1 ≤ i , k ≤ n (h_{ik})_{1\le i,k\le n} ( h ik ) 1 ≤ i , k ≤ n F n \mathcal{F}_n F n Π \Pi Π { 1 , … , n } \{1,\ldots,n\} { 1 , … , n } F n \mathcal{F}_n F n
h i ⋅ ≡ 1 n ∑ k h i k , h ⋅ k ≡ 1 n ∑ i h i k , h ⋅ ⋅ ≡ 1 n 2 ∑ i , k h i k , h_{i\cdot} \equiv\frac{1}{n}\sum_k h_{ik}, \qquad h_{\cdot k} \equiv\frac{1}{n}\sum_i h_{ik}, \qquad h_{\cdot\cdot} \equiv\frac{1}{n^2}\sum_{i,k}h_{ik}, h i ⋅ ≡ n 1 k ∑ h ik , h ⋅ k ≡ n 1 i ∑ h ik , h ⋅⋅ ≡ n 2 1 i , k ∑ h ik , and the doubly centered array
h ~ i k ≡ h i k − h i ⋅ − h ⋅ k + h ⋅ ⋅ . \widetilde{h}_{ik} \equiv h_{ik}-h_{i\cdot}-h_{\cdot k}+h_{\cdot\cdot}. h ik ≡ h ik − h i ⋅ − h ⋅ k + h ⋅⋅ . Then, conditional on F n \mathcal{F}_n F n
E [ 1 n ∑ i h i , Π ( i ) ∣ F n ] = h ⋅ ⋅ , (15) \mathbb{E}\left[\left.\frac{1}{n}\sum_i h_{i,\Pi(i)}\right|\mathcal{F}_n\right]=h_{\cdot\cdot}, \tag*{(15)} E [ n 1 i ∑ h i , Π ( i ) F n ] = h ⋅⋅ , (15) Var ( 1 n ∑ i h i , Π ( i ) ∣ F n ) = 1 n 2 ( n − 1 ) ∑ i , k h ~ i k 2 ≤ 1 n 2 ( n − 1 ) ∑ i , k h i k 2 . (16) \operatorname{Var}\left(\left.\frac{1}{n}\sum_i h_{i,\Pi(i)}\right|\mathcal{F}_n\right)=\frac{1}{n^2(n-1)}\sum_{i,k}\widetilde{h}_{ik}^{\,2}\le\frac{1}{n^2(n-1)}\sum_{i,k}h_{ik}^{\,2}. \tag*{(16)} Var ( n 1 i ∑ h i , Π ( i ) F n ) = n 2 ( n − 1 ) 1 i , k ∑ h ik 2 ≤ n 2 ( n − 1 ) 1 i , k ∑ h ik 2 . (16) In particular, let ( q i ) i (q_i)_i ( q i ) i ( A i k ) i , k (A_{ik})_{i,k} ( A ik ) i , k F n \mathcal{F}_n F n A i ⋅ ≡ n − 1 ∑ k A i k A_{i\cdot}\equiv n^{-1}\sum_k A_{ik} A i ⋅ ≡ n − 1 ∑ k A ik A i ⋅ = 0 A_{i\cdot}=0 A i ⋅ = 0 i i i h i k = q i A i k h_{ik}=q_iA_{ik} h ik = q i A ik
E [ 1 n ∑ i q i A i , Π ( i ) ∣ F n ] = 0 \mathbb{E}\left[\left.\frac{1}{\sqrt{n}}\sum_i q_iA_{i,\Pi(i)}\right|\mathcal{F}_n\right]=0 E [ n 1 i ∑ q i A i , Π ( i ) F n ] = 0 and
Var ( 1 n ∑ i q i A i , Π ( i ) ∣ F n ) ≤ 1 n ( n − 1 ) ∑ i , k q i 2 A i k 2 . \operatorname{Var}\left(\left.\frac{1}{\sqrt{n}}\sum_i q_iA_{i,\Pi(i)}\right|\mathcal{F}_n\right)\le\frac{1}{n(n-1)}\sum_{i,k}q_i^2A_{ik}^2. Var ( n 1 i ∑ q i A i , Π ( i ) F n ) ≤ n ( n − 1 ) 1 i , k ∑ q i 2 A ik 2 . Proof. Put c i k ≡ h i k − h i ⋅ c_{ik}\equiv h_{ik}-h_{i\cdot} c ik ≡ h ik − h i ⋅
∑ k c i k = 0 , c ⋅ k = h ⋅ k − h ⋅ ⋅ , c i k − c ⋅ k = h ~ i k , \sum_k c_{ik}=0,\qquad c_{\cdot k}=h_{\cdot k}-h_{\cdot\cdot},\qquad c_{ik}-c_{\cdot k}=\widetilde{h}_{ik}, k ∑ c ik = 0 , c ⋅ k = h ⋅ k − h ⋅⋅ , c ik − c ⋅ k = h ik , and
1 n ∑ i h i , Π ( i ) = h ⋅ ⋅ + 1 n ∑ i c i , Π ( i ) . \frac{1}{n}\sum_i h_{i,\Pi(i)}=h_{\cdot\cdot}+\frac{1}{n}\sum_i c_{i,\Pi(i)}. n 1 i ∑ h i , Π ( i ) = h ⋅⋅ + n 1 i ∑ c i , Π ( i ) . Conditional on F n \mathcal{F}_n F n ( c i k ) (c_{ik}) ( c ik ) h ⋅ ⋅ h_{\cdot\cdot} h ⋅⋅ Π \Pi Π ( c i k ) (c_{ik}) ( c ik )
For the inequality, the definition of h ~ \widetilde{h} h
∑ k h ~ i k = 0 , ∑ i h ~ i k = 0. \sum_k\widetilde{h}_{ik}=0,\qquad\sum_i\widetilde{h}_{ik}=0. k ∑ h ik = 0 , i ∑ h ik = 0. Consequently,
∑ i , k h ~ i k ( h i ⋅ + h ⋅ k − h ⋅ ⋅ ) = ∑ i h i ⋅ ∑ k h ~ i k + ∑ k h ⋅ k ∑ i h ~ i k − h ⋅ ⋅ ∑ i , k h ~ i k = 0. \sum_{i,k} \tilde{h}_{ik}(h_{i\cdot}+h_{\cdot k}-h_{\cdot\cdot})=\sum_i h_{i\cdot}\sum_k\tilde{h}_{ik}+\sum_k h_{\cdot k}\sum_i\tilde{h}_{ik}-h_{\cdot\cdot}\sum_{i,k}\tilde{h}_{ik}
=0. i , k ∑ h ~ ik ( h i ⋅ + h ⋅ k − h ⋅⋅ ) = i ∑ h i ⋅ k ∑ h ~ ik + k ∑ h ⋅ k i ∑ h ~ ik − h ⋅⋅ i , k ∑ h ~ ik = 0. Expanding h i k = h ~ i k + h i ⋅ + h ⋅ k − h ⋅ ⋅ h_{ik}=\tilde{h}_{ik}+h_{i\cdot}+h_{\cdot k}-h_{\cdot\cdot} h ik = h ~ ik + h i ⋅ + h ⋅ k − h ⋅⋅
∑ i , k h i k 2 = ∑ i , k h ~ i k 2 + ∑ i , k ( h i ⋅ + h ⋅ k − h ⋅ ⋅ ) 2 ≥ ∑ i , k h ~ i k 2 . \sum_{i,k}h_{ik}^{2}=\sum_{i,k}\tilde{h}_{ik}^{2}+\sum_{i,k}(h_{i\cdot}+h_{\cdot k}-h_{\cdot\cdot})^{2}\geq\sum_{i,k}\tilde{h}_{ik}^{2}. i , k ∑ h ik 2 = i , k ∑ h ~ ik 2 + i , k ∑ ( h i ⋅ + h ⋅ k − h ⋅⋅ ) 2 ≥ i , k ∑ h ~ ik 2 . Finally, if h i k = q i A i k h_{ik}=q_iA_{ik} h ik = q i A ik A i ⋅ = 0 A_{i\cdot}=0 A i ⋅ = 0
h i ⋅ = q i A i ⋅ = 0 , h ⋅ ⋅ = 0. h_{i\cdot}=q_iA_{i\cdot}=0,\qquad h_{\cdot\cdot}=0. h i ⋅ = q i A i ⋅ = 0 , h ⋅⋅ = 0. Writing M n = n − 1 ∑ i h i , Π ( i ) M_n=n^{-1}\sum_i h_{i,\Pi(i)} M n = n − 1 ∑ i h i , Π ( i ) n − 1 / 2 ∑ i q i A i , Π ( i ) = n M n n^{-1/2}\sum_i q_iA_{i,\Pi(i)}=\sqrt{n}M_n n − 1/2 ∑ i q i A i , Π ( i ) = n M n
E [ n M n ∣ F n ] = n h ⋅ ⋅ = 0 , \mathbb{E}[\sqrt{n}M_n\mid\mathcal{F}_n]=\sqrt{n}h_{\cdot\cdot}=0, E [ n M n ∣ F n ] = n h ⋅⋅ = 0 , Var ( n M n ∣ F n ) = n Var ( M n ∣ F n ) ≤ n n 2 ( n − 1 ) ∑ i , k h i k 2 = 1 n ( n − 1 ) ∑ i , k q i 2 A i k 2 . \operatorname{Var}(\sqrt{n}M_n\mid\mathcal{F}_n)=n\operatorname{Var}(M_n\mid\mathcal{F}_n)
\leq\frac{n}{n^{2}(n-1)}\sum_{i,k}h_{ik}^{2}
=\frac{1}{n(n-1)}\sum_{i,k}q_i^{2}A_{ik}^{2}. Var ( n M n ∣ F n ) = n Var ( M n ∣ F n ) ≤ n 2 ( n − 1 ) n i , k ∑ h ik 2 = n ( n − 1 ) 1 i , k ∑ q i 2 A ik 2 . Lemma 14 (Studentizer transfer under mean-square perturbations). Let R n i ≡ U n i + Δ n i R_{ni}\equiv U_{ni}+\Delta_{ni} R ni ≡ U ni + Δ ni V ^ n ( R ) ≡ n − 1 ∑ i R n i 2 − ( n − 1 ∑ i R n i ) 2 \widehat{V}_n(R)\equiv n^{-1}\sum_iR_{ni}^{2}-(n^{-1}\sum_iR_{ni})^{2} V n ( R ) ≡ n − 1 ∑ i R ni 2 − ( n − 1 ∑ i R ni ) 2 τ n 2 = O P ( 1 ) \tau_n^{2}=O_{\mathbb{P}}(1) τ n 2 = O P ( 1 )
1 n ∑ i U n i 2 − τ n 2 = o P ( 1 ) , 1 n ∑ i U n i = O P ( 1 ) , 1 n ∑ i Δ n i 2 = o P ( 1 ) . \frac{1}{n}\sum_iU_{ni}^{2}-\tau_n^{2}=o_{\mathbb{P}}(1),\qquad
\frac{1}{\sqrt{n}}\sum_iU_{ni}=O_{\mathbb{P}}(1),\qquad
\frac{1}{n}\sum_i\Delta_{ni}^{2}=o_{\mathbb{P}}(1). n 1 i ∑ U ni 2 − τ n 2 = o P ( 1 ) , n 1 i ∑ U ni = O P ( 1 ) , n 1 i ∑ Δ ni 2 = o P ( 1 ) . Then
V ^ n ( R ) − τ n 2 = o P ( 1 ) . \widehat{V}_n(R)-\tau_n^{2}=o_{\mathbb{P}}(1). V n ( R ) − τ n 2 = o P ( 1 ) . If, in addition, τ n 2 \tau_n^{2} τ n 2 V ^ n ( R ) / τ n 2 ⟶ P 1 \widehat{V}_n(R)/\tau_n^{2}\overset{\mathbb{P}}{\longrightarrow}1 V n ( R ) / τ n 2 ⟶ P 1
Proof. By Cauchy–Schwarz,
∣ 1 n ∑ i U n i Δ n i ∣ ≤ ( 1 n ∑ i U n i 2 ) 1 / 2 ( 1 n ∑ i Δ n i 2 ) 1 / 2 = o P ( 1 ) . \left|\frac{1}{n}\sum_iU_{ni}\Delta_{ni}\right|
\leq\left(\frac{1}{n}\sum_iU_{ni}^{2}\right)^{1/2}
\left(\frac{1}{n}\sum_i\Delta_{ni}^{2}\right)^{1/2}
=o_{\mathbb{P}}(1). n 1 i ∑ U ni Δ ni ≤ ( n 1 i ∑ U ni 2 ) 1/2 ( n 1 i ∑ Δ ni 2 ) 1/2 = o P ( 1 ) . Hence n − 1 ∑ i R n i 2 = τ n 2 + o P ( 1 ) n^{-1}\sum_iR_{ni}^{2}=\tau_n^{2}+o_{\mathbb{P}}(1) n − 1 ∑ i R ni 2 = τ n 2 + o P ( 1 )
∣ 1 n ∑ i Δ n i ∣ 2 ≤ 1 n ∑ i Δ n i 2 = o P ( 1 ) , 1 n ∑ i U n i = O P ( n − 1 / 2 ) = o P ( 1 ) . \left|\frac{1}{n}\sum_i\Delta_{ni}\right|^{2}\leq\frac{1}{n}\sum_i\Delta_{ni}^{2}=o_{\mathbb{P}}(1),\qquad
\frac{1}{n}\sum_iU_{ni}=O_{\mathbb{P}}(n^{-1/2})=o_{\mathbb{P}}(1). n 1 i ∑ Δ ni 2 ≤ n 1 i ∑ Δ ni 2 = o P ( 1 ) , n 1 i ∑ U ni = O P ( n − 1/2 ) = o P ( 1 ) . Thus ( n − 1 ∑ i R n i ) 2 = o P ( 1 ) (n^{-1}\sum_iR_{ni})^{2}=o_{\mathbb{P}}(1) ( n − 1 ∑ i R ni ) 2 = o P ( 1 )
Auxiliary regularity conditions
The conditions collected in this section are the technical regularity requirements used in the asymptotic analysis of § 3 and § 4 . We first state the condition on the fourth moment of the outcome shared by all three procedures. The remaining assumptions are then grouped according to the section in which the corresponding procedure is introduced.
Shared moment conditions on the outcome
Assumption 6 (Shared moment conditions on the outcome). Assume
C Y , 4 ≡ ess sup z E [ ∣ Y ∣ 4 ∣ Z = z ] < ∞ . C_{Y,4} \equiv\operatorname{ess\,sup}_{z}\mathbb{E}\left[|Y|^{4}\mid Z=z\right] < \infty. C Y , 4 ≡ ess sup z E [ ∣ Y ∣ 4 ∣ Z = z ] < ∞. Regularity conditions for Section 3: the GLM score test
For μ \mu μ ϑ \vartheta ϑ
c ϑ ( μ ) ≡ 1 V ϑ ( μ ) g ′ ( μ ) , W ϑ ( μ ) ≡ 1 V ϑ ( μ ) g ′ ( μ ) 2 , κ ϑ ( μ ) ≡ ∂ μ c ϑ ( μ ) g ′ ( μ ) . c_{\vartheta}(\mu) \equiv\frac{1}{V_{\vartheta}(\mu)g'(\mu)}, \qquad W_{\vartheta}(\mu) \equiv\frac{1}{V_{\vartheta}(\mu)g'(\mu)^2}, \qquad\kappa_{\vartheta}(\mu) \equiv\frac{\partial_{\mu}c_{\vartheta}(\mu)}{g'(\mu)}. c ϑ ( μ ) ≡ V ϑ ( μ ) g ′ ( μ ) 1 , W ϑ ( μ ) ≡ V ϑ ( μ ) g ′ ( μ ) 2 1 , κ ϑ ( μ ) ≡ g ′ ( μ ) ∂ μ c ϑ ( μ ) . Write c ⋆ = c ϑ ⋆ c_{\star}=c_{\vartheta_{\star}} c ⋆ = c ϑ ⋆ W ⋆ = W ϑ ⋆ W_{\star}=W_{\vartheta_{\star}} W ⋆ = W ϑ ⋆ κ ⋆ = κ ϑ ⋆ \kappa_{\star}=\kappa_{\vartheta_{\star}} κ ⋆ = κ ϑ ⋆
Assumption 7 (GLM auxiliary regularity). In addition to Assumption 1 and Assumption 6, assume:
(i) There is a constant C Z < ∞ C_Z<\infty C Z < ∞ such that ∥ Z i ∥ ≤ C Z \lVert Z_i\rVert\le C_Z ∥ Z i ∥ ≤ C Z almost surely, and one coordinate of Z i Z_i Z i is identically one.
(ii) E [ Z i Z i T ] \mathbb{E}[Z_iZ_i^{\mathsf{T}}] E [ Z i Z i T ] is positive definite.
(iii) Let
K μ ≡ { g − 1 ( z T β ) : ∥ z ∥ ≤ C Z } . K_{\mu}\equiv\left\{g^{-1}(z^{\mathsf{T}}\beta):\lVert z\rVert\le C_Z\right\}. K μ ≡ { g − 1 ( z T β ) : ∥ z ∥ ≤ C Z } . There are an open neighbourhood N \mathcal{N} N of K μ K_{\mu} K μ within the mean space and a compact interval Θ 0 \Theta_0 Θ 0 of dispersion values containing ϑ ⋆ \vartheta_{\star} ϑ ⋆ in its interior. The inverse link g − 1 g^{-1} g − 1 is continuously differentiable with bounded derivative on g ( N ) g(\mathcal{N}) g ( N ) and μ ↦ c ϑ ( μ ) \mu\mapsto c_{\vartheta}(\mu) μ ↦ c ϑ ( μ ) is continuously differentiable on N \mathcal{N} N for every ϑ ∈ Θ 0 \vartheta\in\Theta_0 ϑ ∈ Θ 0
There are deterministic constants 0 < v − ≤ v + < ∞ 0<v_{-}\le v_{+}<\infty 0 < v − ≤ v + < ∞ and 0 < g − ≤ g + < ∞ 0<g_{-}\le g_{+}<\infty 0 < g − ≤ g + < ∞ such that, for all ( μ , ϑ ) ∈ N × Θ 0 (\mu,\vartheta)\in\mathcal{N}\times\Theta_0 ( μ , ϑ ) ∈ N × Θ 0
v − ≤ V ϑ ( μ ) ≤ v + , g − ≤ ∣ g ′ ( μ ) ∣ ≤ g + . v_{-}\le V_{\vartheta}(\mu)\le v_{+}, \qquad g_{-}\le\lvert g'(\mu)\rvert\le g_{+}. v − ≤ V ϑ ( μ ) ≤ v + , g − ≤ ∣ g ′ ( μ )∣ ≤ g + . There are also deterministic constants C 0 , L 0 < ∞ C_0, L_0 < \infty C 0 , L 0 < ∞ f ∈ { c , W , κ } f \in\{c,W,\kappa\} f ∈ { c , W , κ } ( μ , ϑ ) , ( μ ′ , ϑ ′ ) ∈ N × Θ 0 (\mu,\vartheta),(\mu',\vartheta') \in\mathcal{N} \times\Theta_0 ( μ , ϑ ) , ( μ ′ , ϑ ′ ) ∈ N × Θ 0
∣ f ϑ ( μ ) ∣ ≤ C 0 , ∣ f ϑ ( μ ) − f ϑ ′ ( μ ′ ) ∣ ≤ L 0 ( ∣ μ − μ ′ ∣ + ∣ ϑ − ϑ ′ ∣ ) . |f_{\vartheta}(\mu)| \le C_0,\qquad|f_{\vartheta}(\mu)-f_{\vartheta'}(\mu')| \le L_0\left(|\mu-\mu'|+|\vartheta-\vartheta'|\right). ∣ f ϑ ( μ ) ∣ ≤ C 0 , ∣ f ϑ ( μ ) − f ϑ ′ ( μ ′ ) ∣ ≤ L 0 ( ∣ μ − μ ′ ∣ + ∣ ϑ − ϑ ′ ∣ ) . (iv) The limit of the profile information and the permutation score variance are positive, respectively:
I γ ≡ E [ R i 2 ] > 0 , E [ S i ] > 0. I_{\gamma} \equiv\mathbb{E}[R_i^2] > 0,\qquad\mathbb{E}[S_i] > 0. I γ ≡ E [ R i 2 ] > 0 , E [ S i ] > 0. (v) The observed oracle score has finite fourth moment:
E [ ( δ i ξ i ) 4 ] < ∞ . \mathbb{E}[(\delta_i\xi_i)^4] < \infty. E [( δ i ξ i ) 4 ] < ∞. Regularity conditions for Section 4: the GCM and PCM tests
Assumption 8 (GCM auxiliary regularity). In addition to Assumption 6, assume:
(i) X \mathbf{X} X
C X , 4 ≡ ess sup z E [ ∣ X ∣ 4 ∣ Z = z ] < ∞ . C_{\mathrm{X},4} \equiv\operatorname{ess\,sup}_{z}\mathbb{E}[|\mathbf{X}|^{4}\mid Z=z] < \infty. C X , 4 ≡ ess sup z E [ ∣ X ∣ 4 ∣ Z = z ] < ∞. (ii) The variance of the observed oracle summand is nondegenerate:
σ G C M 2 ≡ Var ( ( X − μ x ( Z ) ) ( Y − μ y ( Z ) ) ) > 0. \sigma_{\mathrm{GCM}}^{2} \equiv\operatorname{Var}((\mathbf{X}-\mu_x(Z))(\mathbf{Y}-\mu_y(Z))) > 0. σ GCM 2 ≡ Var (( X − μ x ( Z )) ( Y − μ y ( Z ))) > 0. Assumption 9 (PCM auxiliary regularity). Let H n \mathcal{H}_n H n D 2 \mathcal{D}_2 D 2
(i) There exists a deterministic constant c Y > 0 c_Y>0 c Y > 0
Var ( Y ∣ Z = z ) ≥ c Y for almost every z . \operatorname{Var}(Y\mid Z=z)\ge c_Y \qquad\text{for almost every }z. Var ( Y ∣ Z = z ) ≥ c Y for almost every z . (ii) The observed scale of the learned direction satisfies P ( σ f , n 2 = 0 ) → 0 \mathbb{P}(\sigma_{f,n}^{2}=0)\to0 P ( σ f , n 2 = 0 ) → 0
a ≡ f ^ ( X , Z ) − μ f ( Z ) , a\equiv\hat f(\mathbf{X},Z)-\mu_f(Z), a ≡ f ^ ( X , Z ) − μ f ( Z ) , ess sup z E [ a 2 ∣ Z = z , H n ] σ f , n 2 = O P ( 1 ) , E [ a 4 ∣ H n ] σ f , n 4 = O P ( 1 ) . (17) \operatorname{ess\,sup}_{z}\frac{\mathbb{E}[a^{2}\mid Z=z,\mathcal{H}_n]}{\sigma_{f,n}^{2}}=O_{\mathbb{P}}(1),\qquad\frac{\mathbb{E}[a^{4}\mid\mathcal{H}_n]}{\sigma_{f,n}^{4}}=O_{\mathbb{P}}(1). \tag*{(17)} ess sup z σ f , n 2 E [ a 2 ∣ Z = z , H n ] = O P ( 1 ) , σ f , n 4 E [ a 4 ∣ H n ] = O P ( 1 ) . (17) (iii) The scale under the product law satisfies P ( σ f , n , p e r m 2 = 0 ) → 0 \mathbb{P}(\sigma_{f,n,\mathrm{perm}}^{2}=0)\to0 P ( σ f , n , perm 2 = 0 ) → 0 H n \mathcal{H}_n H n F F F x , z x,z x , z
∣ f ^ ( x , z ) − b p e r m ( z ) ∣ ≤ F ( x ) , E [ F ( X ) 4 ∣ H n ] σ f , n , p e r m 4 = O P ( 1 ) . (18) |\hat f(x,z)-b^{\mathrm{perm}}(z)|\le F(x),\qquad\frac{\mathbb{E}[F(\mathbf{X})^{4}\mid\mathcal{H}_n]}{\sigma_{f,n,\mathrm{perm}}^{4}}=O_{\mathbb{P}}(1). \tag*{(18)} ∣ f ^ ( x , z ) − b perm ( z ) ∣ ≤ F ( x ) , σ f , n , perm 4 E [ F ( X ) 4 ∣ H n ] = O P ( 1 ) . (18) Consequences for the PCM permutation array
Proposition 5 (Approximation under the product law for the PCM permutation array). Under Assumptions 6 and 9(iii), let
n ≡ ∣ D 1 ∣ , B i k ≡ f ^ ( X k , Z i ) , B i ⋅ ≡ 1 n ∑ k ∈ D 1 B i k . n\equiv|\mathcal{D}_1|,\qquad B_{ik}\equiv\hat f(X_k,Z_i),\qquad B_{i\cdot}\equiv\frac{1}{n}\sum_{k\in\mathcal{D}_1}B_{ik}. n ≡ ∣ D 1 ∣ , B ik ≡ f ^ ( X k , Z i ) , B i ⋅ ≡ n 1 k ∈ D 1 ∑ B ik . All sums and maxima below are over indices in D 1 \mathcal{D}_1 D 1
1 n 2 σ f , n , p e r m 2 ∑ i , k ∈ D 1 ( B i k − B i ⋅ ) 2 ⟶ P 1 , (19) \frac{1}{n^2\sigma_{f,n,\mathrm{perm}}^2}\sum_{i,k\in\mathcal{D}_1}(B_{ik}-B_{i\cdot})^2 \overset{\mathbb{P}}{\longrightarrow} 1, \tag*{(19)} n 2 σ f , n , perm 2 1 i , k ∈ D 1 ∑ ( B ik − B i ⋅ ) 2 ⟶ P 1 , (19) max i n − 1 ∑ k ( B i k − B i ⋅ ) 2 σ f , n , p e r m 2 = O P ( 1 ) , (20) \frac{\max_i n^{-1}\sum_{k}(B_{ik}-B_{i\cdot})^2}{\sigma_{f,n,\mathrm{perm}}^2}=O_{\mathbb{P}}(1), \tag*{(20)} σ f , n , perm 2 max i n − 1 ∑ k ( B ik − B i ⋅ ) 2 = O P ( 1 ) , (20) max i , k { Y i − μ y ( Z i ) } 2 ( B i k − B i ⋅ ) 2 n σ f , n , p e r m 2 = o P ( 1 ) . (21) \frac{\max_{i,k}\{Y_i-\mu_y(Z_i)\}^2(B_{ik}-B_{i\cdot})^2}{n\sigma_{f,n,\mathrm{perm}}^2}=o_{\mathbb{P}}(1). \tag*{(21)} n σ f , n , perm 2 max i , k { Y i − μ y ( Z i ) } 2 ( B ik − B i ⋅ ) 2 = o P ( 1 ) . (21) Moreover,
1 n σ f , n , p e r m 2 ∑ i ∈ D 1 { B i ⋅ − b p e r m ( Z i ) } 2 = O P ( n − 1 ) , (22) \frac{1}{n\sigma_{f,n,\mathrm{perm}}^2}\sum_{i\in\mathcal{D}_1}\{B_{i\cdot}-b^{\mathrm{perm}}(Z_i)\}^2=O_{\mathbb{P}}(n^{-1}), \tag*{(22)} n σ f , n , perm 2 1 i ∈ D 1 ∑ { B i ⋅ − b perm ( Z i ) } 2 = O P ( n − 1 ) , (22) and if E n , y P C M = o P ( 1 ) E_{n,y}^{\mathrm{PCM}}=o_{\mathbb{P}}(1) E n , y PCM = o P ( 1 )
1 n 2 σ f , n , p e r m 2 ∑ i , k ∈ D 1 { μ y ( Z i ) − μ ^ y ( Z i ) } 2 ( B i k − B i ⋅ ) 2 = o P ( 1 ) . (23) \frac{1}{n^2\sigma_{f,n,\mathrm{perm}}^2}\sum_{i,k\in\mathcal{D}_1}\{\mu_y(Z_i)-\widehat{\mu}_y(Z_i)\}^2(B_{ik}-B_{i\cdot})^2=o_{\mathbb{P}}(1). \tag*{(23)} n 2 σ f , n , perm 2 1 i , k ∈ D 1 ∑ { μ y ( Z i ) − μ y ( Z i ) } 2 ( B ik − B i ⋅ ) 2 = o P ( 1 ) . (23) Proof. Write S n 2 = σ f , n , p e r m 2 S_n^2=\sigma_{f,n,\mathrm{perm}}^2 S n 2 = σ f , n , perm 2 { s n 2 > 0 } \{s_n^2>0\} { s n 2 > 0 }
h ( x , z ) ≡ f ^ ( x , z ) − b p e r m ( z ) , H i k ≡ h ( X k , Z i ) , H ‾ i ≡ 1 n ∑ k H i k . h(x,z)\equiv\widehat{f}(x,z)-b^{\mathrm{perm}}(z),\qquad H_{ik}\equiv h(X_k,Z_i),\qquad\overline{H}_i\equiv\frac{1}{n}\sum_k H_{ik}. h ( x , z ) ≡ f ( x , z ) − b perm ( z ) , H ik ≡ h ( X k , Z i ) , H i ≡ n 1 k ∑ H ik . Given H n \mathcal{H}_n H n
∫ h ( x , z ) d P X ( x ) = 0 , ∬ h ( x , z ) 2 d P X ( x ) d P Z ( z ) = s n 2 . \int h(x,z)\,dP_X(x)=0,\qquad\iint h(x,z)^2\,dP_X(x)\,dP_Z(z)=s_n^2. ∫ h ( x , z ) d P X ( x ) = 0 , ∬ h ( x , z ) 2 d P X ( x ) d P Z ( z ) = s n 2 . Since B i k − B i ⋅ = H i k − H ‾ i B_{ik}-B_{i\cdot}=H_{ik}-\overline{H}_i B ik − B i ⋅ = H ik − H i
1 n 2 ∑ i , k ( B i k − B i ⋅ ) 2 = 1 n 2 ∑ i , k H i k 2 − 1 n ∑ i H ‾ i 2 . (24) \frac{1}{n^2}\sum_{i,k}(B_{ik}-B_{i\cdot})^2=\frac{1}{n^2}\sum_{i,k}H_{ik}^2-\frac{1}{n}\sum_i\overline{H}_i^2. \tag*{(24)} n 2 1 i , k ∑ ( B ik − B i ⋅ ) 2 = n 2 1 i , k ∑ H ik 2 − n 1 i ∑ H i 2 . (24) By (19) and conditional Jensen’s inequality, for r ∈ { 2 , 4 } r\in\{2,4\} r ∈ { 2 , 4 }
E [ F ( X ) r ∣ H n ] s n r ≤ { E [ F ( X ) 4 ∣ H n ] s n 4 } r / 4 = O P ( 1 ) . (25) \frac{\mathbb{E}[F(X)^r\mid\mathcal{H}_n]}{s_n^r}\leq\left\{\frac{\mathbb{E}[F(X)^4\mid\mathcal{H}_n]}{s_n^4}\right\}^{r/4}=O_{\mathbb{P}}(1). \tag*{(25)} s n r E [ F ( X ) r ∣ H n ] ≤ { s n 4 E [ F ( X ) 4 ∣ H n ] } r /4 = O P ( 1 ) . (25) Let
U n ≡ 1 n ( n − 1 ) ∑ i ≠ k H i k 2 . U_n\equiv\frac{1}{n(n-1)}\sum_{i\ne k}H_{ik}^2. U n ≡ n ( n − 1 ) 1 i = k ∑ H ik 2 . For i ≠ k i\ne k i = k X k X_k X k Z i Z_i Z i H n \mathcal{H}_n H n E [ U n ∣ H n ] = s n 2 \mathbb{E}[U_n\mid\mathcal{H}_n]=s_n^2 E [ U n ∣ H n ] = s n 2 H i k 2 H_{ik}^2 H ik 2 H j ℓ 2 H_{j\ell}^2 H j ℓ 2 H n \mathcal{H}_n H n { i , k } ∩ { j , ℓ } = ∅ \{i,k\}\cap\{j,\ell\}=\varnothing { i , k } ∩ { j , ℓ } = ∅ ( i , k ) (i,k) ( i , k ) i ≠ k i\ne k i = k 4 ( n − 1 ) 4(n-1) 4 ( n − 1 ) ( j , ℓ ) (j,\ell) ( j , ℓ ) j ≠ ℓ j\ne\ell j = ℓ ( i , k ) (i,k) ( i , k )
∣ Cov ( H i k 2 , H j ℓ 2 ∣ H n ) ∣ ≤ 2 E [ F ( X ) 4 ∣ H n ] . \left|\operatorname{Cov}(H_{ik}^2,H_{j\ell}^2\mid\mathcal{H}_n)\right|\leq2\mathbb{E}[F(X)^4\mid\mathcal{H}_n]. Cov ( H ik 2 , H j ℓ 2 ∣ H n ) ≤ 2 E [ F ( X ) 4 ∣ H n ] . Consequently,
Var ( U n ∣ H n ) ≤ n ( n − 1 ) 4 ( n − 1 ) 2 E [ F ( X ) 4 ∣ H n ] n 2 ( n − 1 ) 2 = 8 n E [ F ( X ) 4 ∣ H n ] = O P ( s n 4 / n ) . \operatorname{Var}(U_n \mid\mathcal{H}_n) \le\frac{n(n-1)4(n-1)2\mathbb{E}[F(\mathbf{X})^4 \mid\mathcal{H}_n]}{n^2(n-1)^2}
= \frac{8}{n}\mathbb{E}[F(\mathbf{X})^4 \mid\mathcal{H}_n] = O_{\mathrm{P}}(s_n^4/n). Var ( U n ∣ H n ) ≤ n 2 ( n − 1 ) 2 n ( n − 1 ) 4 ( n − 1 ) 2 E [ F ( X ) 4 ∣ H n ] = n 8 E [ F ( X ) 4 ∣ H n ] = O P ( s n 4 / n ) . Thus E [ U n / s n 2 − 1 ∣ H n ] = 0 \mathbb{E}[U_n/s_n^2-1 \mid\mathcal{H}_n]=0 E [ U n / s n 2 − 1 ∣ H n ] = 0 Var ( U n / s n 2 − 1 ∣ H n ) = O P ( n − 1 ) \operatorname{Var}(U_n/s_n^2-1 \mid\mathcal{H}_n)=O_{\mathrm{P}}(n^{-1}) Var ( U n / s n 2 − 1 ∣ H n ) = O P ( n − 1 )
U n / s n 2 ⟶ P 1. U_n/s_n^2 \overset{\mathrm{P}}{\longrightarrow} 1. U n / s n 2 ⟶ P 1. For the diagonal contribution, conditional Markov and the r = 2 r=2 r = 2 (25) give
n − 1 ∑ i F ( X i ) 2 s n 2 = O P ( 1 ) , n − 2 ∑ i H i i 2 s n 2 ≤ 1 n n − 1 ∑ i F ( X i ) 2 s n 2 = O P ( n − 1 ) . \frac{n^{-1}\sum_i F(X_i)^2}{s_n^2}=O_{\mathrm{P}}(1), \qquad\frac{n^{-2}\sum_i H_{ii}^2}{s_n^2}\leq\frac{1}{n}\frac{n^{-1}\sum_i F(X_i)^2}{s_n^2}=O_{\mathrm{P}}(n^{-1}). s n 2 n − 1 ∑ i F ( X i ) 2 = O P ( 1 ) , s n 2 n − 2 ∑ i H ii 2 ≤ n 1 s n 2 n − 1 ∑ i F ( X i ) 2 = O P ( n − 1 ) . For the row means, conditional on ( X i , Z i , H n ) (X_i,Z_i,\mathcal{H}_n) ( X i , Z i , H n ) H i k H_{ik} H ik k ≠ i k\ne i k = i
E [ H i k ∣ X i , Z i , H n ] = ∫ h ( x , Z i ) d P X ( x ) = 0. \mathbb{E}[H_{ik}\mid X_i,Z_i,\mathcal{H}_n]=\int h(x,Z_i)\,dP_X(x)=0. E [ H ik ∣ X i , Z i , H n ] = ∫ h ( x , Z i ) d P X ( x ) = 0. Since H i i H_{ii} H ii H ˉ i 2 \bar{H}_i^2 H ˉ i 2
E [ H ˉ i 2 ∣ H n ] = 1 n 2 E [ H i i 2 ∣ H n ] + 1 n 2 ∑ k ≠ i E [ H i k 2 ∣ H n ] = 1 n 2 E [ H i i 2 ∣ H n ] + n − 1 n 2 s n 2 ≤ 1 n 2 E [ F ( X ) 2 ∣ H n ] + n − 1 n 2 s n 2 . \begin{aligned}
\mathbb{E}[\bar{H}_i^2\mid\mathcal{H}_n]=\frac{1}{n^2}\mathbb{E}[H_{ii}^2\mid\mathcal{H}_n]+\frac{1}{n^2}\sum_{k\ne i}\mathbb{E}[H_{ik}^2\mid\mathcal{H}_n] \\
&=\frac{1}{n^2}\mathbb{E}[H_{ii}^2\mid\mathcal{H}_n]+\frac{n-1}{n^2}s_n^2 \\
&\leq\frac{1}{n^2}\mathbb{E}[F(\mathbf{X})^2\mid\mathcal{H}_n]+\frac{n-1}{n^2}s_n^2.
\end{aligned} E [ H ˉ i 2 ∣ H n ] = n 2 1 E [ H ii 2 ∣ H n ] + n 2 1 k = i ∑ E [ H ik 2 ∣ H n ] = n 2 1 E [ H ii 2 ∣ H n ] + n 2 n − 1 s n 2 ≤ n 2 1 E [ F ( X ) 2 ∣ H n ] + n 2 n − 1 s n 2 . Dividing by s n 2 s_n^2 s n 2 i i i r = 2 r=2 r = 2 (25) , conditional Markov yields
1 n s n 2 ∑ i H ˉ i 2 = O P ( n − 1 ) . \frac{1}{ns_n^2}\sum_i\bar{H}_i^2=O_{\mathrm{P}}(n^{-1}). n s n 2 1 i ∑ H ˉ i 2 = O P ( n − 1 ) . Since H ˉ i = B i ⋅ − b p e r m ( Z i ) \bar{H}_i=B_{i\cdot}-b^{\mathrm{perm}}(Z_i) H ˉ i = B i ⋅ − b perm ( Z i ) (22) . Substituting into (24) gives
∑ i , k ( B i k − B i ⋅ ) 2 n 2 s n 2 = n − 1 n U n s n 2 + ∑ i H i i 2 n 2 s n 2 − ∑ i H ˉ i 2 n s n 2 ⟶ P 1 , \frac{\sum_{i,k}(B_{ik}-B_{i\cdot})^2}{n^2s_n^2}
=\frac{n-1}{n}\frac{U_n}{s_n^2}+\frac{\sum_i H_{ii}^2}{n^2s_n^2}-\frac{\sum_i\bar{H}_i^2}{ns_n^2}
\overset{\mathrm{P}}{\longrightarrow}1, n 2 s n 2 ∑ i , k ( B ik − B i ⋅ ) 2 = n n − 1 s n 2 U n + n 2 s n 2 ∑ i H ii 2 − n s n 2 ∑ i H ˉ i 2 ⟶ P 1 , which proves (19) .
For the row bound,
max i 1 n ∑ k ( B i k − B i ⋅ ) 2 = max i { 1 n ∑ k H i k 2 − H ˉ i 2 } ≤ 1 n ∑ k F ( X k ) 2 = O P ( s n 2 ) , \max_i\frac{1}{n}\sum_k(B_{ik}-B_{i\cdot})^2
=\max_i\left\{\frac{1}{n}\sum_k H_{ik}^2-\bar{H}_i^2\right\}
\leq\frac{1}{n}\sum_k F(X_k)^2=O_{\mathrm{P}}(s_n^2), i max n 1 k ∑ ( B ik − B i ⋅ ) 2 = i max { n 1 k ∑ H ik 2 − H ˉ i 2 } ≤ n 1 k ∑ F ( X k ) 2 = O P ( s n 2 ) , proving (20) . If E n , y P C M = O P ( 1 ) E_{n,y}^{\mathrm{PCM}}=O_{\mathrm{P}}(1) E n , y PCM = O P ( 1 ) e y , i = μ y ( Z i ) − μ ^ y ( Z i ) e_{y,i}=\mu_y(Z_i)-\hat{\mu}_y(Z_i) e y , i = μ y ( Z i ) − μ ^ y ( Z i )
1 n 2 s n 2 ∑ i , k e y , i 2 ( B i k − B i ⋅ ) 2 ≤ E n , y P C M max i n − 1 ∑ k ( B i k − B i ⋅ ) 2 s n 2 = O P ( 1 ) , \frac{1}{n^2s_n^2}\sum_{i,k}e_{y,i}^2(B_{ik}-B_{i\cdot})^2
\leq E_{n,y}^{\mathrm{PCM}}\frac{\max_i n^{-1}\sum_k(B_{ik}-B_{i\cdot})^2}{s_n^2}=O_{\mathrm{P}}(1), n 2 s n 2 1 i , k ∑ e y , i 2 ( B ik − B i ⋅ ) 2 ≤ E n , y PCM s n 2 max i n − 1 ∑ k ( B ik − B i ⋅ ) 2 = O P ( 1 ) , which proves (23) .
Finally, ∣ H i k − H ˉ i ∣ ≤ 2 max k F ( X k ) |H_{ik}-\bar{H}_i| \le2\max_k F(X_k) ∣ H ik − H ˉ i ∣ ≤ 2 max k F ( X k ) b ≡ Y − μ y ( Z ) b \equiv Y-\mu_y(Z) b ≡ Y − μ y ( Z ) b i ≡ Y i − μ y ( Z i ) b_i \equiv Y_i-\mu_y(Z_i) b i ≡ Y i − μ y ( Z i ) E [ b 4 ] < ∞ \mathbb{E}[b^4] < \infty E [ b 4 ] < ∞ ε > 0 \varepsilon> 0 ε > 0
P ( max i ∣ b i ∣ > ε n 1 / 4 ) ≤ n P ( ∣ b ∣ > ε n 1 / 4 ) ≤ ε − 4 E [ b 4 1 { ∣ b ∣ > ε n 1 / 4 } ] ⟶ 0. \mathbb{P}\left(\max_i |b_i| > \varepsilon n^{1/4}\right) \le n\mathbb{P}\left(|b| > \varepsilon n^{1/4}\right)
\le\varepsilon^{-4}\mathbb{E}\left[b^4 1\{|b| > \varepsilon n^{1/4}\}\right] \longrightarrow0. P ( i max ∣ b i ∣ > ε n 1/4 ) ≤ n P ( ∣ b ∣ > ε n 1/4 ) ≤ ε − 4 E [ b 4 1 { ∣ b ∣ > ε n 1/4 } ] ⟶ 0. Thus max i b i 2 = O P ( n 1 / 2 ) \max_i b_i^2 = O_{\mathbb{P}}(n^{1/2}) max i b i 2 = O P ( n 1/2 ) F ( X k ) / s n F(X_k)/s_n F ( X k ) / s n H n \mathcal{H}_n H n q = 4 q=4 q = 4
max k F ( X k ) 2 s n 2 = O P ( n 1 / 2 ) . \frac{\max_k F(X_k)^2}{s_n^2} = O_{\mathbb{P}}(n^{1/2}). s n 2 max k F ( X k ) 2 = O P ( n 1/2 ) . Therefore
max i , k b i 2 ( B i k − B i ⋅ ) 2 n s n 2 ≤ 4 max i b i 2 n 1 / 2 max k F ( X k ) 2 n 1 / 2 s n 2 = O P ( 1 ) , \frac{\max_{i,k} b_i^2(B_{ik}-B_{i\cdot})^2}{n s_n^2}
\le4\frac{\max_i b_i^2}{n^{1/2}}\frac{\max_k F(X_k)^2}{n^{1/2}s_n^2}
=O_{\mathbb{P}}(1), n s n 2 max i , k b i 2 ( B ik − B i ⋅ ) 2 ≤ 4 n 1/2 max i b i 2 n 1/2 s n 2 max k F ( X k ) 2 = O P ( 1 ) , proving (21) .
Proofs of size validity
Finite-sample validity
Proof of Proposition 1
Proof. By hypothesis, the joint distribution of the data is invariant under G \mathcal{G} G π ∈ G \pi\in\mathcal{G} π ∈ G T n ( π ) T_n^{(\pi)} T n ( π ) [16 ] therefore gives
P { T n > Q 1 − α ( T n ( Π ) ∣ F n ) } ≤ α . \mathbb{P}\{T_n > Q_{1-\alpha}(T_n^{(\Pi)} \mid\mathcal{F}_n)\} \le\alpha. P { T n > Q 1 − α ( T n ( Π ) ∣ F n )} ≤ α . For Monte Carlo calibration, take the identity together with B B B Π 1 , … , Π B \Pi_1,\ldots,\Pi_B Π 1 , … , Π B G \mathcal{G} G [16 ] ,
p = 1 + ∑ b = 1 B 1 { T n ( Π b ) ≥ T n } B + 1 p=\frac{1+\sum_{b=1}^{B}1\{T_n^{(\Pi_b)}\ge T_n\}}{B+1} p = B + 1 1 + ∑ b = 1 B 1 { T n ( Π b ) ≥ T n } is super-uniform, so P ( p ≤ α ) ≤ α \mathbb{P}(p\le\alpha)\le\alpha P ( p ≤ α ) ≤ α H 0 M I H_0^{\mathrm{MI}} H 0 MI ( X ( π ) , Y , Z ) = d ( X , Y , Z ) (X^{(\pi)},Y,Z)\overset{d}{=}(X,Y,Z) ( X ( π ) , Y , Z ) = d ( X , Y , Z ) π \pi π
Oracle approximation and calibration
Proof of Theorem 1
Proof. All ratios below are evaluated on the events where their scales are positive; arbitrary fixed conventions on the complementary events do not affect any limit, since their probabilities tend to zero. Define the oracle sums and second moments by
S s , n ≡ 1 n ∑ i U n i , M s , n ≡ 1 n ∑ i U n i 2 , S_{s,n} \equiv\frac{1}{\sqrt{n}}\sum_i U_{ni}, \qquad M_{s,n} \equiv\frac{1}{n}\sum_i U_{ni}^{2}, S s , n ≡ n 1 i ∑ U ni , M s , n ≡ n 1 i ∑ U ni 2 , S p , n ≡ 1 n ∑ i C n , Π ( i ) , M p , n ≡ 1 n ∑ i C n , Π ( i ) 2 . S_{p,n} \equiv\frac{1}{\sqrt{n}}\sum_i C_{n,\Pi(i)}, \qquad M_{p,n} \equiv\frac{1}{n}\sum_i C_{n,\Pi(i)}^{2}. S p , n ≡ n 1 i ∑ C n , Π ( i ) , M p , n ≡ n 1 i ∑ C n , Π ( i ) 2 . Step 1: Observed oracle Let H n ⊆ F n \mathcal{H}_n \subseteq\mathcal{F}_n H n ⊆ F n H n \mathcal{H}_n H n U n 1 , … , U n n U_{n1},\ldots,U_{nn} U n 1 , … , U nn σ s , n 2 \sigma_{s,n}^{2} σ s , n 2
∑ i = 1 n E [ ∣ U n i ∣ 4 ∣ H n ] { ∑ i = 1 n E [ U n i 2 ∣ H n ] } 2 = E [ U n 1 4 ∣ H n ] n σ s , n 4 = o P ( 1 ) . \frac{\sum_{i=1}^{n}\mathbb{E}[|U_{ni}|^{4}\mid\mathcal{H}_n]}{\left\{\sum_{i=1}^{n}\mathbb{E}[U_{ni}^{2}\mid\mathcal{H}_n]\right\}^{2}}=\frac{\mathbb{E}[U_{n1}^{4}\mid\mathcal{H}_n]}{n\sigma_{s,n}^{4}}=o_P(1). { ∑ i = 1 n E [ U ni 2 ∣ H n ] } 2 ∑ i = 1 n E [ ∣ U ni ∣ 4 ∣ H n ] = n σ s , n 4 E [ U n 1 4 ∣ H n ] = o P ( 1 ) . Lemma 6 gives
S s , n σ s , n ∣ H n → d , p N ( 0 , 1 ) , S s , n σ s , n → d N ( 0 , 1 ) , \left.\frac{S_{s,n}}{\sigma_{s,n}}\right|\mathcal{H}_n \xrightarrow{d,p} N(0,1), \qquad\frac{S_{s,n}}{\sigma_{s,n}}\xrightarrow{d}N(0,1), σ s , n S s , n H n d , p N ( 0 , 1 ) , σ s , n S s , n d N ( 0 , 1 ) , by Lemma 4. Moreover,
E [ M s , n σ s , n 2 − 1 ∣ H n ] = 0 , Var ( M s , n σ s , n 2 ∣ H n ) ≤ E [ U n 1 4 ∣ H n ] n σ s , n 4 = o P ( 1 ) . \mathbb{E}\left[\left.\frac{M_{s,n}}{\sigma_{s,n}^{2}}-1\right|\mathcal{H}_n\right]=0,\qquad\operatorname{Var}\left(\left.\frac{M_{s,n}}{\sigma_{s,n}^{2}}\right|\mathcal{H}_n\right)\leq\frac{\mathbb{E}[U_{n1}^{4}\mid\mathcal{H}_n]}{n\sigma_{s,n}^{4}}=o_P(1). E [ σ s , n 2 M s , n − 1 H n ] = 0 , Var ( σ s , n 2 M s , n H n ) ≤ n σ s , n 4 E [ U n 1 4 ∣ H n ] = o P ( 1 ) . Applying Lemma 8(b) to M s , n / σ s , n 2 − 1 M_{s,n}/\sigma_{s,n}^{2}-1 M s , n / σ s , n 2 − 1 G n = H n \mathcal{G}_n=\mathcal{H}_n G n = H n
M s , n σ s , n 2 → p 1. \frac{M_{s,n}}{\sigma_{s,n}^{2}}\xrightarrow{p}1. σ s , n 2 M s , n p 1. Step 2: Permutation oracle Put D i k ≡ C n , i k − C n , ⋅ k D_{ik}\equiv C_{n,ik}-C_{n,\cdot k} D ik ≡ C n , ik − C n , ⋅ k C n , ⋅ k C_{n,\cdot k} C n , ⋅ k ( D i k ) (D_{ik}) ( D ik ) F n \mathcal{F}_n F n
E [ S p , n ∣ F n ] = 0 , \mathbb{E}[S_{p,n}\mid\mathcal{F}_n]=0, E [ S p , n ∣ F n ] = 0 , v p , n 2 ≡ Var ( S p , n ∣ F n ) = 1 n ( n − 1 ) ∑ i , k D i k 2 = n n − 1 σ p , n 2 − 1 n − 1 ∑ k C n , ⋅ k 2 . v_{p,n}^{2}\equiv\operatorname{Var}(S_{p,n}\mid\mathcal{F}_n)=\frac{1}{n(n-1)}\sum_{i,k}D_{ik}^{2}=\frac{n}{n-1}\sigma_{p,n}^{2}-\frac{1}{n-1}\sum_k C_{n,\cdot k}^{2}. v p , n 2 ≡ Var ( S p , n ∣ F n ) = n ( n − 1 ) 1 i , k ∑ D ik 2 = n − 1 n σ p , n 2 − n − 1 1 k ∑ C n , ⋅ k 2 . The assumption on the column mean gives v p , n 2 / σ p , n 2 → p 1 v_{p,n}^{2}/\sigma_{p,n}^{2}\xrightarrow{p}1 v p , n 2 / σ p , n 2 p 1 v p , n 2 > 0 v_{p,n}^{2}>0 v p , n 2 > 0 max i , k D i k 2 ≤ 4 max i , k C n , i k 2 \max_{i,k}D_{ik}^{2}\leq4\max_{i,k}C_{n,ik}^{2} max i , k D ik 2 ≤ 4 max i , k C n , ik 2
max i , k D i k 2 n − 1 ∑ i , k D i k 2 ≤ 4 n n − 1 max i , k C n , i k 2 n σ p , n 2 σ p , n 2 v p , n 2 = o P ( 1 ) . \frac{\max_{i,k}D_{ik}^{2}}{n^{-1}\sum_{i,k}D_{ik}^{2}}\leq\frac{4n}{n-1}\frac{\max_{i,k}C_{n,ik}^{2}}{n\sigma_{p,n}^{2}}\frac{\sigma_{p,n}^{2}}{v_{p,n}^{2}}=o_P(1). n − 1 ∑ i , k D ik 2 max i , k D ik 2 ≤ n − 1 4 n n σ p , n 2 max i , k C n , ik 2 v p , n 2 σ p , n 2 = o P ( 1 ) . Lemma 12 and conditional Slutsky (Lemma 3) consequently give
S p , n σ p , n ∣ F n → d , p N ( 0 , 1 ) . \left.\frac{S_{p,n}}{\sigma_{p,n}}\right|\mathcal{F}_n\xrightarrow{d,p}N(0,1). σ p , n S p , n F n d , p N ( 0 , 1 ) . Apply Lemma 13 to h i k = C n , i k 2 h_{ik}=C_{n,ik}^{2} h ik = C n , ik 2 M p , n = n − 1 ∑ i C n , Π ( i ) 2 M_{p,n}=n^{-1}\sum_i C_{n,\Pi(i)}^{2} M p , n = n − 1 ∑ i C n , Π ( i ) 2 σ p , n 2 = n − 2 ∑ i , k C n , i k 2 \sigma_{p,n}^{2}=n^{-2}\sum_{i,k}C_{n,ik}^{2} σ p , n 2 = n − 2 ∑ i , k C n , ik 2 (15) gives
E [ M p , n / σ p , n 2 ∣ F n ] = 1. \mathbb{E}[M_{p,n}/\sigma_{p,n}^{2}\mid\mathcal{F}_n]=1. E [ M p , n / σ p , n 2 ∣ F n ] = 1. Applying the variance bound in (16) with h i k = C n , i k 2 h_{ik}=C_{n,ik}^{2} h ik = C n , ik 2
Var ( M p , n / σ p , n 2 ∣ F n ) = 1 σ p , n 4 Var ( M p , n ∣ F n ) ≤ ∑ i , k C n , i k 4 n 2 ( n − 1 ) σ p , n 4 ≤ ( max i , k C n , i k 2 ) ∑ i , k C n , i k 2 n 2 ( n − 1 ) σ p , n 4 = n n − 1 max i , k C n , i k 2 n σ p , n 2 = o P ( 1 ) , \begin{aligned}
\operatorname{Var}(M_{p,n}/\sigma_{p,n}^{2}\mid\mathcal{F}_{n})
&=\frac{1}{\sigma_{p,n}^{4}}\operatorname{Var}(M_{p,n}\mid\mathcal{F}_{n})\\
&\leq\frac{\sum_{i,k}C_{n,ik}^{4}}{n^{2}(n-1)\sigma_{p,n}^{4}}\\
&\leq\frac{(\max_{i,k}C_{n,ik}^{2})\sum_{i,k}C_{n,ik}^{2}}{n^{2}(n-1)\sigma_{p,n}^{4}}\\
&=\frac{n}{n-1}\frac{\max_{i,k}C_{n,ik}^{2}}{n\sigma_{p,n}^{2}}=o_{P}(1),
\end{aligned} Var ( M p , n / σ p , n 2 ∣ F n ) = σ p , n 4 1 Var ( M p , n ∣ F n ) ≤ n 2 ( n − 1 ) σ p , n 4 ∑ i , k C n , ik 4 ≤ n 2 ( n − 1 ) σ p , n 4 ( max i , k C n , ik 2 ) ∑ i , k C n , ik 2 = n − 1 n n σ p , n 2 max i , k C n , ik 2 = o P ( 1 ) , where the last equality uses ∑ i , k C n , i k 2 = n 2 σ p , n 2 \sum_{i,k}C_{n,ik}^{2}=n^{2}\sigma_{p,n}^{2} ∑ i , k C n , ik 2 = n 2 σ p , n 2 M p , n / σ p , n 2 − 1 M_{p,n}/\sigma_{p,n}^{2}-1 M p , n / σ p , n 2 − 1 G n = F n \mathcal{G}_{n}=\mathcal{F}_{n} G n = F n
M p , n σ p , n 2 ⟶ P 1. \frac{M_{p,n}}{\sigma_{p,n}^{2}}\mathrel{\mathop{\longrightarrow}^{P}}1. σ p , n 2 M p , n ⟶ P 1. Step 3: Transfer to fitted statistics Write
S ^ s , n = 1 n ∑ i R ^ i , S ^ p , n = 1 n ∑ i R ^ i ( Π ) , V ^ s , n = V ^ n , V ^ p , n = V ^ n ( Π ) . \widehat{S}_{s,n}=\frac{1}{\sqrt{n}}\sum_i\widehat{R}_i,\qquad
\widehat{S}_{p,n}=\frac{1}{\sqrt{n}}\sum_i\widehat{R}^{(\Pi)}_i,\qquad
\widehat{V}_{s,n}=\widehat{V}_n,\qquad
\widehat{V}_{p,n}=\widehat{V}^{(\Pi)}_n. S s , n = n 1 i ∑ R i , S p , n = n 1 i ∑ R i ( Π ) , V s , n = V n , V p , n = V n ( Π ) . Steps 1–2, together with Lemma 4 for the permutation oracle, give, for r = s , p r=s,p r = s , p
M r , n σ r , n 2 ⟶ P 1 , S r , n σ r , n = O P ( 1 ) . (26) \frac{M_{r,n}}{\sigma_{r,n}^{2}}\mathrel{\mathop{\longrightarrow}^{P}}1,\qquad
\frac{S_{r,n}}{\sigma_{r,n}}=O_{P}(1). \tag*{(26)} σ r , n 2 M r , n ⟶ P 1 , σ r , n S r , n = O P ( 1 ) . (26) The second bound in condition (iii) of Theorem 1 gives
1 n ∑ i ( Δ r , n i σ r , n ) 2 = O P ( 1 ) . (27) \frac{1}{n}\sum_i\left(\frac{\Delta_{r,ni}}{\sigma_{r,n}}\right)^2=O_{P}(1). \tag*{(27)} n 1 i ∑ ( σ r , n Δ r , ni ) 2 = O P ( 1 ) . (27) Apply Lemma 14, with τ n 2 = 1 \tau_n^2=1 τ n 2 = 1
( U n i σ s , n , Δ s , n i σ s , n ) and ( C n , i Π ( i ) σ p , n , Δ p , n i σ p , n ) . \left(\frac{U_{ni}}{\sigma_{s,n}},\frac{\Delta_{s,ni}}{\sigma_{s,n}}\right)
\quad\text{and}\quad
\left(\frac{C_{n,i\Pi(i)}}{\sigma_{p,n}},\frac{\Delta_{p,ni}}{\sigma_{p,n}}\right). ( σ s , n U ni , σ s , n Δ s , ni ) and ( σ p , n C n , i Π ( i ) , σ p , n Δ p , ni ) . Equations (26) –(27) then yield
V ^ r , n σ r , n 2 ⟶ P 1 , r = s , p . (28) \frac{\widehat{V}_{r,n}}{\sigma_{r,n}^{2}}\mathrel{\mathop{\longrightarrow}^{P}}1,\qquad r=s,p. \tag*{(28)} σ r , n 2 V r , n ⟶ P 1 , r = s , p . (28) For the observed numerator, the first bound in condition (iii) and Step 1 give
S ^ s , n σ s , n = S s , n σ s , n + S ^ s , n − S s , n σ s , n ⟶ d N ( 0 , 1 ) . \frac{\widehat{S}_{s,n}}{\sigma_{s,n}}
=
\frac{S_{s,n}}{\sigma_{s,n}}
+
\frac{\widehat{S}_{s,n}-S_{s,n}}{\sigma_{s,n}}
\mathrel{\mathop{\longrightarrow}^{d}}N(0,1). σ s , n S s , n = σ s , n S s , n + σ s , n S s , n − S s , n ⟶ d N ( 0 , 1 ) . Together with (28) , ordinary Slutsky gives
T n = S ^ s , n / σ s , n ( V ^ s , n / σ s , n 2 ) 1 / 2 ⟶ d N ( 0 , 1 ) . (29) T_n=\frac{\widehat{S}_{s,n}/\sigma_{s,n}}{(\widehat{V}_{s,n}/\sigma_{s,n}^{2})^{1/2}}
\mathrel{\mathop{\longrightarrow}^{d}}N(0,1). \tag*{(29)} T n = ( V s , n / σ s , n 2 ) 1/2 S s , n / σ s , n ⟶ d N ( 0 , 1 ) . (29) For the permutation numerator, Step 2 and the first bound in condition (iii), followed by conditional Slutsky (Lemma 3), give
S ^ p , n σ p , n ∣ F n → d , p N ( 0 , 1 ) . \left.\frac{\widehat{S}_{p,n}}{\sigma_{p,n}}\right\vert\mathcal{F}_n \xrightarrow{d,p} N(0,1). σ p , n S p , n F n d , p N ( 0 , 1 ) . By (28) , ( V ^ p , n / σ p , n 2 ) − 1 / 2 → p 1 \left(\widehat{V}_{p,n}/\sigma_{p,n}^{2}\right)^{-1/2}\xrightarrow{p}1 ( V p , n / σ p , n 2 ) − 1/2 p 1
T n ( Π ) = S ^ p , n σ p , n ( V ^ p , n σ p , n 2 ) − 1 / 2 ∣ F n → d , p N ( 0 , 1 ) . (30) T_n^{(\Pi)}=\frac{\widehat{S}_{p,n}}{\sigma_{p,n}}\left(\frac{\widehat{V}_{p,n}}{\sigma_{p,n}^{2}}\right)^{-1/2}\mathrel{\Big|}\mathcal{F}_n\xrightarrow{d,p}N(0,1). \tag*{(30)} T n ( Π ) = σ p , n S p , n ( σ p , n 2 V p , n ) − 1/2 F n d , p N ( 0 , 1 ) . (30) Step 4: Calibration Let
q n , 1 − α ≡ Q 1 − α ( T n ( Π ) ∣ F n ) . q_{n,1-\alpha}\equiv Q_{1-\alpha}\left(T_n^{(\Pi)}\mid\mathcal{F}_n\right). q n , 1 − α ≡ Q 1 − α ( T n ( Π ) ∣ F n ) . By (30) and Lemma 5,
q n , 1 − α → p z 1 − α . q_{n,1-\alpha}\xrightarrow{p}z_{1-\alpha}. q n , 1 − α p z 1 − α . Together with (29) , Slutsky’s theorem gives
T n − q n , 1 − α → d N ( − z 1 − α , 1 ) . T_n-q_{n,1-\alpha}\xrightarrow{d}N(-z_{1-\alpha},1). T n − q n , 1 − α d N ( − z 1 − α , 1 ) . Since the limiting distribution is continuous at zero,
P ( T n > q n , 1 − α ) = P ( T n − q n , 1 − α > 0 ) ⟶ 1 − Φ ( z 1 − α ) = α . \mathbb{P}(T_n>q_{n,1-\alpha})=\mathbb{P}(T_n-q_{n,1-\alpha}>0)\longrightarrow1-\Phi(z_{1-\alpha})=\alpha. P ( T n > q n , 1 − α ) = P ( T n − q n , 1 − α > 0 ) ⟶ 1 − Φ ( z 1 − α ) = α . Asymptotic validity
GLM score test
For b b b β \beta β ϑ ∈ Θ 0 \vartheta\in\Theta_0 ϑ ∈ Θ 0
s i ( b ; ϑ ) ≡ ( Y i − μ i ( b ) ) c ϑ ( μ i ( b ) ) , W i ( b ; ϑ ) ≡ W ϑ ( μ i ( b ) ) . s_i(b;\vartheta)\equiv(Y_i-\mu_i(b))c_\vartheta(\mu_i(b)),\qquad W_i(b;\vartheta)\equiv W_\vartheta(\mu_i(b)). s i ( b ; ϑ ) ≡ ( Y i − μ i ( b )) c ϑ ( μ i ( b )) , W i ( b ; ϑ ) ≡ W ϑ ( μ i ( b )) . The oracle quantities satisfy s i ( β ; ϑ ⋆ ) = ξ i s_i(\beta;\vartheta_\star)=\xi_i s i ( β ; ϑ ⋆ ) = ξ i W i ( β ; ϑ ⋆ ) = W i W_i(\beta;\vartheta_\star)=W_i W i ( β ; ϑ ⋆ ) = W i ( β ^ , ϑ ^ ) (\widehat{\beta},\widehat{\vartheta}) ( β , ϑ )
∇ b s i ( b ; ϑ ) = { − W i ( b ; ϑ ) + ( Y i − μ i ( b ) ) κ ϑ ( μ i ( b ) ) } Z i . (31) \nabla_b s_i(b;\vartheta)=\left\{-W_i(b;\vartheta)+(Y_i-\mu_i(b))\kappa_\vartheta(\mu_i(b))\right\}Z_i. \tag*{(31)} ∇ b s i ( b ; ϑ ) = { − W i ( b ; ϑ ) + ( Y i − μ i ( b )) κ ϑ ( μ i ( b )) } Z i . (31) Lemmas 15–19 assume Assumptions 1, 6, and 7.
Lemma 15 (Local plug-in bounds). There are a neighbourhood B B B β \beta β C < ∞ C<\infty C < ∞ b , b ′ ∈ B b,b'\in B b , b ′ ∈ B ϑ , ϑ ′ ∈ Θ 0 \vartheta,\vartheta'\in\Theta_0 ϑ , ϑ ′ ∈ Θ 0 i i i
∣ μ i ( b ) − μ i ( b ′ ) ∣ ≤ C ∥ b − b ′ ∥ , |\mu_i(b)-\mu_i(b')|\le C\lVert b-b'\rVert, ∣ μ i ( b ) − μ i ( b ′ ) ∣ ≤ C ∥ b − b ′ ∥ , ∣ s i ( b ; ϑ ) − s i ( b ′ ; ϑ ′ ) ∣ ≤ C ( 1 + ∣ Y i ∣ ) { ∥ b − b ′ ∥ + ∣ ϑ − ϑ ′ ∣ } , |s_i(b;\vartheta)-s_i(b';\vartheta')|\le C(1+|Y_i|)\{\lVert b-b'\rVert+|\vartheta-\vartheta'|\}, ∣ s i ( b ; ϑ ) − s i ( b ′ ; ϑ ′ ) ∣ ≤ C ( 1 + ∣ Y i ∣ ) {∥ b − b ′ ∥ + ∣ ϑ − ϑ ′ ∣ } , ∣ s i ( b ; ϑ ) m − s i ( b ′ ; ϑ ′ ) m ∣ ≤ C ( 1 + ∣ Y i ∣ m ) { ∥ b − b ′ ∥ + ∣ ϑ − ϑ ′ ∣ } , m ∈ { 2 , 4 } , |s_i(b;\vartheta)^m-s_i(b';\vartheta')^m|\le C(1+|Y_i|^m)\{\lVert b-b'\rVert+|\vartheta-\vartheta'|\},\qquad m\in\{2,4\}, ∣ s i ( b ; ϑ ) m − s i ( b ′ ; ϑ ′ ) m ∣ ≤ C ( 1 + ∣ Y i ∣ m ) {∥ b − b ′ ∥ + ∣ ϑ − ϑ ′ ∣ } , m ∈ { 2 , 4 } , ∣ W i ( b ; ϑ ) m − W i ( b ′ ; ϑ ′ ) m ∣ ≤ C { ∥ b − b ′ ∥ + ∣ ϑ − ϑ ′ ∣ } , m ∈ { 1 , 2 } . |W_i(b;\vartheta)^m-W_i(b';\vartheta')^m|\le C\{\lVert b-b'\rVert+|\vartheta-\vartheta'|\},\qquad m\in\{1,2\}. ∣ W i ( b ; ϑ ) m − W i ( b ′ ; ϑ ′ ) m ∣ ≤ C {∥ b − b ′ ∥ + ∣ ϑ − ϑ ′ ∣ } , m ∈ { 1 , 2 } . The same bound, with envelope C ( 1 + ∣ Y i ∣ ) C(1+|Y_i|) C ( 1 + ∣ Y i ∣ ) ∇ b s i ( b ; ϑ ) \nabla_b s_i(b;\vartheta) ∇ b s i ( b ; ϑ )
Proof. Choose a closed neighbourhood B B B β \beta β
μ i ( b ) = g − 1 ( Z i T b ) ∈ N 0 ⊂ N for all b ∈ B and all i . \mu_i(b) = g^{-1}(Z_i^\mathsf{T}b) \in\mathcal{N}_0 \subset\mathcal{N} \qquad\text{for all } b \in B \text{ and all } i. μ i ( b ) = g − 1 ( Z i T b ) ∈ N 0 ⊂ N for all b ∈ B and all i . This is possible because ∥ Z i ∥ ≤ C Z \lVert Z_i\rVert\le C_Z ∥ Z i ∥ ≤ C Z K μ ⊂ N K_\mu\subset\mathcal{N} K μ ⊂ N C μ < ∞ C_\mu< \infty C μ < ∞ ∣ μ ∣ ≤ C μ |\mu| \le C_\mu ∣ μ ∣ ≤ C μ N 0 \mathcal{N}_0 N 0 f ∈ { c , W , κ } f \in\{c,W,\kappa\} f ∈ { c , W , κ }
∣ f ϑ ( μ ) ∣ ≤ C 0 , ∣ f ϑ ( μ ) − f ϑ ′ ( μ ′ ) ∣ ≤ L 0 { ∣ μ − μ ′ ∣ + ∣ ϑ − ϑ ′ ∣ } . |f_{\vartheta}(\mu)| \le C_0, \qquad|f_{\vartheta}(\mu) - f_{\vartheta'}(\mu')| \le L_0\{|\mu-\mu'| + |\vartheta-\vartheta'|\}. ∣ f ϑ ( μ ) ∣ ≤ C 0 , ∣ f ϑ ( μ ) − f ϑ ′ ( μ ′ ) ∣ ≤ L 0 { ∣ μ − μ ′ ∣ + ∣ ϑ − ϑ ′ ∣ } . Let
Δ ≡ ∥ b − b ′ ∥ + ∣ ϑ − ϑ ′ ∣ , μ ≡ μ i ( b ) , μ ′ ≡ μ i ( b ′ ) . \Delta\equiv\lVert b-b'\rVert+ |\vartheta-\vartheta'|,\qquad\mu\equiv\mu_i(b),\qquad\mu' \equiv\mu_i(b'). Δ ≡ ∥ b − b ′ ∥ + ∣ ϑ − ϑ ′ ∣ , μ ≡ μ i ( b ) , μ ′ ≡ μ i ( b ′ ) . Since ( g − 1 ) ′ (g^{-1})' ( g − 1 ) ′ g ( N 0 ) g(\mathcal{N}_0) g ( N 0 )
∣ μ − μ ′ ∣ ≤ sup η ∈ g ( N 0 ) ∣ ( g − 1 ) ′ ( η ) ∣ ∥ Z i T ( b − b ′ ) ∥ ≤ C ∥ b − b ′ ∥ . (32) |\mu-\mu'| \le\sup_{\eta\in g(\mathcal{N}_0)} |(g^{-1})'(\eta)|\lVert Z_i^\mathsf{T}(b-b')\rVert\le C\lVert b-b'\rVert. \tag*{(32)} ∣ μ − μ ′ ∣ ≤ η ∈ g ( N 0 ) sup ∣ ( g − 1 ) ′ ( η ) ∣ ∥ Z i T ( b − b ′ )∥ ≤ C ∥ b − b ′ ∥ . (32) For the score contribution,
s i ( b ; ϑ ) − s i ( b ′ ; ϑ ′ ) = ( Y i − μ ) { c ϑ ( μ ) − c ϑ ′ ( μ ′ ) } + ( μ ′ − μ ) c ϑ ′ ( μ ′ ) , s_i(b;\vartheta)-s_i(b';\vartheta') = (Y_i-\mu)\{c_{\vartheta}(\mu)-c_{\vartheta'}(\mu')\}+(\mu'-\mu)c_{\vartheta'}(\mu'), s i ( b ; ϑ ) − s i ( b ′ ; ϑ ′ ) = ( Y i − μ ) { c ϑ ( μ ) − c ϑ ′ ( μ ′ )} + ( μ ′ − μ ) c ϑ ′ ( μ ′ ) , so, using (32) ,
∣ s i ( b ; ϑ ) − s i ( b ′ ; ϑ ′ ) ∣ ≤ ( ∣ Y i ∣ + C μ ) L 0 { ∣ μ − μ ′ ∣ + ∣ ϑ − ϑ ′ ∣ } + C 0 ∣ μ − μ ′ ∣ ≤ C ( 1 + ∣ Y i ∣ ) Δ . |s_i(b;\vartheta)-s_i(b';\vartheta')| \le(|Y_i|+C_\mu)L_0\{|\mu-\mu'|+|\vartheta-\vartheta'|\}+C_0|\mu-\mu'|
\le C(1+|Y_i|)\Delta. ∣ s i ( b ; ϑ ) − s i ( b ′ ; ϑ ′ ) ∣ ≤ ( ∣ Y i ∣ + C μ ) L 0 { ∣ μ − μ ′ ∣ + ∣ ϑ − ϑ ′ ∣ } + C 0 ∣ μ − μ ′ ∣ ≤ C ( 1 + ∣ Y i ∣ ) Δ. Also
∣ s i ( b ; ϑ ) ∣ ≤ ( ∣ Y i ∣ + C μ ) C 0 ≤ C ( 1 + ∣ Y i ∣ ) , |s_i(b;\vartheta)| \le(|Y_i|+C_\mu)C_0 \le C(1+|Y_i|), ∣ s i ( b ; ϑ ) ∣ ≤ ( ∣ Y i ∣ + C μ ) C 0 ≤ C ( 1 + ∣ Y i ∣ ) , and the same bound holds at ( b ′ , ϑ ′ ) (b',\vartheta') ( b ′ , ϑ ′ ) m ∈ { 2 , 4 } m\in\{2,4\} m ∈ { 2 , 4 }
∣ s i ( b ; ϑ ) m − s i ( b ′ ; ϑ ′ ) m ∣ ≤ m max { ∣ s i ( b ; ϑ ) ∣ , ∣ s i ( b ′ ; ϑ ′ ) ∣ } m − 1 ∣ s i ( b ; ϑ ) − s i ( b ′ ; ϑ ′ ) ∣ ≤ C ( 1 + ∣ Y i ∣ ) m Δ ≤ C ( 1 + ∣ Y i ∣ m ) Δ . |s_i(b;\vartheta)^m-s_i(b';\vartheta')^m| \le m\max\{|s_i(b;\vartheta)|,|s_i(b';\vartheta')|\}^{m-1}|s_i(b;\vartheta)-s_i(b';\vartheta')|
\le C(1+|Y_i|)^m\Delta\le C(1+|Y_i|^m)\Delta. ∣ s i ( b ; ϑ ) m − s i ( b ′ ; ϑ ′ ) m ∣ ≤ m max { ∣ s i ( b ; ϑ ) ∣ , ∣ s i ( b ′ ; ϑ ′ ) ∣ } m − 1 ∣ s i ( b ; ϑ ) − s i ( b ′ ; ϑ ′ ) ∣ ≤ C ( 1 + ∣ Y i ∣ ) m Δ ≤ C ( 1 + ∣ Y i ∣ m ) Δ. Likewise,
∣ W i ( b ; ϑ ) − W i ( b ′ ; ϑ ′ ) ∣ ≤ L 0 { ∣ μ − μ ′ ∣ + ∣ ϑ − ϑ ′ ∣ } ≤ C Δ , |W_i(b;\vartheta)-W_i(b';\vartheta')| \le L_0\{|\mu-\mu'|+|\vartheta-\vartheta'|\}\le C\Delta, ∣ W i ( b ; ϑ ) − W i ( b ′ ; ϑ ′ ) ∣ ≤ L 0 { ∣ μ − μ ′ ∣ + ∣ ϑ − ϑ ′ ∣ } ≤ C Δ , and hence, for m ∈ { 1 , 2 } m\in\{1,2\} m ∈ { 1 , 2 }
∣ W i ( b ; ϑ ) m − W i ( b ′ ; ϑ ′ ) m ∣ ≤ m C 0 m − 1 ∣ W i ( b ; ϑ ) − W i ( b ′ ; ϑ ′ ) ∣ ≤ C Δ . |W_i(b;\vartheta)^m-W_i(b';\vartheta')^m| \le mC_0^{m-1}|W_i(b;\vartheta)-W_i(b';\vartheta')|
\le C\Delta. ∣ W i ( b ; ϑ ) m − W i ( b ′ ; ϑ ′ ) m ∣ ≤ m C 0 m − 1 ∣ W i ( b ; ϑ ) − W i ( b ′ ; ϑ ′ ) ∣ ≤ C Δ. Finally, by (31) ,
∥ ∇ b s i ( b ; ϑ ) − ∇ b s i ( b ′ ; ϑ ′ ) ∥ ≤ ∥ Z i ∥ ( ∣ W i ( b ; ϑ ) − W i ( b ′ ; ϑ ′ ) ∣ + ∣ ( Y i − μ ) κ ϑ ( μ ) − ( Y i − μ ′ ) κ ϑ ′ ( μ ′ ) ∣ ) . \lVert\nabla_b s_i(b;\vartheta)-\nabla_b s_i(b';\vartheta')\rVert
\le\lVert Z_i\rVert\bigl(|W_i(b;\vartheta)-W_i(b';\vartheta')|+|(Y_i-\mu)\kappa_{\vartheta}(\mu)-(Y_i-\mu')\kappa_{\vartheta'}(\mu')|\bigr). ∥ ∇ b s i ( b ; ϑ ) − ∇ b s i ( b ′ ; ϑ ′ )∥ ≤ ∥ Z i ∥ ( ∣ W i ( b ; ϑ ) − W i ( b ′ ; ϑ ′ ) ∣ + ∣ ( Y i − μ ) κ ϑ ( μ ) − ( Y i − μ ′ ) κ ϑ ′ ( μ ′ ) ∣ ) . The second term satisfies
∣ ( Y i − μ ) κ ϑ ( μ ) − ( Y i − μ ′ ) κ ϑ ′ ( μ ′ ) ∣ ≤ ( ∣ Y i ∣ + C μ ) ∣ κ ϑ ( μ ) − κ ϑ ′ ( μ ′ ) ∣ + C 0 ∣ μ − μ ′ ∣ ≤ C ( 1 + ∣ Y i ∣ ) Δ . |(Y_i-\mu)\kappa_{\vartheta}(\mu)-(Y_i-\mu')\kappa_{\vartheta'}(\mu')|
\le(|Y_i|+C_\mu)|\kappa_{\vartheta}(\mu)-\kappa_{\vartheta'}(\mu')|+C_0|\mu-\mu'|
\le C(1+|Y_i|)\Delta. ∣ ( Y i − μ ) κ ϑ ( μ ) − ( Y i − μ ′ ) κ ϑ ′ ( μ ′ ) ∣ ≤ ( ∣ Y i ∣ + C μ ) ∣ κ ϑ ( μ ) − κ ϑ ′ ( μ ′ ) ∣ + C 0 ∣ μ − μ ′ ∣ ≤ C ( 1 + ∣ Y i ∣ ) Δ. Since ∥ Z i ∥ ≤ C Z \lVert Z_i\rVert\le C_Z ∥ Z i ∥ ≤ C Z
∥ ∇ b s i ( b ; ϑ ) − ∇ b s i ( b ′ ; ϑ ′ ) ∥ ≤ C ( 1 + ∣ Y i ∣ ) Δ , \lVert\nabla_b s_i(b;\vartheta)-\nabla_b s_i(b';\vartheta')\rVert\le C(1+|Y_i|)\Delta, ∥ ∇ b s i ( b ; ϑ ) − ∇ b s i ( b ′ ; ϑ ′ )∥ ≤ C ( 1 + ∣ Y i ∣ ) Δ , which proves all the stated bounds.
Lemma 16 (Empirical GLM quantities).
1 n Z T W ^ Z → p M , ( 1 n Z T W ^ Z ) − 1 → p M − 1 , \frac{1}{n}Z^{\mathsf{T}}\widehat{W}Z \xrightarrow{p} M,
\qquad
\left(\frac{1}{n}Z^{\mathsf{T}}\widehat{W}Z\right)^{-1} \xrightarrow{p} M^{-1}, n 1 Z T W Z p M , ( n 1 Z T W Z ) − 1 p M − 1 , 1 n X T W ^ X → p E [ W i X i 2 ] , 1 n Z T W ^ X → p r , \frac{1}{n}X^{\mathsf{T}}\widehat{W}X \xrightarrow{p} \mathbb{E}[W_iX_i^2],
\qquad
\frac{1}{n}Z^{\mathsf{T}}\widehat{W}X \xrightarrow{p} r, n 1 X T W X p E [ W i X i 2 ] , n 1 Z T W X p r , 1 n ∑ i = 1 n W ^ i → p E [ W i ] , 1 n ∑ i = 1 n W ^ i 2 → p E [ W i 2 ] . \frac{1}{n}\sum_{i=1}^{n}\widehat{W}_i \xrightarrow{p} \mathbb{E}[W_i],
\qquad
\frac{1}{n}\sum_{i=1}^{n}\widehat{W}_i^2 \xrightarrow{p} \mathbb{E}[W_i^2]. n 1 i = 1 ∑ n W i p E [ W i ] , n 1 i = 1 ∑ n W i 2 p E [ W i 2 ] . For each fixed coordinate j j j
1 n ∑ i = 1 n W ^ i 2 Z i j 2 → p E [ W i 2 Z i j 2 ] . \frac{1}{n}\sum_{i=1}^{n}\widehat{W}_i^2 Z_{ij}^2 \xrightarrow{p} \mathbb{E}[W_i^2 Z_{ij}^2]. n 1 i = 1 ∑ n W i 2 Z ij 2 p E [ W i 2 Z ij 2 ] . Proof. By Lemma 15, on an event whose probability tends to one,
∥ 1 n ∑ i Z i Z i T ( W ^ i − W i ) ∥ ≤ C ( ∥ β ^ − β ∥ + ∣ ϑ ^ − ϑ ⋆ ∣ ) 1 n ∑ i ∥ Z i ∥ 2 = o P ( 1 ) . \left\|\frac{1}{n}\sum_i Z_iZ_i^{\mathsf{T}}(\widehat{W}_i-W_i)\right\|
\leq C\left(\|\widehat{\beta}-\beta\|+|\widehat{\vartheta}-\vartheta_\star|\right)\frac{1}{n}\sum_i\|Z_i\|^2=o_P(1). n 1 i ∑ Z i Z i T ( W i − W i ) ≤ C ( ∥ β − β ∥ + ∣ ϑ − ϑ ⋆ ∣ ) n 1 i ∑ ∥ Z i ∥ 2 = o P ( 1 ) . The oracle average n − 1 ∑ i W i Z i Z i T n^{-1}\sum_i W_iZ_iZ_i^{\mathsf{T}} n − 1 ∑ i W i Z i Z i T M M M M M M
P ( Z T W ^ Z is singular ) ≤ P { ∥ n − 1 Z T W ^ Z − M ∥ ≥ λ min ( M ) } ⟶ 0. \mathbb{P}(Z^{\mathsf{T}}\widehat{W}Z\text{ is singular})
\leq\mathbb{P}\left\{\left\|n^{-1}Z^{\mathsf{T}}\widehat{W}Z-M\right\|\geq\lambda_{\min}(M)\right\}\longrightarrow0. P ( Z T W Z is singular ) ≤ P { n − 1 Z T W Z − M ≥ λ m i n ( M ) } ⟶ 0. Similarly,
∣ 1 n ∑ i X i 2 ( W ^ i − W i ) ∣ ≤ C ( ∥ β ^ − β ∥ + ∣ ϑ ^ − ϑ ⋆ ∣ ) 1 n ∑ i X i 2 = o P ( 1 ) , \left|\frac{1}{n}\sum_i X_i^2(\widehat{W}_i-W_i)\right|
\leq C\left(\|\widehat{\beta}-\beta\|+|\widehat{\vartheta}-\vartheta_\star|\right)\frac{1}{n}\sum_i X_i^2=o_P(1), n 1 i ∑ X i 2 ( W i − W i ) ≤ C ( ∥ β − β ∥ + ∣ ϑ − ϑ ⋆ ∣ ) n 1 i ∑ X i 2 = o P ( 1 ) , ∥ 1 n ∑ i Z i X i ( W ^ i − W i ) ∥ ≤ C ( ∥ β ^ − β ∥ + ∣ ϑ ^ − ϑ ⋆ ∣ ) 1 n ∑ i ∥ Z i ∥ ∣ X i ∣ = o P ( 1 ) . \left\|\frac{1}{n}\sum_i Z_iX_i(\widehat{W}_i-W_i)\right\|
\leq C\left(\|\widehat{\beta}-\beta\|+|\widehat{\vartheta}-\vartheta_\star|\right)\frac{1}{n}\sum_i\|Z_i\||X_i|=o_P(1). n 1 i ∑ Z i X i ( W i − W i ) ≤ C ( ∥ β − β ∥ + ∣ ϑ − ϑ ⋆ ∣ ) n 1 i ∑ ∥ Z i ∥∣ X i ∣ = o P ( 1 ) . The corresponding oracle averages converge almost surely to E [ W i X i 2 ] \mathbb{E}[W_iX_i^2] E [ W i X i 2 ] r r r
1 n ∑ i ∣ W ^ i m − W i m ∣ ≤ C ( ∥ β ^ − β ∥ + ∣ ϑ ^ − ϑ ⋆ ∣ ) = o P ( 1 ) , m ∈ { 1 , 2 } , \frac{1}{n}\sum_i|\widehat{W}_i^m-W_i^m|
\leq C\left(\|\widehat{\beta}-\beta\|+|\widehat{\vartheta}-\vartheta_\star|\right)=o_P(1),
\qquad m\in\{1,2\}, n 1 i ∑ ∣ W i m − W i m ∣ ≤ C ( ∥ β − β ∥ + ∣ ϑ − ϑ ⋆ ∣ ) = o P ( 1 ) , m ∈ { 1 , 2 } , and, because Z i j Z_{ij} Z ij
1 n ∑ i Z i j 2 ∣ W ^ i 2 − W i 2 ∣ = o P ( 1 ) . \frac{1}{n}\sum_i Z_{ij}^2|\widehat{W}_i^2-W_i^2|=o_P(1). n 1 i ∑ Z ij 2 ∣ W i 2 − W i 2 ∣ = o P ( 1 ) . The strong law applied to the oracle terms establishes the remaining claims.
Lemma 17 (Dispersion plug-in negligibility).
1 n ∑ i = 1 n ( X i Z i ) { s i ( β ; ϑ ^ ) − s i ( β ; ϑ ⋆ ) } = o P ( 1 ) . \frac{1}{\sqrt{n}}\sum_{i=1}^{n}
\begin{pmatrix}
X_i\\
Z_i
\end{pmatrix}
\{s_i(\beta;\widehat{\vartheta})-s_i(\beta;\vartheta_\star)\}
=o_P(1). n 1 i = 1 ∑ n ( X i Z i ) { s i ( β ; ϑ ) − s i ( β ; ϑ ⋆ )} = o P ( 1 ) . Proof. It is enough to prove the claim for one coordinate. Let q ( X , Z ) q(\mathbf{X},\mathbf{Z}) q ( X , Z ) X \mathbf{X} X Z \mathbf{Z} Z ϑ ∈ Θ 0 \vartheta\in\Theta_0 ϑ ∈ Θ 0
f ϑ ( O ) ≡ q ( X , Z ) { Y − μ β ( Z ) } c ϑ ( μ β ( Z ) ) , O = ( X , Y , Z ) . f_{\vartheta}(O) \equiv q(\mathbf{X},\mathbf{Z})\{Y-\mu_{\beta}(\mathbf{Z})\}c_{\vartheta}(\mu_{\beta}(\mathbf{Z})), \qquad O=(\mathbf{X},Y,\mathbf{Z}). f ϑ ( O ) ≡ q ( X , Z ) { Y − μ β ( Z )} c ϑ ( μ β ( Z )) , O = ( X , Y , Z ) . Let P P P O O O P h = E [ h ( O ) ] Ph=\mathbb{E}[h(O)] P h = E [ h ( O )] (8) , P f ϑ = 0 Pf_{\vartheta}=0 P f ϑ = 0 ϑ ∈ Θ 0 \vartheta\in\Theta_0 ϑ ∈ Θ 0 ( X , Z ) (\mathbf{X},\mathbf{Z}) ( X , Z )
P f ϑ = E [ q ( X , Z ) c ϑ ( μ β ( Z ) ) E [ Y − μ β ( Z ) ∣ X , Z ] ] = 0. Pf_{\vartheta}=\mathbb{E}[q(\mathbf{X},\mathbf{Z})c_{\vartheta}(\mu_{\beta}(\mathbf{Z}))\mathbb{E}[Y-\mu_{\beta}(\mathbf{Z})\mid\mathbf{X},\mathbf{Z}]]=0. P f ϑ = E [ q ( X , Z ) c ϑ ( μ β ( Z )) E [ Y − μ β ( Z ) ∣ X , Z ]] = 0. Assumption 7(iii) yields, for a constant C C C ϑ , ϑ ′ \vartheta,\vartheta' ϑ , ϑ ′
∣ f ϑ ( O ) − f ϑ ′ ( O ) ∣ ≤ L ( O ) ∣ ϑ − ϑ ′ ∣ , L ( O ) ≡ C ∣ q ( X , Z ) ∣ ( 1 + ∣ Y ∣ ) . (33) |f_{\vartheta}(O)-f_{\vartheta'}(O)|\leq L(O)|\vartheta-\vartheta'|,\qquad L(O)\equiv C|q(\mathbf{X},\mathbf{Z})|(1+|Y|). \tag*{(33)} ∣ f ϑ ( O ) − f ϑ ′ ( O ) ∣ ≤ L ( O ) ∣ ϑ − ϑ ′ ∣ , L ( O ) ≡ C ∣ q ( X , Z ) ∣ ( 1 + ∣ Y ∣ ) . (33) The envelope is square integrable. This is immediate when q q q Z \mathbf{Z} Z q = X q=\mathbf{X} q = X
E [ X 2 Y 2 ] ≤ { E [ X 4 ] E [ Y 4 ] } 1 / 2 < ∞ , \mathbb{E}[\mathbf{X}^{2}Y^{2}]\leq\{\mathbb{E}[\mathbf{X}^{4}]\mathbb{E}[Y^{4}]\}^{1/2}<\infty, E [ X 2 Y 2 ] ≤ { E [ X 4 ] E [ Y 4 ] } 1/2 < ∞ , and hence P L 2 < ∞ PL^{2}<\infty P L 2 < ∞
Since Θ 0 \Theta_0 Θ 0 R \mathbb{R} R (33) and P L 2 < ∞ PL^{2}<\infty P L 2 < ∞ F ≡ { f ϑ : ϑ ∈ Θ 0 } \mathcal{F}\equiv\{f_{\vartheta}:\vartheta\in\Theta_0\} F ≡ { f ϑ : ϑ ∈ Θ 0 } P P P [33 ] . Moreover,
P ( f ϑ ^ − f ϑ ⋆ ) 2 ≤ P L 2 ∣ ϑ ^ − ϑ ⋆ ∣ 2 = o P ( 1 ) , P(f_{\widehat{\vartheta}}-f_{\vartheta_\star})^{2}\leq PL^{2}|\widehat{\vartheta}-\vartheta_\star|^{2}=o_{P}(1), P ( f ϑ − f ϑ ⋆ ) 2 ≤ P L 2 ∣ ϑ − ϑ ⋆ ∣ 2 = o P ( 1 ) , by consistency of ϑ ^ \widehat{\vartheta} ϑ [33 ] therefore gives
G n ( f ϑ ^ − f ϑ ⋆ ) = o P ( 1 ) , G n h ≡ 1 n ∑ i { h ( X i , Y i , Z i ) − P h } . \mathbb{G}_{n}(f_{\widehat{\vartheta}}-f_{\vartheta_\star})=o_{P}(1),\qquad\mathbb{G}_{n}h\equiv\frac{1}{\sqrt{n}}\sum_i\{h(X_i,Y_i,Z_i)-Ph\}. G n ( f ϑ − f ϑ ⋆ ) = o P ( 1 ) , G n h ≡ n 1 i ∑ { h ( X i , Y i , Z i ) − P h } . Since P f ϑ = 0 Pf_{\vartheta}=0 P f ϑ = 0 ϑ ∈ Θ 0 \vartheta\in\Theta_0 ϑ ∈ Θ 0
1 n ∑ i = 1 n q ( X i , Z i ) { s i ( β ; ϑ ^ ) − s i ( β ; ϑ ⋆ ) } = o P ( 1 ) . \frac{1}{\sqrt{n}}\sum_{i=1}^{n}q(X_i,Z_i)\{s_i(\beta;\widehat{\vartheta})-s_i(\beta;\vartheta_\star)\}=o_{P}(1). n 1 i = 1 ∑ n q ( X i , Z i ) { s i ( β ; ϑ ) − s i ( β ; ϑ ⋆ )} = o P ( 1 ) . Applying the same argument to q = X q=\mathbf{X} q = X Z \mathbf{Z} Z □ \square □
Lemma 18 (Asymptotic linearity of the null estimator).
n ( β ^ − β ) = M − 1 1 n ∑ i = 1 n Z i ξ i + o P ( 1 ) . \sqrt{n}(\widehat{\beta}-\beta)=M^{-1}\frac{1}{\sqrt{n}}\sum_{i=1}^{n}Z_i\xi_i+o_{P}(1). n ( β − β ) = M − 1 n 1 i = 1 ∑ n Z i ξ i + o P ( 1 ) . Proof. Set J n ( b ; ϑ ) ≡ n − 1 ∑ i Z i ∇ b s i ( b ; ϑ ) ⊤ J_n(b;\vartheta)\equiv n^{-1}\sum_i Z_i\nabla_b s_i(b;\vartheta)^\top J n ( b ; ϑ ) ≡ n − 1 ∑ i Z i ∇ b s i ( b ; ϑ ) ⊤
0 = U n ( β ^ ; ϑ ^ ) = U n ( β ; ϑ ^ ) + J ‾ n ( β ^ − β ) , J ‾ n ≡ ∫ 0 1 J n ( β + t ( β ^ − β ) ; ϑ ^ ) d t . 0=U_n(\widehat{\beta};\widehat{\vartheta})=U_n(\beta;\widehat{\vartheta})+\overline{J}_n(\widehat{\beta}-\beta),\qquad\overline{J}_n\equiv\int_0^1J_n(\beta+t(\widehat{\beta}-\beta);\widehat{\vartheta})\,dt. 0 = U n ( β ; ϑ ) = U n ( β ; ϑ ) + J n ( β − β ) , J n ≡ ∫ 0 1 J n ( β + t ( β − β ) ; ϑ ) d t . Lemma 17 gives
n U n ( β ; ϑ ^ ) = 1 n ∑ i Z i ξ i + o P ( 1 ) . \sqrt{n}U_n(\beta;\widehat{\vartheta})=\frac{1}{\sqrt{n}}\sum_i Z_i\xi_i+o_{P}(1). n U n ( β ; ϑ ) = n 1 i ∑ Z i ξ i + o P ( 1 ) . By (31) ,
J n ( β ; ϑ ⋆ ) = − 1 n ∑ i W i Z i Z i T + 1 n ∑ i ( Y i − μ i ) κ ⋆ ( μ i ) Z i Z i T ⟶ p − M . J_n(\beta;\vartheta_\star)=-\frac{1}{n}\sum_i W_iZ_iZ_i^{\mathsf T}+\frac{1}{n}\sum_i(Y_i-\mu_i)\kappa_\star(\mu_i)Z_iZ_i^{\mathsf T}\overset{p}{\longrightarrow}-M. J n ( β ; ϑ ⋆ ) = − n 1 i ∑ W i Z i Z i T + n 1 i ∑ ( Y i − μ i ) κ ⋆ ( μ i ) Z i Z i T ⟶ p − M . Here the second average has mean zero because E [ Y i − μ i ∣ Z i ] = 0 \mathbb{E}[Y_i-\mu_i\mid Z_i]=0 E [ Y i − μ i ∣ Z i ] = 0 ∥ Z i ∥ ≤ C Z \lVert Z_i\rVert\le C_Z ∥ Z i ∥ ≤ C Z
∥ J ‾ n − J n ( β ; ϑ ⋆ ) ∥ ≤ ∫ 0 1 1 n ∑ i ∥ Z i ∥ ∥ ∇ b s i ( β + t ( β ^ − β ) ; ϑ ^ ) − ∇ b s i ( β ; ϑ ⋆ ) ∥ d t ≤ C C Z { ∥ β ^ − β ∥ + ∣ ϑ ^ − ϑ ⋆ ∣ } 1 n ∑ i ( 1 + ∣ Y i ∣ ) = o P ( 1 ) . \begin{aligned}
\lVert\overline{J}_n-J_n(\beta;\vartheta_\star)\rVert\le\int_0^1\frac{1}{n}\sum_i\lVert Z_i\rVert\left\lVert\nabla_b s_i(\beta+t(\widehat{\beta}-\beta);\widehat{\vartheta})-\nabla_b s_i(\beta;\vartheta_\star)\right\rVert\,dt \\
&\le CC_Z\{\lVert\widehat{\beta}-\beta\rVert+\lvert\widehat{\vartheta}-\vartheta_\star\rvert\}\frac{1}{n}\sum_i(1+\lvert Y_i\rvert)=o_{\mathbb{P}}(1).
\end{aligned} ∥ J n − J n ( β ; ϑ ⋆ )∥ ≤ ∫ 0 1 n 1 i ∑ ∥ Z i ∥ ∇ b s i ( β + t ( β − β ) ; ϑ ) − ∇ b s i ( β ; ϑ ⋆ ) d t ≤ C C Z {∥ β − β ∥ + ∣ ϑ − ϑ ⋆ ∣} n 1 i ∑ ( 1 + ∣ Y i ∣) = o P ( 1 ) . Thus J ‾ n ⟶ p − M \overline{J}_n\overset{p}{\longrightarrow}-M J n ⟶ p − M M M M J ‾ n \overline{J}_n J n J ‾ n − 1 ⟶ p − M − 1 \overline{J}_n^{-1}\overset{p}{\longrightarrow}-M^{-1} J n − 1 ⟶ p − M − 1
The vectors Z i ξ i Z_i\xi_i Z i ξ i
E ∥ 1 n ∑ i Z i ξ i ∥ 2 = E ∥ Z i ξ i ∥ 2 ≤ C Z 2 C ψ 2 v ‾ Y < ∞ . \mathbb{E}\left\lVert\frac{1}{\sqrt{n}}\sum_i Z_i\xi_i\right\rVert^2=\mathbb{E}\lVert Z_i\xi_i\rVert^2\le C_Z^2C_\psi^2\overline{v}_Y<\infty. E n 1 i ∑ Z i ξ i 2 = E ∥ Z i ξ i ∥ 2 ≤ C Z 2 C ψ 2 v Y < ∞. Hence n − 1 / 2 ∑ i Z i ξ i = O P ( 1 ) n^{-1/2}\sum_i Z_i\xi_i=O_{\mathbb{P}}(1) n − 1/2 ∑ i Z i ξ i = O P ( 1 )
n ( β ^ − β ) = − J ‾ n − 1 { 1 n ∑ i Z i ξ i + o P ( 1 ) } = M − 1 1 n ∑ i Z i ξ i + o P ( 1 ) . \begin{aligned}
\sqrt{n}(\widehat{\beta}-\beta)=-\overline{J}_n^{-1}\left\{\frac{1}{\sqrt{n}}\sum_i Z_i\xi_i+o_{\mathbb{P}}(1)\right\} \\
&=M^{-1}\frac{1}{\sqrt{n}}\sum_i Z_i\xi_i+o_{\mathbb{P}}(1).
\end{aligned} n ( β − β ) = − J n − 1 { n 1 i ∑ Z i ξ i + o P ( 1 ) } = M − 1 n 1 i ∑ Z i ξ i + o P ( 1 ) . Lemma 19 (Moments of the fitted working residuals). For m ∈ { 2 , 4 } m\in\{2,4\} m ∈ { 2 , 4 }
1 n ∑ i = 1 n ϵ ^ i m ⟶ E [ ξ i m ] a l m o s t s u r e l y . \frac{1}{n}\sum_{i=1}^n\widehat{\epsilon}_i^m\longrightarrow\mathbb{E}[\xi_i^m]\qquad almost\ surely. n 1 i = 1 ∑ n ϵ i m ⟶ E [ ξ i m ] a l m os t s u r e l y . In particular,
lim inf n 1 n ∑ i ϵ ^ i 2 > 0 , lim sup n 1 n ∑ i ϵ ^ i 4 < ∞ a l m o s t s u r e l y . \liminf_n\frac{1}{n}\sum_i\widehat{\epsilon}_i^2>0,\qquad\limsup_n\frac{1}{n}\sum_i\widehat{\epsilon}_i^4<\infty\qquad almost\ surely. n lim inf n 1 i ∑ ϵ i 2 > 0 , n lim sup n 1 i ∑ ϵ i 4 < ∞ a l m os t s u r e l y . Proof. By Lemma 15 and the almost-sure consistency of ( β ^ , ϑ ^ ) (\widehat{\beta},\widehat{\vartheta}) ( β , ϑ ) n n n
∣ 1 n ∑ i ( ϵ ^ i m − ξ i m ) ∣ ≤ C ( ∥ β ^ − β ∥ + ∣ ϑ ^ − ϑ ⋆ ∣ ) 1 n ∑ i ( 1 + ∣ Y i ∣ m ) , m ∈ { 2 , 4 } . \left\lvert\frac{1}{n}\sum_i(\widehat{\epsilon}_i^m-\xi_i^m)\right\rvert\le C(\lVert\widehat{\beta}-\beta\rVert+\lvert\widehat{\vartheta}-\vartheta_\star\rvert)\frac{1}{n}\sum_i(1+\lvert Y_i\rvert^m),\qquad m\in\{2,4\}. n 1 i ∑ ( ϵ i m − ξ i m ) ≤ C (∥ β − β ∥ + ∣ ϑ − ϑ ⋆ ∣) n 1 i ∑ ( 1 + ∣ Y i ∣ m ) , m ∈ { 2 , 4 } . The first factor tends to zero almost surely and the second converges almost surely to a finite constant. The strong law also gives n − 1 ∑ i ξ i m → E [ ξ i m ] n^{-1}\sum_i\xi_i^m\to\mathbb{E}[\xi_i^m] n − 1 ∑ i ξ i m → E [ ξ i m ] E [ ξ i 2 ] = E [ S i ] > 0 \mathbb{E}[\xi_i^2]=\mathbb{E}[S_i]>0 E [ ξ i 2 ] = E [ S i ] > 0 E [ ξ i 4 ] < ∞ \mathbb{E}[\xi_i^4]<\infty E [ ξ i 4 ] < ∞
Proof of Proposition 3
Proof. Because U n ( β ^ ; ϑ ^ ) = 0 U_n(\hat{\beta};\hat{\vartheta})=0 U n ( β ^ ; ϑ ^ ) = 0 ∑ i Z i ϵ ^ i = 0 \sum_i Z_i\hat{\epsilon}_i=0 ∑ i Z i ϵ ^ i = 0 ∑ i ν ^ i G L M ϵ ^ i = ∑ i X i ϵ ^ i \sum_i \hat{\nu}^{\mathrm{GLM}}_i\hat{\epsilon}_i=\sum_i X_i\hat{\epsilon}_i ∑ i ν ^ i GLM ϵ ^ i = ∑ i X i ϵ ^ i N n ( b ; ϑ ) = n − 1 / 2 ∑ i X i s i ( b ; ϑ ) N_n(b;\vartheta)=n^{-1/2}\sum_i X_i s_i(b;\vartheta) N n ( b ; ϑ ) = n − 1/2 ∑ i X i s i ( b ; ϑ ) (9) is N n ( β ^ ; ϑ ^ ) N_n(\hat{\beta};\hat{\vartheta}) N n ( β ^ ; ϑ ^ )
Step 1: Expansion of the numerator Define K n ( b ; ϑ ) ≡ n − 1 ∑ i X i ∇ b s i ( b ; ϑ ) ⊤ K_n(b;\vartheta)\equiv n^{-1}\sum_i X_i\nabla_b s_i(b;\vartheta)^\top K n ( b ; ϑ ) ≡ n − 1 ∑ i X i ∇ b s i ( b ; ϑ ) ⊤
N n ( β ^ ; ϑ ^ ) = N n ( β ; ϑ ^ ) + K ˉ n n ( β ^ − β ) , K ˉ n = ∫ 0 1 K n ( β + t ( β ^ − β ) ; ϑ ^ ) d t . N_n(\hat{\beta};\hat{\vartheta})=N_n(\beta;\hat{\vartheta})+\bar{K}_n\sqrt{n}(\hat{\beta}-\beta),\qquad\bar{K}_n=\int_0^1K_n(\beta+t(\hat{\beta}-\beta);\hat{\vartheta})\,dt. N n ( β ^ ; ϑ ^ ) = N n ( β ; ϑ ^ ) + K ˉ n n ( β ^ − β ) , K ˉ n = ∫ 0 1 K n ( β + t ( β ^ − β ) ; ϑ ^ ) d t . Lemma 17 implies
N n ( β ; ϑ ^ ) = 1 n ∑ i X i ξ i + o P ( 1 ) . N_n(\beta;\hat{\vartheta})=\frac{1}{\sqrt{n}}\sum_i X_i\xi_i+o_{\mathrm{P}}(1). N n ( β ; ϑ ^ ) = n 1 i ∑ X i ξ i + o P ( 1 ) . At ( b , ϑ ) = ( β , ϑ ⋆ ) (b,\vartheta)=(\beta,\vartheta_\star) ( b , ϑ ) = ( β , ϑ ⋆ ) (31) gives
K n ( β ; ϑ ⋆ ) = − 1 n ∑ i X i W i Z i ⊤ + 1 n ∑ i X i ( Y i − μ i ) κ ⋆ ( μ i ) Z i ⊤ . K_n(\beta;\vartheta_\star)=-\frac{1}{n}\sum_iX_iW_iZ_i^\top+\frac{1}{n}\sum_iX_i(Y_i-\mu_i)\kappa_\star(\mu_i)Z_i^\top. K n ( β ; ϑ ⋆ ) = − n 1 i ∑ X i W i Z i ⊤ + n 1 i ∑ X i ( Y i − μ i ) κ ⋆ ( μ i ) Z i ⊤ . The first average converges to − r ⊤ -r^\top − r ⊤ (8) gives
E [ X i ( Y i − μ i ) ∣ Z i ] = E [ X i E [ Y i − μ i ∣ X i , Z i ] ∣ Z i ] = 0. \mathbb{E}[X_i(Y_i-\mu_i)\mid Z_i]=\mathbb{E}[X_i\mathbb{E}[Y_i-\mu_i\mid X_i,Z_i]\mid Z_i]=0. E [ X i ( Y i − μ i ) ∣ Z i ] = E [ X i E [ Y i − μ i ∣ X i , Z i ] ∣ Z i ] = 0. Its summands are integrable, so the strong law sends it to zero. Furthermore, by Lemma 15,
sup 0 ≤ t ≤ 1 ∥ K n ( β + t ( β ^ − β ) ; ϑ ^ ) − K n ( β ; ϑ ⋆ ) ∥ ≤ C ( ∥ β ^ − β ∥ + ∣ ϑ ^ − ϑ ⋆ ∣ ) 1 n ∑ i ∣ X i ∣ ( 1 + ∣ Y i ∣ ) = o P ( 1 ) , \sup_{0\leq t\leq1}\left\|K_n(\beta+t(\hat{\beta}-\beta);\hat{\vartheta})-K_n(\beta;\vartheta_\star)\right\|
\leq C(\|\hat{\beta}-\beta\|+|\hat{\vartheta}-\vartheta_\star|)\frac{1}{n}\sum_i|X_i|(1+|Y_i|)=o_{\mathrm{P}}(1), 0 ≤ t ≤ 1 sup K n ( β + t ( β ^ − β ) ; ϑ ^ ) − K n ( β ; ϑ ⋆ ) ≤ C ( ∥ β ^ − β ∥ + ∣ ϑ ^ − ϑ ⋆ ∣ ) n 1 i ∑ ∣ X i ∣ ( 1 + ∣ Y i ∣ ) = o P ( 1 ) , where the sample average is O P ( 1 ) O_{\mathrm{P}}(1) O P ( 1 ) K ˉ n → p − r ⊤ \bar{K}_n\xrightarrow{\mathrm{p}}-r^\top K ˉ n p − r ⊤ Lemma 18 now yields
N n ( β ^ ; ϑ ^ ) = 1 n ∑ i X i ξ i − r ⊤ M − 1 1 n ∑ i Z i ξ i + o P ( 1 ) = 1 n ∑ i ( X i − θ ⊤ Z i ) ξ i + o P ( 1 ) . (34) \begin{aligned}
N_n(\hat{\beta};\hat{\vartheta})=\frac{1}{\sqrt{n}}\sum_iX_i\xi_i-r^\top M^{-1}\frac{1}{\sqrt{n}}\sum_iZ_i\xi_i+o_{\mathrm{P}}(1) \\
&=\frac{1}{\sqrt{n}}\sum_i(X_i-\theta^\top Z_i)\xi_i+o_{\mathrm{P}}(1). \tag*{(34)}
\end{aligned} N n ( β ^ ; ϑ ^ ) = n 1 i ∑ X i ξ i − r ⊤ M − 1 n 1 i ∑ Z i ξ i + o P ( 1 ) = n 1 i ∑ ( X i − θ ⊤ Z i ) ξ i + o P ( 1 ) . (34) Step 2: Limit of the oracle numerator Let U n i = δ i ξ i U_{ni}=\delta_i\xi_i U ni = δ i ξ i δ i \delta_i δ i § 3 . Under the null, E [ ξ i ∣ X i , Z i ] = E [ Y i − μ i ∣ X i , Z i ] / ( v i g i ′ ) = 0 \mathbb{E}[\xi_i\mid X_i,Z_i]=\mathbb{E}[Y_i-\mu_i\mid X_i,Z_i]/(v_i g_i')=0 E [ ξ i ∣ X i , Z i ] = E [ Y i − μ i ∣ X i , Z i ] / ( v i g i ′ ) = 0 E [ U n i ] = 0 \mathbb{E}[U_{ni}]=0 E [ U ni ] = 0
E [ U n i 2 ] = E [ δ i 2 ξ i 2 ] = τ s 2 < ∞ \mathbb{E}[U_{ni}^2]=\mathbb{E}[\delta_i^2\xi_i^2]=\tau_s^2<\infty E [ U ni 2 ] = E [ δ i 2 ξ i 2 ] = τ s 2 < ∞ by Remark 16. Therefore the i.i.d. central limit theorem and (34) give
N n ( β ^ ; ϑ ^ ) → d N ( 0 , τ s 2 ) . N_n(\hat{\beta};\hat{\vartheta})\xrightarrow{d}N(0,\tau_s^2). N n ( β ^ ; ϑ ^ ) d N ( 0 , τ s 2 ) . Step 3: Limit of the profile studentizer The fitted weights in V ~ n G L M \widetilde{V}_n^{\mathrm{GLM}} V n GLM W ^ i = W i ( β ^ ; ϑ ^ ) \widehat{W}_i = W_i(\widehat{\beta};\widehat{\vartheta}) W i = W i ( β ; ϑ )
1 n X T W ^ X ⟶ p E [ W i X i 2 ] , 1 n Z T W ^ X ⟶ p r , 1 n Z T W ^ Z ⟶ p M . \frac{1}{n}X^{\mathsf{T}}\widehat{W}X \mathrel{\overset{p}{\longrightarrow}} \mathbb{E}[W_iX_i^2], \qquad\frac{1}{n}Z^{\mathsf{T}}\widehat{W}X \mathrel{\overset{p}{\longrightarrow}} r, \qquad\frac{1}{n}Z^{\mathsf{T}}\widehat{W}Z \mathrel{\overset{p}{\longrightarrow}} M. n 1 X T W X ⟶ p E [ W i X i 2 ] , n 1 Z T W X ⟶ p r , n 1 Z T W Z ⟶ p M . Since M M M
V ~ n G L M = 1 n X T W ^ X − ( 1 n X T W ^ Z ) ( 1 n Z T W ^ Z ) − 1 ( 1 n Z T W ^ X ) \widetilde{V}_n^{\mathrm{GLM}} = \frac{1}{n}X^{\mathsf{T}}\widehat{W}X - \left(\frac{1}{n}X^{\mathsf{T}}\widehat{W}Z\right)\left(\frac{1}{n}Z^{\mathsf{T}}\widehat{W}Z\right)^{-1}\left(\frac{1}{n}Z^{\mathsf{T}}\widehat{W}X\right) V n GLM = n 1 X T W X − ( n 1 X T W Z ) ( n 1 Z T W Z ) − 1 ( n 1 Z T W X ) ⟶ p E [ W i X i 2 ] − r T M − 1 r . \mathrel{\overset{p}{\longrightarrow}} \mathbb{E}[W_iX_i^2] - r^{\mathsf{T}}M^{-1}r. ⟶ p E [ W i X i 2 ] − r T M − 1 r . Since θ = M − 1 r \theta= M^{-1}r θ = M − 1 r
E [ W i ( X i − θ T Z i ) 2 ] = E [ W i X i 2 ] − 2 θ T r + θ T M θ = E [ W i X i 2 ] − r T M − 1 r = E [ R i 2 ] . \begin{aligned}
\mathbb{E}[W_i(X_i-\theta^{\mathsf{T}}Z_i)^2] = \mathbb{E}[W_iX_i^2] - 2\theta^{\mathsf{T}}r + \theta^{\mathsf{T}}M\theta\\
&= \mathbb{E}[W_iX_i^2] - r^{\mathsf{T}}M^{-1}r = \mathbb{E}[R_i^2].
\end{aligned} E [ W i ( X i − θ T Z i ) 2 ] = E [ W i X i 2 ] − 2 θ T r + θ T M θ = E [ W i X i 2 ] − r T M − 1 r = E [ R i 2 ] . Thus V ~ n G L M ⟶ p E [ R i 2 ] > 0 \widetilde{V}_n^{\mathrm{GLM}} \mathrel{\overset{p}{\longrightarrow}} \mathbb{E}[R_i^2] > 0 V n GLM ⟶ p E [ R i 2 ] > 0
T ~ n G L M ⟶ d N ( 0 , τ s 2 E [ R i 2 ] ) . \widetilde{T}_n^{\mathrm{GLM}} \mathrel{\overset{d}{\longrightarrow}} N\left(0,\frac{\tau_s^2}{\mathbb{E}[R_i^2]}\right). T n GLM ⟶ d N ( 0 , E [ R i 2 ] τ s 2 ) . When Var ( Y i ∣ X i , Z i ) = v i \operatorname{Var}(Y_i\mid X_i,Z_i)=v_i Var ( Y i ∣ X i , Z i ) = v i E [ ε i 2 ∣ X i , Z i ] = W i \mathbb{E}[\varepsilon_i^2\mid X_i,Z_i]=W_i E [ ε i 2 ∣ X i , Z i ] = W i τ s 2 = E [ W i δ i 2 ] = E [ R i 2 ] \tau_s^2=\mathbb{E}[W_i\delta_i^2]=\mathbb{E}[R_i^2] τ s 2 = E [ W i δ i 2 ] = E [ R i 2 ]
Proof of Proposition 2
Proof. Write Δ s , n i = R ^ i G L M − U n i \Delta_{s,ni}=\widehat{R}_i^{\mathrm{GLM}}-U_{ni} Δ s , ni = R i GLM − U ni U n i = δ i ξ i U_{ni}=\delta_i\xi_i U ni = δ i ξ i
Step 1: Numerator linearization The nuisance score equation implies ∑ i Z i ε ^ i = 0 \sum_i Z_i\widehat{\varepsilon}_i=0 ∑ i Z i ε i = 0
1 n ∑ i R ^ i G L M = 1 n ∑ i X i ε ^ i . \frac{1}{\sqrt{n}}\sum_i\widehat{R}_i^{\mathrm{GLM}}=\frac{1}{\sqrt{n}}\sum_i X_i\widehat{\varepsilon}_i. n 1 i ∑ R i GLM = n 1 i ∑ X i ε i . Steps 1–2 in the proof of Proposition 3 give
1 n ∑ i = 1 n R ^ i G L M = 1 n ∑ i = 1 n U n i + o P ( 1 ) ⟶ d N ( 0 , τ s 2 ) . (35) \frac{1}{\sqrt{n}}\sum_{i=1}^{n}\widehat{R}_i^{\mathrm{GLM}}=\frac{1}{\sqrt{n}}\sum_{i=1}^{n}U_{ni}+o_{\mathbb{P}}(1)\mathrel{\overset{d}{\longrightarrow}}N(0,\tau_s^2). \tag*{(35)} n 1 i = 1 ∑ n R i GLM = n 1 i = 1 ∑ n U ni + o P ( 1 ) ⟶ d N ( 0 , τ s 2 ) . (35) In particular, n − 1 / 2 ∑ i U n i = O P ( 1 ) n^{-1/2}\sum_i U_{ni}=O_{\mathbb{P}}(1) n − 1/2 ∑ i U ni = O P ( 1 ) n − 1 / 2 ∑ i R ^ i G L M = O P ( 1 ) n^{-1/2}\sum_i\widehat{R}_i^{\mathrm{GLM}}=O_{\mathbb{P}}(1) n − 1/2 ∑ i R i GLM = O P ( 1 )
Step 2: Mean-square approximation of the fitted score contributions Lemma 16 gives
θ ^ = ( 1 n Z T W ^ Z ) − 1 ( 1 n Z T W ^ X ) ⟶ p M − 1 r = θ . (36) \widehat{\theta}=\left(\frac{1}{n}Z^{\mathsf{T}}\widehat{W}Z\right)^{-1}\left(\frac{1}{n}Z^{\mathsf{T}}\widehat{W}X\right)\mathrel{\overset{p}{\longrightarrow}}M^{-1}r=\theta. \tag*{(36)} θ = ( n 1 Z T W Z ) − 1 ( n 1 Z T W X ) ⟶ p M − 1 r = θ . (36) Because ν ^ i G L M = X i − Z i T θ ^ \widehat{\nu}_i^{\mathrm{GLM}}=X_i-Z_i^{\mathsf{T}}\widehat{\theta} ν i GLM = X i − Z i T θ δ i = X i − Z i T θ \delta_i=X_i-Z_i^{\mathsf{T}}\theta δ i = X i − Z i T θ
Δ s , n i = ( δ i − Z i T ( θ ^ − θ ) ) ε ^ i − δ i ξ i = − Z i T ( θ ^ − θ ) ε ^ i + δ i ( ε ^ i − ξ i ) . \begin{aligned}
\Delta_{s,ni}=(\delta_i-Z_i^{\mathsf{T}}(\widehat{\theta}-\theta))\widehat{\varepsilon}_i-\delta_i\xi_i \\
&=-Z_i^{\mathsf{T}}(\widehat{\theta}-\theta)\widehat{\varepsilon}_i+\delta_i(\widehat{\varepsilon}_i-\xi_i).
\end{aligned} Δ s , ni = ( δ i − Z i T ( θ − θ )) ε i − δ i ξ i = − Z i T ( θ − θ ) ε i + δ i ( ε i − ξ i ) . Hence ( u + v ) 2 ≤ 2 u 2 + 2 v 2 (u+v)^2 \le2u^2+2v^2 ( u + v ) 2 ≤ 2 u 2 + 2 v 2 ∥ Z i ∥ ≤ C Z \lVert Z_i\rVert\le C_Z ∥ Z i ∥ ≤ C Z
1 n ∑ i Δ s , n i 2 ≤ 2 C Z 2 ∥ θ ^ − θ ∥ 2 1 n ∑ i ϵ ^ i 2 + 2 n ∑ i δ i 2 ( ϵ ^ i − ξ i ) 2 . \frac{1}{n}\sum_i \Delta_{s,ni}^2 \le2C_Z^2\lVert\hat{\theta}-\theta\rVert^2\frac{1}{n}\sum_i\hat{\epsilon}_i^2+\frac{2}{n}\sum_i\delta_i^2(\hat{\epsilon}_i-\xi_i)^2. n 1 i ∑ Δ s , ni 2 ≤ 2 C Z 2 ∥ θ ^ − θ ∥ 2 n 1 i ∑ ϵ ^ i 2 + n 2 i ∑ δ i 2 ( ϵ ^ i − ξ i ) 2 . The first term is o P ( 1 ) o_P(1) o P ( 1 ) (36) and Lemma 19 . For the second term, Lemma 15 gives, eventually almost surely,
∣ ϵ ^ i − ξ i ∣ ≤ C ( 1 + ∣ Y i ∣ ) { ∥ β ^ − β ∥ + ∣ ϑ ^ − ϑ ⋆ ∣ } . |\hat{\epsilon}_i-\xi_i|\le C(1+|Y_i|)\{\lVert\hat{\beta}-\beta\rVert+|\hat{\vartheta}-\vartheta_\star|\}. ∣ ϵ ^ i − ξ i ∣ ≤ C ( 1 + ∣ Y i ∣ ) {∥ β ^ − β ∥ + ∣ ϑ ^ − ϑ ⋆ ∣ } . Therefore
1 n ∑ i δ i 2 ( ϵ ^ i − ξ i ) 2 ≤ C { ∥ β ^ − β ∥ + ∣ ϑ ^ − ϑ ⋆ ∣ } 2 1 n ∑ i δ i 2 ( 1 + ∣ Y i ∣ ) 2 . \frac{1}{n}\sum_i\delta_i^2(\hat{\epsilon}_i-\xi_i)^2\le C\{\lVert\hat{\beta}-\beta\rVert+|\hat{\vartheta}-\vartheta_\star|\}^2\frac{1}{n}\sum_i\delta_i^2(1+|Y_i|)^2. n 1 i ∑ δ i 2 ( ϵ ^ i − ξ i ) 2 ≤ C {∥ β ^ − β ∥ + ∣ ϑ ^ − ϑ ⋆ ∣ } 2 n 1 i ∑ δ i 2 ( 1 + ∣ Y i ∣ ) 2 . The last sample average is O P ( 1 ) O_P(1) O P ( 1 ) Z Z Z δ i 2 ≤ C ( 1 + X i 2 ) \delta_i^2\le C(1+X_i^2) δ i 2 ≤ C ( 1 + X i 2 )
E [ X i 2 Y i 2 ] ≤ { E [ X i 4 ] E [ Y i 4 ] } 1 / 2 < ∞ . \mathbb{E}[X_i^2Y_i^2]\le\{\mathbb{E}[X_i^4]\mathbb{E}[Y_i^4]\}^{1/2}<\infty. E [ X i 2 Y i 2 ] ≤ { E [ X i 4 ] E [ Y i 4 ] } 1/2 < ∞. Thus the consistency of ( β ^ , ϑ ^ ) (\hat{\beta},\hat{\vartheta}) ( β ^ , ϑ ^ )
1 n ∑ i Δ s , n i 2 = o P ( 1 ) . (37) \frac{1}{n}\sum_i\Delta_{s,ni}^2=o_P(1). \tag*{(37)} n 1 i ∑ Δ s , ni 2 = o P ( 1 ) . (37) Step 3: Consistency of the empirical studentizer By the law of large numbers,
1 n ∑ i U n i 2 → p E [ δ i 2 ξ i 2 ] = τ s 2 . (38) \frac{1}{n}\sum_i U_{ni}^2\xrightarrow{p}\mathbb{E}[\delta_i^2\xi_i^2]=\tau_s^2. \tag*{(38)} n 1 i ∑ U ni 2 p E [ δ i 2 ξ i 2 ] = τ s 2 . (38) By (35) , n − 1 / 2 ∑ i U n i = O P ( 1 ) n^{-1/2}\sum_iU_{ni}=O_P(1) n − 1/2 ∑ i U ni = O P ( 1 ) (38) and (37) verify the remaining hypotheses of Lemma 14 with oracle scores U n i U_{ni} U ni Δ s , n i \Delta_{s,ni} Δ s , ni τ n 2 = τ s 2 \tau_n^2=\tau_s^2 τ n 2 = τ s 2
V ^ n G L M → p τ s 2 . (39) \widehat{V}_n^{\mathrm{GLM}}\xrightarrow{p}\tau_s^2. \tag*{(39)} V n GLM p τ s 2 . (39) Because τ s 2 > 0 \tau_s^2>0 τ s 2 > 0 (35) proves
T n G L M → d N ( 0 , 1 ) . T_n^{\mathrm{GLM}}\xrightarrow{d}N(0,1). T n GLM d N ( 0 , 1 ) . Proof of Theorem 2
Proof. We verify Theorem 1 with observed oracle U n i = δ i ξ i U_{ni}=\delta_i\xi_i U ni = δ i ξ i
C n , i k = ( X k − X ˉ ) ξ i , s X , n 2 = 1 n ∑ k ( X k − X ˉ ) 2 . C_{n,ik}=(X_k-\bar{X})\xi_i,\qquad s_{X,n}^2=\frac{1}{n}\sum_k(X_k-\bar{X})^2. C n , ik = ( X k − X ˉ ) ξ i , s X , n 2 = n 1 k ∑ ( X k − X ˉ ) 2 . Take the trivial training field in Theorem 1(i). Under (8) , E [ ξ i ∣ X i , Z i ] = 0 \mathbb{E}[\xi_i\mid X_i,Z_i]=0 E [ ξ i ∣ X i , Z i ] = 0 E [ U n i ] = 0 \mathbb{E}[U_{ni}]=0 E [ U ni ] = 0 U n i U_{ni} U ni τ s 2 \tau_s^2 τ s 2 § 3 and the hypothesis of positive variance of Proposition 2 give σ s , n 2 = E [ δ i 2 ξ i 2 ] = τ s 2 > 0 \sigma_{s,n}^{2} = \mathbb{E}[\delta_{i}^{2}\xi_{i}^{2}] = \tau_{s}^{2} > 0 σ s , n 2 = E [ δ i 2 ξ i 2 ] = τ s 2 > 0
E [ U n i 4 ] n σ s , n 4 = E [ ( δ i ξ i ) 4 ] n τ s 4 ⟶ 0. \frac{\mathbb{E}[U_{ni}^{4}]}{n\sigma_{s,n}^{4}} = \frac{\mathbb{E}[(\delta_{i}\xi_{i})^{4}]}{n\tau_{s}^{4}} \longrightarrow0. n σ s , n 4 E [ U ni 4 ] = n τ s 4 E [( δ i ξ i ) 4 ] ⟶ 0. For the two requirements for the observed score in Theorem 1(iii), set Δ s , n i ≡ R ^ i G L M − U n i \Delta_{s,ni} \equiv\hat{R}_{i}^{\mathrm{GLM}} - U_{ni} Δ s , ni ≡ R ^ i GLM − U ni (35) states that n − 1 / 2 ∑ i Δ s , n i = o P ( 1 ) n^{-1/2}\sum_i\Delta_{s,ni}=o_{\mathbb{P}}(1) n − 1/2 ∑ i Δ s , ni = o P ( 1 ) (37) states that n − 1 ∑ i Δ s , n i 2 = o P ( 1 ) n^{-1}\sum_i\Delta_{s,ni}^{2}=o_{\mathbb{P}}(1) n − 1 ∑ i Δ s , ni 2 = o P ( 1 ) σ s , n = τ s > 0 \sigma_{s,n}=\tau_s>0 σ s , n = τ s > 0
1 n σ s , n ∑ i Δ s , n i = o P ( 1 ) , 1 n σ s , n 2 ∑ i Δ s , n i 2 = o P ( 1 ) . \frac{1}{\sqrt{n}\,\sigma_{s,n}}\sum_i\Delta_{s,ni}=o_{\mathbb{P}}(1),
\qquad
\frac{1}{n\sigma_{s,n}^{2}}\sum_i\Delta_{s,ni}^{2}=o_{\mathbb{P}}(1). n σ s , n 1 i ∑ Δ s , ni = o P ( 1 ) , n σ s , n 2 1 i ∑ Δ s , ni 2 = o P ( 1 ) . Permutation oracle. Every row of ( C n , i k ) (C_{n,ik}) ( C n , ik ) σ p , n 2 = n − 2 ∑ i , k C n , i k 2 = s X , n 2 n − 1 ∑ i ξ i 2 \sigma_{p,n}^{2}=n^{-2}\sum_{i,k}C_{n,ik}^{2}=s_{X,n}^{2}n^{-1}\sum_i\xi_i^{2} σ p , n 2 = n − 2 ∑ i , k C n , ik 2 = s X , n 2 n − 1 ∑ i ξ i 2 E X i = 0 \mathbb{E}X_i=0 E X i = 0 E X i 2 = 1 \mathbb{E}X_i^{2}=1 E X i 2 = 1 s X , n 2 → p 1 s_{X,n}^{2}\xrightarrow{p}1 s X , n 2 p 1 n − 1 ∑ i ξ i 2 → p τ p 2 > 0 n^{-1}\sum_i\xi_i^{2}\xrightarrow{p}\tau_p^{2}>0 n − 1 ∑ i ξ i 2 p τ p 2 > 0 P ( σ p , n 2 > 0 ) → 1 \mathbb{P}(\sigma_{p,n}^{2}>0)\to1 P ( σ p , n 2 > 0 ) → 1 ξ ˉ = n − 1 ∑ i ξ i \bar{\xi}=n^{-1}\sum_i\xi_i ξ ˉ = n − 1 ∑ i ξ i
∑ k C n , ⋅ k 2 ( n − 1 ) σ p , n 2 = n n − 1 ξ ˉ 2 n − 1 ∑ i ξ i 2 = o P ( 1 ) . \frac{\sum_k C_{n,\cdot k}^{2}}{(n-1)\sigma_{p,n}^{2}}
=
\frac{n}{n-1}\frac{\bar{\xi}^{2}}{n^{-1}\sum_i\xi_i^{2}}
=o_{\mathbb{P}}(1). ( n − 1 ) σ p , n 2 ∑ k C n , ⋅ k 2 = n − 1 n n − 1 ∑ i ξ i 2 ξ ˉ 2 = o P ( 1 ) . For any i.i.d. V i V_i V i E ∣ V i ∣ 4 < ∞ \mathbb{E}|V_i|^{4}<\infty E ∣ V i ∣ 4 < ∞ n P ( ∣ V 1 ∣ 4 > ϵ n ) = o ( 1 ) n\mathbb{P}(|V_1|^{4}>\epsilon n)=o(1) n P ( ∣ V 1 ∣ 4 > ϵ n ) = o ( 1 ) max i V i 2 = o P ( n ) \max_iV_i^{2}=o_{\mathbb{P}}(\sqrt{n}) max i V i 2 = o P ( n ) X i X_i X i ξ i \xi_i ξ i X ˉ = O P ( n − 1 / 2 ) \bar{X}=O_{\mathbb{P}}(n^{-1/2}) X ˉ = O P ( n − 1/2 )
max i , k C n , i k 2 n σ p , n 2 = max k ( X k − X ˉ ) 2 max i ξ i 2 n σ p , n 2 = o P ( 1 ) . \frac{\max_{i,k}C_{n,ik}^{2}}{n\sigma_{p,n}^{2}}
=
\frac{\max_k(X_k-\bar{X})^{2}\max_i\xi_i^{2}}{n\sigma_{p,n}^{2}}
=o_{\mathbb{P}}(1). n σ p , n 2 max i , k C n , ik 2 = n σ p , n 2 max k ( X k − X ˉ ) 2 max i ξ i 2 = o P ( 1 ) . These equations verify both array conditions in Theorem 1(ii).
Permutation nuisance remainder. Put e i ≡ ϵ ^ i − ξ i e_i\equiv\hat{\epsilon}_i-\xi_i e i ≡ ϵ ^ i − ξ i ( β ^ , ϑ ^ ) (\hat{\beta},\hat{\vartheta}) ( β ^ , ϑ ^ )
1 n ∑ i e i 2 ≤ C { ∥ β ^ − β ∥ + ∣ ϑ ^ − ϑ ⋆ ∣ } 2 1 n ∑ i ( 1 + ∣ Y i ∣ ) 2 = o P ( 1 ) . \frac{1}{n}\sum_i e_i^{2}\le C\{\|\hat{\beta}-\beta\|+|\hat{\vartheta}-\vartheta_{\star}|\}^{2}\frac{1}{n}\sum_i(1+|Y_i|)^{2}=o_{\mathbb{P}}(1). n 1 i ∑ e i 2 ≤ C { ∥ β ^ − β ∥ + ∣ ϑ ^ − ϑ ⋆ ∣ } 2 n 1 i ∑ ( 1 + ∣ Y i ∣ ) 2 = o P ( 1 ) . Because the nuisance design contains an intercept, ∑ i ϵ ^ i = 0 \sum_i\hat{\epsilon}_i=0 ∑ i ϵ ^ i = 0 ∑ i Z i ϵ ^ i = 0 \sum_i Z_i\hat{\epsilon}_i=0 ∑ i Z i ϵ ^ i = 0 Δ p , n i ≡ R ^ i G L M , ( Π ) − C n , i Π ( i ) \Delta_{p,ni}\equiv\hat{R}_{i}^{\mathrm{GLM},(\Pi)}-C_{n,i\Pi(i)} Δ p , ni ≡ R ^ i GLM , ( Π ) − C n , i Π ( i )
1 n ∑ i Δ p , n i = 1 n ∑ i ( X Π ( i ) − X ˉ ) e i . \frac{1}{\sqrt{n}}\sum_i\Delta_{p,ni}
=
\frac{1}{\sqrt{n}}\sum_i(X_{\Pi(i)}-\bar{X})e_i. n 1 i ∑ Δ p , ni = n 1 i ∑ ( X Π ( i ) − X ˉ ) e i . Conditional on F n \mathcal{F}_n F n
n n − 1 s X , n 2 1 n ∑ i e i 2 = o P ( 1 ) \frac{n}{n-1}s_{X,n}^{2}\frac{1}{n}\sum_i e_i^{2}=o_{\mathbb{P}}(1) n − 1 n s X , n 2 n 1 i ∑ e i 2 = o P ( 1 ) by Lemma 10. Lemma 8(b) applied to this zero-mean conditional sum proves n − 1 / 2 ∑ i Δ p , n i = o P ( 1 ) n^{-1/2}\sum_i\Delta_{p,ni}=o_{\mathbb{P}}(1) n − 1/2 ∑ i Δ p , ni = o P ( 1 ) r = p r=p r = p
For the squared remainder, define
M ^ n = 1 n Z ⊤ W ^ Z , r ^ n ( Π ) = 1 n ∑ i X Π ( i ) W ^ i Z i . \widehat{M}_n = \frac{1}{n}Z^\top\widehat{W}Z,\qquad\widehat{r}_n^{(\Pi)} = \frac{1}{n}\sum_i X_{\Pi(i)}\widehat{W}_iZ_i. M n = n 1 Z ⊤ W Z , r n ( Π ) = n 1 i ∑ X Π ( i ) W i Z i . Conditional on F n \mathcal{F}_n F n j j j
E [ r ^ n , j ( Π ) ∣ F n ] = X ‾ 1 n ∑ i W ^ i Z i j = o P ( 1 ) , \mathbb{E}[\widehat{r}_{n,j}^{(\Pi)} \mid\mathcal{F}_n] = \overline{X}\frac{1}{n}\sum_i \widehat{W}_iZ_{ij} = o_{\mathrm{P}}(1), E [ r n , j ( Π ) ∣ F n ] = X n 1 i ∑ W i Z ij = o P ( 1 ) , Var ( r ^ n , j ( Π ) ∣ F n ) ≤ 1 n − 1 ( 1 n ∑ i X i 2 ) ( 1 n ∑ i W ^ i 2 Z i j 2 ) = O P ( n − 1 ) . \operatorname{Var}(\widehat{r}_{n,j}^{(\Pi)} \mid\mathcal{F}_n) \le\frac{1}{n-1}\left(\frac{1}{n}\sum_i X_i^2\right)\left(\frac{1}{n}\sum_i \widehat{W}_i^2 Z_{ij}^2\right) = O_{\mathrm{P}}(n^{-1}). Var ( r n , j ( Π ) ∣ F n ) ≤ n − 1 1 ( n 1 i ∑ X i 2 ) ( n 1 i ∑ W i 2 Z ij 2 ) = O P ( n − 1 ) . Here X ‾ = o P ( 1 ) \overline{X} = o_{\mathrm{P}}(1) X = o P ( 1 ) n − 1 ∑ i X i 2 → p 1 n^{-1}\sum_i X_i^2 \xrightarrow{\mathrm{p}} 1 n − 1 ∑ i X i 2 p 1 ∥ Z i ∥ \lVert Z_i\rVert ∥ Z i ∥ W ^ i \widehat{W}_i W i r ^ n , j ( Π ) − E [ r ^ n , j ( Π ) ∣ F n ] \widehat{r}_{n,j}^{(\Pi)}-\mathbb{E}[\widehat{r}_{n,j}^{(\Pi)}\mid\mathcal{F}_n] r n , j ( Π ) − E [ r n , j ( Π ) ∣ F n ] r ^ n , j ( Π ) = o P ( 1 ) \widehat{r}_{n,j}^{(\Pi)}=o_{\mathrm{P}}(1) r n , j ( Π ) = o P ( 1 ) p p p r ^ n ( Π ) = o P ( 1 ) \widehat{r}_n^{(\Pi)}=o_{\mathrm{P}}(1) r n ( Π ) = o P ( 1 ) M ^ n → p M \widehat{M}_n\xrightarrow{\mathrm{p}}M M n p M M M M
θ ^ ( Π ) = M ^ n − 1 r ^ n ( Π ) = o P ( 1 ) . (40) \widehat{\theta}^{(\Pi)}=\widehat{M}_n^{-1}\widehat{r}_n^{(\Pi)}=o_{\mathrm{P}}(1). \tag*{(40)} θ ( Π ) = M n − 1 r n ( Π ) = o P ( 1 ) . (40) Using
Δ p , n i = ( X Π ( i ) − X ‾ ) e i + ( X ‾ − Z i ⊤ θ ^ ( Π ) ) ϵ ^ i , \Delta_{p,ni}=(X_{\Pi(i)}-\overline{X})e_i+(\overline{X}-Z_i^\top\widehat{\theta}^{(\Pi)})\widehat{\epsilon}_i, Δ p , ni = ( X Π ( i ) − X ) e i + ( X − Z i ⊤ θ ( Π ) ) ϵ i , we obtain
1 n ∑ i { Δ p , n i } 2 ≤ 2 n ∑ i ( X Π ( i ) − X ‾ ) 2 e i 2 + 4 { X ‾ 2 + C Z 2 ∥ θ ^ ( Π ) ∥ 2 } 1 n ∑ i ϵ ^ i 2 = o P ( 1 ) . \frac{1}{n}\sum_i\{\Delta_{p,ni}\}^2\leq\frac{2}{n}\sum_i(X_{\Pi(i)}-\overline{X})^2e_i^2
+4\{\overline{X}^2+C_Z^2\lVert\widehat{\theta}^{(\Pi)}\rVert^2\}\frac{1}{n}\sum_i\widehat{\epsilon}_i^2=o_{\mathrm{P}}(1). n 1 i ∑ { Δ p , ni } 2 ≤ n 2 i ∑ ( X Π ( i ) − X ) 2 e i 2 + 4 { X 2 + C Z 2 ∥ θ ( Π ) ∥ 2 } n 1 i ∑ ϵ i 2 = o P ( 1 ) . Indeed, the first term has conditional expectation 2 s X , n 2 n − 1 ∑ i e i 2 = o P ( 1 ) 2s_{X,n}^2n^{-1}\sum_i e_i^2=o_{\mathrm{P}}(1) 2 s X , n 2 n − 1 ∑ i e i 2 = o P ( 1 ) o P ( 1 ) o_{\mathrm{P}}(1) o P ( 1 ) (40) , X ‾ = o P ( 1 ) \overline{X}=o_{\mathrm{P}}(1) X = o P ( 1 ) σ p , n 2 → p τ p 2 > 0 \sigma_{p,n}^2\xrightarrow{\mathrm{p}}\tau_p^2>0 σ p , n 2 p τ p 2 > 0
1 n σ p , n ∑ i Δ p , n i = o P ( 1 ) , 1 n σ p , n 2 ∑ i { Δ p , n i } 2 = o P ( 1 ) . \frac{1}{\sqrt{n}\sigma_{p,n}}\sum_i\Delta_{p,ni}=o_{\mathrm{P}}(1),\qquad
\frac{1}{n\sigma_{p,n}^2}\sum_i\{\Delta_{p,ni}\}^2=o_{\mathrm{P}}(1). n σ p , n 1 i ∑ Δ p , ni = o P ( 1 ) , n σ p , n 2 1 i ∑ { Δ p , ni } 2 = o P ( 1 ) . These are the two r = p r=p r = p r = s r=s r = s
Proof of Theorem 3
Proof. Conditional on F n \mathcal{F}_n F n ∑ i ν ^ i G L M , ( Π ) ϵ ^ i = ∑ i X Π ( i ) ϵ ^ i \sum_i\widehat{\nu}_i^{\mathrm{GLM},(\Pi)}\widehat{\epsilon}_i=\sum_iX_{\Pi(i)}\widehat{\epsilon}_i ∑ i ν i GLM , ( Π ) ϵ i = ∑ i X Π ( i ) ϵ i T ~ n G L M , ( Π ) = H n D n \widetilde{T}_n^{\mathrm{GLM},(\Pi)}=H_nD_n T n GLM , ( Π ) = H n D n H n ≡ ∑ i X Π ( i ) ϵ ^ i / ∑ i ϵ ^ i 2 H_n\equiv\sum_iX_{\Pi(i)}\widehat{\epsilon}_i/\sqrt{\sum_i\widehat{\epsilon}_i^2} H n ≡ ∑ i X Π ( i ) ϵ i / ∑ i ϵ i 2
D n 2 ≡ n − 1 ∑ i ϵ ^ i 2 n − 1 X ( Π ) ⊤ W ^ X ( Π ) − ( n − 1 X ( Π ) ⊤ W ^ Z ) ( n − 1 Z ⊤ W ^ Z ) − 1 ( n − 1 Z ⊤ W ^ X ( Π ) ) . (41) D_n^2\equiv\frac{n^{-1}\sum_i\widehat{\epsilon}_i^2}{n^{-1}X^{(\Pi)\top}\widehat{W}X^{(\Pi)}-\left(n^{-1}X^{(\Pi)\top}\widehat{W}Z\right)\left(n^{-1}Z^\top\widehat{W}Z\right)^{-1}\left(n^{-1}Z^\top\widehat{W}X^{(\Pi)}\right)}. \tag*{(41)} D n 2 ≡ n − 1 X ( Π ) ⊤ W X ( Π ) − ( n − 1 X ( Π ) ⊤ W Z ) ( n − 1 Z ⊤ W Z ) − 1 ( n − 1 Z ⊤ W X ( Π ) ) n − 1 ∑ i ϵ i 2 . (41) By Lemma 19 , ∑ i ϵ ^ i 2 > 0 \sum_i \hat{\epsilon}_i^2 > 0 ∑ i ϵ ^ i 2 > 0 c n i = ϵ ^ i / ∑ j ϵ ^ j 2 c_{ni} = \hat{\epsilon}_i/\sqrt{\sum_j \hat{\epsilon}_j^2} c ni = ϵ ^ i / ∑ j ϵ ^ j 2 ∑ i c n i = 0 \sum_i c_{ni} = 0 ∑ i c ni = 0 ∑ i c n i 2 = 1 \sum_i c_{ni}^2 = 1 ∑ i c ni 2 = 1 d > 0 d > 0 d > 0
∑ i c n i 2 1 { n ∣ c n i ∣ ≥ d } ≤ n d 2 ∑ i c n i 4 = 1 d 2 n − 1 ∑ i ϵ ^ i 4 ( n − 1 ∑ i ϵ ^ i 2 ) 2 . \sum_i c_{ni}^2 \mathbf{1}\{\sqrt{n}|c_{ni}| \ge d\} \le\frac{n}{d^2}\sum_i c_{ni}^4
= \frac{1}{d^2}\frac{n^{-1}\sum_i \hat{\epsilon}_i^4}{(n^{-1}\sum_i \hat{\epsilon}_i^2)^2}. i ∑ c ni 2 1 { n ∣ c ni ∣ ≥ d } ≤ d 2 n i ∑ c ni 4 = d 2 1 ( n − 1 ∑ i ϵ ^ i 2 ) 2 n − 1 ∑ i ϵ ^ i 4 . By Lemma 19 , the ratio on the right is almost surely bounded for all sufficiently large n n n n → ∞ n \to\infty n → ∞ d → ∞ d \to\infty d → ∞
P ( max i ≤ n ∣ X i ∣ > ε n ) ≤ E [ X i 4 ] ε 4 n ⟶ 0. \mathbb{P}\left(\max_{i\le n}|X_i| > \varepsilon\sqrt{n}\right) \le\frac{\mathbb{E}[X_i^4]}{\varepsilon^4 n} \longrightarrow0. P ( i ≤ n max ∣ X i ∣ > ε n ) ≤ ε 4 n E [ X i 4 ] ⟶ 0. Since Π \Pi Π F n \mathcal{F}_n F n H n H_n H n F n \mathcal{F}_n F n ( X , c n ) (X,c_n) ( X , c n )
H n ∣ F n → d , p N ( 0 , 1 ) . (42) H_n \mid\mathcal{F}_n \xrightarrow{d,p} N(0,1). \tag*{(42)} H n ∣ F n d , p N ( 0 , 1 ) . (42) It remains to show that D n 2 → p σ p 2 D_n^2 \xrightarrow{p} \sigma_p^2 D n 2 p σ p 2 Lemma 19 gives
1 n ∑ i ϵ ^ i 2 → p E [ ξ i 2 ] = E [ S i ] . (43) \frac{1}{n}\sum_i \hat{\epsilon}_i^2 \xrightarrow{p} \mathbb{E}[\xi_i^2] = \mathbb{E}[S_i]. \tag*{(43)} n 1 i ∑ ϵ ^ i 2 p E [ ξ i 2 ] = E [ S i ] . (43) Let Q n ( Π ) ≡ n − 1 ∑ i W ^ i X Π ( i ) 2 Q_n^{(\Pi)} \equiv n^{-1}\sum_i \widehat{W}_i X_{\Pi(i)}^2 Q n ( Π ) ≡ n − 1 ∑ i W i X Π ( i ) 2
E [ Q n ( Π ) ∣ F n ] = ( 1 n ∑ i W ^ i ) ( 1 n ∑ i X i 2 ) → p E [ W i ] . \mathbb{E}[Q_n^{(\Pi)} \mid\mathcal{F}_n] = \left(\frac{1}{n}\sum_i \widehat{W}_i\right)\left(\frac{1}{n}\sum_i X_i^2\right) \xrightarrow{p} \mathbb{E}[W_i]. E [ Q n ( Π ) ∣ F n ] = ( n 1 i ∑ W i ) ( n 1 i ∑ X i 2 ) p E [ W i ] . By Lemma 10,
Var ( Q n ( Π ) ∣ F n ) ≤ 1 n − 1 ( 1 n ∑ i W ^ i 2 ) ( 1 n ∑ i X i 4 ) → p 0. \operatorname{Var}(Q_n^{(\Pi)} \mid\mathcal{F}_n) \le\frac{1}{n-1}\left(\frac{1}{n}\sum_i \widehat{W}_i^2\right)\left(\frac{1}{n}\sum_i X_i^4\right) \xrightarrow{p} 0. Var ( Q n ( Π ) ∣ F n ) ≤ n − 1 1 ( n 1 i ∑ W i 2 ) ( n 1 i ∑ X i 4 ) p 0. Therefore
E [ Q n ( Π ) − E [ W i ] ∣ F n ] → p 0 , Var ( Q n ( Π ) − E [ W i ] ∣ F n ) → p 0. \mathbb{E}[Q_n^{(\Pi)}-\mathbb{E}[W_i]\mid\mathcal{F}_n]\xrightarrow{p}0,\qquad
\operatorname{Var}(Q_n^{(\Pi)}-\mathbb{E}[W_i]\mid\mathcal{F}_n)\xrightarrow{p}0. E [ Q n ( Π ) − E [ W i ] ∣ F n ] p 0 , Var ( Q n ( Π ) − E [ W i ] ∣ F n ) p 0. Lemma 8(b) gives
Q n ( Π ) → p E [ W i ] . (44) Q_n^{(\Pi)} \xrightarrow{p} \mathbb{E}[W_i]. \tag*{(44)} Q n ( Π ) p E [ W i ] . (44) The calculation in (40) , established in the proof of Theorem 2, gives θ ^ ( Π ) = o P ( 1 ) \hat{\theta}^{(\Pi)} = o_P(1) θ ^ ( Π ) = o P ( 1 ) M ^ n = n − 1 Z ⊤ W ^ Z \widehat{M}_n = n^{-1}Z^\top\widehat{W}Z M n = n − 1 Z ⊤ W Z r ^ n ( Π ) = n − 1 Z ⊤ W ^ X ( Π ) = M ^ n θ ^ ( Π ) \hat{r}_n^{(\Pi)} = n^{-1}Z^\top\widehat{W}X^{(\Pi)} = \widehat{M}_n\hat{\theta}^{(\Pi)} r ^ n ( Π ) = n − 1 Z ⊤ W X ( Π ) = M n θ ^ ( Π ) M ^ n → p M \widehat{M}_n \xrightarrow{p} M M n p M
( r ^ n ( Π ) ) ⊤ M ^ n − 1 r ^ n ( Π ) = ( θ ^ ( Π ) ) ⊤ M ^ n θ ^ ( Π ) = o P ( 1 ) . (45) \left(\hat{r}_n^{(\Pi)}\right)^\top\widehat{M}_n^{-1}\hat{r}_n^{(\Pi)}
= \left(\hat{\theta}^{(\Pi)}\right)^\top\widehat{M}_n\hat{\theta}^{(\Pi)}
= o_P(1). \tag*{(45)} ( r ^ n ( Π ) ) ⊤ M n − 1 r ^ n ( Π ) = ( θ ^ ( Π ) ) ⊤ M n θ ^ ( Π ) = o P ( 1 ) . (45) Combining (43) , (44) , and (45) gives
D n 2 ⟶ p E [ S i ] E [ W i ] = σ p 2 . D_n^2 \overset{p}{\longrightarrow} \frac{\mathbb{E}[S_i]}{\mathbb{E}[W_i]}=\sigma_p^2. D n 2 ⟶ p E [ W i ] E [ S i ] = σ p 2 . Because σ p 2 > 0 \sigma_p^2>0 σ p 2 > 0 D n ⟶ p σ p D_n \overset{p}{\longrightarrow} \sigma_p D n ⟶ p σ p
T ~ n G L M , ( Π ) ∣ F n ⟶ d , p N ( 0 , σ p 2 ) . \widetilde{T}_n^{\mathrm{GLM},(\Pi)}\mid\mathcal{F}_n \overset{d,p}{\longrightarrow} N(0,\sigma_p^2). T n GLM , ( Π ) ∣ F n ⟶ d , p N ( 0 , σ p 2 ) . Lemma 5 then gives
Q 1 − α ( T ~ n G L M , ( Π ) ∣ F n ) ⟶ p σ p z 1 − α . Q_{1-\alpha}\left(\widetilde{T}_n^{\mathrm{GLM},(\Pi)}\mid\mathcal{F}_n\right)\overset{p}{\longrightarrow}\sigma_p z_{1-\alpha}. Q 1 − α ( T n GLM , ( Π ) ∣ F n ) ⟶ p σ p z 1 − α . If Var ( Y i ∣ X i , Z i ) = c v i \operatorname{Var}(Y_i\mid X_i,Z_i)=cv_i Var ( Y i ∣ X i , Z i ) = c v i c > 0 c>0 c > 0 E [ ε i 2 ∣ X i , Z i ] = c W i \mathbb{E}[\varepsilon_i^2\mid X_i,Z_i]=cW_i E [ ε i 2 ∣ X i , Z i ] = c W i τ s 2 = c E [ R i 2 ] \tau_s^2=c\mathbb{E}[R_i^2] τ s 2 = c E [ R i 2 ] τ p 2 = c E [ W i ] \tau_p^2=c\mathbb{E}[W_i] τ p 2 = c E [ W i ] σ s 2 = σ p 2 = c \sigma_s^2=\sigma_p^2=c σ s 2 = σ p 2 = c T ~ n G L M ⟶ d N ( 0 , c ) \widetilde{T}_n^{\mathrm{GLM}}\overset{d}{\longrightarrow}N(0,c) T n GLM ⟶ d N ( 0 , c ) c z 1 − α \sqrt{c}z_{1-\alpha} c z 1 − α α \alpha α
Proof of Theorem 4
Proof. Write ν i = ν i G C M \nu_i=\nu_i^{\mathrm{GCM}} ν i = ν i GCM ϵ i = ϵ i G C M \epsilon_i=\epsilon_i^{\mathrm{GCM}} ϵ i = ϵ i GCM § 4.1 , and set
e x , i = μ x ( Z i ) − μ ^ x ( Z i ) , e y , i = μ y ( Z i ) − μ ^ y ( Z i ) . e_{x,i}=\mu_x(Z_i)-\widehat{\mu}_x(Z_i),\qquad e_{y,i}=\mu_y(Z_i)-\widehat{\mu}_y(Z_i). e x , i = μ x ( Z i ) − μ x ( Z i ) , e y , i = μ y ( Z i ) − μ y ( Z i ) . By Assumption 8(i) and Remark 15,
E [ ν i 2 ∣ Z i ] ≤ 4 C X 1 / 2 , E [ ν i 4 ∣ Z i ] ≤ 16 C X , 4 , E [ ϵ i 2 ∣ Z i ] ≤ v ˉ Y , E [ ϵ i 4 ∣ Z i ] ≤ 16 C Y , 4 . (46) \begin{aligned}
\mathbb{E}[\nu_i^2\mid Z_i]&\le4C_X^{1/2}, & \mathbb{E}[\nu_i^4\mid Z_i]&\le16C_{X,4},\\
\mathbb{E}[\epsilon_i^2\mid Z_i]&\le\bar v_Y, & \mathbb{E}[\epsilon_i^4\mid Z_i]&\le16C_{Y,4}. \tag*{(46)}
\end{aligned} E [ ν i 2 ∣ Z i ] E [ ϵ i 2 ∣ Z i ] ≤ 4 C X 1/2 , ≤ v ˉ Y , E [ ν i 4 ∣ Z i ] E [ ϵ i 4 ∣ Z i ] ≤ 16 C X , 4 , ≤ 16 C Y , 4 . (46) Condition (i): observed oracle Take the trivial training field and U n i = ν i ϵ i U_{ni}=\nu_i\epsilon_i U ni = ν i ϵ i H 0 C I H_0^{\mathrm{CI}} H 0 CI E [ U n i ∣ X i , Z i ] = ν i E [ ϵ i ∣ X i , Z i ] = 0 \mathbb{E}[U_{ni}\mid X_i,Z_i]=\nu_i\mathbb{E}[\epsilon_i\mid X_i,Z_i]=0 E [ U ni ∣ X i , Z i ] = ν i E [ ϵ i ∣ X i , Z i ] = 0 X i X_i X i Y i Y_i Y i Z i Z_i Z i (46) give
σ s , n 2 = E [ U n i 2 ] = σ G C M 2 > 0 , E [ U n i 4 ] n σ s , n 4 = E { E [ ν i 4 ∣ Z i ] E [ ϵ i 4 ∣ Z i ] } n σ G C M 4 ≤ 256 C X , 4 C Y , 4 n σ G C M 4 ⟶ 0. \sigma_{s,n}^2=\mathbb{E}[U_{ni}^2]=\sigma_{\mathrm{GCM}}^2>0,\qquad
\frac{\mathbb{E}[U_{ni}^4]}{n\sigma_{s,n}^4}
=\frac{\mathbb{E}\{\mathbb{E}[\nu_i^4\mid Z_i]\mathbb{E}[\epsilon_i^4\mid Z_i]\}}{n\sigma_{\mathrm{GCM}}^4}
\le\frac{256C_{X,4}C_{Y,4}}{n\sigma_{\mathrm{GCM}}^4}\longrightarrow0. σ s , n 2 = E [ U ni 2 ] = σ GCM 2 > 0 , n σ s , n 4 E [ U ni 4 ] = n σ GCM 4 E { E [ ν i 4 ∣ Z i ] E [ ϵ i 4 ∣ Z i ]} ≤ n σ GCM 4 256 C X , 4 C Y , 4 ⟶ 0. The positive variance is Assumption 8(ii).
Condition (ii): permutation oracle Take
C n , i k = ( X k − X ˉ ) ϵ i , s X , n 2 = 1 n ∑ k ( X k − X ˉ ) 2 . C_{n,ik}=(X_k-\bar X)\epsilon_i,\qquad s_{X,n}^2=\frac{1}{n}\sum_k(X_k-\bar X)^2. C n , ik = ( X k − X ˉ ) ϵ i , s X , n 2 = n 1 k ∑ ( X k − X ˉ ) 2 . The array is F n \mathcal{F}_n F n ∑ k C n , i k = 0 \sum_k C_{n,ik}=0 ∑ k C n , ik = 0 i i i
σ p , n 2 = 1 n 2 ∑ i , k C n , i k 2 = s X , n 2 1 n ∑ i ϵ i 2 ⟶ p Var ( X ) E [ ϵ i 2 ] = σ G C M , 0 2 > 0. (47) \sigma_{p,n}^2=\frac{1}{n^2}\sum_{i,k}C_{n,ik}^2=s_{X,n}^2\frac{1}{n}\sum_i\epsilon_i^2\overset{p}{\longrightarrow}\operatorname{Var}(X)\mathbb{E}[\epsilon_i^2]=\sigma_{\mathrm{GCM},0}^2>0. \tag*{(47)} σ p , n 2 = n 2 1 i , k ∑ C n , ik 2 = s X , n 2 n 1 i ∑ ϵ i 2 ⟶ p Var ( X ) E [ ϵ i 2 ] = σ GCM , 0 2 > 0. (47) The law of large numbers gives the convergence; positivity follows from Remark 17. For ϵ ˉ = n − 1 ∑ i ϵ i \bar{\epsilon}=n^{-1}\sum_i\epsilon_i ϵ ˉ = n − 1 ∑ i ϵ i C n , ⋅ k = ( X k − X ˉ ) ϵ ˉ C_{n,\cdot k}=(X_k-\bar{X})\bar{\epsilon} C n , ⋅ k = ( X k − X ˉ ) ϵ ˉ
∑ k C n , h 2 ( n − 1 ) σ p , n 2 = n n − 1 ϵ ˉ 2 1 n ∑ i ϵ i 2 = o P ( 1 ) , (48) \frac{\sum_k C_{n,h}^2}{(n-1)\sigma_{p,n}^2}=\frac{n}{n-1}\frac{\bar{\epsilon}^2}{\frac{1}{n}\sum_i\epsilon_i^2}=o_P(1), \tag*{(48)} ( n − 1 ) σ p , n 2 ∑ k C n , h 2 = n − 1 n n 1 ∑ i ϵ i 2 ϵ ˉ 2 = o P ( 1 ) , (48) because ϵ ˉ = O P ( n − 1 / 2 ) \bar{\epsilon}=O_P(n^{-1/2}) ϵ ˉ = O P ( n − 1/2 ) 1 n ∑ i ϵ i 2 → p E [ ϵ i 2 ] > 0 \frac{1}{n}\sum_i\epsilon_i^2 \xrightarrow{p} \mathbb{E}[\epsilon_i^2]>0 n 1 ∑ i ϵ i 2 p E [ ϵ i 2 ] > 0 E ∣ X i ∣ 4 ≤ C X , 4 \mathbb{E}|X_i|^4\le C_{X,4} E ∣ X i ∣ 4 ≤ C X , 4 E ∣ ϵ i ∣ 4 ≤ 16 C Y , 4 \mathbb{E}|\epsilon_i|^4\le16C_{Y,4} E ∣ ϵ i ∣ 4 ≤ 16 C Y , 4 V i = X i V_i=X_i V i = X i V i = ϵ i V_i=\epsilon_i V i = ϵ i ϵ > 0 \epsilon>0 ϵ > 0 n P ( ∣ V 1 ∣ 4 > ϵ n ) → 0 nP(|V_1|^4>\epsilon n)\to0 n P ( ∣ V 1 ∣ 4 > ϵ n ) → 0 max i V i 2 = o P ( n ) \max_i V_i^2=o_P(\sqrt{n}) max i V i 2 = o P ( n ) ∣ X ˉ ∣ ≤ max k ∣ X k ∣ |\bar{X}|\le\max_k|X_k| ∣ X ˉ ∣ ≤ max k ∣ X k ∣
max k ( X k − X ˉ ) 2 ≤ 4 max k X k 2 = o P ( n ) . \max_k(X_k-\bar{X})^2\le4\max_k X_k^2=o_P(\sqrt{n}). k max ( X k − X ˉ ) 2 ≤ 4 k max X k 2 = o P ( n ) . Together with max i ϵ i 2 = o P ( n ) \max_i\epsilon_i^2=o_P(\sqrt{n}) max i ϵ i 2 = o P ( n ) (47) , this yields
max i , k C n , i k 2 n σ p , n 2 = max k ( X k − X ˉ ) 2 max i ϵ i 2 n σ p , n 2 = o P ( 1 ) . (49) \frac{\max_{i,k}C_{n,ik}^2}{n\sigma_{p,n}^2}
=\frac{\max_k(X_k-\bar{X})^2\max_i\epsilon_i^2}{n\sigma_{p,n}^2}=o_P(1). \tag*{(49)} n σ p , n 2 max i , k C n , ik 2 = n σ p , n 2 max k ( X k − X ˉ ) 2 max i ϵ i 2 = o P ( 1 ) . (49) Equations (48) –(49) are the two array requirements of Theorem 1(ii).
Condition (iii): observed fitted scores Write R ^ i G C M = U n i + Δ s , n i \hat{R}^{\mathrm{GCM}}_i=U_{ni}+\Delta_{s,ni} R ^ i GCM = U ni + Δ s , ni Δ s , n i = e x , i ϵ i + ν i e y , i + e x , i e y , i \Delta_{s,ni}=e_{x,i}\epsilon_i+\nu_i e_{y,i}+e_{x,i}e_{y,i} Δ s , ni = e x , i ϵ i + ν i e y , i + e x , i e y , i ( ω x , ω y ) (\omega_x,\omega_y) ( ω x , ω y ) § 9 , set H x = σ ( ( X i , Z i ) i ≤ n , ω x ) \mathcal{H}_x=\sigma((X_i,Z_i)_{i\le n},\omega_x) H x = σ (( X i , Z i ) i ≤ n , ω x ) H y = σ ( ( Y i , Z i ) i ≤ n , ω y ) \mathcal{H}_y=\sigma((Y_i,Z_i)_{i\le n},\omega_y) H y = σ (( Y i , Z i ) i ≤ n , ω y ) e x , i e_{x,i} e x , i e y , i e_{y,i} e y , i H x \mathcal{H}_x H x H y \mathcal{H}_y H y H 0 C I \mathcal{H}_0^{\mathrm{CI}} H 0 CI ϵ i \epsilon_i ϵ i H x \mathcal{H}_x H x ν i \nu_i ν i H y \mathcal{H}_y H y
Var ( 1 n ∑ i e x , i ϵ i ∣ H x ) = 1 n ∑ i e x , i 2 E [ ϵ i 2 ∣ Z i ] ≤ v ˉ Y E n , x = o P ( 1 ) , \operatorname{Var}\left(\left.\frac{1}{\sqrt{n}}\sum_i e_{x,i}\epsilon_i\right|\mathcal{H}_x\right)=\frac{1}{n}\sum_i e_{x,i}^2\mathbb{E}[\epsilon_i^2\mid Z_i]\le\bar{v}_Y E_{n,x}=o_P(1), Var ( n 1 i ∑ e x , i ϵ i H x ) = n 1 i ∑ e x , i 2 E [ ϵ i 2 ∣ Z i ] ≤ v ˉ Y E n , x = o P ( 1 ) , Var ( 1 n ∑ i ν i e y , i ∣ H y ) = 1 n ∑ i e y , i 2 E [ ν i 2 ∣ Z i ] ≤ 4 C X , 4 1 / 2 E n , y = o P ( 1 ) . (50) \operatorname{Var}\left(\left.\frac{1}{\sqrt{n}}\sum_i\nu_i e_{y,i}\right|\mathcal{H}_y\right)=\frac{1}{n}\sum_i e_{y,i}^2\mathbb{E}[\nu_i^2\mid Z_i]\le4C_{X,4}^{1/2}E_{n,y}=o_P(1). \tag*{(50)} Var ( n 1 i ∑ ν i e y , i H y ) = n 1 i ∑ e y , i 2 E [ ν i 2 ∣ Z i ] ≤ 4 C X , 4 1/2 E n , y = o P ( 1 ) . (50) Their conditional means are zero, so Lemma 8(b) makes both sums o P ( 1 ) o_P(1) o P ( 1 )
E [ 1 n ∑ i e x , i 2 ϵ i 2 ∣ H x ] ≤ v ˉ Y E n , x = o P ( 1 ) , \mathbb{E}\left[\left.\frac{1}{n}\sum_i e_{x,i}^2\epsilon_i^2\right|\mathcal{H}_x\right]\le\bar{v}_Y E_{n,x}=o_P(1), E [ n 1 i ∑ e x , i 2 ϵ i 2 H x ] ≤ v ˉ Y E n , x = o P ( 1 ) , E [ 1 n ∑ i ν i 2 e y , i 2 ∣ H y ] ≤ 4 C X , 4 1 / 2 E n , y = o P ( 1 ) . (51) \mathbb{E}\left[\left.\frac{1}{n}\sum_i\nu_i^2e_{y,i}^2\right|\mathcal{H}_y\right]\le4C_{X,4}^{1/2}E_{n,y}=o_P(1). \tag*{(51)} E [ n 1 i ∑ ν i 2 e y , i 2 H y ] ≤ 4 C X , 4 1/2 E n , y = o P ( 1 ) . (51) Lemma 8(a) applies to both nonnegative averages. The product term satisfies
∣ 1 n ∑ i e x , i e y , i ∣ 2 ≤ n E n , x E n , y = o P ( 1 ) , 1 n ∑ i e x , i 2 e y , i 2 ≤ n E n , x E n , y = o P ( 1 ) . (52) \left|\frac{1}{\sqrt{n}}\sum_i e_{x,i}e_{y,i}\right|^2\le nE_{n,x}E_{n,y}=o_P(1),\qquad
\frac{1}{n}\sum_i e_{x,i}^2e_{y,i}^2\le nE_{n,x}E_{n,y}=o_P(1). \tag*{(52)} n 1 i ∑ e x , i e y , i 2 ≤ n E n , x E n , y = o P ( 1 ) , n 1 i ∑ e x , i 2 e y , i 2 ≤ n E n , x E n , y = o P ( 1 ) . (52) The o P ( 1 ) o_P(1) o P ( 1 ) (50) –(52) use exactly E n , x , E n , y = o P ( 1 ) E_{n,x},E_{n,y}=o_P(1) E n , x , E n , y = o P ( 1 ) n E n , x E n , y = o P ( 1 ) nE_{n,x}E_{n,y}=o_P(1) n E n , x E n , y = o P ( 1 ) (50) , (51) , and (52) by the triangle inequality and ( u + v + w ) 2 ≤ 3 ( u 2 + v 2 + w 2 ) (u+v+w)^2 \le3(u^2+v^2+w^2) ( u + v + w ) 2 ≤ 3 ( u 2 + v 2 + w 2 )
∣ 1 n ∑ i Δ s , n i ∣ ≤ ∣ 1 n ∑ i e x , i ϵ i ∣ + ∣ 1 n ∑ i ν i e y , i ∣ + ∣ 1 n ∑ i e x , i e y , i ∣ = o P ( 1 ) , \left|\frac{1}{\sqrt{n}}\sum_i \Delta_{s,ni}\right| \le\left|\frac{1}{\sqrt{n}}\sum_i e_{x,i}\epsilon_i\right|+\left|\frac{1}{\sqrt{n}}\sum_i \nu_i e_{y,i}\right|+\left|\frac{1}{\sqrt{n}}\sum_i e_{x,i}e_{y,i}\right|=o_{\mathrm{P}}(1), n 1 i ∑ Δ s , ni ≤ n 1 i ∑ e x , i ϵ i + n 1 i ∑ ν i e y , i + n 1 i ∑ e x , i e y , i = o P ( 1 ) , 1 n ∑ i Δ s , n i 2 ≤ 3 { 1 n ∑ i e x , i 2 ϵ i 2 + 1 n ∑ i ν i 2 e y , i 2 + 1 n ∑ i e x , i 2 e y , i 2 } = o P ( 1 ) . \frac{1}{n}\sum_i \Delta_{s,ni}^2 \le3\left\{\frac{1}{n}\sum_i e_{x,i}^2\epsilon_i^2+\frac{1}{n}\sum_i \nu_i^2e_{y,i}^2+\frac{1}{n}\sum_i e_{x,i}^2e_{y,i}^2\right\}=o_{\mathrm{P}}(1). n 1 i ∑ Δ s , ni 2 ≤ 3 { n 1 i ∑ e x , i 2 ϵ i 2 + n 1 i ∑ ν i 2 e y , i 2 + n 1 i ∑ e x , i 2 e y , i 2 } = o P ( 1 ) . Therefore
1 n ∑ i R ^ i G C M = 1 n ∑ i U n i + o P ( 1 ) , 1 n ∑ i Δ s , n i 2 = o P ( 1 ) . (53) \frac{1}{\sqrt{n}}\sum_i \widehat{R}_i^{\mathrm{GCM}}=\frac{1}{\sqrt{n}}\sum_i U_{ni}+o_{\mathrm{P}}(1),\qquad\frac{1}{n}\sum_i\Delta_{s,ni}^2=o_{\mathrm{P}}(1). \tag*{(53)} n 1 i ∑ R i GCM = n 1 i ∑ U ni + o P ( 1 ) , n 1 i ∑ Δ s , ni 2 = o P ( 1 ) . (53) Since σ s , n 2 = σ G C M 2 > 0 \sigma_{s,n}^2=\sigma_{\mathrm{GCM}}^2>0 σ s , n 2 = σ GCM 2 > 0 r = s r=s r = s
1 n σ s , n ∑ i Δ s , n i = o P ( 1 ) , 1 n σ s , n 2 ∑ i Δ s , n i 2 = o P ( 1 ) . \frac{1}{\sqrt{n}\sigma_{s,n}}\sum_i\Delta_{s,ni}=o_{\mathrm{P}}(1),\qquad\frac{1}{n\sigma_{s,n}^2}\sum_i\Delta_{s,ni}^2=o_{\mathrm{P}}(1). n σ s , n 1 i ∑ Δ s , ni = o P ( 1 ) , n σ s , n 2 1 i ∑ Δ s , ni 2 = o P ( 1 ) . The ordinary law of large numbers gives n − 1 ∑ i U n i 2 → p σ G C M 2 n^{-1}\sum_i U_{ni}^2 \xrightarrow{p}\sigma_{\mathrm{GCM}}^2 n − 1 ∑ i U ni 2 p σ GCM 2 n − 1 / 2 ∑ i U n i = O P ( 1 ) n^{-1/2}\sum_i U_{ni}=O_{\mathrm{P}}(1) n − 1/2 ∑ i U ni = O P ( 1 ) (53) therefore also yields
V ^ n G C M → p σ G C M 2 . (54) \widehat{V}_n^{\mathrm{GCM}}\xrightarrow{p}\sigma_{\mathrm{GCM}}^2. \tag*{(54)} V n GCM p σ GCM 2 . (54) Condition (iii): permuted fitted scores Write
e ~ x , i = X ‾ − μ ^ x ( Π ) ( Z i ) , Δ p , n i = ( X Π ( i ) − X ‾ ) e y , i + e ~ x , i ϵ i + e ~ x , i e y , i . \widetilde{e}_{x,i}=\overline{X}-\widehat{\mu}_x^{(\Pi)}(Z_i),\qquad\Delta_{p,ni}=(X_{\Pi(i)}-\overline{X})e_{y,i}+\widetilde{e}_{x,i}\epsilon_i+\widetilde{e}_{x,i}e_{y,i}. e x , i = X − μ x ( Π ) ( Z i ) , Δ p , ni = ( X Π ( i ) − X ) e y , i + e x , i ϵ i + e x , i e y , i . The first term in the normalized sum has conditional mean zero and, by Lemma 10,
Var Π ( 1 n ∑ i ( X Π ( i ) − X ‾ ) e y , i | F n ) ≤ n n − 1 s X , n 2 E n , y = o P ( 1 ) . (55) \operatorname{Var}_{\Pi}\left(\frac{1}{\sqrt{n}}\sum_i(X_{\Pi(i)}-\overline{X})e_{y,i}\,\middle|\,\mathcal{F}_n\right)\le\frac{n}{n-1}s_{X,n}^2E_{n,y}=o_{\mathrm{P}}(1). \tag*{(55)} Var Π ( n 1 i ∑ ( X Π ( i ) − X ) e y , i F n ) ≤ n − 1 n s X , n 2 E n , y = o P ( 1 ) . (55) For the second term, condition on G n = σ ( H x , Π ) \mathcal{G}_n=\sigma(\mathcal{H}_x,\Pi) G n = σ ( H x , Π ) e ~ x , i \widetilde{e}_{x,i} e x , i G n \mathcal{G}_n G n ϵ i \epsilon_i ϵ i
Var ( 1 n ∑ i e ~ x , i ϵ i | G n ) = 1 n ∑ i e ~ x , i 2 E [ ϵ i 2 ∣ Z i ] ≤ v ‾ Y E ~ n , x = o P ( 1 ) . (56) \operatorname{Var}\left(\frac{1}{\sqrt{n}}\sum_i\widetilde{e}_{x,i}\epsilon_i\,\middle|\,\mathcal{G}_n\right)=\frac{1}{n}\sum_i\widetilde{e}_{x,i}^2\mathbb{E}[\epsilon_i^2\mid Z_i]\le\overline{v}_Y\widetilde{E}_{n,x}=o_{\mathrm{P}}(1). \tag*{(56)} Var ( n 1 i ∑ e x , i ϵ i G n ) = n 1 i ∑ e x , i 2 E [ ϵ i 2 ∣ Z i ] ≤ v Y E n , x = o P ( 1 ) . (56) Lemma 8(b) applies to both sums. The product term satisfies
∣ 1 n ∑ i e ~ x , i e y , i ∣ 2 ≤ n E ~ n , x E n , y = o P ( 1 ) . (57) \left|\frac{1}{\sqrt{n}}\sum_i\widetilde{e}_{x,i}e_{y,i}\right|^2\le n\widetilde{E}_{n,x}E_{n,y}=o_{\mathrm{P}}(1). \tag*{(57)} n 1 i ∑ e x , i e y , i 2 ≤ n E n , x E n , y = o P ( 1 ) . (57) For the mean-square requirement, conditional expectation and Lemma 8(a) give
E Π [ 1 n ∑ i ( X Π ( i ) − X ‾ ) 2 e y , i 2 | F n ] = s X , n 2 E n , y = o P ( 1 ) , \mathbb{E}_{\Pi}\left[\frac{1}{n}\sum_i(X_{\Pi(i)}-\overline{X})^2e_{y,i}^2\,\middle|\,\mathcal{F}_n\right]=s_{X,n}^2E_{n,y}=o_{\mathrm{P}}(1), E Π [ n 1 i ∑ ( X Π ( i ) − X ) 2 e y , i 2 F n ] = s X , n 2 E n , y = o P ( 1 ) , E [ 1 n ∑ i e ~ x , i 2 ϵ i 2 | G n ] ≤ v ‾ Y E ~ n , x = o P ( 1 ) . (58) \mathbb{E}\left[\frac{1}{n}\sum_i\widetilde{e}_{x,i}^2\epsilon_i^2\,\middle|\,\mathcal{G}_n\right]\le\overline{v}_Y\widetilde{E}_{n,x}=o_{\mathrm{P}}(1). \tag*{(58)} E [ n 1 i ∑ e x , i 2 ϵ i 2 G n ] ≤ v Y E n , x = o P ( 1 ) . (58) Finally,
1 n ∑ i e ~ x , i 2 e y , i 2 ≤ n E ~ n , x E n , y = o P ( 1 ) . (59) \frac{1}{n}\sum_i \widetilde{e}_{x,i}^{2}e_{y,i}^{2}\le n\widetilde{E}_{n,x}E_{n,y}=o_{\mathbb{P}}(1). \tag*{(59)} n 1 i ∑ e x , i 2 e y , i 2 ≤ n E n , x E n , y = o P ( 1 ) . (59) Here Assumption 3 supplies E ~ n , x = o P ( 1 ) \widetilde{E}_{n,x}=o_{\mathbb{P}}(1) E n , x = o P ( 1 ) n E n , y E ~ n , x = o P ( 1 ) nE_{n,y}\widetilde{E}_{n,x}=o_{\mathbb{P}}(1) n E n , y E n , x = o P ( 1 ) E n , y = o P ( 1 ) E_{n,y}=o_{\mathbb{P}}(1) E n , y = o P ( 1 ) (55) , (56) , (57) , (58) , and (59) gives
∣ 1 n ∑ i Δ p , n i ∣ ≤ ∣ 1 n ∑ i ( X Π ( i ) − X ‾ ) e y , i ∣ + ∣ 1 n ∑ i e ~ x , i ϵ i ∣ + ∣ 1 n ∑ i e ~ x , i e y , i ∣ = o P ( 1 ) , \left|\frac{1}{\sqrt{n}}\sum_i\Delta_{p,ni}\right|
\leq\left|\frac{1}{\sqrt{n}}\sum_i(X_{\Pi(i)}-\overline{X})e_{y,i}\right|+\left|\frac{1}{\sqrt{n}}\sum_i\widetilde{e}_{x,i}\epsilon_i\right|+\left|\frac{1}{\sqrt{n}}\sum_i\widetilde{e}_{x,i}e_{y,i}\right|=o_{\mathbb{P}}(1), n 1 i ∑ Δ p , ni ≤ n 1 i ∑ ( X Π ( i ) − X ) e y , i + n 1 i ∑ e x , i ϵ i + n 1 i ∑ e x , i e y , i = o P ( 1 ) , 1 n ∑ i Δ p , n i 2 ≤ 3 { 1 n ∑ i ( X Π ( i ) − X ‾ ) 2 e y , i 2 + 1 n ∑ i e ~ x , i 2 ϵ i 2 + 1 n ∑ i e ~ x , i 2 e y , i 2 } = o P ( 1 ) . \frac{1}{n}\sum_i\Delta_{p,ni}^{2}\leq3\left\{\frac{1}{n}\sum_i(X_{\Pi(i)}-\overline{X})^{2}e_{y,i}^{2}+\frac{1}{n}\sum_i\widetilde{e}_{x,i}^{2}\epsilon_i^{2}+\frac{1}{n}\sum_i\widetilde{e}_{x,i}^{2}e_{y,i}^{2}\right\}=o_{\mathbb{P}}(1). n 1 i ∑ Δ p , ni 2 ≤ 3 { n 1 i ∑ ( X Π ( i ) − X ) 2 e y , i 2 + n 1 i ∑ e x , i 2 ϵ i 2 + n 1 i ∑ e x , i 2 e y , i 2 } = o P ( 1 ) . By (47) , σ p , n − 1 = O P ( 1 ) \sigma_{p,n}^{-1}=O_{\mathbb{P}}(1) σ p , n − 1 = O P ( 1 ) σ p , n − 2 = O P ( 1 ) \sigma_{p,n}^{-2}=O_{\mathbb{P}}(1) σ p , n − 2 = O P ( 1 )
1 n σ p , n ∑ i Δ p , n i = o P ( 1 ) , \frac{1}{\sqrt{n}\,\sigma_{p,n}}\sum_i\Delta_{p,ni}=o_{\mathbb{P}}(1), n σ p , n 1 i ∑ Δ p , ni = o P ( 1 ) , 1 n σ p , n 2 ∑ i Δ p , n i 2 = o P ( 1 ) . \frac{1}{n\sigma_{p,n}^{2}}\sum_i\Delta_{p,ni}^{2}=o_{\mathbb{P}}(1). n σ p , n 2 1 i ∑ Δ p , ni 2 = o P ( 1 ) . All conditions of Theorem 1 are now verified, and its conclusion gives the two normal limits and the level statement of Theorem 4.
Proof of Theorem 5
Proof. Apply Theorem 1 with sample size n ⋆ = ∣ D 1 ∣ n_{\star}=|\mathcal{D}_{1}| n ⋆ = ∣ D 1 ∣ § 4.2 . All sums and maxima below are over D 1 \mathcal{D}_{1} D 1 H n \mathcal{H}_{n} H n H n \mathcal{H}_{n} H n ν i = ν i P C M \nu_i=\nu_i^{\mathrm{PCM}} ν i = ν i PCM ϵ i = ϵ i P C M \epsilon_i=\epsilon_i^{\mathrm{PCM}} ϵ i = ϵ i PCM
e y , i = μ y ( Z i ) − μ ^ y ( Z i ) , e f , i = μ f ( Z i ) − μ ^ f ( Z i ) . e_{y,i}=\mu_y(Z_i)-\widehat{\mu}_y(Z_i),\qquad e_{f,i}=\mu_f(Z_i)-\widehat{\mu}_f(Z_i). e y , i = μ y ( Z i ) − μ y ( Z i ) , e f , i = μ f ( Z i ) − μ f ( Z i ) . Assumption 9(i) and Remark 15 give the explicit bounds
c Y ≤ V Y ( Z i ) ≤ v ‾ Y , E [ ϵ i 4 ∣ Z i ] ≤ m ‾ Y , 4 almost surely . (60) c_Y\leq V_Y(Z_i)\leq\overline{v}_Y,\qquad\mathbb{E}[\epsilon_i^4\mid Z_i]\leq\overline{m}_{Y,4}\quad\text{almost surely}. \tag*{(60)} c Y ≤ V Y ( Z i ) ≤ v Y , E [ ϵ i 4 ∣ Z i ] ≤ m Y , 4 almost surely . (60) All ratios below are evaluated on the events where their denominators are positive; Assumption 9(ii)–(iii) makes those events have probability tending to one.
Condition (i): observed oracle scores Take U n i = ν i ϵ i U_{ni}=\nu_i\epsilon_i U ni = ν i ϵ i H n \mathcal{H}_{n} H n H 0 C I \mathcal{H}_{0}^{\mathrm{CI}} H 0 CI E [ ϵ i ∣ X i , Z i , H n ] = 0 \mathbb{E}[\epsilon_i\mid X_i,Z_i,\mathcal{H}_{n}]=0 E [ ϵ i ∣ X i , Z i , H n ] = 0 U n i U_{ni} U ni σ s , n 2 = E [ U n i 2 ∣ H n ] \sigma_{s,n}^{2}=\mathbb{E}[U_{ni}^{2}\mid\mathcal{H}_{n}] σ s , n 2 = E [ U ni 2 ∣ H n ] σ f , n 2 = E [ ν i 2 ∣ H n ] \sigma_{f,n}^{2}=\mathbb{E}[\nu_i^{2}\mid\mathcal{H}_{n}] σ f , n 2 = E [ ν i 2 ∣ H n ]
σ s , n 2 = E [ ν i 2 V Y ( Z i ) ∣ H n ] , c Y σ f , n 2 ≤ σ s , n 2 ≤ v ‾ Y σ f , n 2 . (61) \sigma_{s,n}^{2}=\mathbb{E}[\nu_i^{2}V_Y(Z_i)\mid\mathcal{H}_{n}],\qquad c_Y\sigma_{f,n}^{2}\leq\sigma_{s,n}^{2}\leq\overline{v}_Y\sigma_{f,n}^{2}. \tag*{(61)} σ s , n 2 = E [ ν i 2 V Y ( Z i ) ∣ H n ] , c Y σ f , n 2 ≤ σ s , n 2 ≤ v Y σ f , n 2 . (61) In particular P ( σ s , n 2 > 0 ) → 1 \mathbb{P}(\sigma_{s,n}^{2}>0)\to1 P ( σ s , n 2 > 0 ) → 1 (17) give
E [ U n i 4 ∣ H n ] n ⋆ σ s , n 4 = E [ ν i 4 E [ ϵ i 4 ∣ Z i ] ∣ H n ] n ⋆ σ s , n 4 ≤ m ‾ Y , 4 E [ ν i 4 ∣ H n ] n ⋆ c Y 2 σ f , n 4 = o P ( 1 ) . \frac{\mathbb{E}[U_{ni}^{4}\mid\mathcal{H}_{n}]}{n_{\star}\sigma_{s,n}^{4}}
=\frac{\mathbb{E}[\nu_i^{4}\mathbb{E}[\epsilon_i^{4}\mid Z_i]\mid\mathcal{H}_{n}]}{n_{\star}\sigma_{s,n}^{4}}
\leq\frac{\overline{m}_{Y,4}\mathbb{E}[\nu_i^{4}\mid\mathcal{H}_{n}]}{n_{\star}c_Y^{2}\sigma_{f,n}^{4}}
=o_{\mathbb{P}}(1). n ⋆ σ s , n 4 E [ U ni 4 ∣ H n ] = n ⋆ σ s , n 4 E [ ν i 4 E [ ϵ i 4 ∣ Z i ] ∣ H n ] ≤ n ⋆ c Y 2 σ f , n 4 m Y , 4 E [ ν i 4 ∣ H n ] = o P ( 1 ) . This verifies every part of Theorem 1(i).
Condition (ii): permutation oracle array Put B i k = f ^ ( X k , Z i ) B_{ik}=\hat{f}(X_k,Z_i) B ik = f ^ ( X k , Z i ) B i ⋅ = n ⋆ − 1 ∑ k B i k B_{i\cdot}=n_{\star}^{-1}\sum_k B_{ik} B i ⋅ = n ⋆ − 1 ∑ k B ik
A i k = B i k − B i ⋅ , r i = 1 n ⋆ ∑ k A i k 2 , D n = 1 n ⋆ ∑ i r i , C n , i k = ϵ i A i k . A_{ik}=B_{ik}-B_{i\cdot},\qquad r_i=\frac{1}{n_{\star}}\sum_k A_{ik}^{2},\qquad D_n=\frac{1}{n_{\star}}\sum_i r_i,\qquad C_{n,ik}=\epsilon_i A_{ik}. A ik = B ik − B i ⋅ , r i = n ⋆ 1 k ∑ A ik 2 , D n = n ⋆ 1 i ∑ r i , C n , ik = ϵ i A ik . This array is F n \mathcal{F}_n F n
σ p , n 2 = 1 n ⋆ 2 ∑ i , k C n , i k 2 = 1 n ⋆ ∑ i ϵ i 2 r i . (62) \sigma_{p,n}^{2}=\frac{1}{n_{\star}^{2}}\sum_{i,k}C_{n,ik}^{2}=\frac{1}{n_{\star}}\sum_i\epsilon_i^{2}r_i. \tag*{(62)} σ p , n 2 = n ⋆ 2 1 i , k ∑ C n , ik 2 = n ⋆ 1 i ∑ ϵ i 2 r i . (62) Let H X , n = σ ( H n , ( X i , Z i ) i ∈ D 1 ) \mathcal{H}_{X,n}=\sigma(\mathcal{H}_n,(X_i,Z_i)_{i\in D_1}) H X , n = σ ( H n , ( X i , Z i ) i ∈ D 1 )
D n / σ f , n , p e r m 2 → p 1 , max i r i / σ f , n , p e r m 2 = O P ( 1 ) , max i , k ϵ i 2 A i k 2 n ⋆ σ f , n , p e r m 2 = o P ( 1 ) (63) D_n/\sigma_{f,n,\mathrm{perm}}^{2}\xrightarrow{p}1,\qquad\max_i r_i/\sigma_{f,n,\mathrm{perm}}^{2}=O_{\mathbb{P}}(1),\qquad\frac{\max_{i,k}\epsilon_i^{2}A_{ik}^{2}}{n_{\star}\sigma_{f,n,\mathrm{perm}}^{2}}=o_{\mathbb{P}}(1) \tag*{(63)} D n / σ f , n , perm 2 p 1 , i max r i / σ f , n , perm 2 = O P ( 1 ) , n ⋆ σ f , n , perm 2 max i , k ϵ i 2 A ik 2 = o P ( 1 ) (63) by (19) –(21) . Conditional on H X , n \mathcal{H}_{X,n} H X , n ϵ i \epsilon_i ϵ i E [ ϵ i 2 ∣ H X , n ] = V Y ( Z i ) \mathbb{E}[\epsilon_i^{2}\mid\mathcal{H}_{X,n}]=V_Y(Z_i) E [ ϵ i 2 ∣ H X , n ] = V Y ( Z i )
E [ σ p , n 2 ∣ H X , n ] = 1 n ⋆ ∑ i V Y ( Z i ) r i ∈ [ c Y D n , v ‾ Y D n ] , \mathbb{E}[\sigma_{p,n}^{2}\mid\mathcal{H}_{X,n}]=\frac{1}{n_{\star}}\sum_i V_Y(Z_i)r_i\in[c_YD_n,\overline{v}_YD_n], E [ σ p , n 2 ∣ H X , n ] = n ⋆ 1 i ∑ V Y ( Z i ) r i ∈ [ c Y D n , v Y D n ] , Var ( σ p , n 2 ∣ H X , n ) ≤ 1 n ⋆ 2 ∑ i r i 2 E [ ϵ i 4 ∣ Z i ] ≤ m ‾ Y , 4 n ⋆ ( max i r i ) D n . \operatorname{Var}(\sigma_{p,n}^{2}\mid\mathcal{H}_{X,n})\leq\frac{1}{n_{\star}^{2}}\sum_i r_i^{2}\mathbb{E}[\epsilon_i^{4}\mid Z_i]\leq\frac{\overline{m}_{Y,4}}{n_{\star}}(\max_i r_i)D_n. Var ( σ p , n 2 ∣ H X , n ) ≤ n ⋆ 2 1 i ∑ r i 2 E [ ϵ i 4 ∣ Z i ] ≤ n ⋆ m Y , 4 ( i max r i ) D n . The last bound divided by σ f , n , p e r m 4 \sigma_{f,n,\mathrm{perm}}^{4} σ f , n , perm 4 O P ( n ⋆ − 1 ) O_{\mathbb{P}}(n_{\star}^{-1}) O P ( n ⋆ − 1 ) ( σ p , n 2 − E [ σ p , n 2 ∣ H X , n ] ) / σ f , n , p e r m 2 = o P ( 1 ) (\sigma_{p,n}^{2}-\mathbb{E}[\sigma_{p,n}^{2}\mid\mathcal{H}_{X,n}])/\sigma_{f,n,\mathrm{perm}}^{2}=o_{\mathbb{P}}(1) ( σ p , n 2 − E [ σ p , n 2 ∣ H X , n ]) / σ f , n , perm 2 = o P ( 1 ) (63) , this proves
P ( c Y 4 ≤ σ p , n 2 σ f , n , p e r m 2 ≤ 2 v ‾ Y ) ⟶ 1 , σ f , n , p e r m 2 / σ p , n 2 = O P ( 1 ) , P ( σ p , n 2 > 0 ) ⟶ 1. (64) \mathbb{P}\left(\frac{c_Y}{4}\leq\frac{\sigma_{p,n}^{2}}{\sigma_{f,n,\mathrm{perm}}^{2}}\leq2\overline{v}_Y\right)\longrightarrow1,\qquad\sigma_{f,n,\mathrm{perm}}^{2}/\sigma_{p,n}^{2}=O_{\mathbb{P}}(1),\qquad\mathbb{P}(\sigma_{p,n}^{2}>0)\longrightarrow1. \tag*{(64)} P ( 4 c Y ≤ σ f , n , perm 2 σ p , n 2 ≤ 2 v Y ) ⟶ 1 , σ f , n , perm 2 / σ p , n 2 = O P ( 1 ) , P ( σ p , n 2 > 0 ) ⟶ 1. (64) For C n , ⋅ k = n ⋆ − 1 ∑ i ϵ i A i k C_{n,\cdot k}=n_{\star}^{-1}\sum_i\epsilon_iA_{ik} C n , ⋅ k = n ⋆ − 1 ∑ i ϵ i A ik (60) give
E [ 1 n ⋆ − 1 ∑ k C n , ⋅ k 2 ∣ H X , n ] = 1 n ⋆ 2 ( n ⋆ − 1 ) ∑ i , k V Y ( Z i ) A i k 2 ≤ v ‾ Y D n n ⋆ − 1 . \mathbb{E}\left[\left.\frac{1}{n_{\star}-1}\sum_k C_{n,\cdot k}^{2}\right|\mathcal{H}_{X,n}\right]=\frac{1}{n_{\star}^{2}(n_{\star}-1)}\sum_{i,k}V_Y(Z_i)A_{ik}^{2}\leq\frac{\overline{v}_YD_n}{n_{\star}-1}. E [ n ⋆ − 1 1 k ∑ C n , ⋅ k 2 H X , n ] = n ⋆ 2 ( n ⋆ − 1 ) 1 i , k ∑ V Y ( Z i ) A ik 2 ≤ n ⋆ − 1 v Y D n . After division by σ f , n , p e r m 2 \sigma_{f,n,\mathrm{perm}}^{2} σ f , n , perm 2 o P ( 1 ) o_{\mathbb{P}}(1) o P ( 1 ) (64) imply ∑ k C n , ⋅ k 2 / { ( n ⋆ − 1 ) σ p , n 2 } = o P ( 1 ) \sum_k C_{n,\cdot k}^{2}/\{(n_{\star}-1)\sigma_{p,n}^{2}\}=o_{\mathbb{P}}(1) ∑ k C n , ⋅ k 2 / {( n ⋆ − 1 ) σ p , n 2 } = o P ( 1 )
max i , k C n , i k 2 n ⋆ σ p , n 2 = max i , k ϵ i 2 A i k 2 σ f , n , p e r m 2 n ⋆ σ f , n , p e r m 2 σ p , n 2 = o P ( 1 ) \frac{\max_{i,k}C_{n,ik}^{2}}{n_{\star}\sigma_{p,n}^{2}}=\frac{\max_{i,k}\epsilon_i^{2}A_{ik}^{2}\sigma_{f,n,\mathrm{perm}}^{2}}{n_{\star}\sigma_{f,n,\mathrm{perm}}^{2}\sigma_{p,n}^{2}}=o_{\mathbb{P}}(1) n ⋆ σ p , n 2 max i , k C n , ik 2 = n ⋆ σ f , n , perm 2 σ p , n 2 max i , k ϵ i 2 A ik 2 σ f , n , perm 2 = o P ( 1 ) by (63) and (64) . Thus both requirements in Theorem 1(ii) hold.
Condition (iii): observed fitted scores The score difference is exactly
Δ s , n i = R ^ i P C M − U n i = e f , i ϵ i + ν i e y , i + e f , i e y , i . (65) \Delta_{s,ni}=\widehat{R}^{\mathrm{PCM}}_i-U_{ni}=e_{f,i}\epsilon_i+\nu_i e_{y,i}+e_{f,i}e_{y,i}. \tag*{(65)} Δ s , ni = R i PCM − U ni = e f , i ϵ i + ν i e y , i + e f , i e y , i . (65) Let H Y , n = σ ( H n , ( Y i , Z i ) i ∈ D 1 ) \mathcal{H}_{Y,n} = \sigma(\mathcal{H}_{n},(Y_i,Z_i)_{i\in D_1}) H Y , n = σ ( H n , ( Y i , Z i ) i ∈ D 1 ) § 9 make e f , i e_{f,i} e f , i H X , n \mathcal{H}_{X,n} H X , n e y , i e_{y,i} e y , i H Y , n \mathcal{H}_{Y,n} H Y , n H 0 C I H_0^{\mathrm{CI}} H 0 CI ϵ i \epsilon_i ϵ i H X , n \mathcal{H}_{X,n} H X , n ν i \nu_i ν i H Y , n \mathcal{H}_{Y,n} H Y , n
Var ( ∑ i e f , i ϵ i n ⋆ σ f , n ∣ H X , n ) = E [ ∑ i e f , i 2 ϵ i 2 n ⋆ σ f , n 2 ∣ H X , n ] = 1 n ⋆ σ f , n 2 ∑ i e f , i 2 V Y ( Z i ) ≤ v ˉ Y E n , f P C M σ f , n 2 = o P ( 1 ) , \begin{aligned}
\operatorname{Var}\left(\left.\frac{\sum_i e_{f,i}\epsilon_i}{\sqrt{n_\star}\sigma_{f,n}}\right|\mathcal{H}_{X,n}\right)
&=\mathbb{E}\left[\left.\frac{\sum_i e_{f,i}^{2}\epsilon_i^{2}}{n_\star\sigma_{f,n}^{2}}\right|\mathcal{H}_{X,n}\right]\\
&=\frac{1}{n_\star\sigma_{f,n}^{2}}\sum_i e_{f,i}^{2}V_Y(Z_i)\leq\bar{v}_Y\frac{E_{n,f}^{\mathrm{PCM}}}{\sigma_{f,n}^{2}}=o_{\mathbb{P}}(1),
\end{aligned} Var ( n ⋆ σ f , n ∑ i e f , i ϵ i H X , n ) = E [ n ⋆ σ f , n 2 ∑ i e f , i 2 ϵ i 2 H X , n ] = n ⋆ σ f , n 2 1 i ∑ e f , i 2 V Y ( Z i ) ≤ v ˉ Y σ f , n 2 E n , f PCM = o P ( 1 ) , Var ( ∑ i ν i e y , i n ⋆ σ f , n ∣ H Y , n ) = E [ ∑ i ν i 2 e y , i 2 n ⋆ σ f , n 2 ∣ H Y , n ] = 1 n ⋆ σ f , n 2 ∑ i e y , i 2 E [ ν i 2 ∣ Z i , H n ] ≤ { ess sup z E [ ν i 2 ∣ Z i = z , H n ] σ f , n 2 } E n , y P C M = o P ( 1 ) . (66) \begin{aligned}
\operatorname{Var}\left(\left.\frac{\sum_i \nu_i e_{y,i}}{\sqrt{n_\star}\sigma_{f,n}}\right|\mathcal{H}_{Y,n}\right)
&=\mathbb{E}\left[\left.\frac{\sum_i \nu_i^{2}e_{y,i}^{2}}{n_\star\sigma_{f,n}^{2}}\right|\mathcal{H}_{Y,n}\right]\\
&=\frac{1}{n_\star\sigma_{f,n}^{2}}\sum_i e_{y,i}^{2}\mathbb{E}[\nu_i^{2}\mid Z_i,\mathcal{H}_n]\\
&\leq\left\{\operatorname*{ess\,sup}_{z}\frac{\mathbb{E}[\nu_i^{2}\mid Z_i=z,\mathcal{H}_n]}{\sigma_{f,n}^{2}}\right\}E_{n,y}^{\mathrm{PCM}}=o_{\mathbb{P}}(1). \tag*{(66)}
\end{aligned} Var ( n ⋆ σ f , n ∑ i ν i e y , i H Y , n ) = E [ n ⋆ σ f , n 2 ∑ i ν i 2 e y , i 2 H Y , n ] = n ⋆ σ f , n 2 1 i ∑ e y , i 2 E [ ν i 2 ∣ Z i , H n ] ≤ { z ess sup σ f , n 2 E [ ν i 2 ∣ Z i = z , H n ] } E n , y PCM = o P ( 1 ) . (66) The last bounds use (17) and Assumption 4. Lemma 8(a)–(b) therefore makes both normalized sums and both normalized mean squares o P ( 1 ) o_{\mathbb{P}}(1) o P ( 1 )
∣ n ⋆ − 1 / 2 ∑ i e f , i e y , i ∣ 2 σ f , n 2 ≤ n ⋆ E n , f P C M E n , y P C M σ f , n 2 = o P ( 1 ) , \frac{\left|n_\star^{-1/2}\sum_i e_{f,i}e_{y,i}\right|^2}{\sigma_{f,n}^{2}}
\leq\frac{n_\star E_{n,f}^{\mathrm{PCM}}E_{n,y}^{\mathrm{PCM}}}{\sigma_{f,n}^{2}}=o_{\mathbb{P}}(1), σ f , n 2 n ⋆ − 1/2 ∑ i e f , i e y , i 2 ≤ σ f , n 2 n ⋆ E n , f PCM E n , y PCM = o P ( 1 ) , 1 n ⋆ σ f , n 2 ∑ i e f , i 2 e y , i 2 ≤ n ⋆ E n , f P C M E n , y P C M σ f , n 2 = o P ( 1 ) . (67) \frac{1}{n_\star\sigma_{f,n}^{2}}\sum_i e_{f,i}^{2}e_{y,i}^{2}
\leq\frac{n_\star E_{n,f}^{\mathrm{PCM}}E_{n,y}^{\mathrm{PCM}}}{\sigma_{f,n}^{2}}=o_{\mathbb{P}}(1). \tag*{(67)} n ⋆ σ f , n 2 1 i ∑ e f , i 2 e y , i 2 ≤ σ f , n 2 n ⋆ E n , f PCM E n , y PCM = o P ( 1 ) . (67) Combining (66) and (67) in (65) , the triangle inequality and ( u + v + w ) 2 ≤ 3 ( u 2 + v 2 + w 2 ) (u+v+w)^2\leq3(u^2+v^2+w^2) ( u + v + w ) 2 ≤ 3 ( u 2 + v 2 + w 2 )
∣ ∑ i Δ s , n i ∣ n ⋆ σ f , n ≤ ∣ ∑ i e f , i ϵ i ∣ + ∣ ∑ i ν i e y , i ∣ + ∣ ∑ i e f , i e y , i ∣ n ⋆ σ f , n = o P ( 1 ) , \frac{\left|\sum_i\Delta_{s,ni}\right|}{\sqrt{n_\star}\sigma_{f,n}}
\leq\frac{\left|\sum_i e_{f,i}\epsilon_i\right|+\left|\sum_i\nu_i e_{y,i}\right|+\left|\sum_i e_{f,i}e_{y,i}\right|}{\sqrt{n_\star}\sigma_{f,n}}=o_{\mathbb{P}}(1), n ⋆ σ f , n ∣ ∑ i Δ s , ni ∣ ≤ n ⋆ σ f , n ∣ ∑ i e f , i ϵ i ∣ + ∣ ∑ i ν i e y , i ∣ + ∣ ∑ i e f , i e y , i ∣ = o P ( 1 ) , ∑ i Δ s , n i 2 n ⋆ σ f , n 2 ≤ 3 n ⋆ σ f , n 2 ∑ i ( e f , i 2 ϵ i 2 + ν i 2 e y , i 2 + e f , i 2 e y , i 2 ) = o P ( 1 ) . \frac{\sum_i\Delta_{s,ni}^{2}}{n_\star\sigma_{f,n}^{2}}
\leq\frac{3}{n_\star\sigma_{f,n}^{2}}\sum_i\left(e_{f,i}^{2}\epsilon_i^{2}+\nu_i^{2}e_{y,i}^{2}+e_{f,i}^{2}e_{y,i}^{2}\right)=o_{\mathbb{P}}(1). n ⋆ σ f , n 2 ∑ i Δ s , ni 2 ≤ n ⋆ σ f , n 2 3 i ∑ ( e f , i 2 ϵ i 2 + ν i 2 e y , i 2 + e f , i 2 e y , i 2 ) = o P ( 1 ) . Finally, σ f , n / σ s , n ≤ c Y − 1 / 2 \sigma_{f,n}/\sigma_{s,n}\leq c_Y^{-1/2} σ f , n / σ s , n ≤ c Y − 1/2 σ f , n 2 / σ s , n 2 ≤ c Y − 1 \sigma_{f,n}^{2}/\sigma_{s,n}^{2}\leq c_Y^{-1} σ f , n 2 / σ s , n 2 ≤ c Y − 1 (61) . Multiplying the two displayed bounds by these factors gives
∑ i Δ s , n i n ⋆ σ s , n = o P ( 1 ) , ∑ i Δ s , n i 2 n ⋆ σ s , n 2 = o P ( 1 ) . \frac{\sum_i\Delta_{s,ni}}{\sqrt{n_\star}\sigma_{s,n}}=o_{\mathbb{P}}(1),\qquad\frac{\sum_i\Delta_{s,ni}^{2}}{n_\star\sigma_{s,n}^{2}}=o_{\mathbb{P}}(1). n ⋆ σ s , n ∑ i Δ s , ni = o P ( 1 ) , n ⋆ σ s , n 2 ∑ i Δ s , ni 2 = o P ( 1 ) . These are the requirements for the observed score of Theorem 1(iii).
Condition (iii): permuted fitted scores Set e ~ f , i = B i − μ ^ f ( Π ) ( Z i ) \widetilde{e}_{f,i}=B_i-\widehat{\mu}^{(\Pi)}_f(Z_i) e f , i = B i − μ f ( Π ) ( Z i ) E ~ n , f P C M = n ⋆ − 1 ∑ i e ~ f , i 2 \widetilde{E}_{n,f}^{\mathrm{PCM}}=n_\star^{-1}\sum_i\widetilde{e}_{f,i}^{2} E n , f PCM = n ⋆ − 1 ∑ i e f , i 2
Δ p , n i = R ^ i P C M , ( Π ) − C n , i Π ( i ) = e y , i A I , i Π ( i ) + ϵ i e ~ f , i + e y , i e ~ f , i . (68) \Delta_{p,ni}=\widehat{R}_i^{\mathrm{PCM},(\Pi)}-C_{n,i\Pi(i)}=e_{y,i}A_{I,i\Pi(i)}+\epsilon_i\widetilde{e}_{f,i}+e_{y,i}\widetilde{e}_{f,i}. \tag*{(68)} Δ p , ni = R i PCM , ( Π ) − C n , i Π ( i ) = e y , i A I , i Π ( i ) + ϵ i e f , i + e y , i e f , i . (68) For the first term, apply the final assertion of Lemma 13 with q i = e y , i q_i=e_{y,i} q i = e y , i ( A i k ) (A_{ik}) ( A ik )
Var Π ( ∑ i e y , i A i , Π ( i ) n ⋆ σ f , n , p e r m ∣ F n ) ≤ 1 n ⋆ ( n ⋆ − 1 ) σ f , n , p e r m 2 ∑ i , k e y , i 2 A i k 2 = o P ( 1 ) . (69) \operatorname{Var}_{\Pi}\left(\left.\frac{\sum_i e_{y,i}A_{i,\Pi(i)}}{\sqrt{n_\star}\sigma_{f,n,\mathrm{perm}}}\right|\mathcal{F}_n\right)
\le\frac{1}{n_\star(n_\star-1)\sigma_{f,n,\mathrm{perm}}^2}\sum_{i,k}e_{y,i}^2A_{ik}^2=o_{\mathrm{P}}(1). \tag*{(69)} Var Π ( n ⋆ σ f , n , perm ∑ i e y , i A i , Π ( i ) F n ) ≤ n ⋆ ( n ⋆ − 1 ) σ f , n , perm 2 1 i , k ∑ e y , i 2 A ik 2 = o P ( 1 ) . (69) The last step is (23) , whose right side differs by the factor n ⋆ / ( n ⋆ − 1 ) n_\star/(n_\star-1) n ⋆ / ( n ⋆ − 1 ) G n = σ ( H X , n , Π ) \mathcal{G}_n=\sigma(\mathcal{H}_{X,n},\Pi) G n = σ ( H X , n , Π ) ε ~ f , i \tilde{\varepsilon}_{f,i} ε ~ f , i G n \mathcal{G}_n G n ϵ i \epsilon_i ϵ i
Var ( ∑ i ϵ i ϵ ~ f , i n ⋆ σ f , n , p e r m ∣ G n ) = 1 n ⋆ σ f , n , p e r m 2 ∑ i V Y ( Z i ) ϵ ~ f , i 2 ≤ v ˉ Y E ~ n , f P C M σ f , n , p e r m 2 = o P ( 1 ) . (70) \begin{aligned}
\operatorname{Var}\left(\left.\frac{\sum_i\epsilon_i\tilde{\epsilon}_{f,i}}{\sqrt{n_\star}\sigma_{f,n,\mathrm{perm}}}\right|\mathcal{G}_n\right)
=\frac{1}{n_\star\sigma_{f,n,\mathrm{perm}}^2}\sum_i V_Y(Z_i)\tilde{\epsilon}_{f,i}^2 \\
&\le\frac{\bar{v}_Y\widetilde{E}^{\mathrm{PCM}}_{n,f}}{\sigma_{f,n,\mathrm{perm}}^2}=o_{\mathrm{P}}(1). \tag*{(70)}
\end{aligned} Var ( n ⋆ σ f , n , perm ∑ i ϵ i ϵ ~ f , i G n ) = n ⋆ σ f , n , perm 2 1 i ∑ V Y ( Z i ) ϵ ~ f , i 2 ≤ σ f , n , perm 2 v ˉ Y E n , f PCM = o P ( 1 ) . (70) The last bound is Assumption 5. The third sum is controlled deterministically by its other part:
∣ n ⋆ − 1 / 2 ∑ i e y , i ϵ ~ f , i ∣ 2 σ f , n , p e r m 2 ≤ n ⋆ E n , y P C M E ~ n , f P C M σ f , n , p e r m 2 = o P ( 1 ) (71) \frac{\left|n_\star^{-1/2}\sum_i e_{y,i}\tilde{\epsilon}_{f,i}\right|^2}{\sigma_{f,n,\mathrm{perm}}^2}
\le\frac{n_\star E^{\mathrm{PCM}}_{n,y}\widetilde{E}^{\mathrm{PCM}}_{n,f}}{\sigma_{f,n,\mathrm{perm}}^2}=o_{\mathrm{P}}(1) \tag*{(71)} σ f , n , perm 2 n ⋆ − 1/2 ∑ i e y , i ϵ ~ f , i 2 ≤ σ f , n , perm 2 n ⋆ E n , y PCM E n , f PCM = o P ( 1 ) (71) by the same assumption. Applying Lemma 8(b) to (69) and (70) , and combining with (71) in (68) , gives
∣ ∑ i Δ p , n i ∣ n ⋆ σ f , n , p e r m ≤ ∣ ∑ i e y , i A i , Π ( i ) ∣ + ∣ ∑ i ϵ i ϵ ~ f , i ∣ + ∣ ∑ i e y , i ϵ ~ f , i ∣ n ⋆ σ f , n , p e r m = o P ( 1 ) . \frac{\left|\sum_i\Delta_{p,ni}\right|}{\sqrt{n_\star}\sigma_{f,n,\mathrm{perm}}}
\le\frac{\left|\sum_i e_{y,i}A_{i,\Pi(i)}\right|+\left|\sum_i\epsilon_i\tilde{\epsilon}_{f,i}\right|+\left|\sum_i e_{y,i}\tilde{\epsilon}_{f,i}\right|}{\sqrt{n_\star}\sigma_{f,n,\mathrm{perm}}}=o_{\mathrm{P}}(1). n ⋆ σ f , n , perm ∣ ∑ i Δ p , ni ∣ ≤ n ⋆ σ f , n , perm ∑ i e y , i A i , Π ( i ) + ∣ ∑ i ϵ i ϵ ~ f , i ∣ + ∣ ∑ i e y , i ϵ ~ f , i ∣ = o P ( 1 ) . For the requirement for the squared score, conditional permutation expectation and (23) give
E Π [ 1 n ⋆ σ f , n , p e r m 2 ∑ i e y , i 2 A i , Π ( i ) 2 ∣ F n ] = 1 n ⋆ 2 σ f , n , p e r m 2 ∑ i , k e y , i 2 A i k 2 = o P ( 1 ) . (72) \mathbb{E}_{\Pi}\left[\left.\frac{1}{n_\star\sigma_{f,n,\mathrm{perm}}^2}\sum_i e_{y,i}^2A_{i,\Pi(i)}^2\right|\mathcal{F}_n\right]
=\frac{1}{n_\star^2\sigma_{f,n,\mathrm{perm}}^2}\sum_{i,k}e_{y,i}^2A_{ik}^2=o_{\mathrm{P}}(1). \tag*{(72)} E Π [ n ⋆ σ f , n , perm 2 1 i ∑ e y , i 2 A i , Π ( i ) 2 F n ] = n ⋆ 2 σ f , n , perm 2 1 i , k ∑ e y , i 2 A ik 2 = o P ( 1 ) . (72) Conditional on G n \mathcal{G}_n G n
E [ ∑ i ϵ i 2 ϵ ~ f , i 2 n ⋆ σ f , n , p e r m 2 ∣ G n ] = 1 n ⋆ σ f , n , p e r m 2 ∑ i V Y ( Z i ) ϵ ~ f , i 2 ≤ v ˉ Y E ~ n , f P C M σ f , n , p e r m 2 = o P ( 1 ) . (73) \begin{aligned}
\mathbb{E}\left[\left.\frac{\sum_i\epsilon_i^2\tilde{\epsilon}_{f,i}^2}{n_\star\sigma_{f,n,\mathrm{perm}}^2}\right|\mathcal{G}_n\right]
=\frac{1}{n_\star\sigma_{f,n,\mathrm{perm}}^2}\sum_i V_Y(Z_i)\tilde{\epsilon}_{f,i}^2 \\
&\le\frac{\bar{v}_Y\widetilde{E}^{\mathrm{PCM}}_{n,f}}{\sigma_{f,n,\mathrm{perm}}^2}=o_{\mathrm{P}}(1). \tag*{(73)}
\end{aligned} E [ n ⋆ σ f , n , perm 2 ∑ i ϵ i 2 ϵ ~ f , i 2 G n ] = n ⋆ σ f , n , perm 2 1 i ∑ V Y ( Z i ) ϵ ~ f , i 2 ≤ σ f , n , perm 2 v ˉ Y E n , f PCM = o P ( 1 ) . (73) The remaining square satisfies
1 n ⋆ σ f , n , p e r m 2 ∑ i e y , i 2 ϵ ~ f , i 2 ≤ n ⋆ E n , y P C M E ~ n , f P C M σ f , n , p e r m 2 = o P ( 1 ) . (74) \frac{1}{n_\star\sigma_{f,n,\mathrm{perm}}^2}\sum_i e_{y,i}^2\tilde{\epsilon}_{f,i}^2
\le\frac{n_\star E^{\mathrm{PCM}}_{n,y}\widetilde{E}^{\mathrm{PCM}}_{n,f}}{\sigma_{f,n,\mathrm{perm}}^2}=o_{\mathrm{P}}(1). \tag*{(74)} n ⋆ σ f , n , perm 2 1 i ∑ e y , i 2 ϵ ~ f , i 2 ≤ σ f , n , perm 2 n ⋆ E n , y PCM E n , f PCM = o P ( 1 ) . (74) Lemma 8(a), (72) , (73) , and (74) , together with ( u + v + w ) 2 ≤ 3 ( u 2 + v 2 + w 2 ) (u+v+w)^2 \le3(u^2+v^2+w^2) ( u + v + w ) 2 ≤ 3 ( u 2 + v 2 + w 2 )
∑ i Δ p , n i 2 n ⋆ σ f , n , p e r m 2 ≤ 3 n ⋆ σ f , n , p e r m 2 ∑ i ( e y , i 2 A i , Π ( i ) 2 + e y , i 2 e f , i 2 + e y , i 2 e f , i 2 ) = O P ( 1 ) . \frac{\sum_i \Delta_{p,ni}^2}{n_{\star}\sigma_{f,n,\mathrm{perm}}^2}
\le\frac{3}{n_{\star}\sigma_{f,n,\mathrm{perm}}^2}\sum_i\left(e_{y,i}^2 A_{i,\Pi(i)}^2+e_{y,i}^2e_{f,i}^2+e_{y,i}^2e_{f,i}^2\right)=O_{\mathrm{P}}(1). n ⋆ σ f , n , perm 2 ∑ i Δ p , ni 2 ≤ n ⋆ σ f , n , perm 2 3 i ∑ ( e y , i 2 A i , Π ( i ) 2 + e y , i 2 e f , i 2 + e y , i 2 e f , i 2 ) = O P ( 1 ) . Finally, σ f , n , p e r m 2 / σ p , n 2 = O P ( 1 ) \sigma_{f,n,\mathrm{perm}}^2/\sigma_{p,n}^2=O_{\mathrm{P}}(1) σ f , n , perm 2 / σ p , n 2 = O P ( 1 ) (62) –(64) . Therefore the conversion to the Theorem 1 scale is
∑ i Δ p , n i n ⋆ σ p , n = σ f , n , p e r m σ p , n ∑ i Δ p , n i n ⋆ σ f , n , p e r m = o P ( 1 ) , \frac{\sum_i\Delta_{p,ni}}{\sqrt{n_{\star}}\sigma_{p,n}}
=\frac{\sigma_{f,n,\mathrm{perm}}}{\sigma_{p,n}}\frac{\sum_i\Delta_{p,ni}}{\sqrt{n_{\star}}\sigma_{f,n,\mathrm{perm}}}
=o_{\mathrm{P}}(1), n ⋆ σ p , n ∑ i Δ p , ni = σ p , n σ f , n , perm n ⋆ σ f , n , perm ∑ i Δ p , ni = o P ( 1 ) , ∑ i Δ p , n i 2 n ⋆ σ p , n 2 = σ f , n , p e r m 2 σ p , n 2 ∑ i Δ p , n i 2 n ⋆ σ f , n , p e r m 2 = o P ( 1 ) . \frac{\sum_i\Delta_{p,ni}^2}{n_{\star}\sigma_{p,n}^2}
=\frac{\sigma_{f,n,\mathrm{perm}}^2}{\sigma_{p,n}^2}\frac{\sum_i\Delta_{p,ni}^2}{n_{\star}\sigma_{f,n,\mathrm{perm}}^2}
=o_{\mathrm{P}}(1). n ⋆ σ p , n 2 ∑ i Δ p , ni 2 = σ p , n 2 σ f , n , perm 2 n ⋆ σ f , n , perm 2 ∑ i Δ p , ni 2 = o P ( 1 ) . These verify the two requirements for the permutation score of Theorem 1(iii). All conditions of Theorem 1 are now verified, so its conclusion gives Theorem 5. □
Stability of permutation refitting
This appendix gives sufficient conditions for Assumptions 3 and 5.
Learner setup and complexity measures
Let n ⋆ = n n_{\star}=n n ⋆ = n n ⋆ = ∣ D 1 ∣ n_{\star}=|\mathcal{D}_1| n ⋆ = ∣ D 1 ∣ n ⋆ ≥ 2 n_{\star}\ge2 n ⋆ ≥ 2 1 , … , n ⋆ 1,\ldots,n_{\star} 1 , … , n ⋆ H n \mathcal{H}_n H n λ n \lambda_n λ n F n \mathcal{F}_n F n u = ( u i ) i = 1 n ⋆ \mathbf{u}=(u_i)_{i=1}^{n_{\star}} u = ( u i ) i = 1 n ⋆ Π \Pi Π ϕ : Z → R d \phi:\mathcal{Z}\to\mathbb{R}^d ϕ : Z → R d
Φ = ( ϕ ( Z 1 ) ⊤ , … , ϕ ( Z n ⋆ ) ⊤ ) , P 1 = n ⋆ − 1 11 ⊤ , H = I − P 1 , Φ c = H Φ . \Phi=\left(\phi(Z_1)^{\top},\ldots,\phi(Z_{n_{\star}})^{\top}\right),\qquad P_1=n_{\star}^{-1}\mathbf{1}\mathbf{1}^{\top},\qquad H=I-P_1,\qquad\Phi_c=H\Phi. Φ = ( ϕ ( Z 1 ) ⊤ , … , ϕ ( Z n ⋆ ) ⊤ ) , P 1 = n ⋆ − 1 1 1 ⊤ , H = I − P 1 , Φ c = H Φ. Definition 2 (Linear smoother). A regression rule is a linear smoother if its fitted vector is S u S\mathbf{u} S u F n \mathcal{F}_n F n S S S S 1 = 1 S\mathbf{1}=\mathbf{1} S 1 = 1 C ( S ) = ∥ S ∥ F 2 C(S)=\lVert S\rVert_{\mathrm{F}}^2 C ( S ) = ∥ S ∥ F 2
Now we instantiate the linear smoother definition with common learners.
Ordinary least squares. Let Φ ~ = [ 1 , Φ ] \widetilde{\Phi}=[\mathbf{1},\Phi] Φ = [ 1 , Φ ] d + 1 d+1 d + 1
min α ∈ R , γ ∈ R d ∥ u − α 1 − Φ ~ γ ∥ 2 2 . \min_{\alpha\in\mathbb{R},\,\gamma\in\mathbb{R}^d}\lVert\mathbf{u}-\alpha\mathbf{1}-\widetilde{\Phi}\gamma\rVert_2^2. α ∈ R , γ ∈ R d min ∥ u − α 1 − Φ γ ∥ 2 2 . Its fitted vector is S O L S u S_{\mathrm{OLS}}\mathbf{u} S OLS u S O L S = Φ ~ ( Φ ~ ⊤ Φ ~ ) − 1 Φ ~ ⊤ S_{\mathrm{OLS}}=\widetilde{\Phi}(\widetilde{\Phi}^{\top}\widetilde{\Phi})^{-1}\widetilde{\Phi}^{\top} S OLS = Φ ( Φ ⊤ Φ ) − 1 Φ ⊤
C O L S = C ( S O L S ) = d + 1. C_{\mathrm{OLS}}=C(S_{\mathrm{OLS}})=d+1. C OLS = C ( S OLS ) = d + 1. min α ∈ R , γ ∈ R d { ∥ u − α 1 − Φ γ ∥ 2 2 + λ n ∥ γ ∥ 2 2 } . \min_{\alpha\in\mathbb{R},\,\gamma\in\mathbb{R}^d}\left\{\lVert\mathbf{u}-\alpha\mathbf{1}-\Phi\gamma\rVert_2^2+\lambda_n\lVert\gamma\rVert_2^2\right\}. α ∈ R , γ ∈ R d min { ∥ u − α 1 − Φ γ ∥ 2 2 + λ n ∥ γ ∥ 2 2 } . Its fitted vector is S r i d g e ( λ n ) u S_{\mathrm{ridge}}(\lambda_n)\mathbf{u} S ridge ( λ n ) u S r i d g e ( λ n ) = P 1 + Φ c ( Φ c T Φ c + λ n I d ) − 1 Φ c T S_{\mathrm{ridge}}(\lambda_n)=P_1+\Phi_c(\Phi_c^{\mathsf{T}}\Phi_c+\lambda_n I_d)^{-1}\Phi_c^{\mathsf{T}} S ridge ( λ n ) = P 1 + Φ c ( Φ c T Φ c + λ n I d ) − 1 Φ c T ρ 1 , … , ρ d \rho_1,\ldots,\rho_d ρ 1 , … , ρ d Φ c T Φ c \Phi_c^{\mathsf{T}}\Phi_c Φ c T Φ c
C r i d g e ( λ n ) = C { S r i d g e ( λ n ) } = 1 + ∑ j = 1 d ( ρ j ρ j + λ n ) 2 ≤ d + 1. (75) \mathcal{C}_{\mathrm{ridge}}(\lambda_n)=\mathcal{C}\{S_{\mathrm{ridge}}(\lambda_n)\}=1+\sum_{j=1}^{d}\left(\frac{\rho_j}{\rho_j+\lambda_n}\right)^2\leq d+1. \tag*{(75)} C ridge ( λ n ) = C { S ridge ( λ n )} = 1 + j = 1 ∑ d ( ρ j + λ n ρ j ) 2 ≤ d + 1. (75) Kernel ridge regression. Let k k k Z \mathcal{Z} Z H k \mathcal{H}_k H k K i ℓ = k ( Z i , Z ℓ ) K_{i\ell}=k(Z_i,Z_\ell) K i ℓ = k ( Z i , Z ℓ ) K c = H K H K_c=HKH K c = H K H H H H H = I − n ⋆ − 1 11 T H=I-n_\star^{-1}\mathbf{1}\mathbf{1}^{\mathsf{T}} H = I − n ⋆ − 1 1 1 T λ n > 0 \lambda_n>0 λ n > 0 h = ( h ( Z i ) ) i = 1 n ⋆ \mathbf{h}=(h(Z_i))_{i=1}^{n_\star} h = ( h ( Z i ) ) i = 1 n ⋆
min α ∈ R , h ∈ H k { ∥ u − α 1 − h ∥ 2 2 + λ n ∥ h ∥ H k 2 } . \min_{\alpha\in\mathbb{R},\,\mathbf{h}\in\mathcal{H}_k}\left\{\|\mathbf{u}-\alpha\mathbf{1}-\mathbf{h}\|_2^2+\lambda_n\|\mathbf{h}\|_{\mathcal{H}_k}^2\right\}. α ∈ R , h ∈ H k min { ∥ u − α 1 − h ∥ 2 2 + λ n ∥ h ∥ H k 2 } . The unpenalized intercept and the representer theorem give the fitted vector S K R R ( λ n ) u S_{\mathrm{KRR}}(\lambda_n)\mathbf{u} S KRR ( λ n ) u S K R R ( λ n ) = P 1 + K c ( K c + λ n I ) − 1 H S_{\mathrm{KRR}}(\lambda_n)=P_1+K_c(K_c+\lambda_n I)^{-1}H S KRR ( λ n ) = P 1 + K c ( K c + λ n I ) − 1 H ν 1 , … , ν n ⋆ \nu_1,\ldots,\nu_{n_\star} ν 1 , … , ν n ⋆ K c K_c K c
C K R R ( λ n ) = C { S K R R ( λ n ) } = 1 + ∑ j = 1 n ⋆ ( ν j ν j + λ n ) 2 . (76) \mathcal{C}_{\mathrm{KRR}}(\lambda_n)=\mathcal{C}\{S_{\mathrm{KRR}}(\lambda_n)\}=1+\sum_{j=1}^{n_\star}\left(\frac{\nu_j}{\nu_j+\lambda_n}\right)^2. \tag*{(76)} C KRR ( λ n ) = C { S KRR ( λ n )} = 1 + j = 1 ∑ n ⋆ ( ν j + λ n ν j ) 2 . (76) Beyond linear smoothers: lasso. For λ n > 0 \lambda_n>0 λ n > 0
min α ∈ R , γ ∈ R d { 1 2 n ⋆ ∥ u − α 1 − Φ γ ∥ 2 2 + λ n ∥ γ ∥ 1 } . \min_{\alpha\in\mathbb{R},\,\boldsymbol{\gamma}\in\mathbb{R}^d}\left\{\frac{1}{2n_\star}\|\mathbf{u}-\alpha\mathbf{1}-\Phi\boldsymbol{\gamma}\|_2^2+\lambda_n\|\boldsymbol{\gamma}\|_1\right\}. α ∈ R , γ ∈ R d min { 2 n ⋆ 1 ∥ u − α 1 − Φ γ ∥ 2 2 + λ n ∥ γ ∥ 1 } . Remark 18 (Eliminating the intercept). The unpenalized intercept α \alpha α a n > 0 a_n>0 a n > 0 P n ( γ ) \mathcal{P}_n(\boldsymbol{\gamma}) P n ( γ ) α \alpha α
γ ^ ∈ a r g m i n γ ∈ R d { a n ∥ H u − Φ c γ ∥ 2 2 + P n ( γ ) } . (77) \widehat{\boldsymbol{\gamma}}\in\mathop{\rm argmin}_{\boldsymbol{\gamma}\in\mathbb{R}^d}\left\{a_n\|H\mathbf{u}-\Phi_c\boldsymbol{\gamma}\|_2^2+\mathcal{P}_n(\boldsymbol{\gamma})\right\}. \tag*{(77)} γ ∈ argmin γ ∈ R d { a n ∥ H u − Φ c γ ∥ 2 2 + P n ( γ ) } . (77) Here a n = 1 a_n=1 a n = 1 a n = ( 2 n ⋆ ) − 1 a_n=(2n_\star)^{-1} a n = ( 2 n ⋆ ) − 1 λ n ∥ γ ∥ 2 2 \lambda_n\|\boldsymbol{\gamma}\|_2^2 λ n ∥ γ ∥ 2 2 λ n ∥ γ ∥ 1 \lambda_n\|\boldsymbol{\gamma}\|_1 λ n ∥ γ ∥ 1 α ^ = u ‾ − ϕ ‾ T γ ^ \widehat{\alpha}=\overline{u}-\overline{\boldsymbol{\phi}}^{\mathsf{T}}\widehat{\boldsymbol{\gamma}} α = u − ϕ T γ u ‾ = n ⋆ − 1 ∑ i u i \overline{u}=n_\star^{-1}\sum_i u_i u = n ⋆ − 1 ∑ i u i ϕ ‾ = n ⋆ − 1 ∑ i ϕ ( Z i ) \overline{\boldsymbol{\phi}}=n_\star^{-1}\sum_i\phi(Z_i) ϕ = n ⋆ − 1 ∑ i ϕ ( Z i ) Φ c γ \Phi_c\boldsymbol{\gamma} Φ c γ H h H\mathbf{h} H h h = ( h ( Z i ) ) i = 1 n ⋆ \mathbf{h}=(h(Z_i))_{i=1}^{n_\star} h = ( h ( Z i ) ) i = 1 n ⋆ h ∈ H k \mathbf{h}\in\mathcal{H}_k h ∈ H k a n = 1 a_n=1 a n = 1 λ n ∥ h ∥ H k 2 \lambda_n\|\mathbf{h}\|_{\mathcal{H}_k}^2 λ n ∥ h ∥ H k 2 u ‾ − n ⋆ − 1 ∑ i h ^ ( Z i ) \overline{u}-n_\star^{-1}\sum_i\widehat{h}(Z_i) u − n ⋆ − 1 ∑ i h ( Z i )
Additional notation. For a F n \mathcal{F}_n F n A A A u i = A i , Π ( i ) u_i=A_{i,\Pi(i)} u i = A i , Π ( i ) m i = A i ⋅ = n ⋆ − 1 ∑ k = 1 n ⋆ A i k m_i=A_{i\cdot}=n_\star^{-1}\sum_{k=1}^{n_\star}A_{ik} m i = A i ⋅ = n ⋆ − 1 ∑ k = 1 n ⋆ A ik ϕ c ( Z i ) = ϕ ( Z i ) − ϕ ‾ \phi_c(Z_i)=\phi(Z_i)-\overline{\phi} ϕ c ( Z i ) = ϕ ( Z i ) − ϕ
Δ n ( A ) = ∥ 1 n ⋆ ∑ i ϕ c ( Z i ) ( u i − m i ) ∥ ∞ , v ϕ , max = max ℓ ≤ d 1 n ⋆ ∑ i ϕ c , ℓ ( Z i ) 2 . (78) \Delta_n(A)=\left\|\frac{1}{n_\star}\sum_i\phi_c(Z_i)(u_i-m_i)\right\|_\infty,\qquad v_{\phi,\max}=\max_{\ell\leq d}\frac{1}{n_\star}\sum_i\phi_{c,\ell}(Z_i)^2. \tag*{(78)} Δ n ( A ) = n ⋆ 1 i ∑ ϕ c ( Z i ) ( u i − m i ) ∞ , v ϕ , m a x = ℓ ≤ d max n ⋆ 1 i ∑ ϕ c , ℓ ( Z i ) 2 . (78) For GCM, A i k = X k A_{ik}=X_k A ik = X k A i k = B i k A_{ik}=B_{ik} A ik = B ik
Main stability results
We divide the analysis into two parts: the GCM and the PCM.
GCM
For the GCM, the conditional permutation target is the constant vector X ˉ 1 \bar{X}\mathbf{1} X ˉ 1
Corollary 1 (Explicit GCM stability rates for common smoothers). Assume E [ X 2 ] < ∞ \mathbb{E}[X^2] < \infty E [ X 2 ] < ∞ Then the following results hold.
Learner complexity results. We have E ~ n , x = O P ( C O L S / n ⋆ ) \widetilde{E}_{n,x} = O_{\mathbb{P}}(C_{\mathrm{OLS}}/n_{\star}) E n , x = O P ( C OLS / n ⋆ ) E ~ n , x = O P ( C r i d g e ( λ n ) / n ⋆ ) \widetilde{E}_{n,x} = O_{\mathbb{P}}(C_{\mathrm{ridge}}(\lambda_n)/n_{\star}) E n , x = O P ( C ridge ( λ n ) / n ⋆ ) E ~ n , x = O P ( C K R R ( λ n ) / n ⋆ ) \widetilde{E}_{n,x} = O_{\mathbb{P}}(C_{\mathrm{KRR}}(\lambda_n)/n_{\star}) E n , x = O P ( C KRR ( λ n ) / n ⋆ )
Permutation consistency. GCM permutation consistency holds whenever the corresponding variance complexity C O L S C_{\mathrm{OLS}} C OLS C r i d g e ( λ n ) C_{\mathrm{ridge}}(\lambda_n) C ridge ( λ n ) C K R R ( λ n ) C_{\mathrm{KRR}}(\lambda_n) C KRR ( λ n ) o P ( n ⋆ ) o_{\mathbb{P}}(n_{\star}) o P ( n ⋆ )
Product of permutation errors. The product condition in Assumption 3 holds if, in addition, E n , y E_{n,y} E n , y o P ( 1 ) o_{\mathbb{P}}(1) o P ( 1 )
Theorem 8 (Lasso verifies GCM permutation stability). Use the lasso setup above with u i = X Π ( i ) u_i = X_{\Pi(i)} u i = X Π ( i ) Define
Δ n G C M ≡ Δ n ( A ) = ∥ 1 n ⋆ ∑ i = 1 n ⋆ { ϕ ( Z i ) − ϕ ˉ } { X Π ( i ) − X ˉ } ∥ ∞ . \Delta_n^{\mathrm{GCM}} \equiv\Delta_n(A) =
\left\lVert\frac{1}{n_{\star}}\sum_{i=1}^{n_{\star}}\{\phi(Z_i)-\bar{\phi}\}\{X_{\Pi(i)}-\bar{X}\}\right\rVert_{\infty}. Δ n GCM ≡ Δ n ( A ) = n ⋆ 1 i = 1 ∑ n ⋆ { ϕ ( Z i ) − ϕ ˉ } { X Π ( i ) − X ˉ } ∞ . If
P ( Δ n G C M ≤ λ n / 2 ) ⟶ 1 , (79) \mathbb{P}\left(\Delta_n^{\mathrm{GCM}} \le\lambda_n/2\right) \longrightarrow1, \tag*{(79)} P ( Δ n GCM ≤ λ n /2 ) ⟶ 1 , (79) then
P ( E ~ n , x = 0 ) ⟶ 1. (80) \mathbb{P}\left(\widetilde{E}_{n,x}=0\right) \longrightarrow1. \tag*{(80)} P ( E n , x = 0 ) ⟶ 1. (80) Consequently, E ~ n , x = o P ( 1 ) \widetilde{E}_{n,x}=o_{\mathbb{P}}(1) E n , x = o P ( 1 ) n ⋆ E n , y E ~ n , x = o P ( 1 ) n_{\star}E_{n,y}\widetilde{E}_{n,x}=o_{\mathbb{P}}(1) n ⋆ E n , y E n , x = o P ( 1 ) d = d n ⟶ ∞ d=d_n\longrightarrow\infty d = d n ⟶ ∞ c G > 0 c_G>0 c G > 0
P ( Δ n G C M > t ∣ F n ) ≤ 2 d exp ( − c G n ⋆ t 2 ) , t > 0. (81) \mathbb{P}\left(\Delta_n^{\mathrm{GCM}}>t\mid\mathcal{F}_n\right)\le2d\exp(-c_G n_{\star}t^2), \qquad t>0. \tag*{(81)} P ( Δ n GCM > t ∣ F n ) ≤ 2 d exp ( − c G n ⋆ t 2 ) , t > 0. (81) Then there exists a constant C > 0 C>0 C > 0 λ n = C log d n ⋆ \lambda_n=C\sqrt{\frac{\log d}{n_{\star}}} λ n = C n ⋆ l o g d (79) .
Remark 19 (A heavy tail scenario). If only the weaker second-moment assumptions E [ X 2 ] < ∞ \mathbb{E}[X^2]<\infty E [ X 2 ] < ∞ v ϕ , max = O P ( 1 ) v_{\phi,\max}=O_{\mathbb{P}}(1) v ϕ , m a x = O P ( 1 ) Δ n G C M = O P ( d / n ⋆ ) \Delta_n^{\mathrm{GCM}}=O_{\mathbb{P}}(\sqrt{d/n_{\star}}) Δ n GCM = O P ( d / n ⋆ ) d / n ⋆ = o ( λ n ) \sqrt{d/n_{\star}}=o(\lambda_n) d / n ⋆ = o ( λ n )
Remark 20 (A primitive GCM score condition). Suppose that, with probability tending to one, the centered observations X k − X ˉ X_k-\bar{X} X k − X ˉ ϕ c , ℓ ( Z i ) \phi_{c,\ell}(Z_i) ϕ c , ℓ ( Z i ) K X K_X K X K ϕ K_{\phi} K ϕ (81) with tail constant ( 32 K X 2 K ϕ 2 ) − 1 (32K_X^2K_{\phi}^2)^{-1} ( 32 K X 2 K ϕ 2 ) − 1 d → ∞ d\to\infty d → ∞ C ( log d ) / n ⋆ C\sqrt{(\log d)/n_{\star}} C ( log d ) / n ⋆ C > 8 2 K X K ϕ C>8\sqrt{2}K_XK_{\phi} C > 8 2 K X K ϕ (79) .
PCM
For the PCM, write the finite-sample and population permutation targets as m = ( B i ⋅ ) i ∈ D 1 \mathbf{m}=(B_{i\cdot})_{i\in D_1} m = ( B i ⋅ ) i ∈ D 1 b = ( b p e r m ( Z i ) ) i ∈ D 1 \mathbf{b}=(b^{\mathrm{perm}}(Z_i))_{i\in D_1} b = ( b perm ( Z i ) ) i ∈ D 1
b p e r m ( z ) = α n + ϕ ( z ) ⊤ γ n + r n ( z ) , P Z -a.s. , (82) b^{\mathrm{perm}}(z)=\alpha_n+\phi(z)^\top\gamma_n+r_n(z),\qquad P_Z\text{-a.s.}, \tag*{(82)} b perm ( z ) = α n + ϕ ( z ) ⊤ γ n + r n ( z ) , P Z -a.s. , (82) Write r n = ( r n ( Z i ) ) i ∈ D 1 \mathbf{r}_n=(r_n(Z_i))_{i\in D_1} r n = ( r n ( Z i ) ) i ∈ D 1
b p e r m ( z ) = α n , K R R + h n ( z ) + r n , K R R ( z ) , h n ∈ H k . b^{\mathrm{perm}}(z)=\alpha_{n,\mathrm{KRR}}+h_n(z)+r_{n,\mathrm{KRR}}(z),\qquad h_n\in\mathcal{H}_k. b perm ( z ) = α n , KRR + h n ( z ) + r n , KRR ( z ) , h n ∈ H k . Now we present the explicit PCM stability rates for common smoothers.
Corollary 2 (Explicit PCM stability rates for common smoothers). Assume the conditions of Proposition 5 and σ f , n , p e r m 2 > 0 \sigma^2_{f,n,\mathrm{perm}}>0 σ f , n , perm 2 > 0 Under the target decompositions above, the following results hold.
Learner complexity results. E ~ n , f P C M = O P ( C O L S σ f , n , p e r m 2 + ∑ i r n ( Z i ) 2 ) / n ⋆ \widetilde{E}^{\mathrm{PCM}}_{n,f}=O_{\mathbb{P}}\left(C_{\mathrm{OLS}}\sigma^2_{f,n,\mathrm{perm}}+\sum_i r_n(Z_i)^2\right)/n_\star E n , f PCM = O P ( C OLS σ f , n , perm 2 + ∑ i r n ( Z i ) 2 ) / n ⋆ E ~ n , f P C M = O P ( C r i d g e ( λ n ) σ f , n , p e r m 2 + ∑ i r n ( Z i ) 2 + λ n ∥ γ n ∥ 2 2 ) / n ⋆ \widetilde{E}^{\mathrm{PCM}}_{n,f}=O_{\mathbb{P}}\left(C_{\mathrm{ridge}}(\lambda_n)\sigma^2_{f,n,\mathrm{perm}}+\sum_i r_n(Z_i)^2+\lambda_n\lVert\gamma_n\rVert_2^2\right)/n_\star E n , f PCM = O P ( C ridge ( λ n ) σ f , n , perm 2 + ∑ i r n ( Z i ) 2 + λ n ∥ γ n ∥ 2 2 ) / n ⋆ E ~ n , f P C M = O P ( C K R R ( λ n ) σ f , n , p e r m 2 + ∑ i r n , K R R ( Z i ) 2 + λ n ∥ h n ∥ H k 2 ) / n ⋆ \widetilde{E}^{\mathrm{PCM}}_{n,f}=O_{\mathbb{P}}\left(C_{\mathrm{KRR}}(\lambda_n)\sigma^2_{f,n,\mathrm{perm}}+\sum_i r_{n,\mathrm{KRR}}(Z_i)^2+\lambda_n\lVert h_n\rVert_{\mathcal{H}_k}^2\right)/n_\star E n , f PCM = O P ( C KRR ( λ n ) σ f , n , perm 2 + ∑ i r n , KRR ( Z i ) 2 + λ n ∥ h n ∥ H k 2 ) / n ⋆
Permutation consistency. The first part of Assumption 5 follows whenever the corresponding right-hand side is o P ( σ f , n , p e r m 2 ) o_{\mathbb{P}}(\sigma^2_{f,n,\mathrm{perm}}) o P ( σ f , n , perm 2 )
Product of permutation errors. The condition on the product of the errors follows if the same right-hand side multiplied by n ⋆ E n , y P C M n_\star E^{\mathrm{PCM}}_{n,y} n ⋆ E n , y PCM o P ( σ f , n , p e r m 2 ) o_{\mathbb{P}}(\sigma^2_{f,n,\mathrm{perm}}) o P ( σ f , n , perm 2 )
Now we present the explicit PCM permutation stability result for the lasso.
Theorem 9 (Lasso verifies PCM permutation stability). Use the lasso setup above with u i = f ^ ( X Π ( i ) , Z i ) u_i=\hat{f}(X_{\Pi(i)},Z_i) u i = f ^ ( X Π ( i ) , Z i ) D 1 D_1 D 1 A i k = B i k A_{ik}=B_{ik} A ik = B ik Δ n P C M ≡ Δ n ( A ) \Delta_n^{\mathrm{PCM}}\equiv\Delta_n(A) Δ n PCM ≡ Δ n ( A ) σ f , n , p e r m 2 > 0 \sigma^2_{f,n,\mathrm{perm}}>0 σ f , n , perm 2 > 0 Proposition 5 hold, and (82) holds. If
P ( Δ n P C M ≤ λ n / 2 ) → 1 , (83) \mathbb{P}\left(\Delta_n^{\mathrm{PCM}}\leq\lambda_n/2\right)\to1, \tag*{(83)} P ( Δ n PCM ≤ λ n /2 ) → 1 , (83) then
E ~ n , f P C M σ f , n , p e r m 2 = O P ( ∥ r n ∥ 2 2 n ⋆ σ f , n , p e r m 2 + λ n ∥ γ n ∥ 1 σ f , n , p e r m 2 + 1 n ⋆ ) . (84) \frac{\widetilde{E}^{\mathrm{PCM}}_{n,f}}{\sigma^2_{f,n,\mathrm{perm}}}
=O_{\mathbb{P}}\left(
\frac{\lVert r_n\rVert_2^2}{n_\star\sigma^2_{f,n,\mathrm{perm}}}
+\frac{\lambda_n\lVert\gamma_n\rVert_1}{\sigma^2_{f,n,\mathrm{perm}}}
+\frac{1}{n_\star}
\right). \tag*{(84)} σ f , n , perm 2 E n , f PCM = O P ( n ⋆ σ f , n , perm 2 ∥ r n ∥ 2 2 + σ f , n , perm 2 λ n ∥ γ n ∥ 1 + n ⋆ 1 ) . (84) Consequently, if the right-hand side is o P ( 1 ) o_{\mathbb{P}}(1) o P ( 1 ) the first part of Assumption 5 holds; if its product with n ⋆ E n , y P C M n_\star E^{\mathrm{PCM}}_{n,y} n ⋆ E n , y PCM is also o P ( 1 ) o_{\mathbb{P}}(1) o P ( 1 ) then the second part holds.
The preceding theorem does not require sparsity. When S n ≡ supp ( γ n ) S_n\equiv\operatorname{supp}(\gamma_n) S n ≡ supp ( γ n ) s n ≡ ∣ S n ∣ s_n\equiv|S_n| s n ≡ ∣ S n ∣ d = d n → ∞ d=d_n\to\infty d = d n → ∞ s n ≥ 1 s_n\geq1 s n ≥ 1
κ n 2 ≡ inf δ ≠ 0 s n ∥ Φ c δ ∥ 2 2 n ⋆ ∥ δ S n ∥ 1 2 , ∥ δ S n c ∥ 1 ≤ 4 ∥ δ S n ∥ 1 . (85) \kappa_n^2\equiv\inf_{\delta\neq0}
\frac{s_n\lVert\Phi_c\delta\rVert_2^2}{n_\star\lVert\delta_{S_n}\rVert_1^2},
\qquad\lVert\delta_{S_n^c}\rVert_1\leq4\lVert\delta_{S_n}\rVert_1. \tag*{(85)} κ n 2 ≡ δ = 0 inf n ⋆ ∥ δ S n ∥ 1 2 s n ∥ Φ c δ ∥ 2 2 , ∥ δ S n c ∥ 1 ≤ 4 ∥ δ S n ∥ 1 . (85) Set κ n = 1 \kappa_n=1 κ n = 1 s n = 0 s_n=0 s n = 0 s n / κ n 2 s_n/\kappa_n^2 s n / κ n 2
Corollary 3 (Sparse oracle rate for the PCM lasso). In the setting of Theorem 9, suppose that, for some κ 0 > 0 \kappa_0>0 κ 0 > 0 P ( s n = 0 or κ n ≥ κ 0 ) → 1 \mathbb{P}(s_n=0\text{ or }\kappa_n\geq\kappa_0)\to1 P ( s n = 0 or κ n ≥ κ 0 ) → 1 Suppose also that, for some fixed c 0 > 0 c_0>0 c 0 > 0 with probability tending to one,
P ( Δ n P C M > t σ f , n , p e r m ∣ F n ) ≤ 2 d exp ( − c 0 n ⋆ t 2 ) , t > 0. (86) \mathbb{P}\left(\Delta_n^{\mathrm{PCM}}>t\sigma_{f,n,\mathrm{perm}}\mid\mathcal{F}_n\right)\leq2d\exp(-c_0n_\star t^2),\qquad t>0. \tag*{(86)} P ( Δ n PCM > t σ f , n , perm ∣ F n ) ≤ 2 d exp ( − c 0 n ⋆ t 2 ) , t > 0. (86) Choose
λ n = C σ f , n , p e r m log d n ⋆ , C > 2 c 0 . \lambda_n = C\sigma_{f,n,\mathrm{perm}}\sqrt{\frac{\log d}{n_\star}}, \qquad C > \frac{2}{\sqrt{c_0}}. λ n = C σ f , n , perm n ⋆ log d , C > c 0 2 . Then
E ~ n , f P C M σ f , n , p e r m 2 = O P ( ∥ r n ∥ 2 2 n ⋆ σ f , n , p e r m 2 + s n log d n ⋆ κ n 2 + 1 n ⋆ ) = O P ( ∥ r n ∥ 2 2 n ⋆ σ f , n , p e r m 2 + s n log d n ⋆ + 1 n ⋆ ) . (87) \frac{\widetilde{E}^{\mathrm{PCM}}_{n,f}}{\sigma^2_{f,n,\mathrm{perm}}}
= O_{\mathrm{P}}\left(\frac{\lVert r_n\rVert_2^2}{n_\star\sigma^2_{f,n,\mathrm{perm}}}+\frac{s_n\log d}{n_\star\kappa_n^2}+\frac{1}{n_\star}\right)
= O_{\mathrm{P}}\left(\frac{\lVert r_n\rVert_2^2}{n_\star\sigma^2_{f,n,\mathrm{perm}}}+\frac{s_n\log d}{n_\star}+\frac{1}{n_\star}\right). \tag*{(87)} σ f , n , perm 2 E n , f PCM = O P ( n ⋆ σ f , n , perm 2 ∥ r n ∥ 2 2 + n ⋆ κ n 2 s n log d + n ⋆ 1 ) = O P ( n ⋆ σ f , n , perm 2 ∥ r n ∥ 2 2 + n ⋆ s n log d + n ⋆ 1 ) . (87) Hence the first part of Assumption 5 follows whenever
∥ r n ∥ 2 2 n ⋆ σ f , n , p e r m 2 + s n log d n ⋆ = o P ( 1 ) , \frac{\lVert r_n\rVert_2^2}{n_\star\sigma^2_{f,n,\mathrm{perm}}}+\frac{s_n\log d}{n_\star}=o_{\mathrm{P}}(1), n ⋆ σ f , n , perm 2 ∥ r n ∥ 2 2 + n ⋆ s n log d = o P ( 1 ) , and the condition on the product of the errors of Assumption 5 follows if the left-hand side, plus n ⋆ − 1 n^{-1}_\star n ⋆ − 1 n ⋆ E n , y P C M n_\star E^{\mathrm{PCM}}_{n,y} n ⋆ E n , y PCM o P ( 1 ) o_{\mathrm{P}}(1) o P ( 1 ) r n = 0 r_n=0 r n = 0
s n log d n ⋆ = o P ( 1 ) , E n , y P C M ( s n log d + 1 ) = o P ( 1 ) . \frac{s_n\log d}{n_\star}=o_{\mathrm{P}}(1), \qquad E^{\mathrm{PCM}}_{n,y}(s_n\log d+1)=o_{\mathrm{P}}(1). n ⋆ s n log d = o P ( 1 ) , E n , y PCM ( s n log d + 1 ) = o P ( 1 ) . Remark 21 (A primitive score condition). The normalized score bound (86) follows from the stronger part of Lemma 23. For example, it holds with c 0 = ( 32 K A 2 K ϕ 2 ) − 1 c_0=(32K_A^2K_\phi^2)^{-1} c 0 = ( 32 K A 2 K ϕ 2 ) − 1
max i , k ∣ B i k − B i ⋅ ∣ ≤ K A σ f , n , p e r m , max i , ℓ ∣ ϕ c , ℓ ( Z i ) ∣ ≤ K ϕ \max_{i,k}|B_{ik}-B_{i\cdot}|\le K_A\sigma_{f,n,\mathrm{perm}}, \qquad\max_{i,\ell}|\phi_{c,\ell}(Z_i)|\le K_\phi i , k max ∣ B ik − B i ⋅ ∣ ≤ K A σ f , n , perm , i , ℓ max ∣ ϕ c , ℓ ( Z i ) ∣ ≤ K ϕ for fixed constants K A , K ϕ K_A,K_\phi K A , K ϕ
Proofs for linear smoothers
General bound for linear smoothers
Write m = E [ u ∣ F n ] \mathbf{m}=\mathbb{E}[\mathbf{u}\mid\mathcal{F}_n] m = E [ u ∣ F n ] Ω Π = Cov ( u ∣ F n ) \Omega_\Pi=\operatorname{Cov}(\mathbf{u}\mid\mathcal{F}_n) Ω Π = Cov ( u ∣ F n ) E ~ n = n ⋆ − 1 ∥ S u − m ∥ 2 2 \widetilde{E}_n=n_\star^{-1}\lVert S\mathbf{u}-\mathbf{m}\rVert_2^2 E n = n ⋆ − 1 ∥ S u − m ∥ 2 2
Lemma 20 (Bias–variance principle for permutation refits). For any F n \mathcal{F}_n F n S S S
E [ E ~ n ∣ F n ] = 1 n ⋆ Tr ( S Ω Π S ⊤ ) + 1 n ⋆ ∥ ( I − S ) m ∥ 2 2 . (88) \mathbb{E}[\widetilde{E}_n\mid\mathcal{F}_n]
=\frac{1}{n_\star}\operatorname{Tr}(S\Omega_\Pi S^\top)+\frac{1}{n_\star}\lVert(I-S)\mathbf{m}\rVert_2^2. \tag*{(88)} E [ E n ∣ F n ] = n ⋆ 1 Tr ( S Ω Π S ⊤ ) + n ⋆ 1 ∥( I − S ) m ∥ 2 2 . (88) ≤ ∥ Ω Π ∥ o p ∥ S ∥ F 2 n ⋆ + 1 n ⋆ ∥ ( I − S ) m ∥ 2 2 . (89) \le\frac{\lVert\Omega_\Pi\rVert_{\mathrm{op}}\lVert S\rVert_F^2}{n_\star}+\frac{1}{n_\star}\lVert(I-S)\mathbf{m}\rVert_2^2. \tag*{(89)} ≤ n ⋆ ∥ Ω Π ∥ op ∥ S ∥ F 2 + n ⋆ 1 ∥( I − S ) m ∥ 2 2 . (89) Consequently,
E ~ n = O P ( ∥ Ω Π ∥ o p ∥ S ∥ F 2 + ∥ ( I − S ) m ∥ 2 2 n ⋆ ) . \widetilde{E}_n=O_{\mathrm{P}}\left(\frac{\lVert\Omega_\Pi\rVert_{\mathrm{op}}\lVert S\rVert_F^2+\lVert(I-S)\mathbf{m}\rVert_2^2}{n_\star}\right). E n = O P ( n ⋆ ∥ Ω Π ∥ op ∥ S ∥ F 2 + ∥( I − S ) m ∥ 2 2 ) . Theorem 10 (Permutation stability of linear smoothers for the GCM). Suppose the permuted X X X Z Z Z F n \mathcal{F}_n F n S S S S 1 = 1 S\mathbf{1}=\mathbf{1} S 1 = 1 E [ X 2 ] < ∞ \mathbb{E}[X^2]<\infty E [ X 2 ] < ∞
E ~ n , x = O P ( ∥ S ∥ F 2 n ⋆ ) and n ⋆ E n , y E ~ n , x = O P ( E n , y ∥ S ∥ F 2 ) . \widetilde{E}_{n,x}=O_{\mathrm{P}}\left(\frac{\lVert S\rVert_F^2}{n_\star}\right)
\qquad\text{and}\qquad
n_\star E_{n,y}\widetilde{E}_{n,x}=O_{\mathrm{P}}\left(E_{n,y}\lVert S\rVert_F^2\right). E n , x = O P ( n ⋆ ∥ S ∥ F 2 ) and n ⋆ E n , y E n , x = O P ( E n , y ∥ S ∥ F 2 ) . Consequently, ∥ S ∥ F 2 = o P ( n ⋆ ) \|S\|_{F}^{2}=o_{\mathrm{P}}(n_{\star}) ∥ S ∥ F 2 = o P ( n ⋆ ) E ~ n , x = o P ( 1 ) \widetilde{E}_{n,x}=o_{\mathrm{P}}(1) E n , x = o P ( 1 ) E n , y ∥ S ∥ F 2 = o P ( 1 ) E_{n,y}\|S\|_{F}^{2}=o_{\mathrm{P}}(1) E n , y ∥ S ∥ F 2 = o P ( 1 ) n ⋆ E n , y E ~ n , x = o P ( 1 ) n_{\star}E_{n,y}\widetilde{E}_{n,x}=o_{\mathrm{P}}(1) n ⋆ E n , y E n , x = o P ( 1 )
Theorem 11 (Permutation stability of linear smoothers for the PCM). Suppose the permuted projection regression on D 1 \mathcal{D}_{1} D 1 F n \mathcal{F}_{n} F n S S S P ( σ f , n , p e r m 2 > 0 ) → 1 \mathbb{P}(\sigma^{2}_{f,n,\mathrm{perm}}>0)\to1 P ( σ f , n , perm 2 > 0 ) → 1
max i n ⋆ − 1 ∑ k ∈ D 1 ( B i k − B i ⋅ ) 2 σ f , n , p e r m 2 = O P ( 1 ) \frac{\max_i n_{\star}^{-1}\sum_{k\in\mathcal{D}_{1}}(B_{ik}-B_{i\cdot})^{2}}{\sigma^{2}_{f,n,\mathrm{perm}}}=O_{\mathrm{P}}(1) σ f , n , perm 2 max i n ⋆ − 1 ∑ k ∈ D 1 ( B ik − B i ⋅ ) 2 = O P ( 1 ) holds. Then, on { σ f , n , p e r m 2 > 0 } \{\sigma^{2}_{f,n,\mathrm{perm}}>0\} { σ f , n , perm 2 > 0 }
E ~ n , f P C M σ f , n , p e r m 2 = O P ( ∥ S ∥ F 2 n ⋆ + ∥ ( I − S ) m ∥ 2 2 n ⋆ σ f , n , p e r m 2 ) . (90) \frac{\widetilde{E}^{\mathrm{PCM}}_{n,f}}{\sigma^{2}_{f,n,\mathrm{perm}}}
=O_{\mathrm{P}}\left(\frac{\|S\|_{F}^{2}}{n_{\star}}+\frac{\|(I-S)\mathbf{m}\|_{2}^{2}}{n_{\star}\sigma^{2}_{f,n,\mathrm{perm}}}\right). \tag*{(90)} σ f , n , perm 2 E n , f PCM = O P ( n ⋆ ∥ S ∥ F 2 + n ⋆ σ f , n , perm 2 ∥ ( I − S ) m ∥ 2 2 ) . (90) n ⋆ E n , y P C M E ~ n , f P C M σ f , n , p e r m 2 = O P ( E n , y P C M ∥ S ∥ F 2 + E n , y P C M ∥ ( I − S ) m ∥ 2 2 σ f , n , p e r m 2 ) . (91) \frac{n_{\star}E^{\mathrm{PCM}}_{n,y}\widetilde{E}^{\mathrm{PCM}}_{n,f}}{\sigma^{2}_{f,n,\mathrm{perm}}}
=O_{\mathrm{P}}\left(E^{\mathrm{PCM}}_{n,y}\|S\|_{F}^{2}+\frac{E^{\mathrm{PCM}}_{n,y}\|(I-S)\mathbf{m}\|_{2}^{2}}{\sigma^{2}_{f,n,\mathrm{perm}}}\right). \tag*{(91)} σ f , n , perm 2 n ⋆ E n , y PCM E n , f PCM = O P ( E n , y PCM ∥ S ∥ F 2 + σ f , n , perm 2 E n , y PCM ∥ ( I − S ) m ∥ 2 2 ) . (91) Consequently, ∥ S ∥ F 2 = o P ( n ⋆ ) \|S\|_{F}^{2}=o_{\mathrm{P}}(n_{\star}) ∥ S ∥ F 2 = o P ( n ⋆ )
∥ ( I − S ) m ∥ 2 2 n ⋆ σ f , n , p e r m 2 = o P ( 1 ) \frac{\|(I-S)\mathbf{m}\|_{2}^{2}}{n_{\star}\sigma^{2}_{f,n,\mathrm{perm}}}=o_{\mathrm{P}}(1) n ⋆ σ f , n , perm 2 ∥ ( I − S ) m ∥ 2 2 = o P ( 1 ) imply the first part of Assumption 5. If, in addition,
E n , y P C M ∥ S ∥ F 2 = o P ( 1 ) , E n , y P C M ∥ ( I − S ) m ∥ 2 2 σ f , n , p e r m 2 = o P ( 1 ) , E^{\mathrm{PCM}}_{n,y}\|S\|_{F}^{2}=o_{\mathrm{P}}(1),\qquad
\frac{E^{\mathrm{PCM}}_{n,y}\|(I-S)\mathbf{m}\|_{2}^{2}}{\sigma^{2}_{f,n,\mathrm{perm}}}=o_{\mathrm{P}}(1), E n , y PCM ∥ S ∥ F 2 = o P ( 1 ) , σ f , n , perm 2 E n , y PCM ∥ ( I − S ) m ∥ 2 2 = o P ( 1 ) , then the second part also holds.
Proof of Lemma 20. The decomposition
S u − m = S ( u − m ) − ( I − S ) m S\mathbf{u}-\mathbf{m}=S(\mathbf{u}-\mathbf{m})-(I-S)\mathbf{m} S u − m = S ( u − m ) − ( I − S ) m has conditional cross term zero, since E [ u − m ∣ F n ] = 0 \mathbb{E}[\mathbf{u}-\mathbf{m}\mid\mathcal{F}_{n}]=0 E [ u − m ∣ F n ] = 0
E [ E ~ n ∣ F n ] = 1 n ⋆ E [ ∥ S ( u − m ) ∥ 2 2 ∣ F n ] + ∥ ( I − S ) m ∥ 2 2 n ⋆ = Tr ( S Ω Π S T ) n ⋆ + ∥ ( I − S ) m ∥ 2 2 n ⋆ . \begin{aligned}
\mathbb{E}[\widetilde{E}_{n}\mid\mathcal{F}_{n}]
=\frac{1}{n_{\star}}\mathbb{E}[\|S(\mathbf{u}-\mathbf{m})\|_{2}^{2}\mid\mathcal{F}_{n}]
+\frac{\|(I-S)\mathbf{m}\|_{2}^{2}}{n_{\star}} \\
&=\frac{\operatorname{Tr}(S\Omega_{\Pi}S^{\mathsf{T}})}{n_{\star}}
+\frac{\|(I-S)\mathbf{m}\|_{2}^{2}}{n_{\star}}.
\end{aligned} E [ E n ∣ F n ] = n ⋆ 1 E [ ∥ S ( u − m ) ∥ 2 2 ∣ F n ] + n ⋆ ∥ ( I − S ) m ∥ 2 2 = n ⋆ Tr ( S Ω Π S T ) + n ⋆ ∥ ( I − S ) m ∥ 2 2 . Because 0 ⪯ Ω Π ⪯ ∥ Ω Π ∥ o p I 0\preceq\Omega_{\Pi}\preceq\|\Omega_{\Pi}\|_{\mathrm{op}}I 0 ⪯ Ω Π ⪯ ∥ Ω Π ∥ op I
Tr ( S Ω Π S T ) ≤ ∥ Ω Π ∥ o p Tr ( S S T ) = ∥ Ω Π ∥ o p ∥ S ∥ F 2 . \operatorname{Tr}(S\Omega_{\Pi}S^{\mathsf{T}})
\leq\|\Omega_{\Pi}\|_{\mathrm{op}}\operatorname{Tr}(SS^{\mathsf{T}})
=\|\Omega_{\Pi}\|_{\mathrm{op}}\|S\|_{F}^{2}. Tr ( S Ω Π S T ) ≤ ∥ Ω Π ∥ op Tr ( S S T ) = ∥ Ω Π ∥ op ∥ S ∥ F 2 . This proves (88) –(89) . Conditional Markov gives the stated stochastic order: for any nonnegative F n \mathcal{F}_{n} F n b n b_{n} b n E [ E ~ n ∣ F n ] \mathbb{E}[\widetilde{E}_{n}\mid\mathcal{F}_{n}] E [ E n ∣ F n ] P ( E ~ n > M b n , b n > 0 ) ≤ M − 1 \mathbb{P}(\widetilde{E}_{n}>Mb_{n},b_{n}>0)\leq M^{-1} P ( E n > M b n , b n > 0 ) ≤ M − 1 E ~ n = 0 \widetilde{E}_{n}=0 E n = 0 { b n = 0 } \{b_{n}=0\} { b n = 0 }
Lemma 21 (Covariance of a permutation response generated from an array). Let n ⋆ ≥ 2 n_{\star} \ge2 n ⋆ ≥ 2 ( A i k ) 1 ≤ i , k ≤ n ⋆ (A_{ik})_{1\le i,k\le n_{\star}} ( A ik ) 1 ≤ i , k ≤ n ⋆ F n \mathcal{F}_n F n Π \Pi Π F n \mathcal{F}_n F n
u i ≡ A i , Π ( i ) , A i ⋅ ≡ 1 n ⋆ ∑ k = 1 n ⋆ A i k , v A , max ≡ max i 1 n ⋆ ∑ k = 1 n ⋆ ( A i k − A i ⋅ ) 2 . u_i \equiv A_{i,\Pi(i)}, \qquad A_{i\cdot} \equiv\frac{1}{n_{\star}}\sum_{k=1}^{n_{\star}} A_{ik}, \qquad v_{A,\max} \equiv\max_i \frac{1}{n_{\star}}\sum_{k=1}^{n_{\star}}(A_{ik}-A_{i\cdot})^2. u i ≡ A i , Π ( i ) , A i ⋅ ≡ n ⋆ 1 k = 1 ∑ n ⋆ A ik , v A , m a x ≡ i max n ⋆ 1 k = 1 ∑ n ⋆ ( A ik − A i ⋅ ) 2 . Then m = ( A i ⋅ ) i = 1 n ⋆ \mathbf{m}=(A_{i\cdot})_{i=1}^{n_{\star}} m = ( A i ⋅ ) i = 1 n ⋆
∥ Ω Π ∥ o p ≤ n ⋆ n ⋆ − 1 v A , max . (92) \lVert\Omega_{\Pi}\rVert_{\mathrm{op}} \le\frac{n_{\star}}{n_{\star}-1}v_{A,\max}. \tag*{(92)} ∥ Ω Π ∥ op ≤ n ⋆ − 1 n ⋆ v A , m a x . (92) For GCM, A i k = X k A_{ik}=X_k A ik = X k v A , max = O P ( 1 ) v_{A,\max}=O_{\mathbb{P}}(1) v A , m a x = O P ( 1 ) E [ X 2 ] < ∞ \mathbb{E}[X^2]<\infty E [ X 2 ] < ∞ A i k = B i k A_{ik}=B_{ik} A ik = B ik v A , max / σ f , n , p e r m 2 = O P ( 1 ) v_{A,\max}/\sigma^2_{f,n,\mathrm{perm}}=O_{\mathbb{P}}(1) v A , m a x / σ f , n , perm 2 = O P ( 1 )
Proof. For every q ∈ R n ⋆ q\in\mathbb{R}^{n_{\star}} q ∈ R n ⋆ h i k = q i ( A i k − A i ⋅ ) h_{ik}=q_i(A_{ik}-A_{i\cdot}) h ik = q i ( A ik − A i ⋅ )
q ⊤ Ω Π q = Var ( ∑ i q i A i , Π ( i ) ∣ F n ) q^{\top}\Omega_{\Pi}q=\operatorname{Var}\left(\left.\sum_i q_iA_{i,\Pi(i)}\right|\mathcal{F}_n\right) q ⊤ Ω Π q = Var ( i ∑ q i A i , Π ( i ) F n ) ≤ 1 n ⋆ − 1 ∑ i q i 2 ∑ k ( A i k − A i ⋅ ) 2 \le\frac{1}{n_{\star}-1}\sum_i q_i^2\sum_k(A_{ik}-A_{i\cdot})^2 ≤ n ⋆ − 1 1 i ∑ q i 2 k ∑ ( A ik − A i ⋅ ) 2 ≤ n ⋆ v A , max n ⋆ − 1 ∑ i q i 2 = n ⋆ n ⋆ − 1 v A , max ∥ q ∥ 2 2 . \le\frac{n_{\star}v_{A,\max}}{n_{\star}-1}\sum_i q_i^2=\frac{n_{\star}}{n_{\star}-1}v_{A,\max}\lVert q\rVert_2^2. ≤ n ⋆ − 1 n ⋆ v A , m a x i ∑ q i 2 = n ⋆ − 1 n ⋆ v A , m a x ∥ q ∥ 2 2 . Taking the supremum over ∥ q ∥ 2 = 1 \lVert q\rVert_2=1 ∥ q ∥ 2 = 1 (92) . For GCM, v A , max = n ⋆ − 1 ∑ k ( X k − X ˉ ) 2 = O P ( 1 ) v_{A,\max}=n_{\star}^{-1}\sum_k(X_k-\bar{X})^2=O_{\mathbb{P}}(1) v A , m a x = n ⋆ − 1 ∑ k ( X k − X ˉ ) 2 = O P ( 1 ) (20) .
Proof of Theorem 10. For the GCM, m = X ˉ 1 \mathbf{m}=\bar{X}\mathbf{1} m = X ˉ 1
1 n ⋆ ∥ ( I − S ) m ∥ 2 2 = X ˉ 2 1 n ⋆ ∥ ( I − S ) 1 ∥ 2 2 = 0 \frac{1}{n_{\star}}\lVert(I-S)\mathbf{m}\rVert_2^2=\bar{X}^2\frac{1}{n_{\star}}\lVert(I-S)\mathbf{1}\rVert_2^2=0 n ⋆ 1 ∥( I − S ) m ∥ 2 2 = X ˉ 2 n ⋆ 1 ∥( I − S ) 1 ∥ 2 2 = 0 because S 1 = 1 S\mathbf{1}=\mathbf{1} S 1 = 1 E [ X 2 ] < ∞ \mathbb{E}[X^2]<\infty E [ X 2 ] < ∞ ∥ Ω Π ∥ o p = O P ( 1 ) \lVert\Omega_{\Pi}\rVert_{\mathrm{op}}=O_{\mathbb{P}}(1) ∥ Ω Π ∥ op = O P ( 1 ) Lemma 20 now yields
E ~ n , x = O P ( ∥ S ∥ F 2 n ⋆ ) . \widetilde{E}_{n,x}=O_{\mathbb{P}}\left(\frac{\lVert S\rVert_F^2}{n_{\star}}\right). E n , x = O P ( n ⋆ ∥ S ∥ F 2 ) . Moreover,
n ⋆ E n , y E ~ n , x = O P ( E n , y ∥ S ∥ F 2 ) . n_{\star}E_{n,y}\widetilde{E}_{n,x}=O_{\mathbb{P}}(E_{n,y}\lVert S\rVert_F^2). n ⋆ E n , y E n , x = O P ( E n , y ∥ S ∥ F 2 ) . The two stated sufficient conditions yield the corresponding o P ( 1 ) o_{\mathbb{P}}(1) o P ( 1 )
Proof of Theorem 11. For u i = B i , Π ( i ) u_i=B_{i,\Pi(i)} u i = B i , Π ( i ) m i = B i ⋅ m_i=B_{i\cdot} m i = B i ⋅
∥ Ω Π ∥ o p σ f , n , p e r m 2 ≤ n ⋆ n ⋆ − 1 max i n ⋆ − 1 ∑ k ( B i k − B i ⋅ ) 2 σ f , n , p e r m 2 = O P ( 1 ) . \frac{\lVert\Omega_{\Pi}\rVert_{\mathrm{op}}}{\sigma^2_{f,n,\mathrm{perm}}}
\le\frac{n_{\star}}{n_{\star}-1}\frac{\max_i n_{\star}^{-1}\sum_k(B_{ik}-B_{i\cdot})^2}{\sigma^2_{f,n,\mathrm{perm}}}
=O_{\mathbb{P}}(1). σ f , n , perm 2 ∥ Ω Π ∥ op ≤ n ⋆ − 1 n ⋆ σ f , n , perm 2 max i n ⋆ − 1 ∑ k ( B ik − B i ⋅ ) 2 = O P ( 1 ) . The conditional bound (89) therefore becomes
E [ E ~ n , f P C M σ f , n , p e r m 2 ∣ F n ] ≤ ∥ Ω Π ∥ o p σ f , n , p e r m 2 ∥ S ∥ F 2 n ⋆ + ∥ ( I − S ) m ∥ 2 2 n ⋆ σ f , n , p e r m 2 . \mathbb{E}\left[\left.\frac{\widetilde{E}^{\mathrm{PCM}}_{n,f}}{\sigma^2_{f,n,\mathrm{perm}}}\right|\mathcal{F}_n\right]
\le\frac{\lVert\Omega_{\Pi}\rVert_{\mathrm{op}}}{\sigma^2_{f,n,\mathrm{perm}}}\frac{\lVert S\rVert_F^2}{n_{\star}}+\frac{\lVert(I-S)\mathbf{m}\rVert_2^2}{n_{\star}\sigma^2_{f,n,\mathrm{perm}}}. E [ σ f , n , perm 2 E n , f PCM F n ] ≤ σ f , n , perm 2 ∥ Ω Π ∥ op n ⋆ ∥ S ∥ F 2 + n ⋆ σ f , n , perm 2 ∥( I − S ) m ∥ 2 2 . Conditional Markov, as in Lemma 20 , gives (90) . Multiplying by the F n \mathcal{F}_n F n n ⋆ E n , y P C M n_{\star}E^{\mathrm{PCM}}_{n,y} n ⋆ E n , y PCM (91) and the stated sufficient conditions.
OLS, ridge, and KRR
Lemma 22 (Properties of the three linear smoothers). For each of the OLS, ridge, and KRR matrices in Subsection G.1,
S 1 = 1 , ∥ S ∥ o p = 1 , ∥ I − S ∥ o p ≤ 1. S1 = 1,\qquad\lVert S\rVert_{\mathrm{op}} = 1,\qquad\lVert I - S\rVert_{\mathrm{op}} \le1. S 1 = 1 , ∥ S ∥ op = 1 , ∥ I − S ∥ op ≤ 1. Their variance complexities are the expressions given there.
Proof. OLS. By the assumption of full column rank on Φ ~ \widetilde{\Phi} Φ
S O L S T = S O L S , S O L S 2 = S O L S , S O L S Φ ~ = Φ ~ . S_{\mathrm{OLS}}^{\mathsf{T}} = S_{\mathrm{OLS}},\qquad S_{\mathrm{OLS}}^{2} = S_{\mathrm{OLS}},\qquad S_{\mathrm{OLS}}\widetilde{\Phi} = \widetilde{\Phi}. S OLS T = S OLS , S OLS 2 = S OLS , S OLS Φ = Φ . Thus S O L S S_{\mathrm{OLS}} S OLS col ( Φ ~ ) \operatorname{col}(\widetilde{\Phi}) col ( Φ ) Φ ~ \widetilde{\Phi} Φ 1 1 1
S O L S 1 = 1 , ∥ S O L S ∥ F 2 = Tr ( S O L S T S O L S ) = Tr ( S O L S ) = d + 1. S_{\mathrm{OLS}}1 = 1,\qquad\lVert S_{\mathrm{OLS}}\rVert_{F}^{2} = \operatorname{Tr}(S_{\mathrm{OLS}}^{\mathsf{T}}S_{\mathrm{OLS}}) = \operatorname{Tr}(S_{\mathrm{OLS}}) = d + 1. S OLS 1 = 1 , ∥ S OLS ∥ F 2 = Tr ( S OLS T S OLS ) = Tr ( S OLS ) = d + 1. Ridge. Eliminating the intercept gives the normal equations and fitted vector
( Φ ~ c T Φ ~ c + λ n I d ) γ ^ ( Π ) = Φ ~ c T u , μ ^ ( Π ) = P 1 u + Φ ~ c γ ^ ( Π ) = { P 1 + Φ ~ c ( Φ ~ c T Φ ~ c + λ n I d ) − 1 Φ ~ c T } u . \begin{aligned}
(\widetilde{\Phi}_{c}^{\mathsf{T}}\widetilde{\Phi}_{c}+\lambda_{n}I_{d})\widehat{\gamma}^{(\Pi)} &= \widetilde{\Phi}_{c}^{\mathsf{T}}u,\\
\widehat{\mu}^{(\Pi)} &= P_{1}u+\widetilde{\Phi}_{c}\widehat{\gamma}^{(\Pi)}\\
&= \left\{P_{1}+\widetilde{\Phi}_{c}(\widetilde{\Phi}_{c}^{\mathsf{T}}\widetilde{\Phi}_{c}+\lambda_{n}I_{d})^{-1}\widetilde{\Phi}_{c}^{\mathsf{T}}\right\}u.
\end{aligned} ( Φ c T Φ c + λ n I d ) γ ( Π ) μ ( Π ) = Φ c T u , = P 1 u + Φ c γ ( Π ) = { P 1 + Φ c ( Φ c T Φ c + λ n I d ) − 1 Φ c T } u . Because Φ ~ c T 1 = 0 \widetilde{\Phi}_{c}^{\mathsf{T}}1=0 Φ c T 1 = 0 S r i d g e ( λ n ) 1 = 1 S_{\mathrm{ridge}}(\lambda_{n})1=1 S ridge ( λ n ) 1 = 1 v j v_{j} v j Φ ~ c T Φ ~ c \widetilde{\Phi}_{c}^{\mathsf{T}}\widetilde{\Phi}_{c} Φ c T Φ c ρ j > 0 \rho_{j}>0 ρ j > 0 q j = Φ ~ c v j / ρ j q_{j}=\widetilde{\Phi}_{c}v_{j}/\sqrt{\rho_{j}} q j = Φ c v j / ρ j 1 1 1
S r i d g e ( λ n ) q j = Φ ~ c ( Φ ~ c T Φ ~ c + λ n I d ) − 1 Φ ~ c T Φ ~ c v j ρ j = ρ j ρ j + λ n q j . S_{\mathrm{ridge}}(\lambda_{n})q_{j}=\frac{\widetilde{\Phi}_{c}(\widetilde{\Phi}_{c}^{\mathsf{T}}\widetilde{\Phi}_{c}+\lambda_{n}I_{d})^{-1}\widetilde{\Phi}_{c}^{\mathsf{T}}\widetilde{\Phi}_{c}v_{j}}{\sqrt{\rho_{j}}}=\frac{\rho_{j}}{\rho_{j}+\lambda_{n}}q_{j}. S ridge ( λ n ) q j = ρ j Φ c ( Φ c T Φ c + λ n I d ) − 1 Φ c T Φ c v j = ρ j + λ n ρ j q j . On the orthogonal complement of span ( 1 ) + col ( Φ ~ c ) \operatorname{span}(1)+\operatorname{col}(\widetilde{\Phi}_{c}) span ( 1 ) + col ( Φ c ) ρ j \rho_{j} ρ j (75) :
∥ S r i d g e ( λ n ) ∥ F 2 = 1 + ∑ j = 1 d ( ρ j ρ j + λ n ) 2 ≤ d + 1. \lVert S_{\mathrm{ridge}}(\lambda_{n})\rVert_{F}^{2}=1+\sum_{j=1}^{d}\left(\frac{\rho_{j}}{\rho_{j}+\lambda_{n}}\right)^{2}\le d+1. ∥ S ridge ( λ n ) ∥ F 2 = 1 + j = 1 ∑ d ( ρ j + λ n ρ j ) 2 ≤ d + 1. KRR. The centered kernel matrix satisfies
K c 1 = 0 , H K c = K c H = K c . K_{c}1=0,\qquad HK_{c}=K_{c}H=K_{c}. K c 1 = 0 , H K c = K c H = K c . Consequently S K R R ( λ n ) 1 = 1 S_{\mathrm{KRR}}(\lambda_{n})1=1 S KRR ( λ n ) 1 = 1 K c K_{c} K c n ⋆ − 1 / 2 1 n_{\star}^{-1/2}1 n ⋆ − 1/2 1 q j q_{j} q j H q j = q j Hq_{j}=q_{j} H q j = q j K c q j = ν j q j K_{c}q_{j}=\nu_{j}q_{j} K c q j = ν j q j
S K R R ( λ n ) q j = P 1 q j + K c ( K c + λ n I n ⋆ ) − 1 H q j = ν j ν j + λ n q j . S_{\mathrm{KRR}}(\lambda_{n})q_{j}=P_{1}q_{j}+K_{c}(K_{c}+\lambda_{n}I_{n_{\star}})^{-1}Hq_{j}=\frac{\nu_{j}}{\nu_{j}+\lambda_{n}}q_{j}. S KRR ( λ n ) q j = P 1 q j + K c ( K c + λ n I n ⋆ ) − 1 H q j = ν j + λ n ν j q j . This gives a real orthonormal eigenbasis for the smoother, proving symmetry, and gives
∥ S K R R ( λ n ) ∥ F 2 = 1 + ∑ j = 1 n ⋆ ( ν j ν j + λ n ) 2 , \lVert S_{\mathrm{KRR}}(\lambda_{n})\rVert_{F}^{2}=1+\sum_{j=1}^{n_{\star}}\left(\frac{\nu_{j}}{\nu_{j}+\lambda_{n}}\right)^{2}, ∥ S KRR ( λ n ) ∥ F 2 = 1 + j = 1 ∑ n ⋆ ( ν j + λ n ν j ) 2 , where the summand for the constant direction is zero. This is (76) .
In all three cases, the eigenvalues of S S S [ 0 , 1 ] [0,1] [ 0 , 1 ] 1 1 1
∥ S ∥ o p = 1 , ∥ I − S ∥ o p ≤ 1. \lVert S\rVert_{\mathrm{op}}=1,\qquad\lVert I-S\rVert_{\mathrm{op}}\le1. ∥ S ∥ op = 1 , ∥ I − S ∥ op ≤ 1. Proof of Corollary 1. Apply Theorem 10 using the reproduction of constants and the complexities established in Lemma 22 .
PCM target approximation and explicit rates
We use (22) to transfer the decompositions of the population target to the exact permutation target.
Proposition 6 (Population approximation of the PCM permutation target). Suppose σ f , n , p e r m 2 > 0 \sigma_{f,n,\mathrm{perm}}^{2}>0 σ f , n , perm 2 > 0
∥ ( I − S O L S ) m ∥ 2 2 n ⋆ σ f , n , p e r m 2 ≤ 2 ∑ i ∈ D 1 r n ( Z i ) 2 n ⋆ σ f , n , p e r m 2 + O P ( n ⋆ − 1 ) . (93) \frac{\lVert(I-S_{\mathrm{OLS}})m\rVert_{2}^{2}}{n_{\star}\sigma_{f,n,\mathrm{perm}}^{2}}
\le2\frac{\sum_{i\in D_{1}}r_{n}(Z_{i})^{2}}{n_{\star}\sigma_{f,n,\mathrm{perm}}^{2}}+O_{\mathbb{P}}(n_{\star}^{-1}). \tag*{(93)} n ⋆ σ f , n , perm 2 ∥( I − S OLS ) m ∥ 2 2 ≤ 2 n ⋆ σ f , n , perm 2 ∑ i ∈ D 1 r n ( Z i ) 2 + O P ( n ⋆ − 1 ) . (93) ∥ ( I − S r i d g e ( λ n ) ) m ∥ 2 2 n ⋆ σ f , n , p e r m 2 ≤ 2 ∑ i ∈ D 1 r n ( Z i ) 2 + λ n ∥ γ n ∥ 2 2 n ⋆ σ f , n , p e r m 2 + O P ( n ⋆ − 1 ) . (94) \frac{\lVert(I-S_{\mathrm{ridge}}(\lambda_{n}))m\rVert_{2}^{2}}{n_{\star}\sigma_{f,n,\mathrm{perm}}^{2}}
\le2\frac{\sum_{i\in D_{1}}r_{n}(Z_{i})^{2}+\lambda_{n}\lVert\gamma_{n}\rVert_{2}^{2}}{n_{\star}\sigma_{f,n,\mathrm{perm}}^{2}}+O_{\mathbb{P}}(n_{\star}^{-1}). \tag*{(94)} n ⋆ σ f , n , perm 2 ∥( I − S ridge ( λ n )) m ∥ 2 2 ≤ 2 n ⋆ σ f , n , perm 2 ∑ i ∈ D 1 r n ( Z i ) 2 + λ n ∥ γ n ∥ 2 2 + O P ( n ⋆ − 1 ) . (94) For KRR,
∥ ( I − S K R R ( λ n ) ) m ∥ 2 2 n ⋆ σ f , n , p e r m 2 ≤ 2 ∑ i ∈ D 1 r n , K R R ( Z i ) 2 + λ n ∥ h n ∥ H k 2 n ⋆ σ f , n , p e r m 2 + O P ( n ⋆ − 1 ) . (95) \frac{\lVert(I-S_{\mathrm{KRR}}(\lambda_{n}))m\rVert_{2}^{2}}{n_{\star}\sigma_{f,n,\mathrm{perm}}^{2}}
\le2\frac{\sum_{i\in D_{1}}r_{n,\mathrm{KRR}}(Z_{i})^{2}+\lambda_{n}\lVert h_{n}\rVert_{\mathcal{H}_{k}}^{2}}{n_{\star}\sigma_{f,n,\mathrm{perm}}^{2}}+O_{\mathbb{P}}(n_{\star}^{-1}). \tag*{(95)} n ⋆ σ f , n , perm 2 ∥( I − S KRR ( λ n )) m ∥ 2 2 ≤ 2 n ⋆ σ f , n , perm 2 ∑ i ∈ D 1 r n , KRR ( Z i ) 2 + λ n ∥ h n ∥ H k 2 + O P ( n ⋆ − 1 ) . (95) Proof. The exact and population target vectors satisfy
∥ m − b ∥ 2 2 = ∑ i ∈ D 1 { B i ⋅ − b p e r m ( Z i ) } 2 . \lVert m-b\rVert_{2}^{2}=\sum_{i\in D_{1}}\{B_{i\cdot}-b^{\mathrm{perm}}(Z_{i})\}^{2}. ∥ m − b ∥ 2 2 = i ∈ D 1 ∑ { B i ⋅ − b perm ( Z i ) } 2 . Thus Proposition 5, specifically (22) , gives
∥ m − b ∥ 2 2 n ⋆ σ f , n , p e r m 2 = O P ( n ⋆ − 1 ) . (96) \frac{\lVert m-b\rVert_{2}^{2}}{n_{\star}\sigma_{f,n,\mathrm{perm}}^{2}}=O_{\mathbb{P}}(n_{\star}^{-1}). \tag*{(96)} n ⋆ σ f , n , perm 2 ∥ m − b ∥ 2 2 = O P ( n ⋆ − 1 ) . (96) For any of the three smoothers, Lemma 22 gives ∥ I − S ∥ o p ≤ 1 \lVert I-S\rVert_{\mathrm{op}}\le1 ∥ I − S ∥ op ≤ 1
∥ ( I − S ) m ∥ 2 2 = ∥ ( I − S ) b + ( I − S ) ( m − b ) ∥ 2 2 ≤ 2 ∥ ( I − S ) b ∥ 2 2 + 2 ∥ ( I − S ) ( m − b ) ∥ 2 2 ≤ 2 ∥ ( I − S ) b ∥ 2 2 + 2 ∥ m − b ∥ 2 2 . \begin{aligned}
\lVert(I-S)m\rVert_{2}^{2}
&=\lVert(I-S)b+(I-S)(m-b)\rVert_{2}^{2}\\
&\le2\lVert(I-S)b\rVert_{2}^{2}+2\lVert(I-S)(m-b)\rVert_{2}^{2}\\
&\le2\lVert(I-S)b\rVert_{2}^{2}+2\lVert m-b\rVert_{2}^{2}.
\end{aligned} ∥( I − S ) m ∥ 2 2 = ∥( I − S ) b + ( I − S ) ( m − b ) ∥ 2 2 ≤ 2 ∥( I − S ) b ∥ 2 2 + 2 ∥( I − S ) ( m − b ) ∥ 2 2 ≤ 2 ∥( I − S ) b ∥ 2 2 + 2 ∥ m − b ∥ 2 2 . Dividing by n ⋆ σ f , n , p e r m 2 n_{\star}\sigma_{f,n,\mathrm{perm}}^{2} n ⋆ σ f , n , perm 2
∥ ( I − S ) m ∥ 2 2 n ⋆ σ f , n , p e r m 2 ≤ 2 ∥ ( I − S ) b ∥ 2 2 n ⋆ σ f , n , p e r m 2 + O P ( n ⋆ − 1 ) . (97) \frac{\lVert(I-S)m\rVert_{2}^{2}}{n_{\star}\sigma_{f,n,\mathrm{perm}}^{2}}
\le2\frac{\lVert(I-S)b\rVert_{2}^{2}}{n_{\star}\sigma_{f,n,\mathrm{perm}}^{2}}+O_{\mathbb{P}}(n_{\star}^{-1}). \tag*{(97)} n ⋆ σ f , n , perm 2 ∥( I − S ) m ∥ 2 2 ≤ 2 n ⋆ σ f , n , perm 2 ∥( I − S ) b ∥ 2 2 + O P ( n ⋆ − 1 ) . (97) It remains to bound the residual for the population target for each smoother.
OLS. By (82) , b = α n 1 + Φ γ n + r n \mathbf{b}=\alpha_n\mathbf{1}+\Phi\boldsymbol{\gamma}_n+\mathbf{r}_n b = α n 1 + Φ γ n + r n
( I − S O L S ) b = ( I − S O L S ) r n , (I-S_{\mathrm{OLS}})\mathbf{b}=(I-S_{\mathrm{OLS}})\mathbf{r}_n, ( I − S OLS ) b = ( I − S OLS ) r n , ∥ ( I − S O L S ) b ∥ 2 2 ≤ ∥ r n ∥ 2 2 = ∑ i ∈ D 1 r n ( Z i ) 2 . \left\|(I-S_{\mathrm{OLS}})\mathbf{b}\right\|_2^2\leq\|\mathbf{r}_n\|_2^2=\sum_{i\in\mathcal{D}_1}r_n(Z_i)^2. ∥ ( I − S OLS ) b ∥ 2 2 ≤ ∥ r n ∥ 2 2 = i ∈ D 1 ∑ r n ( Z i ) 2 . Substitution into (97) proves (93) .
Ridge. Let ( α ^ b , γ ^ b ) (\widehat{\alpha}_b,\widehat{\boldsymbol{\gamma}}_b) ( α b , γ b ) b \mathbf{b} b α ^ b 1 + Φ γ ^ b = S r i d g e ( λ n ) b \widehat{\alpha}_b\mathbf{1}+\Phi\widehat{\boldsymbol{\gamma}}_b=S_{\mathrm{ridge}}(\lambda_n)\mathbf{b} α b 1 + Φ γ b = S ridge ( λ n ) b
∥ ( I − S r i d g e ( λ n ) ) b ∥ 2 2 ≤ ∥ b − α ^ b 1 − Φ γ ^ b ∥ 2 2 + λ n ∥ γ ^ b ∥ 2 2 ≤ ∥ b − α n 1 − Φ γ n ∥ 2 2 + λ n ∥ γ n ∥ 2 2 = ∑ i ∈ D 1 r n ( Z i ) 2 + λ n ∥ γ n ∥ 2 2 . \begin{aligned}
\left\|(I-S_{\mathrm{ridge}}(\lambda_n))\mathbf{b}\right\|_2^2\leq\left\|\mathbf{b}-\widehat{\alpha}_b\mathbf{1}-\Phi\widehat{\boldsymbol{\gamma}}_b\right\|_2^2+\lambda_n\left\|\widehat{\boldsymbol{\gamma}}_b\right\|_2^2 \\
&\leq\left\|\mathbf{b}-\alpha_n\mathbf{1}-\Phi\boldsymbol{\gamma}_n\right\|_2^2+\lambda_n\left\|\boldsymbol{\gamma}_n\right\|_2^2 \\
&=\sum_{i\in\mathcal{D}_1}r_n(Z_i)^2+\lambda_n\left\|\boldsymbol{\gamma}_n\right\|_2^2.
\end{aligned} ∥ ( I − S ridge ( λ n )) b ∥ 2 2 ≤ ∥ b − α b 1 − Φ γ b ∥ 2 2 + λ n ∥ γ b ∥ 2 2 ≤ ∥ b − α n 1 − Φ γ n ∥ 2 2 + λ n ∥ γ n ∥ 2 2 = i ∈ D 1 ∑ r n ( Z i ) 2 + λ n ∥ γ n ∥ 2 2 . Substitution into (97) proves (94) .
KRR. Let ( α ^ b , h ^ b ) (\widehat{\alpha}_b,\widehat{h}_b) ( α b , h b ) b \mathbf{b} b h ^ b = ( h ^ b ( Z i ) ) i ∈ D 1 \widehat{\mathbf{h}}_b=(\widehat{h}_b(Z_i))_{i\in\mathcal{D}_1} h b = ( h b ( Z i ) ) i ∈ D 1 α ^ b 1 + h ^ b = S K R R ( λ n ) b \widehat{\alpha}_b\mathbf{1}+\widehat{\mathbf{h}}_b=S_{\mathrm{KRR}}(\lambda_n)\mathbf{b} α b 1 + h b = S KRR ( λ n ) b
∥ ( I − S K R R ( λ n ) ) b ∥ 2 2 ≤ ∥ b − α ^ b 1 − h ^ b ∥ 2 2 + λ n ∥ h ^ b ∥ H k 2 ≤ ∑ i ∈ D 1 { b p e r m ( Z i ) − α n , K R R − h n ( Z i ) } 2 + λ n ∥ h n ∥ H k 2 = ∑ i ∈ D 1 r n , K R R ( Z i ) 2 + λ n ∥ h n ∥ H k 2 . \begin{aligned}
\left\|(I-S_{\mathrm{KRR}}(\lambda_n))\mathbf{b}\right\|_2^2\leq\left\|\mathbf{b}-\widehat{\alpha}_b\mathbf{1}-\widehat{\mathbf{h}}_b\right\|_2^2+\lambda_n\left\|\widehat{h}_b\right\|_{\mathcal{H}_k}^2 \\
&\leq\sum_{i\in\mathcal{D}_1}\left\{b^{\mathrm{perm}}(Z_i)-\alpha_{n,\mathrm{KRR}}-h_n(Z_i)\right\}^2+\lambda_n\left\|h_n\right\|_{\mathcal{H}_k}^2 \\
&=\sum_{i\in\mathcal{D}_1}r_{n,\mathrm{KRR}}(Z_i)^2+\lambda_n\left\|h_n\right\|_{\mathcal{H}_k}^2.
\end{aligned} ∥ ( I − S KRR ( λ n )) b ∥ 2 2 ≤ b − α b 1 − h b 2 2 + λ n h b H k 2 ≤ i ∈ D 1 ∑ { b perm ( Z i ) − α n , KRR − h n ( Z i ) } 2 + λ n ∥ h n ∥ H k 2 = i ∈ D 1 ∑ r n , KRR ( Z i ) 2 + λ n ∥ h n ∥ H k 2 . Substitution into (97) proves (95) .
Proof of Corollary 2. Substitute the three bounds of Proposition 6 and the complexities in Lemma 22 into Theorem 11. Each C ( S ) ≥ 1 C(S)\geq1 C ( S ) ≥ 1 O P ( n ⋆ − 1 ) O_{\mathrm P}(n_\star^{-1}) O P ( n ⋆ − 1 ) C ( S ) / n ⋆ C(S)/n_\star C ( S ) / n ⋆ n ⋆ E n , y P C M n_\star E^{\mathrm{PCM}}_{n,y} n ⋆ E n , y PCM
Proofs for lasso
A generic bound for the permutation score
Lemma 23 (Permutation score bound for lasso). Let ( A i k ) 1 ≤ i , k ≤ n ⋆ (A_{ik})_{1\leq i,k\leq n_\star} ( A ik ) 1 ≤ i , k ≤ n ⋆ v A , max v_{A,\max} v A , m a x Δ n ( A ) \Delta_n(A) Δ n ( A ) v ϕ , max v_{\phi,\max} v ϕ , m a x (78) . Then
E [ Δ n ( A ) 2 ∣ F n ] ≤ d v A , max v ϕ , max n ⋆ − 1 . (98) \mathbb{E}\left[\Delta_n(A)^2\mid\mathcal{F}_n\right]\leq\frac{d\,v_{A,\max}v_{\phi,\max}}{n_\star-1}. \tag*{(98)} E [ Δ n ( A ) 2 ∣ F n ] ≤ n ⋆ − 1 d v A , m a x v ϕ , m a x . (98) Consequently, if v A , max = O P ( 1 ) v_{A,\max}=O_{\mathrm P}(1) v A , m a x = O P ( 1 ) v ϕ , max = O P ( 1 ) v_{\phi,\max}=O_{\mathrm P}(1) v ϕ , m a x = O P ( 1 ) Δ n ( A ) = O P ( d / n ⋆ ) \Delta_n(A)=O_{\mathrm P}(\sqrt{d/n_\star}) Δ n ( A ) = O P ( d / n ⋆ )
If, in addition, for some positive F n \mathcal{F}_n F n M A , n M_{A,n} M A , n M ϕ , n M_{\phi,n} M ϕ , n
max i , k ∣ A i k − A i ⋅ ∣ ≤ M A , n , max i , ℓ ∣ ϕ c , ℓ ( Z i ) ∣ ≤ M ϕ , n , \max_{i,k}|A_{ik}-A_{i\cdot}|\le M_{A,n},\qquad\max_{i,\ell}|\phi_{c,\ell}(Z_i)|\le M_{\phi,n}, i , k max ∣ A ik − A i ⋅ ∣ ≤ M A , n , i , ℓ max ∣ ϕ c , ℓ ( Z i ) ∣ ≤ M ϕ , n , then, conditionally on F n \mathcal{F}_n F n
P { Δ n ( A ) ≥ t ∣ F n } ≤ 2 d exp { − n ⋆ t 2 32 M A , n 2 M ϕ , n 2 } , t > 0. (99) \mathbb{P}\{\Delta_n(A)\ge t\mid\mathcal{F}_n\}\le2d\exp\left\{-\frac{n_{\star}t^2}{32M_{A,n}^2M_{\phi,n}^2}\right\},\qquad t>0. \tag*{(99)} P { Δ n ( A ) ≥ t ∣ F n } ≤ 2 d exp { − 32 M A , n 2 M ϕ , n 2 n ⋆ t 2 } , t > 0. (99) Proof. For each ℓ ≤ d \ell\le d ℓ ≤ d T ℓ = n ⋆ − 1 ∑ i ϕ c , ℓ ( Z i ) ( A i , Π ( i ) − A i ⋅ ) T_\ell=n_{\star}^{-1}\sum_i\phi_{c,\ell}(Z_i)(A_{i,\Pi(i)}-A_{i\cdot}) T ℓ = n ⋆ − 1 ∑ i ϕ c , ℓ ( Z i ) ( A i , Π ( i ) − A i ⋅ ) E [ T ℓ ∣ F n ] = 0 \mathbb{E}[T_\ell\mid\mathcal{F}_n]=0 E [ T ℓ ∣ F n ] = 0
E [ T ℓ 2 ∣ F n ] ≤ ∑ i ϕ c , ℓ ( Z i ) 2 ∑ k ( A i k − A i ⋅ ) 2 n ⋆ 2 ( n ⋆ − 1 ) ≤ v A , max n ⋆ − 1 { 1 n ⋆ ∑ i ϕ c , ℓ ( Z i ) 2 } ≤ v A , max v ϕ , max n ⋆ − 1 . \begin{aligned}
\mathbb{E}[T_\ell^2\mid\mathcal{F}_n]\le\frac{\sum_i\phi_{c,\ell}(Z_i)^2\sum_k(A_{ik}-A_{i\cdot})^2}{n_{\star}^2(n_{\star}-1)} \\
&\le\frac{v_{A,\max}}{n_{\star}-1}\left\{\frac{1}{n_{\star}}\sum_i\phi_{c,\ell}(Z_i)^2\right\}\le\frac{v_{A,\max}v_{\phi,\max}}{n_{\star}-1}.
\end{aligned} E [ T ℓ 2 ∣ F n ] ≤ n ⋆ 2 ( n ⋆ − 1 ) ∑ i ϕ c , ℓ ( Z i ) 2 ∑ k ( A ik − A i ⋅ ) 2 ≤ n ⋆ − 1 v A , m a x { n ⋆ 1 i ∑ ϕ c , ℓ ( Z i ) 2 } ≤ n ⋆ − 1 v A , m a x v ϕ , m a x . Since Δ n ( A ) 2 = max ℓ T ℓ 2 ≤ ∑ ℓ T ℓ 2 \Delta_n(A)^2=\max_\ell T_\ell^2\le\sum_\ell T_\ell^2 Δ n ( A ) 2 = max ℓ T ℓ 2 ≤ ∑ ℓ T ℓ 2 (98) ; conditional Markov then gives the stated O P ( d / n ⋆ ) O_{\mathbb{P}}(\sqrt{d/n_{\star}}) O P ( d / n ⋆ )
It remains to prove (99) . Fix a coordinate ℓ \ell ℓ T ℓ = g ℓ ( Π ) T_\ell=g_\ell(\Pi) T ℓ = g ℓ ( Π ) Π ( 1 ) , … , Π ( n ⋆ ) \Pi(1),\ldots,\Pi(n_{\star}) Π ( 1 ) , … , Π ( n ⋆ )
M r = E [ T ℓ ∣ F n , Π ( 1 ) , … , Π ( r ) ] , r = 0 , … , n ⋆ , \mathcal{M}_r=\mathbb{E}[T_\ell\mid\mathcal{F}_n,\Pi(1),\ldots,\Pi(r)],\qquad r=0,\ldots,n_{\star}, M r = E [ T ℓ ∣ F n , Π ( 1 ) , … , Π ( r )] , r = 0 , … , n ⋆ , be the corresponding Doob martingale. Conditional on the first r − 1 r-1 r − 1 a a a b b b Π ( r ) \Pi(r) Π ( r ) π \pi π π ( r ) = a \pi(r)=a π ( r ) = a π ′ \pi' π ′ π ′ ( r ) = b \pi'(r)=b π ′ ( r ) = b r r r s = π − 1 ( b ) > r s=\pi^{-1}(b)>r s = π − 1 ( b ) > r
g ℓ ( π ) − g ℓ ( π ′ ) = 1 n ⋆ { ϕ c , ℓ ( Z r ) ( A r a − A r b ) + ϕ c , ℓ ( Z s ) ( A s b − A s a ) } , g_\ell(\pi)-g_\ell(\pi')=\frac{1}{n_{\star}}\{\phi_{c,\ell}(Z_r)(A_{ra}-A_{rb})+\phi_{c,\ell}(Z_s)(A_{sb}-A_{sa})\}, g ℓ ( π ) − g ℓ ( π ′ ) = n ⋆ 1 { ϕ c , ℓ ( Z r ) ( A r a − A r b ) + ϕ c , ℓ ( Z s ) ( A s b − A s a )} , ∣ g ℓ ( π ) − g ℓ ( π ′ ) ∣ ≤ M ϕ , n n ⋆ { ∣ A r a − A r ⋅ ∣ + ∣ A r b − A r ⋅ ∣ + ∣ A s b − A s ⋅ ∣ + ∣ A s a − A s ⋅ ∣ } ≤ 4 M A , n M ϕ , n n ⋆ . \begin{aligned}
|g_\ell(\pi)-g_\ell(\pi')|\le\frac{M_{\phi,n}}{n_{\star}}\{|A_{ra}-A_{r\cdot}|+|A_{rb}-A_{r\cdot}|+|A_{sb}-A_{s\cdot}|+|A_{sa}-A_{s\cdot}|\} \\
&\le\frac{4M_{A,n}M_{\phi,n}}{n_{\star}}.
\end{aligned} ∣ g ℓ ( π ) − g ℓ ( π ′ ) ∣ ≤ n ⋆ M ϕ , n { ∣ A r a − A r ⋅ ∣ + ∣ A r b − A r ⋅ ∣ + ∣ A s b − A s ⋅ ∣ + ∣ A s a − A s ⋅ ∣ } ≤ n ⋆ 4 M A , n M ϕ , n . The swap is a bijection between the two sets of completions. Averaging shows that the conditional expectations corresponding to any two admissible values of Π ( r ) \Pi(r) Π ( r ) M r − 1 \mathcal{M}_{r-1} M r − 1
∣ M r − M r − 1 ∣ ≤ 4 M A , n M ϕ , n n ⋆ a.s. conditionally on F n . |\mathcal{M}_r-\mathcal{M}_{r-1}|\le\frac{4M_{A,n}M_{\phi,n}}{n_{\star}}\qquad\text{a.s. conditionally on }\mathcal{F}_n. ∣ M r − M r − 1 ∣ ≤ n ⋆ 4 M A , n M ϕ , n a.s. conditionally on F n . Moreover, M 0 = E [ T ℓ ∣ F n ] = 0 \mathcal{M}_0=\mathbb{E}[T_\ell\mid\mathcal{F}_n]=0 M 0 = E [ T ℓ ∣ F n ] = 0 M n ⋆ = T ℓ \mathcal{M}_{n_{\star}}=T_\ell M n ⋆ = T ℓ
P ( ∣ T ℓ ∣ ≥ t ∣ F n ) ≤ 2 exp { − t 2 2 ∑ r = 1 n ⋆ ( 4 M A , n M ϕ , n / n ⋆ ) 2 } = 2 exp { − n ⋆ t 2 32 M A , n 2 M ϕ , n 2 } . \begin{aligned}
\mathbb{P}(|T_\ell|\ge t\mid\mathcal{F}_n)\le2\exp\left\{-\frac{t^2}{2\sum_{r=1}^{n_{\star}}(4M_{A,n}M_{\phi,n}/n_{\star})^2}\right\} \\
&=2\exp\left\{-\frac{n_{\star}t^2}{32M_{A,n}^2M_{\phi,n}^2}\right\}.
\end{aligned} P ( ∣ T ℓ ∣ ≥ t ∣ F n ) ≤ 2 exp { − 2 ∑ r = 1 n ⋆ ( 4 M A , n M ϕ , n / n ⋆ ) 2 t 2 } = 2 exp { − 32 M A , n 2 M ϕ , n 2 n ⋆ t 2 } . Finally, { Δ n ( A ) ≥ t } ⊆ ⋃ ℓ = 1 d { ∣ T ℓ ∣ ≥ t } \{\Delta_n(A)\ge t\}\subseteq\bigcup_{\ell=1}^d\{|T_\ell|\ge t\} { Δ n ( A ) ≥ t } ⊆ ⋃ ℓ = 1 d { ∣ T ℓ ∣ ≥ t } (99) .
Proof for the GCM
Proof of Theorem 8. Write X ( Π ) = ( X Π ( i ) ) i = 1 n ⋆ X^{(\Pi)} = (X_{\Pi(i)})_{i=1}^{n_\star} X ( Π ) = ( X Π ( i ) ) i = 1 n ⋆ X ˉ \bar{X} X ˉ H X ( Π ) = X ( Π ) − X ˉ 1 HX^{(\Pi)} = X^{(\Pi)} - \bar{X}1 H X ( Π ) = X ( Π ) − X ˉ 1 (77) to lasso and comparing its minimizer with zero gives
∥ H X ( Π ) − Φ c γ ^ ( Π ) ∥ 2 2 2 n ⋆ + λ n ∥ γ ^ ( Π ) ∥ 1 ≤ ∥ H X ( Π ) ∥ 2 2 2 n ⋆ . \frac{\left\|HX^{(\Pi)}-\Phi_c\hat{\gamma}^{(\Pi)}\right\|_2^2}{2n_\star}+\lambda_n\left\|\hat{\gamma}^{(\Pi)}\right\|_1\leq\frac{\left\|HX^{(\Pi)}\right\|_2^2}{2n_\star}. 2 n ⋆ H X ( Π ) − Φ c γ ^ ( Π ) 2 2 + λ n γ ^ ( Π ) 1 ≤ 2 n ⋆ H X ( Π ) 2 2 . Expanding the square and cancelling ∥ H X ( Π ) ∥ 2 2 / ( 2 n ⋆ ) \left\|HX^{(\Pi)}\right\|_2^2/(2n_\star) H X ( Π ) 2 2 / ( 2 n ⋆ )
∥ Φ c γ ^ ( Π ) ∥ 2 2 2 n ⋆ + λ n ∥ γ ^ ( Π ) ∥ 1 ≤ ( H X ( Π ) ) ⊤ Φ c γ ^ ( Π ) n ⋆ ≤ ∥ Φ c ⊤ ( X ( Π ) − X ˉ 1 ) n ⋆ ∥ ∞ ∥ γ ^ ( Π ) ∥ 1 = Δ n G C M ∥ γ ^ ( Π ) ∥ 1 . \begin{aligned}
\frac{\left\|\Phi_c\hat{\gamma}^{(\Pi)}\right\|_2^2}{2n_\star}+\lambda_n\left\|\hat{\gamma}^{(\Pi)}\right\|_1\leq\frac{\left(HX^{(\Pi)}\right)^\top\Phi_c\hat{\gamma}^{(\Pi)}}{n_\star} \\
&\leq\left\|\frac{\Phi_c^\top\left(X^{(\Pi)}-\bar{X}1\right)}{n_\star}\right\|_\infty\left\|\hat{\gamma}^{(\Pi)}\right\|_1 \\
&=\Delta_n^{\mathrm{GCM}}\left\|\hat{\gamma}^{(\Pi)}\right\|_1.
\end{aligned} 2 n ⋆ Φ c γ ^ ( Π ) 2 2 + λ n γ ^ ( Π ) 1 ≤ n ⋆ ( H X ( Π ) ) ⊤ Φ c γ ^ ( Π ) ≤ n ⋆ Φ c ⊤ ( X ( Π ) − X ˉ 1 ) ∞ γ ^ ( Π ) 1 = Δ n GCM γ ^ ( Π ) 1 . where the second inequality is Hölder’s inequality. Thus, on { Δ n G C M ≤ λ n / 2 } \{\Delta_n^{\mathrm{GCM}}\leq\lambda_n/2\} { Δ n GCM ≤ λ n /2 }
0 ≤ ∥ Φ c γ ^ ( Π ) ∥ 2 2 2 n ⋆ + λ n 2 ∥ γ ^ ( Π ) ∥ 1 ≤ 0. 0\leq\frac{\left\|\Phi_c\hat{\gamma}^{(\Pi)}\right\|_2^2}{2n_\star}+\frac{\lambda_n}{2}\left\|\hat{\gamma}^{(\Pi)}\right\|_1\leq0. 0 ≤ 2 n ⋆ Φ c γ ^ ( Π ) 2 2 + 2 λ n γ ^ ( Π ) 1 ≤ 0. Since λ n > 0 \lambda_n>0 λ n > 0 γ ^ ( Π ) = 0 \hat{\gamma}^{(\Pi)}=0 γ ^ ( Π ) = 0 X ˉ 1 \bar{X}1 X ˉ 1 E ~ n , x = 0 \tilde{E}_{n,x}=0 E ~ n , x = 0 (80) ; moreover, for every ε > 0 \varepsilon>0 ε > 0
P ( n ⋆ E n , y E ~ n , x > ε ) ≤ P ( Δ n G C M > λ n / 2 ) ⟶ 0. \mathbb{P}\left(n_\star E_{n,y}\tilde{E}_{n,x}>\varepsilon\right)\leq\mathbb{P}\left(\Delta_n^{\mathrm{GCM}}>\lambda_n/2\right)\longrightarrow0. P ( n ⋆ E n , y E ~ n , x > ε ) ≤ P ( Δ n GCM > λ n /2 ) ⟶ 0. Under the second-moment conditions, Lemma 23 with A i k = X k A_{ik}=X_k A ik = X k Δ n G C M = O P ( d / n ⋆ ) \Delta_n^{\mathrm{GCM}}=O_{\mathbb{P}}\left(\sqrt{d/n_\star}\right) Δ n GCM = O P ( d / n ⋆ ) v A , max = n ⋆ − 1 ∑ k ( X k − X ˉ ) 2 = O P ( 1 ) v_{A,\max}=n_\star^{-1}\sum_k(X_k-\bar{X})^2=O_{\mathbb{P}}(1) v A , m a x = n ⋆ − 1 ∑ k ( X k − X ˉ ) 2 = O P ( 1 ) d / n ⋆ = o ( λ n ) \sqrt{d/n_\star}=o(\lambda_n) d / n ⋆ = o ( λ n ) G n G_n G n (81) holds. Since P ( G n ) → 1 \mathbb{P}(G_n)\to1 P ( G n ) → 1 d → ∞ d\to\infty d → ∞ C > 2 / c G C>2/\sqrt{c_G} C > 2/ c G λ n = C log d / n ⋆ \lambda_n=C\sqrt{\log d/n_\star} λ n = C log d / n ⋆
P ( Δ n G C M > λ n / 2 ) ≤ P ( G n c ) + 2 d 1 − c G C 2 / 4 ⟶ 0. \mathbb{P}\left(\Delta_n^{\mathrm{GCM}}>\lambda_n/2\right)\leq\mathbb{P}(G_n^c)+2d^{1-c_GC^2/4}\longrightarrow0. P ( Δ n GCM > λ n /2 ) ≤ P ( G n c ) + 2 d 1 − c G C 2 /4 ⟶ 0. Proofs for the PCM
Proof of Theorem 9. Centered fit. Write ε ( Π ) = u − m \varepsilon^{(\Pi)}=u-m ε ( Π ) = u − m
m = b + ( m − b ) = α n 1 + Φ γ n + r n + ( m − b ) , (100) m=b+(m-b)=\alpha_n1+\Phi\gamma_n+r_n+(m-b), \tag*{(100)} m = b + ( m − b ) = α n 1 + Φ γ n + r n + ( m − b ) , (100) and (96) gives
∥ m − b ∥ 2 2 n ⋆ σ f , n , p e r m 2 = O P ( n ⋆ − 1 ) . \frac{\left\|m-b\right\|_2^2}{n_\star\sigma_{f,n,\mathrm{perm}}^2}=O_{\mathbb{P}}(n_\star^{-1}). n ⋆ σ f , n , perm 2 ∥ m − b ∥ 2 2 = O P ( n ⋆ − 1 ) . Specializing (77) to lasso, its minimizing property yields
∥ H u − Φ c γ ^ ( Π ) ∥ 2 2 2 n ⋆ + λ n ∥ γ ^ ( Π ) ∥ 1 ≤ ∥ H u − Φ c γ n ∥ 2 2 2 n ⋆ + λ n ∥ γ n ∥ 1 . \frac{\left\|Hu-\Phi_c\hat{\gamma}^{(\Pi)}\right\|_2^2}{2n_\star}+\lambda_n\left\|\hat{\gamma}^{(\Pi)}\right\|_1
\leq\frac{\left\|Hu-\Phi_c\gamma_n\right\|_2^2}{2n_\star}+\lambda_n\left\|\gamma_n\right\|_1. 2 n ⋆ H u − Φ c γ ^ ( Π ) 2 2 + λ n γ ^ ( Π ) 1 ≤ 2 n ⋆ ∥ H u − Φ c γ n ∥ 2 2 + λ n ∥ γ n ∥ 1 . For either coefficient vector γ \gamma γ H 2 = H H^2=H H 2 = H H Φ c = Φ c H\Phi_c=\Phi_c H Φ c = Φ c
∥ H u − Φ c γ ∥ 2 2 = ∥ H ε ( Π ) ∥ 2 2 + ∥ H m − Φ c γ ∥ 2 2 + 2 ( ε ( Π ) ) ⊤ H m − 2 ( ε ( Π ) ) ⊤ Φ c γ . \lVert H\mathbf{u}-\Phi_c\gamma\rVert_2^2=\lVert H\boldsymbol{\varepsilon}^{(\Pi)}\rVert_2^2+\lVert H\mathbf{m}-\Phi_c\gamma\rVert_2^2+2(\boldsymbol{\varepsilon}^{(\Pi)})^\top H\mathbf{m}-2(\boldsymbol{\varepsilon}^{(\Pi)})^\top\Phi_c\gamma. ∥ H u − Φ c γ ∥ 2 2 = ∥ H ε ( Π ) ∥ 2 2 + ∥ H m − Φ c γ ∥ 2 2 + 2 ( ε ( Π ) ) ⊤ H m − 2 ( ε ( Π ) ) ⊤ Φ c γ . The terms ∥ H ε ( Π ) ∥ 2 2 \lVert H\boldsymbol{\varepsilon}^{(\Pi)}\rVert_2^2 ∥ H ε ( Π ) ∥ 2 2 2 ( ε ( Π ) ) ⊤ H m 2(\boldsymbol{\varepsilon}^{(\Pi)})^\top H\mathbf{m} 2 ( ε ( Π ) ) ⊤ H m { Δ n P C M ≤ λ n / 2 } \{\Delta_n^{\mathrm{PCM}}\leq\lambda_n/2\} { Δ n PCM ≤ λ n /2 }
∥ H m − Φ c γ ^ ( Π ) ∥ 2 2 2 n ⋆ + λ n ∥ γ ^ ( Π ) ∥ 1 ≤ ∥ H m − Φ c γ n ∥ 2 2 2 n ⋆ + λ n ∥ γ n ∥ 1 + ( ε ( Π ) ) ⊤ Φ c ( γ ^ ( Π ) − γ n ) n ⋆ ≤ ∥ H m − Φ c γ n ∥ 2 2 2 n ⋆ + λ n ∥ γ n ∥ 1 + λ n 2 ∥ γ ^ ( Π ) − γ n ∥ 1 . (101) \begin{aligned}
\frac{\lVert H\mathbf{m}-\Phi_c\widehat{\gamma}^{(\Pi)}\rVert_2^2}{2n_\star}+\lambda_n\lVert\widehat{\gamma}^{(\Pi)}\rVert_1
\leq\frac{\lVert H\mathbf{m}-\Phi_c\gamma_n\rVert_2^2}{2n_\star}+\lambda_n\lVert\gamma_n\rVert_1+\frac{(\boldsymbol{\varepsilon}^{(\Pi)})^\top\Phi_c(\widehat{\gamma}^{(\Pi)}-\gamma_n)}{n_\star} \\
&\leq\frac{\lVert H\mathbf{m}-\Phi_c\gamma_n\rVert_2^2}{2n_\star}+\lambda_n\lVert\gamma_n\rVert_1+\frac{\lambda_n}{2}\lVert\widehat{\gamma}^{(\Pi)}-\gamma_n\rVert_1. \tag*{(101)}
\end{aligned} 2 n ⋆ ∥ H m − Φ c γ ( Π ) ∥ 2 2 + λ n ∥ γ ( Π ) ∥ 1 ≤ 2 n ⋆ ∥ H m − Φ c γ n ∥ 2 2 + λ n ∥ γ n ∥ 1 + n ⋆ ( ε ( Π ) ) ⊤ Φ c ( γ ( Π ) − γ n ) ≤ 2 n ⋆ ∥ H m − Φ c γ n ∥ 2 2 + λ n ∥ γ n ∥ 1 + 2 λ n ∥ γ ( Π ) − γ n ∥ 1 . (101) Here ∥ Φ c ⊤ ε ( Π ) / n ⋆ ∥ ∞ = Δ n P C M \lVert\Phi_c^\top\boldsymbol{\varepsilon}^{(\Pi)}/n_\star\rVert_\infty=\Delta_n^{\mathrm{PCM}} ∥ Φ c ⊤ ε ( Π ) / n ⋆ ∥ ∞ = Δ n PCM
∥ γ ^ ( Π ) − γ n ∥ 1 ≤ ∥ γ ^ ( Π ) ∥ 1 + ∥ γ n ∥ 1 , H m − Φ c γ n = H r n + H ( m − b ) , \lVert\widehat{\gamma}^{(\Pi)}-\gamma_n\rVert_1\leq\lVert\widehat{\gamma}^{(\Pi)}\rVert_1+\lVert\gamma_n\rVert_1,
\qquad
H\mathbf{m}-\Phi_c\gamma_n=H\mathbf{r}_n+H(\mathbf{m}-\mathbf{b}), ∥ γ ( Π ) − γ n ∥ 1 ≤ ∥ γ ( Π ) ∥ 1 + ∥ γ n ∥ 1 , H m − Φ c γ n = H r n + H ( m − b ) , we obtain
∥ H m − Φ c γ ^ ( Π ) ∥ 2 2 n ⋆ + λ n ∥ γ ^ ( Π ) ∥ 1 ≤ ∥ H m − Φ c γ n ∥ 2 2 n ⋆ + 3 λ n ∥ γ n ∥ 1 ≤ 2 ∥ r n ∥ 2 2 + 2 ∥ m − b ∥ 2 2 n ⋆ + 3 λ n ∥ γ n ∥ 1 . (102) \begin{aligned}
\frac{\lVert H\mathbf{m}-\Phi_c\widehat{\gamma}^{(\Pi)}\rVert_2^2}{n_\star}+\lambda_n\lVert\widehat{\gamma}^{(\Pi)}\rVert_1
\leq\frac{\lVert H\mathbf{m}-\Phi_c\gamma_n\rVert_2^2}{n_\star}+3\lambda_n\lVert\gamma_n\rVert_1 \\
&\leq\frac{2\lVert\mathbf{r}_n\rVert_2^2+2\lVert\mathbf{m}-\mathbf{b}\rVert_2^2}{n_\star}+3\lambda_n\lVert\gamma_n\rVert_1. \tag*{(102)}
\end{aligned} n ⋆ ∥ H m − Φ c γ ( Π ) ∥ 2 2 + λ n ∥ γ ( Π ) ∥ 1 ≤ n ⋆ ∥ H m − Φ c γ n ∥ 2 2 + 3 λ n ∥ γ n ∥ 1 ≤ n ⋆ 2 ∥ r n ∥ 2 2 + 2 ∥ m − b ∥ 2 2 + 3 λ n ∥ γ n ∥ 1 . (102) The last inequality uses ∥ H ∥ o p ≤ 1 \lVert H\rVert_{\mathrm{op}}\leq1 ∥ H ∥ op ≤ 1 ∥ a + b ∥ 2 2 ≤ 2 ∥ a ∥ 2 2 + 2 ∥ b ∥ 2 2 \lVert a+b\rVert_2^2\leq2\lVert a\rVert_2^2+2\lVert b\rVert_2^2 ∥ a + b ∥ 2 2 ≤ 2 ∥ a ∥ 2 2 + 2 ∥ b ∥ 2 2
Intercept and total error. With m ˉ = n ⋆ − 1 1 ⊤ m \bar{m}=n_\star^{-1}\mathbf{1}^{\top}\mathbf{m} m ˉ = n ⋆ − 1 1 ⊤ m
u ˉ 1 + Φ c γ ^ ( Π ) − m = ( u ˉ − m ˉ ) 1 + Φ c γ ^ ( Π ) − H m . \bar{u}\mathbf{1}+\Phi_c\widehat{\gamma}^{(\Pi)}-\mathbf{m}=(\bar{u}-\bar{m})\mathbf{1}+\Phi_c\widehat{\gamma}^{(\Pi)}-H\mathbf{m}. u ˉ 1 + Φ c γ ( Π ) − m = ( u ˉ − m ˉ ) 1 + Φ c γ ( Π ) − H m . The two terms on the right are orthogonal because 1 ⊤ Φ c = 0 \mathbf{1}^{\top}\Phi_c=0 1 ⊤ Φ c = 0 1 ⊤ H = 0 \mathbf{1}^{\top}H=0 1 ⊤ H = 0
E ~ n , f P C M = ( u ˉ − m ˉ ) 2 + ∥ Φ c γ ^ ( Π ) − H m ∥ 2 2 n ⋆ . (103) \widetilde{E}_{n,f}^{\mathrm{PCM}}=(\bar{u}-\bar{m})^2+\frac{\lVert\Phi_c\widehat{\gamma}^{(\Pi)}-H\mathbf{m}\rVert_2^2}{n_\star}. \tag*{(103)} E n , f PCM = ( u ˉ − m ˉ ) 2 + n ⋆ ∥ Φ c γ ( Π ) − H m ∥ 2 2 . (103) By Lemma 21 and (20) , with A = B A=B A = B
E [ ( u ˉ − m ˉ ) 2 σ f , n , p e r m 2 ∣ F n ] = 1 ⊤ Ω Π 1 n ⋆ 2 σ f , n , p e r m 2 ≤ ∥ Ω Π ∥ o p n ⋆ σ f , n , p e r m 2 ≤ v A , max ( n ⋆ − 1 ) σ f , n , p e r m 2 = O P ( n ⋆ − 1 ) . \mathbb{E}\left[\left.\frac{(\bar{u}-\bar{m})^2}{\sigma_{f,n,\mathrm{perm}}^2}\right|\mathcal{F}_n\right]=\frac{\mathbf{1}^{\top}\Omega_\Pi\mathbf{1}}{n_\star^2\sigma_{f,n,\mathrm{perm}}^2}
\leq\frac{\lVert\Omega_\Pi\rVert_{\mathrm{op}}}{n_\star\sigma_{f,n,\mathrm{perm}}^2}
\leq\frac{v_{A,\max}}{(n_\star-1)\sigma_{f,n,\mathrm{perm}}^2}=O_{\mathbb{P}}(n_\star^{-1}). E [ σ f , n , perm 2 ( u ˉ − m ˉ ) 2 F n ] = n ⋆ 2 σ f , n , perm 2 1 ⊤ Ω Π 1 ≤ n ⋆ σ f , n , perm 2 ∥ Ω Π ∥ op ≤ ( n ⋆ − 1 ) σ f , n , perm 2 v A , m a x = O P ( n ⋆ − 1 ) . Conditional Markov gives
( u ˉ − m ˉ ) 2 σ f , n , p e r m 2 = O P ( n ⋆ − 1 ) . (104) \frac{(\bar{u}-\bar{m})^2}{\sigma_{f,n,\mathrm{perm}}^2}=O_{\mathbb{P}}(n_\star^{-1}). \tag*{(104)} σ f , n , perm 2 ( u ˉ − m ˉ ) 2 = O P ( n ⋆ − 1 ) . (104) Combining (102) and (103) , on the event where the score bound holds,
E ~ n , f P C M σ f , n , p e r m 2 ≤ ( u ˉ − m ˉ ) 2 σ f , n , p e r m 2 + 2 ∥ r n ∥ 2 2 + 2 ∥ m − b ∥ 2 2 n ⋆ σ f , n , p e r m 2 + 3 λ n ∥ γ n ∥ 1 σ f , n , p e r m 2 . \frac{\widetilde{E}_{n,f}^{\mathrm{PCM}}}{\sigma_{f,n,\mathrm{perm}}^{2}} \le\frac{(\bar{u}-\bar{m})^{2}}{\sigma_{f,n,\mathrm{perm}}^{2}}+\frac{2\lVert r_n\rVert_{2}^{2}+2\lVert m-b\rVert_{2}^{2}}{n_{\star}\sigma_{f,n,\mathrm{perm}}^{2}}+\frac{3\lambda_n\lVert\gamma_n\rVert_1}{\sigma_{f,n,\mathrm{perm}}^{2}}. σ f , n , perm 2 E n , f PCM ≤ σ f , n , perm 2 ( u ˉ − m ˉ ) 2 + n ⋆ σ f , n , perm 2 2 ∥ r n ∥ 2 2 + 2 ∥ m − b ∥ 2 2 + σ f , n , perm 2 3 λ n ∥ γ n ∥ 1 . This event has probability tending to one; using (96) and (104) therefore proves (84) . □
Proof of Corollary 3. Score domination. Let G n G_n G n (86) holds, so P ( G n ) → 1 \mathbb{P}(G_n)\to1 P ( G n ) → 1 G n G_n G n t = ( C / 2 ) log d / n ⋆ t=(C/2)\sqrt{\log d/n_{\star}} t = ( C /2 ) log d / n ⋆
P ( Δ n P C M > λ n / 2 ∣ F n ) ≤ 2 d exp ( − c 0 C 2 4 log d ) = 2 d 1 − c 0 C 2 / 4 . \mathbb{P}\left(\Delta_n^{\mathrm{PCM}}>\lambda_n/2\mid\mathcal{F}_n\right)\le2d\exp\left(-\frac{c_0C^2}{4}\log d\right)=2d^{1-c_0C^2/4}. P ( Δ n PCM > λ n /2 ∣ F n ) ≤ 2 d exp ( − 4 c 0 C 2 log d ) = 2 d 1 − c 0 C 2 /4 . Since C > 2 / c 0 C>2/\sqrt{c_0} C > 2/ c 0 d → ∞ d\to\infty d → ∞
P ( Δ n P C M > λ n / 2 ) ≤ P ( G n c ) + 2 d 1 − c 0 C 2 / 4 ⟶ 0 , \mathbb{P}\left(\Delta_n^{\mathrm{PCM}}>\lambda_n/2\right)\le\mathbb{P}(G_n^c)+2d^{1-c_0C^2/4}\longrightarrow0, P ( Δ n PCM > λ n /2 ) ≤ P ( G n c ) + 2 d 1 − c 0 C 2 /4 ⟶ 0 , which proves (83) .
Sparse prediction bound. Let
δ ≡ γ ^ ( Π ) − γ n , \delta\equiv\widehat{\gamma}^{(\Pi)}-\gamma_n, δ ≡ γ ( Π ) − γ n , E n ≡ 1 n ⋆ ∥ H m − Φ c γ ^ ( Π ) ∥ 2 2 , \mathcal{E}_n\equiv\frac{1}{n_{\star}}\lVert Hm-\Phi_c\widehat{\gamma}^{(\Pi)}\rVert_2^2, E n ≡ n ⋆ 1 ∥ H m − Φ c γ ( Π ) ∥ 2 2 , R n ≡ 1 n ⋆ ∥ H r n + H ( m − b ) ∥ 2 2 . R_n\equiv\frac{1}{n_{\star}}\lVert Hr_n+H(m-b)\rVert_2^2. R n ≡ n ⋆ 1 ∥ H r n + H ( m − b ) ∥ 2 2 . Equation (100) gives
E n = 1 n ⋆ ∥ H r n + H ( m − b ) − Φ c δ ∥ 2 2 . \mathcal{E}_n=\frac{1}{n_{\star}}\lVert Hr_n+H(m-b)-\Phi_c\delta\rVert_2^2. E n = n ⋆ 1 ∥ H r n + H ( m − b ) − Φ c δ ∥ 2 2 . Also, by (96) ,
R n ≤ 2 ∥ r n ∥ 2 2 + 2 ∥ m − b ∥ 2 2 n ⋆ , R_n\le\frac{2\lVert r_n\rVert_2^2+2\lVert m-b\rVert_2^2}{n_{\star}}, R n ≤ n ⋆ 2 ∥ r n ∥ 2 2 + 2 ∥ m − b ∥ 2 2 , R n σ f , n , p e r m 2 = O P ( ∥ r n ∥ 2 2 n ⋆ σ f , n , p e r m 2 + 1 n ⋆ ) . (105) \frac{R_n}{\sigma_{f,n,\mathrm{perm}}^2}=O_{\mathbb{P}}\left(\frac{\lVert r_n\rVert_2^2}{n_{\star}\sigma_{f,n,\mathrm{perm}}^2}+\frac{1}{n_{\star}}\right). \tag*{(105)} σ f , n , perm 2 R n = O P ( n ⋆ σ f , n , perm 2 ∥ r n ∥ 2 2 + n ⋆ 1 ) . (105) On { Δ n P C M ≤ λ n / 2 } \{\Delta_n^{\mathrm{PCM}}\le\lambda_n/2\} { Δ n PCM ≤ λ n /2 } (101) and the support decomposition
∥ γ n + δ ∥ 1 = ∥ γ n , S n + δ S n ∥ 1 + ∥ δ S n c ∥ 1 ≥ ∥ γ n ∥ 1 − ∥ δ S n ∥ 1 + ∥ δ S n c ∥ 1 . \lVert\gamma_n+\delta\rVert_1=\lVert\gamma_{n,S_n}+\delta_{S_n}\rVert_1+\lVert\delta_{S_n^c}\rVert_1
\ge\lVert\gamma_n\rVert_1-\lVert\delta_{S_n}\rVert_1+\lVert\delta_{S_n^c}\rVert_1. ∥ γ n + δ ∥ 1 = ∥ γ n , S n + δ S n ∥ 1 + ∥ δ S n c ∥ 1 ≥ ∥ γ n ∥ 1 − ∥ δ S n ∥ 1 + ∥ δ S n c ∥ 1 . Multiplying by two and collecting the two parts of ∥ δ ∥ 1 \lVert\delta\rVert_1 ∥ δ ∥ 1
E n + λ n ∥ δ S n c ∥ 1 ≤ R n + 3 λ n ∥ δ S n ∥ 1 . (106) \mathcal{E}_n+\lambda_n\lVert\delta_{S_n^c}\rVert_1\le R_n+3\lambda_n\lVert\delta_{S_n}\rVert_1. \tag*{(106)} E n + λ n ∥ δ S n c ∥ 1 ≤ R n + 3 λ n ∥ δ S n ∥ 1 . (106) If s n = 0 s_n = 0 s n = 0 E n ≤ R n \mathcal{E}_n \le R_n E n ≤ R n δ = 0 \boldsymbol{\delta} = 0 δ = 0 E n = R n \mathcal{E}_n = R_n E n = R n s n ≥ 1 s_n \ge1 s n ≥ 1 δ ≠ 0 \boldsymbol{\delta} \ne0 δ = 0 { κ n ≥ κ 0 } \{\kappa_n \ge\kappa_0\} { κ n ≥ κ 0 } λ n ∥ δ S n ∥ 1 < R n \lambda_n\|\boldsymbol{\delta}_{S_n}\|_1 < R_n λ n ∥ δ S n ∥ 1 < R n (106) gives E n ≤ 4 R n \mathcal{E}_n \le4R_n E n ≤ 4 R n
λ n ∥ δ S n c ∥ 1 ≤ R n + 3 λ n ∥ δ S n ∥ 1 ≤ 4 λ n ∥ δ S n ∥ 1 , \lambda_n\|\boldsymbol{\delta}_{S_n^c}\|_1 \le R_n + 3\lambda_n\|\boldsymbol{\delta}_{S_n}\|_1 \le4\lambda_n\|\boldsymbol{\delta}_{S_n}\|_1, λ n ∥ δ S n c ∥ 1 ≤ R n + 3 λ n ∥ δ S n ∥ 1 ≤ 4 λ n ∥ δ S n ∥ 1 , so δ \boldsymbol{\delta} δ (85) . Compatibility and the triangle inequality give
∥ δ S n ∥ 1 ≤ s n κ n ∥ Φ c δ ∥ 2 n ⋆ ≤ s n κ n ( R n + E n ) . \|\boldsymbol{\delta}_{S_n}\|_1 \le\frac{\sqrt{s_n}}{\kappa_n}\frac{\|\Phi_c\boldsymbol{\delta}\|_2}{\sqrt{n_\star}} \le\frac{\sqrt{s_n}}{\kappa_n}\left(\sqrt{R_n}+\sqrt{\mathcal{E}_n}\right). ∥ δ S n ∥ 1 ≤ κ n s n n ⋆ ∥ Φ c δ ∥ 2 ≤ κ n s n ( R n + E n ) . Substituting into (106) ,
E n ≤ R n + 3 s n λ n κ n ( E n + R n ) . \mathcal{E}_n \le R_n + \frac{3\sqrt{s_n}\lambda_n}{\kappa_n}\left(\sqrt{\mathcal{E}_n}+\sqrt{R_n}\right). E n ≤ R n + κ n 3 s n λ n ( E n + R n ) . The two applications of x y ≤ ( x 2 + y 2 ) / 2 xy \le(x^2+y^2)/2 x y ≤ ( x 2 + y 2 ) /2
3 s n λ n κ n E n ≤ E n 2 + 9 s n λ n 2 2 κ n 2 , \frac{3\sqrt{s_n}\lambda_n}{\kappa_n}\sqrt{\mathcal{E}_n} \le\frac{\mathcal{E}_n}{2}+\frac{9s_n\lambda_n^2}{2\kappa_n^2}, κ n 3 s n λ n E n ≤ 2 E n + 2 κ n 2 9 s n λ n 2 , 3 s n λ n κ n R n ≤ R n 2 + 9 s n λ n 2 2 κ n 2 . \frac{3\sqrt{s_n}\lambda_n}{\kappa_n}\sqrt{R_n} \le\frac{R_n}{2}+\frac{9s_n\lambda_n^2}{2\kappa_n^2}. κ n 3 s n λ n R n ≤ 2 R n + 2 κ n 2 9 s n λ n 2 . Consequently,
E n ≤ 3 2 R n + 1 2 E n + 9 s n λ n 2 κ n 2 , E n ≤ 3 R n + 18 s n λ n 2 κ n 2 . \mathcal{E}_n \le\frac{3}{2}R_n+\frac{1}{2}\mathcal{E}_n+\frac{9s_n\lambda_n^2}{\kappa_n^2}, \qquad\mathcal{E}_n \le3R_n+\frac{18s_n\lambda_n^2}{\kappa_n^2}. E n ≤ 2 3 R n + 2 1 E n + κ n 2 9 s n λ n 2 , E n ≤ 3 R n + κ n 2 18 s n λ n 2 . Combining all cases gives, with probability tending to one,
E n ≤ 4 R n + 18 s n λ n 2 κ n 2 . \mathcal{E}_n \le4R_n+\frac{18s_n\lambda_n^2}{\kappa_n^2}. E n ≤ 4 R n + κ n 2 18 s n λ n 2 . Using (105) , the exact decomposition (103) , and (104) ,
E ~ n , f P C M σ f , n , p e r m 2 = O P ( ∥ r n ∥ 2 2 n ⋆ σ f , n , p e r m 2 + s n λ n 2 κ n 2 σ f , n , p e r m 2 + 1 n ⋆ ) . \frac{\widetilde{\mathcal{E}}^{\mathrm{PCM}}_{n,f}}{\sigma^2_{f,n,\mathrm{perm}}}
=O_{\mathrm{P}}\left(\frac{\|\mathbf{r}_n\|_2^2}{n_\star\sigma^2_{f,n,\mathrm{perm}}}+\frac{s_n\lambda_n^2}{\kappa_n^2\sigma^2_{f,n,\mathrm{perm}}}+\frac{1}{n_\star}\right). σ f , n , perm 2 E n , f PCM = O P ( n ⋆ σ f , n , perm 2 ∥ r n ∥ 2 2 + κ n 2 σ f , n , perm 2 s n λ n 2 + n ⋆ 1 ) . Substitution of λ n 2 = C 2 σ f , n , p e r m 2 log d / n ⋆ \lambda_n^2=C^2\sigma^2_{f,n,\mathrm{perm}}\log d/n_\star λ n 2 = C 2 σ f , n , perm 2 log d / n ⋆ (87) ; P ( s n = 0 or κ n ≥ κ 0 ) → 1 \mathbb{P}(s_n=0\text{ or }\kappa_n\ge\kappa_0)\to1 P ( s n = 0 or κ n ≥ κ 0 ) → 1
Local power under permutation calibration
Fix α ∈ ( 0 , 1 ) \alpha\in(0,1) α ∈ ( 0 , 1 )
A common lemma for transferring critical values
Lemma 24 (Transfer of critical values under contiguity). Let P 0 ( n ) P_0^{(n)} P 0 ( n ) P n ( n ) P_n^{(n)} P n ( n ) P n ( n ) P_n^{(n)} P n ( n ) P 0 ( n ) P_0^{(n)} P 0 ( n ) c n c_n c n c ∈ R c \in\mathbb{R} c ∈ R
c n → P 0 ( n ) c , c_n \xrightarrow{P_0^{(n)}} c, c n P 0 ( n ) c , then also c n → P n ( n ) c c_n \xrightarrow{P_n^{(n)}} c c n P n ( n ) c T n → d T T_n \xrightarrow{d} T T n d T P n ( n ) P_n^{(n)} P n ( n ) P ( T = c ) = 0 \mathbb{P}(T=c)=0 P ( T = c ) = 0
P n ( n ) ( T n > c n ) ⟶ P ( T > c ) . P_n^{(n)}(T_n>c_n) \longrightarrow\mathbb{P}(T>c). P n ( n ) ( T n > c n ) ⟶ P ( T > c ) . Proof. For every ε > 0 \varepsilon>0 ε > 0
P 0 ( n ) ( ∣ c n − c ∣ > ε ) → 0 ⟹ P n ( n ) ( ∣ c n − c ∣ > ε ) → 0. P_0^{(n)}(|c_n-c|>\varepsilon)\to0 \quad\Longrightarrow\quad P_n^{(n)}(|c_n-c|>\varepsilon)\to0. P 0 ( n ) ( ∣ c n − c ∣ > ε ) → 0 ⟹ P n ( n ) ( ∣ c n − c ∣ > ε ) → 0. Hence, under P n ( n ) P_n^{(n)} P n ( n ) T n − c n → d T − c T_n-c_n\xrightarrow{d}T-c T n − c n d T − c P ( T − c = 0 ) = 0 \mathbb{P}(T-c=0)=0 P ( T − c = 0 ) = 0
P n ( n ) ( T n > c n ) = P n ( n ) ( T n − c n > 0 ) ⟶ P ( T − c > 0 ) . P_n^{(n)}(T_n>c_n)=P_n^{(n)}(T_n-c_n>0)\longrightarrow\mathbb{P}(T-c>0). P n ( n ) ( T n > c n ) = P n ( n ) ( T n − c n > 0 ) ⟶ P ( T − c > 0 ) . Local power of the GLM score with permutation calibration
Fix a null parameter ( 0 , β 0 , ϑ 0 ) (0,\beta_0,\vartheta_0) ( 0 , β 0 , ϑ 0 ) § 3 . Let P 0 P_0 P 0 P 0 ( n ) = P 0 ⊗ n P_0^{(n)}=P_0^{\otimes n} P 0 ( n ) = P 0 ⊗ n E 0 \mathbb{E}_0 E 0 P 0 P_0 P 0 μ 0 ( Z ) ≡ g − 1 ( Z ⊤ β 0 ) \mu_0(Z)\equiv g^{-1}(Z^{\top}\beta_0) μ 0 ( Z ) ≡ g − 1 ( Z ⊤ β 0 ) v 0 ≡ V ϑ 0 ( μ 0 ( Z ) ) v_0\equiv V_{\vartheta_0}(\mu_0(Z)) v 0 ≡ V ϑ 0 ( μ 0 ( Z )) g 0 ′ ≡ g ′ ( μ 0 ( Z ) ) g'_0\equiv g'(\mu_0(Z)) g 0 ′ ≡ g ′ ( μ 0 ( Z )) ξ 0 ≡ ( Y − μ 0 ( Z ) ) / ( v 0 g 0 ′ ) \xi_0\equiv(Y-\mu_0(Z))/(v_0g'_0) ξ 0 ≡ ( Y − μ 0 ( Z )) / ( v 0 g 0 ′ ) W 0 ≡ 1 / { v 0 ( g 0 ′ ) 2 } W_0\equiv1/\{v_0(g'_0)^2\} W 0 ≡ 1/ { v 0 ( g 0 ′ ) 2 }
M 0 ≡ E 0 [ W 0 Z Z ⊤ ] , r 0 ≡ E 0 [ W 0 Z X ] , θ 0 ≡ M 0 − 1 r 0 , δ 0 ≡ X − θ 0 ⊤ Z , M_0\equiv\mathbb{E}_0[W_0ZZ^{\top}],\qquad r_0\equiv\mathbb{E}_0[W_0ZX],\qquad\theta_0\equiv M_0^{-1}r_0,\qquad\delta_0\equiv X-\theta_0^{\top}Z, M 0 ≡ E 0 [ W 0 Z Z ⊤ ] , r 0 ≡ E 0 [ W 0 Z X ] , θ 0 ≡ M 0 − 1 r 0 , δ 0 ≡ X − θ 0 ⊤ Z , and define the profile information
I G L M ≡ E 0 [ W 0 δ 0 2 ] = E 0 [ W 0 X 2 ] − r 0 ⊤ M 0 − 1 r 0 . I_{\mathrm{GLM}}\equiv\mathbb{E}_0[W_0\delta_0^2]=\mathbb{E}_0[W_0X^2]-r_0^{\top}M_0^{-1}r_0. I GLM ≡ E 0 [ W 0 δ 0 2 ] = E 0 [ W 0 X 2 ] − r 0 ⊤ M 0 − 1 r 0 . The second equality follows from E 0 [ W 0 δ 0 Z ] = 0 \mathbb{E}_0[W_0\delta_0Z]=0 E 0 [ W 0 δ 0 Z ] = 0
Assumption 10 (Local GLM alternatives for power). Assume that, in a neighbourhood of ( 0 , β 0 ) (0,\beta_0) ( 0 , β 0 ) Y ∣ X , Z Y\mid X,Z Y ∣ X , Z p ( ⋅ ; g − 1 ( X γ + Z ⊤ β ) , ϑ 0 ) p(\cdot;g^{-1}(X\gamma+Z^{\top}\beta),\vartheta_0) p ( ⋅ ; g − 1 ( X γ + Z ⊤ β ) , ϑ 0 ) ( X , Z ) (X,Z) ( X , Z ) ( 0 , β 0 ) (0,\beta_0) ( 0 , β 0 ) P 0 P_0 P 0 ϑ ⋆ = ϑ 0 \vartheta_\star=\vartheta_0 ϑ ⋆ = ϑ 0 h γ ∈ R h_\gamma\in\mathbb{R} h γ ∈ R k ∈ R p k\in\mathbb{R}^p k ∈ R p
γ n = h γ n , β n = β 0 + k n . \gamma_n=\frac{h_\gamma}{\sqrt{n}},\qquad\beta_n=\beta_0+\frac{k}{\sqrt{n}}. γ n = n h γ , β n = β 0 + n k . Write h = ( h γ , k ) h=(h_\gamma,k) h = ( h γ , k ) n n n P n , h ( n ) P_{n,h}^{(n)} P n , h ( n )
Here h γ h_\gamma h γ γ n \gamma_n γ n n − 1 / 2 n^{-1/2} n − 1/2 k k k β n \beta_n β n
Theorem 12 (GLM score: local power is unchanged by permutation calibration). Under Assumption 10,
T n G L M ⟶ d N ( h γ I G L M , 1 ) under P n , h ( n ) . (107) T_n^{\mathrm{GLM}} \mathrel{\overset{d}{\longrightarrow}} N\left(h_\gamma\sqrt{I_{\mathrm{GLM}}},1\right)\qquad\text{under }P_{n,h}^{(n)}. \tag*{(107)} T n GLM ⟶ d N ( h γ I GLM , 1 ) under P n , h ( n ) . (107) Moreover, if
q n , α G L M , π ≡ Q 1 − α ( T n G L M , ( Π ) ∣ F n ) , q_{n,\alpha}^{\mathrm{GLM},\pi}\equiv Q_{1-\alpha}\left(T_n^{\mathrm{GLM},(\Pi)}\mid\mathcal{F}_n\right), q n , α GLM , π ≡ Q 1 − α ( T n GLM , ( Π ) ∣ F n ) , then
q n , α G L M , π = z 1 − α + o P n , h ( n ) ( 1 ) , (108) q_{n,\alpha}^{\mathrm{GLM},\pi}=z_{1-\alpha}+o_{P_{n,h}^{(n)}}(1), \tag*{(108)} q n , α GLM , π = z 1 − α + o P n , h ( n ) ( 1 ) , (108) and therefore
P n , h ( n ) ( T n G L M > q n , α G L M , π ) ⟶ 1 − Φ ( z 1 − α − h γ I G L M ) . (109) P_{n,h}^{(n)}\left(T_n^{\mathrm{GLM}}>q_{n,\alpha}^{\mathrm{GLM},\pi}\right)\longrightarrow1-\Phi\left(z_{1-\alpha}-h_\gamma\sqrt{I_{\mathrm{GLM}}}\right). \tag*{(109)} P n , h ( n ) ( T n GLM > q n , α GLM , π ) ⟶ 1 − Φ ( z 1 − α − h γ I GLM ) . (109) The same expression is the local power of the usual score test 1 { T n G L M > z 1 − α } 1\{T_n^{\mathrm{GLM}}>z_{1-\alpha}\} 1 { T n GLM > z 1 − α } α \alpha α h γ = 0 h_\gamma=0 h γ = 0 h γ h_\gamma h γ
Proof. At ( 0 , β 0 ) (0,\beta_0) ( 0 , β 0 ) ( h γ , k ) (h_\gamma,k) ( h γ , k ) ℓ ˙ h = ( h γ X + k ⊤ Z ) ξ 0 \dot{\ell}_h=(h_\gamma X+k^\top Z)\xi_0 ℓ ˙ h = ( h γ X + k ⊤ Z ) ξ 0 E 0 ( ξ 0 ∣ X , Z ) = 0 \mathbb{E}_0(\xi_0\mid X,Z)=0 E 0 ( ξ 0 ∣ X , Z ) = 0 E 0 ( ξ 0 2 ∣ X , Z ) = W 0 \mathbb{E}_0(\xi_0^2\mid X,Z)=W_0 E 0 ( ξ 0 2 ∣ X , Z ) = W 0 [33 ] ) gives
log d P n , h ( n ) d P 0 ( n ) = 1 n ∑ i = 1 n ℓ ˙ h , i − 1 2 E 0 ( ℓ ˙ h 2 ) + o P 0 ( n ) ( 1 ) , \log\frac{dP_{n,h}^{(n)}}{dP_0^{(n)}}=\frac{1}{\sqrt{n}}\sum_{i=1}^{n}\dot{\ell}_{h,i}-\frac{1}{2}\mathbb{E}_0(\dot{\ell}_h^2)+o_{P_0^{(n)}}(1), log d P 0 ( n ) d P n , h ( n ) = n 1 i = 1 ∑ n ℓ ˙ h , i − 2 1 E 0 ( ℓ ˙ h 2 ) + o P 0 ( n ) ( 1 ) , so P n , h ( n ) ◃ P 0 ( n ) P_{n,h}^{(n)}\triangleleft P_0^{(n)} P n , h ( n ) ◃ P 0 ( n ) [33 ] ). Equation (35) in the proof of Proposition 2, evaluated at the null parameter, gives
1 n ∑ i = 1 n ν ^ i G L M ϵ ^ i = 1 n ∑ i = 1 n δ 0 , i ξ 0 , i + o P 0 ( n ) ( 1 ) . \frac{1}{\sqrt{n}}\sum_{i=1}^{n}\hat{\nu}_i^{\mathrm{GLM}}\hat{\epsilon}_i=\frac{1}{\sqrt{n}}\sum_{i=1}^{n}\delta_{0,i}\xi_{0,i}+o_{P_0^{(n)}}(1). n 1 i = 1 ∑ n ν ^ i GLM ϵ ^ i = n 1 i = 1 ∑ n δ 0 , i ξ 0 , i + o P 0 ( n ) ( 1 ) . For the studentizer, equation (39) and E 0 ( ξ 0 2 ∣ X , Z ) = W 0 \mathbb{E}_0(\xi_0^2\mid X,Z)=W_0 E 0 ( ξ 0 2 ∣ X , Z ) = W 0
V ^ n G L M ⟶ P 0 ( n ) E 0 [ δ 0 2 ξ 0 2 ] = E 0 [ δ 0 2 E 0 ( ξ 0 2 ∣ X , Z ) ] = E 0 [ δ 0 2 W 0 ] = I G L M . \widehat{V}_n^{\mathrm{GLM}}\mathrel{\overset{P_0^{(n)}}{\longrightarrow}}\mathbb{E}_0[\delta_0^2\xi_0^2]=\mathbb{E}_0[\delta_0^2\mathbb{E}_0(\xi_0^2\mid X,Z)]=\mathbb{E}_0[\delta_0^2W_0]=I_{\mathrm{GLM}}. V n GLM ⟶ P 0 ( n ) E 0 [ δ 0 2 ξ 0 2 ] = E 0 [ δ 0 2 E 0 ( ξ 0 2 ∣ X , Z )] = E 0 [ δ 0 2 W 0 ] = I GLM . Hence
T n G L M = 1 n I G L M ∑ i = 1 n δ 0 , i ξ 0 , i + o P 0 ( n ) ( 1 ) . (110) T_n^{\mathrm{GLM}}=\frac{1}{\sqrt{nI_{\mathrm{GLM}}}}\sum_{i=1}^{n}\delta_{0,i}\xi_{0,i}+o_{P_0^{(n)}}(1). \tag*{(110)} T n GLM = n I GLM 1 i = 1 ∑ n δ 0 , i ξ 0 , i + o P 0 ( n ) ( 1 ) . (110) Contiguity transfers the remainder in (110) to o P n , h ( n ) ( 1 ) o_{P_{n,h}^{(n)}}(1) o P n , h ( n ) ( 1 )
Since E 0 ( W 0 Z δ 0 ) = 0 \mathbb{E}_0(W_0Z\delta_0)=0 E 0 ( W 0 Z δ 0 ) = 0 X = δ 0 + θ 0 ⊤ Z X=\delta_0+\theta_0^\top Z X = δ 0 + θ 0 ⊤ Z
Cov 0 ( δ 0 ξ 0 , ℓ ˙ h ) = E 0 { W 0 δ 0 ( h γ X + k ⊤ Z ) } = h γ E 0 ( W 0 δ 0 2 ) = h γ I G L M . \operatorname{Cov}_0(\delta_0\xi_0,\dot{\ell}_h)=\mathbb{E}_0\{W_0\delta_0(h_\gamma X+k^\top Z)\}
=h_\gamma\mathbb{E}_0(W_0\delta_0^2)=h_\gamma I_{\mathrm{GLM}}. Cov 0 ( δ 0 ξ 0 , ℓ ˙ h ) = E 0 { W 0 δ 0 ( h γ X + k ⊤ Z )} = h γ E 0 ( W 0 δ 0 2 ) = h γ I GLM . Both summands are centered and square integrable under P 0 P_0 P 0 E 0 [ δ 0 2 ξ 0 2 ] = I G L M < ∞ \mathbb{E}_0[\delta_0^2\xi_0^2]=I_{\mathrm{GLM}}<\infty E 0 [ δ 0 2 ξ 0 2 ] = I GLM < ∞ E 0 [ ℓ ˙ h 2 ] < ∞ \mathbb{E}_0[\dot{\ell}_h^2]<\infty E 0 [ ℓ ˙ h 2 ] < ∞ P 0 ( n ) P_0^{(n)} P 0 ( n )
1 n ∑ i = 1 n ( δ 0 , i ξ 0 , i I G L M , ℓ ˙ h , i ) → d N 2 ( ( 0 0 ) , ( 1 h γ I G L M h γ I G L M E 0 [ ℓ ˙ h 2 ] ) ) . \frac{1}{\sqrt{n}}\sum_{i=1}^{n}\left(\frac{\delta_{0,i}\xi_{0,i}}{\sqrt{I_{\mathrm{GLM}}}},\dot{\ell}_{h,i}\right)\xrightarrow{d}N_2\left(\begin{pmatrix}0\\0\end{pmatrix},\begin{pmatrix}1&h_\gamma\sqrt{I_{\mathrm{GLM}}}\\h_\gamma\sqrt{I_{\mathrm{GLM}}}&\mathbb{E}_0[\dot{\ell}_h^2]\end{pmatrix}\right). n 1 i = 1 ∑ n ( I GLM δ 0 , i ξ 0 , i , ℓ ˙ h , i ) d N 2 ( ( 0 0 ) , ( 1 h γ I GLM h γ I GLM E 0 [ ℓ ˙ h 2 ] ) ) . Together with the likelihood-ratio expansion, Le Cam’s third lemma (van der Vaart, [33 ] , Example 6.7) gives
1 n I G L M ∑ i = 1 n δ 0 , i ξ 0 , i → d N ( h γ I G L M , 1 ) \frac{1}{\sqrt{nI_{\mathrm{GLM}}}}\sum_{i=1}^{n}\delta_{0,i}\xi_{0,i}\xrightarrow{d}N(h_\gamma\sqrt{I_{\mathrm{GLM}}},1) n I GLM 1 i = 1 ∑ n δ 0 , i ξ 0 , i d N ( h γ I GLM , 1 ) under P n , h ( n ) P_{n,h}^{(n)} P n , h ( n )
Equation (110) and the transferred remainder then yield (107) .
Finally, Theorem 2 gives q n , α G L M , π → z 1 − α q_{n,\alpha}^{\mathrm{GLM},\pi}\to z_{1-\alpha} q n , α GLM , π → z 1 − α P 0 ( n ) P_0^{(n)} P 0 ( n ) Lemma 24 transfers this convergence to P n , h ( n ) P_{n,h}^{(n)} P n , h ( n ) (108) –(109) .
Local power of the GCM with permutation calibration
Assumption 11 (GPLM local alternatives for GCM power). Let P β , g P_{\beta,g} P β , g ( X , Z ) (X,Z) ( X , Z ) P X Z P_{XZ} P X Z Y ∣ X , Z Y\mid X,Z Y ∣ X , Z
f η ( y ) = exp { y η − ψ ( η ) } , η = β X + g ( Z ) , f_\eta(y)=\exp\{y\eta-\psi(\eta)\},\qquad\eta=\beta X+g(Z), f η ( y ) = exp { y η − ψ ( η )} , η = β X + g ( Z ) , with respect to a fixed measure λ \lambda λ f η f_\eta f η J J J P 0 = P 0 , g 0 P_0=P_{0,g_0} P 0 = P 0 , g 0 P 0 ( n ) = P 0 ⊗ n P_0^{(n)}=P_0^{\otimes n} P 0 ( n ) = P 0 ⊗ n E 0 \mathbb{E}_0 E 0 P 0 P_0 P 0 h = ( h β , h g ) h=(h_\beta,h_g) h = ( h β , h g ) h β ∈ R h_\beta\in\mathbb{R} h β ∈ R h g ∈ L 2 ( P Z ) h_g\in L^2(P_Z) h g ∈ L 2 ( P Z )
β n = h β n , g n = g 0 + h g n , P n , h ( n ) = P β n , g n ⊗ n . \beta_n=\frac{h_\beta}{\sqrt{n}},\qquad g_n=g_0+\frac{h_g}{\sqrt{n}},\qquad P_{n,h}^{(n)}=P_{\beta_n,g_n}^{\otimes n}. β n = n h β , g n = g 0 + n h g , P n , h ( n ) = P β n , g n ⊗ n . Assume g 0 ( Z ) + t { h β X + h g ( Z ) } ∈ J g_0(Z)+t\{h_\beta X+h_g(Z)\}\in J g 0 ( Z ) + t { h β X + h g ( Z )} ∈ J t t t P 0 P_0 P 0
ψ ′ ′ \psi^{\prime\prime} ψ ′′ J J J
the support of ( X , Z ) (X,Z) ( X , Z ) g 0 g_0 g 0 h g h_g h g Z Z Z δ > 0 \delta>0 δ > 0 ψ \psi ψ J J J
{ g 0 ( z ) + t { h β x + h g ( z ) } : ( x , z ) ∈ supp ( X , Z ) , ∣ t ∣ ≤ δ } . \{g_0(z)+t\{h_\beta x+h_g(z)\}:(x,z)\in\operatorname{supp}(X,Z),\ |t|\le\delta\}. { g 0 ( z ) + t { h β x + h g ( z )} : ( x , z ) ∈ supp ( X , Z ) , ∣ t ∣ ≤ δ } . Throughout this subsection, μ x \mu_x μ x μ y \mu_y μ y σ G C M 2 \sigma_{\mathrm{GCM}}^2 σ GCM 2 P 0 P_0 P 0 μ x ( z ) = E 0 [ X ∣ Z = z ] \mu_x(z)=\mathbb{E}_0[X\mid Z=z] μ x ( z ) = E 0 [ X ∣ Z = z ] μ y ( z ) = ψ ′ ( g 0 ( z ) ) \mu_y(z)=\psi^\prime(g_0(z)) μ y ( z ) = ψ ′ ( g 0 ( z )) X ⊥ ⊥ Y ∣ Z X\perp\!\!\!\perp Y\mid Z X ⊥ ⊥ Y ∣ Z P 0 P_0 P 0
σ G C M 2 = E 0 [ Var 0 ( X ∣ Z ) Var 0 ( Y ∣ Z ) ] > 0 , \sigma_{\mathrm{GCM}}^2=\mathbb{E}_0[\operatorname{Var}_0(X\mid Z)\operatorname{Var}_0(Y\mid Z)]>0, σ GCM 2 = E 0 [ Var 0 ( X ∣ Z ) Var 0 ( Y ∣ Z )] > 0 , where positivity is Assumption 8(ii).
Theorem 13 (GCM: local power is unchanged by permutation calibration). Under Assumption 11,
T n G C M ⟶ d N ( h β σ G C M , 1 ) under P n , h ( n ) . (111) T_n^{\mathrm{GCM}} \overset{d}{\longrightarrow} N(h_\beta\sigma_{\mathrm{GCM}},1)\quad\text{under }P_{n,h}^{(n)}. \tag*{(111)} T n GCM ⟶ d N ( h β σ GCM , 1 ) under P n , h ( n ) . (111) If
q n , α G C M , π ≡ Q 1 − α ( T n G C M , ( Π ) ∣ F n ) , q_{n,\alpha}^{\mathrm{GCM},\pi}\equiv\mathbb{Q}_{1-\alpha}\left(T_n^{\mathrm{GCM},(\Pi)}\mid\mathcal{F}_n\right), q n , α GCM , π ≡ Q 1 − α ( T n GCM , ( Π ) ∣ F n ) , then
q n , α G C M , π = z 1 − α + o P n , h ( n ) ( 1 ) , (112) q_{n,\alpha}^{\mathrm{GCM},\pi}=z_{1-\alpha}+o_{P_{n,h}^{(n)}}(1), \tag*{(112)} q n , α GCM , π = z 1 − α + o P n , h ( n ) ( 1 ) , (112) and
P n , h ( n ) ( T n G C M > q n , α G C M , π ) ⟶ 1 − Φ ( z 1 − α − h β σ G C M ) . (113) P_{n,h}^{(n)}\left(T_n^{\mathrm{GCM}}>q_{n,\alpha}^{\mathrm{GCM},\pi}\right)\longrightarrow1-\Phi(z_{1-\alpha}-h_\beta\sigma_{\mathrm{GCM}}). \tag*{(113)} P n , h ( n ) ( T n GCM > q n , α GCM , π ) ⟶ 1 − Φ ( z 1 − α − h β σ GCM ) . (113) The GCM calibrated using a normal distribution with cutoff z 1 − α z_{1-\alpha} z 1 − α
Proof. Define
U ≡ ( X − μ x ( Z ) ) ( Y − μ y ( Z ) ) . U\equiv(X-\mu_x(Z))(Y-\mu_y(Z)). U ≡ ( X − μ x ( Z )) ( Y − μ y ( Z )) . Conditional independence under P 0 P_0 P 0 E 0 [ U ] = 0 \mathbb{E}_0[U]=0 E 0 [ U ] = 0
E 0 [ U 2 ] = E 0 [ Var 0 ( X ∣ Z ) Var 0 ( Y ∣ Z ) ] = σ G C M 2 . \mathbb{E}_0[U^2]=\mathbb{E}_0[\operatorname{Var}_0(X\mid Z)\operatorname{Var}_0(Y\mid Z)]=\sigma_{\mathrm{GCM}}^2. E 0 [ U 2 ] = E 0 [ Var 0 ( X ∣ Z ) Var 0 ( Y ∣ Z )] = σ GCM 2 . Equations (53) and (54) in the proof of Theorem 4 give
1 n ∑ i = 1 n R ^ i G C M = 1 n ∑ i = 1 n U i + o P 0 ( n ) ( 1 ) , V ^ n G C M = σ G C M 2 + o P 0 ( n ) ( 1 ) . \frac{1}{\sqrt{n}}\sum_{i=1}^{n}\widehat{R}^{\mathrm{GCM}}_i=\frac{1}{\sqrt{n}}\sum_{i=1}^{n}U_i+o_{P_0^{(n)}}(1),\qquad\widehat{V}^{\mathrm{GCM}}_n=\sigma_{\mathrm{GCM}}^2+o_{P_0^{(n)}}(1). n 1 i = 1 ∑ n R i GCM = n 1 i = 1 ∑ n U i + o P 0 ( n ) ( 1 ) , V n GCM = σ GCM 2 + o P 0 ( n ) ( 1 ) . Since σ G C M > 0 \sigma_{\mathrm{GCM}}>0 σ GCM > 0 n − 1 / 2 ∑ i U i = O P 0 ( n ) ( 1 ) n^{-1/2}\sum_i U_i=O_{P_0^{(n)}}(1) n − 1/2 ∑ i U i = O P 0 ( n ) ( 1 )
T n G C M = 1 σ G C M n ∑ i = 1 n U i + o P 0 ( n ) ( 1 ) . (114) T_n^{\mathrm{GCM}}=\frac{1}{\sigma_{\mathrm{GCM}}\sqrt{n}}\sum_{i=1}^{n}U_i+o_{P_0^{(n)}}(1). \tag*{(114)} T n GCM = σ GCM n 1 i = 1 ∑ n U i + o P 0 ( n ) ( 1 ) . (114) Put Δ ( X , Z ) = h β X + h g ( Z ) \Delta(X,Z)=h_\beta X+h_g(Z) Δ ( X , Z ) = h β X + h g ( Z ) P t = P t h β , g 0 + t h g P_t=P_{th_\beta,g_0+th_g} P t = P t h β , g 0 + t h g p t p_t p t P X Z ⊗ λ P_{XZ}\otimes\lambda P X Z ⊗ λ
∂ t p t ( x , y , z ) = 1 2 Δ ( x , z ) { y − ψ ′ ( g 0 ( z ) + t Δ ( x , z ) ) } p t ( x , y , z ) . \partial_t\sqrt{p_t(x,y,z)}=\frac{1}{2}\Delta(x,z)\{y-\psi'(g_0(z)+t\Delta(x,z))\}\sqrt{p_t(x,y,z)}. ∂ t p t ( x , y , z ) = 2 1 Δ ( x , z ) { y − ψ ′ ( g 0 ( z ) + t Δ ( x , z ))} p t ( x , y , z ) . Thus its score at zero is ℓ ˙ h = Δ ( X , Z ) { Y − μ y ( Z ) } \dot{\ell}_h=\Delta(X,Z)\{Y-\mu_y(Z)\} ℓ ˙ h = Δ ( X , Z ) { Y − μ y ( Z )}
4 ∫ ( ∂ t p t ) 2 d ( P X Z ⊗ λ ) = E 0 [ Δ 2 ψ ′ ′ { g 0 ( Z ) + t Δ } ] ≡ I ( t ) . 4\int(\partial_t\sqrt{p_t})^2\,d(P_{XZ}\otimes\lambda)=\mathbb{E}_0[\Delta^2\psi''\{g_0(Z)+t\Delta\}]\equiv I(t). 4 ∫ ( ∂ t p t ) 2 d ( P X Z ⊗ λ ) = E 0 [ Δ 2 ψ ′′ { g 0 ( Z ) + t Δ }] ≡ I ( t ) . Under Assumption 11(i), writing v = ψ ′ ′ > 0 v=\psi''>0 v = ψ ′′ > 0
I ( t ) = v E 0 [ Δ 2 ] ≤ 2 v { h β 2 E 0 [ X 2 ] + E 0 [ h g ( Z ) 2 ] } < ∞ , I(t)=v\mathbb{E}_0[\Delta^2]\leq2v\{h_\beta^2\mathbb{E}_0[X^2]+\mathbb{E}_0[h_g(Z)^2]\}<\infty, I ( t ) = v E 0 [ Δ 2 ] ≤ 2 v { h β 2 E 0 [ X 2 ] + E 0 [ h g ( Z ) 2 ]} < ∞ , by Assumption 8(i) and h g ∈ L 2 ( P Z ) h_g\in L^2(P_Z) h g ∈ L 2 ( P Z ) I ( t ) I(t) I ( t ) Δ \Delta Δ ψ ′ ′ \psi'' ψ ′′
sup ∣ t ∣ ≤ δ ∣ Δ 2 ψ ′ ′ { g 0 ( Z ) + t Δ } ∣ ≤ ∥ Δ ∥ ∞ 2 sup ( x , z ) ∈ supp ( X , Z ) ∣ t ∣ ≤ δ ∣ ψ ′ ′ { g 0 ( z ) + t Δ ( x , z ) } ∣ < ∞ . \sup_{|t|\leq\delta}|\Delta^2\psi''\{g_0(Z)+t\Delta\}|\leq\|\Delta\|_\infty^2\sup_{\substack{(x,z)\in\operatorname{supp}(X,Z)\\ |t|\leq\delta}}|\psi''\{g_0(z)+t\Delta(x,z)\}|<\infty. ∣ t ∣ ≤ δ sup ∣ Δ 2 ψ ′′ { g 0 ( Z ) + t Δ } ∣ ≤ ∥Δ ∥ ∞ 2 ( x , z ) ∈ supp ( X , Z ) ∣ t ∣ ≤ δ sup ∣ ψ ′′ { g 0 ( z ) + t Δ ( x , z )} ∣ < ∞. Dominated convergence gives I ( t ) → I ( 0 ) I(t) \to I(0) I ( t ) → I ( 0 ) t → 0 t \to0 t → 0 [33 ] gives differentiability in quadratic mean at zero. Theorem 7.2 therein yields
log d P n , h ( n ) d P 0 ( n ) = 1 n ∑ i = 1 n ℓ ˙ h , i − 1 2 I ( 0 ) + o P 0 ( n ) ( 1 ) . \log\frac{dP_{n,h}^{(n)}}{dP_0^{(n)}} = \frac{1}{\sqrt{n}}\sum_{i=1}^{n}\dot{\ell}_{h,i} - \frac{1}{2}I(0) + o_{P_0^{(n)}}(1). log d P 0 ( n ) d P n , h ( n ) = n 1 i = 1 ∑ n ℓ ˙ h , i − 2 1 I ( 0 ) + o P 0 ( n ) ( 1 ) . Since E 0 [ ℓ ˙ h ] = 0 \mathbb{E}_0[\dot{\ell}_h] = 0 E 0 [ ℓ ˙ h ] = 0 E 0 [ ℓ ˙ h 2 ] = I ( 0 ) < ∞ \mathbb{E}_0[\dot{\ell}_h^2] = I(0) < \infty E 0 [ ℓ ˙ h 2 ] = I ( 0 ) < ∞ [33 Lemma 6.4] give P n , h ( n ) ◃ P 0 ( n ) P_{n,h}^{(n)} \triangleleft P_0^{(n)} P n , h ( n ) ◃ P 0 ( n )
Using conditional independence and E 0 [ X − μ x ( Z ) ∣ Z ] = 0 \mathbb{E}_0[X - \mu_x(Z) \mid Z] = 0 E 0 [ X − μ x ( Z ) ∣ Z ] = 0
E 0 [ U ℓ ˙ h ] = E 0 [ Var 0 ( Y ∣ Z ) E 0 { ( X − μ x ( Z ) ) ( h β X + h g ( Z ) ) ∣ Z } ] = h β E 0 [ Var 0 ( Y ∣ Z ) Var 0 ( X ∣ Z ) ] = h β σ G C M 2 . \mathbb{E}_0[U\dot{\ell}_h] = \mathbb{E}_0[\operatorname{Var}_0(Y \mid Z)\mathbb{E}_0\{(X - \mu_x(Z))(h_\beta X + h_g(Z)) \mid Z\}]
= h_\beta\mathbb{E}_0[\operatorname{Var}_0(Y \mid Z)\operatorname{Var}_0(X \mid Z)] = h_\beta\sigma_{\mathrm{GCM}}^2. E 0 [ U ℓ ˙ h ] = E 0 [ Var 0 ( Y ∣ Z ) E 0 {( X − μ x ( Z )) ( h β X + h g ( Z )) ∣ Z }] = h β E 0 [ Var 0 ( Y ∣ Z ) Var 0 ( X ∣ Z )] = h β σ GCM 2 . The centered summands have finite second moments, so under P 0 ( n ) P_0^{(n)} P 0 ( n )
1 n ∑ i = 1 n ( U i / σ G C M ℓ ˙ h , i ) → d N 2 ( ( 0 0 ) , ( 1 h β σ G C M h β σ G C M I ( 0 ) ) ) . \frac{1}{\sqrt{n}}\sum_{i=1}^{n}
\begin{pmatrix}
U_i/\sigma_{\mathrm{GCM}} \\
\dot{\ell}_{h,i}
\end{pmatrix}
\xrightarrow{d}
N_2\left(
\begin{pmatrix}
0\\
0
\end{pmatrix},
\begin{pmatrix}
1 & h_\beta\sigma_{\mathrm{GCM}}\\
h_\beta\sigma_{\mathrm{GCM}} & I(0)
\end{pmatrix}
\right). n 1 i = 1 ∑ n ( U i / σ GCM ℓ ˙ h , i ) d N 2 ( ( 0 0 ) , ( 1 h β σ GCM h β σ GCM I ( 0 ) ) ) . Together with the likelihood-ratio expansion, Le Cam’s third lemma [33 Example 6.7] gives
1 σ G C M n ∑ i = 1 n U i → d N ( h β σ G C M , 1 ) under P n , h ( n ) . \frac{1}{\sigma_{\mathrm{GCM}}\sqrt{n}}\sum_{i=1}^{n} U_i \xrightarrow{d} N(h_\beta\sigma_{\mathrm{GCM}}, 1) \quad\text{under } P_{n,h}^{(n)}. σ GCM n 1 i = 1 ∑ n U i d N ( h β σ GCM , 1 ) under P n , h ( n ) . Contiguity transfers the remainder in (114) , proving (111) . Finally, Theorem 4 and Lemma 5 give q n , α G C M , π → z 1 − α q_{n,\alpha}^{\mathrm{GCM},\pi} \to z_{1-\alpha} q n , α GCM , π → z 1 − α P 0 ( n ) P_0^{(n)} P 0 ( n ) P n , h ( n ) P_{n,h}^{(n)} P n , h ( n ) (112) –(113) .
Local power of the PCM with permutation calibration
In Algorithm 4, use OLS with an intercept in Z Z Z n 1 , n 2 → ∞ n_1, n_2 \to\infty n 1 , n 2 → ∞ n 1 + n 2 = n n_1 + n_2 = n n 1 + n 2 = n n / 2 n/2 n /2 n 1 n_1 n 1
Assumption 12 (Local linear alternatives for OLS PCM power). Let ( X , Z , ζ ) (X, Z, \zeta) ( X , Z , ζ ) p p p E [ ζ ∣ X , Z ] = 0 \mathbb{E}[\zeta\mid X, Z] = 0 E [ ζ ∣ X , Z ] = 0 ζ ⊥ ⊥ X ∣ Z \zeta\perp\!\!\!\perp X \mid Z ζ ⊥ ⊥ X ∣ Z h ∈ R h \in\mathbb{R} h ∈ R γ 0 ∈ R p \gamma_0 \in\mathbb{R}^p γ 0 ∈ R p
Y = β n X + γ 0 T Z + ζ , β n = h / n . Y = \beta_n X + \gamma_0^\mathsf{T} Z + \zeta, \qquad\beta_n = h/\sqrt{n}. Y = β n X + γ 0 T Z + ζ , β n = h / n . Write P n , h ( n ) P_{n,h}^{(n)} P n , h ( n ) n n n P 0 ( n ) P_0^{(n)} P 0 ( n ) β = 0 \beta= 0 β = 0 P n , h ( n ) ◃ P 0 ( n ) P_{n,h}^{(n)} \triangleleft P_0^{(n)} P n , h ( n ) ◃ P 0 ( n )
E [ ∣ X ∣ 4 + ∥ Z ∥ 4 + ∣ ζ ∣ 4 ] < ∞ , E [ W W T ] ≻ 0 , W = ( X , Z T ) T . \mathbb{E}[|X|^4 + \lVert Z\rVert^4 + |\zeta|^4] < \infty, \qquad\mathbb{E}[WW^\mathsf{T}] \succ0, \qquad W = (X, Z^\mathsf{T})^\mathsf{T}. E [ ∣ X ∣ 4 + ∥ Z ∥ 4 + ∣ ζ ∣ 4 ] < ∞ , E [ W W T ] ≻ 0 , W = ( X , Z T ) T . Define
η X = { E [ Z Z T ] } − 1 E [ Z X ] , ξ = X − η X T Z , σ ξ 2 = E [ ξ 2 ] , \eta_X = \{\mathbb{E}[ZZ^\mathsf{T}]\}^{-1}\mathbb{E}[ZX], \qquad\xi= X - \eta_X^\mathsf{T}Z, \qquad\sigma_\xi^2 = \mathbb{E}[\xi^2], η X = { E [ Z Z T ] } − 1 E [ Z X ] , ξ = X − η X T Z , σ ξ 2 = E [ ξ 2 ] , and assume σ ξ ζ 2 ≡ E [ ξ 2 ζ 2 ] > 0 \sigma_{\xi\zeta}^2 \equiv\mathbb{E}[\xi^2\zeta^2] > 0 σ ξ ζ 2 ≡ E [ ξ 2 ζ 2 ] > 0
Here h h h β n \beta_n β n n − 1 / 2 n^{-1/2} n − 1/2
σ β 2 = σ ξ ζ 2 σ ξ 4 , \sigma_\beta^2 = \frac{\sigma_{\xi\zeta}^2}{\sigma_\xi^4}, σ β 2 = σ ξ 4 σ ξ ζ 2 , and define
a n = n 2 β n σ β , b n = n 1 β n σ ξ 2 σ ξ ξ . (115) a_n = \frac{\sqrt{n_2}\beta_n}{\sigma_\beta}, \qquad b_n = \frac{\sqrt{n_1}\beta_n\sigma_\xi^2}{\sigma_{\xi\xi}}. \tag*{(115)} a n = σ β n 2 β n , b n = σ ξ ξ n 1 β n σ ξ 2 . (115) Theorem 14 (PCM with OLS: local power is unchanged by permutation calibration). Fix α ∈ ( 0 , 1 ) \alpha\in(0,1) α ∈ ( 0 , 1 ) Under Assumption 12, suppose a n → a a_n \to a a n → a b n → b b_n \to b b n → b P n , h ( n ) P_{n,h}^{(n)} P n , h ( n )
T n P C M → d T ∞ P C M , T ∞ P C M ∼ Φ ( a ) N ( b , 1 ) + Φ ( − a ) N ( − b , 1 ) . (116) T_n^{\mathrm{PCM}} \xrightarrow{d} T_\infty^{\mathrm{PCM}}, \qquad T_\infty^{\mathrm{PCM}} \sim\Phi(a)N(b,1) + \Phi(-a)N(-b,1). \tag*{(116)} T n PCM d T ∞ PCM , T ∞ PCM ∼ Φ ( a ) N ( b , 1 ) + Φ ( − a ) N ( − b , 1 ) . (116) Moreover, the conditional permutation quantile
q n , α P C M , π ≡ Q 1 − α ( T n P C M , ( Π ) ∣ F n ) q_{n,\alpha}^{\mathrm{PCM},\pi} \equiv Q_{1-\alpha}\left(T_n^{\mathrm{PCM},(\Pi)} \mid\mathcal{F}_n\right) q n , α PCM , π ≡ Q 1 − α ( T n PCM , ( Π ) ∣ F n ) satisfies
q n , α P C M , π = z 1 − α + o P n , h ( n ) ( 1 ) , (117) q_{n,\alpha}^{\mathrm{PCM},\pi} = z_{1-\alpha} + o_{P_{n,h}^{(n)}}(1), \tag*{(117)} q n , α PCM , π = z 1 − α + o P n , h ( n ) ( 1 ) , (117) and
P n , h ( n ) ( T n P C M > q n , α P C M , π ) ⟶ Φ ( a ) Φ ( z α + b ) + Φ ( − a ) Φ ( z α − b ) . (118) P_{n,h}^{(n)}\left(T_n^{\mathrm{PCM}} > q_{n,\alpha}^{\mathrm{PCM},\pi}\right) \longrightarrow\Phi(a)\Phi(z_\alpha+b) + \Phi(-a)\Phi(z_\alpha-b). \tag*{(118)} P n , h ( n ) ( T n PCM > q n , α PCM , π ) ⟶ Φ ( a ) Φ ( z α + b ) + Φ ( − a ) Φ ( z α − b ) . (118) The statistic calibrated using a normal distribution with cutoff z 1 − α z_{1-\alpha} z 1 − α [26 Proposition S37] . For n 1 = n 2 = n / 2 n_1=n_2=n/2 n 1 = n 2 = n /2
a = b = h σ ξ 2 2 σ ξ ζ . a=b=\frac{h\sigma_\xi^2}{\sqrt{2}\sigma_{\xi\zeta}}. a = b = 2 σ ξ ζ h σ ξ 2 . Proof. OLS factorization and sampling limit. Probability orders in this part refer to P n , h ( n ) P_{n,h}^{(n)} P n , h ( n ) m ^ p ( x , z ) = β ^ x + γ ^ ⊤ z \widehat m_p(x,z)=\widehat\beta x+\widehat\gamma^\top z m p ( x , z ) = β x + γ ⊤ z m ^ q ( z ) = χ ^ ⊤ z \widehat m_q(z)=\widehat\chi^\top z m q ( z ) = χ ⊤ z Z 1 , X 1 Z_1,X_1 Z 1 , X 1 H 1 H_1 H 1 Z 1 Z_1 Z 1
V ^ P C M = ( I − H 1 ) { β ^ X 1 + Z 1 ( γ ^ − χ ^ ) } = β ^ ( I − H 1 ) X 1 . \widehat{V}^{\mathrm{PCM}}=(I-H_1)\{\widehat\beta X_1+Z_1(\widehat\gamma-\widehat\chi)\}=\widehat\beta(I-H_1)X_1. V PCM = ( I − H 1 ) { β X 1 + Z 1 ( γ − χ )} = β ( I − H 1 ) X 1 . For the OLS coefficients for the inference fold η ^ X , η ^ Y \widehat\eta_X,\widehat\eta_Y η X , η Y R i = ( X i − Z i ⊤ η ^ X ) ( Y i − Z i ⊤ η ^ Y ) R_i=(X_i-Z_i^\top\widehat\eta_X)(Y_i-Z_i^\top\widehat\eta_Y) R i = ( X i − Z i ⊤ η X ) ( Y i − Z i ⊤ η Y )
R ^ i P C M = β ^ R i , T n P C M = sgn ( β ^ ) Q n , Q n = n 1 − 1 / 2 ∑ i ∈ D 1 R i S n 1 ( R ) . (119) \widehat R_i^{\mathrm{PCM}}=\widehat\beta R_i,\qquad T_n^{\mathrm{PCM}}=\operatorname{sgn}(\widehat\beta)Q_n,\qquad Q_n=\frac{n_1^{-1/2}\sum_{i\in D_1}R_i}{\sqrt{S_{n_1}(R)}}. \tag*{(119)} R i PCM = β R i , T n PCM = sgn ( β ) Q n , Q n = S n 1 ( R ) n 1 − 1/2 ∑ i ∈ D 1 R i . (119) Here S n 1 ( R ) = n 1 − 1 ∑ i R i 2 − ( n 1 − 1 ∑ i R i ) 2 S_{n_1}(R)=n_1^{-1}\sum_i R_i^2-(n_1^{-1}\sum_iR_i)^2 S n 1 ( R ) = n 1 − 1 ∑ i R i 2 − ( n 1 − 1 ∑ i R i ) 2 Q n = 0 Q_n=0 Q n = 0 sgn ( 0 ) = 0 \operatorname{sgn}(0)=0 sgn ( 0 ) = 0 β ^ \widehat\beta β Q n Q_n Q n
The population coefficient of Y Y Y Z Z Z η Y , n = γ 0 + β n η X \eta_{Y,n}=\gamma_0+\beta_n\eta_X η Y , n = γ 0 + β n η X ϵ n , i = ζ i + β n ξ i \epsilon_{n,i}=\zeta_i+\beta_n\xi_i ϵ n , i = ζ i + β n ξ i d X = η ^ X − η X d_X=\widehat\eta_X-\eta_X d X = η X − η X d Y = η ^ Y − η Y , n d_Y=\widehat\eta_Y-\eta_{Y,n} d Y = η Y − η Y , n
( d X d Y ) = ( 1 n 1 ∑ i ∈ D 1 Z i Z i ⊤ ) − 1 1 n 1 ∑ i ∈ D 1 Z i ( ξ i ϵ n , i ) = O P ( n 1 − 1 / 2 ) , (d_X\ \ d_Y)=\left(\frac{1}{n_1}\sum_{i\in D_1}Z_iZ_i^\top\right)^{-1}\frac{1}{n_1}\sum_{i\in D_1}Z_i(\xi_i\ \ \epsilon_{n,i})=O_P(n_1^{-1/2}), ( d X d Y ) = ( n 1 1 i ∈ D 1 ∑ Z i Z i ⊤ ) − 1 n 1 1 i ∈ D 1 ∑ Z i ( ξ i ϵ n , i ) = O P ( n 1 − 1/2 ) , since Z i ξ i Z_i\xi_i Z i ξ i Z i ϵ n , i Z_i\epsilon_{n,i} Z i ϵ n , i D 1 D_1 D 1
1 n 1 ∑ i ( R i − ξ i ϵ n , i ) = − d X T Z 1 T Z 1 d Y n 1 = O P ( n 1 − 1 / 2 ) . \frac{1}{\sqrt{n_1}}\sum_i(R_i-\xi_i\epsilon_{n,i})=-\frac{d_X^{\mathsf T}Z_1^{\mathsf T}Z_1d_Y}{\sqrt{n_1}}=O_{\mathbb P}(n_1^{-1/2}). n 1 1 i ∑ ( R i − ξ i ϵ n , i ) = − n 1 d X T Z 1 T Z 1 d Y = O P ( n 1 − 1/2 ) . Moreover, R i − ξ i ζ i = − ξ i Z i T d Y − ϵ n , i Z i T d X + ( Z i T d X ) ( Z i T d Y ) + β n ξ i 2 R_i-\xi_i\zeta_i=-\xi_iZ_i^{\mathsf T}d_Y-\epsilon_{n,i}Z_i^{\mathsf T}d_X+(Z_i^{\mathsf T}d_X)(Z_i^{\mathsf T}d_Y)+\beta_n\xi_i^2 R i − ξ i ζ i = − ξ i Z i T d Y − ϵ n , i Z i T d X + ( Z i T d X ) ( Z i T d Y ) + β n ξ i 2
1 n 1 ∑ i ( R i − ξ i ζ i ) 2 ≤ 4 ∥ d Y ∥ 2 1 n 1 ∑ i ξ i 2 ∥ Z i ∥ 2 + 4 ∥ d X ∥ 2 1 n 1 ∑ i ϵ n , i 2 ∥ Z i ∥ 2 + 4 ∥ d X ∥ 2 ∥ d Y ∥ 2 1 n 1 ∑ i ∥ Z i ∥ 4 + 4 β n 2 n 1 ∑ i ξ i 4 = O P ( n 1 − 1 + β n 2 ) = o P ( 1 ) . \begin{aligned}
\frac{1}{n_1}\sum_i(R_i-\xi_i\zeta_i)^2
&\le4\lVert d_Y\rVert^2\frac{1}{n_1}\sum_i\xi_i^2\lVert Z_i\rVert^2
+4\lVert d_X\rVert^2\frac{1}{n_1}\sum_i\epsilon_{n,i}^2\lVert Z_i\rVert^2\\
&\quad+4\lVert d_X\rVert^2\lVert d_Y\rVert^2\frac{1}{n_1}\sum_i\lVert Z_i\rVert^4
+\frac{4\beta_n^2}{n_1}\sum_i\xi_i^4\\
&=O_{\mathbb P}(n_1^{-1}+\beta_n^2)=o_{\mathbb P}(1).
\end{aligned} n 1 1 i ∑ ( R i − ξ i ζ i ) 2 ≤ 4 ∥ d Y ∥ 2 n 1 1 i ∑ ξ i 2 ∥ Z i ∥ 2 + 4 ∥ d X ∥ 2 n 1 1 i ∑ ϵ n , i 2 ∥ Z i ∥ 2 + 4 ∥ d X ∥ 2 ∥ d Y ∥ 2 n 1 1 i ∑ ∥ Z i ∥ 4 + n 1 4 β n 2 i ∑ ξ i 4 = O P ( n 1 − 1 + β n 2 ) = o P ( 1 ) . Each average on the right is O P ( 1 ) O_{\mathbb P}(1) O P ( 1 ) ξ i ζ i \xi_i\zeta_i ξ i ζ i σ ξ ζ 2 \sigma_{\xi\zeta}^2 σ ξ ζ 2 β n n 1 − 1 / 2 ∑ i ( ξ i 2 − σ ξ 2 ) = O P ( ∣ β n ∣ ) = o P ( 1 ) \beta_n n_1^{-1/2}\sum_i(\xi_i^2-\sigma_\xi^2)=O_{\mathbb P}(|\beta_n|)=o_{\mathbb P}(1) β n n 1 − 1/2 ∑ i ( ξ i 2 − σ ξ 2 ) = O P ( ∣ β n ∣ ) = o P ( 1 )
S n 1 ( R ) = σ ξ ζ 2 + o P ( 1 ) , S_{n_1}(R)=\sigma_{\xi\zeta}^2+o_{\mathbb P}(1), S n 1 ( R ) = σ ξ ζ 2 + o P ( 1 ) , Q n = 1 σ ξ ζ n 1 ∑ i ξ i ζ i + b n + o P ( 1 ) . Q_n=\frac{1}{\sigma_{\xi\zeta}\sqrt{n_1}}\sum_i\xi_i\zeta_i+b_n+o_{\mathbb P}(1). Q n = σ ξ ζ n 1 1 i ∑ ξ i ζ i + b n + o P ( 1 ) . Here b n = O ( 1 ) b_n=O(1) b n = O ( 1 ) σ ξ ζ \sigma_{\xi\zeta} σ ξ ζ
For the training coefficient, with W i = ( X i , Z i T ) T W_i=(X_i,Z_i^{\mathsf T})^{\mathsf T} W i = ( X i , Z i T ) T E [ W W T ] \mathbb E[WW^{\mathsf T}] E [ W W T ]
n 2 ( β ^ − β n ) = [ ( 1 n 2 ∑ i ∈ D 2 W i W i T ) − 1 1 n 2 ∑ i ∈ D 2 W i ζ i ] 1 = 1 σ ξ 2 n 2 ∑ i ∈ D 2 ξ i ζ i + o P ( 1 ) , \begin{aligned}
\sqrt{n_2}(\hat\beta-\beta_n)
&=\left[\left(\frac{1}{n_2}\sum_{i\in D_2}W_iW_i^{\mathsf T}\right)^{-1}
\frac{1}{\sqrt{n_2}}\sum_{i\in D_2}W_i\zeta_i\right]_1\\
&=\frac{1}{\sigma_\xi^2\sqrt{n_2}}\sum_{i\in D_2}\xi_i\zeta_i+o_{\mathbb P}(1),
\end{aligned} n 2 ( β ^ − β n ) = ( n 2 1 i ∈ D 2 ∑ W i W i T ) − 1 n 2 1 i ∈ D 2 ∑ W i ζ i 1 = σ ξ 2 n 2 1 i ∈ D 2 ∑ ξ i ζ i + o P ( 1 ) , where [ ⋅ ] 1 [\cdot]_1 [ ⋅ ] 1
( n 2 β ^ / σ β Q n ) → d N 2 ( ( a b ) , I 2 ) . \begin{pmatrix}
\sqrt{n_2}\hat\beta/\sigma_\beta\\
Q_n
\end{pmatrix}
\xrightarrow{d}
N_2\left(\begin{pmatrix}a\\b\end{pmatrix},I_2\right). ( n 2 β ^ / σ β Q n ) d N 2 ( ( a b ) , I 2 ) . The first limiting coordinate has no atom at zero, so P ( β ^ = 0 ) → 0 \mathbb P(\hat\beta=0)\to0 P ( β ^ = 0 ) → 0 (119) proves (116) .
Permutation distribution under the base null. Here probability orders refer to P 0 ( n ) \mathbb P_0^{(n)} P 0 ( n ) D 1 D_1 D 1 c i = X i − X ˉ 1 c_i=X_i-\bar X_1 c i = X i − X ˉ 1 X ˉ 1 = n 1 − 1 ∑ i X i \bar X_1=n_1^{-1}\sum_iX_i X ˉ 1 = n 1 − 1 ∑ i X i e i = Y i − Z i T γ ^ Y e_i=Y_i-Z_i^{\mathsf T}\hat\gamma_Y e i = Y i − Z i T γ ^ Y ∑ i c i = ∑ i e i = 0 \sum_i c_i=\sum_i e_i=0 ∑ i c i = ∑ i e i = 0 H 1 e = 0 H_1e=0 H 1 e = 0 e i = ζ i − Z i T d Y e_i=\zeta_i-Z_i^{\mathsf T}d_Y e i = ζ i − Z i T d Y d Y = O P ( n 1 − 1 / 2 ) d_Y=O_{\mathbb P}(n_1^{-1/2}) d Y = O P ( n 1 − 1/2 ) n 1 − 1 ∑ i ( Z i T d Y ) 2 = O P ( n 1 − 1 ) n_1^{-1}\sum_i(Z_i^{\mathsf T}d_Y)^2=O_{\mathbb P}(n_1^{-1}) n 1 − 1 ∑ i ( Z i T d Y ) 2 = O P ( n 1 − 1 )
1 n 1 ∑ i c i 2 → p Var ( X ) > 0 , 1 n 1 ∑ i e i 2 → p E [ ζ 2 ] > 0. \frac{1}{n_1}\sum_i c_i^2\xrightarrow{p}\operatorname{Var}(X)>0,\qquad
\frac{1}{n_1}\sum_i e_i^2\xrightarrow{p}\mathbb E[\zeta^2]>0. n 1 1 i ∑ c i 2 p Var ( X ) > 0 , n 1 1 i ∑ e i 2 p E [ ζ 2 ] > 0. For any i.i.d. V i V_i V i E ∣ V i ∣ 4 < ∞ \mathbb{E}|V_i|^4 < \infty E ∣ V i ∣ 4 < ∞
P ( max i ∣ V i ∣ 4 > ε n 1 ) ≤ ε − 1 E [ ∣ V 1 ∣ 4 1 { ∣ V 1 ∣ 4 > ε n 1 } ] ⟶ 0 , ε > 0. \mathbb{P}\left(\max_i |V_i|^4 > \varepsilon n_1\right) \le\varepsilon^{-1}\mathbb{E}\left[|V_1|^4 \mathbf{1}\{|V_1|^4 > \varepsilon n_1\}\right] \longrightarrow0,\qquad\varepsilon>0. P ( i max ∣ V i ∣ 4 > ε n 1 ) ≤ ε − 1 E [ ∣ V 1 ∣ 4 1 { ∣ V 1 ∣ 4 > ε n 1 } ] ⟶ 0 , ε > 0. Apply this to X i X_i X i ζ i \zeta_i ζ i ∥ Z i ∥ \lVert Z_i\rVert ∥ Z i ∥
c i 4 + e i 4 ≤ 8 { ∣ X i ∣ 4 + ∣ X ˉ 1 ∣ 4 + ∣ ζ i ∣ 4 + ∥ d Y ∥ 4 ∥ Z i ∥ 4 } c_i^4+e_i^4\le8\{|X_i|^4+|\bar X_1|^4+|\zeta_i|^4+\lVert d_Y\rVert^4\lVert Z_i\rVert^4\} c i 4 + e i 4 ≤ 8 { ∣ X i ∣ 4 + ∣ X ˉ 1 ∣ 4 + ∣ ζ i ∣ 4 + ∥ d Y ∥ 4 ∥ Z i ∥ 4 } and the law of large numbers imply n 1 − 1 ∑ i ( c i 4 + e i 4 ) = O P ( 1 ) n_1^{-1}\sum_i(c_i^4+e_i^4)=O_{\mathbb P}(1) n 1 − 1 ∑ i ( c i 4 + e i 4 ) = O P ( 1 ) n 1 − 1 max i ( c i 4 + e i 4 ) = o P ( 1 ) n_1^{-1}\max_i(c_i^4+e_i^4)=o_{\mathbb P}(1) n 1 − 1 max i ( c i 4 + e i 4 ) = o P ( 1 )
σ p , n 2 = ( 1 n 1 ∑ i e i 2 ) ( 1 n 1 ∑ i c i 2 ) ⟶ p Var ( X ) E [ ζ 2 ] > 0. \sigma_{p,n}^2=\left(\frac{1}{n_1}\sum_i e_i^2\right)\left(\frac{1}{n_1}\sum_i c_i^2\right)\mathop{\longrightarrow}^{p}\operatorname{Var}(\mathbf X)\mathbb E[\zeta^2]>0. σ p , n 2 = ( n 1 1 i ∑ e i 2 ) ( n 1 1 i ∑ c i 2 ) ⟶ p Var ( X ) E [ ζ 2 ] > 0. The array C n , i k = e i c k C_{n,ik}=e_ic_k C n , ik = e i c k
max i , k C n , i k 2 n 1 σ p , n 2 = ( max i e i 4 / n 1 ) 1 / 2 ( max k c k 4 / n 1 ) 1 / 2 σ p , n 2 = o P ( 1 ) . \frac{\max_{i,k}C_{n,ik}^2}{n_1\sigma_{p,n}^2}
=\frac{(\max_i e_i^4/n_1)^{1/2}(\max_k c_k^4/n_1)^{1/2}}{\sigma_{p,n}^2}
=o_{\mathbb P}(1). n 1 σ p , n 2 max i , k C n , ik 2 = σ p , n 2 ( max i e i 4 / n 1 ) 1/2 ( max k c k 4 / n 1 ) 1/2 = o P ( 1 ) . Its permutation sum has conditional variance n 1 2 σ p , n 2 / ( n 1 − 1 ) n_1^2\sigma_{p,n}^2/(n_1-1) n 1 2 σ p , n 2 / ( n 1 − 1 )
∑ i e i c Π ( i ) n 1 σ p , n ∣ F n ⟶ d , p N ( 0 , 1 ) . (120) \left.\frac{\sum_i e_i c_{\Pi(i)}}{\sqrt{n_1}\sigma_{p,n}}\right|\mathcal F_n
\mathop{\longrightarrow}^{d,p}N(0,1). \tag*{(120)} n 1 σ p , n ∑ i e i c Π ( i ) F n ⟶ d , p N ( 0 , 1 ) . (120) For the second moment, Lemma 10 gives
E Π [ 1 n 1 ∑ i e i 2 c Π ( i ) 2 ∣ F n ] = σ p , n 2 , \mathbb E_\Pi\left[\left.\frac{1}{n_1}\sum_i e_i^2c_{\Pi(i)}^2\right|\mathcal F_n\right]=\sigma_{p,n}^2, E Π [ n 1 1 i ∑ e i 2 c Π ( i ) 2 F n ] = σ p , n 2 , Var Π ( 1 n 1 ∑ i e i 2 c Π ( i ) 2 ∣ F n ) ≤ ( n 1 − 1 ∑ i e i 4 ) ( n 1 − 1 ∑ i c i 4 ) n 1 − 1 = O P ( n 1 − 1 ) . \operatorname{Var}_\Pi\left(\left.\frac{1}{n_1}\sum_i e_i^2c_{\Pi(i)}^2\right|\mathcal F_n\right)
\le\frac{(n_1^{-1}\sum_i e_i^4)(n_1^{-1}\sum_i c_i^4)}{n_1-1}
=O_{\mathbb P}(n_1^{-1}). Var Π ( n 1 1 i ∑ e i 2 c Π ( i ) 2 F n ) ≤ n 1 − 1 ( n 1 − 1 ∑ i e i 4 ) ( n 1 − 1 ∑ i c i 4 ) = O P ( n 1 − 1 ) . Lemma 8(b) thus gives n 1 − 1 ∑ i e i 2 c Π ( i ) 2 − σ p , n 2 = o P ( 1 ) n_1^{-1}\sum_i e_i^2c_{\Pi(i)}^2-\sigma_{p,n}^2=o_{\mathbb P}(1) n 1 − 1 ∑ i e i 2 c Π ( i ) 2 − σ p , n 2 = o P ( 1 )
The OLS refit gives R ^ i P C M , ( Π ) = β ^ R i ( Π ) \hat R_i^{\mathrm{PCM},(\Pi)}=\hat\beta R_i^{(\Pi)} R ^ i PCM , ( Π ) = β ^ R i ( Π ) R i ( Π ) = e i { c Π ( i ) − ( H 1 c Π ) i } R_i^{(\Pi)}=e_i\{c_{\Pi(i)}-(H_1c_\Pi)_i\} R i ( Π ) = e i { c Π ( i ) − ( H 1 c Π ) i } H 1 e = 0 H_1e=0 H 1 e = 0 ∑ i R i ( Π ) = ∑ i e i c Π ( i ) \sum_iR_i^{(\Pi)}=\sum_i e_ic_{\Pi(i)} ∑ i R i ( Π ) = ∑ i e i c Π ( i ) c c c E Π [ c Π c Π ⊤ ∣ F n ] = ∥ c ∥ 2 ( I − 11 ⊤ / n 1 ) / ( n 1 − 1 ) \mathbb E_\Pi[c_\Pi c_\Pi^\top\mid\mathcal F_n]=\lVert c\rVert^2(I-\mathbf1\mathbf1^\top/n_1)/(n_1-1) E Π [ c Π c Π ⊤ ∣ F n ] = ∥ c ∥ 2 ( I − 1 1 ⊤ / n 1 ) / ( n 1 − 1 ) H 1 2 = H 1 H_1^2=H_1 H 1 2 = H 1 H 1 1 = 1 H_1\mathbf1=\mathbf1 H 1 1 = 1 rank ( H 1 ) ≤ p \operatorname{rank}(H_1)\le p rank ( H 1 ) ≤ p
E Π [ 1 n 1 ∑ i { R i ( Π ) − e i c Π ( i ) } 2 ∣ F n ] ≤ max i e i 2 n 1 E Π [ ∥ H 1 c Π ∥ 2 ∣ F n ] = max i e i 2 n 1 ∥ c ∥ 2 n 1 − 1 { rank ( H 1 ) − 1 } = o P ( 1 ) . \begin{aligned}
\mathbb E_\Pi\left[\left.\frac{1}{n_1}\sum_i\{R_i^{(\Pi)}-e_ic_{\Pi(i)}\}^2\right|\mathcal F_n\right]
\le\frac{\max_i e_i^2}{n_1}\mathbb E_\Pi[\lVert H_1c_\Pi\rVert^2\mid\mathcal F_n] \\
&=\frac{\max_i e_i^2}{n_1}\frac{\lVert c\rVert^2}{n_1-1}\{\operatorname{rank}(H_1)-1\}
=o_{\mathbb P}(1).
\end{aligned} E Π [ n 1 1 i ∑ { R i ( Π ) − e i c Π ( i ) } 2 F n ] ≤ n 1 max i e i 2 E Π [∥ H 1 c Π ∥ 2 ∣ F n ] = n 1 max i e i 2 n 1 − 1 ∥ c ∥ 2 { rank ( H 1 ) − 1 } = o P ( 1 ) . Lemma 8(a) gives the mean-square error o P ( 1 ) o_{\mathbb P}(1) o P ( 1 ) O P ( 1 ) O_{\mathbb P}(1) O P ( 1 ) (120) and Lemma 4. Lemma 14 therefore yields S n 1 ( R ( Π ) ) / σ p , n 2 ⟶ p 1 S_{n_1}(R^{(\Pi)})/\sigma_{p,n}^2\mathop{\longrightarrow}^{p}1 S n 1 ( R ( Π ) ) / σ p , n 2 ⟶ p 1
∑ i R i ( Π ) n 1 S n 1 ( R ( Π ) ) ∣ F n ⟶ d , p N ( 0 , 1 ) . \left.\frac{\sum_i R_i^{(\Pi)}}{\sqrt{n_1S_{n_1}(R^{(\Pi)})}}\right|\mathcal F_n
\mathop{\longrightarrow}^{d,p}N(0,1). n 1 S n 1 ( R ( Π ) ) ∑ i R i ( Π ) F n ⟶ d , p N ( 0 , 1 ) . Multiplication by the F n \mathcal{F}_n F n β ^ \hat{\beta} β ^ P 0 ( n ) ( β ^ = 0 ) → 0 P_0^{(n)}(\hat{\beta}=0) \to0 P 0 ( n ) ( β ^ = 0 ) → 0 q n , α P C M , π → P 0 ( n ) z 1 − α q_{n,\alpha}^{\mathrm{PCM},\pi} \xrightarrow{P_0^{(n)}} z_{1-\alpha} q n , α PCM , π P 0 ( n ) z 1 − α
Finally, contiguity and Lemma 24 , applied to the continuous mixture in (116) , give (117) and (118) . The deterministic cutoff z 1 − α z_{1-\alpha} z 1 − α (116) ; the expression for equally sized folds follows from (115) and the definition of σ β \sigma_\beta σ β
