Introduction
In survey experiments, random assignment of treatment identifies the intention-to-treat (ITT) effect, but this estimand does not distinguish respondents who pay attention to or engage with the treatment from those who do not. To assess attentiveness and treatment engagement, researchers commonly administer attention or manipulation checks to determine, for example, whether respondents carefully read instructions or understood the intended manipulation (e.g., Berinsky et al., [4 ] ; Kane and Barabas, [20 ] ). Although informative, these checks are imperfect measures of respondents’ underlying attentiveness or engagement. Some inattentive respondents may pass a check by chance, whereas successfully manipulated respondents may fail because they misunderstand the manipulation check questions. In addition, when checks are administered after treatment assignment, treatment itself may also affect measurement error (e.g., Montgomery et al., [27 ] ; Aronow et al., [3 ] ; Blackwell et al., [5 ] ). Therefore, restricting the analysis to respondents who pass these checks can introduce substantial bias.
These challenges raise a fundamental question: how can researchers use imperfect attention and manipulation checks to estimate treatment effects among respondents who would be attentive or successfully manipulated under treatment? We propose a general statistical framework for estimating this quantity, which we call the engaged average treatment effect (eATE), where engagement refers to attentiveness or successful manipulation, depending on the application. Our framework accommodates both pre-treatment engagement, which is unaffected by treatment assignment, and post-treatment engagement, which may depend on assignment. We show that the eATE can be identified using three potentially imperfect checks that are conditionally independent given true, unobserved engagement status, treatment assignment, and pre-treatment covariates.
Building on this identification result, we develop an estimation procedure that accommodates flexible covariate modeling and provides valid uncertainty quantification. Specifically, we use spline approximations to allow both engagement probabilities and the measurement qualities of each check to smoothly vary with respondent characteristics. We then propose a cross-fitted estimator with a valid asymptotic confidence interval that accounts for uncertainty in fitting the measurement model, leveraging the results from the semiparametric statistics literature (e.g., Tsiatis, [31 ] ; Chernozhukov et al., [9 ] ). Lastly, we also develop a statistical test that can be used to diagnose the potential violations of the relevance conditions under the conditional independence assumptions.
We apply the proposed methodology to two published experiments, one with attention checks and the other with post-treatment manipulation checks. In the civil liberties framing experiment of Berinsky et al. [4 ] , our approach yields a larger estimated effect on rally approval among attentive respondents than either the full-sample ITT estimate or the estimate among respondents who pass the observed attention checks. In the digital media literacy experiment of Guess et al. [14 ] , we estimate that the proportion of respondents who would be successfully manipulated exceeds the proportion who pass all three manipulation checks. The resulting eATE estimates are smaller in absolute magnitude than the instrumental variable estimates but larger than the ITT estimates. Together, these applications demonstrate how accounting for measurement error in attention and manipulation checks can materially change the estimated magnitude of treatment effects.
Related literature. This paper contributes to the literature on attention and manipulation checks by developing a new methodological framework that accommodates both pre- and post-treatment checks with measurement error. Although these checks are widely used in survey experiments across the social science, there exist few statistical methods that properly account for measurement error in these checks.
For attention checks, administering them before treatment avoids post-treatment selection bias, but the checks themselves may prime respondents and alter their subsequent responses. In contrast, restricting the analysis to respondents who pass a post-treatment check may introduce selection bias [27 , 3 ] . These concerns have motivated partial identification approaches to post-treatment selection [3 ] , imperfect screening [33 , 32 ] , and the trade-off between priming and post-treatment bias when measuring moderators [5 ] . Unlike these partial identification approaches, we propose a unified methodological framework that provides point identification of the eATE under the key conditional independence assumption regardless of whether checks are done before or after treatment. Our methodology also quantifies estimation uncertainty, providing asymptotically valid confidence intervals for the eATE estimates.
Mathur (2025b) takes a different approach, treating attention checks as tools for identifying respondents whose responses are free of measurement error. The author proposes adjusting for covariates that affect both attentiveness and the outcome when analyzing respondents who pass the checks. This approach, however, assumes that passing the attention checks accurately identifies attentive respondents. In contrast, our methodology accounts for measurement error that is inherent in attention checks.
Accounting for manipulation checks is more challenging because successful manipulation can only be assessed after treatment assignment. Aronow et al. (2019) and Blackwell et al. (2025) apply partial identification approaches to this post-treatment selection problem. Others, including Mathur (2025a), propose the use of instrumental variables to estimate the eATE where randomized treatment assignment serves as an instrument for the observed manipulation status. However, this approach requires the exclusion restriction that treatment assignment affects the outcome only through the observed manipulation status, essentially precluding the possibility of any measurement error in manipulation checks and the existence of alternative causal pathways. In contrast, our approach does not require any individual manipulation check to perfectly measure the underlying manipulation status and allows treatment assignment to affect the outcome directly among respondents who would be manipulated under both treatment conditions.
More broadly, Hazlett et al. (2026) study the eATE in the context of post-treatment selection, which they call the treatment reactive average causal effect (TRACE). The authors propose a partial identification approach that allows for a nonzero average treatment effect among respondents who would not react to the treatment. However, their approach does not explicitly account for measurement error in the observed post-treatment variable. In contrast, our methodology accounts for measurement error in manipulation checks and provides point identification of the eATE, by assuming a zero average treatment effect among respondents who would not be manipulated under either treatment condition.
Finally, the proposed methodology builds on the literature on measurement error and proxy-based identification, which exploits multiple imperfect measurements of a latent variable to identify causal effects under the assumptions of non-differential measurement error. Such approaches have been developed for unobserved confounders [21 , 26 , 30 , 10 ] , unobserved treatments [35 ] , and latent outcomes [15 ] , with a wide range of applications, including unobserved demographic characteristics [25 , 6 ] and text annotations [28 , 12 ] . We extend this approach to the setting of attention and manipulation checks in randomized experiments, where respondents’ underlying attentiveness or engagement is latent and each observed check may contain measurement error.
Organization of the paper. The rest of this paper proceeds as follows. Section 2 introduces two empirical applications. Section 3 presents the identification results. Section 4 then develops estimation and inference procedures. § 5 discusses a diagnostics test regarding the key identification assumption. § 6 then confirms the performance on the simulated data, and § 7 applies the proposed methodology to the motivating examples introduced earlier. Finally, § 8 concludes by discussing directions for future research.
Motivating Empirical Applications
This section introduces two motivating applications: one uses attention checks to assess respondents’ attentiveness, and the other uses manipulation checks to evaluate their engagement with the treatment stimulus. Although the checks are administered both before and after treatment in the first application and only after treatment in the second, our proposed methodology accommodates both settings, provided that at least three well-designed checks are available (see § 3 ). We revisit both applications in § 7 , where we analyze their data using the proposed methodology.
Civil Liberties Framing Experiment with Attention Checks
A central question in the study of public opinion is whether people’s preferences depend on how political issues are presented. In a classic study, Nelson et al. [29 ] examined this question through a survey experiment on civil liberties. Respondents read an article about a proposed Ku Klux Klan (KKK) rally that presented one of two frames. The free speech frame emphasized the KKK’s right to express its views and the public’s right to hear them, whereas the public safety frame emphasized the potential for violence and threats to public order. The authors found that respondents exposed to the free speech frame were more willing to allow the rally than those exposed to the public safety frame. This finding suggests that news framing can shape political tolerance by emphasizing different considerations.
Although the original study found a framing effect, respondent inattention may attenuate the observed effect. To explore this possibility, Berinsky et al. [4 ] replicated the civil liberties framing experiment using multiple attention checks. Figure 1 illustrates the experimental design.
Figure 1. Experimental design and attention checks in the civil liberties framing experiment of Berinsky et al. [4 ] . Respondents read a news article emphasizing either public safety or free speech. They then report whether the university should allow the KKK rally. The authors measured the attentiveness using attention checks.
The authors administered four attention checks throughout the survey, randomizing their order for each respondent. The checks asked about respondents’ favorite color, current feelings, website preferences, and political interest. Rather than answering these questions directly, respondents were instructed to select specific options to demonstrate that they had read the instructions carefully. For example, the favorite-color question instructed respondents to select both red and green, regardless of their actual preferences. The first screener was administered before the framing experiment, whereas the remaining three were administered afterwards.
This example highlights an important limitation of attention checks. As Berinsky et al. [4 ] acknowledge, these checks are imperfect measures of respondents’ underlying attentiveness. Inattentive respondents may pass by chance, whereas attentive respondents may fail because they misread the instructions or make a response error. Indeed, Table 1 shows only modest correlations and varying pass rates across respondents’ performance on individual checks. Therefore, respondents who pass the checks need not be representative of truly attentive respondents. Without additional restrictions on the measurement error process, the resulting bias can be large and operate in either direction.
Table 1. Pairwise Pearson correlations and marginal pass rates for the four attention checks used in the experiment of Berinsky et al. [4 ] (based on the civil liberties framing experiment, Wave 2 of Study 1). Web, Interest, Color, and Feeling refer to attention checks that ask about respondents’ website preferences, political interest, favorite color, and current feelings, respectively. Each attention check is coded as one if the respondent correctly follows the instructions and zero otherwise.
Although Berinsky et al. [4 ] recommend using multiple checks, aggregating multiple imperfect measures does not, by itself, resolve the measurement error problem. In their analysis, the authors count the number of checks each respondent passes and report treatment effects separately for each count. They find that the estimated effect is attenuated for those who passed 0 or 1 check when compared to those who passed all checks. However, it is unclear whether this difference reflects a true difference in attentiveness or other sources of heterogeneity, such as differences in respondents’ unobserved characteristics that are correlated with both their attentiveness and their treatment effect.
This study calls for a statistical framework that accounts for measurement error problems of attention checks while allowing for heterogeneity in treatment effects across respondents with different levels of attentiveness.
Digital Media Literacy Experiment with Manipulation Checks
The spread of misinformation on social media has raised growing concerns about citizens’ ability to evaluate political information ([1 , 22 ] ). One potential solution is to improve individuals’ digital media literacy. To investigate the effectiveness of this approach, Guess et al. ((2020)) examine whether a brief digital media literacy intervention improves people’s ability to distinguish mainstream news from false news. In this experiment, respondents are randomly assigned to the treatment condition that asks them to read ten tips from Facebook’s campaign for identifying false news or to the control condition that provides no tips. The tips encourage readers to scrutinize web addresses and news sources, examine supporting evidence, and compare reports across outlets. Respondents subsequently rate the accuracy of news headlines on a four-point scale. The outcome of interest is the difference between respondents’ perceived accuracy ratings for mainstream and false news headlines.
A fundamental challenge in this study is that treatment assignment does not necessarily guarantee successful intervention. The effectiveness of the intervention depends on whether respondents pay attention to the tips and understand the advice. To assess whether respondents successfully receive the intervention, Guess et al. [14 ] administer three manipulation checks in the form of comprehension questions that cover three aspects of the ten tips after the intervention (see Figure 2 for the illustration of the experimental design). The authors define successful manipulation as correctly answering all three questions on the first attempt. However, as shown in Table 2 , these checks are only moderately correlated. Moreover, even though the pass rate for each check is relatively high, only 66% of treated respondents passed all three checks.
Figure 2. Experimental design and manipulation checks in the digital media literacy experiment of Guess et al. [14 ] . Respondents receive the digital media literacy intervention and evaluate the accuracy of mainstream and false news headlines. True manipulation is modeled as a latent post-treatment variable that affects respondents’ outcomes and their performance on the observed manipulation checks. The direct arrow from treatment to outcome allows treatment effects beyond those operating through manipulation status.
Table 2. Pairwise Pearson correlations and pass rates for the three manipulation checks used in the digital media literacy experiment of Guess et al. [14 ] . URLs, Misspellings, and Sources refer to checks assessing respondents’ understanding of misleading web addresses, spelling errors, and corroboration by multiple trusted sources, respectively. Each check is coded as one if the respondent correctly answers the question on the first attempt and zero otherwise. The last row of panel (b) reports the proportion of respondents who passed all three checks.
In addition to potential measurement error in the observed manipulation checks, this example highlights additional challenges of using post-treatment checks. Since the checks are conducted after treatment assignment, subgroup analyses restricted to respondents who pass the checks may suffer from post-treatment selection bias (Montgomery et al. [27 ] ; Aronow et al. [3 ] ). As a result, a common approach in this context is to use instrumental variables (e.g., Mathur [23 ] ). However, the validity of the instrumental variable approach relies on the exclusion restriction, which may be difficult to justify in many applications, including this study. For example, respondents who fail to correctly answer certain comprehension questions may nevertheless understand and apply some of the tips, improving their ability to distinguish mainstream news from false news. Therefore, treatment assignment may affect the outcome even among respondents classified as failing the manipulation checks, violating the exclusion restriction.
This study motivates the development of a statistical framework that accounts for imperfect measurement of manipulation status while avoiding the full exclusion restriction required by the standard instrumental variable approach.
The Proposed Framework
In this section, we introduce the proposed methodology for estimating the engaged average treatment effect (eATE) in survey experiments with imperfect attention or manipulation checks. We begin by defining the eATE as the causal quantity of interest and then establish its identification under the key conditional independence assumption. We then briefly discuss how the proposed methodology relaxes the exclusion restriction assumption required by the standard instrumental variable approach.
The engaged Average Treatment Effect (eATE)
Consider a simple random sample of n n n respondents from a population of interest. For each respondent i = 1 , … , n i = 1, \ldots, n i = 1 , … , n , let T i ∈ T = { 0 , 1 } T_i \in\mathcal{T} = \{0,1\} T i ∈ T = { 0 , 1 } denote the treatment assignment, where T i = 1 T_i = 1 T i = 1 indicates treatment and T i = 0 T_i = 0 T i = 0 indicates the control condition. We may also observe the pre-treatment covariates, which we denote X i X_i X i when available. Let Y i ( t ) Y_i(t) Y i ( t ) denote the potential outcome under treatment assignment t ∈ T t \in\mathcal{T} t ∈ T . We use Y i = Y i ( T i ) Y_i = Y_i(T_i) Y i = Y i ( T i ) to denote the observed outcome.
Our causal quantity of interest is the average treatment effect among respondents who are attentive or successfully manipulated, which we call the engaged average treatment effect (eATE). Because our framework encompasses both attention and manipulation checks, we use the term engagement to refer to attention or successful manipulation, depending on the application.
Formally, let W i ( t ) ∈ { 0 , 1 } W_i(t) \in\{0,1\} W i ( t ) ∈ { 0 , 1 } denote the potential engagement status under treatment assignment t t t , where W i ( t ) = 1 W_i(t) = 1 W i ( t ) = 1 indicates that respondent i i i would be engaged under this assignment. We denote the realized engagement status by W i = W i ( T i ) W_i = W_i(T_i) W i = W i ( T i ) . This notation accommodates both pre-treatment attention and post-treatment manipulation. When engagement represents pre-treatment attention, treatment assignment does not affect engagement status so that W i ( 1 ) = W i ( 0 ) = W i W_i(1) = W_i(0) = W_i W i ( 1 ) = W i ( 0 ) = W i for every respondent. In contrast, when engagement represents post-treatment manipulation, engagement status may depend on treatment assignment, allowing W i ( 1 ) ≠ W i ( 0 ) W_i(1) \ne W_i(0) W i ( 1 ) = W i ( 0 ) for some respondents. We begin by making two standard assumptions about survey experiments. First, we assume consistency, requiring that each respondent’s potential outcomes and engagement status depend only on their own treatment assignment. This assumption rules out spillover effects between respondents.
Assumption 1 (Consistency). The observed outcome and realized engagement status satisfy
Y i = Y i ( T i ) and W i = W i ( T i ) , for all i = 1 , … , n . Y_i = Y_i(T_i) \quad\text{and} \quad W_i = W_i(T_i), \quad\text{for all } i = 1,\ldots,n. Y i = Y i ( T i ) and W i = W i ( T i ) , for all i = 1 , … , n .
Second, we assume that treatment is randomly assigned, independently of all potential quantities conditional on pre-treatment covariates. This accommodates a variety of settings, including complete and stratified randomization.
Assumption 2 (Randomization of Treatment Assignment). Treatment assignment satisfies
T i ⊥ ⊥ { Y i ( 1 ) , Y i ( 0 ) , W i ( 1 ) , W i ( 0 ) } ∣ X i , T_i \perp\!\!\!\perp\{Y_i(1),Y_i(0),W_i(1),W_i(0)\} \mid X_i, T i ⊥ ⊥ { Y i ( 1 ) , Y i ( 0 ) , W i ( 1 ) , W i ( 0 )} ∣ X i , with the known assignment probability π t ( x ) : = Pr ( T i = t ∣ X i = x ) > 0 \pi_t(x) := \Pr(T_i = t \mid X_i = x) > 0 π t ( x ) := Pr ( T i = t ∣ X i = x ) > 0 for all x x x and t ∈ T t \in\mathcal{T} t ∈ T .
Under Assumptions 1 and 2, the average treatment effect (ATE) is identified by the mean difference in outcome between the treatment and control groups with necessary covariate adjustment. Our primary quantity of interest, however, is the average treatment effect among respondents who would be engaged under treatment, or the engaged average treatment effect (eATE), which we define as:
Ψ : = E [ Y i ( 1 ) − Y i ( 0 ) ∣ W i ( 1 ) = 1 ] . (1) \Psi:= \mathbb{E}[Y_i(1) - Y_i(0) \mid W_i(1) = 1]. \tag*{(1)} Ψ := E [ Y i ( 1 ) − Y i ( 0 ) ∣ W i ( 1 ) = 1 ] . (1)
When engagement represents pre-treatment attention, Ψ \Psi Ψ reduces to the average treatment effect among attentive respondents, i.e., Ψ = E [ Y i ( 1 ) − Y i ( 0 ) ∣ W i = 1 ] \Psi= \mathbb{E}[Y_i(1) - Y_i(0) \mid W_i = 1] Ψ = E [ Y i ( 1 ) − Y i ( 0 ) ∣ W i = 1 ] . In this case, the target subgroup is the same under both treatment conditions.
When engagement represents post-treatment manipulation, Ψ \Psi Ψ instead represents the average treatment effect among respondents who would be successfully manipulated if assigned to treatment, regardless of their engagement status under control. In the literature on manipulation checks, some researchers have studied Ψ \Psi Ψ using a partial identification approach [16 ] . However, Ψ \Psi Ψ is different from another popular causal quantity of interest in the literature, which is the average effect among “always-manipulated” respondents [3 , 32 ] , defined as,
E [ Y i ( 1 ) − Y i ( 0 ) ∣ W i ( 1 ) = W i ( 0 ) = 1 ] . \mathbb{E}[Y_i(1) - Y_i(0) \mid W_i(1) = W_i(0) = 1]. E [ Y i ( 1 ) − Y i ( 0 ) ∣ W i ( 1 ) = W i ( 0 ) = 1 ] .
This quantity targets respondents who would be manipulated regardless of treatment assignment, but this subgroup is often small and may not even exist in some experiments where the control condition is no manipulation (rather than comparing two different manipulations). In addition, membership in this subgroup is never directly observed [13 ] . As a consequence, the identification of this quantity requires a stringent assumption, such as principal ignorability [11 ] , or necessitates a partial identification approach (e.g., [3 , 32 ] .
Identification when Engagement Status is Observed
For the sake of exposition, we first consider the case where the (true) engagement status W i W_i W i is observed without measurement error. Under Assumptions 1 and 2, the ATE is identified by the mean difference in outcome between the treatment and control groups with covariate adjustment:
τ : = E [ Y i ( 1 ) − Y i ( 0 ) ] = E [ E [ Y i ∣ T i = 1 , X i ] − E [ Y i ∣ T i = 0 , X i ] ] . \tau:= \mathbb{E}[Y_i(1) - Y_i(0)] = \mathbb{E}\left[\mathbb{E}[Y_i \mid T_i = 1, X_i] - \mathbb{E}[Y_i \mid T_i = 0, X_i]\right]. τ := E [ Y i ( 1 ) − Y i ( 0 )] = E [ E [ Y i ∣ T i = 1 , X i ] − E [ Y i ∣ T i = 0 , X i ] ] .
We show that, under two additional assumptions, this ATE can be rescaled to identify the eATE. First, we assume that treatment does not, on average, affect respondents who would not be engaged under either treatment condition. For pre-treatment attention, this requires inattentive respondents to be unaffected by treatment on average. For post-treatment manipulation, the same restriction applies to respondents who would not be successfully manipulated under either treatment condition. We emphasize that this assumption needs to hold only on average and does not require the treatment effect to be zero for every respondent in this subgroup.
Assumption 3 (Zero average treatment effect without engagement). Treatment has no average effect among respondents who would not be engaged under either assignment:
E [ Y i ( 1 ) − Y i ( 0 ) ∣ W i ( 1 ) = W i ( 0 ) = 0 ] = 0. \mathbb{E}[Y_i(1) - Y_i(0) \mid W_i(1) = W_i(0) = 0] = 0. E [ Y i ( 1 ) − Y i ( 0 ) ∣ W i ( 1 ) = W i ( 0 ) = 0 ] = 0.
Second, we assume that treatment cannot reduce engagement for any respondent, thereby ruling out respondents who would be engaged under the control condition but not under the treatment. This monotonicity condition is naturally satisfied in many survey experiments. For pre-treatment attention checks, the assumption holds trivially because W i ( 1 ) = W i ( 0 ) W_i(1) = W_i(0) W i ( 1 ) = W i ( 0 ) . For post-treatment manipulation checks, the assumption is satisfied in one-sided treatment designs where manipulation is only active under treatment, ensuring that control respondents are never engaged, i.e., W i ( 0 ) = 0 W_i(0) = 0 W i ( 0 ) = 0 for every respondent. The assumption is also applicable in two-sided treatment designs where manipulation is active under both treatment and control, provided that the treatment condition is designed to be more engaging for every respondent than the control condition.
Assumption 4 (Monotonicity). For every respondent,
W i ( 1 ) ≥ W i ( 0 ) . (2) W_i(1) \ge W_i(0). \tag*{(2)} W i ( 1 ) ≥ W i ( 0 ) . (2)
Under Assumption 4, respondents with W i ( 1 ) = 0 W_i(1) = 0 W i ( 1 ) = 0 are precisely those who would not be engaged under either treatment condition, i.e., W i ( 1 ) = W i ( 0 ) = 0 W_i(1) = W_i(0) = 0 W i ( 1 ) = W i ( 0 ) = 0 . Assumption 3 therefore implies that treatment has no average effect outside the target subgroup. Applying the law of iterated expectations gives
E [ Y i ( 1 ) − Y i ( 0 ) ] = E [ Y i ( 1 ) − Y i ( 0 ) ∣ W i ( 1 ) = 1 ] Pr ( W i ( 1 ) = 1 ) + E [ Y i ( 1 ) − Y i ( 0 ) ∣ W i ( 1 ) = 0 ] Pr ( W i ( 1 ) = 0 ) = Ψ Pr ( W i ( 1 ) = 1 ) . \begin{aligned}
\mathbb{E}[Y_i(1) - Y_i(0)] ={}& \mathbb{E}[Y_i(1) - Y_i(0) \mid W_i(1) = 1]\Pr(W_i(1) = 1) \\
&+ \mathbb{E}[Y_i(1) - Y_i(0) \mid W_i(1) = 0]\Pr(W_i(1) = 0) \\
={}& \Psi\Pr(W_i(1) = 1).
\end{aligned} E [ Y i ( 1 ) − Y i ( 0 )] = = E [ Y i ( 1 ) − Y i ( 0 ) ∣ W i ( 1 ) = 1 ] Pr ( W i ( 1 ) = 1 ) + E [ Y i ( 1 ) − Y i ( 0 ) ∣ W i ( 1 ) = 0 ] Pr ( W i ( 1 ) = 0 ) Ψ Pr ( W i ( 1 ) = 1 ) .
As consistency and randomization imply Pr ( W i ( 1 ) = 1 ) = E [ Pr ( W i = 1 ∣ T i = 1 , X i ) ] \Pr(W_i(1) = 1) = \mathbb{E}[\Pr(W_i = 1 \mid T_i = 1, X_i)] Pr ( W i ( 1 ) = 1 ) = E [ Pr ( W i = 1 ∣ T i = 1 , X i )] , we can identify the eATE as,
Ψ = E [ E [ Y i ∣ T i = 1 , X i ] − E [ Y i ∣ T i = 0 , X i ] ] E [ Pr ( W i = 1 ∣ T i = 1 , X i ) ] , (3) \Psi=
\frac{\mathbb{E}[\mathbb{E}[Y_i \mid T_i = 1, X_i] - \mathbb{E}[Y_i \mid T_i = 0, X_i]]}
{\mathbb{E}[\Pr(W_i = 1 \mid T_i = 1, X_i)]}, \tag*{(3)} Ψ = E [ Pr ( W i = 1 ∣ T i = 1 , X i )] E [ E [ Y i ∣ T i = 1 , X i ] − E [ Y i ∣ T i = 0 , X i ]] , (3)
provided that some treated respondents are engaged, i.e., Pr ( W i = 1 ∣ T i = 1 , X i = x ) > 0 \Pr(W_i = 1 \mid T_i = 1, X_i = x) > 0 Pr ( W i = 1 ∣ T i = 1 , X i = x ) > 0 (this assumption is formally stated below). We note that in this setting, the ATE never exceeds the eATE in magnitude since the denominator is bounded between 0 and 1. This happens because we assume no effect without engagement (Assumption 3).
Recovering Latent Engagement Status from Multiple Checks
The preceding identification result was obtained under the assumption that the true engagement status is directly observed. However, as discussed in § 2 , this assumption is too stringent in practice because attention and manipulation checks are frequently subject to measurement error. Fortunately, we show that it is possible to recover the proportion of truly engaged respondents from multiple noisy attention or manipulation checks. The use of multiple checks is a common setup in applied research and is recommended as a best practice [4 ] . We require three checks in order to identify the eATE.
Specifically, for each respondent i i i , let W ~ j i \widetilde{W}_{ji} W j i denote the j j j th observed check, where j ∈ { 1 , 2 , 3 } j \in\{1,2,3\} j ∈ { 1 , 2 , 3 } . These checks may be binary, ordinal, or continuous. Our goal is to recover the proportion of respondents who would be engaged under treatment, Pr ( W i ( 1 ) = 1 ) \Pr(W_i(1)=1) Pr ( W i ( 1 ) = 1 ) , which appears in the denominator of Equation (3) .
We impose two assumptions regarding the quality of checks. First, we assume that, among the treated respondents, the measurement errors of the checks are mutually independent given true engagement status and pre-treatment characteristics. This assumption is formalized as follows:
Assumption 5 (Conditional independence of measurement errors among checks).
W ~ 1 i ⊥ ⊥ W ~ 2 i ⊥ ⊥ W ~ 3 i ∣ W i , T i = 1 , X i . (4) \widetilde{W}_{1i} \perp\!\!\!\perp\widetilde{W}_{2i} \perp\!\!\!\perp\widetilde{W}_{3i} \mid W_i,T_i=1,X_i. \tag*{(4)} W 1 i ⊥ ⊥ W 2 i ⊥ ⊥ W 3 i ∣ W i , T i = 1 , X i . (4) Or equivalently, Pr ( W ~ 1 i , W ~ 2 i , W ~ 3 i ∣ W i , T i = 1 , X i ) = Pr ( W ~ 1 i ∣ W i , T i = 1 , X i ) Pr ( W ~ 2 i ∣ W i , T i = 1 , X i ) Pr ( W ~ 3 i ∣ W i , T i = 1 , X i ) \Pr(\widetilde{W}_{1i},\widetilde{W}_{2i},\widetilde{W}_{3i}\mid W_i,T_i=1,X_i)=\Pr(\widetilde{W}_{1i}\mid W_i,T_i=1,X_i)\Pr(\widetilde{W}_{2i}\mid W_i,T_i=1,X_i)\Pr(\widetilde{W}_{3i}\mid W_i,T_i=1,X_i) Pr ( W 1 i , W 2 i , W 3 i ∣ W i , T i = 1 , X i ) = Pr ( W 1 i ∣ W i , T i = 1 , X i ) Pr ( W 2 i ∣ W i , T i = 1 , X i ) Pr ( W 3 i ∣ W i , T i = 1 , X i ) .
Figure 3 graphically illustrates the requirement of Assumption 5. Among the treated respondents, we allow the checks to be associated with one another, but only through the true latent engagement status and the observed pre-treatment covariates. For example, a common comprehension process that affects two or more checks beyond W i W_i W i and X i X_i X i would violate the required assumption.
Figure 3. Directed Acyclic Graph for the proposed methodology. The dashed circle denotes the unobserved true engagement status. We assume that, conditional on W W W , X X X , and T = 1 T = 1 T = 1 , the three checks are mutually independent. The three checks (W ~ 1 \widetilde{W}_1 W 1 , W ~ 2 \widetilde{W}_2 W 2 , and W ~ 3 \widetilde{W}_3 W 3 ) and the engagement status W W W can be pre-treatment, in which case there are no direct arrows from T T T to these variables.
Second, we require that, among treated respondents, the true latent engagement variable can take both values within each covariate stratum, and checks can distinguish engaged from unengaged respondents on average. This condition is analogous to the relevance condition in the instrumental variable approach. Specifically, among treated respondents with the same pre-treatment covariates, engaged respondents must have a higher average score on each check than unengaged ones, meaning that the engaged respondents are more likely to pass than unengaged respondents.
Assumption 6 (Positivity and Relevance). For almost every x x x , the following conditions hold:
(i) (Positivity) 0 < Pr ( W i = 1 ∣ T i = 1 , X i = x ) < 1 0<\Pr(W_i=1\mid T_i=1,X_i=x)<1 0 < Pr ( W i = 1 ∣ T i = 1 , X i = x ) < 1 .
(ii) (Relevance of checks) For j ∈ { 1 , 2 , 3 } j\in\{1,2,3\} j ∈ { 1 , 2 , 3 } ,
E [ W ~ j i ∣ W i = 1 , T i = 1 , X i = x ] > E [ W ~ j i ∣ W i = 0 , T i = 1 , X i = x ] . \mathbb{E}[\widetilde{W}_{ji}\mid W_i=1,T_i=1,X_i=x]>\mathbb{E}[\widetilde{W}_{ji}\mid W_i=0,T_i=1,X_i=x]. E [ W j i ∣ W i = 1 , T i = 1 , X i = x ] > E [ W j i ∣ W i = 0 , T i = 1 , X i = x ] .
Under Assumption 5, these conditions have observable implications, which we leverage to develop a statistical test of whether they hold in a given application (see § 5 ).
Under these assumptions, we can identify the conditional probability of engagement among treated respondents.
Proposition 1 (Recovery of latent engagement probabilities). Suppose that Assumptions 5 and 6 hold. Then, for almost every x x x , the conditional engagement probability is identified as
Pr ( W i = 1 ∣ T i = 1 , X i = x ) = 1 − e 0 ⊤ G 1 ( 1 , x ) − 1 E [ V 1 i ∣ T i = 1 , X i = x ] , (5) \Pr(W_i=1\mid T_i=1,X_i=x)=1-e_0^\top G_1(1,x)^{-1}\mathbb{E}[V_{1i}\mid T_i=1,X_i=x], \tag*{(5)} Pr ( W i = 1 ∣ T i = 1 , X i = x ) = 1 − e 0 ⊤ G 1 ( 1 , x ) − 1 E [ V 1 i ∣ T i = 1 , X i = x ] , (5)
where e 0 = ( 1 , 0 ) ⊤ \mathbf{e}_0 = (1, 0)^\top e 0 = ( 1 , 0 ) ⊤ , V 1 i : = ( 1 , W ~ 1 i ) ⊤ \mathbf{V}_{1i} := (1, \widetilde{W}_{1i})^\top V 1 i := ( 1 , W 1 i ) ⊤ , and G 1 ( 1 , x ) \mathbf{G}_1(1, x) G 1 ( 1 , x ) is the matrix of conditional mean vectors defined in Equation (8) below and identifiable from the observed data. If, in addition, Assumptions 1–4 hold, then we have,
Ψ = E [ E [ Y i ∣ T i = 1 , X i ] − E [ Y i ∣ T i = 0 , X i ] ] E [ Pr ( W i = 1 ∣ T i = 1 , X i ) ] . (6) \Psi= \frac{\mathbb{E}[\mathbb{E}[Y_i \mid T_i = 1, X_i] - \mathbb{E}[Y_i \mid T_i = 0, X_i]]}{\mathbb{E}[\Pr(W_i = 1 \mid T_i = 1, X_i)]}. \tag*{(6)} Ψ = E [ Pr ( W i = 1 ∣ T i = 1 , X i )] E [ E [ Y i ∣ T i = 1 , X i ] − E [ Y i ∣ T i = 0 , X i ]] . (6)
Appendix S1.1 provides the proof of the proposition and the recovery and calibration formulas used for estimation. This result builds on the growing literature on measurement error and proxy variables and their applications to social science research (e.g., Kuroki and Pearl [21 ] ; Zhou and Tchetgen Tchetgen [35 ] ; McCartan et al. [25 ] ; Nakamura [28 ] ; Chasalow et al. [6 ] ; Egami and Shin [12 ] ).
The essence of the identification argument relies on the following factorization of the conditional moments of the checks used in the literature (see, e.g., Hu [17 ] ; Zhou and Tchetgen Tchetgen [35 ] ; Nakamura [28 ] ; Egami and Shin [12 ] ; Guo et al. [15 ] , for similar ideas). For j = 1 , 2 j = 1, 2 j = 1 , 2 , let V j i : = ( 1 , W ~ j i ) ⊤ \mathbf{V}_{ji} := (1, \widetilde{W}_{ji})^\top V j i := ( 1 , W j i ) ⊤ , which augments the j j j th check with a constant. By the law of iterated expectations and the conditional independence of the first two checks under Assumption 5, we have
E [ V 1 i V 2 i ⊤ ∣ T i = 1 , X i = x ] = ∑ w = 0 1 E [ V 1 i V 2 i ⊤ ∣ W i = w , T i = 1 , X i = x ] Pr ( W i = w ∣ T i = 1 , X i = x ) = ∑ w = 0 1 E [ V 1 i ∣ W i = w , T i = 1 , X i = x ] E [ V 2 i ∣ W i = w , T i = 1 , X i = x ] ⊤ Pr ( W i = w ∣ T i = 1 , X i = x ) = G 1 ( 1 , x ) [ Pr ( W i = 0 ∣ T i = 1 , X i = x ) 0 0 Pr ( W i = 1 ∣ T i = 1 , X i = x ) ] G 2 ( 1 , x ) ⊤ , (7) \begin{aligned}
\mathbb{E}[\mathbf{V}_{1i}\mathbf{V}_{2i}^{\top} \mid T_i = 1, X_i = x]
&= \sum_{w=0}^{1} \mathbb{E}[\mathbf{V}_{1i}\mathbf{V}_{2i}^{\top} \mid W_i = w, T_i = 1, X_i = x]\Pr(W_i = w \mid T_i = 1, X_i = x) \\
&= \sum_{w=0}^{1} \mathbb{E}[\mathbf{V}_{1i} \mid W_i = w, T_i = 1, X_i = x]\mathbb{E}[\mathbf{V}_{2i} \mid W_i = w, T_i = 1, X_i = x]^\top\Pr(W_i = w \mid T_i = 1, X_i = x) \\
&= \mathbf{G}_1(1,x)
\begin{bmatrix}
\Pr(W_i = 0 \mid T_i = 1, X_i = x) & 0 \\
0 & \Pr(W_i = 1 \mid T_i = 1, X_i = x)
\end{bmatrix}
\mathbf{G}_2(1,x)^\top, \tag*{(7)}
\end{aligned} E [ V 1 i V 2 i ⊤ ∣ T i = 1 , X i = x ] = w = 0 ∑ 1 E [ V 1 i V 2 i ⊤ ∣ W i = w , T i = 1 , X i = x ] Pr ( W i = w ∣ T i = 1 , X i = x ) = w = 0 ∑ 1 E [ V 1 i ∣ W i = w , T i = 1 , X i = x ] E [ V 2 i ∣ W i = w , T i = 1 , X i = x ] ⊤ Pr ( W i = w ∣ T i = 1 , X i = x ) = G 1 ( 1 , x ) [ Pr ( W i = 0 ∣ T i = 1 , X i = x ) 0 0 Pr ( W i = 1 ∣ T i = 1 , X i = x ) ] G 2 ( 1 , x ) ⊤ , (7)
for almost every x x x , where, for j = 1 , 2 j = 1, 2 j = 1 , 2 and t ∈ { 0 , 1 } t \in\{0,1\} t ∈ { 0 , 1 } , we define
G j ( t , x ) : = [ 1 E [ W ~ j i ∣ W i = 0 , T i = t , X i = x ] 1 E [ W ~ j i ∣ W i = 1 , T i = t , X i = x ] E [ W ~ j i ∣ W i = 0 , T i = t , X i = x ] E [ W ~ j i ∣ W i = 1 , T i = t , X i = x ] ] . (8) G_j(t,x) := \begin{bmatrix}
\frac{1}{\mathbb{E}[\widetilde{W}_{ji}\mid W_i=0,T_i=t,X_i=x]} & \frac{1}{\mathbb{E}[\widetilde{W}_{ji}\mid W_i=1,T_i=t,X_i=x]} \\
\mathbb{E}[\widetilde{W}_{ji}\mid W_i=0,T_i=t,X_i=x] & \mathbb{E}[\widetilde{W}_{ji}\mid W_i=1,T_i=t,X_i=x]
\end{bmatrix}. \tag*{(8)} G j ( t , x ) := [ E [ W j i ∣ W i = 0 , T i = t , X i = x ] 1 E [ W j i ∣ W i = 0 , T i = t , X i = x ] E [ W j i ∣ W i = 1 , T i = t , X i = x ] 1 E [ W j i ∣ W i = 1 , T i = t , X i = x ] ] . (8)
Here, the columns of G j ( t , x ) G_j(t,x) G j ( t , x ) contain the conditional mean vectors of V j i V_{ji} V j i for unengaged and engaged respondents, respectively. Thus, the observed moment matrix is a weighted sum of the products of these subgroup-specific mean vectors, with weights given by the subgroup probabilities.
Assumption 5 also implies that the third check is jointly independent of the first two checks conditional on engagement status and covariates among treated respondents. The conditional moment weighted by W ~ 3 i \widetilde{W}_{3i} W 3 i therefore admits another factorization:
E [ W ~ 3 i V 1 i V 2 i T ∣ T i = 1 , X i = x ] = G 1 ( 1 , x ) [ Pr ( W i = 0 ∣ T i = 1 , X i = x ) m 1 ( 0 , x ) 0 0 Pr ( W i = 1 ∣ T i = 1 , X i = x ) m 1 ( 1 , x ) ] G 2 ( 1 , x ) T , (9) \mathbb{E}[\widetilde{W}_{3i}V_{1i}V_{2i}^{\mathsf{T}}\mid T_i=1,X_i=x]
=G_1(1,x)
\begin{bmatrix}
\Pr(W_i=0\mid T_i=1,X_i=x)m_1(0,x) & 0 \\
0 & \Pr(W_i=1\mid T_i=1,X_i=x)m_1(1,x)
\end{bmatrix}
G_2(1,x)^{\mathsf{T}}, \tag*{(9)} E [ W 3 i V 1 i V 2 i T ∣ T i = 1 , X i = x ] = G 1 ( 1 , x ) [ Pr ( W i = 0 ∣ T i = 1 , X i = x ) m 1 ( 0 , x ) 0 0 Pr ( W i = 1 ∣ T i = 1 , X i = x ) m 1 ( 1 , x ) ] G 2 ( 1 , x ) T , (9)
where m 1 ( w , x ) : = E [ W ~ 3 i ∣ W i = w , T i = 1 , X i = x ] m_1(w,x):=\mathbb{E}[\widetilde{W}_{3i}\mid W_i=w,T_i=1,X_i=x] m 1 ( w , x ) := E [ W 3 i ∣ W i = w , T i = 1 , X i = x ] . The left-hand sides of Equations (7) and (9) are identified from the observed distribution, whereas their right-hand sides contain the same unknown subgroup probabilities and check means. The only difference is that the second factorization additionally weights each subgroup by its conditional mean of the third check.
This common structure allows us to separate the unknown quantities. Specifically, multiplying the second moment matrix on the right by the inverse of the first cancels the subgroup probabilities and G 2 ( 1 , x ) G_2(1,x) G 2 ( 1 , x ) :
E [ W ~ 3 i V 1 i V 2 i T ∣ T i = 1 , X i = x ] E [ V 1 i V 2 i T ∣ T i = 1 , X i = x ] − 1 = G 1 ( 1 , x ) [ m 1 ( 0 , x ) 0 0 m 1 ( 1 , x ) ] G 1 ( 1 , x ) − 1 . \mathbb{E}[\widetilde{W}_{3i}V_{1i}V_{2i}^{\mathsf{T}}\mid T_i=1,X_i=x]\mathbb{E}[V_{1i}V_{2i}^{\mathsf{T}}\mid T_i=1,X_i=x]^{-1}
=G_1(1,x)
\begin{bmatrix}
m_1(0,x) & 0 \\
0 & m_1(1,x)
\end{bmatrix}
G_1(1,x)^{-1}. E [ W 3 i V 1 i V 2 i T ∣ T i = 1 , X i = x ] E [ V 1 i V 2 i T ∣ T i = 1 , X i = x ] − 1 = G 1 ( 1 , x ) [ m 1 ( 0 , x ) 0 0 m 1 ( 1 , x ) ] G 1 ( 1 , x ) − 1 .
Here, the right-hand side is an eigenvalue decomposition (EVD) of a matrix identified from the observed distribution. Although an eigendecomposition is not unique in general, three features of this problem make identification possible. First, the relevance condition regarding the third check in Assumption 6 ensures that the eigenvalues are distinct, identifying the eigenvectors up to scale and order. Second, because the first entry of each column of G 1 ( 1 , x ) G_1(1,x) G 1 ( 1 , x ) equals one, normalizing each eigenvector to have first entry one fixes its scale. Finally, the strict ordering of check means in Assumption 6 then assigns the eigenvector with the larger second entry to engaged respondents. This identifies the matrix of check means G 1 ( 1 , x ) G_1(1,x) G 1 ( 1 , x ) , and this is sufficient to identify the conditional engagement probability Pr ( W i = 1 ∣ T i = 1 , X i = x ) \Pr(W_i=1\mid T_i=1,X_i=x) Pr ( W i = 1 ∣ T i = 1 , X i = x ) since
E [ V 1 i ∣ T i = 1 , X i = x ] = ∑ w E [ V 1 i ∣ W i = w , T i = 1 , X i = x ] Pr ( W i = w ∣ T i = 1 , X i = x ) = G 1 ( 1 , x ) [ Pr ( W i = 0 ∣ T i = 1 , X i = x ) Pr ( W i = 1 ∣ T i = 1 , X i = x ) ] . \begin{aligned}
\mathbb{E}[V_{1i}\mid T_i=1,X_i=x]
=\sum_w \mathbb{E}[V_{1i}\mid W_i=w,T_i=1,X_i=x]\Pr(W_i=w\mid T_i=1,X_i=x) \\
&=G_1(1,x)
\begin{bmatrix}
\Pr(W_i=0\mid T_i=1,X_i=x) \\
\Pr(W_i=1\mid T_i=1,X_i=x)
\end{bmatrix}.
\end{aligned} E [ V 1 i ∣ T i = 1 , X i = x ] = w ∑ E [ V 1 i ∣ W i = w , T i = 1 , X i = x ] Pr ( W i = w ∣ T i = 1 , X i = x ) = G 1 ( 1 , x ) [ Pr ( W i = 0 ∣ T i = 1 , X i = x ) Pr ( W i = 1 ∣ T i = 1 , X i = x ) ] .
Connection with Instrumental Variable Approach
We now discuss how the proposed approach is related to instrumental variables methods, which are often used to address the failure of manipulation ([23 ] ). Consider a setting in which W ~ i ( t ) = 1 \widetilde{W}_i(t)=1 W i ( t ) = 1 represents successful manipulation under treatment assignment T i = t T_i=t T i = t for t = 0 , 1 t=0,1 t = 0 , 1 . Under monotonicity, i.e., W i ( 1 ) ≥ W i ( 0 ) W_i(1) \ge W_i(0) W i ( 1 ) ≥ W i ( 0 ) , which is equivalent to Assumption 4, and the exclusion restriction (i.e., T i T_i T i affects Y i Y_i Y i only through W i W_i W i ), the complier average treatment effect can be identified as [2 ]
E [ Y i ( W i = 1 ) − Y i ( W i = 0 ) ∣ W i ( 1 ) = 1 , W i ( 0 ) = 0 ] = E [ E [ Y i ∣ T i = 1 , X i ] − E [ Y i ∣ T i = 0 , X i ] ] E [ Pr ( W i = 1 ∣ T i = 1 , X i ) − Pr ( W i = 1 ∣ T i = 0 , X i ) ] \mathbb{E}[Y_i(W_i=1)-Y_i(W_i=0)\mid W_i(1)=1,W_i(0)=0]
=
\frac{\mathbb{E}[\mathbb{E}[Y_i\mid T_i=1,X_i]-\mathbb{E}[Y_i\mid T_i=0,X_i]]}{\mathbb{E}[\Pr(W_i=1\mid T_i=1,X_i)-\Pr(W_i=1\mid T_i=0,X_i)]} E [ Y i ( W i = 1 ) − Y i ( W i = 0 ) ∣ W i ( 1 ) = 1 , W i ( 0 ) = 0 ] = E [ Pr ( W i = 1 ∣ T i = 1 , X i ) − Pr ( W i = 1 ∣ T i = 0 , X i )] E [ E [ Y i ∣ T i = 1 , X i ] − E [ Y i ∣ T i = 0 , X i ]]
This expression can also be applied using a proxy measure, so long as the proxy preserves the difference in mean engagement between the treatment and control conditions, i.e.,
E [ E [ W ~ 1 i ∣ T i = 1 , X i ] − E [ W ~ 1 i ∣ T i = 0 , X i ] ] = E [ E [ W i ∣ T i = 1 , X i ] − E [ W i ∣ T i = 0 , X i ] ] . \mathbb{E}[\mathbb{E}[\widetilde{W}_{1i}\mid T_i=1,X_i]-\mathbb{E}[\widetilde{W}_{1i}\mid T_i=0,X_i]]
=
\mathbb{E}[\mathbb{E}[W_i\mid T_i=1,X_i]-\mathbb{E}[W_i\mid T_i=0,X_i]]. E [ E [ W 1 i ∣ T i = 1 , X i ] − E [ W 1 i ∣ T i = 0 , X i ]] = E [ E [ W i ∣ T i = 1 , X i ] − E [ W i ∣ T i = 0 , X i ]] .
Importantly, under a one-sided treatment design (i.e., Pr ( W i ( 0 ) = 1 ) = 0 \Pr(W_i(0)=1)=0 Pr ( W i ( 0 ) = 1 ) = 0 ), both approaches target the same principal stratum.
This comparison highlights two key differences between the instrumental variable approach and our proposed methodology. First, our approach does not require the exclusion restriction, allowing treatment assignment to directly affect the outcome even when engagement status remains unchanged. In particular, we allow nonzero average treatment effect among respondents who would be engaged under both treatment conditions:
E [ Y i ( 1 ) − Y i ( 0 ) ∣ W i ( 1 ) = 1 , W i ( 0 ) = 1 ] ≠ 0. \mathbb{E}[Y_i(1)-Y_i(0)\mid W_i(1)=1,W_i(0)=1]\ne0. E [ Y i ( 1 ) − Y i ( 0 ) ∣ W i ( 1 ) = 1 , W i ( 0 ) = 1 ] = 0.
In contrast, the instrumental variable approach assumes that this average treatment effect is zero, which may be too restrictive in practice. For example, when treatment is designed to induce anxiety, some respondents may remain anxious regardless of treatment assignment. If the treatment affects the outcomes of these respondents, then the exclusion restriction will be violated.
Second, the instrumental variable approach ignores potential measurement error in manipulation checks. In contrast, our approach allows for biased proxies and does not place a restriction on the conditional distribution of each check given engagement status, treatment assignment, and pre-treatment covariates, so long as the conditional independence (Assumption 5) and relevance (Assumption 6) conditions are satisfied.
Estimation and Inference
In this section, building on the identification results above, we develop an estimation procedure that accommodates flexible covariate modeling and provides valid uncertainty quantification. To develop an estimator of Ψ \Psi Ψ and characterize its uncertainty, there are two challenges we must overcome. First, the identification argument recovers the measurement model separately at each covariate value X i = x X_i=x X i = x . When covariates are continuous or define many small strata, separately estimating the required conditional moments can be unstable. Second, inference must account for estimation of the latent engagement probabilities Pr ( W i = 1 ∣ T i = 1 , X i = x ) \Pr(W_i=1\mid T_i=1,X_i=x) Pr ( W i = 1 ∣ T i = 1 , X i = x ) and measurement functions m 1 ( w , x ) m_1(w,x) m 1 ( w , x ) .
We address these challenges by approximating the dependence on covariates using spline functions and then propagating first-stage estimation error using methods from semiparametric statistics. Formally, let q ( x ) : = Pr ( W i = 1 ∣ T i = 1 , X i = x ) q(x):=\Pr(W_i=1\mid T_i=1,X_i=x) q ( x ) := Pr ( W i = 1 ∣ T i = 1 , X i = x ) denote the conditional engagement probability and Q : = Pr ( W i ( 1 ) = 1 ) = E [ q ( X i ) ] Q:=\Pr(W_i(1)=1)=\mathbb{E}[q(X_i)] Q := Pr ( W i ( 1 ) = 1 ) = E [ q ( X i )] . Together with the four check means in G 1 ( 1 , x ) G_1(1,x) G 1 ( 1 , x ) and G 2 ( 1 , x ) G_2(1,x) G 2 ( 1 , x ) and the two third-check means m 1 ( 0 , x ) m_1(0,x) m 1 ( 0 , x ) and m 1 ( 1 , x ) m_1(1,x) m 1 ( 1 , x ) , there are seven unknown scalar functions of the covariates.
Let L n L_n L n denote the number of basis functions, which may increase with the sample size n n n , and let φ 1 ( x ) , … , φ L n ( x ) \varphi_1(x), \ldots, \varphi_{L_n}(x) φ 1 ( x ) , … , φ L n ( x ) be a user-specified spline basis satisfying φ ℓ ( x ) ≥ 0 \varphi_\ell(x) \ge0 φ ℓ ( x ) ≥ 0 and ∑ ℓ = 1 L n φ ℓ ( x ) = 1 \sum_{\ell=1}^{L_n} \varphi_\ell(x) = 1 ∑ ℓ = 1 L n φ ℓ ( x ) = 1 . Researchers may choose among different spline bases; in the empirical applications of § 7 , we use piecewise-linear (degree-one) spline bases. Specifically, we approximate the seven unknown scalar functions by
q ( x ; θ ) = ∑ ℓ = 1 L n φ ℓ ( x ) ω 1 ℓ , G j ( 1 , x ; θ ) = [ 1 ∑ ℓ = 1 L n φ ℓ ( x ) λ j 0 ℓ 1 ∑ ℓ = 1 L n φ ℓ ( x ) λ j 1 ℓ ] , m 1 ( w , x ; θ ) = ∑ ℓ = 1 L n φ ℓ ( x ) ξ w ℓ , (10) q(x;\theta)=\sum_{\ell=1}^{L_n}\varphi_\ell(x)\omega_{1\ell},\quad G_j(1,x;\theta)=\begin{bmatrix}\dfrac{1}{\sum_{\ell=1}^{L_n}\varphi_\ell(x)\lambda_{j0\ell}} & \dfrac{1}{\sum_{\ell=1}^{L_n}\varphi_\ell(x)\lambda_{j1\ell}}\end{bmatrix},\quad m_1(w,x;\theta)=\sum_{\ell=1}^{L_n}\varphi_\ell(x)\xi_{w\ell}, \tag*{(10)} q ( x ; θ ) = ℓ = 1 ∑ L n φ ℓ ( x ) ω 1 ℓ , G j ( 1 , x ; θ ) = [ ∑ ℓ = 1 L n φ ℓ ( x ) λ j 0 ℓ 1 ∑ ℓ = 1 L n φ ℓ ( x ) λ j 1 ℓ 1 ] , m 1 ( w , x ; θ ) = ℓ = 1 ∑ L n φ ℓ ( x ) ξ w ℓ , (10)
for j ∈ { 1 , 2 } j \in\{1,2\} j ∈ { 1 , 2 } and w ∈ { 0 , 1 } w \in\{0,1\} w ∈ { 0 , 1 } , and θ \theta θ is the parameter vector that collects { ω 1 ℓ , λ j 0 ℓ , λ j 1 ℓ , ξ 0 ℓ , ξ 1 ℓ } ℓ = 1 L n \{\omega_{1\ell},\lambda_{j0\ell},\lambda_{j1\ell},\xi_{0\ell},\xi_{1\ell}\}_{\ell=1}^{L_n} { ω 1 ℓ , λ j 0 ℓ , λ j 1 ℓ , ξ 0 ℓ , ξ 1 ℓ } ℓ = 1 L n for j = 1 , 2 j=1,2 j = 1 , 2 . We impose 0 < ω 1 ℓ < 1 0 < \omega_{1\ell} < 1 0 < ω 1 ℓ < 1 , λ j 0 ℓ < λ j 1 ℓ \lambda_{j0\ell} < \lambda_{j1\ell} λ j 0 ℓ < λ j 1 ℓ , and ξ 0 ℓ < ξ 1 ℓ \xi_{0\ell} < \xi_{1\ell} ξ 0 ℓ < ξ 1 ℓ for every ℓ \ell ℓ and j j j . Because the basis functions are nonnegative and sum to one, these restrictions ensure positivity and proxy relevance in Assumption 6. We use a common basis for convenience, but each function has its own coefficients, allowing different relationships with the covariates. However, using a common basis is not required, and different bases may be used for different functions.
We estimate θ \theta θ using the conditional moment restrictions in Equations (7) and (9) . Let ρ i ( θ ) \rho_i(\theta) ρ i ( θ ) be the eight-dimensional vector obtained by stacking the columns of the residual matrices, defined as,
ρ i ( θ ) : = [ vec { R 1 i ( θ ) } vec { R 2 i ( θ ) } ] ∈ R 8 , \rho_i(\theta):=\begin{bmatrix}\operatorname{vec}\{R_{1i}(\theta)\}\\ \operatorname{vec}\{R_{2i}(\theta)\}\end{bmatrix}\in\mathbb{R}^8, ρ i ( θ ) := [ vec { R 1 i ( θ )} vec { R 2 i ( θ )} ] ∈ R 8 ,
where vec ( ⋅ ) \operatorname{vec}(\cdot) vec ( ⋅ ) stacks the columns of a matrix, and
R 1 i ( θ ) : = V 1 i V 2 i T − G 1 ( 1 , X i ; θ ) [ 1 − q ( X i ; θ ) 0 0 q ( X i ; θ ) ] G 2 ( 1 , X i ; θ ) T , R_{1i}(\theta):=V_{1i}V_{2i}^{\mathsf{T}}-G_1(1,X_i;\theta)\begin{bmatrix}1-q(X_i;\theta)&0\\0&q(X_i;\theta)\end{bmatrix}G_2(1,X_i;\theta)^{\mathsf{T}}, R 1 i ( θ ) := V 1 i V 2 i T − G 1 ( 1 , X i ; θ ) [ 1 − q ( X i ; θ ) 0 0 q ( X i ; θ ) ] G 2 ( 1 , X i ; θ ) T ,
R 2 i ( θ ) : = W ~ 3 i V 1 i V 2 i T − G 1 ( 1 , X i ; θ ) [ { 1 − q ( X i ; θ ) } m 1 ( 0 , X i ; θ ) 0 0 q ( X i ; θ ) m 1 ( 1 , X i ; θ ) ] G 2 ( 1 , X i ; θ ) T . (11) R_{2i}(\theta):=\widetilde{W}_{3i}V_{1i}V_{2i}^{\mathsf{T}}-G_1(1,X_i;\theta)\begin{bmatrix}\{1-q(X_i;\theta)\}m_1(0,X_i;\theta)&0\\0&q(X_i;\theta)m_1(1,X_i;\theta)\end{bmatrix}G_2(1,X_i;\theta)^{\mathsf{T}}. \tag*{(11)} R 2 i ( θ ) := W 3 i V 1 i V 2 i T − G 1 ( 1 , X i ; θ ) [ { 1 − q ( X i ; θ )} m 1 ( 0 , X i ; θ ) 0 0 q ( X i ; θ ) m 1 ( 1 , X i ; θ ) ] G 2 ( 1 , X i ; θ ) T . (11)
When the candidate functions equal the population functions, the identification restrictions imply
E [ ρ i ( θ ) ∣ T i = 1 , X i ] = 0. (12) \mathbb{E}[\rho_i(\theta)\mid T_i=1,X_i]=0. \tag*{(12)} E [ ρ i ( θ ) ∣ T i = 1 , X i ] = 0. (12)
We therefore estimate the coefficients by minimizing a regularized sum of squared moment residuals among treated respondents:
θ ^ ∈ arg min θ ∈ Θ { 1 n ∑ i = 1 n 1 { T i = 1 } ∥ ρ i ( θ ) ∥ 2 2 + λ n Ω n ( θ ) } . (13) \widehat{\theta}\in\mathop{\arg\min}_{\theta\in\Theta}\left\{\frac{1}{n}\sum_{i=1}^{n}\mathbb{1}\{T_i=1\}\lVert\rho_i(\theta)\rVert_2^2+\lambda_n\Omega_n(\theta)\right\}. \tag*{(13)} θ ∈ arg min θ ∈ Θ { n 1 i = 1 ∑ n 1 { T i = 1 } ∥ ρ i ( θ ) ∥ 2 2 + λ n Ω n ( θ ) } . (13)
where Θ \Theta Θ is the parameter space, Ω n \Omega_n Ω n is the regularization term of the nuisance functions, and λ n ≥ 0 \lambda_n\ge0 λ n ≥ 0 controls the strength of regularization. The regularization term is especially important when researchers include a large number of pre-treatment covariates. Throughout the paper, we use the squared L 2 L_2 L 2 norm of adjacent coefficient differences used in Chen and Pouzo (2012):
Ω n = ∑ r = 1 R − 1 ∥ θ r + 1 − θ r ∥ 2 2 τ r + 1 − τ r , \Omega_n=\sum_{r=1}^{R-1}\frac{\lVert\theta_{r+1}-\theta_r\rVert_2^2}{\tau_{r+1}-\tau_r}, Ω n = r = 1 ∑ R − 1 τ r + 1 − τ r ∥ θ r + 1 − θ r ∥ 2 2 ,
where τ 1 < ⋯ < τ R \tau_1 < \cdots< \tau_R τ 1 < ⋯ < τ R are the R R R knots, including the boundary knots, and θ r \boldsymbol{\theta}_r θ r collects the nuisance-function coefficients at the rth knot. For a piecewise-linear basis, R = L n R = L_n R = L n , and it is recommended to choose L n L_n L n of order n 1 / 5 n^{1/5} n 1/5 and λ n \lambda_n λ n of order L n / n L_n/n L n / n [8 ] . However, researchers may use other forms of regularization depending on the application.
This objective is nonlinear due to the cross products of the spline basis and can be computationally challenging. However, we can simplify the computation by projecting the residual ρ i ( θ ) \boldsymbol{\rho}_i(\boldsymbol{\theta}) ρ i ( θ ) onto another sufficiently rich basis of X i X_i X i [7 , 8 ] . Specifically, we consider a separate spline basis h ( x ) \boldsymbol{h}(x) h ( x ) that spans the products of up to four original spline basis functions in Equation (10) (i.e., three check means and an engagement probability). Because ρ i ( θ ) \boldsymbol{\rho}_i(\boldsymbol{\theta}) ρ i ( θ ) contains products of at most four candidate functions, this basis represents every such moment exactly.
By the orthogonality of least squares, we can then write the residuals as the sum of projection and projection error:
1 n ∑ i = 1 n 1 { T i = 1 } ∥ ρ i ( θ ) ∥ 2 2 = 1 n ∑ i = 1 n ∑ r = 1 8 T i ∥ h ( X i ) ⊤ β ^ r ( θ ) ∥ 2 2 + 1 n ∑ i = 1 n ∑ r = 1 8 T i ∥ ρ i r ( θ ) − h ( X i ) ⊤ β ^ r ( θ ) ∥ 2 2 \frac{1}{n}\sum_{i=1}^{n}\mathbb{1}\{T_i=1\}\lVert\boldsymbol{\rho}_i(\boldsymbol{\theta})\rVert_2^2
=
\frac{1}{n}\sum_{i=1}^{n}\sum_{r=1}^{8}T_i\left\lVert\boldsymbol{h}(X_i)^\top\widehat{\boldsymbol{\beta}}_r(\boldsymbol{\theta})\right\rVert_2^2
+
\frac{1}{n}\sum_{i=1}^{n}\sum_{r=1}^{8}T_i\left\lVert\rho_{ir}(\boldsymbol{\theta})-\boldsymbol{h}(X_i)^\top\widehat{\boldsymbol{\beta}}_r(\boldsymbol{\theta})\right\rVert_2^2 n 1 i = 1 ∑ n 1 { T i = 1 } ∥ ρ i ( θ ) ∥ 2 2 = n 1 i = 1 ∑ n r = 1 ∑ 8 T i h ( X i ) ⊤ β r ( θ ) 2 2 + n 1 i = 1 ∑ n r = 1 ∑ 8 T i ρ i r ( θ ) − h ( X i ) ⊤ β r ( θ ) 2 2
where ρ i r ( θ ) \rho_{ir}(\boldsymbol{\theta}) ρ i r ( θ ) is the r-th component of the residual ρ i ( θ ) \boldsymbol{\rho}_i(\boldsymbol{\theta}) ρ i ( θ ) (r ∈ { 1 , ⋯ , 8 } r\in\{1,\cdots,8\} r ∈ { 1 , ⋯ , 8 } ) and β ^ r \widehat{\boldsymbol{\beta}}_r β r is its projection onto the column space spanned by h ( X i ) \boldsymbol{h}(X_i) h ( X i ) defined as
β ^ r ( θ ) = { ∑ i = 1 n 1 { T i = 1 } h ( X i ) h ( X i ) ⊤ } − 1 ∑ i = 1 n 1 { T i = 1 } h ( X i ) ρ i r ( θ ) . \widehat{\boldsymbol{\beta}}_r(\boldsymbol{\theta})=
\left\{\sum_{i=1}^{n}\mathbb{1}\{T_i=1\}\boldsymbol{h}(X_i)\boldsymbol{h}(X_i)^\top\right\}^{-1}
\sum_{i=1}^{n}\mathbb{1}\{T_i=1\}\boldsymbol{h}(X_i)\rho_{ir}(\boldsymbol{\theta}). β r ( θ ) = { i = 1 ∑ n 1 { T i = 1 } h ( X i ) h ( X i ) ⊤ } − 1 i = 1 ∑ n 1 { T i = 1 } h ( X i ) ρ i r ( θ ) .
Since the basis h ( x ) \boldsymbol{h}(x) h ( x ) contains all the terms in the original spline basis, the projection error does not depend on θ \boldsymbol{\theta} θ and thus Equation (13) can be simplified as
θ ^ ∈ argmin θ ∈ Θ 1 n ∑ r = 1 8 ( β ^ r ( θ ) ⊤ { ∑ i = 1 n 1 { T i = 1 } h ( X i ) h ( X i ) ⊤ } β ^ r ( θ ) ) + λ n Ω n ( θ ) . (14) \widehat{\boldsymbol{\theta}}\in\operatorname*{argmin}_{\boldsymbol{\theta}\in\Theta}
\frac{1}{n}\sum_{r=1}^{8}
\left(
\widehat{\boldsymbol{\beta}}_r(\boldsymbol{\theta})^\top
\left\{\sum_{i=1}^{n}\mathbb{1}\{T_i=1\}\boldsymbol{h}(X_i)\boldsymbol{h}(X_i)^\top\right\}
\widehat{\boldsymbol{\beta}}_r(\boldsymbol{\theta})
\right)
+\lambda_n\Omega_n(\boldsymbol{\theta}). \tag*{(14)} θ ∈ θ ∈ Θ argmin n 1 r = 1 ∑ 8 ( β r ( θ ) ⊤ { i = 1 ∑ n 1 { T i = 1 } h ( X i ) h ( X i ) ⊤ } β r ( θ ) ) + λ n Ω n ( θ ) . (14)
The optimization problem in Equation (14) is computationally easier since the inner term (Gram matrix) does not depend on the parameter θ \boldsymbol{\theta} θ and thus we only need to compute once.
We then propagate the first-stage estimation error, building on recent developments in the semiparametric statistics literature (e.g., [9 , 31 ] ). Specifically, we consider the influence function , a function that satisfies
E [ ϕ ( T i , Y i , X i , W ~ 1 i , W ~ 2 i , W ~ 3 i ; η , Ψ ) ] = 0 \mathbb{E}\left[\phi(T_i,Y_i,X_i,\widetilde{W}_{1i},\widetilde{W}_{2i},\widetilde{W}_{3i};\boldsymbol{\eta},\Psi)\right]=0 E [ ϕ ( T i , Y i , X i , W 1 i , W 2 i , W 3 i ; η , Ψ ) ] = 0
and
n ( Ψ ^ − Ψ ) = 1 n ∑ i = 1 n ϕ ( T i , Y i , X i , W ~ 1 i , W ~ 2 i , W ~ 3 i ; η , Ψ ) + o p ( 1 ) , \sqrt{n}(\widehat{\Psi}-\Psi)=\frac{1}{\sqrt{n}}\sum_{i=1}^{n}\phi(T_i,Y_i,X_i,\widetilde{W}_{1i},\widetilde{W}_{2i},\widetilde{W}_{3i};\boldsymbol{\eta},\Psi)+o_p(1), n ( Ψ − Ψ ) = n 1 i = 1 ∑ n ϕ ( T i , Y i , X i , W 1 i , W 2 i , W 3 i ; η , Ψ ) + o p ( 1 ) ,
where η \boldsymbol{\eta} η is the collection of nuisance functions and o p ( 1 ) o_p(1) o p ( 1 ) refers to the error that is negligible as n → ∞ n\to\infty n → ∞ . For the influence function below, the first condition serves as a moment condition to estimate the parameter of interest Ψ \Psi Ψ , whereas the second expression is useful to derive its asymptotic variance using the central limit theorem. The influence function is a key concept in the recent development of semiparametric statistics, since estimating equations based on it are insensitive to first-order errors in the nuisance estimates, a property known as Neyman orthogonality [9 ] . In our setup, the influence function is derived as follows.
Proposition 2 (Influence function). Under Assumptions 1–6 , provided that the component influence functions below are square-integrable, an influence function for Ψ \Psi Ψ is given by
ϕ ( T i , Y i , X i , W ~ 1 i , W ~ 2 i , W ~ 3 i ; θ , μ 0 , μ 1 , Q , Ψ ) : = ϕ 1 ( T i , Y i , X i ; μ 1 ) − ϕ 0 ( T i , Y i , X i ; μ 0 ) − Ψ ϕ Q ( T i , X i , W ~ 1 i , W ~ 2 i , W ~ 3 i ; θ ) Q , (15) \phi(T_i,Y_i,X_i,\widetilde{W}_{1i},\widetilde{W}_{2i},\widetilde{W}_{3i};\theta,\mu_0,\mu_1,Q,\Psi) := \frac{\phi_1(T_i,Y_i,X_i;\mu_1)-\phi_0(T_i,Y_i,X_i;\mu_0)-\Psi\phi^Q(T_i,X_i,\widetilde{W}_{1i},\widetilde{W}_{2i},\widetilde{W}_{3i};\theta)}{Q}, \tag*{(15)} ϕ ( T i , Y i , X i , W 1 i , W 2 i , W 3 i ; θ , μ 0 , μ 1 , Q , Ψ ) := Q ϕ 1 ( T i , Y i , X i ; μ 1 ) − ϕ 0 ( T i , Y i , X i ; μ 0 ) − Ψ ϕ Q ( T i , X i , W 1 i , W 2 i , W 3 i ; θ ) , (15) where ϕ t ( T i , Y i , X i ; μ t ) \phi_t(T_i,Y_i,X_i;\mu_t) ϕ t ( T i , Y i , X i ; μ t ) is an uncentered influence function for Ψ t : = E [ μ t ( X i ) ] \Psi_t := \mathbb{E}[\mu_t(X_i)] Ψ t := E [ μ t ( X i )] defined as
ϕ t ( T i , Y i , X i ; μ t ) : = μ t ( X i ) + 1 { T i = t } π t ( X i ) { Y i − μ t ( X i ) } , \phi_t(T_i,Y_i,X_i;\mu_t) := \mu_t(X_i) + \frac{1\{T_i=t\}}{\pi_t(X_i)}\{Y_i-\mu_t(X_i)\}, ϕ t ( T i , Y i , X i ; μ t ) := μ t ( X i ) + π t ( X i ) 1 { T i = t } { Y i − μ t ( X i )} , with μ t ( x ) : = E [ Y i ∣ T i = t , X i = x ] \mu_t(x) := \mathbb{E}[Y_i \mid T_i=t,X_i=x] μ t ( x ) := E [ Y i ∣ T i = t , X i = x ] . Similarly, ϕ Q ( T i , X i , W ~ 1 i , W ~ 2 i , W ~ 3 i ; θ ) \phi^Q(T_i,X_i,\widetilde{W}_{1i},\widetilde{W}_{2i},\widetilde{W}_{3i};\theta) ϕ Q ( T i , X i , W 1 i , W 2 i , W 3 i ; θ ) is an uncentered influence function for Q = Pr ( W i ( 1 ) = 1 ) Q = \Pr(W_i(1)=1) Q = Pr ( W i ( 1 ) = 1 ) defined as
ϕ Q ( T i , X i , W ~ 1 i , W ~ 2 i , W ~ 3 i ; θ ) : = q ( X i ) + 1 { T i = 1 } π 1 ( X i ) { 1 − S 0 i + C i − q ( X i ) } , \phi^Q(T_i,X_i,\widetilde{W}_{1i},\widetilde{W}_{2i},\widetilde{W}_{3i};\theta) := q(X_i) + \frac{1\{T_i=1\}}{\pi_1(X_i)}\{1-S_{0i}+C_i-q(X_i)\}, ϕ Q ( T i , X i , W 1 i , W 2 i , W 3 i ; θ ) := q ( X i ) + π 1 ( X i ) 1 { T i = 1 } { 1 − S 0 i + C i − q ( X i )} , where
S j i : = [ G j ( 1 , X i ) ] 2 , 2 − W ~ j i [ G j ( 1 , X i ) ] 2 , 2 − [ G j ( 1 , X i ) ] 2 , 1 , j = 1 , 2 , S 0 i : = S 1 i S 2 i . S_{ji} := \frac{[G_j(1,X_i)]_{2,2}-\widetilde{W}_{ji}}{[G_j(1,X_i)]_{2,2}-[G_j(1,X_i)]_{2,1}}, \qquad j=1,2, \qquad S_{0i}:=S_{1i}S_{2i}. S j i := [ G j ( 1 , X i ) ] 2 , 2 − [ G j ( 1 , X i ) ] 2 , 1 [ G j ( 1 , X i ) ] 2 , 2 − W j i , j = 1 , 2 , S 0 i := S 1 i S 2 i . C i : = W ~ 3 i − m 1 ( 1 , X i ) m 1 ( 1 , X i ) − m 1 ( 0 , X i ) { S 1 i + S 2 i − 2 S 0 i } . C_i := \frac{\widetilde{W}_{3i}-m_1(1,X_i)}{m_1(1,X_i)-m_1(0,X_i)}\{S_{1i}+S_{2i}-2S_{0i}\}. C i := m 1 ( 1 , X i ) − m 1 ( 0 , X i ) W 3 i − m 1 ( 1 , X i ) { S 1 i + S 2 i − 2 S 0 i } . The proof is given in Appendix § 9.2.2 . Here, we can see that the uncentered influence functions take the canonical form of augmented inverse propensity score except that ϕ Q \phi^Q ϕ Q uses 1 − S 0 i + C i 1-S_{0i}+C_i 1 − S 0 i + C i instead of W i W_i W i since it is latent and thus needs to be estimated.
We then obtain the proposed estimator by substituting fitted functions into the estimating equations for Ψ 0 \Psi_0 Ψ 0 , Ψ 1 \Psi_1 Ψ 1 , and Q Q Q . We use K K K -fold cross-fitting so that each respondent’s contribution is evaluated using functions estimated without that respondent’s fold ([9 , 34 ] ). For a fixed K ≥ 2 K \ge2 K ≥ 2 , the procedure is as follows.
Randomly partition the observations into K K K folds, I 1 , … , I K \mathcal{I}_1,\ldots,\mathcal{I}_K I 1 , … , I K , of approximately equal size.
For each fold k k k , estimate the measurement model using observations outside I k \mathcal{I}_k I k :
θ ^ ( − k ) ∈ arg min θ ∈ Θ 1 n − ∣ I k ∣ ∑ r = 1 8 ( β ^ r ( θ ) T { ∑ i ∉ I k T i h ( X i ) h ( X i ) T } β ^ r ( θ ) ) + λ n Ω n ( θ ) \widehat{\theta}^{(-k)} \in\mathop{\arg\min}_{\theta\in\Theta} \frac{1}{n-|\mathcal{I}_k|}\sum_{r=1}^{8}\left(\widehat{\beta}_r(\theta)^\mathsf{T}\left\{\sum_{i\notin\mathcal{I}_k}T_i h(X_i)h(X_i)^\mathsf{T}\right\}\widehat{\beta}_r(\theta)\right)+\lambda_n\Omega_n(\theta) θ ( − k ) ∈ arg min θ ∈ Θ n − ∣ I k ∣ 1 r = 1 ∑ 8 β r ( θ ) T ⎩ ⎨ ⎧ i ∈ / I k ∑ T i h ( X i ) h ( X i ) T ⎭ ⎬ ⎫ β r ( θ ) + λ n Ω n ( θ )
Substitute θ ^ ( − k ) \widehat{\theta}^{(-k)} θ ( − k ) into Equation (10) to obtain the fitted engagement probabilities, the means of the first two checks, and subgroup-specific means of the third check. Also estimate μ t ( x ) = E [ Y i ∣ T i = t , X i = x ] \mu_t(x)=\mathbb{E}[Y_i\mid T_i=t,X_i=x] μ t ( x ) = E [ Y i ∣ T i = t , X i = x ] using the training observations in arm t t t , for t = 0 , 1 t=0,1 t = 0 , 1 .
Compute
Ψ ^ t = 1 n ∑ k = 1 K ∑ i ∈ I k [ μ ^ t ( − k ) ( X i ) + 1 { T i = t } π t ( X i ) { Y i − μ ^ t ( − k ) ( X i ) } ] , (16) \widehat{\Psi}_t=\frac{1}{n}\sum_{k=1}^{K}\sum_{i\in\mathcal{I}_k}\left[\widehat{\mu}_t^{(-k)}(X_i)+\frac{1\{T_i=t\}}{\pi_t(X_i)}\{Y_i-\widehat{\mu}_t^{(-k)}(X_i)\}\right], \tag*{(16)} Ψ t = n 1 k = 1 ∑ K i ∈ I k ∑ [ μ t ( − k ) ( X i ) + π t ( X i ) 1 { T i = t } { Y i − μ t ( − k ) ( X i )} ] , (16)
Q ^ = 1 n ∑ k = 1 K ∑ i ∈ I k [ q ^ ( − k ) ( X i ) + 1 { T i = 1 } π 1 ( X i ) { 1 − S ^ 0 i ( − k ) + C ^ i ( − k ) − q ^ ( − k ) ( X i ) } ] . (17) \widehat{Q}=\frac{1}{n}\sum_{k=1}^{K}\sum_{i\in\mathcal{I}_k}\left[\widehat{q}^{(-k)}(X_i)+\frac{1\{T_i=1\}}{\pi_1(X_i)}\{1-\widehat{S}_{0i}^{(-k)}+\widehat{C}_i^{(-k)}-\widehat{q}^{(-k)}(X_i)\}\right]. \tag*{(17)} Q = n 1 k = 1 ∑ K i ∈ I k ∑ [ q ( − k ) ( X i ) + π 1 ( X i ) 1 { T i = 1 } { 1 − S 0 i ( − k ) + C i ( − k ) − q ( − k ) ( X i )} ] . (17)
and set
Ψ ^ = Ψ ^ 1 − Ψ ^ 0 Q ^ . (18) \widehat{\Psi}=\frac{\widehat{\Psi}_1-\widehat{\Psi}_0}{\widehat{Q}}. \tag*{(18)} Ψ = Q Ψ 1 − Ψ 0 . (18)
Finally, we establish the asymptotic distribution of the proposed estimator under the regularity conditions of Assumption 7 in Appendix S1.3.
Proposition 3 (Asymptotic normality). Under Assumptions 1–7,
n ( Ψ ^ − Ψ ) ⟶ d N ( 0 , E [ ϕ ( T i , Y i , X i , W ~ 1 i , W ~ 2 i , W ~ 3 i ; θ , μ 0 , μ 1 , Q , Ψ ) 2 ] ) . (19) \sqrt{n}(\widehat{\Psi}-\Psi)\mathrel{\overset{d}{\longrightarrow}}\mathcal{N}\left(0,\mathbb{E}\left[\phi(T_i,Y_i,X_i,\widetilde{W}_{1i},\widetilde{W}_{2i},\widetilde{W}_{3i};\theta,\mu_0,\mu_1,Q,\Psi)^2\right]\right). \tag*{(19)} n ( Ψ − Ψ ) ⟶ d N ( 0 , E [ ϕ ( T i , Y i , X i , W 1 i , W 2 i , W 3 i ; θ , μ 0 , μ 1 , Q , Ψ ) 2 ] ) . (19)
The proof is given in Appendix S1.3. We can thus calculate the standard errors by
se ^ ( Ψ ^ ) = { 1 n 2 ∑ k = 1 K ∑ i ∈ I k ( ϕ ^ i − ϕ ^ ‾ ) 2 } 1 / 2 . \widehat{\operatorname{se}}(\widehat{\Psi})=\left\{\frac{1}{n^2}\sum_{k=1}^{K}\sum_{i\in\mathcal{I}_k}\left(\widehat{\phi}_i-\overline{\widehat{\phi}}\right)^2\right\}^{1/2}. se ( Ψ ) = { n 2 1 k = 1 ∑ K i ∈ I k ∑ ( ϕ i − ϕ ) 2 } 1/2 .
where ϕ ^ i : = ϕ ( T i , Y i , X i , W ~ 1 i , W ~ 2 i , W ~ 3 i ; θ ^ ( − k ) , μ ^ 0 ( − k ) , μ ^ 1 ( − k ) , Q ^ , Ψ ^ ) \widehat{\phi}_i:=\phi(T_i,Y_i,X_i,\widetilde{W}_{1i},\widetilde{W}_{2i},\widetilde{W}_{3i};\widehat{\theta}^{(-k)},\widehat{\mu}_0^{(-k)},\widehat{\mu}_1^{(-k)},\widehat{Q},\widehat{\Psi}) ϕ i := ϕ ( T i , Y i , X i , W 1 i , W 2 i , W 3 i ; θ ( − k ) , μ 0 ( − k ) , μ 1 ( − k ) , Q , Ψ ) is the estimated influence function and ϕ ^ ‾ = n − 1 ∑ k = 1 K ∑ i ∈ I k ϕ ^ i \overline{\widehat{\phi}}=n^{-1}\sum_{k=1}^{K}\sum_{i\in\mathcal{I}_k}\widehat{\phi}_i ϕ = n − 1 ∑ k = 1 K ∑ i ∈ I k ϕ i is its empirical average. An asymptotic 95% confidence interval is then obtained as Ψ ^ ± 1.96 se ^ ( Ψ ^ ) \widehat{\Psi}\pm1.96\widehat{\operatorname{se}}(\widehat{\Psi}) Ψ ± 1.96 se ( Ψ ) .
Testing the Relevance Assumption
The key identification assumptions of the proposed methodology are the conditional independence of checks among treated individuals (Assumption 5) and the relevance of checks (Assumption 6). Although these assumptions involve unobserved engagement status, the relevance condition has observable implications under the conditional independence assumption. Specifically, we can assess the relevance assumption through associations among observed variables. The following proposition provides the covariance identities underlying these diagnostics.
Proposition 4 (Covariance identity). Consider a treatment condition t t t , under which the conditional independence restrictions in Assumption 5 hold. Then, we have,
Cov ( W ~ j i , W ~ j ′ i ∣ T i = t , X i = x ) = d t j ( x ) d t j ′ ( x ) V ( W i ∣ T i = t , X i = x ) , 1 ≤ j < j ′ ≤ 3 , (20) \operatorname{Cov}(\widetilde{W}_{ji},\widetilde{W}_{j'i}\mid T_i=t,X_i=x)=d_t^j(x)d_t^{j'}(x)\operatorname{V}(W_i\mid T_i=t,X_i=x),\qquad1\leq j<j'\leq3, \tag*{(20)} Cov ( W j i , W j ′ i ∣ T i = t , X i = x ) = d t j ( x ) d t j ′ ( x ) V ( W i ∣ T i = t , X i = x ) , 1 ≤ j < j ′ ≤ 3 , (20) where
d t j ( x ) : = E [ W ~ j i ∣ W i = 1 , T i = t , X i = x ] − E [ W ~ j i ∣ W i = 0 , T i = t , X i = x ] , j ∈ { 1 , 2 , 3 } . d_t^j(x):=\mathbb{E}[\widetilde{W}_{ji}\mid W_i=1,T_i=t,X_i=x]-\mathbb{E}[\widetilde{W}_{ji}\mid W_i=0,T_i=t,X_i=x],\qquad j\in\{1,2,3\}. d t j ( x ) := E [ W j i ∣ W i = 1 , T i = t , X i = x ] − E [ W j i ∣ W i = 0 , T i = t , X i = x ] , j ∈ { 1 , 2 , 3 } .
The proof is given in Appendix S1.4.
Proposition 4 shows that a zero covariance between checks implies that at least one of three factors is zero: the first check fails to distinguish engaged respondents from the unengaged ones, the second check fails to distinguish these groups, or engagement status does not vary within the covariate stratum. In the treated group, the last factor is given by,
V ( W i ∣ T i = 1 , X i = x ) = q ( x ) { 1 − q ( x ) } , \operatorname{V}(W_i\mid T_i=1,X_i=x)=q(x)\{1-q(x)\}, V ( W i ∣ T i = 1 , X i = x ) = q ( x ) { 1 − q ( x )} ,
which is strictly positive under the positivity condition given in Assumption 6. Therefore, under conditional independence and positivity, the covariance between any two checks is nonzero if and only if both checks have different conditional means across engagement groups. Moreover, the positive ordering imposed by the relevance condition implies that all three pairwise conditional covariances are strictly positive. These observable implications allow us to assess the strength of the relevance condition without first recovering the latent engagement status.
We can implement these diagnostics using pairwise regressions with covariate adjustment. Specifically, among treated respondents, we consider the following regression for each pair ( j , k ) ∈ { ( 1 , 2 ) , ( 1 , 3 ) , ( 2 , 3 ) } (j,k) \in\{(1,2),(1,3),(2,3)\} ( j , k ) ∈ {( 1 , 2 ) , ( 1 , 3 ) , ( 2 , 3 )} :
W ~ j i = p 0 ( X i ) ⊤ ζ j k + W ~ k i p 1 ( X i ) ⊤ ν j k + e j k , i , (21) \widetilde{W}_{ji} = \mathbf{p}_0(X_i)^{\top}\zeta_{jk} + \widetilde{W}_{ki}\mathbf{p}_1(X_i)^{\top}\nu_{jk} + e_{jk,i}, \tag*{(21)} W j i = p 0 ( X i ) ⊤ ζ j k + W k i p 1 ( X i ) ⊤ ν j k + e j k , i , (21)
where p 0 ( X i ) \mathbf{p}_0(X_i) p 0 ( X i ) and p 1 ( X i ) \mathbf{p}_1(X_i) p 1 ( X i ) are user-specified covariate basis vectors that include an intercept, and ζ j k \zeta_{jk} ζ j k and ν j k \nu_{jk} ν j k are pair-specific coefficient vectors. The first term adjusts for the covariates, while the second allows the association between checks to vary with the covariates. For each pair, we conduct an F F F -test of H 0 : ν j k = 0 H_0:\nu_{jk}=0 H 0 : ν j k = 0 to assess conditional association between checks. We then use an intersection–union test to assess whether all three pairs exhibit conditional association. Specifically, we test
H 0 : ν j k = 0 for at least one pair ( j , k ) against H 1 : ν j k ≠ 0 for all three pairs. H_0:\nu_{jk}=0 \text{ for at least one pair } (j,k) \quad\text{against} \quad H_1:\nu_{jk}\ne0 \text{ for all three pairs.} H 0 : ν j k = 0 for at least one pair ( j , k ) against H 1 : ν j k = 0 for all three pairs.
The resulting p p p -value is
p I U T = max { p 12 , p 13 , p 23 } . p_{\mathrm{IUT}}=\max\{p_{12},p_{13},p_{23}\}. p IUT = max { p 12 , p 13 , p 23 } .
We reject the null hypothesis at level α \alpha α if all three pairwise p p p -values are below α \alpha α .
Simulation Studies
In this section, we conduct simulation studies to examine the performance of the proposed estimator under varying degrees of proxy quality, considering both attention and manipulation checks.
Setup
For n = 1000 n=1000 n = 1000 respondents, we independently draw a pre-treatment covariate X i ∼ Uniform ( − 1 , 1 ) X_i \sim\operatorname{Uniform}(-1,1) X i ∼ Uniform ( − 1 , 1 ) and treatment assignment T i ∼ Bernoulli ( 1 / 2 ) T_i \sim\operatorname{Bernoulli}(1/2) T i ∼ Bernoulli ( 1/2 ) . We then generate potential engagement status as
W i ( 1 ) = 1 { U i ≤ 0.6 + 0.2 X i } , W i ( 0 ) = 1 { U i ≤ γ ( 0.6 + 0.2 X i ) } . W_i(1)=\mathbb{1}\{U_i\le0.6+0.2X_i\}, \qquad W_i(0)=\mathbb{1}\{U_i\le\gamma(0.6+0.2X_i)\}. W i ( 1 ) = 1 { U i ≤ 0.6 + 0.2 X i } , W i ( 0 ) = 1 { U i ≤ γ ( 0.6 + 0.2 X i )} .
where U i ∼ Uniform ( 0 , 1 ) U_i \sim\operatorname{Uniform}(0,1) U i ∼ Uniform ( 0 , 1 ) . We set γ = 1 \gamma=1 γ = 1 for attention, so that engagement is unaffected by treatment, and γ = 0.4 \gamma=0.4 γ = 0.4 for manipulation, so that treatment can increase engagement.
Throughout the simulation, we assume that realized engagement W i = W i ( T i ) W_i=W_i(T_i) W i = W i ( T i ) is unobserved. Instead, we observe three binary checks, generated as
W ~ j i = 1 { a j + 0.3 X i + κ ( 2 W i − 1 ) + Z j i > 0 } , j ∈ { 1 , 2 , 3 } , \widetilde{W}_{ji}=\mathbb{1}\{a_j+0.3X_i+\kappa(2W_i-1)+Z_{ji}>0\}, \qquad j\in\{1,2,3\}, W j i = 1 { a j + 0.3 X i + κ ( 2 W i − 1 ) + Z j i > 0 } , j ∈ { 1 , 2 , 3 } ,
where Z j i ∼ N ( 0 , 1 ) Z_{ji}\sim\mathcal{N}(0,1) Z j i ∼ N ( 0 , 1 ) ( j ∈ { 1 , 2 , 3 } ) (j\in\{1,2,3\}) ( j ∈ { 1 , 2 , 3 }) is the error in the observed checks and ( a 1 , a 2 , a 3 ) = ( − 0.5 , 0 , 0.5 ) (a_1,a_2,a_3)=(-0.5,0,0.5) ( a 1 , a 2 , a 3 ) = ( − 0.5 , 0 , 0.5 ) allows the checks to differ in difficulty. Here, κ \kappa κ controls the quality of checks: a larger κ \kappa κ corresponds to more accurate measurement of W i W_i W i . We finally generate potential outcomes as
Y i ( t ) = X i 2 + 0.5 W i ( 0 ) + t W i ( 1 ) ( 1 + X i + { Z 1 i + Z 2 i + Z 3 i } ) + ε i , t ∈ { 0 , 1 } , Y_i(t)=X_i^2+0.5W_i(0)+tW_i(1)(1+X_i+\{Z_{1i}+Z_{2i}+Z_{3i}\})+\varepsilon_i, \qquad t\in\{0,1\}, Y i ( t ) = X i 2 + 0.5 W i ( 0 ) + t W i ( 1 ) ( 1 + X i + { Z 1 i + Z 2 i + Z 3 i }) + ε i , t ∈ { 0 , 1 } ,
where ε i ∼ Uniform ( − 1 , 1 ) \varepsilon_i\sim\operatorname{Uniform}(-1,1) ε i ∼ Uniform ( − 1 , 1 ) is an outcome error, and we observe Y i = Y i ( T i ) Y_i=Y_i(T_i) Y i = Y i ( T i ) .
Under this data-generating process, we consider attention and manipulation checks and vary the check quality κ ∈ { 1.0 , 1.5 , 2.0 , 2.5 } \kappa\in\{1.0,1.5,2.0,2.5\} κ ∈ { 1.0 , 1.5 , 2.0 , 2.5 } . For each configuration, we generate 1000 Monte Carlo replications to evaluate the performance of the estimator.
In each replication, we implement the proposed estimator using five-fold cross-fitting and piecewise-linear splines in X i X_i X i . Following the penalized sieve estimation literature [8 ] , we use L n = 4 L_n = 4 L n = 4 spline basis functions and set the regularization weight to λ n = 0.004 \lambda_n = 0.004 λ n = 0.004 . Within each training sample, we estimate the conditional engagement probability and measurement functions using treated respondents by minimizing the objective in Equation (14) .
We also estimate the conditional outcome means separately by treatment arm using the same spline basis. We then evaluate the fitted functions on the held-out observations and combine their contributions across folds to obtain the corrected engagement probability estimate Q ^ \widehat{Q} Q and the eATE estimate Ψ ^ = ( Ψ ^ 1 − Ψ ^ 0 ) / Q ^ \widehat{\Psi} = (\widehat{\Psi}_1 - \widehat{\Psi}_0)/\widehat{Q} Ψ = ( Ψ 1 − Ψ 0 ) / Q , as defined in Equations (17) and (18) , using the known treatment assignment probabilities π 0 ( x ) = π 1 ( x ) = 1 / 2 \pi_0(x) = \pi_1(x) = 1/2 π 0 ( x ) = π 1 ( x ) = 1/2 .
We compare the proposed estimator with three conventional approaches. All three estimators adjust for the pre-treatment covariate X i X_i X i using the same spline basis as the proposed estimator. First, we consider the intention-to-treat (ITT) effect by regressing outcomes on treatment assignment and the spline basis functions of X i X_i X i . Unlike the eATE, this estimator averages treatment effects over both engaged and unengaged respondents. Second, for attention checks, we consider a subgroup analysis that estimates the treatment effect among respondents who pass all three observed checks, i.e., W ~ 1 i = W ~ 2 i = W ~ 3 i = 1 \widetilde{W}_{1i} = \widetilde{W}_{2i} = \widetilde{W}_{3i} = 1 W 1 i = W 2 i = W 3 i = 1 , using the same covariate-adjusted regression. This approach treats the observed checks as accurate measures of engagement and is therefore subject to bias from measurement error. Third, for manipulation checks, we consider an instrumental variable (IV) estimator that uses randomized treatment assignment as an instrument for an indicator of passing all three manipulation checks, adjusting for X i X_i X i in both stages. This approach is also sensitive to measurement error in the checks and additionally relies on the exclusion restriction. In contrast, our proposed estimator explicitly accounts for measurement error and does not require the standard IV exclusion restriction.
Results
Figure 4 presents the simulation results for attention checks in the left column and manipulation checks in the right column. Across all levels of check accuracy (horizontal axis), the proposed estimator (red solid circles) exhibits negligible bias (top row), low RMSE (middle), and empirical coverage (bottom) close to the nominal 95% level. In contrast, the conventional estimators generally exhibit substantial bias and poor coverage. In both attention and manipulation settings, the intention-to-treat estimator is biased toward zero because it averages treatment effects over both engaged and unengaged respondents. For pre-treatment attention checks, restricting the analysis to respondents who pass all three checks also produces a substantial bias when the checks are imperfect. Although this bias decreases as check accuracy increases, the subgroup estimator continues to exhibit undercoverage even when the accuracy of each check is relatively high. For post-treatment manipulation checks, the instrumental variable estimator exhibits substantial bias and poor coverage even when the checks are nearly perfectly accurate (κ = 2.5 \kappa= 2.5 κ = 2.5 ). This bias persists because the exclusion restriction is violated, a problem that cannot be resolved by improving the accuracy of the observed checks alone.
Figure 4. Simulation results comparing the proposed methodology with conventional estimators for attention checks (left) and manipulation checks (right). The three rows report bias, root mean squared error (RMSE), and empirical coverage of 95% confidence intervals for the engaged average treatment effect (eATE). The horizontal axis shows the check-quality parameter κ \kappa κ (top) and the corresponding accuracy of a single check (bottom). Each setting uses n = 1000 n = 1000 n = 1000 respondents and 1000 Monte Carlo replications. The dashed horizontal lines indicate zero bias, zero RMSE, and nominal 95% coverage, respectively. The conventional estimators include the intention-to-treat estimator, the subgroup estimator restricted to respondents who pass all three attention checks, and the instrumental variable estimator for manipulation checks.
Results of Empirical Analyses
In this section, we apply the proposed methodology to the two motivating examples introduced in § 2 : the framing experiment of Berinsky et al. (2014), which uses attention checks, and the digital media literacy experiment of Guess et al. (2020), which uses manipulation checks. In both applications, we compare our proposed approach with conventional methods to examine how accounting for measurement error in the checks affects the estimated treatment effects.
Civil Liberties Framing Experiment
Civil Liberties Framing Experiment
We first apply the proposed methodology to the civil liberties framing experiment introduced in § 2.1 . Specifically, we analyze the replication experiment of Nelson et al. [29 ] administered in the second wave of Study 1 in Berinsky et al. [4 ] , a two-wave online panel conducted by Survey Sampling International in June–July 2011.
Among 381 respondents with observed treatment assignment and outcome, five have missing attention check responses, leaving N = 376 N = 376 N = 376 respondents for the analysis sample. As described in § 2.1 , our treatment of interest is assignment to the public safety frame rather than the free speech frame, which we code as treatment and control, respectively. The outcome is a binary indicator that is equal to one if the respondent thinks that the university should allow the KKK rally and zero otherwise. We then construct four binary attention check variables indexed by their order of administration. Specifically, W ~ j i \widetilde{W}_{ji} W j i equals one if respondent i i i correctly answered the screener administered in position j j j , and zero otherwise, for j = 1 , … , 4 j = 1, \ldots, 4 j = 1 , … , 4 . The first screener was administered before the framing experiment, whereas the remaining three were administered afterwards. Because screener types were randomly assigned to administration positions, the content of the screener corresponding to each W ~ j i \widetilde{W}_{ji} W j i can differ across respondents.
We then apply the proposed methodology to estimate the average treatment effect among engaged respondents or eATE. Specifically, we use the first three attention checks since their timing is the closest to the framing experiment we reanalyze. We use five-fold cross-fitting and adjust for age and gender. To model the dependence on these covariates, we use an additive basis consisting of three piecewise-linear spline basis functions for age and a gender indicator, and set the regularization parameter to λ n = 3 / 376 ≈ 0.008 \lambda_n = 3/376 \approx0.008 λ n = 3/376 ≈ 0.008 . We also estimate the conditional outcome means separately by treatment condition using the same covariate basis.
We compare the eATE estimate from our proposed methodology with the conventional estimates based on three different samples: all respondents in the analysis sample, respondents who passed the screener immediately preceding the experiment, and respondents who passed all four screeners. The first benchmark estimates the intention-to-treat (ITT) effect in the full analysis sample, while the other two restrict the sample based on observed screener responses. To ensure comparability, we adjust all three benchmark estimates for the same covariates used in our approach, with the same age spline specification and gender indicator.
Figure 5 reports the results. Across all four approaches, we find that the public safety frame substantially reduces respondents’ probability of supporting the rally relative to the free speech frame. Specifically, the estimated average effect is − 0.359 -0.359 − 0.359 (95% CI: [ − 0.451 , − 0.267 ] [-0.451, -0.267] [ − 0.451 , − 0.267 ] ) among all respondents, − 0.411 -0.411 − 0.411 (95% CI: [ − 0.525 , − 0.296 ] [-0.525, -0.296] [ − 0.525 , − 0.296 ] ) among respondents who passed the screener immediately preceding the experiment, and − 0.397 -0.397 − 0.397 (95% CI: [ − 0.524 , − 0.270 ] [-0.524, -0.270] [ − 0.524 , − 0.270 ] ) among those who passed all four screeners. Our proposed approach estimates the eATE of − 0.479 -0.479 − 0.479 (95% CI: [ − 0.616 , − 0.342 ] [-0.616, -0.342] [ − 0.616 , − 0.342 ] ) among engaged respondents, corresponding to a 47.9 percentage-point reduction in support for allowing the rally. This estimate is approximately 12.0 percentage points larger in magnitude than the ITT estimate among all respondents. To assess the relevance condition required for identification, we conduct the pairwise F F F -tests described in § 5 among respondents assigned to the public safety frame. The resulting F F F -statistics are 7.9 for checks 1 and 2, 6.4 for checks 1 and 3, and 10.6 for checks 2 and 3, and the intersection–union test described in § 5 also rejects the null hypothesis that at least one pair lacks conditional association (p I U T < 0.0001 p_{\mathrm{IUT}} < 0.0001 p IUT < 0.0001 ). These results support the relevance condition.
Figure 5. Estimated effects of the public safety frame relative to the free speech frame on support for allowing the KKK rally in the civil liberties framing experiment of [4 ] . The dashed vertical line indicates zero. The first estimator uses all respondents in the analysis sample. The second and third estimators restrict the analysis to respondents who passed the screener immediately preceding the experiment and those who passed all four screeners in the second wave, respectively. The proposed estimator accounts for measurement error in the first three screeners to estimate the average treatment effect among attentive respondents.
These findings suggest that restricting the analysis to respondents who pass the observed attention checks may not fully account for measurement error in these checks. In particular, even respondents who pass all four screeners may include inattentive respondents, while some attentive respondents may fail one or more screeners. By accounting for such measurement error under the identification assumptions, our proposed approach yields a larger treatment effect estimate in magnitude than the conventional subgroup analyses. Although the confidence intervals overlap, the lower bound of the 95% confidence interval shifts from −0.451 for the ITT estimate to −0.616 for the proposed eATE estimate, while the upper bound shifts from −0.267 to −0.342. This pattern is consistent with conventional approaches understating the magnitude of the framing effect among attentive respondents.
Digital Media Literacy Experiment
We next apply the proposed methodology to the data from the digital media literacy experiment ([14 ] ) introduced in § 2.2 . Specifically, we analyze the first wave of the U.S. YouGov survey, conducted from November 20 to December 27, 2018. The sample consists of 4,907 respondents, including 2,454 assigned to the digital media literacy intervention and 2,453 assigned to the control condition. In the experiment, each respondent was asked to evaluate four mainstream (true) and two false news headlines on a four-point scale ranging from 1 (not at all accurate) to 4 (very accurate). We consider three outcomes: perceived accuracy of false news headlines, perceived accuracy of mainstream news headlines, and discernment, defined as the average difference in accuracy rating between mainstream and false news. Following the original paper, we construct three binary manipulation checks indicating whether respondents correctly answered each comprehension question on their first attempt. These questions assess their understanding of misleading web addresses, spelling errors, and corroboration by multiple trusted sources.
We then apply the proposed approach to estimate the average treatment effect among respondents who would be successfully manipulated under treatment (the eATE). Specifically, we use five-fold cross-fitting and adjust for multiple pre-treatment covariates, including age, gender, race, and education. For the first two outcomes, we conduct the analysis at the headline level, including headline fixed effects to account for differences across headlines. Because each respondent evaluates multiple headlines, we cluster standard errors at the respondent level and ensure that all observations from the same respondent belong to the same cross-fitting fold. For discernment, we conduct the analysis at the respondent level. To model the dependence on covariates, we use an additive basis consisting of five piecewise-linear spline basis functions for age and indicators for gender, race, and education, and set the regularization parameter to λ n = 5 / 4907 ≈ 0.0010 \lambda_n = 5/4907 \approx0.0010 λ n = 5/4907 ≈ 0.0010 .
Figure 6. Estimated effects of the digital media literacy intervention on perceived accuracy of false news (left), perceived accuracy of mainstream news (middle), and discernment (right) in Guess et al. [14 ] . The dashed vertical line indicates zero. The intention-to-treat estimator measures the effect of treatment assignment. The instrumental variable estimator uses randomized treatment assignment as an instrument for passing the manipulation checks, which is defined as correctly answering all three comprehension questions on the first attempt. The proposed estimator of the eATE accounts for measurement error in these checks to estimate the effect among respondents who would be successfully manipulated under treatment.
As benchmark comparisons, we replicate the two original estimators: the intention-to-treat (ITT) estimator and the instrumental variable (IV) estimator. The former estimates the effect of randomized assignment, whereas the latter uses treatment assignment as an instrument for observed treatment receipt, defined as passing all three comprehension questions on the first attempt among treated respondents. To ensure comparability with the proposed estimator, we also adjust for the same pre-treatment covariates (i.e., age, gender, race, and education) in all three analyses. Following the original analysis, we include headline fixed effects and cluster standard errors at the respondent level for the first two outcomes.
Figure 6 reports the results. Across all three outcomes, we find that the intervention reduces the perceived accuracy of both false and mainstream news, with a larger reduction for false news, thereby improving respondents’ ability to distinguish between mainstream and false news. Specifically, our proposed approach estimates that the intervention reduces perceived accuracy by 0.233 points for false news and 0.059 points for mainstream news, and increases discernment by 0.169 points. To assess the relevance condition required for identification, we conduct the pairwise F F F -tests described in § 5 . The resulting F F F -statistics are 21.8, 16.9, and 16.7, with all three p p p -values below 0.0001. The intersection–union test also rejects the null hypothesis that at least one pair lacks conditional association (p I U T < 0.0001 p_{\mathrm{IUT}} < 0.0001 p IUT < 0.0001 ). Overall, the estimates based on our proposed methodology are larger in absolute magnitude than the ITT estimates but smaller than the IV estimates.
The difference between the proposed and IV methods is largely driven by the assumptions made about manipulation. The IV approach assumes that manipulation is successful for those respondents who correctly answer all three comprehension questions on the first attempt. Under this definition, only 65.6% of treated respondents are classified as successfully manipulated. In contrast, our proposed approach accounts for measurement error in each manipulation check and estimates the proportion of successfully manipulated respondents to be 0.836 with a standard error of 0.015, approximately 18 percentage points higher than the observed proportion. Since both approaches rescale the intention-to-treat effect by the estimated proportion of successfully manipulated respondents, the smaller denominator used by the IV approach leads to larger treatment effect estimates. These findings suggest that requiring respondents to pass all three checks for successful manipulation may be overly restrictive and may overstate the magnitude of the treatment effect among engaged respondents.
Concluding Remarks
In this paper, we propose a general statistical framework for analyzing survey experiments with attention and manipulation checks. Our unified approach accommodates both pre-treatment attention checks and post-treatment manipulation checks while accounting for measurement error in the observed checks. We show that it is possible to use three imperfect checks to recover the proportion of respondents who would be engaged under treatment and identify the average treatment effect for this subgroup (eATE), under the assumption that these three checks are conditionally independent of one another given treatment, true engagement status, and the set of pre-treatment covariates. We then develop a flexible estimator, derive its asymptotic confidence intervals, and provide a diagnostic test for the key identification assumptions. Through the reanalysis of a civil liberties framing experiment and a digital media literacy experiment, we demonstrate how accounting for measurement error can yield estimated treatment effects that are different from those based on conventional subgroup and instrumental variable analyses.
An important direction for future research, which we are currently pursuing, is to incorporate richer measurements of respondents’ engagement. Eye tracking, for example, provides detailed information about which parts of a stimulus respondents view, how long they view them, and whether they return to particular pieces of information (e.g., Jenke et al., [18 ] ; Jenke and Sullivan, [19 ] ). However, these measurements are also subject to measurement error. Future research could extend our framework to accommodate such richer, potentially high-dimensional measurements of respondents’ engagement.
Proofs
Proof of Proposition 1
We begin by writing Equations (7) and (9) as
E [ V 1 i V 2 i T ∣ T i = 1 , X i = x ] = G 1 ( 1 , x ) diag { 1 − q ( x ) , q ( x ) } G 2 ( 1 , x ) T , \mathbb{E}[V_{1i}V_{2i}^{\mathsf{T}} \mid T_i=1, X_i=x] = G_1(1,x)\operatorname{diag}\{1-q(x),q(x)\}G_2(1,x)^{\mathsf{T}}, E [ V 1 i V 2 i T ∣ T i = 1 , X i = x ] = G 1 ( 1 , x ) diag { 1 − q ( x ) , q ( x )} G 2 ( 1 , x ) T ,
E [ W ~ 3 i V 1 i V 2 i T ∣ T i = 1 , X i = x ] = G 1 ( 1 , x ) diag { ( 1 − q ( x ) ) m 1 ( 0 , x ) , q ( x ) m 1 ( 1 , x ) } G 2 ( 1 , x ) T . \mathbb{E}[\widetilde{W}_{3i}V_{1i}V_{2i}^{\mathsf{T}} \mid T_i=1, X_i=x] = G_1(1,x)\operatorname{diag}\{(1-q(x))m_1(0,x),q(x)m_1(1,x)\}G_2(1,x)^{\mathsf{T}}. E [ W 3 i V 1 i V 2 i T ∣ T i = 1 , X i = x ] = G 1 ( 1 , x ) diag {( 1 − q ( x )) m 1 ( 0 , x ) , q ( x ) m 1 ( 1 , x )} G 2 ( 1 , x ) T .
The relevance assumption (Assumption 6(ii)) implies that G j ( 1 , x ) G_j(1,x) G j ( 1 , x ) is invertible for j = 1 , 2 j=1,2 j = 1 , 2 because their determinants are the strictly positive differences between the two check means. The positivity condition (Assumption 6(i)) gives 0 < q ( x ) < 1 0<q(x)<1 0 < q ( x ) < 1 . Thus, the first observed moment matrix is invertible, with
E [ V 1 i V 2 i T ∣ T i = 1 , X i = x ] − 1 = { G 2 ( 1 , x ) T } − 1 diag { 1 1 − q ( x ) , 1 q ( x ) } G 1 ( 1 , x ) − 1 . \mathbb{E}[V_{1i}V_{2i}^{\mathsf{T}} \mid T_i=1, X_i=x]^{-1}
=
\left\{G_2(1,x)^{\mathsf{T}}\right\}^{-1}
\operatorname{diag}\left\{\frac{1}{1-q(x)},\frac{1}{q(x)}\right\}
G_1(1,x)^{-1}. E [ V 1 i V 2 i T ∣ T i = 1 , X i = x ] − 1 = { G 2 ( 1 , x ) T } − 1 diag { 1 − q ( x ) 1 , q ( x ) 1 } G 1 ( 1 , x ) − 1 .
Multiplying the third-check-weighted matrix by this inverse yields
E [ W ~ 3 i V 1 i V 2 i T ∣ T i = 1 , X i = x ] E [ V 1 i V 2 i T ∣ T i = 1 , X i = x ] − 1 = G 1 ( 1 , x ) diag { m 1 ( 0 , x ) , m 1 ( 1 , x ) } G 1 ( 1 , x ) − 1 . \begin{aligned}
&\mathbb{E}[\widetilde{W}_{3i}V_{1i}V_{2i}^{\mathsf{T}} \mid T_i=1, X_i=x]\mathbb{E}[V_{1i}V_{2i}^{\mathsf{T}} \mid T_i=1, X_i=x]^{-1}\\
&\qquad=G_1(1,x)\operatorname{diag}\{m_1(0,x),m_1(1,x)\}G_1(1,x)^{-1}.
\end{aligned} E [ W 3 i V 1 i V 2 i T ∣ T i = 1 , X i = x ] E [ V 1 i V 2 i T ∣ T i = 1 , X i = x ] − 1 = G 1 ( 1 , x ) diag { m 1 ( 0 , x ) , m 1 ( 1 , x )} G 1 ( 1 , x ) − 1 .
The relevance assumption about the third check ensures that the two eigenvalues are distinct. The right eigenvectors therefore identify the columns of G 1 ( 1 , x ) G_1(1,x) G 1 ( 1 , x ) up to scale and order. Normalizing each eigenvector to have first entry one fixes its scale. The relevance condition assigns the column with the larger second entry to engaged respondents, identifying the ordered columns and the corresponding third-check means.
Exchanging the roles of the checks gives
E [ W ~ 3 i V 2 i V 1 i T ∣ T i = 1 , X i = x ] E [ V 2 i V 1 i T ∣ T i = 1 , X i = x ] − 1 = G 2 ( 1 , x ) diag { m 1 ( 0 , x ) , m 1 ( 1 , x ) } G 2 ( 1 , x ) − 1 . \begin{aligned}
&\mathbb{E}[\widetilde{W}_{3i}V_{2i}V_{1i}^{\mathsf{T}} \mid T_i=1, X_i=x]\mathbb{E}[V_{2i}V_{1i}^{\mathsf{T}} \mid T_i=1, X_i=x]^{-1}\\
&\qquad=G_2(1,x)\operatorname{diag}\{m_1(0,x),m_1(1,x)\}G_2(1,x)^{-1}.
\end{aligned} E [ W 3 i V 2 i V 1 i T ∣ T i = 1 , X i = x ] E [ V 2 i V 1 i T ∣ T i = 1 , X i = x ] − 1 = G 2 ( 1 , x ) diag { m 1 ( 0 , x ) , m 1 ( 1 , x )} G 2 ( 1 , x ) − 1 .
Normalizing its eigenvectors and matching their eigenvalues to the labeled third-check means identifies G 2 ( 1 , x ) G_2(1,x) G 2 ( 1 , x ) . The recovered ordering agrees with the positive check-mean difference required for the second check.
Now, for either check, the following equality holds,
E [ V j i ∣ T i = 1 , X i = x ] = G j ( 1 , x ) [ 1 − q ( x ) q ( x ) ] . \mathbb{E}[V_{ji} \mid T_i=1, X_i=x] = G_j(1,x)
\begin{bmatrix}
1-q(x)\\
q(x)
\end{bmatrix}. E [ V j i ∣ T i = 1 , X i = x ] = G j ( 1 , x ) [ 1 − q ( x ) q ( x ) ] .
Multiplication by the identified inverse of G j ( 1 , x ) G_j(1,x) G j ( 1 , x ) therefore identifies q ( x ) q(x) q ( x ) .
Define
ℓ j 0 ( 1 , x ) = G j ( 1 , x ) − T e 0 , j = 1 , 2. (22) \ell_{j0}(1,x)=G_j(1,x)^{-\mathsf{T}}e_0,\qquad j=1,2. \tag*{(22)} ℓ j 0 ( 1 , x ) = G j ( 1 , x ) − T e 0 , j = 1 , 2. (22)
Since these coefficients satisfy ℓ j 0 ( 1 , x ) T G j ( 1 , x ) = ( 1 , 0 ) \ell_{j0}(1,x)^{\mathsf{T}}G_j(1,x)=(1,0) ℓ j 0 ( 1 , x ) T G j ( 1 , x ) = ( 1 , 0 ) , we have
E [ ℓ j 0 ( 1 , x ) T V j i ∣ W i = w , T i = 1 , X i = x ] = 1 − w . \mathbb{E}[\ell_{j0}(1,x)^{\mathsf{T}}V_{ji}\mid W_i=w,T_i=1,X_i=x]=1-w. E [ ℓ j 0 ( 1 , x ) T V j i ∣ W i = w , T i = 1 , X i = x ] = 1 − w .
In particular,
1 − q ( x ) = ℓ j 0 ( 1 , x ) T E [ V j i ∣ T i = 1 , X i = x ] , j = 1 , 2. 1-q(x)=\ell_{j0}(1,x)^{\mathsf{T}}\mathbb{E}[V_{ji}\mid T_i=1,X_i=x],\qquad j=1,2. 1 − q ( x ) = ℓ j 0 ( 1 , x ) T E [ V j i ∣ T i = 1 , X i = x ] , j = 1 , 2.
The coefficients have the explicit form
ℓ j 0 ( 1 , x ) ⊤ = ( [ G j ( 1 , x ) ] 2 , 2 , − 1 ) [ G j ( 1 , x ) ] 2 , 2 − [ G j ( 1 , x ) ] 2 , 1 , \ell_{j0}(1,x)^{\top}=\frac{\left([G_j(1,x)]_{2,2},-1\right)}{[G_j(1,x)]_{2,2}-[G_j(1,x)]_{2,1}}, ℓ j 0 ( 1 , x ) ⊤ = [ G j ( 1 , x ) ] 2 , 2 − [ G j ( 1 , x ) ] 2 , 1 ( [ G j ( 1 , x ) ] 2 , 2 , − 1 ) ,
where the denominator is strictly positive because of the relevance condition.
Consistency and randomization imply q ( x ) = Pr ( W i ( 1 ) = 1 ∣ X i = x ) q(x)=\Pr(W_i(1)=1\mid X_i=x) q ( x ) = Pr ( W i ( 1 ) = 1 ∣ X i = x ) . For either j = 1 , 2 j=1,2 j = 1 , 2 , the law of iterated expectations gives
Q = ∫ q ( x ) d P X ( x ) = 1 − ∫ ℓ j 0 ( 1 , x ) ⊤ E [ V j i ∣ T i = 1 , X i = x ] d P X ( x ) = 1 − E [ E [ S j i ∣ T i = 1 , X i ] ] . \begin{aligned}
Q&=\int q(x)\,\mathrm{d}P_X(x)\\
&=1-\int\ell_{j0}(1,x)^{\top}\mathbb{E}[V_{ji}\mid T_i=1,X_i=x]\,\mathrm{d}P_X(x)\\
&=1-\mathbb{E}\left[\mathbb{E}[S_{ji}\mid T_i=1,X_i]\right].
\end{aligned} Q = ∫ q ( x ) d P X ( x ) = 1 − ∫ ℓ j 0 ( 1 , x ) ⊤ E [ V j i ∣ T i = 1 , X i = x ] d P X ( x ) = 1 − E [ E [ S j i ∣ T i = 1 , X i ] ] .
where S j i : = ℓ j 0 ( 1 , X i ) ⊤ V j i S_{ji}:=\ell_{j0}(1,X_i)^{\top}V_{ji} S j i := ℓ j 0 ( 1 , X i ) ⊤ V j i . Combining this result with Assumptions 3 and 4, as in § 3.2 , yields
Ψ = E [ E [ Y i ∣ T i = 1 , X i ] − E [ Y i ∣ T i = 0 , X i ] ] 1 − E [ E [ S j i ∣ T i = 1 , X i ] ] , j = 1 , 2. \Psi=\frac{\mathbb{E}\left[\mathbb{E}[Y_i\mid T_i=1,X_i]-\mathbb{E}[Y_i\mid T_i=0,X_i]\right]}{1-\mathbb{E}\left[\mathbb{E}[S_{ji}\mid T_i=1,X_i]\right]},\qquad j=1,2. Ψ = 1 − E [ E [ S j i ∣ T i = 1 , X i ] ] E [ E [ Y i ∣ T i = 1 , X i ] − E [ Y i ∣ T i = 0 , X i ] ] , j = 1 , 2.
This proves the proposition.
Proof of Proposition 2
Throughout the remainder of the appendix, we use the following notation
ϕ t i : = ϕ t ( T i , Y i , X i ; μ t ) , t = 0 , 1 , \phi_{ti}:=\phi_t(T_i,Y_i,X_i;\mu_t),\qquad t=0,1, ϕ t i := ϕ t ( T i , Y i , X i ; μ t ) , t = 0 , 1 ,
ϕ i Q : = ϕ Q ( T i , X i , W ~ 1 i , W ~ 2 i , W ~ 3 i ; θ ) , \phi_i^Q:=\phi^Q(T_i,X_i,\widetilde{W}_{1i},\widetilde{W}_{2i},\widetilde{W}_{3i};\theta), ϕ i Q := ϕ Q ( T i , X i , W 1 i , W 2 i , W 3 i ; θ ) ,
ϕ i : = ϕ ( T i , Y i , X i , W ~ 1 i , W ~ 2 i , W ~ 3 i ; θ , μ 0 , μ 1 , Q , Ψ ) . \phi_i:=\phi(T_i,Y_i,X_i,\widetilde{W}_{1i},\widetilde{W}_{2i},\widetilde{W}_{3i};\theta,\mu_0,\mu_1,Q,\Psi). ϕ i := ϕ ( T i , Y i , X i , W 1 i , W 2 i , W 3 i ; θ , μ 0 , μ 1 , Q , Ψ ) .
These functions are evaluated at the true nuisance functions and parameter values. We first establish the calibration properties of the surrogates and derive the influence function when W i W_i W i is observed. We then show that replacing W i W_i W i with 1 − S 0 i + C i 1-S_{0i}+C_i 1 − S 0 i + C i preserves the score expectations required for an observed-data influence function.
Lemmas and their Proofs
We begin by proving two lemmas.
Lemma 1 (Calibration and the correction term). Under Assumptions 5 and 6, for w = 0 , 1 w=0,1 w = 0 , 1 and almost every x x x ,
E [ S j i ∣ W i = w , T i = 1 , X i = x ] = 1 − w , j = 1 , 2 , \mathbb{E}[S_{ji}\mid W_i=w,T_i=1,X_i=x]=1-w,\qquad j=1,2, E [ S j i ∣ W i = w , T i = 1 , X i = x ] = 1 − w , j = 1 , 2 , E [ S 0 i ∣ W i = w , T i = 1 , X i = x ] = 1 − w , \mathbb{E}[S_{0i}\mid W_i=w,T_i=1,X_i=x]=1-w, E [ S 0 i ∣ W i = w , T i = 1 , X i = x ] = 1 − w , E [ C i ∣ W i = w , T i = 1 , X i = x ] = 0. \mathbb{E}[C_i\mid W_i=w,T_i=1,X_i=x]=0. E [ C i ∣ W i = w , T i = 1 , X i = x ] = 0. Consequently,
E [ 1 − S 0 i + C i ∣ W i = w , T i = 1 , X i = x ] = w , \mathbb{E}[1-S_{0i}+C_i\mid W_i=w,T_i=1,X_i=x]=w, E [ 1 − S 0 i + C i ∣ W i = w , T i = 1 , X i = x ] = w , E [ 1 − S 0 i + C i ∣ T i = 1 , X i = x ] = q ( x ) . \mathbb{E}[1-S_{0i}+C_i\mid T_i=1,X_i=x]=q(x). E [ 1 − S 0 i + C i ∣ T i = 1 , X i = x ] = q ( x ) .
Proof By (22) , we have,
E [ S j i ∣ W i = w , T i = 1 , X i = x ] = ℓ j 0 ( 1 , x ) ⊤ E [ V j i ∣ W i = w , T i = 1 , X i = x ] = 1 − w , \mathbb{E}[S_{ji}\mid W_i=w,T_i=1,\mathbf{X}_i=\mathbf{x}]=\ell_{j0}(1,\mathbf{x})^\top\mathbb{E}[V_{ji}\mid W_i=w,T_i=1,\mathbf{X}_i=\mathbf{x}]
=1-w, E [ S j i ∣ W i = w , T i = 1 , X i = x ] = ℓ j 0 ( 1 , x ) ⊤ E [ V j i ∣ W i = w , T i = 1 , X i = x ] = 1 − w ,
for j = 1 , 2 j=1,2 j = 1 , 2 . Since S 0 i = S 1 i S 2 i S_{0i}=S_{1i}S_{2i} S 0 i = S 1 i S 2 i , conditional independence of the first two checks gives,
E [ S 0 i ∣ W i = w , T i = 1 , X i = x ] = E [ S 1 i ∣ W i = w , T i = 1 , X i = x ] E [ S 2 i ∣ W i = w , T i = 1 , X i = x ] = ( 1 − w ) 2 = 1 − w . \mathbb{E}[S_{0i}\mid W_i=w,T_i=1,\mathbf{X}_i=\mathbf{x}]=\mathbb{E}[S_{1i}\mid W_i=w,T_i=1,\mathbf{X}_i=\mathbf{x}]\mathbb{E}[S_{2i}\mid W_i=w,T_i=1,\mathbf{X}_i=\mathbf{x}]
=(1-w)^2=1-w. E [ S 0 i ∣ W i = w , T i = 1 , X i = x ] = E [ S 1 i ∣ W i = w , T i = 1 , X i = x ] E [ S 2 i ∣ W i = w , T i = 1 , X i = x ] = ( 1 − w ) 2 = 1 − w .
The last equality follows because W i W_i W i is binary. Therefore,
E [ S 1 i + S 2 i − 2 S 0 i ∣ W i = w , T i = 1 , X i = x ] = ( 1 − w ) + ( 1 − w ) − 2 ( 1 − w ) = 0. \mathbb{E}[S_{1i}+S_{2i}-2S_{0i}\mid W_i=w,T_i=1,\mathbf{X}_i=\mathbf{x}]=(1-w)+(1-w)-2(1-w)=0. E [ S 1 i + S 2 i − 2 S 0 i ∣ W i = w , T i = 1 , X i = x ] = ( 1 − w ) + ( 1 − w ) − 2 ( 1 − w ) = 0.
Recall that the correction term is
C i = W ~ 3 i − m 1 ( 1 , X i ) m 1 ( 1 , X i ) − m 1 ( 0 , X i ) ( S 1 i + S 2 i − 2 S 0 i ) . C_i=\frac{\widetilde{W}_{3i}-m_1(1,\mathbf{X}_i)}{m_1(1,\mathbf{X}_i)-m_1(0,\mathbf{X}_i)}(S_{1i}+S_{2i}-2S_{0i}). C i = m 1 ( 1 , X i ) − m 1 ( 0 , X i ) W 3 i − m 1 ( 1 , X i ) ( S 1 i + S 2 i − 2 S 0 i ) .
The third check is independent of the first two checks conditional on { W i , T i = 1 , X i } \{W_i,T_i=1,\mathbf{X}_i\} { W i , T i = 1 , X i } . Hence,
E [ C i ∣ W i = w , T i = 1 , X i = x ] = m 1 ( w , x ) − m 1 ( 1 , x ) m 1 ( 1 , x ) − m 1 ( 0 , x ) E [ S 1 i + S 2 i − 2 S 0 i ∣ W i = w , T i = 1 , X i = x ] = 0. \mathbb{E}[C_i\mid W_i=w,T_i=1,\mathbf{X}_i=\mathbf{x}]
=\frac{m_1(w,\mathbf{x})-m_1(1,\mathbf{x})}{m_1(1,\mathbf{x})-m_1(0,\mathbf{x})}\mathbb{E}[S_{1i}+S_{2i}-2S_{0i}\mid W_i=w,T_i=1,\mathbf{X}_i=\mathbf{x}]=0. E [ C i ∣ W i = w , T i = 1 , X i = x ] = m 1 ( 1 , x ) − m 1 ( 0 , x ) m 1 ( w , x ) − m 1 ( 1 , x ) E [ S 1 i + S 2 i − 2 S 0 i ∣ W i = w , T i = 1 , X i = x ] = 0.
Combining these equalities and then averaging over W i W_i W i yields
E [ 1 − S 0 i + C i ∣ W i = w , T i = 1 , X i = x ] = 1 − ( 1 − w ) + 0 = w , \mathbb{E}[1-S_{0i}+C_i\mid W_i=w,T_i=1,\mathbf{X}_i=\mathbf{x}]=1-(1-w)+0=w, E [ 1 − S 0 i + C i ∣ W i = w , T i = 1 , X i = x ] = 1 − ( 1 − w ) + 0 = w ,
E [ 1 − S 0 i + C i ∣ T i = 1 , X i = x ] = E [ W i ∣ T i = 1 , X i = x ] = q ( x ) . \mathbb{E}[1-S_{0i}+C_i\mid T_i=1,\mathbf{X}_i=\mathbf{x}]=\mathbb{E}[W_i\mid T_i=1,\mathbf{X}_i=\mathbf{x}]=q(\mathbf{x}). E [ 1 − S 0 i + C i ∣ T i = 1 , X i = x ] = E [ W i ∣ T i = 1 , X i = x ] = q ( x ) .
Lemma 2 (Full-data influence functions). Suppose that Assumptions 1–4 hold, 0 < π 1 ( x ) < 1 0<\pi_1(\mathbf{x})<1 0 < π 1 ( x ) < 1 for almost every x \mathbf{x} x , Q > 0 Q>0 Q > 0 , and E [ Y i 2 ] < ∞ \mathbb{E}[Y_i^2]<\infty E [ Y i 2 ] < ∞ . Assume also that ϕ 0 i \phi_{0i} ϕ 0 i , ϕ 1 i \phi_{1i} ϕ 1 i , and ϕ i Q , f u l l \phi_i^{Q,\mathrm{full}} ϕ i Q , full below are square-integrable. If the realized engagement status W i W_i W i is observed, an uncentered influence function for Q Q Q and an influence function for Ψ \Psi Ψ are given by
ϕ i Q , f u l l : = ϕ Q , f u l l ( T i , X i , W i ; q ) = q ( X i ) + 1 { T i = 1 } π 1 ( X i ) { W i − q ( X i ) } , \phi_i^{Q,\mathrm{full}}:=\phi^{Q,\mathrm{full}}(T_i,\mathbf{X}_i,W_i;q)=q(\mathbf{X}_i)+\frac{1\{T_i=1\}}{\pi_1(\mathbf{X}_i)}\{W_i-q(\mathbf{X}_i)\}, ϕ i Q , full := ϕ Q , full ( T i , X i , W i ; q ) = q ( X i ) + π 1 ( X i ) 1 { T i = 1 } { W i − q ( X i )} , ϕ i f u l l : = ϕ f u l l ( T i , Y i , X i , W i ; q , μ 0 , μ 1 , Q , Ψ ) = ϕ 1 i − ϕ 0 i − Ψ ϕ i Q , f u l l Q , \phi_i^{\mathrm{full}}:=\phi^{\mathrm{full}}(T_i,Y_i,\mathbf{X}_i,W_i;q,\mu_0,\mu_1,Q,\Psi)=\frac{\phi_{1i}-\phi_{0i}-\Psi\phi_i^{Q,\mathrm{full}}}{Q}, ϕ i full := ϕ full ( T i , Y i , X i , W i ; q , μ 0 , μ 1 , Q , Ψ ) = Q ϕ 1 i − ϕ 0 i − Ψ ϕ i Q , full , where ϕ t i \phi_{ti} ϕ t i is the uncentered influence function for Ψ t \Psi_t Ψ t defined in Proposition 2.
Proof Consider any regular parametric submodel of the full-data model through the true law. Denote its score by s F s^F s F and the marginal score for X i \mathbf{X}_i X i by s X s_X s X . Then,
E [ s F ] = 0 , E [ s X ] = 0 , E [ s F ∣ X i ] = s X . \mathbb{E}[s^F]=0,\qquad\mathbb{E}[s_X]=0,\qquad\mathbb{E}[s^F\mid\mathbf{X}_i]=s_X. E [ s F ] = 0 , E [ s X ] = 0 , E [ s F ∣ X i ] = s X .
We first derive the influence function for Ψ t \Psi_t Ψ t . Denote differentiation along the submodel at the true law by a dot. Then, differentiating the conditional mean gives
μ ˙ t ( x ) = E [ Y i { s F − E [ s F ∣ T i = t , X i = x ] } ∣ T i = t , X i = x ] = E [ { Y i − μ t ( x ) } s F ∣ T i = t , X i = x ] . \dot{\mu}_t(x) = \mathbb{E}\left[Y_i\{s^F-\mathbb{E}[s^F\mid T_i=t,X_i=x]\}\mid T_i=t,X_i=x\right]
= \mathbb{E}\left[\{Y_i-\mu_t(x)\}s^F\mid T_i=t,X_i=x\right]. μ ˙ t ( x ) = E [ Y i { s F − E [ s F ∣ T i = t , X i = x ]} ∣ T i = t , X i = x ] = E [ { Y i − μ t ( x )} s F ∣ T i = t , X i = x ] .
Since Ψ t = ∫ μ t ( x ) d P X ( x ) \Psi_t=\int\mu_t(x)\,\mathrm{d}P_X(x) Ψ t = ∫ μ t ( x ) d P X ( x ) ,
Ψ ˙ t = ∫ μ ˙ t ( x ) d P X ( x ) + E [ μ t ( X i ) s X ] = E [ 1 { T i = t } π t ( X i ) { Y i − μ t ( X i ) } s F ] + E [ { μ t ( X i ) − Ψ t } s F ] = E [ ( 1 { T i = t } π t ( X i ) { Y i − μ t ( X i ) } + { μ t ( X i ) − Ψ t } ) s F ] . \begin{aligned}
\dot{\Psi}_t = \int\dot{\mu}_t(x)\,\mathrm{d}P_X(x)+\mathbb{E}[\mu_t(X_i)s_X] \\
&= \mathbb{E}\left[\frac{1\{T_i=t\}}{\pi_t(X_i)}\{Y_i-\mu_t(X_i)\}s^F\right]+\mathbb{E}[\{\mu_t(X_i)-\Psi_t\}s^F] \\
&= \mathbb{E}\left[\left(\frac{1\{T_i=t\}}{\pi_t(X_i)}\{Y_i-\mu_t(X_i)\}+\{\mu_t(X_i)-\Psi_t\}\right)s^F\right].
\end{aligned} Ψ ˙ t = ∫ μ ˙ t ( x ) d P X ( x ) + E [ μ t ( X i ) s X ] = E [ π t ( X i ) 1 { T i = t } { Y i − μ t ( X i )} s F ] + E [{ μ t ( X i ) − Ψ t } s F ] = E [ ( π t ( X i ) 1 { T i = t } { Y i − μ t ( X i )} + { μ t ( X i ) − Ψ t } ) s F ] .
Similarly, differentiating q ( x ) = P ( W i = 1 ∣ T i = 1 , X i = x ) q(x)=\mathbb{P}(W_i=1\mid T_i=1,X_i=x) q ( x ) = P ( W i = 1 ∣ T i = 1 , X i = x ) gives
q ˙ ( x ) = E [ W i { s F − E [ s F ∣ T i = 1 , X i = x ] } ∣ T i = 1 , X i = x ] = E [ { W i − q ( x ) } s F ∣ T i = 1 , X i = x ] . \dot{q}(x)=\mathbb{E}\left[W_i\{s^F-\mathbb{E}[s^F\mid T_i=1,X_i=x]\}\mid T_i=1,X_i=x\right]
=\mathbb{E}\left[\{W_i-q(x)\}s^F\mid T_i=1,X_i=x\right]. q ˙ ( x ) = E [ W i { s F − E [ s F ∣ T i = 1 , X i = x ]} ∣ T i = 1 , X i = x ] = E [ { W i − q ( x )} s F ∣ T i = 1 , X i = x ] .
Therefore,
Q ˙ = ∫ q ˙ ( x ) d P X ( x ) + E [ q ( X i ) s X ] = E [ ( 1 { T i = 1 } π 1 ( X i ) { W i − q ( X i ) } + { q ( X i ) − Q } ) s F ] . \dot{Q}=\int\dot{q}(x)\,\mathrm{d}P_X(x)+\mathbb{E}[q(X_i)s_X]
=\mathbb{E}\left[\left(\frac{1\{T_i=1\}}{\pi_1(X_i)}\{W_i-q(X_i)\}+\{q(X_i)-Q\}\right)s^F\right]. Q ˙ = ∫ q ˙ ( x ) d P X ( x ) + E [ q ( X i ) s X ] = E [ ( π 1 ( X i ) 1 { T i = 1 } { W i − q ( X i )} + { q ( X i ) − Q } ) s F ] .
Finally, since Ψ = ( Ψ 1 − Ψ 0 ) / Q \Psi=(\Psi_1-\Psi_0)/Q Ψ = ( Ψ 1 − Ψ 0 ) / Q , by quotient rule,
Ψ ˙ = ( Ψ ˙ 1 − Ψ ˙ 0 ) Q − ( Ψ 1 − Ψ 0 ) Q ˙ Q 2 = Ψ ˙ 1 − Ψ ˙ 0 − Ψ Q ˙ Q = E [ ϕ 1 i − ϕ 0 i − Ψ ϕ i Q , f u l l Q s F ] . \dot{\Psi}=\frac{(\dot{\Psi}_1-\dot{\Psi}_0)Q-(\Psi_1-\Psi_0)\dot{Q}}{Q^2}
=\frac{\dot{\Psi}_1-\dot{\Psi}_0-\Psi\dot{Q}}{Q}
=\mathbb{E}\left[\frac{\phi_{1i}-\phi_{0i}-\Psi\phi_i^{Q,\mathrm{full}}}{Q}s^F\right]. Ψ ˙ = Q 2 ( Ψ ˙ 1 − Ψ ˙ 0 ) Q − ( Ψ 1 − Ψ 0 ) Q ˙ = Q Ψ ˙ 1 − Ψ ˙ 0 − Ψ Q ˙ = E [ Q ϕ 1 i − ϕ 0 i − Ψ ϕ i Q , full s F ] .
which establishes the desired result. □ \square □
Proof of Proposition 2
We now relate the full-data and observed-data influence functions, following the score argument in Zhou and Tchetgen Tchetgen [35 ] .
Proof Let s F s^F s F be the score of any regular parametric submodel of the full-data model, as in Lemma 2. Marginalizing the full-data likelihood over W i W_i W i gives the observed-data score
s O = E [ s F ∣ O i ] . s^O=\mathbb{E}[s^F\mid O_i]. s O = E [ s F ∣ O i ] .
Since ϕ i Q \phi_i^Q ϕ i Q is measurable with respect to O i O_i O i ,
E [ ϕ i Q s O ] = E [ ϕ i Q E [ s F ∣ O i ] ] = E [ ϕ i Q s F ] . \mathbb{E}[\phi_i^Q s^O]=\mathbb{E}[\phi_i^Q\mathbb{E}[s^F\mid O_i]]=\mathbb{E}[\phi_i^Q s^F]. E [ ϕ i Q s O ] = E [ ϕ i Q E [ s F ∣ O i ]] = E [ ϕ i Q s F ] .
Thus, by Lemma 2, it suffices to show that
E [ ( ϕ i Q − ϕ i Q , f u l l ) s F ] = 0 (23) \mathbb{E}[(\phi_i^Q-\phi_i^{Q,\mathrm{full}})s^F]=0 \tag*{(23)} E [( ϕ i Q − ϕ i Q , full ) s F ] = 0 (23)
for every such score, and that E [ ϕ i Q ] = Q \mathbb{E}[\phi_i^Q]=Q E [ ϕ i Q ] = Q .
Under Assumption 5, the conditional full-data density in the treated arm factors as
p ( W i ∣ T i = 1 , X i ) ∏ j = 1 3 p ( W ~ j i ∣ W i , T i = 1 , X i ) p ( Y i ∣ W i , W ~ 1 i , W ~ 2 i , W ~ 3 i , T i = 1 , X i ) . p(W_i \mid T_i=1,\mathbf{X}_i)\prod_{j=1}^{3}p(\widetilde{W}_{ji}\mid W_i,T_i=1,\mathbf{X}_i)p(Y_i\mid W_i,\widetilde{W}_{1i},\widetilde{W}_{2i},\widetilde{W}_{3i},T_i=1,\mathbf{X}_i). p ( W i ∣ T i = 1 , X i ) j = 1 ∏ 3 p ( W j i ∣ W i , T i = 1 , X i ) p ( Y i ∣ W i , W 1 i , W 2 i , W 3 i , T i = 1 , X i ) .
Differentiating the log likelihood therefore yields
s F = s X + 1 { T i = 1 } ( s W + s 1 + s 2 + s 3 + s Y ) + 1 { T i = 0 } s c t r , s^F=s_X+1\{T_i=1\}(s_W+s_1+s_2+s_3+s_Y)+1\{T_i=0\}s_{\mathrm{ctr}}, s F = s X + 1 { T i = 1 } ( s W + s 1 + s 2 + s 3 + s Y ) + 1 { T i = 0 } s ctr ,
where s c t r s_{\mathrm{ctr}} s ctr is the score of the conditional full-data distribution in the control arm. There is no treatment score because the randomization probabilities are fixed. The conditional scores satisfy
E [ s W ∣ T i = 1 , X i ] = 0 , \mathbb{E}[s_W\mid T_i=1,\mathbf{X}_i]=0, E [ s W ∣ T i = 1 , X i ] = 0 ,
E [ s j ∣ W i , T i = 1 , X i ] = 0 , j = 1 , 2 , 3 , \mathbb{E}[s_j\mid W_i,T_i=1,\mathbf{X}_i]=0,\qquad j=1,2,3, E [ s j ∣ W i , T i = 1 , X i ] = 0 , j = 1 , 2 , 3 ,
E [ s Y ∣ W i , W ~ 1 i , W ~ 2 i , W ~ 3 i , T i = 1 , X i ] = 0. \mathbb{E}[s_Y\mid W_i,\widetilde{W}_{1i},\widetilde{W}_{2i},\widetilde{W}_{3i},T_i=1,\mathbf{X}_i]=0. E [ s Y ∣ W i , W 1 i , W 2 i , W 3 i , T i = 1 , X i ] = 0.
Now, the difference between the two expressions for Q Q Q is
ϕ i Q − ϕ i Q , f u l l = 1 { T i = 1 } π 1 ( X i ) { 1 − S 0 i + C i − W i } . \phi_i^Q-\phi_i^{Q,\mathrm{full}}=\frac{1\{T_i=1\}}{\pi_1(\mathbf{X}_i)}\{1-S_{0i}+C_i-W_i\}. ϕ i Q − ϕ i Q , full = π 1 ( X i ) 1 { T i = 1 } { 1 − S 0 i + C i − W i } .
To verify Equation (23) , we first calculate the conditional expectation of this difference given each check and the latent engagement status.
For the third check, conditional independence and Lemma 1 give
E [ S 0 i ∣ W i = w , W ~ 3 i , T i = 1 , X i = x ] = 1 − w , \mathbb{E}[S_{0i}\mid W_i=w,\widetilde{W}_{3i},T_i=1,\mathbf{X}_i=\mathbf{x}]=1-w, E [ S 0 i ∣ W i = w , W 3 i , T i = 1 , X i = x ] = 1 − w ,
E [ S 1 i + S 2 i − 2 S 0 i ∣ W i = w , W ~ 3 i , T i = 1 , X i = x ] = 0. \mathbb{E}[S_{1i}+S_{2i}-2S_{0i}\mid W_i=w,\widetilde{W}_{3i},T_i=1,\mathbf{X}_i=\mathbf{x}]=0. E [ S 1 i + S 2 i − 2 S 0 i ∣ W i = w , W 3 i , T i = 1 , X i = x ] = 0.
The third-check factor in C i C_i C i is fixed under this conditioning. Consequently,
E [ 1 − S 0 i + C i − W i ∣ W i = w , W ~ 3 i , T i = 1 , X i = x ] = 1 − ( 1 − w ) + W ~ 3 i − m 1 ( 1 , x ) m 1 ( 1 , x ) − m 1 ( 0 , x ) × 0 − w = 0. (24) \begin{aligned}
\mathbb{E}[1-S_{0i}+C_i-W_i\mid W_i=w,\widetilde{W}_{3i},T_i=1,\mathbf{X}_i=\mathbf{x}]
&=1-(1-w)+\frac{\widetilde{W}_{3i}-m_1(1,\mathbf{x})}{m_1(1,\mathbf{x})-m_1(0,\mathbf{x})}\times0-w=0. \tag*{(24)}
\end{aligned} E [ 1 − S 0 i + C i − W i ∣ W i = w , W 3 i , T i = 1 , X i = x ] = 1 − ( 1 − w ) + m 1 ( 1 , x ) − m 1 ( 0 , x ) W 3 i − m 1 ( 1 , x ) × 0 − w = 0. (24)
For either of the first two checks, let j ∈ { 1 , 2 } j\in\{1,2\} j ∈ { 1 , 2 } . Conditional independence of the checks implies
E [ S 0 i ∣ W i = w , W ~ j i , T i = 1 , X i = x ] = ( 1 − w ) S j i , \mathbb{E}[S_{0i}\mid W_i=w,\widetilde{W}_{ji},T_i=1,\mathbf{X}_i=\mathbf{x}]=(1-w)S_{ji}, E [ S 0 i ∣ W i = w , W j i , T i = 1 , X i = x ] = ( 1 − w ) S j i ,
E [ S 1 i + S 2 i − 2 S 0 i ∣ W i = w , W ~ j i , T i = 1 , X i = x ] = S j i + ( 1 − w ) − 2 ( 1 − w ) S j i . \mathbb{E}[S_{1i}+S_{2i}-2S_{0i}\mid W_i=w,\widetilde{W}_{ji},T_i=1,\mathbf{X}_i=\mathbf{x}]=S_{ji}+(1-w)-2(1-w)S_{ji}. E [ S 1 i + S 2 i − 2 S 0 i ∣ W i = w , W j i , T i = 1 , X i = x ] = S j i + ( 1 − w ) − 2 ( 1 − w ) S j i .
Since W i W_i W i is binary,
m 1 ( w , x ) − m 1 ( 1 , x ) m 1 ( 1 , x ) − m 1 ( 0 , x ) = w − 1. \frac{m_1(w,\mathbf{x})-m_1(1,\mathbf{x})}{m_1(1,\mathbf{x})-m_1(0,\mathbf{x})}=w-1. m 1 ( 1 , x ) − m 1 ( 0 , x ) m 1 ( w , x ) − m 1 ( 1 , x ) = w − 1.
Hence,
E [ C i ∣ W i = w , W ~ j i , T i = 1 , X i = x ] = ( w − 1 ) { S j i + ( 1 − w ) − 2 ( 1 − w ) S j i } = ( w − 1 ) ( 2 w − 1 ) S j i + ( w − 1 ) ( 1 − w ) = ( 1 − w ) ( S j i − 1 ) , \begin{aligned}
\mathbb{E}[C_i\mid W_i=w,\widetilde{W}_{ji},T_i=1,\mathbf{X}_i=\mathbf{x}]
&=(w-1)\{S_{ji}+(1-w)-2(1-w)S_{ji}\}\\
&=(w-1)(2w-1)S_{ji}+(w-1)(1-w) \\
&= (1-w)(S_{ji}-1),
\end{aligned} E [ C i ∣ W i = w , W j i , T i = 1 , X i = x ] = ( w − 1 ) { S j i + ( 1 − w ) − 2 ( 1 − w ) S j i } = ( w − 1 ) ( 2 w − 1 ) S j i + ( w − 1 ) ( 1 − w ) = ( 1 − w ) ( S j i − 1 ) ,
where the last equality uses w 2 = w w^2=w w 2 = w . Therefore,
E [ 1 − S 0 i + C i − W i ∣ W i = w , W ~ j i , T i = 1 , X i = x ] = 1 − ( 1 − w ) S j i + ( 1 − w ) ( S j i − 1 ) − w = 0 , j = 1 , 2. (25) \mathbb{E}[1-S_{0i}+C_i-W_i\mid W_i=w,\widetilde{W}_{ji},T_i=1,X_i=x]
=1-(1-w)S_{ji}+(1-w)(S_{ji}-1)-w=0,\qquad j=1,2. \tag*{(25)} E [ 1 − S 0 i + C i − W i ∣ W i = w , W j i , T i = 1 , X i = x ] = 1 − ( 1 − w ) S j i + ( 1 − w ) ( S j i − 1 ) − w = 0 , j = 1 , 2. (25)
To apply these identities to the score, Equations (S3) and (S4) imply
E [ 1 − S 0 i + C i − W i ∣ W i , W ~ j i , T i = 1 , X i ] = 0 , j = 1 , 2 , 3 , \mathbb{E}[1-S_{0i}+C_i-W_i\mid W_i,\widetilde{W}_{ji},T_i=1,X_i]=0,\qquad j=1,2,3, E [ 1 − S 0 i + C i − W i ∣ W i , W j i , T i = 1 , X i ] = 0 , j = 1 , 2 , 3 ,
E [ 1 − S 0 i + C i − W i ∣ W i , T i = 1 , X i ] = 0 , \mathbb{E}[1-S_{0i}+C_i-W_i\mid W_i,T_i=1,X_i]=0, E [ 1 − S 0 i + C i − W i ∣ W i , T i = 1 , X i ] = 0 ,
E [ 1 − S 0 i + C i − W i ∣ T i = 1 , X i ] = 0. \mathbb{E}[1-S_{0i}+C_i-W_i\mid T_i=1,X_i]=0. E [ 1 − S 0 i + C i − W i ∣ T i = 1 , X i ] = 0.
The last two equalities follow by iterated expectations. Since s X s_X s X is a function of X i X_i X i , s W s_W s W is a function of ( W i , X i ) (W_i,X_i) ( W i , X i ) in the treated arm, and each s j s_j s j is a function of ( W ~ j i , W i , X i ) (\widetilde{W}_{ji},W_i,X_i) ( W j i , W i , X i ) in that arm,
E [ ( 1 − S 0 i + C i − W i ) s X ∣ T i = 1 , X i ] = s X E [ ( 1 − S 0 i + C i − W i ) ∣ T i = 1 , X i ] = 0 , \mathbb{E}[(1-S_{0i}+C_i-W_i)s_X\mid T_i=1,X_i]
=s_X\mathbb{E}[(1-S_{0i}+C_i-W_i)\mid T_i=1,X_i]=0, E [( 1 − S 0 i + C i − W i ) s X ∣ T i = 1 , X i ] = s X E [( 1 − S 0 i + C i − W i ) ∣ T i = 1 , X i ] = 0 ,
E [ ( 1 − S 0 i + C i − W i ) s W ∣ T i = 1 , X i ] = E [ s W E [ ( 1 − S 0 i + C i − W i ) ∣ W i , T i = 1 , X i ] ∣ T i = 1 , X i ] = 0 , \mathbb{E}[(1-S_{0i}+C_i-W_i)s_W\mid T_i=1,X_i]
=\mathbb{E}[s_W\mathbb{E}[(1-S_{0i}+C_i-W_i)\mid W_i,T_i=1,X_i]\mid T_i=1,X_i]=0, E [( 1 − S 0 i + C i − W i ) s W ∣ T i = 1 , X i ] = E [ s W E [( 1 − S 0 i + C i − W i ) ∣ W i , T i = 1 , X i ] ∣ T i = 1 , X i ] = 0 ,
E [ ( 1 − S 0 i + C i − W i ) s j ∣ T i = 1 , X i ] = E [ s j E [ 1 − S 0 i + C i − W i ∣ W i , W ~ j i , T i = 1 , X i ] ∣ T i = 1 , X i ] = 0 \mathbb{E}[(1-S_{0i}+C_i-W_i)s_j\mid T_i=1,X_i]
=\mathbb{E}\left[s_j\mathbb{E}[1-S_{0i}+C_i-W_i\mid W_i,\widetilde{W}_{ji},T_i=1,X_i]\mid T_i=1,X_i\right]=0 E [( 1 − S 0 i + C i − W i ) s j ∣ T i = 1 , X i ] = E [ s j E [ 1 − S 0 i + C i − W i ∣ W i , W j i , T i = 1 , X i ] ∣ T i = 1 , X i ] = 0
for j = 1 , 2 , 3 j=1,2,3 j = 1 , 2 , 3 . For the outcome score, 1 − S 0 i + C i − W i 1-S_{0i}+C_i-W_i 1 − S 0 i + C i − W i is measurable given W i W_i W i , the three checks, and X i X_i X i . Thus,
E [ ( 1 − S 0 i + C i − W i ) s Y ∣ W i , W ~ 1 i , W ~ 2 i , W ~ 3 i , T i = 1 , X i ] = ( 1 − S 0 i + C i − W i ) E [ s Y ∣ W i , W ~ 1 i , W ~ 2 i , W ~ 3 i , T i = 1 , X i ] = 0. \mathbb{E}[(1-S_{0i}+C_i-W_i)s_Y\mid W_i,\widetilde{W}_{1i},\widetilde{W}_{2i},\widetilde{W}_{3i},T_i=1,X_i]
=(1-S_{0i}+C_i-W_i)\mathbb{E}[s_Y\mid W_i,\widetilde{W}_{1i},\widetilde{W}_{2i},\widetilde{W}_{3i},T_i=1,X_i]=0. E [( 1 − S 0 i + C i − W i ) s Y ∣ W i , W 1 i , W 2 i , W 3 i , T i = 1 , X i ] = ( 1 − S 0 i + C i − W i ) E [ s Y ∣ W i , W 1 i , W 2 i , W 3 i , T i = 1 , X i ] = 0.
The control-arm contribution is zero because 1 { T i = 1 } 1 { T i = 0 } = 0 1\{T_i=1\}1\{T_i=0\}=0 1 { T i = 1 } 1 { T i = 0 } = 0 . Combining these results gives
E [ ( ϕ i Q − ϕ i Q , f u l l ) s F ] = E [ 1 { T i = 1 } π 1 ( X i ) ( 1 − S 0 i + C i − W i ) s F ] = ∫ E [ ( 1 − S 0 i + C i − W i ) ( s X + s W + s 1 + s 2 + s 3 + s Y ) ∣ T i = 1 , X i = x ] d P X ( x ) = 0. \begin{aligned}
\mathbb{E}[(\phi_i^Q-\phi_i^{Q,\mathrm{full}})s^F]
=\mathbb{E}\left[\frac{1\{T_i=1\}}{\pi_1(X_i)}(1-S_{0i}+C_i-W_i)s^F\right] \\
&=\int\mathbb{E}[(1-S_{0i}+C_i-W_i)(s_X+s_W+s_1+s_2+s_3+s_Y)\mid T_i=1,X_i=x]\,dP_X(x) \\
&=0.
\end{aligned} E [( ϕ i Q − ϕ i Q , full ) s F ] = E [ π 1 ( X i ) 1 { T i = 1 } ( 1 − S 0 i + C i − W i ) s F ] = ∫ E [( 1 − S 0 i + C i − W i ) ( s X + s W + s 1 + s 2 + s 3 + s Y ) ∣ T i = 1 , X i = x ] d P X ( x ) = 0.
It follows that
Q ˙ = E [ ϕ i Q , f u l l s F ] = E [ ϕ i Q s F ] = E [ ϕ i Q s O ] = E [ ( ϕ i Q − Q ) s O ] . \dot{Q}=\mathbb{E}[\phi_i^{Q,\mathrm{full}}s^F]=\mathbb{E}[\phi_i^Qs^F]=\mathbb{E}[\phi_i^Qs^O]=\mathbb{E}[(\phi_i^Q-Q)s^O]. Q ˙ = E [ ϕ i Q , full s F ] = E [ ϕ i Q s F ] = E [ ϕ i Q s O ] = E [( ϕ i Q − Q ) s O ] .
Moreover, Lemma 1 implies
E [ ϕ i Q ] = E [ q ( X i ) ] + ∫ { E [ 1 − S 0 i + C i ∣ T i = 1 , X i = x ] − q ( x ) } d P X ( x ) = Q . \mathbb{E}[\phi_i^Q]=\mathbb{E}[q(X_i)]+\int\{\mathbb{E}[1-S_{0i}+C_i\mid T_i=1,X_i=x]-q(x)\}\,dP_X(x)=Q. E [ ϕ i Q ] = E [ q ( X i )] + ∫ { E [ 1 − S 0 i + C i ∣ T i = 1 , X i = x ] − q ( x )} d P X ( x ) = Q .
Thus, ϕ i Q − Q \phi_i^Q-Q ϕ i Q − Q is an observed-data influence function for Q Q Q .
Finally, ϕ t i \phi_{ti} ϕ t i is observed-data measurable, so
ψ ˙ t = E [ ϕ t i s F ] = E [ ϕ t i E [ s F ∣ O i ] ] = E [ ϕ t i s O ] . \dot{\psi}_t=\mathbb{E}[\phi_{ti}s^F]=\mathbb{E}[\phi_{ti}\mathbb{E}[s^F\mid O_i]]=\mathbb{E}[\phi_{ti}s^O]. ψ ˙ t = E [ ϕ t i s F ] = E [ ϕ t i E [ s F ∣ O i ]] = E [ ϕ t i s O ] .
Applying the quotient rule,
Ψ ˙ = Ψ ˙ 1 − Ψ ˙ 0 − Ψ Q ˙ Q = E [ ϕ 1 i − ϕ 0 i − Ψ ϕ i Q Q − s O ] = E [ ϕ i s O ] . \dot{\Psi} = \frac{\dot{\Psi}_1 - \dot{\Psi}_0 - \Psi\dot{Q}}{Q} = \mathbb{E}\left[\frac{\phi_{1i} - \phi_{0i} - \Psi\phi_i^Q}{Q} - s^O\right] = \mathbb{E}\left[\phi_i^{s^O}\right]. Ψ ˙ = Q Ψ ˙ 1 − Ψ ˙ 0 − Ψ Q ˙ = E [ Q ϕ 1 i − ϕ 0 i − Ψ ϕ i Q − s O ] = E [ ϕ i s O ] .
Also,
E [ ϕ i ] = E [ ϕ 1 i ] − E [ ϕ 0 i ] − Ψ E [ ϕ i Q ] Q = Ψ 1 − Ψ 0 − Ψ Q Q = 0. \mathbb{E}[\phi_i] = \frac{\mathbb{E}[\phi_{1i}] - \mathbb{E}[\phi_{0i}] - \Psi\mathbb{E}[\phi_i^Q]}{Q}
= \frac{\Psi_1 - \Psi_0 - \Psi Q}{Q} = 0. E [ ϕ i ] = Q E [ ϕ 1 i ] − E [ ϕ 0 i ] − Ψ E [ ϕ i Q ] = Q Ψ 1 − Ψ 0 − Ψ Q = 0.
Thus, these identities establish the influence function in Equation (15) . ∎
Proof of Proposition 3
Assumption 7 (Regularity Conditions). There exist fixed constants c 1 > 0 c_{1} > 0 c 1 > 0 and c 2 ∈ ( 0 , 1 / 2 ) c_{2} \in(0,1/2) c 2 ∈ ( 0 , 1/2 ) , and a deterministic sequence δ n > 0 \delta_{n} > 0 δ n > 0 with δ n → 0 \delta_{n} \to0 δ n → 0 , such that the following conditions hold. We suppress the superscript ( − k ) (-k) ( − k ) for notational simplicity.
(a) (Boundedness ) The outcome and numerically coded checks satisfy
Pr ( max { ∣ Y i ∣ , ∣ W ~ 1 i ∣ , ∣ W ~ 2 i ∣ , ∣ W ~ 3 i ∣ } ≤ c 1 ) = 1. \Pr\left(\max\{|Y_{i}|,|\widetilde{W}_{1i}|,|\widetilde{W}_{2i}|,|\widetilde{W}_{3i}|\} \le c_{1}\right)=1. Pr ( max { ∣ Y i ∣ , ∣ W 1 i ∣ , ∣ W 2 i ∣ , ∣ W 3 i ∣ } ≤ c 1 ) = 1. All estimated nuisance functions are also uniformly bounded in absolute value by c 1 c_{1} c 1 almost surely, componentwise for vector- and matrix-valued functions.
(b) (Positivity and Relevance ) For almost every x x x ,
c 2 ≤ π t ( x ) ≤ 1 − c 2 , t = 0 , 1 , c 2 ≤ q ( x ) ≤ 1 − c 2 , σ min { G j ( 1 , x ) } ≥ c 2 , j = 1 , 2 , [ G j ( 1 , x ) ] 2 , 2 − [ G j ( 1 , x ) ] 2 , 1 ≥ c 2 , j = 1 , 2 , ∣ m 1 ( 1 , x ) − m 1 ( 0 , x ) ∣ ≥ c 2 . \begin{aligned}
c_{2} &\le\pi_{t}(x) \le1-c_{2}, && t=0,1,\\
c_{2} &\le q(x) \le1-c_{2},\\
\sigma_{\min}\{G_{j}(1,x)\} &\ge c_{2}, && j=1,2,\\
[G_{j}(1,x)]_{2,2}-[G_{j}(1,x)]_{2,1} &\ge c_{2}, && j=1,2,\\
|m_{1}(1,x)-m_{1}(0,x)| &\ge c_{2}.
\end{aligned} c 2 c 2 σ m i n { G j ( 1 , x )} [ G j ( 1 , x ) ] 2 , 2 − [ G j ( 1 , x ) ] 2 , 1 ∣ m 1 ( 1 , x ) − m 1 ( 0 , x ) ∣ ≤ π t ( x ) ≤ 1 − c 2 , ≤ q ( x ) ≤ 1 − c 2 , ≥ c 2 , ≥ c 2 , ≥ c 2 . t = 0 , 1 , j = 1 , 2 , j = 1 , 2 , Here, σ min \sigma_{\min} σ m i n denotes the smallest singular value. The estimated functions q ^ \widehat{q} q , G ^ j \widehat{G}_{j} G j , and m ^ 1 \widehat{m}_{1} m 1 satisfy the corresponding bounds almost surely.
(c) (Nuisance Function Estimation ) With probability tending to one, the nuisance estimates satisfy
E [ ∣ μ ^ t ( X i ) − μ t ( X i ) ∣ 2 ] 1 / 2 ≤ δ n , t = 0 , 1 , \mathbb{E}\left[\left|\widehat{\mu}_{t}(X_{i})-\mu_{t}(X_{i})\right|^{2}\right]^{1/2}\le\delta_{n},\qquad t=0,1, E [ ∣ μ t ( X i ) − μ t ( X i ) ∣ 2 ] 1/2 ≤ δ n , t = 0 , 1 , E [ ∣ q ^ ( X i ) − q ( X i ) ∣ 2 ] 1 / 2 ≤ δ n , \mathbb{E}\left[\left|\widehat{q}(X_{i})-q(X_{i})\right|^{2}\right]^{1/2}\le\delta_{n}, E [ ∣ q ( X i ) − q ( X i ) ∣ 2 ] 1/2 ≤ δ n , E [ ∑ r = 0 1 { [ ℓ ^ j 0 ( 1 , X i ) ] r − [ ℓ j 0 ( 1 , X i ) ] r } 2 ] 1 / 2 ≤ n − 1 / 4 δ n , j = 1 , 2 , \mathbb{E}\left[\sum_{r=0}^{1}\left\{[\widehat{\ell}_{j0}(1,X_{i})]_{r}-[\ell_{j0}(1,X_{i})]_{r}\right\}^{2}\right]^{1/2}\le n^{-1/4}\delta_{n},\qquad j=1,2, E [ r = 0 ∑ 1 { [ ℓ j 0 ( 1 , X i ) ] r − [ ℓ j 0 ( 1 , X i ) ] r } 2 ] 1/2 ≤ n − 1/4 δ n , j = 1 , 2 , E [ ∣ m ^ 1 ( w , X i ) − m 1 ( w , X i ) ∣ 2 ] 1 / 2 ≤ n − 1 / 4 δ n , w = 0 , 1. (26) \mathbb{E}\left[\left|\widehat{m}_{1}(w,X_{i})-m_{1}(w,X_{i})\right|^{2}\right]^{1/2}\le n^{-1/4}\delta_{n},\qquad w=0,1. \tag*{(26)} E [ ∣ m 1 ( w , X i ) − m 1 ( w , X i ) ∣ 2 ] 1/2 ≤ n − 1/4 δ n , w = 0 , 1. (26) Here, r = 0 , 1 r=0,1 r = 0 , 1 indexes the two entries of ℓ j 0 \ell_{j0} ℓ j 0 .
(d) (Nondegenerate Variance ) The influence function in Equation (15) satisfies
0 < E [ ϕ i 2 ] < ∞ . 0 < \mathbb{E}[\phi_{i}^{2}] < \infty. 0 < E [ ϕ i 2 ] < ∞.
Proof. Let n k = ∣ I k ∣ n_{k}=|\mathcal{I}_{k}| n k = ∣ I k ∣ . For each fold k k k , we condition on the observations outside I k \mathcal{I}_{k} I k and on the random partition. Throughout the proof, c c c denotes a finite positive constant that may change from line to line. Also, we use the following fact in STEP2. For any nonnegative measurable function g g g , iterated expectations and Assumption 7(b) imply
E [ g ( X i ) ∣ T i = 1 ] = E [ π 1 ( X i ) g ( X i ) ] Pr ( T i = 1 ) ≤ 1 c 2 E [ g ( X i ) ] , \mathbb{E}[g(X_{i})\mid T_{i}=1]=\frac{\mathbb{E}[\pi_{1}(X_{i})g(X_{i})]}{\Pr(T_{i}=1)}\le\frac{1}{c_{2}}\mathbb{E}[g(X_{i})], E [ g ( X i ) ∣ T i = 1 ] = Pr ( T i = 1 ) E [ π 1 ( X i ) g ( X i )] ≤ c 2 1 E [ g ( X i )] ,
since Pr ( T i = 1 ) = E [ π 1 ( X i ) ] ≥ c 2 \Pr(T_{i}=1)=\mathbb{E}[\pi_{1}(X_{i})]\ge c_{2} Pr ( T i = 1 ) = E [ π 1 ( X i )] ≥ c 2 . We use this bound when transferring the marginal nuisance-rate conditions to the treated arm.
STEP1: Linearizing numerator We first consider the marginal outcome means. Subtracting the true uncentered influence function from its fitted counterpart gives
H t i : = μ ^ t ( X i ) + 1 { T i = t } π t ( X i ) { Y i − μ ^ t ( X i ) } − ϕ t i = μ ^ t ( X i ) − μ t ( X i ) − 1 { T i = t } π t ( X i ) { μ ^ t ( X i ) − μ t ( X i ) } = ( 1 − 1 { T i = t } π t ( X i ) ) { μ ^ t ( X i ) − μ t ( X i ) } , t = 0 , 1. (27) \begin{aligned}
H_{ti} := \widehat{\mu}_t(X_i) + \frac{1\{T_i=t\}}{\pi_t(X_i)}\{Y_i-\widehat{\mu}_t(X_i)\}-\phi_{ti} \\
&= \widehat{\mu}_t(X_i)-\mu_t(X_i)-\frac{1\{T_i=t\}}{\pi_t(X_i)}\{\widehat{\mu}_t(X_i)-\mu_t(X_i)\} \\
&= \left(1-\frac{1\{T_i=t\}}{\pi_t(X_i)}\right)\{\widehat{\mu}_t(X_i)-\mu_t(X_i)\}, \qquad t=0,1. \tag*{(27)}
\end{aligned} H t i := μ t ( X i ) + π t ( X i ) 1 { T i = t } { Y i − μ t ( X i )} − ϕ t i = μ t ( X i ) − μ t ( X i ) − π t ( X i ) 1 { T i = t } { μ t ( X i ) − μ t ( X i )} = ( 1 − π t ( X i ) 1 { T i = t } ) { μ t ( X i ) − μ t ( X i )} , t = 0 , 1. (27)
Consequently, (16) implies
Ψ ^ t − Ψ t = 1 n ∑ k = 1 K ∑ i ∈ I k { ϕ t i + H t i ( − k ) } − Ψ t = 1 n ∑ i = 1 n ( ϕ t i − Ψ t ) + ∑ k = 1 K n k n 1 n k ∑ i ∈ I k H t i ( − k ) . (28) \widehat{\Psi}_t-\Psi_t=\frac{1}{n}\sum_{k=1}^{K}\sum_{i\in\mathcal{I}_k}\{\phi_{ti}+H_{ti}^{(-k)}\}-\Psi_t=\frac{1}{n}\sum_{i=1}^{n}(\phi_{ti}-\Psi_t)+\sum_{k=1}^{K}\frac{n_k}{n}\frac{1}{n_k}\sum_{i\in\mathcal{I}_k}H_{ti}^{(-k)}. \tag*{(28)} Ψ t − Ψ t = n 1 k = 1 ∑ K i ∈ I k ∑ { ϕ t i + H t i ( − k ) } − Ψ t = n 1 i = 1 ∑ n ( ϕ t i − Ψ t ) + k = 1 ∑ K n n k n k 1 i ∈ I k ∑ H t i ( − k ) . (28)
For a fixed fold, iterated expectations give
E [ H t i ] = E [ E [ ( 1 − 1 { T i = t } π t ( X i ) ) { μ ^ t ( X i ) − μ t ( X i ) } | X i ] ] = E [ { μ ^ t ( X i ) − μ t ( X i ) } { 1 − Pr ( T i = t ∣ X i ) π t ( X i ) } ] = E [ { μ ^ t ( X i ) − μ t ( X i ) } ( 1 − π t ( X i ) π t ( X i ) ) ] = 0. \begin{aligned}
\mathbb{E}[H_{ti}]=\mathbb{E}\left[\mathbb{E}\left[\left(1-\frac{1\{T_i=t\}}{\pi_t(X_i)}\right)\{\widehat{\mu}_t(X_i)-\mu_t(X_i)\}\,\middle|\,X_i\right]\right] \\
&=\mathbb{E}\left[\{\widehat{\mu}_t(X_i)-\mu_t(X_i)\}\left\{1-\frac{\Pr(T_i=t\mid X_i)}{\pi_t(X_i)}\right\}\right] \\
&=\mathbb{E}\left[\{\widehat{\mu}_t(X_i)-\mu_t(X_i)\}\left(1-\frac{\pi_t(X_i)}{\pi_t(X_i)}\right)\right] \\
&=0.
\end{aligned} E [ H t i ] = E [ E [ ( 1 − π t ( X i ) 1 { T i = t } ) { μ t ( X i ) − μ t ( X i )} X i ] ] = E [ { μ t ( X i ) − μ t ( X i )} { 1 − π t ( X i ) Pr ( T i = t ∣ X i ) } ] = E [ { μ t ( X i ) − μ t ( X i )} ( 1 − π t ( X i ) π t ( X i ) ) ] = 0.
The third equality follows from randomization. The same conditioning gives the second-moment bound
E [ H t i 2 ] = E [ ∣ μ ^ t ( X i ) − μ t ( X i ) ∣ 2 E [ ( 1 − 1 { T i = t } π t ( X i ) ) 2 | X i ] ] = E [ ∣ μ ^ t ( X i ) − μ t ( X i ) ∣ 2 { π t ( X i ) ( 1 − π t ( X i ) − 1 ) 2 + ( 1 − π t ( X i ) ) } ] = E [ 1 − π t ( X i ) π t ( X i ) ∣ μ ^ t ( X i ) − μ t ( X i ) ∣ 2 ] ≤ 1 c 2 E [ ∣ μ ^ t ( X i ) − μ t ( X i ) ∣ 2 ] ≤ 1 c 2 δ n 2 . \begin{aligned}
\mathbb{E}[H_{ti}^{2}]=\mathbb{E}\left[|\widehat{\mu}_t(X_i)-\mu_t(X_i)|^2\mathbb{E}\left[\left(1-\frac{1\{T_i=t\}}{\pi_t(X_i)}\right)^2\,\middle|\,X_i\right]\right] \\
&=\mathbb{E}\left[|\widehat{\mu}_t(X_i)-\mu_t(X_i)|^2\{\pi_t(X_i)(1-\pi_t(X_i)^{-1})^2+(1-\pi_t(X_i))\}\right] \\
&=\mathbb{E}\left[\frac{1-\pi_t(X_i)}{\pi_t(X_i)}|\widehat{\mu}_t(X_i)-\mu_t(X_i)|^2\right] \\
&\leq\frac{1}{c_2}\mathbb{E}[|\widehat{\mu}_t(X_i)-\mu_t(X_i)|^2]\leq\frac{1}{c_2}\delta_n^2.
\end{aligned} E [ H t i 2 ] = E [ ∣ μ t ( X i ) − μ t ( X i ) ∣ 2 E [ ( 1 − π t ( X i ) 1 { T i = t } ) 2 X i ] ] = E [ ∣ μ t ( X i ) − μ t ( X i ) ∣ 2 { π t ( X i ) ( 1 − π t ( X i ) − 1 ) 2 + ( 1 − π t ( X i ))} ] = E [ π t ( X i ) 1 − π t ( X i ) ∣ μ t ( X i ) − μ t ( X i ) ∣ 2 ] ≤ c 2 1 E [ ∣ μ t ( X i ) − μ t ( X i ) ∣ 2 ] ≤ c 2 1 δ n 2 .
The inequalities use Assumption 7(b)–(c). Conditional independence of the evaluation observations now implies
E [ { n n k ∑ i ∈ I k H t i } 2 ] = n n k 2 ∑ i ∈ I k E [ H t i 2 ] = n n k E [ H t i 2 ] ≤ n n k c 2 δ n 2 . \mathbb{E}\left[\left\{\frac{\sqrt{n}}{n_k}\sum_{i\in\mathcal{I}_k}H_{ti}\right\}^2\right]=\frac{n}{n_k^2}\sum_{i\in\mathcal{I}_k}\mathbb{E}[H_{ti}^2]=\frac{n}{n_k}\mathbb{E}[H_{ti}^2]\leq\frac{n}{n_kc_2}\delta_n^2. E { n k n i ∈ I k ∑ H t i } 2 = n k 2 n i ∈ I k ∑ E [ H t i 2 ] = n k n E [ H t i 2 ] ≤ n k c 2 n δ n 2 .
Here, the cross terms vanish because each H t i H_{ti} H t i has conditional mean zero. Thus, for every ε > 0 \varepsilon>0 ε > 0 , conditional Chebyshev’s inequality yields
Pr ( ∣ n n k ∑ i ∈ I k H t i ∣ > ε | training observations and partition ) ≤ n n k ε 2 E [ H t i 2 ] ≤ n n k ε 2 c 2 δ n 2 . \Pr\left(\left|\frac{\sqrt{n}}{n_k}\sum_{i\in\mathcal{I}_k}H_{ti}\right|>\varepsilon\;\middle|\;\text{training observations and partition}\right)\leq\frac{n}{n_k\varepsilon^2}\mathbb{E}[H_{ti}^2]\leq\frac{n}{n_k\varepsilon^2c_2}\delta_n^2. Pr ( n k n i ∈ I k ∑ H t i > ε training observations and partition ) ≤ n k ε 2 n E [ H t i 2 ] ≤ n k ε 2 c 2 n δ n 2 .
The right-hand side tends to zero because n / n k n/n_k n / n k is bounded and δ n → 0 \delta_n \to0 δ n → 0 . The rate event also has probability tending to one, so the fold average is o p ( n − 1 / 2 ) o_p(n^{-1/2}) o p ( n − 1/2 ) unconditionally. Since K K K is fixed, its weighted average over folds is o p ( n − 1 / 2 ) o_p(n^{-1/2}) o p ( n − 1/2 ) . Therefore,
Ψ ^ t − Ψ t = 1 n ∑ i = 1 n ( ϕ t i − Ψ t ) + o p ( n − 1 / 2 ) , t = 0 , 1. (29) \widehat{\Psi}_t-\Psi_t=\frac{1}{n}\sum_{i=1}^{n}(\phi_{ti}-\Psi_t)+o_p(n^{-1/2}),\qquad t=0,1. \tag*{(29)} Ψ t − Ψ t = n 1 i = 1 ∑ n ( ϕ t i − Ψ t ) + o p ( n − 1/2 ) , t = 0 , 1. (29)
STEP2: Linearizing denominator Next, we consider the engagement probability. For a fixed fold, define the difference between the fitted and true uncentered influence functions by
H i Q : = q ^ ( X i ) + T i π 1 ( X i ) { 1 − S ^ 0 i + C ^ i − q ^ ( X i ) } − ϕ i Q = q ^ ( X i ) − q ( X i ) + T i π 1 ( X i ) { − S ^ 0 i + S 0 i + C ^ i − C i − q ^ ( X i ) + q ( X i ) } = ( 1 − T i π 1 ( X i ) ) { q ^ ( X i ) − q ( X i ) } + T i π 1 ( X i ) { − S ^ 0 i + S 0 i + C ^ i − C i } . \begin{aligned}
H_i^Q:=\widehat{q}(X_i)+\frac{T_i}{\pi_1(X_i)}\{1-\widehat{S}_{0i}+\widehat{C}_i-\widehat{q}(X_i)\}-\phi_i^Q \\
&=\widehat{q}(X_i)-q(X_i)+\frac{T_i}{\pi_1(X_i)}\{-\widehat{S}_{0i}+S_{0i}+\widehat{C}_i-C_i-\widehat{q}(X_i)+q(X_i)\} \\
&=\left(1-\frac{T_i}{\pi_1(X_i)}\right)\{\widehat{q}(X_i)-q(X_i)\}+\frac{T_i}{\pi_1(X_i)}\{-\widehat{S}_{0i}+S_{0i}+\widehat{C}_i-C_i\}.
\end{aligned} H i Q := q ( X i ) + π 1 ( X i ) T i { 1 − S 0 i + C i − q ( X i )} − ϕ i Q = q ( X i ) − q ( X i ) + π 1 ( X i ) T i { − S 0 i + S 0 i + C i − C i − q ( X i ) + q ( X i )} = ( 1 − π 1 ( X i ) T i ) { q ( X i ) − q ( X i )} + π 1 ( X i ) T i { − S 0 i + S 0 i + C i − C i } .
Equation (17) then gives
Q ^ − Q = 1 n ∑ k = 1 K ∑ i ∈ I k { ϕ i Q + H i Q , ( − k ) } − Q = 1 n ∑ i = 1 n ( ϕ i Q − Q ) + ∑ k = 1 K n k n 1 n k ∑ i ∈ I k H i Q , ( − k ) = 1 n ∑ i = 1 n ( ϕ i Q − Q ) + ∑ k = 1 K n k n { 1 n k ∑ i ∈ I k H i Q , ( − k ) − E [ H i Q , ( − k ) ] } + ∑ k = 1 K n k n E [ H i Q , ( − k ) ] . (30) \begin{aligned}
\widehat{Q}-Q=\frac{1}{n}\sum_{k=1}^{K}\sum_{i\in\mathcal{I}_k}\{\phi_i^Q+H_i^{Q,(-k)}\}-Q \\
&=\frac{1}{n}\sum_{i=1}^{n}(\phi_i^Q-Q)+\sum_{k=1}^{K}\frac{n_k}{n}\frac{1}{n_k}\sum_{i\in\mathcal{I}_k}H_i^{Q,(-k)} \\
&=\frac{1}{n}\sum_{i=1}^{n}(\phi_i^Q-Q)+\sum_{k=1}^{K}\frac{n_k}{n}\left\{\frac{1}{n_k}\sum_{i\in\mathcal{I}_k}H_i^{Q,(-k)}-\mathbb{E}[H_i^{Q,(-k)}]\right\}+\sum_{k=1}^{K}\frac{n_k}{n}\mathbb{E}[H_i^{Q,(-k)}]. \tag*{(30)}
\end{aligned} Q − Q = n 1 k = 1 ∑ K i ∈ I k ∑ { ϕ i Q + H i Q , ( − k ) } − Q = n 1 i = 1 ∑ n ( ϕ i Q − Q ) + k = 1 ∑ K n n k n k 1 i ∈ I k ∑ H i Q , ( − k ) = n 1 i = 1 ∑ n ( ϕ i Q − Q ) + k = 1 ∑ K n n k { n k 1 i ∈ I k ∑ H i Q , ( − k ) − E [ H i Q , ( − k ) ] } + k = 1 ∑ K n n k E [ H i Q , ( − k ) ] . (30)
We bound the centered average and the population expectation separately.
STEP2-1: Bounding surrogate engagement S 0 i S_{0i} S 0 i To bound the centered average, we first control the error in the measurement correction. Bounded checks imply ∣ m 1 ( w , x ) ∣ ≤ c 1 |m_1(w,x)|\le c_1 ∣ m 1 ( w , x ) ∣ ≤ c 1 and ∣ [ G j ( 1 , x ) ] 2 , w + 1 ∣ ≤ c 1 |[G_j(1,x)]_{2,w+1}|\le c_1 ∣ [ G j ( 1 , x ) ] 2 , w + 1 ∣ ≤ c 1 for w = 0 , 1 w=0,1 w = 0 , 1 . Thus, the formula for S j i S_{ji} S j i gives
∣ S j i ∣ ≤ ∣ [ G j ( 1 , X i ) ] 2 , 2 ∣ + ∣ W ~ j i ∣ ∣ [ G j ( 1 , X i ) ] 2 , 2 − [ G j ( 1 , X i ) ] 2 , 1 ∣ ≤ 2 c 1 c 2 , j = 1 , 2. |S_{ji}|\le\frac{|[G_j(1,X_i)]_{2,2}|+|\widetilde{W}_{ji}|}{|[G_j(1,X_i)]_{2,2}-[G_j(1,X_i)]_{2,1}|}\le\frac{2c_1}{c_2},\qquad j=1,2. ∣ S j i ∣ ≤ ∣ [ G j ( 1 , X i ) ] 2 , 2 − [ G j ( 1 , X i ) ] 2 , 1 ∣ ∣ [ G j ( 1 , X i ) ] 2 , 2 ∣ + ∣ W j i ∣ ≤ c 2 2 c 1 , j = 1 , 2.
The fitted functions satisfy the same boundedness and relevance conditions, so ∣ S ^ j i ∣ ≤ 2 c 1 / c 2 |\widehat{S}_{ji}|\le2c_1/c_2 ∣ S j i ∣ ≤ 2 c 1 / c 2 as well. By Equation (22) ,
S ^ j i − S j i = { ℓ ^ j 0 ( 1 , X i ) − ℓ j 0 ( 1 , X i ) } ⊤ V j i = ∑ r = 0 1 { [ ℓ ^ j 0 ( 1 , X i ) ] r − [ ℓ j 0 ( 1 , X i ) ] r } [ V j i ] r . \widehat{S}_{ji}-S_{ji}=\{\widehat{\ell}_{j0}(1,X_i)-\ell_{j0}(1,X_i)\}^{\top}V_{ji}=\sum_{r=0}^{1}\{[\widehat{\ell}_{j0}(1,X_i)]_r-[\ell_{j0}(1,X_i)]_r\}[V_{ji}]_r. S j i − S j i = { ℓ j 0 ( 1 , X i ) − ℓ j 0 ( 1 , X i ) } ⊤ V j i = r = 0 ∑ 1 {[ ℓ j 0 ( 1 , X i ) ] r − [ ℓ j 0 ( 1 , X i ) ] r } [ V j i ] r .
Thus, we have
∣ S ^ j i − S j i ∣ 2 ≤ ( 1 + ∣ W ~ j i ∣ 2 ) ∑ r = 0 1 ∣ [ ℓ ^ j 0 ( 1 , X i ) ] r − [ ℓ j 0 ( 1 , X i ) ] r ∣ 2 ≤ ( 1 + c 1 2 ) ∑ r = 0 1 ∣ [ ℓ ^ j 0 ( 1 , X i ) ] r − [ ℓ j 0 ( 1 , X i ) ] r ∣ 2 . |\widehat{S}_{ji}-S_{ji}|^2\le(1+|\widetilde{W}_{ji}|^2)\sum_{r=0}^{1}|[\widehat{\ell}_{j0}(1,X_i)]_r-[\ell_{j0}(1,X_i)]_r|^2\le(1+c_1^2)\sum_{r=0}^{1}|[\widehat{\ell}_{j0}(1,X_i)]_r-[\ell_{j0}(1,X_i)]_r|^2. ∣ S j i − S j i ∣ 2 ≤ ( 1 + ∣ W j i ∣ 2 ) r = 0 ∑ 1 ∣ [ ℓ j 0 ( 1 , X i ) ] r − [ ℓ j 0 ( 1 , X i ) ] r ∣ 2 ≤ ( 1 + c 1 2 ) r = 0 ∑ 1 ∣ [ ℓ j 0 ( 1 , X i ) ] r − [ ℓ j 0 ( 1 , X i ) ] r ∣ 2 .
where the first inequality follows from the Cauchy–Schwarz inequality and the second from Assumption 7(a). Therefore, by taking conditional expectations, we obtain
E [ ∣ S ^ j i − S j i ∣ 2 ∣ T i = 1 ] 1 / 2 ≤ 1 + c 2 2 E [ ∑ r = 0 1 ∣ [ ℓ ^ j 0 ( 1 , X i ) ] r − [ ℓ j 0 ( 1 , X i ) ] r ∣ 2 ∣ T i = 1 ] 1 / 2 ≤ 1 + c 1 2 c 2 E [ ∑ r = 0 1 ∣ [ ℓ ^ j 0 ( 1 , X i ) ] r − [ ℓ j 0 ( 1 , X i ) ] r ∣ 2 ] 1 / 2 ≤ 1 + c 1 2 c 2 n − 1 / 4 δ n , j = 1 , 2. (31) \begin{aligned}
\mathbb{E}\left[\left|\widehat{S}_{ji}-S_{ji}\right|^2\mid T_i=1\right]^{1/2}
\leq\sqrt{1+c_2^2\mathbb{E}\left[\left.\sum_{r=0}^{1}\left|[\widehat{\ell}_{j0}(1,X_i)]_r-[\ell_{j0}(1,X_i)]_r\right|^2\right|T_i=1\right]^{1/2}} \\
&\leq\sqrt{\frac{1+c_1^2}{c_2}\mathbb{E}\left[\sum_{r=0}^{1}\left|[\widehat{\ell}_{j0}(1,X_i)]_r-[\ell_{j0}(1,X_i)]_r\right|^2\right]}^{1/2} \\
&\leq\sqrt{\frac{1+c_1^2}{c_2}}n^{-1/4}\delta_n,\qquad j=1,2. \tag*{(31)}
\end{aligned} E [ S j i − S j i 2 ∣ T i = 1 ] 1/2 ≤ 1 + c 2 2 E [ r = 0 ∑ 1 [ ℓ j 0 ( 1 , X i ) ] r − [ ℓ j 0 ( 1 , X i ) ] r 2 T i = 1 ] 1/2 ≤ c 2 1 + c 1 2 E [ r = 0 ∑ 1 [ ℓ j 0 ( 1 , X i ) ] r − [ ℓ j 0 ( 1 , X i ) ] r 2 ] 1/2 ≤ c 2 1 + c 1 2 n − 1/4 δ n , j = 1 , 2. (31)
where the second inequality uses the covariate-moment bound at the beginning of the proof (by taking g ( X i ) = ∑ r = 0 1 ∣ [ ℓ ^ j 0 ( 1 , X i ) ] r − [ ℓ j 0 ( 1 , X i ) ] r ∣ 2 g(X_i)=\sum_{r=0}^{1}|[\widehat{\ell}_{j0}(1,X_i)]_r-[\ell_{j0}(1,X_i)]_r|^2 g ( X i ) = ∑ r = 0 1 ∣ [ ℓ j 0 ( 1 , X i ) ] r − [ ℓ j 0 ( 1 , X i ) ] r ∣ 2 ), and the last inequality uses Assumption 7(c).
For the product surrogate, adding and subtracting S 1 i S ^ 2 i S_{1i}\widehat{S}_{2i} S 1 i S 2 i gives
S ^ 0 i − S 0 i = S ^ 1 i S ^ 2 i − S 1 i S 2 i = S ^ 1 i S ^ 2 i − S 1 i S ^ 2 i + S 1 i S ^ 2 i − S 1 i S 2 i = S ^ 2 i ( S ^ 1 i − S 1 i ) + S 1 i ( S ^ 2 i − S 2 i ) . \widehat{S}_{0i}-S_{0i}=\widehat{S}_{1i}\widehat{S}_{2i}-S_{1i}S_{2i}=\widehat{S}_{1i}\widehat{S}_{2i}-S_{1i}\widehat{S}_{2i}+S_{1i}\widehat{S}_{2i}-S_{1i}S_{2i}=\widehat{S}_{2i}(\widehat{S}_{1i}-S_{1i})+S_{1i}(\widehat{S}_{2i}-S_{2i}). S 0 i − S 0 i = S 1 i S 2 i − S 1 i S 2 i = S 1 i S 2 i − S 1 i S 2 i + S 1 i S 2 i − S 1 i S 2 i = S 2 i ( S 1 i − S 1 i ) + S 1 i ( S 2 i − S 2 i ) .
The triangle inequality for the square root of the second moment therefore implies
E [ ∣ S ^ 0 i − S 0 i ∣ 2 ∣ T i = 1 ] 1 / 2 ≤ E [ ∣ S ^ 2 i ( S ^ 1 i − S 1 i ) ∣ 2 ∣ T i = 1 ] 1 / 2 + E [ ∣ S 1 i ( S ^ 2 i − S 2 i ) ∣ 2 ∣ T i = 1 ] 1 / 2 ≤ 2 c 1 c 2 ∑ j = 1 2 E [ ∣ S ^ j i − S j i ∣ 2 ∣ T i = 1 ] 1 / 2 ≤ c n − 1 / 4 δ n . (32) \begin{aligned}
\mathbb{E}\left[\left|\widehat{S}_{0i}-S_{0i}\right|^2\mid T_i=1\right]^{1/2}
\leq\mathbb{E}\left[\left|\widehat{S}_{2i}(\widehat{S}_{1i}-S_{1i})\right|^2\mid T_i=1\right]^{1/2}
+\mathbb{E}\left[\left|S_{1i}(\widehat{S}_{2i}-S_{2i})\right|^2\mid T_i=1\right]^{1/2} \\
&\leq\frac{2c_1}{c_2}\sum_{j=1}^{2}\mathbb{E}\left[\left|\widehat{S}_{ji}-S_{ji}\right|^2\mid T_i=1\right]^{1/2} \\
&\leq cn^{-1/4}\delta_n. \tag*{(32)}
\end{aligned} E [ S 0 i − S 0 i 2 ∣ T i = 1 ] 1/2 ≤ E [ S 2 i ( S 1 i − S 1 i ) 2 ∣ T i = 1 ] 1/2 + E [ S 1 i ( S 2 i − S 2 i ) 2 ∣ T i = 1 ] 1/2 ≤ c 2 2 c 1 j = 1 ∑ 2 E [ S j i − S j i 2 ∣ T i = 1 ] 1/2 ≤ c n − 1/4 δ n . (32)
Here, the second inequality uses the uniform bounds on the multiplying factors, and the third uses (31) .
STEP2-2: Bounding correction term C i C_i C i For the correction term, write
A i : = S 1 i + S 2 i − 2 S 0 i , A ^ i : = S ^ 1 i + S ^ 2 i − 2 S ^ 0 i , A_i:=S_{1i}+S_{2i}-2S_{0i},\qquad\widehat{A}_i:=\widehat{S}_{1i}+\widehat{S}_{2i}-2\widehat{S}_{0i}, A i := S 1 i + S 2 i − 2 S 0 i , A i := S 1 i + S 2 i − 2 S 0 i ,
B i : = W ~ 3 i − m 1 ( 1 , X i ) m 1 ( 1 , X i ) − m 1 ( 0 , X i ) , B ^ i : = W ~ 3 i − m ^ 1 ( 1 , X i ) m ^ 1 ( 1 , X i ) − m ^ 1 ( 0 , X i ) . B_i:=\frac{\widetilde{W}_{3i}-m_1(1,X_i)}{m_1(1,X_i)-m_1(0,X_i)},\qquad\widehat{B}_i:=\frac{\widetilde{W}_{3i}-\widehat{m}_1(1,X_i)}{\widehat{m}_1(1,X_i)-\widehat{m}_1(0,X_i)}. B i := m 1 ( 1 , X i ) − m 1 ( 0 , X i ) W 3 i − m 1 ( 1 , X i ) , B i := m 1 ( 1 , X i ) − m 1 ( 0 , X i ) W 3 i − m 1 ( 1 , X i ) .
Then C i = A i B i C_i=A_iB_i C i = A i B i and C ^ i = A ^ i B ^ i \widehat{C}_i=\widehat{A}_i\widehat{B}_i C i = A i B i . By the triangle inequality, the bounds already established imply
∣ A i ∣ ≤ ∣ S 1 i ∣ + ∣ S 2 i ∣ + 2 ∣ S 1 i S 2 i ∣ ≤ 4 c 1 c 2 + 8 c 1 2 c 2 2 , |A_i|\leq|S_{1i}|+|S_{2i}|+2|S_{1i}S_{2i}|\leq\frac{4c_1}{c_2}+\frac{8c_1^2}{c_2^2}, ∣ A i ∣ ≤ ∣ S 1 i ∣ + ∣ S 2 i ∣ + 2∣ S 1 i S 2 i ∣ ≤ c 2 4 c 1 + c 2 2 8 c 1 2 ,
∣ B i ∣ ≤ ∣ W ~ 3 i ∣ + ∣ m 1 ( 1 , X i ) ∣ ∣ m 1 ( 1 , X i ) − m 1 ( 0 , X i ) ∣ ≤ 2 c 1 c 2 . |B_i|\leq\frac{|\widetilde{W}_{3i}|+|m_1(1,X_i)|}{|m_1(1,X_i)-m_1(0,X_i)|}\leq\frac{2c_1}{c_2}. ∣ B i ∣ ≤ ∣ m 1 ( 1 , X i ) − m 1 ( 0 , X i ) ∣ ∣ W 3 i ∣ + ∣ m 1 ( 1 , X i ) ∣ ≤ c 2 2 c 1 .
The same bounds hold for A ^ i \widehat{A}_i A i and B ^ i \widehat{B}_i B i . Also,
A ^ i − A i = ( S ^ 1 i − S 1 i ) + ( S ^ 2 i − S 2 i ) − 2 ( S ^ 0 i − S 0 i ) . \widehat{A}_i-A_i=(\widehat{S}_{1i}-S_{1i})+(\widehat{S}_{2i}-S_{2i})-2(\widehat{S}_{0i}-S_{0i}). A i − A i = ( S 1 i − S 1 i ) + ( S 2 i − S 2 i ) − 2 ( S 0 i − S 0 i ) .
Applying the triangle inequality gives
E [ ∣ A ^ i − A i ∣ 2 ∣ T i = 1 ] 1 / 2 ≤ ∑ j = 1 2 E [ ∣ S ^ j i − S j i ∣ 2 ∣ T i = 1 ] 1 / 2 + 2 E [ ∣ S ^ 0 i − S 0 i ∣ 2 ∣ T i = 1 ] 1 / 2 \mathbb{E}\left[|\widehat{A}_i-A_i|^2\mid T_i=1\right]^{1/2}
\leq\sum_{j=1}^{2}\mathbb{E}\left[|\widehat{S}_{ji}-S_{ji}|^2\mid T_i=1\right]^{1/2}
+2\mathbb{E}\left[|\widehat{S}_{0i}-S_{0i}|^2\mid T_i=1\right]^{1/2} E [ ∣ A i − A i ∣ 2 ∣ T i = 1 ] 1/2 ≤ j = 1 ∑ 2 E [ ∣ S j i − S j i ∣ 2 ∣ T i = 1 ] 1/2 + 2 E [ ∣ S 0 i − S 0 i ∣ 2 ∣ T i = 1 ] 1/2
≤ 2 1 + c 1 2 c 2 n − 1 / 4 δ n + 2 E [ ∣ S ^ 0 i − S 0 i ∣ 2 ∣ T i = 1 ] 1 / 2 ≤ c n − 1 / 4 δ n . (33) \begin{aligned}
\le2\sqrt{\frac{1+c_1^2}{c_2}}n^{-1/4}\delta_n+2\mathbb{E}[|\widehat{S}_{0i}-S_{0i}|^2\mid T_i=1]^{1/2} \\
&\le cn^{-1/4}\delta_n. \tag*{(33)}
\end{aligned} ≤ 2 c 2 1 + c 1 2 n − 1/4 δ n + 2 E [ ∣ S 0 i − S 0 i ∣ 2 ∣ T i = 1 ] 1/2 ≤ c n − 1/4 δ n . (33)
The last inequality follows by substituting the preceding surrogate bounds. To compare the third-check factors, first add and subtract the fraction with numerator W ~ 3 i − m 1 ( 1 , X i ) \widetilde{W}_{3i}-m_1(1,X_i) W 3 i − m 1 ( 1 , X i ) and the fitted denominator:
B ^ i − B i = W ~ 3 i − m ^ 1 ( 1 , X i ) m ^ 1 ( 1 , X i ) − m ^ 1 ( 0 , X i ) − W ~ 3 i − m 1 ( 1 , X i ) m 1 ( 1 , X i ) − m 1 ( 0 , X i ) = − m ^ 1 ( 1 , X i ) − m 1 ( 1 , X i ) m ^ 1 ( 1 , X i ) − m ^ 1 ( 0 , X i ) + { W ~ 3 i − m 1 ( 1 , X i ) } { 1 m ^ 1 ( 1 , X i ) − m ^ 1 ( 0 , X i ) − 1 m 1 ( 1 , X i ) − m 1 ( 0 , X i ) } = − m ^ 1 ( 1 , X i ) − m 1 ( 1 , X i ) m ^ 1 ( 1 , X i ) − m ^ 1 ( 0 , X i ) − W ~ 3 i − m 1 ( 1 , X i ) { m ^ 1 ( 1 , X i ) − m ^ 1 ( 0 , X i ) } { m 1 ( 1 , X i ) − m 1 ( 0 , X i ) } × [ { m ^ 1 ( 1 , X i ) − m 1 ( 1 , X i ) } − { m ^ 1 ( 0 , X i ) − m 1 ( 0 , X i ) } ] . \begin{aligned}
\widehat{B}_i-B_i
&=\frac{\widetilde{W}_{3i}-\widehat{m}_1(1,X_i)}{\widehat{m}_1(1,X_i)-\widehat{m}_1(0,X_i)}
-\frac{\widetilde{W}_{3i}-m_1(1,X_i)}{m_1(1,X_i)-m_1(0,X_i)}\\
&=-\frac{\widehat{m}_1(1,X_i)-m_1(1,X_i)}{\widehat{m}_1(1,X_i)-\widehat{m}_1(0,X_i)}\\
&\quad+\{\widetilde{W}_{3i}-m_1(1,X_i)\}\left\{\frac{1}{\widehat{m}_1(1,X_i)-\widehat{m}_1(0,X_i)}-\frac{1}{m_1(1,X_i)-m_1(0,X_i)}\right\}\\
&=-\frac{\widehat{m}_1(1,X_i)-m_1(1,X_i)}{\widehat{m}_1(1,X_i)-\widehat{m}_1(0,X_i)}\\
&\quad-\frac{\widetilde{W}_{3i}-m_1(1,X_i)}{\{\widehat{m}_1(1,X_i)-\widehat{m}_1(0,X_i)\}\{m_1(1,X_i)-m_1(0,X_i)\}}\\
&\qquad\times[\{\widehat{m}_1(1,X_i)-m_1(1,X_i)\}-\{\widehat{m}_1(0,X_i)-m_1(0,X_i)\}].
\end{aligned} B i − B i = m 1 ( 1 , X i ) − m 1 ( 0 , X i ) W 3 i − m 1 ( 1 , X i ) − m 1 ( 1 , X i ) − m 1 ( 0 , X i ) W 3 i − m 1 ( 1 , X i ) = − m 1 ( 1 , X i ) − m 1 ( 0 , X i ) m 1 ( 1 , X i ) − m 1 ( 1 , X i ) + { W 3 i − m 1 ( 1 , X i )} { m 1 ( 1 , X i ) − m 1 ( 0 , X i ) 1 − m 1 ( 1 , X i ) − m 1 ( 0 , X i ) 1 } = − m 1 ( 1 , X i ) − m 1 ( 0 , X i ) m 1 ( 1 , X i ) − m 1 ( 1 , X i ) − { m 1 ( 1 , X i ) − m 1 ( 0 , X i )} { m 1 ( 1 , X i ) − m 1 ( 0 , X i )} W 3 i − m 1 ( 1 , X i ) × [{ m 1 ( 1 , X i ) − m 1 ( 1 , X i )} − { m 1 ( 0 , X i ) − m 1 ( 0 , X i )}] .
Therefore, using the boundedness in Assumption 7(a)–(b), we have
∣ B ^ i − B i ∣ ≤ 1 c 2 ∣ m ^ 1 ( 1 , X i ) − m 1 ( 1 , X i ) ∣ + 2 c 1 c 2 2 ∣ { m ^ 1 ( 1 , X i ) − m 1 ( 1 , X i ) } − { m ^ 1 ( 0 , X i ) − m 1 ( 0 , X i ) } ∣ ≤ ( 1 c 2 + 2 c 1 c 2 2 ) ∣ m ^ 1 ( 1 , X i ) − m 1 ( 1 , X i ) ∣ + 2 c 1 c 2 2 ∣ m ^ 1 ( 0 , X i ) − m 1 ( 0 , X i ) ∣ . \begin{aligned}
|\widehat{B}_i-B_i|
&\le\frac{1}{c_2}|\widehat{m}_1(1,X_i)-m_1(1,X_i)|\\
&\quad+\frac{2c_1}{c_2^2}|\{\widehat{m}_1(1,X_i)-m_1(1,X_i)\}-\{\widehat{m}_1(0,X_i)-m_1(0,X_i)\}|\\
&\le\left(\frac{1}{c_2}+\frac{2c_1}{c_2^2}\right)|\widehat{m}_1(1,X_i)-m_1(1,X_i)|+\frac{2c_1}{c_2^2}|\widehat{m}_1(0,X_i)-m_1(0,X_i)|.
\end{aligned} ∣ B i − B i ∣ ≤ c 2 1 ∣ m 1 ( 1 , X i ) − m 1 ( 1 , X i ) ∣ + c 2 2 2 c 1 ∣ { m 1 ( 1 , X i ) − m 1 ( 1 , X i )} − { m 1 ( 0 , X i ) − m 1 ( 0 , X i )} ∣ ≤ ( c 2 1 + c 2 2 2 c 1 ) ∣ m 1 ( 1 , X i ) − m 1 ( 1 , X i ) ∣ + c 2 2 2 c 1 ∣ m 1 ( 0 , X i ) − m 1 ( 0 , X i ) ∣.
Taking second moments and applying the triangle inequality gives
E [ ∣ B ^ i − B i ∣ 2 ∣ T i = 1 ] 1 / 2 ≤ ( 1 c 2 + 2 c 1 c 2 2 ) E [ ∣ m ^ 1 ( 1 , X i ) − m 1 ( 1 , X i ) ∣ 2 ∣ T i = 1 ] 1 / 2 + 2 c 1 c 2 2 E [ ∣ m ^ 1 ( 0 , X i ) − m 1 ( 0 , X i ) ∣ 2 ∣ T i = 1 ] 1 / 2 ≤ 1 c 2 ( 1 c 2 + 2 c 1 c 2 2 ) E [ ∣ m ^ 1 ( 1 , X i ) − m 1 ( 1 , X i ) ∣ 2 ] 1 / 2 + 2 c 1 c 2 5 / 2 E [ ∣ m ^ 1 ( 0 , X i ) − m 1 ( 0 , X i ) ∣ 2 ] 1 / 2 ≤ c n − 1 / 4 δ n . (34) \begin{aligned}
\mathbb{E}[|\widehat{B}_i-B_i|^2\mid T_i=1]^{1/2}
&\le\left(\frac{1}{c_2}+\frac{2c_1}{c_2^2}\right)\mathbb{E}[|\widehat{m}_1(1,X_i)-m_1(1,X_i)|^2\mid T_i=1]^{1/2}\\
&\quad+\frac{2c_1}{c_2^2}\mathbb{E}[|\widehat{m}_1(0,X_i)-m_1(0,X_i)|^2\mid T_i=1]^{1/2}\\
&\le\frac{1}{\sqrt{c_2}}\left(\frac{1}{c_2}+\frac{2c_1}{c_2^2}\right)\mathbb{E}[|\widehat{m}_1(1,X_i)-m_1(1,X_i)|^2]^{1/2}\\
&\quad+\frac{2c_1}{c_2^{5/2}}\mathbb{E}[|\widehat{m}_1(0,X_i)-m_1(0,X_i)|^2]^{1/2}\\
&\le cn^{-1/4}\delta_n. \tag*{(34)}
\end{aligned} E [ ∣ B i − B i ∣ 2 ∣ T i = 1 ] 1/2 ≤ ( c 2 1 + c 2 2 2 c 1 ) E [ ∣ m 1 ( 1 , X i ) − m 1 ( 1 , X i ) ∣ 2 ∣ T i = 1 ] 1/2 + c 2 2 2 c 1 E [ ∣ m 1 ( 0 , X i ) − m 1 ( 0 , X i ) ∣ 2 ∣ T i = 1 ] 1/2 ≤ c 2 1 ( c 2 1 + c 2 2 2 c 1 ) E [ ∣ m 1 ( 1 , X i ) − m 1 ( 1 , X i ) ∣ 2 ] 1/2 + c 2 5/2 2 c 1 E [ ∣ m 1 ( 0 , X i ) − m 1 ( 0 , X i ) ∣ 2 ] 1/2 ≤ c n − 1/4 δ n . (34)
where the second inequality uses the covariate-moment bound at the beginning of the proof, and the last inequality uses the rates for the two third-check means.
Adding and subtracting A i B ^ i A_i\widehat{B}_i A i B i gives
C ^ i − C i = A ^ i B ^ i − A i B i = A ^ i B ^ i − A i B ^ i + A i B ^ i − A i B i = B ^ i ( A ^ i − A i ) + A i ( B ^ i − B i ) . \widehat{C}_i-C_i=\widehat{A}_i\widehat{B}_i-A_iB_i=\widehat{A}_i\widehat{B}_i-A_i\widehat{B}_i+A_i\widehat{B}_i-A_iB_i=\widehat{B}_i(\widehat{A}_i-A_i)+A_i(\widehat{B}_i-B_i). C i − C i = A i B i − A i B i = A i B i − A i B i + A i B i − A i B i = B i ( A i − A i ) + A i ( B i − B i ) .
We therefore obtain
E [ ∣ C ^ i − C i ∣ 2 ∣ T i = 1 ] 1 / 2 ≤ E [ ∣ B ^ i ( A ^ i − A i ) ∣ 2 ∣ T i = 1 ] 1 / 2 + E [ ∣ A i ( B ^ i − B i ) ∣ 2 ∣ T i = 1 ] 1 / 2 \mathbb{E}[|\widehat{C}_i-C_i|^2\mid T_i=1]^{1/2}
\le\mathbb{E}[|\widehat{B}_i(\widehat{A}_i-A_i)|^2\mid T_i=1]^{1/2}+\mathbb{E}[|A_i(\widehat{B}_i-B_i)|^2\mid T_i=1]^{1/2} E [ ∣ C i − C i ∣ 2 ∣ T i = 1 ] 1/2 ≤ E [ ∣ B i ( A i − A i ) ∣ 2 ∣ T i = 1 ] 1/2 + E [ ∣ A i ( B i − B i ) ∣ 2 ∣ T i = 1 ] 1/2
≤ 2 c 1 c 2 E [ ∣ A ^ i − A i ∣ 2 ∣ T i = 1 ] 1 / 2 + ( 4 c 1 c 2 + 8 c 1 2 c 2 2 ) E [ ∣ B ^ i − B i ∣ 2 ∣ T i = 1 ] 1 / 2 ≤ c n − 1 / 4 δ n + ( 4 c 1 c 2 + 8 c 1 2 c 2 2 ) E [ ∣ B ^ i − B i ∣ 2 ∣ T i = 1 ] 1 / 2 ≤ c n − 1 / 4 δ n . (35) \begin{aligned}
&\le\frac{2c_1}{c_2}\mathbb{E}[|\widehat{A}_i-A_i|^2\mid T_i=1]^{1/2}+\left(\frac{4c_1}{c_2}+\frac{8c_1^2}{c_2^2}\right)\mathbb{E}[|\widehat{B}_i-B_i|^2\mid T_i=1]^{1/2}\\
&\le cn^{-1/4}\delta_n+\left(\frac{4c_1}{c_2}+\frac{8c_1^2}{c_2^2}\right)\mathbb{E}[|\widehat{B}_i-B_i|^2\mid T_i=1]^{1/2}\\
&\le cn^{-1/4}\delta_n. \tag*{(35)}
\end{aligned} ≤ c 2 2 c 1 E [ ∣ A i − A i ∣ 2 ∣ T i = 1 ] 1/2 + ( c 2 4 c 1 + c 2 2 8 c 1 2 ) E [ ∣ B i − B i ∣ 2 ∣ T i = 1 ] 1/2 ≤ c n − 1/4 δ n + ( c 2 4 c 1 + c 2 2 8 c 1 2 ) E [ ∣ B i − B i ∣ 2 ∣ T i = 1 ] 1/2 ≤ c n − 1/4 δ n . (35)
Here, the first inequality is the triangle inequality, and the second uses boundedness of the multiplying factors. The third substitutes the bound for A ^ i − A i \widehat{A}_i-A_i A i − A i , and the fourth substitutes the bound for B ^ i − B i \widehat{B}_i-B_i B i − B i . Another application of the triangle inequality yields
E [ ∣ − S ^ 0 i + S 0 i + C ^ i − C i ∣ 2 ∣ T i = 1 ] 1 / 2 ≤ E [ ∣ S ^ 0 i − S 0 i ∣ 2 ∣ T i = 1 ] 1 / 2 + E [ ∣ C ^ i − C i ∣ 2 ∣ T i = 1 ] 1 / 2 ≤ c n − 1 / 4 δ n + E [ ∣ C ^ i − C i ∣ 2 ∣ T i = 1 ] 1 / 2 ≤ c n − 1 / 4 δ n . (36) \begin{aligned}
\mathbb{E}[|-\widehat{S}_{0i}+S_{0i}+\widehat{C}_i-C_i|^2\mid T_i=1]^{1/2}
&\le\mathbb{E}[|\widehat{S}_{0i}-S_{0i}|^2\mid T_i=1]^{1/2}+\mathbb{E}[|\widehat{C}_i-C_i|^2\mid T_i=1]^{1/2}\\
&\le cn^{-1/4}\delta_n+\mathbb{E}[|\widehat{C}_i-C_i|^2\mid T_i=1]^{1/2}\\
&\le cn^{-1/4}\delta_n. \tag*{(36)}
\end{aligned} E [ ∣ − S 0 i + S 0 i + C i − C i ∣ 2 ∣ T i = 1 ] 1/2 ≤ E [ ∣ S 0 i − S 0 i ∣ 2 ∣ T i = 1 ] 1/2 + E [ ∣ C i − C i ∣ 2 ∣ T i = 1 ] 1/2 ≤ c n − 1/4 δ n + E [ ∣ C i − C i ∣ 2 ∣ T i = 1 ] 1/2 ≤ c n − 1/4 δ n . (36)
Bounding centered average of H i Q H_i^Q H i Q
We can now bound the centered average of H i Q H_i^Q H i Q in L 2 L_2 L 2 norm. Applying the triangle inequality to its two terms gives
E [ ( H i Q ) 2 ] 1 / 2 ≤ E [ [ ( 1 − T i π 1 ( X i ) ) 2 ∣ q ^ ( X i ) − q ( X i ) ∣ 2 ] ] 1 / 2 + E [ [ T i 2 π 1 ( X i ) 2 ∣ − S ^ 0 i + S 0 i + C ^ i − C i ∣ 2 ] ] 1 / 2 ≤ 1 c 2 E [ ∣ q ^ ( X i ) − q ( X i ) ∣ 2 ] 1 / 2 + 1 c 2 E [ ∣ − S ^ 0 i + S 0 i + C ^ i − C i ∣ 2 ∣ T i = 1 ] 1 / 2 ≤ 1 c 2 δ n + 1 c 2 E [ ∣ − S ^ 0 i + S 0 i + C ^ i − C i ∣ 2 ∣ T i = 1 ] 1 / 2 ≤ 1 c 2 δ n + c n − 1 / 4 δ n ≤ c δ n . (37) \begin{aligned}
\mathbb{E}[(H_i^Q)^2]^{1/2}
&\le\mathbb{E}\left[\left[\left(1-\frac{T_i}{\pi_1(X_i)}\right)^2|\widehat{q}(X_i)-q(X_i)|^2\right]\right]^{1/2}
+\mathbb{E}\left[\left[\frac{T_i^2}{\pi_1(X_i)^2}|-\widehat{S}_{0i}+S_{0i}+\widehat{C}_i-C_i|^2\right]\right]^{1/2}\\
&\le\frac{1}{\sqrt{c_2}}\mathbb{E}[|\widehat{q}(X_i)-q(X_i)|^2]^{1/2}
+\frac{1}{c_2}\mathbb{E}[|-\widehat{S}_{0i}+S_{0i}+\widehat{C}_i-C_i|^2\mid T_i=1]^{1/2}\\
&\le\frac{1}{\sqrt{c_2}}\delta_n+\frac{1}{c_2}\mathbb{E}[|-\widehat{S}_{0i}+S_{0i}+\widehat{C}_i-C_i|^2\mid T_i=1]^{1/2}\\
&\le\frac{1}{\sqrt{c_2}}\delta_n+cn^{-1/4}\delta_n\\
&\le c\delta_n. \tag*{(37)}
\end{aligned} E [( H i Q ) 2 ] 1/2 ≤ E [ [ ( 1 − π 1 ( X i ) T i ) 2 ∣ q ( X i ) − q ( X i ) ∣ 2 ] ] 1/2 + E [ [ π 1 ( X i ) 2 T i 2 ∣ − S 0 i + S 0 i + C i − C i ∣ 2 ] ] 1/2 ≤ c 2 1 E [ ∣ q ( X i ) − q ( X i ) ∣ 2 ] 1/2 + c 2 1 E [ ∣ − S 0 i + S 0 i + C i − C i ∣ 2 ∣ T i = 1 ] 1/2 ≤ c 2 1 δ n + c 2 1 E [ ∣ − S 0 i + S 0 i + C i − C i ∣ 2 ∣ T i = 1 ] 1/2 ≤ c 2 1 δ n + c n − 1/4 δ n ≤ c δ n . (37)
For the second inequality, the first coefficient follows by conditioning on X i X_i X i and using the same Bernoulli second-moment calculation as for H T i H_{Ti} H T i , together with π 1 ( X i ) ≥ c 2 \pi_1(X_i)\ge c_2 π 1 ( X i ) ≥ c 2 . For the second coefficient, we use T i 2 = T i T_i^2=T_i T i 2 = T i and
E [ T i π 1 ( X i ) 2 ∣ − S ^ 0 i + S 0 i + C ^ i − C i ∣ 2 ] ≤ Pr ( T i = 1 ) c 2 2 E [ ∣ − S ^ 0 i + S 0 i + C ^ i − C i ∣ 2 ∣ T i = 1 ] ≤ 1 c 2 2 E [ ∣ − S ^ 0 i + S 0 i + C ^ i − C i ∣ 2 ∣ T i = 1 ] . \begin{aligned}
\mathbb{E}\left[\frac{T_i}{\pi_1(X_i)^2}|-\widehat{S}_{0i}+S_{0i}+\widehat{C}_i-C_i|^2\right]
&\le\frac{\Pr(T_i=1)}{c_2^2}\mathbb{E}[|-\widehat{S}_{0i}+S_{0i}+\widehat{C}_i-C_i|^2\mid T_i=1]\\
&\le\frac{1}{c_2^2}\mathbb{E}[|-\widehat{S}_{0i}+S_{0i}+\widehat{C}_i-C_i|^2\mid T_i=1].
\end{aligned} E [ π 1 ( X i ) 2 T i ∣ − S 0 i + S 0 i + C i − C i ∣ 2 ] ≤ c 2 2 Pr ( T i = 1 ) E [ ∣ − S 0 i + S 0 i + C i − C i ∣ 2 ∣ T i = 1 ] ≤ c 2 2 1 E [ ∣ − S 0 i + S 0 i + C i − C i ∣ 2 ∣ T i = 1 ] .
Within an evaluation fold, the centered variables H i Q − E [ H i Q ] H_i^Q-\mathbb{E}[H_i^Q] H i Q − E [ H i Q ] are conditionally independent. Hence,
E [ { n n k ∑ i ∈ I k ( H i Q − E [ H i Q ] ) } 2 ] = n n k E [ ( H i Q − E [ H i Q ] ) 2 ] ≤ n n k E [ ( H i Q ) 2 ] ≤ c n n k δ n 2 . \mathbb{E}\left[\left\{\frac{\sqrt{n}}{n_k}\sum_{i\in\mathcal{I}_k}\left(H_i^Q-\mathbb{E}[H_i^Q]\right)\right\}^2\right]
=\frac{n}{n_k}\mathbb{E}[(H_i^Q-\mathbb{E}[H_i^Q])^2]
\le\frac{n}{n_k}\mathbb{E}[(H_i^Q)^2]
\le\frac{c\,n}{n_k}\delta_n^2. E { n k n i ∈ I k ∑ ( H i Q − E [ H i Q ] ) } 2 = n k n E [( H i Q − E [ H i Q ] ) 2 ] ≤ n k n E [( H i Q ) 2 ] ≤ n k c n δ n 2 .
Conditional Chebyshev’s inequality therefore bounds the probability that the absolute value of this scaled average exceeds ε \varepsilon ε by c ( n / n k ) δ n 2 / ε 2 c(n/n_k)\delta_n^2/\varepsilon^2 c ( n / n k ) δ n 2 / ε 2 on the rate event. This bound tends to zero, and the complement of the event has vanishing probability. Averaging over the fixed number of folds gives
∑ k = 1 K n k n { 1 n k ∑ i ∈ I k H i Q , ( − k ) − E [ H i Q , ( − k ) ] } = o p ( n − 1 / 2 ) . (38) \sum_{k=1}^{K}\frac{n_k}{n}\left\{\frac{1}{n_k}\sum_{i\in\mathcal{I}_k}H_i^{Q,(-k)}-\mathbb{E}[H_i^{Q,(-k)}]\right\}=o_p(n^{-1/2}). \tag*{(38)} k = 1 ∑ K n n k { n k 1 i ∈ I k ∑ H i Q , ( − k ) − E [ H i Q , ( − k ) ] } = o p ( n − 1/2 ) . (38)
STEP2-4: Bounding the conditional population remainder E [ H i Q ] \mathbb{E}[H_i^Q] E [ H i Q ] . It remains to bound the conditional population remainder E [ H i Q ] \mathbb{E}[H_i^Q] E [ H i Q ] . The contribution from the fitted engagement probability has expectation
E [ ( 1 − T i π 1 ( X i ) ) { q ^ ( X i ) − q ( X i ) } ] = E [ { q ^ ( X i ) − q ( X i ) } E [ 1 − T i π 1 ( X i ) ∣ X i ] ] = E [ { q ^ ( X i ) − q ( X i ) } { 1 − Pr ( T i = 1 ∣ X i ) π 1 ( X i ) } ] = E [ { q ^ ( X i ) − q ( X i ) } ( 1 − π 1 ( X i ) π 1 ( X i ) ) ] = 0. \begin{aligned}
\mathbb{E}\left[\left(1-\frac{T_i}{\pi_1(X_i)}\right)\{\widehat{q}(X_i)-q(X_i)\}\right]
&=\mathbb{E}\left[\{\widehat{q}(X_i)-q(X_i)\}\mathbb{E}\left[\left.1-\frac{T_i}{\pi_1(X_i)}\right|X_i\right]\right]\\
&=\mathbb{E}\left[\{\widehat{q}(X_i)-q(X_i)\}\left\{1-\frac{\Pr(T_i=1\mid X_i)}{\pi_1(X_i)}\right\}\right]\\
&=\mathbb{E}\left[\{\widehat{q}(X_i)-q(X_i)\}\left(1-\frac{\pi_1(X_i)}{\pi_1(X_i)}\right)\right]\\
&=0.
\end{aligned} E [ ( 1 − π 1 ( X i ) T i ) { q ( X i ) − q ( X i )} ] = E [ { q ( X i ) − q ( X i )} E [ 1 − π 1 ( X i ) T i X i ] ] = E [ { q ( X i ) − q ( X i )} { 1 − π 1 ( X i ) Pr ( T i = 1 ∣ X i ) } ] = E [ { q ( X i ) − q ( X i )} ( 1 − π 1 ( X i ) π 1 ( X i ) ) ] = 0.
where the third equality uses randomization. Consequently,
E [ H i Q ] = E [ ( 1 − T i π 1 ( X i ) ) { q ^ ( X i ) − q ( X i ) } + T i π 1 ( X i ) { − S ^ 0 i + S 0 i + C ^ i − C i } ] = E [ T i π 1 ( X i ) { − S ^ 0 i + S 0 i + C ^ i − C i } ] = Pr ( T i = 1 ) E [ − S ^ 0 i + S 0 i + C ^ i − C i π 1 ( X i ) ∣ T i = 1 ] . (39) \begin{aligned}
\mathbb{E}[H_i^Q]
&=\mathbb{E}\left[\left(1-\frac{T_i}{\pi_1(X_i)}\right)\{\widehat{q}(X_i)-q(X_i)\}+\frac{T_i}{\pi_1(X_i)}\{-\widehat{S}_{0i}+S_{0i}+\widehat{C}_i-C_i\}\right]\\
&=\mathbb{E}\left[\frac{T_i}{\pi_1(X_i)}\{-\widehat{S}_{0i}+S_{0i}+\widehat{C}_i-C_i\}\right]\\
&=\Pr(T_i=1)\mathbb{E}\left[\left.\frac{-\widehat{S}_{0i}+S_{0i}+\widehat{C}_i-C_i}{\pi_1(X_i)}\right|T_i=1\right]. \tag*{(39)}
\end{aligned} E [ H i Q ] = E [ ( 1 − π 1 ( X i ) T i ) { q ( X i ) − q ( X i )} + π 1 ( X i ) T i { − S 0 i + S 0 i + C i − C i } ] = E [ π 1 ( X i ) T i { − S 0 i + S 0 i + C i − C i } ] = Pr ( T i = 1 ) E [ π 1 ( X i ) − S 0 i + S 0 i + C i − C i T i = 1 ] . (39)
We calculate this expectation by first conditioning on W i = w W_i=w W i = w , T i = 1 T_i=1 T i = 1 , and X i = x X_i=x X i = x . Calibration gives
E [ S j i ∣ W i = w , T i = 1 , X i = x ] = 1 − w , j = 1 , 2. \mathbb{E}[S_{ji}\mid W_i=w,T_i=1,X_i=x]=1-w,\qquad j=1,2. E [ S j i ∣ W i = w , T i = 1 , X i = x ] = 1 − w , j = 1 , 2.
Since S j i S_{ji} S j i is a function of the j j j th check and X i X_i X i , conditional independence of the first two checks implies
E [ S 0 i ∣ W i = w , T i = 1 , X i = x ] = E [ S 1 i S 2 i ∣ W i = w , T i = 1 , X i = x ] = E [ S 1 i ∣ W i = w , T i = 1 , X i = x ] E [ S 2 i ∣ W i = w , T i = 1 , X i = x ] = ( 1 − w ) 2 = 1 − w . \begin{aligned}
\mathbb{E}[S_{0i}\mid W_i=w,T_i=1,X_i=x]
&=\mathbb{E}[S_{1i}S_{2i}\mid W_i=w,T_i=1,X_i=x]\\
&=\mathbb{E}[S_{1i}\mid W_i=w,T_i=1,X_i=x]\mathbb{E}[S_{2i}\mid W_i=w,T_i=1,X_i=x]\\
&=(1-w)^2\\
&=1-w.
\end{aligned} E [ S 0 i ∣ W i = w , T i = 1 , X i = x ] = E [ S 1 i S 2 i ∣ W i = w , T i = 1 , X i = x ] = E [ S 1 i ∣ W i = w , T i = 1 , X i = x ] E [ S 2 i ∣ W i = w , T i = 1 , X i = x ] = ( 1 − w ) 2 = 1 − w .
The third equality substitutes the calibration identities, and the last equality uses w ∈ { 0 , 1 } w\in\{0,1\} w ∈ { 0 , 1 } . It thus follows that
E [ A i ∣ W i = w , T i = 1 , X i = x ] = E [ S 1 i ∣ W i = w , T i = 1 , X i = x ] + E [ S 2 i ∣ W i = w , T i = 1 , X i = x ] − 2 E [ S 0 i ∣ W i = w , T i = 1 , X i = x ] = ( 1 − w ) + ( 1 − w ) − 2 ( 1 − w ) = 0. (40) \begin{aligned}
\mathbb{E}[A_i\mid W_i=w,T_i=1,X_i=x]
&=\mathbb{E}[S_{1i}\mid W_i=w,T_i=1,X_i=x]+\mathbb{E}[S_{2i}\mid W_i=w,T_i=1,X_i=x]\\
&\quad-2\mathbb{E}[S_{0i}\mid W_i=w,T_i=1,X_i=x]\\
&=(1-w)+(1-w)-2(1-w)\\
&=0. \tag*{(40)}
\end{aligned} E [ A i ∣ W i = w , T i = 1 , X i = x ] = E [ S 1 i ∣ W i = w , T i = 1 , X i = x ] + E [ S 2 i ∣ W i = w , T i = 1 , X i = x ] − 2 E [ S 0 i ∣ W i = w , T i = 1 , X i = x ] = ( 1 − w ) + ( 1 − w ) − 2 ( 1 − w ) = 0. (40)
The difference between the fitted and true product surrogates can be expanded as
S ^ 0 i − S 0 i = S ^ 1 i S ^ 2 i − S 1 i S 2 i = { S 1 i + ( S ^ 1 i − S 1 i ) } { S 2 i + ( S ^ 2 i − S 2 i ) } − S 1 i S 2 i = ( S ^ 1 i − S 1 i ) S 2 i + S 1 i ( S ^ 2 i − S 2 i ) + ( S ^ 1 i − S 1 i ) ( S ^ 2 i − S 2 i ) . \begin{aligned}
\widehat{S}_{0i}-S_{0i}
&=\widehat{S}_{1i}\widehat{S}_{2i}-S_{1i}S_{2i}\\
&=\{S_{1i}+(\widehat{S}_{1i}-S_{1i})\}\{S_{2i}+(\widehat{S}_{2i}-S_{2i})\}-S_{1i}S_{2i}\\
&=(\widehat{S}_{1i}-S_{1i})S_{2i}+S_{1i}(\widehat{S}_{2i}-S_{2i})+(\widehat{S}_{1i}-S_{1i})(\widehat{S}_{2i}-S_{2i}).
\end{aligned} S 0 i − S 0 i = S 1 i S 2 i − S 1 i S 2 i = { S 1 i + ( S 1 i − S 1 i )} { S 2 i + ( S 2 i − S 2 i )} − S 1 i S 2 i = ( S 1 i − S 1 i ) S 2 i + S 1 i ( S 2 i − S 2 i ) + ( S 1 i − S 1 i ) ( S 2 i − S 2 i ) .
Taking the conditional expectation first gives
E [ S ^ 0 i − S 0 i ∣ W i = w , T i = 1 , X i = x ] = E [ ( S ^ 1 i − S 1 i ) S 2 i ∣ W i = w , T i = 1 , X i = x ] + E [ S 1 i ( S ^ 2 i − S 2 i ) ∣ W i = w , T i = 1 , X i = x ] + E [ ( S ^ 1 i − S 1 i ) ( S ^ 2 i − S 2 i ) ∣ W i = w , T i = 1 , X i = x ] . \begin{aligned}
\mathbb{E}[\widehat{S}_{0i}-S_{0i}\mid W_i=w,T_i=1,X_i=x]
&=\mathbb{E}[(\widehat{S}_{1i}-S_{1i})S_{2i}\mid W_i=w,T_i=1,X_i=x]\\
&\quad+\mathbb{E}[S_{1i}(\widehat{S}_{2i}-S_{2i})\mid W_i=w,T_i=1,X_i=x]\\
&\quad+\mathbb{E}[(\widehat{S}_{1i}-S_{1i})(\widehat{S}_{2i}-S_{2i})\mid W_i=w,T_i=1,X_i=x].
\end{aligned} E [ S 0 i − S 0 i ∣ W i = w , T i = 1 , X i = x ] = E [( S 1 i − S 1 i ) S 2 i ∣ W i = w , T i = 1 , X i = x ] + E [ S 1 i ( S 2 i − S 2 i ) ∣ W i = w , T i = 1 , X i = x ] + E [( S 1 i − S 1 i ) ( S 2 i − S 2 i ) ∣ W i = w , T i = 1 , X i = x ] .
Each fitted surrogate is evaluated with its training sample held fixed, so it remains a function of its own check and X i X_i X i . Conditional independence therefore gives
E [ S ^ 0 i − S 0 i ∣ W i = w , T i = 1 , X i = x ] = E [ S ^ 1 i − S 1 i ∣ W i = w , T i = 1 , X i = x ] E [ S 2 i ∣ W i = w , T i = 1 , X i = x ] + E [ S 1 i ∣ W i = w , T i = 1 , X i = x ] E [ S ^ 2 i − S 2 i ∣ W i = w , T i = 1 , X i = x ] + E [ ( S ^ 1 i − S 1 i ) ( S ^ 2 i − S 2 i ) ∣ W i = w , T i = 1 , X i = x ] . \begin{aligned}
\mathbb{E}[\widehat{S}_{0i}-S_{0i}\mid W_i=w,T_i=1,X_i=x]
&=\mathbb{E}[\widehat{S}_{1i}-S_{1i}\mid W_i=w,T_i=1,X_i=x]\mathbb{E}[S_{2i}\mid W_i=w,T_i=1,X_i=x]\\
&\quad+\mathbb{E}[S_{1i}\mid W_i=w,T_i=1,X_i=x]\mathbb{E}[\widehat{S}_{2i}-S_{2i}\mid W_i=w,T_i=1,X_i=x]\\
&\quad+\mathbb{E}[(\widehat{S}_{1i}-S_{1i})(\widehat{S}_{2i}-S_{2i})\mid W_i=w,T_i=1,X_i=x].
\end{aligned} E [ S 0 i − S 0 i ∣ W i = w , T i = 1 , X i = x ] = E [ S 1 i − S 1 i ∣ W i = w , T i = 1 , X i = x ] E [ S 2 i ∣ W i = w , T i = 1 , X i = x ] + E [ S 1 i ∣ W i = w , T i = 1 , X i = x ] E [ S 2 i − S 2 i ∣ W i = w , T i = 1 , X i = x ] + E [( S 1 i − S 1 i ) ( S 2 i − S 2 i ) ∣ W i = w , T i = 1 , X i = x ] .
Substituting the calibration identities now yields
E [ S ^ 0 i − S 0 i ∣ W i = w , T i = 1 , X i = x ] = ( 1 − w ) E [ S ^ 1 i − S 1 i ∣ W i = w , T i = 1 , X i = x ] + ( 1 − w ) E [ S ^ 2 i − S 2 i ∣ W i = w , T i = 1 , X i = x ] + E [ ( S ^ 1 i − S 1 i ) ( S ^ 2 i − S 2 i ) ∣ W i = w , T i = 1 , X i = x ] = ( 1 − w ) ∑ j = 1 2 E [ S ^ j i − S j i ∣ W i = w , T i = 1 , X i = x ] + E [ ( S ^ 1 i − S 1 i ) ( S ^ 2 i − S 2 i ) ∣ W i = w , T i = 1 , X i = x ] . (41) \begin{aligned}
\mathbb{E}[\widehat{S}_{0i}-S_{0i}\mid W_i=w,T_i=1,X_i=x]
&=(1-w)\mathbb{E}[\widehat{S}_{1i}-S_{1i}\mid W_i=w,T_i=1,X_i=x]\\
&\quad+(1-w)\mathbb{E}[\widehat{S}_{2i}-S_{2i}\mid W_i=w,T_i=1,X_i=x]\\
&\quad+\mathbb{E}[(\widehat{S}_{1i}-S_{1i})(\widehat{S}_{2i}-S_{2i})\mid W_i=w,T_i=1,X_i=x]\\
&=(1-w)\sum_{j=1}^{2}\mathbb{E}[\widehat{S}_{ji}-S_{ji}\mid W_i=w,T_i=1,X_i=x]\\
&\quad+\mathbb{E}[(\widehat{S}_{1i}-S_{1i})(\widehat{S}_{2i}-S_{2i})\mid W_i=w,T_i=1,X_i=x]. \tag*{(41)}
\end{aligned} E [ S 0 i − S 0 i ∣ W i = w , T i = 1 , X i = x ] = ( 1 − w ) E [ S 1 i − S 1 i ∣ W i = w , T i = 1 , X i = x ] + ( 1 − w ) E [ S 2 i − S 2 i ∣ W i = w , T i = 1 , X i = x ] + E [( S 1 i − S 1 i ) ( S 2 i − S 2 i ) ∣ W i = w , T i = 1 , X i = x ] = ( 1 − w ) j = 1 ∑ 2 E [ S j i − S j i ∣ W i = w , T i = 1 , X i = x ] + E [( S 1 i − S 1 i ) ( S 2 i − S 2 i ) ∣ W i = w , T i = 1 , X i = x ] . (41)
For the third-check factor, adding and subtracting m ^ 1 ( w , x ) \widehat{m}_1(w,x) m 1 ( w , x ) in the numerator gives
m 1 ( w , x ) − m ^ 1 ( 1 , x ) m ^ 1 ( 1 , x ) − m ^ 1 ( 0 , x ) = m ^ 1 ( w , x ) − m ^ 1 ( 1 , x ) − { m ^ 1 ( w , x ) − m 1 ( w , x ) } m ^ 1 ( 1 , x ) − m ^ 1 ( 0 , x ) = m ^ 1 ( w , x ) − m ^ 1 ( 1 , x ) m ^ 1 ( 1 , x ) − m ^ 1 ( 0 , x ) − m ^ 1 ( w , x ) − m 1 ( w , x ) m ^ 1 ( 1 , x ) − m ^ 1 ( 0 , x ) . \begin{aligned}
\frac{m_1(w,x)-\widehat{m}_1(1,x)}{\widehat{m}_1(1,x)-\widehat{m}_1(0,x)}
&=\frac{\widehat{m}_1(w,x)-\widehat{m}_1(1,x)-\{\widehat{m}_1(w,x)-m_1(w,x)\}}{\widehat{m}_1(1,x)-\widehat{m}_1(0,x)}\\
&=\frac{\widehat{m}_1(w,x)-\widehat{m}_1(1,x)}{\widehat{m}_1(1,x)-\widehat{m}_1(0,x)}
-\frac{\widehat{m}_1(w,x)-m_1(w,x)}{\widehat{m}_1(1,x)-\widehat{m}_1(0,x)}.
\end{aligned} m 1 ( 1 , x ) − m 1 ( 0 , x ) m 1 ( w , x ) − m 1 ( 1 , x ) = m 1 ( 1 , x ) − m 1 ( 0 , x ) m 1 ( w , x ) − m 1 ( 1 , x ) − { m 1 ( w , x ) − m 1 ( w , x )} = m 1 ( 1 , x ) − m 1 ( 0 , x ) m 1 ( w , x ) − m 1 ( 1 , x ) − m 1 ( 1 , x ) − m 1 ( 0 , x ) m 1 ( w , x ) − m 1 ( w , x ) .
Because w w w is binary,
m ^ 1 ( w , x ) − m ^ 1 ( 1 , x ) = ( 1 − w ) m ^ 1 ( 0 , x ) + w m ^ 1 ( 1 , x ) − m ^ 1 ( 1 , x ) = ( 1 − w ) m ^ 1 ( 0 , x ) + ( w − 1 ) m ^ 1 ( 1 , x ) = ( w − 1 ) { m ^ 1 ( 1 , x ) − m ^ 1 ( 0 , x ) } . \begin{aligned}
\widehat{m}_1(w,x)-\widehat{m}_1(1,x)
&=(1-w)\widehat{m}_1(0,x)+w\widehat{m}_1(1,x)-\widehat{m}_1(1,x)\\
&=(1-w)\widehat{m}_1(0,x)+(w-1)\widehat{m}_1(1,x)\\
&=(w-1)\{\widehat{m}_1(1,x)-\widehat{m}_1(0,x)\}.
\end{aligned} m 1 ( w , x ) − m 1 ( 1 , x ) = ( 1 − w ) m 1 ( 0 , x ) + w m 1 ( 1 , x ) − m 1 ( 1 , x ) = ( 1 − w ) m 1 ( 0 , x ) + ( w − 1 ) m 1 ( 1 , x ) = ( w − 1 ) { m 1 ( 1 , x ) − m 1 ( 0 , x )} .
Substitution into the preceding fraction then gives
m 1 ( w , x ) − m ^ 1 ( 1 , x ) m ^ 1 ( 1 , x ) − m ^ 1 ( 0 , x ) = ( w − 1 ) { m ^ 1 ( 1 , x ) − m ^ 1 ( 0 , x ) } m ^ 1 ( 1 , x ) − m ^ 1 ( 0 , x ) − m ^ 1 ( w , x ) − m 1 ( w , x ) m ^ 1 ( 1 , x ) − m ^ 1 ( 0 , x ) = ( w − 1 ) − m ^ 1 ( w , x ) − m 1 ( w , x ) m ^ 1 ( 1 , x ) − m ^ 1 ( 0 , x ) . \begin{aligned}
\frac{m_1(w,x)-\widehat{m}_1(1,x)}{\widehat{m}_1(1,x)-\widehat{m}_1(0,x)}
&=\frac{(w-1)\{\widehat{m}_1(1,x)-\widehat{m}_1(0,x)\}}{\widehat{m}_1(1,x)-\widehat{m}_1(0,x)}
-\frac{\widehat{m}_1(w,x)-m_1(w,x)}{\widehat{m}_1(1,x)-\widehat{m}_1(0,x)}\\
&=(w-1)-\frac{\widehat{m}_1(w,x)-m_1(w,x)}{\widehat{m}_1(1,x)-\widehat{m}_1(0,x)}.
\end{aligned} m 1 ( 1 , x ) − m 1 ( 0 , x ) m 1 ( w , x ) − m 1 ( 1 , x ) = m 1 ( 1 , x ) − m 1 ( 0 , x ) ( w − 1 ) { m 1 ( 1 , x ) − m 1 ( 0 , x )} − m 1 ( 1 , x ) − m 1 ( 0 , x ) m 1 ( w , x ) − m 1 ( w , x ) = ( w − 1 ) − m 1 ( 1 , x ) − m 1 ( 0 , x ) m 1 ( w , x ) − m 1 ( w , x ) .
The same binary identity applied to the true means gives
m 1 ( w , x ) − m 1 ( 1 , x ) m 1 ( 1 , x ) − m 1 ( 0 , x ) = ( w − 1 ) { m 1 ( 1 , x ) − m 1 ( 0 , x ) } m 1 ( 1 , x ) − m 1 ( 0 , x ) = w − 1. \frac{m_1(w,x)-m_1(1,x)}{m_1(1,x)-m_1(0,x)}
=\frac{(w-1)\{m_1(1,x)-m_1(0,x)\}}{m_1(1,x)-m_1(0,x)}
=w-1. m 1 ( 1 , x ) − m 1 ( 0 , x ) m 1 ( w , x ) − m 1 ( 1 , x ) = m 1 ( 1 , x ) − m 1 ( 0 , x ) ( w − 1 ) { m 1 ( 1 , x ) − m 1 ( 0 , x )} = w − 1.
We next calculate the mean of the correction difference. First, its product representation implies
E [ C ^ i − C i ∣ W i = w , T i = 1 , X i = x ] = E [ B ^ i A ^ i ∣ W i = w , T i = 1 , X i = x ] − E [ B i A i ∣ W i = w , T i = 1 , X i = x ] . \mathbb{E}[\widehat{C}_i-C_i \mid W_i=w,T_i=1,X_i=x]
= \mathbb{E}[\widehat{B}_i\widehat{A}_i \mid W_i=w,T_i=1,X_i=x]-\mathbb{E}[B_iA_i \mid W_i=w,T_i=1,X_i=x]. E [ C i − C i ∣ W i = w , T i = 1 , X i = x ] = E [ B i A i ∣ W i = w , T i = 1 , X i = x ] − E [ B i A i ∣ W i = w , T i = 1 , X i = x ] .
Conditional independence of the third check from the first two checks gives
E [ C ^ i − C i ∣ W i = w , T i = 1 , X i = x ] = E [ B ^ i ∣ W i = w , T i = 1 , X i = x ] E [ A ^ i ∣ W i = w , T i = 1 , X i = x ] − E [ B i ∣ W i = w , T i = 1 , X i = x ] E [ A i ∣ W i = w , T i = 1 , X i = x ] . \begin{aligned}
\mathbb{E}[\widehat{C}_i-C_i \mid W_i=w,T_i=1,X_i=x]
&= \mathbb{E}[\widehat{B}_i \mid W_i=w,T_i=1,X_i=x]\mathbb{E}[\widehat{A}_i \mid W_i=w,T_i=1,X_i=x]\\
&\quad-\mathbb{E}[B_i \mid W_i=w,T_i=1,X_i=x]\mathbb{E}[A_i \mid W_i=w,T_i=1,X_i=x].
\end{aligned} E [ C i − C i ∣ W i = w , T i = 1 , X i = x ] = E [ B i ∣ W i = w , T i = 1 , X i = x ] E [ A i ∣ W i = w , T i = 1 , X i = x ] − E [ B i ∣ W i = w , T i = 1 , X i = x ] E [ A i ∣ W i = w , T i = 1 , X i = x ] .
Using the conditional mean of the third check yields
E [ C ^ i − C i ∣ W i = w , T i = 1 , X i = x ] = m ^ 1 ( w , x ) − m ^ 1 ( 1 , x ) m ^ 1 ( 1 , x ) − m ^ 1 ( 0 , x ) E [ A ^ i ∣ W i = w , T i = 1 , X i = x ] − m 1 ( w , x ) − m 1 ( 1 , x ) m 1 ( 1 , x ) − m 1 ( 0 , x ) E [ A i ∣ W i = w , T i = 1 , X i = x ] = m ^ 1 ( w , x ) − m ^ 1 ( 1 , x ) m ^ 1 ( 1 , x ) − m ^ 1 ( 0 , x ) E [ A ^ i ∣ W i = w , T i = 1 , X i = x ] − ( w − 1 ) E [ A i ∣ W i = w , T i = 1 , X i = x ] . \begin{aligned}
\mathbb{E}[\widehat{C}_i-C_i \mid W_i=w,T_i=1,X_i=x]
&= \frac{\widehat{m}_1(w,x)-\widehat{m}_1(1,x)}{\widehat{m}_1(1,x)-\widehat{m}_1(0,x)}
\mathbb{E}[\widehat{A}_i \mid W_i=w,T_i=1,X_i=x]\\
&\quad-\frac{m_1(w,x)-m_1(1,x)}{m_1(1,x)-m_1(0,x)}
\mathbb{E}[A_i \mid W_i=w,T_i=1,X_i=x]\\
&= \frac{\widehat{m}_1(w,x)-\widehat{m}_1(1,x)}{\widehat{m}_1(1,x)-\widehat{m}_1(0,x)}
\mathbb{E}[\widehat{A}_i \mid W_i=w,T_i=1,X_i=x]\\
&\quad-(w-1)\mathbb{E}[A_i \mid W_i=w,T_i=1,X_i=x].
\end{aligned} E [ C i − C i ∣ W i = w , T i = 1 , X i = x ] = m 1 ( 1 , x ) − m 1 ( 0 , x ) m 1 ( w , x ) − m 1 ( 1 , x ) E [ A i ∣ W i = w , T i = 1 , X i = x ] − m 1 ( 1 , x ) − m 1 ( 0 , x ) m 1 ( w , x ) − m 1 ( 1 , x ) E [ A i ∣ W i = w , T i = 1 , X i = x ] = m 1 ( 1 , x ) − m 1 ( 0 , x ) m 1 ( w , x ) − m 1 ( 1 , x ) E [ A i ∣ W i = w , T i = 1 , X i = x ] − ( w − 1 ) E [ A i ∣ W i = w , T i = 1 , X i = x ] .
The last equality uses the binary identity for the true third-check means. (40) then removes the last term:
E [ C ^ i − C i ∣ W i = w , T i = 1 , X i = x ] = m ^ 1 ( w , x ) − m ^ 1 ( 1 , x ) m ^ 1 ( 1 , x ) − m ^ 1 ( 0 , x ) E [ A ^ i ∣ W i = w , T i = 1 , X i = x ] . \mathbb{E}[\widehat{C}_i-C_i \mid W_i=w,T_i=1,X_i=x]
= \frac{\widehat{m}_1(w,x)-\widehat{m}_1(1,x)}{\widehat{m}_1(1,x)-\widehat{m}_1(0,x)}
\mathbb{E}[\widehat{A}_i \mid W_i=w,T_i=1,X_i=x]. E [ C i − C i ∣ W i = w , T i = 1 , X i = x ] = m 1 ( 1 , x ) − m 1 ( 0 , x ) m 1 ( w , x ) − m 1 ( 1 , x ) E [ A i ∣ W i = w , T i = 1 , X i = x ] .
Substituting the identity for the fitted fraction gives
E [ C ^ i − C i ∣ W i = w , T i = 1 , X i = x ] = { ( w − 1 ) − m ^ 1 ( w , x ) − m 1 ( w , x ) m ^ 1 ( 1 , x ) − m ^ 1 ( 0 , x ) } E [ A ^ i ∣ W i = w , T i = 1 , X i = x ] = ( w − 1 ) E [ A ^ i ∣ W i = w , T i = 1 , X i = x ] − m ^ 1 ( w , x ) − m 1 ( w , x ) m ^ 1 ( 1 , x ) − m ^ 1 ( 0 , x ) E [ A ^ i ∣ W i = w , T i = 1 , X i = x ] . \begin{aligned}
\mathbb{E}[\widehat{C}_i-C_i \mid W_i=w,T_i=1,X_i=x]
&= \left\{(w-1)-\frac{\widehat{m}_1(w,x)-m_1(w,x)}{\widehat{m}_1(1,x)-\widehat{m}_1(0,x)}\right\}
\mathbb{E}[\widehat{A}_i \mid W_i=w,T_i=1,X_i=x]\\
&= (w-1)\mathbb{E}[\widehat{A}_i \mid W_i=w,T_i=1,X_i=x]\\
&\quad-\frac{\widehat{m}_1(w,x)-m_1(w,x)}{\widehat{m}_1(1,x)-\widehat{m}_1(0,x)}
\mathbb{E}[\widehat{A}_i \mid W_i=w,T_i=1,X_i=x].
\end{aligned} E [ C i − C i ∣ W i = w , T i = 1 , X i = x ] = { ( w − 1 ) − m 1 ( 1 , x ) − m 1 ( 0 , x ) m 1 ( w , x ) − m 1 ( w , x ) } E [ A i ∣ W i = w , T i = 1 , X i = x ] = ( w − 1 ) E [ A i ∣ W i = w , T i = 1 , X i = x ] − m 1 ( 1 , x ) − m 1 ( 0 , x ) m 1 ( w , x ) − m 1 ( w , x ) E [ A i ∣ W i = w , T i = 1 , X i = x ] .
Since the conditional mean of A i A_i A i is zero, this is also
E [ C ^ i − C i ∣ W i = w , T i = 1 , X i = x ] = ( w − 1 ) E [ A ^ i − A i ∣ W i = w , T i = 1 , X i = x ] − m ^ 1 ( w , x ) − m 1 ( w , x ) m ^ 1 ( 1 , x ) − m ^ 1 ( 0 , x ) E [ A ^ i − A i ∣ W i = w , T i = 1 , X i = x ] . (42) \begin{aligned}
\mathbb{E}[\widehat{C}_i-C_i \mid W_i=w,T_i=1,X_i=x]
&= (w-1)\mathbb{E}[\widehat{A}_i-A_i \mid W_i=w,T_i=1,X_i=x]\\
&\quad-\frac{\widehat{m}_1(w,x)-m_1(w,x)}{\widehat{m}_1(1,x)-\widehat{m}_1(0,x)}
\mathbb{E}[\widehat{A}_i-A_i \mid W_i=w,T_i=1,X_i=x]. \tag*{(42)}
\end{aligned} E [ C i − C i ∣ W i = w , T i = 1 , X i = x ] = ( w − 1 ) E [ A i − A i ∣ W i = w , T i = 1 , X i = x ] − m 1 ( 1 , x ) − m 1 ( 0 , x ) m 1 ( w , x ) − m 1 ( w , x ) E [ A i − A i ∣ W i = w , T i = 1 , X i = x ] . (42)
We can now isolate the first-order terms in the remainder. Substituting (42) first gives
E [ − S ^ 0 i + S 0 i + C ^ i − C i ∣ W i = w , T i = 1 , X i = x ] = − E [ S ^ 0 i − S 0 i ∣ W i = w , T i = 1 , X i = x ] + ( w − 1 ) E [ A ^ i − A i ∣ W i = w , T i = 1 , X i = x ] − m ^ 1 ( w , x ) − m 1 ( w , x ) m ^ 1 ( 1 , x ) − m ^ 1 ( 0 , x ) E [ A ^ i − A i ∣ W i = w , T i = 1 , X i = x ] . \begin{aligned}
\mathbb{E}[-\widehat{S}_{0i}+S_{0i}+\widehat{C}_i-C_i \mid W_i=w,T_i=1,X_i=x]
= -\mathbb{E}[\widehat{S}_{0i}-S_{0i} \mid W_i=w,T_i=1,X_i=x] \\
&+ (w-1)\mathbb{E}[\widehat{A}_i-A_i \mid W_i=w,T_i=1,X_i=x]
-\frac{\widehat{m}_1(w,x)-m_1(w,x)}{\widehat{m}_1(1,x)-\widehat{m}_1(0,x)}\mathbb{E}[\widehat{A}_i-A_i \mid W_i=w,T_i=1,X_i=x].
\end{aligned} E [ − S 0 i + S 0 i + C i − C i ∣ W i = w , T i = 1 , X i = x ] = − E [ S 0 i − S 0 i ∣ W i = w , T i = 1 , X i = x ] + ( w − 1 ) E [ A i − A i ∣ W i = w , T i = 1 , X i = x ] − m 1 ( 1 , x ) − m 1 ( 0 , x ) m 1 ( w , x ) − m 1 ( w , x ) E [ A i − A i ∣ W i = w , T i = 1 , X i = x ] .
The definition of A i A_i A i implies
A ^ i − A i = ( S ^ 1 i − S 1 i ) + ( S ^ 2 i − S 2 i ) − 2 ( S ^ 0 i − S 0 i ) . \widehat{A}_i-A_i=(\widehat{S}_{1i}-S_{1i})+(\widehat{S}_{2i}-S_{2i})-2(\widehat{S}_{0i}-S_{0i}). A i − A i = ( S 1 i − S 1 i ) + ( S 2 i − S 2 i ) − 2 ( S 0 i − S 0 i ) .
Substituting this expression into the term multiplied by w − 1 w-1 w − 1 yields
E [ − S ^ 0 i + S 0 i + C ^ i − C i ∣ W i = w , T i = 1 , X i = x ] = − E [ S ^ 0 i − S 0 i ∣ W i = w , T i = 1 , X i = x ] + ( w − 1 ) ∑ j = 1 2 E [ S ^ j i − S j i ∣ W i = w , T i = 1 , X i = x ] − 2 ( w − 1 ) E [ S ^ 0 i − S 0 i ∣ W i = w , T i = 1 , X i = x ] − m ^ 1 ( w , x ) − m 1 ( w , x ) m ^ 1 ( 1 , x ) − m ^ 1 ( 0 , x ) E [ A ^ i − A i ∣ W i = w , T i = 1 , X i = x ] = ( w − 1 ) ∑ j = 1 2 E [ S ^ j i − S j i ∣ W i = w , T i = 1 , X i = x ] + ( 1 − 2 w ) E [ S ^ 0 i − S 0 i ∣ W i = w , T i = 1 , X i = x ] − m ^ 1 ( w , x ) − m 1 ( w , x ) m ^ 1 ( 1 , x ) − m ^ 1 ( 0 , x ) E [ A ^ i − A i ∣ W i = w , T i = 1 , X i = x ] . \begin{aligned}
\mathbb{E}[-\widehat{S}_{0i}+S_{0i}+\widehat{C}_i-C_i \mid W_i=w,T_i=1,X_i=x]
={}&-\mathbb{E}[\widehat{S}_{0i}-S_{0i}\mid W_i=w,T_i=1,X_i=x]\\
&+(w-1)\sum_{j=1}^{2}\mathbb{E}[\widehat{S}_{ji}-S_{ji}\mid W_i=w,T_i=1,X_i=x]\\
&-2(w-1)\mathbb{E}[\widehat{S}_{0i}-S_{0i}\mid W_i=w,T_i=1,X_i=x]\\
&-\frac{\widehat{m}_1(w,x)-m_1(w,x)}{\widehat{m}_1(1,x)-\widehat{m}_1(0,x)}\mathbb{E}[\widehat{A}_i-A_i\mid W_i=w,T_i=1,X_i=x]\\
={}&(w-1)\sum_{j=1}^{2}\mathbb{E}[\widehat{S}_{ji}-S_{ji}\mid W_i=w,T_i=1,X_i=x]\\
&+(1-2w)\mathbb{E}[\widehat{S}_{0i}-S_{0i}\mid W_i=w,T_i=1,X_i=x]\\
&-\frac{\widehat{m}_1(w,x)-m_1(w,x)}{\widehat{m}_1(1,x)-\widehat{m}_1(0,x)}\mathbb{E}[\widehat{A}_i-A_i\mid W_i=w,T_i=1,X_i=x].
\end{aligned} E [ − S 0 i + S 0 i + C i − C i ∣ W i = w , T i = 1 , X i = x ] = = − E [ S 0 i − S 0 i ∣ W i = w , T i = 1 , X i = x ] + ( w − 1 ) j = 1 ∑ 2 E [ S j i − S j i ∣ W i = w , T i = 1 , X i = x ] − 2 ( w − 1 ) E [ S 0 i − S 0 i ∣ W i = w , T i = 1 , X i = x ] − m 1 ( 1 , x ) − m 1 ( 0 , x ) m 1 ( w , x ) − m 1 ( w , x ) E [ A i − A i ∣ W i = w , T i = 1 , X i = x ] ( w − 1 ) j = 1 ∑ 2 E [ S j i − S j i ∣ W i = w , T i = 1 , X i = x ] + ( 1 − 2 w ) E [ S 0 i − S 0 i ∣ W i = w , T i = 1 , X i = x ] − m 1 ( 1 , x ) − m 1 ( 0 , x ) m 1 ( w , x ) − m 1 ( w , x ) E [ A i − A i ∣ W i = w , T i = 1 , X i = x ] .
Here, the last equality collects the two coefficients on the product-surrogate error, using − 1 − 2 ( w − 1 ) = 1 − 2 w -1-2(w-1)=1-2w − 1 − 2 ( w − 1 ) = 1 − 2 w . Next, Equation (S20) gives
E [ − S ^ 0 i + S 0 i + C ^ i − C i ∣ W i = w , T i = 1 , X i = x ] = ( w − 1 ) ∑ j = 1 2 E [ S ^ j i − S j i ∣ W i = w , T i = 1 , X i = x ] + ( 1 − 2 w ) ( 1 − w ) ∑ j = 1 2 E [ S ^ j i − S j i ∣ W i = w , T i = 1 , X i = x ] + ( 1 − 2 w ) E [ ( S ^ 1 i − S 1 i ) ( S ^ 2 i − S 2 i ) ∣ W i = w , T i = 1 , X i = x ] − m ^ 1 ( w , x ) − m 1 ( w , x ) m ^ 1 ( 1 , x ) − m ^ 1 ( 0 , x ) E [ A ^ i − A i ∣ W i = w , T i = 1 , X i = x ] = { ( w − 1 ) + ( 1 − 2 w ) ( 1 − w ) } ∑ j = 1 2 E [ S ^ j i − S j i ∣ W i = w , T i = 1 , X i = x ] + ( 1 − 2 w ) E [ ( S ^ 1 i − S 1 i ) ( S ^ 2 i − S 2 i ) ∣ W i = w , T i = 1 , X i = x ] − m ^ 1 ( w , x ) − m 1 ( w , x ) m ^ 1 ( 1 , x ) − m ^ 1 ( 0 , x ) E [ A ^ i − A i ∣ W i = w , T i = 1 , X i = x ] . \begin{aligned}
\mathbb{E}[-\widehat{S}_{0i}+S_{0i}+\widehat{C}_i-C_i\mid W_i=w,T_i=1,X_i=x]
={}&(w-1)\sum_{j=1}^{2}\mathbb{E}[\widehat{S}_{ji}-S_{ji}\mid W_i=w,T_i=1,X_i=x]\\
&+(1-2w)(1-w)\sum_{j=1}^{2}\mathbb{E}[\widehat{S}_{ji}-S_{ji}\mid W_i=w,T_i=1,X_i=x]\\
&+(1-2w)\mathbb{E}[(\widehat{S}_{1i}-S_{1i})(\widehat{S}_{2i}-S_{2i})\mid W_i=w,T_i=1,X_i=x]\\
&-\frac{\widehat{m}_1(w,x)-m_1(w,x)}{\widehat{m}_1(1,x)-\widehat{m}_1(0,x)}\mathbb{E}[\widehat{A}_i-A_i\mid W_i=w,T_i=1,X_i=x]\\
={}&\left\{(w-1)+(1-2w)(1-w)\right\}\sum_{j=1}^{2}\mathbb{E}[\widehat{S}_{ji}-S_{ji}\mid W_i=w,T_i=1,X_i=x]\\
&+(1-2w)\mathbb{E}[(\widehat{S}_{1i}-S_{1i})(\widehat{S}_{2i}-S_{2i})\mid W_i=w,T_i=1,X_i=x]\\
&-\frac{\widehat{m}_1(w,x)-m_1(w,x)}{\widehat{m}_1(1,x)-\widehat{m}_1(0,x)}\mathbb{E}[\widehat{A}_i-A_i\mid W_i=w,T_i=1,X_i=x].
\end{aligned} E [ − S 0 i + S 0 i + C i − C i ∣ W i = w , T i = 1 , X i = x ] = = ( w − 1 ) j = 1 ∑ 2 E [ S j i − S j i ∣ W i = w , T i = 1 , X i = x ] + ( 1 − 2 w ) ( 1 − w ) j = 1 ∑ 2 E [ S j i − S j i ∣ W i = w , T i = 1 , X i = x ] + ( 1 − 2 w ) E [( S 1 i − S 1 i ) ( S 2 i − S 2 i ) ∣ W i = w , T i = 1 , X i = x ] − m 1 ( 1 , x ) − m 1 ( 0 , x ) m 1 ( w , x ) − m 1 ( w , x ) E [ A i − A i ∣ W i = w , T i = 1 , X i = x ] { ( w − 1 ) + ( 1 − 2 w ) ( 1 − w ) } j = 1 ∑ 2 E [ S j i − S j i ∣ W i = w , T i = 1 , X i = x ] + ( 1 − 2 w ) E [( S 1 i − S 1 i ) ( S 2 i − S 2 i ) ∣ W i = w , T i = 1 , X i = x ] − m 1 ( 1 , x ) − m 1 ( 0 , x ) m 1 ( w , x ) − m 1 ( w , x ) E [ A i − A i ∣ W i = w , T i = 1 , X i = x ] .
The coefficient on the first-order surrogate errors is
( w − 1 ) + ( 1 − 2 w ) ( 1 − w ) = w − 1 + 1 − 3 w + 2 w 2 (w-1)+(1-2w)(1-w)=w-1+1-3w+2w^2 ( w − 1 ) + ( 1 − 2 w ) ( 1 − w ) = w − 1 + 1 − 3 w + 2 w 2
= 2 w 2 − 2 w = − 2 w ( 1 − w ) = 0 , \begin{aligned}
&= 2w^{2}-2w \\
&= -2w(1-w) \\
&= 0,
\end{aligned} = 2 w 2 − 2 w = − 2 w ( 1 − w ) = 0 ,
where the last equality uses w ∈ { 0 , 1 } w \in\{0,1\} w ∈ { 0 , 1 } . Thus,
E [ − S ^ 0 i + S 0 i + C ^ i − C i ∣ W i = w , T i = 1 , X i = x ] = ( 1 − 2 w ) E [ ( S ^ 1 i − S 1 i ) ( S ^ 2 i − S 2 i ) ∣ W i = w , T i = 1 , X i = x ] − m ^ 1 ( w , x ) − m 1 ( w , x ) m ^ 1 ( 1 , x ) − m ^ 1 ( 0 , x ) E [ A ^ i − A i ∣ W i = w , T i = 1 , X i = x ] . \begin{aligned}
\mathbb{E}[-\widehat{S}_{0i}+S_{0i}+\widehat{C}_{i}-C_{i}\mid W_{i}=w,T_{i}=1,X_{i}=x]
&= (1-2w)\mathbb{E}[(\widehat{S}_{1i}-S_{1i})(\widehat{S}_{2i}-S_{2i})\mid W_{i}=w,T_{i}=1,X_{i}=x] \\
&\quad-\frac{\widehat{m}_{1}(w,x)-m_{1}(w,x)}{\widehat{m}_{1}(1,x)-\widehat{m}_{1}(0,x)}
\mathbb{E}[\widehat{A}_{i}-A_{i}\mid W_{i}=w,T_{i}=1,X_{i}=x].
\end{aligned} E [ − S 0 i + S 0 i + C i − C i ∣ W i = w , T i = 1 , X i = x ] = ( 1 − 2 w ) E [( S 1 i − S 1 i ) ( S 2 i − S 2 i ) ∣ W i = w , T i = 1 , X i = x ] − m 1 ( 1 , x ) − m 1 ( 0 , x ) m 1 ( w , x ) − m 1 ( w , x ) E [ A i − A i ∣ W i = w , T i = 1 , X i = x ] .
Returning to Equation (S18), iterated expectations give
E [ H i Q ] = Pr ( T i = 1 ) E [ 1 π 1 ( X i ) E [ − S ^ 0 i + S 0 i + C ^ i − C i ∣ W i , T i = 1 , X i ] ∣ T i = 1 ] = Pr ( T i = 1 ) E [ 1 − 2 W i π 1 ( X i ) E [ ( S ^ 1 i − S 1 i ) ( S ^ 2 i − S 2 i ) ∣ W i , T i = 1 , X i ] ∣ T i = 1 ] − Pr ( T i = 1 ) E [ m ^ 1 ( W i , X i ) − m 1 ( W i , X i ) π 1 ( X i ) { m ^ 1 ( 1 , X i ) − m ^ 1 ( 0 , X i ) } E [ A ^ i − A i ∣ W i , T i = 1 , X i ] ∣ T i = 1 ] . \begin{aligned}
\mathbb{E}[H_{i}^{Q}]
&=\Pr(T_{i}=1)\mathbb{E}\left[\left.\frac{1}{\pi_{1}(X_{i})}\mathbb{E}[-\widehat{S}_{0i}+S_{0i}+\widehat{C}_{i}-C_{i}\mid W_{i},T_{i}=1,X_{i}]\right|T_{i}=1\right] \\
&=\Pr(T_{i}=1)\mathbb{E}\left[\left.\frac{1-2W_{i}}{\pi_{1}(X_{i})}\mathbb{E}[(\widehat{S}_{1i}-S_{1i})(\widehat{S}_{2i}-S_{2i})\mid W_{i},T_{i}=1,X_{i}]\right|T_{i}=1\right] \\
&\quad-\Pr(T_{i}=1)\mathbb{E}\left[\left.\frac{\widehat{m}_{1}(W_{i},X_{i})-m_{1}(W_{i},X_{i})}{\pi_{1}(X_{i})\{\widehat{m}_{1}(1,X_{i})-\widehat{m}_{1}(0,X_{i})\}}\mathbb{E}[\widehat{A}_{i}-A_{i}\mid W_{i},T_{i}=1,X_{i}]\right|T_{i}=1\right].
\end{aligned} E [ H i Q ] = Pr ( T i = 1 ) E [ π 1 ( X i ) 1 E [ − S 0 i + S 0 i + C i − C i ∣ W i , T i = 1 , X i ] T i = 1 ] = Pr ( T i = 1 ) E [ π 1 ( X i ) 1 − 2 W i E [( S 1 i − S 1 i ) ( S 2 i − S 2 i ) ∣ W i , T i = 1 , X i ] T i = 1 ] − Pr ( T i = 1 ) E [ π 1 ( X i ) { m 1 ( 1 , X i ) − m 1 ( 0 , X i )} m 1 ( W i , X i ) − m 1 ( W i , X i ) E [ A i − A i ∣ W i , T i = 1 , X i ] T i = 1 ] .
Both multiplying coefficients are functions of ( W i , X i ) (W_{i},X_{i}) ( W i , X i ) . Applying iterated expectations to the first term and then to the second term gives
E [ H i Q ] = Pr ( T i = 1 ) E [ 1 − 2 W i π 1 ( X i ) ( S ^ 1 i − S 1 i ) ( S ^ 2 i − S 2 i ) ∣ T i = 1 ] − Pr ( T i = 1 ) E [ m ^ 1 ( W i , X i ) − m 1 ( W i , X i ) π 1 ( X i ) { m ^ 1 ( 1 , X i ) − m ^ 1 ( 0 , X i ) } ( A ^ i − A i ) ∣ T i = 1 ] = Pr ( T i = 1 ) E [ 1 − 2 W i π 1 ( X i ) ( S ^ 1 i − S 1 i ) ( S ^ 2 i − S 2 i ) ∣ T i = 1 ] − Pr ( T i = 1 ) E [ m ^ 1 ( W i , X i ) − m 1 ( W i , X i ) π 1 ( X i ) { m ^ 1 ( 1 , X i ) − m ^ 1 ( 0 , X i ) } ( A ^ i − A i ) ∣ T i = 1 ] . (43) \begin{aligned}
\mathbb{E}[H_{i}^{Q}]
&=\Pr(T_{i}=1)\mathbb{E}\left[\left.\frac{1-2W_{i}}{\pi_{1}(X_{i})}(\widehat{S}_{1i}-S_{1i})(\widehat{S}_{2i}-S_{2i})\right|T_{i}=1\right] \\
&\quad-\Pr(T_{i}=1)\mathbb{E}\left[\left.\frac{\widehat{m}_{1}(W_{i},X_{i})-m_{1}(W_{i},X_{i})}{\pi_{1}(X_{i})\{\widehat{m}_{1}(1,X_{i})-\widehat{m}_{1}(0,X_{i})\}}(\widehat{A}_{i}-A_{i})\right|T_{i}=1\right] \\
&=\Pr(T_{i}=1)\mathbb{E}\left[\left.\frac{1-2W_{i}}{\pi_{1}(X_{i})}(\widehat{S}_{1i}-S_{1i})(\widehat{S}_{2i}-S_{2i})\right|T_{i}=1\right] \\
&\quad-\Pr(T_{i}=1)\mathbb{E}\left[\left.\frac{\widehat{m}_{1}(W_{i},X_{i})-m_{1}(W_{i},X_{i})}{\pi_{1}(X_{i})\{\widehat{m}_{1}(1,X_{i})-\widehat{m}_{1}(0,X_{i})\}}(\widehat{A}_{i}-A_{i})\right|T_{i}=1\right]. \tag*{(43)}
\end{aligned} E [ H i Q ] = Pr ( T i = 1 ) E [ π 1 ( X i ) 1 − 2 W i ( S 1 i − S 1 i ) ( S 2 i − S 2 i ) T i = 1 ] − Pr ( T i = 1 ) E [ π 1 ( X i ) { m 1 ( 1 , X i ) − m 1 ( 0 , X i )} m 1 ( W i , X i ) − m 1 ( W i , X i ) ( A i − A i ) T i = 1 ] = Pr ( T i = 1 ) E [ π 1 ( X i ) 1 − 2 W i ( S 1 i − S 1 i ) ( S 2 i − S 2 i ) T i = 1 ] − Pr ( T i = 1 ) E [ π 1 ( X i ) { m 1 ( 1 , X i ) − m 1 ( 0 , X i )} m 1 ( W i , X i ) − m 1 ( W i , X i ) ( A i − A i ) T i = 1 ] . (43)
To bound the second product, we first control the error in the third-check mean evaluated at the realized latent state. Conditioning on X i X_{i} X i within the treated arm gives
E [ ∣ m ^ 1 ( W i , X i ) − m 1 ( W i , X i ) ∣ 2 ∣ T i = 1 ] = E [ E [ ∣ m ^ 1 ( W i , X i ) − m 1 ( W i , X i ) ∣ 2 ∣ X i , T i = 1 ] ∣ T i = 1 ] = E [ { 1 − q ( X i ) } ∣ m ^ 1 ( 0 , X i ) − m 1 ( 0 , X i ) ∣ 2 + q ( X i ) ∣ m ^ 1 ( 1 , X i ) − m 1 ( 1 , X i ) ∣ 2 ∣ T i = 1 ] . \begin{aligned}
\mathbb{E}[|\widehat{m}_{1}(W_{i},X_{i})-m_{1}(W_{i},X_{i})|^{2}\mid T_{i}=1]
&=\mathbb{E}\left[\left.\mathbb{E}[|\widehat{m}_{1}(W_{i},X_{i})-m_{1}(W_{i},X_{i})|^{2}\mid X_{i},T_{i}=1]\right|T_{i}=1\right] \\
&=\mathbb{E}[\{1-q(X_{i})\}|\widehat{m}_{1}(0,X_{i})-m_{1}(0,X_{i})|^{2} \\
&\quad+q(X_{i})|\widehat{m}_{1}(1,X_{i})-m_{1}(1,X_{i})|^{2}\mid T_{i}=1].
\end{aligned} E [ ∣ m 1 ( W i , X i ) − m 1 ( W i , X i ) ∣ 2 ∣ T i = 1 ] = E [ E [ ∣ m 1 ( W i , X i ) − m 1 ( W i , X i ) ∣ 2 ∣ X i , T i = 1 ] T i = 1 ] = E [{ 1 − q ( X i )} ∣ m 1 ( 0 , X i ) − m 1 ( 0 , X i ) ∣ 2 + q ( X i ) ∣ m 1 ( 1 , X i ) − m 1 ( 1 , X i ) ∣ 2 ∣ T i = 1 ] .
Using the covariate-moment bound at the beginning of the proof, we obtain
E [ ∣ m ^ 1 ( W i , X i ) − m 1 ( W i , X i ) ∣ 2 ∣ T i = 1 ] ≤ 1 c 2 E [ { 1 − q ( X i ) } ∣ m ^ 1 ( 0 , X i ) − m 1 ( 0 , X i ) ∣ 2 + q ( X i ) ∣ m ^ 1 ( 1 , X i ) − m 1 ( 1 , X i ) ∣ 2 ] ≤ 1 c 2 E [ ∣ m ^ 1 ( 0 , X i ) − m 1 ( 0 , X i ) ∣ 2 + ∣ m ^ 1 ( 1 , X i ) − m 1 ( 1 , X i ) ∣ 2 ] = 1 c 2 ∑ w = 0 1 E [ ∣ m ^ 1 ( w , X i ) − m 1 ( w , X i ) ∣ 2 ] ≤ 2 c 2 n − 1 / 2 δ n . \begin{aligned}
\mathbb{E}[|\widehat{m}_{1}(W_{i},X_{i})-m_{1}(W_{i},X_{i})|^{2}\mid T_{i}=1]
&\leq\frac{1}{c_{2}}\mathbb{E}[\{1-q(X_{i})\}|\widehat{m}_{1}(0,X_{i})-m_{1}(0,X_{i})|^{2} \\
&\quad+q(X_{i})|\widehat{m}_{1}(1,X_{i})-m_{1}(1,X_{i})|^{2}] \\
&\leq\frac{1}{c_{2}}\mathbb{E}[|\widehat{m}_{1}(0,X_{i})-m_{1}(0,X_{i})|^{2} \\
&\quad+|\widehat{m}_{1}(1,X_{i})-m_{1}(1,X_{i})|^{2}] \\
&=\frac{1}{c_{2}}\sum_{w=0}^{1}\mathbb{E}[|\widehat{m}_{1}(w,X_{i})-m_{1}(w,X_{i})|^{2}] \\
&\le\frac{2}{c_{2}}n^{-1/2}\delta_{n}.
\end{aligned} E [ ∣ m 1 ( W i , X i ) − m 1 ( W i , X i ) ∣ 2 ∣ T i = 1 ] ≤ c 2 1 E [{ 1 − q ( X i )} ∣ m 1 ( 0 , X i ) − m 1 ( 0 , X i ) ∣ 2 + q ( X i ) ∣ m 1 ( 1 , X i ) − m 1 ( 1 , X i ) ∣ 2 ] ≤ c 2 1 E [ ∣ m 1 ( 0 , X i ) − m 1 ( 0 , X i ) ∣ 2 + ∣ m 1 ( 1 , X i ) − m 1 ( 1 , X i ) ∣ 2 ] = c 2 1 w = 0 ∑ 1 E [ ∣ m 1 ( w , X i ) − m 1 ( w , X i ) ∣ 2 ] ≤ c 2 2 n − 1/2 δ n .
The second inequality uses 0 ≤ q ( X i ) ≤ 1 0 \le q(X_i) \le1 0 ≤ q ( X i ) ≤ 1 , and the last inequality uses Assumption 7(c). Taking square roots yields
E [ ∣ m ^ 1 ( W i , X i ) − m 1 ( W i , X i ) ∣ 2 ∣ T i = 1 ] 1 / 2 ≤ 2 c 2 n − 1 / 4 δ n . \mathbb{E}\left[\left|\widehat{m}_{1}(W_i,X_i)-m_{1}(W_i,X_i)\right|^2 \mid T_i=1\right]^{1/2} \le\sqrt{\frac{2}{c_{2}}}n^{-1/4}\delta_n. E [ ∣ m 1 ( W i , X i ) − m 1 ( W i , X i ) ∣ 2 ∣ T i = 1 ] 1/2 ≤ c 2 2 n − 1/4 δ n .
Applying the triangle inequality to Equation (43) gives
∣ E [ H i Q ] ∣ ≤ Pr ( T i = 1 ) ∣ E [ 1 − 2 W i π 1 ( X i ) ( S ^ 1 i − S 1 i ) ( S ^ 2 i − S 2 i ) ∣ T i = 1 ] ∣ + Pr ( T i = 1 ) ∣ E [ m ^ 1 ( W i , X i ) − m 1 ( W i , X i ) π 1 ( X i ) { m ^ 1 ( 1 , X i ) − m ^ 1 ( 0 , X i ) } ( A ^ i − A i ) ∣ T i = 1 ] ∣ ≤ 1 c 2 E [ ∣ 1 − 2 W i ∣ ∣ S ^ 1 i − S 1 i ∣ ∣ S ^ 2 i − S 2 i ∣ ∣ T i = 1 ] + 1 c 2 E [ ∣ m ^ 1 ( W i , X i ) − m 1 ( W i , X i ) ∣ ∣ m ^ 1 ( 1 , X i ) − m ^ 1 ( 0 , X i ) ∣ ∣ A ^ i − A i ∣ ∣ T i = 1 ] . \begin{aligned}
\left|\mathbb{E}[H_i^Q]\right|
&\le\Pr(T_i=1)\left|\mathbb{E}\left[\frac{1-2W_i}{\pi_1(X_i)}(\widehat{S}_{1i}-S_{1i})(\widehat{S}_{2i}-S_{2i})\mid T_i=1\right]\right|\\
&\quad+\Pr(T_i=1)\left|\mathbb{E}\left[\frac{\widehat{m}_1(W_i,X_i)-m_1(W_i,X_i)}{\pi_1(X_i)\{\widehat{m}_1(1,X_i)-\widehat{m}_1(0,X_i)\}}(\widehat{A}_i-A_i)\mid T_i=1\right]\right|\\
&\le\frac{1}{c_2}\mathbb{E}\left[\left|1-2W_i\right|\left|\widehat{S}_{1i}-S_{1i}\right|\left|\widehat{S}_{2i}-S_{2i}\right|\mid T_i=1\right]\\
&\quad+\frac{1}{c_2}\mathbb{E}\left[\frac{\left|\widehat{m}_1(W_i,X_i)-m_1(W_i,X_i)\right|}{\left|\widehat{m}_1(1,X_i)-\widehat{m}_1(0,X_i)\right|}\left|\widehat{A}_i-A_i\right|\mid T_i=1\right].
\end{aligned} E [ H i Q ] ≤ Pr ( T i = 1 ) E [ π 1 ( X i ) 1 − 2 W i ( S 1 i − S 1 i ) ( S 2 i − S 2 i ) ∣ T i = 1 ] + Pr ( T i = 1 ) E [ π 1 ( X i ) { m 1 ( 1 , X i ) − m 1 ( 0 , X i )} m 1 ( W i , X i ) − m 1 ( W i , X i ) ( A i − A i ) ∣ T i = 1 ] ≤ c 2 1 E [ ∣ 1 − 2 W i ∣ S 1 i − S 1 i S 2 i − S 2 i ∣ T i = 1 ] + c 2 1 E [ ∣ m 1 ( 1 , X i ) − m 1 ( 0 , X i ) ∣ ∣ m 1 ( W i , X i ) − m 1 ( W i , X i ) ∣ A i − A i ∣ T i = 1 ] .
The second inequality uses Pr ( T i = 1 ) ≤ 1 \Pr(T_i=1)\le1 Pr ( T i = 1 ) ≤ 1 and π 1 ( X i ) ≥ c 2 \pi_1(X_i)\ge c_2 π 1 ( X i ) ≥ c 2 . Since ∣ 1 − 2 W i ∣ = 1 \left|1-2W_i\right|=1 ∣ 1 − 2 W i ∣ = 1 , the first expectation becomes the expectation of the product of the two surrogate errors.
∣ E [ H i Q ] ∣ ≤ 1 c 2 E [ ∣ S ^ 1 i − S 1 i ∣ ∣ S ^ 2 i − S 2 i ∣ ∣ T i = 1 ] + 1 c 2 E [ ∣ m ^ 1 ( W i , X i ) − m 1 ( W i , X i ) ∣ ∣ m ^ 1 ( 1 , X i ) − m ^ 1 ( 0 , X i ) ∣ ∣ A ^ i − A i ∣ ∣ T i = 1 ] . \left|\mathbb{E}[H_i^Q]\right|\le\frac{1}{c_2}\mathbb{E}\left[\left|\widehat{S}_{1i}-S_{1i}\right|\left|\widehat{S}_{2i}-S_{2i}\right|\mid T_i=1\right]
+\frac{1}{c_2}\mathbb{E}\left[\frac{\left|\widehat{m}_1(W_i,X_i)-m_1(W_i,X_i)\right|}{\left|\widehat{m}_1(1,X_i)-\widehat{m}_1(0,X_i)\right|}\left|\widehat{A}_i-A_i\right|\mid T_i=1\right]. E [ H i Q ] ≤ c 2 1 E [ S 1 i − S 1 i S 2 i − S 2 i ∣ T i = 1 ] + c 2 1 E [ ∣ m 1 ( 1 , X i ) − m 1 ( 0 , X i ) ∣ ∣ m 1 ( W i , X i ) − m 1 ( W i , X i ) ∣ A i − A i ∣ T i = 1 ] .
The lower bound on the fitted third-check gap then gives
∣ E [ H i Q ] ∣ ≤ 1 c 2 E [ ∣ S ^ 1 i − S 1 i ∣ ∣ S ^ 2 i − S 2 i ∣ ∣ T i = 1 ] + 1 c 2 2 E [ ∣ m ^ 1 ( W i , X i ) − m 1 ( W i , X i ) ∣ ∣ A ^ i − A i ∣ ∣ T i = 1 ] . \left|\mathbb{E}[H_i^Q]\right|\le\frac{1}{c_2}\mathbb{E}\left[\left|\widehat{S}_{1i}-S_{1i}\right|\left|\widehat{S}_{2i}-S_{2i}\right|\mid T_i=1\right]
+\frac{1}{c_2^2}\mathbb{E}\left[\left|\widehat{m}_1(W_i,X_i)-m_1(W_i,X_i)\right|\left|\widehat{A}_i-A_i\right|\mid T_i=1\right]. E [ H i Q ] ≤ c 2 1 E [ S 1 i − S 1 i S 2 i − S 2 i ∣ T i = 1 ] + c 2 2 1 E [ ∣ m 1 ( W i , X i ) − m 1 ( W i , X i ) ∣ A i − A i ∣ T i = 1 ] .
Applying the Cauchy–Schwarz inequality separately to the two products yields
∣ E [ H i Q ] ∣ ≤ 1 c 2 E [ ∣ S ^ 1 i − S 1 i ∣ 2 ∣ T i = 1 ] 1 / 2 E [ ∣ S ^ 2 i − S 2 i ∣ 2 ∣ T i = 1 ] 1 / 2 + 1 c 2 2 E [ ∣ m ^ 1 ( W i , X i ) − m 1 ( W i , X i ) ∣ 2 ∣ T i = 1 ] 1 / 2 × E [ ∣ A ^ i − A i ∣ 2 ∣ T i = 1 ] 1 / 2 . \begin{aligned}
\left|\mathbb{E}[H_i^Q]\right|
&\le\frac{1}{c_2}\mathbb{E}\left[\left|\widehat{S}_{1i}-S_{1i}\right|^2\mid T_i=1\right]^{1/2}\mathbb{E}\left[\left|\widehat{S}_{2i}-S_{2i}\right|^2\mid T_i=1\right]^{1/2}\\
&\quad+\frac{1}{c_2^2}\mathbb{E}\left[\left|\widehat{m}_1(W_i,X_i)-m_1(W_i,X_i)\right|^2\mid T_i=1\right]^{1/2}\\
&\qquad\qquad\times\mathbb{E}\left[\left|\widehat{A}_i-A_i\right|^2\mid T_i=1\right]^{1/2}.
\end{aligned} E [ H i Q ] ≤ c 2 1 E [ S 1 i − S 1 i 2 ∣ T i = 1 ] 1/2 E [ S 2 i − S 2 i 2 ∣ T i = 1 ] 1/2 + c 2 2 1 E [ ∣ m 1 ( W i , X i ) − m 1 ( W i , X i ) ∣ 2 ∣ T i = 1 ] 1/2 × E [ A i − A i 2 ∣ T i = 1 ] 1/2 .
First inserting Equation (31) , then the bound for the third-check mean, and finally Equation (33) , we obtain
∣ E [ H i Q ] ∣ ≤ 1 c 2 { 1 + c 1 2 c 2 n − 1 / 4 δ n } 2 + 1 c 2 2 E [ ∣ m ^ 1 ( W i , X i ) − m 1 ( W i , X i ) ∣ 2 ∣ T i = 1 ] 1 / 2 E [ ∣ A ^ i − A i ∣ 2 ∣ T i = 1 ] 1 / 2 ≤ 1 c 2 { 1 + c 1 2 c 2 n − 1 / 4 δ n } 2 + 2 / c 2 c 2 2 n − 1 / 4 δ n E [ ∣ A ^ i − A i ∣ 2 ∣ T i = 1 ] 1 / 2 . ≤ 1 c 2 { 1 + c 1 2 c 2 n − 1 / 4 δ n } 2 + 2 / c 2 c 2 2 n − 1 / 4 δ n ( c n − 1 / 4 δ n ) = ( 1 + c 1 2 c 2 2 + 2 / c 2 c c 2 2 ) n − 1 / 2 δ n 2 ≤ c n − 1 / 2 δ n 2 = o ( n − 1 / 2 ) . (44) \begin{aligned}
\left|\mathbb{E}[H_i^Q]\right|
&\le\frac{1}{c_2}\left\{\sqrt{\frac{1+c_1^2}{c_2}}n^{-1/4}\delta_n\right\}^2
+\frac{1}{c_2^2}\mathbb{E}\left[\left|\widehat{m}_1(W_i,X_i)-m_1(W_i,X_i)\right|^2\mid T_i=1\right]^{1/2}\mathbb{E}\left[\left|\widehat{A}_i-A_i\right|^2\mid T_i=1\right]^{1/2}\\
&\le\frac{1}{c_2}\left\{\sqrt{\frac{1+c_1^2}{c_2}}n^{-1/4}\delta_n\right\}^2
+\frac{\sqrt{2/c_2}}{c_2^2}n^{-1/4}\delta_n\mathbb{E}\left[\left|\widehat{A}_i-A_i\right|^2\mid T_i=1\right]^{1/2}. \\
&\le\frac{1}{c_2}\left\{\sqrt{\frac{1+c_1^2}{c_2}}n^{-1/4}\delta_n\right\}^2+\frac{\sqrt{2/c_2}}{c_2^2}n^{-1/4}\delta_n(cn^{-1/4}\delta_n) \\
&=\left(\frac{1+c_1^2}{c_2^2}+\frac{\sqrt{2/c_2}\,c}{c_2^2}\right)n^{-1/2}\delta_n^2 \\
&\le cn^{-1/2}\delta_n^2 \\
&=o(n^{-1/2}). \tag*{(44)}
\end{aligned} E [ H i Q ] ≤ c 2 1 ⎩ ⎨ ⎧ c 2 1 + c 1 2 n − 1/4 δ n ⎭ ⎬ ⎫ 2 + c 2 2 1 E [ ∣ m 1 ( W i , X i ) − m 1 ( W i , X i ) ∣ 2 ∣ T i = 1 ] 1/2 E [ A i − A i 2 ∣ T i = 1 ] 1/2 ≤ c 2 1 ⎩ ⎨ ⎧ c 2 1 + c 1 2 n − 1/4 δ n ⎭ ⎬ ⎫ 2 + c 2 2 2/ c 2 n − 1/4 δ n E [ A i − A i 2 ∣ T i = 1 ] 1/2 . ≤ c 2 1 ⎩ ⎨ ⎧ c 2 1 + c 1 2 n − 1/4 δ n ⎭ ⎬ ⎫ 2 + c 2 2 2/ c 2 n − 1/4 δ n ( c n − 1/4 δ n ) = ( c 2 2 1 + c 1 2 + c 2 2 2/ c 2 c ) n − 1/2 δ n 2 ≤ c n − 1/2 δ n 2 = o ( n − 1/2 ) . (44)
The last equality uses δ n → 0 \delta_n \to0 δ n → 0 . These bounds hold on the training-sample event in Assumption 7(c). Since that event has probability tending to one, each fold’s conditional population remainder is o p ( n − 1 / 2 ) o_p(n^{-1/2}) o p ( n − 1/2 ) . With a fixed number of folds,
∑ k = 1 K n k n E [ H i Q , ( − k ) ] = o p ( n − 1 / 2 ) . \sum_{k=1}^{K}\frac{n_k}{n}\mathbb{E}[H_i^{Q,(-k)}]=o_p(n^{-1/2}). k = 1 ∑ K n n k E [ H i Q , ( − k ) ] = o p ( n − 1/2 ) .
STEP 3: Applying CLT Substituting the centered empirical bound in Equation (38) into Equation (30) first gives
Q ^ − Q = 1 n ∑ i = 1 n ( ϕ i Q − Q ) + o p ( n − 1 / 2 ) + ∑ k = 1 K n k n E [ H i Q , ( − k ) ] . \widehat{Q}-Q=\frac{1}{n}\sum_{i=1}^{n}(\phi_i^Q-Q)+o_p(n^{-1/2})+\sum_{k=1}^{K}\frac{n_k}{n}\mathbb{E}[H_i^{Q,(-k)}]. Q − Q = n 1 i = 1 ∑ n ( ϕ i Q − Q ) + o p ( n − 1/2 ) + k = 1 ∑ K n n k E [ H i Q , ( − k ) ] .
Substituting the population remainder bound then yields
Q ^ − Q = 1 n ∑ i = 1 n ( ϕ i Q − Q ) + o p ( n − 1 / 2 ) + o p ( n − 1 / 2 ) = 1 n ∑ i = 1 n ( ϕ i Q − Q ) + o p ( n − 1 / 2 ) . (45) \widehat{Q}-Q=\frac{1}{n}\sum_{i=1}^{n}(\phi_i^Q-Q)+o_p(n^{-1/2})+o_p(n^{-1/2})=\frac{1}{n}\sum_{i=1}^{n}(\phi_i^Q-Q)+o_p(n^{-1/2}). \tag*{(45)} Q − Q = n 1 i = 1 ∑ n ( ϕ i Q − Q ) + o p ( n − 1/2 ) + o p ( n − 1/2 ) = n 1 i = 1 ∑ n ( ϕ i Q − Q ) + o p ( n − 1/2 ) . (45)
Finally, we consider the ratio estimator. Proposition 2 proves that the uncentered influence function satisfies
E [ ϕ t i − Ψ t ] = 0 , t = 0 , 1 , \mathbb{E}[\phi_{ti}-\Psi_t]=0,\qquad t=0,1, E [ ϕ t i − Ψ t ] = 0 , t = 0 , 1 ,
E [ ϕ i Q − Q ] = 0. \mathbb{E}[\phi_i^Q-Q]=0. E [ ϕ i Q − Q ] = 0.
The outcomes and checks are bounded by Assumption 7(a). The denominators in the calibrated surrogates and correction term are bounded away from zero by Assumption 7(b). Consequently, these surrogates and the correction term are bounded. Since π t ( X i ) ≥ c 2 \pi_t(X_i)\ge c_2 π t ( X i ) ≥ c 2 for t = 0 , 1 t=0,1 t = 0 , 1 by Assumption 7(b), the component influence functions have finite second moments. The central limit theorem therefore implies
1 n ∑ i = 1 n ( ϕ t i − Ψ t ) = O p ( 1 ) , t = 0 , 1 , \frac{1}{\sqrt{n}}\sum_{i=1}^{n}(\phi_{ti}-\Psi_t)=O_p(1),\qquad t=0,1, n 1 i = 1 ∑ n ( ϕ t i − Ψ t ) = O p ( 1 ) , t = 0 , 1 ,
1 n ∑ i = 1 n ( ϕ i Q − Q ) = O p ( 1 ) . \frac{1}{\sqrt{n}}\sum_{i=1}^{n}(\phi_i^Q-Q)=O_p(1). n 1 i = 1 ∑ n ( ϕ i Q − Q ) = O p ( 1 ) .
Equation (29) then gives Ψ ^ t − Ψ t = O p ( n − 1 / 2 ) \widehat{\Psi}_t-\Psi_t=O_p(n^{-1/2}) Ψ t − Ψ t = O p ( n − 1/2 ) for t = 0 , 1 t=0,1 t = 0 , 1 . Equation (45) gives Q ^ − Q = O p ( n − 1 / 2 ) \widehat{Q}-Q=O_p(n^{-1/2}) Q − Q = O p ( n − 1/2 ) . Assumption 7(b) also implies
Q = E [ q ( X i ) ] ≥ c 2 . Q=\mathbb{E}[q(X_i)]\ge c_2. Q = E [ q ( X i )] ≥ c 2 .
Thus, Pr ( Q ^ ≥ Q / 2 ) → 1 \Pr(\widehat{Q} \ge Q/2) \to1 Pr ( Q ≥ Q /2 ) → 1 . On this event,
1 Q ^ − 1 Q = Q − Q ^ Q Q ^ = − Q ^ − Q Q Q ^ . \frac{1}{\widehat{Q}}-\frac{1}{Q}=\frac{Q-\widehat{Q}}{Q\widehat{Q}}=-\frac{\widehat{Q}-Q}{Q\widehat{Q}}. Q 1 − Q 1 = Q Q Q − Q = − Q Q Q − Q .
The denominator is bounded away from zero on this event, so
1 Q ^ − 1 Q = O p ( n − 1 / 2 ) . \frac{1}{\widehat{Q}}-\frac{1}{Q}=O_p(n^{-1/2}). Q 1 − Q 1 = O p ( n − 1/2 ) .
Using the definition of Ψ ^ \widehat{\Psi} Ψ , we obtain
Ψ ^ − Ψ = Ψ ^ 1 − Ψ ^ 0 Q ^ − Ψ = Ψ ^ 1 − Ψ ^ 0 − Ψ Q ^ Q ^ = ( Ψ ^ 1 − Ψ 1 ) − ( Ψ ^ 0 − Ψ 0 ) − Ψ ( Q ^ − Q ) + ( Ψ 1 − Ψ 0 − Ψ Q ) Q ^ = ( Ψ ^ 1 − Ψ 1 ) − ( Ψ ^ 0 − Ψ 0 ) − Ψ ( Q ^ − Q ) Q ^ . \begin{aligned}
\widehat{\Psi}-\Psi
&=\frac{\widehat{\Psi}_1-\widehat{\Psi}_0}{\widehat{Q}}-\Psi\\
&=\frac{\widehat{\Psi}_1-\widehat{\Psi}_0-\Psi\widehat{Q}}{\widehat{Q}}\\
&=\frac{(\widehat{\Psi}_1-\Psi_1)-(\widehat{\Psi}_0-\Psi_0)-\Psi(\widehat{Q}-Q)+(\Psi_1-\Psi_0-\Psi Q)}{\widehat{Q}}\\
&=\frac{(\widehat{\Psi}_1-\Psi_1)-(\widehat{\Psi}_0-\Psi_0)-\Psi(\widehat{Q}-Q)}{\widehat{Q}}.
\end{aligned} Ψ − Ψ = Q Ψ 1 − Ψ 0 − Ψ = Q Ψ 1 − Ψ 0 − Ψ Q = Q ( Ψ 1 − Ψ 1 ) − ( Ψ 0 − Ψ 0 ) − Ψ ( Q − Q ) + ( Ψ 1 − Ψ 0 − Ψ Q ) = Q ( Ψ 1 − Ψ 1 ) − ( Ψ 0 − Ψ 0 ) − Ψ ( Q − Q ) .
The last equality uses Ψ 1 − Ψ 0 = Ψ Q \Psi_1-\Psi_0=\Psi Q Ψ 1 − Ψ 0 = Ψ Q . Substituting (29) first gives
n ( Ψ ^ − Ψ ) = 1 Q ^ { 1 n ∑ i = 1 n [ ( ϕ 1 i − Ψ 1 ) − ( ϕ 0 i − Ψ 0 ) ] − Ψ n ( Q ^ − Q ) } + o p ( 1 ) . \sqrt{n}(\widehat{\Psi}-\Psi)=\frac{1}{\widehat{Q}}\left\{\frac{1}{\sqrt{n}}\sum_{i=1}^{n}\left[(\phi_{1i}-\Psi_1)-(\phi_{0i}-\Psi_0)\right]-\Psi\sqrt{n}(\widehat{Q}-Q)\right\}+o_p(1). n ( Ψ − Ψ ) = Q 1 { n 1 i = 1 ∑ n [ ( ϕ 1 i − Ψ 1 ) − ( ϕ 0 i − Ψ 0 ) ] − Ψ n ( Q − Q ) } + o p ( 1 ) .
Here, 1 / Q ^ = O p ( 1 ) 1/\widehat{Q}=O_p(1) 1/ Q = O p ( 1 ) preserves the order of the remainder. Next, substituting (45) yields
n ( Ψ ^ − Ψ ) = 1 Q ^ n ∑ i = 1 n [ ( ϕ 1 i − Ψ 1 ) − ( ϕ 0 i − Ψ 0 ) − Ψ ( ϕ i Q − Q ) ] + o p ( 1 ) = 1 Q ^ n ∑ i = 1 n [ ϕ 1 i − ϕ 0 i − Ψ ϕ i Q − ( Ψ 1 − Ψ 0 − Ψ Q ) ] + o p ( 1 ) = 1 Q ^ n ∑ i = 1 n ( ϕ 1 i − ϕ 0 i − Ψ ϕ i Q ) + o p ( 1 ) . \begin{aligned}
\sqrt{n}(\widehat{\Psi}-\Psi)
&=\frac{1}{\widehat{Q}\sqrt{n}}\sum_{i=1}^{n}\left[(\phi_{1i}-\Psi_1)-(\phi_{0i}-\Psi_0)-\Psi(\phi_i^Q-Q)\right]+o_p(1)\\
&=\frac{1}{\widehat{Q}\sqrt{n}}\sum_{i=1}^{n}\left[\phi_{1i}-\phi_{0i}-\Psi\phi_i^Q-(\Psi_1-\Psi_0-\Psi Q)\right]+o_p(1)\\
&=\frac{1}{\widehat{Q}\sqrt{n}}\sum_{i=1}^{n}\left(\phi_{1i}-\phi_{0i}-\Psi\phi_i^Q\right)+o_p(1).
\end{aligned} n ( Ψ − Ψ ) = Q n 1 i = 1 ∑ n [ ( ϕ 1 i − Ψ 1 ) − ( ϕ 0 i − Ψ 0 ) − Ψ ( ϕ i Q − Q ) ] + o p ( 1 ) = Q n 1 i = 1 ∑ n [ ϕ 1 i − ϕ 0 i − Ψ ϕ i Q − ( Ψ 1 − Ψ 0 − Ψ Q ) ] + o p ( 1 ) = Q n 1 i = 1 ∑ n ( ϕ 1 i − ϕ 0 i − Ψ ϕ i Q ) + o p ( 1 ) .
The last equality again uses Ψ 1 − Ψ 0 = Ψ Q \Psi_1-\Psi_0=\Psi Q Ψ 1 − Ψ 0 = Ψ Q . Adding and subtracting the same sum divided by Q Q Q gives
n ( Ψ ^ − Ψ ) = 1 Q n ∑ i = 1 n ( ϕ 1 i − ϕ 0 i − Ψ ϕ i Q ) + ( 1 Q ^ − 1 Q ) 1 n ∑ i = 1 n ( ϕ 1 i − ϕ 0 i − Ψ ϕ i Q ) + o p ( 1 ) . \sqrt{n}(\widehat{\Psi}-\Psi)=\frac{1}{Q\sqrt{n}}\sum_{i=1}^{n}\left(\phi_{1i}-\phi_{0i}-\Psi\phi_i^Q\right)+\left(\frac{1}{\widehat{Q}}-\frac{1}{Q}\right)\frac{1}{\sqrt{n}}\sum_{i=1}^{n}\left(\phi_{1i}-\phi_{0i}-\Psi\phi_i^Q\right)+o_p(1). n ( Ψ − Ψ ) = Q n 1 i = 1 ∑ n ( ϕ 1 i − ϕ 0 i − Ψ ϕ i Q ) + ( Q 1 − Q 1 ) n 1 i = 1 ∑ n ( ϕ 1 i − ϕ 0 i − Ψ ϕ i Q ) + o p ( 1 ) .
The first factor in the product is O p ( n − 1 / 2 ) O_p(n^{-1/2}) O p ( n − 1/2 ) . For the second factor, the identity
1 n ∑ i = 1 n ( ϕ 1 i − ϕ 0 i − Ψ ϕ i Q ) = 1 n ∑ i = 1 n [ ( ϕ 1 i − Ψ 1 ) − ( ϕ 0 i − Ψ 0 ) − Ψ ( ϕ i Q − Q ) ] \frac{1}{\sqrt{n}}\sum_{i=1}^{n}\left(\phi_{1i}-\phi_{0i}-\Psi\phi_i^Q\right)=\frac{1}{\sqrt{n}}\sum_{i=1}^{n}\left[(\phi_{1i}-\Psi_1)-(\phi_{0i}-\Psi_0)-\Psi(\phi_i^Q-Q)\right] n 1 i = 1 ∑ n ( ϕ 1 i − ϕ 0 i − Ψ ϕ i Q ) = n 1 i = 1 ∑ n [ ( ϕ 1 i − Ψ 1 ) − ( ϕ 0 i − Ψ 0 ) − Ψ ( ϕ i Q − Q ) ]
shows that it is O p ( 1 ) O_p(1) O p ( 1 ) . Their product is therefore O p ( n − 1 / 2 ) = o p ( 1 ) O_p(n^{-1/2}) = o_p(1) O p ( n − 1/2 ) = o p ( 1 ) . It follows that
n ( Ψ ^ − Ψ ) = 1 Q n ∑ i = 1 n ( ϕ 1 i − ϕ 0 i − Ψ ϕ i Q ) + o p ( 1 ) = 1 n ∑ i = 1 n ϕ i + o p ( 1 ) . \sqrt{n}(\widehat{\Psi} - \Psi) = \frac{1}{Q\sqrt{n}}\sum_{i=1}^{n}(\phi_{1i} - \phi_{0i} - \Psi\phi_i^Q) + o_p(1) = \frac{1}{\sqrt{n}}\sum_{i=1}^{n}\phi_i + o_p(1). n ( Ψ − Ψ ) = Q n 1 i = 1 ∑ n ( ϕ 1 i − ϕ 0 i − Ψ ϕ i Q ) + o p ( 1 ) = n 1 i = 1 ∑ n ϕ i + o p ( 1 ) .
where the second equality uses (15) .
Finally,
E [ ϕ i ] = E [ ϕ 1 i ] − E [ ϕ 0 i ] − Ψ E [ ϕ i Q ] Q = Ψ 1 − Ψ 0 − Ψ Q Q = 0. \mathbb{E}[\phi_i] = \frac{\mathbb{E}[\phi_{1i}] - \mathbb{E}[\phi_{0i}] - \Psi\mathbb{E}[\phi_i^Q]}{Q} = \frac{\Psi_1 - \Psi_0 - \Psi Q}{Q} = 0. E [ ϕ i ] = Q E [ ϕ 1 i ] − E [ ϕ 0 i ] − Ψ E [ ϕ i Q ] = Q Ψ 1 − Ψ 0 − Ψ Q = 0.
The observations are independent and identically distributed, so the ϕ i \phi_i ϕ i are also independent and identically distributed. Assumption 7(d) gives 0 < E [ ϕ i 2 ] < ∞ 0 < \mathbb{E}[\phi_i^2] < \infty 0 < E [ ϕ i 2 ] < ∞ . The central limit theorem therefore yields
n ( Ψ ^ − Ψ ) = 1 n ∑ i = 1 n ϕ i + o p ( 1 ) ⟶ d N ( 0 , E [ ϕ i 2 ] ) . \sqrt{n}(\widehat{\Psi} - \Psi) = \frac{1}{\sqrt{n}}\sum_{i=1}^{n}\phi_i + o_p(1) \mathrel{\overset{d}{\longrightarrow}} \mathcal{N}(0,\mathbb{E}[\phi_i^2]). n ( Ψ − Ψ ) = n 1 i = 1 ∑ n ϕ i + o p ( 1 ) ⟶ d N ( 0 , E [ ϕ i 2 ]) .
Proof of Proposition 4
Proof. Now, since W i W_i W i is binary, we can write
E [ W ~ j i ∣ W i , T i = t , X i = x ] = E [ W ~ j i ∣ W i = 0 , T i = t , X i = x ] + W i d t j ( x ) . (46) \mathbb{E}[\widetilde{W}_{ji} \mid W_i,T_i=t,X_i=x] = \mathbb{E}[\widetilde{W}_{ji} \mid W_i=0,T_i=t,X_i=x] + W_i d_t^j(x). \tag*{(46)} E [ W j i ∣ W i , T i = t , X i = x ] = E [ W j i ∣ W i = 0 , T i = t , X i = x ] + W i d t j ( x ) . (46)
By the conditional law of total covariance, we have
Cov ( W ~ j i , W ~ j ′ , i ∣ T i = t , X i = x ) = E [ Cov ( W ~ j i , W ~ j ′ , i ∣ W i , T i = t , X i = x ) ∣ T i = t , X i = x ] + Cov ( E [ W ~ j i ∣ W i , T i = t , X i = x ] , E [ W ~ j ′ , i ∣ W i , T i = t , X i = x ] ∣ T i = t , X i = x ) . \begin{aligned}
\operatorname{Cov}(\widetilde{W}_{ji},\widetilde{W}_{j',i}\mid T_i=t,X_i=x)
={}&\mathbb{E}[\operatorname{Cov}(\widetilde{W}_{ji},\widetilde{W}_{j',i}\mid W_i,T_i=t,X_i=x)\mid T_i=t,X_i=x]\\
&+\operatorname{Cov}\bigl(\mathbb{E}[\widetilde{W}_{ji}\mid W_i,T_i=t,X_i=x],\mathbb{E}[\widetilde{W}_{j',i}\mid W_i,T_i=t,X_i=x]\mid T_i=t,X_i=x\bigr).
\end{aligned} Cov ( W j i , W j ′ , i ∣ T i = t , X i = x ) = E [ Cov ( W j i , W j ′ , i ∣ W i , T i = t , X i = x ) ∣ T i = t , X i = x ] + Cov ( E [ W j i ∣ W i , T i = t , X i = x ] , E [ W j ′ , i ∣ W i , T i = t , X i = x ] ∣ T i = t , X i = x ) .
The first term is zero by conditional independence of the two checks. We then apply Equation (46) to each check in the second term. Conditional on ( t , x ) (t,x) ( t , x ) , the intercepts and the two separations are constants, leaving
Cov ( W ~ j i , W ~ j ′ , i ∣ T i = t , X i = x ) = Cov ( W i d t j ( x ) , W i d t j ′ ( x ) ∣ T i = t , X i = x ) = d t j ( x ) d t j ′ ( x ) V ( W i ∣ T i = t , X i = x ) . \begin{aligned}
\operatorname{Cov}(\widetilde{W}_{ji},\widetilde{W}_{j',i}\mid T_i=t,X_i=x)
&=\operatorname{Cov}\bigl(W_i d_t^j(x),W_i d_t^{j'}(x)\mid T_i=t,X_i=x\bigr)\\
&=d_t^j(x)d_t^{j'}(x)V(W_i\mid T_i=t,X_i=x).
\end{aligned} Cov ( W j i , W j ′ , i ∣ T i = t , X i = x ) = Cov ( W i d t j ( x ) , W i d t j ′ ( x ) ∣ T i = t , X i = x ) = d t j ( x ) d t j ′ ( x ) V ( W i ∣ T i = t , X i = x ) .
This proves Equation (20) for each ( j , j ′ ) ∈ { ( 1 , 2 ) , ( 1 , 3 ) , ( 2 , 3 ) } (j,j')\in\{(1,2),(1,3),(2,3)\} ( j , j ′ ) ∈ {( 1 , 2 ) , ( 1 , 3 ) , ( 2 , 3 )} .