betaFeedback

WorksEconomics, Econometrics and Finance

Design-Based Inference for Fuzzy Difference-in-Differences

Abstract

I study fuzzy difference-in-differences designs from a design-based perspective, in which potential outcomes and treatment decisions are fixed and only the assignment of a policy is random, with heterogeneous and possibly unknown assignment probabilities. The Wald-DiD design estimand equals an assignment-weighted complier effect plus contamination from switchers, from changes in always-takers' effects and from trends in untreated outcomes. I derive the conditions under which the estimand holds a clear causal interpretation. The conditions are weaker than parallel trends in both outcomes and treatments. We propose an Anderson–Rubin-type test with a conservative Neyman variance that remains valid under weak first stages, a sensitivity analysis for the complier effect, and estimators for designs with known assignment probabilities. An application to the Oregon Health Insurance Experiment illustrates the results.

Introduction

Difference-in-differences (DiD) is a popular method for causal inference among applied social scientists. In its canonical format, there are two time periods. In the first period, all units are untreated, and in the second period, some units take treatment and other remain untreated. Under the parallel trends assumption, the DiD estimand identifies the average treatment effect on the treated (ATT). However, many difference-in-differences applications are fuzzy: a policy changes the incentive to take a treatment, but take-up is imperfect. Some units are treated before the policy and start treatment for reasons unrelated to it, while some units are exposed to the policy and remain untreated. The canonical estimator in this setting is the Wald-DiD ratio, which divides the DiD in outcomes by the DiD in treatment rates [14]. de Chaisemartin and D’Haultfœuille [11] showed that, in the conventional sampling-based framework, the Wald-DiD estimand identifies a local average treatment effect only under strong homogeneity restrictions: parallel trends must be combined with stability of treatment effects over time and homogeneity of effects between groups.

A growing literature argues that, in many DiD applications, uncertainty is more naturally attributed to the assignment of the policy than to sampling from a superpopulation [1, 5, 2, 31]. Rambachan and Roth [29] develop a design-based framework for quasi-experiments in which units have heterogeneous and unknown assignment probabilities, so that selection into treatment is permitted, and show how the biases of popular estimators can be expressed as finite-population covariances between assignment probabilities and potential outcomes. This paper brings that perspective to the fuzzy DiD framework.

I consider a two-period panel in which a policy (the assignment) becomes available in the second period. Treatment status is binary and monotone in both assignment and time, which partitions the population into never-takers, always-takers, compliers and switchers (units that adopt treatment in the second period irrespective of assignment).

I derive an exact causal decomposition of the design-based Wald-DiD estimand. It equals an assignment-probability-weighted average of complier effects plus three contamination terms: the effects of treatment on switchers, the change over time in the effects of treatment on always-takers, and trends in untreated outcomes. Each term enters the expression through its finite-population covariance with the assignment probabilities. All three vanish under complete randomization, so in the design-based framework the problems highlighted by de Chaisemartin and D’Haultfœuille [11] arise from the interaction between heterogeneity in assignment probabilities interacting and heterogeneity in effects and trends.

Second, I characterize the conditions under which the Wald-DiD design estimand equals the weighted complier effect: the assignment probabilities must be uncorrelated with a single contamination index that combines untreated trends, switcher effects and the drift in always-takers’ effects. This condition is necessary and sufficient, and it is strictly weaker than the pair of orthogonality conditions (design-based parallel trends for outcomes and for treatment) that would follow from applying existing results directly (Hudson et al., [19]; Rambachan and Roth, [29]). For example, it holds under design-based parallel trends and homogeneous effects even when switchers are concentrated among units likely to be assigned.

Third, I propose an Anderson–Rubin-type test based on a difference in means of the null-adjusted outcome change, studentized by a Neyman-type variance estimator. The test is asymptotically valid under the rejective-sampling design of Rambachan and Roth [29], with a conservative variance estimator whose slack we characterize, and its validity does not depend on the strength of the first stage. Inverting the test gives Fieller-type confidence sets, which are unbounded exactly when the first stage is not significant.

Fourth, I develop a sensitivity analysis for the complier effect. Bounds on the two contamination covariances deliver a closed-form identified set and a confidence set that is again robust to weak identification. Tests of the null of no complier effect depend only on the reduced-form contamination bound, which yields a simple breakdown value. When the assignment probabilities are known, as in stratified experiments and lotteries, the contamination terms are themselves estimable and the complier effect is point identified.

The results are illustrated with the Oregon Health Insurance Experiment [16, 37], in which the right to apply for Medicaid was allocated by lottery and compliance was imperfect: some individuals who were not selected enrolled through other routes, and selection probabilities differed by household size. Because the probabilities are known, the application serves as a benchmark: it measures the contamination that an analyst ignoring the heterogeneity in assignment probabilities would face.

Related literature

The paper contributes to the literature on DiD with heterogeneous effects (de Chaisemartin and D’Haultfœuille, 2018, 2020; Callaway and Sant’Anna, 2021; Goodman-Bacon, 2021; Sun and Abraham, 2021; Borusyak et al., 2024; see de Chaisemartin and D’Haultfœuille, 2023 and Roth et al., 2023 for reviews), and specifically to fuzzy DiD [11]. It also contributes to design-based inference [33, 24, 1, 5, 31, 29] (Li and Ding 2017), in particular for IV [23, 25]. We share the assignment model of Rambachan and Roth [29], who analyse two-stage least squares under strong-instrument asymptotics. Our contributions relative to their work are the fuzzy DiD decomposition and characterization, inference that is robust to weak identification [34, 3], and a sensitivity analysis that bounds the first-stage and reduced-form contamination jointly, in the spirit of Rosenbaum (2002), Manski and Pepper [27], Cinelli and Hazlett [9] and Rambachan and Roth [28].

The rest of the paper is organized as follows: § 2 introduces the framework. § 3 presents the identification results, § 4 inference, and § 5 the sensitivity analysis and the case of known assignment probabilities. § 6 summarizes simulations and § 7 presents the application. Proofs of the main results are in Appendix § 9. The Online Appendix contains the simulation design, additional simulation results and the proof of an auxiliary lemma.

Setup

Potential outcomes and assignment

We observe a balanced panel of NN units, i=1,…,Ni = 1, \ldots, N, in two periods, t∈{0,1}t \in\{0, 1\}. A policy becomes available in period 1. Let Zi∈{0,1}Z_i \in\{0, 1\} indicate whether unit ii is assigned to the policy in period 1; no unit is assigned in period 0. Treatment is binary. We write Di0D_{i0} for treatment status in period 0 and Di1(z)D_{i1}(z) for potential treatment status in period 1 under assignment zz. Potential outcomes are Yit(d)Y_{it}(d), d∈{0,1}d \in\{0, 1\}. This notation, which is closer to the IV literature than that of de Chaisemartin and D’Haultfœuille [11], encodes two maintained restrictions.

Assumption 1 (Exclusion and no anticipation). Assignment affects outcomes only through treatment in both periods, and period-0 treatment and outcomes do not depend on period-1 assignment, so that the observed data are Di0D_{i0}, Yi0=Yi0(Di0)Y_{i0} = Y_{i0}(D_{i0}), Di1=Di1(Zi)D_{i1} = D_{i1}(Z_i) and Yi1=Yi1(Di1(Zi))Y_{i1} = Y_{i1}(D_{i1}(Z_i)).

Let τit=Yit(1)−Yit(0)\tau_{it} = Y_{it}(1) - Y_{it}(0) be the unit-level effect in period tt, Δτi=τi1−τi0\Delta\tau_i = \tau_{i1} - \tau_{i0} its change over time, and δi=Yi1(0)−Yi0(0)\delta_i = Y_{i1}(0) - Y_{i0}(0) the untreated outcome trend. The first differences, as functions of assignment, are

ΔDi(z)=Di1(z)−Di0,ΔYi(Di(z))=Yi1(Di1(z))−Yi0(Di0),z∈{0,1},(1)\Delta D_i(z)=D_{i1}(z)-D_{i0}, \qquad\Delta Y_i(D_i(z))=Y_{i1}(D_{i1}(z))-Y_{i0}(D_{i0}), \qquad z\in\{0,1\}, \tag*{(1)}

and the observed differences are ΔDi=ΔDi(Zi)\Delta D_i=\Delta D_i(Z_i) and ΔYi=ΔYi(Di(Zi))\Delta Y_i=\Delta Y_i(D_i(Z_i)).

Assumption 2 (Monotonicity in assignment and time). For all ii, (a) Di1(1)≥Di1(0)D_{i1}(1)\ge D_{i1}(0); (b) Di1(z)≥Di0D_{i1}(z)\ge D_{i0} for z∈{0,1}z\in\{0,1\}.

Part (a) is the usual IV monotonicity condition [20]. Part (b) rules out treatment discontinuation. Together they leave four treatment paths, which partition the population into the groups in Table 1. In particular, always-takers are exactly the units with Di0=1D_{i0}=1, and their number NA=∑iDi0N_A=\sum_i D_{i0} is observed.

TypeDi0D_{i0}Di1(0)D_{i1}(0)Di1(1)D_{i1}(1)ΔDi(0)\Delta D_i(0)ΔYi(Di(0))\Delta Y_i(D_i(0))ΔYi(Di(1))−ΔYi(Di(0))\Delta Y_i(D_i(1))-\Delta Y_i(D_i(0))
Never-takers N\mathcal{N}0000δi\delta_i0
Always-takers A\mathcal{A}1110δi+Δτi\delta_i+\Delta\tau_i0
Compliers C\mathcal{C}0010δi\delta_iτi1\tau_{i1}
Switchers S\mathcal{S}0111δi+τi1\delta_i+\tau_{i1}0

Table 1. Compliance types under Assumption 2.

Following the design-based approach, we condition on the potential outcomes and treatments FN={Di0,Di1(⋅),Yi0(⋅),Yi1(⋅)}i=1N\mathcal{F}_N=\{D_{i0},D_{i1}(\cdot),Y_{i0}(\cdot),Y_{i1}(\cdot)\}_{i=1}^N, so that the only source of randomness is the assignment vector Z=(Z1,…,ZN)′\mathbf{Z}=(Z_1,\ldots,Z_N)^\prime. We adopt the assignment model of Rambachan and Roth [29].

Assumption 3 (Assignment design). For some fixed N1∈{1,…,N−1}N_1\in\{1,\ldots,N-1\} and unknown pi∈(0,1)p_i\in(0,1), possibly dependent on FN\mathcal{F}_N,

P(Z=z∣FN)∝∏i=1Npizi(1−pi)1−ziif ∑izi=N1,and 0 otherwise.\mathbb{P}(\mathbf{Z}=\mathbf{z}\mid\mathcal{F}_N)\propto\prod_{i=1}^{N}p_i^{z_i}(1-p_i)^{1-z_i}\quad\text{if }\sum_i z_i=N_1,\qquad\text{and }0\text{ otherwise.}

Assumption 3 describes independent Bernoulli assignment conditional on the number of assigned units, also known as rejective or conditional Poisson sampling [18, 8]. We write PR\mathbb{P}_{R}, ER\mathbb{E}_{R} and VR\mathbb{V}_{R} for probabilities, expectations and variances over this distribution, N0=N−N1N_{0}=N-N_{1}, pN=N1/Np_{N}=N_{1}/N, and πi=PR(Zi=1)\pi_{i}=\mathbb{P}_{R}(Z_{i}=1) for the marginal assignment probability, so that ∑iπi=N1\sum_{i}\pi_{i}=N_{1}. Because the pip_{i} may depend on potential outcomes and potential take-up, the design allows rich forms of selection into assignment. Complete randomization is the special case pi≡pp_{i}\equiv p, which implies πi≡pN\pi_{i}\equiv p_{N}. Finite-population covariances are denoted Cov⁡N(xi,yi)=N−1∑i(xi−xˉ)(yi−yˉ)\operatorname{Cov}_{N}(x_{i},y_{i})=N^{-1}\sum_{i}(x_{i}-\bar{x})(y_{i}-\bar{y}).

Estimators and the design-based estimand

For any variable Xi=Xi(Zi)X_{i}=X_{i}(Z_{i}), let τ^X=N1−1∑iZiXi−N0−1∑i(1−Zi)Xi\hat{\tau}_{X}=N_{1}^{-1}\sum_{i}Z_{i}X_{i}-N_{0}^{-1}\sum_{i}(1-Z_{i})X_{i} denote the difference in means. The reduced-form and first-stage DiD estimators are τ^RF=τ^ΔY\hat{\tau}_{\mathrm{RF}}=\hat{\tau}_{\Delta Y} and τ^FS=τ^ΔD\hat{\tau}_{\mathrm{FS}}=\hat{\tau}_{\Delta D}, and the Wald-DiD estimator is β^=τ^RF/τ^FS\hat{\beta}=\hat{\tau}_{\mathrm{RF}}/\hat{\tau}_{\mathrm{FS}}. In a balanced two-period panel, τ^RF\hat{\tau}_{\mathrm{RF}} and τ^FS\hat{\tau}_{\mathrm{FS}} coincide with the two-way fixed effects DiD coefficients, and β^\hat{\beta} coincides with the IV estimator in a regression of ΔYi\Delta Y_{i} on ΔDi\Delta D_{i} instrumented by ZiZ_{i}. Define the weights

bi=πiN1,ai=πiN1−1−πiN0=πi−pNNpN(1−pN).(2)b_{i}=\frac{\pi_{i}}{N_{1}},\qquad a_{i}=\frac{\pi_{i}}{N_{1}}-\frac{1-\pi_{i}}{N_{0}}=\frac{\pi_{i}-p_{N}}{Np_{N}(1-p_{N})}. \tag*{(2)}

which satisfy ∑ibi=1\sum_{i}b_{i}=1 and ∑iai=0\sum_{i}a_{i}=0. Let τRF=ER[τ^RF]\tau_{\mathrm{RF}}=\mathbb{E}_{R}[\hat{\tau}_{\mathrm{RF}}] and τFS=ER[τ^FS]\tau_{\mathrm{FS}}=\mathbb{E}_{R}[\hat{\tau}_{\mathrm{FS}}]. Following Rambachan and Roth [29], the Wald-DiD design estimand is

βR=τRFτFSwhenever τFS≠0.(3)\beta_{R}=\frac{\tau_{\mathrm{RF}}}{\tau_{\mathrm{FS}}}\qquad\text{whenever }\tau_{\mathrm{FS}}\ne0. \tag*{(3)}

The parameter βR\beta_{R} is well defined given FN\mathcal{F}_{N} and the design, and τRF\tau_{\mathrm{RF}} and τFS\tau_{\mathrm{FS}} are unbiasedly estimable. The question is what βR\beta_{R} means. The natural causal benchmark is the assignment-weighted complier effect

βC=∑i∈Cπiτi1∑i∈Cπi=ER[∑i∈CZiτi1]ER[∑i∈CZi].(4)\beta_{C}=\frac{\sum_{i\in C}\pi_{i}\tau_{i1}}{\sum_{i\in C}\pi_{i}}=\frac{\mathbb{E}_{R}\left[\sum_{i\in C}Z_{i}\tau_{i1}\right]}{\mathbb{E}_{R}\left[\sum_{i\in C}Z_{i}\right]}. \tag*{(4)}

the expected average effect among assigned compliers. It is the design analogue of a local average treatment effect on the treated (the LATEπ\mathrm{LATE}_{\pi} of Rambachan and Roth [29]). Under complete randomization it reduces to the complier average effect LATEC=NC−1∑i∈Cτi1\mathrm{LATE}_{C}=N_{C}^{-1}\sum_{i\in\mathcal{C}}\tau_{i1}. Throughout we assume NC≥1N_{C}\ge1.

Identification

We begin with a general lemma that requires neither monotonicity nor exclusion.

Lemma 1 (Design expectation of a difference in means). Under Assumption 3, for any fixed potential responses {Xi(0),Xi(1)}i=1N\{X_i(0),X_i(1)\}_{i=1}^{N},

ER[τ~X]=∑i=1Nbi{Xi(1)−Xi(0)}+∑i=1NaiXi(0),∑i=1NaiXi(0)=Cov⁡N(πi,Xi(0))pN(1−pN).\mathbb{E}_{R}[\widetilde{\tau}_{X}]=\sum_{i=1}^{N}b_i\{X_i(1)-X_i(0)\}+\sum_{i=1}^{N}a_iX_i(0),\qquad\sum_{i=1}^{N}a_iX_i(0)=\frac{\operatorname{Cov}_{N}(\pi_i,X_i(0))}{p_N(1-p_N)}.

The first term is a π\pi-weighted average effect of assignment on XX. The second is a selection term that vanishes when assignment probabilities are uncorrelated with the response without assignment. Lemma 1, and hence Theorem 1 below, uses Assumption 3 only through the facts that ∑iZi=N1\sum_i Z_i=N_1 is fixed and that πi∈(0,1)\pi_i\in(0,1); both results therefore hold for any such design. Applying Lemma 1 to Xi=ΔDiX_i=\Delta D_i and Xi=ΔYiX_i=\Delta Y_i and using Table 1 yields our main result. Define the design contamination terms

RN=∑i=1NaiΔYi(Di(0))=Cov⁡N(πi,ΔYi(Di(0)))pN(1−pN),(5)R_N=\sum_{i=1}^{N}a_i\Delta Y_i(D_i(0))=\frac{\operatorname{Cov}_{N}(\pi_i,\Delta Y_i(D_i(0)))}{p_N(1-p_N)}, \tag*{(5)}
FN=∑i=1NaiΔDi(0)=Cov⁡N(πi,1{i∈S})pN(1−pN),F_N=\sum_{i=1}^{N}a_i\Delta D_i(0)=\frac{\operatorname{Cov}_{N}(\pi_i,1\{i\in\mathcal{S}\})}{p_N(1-p_N)},

and the complier mass BC=∑i∈Cbi>0B_C=\sum_{i\in\mathcal{C}}b_i>0.

Theorem 1 (Decomposition of the Wald-DiD design estimand). Suppose Assumptions 1–3 hold. (a) The first stage and reduced form satisfy τFS=BC+FN\tau_{\mathrm{FS}}=B_C+F_N and τRF=BCβC+RN\tau_{\mathrm{RF}}=B_C\beta_C+R_N,

where

FN=∑i∈Sai,(6)F_N = \sum_{i\in\mathcal{S}} a_i, \tag*{(6)}
RN=∑i∈Saiτi1+∑i∈AaiΔτi+∑i=1Naiδi.(7)R_N = \sum_{i\in\mathcal{S}} a_i\tau_{i1} + \sum_{i\in\mathcal{A}} a_i\Delta\tau_i + \sum_{i=1}^{N} a_i\delta_i. \tag*{(7)}

(b) If τFS≠0\tau_{\mathrm{FS}} \ne0,

βR−βC=RN−βCFNτFS=∑i∈Sai(τi1−βC)+∑i∈AaiΔτi+∑i=1Naiδi∑i∈Cbi+∑i∈Sai.(8)\beta_R-\beta_C = \frac{R_N-\beta_C F_N}{\tau_{\mathrm{FS}}} = \frac{\sum_{i\in\mathcal{S}}a_i(\tau_{i1}-\beta_C)+\sum_{i\in\mathcal{A}}a_i\Delta\tau_i+\sum_{i=1}^{N}a_i\delta_i}{\sum_{i\in\mathcal{C}}b_i+\sum_{i\in\mathcal{S}}a_i}. \tag*{(8)}

(c) Let Γi=ΔYi(Di(0))−βCΔDi(0)=δi+1{i∈A}Δτi+1{i∈S}(τi1−βC)\Gamma_i=\Delta Y_i(D_i(0))-\beta_C\Delta D_i(0)=\delta_i+1\{i\in\mathcal{A}\}\Delta\tau_i+1\{i\in\mathcal{S}\}(\tau_{i1}-\beta_C). If τFS≠0\tau_{\mathrm{FS}}\ne0,

βR=βC\beta_R=\beta_C if and only if Cov⁡N(πi,Γi)=0\operatorname{Cov}_N(\pi_i,\Gamma_i)=0.

Equation (8) isolates three channels through which βR\beta_R departs from a causal parameter, which are the channels that de Chaisemartin and D’Haultfœuille [11] identify in the sampling-based framework. Switchers contribute the deviation of their effects from βC\beta_C, weighted by aia_i. Always-takers contribute the change in their treatment effects between periods. Trends in untreated outcomes contribute whenever their covariance with the assignment probabilities is different from zero, which is a design-based failure of parallel trends. In the design-based framework, all three channels operate only through heterogeneity in πi\pi_i.

Corollary 1 (Complete randomization). If πi=pN\pi_i=p_N for all ii, then ai=0a_i=0, τFS=NC/N\tau_{\mathrm{FS}}=N_C/N and βR=LATEC\beta_R=\mathrm{LATE}_C, with no restriction on switchers, always-takers or trends.

Corollary 2 (Sufficient conditions). Under the conditions of Theorem 1, βR=βC\beta_R=\beta_C if (a) Cov⁡N(πi,δi)=0\operatorname{Cov}_N(\pi_i,\delta_i)=0; (b) Cov⁡N(πi,1{i∈A}Δτi)=0\operatorname{Cov}_N(\pi_i,1\{i\in\mathcal{A}\}\Delta\tau_i)=0; and (c) Cov⁡N(πi,1{i∈S}(τi1−βC))=0\operatorname{Cov}_N(\pi_i,1\{i\in\mathcal{S}\}(\tau_{i1}-\beta_C))=0. In particular, (a)–(c) hold if Cov⁡N(πi,δi)=0\operatorname{Cov}_N(\pi_i,\delta_i)=0, τi1\tau_{i1} is constant and Δτi=0\Delta\tau_i=0 for i∈Ai\in\mathcal{A}, even if FN≠0F_N\ne0.

Condition (a) is design-based parallel trends. Condition (b) holds, for example, if effects on always-takers are stable over time, and condition (c) if switchers’ effects equal βC\beta_C. The last statement of Corollary 2 shows that switchers may be concentrated among units with high assignment probabilities. Conditions (a)–(c) of Corollary 2 are design-based counterparts of the parallel trends, stable treatment effect and homogeneous treatment effect assumptions of de Chaisemartin and D’Haultfœuille [11]. Unlike in their framework, none of them is needed under complete randomization (Corollary 1).

Remark 1 (Relation to existing conditions). Applied to first-differenced data, the IV results of Rambachan and Roth [29] give βR=βC\beta_R = \beta_C under RN=FN=0R_N = F_N = 0, that is, design-based parallel trends in both potential outcomes and potential treatments. Theorem 1(c) shows that the sharp requirement is the single restriction RN=βCFNR_N = \beta_C F_N, since Cov⁡N(πi,Γi)=pN(1−pN)(RN−βCFN)\operatorname{Cov}_N(\pi_i, \Gamma_i) = p_N(1-p_N)(R_N - \beta_C F_N).

Remark 2 (Negative weights). Theorem 1 can be written as

βR=∑i∈C∪Sωiτi1+τFS−1(∑i∈AaiΔτi+∑iaiδi)\beta_R = \sum_{i\in C\cup S} \omega_i\tau_{i1} + \tau_{\mathrm{FS}}^{-1}\left(\sum_{i\in A}a_i\Delta\tau_i + \sum_i a_i\delta_i\right)

with ωi=bi/τFS\omega_i = b_i/\tau_{\mathrm{FS}} for compliers and ωi=ai/τFS\omega_i = a_i/\tau_{\mathrm{FS}} for switchers. The ωi\omega_i sum to one, but switchers with πi<pN\pi_i < p_N receive negative weight. Hence, even under design-based parallel trends, βR\beta_R can be negative when every τi1\tau_{i1} is positive. This parallels the negative-weights results of de Chaisemartin and D’Haultfœuille [11, 12].

Remark 3 (Testable implications and always-takers). Because Di0D_{i0} and Yi0Y_{i0} are predetermined, Lemma 1 implies that ER[τ^W]=Cov⁡N(πi,Wi)/{pN(1−pN)}E_R[\widehat{\tau}_W] = \operatorname{Cov}_N(\pi_i, W_i)/\{p_N(1-p_N)\} for any predetermined WiW_i. Balance tests on Di0D_{i0} (that is, on 1{i∈A}\mathbf{1}\{i\in A\}), on Yi0Y_{i0} or on pre-period trends are therefore informative about the covariance of πi\pi_i with the components of Γi\Gamma_i, and Theorem 2 below delivers valid inference for them. Moreover, since always-takers are observed at baseline, one can drop units with Di0=1D_{i0} = 1. Conditional on the number of assigned units in the remaining subpopulation, the assignment vector there is again of the form in Assumption 3, and the term ∑i∈AaiΔτi\sum_{i\in A} a_i\Delta\tau_i disappears.

Inference

Theorem 1 shows that interpreting βR\beta_R as βC\beta_C requires a covariance restriction that the design itself rarely justifies. We therefore develop inference for βR\beta_R, which requires no such restriction, and return to the complier effect in § 5. Assumption 2 is not used in this section.

Test statistic

For a hypothesized value b∈Rb \in\mathbb{R}, let Ri(z;b)=ΔYi(z)−bΔDi(z)R_i(z;b)=\Delta Y_i(z)-b\Delta D_i(z) and Ri(b)=ΔYi−bΔDiR_i(b)=\Delta Y_i-b\Delta D_i. Since the difference in means is linear,

M^(b)=τ^RF−bτ^FS=τ^R(b),ER[M^(b)]=mN(b):=τRF−bτFS.\widehat{M}(b)=\widehat{\tau}_{\mathrm{RF}}-b\widehat{\tau}_{\mathrm{FS}}=\widehat{\tau}_{R}(b), \qquad\mathbb{E}_{R}[\widehat{M}(b)]=m_N(b):=\tau_{\mathrm{RF}}-b\tau_{\mathrm{FS}}.

When τFS≠0\tau_{\mathrm{FS}}\ne0, H0:βR=bH_0:\beta_R=b is equivalent to mN(b)=0m_N(b)=0. Let s^XW,1\widehat{s}_{XW,1} and s^XW,0\widehat{s}_{XW,0} be the sample covariances of (Xi,Wi)(X_i,W_i) among assigned and unassigned units, with denominators N1−1N_1-1 and N0−1N_0-1, and v^XW=s^XW,1/N1+s^XW,0/N0\widehat{v}_{XW}=\widehat{s}_{XW,1}/N_1+\widehat{s}_{XW,0}/N_0. The Neyman-type variance estimator of M^(b)\widehat{M}(b) is

V^(b)=v^YY−2bv^YD+b2v^DD,(9)\widehat{V}(b)=\widehat{v}_{YY}-2b\widehat{v}_{YD}+b^2\widehat{v}_{DD}, \tag*{(9)}

where YY and DD stand for ΔY\Delta Y and ΔD\Delta D. Equivalently, V^(b)\widehat{V}(b) is the HC2 variance of the coefficient on ZiZ_i in an ordinary least squares regression of Ri(b)R_i(b) on (1,Zi)(1,Z_i) ([21]). The Anderson–Rubin-type statistic is TN(b)=M^(b)/V^(b)1/2T_N(b)=\widehat{M}(b)/\widehat{V}(b)^{1/2}.

Asymptotic framework

We consider a sequence of finite populations indexed by NN, with all quantities implicitly depending on NN. Let ci(b)=Ri(1;b)/N1+Ri(0;b)/N0c_i(b)=R_i(1;b)/N_1+R_i(0;b)/N_0, π~i=πi(1−πi)\widetilde{\pi}_i=\pi_i(1-\pi_i), c‾(b)=∑iπ~ici(b)/∑iπ~i\overline{c}(b)=\sum_i\widetilde{\pi}_ic_i(b)/\sum_i\widetilde{\pi}_i and

VN(b)=∑i=1Nπ~i{ci(b)−c‾(b)}2.(10)V_N(b)=\sum_{i=1}^{N}\widetilde{\pi}_i\left\{c_i(b)-\overline{c}(b)\right\}^{2}. \tag*{(10)}

Assumption 4 (Regularity). For a given bb, (a) there is η∈(0,1/2)\eta\in(0,1/2) such that η≤πi≤1−η\eta\le\pi_i\le1-\eta for all ii and NN, and pN→p∈(0,1)p_N\to p\in(0,1); (b) sup⁡NN−1∑i{ΔYi(z)4+ΔDi(z)4}<∞\sup_N N^{-1}\sum_i\{\Delta Y_i(z)^4+\Delta D_i(z)^4\}<\infty for z∈{0,1}z\in\{0,1\}; (c) lim inf⁡NNVN(b)>0\liminf_N NV_N(b)>0.

Part (a) requires overlap. Part (b) is a standard moment bound for finite-population limit theorems ((Li and Ding, 2017)). Part (c) rules out degenerate cases in which M^(b)\widehat{M}(b) has negligible design variance. Define m1(b)=N1−1∑iπiRi(1;b)m_1(b)=N_1^{-1}\sum_i\pi_iR_i(1;b), m0(b)=N0−1∑i(1−πi)Ri(0;b)m_0(b)=N_0^{-1}\sum_i(1-\pi_i)R_i(0;b), ei(z;b)=Ri(z;b)−mz(b)e_i(z;b)=R_i(z;b)-m_z(b) and

VN+(b)=∑i=1Nπiei(1;b)2N12+∑i=1N(1−πi)ei(0;b)2N02.(11)V_N^+(b)=\sum_{i=1}^{N}\frac{\pi_i e_i(1;b)^2}{N_1^2}+\sum_{i=1}^{N}\frac{(1-\pi_i)e_i(0;b)^2}{N_0^2}. \tag*{(11)}

Theorem 2 (Design-based Anderson–Rubin inference). Suppose Assumptions 1, 3 and 4 hold for a given bb. (a) M^(b)−mN(b)=∑i(Zi−πi)ci(b)\widehat{M}(b)-m_N(b)=\sum_i(Z_i-\pi_i)c_i(b), VR[M^(b)]=VN(b){1+o(1)}V_{\mathsf R}[\widehat{M}(b)]=V_N(b)\{1+o(1)\}, and {M^(b)−mN(b)}/VR[M^(b)]1/2→dN(0,1)\{\widehat{M}(b)-m_N(b)\}/V_{\mathsf R}[\widehat{M}(b)]^{1/2}\xrightarrow{d}N(0,1). (b) N{V^(b)−VN+(b)}→p0N\{\widehat{V}(b)-V_N^+(b)\}\xrightarrow{p}0, and

VN+(b)−VN(b)≥∑i=1N{πiei(1;b)N1−(1−πi)ei(0;b)N0}2≥0.(12)V_N^+(b)-V_N(b)\geq\sum_{i=1}^{N}\left\{\frac{\pi_i e_i(1;b)}{N_1}-\frac{(1-\pi_i)e_i(0;b)}{N_0}\right\}^2\geq0. \tag*{(12)}

(c) If mN(b)=0m_N(b)=0 for all NN, then lim sup⁡N→∞PR{∣TN(b)∣>z1−α/2}≤α\limsup_{N\to\infty}\mathop{\mathrm{P}}_{\mathsf R}\{|T_N(b)|>z_{1-\alpha/2}\}\leq\alpha for every α∈(0,1)\alpha\in(0,1).

Part (c) does not involve τFS\tau_{\mathrm{FS}}. The test therefore remains valid when the first stage is weak, for instance along sequences with τFS=h/N\tau_{\mathrm{FS}}=h/\sqrt{N}, under which Wald inference fails ([34]). Part (b) shows that the variance estimator is conservative even when the effects of assignment on Ri(b)R_i(b) are constant, unless πi\pi_i is constant. This mirrors Rambachan and Roth [29] for the difference in means. Sharper but more complex variance bounds in the spirit of Aronow et al. [4] could be developed.

Corollary 3 (Anderson–Rubin confidence set). Let CSαAR={b:M^(b)2≤z1−α/22V^(b)}\mathrm{CS}^{\mathrm{AR}}_\alpha=\{b:\widehat{M}(b)^2\leq z_{1-\alpha/2}^2\widehat{V}(b)\}. If τFS≠0\tau_{\mathrm{FS}}\ne0 and Assumption 4 holds at b=βRb=\beta_{\mathrm R}, lim inf⁡NPR(βR∈CSαAR)≥1−α\liminf_{N}\mathop{\mathrm{P}}_{\mathsf R}(\beta_{\mathrm R}\in\mathrm{CS}^{\mathrm{AR}}_\alpha)\geq1-\alpha. With κ=z1−α/22\kappa=z_{1-\alpha/2}^2, A=τ^FS2−κv^DDA=\widehat{\tau}_{\mathrm{FS}}^2-\kappa\widehat{v}_{\mathrm{DD}}, B=τ^RFτ^FS−κv^YDB=\widehat{\tau}_{\mathrm{RF}}\widehat{\tau}_{\mathrm{FS}}-\kappa\widehat{v}_{\mathrm{YD}} and C=τ^RF2−κv^YYC=\widehat{\tau}_{\mathrm{RF}}^2-\kappa\widehat{v}_{\mathrm{YY}}, CSαAR={b:Ab2−2Bb+C≤0}\mathrm{CS}^{\mathrm{AR}}_\alpha=\{b:Ab^2-2Bb+C\leq0\}, which is (a) the bounded interval [(B−B2−AC)/A,(B+B2−AC)/A][(B-\sqrt{B^2-AC})/A,(B+\sqrt{B^2-AC})/A], containing β^\widehat{\beta}, if A>0A>0; (b) the union of two disjoint half-lines if A<0A<0 and B2>ACB^2>AC; (c) the real line if A<0A<0 and B2<ACB^2<AC. Case (a) occurs if and only if the first-stage tt-test rejects τFS=0\tau_{\mathrm{FS}}=0 at level α\alpha.

Corollary 4 (Wald inference under a strong first stage). If Assumption 4 holds at b=βRb=\beta_R and lim inf⁡N∣τFS∣>0\liminf_N|\tau_{\mathrm{FS}}|>0, then, with se^=V^(β^)1/2/∣τ^FS∣\widehat{\mathrm{se}}=\widehat{V}(\widehat{\beta})^{1/2}/|\widehat{\tau}_{\mathrm{FS}}|, lim sup⁡NPR(∣β^−βR∣>z1−α/2se^)≤α\limsup_N\mathbb{P}_{\mathcal{R}}(|\widehat{\beta}-\beta_R|>z_{1-\alpha/2}\widehat{\mathrm{se}})\le\alpha.

Remark 4 (Fixed populations and exact tests). The null mN(b)=0m_N(b)=0 is not sharp, so Fisher randomization tests are not directly available without further restrictions [23, 25]. In addition, unlike in a randomized experiment, the researcher does not know the πi\pi_i. Berry–Esseen bounds for rejective sampling, as used by Rambachan and Roth [29], can replace the sequence-of-populations argument.

Sensitivity analysis and known designs

Bounded contamination

Theorem 1(a) gives the identity

βC=τRF−RNτFS−FN,(13)\beta_C=\frac{\tau_{\mathrm{RF}}-R_N}{\tau_{\mathrm{FS}}-F_N}, \tag*{(13)}

so the complier effect is determined by the identified (τRF,τFS)(\tau_{\mathrm{RF}},\tau_{\mathrm{FS}}) and the two contamination terms in (5). These terms have direct interpretations: RNR_N is the covariance between assignment probabilities and the outcome changes units would experience without the policy, and FNF_N is the covariance between assignment probabilities and switching. We bound them.

Assumption 5 (Bounded design contamination). ∣RN∣≤MR|R_N|\le M_R and ∣FN∣≤MF|F_N|\le M_F for known MR,MF≥0M_R,M_F\ge0.

Proposition 1 (Identified set). Under Assumptions 1–3 and 5, βC∈ΘI={(τRF−r)/(τFS−f):∣r∣≤MR, ∣f∣≤MF, τFS−f>0}\beta_C\in\Theta_I=\{(\tau_{\mathrm{RF}}-r)/(\tau_{\mathrm{FS}}-f):|r|\le M_R,\ |f|\le M_F,\ \tau_{\mathrm{FS}}-f>0\}. If MF<τFSM_F<\tau_{\mathrm{FS}}, ΘI=[βlo,βhi]\Theta_I=[\beta^{\mathrm{lo}},\beta^{\mathrm{hi}}] with

βlo=min⁡s∈{−1,1}τRF−MRτFS+sMF,βhi=max⁡s∈{−1,1}τRF+MRτFS+sMF.\beta^{\mathrm{lo}}=\min_{s\in\{-1,1\}}\frac{\tau_{\mathrm{RF}}-M_R}{\tau_{\mathrm{FS}}+sM_F},\qquad \beta^{\mathrm{hi}}=\max_{s\in\{-1,1\}}\frac{\tau_{\mathrm{RF}}+M_R}{\tau_{\mathrm{FS}}+sM_F}.

Theorem 3 (Sensitivity confidence set). Let

CSαS(MR,MF)={b:∣M^(b)∣≤MR+∣b∣MF+z1−α/2V^(b)1/2}.\mathit{CS}^{S}_{\alpha}(M_R,M_F)=\left\{b:|\widehat{M}(b)|\le M_R+|b|M_F+z_{1-\alpha/2}\widehat{V}(b)^{1/2}\right\}.

Under Assumptions 11–33 and 55, and Assumption 44 at b=βCb = \beta_C, lim inf⁡NP{βC∈CSαS(MR,MF)}≥1−α\liminf_N \mathbb{P}\{\beta_C \in\mathrm{CS}_{\alpha}^{S}(M_R, M_F)\} \ge1-\alpha. For MR=MF=0M_R = M_F = 0, CSαS\mathrm{CS}_{\alpha}^{S} equals CSαAR\mathrm{CS}_{\alpha}^{AR}.

The confidence set requires no lower bound on the first stage and is computed by a one-dimensional grid search. Because it uses a two-sided critical value at every bb, it is conservative when ΘI\Theta_I is wide relative to sampling uncertainty; refinements in the spirit of Imbens and Manski [22] and Stoye [35] are possible.

Corollary 5 (Robustness of the null of no effect). 0∈CSαS(MR,MF)0 \in\mathrm{CS}_{\alpha}^{S}(M_R, M_F) if and only if ∣τ^RF∣≤MR+z1−α/2v^YY1/2\lvert\widehat{\tau}_{RF}\rvert\le M_R + z_{1-\alpha/2}\widehat{v}_{YY}^{1/2}, whatever the value of MFM_F. Hence the breakdown value for rejecting βC=0\beta_C = 0 is MR∗=max⁡{∣τ^RF∣−z1−α/2v^YY1/2,0}M_R^* = \max\{\lvert\widehat{\tau}_{RF}\rvert- z_{1-\alpha/2}\widehat{v}_{YY}^{1/2}, 0\}, or pN(1−pN)MR∗p_N(1-p_N)M_R^* in covariance units.

Corollary 55 has a simple practical message. Conclusions about the existence and sign of a complier effect depend only on how much the assignment probabilities may covary with outcome changes in the absence of the policy. Conclusions about its magnitude also depend on how they covary with switching.

Remark 5 (Calibration). By the Cauchy–Schwarz inequality,

∣RN∣≤σπSD⁡N(ΔYi(0))pN(1−pN),∣FN∣≤σπ{sS(1−sS)}1/2pN(1−pN),\lvert R_N\rvert\le\frac{\sigma_\pi\operatorname{SD}_N(\Delta Y_i(0))}{p_N(1-p_N)}, \qquad\lvert F_N\rvert\le\frac{\sigma_\pi\{s_S(1-s_S)\}^{1/2}}{p_N(1-p_N)},

where σπ=SD⁡N(πi)\sigma_\pi= \operatorname{SD}_N(\pi_i) measures the departure from complete randomization and sSs_S is the switcher share. The (1−πi)(1-\pi_i)-weighted analogues of SD⁡N(ΔYi(0))\operatorname{SD}_N(\Delta Y_i(0)) and sSs_S are estimable from unassigned units. This reduces the analysis to a single interpretable parameter σπ\sigma_\pi, possibly scaled by assumed correlations. Alternatively, following Rambachan and Roth [28], the covariance of πi\pi_i with pre-period trends, which is estimable by Remark 33, can benchmark Cov⁡N(πi,δi)\operatorname{Cov}_N(\pi_i,\delta_i), and the decomposition (7) allows each channel to be bounded separately.

Remark 6 (Relaxing monotonicity and exclusion). Lemma 11 does not use Assumption 22, and Theorems 22 and 33 use only identity (13). Time monotonicity is needed only to interpret RNR_N and FNF_N through compliance types; units that stop treatment enter RNR_N and FNF_N and require no change. If assignment monotonicity fails, with defiers D\mathcal{D} and BD=∑i∈DbiB_{\mathcal{D}}=\sum_{i\in\mathcal{D}}b_i, then (13) holds with RNR_N replaced by RN−∑i∈Dbiτi1R_N-\sum_{i\in\mathcal{D}}b_i\tau_{i1} and FNF_N by FN−BDF_N-B_{\mathcal{D}}. Bounds BD≤BˉB_{\mathcal{D}}\le\bar{B} and ∣τi1∣≤τˉ|\tau_{i1}|\le\bar{\tau} on D\mathcal{D} thus enlarge (MR,MF)(M_R,M_F) to (MR+Bˉτˉ,MF+Bˉ)(M_R+\bar{B}\bar{\tau},M_F+\bar{B}) (cf. [10]). Similarly, a direct effect γi\gamma_i of assignment on period-1 outcomes with ∣γi∣≤γˉ|\gamma_i|\le\bar{\gamma} adds ∑ibiγi∈[−γˉ,γˉ]\sum_i b_i\gamma_i\in[-\bar{\gamma},\bar{\gamma}] to τRF\tau_{RF}, since ∑ibi=1\sum_i b_i=1, and enlarges MRM_R to MR+γˉM_R+\bar{\gamma}.

Known assignment probabilities

In stratified experiments and in lotteries with known odds, the πi\pi_i are known, and the contamination terms are no longer nuisance parameters to be bounded but estimable quantities.

We state the result for stratified designs, which covers the application in § 7.

Assumption 6 (Stratified assignment). The units are partitioned into fixed strata G1,…,GG\mathcal{G}_1,\ldots,\mathcal{G}_G of sizes NgN_g, and Z\mathbf{Z} is completely randomized within each stratum, independently across strata, with fixed N1g∈{1,…,Ng−1}N_{1g}\in\{1,\ldots,N_g-1\} assigned units in stratum gg.

Under Assumption 6, πi=πg:=N1g/Ng\pi_i=\pi_g:=N_{1g}/N_g for i∈Ggi\in\mathcal{G}_g, and N1=∑gN1gN_1=\sum_gN_{1g} is fixed, so Lemma 1 and Theorem 1 apply. Let Xˉzg\bar{X}_{zg} be the mean of XiX_i among units in stratum gg with Zi=zZ_i=z, N0g=Ng−N1gN_{0g}=N_g-N_{1g}, and define the stratified and contamination estimators

θ^X=∑g=1GN1gN1(Xˉ1g−Xˉ0g),ρ^X=∑g=1G(N1gN1−N0gN0)Xˉ0g.(14)\hat{\theta}_X=\sum_{g=1}^{G}\frac{N_{1g}}{N_1}\left(\bar{X}_{1g}-\bar{X}_{0g}\right),\qquad \hat{\rho}_X=\sum_{g=1}^{G}\left(\frac{N_{1g}}{N_1}-\frac{N_{0g}}{N_0}\right)\bar{X}_{0g}. \tag*{(14)}

which satisfy τ^X=θ^X+ρ^X\hat{\tau}_X=\hat{\theta}_X+\hat{\rho}_X identically. Let M~(b)=θ^ΔY−bθ^ΔD\widetilde{M}(b)=\hat{\theta}_{\Delta Y}-b\hat{\theta}_{\Delta D}, and let V~(b)=∑g(N1g/N1)2{s^1g2(b)/N1g+s^0g2(b)/N0g}\widetilde{V}(b)=\sum_g(N_{1g}/N_1)^2\{\hat{s}_{1g}^2(b)/N_{1g}+\hat{s}_{0g}^2(b)/N_{0g}\}, where s^zg2(b)\hat{s}_{zg}^2(b) is the sample variance of Ri(b)R_i(b) among units in stratum gg with Zi=zZ_i=z.

Proposition 2 (Known assignment probabilities). Suppose Assumptions 1, 2 and 6 hold. (a) ER[θ^ΔD]=BC\mathbb{E}_{R}[\hat{\theta}_{\Delta D}]=B_C, ER[θ^ΔY]=BCβC\mathbb{E}_{R}[\hat{\theta}_{\Delta Y}]=B_C\beta_C, ER[ρ^ΔD]=FN\mathbb{E}_{R}[\hat{\rho}_{\Delta D}]=F_N and ER[ρ^ΔY]=RN\mathbb{E}_{R}[\hat{\rho}_{\Delta Y}]=R_N; hence βC=ER[θ^ΔY]/ER[θ^ΔD]\beta_C=\mathbb{E}_{R}[\hat{\theta}_{\Delta Y}]/\mathbb{E}_{R}[\hat{\theta}_{\Delta D}] and βR−βC=(RN−βCFN)/τFS\beta_R-\beta_C=(R_N-\beta_CF_N)/\tau_{FS} are identified. (b) Suppose further that GG is fixed, that η≤πg≤1−η\eta\le\pi_g\le1-\eta for some η>0\eta>0 and Ng/N→λg>0N_g/N\to\lambda_g>0, that the moment

bound of Assumption 4(b) holds within each stratum, and that lim inf⁡NNVR[M~(b)]>0\liminf_N N\mathbb{V}_R[\widetilde{M}(b)] > 0.

Under H0:βC=bH_0:\beta_C=b, lim sup⁡NPR{∣M~(b)∣>z1−α/2V~(b)1/2}≤α\limsup_N\mathbb{P}_R\{|\widetilde{M}(b)| > z_{1-\alpha/2}\widetilde{V}(b)^{1/2}\} \le\alpha.

Part (a) holds without any restriction on trends, switchers or always-takers: with a known design, randomization alone identifies βC\beta_C. The same argument applied to period-1 levels shows that the stratified IV estimator in levels has the same estimand, because ER[θ^Y0]=ER[θ^D0]=0\mathbb{E}_R[\widehat{\theta}_{Y0}] = \mathbb{E}_R[\widehat{\theta}_{D0}] = 0 for predetermined variables. The role of differencing is then only to improve precision, as in covariate adjustment. Part (b) gives a stratified Anderson–Rubin test that is again robust to a weak first stage. The estimators ρ^ΔY\widehat{\rho}_{\Delta Y} and ρ^ΔD\widehat{\rho}_{\Delta D} measure the contamination that the pooled Wald-DiD estimator incurs by ignoring the heterogeneity in πi\pi_i; they depend only on unassigned units, whose outcome changes reveal ΔYi(Di(0))\Delta Y_i(D_i(0)) and ΔDi(0)\Delta D_i(0).

Simulation evidence

We draw one finite population per design, hold its potential outcomes and potential treatment decisions fixed, and redraw only the assignment, 5,000 times, from Assumption 3 with N1=N/2N_1=N/2 and pi=logit⁡−1(c+γWi)p_i=\operatorname{logit}^{-1}(c+\gamma W_i) for a unit characteristic Wi∼N(0,1)W_i\sim N(0,1). Compliance-type shares are (0.40,0.10,0.35,0.15)(0.40,0.10,0.35,0.15) for (N,A,C,S)(N,A,C,S), and effects and trends depend on WiW_i. In D1 assignment is completely randomized (γ=0\gamma=0). In D2 assignment probabilities are correlated with untreated trends. In D3 switchers are concentrated among high-πi\pi_i units, with design-based parallel trends and homogeneous effects. D4 is as D3 but with heterogeneous effects. D5 has a weak first stage (N=200N=200, 4% compliers) under complete randomization. Section S2 of the Online Appendix gives the details and additional results.

Table 2 confirms the theory. In every design the AR set covers the design estimand βR\beta_R at or above the nominal rate, up to simulation error. The variance estimator is conservative, more so when assignment probabilities are heterogeneous (D2–D4), as predicted by (12). Under complete randomization, βR\beta_R equals the complier effect (Corollary 1). When assignment probabilities covary with untreated trends (D2), or with switching when effects are heterogeneous (D4), βR\beta_R departs substantially from βC\beta_C, and inference that treats βR\beta_R as a complier effect fails. In D3, switchers are concentrated among likely-assigned units (FN=0.071F_N = 0.071), yet βR=βC\beta_R = \beta_C because RN=βCFNR_N = \beta_C F_N, which illustrates that the condition in Theorem 1(c) is strictly weaker than RN=FN=0R_N = F_N = 0. The sensitivity set with correctly specified bounds restores coverage of βC\beta_C. With a weak first stage (D5), the Wald interval undercovers (78%) and the estimator is badly median biased, whereas the AR set retains nominal coverage.

NNβR\beta_RβC\beta_CτFS\tau_{\mathrm{FS}}BiasCoverage of βR\beta_RCoverage of βC\beta_CV‾/V‾R\overline{V}/\overline{V}_R
WaldARARSens.
D110000.9890.9890.3490.0000.9620.9600.9600.9601.05
D210002.5611.2540.3770.0060.9850.9850.0000.9971.57
D310001.0001.0000.412−0.001-0.0010.9690.9680.9681.0001.21
D410001.5391.1950.405−0.002-0.0020.9750.9740.1231.0001.34
D52001.0341.0790.0202.2880.7820.9510.9510.9521.01

Table 2. Design-based Monte Carlo results.

Note: Results over 5,000 assignment draws per design. Bias is the median of β^−βR\widehat{\beta}-\beta_R. Wald is the interval of Corollary 4, AR is CS0.05AR\mathrm{CS}^{\mathrm{AR}}_{0.05} and Sens. is CS0.05S(∣RN∣,∣FN∣)\mathrm{CS}^{\mathrm{S}}_{0.05}(|R_N|, |F_N|), with bounds set at the true contamination.

The last column is the mean of V^(βR)\widehat{V}(\beta_R) divided by the simulation variance of M^(βR)\widehat{M}(\beta_R). In D5, CS0.05AR\mathrm{CS}^{\mathrm{AR}}_{0.05} is unbounded in 94% of draws.

Empirical application: the Oregon Health Insurance Experiment

Setting and fit with the framework

In 2008 the state of Oregon reopened its Medicaid programme for low-income uninsured adults (OHP Standard) and allocated the right to apply by lottery among the individuals on a waiting list [16]. Eight drawings took place between March and September 2008. Selected individuals, together with the members of their household on the list, could apply, and about 30% of those selected enrolled. Taubman et al. [37] linked the list to emergency-department (ED) records from the Portland-area hospitals covering 1 January 2007 to 30 September 2009. We use their individual-level public-use data [15].

The mapping to our framework is direct. The units are the individuals in the ED sample, ZiZ_i indicates selection in any drawing, and YitY_{it} is the number of ED visits per year in period tt, where period 0 runs from 1 January 2007 to 9 March 2008 and period 1 from 10 March 2008, the earliest notification date, to 30 September 2009. The treatment Di1D_{i1} indicates enrolment in any Medicaid programme during period 1, the first-stage measure of Taubman et al. [37]. The public-use files contain no measure of enrolment before the lottery, and we set Di0=0D_{i0}=0. Then there are no always-takers, Assumption 2(b) holds trivially, and the always-taker channel in (7) is absent.

Three features make the experiment a good fit. First, compliance is imperfect in both directions. Many selected individuals did not enrol, and some individuals who were not selected enrolled through other routes, mostly in OHP Plus. The latter are switchers in Table 1, a group that a cross-sectional IV analysis classifies as always-takers. Second, the state drew individuals but extended selection to the whole household, a procedure that favoured larger households [16]. Assignment probabilities are therefore heterogeneous, which is precisely what generates the contamination terms of Theorem 1. Third, the ED data form a panel with outcomes measured before randomization.

Unlike in the quasi-experiments for which § 4 and § 5 are designed, the assignment probabilities are known up to the number of household members on the list (1, 2 or 3), within which selection is random. The application is therefore a benchmark. Proposition 2 identifies βC\beta_C, so we can estimate the contamination (RN,FN)(R_N, F_N) that an analyst who ignores the heterogeneity in πi\pi_i would face, compare the pooled Wald-DiD estimate with β^C=θ^ΔY/θ^ΔD\hat{\beta}_C = \hat{\theta}_{\Delta Y}/\hat{\theta}_{\Delta D}, and compare the estimated contamination with the breakdown value of § 5.

Two features of the lottery require adaptation. Because selection is at the household level, all variances treat households as the units of assignment and are computed from household totals of residuals within each stratum and arm. Because the number of selected individuals was not fixed in advance, we condition on it.

Results

The ED sample contains 24,615 individuals with non-missing outcomes in 22,159 households. The overall selection rate is pN=0.390p_N = 0.390, and the selection rates for one, two and three household members on the list are 0.344, 0.571 and 0.885, so that σπ=0.094\sigma_\pi= 0.094. Among individuals who were not selected, a share 0.151 enrolled in Medicaid during period 1; these are the switchers.

Table 3 reports the pooled estimators of § 2, which ignore the strata and estimate τFS\tau_{FS}, τRF\tau_{RF} and βR\beta_R, and the stratified estimators of Proposition 2, which estimate BCB_C, BCβCB_C\beta_C and βC\beta_C. Selection raised Medicaid enrolment by 0.241, so the first stage is strong and the AR sets are bounded intervals. For ED visits per year, the stratified estimate of the complier effect is 0.292 (standard error 0.077), against a period-1 mean of 0.64 visits per year. among non-selected individuals, consistent with the increase in ED use found by Taubman et al. [37]. The pooled Wald-DiD estimate is 0.266 (0.077). Their difference, −0.026, estimates the bias (RN−βCFN)/τFS(R_N-\beta_C F_N)/\tau_{FS} of Theorem 1(b). Both contamination estimates are small: ρ^ΔY=−0.007\hat{\rho}_{\Delta Y}=-0.007 (0.005) and ρ^ΔD=−0.001\hat{\rho}_{\Delta D}=-0.001 (0.001), so switching is essentially uncorrelated with the assignment probabilities. The breakdown value of Corollary 5 for the pooled analysis is MR∗=0.028M_R^*=0.028, about four times ∣ρ^ΔY∣\lvert\hat{\rho}_{\Delta Y}\rvert. An analyst who did not know the design, but was willing to bound the reduced-form contamination at a few hundredths of a visit per year, would therefore have correctly concluded that Medicaid increases ED use among compliers.

First stageReduced formWald-DiD95% AR setρ^ΔY\hat{\rho}_{\Delta Y}ρ^ΔD\hat{\rho}_{\Delta D}
ED visits per year
Pooled (ignores strata)0.2400.0640.266[0.115, 0.417]−0.007−0.001
(0.006)(0.018)(0.077)(0.005)(0.001)
Stratified (known design)0.2410.0710.292[0.142, 0.443]
(0.006)(0.019)(0.077)
Any ED visit
Pooled (ignores strata)0.2400.0140.060[0.003, 0.116]−0.002−0.001
(0.006)(0.007)(0.029)(0.002)(0.001)
Stratified (known design)0.2410.0160.068[0.011, 0.125]
(0.006)(0.007)(0.029)

Table 3. Medicaid and emergency-department use in Oregon. Note: Pooled rows report τ^FS\hat{\tau}_{FS}, τ^RF\hat{\tau}_{RF}, β^\hat{\beta} and CS0.05AR\mathrm{CS}^{\mathrm{AR}}_{0.05} (§ 4). Stratified rows report θ^ΔD\hat{\theta}_{\Delta D}, θ^ΔY\hat{\theta}_{\Delta Y}, β^C\hat{\beta}_{C} and the stratified AR set of Proposition 2. The last two columns report the contamination estimators ρ^ΔY\hat{\rho}_{\Delta Y} and ρ^ΔD\hat{\rho}_{\Delta D} in (14). Strata are household size on the lottery list. The treatment is Medicaid enrolment. Standard errors in parentheses treat households as the units of assignment. ED visits per year are visit counts, truncated and censored as in the public-use files, divided by the length of each period in years. Any ED visit compares an indicator for any visit in period 1 with the same indicator in period 0, whose windows differ in length. Sample: 24,615 individuals in the ED sample of Taubman et al. [37].

Differencing is what makes the pooled analysis robust. Larger households, which were more likely to be selected, used the ED less before the lottery: the pooled difference in pre-period ED visits per year between selected and non-selected individuals is −0.070 (0.020), whereas the stratified difference is −0.011 (0.020), consistent with successful randomization within strata (Remark 3). Consequently, the pooled IV estimate in period-1 levels is −0.026 (0.085), whereas the stratified levels estimate, whose estimand is again βC\beta_C, is 0.245 (0.081). Selection on household size thus enters levels strongly but trends only weakly, and the Wald-DiD estimand inherits only the latter. The any-visit outcome gives the same qualitative picture.

Conclusion

This paper develops a design-based framework for the fuzzy difference-in-differences setting. This approach is motivated by settings in which researchers wish to remain agnostic about the statistical process behind the sampling and formation of potential outcomes and treatments. Viewed through the lens of the assignment mechanism, fuzzy DiD has a transparent structure. The Wald-DiD design estimand equals an assignment-weighted complier effect plus contamination from switchers, always-takers and trends in untreated outcomes, all of which affect the estimand through the covariance between assignment probabilities and a single contamination index. When this covariance cannot be argued to vanish, the design still supports valid, weak-identification-robust and conservative inference on the Wald-DiD estimand, and a sensitivity analysis for the complier effect whose conclusions about the existence of an effect depend only on the reduced-form contamination. When the design is known, the contamination is estimable and the complier effect is identified. Extensions to staggered adoption ([5, 31]), clustered assignment, covariate adjustment and alternative fuzzy DiD estimands such as the time-corrected Wald ratio of de Chaisemartin and D’Haultfœuille [11] are natural next steps.

Proofs

Throughout, p=pNp = p_N and all sums run over i=1,…,Ni = 1,\ldots,N unless indicated otherwise. We use the following properties of the rejective design. Their proof is routine and is given in Section S3 of the Online Appendix.

Lemma 2. Suppose Assumption 3 holds and let dN=∑iπ~id_N = \sum_i \tilde{\pi}_i. (a) For i≠ji \ne j, Cov⁡R(Zi,Zj)≤0\operatorname{Cov}_{R}(Z_i,Z_j) \le0; consequently, for any real weights wiw_i, V⁡R[∑iZiwi]≤2∑iπ~iwi2≤12∑iwi2\operatorname{V}_{R}\left[\sum_i Z_iw_i\right] \le2\sum_i \tilde{\pi}_i w_i^2 \le\frac{1}{2}\sum_i w_i^2. (b) If dN→∞d_N \to\infty, then for any array {yi}\{y_i\} with yˉ=∑iπ~iyi/dN\bar{y} = \sum_i \tilde{\pi}_i y_i/d_N, V⁡R[∑iZiyi]={1+o(1)}∑iπ~i(yi−yˉ)2\operatorname{V}_{R}\left[\sum_i Z_i y_i\right] = \{1+o(1)\}\sum_i \tilde{\pi}_i(y_i-\bar{y})^2. (c) If, in addition, max⁡i(yi−yˉ)2/∑iπ~i(yi−yˉ)2→0\max_i(y_i-\bar{y})^2/\sum_i \tilde{\pi}_i(y_i-\bar{y})^2 \to0, then ∑i(Zi−πi)yi/{∑iπ~i(yi−yˉ)2}1/2→dN(0,1)\sum_i(Z_i-\pi_i)y_i/\{\sum_i \tilde{\pi}_i(y_i-\bar{y})^2\}^{1/2} \xrightarrow{d} N(0,1).

Proofs for Section 3

Proof of Lemma 1. Since N1N_1 and N0N_0 are fixed, E⁡R[τ~X]=∑i{πiXi(1)/N1−(1−πi)Xi(0)/N0}\operatorname{E}_{R}[\tilde{\tau}_X] = \sum_i\{\pi_iX_i(1)/N_1 - (1-\pi_i)X_i(0)/N_0\}. Adding and subtracting πiXi(0)/N1\pi_iX_i(0)/N_1 gives ∑ibi{Xi(1)−Xi(0)}+∑iaiXi(0)\sum_i b_i\{X_i(1)-X_i(0)\}+\sum_i a_iX_i(0).

For the second claim, ai={πi(N0+N1)−N1}/(N1N0)=(πi−p)/{Np(1−p)}a_i = \{\pi_i(N_0+N_1)-N_1\}/(N_1N_0) = (\pi_i-p)/\{Np(1-p)\}. Since ∑iπi=N1\sum_i\pi_i=N_1, the mean of πi\pi_i is pp and ∑i(πi−p)Xi(0)=NCov⁡N(πi,Xi(0))\sum_i(\pi_i-p)X_i(0)=N\operatorname{Cov}_{N}(\pi_i,X_i(0)).

Proof of Theorem 1. (a) By Table 1, ΔDi(1)−ΔDi(0)=1{i∈C}\Delta D_i(1)-\Delta D_i(0)=1\{i\in\mathcal{C}\} and ΔDi(0)=1{i∈S}\Delta D_i(0)=1\{i\in\mathcal{S}\}, so Lemma 1 gives τFS=∑i∈Cbi+∑i∈Sai=BC+FN\tau_{\mathrm{FS}}=\sum_{i\in\mathcal{C}}b_i+\sum_{i\in\mathcal{S}}a_i=B_C+F_N. Likewise, ΔYi(1)−ΔYi(0)=1{i∈C}τi1\Delta Y_i(1)-\Delta Y_i(0)=1\{i\in\mathcal{C}\}\tau_{i1} and ΔYi(0)=δi+1{i∈A}Δτi+1{i∈S}τi1\Delta Y_i(0)=\delta_i+1\{i\in\mathcal{A}\}\Delta\tau_i+1\{i\in\mathcal{S}\}\tau_{i1}. Hence τRF=∑i∈Cbiτi1+RN\tau_{\mathrm{RF}}=\sum_{i\in\mathcal{C}}b_i\tau_{i1}+R_N with RNR_N as in (7), and ∑i∈Cbiτi1=BCβC\sum_{i\in\mathcal{C}}b_i\tau_{i1}=B_C\beta_C by (4). Finally, BC>0B_C>0 because NC≥1N_C\ge1 and πi>0\pi_i>0 for all ii.

(b) From (a), τRF−βCτFS=RN−βCFN\tau_{\mathrm{RF}}-\beta_C\tau_{\mathrm{FS}}=R_N-\beta_CF_N. Dividing by τFS\tau_{\mathrm{FS}} and substituting (6) and (7) gives (8). (c) RN−βCFN=∑iai{ΔYi(0)−βCΔDi(0)}=∑iaiΓi=Cov⁡N(πi,Γi)/{p(1−p)}R_N-\beta_CF_N=\sum_i a_i\{\Delta Y_i(0)-\beta_C\Delta D_i(0)\}=\sum_i a_i\Gamma_i=\operatorname{Cov}_{N}(\pi_i,\Gamma_i)/\{p(1-p)\} by Lemma 1. Since τFS≠0\tau_{\mathrm{FS}}\ne0, part (b) shows that βR=βC\beta_R=\beta_C if and only if this quantity is zero.

Proof of Corollaries 1 and 2. If πi≡p\pi_i\equiv p, then ai=0a_i=0 and bi=1/Nb_i=1/N, so RN=FN=0R_N=F_N=0, BC=NC/NB_C=N_C/N and βC=LATE⁡C\beta_C=\operatorname{LATE}_C, and Theorem 1(a) gives Corollary 1. For Corollary 2, Cov⁡N(πi,Γi)\operatorname{Cov}_{N}(\pi_i,\Gamma_i) is the sum of the three covariances in (a)–(c) by linearity, so Theorem 1(c) applies. If τi1=τ\tau_{i1}=\tau for all ii and Δτi=0\Delta\tau_i=0 for i∈Ai\in\mathcal{A}, then βC=τ\beta_C=\tau and (b) and (c) hold trivially, whatever the value of FNF_N.

Proofs for Section 4

Proof of Theorem 2. Fix bb and suppress it from the notation, writing Ri(z)=Ri(z;b)R_i(z)=R_i(z;b) and ei(z)=ei(z;b)e_i(z)=e_i(z;b).

Step 1 (exact representation). Since N1N_1 is fixed, M^=∑iZiRi(1)/N1−∑i(1−Zi)Ri(0)/N0=∑iZici−∑iRi(0)/N0\widehat{M}=\sum_i Z_iR_i(1)/N_1-\sum_i(1-Z_i)R_i(0)/N_0=\sum_i Z_ic_i-\sum_iR_i(0)/N_0. Taking expectations, mN=∑iπici−∑iRi(0)/N0m_N=\sum_i\pi_ic_i-\sum_iR_i(0)/N_0, so M^−mN=∑i(Zi−πi)ci\widehat{M}-m_N=\sum_i(Z_i-\pi_i)c_i.

Step 2 (asymptotic normality). Assumption 4(a) gives dN≥Nη(1−η)→∞d_N\geq N\eta(1-\eta)\to\infty.

By Lemma 2(b) with yi=ciy_i=c_i, VR[M^]=VN{1+o(1)}V_{\mathcal{R}}[\widehat{M}]=V_N\{1+o(1)\}. Assumption 4(b) implies N−1∑iRi(z)4≤KN^{-1}\sum_iR_i(z)^4\leq K for some K<∞K<\infty, and hence max⁡iRi(z)2≤(∑iRi(z)4)1/2≤(KN)1/2\max_iR_i(z)^2\leq(\sum_iR_i(z)^4)^{1/2}\leq(KN)^{1/2}.

Because N1,N0≥cNN_1,N_0\geq cN for some c>0c>0 and large NN, max⁡ici2=O(N−3/2)\max_i c_i^2=O(N^{-3/2}) and max⁡i(ci−cˉ)2=O(N−3/2)\max_i(c_i-\bar{c})^2=O(N^{-3/2}). By Assumption 4(c), VN≥κ/NV_N\geq\kappa/N for some κ>0\kappa>0, so the ratio in Lemma 2(c) is O(N−1/2)O(N^{-1/2}), and part (a) follows.

Step 3 (variance estimator). Let Rˉ1=N1−1∑iZiRi\bar{R}_1=N_1^{-1}\sum_iZ_iR_i. Then Rˉ1−m1=N1−1∑iZiei(1)\bar{R}_1-m_1=N_1^{-1}\sum_iZ_ie_i(1) and

s^12=1N1−1{∑iZiei(1)2−N1(Rˉ1−m1)2}.\widehat{s}_1^2=\frac{1}{N_1-1}\left\{\sum_i Z_ie_i(1)^2-N_1(\bar{R}_1-m_1)^2\right\}.

First, ER[∑iZiei(1)2]=∑iπiei(1)2\mathbb{E}_{\mathcal{R}}[\sum_iZ_ie_i(1)^2]=\sum_i\pi_ie_i(1)^2. Since ∣m1∣≤N1−1∑i∣Ri(1)∣=O(1)|m_1|\leq N_1^{-1}\sum_i|R_i(1)|=O(1), ∑iei(1)4=O(N)\sum_i e_i(1)^4=O(N), and Lemma 2(a) gives VR[N−1∑iZiei(1)2]≤(4N2)−1∑iei(1)4=O(N−1)V_{\mathcal{R}}[N^{-1}\sum_iZ_ie_i(1)^2]\leq(4N^2)^{-1}\sum_i e_i(1)^4=O(N^{-1}). Second, ER[∑iZiei(1)]=∑iπiRi(1)−N1m1=0\mathbb{E}_{\mathcal{R}}[\sum_iZ_ie_i(1)]=\sum_i\pi_iR_i(1)-N_1m_1=0 and, by Lemma 2(a), VR[∑iZiei(1)]≤12∑iei(1)2=O(N)V_{\mathcal{R}}[\sum_iZ_ie_i(1)]\leq\frac{1}{2}\sum_i e_i(1)^2=O(N), so Rˉ1−m1=Op(N−1/2)\bar{R}_1-m_1=O_p(N^{-1/2}). Therefore s^12=N1−1∑iπiei(1)2+Op(N−1/2)\widehat{s}_1^2=N_1^{-1}\sum_i\pi_ie_i(1)^2+O_p(N^{-1/2}) and, analogously, s^02=N0−1∑i(1−πi)ei(0)2+Op(N−1/2)\widehat{s}_0^2=N_0^{-1}\sum_i(1-\pi_i)e_i(0)^2+O_p(N^{-1/2}). Since V^=s^12/N1+s^02/N0\widehat{V}=\widehat{s}_1^2/N_1+\widehat{s}_0^2/N_0 (expression (9) is the same quantity computed from the sample moments of ΔYi\Delta Y_i and ΔDi\Delta D_i),

N(V^−VN+)=Op(N−1/2).N(\widehat{V}-V_N^+)=O_p(N^{-1/2}).

Step 4 (conservativeness). Let k=m1/N1+m0/N0k=m_1/N_1+m_0/N_0, so that ci−k=ei(1)/N1+ei(0)/N0c_i-k=e_i(1)/N_1+e_i(0)/N_0.

Direct expansion gives

πiei(1)2N12+(1−πi)ei(0)2N02−π~i{ei(1)N1+ei(0)N0}2={πiei(1)N1−(1−πi)ei(0)N0}2.\frac{\pi_i e_i(1)^2}{N_1^2}+\frac{(1-\pi_i)e_i(0)^2}{N_0^2}-\tilde{\pi}_i\left\{\frac{e_i(1)}{N_1}+\frac{e_i(0)}{N_0}\right\}^2=\left\{\frac{\pi_i e_i(1)}{N_1}-\frac{(1-\pi_i)e_i(0)}{N_0}\right\}^2.

Summing over ii, VN+−∑iπ~i(ci−k)2V_N^+ - \sum_i \tilde{\pi}_i(c_i-k)^2 equals the right-hand side of (12). Since cˉ\bar{c} minimizes x↦∑iπ~i(ci−x)2x \mapsto\sum_i \tilde{\pi}_i(c_i-x)^2, ∑iπ~i(ci−k)2≥VN\sum_i \tilde{\pi}_i(c_i-k)^2 \ge V_N, which proves (12).

Step 5 (test validity). Let GN=(M^−mN)/VR[M^]1/2G_N = (\widehat{M}-m_N)/V_{\mathcal{R}}[\widehat{M}]^{1/2}, so that GN→dN(0,1)G_N \xrightarrow{d} N(0,1) by Step 2.

Under mN=0m_N=0, TN=GN{VR[M^]/V^}1/2T_N=G_N\{\mathopen{}V_{\mathcal{R}}[\widehat{M}]/\widehat{V}\mathclose{}\}^{1/2}. By Steps 2–4 and NVN≥κNV_N\ge\kappa,

V^VR[M^]=VN++op(N−1)VN{1+o(1)}≥VN+op(N−1)VN{1+o(1)}=1+op(1).\frac{\widehat{V}}{V_{\mathcal{R}}[\widehat{M}]} = \frac{V_N^+ + o_p(N^{-1})}{V_N\{1+o(1)\}} \ge \frac{V_N + o_p(N^{-1})}{V_N\{1+o(1)\}} =1+o_p(1).

For any ϵ∈(0,1)\epsilon\in(0,1), PR(∣TN∣>z)≤PR{∣GN∣>z(1−ϵ)}+PR{V^<(1−ϵ)2VR[M^]}\mathbb{P}_{\mathcal{R}}(|T_N|>z)\le\mathbb{P}_{\mathcal{R}}\{|G_N|>z(1-\epsilon)\}+\mathbb{P}_{\mathcal{R}}\{\widehat{V}<(1-\epsilon)^2V_{\mathcal{R}}[\widehat{M}]\}. The second term vanishes, so lim sup⁡NPR(∣TN∣>z)≤2{1−Φ(z(1−ϵ))}\limsup_N\mathbb{P}_{\mathcal{R}}(|T_N|>z)\le2\{1-\Phi(z(1-\epsilon))\}. Letting ϵ↓0\epsilon\downarrow0 with z=z1−α/2z=z_{1-\alpha/2} proves part (c).

Proof of Corollary 3. Since τFS≠0\tau_{\mathrm{FS}}\ne0, mN(βR)=0m_N(\beta_R)=0, and Theorem 2(c) at b=βRb=\beta_R gives the coverage statement. Expanding M^(b)2−κV^(b)\widehat{M}(b)^2-\kappa\widehat{V}(b) with (9) gives Ab2−2Bb+CAb^2-2Bb+C. If A>0A>0, the quadratic is convex and equals −κV^(β^)≤0-\kappa\widehat{V}(\widehat{\beta})\le0 at b=β^b=\widehat{\beta}, because M^(β^)=0\widehat{M}(\widehat{\beta})=0; hence B2≥ACB^2\ge AC and the set is the stated interval. If A<0A<0, the quadratic is concave, and its sublevel set {≤0}\{\le0\} is the complement of the open interval between the real roots when B2>ACB^2>AC and the whole line when B2<ACB^2<AC. Finally, A>0A>0 if and only if ∣τ^FS∣/v^DD1/2>z1−α/2|\widehat{\tau}_{\mathrm{FS}}|/\widehat{v}_{DD}^{1/2}>z_{1-\alpha/2}.

Proof of Corollary 4. Since τ^RF−βRτ^FS=M^(βR)\widehat{\tau}_{\mathrm{RF}}-\beta_R\widehat{\tau}_{\mathrm{FS}}=\widehat{M}(\beta_R), β^−βR=M^(βR)/τ^FS\widehat{\beta}-\beta_R=\widehat{M}(\beta_R)/\widehat{\tau}_{\mathrm{FS}}, and the event ∣β^−βR∣>zse^|\widehat{\beta}-\beta_R|>z\widehat{\mathrm{se}} is ∣M^(βR)∣>zV^(β^)1/2|\widehat{M}(\beta_R)|>z\widehat{V}(\widehat{\beta})^{1/2}. Steps 2 and 3 of the proof of Theorem 2 applied to ΔDi\Delta D_i give τ^FS−τFS=Op(N−1/2)\widehat{\tau}_{\mathrm{FS}}-\tau_{\mathrm{FS}}=O_p(N^{-1/2}). In addition, VN(βR)≤14∑ici(βR)2=O(N−1)V_N(\beta_R)\le\frac{1}{4}\sum_i c_i(\beta_R)^2=O(N^{-1}), so M^(βR)=Op(N−1/2)\widehat{M}(\beta_R)=O_p(N^{-1/2}) and β^−βR=Op(N−1/2)\widehat{\beta}-\beta_R=O_p(N^{-1/2}). Since v^YY,v^DD=Op(N−1)\widehat{v}_{YY},\widehat{v}_{DD}=O_p(N^{-1}) and ∣v^YD∣≤(v^YYv^DD)1/2|\widehat{v}_{YD}|\le(\widehat{v}_{YY}\widehat{v}_{DD})^{1/2},

V^(β^)−V^(βR)=−2(β^−βR)v^YD+(β^2−βR2)v^DD=op(N−1).\widehat{V}(\widehat{\beta})-\widehat{V}(\beta_R) =-2(\widehat{\beta}-\beta_R)\widehat{v}_{YD} +(\widehat{\beta}^{2}-\beta_R^{2})\widehat{v}_{DD} =o_p(N^{-1}).

Step 5 of the proof of Theorem 2 then applies with V^(β^)\widehat{V}(\widehat{\beta}) in place of V^(βR)\widehat{V}(\beta_R).

Proofs for Section 5

Proof of Proposition 1. By (13), βC=g(RN,FN)\beta_C=g(R_N,F_N) with g(r,f)=(τRF−r)/(τFS−f)g(r,f)=(\tau_{\mathrm{RF}}-r)/(\tau_{\mathrm{FS}}-f), ∣RN∣≤MR|R_N|\le M_R, ∣FN∣≤MF|F_N|\le M_F and τFS−FN=BC>0\tau_{\mathrm{FS}}-F_N=B_C>0; hence βC∈ΘI\beta_C\in\Theta_I. If MF<τFSM_F<\tau_{\mathrm{FS}}, gg is continuous on the compact and connected box [−MR,MR]×[−MF,MF][-M_R,M_R]\times[-M_F,M_F], so its image is a compact interval. For fixed ff, gg is strictly decreasing in rr. For fixed rr, ∂g/∂f=(τRF−r)/(τFS−f)2\partial g/\partial f=(\tau_{RF}-r)/(\tau_{FS}-f)^2 has constant sign, so gg is monotone in ff. The minimum is therefore attained at r=MRr=M_R and f∈{−MF,MF}f\in\{-M_F,M_F\}, and the maximum at r=−MRr=-M_R and f∈{−MF,MF}f\in\{-M_F,M_F\}.

Proof of Theorem 3. By (13), mN(βC)=τRF−βCτFS=RN−βCFNm_N(\beta_C)=\tau_{RF}-\beta_C\tau_{FS}=R_N-\beta_CF_N, so ∣mN(βC)∣≤MR+∣βC∣MF|m_N(\beta_C)|\le M_R+|\beta_C|M_F under Assumption 5. Hence

PR{βC∉CαS(MR,MF)}≤PR{∣M^(βC)−mN(βC)∣>z1−α/2V^(βC)1/2}.\mathbb{P}_{R}\left\{\beta_C\notin\mathcal{C}_{\alpha}^{S}(M_R,M_F)\right\}\le\mathbb{P}_{R}\left\{\left|\widehat{M}(\beta_C)-m_N(\beta_C)\right|>z_{1-\alpha/2}\widehat{V}(\beta_C)^{1/2}\right\}.

Steps 1–5 of the proof of Theorem 2, applied at b=βCb=\beta_C to the centred statistic M^(βC)−mN(βC)\widehat{M}(\beta_C)-m_N(\beta_C), show that the right-hand side has limit superior at most α\alpha. The case MR=MF=0M_R=M_F=0 is immediate from the definitions.

Proof of Corollary 5. At b=0b=0, M^(0)=τ^RF\widehat{M}(0)=\widehat{\tau}_{RF}, V^(0)=v^YY\widehat{V}(0)=\widehat{v}_{YY} and ∣b∣MF=0|b|M_F=0.

Proof of Proposition 2. (a) Under Assumption 6, ER[X‾zg]=Ng−1∑i∈GgXi(z)\mathbb{E}_R[\overline{X}_{zg}]=N_g^{-1}\sum_{i\in\mathcal{G}_g}X_i(z). Hence

ER[θ^X]=∑g∑i∈GgN1gN1Ng{Xi(1)−Xi(0)}=∑ibi{Xi(1)−Xi(0)},\mathbb{E}_R[\widehat{\theta}_X]=\sum_g\sum_{i\in\mathcal{G}_g}\frac{N_{1g}}{N_1N_g}\{X_i(1)-X_i(0)\}=\sum_i b_i\{X_i(1)-X_i(0)\},

since πi=N1g/Ng\pi_i=N_{1g}/N_g for i∈Ggi\in\mathcal{G}_g, and similarly

ER[ρ^X]=∑g∑i∈Gg(πgN1−1−πgN0)Xi(0)=∑iaiXi(0).\mathbb{E}_R[\widehat{\rho}_X]=\sum_g\sum_{i\in\mathcal{G}_g}\left(\frac{\pi_g}{N_1}-\frac{1-\pi_g}{N_0}\right)X_i(0)=\sum_i a_iX_i(0).

By Table 1, ∑ibi{ΔDi(1)−ΔDi(0)}=BC\sum_i b_i\{\Delta D_i(1)-\Delta D_i(0)\}=B_C, ∑ibi{ΔYi(1)−ΔYi(0)}=BCβC\sum_i b_i\{\Delta Y_i(1)-\Delta Y_i(0)\}=B_C\beta_C, ∑iaiΔDi(0)=FN\sum_i a_i\Delta D_i(0)=F_N and ∑iaiΔYi(0)=RN\sum_i a_i\Delta Y_i(0)=R_N. The identity τ^X=θ^X+ρ^X\widehat{\tau}_X=\widehat{\theta}_X+\widehat{\rho}_X follows from τ^X=∑g{(N1g/N1)X‾1g−(N0g/N0)X‾0g}\widehat{\tau}_X=\sum_g\{(N_{1g}/N_1)\overline{X}_{1g}-(N_{0g}/N_0)\overline{X}_{0g}\}, and the expression for βR−βC\beta_R-\beta_C is Theorem 1(b).

(b) Under H0:βC=bH_0:\beta_C=b, part (a) gives ER[M^(b)]=BC(βC−b)=0\mathbb{E}_R[\widehat{M}(b)]=B_C(\beta_C-b)=0. Write M^(b)=∑gwg(R‾1g−R‾0g)\widehat{M}(b)=\sum_g w_g(\overline{R}_{1g}-\overline{R}_{0g}) with wg=N1g/N1w_g=N_{1g}/N_1 and R‾zg\overline{R}_{zg} the mean of Ri(b)R_i(b) in stratum gg and arm zz. The summands are independent across strata. Within each stratum, assignment is completely randomized with N1g/Ng∈[η,1−η]N_{1g}/N_g \in[\eta,1-\eta], and the moment bound implies the conditions of the finite-population central limit theorem of Li and Ding (2017, Theorem 1). Since GG is fixed and NV⁡R[M~(b)]N\operatorname{V}_{R}[\widetilde{M}(b)] is bounded away from zero, M~(b)/V⁡R[M~(b)]1/2→dN(0,1)\widetilde{M}(b)/\operatorname{V}_{R}[\widetilde{M}(b)]^{1/2} \xrightarrow{d} N(0,1). Within stratum gg, V⁡R[R‾1g−R‾0g]=S1g2/N1g+S0g2/N0g−Sτg2/Ng\operatorname{V}_{R}[\overline{R}_{1g}-\overline{R}_{0g}]=S^{2}_{1g}/N_{1g}+S^{2}_{0g}/N_{0g}-S^{2}_{\tau g}/N_g, where Szg2S^{2}_{zg} is the finite-population variance of Ri(z;b)R_i(z;b) in stratum gg and Sτg2S^{2}_{\tau g} that of Ri(1;b)−Ri(0;b)R_i(1;b)-R_i(0;b); and S^zg2(b)−Szg2→p0\widehat{S}^{2}_{zg}(b)-S^{2}_{zg}\xrightarrow{p}0 (Li and Ding, 2017). Hence N{V~(b)−V⁡R[M~(b)]}≥op(1)N\{\widetilde{V}(b)-\operatorname{V}_{R}[\widetilde{M}(b)]\}\geq o_p(1), and the argument of Step 5 of the proof of Theorem 2 completes the proof.

References

  1. [1]Abadie, A., Athey, S., Imbens, G. W., and Wooldridge, J. M. (2020). Sampling-based versus design-based uncertainty in regression analysis. Econometrica, 88:265–296.DOI
  2. [2]Abadie, A., Athey, S., Imbens, G. W., and Wooldridge, J. M. (2023). When should you adjust standard errors for clustering? Quarterly Journal of Economics, 138:1–35.
  3. [3]Andrews, I., Stock, J. H., and Sun, L. (2019). Weak instruments in instrumental variables regression: Theory and practice. Annual Review of Economics, 11:727–753.DOI
  4. [4]Aronow, P. M., Green, D. P., and Lee, D. K. K. (2014). Sharp bounds on the variance in randomized experiments. Annals of Statistics, 42:850–871.
  5. [5]Athey, S. and Imbens, G. W. (2022). Design-based analysis in difference-in-differences settings with staggered adoption. Journal of Econometrics, 226:62–79.
  6. [8]Chen, X.-H., Dempster, A. P., and Liu, J. S. (1994). Weighted finite population sampling to maximize entropy. Biometrika, 81:457–469.
  7. [9]Cinelli, C. and Hazlett, C. (2020). Making sense of sensitivity: Extending omitted variable bias. Journal of the Royal Statistical Society, Series B, 82:39–67.
  8. [10]de Chaisemartin, C. (2017). Tolerating defiance? Local average treatment effects without monotonicity. Quantitative Economics, 8:367–396.
  9. [11]de Chaisemartin, C. and D’Haultfœuille, X. (2018). Fuzzy differences-in-differences. Review of Economic Studies, 85:999–1028.
  10. [12]de Chaisemartin, C. and D’Haultfœuille, X. (2020). Two-way fixed effects estimators with heterogeneous treatment effects. American Economic Review, 110(9):2964–2996.
  11. [14]Duflo, E. (2001). Schooling and labor market consequences of school construction in Indonesia: Evidence from an unusual policy experiment. American Economic Review, 91(4):795–813.
  12. [15]Finkelstein, A. and Baicker, K. (2020). The Oregon health insurance experiment: Data. National Bureau of Economic Research. https://doi.org/10.60592/rx0z-sp89.
  13. [16]Finkelstein, A., Taubman, S., Wright, B., Bernstein, M., Gruber, J., Newhouse, J. P., Allen, H., Baicker, K., and Oregon Health Study Group (2012). The Oregon health insurance experiment: Evidence from the first year. Quarterly Journal of Economics, 127:1057–1106.
  14. [18]Hájek, J. (1964). Asymptotic theory of rejective sampling with varying probabilities from a finite population. Annals of Mathematical Statistics, 35:1491–1523.DOI
  15. [19]Hudson, S., Hull, P., and Liebersohn, J. (2017). Interpreting instrumented difference-in-differences. Working paper, Unpublished manuscript.
  16. [20]Imbens, G. W. and Angrist, J. D. (1994). Identification and estimation of local average treatment effects. Econometrica, 62:467–475.
  17. [21]Imbens, G. W. and Kolesár, M. (2016). Robust standard errors in small samples: Some practical advice. Review of Economics and Statistics, 98:701–712.
  18. [22]Imbens, G. W. and Manski, C. F. (2004). Confidence intervals for partially identified parameters. Econometrica, 72:1845–1857.
  19. [23]Imbens, G. W. and Rosenbaum, P. R. (2005). Robust, accurate confidence intervals with a weak instrument: Quarter of birth and education. Journal of the Royal Statistical Society, Series A, 168:109–126.DOI
  20. [24]Imbens, G. W. and Rubin, D. B. (2015). Causal Inference for Statistics, Social, and Biomedical Sciences: An Introduction. Cambridge University Press, Cambridge.
  21. [25]Kang, H., Peck, L., and Keele, L. (2018). Inference for instrumental variables: A randomization inference approach. Journal of the Royal Statistical Society, Series A, 181:1231–1254.arxiv.org/abs/1606.04146
  22. [27]Manski, C. F. and Pepper, J. V. (2018). How do right-to-carry laws affect crime rates? Coping with ambiguity using bounded-variation assumptions. Review of Economics and Statistics, 100:232–244.
  23. [28]Rambachan, A. and Roth, J. (2023). A more credible approach to parallel trends. Review of Economic Studies, 90:2555–2591.DOI
  24. [29]Rambachan, A. and Roth, J. (2026). Design-based uncertainty for quasi-experiments. Journal of the American Statistical Association, 121:477–491.DOI
  25. [31]Roth, J. and Sant’Anna, P. H. C. (2023). Efficient estimation for staggered rollout designs. Journal of Political Economy Microeconomics, 1:669–709.arxiv.org/abs/2102.01291
  26. [33]Splawa-Neyman, J. (1990). On the application of probability theory to agricultural experiments. Essay on principles. Section 9. Statistical Science, 5:465–472. Translated and edited by D. M. Dabrowska and T. P. Speed from the Polish original (1923).
  27. [34]Staiger, D. and Stock, J. H. (1997). Instrumental variables regression with weak instruments. Econometrica, 65:557–586.
  28. [35]Stoye, J. (2009). More on confidence intervals for partially identified parameters. Econometrica, 77:1299–1315.DOI
  29. [37]Taubman, S. L., Allen, H. L., Wright, B. J., Baicker, K., and Finkelstein, A. N. (2014). Medicaid increases emergency-department use: Evidence from Oregon’s health insurance experiment. Science, 343:263–268.

Paper details

vlnc:W_5myfwrgk
Version
v1 · latest
Published
Fresh — added to the Hub within the last 30 days
Added
License
CC BY 4.0
Usage

views · 0

downloads · 0

A view counts once per browser per day, a download each time the PDF is requested.

  • Economics, Econometrics and Finance

Not recorded for this paper

View statement
Looking for related papers…
Contents