WorksEconomics, Econometrics and Finance
Design-Based Inference for Fuzzy Difference-in-Differences
Abstract
I study fuzzy difference-in-differences designs from a design-based perspective, in which potential outcomes and treatment decisions are fixed and only the assignment of a policy is random, with heterogeneous and possibly unknown assignment probabilities. The Wald-DiD design estimand equals an assignment-weighted complier effect plus contamination from switchers, from changes in always-takers' effects and from trends in untreated outcomes. I derive the conditions under which the estimand holds a clear causal interpretation. The conditions are weaker than parallel trends in both outcomes and treatments. We propose an Anderson–Rubin-type test with a conservative Neyman variance that remains valid under weak first stages, a sensitivity analysis for the complier effect, and estimators for designs with known assignment probabilities. An application to the Oregon Health Insurance Experiment illustrates the results.
Introduction
Difference-in-differences (DiD) is a popular method for causal inference among applied social scientists. In its canonical format, there are two time periods. In the first period, all units are untreated, and in the second period, some units take treatment and other remain untreated. Under the parallel trends assumption, the DiD estimand identifies the average treatment effect on the treated (ATT). However, many difference-in-differences applications are fuzzy: a policy changes the incentive to take a treatment, but take-up is imperfect. Some units are treated before the policy and start treatment for reasons unrelated to it, while some units are exposed to the policy and remain untreated. The canonical estimator in this setting is the Wald-DiD ratio, which divides the DiD in outcomes by the DiD in treatment rates [14]. de Chaisemartin and D’Haultfœuille [11] showed that, in the conventional sampling-based framework, the Wald-DiD estimand identifies a local average treatment effect only under strong homogeneity restrictions: parallel trends must be combined with stability of treatment effects over time and homogeneity of effects between groups.
A growing literature argues that, in many DiD applications, uncertainty is more naturally attributed to the assignment of the policy than to sampling from a superpopulation [1, 5, 2, 31]. Rambachan and Roth [29] develop a design-based framework for quasi-experiments in which units have heterogeneous and unknown assignment probabilities, so that selection into treatment is permitted, and show how the biases of popular estimators can be expressed as finite-population covariances between assignment probabilities and potential outcomes. This paper brings that perspective to the fuzzy DiD framework.
I consider a two-period panel in which a policy (the assignment) becomes available in the second period. Treatment status is binary and monotone in both assignment and time, which partitions the population into never-takers, always-takers, compliers and switchers (units that adopt treatment in the second period irrespective of assignment).
I derive an exact causal decomposition of the design-based Wald-DiD estimand. It equals an assignment-probability-weighted average of complier effects plus three contamination terms: the effects of treatment on switchers, the change over time in the effects of treatment on always-takers, and trends in untreated outcomes. Each term enters the expression through its finite-population covariance with the assignment probabilities. All three vanish under complete randomization, so in the design-based framework the problems highlighted by de Chaisemartin and D’Haultfœuille [11] arise from the interaction between heterogeneity in assignment probabilities interacting and heterogeneity in effects and trends.
Second, I characterize the conditions under which the Wald-DiD design estimand equals the weighted complier effect: the assignment probabilities must be uncorrelated with a single contamination index that combines untreated trends, switcher effects and the drift in always-takers’ effects. This condition is necessary and sufficient, and it is strictly weaker than the pair of orthogonality conditions (design-based parallel trends for outcomes and for treatment) that would follow from applying existing results directly (Hudson et al., [19]; Rambachan and Roth, [29]). For example, it holds under design-based parallel trends and homogeneous effects even when switchers are concentrated among units likely to be assigned.
Third, I propose an Anderson–Rubin-type test based on a difference in means of the null-adjusted outcome change, studentized by a Neyman-type variance estimator. The test is asymptotically valid under the rejective-sampling design of Rambachan and Roth [29], with a conservative variance estimator whose slack we characterize, and its validity does not depend on the strength of the first stage. Inverting the test gives Fieller-type confidence sets, which are unbounded exactly when the first stage is not significant.
Fourth, I develop a sensitivity analysis for the complier effect. Bounds on the two contamination covariances deliver a closed-form identified set and a confidence set that is again robust to weak identification. Tests of the null of no complier effect depend only on the reduced-form contamination bound, which yields a simple breakdown value. When the assignment probabilities are known, as in stratified experiments and lotteries, the contamination terms are themselves estimable and the complier effect is point identified.
The results are illustrated with the Oregon Health Insurance Experiment [16, 37], in which the right to apply for Medicaid was allocated by lottery and compliance was imperfect: some individuals who were not selected enrolled through other routes, and selection probabilities differed by household size. Because the probabilities are known, the application serves as a benchmark: it measures the contamination that an analyst ignoring the heterogeneity in assignment probabilities would face.
Related literature
The paper contributes to the literature on DiD with heterogeneous effects (de Chaisemartin and D’Haultfœuille, 2018, 2020; Callaway and Sant’Anna, 2021; Goodman-Bacon, 2021; Sun and Abraham, 2021; Borusyak et al., 2024; see de Chaisemartin and D’Haultfœuille, 2023 and Roth et al., 2023 for reviews), and specifically to fuzzy DiD [11]. It also contributes to design-based inference [33, 24, 1, 5, 31, 29] (Li and Ding 2017), in particular for IV [23, 25]. We share the assignment model of Rambachan and Roth [29], who analyse two-stage least squares under strong-instrument asymptotics. Our contributions relative to their work are the fuzzy DiD decomposition and characterization, inference that is robust to weak identification [34, 3], and a sensitivity analysis that bounds the first-stage and reduced-form contamination jointly, in the spirit of Rosenbaum (2002), Manski and Pepper [27], Cinelli and Hazlett [9] and Rambachan and Roth [28].
The rest of the paper is organized as follows: § 2 introduces the framework. § 3 presents the identification results, § 4 inference, and § 5 the sensitivity analysis and the case of known assignment probabilities. § 6 summarizes simulations and § 7 presents the application. Proofs of the main results are in Appendix § 9. The Online Appendix contains the simulation design, additional simulation results and the proof of an auxiliary lemma.
Setup
Potential outcomes and assignment
We observe a balanced panel of units, , in two periods, . A policy becomes available in period 1. Let indicate whether unit is assigned to the policy in period 1; no unit is assigned in period 0. Treatment is binary. We write for treatment status in period 0 and for potential treatment status in period 1 under assignment . Potential outcomes are , . This notation, which is closer to the IV literature than that of de Chaisemartin and D’Haultfœuille [11], encodes two maintained restrictions.
Assumption 1 (Exclusion and no anticipation). Assignment affects outcomes only through treatment in both periods, and period-0 treatment and outcomes do not depend on period-1 assignment, so that the observed data are , , and .
Let be the unit-level effect in period , its change over time, and the untreated outcome trend. The first differences, as functions of assignment, are
and the observed differences are and .
Assumption 2 (Monotonicity in assignment and time). For all , (a) ; (b) for .
Part (a) is the usual IV monotonicity condition [20]. Part (b) rules out treatment discontinuation. Together they leave four treatment paths, which partition the population into the groups in Table 1. In particular, always-takers are exactly the units with , and their number is observed.
| Type | ||||||
| Never-takers | 0 | 0 | 0 | 0 | 0 | |
| Always-takers | 1 | 1 | 1 | 0 | 0 | |
| Compliers | 0 | 0 | 1 | 0 | ||
| Switchers | 0 | 1 | 1 | 1 | 0 |
Table 1. Compliance types under Assumption 2.
Following the design-based approach, we condition on the potential outcomes and treatments , so that the only source of randomness is the assignment vector . We adopt the assignment model of Rambachan and Roth [29].
Assumption 3 (Assignment design). For some fixed and unknown , possibly dependent on ,
Assumption 3 describes independent Bernoulli assignment conditional on the number of assigned units, also known as rejective or conditional Poisson sampling [18, 8]. We write , and for probabilities, expectations and variances over this distribution, , , and for the marginal assignment probability, so that . Because the may depend on potential outcomes and potential take-up, the design allows rich forms of selection into assignment. Complete randomization is the special case , which implies . Finite-population covariances are denoted .
Estimators and the design-based estimand
For any variable , let denote the difference in means. The reduced-form and first-stage DiD estimators are and , and the Wald-DiD estimator is . In a balanced two-period panel, and coincide with the two-way fixed effects DiD coefficients, and coincides with the IV estimator in a regression of on instrumented by . Define the weights
which satisfy and . Let and . Following Rambachan and Roth [29], the Wald-DiD design estimand is
The parameter is well defined given and the design, and and are unbiasedly estimable. The question is what means. The natural causal benchmark is the assignment-weighted complier effect
the expected average effect among assigned compliers. It is the design analogue of a local average treatment effect on the treated (the of Rambachan and Roth [29]). Under complete randomization it reduces to the complier average effect . Throughout we assume .
Identification
We begin with a general lemma that requires neither monotonicity nor exclusion.
Lemma 1 (Design expectation of a difference in means). Under Assumption 3, for any fixed potential responses ,
The first term is a -weighted average effect of assignment on . The second is a selection term that vanishes when assignment probabilities are uncorrelated with the response without assignment. Lemma 1, and hence Theorem 1 below, uses Assumption 3 only through the facts that is fixed and that ; both results therefore hold for any such design. Applying Lemma 1 to and and using Table 1 yields our main result. Define the design contamination terms
and the complier mass .
Theorem 1 (Decomposition of the Wald-DiD design estimand). Suppose Assumptions 1–3 hold. (a) The first stage and reduced form satisfy and ,
where
(b) If ,
(c) Let . If ,
if and only if .
Equation (8) isolates three channels through which departs from a causal parameter, which are the channels that de Chaisemartin and D’Haultfœuille [11] identify in the sampling-based framework. Switchers contribute the deviation of their effects from , weighted by . Always-takers contribute the change in their treatment effects between periods. Trends in untreated outcomes contribute whenever their covariance with the assignment probabilities is different from zero, which is a design-based failure of parallel trends. In the design-based framework, all three channels operate only through heterogeneity in .
Corollary 1 (Complete randomization). If for all , then , and , with no restriction on switchers, always-takers or trends.
Corollary 2 (Sufficient conditions). Under the conditions of Theorem 1, if (a) ; (b) ; and (c) . In particular, (a)–(c) hold if , is constant and for , even if .
Condition (a) is design-based parallel trends. Condition (b) holds, for example, if effects on always-takers are stable over time, and condition (c) if switchers’ effects equal . The last statement of Corollary 2 shows that switchers may be concentrated among units with high assignment probabilities. Conditions (a)–(c) of Corollary 2 are design-based counterparts of the parallel trends, stable treatment effect and homogeneous treatment effect assumptions of de Chaisemartin and D’Haultfœuille [11]. Unlike in their framework, none of them is needed under complete randomization (Corollary 1).
Remark 1 (Relation to existing conditions). Applied to first-differenced data, the IV results of Rambachan and Roth [29] give under , that is, design-based parallel trends in both potential outcomes and potential treatments. Theorem 1(c) shows that the sharp requirement is the single restriction , since .
Remark 2 (Negative weights). Theorem 1 can be written as
with for compliers and for switchers. The sum to one, but switchers with receive negative weight. Hence, even under design-based parallel trends, can be negative when every is positive. This parallels the negative-weights results of de Chaisemartin and D’Haultfœuille [11, 12].
Remark 3 (Testable implications and always-takers). Because and are predetermined, Lemma 1 implies that for any predetermined . Balance tests on (that is, on ), on or on pre-period trends are therefore informative about the covariance of with the components of , and Theorem 2 below delivers valid inference for them. Moreover, since always-takers are observed at baseline, one can drop units with . Conditional on the number of assigned units in the remaining subpopulation, the assignment vector there is again of the form in Assumption 3, and the term disappears.
Inference
Theorem 1 shows that interpreting as requires a covariance restriction that the design itself rarely justifies. We therefore develop inference for , which requires no such restriction, and return to the complier effect in § 5. Assumption 2 is not used in this section.
Test statistic
For a hypothesized value , let and . Since the difference in means is linear,
When , is equivalent to . Let and be the sample covariances of among assigned and unassigned units, with denominators and , and . The Neyman-type variance estimator of is
where and stand for and . Equivalently, is the HC2 variance of the coefficient on in an ordinary least squares regression of on ([21]). The Anderson–Rubin-type statistic is .
Asymptotic framework
We consider a sequence of finite populations indexed by , with all quantities implicitly depending on . Let , , and
Assumption 4 (Regularity). For a given , (a) there is such that for all and , and ; (b) for ; (c) .
Part (a) requires overlap. Part (b) is a standard moment bound for finite-population limit theorems ((Li and Ding, 2017)). Part (c) rules out degenerate cases in which has negligible design variance. Define , , and
Theorem 2 (Design-based Anderson–Rubin inference). Suppose Assumptions 1, 3 and 4 hold for a given . (a) , , and . (b) , and
(c) If for all , then for every .
Part (c) does not involve . The test therefore remains valid when the first stage is weak, for instance along sequences with , under which Wald inference fails ([34]). Part (b) shows that the variance estimator is conservative even when the effects of assignment on are constant, unless is constant. This mirrors Rambachan and Roth [29] for the difference in means. Sharper but more complex variance bounds in the spirit of Aronow et al. [4] could be developed.
Corollary 3 (Anderson–Rubin confidence set). Let . If and Assumption 4 holds at , . With , , and , , which is (a) the bounded interval , containing , if ; (b) the union of two disjoint half-lines if and ; (c) the real line if and . Case (a) occurs if and only if the first-stage -test rejects at level .
Corollary 4 (Wald inference under a strong first stage). If Assumption 4 holds at and , then, with , .
Remark 4 (Fixed populations and exact tests). The null is not sharp, so Fisher randomization tests are not directly available without further restrictions [23, 25]. In addition, unlike in a randomized experiment, the researcher does not know the . Berry–Esseen bounds for rejective sampling, as used by Rambachan and Roth [29], can replace the sequence-of-populations argument.
Sensitivity analysis and known designs
Bounded contamination
Theorem 1(a) gives the identity
so the complier effect is determined by the identified and the two contamination terms in (5). These terms have direct interpretations: is the covariance between assignment probabilities and the outcome changes units would experience without the policy, and is the covariance between assignment probabilities and switching. We bound them.
Assumption 5 (Bounded design contamination). and for known .
Proposition 1 (Identified set). Under Assumptions 1–3 and 5, . If , with
Theorem 3 (Sensitivity confidence set). Let
Under Assumptions – and , and Assumption at , . For , equals .
The confidence set requires no lower bound on the first stage and is computed by a one-dimensional grid search. Because it uses a two-sided critical value at every , it is conservative when is wide relative to sampling uncertainty; refinements in the spirit of Imbens and Manski [22] and Stoye [35] are possible.
Corollary 5 (Robustness of the null of no effect). if and only if , whatever the value of . Hence the breakdown value for rejecting is , or in covariance units.
Corollary has a simple practical message. Conclusions about the existence and sign of a complier effect depend only on how much the assignment probabilities may covary with outcome changes in the absence of the policy. Conclusions about its magnitude also depend on how they covary with switching.
Remark 5 (Calibration). By the Cauchy–Schwarz inequality,
where measures the departure from complete randomization and is the switcher share. The -weighted analogues of and are estimable from unassigned units. This reduces the analysis to a single interpretable parameter , possibly scaled by assumed correlations. Alternatively, following Rambachan and Roth [28], the covariance of with pre-period trends, which is estimable by Remark , can benchmark , and the decomposition (7) allows each channel to be bounded separately.
Remark 6 (Relaxing monotonicity and exclusion). Lemma does not use Assumption , and Theorems and use only identity (13). Time monotonicity is needed only to interpret and through compliance types; units that stop treatment enter and and require no change. If assignment monotonicity fails, with defiers and , then (13) holds with replaced by and by . Bounds and on thus enlarge to (cf. [10]). Similarly, a direct effect of assignment on period-1 outcomes with adds to , since , and enlarges to .
Known assignment probabilities
In stratified experiments and in lotteries with known odds, the are known, and the contamination terms are no longer nuisance parameters to be bounded but estimable quantities.
We state the result for stratified designs, which covers the application in § 7.
Assumption 6 (Stratified assignment). The units are partitioned into fixed strata of sizes , and is completely randomized within each stratum, independently across strata, with fixed assigned units in stratum .
Under Assumption 6, for , and is fixed, so Lemma 1 and Theorem 1 apply. Let be the mean of among units in stratum with , , and define the stratified and contamination estimators
which satisfy identically. Let , and let , where is the sample variance of among units in stratum with .
Proposition 2 (Known assignment probabilities). Suppose Assumptions 1, 2 and 6 hold. (a) , , and ; hence and are identified. (b) Suppose further that is fixed, that for some and , that the moment
bound of Assumption 4(b) holds within each stratum, and that .
Under , .
Part (a) holds without any restriction on trends, switchers or always-takers: with a known design, randomization alone identifies . The same argument applied to period-1 levels shows that the stratified IV estimator in levels has the same estimand, because for predetermined variables. The role of differencing is then only to improve precision, as in covariate adjustment. Part (b) gives a stratified Anderson–Rubin test that is again robust to a weak first stage. The estimators and measure the contamination that the pooled Wald-DiD estimator incurs by ignoring the heterogeneity in ; they depend only on unassigned units, whose outcome changes reveal and .
Simulation evidence
We draw one finite population per design, hold its potential outcomes and potential treatment decisions fixed, and redraw only the assignment, 5,000 times, from Assumption 3 with and for a unit characteristic . Compliance-type shares are for , and effects and trends depend on . In D1 assignment is completely randomized (). In D2 assignment probabilities are correlated with untreated trends. In D3 switchers are concentrated among high- units, with design-based parallel trends and homogeneous effects. D4 is as D3 but with heterogeneous effects. D5 has a weak first stage (, 4% compliers) under complete randomization. Section S2 of the Online Appendix gives the details and additional results.
Table 2 confirms the theory. In every design the AR set covers the design estimand at or above the nominal rate, up to simulation error. The variance estimator is conservative, more so when assignment probabilities are heterogeneous (D2–D4), as predicted by (12). Under complete randomization, equals the complier effect (Corollary 1). When assignment probabilities covary with untreated trends (D2), or with switching when effects are heterogeneous (D4), departs substantially from , and inference that treats as a complier effect fails. In D3, switchers are concentrated among likely-assigned units (), yet because , which illustrates that the condition in Theorem 1(c) is strictly weaker than . The sensitivity set with correctly specified bounds restores coverage of . With a weak first stage (D5), the Wald interval undercovers (78%) and the estimator is badly median biased, whereas the AR set retains nominal coverage.
| Bias | Coverage of | Coverage of | ||||||||
| Wald | AR | AR | Sens. | |||||||
| D1 | 1000 | 0.989 | 0.989 | 0.349 | 0.000 | 0.962 | 0.960 | 0.960 | 0.960 | 1.05 |
| D2 | 1000 | 2.561 | 1.254 | 0.377 | 0.006 | 0.985 | 0.985 | 0.000 | 0.997 | 1.57 |
| D3 | 1000 | 1.000 | 1.000 | 0.412 | 0.969 | 0.968 | 0.968 | 1.000 | 1.21 | |
| D4 | 1000 | 1.539 | 1.195 | 0.405 | 0.975 | 0.974 | 0.123 | 1.000 | 1.34 | |
| D5 | 200 | 1.034 | 1.079 | 0.020 | 2.288 | 0.782 | 0.951 | 0.951 | 0.952 | 1.01 |
Table 2. Design-based Monte Carlo results.
Note: Results over 5,000 assignment draws per design. Bias is the median of . Wald is the interval of Corollary 4, AR is and Sens. is , with bounds set at the true contamination.
The last column is the mean of divided by the simulation variance of . In D5, is unbounded in 94% of draws.
Empirical application: the Oregon Health Insurance Experiment
Setting and fit with the framework
In 2008 the state of Oregon reopened its Medicaid programme for low-income uninsured adults (OHP Standard) and allocated the right to apply by lottery among the individuals on a waiting list [16]. Eight drawings took place between March and September 2008. Selected individuals, together with the members of their household on the list, could apply, and about 30% of those selected enrolled. Taubman et al. [37] linked the list to emergency-department (ED) records from the Portland-area hospitals covering 1 January 2007 to 30 September 2009. We use their individual-level public-use data [15].
The mapping to our framework is direct. The units are the individuals in the ED sample, indicates selection in any drawing, and is the number of ED visits per year in period , where period 0 runs from 1 January 2007 to 9 March 2008 and period 1 from 10 March 2008, the earliest notification date, to 30 September 2009. The treatment indicates enrolment in any Medicaid programme during period 1, the first-stage measure of Taubman et al. [37]. The public-use files contain no measure of enrolment before the lottery, and we set . Then there are no always-takers, Assumption 2(b) holds trivially, and the always-taker channel in (7) is absent.
Three features make the experiment a good fit. First, compliance is imperfect in both directions. Many selected individuals did not enrol, and some individuals who were not selected enrolled through other routes, mostly in OHP Plus. The latter are switchers in Table 1, a group that a cross-sectional IV analysis classifies as always-takers. Second, the state drew individuals but extended selection to the whole household, a procedure that favoured larger households [16]. Assignment probabilities are therefore heterogeneous, which is precisely what generates the contamination terms of Theorem 1. Third, the ED data form a panel with outcomes measured before randomization.
Unlike in the quasi-experiments for which § 4 and § 5 are designed, the assignment probabilities are known up to the number of household members on the list (1, 2 or 3), within which selection is random. The application is therefore a benchmark. Proposition 2 identifies , so we can estimate the contamination that an analyst who ignores the heterogeneity in would face, compare the pooled Wald-DiD estimate with , and compare the estimated contamination with the breakdown value of § 5.
Two features of the lottery require adaptation. Because selection is at the household level, all variances treat households as the units of assignment and are computed from household totals of residuals within each stratum and arm. Because the number of selected individuals was not fixed in advance, we condition on it.
Results
The ED sample contains 24,615 individuals with non-missing outcomes in 22,159 households. The overall selection rate is , and the selection rates for one, two and three household members on the list are 0.344, 0.571 and 0.885, so that . Among individuals who were not selected, a share 0.151 enrolled in Medicaid during period 1; these are the switchers.
Table 3 reports the pooled estimators of § 2, which ignore the strata and estimate , and , and the stratified estimators of Proposition 2, which estimate , and . Selection raised Medicaid enrolment by 0.241, so the first stage is strong and the AR sets are bounded intervals. For ED visits per year, the stratified estimate of the complier effect is 0.292 (standard error 0.077), against a period-1 mean of 0.64 visits per year. among non-selected individuals, consistent with the increase in ED use found by Taubman et al. [37]. The pooled Wald-DiD estimate is 0.266 (0.077). Their difference, −0.026, estimates the bias of Theorem 1(b). Both contamination estimates are small: (0.005) and (0.001), so switching is essentially uncorrelated with the assignment probabilities. The breakdown value of Corollary 5 for the pooled analysis is , about four times . An analyst who did not know the design, but was willing to bound the reduced-form contamination at a few hundredths of a visit per year, would therefore have correctly concluded that Medicaid increases ED use among compliers.
| First stage | Reduced form | Wald-DiD | 95% AR set | |||
| ED visits per year | ||||||
| Pooled (ignores strata) | 0.240 | 0.064 | 0.266 | [0.115, 0.417] | −0.007 | −0.001 |
| (0.006) | (0.018) | (0.077) | (0.005) | (0.001) | ||
| Stratified (known design) | 0.241 | 0.071 | 0.292 | [0.142, 0.443] | ||
| (0.006) | (0.019) | (0.077) | ||||
| Any ED visit | ||||||
| Pooled (ignores strata) | 0.240 | 0.014 | 0.060 | [0.003, 0.116] | −0.002 | −0.001 |
| (0.006) | (0.007) | (0.029) | (0.002) | (0.001) | ||
| Stratified (known design) | 0.241 | 0.016 | 0.068 | [0.011, 0.125] | ||
| (0.006) | (0.007) | (0.029) |
Table 3. Medicaid and emergency-department use in Oregon. Note: Pooled rows report , , and (§ 4). Stratified rows report , , and the stratified AR set of Proposition 2. The last two columns report the contamination estimators and in (14). Strata are household size on the lottery list. The treatment is Medicaid enrolment. Standard errors in parentheses treat households as the units of assignment. ED visits per year are visit counts, truncated and censored as in the public-use files, divided by the length of each period in years. Any ED visit compares an indicator for any visit in period 1 with the same indicator in period 0, whose windows differ in length. Sample: 24,615 individuals in the ED sample of Taubman et al. [37].
Differencing is what makes the pooled analysis robust. Larger households, which were more likely to be selected, used the ED less before the lottery: the pooled difference in pre-period ED visits per year between selected and non-selected individuals is −0.070 (0.020), whereas the stratified difference is −0.011 (0.020), consistent with successful randomization within strata (Remark 3). Consequently, the pooled IV estimate in period-1 levels is −0.026 (0.085), whereas the stratified levels estimate, whose estimand is again , is 0.245 (0.081). Selection on household size thus enters levels strongly but trends only weakly, and the Wald-DiD estimand inherits only the latter. The any-visit outcome gives the same qualitative picture.
Conclusion
This paper develops a design-based framework for the fuzzy difference-in-differences setting. This approach is motivated by settings in which researchers wish to remain agnostic about the statistical process behind the sampling and formation of potential outcomes and treatments. Viewed through the lens of the assignment mechanism, fuzzy DiD has a transparent structure. The Wald-DiD design estimand equals an assignment-weighted complier effect plus contamination from switchers, always-takers and trends in untreated outcomes, all of which affect the estimand through the covariance between assignment probabilities and a single contamination index. When this covariance cannot be argued to vanish, the design still supports valid, weak-identification-robust and conservative inference on the Wald-DiD estimand, and a sensitivity analysis for the complier effect whose conclusions about the existence of an effect depend only on the reduced-form contamination. When the design is known, the contamination is estimable and the complier effect is identified. Extensions to staggered adoption ([5, 31]), clustered assignment, covariate adjustment and alternative fuzzy DiD estimands such as the time-corrected Wald ratio of de Chaisemartin and D’Haultfœuille [11] are natural next steps.
Proofs
Throughout, and all sums run over unless indicated otherwise. We use the following properties of the rejective design. Their proof is routine and is given in Section S3 of the Online Appendix.
Lemma 2. Suppose Assumption 3 holds and let . (a) For , ; consequently, for any real weights , . (b) If , then for any array with , . (c) If, in addition, , then .
Proofs for Section 3
Proof of Lemma 1. Since and are fixed, . Adding and subtracting gives .
For the second claim, . Since , the mean of is and .
Proof of Theorem 1. (a) By Table 1, and , so Lemma 1 gives . Likewise, and . Hence with as in (7), and by (4). Finally, because and for all .
(b) From (a), . Dividing by and substituting (6) and (7) gives (8). (c) by Lemma 1. Since , part (b) shows that if and only if this quantity is zero.
Proof of Corollaries 1 and 2. If , then and , so , and , and Theorem 1(a) gives Corollary 1. For Corollary 2, is the sum of the three covariances in (a)–(c) by linearity, so Theorem 1(c) applies. If for all and for , then and (b) and (c) hold trivially, whatever the value of .
Proofs for Section 4
Proof of Theorem 2. Fix and suppress it from the notation, writing and .
Step 1 (exact representation). Since is fixed, . Taking expectations, , so .
Step 2 (asymptotic normality). Assumption 4(a) gives .
By Lemma 2(b) with , . Assumption 4(b) implies for some , and hence .
Because for some and large , and . By Assumption 4(c), for some , so the ratio in Lemma 2(c) is , and part (a) follows.
Step 3 (variance estimator). Let . Then and
First, . Since , , and Lemma 2(a) gives . Second, and, by Lemma 2(a), , so . Therefore and, analogously, . Since (expression (9) is the same quantity computed from the sample moments of and ),
Step 4 (conservativeness). Let , so that .
Direct expansion gives
Summing over , equals the right-hand side of (12). Since minimizes , , which proves (12).
Step 5 (test validity). Let , so that by Step 2.
Under , . By Steps 2–4 and ,
For any , . The second term vanishes, so . Letting with proves part (c).
Proof of Corollary 3. Since , , and Theorem 2(c) at gives the coverage statement. Expanding with (9) gives . If , the quadratic is convex and equals at , because ; hence and the set is the stated interval. If , the quadratic is concave, and its sublevel set is the complement of the open interval between the real roots when and the whole line when . Finally, if and only if .
Proof of Corollary 4. Since , , and the event is . Steps 2 and 3 of the proof of Theorem 2 applied to give . In addition, , so and . Since and ,
Step 5 of the proof of Theorem 2 then applies with in place of .
Proofs for Section 5
Proof of Proposition 1. By (13), with , , and ; hence . If , is continuous on the compact and connected box , so its image is a compact interval. For fixed , is strictly decreasing in . For fixed , has constant sign, so is monotone in . The minimum is therefore attained at and , and the maximum at and .
Proof of Theorem 3. By (13), , so under Assumption 5. Hence
Steps 1–5 of the proof of Theorem 2, applied at to the centred statistic , show that the right-hand side has limit superior at most . The case is immediate from the definitions.
Proof of Corollary 5. At , , and .
Proof of Proposition 2. (a) Under Assumption 6, . Hence
since for , and similarly
By Table 1, , , and . The identity follows from , and the expression for is Theorem 1(b).
(b) Under , part (a) gives . Write with and the mean of in stratum and arm . The summands are independent across strata. Within each stratum, assignment is completely randomized with , and the moment bound implies the conditions of the finite-population central limit theorem of Li and Ding (2017, Theorem 1). Since is fixed and is bounded away from zero, . Within stratum , , where is the finite-population variance of in stratum and that of ; and (Li and Ding, 2017). Hence , and the argument of Step 5 of the proof of Theorem 2 completes the proof.
References
- [1]Abadie, A., Athey, S., Imbens, G. W., and Wooldridge, J. M. (2020). Sampling-based versus design-based uncertainty in regression analysis. Econometrica, 88:265–296.DOI
- [2]Abadie, A., Athey, S., Imbens, G. W., and Wooldridge, J. M. (2023). When should you adjust standard errors for clustering? Quarterly Journal of Economics, 138:1–35.
- [3]Andrews, I., Stock, J. H., and Sun, L. (2019). Weak instruments in instrumental variables regression: Theory and practice. Annual Review of Economics, 11:727–753.DOI
- [4]Aronow, P. M., Green, D. P., and Lee, D. K. K. (2014). Sharp bounds on the variance in randomized experiments. Annals of Statistics, 42:850–871.
- [5]Athey, S. and Imbens, G. W. (2022). Design-based analysis in difference-in-differences settings with staggered adoption. Journal of Econometrics, 226:62–79.
- [8]Chen, X.-H., Dempster, A. P., and Liu, J. S. (1994). Weighted finite population sampling to maximize entropy. Biometrika, 81:457–469.
- [9]Cinelli, C. and Hazlett, C. (2020). Making sense of sensitivity: Extending omitted variable bias. Journal of the Royal Statistical Society, Series B, 82:39–67.
- [10]de Chaisemartin, C. (2017). Tolerating defiance? Local average treatment effects without monotonicity. Quantitative Economics, 8:367–396.
- [11]de Chaisemartin, C. and D’Haultfœuille, X. (2018). Fuzzy differences-in-differences. Review of Economic Studies, 85:999–1028.
- [12]de Chaisemartin, C. and D’Haultfœuille, X. (2020). Two-way fixed effects estimators with heterogeneous treatment effects. American Economic Review, 110(9):2964–2996.
- [14]Duflo, E. (2001). Schooling and labor market consequences of school construction in Indonesia: Evidence from an unusual policy experiment. American Economic Review, 91(4):795–813.
- [15]Finkelstein, A. and Baicker, K. (2020). The Oregon health insurance experiment: Data. National Bureau of Economic Research. https://doi.org/10.60592/rx0z-sp89.
- [16]Finkelstein, A., Taubman, S., Wright, B., Bernstein, M., Gruber, J., Newhouse, J. P., Allen, H., Baicker, K., and Oregon Health Study Group (2012). The Oregon health insurance experiment: Evidence from the first year. Quarterly Journal of Economics, 127:1057–1106.
- [18]Hájek, J. (1964). Asymptotic theory of rejective sampling with varying probabilities from a finite population. Annals of Mathematical Statistics, 35:1491–1523.DOI
- [19]Hudson, S., Hull, P., and Liebersohn, J. (2017). Interpreting instrumented difference-in-differences. Working paper, Unpublished manuscript.
- [20]Imbens, G. W. and Angrist, J. D. (1994). Identification and estimation of local average treatment effects. Econometrica, 62:467–475.
- [21]Imbens, G. W. and Kolesár, M. (2016). Robust standard errors in small samples: Some practical advice. Review of Economics and Statistics, 98:701–712.
- [22]Imbens, G. W. and Manski, C. F. (2004). Confidence intervals for partially identified parameters. Econometrica, 72:1845–1857.
- [23]Imbens, G. W. and Rosenbaum, P. R. (2005). Robust, accurate confidence intervals with a weak instrument: Quarter of birth and education. Journal of the Royal Statistical Society, Series A, 168:109–126.DOI
- [24]Imbens, G. W. and Rubin, D. B. (2015). Causal Inference for Statistics, Social, and Biomedical Sciences: An Introduction. Cambridge University Press, Cambridge.
- [25]Kang, H., Peck, L., and Keele, L. (2018). Inference for instrumental variables: A randomization inference approach. Journal of the Royal Statistical Society, Series A, 181:1231–1254.arxiv.org/abs/1606.04146
- [27]Manski, C. F. and Pepper, J. V. (2018). How do right-to-carry laws affect crime rates? Coping with ambiguity using bounded-variation assumptions. Review of Economics and Statistics, 100:232–244.
- [28]Rambachan, A. and Roth, J. (2023). A more credible approach to parallel trends. Review of Economic Studies, 90:2555–2591.DOI
- [29]Rambachan, A. and Roth, J. (2026). Design-based uncertainty for quasi-experiments. Journal of the American Statistical Association, 121:477–491.DOI
- [31]Roth, J. and Sant’Anna, P. H. C. (2023). Efficient estimation for staggered rollout designs. Journal of Political Economy Microeconomics, 1:669–709.arxiv.org/abs/2102.01291
- [33]Splawa-Neyman, J. (1990). On the application of probability theory to agricultural experiments. Essay on principles. Section 9. Statistical Science, 5:465–472. Translated and edited by D. M. Dabrowska and T. P. Speed from the Polish original (1923).
- [34]Staiger, D. and Stock, J. H. (1997). Instrumental variables regression with weak instruments. Econometrica, 65:557–586.
- [35]Stoye, J. (2009). More on confidence intervals for partially identified parameters. Econometrica, 77:1299–1315.DOI
- [37]Taubman, S. L., Allen, H. L., Wright, B. J., Baicker, K., and Finkelstein, A. N. (2014). Medicaid increases emergency-department use: Evidence from Oregon’s health insurance experiment. Science, 343:263–268.