betaFeedback

WorksMathematics

The precision of fractional factorial designs under randomization

Abstract

A regular fraction of a two-level factorial design runs fewer treatment combinations (cells), with more units in each. With the number of units fixed, more units per cell do not by themselves reduce the variance. Averaged over the fractions of one defining contrast subgroup, a fraction's variance is smaller than the full design's by the unit-to-unit variation of the aliased effects, a gain the usual variance estimator misses. Assuming the aliased effects are zero, we propose a full-design estimator that weights the fractions' contrasts by inverse variance. Its estimated variance never exceeds the saturated estimator's. In large samples, it is proportional to the harmonic mean of the fractions' average cell variances, and a fraction's, on average, to the arithmetic mean. We test this assumption, but choosing between the two estimators by the test can lose coverage. The results extend to unequal allocations, and the large-sample results to covariate adjustment.

Introduction

A 2K2^{K} factorial experiment assigns units to all combinations of KK two-level factors. When 2K2^{K} is large, experimenters often run a regular fraction, which uses 2K−p2^{K-p} of the treatment combinations (cells) (Box and Hunter, 1961a,b; Wu and Hamada, 2021, §5). With the same number of units, each cell in the fraction receives 2p2^{p} times as many units as a cell of the full design. However, this comes with the cost of aliasing. That is, each factorial effect outside the defining contrast subgroup is confounded with 2p−12^{p}-1 others, and the fraction estimates their signed sum. The usual justification is the effect hierarchy principle, under which the aliased interactions are negligible ([9] (Wu and Hamada 2021)).

We study whether running a fraction improves precision in the design-based framework, in which the potential outcomes and covariates are fixed, and the assignment is the sole source of randomness. Dasgupta, Pillai, and Rubin [5] developed this framework for 2K2^K designs, and Pashley and Bind [9] extended it to fractions. Under additivity, i.e., when every factorial effect is the same for every unit, Pashley and Bind [9] found that a fraction and the full design with the same number of units have the same variance. Zhao and Ding [12] compared a fraction with a full design, analyzed by a factor-unsaturated regression, via simulation, and left the theory to future research. Equal precision with a fixed number of units is also the classical conclusion under a linear model with constant error variance (e.g., [4]).

This article gives the comparison without additivity, for regular fractions and balanced designs with the same number of units. The extra units per cell do not reduce the variance by themselves. Averaged over the 2p2^p fractions with the same defining contrast subgroup, the variance of a fraction is smaller than that of the full design by the unit-to-unit variation of the aliased effects. The usual variance estimator does not detect this gain, and in our simulation, neither does the sharp variance bound of Aronow, Green, and Lee [1], when extended to a coset (§S8.8 of the Supplementary Material). Under the fraction’s assumption that the aliased effects are zero, we propose a weighted estimator under the full design, which combines the contrasts of these fractions with inverse-variance weights. In large samples, its estimated variance is proportional to the harmonic mean of the average cell variances of the fractions, while that of a fraction, on average, is proportional to their arithmetic mean. The estimated variance of the weighted estimator is also never larger than that of the saturated estimator, which needs no assumption on the aliased effects. Under the full design, we also provide a test for the fraction’s assumption. However, choosing between the weighted and saturated estimators by this test can lose coverage, as is common after a pretest [6]. In large samples, the results extend to covariate adjustment by the fully interacted regression of Lin [8]. The exact and large-sample results, and the test, extend to unequal allocations, for which Ravichandran et al. [10] derived the optimal allocations in 2K2^{K} designs. The comparisons with the fractions then hold when each cell of a fraction has 2p2^{p} times as many units as the same cell of the full design. When the potential outcomes do not depend on the stage, all results also hold when the cosets of the full design are run in separate stages planned in advance. A fold-over planned in advance is then the full design when p=1p=1, and a fraction with a smaller defining contrast subgroup when p≥2p \ge2 (§S7 of the Supplementary Material).

Setup

Factorial effects

Consider NN units and KK factors with levels −1-1 and +1+1. A treatment combination (cell) is z=(z1,…,zK)∈Z≡{−1,+1}Kz=(z_1,\ldots,z_K)\in\mathcal{Z}\equiv\{-1,+1\}^{K}, and unit ii has potential outcome Yi(z)Y_i(z). Write Y‾(z)=N−1∑i=1NYi(z)\overline{Y}(z)=N^{-1}\sum_{i=1}^{N}Y_i(z), and assume the stable unit treatment value assumption (SUTVA) holds throughout. A factorial effect is labeled by a subset kk of {1,…,K}\{1,\ldots,K\}, written as a word, e.g., 23 stands for {2,3}\{2,3\}, with the empty word being ∅\emptyset. For two words kk and ww, the product kwkw denotes their symmetric difference. The contrast of word kk is gk(z)=∏ℓ∈kzℓg_k(z)=\prod_{\ell\in k}z_\ell, with g∅(z)=1g_{\emptyset}(z)=1, so that gk(z)gw(z)=gkw(z)g_k(z)g_w(z)=g_{kw}(z). Following Dasgupta, Pillai, and Rubin [5], the factorial effects of word k≠∅k\ne\emptyset for unit ii and for the population respectively are

τi(k)=12K−1∑z∈Zgk(z)Yi(z),τ(k)=1N∑i=1Nτi(k).(1)\tau_i(k)=\frac{1}{2^{K-1}}\sum_{z\in\mathcal{Z}}g_k(z)Y_i(z),\qquad\tau(k)=\frac{1}{N}\sum_{i=1}^{N}\tau_i(k). \tag*{(1)}

For unit-level quantities b1,…,bNb_1,\ldots,b_N, write S2(b)=(N−1)−1∑i=1N(bi−b‾)2S^2(b)=(N-1)^{-1}\sum_{i=1}^{N}(b_i-\overline{b})^2 for their finite-population variance. Thus, S2{τ(k)}S^2\{\tau(k)\} is the variance of τ1(k),…,τN(k)\tau_1(k),\ldots,\tau_N(k), and the cell variance S2(z)≡S2{Y(z)}S^2(z) \equiv S^2\{Y(z)\} is that of Y1(z),…,YN(z)Y_1(z), \ldots, Y_N(z).

Regular fractions

A regular fraction is built from pp generator words w1,…,wpw_1,\ldots,w_p such that no product of a nonempty subset of them equals ∅\emptyset. The defining contrast subgroup GG is the set of J=2pJ=2^p products of subsets of the generators. Each sign vector t∈{−1,+1}pt\in\{-1,+1\}^p defines a coset {z∈Z:gwr(z)=tr, r=1,…,p}\{z\in\mathcal{Z}:g_{w_r}(z)=t_r,\ r=1,\ldots,p\}. Denote the JJ cosets by C1,…,CJC_1,\ldots,C_J, with C1C_1 given by t=(1,…,1)t=(1,\ldots,1). Each coset has 2K−p2^{K-p} cells, and the JJ cosets partition Z\mathcal{Z}. For w∈Gw\in G, the contrast gwg_w is constant on CjC_j, and denote its value by χj(w)\chi_j(w). A fraction runs the cells of one coset. For example, with K=3K=3 and the generator 123, the two cosets are C1={z:z1z2z3=1}C_1=\{z:z_1z_2z_3=1\} and C2={z:z1z2z3=−1}C_2=\{z:z_1z_2z_3=-1\}. Fix a target word k∉Gk\notin G. For w∈G∖{∅}w\in G\setminus\{\emptyset\}, the word kwkw is nonempty because k∉Gk\notin G, and gkw(z)=χj(w)gk(z)g_{kw}(z)=\chi_j(w)g_k(z) for z∈Cjz\in C_j. Thus, on the cells of one coset, the contrasts of kk and kwkw agree up to sign. We therefore call τi(kw)\tau_i(kw) and τ(kw)\tau(kw), for w∈G∖{∅}w\in G\setminus\{\emptyset\}, the aliased effects of unit ii and of the population. The coset contrast of unit ii and its population average respectively are

τ~j,i(k)=12K−p−1∑z∈Cjgk(z)Yi(z),τ~j(k)=1N∑i=1Nτ~j,i(k).\widetilde{\tau}_{j,i}(k)=\frac{1}{2^{K-p-1}}\sum_{z\in C_j}g_k(z)Y_i(z),\qquad\widetilde{\tau}_j(k)=\frac{1}{N}\sum_{i=1}^{N}\widetilde{\tau}_{j,i}(k).

Lemma 1. For every unit ii and every coset jj,

τ~j,i(k)=τi(k)+∑w∈G∖{∅}χj(w)τi(kw).(2)\widetilde{\tau}_{j,i}(k)=\tau_i(k)+\sum_{w\in G\setminus\{\emptyset\}}\chi_j(w)\tau_i(kw). \tag*{(2)}

§S1 of the Supplementary Material proves Lemma 1. The coset contrast is the target effect plus the J−1J-1 aliased effects τi(kw)\tau_i(kw), each with the sign χj(w)\chi_j(w).

Designs and estimators

Definition 1 (Designs). Let N/2K≥2N/2^K\geq2 be an integer. The full design D\mathcal{D} assigns the NN units by complete randomization to the 2K2^K cells of Z\mathcal{Z}, with N/2KN/2^K units in each cell. For

j=1,…,Jj = 1, \ldots, J, the fractional design DjD_j assigns the NN units by complete randomization to the 2K−p2^{K-p} cells of Cj\mathcal{C}_j, with N/2K−pN/2^{K-p} units in each cell.

Running the cosets of D\mathcal{D} in separate stages, after splitting the units completely at random, gives the same distribution of assignments as D\mathcal{D}. Therefore, the results below also hold for staged designs when the potential outcomes do not depend on the stage, including a fold-over planned in advance when p=1p = 1. For p≥2p \ge2, a fold-over is a fraction whose defining contrast subgroup has 2p−12^{p-1} words, and the results for fractions extend (§S7.3 of the Supplementary Material).

References

  1. [1]Aronow, P. M., Green, D. P., and Lee, D. K. K. (2014), “Sharp bounds on the variance in randomized experiments,” The Annals of Statistics, 42, 850–871.
  2. [4]Collins, L. M., Dziak, J. J., and Li, R. (2009), “Design of experiments with multiple independent variables: A resource management perspective on complete and reduced factorial designs,” Psychological Methods, 14, 202–224.DOI
  3. [5]Dasgupta, T., Pillai, N. S., and Rubin, D. B. (2015), “Causal inference from 2^K factorial designs by using potential outcomes,” Journal of the Royal Statistical Society: Series B (Statistical Methodology), 77, 727–753.
  4. [6]Leeb, H. and Pötscher, B. M. (2005), “Model selection and inference: Facts and fiction,” Econometric Theory, 21, 21–59.DOI
  5. [8]Lin, W. (2013), “Agnostic notes on regression adjustments to experimental data: Reexamining Freedman’s critique,” The Annals of Applied Statistics, 7, 295–318.DOI
  6. [9]Pashley, N. E. and Bind, M.-A. C. (2023), “Causal inference for multiple treatments using fractional factorial designs,” The Canadian Journal of Statistics, 51, 444–468.arxiv.org/abs/1905.07596
  7. [10]Ravichandran, A., Pashley, N. E., Libgober, B., and Dasgupta, T. (2024), “Optimal allocation of sample size for randomization-based inference from 2^K factorial designs,” Journal of Causal Inference, 12, 20230046.DOI
  8. [12]Zhao, A. and Ding, P. (2023), “Covariate adjustment in multiarmed, possibly factorial experiments,” Journal of the Royal Statistical Society: Series B (Statistical Methodology), 85, 1–23.arxiv.org/abs/2112.10557

Paper details

vlnc:W_cmyeg3sm
Version
v1 · latest
Published
Fresh — added to the Hub within the last 30 days
Added
License
CC BY 4.0
Usage

views · 3

downloads · 0

A view counts once per browser per day, a download each time the PDF is requested.

  • Mathematics

Not recorded for this paper

View statement
Looking for related papers…
Contents