betaFeedback

WorksMathematics

Ratio-of-mediator-probability weighting for causal mediation analysis: estimation, extensions, and practical guidance

Abstract

Ratio-of-mediator-probability weighting (RMPW) estimates natural direct and indirect effects by reweighting mediator distributions across treatment conditions rather than modeling the outcome, thereby accommodating treatment-by-mediator interaction. This article brings existing RMPW methods together within a common estimation framework, incorporates a heteroskedastic normal model for continuous mediators, and extends two-step variance estimation to multiple parametric weight components. The framework is illustrated using a national health survey to decompose disparities in cigarette smoking by sexual-minority status and a randomized welfare-to-work experiment to examine pathways from program assignment through employment to maternal depressive symptoms. Targeted simulations evaluate model specification, alternative weighting strategies, treatment-selection adjustment, weight stability, variance estimation, and continuous treatments. Parametric RMPW performed well when models were adequately specified and weights were stable, but correct specification alone did not prevent unstable estimation from concentrated weights. Stratification-based weighting provided some robustness to model misspecification. Standard errors that treated estimated weights as fixed were unreliable, whereas two-step and bootstrap inference performed well under appropriate conditions. The results provide practical guidance for selecting weighting strategies, diagnosing instability, and conducting inference. The methods are implemented in the community-contributed Stata command mmwsmed.

Introduction

Evaluations of health care interventions typically focus on whether an intervention changes an outcome, but an estimate of the overall treatment effect does not explain how that effect was produced. Many interventions are explicitly designed to operate through intermediate processes, for example, by changing patient behavior, treatment adherence, engagement with care, or other modifiable mechanisms that subsequently affect health outcomes. Mediation analysis examines these causal pathways by decomposing the overall effect of a treatment into components operating through a mediator of interest and through alternative pathways. Understanding these mechanisms can help confirm that an effective intervention operates as intended and, when an intervention fails, identify where the hypothesized causal pathway breaks down and how the intervention might be improved [19, 31].

Methods for conducting mediation analysis have expanded substantially and can be viewed as arising from three broad analytic conventions. Parametric regression and structural equation models estimate mediated effects through models for the mediator and outcome, commonly using products or differences of regression coefficients [2, 21, 34]. The potential-outcomes framework instead defines direct and indirect effects as contrasts between counterfactual outcomes and makes explicit the assumptions required for their causal identification [28, 15] (Pearl 2001). Within the potential-outcomes framework itself, a large collection of estimators has developed from combinations of weighting, model-based prediction, and hybrid strategies, with different modeling requirements and robustness properties [25]. More recently, flexible nonparametric and machine-learning approaches have provided alternatives that reduce reliance on prespecified functional forms, permit data-adaptive estimation of nuisance functions, and can accommodate complex nonlinearities and interactions [20, 6, 7]. Thus, the proliferation of mediation methods reflects not only alternative estimation strategies but also differences in the causal structures, identifying assumptions, and scientific questions they are designed to address.

Within the important setting in which natural direct and indirect effects are identified, Hong [8] and Hong et al. [11] developed ratio-of-mediator-probability weighting (RMPW). The premise is to estimate counterfactual potential-outcome means by reweighting the observed mediator distribution under one treatment condition to resemble the distribution it would have had under another. For example, the counterfactual outcome that treated individuals would have had if their mediator had followed its distribution under control is estimated by weighting treated individuals by the ratio of the mediator’s probability under control to its probability under treatment. Because the outcome itself is not modeled, RMPW accommodates treatment-by-mediator interaction without requiring that interaction to be specified in an outcome model and can be used with different types of outcomes [11]. Hong subsequently developed a broad framework encompassing parametric and stratification-based nonparametric mediator weights, multivalued and nonrandom treatments, moderated mediation, and multiple mediators [11]. Other work has extended the available estimation tools: Bein et al. [3] developed two-step variance estimation that accounts for estimation of the weights; mediator-probability ratios can alternatively be expressed through treatment odds [32, 14]; and generalized propensity-score weighting extends mediation weighting to continuous treatments [13].

This increasingly broad framework gives investigators considerable flexibility, but it also creates consequential choices in applied analyses. Mediator weights may be constructed from a parametric model, from stratification, or through treatment odds; nonrandom treatment requires an additional treatment-selection model; continuous mediators require specification of a conditional density; and continuous treatments require local weighting and a bandwidth choice. Inference must additionally account for the fact that the weights themselves are estimated. These choices involve tradeoffs between model specification, robustness, and weight stability that are not apparent from asymptotic identification results alone. The simulations of Hong et al. [11] demonstrated good performance of RMPW in the original binary-treatment, binary-mediator setting and showed that parametric mediator weights can be sensitive to model misspecification. However, the RMPW framework has since come to encompass a much broader range of mediator, outcome, and treatment types and weighting strategies, for which finite-sample performance has received less systematic evaluation. This is particularly important for weighting estimators because a correctly specified model can nevertheless generate highly variable weights and unstable estimation [5, 1].

This article makes three contributions. First, it brings these developments together within a unified estimation framework. For discrete treatments, the basic estimator and its extensions share a single data-duplication estimator and differ primarily in how the component weights are constructed. Second, it incorporates a heteroskedastic normal mediator model that jointly estimates the conditional mean and variance of a continuous mediator, rather than obtaining them from separately fitted conditional moments. It also extends the two-step variance estimator of Bein et al. [3] to propagate uncertainty from all parametric models used to construct mediator and treatment-selection weights. Third, targeted simulation experiments evaluate the finite-sample consequences of the principal choices available within the framework. The experiments examine mediator-model misspecification, alternative weighting strategies, treatment confounding, weight instability, continuous treatments, and variance estimation. The resulting evidence is used to develop practical guidance for choosing models and weights, diagnosing instability, selecting a variance estimator, and reporting an RMPW analysis. The methods are implemented in mmwsmed, a community-contributed Stata command [18].

Section 2 defines the causal estimands and identification assumptions and develops RMPW estimation and its extensions. Section 3 illustrates the framework using a national health survey and a randomized welfare-to-work experiment; Section 4 describes the targeted simulation experiments, and Section 5 reports their results; Section 6 translates the findings into practical guidance for investigators and discusses limitations; and Section 7 concludes.

Methods

Causal estimands and identification

Let TT denote treatment, with levels 0,1,…,L−10, 1, \ldots, L - 1, one of which, cc, is the control condition; let MM denote the mediator, YY the outcome, and X\mathbf{X} a vector of pretreatment covariates. For each individual, M(t)M(t) denotes the potential value of the mediator under treatment level tt, and Y(t,m)Y(t,m) the potential outcome under treatment tt and mediator value mm. The composite potential outcome Y(t,M(t′))Y(t, M(t')) is the outcome under treatment tt when the mediator takes the value it would have taken under treatment t′t' [28, 15, 34, 24] (Pearl 2001). The causal effects considered here are linear contrasts of population potential-outcome means E{Y(a,M(b))}E\{Y(a, M(b))\}.

For a treatment level t≠ct \ne c, four potential-outcome means define the effects considered in this article. The means E{Y(t,M(t))}E\{Y(t, M(t))\} and E{Y(c,M(c))}E\{Y(c, M(c))\} are the mean outcomes under treatment and control, respectively. The cross-world mean E{Y(t,M(c))}E\{Y(t, M(c))\} is the mean outcome if treatment were set to tt but the mediator followed the distribution it would have under control, and E{Y(c,M(t))}E\{Y(c, M(t))\} is defined analogously with treatment set to control and the mediator following its distribution under treatment. Table 2 summarizes the resulting effects.

EffectDefinitionInterpretation
TEYtt−YccY_{tt} - Y_{cc}total effect of the treatment
NIEYtt−YtcY_{tt} - Y_{tc}effect transmitted through the mediator, under treatment
NDEYtc−YccY_{tc} - Y_{cc}effect not transmitted through the mediator, with the mediator at its control value
PNIEYct−YccY_{ct} - Y_{cc}effect transmitted through the mediator, under control
TNDEYtt−YctY_{tt} - Y_{ct}effect not transmitted through the mediator, with the mediator at its treated value
INTNIE −- PNIE == TNDE −- NDEnatural treatment-by-mediator interaction

Table 2. Causal effects considered for treatment level tt compared with the control level cc. Yab=E[Y(a,M(b))]Y_{ab} = \mathrm{E}[Y(a, M(b))]. With a continuous treatment, tt and cc are the evaluation dose dtd_t and the control dose d0d_0.

For a treatment level t≠ct \ne c compared with the control condition, the total effect is

TE=E{Y(t,M(t))}−E{Y(c,M(c))}.\mathrm{TE} = E\{Y(t, M(t))\} - E\{Y(c, M(c))\}.

The natural indirect effect and natural direct effect are

NIE=E{Y(t,M(t))}−E{Y(t,M(c))},\mathrm{NIE} = E\{Y(t, M(t))\} - E\{Y(t, M(c))\},

and

NDE=E{Y(t,M(c))}−E{Y(c,M(c))},\mathrm{NDE}=E\{Y(t,M(c))\}-E\{Y(c,M(c))\},

so that TE=NIE+NDE\mathrm{TE}=\mathrm{NIE}+\mathrm{NDE}. The alternative decomposition uses the pure natural indirect effect

PNIE=E{Y(c,M(t))}−E{Y(c,M(c))},\mathrm{PNIE}=E\{Y(c,M(t))\}-E\{Y(c,M(c))\},

and the total natural direct effect

TNDE=E{Y(t,M(t))}−E{Y(c,M(t))},\mathrm{TNDE}=E\{Y(t,M(t))\}-E\{Y(c,M(t))\},

so that TE=PNIE+TNDE\mathrm{TE}=\mathrm{PNIE}+\mathrm{TNDE}. The natural treatment-by-mediator interaction is

INT=NIE−PNIE=TNDE−NDE,\mathrm{INT}=\mathrm{NIE}-\mathrm{PNIE}=\mathrm{TNDE}-\mathrm{NDE},

which measures the extent to which the indirect effect depends on treatment, or equivalently the direct effect depends on the mediator distribution (Hong et al., 2015; Hong, 2015; VanderWeele, 2014). These alternative decompositions are useful when treatment and mediator interact in their effects on the outcome, because the natural indirect and direct effects then depend on the treatment level or mediator distribution at which they are defined.

For a multivalued treatment, these effects are defined separately for every treatment level tt relative to the control level cc. For a continuous treatment, tt and cc are replaced by the evaluation dose dtd_t and control dose d0d_0. Extensions of RMPW to multiple mediators, moderated mediation, and controlled direct effects are described in the Supplementary Material; their finite-sample performance is not evaluated here.

Identification of natural direct and indirect effects requires assumptions linking these counterfactual quantities to the observed data [28, 15, 34, 24]. Conditional on pretreatment covariates X\mathbf{X}, these require that (i) treatment assignment is independent of the potential mediator and outcomes; (ii) the mediator is independent of the potential outcomes conditional on treatment; and (iii) no posttreatment variable confounds the mediator–outcome relationship. The last condition is particularly important because natural direct and indirect effects are generally not identified when a variable affected by treatment also confounds the mediator–outcome relationship [35].

Positivity requires a positive probability or density of the relevant treatment and mediator values at covariate patterns represented in the target population, and consistency requires the observed mediator and outcome to equal their corresponding potential values under the treatment actually received. When treatment is randomized, the treatment ignorability condition holds by design, but mediator assignment remains nonrandom; X\mathbf{X} must therefore still contain pretreatment confounders of the mediator–outcome relationship [15, 31] (Hong 2015). When treatment is observational, X\mathbf{X} must additionally contain sufficient covariates to control treatment confounding.

Ratio-of-mediator-probability weighting

Under these assumptions, E{Y(t,M(t))}E\{Y(t,M(t))\} and E{Y(c,M(c))}E\{Y(c,M(c))\} are identified from the observed treatment groups, with additional treatment-selection weighting when treatment is not randomized. The cross-world mean E{Y(t,M(c))}E\{Y(t,M(c))\} is estimated from individuals who received treatment tt using the ratio-of-mediator-probability weight

ωt→c=Pr⁡(M=m∣T=c,X)Pr⁡(M=m∣T=t,X)(1)\omega_{t\to c}=\frac{\Pr(M=m\mid T=c,\mathbf{X})}{\Pr(M=m\mid T=t,\mathbf{X})} \tag*{(1)}

where mm is the individual’s observed mediator value [8] (Hong et al. 2015; Hong 2015). For a continuous mediator, the probabilities in Equation (1) are replaced by conditional densities. The weight transforms the mediator distribution among individuals receiving treatment tt into the distribution it would have had under control, conditional on X\mathbf{X}. The reverse weight ωc→t\omega_{c\to t} applied to the control group identifies E{Y(c,M(t))}E\{Y(c,M(t))\}, which is required for the PNIE, TNDE, and INT.

The essential attraction of RMPW is that this reweighting leaves the observed outcome-generating process within each treatment group intact. Consequently, estimation of the natural effects does not require an outcome model and permits the relation between the mediator and outcome to differ by treatment [11, 11, 25].

Following Hong (2015), estimation proceeds by data duplication. For each requested potential-outcome mean E{Y(a,M(b))}E\{Y(a,M(b))\}, the observations at treatment level aa are copied and assigned weight ωa→b\omega_{a\to b}, equal to one when a=ba=b, multiplied by any treatment-selection weight. The required copies are stacked, and a single weighted outcome regression on indicators for the copies estimates the potential-outcome means jointly. The causal effects are linear contrasts of these means.

More generally, the weight assigned to individual ii in copy cc can be written as

Wic=uiτi∏kωick,(2)W_{ic}=u_i\tau_i\prod_k\omega_{ick}, \tag*{(2)}

where uiu_i is a sampling weight, τi\tau_i is a treatment-selection weight, and ωick\omega_{ick} is the mediator-probability or density ratio for mediator kk required by that copy. When sampling weights are required for population inference, they should also be incorporated when estimating the models used to construct the mediation weights [4]. Components that are not required for a particular analysis equal one. Thus, the discrete-treatment RMPW extensions considered here share the same data-duplication estimator and differ primarily in how the component weights are constructed; the continuous-treatment estimator of § 2.4.1 replaces these weights with kernel-localized generalized propensity-score weights. This common representation also provides the basis for the extension of two-step variance estimation described in § 2.4.2.

Extensions and estimation choices

The basic RMPW construction requires choices about how the mediator distribution is represented, how treatment selection is handled when treatment is not randomized, and how uncertainty in the estimated weights is incorporated into inference. Continuous treatments require a related generalized propensity-score construction. Table 1 summarizes the methodological basis for these components, the developments introduced here, and the aspects evaluated in the simulation experiments. These choices are described below independently of the simulation designs used later to evaluate them.

Table 1: Components of the RMPW framework
ComponentPrevious basisDeveloped hereEvaluated here
Basic RMPWHong [8]; Hong et al. (2015); Hong (2015)Common weight representation (Equation (2))Replication of Hong et al. (2015); continuous mediators, binary and count outcomes, and multivalued treatments
Nonparametric RMPWHong et al. (2015); Hong (2015)—Number of strata and residual bias; misspecification designs extending those of Hong et al. (2015); bootstrap inference
Continuous mediatorSeparately estimated conditional moments (Hong, 2015, sec. 12.1.2)Heteroskedastic normal model: joint maximum-likelihood mean and log-variance equations, with guaranteed positive fitted varianceConstant-variance, heteroskedastic, and odds-ratio weights under increasing heteroskedasticity
Odds-ratio mediator weightsTchetgen Tchetgen [32]; Huber [14]—Comparison with ratio weights for continuous mediators and ordinal treatments; bootstrap inference
Nonrandom treatmentHong (2015, sec. 12.2.1); Hong [9]; Linden [17]—IPTW versus MMWS; omitted treatment-selection adjustment; binary, multivalued, and ordinal treatments; heteroskedastic treatment assignment
Continuous treatmentHsu et al. [13]—Default versus halved bandwidth
Two-step varianceBein et al. [3]Extension to stacked estimating equations for all parametric weight components (mediator, treatment-selection, marginal-share, and odds-ratio models)Comparison with known-weight and bootstrap standard errors across designs
Weight diagnosticsKish [16]; general weighting literature [5, 1]—Correspondence of effective sample size and weight concentration with estimator instability

Table 1. Components of the RMPW framework: previous methodological basis, methodological development in this article, and finite-sample evaluation in this article.

Mediator weights

Parametric ratio weights. For the RMPW estimator in Equation (1), the conditional distribution of the mediator is modeled separately within each treatment level [8] (Hong et al. 2015; Hong 2015). The fitted models provide, for every observation, the probability or density of its observed mediator value under each treatment level, and ratios of these predictions form the mediator weights. The model therefore depends on the scale and distribution of the mediator: binary and ordinal mediators require probability models, count mediators may be represented by an appropriate count distribution, and continuous mediators require a conditional density. With a multivalued treatment, a separate mediator model is fitted within each treatment level.

For a continuous mediator, a normal linear model commonly assumes constant conditional variance within each treatment level. Hong’s construction allows the variance to depend on covariates by estimating E(M∣X)E(M \mid X) and E(M2∣X)E(M^2 \mid X) separately and obtaining the conditional variance from E(M2∣X)−{E(M∣X)}2E(M^2 \mid X) - \{E(M \mid X)\}^2 (Hong, 2015, sec. 12.1.2). Because separately fitted mean equations need not imply a positive variance for every observation, this construction can yield negative fitted variances.

Here I instead use a heteroskedastic normal model in which the conditional mean and variance are estimated jointly:

M∣X,T=t∼N{x′βt,exp⁡(z′γt)},M \mid X,T=t \sim N\{x^{\prime}\beta_t,\exp(z^{\prime}\gamma_t)\},

where x contains covariates of the conditional mean and z contains covariates of the conditional variance. The mean and variance equations are estimated jointly by maximum likelihood; for treatment level tt, the log-likelihood is ℓt(βt,γt)=−12∑i:Ti=twi{log⁡(2π)+zi′γt+(Mi−xi′βt)2/exp⁡(zi′γt)}\ell_t(\beta_t,\gamma_t)=-\frac{1}{2}\sum_{i:T_i=t}w_i\{\log(2\pi)+z_i^{\prime}\gamma_t+(M_i-x_i^{\prime}\beta_t)^2/\exp(z_i^{\prime}\gamma_t)\}, where wiw_i is any sampling or treatment-selection weight. The exponential variance specification guarantees a positive fitted variance and permits the mediator density, and therefore the RMPW density ratio, to reflect treatment-specific and covariate-dependent heteroskedasticity.

Odds-ratio weights. An alternative avoids modeling the mediator distribution directly. By Bayes’ rule, the mediator-probability ratio can be expressed through two treatment models, one conditional on the mediator and one without it [32, 14]:

ωa→b=Pr⁡(T=b∣M,C)Pr⁡(T=a∣C)Pr⁡(T=a∣M,C)Pr⁡(T=b∣C),(3)\omega_{a\to b}=\frac{\Pr(T=b\mid M,C)\Pr(T=a\mid C)}{\Pr(T=a\mid M,C)\Pr(T=b\mid C)}, \tag*{(3)}

where C denotes the mediator-model covariates. This representation can be useful when the mediator distribution is difficult to specify, particularly for nonstandard continuous or count mediators. It does not eliminate modeling assumptions; rather, it transfers them from the mediator distribution to the treatment distribution conditional on the mediator and covariates [32, 14, 25].

Stratification-based nonparametric ratio weights. The nonparametric RMPW estimator replaces fitted mediator probabilities with observed proportions within propensity-score strata [11]. Observations are first stratified on the linear predictor of the mediator model under treatment and then, within those strata, on the corresponding predictor under control. Within each cross-stratum, the mediator probability is estimated by its observed proportion at the relevant treatment level. This marginal mean weighting through stratification (MMWS) construction [9, 17] reduces reliance on the functional form of the mediator model, at the cost of residual differences in mediator propensity within strata and increasingly sparse cross-strata as the number of strata increases.

Nonrandom treatment

When treatment is not randomized, mediator weighting alone does not address confounding of treatment with the mediator or outcome. Hong therefore combines the mediator weight with a treatment-selection weight (Hong, 2015, sec. 12.2.1). One approach uses the stabilized inverse-probability-of-treatment weight (IPTW)

Pr⁡(T=z)Pr⁡(T=z∣X)\frac{\Pr(T=z)}{\Pr(T=z\mid X)}

for an individual observed at treatment level zz. The treatment model may be binary or multivalued, as appropriate to the treatment. As with inverse-probability weighting more generally, performance depends on adequate treatment-model specification and sufficient overlap in the estimated treatment probabilities [5, 1].

As an alternative, MMWS treatment weights [9, 17] stratify observations on the estimated treatment propensity and weight observations within each stratum to reproduce the marginal treatment distribution. IPTW therefore relies directly on the fitted propensity score, whereas MMWS replaces individual inverse-probability weights with stratum-specific weights. The latter can reduce the influence of extreme estimated propensities but may retain residual treatment confounding within strata [9, 17].

Continuous treatment

For a continuous treatment, exact treatment levels have probability zero and the discrete-treatment RMPW construction cannot be applied directly. I therefore use the generalized propensity-score (GPS) weighting estimator of Hsu et al. [13]. Two conditional treatment densities are estimated: f1(d∣X)f_1(d \mid\mathbf{X}), conditional on pretreatment covariates, and f2(d∣M,X)f_2(d \mid M,\mathbf{X}), additionally conditional on the mediator. Let Kd(T)K_d(T) denote an Epanechnikov kernel in (T−d)/h(T-d)/h. The potential-outcome means at evaluation dose dtd_t and control dose d0d_0 are estimated as normalized weighted means with weights

E{Y(dt,M(dt))}:Kdt(T)f1(dt∣X),E\{Y(d_t,M(d_t))\}:\quad\frac{K_{d_t}(T)}{f_1(d_t \mid\mathbf{X})},

and

E{Y(dt,M(d0))}:Kdt(T)f2(d0∣M,X)f2(dt∣M,X)f1(d0∣X),E\{Y(d_t,M(d_0))\}:\quad\frac{K_{d_t}(T)f_2(d_0 \mid M,\mathbf{X})}{f_2(d_t \mid M,\mathbf{X})f_1(d_0 \mid\mathbf{X})},

with symmetric expressions for E{Y(d0,M(d0))}E\{Y(d_0,M(d_0))\} and E{Y(d0,M(dt))}E\{Y(d_0,M(d_t))\}. To limit the influence of extreme weights, observations whose normalized weight wi/∑jwjw_i/\sum_j w_j exceeds a threshold (0.1 in all analyses here) in any of the potential-outcome means are removed from all means, and the means are then computed from the remaining observations with renormalized weights. The bandwidth hh determines how strongly observations are localized around the evaluation dose: smaller bandwidths make the comparison more local but reduce the effective information contributing to each weighted mean [13]. Inference for this estimator is obtained by bootstrap.

Variance estimation

RMPW and the related weighting estimators considered here are multistep estimators: models used to construct mediator or treatment-selection weights are estimated before the final weighted outcome model. A conventional robust variance estimator applied only to the final outcome model treats the estimated weights as fixed and therefore does not propagate sampling uncertainty from the models used to construct them [3].

Bein et al. [3] developed a two-step variance estimator for RMPW that combines the estimating equations for the weight model and weighted outcome model and obtains an M-estimation sandwich variance. I extend this construction to propagate uncertainty from all parametric models used to construct mediator and treatment-selection weights by stacking their estimating equations with those of the weighted outcome model. The resulting variance estimator accommodates ratio and odds-ratio mediator weights, including Poisson and heteroskedastic normal mediator models; binary and multivalued treatment-selection models; and the final weighted outcome model. This permits the same two-step framework to be used when both mediator and treatment-selection weights are estimated. The stacked estimating equations, sandwich variance, and delta-method transformation to the causal effects are given in Supplementary Section S2.

The derivative-based two-step estimator does not cover stratification-based RMPW or MMWS weights, which are step functions of estimated propensity scores. Bootstrap inference provides a general alternative for these estimators and can also be used with parametric weights when finite-sample performance of the sandwich approximation is a concern. All models, strata, and weights are re-estimated within each bootstrap resample.

Weight diagnostics

Estimated treatment or mediator probabilities near the boundaries can cause a small number of observations to receive a large share of the total weight ([5, 1]). I therefore consider descriptive diagnostics of two kinds. The first concerns concentration of the weights. Within each weighted copy, I examine the largest observation’s share of the total weight and Kish’s effective sample size,

ESS=(∑iwi)2∑iwi2,\mathrm{ESS} = \frac{\left(\sum_i w_i\right)^2}{\sum_i w_i^2},

([16]), expressed as a percentage of the number of observations in the copy. For an analysis involving several weighted copies, the smallest ESS across copies provides a summary of the least stable component of the estimator.

The second concerns whether the weights accomplish what they are intended to do. Each cross-world copy is weighted to reproduce the mediator distribution of the other treatment group, so the weighted distribution of the mediator in that copy—the proportion at each level of a categorical mediator, or the mean of a continuous or count mediator—can be compared with the corresponding distribution in the targeted treatment group, weighted by any treatment-selection weights ((Hong, 2015, p. 290)). Substantial discrepancies indicate a misspecified mediator model (or, for odds-ratio weights, treatment model) or limited overlap. Covariate balance after weighting can be assessed in the same spirit ([1] (Hong 2015)). All of these quantities are used as warning indicators rather than formal criteria for estimator validity, and satisfactory diagnostics do not establish the identification assumptions.

Empirical illustrations

Two applications illustrate the framework. The first, based on a national health survey, combines an observational exposure, treatment-selection weighting, survey sampling weights, and alternative constructions of the mediator weights, and shows how weight diagnostics and inference operate together in a realistic analysis. The second, a randomized welfare-to-work experiment, complements it by illustrating alternative representations of the mediator and a continuous outcome.

National Survey on Drug Use and Health

Sexual-minority adults experience disparities in cigarette smoking and other substance-use outcomes relative to heterosexual adults, with particularly pronounced disparities among women ([30]). Earlier smoking initiation has been identified as one pathway contributing to these disparities in adulthood ([29]). I use the publicly available National Survey on Drug Use and Health (NSDUH) data analyzed by Coffman et al. [4] and distributed with the twangMediation package for R ([22]), comprising 40,293 adult women, of whom 4,130 identified as lesbian, gay, or bisexual (LGB). The exposure is sexual-minority status (LGB vs. heterosexual), the mediator is smoking initiation before age 15, and the outcome is current (past-month) cigarette smoking.

Sexual-minority status is not an assignable treatment, so the natural effects are interpreted here as a decomposition of the covariate-adjusted smoking disparity rather than as effects of a manipulable intervention. Under the identification assumptions of § 2.1, the NIE represents the component of the disparity associated with differences in early smoking initiation, whereas the NDE represents the disparity remaining after equalizing its distribution.

Because the exposure is observational, stabilized IPTW weights were estimated from a logit model containing age, race and ethnicity, education, household income, and employment status. The mediator models used the same covariates. Analyses were conducted both without and with the NSDUH sampling weights; when used, the sampling weights entered all fitted models and the potential-outcome means. Parametric RMPW with logit mediator models was compared with odds-ratio weights and stratification-based nonparametric RMPW, and robustness to a probit treatment model and restriction to common support was examined (Table 3).

Weighting methodNIENDETEINTESS (%)
Without NSDUH sampling weights
Parametric RMPW0.028 (0.003)0.097 (0.008)0.125 (0.008)−0.005 (0.003)60.3
Odds-ratio weights0.028 (0.003)0.097 (0.008)0.125 (0.008)−0.001 (0.003)58.7
Nonparametric RMPW, 3×33 \times30.028 (0.003)0.097 (0.009)0.125 (0.009)−0.002 (0.002)59.9
Parametric RMPW, probit treatment model0.028 (0.003)0.097 (0.008)0.124 (0.008)−0.005 (0.003)59.4
With NSDUH sampling weights
Parametric RMPW0.018 (0.004)0.080 (0.013)0.098 (0.014)−0.009 (0.005)15.7
Odds-ratio weights0.024 (0.004)0.075 (0.014)0.098 (0.014)0.001 (0.004)14.3
Nonparametric RMPW, 3×33 \times30.018 (0.006)0.080 (0.014)0.098 (0.014)−0.006 (0.006)15.6
Parametric RMPW, probit treatment model0.018 (0.004)0.079 (0.013)0.098 (0.013)−0.009 (0.005)15.5

Table 3. NSDUH data: decomposition of the disparity in current smoking by sexual-minority status into components through and not through early smoking initiation; estimates (standard errors) by weighting method, without and with the NSDUH sampling weights.

Incorporating the NSDUH sampling weights reduced the estimated smoking disparity and both of its components while substantially increasing weight concentration. Parametric and nonparametric RMPW nevertheless produced very similar decompositions both with and without sampling weights. Odds-ratio weights agreed closely with these estimators without sampling weights but assigned somewhat more of the weighted disparity to the indirect pathway. The mediator distribution check (§ 2.4.3) is informative here. With sampling weights, parametric and nonparametric RMPW reproduced the target distributions more closely than odds-ratio weighting, favoring the mediator-probability weights in this application. The estimated treatment-by-mediator interaction was small throughout. Results were essentially unchanged when the treatment model was changed from logit to probit or the analysis was restricted to common support.

For parametric RMPW, two-step, PSU-clustered two-step, and PSU-cluster bootstrap standard errors were similar. For nonparametric RMPW, however, treating the step-function weights as known understated uncertainty relative to the bootstrap, consistent with the simulation results in § 5.4. The analyses incorporate the NSDUH sampling weights and PSU clustering but not survey strata and therefore do not constitute full design-based survey inference. Overall, the example illustrates that population weighting can affect both the estimated decomposition and weight stability, that diagnostics can help distinguish among alternative mediator-weight constructions, and that inference should reflect how the weights are estimated.

Riverside welfare-to-work experiment

The second illustration complements the first: treatment was randomized, the outcome is continuous, and the mediator can be represented in several ways. It uses data from the Riverside site of the National Evaluation of Welfare-to-Work Strategies (NEWWS), analyzed by Hong et al. (2015) and Hong (2015, ch. 11) and distributed with the rmpw package for R (Qin et al., [27]). Mothers with at least one child aged 3–5 years were randomly assigned to a labor force attachment (LFA) program emphasizing rapid entry into employment (n=208n = 208) or to a control condition (n=486n = 486). The outcome is maternal depressive symptoms two years after randomization, and the mediator is employment during those two years. LFA increased employment but had no statistically significant overall effect on depressive symptoms. Hong et al. (2015) proposed that this overall effect could conceal opposing pathways—program-induced employment might reduce depressive symptoms, while other features of the program, such as the stress associated with the threat of sanctions, could operate in the opposite direction—and that the relationship between employment and depression might differ by treatment assignment. Employment can be represented as a binary indicator, a three-category measure, or the number of quarters employed (0–8). Mediator models adjusted for nine baseline covariates (prior employment, four measures of depressive symptoms, two barriers to work, Hispanic ethnicity, and marital status), which were also included in the outcome model for precision. Because treatment was randomized, no treatment-selection weights were required (Table 4).

Mediator model / weightsStd. err.NIENDEPNIEINTESS (%)
Binary mediator (emp), logit model
Parametric RMPWtwo-step−0.871 (0.475)1.277 (0.861)0.318 (0.275)−1.189 (0.555)68.8
bootstrap−0.871 (0.620)1.277 (0.942)0.318 (0.302)−1.189 (0.691)
Categorical mediator (emp_cat)
Ordered logittwo-step−0.865 (0.394)1.269 (0.801)0.134 (0.270)−0.999 (0.470)67.1
bootstrap−0.865 (0.456)1.269 (0.816)0.134 (0.298)−0.999 (0.546)
Multinomial logittwo-step−0.886 (0.473)1.282 (0.843)0.071 (0.367)−0.957 (0.595)59.3
Count mediator (emp_cont, quarters employed)
Odds ratiotwo-step−0.616 (0.280)1.018 (0.738)−0.234 (0.190)−0.382 (0.325)72.2
bootstrap−0.616 (0.282)1.018 (0.718)−0.234 (0.201)−0.382 (0.331)

Table 4. NEWWS Riverside data: estimates (standard errors) by mediator representation, weighting method, and variance estimator.

With employment represented as a binary mediator, parametric RMPW indicated opposing pathways: LFA increased employment (65% vs. 40% employed), the indirect effect through employment was beneficial (NIE = −0.87-0.87), the direct effect was detrimental (NDE = 1.281.28), and the natural treatment-by-mediator interaction was negative (INT = −1.19-1.19), consistent with the hypothesis that employment was more beneficial under LFA than under the control condition. Representing employment by three ordered categories, modeled by ordered or multinomial logit, gave nearly identical estimates. Bootstrap standard errors of the NIE and the interaction were 25%–30% larger than two-step standard errors for the binary mediator, and with bootstrap inference the interaction was no longer statistically significant. The number of quarters employed is bounded, with 53% of observations equal to zero, making normal and Poisson mediator models unattractive. Odds-ratio weights avoided modeling this distribution directly and gave a smaller NIE (−0.62-0.62) and interaction (−0.38-0.38), with stable weights and similar two-step and bootstrap standard errors. This illustrates both the attraction and the tradeoff of odds-ratio weighting: it avoids specifying a difficult mediator distribution but shifts the modeling assumptions to the treatment model conditional on the mediator and covariates.

Simulation evaluation

The empirical illustrations demonstrate several choices that arise in applying RMPW, including how treatment selection is addressed, how the mediator is represented and modeled, how mediator weights are constructed, and how weight stability and uncertainty are assessed. Targeted simulation experiments were therefore used to evaluate the finite-sample consequences of these and related choices under conditions in which the true causal effects are known. The experiments broaden the settings evaluated by Hong et al. [11] beyond the original binary-treatment, binary-mediator design and examine mediator-model specification, alternative weighting strategies, treatment-selection adjustment, weight instability, variance estimation, and continuous-treatment weighting. The simulations were designed and reported following the ADEM P framework—aims, data-generating mechanisms, estimands, methods, and performance measures—of Morris et al. [23]. Complete data-generating mechanisms, true estimand values, models, sample sizes, and methods for all designs are given in Supplementary Table S2.

Simulation designs

The simulations begin by anchoring the evaluation to Hong et al. [11]. A replication experiment reproduces their three randomized binary-treatment, binary-mediator scenarios. A misspecification experiment reconstructs their mediator-model misspecification analysis by adding either a nonlinear squared term or a nonadditive product term to the mediator model and omitting that term from the fitted model. Because the original article did not report the covariate coefficients or complete misspecification data-generating mechanisms, these components were reconstructed to preserve the reported marginal mediator probabilities and true effects. These experiments compare parametric RMPW with correctly and incorrectly specified mediator models and stratification-based nonparametric RMPW.

A second set of designs examines whether the weighting framework continues to perform well beyond the original binary-mediator, continuous-outcome setting. The designs include continuous and count mediators, binary and count outcomes, and a three-level treatment. A continuous-mediator design also allows covariate effects and residual variance to differ between treatment groups, providing a setting in which the parametric density-ratio model is correctly specified but a simple treatment model linear in the mediator, as used for the basic odds-ratio weight, is misspecified.

Nonrandom treatment assignment is examined in designs that compare adjustment for mediator selection alone with IPTW and MMWS treatment-selection weights and examine omission of a treatment–outcome confounder from the treatment model. A further design uses an ordinal three-level treatment and compares alternative multivalued treatment models.

Two sets of designs examine the relation between model specification and weight stability. In the first, treatment is generated from logit, probit, and heteroskedastic probit models, and each model is fitted. In the second, a continuous mediator has a conditional variance that depends on treatment and covariates; increasing degrees of heteroskedasticity are used to compare the conventional constant-variance density-ratio model, the heteroskedastic normal model described above, and odds-ratio weights. Because increasing heteroskedasticity can also make the mediator densities under the two treatment groups increasingly dissimilar, this design provides a direct assessment of the relation between model specification, weight concentration, and estimator performance.

Inference is examined by reusing representative binary-mediator and three-level-treatment designs with nonrandom treatment at N=800N = 800 to compare two-step and bootstrap inference for IPTW and bootstrap inference for MMWS, nonparametric RMPW, and odds-ratio mediator weights. Additional two-step-versus-bootstrap comparisons are made in the heteroskedastic continuous-mediator design.

Finally, the GPS estimator of Hsu et al. [13] is evaluated for a continuous treatment. A linear Gaussian treatment model is evaluated at two treatment contrasts, and the default bandwidth, sd⁡(T) 2.34/N1/4\operatorname{sd}(T)\,2.34/N^{1/4}, is compared with one-half of that value to examine the bias–variance consequences of more localized kernel weighting.

Across the simulations, the principal estimands are the NIE and NDE for each treatment level relative to control: level 1 versus 0 for binary treatments, levels 1 and 2 versus 0 for three-level treatments, and doses 2 and 0 versus 1 for the continuous treatment. True values were obtained analytically when possible and otherwise by Gauss–Hermite integration over the covariate distribution. Unless otherwise stated, mediator and treatment models include the covariates as linear additive terms. An unadjusted estimator obtained from a mediator model without covariates provides the reference for the proportion of bias removed in the randomized-treatment and nonrandom-treatment designs.

Performance measures

For each estimand, performance is summarized by bias, empirical standard error, mean model standard error, coverage of nominal 95% confidence intervals, root mean squared error, and the proportion of replications that fail to produce an estimate. To facilitate comparison with Hong et al. [11], the randomized- and nonrandom-treatment designs also report the proportion of the naive estimator’s absolute bias removed,

1−∣bias⁡∣∣bias⁡naive∣.1 - \frac{\lvert\operatorname{bias}\rvert}{\lvert\operatorname{bias}_{\mathrm{naive}}\rvert}.

The accuracy of model-based inference is assessed by the relative error of the mean estimated standard error,

SE^EmpSE−1\frac{\widehat{\mathrm{SE}}}{\mathrm{EmpSE}} - 1

where EmpSE is the empirical standard error of the estimates across replications. A value of zero indicates agreement with the empirical sampling variability, whereas negative and positive values indicate underestimation and overestimation, respectively. For designs specifically examining weight instability, the smallest Kish ESS across weighted copies, the largest observation’s share of a copy’s total weight, and the proportion of replications in which at least one copy has ESS below 5% are also recorded.

Each design is evaluated at N=800N = 800 and N=5,000N = 5{,}000, except the bootstrap-inference designs, which use N=800N = 800. The simulations use 1,000 replications except for the two-step-versus-bootstrap comparisons, which use 500. Bootstrap standard errors are based on 200 resamples. Monte Carlo standard errors are computed for all performance measures following Morris et al. [23], with the largest value reported in each table’s notes. With 1,000 replications, the Monte Carlo standard error of 95% coverage is approximately 0.007 and that of estimated bias is 3.2% of the empirical standard error; with 500 replications, the corresponding values are 0.010 and 4.5%.

All simulation analyses were performed with the community-contributed package mmwsmed [18] in Stata version 19.

Simulation results

Mediator-model specification and alternative weights

The replication of Hong et al. [11] confirmed the performance of RMPW in the original binary-treatment, binary-mediator setting (Supplementary Table S3). The naive estimator was biased in every scenario, with NIE coverage of at most 0.18. Parametric RMPW removed 98%–99% of the bias, while stratification-based nonparametric RMPW removed 81%–84% with 3×33 \times3 strata and 78%–90% with 4×44 \times4 strata. These results closely reproduced the proportions reported by Hong et al. [11] (Supplementary Table S4). At N=800N = 800, increasing from 3×33 \times3 to 4×44 \times4 strata did not consistently reduce bias. Coverage for the nonparametric estimators, whose default standard errors treat the weights as known, was below nominal in several settings.

Mediator-model misspecification produced a clear difference between parametric and stratification-based weighting (Supplementary Table S5). When a nonlinear or nonadditive term was omitted from the mediator model, the proportion of NIE bias removed by parametric RMPW declined as the strength of the omitted term increased. In the nonlinear designs, bias removal declined from 51%–53% at δ=0.25\delta= 0.25 to about 20% at δ=1\delta= 1; in the nonadditive designs, it declined from 79%–81% to 61%–65%. At N=5,000N = 5{,}000, NIE coverage was as low as 0.41. Nonparametric RMPW removed 65%–88% of the NIE bias across these designs and had the smallest NIE root mean squared error in every design and at both sample sizes, although its NIE coverage remained below nominal (0.65–0.85). Correctly specifying the parametric mediator model reduced bias but could substantially increase sampling variability; in the most extreme designs, its empirical standard error was as much as 2.8 times that of the misspecified model.

Across different mediator, outcome, and treatment types, parametric RMPW was essentially unbiased when its mediator model was adequately specified (Supplementary Table S6). Absolute bias was at most 0.006 for continuous and count mediators, binary and count outcomes, and a three-level treatment. Odds-ratio weights performed similarly when a continuous mediator had common covariate effects and residual variance across treatment groups. When these features differed by treatment group, however, odds-ratio weights removed only 44%–46% of the bias, leaving bias of approximately 0.06 for both the NIE and NDE at both sample sizes. Stratification-based nonparametric rMPW removed 82%–97% of the bias in the designs in which it was evaluated.

Treatment-selection adjustment

With nonrandom treatment, mediator weighting alone was insufficient to recover both natural effects (Supplementary Table S7). Adjusting only for mediator selection removed 80%–100% of the NIE bias but left NDE bias of 0.11–0.55, exceeding the bias of the naive estimator in every design. Adding IPTW treatment-selection weights reduced the absolute NDE bias to at most 0.009. MMWS treatment weights retained some NDE bias, ranging from 0.012 to 0.058 with five strata and from 0.005 to 0.029 with ten strata. When a treatment–outcome confounder was omitted from the treatment model, absolute NDE bias ranged from 0.015 to 0.087, whereas NIE bias remained at most 0.007.

For an ordinal three-level treatment, IPTW based on multinomial logit, ordered logit, and ordered probit treatment models was essentially unbiased, with absolute bias at most 0.013 and coverage of 0.930–0.956 (Supplementary Table S8). Odds-ratio mediator weights, which rely on a model for treatment conditional on the mediator, had NDE bias as large as −0.036 and coverage as low as 0.859.

Correct specification does not ensure stable weights

Correct model specification did not ensure stable weighting or good finite-sample performance. When treatment was generated by conventional logit or probit models, the alternative treatment models performed similarly (Supplementary Table S9). Under strong heteroskedasticity (g=0.50g = 0.50), however, the correctly specified heteroskedastic probit model performed worse than the simpler logit model, with greater bias, sampling variability, and undercoverage. This deterioration coincided with greater weight concentration: at N=5,000N = 5{,}000, the mean smallest effective sample size was 19.2% for the heteroskedastic probit model versus approximately 30% for the logit model, and very low effective sample sizes became more frequent. Both the estimation problems and weight concentration diminished as heteroskedasticity weakened.

The same pattern was more pronounced with a heteroskedastic continuous mediator (Supplementary Table S10). Assuming constant variance produced increasing bias and severe undercoverage as heteroskedasticity increased. Modeling the conditional variance substantially reduced the bias but did not prevent large sampling variability and poor coverage under strong heteroskedasticity; odds-ratio weights also deteriorated as heteroskedasticity increased. These problems again coincided with increasingly concentrated weights: at c=0.40c = 0.40 and N=5,000N = 5{,}000, the mean smallest effective sample size was 15.5% with the heteroskedastic mediator model and 9.6% with odds-ratio weights. In both experiments, deterioration in estimator performance coincided with deterioration in the weight diagnostics.

Variance estimation

Standard errors that treated estimated weights as fixed were unreliable in both directions (Supplementary Tables S3 and S5–S7). They understated sampling variability for several mediator-weighting estimators but overstated it for some treatment-selection estimators. Two-step standard errors generally performed well in large samples when the weights were stable, but could substantially understate sampling variability when the weights were highly variable, including in the mediator-model misspecification and heteroskedastic continuous-mediator designs (Supplementary Tables S5 and S11). In the nonrandom-treatment designs, where both mediator and treatment-selection weights were estimated, the generalized two-step standard errors produced coverage close to the nominal level when the weights were stable (Supplementary Table S7).

Bootstrap inference generally more closely reproduced empirical sampling variability at N=800N = 800 (Supplementary Table S12). Coverage was near nominal for IPTW, MMWS, and nonparametric RMPW; poorer coverage for some odds-ratio estimates coincided with appreciable estimator bias. Bootstrap inference also improved coverage relative to two-step inference in the heteroskedastic continuous-mediator design at N=800N = 800.

Bootstrap inference did not, however, remedy severe weight instability. At N=5,000N = 5{,}000 under strong mediator heteroskedasticity, both two-step and bootstrap standard errors substantially underestimated the empirical sampling variability (Supplementary Table S11). Estimation failures were uncommon overall: failures occurred in only 17 simulation cells, with at most 11 of 1,000 replications failing in any cell, and bootstrap fitting failures were rare.

Continuous-treatment weighting

GPS weighting with the default bandwidth produced little bias and coverage near the nominal level in the continuous-treatment design (Supplementary Table S13). Halving the bandwidth increased sampling variability and, at N=800N = 800, produced appreciable NDE bias and lower coverage.

Discussion

The empirical illustrations and simulation experiments together highlight three distinct considerations in applying RMPW: whether the causal effect is identified, whether the models used to construct the weights are sufficiently well specified and produce stable weights, and whether inference adequately accounts for estimation of those weights. These considerations are related but not interchangeable. No weighting strategy can overcome failure of the identification assumptions; correct specification of a weight model does not guarantee good finite-sample performance when the resulting weights are highly concentrated; and a better variance estimator cannot repair an unstable point estimator.

Parametric RMPW performed well across different mediator, outcome, and treatment types when the models producing the weights were adequately specified and the weights were stable. At the same time, the simulations clarify an important tradeoff between specification and robustness. Stratification-based nonparametric RMPW was less sensitive to nonlinear and nonadditive misspecification of the mediator model but retained some residual bias, whereas correctly specified parametric models could reduce bias at the cost of substantially greater sampling variability. Alternative weighting formulations likewise changed rather than eliminated the modeling assumptions: odds-ratio weighting avoided direct specification of the mediator distribution but instead required an adequate model for treatment conditional on the mediator and covariates [32, 14]. The NSDUH illustration reinforces this point: alternative mediator-weight constructions gave similar results without sampling weights but differed more once sampling weights were incorporated, and reproduction of the target mediator distribution favored the mediator-probability weights.

Perhaps the most consequential finding is that model specification and weight stability should be treated as separate properties of a weighting analysis. In the treatment-model experiment, under strong heteroskedasticity, the correctly specified heteroskedastic probit model generated more concentrated weights and poorer finite-sample performance than the simpler logit model; in the continuous-mediator experiment, modeling the conditional variance correctly reduced bias but still produced concentrated weights and poor coverage under strong heteroskedasticity. The NSDUH analysis provides a complementary empirical example: incorporating sampling weights changed the target population estimates while sharply reducing effective sample size. The problem was especially pronounced in the simulations for continuous mediators with strongly treatment- or covariate-dependent variances. Ratios of continuous densities can become extremely variable when the treatment groups have limited overlap in their mediator distributions, so increasing sample size need not provide the stabilization expected from conventional large-sample arguments. The effective-sample-size diagnostics deteriorated in the same settings in which bias, sampling variability, and coverage deteriorated. This does not imply that investigators should deliberately misspecify their models. Rather, it means that goodness of fit and substantive plausibility should be considered together with the behavior of the weights implied by those models.

The variance results lead to a related conclusion. Standard errors that treated estimated weights as fixed were unreliable in both directions. The generalized two-step estimator, building on the approach of Bein et al. [3], performed well for many parametric weighting estimators when samples were sufficiently large and weights were stable, while bootstrap inference generally performed better at moderate sample sizes and provides a natural approach for the step-function weights used by nonparametric RMPW and MMWS. The empirical illustrations showed the same general pattern: two-step and bootstrap inference were similar for parametric weights, whereas bootstrap inference was more consequential for the nonparametric weights in NSDUH and for some effects in Riverside. Neither approach, however, produced reliable inference in the simulations when the point estimator itself was severely destabilized by extreme weights. Variance estimation and weight diagnostics should therefore be viewed as complementary components of an RMPW analysis rather than as substitutes for one another.

Practical guidance

These findings lead to several practical recommendations, summarized in Table 5.

IssueRecommendation
Identification and covariatesDefine the causal estimand and assess the identification assumptions before choosing an estimator. Adjust the mediator model for pretreatment mediator–outcome confounders even with randomized treatment; with nonrandom treatment, additionally adjust for treatment confounders.
Discrete mediatorUse a suitable parametric mediator model when its functional form is credible. For binary or ordinal mediators, compare with stratification-based nonparametric RMPW as a robustness analysis for functional-form specification; avoid overly fine strata when sample sizes within cross-strata are small.
Count mediatorUse a count model when its distributional assumptions are plausible; otherwise consider odds-ratio weighting, recognizing that this shifts the modeling assumptions to the treatment model conditional on the mediator.
Continuous mediatorExamine the conditional mean and variance of the mediator across treatment groups. Model heteroskedasticity when present and inspect weight stability carefully; use odds-ratio weighting only when the treatment model conditional on the mediator can be adequately specified.
Nonrandom treatmentModel treatment selection explicitly. IPTW performs well with an adequately specified treatment model; stratification-based treatment weights can reduce sensitivity to extreme propensities but may retain residual confounding. Balance model flexibility against weight stability.
Continuous treatmentUse GPS weighting at evaluation doses with adequate support. Treat bandwidth as a bias–variance choice rather than assuming that a smaller bandwidth is preferable.
Weight diagnosticsExamine covariate balance, reproduction of the target mediator distribution, effective sample size, and concentration of the weights. Treat unusually low effective sample size or highly concentrated weights as warnings requiring investigation, not as formal validity thresholds.
Variance estimationAvoid standard errors that treat estimated weights as fixed. Use two-step inference for parametric weights in sufficiently large samples with reasonably stable weights; use bootstrap inference for step-function weights and when finite-sample approximation is a concern. Refit all models and weights within each bootstrap resample.
ReportingReport the estimand and identification assumptions, models and covariates, weighting strategy, variance estimator, balance and weight-stability diagnostics, bootstrap fitting failures, overlap restrictions, and sensitivity to substantively different weighting approaches.

Table 5. Practical recommendations for ratio-of-mediator-probability weighting analyses.

Begin with the causal estimand and identification assumptions. The choice of weighting estimator comes after the causal question has been defined. Natural direct and indirect effects require assumptions about treatment assignment, mediator–outcome confounding, positivity, and consistency ([15, 34, 24]). Pretreatment covariates used to model the mediator should therefore include confounders of the mediator–outcome relationship even when treatment is randomized, because the mediator itself is not randomized. When treatment is observational, treatment confounding must additionally be addressed. Treatment-induced mediator–outcome confounding requires particular care because the natural effects considered here are generally not identified under the standard assumptions ([35]).

Choose the mediator-weighting strategy to match the modeling problem. When a credible parametric model is available for the mediator distribution, parametric ratio weights provide a natural starting point. For binary and ordinal mediators, stratification-based nonparametric RMPW can provide a useful robustness analysis for functional-form specification, although increasingly fine cross-stratification can create sparse cells and need not improve performance in modest samples. Count and continuous mediators require particular attention to their conditional distributions. For continuous mediators, the conditional variance should be examined as well as the conditional mean; imposing constant variance when substantial heteroskedasticity is present can produce biased density ratios. Odds-ratio weighting may be attractive when direct modeling of the mediator distribution is difficult, as in the bounded, zero-heavy count mediator in the Riverside example, but the required model for treatment conditional on the mediator and covariates must itself be credible.

Address treatment selection separately when treatment is not randomized. Mediator weights address mediator selection; they do not in general remove confounding of treatment with the mediator and outcome. A treatment-selection weight is therefore required when treatment is observational under the identification strategy considered here (Hong, 2015). IPTW performed well when the treatment model included the relevant confounders. Stratification-based treatment weights reduced reliance on individual inverse probabilities but retained residual confounding when strata were too coarse. More flexible treatment models should be considered when substantively justified, but flexibility should not be pursued without examining the stability of the resulting weights.

Examine the weights before interpreting the effects. Investigators should examine overlap and covariate balance and assess whether weighting reproduces the intended target distribution [5, 1]. For RMPW, effective sample size and the largest observation’s share of the total weight indicate whether a small part of the sample dominates a weighted potential-outcome mean. The simulations do not establish universal thresholds for these diagnostics. Unusually concentrated weights should instead prompt investigation of overlap, model specification, and whether the chosen weighting formulation is appropriate for the observed treatment and mediator distributions. Restricting an analysis to a prespecified region of adequate common support may be informative when lack of overlap is responsible for instability, but doing so changes the population to which the resulting causal effect applies.

Match inference to the way the weights are estimated. Standard errors that treat estimated weights as fixed should generally be avoided. For parametric weights, the two-step estimator provides a computationally convenient way to propagate uncertainty from estimation of the weight models and performed well in sufficiently large samples when the weights were reasonably stable. Bootstrap inference provides an alternative in moderate samples and is required for the stratification-based estimators considered here because their step-function weights are not covered by the derivative-based variance calculation. All models, strata, and weights should be re-estimated within each bootstrap resample. Repeated fitting failures should be treated as evidence of sparse data or unstable models rather than simply discarded as a computational inconvenience.

Use additional care with continuous treatments. For continuous treatments, the evaluation doses should lie in regions with adequate support, and the bandwidth should be regarded as a bias–variance choice rather than a tuning parameter for which smaller is necessarily better. In the designs considered here, the default bandwidth used by the GPS estimator of Hsu et al. [13] performed well, whereas halving it increased sampling variability and finite-sample bias. Because only two bandwidths and a single data-generating treatment model were examined, these results support caution rather than a general bandwidth-selection rule.

Report the information needed to evaluate the weighting analysis. An RMPW analysis should report the causal estimands and identification assumptions; the mediator and, when applicable, treatment models and their covariates; the weighting strategy; the variance estimator; and relevant balance and weight-stability diagnostics. Bootstrap analyses should describe the re-sampling procedure and report fitting failures. Restrictions based on overlap or common support should be reported together with their implications for the target population. When alternative weighting strategies encode meaningfully different modeling assumptions, presenting those analyses as robustness checks can help readers assess how strongly the conclusions depend on a particular specification.

Limitations

The simulation experiments were designed to isolate several practically important choices in RMPW implementation rather than exhaust the full range of data-generating mechanisms, estimators, and applications. They considered representative mediator, outcome, and treatment types at principally two sample sizes, and only selected forms of model misspecification and weight instability. Extensions such as multiple mediators and moderated mediation were not evaluated, and the continuous-treatment experiment considered a single treatment model and two bandwidths. The results should therefore be interpreted as evidence about the methods under the conditions examined rather than as rankings that necessarily generalize to other applications.

The simulations also assume that the natural effects are identified and therefore do not address violations such as unmeasured mediator–outcome confounding; weighting-based sensitivity analysis provides a complementary approach to this problem [12]. Finally, although weight instability was examined directly, the simulations did not compare remedies such as weight truncation or alternative weighting targets. Such modifications can alter the bias–variance tradeoff and, in some cases, the population to which the estimand applies, and therefore require separate evaluation.

Conclusion

Ratio-of-mediator-probability weighting provides a flexible approach to causal mediation analysis that has evolved into a broad family of weighting methods for different mediator and treatment types. This article brings these methods together within a common estimation framework for discrete treatments, incorporates a heteroskedastic normal model for continuous mediators, extends two-step variance estimation to multiple parametric weight components, and broadens the finite-sample evidence available to guide their application. The results show that successful application requires more than correct specification of the models used to construct the weights: investigators must also assess whether those models produce stable weights and use an inference procedure that appropriately accounts for weight estimation. These considerations—identification, model specification and weight stability, and inference—provide a practical framework for applying and evaluating RMPW analyses. The estimators, variance procedures, and diagnostics examined here are implemented in the community-contributed Stata command mmwsmed [18].

IPTW, inverse-probability-of-treatment weighting; MMWS, marginal mean weighting through stratification. The derivation of the generalized two-step variance estimator is given in Supplementary § 10.

N=40,293N = 40{,}293 adult women (4,130 LGB, 36,163 heterosexual). Exposure: LGB vs. heterosexual; mediator: smoking initiation before age 15; outcome: past-month cigarette smoking (effects are differences in prevalence). Treatment-selection (IPTW) and mediator models adjust for age, race and ethnicity, education, household income, and employment status; the treatment model is logit unless stated, and the mediator models are logit. With sampling weights, the weights enter every model and the potential-outcome means. Standard errors are two-step, except for nonparametric RMPW (bootstrap, 1,000 resamples of the 100 primary sampling units). ESS: smallest Kish effective sample size of the four weighted copies, as a percentage of the copy’s observations.

N=694N = 694 (486 control, 208 LFA). All analyses adjust for the nine covariates in the mediator and outcome models.

Two-step: standard errors that account for the estimation of the weights; bootstrap: 1,000 resamples. ESS: smallest Kish effective sample size of the two weighted copies, as a percentage of the copy’s observations. The total effect was 0.40–0.41 in every analysis.

Supplementary material

Additional RMPW estimands and identification conditions

Two mediators. With two mediators M1M_1 and M2M_2, the potential-outcome means depend on their causal ordering. In the parallel (concurrent) design, neither mediator affects the other, and define

Y(z,a,b)=E{Y(z,M1(a),M2(b))}.Y(z,a,b) = E\{Y(z,M_1(a),M_2(b))\}.

The total effect Y(t,t,t)−Y(c,c,c)Y(t,t,t)-Y(c,c,c) is decomposed into the natural direct effect Y(t,c,c)−Y(c,c,c)Y(t,c,c)-Y(c,c,c) and the joint indirect effect Y(t,t,t)−Y(t,c,c)Y(t,t,t)-Y(t,c,c). The latter can be decomposed into the indirect effect through M1M_1 with M2M_2 at its control or treated value,

NIE10=Y(t,t,c)−Y(t,c,c),NIE11=Y(t,t,t)−Y(t,c,t).\mathrm{NIE}_{10}=Y(t,t,c)-Y(t,c,c), \qquad\mathrm{NIE}_{11}=Y(t,t,t)-Y(t,c,t).

and the corresponding effects through M2M_2,

NIE20=Y(t,c,t)−Y(t,c,c),NIE21=Y(t,t,t)−Y(t,t,c).\mathrm{NIE}_{20}=Y(t,c,t)-Y(t,c,c), \qquad\mathrm{NIE}_{21}=Y(t,t,t)-Y(t,t,c).

The mediated interaction is

M1xM2=NIE21−NIE20,\mathrm{M1xM2}=\mathrm{NIE}_{21}-\mathrm{NIE}_{20},

which is nonzero when the effect of one mediator on the outcome depends on the value of the other ((Hong, 2015, sec. 13.2)).

In the sequential (consecutive) design, M1M_1 precedes and may affect M2M_2, and M2(t,M1(t′))M_2(t,M_1(t')) denotes the potential value of the second mediator under treatment tt when the first mediator takes its value under t′t'. The weight proposed by Hong (2015, eq. 13.16) identifies potential-outcome means of the form

Y(z,a,b)=E{Y(z,M1(a),M2(b,M1(a)))}.Y(z,a,b)=E\{Y(z,M_1(a),M_2(b,M_1(a)))\}.

so that M2M_2 responds to the individual’s own value of M1M_1. Using these means, the NDE, joint NIE, indirect effects through M1M_1 and M2M_2, their mediated interaction, the corresponding pure indirect effects, and the treatment-by-mediator interactions can be defined as shown in Table S1. The NDE, joint NIE, and TE coincide with the definitions of Hong [10].

EffectDefinitionInterpretation
Two parallel mediators; Y(z,a,b)=E[Y(z,M1(a),M2(b))]Y(z,a,b)=\mathrm{E}[Y(z,M_{1}(a),M_{2}(b))]
TE, NDEY(t,t,t)−Y(c,c,c)Y(t,t,t)-Y(c,c,c); Y(t,c,c)−Y(c,c,c)Y(t,c,c)-Y(c,c,c)total and natural direct effects
NIEY(t,t,t)−Y(t,c,c)Y(t,t,t)-Y(t,c,c)joint indirect effect through both mediators
NIE1_0, NIE1_1Y(t,t,c)−Y(t,c,c)Y(t,t,c)-Y(t,c,c); Y(t,t,t)−Y(t,c,t)Y(t,t,t)-Y(t,c,t)indirect effect through M1M_{1}, with M2M_{2} at its control or treated value
NIE2_0, NIE2_1Y(t,c,t)−Y(t,c,c)Y(t,c,t)-Y(t,c,c); Y(t,t,t)−Y(t,t,c)Y(t,t,t)-Y(t,t,c)indirect effect through M2M_{2}, with M1M_{1} at its control or treated value
M1xM2NIE2_1 − NIE2_0indirect effect through the M1M_{1}-by-M2M_{2} interaction
Two sequential mediators; Y(z,a,b)=E[Y(z,M1(a),M2(b,M1(a)))]Y(z,a,b)=\mathrm{E}[Y(z,M_{1}(a),M_{2}(b,M_{1}(a)))]
TE, NDE, NIEas for parallel mediatorstotal, natural direct, and joint indirect effects
NIE1Y(t,t,c)−Y(t,c,c)Y(t,t,c)-Y(t,c,c)indirect effect through M1M_{1}, including its effect on M2M_{2} under control
NIE2Y(t,c,t)−Y(t,c,c)Y(t,c,t)-Y(t,c,c)indirect effect through M2M_{2}, with M1M_{1} at its control value
M1xM2Y(t,t,t)−Y(t,t,c)−Y(t,c,t)+Y(t,c,c)Y(t,t,t)-Y(t,t,c)-Y(t,c,t)+Y(t,c,c)indirect effect through the M1M_{1}-by-M2M_{2} interaction
PNIE1, PNIE2Y(c,t,c)−Y(c,c,c)Y(c,t,c)-Y(c,c,c); Y(c,c,t)−Y(c,c,c)Y(c,c,t)-Y(c,c,c)indirect effects through M1M_{1} and M2M_{2}, under control
ZxM1, ZxM2NIE1 − PNIE1; NIE2 − PNIE2treatment-by-mediator interactions
Other
effect_diffeffect at moderator level gg minus effect at the first leveldifference in an effect between subpopulations
POmeansYabY_{ab} or Y(z,a,b)Y(z,a,b)potential-outcome means

Table S1. Effects for two mediators and moderated mediation, for treatment level tt compared with the control level cc.

Moderated mediation and controlled direct effects. For a pretreatment moderator GG, the models, weights, and effects may be estimated within each level gg of GG, with differences in each effect defined relative to a reference level of the moderator ((Hong, 2015, sec. 13.1)). Controlled direct effects can additionally be estimated from a weighted outcome regression containing treatment, mediator, and their interaction ((Hong, 2015)).

Identification with two mediators. With two parallel mediators, conditions (ii) and (iii) of § 2.1 apply to each mediator, and the mediators must additionally be conditionally independent of each other given treatment and XX ((Hong, 2015, sec. 13.2.2)). Sequential mediators require stronger conditions ((Hong, 2015, sec. 13.3.2)): given treatment and XX, M1M_1 must be independent not only of the potential outcomes but also of the potential values of M2M_2; and given treatment, M1M_1, and XX, M2M_2 must be independent of the potential outcomes. Because M1M_1 is itself a posttreatment variable, the M2M_2–outcome relationship may be confounded by M1M_1, which the weights condition on, but by no other variable affected by the treatment or by M1M_1.

Generalized two-step variance estimator

This section states the estimating equations stacked by the two-step variance estimator of Section 2.3.4, as implemented in mmwsmed.

Weights and outcome model. Estimation proceeds in the duplicated data (Section 2.2). Copy c=1,…,Cc = 1,\ldots,C targets the potential-outcome mean E{Y(ac,M(bc))}E\{Y(a_c, M(b_c))\} and contains the individuals with Ti=acT_i = a_c. The weight of individual ii in copy cc is

Wic(θ)=uiτi(θT,π)∏kωick(θM),W_{ic}(\boldsymbol{\theta}) = u_i \tau_i(\boldsymbol{\theta}_T,\boldsymbol{\pi}) \prod_k \omega_{ick}(\boldsymbol{\theta}_M),

where uiu_i is a sampling weight (one if none); τi=πTi/Pr⁡(T=Ti∣Xi;θT)\tau_i = \pi_{T_i}/\Pr(T = T_i \mid X_i;\boldsymbol{\theta}_T) is the stabilized IPTW (one under randomization), with marginal treatment shares π=(πz)\boldsymbol{\pi} = (\pi_z) and treatment-model parameters θT\boldsymbol{\theta}_T; and ωick=fk(Mik∣bc,Xi;θM)/fk(Mik∣ac,Xi;θM)\omega_{ick} = f_k(M_{ik} \mid b_c, X_i;\boldsymbol{\theta}_M)/f_k(M_{ik} \mid a_c, X_i;\boldsymbol{\theta}_M) is the mediator-probability (or density) ratio for mediator kk, equal to one when ac=bca_c = b_c. For odds-ratio weights, ωick\omega_{ick} is (3), and θM\boldsymbol{\theta}_M contains the parameters of the treatment models with and without the mediator. Collect the parameters of all weight components in θ=(θ1,…,θJ)\boldsymbol{\theta} = (\boldsymbol{\theta}_1,\ldots,\boldsymbol{\theta}_J), one block per fitted model (a mediator model for each treatment level, the treatment-selection model, the marginal shares, or an odds-weight treatment model).

The outcome model is a generalized linear model μic=g−1(dicλ)\mu_{ic} = g^{-1}(\mathbf{d}_{ic}\boldsymbol{\lambda}), where dic\mathbf{d}_{ic} contains the copy indicators and, optionally, outcome covariates centered at their (sampling-weighted) sample means, v~i\tilde{\mathbf{v}}_i, so that λ=(λ1,…,λC,λ′)\boldsymbol{\lambda} = (\lambda_1,\ldots,\lambda_C,\boldsymbol{\lambda}'); gg is the identity, logit, probit, or log link. Its weighted estimating function for individual ii is

ψλ,i(λ,θ)=∑cWic(θ)dicαic(Yi−μic),\boldsymbol{\psi}_{\lambda,i}(\boldsymbol{\lambda},\boldsymbol{\theta}) = \sum_c W_{ic}(\boldsymbol{\theta})\mathbf{d}_{ic}\alpha_{ic}(Y_i-\mu_{ic}),

with αic=1\alpha_{ic}=1 for canonical links and αic=ϕ(ηic)/[Φ(ηic){1−Φ(ηic)}]\alpha_{ic}=\phi(\eta_{ic})/[\Phi(\eta_{ic})\{1-\Phi(\eta_{ic})\}] for the probit link, ηic=dic′λ\eta_{ic}=\mathbf{d}_{ic}'\boldsymbol{\lambda}. The sum over copies reflects that all copies of an individual derive from one observation.

Weight-model estimating functions. Each weight component jj is estimated by solving ∑iψj,i(θj)=0\sum_i\boldsymbol{\psi}_{j,i}(\boldsymbol{\theta}_j)=0, where ψj,i\boldsymbol{\psi}_{j,i} is the (weighted) maximum-likelihood score of that model. For the marginal treatment shares, ψπz,i=ui{1(Ti=z)−πz}\boldsymbol{\psi}_{\pi_z,i}=u_i\{1(T_i=z)-\pi_z\}. The mediator models are fitted with weights uiτiu_i\tau_i and therefore depend on the treatment-selection parameters: ψM,i=uiτi(θT,π)sM,i(θM)\boldsymbol{\psi}_{M,i}=u_i\tau_i(\boldsymbol{\theta}_T,\boldsymbol{\pi})\mathbf{s}_{M,i}(\boldsymbol{\theta}_M), with sM,i\mathbf{s}_{M,i} the individual score. The full stacked estimating function is

Ψi=(ψT,i′,ψπ,i′,ψM,i′,ψλ,i′)′,\boldsymbol{\Psi}_i = \left(\boldsymbol{\psi}_{T,i}',\boldsymbol{\psi}_{\pi,i}',\boldsymbol{\psi}_{M,i}',\boldsymbol{\psi}_{\lambda,i}'\right)',

in which the treatment-selection and marginal-share blocks are absent under randomization and the mediator block includes the odds-weight treatment models when those weights are used.

Sandwich variance. Let AN=−∑i∂Ψi/∂(θ′,λ′)\mathbf{A}_N=-\sum_i\partial\boldsymbol{\Psi}_i/\partial(\boldsymbol{\theta}',\boldsymbol{\lambda}') and BN=∑q(∑i∈qΨi)(∑i∈qΨi)′\mathbf{B}_N=\sum_q(\sum_{i\in q}\boldsymbol{\Psi}_i)(\sum_{i\in q}\boldsymbol{\Psi}_i)', where qq indexes clusters (individuals in the absence of clustering). The variance of (θ^,λ^)(\hat{\boldsymbol{\theta}},\hat{\boldsymbol{\lambda}}) is estimated by AN−1BNAN−T\mathbf{A}_N^{-1}\mathbf{B}_N\mathbf{A}_N^{-\mathrm{T}}, the empirical sandwich A−1BA−T/N\mathbf{A}^{-1}\mathbf{B}\mathbf{A}^{-\mathrm{T}}/N written in sums. Because the stack is block-lower-triangular (the weight models do not depend on λ\boldsymbol{\lambda}, and the mediator models depend on θT\boldsymbol{\theta}_T and π\boldsymbol{\pi} but not conversely), the λ\boldsymbol{\lambda} block can be computed through influence functions. For each weight component, let Hj=−∑i∂ψj,i/∂θj′\mathbf{H}_j=-\sum_i\partial\boldsymbol{\psi}_{j,i}/\partial\boldsymbol{\theta}_j' and

IF⁡j,i=Hj−1ψj,i(treatment and marginal-share models),(4)\operatorname{IF}_{j,i}=\mathbf{H}_j^{-1}\boldsymbol{\psi}_{j,i}\quad\text{(treatment and marginal-share models),} \tag*{(4)}
IFM,i=HM−1{ψM,i+∑j∈{T,π}(∑lψM,l∂log⁡τl∂θj)IFj,i}(mediator models).\mathrm{IF}_{M,i}=\mathbf{H}_{M}^{-1}\left\{\psi_{M,i}+\sum_{j\in\{T,\pi\}}\left(\sum_l\psi_{M,l}\frac{\partial\log\tau_l}{\partial\theta_j}\right)\mathrm{IF}_{j,i}\right\} \quad\text{(mediator models).}

Then, with

Q=∑i∑cWicdicdic′{αic∂μic∂ηic−(Yi−μic)∂αic∂ηic},Dj=∑i∑cWicdicαic(Yi−μic)∂log⁡Wic∂θj,\mathbf{Q}=\sum_i\sum_c W_{ic}d_{ic}d_{ic}^{\prime}\left\{\alpha_{ic}\frac{\partial\mu_{ic}}{\partial\eta_{ic}}-(Y_i-\mu_{ic})\frac{\partial\alpha_{ic}}{\partial\eta_{ic}}\right\},\qquad \mathbf{D}_j=\sum_i\sum_c W_{ic}d_{ic}\alpha_{ic}(Y_i-\mu_{ic})\frac{\partial\log W_{ic}}{\partial\theta_j},

and Ui=ψλ,i+∑jDjIFj,i\mathbf{U}_i=\psi_{\lambda,i}+\sum_j\mathbf{D}_j\mathrm{IF}_{j,i},

Var⁡^(λ^)=Q−1{∑q(∑i∈qUi)(∑i∈qUi)′}Q−1.\widehat{\operatorname{Var}}(\hat{\lambda})=\mathbf{Q}^{-1}\left\{\sum_q\left(\sum_{i\in q}\mathbf{U}_i\right)\left(\sum_{i\in q}\mathbf{U}_i\right)^{\prime}\right\}\mathbf{Q}^{-1}.

which equals the λ\lambda block of AN−1BNAN−⊤\mathbf{A}_N^{-1}\mathbf{B}_N\mathbf{A}_N^{-\top}. The derivatives of the log weight are ∂log⁡Wic/∂θj=hjcGj,i\partial\log W_{ic}/\partial\theta_j=h_{jc}\mathbf{G}_{j,i}, where Gj,i\mathbf{G}_{j,i} is the derivative of the log fitted probability or density of model jj for individual ii and hjch_{jc} indicates how that probability enters copy cc’s weight: −1-1 for the treatment-selection model (denominator of τi\tau_i); +1+1 for the marginal share; for the mediator model at treatment level tt, +1+1 if t=bct=b_c, −1-1 if t=act=a_c, and 0 otherwise (so the mediator models do not contribute when ac=bca_c=b_c); and the corresponding signs of the log-odds terms in Equation (3) for odds-ratio weights. For the heteroskedastic normal mediator model, for example, Gi=(xi′ri/σi2, zi′(ri2/σi2−1)/2)′\mathbf{G}_i=\left(x_i^{\prime}r_i/\sigma_i^2,\ z_i^{\prime}(r_i^2/\sigma_i^2-1)/2\right)^{\prime}, with ri=Mi−xi′βtr_i=M_i-x_i^{\prime}\beta_t and σi2=exp⁡(zi′γt)\sigma_i^2=\exp(z_i^{\prime}\gamma_t).

Potential-outcome means and effects. For a linear outcome model with centered covariates, the copy coefficients in λ\lambda are the potential-outcome means ν\nu. For a nonlinear outcome model, the means are standardized, ν^c=∑iuigc−1(λ^c+v~i′λ^v)/∑iui\hat{\nu}_c=\sum_i u_i g_c^{-1}(\hat{\lambda}_c+\tilde{\mathbf{v}}_i^{\prime}\hat{\lambda}_v)/\sum_i u_i, and Var⁡^(ν^)=JVar⁡^(λ^)J′\widehat{\operatorname{Var}}(\hat{\nu})=\mathbf{J}\widehat{\operatorname{Var}}(\hat{\lambda})\mathbf{J}^{\prime}, with J=∂ν^/∂λ′\mathbf{J}=\partial\hat{\nu}/\partial\lambda^{\prime}; in both cases the empirical covariate distribution over which the means are averaged is treated as fixed. The causal effects are linear contrasts δ=Lν\delta=\mathbf{L}\nu (Table 2), with Var⁡^(δ^)=LVar⁡^(ν^)L′\widehat{\operatorname{Var}}(\hat{\delta})=\mathbf{L}\widehat{\operatorname{Var}}(\hat{\nu})\mathbf{L}^{\prime}.

Relation to Bein et al. With a single mediator, randomized treatment, no sampling weights or outcome covariates, and a linear outcome model, the treatment-selection and marginal-share blocks are absent and the stack reduces to the mediator-model scores and the weighted estimating equations for the potential-outcome means, the construction of Bein et al. [3].

Coverage. The estimator applies whenever every weight component is a smooth function of parameters estimated by solving estimating equations. As implemented, this includes logit, probit, Poisson, ordered logit, multinomial logit, normal, and heteroskedastic normal mediator models; logit, probit, heteroskedastic probit, ordered, and multinomial logit treatment-selection models with estimated marginal shares; the treatment models underlying odds-ratio weights; and linear, logit, probit, and Poisson outcome models. It does not apply to stratification-based nonparametric RMPW or MMWS weights, which are step functions of estimated propensity scores: the weights are not differentiable in the propensity-model parameters, so Dj\mathbf{D}_j is not defined and stratum boundaries contribute variability that a first-order expansion does not capture. Nor does it apply to the kernel-based GPS estimator for continuous treatment. Bootstrap inference is used in these cases.

Yab=E[Y(a,M(b))]Y_{ab}=\mathrm{E}[Y(a,M(b))] for one mediator.

Simulation design and complete results

Supplementary Table S2 gives the data-generating mechanisms, true values of the estimands, and methods compared for every simulation design. Supplementary Table S3–Table S13 report the complete results for the designs summarized in § 5.

DesignTreatment TTMediator MMOutcome YYNIENDE
Replication of Hong et al. (2015). N=800N = 800, 5,000; 1,000 replications.
All designs: T∼Bernoulli⁡(0.5)T \sim\operatorname{Bernoulli}(0.5); \logit\Pr(M = 1 \mid T = 0) = \alpha_0 + X_1 + 0.5X_2, \logit\Pr(M = 1 \mid T = 1) = \alpha_1 + 0.5X_2 + X_3; Y=μTM+0.5S+εY = \mu_{TM} + 0.5S + \varepsilon, with μ00=0\mu_{00} = 0, μ10=θ1\mu_{10} = \theta_1, μ01=θ2\mu_{01} = \theta_2, μ11=θ1+θ2+θ3\mu_{11} = \theta_1 + \theta_2 + \theta_3.
(a)RandomizedE⁡(π0,π1)=(0.392,0.661)\operatorname{E}(\pi_0,\pi_1) = (0.392, 0.661)θ=(0.392,1,−1)\theta= (0.392, 1, -1)00
(b)RandomizedE⁡(π0,π1)=(0.392,0.661)\operatorname{E}(\pi_0,\pi_1) = (0.392, 0.661)θ=(2.158,0.5,−3.5)\theta= (2.158, 0.5, -3.5)−0.807-0.8070.7870.787
(c)RandomizedE⁡(π0,π1)=(0.194,0.807)\operatorname{E}(\pi_0,\pi_1) = (0.194, 0.807)θ=(1.112,0.5,−1.75)\theta= (1.112, 0.5, -1.75)−0.766-0.7660.7730.773
Methods: naive (mediator model without covariates); RMPW (known, two-step); NP 3×33 \times3 and 4×44 \times4 (known).
Mediator-model misspecification. N=800N = 800, 5,000; 1,000 replications.
All designs: treatment and outcome of design (b); the mediator models of the replication designs plus a nonlinear (q) or nonadditive (i) term of strength δ\delta, with α0,α1\alpha_0,\alpha_1 recalibrated so that E⁡(π0,π1)=(0.392,0.661)\operatorname{E}(\pi_0,\pi_1) = (0.392, 0.661).
NonlinearRandomizedT=0T = 0: +δ(X1−1)+\delta(X_1 - 1); T=1T = 1: +δ(X3−1)+\delta(X_3 - 1); δ=0.25,0.50,1\delta= 0.25, 0.50, 1as (b)−0.807-0.8070.7870.787
NonadditiveRandomizedT=0T = 0: +δX1X2+\delta X_1X_2; T=1T = 1: +δX2X3+\delta X_2X_3; δ=0.25,0.50,1\delta= 0.25, 0.50, 1as (b)−0.807-0.8070.7870.787
Methods: naive; RMPW with the linear additive (misspecified) mediator model (two-step); RMPW with the correct mediator model (two-step); NP 3×33 \times3 and 4×44 \times4 (known).
Alternative mediator, outcome, and treatment types. N=800N = 800, 5,000; 1,000 replications.
Continuous MM, commonBernoulli⁡(0.5)\operatorname{Bernoulli}(0.5)M=0.5T+0.5S+UM = 0.5T + 0.5S + UY=0.2T+0.4M+0.3TM+0.5S+εY = 0.2T + 0.4M + 0.3TM + 0.5S + \varepsilon0.3500.200
Continuous MM, arm-specificBernoulli⁡(0.5)\operatorname{Bernoulli}(0.5)T=0T = 0: M=0.5X1+0.25X2+UM = 0.5X_1 + 0.25X_2 + U; T=1T = 1: M=0.5+0.25X2+0.5X3+1.2UM = 0.5 + 0.25X_2 + 0.5X_3 + 1.2Uas above0.3500.200
Count MMBernoulli⁡(0.5)\operatorname{Bernoulli}(0.5)M∼Poisson⁡(λT)M \sim\operatorname{Poisson}(\lambda_T); ln⁡λ0=0.637+0.3X1+0.15X2\ln\lambda_0 = 0.637 + 0.3X_1 + 0.15X_2, ln⁡λ1=1.042+0.15X2+0.3X3\ln\lambda_1 = 1.042 + 0.15X_2 + 0.3X_3; E⁡(M∣T)=2,3\operatorname{E}(M \mid T) = 2, 3Y=0.2T+0.3M+0.1TM+0.5S+εY = 0.2T + 0.3M + 0.1TM + 0.5S + \varepsilon0.4000.400
Binary YYBernoulli⁡(0.5)\operatorname{Bernoulli}(0.5)Binary, as replication design (b)Pr⁡(Y=1)=Λ(−1+0.6T+1.2M+0.4TM+0.4S)\Pr(Y = 1) = \Lambda(-1 + 0.6T + 1.2M + 0.4TM + 0.4S)0.0930.145
Count YYBernoulli⁡(0.5)\operatorname{Bernoulli}(0.5)as aboveY∼Poisson⁡{exp⁡(0.3T+0.7M−0.2TM+0.2S)}Y \sim\operatorname{Poisson}\{\exp(0.3T + 0.7M - 0.2TM + 0.2S)\}0.2450.305
Three-level TTUniform on {0,1,2}\{0, 1, 2\}\logit\Pr(M = 1 \mid T = t): α0+X1+0.5X2\alpha_0 + X_1 + 0.5X_2; α1+0.5X2+X3\alpha_1 + 0.5X_2 + X_3; α2+0.5S\alpha_2 + 0.5S; E⁡(π0,π1,π2)=(0.40,0.60,0.75)\operatorname{E}(\pi_0,\pi_1,\pi_2) = (0.40, 0.60, 0.75)Y=βT+γTM+0.5S+εY = \beta_T + \gamma_T M + 0.5S + \varepsilon; β=(0,0.3,0.5)\beta= (0, 0.3, 0.5), γ=(0.6,0.8,1)\gamma= (0.6, 0.8, 1)0.160, 0.380; 0.350, 0.660
Methods: naive; RMPW (known, two-step); odds (known, two-step) for the continuous mediators; NP 4×44 \times4 for the binary and count outcomes and 3×33 \times3 for the three-level treatment (known).
Nonrandom treatment assignment. N=800N = 800, 5,000; 1,000 replications.
DesignTreatment TTMediator MMOutcome YYNIENDE
Binary MMPr⁡(T=1)=Λ(v)\Pr(T=1)=\Lambda(v)logit Pr⁡(M=1∣T)\Pr(M=1\mid T) as replication design (b) +0.5W1+0.5W_1; E(π0,π1)=(0.392,0.661)E(\pi_0,\pi_1)=(0.392,0.661)Y=0.3T+(0.6+0.3T)M+0.5S+0.4W1+0.4W2+εY=0.3T+(0.6+0.3T)M+0.5S+0.4W_1+0.4W_2+\varepsilon0.2420.418
Continuous MMas aboveM=T+0.5S+0.5W1+UM=T+0.5S+0.5W_1+UPr⁡(Y=1)=Λ{−1+0.5T+(0.6+0.2T)M+0.3S+0.4W1+0.4(W2−0.5)}\Pr(Y=1)=\Lambda\{-1+0.5T+(0.6+0.2T)M+0.3S+0.4W_1+0.4(W_2-0.5)\}0.1330.092
Three-level TTMultinomial logit; log odds vs. T=0T=0: v1v_1 (T=1T=1), v2v_2 (T=2T=2)as randomized three-level T+0.5W1T+0.5W_1as randomized three-level T+0.4W1+0.4W2T+0.4W_1+0.4W_20.160<br>0.3500.380<br>0.660
Methods: naive; mediator selection only (no treatment weights); IPTW (known, two-step); MMWS 5 and 10 (known); IPTW with a misspecified treatment model omitting W2W_2 (known).
Treatment-model specification and weight stability. N=800N=800, 5,000; 1,000 replications.
LogitPr⁡(T=1)=Λ(v)\Pr(T=1)=\Lambda(v)as nonrandom binary MMas nonrandom binary MM0.2420.418
ProbitPr⁡(T=1)=Φ(v/1.6)\Pr(T=1)=\Phi(v/1.6)as aboveas above0.2420.418
Heterosk. probitPr⁡(T=1)=Φ{v/(1.6egW1)}\Pr(T=1)=\Phi\{v/(1.6e^{gW_1})\}; g=0.50,0.25,0.10g=0.50,0.25,0.10as aboveas above0.2420.418
Methods: IPTW from logit, probit, and heteroskedastic probit (variance covariate W1W_1) treatment models (two-step).
Ordinal treatment. N=800N=800, 5,000; 1,000 replications.
Ordinal TTOrdered logit: T∗=0.6v+T^*=0.6v+ logistic error; cutpoints −0.4-0.4, 0.80.8as nonrandom three-level TTas nonrandom three-level TT0.160<br>0.3500.380<br>0.660
Methods: IPTW from multinomial logit, ordered logit, and ordered probit treatment models; odds from multinomial logit and ordered logit (two-step).
Heteroskedastic continuous mediator. N=800N=800, 5,000.
Four levels of ccas nonrandom binary MMM=T+0.5S+0.5W1+exp⁡{0.5(1.5cX1+cTX2)}UM=T+0.5S+0.5W_1+\exp\{0.5(1.5cX_1+cTX_2)\}U; c=0.40,0.20,0.10,0c=0.40,0.20,0.10,0Y=0.3T+(0.5+0.2T)M+0.5S+0.4W1+0.4W2+εY=0.3T+(0.5+0.2T)M+0.5S+0.4W_1+0.4W_2+\varepsilon0.7000.300
Methods: (a) 1,000 replications: constant-variance mediator model; heteroskedastic model with the variance depending on all mediator covariates or on X1X_1 and X2X_2; odds (all with IPTW, two-step). (b) 500 replications: heteroskedastic model (all cc) and constant-variance model (c=0c=0), two-step vs. bootstrap on the same data sets.
Continuous treatment. N=800N=800, 5,000; 1,000 replications.
Continuous TTT=1+0.5X1+0.5X2+εTT=1+0.5X_1+0.5X_2+\varepsilon_TM=0.5+0.6T+0.4X1+0.3X2+UM=0.5+0.6T+0.4X_1+0.3X_2+UY=0.3T+0.5M+0.2TM+0.5X1+0.5X2+εY=0.3T+0.5M+0.2TM+0.5X_1+0.5X_2+\varepsilon0.540<br>-0.3000.520<br>-0.520
Covariates X1X_1, X2X_2 only. Estimands: dose 2 vs. 1 and dose 0 vs. 1. Methods: GPS weighting [13] with the default bandwidth and half of it (bootstrap).
Variance estimation. N=800N=800; 500 replications.
Binary MM; three-level TTas nonrandom designsas nonrandom designsas nonrandom designs

Table S2. Simulation design: data-generating mechanisms, true values of the estimands, and methods compared, by simulation design [23].

DesignNNMethodBias%BREmpSEModSECovRMSEFail
A. Natural indirect effect
(a)800Naive0.1640.0000.0410.0370.0110.1700
RMPW, known weights0.0030.9820.0790.0710.9460.0790
RMPW, two-step0.0030.9820.0790.0660.9330.0790
Nonparametric 3×33 \times30.0300.8170.0590.0630.9350.0660
Nonparametric 4×44 \times40.0190.8870.0620.0680.9570.0650
(a)5,000Naive0.1620.0000.0170.0150.0000.1630
RMPW, known weights−0.002-0.0020.9910.0290.0300.9610.0290
RMPW, two-step−0.002-0.0020.9910.0290.0280.9390.0290
Nonparametric 3×33 \times30.0260.8370.0210.0230.8070.0340
Nonparametric 4×44 \times40.0170.8960.0220.0250.9230.0280
(b)800Naive0.1620.0000.0890.0370.1790.1850
RMPW, known weights−0.004-0.0040.9770.1320.1050.8670.1320
RMPW, two-step−0.004-0.0040.9770.1320.1320.9450.1320
Nonparametric 3×33 \times30.0260.8420.1130.0900.8570.1160
Nonparametric 4×44 \times40.0360.7780.1200.1010.8500.1250
(b)5,000Naive0.1640.0000.0370.0150.0010.1680
RMPW, known weights0.0010.9940.0550.0440.8780.0550
RMPW, two-step0.0010.9940.0550.0540.9500.0550
Nonparametric 3×33 \times30.0270.8330.0460.0330.7540.0530
Nonparametric 4×44 \times40.0180.8920.0460.0360.8300.0490
(c)800Naive0.3820.0000.1020.1000.0470.3960
RMPW, known weights0.0070.9810.1680.1600.9390.1680
RMPW, two-step0.0070.9810.1680.1420.9140.1680
Nonparametric 3×33 \times30.0730.8100.1330.1550.9220.1520
Nonparametric 4×44 \times40.0820.7860.1390.1490.8990.1610
(c)5,000Naive0.3870.0000.0420.0400.0000.3890
RMPW, known weights0.0040.9890.0690.0710.9460.0690
RMPW, two-step0.0040.9890.0690.0640.9230.0690
Nonparametric 3×33 \times30.0680.8250.0510.0600.8110.0850
Nonparametric 4×44 \times40.0440.8870.0540.0650.9100.0700
B. Natural direct effect
(a)800Naive−0.164-0.1640.0000.1050.1070.6660.1950
RMPW, known weights−0.003-0.0030.9830.1290.1240.9520.1290
RMPW, two-step−0.003-0.0030.9830.1290.1210.9480.1290
Nonparametric 3×33 \times3−0.030-0.0300.8190.1170.1190.9510.1200
Nonparametric 4×44 \times4−0.019-0.0190.8850.1210.1230.9550.1220
(a)5,000Naive−0.161-0.1610.0000.0400.0430.0280.1660
RMPW, known weights0.0030.9840.0490.0500.9480.0490
RMPW, two-step0.0030.9840.0490.0490.9460.0490
Nonparametric 3×33 \times3−0.025-0.0250.8420.0450.0460.9250.0510
Nonparametric 4×44 \times4−0.016-0.0160.9010.0450.0470.9480.0480
(b)800Naive−0.165-0.1650.0000.1270.1190.6950.2080
RMPW, known weights0.0010.9930.1480.1550.9610.1480
RMPW, two-step0.0010.9930.1480.1450.9490.1480
Nonparametric 3×33 \times3−0.027-0.0270.8370.1310.1450.9570.1330
Nonparametric 4×44 \times4−0.024-0.0240.8550.1390.1530.9540.1410
(b)5,000Naive−0.162-0.1620.0000.0550.0470.1110.1710
RMPW, known weights0.0010.9920.0610.0630.9510.0610
RMPW, two-step0.0010.9920.0610.0590.9440.0610
DesignNNMethodBias%BREmpSEModSECovRMSEFail
Nonparametric 3×33 \times3−0.0250.8440.0540.0560.9360.0600
Nonparametric 4×44 \times4−0.0160.9040.0540.0570.9540.0560
(c)800Naive−0.3820.0000.1380.1370.2140.4070
RMPW, known weights−0.0070.9810.1880.1870.9450.1880
RMPW, two-step−0.0070.9810.1880.1680.9260.1880
Nonparametric 3×33 \times3−0.0680.8210.1690.1840.9260.1820
Nonparametric 4×44 \times4−0.0640.8320.1730.1820.9310.1850
(c)5,000Naive−0.3880.0000.0570.0550.0000.3920
RMPW, known weights−0.0050.9870.0770.0810.9570.0770
RMPW, two-step−0.0050.9870.0770.0740.9360.0770
Nonparametric 3×33 \times3−0.0690.8230.0620.0710.8610.0920
Nonparametric 4×44 \times4−0.0450.8850.0640.0750.9170.0780

Table S3. Replication of Hong et al. (2015): randomized binary treatment, binary mediator, and continuous outcome; 1,000 replications per cell.

DesignNNMethodBiasEmpSEModSECovRMSEFail
A. Natural indirect effect
dose 2 vs. 1800Default bandwidth0.0050.1280.1180.9260.1280
Half bandwidth−0.0400.1470.1380.8860.1530
dose 2 vs. 15,000Default bandwidth0.0050.0790.0730.9340.0790
Half bandwidth−0.0130.0980.0920.8930.0990
dose 0 vs. 1800Default bandwidth−0.0210.0980.0940.9500.1010
Half bandwidth0.0270.1130.1100.9120.1160
dose 0 vs. 15,000Default bandwidth−0.0080.0660.0580.9240.0660
Half bandwidth0.0080.0780.0720.9080.0780
B. Natural direct effect
dose 2 vs. 1800Default bandwidth−0.0480.2600.2590.9340.2650
Half bandwidth0.1500.3250.3300.9060.3580
dose 2 vs. 15,000Default bandwidth−0.0420.1550.1530.9530.1600
Half bandwidth0.0270.2000.1990.9340.2010
dose 0 vs. 1800Default bandwidth0.0390.2330.2250.9410.2360
Half bandwidth−0.1240.2860.2880.9140.3120
dose 0 vs. 15,000Default bandwidth0.0280.1330.1300.9510.1360
Half bandwidth−0.0250.1710.1700.9280.1730

Table S13. Continuous treatment: GPS weighting under alternative bandwidths, with bootstrap standard errors (200 resamples); 1,000 replications per cell.

Methods: IPTW (two-step, bootstrap); MMWS 5 and 10, NP 3×33 \times3, and odds (bootstrap).

Notes: Covariates are independent: X1,X2,X3,W1∼N(0,1)X_1, X_2, X_3, W_1 \sim N(0,1) and W2∼Bernoulli⁡(0.5)W_2 \sim\operatorname{Bernoulli}(0.5) (W1W_1 and W2W_2 only in designs with nonrandom treatment); S=X1+X2+X3S = X_1 + X_2 + X_3; ε,U∼N(0,1)\varepsilon, U \sim N(0,1); and Λ\Lambda and Φ\Phi are the logistic and standard normal distribution functions. Treatment indices: v=0.5X1−0.5X2+0.3X3+0.8W1+0.8(W2−0.5)v = 0.5X_1 - 0.5X_2 + 0.3X_3 + 0.8W_1 + 0.8(W_2 - 0.5) and v2=−0.3X1+0.5X2+0.5X3+0.5W1−0.8(W2−0.5)v_2 = -0.3X_1 + 0.5X_2 + 0.5X_3 + 0.5W_1 - 0.8(W_2 - 0.5). Mediator intercepts αt\alpha_t were calibrated so that E{Pr⁡(M=1∣T=t)}E\{\Pr(M = 1 \mid T = t)\} takes the stated values. True values are the NIE and NDE of treatment level 1 vs. 0 (and 2 vs. 0 where two contrasts are shown); for binary and count outcomes they are on the mean (probability or count) scale and were obtained by Gauss–Hermite integration over the covariates. The analysis models used the covariates X1X_1–X3X_3 (plus W1W_1 for the mediator and X1X_1–X3X_3, W1W_1, W2W_2 for the treatment in designs with nonrandom treatment) as linear additive terms, except where stated. Known: standard errors accounting for estimation of the weights as known; two-step: standard errors accounting for estimation of the weights using the generalized two-step estimator described in § 2.4.2 and Supplementary Section S2; bootstrap: 200 resamples. RMPW: parametric ratio-of-mediator-probability weights; NP k×kk \times k: nonparametric RMPW with kk strata of each mediator propensity score; odds: odds-ratio mediator weights; IPTW: inverse-probability-of-treatment weights; MMWS kk: marginal mean weighting through stratification with kk strata.

Bias, mean estimate minus the true value; EmpSE, empirical standard error; ModSE, mean model standard error; Cov, coverage of nominal 95% intervals; RMSE, root mean squared error; Fail, replications without an estimate. %BR, proportion of the naive estimator’s bias removed. Scenario (a): NIE = NDE = 0; (b): NIE = −0.807, NDE = 0.787; (c): NIE = −0.766, NDE = 0.773. Maximum Monte Carlo standard errors: bias 0.006; EmpSE 0.004; ModSE 0.002; coverage 0.015; RMSE 0.005.

ScenarioN=5,000N = 5{,}000N=800N = 800
RMPWNP 3×33 \times3NP 4×44 \times4RMPWNP 3×33 \times3NP 4×44 \times4
Natural indirect effect
(a)0.991 (0.998)0.837 (0.856)0.896 (0.904)0.982 (0.985)0.817 (0.856)0.887 (0.885)
(b)0.994 (0.991)0.833 (0.865)0.892 (0.913)0.977 (0.990)0.842 (0.864)0.778 (0.874)
(c)0.989 (0.999)0.825 (0.856)0.887 (0.904)0.981 (0.994)0.810 (0.823)0.786 (0.776)
Natural direct effect
(a)0.984 (0.999)0.842 (0.857)0.901 (0.905)0.983 (0.984)0.819 (0.843)0.885 (0.872)
(b)0.992 (0.980)0.844 (0.875)0.904 (0.923)0.993 (0.999)0.837 (0.854)0.855 (0.864)
(c)0.987 (0.995)0.823 (0.859)0.885 (0.908)0.981 (0.988)0.821 (0.818)0.832 (0.771)

Table S4. Replication of Hong et al. (2015): proportion of naive-estimator bias removed, this replication and (in parentheses) Hong et al. [11].

NP, nonparametric RMPW. Monte Carlo standard errors of the proportions in this replication are at most 0.03.

DesignNNMethodBias%BREmpSEModSECovRMSEFail
A. Natural indirect effect
Nonlinear, δ=0.25\delta= 0.25800Naive0.1510.0000.0880.0910.6010.1750
RMPW, misspecified0.0710.5300.1200.1220.9200.1390
RMPW, correct model0.0100.9370.1520.1330.9490.1520
Nonparametric 3×33 \times30.0280.8170.1130.0910.8510.1160
Nonparametric 4×44 \times40.0380.7460.1200.1010.8490.1260
Nonlinear, δ=0.25\delta= 0.255,000Naive0.1530.0000.0360.0360.0180.1570
RMPW, misspecified0.0750.5100.0480.0490.6670.0890
RMPW, correct model0.0020.9890.0680.0590.9400.0680
Nonparametric 3×33 \times30.0290.8080.0430.0330.7640.0520
Nonparametric 4×44 \times40.0190.8740.0450.0370.8450.0490
Nonlinear, δ=0.50\delta= 0.50800Naive0.1420.0000.0920.0920.6440.1690
RMPW, misspecified0.1080.2370.1150.1170.8520.1580
RMPW, correct model0.0270.8120.1710.1400.9120.1730
Nonparametric 3×33 \times30.0360.7460.1120.0870.8230.1170
Nonparametric 4×44 \times40.0500.6500.1180.0950.8150.1280
Nonlinear, δ=0.50\delta= 0.505,000Naive0.1390.0000.0380.0370.0390.1440
RMPW, misspecified0.1020.2690.0470.0460.4100.1120
RMPW, correct model0.0060.9560.0870.0680.9210.0870
Nonparametric 3×33 \times30.0320.7690.0450.0310.7210.0550
Nonparametric 4×44 \times40.0250.8190.0460.0360.8080.0530
Nonlinear, δ=1\delta= 1800Naive0.1110.0000.0970.0950.7520.1470
RMPW, misspecified0.0890.1960.1120.1130.8760.1430
RMPW, correct model0.0370.6620.1820.1480.8980.1860
Nonparametric 3×33 \times30.0310.7160.1100.0780.7830.1140
Nonparametric 4×44 \times40.0370.6620.1160.0900.8160.1220
Nonlinear, δ=1\delta= 15,000Naive0.1140.0000.0380.0380.1520.1200
RMPW, misspecified0.0920.1960.0460.0450.4710.1020
RMPW, correct model0.0210.8160.1170.0790.8510.1191
Nonparametric 3×33 \times30.0340.7030.0450.0280.6540.0560
Nonparametric 4×44 \times40.0290.7430.0450.0320.7490.0540
Nonadditive, δ=0.25\delta= 0.25800Naive0.1520.0000.0910.0900.6040.1770
RMPW, misspecified0.0320.7880.1280.1260.9310.1320
RMPW, correct model0.0090.9420.1400.1340.9360.1400
Nonparametric 3×33 \times30.0300.8000.1130.0900.8430.1170
Nonparametric 4×44 \times40.0380.7460.1190.0990.8500.1250
Nonadditive, δ=0.25\delta= 0.255,000Naive0.1540.0000.0350.0360.0100.1580
RMPW, misspecified0.0300.8070.0500.0510.9060.0580
RMPW, correct model0.0000.9990.0600.0570.9590.0600
Nonparametric 3×33 \times30.0280.8170.0430.0320.7700.0510
Nonparametric 4×44 \times40.0190.8790.0440.0360.8470.0480
Nonadditive, δ=0.50\delta= 0.50800Naive0.1440.0000.0900.0920.6480.1700
RMPW, misspecified0.0460.6780.1260.1240.9350.1340
RMPW, correct model0.0110.9260.1620.1390.9330.1630
Nonparametric 3×33 \times30.0350.7570.1140.0860.8200.1190
Nonparametric 4×44 \times40.0370.7410.1210.0970.8230.1260
Nonadditive, δ=0.50\delta= 0.505,000Naive0.1420.0000.0360.0370.0300.1470
RMPW, misspecified0.0450.6860.0490.0490.8470.0660
RMPW, correct model0.0001.0000.0710.0630.9400.0710
DesignNNMethodBias%BREmpSEModSECovRMSEFail
Nonparametric 3×33 \times30.0320.7780.0430.0310.7280.0530
Nonparametric 4×44 \times40.0220.8460.0450.0350.8200.0500
Nonadditive, δ=1\delta= 1800Naive0.1120.0000.0920.0950.7650.1450
RMPW, misspecified0.0400.6470.1180.1200.9430.1250
RMPW, correct model0.0110.9010.1820.1460.9460.1820
Nonparametric 3×33 \times30.0280.7530.1080.0810.8310.1110
Nonparametric 4×44 \times40.0260.7650.1130.0910.8350.1160
Nonadditive, δ=1\delta= 15,000Naive0.1170.0000.0360.0380.1210.1220
RMPW, misspecified0.0460.6070.0460.0480.8360.0650
RMPW, correct model0.0030.9730.1270.0750.9260.1270
Nonparametric 3×33 \times30.0310.7340.0430.0290.7060.0530
Nonparametric 4×44 \times40.0240.7990.0430.0330.8060.0490
B. Natural direct effect
Nonlinear, δ=0.25\delta= 0.25800Naive−0.154-0.1540.0000.1290.1310.7750.2010
RMPW, misspecified−0.074-0.0740.5200.1350.1360.9100.1540
RMPW, correct model−0.013-0.0130.9190.1640.1460.9360.1640
Nonparametric 3×33 \times3−0.029-0.0290.8110.1310.1450.9550.1340
Nonparametric 4×44 \times4−0.020-0.0200.8670.1410.1520.9580.1420
Nonlinear, δ=0.25\delta= 0.255,000Naive−0.154-0.1540.0000.0510.0520.1620.1630
RMPW, misspecified−0.077-0.0770.5040.0550.0550.7100.0940
RMPW, correct model−0.003-0.0030.9790.0740.0640.9320.0740
Nonparametric 3×33 \times3−0.031-0.0310.7990.0520.0550.9270.0610
Nonparametric 4×44 \times4−0.021-0.0210.8650.0520.0580.9550.0560
Nonlinear, δ=0.50\delta= 0.50800Naive−0.144-0.1440.0000.1330.1310.7980.1960
RMPW, misspecified−0.111-0.1110.2320.1320.1310.8690.1720
RMPW, correct model−0.029-0.0290.7970.1850.1530.9180.1870
Nonparametric 3×33 \times3−0.038-0.0380.7390.1320.1420.9470.1370
Nonparametric 4×44 \times4−0.030-0.0300.7940.1380.1480.9570.1410
Nonlinear, δ=0.50\delta= 0.505,000Naive−0.140-0.1400.0000.0510.0520.2330.1490
RMPW, misspecified−0.102-0.1020.2680.0520.0520.4990.1150
RMPW, correct model−0.007-0.0070.9520.0930.0730.9120.0930
Nonparametric 3×33 \times3−0.033-0.0330.7660.0520.0540.9140.0610
Nonparametric 4×44 \times4−0.026-0.0260.8160.0520.0570.9410.0580
Nonlinear, δ=1\delta= 1800Naive−0.111-0.1110.0000.1310.1310.8550.1720
RMPW, misspecified−0.089-0.0890.1950.1280.1280.8950.1560
RMPW, correct model−0.038-0.0380.6590.1950.1610.9030.1990
Nonparametric 3×33 \times3−0.031-0.0310.7180.1290.1360.9570.1330
Nonparametric 4×44 \times4−0.023-0.0230.7900.1370.1450.9550.1390
Nonlinear, δ=1\delta= 15,000Naive−0.114-0.1140.0000.0540.0520.4130.1270
RMPW, misspecified−0.092-0.0920.1950.0530.0510.5630.1060
RMPW, correct model−0.021-0.0210.8120.1210.0840.8470.1221
Nonparametric 3×33 \times3−0.034-0.0340.6990.0540.0520.8880.0640
Nonparametric 4×44 \times4−0.030-0.0300.7400.0540.0550.9120.0620
Nonadditive, δ=0.25\delta= 0.25800Naive−0.147-0.1470.0000.1300.1300.7950.1960
RMPW, misspecified−0.027-0.0270.8140.1410.1400.9420.1430
RMPW, correct model−0.004-0.0040.9730.1510.1460.9450.1510
Nonparametric 3×33 \times3−0.025-0.0250.8290.1300.1440.9640.1330
Nonparametric 4×44 \times4−0.012-0.0120.9180.1370.1510.9680.1370
Nonadditive, δ=0.25\delta= 0.255,000Naive−0.154-0.1540.0000.0510.0520.1610.1620
RMPW, misspecified−0.030-0.0300.8040.0570.0570.9120.0640
RMPW, correct model0.0000.9980.0660.0620.9430.0660
Nonparametric 3×33 \times3−0.029-0.0290.8150.0520.0550.9330.0590
Nonparametric 4×44 \times4−0.019-0.0190.8770.0520.0570.9600.0550

Table S5. Table S5: Mediator-model misspecification: nonlinear and nonadditive departures from the mediator model (scenario (b) of Hong et al. [11] with a nonlinear or nonadditive term of strength δ\delta omitted from the mediator model); 1,000 replications per cell.

DesignNNMethodBias%BREmpSEModSECovRMSEFail
Nonadditive, δ=0.50\delta= 0.50800Naive−0.142-0.1420.0000.1260.1310.8060.1900
RMPW, misspecified−0.045-0.0450.6860.1360.1370.9390.1430
RMPW, correct model−0.009-0.0090.9380.1710.1500.9310.1710
Nonparametric 3×33 \times3−0.032-0.0320.7720.1290.1410.9620.1330
Nonparametric 4×44 \times4−0.012-0.0120.9140.1350.1480.9640.1350
Nonadditive, δ=0.50\delta= 0.505,000Naive−0.142-0.1420.0000.0540.0520.2280.1520
RMPW, misspecified−0.045-0.0450.6850.0570.0550.8510.0730
RMPW, correct model0.0001.0000.0770.0680.9380.0770
Nonparametric 3×33 \times3−0.032-0.0320.7780.0530.0540.9130.0610
Nonparametric 4×44 \times4−0.022-0.0220.8460.0530.0560.9340.0580
Nonadditive, δ=1\delta= 1800Naive−0.114-0.1140.0000.1290.1310.8660.1720
RMPW, misspecified−0.042-0.0420.6360.1320.1330.9390.1390
RMPW, correct model−0.013-0.0130.8850.1980.1580.9390.1980
Nonparametric 3×33 \times3−0.027-0.0270.7600.1290.1380.9440.1320
Nonparametric 4×44 \times4−0.003-0.0030.9770.1370.1440.9600.1370
Nonadditive, δ=1\delta= 15,000Naive−0.117-0.1170.0000.0520.0530.3830.1280
RMPW, misspecified−0.046-0.0460.6060.0530.0530.8590.0700
RMPW, correct model−0.003-0.0030.9720.1320.0790.9170.1320
Nonparametric 3×33 \times3−0.031-0.0310.7330.0510.0530.9190.0600
Nonparametric 4×44 \times4−0.024-0.0240.7980.0520.0550.9410.0570

Table 6.

Bias, mean estimate minus the true value; EmpSE, empirical standard error; ModSE, mean model standard error; Cov, coverage of nominal 95% intervals; RMSE, root mean squared error; Fail, replications without an estimate. %BR, proportion of the naive estimator’s bias removed. True values: NIE=−0.807\mathrm{NIE} = -0.807, NDE=0.787\mathrm{NDE} = 0.787. Misspecified: linear additive mediator model; correct model: mediator model with the omitted term; both with two-step standard errors. The nonparametric standard errors treat the weights as known. Maximum Monte Carlo standard errors: bias 0.006; EmpSE 0.004; coverage 0.016.

DesignNNMethodBias%BREmpSEModSECovRMSEFail
A. Natural indirect effect
Continuous MM, common800Naive0.2130.0000.1090.0500.0890.2390
RMPW, known weights−0.0030.9880.0670.0580.9090.0670
RMPW, two-step−0.0030.9880.0670.0660.9500.0670
Odds0.0001.0000.0680.0540.8750.0680
Odds, two-step0.0001.0000.0680.0670.9520.0680
Continuous MM, common5,000Naive0.2150.0000.0450.0190.0000.2200
RMPW, known weights0.0010.9940.0250.0220.9140.0250
RMPW, two-step0.0010.9940.0250.0240.9330.0250
Odds0.0020.9920.0280.0220.8760.0280
Odds, two-step0.0020.9920.0280.0270.9390.0280
Continuous MM, arm-specific800Naive0.1090.0000.0830.0410.3150.1370
RMPW, known weights−0.0030.9700.1040.0840.8930.1040
RMPW, two-step−0.0030.9700.1040.0960.9440.1040
Odds0.0610.4360.0780.0490.7020.0990
Odds, two-step0.0610.4360.0780.0780.8960.0990
Continuous MM, arm-specific5,000Naive0.1070.0000.0340.0160.0110.1120
RMPW, known weights−0.0010.9950.0450.0360.8990.0450
RMPW, two-step−0.0010.9950.0450.0410.9420.0450
Odds0.0580.4550.0320.0200.2440.0660
Odds, two-step0.0580.4550.0320.0310.5300.0660
Count MM800Naive0.2780.0000.0990.0590.0250.2950
RMPW, known weights−0.0020.9920.1090.0870.9310.1090
RMPW, two-step−0.0020.9920.1090.0910.9400.1090
Count MM5,000Naive0.2820.0000.0390.0240.0000.2840
RMPW, known weights−0.0020.9940.0410.0380.9400.0410
RMPW, two-step−0.0020.9940.0410.0390.9550.0410
Binary YY800Naive0.0240.0000.0200.0130.5190.0310
RMPW, known weights−0.0010.9470.0310.0280.9200.0310
RMPW, two-step−0.0010.9470.0310.0290.9360.0310
Nonparametric 4 × 4−0.0010.9710.0320.0290.9160.0320
Binary YY5,000Naive0.0250.0000.0080.0050.0150.0270
RMPW, known weights0.0000.9850.0120.0110.9190.0120
RMPW, two-step0.0000.9850.0120.0120.9380.0120
Nonparametric 4 × 40.0030.8780.0110.0100.9180.0110
Count YY800Naive0.1170.0000.0620.0410.2610.1320
RMPW, known weights0.0010.9950.0930.0780.9220.0930
RMPW, two-step0.0010.9950.0930.0810.9280.0930
Nonparametric 4 × 40.0050.9580.0860.0770.9270.0860
Count YY5,000Naive0.1150.0000.0240.0160.0000.1170
RMPW, known weights0.0000.9960.0330.0320.9490.0330
RMPW, two-step0.0000.9960.0330.0330.9550.0330
Nonparametric 4 × 40.0120.8930.0300.0280.9070.0320
Three-level TT, 1 vs. 0800Naive0.1210.0000.0680.0320.1590.1390
RMPW, known weights0.0040.9650.0770.0690.9260.0770
RMPW, two-step0.0040.9650.0770.0730.9440.0770
Nonparametric 3 × 30.0200.8380.0740.0650.9190.0760
Three-level TT, 1 vs. 05,000Naive0.1190.0000.0280.0130.0000.1230
RMPW, known weights0.0001.0000.0320.0280.9180.0320
RMPW, two-step0.0001.0000.0320.0300.9370.0320

Table S6. Alternative mediator, outcome, and treatment types: randomized treatment; 1,000 replications per cell.

DesignNNMethodBias%BREmpSEModSECovRMSEFail
Nonparametric 3×33 \times30.0200.8350.0270.0230.8320.0330
Three-level TT, 2 vs. 0800Naive0.2280.0000.0920.0650.1070.2460
RMPW, known weights−0.0040.9810.0840.0750.9260.0840
RMPW, two-step−0.0040.9810.0840.0770.9400.0840
Nonparametric 3×33 \times30.0300.8700.0860.0790.9110.0910
Three-level TT, 2 vs. 05,000Naive0.2310.0000.0360.0260.0000.2330
RMPW, known weights0.0000.9990.0300.0290.9530.0300
RMPW, two-step0.0000.9990.0300.0300.9550.0300
Nonparametric 3×33 \times30.0360.8430.0290.0280.7440.0460
B. Natural direct effect
Continuous MM, common800Naive−0.2150.0000.0960.1360.7290.2350
RMPW, known weights0.0010.9950.1240.1390.9750.1240
RMPW, two-step0.0010.9950.1240.1270.9560.1240
Odds−0.0010.9940.1220.1370.9690.1220
Odds, two-step−0.0010.9940.1220.1280.9530.1220
Continuous MM, common5,000Naive−0.2160.0000.0370.0540.0010.2190
RMPW, known weights−0.0020.9910.0500.0550.9650.0500
RMPW, two-step−0.0020.9910.0500.0500.9500.0500
Odds−0.0020.9890.0520.0550.9650.0520
Odds, two-step−0.0020.9890.0520.0510.9470.0520
Continuous MM, arm-specific800Naive−0.1060.0000.0990.1160.8900.1450
RMPW, known weights0.0060.9450.1400.1410.9600.1400
RMPW, two-step0.0060.9450.1400.1310.9460.1400
Odds−0.0590.4460.1170.1270.9400.1310
Odds, two-step−0.0590.4460.1170.1150.9150.1310
Continuous MM, arm-specific5,000Naive−0.1080.0000.0380.0460.3190.1140
RMPW, known weights0.0000.9980.0580.0580.9660.0580
RMPW, two-step0.0000.9980.0580.0540.9560.0580
Odds−0.0590.4520.0460.0500.8100.0750
Odds, two-step−0.0590.4520.0460.0460.7460.0750
Count MM800Naive−0.2810.0000.1040.1170.2950.2990
RMPW, known weights0.0000.9990.1360.1370.9640.1360
RMPW, two-step0.0000.9990.1360.1290.9480.1360
Count MM5,000Naive−0.2810.0000.0420.0470.0000.2840
RMPW, known weights0.0020.9930.0550.0570.9620.0550
RMPW, two-step0.0020.9930.0550.0540.9530.0550
Binary YY800Naive−0.0260.0000.0350.0380.9210.0430
RMPW, known weights0.0000.9820.0450.0450.9550.0450
RMPW, two-step0.0000.9820.0450.0430.9480.0450
Nonparametric 4×44 \times4−0.0020.9140.0450.0460.9520.0450
Binary YY5,000Naive−0.0250.0000.0150.0150.6200.0290
RMPW, known weights0.0000.9820.0190.0180.9400.0190
RMPW, two-step0.0000.9820.0190.0170.9280.0190
Nonparametric 4×44 \times4−0.0030.8760.0180.0180.9400.0180
Count YY800Naive−0.1140.0000.1020.1130.8500.1530
RMPW, known weights0.0020.9840.1360.1370.9600.1360
RMPW, two-step0.0020.9840.1360.1290.9430.1360
Nonparametric 4×44 \times4−0.0100.9110.1310.1360.9600.1320
Count YY5,000Naive−0.1170.0000.0410.0450.2420.1230
RMPW, known weights−0.0010.9900.0520.0550.9590.0520
RMPW, two-step−0.0010.9900.0520.0520.9500.0520
Nonparametric 4×44 \times4−0.0140.8810.0500.0520.9430.0520

Table continued (PDF p. 40).

DesignNNMethodBias%BREmpSEModSECovRMSEFail
Three-level TT, 1 vs. 0800Naive−0.1200.0000.1170.1290.8650.1670
RMPW, known weights−0.0040.9690.1400.1440.9540.1400
RMPW, two-step−0.0040.9690.1400.1380.9390.1400
Nonparametric 3×33 \times3−0.0060.9490.1600.1420.9110.1600
Three-level TT, 1 vs. 05,000Naive−0.1210.0000.0480.0510.3520.1300
RMPW, known weights−0.0020.9870.0590.0580.9390.0590
RMPW, two-step−0.0020.9870.0590.0560.9290.0590
Nonparametric 3×33 \times3−0.0210.8250.0550.0550.9290.0590
Three-level TT, 2 vs. 0800Naive−0.2280.0000.1280.1450.6580.2620
RMPW, known weights0.0040.9820.1460.1490.9570.1460
RMPW, two-step0.0040.9820.1460.1400.9470.1460
Nonparametric 3×33 \times3−0.0330.8580.1500.1530.9440.1530
Three-level TT, 2 vs. 05,000Naive−0.2310.0000.0540.0580.0160.2380
RMPW, known weights−0.0010.9960.0580.0590.9530.0580
RMPW, two-step−0.0010.9960.0580.0560.9380.0580
Nonparametric 3×33 \times3−0.0370.8410.0580.0590.9100.0680

Table 7. (continued)

Bias, mean estimate minus the true value; EmpSE, empirical standard error; ModSE, mean model standard error; Cov, coverage of nominal 95% intervals; RMSE, root mean squared error; Fail, replications without an estimate. %BR, proportion of the naive estimator’s bias removed. Continuous mediator with common or arm-specific covariate effects and residual variance; binary and count outcomes with a binary mediator; three-level treatment with a binary mediator. Maximum Monte Carlo standard errors: bias 0.005; EmpSE 0.004; ModSE 0.001; coverage 0.016; RMSE 0.006.

(continued)

DesignNNMethodBias%BREmpSEModSECovRMSEFail
Three-level TT, 2 vs. 05,000IPTW, W2W_2 omitted−0.0660.0050.1530.1910.9840.1670
Three-level TT, 2 vs. 05,000Naive0.0700.0000.0580.0640.8360.0910
Three-level TT, 2 vs. 05,000Mediator selection only0.443−5.2910.0620.0650.0000.4470
Three-level TT, 2 vs. 05,000IPTW, known weights0.0000.9940.0670.0810.9890.0670
Three-level TT, 2 vs. 05,000IPTW, two-step0.0000.9940.0670.0630.9290.0670
Three-level TT, 2 vs. 05,000MMWS 5 strata0.0390.4520.0600.0770.9650.0710
Three-level TT, 2 vs. 05,000MMWS 10 strata0.0140.8080.0630.0790.9840.0640
Three-level TT, 2 vs. 05,000IPTW, W2W_2 omitted−0.0700.0070.0590.0760.9080.0920

Table 8.

Bias, mean estimate minus the true value; EmpSE, empirical standard error; ModSE, mean model standard error; Cov, coverage of nominal 95% intervals; RMSE, root mean squared error; Fail, replications without an estimate. %BR, proportion of the naive estimator’s bias removed. Binary mediator: binary treatment and mediator; continuous mediator: binary treatment, continuous mediator, binary outcome; three-level: three-level treatment, binary mediator. MMWS and the misspecified model use standard errors that treat the weights as known. Maximum Monte Carlo standard errors: bias 0.005; EmpSE 0.004; ModSE 0.001; coverage 0.016; RMSE 0.005.

DesignNNMethodBiasEmpSEModSECovRMSEFail
A. Natural indirect effect
1 vs. 0800IPTW, mlogit0.0000.0980.0830.9340.0980
IPTW, ologit0.0010.0990.0850.9410.0990
IPTW, oprobit0.0010.0990.0850.9420.0990
Odds, mlogit0.0030.0520.0520.9470.0520
Odds, ologit0.0170.0430.0430.9510.0460
1 vs. 05,000IPTW, mlogit0.0000.0340.0340.9500.0340
IPTW, ologit0.0000.0340.0350.9560.0340
IPTW, oprobit0.0000.0340.0350.9540.0340
Odds, mlogit0.0000.0200.0200.9530.0200
Odds, ologit0.0160.0170.0170.8590.0230
2 vs. 0800IPTW, mlogit−0.0050.0970.0910.9330.0970
IPTW, ologit−0.0040.0950.0900.9380.0950
IPTW, oprobit−0.0050.0960.0910.9350.0960
Odds, mlogit0.0200.0890.0860.9450.0910
Odds, ologit0.0120.0870.0850.9550.0880
2 vs. 05,000IPTW, mlogit−0.0010.0360.0350.9470.0360
IPTW, ologit−0.0010.0360.0340.9460.0360
IPTW, oprobit−0.0010.0360.0350.9480.0360
Odds, mlogit0.0200.0340.0340.9200.0390
Odds, ologit0.0130.0330.0340.9540.0360
B. Natural direct effect
1 vs. 0800IPTW, mlogit−0.0080.1360.1230.9310.1370
IPTW, ologit−0.0050.1630.1500.9380.1630
IPTW, oprobit−0.0030.1630.1500.9430.1630
Odds, mlogit−0.0100.1070.1060.9470.1070
Odds, ologit−0.0210.1380.1370.9470.1400
1 vs. 05,000IPTW, mlogit−0.0070.0500.0500.9480.0510
IPTW, ologit−0.0020.0620.0610.9470.0620
IPTW, oprobit0.0010.0620.0610.9450.0620
Odds, mlogit−0.0070.0430.0420.9500.0430
Odds, ologit−0.0180.0550.0550.9380.0580
2 vs. 0800IPTW, mlogit−0.0110.1410.1280.9340.1410
IPTW, ologit0.0030.1410.1280.9300.1410
IPTW, oprobit−0.0080.1420.1280.9320.1420
Odds, mlogit−0.0360.1400.1300.9350.1440
Odds, ologit−0.0140.1390.1300.9360.1400
2 vs. 05,000IPTW, mlogit−0.0130.0540.0510.9370.0550
IPTW, ologit0.0010.0540.0510.9410.0540
IPTW, oprobit−0.0110.0540.0510.9430.0550
Odds, mlogit−0.0350.0530.0520.8970.0630
Odds, ologit−0.0130.0530.0520.9490.0540

Table S8. Table S8: Ordinal treatment: alternative treatment and mediator-weight models, with the treatment generated by an ordered logit model (two-step standard errors); 1,000 replications per cell.

Bias, mean estimate minus the true value; EmpSE, empirical standard error; ModSE, mean model standard error; Cov, coverage of nominal 95% intervals; RMSE, root mean squared error; Fail, replications without an estimate. True values: 1 vs. 0, NIE = 0.16, NDE = 0.38; 2 vs. 0, NIE = 0.35, NDE = 0.66. Maximum Monte Carlo standard errors: bias 0.005; EmpSE 0.004; ModSE 0.001; coverage 0.011; RMSE 0.004.

DesignNNMethodBiasEmpSEModSECovRMSEESSESS<5%Fail
A. Natural indirect effect
Logit800Logit−0.0080.0960.0820.9390.097––0
Probit−0.0100.0990.0840.9400.100––0
Het. probit−0.0100.0990.0830.9400.100––0
Logit5,000Logit−0.0030.0350.0340.9520.035––0
Probit−0.0040.0360.0350.9500.036––0
Het. probit−0.0040.0360.0350.9550.036––0
Probit800Logit−0.0080.0940.0820.9360.095––0
Probit−0.0090.0960.0830.9360.096––0
Het. probit−0.0090.0960.0820.9330.096––0
Probit5,000Logit−0.0040.0340.0340.9530.034––0
Probit−0.0040.0360.0350.9510.036––0
Het. probit−0.0040.0350.0350.9520.035––0
Het. probit, g=0.50g = 0.50800Logit−0.0070.0980.0830.9230.09929.60.0010
Probit−0.0070.0990.0830.9210.09928.00.0040
Het. probit−0.0180.1190.0920.9140.12026.10.0211
Het. probit, g=0.50g = 0.505,000Logit−0.0030.0370.0350.9370.03729.90.0010
Probit−0.0030.0380.0350.9350.03826.20.0100
Het. probit−0.0090.0600.0440.9280.06119.20.0750
Het. probit, g=0.25g = 0.25800Logit−0.0040.0900.0800.9380.09029.80.0000
Probit−0.0040.0910.0810.9350.09128.20.0060
Het. probit−0.0130.0990.0860.9310.10027.30.0090
Het. probit, g=0.25g = 0.255,000Logit−0.0030.0360.0340.9440.03630.00.0010
Probit−0.0030.0360.0350.9480.03626.50.0111
Het. probit−0.0080.0450.0390.9510.04622.30.0270
Het. probit, g=0.10g = 0.10800Logit−0.0100.0900.0820.9380.09130.10.0000
Probit−0.0110.0910.0820.9380.09228.90.0010
Het. probit−0.0140.0950.0840.9350.09628.70.0030
Het. probit, g=0.10g = 0.105,000Logit0.0000.0360.0340.9450.03630.30.0000
Probit0.0000.0360.0350.9460.03628.20.0010
Het. probit−0.0030.0390.0360.9400.03925.70.0080
B. Natural direct effect
Logit800Logit0.0120.1340.1190.9270.135––0
Probit0.0010.1420.1220.9200.142––0
Het. probit0.0070.1420.1200.9190.142––0
Logit5,000Logit0.0030.0520.0500.9360.052––0
Probit−0.0100.0560.0530.9290.057––0
Het. probit−0.0090.0570.0520.9300.057––0
Probit800Logit0.0200.1300.1180.9280.132––0
Probit0.0110.1360.1210.9250.136––0
Het. probit0.0140.1370.1200.9200.137––0
Probit5,000Logit0.0170.0490.0500.9400.052––0
Probit0.0040.0540.0530.9590.054––0
Het. probit0.0040.0530.0520.9530.053––0
Het. probit, g=0.50g = 0.50800Logit0.0030.1400.1260.9310.14029.60.0010
Probit−0.0130.1660.1310.9270.16628.00.0040
Het. probit0.0690.1670.1290.8760.18026.10.0211
Het. probit, g=0.50g = 0.505,000Logit−0.0020.0540.0530.9490.05429.90.0010

Table S9. Table S9: Treatment-model specification and weight stability: alternative fitted treatment models (IPTW, two-step standard errors); rows by data-generating treatment model; 1,000 replications per cell.

(continued)

DesignNNMethodBiasEmpSEModSECovRMSEESSESS<5%Fail
Probit−0.0230.0680.0600.9490.07226.20.0100
Het. probit0.0410.0910.0640.8060.10019.20.0750
Het. probit, g=0.25g = 0.25800Logit0.0060.1320.1210.9350.13229.80.0000
Probit−0.0090.1490.1270.9360.14928.20.0060
Het. probit0.0300.1440.1210.9130.14727.30.0090
Het. probit, g=0.25g = 0.255,000Logit0.0060.0510.0520.9540.05230.00.0010
Probit−0.0160.0720.0600.9580.07326.50.0111
Het. probit0.0170.0700.0580.9130.07222.30.0270
Het. probit, g=0.10g = 0.10800Logit0.0230.1290.1190.9220.13130.10.0000
Probit0.0130.1360.1210.9270.13628.90.0010
Het. probit0.0260.1330.1180.9180.13528.70.0030
Het. probit, g=0.10g = 0.105,000Logit0.0120.0510.0500.9390.05330.30.0000
Probit−0.0030.0570.0540.9490.05728.20.0010
Het. probit0.0030.0610.0540.9370.06125.70.0080

Table 9.

Bias, mean estimate minus the true value; EmpSE, empirical standard error; ModSE, mean model standard error; Cov, coverage of nominal 95% intervals; RMSE, root mean squared error; Fail, replications without an estimate. Design: data-generating treatment model (Het. probit, heteroskedastic probit with standard deviation exp⁡(gW1)\exp(gW_1)); Method: fitted treatment model. ESS, mean of the smallest Kish effective sample size of any copy (% of its observations); ESS<5%, proportion of replications in which some copy’s ESS fell below 5% (recorded for the heteroskedastic probit designs only). True NIE = 0.242, NDE = 0.418. Maximum Monte Carlo standard errors: bias 0.005; EmpSE 0.004; ModSE 0.002; coverage 0.013; RMSE 0.015.

DesignNNMethodBiasEmpSEModSECovRMSEESSESS<5%Fail
c=0.40c = 0.40800Constant variance−0.2350.2070.1700.4780.31318.50.12911
Heteroskedastic−0.0950.2340.1850.7580.25219.30.0492
Heteroskedastic (X1,X2X_1, X_2)−0.0900.2450.1790.7530.26119.60.0562
Odds−0.1440.2640.1680.6360.30116.00.1252
c=0.40c = 0.405,000Constant variance−0.2010.1030.0790.1720.22611.70.1982
Heteroskedastic−0.0360.1460.0960.7510.15015.50.0731
Heteroskedastic (X1,X2X_1, X_2)−0.0380.1450.0970.7460.15015.50.0741
Odds−0.0850.2660.1240.5440.2809.60.2721
c=0.20c = 0.20800Constant variance−0.2040.1550.1300.5540.25623.00.0261
Heteroskedastic−0.0670.1900.1610.8370.20122.50.0302
Heteroskedastic (X1,X2X_1, X_2)−0.0690.1840.1510.8390.19622.80.0241
Odds−0.1580.1730.1390.6480.23421.40.0220
c=0.20c = 0.205,000Constant variance−0.1660.0590.0540.1730.17619.60.0220
Heteroskedastic−0.0100.1290.0730.8750.12920.80.0180
Heteroskedastic (X1,X2X_1, X_2)−0.0110.1020.0730.8830.10221.10.0180
Odds−0.1420.0750.0670.3680.16119.00.0220
c=0.10c = 0.10800Constant variance−0.1280.1450.1300.7510.19324.20.0090
Heteroskedastic−0.0430.1790.1520.8970.18423.20.0271
Heteroskedastic (X1,X2X_1, X_2)−0.0470.1710.1440.8780.17723.60.0170
Odds−0.0940.1690.1430.7880.19423.70.0140
c=0.10c = 0.105,000Constant variance−0.1010.0560.0530.4710.11622.40.0060
Heteroskedastic−0.0120.0600.0570.9120.06122.80.0050
Heteroskedastic (X1,X2X_1, X_2)−0.0120.0630.0590.9090.06423.30.0040
Odds−0.0850.0750.0660.6400.11421.80.0060
c=0.00c = 0.00800Constant variance−0.0400.1530.1380.9080.15825.00.0080
Heteroskedastic−0.0380.1930.1500.9050.19723.60.0252
Heteroskedastic (X1,X2X_1, X_2)−0.0410.1620.1430.9060.16723.90.0192
Odds−0.0110.1960.1550.8880.19624.90.0120
c=0.00c = 0.005,000Constant variance−0.0110.0620.0570.9120.06324.10.0030
Heteroskedastic−0.0110.0610.0540.9170.06223.60.0030
Heteroskedastic (X1,X2X_1, X_2)−0.0130.0580.0540.9080.06024.00.0020
Odds−0.0040.0750.0700.9210.07523.60.0020
B. Natural direct effect
c=0.40c = 0.40800Constant variance0.2370.2280.1930.6090.32918.50.12911
Heteroskedastic0.0970.2500.1980.7730.26819.30.0492
Heteroskedastic (X1,X2X_1, X_2)0.0930.2630.1930.7850.27919.60.0562
Odds0.1470.2850.1920.7040.32116.00.1252
c=0.40c = 0.405,000Constant variance0.2010.1100.0870.2120.22911.70.1982
Heteroskedastic0.0360.1500.1020.7760.15415.50.0731
Heteroskedastic (X1,X2X_1, X_2)0.0390.1480.1030.7740.15315.50.0741
Odds0.0850.2680.1300.5860.2819.60.2721
c=0.20c = 0.20800Constant variance0.2030.1790.1570.6580.27123.00.0261
Heteroskedastic0.0690.2130.1780.8510.22322.50.0302
Heteroskedastic (X1,X2X_1, X_2)0.0700.2020.1680.8490.21422.80.0241
Odds0.1580.2170.1680.7350.26921.40.0220
c=0.20c = 0.205,000Constant variance0.1670.0690.0670.2720.18119.60.0220
Heteroskedastic0.0120.1310.0810.8990.13220.80.0180
Heteroskedastic (X1,X2X_1, X_2)0.0130.1050.0810.8960.10621.10.0180
DesignNNMethodBiasEmpSEModSECovRMSEESSESS<5%Fail
Odds0.1440.0830.0780.4600.16619.00.0220
c=0.10c = 0.10800Constant variance0.1310.1650.1540.8110.21124.20.0090
Heteroskedastic0.0460.1910.1710.9020.19623.20.0271
Heteroskedastic (X1,X2X_1, X_2)0.0500.1850.1630.8980.19223.60.0170
Odds0.0970.1960.1700.8340.21823.70.0140
c=0.10c = 0.105,000Constant variance0.1030.0670.0640.5910.12222.40.0060
Heteroskedastic0.0130.0690.0670.9300.07022.80.0050
Heteroskedastic (X1,X2X_1, X_2)0.0140.0710.0680.9280.07223.30.0040
Odds0.0870.0840.0770.7000.12121.80.0060
c=0.00c = 0.00800Constant variance0.0460.1800.1590.8980.18625.00.0080
Heteroskedastic0.0450.2130.1680.8940.21723.60.0252
Heteroskedastic (X1,X2X_1, X_2)0.0480.1870.1630.8990.19323.90.0192
Odds0.0170.2210.1800.8910.22224.90.0120
c=0.00c = 0.005,000Constant variance0.0090.0710.0660.9290.07124.10.0030
Heteroskedastic0.0090.0700.0640.9230.07123.60.0030
Heteroskedastic (X1,X2X_1, X_2)0.0110.0680.0640.9270.06824.00.0020
Odds0.0020.0850.0800.9360.08523.60.0020

Table S10. Heteroskedastic continuous mediator: alternative mediator-weight models, ln⁡var⁡(M∣T,X)=1.5cX1+cTX2\ln\operatorname{var}(M \mid T, X) = 1.5cX_1 + cTX_2 (IPTW, two-step standard errors); 1,000 replications per cell.

Bias, mean estimate minus the true value; EmpSE, empirical standard error; ModSE, mean model standard error; Cov, coverage of nominal 95% intervals; RMSE, root mean squared error; Fail, replications without an estimate.

ESS, mean of the smallest Kish effective sample size of any copy (% of its observations); ESS<5%, proportion of replications in which some copy’s ESS fell below 5%. True NIE = 0.70, NDE = 0.30. Maximum Monte Carlo standard errors: bias 0.009; EmpSE 0.006; ModSE 0.007; coverage 0.016; RMSE 0.029.

DesignNNMethodBiasEmpSEModSECovRMSEFail
A. Natural indirect effect
c=0.40c = 0.40800Heteroskedastic, two-step−0.0880.2410.1890.7840.2560
Heteroskedastic, bootstrap−0.0880.2410.2250.8620.2560
c=0.40c = 0.405,000Heteroskedastic, two-step−0.0240.2130.1040.7660.2140
Heteroskedastic, bootstrap−0.0240.2130.1140.7860.2140
c=0.20c = 0.20800Heteroskedastic, two-step−0.0570.1850.1630.8440.1930
Heteroskedastic, bootstrap−0.0570.1850.2020.8960.1930
c=0.20c = 0.205,000Heteroskedastic, two-step−0.0030.1260.0730.8760.1260
Heteroskedastic, bootstrap−0.0030.1260.0810.8980.1260
c=0.10c = 0.10800Heteroskedastic, two-step−0.0480.1680.1520.8820.1750
Heteroskedastic, bootstrap−0.0480.1680.1940.9320.1750
c=0.10c = 0.105,000Heteroskedastic, two-step−0.0100.0630.0580.8980.0640
Heteroskedastic, bootstrap−0.0100.0630.0660.9220.0640
c=0.00c = 0.00800Heteroskedastic, two-step−0.0340.1830.1530.8920.1860
Heteroskedastic, bootstrap−0.0340.1830.1910.9540.1860
Constant variance, two-step−0.0410.1730.1370.8800.1770
Constant variance, bootstrap−0.0410.1730.1550.9060.1770
c=0.00c = 0.005,000Heteroskedastic, two-step−0.0080.0580.0540.9060.0590
Heteroskedastic, bootstrap−0.0080.0580.0610.9280.0590
Constant variance, two-step−0.0070.0620.0580.9220.0620
Constant variance, bootstrap−0.0070.0620.0600.9240.0620
B. Natural direct effect
c=0.40c = 0.40800Heteroskedastic, two-step0.0840.2550.2030.8020.2690
Heteroskedastic, bootstrap0.0840.2550.2370.8620.2690
c=0.40c = 0.405,000Heteroskedastic, two-step0.0240.2150.1100.7940.2160
Heteroskedastic, bootstrap0.0240.2150.1200.8160.2160
c=0.20c = 0.20800Heteroskedastic, two-step0.0550.2020.1810.8720.2090
Heteroskedastic, bootstrap0.0550.2020.2170.9200.2090
c=0.20c = 0.205,000Heteroskedastic, two-step0.0050.1300.0810.9220.1300
Heteroskedastic, bootstrap0.0050.1300.0900.9360.1300
c=0.10c = 0.10800Heteroskedastic, two-step0.0480.1770.1690.9000.1830
Heteroskedastic, bootstrap0.0480.1770.2080.9380.1830
c=0.10c = 0.105,000Heteroskedastic, two-step0.0110.0720.0680.9260.0730
Heteroskedastic, bootstrap0.0110.0720.0750.9320.0730
c=0.00c = 0.00800Heteroskedastic, two-step0.0430.1940.1710.9060.1980
Heteroskedastic, bootstrap0.0430.1940.2060.9340.1980
Constant variance, two-step0.0510.1880.1570.8920.1950
Constant variance, bootstrap0.0510.1880.1740.9100.1950
c=0.00c = 0.005,000Heteroskedastic, two-step0.0070.0700.0640.9140.0700
Heteroskedastic, bootstrap0.0070.0700.0700.9160.0700
Constant variance, two-step0.0060.0730.0680.9220.0740
Constant variance, bootstrap0.0060.0730.0690.9220.0740

Table S11. Table S11: Heteroskedastic continuous mediator: two-step versus bootstrap standard errors (same simulated data sets for each pair); 500 replications per cell, 200 bootstrap resamples.

Bias, mean estimate minus the true value; EmpSE, empirical standard error; ModSE, mean model standard error; Cov, coverage of nominal 95% intervals; RMSE, root mean squared error; Fail, replications without an estimate. True NIE = 0.70, NDE = 0.30. Maximum Monte Carlo standard errors: bias 0.011; EmpSE 0.008; ModSE 0.007; coverage 0.019; RMSE 0.037.

DesignNNMethodBiasEmpSEModSECovRMSEFailed res.Fail
A. Natural indirect effect
Binary MM800IPTW, two-step−0.0090.0910.0820.9400.0910.000
IPTW, bootstrap−0.0090.0910.0900.9580.0910.000
MMWS 5, bootstrap−0.0070.0860.0880.9660.0860.000
MMWS 10, bootstrap−0.0080.0910.0910.9600.0910.000
Nonparametric, bootstrap0.0260.0790.0880.9700.0830.000
Odds, bootstrap−0.0040.0560.0560.9440.0560.000
Three-level TT, 1 vs. 0800IPTW, two-step−0.0040.1050.0910.9200.1050.000
IPTW, bootstrap−0.0040.1050.1040.9520.1050.040
MMWS 5, bootstrap−0.0080.0990.1030.9600.0990.010
MMWS 10, bootstrap−0.0100.1040.1060.9540.1040.020
Nonparametric, bootstrap0.0110.0940.1110.9820.0940.000
Odds, bootstrap−0.0410.0530.0520.8100.0670.010
Three-level TT, 2 vs. 0800IPTW, two-step−0.0060.1110.1030.9540.1110.000
IPTW, bootstrap−0.0060.1110.1210.9740.1110.040
MMWS 5, bootstrap0.0000.1060.1230.9800.1060.010
MMWS 10, bootstrap−0.0060.1120.1250.9740.1120.020
Nonparametric, bootstrap0.0460.1100.1310.9820.1190.000
Odds, bootstrap−0.0110.0920.0930.9520.0920.010
B. Natural direct effect
Binary MM800IPTW, two-step0.0040.1310.1190.9280.1310.000
IPTW, bootstrap0.0040.1310.1260.9480.1310.000
MMWS 5, bootstrap0.0520.1220.1210.9360.1330.000
MMWS 10, bootstrap0.0200.1250.1270.9420.1270.000
Nonparametric, bootstrap−0.0310.1260.1250.9460.1300.000
Odds, bootstrap−0.0020.1140.1080.9280.1140.000
Three-level TT, 1 vs. 0800IPTW, two-step0.0110.1680.1470.9340.1680.000
IPTW, bootstrap0.0110.1680.1600.9540.1680.040
MMWS 5, bootstrap0.0510.1540.1570.9500.1620.010
MMWS 10, bootstrap0.0250.1650.1650.9560.1660.020
Nonparametric, bootstrap0.0340.1890.2110.9640.1920.000
Odds, bootstrap0.0480.1460.1450.9280.1540.010
Three-level TT, 2 vs. 0800IPTW, two-step−0.0010.1740.1560.9300.1740.000
IPTW, bootstrap−0.0010.1740.1720.9520.1740.040
MMWS 5, bootstrap0.0350.1570.1690.9640.1610.010
MMWS 10, bootstrap0.0100.1700.1770.9580.1700.020
Nonparametric, bootstrap−0.0490.1770.2060.9740.1840.000
Odds, bootstrap0.0040.1810.1640.9340.1810.010

Table S12. Variance estimation: two-step and bootstrap inference across weighting methods (200 bootstrap resamples), N=800N = 800; 500 replications per cell.

Bias, mean estimate minus the true value; EmpSE, empirical standard error; ModSE, mean model standard error; Cov, coverage of nominal 95% intervals; RMSE, root mean squared error; Fail, replications without an estimate. Failed res., mean number of failed bootstrap resamples per replication. Designs as in Table S7. Maximum Monte Carlo standard errors: bias 0.008; EmpSE 0.006; ModSE 0.002; coverage 0.018; RMSE 0.007.

DesignNNMethodBias%BREmpSEModSECovRMSEFail
A. Natural indirect effect
Binary MM800Naive0.3220.0000.0720.0500.0000.3300
Mediator selection only−0.0160.9490.0830.0720.9200.0850
IPTW, known weights−0.0110.9670.0970.0840.9350.0970
IPTW, two-step−0.0110.9670.0970.0820.9340.0970
MMWS 5 strata−0.0090.9730.0940.0810.9330.0950
MMWS 10 strata−0.0090.9720.0960.0830.9340.0960
IPTW, W2W_2 omitted−0.0070.9770.0940.0820.9320.0940
Binary MM5,000Naive0.3210.0000.0290.0200.0000.3220
Mediator selection only−0.0150.9530.0310.0290.9210.0350
IPTW, known weights−0.0030.9920.0350.0340.9610.0350
IPTW, two-step−0.0030.9920.0350.0340.9600.0350
MMWS 5 strata−0.0020.9930.0340.0330.9550.0340
MMWS 10 strata−0.0020.9940.0350.0340.9560.0350
IPTW, W2W_2 omitted−0.0020.9950.0340.0330.9580.0340
Continuous MM800Naive0.1290.0000.0280.0250.0000.1320
Mediator selection only−0.0030.9800.0330.0310.9380.0330
IPTW, known weights−0.0060.9520.0340.0340.9510.0340
IPTW, two-step−0.0060.9520.0340.0330.9370.0340
MMWS 5 strata−0.0050.9640.0330.0340.9500.0340
MMWS 10 strata−0.0050.9590.0340.0340.9490.0340
IPTW, W2W_2 omitted−0.0050.9590.0340.0340.9490.0340
Continuous MM5,000Naive0.1300.0000.0110.0100.0000.1300
Mediator selection only0.0000.9980.0120.0130.9590.0120
IPTW, known weights−0.0010.9930.0130.0140.9640.0130
IPTW, two-step−0.0010.9930.0130.0130.9520.0130
MMWS 5 strata0.0000.9990.0130.0140.9650.0130
MMWS 10 strata0.0000.9960.0130.0140.9700.0130
IPTW, W2W_2 omitted−0.0010.9960.0130.0140.9630.0130
Three-level TT, 1 vs. 0800Naive0.2120.0000.0750.0400.0110.2250
Mediator selection only−0.0380.8220.0760.0630.8370.0850
IPTW, known weights−0.0080.9640.1040.0920.9320.1040
IPTW, two-step−0.0080.9640.1040.0910.9350.1040
MMWS 5 strata−0.0080.9610.1000.0880.9200.1000
MMWS 10 strata−0.0090.9560.1030.0910.9270.1040
IPTW, W2W_2 omitted0.0020.9910.1000.0860.9130.1000
Three-level TT, 1 vs. 05,000Naive0.2130.0000.0300.0160.0000.2150
Mediator selection only−0.0420.8050.0300.0250.6100.0510
IPTW, known weights0.0000.9990.0450.0430.9400.0450
IPTW, two-step0.0000.9990.0450.0410.9280.0450
MMWS 5 strata−0.0040.9800.0420.0390.9340.0420
MMWS 10 strata−0.0010.9950.0440.0410.9430.0440
IPTW, W2W_2 omitted0.0020.9930.0420.0390.9280.0420
Three-level TT, 2 vs. 0800Naive0.4110.0000.1070.0850.0010.4250
Mediator selection only0.0400.9020.0990.0910.8940.1060
IPTW, known weights−0.0030.9940.1130.0960.9180.1130
IPTW, two-step−0.0030.9940.1130.1020.9330.1130
MMWS 5 strata0.0010.9980.1060.0940.9240.1060
MMWS 10 strata−0.0010.9970.1130.0970.9110.1130
IPTW, W2W_2 omitted−0.0050.9870.1000.0890.9190.1000
Three-level TT, 2 vs. 05,000Naive0.4130.0000.0430.0340.0000.4160
Mediator selection only0.0410.9010.0370.0350.7680.0550
DesignNNMethodBias%BREmpSEModSECovRMSEFail
IPTW, known weights−0.0010.9980.0390.0350.9240.0390
IPTW, two-step−0.0010.9980.0390.0380.9450.0390
MMWS 5 strata0.0040.9910.0370.0350.9300.0370
MMWS 10 strata0.0010.9970.0380.0350.9250.0380
IPTW, W2W_2 omitted−0.0010.9970.0350.0320.9290.0350
B. Natural direct effect
Binary MM800Naive0.1640.0000.1070.1180.7370.1960
Mediator selection only0.502−2.0590.1310.1310.0330.5190
IPTW, known weights0.0090.9450.1310.1640.9900.1320
IPTW, two-step0.0090.9450.1310.1200.9280.1320
MMWS 5 strata0.0580.6460.1220.1530.9820.1350
MMWS 10 strata0.0290.8230.1250.1590.9830.1280
IPTW, W2W_2 omitted0.0870.4710.1240.1560.9600.1520
Binary MM5,000Naive0.1670.0000.0420.0470.0410.1720
Mediator selection only0.502−2.0140.0530.0520.0000.5050
IPTW, known weights0.0010.9970.0520.0660.9870.0520
IPTW, two-step0.0010.9970.0520.0500.9510.0520
MMWS 5 strata0.0530.6840.0490.0610.9100.0720
MMWS 10 strata0.0210.8730.0500.0630.9800.0540
IPTW, W2W_2 omitted0.0780.5290.0490.0630.8140.0930
Continuous MM800Naive−0.0160.0000.0350.0410.9630.0390
Mediator selection only0.116−6.4730.0440.0460.2930.1240
IPTW, known weights0.0070.5630.0480.0540.9720.0480
IPTW, two-step0.0070.5630.0480.0460.9480.0480
MMWS 5 strata0.016−0.0590.0460.0520.9720.0490
MMWS 10 strata0.0100.3530.0480.0540.9760.0490
IPTW, W2W_2 omitted0.019−0.2220.0460.0530.9670.0500
Continuous MM5,000Naive−0.0170.0000.0140.0160.8620.0220
Mediator selection only0.113−5.8220.0180.0190.0000.1140
IPTW, known weights0.0010.9330.0190.0220.9850.0190
IPTW, two-step0.0010.9330.0190.0190.9530.0190
MMWS 5 strata0.0120.2600.0180.0210.9490.0220
MMWS 10 strata0.0050.6890.0180.0220.9810.0190
IPTW, W2W_2 omitted0.0150.1200.0180.0210.9350.0230
Three-level TT, 1 vs. 0800Naive0.2840.0000.1260.1360.4460.3110
Mediator selection only0.534−0.8790.1520.1460.0450.5560
IPTW, known weights0.0020.9930.1640.1960.9830.1640
IPTW, two-step0.0020.9930.1640.1460.9210.1640
MMWS 5 strata0.0440.8460.1550.1890.9770.1610
MMWS 10 strata0.0140.9520.1630.1980.9830.1630
IPTW, W2W_2 omitted0.0740.7400.1510.1860.9720.1680
Three-level TT, 1 vs. 05,000Naive0.2940.0000.0480.0550.0000.2980
Mediator selection only0.549−0.8660.0550.0590.0000.5520
IPTW, known weights0.0010.9950.0720.0840.9860.0720
IPTW, two-step0.0010.9950.0720.0650.9270.0720
MMWS 5 strata0.0470.8400.0630.0780.9530.0780
MMWS 10 strata0.0160.9440.0660.0820.9810.0680
IPTW, W2W_2 omitted0.0770.7390.0630.0780.8850.0990
Three-level TT, 2 vs. 0800Naive0.0660.0000.1490.1610.9450.1630
Mediator selection only0.438−5.5900.1590.1650.2390.4650
IPTW, known weights0.0000.9990.1670.2030.9840.1670
IPTW, two-step0.0000.9990.1670.1540.9340.1670
MMWS 5 strata0.0370.4430.1570.1950.9860.1610
MMWS 10 strata0.0080.8840.1670.2040.9830.1680

Table S7. Nonrandom treatment assignment: alternative treatment-selection adjustments; 1,000 replications per cell.

Bias, mean estimate minus the true value; EmpSE, empirical standard error; ModSE, mean model standard error; Cov, coverage of nominal 95% intervals; RMSE, root mean squared error; Fail, replications without an estimate. True values: 2 vs. 1, NIE = 0.54, NDE = 0.52; 0 vs. 1, NIE = −0.30, NDE = −0.52. Maximum Monte Carlo standard errors: bias 0.010; EmpSE 0.007; ModSE 0.002; coverage 0.010; RMSE 0.007.

References

  1. [1]Peter C. Austin and Elizabeth A. Stuart. Moving towards best practice when using inverse probabil- ity of treatment weighting (IPTW) using the propensity score to estimate causal treatment effects in observational studies. Statistics in Medicine, 34(28):3661–3679, 2015. doi: 10.1002/sim.6607.DOI
  2. [2]Reuben M. Baron and David A. Kenny. The moderator–mediator variable distinction in social psy- chological research: Conceptual, strategic, and statistical considerations. Journal of Personality and Social Psychology, 51(6):1173–1182, 1986. doi: 10.1037/0022-3514.51.6.1173.pubmed.ncbi.nlm.nih.gov/3806354
  3. [3]Edward Bein, Jonah Deutsch, Guanglei Hong, Kristin E. Porter, Xu Qin, and Cheng Yang. Two-step estimation in ratio-of-mediator-probability weighted causal mediation analysis. Statistics in Medicine, 37(8):1304–1324, 2018. doi: 10.1002/sim.7581.DOI
  4. [4]Donna L. Coffman, Haoyu Zhou, Katherine E. Castellano, Megan S. Schuler, and Daniel F. McCaffrey. Sampling weighting strategies in causal mediation analysis. BMC Medical Research Methodology, 24:133, 2024. doi: 10.1186/s12874-024-02262-x.DOI
  5. [5]Stephen R. Cole and Miguel A. Hernán. Constructing inverse probability weights for marginal structural models. American Journal of Epidemiology, 168(6):656–664, 2008. doi: 10.1093/aje/kwn164.DOI
  6. [6]Helmut Farbmacher, Martin Huber, Lukáš Lafférs, Henrika Langen, and Martin Spindler. Causal mediation analysis with double machine learning. The Econometrics Journal, 25(2):277–300, 2022. doi: 10.1093/ectj/utac003.
  7. [7]Nima S. Hejazi, Kara E. Rudolph, and Iván Díaz. medoutcon: Nonparametric efficient causal mediation analysis with machine learning in R. Journal of Open Source Software, 7(69):3979, 2022. doi: 10.21105/joss.03979.DOI
  8. [8]Guanglei Hong. Ratio of mediator probability weighting for estimating natural direct and indirect effects. In JSM Proceedings, Biometrics Section, pages 2401–2415, Alexandria, VA, 2010. American Statistical Association.
  9. [9]Guanglei Hong. Marginal mean weighting through stratification: a generalized method for evaluating multivalued and multiple treatments with nonexperimental data. Psychological Methods, 17:44–60, 2012. doi: 10.1037/a0024918.DOI
  10. [10]Guanglei Hong. Causality in a Social World: Moderation, Mediation and Spill-over. Wiley, Chichester, UK, 2015. doi: 10.1002/9781119030638.DOI
  11. [11]Guanglei Hong, Jonah Deutsch, and Heather D. Hill. Ratio-of-mediator-probability weighting for causal mediation analysis in the presence of treatment-by-mediator interaction. Journal of Educational and Behavioral Statistics, 40(3):307–340, 2015. doi: 10.3102/1076998615583902.DOI
  12. [12]Guanglei Hong, Xu Qin, and Fan Yang. Weighting-based sensitivity analysis in causal mediation studies. Journal of Educational and Behavioral Statistics, 43(1):32–56, 2018. doi: 10.3102/1076998617749561.DOI
  13. [13]Yu-Chin Hsu, Martin Huber, Ying-Ying Lee, and Layal Lettry. Direct and indirect effects of continuous treatments based on generalized propensity score weighting. Journal of Applied Econometrics, 35(7):814–840, 2020. doi: 10.1002/jae.2765.DOI
  14. [14]Martin Huber. Identifying causal mechanisms (primarily) based on inverse probability weighting. Journal of Applied Econometrics, 29:920–943, 2014. doi: 10.1002/jae.2341.DOI
  15. [15]Kosuke Imai, Luke Keele, and Dustin Tingley. A general approach to causal mediation analysis. Psychological Methods, 15:309–334, 2010. doi: 10.1037/a0020761.DOI
  16. [16]Leslie Kish. Survey Sampling. Wiley, New York, 1965.
  17. [17]Ariel Linden. Combining propensity score-based stratification and weighting to improve causal inference in the evaluation of health care interventions. Journal of Evaluation in Clinical Practice, 20:1065–1071, 2014. doi: 10.1111/jep.12254.DOI
  18. [18]Ariel Linden. MMWSMED: Stata module for conducting causal mediation analysis using ratio-of-mediator-probability weighting with parametric or nonparametric weights. Statistical Software Components S460014, Boston College Department of Economics, 2026.
  19. [19]Ariel Linden and Kristian Bernt Karlson. Using mediation analysis to identify causal mechanisms in disease management interventions. Health Services and Outcomes Research Methodology, 13:86–108, 2013. doi: 10.1007/s10742-013-0106-5.DOI
  20. [20]Ariel Linden and Paul R. Yarnold. Identifying causal mechanisms in health care interventions using classification tree analysis. Journal of Evaluation in Clinical Practice, 24:353–361, 2018. doi: 10.1111/jep.12848.DOI
  21. [21]David P. MacKinnon, Amanda J. Fairchild, and Matthew S. Fritz. Mediation analysis. Annual Review of Psychology, 58:593–614, 2007. doi: 10.1146/annurev.psych.58.110405.085542.DOI
  22. [22]Dan McCaffrey, Katherine Castellano, Donna Coffman, Brian Vegetable, Megan Schuler, and Haoyu Zhou. twangMediation: Twang Causal Mediation Modeling via Weighting, 2026. URL https://CRAN.R-project.org/package=twangMediation. R package version 1.2.1.
  23. [23]Tim P. Morris, Ian R. White, and Michael J. Crowther. Using simulation studies to evaluate statistical methods. Statistics in Medicine, 38(11):2074–2102, 2019. doi: 10.1002/sim.8086.DOI
  24. [24]Trang Quynh Nguyen, Ian Schmid, Elizabeth L. Ogburn, and Elizabeth A. Stuart. Clarifying causal mediation analysis: Effect identification via three assumptions and five potential outcomes. Journal of Causal Inference, 10(1):246–279, 2022. doi: 10.1515/jci-2021-0049.DOI
  25. [25]Trang Quynh Nguyen, Elizabeth L. Ogburn, Ian Schmid, Elizabeth B. Sarker, Noah Greifer, Ina M. Koning, and Elizabeth A. Stuart. Causal mediation analysis: From simple to more robust strategies for estimation of marginal natural (in)direct effects. Statistics Surveys, 17:1–41, 2023. doi: 10.1214/22-SS140.DOI
  26. [27]Xu Qin, Guanglei Hong, and Fan Yang. rmpw: Causal Mediation Analysis Using Weighting Approach, 2026. URL https://CRAN.R-project.org/package=rmpw. R package version 0.0.7.
  27. [28]James M. Robins and Sander Greenland. Identifiability and exchangeability for direct and indirect effects. Epidemiology, 3(2):143–155, 1992. doi: 10.1097/00001648-199203000-00013.DOI
  28. [29]Megan S. Schuler and Rebecca L. Collins. Early alcohol and smoking initiation: a contributor to sexual minority disparities in adult use. American Journal of Preventive Medicine, 57(6):808–817, 2019. doi: 10.1016/j.amepre.2019.07.020.DOI
  29. [30]Megan S. Schuler, Cara E. Rice, Rebecca J. Evans-Polce, and Rebecca L. Collins. Disparities in substance use behaviors and disorders among adult sexual minorities by age, gender, and sexual identity. Drug and Alcohol Dependence, 189:139–146, 2018. doi: 10.1016/j.drugalcdep.2018.05.008.DOI
  30. [31]Megan S. Schuler, Donna L. Coffman, Elizabeth A. Stuart, Trang Q. Nguyen, Brian Vegavetable, and Daniel F. McCaffrey. Practical challenges in mediation analysis: a guide for applied researchers. Health Services and Outcomes Research Methodology, 25(1):57–84, 2025. doi: 10.1007/s10742-024-00327-4.DOI
  31. [32]Eric J. Tchetgen Tchetgen. Inverse odds ratio-weighted estimation for causal mediation analysis. Statistics in Medicine, 32:4567–4580, 2013. doi: 10.1002/sim.5864.DOI
  32. [34]Tyler J. VanderWeele. Explanation in Causal Inference: Methods for Mediation and Interaction. Oxford University Press, New York, 2015.
  33. [35]Tyler J. VanderWeele, Stijn Vansteelandt, and James M. Robins. Effect decomposition in the presence of an exposure-induced mediator-outcome confounder. Epidemiology, 25(2):300–306, 2014. doi: 10.1097/EDE.0000000000000034.DOI

Paper details

vlnc:W_cx7jt86a
Version
v1 · latest
Published
Fresh — added to the Hub within the last 30 days
Added
License
CC BY 4.0
Usage

views · 0

downloads · 0

A view counts once per browser per day, a download each time the PDF is requested.

  • Mathematics

Not recorded for this paper

View statement
Looking for related papers…
Contents