betaFeedback

WorksMathematics

LaMe: a Joint Latent Mediation Framework for High-Dimensional Systems

Abstract

Mediation analysis seeks to understand how an exposure influences an outcome through intermediate variables. However, as increasingly rich datasets become available, hundreds to thousands of potential mediators make these systems difficult to model and hard to interpret. Latent-variable approaches offer a natural way to accommodate these high-dimensional settings. However, existing latent mediation methods face limitations, including overparameterization, biased inference, or limited scalability. We develop LaMe, a joint linear-Gaussian framework that summarizes a high-dimensional exposure-mediator-outcome system through a small number of latent variables. We identify natural direct and indirect effects under an explicit assumption that the manifest measurements fully capture the latent mediators, and characterize conditions under which the latent representation effectively restricts the observed cross-covariance structure. Simulations assess recovery of observed association parameters, natural effects, and manifest features associated with latent mediation pathways across a range of challenging settings. Four real-world applications demonstrate competitive predictive performance with compact latent representations, including single-dimensional representations of biomarker and proteomic measurements. Together, these results establish LaMe as a practical framework for estimating and summarizing multivariate mediation systems under explicit structural and measurement assumptions.

Introduction

Deciphering how complex exposures exert their effects through intermediary processes is a cornerstone of modern epidemiology. In the NHANES data considered here, for example, lifestyle characteristics may directly influence glycohemoglobin levels, but might also affect blood biomarkers which in turn cause these variations. However, the continued emergence of large-scale, high-dimensional datasets poses new challenges to established mediation analysis methods. Analyses conducted on biological data are particularly affected, as molecular profiling can yield hundreds or thousands of correlated molecular measurements, only a subset of which may be relevant to the mediating process linking an exposure to an outcome. Latent variable models have been introduced to extend classical mediation models to these settings [1, 7, 12] (Chén, Crainiceanu, Ogburn, Caffo, Wager, and Lindquist 2017). This approach brings a number of improvements over non-latent mediation techniques [20] (Zhu, He, Ma, Hu, Li, and Li 2024) (e.g. based on OLS, LASSO or ElasticNet): noise decomposition, information compression, interpretability and robustness of the estimates.

However, existing latent mediation approaches face several limitations. Stepwise methods estimate latent scores separately from subsets of the observed data and use the resulting scores in subsequent regressions on remaining observed variables, potentially inferring latent variables that are irrelevant to mediation systems [12] (Chén et al. 2017). Joint approaches avoid this issue by estimating latent variables along with their relation to observed variables, but may be overparameterized, prone to overfitting, limited in scalability, and typically assume univariate exposures, univariate outcomes, and the conditional independence among manifest variables given the latent mediators whereas biological measurement are generally correlated [1, 7]. To address these limitations, we introduce the Latent Mediation (LaMe) framework, which jointly models multivariate exposures, outcomes and high-dimensional manifest variables through a small number of latent mediators without assuming diagonality of noise covariances, while explicitly accounting for direct exposure-outcome effects.

This approach makes three main methodological contributions. First, LaMe is novel in that it infers latent mediators from a joint probabilistic model with multivariate exposures and outcomes, high-dimensional manifest variables, and non-diagonal residual covariance matrices. As a result, it avoids biases associated with estimating latent scores separately from the rest of the system as in stepwise procedures and estimates all its parameters based on all available information rather

This paper has been submitted for consideration for publication in Biometrics than subsets of observed variables. We characterize when the representation imposes a genuine restriction on cross-block relationships, distinguishing dimension reduction from a reparameterization of unrestricted regression. Second, LaMe connects joint probabilistic estimation methods [1, 7] with representations in which latent mediators are exactly recoverable from the manifest measurements [12, 21] (Chén et al. 2017). Under this explicit restriction and suitable rank conditions, the aggregate natural direct and indirect effects are statistically identified from the observed data. Throughout this article, identifiability refers to statistical identification from the observed-data distribution, rather than causal identification of natural effects. We also introduce a score to assess which manifest variables are associated with exposures and informative about outcomes through the latent mediators, independently of how the latent variables are represented. Third, we derive closed-form EM updates for two equivalent likelihood factorizations, allowing computational efficiency to be improved when manifest measurements and exposures have substantially different dimensions. Simulations assess parameter recovery, effect recovery under the exact recoverability assumption, and feature ranking; real-world applications assess predictive performance and the compactness of the learned representations. These results position LaMe as a useful methodological option.

Background

Let N>1N > 1 be the number of observations and, for each observation, let I≥1I \ge1 be the number of exposures, J≥1J \ge1 the number of manifest variables, and L≥1L \ge1 the number of outcomes. Then, let X∈RN×IX \in\mathbb{R}^{N \times I} denote the exposure matrix, M∈RN×JM \in\mathbb{R}^{N \times J} the manifest variables matrix and Y∈RN×LY \in\mathbb{R}^{N \times L} the outcome matrix.

Non-latent models

In their simplest form, linear mediation models are specified by the following set of structural causal equations (see Figure 1-left):

Figure 1. Non-latent (left-panel) and latent (middle-panel) mediation systems, and overview of several associated inference procedures (right-panel) — (left and middle panels) Variables XX, MM, SS and YY are specified in the ovals, structural coefficients matrices β\beta are shown in rounded rectangles, and the system’s dimensions are given in white squares. Arrows represent causal relations between variables. For variables with a causal path to the outcome, squares’ color represents the total causal effect of a unit intervention on the corresponding variable on the outcome YY under the model specified in Eq. (2)–in this case, the sum of all causal paths leading to YY; in the latent graph, the colors of the manifest-variable squares represent their conditional associations with YY, not causal effects. For both squares and arrows, blue represents a negative relation, red represents a positive relation, and color intensity represents its strength. (right-panel) Common optimization approaches to mediation systems and the order in which they infer parameters. Alt text: three-panel schematic comparing non-latent and latent mediation systems and their inference strategies. Exposures influence outcomes directly and through latent mediators; red and blue arrows indicate positive and negative relationships. The right panel groups methods by two-step, three-step, or fully joint estimation.

X=εxM=Xβ(xm)+εmY=Xβ(xy)+Mβ(my)+εy(1)\begin{aligned} X &= \varepsilon_x\\ M &= X\beta^{(xm)}+\varepsilon_m\\ Y &= X\beta^{(xy)}+M\beta^{(my)}+\varepsilon_y \tag*{(1)} \end{aligned}

The random vectors εx,n\varepsilon_{x,n}, εm,n\varepsilon_{m,n} and εy,n\varepsilon_{y,n} corresponding to the nthn^{\mathrm{th}} observation are usually assumed to be mean-zero Gaussians and mutually independent. Formally, for each observation nn, εx,n∼N(0,Σx)\varepsilon_{x,n}\sim\mathcal{N}(0,\Sigma_x), εm,n∼N(0,Σm∣x)\varepsilon_{m,n}\sim\mathcal{N}(0,\Sigma_{m\mid x}) and εy,n∼N(0,Σy∣x,m)\varepsilon_{y,n}\sim\mathcal{N}(0,\Sigma_{y\mid x,m}); these terms account for noise and/or exogenous variables. Coefficient matrices are β(xm)∈RI×J\beta^{(xm)}\in\mathbb{R}^{I\times J}, β(xy)∈RI×L\beta^{(xy)}\in\mathbb{R}^{I\times L}, β(my)∈RJ×L\beta^{(my)}\in\mathbb{R}^{J\times L} and noise covariances are Σm∣x∈RJ×J\Sigma_{m\mid x}\in\mathbb{R}^{J\times J}, Σy∣x,m∈RL×L\Sigma_{y\mid x,m}\in\mathbb{R}^{L\times L}. We consider that all variables are centered. As shown in Web Appendix 6, the inclusion of confounders CC between XX, MM and YY is straightforward and is therefore omitted here for clarity.

According to recent surveys [17, 20] (Zhu et al. 2024), most high-dimensional mediation analyses published in the literature rely on non-latent linear mediation models, under which the parameters of Eq. (1) are estimated using sequential regression procedures combined with penalization. Classical mediation analysis is based on ordinary least squares (OLS) estimation of β(xm)\beta^{(xm)}, β(my)\beta^{(my)} and β(xy)\beta^{(xy)}, while extensions to high-dimensional settings incorporate regularization to select the relevant mediators or improve estimation or prediction accuracy. Typical examples include the LASSO, Ridge and ElasticNet regressions. Another example is the Pathway LASSO model (Zhao and Luo, 2022), which introduces a nonlinear penalty targeting entire indirect paths rather than individual coefficients. These approaches allow for straightforward coefficient estimation, quickly converge to global optima, and provide readily interpretable results. On the other hand, they heavily rely on hyperparameter tuning and may yield unstable and selection-inconsistent estimates in the presence of strongly correlated mediators or high-dimensional regimes, where mediator selection can become effectively arbitrary [18] (Ye, Zhu, and Coffman 2021).

Latent models

Latent mediators.

Structural equations. To overcome some of these limitations, latent models have been suggested, which assume the existence of unobserved variables, denoted by S∈RN×KS\in\mathbb{R}^{N\times K}, where KK is the number of latent dimensions. Following the latent mediation formulations of [1, 7], LaMe is specified by the following structural equations (see Figure 1-middle):

X=εxS=Xβ(xs)+εsM=Sβ(sm)+εmY=Xβ(xy)+Sβ(sy)+εy(2)\begin{aligned} X &= \varepsilon_x\\ S &= X\beta^{(xs)}+\varepsilon_s\\ M &= S\beta^{(sm)}+\varepsilon_m\\ Y &= X\beta^{(xy)}+S\beta^{(sy)}+\varepsilon_y \tag*{(2)} \end{aligned}

The coefficient matrices of the model are β(xs)∈RI×K\beta^{(xs)}\in\mathbb{R}^{I\times K}, β(sm)∈RK×J\beta^{(sm)}\in\mathbb{R}^{K\times J}, β(xy)∈RI×L\beta^{(xy)}\in\mathbb{R}^{I\times L}, β(sy)∈RK×L\beta^{(sy)}\in\mathbb{R}^{K\times L}. The noise vectors εx,n\varepsilon_{x,n}, εs,n\varepsilon_{s,n}, εm,n\varepsilon_{m,n} and εy,n\varepsilon_{y,n} corresponding to the nthn^{\mathrm{th}} observation are assumed to be mean-zero Gaussians and mutually independent. Formally, for each observation nn, εx,n∼N(0,Σx)\varepsilon_{x,n}\sim\mathcal{N}(0,\Sigma_x), εs,n∼N(0,Σs)\varepsilon_{s,n}\sim\mathcal{N}(0,\Sigma_s), εm,n∼N(0,Σm∣s)\varepsilon_{m,n}\sim\mathcal{N}(0,\Sigma_{m\mid s}) and εy,n∼N(0,Σy∣x,s)\varepsilon_{y,n}\sim\mathcal{N}(0,\Sigma_{y\mid x,s}), for some noise covariance matrices Σx∈RI×I\Sigma_x\in\mathbb{R}^{I\times I}, Σs∈RK×K\Sigma_s\in\mathbb{R}^{K\times K}, Σm∣s∈RJ×J\Sigma_{m\mid s}\in\mathbb{R}^{J\times J}, Σy∣x,s∈RL×L\Sigma_{y\mid x,s}\in\mathbb{R}^{L\times L}.

To simplify the exposition, we omit observed confounders (say CC) from the subsequent derivations. As shown in Web Appendix 6, the inclusion of confounders between XX, SS and YY is straightforward and requires only replacing the exposure variable with a composite variable ξ:=[X,C]\xi:= [X,C], and appropriately partitioning the estimated coefficients corresponding to XX and CC.

Natural effects. Under sequential ignorability, namely {Y(x,s),S(x′)}⊥X∣C=c\{Y(x,s),S(x')\}\perp X\mid C=c and Y(x,s)⊥S(x′)∣X=x′,C=cY(x,s)\perp S(x')\mid X=x',C=c where Y(x,s)Y(x,s) and S(x′)S(x') denote the counterfactual outcome and mediator under the corresponding interventions, the natural direct and indirect effects associated with a one-unit increase in the exposures are given by [1, 7]:

NDE=β(xy);NIE=β(xs)β(sy),(3)NDE=\beta^{(xy)}\quad;\qquad NIE=\beta^{(xs)}\beta^{(sy)}, \tag*{(3)}

Stepwise approaches.

Stepwise approaches to latent mediation generally define SS as a deterministic linear combination of manifest variables. These methods typically derive latent scores from an initial model, for instance from Principal Component Analysis (PCA), Partial Least Squares (PLS) or their variants; then, these scores enter as observed mediators in the subsequent estimation of a non-latent mediation model [4, 11, 21] (Hu, DeSimone, and Wang 2024). However, most existing stepwise approaches do not use all available data to derive their latent scores. For instance, approaches based on PCA optimize SS to explain the variance of MM, and approaches based on PLS optimize SS to explain the covariance between XX and MM, or MM and YY, regardless of whether the resulting scores best characterize mediation between XX and YY. Conversely, a correctly specified model accounting for the joint variance and covariance of (X,M,Y)(X, M, Y) does not require latent scores to be selected using only a subset of the observed relationships.

Joint modeling

A joint approach that maximizes a likelihood corresponding to the full structural model may therefore be preferable.

The latent approaches of [12] (Chén et al. 2017) both consider an univariate exposure and outcome, and both estimate the first latent direction and the associated path coefficients from a joint objective based on all observed data. In (Chén et al., 2017), the model extracts subsequent directions through approximate deflation while keeping earlier directions fixed. Consequently, the retained directions generally differ from those estimated by maximizing the joint likelihood of a full multivariate latent model; in particular, the sequential objective does not include cross-score residual covariances. Finally, the authors remark that inference may become cumbersome when J≫1J \gg1 without resorting to additional data augmentation techniques. The model introduced in [12] extends this approach beyond linear relationships to model SS as an arbitrary deterministic function of MM. However, this extension is limited to a single latent variable. Moreover, score selection and the estimated direct–indirect decomposition are sensitive to the scale of YY, as shown in Web Appendix 9. Finally, standardization of the univariate score is imposed algorithmically rather than incorporated into the objective function, complicating the analytical characterization of the resulting estimator.

Joint probabilistic formulations, on the other hand, explicitly account for invariance to latent rescaling. In [1], the authors formulate a joint probabilistic latent model with a single latent mediator. The approach was later extended by [7] to handle multivariate latent mediators. In that work, the authors assume a univariate exposure and outcome, and constrain the conditional noise covariance of manifest variables given latent mediators Σm∣s\Sigma_{m|s} to be diagonal. In this setting, however, we show in Web Appendix 10 that the non-regularized and correctly specified model from [7] is equivalent to Factor Analysis of MM followed by linear regressions on the resulting SS when K>1K > 1. Additionally, the diagonal constraint on Σm∣s\Sigma_{m|s} imposes conditional independence of manifest variables given the latent mediators, which is not necessarily true when MM represents biomarkers. Finally, Condition 1 in [7] is required for identification of the effects, but the proposed model does not guarantee that this condition holds.

Methods

We develop LaMe to extend existing approaches by jointly modeling all observed variables, allowing non-diagonal noise covariance matrices and multivariate exposures, outcomes and latent information bottleneck. Under the exact recoverability assumption for SS, the framework also permits identification of natural effects.

We first detail the exact recoverability and information bottleneck assumptions made by our approach, then we develop a general inference procedure for an auxiliary model that does not assume exact recoverability, and we finally show in Proposition 1 that exact recoverability can be recovered through a likelihood-equivalent reparametrization of the model post-estimation. This proposition further establishes that the parametric representations of natural direct and indirect effects are statistically identified under this condition.

Assumptions

Exact recoverability

Following score-based mediation approaches, which define latent mediators as linear combinations of manifest variables [12, 21] (Chén et al. 2017), we assume the same kind of deterministic relationship holds within our probabilistic framework. Specifically, we will work under what we call the exact recoverability assumption, under which S=E[S∣M]:=Mβ(ms)S = \mathbb{E}[S|M] := M\beta^{(\mathrm{ms})}. We emphasize that this equality does not imply that the measurements MM are causes of SS; they remain the manifest variables causally influenced by SS, with SS causally influenced by XX. This requirement is equivalent to:

Σm∣sβ(ms)=0(4)\Sigma_{m|s}\beta^{(\mathrm{ms})} = 0 \tag*{(4)}

where β(ms)=((β(sm))TΣsβ(sm)+Σm∣s)−1(β(sm))T(Σs∣x+(β(xs))TΣxβ(xs))\beta^{(\mathrm{ms})} = \left((\beta^{(\mathrm{sm})})^{\mathsf{T}}\Sigma_s\beta^{(\mathrm{sm})}+\Sigma_{m|s}\right)^{-1}(\beta^{(\mathrm{sm})})^{\mathsf{T}}\left(\Sigma_{s|x}+(\beta^{(\mathrm{xs})})^{\mathsf{T}}\Sigma_x\beta^{(\mathrm{xs})}\right).

We point out that the exact-recoverability approximation becomes increasingly accurate as the number of manifest variables JJ grows, making it well suited to high-dimensional settings. To see this, consider the difference between the unconstrained latent variable S′S' in Eq. (2) and its exactly recoverable counterpart S∗=Mβ(ms)S^* = M\beta^{(\mathrm{ms})}. Assuming that additional manifest variables provide non-vanishing, non-redundant information about SS, the measurement precision β(sm)Σm∣s−1(β(sm))T\beta^{(\mathrm{sm})}\Sigma_{m|s}^{-1}(\beta^{(\mathrm{sm})})^{\mathsf{T}} grows linearly with JJ. Then, we show in Web Appendix 7 that E[∥S∗−S′∥2]=O(J−1)\mathbb{E}[\lVert S^* - S'\rVert^2] = \mathcal{O}(J^{-1}), meaning that S′→J→∞S∗S' \xrightarrow{J\to\infty} S^*. Thus, as the number of informative manifest variables increases, the latent variable SS of the unconstrained model in Eq. (2) asymptotically converges to the latent mediator that is exactly recoverable from MM. Although exact recoverability is an idealization for finite JJ, it therefore becomes a reasonable approximation when sufficiently many informative manifest variables are available, supporting its use in high-dimensional settings.

We show in Proposition 1 below that the joint model specified in Eq. (2) can always be formulated to satisfy exact recoverability.

Information bottleneck

References

  1. [1]Albert, J. M., Geng, C., and Nelson, S. (2016). Causal mediation analysis with a latent mediator. *Biometrical Journal* 58, 535–548.DOI
  2. [2]Anderson, T. W., Rubin, H., et al. (1956). Statistical inference in factor analysis. In *Proceedings of the third Berkeley symposium on mathematical statistics and probability*, volume 5, pages 111–150.
  3. [3]Bishop, C. M. (2006). *Pattern Recognition and Machine Learning*. New York: Springer.
  4. [4]Cai, X., Wang, Q., and Zhu, Y. (2023). Mediation analysis with latent factors using simultaneous group-wise and parameter-wise penalization. *Stat* 12, 630.DOI
  5. [5]Chechik, G., Globerson, A., Tishby, N., and Weiss, Y. (2005). Information bottleneck for gaussian variables. *Journal of Machine Learning Research* 6, 165–188.
  6. [7]Derkach, A., Pfeiffer, R. M., Chen, T.-H., and Sampson, J. N. (2019). High dimensional mediation analysis with latent variables. *Biometrics* 75, 745–756.DOI
  7. [8]Hayashi, F. (2000). *Econometrics*. Princeton University Press.
  8. [11]Kawano, S., Fujisawa, H., Takada, T., and Shiroishi, T. (2018). Sparse principal component regression for generalized linear models. *Computational Statistics & Data Analysis* 124, 180–196.DOI
  9. [12]Nath, T., Caffo, B. B., Wager, T., and Lindquist, M. A. (2023). A machine learning based approach towards high-dimensional mediation analysis. *NeuroImage* 268, 119843.DOI
  10. [13]National Center for Health Statistics (2024). National Health and Nutrition Examination Survey, August 2021–August 2023. U.S. Department of Health and Human Services, Centers for Disease Control and Prevention. Available at https://www.cdc.gov/nchs/nhanes/continuousnhanes/default.aspx?Cycle=2021–2023 (accessed June 26, 2025).
  11. [14]Neal, R. M. and Hinton, G. E. (1998). *A View of the Em Algorithm that Justifies Incremental, Sparse, and other Variants*, pages 355–368. Springer Netherlands.DOI
  12. [15]Organisation for Economic Co-operation and Development (2023). PISA 2022 Database. OECD. Available at https://www.oecd.org/en/data/datasets/pisa-2022-database.html (accessed June 26, 2025).DOI
  13. [16]Redmond, M. (2002). Communities and Crime. UCI Machine Learning Repository. Available at https://doi.org/10.24432/C53W3X (accessed June 26, 2025).
  14. [17]Rijnhart, J. J. M., Lamp, S. J., Valente, M. J., MacKinnon, D. P., Twisk, J. W. R., and Heymans, M. W. (2021). Mediation analysis methods used in observational research: a scoping review and recommendations. *BMC Med Res Methodol* 21,
  15. [18]Wang, H., Lengerich, B. J., Aragam, B., and Xing, E. P. (2018). Precision lasso: accounting for correlations and linear dependencies in high-dimensional genomic data. *Bioinformatics* 35, 1181–1187.DOI
  16. [20]Zeng, P., Shao, Z., and Zhou, X. (2021). Statistical methods for mediation analysis in the era of high-throughput genomics: Current successes and future challenges. *Computational and Structural Biotechnology Journal* 19, 3209–3224.DOI
  17. [21]Zhao, Y., Lindquist, M. A., and Caffo, B. S. (2020). Sparse principal component based high-dimensional mediation analysis. *Computational Statistics and Data Analysis* 142, 106835.DOI

Paper details

vlnc:W_bww4ufds
Version
v1 · latest
Published
Fresh — added to the Hub within the last 30 days
Added
License
CC BY 4.0
Usage

views · 10

downloads · 0

A view counts once per browser per day, a download each time the PDF is requested.

  • Mathematics

Not recorded for this paper

View statement
Looking for related papers…
Contents