betaFeedback

WorksMathematics

A Kernel-Based Test for Multi-Way Conditional Independence

Abstract

Conditional independence involving more than two variables, or multi-way conditional independence, is a fundamental probabilistic structure useful in many areas of statistical research, including latent-variable and graphical models. However, most existing works focus on pairwise conditional independence. To address this gap, we propose a kernel-based test of multi-way conditional independence. Specifically, we characterize this hypothesis through the simultaneous vanishing of a vector of conditional cross-covariance operators and construct a test statistic based on the norm of the vector of their estimators. We derive the statistic's asymptotic distributions under the null hypothesis and local alternatives. As a related methodological contribution, we also develop a test of multi-way unconditional independence and provide a procedure for its implementation. An application to RNA data illustrates the method's use in genomics.

Introduction

Conditional independence is a fundamental concept in statistics and provides the structural basis for a wide range of methods, including graphical models [23], causal inference [38], sufficient dimension reduction [11, 25], and latent-variable models [5, 1]. However, most of the existing works on this problem are concerned with conditional independence between two variables given a third, and less attention has been given to conditional independence among several variables.

In latent-variable models, in particular multivariate settings are common. The dependence among several observed variables is assumed to be fully explained by a latent state ZZ. For observed variables X1,…,XdX_1,\ldots,X_d, this assumption is expressed as

PX1,…,Xd∣Z=⨂j=1dPXj∣ZPZ-almost surely.P_{X_1,\ldots,X_d\mid Z}=\bigotimes_{j=1}^{d}P_{X_j\mid Z}\quad P_Z\text{-almost surely}.

We refer to this property as multi-way conditional independence and write X1⊥⋯⊥Xd∣ZX_1 \perp\cdots\perp X_d \mid Z. This factorization is imposed as a key structural assumption in many latent-variable models. Under additional model-specific conditions, this factorization enables identification of the proportions and component distributions ([17, 1]) and supports their nonparametric or moment-based estimation ([17, 6, 3]). It is therefore essential to test whether this condition actually holds. Residual dependence after conditioning on ZZ may indicate omitted latent structure, direct interactions, or model misspecification, thus undermining subsequent inference.

The question of whether a shared observed/latent state accounts for joint dependence is particularly relevant in genomics and neuroscience. Gene-expression latent-variable models represent dependence among genes through shared latent factors and observed covariates ([8, 31]). Neural population models similarly use shared latent states or common inputs to explain correlated activity ([32] (Yu et al. 2009)), while cortical recordings reveal higher-order correlations that persist after accounting for stimulus-dependent activity ([21]). These examples motivate testing whether a given state ZZ fully explains joint dependence through the conditional factorization above.

Although model-specific diagnostics have been developed for certain parametric latent-class models ([16, 36, 30]), a general nonparametric test for multi-way conditional independence has yet to be developed. In contrast, nonparametric testing of bivariate conditional independence has been extensively studied; see Li and Fan (2020) for a comprehensive review. Representative procedures are based on nonparametric Hellinger distances (Su and White, 2008), maximal nonlinear conditional correlation ([19]), more general metric characterizations ([18]), permutation-based reductions to two-sample test ([12]), regression residuals under structural restrictions ([52, 43]), and distribution-free transformations ([7]).

Among these nonparametric bivariate conditional independence tests, the operator-based approach in reproducing kernel Hilbert spaces provides the foundation for our method. Under suitable conditions on the kernels, bivariate conditional independence can be characterized by the vanishing of the conditional covariance operator ([13, 14]), which forms the basis of the kernel conditional independence test proposed by Zhang et al. [53]. Subsequent work has developed related operator-based methods for nonparametric graphical modeling and high-dimensional conditioning ([24] (Li and Kim 2024)), as well as extensions to functional data ([25, 47]). Alternative kernel formulations include kernel partial correlation and distance-based characterizations of conditional dependence ([20, 44]).

Building on this operator characterization, we extend the bivariate framework to multi-way conditional independence. The key observation is simple. For any fixed ordering of the variables, multi-way conditional independence can be decomposed into a sequence of pairwise conditional independence conditions:

X1⊥⋯⊥Xd∣Z⟺Xj⊥X{j+1,…,d}∣Z,for all j=1,…,d−1.X_1 \perp\cdots\perp X_d \mid Z \quad\Longleftrightarrow\quad X_j \perp X_{\{j+1,\ldots,d\}} \mid Z,\quad\text{for all } j=1,\ldots,d-1.

Thus, the multivariate null hypothesis can be expressed through d−1d-1 bivariate conditional-independence restrictions. To construct the test statistic, we apply the conditional covariance operator characterization to each restriction and combine the resulting operators into a single omnibus test statistic of multi-way conditional independence. We further develop the asymptotic distribution of our test statistic under the null and local alternative distributions. To the best of our knowledge, this is the first general nonparametric test for multi-way conditional independence.

In particular, the multi-way unconditional independence test can be regarded as a special case by taking ZZ as constant, which also adds up to the existing literature. For example, joint independence based on the dd-variable Hilbert–Schmidt independence criterion ([39]) and distance-based measures ([33, 9]). We stay brief here as this unconditional degeneration is not the primary focus of the paper.

The remainder of the paper is organized as follows. § 2 reviews reproducing kernel Hilbert spaces and cross-covariance operators, which provide a framework for characterizing nonlinear dependence using linear operators. We construct the test statistics in § 3 and study their asymptotic properties in § 4. The unconditional case then follows as a special case in § 5. Finally, § 6 describes the numerical implementation, and § 7 presents simulation studies and applies our method to the OneK1K single-cell RNA-seq dataset.

Background

Background on Reproducing Kernel Hilbert Spaces

Let ΩXj=Rpj\Omega_{X_j}=\mathbb{R}^{p_j} for j=1,…,dj=1,\ldots,d, and ΩZ=RpZ\Omega_Z=\mathbb{R}^{p_Z}.1 Let (Ω,F,P)(\Omega,\mathcal{F},\mathbb{P}) be a probability space, and let Xj:Ω→ΩXjX_j:\Omega\to\Omega_{X_j}, j=1,…,dj=1,\ldots,d, and Z:Ω→ΩZZ:\Omega\to\Omega_Z be random elements. For any subset J⊂{1,…,d}J \subset\{1,\ldots,d\}, let XJ=(Xj)j∈JX_J=(X_j)_{j\in J} denote the corresponding subvector. We use the pointwise product to define the kernel on the product space:

κXJ(xJ,xJ′)=∏ℓ∈Jκℓ(xℓ,xℓ′).\kappa_{X_J}(x_J,x'_J)=\prod_{\ell\in J}\kappa_\ell(x_\ell,x'_\ell).

Let HXJ\mathcal{H}_{X_J} denote the reproducing kernel Hilbert space induced by κXJ\kappa_{X_J}.

We further define augmented variables X¨J=(XJ,Z)\ddot{X}_J=(X_J,Z). Here and throughout the paper, a double dot over a variable indicates that the variable has been augmented by adjoining ZZ. The associated kernel is then defined as

κX¨J((xJ,z),(xJ′,z′))=κXJ(xJ,xJ′)κZ(z,z′).\kappa_{\ddot{X}_J}((x_J,z),(x'_J,z'))=\kappa_{X_J}(x_J,x'_J)\kappa_Z(z,z').

Let HX¨J\mathcal{H}_{\ddot{X}_J} denote the reproducing kernel Hilbert space induced by κX¨J\kappa_{\ddot{X}_J}. Its inner product is also given by the product

⟨f1g1,f2g2⟩HX¨J=⟨f1,f2⟩HXJ⟨g1,g2⟩HZ.\langle f_1g_1,f_2g_2\rangle_{\mathcal{H}_{\ddot{X}_J}}=\langle f_1,f_2\rangle_{\mathcal{H}_{X_J}}\langle g_1,g_2\rangle_{\mathcal{H}_Z}.

where f1,f2∈HXJf_1,f_2\in\mathcal{H}_{X_J}, g1,g2∈HZg_1,g_2\in\mathcal{H}_Z, and (fg)(xJ,z)=f(xJ)g(z)(fg)(x_J,z)=f(x_J)g(z). This definition extends by bilinearity to finite linear combinations and then by continuity to the whole space.

Assumption 1. The kernels κXJ\kappa_{X_J} and κZ\kappa_Z are continuous, bounded, and positive-definite. The conditions hold for all J⊂{1,…,d}J\subset\{1,\ldots,d\}.

By the boundedness of κXJ\kappa_{X_J}, the expectation functional f↦E[f(XJ)]f\mapsto\mathbb{E}[f(X_J)] is a bounded linear functional on HXJ\mathcal{H}_{X_J}. Applying Riesz representation theorem, there exists a unique μXJ∈HXJ\mu_{X_J}\in\mathcal{H}_{X_J} such that

⟨f,μXJ⟩HXJ=E[f(XJ)].\langle f,\mu_{X_J}\rangle_{\mathcal{H}_{X_J}}=\mathbb{E}[f(X_J)].

We define μXJ\mu_{X_J} to be the kernel mean embedding of XJX_J. Similarly, we define the kernel mean embeddings μZ∈HZ\mu_Z\in\mathcal{H}_Z for ZZ and μX¨J∈HX¨J\mu_{\ddot{X}_J}\in\mathcal{H}_{\ddot{X}_J} for X¨J\ddot{X}_J, respectively. Then the corresponding empirical kernel mean embeddings are denoted by μ^XJ=En[κXJ(⋅,XJ)]\hat{\mu}_{X_J}=\mathbb{E}_n[\kappa_{X_J}(\cdot,X_J)], μ^Z=En[κZ(⋅,Z)]\hat{\mu}_Z=\mathbb{E}_n[\kappa_Z(\cdot,Z)], and μ^X¨J=En[κX¨J(⋅,X¨J)]\hat{\mu}_{\ddot{X}_J}=\mathbb{E}_n[\kappa_{\ddot{X}_J}(\cdot,\ddot{X}_J)], where En\mathbb{E}_n stands for the sample average based on i.i.d. data {(X1,i,…,Xd,i,Zi);i=1,…,n}\{(X_{1,i},\ldots,X_{d,i},Z_i); i=1,\ldots,n\}.

References

  1. [1]Allman, E. S., Matias, C., and Rhodes, J. A. (2009), “Identifiability of Parameters in Latent Structure Models with Many Observed Variables,” The Annals of Statistics, 37, 3099–3132.DOI
  2. [2]Allocco, D. J., Kohane, I. S., and Butte, A. J. (2004), “Quantifying the Relationship between Co-expression, Co-regulation and Gene Function,” BMC Bioinformatics, 5, 18.DOI
  3. [3]Anandkumar, A., Ge, R., Hsu, D., Kakade, S. M., and Telgarsky, M. (2014), “Tensor Decompositions for Learning Latent Variable Models,” Journal of Machine Learning Research, 15, 2773–2832.
  4. [4]Argelaguet, R., Arnol, D., Bredikhin, D., et al. (2020), “MOFA+: A Statistical Framework for Comprehensive Integration of Multi-Modal Single-Cell Data,” Genome Biology, 21, 111.DOI
  5. [5]Bartholomew, D. J., Knott, M., and Moustaki, I. (2011), Latent Variable Models and Factor Analysis: A Unified Approach, Wiley Series in Probability and Statistics, Chichester: John Wiley & Sons, 3rd ed.
  6. [6]Bonhomme, S., Jochmans, K., and Robin, J.-M. (2016), “Non-Parametric Estimation of Finite Mixtures from Repeated Measurements,” Journal of the Royal Statistical Society: Series B (Statistical Methodology), 78, 211–229.DOI
  7. [7]Cai, Z., Li, R., and Zhang, Y. (2022), “A Distribution Free Conditional Independence Test with Applications to Causal Discovery,” Journal of Machine Learning Research, 23, 1–41.arxiv.org/abs/2105.07555
  8. [8]Carvalho, C. M., Chang, J., Lucas, J. E., Nevins, J. R., Wang, Q., and West, M. (2008), “High-Dimensional Sparse Factor Modeling: Applications in Gene Expression Genomics,” Journal of the American Statistical Association, 103, 1438–1456.pubmed.ncbi.nlm.nih.gov/21218139
  9. [9]Chakraborty, S. and Zhang, X. (2019), “Distance Metrics for Measuring Joint Dependence with Application to Causal Inference,” Journal of the American Statistical Association, 114, 1638–1650.arxiv.org/abs/1711.09179
  10. [10]Conway, J. (2019), A Course in Functional Analysis, Graduate Texts in Mathematics, Springer New York.
  11. [11]Cook, R. D. (1998), Regression Graphics: Ideas for Studying Regressions through Graphics, New York: John Wiley & Sons.
  12. [12]Doran, G., Muandet, K., Zhang, K., and Schölkopf, B. (2014), “A Permutation-Based Kernel Conditional Independence Test,” in Proceedings of the 30th Conference on Uncertainty in Artificial Intelligence (UAI2014), Oregon: AUAI Press Corvallis, pp. 132–141.
  13. [13]Fukumizu, K., Bach, F. R., and Jordan, M. I. (2004), “Dimensionality reduction for supervised learning with reproducing kernel Hilbert spaces,” Journal of Machine Learning Research, 5, 73–99.DOI
  14. [14]Fukumizu, K., Gretton, A., Sun, X., and Schölkopf, B. (2008), “Kernel measures of conditional dependence,” Advances in neural information processing systems, 489–496.
  15. [15]Gretton, A., Bousquet, O., Smola, A., and Schölkopf, B. (2005), “Measuring Statistical Dependence with Hilbert-Schmidt Norms,” in Algorithmic Learning Theory, eds. Jain, S., Simon, H. U., and Tomita, E., Berlin, Heidelberg: Springer Berlin Heidelberg, pp. 63–77.
  16. [16]Hagenaars, J. A. (1988), “Latent Structure Models with Direct Effects between Indicators,” Sociological Methods & Research, 16, 379–405.DOI
  17. [17]Hall, P. and Zhou, X.-H. (2003), “Nonparametric Estimation of Component Distributions in a Multivariate Mixture,” The Annals of Statistics, 31, 201–224.DOI
  18. [18]Huang, M., Sun, Y., and White, H. (2016), “A Flexible Nonparametric Test for Conditional Independence,” Econometric Theory, 32, 1434–1482.
  19. [19]Huang, T.-M. (2010), “Testing conditional independence using maximal nonlinear conditional correlation,” The Annals of Statistics, 38, 2047 – 2091.DOI
  20. [20]Huang, Z., Deb, N., and Sen, B. (2022), “Kernel Partial Correlation Coefficient — a Measure of Conditional Dependence,” Journal of Machine Learning Research, 23, 1–58.
  21. [21]Köster, U., Sohl-Dickstein, J., Gray, C. M., and Olshausen, B. A. (2014), “Modeling Higher-Order Correlations within Cortical Microcolumns,” PLoS Computational Biology, 10, e1003684.DOI
  22. [22]Kotliar, D., Veres, A., Nagy, M. A., Tabrizi, S., Hodis, E., Melton, D. A., and Sabeti, P. C. (2019), “Identifying gene expression programs of cell-type identity and cellular activity with single-cell RNA-Seq,” eLife, 8, e43803.DOI
  23. [23]Lauritzen, S. (1996), Graphical models, Oxford: Clarendon Press.
  24. [24]Lee, K.-Y., Li, B., and Zhao, H. (2016a), “On an additive partial correlation operator and nonparametric estimation of graphical models,” Biometrika, 103, 513–530. — (2016b), “Variable selection via additive conditional independence,” Journal of the Royal Statistical Society Series B: Statistical Methodology, 78, 1037–1055.
  25. [25]Li, B. (2018a), “Linear operator-based statistical analysis: A useful paradigm for big data,” Canadian Journal of Statistics, 46, 79–103.DOI
  26. [30]Li Donni, P. and Thomas, R. (2020), “Latent Class Models for Multiple Ordered Categorical Health Data: Testing Violation of the Local Independence Assumption,” Empirical Economics, 59, 1903–1931.
  27. [31]Lopez, R., Regier, J., Cole, M. B., Jordan, M. I., and Yosef, N. (2018), “Deep Generative Modeling for Single-Cell Transcriptomics,” Nature Methods, 15, 1053–1058.DOI
  28. [32]Macke, J. H., Opper, M., and Bethge, M. (2011), “Common Input Explains Higher-Order Correlations and Entropy in a Simple Model of Neural Population Activity,” Physical Review Letters, 106, 208102.
  29. [33]Matteson, D. S. and Tsay, R. S. (2017), “Independent Component Analysis via Distance Covariance,” Journal of the American Statistical Association, 112, 623–637.arxiv.org/abs/1306.4911
  30. [34]Morgan, M. J. and Liu, Z.-g. (2011), “Crosstalk of Reactive Oxygen Species and NF-κB Signaling,” Cell Research, 21, 103–115.DOI
  31. [35]Mussbacher, M., Derler, M., Basilio, J., and Schmid, J. A. (2023), “NF-κB in Monocytes and Macrophages: An Inflammatory Master Regulator in Multitalented Immune Cells,” Frontiers in Immunology, 14, 1134661.DOI
  32. [36]Oberski, D. L., van Kollenburg, G. H., and Vermunt, J. K. (2013), “A Monte Carlo Evaluation of Three Methods to Detect Local Dependence in Binary Data Latent Class Models,” Advances in Data Analysis and Classification, 7, 267–279.DOI
  33. [37]Oelen, R., de Vries, D. H., Brugge, H., et al. (2022), “Single-Cell RNA-Sequencing of Peripheral Blood Mononuclear Cells Reveals Widespread, Context-Specific Gene Expression Regulation upon Pathogenic Exposure,” Nature Communications, 13, 3267.DOI
  34. [38]Pearl, J. (2009), Causality: Models, Reasoning, and Inference, Cambridge: Cambridge University Press, 2nd ed.
  35. [39]Pfister, N., Bühlmann, P., Schölkopf, B., and Peters, J. (2018), “Kernel-Based Tests for Joint Independence,” Journal of the Royal Statistical Society: Series B (Statistical Methodology), 80, 5–31.arxiv.org/abs/1603.00285
  36. [40]Piasecka, B., Duffy, D., Urrutia, A., Quach, H., Patin, E., Posseme, C., Bergstedt, J., Charbit, B., Rouilly, V., MacPherson, C. R., et al. (2018), “Distinctive roles of age, sex, and genetics in shaping transcriptional variation of human immune responses to microbial challenges,” Proceedings of the National Academy of Sciences, 115, E488–E497.DOI
  37. [41]Ribeiro, D. M., Rubinacci, S., Ramisch, A., Hofmeister, R. J., Dermitzakis, E. T., and Delaneau, O. (2021), “The Molecular Basis, Genetic Control and Pleiotropic Effects of Local Gene Co-expression,” Nature Communications, 12, 4842.DOI
  38. [42]Sang, P. and Li, B. (2026), “Nonlinear function-on-function regression by RKHS,” Journal of Machine Learning Research. To appear.
  39. [43]Shah, R. D. and Peters, J. (2020), “The Hardness of Conditional Independence Testing and the Generalised Covariance Measure,” The Annals of Statistics, 48, 1514–1538.
  40. [44]Sheng, T. and Sriperumbudur, B. K. (2023), “On Distance and Kernel Measures of Conditional Dependence,” Journal of Machine Learning Research, 24, 1–16.
  41. [47]Tang, Y. and Li, B. (2026), “A Reproducing-Kernel-Based Nonparametric Test for Conditional Independence of Functional Data,”.DOI
  42. [48]Wong, K. L., Tai, J. J.-Y., Wong, W.-C., Han, H., Sem, X., Yeap, W.-H., Kourilsky, P., and Wong, S.-C. (2011), “Gene expression profiling reveals the defining features of the classical, intermediate, and nonclassical human monocyte subsets,” Blood, 118, e16–e31.DOI
  43. [49]Xue, A., Yazar, S., Neavin, D., and Powell, J. E. (2023), “Pitfalls and Opportunities for Applying Latent Variables in Single-Cell eQTL Analyses,” Genome Biology, 24, 33.
  44. [50]Yazar, S., Alquicira-Hernandez, J., Wing, K., Senabouth, A., Gordon, G., Andersen, S., Lu, Q., Rowson, A., Taylor, T. R., Clarke, L., et al. (2022), “Single-cell eQTL mapping identifies cell type-specific genetic control of autoimmune disease,” Science, 376, eabf3041.DOI
  45. [52]Zhang, H., Zhou, S., Zhang, K., and Guan, J. (2017), “Causal Discovery Using Regression-Based Conditional Independence Tests,” Proceedings of the AAAI Conference on Artificial Intelligence, 31.DOI
  46. [53]Zhang, K., Peters, J., Janzing, D., and Schölkopf, B. (2012), “Kernel-based conditional independence test and application in causal discovery,” arXiv preprint arXiv:1202.3775.arxiv.org/abs/1202.3775

Paper details

vlnc:W_47rvqshu
Version
v1 · latest
Published
Fresh — added to the Hub within the last 30 days
Added
License
CC BY 4.0
Usage

views · 0

downloads · 0

A view counts once per browser per day, a download each time the PDF is requested.

  • Mathematics

Not recorded for this paper

View statement
Looking for related papers…
Contents