betaFeedback

WorksComputer Science

When Covariance Is Not Enough: Kernel-Based Regularization for Single-Cell Counterfactuals

Abstract

Counterfactual editing of single cells, predicting how an observed cell would look had one factor been different, enables in-silico screening where experiments are infeasible, but is useful only if the edit is faithful to the individual and not merely to the condition. Concept bottleneck generative models make the effects of interventions interpretable by decomposing a cell's representation into known and unknown factors. Previous work uses a linear cross-covariance penalty to disentangle known and unknown dimensions. We demonstrate that this first-order criterion does not fully separate information when non-linear interactions are present. We first demonstrate the inadequacy of linear cross-factor penalization with simulation experiments with controlled edits and known non-linear covariate effects. We then demonstrate that adding a nonlinear kernel criterion, the normalized Hilbert-Schmidt Independence Criterion, is able to recover the true intervention effects on simulation data. Finally, we apply the non-linear concept bottleneck model to a complex single-cell RNA-seq dataset and find that it improves the ability to generate counterfactual expression profiles of cells under different tissue environments.

Introduction

Counterfactual modeling asks what an observation would have looked like had one of its underlying factors been different when other factors are held fixed. The dominant framework is Pearl’s abduction–action–prediction [1]: infer the latent state that explains the observation, intervene on a single factor, predict the outcome. The edit is useful only if it is at once effective, it produces the requested change, and faithful, it alters nothing else. These models are particularly valuable to observational samples in biology, where high-dimensional measurements are assumed to lie in a lower dimensional space, exhaustive sampling of potential conditions is expensive and infeasible, and a sample can usually only be measured once with a single modality. Common biological counterfactual questions include what a cell’s gene expression would be in a different environment, e.g. different tissue, or under a different context, e.g. alternative drug treatment. Single-cell RNA sequencing [2], measuring the gene expression of individual cells [3] from a sample, provides increasing resolution into disease biology at ever increasing cell numbers and resolution [4, 5]. Improved throughput and decreasing costs have enabled the production of reference atlases across tissues and diseases [6, 7, 8, 9, 10], and enabled training of latent-variable models of the cell types and states [11]. However, while modern foundation models pretrained on tens of millions of cells [12, 13] can describe cellular states, they have fallen short on valuable counterfactual prediction. Improved counterfactual models of single-cell RNA-seq data would enable the exploration of the combinatorial space of cell types, patients, treatments and doses that is infeasible at current experiment scales.

Existing models of cellular response follow two overlapping directions. The first predicts intervention effects at the level of populations rather than of a given cell, whether by assuming a fixed additive shift in latent space [14, 15], conditioning a decoder on the perturbation [16], propagating it through a prior knowledge graph [17], or mapping unpaired control and treated populations onto one another by optimal transport [18, 19]; the transport maps do act per cell, but they match distributions rather than reconstruct a named factor. The second uses interpretable representations with biologically named components [20]. Several methods combine these directions through latent-space arithmetic over disentangled factors [21, 22], latent optimization with capacity-constrained unknown-attribute embeddings [23], or causally structured concept embeddings decoded by a diffusion model [24]. All emit a profile per input cell, but their evaluations do not explicitly test whether an edit preserves donor-specific identity or matches the source donor’s response under the target condition.

The concept bottleneck generative model scCBGM combines both approaches [25] by implementing the concept bottleneck idea [26] for a generative model of cells. The model decomposes each cell’s latent representation into a supervised block of observed concepts, such as cell type, tissue or treatment, and a set of unknown factors that account for other unrecorded variation, from donor identity to library size and batch. Counterfactual predictions for the known factors can be generated via abduction–action–prediction: encode a cell, replace one concept with its target value, and decode from the same unknown factors. Because the concepts are explicit, the edit can be checked rather than trusted, which makes it a plausible route to personalized prediction. This counterfactual, however, is only valid if the two blocks are genuinely independent, such that editing a concept does not alter another concept’s effect, e.g. a donor-specific signal that makes the prediction personal. To ensure disentanglement of the latent factors and concepts, scCBGM enforces independence with a penalty on the linear cross-covariance between the blocks, a first-order criterion blind to any dependence whose conditional means agree. Kernel criteria are the known remedy in this setting: Lopez et al. constrained a single-cell VAE with the dd-variable Hilbert-Schmidt Independence Criterion to keep the latent independent of observed nuisance factors [27]. Here both arguments are learned, the concept head on one side and the unknown block on the other, and the question is not invariance to a nuisance but whether an edit made through the concept pathway stays faithful to the individual carried in the unknowns.

We show that the linear penalty is not always enough. On a matched pair of simulations differing in exactly one place, the cross-covariance penalty recovers donor-specific counterfactuals when the edited condition enters as an additive mean shift shared across donors, and fails at every penalty strength when it instead modulates the variance of a donor-specific signature, a regime in which the dependence between the blocks is entirely second-order and the penalty reports no violation. Adding a kernel criterion, the normalized Hilbert-Schmidt Independence Criterion [28], recovers the donor-specific edit, and the same ordering holds on the TAURUS inflammatory bowel disease cohort [29], where the added kernel penalty produces an accurate rectum → ileum edit.

Results

A single-cell simulator with known donor ground truth

We ask whether a concept bottleneck generative model edits a cell in a donor-specific way. The original single cell concept bottleneck paper established that flipping such a concept can move cells into the target condition [25]. A donor-specific edit, however, should capture the expression of the cells that donor has in the target condition rather than a generic population from the target condition. To that end we implemented a simulator in which the donor effect is known by construction, so that the correct donor-specific answer is available as ground truth even for donors the model has never seen.

The simulation uses a hierarchical Poisson model to generate single-cell like data [25] (§ 4.2). Every cell draws a donor, a tissue, a cell type and a set of binary latent concepts from a fixed hierarchy. Its Poisson log-rate is a sum of gene-wise effects: a baseline expression level, a cell-type effect, a donor fingerprint, a tissue effect, the latent concept effects, a per-cell library-size offset and gene noise (Eq. (5)). The two-level tissue axis plays the role of the editable binary concept is_ileum. Counts are then drawn by inverse-CDF transform of stored exogenous uniforms (Eq. (6)), so the sampler is a deterministic function of its latent representation and we can regenerate a dataset with one effect changed and every other draw held fixed. The counts are then pushed through the pre-processing used throughout this work (median library-size normalization, log⁡(1+x)\log(1+x), PCA to 128 components), so all training and scoring happen in a 128-dimensional PCA space. The generator produces 240,000 cells over 150 donors, 4 cell types and 3,000 genes (Table 4). Figure 1 illustrates the simulated dataset with four cell types (A), the donor structure (B), and flipped tissue concept identity (C).

Structure of the synthetic donor-holdout dataset

Figure 1. Structure of the synthetic donor-holdout dataset (240,000 cells, 150 donors, 20 of them held out). All panels show the same UMAP, fitted once on a uniform random subsample of 40,000 cells in the 128-dimensional PCA space the model is trained in, and recoloured: (A) the 4 simulated cell types; (B) donor identity, with 8 donors highlighted (donor_000, donor_001, donor_002, donor_003, donor_014, donor_015, donor_020, donor_021) and the remaining donors in gray; (C) the binary tissue concept is_ileum, which is what the counterfactual flips; (D) training donors versus the 20 held-out donors.

SymbolMeaningValue
NNcells240,000
FFgenes3,000
BBdonors (20 held out)150
TTtissues (is_ileum)2
UUcell types4
CClatent concepts3
[ℓ,h][\ell,h]baseline-expression range[1,6][1,6]
σB\sigma_Bdonor s.d.0.30
σU\sigma_Ucell-type s.d.0.20
σT\sigma_Ttissue s.d.0.35
σC\sigma_Cconcept s.d.0.05
σε\sigma_\varepsilongene-noise s.d.0.02
σL\sigma^Llibrary-size s.d.Unif⁡(0.01,0.03)\operatorname{Unif}(0.01,0.03)
(a,b)(a,b)zero-inflation Beta parameters(1,0.5)(1,0.5)
RRdonor factor rank12
∣V∣\lvert V\rvertvariance-block genes150
ρ\rhovariance-channel base scale0.5
MMileum variance multiplier3
–PCA components128
–seed0

Table 4. Generator settings for the matched linear / heteroskedastic pair.

Donor effects must be low rank for held-out donors to be predictable

In the original single cell concept bottleneck simulator, the donor fingerprint γb,g\gamma_{b,g}, the effect of donor bb on gene gg, is drawn i.i.d. per gene, which makes each donor an isolated point in gene space: nothing learned from the training donors carries information about an unseen donor’s vector γb\gamma_b, and the donor holdout is unsolvable in principle rather than merely hard. To properly model donor-specific effects, we replace the donor block by a shared factor model γ=HW\gamma= HW with H∈RB×RH \in\mathbb{R}^{B \times R} and W∈RR×FW \in\mathbb{R}^{R \times F} (Eq. (7)), where B=150B = 150 is the number of donors, F=3,000F = 3{,}000 the number of genes, and R=12R = 12 the rank: HH gives each donor RR loadings and WW holds RR gene programs shared across donors, the analogue of a small number of transcriptional programs that individuals up- or downregulate to different degrees rather than FF independent per-gene effects. The program matrix WW is drawn once and shared, and its variance is scaled by 1/R1/R so that the per-gene donor variance Var⁡(γb,g)=σB2\operatorname{Var}(\gamma_{b,g}) = \sigma_B^2 is unchanged, where σB\sigma_B is the donor-effect standard deviation: only the rank of the donor signal differs, not its strength. Every donor, held out or not, then lies in the same RR-dimensional subspace, so a donor readout learned on training donors can transfer to unseen ones. All results below use this low-rank generator (Table 4).

Linear regime: a covariance penalty is enough

We simulate data with fixed donor, tissue (non-ileum and ileum), and cell type of various library sizes and noisy gene expression observations. When training the model, the cell type and tissue are observed for each cell in a supervised concept vector c^\hat{c}, and the unknown-factor block uu, should account for the donor, library size and the gene noise. We test how well the model performs when generating a counterfactual by switching only the tissue concept to ileum (§ 5.1). Donor identity is an unknown concept, so for the model to make a successful donor-specific edit the donor identity must survive in uu and the decoder must recombine it with the flipped concept. What keeps c^\hat{c} and uu from entangling is an orthogonality penalty λ⊥L⊥(c^,u)\lambda_{\perp}\mathcal{L}_{\perp}(\hat{c},u), where L⊥\mathcal{L}_{\perp} penalizes dependence between the two blocks and λ⊥\lambda_{\perp} sets its strength; the original model uses a linear cross-covariance penalty, and we start there.

We evaluate the model on held-out donors: 20 of the 150 donors are withheld from training entirely (Figure 1D), and at inference each unseen donor’s non-ileum cells are edited to ileum and compared against that donor’s observed ileal cells of the same cell type. Absolute distances are not comparable across cell types, so we score each held-out (donor bb, cell type kk) pair by a ratio of squared Maximum Mean Discrepancies (MMD),

rMMD⁡b,k=MMD⁡(X^b,k,Xb,ktgt)MMD⁡(X⋅,ktgt,Xb,ktgt)(1)\operatorname{rMMD}_{b,k} = \frac{\operatorname{MMD}(\hat{X}_{b,k}, X_{b,k}^{\mathrm{tgt}})}{\operatorname{MMD}(X_{\cdot,k}^{\mathrm{tgt}}, X_{b,k}^{\mathrm{tgt}})} \tag*{(1)}

where X^b,k\hat{X}_{b,k} is the edit of donor bb’s type-kk source cells, Xb,ktgtX_{b,k}^{\mathrm{tgt}} that donor’s own real target cells of type kk, and X⋅,ktgtX_{\cdot,k}^{\mathrm{tgt}} the generic target-condition population of type kk pooled over all training donors (§ 6). The denominator is thus the best "get the condition right but ignore the donor" answer: a value near or above 1 means a generic target-condition cell, and a value below 1 means the edit is closer to that donor’s target than the pooled population is, which is exactly the donor-specific property. We report the median over all held-out (donor, cell type) rows and, where stated, the fraction below 1.

We first evaluate a simulation where the donor-specific effect is an additive mean shift shared by all donors, so the donor-specific part of the answer is entirely the donor’s own baseline, which the model must carry in uu. Here, the original concept bottleneck model with a linear orthogonality penalty generalizes to unseen donors (Table 1, top block): the cross-covariance penalty gives a median held-out-donor rMMD of 0.09, 0.06 and 0.06 at λ⊥=0.5\lambda_{\perp} = 0.5, 1 and 5, all well below 1. Row A of Figure 2 shows this per donor: for four donors never seen in training, the edited cells (red) land on the withheld real ileal cells (green) for all four cell types, and the arrows track the real source-to-target displacement. This is the regime in which the method behaves as advertised.

Donor-holdout counterfactual on the matched linear / heteroskedastic simulation pair

Figure 2. Donor-holdout counterfactual on the matched linear / heteroskedastic simulation pair. Rows: (A) linear regime with a covariance penalty, (B) heteroskedastic regime with the same covariance penalty, (C) heteroskedastic regime with covariance + nHSIC; all at λ⊥=0.5\lambda_{\perp}=0.5. Columns are held-out donors (donor_014, donor_015, donor_020, donor_024), never seen in training. Each panel shows all 4 simulated cell types of that donor: the retained is_ileum=non_ileum cells (blue), the withheld real is_ileum=ileum cells (green), and the model’s counterfactual edit of the blue cells (red). Rows B and C share one UMAP, fitted jointly on their real cells and both predictions, so they are directly comparable and only the red cloud moves; row A is a different dataset and therefore a different coordinate system. Each group is subsampled to at most 300 cells per donor and cell type, so the three groups are drawn at comparable density. The three groups are drawn in a single shuffled scatter, so neither is systematically painted over the other and a true overlap reads as a blend. Arrows run from each cell type’s real source cloud to the model’s counterfactual for that cloud (centroids), i.e. the edit the model actually made; the edit succeeds when the arrow head lands on the green held-out target. They are a reading guide in UMAP coordinates, not a displacement in any metric space. The gray backdrop is the real cells of all 20 held-out donors (20,000 of 32,769 drawn), so each gray blob is another donor. Panels are framed, and tinted by regime – rows B and C share both an embedding and a tint, row A neither. Panel labels give that donor’s median rMMD over the cell types scored for it (a cell type is scored only where the donor has enough cells on both sides); run-level medians over all 20 held-out donors: (A) 0.09, (B) 1.30, (C) 0.15 (lower is better; < 1 beats mapping the cells to the closest existing population).

RegimeOrth. lossλ⊥=0.5\lambda_{\perp}=0.5λ⊥=1\lambda_{\perp}=1λ⊥=5\lambda_{\perp}=5
Linear (mean shift)cov0.090.060.06
Linear (mean shift)nHSIC0.100.080.06
Linear (mean shift)cov + nHSIC0.030.242.02
Nonlinear (heteroskedastic)cov1.301.301.09
Nonlinear (heteroskedastic)nHSIC1.090.690.35
Nonlinear (heteroskedastic)cov + nHSIC0.150.212.16

Table 1. Median per-donor rMMDr\mathrm{MMD} on held-out donors for the matched linear/heteroskedastic simulation pair. Bold = beats the baseline (< 1). A covariance penalty suffices in the linear regime but fails at every strength in the heteroskedastic one, where the condition enters through a conditional variance; adding nHSIC recovers a donor-specific edit.

Heteroskedastic regime: a covariance penalty fails, nHSIC recovers the edit

A general linear additive shift is an unrealistic assumption for true single-cell RNA-seq observations. We therefore build a matched member of the pair, generated from the same seed and differing only on a block VV of the 150 highest-expression genes. On VV the tissue main effect is removed and replaced by a donor–tissue variance interaction (Eq. (8)): ϕb\phi_b is a per-donor signature on VV (drawn once, zero-mean across donors), m(tn)m(t_n) is a tissue multiplier equal to 1 in non-ileum and MM in ileum, ρ=0.5\rho= 0.5 scales the channel and M=3M = 3 is the ileum variance multiplier. Each donor thus carries its own signature ϕb\phi_b whose amplitude grows by MM in the ileum. Because E[ϕ]=0\mathbb{E}[\phi] = 0, the condition and the block are uncorrelated, Cov⁡(is_ileum,V)=0\operatorname{Cov}(\mathrm{is\_ileum},V) = 0, while the conditional variance ratio between the two tissues is M2M^2. The correct edit is to amplify each donor’s own

baseline signature, and the information needed is present in the donor’s source cells, so the task stays well posed.

In this regime the covariance penalty fails at every strength (Table 1, bottom block): median rMMDr\mathrm{MMD} 1.30, 1.30 and 1.09, with only 35%, 37% and 46% of the individual (donor, cell type) rows below 1. Row B of Figure 2 shows the failure mode. The edit is not absent, the red cloud does move away from blue, but it moves toward a generic ileal position and does not reach the donor’s own green cells.

By contrast, adding a kernel penalty [28] recovers the edit. The normalized Hilbert–Schmidt Independence Criterion (nHSIC) measures the dependence between c^\hat{c} and uu through kernels rather than covariance, so it registers the higher-order dependence that a covariance penalty is blind to (§ 5). With nHSIC alone the median rMMDr\mathrm{MMD} falls from 1.09 to 0.69 to 0.35 as λ⊥\lambda_{\perp} goes from 0.5 to 1 to 5; combining the two penalties is better still, 0.15 at λ⊥=0.5\lambda_{\perp}=0.5 and 0.21 at λ⊥=1\lambda_{\perp}=1, with 95% of the individual donor rows below 1 in both cases. Row C of Figure 2 is the same four held-out donors in the same embedding as row B, and the only thing that changes is that the red cloud now lands on green. Since rows B and C share a UMAP, the comparison is direct.

We find that success is dependent on the strength on the penalty. At λ⊥=5\lambda_{\perp}=5 the combined penalty breaks down in both regimes, rMMDr\mathrm{MMD} 2.16 in the heteroskedastic member and 2.02 in the linear one. This is not an inert decoder. The edit ratio measures how far the decoder’s output moves when the concept is flipped from source to target, relative to the source reconstruction (§ 5.1); within these three runs it grows monotonically with λ⊥\lambda_{\perp} (0.05, 0.08, 0.90). The model is making a large edit that leaves the target population, i.e. over-regularizing the bottleneck drives the prediction off the data manifold rather than suppressing the edit. The useful range is therefore bounded on both sides, and the two failure modes look nothing alike.

Real data: held-out subtype counterfactual in the Taurus cohort

Finally, we evaluate linear and non-linear penalties in concept bottleneck models on the TAURUS cohort [29]. We use the pooled subtype holdout used by the original authors: all cells of one fine subtype in the target condition are withheld from training, the retained cells of that subtype in the source condition are edited, and the prediction is compared to the withheld cells. The subtype was chosen by a model-free screen of all 36 location-agnostic subtypes, ranking them by the real source-versus-target distance and by the number of donors that carry the subtype on both sides (§ 6), since a subtype whose two conditions barely differ makes the ratio metric uninformative. We selected stem_LGR5pos (per-donor real gap MMD 0.20, 14 donors on both sides). The holdout removes all 4,100 ileal stem_LGR5pos cells of 987,743; at inference the 6,342 retained rectal cells of that subtype are edited non_ileum → ileum.

We compare three runs that differ only in the orthogonality penalty and its weight (Table 2). The covariance penalty at λ⊥=0.5\lambda_{\perp} = 0.5 does not produce an edit. The decoder barely responds to the flip (edit ratio 0.04), and the prediction stays essentially where the source cells were: its MMD to the withheld ileal cells is 0.231, against 0.244 for the un-edited epithelial non-ileum population, which is the closest of the 14 candidates and hence the denominator of the ratio. Read alone, the ratios (rMMD 0.95, rSinkhorn 0.50) are both below 1 and would suggest success; they are below 1 only because the source population is itself the best available answer. Raising the same penalty to λ⊥=10\lambda_{\perp} = 10 does not fix it (numerator 0.234, rMMD 0.96, rSinkhorn 0.46), although the decoder does now respond to the concept (edit ratio 0.22).

Orthogonality penaltyλ⊥\lambda_{\perp}rMMD ↓\downarrowrSinkhorn ↓\downarrow
Covariance0.50.950.50
Covariance100.960.46
Covariance + nHSIC100.410.14

Table 2. Held-out counterfactual accuracy on the taurus stem holdout (Rectum →\rightarrow terminal ileum, stem_LGR5pos). rMMD is the ratio metric of Eq. (19): <1< 1 means the edited cells are closer to the real held-out ileal cells than the closest existing cell population is. All three runs are below 1, but the covariance-only numerators (0.231 and 0.234) are essentially the denominator (0.244), i.e. the prediction is no closer to the target than the un-edited source it was made from; only cov + nHSIC (bold) makes a complete edit (edit ratio 0.67 vs. 0.04 and 0.22).

The combined covariance and nHSIC penalty at λ⊥=10\lambda_{\perp} = 10 gives rMMD 0.41 and rSinkhorn 0.14, a numerator of 0.100 against the same denominator of 0.244, and an edit ratio of 0.67. Figure 3 shows the three runs in a shared embedding and reproduces the ordering seen in the simulation: under cov λ⊥=0.5\lambda_{\perp} = 0.5 the prediction sits on top of the source cells, under cov λ⊥=10\lambda_{\perp} = 10 it is displaced but stops short of the withheld ileal cells, and under cov + nHSIC it overlaps them.

Counterfactual edit of the held-out stem_LGR5pos subtype under three orthogonality penalties

Figure 3. Counterfactual edit of the held-out stem_LGR5pos subtype (rectum →\to terminal ileum) under three orthogonality penalties. Every is_ileum=ileum stem_LGR5pos cell (n=4,100n=4{,}100, green) was withheld from training; at inference the retained rectal cells of the same subtype (n=6,342n=6{,}342, blue) are edited to ileum (red) and compared against them. The gray backdrop is the rest of the epithelial lineage, shaded by fine subtype (largest subtype lightest); a uniform random sample of 30,000 of the 317,976 cells is drawn, to keep the UMAP tractable; sampling is uniform, so subtype proportions are preserved.

We find that the qualitative ranking found in the heteroskedastic simulation, where a linear orthogonality penalty leaves an incomplete edit and a kernel criterion completes it, is consistent in the real data. The success metric of rMMD <1< 1 is necessary but not sufficient: all three runs clear that bar, but only the run with the kernel penalty clears it by a margin that corresponds to an actual edit. We therefore read the ratio together with the numerator, the winning denominator population and the decoder edit ratio.

Discussion

We asked whether a concept bottleneck generative model edits a cell in a donor-specific way, and the answer depends on how the edited condition enters the data. On a matched pair of simulations with linear and non-linear donor specific effects, the linear cross-covariance penalty of scCBGM [25] suffices when the condition is an additive mean shift shared by all donors and fails when it instead modulates the variance of a donor-specific signature, regardless of the strength of the linear orthogonality penalty. The failure is not an absent edit with a high enough penalty: the decoder moves the cells, but to a generic target-condition position rather than that donor’s own. Adding a kernel criterion, nHSIC, to the linear one recovers the donor-specific edit, and the same holds on Taurus [29], where the covariance-only numerators sit at the denominator while cov + nHSIC completes the edit. Neither regime is a strawman. The additive one is plausibly what the interferon-β\beta stimulation benchmark of Kang et al. [30] provides, on which scCBGM was evaluated with the covariance penalty alone: a coordinated up-regulation of an interferon program largely shared across cells and donors, hence a first-order effect that a first-order criterion can keep out of UU. The heteroskedastic regime is what one expects when a perturbation amplifies an individual’s existing program instead of adding a fixed offset, as when inflammation exaggerates a donor’s baseline response. The low-rank donor structure is likewise biologically grounded, individuals differing mainly by up- or downregulating a limited set of shared transcriptional programs rather than through FF independent per-gene effects, and it is what makes an unseen donor predictable at all. We therefore recommend adding the kernel penalty rather than replacing the linear one. The terms are complementary: cross-covariance is a cheap, low-variance estimate of the first-order dependence, whereas nHSIC must estimate a full kernel dependence from a few hundred cells, so keeping both anchors the first order and leaves the kernel term only what covariance cannot see. The combination reaches the best value in both regimes at the smallest weight (0.03 and 0.15 at λ⊥=0.5\lambda_{\perp}=0.5), where nHSIC alone needs λ⊥=5\lambda_{\perp}=5 to reach 0.35.

Hyperparameter sensitivity is the main current weakness of nonlinear concept bottleneck models. We find that over-regularization is a potential issue with the combined kernel and linear penalty. Tuning on rMMD alone would miss this, and we find that joint evaluation with the edit ratio and UMAPs is required. The best setting differed between our two datasets (λ⊥=0.5\lambda_{\perp}=0.5 in simulation, 10 on Taurus). We did not explore the estimator’s batch-size dependence and its median-heuristic bandwidths. Our results are also single-seed, reported without intervals, live in a 128-dimensional PCA space under a Gaussian loss rather than in count space, and rest on one simulator and one cohort with a single held-out subtype.

Growing longitudinal and pan-tissue collection will eventually make further evaluations and development feasible. As data increases, we can move towards patient, rather than cell level, prediction and generation - treating each patient as a distribution over cells and the cohort as a distribution over those distributions, simultaneously generating cell type abundances and cell states. With such a readout the applications follow from the counterfactual framing, in-silico treatment screening in a specific patient, imputing a tissue that was never biopsied, mapping an individual along an axis such as age, and transferring a biomarker from accessible blood to the tissue where the disease sits. Each is an edit that must be faithful to the individual and not merely to the condition, and which independence criterion is used decides whether it is.

Methods

Single cell RNA sequencing preprocessing

Raw single cell counts from the Taurus [29] dataset object TAURUS_raw_counts_annotated_final.h5ad were downloaded from https://zenodo.org/records/14007626 and reduced to the 3,000 most highly-variable genes (sc.highly_variable_genes, seurat_v3 flavor on raw counts, mitochondrial/ribosomal genes excluded), median full-library-size normalized and log1p-transformed, then embedded into 128 principal components (scanpy PCA, zero-centered, fit on all cells) [31]. From the cell-level annotations (obs) we derived the three columns. The fine subtype was made location-agnostic by stripping the leading Ileal_/Non_ileal_ prefix from minor (minor_agnostic), which defines the held-out subtype stem_LGR5pos. The tissue axis was collapsed to a binary concept is_ileum (ileum if Site == Terminal_Ileum, else non_ileum), since the four colon sub-regions are transcriptomically hard to distinguish. The final major cell-type concept was obtained by a deterministic union-merge of the 22-value major column into seven major lineages (Table 3); the Ileal_epithelium and Non_ileal_epithelium majors are merged into a single epithelial lineage so the cell-type concept is not a proxy for is_ileum.

Major cell typeOriginal major values merged
epithelialIleal_epithelium, Non_ileal_epithelium
T_ILCCD4_T, CD8_T, Unconventional_T, Innate_lymphocytes
B lineageB, Plasma
myeloid cellMono_macro, DC, Mast, Cycling_MNP
endothelialEndothelium
glial cellGlial
mesenchymalFibroblast, Epi_fibroblast, LP_fibroblast, Myofibroblast, C3pos_fibroblast, THY1pos_FAPpos_PDPNpos_fibroblast, Pericyte, Cycling_stroma

Table 3. Cell-type concept: mapping of the 22 major values onto seven lineages.

Synthetic data

Hierarchical Poisson generative model. We adapt the hierarchical Poisson single-cell generator of [25] (conceptlab.datagen.OmicsDataGenerator), reused unchanged as the base model and extended by the two terms below. Each cell nn draws a donor bnb_n, a tissue tnt_n, a cell type unu_n and a binary latent concept vector vn∈{0,1}Cv_n \in\{0,1\}^{C},

p(B)∼Dir⁡(10⋅1B),p(T)∼Dir⁡(10⋅1T),pt(U)∼Dir⁡(1U),pu(C)∼Dir⁡(151C),p^{(B)} \sim\operatorname{Dir}(10 \cdot\mathbf{1}_{B}), \quad p^{(T)} \sim\operatorname{Dir}(10 \cdot\mathbf{1}_{T}), \quad p_t^{(U)} \sim\operatorname{Dir}(\mathbf{1}_{U}), \quad p_u^{(C)} \sim\operatorname{Dir}(\frac{1}{5}\mathbf{1}_{C}),
bn∼Cat⁡(p(B)),tn∼Cat⁡(p(T)),un∼Cat⁡(ptn(U)),vn,c∼Bern⁡(σ(ζn,c)),ζn=clip⁡(logit⁡pun(C),−8,8),(2)b_n \sim\operatorname{Cat}(p^{(B)}), \quad t_n \sim\operatorname{Cat}(p^{(T)}), \quad u_n \sim\operatorname{Cat}(p_{t_n}^{(U)}), \quad v_{n,c} \sim\operatorname{Bern}(\sigma(\zeta_{n,c})), \quad\zeta_n = \operatorname{clip}(\operatorname{logit}p_{u_n}^{(C)},-8,8), \tag*{(2)}

with gene-wise additive effects

wg∼Unif⁡(log⁡ℓ,log⁡h),γb,g∼N(0,σB2),ωu,g∼N(0,σU2),τt,g∼N(0,σT2),(3)w_g \sim\operatorname{Unif}(\log\ell,\log h), \quad\gamma_{b,g} \sim\mathcal{N}(0,\sigma_B^2), \quad\omega_{u,g} \sim\mathcal{N}(0,\sigma_U^2), \quad\tau_{t,g} \sim\mathcal{N}(0,\sigma_T^2), \tag*{(3)}
sc,g∼N(0,σC2)⋅1[mc,g=1],mc,g∼Bern⁡(πc),πc=1−Beta⁡(a,b),(4)s_{c,g} \sim\mathcal{N}(0,\sigma_C^2) \cdot\mathbf{1}[m_{c,g}=1], \quad m_{c,g} \sim\operatorname{Bern}(\pi_c), \quad\pi_c = 1-\operatorname{Beta}(a,b), \tag*{(4)}

a per-cell library offset ℓn∼N(0,(σbnL)2)\ell_n \sim\mathcal{N}(0,(\sigma_{b_n}^{L})^2), σbL∼Unif⁡(0.01,0.03)\sigma_b^{L} \sim\operatorname{Unif}(0.01,0.03), and gene noise εn,g∼N(0,σε2)\varepsilon_{n,g} \sim\mathcal{N}(0,\sigma_{\varepsilon}^2). The Poisson log-rate is additive,

log⁡λn,g=wg+∑csc,gvn,c+ωun,g+γbn,g+τtn,g+ℓn+εn,g,(5)\log\lambda_{n,g} = w_g + \sum_c s_{c,g}v_{n,c} + \omega_{u_n,g} + \gamma_{b_n,g} + \tau_{t_n,g} + \ell_n + \varepsilon_{n,g}, \tag*{(5)}

and counts are drawn by inverse-CDF transform of a stored exogenous uniform an,g∼Unif⁡(0,1)a_{n,g} \sim\operatorname{Unif}(0,1),

xn,g=FPois⁡(λn,g)−1(an,g).(6)x_{n,g}=F_{\operatorname{Pois}(\lambda_{n,g})}^{-1}(a_{n,g}). \tag*{(6)}

Keeping an,ga_{n,g} and εn,g\varepsilon_{n,g} makes the sampler a deterministic function of the latents: we call the generator, read the coefficients and exogenous draws back out, and recompute (6)–(7) ourselves, which is what lets us inject the terms below while holding every other draw fixed. The T=2T=2 tissue axis is the editable binary concept is_ileum; the model conditions on \{\text{celltype},\text{is_ileum}\}, while vv, the donor fingerprint γ\gamma, the library offset and the noise are unobserved and must be absorbed by the unknown factors UU.

Low-rank donor structure. In the base model γb,g\gamma_{b,g} is i.i.d. per gene, so donors are isolated points and a held-out donor is unpredictable in principle. We replace the donor block by a shared factor model

γ=HW,H∈RB×R, Hb,r∼N(0,1),W∈RR×F, Wr,g∼N(0,σB2R),(7)\gamma=HW,\qquad H\in\mathbb{R}^{B\times R},\ H_{b,r}\sim\mathcal{N}(0,1),\qquad W\in\mathbb{R}^{R\times F},\ W_{r,g}\sim\mathcal{N}\left(0,\frac{\sigma_B^2}{R}\right), \tag*{(7)}

with the RR dense programs WW drawn once and shared by all donors. Since Var⁡(γb,g)=R⋅σB2/R=σB2\operatorname{Var}(\gamma_{b,g})=R\cdot\sigma_B^2/R=\sigma_B^2, only the rank of the donor signal changes, not its strength; every donor, held-out ones included, lies on the same RR-dimensional subspace, so a donor readout learned on training donors transfers.

Nonlinear (heteroskedastic) counterfactual axis and matched linear control. The two members are generated from the same seed, hence share all coefficients, per-cell latents and exogenous draws, and differ only on a block VV of the ∣V∣\lvert V\rvert highest-baseline genes. In the linear member every gene keeps its additive tissue effect, so the is_ileum edit is the fixed shift τ1,⋅−τ0,⋅\tau_{1,\cdot}-\tau_{0,\cdot}, identical for all donors. In the nonlinear member the tissue main effect on VV is removed (τt,g=0\tau_{t,g}=0, g∈Vg\in V) and replaced by a donor–tissue variance interaction

log⁡λn,g+=ρ ϕbnm(tn),g∈V,ϕb∼N(0,1) with ∑bϕb=0,m(0)=1, m(1)=M,(8)\log\lambda_{n,g}+=\rho\,\phi_{b_n}m(t_n),\qquad g\in V,\qquad\phi_b\sim\mathcal{N}(0,1)\ \text{with}\ \sum_b\phi_b=0,\qquad m(0)=1,\ m(1)=M, \tag*{(8)}

i.e. each donor has its own signature ϕb\phi_b on VV whose amplitude is multiplied by MM in the ileum. By construction

\mathbb{E}[V\mid t]=\rho\,m(t)\mathbb{E}[\phi]=0\Rightarrow\operatorname{Cov}(\text{is_ileum},V)=0,\qquad \frac{\operatorname{Var}[V\mid\text{ileum}]}{\operatorname{Var}[V\mid\text{non\_ileum}]}=M^2, \tag*{(9)}

so the channel is invisible to a covariance penalty but visible to a kernel one: the means agree in both conditions and the dependence is entirely second-order. It also keeps the held-out-donor counterfactual well posed, since ϕb\phi_b is present in both conditions (amplitude ρ\rho at the source, ρM\rho M in the ileum) and is therefore readable from a donor’s source cells; the correct edit is to amplify each donor’s own baseline signature by MM, whereas storing the raw source amplitude in UU under-scales it. A CPU oracle verifies all three properties before training (measured Cov⁡≈0\operatorname{Cov}\approx0, variance ratio ≈M2\approx M^2, and per-donor Pearson r≥0.75r\geq0.75 between the source VV-mean and the true ϕb\phi_b). Both members are then passed through the identical pipeline used for the real data, median library-size normalization, log⁡(1+x)\log(1+x), PCA to 128 components, and evaluated under a donor holdout: 20 of the 150 donors are withheld from training entirely, and the is_ileum edit is scored per cell type on those unseen donors.

Final parameters used. All parameters finally used for the simulation are listed in Table 4.

Normalized Hilbert Schmidt Independence Criterion

The nHSIC orthogonality loss. HSIC as a differentiable penalty inside a single-cell VAE follows [27]; we use the normalized, centered-kernel-alignment form and apply it between the two bottleneck blocks rather than between the latent and a nuisance variable. The bottleneck of cell ii splits into a concept vector ci∈Rdcc_i \in\mathbb{R}^{d_c} and an unknown-factor vector ui∈Rduu_i \in\mathbb{R}^{d_u}. The two blocks have different dimensions and cannot be compared directly; instead we compare their pairwise-similarity patterns over a minibatch of nn cells. With an RBF kernel k(a,b)=exp⁡(−∥a−b∥2/2σ2)k(a,b)=\exp(-\lVert a-b\rVert^2/2\sigma^2) this gives two n×nn \times n Gram matrices

Kij=exp⁡(−∥ci−cj∥22σc2),Lij=exp⁡(−∥ui−uj∥22σu2),i,j=1,…,n,(10)K_{ij}=\exp\left(-\frac{\lVert c_i-c_j\rVert^2}{2\sigma_c^2}\right), \qquad L_{ij}=\exp\left(-\frac{\lVert u_i-u_j\rVert^2}{2\sigma_u^2}\right), \qquad i,j=1,\ldots,n, \tag*{(10)}

where entry ijij is the similarity of cells ii and jj in the respective channel (1 = identical, 0 = far apart). The bandwidths use the median heuristic, σ2=median⁡{∥xi−xj∥2}i≠j\sigma^2=\operatorname{median}\{\lVert x_i-x_j\rVert^2\}_{i\ne j}, evaluated per batch and detached from the computational graph, so the optimizer cannot trivialize the penalty by inflating σ\sigma until all similarities collapse to 1. Both matrices are centered with

H=In−1n11T,(11)H=I_n-\frac{1}{n}\mathbf{1}\mathbf{1}^{\mathsf{T}}, \tag*{(11)}

which subtracts row, column and grand means in the implicit feature space, so dependence is judged from deviations rather than from the overall similarity level. Alignment of the two centered patterns is measured by the (biased) Hilbert-Schmidt Independence Criterion [28].

HSIC⁡(c,u)=1(n−1)2tr⁡(KHLH),tr⁡(KHLH)=∑i,j(HKH)ij(HLH)ij,(12)\operatorname{HSIC}(c,u)=\frac{1}{(n-1)^2}\operatorname{tr}(KHLH), \qquad\operatorname{tr}(KHLH)=\sum_{i,j}(HKH)_{ij}(HLH)_{ij}, \tag*{(12)}

i.e. an entrywise inner product of the two centered similarity tables: large when cells that are alike in concept space are also alike in unknown space, zero iff cc and uu are independent in the RKHS (nonlinear dependence included, unlike a cross-covariance penalty). Its magnitude, however, depends on the feature scales and on nn. Normalizing by the geometric mean of the two self-alignments gives the centered kernel alignment we use as the orthogonal loss,

L⊥(c,u)=LnHSIC(c,u)=tr⁡(KHLH)tr⁡(KHKH)⋅tr⁡(LHLH)∈[0,1],(13)\mathcal{L}_{\perp}(c,u)=\mathcal{L}_{\mathrm{nHSIC}}(c,u)=\frac{\operatorname{tr}(KHLH)}{\sqrt{\operatorname{tr}(KHKH)\cdot\operatorname{tr}(LHLH)}}\in[0,1], \tag*{(13)}

with 0 = independent and 1 = fully dependent. The (n−1)−2(n-1)^{-2} factors cancel, so the value is scale-invariant and comparable across batches, and its weight in the objective is on an interpretable scale (λ⊥≈1\lambda_{\perp}\approx1–1010).

Numerical details. Three guards make the criterion differentiable in practice. (i) Squared distances are expanded as Dij=max⁡(∥xi∥2+∥xj∥2−2xiTxj,0)D_{ij}=\max(\lVert x_i\rVert^2+\lVert x_j\rVert^2-2x_i^{\mathsf{T}}x_j,0) rather than obtained from a Euclidean-distance primitive, whose gradient is undefined at coincident rows (a batch of near-identical concept predictions would emit NaNs); the max⁡(⋅,0)\max(\cdot,0) only absorbs floating-point round-off. (ii) The traces are evaluated as tr⁡(AHBH)=∑ij(AH)ij(BH)ji\operatorname{tr}(AHBH)=\sum_{ij}(AH)_{ij}(BH)_{ji}, avoiding the full matrix products. (iii) The denominator is clamped inside the square root, max⁡(tr⁡(KHKH)tr⁡(LHLH),ε)\sqrt{\max(\operatorname{tr}(KHKH)\operatorname{tr}(LHLH),\varepsilon)} with ε=10−12\varepsilon=10^{-12}: if the concept predictions collapse, tr⁡(KHKH)→0\operatorname{tr}(KHKH)\to0 and ddss\frac{d}{ds}\sqrt{s} diverges, so clamping the argument bounds the backward pass at 12ε−1/2\frac{1}{2}\varepsilon^{-1/2} while leaving non-degenerate batches unchanged. The loss returns 0 for n<4n<4, where the estimator is meaningless. Cost is O(n2)O(n^2) per batch.

Model and training

Architecture. We use the concept-bottleneck VAE of [25] unchanged, and modify only the orthogonality penalty (§ 5). An encoder maps a cell x∈Rdx\in\mathbb{R}^{d} to (μ,log⁡σ2)(\mu,\log\sigma^2) through a linear embedding into R1024\mathbb{R}^{1024} followed by 4 residual blocks (LINEAR→\toLAYERNORM→\toRELU→\toDROPOUT, p=0.1p=0.1, plus identity skip); z∈R128z\in\mathbb{R}^{128} is drawn by the reparametrization trick [32]. The bottleneck splits zz into two heads,

c^=sigmoid⁡(Wcz+bc)∈[0,1]dc,u=ReLU⁡(Wuz+bu)∈R≥0du,du=128,(14)\widehat{c}=\operatorname{sigmoid}(W_cz+b_c)\in[0,1]^{d_c},\qquad u=\operatorname{ReLU}(W_uz+b_u)\in\mathbb{R}_{\geq0}^{d_u},\qquad d_u=128, \tag*{(14)}

where c^\widehat{c} are the supervised concepts and uu the unknown factors that must absorb everything else (donor identity, library size, unmodelled state). The skip decoder embeds c^\widehat{c} and uu separately into R1024\mathbb{R}^{1024} and passes the uu-embedding through 4 residual blocks, each of which re-concatenates the concept embedding before its linear layer, so the concept vector is re-injected at every depth; a final linear layer returns x^∈Rd\widehat{x}\in\mathbb{R}^{d}. During training the decoder is conditioned on the predicted c^\widehat{c} (never on the ground-truth one-hot), so the concept pathway is the only route by which concept information can reach the reconstruction. A counterfactual replaces only the edited concept block by its target one-hot, and decodes from the same uu: abduction of uu, action on cc, prediction.

Objective. With a minibatch of nn cells,

L=MSE⁡(x^,x)⏟reconstruction+β[−12(1+log⁡σ2−μ2−σ2)]⏟KL+λcdc⋅BCE⁡(clip⁡(c^,0,1),c)+λ⊥L⊥(c^,u),(15)\mathcal{L}=\underbrace{\operatorname{MSE}(\widehat{x},x)}_{\text{reconstruction}}+\beta\underbrace{\left[-\frac{1}{2}\left(1+\log\sigma^2-\mu^2-\sigma^2\right)\right]}_{\mathrm{KL}}+\lambda_c d_c\cdot\operatorname{BCE}(\operatorname{clip}(\widehat{c},0,1),c)+\lambda_{\perp}\mathcal{L}_{\perp}(\widehat{c},u), \tag*{(15)}

all terms averaged over cells and coordinates (⋅ˉ\bar{\cdot} denotes the mean over the batch and latent dimensions). The clip to [0,1][0,1] guards the BCE against sigmoid round-off. Concepts are one-hot encoded and scored per column, so a multi-class concept contributes one BCE term per level. The orthogonality term is either the linear cross-covariance penalty

Lcov(c^,u)=∥Cov⁡^(u,c^)∥F2,Cov⁡^(u,c^)=1n−1u~Tc~(16)\mathcal{L}_{\mathrm{cov}}(\widehat{c},u)=\lVert\widehat{\operatorname{Cov}}(u,\widehat{c})\rVert_F^2,\qquad\widehat{\operatorname{Cov}}(u,\widehat{c})=\frac{1}{n-1}\widetilde{u}^{\mathsf{T}}\widetilde{c} \tag*{(16)}

(\tilde{\cdot} = batch-centered), our normalized-HSIC penalty LnHSIC\mathcal{L}_{\mathrm{nHSIC}} (Sec. 4.3), or their sum Lcov+LnHSIC\mathcal{L}_{\mathrm{cov}}+\mathcal{L}_{\mathrm{nHSIC}} (cov_nhsic); a single weight λ⊥\lambda_{\perp} multiplies whichever is selected.

Class-balanced concept loss. The binary tissue concept is strongly imbalanced (83% non-ileum) in the Taurus dataset, and an unweighted BCE lets the head reach a high accuracy while under-fitting the minority level, which is exactly the concept we edit. We therefore rescale the positive entries of column jj by wj=(1−pj)/pjw_j=(1-p_j)/p_j, with pjp_j the training prevalence of that level, keeping reduction='mean' so the loss scale of the other concepts is unchanged. For the taurus runs this gives w=4.88w=4.88 on is_ileum; the cell-type concept is left unweighted. The synthetic runs use the unweighted loss (all levels are near-balanced by construction).

Inputs and concepts. Both settings enter the model in the same representation: median library-size normalization, log⁡(1+x)\log(1+x), the top 3,000 highly variable genes (seurat_v3), and PCA to 128 components fitted once on all cells, so d=128d=128. This means the whole model training and evaluations run in PCA space. Concepts are supervised one-hot blocks: taurus uses the 7-way major lineage label plus the binary is_ileum (dc=8d_c=8); the simulation uses the 4-way celltype plus the binary is_ileum (dc=5d_c=5). Donor identity and sub cell type are never concepts, but must be recovered in uu, which is what the counterfactual evaluation tests.

Splits. Two nested splits are used, with different purposes. (i) Evaluation holdout: Removed from training entirely, never seen in any form. For taurus this is a global subtype holdout: all cells of the held-out subtype in the target condition (is_ileum = ileum ×\times stem_LGR5pos) are withheld, 4,100 of 987,743 cells, leaving 983,643 for training. At evaluation the retained Rectum stem_LGR5pos cells are edited non_ileum →\to ileum and compared to the real held-out ileal cells, per donor for the 14 donors that carry the subtype on both sides, and pooled. For the simulation this is a donor holdout: 20 of the 150 donors are withheld in full (207,231 of 240,000 cells remain), and the edit is scored per cell type on donors the model has never seen (§ 4.2). (ii) Train/validation: A uniformly random 10% of the remaining cells is held out as a validation set for monitoring, early stopping and checkpoint selection only: 885,279 / 98,364 for taurus and 186,508 / 20,723 for the simulation. This split is disjoint from (i) and is drawn from a fixed seed, so training and inference reconstruct identical splits from the observation table without caching them.

Optimization. Adam [33], learning rate 3×10−43 \times10^{-4} with an exponential per-epoch decay (γ=0.997\gamma=0.997), batch size 512 with reshuffling each epoch, at most 256 epochs. Training stops early when the total validation loss has not improved for patience consecutive epochs (min_delta = 0), and the checkpoint with the lowest total validation loss, not the last one, is the model used for every reported evaluation. Gradients are clipped to a global norm of 1.0 in the synthetic runs and left unclipped in the taurus runs. All runs use seed 0 for the data split, weight initialization, dropout and batch order.

Training diagnostics. Two quantities are logged every 10 epochs, since the scalar concept loss hides the two failure modes we care about. Per-concept accuracy (argmax within a one-hot block, threshold 0.5 for binary concepts) is compared against the majority-class rate of that concept, detecting a collapsed concept head. The edit ratio measures whether edits reach the output: validation cells are encoded once and decoded twice from the same uu, with the edited concept block set to source and to target,

edit ratio=∥dec⁡(ctgt,u)−dec⁡(csrc,u)∥2∥dec⁡(csrc,u)∥2.(17)\mathrm{edit\ ratio}=\frac{\left\lVert\operatorname{dec}\left(c^{\mathrm{tgt}},u\right)-\operatorname{dec}\left(c^{\mathrm{src}},u\right)\right\rVert_2}{\left\lVert\operatorname{dec}\left(c^{\mathrm{src}},u\right)\right\rVert_2}. \tag*{(17)}

A ratio near zero means the decoder reconstructs from uu alone and ignores the concept, so no counterfactual can propagate regardless of concept accuracy.

Configurations. Table 5 lists the settings that differ between runs; everything else is shared. On the real data we report the selected model (cov + nHSIC, λ⊥=10\lambda_{\perp}=10) together with the two covariance-only baselines at λ⊥=0.5\lambda_{\perp}=0.5 and λ⊥=10\lambda_{\perp}=10, which isolate the penalty type from its strength. The baselines were run with the more generous stopping patience (50 vs. 25) and still converged and stopped much earlier (87 and 71 epochs, best epochs 37 and 21) than the selected model (173 epochs, best epoch 148), so their weaker edits are not a consequence of a shorter budget. On the simulation we train the full grid {cov, nHSIC, cov + nHSIC} ×\times {0.5, 1, 5} on both members of the matched pair (18 runs) with a single shared configuration on NVIDIA A100 GPUs.

taurus (Rectum → ileum, stem)Synthetic pair
cov baselinesselectedgrid
Orthogonality penaltycovcov + nHSICcov / nHSIC / cov + nHSIC
λ⊥\lambda_{\perp}0.5, 10100.5, 1, 5
Class-balanced conceptsis_ileumis_ileumnone
Patience502550
Gradient clipping––1.0
Epochs run (best epoch)71 (21), 87 (37)173 (148)up to 256
Sharedd=128d=128 (PCA), latent 128, du=128d_u=128, hidden 1024, 4 layers, dropout 0.1, skip decoder, β=10−5\beta=10^{-5}, λc=0.1\lambda_c=0.1, Adam 3×10−43\times10^{-4}, γlr=0.997\gamma_{\mathrm{lr}}=0.997, batch 512, max 256 epochs, seed 0, val. fraction 0.1.

Table 5. Training configurations. All models share one architecture and optimizer; the runs differ only in the orthogonality penalty and its weight (plus the stopping patience and gradient clipping noted above).

Evaluation

Distances. Two distances between two sets of cells are used, both computed in the 128-dimensional PCA space the model is trained in. The first is the unbiased squared maximum mean discrepancy, written MMD throughout, with a Gaussian kernel,

MMD⁡(X,Y)=1n(n−1)∑i≠jk(xi,xj)+1m(m−1)∑i≠jk(yi,yj)−2nm∑i,jk(xi,yj),(18)\operatorname{MMD}(X,Y)=\frac{1}{n(n-1)}\sum_{i\ne j}k(x_i,x_j)+\frac{1}{m(m-1)}\sum_{i\ne j}k(y_i,y_j)-\frac{2}{nm}\sum_{i,j}k(x_i,y_j), \tag*{(18)}

with k(a,b)=exp⁡(−ς∥a−b∥2)k(a,b)=\exp(-\varsigma\lVert a-b\rVert^2) and the bandwidth set by the median heuristic on the pooled sample, ς=1/(2median⁡{∥zi−zj∥}2)\varsigma=1/(2\operatorname{median}\{\lVert z_i-z_j\rVert\}^2), z=X∪Yz=X\cup Y. Self-similarity terms are excluded and the estimate is clipped at 0. The second is the debiased Sinkhorn divergence S(X,Y)=OT⁡ε(X,Y)−12OT⁡ε(X,X)−12OT⁡ε(Y,Y)S(X,Y)=\operatorname{OT}_{\varepsilon}(X,Y)-\frac{1}{2}\operatorname{OT}_{\varepsilon}(X,X)-\frac{1}{2}\operatorname{OT}_{\varepsilon}(Y,Y) with entropic regularization ε=0.1\varepsilon=0.1 on squared-Euclidean cost matrices, each normalized by its own maximum before solving and the divergence rescaled back. Both groups are subsampled to at most 2,000 cells with a fixed seed before either distance is computed.

Ratio metrics. Absolute distances are not comparable across subtypes or datasets, so, following [25], we normalize by the distance from the target to the closest population that already exists in the training data. Writing X^\hat{X} for the edited cells and XtgtX^{\mathrm{tgt}} for the real withheld target cells,

rMMD⁡=MMD⁡(X^,Xtgt)min⁡p∈PMMD⁡(Xp,Xtgt),rSinkhorn⁡=S(X^,Xtgt)min⁡p∈PS(Xp,Xtgt).(19)\operatorname{rMMD}=\frac{\operatorname{MMD}(\hat{X},X^{\mathrm{tgt}})}{\min_{p\in\mathcal{P}}\operatorname{MMD}(X_p,X^{\mathrm{tgt}})},\qquad \operatorname{rSinkhorn}=\frac{S(\hat{X},X^{\mathrm{tgt}})}{\min_{p\in\mathcal{P}}S(X_p,X^{\mathrm{tgt}})}. \tag*{(19)}

where P\mathcal{P} is a set of candidate populations drawn from the training cells only. A value below 1 means the edit is a better description of the target than the best available "map the cells onto the nearest existing population" answer; the numerator, the identity of the winning population p∗p^{*} and the full candidate list are recorded for every run. The minimum is taken separately for each metric, so the winner may differ between rMMD and rSinkhorn.

Candidate populations, pooled subtype holdout (Taurus). Here P\mathcal{P} is the set of joint combinations of the model’s own concept columns: every observed pair (major lineage, is_ileum) among the training cells, with populations of fewer than 10 cells dropped. For the Taurus runs this yields 14 candidates from the 7 lineages crossed with the binary tissue concept. Fine subtypes are deliberately not used to define candidates, so the baseline is "the closest population the model is allowed to name", not "the same subtype in the target condition", which would not exist for a held-out subtype. For all three Taurus runs the winning population was (epithelial, non_ileum) at MMD 0.244, i.e. the un-edited source lineage, and the same population also won under the Sinkhorn divergence.

Candidate populations, donor holdout (simulation). The donor holdout scores each (held-out donor bb, cell type kk) pair separately. The numerator is the distance between the edit of that donor’s source cells of type kk and the same donor’s real target cells of type kk. The denominator is the distance between the generic training population of the target condition and cell type kk, pooled over all training donors, and that same donor’s real target cells. The ratio therefore isolates donor specificity: a model that predicts the condition correctly but ignores who the donor is scores ≈1\approx1 by construction, and only a model that recovers the donor’s own position scores below it. A (donor, cell type) row is scored only if both sides have at least 20 cells and the denominator population has at least 10; we report the median over the surviving rows and, where stated, the fraction of rows below 1. For the final simulation pair this gives 65 rows across the 20 held-out donors.

Subtype screen. The held-out subtype for the pooled evaluation was chosen before any training by a model-free screen. For each of the 36 location-agnostic fine subtypes we computed the real source-versus-target distance (MMD and Sinkhorn, per donor and pooled) together with the number of donors carrying the subtype on both sides and the cell counts per side. Subtypes whose two conditions barely differ are unsuitable, because the denominator of Eq. (19) then collapses onto the source population and the ratio becomes unstable. stem_LGR5pos ranked third by per-donor MMD (0.201; pooled 0.146) with 14 paired donors, 12,614 source and 4,100 target cells, and was preferred over the two larger-gap subtypes for its higher paired-donor count and its coherence in the target condition. Because the holdout is baked into training, each candidate subtype would require its own model, so the screen is run first and only the selected subtype’s hold-out model is trained.

Use of LLMs

During the preparation of this work, the authors used Claude Opus 4.8, Opus 5 and Opus 5.5 to improve the code, language, readability, and flow of the manuscript. After using this tool, the authors reviewed and edited the content as needed and take full responsibility for the scientific integrity and final content of the publication.

References

  1. [1]Judea Pearl. Causality: Models, Reasoning, and Inference. 2nd. Cambridge, UK: Cambridge University Press, 2009.
  2. [2]Evan Z Macosko et al. “Highly parallel genome-wide expression profiling of individual cells using nanoliter droplets”. en. In: Cell 161.5 (May 2015), pp. 1202–1214. DOI: 10.1016/j.cell.2015.05.002.DOI
  3. [3]Fuchou Tang et al. “mRNA-Seq whole-transcriptome analysis of a single cell”. en. In: Nat. Methods 6.5 (May 2009), pp. 377–382. DOI: 10.1038/nmeth.1315.DOI
  4. [4]Alexandra-Chloé Villani et al. “Single-cell RNA-seq reveals new types of human blood dendritic cells, monocytes, and progenitors”. en. In: Science 356.6335 (Apr. 2017), eaah4573. DOI: 10.1126/science.aah4573.DOI
  5. [5]Cyril Neftel et al. “An integrative model of cellular states, plasticity, and genetics for glioblastoma”. en. In: Cell 178.4 (Aug. 2019), 835–849.e21. DOI: 10.1016/j.cell.2019.06.024.DOI
  6. [6]Tabula Muris Consortium. “Single-cell transcriptomics of 20 mouse organs creates a Tabula Muris”. en. In: Nature 562.7727 (Oct. 2018), pp. 367–372. DOI: 10.1038/s41586-018-0590-4.
  7. [7]Aviv Regev et al. “The Human Cell Atlas”. en. In: Elife 6 (Dec. 2017), e27041. DOI: 10.7554/eLife.27041.DOI
  8. [8]Tabula Sapiens Consortium. “The Tabula Sapiens: A multiple-organ, single-cell transcriptomic atlas of humans”. en. In: Science 376.6594 (May 2022), eabl4896. DOI: 10.1126/science.abl4896.
  9. [9]Korbinian Traeuble et al. “Integrated single-cell atlas of human atherosclerotic plaques”. en. In: Nat. Commun. 16.1 (Sept. 2025), p. 8255. DOI: 10.1038/s41467-025-63202-x.DOI
  10. [10]Cristian Ruiz-Moreno et al. “Charting the single-cell and spatial landscape of IDH-wild-type glioblastoma with GBmap”. en. In: Neuro Oncol. 27.9 (2025), pp. 2281–2295. DOI: 10.1093/neuonc/noaf113.
  11. [11]Romain Lopez et al. “Deep generative modeling for single-cell transcriptomics”. en. In: Nat. Methods 15.12 (Dec. 2018), pp. 1053–1058. DOI: 10.1038/s41592-018-0229-2.DOI
  12. [12]Christina V Theodoris et al. “Transfer learning enables predictions in network biology”. en. In: Nature 618.7965 (June 2023), pp. 616–624. DOI: 10.1038/s41586-023-06139-9.DOI
  13. [13]Haotian Cui et al. “scGPT: toward building a foundation model for single-cell multi-omics using generative AI”. en. In: Nat. Methods 21.8 (Aug. 2024), pp. 1470–1480. DOI: 10.1038/s41592-024-02201-0.DOI
  14. [14]Mohammad Lotfollahi, F Alexander Wolf, and Fabian J Theis. “scGen predicts single-cell perturbation responses”. en. In: Nat. Methods 16.8 (Aug. 2019), pp. 715–721. DOI: 10.1038/s41592-019-0494-8.DOI
  15. [15]Omar Kana et al. “Generative modeling of single-cell gene expression for dose-dependent chemical perturbations”. en. In: Patterns 4.8 (Aug. 2023), p. 100817. DOI: 10.1016/j.patter.2023.100817.DOI
  16. [16]Mohammad Lotfollahi et al. “Predicting cellular responses to complex perturbations in high-throughput screens”. en. In: Mol. Syst. Biol. 19.6 (June 2023), e11517. DOI: 10.15252/msb.202211517.DOI
  17. [17]Yusuf Roohani, Kexin Huang, and Jure Leskovec. “Predicting transcriptional outcomes of novel multigene perturbations with GEARS”. en. In: Nat. Biotechnol. 42.6 (June 2024), pp. 927–935. DOI: 10.1038/s41587-023-01905-6.DOI
  18. [18]Charlotte Bunne et al. “Learning single-cell perturbation responses using neural optimal transport”. en. In: Nat. Methods 20.11 (Nov. 2023), pp. 1759–1768. DOI: 10.1038/s41592-023-01969-x.DOI
  19. [19]Mingze Dong et al. “Causal identification of single-cell experimental perturbation effects with CINEMA-OT”. en. In: Nat. Methods 20.11 (Nov. 2023), pp. 1769–1779. DOI: 10.1038/s41592-023-02040-5.DOI
  20. [20]Mohammad Lotfollahi et al. “Biologically informed deep learning to query gene programs in single-cell atlases”. en. In: Nat. Cell Biol. 25.2 (Feb. 2023), pp. 337–350. DOI: 10.1038/s41556-022-01072-x.DOI
  21. [21]Hengshi Yu and Joshua D Welch. “MichiGAN: sampling from disentangled representations of single-cell data using generative adversarial networks”. en. In: Genome Biol. 22.1 (May 2021), p. 158. DOI: 10.1186/s13059-021-02373-4.DOI
  22. [22]Ziqi Zhang et al. “scDisInFact: disentangled learning for integration and prediction of multi-batch multi-condition single-cell RNA-sequencing data”. en. In: Nat. Commun. 15.1 (Jan. 2024), p. 912. DOI: 10.1038/s41467-024-45227-w.DOI
  23. [23]Zoe Piran et al. “Disentanglement of single-cell data with biolord”. en. In: Nat. Biotechnol. 42.11 (Nov. 2024), pp. 1678–1683. DOI: 10.1038/s41587-023-02079-x.DOI
  24. [24]Yicheng Gao et al. “Causal disentanglement for single-cell representations and controllable counterfactual generation”. In: Nat. Commun. 16 (2025). DOI: 10.1038/s41467-025-62008-1.DOI
  25. [25]Alma Andersson et al. scCBGM: Interpretable Single-Cell Counterfactual Editing. 2026. arXiv: 2606.07760 [cs.LG]. URL: https://arxiv.org/abs/2606.07760.
  26. [26]Pang Wei Koh et al. “Concept bottleneck models”. In: Proceedings of the 37th International Conference on Machine Learning (ICML). Vol. 119. PMLR. 2020, pp. 5338–5348.arxiv.org/abs/2007.04612
  27. [27]Romain Lopez et al. “Information Constraints on Auto-Encoding Variational Bayes”. In: Advances in Neural Information Processing Systems (NeurIPS). Vol. 31. 2018, pp. 6114–6125.arxiv.org/abs/1805.08672
  28. [28]Arthur Gretton et al. “Measuring statistical dependence with Hilbert–Schmidt norms”. In: Algorithmic Learning Theory (ALT). Vol. 3734. Lecture Notes in Computer Science. Springer, 2005, pp. 63–77. DOI: 10.1007/11564089_7.DOI
  29. [29]Tom Thomas et al. “A longitudinal single-cell atlas of anti-tumour necrosis factor treatment in inflammatory bowel disease”. en. In: Nat. Immunol. 25.11 (Nov. 2024), pp. 2152–2165. DOI: https://doi.org/10.1038/s41590-024-01994-8.
  30. [30]Hyun Min Kang et al. “Multiplexed droplet single-cell RNA-sequencing using natural genetic variation”. en. In: Nat. Biotechnol. 36.1 (Jan. 2018), pp. 89–94. DOI: 10.1038/nbt.4042.DOI
  31. [31]F Alexander Wolf, Philipp Angerer, and Fabian J Theis. “SCANPY: large-scale single-cell gene expression data analysis”. en. In: Genome Biol. 19.1 (Feb. 2018), p. 15. DOI: 10.1186/s13059-017-1382-0.DOI
  32. [32]Diederik P. Kingma and Max Welling. “Auto-Encoding Variational Bayes”. In: 2nd International Conference on Learning Representations, ICLR 2014, Banff, AB, Canada, April 14-16, 2014, Conference Track Proceedings. 2014. URL: http://arxiv.org/abs/1312.6114.
  33. [33]Diederik P. Kingma and Jimmy Ba. “Adam: A Method for Stochastic Optimization”. In: International Conference on Learning Representations (ICLR). 2015. URL: https://arxiv.org/abs/1412.6980.

Paper details

vlnc:W_r9msty4u
Version
v1 · latest
Published
Fresh — added to the Hub within the last 30 days
Added
License
CC BY 4.0
Usage

views · 12

downloads · 0

A view counts once per browser per day, a download each time the PDF is requested.

  • Computer Science

Claude Opus 4.8 (bare), Claude Opus 5 (bare), Opus 5.5

View statement
Looking for related papers…
Contents