betaFeedback

WorksComputer Science

Learning Biological Hierarchies in Single-Cell Foundation Models

Abstract

Cell types are organized by taxonomic relationships that define lineages in the so-called cell ontology. However, most existing foundation models typically ignore cell-type lineages encoded in this cell ontology. We introduce a biologically informed foundation model called scOntoFM, which embeds hierarchical cell ontology into representation learning. Using Lowest Common Ancestor (LCA) distances from the ontology graph, our framework pairs efficient offline triplet sampling with hierarchical cell-ontology learning. This approach jointly preserves local neighborhoods and shapes the global embedding geometry to reflect the ontology structure. Across diverse cell- and gene-level benchmarks, the model consistently improves performance, with especially strong gains in zero-shot settings. Our model also balances batch integration with biological structure preservation, providing complementary perspectives to existing embedding evaluation frameworks and supporting scalable biological discovery.

Introduction

Current single-cell RNA sequencing (scRNA-seq) technology has revolutionized our ability to characterize cellular heterogeneity and uncover the molecular mechanisms underlying healthy and diseased tissues [25]. Driven by advances in high-throughput sequencing and large-scale atlas construction, the field has witnessed an explosion of massive single-cell datasets in recent years [42, 14, 40, 1]. This data deluge has created an urgent demand for computational frameworks capable of fully utilizing these resources for broad biological discovery. Inspired by the transformative success of Large Language Models (LLMs) in Natural Language Processing (NLP), the community has recently developed Single-Cell Foundation Models (scFMs), such as Geneformer [45], scGPT [10], scFoundation [22], scCello [54], CellWhisperer [38], etc. By pretraining on large corpora of cell-level transcriptomic data, these models aim to learn generalizable cellular representations that can be transferred to diverse downstream tasks, ranging from cell-type annotation to gene-perturbation prediction [44]. We provide a detailed description of the Related Work in Appendix § 7.

Contrastive learning is widely adopted in current scFMs as an auxiliary objective to learn discriminative features between cell types [56, 24, 54, 13]. Among these approaches, ontology-aware methods such as Similarity [24] and scCello [54] further incorporate cell ontology as supervised signals during model training, leveraging online triplet sampling and vanilla contrastive objectives to encourage biologically meaningful representation learning. However, scaling such frameworks to atlas-level datasets introduces substantial computational challenges, as standard online triplet sampling is highly inefficient for massive datasets. In random mini-batches, informative positive pairs are rarely encountered (Appendix Figure (AFigure) 7), leading to weak supervision and suboptimal metric learning [18, 43]. Moreover, existing methods such as scCello mainly focus on local cell-type relational alignment guided by the ontology structure, while overlooking the global hierarchical structure. ture encoded in the cell ontology [34, 57]. Consequently, the learned latent space may not faithfully reflect biological phylogeny and developmental lineages, limiting both biological interpretability and the ability to capture dynamic cellular processes [47, 50].

To address these challenges, we propose scOntoFM, a novel biologically informed foundation model that explicitly integrates hierarchical cell ontology into a representation learning framework. Cell Ontology [34] provides a curated hierarchy of cell-type identities that complements transcriptomic supervision. Consistent with prior single-cell studies linking ontology relationships to transcriptomic similarity [48, 37, 50], we used the Lowest Common Ancestor (LCA) distance [39, 5] from the cell ontology graph as an interpretable structural prior and an approximate proxy for transcriptomic distance. This prior serves as a guide for both data sampling and loss optimization in scOntoFM. Our contributions are threefold.

  • Efficient Ontology-aware Offline Sampling: We introduce the Cell-Dataset Index Map, a hierarchical indexing structure for efficient offline triplet sampling across cell types and datasets. Guided by the LCA distance matrix, it prioritizes cross-dataset positives and semi-hard and hard negatives drawn from semantically close cell types in the ontology graph, thus overcoming the inefficiency of random online sampling commonly used in current scFMs, e.g., scCello [54].

  • Hierarchical Structure Learning: We incorporate the ontology structure into representation learning through a dynamic margin triplet loss, where the margin is determined by the LCA distance between the anchor and negative cell types and the ontology depth of the anchor cell type. In addition, a complementary rank-order alignment loss enforces consistency between the relative ordering of cell-type centroid distances in the embedding space and their ontology-based LCA distances within each batch. These two synergistic losses enforce both fine-grained local discrimination and global hierarchical consistency, producing an embedding space that better reflects the structure of the cell-type ontology.

  • Comprehensive Evaluation: We benchmark our model against state-of-the-art (SOTA) methods across diverse cell- and gene-level tasks. Our results demonstrate superior generalizability, particularly in zero-shot settings. Crucially, as highlighted by recent critiques of embedding metrics (e.g., the “drifting islands” phenomenon) [47], purely aligning cell embeddings with cell-type annotations will lead to biologically problematic embeddings. Our evaluation confirms that our approach achieves a superior balance between the well established scIB metrics [32], which measure the proximity of cells that belong to the same cell-type cluster, and how well cells of the same cell type mix-up across batches/datasets in single cell studies, and the more rigorous biological structure preservation metric of model embeddings [50].

Methods

Figure 1 provides a high-level overview of the proposed scOntoFM workflow. We first describe the pre-training data processing, the construction of the Lowest Common Ancestor (LCA) distance matrix in § 2.1, which is followed by the offline triplet sampling, model architecture, and model pre-training objectives in § 2.2, § 2.3 and § 2.4, respectively.

Figure 1. Proposed model workflow. (a) scOntoFM pre-training data corpus details (left), and the distribution of major tissues and diseases in the corpus (right). (b) An illustration of cell ontology graph and its derived LCA distance matrix. (c) Ontology-aware loss terms in scOntoFM, the dynamic margin α(ca,cn)\alpha(c_a,c_n) is controlled by the distance between the anchor cell-type cac_a and negative cell-type cnc_n, and the depth of the anchor cac_a in the ontology graph. Down below is the hierarchical ontology alignment loss, in which Δemb\Delta^{\mathrm{emb}} and Δ~onto∈RB×B\widetilde{\Delta}^{\mathrm{onto}} \in\mathbb{R}^{B \times B} represent correspondingly the distance matrix of our model embeddings and its corresponding LCA distance matrix in the batch (of size BB), and where rembr^{\mathrm{emb}} and rontor^{\mathrm{onto}} are the soft-ranking rows in their respective distance matrices. ρS(⋅,⋅)\rho_S(\cdot,\cdot) is the Spearman correlation for the soft-ranking vectors. (d) Downstream tasks of scOntoFM, from left to right and grouped into cell- and gene-level tasks as presented in the results.

Data Processing

Single-cell Data Let D={xn,cn}n=0N\mathcal{D} = \{x_n, c_n\}_{n=0}^{N} denote a single-cell RNA-seq dataset, where xn∈R∣M∣x_n \in\mathbb{R}^{|\mathcal{M}|} is the raw gene expression vector in cell nn, and M={m1,…,m∣M∣}\mathcal{M} = \{m_1, \ldots, m_{|\mathcal{M}|}\} represents the global gene set (vocabulary) of size ∣M∣|\mathcal{M}| defined by our pre-training corpus. Each cell is associated with a cell type label cn∈Vc_n \in\mathcal{V}, where V\mathcal{V} represents the set of all cell type nodes in the cell ontology graph, which we will introduce in the next section. An overview of our pre-training data corpus is presented in Figure 1(a).

We follow the same data pre-processing steps as Geneformer [45]: first we normalize the library size of each cell to 10,000 to account for sequencing depth variation, and re-normalize each gene expression value with its non-zero median over the entire pre-training dataset, i.e.,

xn,m′=104xn,m∑m′∈Mxn,m′,xn,m′′=xn,m′median⁡{xn,m′∣xn,m′>0},n=1,…,N, m∈Mx'_{n,m} = 10^{4} \frac{x_{n,m}}{\sum_{m' \in\mathcal{M}} x_{n,m'}}, \qquad x''_{n,m} = \frac{x'_{n,m}}{\operatorname{median}\{x'_{n,m} \mid x'_{n,m} > 0\}}, \qquad n = 1,\ldots,N,\ m \in\mathcal{M}

where xn,m′′x''_{n,m} is the normalized gene expression for cell nn and gene mm. Following normalization, we perform rank-value encoding to generate the inputs of the model. For each cell nn, we sort the normalized expression values xn,m′′x''_{n,m} in descending order. The resulting input is a sequence of gene tokens Sn=(m(1),m(2),…,m(L))S_n = (m_{(1)},m_{(2)},\ldots,m_{(L)}), where each token m(k)∈Mm_{(k)} \in\mathcal{M} is the gene corresponding to the kk-th highest expression value in cell nn. We truncate this gene sequence to a maximum length of L=2,000L = 2{,}000, and then append the [CLS] and [EOS] tokens at the beginning and the end of the sequence, respectively. By defining the sequence elements directly as members of M\mathcal{M}, we ensure that the input tokens map one-to-one with the pre-defined model vocabulary.

LCA Distance Matrix Construction To quantify biological similarity, we leverage the hierarchical structure of the cell ontology graph G=(V,E)\mathcal{G} = (\mathcal{V},\mathcal{E}) from the Open Biological and Biomedical Ontology (OBO) Foundry [41], where V\mathcal{V} denotes cell-type nodes in the graph and every cell label cic_i in our dataset is an instance of a node v∈Vv \in\mathcal{V}; E\mathcal{E} is the directed edge between two nodes, encoding the hierarchical relationship (“is a subtype of”) between cell types. We construct a pairwise distance matrix Δ∈R∣V∣×∣V∣\Delta\in\mathbb{R}^{|\mathcal{V}| \times|\mathcal{V}|} derived from the Lowest Common Ancestor (LCA) based on G\mathcal{G} Figure 1(b). For any two types of cells ci,cj∈Vc_i,c_j \in\mathcal{V}, their raw semantic distance dG(ci,cj)d_{\mathcal{G}}(c_i,c_j) is calculated as the sum of the shortest path distances from each node to their common cs=LCA(ci,cj)c_s = \mathrm{LCA}(c_i,c_j):

Δi,j=dG(ci,cj)=δpath(ci,cs)+δpath(cj,cs),(1)\Delta_{i,j} = d_{\mathcal{G}}(c_i,c_j) = \delta_{\mathrm{path}}(c_i,c_s) + \delta_{\mathrm{path}}(c_j,c_s), \tag*{(1)}

where δpath(ci,cs)\delta_{\mathrm{path}}(c_i,c_s) and δpath(cj,cs)\delta_{\mathrm{path}}(c_j,c_s) denote the number of edges on the shortest path between nodes (ci,cj)(c_i,c_j) to their lowest common ancestor csc_s in G\mathcal{G}, respectively. To facilitate stable training with the dynamic triplet margin loss in § 2.4, we compute the normalized distance d~(ci,cj)\widetilde{d}(c_i,c_j) element wise and obtain Δ~i,j\widetilde{\Delta}_{i,j} by scaling dG(ci,cj)d_{\mathcal{G}}(c_i,c_j) by the maximum distance on the ontology graph. The matrix Δ~\widetilde{\Delta}, with elements Δ~i,j\widetilde{\Delta}_{i,j}, serves as the ground truth for biological distinctness during the extraction of negative pairs in § 2.2 and to determine the dynamic margin for the triplet loss defined in § 2.4. We provide more empirical evidence about the relationship between LCA-based distance and transcriptomic similarity in Section 4 and Appendix § 8.2.

Offline Triplet Sampling

Contrastive learning [8] is a representation learning framework that aims to learn an embedding space such that it encourages semantically similar samples to be closer while pushing dissimilar samples apart. In contrastive learning, an anchor sample serves as a reference that is used to compare, by proxy, positive and negative samples. The general concept is shown in Figure 1(c). Efficiently mining of informative triplets is critical for contrastive learning. While standard online triplet sampling, like in SCimilarity and scCello, is highly inefficient at massive scale, since random mini-batches are unlikely to contain sufficiently informative positive pairs. Here, we propose a hierarchical indexing structure, which we term the Cell-Dataset Index Map (Ω\Omega), to enable O(1)O(1) retrieval of candidate samples. Let V\mathcal{V} be the set of cell types and K\mathcal{K} be the set of all dataset identifiers (e.g., studies). We define Ω\Omega as the collection of index sets Ic,k\mathcal{I}_{c,k}, each of which contains the indices of cells belonging to type c∈Vc \in\mathcal{V} and dataset k∈Kk \in\mathcal{K}: Ω={Ic,k}c∈V,k∈K\Omega= \{\mathcal{I}_{c,k}\}_{c\in\mathcal{V},k\in\mathcal{K}}, where again Ic,k\mathcal{I}_{c,k} denotes the set of indices for all cells of type cc found in dataset kk. Additionally, for any cell type cc, we denote the set of datasets containing this cell type as Kc={k∈K∣Ic,k≠∅}\mathcal{K}_{c} = \{k \in\mathcal{K} \mid\mathcal{I}_{c,k} \ne\emptyset\}. For each training step, we select an anchor cell xax_a with cell type cac_a from dataset kak_a. Specifically, we construct triplets (xa,xp,xn)(x_a, x_p, x_n) as follows.

Positive Sampling To promote batch mixing for the same cell type, we prioritize sampling cross-dataset positive pairs from different datasets. We define a pool of positive candidates, P+\mathcal{P}^{+}, designed to prioritize cross-dataset pairs while also handling edge cases where a cell type is unique to a single dataset. If the cell type cac_a exists in multiple datasets (∣Kca∣>1|\mathcal{K}_{c_a}| > 1), we construct the pool by taking the union of all instances from the datasets in Kca\mathcal{K}_{c_a} other than kak_a: P+=⋃k∈Kca∖{ka}Ica,k\mathcal{P}^{+} = \bigcup_{k\in\mathcal{K}_{c_a}\setminus\{k_a\}} \mathcal{I}_{c_a,k}, and then we uniformly sample xp∼Uniform⁡(P+)x_p \sim\operatorname{Uniform}(\mathcal{P}^{+}). If the cell type only exists in the anchor’s dataset (i.e., P+=∅\mathcal{P}^{+} = \emptyset), we default to sampling xpx_p from the anchor’s own dataset index Ica,ka\mathcal{I}_{c_a,k_a}.

Negative Sampling Randomly selected negative samples (e.g., “T cell” vs. “Neuron”) are often too easy to distinguish, producing zero loss and vanishing gradients. To ensure robust learning, we specifically mine “semi-hard” to “hard” negatives—cell types that are biologically distinct but semantically close to the anchor.

We utilize the Cell Ontology graph to measure semantic proximity. Recall that Δ\Delta is the pre-computed LCA distance matrix between all cell types. For an anchor with cell-type cac_a, we first define the set of valid negative cell types, denoted as Vn\mathcal{V}_n, excluding the anchor type cac_a and its lineage (ancestors and descendants) to avoid conflicting training signals (ambiguous pairs) during training: Vn={v∈V∣v≠ca∧v∉Lineage⁡(ca)}\mathcal{V}_n = \{v \in\mathcal{V} \mid v \ne c_a \land v \notin\operatorname{Lineage}(c_a)\}. To identify the most informative negatives, we rank all valid types based on their proximity to the anchor. Let rank⁡(ca,v)\operatorname{rank}(c_a,v) denote the ordinal rank of the anchor cell type cac_a and all other valid negative cell types v∈Vnv \in\mathcal{V}_n when sorted by ascending LCA distance Δ(ca,v)\Delta(c_a,v). We select the set of hard to semi-hard negative candidates by following Chard={v∈Vn∣rank⁡(ca,v)≤M}\mathcal{C}_{\mathrm{hard}} = \{v \in\mathcal{V}_n \mid\operatorname{rank}(c_a,v) \le M\}, and we set M=50M = 50 by default, which provides a diverse set of closely-related cell types to sample from, stabilizing training while maintaining optimization “difficulty”. By discriminating against these biologically similar types, the model learns to capture high-resolution lineage markers rather than than just broad lineage differences. Then the negative cell instance xnx_n is uniformly sampled from: xn∼⋃c∈ChardIc,kx_n \sim\bigcup_{c\in\mathcal{C}_{\mathrm{hard}}} \mathcal{I}_{c,k}.

Since the collection of index sets Ω\Omega pre-groups indices, we avoid costly filtering operations during training. The sampling logic described above is implemented as a vectorized look-up operation. We pre-compute the valid positive dataset indices and negative cell type candidates for every batch, allowing us to generate millions of triplets in parallel using efficient tensor gathering operations on the GPU. To support this empirically, the computational cost of constructing the Cell-Dataset Index Map is summarized in Table 8.

MetricValue
CPUIntel Xeon Gold 6252
Time Consumption∼5\sim5 minutes
Peak Memory Usage6.5 GB
Final File Size132 MB

Table 8. Computational cost of constructing the Cell-Dataset Index Map.

Model Architecture

We adopt a transformer architecture similar to the Geneformer [45]. The model consists of 6 transformer layers with 256 hidden units and 4 attention heads. The input to the model is the rank-ordered sequence of gene tokens SnS_n, which is embedded as follows: H0=E(Sn)+Epos\mathbf{H}_0 = \mathrm{E}(S_n) + \mathbf{E}_{pos}, where E(⋅)\mathrm{E}(\cdot) is the gene-token embedding layer, Epos\mathbf{E}_{pos} represents the learnable positional embedding, and the embedding H0\mathbf{H}_0 is the embedded input used by the model. A special [CLS] token is added to the beginning of the sequence to aggregate global cell representations, and [EOS] is added to the end of the input gene sequence SnS_n. We use the output of the final hidden layer of the transformer model associated with the [CLS] token, denoted as znz_n, as the low-dimensional cell embedding.

Learning Objective

We optimize our model using three objectives, namely i) the standard MLM term, ii) the dynamic margin triplet loss, and iii) the global ontology alignment loss. The latter two loss objectives leverage the LCA distance matrix to guide the model to learn biologically meaningful representations. Compared with existing methods such as scCello [54], which employs a fixed scaling parameter to learn ontology-informed pairwise relational alignment in its contrastive loss and focuses on local cell-type relationships, our approach uses a dynamic margin triplet loss to enable finer-grained discrimination of cell states, while the rank-order alignment loss further preserves the global hierarchical structure in the embedding space.

Masked Language Modeling (MLM) Like in BERT [12], we apply the MLM strategy with our input sequence SnS_n, and the masking ratio is set to 15%. For the selected masking position, we apply the same masking rule as used in BERT, (i) there is 80% chance that the token is replaced by the [MASK] token; (ii) 10% probability that the token will be replaced by a random token; and (iii) 10% chance that the token will remain unchanged. Let Jmask⊂1,…,L\mathcal{J}_{\mathrm{mask}} \subset1,\ldots,L, denote the set of masked position indices. We write the MLM loss as follows

LMLM=−∑j∈Jmasklog⁡P(tj∣Sn\Jmask),(2)\mathcal{L}_{\mathrm{MLM}}=-\sum_{j\in\mathcal{J}_{\mathrm{mask}}}\log P(t_j\mid S_n\backslash\mathcal{J}_{\mathrm{mask}}), \tag*{(2)}

where tjt_j is the original token in the masked position and Sn\JmaskS_n\backslash\mathcal{J}_{\mathrm{mask}} represents the masked sequence. This objective enables the model to learn gene–gene co-expression patterns that can be used to reconstruct missing gene expression values as previously described in [45].

Dynamic Ontology-Aware Triplet Loss To enforce the biological hierarchy in the embedding space, we employ a triplet loss with a dynamic margin α(⋅,⋅)\alpha(\cdot,\cdot) (Figure 1(c) top panel). Specifically, α(⋅,⋅)\alpha(\cdot,\cdot) adapts to the specific semantic relationship between the anchor and the negative sample based on their positions in the cell-type ontology instead of setting a fixed margin as in the original triplet loss [18]. Specifically, we define the dynamic margin α(ca,cn)\alpha(c_a,c_n) as:

α(ca,cn)=ϵ+(1−δ~(ca,cn))+depth⁡(ca),(3)\alpha(c_a,c_n)=\epsilon+(1-\widetilde{\delta}(c_a,c_n))+\operatorname{depth}(c_a), \tag*{(3)}

where ϵ\epsilon is a small constant offset (set to 0.1 by default), δ~\widetilde{\delta} denotes the normalized LCA distance between cell types cac_a and cnc_n, and depth⁡(ca)\operatorname{depth}(c_a) is the normalized depth of the anchor node in the ontology graph. This design encourages a larger separation margin for semantically similar cell types, i.e., small δ~\widetilde{\delta}, leading to larger margin, and for anchors corresponding to fine-grained cell types, i.e., large depth, as well. The offset term ϵ\epsilon ensures a minimum separation for shallow and biologically distinct nodes, in which these cases are easily distinguishable by the model. The resulting triplet loss function is formulated as follows.

LTri=max⁡(0,∥za−zp∥22−∥za−zn∥22+α(ca,cn)).(4)\mathcal{L}_{\mathrm{Tri}}=\max\left(0,\left\lVert z_a-z_p\right\rVert_2^2-\left\lVert z_a-z_n\right\rVert_2^2+\alpha(c_a,c_n)\right). \tag*{(4)}

Consequently, this design forces the model to learn more discriminative representations that capture subtle biological distinctions, i.e., CD4⁺ T cells and CD8⁺ T cells, relative to naive T cells.

Global Cell-type Ontology Alignment Although the dynamic margin triplet loss effectively enforces local discrimination—ensuring that fine-grained, semantically similar subtypes are pushed apart to prevent collapse—, however, it does not inherently preserve the broader cell-type hierarchical lineage in a mini-batch. Here, we introduce a complementary objective that aligns the geometry of the embedding space with the global hierarchical relationships encoded by the cell. Concretely, we propose a Rank-Order Alignment Loss that encourages consistency between (i) pairwise distances in the learned embedding space, and (ii) reference ontological distances, by matching their relative (rank) order (see Figure 1(c) lower panel).

Let Z∈RB×d\mathbf{Z}\in\mathbb{R}^{B\times d} denote cell embeddings in a mini-batch, where BB is the batch size and dd is the embedding dimension, which we set to d=256d=256 in our case. We firstly compute the pairwise cosine distance matrix in the embedding space in the current batch: Δemb=1−ZZT∥Z∥2∥Z∥2T\Delta^{\mathrm{emb}}=\mathbf{1}-\frac{\mathbf{Z}\mathbf{Z}^{\mathsf{T}}}{\lVert\mathbf{Z}\rVert_2\lVert\mathbf{Z}\rVert_2^{\mathsf{T}}}. Following this, we get the embedding distance matrix Δemb∈RB×B\Delta^{\mathrm{emb}}\in\mathbb{R}^{B\times B} in the batch. Regarding the reference ontology distances, recall that Δ~\widetilde{\Delta} is the normalized ontology LCA distance, here we denote Δ~onto∈RB×B\widetilde{\Delta}^{\mathrm{onto}}\in\mathbb{R}^{B\times B} as the corresponding LCA ontology distance sub-matrix in the current batch. Let Δnemb\Delta_n^{\mathrm{emb}} and Δ~nonto\widetilde{\Delta}_n^{\mathrm{onto}} be the vectors obtained from the nn-th row of Δemb\Delta^{\mathrm{emb}}, and Δ~onto\widetilde{\Delta}^{\mathrm{onto}}, respectively. Our goal is to make the ranking of neighbors induced by Δnemb\Delta_n^{\mathrm{emb}} match the ranking induced by Δ~nonto\widetilde{\Delta}_n^{\mathrm{onto}}, i.e., to maximize the row-wise Spearman rank correlation between the two distance vectors. Since the Spearman correlation involves a non-differentiable sorting/ranking operation, we adopt the differentiable soft-ranking operator [6]. Specifically, we compute soft rank vectors:

rnemb=Rsoft(Δnemb),rnonto=Rsoft(Δ~nonto),\mathbf{r}^{\mathrm{emb}}_n = \mathcal{R}_{\mathrm{soft}}\left(\Delta^{\mathrm{emb}}_n\right), \quad\mathbf{r}^{\mathrm{onto}}_n = \mathcal{R}_{\mathrm{soft}}\left(\widetilde{\Delta}^{\mathrm{onto}}_n\right),

where Rsoft(⋅)\mathcal{R}_{\mathrm{soft}}(\cdot) denotes the differentiable ranking operator. We then define the global ontology alignment objective as follows.

LontoAlign=1−1B∑n=1BρS(rnemb,rnonto),(5)\mathcal{L}_{\mathrm{ontoAlign}} = 1 - \frac{1}{B} \sum_{n=1}^{B} \rho_S\left(\mathbf{r}^{\mathrm{emb}}_n, \mathbf{r}^{\mathrm{onto}}_n\right), \tag*{(5)}

where ρS(⋅,⋅)\rho_S(\cdot,\cdot) is the Spearman rank correlation. Minimizing LontoAlign\mathcal{L}_{\mathrm{ontoAlign}} encourages the learned embedding space to preserve, for each cell, the relative ordering of ontological distances within each batch, thus aligning the global embedding structure with the hierarchical relationships defined by the ontology.

Complete Objective. The complete loss function is a weighted sum of the three terms in (2), (4) and (5): LTotal=LMLM+λ1LTri+λ2LontoAlign\mathcal{L}_{\mathrm{Total}} = \mathcal{L}_{\mathrm{MLM}} + \lambda_1 \mathcal{L}_{\mathrm{Tri}} + \lambda_2 \mathcal{L}_{\mathrm{ontoAlign}}, where λ1\lambda_1 and λ2\lambda_2 control the contribution of local and global ontology structure preserving losses, respectively. For simplicity, we set λ1=λ2=1\lambda_1 = \lambda_2 = 1 in all of our experiments. The dynamic triplet margin loss enables us to discriminate subtle cell-state differences, while the ontology alignment loss imposes a global hierarchical structure on the learned representation to preserve biological fidelity. The trade-off and synergy between these complementary objectives are explicitly analyzed in the ablation study in Section 4.

Experiments

Pre-Training

Pretrained Dataset Construction. We downloaded 234 Homo sapiens scRNA-seq datasets from the CZI [1] data portal (2025-01-30 release), covering 210 tissues, 566 cell types, 58 disease types in the pretraining phase (see Figure 1(a)). All datasets are from Homo Sapiens and we retained only those sequencing using the 10X platform. We filtered out low-quality cells with fewer than 200 detected genes and genes expressed in fewer than 5 cells, and excluded malignant cells, non-primary samples, and cells whose annotated cell types are not present in the Cell Ontology graph. After such filtering, approximately 27.5 million cells remained.

Pretrained Model Architecture and Configurations. We adopt the same BERT-style architecture as scCello [54] and Geneformer [45], with six Transformer layers and four attention heads per layer, a 512-dimensional intermediate layer and a 256-dimensional hidden size. We pretrain the model using AdamW (learning rate 10−310^{-3}, weight decay 0.001) with a warm-up of approximately 1% of the total training steps. Training is performed for three epochs on six NVIDIA H200 GPUs with a batch size of 64 per GPU and a gradient accumulation step of 4 (effective batch size = 64 × number of GPUs × 4); the complete training process takes approximately 30 hours. Additional training details and hyperparameter settings are provided in Appendix § 8.3.

Downstream Task Settings

In the following, we describe the state-of-the-art (SOTA) baselines and provide details on the datasets and evaluation metrics used for each downstream task. Additional information on the baselines and evaluation metrics is provided in Appendix § 8.5 and Appendix § 8.§ 14.

Benchmark baselines. We benchmark our model against five SOTA scFMs, namely, Geneformer [45], scGPT [10], scCello [54], SCimilarity [24] and CellPLM [49], on four downstream tasks spanning both cell- and gene-level evaluations. The illustration of the downstream tasks considered is shown in Figure 1(d).

Cell-type Annotation. To benchmark zero-shot cell-type clustering, we evaluate on nine datasets excluded from pretraining, spanning blood [19], heart [26], liver [15], kidney [28], pancreas [4], lung [27], breast [20], adipose tissue [16], and bone marrow [55], with sizes ranging from 30k to 348k cells (Appendix § 8.§ 12.2). Specifically, the HHA (blood), kidney and Azimuth PBMC [23] datasets are selected for model robustness analysis in cell-type annotation, as they provide author-annotated labels spanning multiple levels of granularity, from coarse to fine. We further construct an evaluation set of 340 overlapping cell types from 212 unseen datasets (ATable 11). Performance is assessed using scIB metrics [32], including ARI, NMI, and ASWlabel_{label}, and their arithmetic mean (AvgBio), along with ASWlabel/batch_{label/batch} and GraphConn, and their mean (AvgBatch). The overall scIB score is calculated as a weighted average of these two aggregated metrics (0.6×0.6 \timesAvgBio +0.4×+ 0.4 \timesAvgBatch). We also report scGraph-RWR and scGraph-LCA [47, 50] to evaluate the preservation of biological structure captured by model embeddings. For cell-type annotation fine-tuning (Appendix § 8.§ 12.3), we use the pancreas and HIHA datasets, reporting accuracy and macro-F1 for the predicted labels. Moreover, we compare with hierarchical cross-entropy (HCE) classifiers [11], using the authors’ released MLP and TabNet checkpoints.

Novel Cell Identification We use the same 212 unseen datasets as previously mentioned in the cell-type annotation task. However, here we focus on cell types that are absent from pretraining. After filtering rare types (<2,000<2{,}000 cells) and subsampling abundant ones (>10,000>10{,}000 cells), the benchmark contains 99 novel cell types and 774,215 cells (ATable 13, Appendix § 8.§ 12.4). Performance is measured using per-cell-type accuracy and macro-accuracy to account for class imbalance.

Gene Function Prediction We evaluate both static and contextual gene embeddings. For static embeddings, Gene Ontology (GO) gene sets from MSigDB [31] are grouped into Biological Process (BP), Cellular Component (CC), and Molecular Function (MF), and evaluated separately. For contextual embeddings, we conduct a case study on lung adenocarcinoma [27] using decoupleR [3]. Details are provided in Appendix § 8.§ 12.5. We report micro/macro precision and micro/macro F1 scores.

Cross-modal Spatial Gene Imputation We evaluate spatial gene imputation using scRNA-seq references from lung cancer [27], breast cancer [51], and brain tissue from SEA-AD [17], paired with corresponding spatial datasets. Additional details are summarized in Appendix § 8.§ 13. Performance is quantified using Pearson, Spearman, and Kendall’s τ\tau correlations between imputed and ground-truth spatial expression [35].

Results

We first examine the relationship between cell ontology distance and transcriptomic distance to assess the biological motivation for ontology-guided learning. We then evaluate scOntoFM on cell-level tasks, including cell-type annotation and novel cell-type identification, followed by gene-level tasks, namely, gene function prediction and spatial gene imputation. Finally, we analyze the contributions of the training objectives through ablation studies. For all tables reported in the following, bold indicates the best performance and <u>underlining</u> denotes the second-best results.

Relationship Between Ontology and Transcriptomic Distances To examine the relationship between ontology proximity and transcriptional similarity, we compared LCA distances with the pairwise cosine distances between cell-type centroids in PCA space, using Tabula Sapiens v2 and the nine datasets used for zero-shot clustering. As shown in Figure 2, the mean transcriptional distance between cell types increased monotonically over the first ten LCA distance levels (Pearson r=0.91r = 0.91, Spearman ρ=1\rho= 1), then plateaued (AFigure 9). Consistent with this trend, we observed positive correlations across all nine zero-shot clustering datasets, with Pearson rr ranging from 0.31 to 0.58 and Spearman ρ\rho from 0.27 to 0.60 (Table 1). Together, these results indicate that ontology-near cell types tend, on average, to be more transcriptionally similar, supporting LCA distance as an informative structural prior, while its limitations and more details are discussed in Appendix § 8.2.

Figure 2. Cell-type pairwise transcriptomic distance in PCA space across ten LCA ontology levels in Tabula Sapiens v2. Boxes show the distance distributions, and the red line denotes the per-level mean.

DatasetPearson rrSpearman ρ\rho# Cell types
HIHA0.580.6038
Heart0.470.4312
Lung0.510.5510
Liver0.370.2918
Kidney0.310.2733
Pancreas0.380.3012
Breast TNBC0.540.316
Adipose tissue0.350.2816
Bone marrow0.390.3640
Overall0.410.40130

Table 1. Correlation between LCA distance and cell-type centroids cosine distance in PCA space over the nine zero-shot clustering datasets. “Overall” pools pairs across all the datasets.

Cell Type Annotation (Zero-shot Clustering) We first evaluate the performance of our model with SOTA methods across nine datasets of diverse tissues. As shown in the radar plot of Figure 3 and ATables 16 and 17, scOntoFM achieves a mean scIB score comparable to SCimilarity (0.7958 vs. 0.7960) and above the remaining baselines. We then investigate performance using biologically structure preserving metrics (scGraph-LCA and scGraph-RWR) in Figure 3 right panel, ATables 18 and 19. Notably, our model achieves substantially higher performance than SOTA methods on these two metrics across most tissues, highlighting the complementary nature of biological conservation versus scIB metrics. To further illustrate this, we examined the cell-type embedding distance heatmap of Bone Marrow tissue in AFigure 11(c), which aligns well with the LCA-based ontology distance heatmap as well as the RWR-based ontology distance heatmap (AFigure 12). Specifically, we investigate the hematopoietic stem cell and mark it as the rooted cell type in the distance umap plot (see AFigure 11(a,b,d)), which precisely illustrates the developmental lineage of hematopoietic and myeloid-related cell types, as well as aligns well with G2M and S phase cell cycle scores reported in the original study [55].

Figure 3. Radar plots of zero-shot clustering benchmark across nine tissues in scIB (left) and scGraph-LCA (Spearman correlation) (right) metrics across six SOTA methods.

We further evaluate the robustness of our model embeddings (Table 2) under varying annotation granularity in HIHA and Kidney dataset, which contain author annotations from coarse to fine-grained labels. We calculate the agreement scores (ARI) (Table 25) and visualize the cell-type annotation flow in sankey plot (Figure 17, Figure 18) between the cell ontology reference and different annotation levels. Leveraging its ontology-aware design, scOntoFM achieves competitive clustering across varying resolutions, particularly at finer annotation levels (e.g., AIFI_L2/L3 and Subclass_L2), indicating its superior ability to capture subtle cellular states. Although performance is comparatively lower on coarse-grained annotations (e.g., AIFI_L1 and Subclass_L1), this is expected due to their weaker alignment with the fine-grained Cell Ontology labels used during pretraining.

Table 2: Evaluation of zero-shot clustering robustness under varying annotation levels using the scIB score in HIHA and Kidney datasets.

ModelHIHAKidney
AIFI_L1AIFI_L2AIFI_L3Subclass_L1Subclass_L2
Ours0.6810.7820.6710.7400.648
Geneformer0.5100.4730.3980.5850.520
scGPT0.6700.6220.5630.7130.631
scCello0.7010.6000.5060.7700.639
cellPLM0.7600.7220.6660.7210.640

Table 2.

DatasetReferenceTarget# Ref Labels# Target LabelsAgreement Score
HIHAcell_typeAIFI_L13890.259
HIHAcell_typeAIFI_L238290.970
HIHAcell_typeAIFI_L338710.830
Kidneycell_typeSubclassLevel133170.933
Kidneycell_typeSubclassLevel233970.407

Table 25. Agreement scores between reference and target annotations across datasets.

Figure 17. From left to right: Sankey plot of cell type annotation flow from cell_type (CZI provided ontology labels) to author annotated labels (from coarse to fine-grained) in HIHA dataset

Figure 18. From left to right: Sankey plot of cell type annotation flow from cell_type (CZI provided ontology labels) to author annotated labels (from coarse to fine-grained) in Kidney dataset

To assess the generalization of our model in real world scenarios, we evaluated 340 cell types present in the pre-training cell-type labels but sourced from entirely new, unseen datasets. Although our method shows comparable performance to cellPLM in standard scIB metrics (Table 20), it outperforms cellPLM by a significant margin in scGraph-LCA. Collectively, these results demonstrate that our embeddings better preserve cell-type ontology structure and exhibit superior biological conservation, a critical aspect often under-emphasized in prior foundation model evaluation.

MethodNMIARIASWAvg BioscGraph-LCA SpearmanscGraph-LCA Kendall’s τ\tauscGraph-LCA PearsonscGraph-RWR SpearmanscGraph-RWR Kendall’s τ\tauscGraph-RWR Pearson
Ours0.78040.34310.50320.54220.51140.38870.56110.09370.05690.4427
Geneformer0.72130.22410.47170.47240.29620.21570.32780.13690.09160.3605
scGPT0.77400.30140.50740.52760.13390.09620.20580.06820.04470.3638
CellPLM0.79150.33340.52430.54980.09800.06870.19330.14360.09620.3980
scCello0.76930.30920.51990.53280.24170.17510.33840.06700.04270.4768

Table 20. Overall performance of zero-shot clustering of 340 in-distribution cell types in unseen datasets across SOTA methods using scIB metrics, scGraph-LCA, and scGraph-RWR. Best result per column is shown in bold.

Cell Type Annotation (Fine-tuning) We conduct cell type classification benchmarks on the HIHA and Pancreas datasets, with fixed held-out donors and identical hyperparameters for all the HCE classifiers. Averaged over five random seeds, scOntoFM achieves the best overall accuracy and maintains strong Macro-F1 scores (Table 3), indicating its effectiveness in learning discriminative and generalizable representations for accurate cell type classification beyond zero-shot clustering. To further evaluate our model robustness to label noise, we progressively inject noise into the training annotations of the well-annotated Azimuth PBMC dataset by randomly replacing fine-grained labels with their corresponding coarser categories. scOntoFM demonstrates highly stable and accurate cell annotation even under increasing noise levels (Table 26). When the noise ratio reaches 50%, the model reliably falls back to predicting broader parent categories. This behavior is consistent with biological hierarchies, ensuring structurally coherent predictions despite unreliable fine-grained annotations.

Table 3: Cell-type annotation benchmark (fine-tuning) on HIHA and Pancreas datasets across six SOTA methods.

ModelHIHAPancreas
AccuracyMacro F1AccuracyMacro F1
Ours0.915 ± 0.000.771 ± 0.010.992 ± 0.000.969 ± 0.01
Geneformer0.701 ± 0.010.395 ± 0.010.935 ± 0.000.840 ± 0.01
scGPT0.839 ± 0.000.613 ± 0.020.973 ± 0.000.919 ± 0.01
scCello0.811 ± 0.010.543 ± 0.010.977 ± 0.000.960 ± 0.00
cellPLM0.907 ± 0.000.792 ± 0.010.968 ± 0.000.902 ± 0.01
HCE-TabNet0.910 ± 0.000.710 ± 0.020.980 ± 0.000.900 ± 0.01

Table 3.

NoiseAccuracy (mean ± std)Macro F1 (mean ± std)Parent (mean ± std)Child (mean ± std)Sibling (mean ± std)
0.10.941 ± 0.0010.806 ± 0.0170.001 ± 0.0000.003 ± 0.0010.014 ± 0.000
0.20.940 ± 0.0010.752 ± 0.0130.001 ± 0.0000.003 ± 0.0010.015 ± 0.000
0.30.934 ± 0.0020.691 ± 0.0080.001 ± 0.0000.005 ± 0.0010.015 ± 0.001
0.40.834 ± 0.0360.572 ± 0.0240.001 ± 0.0010.053 ± 0.0210.018 ± 0.010
0.50.150 ± 0.0180.207 ± 0.0250.000 ± 0.0000.552 ± 0.0020.035 ± 0.003

Table 26. Impact of noise proportion on performance and hierarchical error distribution.

Novel Cell Identification

To evaluate the hierarchical representations of known cell types and their generalizability to unseen cell types, we consider a more challenging setting: novel cell-type identification. We leverage the full cell-ontology LCA distance matrix to infer embeddings for unseen cell-type centroids via a weighted average of seen centroid (see Appendix § 8.§ 12.4 for details). First, we examine the biological structure coherence of seen cell types. As visualized in AFigure 13(a, b), the centroid embeddings exhibit distinct separation among major lineages, including endothelial cells, leukocytes, myeloid cells, and hematopoietic oligopotent progenitors, confirming that the model captures biologically meaningful embeddings. Quantitatively, as shown in Figure 4, our method outperforms all SOTA baselines, achieving an overall macro-accuracy of 28.59% across 99 novel cell types (ATable 13). And the per-cell-type rank distribution reveals that our model attains the lowest average rank (2.85) across the 99 unseen types—an improvement of 13.65% over the second-best method, scCello (3.24). This underscores the model’s superior capability in recognizing previously unseen biological states.

Figure 4. Bar plot of macro-accuracy for novel cell identification (left) and violin plot of per-cell type accuracy rank distributions (right); ♦ denotes each method’s average rank.

To thoroughly examine this ability, we selected three unseen cell types from the hematopoietic lineage where scOntoFM achieved high predictive accuracy: megakaryocyte progenitor cells, immature neutrophils, and CD14-positive/CD16-low monocytes (AFigure 13(c, d)). We retrieved the top 5 closest seen cell types for each of these unseen targets from the pretraining dataset and performed co-clustering with the unseen cell types. As shown in AFigure 14(a, b, c), the unseen cell types cluster proximally to their biologically related seen counterparts in the embedding space. Crucially, this embedding proximity is consistent with the LCA distance heatmap (AFigure 14(d)), which validates that the model successfully extrapolates the correct ontological neighborhood for novel cells.

Gene Function Prediction scFMs inherently learn informative gene embeddings from diverse cellular contexts and disease conditions. In this section, we systematically explore the biological knowledge encoded by these embeddings; specifically, we compare the static gene representations learned by scFMs to make functional predictions about GO terms, namely, BP, CC, and MF. As shown in Table 4, our method achieves the highest classification performance for all baselines, achieving top scores in both Micro-F1 (0.917±0.001) and Macro-F1 (0.916±0.001) using 20-fold cross validation. This indicates that our model offers a superior balance of precision and recall when assigning functional labels. Furthermore, in terms of Average Precision (AP), our method remains highly competitive, performing on par with Geneformer and scGPT (e.g., Micro AP of 0.921 vs. 0.922) and outperforming scCello. These results demonstrate that our biologically informed training strategy enables the model to capture robust, semantically meaningful gene representations that generalize well to standard functional annotation tasks. We also show GO namespace classification results by aggregating the corresponding gene embeddings for each GO term in ATable 21.

Table 4: Gene function prediction performance across SOTA methods. Classification accuracy (mean ± std) is calculated over 20-fold cross validation, best values are highlighted in bold.

MethodF1 MicroF1 MacroAP MicroAP Macro
Ours0.917 ± 0.0010.916 ± 0.0010.921 ± 0.0020.900 ± 0.002
Geneformer0.912 ± 0.0010.911 ± 0.0010.922 ± 0.0020.902 ± 0.003
scCello0.913 ± 0.0000.911 ± 0.0000.916 ± 0.0020.891 ± 0.003
scGPT0.908 ± 0.0010.906 ± 0.0010.922 ± 0.0020.902 ± 0.003

Table 4.

Beyond static representations, we evaluate the model’s capacity to generate context-aware gene embeddings by applying our model on a metastatic lung adenocarcinoma dataset [27]. We focus on the GLYCOLYSIS hallmark gene set to visualize how the model captures pathway activity across different cellular contexts, and we also visualize the enrichment of score of COAGULATION and ALLOGRAFT_REJECTION hallmarks in AFigure 15. As shown in Figure 5, the enrichment scores for the GLYCOLYSIS hallmark are notably elevated in malignant cell populations, as well as cancer-associated fibroblasts (CAFs), and tumor-associated macrophages (TAMs). This pattern reflects the high proliferative and metastatic activity of tumor cells and the immunosuppressive phenotype of associated immune cells within the tumor microenvironment [36].

Figure 5. scOntoFM-derived cell-type clustering (Left) in lung cancer and the enrichment heatmap (Right) for the GLYCOLYSIS hallmark gene set.

Cross-modal Spatial Gene Imputation High-resolution spatial transcriptomics technologies are often constrained by the limited number of detectable genes. To address this challenge, we extend our framework to a cross-modal setting that leverages single-cell reference data to impute unmeasured genes in spatial datasets, a critical task to enhance spatial resolution [30, 21]. As shown in Table 5, our model consistently outperforms all baseline methods across all evaluation metrics. In particular, we achieve the highest Pearson, Spearman, and Kendall’s τ\tau correlations in both the Lung and Breast cancer dataset, outperforming the nearest rival (Geneformer). We further evaluate our method on an independent SEA-AD brain spatial transcriptomics dataset (ATable 22, [16]). Despite being surpassed only by CellPLM which was trained on spatial data, our model consistently outperforms other single-modality baselines, demonstrating scOntoFM’s strong cross-modal generalization by recovering meaningful spatial patterns.

Table 5: Spatial gene imputation performance on Lung and Breast cancer datasets. Pearson, Spearman, and Kendall’s τ\tau correlation are calculated over 10-fold cross validation on held-out gene sets, best values are in bold.

Lung cancerBreast cancer
MethodPearsonSpearmanKendall’s τ\tauPearsonSpearmanKendall’s τ\tau
Ours0.319 ± 0.2040.282 ± 0.1950.223 ± 0.1480.388 ± 0.1980.369 ± 0.1980.279 ± 0.147
Geneformer0.314 ± 0.2040.275 ± 0.1980.218 ± 0.1500.369 ± 0.1840.357 ± 0.1980.270 ± 0.147
CellPLM0.272 ± 0.1900.225 ± 0.1680.179 ± 0.1280.152 ± 0.1070.125 ± 0.0970.093 ± 0.072
scCello0.274 ± 0.1870.233 ± 0.1750.185 ± 0.1310.105 ± 0.0980.101 ± 0.0920.075 ± 0.070
scGPT0.308 ± 0.2010.255 ± 0.1910.202 ± 0.1440.290 ± 0.1760.271 ± 0.1680.201 ± 0.121

Table 5.

Ablation Study We conduct an ablation study to assess the contributions of the MLM loss, dynamic margin ranking loss, and ontology alignment (Figure 6, ATable 23, ATable 24) in the zero-shot clustering task across nine tissues. To ensure fair comparison, all ablated variants are pretrained on the same 27.5M-cell corpus as scOntoFM. Removing ontology alignment improves scIB performance but leads to a substantial degradation in scGraph-LCA, indicating weakened ontology consistency. In contrast, excluding the dynamic margin ranking loss enhances scGraph-LCA but slightly reduces scIB, suggesting complementary roles of these two components. Although individual components favor specific metrics, the full model (scOntoFM) achieves a balanced trade-off, simultaneously maintaining strong scIB performance and substantially improving scGraph-LCA. These results demonstrate that jointly optimizing dynamic margin ranking and ontology alignment is essential to balancing integration quality and ontology-aware structural consistency.

Figure 6. Pareto analysis of scOntoFM loss components in scIB and scGraph-LCA. Highlighted points are the mean across nine tissues.

Conclusion

We presented scOntoFM, a novel foundation model that integrates hierarchical cell ontologies directly into the pretraining framework. Through the use of efficient offline triplet sampling and a dynamic, hierarchy-aware loss function, our approach addresses key limitations in existing methods regarding biological interpretability. Our results show that scOntoFM not only achieves SOTA performance across diverse downstream tasks, but also significantly improves the preservation of biological structure, offering a more robust tool for single-cell analysis.

Limitations This work has several limitations. i) scOntoFM depends on the quality of the cell ontology and source annotations: incomplete coverage, uneven granularity, and inconsistent labels can introduce ambiguous supervision and affect the learned representations. ii) triplet sampling relies on ontology relationships without jointly considering transcriptomic similarity. Consequently, it may overlook context-specific variation, as closely related cell types can exhibit distinct expression patterns across tissues, diseases, or developmental stages. iii) pretraining requires substantial computational resources and curated annotations, while the storage requirements of the Cell-Dataset Index Map increase with corpus size. Future work will focus on developing a unified cell embedding space that supports harmonized, ontology-grounded annotations across cell atlases, together with covariate-conditioned sampling. These extensions aim to preserve a shared global structure while capturing context-specific variation and disease- and development-associated changes in cell state.

Related Work

Existing single-cell foundation models (scFMs) can be broadly categorized into three groups based on their tokenization and data processing strategies [44]. The first group [45, 54, 7], exemplified by Geneformer, employs rank-based tokenization, where genes are ordered by expression level to form sequence-like inputs for Masked Language Modeling (MLM). The second group [52, 10], (e.g., scGPT) extends this by discretizing gene expression values into consecutive bins, hence enabling joint learning of gene and cell representations. The third group [49, 22], known as value projection methods (e.g., CellPLM, scFoundation), bypasses discrete tokenization by projecting raw expression counts directly into a continuous embedding space. Other recent studies [9, 29, 38], have explored multi-modal learning to align single-cell data with natural language, thus leveraging the power of large language models; these methods demonstrate great potential for downstream biological discovery.

Hierarchy-aware representation learning has a long history outside single-cell biology. Hierarchical triplet loss [18] and dynamic metric learning [43] adapt margins to a class taxonomy; Bertinetto et al. [5] show that LCA-height-weighted losses produce "better mistakes" on ImageNet; and Shi et al. [39] find that LCA-based taxonomic distance predicts out-of-distribution generalization across vision models. Within single-cell biology, OnClass [48] propagates labels over the Cell Ontology to classify unseen types, treeArches [33] progressively builds and extends cell atlas according to the hierarchies of cell types, UCE [37] evaluates embeddings against ontology-derived distances, and hierarchical cross-entropy [11] trains annotation models against the ontology tree. scOntoFM draws on both lines: it uses LCA distance both to mine triplets offline and as the target of a rank-order alignment loss during foundation-model pre-training.

In terms of incorporating cell ontology into model pretraining, Heimberg et al. [24] proposed SCimilarity, a deep metric learning–based foundation model that combines a standard triplet loss with a reconstruction objective to embed single-cell RNA-sequencing data into a unified space. Specifically, it leverages cell ontology information to guide the sampling of semi-hard negative triplets within mini-batches. Along a similar line, scCello [54] employs the Personalized PageRank (PPR) algorithm to quantify structural similarity between nodes in the cell ontology graph, thereby guiding the model to learn pairwise cell-type distance relationships that aligns with PPR structure.

Unlike previous approaches, we proposes a more coherent and unified framework that integrates ontology information not only into the triplet sampling stage but also into biologically constrained representation learning throughout pre-training. Concretely, compared to scCello and SCimilarity, scOntoFM enjoys special novelty as follows: i) Efficient offline ontology-aware triplet sampling, which mines more informative positive and negative pairs and facilitate effective contrastive learning. While standard online triplet sampling, like in scCello and SCimilarity, is highly inefficient for massive datasets and the probability of encountering informative positive pairs is low in a random mini-batch (see Figure 7). ii) The ontology-aware dynamic margin and global ontology alignment loss terms enables the model capture more fine-grained cell type information (see Table 2), while also maintains a faithful biological structure versus scCello (see Figure 3 right panel, AFigure 11, 12). iii) Comprehensive evaluation on both cell- and gene level task. While scCello mainly concentrates on cell level tasks, scOntoFM provides a more dedicated evaluation for scFMs in gene-level tasks (see Section 4).

Figure 7. Distribution of positives per sample under online random sampling in 10% of our pretraining dataset. Pink bars show the absolute frequency of positive samples, and purple bars show the corresponding proportions.

EXPERIMENTS

Online-triplet sampling

In Figure 7, we show an experiment of online triplet sampling for 10% of our pre-training data. Around 46.78% of them have zero positives, and 21.03% only have one positives, indicating the highly in-efficiency for online triplet sampling in large data corpus.

Relationship between LCA distance and transcriptomic similarity

To test the premise that proximity in the cell ontology reflects transcriptomic proximity, we computed per cell-type transcriptomic centroids in PCA space by averaging the pre-processed gene expression through Scanpy pipeline (library size normalization, log-transform, with highly variable genes set to 2000, and principle components set to 50) over all cells belonging to that type, and measured the cosine distance between every pair of cell-type profiles. We compared these transcriptomic distances to the raw LCA distances Δi,j\Delta_{i,j} from 1 in Tabula Sapiens v2 dataset (173 cell types across 28 organs) and the nine diverse tissue datasets from our zero-shot clustering evaluation.

Correlation Analysis on Tabula Sapiens v2. As shown in Table 6 and Figure 2, grouping cell pairs by the first 10 LCA distance reveals a strong positive correlation and shows a strictly monotone increase in mean expression distance (Pearson r=0.91r = 0.91, p<0.001p < 0.001). The within-level standard deviation shrinks with LCA distance: close relatives are heterogeneous (some sibling types are transcriptionally far apart), whereas distant types are uniformly dissimilar. This broad, multi-organ analysis supports the biological premise that ontology-near cell types tend, on average, to be more transcriptionally similar, which also aligns with findings in OnClass [48] and UCE [37].

LCA distance12345678910
Mean Expr. dist. ± sd0.55±0.370.72±0.330.82±0.300.91±0.230.96±0.200.99±0.171.00±0.161.02±0.151.04±0.131.06±0.12

Table 6. The relationship between ontology LCA distance and transcriptomic distance in Tabula Sapiens v2.

Generalization Across Nine Zero-shot Clustering Datasets. As showed in Table 1 and AFigure 8, we observed a consistent positive correlation in every tissue, with an overall Pearson r=0.41r = 0.41 (p<0.001p < 0.001) across 130 unique cell types. Correlations are strongest in immune-rich tissues (HIHA, Lung, Breast TNBC), where the ontology is deepest, and weakest in Kidney, Adipose and Liver, where many annotated types are shallow epithelial or stromal nodes.

Discordant pairs. Plotting pre-trained global cell-type centroid distances against Δ~i,j\widetilde{\Delta}_{i,j} (Figure 10) isolates two kinds of off-diagonal pairs. (i) Ontology-far, transcriptome-close: almost entirely functional T-cell subsets that the ontology separates but the transcriptome does not (e.g., CD8+ cytotoxic vs. CD4+ cytotoxic T cells; γδ intra-epithelial vs. effector-memory CD8+ T cells), plus cross-tissue mesenchyme (ileal smooth muscle vs. kidney fibroblast). (ii) Ontology-close, transcriptome-far: dominated by generic parent terms (“professional antigen presenting cell”, “connective tissue cell”) that sit one edge from many children yet are transcriptionally far from all of them. The ontology is thus an informative prior rather than a complete model of expression; the MLM objective keeps the embedding faithful to the transcriptome in exactly these regions, which is why the full model (not the ontology-only ablations) gives the best balance in Figure 6.

Figure 10. Scatter plot of the pairwise transcriptomic cell-type centroid distance versus the normalized LCA distance Δ~i,j\widetilde{\Delta}_{i,j}. The off-diagonal points are highlighted with red and blue.

Pre-trained Model Configuration (Table 7)

The pre-trained model backbone, architecture, and the training details are listed in Table 7.

ComponentConfiguration
Backbone ModelBERT (BertForMaskedLM)
BERT ArchitectureVocal size: 19941<br>Hidden size: 256<br>Layers: 6<br>Attention heads: 4<br>Intermediate size: 512<br>Activation: ReLU<br>Max position embeddings: 2002
RegularizationHidden dropout: 0.02<br>Attention dropout: 0.02<br>LayerNorm ϵ\epsilon: 1×10−121 \times10^{-12}
OptimizerAdamW (lr =1×10−3= 1 \times10^{-3}, weight decay =0.001= 0.001)
Learning Rate ScheduleWarmupDecayLR<br>Warmup steps: 1% of total training steps
Distributed TrainingDeepSpeed with ZeRO stage 2<br>Overlap communication enabled<br>Reduce / allgather bucket size: 5×1065 \times10^{6}<br>Contiguous gradients enabled
PrecisionFP16 mixed precision
Training HyperparametersBatch size per GPU: 64<br>Gradient accumulation steps: 4<br>Effective batch size: 64×64 \times world size ×4\times4<br>Epochs: 3<br>Gradient clipping: 1.0

Table 7. Model architecture and training configuration.

Computation cost of the Cell-Dataset Index Map (Ω\Omega)

We construct the Cell-Dataset Index Map by extracting specific columns (cell_type_ontology_term_id, dataset_id) from the Hugging Face pre-training dataset and applying pandas’ groupby operation. The process takes about 5 minutes and uses 6.5 GB of peak memory on an Intel Xeon Gold 6252 CPU. The final file size is 132 MB, ensuring that scalability to a larger corpus is highly feasible.

Figure 8. Scatter plots of LCA distance versus cell-type pairwise transcriptomic distance for each of the nine tissues.

Baselines

Geneformer. We followed the official guideline of Geneformer [45] and downloaded the pre-training weights of Geneformer-V1-10M on https://huggingface.co/ctheodoris/Geneformer/tree/main.

scGPT. We follow the tutorial of running scGPT [10] from https://scgpt.readthedocs.io/en/latest/ and downloaded the whole-human model checkpoint (https://drive.google.com/drive/folders/1oWh_-ZRdhtoGQ2Fw24HP41FgLoomVo-y?usp=sharing) as indicated in its github repository.

scCello. The pre-trained zero-shot scCello [54] model weights are downloaded from hugging face with following link: https://huggingface.co/katarinayuan/scCello-zeroshot.

CellPLM. We refer the official github repository of CellPLM [49] (https://github.com/OmicsML/CellPLM) and download its latest version checkpoint: 20230926_85M from https://github.com/OmicsML/CellPLM/tree/main/ckpt.

SCimilarity. We refer the official github repository of SCimilarity [24] (https://github.com/Genentech/scimilarity) and download its latest model v_1.1 from https://zenodo.org/records/10685499.

To provide a clear overview of each baseline, in the following Table 9, we summarize and compare models in terms of pre-training data, parameter size, and training strategy.

ModelPretraining Data Size# ParametersModeling Strategy
geneformer30M10,316,196MLM, no ontology
scGPT33M51,330,049Auto-regressive, no ontology
cellPLM11.4M82,402,543VAE
scCello22M10,683,654MLM + ontology
SCimilarity7.9M62,332,627Autoencoder + ontology
scOntoFM27.5M8,846,848MLM + ontology

Table 9. Model comparison in terms of pretraining data size, model scale, and modeling strategy.

Figure 9. Cell-type pairwise transcriptomic distance in PCA space across raw LCA ontology distance over 1 to 16 levels in Tabula Sapiens v2. Boxes show the distance distributions, and the red line denotes the per-level mean.

Zero-shot Clustering Datasets Details

For the zero-shot evaluation datasets summarized in Table 10, we retained only primary cells from the original datasets, and no additional filtering was applied. Each data sources are listed down below in the table.

Dataset# CellsSource
Blood (HIHA)263,917https://cellxgene.cziscience.com/e/e522d2cd-7927-4e59-a4ed-064009569279.cxg/
Heart211,061https://cellxgene.cziscience.com/e/d567b692-c374-4628-a508-8008f6778f22.cxg/
Liver72,710https://cellxgene.cziscience.com/e/84cfa5aa-a0d1-415e-ac47-eb66c9cf8170.cxg/
Kidney (Atlas V2)348,984https://cellxgene.cziscience.com/collections/9c9d04c4-8899-417f-bb6f-6107dcadf14f
Pancreas245,878https://cellxgene.cziscience.com/e/e51bae9a-c747-4b64-904a-4da7cda218ab.cxg/
Breast TNBC52,681https://cellxgene.cziscience.com/e/9a64bf99-ebe5-4276-93a8-bee9dff1cd47.cxg/
Lung30,472https://www.ncbi.nlm.nih.gov/geo/query/acc.cgi?acc=GSE131907
Adipose Tissue166,149https://cellxgene.cziscience.com/e/ca46ffe6-0a26-4d8d-82fa-81722daf1102.cxg/
Bone Marrow263,159https://cellxgene.cziscience.com/collections/f6c50495-3361-40ed-a819-fb9644396ed9

Table 10. Summary of zero-shot clustering evaluation datasets used in this study.

Unseen Evaluation Datasets In Table 11, we show the list of query dataset ids from CZI data portal.

Dataset IDDataset IDDataset ID
99e317c0-1fab-4f73-b511-859e752fd1c36725ee8e-ef5b-4e68-8901-61bd14a1fe73e2808a6e-e2ea-41b9-b38c-4a08f1677f02
98f5d518-1e60-4307-b732-ab045cd3cbc2bab7432a-5cfe-45ea-928c-422d03c45cdd6c87755e-a671-41a8-9c7e-4e43b850a57b
e6361237-ac4e-4c5d-ad8f-f16aca0c0a8f242c6e7f-9016-4048-af70-d631f5eea188d5a2d011-3765-4dad-ac36-3fff087589fe
43245158-5ae1-4e71-a9a6-67eef49c26bca65bcc2d-4243-44c1-a262-ab7dcddfcf864e374783-0152-423e-bcb6-b87cc232f64d
5e7fff82-b8eb-4220-bf65-c6feb6765ea18d1dd010-5cbc-43fb-83f8-e0de8e8517dab252b015-b488-4d5c-b16e-968c13e48a2c
72822932-10f6-466f-baf3-a2c1d89364bc881fe679-c6e0-45a3-9427-c4e81be6921f4dd1cd23-fc4d-4fd1-9709-602540f3ca6f
79ef1959-a6b4-4cac-82ca-30feaec48df101209dce-3575-4bed-b1df-129f57fbc03130f5e171-83d7-4fc0-bf75-384f122346b3
5d3fc988-765d-48ba-bfb5-151ef2988cac31285bcd-9a28-48ce-8cc0-543af80261d3dc30c3ec-46d6-4cd8-8ec1-b544a3d0f503
3c361813-5d5e-4868-bfe5-e7591e7b5381234c23d0-d6cd-4612-a40d-e5811727564b76150f40-1989-4977-9e23-696e72d59d9e
171b03c9-516b-499c-a434-fa4f7019198813149914-ea03-4f01-8bf6-b793b667127bdbf0bd35-87f8-4b25-bc90-a3c54f379907
d7f5d8d0-6150-48d7-b094-c34286ad11a1f8d8b443-bca6-4c3c-9042-669dfb7f8030eeacb0c1-2217-4cf6-b8ce-1f0fedf1b569
3e524d45-2265-4440-b903-3c30d51a0b17a20e2f2a-e171-450a-8ddc-8935c2977ad61fc48734-35ee-4372-885c-95fd2674c75a
731e6380-879f-4b0b-9a1f-2150208852efab326369-b63c-48d8-8a7f-82ffa0df70028a554710-08bc-4005-87cd-da9675bdc2e7
0c9a8cfb-6649-4d52-b418-6d8e56bd7afe1a974af7-8b4b-4abb-8aba-23e2979d9518d18736c3-6292-4379-919a-d6d973204c87
9ea52f85-a35e-4b5b-9207-e2f1de7dbbcd87a06a2a-16c6-42f7-af1b-529fad43105151d9716c-50bb-4fd8-8d1e-7d9329969cb7
7c497c66-c2e2-4291-8b96-339e5e65f314ef1a4b05-540c-4f73-8ceb-49dfa800645dbd3e3487-c1e6-4298-b9de-8bdcf3f938d7
14363b6e-3428-45b0-8704-10fd5d6316df17e9d436-a264-4c94-a42d-48c8daf6fdd9e34f3f89-3a48-4173-a06a-9e9803f2e2b9
31f657dc-1875-4c4b-a5ca-ce63b3ef3a823079e9b0-c6db-45c8-b998-a4555a73968ed9f3a82c-24bb-4277-b715-8ec92a33dd32
2856d06c-0ff9-4e01-bfc9-202b74d0b60f74014ef8-d2d0-4cbc-8ba2-037f30753ffdc893ddc3-f25b-45e2-8c9e-155918b4261c
6f7fd0f1-a2ed-4ff1-80d3-33dde731cbc3e500acbf-f166-4c46-b290-f93506cf26d3a4157949-6f2a-40e2-b960-63f6e2bde918
0de831e0-a525-4ec5-b717-df56f2de2bf093091496-be48-4122-b945-9af9c22a753573b7d0aa-035f-4a15-9f1d-f56b70204591
492d9f77-c4c8-485d-92d7-7686fe6c8482f7ec7bd5-04ab-453b-a8a7-c9d1

Table 11. CZI query dataset id for the 212 unseen evaluation datasets.

Cell Type Annotation Fine-tuning

We peformed cell type classification on the HIHA and Pancreas datasets, with fixed held-out testing donors (UP1008 and UP1012 in HIHA; SAMN16547514, SAMN17209843, SAMN17528599, SAMN17831932 in Pancreas) and identical hyperparameters across all scFMs. To ensure a fair comparison, we utilized a standardized Multi-Layer Perceptron (MLP) head classifier for all foundational models. The network takes the extracted 2D continuous cell embeddings as input and processes them through two hidden layers (1024 and 512 dimensions), each followed by 1D Batch Normalization, a ReLU activation, and Dropout regularization (rate of 0.2) to mitigate overfitting. The model was optimized to minimize Cross-Entropy Loss using the AdamW optimizer with a learning rate of 1×10−31 \times10^{-3} and weight decay of 1×10−41 \times10^{-4}, while reserving 10% of the training donor cells as a validation set for early stopping. We also compared with a specialized cell annotation model with hierarchical cross-entropy (HCE) loss [11], using the authors’ released MLP and TabNet checkpoints and their penultimate-layer embeddings, the same donor splits, and the same MLP probe. The results (Table 3, ATable 12) are reported as mean±std over 5 random seeds and confirm that scOntoFM also maintains its significant performance advantage during fine-tuning stage.

Novel Cell Identification

This approach assumes that the biological relationships defined in a Cell Ontology (hierarchical closeness) translate directly to geometric closeness in the cell representation space. Cell types that are closely related in the Cell Ontology (e.g., "CD4+ T cell" and "CD8+ T cell") share significant transcriptomic programs. Therefore, the embedding (centroid) of an unseen cell type can be reliably approximated as a weighted interpolation of its known evolutionary or functional relatives. The novel cell identification process could be summarized as follows:

1. Compute Ontology Affinity Matrix We define an affinity matrix A∈Rm×n\mathbf{A} \in\mathbb{R}^{m \times n} between unseen and seen cell types using a Gaussian kernel on the LCA distances:

Aij=exp⁡(−γ⋅Δ(ui,cj)),(6)A_{ij} = \exp\left(-\gamma\cdot\Delta(u_i,c_j)\right), \tag*{(6)}

where γ\gamma is a scaling hyperparameter, we set it to 1 by default, and uiu_i is the unknown cell type, and cjc_j is the known cell type. The Gaussian kernel enables us to transform the raw LCA (Lowest Common Ancestor) distances into normalized similarity weights.

2. Normalize Weights We normalize the affinities so that the weights for each unseen cell type sum to 1:

Wij=Aij∑k=1nAik.(7)W_{ij} = \frac{A_{ij}}{\sum_{k=1}^{n} A_{ik}}. \tag*{(7)}

Let W∈Rm×n\mathbf{W} \in\mathbb{R}^{m \times n} be the resulting weight matrix.

3. Generate Novel Prototypes We calculate the predicted centroid p^ui\hat{\mathbf{p}}_{u_i} for each novel cell type by linearly combining the known centroids Q∈Rn×d\mathbf{Q} \in\mathbb{R}^{n \times d}.

P^U=W⋅Q,(8)\hat{\mathbf{P}}_{U} = \mathbf{W} \cdot\mathbf{Q}, \tag*{(8)}

where P^U∈Rm×d\hat{\mathbf{P}}_{U} \in\mathbb{R}^{m \times d} contains the estimated embeddings for the unseen type centroids, U={ui}i=1mU = \{u_i\}_{i=1}^{m} is the novel cell-type set containing all the unknown cell types

4. Classification Given a novel cell and and its corresponding embedding zz, which could be classified by finding the nearest unseen centroid using cosine similarity:

y^=argmax⁡ui∈U(z⋅p^ui∥z∥∥p^ui∥).(9)\hat{y} = \underset{u_i \in\mathcal{U}}{\operatorname{argmax}}\left(\frac{z \cdot\hat{p}_{u_i}}{\lVert z\rVert\lVert\hat{p}_{u_i}\rVert}\right). \tag*{(9)}

Novel Cell-type List A summary of the 99 novel cell types is provided in ATable 13, along with the number of cells for each cell type. For cell types with more than 10,000 cells, we randomly subsample 10,000 cells. In total, the novel cell evaluation dataset contains 774,215 cells.

Gene Function Prediction

Static Gene Embedding

To evaluate the semantic quality of the static gene representations, we perform a supervised multilable classification task to predict Gene Ontology (GO) namespace membership by using the static gene embedding extracted from scFMs.

Problem Formulation Let M={m1,m2,…,m∣M∣}\mathcal{M} = \{m_1, m_2, \ldots, m_{|\mathcal{M}|}\} be the set of genes. For each gene mim_i, we extract a static embedding vector ei∈Rde_i \in\mathbb{R}^{d} directly from the input token embedding table of the pre-trained foundation model (e.g., Geneformer, scCello, or scGPT). This approach evaluates the model’s zero-shot understanding of gene identity without updating the encoder weights. The target labels are defined by the three root namespaces of the Gene Ontology: Biological Process (BP), Cellular Component (CC), and Molecular Function (MF). We define the label vector yi∈{0,1}3y_i \in\{0,1\}^{3} for gene mim_i based on memberships defined in the MSigDB C5 gene sets, where yi,j=1y_{i,j} = 1 if the gene belongs to namespace jj.

Gene Ontology Gene Sets The GO terms are downloaded from MSigDB C5 collections, encompassing 7583 BP components, 1042 CC components, and 1855 MF components, forming 10480 gene sets in total.

Model Architecture We employ a 3-layer Multilayer Perceptron (MLP) as a probing classifier. The model architecture and training details are summarized in Table 14. To ensure a fair comparison, we apply same settings for all the scFMs without specifically tuning the hyper-parameters.

CategorySpecification
Input dimensiondd (embedding dimension)
Hidden layer128 units, ReLU activation
Bottleneck layer64 units, ReLU activation
Output layer3 units (multi-label classification)
Dropout rate0.2 (applied after each hidden layer)
Loss functionBinary Cross-Entropy with Logits
OptimizationAdam optimizer
Learning rate1×10−41 \times10^{-4}
Batch size64
Training epochs15
Task formulationMulti-label classification (C=3C = 3)

Table 14. Summary of the architecture and training configuration for static gene embedding GO function classifier.

Experiments and Evaluation To ensure robust evaluation, we perform cross-validation (set kk folder to 20) with stratified sampling for the train/test splits. Performance is reported as the mean and standard deviation across the 20 splits for Micro-F1, Macro-F1, Micro-Average Precision (AP), and Macro-AP, the evaluation metrics will be introduced in more detail in Appendix § 8.11.4.

Contextual Gene embedding

To interpret the biological signals captured by the foundation model, we perform pathway activity inference using the decoupler package [3] for the lung adenocarcinoma dataset [27]. Unlike standard approaches that utilize raw read counts, we leverage the contextualized gene embedding generated by the model.

Input Data Preparation Let H(i)∈RL×d\mathbf{H}^{(i)} \in\mathbb{R}^{L \times d} be the hidden state representation for cell ii extracted from the last layer of the pretrained Transformer model, where LL is the sequence length (number of genes) and dd is the embedding dimension. We obtain a scalar intensity value EijE_{ij} for the jj-th gene in cell ii by averaging over the embedding dimension:

Eij=1d∑k=1dHijk.(10)E_{ij} = \frac{1}{d} \sum_{k=1}^{d} H_{ijk}. \tag*{(10)}

This operation results in a dense matrix E∈RN×M\mathbf{E} \in\mathbb{R}^{N \times M} representing the mean activation intensity of each gene within its specific cellular context.

Pathway Activity Inference We employ the Univariate Linear Model (ULM) in the decoupler package [3] for activity estimation, and download the prior knowledge network from the MSigDB Hallmark collection2. For each cell ii, ULM fits a linear model to determine if the continuous values in Ei\mathbf{E}_i corresponding to a specific pathway gene set are significantly higher than the background distribution. The resulting matrix Y∈RN×K\mathbf{Y} \in\mathbb{R}^{N \times K} contains zz-scores (enrichment scores) representing the inferred activity of KK biological pathways across NN cells. Similarly, we could easily extend this framework to other available gene sets (i.e., MSigDB C3 regulatory taget gene sets, C5 ontology gene sets, or resources inside Omnipath [46]) and perform biological discovery with our model’s contextual embedding. More information about the decoupler package and the analysis pipeline could be found in the official website3.

Differential Pathway Analysis To identify cell-type-specific functional signatures, a differential activity analysis was performed on the matrix Y\mathbf{Y}, containing the pathway enrichment score for each cell. We group cells by their annotated types c∈Cc \in\mathcal{C}, where C\mathcal{C} is the total cell types in the dataset, and apply a tt-test with overestimated variance:

t=μk,c−μk,C∖cσk,c2nc+σk,C∖c2nC∖c.(11)t = \frac{\mu_{k,c} - \mu_{k,\mathcal{C}\setminus c}}{\sqrt{\frac{\sigma_{k,c}^{2}}{n_c} + \frac{\sigma_{k,\mathcal{C}\setminus c}^{2}}{n_{\mathcal{C}\setminus c}}}}. \tag*{(11)}

where μk,c\mu_{k,c} is the mean activity of pathway kk in cell type cc, and C∖c\mathcal{C} \setminus c denotes all the other types. This identifies pathways that are uniquely activated within specific cell populations based on the foundation model’s learned representations.

Spatial Gene Imputation

We perform reference-based spatial gene imputation to evaluate the biological fidelity of the learned cell in zero-shot setting. This task assumes that single-cell and spatial transcriptomics profiles map to a shared latent manifold where proximity implies biological similarity.

Problem Formulation Let Dst\mathcal{D}_{st} and Dsc\mathcal{D}_{sc} denote the spatial transcriptomics and single-cell reference datasets, respectively. Let M\mathcal{M} be the comprehensive set of genes profiled in the single-cell reference, and M′\mathcal{M}' be the set of genes captured in the spatial data, such that M′⊂M\mathcal{M}' \subset\mathcal{M}.

For an arbitrary sample ii from modality m∈{sc,st}m \in\{sc, st\}, let Hi(m)∈RLi×d\mathbf{H}_{i}^{(m)} \in\mathbb{R}^{L_i \times d} denote the contextualized hidden states extracted from the last layer of the pretrained foundation model, where LiL_i represents the sequence length (number of detected genes) and dd is the embedding dimension. We derive a global semantic embedding hi(m)∈Rd\mathbf{h}_{i}^{(m)} \in\mathbb{R}^{d} for this sample by averaging representations across the gene sequence dimension:

hi(m)=1Li∑t=1LiHi,t,:(m).(12)\mathbf{h}_i^{(m)} = \frac{1}{L_i}\sum_{t=1}^{L_i}\mathbf{H}_{i,t,:}^{(m)}. \tag*{(12)}

Formally, we define the spatial dataset as Dst={(xi(st),hi(st))}i=1N\mathcal{D}_{st} = \{(\mathbf{x}_i^{(st)}, \mathbf{h}_i^{(st)})\}_{i=1}^{N}, where hi(st)∈Rd\mathbf{h}_i^{(st)} \in\mathbb{R}^{d} is the average semantic embedding for a spot and xi(st)∈R∣M′∣\mathbf{x}_i^{(st)} \in\mathbb{R}^{|\mathcal{M}'|} is the raw expression vector. Similarly, the reference dataset is defined as Dsc={(xj(sc),hj(sc))}j=1K\mathcal{D}_{sc} = \{(\mathbf{x}_j^{(sc)}, \mathbf{h}_j^{(sc)})\}_{j=1}^{K}, where xj(sc)∈R∣M∣\mathbf{x}_j^{(sc)} \in\mathbb{R}^{|\mathcal{M}|} represents the comprehensive transcriptomic profile. Our objective is to reconstruct the full gene expression profile (specifically for genes in M∖M′\mathcal{M} \setminus\mathcal{M}' or a hold-out test set) for the spatial spots by leveraging the single-cell reference data.

Imputation Workflow We employ a weighted kk-nearest neighbors approach. Given a spatial spot ii, we would like to find the k-nearest cells (which we define as Nki\mathcal{N}_k^i) in the embedding space, where k we set to 50 in our experiment. Then we perform the imputation for the unseen genes (M∖M′\mathcal{M} \setminus\mathcal{M}') in spatial data using a weighted average of the comprehensive genes (M\mathcal{M}) in single cell. More details about how we calculate the weights, and the algorithm implementations are illustrated as follows:

  1. Cosine Similarity Computation: We compute the cosine similarity score between spatial spot ii and reference single cell jj:

Sij=hist⋅hjsc∥hist∥2∥hjsc∥2.(13)S_{ij} = \frac{\mathbf{h}_i^{st} \cdot\mathbf{h}_j^{sc}}{\lVert\mathbf{h}_i^{st}\rVert_2\lVert\mathbf{h}_j^{sc}\rVert_2}. \tag*{(13)}
  1. Neighboring Weights Normalization: For each spatial spot ii, we identify the set Nki\mathcal{N}_k^i of the top-k most similar reference cells. We compute normalized weights using the softmax function:

wij=exp⁡(Sij)∑m∈Nkiexp⁡(Sim)∀j∈Nki.(14)w_{ij} = \frac{\exp(S_{ij})}{\sum_{m \in\mathcal{N}_k^i}\exp(S_{im})} \quad\forall j \in\mathcal{N}_k^i. \tag*{(14)}
  1. Imputation: The imputed expression vector x^ist∈R∣M∣\hat{\mathbf{x}}_i^{st} \in\mathbb{R}^{|\mathcal{M}|} is calculated as:

x^ist=∑j∈Nkiwijxjsc.(15)\hat{\mathbf{x}}_i^{st} = \sum_{j \in\mathcal{N}_k^i} w_{ij}\mathbf{x}_j^{sc}. \tag*{(15)}

This operation effectively transfers the comprehensive gene information from the reference scRNA-seq dataset to the spatial spots.

Spatial Imputation Datasets The spatial imputation datasets, their paired scRNA-seq data, and the shared common genes for model evaluation, are provided in ATable 15. the single cell reference data of lung cancer and breast cancer were retrieved from Kim et al. [27] and [51] (GEO accession number: GSE131907 and GSE176078 respectively). The scRNA-seq of brain tissue was downloaded from SEA-AD (donor_H21_032) [17]. The corresponding lung-tissue spatial dataset (Lung cancer 2) was obtained from the Vizgen MERFISH FFPE Human Immuno-oncology dataset4, the paired breast cancer was sourced from donor (CID3586) in [51] and the brain tissue spatial data was retrieved for donor_H21_032 in SEA-AD.

Experiments and Evaluation We first retrieve the shared common gene sets (M∩M′\mathcal{M} \cap\mathcal{M}') between single cell and spatial transcriptomics data. Then the shared genes are partitioned into 10-fold, we employ k-fold cross-validation to assess the imputed spatial gene performance for each held-out gene sets. Performance is quantified using Pearson Correlation (rr), Spearman Rank Correlation (ρ\rho), and Kendall’s Tau (τ\tau) between the imputed gene vector x^\hat{\mathbf{x}} and ground truth x\mathbf{x} across all test spatial spots. the evaluation metrics will be introduced in more detail in Appendix § 8.11.5.

Evaluation Metrics

We employ different evaluation metrics depending on the downstream tasks. For zero-shot clustering, we report standard scIB metrics [32]: adjust random index (ARI), normalized mutual information (NMI), average silhouette width (ASWlabel_{label}), and their arithmetic mean AvgBio to quantify the separation of different cluster/cell-types. To assess batch mixing, we use ASWlabel/batch\mathrm{ASW}_{label/batch} and GraphConn and report their mean (AvgBatch). And the overall scIB score is calculated as a weighted average of these two average metric (overall = 0.6× AvgBio + 0.4× AvgBatch). We also report scGraph-RWR [50] and scGraph-LCA. The former uses the Random Walk with Restart (RWR) algorithm to obtain cell-type hierarchical distances on the ontology graph and the latter is adapted from scGraph-RWR by replacing the ontology-RWR reference distance matrix with LCA-based distance matrix. Both metrics quantify graph-level similarity between the reference ontology distance matrix and the cell-type centroid embedding distance matrix, providing a complementary evaluation of the intrinsic biological structure of cell embeddings in scFMs [47]. For cell type annotation fine-tuning, we compute overall accuracy to measure general performance and report macro F1 to equally weight all classes regardless of imbalance. For novel cell identification, we compute per-cell-type accuracy and report macro-accuracy to account for class imbalance. For gene function prediction, we report micro/macro precision and micro/macro F1. For spatial gene imputation, we quantify the agreement between imputed and ground-truth spatial expression using Pearson, Spearman, and Kendall’s τ\tau correlation coefficients [35]. The mathematical formulations of each of these metrics are provided in the following sub-sections.

Zero-shot clustering evaluation metrics

Normalized Mutual Information (NMI):

NMI(C,K)=I(C;K)12[H(C)+H(K)]\mathrm{NMI}(C,K)=\frac{I(C;K)}{\frac{1}{2}[H(C)+H(K)]}

where:

  • CC and KK are the true cell type labels and predicted cluster labels.

  • H(C)=−∑iP(ci)log⁡P(ci)H(C)=-\sum_i P(c_i)\log P(c_i) is the entropy of the labels.

  • I(C;K)=∑i,jP(ci,kj)log⁡P(ci,kj)P(ci)P(kj)I(C;K)=\sum_{i,j}P(c_i,k_j)\log\frac{P(c_i,k_j)}{P(c_i)P(k_j)} is the mutual information.

Adjusted Rand Index (ARI):

ARI=RI−E[RI]max⁡(RI)−E[RI]\mathrm{ARI}=\frac{\mathrm{RI}-E[\mathrm{RI}]}{\max(\mathrm{RI})-E[\mathrm{RI}]}

where the unadjusted Rand Index (RI) is:

RI=TP+TNTP+FP+FN+TN=a+b(N2)\mathrm{RI}=\frac{TP+TN}{TP+FP+FN+TN}=\frac{a+b}{\binom{N}{2}}
  • aa: Number of pairs of cells in the same set in CC and the same set in KK.

  • bb: Number of pairs of cells in different sets in CC and different sets in KK.

Average Silhouette Width (ASWlabel_{label}) - Cell Type:

si=bi−aimax⁡(ai,bi),ASWlabel=1N∑i=1Nsi.s_i=\frac{b_i-a_i}{\max(a_i,b_i)},\qquad\mathrm{ASW}_{label}=\frac{1}{N}\sum_{i=1}^{N}s_i.

The commonly used version is the rescaled version down below, and we also use the scaled version in our experiment.

ASWlabel=ASWlabel+12.\mathrm{ASW}_{label}=\frac{\mathrm{ASW}_{label}+1}{2}.

Average Biological Score (AvgBio):

AvgBio=13(NMI+ARI+ASWlabel).\mathrm{AvgBio}=\frac{1}{3}(\mathrm{NMI}+\mathrm{ARI}+\mathrm{ASW}_{label}).

Average Silhouette Width (ASWlabel/batch_{label/batch}) - Batch:

ASWlabel/batch=1∣C∣∑c∈C[1−∣1∣Nc∣∑i∈Ncsibatch∣].\mathrm{ASW}_{label/batch}=\frac{1}{|C|}\sum_{c\in C}\left[1-\left|\frac{1}{|N_c|}\sum_{i\in N_c}s_i^{batch}\right|\right].

where sibatchs_i^{\mathrm{batch}} is the silhouette score calculated using batch labels (instead of cell types) within a specific cell type group cc.

Graph Connectivity (GraphConn):

GraphConn=1∣C∣∑c∈C∣LCC(Gc)∣∣Nc∣\mathrm{GraphConn} = \frac{1}{|C|} \sum_{c \in C} \frac{|\mathrm{LCC}(G_c)|}{|N_c|}

where:

  • GcG_c: The k-nearest neighbor (kNN) subgraph containing only cells of type cc.

  • LCC(Gc)\mathrm{LCC}(G_c): The largest connected component in that subgraph.

Average Batch Score (AvgBatch):

AvgBatch=12(ASWlabel/batch+GraphConn).\mathrm{AvgBatch} = \frac{1}{2}\left(\mathrm{ASW}_{\mathit{label}/\mathit{batch}} + \mathrm{GraphConn}\right).

Overall scIB score: overall score = 0.6 × AvgBio + 0.4 × AvgBatch.

scGraph-LCA To quantify how well the learned latent embedding preserves the hierarchical relationships defined in the Cell Ontology, we employ a correlation-based graph similarity measurement metric, named scGraph-LCA. Our metric generalizes the evaluation protocols proposed in scGraph [47] and scGraph-RWR [50]. It relies on the comparison of two pairwise distance matrices: one derived from the model’s latent space (Δemb\Delta^{\mathrm{emb}}) and one derived from the ontology graph (Δonto\Delta^{\mathrm{onto}}). We will illustrate each component one by one in the following:

Model embedding distance matrix (Δemb\Delta^{\mathrm{emb}})

Following the notation in § 2.1 in the main paper, here, let D={xn,cn}n=0N\mathcal{D} = \{x_n, c_n\}_{n=0}^{N} denotes a single-cell RNA-seq dataset, where each cell nn is associated with a cell type label cn∈Vc_n \in\mathcal{V}. We reuse the notation of Cell-Dataset Index (Ic,k\mathcal{I}_{c,k}) here, let Ic=⋃k∈KcIc,k\mathcal{I}_c = \bigcup_{k \in\mathcal{K}_c} \mathcal{I}_{c,k} represent the set of indices for all cells annotated as type c∈Vc \in\mathcal{V} across all available datasets/batches, here kk could interchangeably represent as batches, since if we only evaluate on one dataset, the number of available dataset is always being 1, however, there may exist multiple batches in one dataset.

For each unique cell type c∈Vc \in\mathcal{V} present in the data, we compute the centroid in the embedding space. To mitigate the effect of outliers, we employ a trimmed mean aggregation. Let {zn∣n∈Ic}\{z_n \mid n \in\mathcal{I}_c\} be the set of embedding vectors for all cells of type cc. We compute the trimmed mean centroid μc\mu_c:

μc=TrimmedMean⁡({zn∣n∈Ic},α),(16)\mu_c = \operatorname{TrimmedMean}\left(\{z_n \mid n \in\mathcal{I}_c\}, \alpha\right), \tag*{(16)}

where α\alpha is the trimming proportion (set to 0.2 in our experiments).

We then construct the pairwise Embedding Distance Matrix Δemb∈R∣V∣×∣V∣\Delta^{\mathrm{emb}} \in\mathbb{R}^{|\mathcal{V}| \times|\mathcal{V}|}, where the entry (i,j)(i,j) represents the Euclidean distance between the centroids of cell types cic_i and cjc_j:

Δijemb=∥μci−μcj∥2.(17)\Delta_{ij}^{\mathrm{emb}} = \lVert\mu_{c_i} - \mu_{c_j} \rVert_2. \tag*{(17)}

Ontology Reference Distance Matrix (Δonto\Delta^{\mathrm{onto}})

The reference matrix encodes the "ground truth" biological distances extracted from the Cell Ontology graph G=(V,E)\mathcal{G} = (\mathcal{V}, \mathcal{E}). , we first introduce the LCA distance matrix, then in the following we will derive the scGraph-RWR based on RWR distance matrix.

As introduced in § 2.1, we utilize the Lowest Common Ancestor (LCA) to capture the hierarchical distances on the ontology graph. For any two types of cells ci,cj∈Vc_i, c_j \in\mathcal{V}, their raw semantic distance dG(ci,cj)d_{\mathcal{G}}(c_i,c_j) is calculated as the sum of the shortest path distances from each node to their common cs=LCA⁡(ci,cj)c_s = \operatorname{LCA}(c_i,c_j):

Δijonto,LCA=δG(ci,cj)=δpath(ci,cs)+δpath(cj,cs),(18)\Delta_{ij}^{\mathrm{onto},\mathrm{LCA}} = \delta_{\mathcal{G}}(c_i,c_j) = \delta_{\mathit{path}}(c_i,c_s) + \delta_{\mathit{path}}(c_j,c_s), \tag*{(18)}

where δpath(ci,cs)\delta_{\mathit{path}}(c_i,c_s) and δpath(cj,cs)\delta_{\mathit{path}}(c_j,c_s) denote the number of edges on the shortest path between nodes (ci,cj)(c_i,c_j) to their lowest common ancestor csc_s in G\mathcal{G}, here we use Δijonto,LCA\Delta_{ij}^{\mathrm{onto},\mathrm{LCA}} to distinguish the RWR-based distance matrix (Δijonto,RWR\Delta_{ij}^{\mathrm{onto},\mathrm{RWR}}) in scGraph-RWR.

Graph Similarity Measurement

To quantify the similarity between the model embedding distance matrix and ontology-based distance matrix, we calculate Pearson Correlation and the rank-based correlation (Spearman, and Kendall’s τ\tau) between the columns of the distance matrices. For each cell type cic_i, let riemb\mathbf{r}_i^{\mathrm{emb}} be the ii-th column of Δemb,LCA\Delta^{\mathrm{emb},\mathrm{LCA}} and rionto\mathbf{r}_i^{\mathrm{onto}} be the ii-th column of Δonto,LCA\Delta^{\mathrm{onto},\mathrm{LCA}}. We compute the agreement score using Pearson Correlation (rr), Spearman’s rank correlation (ρ\rho) and Kendall’s rank correlation coefficient (τ\tau), the full mathematical equations of the above metrics are illustrated in more detail in Appendix § 8.11.5:

scGraph-LCAr=1K∑i=1Kρ(riemb,rionto),(19)\mathrm{scGraph\text{-}LCA}_{r} = \frac{1}{K}\sum_{i=1}^{K}\rho\left(\mathbf{r}_i^{\mathrm{emb}},\mathbf{r}_i^{\mathrm{onto}}\right), \tag*{(19)}
scGraph-LCAρ=1K∑i=1Kρ(riemb,rionto),(20)\mathrm{scGraph\text{-}LCA}_{\rho} = \frac{1}{K}\sum_{i=1}^{K}\rho\left(\mathbf{r}_i^{\mathrm{emb}},\mathbf{r}_i^{\mathrm{onto}}\right), \tag*{(20)}
scGraph-LCAτ=1K∑i=1Kτ(riemb,rionto).(21)\mathrm{scGraph\text{-}LCA}_{\tau} = \frac{1}{K}\sum_{i=1}^{K}\tau\left(\mathbf{r}_i^{\mathrm{emb}},\mathbf{r}_i^{\mathrm{onto}}\right). \tag*{(21)}

A higher correlation indicates that cell types close in the ontology graph are also geometrically proximal in the model’s latent space, demonstrating preservation of biological semantics.

scGraph-RWR According to [50], scGraph-RWR utilizes a weighted ontology graph, in our experiment the weights are download from [48]. Regarding the reference distance matrix, we just need to replace the LCA-based ontology distance matrix with the RWR-based ontology distance matrix by following scGraph-LCA procedures. The process to get Δonto,RWR\Delta^{\mathrm{onto},\mathrm{RWR}} is as follows:

First, we define the column-normalized transition probability matrix P\mathbf{P} as:

P=AD−1,(22)\mathbf{P} = \mathbf{A}\mathbf{D}^{-1}, \tag*{(22)}

where A\mathbf{A} is the adjacency matrix of ontology graph GG and D\mathbf{D} is the diagonal degree matrix. The RWR diffusion matrix R\mathbf{R}, which captures the stationary probability distribution for walkers starting at each node, is computed analytically as:

R=β(I−(1−β)P)−1,(23)\mathbf{R} = \beta\left(\mathbf{I} - (1-\beta)\mathbf{P}\right)^{-1}, \tag*{(23)}

where β\beta is the restart probability (regulating the diffusion scale) and I\mathbf{I} is the identity matrix. Finally, the ontology-derived RWR distance between cell types ii and jj is defined as the complement of the diffusion probability:

Δijonto,RWR=1−Rij.(24)\Delta_{ij}^{\mathrm{onto},\mathrm{RWR}} = 1 - R_{ij}. \tag*{(24)}

Cell annotation (Fine-tuning) metrics

Let CC be the complete set of unique cell types present in the evaluation dataset. The overall classification performance is evaluated using Accuracy and Macro F1 score. The Accuracy is defined as the overall proportion of correctly classified cells out of the total number of cells:

Accuracy=∑c∈CTPcN,\mathrm{Accuracy} = \frac{\sum_{c\in C} TP_c}{N},

where:

  • TPcTP_c: The number of cells correctly classified as cell type cc (True Positives).

  • NN: The total number of cells across all types in the evaluation dataset.

The Macro F1 score is defined as the arithmetic mean of the individual F1 scores computed independently for each cell type c∈Cc \in C, treating all classes equally regardless of their support size:

Macro F1=1∣C∣∑c∈CF1c.\mathrm{Macro\ F1} = \frac{1}{|C|}\sum_{c\in C}\mathrm{F1}_c.

where F1cF1_c is the harmonic mean of the Precision and Recall for a specific cell type cc:

F1c=2×Precisionc×RecallcPrecisionc+RecallcF1_c = 2 \times\frac{\mathrm{Precision}_c \times\mathrm{Recall}_c}{\mathrm{Precision}_c + \mathrm{Recall}_c}

with Precision and Recall defined as:

Precisionc=TPcTPc+FPc,Recallc=TPcTPc+FNc\mathrm{Precision}_c = \frac{TP_c}{TP_c + FP_c}, \qquad\mathrm{Recall}_c = \frac{TP_c}{TP_c + FN_c}
  • FPcFP_c: The number of cells incorrectly classified as cell type cc (False Positives).

  • FNcFN_c: The number of cells truly belonging to cell type cc but incorrectly classified as another type (False Negatives).

  • ∣C∣|C|: The total count of unique cell types in the evaluation set.

Novel cell identification metrics

Let U⊂VU \subset V be the set of unseen cell types (where VV is the total set of cell ontology nodes). The Macro Accuracy over the unseen set UU is defined as the arithmetic mean of the recall (accuracy) for each unseen class:

Macro AccuracyU=1∣U∣∑u∈UAccu\mathrm{Macro\ Accuracy}_U = \frac{1}{|U|} \sum_{u \in U} \mathrm{Acc}_u

where Accu\mathrm{Acc}_u is the accuracy for a specific cell type uu:

Accu=TPuNu\mathrm{Acc}_u = \frac{TP_u}{N_u}
  • TPuTP_u: The number of cells correctly classified as type uu (True Positives).

  • NuN_u: The total number of cells in the dataset with true label uu (Support).

  • ∣U∣|U|: The count of unique unseen cell types.

Gene function prediction

We evaluate the multi-label GO function classification performance using Micro-averaged and Macro-averaged metrics. Let CC be the total number of classes (in which we have three broad GO terms: Biological Process (BP), Cellular Component (CC), and Molecular Function (MF)). For each specific class i∈{1,…,C}i \in\{1, \ldots, C\}, we define:

  • TPiTP_i: True Positives (samples correctly predicted to have GO term ii).

  • FPiFP_i: False Positives (samples incorrectly predicted to have GO term ii).

  • FNiFN_i: False Negatives (samples with GO term ii that were missed).

Macro-Averaged Metrics Macro metrics calculate performance for each GO class independently and then take the unweighted average. This ensures that specific, rare GO class contribute equally to the score as broad, common terms.

Precisionmacro=1C∑i=1CTPiTPi+FPi\mathrm{Precision}_{\mathrm{macro}} = \frac{1}{C} \sum_{i=1}^{C} \frac{TP_i}{TP_i + FP_i}
Recallmacro=1C∑i=1CTPiTPi+FNi\mathrm{Recall}_{\mathrm{macro}} = \frac{1}{C} \sum_{i=1}^{C} \frac{TP_i}{TP_i + FN_i}
F1macro=1C∑i=1C2×TPi2×TPi+FPi+FNiF1_{\mathrm{macro}} = \frac{1}{C} \sum_{i=1}^{C} \frac{2 \times TP_i}{2 \times TP_i + FP_i + FN_i}

Micro-Averaged Metrics Micro metrics aggregate the prediction counts across all GO class globally before calculating the score. This gives a measure of the overall system reliability, weighted by the frequency of the terms.

Precision⁡micro=∑i=1CTPi∑i=1C(TPi+FPi)\operatorname{Precision}_{\mathrm{micro}} = \frac{\sum_{i=1}^{C} TP_i}{\sum_{i=1}^{C}(TP_i + FP_i)}
Recall⁡micro=∑i=1CTPi∑i=1C(TPi+FNi)\operatorname{Recall}_{\mathrm{micro}} = \frac{\sum_{i=1}^{C} TP_i}{\sum_{i=1}^{C}(TP_i + FN_i)}
F1micro=2×Precision⁡micro×Recall⁡microPrecision⁡micro+Recall⁡microF1_{\mathrm{micro}} = \frac{2 \times\operatorname{Precision}_{\mathrm{micro}} \times\operatorname{Recall}_{\mathrm{micro}}}{\operatorname{Precision}_{\mathrm{micro}} + \operatorname{Recall}_{\mathrm{micro}}}

Spatial imputation evaluation metrics

Pearson Correlation (rr)

r=∑(xi−xˉ)(yi−yˉ)∑(xi−xˉ)2∑(yi−yˉ)2r = \frac{\sum(x_i-\bar{x})(y_i-\bar{y})}{\sqrt{\sum(x_i-\bar{x})^2}\sqrt{\sum(y_i-\bar{y})^2}}

Spearman’s Rank Correlation (ρ\rho)

ρ=1−6∑di2n(n2−1)\rho= 1-\frac{6\sum d_i^2}{n(n^2-1)}

where di=rank⁡(xi)−rank⁡(yi)d_i = \operatorname{rank}(x_i)-\operatorname{rank}(y_i).

Kendall’s Tau (τ\tau)

τ=C−D12n(n−1)\tau= \frac{C-D}{\frac{1}{2}n(n-1)}

where CC is the number of concordant pairs and DD is the number of discordant pairs.

RESULTS

Zero-shot clustering SOTA benchmark performance of scIB metrics

Overall scIB score across the evaluation tissues

The overall of the scIB scores for each dataset are provided in ATable 16.

Details of each scIB score across the evaluation tissues

For the details of each scIB metrics: ARI, NMI, ASWlabel_{label}, GraphConn, ASWlabel/batch_{label/batch}, AvgBio, AvgBatch, in HIHA, Heart, Pancreas, Lung, Liver, Breast_TNBC, Kidney, Adipose Tissue, Bone Marrow. Please refer to the ATable 17.

Zero-shot clustering of scGraph-LCA benchmarking performance (Table 18)

Zero-shot clustering of scGraph-RWR benchmarking performance(Table 19)

Zero-shot clustering of 340 in-distribution cell types in unseen datasets

We report the AvgBio metrics (ARI, NMI, ASWlabel_{label}) in scIB (ATabel 20), since we randomly down-sample the cell types with more than 10,000 cells, it is hard for us to observe the batch/dataset id in this process, for simplicity of the implementation, we only calculate the AvgBio related metrics. Additionally, we also report scGraph-LCA and scGraph-RWR in ATabel 20. Note that for the scGraph-RWR metric, 48 cell types are not present in reference embedding, so the total cell types for this metric is 292.

Bone marrow clustering and graph distance visualization

We also demonstrated that the learned embeddings of scOntoFM well reflect the intrinsic biologically hierarchical structure of hematopoietic related cell types (AFigure 11c), showing strong empirical consistency with ontology-based distances (AFigure 12). Furthermore, the ontology-derived distance heatmap in AFigure 11b closely aligns with the G2/M and S-phase cell cycle scores reported in the original study AFigure 11d. Taken together, these results indicate that our model could effectively capture the change in cell states along continuous developmental trajectories.

Clustering and distance heatmaps for the bone marrow dataset

Figure 11. (a) Clustering results of Bone marrow dataset using our model embeddings, colors represent different cell-types. (b) Distance heatmap from root centroid cell-type (hematopoietic stem cell, highlighted as yellow dot in the umap) to all the other cell-types in the dataset. (c) Cell-type centroid embedding distances heatmap for the bone marrow dataset. (d) Cell cycle scores heatmap from original paper [55].

Novel cell identification results

In AFigure 13(a, b), we first demonstrate that our model has learned biological meaningful structure of known cell types, cell-type labels are aggregated into coarser level according to the lineage of ontology tree. Then built on known cell-type centroid embeddings and the comprehensive LCA-based ontology affinity matrix introduced in Appendix § 8.§ 12.4, scOntoFM could infer the unseen cell-type centroids as shown in Figure 13(c, Figure 13d). Concretely, we conducted a specific case study on three unseen hematopoietic cell types: megakaryocyte progenitor cells, immature neutrophils, and CD14-positive/CD16-low monocytes. As Figure 14 shows, our model accurately positions unseen cells near their biological relatives. This alignment with ground-truth ontology distances confirms the model’s ability to correctly infer the biological neighborhood of unknown cell types

Figure 13. (a) Umap visualization of seen cell-type centroid embedding, cell-types are colored by its ontology ancestor at depth 2. (b) Same as (a), but cell-types are colored by its ontology ancestor at depth 1. (c) Co-embedding seen centroid embeddings and unseen cell-type centroid embedding, dots are colored by seen centroid (blue) and unseen centroid (orange). (d) Our model predicted accuracy heatmap for unseen cell-types. The red circles in all the umaps are the cell-types coming from hematopoietic or myeloid cell lineage, in which the unseen cell-types (megakaryocyte progenitor cell, immature neutrophil, CD14-positive, CD16-low monocyte) in Figure 14 are selected from.

Figure 14. (a) Umap of our model embeddings on three unseen cell types and the top 5 closest seen cell types for each unseen cell type. (b) Umap visualization of seen (orange) and unseen (blue) labels. (c) Umap visualization of the three query unseen cell types. (d) LCA distance matrix of unseen cell types and the top 5 closest seen cell types for each unseen cell type.

Figure 12. Left: LCA distance reference heatmap of scGraph-LCA in Bone marrow dataset; Right: Random walk with restart (scGraph-RWR) of ontology graph distance in Bone marrow dataset.

Gene function prediction results

In Figure 15, we show additional results of the COAGULATION hallmark gene set and the ALLOGRAFT_REJECTION hallmark gene set enrichment score in the lung adenocarcinoma dataset [27]. The enrichment scores for the ALLOGRAFT_REJECTION hallmark are notably elevated in immune-related populations—specifically, macrophages, T cells, and NK cells—while remaining low in non-immune lineages. Particularly, the COAGULATION hallmark is highly enriched in the cancer-associated fibroblasts (CAFs), which contributes hypercoagulable state and facilitate tumor immune evasion [53]. These distinct enrichment patterns confirm that our model accurately disentangles the complex functional heterogeneity within the tumor microenvironment.

Cell embeddings and hallmark enrichment score distributions

Figure 15. (a) Our model’s umap embedding visualization colored by cell-types. (b) Violin plot of GLYCOLYSIS hallmark enrichment score distribution for each cell type. (c) Left: COAGULATION hallmark enrichment score heatmap. Right: Violin plot of COAGULATION hallmark enrichment score distribution for each cell type. (d) Left: ALLOGRAFT_REJECTION hallmark enrichment score heatmap. Right: Violin plot of ALLOGRAFT_REJECTION hallmark enrichment score distribution for each cell type.

GO namespace classification by aggregating gene embeddings of the GO terms.

Beyond classifying semantic meaning of single gene to GO terms, we further evaluated GO namespace classification by aggregating the gene embeddings for specific GO terms. The results are reported as mean±std over 5 random seeds. As Table 21 shows, scOntoFM achieves highly competitive results, ranking second only to scGPT, while still leading other Geneformer-like backbone. The performance difference is expected given scGPT’s significantly larger vocabulary size (60,000 vs. 20,000 genes), and embedding dimensionality (512 vs. 256) compared to scOntoFM.

ModelF1 MicroF1 MacroAP MicroAP Macro
Ours<u>0.783</u> ± 0.01<u>0.632</u> ± 0.02<u>0.832</u> ± 0.01<u>0.632</u> ± 0.02
Geneformer0.767 ± 0.010.593 ± 0.020.832 ± 0.010.613 ± 0.02
scCello0.781 ± 0.010.627 ± 0.020.823 ± 0.010.620 ± 0.02
scGPT0.809 ± 0.010.683 ± 0.020.861 ± 0.010.678 ± 0.02

Table 21. GO namespace classification by aggregated gene embeddings. bold indicates the best performance and underlining denotes the second-best results.

Spatial imputation results

In Figure 16, we show an additional violin plot for spatial gene imputation in terms of Pearson, Spearman, and Kendall’s τ\tau correlation metrics across the three evaluation datasets (MERSCOPE Lung cancer 2, breast cancer and SEA-AD). We conducted the Mann-Whitney U tests between our method other scFMs on spatial gene imputation task. Our method is statistical significant vs. other scFMs in both the Lung and breast cancer data except Geneformer.

Violin plots showing Pearson, Spearman, and Kendall’s $\tau$ correlation distributions across imputed genes in Lung Cancer, Breast Cancer, and SEA-AD datasets

Figure 16. (a) Violin plot of the distribution for Pearson, Spearman, and Kendall’s τ\tau correlation across all the imputed genes in MERSCOPE Lung cancer 2 dataset. (b) Violin plot of the distribution for Pearson, Spearman, and Kendall’s τ\tau correlation across all the imputed genes in breast cancer dataset. (c) Violin plot of the distribution for Pearson, Spearman, and Kendall’s τ\tau correlation across all the imputed genes in SEA-AD dataset. Statistical significance is indicated by stars (*), with “ns” denoting no significant difference.

Ablation study about the loss terms in scOntoFM (ATable 24)

Pareto analysis across the nine tissues in our study

The highlighted points represent the average values across each ablation setting. The plot (Figure 6) demonstrates that scOntoFM achieves an optimal balance between standard integration metrics (scIB) and ontology-based metric (scGraph-LCA).

Robustness of scOntoFM to Cell Type Annotation Noise

Cell type annotation noise and inconsistency across datasets can introduce substantial bias during pre-training. To mitigate this, we pre-trained scOntoFM on the CZI CELLxGENE platform, where metadata curation—particularly cell type standardization—is a large-scale, expert-driven effort involving resources such as the Human Cell Atlas (HCA), Tabula Sapiens, and HuBMAP. Pre-training on this well-curated and standardized corpus not only avoids performance degradation but also enables scOntoFM to better capture hierarchical relationships between cell types and preserve biologically meaningful structure.

We evaluate robustness from two complementary aspects. First, robustness across annotation levels in zero-shot clustering: scOntoFM consistently maintains strong performance under varying label granularities (Table 2), particularly at fine-grained levels (e.g., AIFI_L1/L2 in HIHA and SubclassLevel2 in Kidney), highlighting its ability to capture subtle cell states and complex lineage structures. This is further supported by high agreement scores (ARI) between cell ontology references and annotations at different resolutions (see ATable 25, Figure 17, 18), demonstrating stability under label mismatch. Second, resilience to label noise: under controlled noise injection on the Azimuth PBMC dataset, scOntoFM maintains high accuracy and stable performance as noise increases (ATable 26). Notably, at higher noise levels (e.g., 50%), the model predominantly assigns cells to their parent categories, reflecting biologically consistent behavior when fine-grained labels become unreliable.

Broader Impact of scOntoFM

This paper presents work whose goal is to advance the field of Machine Learning and Computational Biology. There are many potential societal consequences of our ontology-based foundation model, particularly in the interpretation of complex single-cell atlases. We hope that by integrating hierarchical biological priors, our model will aid researchers in uncovering accurate biological insights and accelerating disease characterization. However, we acknowledge that pretraining such foundation models is computationally intensive, and future efforts should address the energy efficiency and accessibility of these large-scale systems.

DatasetModelAccuracyMacro-F1
HIHAscOntoFM0.92 ± 0.000.77 ± 0.01
HIHAHCE-TabNet0.91 ± 0.000.71 ± 0.02
HIHAHCE-MLP0.90 ± 0.000.70 ± 0.00
PancreasscOntoFM0.99 ± 0.000.97 ± 0.01
PancreasHCE-TabNet0.98 ± 0.000.90 ± 0.01
PancreasHCE-MLP0.95 ± 0.000.88 ± 0.01

Table 12. Benchmark HCE model in cell-type annotation (fine-tuning) on the HIHA and Pancreas datasets. Results are reported as mean ± standard deviation.

CL IDcellPLMGeneformerOursscCelloscGPTSCimilaritycountCL IDcellPLMGeneformerOursscCelloscGPTSCimilaritycount
CL:00000300.0150.0040.0410.020.002010000CL:000907000.0610.0030.3040.00105888
CL:0000047000.02400010000CL:00090710.9490.7480.8860.4330.8360.7873641
CL:0000060000.00100.00204451CL:00100120.050.010.001000.08210000
CL:000008300.0280.0040.3420.1830.8569257CL:00110310.5760.6750.9380.9830.8730.99810000
CL:000012100.00100.170010000CL:00190210.4080.0890.6490.4470.3570.1674533
CL:000015910.15710.13110.0635088CL:10000220.4980.590.8820.9460.4980.0523513
CL:000033800000010000CL:100004200000010000
CL:0000515000.8290.57700.29610000CL:10003980000007050
CL:00005290.15400.4040.7730010000CL:10004430.9960.0020.9970.7660.9960.99410000
CL:000054900.01200.0010.15805253CL:10005000.00800.0040.0890.0040.0038409
CL:000055000.0080.00100.00505061CL:10015670.0180.0010.0120.1160.0230.02310000
CL:00005520.9740.9730.9750.9730.9680.9863734CL:10015690000009976
CL:00005530.6630.8480.7080.2540.9170.8992564CL:10015760.0190.0220.2090.1670.140.0057386
CL:0000598000.0450.0390.005010000CL:1001609000.0460.04100.0487010
CL:00007000000006364CL:20000160.0080.0060.0180.1020.0130.0043430
CL:00007900.0050.1150.4380.3540.1570.38510000CL:20000290.03100.0840.2380.0570.00610000
CL:00008670.0150.0990.0710.3130.0140.0944554CL:20000540.5640.5360.7470.5760.5840.649320
CL:00009120.3320.1220.3410.2080.3150.16810000CL:4023009000.4210.6420010000
CL:00009200.0540.030.2530.4020.1760.218377CL:40230260000005385
CL:00010330.4010.0110.0180.0780.1470.00110000CL:40230290000006172
CL:000103400000010000CL:4023069000.6190.404006404
CL:00010490.0820.0280.6610.0790.0540.1315698
CL:00010550.8130.8480.8420.8310.8470.9152494
CL:000203200.0750.630.4970.3640.5096718

Table 13. Predicted accuracy per novel cell type across SOTA methods. The best result for each cell type is shown in bold.

cellPLMGeneformerOursscCelloscGPTSCimilaritytotal count
Macro accuracy0.2130.1510.2860.2540.2440.240774,215
Average rank3.7124.2022.8483.2373.4853.515774,215

Table 13.

DatasetST (spots ×\times genes)SC (cells ×\times genes)Shared genes for imputation
Lung cancer31, 520 ×\times 423207, 853 ×\times 20, 793422
Breast cancer4, 784 ×\times 1, 0009, 234 ×\times 19, 098891
SEA-AD18, 085 ×\times 18015, 876 ×\times 20, 080132

Table 15. Summary of spatial transcriptomics (ST) and single-cell (SC) datasets.

DatasetOursGeneformerscGPTscCelloCellPLMSCimilarity
HIHA0.75330.44440.60020.56680.70370.7730
Heart0.83240.68620.83260.86660.78470.8299
Pancreas0.81400.61130.79670.72030.70240.8023
Lung0.84670.83950.83970.82990.83790.8566
Liver0.77300.68530.78860.77200.71020.7921
Breast TNBC0.87240.68240.84180.84230.83770.8601
Kidney0.71850.57850.70200.76010.72110.7149
Adipose Tissue0.86960.65090.81810.85890.80640.8290
Bone Marrow0.68230.64980.69240.66590.67720.7059
Mean0.79580.64760.76800.76470.75350.7960

Table 16. Performance comparison of zero-shot clustering scIB overall score across tissues/datasets. The best results are shown in bold for each dataset.

DatasetMethodNMIARIASWlabel_{\text{label}}GraphConnASWlabel/batch_{\text{label/batch}}AvgBioAvgBatchOverall
HIHAOurs0.75970.64220.60210.88680.87560.66800.88120.7533
HIHAGeneformer0.25000.13550.45600.51170.86870.28050.69020.4444
HIHAscGPT0.51990.26190.49770.82180.89950.42650.86070.6002
HIHAscCello0.47040.32450.48240.64180.91490.42570.77840.5668
HIHAcellPLM0.71010.47550.53650.92080.87550.57400.89820.7037
HIHASCimilarity0.76750.73750.58330.91670.86020.69610.88840.7730
HeartOurs0.85810.82040.67600.92970.87800.78480.90390.8324
HeartGeneformer0.64640.49000.52280.88780.88380.55310.88580.6862
HeartscGPT0.86410.89050.59110.93710.88030.78190.90870.8326
HeartscCello0.90360.93830.65460.94560.89070.83210.91820.8666
HeartcellPLM0.81680.72280.58910.93240.86270.70960.89750.7847
HeartSCimilarity0.85720.81820.67150.94020.86260.78230.90140.8299
PancreasOurs0.85980.79510.60920.97990.82590.75470.90290.8140
PancreasGeneformer0.50450.31870.50820.87260.85250.44380.86250.6113
PancreasscGPT0.83150.85230.52590.91990.85390.73650.88690.7967
PancreasscCello0.64420.54200.57820.96160.87530.58810.91850.7203
PancreascellPLM0.62770.55780.55150.92470.85040.57900.88750.7024
PancreasSCimilarity0.84970.72520.65680.95220.82760.74390.88990.8023
LungOurs0.87220.86960.64690.97970.86530.79620.92250.8467
LungGeneformer0.87970.85120.57740.97750.91150.76940.94450.8395
LungscGPT0.88130.85190.57360.99550.89650.76890.94600.8397
LungscCello0.86640.80430.62110.97810.87940.76400.92870.8299
LungcellPLM0.88230.84770.57860.99420.88670.76960.94040.8379
LungSCimilarity0.88980.87330.68200.98850.84950.81500.91900.8566
LiverOurs0.78710.69350.58240.96000.84220.68770.90110.7730
LiverGeneformer0.65020.46780.52300.93660.84860.54700.89260.6853
LiverscGPT0.79330.75490.57610.96250.85640.70810.90950.7886
LiverscCello0.76740.67630.57460.96590.87560.67280.92070.7720
LivercellPLM0.67380.48500.56390.97420.85420.57420.91420.7102
LiverSCimilarity0.79200.71540.64610.97300.83400.71790.90350.7921
Breast TNBCOurs0.88070.90900.72450.99540.85270.83800.92410.8724
Breast TNBCGeneformer0.60800.39390.52790.99050.89170.50990.94110.6824
Breast TNBCscGPT0.85430.85280.60700.99410.90080.77140.94750.8418
Breast TNBCscCello0.83160.80140.71060.98950.87850.78120.93400.8423
Breast TNBCcellPLM0.82150.84440.64380.99290.88580.76990.93930.8377
Breast TNBCSCimilarity0.85960.86160.72670.99630.85650.81600.92640.8601
KidneyOurs0.71740.57110.58710.95420.76270.62520.85850.7185
KidneyGeneformer0.48740.20080.48780.93920.77720.39200.85820.5785
KidneyscGPT0.71110.54890.53310.95740.75970.59770.85860.7020
KidneyscCello0.74520.68510.60090.96410.80500.67710.88460.7601
KidneycellPLM0.72010.60850.53570.97710.76430.62150.87070.7211
KidneySCimilarity0.75660.53470.61160.96460.70720.63430.83590.7149
Adipose TissueOurs0.90980.93070.64110.96260.90370.82720.93310.8696
Adipose TissueGeneformer0.60910.30700.50120.93900.89840.47240.91870.6509
Adipose TissuescGPT0.86180.75570.59440.97100.90740.73730.93920.8181
Adipose TissuescCello0.87860.89910.63370.95800.92500.80380.94150.8589
Adipose TissuecellPLM0.81510.73590.61200.97750.89150.72100.93450.8064
Adipose TissueSCimilarity0.86390.75670.68680.96850.86920.76910.91880.8290
Bone MarrowOurs0.68090.39190.51350.96500.86010.52880.91260.6823
Bone MarrowGeneformer0.60250.32370.49390.93640.89250.47340.91440.6498
Bone MarrowscGPT0.66540.40020.52670.96930.90050.53070.93490.6924
Bone MarrowscCello0.62560.38670.50750.94840.86140.50660.90490.6659
Bone MarrowcellPLM0.62450.38840.52600.96530.88160.51300.92350.6772
Bone MarrowSCimilarity0.71600.43210.54060.97070.87030.56290.92050.7059

Table 17. Zero-shot clustering benchmarking results of detailed scIB metrics for each of the nine evaluation datasets. The best performance in each metric for each dataset is highlighted in bold.

DatasetSpearman OursSpearman GeneformerSpearman scGPTSpearman scCelloSpearman cellPLMSpearman SCimilarityKendall’s τ\tau OursKendall’s τ\tau GeneformerKendall’s τ\tau scGPTKendall’s τ\tau scCelloKendall’s τ\tau cellPLMKendall’s τ\tau SCimilarityPearson OursPearson GeneformerPearson scGPTPearson scCelloPearson cellPLMPearson SCimilarity
HIHA0.71570.38900.41740.47740.37720.62010.55880.28490.31710.35900.28950.49210.76690.45270.53000.59460.52150.7294
Heart0.74070.42200.66050.41630.51000.50900.83100.34370.52550.32770.42390.44710.77870.60430.71940.64260.65980.9099
Pancreas0.39920.40400.39780.33130.54700.66460.29600.32130.31350.23000.44540.56990.73980.71390.76600.64780.77800.8412
Lung0.67770.67190.40470.46650.46240.59580.56150.56210.32700.36840.37450.48900.83810.81270.73960.75190.74740.7886
Liver0.57040.49050.48240.49250.50230.48420.46910.35870.37990.39230.40080.37810.74110.64570.64230.67180.67160.5901
Breast TNBC0.51120.40000.43280.59690.54870.42260.39560.32610.34700.50850.43900.34610.68990.82980.87740.88080.86930.8425
Kidney0.63780.44230.27600.35540.39120.52040.50830.33680.19210.25730.28780.39100.68570.57480.48800.55790.53820.6273
Adipose Tissue0.61800.49100.41440.53980.46440.63550.47620.37840.32670.40510.30560.50300.78300.67170.64500.71020.66170.7834
Bone Marrow0.66300.35060.24310.52070.28910.59800.53700.24700.17940.39840.22020.47490.70930.41510.41200.61150.44370.6849
Mean0.61120.44900.41430.46560.45470.56110.48790.35540.32330.36180.35680.45060.76810.63620.65460.67470.65870.7376

Table 18. Dataset-wise comparison of zero-shot clustering scGraph-LCA metric under Spearman, Kendall’s τ\tau, and Pearson correlation. Best result per dataset and per metric is shown in bold.

DatasetSpearman OursSpearman GeneformerSpearman scGPTSpearman scCelloSpearman CellPLMKendall’s τ\tau OursKendall’s τ\tau GeneformerKendall’s τ\tau scGPTKendall’s τ\tau scCelloKendall’s τ\tau CellPLMPearson OursPearson GeneformerPearson scGPTPearson scCelloPearson CellPLM
HIHA0.49390.45840.50000.52530.49510.36680.32920.36770.38020.36180.79980.61880.62300.68550.7118
Heart0.34380.41410.31900.43720.39500.22640.32230.21320.32230.30580.89260.84060.88570.91460.9022
Pancreas0.53730.56930.37000.65620.50290.41160.46970.29800.52020.40400.88310.89230.86120.90070.8579
Lung0.53700.37940.32970.29330.29700.44890.27560.23110.24000.21330.92850.88610.91530.95130.9037
Liver0.49740.22920.23930.36650.36490.37710.16770.15940.25520.25730.85250.76040.83070.88980.8258
Breast TNBC0.62000.60000.54000.48000.38000.48000.52000.44000.44000.28000.96010.97020.97930.96390.9464
Kidney0.40450.51740.48250.42940.44720.28230.38010.34890.31200.32020.74190.71050.79170.82680.7128
Adipose Tissue0.58530.41510.45740.44100.37440.44580.30730.33750.32500.26460.83540.78210.86540.86880.8113
Bone Marrow0.55630.33580.29730.44340.38100.40400.23870.21670.31510.28200.74760.53110.63960.74730.6543
Mean0.50840.43540.39280.45250.40420.38250.33450.29030.34560.29880.84910.77690.82130.86100.8140

Table 19. Performance comparison across datasets of scGraph-RWR. Best result per dataset and best mean performance are shown in bold.

MethodPearson ↑\uparrowSpearman ↑\uparrowKendall’s τ\tau ↑\uparrow
Ours0.3820 ± 0.14500.3429 ± 0.14240.2581 ± 0.1025
Geneformer0.3769 ± 0.15800.3388 ± 0.16270.2552 ± 0.1175
CellPLM0.5047 ± 0.17050.4251 ± 0.17650.3247 ± 0.1302
scCello0.3286 ± 0.15430.2901 ± 0.14290.2167 ± 0.1026
scGPT0.3056 ± 0.14700.2435 ± 0.13610.1823 ± 0.0980

Table 22. Spatial gene imputation performance (mean ± std) of Pearson, Spearman, and Kendall’s τ\tau correlations in SEA-AD dataset. Higher is better (↑). Best results are shown in bold.

ModelMLMDynamicMarginOntoAlignscIB ↑\uparrowscGraph-LCA ↑\uparrow
MLM only✅❌❌0.75970.4237
w/o OntoAlign✅✅❌0.80660.3607
w/o DynamicMargin✅❌✅0.78410.7211
scOntoFM✅✅✅0.79580.6112

Table 23. Ablation study of model loss components.

DatasetscIB scOntoFMscIB w/o OntoAlignscIB w/o DynamicMarginscIB MLM onlyscGraph-LCA scOntoFMscGraph-LCA w/o OntoAlignscGraph-LCA w/o DynamicMarginscGraph-LCA MLM only
HIHA0.75330.77450.78480.74030.71570.42820.83010.4787
Heart0.83240.85490.84800.79080.70740.41710.68240.5511
Pancreas0.81400.83880.77670.71910.39920.39570.44080.3351
Lung0.84670.83320.81190.80910.67770.26890.86220.4425
Liver0.77300.77730.78370.75530.57040.36910.69110.4847
Breast_TNBC0.87240.87590.82720.85300.51120.27920.74780.2966
Kidney0.71850.72520.70370.66290.63780.33330.73330.3638
Adipose_tissue0.86960.86430.81850.81540.61800.33890.83450.3986
Bone_marrow0.68230.71530.70210.69150.66300.41550.66730.4624
Mean0.79580.80660.78410.75970.61120.36070.72110.4237

Table 24. Ablation study of the model’s loss terms across the nine evaluation datasets of the overall scIB score and scGraph-LCA metric.

NeurIPS Paper Checklist

Claims

Question: Do the main claims made in the abstract and introduction accurately reflect the paper’s contributions and scope?

Answer: [Yes]

Justification: In this paper, we propose scOntoFM, a novel foundation model that integrates hierarchical cell ontologies directly into the pre-training framework. This main claim is accurately and clearly stated and emphasized in the abstract and introduction section. It reflects our core contribution and scopes.

Guidelines:

  • The answer [N/A] means that the abstract and introduction do not include the claims made in the paper.

  • The abstract and/or introduction should clearly state the claims made, including the contributions made in the paper and important assumptions and limitations. A [No] or [N/A] answer to this question will not be perceived well by the reviewers.

  • The claims made should match theoretical and experimental results, and reflect how much the results can be expected to generalize to other settings.

  • It is fine to include aspirational goals as motivation as long as it is clear that these goals are not attained by the paper.

Limitations

Question: Does the paper discuss the limitations of the work performed by the authors?

Answer: [Yes]

Justification: The paper includes an explicit Limitations paragraph in Section 5, discussing reliance on explicit cell-type labels and scalability constraints from the indexing map, and outlining future directions.

Guidelines:

  • The answer [N/A] means that the paper has no limitation while the answer [No] means that the paper has limitations, but those are not discussed in the paper.

  • The authors are encouraged to create a separate “Limitations” section in their paper.

  • The paper should point out any strong assumptions and how robust the results are to violations of these assumptions (e.g., independence assumptions, noiseless settings, model well-specification, asymptotic approximations only holding locally). The authors should reflect on how these assumptions might be violated in practice and what the implications would be.

  • The authors should reflect on the scope of the claims made, e.g., if the approach was only tested on a few datasets or with a few runs. In general, empirical results often depend on implicit assumptions, which should be articulated.

  • The authors should reflect on the factors that influence the performance of the approach. For example, a facial recognition algorithm may perform poorly when image resolution is low or images are taken in low lighting. Or a speech-to-text system might not be used reliably to provide closed captions for online lectures because it fails to handle technical jargon.

  • The authors should discuss the computational efficiency of the proposed algorithms and how they scale with dataset size.

  • If applicable, the authors should discuss possible limitations of their approach to address problems of privacy and fairness.

  • While the authors might fear that complete honesty about limitations might be used by reviewers as grounds for rejection, a worse outcome might be that reviewers discover limitations that aren’t acknowledged in the paper. The authors should use their best judgment and recognize that individual actions in favor of transparency play an important role in developing norms that preserve the integrity of the community. Reviewers will be specifically instructed to not penalize honesty concerning limitations.

Theory assumptions and proofs

Question: For each theoretical result, does the paper provide the full set of assumptions and a complete (and correct) proof?

Answer: [N/A]

Justification: This paper does not present theorem- or proposition-style theoretical claims requiring formal proofs. Instead, it primarily focuses on methodological development and empirical evaluation in the context of computational biology and single-cell foundation models.

Guidelines:

  • The answer [N/A] means that the paper does not include theoretical results.

  • All the theorems, formulas, and proofs in the paper should be numbered and cross-referenced.

  • All assumptions should be clearly stated or referenced in the statement of any theorems.

  • The proofs can either appear in the main paper or the supplemental material, but if they appear in the supplemental material, the authors are encouraged to provide a short proof sketch to provide intuition.

  • Inversely, any informal proof provided in the core of the paper should be complemented by formal proofs provided in appendix or supplemental material.

  • Theorems and Lemmas that the proof relies upon should be properly referenced.

Experimental result reproducibility

Question: Does the paper fully disclose all the information needed to reproduce the main experimental results of the paper to the extent that it affects the main claims and/or conclusions of the paper (regardless of whether the code and data are provided or not)?

Answer: [Yes]

Justification: We provide all necessary details to reproduce our work. The method is fully specified in Section 2 (including objectives), and experimental setup details are provided in Section 3 plus Appendix § 8/Appendix § 9 (model/training configuration, dataset lists/sources, task protocols, and additional quantitative results).

Guidelines:

  • The answer [N/A] means that the paper does not include experiments.

  • If the paper includes experiments, a [No] answer to this question will not be perceived well by the reviewers: Making the paper reproducible is important, regardless of whether the code and data are provided or not.

  • If the contribution is a dataset and/or model, the authors should describe the steps taken to make their results reproducible or verifiable.

  • Depending on the contribution, reproducibility can be accomplished in various ways. For example, if the contribution is a novel architecture, describing the architecture fully might suffice, or if the contribution is a specific model and empirical evaluation, it may be necessary to either make it possible for others to replicate the model with the same dataset, or provide access to the model. In general, releasing code and data is often one good way to accomplish this, but reproducibility can also be provided via detailed instructions for how to replicate the results, access to a hosted model (e.g., in the case of a large language model), releasing of a model checkpoint, or other means that are appropriate to the research performed.

  • While NeurIPS does not require releasing code, the conference does require all submissions to provide some reasonable avenue for reproducibility, which may depend on the nature of the contribution. For example (a) If the contribution is primarily a new algorithm, the paper should make it clear how to reproduce that algorithm. (b) If the contribution is primarily a new model architecture, the paper should describe the architecture clearly and fully. (c) If the contribution is a new model (e.g., a large language model), then there should either be a way to access this model for reproducing the results or a way to reproduce the model (e.g., with an open-source dataset or instructions for how to construct the dataset).

(d) We recognize that reproducibility may be tricky in some cases, in which case authors are welcome to describe the particular way they provide for reproducibility. In the case of closed-source models, it may be that access to the model is limited in some way (e.g., to registered users), but it should be possible for other researchers to have some path to reproducing or verifying the results.

Open access to data and code

Question: Does the paper provide open access to the data and code, with sufficient instructions to faithfully reproduce the main experimental results, as described in supplemental material?

Answer: [No]

Justification: Our code and datasets will be released upon acceptance.

Guidelines:

  • The answer [N/A] means that paper does not include experiments requiring code.

  • Please see the NeurIPS code and data submission guidelines (https://neurips.cc/public/guides/CodeSubmissionPolicy) for more details.

  • While we encourage the release of code and data, we understand that this might not be possible, so [No] is an acceptable answer. Papers cannot be rejected simply for not including code, unless this is central to the contribution (e.g., for a new open-source benchmark).

  • The instructions should contain the exact command and environment needed to run to reproduce the results. See the NeurIPS code and data submission guidelines (https://neurips.cc/public/guides/CodeSubmissionPolicy) for more details.

  • The authors should provide instructions on data access and preparation, including how to access the raw data, preprocessed data, intermediate data, and generated data, etc.

  • The authors should provide scripts to reproduce all experimental results for the new proposed method and baselines. If only a subset of experiments are reproducible, they should state which ones are omitted from the script and why.

  • At submission time, to preserve anonymity, the authors should release anonymized versions (if applicable).

  • Providing as much information as possible in supplemental material (appended to the paper) is recommended, but including URLs to data and code is permitted.

Experimental setting/details

Question: Does the paper specify all the training and test details (e.g., data splits, hyperparameters, how they were chosen, type of optimizer) necessary to understand the results?

Answer: [Yes]

Justification: All the Experimental setting/details about pre-training and downstream tasks are illustrated in Appendix B.

Guidelines:

  • The answer [N/A] means that the paper does not include experiments.

  • The experimental setting should be presented in the core of the paper to a level of detail that is necessary to appreciate the results and make sense of them.

  • The full details can be provided either with the code, in appendix, or as supplemental material.

Experiment statistical significance

Question: Does the paper report error bars suitably and correctly defined or other appropriate information about the statistical significance of the experiments?

Answer: [Yes]

Justification: The paper reports uncertainty for key experiments using mean ±\pm standard deviation (e.g., Table 4 with 20-fold CV for gene function prediction, Table 5 for spatial imputation), with additional distributional violin plots in Figure 15 for hallmark gene set enrichment score, and error bars in Figure 4 for novel cell identification, as well the statistical significance test in Figure 16 for spatial gene imputation performance.

Guidelines:

  • The answer [N/A] means that the paper does not include experiments.

  • The authors should answer [Yes] if the results are accompanied by error bars, confidence intervals, or statistical significance tests, at least for the experiments that support the main claims of the paper.

  • The factors of variability that the error bars are capturing should be clearly stated (for example, train/test split, initialization, random drawing of some parameter, or overall run with given experimental conditions).

  • The method for calculating the error bars should be explained (closed form formula, call to a library function, bootstrap, etc.)

  • The assumptions made should be given (e.g., Normally distributed errors).

  • It should be clear whether the error bar is the standard deviation or the standard error of the mean.

  • It is OK to report 11-sigma error bars, but one should state it. The authors should preferably report a 22-sigma error bar than state that they have a 96%96\% CI, if the hypothesis of Normality of errors is not verified.

  • For asymmetric distributions, the authors should be careful not to show in tables or figures symmetric error bars that would yield results that are out of range (e.g., negative error rates).

  • If error bars are reported in tables or plots, the authors should explain in the text how they were calculated and reference the corresponding figures or tables in the text.

Experiments compute resources

Question: For each experiment, does the paper provide sufficient information on the computer resources (type of compute workers, memory, time of execution) needed to reproduce the experiments?

Answer: [Yes]

Justification: We report the major pre-training details of scOntoFM (six NVIDIA H200 GPUs, effective batch size, and ∼\sim30-hour pretraining). We also report the computational resources required when constructing the Cell-Dataset Index Map, which takes about 5 minutes and uses 6.5 GB of peak memory on an Intel Xeon Gold 6252 CPU. The final file size is 132 MB.

Guidelines:

  • The answer [N/A] means that the paper does not include experiments.

  • The paper should indicate the type of compute workers CPU or GPU, internal cluster, or cloud provider, including relevant memory and storage.

  • The paper should provide the amount of compute required for each of the individual experimental runs as well as estimate the total compute.

  • The paper should disclose whether the full research project required more compute than the experiments reported in the paper (e.g., preliminary or failed experiments that didn’t make it into the paper).

Code of ethics

Question: Does the research conducted in the paper conform, in every respect, with the NeurIPS Code of Ethics https://neurips.cc/public/EthicsGuidelines?

Answer: [Yes]

Justification: We follow the NeurIPS Code of Ethics throughout the entire project.

Guidelines:

  • The answer [N/A] means that the authors have not reviewed the NeurIPS Code of Ethics.

  • If the authors answer [No], they should explain the special circumstances that require a deviation from the Code of Ethics.

  • The authors should make sure to preserve anonymity (e.g., if there is a special consideration due to laws or regulations in their jurisdiction).

Broader impacts

Question: Does the paper discuss both potential positive societal impacts and negative societal impacts of the work performed?

Answer: [Yes]

Justification: Broader impacts are addressed throughout the introduction and discussion. And we also provide a detailed impact statement in Appendix § 10.

Guidelines:

  • The answer [N/A] means that there is no societal impact of the work performed.

  • If the authors answer [N/A] or [No], they should explain why their work has no societal impact or why the paper does not address societal impact.

  • Examples of negative societal impacts include potential malicious or unintended uses (e.g., disinformation, generating fake profiles, surveillance), fairness considerations (e.g., deployment of technologies that could make decisions that unfairly impact specific groups), privacy considerations, and security considerations.

  • The conference expects that many papers will be foundational research and not tied to particular applications, let alone deployments. However, if there is a direct path to any negative applications, the authors should point it out. For example, it is legitimate to point out that an improvement in the quality of generative models could be used to generate Deepfakes for disinformation. On the other hand, it is not needed to point out that a generic algorithm for optimizing neural networks could enable people to train models that generate Deepfakes faster.

  • The authors should consider possible harms that could arise when the technology is being used as intended and functioning correctly, harms that could arise when the technology is being used as intended but gives incorrect results, and harms following from (intentional or unintentional) misuse of the technology.

  • If there are negative societal impacts, the authors could also discuss possible mitigation strategies (e.g., gated release of models, providing defenses in addition to attacks, mechanisms for monitoring misuse, mechanisms to monitor how a system learns from feedback over time, improving the efficiency and accessibility of ML).

Safeguards

Question: Does the paper describe safeguards that have been put in place for responsible release of data or models that have a high risk for misuse (e.g., pre-trained language models, image generators, or scraped datasets)?

Answer: [N/A]

Justification: This work focuses on representation learning for single-cell transcriptomic analysis and does not involve high-risk generative modeling or biological sequence design. The released code and model checkpoints are intended solely for research use in computational biology. All datasets used in this study are publicly available or distributed under their respective data usage agreements, and no identifiable personal information is included. As such, we do not believe the model poses substantial misuse risks requiring specialized safeguards.

Guidelines:

  • The answer [N/A] means that the paper poses no such risks.

  • Released models that have a high risk for misuse or dual-use should be released with necessary safeguards to allow for controlled use of the model, for example by requiring that users adhere to usage guidelines or restrictions to access the model or implementing safety filters.

  • Datasets that have been scraped from the Internet could pose safety risks. The authors should describe how they avoided releasing unsafe images.

  • We recognize that providing effective safeguards is challenging, and many papers do not require this, but we encourage authors to take this into account and make a best faith effort.

Licenses for existing assets

Question: Are the creators or original owners of assets (e.g., code, data, models), used in the paper, properly credited and are the license and terms of use explicitly mentioned and properly respected?

Answer: [Yes]

Justification: All external assets used in this work, including datasets, software libraries, and pre-existing models, are appropriately cited and credited in the paper.

Guidelines:

  • The answer [N/A] means that the paper does not use existing assets.

  • The authors should cite the original paper that produced the code package or dataset.

  • The authors should state which version of the asset is used and, if possible, include a URL.

  • The name of the license (e.g., CC-BY 4.0) should be included for each asset.

  • For scraped data from a particular source (e.g., website), the copyright and terms of service of that source should be provided.

  • If assets are released, the license, copyright information, and terms of use in the package should be provided. For popular datasets, paperswithcode.com/datasets has curated licenses for some datasets. Their licensing guide can help determine the license of a dataset.

  • For existing datasets that are re-packaged, both the original license and the license of the derived asset (if it has changed) should be provided.

  • If this information is not available online, the authors are encouraged to reach out to the asset’s creators.

  1. New assets

Question: Are new assets introduced in the paper well documented and is the documentation provided alongside the assets?

Answer: [No]

Justification: Code and datasets will be released upon acceptance. Clear documentations will be provided along the assets.

Guidelines:

  • The answer [N/A] means that the paper does not release new assets.

  • Researchers should communicate the details of the dataset/code/model as part of their submissions via structured templates. This includes details about training, license, limitations, etc.

  • The paper should discuss whether and how consent was obtained from people whose asset is used.

  • At submission time, remember to anonymize your assets (if applicable). You can either create an anonymized URL or include an anonymized zip file.

  1. Crowdsourcing and research with human subjects

Question: For crowdsourcing experiments and research with human subjects, does the paper include the full text of instructions given to participants and screenshots, if applicable, as well as details about compensation (if any)?

Answer: [N/A]

Justification: We did not perform crowdsourcing experiments and research with human subjects.

Guidelines:

  • The answer [N/A] means that the paper does not involve crowdsourcing nor research with human subjects.

  • Including this information in the supplemental material is fine, but if the main contribution of the paper involves human subjects, then as much detail as possible should be included in the main paper.

  • According to the NeurIPS Code of Ethics, workers involved in data collection, curation, or other labor should be paid at least the minimum wage in the country of the data collector.

15. Institutional review board (IRB) approvals or equivalent for research with human subjects

Question: Does the paper describe potential risks incurred by study participants, whether such risks were disclosed to the subjects, and whether Institutional Review Board (IRB) approvals (or an equivalent approval/review based on the requirements of your country or institution) were obtained?

Answer: [N/A]

Justification: This paper does not involve crowdsourcing nor research with human subjects.

Guidelines:

  • The answer [N/A] means that the paper does not involve crowdsourcing nor research with human subjects.

  • Depending on the country in which research is conducted, IRB approval (or equivalent) may be required for any human subjects research. If you obtained IRB approval, you should clearly state this in the paper.

  • We recognize that the procedures for this may vary significantly between institutions and locations, and we expect authors to adhere to the NeurIPS Code of Ethics and the guidelines for their institution.

  • For initial submissions, do not include any information that would break anonymity (if applicable), such as the institution conducting the review.

16. Declaration of LLM usage

Question: Does the paper describe the usage of LLMs if it is an important, original, or non-standard component of the core methods in this research? Note that if the LLM is used only for writing, editing, or formatting purposes and does not impact the core methodology, scientific rigor, or originality of the research, declaration is not required.

Answer: [N/A]

Justification: Large language models (LLMs) are not a component of the proposed methodology, experimental pipeline, or scientific contributions of this work.

Guidelines:

  • The answer [N/A] means that the core method development in this research does not involve LLMs as any important, original, or non-standard components.

  • Please refer to our LLM policy in the NeurIPS handbook for what should or should not be described.

References

  1. [1]Shibla Abdulla, Brian Aevermann, Pedro Assis, Seve Badajoz, Sidney M. Bell, Emanuele Bezzi, Batuhan Cakir, Jim Chaffer, Signe Chambers, J. Michael Cherry, Tiffany Chi, Jennifer Chien, Leah Dorman, Pablo Garcia-Nieto, Navib Glorja, Mim Hastie, Daniel Hegeman, Jason Hilton, Timmy Huang, Amanda Infeld, Ana-Maria Istrate, Ivana Jelic, Kuni Katsuya, Yang Joon Kim, Karen Liang, Mike Lin, Maximilian Lombardo, Bailey Marshall, Bruce Martin, Fran McDade, Colin Megill, Nikhil Patel, Alexander Predeus, Brian Raymor, Behnam Robatmili, Dave Rogers, Erica Rutherford, Dana Sadgat, Andrew Shih, Corinn Small, Trent Smith, Prathap Sridharan, Alexander Tarashansky, Norbert Tavares, Harley Thomas, Andrew Tolopko, Meghan Urisko, Joyce Yan, Garabet Yeretsian, Jennifer Zamanian, Arathi Mani, Jonah Cool, and Ambrose Carr. Cz cellxgene discover: A single-cell data platform for scalable exploration, analysis and modeling of aggregated data. November 2023. doi: 10.1101/2023.10.30.563174. URL http://dx.doi.org/10.1101/2023.10.30.563174.
  2. [3]Pau Badia-i Mompel, Jesús Vélez Santiago, Jana Braunger, Celina Geiss, Daniel Dimitrov, Sophia Müller-Dott, Petr Taus, Aurelien Dugourd, Christian H Holland, Ricardo O Ramirez Flores, and Julio Saez-Rodriguez. decoupler: ensemble of computational methods to infer biological activities from omics data. Bioinformatics Advances, 2(1), January 2022. ISSN 2635-0041. doi: 10.1093/bioadv/vbac016. URL http://dx.doi.org/10.1093/bioadv/vbac016.
  3. [4]Khushdeep Bandesh, Efthymios Motakis, Siddhi Nargund, Romy Kursawe, Vijay Selvam, Redwan M Bhuiyan, Giray Naim Ervilmaz, Sai Nivedita Krishnan, Cassandra N. Spracklen, Duygu Ucar, and Michael L. Stitzel. Single-cell decoding of human islet cell type-specific alterations in type 2 diabetes reveals converging genetic- and state-driven β-cell gene expression defects. January 2025. doi: 10.1101/2025.01.17.633590. URL http://dx.doi.org/10.1101/2025.01.17.633590.
  4. [5]Luca Bertinetto, Romain Mueller, Konstantinos Tertikas, Sina Samangooei, and Nicholas A. Lord. Making better mistakes: Leveraging class hierarchies with deep networks. In The IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), June 2020.arxiv.org/abs/1912.09393
  5. [6]Mathieu Blondel, Olivier Teboul, Quentin Berthet, and Josip Djolonga. Fast differentiable sorting and ranking. In International Conference on Machine Learning, pages 950–959. PMLR, 2020.arxiv.org/abs/2002.08871
  6. [7]Junfan Chen, Fabian Schmidt, and Ricardo Henao. Assessing scale and predictive diversity in models for single-cell transcriptomics based on geneformer. PLOS Computational Biology, 22(7):e1013701, 2026. ISSN 1553-7358. doi: 10.1371/journal.pcbi.1013701. URL http://dx.doi.org/10.1371/journal.pcbi.1013701.
  7. [8]Ting Chen, Simon Kornblith, Mohammad Norouzi, and Geoffrey Hinton. A simple framework for contrastive learning of visual representations. In International conference on machine learning, pages 1597–1607. PMLR, 2020.arxiv.org/abs/2002.05709
  8. [9]Yiqun Chen and James Zou. Simple and effective embedding model for single-cell biology built from chatgpt. Nature Biomedical Engineering, 9(4):483–493, December 2024. ISSN 2157-846X. doi: 10.1038/s41551-024-01284-6. URL http://dx.doi.org/10.1038/s41551-024-01284-6.
  9. [10]Haotian Cui, Chloe Wang, Hassaan Maan, Kuan Pang, Fengning Luo, Nan Duan, and Bo Wang. scGPT: toward building a foundation model for single-cell multi-omics using generative ai. Nature Methods, 21(8):1470–1480, February 2024. ISSN 1548-7105. doi: 10.1038/s41592-024-02201-0. URL http://dx.doi.org/10.1038/s41592-024-02201-0.
  10. [11]Sebastiano Cultrera di Montesano, Davide D’Ascెనzo, Srivatsan Raghavan, Ava P. Amini, Peter S. Winter, and Lorin Crawford. Improving atlas-scale single-cell annotation models with hierarchical cross-entropy loss. Nature Computational Science, 6(3):243–249, January 2026. ISSN 2662-8457. doi: 10.1038/s43588-025-00945-z. URL http://dx.doi.org/10.1038/s43588-025-00945-z.
  11. [12]Jiyuan Devlin, Ming-Wei Chang, Kenton Lee, and Kristina Toutanova. BERT: Pre-training of deep bidirectional transformers for language understanding. In Jill Burstein, Christy Doran, and Thamar Solorio, editors, Proceedings of the 2019 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, Volume 1 (Long and Short Papers), pages 4171–4186, Minneapolis, Minnesota, June 2019. Association for Computational Linguistics. doi: 10.18653/v1/N19-1423. URL https://aclanthology.org/N19-1423/.DOI
  12. [13]Jiayuan Ding, Jianhui Lin, Shiyu Jiang, Yixin Wang, Ziyang Miao, Zhaoyu Fang, Jiliang Tang, Min Li, and Xiaojie Qiu. Tabula: A tabular self-supervised foundation model for single-cell transcriptomics. In The Thirty-ninth Annual Conference on Neural Information Processing Systems, 2026. URL https://openreview.net/forum?id=Vk2sfKAdEu.DOI
  13. [14]Joseph R. Ecker, Daniel H. Geschwind, Arnold R. Kriegstein, John Ngai, Pavel Osten, Damon Polioudakis, Aviv Regev, Nenad Sestan, Ian R. Wickersham, and Hongkui Zeng. The brain initiative cell census consortium: Lessons learned toward generating a comprehensive brain cell atlas. Neuron, 96(3):542–557, 2017. ISSN 0896-6273. doi: https://doi.org/10.1016/j.neuron.2017.10.001. URL https://www.sciencedirect.com/science/article/pii/S0896627317309364.
  14. [15]Rachel D. Edgar, Diana Nakib, Damra Camat, Sai Chung, Patricia Lumanto, Jawairia Atif, Catia T. Perciani, Xue-Zhong Ma, Cornelia Thoeni, Nilosa Selvakumaran, Justin Manuel, Blayne Sayed, Koen Huysentruyt, Amanda Ricciuto, Ian McGilvray, Yaron Avitzur, Gary D. Bader, and Sonya A. MacParland. Single-cell atlas of human pediatric liver reveals age-related hepatic gene signatures. Hepatology Communications, 9(11), October 2025. ISSN 2471-254X. doi: 10.1097/hc9.0000000000000813. URL http://dx.doi.org/10.1097/HC9.0000000000000813.
  15. [16]Margo P. Emont, Christopher Jacobs, Adam L. Essene, Deepti Pant, Danielle Tenen, Georgia Colleluori, Angelica Di Vincenzo, Anja M. Jørgensen, Hesam Dashti, Adam Stepek, Elizabeth McGonagle, Sophie Strobel, Samantha Laber, Saaket Agrawal, Gregory P. Westcott, Amrita Kar, Molly L. Veregge, Anton Gulko, Harini Srinivasan, Zachary Kramer, Eleanna De Filippis, Erin Merkel, Jennifer Ducie, Christopher G. Boyd, William Gourash, Anita Courcoulas, Samuel J. Lin, Bernard T. Lee, Donald Morris, Adam Tobias, Amit V. Khera, Melina Claussnitzer, Tune H. Pers, Antonio Giordano, Orr Ashenberg, Aviv Regev, Linus T. Tsai, and Evan D. Rosen. A single-cell atlas of human and mouse white adipose tissue. Nature, 603(7903):926–933, March 2022. ISSN 1476-4687. doi: 10.1038/s41586-022-04518-2. URL http://dx.doi.org/10.1038/s41586-022-04518-2.
  16. [17]Mariano I. Gabitto, Kyle J. Travaglini, Victoria M. Rachleff, Eitan S. Kaplan, Brian Long, Jeanella Ariza, Yi Ding, Joseph T. Mahoney, Nick Dee, Jeff Goldy, Erica J. Melief, Anamika Agrawal, Omar Kana, Xingjian Zhen, Samuel T. Barlow, Krissy Brouner, Jazmin Campos, John Campos, Ambrose J. Carr, Tamara Casper, Rushil Chakrabarty, Michael Clark, Jonah Cool, Rachel D. Alley, Martin Darvas, Song-Lin Ding, Tim Dolbeare, Tom Egdorf, Luke Esposito, Rebecca Ferrer, Lynn E. Fleckenstein, Rohan Gala, Amanda Gary, Emily Gelfand, Jessica Gloe, Nathan Guilford, Junitta Guzman, Daniel Hirschstein, Windy Ho, Madison Hupp, Tim Jarsky, Nelson Johansen, Brian E. Kalmbach, Lisa M. Keene, Sarah Khawand, Mitchell D. Kilgore, Amanda Kirkland, Michael Kunst, Brian R. Lee, Mckaila Leytze, Christine L. Mac Donald, Jocelin Malone, Zoe Maltzer, Naomi Martin, Rachel McCue, Delissa McMillen, Gonzalo Mena, Emma Meyerdierks, Kelly P. Meyers, Tyler Mollenkopf, Mark Montine, Amber L. Nolan, Julie K. Nyhus, Paul A. Olsen, Maiya Pacleb, Chelsea M. Pagan, Nicholas Peña, Trangthanh Pham, Christina Alice Pom, Nadia Postupna, Christine Rimorin, Augustin Ruiz, Giuseppe A. Saldi, Aimee M. Schantz, Nadiya V. Shapovalova, Staci A. Sorensen, Brian Staats, Matt Sullivan, Susan M. Sunkin, Carol Thompson, Michael Tieu, Jonathan T. Ting, Amy Torkelson, Tracy Tran, Nasmil J. Valera Cuevas, Sarah Walling-Bell, Ming-Qiang Wang, Jack Waters, Angela M. Wilson, Ming Xiao, David Haynor, Nicole M. Gatto, Suman Jayadev, Shoaib Mufti, Lydia Ng, Shubhahrata Mukherjee, Paul K. Crane, Caitlin S. Latimer, Boaz P. Levi, Kimberly A. Smith, Jennie L. Close, Jeremy A. Miller, Rebecca D. Hodge, Eric B. Larson, Thomas C. Grabowski, Michael Hawrylycz, C. Dirk Keene, and Ed S. Lein. Integrated multimodal cell atlas of alzheimer’s disease. Nature Neuroscience, 27(12):2366–2383, October 2024. ISSN 1546-1726. doi: 10.1038/s41593-024-01774-5. URL http://dx.doi.org/10.1038/s41593-024-01774-5.
  17. [18]Weifeng Ge. Deep metric learning with hierarchical triplet loss. In Proceedings of the European Conference on Computer Vision (ECCV), September 2018.arxiv.org/abs/1810.06951
  18. [19]Qiuyu Gong, Mehul Sharma, Marla C. Glass, Emma L. Kuan, Aishwarya Chander, Mansi Singh, Lucas T. Graybuck, Zachary J. Thomson, Christian M. LaFrance, Samir Rachid Zaim, Tao Peng, Lauren Y. Okada, Palak C. Genge, Katherine E. Henderson, Elisabeth M. Dornisch, Erik D. Layton, Peter J. Wittig, Alexander T. Heubeck, Nelson M. Mukuka, Julian Reading, Garrett Strawn, Teminjesu Titus-Adewunmi, Kathleen Abadie, Charles R. Roll, Veronica Hernandez, Vaishnavi Parthasarathy, Tyanna J. Stuckey, Blessing Musgrove, Elliott Swanson, Cara Lord, Morgan D. A. Weiss, Cole G. Phalen, Regina R. Mettey, Kevin J. Lee, John B. Johanneson, Erin K. Kawelo, Jessica Garber, Upaasana Krishnan, Megan Smithmyer, E. John Wherry, Laura A. Vella, Sarah E. Henrickson, Mackenzie S. Kopp, Adam K. Savage, Lynne A. Becker, Paul Meijer, Ernest M. Coffey, Jorg J. Goronzy, Mikael Sigvardsson, Cate Speake, Thomas F. Bumol, Ananda W. Goldrath, Troy R. Torgerson, Xiao-jun Li, Peter J. Skene, Jane H. Buckner, and Claire E. Gustafson. Multi-omic profiling reveals age-related immune dynamics in healthy adults. Nature, 648(8094):696–706, October 2025. ISSN 1476-4687. doi: 10.1038/s41586-025-09686-5. URL http://dx.doi.org/10.1038/s41586-025-09686-5.
  19. [20]G. Kenneth Gray, Carman Man-Chung Li, Jennifer M. Rosenbluth, Laura M. Selfors, Nomeda Girnius, Jia-Ren Lin, Ron C.J. Schackmann, Walter L. Goh, Kaitlin Moore, Hana K. Shapiro, Shaolin Mei, Kurt D’Andrea, Katherine L. Nathanson, Peter K. Sorger, Sandro Santagata, Aviv Regev, Judy E. Garber, Deborah A. Dillon, and Joan S. Brugge. A human breast atlas integrating single-cell proteomics and transcriptomics. Developmental Cell, 57(11):1400–1420.e7, June 2022. ISSN 1534-5807. doi: 10.1016/j.devcel.2022.05.003. URL http://dx.doi.org/10.1016/j.devcel.2022.05.003.
  20. [21]Xiangyu Guo and Ricardo Henao. Cross-modal imputation and uncertainty estimation for spatial transcriptomics. In Yingzhen Li, Stephan Mandt, Shipra Agrawal, and Emtiyaz Khan, editors, Proceedings of The 28th International Conference on Artificial Intelligence and Statistics, volume 258 of Proceedings of Machine Learning Research, pages 4375–4383. PMLR, 03–05 May 2025. URL https://proceedings.mlr.press/v258/guo25a.html.
  21. [22]Minsheng Hao, Jing Gong, Xin Zeng, Chiming Liu, Yucheng Guo, Xingyi Cheng, Taifeng Wang, Jianzhu Ma, Xuegong Zhang, and Le Song. Large-scale foundation model on single-cell transcriptomics. Nature Methods, 21(8):1481–1491, June 2024. ISSN 1548-7105. doi: 10.1038/s41592-024-02305-7. URL http://dx.doi.org/10.1038/s41592-024-02305-7.
  22. [23]Yuhan Hao, Stephanie Hao, Erica Andersen-Nissen, William M. Mauck, Shiwei Zheng, Andrew Butler, Maddie J. Lee, Aaron J. Wilk, Charlotte Darby, Michael Zager, Paul Hoffman, Marlon Stoeckius, Efthymia Papalexi, Eleni P. Mimitou, Jaison Jain, Avi Srivastava, Tim Stuart, Lamar M. Fleming, Bertrand Yeung, Angela J. Rogers, Juliana M. McErath, Catherine A. Blish, Raphael Gottardo, Peter Smibert, and Rahul Satija. Integrated analysis of multimodal single-cell data. Cell, 184(13):3573–3587.e29, 2021. ISSN 0092-8674. doi: 10.1016/j.cell.2021.04.048. URL http://dx.doi.org/10.1016/j.cell.2021.04.048.
  23. [24]Graham Heimberg, Tony Kuo, Daryle J. DePianto, Omar Salem, Tobias Heigl, Nathaniel Diamant, Gabriele Scalia, Tommaso Biancalani, Shannon J. Turley, Jason R. Rock, Héctor Corrada Bravo, Josh Kaminker, Jason A. Vander Heiden, and Aviv Regev. A cell atlas foundation model for scalable search of similar human cells. Nature, 638(8052):1085–1094, November 2024. ISSN 1476-4687. doi: 10.1038/s41586-024-08411-y. URL http://dx.doi.org/10.1038/s41586-024-08411-y.
  24. [25]Dragomirka Jovic, Xue Liang, Hua Zeng, Lin Lin, Fengping Xu, and Yonglun Luo. Single-cell rna sequencing technologies and applications: A brief overview. Clinical and Translational Medicine, 12(3), March 2022. ISSN 2001-1326. doi: 10.1002/ctm2.694. URL http://dx.doi.org/10.1002/ctm2.694.
  25. [26]Kazumasa Kanemaru, James Cranley, Daniele Muraro, Antonio M. A. Miranda, Siew Yen Ho, Anna Wilbrey-Clark, Jan Patrick Pett, Krzysztof Polanski, Laura Richardson, Monika Litvinukova, Natshuhiko Kumasaka, Yue Qin, Zuzanna Jablonska, Claudia I. Semprich, Lukas Mach, Monika Dabrowska, Nathan Richoz, Liam Bolt, Lira Mamanova, Rakeshial Kapuge, Sam N. Barnett, Shani Perera, Carlos Talavera-López, Ilaria Mulas, Krishnaa T. Mahbubani, Liz Tuck, Lu Wang, Margaret M. Huang, Martin Prete, Sophie Pritchard, John Dark, Kourosh Saeb-Parsy, Minal Patel, Menna R. Clatworthy, Norbert Hübner, Rasheda A. Chowdhury, Michela Noseda, and Sarah A. Teichmann. Spatially resolved multiomics of human cardiac niches. Nature, 619(7971):801–810, July 2023. ISSN 1476-4687. doi: 10.1038/s41586-023-06311-1. URL http://dx.doi.org/10.1038/s41586-023-06311-1.
  26. [27]Nayoung Kim, Hong Kwan Kim, Kyungjong Lee, Yourae Hong, Jong Ho Cho, Jung Won Choi, Jung-Il Lee, Yeon-Lim Suh, Bo Mi Ku, Hye Hyeon Eum, Soyean Choi, Yoon-La Chae, Je-Gun Joung, Woong-Yang Park, Hyun Ae Jung, Jong-Mu Sun, Se-Hoon Lee, Jin Seok Ahn, Keunchil Park, Myung-Ju Ahn, and Hae-Ock Lee. Single-cell rna sequencing demonstrates the molecular and cellular reprogramming of metastatic lung adenocarcinoma. Nature Communications, 11(1), May 2020. ISSN 2041-1723. doi: 10.1038/s41467-020-16164-1. URL http://dx.doi.org/10.1038/s41467-020-16164-1.
  27. [28]Blue B. Lake, Ricardo Melo Ferreira, Jens Hansen, Rajasree Menon, Jeannine Basta, Heather Thiessen Philbrook, Stephanie Reinert, Robin Fallegger, Asmita K. Lagwankar, Xi Chen, Soumya Vati, Katerina V. Djambazova, Brittany L. Gorman, Nicholas Lucarelli, Debora L. Gisch, Insa M. Schmidt, Viji Nair, Fadhl Alakwaa, Eirini Kefaloyianni, Bo Zhang, Amanda L. Knotten, Madhurima Kaushal, Edgar A. Otto, Melissa A. Farrow, Dinh Diep, Dusan Velickovic, Angela R. Sabo, Elijah Cole, Ian Tamayo, Jovan Tanevski, Kimberly Y. Conklin, Rachel S. G. Sealfon, Yongqun He, Michelle Brennan, Lynn Robbins, Ying-Hua Cheng, Markus Bitzer, Aditya Surapaneni, Steven Menez, Peter V. Kharchenko, Charles E. Alpers, Ulysses G. J. Balis, Laura Barisoni, Ian H. de Boer, Dawit Demeke, Agnes B. Fogo, Joel M. Henderson, Leal Herlitz, Gilbert W. Moeckel, Parmjeet S. Randhawa, Avi Z. Rosenberg, Jennifer A. Schaub, Suman Setty, Frank C. Brosius, Maria L. Caramori, Steven G. Coca, Robert S. Figenshau, Eric H. Kim, Krzysztof Kiryuk, James P. Lash, R. Tyler Miller, John F. O’Toole, Paul M. Palevsky, Eugene P. Rhee, Ana C. Ricardo, Sylvia E. Rosas, Prabir Roy-Chaudhury, Minnie M. Sarwal, John R. Sedor, Robert D. Toto, Adytia Türkmən, Sushrut S. Waikar, James C. Williams, F. P. Wilson, E. Steve Woodle, Evan Z. Macosko, Julio Saez-Rodriguez, Pierre C Dagher, Morgan E. Grams, Petter Bjornstad, Tarek M. El-Achkар, Olga G. Troyanskaya, Nikole Bonevich, Pinaki Sarder, Sanjeev Kumar, Christopher R. Anderton, Jeffrey M. Spraggins, Kumar Sharma, Michael Rauchman, Jonathan Himmelfarb, Joseph P. Gaut, Kidney Precision Medicine Project, Kun Zhang, Ravi Iyengar, Matthias Kretzler, Jeffrey B. Hodgin, Chirag R. Parikh, Michael T. Eadon, Sanjay Jain, HuBMAP consortium, and Kidney Precision Medicine Project. Cellular and spatial drivers of unresolved injury and functional decline in the human kidney. bioRxiv, 2025. doi: 10.1101/2025.09.26.678707. URL https://www.biorxiv.org/content/early/2025/11/24/2025.09.26.678707.DOI
  28. [29]Daniel Levine, Syed A Rizvi, Sacha Lévy, Nazreen Pallikkavalıyaveetil, David Zhang, Xingyu Chen, Sina Ghadermarzi, Ruiming Wu, Zihe Zheng, Ivan Vrkic, Anna Zhong, Daphne Raskin, Insu Han, Antonio Henrique de Oliveira Fonseca, Josue Ortega Caro, Amin Karbasi, Rahul Madhav Dhodapkar, and David van Dijk. Cell2sentence: Teaching large language models the language of biology. In Forty-first International Conference on Machine Learning, 2024. URL https://openreview.net/forum?id=EWt6wsEdvc.DOI
  29. [30]Bin Li, Wen Zhang, Chuang Guo, Hao Xu, Longfei Li, Minghao Fang, Yinlei Hu, Xinye Zhang, Xinfeng Yao, Meifang Tang, Ke Liu, Xuetong Zhao, Jun Lin, Linzhao Cheng, Falai Chen, Tian Xue, and Kun Qun. Benchmarking spatial and single-cell transcriptomics integration methods for transcript distribution prediction and cell type deconvolution. Nature Methods, 19(6):662–670, May 2022. ISSN 1548-7105. doi: 10.1038/s41592-022-01480-9. URL http://dx.doi.org/10.1038/s41592-022-01480-9.
  30. [31]Arthur Liberzon, Aravind Subramanian, Reid Pinchback, Helga Thorvaldsdóttir, Pablo Tamayo, and Jill P. Mesirov. Molecular signatures database (msigdb) 3.0. Bioinformatics, 27(12):1739–1740, May 2011. ISSN 1367-4803. doi: 10.1093/bioinformatics/btr260. URL http://dx.doi.org/10.1093/bioinformatics/btr260.
  31. [32]Malte D. Luecken, M. Büttner, K. Chaichoompu, A. Danese, M. Interlandi, M. F. Mueller, D. C. Strobl, L. Zappia, M. Dugas, M. Colomé-Tatché, and Fabian J. Theis. Benchmarking atlas-level data integration in single-cell genomics. Nature Methods, 19(1):41–50, December 2021. ISSN 1548-7105. doi: 10.1038/s41592-021-01336-8. URL http://dx.doi.org/10.1038/s41592-021-01336-8.
  32. [33]Lieke Michielsen, Mohammad Lotfollahi, Daniel Strobl, Lisa Sikkema, Marcel J T Reinders, Fabian J Theis, and Ahmed Mahfouz. Single-cell reference mapping to construct and extend cell-type hierarchies. NAR Genomics and Bioinformatics, 5(3), 2023. ISSN 2631-9268. doi: 10.1093/nargab/lqad070. URL http://dx.doi.org/10.1093/nargab/lqad070.
  33. [34]David Osumi-Sutherland, Chuan Xu, Maria Keays, Adam P. Levine, Peter V. Kharchenko, Aviv Regev, Ed Lein, and Sarah A. Teichmann. Cell type ontologies of the human cell atlas. Nature Cell Biology, 23(11):1129–1135, November 2021. ISSN 1476-4679. doi: 10.1038/s41556-021-00787-7. URL http://dx.doi.org/10.1038/s41556-021-00787-7.
  34. [35]F. Pedregosa, G. Varoquaux, A. Gramfort, V. Michel, B. Thirion, O. Grisel, M. Blondel, P. Prettenhofer, R. Weiss, V. Dubourg, J. Vanderplas, A. Passos, D. Cournapeau, M. Brucher, M. Perrot, and E. Duchesnay. Scikit-learn: Machine learning in Python. Journal of Machine Learning Research, 12:2825–2830, 2011.arxiv.org/abs/1201.0490
  35. [36]Qiqi Qiao, Shunfeng Hu, and Xin Wang. The regulatory roles and clinical significance of glycolysis in tumor. Cancer Communications, 44(7):761–786, June 2024. ISSN 2523-3548. doi: 10.1002/cac2.12549. URL http://dx.doi.org/10.1002/cac2.12549.
  36. [37]Yanay Rosen, Yusuf Roohani, Ayush Agrawal, Leon Samotorčan, Stephen R. Quake, and Jure Leskovec. Universal cell embedding provides a foundation model for cell biology. Nature, 656(8126):183–191, 2026. ISSN 1476-4687. doi: 10.1038/s41586-026-10689-z. URL http://dx.doi.org/10.1038/s41586-026-10689-z.
  37. [38]Moritz Schaefer, Peter Peneder, Daniel Malzl, Salvo Danilo Lombardo, Mihaela Peycheva, Jake Burton, Anna Hakobyan, Varun Sharma, Thomas Krausgruber, Celine Sin, Jörg Menche, Eleni M. Tomazou, and Christoph Bock. Multimodal learning enables batch-based exploration of single-cell data. Nature Biotechnology, November 2025. ISSN 1546-1696. doi: 10.1038/s41587-025-02857-9. URL http://dx.doi.org/10.1038/s41587-025-02857-9.
  38. [39]Jia Shi, Gautam Gare, Jinjin Tian, Siqi Chai, Zhiqiu Lin, Arun Vasudevan, Di Feng, Francesco Ferroni, and Shu Kong. Lca-on-the-line: Benchmarking out-of-distribution generalization with class taxonomies, 2024. URL https://arxiv.org/abs/2407.16067.
  39. [40]Lisa Sikkema, Ciro Ramírez-Suástegui, Daniel C. Strobl, Tessa E. Gillett, Luke Zappia, Elo Madissoon, Nikolay S. Markov, Laure-Emmanuelle Zaragosi, Yuge Ji, Meshal Ansari, Marie-Jeanne Arguel, Leonie Apperloo, Martin Banchero, Christophe Bécavin, Marijn Berg, Evgeny Chichelnitskiy, Mei-i Chung, Antoine Collin, Aurore C. A. Gay, Janine Gote-Schniering, Baharak Hooshiar Kashani, Kemal Inekci, Manu Jain, Theodore S. Kapellos, Tessa M. Kole, Sylvie Leroy, Christoph H. Mayr, Amanda J. Oliver, Michael von Papen, Lance Peter, Chase J. Taylor, Thomas Walthoeni, Chuan Xu, Linh T. Bui, Carlo De Donno, Leander Dony, Alen Faiz, Minzhe Guo, Austin J. Gutierrez, Lukas Heumos, Ni Huang, Ignacio L. Ibarra, Nathan D. Jackson, Preetish Kadur Lakshminarasimha Murthy, Mohammad Lotfollahi, Tracy Tabib, Carlos Talavera-López, Kyle J. Travaglini, Anna Wilbrey-Clark, Kaylee B. Worlock, Masahiro Yoshida, Yuexin Chen, James S. Hagood, Ahmed Agami, Peter Horvath, Joakim Lundeberg, Charles-Hugo Marquette, Gloria Pryhuber, Christos Samakovis, Xin Sun, Lorraine B. Ware, Kun Zhang, Maarten van den Berge, Yohan Bossé, Tushar J. Desai, Oliver Eickelberg, Naftali Kaminski, Mark A. Krasnow, Robert Lafyatis, Marko Z. Nikolic, Joseph E. Powell, Jayaraj Rajagopal, Mauricio Rojas, Orit Rozenblatt-Rosen, Max A. Seibold, Dean Sheppard, Douglas P. Shepherd, Don D. Sin, Wim Timens, Alexander M. Tsankov, Jeffrey Whitsett, Yan Xu, Nicholas E. Banovich, Pascal Barbry, Thu Elizabeth Duong, Christine S. Falk, Kerstin B. Meyer, Jonathan A. Kropski, Dana Pe’er, Herbert B. Schiller, Purushothama Rao Tata, Joachim L. Schultze, Sara A. Teichmann, Alexander V. Misharin, Martijn C. Nawijn, Malte D. Luecken, and Fabian J. Theis. An integrated cell atlas of the lung in health and disease. Nature Medicine, 29(6):1563–1577, June 2023. ISSN 1546-170X. doi: 10.1038/s41591-023-02327-2. URL http://dx.doi.org/10.1038/s41591-023-02327-2.
  40. [41]Barry Smith, Michael Ashburner, Cornelius Rosse, Jonathan Bard, William Bug, Werner Ceusters, Louis J Goldberg, Karen Eilbeck, Amelia Ireland, Christopher F Mungall, Nico¬les Leontis, Philippe Rocca-Serra, Alan Ruttenberg, Susanna-Assunta Sansone, Richard H Scheuermann, Nigam Shah, Patricia L Whetzel, and Suzanna Lewis. The obo foundry: coordinated evolution of ontologies to support biomedical data integration. Nature Biotechnology, 25(11):1251–1255, November 2007. ISSN 1546-1696. doi: 10.1038/nbt1346. URL http://dx.doi.org/10.1038/nbt1346.
  41. [42]Michael P. Snyder, Shin Lin, Amanda Posgai, Mark Atkinson, Aviv Regev, Jennifer Rood, Orit Rozenblatt-Rosen, Leslie Gaffney, Anna Hupalowska, Rahul Satija, Nils Gehlenborg, Jay Shendure, Julia Laskin, Pehr Harbury, Nicholas A. Nystrom, Jonathan C. Silverstein, Ziv Bar-Joseph, Kun Zhang, Katy Börner, Ying Lin, Richard Conroy, Dena Proccaccini, Ananda L. Roy, Ajay Pillai, Marishka Brown, Zorina S. Galis, Long Cai, Jay Shendure, Cole Trapnell, Shin Lin, Dana Jackson, Michael P. Snyder, Garry Nolan, William James Greenleaf, Ying Lin, Sylvia Plevritis, Sara Ahadi, Stephanie A. Nevins, Hayan Lee, Christian Martijn Schuerch, Sarah Black, Vishal Gautham Venkataramaan, Ed Esplin, Aaron Horning, Amir Bahmani, Kun Zhang, Xin Sun, Sanjay Jain, James Hagood, Gloria Pryhuber, Peter Kharchenko, Mark Atkinson, Bernd Bodenmiller, Todd Brusko, Michael Clare-Salzler, Harry Nick, Kevin Otto, Amanda Posgai, Clive Wasserfall, Marda Jorgensen, Maigan Brusko, Sergio Maffioletti, Richard M. Caprioli, Jeffrey M. Spraggins, Danielle Gutierrez, Nathan Heath Patterson, Elizabeth K. Neumann, Raymond Harris, Mark deCaestecker, Agnes B. Fogo, Raf van de Plas, Ken Lau, Long Cai, Guo-Cheng Yuan, Qian Zhu, Ruben Dries, Peng Yin, Sinem K. Saka, Jocelyn Y. Kishi, Yu Wang, Isabel Goldaracena, Julia Laskin, DongHye Ye, Kristin E. Burnum-Johnson, Paul D. Piehowski, Charles Ansong, Ying Zhu, Pehr Harbury, Tushar Desai, Jay Muly, Peter Chou, Monica Nagendran, Ziv Bar-Joseph, Sarah A. Teichmann, Benedict Paten, Robert F. Murphy, Jian Ma, Vladimir Yu. Kiselev, Carl Kingsford, Allyson Ricarte, Maria Keays, Sushma A. Akoju, Matthew Ruffalo, Nils Gehlenborg, Peter Kharchenko, Margaret Vella, Chuck McCallum, Katy Börner, Leonard E. Cross, Samuel H. Friedman, Randy Heiland, Bruce Herr, Paul Macklin, Ellen M. Quardokus, Lisel Record, James P. Sluka, Griffin M. Weber, Nicholas A. Nystrom, Jonathan C. Silverstein, Philip D. Blood, Alexander J. Ropelewski, William E. Shirey, Robin M. Scibek, Paula Mabee, W. Christopher Lenhardt, Kimberly Robasky, Stavros Michailidis, Rahul Satija, John Marioni, Aviv Regev, Andrew Butler, Tim Stuart, Eyal Fisher, Shila Ghazanfar, Jennifer Rood, Leslie Gaffney, Gokcen Eraslan, Tommaso Biancalani, Eeshit D. Vaishnav, Richard Conroy, Dena Proccaccini, Ananda Roy, Ajay Pillai, Marishka Brown, Zorina Galis, Pothur Srinivas, Aaron Pawlyk, Salvatore Sechi, Elizabeth Wilder, and James Anderson. The human body at cellular resolution: the nih human biomolecular atlas program. Nature, 574(7777):187–192, October 2019. ISSN 1476-4687. doi: 10.1038/s41586-019-1629-x. URL http://dx.doi.org/10.1038/s41586-019-1629-x.
  42. [43]Yifan Sun, Yuke Zhu, Yuhan Zhang, Pengkun Zheng, Xi Qiu, Chi Zhang, and Yichen Wei. Dynamic metric learning: Towards a scalable metric space to accommodate multiple semantic scales. In 2021 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), pages 5389–5398, 2021. doi: 10.1109/CVPR46437.2021.00535.DOI
  43. [44]Artur Szalata, Karin Hrovatin, Sören Becker, Alejandro Tejada-Lapuerta, Haotian Cui, Bo Wang, and Fabian J. Theis. Transformers in single-cell omics: a review and new perspectives. Nature Methods, 21(8):1430–1443, August 2024. ISSN 1548-7105. doi: 10.1038/s41592-024-02353-z. URL http://dx.doi.org/10.1038/s41592-024-02353-z.
  44. [45]Christina V. Theodoris, Ling Xiao, Anant Chopra, Mark D. Chaffin, Zeina R. Al Sayed, Matthew C. Hill, Helene Mantineo, Elizabeth M. Brydon, Zexian Zeng, X. Shirley Liu, and Patrick T. Ellinor. Transfer learning enables predictions in network biology. Nature, 618(7965):616–624, May 2023. ISSN 1476-4687. doi: 10.1038/s41586-023-06139-9. URL http://dx.doi.org/10.1038/s41586-023-06139-9.
  45. [46]Dénes Türei, Tamás Korcsmáros, and Julio Saez-Rodriguez. Omnipath: guidelines and gateway for literature-curated signaling pathway resources. Nature Methods, 13(12):966–967, November 2016. ISSN 1548-7105. doi: 10.1038/nmeth.4077. URL http://dx.doi.org/10.1038/nmeth.4077.
  46. [47]Hanchen Wang, Jure Leskovec, and Aviv Regev. Limitations of cell embedding metrics assessed using drifting islands. Nature Biotechnology, June 2025. ISSN 1546-1696. doi: 10.1038/s41587-025-02702-z. URL http://dx.doi.org/10.1038/s41587-025-02702-z.
  47. [48]Sheng Wang, Angela Oliveira Pisco, Aaron McGeever, Maria Brbic, Marinka Zitnik, Spyros Darmanis, Jure Leskovec, Jim Karkanias, and Russ B. Altman. Leveraging the cell ontology to classify unseen cell types. Nature Communications, 12(1), September 2021. ISSN 2041-1723. doi: 10.1038/s41467-021-25725-x. URL http://dx.doi.org/10.1038/s41467-021-25725-x.
  48. [49]Hongzhi Wen, Wenzhuo Tang, Xinnan Dai, Jiayuan Ding, Wei Jin, Yuying Xie, and Jiliang Tang. CellPLM: Pre-training of cell language model beyond single cells. In The Twelfth International Conference on Learning Representations, 2024.DOI
  49. [50]Jialu Wu, Qing Ye, Yilin Wang, Renling Hu, Yiheng Zhu, Mingze Yin, Tianyue Wang, Jike Wang, Chang-Yu Hsieh, and Tingjun Hou. Biology-driven insights into the power of single-cell foundation models. Genome Biol., 26(1):334, October 2025.DOI
  50. [51]Sunny Z. Wu, Ghamdan Al-Eryani, Daniel Lee Roden, Simon Junankar, Kate Harvey, Alma Andersson, Aatish Thennavan, Chenfei Wang, James R. Torpy, Nenad Bartoncek, Taopeng Wang, Ludvig Larsson, Dominik Kaczorowski, Neil I. Weisenfeld, Cedric R. Uytingco, Jennifer G. Chew, Zachary W. Bent, Chia-Ling Chan, Vikkitharan Gnanasambandapillai, Charles-Antoine Dutertre, Laurence Gluch, Mun N. Hui, Jane Beith, Andrew Parker, Elizabeth Robbins, Davendra Segara, Caroline Cooper, Cindy Mak, Belinda Chan, Sanjay Warrier, Florent Ginhoux, Ewan Millar, Joseph E. Powell, Stephen R. Williams, X. Shirley Liu, Sandra O’Toole, Elgene Lim, Joakim Lundeberg, Charles M. Perou, and Alexander Swarbrick. A single-cell and spatially resolved atlas of human breast cancers. Nature Genetics, 53(9):1334–1347, September 2021. ISSN 1546-1718. doi: 10.1038/s41588-021-00911-1. URL http://dx.doi.org/10.1038/s41588-021-00911-1.
  51. [52]Fan Yang, Wenchuan Wang, Fang Wang, Yuan Fang, Duyu Tang, Junzhou Huang, Hui Lu, and Jianhua Yao. scbert as a large-scale pretrained deep language model for cell type annotation of single-cell rna-seq data. Nature Machine Intelligence, 4(10):852–866, September 2022. ISSN 2522-5839. doi: 10.1038/s42256-022-00534-z. URL http://dx.doi.org/10.1038/s42256-022-00534-z.
  52. [53]Siqian Yang, Shiqi Chen, Yue Zhao, Tao Wu, Yuquan Wang, Tingting Li, Liwan Fu, Ting Ye, Yue-Qing Hu, and Haiquan Chen. Identification of a coagulation-related signature correlated with immune infiltration and their prognostic implications in lung adenocarcinoma. Thoracic Cancer, 14(33):3295–3308, October 2023. ISSN 1759-7714. doi: 10.1111/1759-7714.15121. URL http://dx.doi.org/10.1111/1759-7714.15121.
  53. [54]Xinyu Yuan, Zhihao Zhan, Zuobai Zhang, Manqi Zhou, Jianan Zhao, Boyu Han, Yue Li, and Jian Tang. Cell ontology guided transcriptome foundation model. In The Thirty-eighth Annual Conference on Neural Information Processing Systems, 2024.arxiv.org/abs/2408.12373
  54. [55]Andy G.X. Zeng, Ilaria Iacobucci, Sayyam Shah, Amanda Mitchell, Gordon Wong, Suraj Bansal, David Chen, Qingsong Gao, Hyerin Kim, James A. Kennedy, Andrea Arruda, Mark D. Minden, Torsten Haferlach, Charles G. Mullighan, and John E. Dick. Single-cell transcriptional atlas of human hematopoiesis reveals genetic and hierarchy-based determinants of aberrant aml differentiation. Blood Cancer Discovery, 6(4):307–324, April 2025. ISSN 2643-3249. doi: 10.1158/2643-3230.bcd-24-0342. URL http://dx.doi.org/10.1158/2643-3230.BCD-24-0342.
  55. [56]Suyuan Zhao, Jiahuan Zhang, and Zaiqing Nie. Large-scale cell representation learning via divide-and-conquer contrastive learning, 2023. URL https://arxiv.org/abs/2306.04371.
  56. [57]Qin Zhu, Zuzhi Jiang, Binyamin Zuckerman, Leor Weinberger, Matt Thomson, and Zev J. Gartner. Revealing a coherent cell-state landscape across single-cell datasets with concord. Nature Biotechnology, January 2026. ISSN 1546-1696. doi: 10.1038/s41587-025-02950-z. URL http://dx.doi.org/10.1038/s41587-025-02950-z.

Paper details

vlnc:W_t3vew4k6
Version
v1 · latest
Published
Fresh — added to the Hub within the last 30 days
Added
License
CC BY 4.0
Usage

views · 0

downloads · 0

A view counts once per browser per day, a download each time the PDF is requested.

  • Computer Science

Not recorded for this paper

View statement
Looking for related papers…
Contents