BEND 2.0: Benchmarking the Capabilities Needed to Design DNA

LEADERBOARD machinelearninglifescience.github.io/BEND2

CODE github.com/machinelearninglifescience/BEND2

DATA huggingface.co/datasets/yhsure/BEND2

Introduction

A biologist seeking to change the activity of a regulatory sequence may have hundreds of mutations to choose from. The same mutation can have different effects in different cell types or even on a slightly different genetic background. To help choose what to test, models must distinguish small changes most likely to produce a fine-grained desired response.

Genomic models use different approaches to learn the complex and multifaceted patterns encoded in DNA, drawing on both sequence diversity and measurements of biological function. Models such as Evo2 and DNABERT-2 learn from natural sequence variation (Brixi et al., 2026; Zhou et al., 2024), while GPN-Star uses evolutionary relationships to predict functional constraint (Ya et al., 2026). Other models incorporate functional outcomes rather than relying on natural sequence variation and homology. AlphaGenome, for example, predicts expression, accessibility and splicing from functional genomic data [2], while NTv3 combines sequence learning with such functional supervision [3]. Foundation models and task-specific models have guided the design of sequences with cell type-targeted activity (Gosai et al., 2024) and, more recently, viable bacteriophage genomes (King et al., 2026). However, the central question remains: which features of sequence and function have these models learned well enough to guide experiments?

Looking only at overall predictive accuracy leaves that question unanswered. A sequence-based score may identify a mutation at an evolutionarily constrained position without predicting its effect on expression in a particular cell type. Accurate predictions for individual mutations do not establish whether a model captures interactions between mutations or can predict their effects on a different genetic background. Similarly, distinguishing highly active genes from weakly active genes does not indicate which edit will produce a desired change in expression. Each of these cases calls for more precise testing.

Building on BEND (Marin et al., 2024), we introduce BEND 2.0 to test whether genomic models can guide DNA design. The benchmark is organized around five key capabilities: Prior, Function, Context, Interaction, and Design. Together, our tasks measure what genomic models can predict and whether they can help select DNA sequences that produce a desired response. Table 1 summarizes our selected evaluations.

Model	Prior loss bits ↓	Splicing effects $\rho$ ↑	Cell contrast $\rho$ ↑	Tissue ID % ↑	Background interaction $\rho$ ↑	Single-edit rank percentile ↓	Cell selectivity $U$ ↑	Multi-cell gain $\log_{2}$ ↑
Base composition	1.974	—	—	—	—	—	—	—
Evo2 20B	1.388	0.367	0.041	21.6	0.050	43.5	-0.350	0.275
CENO 1B	1.549	0.295	0.036	9.8	0.010	48.9	-0.318	-0.581
GENERATORv2	1.711	0.388	0.066	17.6	0.046	44.7	-0.256	0.234
DNABERT-2	—	0.198	0.076	5.9	0.020	48.6	-0.302	0.233
NTv3 100M pre	1.537	0.349	0.076	7.8	0.010	47.3	-0.207	0.339
NTv3 100M post	1.492	0.502	0.244	51.0	0.139	30.3	0.211	0.144
Native outputs	—	0.168	0.203	27.5	0.152	32.0	0.200	0.344‡
NTv3 650M pre	1.447	0.187	0.010	7.8	0.009	50.4	-0.061	0.368
NTv3 650M post	1.375	0.564	0.256	56.9	0.187	30.0	0.287	0.568
Native outputs	—	0.461	0.239	45.1	0.170	29.3	0.250	0.482‡
AlphaGenome	—	0.650	0.402	64.7	0.269	25.7	0.515	0.833
Native outputs	—	0.621	0.398	68.6	0.260	24.3	0.384	0.707‡
Enformer	—	—	0.282	31.4	0.181	31.5	0.290	0.700‡
$k$-mer ridge	—	0.318	0.211	7.8	0.032	45.0	-0.097	0.394
Task-trained ResNet	—	0.488	0.020	2.0	-0.001	47.0	-0.215	0.347
SpliceAI	—	0.674	—	—	—	—	—	—
Experimental reference†	—	0.809	0.594	99.0	0.421	11.8	0.962	1.602

Related works. The original BEND benchmark (Marin et al., 2024), GenBench (Liu et al., 2024), and OmniGenBench (Yang et al., 2025) provide systematic evaluations of genomic representations across downstream tasks. In parallel, model-guided regulatory sequence design has demonstrated cell type-targeted activity experimentally (Gosai et al., 2024; Yin et al., 2025; Castillo-Hair et al., 2025), and NucleoBench (Shor et al., 2025) evaluates algorithms for searching sequence space using predictive models as scoring functions. BEND 2.0 connects these directions by testing the predictive distinctions that highly matter for design: resolving context-specific effects, predicting interactions between mutations, and distinguishing candidates with different functional outcomes.

A framework for evaluating genomic models

BEND 2.0 is centered around five complementary capabilities that connect sequence and functional understanding to genomic design.

Sequence priors: How probable is the DNA? We measure how well a model predicts a masked base $x_i$ from surrounding DNA:

This is computed via the negative log-probability, $-\log_2 p_\theta(x_i \mid\text{available DNA context})$, over all queried positions. We report bits per base where lower values indicate better predictions.

Functional prediction: Can sequence predict a functional outcome? For a reference sequence $x$ and an edit $\delta$, the mutation effect $\Delta y$ is the change in an experimental measurement between $x$ and $x+\delta$. A predictor $f_\theta$ estimates this effect:

We test mutation-effect predictions on held-out genes, then evaluate whether the fitted outputs transfer to another assay.

Biological context: How does genomic function depend on the context? For a DNA sequence with an experimental measurement $y_c$ measured in several cells or tissues $c$, we separate its mean across the evaluated contexts $\bar{y}$ from the context-specific deviation $\kappa_c$:

We test mutation-effect differences between cells, expression differences between tissues, and whether predicted profiles across held-out genes identify the correct tissue.

Genetic interactions: Does one mutation alter another’s effect? Let $\Delta y_1$ and $\Delta y_2$ denote the effects of two individual mutations, and $\Delta y_{1+2}$ their combined effect, all relative to a reference sequence in the same context. The interaction $\mathcal{I}_{1,2}$ is the non-additive part of the signal:

We hence score interaction separately since an accurate combined effect can mask poor predictions of interaction.

Genomic design: Which candidate achieves the desired response? Given a target response $y_{\text{target}}$, the model chooses a candidate $a^\star$ among permitted edits or sequences $\mathcal{C}$ whose predicted response $\widehat{y}(a)$ best matches the target $y_{\text{target}}$:

The loss $\ell$ measures mismatch with the target, using the responses relevant to each task: mutation effects or activity across cells.

Predictions and generalization. Most predictions use linear heads fitted to frozen model sequence embeddings. For mutation effects, the linear head weights $w$ are fitted to the difference between variant and reference embeddings $(\phi_\theta(x))$ with $\widehat{\Delta y}=w^\top[\phi_\theta(x+\delta)-\phi_\theta(x)]$. These predictions and native outputs are reported separately. Detailed descriptions of the output types can be found in Appendix § 12.1.§ 1. We compute correlations within each gene or regulatory sequence, then average across the $G$ groups using $\rho_{\text{macro}}=G^{-1}\sum_{g=1}^{G}\rho_g$. This avoids the score being dominated by, e.g., one large gene. For evaluation splits, we group related examples by gene, regulatory element, locus or sequence family. In addition to genomic foundation models, our comparisons include simple predictors, specialist models and experimental references where applicable.

Sequence priors: measuring what models know about DNA

The models learn useful sequence priors. To compare causal and masked sequence models, we mask out the central base of 512 GRCh38 windows of 1,200 bp. For this base reconstruction, NTv3 650M post and Evo2 have the lowest losses among models in Figure 1A. The foundation models outperform simple base composition statistics, but do not seem to have converged on prediction accuracy. For all NTv3 models, we used a fixed 12% additional masking from its training objective, and the final scores averaged the central-base loss over two draws.

Learned sequence knowledge is not always readily accessible. For NTv3 650M post, hiding only the central base gives a loss of 2.199 bits. Masking an additional 12% reduces this to 1.375, the lowest among tested models, although heavier masking eventually worsens performance (Figure 1B). Averaging predicted probabilities across eight masking draws reduces loss further, to 1.346. We tested whether a fixed shift in internal activations could reproduce the benefit of additional masking. We estimated activation differences between single-mask and extra-masked queries, selected layer and shift strength on separate data, and evaluated the shifted model on single-mask queries from held-out genomic blocks. For NTv3 650M post, accuracy rose from 29% to 56%, close to the pre-trained model’s 55% without changing weights. Full results, controls and uncertainty estimates are provided in Appendix § 12.2.§ 1.

A simpler adjustment substantially improves GENERATORv2: temperature calibration lowers loss from 2.168 to 1.711 while leaving accuracy at 50.2% (Figure 1C). Together, these results show that measured sequence-prior performance depends on how the model is queried and its outputs are used.

These gains do not establish that sequence priors capture functional effects or variation between individuals. Evo2 and CENO reconstruct 55.2/62.4% of assay reference bases, yet their allele scores correlate only 0.045/0.029 with reporter effects (Appendix § 12.2.§ 4). Our main prior evaluation uses GRCh38; extending strong sequence priors to natural haplotypes remains an important next step (evident by results in Appendix § 12.2.§ 3).

Functional prediction: from sequence to experimental response

Mutation-effect prediction transfers. Next, we explore whether genomic models can predict how mutations affect RNA splicing. OpenSplice measures changes in exon inclusion across 590,104 mutations in 466 genes (Quarantani et al., 2026). On held-out genes, AlphaGenome’s embedding-fitted linear head reaches macro-average Spearman 0.650, compared with 0.621 for its native outputs and 0.564 for NTv3 650M post. An older specialist model SpliceAI (Jaganathan et al., 2019) reaches 0.674, higher than the top-performing foundation models.

NTv3 enables a comparison of the same architecture before and after functional training (post-training). Post-training improves mutation-effect prediction at both model sizes, and the readout from the 650M post-trained model significantly outperforms both a readout from the matched architecture with random weights (0.144) and a task-trained one-hot ResNet (0.488). Efficiency of training labels and direction analyses are found in Appendix § 12.3.

We then test whether the OpenSplice-trained readouts transfer to a different assay. Readouts trained on OpenSplice are evaluated on 4,618 COMPASS mutations across 88 genes excluded from the OpenSplice training set, without using COMPASS labels for fitting or selection (Koplik et al., 2025). Both AlphaGenome and NTv3 650M post retain substantial predictive accuracy for mutation effects averaged across cell lines, outperforming the $k$-mer baseline fitted on the OpenSplice training set (Figure 2B). Their performance is similar to that of native SpliceAI and AlphaGenome. These results suggest that the models capture patterns that remain useful beyond both the experimental assays and genes used for fitting.

Editing efficiency depends on genomic context. The K562 TRIP library tests editing efficiency across genomic locations while holding the reporter and editing program fixed (Mathis et al., 2025). Both post-trained NTv3 models improve over their pre-trained counterparts, while their differences from AlphaGenome are not statistically resolved (Appendix § 12.3.§ 1).

Biological context: how does function depend on the context

Mutation effects differ between cells. Models may easily predict a mutation’s average effect yet struggle to predict such effect when conditioned on the biological context. Commonly, one may be interested in specific tissues or cell types. In the Siraj dataset, the same mutations are measured in both HepG2 and K562 cells, allowing us to evaluate both their mean effect and the difference between cell types (Siraj et al., 2026). Across 128 regulatory elements, AlphaGenome predicts the mean effect with a correlation of 0.532, compared with 0.398 for the difference between them (Figure 3A/B). It uses its existing prediction tracks without fitting to Siraj measurements. Linear readouts fitted to NTv3 650M post reach 0.492 and 0.256, respectively. Both models predict the mean more accurately than the difference, with a larger gap for NTv3.

Expression profiles distinguish tissues. We also evaluate how well models distinguish expression profiles across tissues, using 5,782 held-out genes measured in 51 GTEx tissues (GTEx Consortium, 2020). For each gene, we pair each tissue with the tissue where measured expression differs most. We then measure how well a model’s difference in expression prediction aligns with the observed difference. Using native outputs, AlphaGenome reaches a correlation of 0.775, compared with 0.594 for NTv3 650M post (Figure 12A). Because the comparison tissue is chosen using measured expression, this effectively tests model predictions for observed tissue contrasts.

A second test asks whether a predicted expression profile across genes identifies the correct tissue among the 51 candidates. Using native outputs, AlphaGenome achieves 68.6% accuracy, compared with 45.1% for NTv3 650M post and merely 7.8% for the k-mer ridge (Figure 12B). Random chance has 2.0% accuracy, while the experimental reference identifies tissues with 99.0% accuracy. This reference uses expression profiles from independent donor halves, averaging both directions. The models generally allow for accurate tissue classification, but a sizeable performance gap remains. The evaluations hold out genes within each tissue (see also Appendix § 12.4.§ 1).

Genetic interactions: predicting departures from additivity

A mutation’s effect depends on its genetic background. Rather than the cell level context of the previous section, different genetic backgrounds may also alter the effect of mutations. The Siraj data allows for isolating this interaction, since it contains the same edit on two paired allelic backgrounds. Across 68,026 paired mutations in 114 regulatory elements, AlphaGenome reaches interaction correlation 0.260, compared with 0.187 for NTv3 650M post’s fitted readout and 0.181 for Enformer [1] (Figure 4A). Predictions are most accurate for mutations within 10 bp of one another and become less accurate with increasing separation (Figure 4C).

Combined-effect accuracy can conceal weak interaction prediction. COMPASS tests non-additive interactions on splicing by measuring single and double mutations relative to the same reference. On its 110-gene panel, NTv3 650M post reaches correlation 0.409 for single-mutation effects and 0.500 for double-mutation effects, but only 0.046 for the interaction remaining after subtracting the two single effects (Figure 4B). Native splice predictors show the same pattern, and replicate correlation for the interaction measurements is 0.171.

To assess how much interaction prediction contributes to overall accuracy, we remove the predicted interaction and retain only the additive effects. NTv3’s double-mutation correlation changes only from 0.500 to 0.493. Similarly, removing AlphaGenome’s predicted background dependence on Siraj changes overall correlation from 0.528 to 0.521, a small but statistically supported decrease of 0.0070 (95% interval: 0.0014–0.0120; Appendix § 12.5.§ 2). This indicates that a model can predict combined effects accurately while recovering little of the interaction between mutations.

Genomic design: selecting sequences for a desired response

We finally ask whether model predictions help choose DNA with a desired response. These tasks evaluate the measured outcomes of model-selected candidates, progressing from matching a single response to controlling expression across multiple cell types. All choices are made within experimentally assayed libraries, with candidate measurements withheld from selection.

Matching a target response. Can a model find an alternative mutation that reproduces the effect of a reference edit? In the Siraj assay, each element contains a natural variant and a measured change in activity, which we define as the target response. Excluding that position leaves 578–597 candidate substitutions per element. A model predicts an effect for every candidate and the reference edit, then ranks candidates by how closely their predicted effects match its own reference prediction. AlphaGenome scores effects as changes in its cell-matched accessibility outputs; NTv3 uses linear heads evaluated on held-out elements. All candidates’ measured effects are withheld until scoring.

For evaluation, we rank edits by how closely their measured effects match the target. Across 331 targets in 110 elements, AlphaGenome’s first choice has an average percentile rank of 24.3, compared with 30.0 for NTv3 650M post and 50 for uniform selection (Figure 5A; lower is better). The top ten choices include at least one edit that matches the reference response within assay uncertainty for 62% of targets with AlphaGenome and 59% with NTv3, compared with 38% under uniform selection (Figure 5B). Model rankings can therefore help find matching edits with fewer tests.

Changing expression selectively. We also test a more challenging task, choosing edits which increase expression in one cell type while minimally changing expression in another. We evaluate 167,958 substitutions across 280 parental backgrounds at 127 Siraj loci, with HepG2 and K562 alternately serving as the target cell. Each model chooses one substitution or no edit by maximizing predicted utility $U_{\mathrm{cell}} = t - \lambda|o|$, where $t$ and $o$ are expression changes relative to the unmutated parent in the target and other cell. Six fixed penalties describe the tradeoff between the two responses (Figure 5C). Complete loci were held out of readout fitting and native-output calibration.

At equal weighting ($\lambda= 1$), calibrated native AlphaGenome achieves measured utility 0.384, compared with 0.287 for NTv3 650M post, 0.211 for NTv3 100M post and zero for no edit. All three models improve over controls that choose edits at the same rate and with similar predicted effects in the target cell. AlphaGenome’s utility is 0.086 higher than its matched control (adjusted 95% interval: 0.030–0.142). With a linear readout fitted to its frozen embeddings, AlphaGenome reaches a utility of 0.515 (Table 1).

To see whether better choices are possible among the substitutions tested in the Siraj assay, we select edits using their measured effects in HepG2 and K562. We use one group of repeat measurements to choose edits and the other to evaluate them, then swap the groups and average the results. On these same evaluation subsets, experimental selection outperforms native AlphaGenome by 0.06 in utility, with higher utility at 108 of 127 loci. This reference demonstrates reproducible opportunities for better selection within the library (Appendix § 12.6.§ 3). Similarly, we test genomic design selective for the specific allelic background in Appendix § 12.6.§ 2.

Selecting multi-cell expression programs. The final task is to select for activity in several cell types while avoiding activity elsewhere. From the twelve-cell Atlas data ((Castillo-Hair et al., 2025)), we aim for sixteen expression patterns: eight with two cell types ON and ten OFF, and eight with three ON and nine OFF. For each pattern, a model needs to choose ten enhancers from a sequence bank of 2,411 enhancers (each 145-nt long). Models therefore need to rank them using predicted activities in the score

where $y$ is $\log_{2}$ reporter activity relative to negative controls. Our score rewards activity in the weakest ON cell and penalizes activity above the control level in the strongest OFF cell. Readouts are fitted on 5,246 other sequences where all twelve possible cell types are represented. We additionally exclude sequence families shared with the candidates.

We compare the models’ ranked choices with enhancers originally designed by Atlas to be active in the target cell types and inactive elsewhere. With linear readouts fitted to native outputs, the gains in utility are 0.707 for AlphaGenome, 0.700 for Enformer, 0.482 for NTv3 650M post and 0.344 for NTv3 100M post (Figure 6A). For all four models, accounting for OFF-cell activity improves selection over using ON-cell activity alone, across all sixteen patterns and both repeat measurements (Appendix § 12.6.§ 4).

For this multi-cell task, however, average utility remains negative: −0.222 for AlphaGenome and −0.228 for Enformer. A linear readout fitted to AlphaGenome’s frozen embeddings raises the gain over random selection to 0.833, but mean utility is still negative (−0.096; Table 1). By comparison, choosing sequences using one repeat measurement and evaluating them using the other gives a mean utility of +0.673. Better matches to the desired expression patterns are therefore available among the tested sequences.

Conclusions

Genomic models capture functional information that transfers across datasets and genes. This genomic understanding helps them select DNA with a desired target response. Functional training largely expands these capabilities: predictions from embedding-fitted linear heads also match or exceed native outputs, suggesting that its benefits extend beyond the prediction head and affect the models more deeply. At the same time, accurate aggregate predictions can conceal weak understanding of cell types and genetic interactions, which on the other hand are highly important for practical downstream use. BEND 2.0 focuses on the capabilities that matter for genomic design, hoping to support the development of models that make this search more effective.

Tasks with less consistent experimental results are often the more difficult tasks for the models, meaning performance should be interpreted in light of the experiment’s reliability. The chosen experimental references are somewhat naive (for example, transferring mutation effects across cell types), yet they still beat current models. The linear heads may also depend on the embedding sizes of the models, and models with different input context lengths may have an effect on task performance. Despite these limits, BEND 2.0 shows where genomic models may help narrow hundreds of candidate mutations to a testable few, and where further progress is needed.

Acknowledgements

This work was supported by the Novo Nordisk Foundation through the Center for Basic Machine Learning Research in Life Science (NNF20OC0062606), CAZAI (NNF22OC0077058), Quantum for Life (NNF20OC0059939), and the NovoSTAR programme, and by the Pioneer Centre for AI (DNRF grant P1). We also acknowledge the Danish Centre for AI Innovation (DCAI) and the Novo Nordisk Foundation (NNF25OC0105165) for supporting our access to the Gefion supercomputer.

AI Use Statement

Generative AI tools were used in evaluating models on HPC clusters, preparing code, organizing the tasks, and preparing the manuscript and supporting materials. The authors reviewed AI-assisted work and take full responsibility for the final manuscript, including its methods, results and conclusions.

Ethics Statement

This work evaluates machine learning models using existing functional-genomics experiments, reference sequences and population-genomic resources. Human genomic data can be identifying. We used the data in accordance with the applicable access and redistribution conditions, which also govern future use. Performance on the contexts presented here should not be assumed to generalize uniformly to other populations or biological settings. Models supporting genomic design may also have dual-use applications. Downstream use therefore requires appropriate biological validation, biosafety review and oversight.

Reproducibility Statement

We provide all code and fixed data splits for running BEND 2.0. We also host the benchmarking data in our own Hugging Face repository so future evaluations do not depend on the original hosts, making our benchmark more sturdy. Data are redistributed under their original terms, with attribution and licensing information included in the repository. Our code includes both setup and evaluation instructions, while our appendices contain additional methodological details and supporting results.

Evaluation details

Holding out data for these experiments does not imply that the sequences were absent from foundation-model pretraining.

Predictions and readouts

Native outputs and fitted linear heads. We evaluate models using native outputs and embedding-fitted linear heads. Native outputs are produced by the models’ existing output heads. Depending on the model, these include self-supervised sequence logits, supervised predictions of functional measurements or genomic annotations. Embedding-fitted linear heads are trained on sequence embeddings while the original model is kept frozen. For multi-cell design, we also fit linear heads to native outputs; these are identified separately. Any task-specific calibration of native outputs is described with the task.

The use of native outputs and fitted linear heads varies depending on the task. Below are examples given for how we obtain variant-effect scores with the different approaches.

To use native outputs, where sequence output logits are available, we score a variant in causal models as its change in log sequence probability. For masked models, the score is calculated as the alternate-versus-reference log odds:

For AlphaGenome, Enformer and post-trained NTv3, we use their native functional predictions, such as DNase accessibility. For each output $c$, we predict the variant’s effect by subtracting the reference prediction from the variant prediction, $f_{\theta}(x+\delta,c)-f_{\theta}(x,c)$.

To measure variant effect using an embedding-fitted linear head, we determine the difference between frozen embeddings. With $\phi_{\theta}(x)$ representing the frozen pooled representation of a sequence and $\Delta_i$ as the measured effect of variant $\delta_i$, we fit ridge regression on the representation change:

Feature standardization is estimated on the training data only. Projections used to reduce feature dimensionality are random, seeded and label-independent, and the ridge penalty $\lambda$ is either fixed in advance or chosen on validation data. Per-task settings are provided in their corresponding sections.

Embedding-fitted linear heads evaluated on COMPASS are fit only on OpenSplice.

NTv3 outputs. Because NTv3 produces multiple native outputs, we define here the native outputs used for each figure. Post-trained NTv3 models produce nucleotide logits, functional tracks and genome annotations [3]. For example, native-output results use nucleotide logits for the sequence-prior task, predicted splice-boundary probabilities for splicing (Figure 2), and cell-matched DNase outputs for cell contrasts (Figure 3). These evaluations use the existing output heads. NTv3 also uses embedding-fitted linear heads and, for multi-cell design, linear heads fitted to native outputs. Throughout the work, we specify the exact settings during each task.

The pre-trained NTv3 100M and 650M models produce nucleotide logits only and have no functional heads.

AlphaGenome fitted readouts. AlphaGenome’s supported input is larger than the benchmark’s sequences, so padding is used. We take frozen 1-bp representations from the final model layer, mean-pool over the inserted bases, project to 512 features, and fit ridge with $\alpha=10$. Siraj and OpenSplice/COMPASS inserts use 16,384-bp N-padded inputs without genomic or vector flanks. We hold out complete elements for Siraj. The COMPASS transfer test in Figure 2B excludes overlapping sequences and removes the 23 shared genes from both datasets. For GTEx, we use 131-kb inputs, scale features using only the training data, and choose $\alpha=100$ using validation genes. The other design readouts follow Appendices F.3 and F.4. Native and fitted results remain separate.

Experimental references. For experimental references, we use one set of measurements to make predictions or choose sequences, then evaluate these predictions or choices using another set. Depending on the setting, the references can be somewhat naive (for example, transferring effects across cell types) or stronger (predicting the same response using repeat measurements).

The split is dependent on the experimental data:

• For OpenSplice, we compare the average of two variant PSI repeats with the third, averaging all three rotations. Effects share the reported wild-type estimate.

• For Siraj, the background-interaction reference compares measurements between HepG2 and K562. Other references compare effects calculated separately from repeats 1–2 and 3–5. We evaluate model predictions against effects estimated from all repeats. For cell-selective design, we instead evaluate both model choices and experimental references using effects relative to the unmutated parent.

• For GTEx, tissue identification averages both donor-half directions (99.0%), and its uncertainty was not re-estimated.

• For Atlas, each of the two RNA measurements is evaluated against the other.

For the Siraj data, the references for cell effects use 127 of the 128 Siraj elements, as one lacks the required count data. Ranking and target-response matching use all 331 targets in 110 elements. We select on one repeat group, test on the other, then swap the groups and average the results.

Uncertainty and scoring. We estimate uncertainty by bootstrapping whole biological groups, such as genes, regulatory elements or loci, following the grouping used for the data splits. This makes measurements sharing a unit stay correlated. When comparing a pre-trained model with a randomly initialized network of the same architecture, we evaluate both on the same bootstrap resample. Experiments with several training seeds jointly resample seeds and biological units. We calculate Spearman correlations within each group and average them, giving every group equal weight:

Sequence priors: Sequence probability and reconstruction

Sampling and controls. We sample 512 autosomal GRCh38 windows, each 1,200 bp long and from distinct 1-Mb blocks. Positions and strands are chosen randomly. Evaluation blocks are separate from those used to fit sequence baselines and calibrate model probabilities.

Uniform DNA costs two bits per base. The tuned Markov model uses order ten and smoothing strength 64, chosen on a 768/256 split and then refitted on all 1,024 fitting windows. Confidence intervals use 4,000 paired resamples of complete genomic blocks. Full settings and seeds are preserved in our codebase.

Sequence probability. For the causal models, we use

Evo2 20B and CENO use base tokens (Brix et al., 2026; Ma et al., 2026). Their probabilities are normalized over all tokens, including non-DNA tokens. For GENERATORv2, we score each sixmer by multiplying the base probabilities at its six positions (Li et al., 2026).

Sequence prediction in repetitive DNA. First, we group windows in three categories based on the count of bases annotated as repeats in GRCh38: none, some but less than half, or at least half. The three causal models have lower base loss than the tuned Markov model in every group, with the largest gains in repeat-rich windows (Figure 7).

Reconstructing a hidden base. We hide base 601 in each window. For each causal model, we insert A, C, G or T and score the entire resulting sequence, including the bases after the changed position. We normalize the four sequence probabilities to obtain a probability for each candidate base:

NTv3 predicts the masked base directly [3]. For the main comparison, we average the base loss over two draws, each masking an additional 12% of the other positions. We also compare a single-mask query with the average base probabilities from eight such draws. This masking rate comes from NTv3’s training procedure. Post-trained NTv3 uses the human species setting.

We report base log loss $-\log_{2} q(x_i \mid x_{\backslash i})$ and accuracy.

How post-training changes access to the sequence prior

Additional masking improves base prediction in post-trained NTv3 despite — somewhat counterintuitively — removing more of the surrounding DNA context (Figure 1B). We investigate this effect and test whether the improved performance can be reproduced while only masking the target base.

Mask placement. We test both NTv3 sizes before and after post-training on the 512 benchmark windows. Each query masks the target base and alters 144 other positions, using two random draws. Masks are spread across the window, placed within 128 bases of the target, or kept more than 256 bases away. For distant masks, we compare scattered and compact patterns with the same number of masks on each side and matched average distances. We also replace the same distant bases with other nucleotides instead of MASK tokens.

For 650M post, distant scattered masks give lower loss than compact masks (1.794 versus 2.167 bits; paired difference 0.373, 95% interval 0.283–0.467). Mask count and average distance therefore do not explain the difference. Base replacement does not reproduce the benefit. The 100M post model behaves similarly, while pre-trained models change much less (Figure 8A–B).

Replacing transformer representations. NTv3 predicts bases using both its final transformer representation and convolutional features that bypass the transformer (skip features) [3]. With only the target base masked, we replace the transformer representation with one obtained from the same DNA with extra masks or base replacements. We keep the skip features fixed, then test the reverse: replacing only the skip features.

For 650M post, uniform masking lowers loss by 0.821 bits. Replacing only the transformer representation reproduces 0.812 bits of this improvement. Replacing only the skip features changes loss by +0.014 bits, with an interval that includes zero. Distant masks and the 100M post model show the same pattern (Figure 8C).

Using unrelated DNA. We then test whether the masking benefit depends on the sequence that was masked. Using 256 pairs of unrelated sequences with similar GC content, we compare transformer representations from the unrelated DNA with and without extra distant masks. The sequence being predicted and its skip features stay fixed. Adding masks to the unrelated sequence lowers loss by 0.246 bits for 650M post (95% interval 0.147–0.348) and 0.612 bits for 100M post (0.325–0.885). Pre-trained models show no clear improvement (Figure 8D). NTv3 uses clean inputs for functional training and masked or altered inputs for language modeling ([3]), so masks may help indicate which task to perform.

Fixed activation shift. The benefit from masking unrelated DNA suggests that part of the effect is shared across sequences. We therefore test whether an average change in transformer activations can improve predictions with only the target base masked.

For each model, we use 768 windows from separate genomic blocks. We compare activations with only the target base masked and with 144 additional randomly placed masks. At each transformer block considered, we average the difference across windows and positions:

Here $h_{\ell,t}(x)$ is the activation at position $t$ of transformer block $\ell$, $L$ is the number of positions, and $N = 768$. This gives one fixed vector per model and block.

We choose the block and shift strength on 128 separate windows. The first block with multiplier 1.5 gives the lowest loss for both post-trained sizes, so we use this setting for all four models, each with its own vector. We then test on 1,536 windows from 48 unused genomic blocks, with varying target positions and both strands equally represented. The shift is added at every position, while model weights stay fixed and only the target base is masked.

We compare against reversed and random shifts of the same magnitude, applied at the same block. We also compare with temperature scaling and affine calibration of the four base logits, fitted on the same 768 windows used to estimate the shifts.

The shift raises accuracy from 28.8% to 51.2% for 100M post and from 29.1% to 55.9% for 650M post, close to the corresponding pre-trained models (Table 2). Pre-trained models change little, and the controls give no comparable improvement. The gains over affine calibration are 20.5 and 25.7 percentage points, respectively (adjusted 95% intervals: 15.7–25.5 and 20.9–30.6). Intervals use block bootstrapping with Bonferroni correction across 28 comparisons.

Condition	NTv3 100M pre	NTv3 650M pre	NTv3 100M post	NTv3 650M post
Unmodified	51.3	55.3	28.8	29.1
Temperature scaling	51.3	55.3	28.8	29.1
Affine calibration	52.0	55.5	30.7	30.2
Activation shift	51.4	<u>55.6</u>	<u>51.2</u>	<u>55.9</u>
Negated shift	<u>51.7</u>	55.0	29.9	28.9
Random shift 1	51.6	55.8	28.9	29.8
Random shift 2	51.3	55.3	27.0	29.6
Random shift 3	51.3	55.1	30.0	29.8
Eight-query average	51.5	55.1	54.4	58.4

Much of the drop in base-prediction accuracy after post-training can therefore be recovered through a fixed activation change, without retraining the model.

Probability calibration

Temperature scaling changes confidence but preserves base and allele-score rankings:

We fit one temperature per model on 1,024 separate windows, choosing $0.1 \le T \le100$ to minimize hidden-base log loss.

Sequence prediction on natural haplotypes

We select 512 nonoverlapping SNP windows on chromosome 22 from the phased 1000 Genomes data for 2,504 participants ((Byrska-Bishop et al., 2022)). We sample 128 sites in each ALT allele frequency range: 1–5%, 20–40%, 60–80% and 95–99%. Each site then has a REF-carrying sequence, an ALT-carrying sequence and the GRCh38 reference. Carrier sequences include all called variants, including indels. We select windows without using model scores.

Predicting the carried allele. We score all four bases in each carrier’s own surrounding DNA, giving the two carriers equal weight before averaging across sites.

Calibration reduces GENERATORv2’s base loss from 3.735 to 2.091 bits. For NTv3 650M post, averaging eight masked queries instead of one reduces loss from 2.337 to 2.225 bits and raises accuracy from 26.3% to 36.3%. All models shown still have higher loss than uniform prediction, which gives two bits (Figure 9A).

Allele preference and surrounding DNA. For each SNP, we score the same REF and ALT alleles in both carriers’ surrounding DNA. We subtract the ALT/REF log odds in the REF background from those in the ALT background. Positive values mean a shift towards ALT.

Across the 396 pairs with different surrounding DNA, Evo2’s preference changes little. NTv3 650M post shows a small shift towards ALT (0.042 bits, 95% interval 0.004–0.093; Figure 9B).

GENERATORv2’s shift towards REF depends on where the SNP falls within a six-base token and on nearby indels. For NTv3, we mask corresponding positions in the two sequences.

Sequence reconstruction and reporter effects

We next evaluate whether models that recover DNA bases also predict how mutations affect expression. Table 3 reports both measurements on the Siraj regulatory sequences, which are 189–200 bp long rather than the 1,200-bp genomic windows used for the main sequence-prior evaluation.

Model	Reference-base recovery (%)	Correlation with RNA effects Spearman $\rho$
Evo2 20B	55.2 [52.5, 58.1]	0.045 [0.027, 0.065]
CENO 1B-1m	62.4 [58.2, 66.6]	0.029 [0.014, 0.044]
GENERATORv2 3B	42.3 [40.0, 44.7]	0.034 [0.018, 0.049]

Reference-base recovery. At each site, we compare four otherwise identical sequences, with A, C, G or T at that position. ’Recovery’ is the frequency with which the original base has the highest full-sequence probability. Random guessing gives 25%. We use 56,742 sites with all three alternative substitutions experimentally available.

Comparing sequence scores with RNA activity. For 170,292 mutations, we compare the change in log sequence probability with the measured increase or decrease in reporter RNA activity in HepG2 and K562, using Spearman correlation. This comparison also includes sites where one or two experimentally possible mutations are missing.

Log probabilities are averaged over the forward and reverse-complement strands. We calculate the metrics within each regulatory element, then average equally across 128 elements. We do not fit expression readouts for this comparison.

The results suggest that reconstructing DNA bases does not necessarily mean predicting how mutations affect expression.

Functional prediction: Variant-effect prediction and label efficiency

Training and evaluation. We use OpenSplice to evaluate how well genomic models predict mutation effects on RNA splicing (Figure 2A). We fit linear readouts to frozen model embeddings, training and testing on separate genes across five cross-validation splits. The 3–6-mer ridge baseline uses the same splits. We also evaluate the native splicing outputs of SpliceAI and AlphaGenome.

To test transfer to another assay, we train the readouts, CNN and ResNet on OpenSplice and evaluate them on COMPASS (Figure 2B). We exclude any overlapping sequences and remove the 23 shared genes from both datasets. The CNN and ResNet use 398 OpenSplice genes for training and another 45 for validation. Trivially, COMPASS measurements are not used for any kind of model selection, but only for evaluation.

Learning from fewer examples. We vary the number of measured OpenSplice variants used to fit the NTv3 linear readouts and train the one-hot CNN. The models use the same training subsets and are evaluated on the same held-out genes. NTv3 readouts greatly outperform the CNN at every tested training-set size, showing strong gains from the use of foundation models (Figure 10).

Direction and magnitude of mutation effects. For mutations that have clearly measured effects ($|\Delta\mathrm{PSI}| > 1.96\sigma$, where $\sigma$ is the reported measurement uncertainty), the NTv3 650M post readout correctly predicts whether exon inclusion increases or decreases in 74.1% of cases, compared with 69.2% for the one-hot ResNet.

Across all mutations, their root mean square errors (adjusted for measurement noise) are respectively 16.55 and 17.27 percentage points. We calculate each metric within genes and then average across genes. NTv3’s mean calibration slope is 0.98 (95% interval: 0.95–1.01), close to the ideal value of one, indicating that predicted effects are on a very similar scale to the measured effects.

Changing focus from predicting effects to attempting an edit

Predicting mutation and edit effects is only part of DNA design: we also need to perform the edit efficiently. We next test whether genomic models can predict how editing efficiency varies with the surrounding DNA.

Experimental setting. We use the K562 TRIP dataset from PRIDICT2 ((Mathis et al., 2025)), which places a 640-bp reporter at 1,196 locations across 23 chromosomes. Editing efficiency is measured for prime editing, adenine editing (ABE8e-SpG), cytosine editing (BE4max) and Cas9 (SpCas9). Prime editing uses one fixed guide while the other three systems share another.

Prediction and evaluation. Models receive the same 8-kb reference-genome window around each integration site (without the inserted reporter). We fit a separate ridge readout to each model’s frozen embeddings for each editing system. Training and testing use five cross-validation splits, keeping complete chromosomes together. Features are scaled using the training data, and the ridge penalty is fixed at $\alpha= 1000$.

We additionally check whether the results depend on this penalty by choosing $\alpha\in\{1, 10, \ldots, 10^{9}\}$ through three-fold cross-validation within the training chromosomes. The chosen $\alpha$ minimizes mean squared prediction error.

We calculate Spearman correlation across the combined held-out predictions. Confidence intervals use 50,000 bootstrap resamples of whole chromosomes, without refitting the readouts. Comparisons between models are adjusted for multiple testing across all 312 comparisons.

Results. Functional training improves NTv3’s predictions of prime- and base-editing efficiency at both model sizes (Figure 11). This improvement remains when the ridge penalty is chosen by cross-validation. AlphaGenome and post-trained NTv3 give the strongest predictions, with no statistically significant differences between them. All models predict Cas9 efficiency substantially less accurately than prime- or base-editing efficiency.

Biological context: Cell, tissue and personal-sequence effects

Tissue expression contrasts and classification

We evaluate expression predictions on 5,782 held-out genes across 51 GTEx tissues (GTEx Consortium, 2020), using $\log(1 + \mathrm{TPM})$ expression.

Expression differences. For each gene, we pair each tissue with the tissue where measured expression differs most. We then calculate the Spearman correlation between predicted and measured differences across all gene–tissue pairs (Figure 12A). Every model and the experimental reference use the same pairs.

Tissue identification. For each tissue, we compare its predicted expression profile across the test genes with the measured profiles of all 51 tissues. The tissue with the highest Spearman correlation is selected. We report the fraction of tissues correctly identified, sharing credit for ties; chance is $1/51 \approx2\%$ (Figure 12B).

Readouts and evaluation. We compare native RNA predictions with linear readouts fitted to frozen model embeddings. For the fitted readouts, embeddings are averaged over the input sequence and projected to 512 features. Ridge regression uses 12,146 training genes, with the penalty chosen using 1,350 separate validation genes.

These readouts use 131,072-base windows centred on each gene’s transcription start site. GENERATORv2 uses a shorter, centred 98,298-base window to fit its token limit. The evaluation holds out genes within the 51 tissues.

The experimental references compare expression measurements from two donor groups. Reported confidence intervals use 10,000 bootstrap resamples of whole genes, recomputing the rankings in each resample.

Personal DNA at seen and unseen loci

We use MAGE data to test whether personal DNA improves gene-expression prediction for new people (Taylor et al., 2024). We reconstruct both chromosome copies for 729 people and 557 chromosome-22 genes, using phased single-nucleotide variants.

We fit linear readouts to one-hot-encoded personal DNA. All fits also include donor and assay covariates. As a control, we repeat the fitting after shuffling whole genotypes among people from the same population and data split. Personal sequence gain is the difference in average within-gene correlation:

Both comparisons hold out people. The second also holds out gene regions, keeping overlapping 8-kb windows together when splitting the data.

At genes seen during training, using the correct personal DNA improves prediction: the gain rises from 0.036 with 1-kb windows to 0.062 with 8-kb windows (Figure 13, left). When the gene region is also unseen, all confidence intervals however include zero, with an 8-kb gain of practically zero (right). Personal DNA thus helps when the readout has learned from the same genes in other people, but shows no clear benefit at new gene regions.

Genetic interactions: Interaction prediction

Predicting COMPASS mutation effects.

We use the linear-readout method described in Appendix C, training on OpenSplice substitutions. Here, we exclude exact sequence overlaps but retain genes shared between the datasets. The interaction analysis includes 15,035 single-mutant and 48,054 double-mutant measurements in 110 genes (Figure 4B).

Before calculating interactions, we subtract each readout’s prediction for an unchanged sequence, so that an unchanged sequence has zero predicted effect. Figure confidence intervals use 10,000 bootstrap samples of whole genes.

How much do predicted interactions contribute?

For COMPASS, we replace each predicted double-mutant effect with the sum of its two single-mutant effects:

For Siraj, we replace a mutation’s predicted effects on its two backgrounds with their mean:

We use 68,026 paired mutations in 114 regulatory elements, averaging measured effects across HepG2 and K562.

We calculate Spearman correlations within genes or regulatory elements and then average them. Confidence intervals use 20,000 paired bootstrap samples of whole genes or elements, with a Bonferroni correction for 22 correlation and RMSE comparisons.

AlphaGenome predicts Siraj interactions with correlation 0.260, yet removing interactions changes its overall effect correlation by only 0.007 (Table 4). A high correlation for overall effects is therefore not enough to show that a model predicts interactions well.

Fine-tuning for interaction prediction

We test whether fine-tuning NTv3 100M post improves interaction prediction compared with a readout fitted to its frozen embeddings. Both approaches use the same Siraj measurements, with separate training, validation and test loci. We update the final decoder block using mutations measured on both backgrounds at 68 training loci and evaluate on 23 test loci. Results average three training runs, and confidence intervals resample test loci.

Fine-tuning improves effect correlation by 0.065 (95% interval 0.005–0.125), but does not clearly improve interaction correlation (change −0.016, 95% interval −0.040–0.007).

Model	Original $\rho$	Without interaction $\rho$	Difference $\Delta\rho$	95% interval
COMPASS: double-mutant effects
AlphaGenome	0.471	0.462	+0.0098	[$-0.0059$, 0.0245]
SpliceAI	0.555	0.559	$-0.0039$	[$-0.0181$, 0.0044]
NTv3 100M post	0.423	0.418	+0.0050	[$-0.0109$, 0.0219]
NTv3 650M post	0.500	0.493	+0.0070	[0.0014, 0.0144]
GENERATORv2	0.257	0.243	+0.0134	[$-0.0066$, 0.0307]
Siraj: mutation effects on paired backgrounds
AlphaGenome	0.528	0.521	+0.0070	[0.0014, 0.0120]
Enformer	0.450	0.451	$-0.0006$	[$-0.0064$, 0.0046]
NTv3 100M post	0.417	0.417	$-0.00003$	[$-0.0057$, 0.0051]
NTv3 650M post	0.487	0.481	+0.0055	[0.0009, 0.0099]
GENERATORv2	0.208	0.211	$-0.0039$	[$-0.0077$, $-0.0001$]
Evo2 20B	0.206	0.209	$-0.0027$	[$-0.0068$, 0.0013]

Genomic design: Target matching and selective expression

Matching a target response

In the Siraj assay, we start from a regulatory element’s reference sequence. The reference edit $\delta_{0}$ introduces an alternative allele. We ask whether a single-base edit at any other position in the reference sequence can produce a similar change in RNA activity. We also test the reverse direction. Notably, Siraj has experimentally measured the entire design space, thus enabling this task.

We retain 331 targets across 110 elements for which the measured reference-edit effect is at least twice its standard error in magnitude. Each model ranks candidate edits by how closely their predicted effects match its own prediction for $\delta_{0}$.

For the first selected edit, we report the percentile $100(r-1)/(N-1)$, where $r$ ranks its measured distance from the target among $N$ eligible edits. Lower percentiles are hence closer matches.

Success within a testing budget. For a budget of $B$ tests, $\operatorname{TopB}_{m}$ contains the first $B$ edits selected by model $m$. A target is matched if at least one selected edit differs from the measured reference effect by no more than one combined standard error:

Here, $\Delta$ is the measured change in RNA activity and SE its standard error. Both percentile ranks and success rates are averaged over targets within each element, then equally across elements.

Selecting mutations for one allelic background

We also test whether AlphaGenome can choose mutations that change expression on one allelic background while leaving the others largely unchanged. The main Siraj panel contains 114 loci with two backgrounds. Two smaller panels contain 11 loci with two backgrounds and 13 loci with four backgrounds.

Choosing mutations. Mutation candidates are substitutions within ten bases of the natural variant positions (excluding the natural variant position). For each cell, target background $t$ and desired direction $d$ ($+1$ for an increase, $-1$ for a decrease), we select the mutation with the highest predicted utility:

where $B$ is the number of backgrounds. We choose no edit if all predicted utilities are non-positive. Results are averaged within loci and then equally across loci.

Model predictions. AlphaGenome receives each experimental haplotype in its genomic context, using 131,072-base inputs. We sum predicted accessibility changes over 501 bases centered on the mutation, normalize each track by its supplied mean, and average the cell-matched ATAC and DNase predictions.

Measured RNA effects. We evaluate repeat measurements 1–2 and 3–5 separately. After summing barcode counts for each sequence, we calculate the $\log_2$ RNA-to-plasmid ratio with a pseudocount of 0.5 and average across repeats. Within each cell and background, we subtract the mean activity of sequences between the first and third activity quartiles. This reference includes both mutated and unmutated sequences.

Matching the target response. To check whether selectivity simply comes from choosing fewer or weaker edits, we compare with a control that ignores effects on the other backgrounds. It chooses the same number of edits within each locus and cell, although the target backgrounds and directions may differ.

The control combines five selection rules covering weaker and stronger predicted target effects. Their proportions are fitted using measured target responses at other loci, aiming to match the selection method’s mean target response. For the 114-locus panel, we use five folds, keeping evaluation loci out of this fitting. Controls for the smaller panels are fitted using the 114-locus panel.

Confidence intervals use 100,000 bootstrap samples of whole loci, refitting the control in each sample. We adjust the confidence intervals for our comparisons using Bonferroni correction.

Results. On the 114-locus panel, selective choices reduce the absolute response on the other background by about 35% in both repeat groups (Figure 14). Mean target responses remain similar: adjusted 95% intervals for their differences lie entirely within $\pm0.05$ $\log_2$ units. Utility gains are 0.196 (adjusted 95% interval 0.139–0.258) and 0.186 (0.131–0.243).

Selecting mutations for one cell type

For the cell-selective design task in Figure 5C, we use the Siraj HepG2/K562 assay (Siraj et al., 2026) to choose mutations that increase expression in one cell type while limiting changes in the other. The analysis includes 167,958 substitutions across 280 parental backgrounds and 127 loci; one locus lacks the required count data. Both cells serve as targets, and all measured substitutions are eligible, including natural-allele substitutions and reversions.

For each repeat, we compare the mutation’s RNA-to-plasmid ratio with that of its unmutated parent, then average across repeats:

where $m0$ denotes the unmutated parent. We also calculate effects separately for repeats 1–2 and 3–5 for the experimental reference below.

Predicting effects. We compare AlphaGenome and Enformer native accessibility outputs, NTv3 100M/650M post readouts, a 3–6-mer ridge baseline and a one-hot ResNet. AlphaGenome and Enformer predictions are summed over 501-bp and 1,024-bp windows, respectively. We convert these accessibility predictions to RNA-effect units by fitting a nonnegative scale factor for each cell. Five folds keep complete loci together, and calibration uses only loci outside the evaluation fold. Training loci receive equal weight.

For NTv3 and the baselines, we combine the predicted mean effect across cells with the predicted HepG2-minus-K562 difference: HepG2 receives the mean plus half the difference, and K562 the mean minus half the difference. These readouts were trained on the published RNA-effect measurements and are evaluated here against effects relative to the unmutated parent, without further calibration.

Choosing and evaluating edits. For each parent and target cell, we choose the substitution maximizing $\widehat{\Delta}_{\mathrm{target}}-\lambda|\widehat{\Delta}_{\mathrm{other}}|$, using $\lambda\in\{0,0.25,0.5,1,2,4\}$. We choose no edit if no mutation has positive predicted utility. The main comparison uses $\lambda=1$.

Measured utility is the target-cell effect minus the absolute other-cell effect. We average these values across backgrounds and target cells within each locus, then equally across loci. Choosing no edit contributes zero.

The control makes an edit for exactly the same parent and target-cell combinations as the selective method. It mixes six target-only selection rules covering weaker and stronger predicted effects, matching the selective method’s average predicted target effect within each combination. Other-cell predictions do not rank the control’s candidates, and measured responses are not used to fit the mixture. The matching therefore applies to predictions; measured RNA effects can still differ.

We use 50,000 bootstrap samples of whole loci, keeping fitted models, calibration factors and selected edits fixed. Confidence intervals for the 24 comparisons across the six models are adjusted using Bonferroni correction.

AlphaGenome, Enformer and both post-trained NTv3 readouts achieve positive utility. AlphaGenome and both NTv3 readouts also improve over their matched controls. Enformer’s gain over its control is uncertain: 0.055 (adjusted 95% interval −0.011–0.120).

Experimental reference. We choose edits using measurements from repeats 1–2 and evaluate them using repeats 3–5, then reverse the groups and average the results. Both directions use the same candidates, penalties, no-edit option and control procedure.

The reference reaches mean utility 0.962 (95% locus-bootstrap interval 0.829–1.092), with gains of 0.371 over target-only selection and 0.292 over the matched control. On average, edits chosen using one group of measurements remain selective in the other.

Selecting enhancers for multi-cell expression programs

The multi-cell task in Figure 6 uses Atlas reporter measurements (Castillo-Hair et al., 2025) (Zenodo record 17410822). We evaluate sixteen expression patterns: eight with two cell types ON and eight with three ON (with the remaining cells OFF). Each pattern uses the same 2,411 distinct 145-nt candidate sequences, regardless of their original design target.

We use twelve cell types with two RNA measurements each. For each cell and its repeat, the activity is the log RNA-to-DNA ratio relative to the median of the 35 negative controls:

Counts are UMI-deduplicated, and we retain the authors’ DNA coverage filter of more than 20 reads in each preparation. The two RNA measurements stem from two separate experimental transfections.

Model predictions. We add the assay’s cloning junctions and pad with Ns to 16,384 bases for AlphaGenome, 196,608 for Enformer and 1,024 for NTv3 post. We average each model’s native human outputs over the insert and apply a $\log(1+x)$ transform. Using the mean of the two RNA measurements for fitting, linear ridge readouts then predict activity in all twelve cells.

Sequences sharing a genomic background or high sequence similarity are grouped into families. Excluding families shared with the candidates leaves 5,246 sequences for fitting. We train on 4,223 and choose the ridge penalty using 1,023 validation sequences, keeping families together. We then refit on all 5,246 sequences. The readout fitted to AlphaGenome’s frozen embeddings uses the same data and fitting procedure.

Selection and evaluation. For each expression pattern, models select the ten sequences with the highest predicted utility (Equation (6)). We evaluate the same choices using each RNA measurement, averaging over the ten sequences and then equally across the sixteen patterns.

We compare these choices with selection based only on the weakest ON-cell prediction (Figure 15), random selection from all candidates, and random selection among sequences which the Atlas authors originally designed for the same ON/OFF pattern. Figure 6A shows gains over this last difficult control, using one point for each RNA measurement.

The experimental reference selects sequences using one RNA measurement and evaluates them using the other, then swaps the measurements. Figure 6B shows one example: AlphaGenome’s ten sequence choices for HepG2/K562, ordered by predicted utility, with activities averaged across both measurements.

Quantification of uncertainty

Confidence intervals

The intervals and point estimates below correspond to Table 1.

Model	Prior loss bits $\downarrow$	Splicing effects $\rho\uparrow$	Cell contrast $\rho\uparrow$	Tissue ID $\% \uparrow$
Base composition	[1.953, 1.997]	—	—	—
Evo2 20B	[1.250, 1.538]	[0.352, 0.381]	[0.022, 0.060]	21.6
CENO 1B	[1.385, 1.736]	[0.282, 0.309]	[0.018, 0.053]	[5.9, 19.6]
GENERATORv2	[1.652, 1.775]	[0.374, 0.402]	[0.050, 0.083]	[9.8, 21.6]
DNABERT-2	—	[0.188, 0.210]	[0.066, 0.087]	[2.0, 9.8]
NTv3 100M pre	[1.454, 1.620]	[0.335, 0.363]	[0.063, 0.089]	[5.9, 15.7]
NTv3 100M post	[1.393, 1.592]	[0.488, 0.516]	[0.215, 0.273]	[35.3, 56.9]
Native outputs	—	[0.151, 0.185]	[0.168, 0.237]	[17.6, 33.3]
NTv3 650M pre	[1.352, 1.539]	[0.174, 0.199]	[-0.001, 0.023]	[3.9, 11.8]
NTv3 650M post	[1.275, 1.475]	[0.550, 0.578]	[0.229, 0.283]	[45.1, 62.7]
Native outputs	—	[0.445, 0.477]	[0.207, 0.271]	[35.3, 47.1]
AlphaGenome	—	[0.634, 0.665]	[0.371, 0.433]	[51.0, 68.6]
Native outputs	—	[0.605, 0.637]	[0.366, 0.431]	[60.8, 72.5]
Enformer	—	—	[0.242, 0.321]	[25.5, 37.3]
$k$-mer ridge	—	[0.307, 0.329]	[0.194, 0.228]	[3.9, 15.7]
Task-trained ResNet	—	[0.477, 0.498]	[0.011, 0.029]	[2.0, 3.9]
SpliceAI	—	[0.659, 0.689]	—	—
Experimental reference	—	[0.792, 0.825]	[0.567, 0.620]	99.0
Model	Background interaction $\rho\uparrow$	Single-edit rank percentile $\downarrow$	Cell selectivity $U \uparrow$	Multi-cell gain $\log_2 \uparrow$
Base composition	—	—	—	—
Evo2 20B	[0.034, 0.066]	[39.5, 47.5]	[-0.418, -0.286]	[0.233, 0.317]
CENO 1B	[-0.007, 0.027]	[44.9, 52.7]	[-0.392, -0.248]	[-0.584, -0.579]
GENERATORv2	[0.030, 0.062]	[40.9, 48.5]	[-0.326, -0.190]	[0.226, 0.243]
DNABERT-2	[0.007, 0.032]	[45.0, 52.2]	[-0.377, -0.230]	[0.224, 0.242]
NTv3 100M pre	[0.000, 0.021]	[43.8, 50.8]	[-0.265, -0.151]	[0.332, 0.347]
NTv3 100M post	[0.110, 0.168]	[25.9, 34.8]	[0.124, 0.299]	[0.106, 0.182]
Native outputs	[0.123, 0.182]	[27.9, 36.2]	[0.101, 0.302]	[0.292, 0.395]‡
NTv3 650M pre	[-0.004, 0.021]	[47.1, 53.9]	[-0.094, -0.034]	[0.345, 0.391]
NTv3 650M post	[0.155, 0.220]	[25.8, 34.5]	[0.198, 0.377]	[0.527, 0.610]
Native outputs	[0.138, 0.201]	[25.5, 33.3]	[0.162, 0.341]	[0.430, 0.534]‡
AlphaGenome	[0.232, 0.304]	[21.3, 30.3]	[0.422, 0.612]	[0.827, 0.839]
Native outputs	[0.219, 0.299]	[20.5, 28.3]	[0.288, 0.483]	[0.701, 0.713]‡
Enformer	[0.141, 0.220]	[27.4, 35.7]	[0.193, 0.391]	[0.692, 0.709]‡
$k$-mer ridge	[0.015, 0.049]	[41.2, 48.9]	[-0.185, -0.009]	[0.368, 0.420]
Task-trained ResNet	[-0.012, 0.011]	[43.5, 50.6]	[-0.281, -0.155]	[0.332, 0.362]
SpliceAI	—	—	—	—
Experimental reference	[0.381, 0.462]	[9.7, 14.0]	[0.829, 1.092]	[1.598, 1.606]