betaFeedback

WorksAgricultural and Biological Sciences

Calpha: Toward Native Biomolecular Intelligence in a General-Purpose Language Model

Abstract

Biomolecular foundation models predict the structures of proteins and their complexes accurately, and large language models (LLMs) reason in natural language, but the two remain separate: the abilities of the former are not part of a general-purpose LLM, and the reasoning of the latter does not enter a structure prediction. Calpha aims at one general model that folds proteins and complexes, understands protein sequence and structure, and lets its language reasoning influence the structures it predicts. This report covers the first step, in which an LLM outputs single-chain protein structures. Qwen3.5-9B reads a sequence enriched with ESM2 residue features. It can respond in text or emit a token that invokes a pretrained SimpleFold structure head, which receives information about the protein only through Qwen's residue states. Under a fixed folding request, the head achieves mean TM-scores of 0.808 on the FoldBench-334 development set, 0.635 on CASP16 domains and 0.900 on a CAMEO-style PDB set. We compare these predictions with a baseline in which ESM2-3B encodes the sequence and a fine-tuned SimpleFold-3B head predicts the coordinates. The two systems also differ in trainable weights, training duration and crop length. For chains of 100–299 residues, all three paired confidence intervals include zero. At 300–600 residues, the mean differences are -0.044, -0.160 and -0.025, respectively. In a separate test, four English request types triggered folding on every evaluated example. Protein training reduced accuracy on a 350-item MMLU-Pro subset from 77.4% to 70.3%. An LLM can therefore output single-chain structures from its own states, with lower long-chain scores in the current comparisons and reduced accuracy on the evaluated MMLU-Pro subset. Complex prediction, structure understanding and the effect of language reasoning on the predicted coordinates remain future work.

Correspondence [email protected]

Homepage https://calpha-ai.com

GitHub https://github.com/yliust/Calpha

Hugging Face https://huggingface.co/liuyong9975/Calpha

Introduction

Foundation models for biomolecular structure. Predicting the three-dimensional structure of a protein from its sequence [1] is now a routine computation. AlphaFold [2, 3] and RoseTTAFold [4] predict single chains from multiple sequence alignments, in many cases with accuracy comparable to experiment. ESMFold [5] and OmegaFold [6] replace the alignment with the representations of a protein language model trained on sequences alone [7, 8]. Prediction has since moved from single chains to assemblies: AlphaFold-Multimer [9] predicts protein complexes, and AlphaFold 3 [10], RoseTTAFold All-Atom [11], Chai-1 [12], Boltz [13, 14] and OpenDDE [15] model proteins together with nucleic acids and small molecules. The architectures have also become simpler and generative. AlphaFlow [16] turns a structure predictor into a flow-matching model [17], and SimpleFold [18] folds proteins with a standard transformer trained by flow matching. Related models design backbones [19, 20], design sequences for a given backbone [21], and model sequence, structure and function jointly [22, 23]. These models now serve as foundation models for structural biology. Each of them, however, maps a fixed kind of input to a fixed kind of output. None takes an instruction in natural language, uses what is written about a protein, or explains its answer.

Large language models. Large language models (LLMs) have developed along a different path. Transformers trained on text at scale [24, 25, 26, 27] follow instructions, answer questions across scientific disciplines, and reason step by step in natural language [28, 29]. Equipped with tools [30], they plan and carry out multi-step research tasks in chemistry [31, 32] and biology [33, 34, 35]. They do not, however, produce molecular structure by themselves. When we ask four frontier chat models to write the CαC_\alpha coordinates of a protein without tools, their mean TM-scores lie between 0.147 and 0.224, against 0.138–0.149 for a random-walk reference on the same targets (§ 4.4).

The gap between the two. The abilities of biomolecular foundation models are therefore important to science but are not yet part of general-purpose LLMs. Existing work connects the two in three ways. In the first, an LLM agent calls a folding or design program as an external tool and reads back its output [34, 35, 36, 37]. The language model chooses what to run, but its reasoning stays outside the predictor and can affect a prediction only through the predictor’s fixed inputs. In the second, a protein encoder is attached to an LLM so that the model can answer questions about a sequence or a structure in text [38, 39, 40, 41, 42], following vision–language models [43, 44, 45]. These models describe proteins and do not generate coordinates. In the third, structure is converted into discrete tokens that a language model reads and writes [46, 47, 48, 49], or a continuous generator is attached to a sequence model [50, 51], as generative heads are attached to language models for images and robot actions [52, 53, 54]. In all three, the reasoning that an LLM carries out in language has not been shown to improve the structure that is predicted.

Our goal. Calpha is a project to build a general-purpose LLM in which these abilities are native. One model should fold proteins and predict complexes, and it should understand protein sequence and structure well enough to discuss them. The abilities that today belong to separate biomolecular foundation models would then be part of a general model and connected to everything else that it knows. Most importantly, we want the model’s natural-language reasoning to take part in the prediction. What the model reads in the literature, what a scientist tells it and what it infers about a mechanism should influence, and in some cases determine, the structure it predicts. A model of this kind could be asked for the bound state of a protein, or could revise a prediction after reasoning about why it is wrong. We see this as a route to better scientific discovery with LLMs. This goal has not been reached. § 7.3 lays out the roadmap, and this report covers its first step.

This report: single-chain structures from an LLM. The first step is to make an LLM output the structure of a single protein chain. In Calpha (Figure 1), Qwen3.5-9B [55] reads a protein sequence inside an ordinary chat turn, together with residue features from ESM2-3B [5]. For a question about the protein, it replies in text. For a folding request, it emits a <structure> token, and its hidden states at the residue positions condition a pretrained SimpleFold-3B flow-matching head [18] that generates all-atom coordinates. The head receives information about the protein only through the language model. The same forward pass therefore decides whether to fold and supplies what the head folds from, with no tool call. The question we can answer at this stage is how much folding accuracy is retained when the structure head receives its input through the chat model.

Figure 1. Calpha inference and training. a Frozen ESM2 features enter Qwen at the amino-acid positions. When Qwen emits <structure>, the front-end mixes residue states from nine layers to condition SimpleFold-3B. b Joint training updates Qwen, the input adapter and a 100M folding head. The second stage freezes Qwen and the pretrained 3B head and trains the 8.4M-parameter conditioning front-end.

What we find. We report three observations. First, the states exported from Qwen support single-chain folding: the head reaches mean TM-scores of 0.808 on our development set, 0.635 on CASP16 domains and 0.900 on a CAMEO-style set. Second, the current Calpha configuration scores lower than the direct ESM2 →\rightarrow SimpleFold3B baseline on chains of 300–600 residues in all three sets. At 100–299 residues, all three paired confidence intervals include zero. The two systems also differ in which weights they update, in training duration and in crop length (Table 2), so the comparison measures their combined effect. Third, four of the five tested folding request types elicited the trigger on every evaluated sequence. Accuracy on the evaluated MMLU-Pro subset [56] fell by 7.1 percentage points after protein training.

CalphaESM2 →\to SimpleFold-3B
ConditioningNine layers of Qwen residue statesESM2 residue features
Updated weights8.4M conditioning front-end; frozen 3B headFine-tuned 3B head
Schedule20,000 steps at crop 256, then 10,000 at crop 51210,000 steps at crop 256
Evaluation checkpointEMA at step 30,000EMA at step 10,000

Table 2. Training configurations for the final paired comparison. Crop is measured in residues. Step counts are cumulative within the structure-head training run.

What remains open. These results concern single chains scored on CαC\alpha atoms under one fixed request. Calpha does not yet predict complexes, and it does not yet take structure as input. In none of our comparisons does routing through the language model make folding more accurate than the direct ESM2 baseline. Whether a change of wording preserves coordinate accuracy, and whether an instruction or a line of reasoning can steer the predicted conformation, are the questions that the goal above depends on. They are open (§ 7).

System and training

Calpha has four parts that run at inference: a protein encoder, a chat model, a small conditioning front-end and a structure head. Two of them, the encoder and the head, are pretrained specialist models used with their public weights. We first describe the inference path, then the two training stages, and Table 1 states for every component where its weights come from and when it is trained.

Component (parameters)Initial weightsStage 1Stage 2At inference
ESM2-3B encoderpublicfrozenfrozenyes
Adapter, ESM2 to language modelnewtrainedfrozenyes
Qwen3.5-9B (≈\approx9B)publictrainedfrozenyes
SimpleFold-100M head (0.1B)publictrained–Calpha-Joint only
Conditioning front-end (8.4M)new–trainedyes
SimpleFold-3B head (2.86B)public–frozenyes

Table 1. Components of Calpha: where the weights come from, what is trained in each stage, and what runs at inference. A dash means the component is not present in that stage.

Inference path

Input. The vocabulary of Qwen3.5-9B [55] is extended with amino-acid tokens and a <structure> token. A protein is written as a span of amino-acid tokens inside an ordinary chat turn. Frozen ESM2-3B [5] computes per-residue features, which a small adapter (a learned mixture over ESM2 layers, an MLP and a zero-initialised gate) maps to the model width and adds to the embeddings of the corresponding amino-acid tokens. The language model therefore sees each residue as “its letter plus what ESM2 knows about it”. ESM2 is thus a second model that reads the sequence; what the design guarantees is that its features reach the structure head only through the language model.

Trigger. The language head is unchanged. Emitting the token <structure> invokes the structure head; otherwise the language model continues generating text. § 5 measures this decision. Whether to fold is thus decided by the model itself, in the same forward pass as any other answer.

Conditioning front-end. After the trigger we read the hidden states at the residue positions from nine layers of the 32-layer model (layers 0, 4, …, 32; width 4096). This gives a tensor of shape (LL, 9, 4096), the same form as the per-layer ESM2 tensor that SimpleFold consumes. The front-end is SimpleFold’s own conditioning entry, rebuilt for this shape: a learned softmax mixture over the nine layers followed by a projection into the head. It has 8.4M parameters.

Structure head. The head is SimpleFold-3B (2.86B parameters) with its public weights: an atom encoder, a residue trunk and an atom decoder trained with flow matching to transport Gaussian noise to all-atom coordinates. We sample with 200 steps and the default noise scale (τ=0.01\tau= 0.01) and report a single sample per target. The head outputs all atoms, but every accuracy number in this report is computed on alpha carbons only; side-chain accuracy was not evaluated.

Training

Stage 1: teach the language model to read proteins and to decide when to fold. All parameters of Qwen3.5-9B, the ESM2 adapter and a small SimpleFold-100M head, initialised from its public weights, are trained jointly for 2B counted tokens. The loss is the sum of three terms: the language cross-entropy, the cross-entropy of the trigger token, and the head’s flow-matching loss with a smooth-lDDT term, the last weighted by 28.9 to match gradient scales. The language side mixes 75% protein tasks (inverse folding, secondary structure, contact prediction, infilling) with 25% general text and reasoning data, all in English; the structure side uses 84,658 PDB chains cropped to 256 residues. The stage-1 model with its 100M head is itself a working folding model and we report it as Calpha-Joint. In trained parameters this stage is by far the larger of the two.

Stage 2: attach a large pretrained head and train only the front-end. The language model is frozen and exports hidden states. The 100M head is replaced by SimpleFold-3B, which is also frozen; only the rebuilt front-end is trained, for 20,000 steps at an effective batch size of 64 (learning rate 10−410^{-4}, EMA 0.999) on the same 84,658 chains, followed by a further 10,000 steps with the crop length raised from 256 to 512 residues. The whole stage took about 11 hours on eight H200 GPUs. The figure of 8.4M trained parameters describes this stage only: the language model whose hidden states it reads was trained in full in stage 1.

The two systems compared on every target

In our ESM2 →\to SimpleFold-3B baseline, ESM2-3B encodes the sequence into residue features. A fine-tuned SimpleFold-3B head takes those features directly and generates the atomic coordinates. In Calpha, the ESM2 features first pass through Qwen. A trained projection maps Qwen’s residue states to the inputs of a frozen SimpleFold-3B head. Both systems start from the same pretrained head weights and use the same 84,658 training chains. Table 2 specifies the checkpoints used for evaluation.

Table 4 also compares two variants. Calpha-Joint is the stage-1 model with its co-trained 100M head. In Calpha-Token, Qwen writes DPLM-2 structure tokens, which the corresponding decoder converts to coordinates [57]. The public SimpleFold-3B baseline uses the released weights.

Mean TM-score by benchmark. In final Calpha (bold), a frozen SimpleFold-3B head predicts coordinates from Qwen residue states; the FoldBench mean is the rightmost point in Figure 2. Failures score zero. The no-homolog columns cover targets without a detected training homolog. Counts appear below the column heads. †Uses an MSA. ‡Published accuracy was not reproduced (BioMatrix) or checked (MultiFlow). See Appendix § 11.

ModelFoldBench-334 All (334)FoldBench-334 No hom. (128)CASP16 All (155)CASP16 No hom. (70)CAMEO-2025 All (200)CAMEO-2025 No hom. (16)
This work
Calpha (LLM + frozen 3B head)0.8080.7790.6350.5670.9000.691
Calpha-Joint (LLM + 100M head, co-trained)0.7440.7170.5820.5020.8450.631
Calpha-Token (DPLM-2 tokens)0.6600.5970.5690.4530.8000.621
Folding models
ESMFold0.8690.8200.7620.6700.9050.855
SimpleFold-3B0.8420.7740.7040.6000.9380.822
SimpleFold-100M0.7950.7240.6590.5700.8950.752
OpenDDE (single sequence)0.4380.5030.3680.3660.3700.398
OpenDDE (+MSA)†0.9380.9170.8720.8150.9580.948
Sequence–structure models
ESM3-open (1.4B)0.7870.7200.6430.5430.8520.739
DPLM-2 (650M)0.6560.5820.5150.4280.7570.608
DPLM-2 Bit (650M)0.6760.5640.5270.4210.8280.557
MultiFlow‡0.2800.3020.2620.2460.2720.265
Sequence–structure–text LLMs
BioMatrix-1.7B‡0.1630.1700.1690.1720.2520.169
BioMatrix-4B‡0.1580.1640.1630.1610.3110.145

Table 4.

What we measured

Table 3. Each measurement follows a different part of the system.

TestInput and populationRecorded outcome
FoldingFixed request template; 334 development chains, 155 CASP16 domains and 200 PDB chainsCαC\alpha coordinates sampled offline from exported states; mean TM-score including failures
Request triggerSequences eligible under the length filter; six English request or question typesFraction of responses emitting <structure> for each type
General abilitySame 350 MMLU-Pro items before and after protein trainingFive-shot accuracy

Table 3. Each measurement follows a different part of the system.

Three sets with different roles. FoldBench-334 contains chains released from 18 January 2023 to 23 October 2024. All development choices were made on this set. CASP16 comprises 155 domain entries corresponding to 72 distinct sequences. We predict domains in isolation, which differs from the official whole-target protocol. Our third set contains 200 single-chain X-ray structures released by the PDB between 1 January and 19 February 2025. We selected them directly from RCSB using CAMEO-style criteria when the CAMEO service was unavailable [58]. Tables abbreviate this set as CAMEO-2025. Nearest-training-structure retrieval already scores 0.844 on it. We report the three sets separately throughout.

Homology buckets. Every target is searched against one training pool of 667,331 chains (PDB [59] plus AFDB-SwissProt [60]) and classified using a minimum sequence identity of 30% and coverage of at least 50% on either side, then assigned to a homolog or no-homolog bucket. The split is 206/128 for FoldBench-334, 85/70 for CASP16 and 184/16 for CAMEO-2025; the last no-homolog bucket is too small to support conclusions on its own. The buckets do not mean quite the same thing across sets: FoldBench-334 homologs were removed from the data used to train our language model, whereas for the other two sets no such filtering was done. We also report four length buckets (<100, 100–299, 300–600, >600 residues).

Scoring and sampling. Our primary measure is TM-score [61] with fixed residue correspondence on alpha carbons. The scorer also records lDDT [62], GDT-TS and RMSD. Failed predictions receive zero TM. We report completion separately. For Cα\alpha the primary result uses one sample per target; Table 8 examines 16. Cα\alpha-Token uses per-target scores computed earlier with the same metric implementation. OpenDDE [15] uses its default top-ranked sample of five. Its MSA-assisted row uses an external alignment and is shown for context. Random walks, nearest-training-structure retrieval and experimental coordinates give three references. The reported geometry check measures adjacent Cα\alpha spacing. Scorer outputs also include nonlocal Cα\alpha proximity and normalized radius of gyration where available; none of these checks validates side-chain geometry.

QuantityPick uses truthFoldBench-334CASP16CAMEO-2025
Single sample (as in Table 4)no0.8080.6350.900
Mean of 16 samples–0.8080.6330.902
Worst of 16yes0.7730.5890.886
Best of 16 (oracle)yes0.8370.6790.919
Consensus pick (most similar to the others)no0.8140.6420.903
Mean range of TM within a target–0.0640.0910.033
Spearman: sample agreement vs. true TM–0.8870.8530.893

Table 8. Sixteen samples per target from Calpha (TM-score). Every TM-score in the table is computed against the true structure. “Pick uses truth” says whether choosing the sample requires the true structure: the oracle does, the consensus pick does not.

Paired comparisons. For each target we subtract the reference system’s TM-score from Cα\alpha’s. We report the mean difference and a percentile 95% interval from 10,000 paired bootstrap resamples of entries. The intervals quantify variation across entries for one trained model per configuration. Related sequences and multiple CASP entries from a single target can reduce the effective sample size. Interval estimates condition on these training runs.

Frontier chat models. GPT, Claude, Gemini and DeepSeek are asked, with one fixed English prompt and no tools, to return an alpha-carbon-only PDB file for the sequence. Because of cost they are run on a fixed, length-stratified subset of 180 targets (60 per set, at most 300 residues), and every other system is rescored on the same subset for comparison.

Baseline checks. Appendix § 11 distinguishes paper-level reproductions, checks against earlier runs, and baselines evaluated only under our protocol. Those distinctions apply to the model comparisons throughout § 4.

Folding results

Accuracy on three single-chain benchmarks

Under a fixed folding request, the final Cα\alpha model uses Qwen residue states to condition a frozen SimpleFold-3B head. It scores 0.808, 0.635 and 0.900 on FoldBench-334, CASP16 and CAMEO-2025, respectively. On all three sets it leads our joint and token variants, while the public SimpleFold-3B model scores higher (Table 4, All columns). FoldBench guided model development. On CASP16, nearest-structure retrieval scores 0.469; on CAMEO-2025 it reaches 0.844, where 184 of 200 chains have a training homolog. Each mean therefore has a different interpretation. The random-walk reference remains close to 0.14 on all three sets.

Where the final systems differ

ESM2→SimpleFold-3B\mathrm{ESM2} \to\mathrm{SimpleFold\text{-}3B} reaches 0.822, 0.682 and 0.917. Both systems use the same pretrained head architecture, but differ in the conditioning path, trainable weights, number of updates and final crop (Table 2). Figure 3 compares their net difference in the two most populated length ranges. Appendix Table 9 gives estimates, target counts and intervals for all groups.

Figure 3. The folding gap widens at 300–600 residues in each benchmark relative to 100–299 residues. Bars show the mean within-target TM-score difference between Calpha and the ESM2 → SimpleFold-3B baseline. Whiskers show bootstrap 95% confidence intervals. Negative values favor the baseline.

At 100–299 residues the paired intervals include zero in all three sets. The CASP16 interval extends to a loss of 0.030 TM, so differences of practical interest remain compatible with these data. At 300–600 residues all three intervals fall below zero. The largest gap occurs on CASP16, where Calpha outperforms the baseline on one of 34 entries. The 16 CASP16 domains shorter than 100 residues also show a negative mean difference with an interval excluding zero. Samples above 600 residues are small (12, 5 and 4 entries).

Figure 4 shows a domain near the median of the long-chain comparison. The homology split is less consistent across sets. The difference on the no-homolog bucket is +0.002 on FoldBench, −0.022 on CASP16 and −0.100 on CAMEO-2025. The last estimate covers 16 entries, nine of which belong to three groups with over 95% sequence identity.

Figure 4. CASP16 domain T1218-D2 (347 residues). Calpha scores 0.408 and the ESM2-conditioned SimpleFold head scores 0.538. Their difference (−0.130) is closest to the group median (−0.131) among the 34 domains of length 300–600, with ties broken by identifier. Backbones are aligned on common observed CαC\alpha positions and share one view and scale.

Comparison with specialist systems

ESMFold exceeds Calpha on the development set and on CASP16. On CAMEO-2025 the paired interval includes zero (Appendix § 9). The public SimpleFold-3B weights score higher on all three sets. Compared with ESM3-open, the paired intervals include zero on FoldBench and CASP16; on CAMEO-2025 the interval favors Calpha. The public DPLM-2 models score lower under our protocol, although their paper’s TM-scores on its own evaluation set were not reproduced (Appendix § 11). OpenDDE with an MSA is shown separately to indicate the accuracy available with additional sequence information. Its sequence database had no historical cutoff.

Comparison with language-model outputs

Calpha-Token (DPLM-2 tokens) reaches 0.660, 0.569 and 0.800. Its FoldBench mean includes one missing prediction scored as zero across 334 targets. Figure 2 includes an earlier ESM3-token model because that was the backbone used for the first flow-head experiment. BioMatrix’s public checkpoints perform close to the random-walk reference on FoldBench; their original folding results were not reproduced. On CAMEO-2025, 15 sequences occur verbatim in the public BioMatrix training set. Its 4B model scores 0.96 on these and 0.28 on the other 185.

Figure 2. FoldBench development runs, ordered by mean TM-score. The final Calpha point matches Table 4 (0.808). ESM3 tokens (0.562 on 333 chains) and Joint training (0.750) are earlier runs. The table uses DPLM-2 tokens (0.660) and a later joint checkpoint (0.744). All other plotted means cover 334 chains. Dates mark evaluation dates in October 2026. Below, each score is for 8JN0 chain A alone (89 residues), selected nearest Calpha’s median, breaking ties by identifier. Traces share one view and scale. Appendix § 10 specifies the training controls.

On the common 180-target subset, four general-purpose chat models produced parseable Cα\alpha output on 707 of 720 requests. Their mean TM-scores lie between 0.147 and 0.224 under the fixed no-tool prompt. Table 5 compares systems on the same sequences.

ModelFoldBench-334CASP16CAMEO-2025
General-purpose chat models (no tools)
GPT (gpt-6.1-sol)0.2080.2240.198
Claude (claude-opus-5-5)0.2130.1960.196
Gemini (gemini-4-argon)0.1970.1870.162
DeepSeek (v4.1-flash)0.1580.1780.147
Same 180 targets, other systems
Random walk (floor)0.1490.1410.138
BioMatrix-4B0.1760.1740.247
ESM3-open0.7600.6590.844
ESMFold0.8410.7500.902
Calpha0.8210.6670.909

Table 5. Frontier chat models asked to output an alpha-carbon-only PDB file, on a fixed subset of 60 targets per set (≤300\le300 residues). TM-score, failures count as zero. Other systems are rescored on the same targets.

Language behavior

§ 4 measured folding under a fixed request. In a conversation, the user can instead ask a question about the same sequence, for which invoking the structure head would be an error. We test whether Qwen makes that distinction under different wordings. A separate before-and-after comparison measures how much general question-answering accuracy it retains after protein training.

Deciding when to fold. For each eligible sequence we evaluate five folding requests and one protein question that calls for a text answer. Table 6 reports the rates. Four request types elicited the trigger throughout the evaluated examples; the fifth elicited it on 95.5–98.2% across the sets. False triggers on the text questions ranged from 0 to 1.4%. Appendix § 11 describes the length filter and the resulting counts. An earlier model trained on mixed Chinese–English data that also contained “fold into structure tokens” tasks triggered on 5% of plain English requests. With the architecture held fixed, the next run removed those token-folding tasks and used English throughout. Triggering improved within the first quarter of training. The two data changes were made together.

RequestFoldBench-334CASP16CAMEO-2025
Training template1.0001.0001.000
Unseen phrasing, close to training1.0001.0001.000
Unseen phrasing, distant1.0001.0001.000
After an unrelated first turn1.0001.0001.000
Phrased like the language-side protein tasks0.9820.9640.955
Protein question that does not ask for a structure (should be 0)0.0140.0000.000

Table 6. Fraction of requests answered with <structure> by the language model of Calpha; one response per eligible target and phrasing.

General ability. On a pre-registered 350-item subset of MMLU-Pro [56, 63], a harder successor of MMLU [64], with five-shot chain-of-thought prompting, the base Qwen3.5-9B scores 0.774 and the stage-1 language model scores 0.703. Accuracy fell by 7.1 percentage points. The trained model still produced answers in the expected format. Reducing the share of general data from 25% to 15% cost a further five points on the same test without improving folding.

From the trigger to a structure. A paraphrase can elicit the same trigger while changing the residue states read by the head. The rates in Table 6 therefore establish how often each request invokes folding. Measuring the fraction of requests that actually return a full-length structure, and the TM-score of those structures, requires following the same requests through coordinate generation and scoring.

What the development experiments explain

Figure 2 compares a random-walk reference with eight folding configurations. Table 7 records which settings changed together. Each trained configuration has one run (Appendix § 10).

ConclusionHeld fixedChangedResultAlso changed (confound)
A 100M head improves on ESM3-token outputfrozen language modeltoken output → flow-matching head (100M) on its hidden states0.562 → 0.712pretrained structure weights; 333 vs. 334 scored
The protein-trained, ESM2-conditioned configuration gives higher TM in the 100M-head comparisonhead, recipeprotein-trained residue states → states from sequence letters in untuned Qwen0.712 → 0.347input feature conditions differ
Freezing the head worked best in this runlanguage model, front-end, datafrozen 3B head → unfrozen0.801 → 0.792 at bestnone known
Random initialization reduced TM at the measured stepco-training recipe, datapretrained 100M head → random initialisation0.672 → 0.270compared at one training step
A longer training crop helps long chains slightlystarting checkpoint and datacrop 256 → 512 residues+0.006continued updates; one training run

Table 7. The comparisons behind each diagnostic conclusion (FoldBench-334, TM-score).

Output path. The early Qwen model writing ESM3 tokens scored 0.562. With a pretrained 100M flow head reading the same frozen backbone, TM rose to 0.713 on the same 333 targets (0.712 on all 334). This comparison includes the decoder’s structural pretraining. In the 100M-head experiments, one-hot sequence input and residue states from a Qwen without protein training scored 0.346 and 0.347. A separate experiment used a trained projector to inject ESM2 into a Qwen without protein training and obtained 0.476 with a 3B head. This shows that specialist features can pass through that backbone. The change in head size and conditioning prevents attribution of the difference from 0.712 to protein training alone.

Pretrained structural knowledge. The structure head was pretrained on far more structures than we train on. With the head frozen and only the 8.4M-parameter front-end trained, Calpha reached 0.801; unfreezing the head from that configuration made the model worse at every checkpoint we measured, from 0.792 at best to 0.771 after 20,000 steps (Appendix § 10.§ 4). A head initialised at random reached only 0.270 under our budget. The historical comparisons with SimpleFold heads conditioned directly on ESM2 features gave gaps of 0.077 and 0.031 with the 100M and 3B heads on the development set. Their training configurations are described in Appendix § 10.§ 2. The final 3B comparison changes the head training schedule as well as the conditioning path.

What did not close the gap. Doubling the training steps, doubling the capacity of the front-end and letting gradients update the language model each changed TM by less than 0.003. Continuing training at crop 512 raised overall TM by 0.006 relative to the crop-256 run. The gain was concentrated in the 300–600-residue group.

Choosing among 16 samples. On CASP16, a single sample scores 0.635. Selecting the sample most similar to the others gives 0.642. An oracle that sees the reference structure reaches 0.679, close to the direct system’s single-sample score of 0.682. The oracle bounds selection within these draws. Across targets, agreement among samples correlates with true TM (Spearman 0.853–0.893 across sets). Using that agreement to assign calibrated confidence to a new target remains to be tested.

Discussion, availability and outlook

Scope of the evidence

The strongest length-dependent finding applies to the two trained systems in Table 2. Their paired gap is negative at 300–600 residues on all three sets. A matched comparison that fixes the trainable head parameters, optimizer schedule and training crops across both conditioning paths would identify the contribution of the path under that recipe. Explaining whether errors arise in local structure or in the arrangement of distant regions also requires measurements across targets.

The reported scores assess CαC\alpha structure for isolated chains or domains. Table 10 reports the fraction of adjacent CαC\alpha distances in the physical range. A random walk constructed with physical step lengths also passes this check. Side-chain placement, all-atom steric quality, complexes, inference latency and peak inference memory remain unevaluated. Third-party training cutoffs and baseline reproduction checks vary by model (Appendix § 11).

Table 10. Completion rate (“Done”) and fraction of consecutive alpha-carbon distances in 3.6–4.0 Å (“Valid”). The check detects broken or stretched chains only: the random walk passes it by construction. The first row is the ground truth passed through the same code. A dash marks a metric that was not recorded.

ModelFoldBench-334CASP16CAMEO-2025
DoneValidDoneValidDoneValid
Ground truth (ceiling)1.000.9961.000.9961.000.996
Calpha (LLM + frozen 3B head)1.000.9951.000.9971.000.998
Calpha-Joint (LLM + 100M head, co-trained)1.000.9431.000.9421.000.962
Calpha-Token (DPLM-2 tokens)1.00–1.00–1.00–
ESM2 → SimpleFold-3B (fine-tuned)1.000.9931.000.9961.000.992
ESMFold1.000.9941.000.9931.000.996
SimpleFold-3B1.000.9981.000.9981.000.998
SimpleFold-100M1.000.9971.000.9971.000.998
OpenDDE (single sequence)1.000.9991.000.9991.000.999
OpenDDE (+MSA)†1.000.9981.000.9991.000.998
ESM3-open (1.4B)1.000.8711.000.7911.000.888
DPLM-2 (650M)1.000.9691.000.9611.000.976
DPLM-2 Bit (650M)1.000.9781.000.9651.000.988
MultiFlow‡1.000.9801.000.9791.000.985
BioMatrix-1.7B‡1.000.8851.000.9101.000.885
BioMatrix-4B‡1.000.9071.000.9151.000.908
Nearest training structure (copy)0.900.9860.820.9910.980.993
Random walk (floor)1.001.0001.001.0001.001.000

Table 10.

Folding accuracy was measured with one request template. Request recognition under paraphrases establishes a language-interface capability. It leaves open whether the residue states produced by those phrasings generate structures of comparable accuracy.

Relation to earlier systems

Single-sequence predictors such as ESMFold [5] and OmegaFold [6] use protein language-model features to fold a sequence. SimpleFold [18] generates coordinates with a flow-matching transformer. Calpha uses its pretrained head and supplies the conditioning features through Qwen.

Protein chat models such as ProtLLM [39] and ProteinGPT [40] answer questions in text. BioMatrix [48] and Trion-1 [49] also generate discrete structure tokens. ESM3 [22] and DPLM-2 [23] use discrete representations for joint sequence and structure modelling. UniGenX [50] and HD-Prot [51] attach continuous generators to sequence models. Calpha combines a chat interface with a continuous head: the same language model decides when to fold and supplies the residue states used to generate coordinates. A chat agent invoking an external folding program is another way to handle these requests [35, 36]. Comparing the two interfaces remains open.

Availability and outlook

The code, the per-target scores and the scorer are available in the code repository. Figure 5 places single-chain folding within the project’s longer-term goals.

Figure 5. The six levels of the Calpha roadmap span molecular generation and understanding. This report evaluates sequence-to-structure generation at L1. Structural understanding at L1 and the later levels are research goals.

Conclusion. Biomolecular foundation models and LLMs have so far been combined mainly through tool calls or text-only interfaces. Calpha aims to make structure prediction and structure understanding native abilities of a general-purpose LLM, so that reasoning in language can shape what is predicted. This report takes the first step. Qwen learns when to invoke folding across several English requests, and its residue states support single-chain folding under the fixed benchmark request. We observe two limitations in these runs. After protein training, accuracy is lower on the evaluated MMLU-Pro subset, and the final configuration scores lower than the ESM2 → SimpleFold-3B baseline on chains of 300–600 residues in all three sets. It also leaves the central part of the goal untested. Routing through a chat language model has not made folding more accurate, and we have not measured whether an instruction or a line of reasoning changes the predicted coordinates. The next steps are to match the training recipes so that the long-chain gap can be explained, to evaluate folding online across phrasings, and to extend the model to complexes and to structure as an input (Figure 5).

Acknowledgements

We thank the authors of SimpleFold [18] for releasing their code and pretrained weights. The structure head of Calpha and the direct ESM2 baseline are built on them.

Additional Tables

Table 11. Paired ΔTM\Delta\mathrm{TM} of Calpha minus each other system over all targets of a set, with bootstrap 95% confidence intervals. *: the interval excludes zero. Positive values mean Calpha is better.

Calpha minusFoldBench-334CASP16CAMEO-2025
Calpha-Joint+0.063 [+0.052, +0.075]*+0.052 [+0.038, +0.068]*+0.055 [+0.043, +0.068]*
Calpha-Token+0.147 [+0.130, +0.165]*+0.066 [+0.048, +0.084]*+0.100 [+0.081, +0.120]*
ESM2 → SimpleFold-3B (fine-tuned)−0.014 [−0.028, +0.000]−0.047 [−0.068, −0.026]*−0.017 [−0.030, −0.004]*
ESMFold−0.062 [−0.080, −0.043]*−0.128 [−0.154, −0.102]*−0.005 [−0.028, +0.019]
SimpleFold-3B−0.034 [−0.052, −0.016]*−0.070 [−0.093, −0.047]*−0.038 [−0.052, −0.023]*
SimpleFold-100M+0.013 [−0.004, +0.030]−0.025 [−0.043, −0.007]*+0.005 [−0.009, +0.021]
OpenDDE (single sequence)+0.370 [+0.340, +0.399]*+0.266 [+0.229, +0.305]*+0.530 [+0.501, +0.559]*
ESM3-open (1.4B)+0.021 [−0.000, +0.042]−0.008 [−0.031, +0.013]+0.048 [+0.023, +0.074]*
DPLM-2 (650M)+0.151 [+0.133, +0.170]*+0.120 [+0.100, +0.141]*+0.143 [+0.118, +0.170]*
DPLM-2 Bit (650M)+0.132 [+0.111, +0.152]*+0.108 [+0.082, +0.133]*+0.072 [+0.052, +0.093]*
BioMatrix-4B‡+0.649 [+0.626, +0.671]*+0.471 [+0.433, +0.510]*+0.589 [+0.546, +0.630]*
Nearest training structure (copy)+0.261 [+0.218, +0.304]*+0.165 [+0.101, +0.229]*+0.056 [+0.028, +0.085]*

Table 11.

Model<100100–299300–600>600
(38)(188)(96)(12)
Calpha (LLM + frozen 3B head)0.8210.8170.8170.536
Calpha-Joint (LLM + 100M head, co-trained)0.8050.7790.6990.369
Calpha-Token (DPLM-2 tokens)0.6500.6530.7080.423
ESM2 → SimpleFold-3B (fine-tuned)0.8330.8130.8610.598
ESMFold0.8010.8530.9300.855
SimpleFold-3B0.7560.8230.9140.821
SimpleFold-100M0.7370.7740.8660.741
OpenDDE (single sequence)0.5930.4540.3600.307
OpenDDE (+MSA)†0.8940.9310.9670.967
ESM3-open (1.4B)0.7080.7570.8750.805
DPLM-2 (650M)0.6210.6350.7310.510
DPLM-2 Bit (650M)0.5980.6530.7600.616
MultiFlow‡0.3270.2920.2500.181
BioMatrix-1.7B‡0.1950.1680.1460.119
BioMatrix-4B‡0.1870.1680.1350.101
Nearest training structure (copy)0.4360.4840.7120.559
Random walk (floor)0.1520.1430.1330.126

Table 12. TM-score by chain length, FoldBench-334. Target counts under the column heads.

Model<100100–299300–600>600
(16)(100)(34)(5)
Calpha (LLM + frozen 3B head)0.5850.6880.5420.347
Calpha-Joint (LLM + 100M head, co-trained)0.5810.6320.4880.229
Calpha-Token (DPLM-2 tokens)0.5600.6060.5080.275
ESM2 → SimpleFold-3B (fine-tuned)0.6100.6960.7020.489
ESMFold0.5900.7780.8160.636
SimpleFold-3B0.5900.7110.7600.547
SimpleFold-100M0.5720.6880.6410.479
OpenDDE (single sequence)0.4140.3920.3080.162
OpenDDE (+MSA)†0.7100.9020.8970.631
ESM3-open (1.4B)0.5990.6640.6190.527
DPLM-2 (650M)0.5610.5450.4350.303
DPLM-2 Bit (650M)0.4810.5470.5160.359
MultiFlow‡0.3960.2580.2320.129
BioMatrix-1.7B‡0.2300.1750.1380.073
BioMatrix-4B‡0.2420.1710.1190.064
Nearest training structure (copy)0.3680.4640.5530.327
Random walk (floor)0.1620.1370.1370.110

Table 13. TM-score by chain length, CASP16.

Model<100100–299300–600>600
(7)(101)(88)(4)
Calpha (LLM + frozen 3B head)0.8110.9290.8970.387
Calpha-Joint (LLM + 100M head, co-trained)0.7830.9150.7930.324
Calpha-Token (DPLM-2 tokens)0.7350.8410.7770.363
ESM2 → SimpleFold-3B (fine-tuned)0.8280.9290.9210.666
ESMFold0.8390.9110.9120.737
SimpleFold-3B0.8310.9390.9540.719
SimpleFold-100M0.7950.9130.8980.555
OpenDDE (single sequence)0.5710.4180.3030.296
OpenDDE (+MSA)†0.8940.9630.9600.894
ESM3-open (1.4B)0.7830.8700.8500.568
DPLM-2 (650M)0.6680.8060.7220.445
DPLM-2 Bit (650M)0.7580.8480.8230.551
MultiFlow‡0.3810.3030.2300.188
BioMatrix-1.7B‡0.2050.2920.2140.138
BioMatrix-4B‡0.2050.3630.2700.099
Nearest training structure (copy)0.7620.8350.8680.659
Random walk (floor)0.1510.1400.1310.115

Table 14. TM-score by chain length, CAMEO-2025.

One pattern in Table 10 is worth stating in words. Over 99% of adjacent Cα distances pass for both Calpha and the experimental structures on every set. Calpha-Joint, with its small co-trained head, is regular up to 600 residues but not beyond: above 600 residues only 4–29% of its alpha-carbon distances are physical, a failure that its TM-score alone would understate.

Development Experiments

This appendix gives the experiments behind § 6. All development results are FoldBench evaluations with one prediction per scored target. Figure 2 labels each trained configuration with the date of its evaluation. Random is a random-walk baseline with no learned parameters, and the randomly initialised head (Scratch) is evaluated at step 1,667. The joint-training run in the figure (0.750) predates the checkpoint reported as Calpha-Joint in Table 4 (0.744).

Replacing token output by a continuous head

At two billion training tokens, the early Qwen model generated ESM3 tokens by greedy decoding. The mean across 333 scored chains was 0.5617. The Calpha-Token checkpoint used in the main benchmark also saw two billion training tokens, but was trained separately to write DPLM-2 tokens. It averaged 0.6623 on its 333 completed chains. Both token runs lacked a score for 8uxt_A. Applying the main benchmark’s zero-for-failure rule gives 0.6603 across all 334 chains. The 0.562 and 0.660 labels therefore describe different checkpoints, token representations and scoring populations.

For the 100M-head experiment we kept the ESM3-token backbone frozen and trained a pretrained SimpleFold-100M head on its hidden states for 10,000 steps. Three controls received the same head training recipe with different conditioning signals (Table 15).

Conditioning of the headTMNo hom.
(none: the language model writes structure tokens)0.562–
Amino-acid one-hot (floor)0.3460.375
Base Qwen hidden states (no protein training)0.3470.372
Hidden states of the protein-trained Qwen0.7120.677
ESM2-3B features (control)0.7890.723

Table 15. Same head and recipe, different conditioning (FoldBench-334; SimpleFold-100M head, 10,000 steps). The first row is the same language model decoding through its own token output (333 scored chains). The four flow-head rows cover 334 chains.

On the 333 shared chains the ESM3-token and 100M-head means were 0.562 and 0.713. This checks the output-path comparison on the same targets. The head starts from the public SimpleFold-100M weights, so part of the gain may come from its structural pretraining and not from the continuous output form. With the 100M head, states from a Qwen without protein training yielded nearly the same mean TM as one-hot sequence input (paired ΔTM\Delta\mathrm{TM} +0.0003, CI [−0.007, +0.008]). In a separate 3B-head configuration, ESM2 features injected through a trained projector gave 0.476 with a Qwen that had received no protein training. The input representation and head size both matter when interpreting this comparison.

Historical comparisons with ESM2-conditioned SimpleFold heads

On FoldBench-334, the 100M-head comparison gave a paired difference of −0.077 TM (CI [−0.090, −0.064]). With the 3B head fine-tuned on both paths for 10,000 steps at crop 256, the historical comparison gave −0.031 (CI [−0.048, −0.015]). The final comparison in Table 9 includes the changes to training listed in Table 2. Each interval pertains to its own configuration.

Table 9. System comparison: paired ΔTM\Delta\mathrm{TM} of Calpha minus the ESM2 →\to SimpleFold-3B baseline (training configurations in Table 2). *: bootstrap 95% CI excludes zero. CAMEO-2025 has four targets above 600 residues, too few for an interval.

BucketFoldBench-334CASP16CAMEO-2025
AllnnΔTM\Delta\mathrm{TM} [95% CI]nnΔTM\Delta\mathrm{TM} [95% CI]nnΔTM\Delta\mathrm{TM} [95% CI]
All334−0.014 [−0.028, +0.000]155−0.047 [−0.068, −0.026]*200−0.017 [−0.030, −0.004]*
Homolog206−0.024 [−0.039, −0.009]*85−0.068 [−0.098, −0.039]*184−0.010 [−0.023, +0.003]
No homolog128+0.002 [−0.026, +0.032]70−0.022 [−0.052, +0.007]16−0.100 [−0.143, −0.060]*
<10038−0.013 [−0.064, +0.043]16−0.024 [−0.051, −0.003]*7−0.017 [−0.070, +0.018]
100–299188+0.004 [−0.015, +0.023]100−0.008 [−0.030, +0.015]101−0.000 [−0.014, +0.018]
300–60096−0.044 [−0.068, −0.021]*34−0.160 [−0.204, −0.119]*88−0.025 [−0.044, −0.007]*
>60012−0.062 [−0.146, +0.019]5−0.143 [−0.375, −0.019]*——

Table 9.

Co-training and structural pretraining

Training the language model jointly with the 100M head, instead of freezing it, raised TM from 0.712 to 0.750 (paired +0.038, CI [+0.023, +0.053]; +0.052 on the no-homolog bucket). Two controls from the same experiment bound what the result depends on. A head initialised at random instead of from the SimpleFold weights reached only 0.270 at the step where the pretrained head reached 0.672. The pretrained head scored higher on all 334 targets at that checkpoint. A run without ESM2 input was stopped early; its in-training probes were at 0.26–0.32 against 0.37–0.57 for the main run at matched steps, which is suggestive but not a formal result. The stage-1 language evaluation is reported in § 5.

Effect of unfreezing the pretrained head

Swapping in the 3B head and retraining it on our 84,658 chains gave 0.791. Freezing it and training only the 8.4M-parameter front-end gave 0.801 (paired +0.0105, CI [+0.002, +0.020]); this comparison also changed the stage-1 language model that supplies the hidden states, so the gain cannot be attributed to freezing alone. The clean test is the reverse experiment (Figure 6): starting from the frozen configuration and unfreezing the head, accuracy was significantly below the frozen model at the first checkpoint measured (−0.010 at 2,500 steps), peaked at 0.792, and fell to 0.771 by 20,000 steps. The measured gap persists through the last checkpoint in this run.

Figure 6. Unfreezing the pretrained 3B head (FoldBench-334). The dashed line marks the starting frozen configuration. All six measured checkpoints score lower after unfreezing; the highest mean is 0.792 at 5,000 updates.

Crop length helps a little; several other things do not

Table 16 summarises the remaining experiments, each changing one setting and all starting from the frozen-head configuration.

QuestionChangeΔTM\Delta\mathrm{TM}
Is the front-end under-trained?10k → 20k steps+0.0001
Is the front-end too small?double its capacity−0.0027
Is the training crop too short?256 → 512 residues+0.0060*
same, with the language model unfrozen+0.0071*
Should the head adapt?unfreeze the 3B head, 10k steps−0.0162*
Should the language model adapt?unfreeze Qwen, crop 256−0.0021
unfreeze Qwen, crop 512−0.0011

Table 16. Experiments that change one setting of the frozen-head configuration (FoldBench-334, paired ΔTM\Delta\mathrm{TM}). *: 95% CI excludes zero.

Crop length. Raising the training crop from 256 to 512 residues is the only change that helped, and the gain falls where the diagnosis predicts: +0.015 on 300–600 residues and nothing below 300. The effect was reproduced along two independent paths, with the language model frozen and unfrozen. It is small, about one fifth of the gap to the public head, and close to the size at which retraining noise could matter; we have not measured that noise.

Unfreezing the language model. Letting gradients from the head update all 9B parameters of Qwen changed nothing overall and made the longest bucket significantly worse (−0.049 above 600 residues at crop 512). This experiment does not test whether additional long-chain training would help.

A recurring shape. Across these experiments, gains landed in the homolog bucket while the no-homolog bucket did not move significantly (for the final configuration against the first frozen one: +0.011* with homologs, −0.002 without). These development experiments were all scored on FoldBench-334; as § 4.2 shows, the homolog pattern does not carry over to CAMEO-2025, so we report it as an observation on our development set and not as an explanation.

Reproduction Gates, Baseline Details and Reproducibility

Reproduction gates. Table 17 records, for each third-party model, the check we ran before using it and its outcome. For ESMFold and ESM3 the check is agreement between this pipeline and our earlier runs, which establishes that the pipeline is repeatable but is not an external validation.

ModelWeightsTraining cut-offGate
ESMFoldesmfold_v1Unverified hereRepeatable: 0.8692 vs. archived 0.8697 (FoldBench-334)
ESM3-openesm3-sm-open-v1Unverified hereRepeatable: 0.7871 vs. archived 0.7880
SimpleFoldpublic 100M / 3Bas releasedEvaluated with our sampler; no external number checked
OpenDDEopendde_v1 (PyPI 1.2.0)2021-09-30No external number checked. The official self-test passes; we ran torch 2.10.0, not the pinned 2.7.1, without an equivalence check
DPLM-2 (650M)dplm2_650mnot statedOwn CAMEO-2022 set: RMSD reproduced (7.59 vs. 7.7 Å); TM not (0.71 vs. 0.79)
DPLM-2 Bit (650M)dplm2_bit_650mnot statedRMSD reproduced (6.38 vs. 6.4 Å); TM not (0.77 vs. 0.84). The only weights released for the DPLM-2.1 paper; they implement its bit-wise token modelling only
MultiFlowZenodo 10714631not statedNot run. Our batched sampler matches the official one on short chains (max alpha-carbon deviation 0.10 Å)
BioMatrix-1.7B/4BBioMatrix-*-SFTnot statedNot reproduced: 0.29 / 0.45 on its own test set vs. 0.84 reported

Table 17. Third-party models: weights, training cut-off where documented, and the outcome of the reproduction gate.

BioMatrix. We prompt with the single instruction shared by all folding rows of its public fine-tuning data, decode greedily, and convert the generated structure tokens to coordinates with the GCP-VQVAE decoder. We excluded an error on our side by decoding the structure tokens of its own training data with the same decoder, which gives TM 0.993. Two explanations remain open: 271 of the 449 sequences of the test set on which the paper reports folding appear verbatim in the public fine-tuning data, and the paper may have evaluated task-specific checkpoints while the public one is merged. We have not contacted the authors. Under our protocol only 24 of 334 FoldBench-334 generations have exactly the right number of structure tokens.

OpenDDE. OpenDDE is run with its command-line defaults (five samples, top-ranked reported) with and without an MSA. The MSAs come from the public ColabFold server [65] with no date filter on the sequence databases, so the MSA row is an upper reference and not a held-out result. The 155 CASP16 domains correspond to 72 distinct sequences; OpenDDE is run once per sequence.

Frontier chat models. The model identifiers are gpt-6.1-sol, claude-opus-5-5, gemini-4-argon and deepseek-v4.1-flash, queried on 2026-11-11 with default reasoning settings and no tools. The DeepSeek responses come from two endpoints serving the same model (87 targets through a gateway, 93 through the vendor API). Parseable output was returned for 180 (GPT), 172 (Claude), 179 (Gemini) and 176 (DeepSeek) of the 180 targets; the rest score zero.

Eligible sequences for the trigger test. The trigger evaluation only forms prompts for sequences of at most 400 residues. This leaves 278, 140 and 155 eligible entries for FoldBench-334, CASP16 and CAMEO-2025, and these are the denominators of the rates in Table 6. The rates for the fifth request type correspond to 273/278, 135/140 and 148/155.

References

  1. [1]Anfinsen, C. B. Principles that govern the folding of protein chains. Science, 181(4096):223–230, 1973. doi: 10.1126/science.181.4096.223.DOI
  2. [2]Senior, A. W., Evans, R., Jumper, J., et al. Improved protein structure prediction using potentials from deep learning. Nature, 577:706–710, 2020.DOI
  3. [3]Jumper, J., Evans, R., Pritzel, A., et al. Highly accurate protein structure prediction with AlphaFold. Nature, 596:583–589, 2021.DOI
  4. [4]Baek, M., DiMaio, F., Anishchenko, I., et al. Accurate prediction of protein structures and interactions using a three-track neural network. Science, 373(6557):871–876, 2021. doi: 10.1126/science.abj8754.DOI
  5. [5]Lin, Z., Akin, H., Rao, R., et al. Evolutionary-scale prediction of atomic-level protein structure with a language model. Science, 379(6637):1123–1130, 2023.DOI
  6. [6]Wu, R., Ding, F., Wang, R., et al. High-resolution de novo structure prediction from primary sequence. bioRxiv preprint, 2022.DOI
  7. [7]Rives, A., Meier, J., Sercu, T., et al. Biological structure and function emerge from scaling unsupervised learning to 250 million protein sequences. Proceedings of the National Academy of Sciences, 118(15):e2016239118, 2021. doi: 10.1073/pnas.2016239118.DOI
  8. [8]Elnaggar, A., Heinzinger, M., Dallago, C., et al. ProtTrans: Toward understanding the language of life through self-supervised learning. IEEE Transactions on Pattern Analysis and Machine Intelligence, 44(10):7112–7127, 2022. doi: 10.1109/TPAMI.2021.3095381.DOI
  9. [9]Evans, R., O’Neill, M., Pritzel, A., et al. Protein complex prediction with AlphaFold-Multimer, 2021. bioRxiv 10.1101/2021.10.04.463034.DOI
  10. [10]Abramson, J., Adler, J., Dunger, J., et al. Accurate structure prediction of biomolecular interactions with AlphaFold 3. Nature, 630:493–500, 2024.DOI
  11. [11]Krishna, R., Wang, J., Ahern, W., et al. Generalized biomolecular modeling and design with RoseTTAFold All-Atom. Science, 384(6693):eadl2528, 2024.DOI
  12. [12]Chai Discovery, Boitreaud, J., Dent, J., McPartlon, M., Meier, J., Reis, V., Rogozhnikov, A., and Wu, K. Chai-1: Decoding the molecular interactions of life. bioRxiv preprint, 2024.DOI
  13. [13]Wohlwend, J., Corso, G., Passaro, S., et al. Boltz-1: Democratizing biomolecular interaction modeling. bioRxiv preprint, 2024.DOI
  14. [14]Passaro, S., Corso, G., Wohlwend, J., et al. Boltz-2: Towards accurate and efficient binding affinity prediction. bioRxiv preprint 2025.06.14.659707, 2025.DOI
  15. [15]Aureka Research. OpenDDE. https://github.com/aurekaresearch/OpenDDE, 2026. arXiv:2607.03787; opendde==1.2.0, accessed 2026-11-11.
  16. [16]Jing, B., Berger, B., and Jaakkola, T. AlphaFold meets flow matching for generating protein ensembles. In Proceedings of the 41st International Conference on Machine Learning, volume 235 of Proceedings of Machine Learning Research, pp. 22277–22303. PMLR, 2024. https://proceedings.mlr.press/v235/jing24a.html.arxiv.org/abs/2402.04845
  17. [17]Lipman, Y., Chen, R. T. Q., Ben-Hamu, H., Nickel, M., and Le, M. Flow matching for generative modeling. In International Conference on Learning Representations, 2023.arxiv.org/abs/2210.02747
  18. [18]Wang, Y., Lu, J., Jaitly, N., Susskind, J., and Bautista, M. A. SimpleFold: Folding proteins is simpler than you think, 2025. arXiv:2509.18480.arxiv.org/abs/2509.18480
  19. [19]Watson, J. L., Juergens, D., Bennett, N. R., et al. De novo design of protein structure and function with RFdiffusion. Nature, 620(7976):1089–1100, 2023. doi: 10.1038/s41586-023-06415-8.DOI
  20. [20]Ingraham, J. B., Baranov, M., Costello, Z., et al. Illuminating protein space with a programmable generative model. Nature, 623(7989):1070–1078, 2023. doi: 10.1038/s41586-023-06728-8.DOI
  21. [21]Dauparas, J., Anishchenko, I., Bennett, N., et al. Robust deep learning–based protein sequence design using ProteinMPNN. Science, 378(6615):49–56, 2022. doi: 10.1126/science.add2187.DOI
  22. [22]Hayes, T., Rao, R., Akin, H., et al. Simulating 500 million years of evolution with a language model. Science, 387(6736):850–858, 2025.DOI
  23. [23]Wang, X., Zheng, Z., Ye, F., Xue, D., Huang, S., and Gu, Q. DPLM-2: A multimodal diffusion protein language model. In International Conference on Learning Representations, 2025.arxiv.org/abs/2410.13782
  24. [24]Vaswani, A., Shazeer, N., Parmar, N., Uszkoreit, J., Jones, L., Gomez, A. N., Kaiser, L., and Polosukhin, I. Attention is all you need. In Advances in Neural Information Processing Systems, 2017.arxiv.org/abs/1706.03762
  25. [25]Brown, T. B., Mann, B., Ryder, N., et al. Language models are few-shot learners. In Advances in Neural Information Processing Systems, 2020.arxiv.org/abs/2005.14165
  26. [26]OpenAI. GPT-4 technical report, 2023. arXiv:2303.08774.arxiv.org/abs/2303.08774
  27. [27]Yang, A., Li, A., Yang, B., et al. Qwen3 technical report. arXiv preprint arXiv:2505.09388, 2025.arxiv.org/abs/2505.09388
  28. [28]Wei, J., Wang, X., Schuurmans, D., Bosma, M., Ichter, B., Xia, F., Chi, E. H., Le, Q. V., and Zhou, D. Chain-of-thought prompting elicits reasoning in large language models. In Advances in Neural Information Processing Systems, 2022.arxiv.org/abs/2201.11903
  29. [29]Guo, D., Yang, D., Zhang, H., et al. DeepSeek-R1: Incentivizing reasoning capability in LLMs via reinforcement learning, 2025. arXiv:2501.12948.DOI
  30. [30]Yao, S., Zhao, J., Yu, D., Du, N., Shafran, I., Narasimhan, K., and Cao, Y. ReAct: Synergizing reasoning and acting in language models. In International Conference on Learning Representations, 2023.arxiv.org/abs/2210.03629
  31. [31]Boiko, D. A., MacKnight, R., Kline, B., and Gomes, G. Autonomous chemical research with large language models. Nature, 624(7992):570–578, 2023. doi: 10.1038/s41586-023-06792-0.DOI
  32. [32]M. Bran, A., Cox, S., Schilter, O., Baldassari, C., White, A. D., and Schwaller, P. Augmenting large language models with chemistry tools. Nature Machine Intelligence, 6(5):525–535, 2024. doi: 10.1038/s42256-024-00832-8.
  33. [33]Gottweis, J., Weng, W.-H., Darwin, A., et al. Towards an AI co-scientist, 2025. arXiv:2502.18864.
  34. [34]Swanson, K., Wu, W., Bulaong, N. L., Pak, J. E., and Zou, J. The virtual lab of AI agents designs new SARS-CoV-2 nanobodies. Nature, 646(8085):716–723, 2025. doi: 10.1038/s41586-025-09442-9.DOI
  35. [35]Huang, K., Zhang, S., Wang, H., et al. Biomni: A general-purpose biomedical AI agent. bioRxiv preprint 2025.05.30.656746, 2025.DOI
  36. [36]Ghafarollahi, A. and Buehler, M. J. ProtAgents: protein discovery via large language model multi-agent collaborations combining physics and machine learning. Digital Discovery, 3(7):1389–1409, 2024. doi: 10.1039/D4DD00013G.DOI
  37. [37]Ge, F., Zhu, J., Zhang, L., et al. AutoBinder Agent: An MCP-based agent for end-to-end protein binder design. arXiv preprint arXiv:2602.00019, 2026.arxiv.org/abs/2602.00019
  38. [38]Xu, M., Yuan, X., Miret, S., and Tang, J. ProtST: Multi-modality learning of protein sequences and biomedical texts. In Proceedings of the 40th International Conference on Machine Learning, volume 202 of Proceedings of Machine Learning Research, pp. 38749–38767. PMLR, 2023. https://proceedings.mlr.press/v202/xu23t.html.arxiv.org/abs/2301.12040
  39. [39]Zhuo, L., Chi, Z., Xu, M., et al. ProtLLM: An interleaved protein-language LLM with protein-as-word pre-training. In Proceedings of the 62nd Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers), pp. 8950–8963. Association for Computational Linguistics, 2024. doi: 10.18653/v1/2024.acl-long.484.DOI
  40. [40]Xiao, Y., Sun, E., Jin, Y., Wang, Q., and Wang, W. ProteinGPT: Multimodal LLM for protein property prediction and structure understanding. arXiv preprint arXiv:2408.11363, 2024. Spotlight at the Machine Learning for Genomics Explorations workshop, ICLR 2025.arxiv.org/abs/2408.11363
  41. [41]Wang, C., Fan, H., Quan, R., Yao, L., and Yang, Y. ProtChatGPT: Towards understanding proteins with hybrid representation and large language models. In Proceedings of the 48th International ACM SIGIR Conference on Research and Development in Information Retrieval, pp. 1076–1086. ACM, 2025. doi: 10.1145/3726302.3730064.DOI
  42. [42]Zhou, X., Han, C., Zhang, Y., et al. Decoding the molecular language of proteins with Evolla. bioRxiv preprint, 2025.DOI
  43. [43]Alayrac, J.-B., Donahue, J., Luc, P., et al. Flamingo: a visual language model for few-shot learning. In Advances in Neural Information Processing Systems, volume 35, pp. 23716–23736. Curran Associates, Inc., 2022.arxiv.org/abs/2204.14198
  44. [44]Li, J., Li, D., Savarese, S., and Hoi, S. BLIP-2: Bootstrapping language-image pre-training with frozen image encoders and large language models. In Proceedings of the 40th International Conference on Machine Learning, volume 202 of Proceedings of Machine Learning Research, pp. 19730–19742. PMLR, 2023. https://proceedings.mlr.press/v202/li23q.html.arxiv.org/abs/2301.12597
  45. [45]Liu, H., Li, C., Wu, Q., and Lee, Y. J. Visual instruction tuning. In Advances in Neural Information Processing Systems, volume 36, pp. 34892–34916. Curran Associates, Inc., 2023.
  46. [46]Chen, Z., Chen, T., Xie, C., Xue, Y., Zhang, X., Zhou, J., and Fang, X. Unifying sequences, structures, and descriptions for any-to-any protein generation with the large multimodal model HelixProTx. arXiv preprint arXiv:2407.09274, 2024.arxiv.org/abs/2407.09274
  47. [47]Ma, Z., Fan, C., Wang, Z., et al. ProTeX: Structure-in-context reasoning and editing of proteins with large language models. arXiv preprint arXiv:2503.08179, 2025.arxiv.org/abs/2503.08179
  48. [48]Pei, Q., Zhou, Z., Duan, Y., Zhao, Y., Li, W., Guo, H., He, L., Li, C., Hsieh, C.-Y., He, C., Yan, R., and Wu, L. BioMatrix: Towards a comprehensive biological foundation model spanning the modality matrix of sequences, structures, and language. arXiv preprint arXiv:2606.22138, 2026. arXiv:2606.22138.arxiv.org/abs/2606.22138
  49. [49]Lu, T., Yang, K., Feng, J., Qiu, K., Ouyang, Y., Wu, Y., Zhang, X., Wang, L., Ma, W.-Y., and Zhou, H. Trion-1: Unifying language, protein sequence, and structure with diffusion language models. arXiv preprint arXiv:2610.31215, 2026. arXiv:2610.31215.arxiv.org/abs/2610.31215
  50. [50]Zhang, G., Li, Y., Luo, R., et al. UniGenX: a unified generative foundation model that couples sequence, structure and function to accelerate scientific design across proteins, molecules and materials. arXiv preprint arXiv:2503.06687, 2025.arxiv.org/abs/2503.06687
  51. [51]Zhou, Y., Qu, H., Liu, Y., Lin, S., Song, L., and Fan, W. HD-Prot: A protein language model for joint sequence-structure modeling with continuous structure tokens. arXiv preprint arXiv:2512.15133, 2025. Long version of a paper to appear at KDD 2026.arxiv.org/abs/2512.15133
  52. [52]Li, T., Tian, Y., Li, H., Deng, M., and He, K. Autoregressive image generation without vector quantization. In Advances in Neural Information Processing Systems, volume 37, pp. 56424–56445. Curran Associates, Inc., 2024.arxiv.org/abs/2406.11838
  53. [53]Zhou, C., Yu, L., Babu, A., et al. Transfusion: Predict the next token and diffuse images with one multi-modal model. In International Conference on Learning Representations (ICLR), 2025. https://openreview.net/forum?id=S12hI0frk6.arxiv.org/abs/2408.11039
  54. [54]Black, K., Brown, N., Driess, D., et al. π₀: A vision-language-action flow model for general robot control. In Robotics: Science and Systems XXI, 2025. doi: 10.15607/RSS.2025.XXI.010.DOI
  55. [55]Qwen. Qwen3.5-9B. Model card, Alibaba Cloud. Accessed 2026.
  56. [56]Wang, Y., Ma, X., Zhang, G., et al. MMLU-Pro: A more robust and challenging multi-task language understanding benchmark. In Advances in Neural Information Processing Systems (Datasets and Benchmarks Track), volume 37, pp. 95266–95290. Curran Associates, Inc., 2024.arxiv.org/abs/2406.01574
  57. [57]Liu, Y. Writability: What makes a protein structure tokenizer a good vehicle for a language model, 2026. Companion technical report.
  58. [58]Robin, X., Haas, J., Gumienny, R., Smolinski, A., Tauriello, G., and Schwede, T. Continuous Automated Model Evaluation (CAMEO)—perspectives on the future of fully automated evaluation of structure prediction methods. Proteins: Structure, Function, and Bioinformatics, 89(12):1977–1986, 2021. doi: 10.1002/prot.26213.DOI
  59. [59]Berman, H. M., Westbrook, J., Feng, Z., Gilliland, G., Bhat, T. N., Weissig, H., Shindyalov, I. N., and Bourne, P. E. The Protein Data Bank. Nucleic Acids Research, 28(1):235–242, 2000. doi: 10.1093/nar/28.1.235.DOI
  60. [60]Varadi, M., Anyango, S., Deshpande, M., et al. AlphaFold Protein Structure Database: massively expanding the structural coverage of protein-sequence space with high-accuracy models. Nucleic Acids Research, 50(D1):D439–D444, 2022. doi: 10.1093/nar/gkab1061.DOI
  61. [61]Zhang, Y. and Skolnick, J. Scoring function for automated assessment of protein structure template quality. Proteins: Structure, Function, and Bioinformatics, 57(4):702–710, 2004.
  62. [62]Mariani, V., Biasini, M., Barbato, A., and Schwede, T. lDDt: a local superposition-free score for comparing protein structures and models using distance difference tests. Bioinformatics, 29(21):2722–2728, 2013.DOI
  63. [63]MMLU-Pro. MMLU-Pro: A more robust and challenging multi-task language understanding benchmark. https://huggingface.co/datasets/TIGER-Lab/MMLU-Pro. Evaluated on a pre-registered 350-item stratified subset with the upstream item list and scorer.
  64. [64]Hendrycks, D., Burns, C., Basart, S., Zou, A., Mazeika, M., Song, D., and Steinhardt, J. Measuring massive multitask language understanding. In International Conference on Learning Representations (ICLR), 2021.
  65. [65]Mirdita, M., Schütze, K., Moriwaki, Y., Heo, L., Ovchinnikov, S., and Steinegger, M. ColabFold: making protein folding accessible to all. Nature Methods, 19(6):679–682, 2022. doi: 10.1038/s41592-022-01488-1.DOI

Paper details

vlnc:W_cf9t4udp
Version
v1 · latest
Published
Fresh — added to the Hub within the last 30 days
Added
License
CC BY 4.0
Usage

views · 0

downloads · 0

A view counts once per browser per day, a download each time the PDF is requested.

  • Agricultural and Biological Sciences

Not recorded for this paper

View statement
Looking for related papers…
Contents