WorksPhysics and Astronomy
Knowledge Distillation of a Protein Language Model Yields a Foundational Implicit Solvent Model
Abstract
Implicit solvent models (ISMs) promise to deliver the accuracy of explicit solvent simulations at a fraction of the computational cost. However, despite decades of development, their accuracy has remained insufficient for many critical applications, particularly for simulating protein folding and the behavior of intrinsically disordered proteins. Developing a transferable, data-driven ISM that overcomes the limitations of traditional analytical formulas remains a central challenge in computational chemistry. Here we address this challenge by introducing a novel strategy that distills the evolutionary information learned by a protein language model, ESM3, into a computationally efficient graph neural network (GNN). We show that this GNN potential, trained on effective energies from ESM3, is robust enough to drive stable, long-timescale molecular dynamics simulations. When combined with a standard electrostatics term, our hybrid model accurately reproduces protein folding free-energy landscapes and predicts the structural ensembles of intrinsically disordered proteins. This approach yields a single, unified model that is transferable across both folded and disordered protein states, resolving a long-standing limitation of conventional ISMs. By successfully distilling evolutionary knowledge into a physical potential, our work delivers a foundational implicit solvent model poised to accelerate the development of predictive, large-scale simulation tools.
TOC Graphic
Figure 7.
Introduction
Implicit solvent models (ISMs) represent a promising middle ground in molecular simulation, offering a significant computational advantage over all-atom explicit solvent simulations while providing greater physical detail than lower-resolution coarse-grained models. [1, 2, 3, 4, 5, 6, 7, 8] The middle-ground promise of ISMs has generated significant enthusiasm. Nevertheless, despite decades of refinement,[9, 10, 11, 12, 13, 14, 15, 16, 17, 18, 19, 20, 21, 22, 23, 24] their accuracy remains well below that of their explicit solvent counterparts. Traditional ISMs, such as popular Generalized Born (GB) and surface-area-based formulations, systematically misrepresent the balance between solvation and intramolecular interactions, frequently leading to artifacts like the over-compaction of disordered proteins, overstabilization of -helical conformations, and exaggerated protein–protein association energies. [25, 26, 27, 28, 29, 30, 31]
These deficiencies stem from two overarching limitations. First, ISMs rely on approximate analytical expressions to compute the solvation free energy, ; such approximations cannot fully capture the complex dependence of on molecular composition, geometry, and conformational state. Second, the parameters of ISMs are rarely optimized in a systematic or data-driven manner to reproduce results from experiment or explicit solvent simulations across diverse protein families. The rise of machine learning (ML) offers a path to overcome these challenges. By replacing the approximate analytical formulas with flexible neural network potentials, ML-based force fields could lead to ISMs with much-improved accuracy. Many promising studies have demonstrated the potential of using ML methodologies for force field development. [26, 32, 33, 34, 35, 36, 37, 38, 39, 40, 41, 42, 43, 44, 45, 46, 47, 48, 49, 50, 51, 52, 53, 54, 55, 56, 57, 58, 59, 60, 61, 62, 63, 64, 65, 66, 67, 68] [69, 70, 71, 72, 73, 74, 75, 76, 77, 78, 79, 80, 81, 82, 83, 84, 85]
Graph neural networks 86–108 (GNNs) are particularly well-suited as neural network potentials. Since molecules are naturally structured as graphs, with atoms as nodes and interactions as edges, GNNs provide a native framework for learning complex, geometry-dependent energetic terms. Moreover, their architecture facilitates the straightforward incorporation of fundamental physical principles, such as translational, rotational, and permutational invariance, which is crucial for building robust and generalizable force fields. However, parameters terizing a GNN as an ISM is not trivial. Unlike typical ML force fields trained on quantum mechanical energies, the target values are not directly known, precluding simple supervised learning. Several theoretically sound approaches have been introduced to optimize ISMs by matching configurational ensembles from explicit solvent simulations. [32, 34, 37] While powerful, these methods are constrained by the finite set of proteins for which such simulation data is available, raising concerns about the transferability of the resulting models to new and diverse protein systems.
In this study, we introduce a novel training strategy to address this transferability challenge by learning a proxy for from a protein language model. We leverage the multimodal model ESM3109, which captures the joint distribution of sequence, structure, and function across billions of proteins and attains near-experimental accuracy in predicting 3D structures directly from sequence. Its conditional probabilities, , and corresponding effective energies, , therefore approximate the true folding free-energy landscape. As solvation dominates folding energetics, these evolution-derived probabilities offer an excellent proxy for solvent-mediated effects.
Putting this hypothesis to the test, we first demonstrate that a GNN with significantly fewer parameters can successfully reproduce the effective energies predicted by ESM3, a remarkable distillation of knowledge. The resulting GNN potential is robust enough to drive long-timescale molecular dynamics (MD) simulations. We show that for several proteins, simulations of up to 500 ns maintain stable structures near their native conformations, providing strong evidence that the evolutionary statistics within ESM3 indeed serve as a high-fidelity proxy for the protein folding free-energy landscape.
To create a physically predictive model, we further combine the distilled GNN potential with a standard GB electrostatic term. This hybrid representation not only reproduces explicit-solvent folding free-energy landscapes with high fidelity but also captures the secondary-structure distributions of intrinsically disordered proteins, an area where traditional ISMs routinely fail. Together, these results demonstrate that our approach yields a genuinely transferable ISM capable of describing both ordered and disordered states within a single framework. By distilling the evolutionary knowledge encoded in a protein language model into a compact neural network potential, we establish the first foundational ISM: a scalable, data-driven model that provides a robust starting point for building the next generation of accurate and efficient protein simulations.
Results
Distilling solvent-sensitive secondary structure preferences into a multiscale GNN
To develop an accurate and transferable machine-learning ISM, we require a flexible architecture capable of representing solvent-mediated interactions across diverse proteins. We therefore adopt Schake, a recently introduced multiscale GNN designed for protein-scale systems.107 Schake combines (i) a short-range SAKE message-passing layer 98 that encodes detailed chemical interactions with (ii) a long-range SchNet message-passing layer 96 that aggregates coarse-grained structural context, enabling both accuracy and scalability.
As noted in the Introduction, training ISMs requires either accurate solvation free energies or extensive explicit-solvent configurational ensembles, both of which remain computationally expensive to obtain at the scale needed for broad transferability, although larger datasets are beginning to emerge. 110–112 We therefore turn to an alternative, data-driven proxy by making use of ESM3, a multimodal protein language model trained across billions of natural protein sequences and structures. 109
ESM3 models the joint distribution of protein sequence and structure, enabling residue-level structural predictions from the sequence alone. These predictions reflect the statistical signatures of folding energetics, which are strongly shaped by solvation. Distilling this sequence–structure relationship into Schake would allow the GNN to inherit solvent-sensitive conformational preferences in much the same way that an accurate solvation free energy would.
While ESM3 produces both secondary- and tertiary-structure distributions at the residue level, we begin by distilling a local observable that is solvent sensitive, interpretable, and naturally compatible with graph-based modeling: the 8 secondary-structure motifs (abbreviated as SS8) detailed by the Define Secondary Structure of Proteins (DSSP) algorithm. 113 SS8 motifs encode hydrogen-bonding patterns and steric preferences, providing a clean and tractable signal for transferring solvent-dependent structural tendencies from ESM3 into a GNN architecture.
To transfer the solvent-informed structural knowledge encoded in ESM3, we train Schake to reproduce the ESM3-predicted SS8 motif likelihoods for proteins in the DISPEF-M dataset. 107 Although Schake was originally designed to operate on full atomistic protein structures, we restrict its inputs to the backbone C, C, and N atoms to reduce computational cost while retaining the essential geometric information required for secondary-structure classification. This backbone-only representation allows Schake to capture solvent-sensitive local structural tendencies with minimal overhead (Figure 1 (PDF p. 6)A–B). DISPEF-M provides an ideal training set, offering diverse structural folds for proteins together with atomistic structural models.
Figure 1 (PDF p. 6). Knowledge distillation enables training of an efficient, multiscale GNN architecture. (A) Capped Ala-Val dipeptide with the , C, and N backbone atoms highlighted. These atoms serve as inputs to the GNN. The color of the left half of the circle indicates the backbone atom type, while the color on the right half indicates the amino acid type. (B) Diagram of the Schake multiscale message-passing scheme. The short-ranged SAKE message-passing layer acts on all backbone , C, and N atoms within the cutoff distance nm, while the long-ranged SchNet message-passing layers acts on only atoms within nm. (C) Diagram of the knowledge distillation training methodology. By inputting an amino acid sequence, ESM3 is used to predict a matrix of SS8 motif likelihoods. A GNN that inputs both an amino acid sequence and a structure is then trained to match the ESM3-predicted matrix of SS8 motif likelihoods using the cross entropy loss function. The sequence and structure of chignolin CLN025 is shown as an example, with the likelihoods for the folded state motifs shown in the matrices.
Despite containing only 45,000 parameters, the distilled Schake model closely matches the SS8 likelihoods of the 1.4 billion-parameter ESM3-open model, achieving an average correct-motif probability of 87.0% compared to ESM3-open’s 89.2% (Figure 2). Furthermore, measured on a Nvidia L40S GPU over 200 repeated inferences, Schake achieves a mean prediction time of 2.16 ms for the 80-residue protein -repressor, approximately 9-fold faster than the corresponding ESM3-open inference (mean 19.23 ms). This substantial speedup is crucial for potential downstream integration into molecular simulations.
Figure 2. Schake matches the SS8 motif predictions from ESM3-open. (A) Confusion matrix for ESM3-open SS8 motif predictions. (B) Architecture of the Schake GNN. Backbone atoms are inputted, and SS8 motif logits are outputted. Refer to the Methods section for a detailed description of the Schake architecture. (C) Confusion matrix for GNN SS8 motif predictions. For (A) and (C), the mean predicted likelihood for each SS8 motif conditioned on the true SS8 motif is displayed. The percentage in the upper right corner indicates the mean predicted likelihood of the true SS8 motif (computed by averaging along the diagonal). Each letter corresponds to a particular SS8 motif: G to -helices, H to -helices, I to -helices, T to hydrogen-bonded turns, E to -sheets, B to -bridges, S to non-hydrogen-bonded bends, and C to non-categorized structures.
To assess the transferability and scalability of the distilled model, we further evaluated Schake on the large-protein subset DISPEF-L. DISPEF-L comprises more than 100,000 proteins ranging from 400 to 800 amino acids, substantially larger than those included in DISPEF-M. Schake maintains strong performance in this much broader regime, achieving an average correct-motif probability of 85.2% (Fig. S1). These results demonstrate that Schake generalizes well beyond its training set and scales effectively to proteins far larger than those encountered during training.
GNN-derived energies reliably distinguish folded and unfolded protein states
In the previous section, we showed that Schake can faithfully reproduce the solvent-sensitive SS8 motif statistics distilled from ESM3, providing a learned signal of how local backbone environments respond to solvation. These SS8 motif likelihoods naturally induce a configuration-dependent energy, constructed as a sum of local contributions from individual backbone atoms, which we define as
where denotes the input configuration and is the GNN-predicted likelihood of SS8 motif at backbone atom . The index runs over all backbone atoms (three per residue) used in the GNN predictions. The binary variable indicates which SS8 motifs are present in the reference (folded) structure, such that the resulting one-state (os) energy selectively stabilizes this reference state. The dimensionless scaling factor controls the overall strength of the GNN-derived energy.
Because the GNN was trained exclusively on folded structures, this energy is expected to behave sensibly for local deviations from the native state: as the conformation perturbs away from the folded structure, the likelihoods assigned to native-state motifs decrease, causing the associated energy to rise. As a result, is minimized in the vicinity of the native state, stabilizing the folded structure locally. Whether this behavior persists for large-scale unfolding events, far outside the training distribution, is a key question for determining whether the GNN-derived energy can contribute meaningfully to an ISM.
To assess this, we next evaluate whether increases appropriately for conformations that depart substantially from the native basin, including globally unfolded and highly extended states. A meaningful solvation-like correction should assign higher energies to such configurations than to the folded structure, even though the model was trained exclusively on native-like conformations.
We first evaluated the behavior of on long atomistic trajectories generated by D. E. Shaw Research,114 which contain repeated folding and unfolding events and therefore provide a stringent test of generalization beyond the folded training set. For these fast-folding proteins, we computed both the RMSD from the native structure and as functions of simulation time. As shown in Figure 3 and S2, the RMSD traces display clear two-state switching behavior, with long plateaus in folded or unfolded basins punctuated by rapid transitions between them. Remarkably, the GNN-derived energy mirrors these transitions almost perfectly: whenever the RMSD increases during an unfolding event, rises in tandem, and it returns to a low value immediately upon refolding. Structures closest to the folded state consistently exhibit the lowest , whereas unfolded conformations yield substantially higher energies. Notably, this behavior is not seen in the GBn2[12] ISM (Fig. S3). The tight correlation of with RMSD demonstrates that the SS8-based energy assignment remains reliable even under large conformational changes far outside of the training set.
Figure 3. GNN energy changes closely correspond to structural changes. was computed across long time-scale MD trajectories generated from a previous work. 114 Each trajectory was subset following the procedure used in a previous GNN study. [32] Results for the four largest proteins are shown here, while results for the other eight proteins are shown in the Fig. S2. RMSD from the folded state (reference structures are detailed in the Methods section) is shown in red on the left y-axis, while is shown in black on the right y-axis. Here, the scaling constant and the temperature K. Note that non-standard amino acids and capping groups are excluded from calculation of . For all proteins, the lowest-energy structure is that with the lowest RMSD from the folded state.
We next tested whether this relationship between unfoldedness and GNN energy persists in much larger proteins and in ensembles deliberately biased toward extended conformations. For each protein, we computed directly as a function of the RMSD from the folded structure. Across all systems examined, increases steadily with RMSD (Fig. S4): native-like states exhibit the lowest energies, partially unfolded states carry intermediate energies, and highly extended configurations yield the highest energies. The consistency of this trend demonstrates that the GNN-derived energy generalizes robustly across sequence length, structural complexity, and sampling regime.
Therefore, the pretrained GNN, without any fine-tuning, produces an energy term that robustly distinguishes folded from unfolded ensembles and penalizes large-scale deviations from the native structure.
GNN-Derived Energies Support Stable ML/MD Simulations
We next assess whether the learned energetic term, , can be used directly within MD simulations to propagate large proteins stably over hundreds of nanoseconds.
We performed 500 ns of ML/MD simulations for the eight largest proteins examined by Nguyen et al. [115], using Schake with , beginning from their experimentally determined folded structures. Across this set, Schake consistently preserves native-like conformations (Figure 4). Even the most divergent configurations sampled remain within 4 Å RMSD of the folded state (Tab. S2), indicating that the GNN-derived energy not only favors but also dynamically confines each system to the native basin. Time traces of both RMSD and (Fig. S5) show that energetic fluctuations closely parallel structural deviations, mirroring the tight correspondence between and RMSD observed for the D. E. Shaw Research trajectories.

Figure 4. Figure 4: ML/MD simulations with Schake maintain structures close to their starting folded conformations. Starting folded structures are shown in yellow; ML/MD structures correspond to the lowest- conformations matching the median trajectory RMSD and are shown in teal. Each protein is labeled with its name, length, and RMSD from the starting structure. Flexible regions (Fig. S6, Tab. S1) are excluded from RMSD calculations.
As a control, we performed matched 500 ns GBn2 simulations for the same proteins. In contrast to Schake, three of the GBn2 trajectories sample structures exceeding 4 Å RMSD from the folded state (Fig. S5, Tab. S3), consistent with the tendency of classical GB models to overstabilize partially unfolded or misfolded compact configurations. This comparison highlights that the GNN-based energetic term exhibits superior dynamical stability relative to traditional ISMs.
To further examine structural integrity, DSSP analysis was performed for both the ML/MD trajectories and their corresponding folded structures. Five proteins show essentially unchanged secondary-structure content over 500 ns (Tab. S2). In homeodomain and -repressor, modest reductions in helical content coincide with increases in hydrogen-bonded turn motifs (Fig. S7), consistent with local rearrangements rather than global unfolding. Protein G exhibits a local disruption in the -sheet between residues 42–46. For comparison, GBn2 trajectories also show deviations in helical content (Tab. S3), including loss of helical structure in that remains stable under Schake.
Multi-state energy formulation enables accurate simulation of folded and intrinsically disordered proteins
The ability to convert SS8 motif likelihoods into an energy function that stabilizes folded conformations in MD simulations is notable in itself. The one-state formulation shows that local structural preferences learned by ESM3 can be distilled into a physically meaningful potential capable of identifying and maintaining native basins.
These results naturally raise a deeper question: can such a distilled energy be pushed beyond qualitative stabilization to achieve quantitative agreement with explicit solvent simulations? This is far from guaranteed, as ESM3 was not trained on explicit-solvent data and is designed to favor the native ensemble rather than balance folded, partially unfolded, and disordered states. Achieving quantitative accuracy therefore requires an energy function that can assign reasonable free energies to alternative local motifs present across unfolded ensembles and IDPs. This motivates the introduction of a multi-state energy formulation.
The multi-state (ms) energy evaluates, at each backbone position, the most probable SS8 motif according to the GNN, without privileging any specific reference state:
Notation follows Eq. (1); to maintain differentiability, the max is smoothly approximated using the LogSumExp function (Eq. S1). Conceptually, treats the GNN as a local motif evaluator, rewarding any structurally plausible environment and allowing the energy to adapt seamlessly across folded, partially folded, and disordered conformations.
As with , we computed along the D. E. Shaw Research trajectories. Across most proteins, correlates strongly with RMSD (Fig. S8), and for all but the two smallest systems (CLN025 and Trp-cage), the folded state remains the global minimum. Thus, the multi-state formulation retains the stabilizing behavior demonstrated in previous sections.
To evaluate the ability of the multi-state formulation to reproduce protein folding landscape, we performed 400 ns umbrella sampling simulations for the four largest fast-folding proteins from Lindorff-Larsen et al. 114. Because is a local motif-based solvation term and does not explicitly encode long-range electrostatic screening, we incorporated it as a correction to the GBn2 ISM, which supplies the continuum electrostatics absent from the GNN. The incorporation of GBn2 also introduces additional stabilization of compact, folded conformations, necessitating the use of a smaller scaling factor () than in the standalone GNN simulations of the previous section. Accordingly, we compared three ISMs: GBn2, GBn2 with the ACE nonpolar term (GBn2/ACE), and GBn2 corrected with (GBn2/GNN). The resulting free energy profiles (Figure 5) were benchmarked against TIP3P explicit solvent. 116
Figure 5. The Schake GNN energy, when combined with GBn2, accurately reproduces protein folding landscapes. Umbrella sampling simulations were performed using three different ISMs, and TIP3P results were obtained from Lindorff-Larsen et al. 114.
For protein G, homeodomain, and -repressor, GBn2/ACE correctly stabilizes the folded state but substantially under-favors unfolded configurations relative to TIP3P. GBn2/GNN remedies this imbalance: it retains the correct folded minimum while more accurately recovering the unfolded-state free energies. For 3D, however, all ISMs fail to produce a folded minimum. Increasing in the GBn2/GNN model restores the folded basin and yields close agreement with TIP3P (Fig. S9).
A key design principle of the multi-state energy is that it should produce a more balanced landscape than the one-state formulation. As a protein moves away from its native structure, the energy should not simply rise monotonically, as occurs with , but should instead allow the model to switch to alternative SS8 motifs that are more prevalent in partially folded or unfolded conformations. This switching behavior can lower the energy of non-native structures and is essential for producing realistic free-energy landscapes in which unfolded states may also correspond to local minima.
To examine whether Schake exhibits this behavior, we analyzed its SS8 predictions for homeodomain along the umbrella sampling trajectories, focusing on residues belonging to its three helices. Since GBn2/GNN closely matches the explicit-solvent free energy profile for this protein (Figure 5), it provides an ideal test case. Across all three helices, Schake’s predicted helix and coil probabilities align with the expected motifs in folded structures and, importantly, transition toward coil-like motifs as more unfolded configurations are sampled (Figs. S10–S12). These results demonstrate that Schake dynamically switches the most probable motif as the local environment changes, confirming that functions as intended and can assign low energies to both folded and unfolded states when appropriate.
We next examined intrinsically disordered proteins, where multi-state behavior is essential. We selected three proteins (IDP 9, 12, 30) from a recently reported explicit solvent simulation dataset 110 and performed four 100 ns ML/MD simulations at a slightly elevated temperature (350 K) to enhance conformational sampling. Simulations were performed under vacuum, GBn2, GBn2/ACE, and GBn2/GNN (). Despite the higher temperature, vacuum and GBn2/ACE still collapse the chains into compact structures, deviating substantially from the extended ensembles observed in explicit solvent; this is a known limitation of existing ISMs for modeling IDPs. In contrast, GBn2/GNN produces extended configurations consistent with the TIP3P reference. Although the GBn2/GNN ensembles are slightly more expanded, the model nonetheless places SS8 motifs in the correct regions (Figs. S13–S15). Notably, GBn2 without any additional correction term most-closely matches the TIP3P reference. However, it is still significant that GBn2/GNN does not collapse the chains, especially since our training set predominantly contained folded structures. These results indicate that the multi-state formulation substantially improves IDP modeling relative to existing ISMs, with further tuning likely to yield quantitative accuracy.
Conclusions and Discussion
In this work, we demonstrate that a data-efficient, multiscale GNN, Schake, can successfully distill secondary-structure distributions learned by the multimodal protein language model ESM3. Schake reproduces SS8 motif likelihoods with high fidelity and generalizes effectively to proteins larger and more diverse than those seen during training. This result alone underscores the surprising efficiency with which evolutionary statistics encoded in protein language models can be transferred to compact GNN architectures.
Building on this representation, we show that the learned GNN potential can function directly as a molecular mechanics energy: it reliably differentiates folded from unfolded configurations, assigns the folded state as energetically favorable, and supports long-timescale ML/MD simulations that preserve key structural features. Across 6.8 of simulations on 11 proteins, Schake maintains near-native conformations and exhibits correlated energetic and structural fluctuations characteristic of physically meaningful folding landscapes.
To extend applicability beyond folded proteins, we introduced a multi-state energy formulation that corrects the GBn2 implicit solvation electrostatics using the GNN-derived potential. This hybrid model accurately reproduces folding free-energy profiles for fast-folding proteins and, notably, captures secondary-structure propensities of intrinsically disordered proteins. The ability to model both folded and disordered states with a single, efficient potential provides strong evidence that distilled protein-language-model statistics offer a powerful basis for implicit solvation modeling.
While these results establish a compelling proof of principle, our present model is not yet a production-ready ISM. Schake was trained primarily on folded structures, and future efforts will benefit from expanding the training set to better represent IDPs, systematically fine-tuning against explicit-solvent simulations, and optimizing GPU kernels to unlock even greater simulation throughput. Nevertheless, the success of our approach demonstrates that knowledge distillation from protein language models is a viable route toward scalable, transferable ISMs. We anticipate that this strategy will enable the development of next-generation ISMs capable of robustly capturing the thermodynamics of proteins across the folded–disordered spectrum.
Methods
Training Schake to predict solvent-sensitive SS8 motifs
Secondary structure targets from DISPEF and ESM3
To train Schake to predict solvent-sensitive secondary structure motifs from protein conformations, we used the DISPEF dataset introduced previously. 107 We focus on the DISPEF-M subset, which contains 19,200 training proteins and 4,800 testing proteins, spanning sizes from 16 to 399 amino acids. For each protein, we extracted Cartesian coordinates for the backbone , carbonyl carbon (C), and amide nitrogen (N) atoms, along with the corresponding amino acid identities.
Secondary structure (SS8) motif labels were assigned using both model predictions and reference annotations. For each protein sequence, SS8 likelihoods were predicted using the ESM3-open protein language model. Version 3.2.0 of the esm package was used. Amino acid sequences were first tokenized using the EsmSequenceTokenizer, and the resulting token sequence was passed through the forward function of ESM3-open. For the th protein containing residues, the model outputs a matrix of SS8 logits (“pseudo-energies”) . These logits were converted to SS8 likelihoods for each residue via a softmax transformation,
yielding a probability matrix .
Each column corresponds to one of the eight DSSP secondary structure classes, ordered as follows: helices, helices, helices, hydrogen-bonded turns, sheets, bridges, non-hydrogen-bonded bends, and all remaining structures.
Reference SS8 labels were computed directly from atomic coordinates using the DSSP algorithm as implemented in the mdtraj package (version 1.10.1). 113 These labels were encoded as one-hot matrices .
Because Schake operates on backbone atoms rather than residues, each residue-level SS8 probability or label was triplicated for the corresponding Cα, C, and N atoms. As a result, both and were expanded to dimension for ESM3 inference and GNN training.
Schake architecture for SS8 likelihood prediction
The Schake GNN architecture is described in detail in prior works. 96,98,107 Originally developed to predict scalar energies for all-atom protein structures, Schake was modified here to predict SS8 motif likelihoods for backbone-only representations. Each backbone atom is assigned an initial embedding
where and are trainable vectors (16 dimensions each) associated with the amino acid identity and backbone atom type, respectively. These embeddings are then passed through two message passing layers to produce updated embeddings . Given an input configuration for protein , Schake outputs a matrix of SS8 pseudo-energies . These outputs are obtained by passing the updated embeddings through a three-layer feed-forward network. All architectural hyperparameters are reported in Tab. S4.
For backbone atom , the predicted likelihood of SS8 motif is obtained via a softmax,
These likelihoods define a local motif energy
which contributes to the total GNN-derived energy as defined in Eqs. (1) and (2). To ensure numerical stability, was added.
Knowledge distillation and training procedure
Schake was trained using a knowledge distillation framework, 117 with ESM3-open serving as the teacher model. Training was performed using PyTorch (version 2.0) 118 and PyTorch Geometric (version 2.3.1). 119
The primary loss was an averaged cross-entropy loss (Eq. S2) between the ESM3-predicted SS8 likelihoods and the Schake-predicted likelihoods . To further anchor predictions to physical structure, a second cross-entropy term between the DSSP-derived one-hot labels and was included. The total training loss was therefore
Training and test-set accuracies are reported in Figure 2 and Fig. S1, respectively. Models were trained for 120 epochs using a batch size of 50 proteins. Optimization was performed with the Adam optimizer,120 using an initial learning rate of , decayed by a factor of 0.9 every three epochs. L2 regularization, with , was applied via weight decay during optimizer initialization.
Classical MD and ML/MD simulation details
All classical MD and ML/MD simulations were performed using OpenMM (version 8.0.0).[60] ML/MD simulations employed the OpenMM-Torch plugin (version 1.0) with PyTorch (version 1.11.0), PyTorch-Cluster (version 1.5.9), and PyTorch-Scatter (version 2.0.8).118,119
Initial structures for CspA, Top7, Hyp Protein, and NTL were obtained from the Protein Data Bank121 (structure IDs 1MJC,122 1QYS,123 1WHZ, and 2HBA,124 respectively), while the initial structures for the proteins shown in Figure 5 and Figure 6 were obtained from previous works.110,114 These initial structures also serve as the reference structures for all RMSD calculation. Simulation inputs were prepared using tleap from AmberTools23.125
Figure 6. ML/MD simulations prevent the collapse of disordered proteins. The solid bar inside each box represents the median radius of gyration value for the three IDPs under different solvent treatments. The bottom and top boundaries of each box represent the first and third quartile values, respectively. The bottom and top whiskers encompass values that are 1.5 times the inter-quartile range below the first quartile and above the third quartile, respectively. TIP3P reference values were obtained from Wang and Zhang 110.
Unless otherwise noted, simulations used the ff14SBonlysc vacuum force field.126,127 Unfolding simulations presented in Fig. S4 were performed using the CHARMM36 vacuum force field.128 Additional simulation parameters and integration details are provided in the Supporting Information,129,130 and performance benchmarks are summarized in Tabs. S5–S7. Trajectory analysis was performed using mdtraj (version 1.10.3)131 and ChimeraX (version 1.10.1).132 Umbrella sampling free energy profiles (Figure 5) were computed using the FastM-BAR package (version 1.4.3).133
Supporting Information
Expanded equations, simulation setup information, and additional supporting figures and tables.
Data Availability
Code for our modified Schake architecture will be made available on GitHub at the following link: https://github.com/ZhangGroup-MITChemistry/Schake_GNN/. We also will include examples of how to use Schake in ML/MD simulations. Additionally, the optimized model parameters from training will be provided.
Acknowledgement
This work was supported by the U.S. National Institutes of Health (Grant R35GM133580). J.A. was supported by the the U.S. National Science Foundation Graduate Research Fellowship. We thank the MIT Office of Research Computing and Data for providing high performance computing resources. This work also used Bridges-2 at the Pittsburgh Supercomputing Center through allocations BIO240299 and CHE240139 from the Advanced Cyberinfrastructure Coordination Ecosystem: Services & Support (ACCESS) program, which is supported by U.S. National Science Foundation grants #2138259, #2138286, #2138307, #2137603, and #2138296.
Competing interests: Authors declare that they have no competing interests.
References
- [1]Latham, A. P.; Zhang, B. Improving Coarse-Grained Protein Force Fields with Small-Angle X-ray Scattering Data. J. Phys. Chem. B 2019, 123, 1026–1034.DOI
- [2]Latham, A. P.; Zhang, B. Maximum Entropy Optimized Force Field for Intrinsically Disordered Proteins. J. Chem. Theory Comput. 2020, 16, 773–781.
- [3]Liu, S.; Wang, C.; Latham, A. P.; Ding, X.; Zhang, B. OpenABC enables flexible, simplified, and efficient GPU accelerated simulations of biomolecular condensates. PLOS Comput. Biol. 2023, 19, e1011442.
- [4]Liu, S.; Wang, C.; Zhang, B. Toward Predictive Coarse-Grained Simulations of Biomolecular Condensates. Biochemistry 2025, 64, 1750–1761.DOI
- [5]Chen, M.; Schafer, N. P.; Zheng, W.; Wolynes, P. G. The Associative Memory, Water Mediated, Structure and Energy Model (AWSEM)-AmyloMETER: Predicting Amyloid Propensity and Fibril Topology Using an Optimized Folding Landscape Model. ACS Chem. Neurosci. 2018, 9, 1027–1039.DOI
- [6]Joseph, J. A.; Reinhardt, A.; Aguirre, A.; Chew, P. Y.; Russell, K. O.; Espinosa, J. R.; Garaizar, A.; Collepardo-Guevara, R. Physics-driven coarse-grained model for biomolecular phase separation with near-quantitative accuracy. Nat. Comput. Sci. 2021, 1, 732–743.DOI
- [7]Dignon, G. L.; Zheng, W.; Kim, Y. C.; Best, R. B.; Mittal, J. Sequence determinants of protein phase behavior from a coarse-grained model. PLOS Comput. Biol. 2018, 14, e1005941.
- [8]Baul, U.; Chakraborty, D.; Mugnai, M. L.; Straub, J. E.; Thirumalai, D. Sequence Effects on Size, Shape, and Structural Heterogeneity in Intrinsically Disordered Proteins. J. Phys. Chem. B 2019, 123, 3462–3474.
- [9]Hawkins, G. D.; Cramer, C. J.; Truhlar, D. G. Parametrized Models of Aqueous Free Energies of Solvation Based on Pairwise Descreening of Solute Atomic Charges from a Dielectric Medium. J. Phys. Chem. 1996, 100, 19824–19839.DOI
- [10]Onufriev, A.; Bashford, D.; Case, D. A. Exploring protein native states and large-scale conformational changes with a modified generalized born model. Proteins 2004, 55, 383–394.
- [11]Mongan, J.; Simmerling, C.; McCammon, J. A.; Case, D. A.; Onufriev, A. Generalized Born model with a simple, robust molecular volume correction. J. Chem. Theory Comput. 2007, 3, 156–169.
- [12]Nguyen, H.; Roe, D. R.; Simmerling, C. Improved Generalized Born Solvent Model Parameters for Protein Simulations. J. Chem. Theory Comput. 2013, 9, 2020–2034.
- [13]Gallicchio, E.; Levy, R. M. AGBNP: an analytic implicit solvent model suitable for molecular dynamics simulations and high-resolution modeling. J. Comput. Chem. 2004, 25, 479–499.
- [14]Chen, J.; Im, W.; Brooks, C. L. Balancing Solvation and Intramolecular Interactions: Toward a Consistent Generalized Born Force Field. J. Am. Chem. Soc. 2006, 128, 3728–3736.
- [15]Lee, M. S.; Salsbury, F. R., Jr.; Brooks, C. L., III Novel generalized Born methods. J. Chem. Phys. 2002, 116, 10606–10614.
- [16]Lee, M. S.; Feig, M.; Salsbury, F. R.; Brooks, C. L. New analytic approximation to the standard molecular volume definition and its application to generalized Born calculations. J. Comput. Chem. 2003, 24, 1348–1356.
- [17]Im, W.; Lee, M. S.; Brooks, C. L. Generalized born model with a simple smoothing function. J. Comput. Chem. 2003, 24, 1691–1702.
- [18]Haberthür, U.; Caflisch, A. FACTS: Fast analytical continuum treatment of solvation. J. Comput. Chem. 2008, 29, 701–715.
- [19]Vitalis, A.; Pappu, R. V. ABSINTH: A new continuum solvation model for simulations of polypeptides in aqueous solutions. J. Comput. Chem. 2009, 30, 673–699.DOI
- [20]Chekmarev, D. S.; Ishida, T.; Levy, R. M. Long-Time Conformational Transitions of Alanine Dipeptide in Aqueous Solution: Continuous and Discrete-State Kinetic Models. J. Phys. Chem. B 2004, 108, 19487–19495.DOI
- [21]Lee, K. H.; Chen, J. Optimization of the GBMV2 Implicit Solvent Force Field for Accurate Simulation of Protein Conformational Equilibria. J. Comput. Chem. 2017, 38, 1332–1341.DOI
- [22]Onufriev, A. V.; Case, D. A. Generalized Born Implicit Solvent Models for Biomolecules. Annu. Rev. Biophys. 2019, 48, 275–296.
- [23]Onufriev, A. V.; Izadi, S. Water models for biomolecular simulations. Wiley Interdiscip. Rev. Comput. Mol. Sci. 2018, 8, e1347.
- [24]Akkermans, R. L. C. Solvation Free Energy of Regular and Azeotropic Molecular Mixtures. J. Phys. Chem. B 2017, 121, 1675–1683.DOI
- [25]Arsiccio, A.; Pisano, R.; Shea, J.-E. A New Transfer Free Energy Based Implicit Solvation Model for the Description of Disordered and Folded Proteins. J. Phys. Chem. B 2022, 126, 6180–6190.
- [26]Greener, J. G. Differentiable simulation to develop molecular dynamics force fields for disordered proteins. Chem. Sci. 2024, 15, 4897–4909.
- [27]Shao, Q.; Zhu, W. Assessing AMBER force fields for protein folding in an implicit solvent. Phys. Chem. Chem. Phys. 2018, 20, 7206–7216.
- [28]Lang, E. J. M.; Baker, E. G.; Woolfson, D. N.; Mulholland, A. J. Generalized Born Implicit Solvent Models Do Not Reproduce Secondary Structures of De Novo Designed Glu/Lys Peptides. J. Chem. Theory Comput. 2022, 18, 4070–4076.DOI
- [29]Shell, M. S.; Ritterson, R.; Dill, K. A. A Test on Peptide Stability of AMBER Force Fields with Implicit Solvation. J. Phys. Chem. B 2008, 112, 6878–6886.DOI
- [30]Best, R. B. Computational and theoretical advances in studies of intrinsically disordered proteins. Curr. Opin. Struct. Biol. 2017, 42, 147–154.DOI
- [31]Bottaro, S.; Lindorff-Larsen, K.; Best, R. B. Variational Optimization of an All-Atom Implicit Solvent Force Field To Match Explicit Solvent Simulation Data. J. Chem. Theory Comput. 2013, 9, 5641–5652.
- [32]Airas, J.; Ding, X.; Zhang, B. Transferable Implicit Solvation via Contrastive Learning of Graph Neural Networks. ACS Cent. Sci. 2023, 9, 2286–2297.
- [33]Katzberger, P.; Riniker, S. Implicit solvent approach based on generalized Born and transferable graph neural networks for molecular dynamics simulations. J. Chem. Phys. 2023, 158, 204101.DOI
- [34]Katzberger, P.; Riniker, S. A general graph neural network based implicit solvation model for organic molecules in water. Chem. Sci. 2024, 15, 10794–10802.DOI
- [35]Katzberger, P.; Hauswirth, L. M.; Kuhn, A. S.; Landrum, G. A.; Riniker, S. Rapid Access to Small Molecule Conformational Ensembles in Organic Solvents Enabled by Graph Neural Network-Based Implicit Solvent Model. J. Am. Chem. Soc. 2025, 147, 13264–13275.DOI
- [36]Katzberger, P.; Pultar, F.; Riniker, S. Transferring Knowledge from MM to QM: A Graph Neural Network-Based Implicit Solvent Model for Small Organic Molecules. J. Chem. Theory Comput. 2025, 21, 7450–7459.DOI
- [37]Chen, Y.; Krämer, A.; Charron, N. E.; Husic, B. E.; Clementi, C.; Noé, F. Machine learning implicit solvation for molecular dynamics. J. Chem. Phys. 2021, 155, 084101.
- [38]Husic, B. E.; Charron, N. E.; Lemm, D.; Wang, J.; Pérez, A.; Majewski, M.; Krämer, A.; Chen, Y.; Olsson, S.; de Fabritiis, G. et al. Coarse graining molecular dynamics with graph neural networks. J. Chem. Phys. 2020, 153, 194101.
- [39]Kozinsky, B.; Musaelian, A.; Johansson, A.; Batzner, S. Scaling the Leading Accuracy of Deep Equivariant Models to Biomolecular Simulations of Realistic Size. Proceedings of the International Conference for High Performance Computing, Networking, Storage and Analysis. 2023; pp 1–12.arxiv.org/abs/2304.10061
- [40]Wang, Z.; Wu, H.; Sun, L.; He, X.; Liu, Z.; Shao, B.; Wang, T.; Liu, T.-Y. Improving machine learning force fields for molecular dynamics simulations with fine-grained force metrics. J. Chem. Phys. 2023, 159, 035101.
- [41]Brunken, C.; Boyer, S.; Omar, M.; Diallo, B. N.; Beguir, K.; Carranza, N. L.; Bent, O. Machine learning of force fields towards molecular dynamics simulations of proteins at DFT accuracy. ICLR 2024 Workshop on Generative and Experimental Perspectives for Biomolecular Design. Vienna, Austria. 2024.
- [42]Yao, S.; Van, R.; Pan, X.; Park, J. H.; Mao, Y.; Pu, J.; Mei, Y.; Shao, Y. Machine learning based implicit solvent model for aqueous-solution alanine dipeptide molecular dynamics simulations. RSC Adv. 2023, 13, 4565–4577.DOI
- [43]Wang, J.; Olsson, S.; Wehmeyer, C.; Pérez, A.; Charron, N. E.; de Fabritiis, G.; Noé, F.; Clementi, C. Machine Learning of Coarse-Grained Molecular Dynamics Force Fields. ACS Cent. Sci. 2019, 5, 755–767.
- [44]Wang, J.; Charron, N.; Husic, B.; Olsson, S.; Noé, F.; Clementi, C. Multi-body effects in a coarse-grained protein force field. J. Chem. Phys. 2021, 154, 164113.DOI
- [45]Fu, X.; Wu, Z.; Wang, W.; Xie, T.; Keten, S.; Gomez-Bombarelli, R.; Jaakkola, T. S. Forces are not Enough: Benchmark and Critical Evaluation for Machine Learning Force Fields with Molecular Simulations. arXiv 2023, arXiv:2210.07237, accessed on 2024-04-09.arxiv.org/abs/2210.07237
- [46]Durumeric, A. E. P.; Chen, Y.; Noé, F.; Clementi, C. Learning data efficient coarse-grained molecular dynamics from forces and noise. arXiv 2024, arXiv:2407.01286, accessed on 2024-09-06.arxiv.org/abs/2407.01286
- [47]Arts, M.; Garcia Satorras, V.; Huang, C.-W.; Zügner, D.; Federici, M.; Clementi, C.; Noé, F.; Pinsler, R.; van den Berg, R. Two for One: Diffusion Models and Force Fields for Coarse-Grained Molecular Dynamics. J. Chem. Theory Comput. 2023, 19, 6151–6159.
- [48]Köhler, J.; Chen, Y.; Krämer, A.; Clementi, C.; Noé, F. Flow-Matching: Efficient Coarse-Graining of Molecular Dynamics without Forces. J. Chem. Theory Comput. 2023, 19, 942–952.DOI
- [49]Loose, T. D.; Sahrmann, P. G.; Qu, T. S.; Voth, G. A. Coarse-Graining with Equivariant Neural Networks: A Path Toward Accurate and Data-Efficient Models. J. Phys. Chem. B 2023, 127, 10564–10572.DOI
- [50]Durumeric, A. E. P.; Voth, G. A. Adversarial-residual-coarse-graining: Applying machine learning theory to systematic molecular coarse-graining. J. Chem. Phys. 2019, 151, 124110.DOI
- [51]Durumeric, A. E. P.; Charron, N. E.; Templeton, C.; Musil, F.; Bonneau, K.; Pasos-Trejo, A. S.; Chen, Y.; Kelkar, A.; Noé, F.; Clementi, C. Machine learned coarse-grained protein force-fields: Are we there yet? Curr. Opin. Struct. Biol. 2023, 79, 102533.DOI
- [52]Ding, X.; Zhang, B. Contrastive Learning of Coarse-Grained Force Fields. J. Chem. Theory Comput. 2022, 18, 6334–6344.DOI
- [53]Ding, X. Optimizing Force Fields with Experimental Data Using Ensemble Reweighting and Potential Contrasting. J. Phys. Chem. B 2024, 128, 6760–6769.DOI
- [54]Chmiela, S.; Sauceda, H. E.; Müller, K.-R.; Tkatchenko, A. Towards exact molecular dynamics simulations with machine-learned force fields. Nat. Commun. 2018, 9, 3887.DOI
- [55]Chmiela, S.; Sauceda, H. E.; Tkatchenko, A.; Müller, K.-R. In Machine Learning Meets Quantum Physics; Schütt, K. T., Chmiela, S., von Lilienfeld, O. A., Tkatchenko, A., Tsuda, K., Müller, K.-R., Eds.; Springer International Publishing, 2020; pp 129–154.
- [56]Duschatko, B. R.; Fu, X.; Owen, C.; Xie, Y.; Musaelian, A.; Jaakkola, T.; Kozinsky, B. Thermodynamically Informed Multimodal Learning of High-Dimensional Free Energy Models in Molecular Coarse Graining. arXiv 2024, arXiv:2405.19386, accessed on 2024-06-24.arxiv.org/abs/2405.19386
- [57]Yang, W.; Templeton, C.; Rosenberger, D.; Bittracher, A.; Nüske, F.; Noé, F.; Clementi, C. Slicing and Dicing: Optimal Coarse-Grained Representation to Preserve Molecular Kinetics. ACS Cent. Sci. 2023, 9, 186–196.DOI
- [58]Corso, G.; Stark, H.; Jegelka, S.; Jaakkola, T.; Barzilay, R. Graph neural networks. Nat. Rev. Methods Primers 2024, 4, 1–13.
- [59]Batzner, S.; Musaelian, A.; Kozinsky, B. Advancing molecular simulation with equivariant interatomic potentials. Nat. Rev. Phys. 2023, 5, 437–438.
- [60]Eastman, P.; Galvelis, R.; Peláez, R. P.; Abreu, C. R. A.; Farr, S. E.; Gallicchio, E.; Gorenko, A.; Henry, M. M.; Hu, F.; Huang, J. et al. OpenMM 8: Molecular Dynamics Simulation with Machine Learning Potentials. J. Phys. Chem. B 2024, 128, 109–116.
- [61]Zheng, S.; He, J.; Liu, C.; Shi, Y.; Lu, Z.; Feng, W.; Ju, F.; Wang, J.; Zhu, J.; Min, Y. et al. Predicting equilibrium distributions for molecular systems with deep learning. Nat. Mach. Intell. 2024, 6, 558–567.
- [62]Sahrmann, P. G.; Loose, T. D.; Durumeric, A. E. P.; Voth, G. A. Utilizing Machine Learning to Greatly Expand the Range and Accuracy of Bottom-Up Coarse-Grained Models through Virtual Particles. J. Chem. Theory Comput. 2023, 19, 4402–4413.
- [63]Bonneau, K.; Lederer, J.; Templeton, C.; Rosenberger, D.; Giambagli, L.; Müller, K.-R.; Clementi, C. Peering inside the black box by learning the relevance of many-body functions in neural network potentials. Nat. Commun. 2025-11-10, 16, 9898.
- [64]Anstine, D. M.; Isayev, O. Machine Learning Interatomic Potentials and Long-Range Physics. J. Phys. Chem. A 2023, 127, 2417–2431.DOI
- [65]Cheng, Z.; Bi, H.; Liu, S.; Chen, J.; Misquitta, A. J.; Yu, K. Developing a Differentiable Long-Range Force Field for Proteins with E(3) Neural Network-Predicted Asymptotic Parameters. J. Chem. Theory Comput. 2024, 20, 5598–5608.
- [66]Faller, C.; Kaltak, M.; Kresse, G. Density-based long-range electrostatic descriptors for machine learning force fields. J. Chem. Phys. 2024, 161, 214701.
- [67]Takaba, K.; J. Friedman, A.; E. Cavender, C.; Kumar Behara, P.; Pulido, I.; M. Henry, M.; MacDermott-Opeskin, H.; R. Iacovella, C.; M. Nagle, A.; Matthew Payne, A. et al. Machine-learned molecular mechanics force fields from large-scale quantum chemical data. Chem. Sci. 2024, 15, 12861–12878.
- [68]Wang, Y.; Pulido, I.; Takaba, K.; Kaminow, B.; Scheen, J.; Wang, L.; Chodera, J. D. EspalomaCharge: Machine Learning-Enabled Ultrafast Partial Charge Assignment. J. Phys. Chem. A 2024, 128, 4160–4167.DOI




