betaFeedback

WorksPhysics and Astronomy

Scaling Graph Neural Networks to Large Proteins

Abstract

Graph neural network (GNN) architectures have emerged as promising force field models, exhibiting high accuracy in predicting complex energies and forces based on atomic identities and Cartesian coordinates. To expand the applicability of GNNs, and machine learning force fields more broadly, optimizing their computational efficiency is critical, especially for large biomolecular systems in classical molecular dynamics simulations. In this study, we address key challenges in existing GNN benchmarks by introducing a dataset, DISPEF, which comprises large, biologically relevant proteins. DISPEF includes 207,454 proteins with sizes up to 12,499 atoms and features diverse chemical environments, spanning folded and disordered regions. The implicit solvation free energies, used as training targets, represent a particularly challenging case due to their many-body nature, providing a stringent test for evaluating the expressiveness of machine learning models. We benchmark the performance of seven GNNs on DISPEF, emphasizing the importance of directly accounting for long-range interactions to enhance model transferability. Additionally, we present a novel multiscale architecture, termed Schake, which delivers transferable and computationally efficient energy and force predictions for large proteins. Our findings offer valuable insights and tools for advancing GNNs in protein modeling applications.

Introduction

Machine learning (ML) force fields have garnered considerable attention in recent years due to their ability to account for many-body effects with higher accuracy and computational efficiency compared to ab initio methods.[1, 2, 3, 4, 5, 6, 7, 8, 9, 10, 11, 12, 13, 14, 15, 16, 17, 18, 19, 20, 21, 22, 23, 24, 25, 26, 27, 28, 29, 30, 31, 32, 33, 34, 35, 36, 37, 38, 39, 40, 41] Given that molecules are naturally structured as graphs, with atoms represented as nodes and interactions as edges, graph neural networks (GNNs) offer a compelling framework for force field modeling. These networks also facilitate the straightforward incorporation of physical symmetries, such as translational, rotational, and permutational symmetries, further boosting their appeal.

Although numerous GNN architectures have been proposed,[42, 43, 44, 45, 46, 47, 48, 49, 50, 51, 52, 53, 54, 55, 56, 57, 58, 59, 60, 61] they generally follow a similar operational principle. Initially, feature vectors are assigned to atoms based on their type. These atomic features are then updated via the message-passing layers, which incorporate information from neighboring atoms. Specifically, each message-passing layer, totaling NlayersN_{\mathrm{layers}}, updates atomic features by aggregating data from neighboring atoms jj within a cutoff radius rcutr_{\mathrm{cut}} from atom ii. As this process repeats across layers, increasingly distant atomic information is incorporated. Finally, the updated atomic features are transformed into contributions to the total potential energy, while force predictions are obtained by calculating the negative gradient of the predicted energy with respect to the atomic positions.

The development of GNN architectures has traditionally prioritized accuracy and expressiveness, with limited attention to computational efficiency. Although ML-based force fields are generally more efficient than quantum mechanical methods, they remain significantly slower compared to highly optimized classical force fields. Notably, several studies have applied GNNs to protein molecules in MD simulations, [1, 2, 3, 4, 5, 6, 7, 8, 9, 10, 11, 12, 13, 14, 15, 57] demonstrating impressive accuracy but encountering limited practical utility due to high computational costs. Consequently, it is essential to systematically assess the performance of GNN architectures on large biomolecules to identify optimal designs and potentially guide future advancements.

Benchmarking the performance of GNN architectures on large biomolecules is complicated by the lack of datasets. Most existing datasets such as QM9, [62] ISO17, [52] MD17, [63] the ANI datasets, [64, 65] the SPICE datasets, [66, 67] and the ZINC dataset, 68 contain only small molecules. While datasets like COMP6v2 [69, 70] and MD22 [71] contain a handful of larger structures, they are still relatively limited in their representation of large, biologically-relevant molecules. Due to these dataset limitations, previous GNN benchmarking studies [13, 72, 73, 74, 75] have not focused on large biomolecules.

In this study, we address the challenges of benchmarking and developing GNN architectures for large, biologically relevant proteins. We first present a dataset, DISPEF (Dataset of Implicit Solvation Protein Energies and Forces), which contains implicit solvation free energies for proteins ranging in size from 16 to 1,022 amino acids. The dataset includes both folded and disordered regions, offering a diverse array of chemical environments. Along with the many-body nature of solvation free energies, these environments present challenging cases for evaluating the effectiveness of ML approaches in mapping atomistic representations to energy. We then benchmark several recent GNN architectures using DISPEF, providing critical insights into key design features essential for ensuring model accuracy and transferability. Building on these benchmark results, we introduce a hybrid, multiscale GNN architecture that demonstrates high efficiency and transferability, particularly to proteins significantly larger than those in the training set. This custom GNN architecture outperforms all existing models examined in this work. Overall, our study offers valuable tools and insights to facilitate the advancement of GNN architectures tailored for protein applications.

Results

DISPEF for benchmarking GNN performance on large proteins

Much of the recent effort to develop more accurate GNNs has largely focused on minimizing prediction errors on widely-used datasets of small, drug-like molecules. However, to develop GNNs that are capable of efficiently and transferably scaling to large and complex proteins containing thousands of atoms, datasets of large protein structures are necessary. To address this data limitation, we’ve constructed a dataset of implicit solvation protein energies and forces (abbreviated as DISPEF) that includes over 200,000 proteins ranging in size from 16 to 1,022 amino acids. This dataset contains AlphaFold2[76, 77]-predicted structures for proteins within the Swiss-Prot database.[78] Since the Swiss-Prot database is hand-curated to minimize sequence redundancy, and since structure predictions from AlphaFold2 contain both higher-confidence folded regions and lower-confidence unfolded / disordered regions, DISPEF covers a wide range of chemical environments.

In addition to all-atom protein structures, we incorporate solvation free energies and the corresponding atomic forces, computed using the GBn2 model,[79, 80] as target values in DISPEF. Although existing databases often provide quantum mechanical energies and forces,[57, 81] calculating these for proteins of biologically relevant size is computationally prohibitive. However, unlike many of the pairwise energy terms commonly used in MD simulations, GB-based implicit solvent models utilize many-body potentials to estimate solvation free energy, accounting for both electrostatic and non-polar contributions.[82] The long-range and many-body characteristics of these models offer robust benchmarks for assessing the capability of GNNs to learn complex relationships that map atomic structures to energy.

We created four subsets of DISPEF that aim to assess different capabilities of GNNs. Representative structures from three of these subsets are displayed in Figure 1. Throughout this manuscript, we’ll refer to the subsets as follows:

  1. DISPEF-S - this consists of 81,341 proteins with less than 2,000 atoms. This subset

DISPEF-S, DISPEF-M, and DISPEF-L protein subsets

Figure 1. DISPEF and its subsets were constructed to evaluate GNN architectures. The first subset (left, denoted DISPEF-S) contains ∼\sim 81,000 proteins with under 2,000 atoms. The second subset (center, denoted DISPEF-M) contains 24,000 proteins with under 400 amino acids. The third subset (right, denoted DISPEF-L) contains over 109,000 proteins ranging in size from 6,800 to 12,499 atoms. A smaller fourth subset, DISPEF-c (not pictured), was constructed to evaluate the computational cost of GNNs.

was further split into training and testing sets consisting of 65,072 and 16,269 proteins, respectively.

  1. DISPEF-M - this consists of 24,000 proteins with less than 400 amino acids (∼\sim 6,800 atoms). This subset was further split into training and testing sets consisting of 19,200 and 4,800 proteins, respectively.

  1. DISPEF-L - this consists of 109,108 proteins with greater than 6,800 atoms and less than 12,499 atoms.

  1. DISPEF-c - this consists of 560 randomly selected proteins, spaced roughly evenly in size from 192 to 12,346 atoms.

DISPEF-S aims to assess the ability of GNNs to produce accurate predictions across a large quantity of relatively small proteins, while DISPEF-M aims to assess the scalability of GNNs across a smaller set of different sized proteins. DISPEF-L aims to assess the transferability of GNNs to structures significantly larger than those in the training set. While DISPEF-L could also be used to train GNNs, this would likely require a GPU with greater than 32 GB of memory. Lastly, the smaller DISPEF-c was constructed to evaluate the computational cost of GNNs. Overall, the construction of DISPEF enables comprehensive investigation into the accuracy, scalability, and transferability of GNN architectures to large, biologically-relevant proteins.

Selection of diverse and efficient GNN architectures

With the creation of DISPEF, we can now benchmark the prediction accuracy of various GNN architectures on large, biologically-relevant proteins. We focus on a wide collection of recently introduced GNNs with diverse architectural designs, whose superior performances and computational efficiency have been documented in prior studies. As such, the SchNet,[52] Graph Network (GN),[2] EGNN,[53] SAKE,[54] Equivariant Transformer (ET),[55, 56] and ViSNet[57] architectures were selected. While GNN architectures that utilize Clebsh-Gordon tensor products to achieve equivariance have proven to be highly accurate for small molecules,[58, 59, 60, 61] they were not investigated in this study due to their well-attested computational cost.[57] Additionally, while not a GNN, we also included ANI[64, 65, 70] in this study due to its computational efficiency and its similar use-cases to GNNs.

While we point to the work of Han et al. 83 for a comprehensive overview of GNN architectures, we’ll briefly detail some of the differences between the selected GNNs. The SchNet architecture utilizes continuous filter convolutions acting on radial basis expansions of pairwise distances rjir_{ji} between atom ii and its neighboring atoms jj to compute the feature update for atom ii.[52] The GN architecture is a modified SchNet architecture that incorporates a radial basis function with trainable parameters and an initial atomic featurization that incorporates information from neighboring atoms.[2] By operating only on the distances between atoms, both SchNet and GN predictions are invariant with respect to E(3)E(3) transformations, corresponding to translational, rotational, and reflectional symmetry. Recent GNN developments have focused on equivariant designs, where predictions will undergo equivalent transformations when a certain transformation was performed on the input. Such designs can be useful for predicting vectorial quantities such as forces or dipole moments, whose directions depend on the molecule orientation. The EGNN architecture utilizes edge features defined between atoms ii and jj to compute the feature update for atom ii, and can additionally be made equivariant by performing a coordinate transformation in each message-passing layer. [53] The SAKE architecture is a recent improvement to EGNN that incorporates attention acting on edge features along with SchNet-style continuous filter convolutions to compute the feature update for atom ii. [8] We opted not to perform coordinate transformation within EGNN and SAKE, as this made training unstable for the large proteins that we focused on in this study. As such, we used E(3)E(3)-invariant versions of EGNN and SAKE. The ET architecture is an equivariant graph transformer that utilizes atomic features and pairwise distances rjir_{ji} to compute the query, key, and value needed to compute attention weights and then the feature update for atom ii. [55] ViSNet, the most recent of the architectures we selected, is an equivariant GNN that utilizes dot products between pairwise displacement unit vectors and between vector rejections to incorporate angular and torsional information. [57]

Lastly, ANI computes feature vectors using the atom types, radial, and angular information from neighboring atoms, and then transforms those feature vectors into energy contributions using an ensemble of feed-forward neural networks (one per atom type). [64, 70, 84]

Larger cutoff distances are required for transferable predictions

We next evaluated the energy prediction accuracy of 6 different GNNs and ANI using our subsets of DISPEF. Performance of GNNs can be highly dependent on hyperparameters, so we carried out a systematic hyperparameter optimization for each architecture. Due to the computational cost for training on DISPEF, the optimization was separated into two stages. First, we optimized on a dataset of MD configurations and their DFT-level energies and forces for chignolin CLN025. [81, 85] Methodological details of this training are discussed in the Supporting Information, and prediction errors can be viewed in Table S1. Then we optimized over DISPEF-S. To reduce training time as much as possible, we implemented a custom, variable-size batch sampler to ensure optimal VRAM usage during training (detailed in the Methods section). From hyperparameter optimization, we found that the number of message-passing layers (NlayersN_{\mathrm{layers}}) and the cutoff distance for identifying neighboring atoms (rcutr_{\mathrm{cut}}) had the most impact on model accuracy and computational efficiency. Both hyperparameters are common to all GNNs investigated in this study. All explored and optimal hyperparameters for all models can be viewed in Tables S2 through S9.

With the optimal hyperparameters, we report the accuracy of each GNN on training and testing sets in the top two panels of Figure 2. Since the energy value inherently increases with increasing protein size, we report the mean arctangent absolute percent error (MAAPE)[86] as a relative error metric (detailed in the Supporting Information). The mean absolute error (MAE) and the r2r^2 value for these models are displayed in Tables S10 and S11. Additionally, we used DISPEF-c to conduct a detailed analysis of the average inference times and maximum VRAM allocations of each model with respect to protein size. Discussion of this analysis can be found in the Supporting Information, along with Figures S3 through S6. For simplicity, however, we show the average inference time and maximum VRAM allocation for a representative structure from DISPEF-M in the bottom panel of Figure 2.

Figure 2. Benchmarking GNN performance on large proteins highlights the advantages of SAKE architectural innovation and significance of increasing rcutr_{\mathrm{cut}} for transferability. In the top two figures, we show the energy mean arctangent absolute percent error[86] (MAAPE) for all models on DISPEF-S and DISPEF-M. The most accurate models are outlined in black, while the testing set errors for GN (which had achieved the lowest testing set errors before we performed further hyperparameter optimization for SAKE) are outlined in gray. MAAPEs for all discussed models are displayed on the figure. Models with rcut=0.5r_{\mathrm{cut}} = 0.5 nm are depicted with solid bars, while models with rcut=0.5r_{\mathrm{cut}} = \sqrt{0.5} and 1 nm are depicted with increasingly more dense diagonal lines through the bars. In the bottom figure, inference times and peak memory allocation as measured on the representative structure of DISPEF-M (UniProt ID: A1W0R3) are displayed. Peak memory allocation is proportional to the width of each circle.

Our initial results suggested that the GN architecture performed best on both the DISPEF-S and DISPEF-M testing sets, while also being relatively computationally efficient (inference time: 141.73 ms, VRAM cost: 0.94 GiB). Notably, the ViSNet architecture, which has demonstrated state-of-the-art energy and force prediction for small molecules and CLN025,[57] showed the weakest performance among the GNNs evaluated in this study. Although further hyperparameter optimization improved accuracy, ViSNet continued to perform suboptimally (Table S3). Enhancing ViSNet’s accuracy would likely require increasing the spherical harmonics degree, which is impractical due to the substantial VRAM requirements.

Our first hyperparameter combination for SAKE (Nlayers=8N_{\mathrm{layers}} = 8, rcut=0.5r_{\mathrm{cut}} = 0.5 nm), referred to as SAKE 8S, achieved the lowest error on the training sets. However, it exhibited a notable decline in accuracy on the test sets, with MAAPE decreases of 2.13% and 3.22% for DISPEF-S and DISPEF-M, respectively. Other architectures (ANI, ET, and ViSNet) also showed large accuracy declines. This performance drop, particularly on the larger proteins in DISPEF-M, suggests that these models may have overfit the training data. These neural networks appear to approximate long-range interaction energies using local atomic environments within the cutoff distance, rather than accurately assigning them to atoms that are spatially distant.

This approximation of long-range interactions results in decreased accuracy on the testing set.

The computational efficiency of SAKE together with its training set accuracy made it a promising candidate for further hyperparameter optimization. As such, further combinations of NlayersN_{\mathrm{layers}} and rcutr_{\mathrm{cut}} were explored. An increase in rcutr_{\mathrm{cut}} to 0.5≈0.707\sqrt{0.5} \approx0.707 nm produced a SAKE model (referred to as SAKE 8M) that surpassed GN in accuracy on both the DISPEF-S and DISPEF-M test sets. However, this came at the cost of higher computational demands, with an inference time of 176.38 ms and a VRAM usage of 0.83 GiB. Reducing NlayersN_{\mathrm{layers}} from 8 to 4, while maintaining rcut=0.5r_{\mathrm{cut}} = \sqrt{0.5} nm, led to a more computationally efficient model (SAKE 4M), reducing inference time to 90.01 ms and preserving the same VRAM usage, though with a trade-off in accuracy. Further increasing rcutr_{\mathrm{cut}} to 1 nm yielded a model (SAKE 4L) with the best performance on the test sets. Nonetheless, the increased computational cost of SAKE 4L, with an inference time of 219.92 ms and a VRAM requirement of 2.07 GiB, emphasizes that while incorporating longer-range information can enhance accuracy, simply increasing rcutr_{\mathrm{cut}} is not a computationally scalable solution.

A multiscale GNN architecture enables transferable and efficient predictions

Our benchmark results suggest that efficient and transferable GNNs need to operate at large cutoff distances with low computational cost. We propose a multiscale architectural design that meets these two criteria. Specifically, we make use of two different message-passing layers that operate on different graphs / subgraphs. A highly accurate SAKE layer updates each atomic featurization using information from all short-ranged neighbors determined by cutoff distance rcutSAKEr_{\mathrm{cut}}^{\mathrm{SAKE}}, followed by a less accurate but more efficient SchNet layer to compute a second update to each atomic featurization using information from long-ranged neighboring alpha carbon atoms that fall in the range of rcutSchNetr_{\mathrm{cut}}^{\mathrm{SchNet}}. We call this mixed model Schake (pronounced as “shock” followed by the letter “A”, or “shock-A”), since it uses both SchNet and SAKE message-passing layers. Recently, a similar design was used in Chroma[87], a denoising probabilistic diffusion model for generating atomistic protein conformations.

The multiscale design significantly expands the attention range of each atom, allowing them to access chemical information at a much longer range. Since short-ranged neighbors are still used to produce their feature updates, each alpha carbon atom in the sparse subgraph would still contain information from its surrounding chemical environment (Figure 3B). Collecting information from alpha carbons allow the atom under consideration to experience the atomic environment at a distance set by rcutSchNetr_{\mathrm{cut}}^{\mathrm{SchNet}}, rather than the much smaller value rcutSAKEr_{\mathrm{cut}}^{\mathrm{SAKE}}.

Schake architecture and multiscale resolution

Figure 3. The Schake architecture enables accurate and efficient incorporation of longer-ranged interactions. (A) Illustration of the Schake architecture. Architectural details are explained in the Methods section and the Supporting Information. (B) Illustration of the multiscale resolution of Schake. Dashed blue space represents the short-ranged environments processed by the SAKE layer, while the white space represents the long-ranged environment processed by the SchNet layer. The solid black circle represents atom ii, the solid multi-colored circles represent the many atom types of short-ranged neighbors jj processed by the SAKE layer, and the solid gray circles represent the alpha carbon atom types of long-ranged neighbors kk processed by the SchNet layer. (C) Energy MAAPE results for DISPEF-S and DISPEF-M along with inference and peak memory allocation measurements for the representative structure of DISPEF-M (UniProt ID: A1W0R3). These figures are formatted identically to Figure 2.

In addition, since a majority of the atom pairs will be processed by the more computationally efficient architecture, SchNet, Schake maintains efficiency. As shown by Figure S1A, for the Schake model with rcutSAKE=0.5r_{\mathrm{cut}}^{\mathrm{SAKE}} = 0.5 nm and rcutSchNet=2.5r_{\mathrm{cut}}^{\mathrm{SchNet}} = 2.5 nm, the SAKE message-passing layer processes ∼20\sim20 - 25%25\% of the total atom pairs, while the SchNet layers processes the remaining ∼75\sim75 - 80%80\% of atom pairs for structures larger than ∼3,000\sim3{,}000 atoms. Furthermore, the total number of processed atom pairs in Schake remains smaller than any model with rcut=1r_{\mathrm{cut}} = 1 nm (Figure S1B), despite a cutoff distance of 2.5 nm.

We evaluated the performance of two Schake models, with rcutSAKE=0.5r_{\mathrm{cut}}^{\mathrm{SAKE}} = 0.5 or 0.5\sqrt{0.5} nm, and rcutSchNet=2.5r_{\mathrm{cut}}^{\mathrm{SchNet}} = 2.5 nm (denoted as Schake S and Schake M, respectively). As shown in Figure 3C, both Schake models are more accurate than all other models across the two testing sets. This accuracy improvement is coupled with a significant decrease in inference times compared to the previously best performing models. Specifically, the Schake models are 2.7×2.7\times (82.25 ms) and 1.7×1.7\times (132.50 ms) faster than the previous most accurate model, SAKE 4L (219.92 ms). While the peak memory allocation of Schake is still relatively high, we discuss causes for this in the Supporting Information. Overall, these results suggest that the Schake architecture is both highly accurate and highly computationally efficient.

Schake enables transferable and scalable energy and force predictions

For ML models to be useful, they should ideally be transferable to both similar-sized and larger proteins outside of the training set. To this end, we next assessed the transferability of Schake and other models to the larger proteins in DISPEF-L. As detailed earlier, DISPEF-L contains over 109,000 proteins ranging in size from 6,800 to 12,499 atoms. Using the two models trained on either DISPEF-S or DISPEF-M for each architecture, we computed energy predictions for all proteins within DISPEF-L. Prediction errors can be viewed in Figure 4.

Figure 4. Training on DISPEF-M results in models that are more transferable to the larger proteins of DISPEF-L. Models with rcut=0.5r_{\mathrm{cut}} = 0.5 nm are depicted with solid bars, while models with rcut=0.5r_{\mathrm{cut}} = \sqrt{0.5} and 1 nm are depicted with increasingly more dense diagonal lines through the bars. For all models, we show the mean arctangent absolute percent error[86] (MAAPE). The best-performing models are outlined in black. While the Schake model trained on DISPEF-S performs poorly on DISPEF-L, the Schake model trained on DISPEF-M achieves the lowest errors of all tested models on DISPEF-L.

These results suggest that across all architectures, greater accuracy for the larger proteins of DISPEF-L is achieved when the training set contains larger structures (i.e., when trained on DISPEF-M). This is especially pronounced with the Schake models, where the larger cutoff distance of rcutSchNet=2.5r_{\mathrm{cut}}^{\mathrm{SchNet}} = 2.5 nm is not fully utilized when training on DISPEF-S. Across the models trained on DISPEF-S, SAKE 4L achieves the best accuracy on DISPEF-L. However, across the models trained on DISPEF-M, both Schake models achieve the best accuracy on DISPEF-L.

When using a GNN as an MD force field, force prediction accuracy is especially important to ensure accurate dynamics. To assess force prediction accuracy, we trained a selection of the most computationally efficient of the architectures we explored on both the energies and forces of structures in DISPEF-M. Due to memory limitations, we could not perform this training on the closest performing model to Schake, SAKE 4L. Energy and force training for each model was conducted using the same hyperparameters used for the energy-only training reported above. Prediction errors from this training are shown in Figure 5. Note that energy errors contribute little to the loss function used during force and energy training (Figure S2). As such, it is not expected for these energy errors to match the results shown in Figure 2 through 4. Despite this, Schake M achieves the lowest energy errors of the tested models. Schake S achieves similar energy errors to SAKE 8S. However, both Schake models outperform all of the other GNNs explored in this study in force prediction accuracy. Schake S is more accurate than the next most accurate model, SAKE 8S, while also being ∼1.4×\sim1.4 \times faster. Additionally, Schake M is substantially more accurate than SAKE 8S while only being ∼1.1×\sim1.1 \times slower. These results again demonstrate the success of our Schake architecture.

Figure 5. The Schake models outperform SchNet, GN, and SAKE at force prediction. (Left) Energy prediction mean arctangent absolute percent error[86] (MAAPE). (Right) Force mean absolute error (MAE). All DISPEF-L energy and force errors are shown in blue text. Models with rcut=0.5r_{\mathrm{cut}} = 0.5 nm are depicted with solid bars, while models with rcut=0.5r_{\mathrm{cut}} = \sqrt{0.5} and 1 nm are depicted with increasingly more dense diagonal lines through the bars.

Conclusions

We have presented DISPEF and its four subsets, named DISPEF-S, DISPEF-M, DISPEF-L, and DISPEF-c, that enable thorough evaluation of the performance of GNN architectures for proteins. Energy inference results for the DISPEF subsets suggest that GNN architectures that are known to perform exceptionally well for small molecules are not necessarily guaranteed to also perform well for large proteins (Tables 2 and 4). As such, DISPEF will help to guide future GNN architecture design toward more accurate predictions for large structures like proteins. Our results in Figure 4 show that longer-range interactions must be accounted for (in a computationally efficient manner) in order to achieve the most accurate predictions for proteins. To this end, we’ve also introduced a multiscale architecture termed “Schake” (Figure 3) that uses a more accurate but more expensive SAKE message-passing layer to pro cess short-ranged neighbors, and a less accurate but more efficient SchNet message-passing layer to process longer-ranged alpha carbon neighbors. This mixed architecture design is not limited to just SAKE and SchNet; in the future, the SAKE layer could be replaced with an even-more accurate message-passing layer. Likewise, the SchNet layer could be replaced with a more efficient scheme to incorporate longer-ranged information. Our training and inference results show that Schake achieves state-of-the-art energy prediction on all of our DISPEF subsets (Figure 3C), and also achieves the smallest force errors when conducting energy and force training on DISPEF-M (Figure 5). Overall, our results provide meaningful tools and insight for the future development of GNNs as broadly-applicable and scalable protein force fields and implicit solvent models.

Methods

Structures, energies, and forces for DISPEF

PDB structures for proteins in the Swiss-Prot database [78] were obtained from the AlphaFold2 protein structure database. [76, 77] We next used OpenMM[88] (ver. 7.7.0) to add hydrogen atoms to all structures. Energy minimization was then performed using OpenMM on the hydrogenated structures with the CHARMM36[89] force field and the GBn2 implicit solvent model. [80] For each minimized structure, we saved the positions (x\mathbf{x}), the solvation free energy computed by the GBn2 implicit solvent (EGBn2E_{\mathrm{GBn2}}), and the corresponding forces (fGBn2=−∇xEGBn2\mathbf{f}_{\mathrm{GBn2}} = -\nabla_{\mathbf{x}} E_{\mathrm{GBn2}}). Units of nm were used for x\mathbf{x}, units of kJ/mol were used for EGBn2E_{\mathrm{GBn2}}, and units of kJ/mol/nm were used for fGBn2\mathbf{f}_{\mathrm{GBn2}}. To ensure that the saved forces were exactly reproducible, the deterministic forces option for OpenMM’s CUDA platform was set to true. We also saved the PDB atom names (for example, the label “CA” for alpha carbons) and the nuclear charges for all atoms in each structure. To enable scaling of the energy loss function by protein size, we also saved the total number of atoms in each structure (natomsn_{\mathrm{atoms}}). This data comprises DISPEF and its subsets. For all of these subsets, x\mathbf{x}, EGBn2E_{\mathrm{GBn2}}, fGBn2\mathbf{f}_{\mathrm{GBn2}}, nuclear charges, PDB atom names, natomsn_{\mathrm{atoms}}, and UniProt entry names were saved. For DISPEF-S, DISPEF-M, and DISPEF-L, EGBn2≥−21,000E_{\mathrm{GBn2}} \geq-21{,}000, −21,000-21{,}000, and −25,000-25{,}000 kcal/mol (respectively) for all structures. All subsets (saved as PyTorch dataset objects) and the code required to load them will be made publicly available (see Data Availability section).

Implementation of GNN Architectures

All models were implemented using PyTorch90 (ver. 2.0) and PyTorch-Geometric91 (ver. 2.3.1). We implemented ET,55 GN,2 and ViSNet57 using the code provided by their respective publications / python packages. The torchANI84 implementation of ANI64,65,70 was used, along with the CUDA-accelerated atomic environment vector computer. The ANI model used here consists of an ensemble of five 4-layer feed-forward NNs, corresponding to the five elements present in our dataset (H, C, N, O, and S).

Our implementation of the SchNet architecture was based on that from the PyTorch-Geometric package. Minor changes inspired by the GN architecture were made to improve accuracy. The expnorm kernel function and the CELU (with α=2\alpha= 2) activation function were used, as opposed to the original Gaussian kernel function and the shifted softplus activation function. Our implementation of EGNN was based on the original PyTorch implementation. We modified this code to implement a cutoff distance (rcutr_{\mathrm{cut}}). Since a PyTorch implementation of SAKE was not previously available, we created our own. We built the SAKE operations onto the EGNN PyTorch code. We further detail our implementation of the SAKE architecture in the Supporting Information. For both EGNN and SAKE, no velocity information was provided, and coordinate transformations were not performed. As such, both implementations were E(3)E(3)-invariant.

Architecture of the multiscale Schake model

Within the llth Schake message-passing layer (Figure 3A), the previous embedding from the (l−1)(l - 1)th message-passing layer for the iith atom, hi(l−1)h_i^{(l-1)}, and hj(l−1)h_j^{(l-1)} for all jj neighboring atoms within distance rji∈[0,rcutSAKE]r_{ji} \in[0,r_{\mathrm{cut}}^{\mathrm{SAKE}}] nm from atom ii, are fed into a SAKE layer to compute a message mi(l)\mathbf{m}_i^{(l)}. The intermediate updated embedding for atom ii, Si(l)=hi(l−1)+mi(l)\mathcal{S}_i^{(l)}=\mathbf{h}_i^{(l-1)}+\mathbf{m}_i^{(l)}, is then computed. Next, Si(l)\mathcal{S}_i^{(l)} and Sk(l)\mathcal{S}_k^{(l)} for all kk neighboring alpha carbon atoms within distance rki∈[rcutSAKE,rcutSchNet]r_{ki}\in[r_{\mathrm{cut}}^{\mathrm{SAKE}},r_{\mathrm{cut}}^{\mathrm{SchNet}}] nm from atom ii are fed into a SchNet layer to compute a message vi(l)\mathbf{v}_i^{(l)}. The embedding is updated again to yield the embedding hi(l)=Si(l)+vi(l)\mathbf{h}_i^{(l)}=\mathcal{S}_i^{(l)}+\mathbf{v}_i^{(l)} for atom ii. This embedding hi(l)\mathbf{h}_i^{(l)} is then either passed into another Schake layer, or used to compute the energy contribution of the iith atom. A detailed description of the Schake architecture is provided in the Supporting Information.

Training on the DISPEF subsets

PyTorch Distributed Data Parallel (DDP) was used to train all models across 8 Nvidia V100 GPUs. All GPUs were provided and managed by the Massachusetts Institute of Technology (MIT) Supercloud and Lincoln Laboratory Supercomputing Center. [92] With the exception of ANI, all models were first trained for 140 epochs on DISPEF-S with an initial learning rate of 1×10−31 \times10^{-3} that decayed every 3 epochs by a factor of 0.875. Next, the parameters obtained from training on DISPEF-S were used to initialize training on DISPEF-M. All models (except for ANI) were trained for 140 epochs on DISPEF-M with an initial learning rate of 1×10−31 \times10^{-3} that decayed every 3 epochs by a factor of 0.875. The Adam optimizer and single-precision floating point format (FP32) were used for all training. The MAE per atom loss function, ℓMAEE/a=1N∑iN1natoms,i∣Epredi−Etargeti∣\ell_{\mathrm{MAE}}^{E/a}=\frac{1}{N}\sum_i^N\frac{1}{n_{\mathrm{atoms},i}}\left|E_{\mathrm{pred}}^i-E_{\mathrm{target}}^i\right|, was used to train all models. Additionally, L2 regularization, ℓreg=λreg∑iθi2\ell_{\mathrm{reg}}=\lambda_{\mathrm{reg}}\sum_i\theta_i^2, acted on the model parameters θ\boldsymbol{\theta}, with λreg=1×10−6\lambda_{\mathrm{reg}}=1 \times10^{-6}. Regularization was implemented through the weight decay option when initializing the Adam optimizer. In total, the loss function was as follows: ℓtot=ℓMAEE/a+ℓreg\ell_{\mathrm{tot}}=\ell_{\mathrm{MAE}}^{E/a}+\ell_{\mathrm{reg}}.

A custom, variable-size batch sampler was used when training all models except ANI, with the atom limit and maximum batch size set to maximize GPU memory utilization.

For the ANI model, training for 860 epochs was first conducted on DISPEF-S with an initial learning rate of 1×10−31 \times10^{-3} that decayed every 3 epochs by a factor of 0.875. After epochs 210, 420, and 640, the learning rate and decay factor were set to 1×10−31 \times10^{-3} and 0.875, 1×10−41 \times10^{-4} and 0.9065, and 5×10−45 \times10^{-4} and 0.886, respectively. Next, the parameters obtained from training on DISPEF-S were then used to initialize training on DISPEF-M. Training on DISPEF-M was conducted for 630 epochs. After epochs 210 and 420, the learning rate and decay rates were set to 1×10−41 \times10^{-4} and 0.9065, and 5×10−55 \times10^{-5} and 0.915, respectively. Following Smith et al. 69, both the AdamW[93] and stochastic gradient descent[94] (SGD) optimizers were used to train ANI. The AdamW optimizer was applied to all of the weights of ANI, while the SGD optimizer was applied to all of the bias parameters. Regularization was only applied to the weights of the 2{sup}nd and 3{sup}rd layers of each feed-forward NN, where λreg=1×10−5\lambda_{\mathrm{reg}} = 1 \times10^{-5} for the 2{sup}nd layer weights and $\lambda_{\mathrm{reg}} = 1 \times 10^{-6}$ for the 3{sup}rd layer weights.

More computationally-demanding force and energy training on DISPEF-M was also conducted for the SchNet, GN, SAKE 8S, SAKE 4M, Schake S, and Schake M architectures. Training was initialized using the optimal model parameters obtained from training these architectures to fit the energies of DISPEF-M. Training was conducted for 90 epochs with an initial learning rate of 1×10−31 \times10^{-3} that decayed every 3 epochs by a factor of 0.875. Optimizers were applied following the same schemes used when training to fit only energies. The MAE per atom loss function was applied to the energies, while the MAE loss function, ℓMAEf=1N13natoms∑iN∑j3natoms∣fpredi,j−ftargeti,j∣\ell_{\mathrm{MAE}}^{f} = \frac{1}{N}\frac{1}{3n_{\mathrm{atoms}}}\sum_{i}^{N}\sum_{j}^{3n_{\mathrm{atoms}}} \left|f_{\mathrm{pred}}^{i,j} - f_{\mathrm{target}}^{i,j}\right|, was applied to the forces. In total, the loss function was as follows: ℓtot=λEℓMAEE/a+λfℓMAEf\ell_{\mathrm{tot}} = \lambda_{\mathrm{E}}\ell_{\mathrm{MAE}}^{\mathrm{E/a}} + \lambda_{f}\ell_{\mathrm{MAE}}^{f}, where λE=λf=0.5\lambda_{\mathrm{E}} = \lambda_{f} = 0.5. Three separate trainings with different random seeds were conducted, and the results for the best performing model are shown in Figure 5.

For ANI and for all of the GNNs explored in this study, the optimal hyperparameters that we selected are displayed in Tables S2 through S9. Additionally, a thorough analysis of computational cost (i.e., inference times and memory allocation) is presented in the Supporting Information, along with Figures S3 through S6 (smoothed using the LOWESS[95] algorithm for ease of interpretation) that show both inference times and memory allocations as a function of protein size.

Implementation of a variable-size batch sampler

While we initially trained our models on the DISPEF subsets using a standard, fixed-size PyTorch batch sampler, we found this to be computationally inefficient. Larger proteins contain more atoms / graph nodes, and thus more VRAM is required for training. While small molecules in datasets like MD17[63] (as one example) do not differ in size that greatly from one another, DISPEF-M contains proteins ranging from 16 to 390 amino acids. This vast difference in protein size renders a fixed-size batch sampler highly computationally inefficient. Even after shuffling the data, it is possible that any given batch will contain only smaller or larger proteins. The memory requirements for a batch containing only larger proteins will be significantly greater than that for a batch containing only smaller proteins. As such, when using a fixed-size batch sampler, the batch size is limited by the size of the largest structures in the dataset. This leaves GPU memory left unused when processing batches containing smaller structures.

To eliminate this inefficiency, we created a variable-size batch sampler. When constructing a batch, our variable-size batch sampler will compute a running summation of the total number of atoms in all proteins added to that batch. If adding the next protein to the current batch would result in the total atom number count being greater than a set limit to the number of atoms per batch, the next protein would be added to a new batch. We found that this variable-size batch sampler results in a speed-up in training time of up to 25%. It is worth noting that this batch sampler introduces some complications when training a model across multiple GPUs using DDP. We discuss these complications in-depth in the Supporting Information and show in Figure S7 that they don’t adversely impact model accuracy.

Supporting Information

Description of the MAAPE relative error metric, our PyTorch implementation of SAKE, the Schake architecture, initial hyperparameter optimization, discussion on computational cost and our variable-sized batch sampler, along with additional supporting figures and tables.

Data Availability

Code for our PyTorch implementation of SAKE, our Schake architecture, and our variable-sized batch sampler will be made available on GitHub at the following link: https://github.com/ZhangGroup-MITChemistry/Schake_GNN/. We also include examples of how to use our implementation of SAKE, Schake, and our variable-sized batch sampler. Additionally, DISPEF-c and an example of how to load it will be made available on the GitHub repository linked above. DISPEF-S, DISPEF-M, and DISPEF-L will be deposited to a suitable data hosting website.

Acknowledgement

This work was supported by the National Institutes of Health (Grant R35GM133580). J.A. was the recipient of the MIT Dean of Science Fellowship and the National Science Foundation Graduate Research Fellowship. Additionally, we thank undergraduate students Jack H. Liu from the MIT Department of Electrical Engineering and Computer Science and Chenxi Ye from the Peking University College of Chemistry and Molecular Engineering for their assistance implementing the ANI architecture using torchANI. We also thank graduate student Ivan Riveros from the MIT Department of Chemistry for his insight on GPU memory bandwidth.

Competing interests: Authors declare that they have no competing interests.

TOC Graphic

Figure 1 (PDF p. 21). TOC Graphic

References

  1. [1]Airas, J.; Ding, X.; Zhang, B. Transferable Implicit Solvation via Contrastive Learning of Graph Neural Networks. ACS Cent. Sci. 2023, 9, 2286–2297.DOI
  2. [2]Majewski, M.; Pérez, A.; Thölke, P.; Doerr, S.; Charron, N. E.; Giorgino, T.; Husic, B. E.; Clementi, C.; Noé, F.; De Fabritiis, G. Machine learning coarse-grained potentials of protein thermodynamics. Nat. Commun. 2023, 14, 5739.DOI
  3. [3]Katzberger, P.; Riniker, S. Implicit solvent approach based on generalized Born and transferable graph neural networks for molecular dynamics simulations. J. Chem. Phys. 2023, 158, 204101.DOI
  4. [4]Katzberger, P.; Riniker, S. A general graph neural network based implicit solvation model for organic molecules in water. Chem. Sci. 2024,DOI
  5. [5]Chen, Y.; Krämer, A.; Charron, N. E.; Husic, B. E.; Clementi, C.; Noé, F. Machine learning implicit solvation for molecular dynamics. J. Chem. Phys. 2021, 155, 084101.DOI
  6. [6]Husic, B. E.; Charron, N. E.; Lemm, D.; Wang, J.; Pérez, A.; Majewski, M.; Krämer, A.; Chen, Y.; Olsson, S.; de Fabritiis, G. et al. Coarse graining molecular dynamics with graph neural networks. J. Chem. Phys. 2020, 153, 194101.DOI
  7. [7]Kozinsky, B.; Musaelian, A.; Johansson, A.; Batzner, S. Scaling the Leading Accuracy of Deep Equivariant Models to Biomolecular Simulations of Realistic Size. Proceedings of the International Conference for High Performance Computing, Networking, Storage and Analysis. 2023; pp 1–12.arxiv.org/abs/2304.10061
  8. [8]Wang, Z.; Wu, H.; Sun, L.; He, X.; Liu, Z.; Shao, B.; Wang, T.; Liu, T.-Y. Improving machine learning force fields for molecular dynamics simulations with fine-grained force metrics. J. Chem. Phys. 2023, 159, 035101.DOI
  9. [9]Brunken, C.; Boyer, S.; Omar, M.; Diallo, B. N.; Beguir, K.; Carranza, N. L.; Bent, O. Machine learning of force fields towards molecular dynamics simulations of proteins at DFT accuracy. ICLR 2024 Workshop on Generative and Experimental Perspectives for Biomolecular Design. Vienna, Austria. 2024.
  10. [10]Yao, S.; Van, R.; Pan, X.; Park, J. H.; Mao, Y.; Pu, J.; Mei, Y.; Shao, Y. Machine learning based implicit solvent model for aqueous-solution alanine dipeptide molecular dynamics simulations. RSC Adv. 2023, 13, 4565–4577.DOI
  11. [11]Wang, J.; Olsson, S.; Wehmeyer, C.; Pérez, A.; Charron, N. E.; de Fabritiis, G.; Noé, F.; Clementi, C. Machine Learning of Coarse-Grained Molecular Dynamics Force Fields. ACS Cent. Sci. 2019, 5, 755–767.arxiv.org/abs/1812.01736
  12. [12]Wang, J.; Charron, N.; Husic, B.; Olsson, S.; Noé, F.; Clementi, C. Multi-body effects in a coarse-grained protein force field. J. Chem. Phys. 2021, 154, 164113.DOI
  13. [13]Fu, X.; Wu, Z.; Wang, W.; Xie, T.; Keten, S.; Gomez-Bombarelli, R.; Jaakkola, T. S. Forces are not Enough: Benchmark and Critical Evaluation for Machine Learning Force Fields with Molecular Simulations. arXiv 2023, arXiv:2210.07237, accessed on 2024-04-09.arxiv.org/abs/2210.07237
  14. [14]Durumeric, A. E. P.; Chen, Y.; Noé, F.; Clementi, C. Learning data efficient coarse-grained molecular dynamics from forces and noise. arXiv 2024, arXiv:2407.01286, accessed on 2024-09-06.arxiv.org/abs/2407.01286
  15. [15]Arts, M.; Garcia Satorras, V.; Huang, C.-W.; Zügner, D.; Federici, M.; Clementi, C.; Noé, F.; Pinsler, R.; van den Berg, R. Two for One: Diffusion Models and Force Fields for Coarse-Grained Molecular Dynamics. J. Chem. Theory Comput. 2023, 19, 6151–6159.arxiv.org/abs/2302.00600
  16. [16]Köhler, J.; Chen, Y.; Krämer, A.; Clementi, C.; Noé, F. Flow-Matching: Efficient Coarse-Graining of Molecular Dynamics without Forces. J. Chem. Theory Comput. 2023, 19, 942–952.DOI
  17. [17]Loose, T. D.; Sahrmann, P. G.; Qu, T. S.; Voth, G. A. Coarse-Graining with Equivariant Neural Networks: A Path Toward Accurate and Data-Efficient Models. J. Phys. Chem. B 2023, 127, 10564–10572.DOI
  18. [18]Durumeric, A. E. P.; Voth, G. A. Adversarial-residual-coarse-graining: Applying machine learning theory to systematic molecular coarse-graining. J. Chem. Phys. 2019, 151, 124110.DOI
  19. [19]Durumeric, A. E. P.; Charron, N. E.; Templeton, C.; Musil, F.; Bonneau, K.; Pasos-Trejo, A. S.; Chen, Y.; Kelkar, A.; Noé, F.; Clementi, C. Machine learned coarse-grained protein force-fields: Are we there yet? Curr. Opin. Struct. Biol. 2023, 79, 102533.DOI
  20. [20]Ding, X.; Zhang, B. Contrastive Learning of Coarse-Grained Force Fields. J. Chem. Theory Comput. 2022, 18, 6334–6344.DOI
  21. [21]Ding, X. Optimizing Force Fields with Experimental Data Using Ensemble Reweighting and Potential Contrasting. ChemRxiv 2024, chemrxiv–2024–vlwb, accessed on 2024-04-09.DOI
  22. [22]Chmiela, S.; Sauceda, H. E.; Müller, K.-R.; Tkatchenko, A. Towards exact molecular dynamics simulations with machine-learned force fields. Nat. Commun. 2018, 9, 3887.DOI
  23. [23]Chmiela, S.; Sauceda, H. E.; Tkatchenko, A.; Müller, K.-R. In Machine Learning Meets Quantum Physics; Schütt, K. T., Chmiela, S., von Lilienfeld, O. A., Tkatchenko, A., Tsuda, K., Müller, K.-R., Eds.; Springer International Publishing, 2020; pp 129–154.
  24. [24]Duschatko, B. R.; Fu, X.; Owen, C.; Xie, Y.; Musae­ lian, A.; Jaakkola, T.; Kozinsky, B. Thermodynamically Informed Multimodal Learning of High-Dimensional Free Energy Models in Molecular Coarse Graining. arXiv 2024, arXiv:2405.19386, accessed on 2024-06-24.arxiv.org/abs/2405.19386
  25. [25]G. Greener, J. Differentiable simulation to develop molecular dynamics force fields for disordered proteins. Chem. Sci. 2024, 15, 4897–4909.DOI
  26. [26]Yang, W.; Templeton, C.; Rosenberger, D.; Bittracher, A.; Nüske, F.; Noé, F.; Clementi, C. Slicing and Dicing: Optimal Coarse-Grained Representation to Preserve Molecular Kinetics. ACS Cent. Sci. 2023, 9, 186–196.DOI
  27. [27]Corso, G.; Stark, H.; Jegelka, S.; Jaakkola, T.; Barzilay, R. Graph neural networks. Nat. Rev. Methods Primers 2024, 4, 1–13.
  28. [28]Batzner, S.; Musaelian, A.; Kozinsky, B. Advancing molecular simulation with equivariant interatomic potentials. Nat. Rev. Phys. 2023, 5, 437–438.DOI
  29. [29]Eastman, P.; Galvelis, R.; Peláez, R. P.; Abreu, C. R. A.; Farr, S. E.; Gallicchio, E.; Gorenko, A.; Henry, M. M.; Hu, F.; Huang, J. et al. OpenMM 8: Molecular Dynamics Simulation with Machine Learning Potentials. J. Phys. Chem. B 2024, 128, 109–116.arxiv.org/abs/2310.03121
  30. [30]Zheng, S.; He, J.; Liu, C.; Shi, Y.; Lu, Z.; Feng, W.; Ju, F.; Wang, J.; Zhu, J.; Min, Y. et al. Predicting equilibrium distributions for molecular systems with deep learning. Nat. Mach. Intell. 2024, 6, 558–567.arxiv.org/abs/2306.05445
  31. [31]Sahrmann, P. G.; Loose, T. D.; Durumeric, A. E. P.; Voth, G. A. Utilizing Machine Learning to Greatly Expand the Range and Accuracy of Bottom-Up Coarse-Grained Models through Virtual Particles. J. Chem. Theory Comput. 2023, 19, 4402–4413.arxiv.org/abs/2212.04530
  32. [32]Bonneau, K.; Lederer, J.; Templeton, C.; Rosenberger, D.; Müller, K.-R.; Clementi, C. Peering inside the black box: Learning the relevance of many-body functions in Neural Network potentials. arXiv 2024, arXiv:2407.04526, accessed on 2024-09-06.arxiv.org/abs/2407.04526
  33. [33]Anstine, D. M.; Isayev, O. Machine Learning Interatomic Potentials and Long-Range Physics. J. Phys. Chem. A 2023, 127, 2417–2431.DOI
  34. [34]Cheng, Z.; Bi, H.; Liu, S.; Chen, J.; Yu, K. Developing Differentiable Long-Range Force Field for Proteins with E(3) Neural Network Predicted Asymptotic Parameters. ChemRxiv 2024, chemrxiv–2024–kkb5h, accessed on 2024-04-09.DOI
  35. [35]Faller, C.; Kaltak, M.; Kresse, G. Density-Based Long-Range Electrostatic Descriptors for Machine Learning Force Fields. arXiv 2024, arXiv:2406.17595, accessed on 2024-08-30.arxiv.org/abs/2406.17595
  36. [36]Takaba, K.; J. Friedman, A.; E. Cavender, C.; Kumar Behara, P.; Pulido, I.; M. Henry, M.; MacDermott-Opeskin, H.; R. Iacovella, C.; M. Nagle, A.; Matthew Payne, A. et al. Machine-learned molecular mechanics force fields from large-scale quantum chemical data. Chem. Sci. 2024, 15, 12861–12878.DOI
  37. [37]Wang, Y.; Pulido, I.; Takaba, K.; Kaminow, B.; Scheen, J.; Wang, L.; Chodera, J. D. EspalomaCharge: Machine Learning-Enabled Ultrafast Partial Charge Assignment. J. Phys. Chem. A 2024, 128, 4160–4167.DOI
  38. [38]Wang, Y.; Fass, J.; Kaminow, B.; E. Herr, J.; Rufa, D.; Zhang, I.; Pulido, I.; Henry, M.; Macdonald, H. E. B.; Takaba, K. et al. End-to-end differentiable construction of molecular mechanics force fields. Chem. Sci. 2022, 13, 12016–12033.DOI
  39. [39]Galvelis, R.; Varela-Rial, A.; Doerr, S.; Fino, R.; Eastman, P.; Markland, T. E.; Chodera, J. D.; De Fabritiis, G. NNP/MM: Accelerating Molecular Dynamics Simulations with Machine Learning Potentials and Molecular Mechanics. J. Chem. Inf. Model. 2023, 63, 5701–5708.DOI
  40. [40]Sabanés Zariquiey, F.; Galvelis, R.; Gallicchio, E.; Chodera, J. D.; Markland, T. E.; De Fabritiis, G. Enhancing Protein–Ligand Binding Affinity Predictions Using Neural Network Potentials. J. Chem. Inf. Model. 2024, 64, 1481–1485.arxiv.org/abs/2401.16062
  41. [41]Barnett, S.; Chodera, J. D. Neural Network Potentials for Enabling Advanced Small-Molecule Drug Discovery and Generative Design. GEN Biotechnology 2024, 3, 119–129.DOI
  42. [42]Unke, O. T.; Meuwly, M. PhysNet: A Neural Network for Predicting Energies, Forces, Dipole Moments, and Partial Charges. J. Chem. Theory Comput. 2019, 15, 3678–3693.DOI
  43. [43]Gasteiger, J.; Groß, J.; Güneşmann, S. Directional Message Passing for Molecular Graphs. arXiv 2022, arXiv:2003.03123, accessed on 2023-07-13.arxiv.org/abs/2003.03123
  44. [44]Gasteiger, J.; Becker, F.; Güneşmann, S. GemNet: Universal Directional Graph Neural Networks for Molecules. arXiv 2022, arXiv:2106.08903, accessed on 2023-07-13.arxiv.org/abs/2106.08903
  45. [45]Anderson, B.; Hy, T.-S.; Kondor, R. Cormorant: Covariant Molecular Neural Networks. arXiv 2019, arXiv:1906.04015, accessed on 2023-7-13.arxiv.org/abs/1906.04015
  46. [46]Brandstetter, J.; Hesselink, R.; van der Pol, E.; Bekkers, E. J.; Welling, M. Geometric and Physical Quantities Improve E(3) Equivariant Message Passing. arXiv 2022, arXiv:2110.02905, accessed on 2023-07-13.arxiv.org/abs/2110.02905
  47. [47]Li, Y.; Liu, L.; Wang, G.; Du, Y.; Chen, P. EGNN: Constructing explainable graph neural networks via knowledge distillation. 2022, 241, 108345.DOI
  48. [48]Schütt, K. T.; Unke, O. T.; Gastegger, M. Equivariant message passing for the prediction of tensorial properties and molecular spectra. arXiv 2021, arXiv:2102.03150, accessed on 2023-07-13.arxiv.org/abs/2102.03150
  49. [49]Thomas, N.; Smidt, T.; Kearnes, S.; Yang, L.; Li, L.; Kohlhoff, K.; Riley, P. Tensor field networks: Rotation- and translation-equivariant neural networks for 3D point clouds. arXiv 2018, arXiv:1802.08219, accessed on 2023-7-13.arxiv.org/abs/1802.08219
  50. [50]Fuchs, F. B.; Worrall, D. E.; Fischer, V.; Welling, M. SE(3)-Transformers: 3D Roto-Translation Equivariant Attention Networks. arXiv 2020, arXiv:2006.10503, accessed on 2023-07-13.arxiv.org/abs/2006.10503
  51. [51]Huang, W.; Han, J.; Rong, Y.; Xu, T.; Sun, F.; Huang, J. Equivariant Graph Mechanics Networks with Constraints. arXiv 2022, arXiv:2203.06442, accessed on 2023-07-19.arxiv.org/abs/2203.06442
  52. [52]Schütt, K. T.; Sauceda, H. E.; Kindermans, P.-J.; Tkatchenko, A.; Müller, K.-R. SchNet – A deep learning architecture for molecules and materials. J. Chem. Phys. 2018, 148, 241722.DOI
  53. [53]Satorras, V. G.; Hoogeboom, E.; Welling, M. E(n) Equivariant Graph Neural Networks. arXiv 2022, arXiv:2102.09844, accessed on 2023-07-19.arxiv.org/abs/2102.09844
  54. [54]Wang, Y.; Chodera, J. D. Spatial Attention Kinetic Networks with E(n)-Equivariance. arXiv 2023, arXiv:2301.08893, accessed on 2023-07-13.arxiv.org/abs/2301.08893
  55. [55]Thölke, P.; de Fabritiis, G. TorchMD-NET: Equivariant Transformers for Neural Network based Molecular Potentials. arXiv 2022, arXiv:2202.02541, accessed on 2023-07-13.arxiv.org/abs/2202.02541
  56. [56]Pelaez, R. P.; Simeon, G.; Galvelis, R.; Mirarchi, A.; Eastman, P.; Doerr, S.; Thölke, P.; Markland, T. E.; De Fabritiis, G. TorchMD-Net 2.0: Fast Neural Network Potentials for Molecular Simulations. J. Chem. Theory Comput. 2024, 20, 4076–4087.DOI
  57. [57]Wang, Y.; Wang, T.; Li, S.; He, X.; Li, M.; Wang, Z.; Zheng, N.; Shao, B.; Liu, T.-Y. Enhancing geometric representations for molecules with equivariant vector-scalar interactive message passing. Nat. Commun. 2024, 15, 313.DOI
  58. [58]Batzner, S.; Musaelian, A.; Sun, L.; Geiger, M.; Mailoa, J. P.; Kornbluth, M.; Molinari, N.; Smidt, T. E.; Kozinsky, B. E(3)-equivariant graph neural networks for data-efficient and accurate interatomic potentials. Nat. Commun. 2022, 13, 2453.DOI
  59. [59]Batatia, I.; Batzner, S.; Kovács, D. P.; Musaelian, A.; Simm, G. N. C.; Drautz, R.; Ortner, C.; Kozinsky, B.; Csányi, G. The Design Space of E(3)-Equivariant Atom- Centered Interatomic Potentials. arXiv 2022, arXiv:2205.06643, accessed on 2024-04-22.arxiv.org/abs/2205.06643
  60. [60]Musaelian, A.; Batzner, S.; Johansson, A.; Sun, L.; Owen, C. J.; Kornbluth, M.; Kozinsky, B. Learning local equivariant representations for large-scale atomistic dynamics. Nat. Commun. 2023, 14, 579.arxiv.org/abs/2204.05249
  61. [61]Batatia, I.; Kovács, D. P.; Simm, G. N. C.; Ortner, C.; Csányi, G. MACE: Higher Order Equivariant Message Passing Neural Networks for Fast and Accurate Force Fields. arXiv 2023, arXiv:2206.07697, accessed on 2024-04-22.arxiv.org/abs/2206.07697
  62. [62]Ramakrishnan, R.; Dral, P. O.; Rupp, M.; von Lilienfeld, O. A. Quantum chemistry structures and properties of 134 kilo molecules. Sci. Data 2014, 1, 140022.DOI
  63. [63]Chmiela, S.; Tkatchenko, A.; Sauceda, H. E.; Poltavsky, I.; Schütt, K. T.; Müller, K.-R. Machine learning of accurate energy-conserving molecular force fields. Sci. Adv. 2017, 3, e1603015.DOI
  64. [64]Smith, J. S.; Isayev, O.; Roitberg, A. E. ANI-1: an extensible neural network potential with DFT accuracy at force field computational cost. Chem. Sci. 2017, 8, 3192–3203.DOI
  65. [65]Smith, J. S.; Zubatyuk, R.; Nebgen, B.; Lubbers, N.; Barros, K.; Roitberg, A. E.; Isayev, O.; Tretiak, S. The ANI-1ccx and ANI-1x data sets, coupled-cluster and density functional theory properties for molecules. Sci. Data 2020, 7, 134.DOI
  66. [66]Eastman, P.; Behara, P. K.; Dotson, D. L.; Galvelis, R.; Herr, J. E.; Horton, J. T.; Mao, Y.; Chodera, J. D.; Pritchard, B. P.; Wang, Y. et al. SPICE, A Dataset of Drug-like Molecules and Peptides for Training Machine Learning Potentials. Sci. Data 2023, 10, 11.arxiv.org/abs/2209.10702
  67. [67]Eastman, P.; Pritchard, B. P.; Chodera, J. D.; Markland, T. E. Nutmeg and SPICE: Models and Data for Biomolecular Machine Learning. J. Chem. Theory Comput. 2024, Sterling, T.; Irwin, J. J. ZINC 15 – Ligand Discovery for Everyone. J. Chem. Inf. Model. 2015, 55, 2324–2337.
  68. [69]Smith, J. S.; Nebgen, B.; Lubbers, N.; Isayev, O.; Roitberg, A. E. Less is more: Sampling chemical space with active learning. J. Chem. Phys. 148, 241733.DOI
  69. [70]Devereux, C.; Smith, J. S.; Huddleston, K. K.; Barros, K.; Zubatyuk, R.; Isayev, O.; Roitberg, A. E. Extending the Applicability of the ANI Deep Learning Molecular Potential to Sulfur and Halogens. J. Chem. Theory Comput. 2020, 16, 4192–4202.DOI
  70. [71]Chmiela, S.; Vassilev-Galindo, V.; Unke, O. T.; Kabylda, A.; Sauceda, H. E.; Tkatchenko, A.; Müller, K.-R. Accurate global machine learning force fields for molecules with hundreds of atoms. Sci. Adv. 2023-01-11, 9, eadf0873.arxiv.org/abs/2209.14865
  71. [72]Wu, Z.; Ramsundar, B.; Feinberg, E. N.; Gomes, J.; Geniesse, C.; Pappu, A. S.; Leswing, K.; Pande, V. MoleculeNet: A Benchmark for Molecular Machine Learning. arXiv 2018, arXiv:1703.00564, accessed on 2024-04-24.arxiv.org/abs/1703.00564
  72. [73]Sorkun, M. C.; Khetan, A.; Er, S. AqSolDB, a curated reference set of aqueous solubility and 2D descriptors for a diverse set of compounds. Sci. Data 2019, 6, 143.DOI
  73. [74]Modee, R.; Laghuvarapu, S.; Priyakumar, U. D. Benchmark study on deep neural network potentials for small organic molecules. J. Comput. Chem. 2022, 43, 308–318.DOI
  74. [75]Dwivedi, V. P.; Joshi, C. K.; Luu, A. T.; Laurent, T.; Bengio, Y.; Bresson, X. Benchmarking Graph Neural Networks. arXiv 2022, arXiv:2003.00982, accessed on 2024-04-24.arxiv.org/abs/2003.00982
  75. [76]Jumper, J.; Evans, R.; Pritzel, A.; Green, T.; Figurnov, M.; Ronneberger, O.; Tunyasuvunakool, K.; Bates, R.; Zídek, A.; Potapenko, A. et al. Highly accurate protein structure prediction with AlphaFold. Nature 2021, 596, 583–589.DOI
  76. [77]Varadi, M.; Anyango, S.; Deshpande, M.; Nair, S.; Natassia, C.; Yordanova, G.; Yuan, D.; Stroe, O.; Wood, G.; Laydon, A. et al. AlphaFold Protein Structure Database: massively expanding the structural coverage of protein-sequence space with high-accuracy models. Nucleic Acids Res. 2022, 50, D439–D444.DOI
  77. [78]The UniProt Consortium UniProt: the Universal Protein Knowledgebase in 2023. Nucleic Acids Res. 2023, 51, D523–D531.DOI
  78. [79]Mongan, J.; Simmerling, C.; McCammon, J. A.; Case, D. A.; Onufriev, A. Generalized Born model with a simple, robust molecular volume correction. J. Chem. Theory Comput. 2007, 3, 156–169.pubmed.ncbi.nlm.nih.gov/21072141
  79. [80]Nguyen, H.; Roe, D. R.; Simmerling, C. Improved Generalized Born Solvent Model Parameters for Protein Simulations. J. Chem. Theory Comput. 2013, 9, 2020–2034.pubmed.ncbi.nlm.nih.gov/25788871
  80. [81]Wang, T.; He, X.; Li, M.; Shao, B.; Liu, T.-Y. AIMD-Chig: Exploring the conformational space of a 166-atom protein Chignolin with ab initio molecular dynamics. Sci. Data 2023, 10, 549.DOI
  81. [82]Onufriev, A. V.; Case, D. A. Generalized Born Implicit Solvent Models for Biomolecules. Annu. Rev. Biophys. 2019, 48, 275–296.DOI
  82. [84]Gao, X.; Ramezanghorbani, F.; Isayev, O.; Smith, J. S.; Roitberg, A. E. TorchANI: A Free and Open Source PyTorch-Based Deep Learning Implementation of the ANI Neural Network Potentials. J. Chem. Inf. Model. 2020, 60, 3408–3415.DOI
  83. [85]Honda, S.; Akiba, T.; Kato, Y. S.; Sawada, Y.; Sekijima, M.; Ishimura, M.; Oishi, A; Watanabe, H.; Odahara, T.; Harata, K. Crystal Structure of a Ten-Amino Acid Protein. J. Am. Chem. Soc. 2008, 130, 15327–15331.DOI
  84. [86]Kim, S.; Kim, H. A new metric of absolute percentage error for intermittent demand forecasts. Int. J. Forecast. 2016, 32, 669–679.DOI
  85. [87]Ingraham, J. B.; Baranov, M.; Costello, Z.; Barber, K. W.; Wang, W.; Ismail, A.; Frappier, V.; Lord, D. M.; Ng-Thow-Hing, C.; Van Vlack, E. R. et al. Illuminating protein space with a programmable generative model. Nature 2023, 623, 1070–1078.DOI
  86. [88]Eastman, P.; Swails, J.; Chodera, J. D.; McGibbon, R. T.; Zhao, Y.; Beauchamp, K. A.; Wang, L.-P.; Simonett, A. C.; Harrigan, M. P.; Stern, C. D. et al. OpenMM 7: Rapid development of high performance algorithms for molecular dynamics. PLOS Comput. Biol. 2017, 13, e1005659.DOI
  87. [89]Best, R. B.; Zhu, X.; Shim, J.; Lopes, P. E. M.; Mittal, J.; Feig, M.; MacKerell, A. D. J. Optimization of the Additive CHARMM All-Atom Protein Force Field Targeting Improved Sampling of the Backbone ϕ, ψ and Side-Chain χ₁ and χ₂ Dihedral Angles. J. Chem. Theory Comput. 2012, 8, 3257–3273.
  88. [92]Reuther, A.; Kepner, J.; Byun, C.; Samsi, S.; Arcand, W.; Bestor, D.; Bergeron, B.; Gadepally, V.; Houle, M.; Hubbell, M. et al. Interactive supercomputing on 40,000 cores for machine learning and data analysis. 2018 IEEE High Performance extreme Computing Conference (HPEC). 2018; p 1–6.DOI
  89. [93]Loschchilov, I.; Hutter, F. Decoupled Weight Decay Regularization. arXiv 2019, arXiv:1711.05101, accessed on 2024-04-22.arxiv.org/abs/1711.05101
  90. [94]Robbins, H.; Monro, S. A Stochastic Approximation Method. Ann. Math. Stat 1951, 22, 400–407.
  91. [95]Cleveland, W. S. Robust Locally Weighted Regression and Smoothing Scatterplots. J. Am. Stat. Assoc. 1979, 74, 829–836.DOI

Paper details

vlnc:W_8sc42rps
Version
v1 · latest
Published
Fresh — added to the Hub within the last 30 days
Added
License
CC BY 4.0
Usage

views · 0

downloads · 0

A view counts once per browser per day, a download each time the PDF is requested.

  • Physics and Astronomy

Not recorded for this paper

View statement
Looking for related papers…
Contents