betaFeedback

WorksPhysics and Astronomy

Convergence of Body-Orders in Linear Atomic Cluster Expansions

Abstract

We study the convergence of a linear atomic cluster expansion (ACE) potential with respect to its basis functions, in terms of the effective two-body interactions of elemental Carbon and Silicon systems. We build ACE potentials with descriptor sets truncated at body-orders K=2K=2 to K=5K=5 trained on a diverse Carbon dataset and on Silicon dimers to pentamers. The potentials trained on diverse structures with standard ACE bases are not able to recover the correct dimer curves much less produce stable curves or solutions. While employing ACE bases removed of self-interactions still does not generalize to the DFT-expected results, properly tailored datasets and basis sets are able to show signs of convergence and stability in the curves and expansions, suggesting a method to build potentials with interpretable bases with respect to the cluster expansion.

Introduction

Decompositions of the total energy of atomic systems by NN-body contributions, or cluster expansions,

E({ri})=V(0)+∑iV(1)(ri)+∑i,ji≠jV(2)(ri,rj)+∑i,j,ki≠j≠k≠iV(3)(ri,rj,rk)+…(1)E(\{\mathbf{r}_i\}) = V^{(0)} + \sum_i V^{(1)}(\mathbf{r}_i) + \sum_{\substack{i,j \\ i \ne j}} V^{(2)}(\mathbf{r}_i,\mathbf{r}_j) + \sum_{\substack{i,j,k \\ i \ne j \ne k \ne i}} V^{(3)}(\mathbf{r}_i,\mathbf{r}_j,\mathbf{r}_k) + \ldots \tag*{(1)}

have been used as the framework to study material properties such as the thermodynamics of substitutional alloys [2, 3] and empirical potentials of condensed matter and molecular systems [4, 5, 6]. While the expansion has been shown to be complete [2, 3, 7] when the number of atoms matches the highest body order of the expansion, its practical effectiveness is limited to cases where the total energy converges after a few body-orders ν\nu [4, 5, 8], as the number of terms increases combinatorially as higher orders are included.

More recently, the expansion has been used in describing local chemical environments for machine learning potentials. The underlying assumption for these descriptors is that the energy of the system is a sum of its individual atomic contributions EiE_i [9],

E({ri})=∑i=1NEi({ri})(2)E(\{\mathbf{r}_i\}) = \sum_{i=1}^{N} E_i(\{\mathbf{r}_i\}) \tag*{(2)}

where NN indicates the total number of atoms in the structure. The atomic energies may then be expressed as an expansion in the number of neighbors.

Improvements on the construction of EiE_i have been made in recent decades, with symmetry-respecting descriptors that use only a few terms in the expansion [9, 10, 11, 12] or take advantage of tensorial operations to reach an arbitrary body order [1, 13, 14, 15]. A detailed review on descriptors of local chemical environments has been done by Musil et al. [16]

A popular version of this resummation is the Atomic Cluster Expansion proposed by Drautz [1]. In this approach, each EiE_i is expanded with a non-orthogonal basis, {Aν}\{A_\nu\}, with unrestricted sums,

Ei({ri})=∑νcν(1)Ai,ν+∑ν1≥ν2ν1ν2cν1ν2(2)Ai,ν1Ai,ν2+…(3)E_i(\{\mathbf{r}_i\}) = \sum_\nu c_\nu^{(1)} A_{i,\nu} + \sum_{\substack{\nu_1 \ge\nu_2 \\ \nu_1 \nu_2}} c_{\nu_1\nu_2}^{(2)} A_{i,\nu_1} A_{i,\nu_2} + \ldots \tag*{(3)}

with single sums over neighbors

Ai,ν=∑jfν(ri,rj),(4)A_{i,\nu} = \sum_j f_\nu(\mathbf{r}_i,\mathbf{r}_j), \tag*{(4)}

and ν\nu indexing the terms of the expansion. The expansion has been shown to be complete [7], and its implementations are effective in machine learning applications [17, 18, 19, 20, 21]. One of the state-of-the-art potentials, MACE [15], extends the formalism of ACE as a graph neural network but the interpretability of the expansion is traded for better expressivity when compared to linear potentials.

In employing interatomic potentials using the ACE framework, the selection of the basis complexity in the number of functions and maximum body-order KK is a heuristic process and no systematic construction has been proposed yet. While ACE potentials have been shown to achieve low fitting errors, the stability of these solutions and the interpretability of the basis functions are not as studied, beyond undertakings to eliminate the self-interactions in Ho et al. [22] and perform uncertainty quantifications in linear models in Chong et al. [23].

In the advent of foundational machine learning models [24] as baselines training neural network potentials that do not need to be trained from scratch, it may also be worth asking if a method to systematically build a body-ordered basis with reasonable lower-order energetics can be established. In this work, we investigate the convergence of linear ACE at different body-orders KK in terms of their projection on the lowest terms of the original cluster expansion Eq. (1). In particular, we consider the convergence of the potential in energy and forces on different datasets as a function of the number and type of basis functions utilized, and compare it to the stability of a real body-ordered expansion. Our study focuses on the first nontrivial term of the expansion: the two-body term. This can be easily extracted from a given potential by studying the energy profile of two isolated atoms, a dimer, as a function of distance:

E2b,i(ri,rj)=V(2)(ri−rj).(5)E_{2b,i}(\mathbf{r}_i,\mathbf{r}_j) = V^{(2)}(\mathbf{r}_i-\mathbf{r}_j). \tag*{(5)}

It is thus natural to ask whether this function might converge to the real dimer dissociation curve, or to another arbitrary function for a dataset that does not contain dimers.

While nonlinearities can improve the performance of machine learning potentials by representing higher-order terms when the expansion is truncated, care should be taken in how they are applied as certain applications may invalidate the body-ordering of the descriptor [14]. In the present work, we do not aim to build the most accurate potentials, but to have a more grounded understanding of their behavior in the ACE framework. As such, we limit our investigations to training simpler linear models as well as not employing very high of a body-order where returns are diminished. Linear ACE models perform comparably well relative to neural network potentials, and have been successfully employed to describe molecular [17] and condensed phases [18].

Methods

Datasets

The datasets we use in the investigations consists of elemental structures of Carbon and Silicon. For Carbon, three datasets were used: the first set consists of 247 dimer structures at distances between 0.5 Å and 8 Å, with half selected for the validation set. The other two are sets of 1000 structures coming from a collection of Carbon allotropes at low pressures (0–0.1 GPa) [25]. One subset contains a homogeneous collection of diamond-like structures clustered using a Euclidean distance measure [26]. The diamond-like dataset is characterized through a histogram of distances to a reference diamond structure in Figure 1. The last subset are structures obtained using farthest point sampling [27] to maximize its diversity. For the diamond-like and diverse datasets, a validation set was constructed from 1000 diverse structures that have not been seen in training. The Carbon data were calculated using the Quantum Espresso package [28] using the rVV10 functional [29] to account for nonlocal interactions.

Histogram of fingerprint distances of the configurations in the Carbon dataset

Figure 1. Histogram of fingerprint distances of the configurations in the Carbon dataset in [25] relative to a reference diamond-like structure (inset). 1000 configurations were chosen within a narrow region in structure space.

Another dataset of molecular Silicon structures taken from [23] was considered as simpler cases where the maximum body order KK in the bases is comparable to the number of neighbors in the structures. For each subset, 90% was used for training and the rest were used for validation.

Descriptor

We use a linear ACE descriptor as implemented by both the pacemaker package [19] and the ACEpotentials.jl [21]. We start from K=2K = 2 body functions and progressively add more terms of the same order. We validate the models at each addition of basis functions, and monitor when the interim model begins to overfit. Before the generalization error increases, we then introduce higher order K=3K = 3 terms while keeping the number of lower order terms fixed, and then repeat the procedure for higher orders. ACE descriptors employ spherical harmonics to describe angular functions, with n,ln,l values chosen to respect rotational symmetry. Low n,ln,l values were chosen to avoid overfitting the model from too many basis functions.

Fitting procedure

Training was done with a least squares error method for both pacemaker and ACEpotentials.jl methods. For pacemaker the loss Lp\mathcal{L}_p is,

Lp=(1−κ)ΔE2+κΔF2+Δcoeff+Δrad(6)\mathcal{L}_p = (1-\kappa)\Delta_E^2 + \kappa\Delta_F^2 + \Delta_{\mathrm{coeff}} + \Delta_{\mathrm{rad}} \tag*{(6)}

where ΔE2\Delta_E^2 and ΔF2\Delta_F^2 are the corresponding energy and force root-mean-square errors (RMSE), and κ\kappa is the coefficient that balances the energy and force contributions to L\mathcal{L}. The term Δcoeff\Delta_{\mathrm{coeff}} regularizes the basis function coefficients2, while Δrad\Delta_{\mathrm{rad}} regularizes the coefficients of the radial components of the basis function. ACEpotentials.jl minimizes the loss LA\mathcal{L}_A through the kernel ridge regression methods,

LA=∥W(y−Ac)∥2+λ∥Γc∥2(7)\mathcal{L}_A = \lVert\mathbf{W}(\mathbf{y}-\mathbf{A}\mathbf{c})\rVert^2 + \lambda\lVert\mathbf{\Gamma}\mathbf{c}\rVert^2 \tag*{(7)}

where W\mathbf{W} encodes the relative loss weightings for energies and forces, y\mathbf{y} contains the labeled data, A\mathbf{A} is the basis matrix and c\mathbf{c} are the parameters to be learned to minimize LA\mathcal{L}_A. Γ\Gamma contains a more general treatment to regularization allowing different penalizing strengths to the coefficients c\mathbf{c}, and λ\lambda dictates the strength of the regularization. We used the Bayesian Linear Regression to let the model dictate appropriate values for λ\lambda and Γ\Gamma. Details of the regression methods are found in Witt et al. [21], and the Supporting Information discusses more details in the numerical setup. Evaluation of dimer curves were done using the Atomic Simulation Environment (ASE) by Hjorth Larsen et al. [30].

Aside from the parameters involving basis set complexities, radial bases/geometric priors, and the loss weightings, other values were set to default for both pacemaker and ACEpotentials.jl to ensure a reasonable set of controlled parameters.

Results and Discussion

Self-interacting ACE bases

We start by fitting the dimer dataset with an increasing number of basis functions, NfN_f, up to 250 elements and K=4K = 4 body-order. The loss and dimer curves are shown in Figure 2, where light green, green, and dark green indicate data from potentials trained using KK up to 2, 3, and 4, respectively. The top plot shows the dependence of the loss function L\mathcal{L} on NfN_f. The validation loss (solid curve) converges for K=2K = 2 and increases with the addition of higher order functions, jumping two orders of magnitude as more K=4K = 4 functions are added.

Loss curves and dimer curves for pacemaker potentials

Figure 2. pacemaker (Top) Loss as a function of number of basis functions, NfN_f, for the dimer dataset. Training and validation errors as dashed and solid curves. Curve colors indicate potentials built from basis sets which are exclusively two-body (light green), up to three-body (green), and up to four-body functions (dark green), respectively. Light solid lines indicate the loss if more functions were introduced without introducing higher KK. (Bottom) dimer curves for each of the potentials trained at certain NfN_f, with reference DFT data as points. Heavy lines represent the potential with the largest NfN_f before functions of higher KK are introduced.

This is of course to be expected, as the dataset consists solely of 2 body structures. The bottom plot highlights the dimer curves of the potentials trained from the dimer set at the highest NfN_f of a particular order KK. We see that all potentials are able to reproduce the dimer curve as expected from the DFT results.

The energy RMSE for the representative potentials are around 5 meV/at, and force RMSE are slightly less than 200 meV/Å for K=2K = 2 and K=3K = 3. The force RMSEs increase to 2700 meV/Å for the overfitting potential at Nf=250N_f = 250, and are due to the short-range region of the curve having energies up to almost two orders of magnitude higher, making the deviations from DFT results more sensitive to errors. A comparison of the forces at the short range region may be found in the Supporting Information B.1.

The results for the homogeneous diamond-like dataset are shown in Figure 3. Unlike the potentials trained from the dimer dataset, the training loss from the diamond-trained potentials reduce by an order of magnitude upon the introduction of the K=3K = 3 functions. The light orange line indicates that given the same NfN_f, the basis set purely constructed from K=2K = 2 functions does not have as pronounced of a decrease in L\mathcal{L}. We see that the validation over the diverse set has losses improving for K=3K = 3, but overfitting occurs at K=4K = 4. The dimer curves from these potentials show a considerable amount of variability even after the loss is converged, and they remain far from the expected dimer behavior dictated by DFT. Of course, the lack of diversity in this dataset, leading to very small variations in the radial distribution function, might be part of the reason why a rather flat 2-body curve is obtained. Moreover, the overfitting for the large descriptor size could contribute to the variability of this contribution.

Top: loss versus number of functions and bottom: dimer curves for the homogeneous, diamond-like dataset

Figure 3. pacemaker (Top) L\mathcal{L} vs. NfN_f and (Bottom) dimer curves for the homogeneous, diamond-like dataset. Orange and pink plots correspond to potentials with only K=2K = 2 and up to K=3K = 3 functions, respectively while purple curves include K=4K = 4 functions. The x-values of the dimer curve start at the minimum distance found in the homogeneous dataset.

We then look at the dimer curves of the potentials for different descriptor complexities, ranging from K=2K = 2 to 5. The blue-green curves are potentials trained with 1000-structures while the orange-red curves are trained with a larger 50000-structure set. Figure 4 shows these curves having higher variability compared to the ones produced by the diamond-like dataset. While the curves look similar as more functions of the same KK are introduced in the set, as seen in the K=4K = 4 potentials, the curves of different KK limits do not seem to converge even when five-body contributions are already added. The variability is much more pronounced, even among potentials with similar maximum KK and more so when adding new terms.

Dimer curves for diverse 1000-structure and 50000-structure datasets

Figure 4. pacemaker Dimer curves for the diverse 1000-structure (blue-green) and 50000-structure (orange-red) for various descriptor sizes. No convergence observed for both sets as higher KK functions are introduced.

We note that the lack of pairs around the 2.0 Å region are due to the structures in the dataset belonging to relatively low pressures. The comparisons between the 1000-structure and the more complete 50000-structure datasets have virtually similar pair distribution functions due to the farthest point sampling methods done to sample the smaller subset. The pair distribution comparisons may also be seen in Supporting Information C.

Finally, we try the same experiment using ACEpotentials.jl using the 1000-structure set with the plots. Figure 5 shows qualitative improvements of the dimer curve in the equilibrium bond length r0r_0 and energy minimum may be attributed to ACEpotentials.jl requiring additional user inputs (i.e. geometric priors) and more rigorous regularization schemes[21]. The routines involving Bayesian Linear Regression also allow for the automatic selection of regularization strengths λ\lambda in Eq. (7) which helps reduce the number of hyperparameter sweeps done. In contrast, the absence of a more involved radial basis construction3 in the qualitative features of

Dimer curves for the 1000-structure Carbon dataset trained with ACEpotentials.jl

Figure 5. ACEpotentials.jl Dimer curves for 1000-structure Carbon dataset trained with the Julia code [21]. Dashed lines indicate the ACE basis (K=3,4,5K = 3,4,5) with self-interactions similar to the one in pacemaker, while the solid lines are trained with a basis purified of self-interactions. Gray bars show the pair distance distribution in the training set. Using a purified basis for the Carbon dataset yields practically identical dimer curves with those from the self-interacting basis for up to K=4K = 4, with a qualitative difference in the energy well for the K=5K = 5 potential.

the dimer curve lead to its greater reliance on a richer dataset.

Still, it seems that the potentials made from purified bases (solid curves) do not produce a marked improvement over the self-interacting basis (dashed lines) as they are mostly overlapping. While it seems that no improvements are seen in the purified basis, the study of the gram matrix conditioning from [22] indicates that the potentials become less stable when the total number of relevant particles JJ (or neighbors in periodic systems) far exceed the maximum body-order, KK, of the basis used to describe the potential4.

We also make sure that the performance of our linear models are reasonable by comparing their loss metrics to the performance of different neural network potentials trained on the same dataset in Pellegrini et al. [31], as shown in Table I. For the 1000-size subsets, we found that the linear ACE models with the most complex descriptor sets (K=4,Nf=930K = 4, N_f = 930 and K=5,Nf=793K = 5, N_f = 793) were able to outperform all PANNA potentials and even the l=1l = 1 version of the NequIP [32] potential. While promising, we may attribute the poor performance of the neural network potentials from the fact that 1000 structures may not be enough data and the resulting potentials may overfit without appropriate regularizations.

However, they perform relatively worse with all other potentials when the complete 50000-size Carbon set is used as the training set, which suggests that the nonlinearities absent in the ACE models provide a considerable improvement in training more complex datasets. Despite this, we believe that the inability of the linear models to capture the effective two-body interaction and converge to a defined shape are not due to the poor LL, but rather the nature and truncation of the expansion employed in the descriptor sets, which is further discussed in the experiments with purified bases for the Silicon dataset.

Linear Potentialsnset=1000n_{\mathrm{set}} = 1000nset=50000n_{\mathrm{set}} = 50000
EFEF
pacemaker
K=4K = 4, Nf=250N_{\mathrm{f}} = 25048.50241.2744.06222.42
K=4K = 4, Nf=493N_{\mathrm{f}} = 49318.30199.2317.55176.48
K=4K = 4, Nf=930N_{\mathrm{f}} = 93015.40167.7315.27166.70
K=5K = 5, Nf=793N_{\mathrm{f}} = 79314.88160.7814.74159.80
ACEpotentials.jl
K=3K = 3, Nf=297N_{\mathrm{f}} = 29738.10336.84--
K=4K = 4, Nf=802N_{\mathrm{f}} = 80218.00210.14--
K=5K = 5, Nf=929N_{\mathrm{f}} = 92916.72202.78--
Neural Networks
PANNA small22.40248.8517.76162.37
PANNA mid20.93241.3612.41137.45
PANNA big17.88242.028.21166.95
NequIP l=1l = 120.40213.007.96110.00
NequIP l=2l = 26.83105.005.2151.40
MACE6.81103.141.8251.31

Table 1. TABLE I. Comparison of linear ACE models to neural network potentials in MAE for energies and forces for the Carbon dataset in [25] for datasets containing nset=1000n_{\mathrm{set}} = 1000 and 50000 structures. Energies (E) are in meV/atom and force components (F) are in meV/Å.

Purified ACE bases

We then repeat the same investigation on a dataset of Silicon structures[23] containing dimers, trimers, tetramers, and pentamers so the maximum number of atomic neighbors matches K=5K = 5. We train potentials from subsets of 100 trimers (Si3_3), 100 tetramers (Si4_4), and 500 pentamers (Si5_5), and a mixture of the structures leaving out the dimers (Si345_{345}) to mimic the diverse Carbon set. Figure 6 highlights that the out-of-distribution Silicon potentials, those trained without dimers, also have curves that are not qualitatively recovered. Additionally, the Si345_{345} potentials, which strikes the closest parallel to the diverse Carbon dataset, exhibit the overlapping of purified and self-interacting potentials albeit at a lesser degree than the Carbon tests. When we look at the coefficients of the expansion, the non-overlap of the lines show the instability of the solutions for the potentials at different KK.

OOD coeffs and curves for Si345, Si3, Si4, and Si5

Figure 6. ACEpotentials.jl (Bottom of each quadrant) Dimer curves of potentials from the Silicon structures in Chong et al. [23] trained at different subsets. Simn\mathrm{Si}_{mn} indicate potentials were trained from datasets that include mm-mers and nn mers, black points show the DFT dimer energies, and gray bars indicate the distributions of pairs in the dataset. Overlapping solid orange (K=4K = 4) and green (K=5K = 5) lines indicate that potentials with purified bases eventually converge to a defined shape when the total number of particles NN match the maximum body order KK. (Upper plots of each quadrant) Coefficients of the purified (cpurec_{\mathrm{pure}}) and self-interacting (cselfc_{\mathrm{self}}) potentials. Blue, orange, and green lines correspond to coefficients of potentials capped at K=3K = 3, K=4K = 4, and K=5K = 5 body-ordered functions while blue, orange, and green shaded regions indicate coefficients corresponding to K=2K = 2, K=3K = 3, K=4K = 4, and K=5K = 5 body-ordered functions. Overlapping lines for a certain region mean that potentials have very similar basis coefficients, suggesting the expansion has stabilized for that particular body-order.

When focusing on the purely trimer trained Si3_3 potentials, we see signatures of convergence at the purified basis. Dimer curves of potentials at all KK overlapping and the basis coefficients cpurec_{\mathrm{pure}} for the 2- and 3-body functions (in the blue region) are consistent with each other. The cpurec_{\mathrm{pure}} corresponding to K≥4K \ge4 functions are 0, which suggests that the self-interactions are indeed removed. In the Si4_4-trained potentials, dimer curves for the K=3K = 3 basis set already deviates from the others due to the lower-order cpurec_{\mathrm{pure}} misrepresenting the higher-order tetramer interactions. We also see this happening for the Si5_5-trained potentials and even for the K=4K = 4 basis sets for their misrepresentation of the 5-body interactions. Overall, the qualitative features of the dimer curves do not seem to converge as basis functions become more complex with higher KK. This makes it difficult to apply the potentials to structures it has not seen during training.

While generalization to the correct DFT curve seems unlikely, we can still see how the coefficients of purified linear ACE models perform when doing interpolative tasks and dimers are included in the training set. It is evident in Figure 7 that the correct DFT results are recovered regardless of basis max KK or whether the basis was purified or not. To understand whether the solution we obtain is stable, we again look at the coefficients of the potentials and see that the purification is successful in disentangling lower-order terms from the contributions of functions at higher KK. It is also worth noting that the cselfc_{\mathrm{self}} for the Si2_2-trained datasets look identical to the purified coefficients, and is possibly due to the fact that the self-interaction terms are not needed to capture the dimer energetics.

ACE coefficients and dimer curves of Silicon potentials

Figure 7. ACEpotentials.jl (Top of each quadrant) ACE coefficients and (bottom of each quadrant) dimer curves of Silicon potentials trained at different subsets, including dimers in all of them. All dimer curves recover the correct DFT values, and purified potentials. All cselfc_{\mathrm{self}}, except for the Si2_2 training, had higher-order renormalizations as seen in the nonzero K=3,4K=3,4 for Si23_{23} and K=4K=4 for Si234_{234}. cpurec_{\mathrm{pure}} coefficients are similar for the K=2,3K=2,3 terms in Si23_{23}-trained potentials, and similar for K=4K=4 terms in the Si234_{234} potentials.

The mismatch in the overlap of cpurec_{\mathrm{pure}} can still be seen in the K=2,3K=2,3 functions of the Si234_{234} sets and the K=2,3,4K=2,3,4 functions of the Si2345_{2345} sets which is the signature of aliasing that occurs when the basis is not complex enough to capture the higher-order energetics. To see if we are consistently getting not only the correct dimer curve, but a stable pair potential, we look at the cpurec_{\mathrm{pure}} of the K=2K=2 basis functions for datasets where max KK matches the maximum number of particles JJ in Figure 8. We include the Si2_2 potential with the K=3K=3 basis as the reference coefficient set. Surprisingly, we see that no other potential was able to capture the reference coefficients completely, with greater deviations coming from the Si234_{234} and Si2345_{2345} potentials. Looking at the discrepancy between training and validation errors between potentials, the high validation error for the Si234_{234} potential suggests that more tetramers need to be sampled. For the Si2345_{2345} potential, both training and validation errors

ACEpotentials.jl cpure of the K = 2 basis functions

Figure 8. ACEpotentials.jl cpurec_{\mathrm{pure}} of the K=2K=2 basis functions for potentials with basis complexities matching the max number of particles, K=JK=J.

being high implies that the K=5K=5 basis functions as well as more pentamers are needed.

Summary and conclusions

In this work, we studied the convergence of linear cluster expansions with respect to the body-ordered basis functions through the effective two-body interactions, E2bE_{2b} of elemental Carbon and Silicon systems. We used the atomic cluster expansion (ACE), with descriptor sets truncated at body-orders K=2K=2 to K=5K=5 trained on a diverse Carbon dataset and on Silicon dimers to pentamers. The potentials trained on diverse structures with standard ACE bases were not able to recover the correct dimer curves much less produce stable curves or solutions. While employing ACE bases removed of self-interactions still did not generalize to the DFT-expected results, properly tailored datasets and basis sets were able to show signs of convergence and stability in the curves and expansions. Stabilizing the coefficients of the expansion order-by-order by procedurally training potentials at bases up to KK from training sets with KK-mers, and possibly utilizing uncertainty quantification metrics as detailed in Chong et al. [23], shows promise as a framework to build a potential with a more interpretable basis using the cluster expansion.

Supporting Information Available

Detailed information on the simulation parameters for both codes is found in Sec. A. Hyperparameter investigations, particularly in balancing the loss contributions between energies and forces, are found in Sec. B. Additional plots may be found in Sec. C, namely force plots between DFT and dimer-trained potential, loss analysis of diverse dataset dimer curves, biasing the diverse dataset with dimer data, learning curves of the diverse dataset runs as well as the extended datasets. Datasets and codes used for calculations may be found in https://github.com/apolmiguel/aceconverge2025.

Acknowledgement

AT thanks Gabriele Coiana and Jerry Ho for fruitful discussions. This study was funded by the European Union – NextGenerationEU – PNRR M4C2-I.1.4, in the framework of the project CN-HPC: National Centre for HPC, Big Data and Quantum Computing - Spoke 7 - (MUR ID: CN 00000013 – CUP: G93C22000600001). FP work was supported by EU – FSE REACT-EU and Italian MUR under contract 45-I-D2100-1 of the PON-RI 2014-2020 initiative. SdG work was supported in part by the European Commission through the MAX Centre of Excellence for supercomputing applications (grant numbers 10109337 and 824143). Computational resources were provided by CINECA.

Simulation Parameters

For the pacemaker tests, the cutoff radius is set to 5 Angstrom. The potential does not use the nonlinear version of the Finnis-Sinclair potential which is set on by default in the pacemaker package. The radial basis is the Chebyshev expansion with the cosine cutoff function. The energy-force cost balancing is set that the RMSE for energies is weighted 50 times more than the forces, and further investigations is shown in Section B. The optimizer is set to BFGS with a maximum of 3000 iterations and a batch size of 100. Details about the input files may be found at the documentation of pacemaker. Table II distinguishes the training parameters done for the dimer curves presented in the main paper.

Training setDescriptor labelTrain stepsValidation setnmax⁡n_{\max}lmax⁡l_{\max}
Dimer 124K=2,Nf=45K = 2, N_f = 453000val_dimer 1231000
Dimer 124K=3,Nf=90K = 3, N_f = 903000val_dimer 12345/80/2
Dimer 124K=4,Nf=250K = 4, N_f = 2503000val_dimer 12345/5/40/2/2
Diamond 1000K=2,Nf=45K = 2, N_f = 453000val_diverse 1231000
Diamond 1000K=3,Nf=90K = 3, N_f = 903000val_diverse 12345/80/2
Diamond 1000K=4,Nf=250K = 4, N_f = 2503000val_diverse 12345/5/40/2/2
Diverse 1000K=2,Nf=45K = 2, N_f = 453000val_diverse 10001000
Diverse 1000K=3,Nf=90K = 3, N_f = 903000val_diverse 100045/80/2
Diverse 1000K=4,Nf=250K = 4, N_f = 2503000val_diverse 100045/5/40/2/2
Diverse 1000K=4,Nf=493K = 4, N_f = 4933000val_diverse 100045/8/40/3/3
Diverse 1000K=4,Nf=960K = 4, N_f = 9603000val_diverse 100045/9/50/6/3
Diverse 1000K=4,Nf=793K = 4, N_f = 7933000val_diverse 100045/8/4/30/3/3/2
Diverse 50000K=4,Nf=250K = 4, N_f = 2503000val_diverse 100045/5/40/2/2
Diverse 50000K=4,Nf=493K = 4, N_f = 493810val_diverse 100045/8/40/3/3
Diverse 50000K=4,Nf=960K = 4, N_f = 960418val_diverse 100045/9/50/6/3
Diverse 50000K=4,Nf=793K = 4, N_f = 7931151val_diverse 100045/8/4/30/3/3/2

Table II. Simulation Parameters for binding curve experiments in pacemaker

We used v0.6.7 of ACEpotentials.jl as it was a stable version for the implementation of the purification algorithm. We used the ACE1x.ace_basis module to create purified bases, and is done by setting pure = true in the arguments. We chose to specifically control the maximum degrees per order, with the K=3,4,5K = 3, 4, 5 basis sets being [24, 20], [24, 20, 16], and [24, 20, 16, 12] respectively. These led to each basis set having 297, 802, and 929 basis functions, respectively5. The number of basis functions were not explicitly selected as the number of functions are consequences of the maximum orders selected by the ACE1x.ace_basis function. We made sure to make the maximum degrees of each order even, with the lower orders having degrees considerably higher than the succeeding ones following the prescription of Ho et al. [22] to ensure that the purified bases span the same space as the original self-interacting basis. We chose rcut=5.0 and expected equilibrium bond length r0=1.287 for the Carbon tests as these are the expected parameters for the dataset, while rcut=5.0, r0=2.207 were chosen for the Silicon datasets.

ACEpotentials.jl training was done using the Bayesian Linear Regression module by using ACEfit.BLR() to automatically determine the best values for the smoothness prior Γ\Gamma and regularization strength λ\lambda. We also found out that this solver provided the most stable dimer curves with respect to changes in basis as compared to LSQR solvers in Witt et al. [21] as seen in Figure S9. Optimizations for LSQR-solved potentials was not done due to the vanishing dimer curves at higher KK. BLR-solved potentials were used mostly “out-of-the-box” values (i.e. Cholesky factorizations used), but qualitative similarities were already present for the ACEpotentials.jl dimer curves. Careful hyperparameter sweeps can be done to optimize the losses, but this is not the main focus of the work.

Comparison of ACEpotentials.jl dimer curves between LSQR and BLR solvers

Figure S9. Comparison of ACEpotentials.jl dimer curves between LSQR (colored) and BLR (gray) solvers for basis sets K=3K = 3 to K=5K = 5 (left to right) and smoothness prior p=4p = 4. Different colors for potentials regressed with LSQR methods indicate different regularization tolerance strengths. K=3K = 3 limited LSQR-solved potentials reaches the BLR-solved potentials as the regularization tolerance is increased. K=4,5K = 4, 5 LSQR-solved potentials had vanishing dimer curves for all tolerances. Energy-to-force loss weighting was set to 50 for all potentials except for the BLR-solved K=5K = 5 which was set to 10 due to issues in getting noninvertible matrices in the solver.

Relative weighting for energy and force losses

In pacemaker, we chose default values for the regularization weights but selected the appropriate energy-to-force loss weighting ratio with the one that simultaneously minimizes the RMSE for energies and forces. Visually, it is the point closest to the origin in the right plot of Figure S10. We see that the best results show for when the energy is weighted 50 times stronger than forces in the regression task (red), and seems to be the start of the dimer curve being “converged” with respect to the coefficient ratio. The study focused on the energetics of the system, so weighting the energies seemed appropriate and our tests showed that increasing the coefficient from 1 to 50 had a marked improvement in the reduction of RMSE for energies, while only slightly penalizing the forces.

RMSE plots for energy, forces, and loss, and RMSE forces versus RMSE energies

Figure S10. (Left, top to bottom) RMSE for energy, forces, and loss for different ratios for coefficients between energy and forces. 10 means energy is weighted 10 times more. (Right) RMSE forces vs. RMSE energies for the different loss coefficient ratios. Faint lines indicate the progression of each of the losses as a function of iterations. Final values are cut at the point where the validation error starts to increase considerably.

We were not able to do the same loss balancing experiments for ACEpotentials.jl due to problems in the matrix for the linear regression being non-invertible. We set the energy-force weighting for the Carbon dataset tests at 10, while it was set at 50 for the Silicon datasets.

Dimer curves of each of the loss coefficient values

Figure S11. Dimer curves of each of the loss coefficient values at the points in the right of Figure S10. For the remainder of the tests in the main paper, the chosen energy loss coefficient is 50 times the forces.

Forces at the short-range region for the dimer-trained potential and the difference between the forces of the DFT and the dimer-trained potential

Figure S12. Forces at the short-range region for the dimer-trained potential and the difference between the forces of the DFT and the dimer-trained potential (green, right y-axis).

Loss $\mathcal{L}$ vs. $N_f$ and dimer curves for the diverse dataset

Figure S13. L\mathcal{L} vs. NfN_f (top) and dimer curves (bottom) for the diverse dataset. Orange and pink plots correspond to potentials with only K=2K = 2 and up to K=3K = 3 functions, respectively while purple curves include K=4K = 4 functions. The x-values of the dimer curve start at the minimum distance found in the homogeneous dataset.

Additional plots

Force plots between DFT and dimer-trained potential

Fig. S12 shows the force values from the reference DFT data (blue scatter) as compared with the values predicted by the NNP (orange line) at the short-range region for the dimer-trained potential with K=4K = 4 and Nf=250N_f = 250. We see that despite relatively low relative errors (maximum at 13%), the absolute value of the forces admit large errors which contribute to the large validation errors at the overfitting region.

Loss analysis of diverse dataset dimer curves

Fig. S13 (top) shows the loss L\mathcal{L} vs. number of functions NfN_f for the potential trained on the diverse dataset and (bottom) the corresponding dimer curves at the terminal NfN_f for each body-order KK. We see from the loss curves that opening the channel for more complex basis functions by increasing KK does help improve training. Further evidence of this comes from the faint lines, which are loss data from potentials still trained with K=2K = 2 (or K=3K = 3) but have the same number of functions as the next higher KK. Despite that, the dimer curve shows that there is a more prominent qualitative change going from K=2K = 2 to K=3K = 3 rather than from K=3K = 3 to K=4K = 4 despite having a smaller drop in losses. From here we see that the parameters at which the loss converges do not indicate that the curves have already converged.

Biasing the diverse dataset with dimer data

Figure S14 shows the effect of biasing the diverse dataset with an increasing number of dimer structures for the pacemaker tests. These tests were done to verify that the linear ACE potentials is very much capable of learning the correct interaction curve if sufficient data is provided. We show that once around 10% of the dataset is composed of dimer structures, the potential is able to approach the DFT data. We also added an additional potential trained on the dataset with 50 dimers added but initialized with a different seed to ensure that initialization effects are minimal.

Dimer curves of potentials trained from the mixture of the 1000 structures from the diverse dataset with an increasing number of structures from the dimer dataset

Figure S14. Dimer curves of potentials trained from the mixture of the 1000 structures from the diverse dataset with an increasing number of structures from the dimer dataset. An additional potential was also trained on the dataset with 50 dimers added but initialized with a different seed to check the initialization effects. Prominent oscillations at short-range are due to absence of bond pairs in structures in the dataset at the sub-1 Å distance region.

Same tests were done for self-interacting and purified potentials for ACEpotentials.jl in Figure S15.

Dimer curves for potentials trained with dimers for carbon

Figure S15. Dimer curves for potentials trained with dimers for carbon. For interpolative tasks we see that linear ACE potentials are able to capture the data well.

Learning curves of diverse dataset runs

Figure S16 and Figure S17 show the learning rates for the diverse dataset with 1000 and 50000 structures, respectively. The different colors indicate the descriptor sets of varying complexity and body-order as controlled by KK and NfN_f. In general, we see that the potentials trained by the dataset with fewer structures approach the overfitting regions much faster and indicate that less complex descriptor sets, which have less parameters, are less susceptible to overfitting. On the other hand, the potentials trained with 50 times the data as compared previously still haven’t overfit. Both learning curves have losses around the same order of magnitude, which indicates that the larger dataset benefits from the more complex descriptor sets6.

Learning rates for the diverse dataset with 1000 structures

Figure S16. Learning rates for (upper left) energy, (lower left) force, and (right) total for the diverse dataset with 1000 structures. Different colors indicate different descriptor sets with varying KK and NfN_f.

Learning rates for the diverse dataset with 50000 structures

Figure S17. Learning rates for (upper left) energy, (lower left) force, and (right) total for the diverse dataset with 50000 structures.

Learning rates for the diverse dataset with 5000 structures

Figure S18. Learning rates for (upper left) energy, (lower left) force, and (right) total for the diverse dataset with 5000 structures. Different colors indicate different descriptor sets with varying KK and NfN_f.

Learning rates for the diverse dataset with 10000 structures

Figure S19. Learning rates for (upper left) energy, (lower left) force, and (right) total for the diverse dataset with 10000 structures.

Learning curves and dimer curves for 5k and 10k runs

Figs. S18 and S19 show the learning rates for the diverse dataset with 5000 and 10000 structures, respectively. The different colors indicate the descriptor sets of varying complexity and body-order as controlled by KK and NfN_f. Less prominent overfitting is observed in the 5000 dataset as compared to the learning curve of the potentials trained on the 1000-structure dataset. The learning curves of the 10000 dataset do not show any signs of overfitting after the runs were completed. Their corresponding dimer curves in Figure S20 and S21 show similar behavior from the potentials trained from the 1000 and 50000 structures in that the dimer curves do not approach a defined curve as higher KK basis functions are introduced in the descriptor set.

Dimer curves for the diverse dataset with 5000 structures

Figure S20. Dimer curves for the diverse dataset with 5000 structures. Different colors indicate different descriptor sets with varying KK and NfN_f.

Dimer curves for the diverse dataset with 10000 structures

Figure S21. Dimer curves for the diverse dataset with 10000 structures. Different colors indicate different descriptor sets with varying KK and NfN_f.

Pair distribution comparison of 1k and 50k diverse Carbon datasets

The pair distributions for the 1k and the 50k-structure datasets in Figure S22 are practically identical, with a lack of distances a 2.0 Å. The region may be supplemented by more varied structures (nn-mers, high-pressure structures) as done in Qamar et al. [20], but a separate investigation must be done for this dataset.

Pair distribution comparisons between the 1000-structure dataset and the complete 50000-structure dataset

Figure S22. Pair distribution comparisons between the 1000-structure dataset and the complete 50000-structure dataset from Shaidu et al. [25].

Model weights and average features of in-domain tests

The mean absolute model weights, cc, with respect to the features corresponding to each body-order KK are tabulated in Table III and IV for the purified and self-interacting bases, respectively. These results verify that potentials with purified bases have negligible coefficients for features KK that have orders higher than the maximum number of atoms in the structures of the training set. Potentials with the original ACE bases will have higher order KK features that are relevant due to the spurious self-interactions masked as lower-order terms. The peculiar case of the Si2_2-trained potentials stem from the fact that the K=2K = 2 functions used (24 functions in the chosen basis) are already sufficient in capturing the pair interactions of the training set which only contains dimers. This applies to both purified and self-interacting bases, making the self-interacting basis functions practically unnecessary in capturing the correct two-body energetics.

TrainsetBasisK=2K = 2K=3K = 3K=4K = 4K=5K = 5
Si2_224.202.024<10−8< 10^{-8}
24.20.162.024<10−8< 10^{-8}<10−8< 10^{-8}
24.20.16.122.024<10−8< 10^{-8}<10−8< 10^{-8}<10−8< 10^{-8}
Si23_{23}24.201.850.392
24.20.161.850.392<10−8< 10^{-8}
24.20.16.121.850.392<10−8< 10^{-8}<10−8< 10^{-8}
Si234_{234}24.201.5690.421
24.20.161.7160.2960.495
24.20.16.121.7160.2960.495<10−8< 10^{-8}
Si2345_{2345}24.201.4290.315
24.20.161.4200.2910.434
24.20.16.121.3700.3140.4820.574

Table 3. Mean absolute model weights for the features each KK of the in-domain tests for the purified bases

TrainsetBasisK=2K = 2K=3K = 3K=4K = 4K=5K = 5
Si2_224.202.024<10−8< 10^{-8}
24.20.162.024<10−8< 10^{-8}<10−8< 10^{-8}
24.20.16.122.024<10−8< 10^{-8}<10−8< 10^{-8}<10−8< 10^{-8}
Si23_{23}24.201.850.392
24.20.161.840.3300.146
24.20.16.121.850.3290.1420.0779
Si234_{234}24.201.5690.421
24.20.161.7140.3590.485
24.20.16.121.7080.3330.4310.315
Si2345_{2345}24.201.4290.315
24.20.161.4100.2800.372
24.20.16.121.3550.3710.4970.521

Table 4. Mean absolute model weights for the features each KK of the in-domain tests for the self-interacting bases

References

  1. [1]R. Drautz, Atomic cluster expansion for accurate and transferable interatomic potentials, Physical Review B 99, 014104 (2019).DOI
  2. [2]J. Sanchez, F. Ducastelle, and D. Gratias, Generalized cluster description of multicomponent systems, Physica A: Statistical Mechanics and its Applications 128, 334 (1984).DOI
  3. [3]L. G. Ferreira, S.-H. Wei, and A. Zunger, First-principles calculation of alloy phase diagrams: The renormalized-interaction approach, Physical Review B 40, 3197 (1989).pubmed.ncbi.nlm.nih.gov/9992258
  4. [4]F. H. Stillinger and T. A. Weber, Computer simulation of local order in condensed phases of silicon, Physical Review B 31, 5262 (1985).pubmed.ncbi.nlm.nih.gov/9936488
  5. [5]R. Biswas and D. R. Hamann, New classical models for silicon structural energies, Physical Review B 36, 6434 (1987).pubmed.ncbi.nlm.nih.gov/9942353
  6. [6]J. Tersoff, Modeling solid-state chemistry: Interatomic potentials for multicomponent systems, Physical Review B 39, 5566 (1989).pubmed.ncbi.nlm.nih.gov/9948964
  7. [7]G. Dusson, M. Bachmayr, G. Csányi, R. Drautz, S. Etter, C. Van Der Oord, and C. Ortner, Atomic cluster expansion: Completeness, efficiency and stability, Journal of Computational Physics 454, 110946 (2022).arxiv.org/abs/1911.03550
  8. [8]R. Drautz, M. Fähnle, and J. M. Sanchez, General relations between many-body potentials and cluster expansions in multicomponent systems, Journal of Physics: Condensed Matter 16, 3843 (2004).DOI
  9. [9]J. Behler and M. Parrinello, Generalized Neural-Network Representation of High-Dimensional Potential-Energy Surfaces, Physical Review Letters 98, 14601 (2007).pubmed.ncbi.nlm.nih.gov/17501293
  10. [10]A. P. Bartók, R. Kondor, and G. Csányi, On representing chemical environments, Physical Review B 87, 184115 (2013).DOI
  11. [11]L. Zhang, J. Han, H. Wang, R. Car, and W. E, Deep Potential Molecular Dynamics: A Scalable Model with the Accuracy of Quantum Mechanics, Physical Review Letters 120, 143001 (2018).DOI
  12. [12]R. Lot, F. Pellegrini, Y. Shaidu, and E. Küçükbenli, PANNA: Properties from Artificial Neural Network Architectures, Computer Physics Communications 256, 107402 (2020).
  13. [13]A. V. Shapeev, Moment Tensor Potentials: A class of systematically improvable interatomic potentials, Multiscale Modeling & Simulation 14, 1153 (2016), arXiv:1512.06054 [cond-mat, physics:physics].DOI
  14. [14]I. Batatia, S. Batizner, D. P. Kovács, A. Musaelian, G. N. C. Simm, R. Drautz, C. Ortner, B. Kozinsky, and G. Csányi, The design space of E(3)-equivariant atom-centred interatomic potentials, Nature Machine Intelligence 7, 56 (2025).DOI
  15. [15]I. Batatia, D. P. Kovacs, G. Simm, C. Ortner, and G. Csanyi, MACE: Higher order equivariant message passing neural networks for fast and accurate force fields, in Advances in Neural Information Processing Systems, Vol. 35, edited by S. Koyejo, S. Mohamed, A. Agarwal, D. Belgrave, K. Cho, and A. Oh (Curran Associates, Inc., 2022) pp. 11423–11436.
  16. [16]F. Musil, A. Grisafi, A. P. Bartók, C. Ortner, G. Csányi, and M. Ceriotti, Physics-Inspired Structural Representations for Molecules and Materials, Chemical Reviews 121, 9759 (2021).DOI
  17. [17]D. P. Kovács, C. V. D. Oord, J. Kucera, A. E. A. Allen, D. J. Cole, C. Ortner, and G. Csányi, Linear Atomic Cluster Expansion Force Fields for Organic Molecules: Beyond RMSE, Journal of Chemical Theory and Computation 17, 7696 (2021).
  18. [18]Y. Lysogorskiy, C. V. D. Oord, A. Bochkarev, S. Menon, M. Rinaldi, T. Hammerschmidt, M. Mrovec, A. Thompson, G. Csányi, C. Ortner, and R. Drautz, Performant implementation of the atomic cluster expansion (PACE) and application to copper and silicon, npj Computational Materials 7, 97 (2021).
  19. [19]A. Bochkarev, Y. Lysogorskiy, S. Menon, M. Qamar, M. Mrovec, and R. Drautz, Efficient parametrization of the atomic cluster expansion, Physical Review Materials 6, 013804 (2022).DOI
  20. [20]M. Qamar, M. Mrovec, Y. Lysogorskiy, A. Bochkarev, and R. Drautz, Atomic Cluster Expansion for Quantum-Accurate Large-Scale Simulations of Carbon, Journal of Chemical Theory and Computation 19, 5151 (2023).DOI
  21. [21]W. C. Witt, C. Van Der Oord, E. Gelzinyte, T. Järvinen, A. Ross, J. P. Darby, C. H. Ho, W. J. Baldwin, M. Sachs, J. Kermode, N. Bernstein, G. Csányi, and C. Ortner, ACEpotentials.jl: A Julia implementation of the atomic cluster expansion, The Journal of Chemical Physics 159, 164101 (2023).
  22. [22]C. H. Ho, T. S. Gutleb, and C. Ortner, Atomic cluster expansion without self-interaction, Journal of Computational Physics 515, 113271 (2024).
  23. [23]S. Chong, F. Bigi, F. Grasselli, P. Loche, M. Kellner, and M. Ceriotti, Prediction rigidities for data-driven chemistry, Faraday Discussions 256, 322 (2025).DOI
  24. [24]I. Batatia, P. Benner, Y. Chiang, A. M. Elena, D. P. Kovács, J. Riebesell, X. R. Advincula, M. Asta, M. Avaylon, W. J. Baldwin, and et al., A foundation model for atomistic materials chemistry. arXiv, March 01, 2024. DOI: 10.48550/arXiv:2401.00096v2. (2024), arXiv:2401.00096 [cond-mat, physics:physics].arxiv.org/abs/2401.00096
  25. [25]Y. Shaidu, E. Küçükbenli, R. Lot, F. Pellegrini, E. Kaxiras, and S. De Gironcoli, A systematic approach to generating accurate neural network potentials: The case of carbon, npj Computational Materials 7, 52 (2021).
  26. [26]A. R. Oganov and M. Valle, How to quantify energy landscapes of solids, The Journal of Chemical Physics 130, 104504 (2009).DOI
  27. [27]Y. Eldar, M. Lindenbaum, M. Porat, and Y. Zeevi, The Farthest Point Strategy for Progressive Image Sampling, IEEE Transactions on Image Processing 6, 1305 (1997).
  28. [28]P. Giannozzi, S. Baroni, N. Bonini, M. Calandra, R. Car, C. Cavazzoni, D. Ceresoli, G. L. Chiarotti, M. Cococcioni, I. Dabo, and et al., QUANTUM ESPRESSO: A modular and open-source software project for quantum simulations of materials, Journal of Physics: Condensed Matter 21, 395502 (2009).DOI
  29. [29]R. Sabatini, T. Gorni, and S. De Gironcoli, Nonlocal van der Waals density functional made simple and efficient, Physical Review B 87, 041108 (2013).DOI
  30. [30]A. Hjorth Larsen, J. Jørgen Mortensen, J. Blomqvist, I. E. Castelli, R. Christensen, M. Dułak, J. Friis, M. N. Groves, B. Hammer, C. Hargus, and et al., The atomic simulation environment—a Python library for working with atoms, Journal of Physics: Condensed Matter 29, 273002 (2017).DOI
  31. [31]F. Pellegrini, R. Lot, Y. Shaidu, and E. Küçükbenli, PANNA 2.0: Efficient neural network interatomic potentials and new architectures, The Journal of Chemical Physics 159, 084117 (2023).arxiv.org/abs/2305.11805
  32. [32]S. Batzner, A. Musaelian, L. Sun, M. Geiger, J. P. Mailoa, M. Kornbluth, N. Molinari, T. E. Smidt, and B. Kozinsky, E(3)-equivariant graph neural networks for data-efficient and accurate interatomic potentials, Nature Communications 13, 2453 (2022).DOI

Paper details

vlnc:W_7g3q5e6t
Version
v1 · latest
Published
Fresh — added to the Hub within the last 30 days
Added
License
CC BY 4.0
Usage

views · 1

downloads · 0

A view counts once per browser per day, a download each time the PDF is requested.

  • Physics and Astronomy

Not recorded for this paper

View statement
Looking for related papers…
Contents