Introduction
One of the central challenges of density functional theory (DFT) research is to design accurate approximations to the exchange-correlation (XC) functional [1 ] . As the one unknown term in Kohn-Sham DFT [2 ] , errors in the XC functional limit the predictive power of DFT for a wide variety of technological problems, including catalysis [3 ] , battery materials [4 ] , and semiconductor physics [5 , 6 ] . Increasing the complexity of the XC functional can improve its accuracy, but often at drastically increased computational cost, which can be prohibitive for large condensed matter simulations. In this work, we use machine learning (ML) to overcome this cost-accuracy trade-off by training a functional that obeys exact physical constraints and is accurate, transferable across a broad range of chemistries, and scalable to hundreds of atoms.
The key cost-accuracy trade-off we address is that between semilocal and hybrid XC functionals. Semilocal functionals, which include the local density approximation (LDA), generalized gradient approximation (GGA), and meta-generalized gradient approximation (meta-GGA) [7 ] , are computationally efficient but insufficiently accurate for many applications. For example, LDA and GGA functionals are incapable of correctly predicting band gaps due to their lack of derivative discontinuity [8 , 9 ] , and they are of limited accuracy for many molecular systems [1 , 10 ] . Meta-GGAs introduce derivative discontinuities into the XC functional and can therefore improve band gap predictions, but they typically account for only 20-50% of the GGA band gap error [11]. Meta-GGAs can be designed to further improve band gap prediction [12, 13], but modifying a given meta-GGA form to predict larger, more accurate band gaps typically reduces the accuracy of that functional for other chemical and material properties, such as bond dissociation energies [13, 14]. On the other hand, hybrid functionals, which mix a fraction of the nonlocal, orbital-dependent exact exchange energy into an XC functional [15], provide significantly improved predictions of band gaps [11, 16] and thermochemical properties [1 , 10 ] over semilocal DFT, but this accuracy comes at a steep additional computational cost, especially within plane-wave DFT. In spite of the remarkable recent progress of efficient implementations for solids [17–22], hybrid DFT is not as efficient or practical as semilocal DFT for these calculations.
Several approaches to design improved XC functionals have been pursued in the literature. Many popular functionals, like PBE [23] and SCAN [24], are designed primarily or entirely based on mathematical constraints obeyed by the exact functional. Constraint-based functionals are often considered trustworthy because they can predict molecular properties without being explicitly tuned to match them, reducing the risk of bias toward a particular property or set of systems [25, 26]. However, some of the most popular and accurate XC functionals for main-group chemistry are empirically fit to chemical data [27, 28]. Recently, significant progress has been made toward combining constraint and data-based design approaches using machine learning [29–35], culminating in the development of the DM21 functional [36], which achieves state-of-the-art accuracy on main-group chemistry benchmarks and also improves the description of static correlation by fitting fractional charge constraints.
While promising, these developments have a few key shortcomings, the most notable of which is their computational cost. DM21 uses a local range-separated hybrid formalism, which is both computationally expensive (even more so than global hybrid DFT) and not widely available for the plane-wave DFT codes typically used for investigating large extended systems. As a result, DM21 has not been applied to solids. To the best of our knowledge, no nonlocal machine-learned functional has yet been implemented for both molecular (Gaussian-type orbital) and extended (plane-wave) DFT calculations, with the exception of the DeePKS approach [37, 38]. However, DeePKS uses geometry-dependent (rather than universal) functionals and is trained for specific systems, so it does not fulfill the role of a broadly chemically transferable model. All other ML XC functionals that have been applied to solids have been limited to the semilocal form [13, 32, 33, 39–41] (with the exception of minimally parameterized van der Waals functionals and one concurrent work by Riemelmoser et al. [42] that introduced a nonlocal functional for C, H, and O-containing systems). This limitation restricts the application space of ML functionals by preventing their use in applications where hybrid DFT accuracy is needed. For example, hybrid DFT has proven useful for studying problems in catalysis within both Gaussian-type orbital and plane-wave codes [43, 44] and for addressing the “band-gap problem” in DFT [45]. In addition to this technical limitation, applications of ML functionals to complex and heterogeneous systems are limited by their lack of transferability and universality, as they have previously only been trained on molecular systems.
These limitations raise the question of whether machine learning models can leverage more efficient features to balance accuracy and computational cost. One possible approach in this direction is to generate nonlocal features by taking integrals of the density over some set of kernel functions [30, 46]. This way, an ML model of the XC functional can capture nonlocal information about the density without evaluating the computationally demanding exact exchange operator required for hybrid DFT. In line with this strategy, we recently introduced the Compressed Scale-Invariant Density Representation (CIDER) formalism for learning the exchange functional [35], in which the nonlocal features were designed to enforce the uniform scaling rule [47] on the learned exchange functional. We demonstrated that the exchange functional can be accurately learned with the CIDER approach. However, the high computational cost of computing the features, lack of implementation for plane-wave DFT, and need to compute and fit the exchange energy density (rather than the total exchange energy) limited the practicality of our model and restricted its application to small molecules.
In this work, we overcome all of these limitations by revising the form of the features, implementing efficient algorithms to evaluate the new features for both molecular and extended systems, and redesigning our Gaussian process model to fit the total exchange energy. These improvements enable the construction of the first nonlocal, universal ML exchange functional for solids, which can be applied to large systems with similar computational cost to semilocal DFT. We use this exchange functional to implement efficient and accurate hybrid DFT surrogates that can be used in a variety of applications. Because our new functional form is both transferable and numerically stable, we believe that our approach will also be useful for learning the full XC functional, allowing us to address problems where DFT accuracy is limited by both exchange and correlation effects.
By training and testing exchange functionals with different feature sets, we find that to learn the exchange functional well enough to reproduce hybrid DFT accuracy, it is necessary to include both a semilocal orbital-dependent feature (i.e. the kinetic energy density) and nonlocal density-dependent features in the model input. Having developed this “nonlocal meta-GGA,” we show that it accurately predicts molecular properties and improves solid-state band gap predictions over standard meta-GGAs, which is a central problem in DFT. The performance for solid-state band gaps is particularly promising because only 9 systems in the training set are solids, with the others being isolated molecules and complexes. We also encounter only a few convergence problems in our self-consistent field (SCF) calculations of 2462 molecular and 453 periodic systems, indicating that the functional has good numerical stability. To demonstrate the transferability and efficiency of the model, we simulate systems with over 500 atoms to predict charged defect transition levels in silicon.
The paper is organized as follows. Section II describes the features and models used in this work. Section III presents our results and discusses the accuracy of the CIDER functionals as well as their computational efficiency within plane-wave DFT. Section IV contains our conclusion. Appendix A provides a detailed description of the computational methods used in this study, and Appendix B provides additional mathematical details for the plane-wave implementation of CIDER.
Theory
In Kohn-Sham DFT [2 ] , it is often convenient to separate the XC energy into exchange and correlation parts, i.e. E x c [ n ] = E x [ n ] + E c [ n ] E_{\rm xc}[n] = E_{\rm x}[n] + E_{\rm c}[n] E xc [ n ] = E x [ n ] + E c [ n ] . The exchange energy E x [ n ] E_{\rm x}[n] E x [ n ] accounts for the antisymmetric nature of the fermionic wave function, while the correlation energy E c [ n ] E_{\rm c}[n] E c [ n ] accounts for additional many-body correlation effects between the electrons. The exchange energy is the main focus of this work and can be trivially generalized from non-spin-polarized DFT to spin-polarized DFT as [48]
E x [ n ↑ , n ↓ ] = 1 2 E x [ 2 n ↑ ] + 1 2 E x [ 2 n ↓ ] , (1) E_{\mathrm{x}}[n_{\uparrow},n_{\downarrow}] = \frac{1}{2}E_{\mathrm{x}}[2n_{\uparrow}] + \frac{1}{2}E_{\mathrm{x}}[2n_{\downarrow}], \tag*{(1)} E x [ n ↑ , n ↓ ] = 2 1 E x [ 2 n ↑ ] + 2 1 E x [ 2 n ↓ ] , (1)
so all formulas in the theory section are written for the non-spin-polarized case for simplicity.
The exact exchange energy can be expressed in terms of the occupied Kohn-Sham orbitals { ϕ i ( r ) } \{\phi_i(\mathbf{r})\} { ϕ i ( r )} as
E x e x a c t [ n ] = − ∑ i j ∫ d 3 r ∫ d 3 r ′ ϕ i ∗ ( r ) ϕ j ( r ) ϕ i ( r ′ ) ϕ j ∗ ( r ′ ) ∣ r − r ′ ∣ . (2) E_{\mathrm{x}}^{\mathrm{exact}}[n] = -\sum_{ij}\int\mathrm{d}^{3}r \int\mathrm{d}^{3}r' \frac{\phi_i^*(\mathbf{r})\phi_j(\mathbf{r})\phi_i(\mathbf{r}')\phi_j^*(\mathbf{r}')}{\lvert\mathbf{r}-\mathbf{r}'\rvert}. \tag*{(2)} E x exact [ n ] = − ij ∑ ∫ d 3 r ∫ d 3 r ′ ∣ r − r ′ ∣ ϕ i ∗ ( r ) ϕ j ( r ) ϕ i ( r ′ ) ϕ j ∗ ( r ′ ) . (2)
Evaluating Eq. (2) is computationally expensive. However, the improved accuracy of hybrid DFT arises from mixing a fraction of E x e x a c t [ n ] E_{\mathrm{x}}^{\mathrm{exact}}[n] E x exact [ n ] into an otherwise semilocal XC functional. These considerations explain the cost-accuracy trade-off between semilocal and hybrid DFT. Our goal is to fit Eq. (2) as accurately as possible with a computationally efficient surrogate model.
The remainder of this section sets up our machine learning-based approach to this problem. Specifically, Sections II A-II C cover the design of the exchange functional form, nonlocal features, and Gaussian process models, and Sections II D-II F cover the computationally efficient implementation of the nonlocal features. The feature implementation requires different algorithms for Gaussian-type orbital and plane-wave DFT codes due to the use of different basis sets and integration grids within these codes.
The simplest approximate exchange functional is the local density approximation (LDA), in which the exchange energy is written as an integral over a function of the density n ( r ) n(\mathbf{r}) n ( r ) at each point in space
E x [ n ] = ∫ d 3 r e x L D A ( n ( r ) ) , (3) E_{\mathrm{x}}[n] = \int\mathrm{d}^{3}r\,e_{\mathrm{x}}^{\mathrm{LDA}}(n(\mathbf{r})), \tag*{(3)} E x [ n ] = ∫ d 3 r e x LDA ( n ( r )) , (3)
where in the Kohn-Sham formalism [2 ] , the density is expressed in terms of the occupied orbitals as
n ( r ) = ∑ i ∣ ϕ i ( r ) ∣ 2 . (4) n(\mathbf{r}) = \sum_i \lvert\phi_i(\mathbf{r})\rvert^2. \tag*{(4)} n ( r ) = i ∑ ∣ ϕ i ( r ) ∣ 2 . (4)
Typically, e x L D A ( n ) e_{\mathrm{x}}^{\mathrm{LDA}}(n) e x LDA ( n ) is defined as the exchange energy density of the uniform electron gas [49]
e x L D A ( n ) = − 3 4 ( 3 π ) 1 / 3 n 4 / 3 . (5) e_{\mathrm{x}}^{\mathrm{LDA}}(n) = -\frac{3}{4}\left(\frac{3}{\pi}\right)^{1/3}n^{4/3}. \tag*{(5)} e x LDA ( n ) = − 4 3 ( π 3 ) 1/3 n 4/3 . (5)
In addition to making Eq. (3) exact for the uniform electron gas, Eq. (5) also satisfies an important property of the exchange functional known as uniform scaling [47, 50], which states that for a scaled density distribution n λ ( r ) = λ 3 n ( λ r ) n_{\lambda}(\mathbf{r}) = \lambda^3 n(\lambda\mathbf{r}) n λ ( r ) = λ 3 n ( λ r ) ,
E x [ n λ ] = λ E x [ n ] , (6) E_{\mathrm{x}}[n_{\lambda}] = \lambda E_{\mathrm{x}}[n], \tag*{(6)} E x [ n λ ] = λ E x [ n ] , (6)
where λ \lambda λ is a positive scalar.
More sophisticated functionals that obey Eq. (6) can be designed by writing the exchange energy density as the product of Eq. (5) and an exchange enhancement factor F x F_{\mathrm{x}} F x :
E x [ n ] = ∫ d 3 r e x L D A ( n ( r ) ) F x ( x ( r ) ) . (7) E_{\mathrm{x}}[n] = \int\mathrm{d}^{3}r\,e_{\mathrm{x}}^{\mathrm{LDA}}(n(\mathbf{r}))F_{\mathrm{x}}(\mathbf{x}(\mathbf{r})). \tag*{(7)} E x [ n ] = ∫ d 3 r e x LDA ( n ( r )) F x ( x ( r )) . (7)
In the above equation, x ( r ) \mathbf{x}(\mathbf{r}) x ( r ) is a vector of scale-invariant features x i x_i x i , where scale-invariance is defined as
x i [ n λ ] ( r ) = x i [ n ] ( λ r ) . (8) x_i[n_{\lambda}](\mathbf{r}) = x_i[n](\lambda\mathbf{r}). \tag*{(8)} x i [ n λ ] ( r ) = x i [ n ] ( λ r ) . (8)
For example, a generalized gradient approximation (GGA) exchange functional is constructed by making F x F_{\mathrm{x}} F x a function of the reduced gradient s s s of the density
s = ∣ ∇ n ∣ 2 ( 3 π 2 ) 1 / 3 n 4 / 3 . (9) s = \frac{\lvert\nabla n\rvert}{2(3\pi^2)^{1/3}n^{4/3}}. \tag*{(9)} s = 2 ( 3 π 2 ) 1/3 n 4/3 ∣ ∇ n ∣ . (9)
Because s s s satisfies Eq. (8) , the GGA form
E x [ n ] = ∫ d 3 r e x L D A ( n ( r ) ) F x ( s ( r ) ) (10) E_{\mathrm{x}}[n] = \int\mathrm{d}^{3}r\,e_{\mathrm{x}}^{\mathrm{LDA}}(n(\mathbf{r}))F_{\mathrm{x}}(s(\mathbf{r})) \tag*{(10)} E x [ n ] = ∫ d 3 r e x LDA ( n ( r )) F x ( s ( r )) (10)
satisfies Eq. (6) . Likewise, a meta-GGA can be constructed by making the exchange enhancement factor a function of two ingredients F x ( s , α ) F_{\mathrm{x}}(s,\alpha) F x ( s , α ) , where
α = τ − τ W τ 0 (11) \alpha= \frac{\tau-\tau_{\mathrm{W}}}{\tau_0} \tag*{(11)} α = τ 0 τ − τ W (11)
is a scale-invariant quantity that depends on the kinetic energy density
τ ( r ) = 1 2 ∑ i ∣ ∇ ϕ i ( r ) ∣ 2 , (12) \tau(\mathbf{r}) = \frac{1}{2}\sum_i \lvert\nabla\phi_i(\mathbf{r})\rvert^2, \tag*{(12)} τ ( r ) = 2 1 i ∑ ∣ ∇ ϕ i ( r ) ∣ 2 , (12)
and τ W = ∣ ∇ n ∣ 2 8 n \tau_{\mathrm{W}} = \frac{\lvert\nabla n\rvert^2}{8n} τ W = 8 n ∣ ∇ n ∣ 2 and τ 0 = 3 10 ( 3 π 2 ) 2 / 3 n 5 / 3 \tau_0 = \frac{3}{10}(3\pi^2)^{2/3}n^{5/3} τ 0 = 10 3 ( 3 π 2 ) 2/3 n 5/3 are the single-orbital and uniform electron gas kinetic energies, respectively.
Updated Nonlocal Features for CIDER
In our initial construction of the CIDER formalism [35], the exchange functional is defined as
E x [ n ] = ∫ d 3 r e x L D A ( n ( r ) ) × F x ( s ( r ) , α ( r ) , { G n l m ( r ) } ) . (13) \begin{aligned}
E_{\mathrm{x}}[n] &= \int\mathrm{d}^{3}r\,e_{\mathrm{x}}^{\mathrm{LDA}}(n(\mathbf{r}))\\
&\quad\times F_{\mathrm{x}}(s(\mathbf{r}),\alpha(\mathbf{r}),\{G_{nlm}(\mathbf{r})\}). \tag*{(13)}
\end{aligned} E x [ n ] = ∫ d 3 r e x LDA ( n ( r )) × F x ( s ( r ) , α ( r ) , { G n l m ( r )}) . (13)
In Eq. (13) , the functions { G n l m ( r ) } \{G_{nlm}(\mathbf{r})\} { G n l m ( r )} are nonlocal features of the density defined by
G n l m ( r ) = ∫ d 3 r ′ g n l m ( r ′ ; r ) n ( r + r ′ ) , (14) G_{nlm}(\mathbf{r}) = \int\mathrm{d}^{3}r'\,g_{nlm}(\mathbf{r}';\mathbf{r})n(\mathbf{r}+\mathbf{r}'), \tag*{(14)} G n l m ( r ) = ∫ d 3 r ′ g n l m ( r ′ ; r ) n ( r + r ′ ) , (14)
where g n l m ( r ′ ; r ) g_{nlm}(\mathbf{r}';\mathbf{r}) g n l m ( r ′ ; r ) is the kernel function
g n l m ( r ′ ; r ) = M Y l m ( r ^ ′ ) ( a ( r ) ∣ r ′ ∣ 2 ) n + l / 2 e − a ( r ) ∣ r ′ ∣ 2 , (15) g_{nlm}(\mathbf{r}';\mathbf{r}) = M Y_{lm}(\hat{\mathbf{r}}')\left(a(\mathbf{r})\lvert\mathbf{r}'\rvert^2\right)^{n+l/2}e^{-a(\mathbf{r})\lvert\mathbf{r}'\rvert^2}, \tag*{(15)} g n l m ( r ′ ; r ) = M Y l m ( r ^ ′ ) ( a ( r ) ∣ r ′ ∣ 2 ) n + l /2 e − a ( r ) ∣ r ′ ∣ 2 , (15)
with M l = B 0 3 / 2 4 π − 1 ( 8 π 3 ) 1 / 3 M_l = B_0^{3/2}\sqrt{4\pi^{-1}\left(\frac{8\pi}{3}\right)^{1/3}} M l = B 0 3/2 4 π − 1 ( 3 8 π ) 1/3 . The position-dependent exponent a ( r ) a(r) a ( r ) is defined as a semilocal quantity
a [ n ] ( r ) = π ( n ( r ) 2 ) 2 / 3 [ B 0 + C 0 ( τ ( r ) τ 0 ( r ) − 1 ) ] , (16) a[n](\mathbf{r})=\pi\left(\frac{n(\mathbf{r})}{2}\right)^{2/3}\left[B_0+C_0\left(\frac{\tau(\mathbf{r})}{\tau_0(\mathbf{r})}-1\right)\right], \tag*{(16)} a [ n ] ( r ) = π ( 2 n ( r ) ) 2/3 [ B 0 + C 0 ( τ 0 ( r ) τ ( r ) − 1 ) ] , (16)
where B 0 B_0 B 0 and C 0 C_0 C 0 are adjustable constants that control how quickly the width of the Gaussian-type function in Eq. (15) decreases as the density and kinetic energy density increase. Equation 16 behaves quadratically under uniform scaling, i.e.
a [ n λ ] ( r ) = λ 2 a [ n ] ( λ r ) . (17) a[n_\lambda](\mathbf{r})=\lambda^2a[n](\lambda\mathbf{r}). \tag*{(17)} a [ n λ ] ( r ) = λ 2 a [ n ] ( λ r ) . (17)
This property ensures that G n l m ( r ) G_{nlm}(\mathbf{r}) G n l m ( r ) is scale-invariant (Eq. (8) ) and therefore that Eq. (13) satisfies the uniform scaling rule (Eq. (6) ).
Using these scale-invariant nonlocal features, we were able to train an ML exchange functional that accurately reproduced hybrid DFT atomization energies [35], but there is room for improvement in making the features more physically intuitive and easier to implement with production-level performance. In this work, we simplify the features by removing the spherical harmonics and instead adding a second position-dependent exponent to the kernel function, resulting in the new features
G i ( r 1 ) = ( B i + B 0 2 ) 3 / 2 ∫ d 3 r 2 Φ ( a ( r 2 ) , b i ( r 1 ) , r 12 ) n ( r 2 ) (18) G_i(\mathbf{r}_1)=\left(\frac{B_i+B_0}{2}\right)^{3/2}\int\mathrm{d}^3\mathbf{r}_2\,\Phi\left(a(\mathbf{r}_2),b_i(\mathbf{r}_1),r_{12}\right)n(\mathbf{r}_2) \tag*{(18)} G i ( r 1 ) = ( 2 B i + B 0 ) 3/2 ∫ d 3 r 2 Φ ( a ( r 2 ) , b i ( r 1 ) , r 12 ) n ( r 2 ) (18)
Φ ( a , b , r ) = exp [ − ( a + b ) r 2 ] , (19) \Phi(a,b,r)=\exp\left[-(a+b)r^2\right], \tag*{(19)} Φ ( a , b , r ) = exp [ − ( a + b ) r 2 ] , (19)
where r 12 = ∣ r 1 − r 2 ∣ r_{12}=|\mathbf{r}_1-\mathbf{r}_2| r 12 = ∣ r 1 − r 2 ∣ . Multiple features G 1 , G 2 , … G_1,G_2,\ldots G 1 , G 2 , … are generated by evaluating Eq. (18) with different choices of exponent b i ( r ) b_i(\mathbf{r}) b i ( r ) , which in turn are tuned by constants B i B_i B i and C i C_i C i as discussed below. Because the position-dependent exponents increase as the density increases, the a a a exponent effectively damps the value of the integrand when r 2 \mathbf{r}_2 r 2 is near the atomic core, which we intend to limit the interaction between the valence region and core region. The prefactor ( B i + B 0 2 ) 3 / 2 \left(\frac{B_i+B_0}{2}\right)^{3/2} ( 2 B i + B 0 ) 3/2 is chosen so that G i = 2 G_i=2 G i = 2 for the non-spin-polarized uniform electron gas.
To retain the scale-invariance of the features, any choice of a [ n ] ( r ) a[n](\mathbf{r}) a [ n ] ( r ) and b i [ n ] ( r ) b_i[n](\mathbf{r}) b i [ n ] ( r ) that satisfies Eq. (17) is sufficient. For example, the exponents can be an orbital-independent function of the density and its gradient
a [ n ] ( r ) = π ( n 2 ) 2 / 3 [ B 0 + C 0 ( ∣ ∇ n ∣ 2 8 n τ 0 ) ] (20) a[n](\mathbf{r})=\pi\left(\frac{n}{2}\right)^{2/3}\left[B_0+C_0\left(\frac{|\nabla n|^2}{8n\tau_0}\right)\right] \tag*{(20)} a [ n ] ( r ) = π ( 2 n ) 2/3 [ B 0 + C 0 ( 8 n τ 0 ∣∇ n ∣ 2 ) ] (20)
b i [ n ] ( r ) = π ( n 2 ) 2 / 3 [ B i + C i ( ∣ ∇ n ∣ 2 8 n τ 0 ) ] (21) b_i[n](\mathbf{r})=\pi\left(\frac{n}{2}\right)^{2/3}\left[B_i+C_i\left(\frac{|\nabla n|^2}{8n\tau_0}\right)\right] \tag*{(21)} b i [ n ] ( r ) = π ( 2 n ) 2/3 [ B i + C i ( 8 n τ 0 ∣∇ n ∣ 2 ) ] (21)
or can be made kinetic-energy dependent, as in our previous work:
a [ n ] ( r ) = π ( n 2 ) 2 / 3 [ B 0 + C 0 ( τ τ 0 − 1 ) ] (22) a[n](\mathbf{r})=\pi\left(\frac{n}{2}\right)^{2/3}\left[B_0+C_0\left(\frac{\tau}{\tau_0}-1\right)\right] \tag*{(22)} a [ n ] ( r ) = π ( 2 n ) 2/3 [ B 0 + C 0 ( τ 0 τ − 1 ) ] (22)
b i [ n ] ( r ) = π ( n 2 ) 2 / 3 [ B i + C i ( τ τ 0 − 1 ) ] . (23) b_i[n](\mathbf{r})=\pi\left(\frac{n}{2}\right)^{2/3}\left[B_i+C_i\left(\frac{\tau}{\tau_0}-1\right)\right]. \tag*{(23)} b i [ n ] ( r ) = π ( 2 n ) 2/3 [ B i + C i ( τ 0 τ − 1 ) ] . (23)
Equations 20 and 21 are used for the nonlocal GGA presented in this work (c.f. § 2 C), while Eqs. 22 and 23 are used for the nonlocal meta-GGA. Throughout Sections II D–II F, the ( B i + B 0 2 ) 3 / 2 \left(\frac{B_i+B_0}{2}\right)^{3/2} ( 2 B i + B 0 ) 3/2 factor and the feature index i i i will be dropped for simplicity, as the inclusion of the scaling factor and the introduction of additional features by introducing new B i B_i B i and C i C_i C i is trivial. The motivation for the form of Eqs. 20–23 is discussed in our previous work introducing CIDER [35], and the heuristic selection of the B i B_i B i and C i C_i C i coefficients is discussed in Appendix A8.
Before continuing, we briefly discuss the construction of the XC potential when nonlocal features are employed. A typical semilocal functional has a multiplicative XC potential
δ E x c δ n ( r ) = ∂ e x c ( r ) ∂ n ( r ) . (24) \frac{\delta E_{\mathrm{xc}}}{\delta n(\mathbf{r})}=\frac{\partial e_{\mathrm{xc}}(\mathbf{r})}{\partial n(\mathbf{r})}. \tag*{(24)} δ n ( r ) δ E xc = ∂ n ( r ) ∂ e xc ( r ) . (24)
A meta-GGA also has a non-multiplicative potential
δ E x c δ τ ( r ) = ∂ e x c ( r ) ∂ τ ( r ) . (25) \frac{\delta E_{\mathrm{xc}}}{\delta\tau(\mathbf{r})}=\frac{\partial e_{\mathrm{xc}}(\mathbf{r})}{\partial\tau(\mathbf{r})}. \tag*{(25)} δ τ ( r ) δ E xc = ∂ τ ( r ) ∂ e xc ( r ) . (25)
This above term is non-multiplicative because it depends on the orbitals through τ ( r ) \tau(\mathbf{r}) τ ( r ) , rather than depending exclusively on the density n ( r ) n(\mathbf{r}) n ( r ) . For a given basis set { χ μ ( r ) } \{\chi_\mu(\mathbf{r})\} { χ μ ( r )} , the contributions of the XC potential to the effective Kohn-Sham Hamiltonian matrix are
V μ ν x c = ∫ d 3 r χ μ ∗ ( r ) χ ν ( r ) δ E x c δ n ( r ) + 1 2 ∫ d 3 r ∇ χ μ ∗ ( r ) ⋅ ∇ χ ν ( r ) δ E x c δ τ ( r ) . (26) V_{\mu\nu}^{\mathrm{xc}}=\int\mathrm{d}^3\mathbf{r}\,\chi_\mu^*(\mathbf{r})\chi_\nu(\mathbf{r})\frac{\delta E_{\mathrm{xc}}}{\delta n(\mathbf{r})}
+\frac{1}{2}\int\mathrm{d}^3\mathbf{r}\,\nabla\chi_\mu^*(\mathbf{r})\cdot\nabla\chi_\nu(\mathbf{r})\frac{\delta E_{\mathrm{xc}}}{\delta\tau(\mathbf{r})}. \tag*{(26)} V μν xc = ∫ d 3 r χ μ ∗ ( r ) χ ν ( r ) δ n ( r ) δ E xc + 2 1 ∫ d 3 r ∇ χ μ ∗ ( r ) ⋅ ∇ χ ν ( r ) δ τ ( r ) δ E xc . (26)
When nonlocal features are included in the XC energy density, e.g.
E x c = ∫ d 3 r e x c ( n ( r ) , ∣ ∇ n ( r ) ∣ , τ ( r ) , G ( r ) ) , (27) E_{\mathrm{xc}}=\int\mathrm{d}^3\mathbf{r}\,e_{\mathrm{xc}}\left(n(\mathbf{r}),|\nabla n(\mathbf{r})|,\tau(\mathbf{r}),G(\mathbf{r})\right), \tag*{(27)} E xc = ∫ d 3 r e xc ( n ( r ) , ∣∇ n ( r ) ∣ , τ ( r ) , G ( r ) ) , (27)
both δ E x c δ n ( r ) \frac{\delta E_{\mathrm{xc}}}{\delta n(\mathbf{r})} δ n ( r ) δ E xc and δ E x c δ τ ( r ) \frac{\delta E_{\mathrm{xc}}}{\delta\tau(\mathbf{r})} δ τ ( r ) δ E xc include terms that depend on other coordinates r ′ \mathbf{r}' r ′ :
δ E x c δ n ( r ) = ∂ e x c ( r ) ∂ n ( r ) + ∂ e x c ( r ) ∂ G ( r ) ∂ G ( r ) ∂ n ( r ) + ∫ d 3 r ′ ∂ e x c ( r ′ ) ∂ G ( r ′ ) δ G ( r ′ ) δ n ( r ′ ) , (28) \frac{\delta E_{\mathrm{xc}}}{\delta n(\mathbf{r})}=\frac{\partial e_{\mathrm{xc}}(\mathbf{r})}{\partial n(\mathbf{r})}+\frac{\partial e_{\mathrm{xc}}(\mathbf{r})}{\partial G(\mathbf{r})}\frac{\partial G(\mathbf{r})}{\partial n(\mathbf{r})}
+\int\mathrm{d}^3\mathbf{r}'\,\frac{\partial e_{\mathrm{xc}}(\mathbf{r}')}{\partial G(\mathbf{r}')}\frac{\delta G(\mathbf{r}')}{\delta n(\mathbf{r}')}, \tag*{(28)} δ n ( r ) δ E xc = ∂ n ( r ) ∂ e xc ( r ) + ∂ G ( r ) ∂ e xc ( r ) ∂ n ( r ) ∂ G ( r ) + ∫ d 3 r ′ ∂ G ( r ′ ) ∂ e xc ( r ′ ) δ n ( r ′ ) δ G ( r ′ ) , (28)
δ E x c δ τ ( r ) = ∂ e x c ( r ) ∂ τ ( r ) + ∂ e x c ( r ) ∂ G ( r ) ∂ G ( r ) ∂ τ ( r ) + ∫ d 3 r ′ ∂ e x c ( r ′ ) ∂ G ( r ′ ) δ G ( r ′ ) δ τ ( r ′ ) . (29) \frac{\delta E_{\mathrm{xc}}}{\delta\tau(\mathbf{r})}=\frac{\partial e_{\mathrm{xc}}(\mathbf{r})}{\partial\tau(\mathbf{r})}+\frac{\partial e_{\mathrm{xc}}(\mathbf{r})}{\partial G(\mathbf{r})}\frac{\partial G(\mathbf{r})}{\partial\tau(\mathbf{r})}
+\int\mathrm{d}^3\mathbf{r}'\,\frac{\partial e_{\mathrm{xc}}(\mathbf{r}')}{\partial G(\mathbf{r}')}\frac{\delta G(\mathbf{r}')}{\delta\tau(\mathbf{r}')}. \tag*{(29)} δ τ ( r ) δ E xc = ∂ τ ( r ) ∂ e xc ( r ) + ∂ G ( r ) ∂ e xc ( r ) ∂ τ ( r ) ∂ G ( r ) + ∫ d 3 r ′ ∂ G ( r ′ ) ∂ e xc ( r ′ ) δ τ ( r ′ ) δ G ( r ′ ) . (29)
In Eq. (28) , ∂ G ( r ) ∂ n ( r ) \frac{\partial G(\mathbf{r})}{\partial n(\mathbf{r})} ∂ n ( r ) ∂ G ( r ) describes the local dependence of G ( r ) G(\mathbf{r}) G ( r ) on the density, and δ G ( r ′ ) δ n ( r ) \frac{\delta G(\mathbf{r}')}{\delta n(\mathbf{r})} δ n ( r ) δ G ( r ′ ) describes the nonlocal dependence on the density. While the expressions for δ E x c δ n ( r ) \frac{\delta E_{\mathrm{xc}}}{\delta n(\mathbf{r})} δ n ( r ) δ E xc and δ E x c δ τ ( r ) \frac{\delta E_{\mathrm{xc}}}{\delta\tau(\mathbf{r})} δ τ ( r ) δ E xc are more complicated than for a meta-GGA, they still contribute to the XC potential matrix exclusively via Eq. (26) . Therefore, any DFT code supporting meta-GGAs can evaluate this type of functional if a routine is added to evaluate the nonlocal contributions to Eqs. (28) and (29) .
Gaussian Process Models for the Exchange Functional
Overview
To study the impact of different features on the accuracy of an ML exchange functional, we implement four functional types that use different sets of features.
SL-GGA is a standard, semilocal GGA like PBE [23], except trained via machine learning. The exchange enhancement factor (F x F_{\mathrm{x}} F x in Eq. (7) ) is a function of s s s only.
NL-GGA is a nonlocal GGA. The exchange enhancement factor is a function of s s s and also three nonlocal features of the form in Eq. (18) , with Eqs. (20) and (21) for the exponents.
SL-MGGA is a standard, semilocal meta-GGA like SCAN [24], except trained via machine learning. The exchange enhancement factor is a function of s s s and α \alpha α .
NL-MGGA is a nonlocal meta-GGA. The exchange enhancement factor is a function of s s s , α \alpha α , and three nonlocal features of the form in Eq. (18) , with Eqs. (22) and (23) for the exponents.
In short, the SL-GGA and SL-MGGA are semilocal functionals, while the NL-GGA and NL-MGGA augment the semilocal functional form with nonlocal features of the density. As described below, Gaussian process models for the exchange energy are trained using each of these four feature sets. Section II C 2 explains how a Gaussian process can be fit to total exchange energies of systems, Section II C 3 lists the specific mathematical form of each feature vector that is used as input to the Gaussian processes, and Section II C 4 lists the covariance kernels for each feature set.
Gaussian process regression is a Bayesian machine learning method in which the values of the predictive function f ( x ) f(x) f ( x ) are treated as dependent random variables. The key ingredient in a Gaussian process is the covariance between values of the function Cov ( f ( x ) , f ( x ′ ) ) = k ( x , x ′ ) \operatorname{Cov}(f(x), f(x')) = k(x, x') Cov ( f ( x ) , f ( x ′ )) = k ( x , x ′ ) , where the user-defined covariance kernel function k ( x , x ′ ) k(x, x') k ( x , x ′ ) determines the relationship between the values of f f f for different inputs x x x . Suppose one has training data of the form ( x i , y i ) (x_i, y_i) ( x i , y i ) , where y i y_i y i is assumed to be the sum of f ( x i ) f(x_i) f ( x i ) and Gaussian-distributed random noise of variance σ n o i s e 2 \sigma_{\mathrm{noise}}^2 σ noise 2 . Then the predictive mean, or expected value, of the function f f f is given by
f ( x ∗ ) = ∑ i k ( x ∗ , x i ) α i . (30) f(x_*) = \sum_i k(x_*, x_i)\alpha_i. \tag*{(30)} f ( x ∗ ) = i ∑ k ( x ∗ , x i ) α i . (30)
with the weight vector α \boldsymbol{\alpha} α defined as
α = ( K + σ n o i s e 2 I ) − 1 y . (31) \boldsymbol{\alpha} = \left(\mathbf{K} + \sigma_{\mathrm{noise}}^2\mathbf{I}\right)^{-1}\mathbf{y}. \tag*{(31)} α = ( K + σ noise 2 I ) − 1 y . (31)
In the above equation K i j = k ( x i , x j ) K_{ij}=k(x_i,x_j) K ij = k ( x i , x j ) is called the covariance matrix, and I \mathbf{I} I is the identity matrix. The reader is referred to the textbook by Rasmussen and Williams [51] for a more thorough introduction to Gaussian processes. The following subsection introduces a modified Gaussian process regression approach for training exchange-correlation functionals.
Fitting the Total Exchange Energy
In this section, we introduce a method to train a Gaussian process to the total exchange energy, as opposed to the nonunique exchange energy density used in our previous work [35]. While we only learn the exchange energy in this work, the approach in this section also applies to the XC energy, so we write the formulas for E x c E_{\mathrm{xc}} E xc for the sake of generality. We index the chemical systems in the training set by i i i and j j j ; while the feature implementations are different for Gaussian-type orbital and plane-wave DFT, the role of systems computed within these two frameworks is identical with regard to model training. The XC energy of a system i i i is given by a numerical integral of the energy density
E x c i = ∑ g ∈ i w g i e x c ( x g i ) , (32) E_{\mathrm{xc}}^i = \sum_{g\in i} w_g^i e_{\mathrm{xc}}(x_g^i), \tag*{(32)} E xc i = g ∈ i ∑ w g i e xc ( x g i ) , (32)
where g g g indexes grid points, w g i w_g^i w g i are the quadrature weights, and x g i x_g^i x g i is the feature vector of molecule i i i at point g g g .
Constructing a Gaussian process for E x c E_{\mathrm{xc}} E xc requires defining the covariance between the XC energies for two different systems, i.e. Cov ( E x c i , E x c j ) \operatorname{Cov}(E_{\mathrm{xc}}^i,E_{\mathrm{xc}}^j) Cov ( E xc i , E xc j ) (c.f. Chapter 2.2 of Ref. [51]). The covariance of sums of random variables a a a , b b b , c c c , and d d d is
Cov ( a + b , c + d ) = Cov ( a , c ) + Cov ( b , c ) + Cov ( a , d ) + Cov ( b , d ) . (33) \operatorname{Cov}(a+b,c+d)=\operatorname{Cov}(a,c)+\operatorname{Cov}(b,c)+\operatorname{Cov}(a,d)+\operatorname{Cov}(b,d). \tag*{(33)} Cov ( a + b , c + d ) = Cov ( a , c ) + Cov ( b , c ) + Cov ( a , d ) + Cov ( b , d ) . (33)
Using Eq. (33) , Cov ( E x c i , E x c j ) \operatorname{Cov}(E_{\mathrm{xc}}^i,E_{\mathrm{xc}}^j) Cov ( E xc i , E xc j ) can be expressed in terms of the covariances between the exchange energy densities as
Cov ( E x c i , E x c j ) = ∑ g ∈ i ∑ h ∈ j w g i w h j Cov ( e x c ( x g i ) , e x c ( x h j ) ) (34) \operatorname{Cov}(E_{\mathrm{xc}}^i,E_{\mathrm{xc}}^j) = \sum_{g\in i}\sum_{h\in j} w_g^i w_h^j \operatorname{Cov}\left(e_{\mathrm{xc}}(x_g^i),e_{\mathrm{xc}}(x_h^j)\right) \tag*{(34)} Cov ( E xc i , E xc j ) = g ∈ i ∑ h ∈ j ∑ w g i w h j Cov ( e xc ( x g i ) , e xc ( x h j ) ) (34)
= ∑ g ∈ i ∑ h ∈ j w g i w h j k x c ( x g i , x h j ) , (35) = \sum_{g\in i}\sum_{h\in j} w_g^i w_h^j k_{\mathrm{xc}}(x_g^i,x_h^j), \tag*{(35)} = g ∈ i ∑ h ∈ j ∑ w g i w h j k xc ( x g i , x h j ) , (35)
where k x c ( x , x ′ ) k_{\mathrm{xc}}(x,x') k xc ( x , x ′ ) is the kernel function for the covariance between the exchange energy densities for feature vectors x \mathbf{x} x and x ′ \mathbf{x'} x ′ . Unfortunately, this expression can be quite expensive to compute because there are tens of thousands of grid points for each small molecule in the training database, and there are several hundred training molecules. Therefore, even for a database of this size, at least 10 12 10^{12} 1 0 12 evaluations of the kernel are required. To circumvent this problem, a set of control points x ~ a \tilde{\mathbf{x}}_a x ~ a can be sampled from the training database to use as a basis set for the feature space (as described in Appendix A § 18 ), effectively creating a sparse Gaussian process [51]. Equation 35 is then approximated as
Cov ( E x c i , E x c j ) ≈ ( k ~ x c i ) T K ~ − 1 k ~ x c j (36) \operatorname{Cov}(E_{\mathrm{xc}}^i,E_{\mathrm{xc}}^j) \approx\left(\tilde{\mathbf{k}}_{\mathrm{xc}}^i\right)^{\mathrm{T}}\tilde{\mathbf{K}}^{-1}\tilde{\mathbf{k}}_{\mathrm{xc}}^j \tag*{(36)} Cov ( E xc i , E xc j ) ≈ ( k ~ xc i ) T K ~ − 1 k ~ xc j (36)
( K ~ ) a b = k e x c ( x ~ a , x ~ b ) (37) \left(\tilde{\mathbf{K}}\right)_{ab}=k_{\mathrm{e}_{\mathrm{xc}}}(\tilde{\mathbf{x}}_a,\tilde{\mathbf{x}}_b) \tag*{(37)} ( K ~ ) ab = k e xc ( x ~ a , x ~ b ) (37)
( k ~ x c i ) a = ∑ g ∈ i w g i k e x c ( x g i , x ~ a ) . (38) \left(\tilde{\mathbf{k}}_{\mathrm{xc}}^i\right)_a=\sum_{g\in i}w_g^i k_{\mathrm{e}_{\mathrm{xc}}}(\mathbf{x}_g^i,\tilde{\mathbf{x}}_a). \tag*{(38)} ( k ~ xc i ) a = g ∈ i ∑ w g i k e xc ( x g i , x ~ a ) . (38)
The approximation of Eq. (35) by Eq. (36) is called the Nyström approximation, and it is covered in more detail in Chapter 8.1 of Ref. [51]. The approximation can be thought of as interpolating the kernel function over the control points x ~ a \tilde{\mathbf{x}}_a x ~ a . Supplemental Material Section S9 investigates the accuracy of this approximation with respect to the number of control points used.
In this work, the exchange functional alone is trained, and the kernel function is expressed in terms of the exchange enhancement factor F x = e x / e x L D A F_{\mathrm{x}}=e_{\mathrm{x}}/e_{\mathrm{x}}^{\mathrm{LDA}} F x = e x / e x LDA . This leads to the adjusted expressions
Cov ( E x i , E x j ) ≈ ( k ~ i ) T K ~ − 1 k ~ j = K i j (39) \operatorname{Cov}(E_{\mathrm{x}}^i,E_{\mathrm{x}}^j)\approx\left(\tilde{\mathbf{k}}^i\right)^{\mathrm{T}}\tilde{\mathbf{K}}^{-1}\tilde{\mathbf{k}}^j=K_{ij} \tag*{(39)} Cov ( E x i , E x j ) ≈ ( k ~ i ) T K ~ − 1 k ~ j = K ij (39)
( K ~ ) a b = k F x ( x ~ a , x ~ b ) (40) \left(\tilde{\mathbf{K}}\right)_{ab}=k_{F_{\mathrm{x}}}(\tilde{\mathbf{x}}_a,\tilde{\mathbf{x}}_b) \tag*{(40)} ( K ~ ) ab = k F x ( x ~ a , x ~ b ) (40)
( k ~ i ) a = ∑ g ∈ i w g i ( e x L D A ) g i k F x ( x g i , x ~ a ) , (41) \left(\tilde{\mathbf{k}}^i\right)_a=\sum_{g\in i}w_g^i\left(e_{\mathrm{x}}^{\mathrm{LDA}}\right)_g^i k_{F_{\mathrm{x}}}(\mathbf{x}_g^i,\tilde{\mathbf{x}}_a), \tag*{(41)} ( k ~ i ) a = g ∈ i ∑ w g i ( e x LDA ) g i k F x ( x g i , x ~ a ) , (41)
where k F x ( x , x ′ ) k_{F_{\mathrm{x}}}(\mathbf{x},\mathbf{x}^{\prime}) k F x ( x , x ′ ) is the covariance kernel for the exchange enhancement factor and ( e x L D A ) g i = e x L D A ( n i ( r g ) ) \left(e_{\mathrm{x}}^{\mathrm{LDA}}\right)_g^i=e_{\mathrm{x}}^{\mathrm{LDA}}(n_i(\mathbf{r}_g)) ( e x LDA ) g i = e x LDA ( n i ( r g )) is the LDA exchange energy density at coordinate g g g for molecule i i i (see Eq. (5) ). Having obtained these covariance expressions, they can be plugged into the predictive function for a Gaussian process to obtain the predicted exchange energy density [51]
e x C I D E R ( x ∗ ) = e x L D A ( n ∗ ) ∑ a k F x ( x ∗ , x ~ a ) α a (42) e_{\mathrm{x}}^{\mathrm{CIDER}}(\mathbf{x}_*)=e_{\mathrm{x}}^{\mathrm{LDA}}(n_*)\sum_a k_{F_{\mathrm{x}}}(\mathbf{x}_*,\tilde{\mathbf{x}}_a)\alpha_a \tag*{(42)} e x CIDER ( x ∗ ) = e x LDA ( n ∗ ) a ∑ k F x ( x ∗ , x ~ a ) α a (42)
α = ∑ i k ~ i { [ K + Σ n o i s e ] − 1 y } i , (43) \alpha=\sum_i\tilde{\mathbf{k}}^i\left\{\left[\mathbf{K}+\mathbf{\Sigma}_{\mathrm{noise}}\right]^{-1}\mathbf{y}\right\}_i, \tag*{(43)} α = i ∑ k ~ i { [ K + Σ noise ] − 1 y } i , (43)
where n ∗ n_* n ∗ and x ∗ \mathbf{x}_* x ∗ are the density and feature vector at a test point and Σ n o i s e \mathbf{\Sigma}_{\mathrm{noise}} Σ noise is the noise matrix for the training points.
Model Types and Feature vectors
This section lists the feature vectors for each model type in terms of s s s (Eq. (9) ), α \alpha α (Eq. (11) ), and the nonlocal features G i G_i G i (Eq. (18) ). All of the feature vectors below are designed such that for the uniform electron gas, x = 0 \mathbf{x}=\mathbf{0} x = 0 . Since the exchange enhancement factor is F x = 1 F_{\mathrm{x}}=1 F x = 1 for the uniform electron gas [49], the uniform electron gas constraint is enforced with a single noiseless training point F x C I D E R ( 0 ) = 1 F_{\mathrm{x}}^{\mathrm{CIDER}}(\mathbf{0})=1 F x CIDER ( 0 ) = 1 . All functionals in this work satisfy the uniform electron gas constraint.
The primary purpose of the transformations performed on the features below is to constrain the ML feature vector elements to fall in a finite range (e.g. [ 0 , 1 ] [0,1] [ 0 , 1 ] ), which makes it easier to map the resulting models to cubic splines as discussed in Section II C 4. Aside from that, the exact form of the feature transformation is a flexible choice, informed by physical considerations, but should not matter much as long as the target is a reasonably smooth function of the transformed features. However, more systematically fine-tuning these transformations in the future could potentially improve model performance slightly.
For the SL-GGA, the feature vector x \mathbf{x} x (Eq. (7) ) is of dimension 1:
x 1 = γ s 2 1 + γ s 2 , (44) x_1=\frac{\gamma s^2}{1+\gamma s^2}, \tag*{(44)} x 1 = 1 + γ s 2 γ s 2 , (44)
where γ = 0.243 \gamma=0.243 γ = 0.243 was empirically chosen by Becke [52] to fit the exact exchange energies of noble gas atoms. This value of γ \gamma γ was also used in our previous work [35] and in the B97-type functionals of the Head-Gordon group [28, 53, 54]. For the SL-MGGA, the feature vector is of dimension 2:
x 1 = γ s 2 1 + γ s 2 (45) x_1=\frac{\gamma s^2}{1+\gamma s^2} \tag*{(45)} x 1 = 1 + γ s 2 γ s 2 (45)
x 2 = 2 1 + α 2 − 1. (46) x_2=\frac{2}{1+\alpha^2}-1. \tag*{(46)} x 2 = 1 + α 2 2 − 1. (46)
The NL-GGA feature vector is of dimension 4:
x 1 = γ s 2 1 + γ s 2 (47) x_1=\frac{\gamma s^2}{1+\gamma s^2} \tag*{(47)} x 1 = 1 + γ s 2 γ s 2 (47)
x 2 = G 1 2 + G 1 − 1 2 (48) x_2=\frac{G_1}{2+G_1}-\frac{1}{2} \tag*{(48)} x 2 = 2 + G 1 G 1 − 2 1 (48)
x 3 = G 2 2 + G 2 − 1 2 (49) x_3=\frac{G_2}{2+G_2}-\frac{1}{2} \tag*{(49)} x 3 = 2 + G 2 G 2 − 2 1 (49)
x 4 = G 3 2 + G 3 − 1 2 . (50) x_4=\frac{G_3}{2+G_3}-\frac{1}{2}. \tag*{(50)} x 4 = 2 + G 3 G 3 − 2 1 . (50)
Lastly, the NL-MGGA feature vector is of dimension 5:
x 1 = γ s 2 1 + γ s 2 (51) x_1=\frac{\gamma s^2}{1+\gamma s^2} \tag*{(51)} x 1 = 1 + γ s 2 γ s 2 (51)
x 2 = 2 1 + α 2 − 1 (52) x_2=\frac{2}{1+\alpha^2}-1 \tag*{(52)} x 2 = 1 + α 2 2 − 1 (52)
x 3 = G 1 2 + G 1 − 1 2 (53) x_3=\frac{G_1}{2+G_1}-\frac{1}{2} \tag*{(53)} x 3 = 2 + G 1 G 1 − 2 1 (53)
x 4 = G 2 2 + G 2 − 1 2 (54) x_4=\frac{G_2}{2+G_2}-\frac{1}{2} \tag*{(54)} x 4 = 2 + G 2 G 2 − 2 1 (54)
x 5 = G 3 2 + G 3 − 1 2 . (55) x_5=\frac{G_3}{2+G_3}-\frac{1}{2}. \tag*{(55)} x 5 = 2 + G 3 G 3 − 2 1 . (55)
The three nonlocal features G 1 , G 2 , G 3 G_1,G_2,G_3 G 1 , G 2 , G 3 are given by Eq. (18) , and the parameterization of the kernel exponents (i.e. the constants B i B_i B i and C i C_i C i in Eqs. 20-23) is described in detail in Appendix A § 19.1 .
Kernels
To fully define the Gaussian process model for the exchange energy, we need to specify the kernel function k F x ( x , x ′ ) k_{\mathrm{Fx}}(\mathbf{x},\mathbf{x}^{\prime}) k Fx ( x , x ′ ) and the noise matrix Σ n o i s e \boldsymbol{\Sigma}_{\mathrm{noise}} Σ noise . The noise matrix is constructed to heuristically approximate the model uncertainty for each data point as described in Appendix A § 19 . To reflect the different number of features in each functional type, a different kernel is used for k F x ( x , x ′ ) k_{\mathrm{Fx}}(\mathbf{x},\mathbf{x}^{\prime}) k Fx ( x , x ′ ) for each, as specified below:
k S L − G G A ( x , x ′ ) = Σ C o v k 1 ( x 1 , x 1 ′ ) (56) k_{\mathrm{SL-GGA}}(\mathbf{x},\mathbf{x}^{\prime})=\Sigma_{\mathrm{Cov}}k_{1}(x_{1},x_{1}^{\prime}) \tag*{(56)} k SL − GGA ( x , x ′ ) = Σ Cov k 1 ( x 1 , x 1 ′ ) (56)
k S L − M G G A ( x , x ′ ) = Σ C o v k 1 ( x 1 , x 1 ′ ) k 2 ( x 2 , x 2 ′ ) (57) k_{\mathrm{SL-MGGA}}(\mathbf{x},\mathbf{x}^{\prime})=\Sigma_{\mathrm{Cov}}k_{1}(x_{1},x_{1}^{\prime})k_{2}(x_{2},x_{2}^{\prime}) \tag*{(57)} k SL − MGGA ( x , x ′ ) = Σ Cov k 1 ( x 1 , x 1 ′ ) k 2 ( x 2 , x 2 ′ ) (57)
k N L − G G A ( x , x ′ ) = Σ C o v k 1 ( x 1 , x 1 ′ ) ∑ i = 2 4 ∑ j = i + 1 4 k i ( x i , x i ′ ) k j ( x j , x j ′ ) (58) k_{\mathrm{NL-GGA}}(\mathbf{x},\mathbf{x}^{\prime})=\Sigma_{\mathrm{Cov}}k_{1}(x_{1},x_{1}^{\prime})\sum_{i=2}^{4}\sum_{j=i+1}^{4}k_{i}(x_{i},x_{i}^{\prime})k_{j}(x_{j},x_{j}^{\prime}) \tag*{(58)} k NL − GGA ( x , x ′ ) = Σ Cov k 1 ( x 1 , x 1 ′ ) i = 2 ∑ 4 j = i + 1 ∑ 4 k i ( x i , x i ′ ) k j ( x j , x j ′ ) (58)
k N L − M G G A ( x , x ′ ) = Σ C o v k 1 ( x 1 , x 1 ′ ) ∑ i = 2 5 ∑ j = i + 1 5 k i ( x i , x i ′ ) k j ( x j , x j ′ ) , (59) k_{\mathrm{NL-MGGA}}(\mathbf{x},\mathbf{x}^{\prime})=\Sigma_{\mathrm{Cov}}k_{1}(x_{1},x_{1}^{\prime})\sum_{i=2}^{5}\sum_{j=i+1}^{5}k_{i}(x_{i},x_{i}^{\prime})k_{j}(x_{j},x_{j}^{\prime}), \tag*{(59)} k NL − MGGA ( x , x ′ ) = Σ Cov k 1 ( x 1 , x 1 ′ ) i = 2 ∑ 5 j = i + 1 ∑ 5 k i ( x i , x i ′ ) k j ( x j , x j ′ ) , (59)
where Σ C o v \Sigma_{\mathrm{Cov}} Σ Cov is a covariance hyperparameter that can be tuned for each model, and the base kernel k i ( x , x ′ ) k_i(\mathbf{x},\mathbf{x}^{\prime}) k i ( x , x ′ ) for all of the above is the squared-exponential kernel
k i ( x , x ′ ) = exp [ − 1 2 ( x − x ′ l i ) 2 ] . (60) k_i(\mathbf{x},\mathbf{x}^{\prime})=\exp\left[-\frac{1}{2}\left(\frac{x-x^{\prime}}{l_i}\right)^2\right]. \tag*{(60)} k i ( x , x ′ ) = exp [ − 2 1 ( l i x − x ′ ) 2 ] . (60)
Equations 58 and 59 use the additive Gaussian process approach of Duvenaud et al. [55].
Because the cost of evaluating Gaussian processes scales linearly with the number of training points, we map the models to cubic splines after training so they can be evaluated efficiently. The semilocal kernels have 1 and 2 features, so they can be mapped to 1 and 2-dimensional cubic splines, respectively. The nonlocal kernels are a sum of 3-dimensional kernels, so they can be mapped to a sum of 3-dimensional cubic splines [35, 56–58]. For a description of the hyperparameter selection process for these models, see Appendix A § 19.1 .
Auxiliary Expansion of the CIDER Features
The rest of this section focuses on the efficient implementation of nonlocal features within Gaussian-type orbital and plane-wave DFT, which is vital for the practicality of CIDER as well as any nonlocal density functional. Evaluating Eq. (18) via simple numerical integration is computationally expensive due to quadratic scaling with the number of grid points. The key insight to designing a faster approach comes from Román-Pérez and Soler [59], who developed a method to expand van der Waals density functionals as a sum of convolutions that could be computed efficiently using the Fast Fourier Transform (FFT). They observed that a function of the form in Eq. (19) , which depends exclusively on the distance between two points and on a semilocal functional of the density at each point, can be approximately expanded as [59]
Φ ( a ( r 2 ) , b ( r 1 ) , r 12 ) ≈ ∑ α β p α a ( a ( r 2 ) ) p β b ( b ( r 1 ) ) Φ α β ( r 12 ) . (61) \Phi\left(a(\mathbf{r}_2),b(\mathbf{r}_1),r_{12}\right)\approx\sum_{\alpha\beta}p_{\alpha}^{a}\left(a(\mathbf{r}_2)\right)p_{\beta}^{b}\left(b(\mathbf{r}_1)\right)\Phi_{\alpha\beta}(r_{12}). \tag*{(61)} Φ ( a ( r 2 ) , b ( r 1 ) , r 12 ) ≈ α β ∑ p α a ( a ( r 2 ) ) p β b ( b ( r 1 ) ) Φ α β ( r 12 ) . (61)
In Eq. (61) , α \alpha α and β \beta β are indices which each correspond to interpolation control points q α q_{\alpha} q α and q β q_{\beta} q β , respectively, and p α a ( a ( r ) ) p_{\alpha}^{a}(a(\mathbf{r})) p α a ( a ( r )) and p β b ( b ( r ) ) p_{\beta}^{b}(b(\mathbf{r})) p β b ( b ( r )) are interpolating functions which take the values p α a ( q α ′ ) = δ α α ′ p_{\alpha}^{a}(q_{\alpha^{\prime}})=\delta_{\alpha\alpha^{\prime}} p α a ( q α ′ ) = δ α α ′ and p β b ( q β ′ ) = δ β β ′ p_{\beta}^{b}(q_{\beta^{\prime}})=\delta_{\beta\beta^{\prime}} p β b ( q β ′ ) = δ β β ′ at the interpolating points. The term Φ α β ( r ) ≡ Φ ( q α , q β , r ) \Phi_{\alpha\beta}(r)\equiv\Phi(q_{\alpha},q_{\beta},r) Φ α β ( r ) ≡ Φ ( q α , q β , r ) .
The Gaussian form of Eq. (19) is particularly conducive to the use of an even-tempered basis for interpolation, in which q α q_{\alpha} q α takes the form
q α = q 0 λ α (62) q_{\alpha}=q_{0}\lambda^{\alpha} \tag*{(62)} q α = q 0 λ α (62)
for some constant λ > 1 \lambda>1 λ > 1 . Román-Pérez and Soler [59] used cubic splines for interpolation, and for the solid-state implementation of CIDER we follow this approach, with p α a ( a ( r ) ) p_{\alpha}^{a}(a(\mathbf{r})) p α a ( a ( r )) being a cubic spline in the transformed coordinate γ ( r ) = ln ( a ( r ) / q 0 ) ln ( λ ) \gamma(\mathbf{r})=\frac{\ln(a(\mathbf{r})/q_{0})}{\ln(\lambda)} γ ( r ) = l n ( λ ) l n ( a ( r ) / q 0 ) . However, for the molecular implementation, we instead treat the control points as a basis set of Gaussians e − q α r 12 2 e^{-q_{\alpha}r_{12}^{2}} e − q α r 12 2 onto which the function e − a ( r 1 ) r 12 2 e^{-a(\mathbf{r}_1)r_{12}^{2}} e − a ( r 1 ) r 12 2 is projected:
p α a ( a ( r ) ) ≡ ∑ β ( S a − 1 ) α β s α ( a ( r ) ) (63) p_{\alpha}^{a}(a(\mathbf{r}))\equiv\sum_{\beta}(S_{a}^{-1})_{\alpha\beta}s_{\alpha}(a(\mathbf{r})) \tag*{(63)} p α a ( a ( r )) ≡ β ∑ ( S a − 1 ) α β s α ( a ( r )) (63)
( S a ) α β = ∫ d 3 r exp [ − ( q α + q β ) ∣ r ∣ 2 ] (64) (S_{a})_{\alpha\beta}=\int\mathrm{d}^{3}r\exp\left[-(q_{\alpha}+q_{\beta})|\mathbf{r}|^{2}\right] \tag*{(64)} ( S a ) α β = ∫ d 3 r exp [ − ( q α + q β ) ∣ r ∣ 2 ] (64)
s α ( a ( r ) ) = ∫ d 3 r ′ exp [ − ( a ( r ) + q α ) ∣ r ′ ∣ 2 ] . (65) s_{\alpha}(a(\mathbf{r}))=\int\mathrm{d}^{3}r^{\prime}\exp\left[-(a(\mathbf{r})+q_{\alpha})|\mathbf{r}^{\prime}|^{2}\right]. \tag*{(65)} s α ( a ( r )) = ∫ d 3 r ′ exp [ − ( a ( r ) + q α ) ∣ r ′ ∣ 2 ] . (65)
As λ → 1 \lambda\to1 λ → 1 , the number of control points (for the interpolation scheme) and number of basis functions (for the basis expansion scheme) both become infinite, and the kernel expansion becomes exact for both approaches. We determined that the cubic interpolation yields better numerical stability for the plane-wave implementation, while the basis expansion form is sufficiently stable for the molecular implementation.
The benefit of expanding the kernel in this manner is that it allows the nonlocal features (Eq. (18) ) to be expressed as a sum of convolutions
G ( r 1 ) = ∑ β p β b ( b ( r 1 ) ) ∑ α ∫ d 3 r 2 Φ α β ( r 12 ) θ α ( r 2 ) , (66) G(\mathbf{r}_1)=\sum_{\beta}p_{\beta}^{b}(b(\mathbf{r}_1))\sum_{\alpha}\int\mathrm{d}^{3}r_2\Phi_{\alpha\beta}(r_{12})\theta_{\alpha}(\mathbf{r}_2), \tag*{(66)} G ( r 1 ) = β ∑ p β b ( b ( r 1 )) α ∑ ∫ d 3 r 2 Φ α β ( r 12 ) θ α ( r 2 ) , (66)
with θ α ( r ) = p α a ( a ( r ) ) n ( r ) \theta_{\alpha}(\mathbf{r})=p_{\alpha}^{a}(a(\mathbf{r}))n(\mathbf{r}) θ α ( r ) = p α a ( a ( r )) n ( r ) . Within plane-wave DFT, the convolutions can be performed with quasi-linear computational complexity by using FFTs [59]. In addition, because CIDER uses a squared exponential for Φ α β ( r 12 ) \Phi_{\alpha\beta}(r_{12}) Φ α β ( r 12 ) (Eq. (19) ), the integrals of Eq. (66) can be evaluated analytically if θ α ( r ) \theta_{\alpha}(\mathbf{r}) θ α ( r ) is projected onto a Gaussian-type orbital basis. We use this approach to design an efficient implementation of nonlocal features for molecular DFT (Section II E).
The Román-Pérez and Soler technique effectively solves the problem of implementing CIDER for an all-electron plane-wave DFT calculation. However, such calculations are infeasible for all but the smallest atoms because a prohibitively large plane-wave basis would be required to describe the core electrons. In addition, pseudopotentials cannot be reliably used with nonlocal features because G ( r ) G(\mathbf{r}) G ( r ) calculated on the pseudo-density is not sufficiently similar to the exact, all-electron feature to get accurate results. To solve this problem, in Section IIF a method is introduced to implement nonlocal features within the Projector-Augmented Wave (PAW) formalism, which requires careful attention to the contributions to G ( r ) G(\mathbf{r}) G ( r ) from the core regions around the atoms.
Before proceeding, we note that while we borrow and expand on numerical techniques developed for nonlocal van der Waals functionals, we emphasize that the functionals in this work are still fit only to the exchange energy and therefore do not account for the correlation effects that give rise to van der Waals interactions. Due to their nonlocality, the nonlocal features in this work could prove useful for learning van der Waals interactions, but demonstrating this will require further research.
Efficient Implementation of Nonlocal CIDER Features for Molecular DFT
Computing Eq. (18) using simple numerical integration is prohibitively expensive because the number of operations is proportional to the square of the number of grid points. While van der Waals functionals like VV10 [60] are usually computed this way within Gaussian-type orbital codes, the contribution of such functionals is sufficiently small and smooth that a very sparse grid can be used. The nonlocal features require a grid of the same density as the semilocal XC grid, and the number of operations per pair of grid points is higher for evaluating Eq. (18) than, for example, the VV10 kernel. It is therefore necessary to reduce the computational complexity of evaluating the nonlocal features for molecular systems.
In this section, we present a charge-partitioning and density fitting-based approach to compute the nonlocal features within Gaussian-type orbital DFT codes for molecular systems, which we have implemented for use in the PySCF code [61, 62]. Much of this approach is based on the work of Franchini et al. [63], who developed a similar method for computing the Coulomb energy. To start, the density is partitioned into atomic contributions using Becke partitioning [64],
n A ( r ) = W A ( r ) n ( r ) (67) n_A(\mathbf{r}) = W_A(\mathbf{r})n(\mathbf{r}) \tag*{(67)} n A ( r ) = W A ( r ) n ( r ) (67)
∑ A W A ( r ) = 1 , (68) \sum_A W_A(\mathbf{r}) = 1, \tag*{(68)} A ∑ W A ( r ) = 1 , (68)
where A A A is the atom index and W A ( r ) W_A(\mathbf{r}) W A ( r ) are the atomic Becke weights. The first step in evaluating G ( r ) G(\mathbf{r}) G ( r ) is to project θ α ( r ) \theta_\alpha(\mathbf{r}) θ α ( r ) onto spherical harmonics for each atom (with L L L being the combined l l l and m m m indices and R A \mathbf{R}_A R A being the position of atom A A A ):
θ α , L A ( r ) = ∫ d Ω A θ α ( r ) W A ( r ) Y L ( Ω A ) (69) \theta^A_{\alpha,L}(\mathbf{r}) = \int d\Omega_A \theta_\alpha(\mathbf{r}) W_A(\mathbf{r})Y_L(\Omega_A) \tag*{(69)} θ α , L A ( r ) = ∫ d Ω A θ α ( r ) W A ( r ) Y L ( Ω A ) (69)
r = R A + ( r cos ϕ sin θ , r sin ϕ sin θ , r cos θ ) , (70) \mathbf{r} = \mathbf{R}_A + (r\cos\phi\sin\theta,r\sin\phi\sin\theta,r\cos\theta), \tag*{(70)} r = R A + ( r cos ϕ sin θ , r sin ϕ sin θ , r cos θ ) , (70)
where Ω A = ( θ , ϕ ) \Omega_A=(\theta,\phi) Ω A = ( θ , ϕ ) is the solid angle and Y L ( Ω ) Y_L(\Omega) Y L ( Ω ) are the spherical harmonics. This projection is done numerically using Lebedev grids [65]. Then each L L L channel is projected onto an even-tempered Gaussian basis { χ A , L , μ ( r ) } \{\chi_{A,L,\mu}(\mathbf{r})\} { χ A , L , μ ( r )} as follows:
χ A , L , μ ( r ) = Y L ( r ^ ) N L , μ r l exp ( − μ r 2 ) (71) \chi_{A,L,\mu}(\mathbf{r}) = Y_L(\hat{\mathbf{r}})N_{L,\mu}r^l\exp(-\mu r^2) \tag*{(71)} χ A , L , μ ( r ) = Y L ( r ^ ) N L , μ r l exp ( − μ r 2 ) (71)
θ ~ α , L , μ A = ∫ d r r 2 + l N l , μ exp ( − μ r 2 ) θ α , L A ( r ) , (72) \tilde{\theta}^A_{\alpha,L,\mu} = \int dr\,r^{2+l}N_{l,\mu}\exp(-\mu r^2)\theta^A_{\alpha,L}(r), \tag*{(72)} θ ~ α , L , μ A = ∫ d r r 2 + l N l , μ exp ( − μ r 2 ) θ α , L A ( r ) , (72)
where N L , μ N_{L,\mu} N L , μ is a normalization constant. The exponents μ \mu μ are constructed using the even-tempered Gaussian approach with tunable parameter β a u x \beta^{\mathrm{aux}} β aux
μ n = μ min ( β a u x ) n n = { 0 , 1 , … , N } (73) \mu_n = \mu_{\min}\left(\beta^{\mathrm{aux}}\right)^n \qquad n=\{0,1,\ldots,N\} \tag*{(73)} μ n = μ m i n ( β aux ) n n = { 0 , 1 , … , N } (73)
A transformation to θ α , L , μ A \theta^A_{\alpha,L,\mu} θ α , L , μ A is then performed by multiplying θ ~ α , L , μ A \tilde{\theta}^A_{\alpha,L,\mu} θ ~ α , L , μ A by the inverse overlap matrix of the χ \chi χ basis for each L L L channel:
θ α , L , μ A = ∑ ν [ ( S A , L X ) − 1 ] μ ν θ ~ α , L , ν A (74) \theta^A_{\alpha,L,\mu} = \sum_\nu\left[\left(S^X_{A,L}\right)^{-1}\right]_{\mu\nu}\tilde{\theta}^A_{\alpha,L,\nu} \tag*{(74)} θ α , L , μ A = ν ∑ [ ( S A , L X ) − 1 ] μν θ ~ α , L , ν A (74)
( S A , L X ) μ ν = ⟨ χ A , L , μ ∣ χ A , L , ν ⟩ . (75) \left(S^X_{A,L}\right)_{\mu\nu} = \langle\chi_{A,L,\mu}|\chi_{A,L,\nu}\rangle. \tag*{(75)} ( S A , L X ) μν = ⟨ χ A , L , μ ∣ χ A , L , ν ⟩ . (75)
For each atom, a second even-tempered basis { ξ A , L , ν ( r ) } \{\xi_{A,L,\nu}(\mathbf{r})\} { ξ A , L , ν ( r )} is introduced to serve as a basis for the convolutions of χ A , L , μ ( r ) \chi_{A,L,\mu}(\mathbf{r}) χ A , L , μ ( r ) . The following projections are computed using analytical Gaussian integrals:
F ~ β , L , ν A = ∑ α , μ θ α , L , μ A × ∫ d 3 r 1 d 3 r 2 Φ α β ( r 12 ) χ A , L , μ ( r 1 ) ξ A , L , ν ( r 2 ) . (76) \begin{aligned}
\tilde{F}^A_{\beta,L,\nu} = \sum_{\alpha,\mu}\theta^A_{\alpha,L,\mu} \\
&\times\int d^3r_1d^3r_2\,\Phi_{\alpha\beta}(r_{12})\chi_{A,L,\mu}(r_1)\xi_{A,L,\nu}(r_2). \tag*{(76)}
\end{aligned} F ~ β , L , ν A = α , μ ∑ θ α , L , μ A × ∫ d 3 r 1 d 3 r 2 Φ α β ( r 12 ) χ A , L , μ ( r 1 ) ξ A , L , ν ( r 2 ) . (76)
The transformation to F β , L , ν A F^A_{\beta,L,\nu} F β , L , ν A is then performed by multiplying F ~ β , L , ν A \tilde{F}^A_{\beta,L,\nu} F ~ β , L , ν A by the inverse overlap matrix of the ξ \xi ξ basis, similarly to Eq. (74) .
The second to last step is to transform F β , L , ν A F^A_{\beta,L,\nu} F β , L , ν A to F β ( r g ) F_\beta(\mathbf{r}_g) F β ( r g ) , where r g \mathbf{r}_g r g are the DFT integration grid points. Because evaluating every ξ \xi ξ orbital of every atom at every r g \mathbf{r}_g r g would be computationally expensive, F β , L , ν A F^A_{\beta,L,\nu} F β , L , ν A is first transformed to a cubic spline F β , L , G , p A F^A_{\beta,L,G,p} F β , L , G , p A , where G G G is a radial coordinate index and p p p indexes the polynomial coefficients of the spline in the interval between points G G G and G + 1 G+1 G + 1 . The transformation from F β , L , G , p A F^A_{\beta,L,G,p} F β , L , G , p A to F β ( r g ) F_\beta(\mathbf{r}_g) F β ( r g ) is then performed as
F β ( r g ) = ∑ A ∑ L Y L ( r ^ g A ) ∑ p = 0 3 ( r g A − r G ) p F β , L , G , p A , (77) F_\beta(\mathbf{r}_g) = \sum_A\sum_L Y_L(\hat{\mathbf{r}}_{gA})\sum_{p=0}^{3}(r_{gA}-r_G)^pF^A_{\beta,L,G,p}, \tag*{(77)} F β ( r g ) = A ∑ L ∑ Y L ( r ^ g A ) p = 0 ∑ 3 ( r g A − r G ) p F β , L , G , p A , (77)
where r g A = r g − R A \mathbf{r}_{gA}=\mathbf{r}_g-\mathbf{R}_A r g A = r g − R A . The spline index G G G is a step function of ∣ r g A ∣ \lvert\mathbf{r}_{gA}\rvert ∣ r g A ∣ . The computational cost of evaluating the splines depends heavily on the efficiency of memory access patterns on the processor, so the coordinates are sorted by G G G before evaluating the contributions to F β ( r g ) F_\beta(\mathbf{r}_g) F β ( r g ) from each atom.
The last and simplest step to compute G ( r ) G(\mathbf{r}) G ( r ) is to evaluate the sum
G ( r ) = ∑ β p β b ( r g ) F β ( r g ) . (78) G(\mathbf{r})=\sum_\beta p_\beta^b(\mathbf{r}_g)F_\beta(\mathbf{r}_g). \tag*{(78)} G ( r ) = β ∑ p β b ( r g ) F β ( r g ) . (78)
After computing the XC energy from G ( r ) G(\mathbf{r}) G ( r ) and the semilocal features, the contributions to the XC potential from the θ α ( r ) \theta_\alpha(\mathbf{r}) θ α ( r ) terms can then be computed by passing backward through the matrix multiplications and spline evaluations detailed above.
The gradient of the total energy with respect to nuclear positions is required for the evaluation of forces. For this implementation of the CIDER functional with nonlocal features, the energy gradient with respect to the nuclear position of atom A A A is given by
∇ A E x c = ∇ A s l E x c + ∑ g ( ∑ α δ E x c δ θ α ( r g ) θ α ( r g ) ) ∇ A w g + ∑ g ( ∑ β δ E x c δ F β ( r g ) ∇ A F β ( r g ) ) w g . (79) \nabla_A E_{\mathrm{xc}}=\nabla_A^{\mathrm{sl}}E_{\mathrm{xc}}+\sum_g\left(\sum_\alpha\frac{\delta E_{\mathrm{xc}}}{\delta\theta_\alpha(\mathbf{r}_g)}\theta_\alpha(\mathbf{r}_g)\right)\nabla_A w_g
+\sum_g\left(\sum_\beta\frac{\delta E_{\mathrm{xc}}}{\delta F_\beta(\mathbf{r}_g)}\nabla_A F_\beta(\mathbf{r}_g)\right)w_g. \tag*{(79)} ∇ A E xc = ∇ A sl E xc + g ∑ ( α ∑ δ θ α ( r g ) δ E xc θ α ( r g ) ) ∇ A w g + g ∑ β ∑ δ F β ( r g ) δ E xc ∇ A F β ( r g ) w g . (79)
where w g w_g w g is the numerical integration quadrature weight corresponding to the coordinate r g \mathbf{r}_g r g . The analytical energy gradient for a semilocal functional ∇ A s l E x c \nabla_A^{\mathrm{sl}}E_{\mathrm{xc}} ∇ A sl E xc is given by Johnson et al. [66] Eq. 11.
The computational bottleneck of this algorithm is the spline evaluation in Eq. (77) , whose cost scales quadratically with system size. Specifically, the cost scales as Θ ( N g r i d N a t o m N β l max 2 ) \Theta(N_{\mathrm{grid}}N_{\mathrm{atom}}N_\beta l_{\max}^2) Θ ( N grid N atom N β l m a x 2 ) , where N g r i d N_{\mathrm{grid}} N grid is the number of integration grid points, N a t o m N_{\mathrm{atom}} N atom is the number of atoms, N β N_\beta N β is the number of kernel interpolation points in Eq. (61) , and l max l_{\max} l m a x is the maximum spherical harmonic order used in the ξ \xi ξ basis. All other components of the algorithm are performed on individual atoms and therefore scale linearly with system size.
Quasi-linear-scaling Implementation of Nonlocal CIDER Features for Plane-Wave DFT with PAW
This section covers our algorithm for evaluating the nonlocal CIDER features within the Projector-Augmented Wave (PAW) method [67], a generalization of the pseudopotential method that can recover the full wave function and electron density. We have implemented this algorithm in the GPAW code [68, 69]. As discussed above, Eqs. (61) and (66) enable the evaluation of G ( r ) G(\mathbf{r}) G ( r ) in O ( N log N ) O(N\log N) O ( N log N ) complexity if the density n ( r ) n(\mathbf{r}) n ( r ) is represented on an FFT grid. However, in the PAW formalism of Blöchl [67], the all-electron density n ( r ) n(\mathbf{r}) n ( r ) is represented as
n ( r ) = n ~ ( r ) + ∑ A ( n A 1 ( r ) − n ~ A 1 ( r ) ) , (80) n(\mathbf{r})=\widetilde{n}(\mathbf{r})+\sum_A\left(n_A^1(\mathbf{r})-\widetilde{n}_A^1(\mathbf{r})\right), \tag*{(80)} n ( r ) = n ( r ) + A ∑ ( n A 1 ( r ) − n A 1 ( r ) ) , (80)
where A A A indexes the atoms, n ~ ( r ) \widetilde{n}(\mathbf{r}) n ( r ) is the pseudo-density on the FFT grid, and the terms with the 1 superscript are represented on radial support grids centered around each atom and extending to some cutoff radius r c r_c r c . Within the cutoff radius of atom A A A , n ~ ( r ) = n ~ A 1 ( r ) \widetilde{n}(\mathbf{r})=\widetilde{n}_A^1(\mathbf{r}) n ( r ) = n A 1 ( r ) and n A 1 ( r ) n_A^1(\mathbf{r}) n A 1 ( r ) is the all-electron density. Outside the cutoff radius of atom A A A , n A 1 ( r ) = n ~ A 1 ( r ) n_A^1(\mathbf{r})=\widetilde{n}_A^1(\mathbf{r}) n A 1 ( r ) = n A 1 ( r ) .
For a semilocal functional, Eq. (80) yields a simple expression for the XC energy E x c [ n ] E_{\mathrm{xc}}[n] E xc [ n ] :
E x c [ n ] = E x c [ n ~ ] + ∑ A ( E x c [ n A 1 ] − E x c [ n ~ A 1 ] ) . (81) E_{\mathrm{xc}}[n]=E_{\mathrm{xc}}[\widetilde{n}]+\sum_A\left(E_{\mathrm{xc}}[n_A^1]-E_{\mathrm{xc}}[\widetilde{n}_A^1]\right). \tag*{(81)} E xc [ n ] = E xc [ n ] + A ∑ ( E xc [ n A 1 ] − E xc [ n A 1 ] ) . (81)
However, the case is not so simple for a nonlocal functional such as CIDER. If Eq. (18) is evaluated on the pseudo-density n ~ ( r ) \widetilde{n}(\mathbf{r}) n ( r ) , the resulting G ( r ) G(\mathbf{r}) G ( r ) is not equal to the all-electron feature even outside the cutoff radius because the convolutions used to compute the feature are nonlocal. For the same reason, evaluating Eq. (18) on the atomic density n A 1 ( r ) n_A^1(\mathbf{r}) n A 1 ( r ) does not yield the all-electron feature inside the cutoff radius. Therefore, implementation of CIDER (and other nonlocal functionals with similar forms, such as some kinetic energy and van der Waals functionals) for PAW must construct a “pseudo-feature” G ~ ( r ) \widetilde{G}(\mathbf{r}) G ( r ) that is smooth but yields the correct feature outside the cutoff radius, as well as construct the “all-electron feature” G ( r ) G(\mathbf{r}) G ( r ) on the radial support grids inside the cutoff radius. Because of this difficulty, current plane-wave implementations [70] of nonlocal functionals like van der Waals correlation use the pseudo-density rather than the full PAW density. When this approach is used, one may need to use PAW datasets with more valence electrons, increasing computational cost [70]. In addition, because CIDER computes the exchange energy, which is of much larger magnitude than the van der Waals correlation energy, higher precision is required. A similar problem exists for kinetic energy density functionals. There is an implementation of semilocal kinetic energy functionals with PAW [71], and of nonlocal kinetic energy functionals with ultrasoft pseudopotentials [72], but no nonlocal kinetic energy functional implementation within PAW. Therefore, by designing an algorithm for computing nonlocal features for CIDER within the PAW formalism, we also enable the full integration of van der Waals functionals and orbital-free DFT [73] into the PAW method.
Starting from Eq. (80) , the all-electron value of G ( r ) G(\mathbf{r}) G ( r ) outside the core region is given by
G ~ ( r ) = ∑ β p β b ( b ~ ( r ) ) [ F ~ β ( r ) + ∑ A Δ F A , β ( r ) ] (82) \widetilde{G}(\mathbf{r})=\sum_{\beta}p_{\beta}^{b}(\widetilde{b}(\mathbf{r}))\left[\widetilde{F}_{\beta}(\mathbf{r})+\sum_{A}\Delta F_{A,\beta}(\mathbf{r})\right] \tag*{(82)} G ( r ) = β ∑ p β b ( b ( r )) [ F β ( r ) + A ∑ Δ F A , β ( r ) ] (82)
F ~ β ( r 1 ) = ∑ α ∫ d 3 r 2 Φ α β ( r 12 ) θ ~ α ( r 2 ) (83) \widetilde{F}_{\beta}(\mathbf{r}_{1})=\sum_{\alpha}\int\mathrm{d}^{3}r_{2}\,\Phi_{\alpha\beta}(r_{12})\widetilde{\theta}_{\alpha}(\mathbf{r}_{2}) \tag*{(83)} F β ( r 1 ) = α ∑ ∫ d 3 r 2 Φ α β ( r 12 ) θ α ( r 2 ) (83)
Δ F A , β ( r 1 ) = ∑ α ∫ d 3 r 2 Φ α β ( r 12 ) ( θ A , α 1 ( r 2 ) − θ ~ A , α 1 ( r 2 ) ) . (84) \Delta F_{A,\beta}(\mathbf{r}_{1})=\sum_{\alpha}\int\mathrm{d}^{3}r_{2}\,\Phi_{\alpha\beta}(r_{12})\left(\theta_{A,\alpha}^{1}(\mathbf{r}_{2})-\widetilde{\theta}_{A,\alpha}^{1}(\mathbf{r}_{2})\right). \tag*{(84)} Δ F A , β ( r 1 ) = α ∑ ∫ d 3 r 2 Φ α β ( r 12 ) ( θ A , α 1 ( r 2 ) − θ A , α 1 ( r 2 ) ) . (84)
The quantities b ~ ( r ) \widetilde{b}(\mathbf{r}) b ( r ) and θ ~ α ( r ) \widetilde{\theta}_{\alpha}(\mathbf{r}) θ α ( r ) are evaluated on the pseudo-density n ~ ( r ) \widetilde{n}(\mathbf{r}) n ( r ) ; θ A , α 1 ( r ) \theta_{A,\alpha}^{1}(\mathbf{r}) θ A , α 1 ( r ) and θ ~ A , α 1 ( r ) \widetilde{\theta}_{A,\alpha}^{1}(\mathbf{r}) θ A , α 1 ( r ) are evaluated on n A 1 ( r ) n_{A}^{1}(\mathbf{r}) n A 1 ( r ) and n ~ A 1 ( r ) \widetilde{n}_{A}^{1}(\mathbf{r}) n A 1 ( r ) , respectively. F ~ β ( r ) \widetilde{F}_{\beta}(\mathbf{r}) F β ( r ) can be evaluated simply using FFTs. To compute Δ F A , β ( r ) \Delta F_{A,\beta}(\mathbf{r}) Δ F A , β ( r ) , the term Δ θ A , α 1 ( r ) = θ A , α 1 ( r ) − θ ~ A , α 1 ( r ) \Delta\theta_{A,\alpha}^{1}(\mathbf{r})=\theta_{A,\alpha}^{1}(\mathbf{r})-\widetilde{\theta}_{A,\alpha}^{1}(\mathbf{r}) Δ θ A , α 1 ( r ) = θ A , α 1 ( r ) − θ A , α 1 ( r ) is first projected onto spherical harmonics and Gaussians as described in the previous section. Because Δ θ A , α 1 ( r ) \Delta\theta_{A,\alpha}^{1}(\mathbf{r}) Δ θ A , α 1 ( r ) is localized within the augmentation region and mostly spherical, far fewer spherical harmonics and Gaussians are required per atom than for the Gaussian-type orbital implementation. This projection yields Δ θ A , α , L , μ 1 \Delta\theta_{A,\alpha,L,\mu}^{1} Δ θ A , α , L , μ 1 . The Gaussians are then analytically projected into reciprocal space to yield Δ θ A , α , L 1 ( k ) \Delta\theta_{A,\alpha,L}^{1}(k) Δ θ A , α , L 1 ( k ) . In reciprocal space,
Δ F A , L , β ( k ) = ∑ α Φ α β ( k ) Δ θ A , α , L 1 ( k ) . (85) \Delta F_{A,L,\beta}(k)=\sum_{\alpha}\Phi_{\alpha\beta}(k)\Delta\theta_{A,\alpha,L}^{1}(k). \tag*{(85)} Δ F A , L , β ( k ) = α ∑ Φ α β ( k ) Δ θ A , α , L 1 ( k ) . (85)
The above equation applies because the convolution Φ α β ( k ) \Phi_{\alpha\beta}(k) Φ α β ( k ) has angular momentum number l = 0 l=0 l = 0 . Note that in this section, denoting a symbol as function of k k k implies that it is the Fourier transform of the real-space function corresponding to that symbol. For example, Φ α β ( k ) \Phi_{\alpha\beta}(k) Φ α β ( k ) is the Fourier transform of Φ α β ( r ) \Phi_{\alpha\beta}(r) Φ α β ( r ) .
Inspired by the PAW method itself, we perform the following trick to approximate Δ F A , β ( r ) \Delta F_{A,\beta}(\mathbf{r}) Δ F A , β ( r ) on the FFT grid. We introduce two sets of functions, { g i A ( r ) } \{g_{i}^{A}(\mathbf{r})\} { g i A ( r )} and { h j A ( r ) } \{h_{j}^{A}(\mathbf{r})\} { h j A ( r )} . The h j A ( r ) h_{j}^{A}(\mathbf{r}) h j A ( r ) functions form a localized basis (i.e. each function vanishes for r > r c r>r_{c} r > r c ) and are meant to fit the high-frequency components of Δ F A , β ( r ) \Delta F_{A,\beta}(\mathbf{r}) Δ F A , β ( r ) introduced by the all-electron density inside the core region. For the current implementation of these functions, we use differences between Gaussians and smooth polynomials that match their derivatives at r c r_{c} r c . The other functions g i A ( r ) g_{i}^{A}(\mathbf{r}) g i A ( r ) are both localized and smooth, and are meant to augment Δ θ A , α 1 ( r ) \Delta\theta_{A,\alpha}^{1}(\mathbf{r}) Δ θ A , α 1 ( r ) to yield the correct values of Δ F A , β ( r ) \Delta F_{A,\beta}(\mathbf{r}) Δ F A , β ( r ) outside of the augmentation region. To do this, least-squares linear regression is used to fit the following function to Δ F A , β ( r ) \Delta F_{A,\beta}(\mathbf{r}) Δ F A , β ( r ) :
y ^ A , β ( r ) = y ^ A , β 1 ( r ) + y ^ A , β 2 ( r ) (86) \widehat{y}_{A,\beta}(\mathbf{r})=\widehat{y}_{A,\beta}^{1}(\mathbf{r})+\widehat{y}_{A,\beta}^{2}(\mathbf{r}) \tag*{(86)} y A , β ( r ) = y A , β 1 ( r ) + y A , β 2 ( r ) (86)
y ^ A , β 1 ( r ) = ∑ j D j β h j A ( r ) (87) \widehat{y}_{A,\beta}^{1}(\mathbf{r})=\sum_{j}D_{j\beta}h_{j}^{A}(\mathbf{r}) \tag*{(87)} y A , β 1 ( r ) = j ∑ D j β h j A ( r ) (87)
y ^ A , β 2 ( r 1 ) = ∑ i , α C i α ∫ d 3 r 2 Φ α β ( r 12 ) g i A ( r 2 ) , (88) \widehat{y}_{A,\beta}^{2}(\mathbf{r}_{1})=\sum_{i,\alpha}C_{i\alpha}\int\mathrm{d}^{3}r_{2}\Phi_{\alpha\beta}(r_{12})g_{i}^{A}(\mathbf{r}_{2}), \tag*{(88)} y A , β 2 ( r 1 ) = i , α ∑ C i α ∫ d 3 r 2 Φ α β ( r 12 ) g i A ( r 2 ) , (88)
where the coefficients C i α C_{i\alpha} C i α and D j β D_{j\beta} D j β are determined by the regression. The necessary integrals are actually performed in reciprocal space since Eq. 85 yields Δ F A , L , β ( k ) \Delta F_{A,L,\beta}(k) Δ F A , L , β ( k ) rather than Δ F A , β ( r ) \Delta F_{A,\beta}(r) Δ F A , β ( r ) . At first glance, this appears to be a complicated and expensive fitting problem, but by separating the basis functions into their individual angular momentum channels and doing a separate regression for each channel, it becomes tractable. The details for this procedure are described in Appendix B1.
Because the functions h j A ( r ) h_{j}^{A}(\mathbf{r}) h j A ( r ) are localized, in the complete basis set limit Δ F A , β ( r ) = y ^ A , β 2 ( r ) \Delta F_{A,\beta}(\mathbf{r})=\widehat{y}_{A,\beta}^{2}(\mathbf{r}) Δ F A , β ( r ) = y A , β 2 ( r ) outside the augmentation region. Therefore, by defining
θ ~ α a u g ( r ) ≡ θ ~ α ( r ) + ∑ A ∑ i C i α A g i A ( r ) , (89) \widetilde{\theta}_{\alpha}^{\mathrm{aug}}(\mathbf{r})\equiv\widetilde{\theta}_{\alpha}(\mathbf{r})+\sum_{A}\sum_{i}C_{i\alpha}^{A}g_{i}^{A}(\mathbf{r}), \tag*{(89)} θ α aug ( r ) ≡ θ α ( r ) + A ∑ i ∑ C i α A g i A ( r ) , (89)
one can compute
G ~ ( r ) = ∑ β p β b ( b ~ ( r ) ) F ~ β a u g ( r ) (90) \widetilde{G}(\mathbf{r})=\sum_{\beta}p_{\beta}^{b}(\widetilde{b}(\mathbf{r}))\widetilde{F}_{\beta}^{\mathrm{aug}}(\mathbf{r}) \tag*{(90)} G ( r ) = β ∑ p β b ( b ( r )) F β aug ( r ) (90)
F ~ β a u g ( r ) = ∑ α ∫ d 3 r 2 Φ α β ( r 12 ) θ ~ α a u g ( r ) , (91) \widetilde{F}_{\beta}^{\mathrm{aug}}(\mathbf{r})=\sum_{\alpha}\int\mathrm{d}^{3}r_{2}\Phi_{\alpha\beta}(r_{12})\widetilde{\theta}_{\alpha}^{\mathrm{aug}}(\mathbf{r}), \tag*{(91)} F β aug ( r ) = α ∑ ∫ d 3 r 2 Φ α β ( r 12 ) θ α aug ( r ) , (91)
which satisfies the condition that the pseudo-feature G ~ ( r ) \widetilde{G}(\mathbf{r}) G ( r ) is equal to the all-electron feature G ( r ) G(\mathbf{r}) G ( r ) outside the augmentation region (within the limit of complete basis sets).
Within the core regions, F ~ β a u g ( r ) \widetilde{F}_{\beta}^{\mathrm{aug}}(\mathbf{r}) F β aug ( r ) is projected onto the radial support grids using a set of projectors in a very similar formalism to PAW itself. This quantity will be called F ~ A , β 1 , a u g ( r ) \widetilde{F}_{A,\beta}^{1,\mathrm{aug}}(\mathbf{r}) F A , β 1 , aug ( r ) , in keeping with the PAW notation, and it takes the form
F ~ A , β 1 , a u g ( r ) = ∑ j B j β A f ~ j A ( r ) + F ~ A , β 1 , 0 ( r ) \widetilde{F}_{A,\beta}^{1,\mathrm{aug}}(\mathbf{r})=\sum_{j}B_{j\beta}^{A}\widetilde{f}_{j}^{A}(\mathbf{r})+\widetilde{F}_{A,\beta}^{1,0}(\mathbf{r}) F A , β 1 , aug ( r ) = j ∑ B j β A f j A ( r ) + F A , β 1 , 0 ( r )
− ∑ j f ~ j A ( r ) [ ∫ d 3 r ′ p ~ j A ( r ′ ) F ~ A , β 1 , 0 ( r ′ ) ] (92) \qquad-\sum_{j}\widetilde{f}_{j}^{A}(\mathbf{r})\left[\int\mathrm{d}^{3}r'\,\widetilde{p}_{j}^{A}(\mathbf{r}')\widetilde{F}_{A,\beta}^{1,0}(\mathbf{r}')\right] \tag*{(92)} − j ∑ f j A ( r ) [ ∫ d 3 r ′ p j A ( r ′ ) F A , β 1 , 0 ( r ′ ) ] (92)
B j β A = d v ∑ g ∈ A F ~ β a u g ( r g ) p ~ j A ( r g ) . (93) B_{j\beta}^{A}=\mathrm{d}v\sum_{g\in A}\widetilde{F}_{\beta}^{\mathrm{aug}}(\mathbf{r}_{g})\widetilde{p}_{j}^{A}(\mathbf{r}_{g}). \tag*{(93)} B j β A = d v g ∈ A ∑ F β aug ( r g ) p j A ( r g ) . (93)
In the above equations, f ~ j A ( r ) \widetilde{f}_{j}^{A}(\mathbf{r}) f j A ( r ) and p ~ j A ( r ) \widetilde{p}_{j}^{A}(\mathbf{r}) p j A ( r ) are sets of atom-centered functions satisfying ⟨ f ~ i A ∣ p ~ j A ⟩ = δ i j \langle\widetilde{f}_{i}^{A}|\widetilde{p}_{j}^{A}\rangle=\delta_{ij} ⟨ f i A ∣ p j A ⟩ = δ ij , and p ~ j A ( r ) \widetilde{p}_{j}^{A}(\mathbf{r}) p j A ( r ) vanish for r > r c r>r_{c} r > r c so that the numerical integral of Eq. 93 can be performed efficiently on the FFT grid points inside the cutoff sphere of atom A A A . The symbol d v \mathrm{d}v d v represents the FFT grid volume element. This is directly analogous to the projector-partial wave pairs in PAW [67]. The construction of f ~ j A ( r ) \widetilde{f}_{j}^{A}(\mathbf{r}) f j A ( r ) and p ~ j A ( r ) \widetilde{p}_{j}^{A}(\mathbf{r}) p j A ( r ) is described in Appendix B2. F ~ A , β 1 , 0 ( r ) \widetilde{F}_{A,\beta}^{1,0}(\mathbf{r}) F A , β 1 , 0 ( r ) is an approximate convolution of θ ~ A , α 1 ( r ) \widetilde{\theta}_{A,\alpha}^{1}(\mathbf{r}) θ A , α 1 ( r ) , and while arbitrary, it helps make the f ~ j A ( r ) \widetilde{f}_{j}^{A}(\mathbf{r}) f j A ( r ) expansion more efficient. Using this construction, the all-electron and pseudo-features on the radial support grids are, respectively,
G A 1 ( r ) = ∑ β p β b ( b A 1 ( r ) ) ( F ~ A , β 1 , a u g ( r ) + y ^ A , β 1 ( r ) ) (94) G_{A}^{1}(\mathbf{r})=\sum_{\beta}p_{\beta}^{b}(b_{A}^{1}(\mathbf{r}))\left(\widetilde{F}_{A,\beta}^{1,\mathrm{aug}}(\mathbf{r})+\widehat{y}_{A,\beta}^{1}(\mathbf{r})\right) \tag*{(94)} G A 1 ( r ) = β ∑ p β b ( b A 1 ( r )) ( F A , β 1 , aug ( r ) + y A , β 1 ( r ) ) (94)
G ~ A 1 ( r ) = ∑ β p β b ( b ~ A 1 ( r ) ) F ~ A , β 1 , a u g ( r ) . (95) \widetilde{G}_{A}^{1}(\mathbf{r})=\sum_{\beta}p_{\beta}^{b}(\widetilde{b}_{A}^{1}(\mathbf{r}))\widetilde{F}_{A,\beta}^{1,\mathrm{aug}}(\mathbf{r}). \tag*{(95)} G A 1 ( r ) = β ∑ p β b ( b A 1 ( r )) F A , β 1 , aug ( r ) . (95)
The exchange correlation energy is then
E x c [ n ] = E x c [ n ~ , G ~ ] + ∑ A ( E x c [ n A 1 , G A 1 ] − E x c [ n ~ A 1 , G ~ A 1 ] ) (96) E_{\mathrm{xc}}[n] = E_{\mathrm{xc}}[\tilde{n},\tilde{G}] + \sum_A \left(E_{\mathrm{xc}}[n_A^1,G_A^1] - E_{\mathrm{xc}}[\tilde{n}_A^1,\tilde{G}_A^1]\right) \tag*{(96)} E xc [ n ] = E xc [ n ~ , G ~ ] + A ∑ ( E xc [ n A 1 , G A 1 ] − E xc [ n ~ A 1 , G ~ A 1 ] ) (96)
where all three terms are semilocal functionals of the density and of the nonlocal features. The first term is evaluated on the FFT grid and the second two on the radial support grids on the atoms.
The use of nonlocal features introduces a few additional terms to the forces and stresses not present for semilocal DFT. The force terms are
F A = F A s l + F A P A S D W (97) \mathbf{F}_A = \mathbf{F}_A^{\mathrm{sl}} + \mathbf{F}_A^{\mathrm{PASDW}} \tag*{(97)} F A = F A sl + F A PASDW (97)
F A P A S D W = ∑ α d v ∑ i g C i α A ∇ G i A ( r g ) ∂ E x c ∂ θ α ( r g ) + ∑ β d v ∑ j g ∂ E x c ∂ B ~ j β A ∇ p ~ j A ( r g ) F ~ β ( r g ) . (98) \mathbf{F}_A^{\mathrm{PASDW}} = \sum_\alpha\mathrm{d}v \sum_{ig} C_{i\alpha}^{A}\nabla G_i^A(\mathbf{r}_g)\frac{\partial E_{\mathrm{xc}}}{\partial\theta_\alpha(\mathbf{r}_g)}
+ \sum_\beta\mathrm{d}v \sum_{jg}\frac{\partial E_{\mathrm{xc}}}{\partial\tilde{B}_{j\beta}^{A}}\nabla\tilde{p}_j^A(\mathbf{r}_g)\tilde{F}_\beta(\mathbf{r}_g). \tag*{(98)} F A PASDW = α ∑ d v i g ∑ C i α A ∇ G i A ( r g ) ∂ θ α ( r g ) ∂ E xc + β ∑ d v j g ∑ ∂ B ~ j β A ∂ E xc ∇ p ~ j A ( r g ) F ~ β ( r g ) . (98)
The stress terms are
σ μ ν = σ μ ν s l + σ μ ν P A S D W (99) \sigma_{\mu\nu} = \sigma_{\mu\nu}^{\mathrm{sl}} + \sigma_{\mu\nu}^{\mathrm{PASDW}} \tag*{(99)} σ μν = σ μν sl + σ μν PASDW (99)
σ μ ν P A S D W = ∑ α ∑ A d v ∑ g { ( r g − R A ) μ ∂ E x c ∂ θ α ( r g ) [ ∑ i C i α A ∇ ν G i A ( r g ) ] + δ μ ν ∑ β ∑ A ∑ j ( ∂ E x c ∂ B ~ j β A B j β A ) } + ∑ β ∑ A d v ∑ g { [ ( r g − R A ) μ F ~ β ( r g ) ] [ ∑ j ∂ E x c ∂ B ~ j β A ∇ ν p ~ j A ( r g ) ] } + ∑ k G k μ G k ν ∣ G k ∣ ∑ β ( ∂ E x c ∂ F ~ β ( G k ) ) ∗ ∑ α Φ α β ′ ( ∣ G k ∣ ) θ α ( G k ) . (100) \begin{aligned}
\sigma_{\mu\nu}^{\mathrm{PASDW}} ={}& \sum_\alpha\sum_A \mathrm{d}v \sum_g \left\{(\mathbf{r}_g-\mathbf{R}_A)_\mu\frac{\partial E_{\mathrm{xc}}}{\partial\theta_\alpha(\mathbf{r}_g)}\left[\sum_i C_{i\alpha}^{A}\nabla_\nu G_i^A(\mathbf{r}_g)\right]\right.\\
&\left.{}+\delta_{\mu\nu}\sum_\beta\sum_A\sum_j\left(\frac{\partial E_{\mathrm{xc}}}{\partial\tilde{B}_{j\beta}^{A}}B_{j\beta}^{A}\right)\right\}\\
&+\sum_\beta\sum_A \mathrm{d}v \sum_g\left\{[(\mathbf{r}_g-\mathbf{R}_A)_\mu\tilde{F}_\beta(\mathbf{r}_g)]\left[\sum_j\frac{\partial E_{\mathrm{xc}}}{\partial\tilde{B}_{j\beta}^{A}}\nabla_\nu\tilde{p}_j^A(\mathbf{r}_g)\right]\right\}\\
&+\sum_k\frac{G_{k\mu}G_{k\nu}}{|\mathbf{G}_k|}\sum_\beta\left(\frac{\partial E_{\mathrm{xc}}}{\partial\tilde{F}_\beta(\mathbf{G}_k)}\right)^*\sum_\alpha\Phi'_{\alpha\beta}(|\mathbf{G}_k|)\theta_\alpha(\mathbf{G}_k). \tag*{(100)}
\end{aligned} σ μν PASDW = α ∑ A ∑ d v g ∑ { ( r g − R A ) μ ∂ θ α ( r g ) ∂ E xc [ i ∑ C i α A ∇ ν G i A ( r g ) ] + δ μν β ∑ A ∑ j ∑ ( ∂ B ~ j β A ∂ E xc B j β A ) ⎭ ⎬ ⎫ + β ∑ A ∑ d v g ∑ { [( r g − R A ) μ F ~ β ( r g )] [ j ∑ ∂ B ~ j β A ∂ E xc ∇ ν p ~ j A ( r g ) ] } + k ∑ ∣ G k ∣ G k μ G k ν β ∑ ( ∂ F ~ β ( G k ) ∂ E xc ) ∗ α ∑ Φ α β ′ ( ∣ G k ∣ ) θ α ( G k ) . (100)
In the above equations, F A s l \mathbf{F}_A^{\mathrm{sl}} F A sl and σ μ ν s l \sigma_{\mu\nu}^{\mathrm{sl}} σ μν sl refer to the force and stress equations for semilocal functionals within the PAW formalism (see Section IIID of Ref. [74]).
There are several significant contributions to the computational cost of the above approach for evaluating G ( r ) G(\mathbf{r}) G ( r ) , which all scale linearly or quasi-linearly with system size. The first contribution comes from computing the Fourier transforms of θ ~ α a u g ( r ) \tilde{\theta}^{\mathrm{aug}}_\alpha(\mathbf{r}) θ ~ α aug ( r ) and F ~ β a u g ( r ) \tilde{F}^{\mathrm{aug}}_\beta(\mathbf{r}) F ~ β aug ( r ) (as well as the functional derivatives with respect to these quantities), which is necessary to compute the convolutions like those in Eq. (91) in reciprocal space. The cost of these operations scales as Θ ( N α N F F T log ( N F F T ) ) \Theta(N_\alpha N_{\mathrm{FFT}}\log(N_{\mathrm{FFT}})) Θ ( N α N FFT log ( N FFT )) , where N α N_\alpha N α is the number of kernel interpolation points in Eq. (61) and N F F T N_{\mathrm{FFT}} N FFT is the size of the FFT grid used for integrating the XC energy. The second contribution arises from computing the convolutions in Eq. (91) in reciprocal space, and scales as Θ ( N α 2 N F F T ) \Theta(N_\alpha^2N_{\mathrm{FFT}}) Θ ( N α 2 N FFT ) . The third contribution comes from computing the PAW correction to θ ~ α a u g ( r ) \tilde{\theta}^{\mathrm{aug}}_\alpha(\mathbf{r}) θ ~ α aug ( r ) (the sum in Eq. (89) ). This and related terms scale as Θ ( N F F T , a t o m N a t o m N α ) \Theta(N_{\mathrm{FFT,atom}}N_{\mathrm{atom}}N_\alpha) Θ ( N FFT , atom N atom N α ) , where N F F T , a t o m N_{\mathrm{FFT,atom}} N FFT , atom is the number of FFT grid points inside the core region of an atom and N a t o m N_{\mathrm{atom}} N atom is the number of atoms. The last contribution consists of all the other PAW correction-related routines (e.g. computing C i α A C_{i\alpha}^{A} C i α A ), which depend only on the density on the atomic radial grids and therefore scale linearly with N a t o m N_{\mathrm{atom}} N atom . The overall scaling with system size N N N is then Θ ( N log ( N ) ) \Theta(N\log(N)) Θ ( N log ( N )) . Notably, none of the components of the algorithm scale with the number of k-points.
Results and Discussion
Using Gaussian process regression, we fit each of the model types in § 2 C (SL-GGA, NL-GGA, SL-MGGA, and NL-MGGA) to match the exact exchange energy on subsets of the GMTKN55 [10 ] and SOL62 [33, 75] databases, as well as a few dozen other training points. We provide a summary of our methods in this section and explain them in more technical detail in Appendix A.
To train the ML functionals, ground-state DFT calculations were first performed using the PBE functional [23]. Then, for each chemical system in the training set, the density and orbitals obtained from the PBE calculation were used to compute the features described in § 2 C3, as well the exact exchange energy of Eq. (2) . The computed features were used as the inputs to the Gaussian process to predict the exact exchange energy. See Appendices A 2 and A 3 for details.
The training and validation systems were chosen as described in Appendix A 4. The SOL62 database [75] contains cohesive energies for a mix of metals and non-metals, for which 9 were used for training, 22 for validation, and 31 for testing. The GMTKN55 database contains 55 sub-databases blocked into five categories: small-molecule properties (“small”), larger molecule reactions and isomerizations (“big”), barrier heights (“barrier”), intermolecular noncovalent interactions (“inter NCI”), and intramolecular noncovalent interactions (“intra-” tra NCI”). The training and validation data consist of twelve sub-databases of GMTKN55 and contain some representative systems from each category, but disproportionately taken from the “small” category. All data not used for training or validation is considered the GMTKN55 test set. This GMTKN55 partitioning results in 280 train data, 163 validation data, and 1062 test data. Further technical details for training are provided in Appendices A 6 and A 7.
The SL-GGA, NL-GGA, SL-MGGA, and NL-MGGA functionals used for benchmarking below are the functionals of each type with the best validation set performance, as described in Appendix A § 19.1 . One important technical note is that each functional was trained via delta-learning on top of some baseline exchange functional. We tried PBE and the Chachiyo GGA [76] as baselines, and Chachiyo proved more accurate for each functional class. However, as described in § 6 C below, slight numerical stability improvements were found when using the NL-MGGA with a PBE baseline (dubbed NL-MGGA-PBE). As described in more detail in § 6 C, we also found that retraining NL-MGGA-PBE with a more diverse training set from GMTKN55 (which is described in Appendix A § 17 ) results in improved numerical stability. Therefore, the resulting functional, dubbed NL-MGGA-DTR, was used for the applications in § 6 D–III G.
When testing and benchmarking the functionals, all calculations (both conventional and CIDER functionals) were performed self-consistently, with the exception of the SOL62 cohesive energies. These were obtained non-self-consistently from PBE orbitals, due to the difficulty of computing the isolated atom energies in plane-wave DFT. This is a small technical barrier that will be addressed in future work. All calculations in the main text used static geometries, with the exception of the defect calculations in § 6 G; in this section, the silicon lattice parameters and defect structures were optimized with both PBE and CIDER.
In § 6 A, we demonstrate that the NL-MGGA is capable of learning the exchange functional more accurately than the other feature sets, and in § 6 B we use 1 and 2-electron systems to provide intuition for why this is. Next, we fine-tune the accuracy and numerical stability of the NL-MGGA (§ 6 C) and demonstrate that the resulting exchange functional can be used to match the accuracy of hybrid DFT (§ 6 D).
Having designed an accurate exchange functional, we demonstrate that it can be used to improve band gap predictions over standard meta-GGAs in § 6 E. In § 6 F, we show that for large systems in plane-wave DFT, our implementation of the NL-MGGA form is similarly efficient to semilocal meta-GGAs and much more efficient than hybrid DFT. Lastly, in § 6 G, we take advantage of our new functional’s efficiency and accurate band gap predictions to compute charge transition levels of point defects in silicon, which is a notoriously challenging task for DFT due to the underestimation of band gaps and the large supercell sizes required to simulate isolated point defects [6 ] .
Taken together, the benchmarks provided below test the capacity of our ML models as well as their transferability across different chemical compositions and structures, including more complicated geometries like defect sites that are not present in the training data. However, our benchmarks mostly lack some system types and properties that play important roles in chemistry and materials science, including out-of-equilibrium geometries (with the exception of the barrier heights in GMTKN55), more complex materials like ternary and quaternary compounds and disordered phases, elastic and vibrational properties, etc. Transferability to these types of systems should not be assumed until future work provides more insight and further improvements to the ML models. For now, our benchmarks of CIDER functionals for predicting bond lengths and lattice constants in Supplemental Material Section S6 provide some initial insight into the ability of CIDER functionals to accurately predict structural properties in addition to energetic properties.
Accuracy of Different Model Types for Molecular and Solid-state Benchmarks
We assess the accuracy of our exchange functionals by substituting them for exact exchange in hybrid functionals to create a “surrogate hybrid” functional. For example, one of the simplest and most popular hybrid functionals is PBE0 [77], which mixes a 1/4 fraction of exact (Hartree–Fock, HF) exchange into the PBE functional [23]:
E x c [ n ] = 3 4 E x P B E [ n ] + 1 4 E x H F [ n ] + E c P B E [ n ] . (101) E_{\mathrm{xc}}[n] = \frac{3}{4}E_{\mathrm{x}}^{\mathrm{PBE}}[n] + \frac{1}{4}E_{\mathrm{x}}^{\mathrm{HF}}[n] + E_{\mathrm{c}}^{\mathrm{PBE}}[n]. \tag*{(101)} E xc [ n ] = 4 3 E x PBE [ n ] + 4 1 E x HF [ n ] + E c PBE [ n ] . (101)
In this section we perform so-called PBE0/CIDER calculations, where the exchange-correlation energy is
E x c [ n ] = 3 4 E x P B E [ n ] + 1 4 E x C I D E R [ n ] + E c P B E [ n ] . (102) E_{\mathrm{xc}}[n] = \frac{3}{4}E_{\mathrm{x}}^{\mathrm{PBE}}[n] + \frac{1}{4}E_{\mathrm{x}}^{\mathrm{CIDER}}[n] + E_{\mathrm{c}}^{\mathrm{PBE}}[n]. \tag*{(102)} E xc [ n ] = 4 3 E x PBE [ n ] + 4 1 E x CIDER [ n ] + E c PBE [ n ] . (102)
One can also generalize PBE0/CIDER with different fractions of CIDER exchange, giving the PBE0(α \alpha α )/CIDER form, which is used in later sections:
E x c [ n ] = ( 1 − α ) E x P B E [ n ] + α E x C I D E R [ n ] + E c P B E [ n ] . (103) E_{\mathrm{xc}}[n] = (1-\alpha)E_{\mathrm{x}}^{\mathrm{PBE}}[n] + \alpha E_{\mathrm{x}}^{\mathrm{CIDER}}[n] + E_{\mathrm{c}}^{\mathrm{PBE}}[n]. \tag*{(103)} E xc [ n ] = ( 1 − α ) E x PBE [ n ] + α E x CIDER [ n ] + E c PBE [ n ] . (103)
Throughout the results, we will also use abbreviations like PBE0/SL-GGA and PBE0/NL-MGGA, which indicate that the specific CIDER functionals used in PBE0/CIDER are the SL-GGA and NL-MGGA, respectively.
All ML functional results in this section refer to the PBE0/CIDER surrogate hybrid form. Because our exchange functionals are trained on exact exchange energies, they are “perfect” if they exactly reproduce the predicted reaction energies of PBE0 on the test set. To measure the error of the exchange functionals, we calculate the average error between PBE0/CIDER and PBE0 for each functional type on GMTKN55 and SOL62. All calculations on GMTKN55 are SCF calculations, but the SOL62 cohesive energy CIDER and PBE0 calculations are performed non-self-consistently with PBE orbitals. See Appendices A 2 and A 3 for details.
The average deviations of the machine-learned functionals from PBE0, using the mean of means (MoM) metric (i.e. the mean of the mean absolute deviations, or MADs, for the sub-databases of GMTKN55), are presented in Figure 1 . Results for all functionals include D4 dispersion corrections [78], with the PBE0/CIDER functionals using the PBE0 dispersion parameters. The nonempirical PBE and r2 ^{2} 2 SCAN [79] functionals, while not designed to reproduce PBE0, are included in Figure 1 to show typical deviations between GGAs, meta-GGAs, and hybrids.
Figure 1. Accuracy of PBE0/CIDER (Eq. (102) ) compared to PBE0 for each functional type on the GMTKN55 database of molecular properties, using the MoM error metric. The database subsets are [10 ] : “all”: Full database; “small”: Basic properties and reaction energies for small systems; “big”: Reaction energies for large systems and isomerization reactions; “barrier”: Barrier heights; “inter NCI”: Intermolecular noncovalent interactions; “intra NCI”: Intramolecular noncovalent interactions.
While the trends vary slightly in the different subsets of GMTKN55, the key trend is that the quality of the fit to exact exchange improves going from SL-GGA to NL-GGA to SL-MGGA to NL-MGGA. Notably, the SL-GGA and NL-GGA suffer from out-of-distribution transferability issues, resulting in larger deviations from PBE0 even compared to the PBE functional on the test set. On the other hand, the ML meta-GGAs are more transferable outside the training data than the ML GGAs, so they match PBE0 more closely than PBE and r2 ^{2} 2 SCAN on the test set. The NL-MGGA is more accurate than the SL-MGGA, achieving an MoM of less than 1 kcal/mol on the test set. The fact that the NL-MGGA outperforms both the NL-GGA and SL-MGGA suggests that the kinetic energy and nonlocal features provide non-redundant information, meaning that nonlocal density features cannot replace the kinetic energy density nor vice versa. As shown in Figure 2 , the same general trends apply for SOL62; the NL-MGGA is the most accurate and transferable ML functional for reproducing PBE0, followed by the SL-MGGA and then the NL-GGA.
Figure 2. Average deviation of PBE and the PBE0/CIDER ML functionals from PBE0 on the SOL62 cohesive energies database.
Because GMTKN55 contains a variety of properties and systems, the performance of the functionals is somewhat dependent on the choice of error metric. However, the general trends are the same as for the MoM metric. See Supplemental Material Section S2A for a discussion of the impact of error metric on the results.
Impact of Nonlocality and Derivative Discontinuity on Model Accuracy
Overall, the SL-MGGA model is more accurate than the NL-GGA model across the databases tested here. However, the fact that NL-GGA improves over SL-GGA and that NL-MGGA improves over SL-MGGA suggests that the nonlocal features contribute useful, non-redundant information about the density for exchange energy prediction. This raises the question: What types of systems require the nonlocal features, and which require the semilocal orbital dependence of the kinetic energy density? We hypothesize that the derivative discontinuity of the exchange functional [80] plays a key role.
Because only the kinetic energy density—and not the nonlocal features of the NL-GGA—includes a derivative discontinuity passing through integer electron number, the NL-GGA (as well as any smooth, pure density functional) cannot be expected to accurately capture energy differences involving a change in electron number, such as ionization energies and electron affinities. On the other hand, existing semilocal meta-GGAs are fundamentally limited by their locality and cannot be expected to improve over GGAs for energy differences between systems of like electron number. This is especially true of one-electron systems, for which the kinetic energy is simply determined by the density gradient.
To explore this hypothesis, we set up two simple sets of systems. The first set is one-electron systems for different arrangements of protons: the hydrogen atom, the H2 + _2^{+} 2 + molecular ion, the linear H3 2 + _3^{2+} 3 2 + molecular ion, and the triangular H3 2 + _3^{2+} 3 2 + molecular ion, all with bond lengths of 0.7 Å, as illustrated in Figure 3 . Because each system contains only one electron with different degrees of delocalization, we expect that only the functionals with nonlocal features can accurately predict energy differences between these systems. Table I shows the MAD between the ML exchange functionals and Hartree-Fock for the energy differences between the hydrogen atom and the molecular ions. As expected, the SL-GGA and SL-MGGA are both less accurate than NL-GGA and NL-MGGA. This confirms that the nonlocal features successfully characterize single-electron delocalization in a way that is not feasible with semilocal functionals.
Figure 3. Geometries and density distributions of the 1-electron systems dataset. a) Hydrogen atom. b) H2 + _2^{+} 2 + . c) H3 2 + _3^{2+} 3 2 + -linear. d) H3 2 + _3^{2+} 3 2 + -triangular.
The second set of systems consists of the He+ ^{+} + ion, the He singlet (closed-shell ground state), and the He triplet (open-shell excited spin state with two same-spin electrons). Because the exchange energy has a discontinuous derivative as the number of electrons of a given spin passes through an integer [80, 81], the energy differences between He+ ^{+} + and He triplet and between He singlet and He triplet can only be described accurately by functionals that capture this derivative discontinuity. As shown in Table I, the absolute deviations from Hartree Fock for the energy differences between these three systems are larger for SL-GGA and NL-GGA than for SL-MGGA and NL-MGGA. This indicates that the derivative discontinuity included in the kinetic energy density is essential for capturing energy differences related to electron addition/removal. Notably, the NL-MGGA is the most accurate rate of the four functional types for both datasets.
Table 1. Mean absolute deviation (kcal/mol) of the four types of ML functional for simple 1 and 2-electron systems compared to Hartree-Fock. a ^{a} a The 1-electron system dataset consists of the energies of H2 + _2^{+} 2 + , H3 2 + _3^{2+} 3 2 + -linear, and H3 2 + _3^{2+} 3 2 + -triangular referenced to the energy of the isolated hydrogen atom. All nearest-neighbor bond lengths are 0.7 Å. b ^{b} b The 2-electron system dataset consists of three energy differences between He+ ^{+} + , He-singlet, and He-triplet, where He-triplet is the neutral helium atom excited state with both electrons having the same spin.
These observations regarding the role of the derivative discontinuity explain why a previous attempt at machine learning a GGA resulted in more accurate atomization energies but less accurate ionization potentials than PBE [31]. It also provides a possible reason for the poor generalization of the SL-GGA and NL-GGA in Figure 1 ; training on properties like ionization potentials and electron affinities that are difficult to describe with orbital-free functionals could lead to over-fitting and decrease the accuracy of other properties. Likewise, the role of nonlocality explains why Kovács et al. [13] found that existing meta-GGAs, as well as 25 meta-GGAs they empirically trained, suffer from a trade-off between cohesive energy accuracy and band gap accuracy. Meta-GGAs are limited in their ability to describe both these properties at once due to their semilocality. In addition, while we do not exhaustively try many possible NLDFs to conclusively show that they can only yield accurate models when paired with orbital-dependent features, the NLDFs we have implemented do need to be paired with a meta-GGA formalism to improve their accuracy. These findings provide evidence that to accurately and transferably fit a nonlocal, orbital-dependent quantity like the exchange energy, an ML functional should contain both orbital dependence and nonlocality in its feature set.
Fine-tuning for Numerical Stability
Because XC functionals are used within SCF calculations, they must be numerically stable and consistently achieve electronic convergence on real systems to be useful for practical applications. For the GMTKN55 dataset, we found that some systems do not quite converge to our default convergence thresholds, which were 10 − 8 10^{-8} 1 0 − 8 Hartree atomic unit (Eh) for the total energy and 10 − 4 10^{-4} 1 0 − 4 Eh for the orbital gradient of the energy. (See Appendix A9 for more details on how we remedied convergence problems.) We found that these calculations could only be converged with a 10 − 7 10^{-7} 1 0 − 7 Eh energy threshold and 2 × 10 − 3 2 \times10^{-3} 2 × 1 0 − 3 Eh gradient threshold. Interestingly, this problem is least severe for the NL-MGGA, as illustrated in Table II , which shows the number of systems for each functional type which did not converge to the 10 − 8 10^{-8} 1 0 − 8 Eh energy and 10 − 4 10^{-4} 1 0 − 4 Eh gradient thresholds. We believe this is because the increased model dimensionality of the NL-MGGA allows it to fit the exchange energy with a smoother function of the input features.
Table II. The number of systems in the GMTKN55 database for which a given functional did not achieve SCF convergence with thresholds of 10 − 8 10^{-8} 1 0 − 8 Eh for total energy and 10 − 4 10^{-4} 1 0 − 4 Eh for the orbital gradient of the energy. All functionals use the PBE0/CIDER form (Eq. (102) ).
Because the NL-MGGA still sees some convergence problems, we decided to explore two modifications of the NL-MGGA for improved numerical stability. The first is the NL-MGGA-PBE, which has the same hyperparameters and feature shape as the NL-MGGA but uses the PBE functional as a baseline rather than the Chachiyo functional [76]. NL-MGGA-PBE was screened during the validation procedure (Appendix A § 19.1 ) but was found to be slightly less accurate than the NL-MGGA with the Chachiyo baseline. However, only 1 system sees a convergence issue in Table II with NL-MGGA-PBE, compared to 3 with NL-MGGA. We also tried training an NL-MGGA model with the PBE baseline and a more chemically diverse subset of GMTKN55, as described in Appendix A § 17 . We refer to this new train/validation/test partition as DTR (for “diverse training”), and it yields
the NL-MGGA-DTR, for which all GMTKN55 systems achieve an orbital gradient convergence of 10 − 4 10^{-4} 1 0 − 4 Eh or lower. In fact, all but 10 of the 2462 GMTKN55 systems converge to within 10 − 9 10^{-9} 1 0 − 9 Eh for energy and 10 − 4.5 10^{-4.5} 1 0 − 4.5 Eh for orbital gradient with NL-MGGA-DTR, and the rest to within 10 − 8 10^{-8} 1 0 − 8 and 10 − 4 10^{-4} 1 0 − 4 Eh, respectively.
As shown in Figure 4 , which displays MoM errors of PBE0/CIDER versus PBE0, the accuracy of these three NL-MGGA variants is similar across the GMTKN55 dataset. The three functional variants also have nearly identical performance for the SQL62 cohesive energies, with MADs compared to PBE0 of 0.13 eV/atom for all three. Therefore, for the remainder of this work, we adopt the NL-MGGA-DTR model as our primary functional of choice for applications, and we name it CIDER23X-NL-MGGA-DTR. This functional is provided (along with several others used in this work) with an early release version of the CiderPress package as described in Section V, and it is the recommended functional for any researchers interested in running DFT calculations with CIDER.
Figure 4. Accuracy of GMTKN55 predictions for the NL-MGGA and the NL-MGGA-PBE and NL-MGGA-DTR variants described in the text (all using the PBE0/CIDER form), compared to PBE0. Train/test partition results are not shown because the train and test data are different for the NL-MGGA-DTR functional.
In addition to the stability of a functional for the purpose of SCF convergence, one might also be interested in other metrics of numerical stability, such as the accuracy of the approximate evaluation of nonlocal density features described in Sections II E and II F and the convergence of the CIDER functional with respect to the size of the grid used to integrate the XC energy. These characteristics are discussed in the Supplemental Material Section S8.
Obtaining Hybrid DFT Accuracy
In this section, we show that by substituting NL-MGGA-DTR for exact exchange in hybrid functionals, accurate predictions of the XC energy can be obtained for molecular systems without additional parameter tuning. In particular, we look at PBE0/CIDER (Eq. (102) ) and another type of functional called PW6B95/CIDER. The original PW6B95 functional [82] is
E x c P W 6 B 95 [ n ] = 0.28 E x H F [ n ] + E x c , s l P W 6 B 95 [ n ] , (104) E_{\mathrm{xc}}^{\mathrm{PW6B95}}[n] = 0.28E_{\mathrm{x}}^{\mathrm{HF}}[n] + E_{\mathrm{xc,sl}}^{\mathrm{PW6B95}}[n], \tag*{(104)} E xc PW6B95 [ n ] = 0.28 E x HF [ n ] + E xc , sl PW6B95 [ n ] , (104)
where E x c , s l P W 6 B 95 E_{\mathrm{xc,sl}}^{\mathrm{PW6B95}} E xc , sl PW6B95 is a meta-GGA term. Just like with PBE0/CIDER, we substitute CIDER in for Hartree-Fock exchange to obtain PW6B95/CIDER:
E x c P W 6 B 95 [ n ] = 0.28 E x C I D E R [ n ] + E x c , s l P W 6 B 95 [ n ] . (105) E_{\mathrm{xc}}^{\mathrm{PW6B95}}[n] = 0.28E_{\mathrm{x}}^{\mathrm{CIDER}}[n] + E_{\mathrm{xc,sl}}^{\mathrm{PW6B95}}[n]. \tag*{(105)} E xc PW6B95 [ n ] = 0.28 E x CIDER [ n ] + E xc , sl PW6B95 [ n ] . (105)
We chose to construct PW6B95/CIDER functionals because the original PW6B95 [82] is among the most accurate functionals on the GMTKN55 database [10 ] [36]. For PBE0/NL-MGGA-DTR, PW6B95/NL-MGGA-DTR, and several semilocal [23, 53, 79, 83] and hybrid [27, 28, 54, 77, 82] functionals, Figure 5 shows the MoM error on all of GMTKN55 as well as the mean absolute error (MAE) on the test partition used for NL-MGGA-DTR. The errors are relative to the high-accuracy benchmark data collected in the GMTKN55 database [10 ] . All functionals include D4 dispersion corrections [78] except for B97M-V [53], ω \omega ω B97X-V [28], and ω \omega ω B97M-V [54], which all use VV10 dispersion [60]. PBE0/CIDER calculations use PBE0 dispersion parameters, and PW6B95/CIDER calculations use PW6B95 dispersion parameters. We show conventional semilocal functionals in shades of blue, conventional hybrid functionals in orange, and CIDER functionals in green.
Figure 5. MoM error for all of GMTKN55 (left) and MAD on the DTR test set (right) relative to GMTKN55 reference values, for semilocal, hybrid, and CIDER functionals.
As Figure 5 shows, the PBE0/NL-MGGA-DTR functional is comparable in accuracy to r2 ^2 2 SCAN, but the combination of systematic errors in the NL-MGGA-DTR functional and in PBE0 itself prevent it from outperforming the most accurate meta-GGA (B97M-V) or any of the hybrid functionals. On the other hand, because PW6B95 is much more accurate than PBE0 on the GMTKN55 dataset, PW6B95/NL-MGGA-DTR is more accurate than all semilocal functionals explored here. It is also more accurate than the PBE0 and B3LYP [27] hybrid functionals and equivalent in accuracy to the range-separated hybrid ω \omega ω B97X-V [28]. This shows that the NL-MGGA-DTR exchange functional can be used to calculate molecular reaction energies with an accuracy that was previously not feasible without exact exchange mixing.
As with Section III A, the analysis of the errors on GMTKN55 depends somewhat on the error metric used. These nuances are discussed in further detail in Supplemental Material Section S2B, which also provides a comparison between the nonlocal and semilocal meta-GGA models.
Using CIDER for Improved Band Gap Prediction
The band gap problem [84, 85], in which semilocal DFT tends to drastically underestimate the band gaps of solids, stands in the way of a variety of important applications like high-throughput screening for optical and electronic properties and semiconductor point defect physics [5 , 6 ] [86]. By introducing a derivative discontinuity into the XC functional, meta-GGAs partly mitigate this problem [87], but they still tend to significantly underestimate band gaps [88]. Exceptions include mBJ [89] and TASK [12], which each have important drawbacks. The mBJ functional provides the exchange potential only, not the energy [89], and TASK does not predict atomization energies and lattice constants as well as SCAN [12–14]. This reinforces the discussion in Section III B regarding the accuracy trade-offs meta-GGAs must make due to their semilocality.
By introducing nonlocality into the exchange energy, PBE0/NL-MGGA-DTR (Eq. (102) or Eq. (103) with α = 0.25 \alpha= 0.25 α = 0.25 ) improves band gap predictions over PBE and r2 ^2 2 SCAN, as shown in Table III . This improvement occurs even though the CIDER functionals are not fit to solid-state band gaps. The PBE0/SL-MGGA also improves band gap predictions over r2 ^2 2 SCAN, but significantly less so than PBE0/NL-MGGA-DTR. The database used for testing covers 255 of the 472 band gaps in the database published by Borldo et al. [45], with La, Yb, and Th-containing systems being excluded as well as systems with a greater than 0.05 eV difference between the spin-orbit-corrected and non-spin-orbit-corrected band gaps (as computed with PBE by Borldo et al. [45]). See Appendix A11 for details, and see Supplemental Material Section S3 for error metrics for all 453 solids in the database that do not contain La, Yb, or Th.
Table III. Error statistics for 255 band gaps from the database published by Borlido et al. [45], in eV. The ML functionals use the PBE0(α \alpha α )/CIDER formalism of Eq. (103) .
To provide a clearer picture of the improved band gap prediction provided by the NL-MGGA-DTR functional, Table IV shows the band gap predictions for a selected subset of systems that are commonly studied [11]. In addition, Figure 6 and Figure 7 contain band structures for silicon (Si) and boron phosphide (BP), respectively, which show that PBE0/NL-MGGA-DTR provides similar band structures to PBE and r2 ^2 2 SCAN but with a wider band gap.
Figure 6. Band structures of silicon with three different functionals. The black dotted line is positioned at the valence band maximum, and the orange dotted line is located at the experimental band gap of 1.17 eV. (a) PBE. (b) r2 ^{2} 2 SCAN. (c) PBE0/NL-MGGA-DTR.
Table IV. Band gaps for a small subset of solids (in eV). CIDER refers to PBE0/NL-MGGA-DTR.
Figure 7. Band structures of boron phosphide with three different functionals. The black dotted line is positioned at the valence band maximum, and the orange dotted line is located at the experimental band gap of 2.10 eV. (a) PBE. (b) r 2 r^{2} r 2 SCAN. (c) PBE0/NL-MGGA-DTR.
There is room for improvement in these band gaps, since PBE0/NL-MGGA-DTR still systematically underestimates band gaps compared to the state-of-the-art HSE06 [90, 91] range-separated hybrid functional, as well as the PBE0 functional to which PBE0/CIDER is an approximation. One possible solution is to tune the fraction of CIDER exchange, which is already done with exact exchange within hybrid DFT to optimize functional accuracy for different applications [92]. As shown in Table III , PBE0(α \alpha α )/NL-MGGA-DTR with α = 0.35 \alpha= 0.35 α = 0.35 has increased band gaps over α = 0.25 \alpha= 0.25 α = 0.25 and similar error statistics to HSE06. However, one drawback of this approach is that because NL-MGGA-DTR increases band gaps less than exact exchange for a given mixing fraction, one might need a large fraction of NL-MGGA-DTR to fit band gaps. This large fraction might not provide accurate predictions of other material properties. Alternative approaches to improve CIDER band gaps include
using physical intuition and exact constraints to tune the derivative discontinuity contribution from the kinetic energy density (as is done in the TASK functional [12]), introducing more nonlocal features to improve the accuracy of the functional in general, and training on fundamental band gaps (i.e. differences between ionization potentials and electron affinities). However, considering the drastic increase in computational cost associated with using HSE06 or PBE0 compared to semilocal DFT, the ability to mitigate the underestimation of band gaps without explicitly fitting them—and while accurately predicting other properties like bond energies—already serves as a major step toward solving the band gap problem.
The key benefit of the nonlocal features used to learn the exchange functional is that the quasi-linear-scaling algorithm used to compute them within plane-wave DFT (Sections IID and IIF) is much faster than the evaluation of exact exchange for large systems and does not depend on the number of k-points. To illustrate this efficiency, Figure 8 a shows the wall time to perform a ground-state SCF calculation for a 216-atom diamond supercell with semilocal and PBE0/CIDER functionals in GPAW [68, 69]. These wall times are compared to those of the hybrid DFT implementation in Quantum ESPRESSO [93, 94], which uses the adaptively compressed exchange (ACE) algorithm [17] to compute the exact exchange energy as efficiently as possible. A Γ \Gamma Γ -point only k-point mesh was used for these calculations. The CIDER functionals, including those with nonlocal features, all take less than 170 seconds (compared to 60 seconds for PBE and 79 seconds for r 2 r^{2} r 2 SCAN). The total wall time for PBE0 is roughly 6.5 times slower than the NL-MGGA-DTR functional, in spite of the use of the ACE algorithm to accelerate the exact exchange calculation. Also, Quantum ESPRESSO has a general performance advantage over the plane-wave version of GPAW, with the 216-atom PBE calculation in Figure 8 a requiring
Figure 8. Computational cost for plane-wave DFT SCF calculations of large systems with different functionals. The semilocal (blue) and CIDER (green) calculations were performed with GPAW on one 64-core Intel Ice Lake node for diamond and two 64-core nodes for silicon. The PBE0 calculation (orange) was performed on one 64-core node with the Quantum ESPRESSO (QE) code [93, 94]. An additional PBE calculation was also performed with QE to compare the computational cost of GPAW and QE. (a) Total SCF calculation wall time for diamond 216-atom bulk supercell with Γ \Gamma Γ -point sampling only. (b) Total SCF calculation wall times for diamond and silicon neutral vacancy formation energies in a 216-atom supercell with 2 × 2 × 2 2 \times2 \times2 2 × 2 × 2 Monkhorst-Pack k-point mesh [95].
60 seconds for GPAW and only 21 seconds for Quantum ESPRESSO. Therefore, more performance optimization in both the GPAW code and the CIDER feature evaluation might further increase the computational advantage of CIDER over hybrid DFT.
The efficiency of CIDER functionals for large systems makes it practical to perform more complicated calculations like defect formation energies. Figure 8b shows the wall time for computing the neutral vacancy formation energies in diamond and silicon using 216-atom supercells for semilocal and PBE0/CIDER functionals. These calculations used a 2 × 2 × 2 2 \times2 \times2 2 × 2 × 2 Monkhorst-Pack k-point mesh [95]. All of the functionals are of comparable computational cost, with the NL-MGGA-DTR model being only about 25% slower than r2SCAN for diamond and 50% slower for silicon. This small decrease in efficiency comes with a significant benefit: As shown in Table V , of the functionals tested, the NL-MGGA model and its variants give the closest match to PBE0 for predicting these vacancy formation energies, with PBE0/NL-MGGA-DTR giving deviations of 0.18 eV and 0.23 eV for diamond and silicon, respectively. The HSE06 and PBE0 formation energies were computed using VASP [96–98]. See Appendix A § 23 for further calculation details.
Table V. The formation energies of the neutral vacancies in diamond and silicon (in eV), computed with 216-atom supercells and a 2 × 2 × 2 2 \times2 \times2 2 × 2 × 2 Monkhorst-Pack k-point mesh [95]. All CIDER functionals use the PBE0/CIDER surrogate hybrid form.
Charged Defect Transition Levels in Silicon
One of the consequences of the band gap problem is that transition levels in charged defects are not accurately described by semilocal DFT, so hybrid DFT is typically used when accurate transition level predictions are needed. In this section, we show that by improving band gap prediction, CIDER is able to accurately predict defect transition levels in silicon. The transition levels associated with the single vacancy [99], as well as the boron [100], phosphorus [101], copper [102], and sulfur [103, 104] substitutionals, were computed as described in Appendix A § 23.1 , using a 512-atom supercell, 2 × 2 × 2 2 \times2 \times2 2 × 2 × 2 Γ \Gamma Γ -centered k-point mesh, and full structural relaxation for each defect and functional.
Figure 9 shows the computed transition levels compared to experiment with two different finite-size correction schemes: the Freysoldt finite-size corrections [105] (Figure 9 a) and the potential alignment correction (Figure 9 b). We computed the transition levels with PBE and PBE0(0.3)/NL-MGGA-DTR (Eq. (103) with α = 0.3 \alpha= 0.3 α = 0.3 ). The latter functional was chosen because its band gap matches the 1.17 eV experimental gap; however, PBE0/NL-MGGA-DTR yields similar results (see Supplemental Material § 32 ). Supplemental Table S13 provides a more detailed list of all computed transition levels. In the rest of this section, PBE0(0.3)/NL-MGGA-DTR is referred to simply as CIDER for brevity.
Figure 9. Electronic transitions levels for the single vacancy and several substitutional defects in silicon computed using different functionals and finite-size correction schemes. The grey bars represent the band gap for the given functional (or experiment). The experimental references are from the following sources: vacancy [99], boron [100], phosphorus [101], copper [102], and sulfur [103, 104]. CIDER refers to PBE0(0.3)/NL-MGGA-DTR. (a) Transition levels computed using Freysoldt finite-size corrections [105]. (b) Transition levels computed using potential alignment correction only.
PBE and CIDER both give reasonable descriptions of the + + / 0 ++/0 + + /0 transition level in the silicon vacancy and the boron acceptor level. However, the phosphorus donor level is not well-described by PBE due to its severe underestimation of the band gap; CIDER remedies the band gap underestimation and therefore predicts a transition level in good agreement with experiment. To correct for band gap underestimation, one could predict defect levels from PBE by scaling the gap (and with it, the transition levels) to the experimental gap, but this makes it difficult to distinguish true shallow levels from deep levels that happen to sit near the (too-low) PBE conduction band edge [5 ] . For example, PBE predicts a transition level of 0.66 eV for the − / − − -/-- − / − − transition of CuSi (Figure 9 a), 0.05 eV above the conduction band minimum. Without further information, it is unclear whether this state is resonant in the conduction band or localized in the gap. CIDER, on the other hand, places the − / − − -/-- − / − − level inside the gap at 0.97 eV, in good agreement with the 1.00 eV level from experiment.
More generally, Figure 9 a shows that paired with the Freysoldt correction, CIDER matches each level in the figure with an error of 0.10 eV or less compared to experiment, with the exception of the + / 0 +/0 + /0 level in CuSi. For this transition, the experimental level is 0.20 eV [102], while the CIDER level is 0.02 eV with the Freysoldt correction and 0.07 eV with the potential alignment correction only. The HSE06 level was previously reported to be 0.21 eV by Sharan et al. [106], in good agreement with experiment. However, Sharan et al. used a 64-atom supercell with a 2 × 2 × 2 2 \times2 \times2 2 × 2 × 2 Monkhorst-Pack k-point mesh [95]. This supercell size is relatively small, and the k-point mesh is both small and fails to sample the band edges, which can interfere with the description of relatively delocalized defect states. When we recalculated the CIDER transition level using Sharan et al.’s supercell size and k-point mesh, we obtained 0.18 eV with Freysoldt corrections, in good agreement with their result and with experiment. In the Supplemental Material (Section S7A), we provide further evidence that the HSE06 level itself is not converged with respect to k-point mesh. This finding illustrates the importance of using well-converged supercell sizes and k-point meshes in point defect calculations, a requirement that is made much easier by the efficiency of the CIDER functionals compared to hybrid DFT. Why this particular transition level is underestimated by both hybrid DFT and CIDER is an interesting question warranting future investigation.
In addition to yielding more accurate transition levels, CIDER predicts somewhat different charge distributions than PBE as well. Figure 10 shows the charge density differences between charge states for several of the transition levels in Figure 9 . There are a few notable takeaways from these plots. First, as expected, the charge density differences are larger for transitions that result in large ionic displacements (as indicated by the MaxDisp in the bottom of each plot). Second, these ionic displacements and resulting charge distortions are noticeable more than 5 Å away from the defect site for some transitions, indicating the importance of large supercells. Lastly, a slightly larger fraction of the charge associated with the transition is contained within a 5 Å sphere around the defect site for CIDER than for PBE (as indicated by the Loc. metric in the bottom of each plot), suggesting that CIDER localizes the defect charge more effectively than PBE. This is reassuring because unphysical charge delocalization is one of the key driving factors of erroneous charge transition level predictions in semiconductor point defects [5 ] .
In summary, our results indicate that the CIDER approach could be a route toward high-accuracy defect calculations at similar cost to semilocal DFT, which would significantly broaden the potential scope of point defects research. In particular, it could be used to improve the accuracy of high-throughput point defects studies [107], which are currently only computationally feasible with semilocal functionals like PBE.
Conclusion
We have demonstrated that by combining the kinetic energy density (a semilocal, orbital-dependent quantity) with nonlocal, density-dependent features, it is possible to design functionals with comparable accuracy to hybrid DFT at a drastically reduced computational cost for plane-wave DFT calculations. We can do this by learning exact exchange energies, without the use of any high-accuracy reference data from experiments or wave function theory. We have shown that our nonlocal meta-GGA model has higher accuracy than semilocal functionals on both molecular benchmarks and solid-state band gap prediction, and that these improvements can assist with applications such as semiconductor point defect calculations.
The developments presented here serve a variety of purposes. As is, the exchange functional we designed can be used as an exact exchange surrogate in scenarios where hybrid DFT is impractical, such as large extended systems. This expands the scope of calculations that can be performed at hybrid DFT accuracy. In addition, the techniques we have introduced to learn smooth, physically constrained functionals and implement efficient nonlocal features could be used in other contexts, such as orbital-free DFT [73] and van der Waals functional design [59, 70, 108].
Figure 10. Change in charge distribution in the (001) plane (integrated over the z z z axis) for several charge transition levels in point defects of silicon. Blue shades represent an increase in electron density, while red shades indicate a decrease in electron density. “MaxDisp” is the maximum ionic displacement between the charge states, and “Loc.” is the fraction of the charge difference contained within a 5 Å sphere around the defect site. Loc. is greater than 1 for the vacancy ++/0 transition because more than 2 electrons are drawn in to the defect region upon addition of an electron pair, depleting electron density in the surrounding region.
Importantly, the ability to systematically vary the descriptor complexity in our machine learning approach allows us to extract physically nontrivial information about the nature of the exchange interactions. Specifically, we find that both nonlocality and derivative discontinuity are independently important for the structure of the exchange energy density functional. This clarifies a key limitation of existing semilocal functionals.
Lastly, our approach can be generalized to learn the full XC functional, providing a path to highly transferable and universal functionals capable of describing challenging and heterogeneous chemistry. We anticipate that these developments will contribute to significantly improving the cost-accuracy trade-off of DFT, accelerating the pace and efficacy of materials and chemical research by providing a stronger theoretical backing in various sub-fields.
Data Availability
The code to run CIDER calculations in both the PySCF and GPAW codes is available at the following Github repository: https://github.com/mir-group/CiderPressLite. The SL-GGA, NL-GGA, SL-MGGA, NL-MGGA, NL-MGGA-PBE, and NL-MGGA-DTR functionals are available for download using the instructions in the Github repository.
Acknowledgments
The authors thank Jennifer Coulter and Stefano Falletta for help with the hybrid DFT benchmark calculation. The authors also thank Stefan Riemelmoser and Georg Kresse for useful discussions. This work was supported by the National Defense Science and Engineering Graduate (NDSEG) Fellowship Program under contract FA9550-21-F-0003, the Camille and Henry Dreyfus Foundation Grant No. ML-22-075, and the Department of Navy award N00014-20-1-2418 issued by the Office of Naval Research. Computational resources were provided by the Harvard University FAS Division of Science Research Computing Group.
Computational Details
Overview
To explore the effectiveness of different features for learning the exchange energy, four types of functional were trained, with the model and feature vector details provided in § 2 C:
SL-GGA: A semilocal GGA model based on the reduced squared gradient s 2 s^2 s 2 (Eq. (9) ).
NL-GGA: A nonlocal model based on s 2 s^2 s 2 and three nonlocal features with Eqs. (20) and (21) for the exponents.
SL-MGGA: A semilocal meta-GGA model based on s 2 s^2 s 2 and the iso-orbital indicator α \alpha α (Eq. (11) ).
NL-MGGA: A nonlocal model based on s 2 s^2 s 2 , α \alpha α , and three nonlocal features with Eqs. (22) and (23) for the exponents.
The purpose of training semilocal GGA and meta-GGA models is to test the expressive power of the nonlocal descriptors. In our previous work (see Table S1 of Ref. [35]), we found that a standard, semilocal meta-GGA could not effectively learn the exchange energy density, but in this work, we determined that a semilocal meta-GGA can learn the total exchange energy much more effectively than it can learn the exchange energy density.
Table S1. WTMAD-2 (kcal/mol) for a few CIDER functionals. PW6B95/NL-MGGA was not evaluated on GMTKN55.
The two primary datasets used in this work were the GMTKN55 [10 ] and SOL62 [33, 75] databases. The GMTKN55 database contains 1505 molecular reaction energies based on 2462 single-point calculations, covering five categories: small-molecule properties, larger molecule reactions and isomerization, barrier heights, intermolecular noncovalent interactions, and intramolecular noncovalent interactions. The SOL62 dataset consists of 42 non-metals and 20 metals as categorized by Zhang et al. [75]. We follow Trepte and Voss [33] in excluding Pb and Th from Zhang et al.’s initial set of 64 solids. Twelve subsets of GMTKN55 and half of SOL62, along with a few other systems, were used for training and validation, as detailed in Section A 4.
Throughout this work, we used pymatgen [109] and ase [110] for structure manipulation and calculation setup, fireworks [111] for automated workflow management, and Scikit-learn [112] for the Gaussian process models. The computational details for the PySCF and GPAW calculations are provided in Appendices A 2 and A 3, respectively. Appendices A 4 and A § 17 describe the training and validation set selection, and Appendix A § 18 covers the selection of control points for the sparse Gaussian processes. Appendix A § 19 describes the selection of noise hyperparameters and training/validation loss functions, and Appendix A § 19.1 describes the list of hyperparameters tested for each model and the model selection procedure. Appendix A § 20 describes how we handled convergence issues for CIDER models on GMTKN55, and Appendix A § 21 describes the PySCF and GPAW settings that are specific to the CIDER functionals. Appendix A § 22 describes the band gap benchmark methods. Appendix A § 23 describes the computational cost benchmarking of CIDER for plane-wave DFT calculations. Finally, Appendix A § 23.1 describes our methodology for computing charged defect transition levels.
Molecular Calculations and Reference Data Generation
Molecular systems, including the nanoclusters discussed in Appendix A 4, were computed using the PySCF code [61, 62]. All calculations were performed using the def2-QZVPPD basis set (with effective core potentials for atoms larger than Kr) [113, 114] for the atomic orbitals and the def2-universal-jkfit auxiliary basis set [115] for the classical Coulomb energy. The seminumercial exchange (SGX) module in PySCF, which is similar to the computationally efficient chain-of-spheres exchange algorithm [116], was used to compute the exchange energy and potential for hybrid functionals. For SGX, level 1 PySCF grids were used with P-junction screening turned off. For a given sub-database in GMTKN55, restricted DFT was used if all systems in that sub-database had S z = 0 S_z = 0 S z = 0 ; otherwise, unrestricted DFT was used. The exception was the PCl3 _3 3 transition state in the INV24 database, for which spin symmetry breaking occurs for some functionals. This system was calculated with unrestricted DFT as well. For SCF calculations, the nonlocal features and XC energy were computed on level 3 PySCF grids.
For benchmarking comparison, the GMTKN55 database was evaluated with the above settings using the PBE [23], revPBE [83], r 2 r^2 r 2 SCAN [79], PBE0 [77], B3LYP [27], and PW6B95 [82] functionals, all with D4 dispersion corrections [78], as well as with B97M-V [53], ω \omega ω B97X-V [28], and ω \omega ω B97M-V [54] functionals.
To collect the exact exchange energy training data, the ground-state orbitals from the PBE calculations were used to compute the exact exchange energy E x e x a c t E_x^{\mathrm{exact}} E x exact via Eq. (2) , and the feature vectors were computed from the corresponding PBE densities. The features were computed on level 1 PySCF grids. For training set feature generation only (not the SCF calculations), the integral in Eq. (18) was computed numerically with the r 2 r_2 r 2 coordinate evaluated on level 3 PySCF grids without pruning. This dense grid is necessary for numerical stability and precision in the core region.
Solid-State Calculations and Reference Data Generation
The electron densities and ground-state energies of the SOL62 dataset were computed in GPAW [68, 69] using the PBE geometries obtained by Trepte and Voss [33], with an energy cutoff of 1000 eV and the same k-point meshes as Trepte and Voss. The h h h parameter in GPAW was set to the minimum value required to avoid aliasing of the density, i.e. 0.096 Å. Fermi-Dirac smearing with a width of 0.01 eV was used for periodic systems. For isolated atoms, fixed occupations were used unless numerical instabilities occurred as a result, which was the case for six atoms (Pb-2, Pd-2, Pt-2, Sn-2, Ta-5, and V-5, where the number is the ground-state magnetic moment 2 S 2S 2 S ). In these cases, Fermi-Dirac smearing with a width of 0.002 eV was used. The default Davidson solver was used for all solids except the alkaline earth metals, which were solved with the conjugate gradient (CG) method. The CG method was used for all isolated atomic systems. To compute cohesive energies for training, the experimental ground state magnetic moments were used for the isolated atom references. The convergence threshold was set to 10 − 6 10^{-6} 1 0 − 6 eV per valence electron for both isolated atoms and periodic systems. The density convergence threshold was set to 10 − 4 10^{-4} 1 0 − 4 per electron for periodic systems and 10 − 3 10^{-3} 1 0 − 3 per electron for isolated atoms. The eigenstates convergence threshold was set to 4 × 10 − 8 4 \times10^{-8} 4 × 1 0 − 8 eV2 ^2 2 /electron for periodic systems and 10 − 5 10^{-5} 1 0 − 5 eV2 ^2 2 /electron for isolated atoms. The cell size for the isolated atoms was chosen to be about 11.7 Å across, with perturbations included to break symmetry. Compared to a larger box size of 17.5 Å, the PBE atomic energies had a root mean square deviation of 0.012 eV and maximum deviation of 0.080 eV (for Zr). Because of the small average deviation, and the fact that the train and test reference values were computed on the atom-in-a-box systems rather than determined by experiment, the smaller boxes were used to lower the computational cost and memory requirements of training and testing. Symmetry constraints were also turned off for the isolated atom calculations.
Training the CIDER models to the solid-state data required computing the exact exchange energy and the nonlocal features for each system. To obtain exact exchange energies, the hybrid module of GPAW was used to compute the non-self-consistent exact exchange energy (EXX) from the PBE orbitals obtained in the above calculations. However, because the k-point meshes above are too dense for evaluation of EXX to be computationally feasible, the periodic systems were first recomputed with a coarser k-point mesh (for each system, the smallest even, Γ-centered mesh with a linear k-point density of at least 4.5 k-points per Å− 1 ^{-1} − 1 ), and the EXX was computed on this mesh. All nonlocal density features were evaluated on the density and orbitals obtained from the denser k-point mesh. They were evaluated with the internal CIDER settings q max = 300 q_{\max}=300 q m a x = 300 and λ = 1.8 \lambda=1.8 λ = 1.8 (see Section A 10 for details).
The PBE0 cohesive energies were computed using the same procedure above, with the PBE0 energies being defined as
E P B E 0 = E P B E , d e n s e − E P B E , c o a r s e + E P B E 0 , c o a r s e , (106) E^{\mathrm{PBE0}}=E^{\mathrm{PBE,dense}}-E^{\mathrm{PBE,coarse}}+E^{\mathrm{PBE0,coarse}}, \tag*{(106)} E PBE0 = E PBE , dense − E PBE , coarse + E PBE0 , coarse , (106)
with dense denoting the calculation done with the dense k-point mesh and coarse denoting the coarser k-point mesh discussed above.
Construction of Training and Validation Sets
The total exchange energy of a molecule is highly basis-set and code-dependent, and it is dominated by the core contributions for large atoms. In practice, only relative exchange energies between systems is relevant for most chemistry and materials science applications. Therefore, rather than training the model to match total exchange energies, we trained to match the exchange energy differences between systems. For example, the target and predicted values for the atomization energy of water would be
E x t a r g e t = E x e x [ n O ] + 2 E x e x [ n H ] − E x e x [ n H 2 O ] (107) E_{\mathrm{x}}^{\mathrm{target}}=E_{\mathrm{x}}^{\mathrm{ex}}[\mathrm{nO}]+2E_{\mathrm{x}}^{\mathrm{ex}}[\mathrm{nH}]-E_{\mathrm{x}}^{\mathrm{ex}}[\mathrm{nH_2O}] \tag*{(107)} E x target = E x ex [ nO ] + 2 E x ex [ nH ] − E x ex [ n H 2 O ] (107)
E x p r e d = E ^ x [ n O ] + 2 E ^ x [ n H ] − E ^ x [ n H 2 O ] , (108) E_{\mathrm{x}}^{\mathrm{pred}}=\hat{E}_{\mathrm{x}}[\mathrm{nO}]+2\hat{E}_{\mathrm{x}}[\mathrm{nH}]-\hat{E}_{\mathrm{x}}[\mathrm{nH_2O}], \tag*{(108)} E x pred = E ^ x [ nO ] + 2 E ^ x [ nH ] − E ^ x [ n H 2 O ] , (108)
where E x e x [ n ] E_{\mathrm{x}}^{\mathrm{ex}}[n] E x ex [ n ] and E ^ x [ n ] \hat{E}_{\mathrm{x}}[n] E ^ x [ n ] are the exact and model exchange energies for density distribution n ( r ) n(\mathbf{r}) n ( r ) , respectively.
The training and validation sets were constructed from four sources:
Molecules: The W4-11 (atomization energies), BH76 (barrier heights), BH76RC (reaction energies for BH76 reactions), MB16-43 (“mindless benchmarking” of reaction energies), S22 (noncovalently bound dimers), G21IP (ionization potentials), PA26 (proton affinities), RG18 (noble gas dimers), ACONF (alkane conformers), ALKBD E10 (alkali/alkaline earth diatomic bond energies), HEAVYSB11 (dissociation energies for molecules with heavy atoms), and SIE4x4 (self-interaction problems) subsets from the GMTKN55 database were used for training and validation. See the original GMTKN55 paper [10 ] and references therein for more details on the datasets. Each set was randomly split into two-thirds training and one-third validation, except for the BH76 and BH76RC subsets. Because the data points in these subsets are based on the same systems, and also because they include the forward and backward reactions for the same chemical equation, a different splitting procedure was chosen to maintain train-validation independence. First, the 34 unique transition states in BH76 were identified and split half-and-half into the training and validation sets. All 40 reactions with the training transition states were used for training, and all 36 reactions with the validation transition states were used for validation. All 17 BH76RC reactions with systems in the BH76 training partition were used for training, and the other 13 reactions were used for validation. All data not in the training or validation set was classified as the GMTKN55 test set. This data partitioning scheme resulted in 280 train data, 163 validation data, and 1062 test data in GMTKN55.
Bulk solids: Half of the SOL62 cohesive energy database (11 randomly selected elemental metals and 20 randomly selected non-metals) was used for training and validation. Nine systems (C-diamond, Li, GaAs, MgO, LiF, ZrC, TiC, Pd, and Pt) were hand-selected for the training set to cover a range of chemical space, while the other 22 systems were used for validation. All data not in the training or validation set was classified as the SOL62 test set.
Nanoclusters: Six nanocluster atomization energies were added to the training set as a “bridge” between the isolated and extended systems in the study. These were Ca13 _{13} 13 and Zn26 _{26} 26 , with the initial structure obtained based on the Materials Project [117] bulk lattice constant and structure optimization at the PBE/def2-TZVP level in Orca [118–120]; C35 _{35} 35 H36 _{36} 36 (hydrogen-capped diamond), obtained from Refs. [121, 122] and also structurally optimized in Orca at the PBE/def2-TZVP level; and Na4 _4 4 Cl4 _4 4 , [Na13 _{13} 13 Cl14 _{14} 14 ], and [Na14 _{14} 14 Cl13 _{13} 13 ]+ ^+ + , all using the Materials Project bulk lattice constant without further structure optimization.
Atoms: Absolute and relative energies of the isolated atoms H-Kr and a few excited spin states were included in the training set. Specifically, the training points consisted of the absolute energies of the H atom and the noble gases He, Ne, Ar, and Kr; the relative energies of the elements with nuclear charge Z Z Z and Z + 1 Z + 1 Z + 1 for Z = { 1 , … , 35 } Z = \{1, \ldots, 35\} Z = { 1 , … , 35 } , and the relative energies of the following ground state/excited state spin pairs: Sc 1/3, Ti 2/4, V 3/5, Cr 6/4, and Ni 2/0. The preceding numbers correspond to the magnetic moment 2 S 2S 2 S .
In addition to fitting to the above training data, the uniform electron gas constraint was enforced as described in § 2 C 3, and the uniform scaling constraint was enforced by the scale-invariance of the input features. The ground-state total energies of the SOL62 solids and isolated atoms were computed with GPAW as described in Section A 3. The ground-state energies for all other training data were evaluated with PySCF as described in Section A 2. The selection of train and validation databases from GMTKN55 includes a wide variety of chemistries and properties. However, because training data is sampled disproportionately from small-molecule properties, there is a large distribution shift between the train/validation and test sets, with the test sets containing more large molecules, isomerization reactions, and barrier heights. The use of this train/test partition therefore serves as a strong test of transferability. The accuracy on SOL62 also strongly reflects the transferability of a model because only 9 solid-state systems were included in the training set, and SOL62 covers a wide variety of chemistries.
A More Diverse Training Set for Numerical Stability
To design a more diverse training set for the NL-MGGA-DTR functional, a kernel function was defined for the “covariance” between two training points in GMTKN55:
k G M T K N 55 ( x , x ′ ) = exp ( − ∥ x − x ′ ∥ 2 328 ) , (109) k_{\mathrm{GMTKN55}}(\mathbf{x},\mathbf{x}') = \exp\left(-\frac{\lVert\mathbf{x}-\mathbf{x}'\rVert^2}{328}\right), \tag*{(109)} k GMTKN55 ( x , x ′ ) = exp ( − 328 ∥ x − x ′ ∥ 2 ) , (109)
where x \mathbf{x} x is the vector of errors for 82 functionals benchmarked on the GMTKN55 database by Goerigk et al. [10 ] with D3(BJ) dispersion corrections [123–125], normalized so that the mean square error across all functionals for that individual data point is 1. This kernel essentially provides a larger similarity score between data points for which the (normalized) errors tend to be similar for a given functional. The kernel matrix was constructed for the entire GMTKN55 database and then factorized using a method called Cholesky decomposition with complete pivoting [126, 127], which factors a given matrix A \mathbf{A} A as follows:
P T A P = L L T , (110) \mathbf{P}^{\mathsf{T}}\mathbf{A}\mathbf{P} = \mathbf{L}\mathbf{L}^{\mathsf{T}}, \tag*{(110)} P T AP = L L T , (110)
In Eq. A5, P \mathbf{P} P is a permutation matrix that orders the vectors x \mathbf{x} x in Equation A4 in decreasing order of their variance, conditioned on the preceding feature vectors, in the space implicitly spanned by the kernel function. Intuitively, this means that later vectors, as ordered by P \mathbf{P} P , are likely to be nearly linearly dependent (in the kernel space) on the earlier vectors. Therefore, by picking the first N N N training points as ordered by P \mathbf{P} P and using the remaining data for testing, we can get a training set whose errors correlate as much as possible with all the test data, effectively making the training set more diverse and potentially preventing out-of-distribution error of the ML model.
We follow this procedure and select the first 225 pivots for the training set, the next 75 pivots for the validation set, and the remaining 1205 pivots for the test set. With this dataset partitioning, more than 99% of the GMTKN55 test set data points contain at least one system not present in the train or validation sets, so the test set is still sufficiently independent to assess the transferability of the functionals to new systems. All training points not contained in GMTKN55 were kept the same as in Appendix A 4.
Control Point Selection for Sparse Gaussian Processes
In Eqs. 37 and 38, a set of control points { x a } \{\mathbf{x}_a\} { x a } is required to evaluate the Gaussian process covariance kernel in the sparse approximation. These control points were initially selected by randomly sampling feature vectors from the PySCF grids for the training set (Appendix A 2). All training data was sampled except the SOL62 data, since the features for these systems were evaluated in GPAW.
Due to the size of the numerical grids from which the features were sampled, there were a large number of potential control points for the sparse Gaussian process, many of which were nearly linearly dependent in the implicit feature space of the covariance kernel function. To reduce the size of the control point set, a Cholesky decomposition with pivoting (Eq. (110) ) was performed on an initial sample size of roughly 10 4 10^4 1 0 4 feature vector samples with a tolerance of 10 − 5 10^{-5} 1 0 − 5 , and only control points corresponding to significant pivot indices were kept. This reduced the number of control points from roughly 10 4 10^4 1 0 4 to 10 2 10^2 1 0 2 .
Selection of Noise Hyperparameters, Training Loss, and Validation Loss
Chemical and material properties have a wide range of magnitudes, from less than 1 kcal/mol for some van der Waals binding energies to greater than 100 kcal/mol for many covalent bond dissociation energies. Correspondingly, different magnitudes of XC errors are expected and acceptable for these properties; an error of 1 kcal/mol might make a van der Waals binding energy prediction useless but be acceptable for an atomization energy. To train the exchange functionals, it was therefore necessary to weight the noise hyperparameter of each training point to reflect the feasible and desired precision for the property of interest. To do so, we used the data provided with GMTKN55 [10 ] to compute the MAD of four hybrid functionals (PBE0-D3(BJ), B3LYP-D3(BJ), PW6B95-D3(BJ), and ω \omega ω B97X-V) compared to the reference data on each sub-database (denoted M A D D B h y b r i d \mathrm{MAD}^{\mathrm{hybrid}}_{\mathrm{DB}} MAD DB hybrid , where DB is the sub-database of interest), and then created characteristic uncertainties for each sub-database as follows (in Hartree atomic units):
σ ~ D B = M A D D B h y b r i d ( σ 0 M A D W 4 − 11 h y b r i d ) , (111) \tilde{\sigma}_{\mathrm{DB}}=\mathrm{MAD}^{\mathrm{hybrid}}_{\mathrm{DB}}\left(\frac{\sigma_0}{\mathrm{MAD}^{\mathrm{hybrid}}_{\mathrm{W4-11}}}\right), \tag*{(111)} σ ~ DB = MAD DB hybrid ( MAD W4 − 11 hybrid σ 0 ) , (111)
where W4-11 is a small-molecule atomization energy sub-database in GMTKN55 and σ 0 = 0.03 E h \sigma_0=0.03\,E_{\mathrm{h}} σ 0 = 0.03 E h . One drawback of the above approach is that it favors high accuracy on systems that are already described well by conventional, dispersion-corrected DFT (like simple van der Waals dimers) and assumes a large uncertainty on systems like self-interaction-dominated bond energies, for which even hybrid functionals with small exchange mixing perform poorly. To remedy this, the noise parameter was augmented with a constant weighting for every system:
σ D B = 2 1 / σ 0 2 + 1 / σ ~ D B 2 . (112) \sigma_{\mathrm{DB}}=\sqrt{\frac{2}{1/\sigma_0^2+1/\tilde{\sigma}_{\mathrm{DB}}^2}}. \tag*{(112)} σ DB = 1/ σ 0 2 + 1/ σ ~ DB 2 2 . (112)
Hyperparameter Selection and Validation Procedure
The models tested in this work have several discrete and continuous hyperparameters to be optimized, which are listed below.
The covariance prefactor hyperparameter Σ C o v \Sigma_{\mathrm{Cov}} Σ Cov in eqs (56) –(59) .
The kernel length-scale (l i l_i l i , where i i i is the feature index) in eq (60) .
For the nonlocal functionals only, the feature exponent parameters B i B_i B i and C i C_i C i in Eqs. (20) –(23) .
The baseline exchange functional on top of which the CIDER model is trained via Δ \Delta Δ -learning. We tested PBE [23] and the Chachiyo GGA [76].
It would be quite challenging and expensive to implement differentiation of the marginal likelihood with respect to the second parameter, and nearly impossible for the third since backpropagation would need to be performed through the feature evaluation step. Therefore, we used various combinations of hyperparameters to train the models, and then for each functional type, we selected the best-performing model on the validation set as the final candidate functional.
To obtain a discrete set of exponent hyperparameters to test for the nonlocal functionals, we developed a reduced number of parameters to specify the feature exponents. Similarly to the previous CIDER work [35], a control parameter A A A tunes B 2 B_2 B 2 and C 2 C_2 C 2 simultaneously, in two different schemes. The first scheme (S1) is
C 2 = A 32 6 5 π ( 6 π 2 ) 2 / 3 ≈ 0.18 A . (113) C_2=\frac{A}{32}\frac{6}{5\pi}(6\pi^2)^{2/3}\approx0.18A. \tag*{(113)} C 2 = 32 A 5 π 6 ( 6 π 2 ) 2/3 ≈ 0.18 A . (113)
The second scheme (S2) provides a different mixture of density-based and gradient/kinetic energy-based terms:
B 2 = A 16 6 5 π ( 6 π 2 ) 2 / 3 ≈ 0.36 A C 2 = B 2 . (114) \begin{aligned}
B_2 &= \frac{A}{16}\frac{6}{5\pi}(6\pi^2)^{2/3} \approx0.36A \\
C_2 &= B_2. \tag*{(114)}
\end{aligned} B 2 C 2 = 16 A 5 π 6 ( 6 π 2 ) 2/3 ≈ 0.36 A = B 2 . (114)
In both schemes, the other constants are determined by B 2 B_2 B 2 and C 2 C_2 C 2 :
B 1 = 1 2 B 2 , B 3 = 2 B 2 , B 0 = D A B 2 (115) B_1 = \frac{1}{2}B_2,\quad B_3 = 2B_2,\quad B_0 = \frac{D}{A}B_2 \tag*{(115)} B 1 = 2 1 B 2 , B 3 = 2 B 2 , B 0 = A D B 2 (115)
C 1 = 1 2 C 2 , C 3 = 2 C 2 , C 0 = D A C 2 , (116) C_1 = \frac{1}{2}C_2,\quad C_3 = 2C_2,\quad C_0 = \frac{D}{A}C_2, \tag*{(116)} C 1 = 2 1 C 2 , C 3 = 2 C 2 , C 0 = A D C 2 , (116)
where D D D is an additional parameter to tune the B 0 B_0 B 0 and C 0 C_0 C 0 coefficients separately from the other coefficients. With these schemes, all 8 parameters { B i } \{B_i\} { B i } , { C i } \{C_i\} { C i } that define the three-feature CIDER model were reduced to two parameters per scheme. For each scheme, we performed a hyperparameter search over the grid A = { 1 / 2 , 1 , 2 , 4 , 8 } A = \{1/2, 1, 2, 4, 8\} A = { 1/2 , 1 , 2 , 4 , 8 } and D = { 1 / 2 , 1 , 2 , 4 , 8 } D = \{1/2, 1, 2, 4, 8\} D = { 1/2 , 1 , 2 , 4 , 8 } , resulting in a total of 50 feature constructions to test.
Because there were already 50 features from which to select the nonlocal functionals, we set the covariance and feature length-scales for these models heuristically:
Σ C o v = R 1 ⟨ Δ F x 2 ⟩ t r a i n (117) \Sigma_{\mathrm{Cov}} = R_1\langle\Delta F_x^2\rangle_{\mathrm{train}} \tag*{(117)} Σ Cov = R 1 ⟨ Δ F x 2 ⟩ train (117)
l i = R 2 ⟨ x i 2 ⟩ t r a i n , (118) l_i = R_2\sqrt{\langle x_i^2\rangle_{\mathrm{train}}}, \tag*{(118)} l i = R 2 ⟨ x i 2 ⟩ train , (118)
where ⟨ ⟩ t r a i n \langle\rangle_{\mathrm{train}} ⟨ ⟩ train is an expectation value over all the grid points in the exchange energy training set, Δ F x \Delta F_x Δ F x is the deviation of the exact exchange enhancement factor at a given point from the baseline functional enhancement factor, x i x_i x i is the i i i -th regularized feature (see § 2 C 3), and R 1 R_1 R 1 and R 2 R_2 R 2 are constants. R 1 R_1 R 1 and R 2 R_2 R 2 were set based on the criteria that the resulting functionals converge consistently in both GTO and plane-wave calculations and exhibit a small total energy deviation between the PAW and all-electron GTO atomization energies for a few small molecules. R 2 = 1 R_2 = 1 R 2 = 1 worked best for both the nonlocal GGA and meta-GGA, while the most practical values of R 1 R_1 R 1 depended on the baseline functional and model type, as shown in Table VI . The meta-GGA models required a smaller covariance prefactor to achieve good stability than the GGA models. We set R 1 R_1 R 1 to be larger by a factor of 20 for functionals with the Chachiyo baseline compared to the PBE baseline because ⟨ Δ F x 2 ⟩ t r a i n \langle\Delta F_x^2\rangle_{\mathrm{train}} ⟨ Δ F x 2 ⟩ train was roughly 20 times larger for PBE than Chachiyo.
Table VI. R 1 R_1 R 1 values for each functional type and baseline functional.
For the semilocal functionals, there was no need to validate over the possible feature constructions, leaving more flexibility to test R 1 R_1 R 1 and R 2 R_2 R 2 systematically. For the functionals with Chachiyo baseline, we tested R 1 = { 1 , 2 , 4 , 8 , 16 , 32 , 64 , 128 } R_1 = \{1, 2, 4, 8, 16, 32, 64, 128\} R 1 = { 1 , 2 , 4 , 8 , 16 , 32 , 64 , 128 } , while for the PBE baseline each R 1 R_1 R 1 was divided by 20 to give a roughly equivalent Σ C o v \Sigma_{\mathrm{Cov}} Σ Cov . For the semilocal functionals, R 2 = { 1 , 1 2 , 1 4 } R_2 = \{1, \frac{1}{2}, \frac{1}{4}\} R 2 = { 1 , 2 1 , 4 1 } were tested, resulting in 24 functionals to validate. Note that for NL-MGGA-DTR, we determined from validation
set SCF calculations that R 1 = 0.1 R_1 = 0.1 R 1 = 0.1 with the PBE baseline resulted in sufficiently stable models, so this larger value was used. In addition, we did not search over feature length-scale for NL-MGGA-DTR, and instead used the features determined to be most accurate for NL-MGGA and NL-MGGA-PBE.
After the functionals were trained, we selected the top 10 functionals of each type and baseline functional, as classified by non-self-consistent performance on the validation set via Eq. A8. To validate these functionals, the GMTKN55 training and validation set reaction energies were predicted using self-consistent calculations with a convergence threshold of 10 − 6 10^{-6} 1 0 − 6 Eh and a gradient threshold of 2 × 10 − 3 2 \times10^{-3} 2 × 1 0 − 3 Eh. For a given functional, if any calculation failed to converge when starting from both the default initial guess of PySCF and from the PBE orbitals, the functional was deemed insufficiently numerically stable and disqualified from further consideration. The non-self-consistent cohesive energies of the SOL62 validation set were also computed. Of the functionals for which all calculations converged, the validation loss consisted of
L v a l = L M O L + L S O L (119) \mathcal{L}_{\mathrm{val}} = \mathcal{L}_{\mathrm{MOL}} + \mathcal{L}_{\mathrm{SOL}} \tag*{(119)} L val = L MOL + L SOL (119)
L M O L = 1 2 ∑ D B N D B / V A L × M S D D B / V A L S C F × ( 1 + σ 0 2 σ D B 2 ) (120) \mathcal{L}_{\mathrm{MOL}} = \frac{1}{2}\sum_{\mathrm{DB}} N_{\mathrm{DB/VAL}} \times\mathrm{MSD}^{\mathrm{SCF}}_{\mathrm{DB/VAL}} \times\left(1+\frac{\sigma_0^2}{\sigma_{\mathrm{DB}}^2}\right) \tag*{(120)} L MOL = 2 1 DB ∑ N DB/VAL × MSD DB/VAL SCF × ( 1 + σ DB 2 σ 0 2 ) (120)
L S O L = M S D S O L 62 / V A L 22 N S C F , (121) \mathcal{L}_{\mathrm{SOL}} = \mathrm{MSD}^{\mathrm{NSCF}}_{\mathrm{SOL62/VAL22}}, \tag*{(121)} L SOL = MSD SOL62/VAL22 NSCF , (121)
where M S D D B / V A L S C F \mathrm{MSD}^{\mathrm{SCF}}_{\mathrm{DB/VAL}} MSD DB/VAL SCF is the self-consistent mean square deviation on the validation partition of GMTKN55 subset DB, N D B / V A L N_{\mathrm{DB/VAL}} N DB/VAL is the number of validation data in DB, and M S D S O L 62 / V A L 22 N S C F \mathrm{MSD}^{\mathrm{NSCF}}_{\mathrm{SOL62/VAL22}} MSD SOL62/VAL22 NSCF is the non-self-consistent mean square deviation of the 22 validation-set cohesive energies per atom. The final functional of each type was selected based on which candidate had the lowest loss for Eq. (119) . The validation results and selected hyperparameters for the final functionals are provided in the Supplemental Material (§ 29 ).
Difficult Convergence Cases
The default convergence tolerance for the calculations performed with CIDER functionals was 10 − 8 10^{-8} 1 0 − 8 Eh, except for the NL-MGGA-PBE and NL-MGGA-DTR calculations, which were converged to 10 − 9 10^{-9} 1 0 − 9 Eh when possible. Some CIDER calculations did not converge to this tolerance within the default PySCF settings. There were two different classes of convergence problems. First, for a few systems, some CIDER calculations did not converge from the PySCF starting guess, but the convergence issues were resolved by starting from PBE ground-state orbitals. Second, for some systems (especially isolated atoms and ions, as well as alkali and alkaline earth-containing systems), a numerical stability issue was observed for some CIDER calculations, in which the energy converged to a reasonable tolerance (less than 10 − 7 10^{-7} 1 0 − 7 Eh), but the orbital gradients oscillated around 10 − 3 10^{-3} 1 0 − 3 Eh without converging fully. In these cases, the energy tolerance was set to 10 − 7 10^{-7} 1 0 − 7 Eh, and the gradient tolerance was set to the (rather large) 2 × 10 − 3 2 \times10^{-3} 2 × 1 0 − 3 Eh. All CIDER calculations converged to these thresholds. PBE starting orbitals were also used for these loose-threshold calculations. Even with these looser thresholds, the achieved energy and orbital gradient convergence is still sufficient to compare thermochemical data in the GMTKN55 database.
CIDER Functional Settings
The CIDER functionals require a few additional settings to specify the precision of the accelerated feature calculations. For the molecular version, these settings are l max l_{\max} l m a x , the maximum spherical harmonic order for auxiliary basis expansions in § 2 E; λ \lambda λ , in Eq. (62) ; β a u x \beta^{\mathrm{aux}} β aux , in Eq. (73) ; and q max q_{\max} q m a x , the maximum value of q α q_{\alpha} q α (Eq. (62) ). The ξ \xi ξ basis in Eq. (76) uses the same parameter λ \lambda λ for the even-tempered basis as Eq. (62) . We chose l max = 10 l_{\max} = 10 l m a x = 10 , λ = β a u x = 1.6 \lambda= \beta^{\mathrm{aux}} = 1.6 λ = β aux = 1.6 , and q max = 1000 36 Z max 2 q_{\max} = \frac{1000}{36} Z_{\max}^{2} q m a x = 36 1000 Z m a x 2 , where Z max Z_{\max} Z m a x is the minimum of 36 and the charge of the largest nucleus in the given GMTKN55 sub-database. These settings were found to be in good agreement with numerical integration of the features. The minimum control point q 0 q_{0} q 0 was set automatically based on the minimum allowed value of the kernel exponent a ( r ) a(r) a ( r ) .
For the plane-wave implementation, only λ = 1.8 \lambda= 1.8 λ = 1.8 and q max = 300 q_{\max} = 300 q m a x = 300 needed to be specified. A larger q max , a t o m = 10 Z 2 q_{\max,\mathrm{atom}} = 10Z^{2} q m a x , atom = 10 Z 2 was used for each atom on the radial support grids, with Z Z Z being the atomic number. In both the Gaussian-type orbital and plane-wave implementations, all other CIDER-specific parameters (like the minimum exponents for the various auxiliary bases) were set automatically based on the chosen basis set and nonlocal feature parameters (i.e. the parameters in Appendix A § 19.1 ).
Band Gap Calculations
Band gaps for the database of Borlido et al. [45] were computed in GPAW [68, 69] as the difference between the conduction band minimum and valence band maximum for a ground-state SCF calculation. The energy cutoff for these calculations was 520 eV, and the grid spacing was set to h = 0.134 h = 0.134 h = 0.134 Å to avoid any aliasing of the density. Fermi-Dirac k-point smearing was used with a width of 0.01 eV, except for the noble gas systems, for which no k-point smearing was used. The k-point mesh for each system was selected to be the smallest even, Γ \Gamma Γ -centered mesh with a linear k-point density of at least 6 k-points per Å− 1 ^{-1} − 1 . For each system, if the PBE band gap were obtained was more than 0.1 eV larger than the PBE band gap obtained by Borlido et al. [45], we inferred that the k-point mesh was too coarse and reran the band gap calculation with a linear k-point density of 10 k-points per Å− 1 ^{-1} − 1 . Using this methodology, we reproduced the PBE band gaps of Borlido et al. with a mean, mean absolute, and maximum error of -0.01, 0.03, and 0.17 eV, respectively, and the SCAN results with a mean, mean absolute, and maximum error of 0.00, 0.03, and 0.15 eV, respectively.
For PBE0/NL-MGGA-DTR and PBE0(0.35)/NL-MGGA-DTR, a few systems did not converge with the default CIDER PAW corrections, and a denser core integration grid and smaller set of feature projectors (i.e. the g i A g_i^A g i A and p i A p_i^A p i A functions of § 2 F) were used. Even with these settings, one system (ReSe2 _2 2 ) did not converge with PBE0(0.35)/NL-MGGA-DTR, and it had to be run with SG15 norm-conserving pseudopotentials (NCPP) [128]. For the 255 calculations in the main text, the mean, mean absolute, and maximum deviations between PBE0/NL-MGGA-DTR with NCPP and PAW were 0.03, 0.12, and 1.12 eV, respectively. In addition, the band gap for ReSe2 _2 2 with PBE0/NL-MGGA-DTR was 1.29 eV with PAW and 1.30 eV with NCPP, so the use of pseudopotentials appears reasonable for this particular case. We think that the convergence issues are due to numerical problems with the PAW corrections for nonlocal features, and improving the numerical stability of these corrections will be a subject of future work.
For the band gaps in Table IV , the same procedure was used, except with denser k-point meshes containing 12 k-points per Å− 1 ^{-1} − 1 . The HSE06 band gaps were computed using VASP [96], with the k-point mesh set to contain 4000 k-points divided by the number of atoms in the unit cell. This is a coarser mesh than was used for the semilocal and CIDER calculations, but we found that the band gap difference compared to larger k-point densities was less than 0.01 eV.
Supercell and Neutral Defect Performance
For the neutral defect calculations, the atomic structure of the 215-atom single vacancy cells of diamond and silicon were obtained using VASP [96–98] with PAW datasets [67, 74], an energy cutoff of 520 eV, the “Accurate” precision setting, and a 2 × 2 × 2 2 \times2 \times2 2 × 2 × 2 Monkhorst-Pack k-point mesh [95], starting with the PBE lattice constant obtained in VASP using the same settings with a 14 × 14 × 14 14 \times14 \times14 14 × 14 × 14 Γ \Gamma Γ -centered k-point mesh. Using these structures, SCF ground-state calculations with static atomic positions were performed in GPAW [68, 69] for the 216atom bulk and 215-atom vacancy cells for each functional. The formation energy of the neutral defect was computed as
E f o r m e n = E d e f e c t − 215 216 E b u l k , (122) E^{\mathrm{formen}} = E^{\mathrm{defect}} - \frac{215}{216} E^{\mathrm{bulk}}, \tag*{(122)} E formen = E defect − 216 215 E bulk , (122)
where the bulk and defect energies were computed with 216 and 215-atom supercells, respectively, with a 2 × 2 × 2 2 \times2 \times2 2 × 2 × 2 Monkhorst-Pack k-point mesh (resulting in 1 reduced k-point for the bulk supercell and 4 reduced k-points for the defect supercell), 520 eV plane-wave cutoff, 0.134 Å grid spacing, and 10 − 5 10^{-5} 1 0 − 5 eV convergence threshold. 523 bands were computed for the bulk, and 520 bands were computed for the defect cell. The time to evaluate the formation energy for each functional was computed as the sum of the time taken for the bulk and vacancy cell calculations, as obtained from the GPAW internal calculation timer. The HSE06 and PBE0 formation energies were obtained using VASP with the same structures, k-point grid, and energy cutoff.
For the Γ-point calculations used in Figure 8 a, GPAW was also used for the semilocal and ML functionals. For the hybrid DFT calculations, Quantum ESPRESSO [93, 94] was used with the adaptively compressed exchange [17] algorithm for exchange, a 38 Rydberg plane-wave cutoff for the wave functions, a 152 Rydberg cutoff for the charge density and potential, a 10 − 6 10^{-6} 1 0 − 6 Rydberg convergence threshold, and SG15 norm-conserving pseudopotentials [128]. The use of a 38 Rydberg (517 eV) plane-wave cutoff might be too small when using norm-conserving pseudopotentials, but for performance benchmarks, it provides the most fair comparison to the 520 eV cutoff used with the PAW setups in GPAW. The PBE calculation with Quantum ESPRESSO used the same settings but with PBE in place of PBE0. The GPAW calculations used 523 bands, and the Quantum ESPRESSO calculations used 432 bands.
Charged Defect Calculations
The charged defect levels were computed with PBE, PBE0/NL-MGGA-DTR, and PBE0(0.3)/NL-MGGA-DTR. The mixing parameter α = 0.3 \alpha= 0.3 α = 0.3 of the last functional was selected to match the zero-temperature experimental band gap of silicon, 1.17 eV. For each functional, the lattice constant, band edges, and band gap were obtained from a unit cell relaxation with 520 eV cutoff, h = 0.134 h = 0.134 h = 0.134 Å, and a 24 × 24 × 24 24 \times24 \times24 24 × 24 × 24 k-point mesh in GPAW [68, 69]. Then, a 512-atom supercell was created for each defect, with perturbed structures used for the vacancy to break symmetry. For each charge state and defect, the geometry was relaxed in GPAW using the BFGS with Line Search algorithm to a tolerance of 0.05 eV/Å with a Γ-point only calculation. This geometry was then used as the starting point for the final geometry relaxation, which was performed with a 2 × 2 × 2 2 \times2 \times2 2 × 2 × 2 Γ-centered k-point mesh and a force convergence tolerance of 0.01 eV/Å. Because PBE0(0.3)/NL-MGGA-DTR is so similar to PBE0/NL-MGGA-DTR, and because the lattice constants are similar, we saved time for this functional by starting the final relaxation from the fractional coordinates of the PBE0/NL-MGGA-DTR calculations.
The formation energies of the defects were calculated as [6 ]
E f [ X q ] = E t o t [ X q ] − E t o t [ b u l k ] − ∑ i n i μ i + q E F + E c o r r [ X q ] , (123) E^{f}[X^{q}] = E_{\mathrm{tot}}[X^{q}] - E_{\mathrm{tot}}[\mathrm{bulk}]
- \sum_i n_i \mu_i + q E_F + E_{\mathrm{corr}}[X^{q}], \tag*{(123)} E f [ X q ] = E tot [ X q ] − E tot [ bulk ] − i ∑ n i μ i + q E F + E corr [ X q ] , (123)
where E t o t E_{\mathrm{tot}} E tot is the total ground-state energy of the supercell DFT calculation, X q X^{q} X q is defect X X X in charge state q q q , E F E_F E F is the Fermi level, and E c o r r [ X q ] E_{\mathrm{corr}}[X^{q}] E corr [ X q ] is a finite-size energy correction for the defect. The stoichiometry changes and chemical potentials n i n_i n i and μ i \mu_i μ i were neglected because they do not impact electronic transition levels. We used the Freysoldt finite-size correction-scheme for E c o r r [ X q ] E_{\mathrm{corr}}[X^{q}] E corr [ X q ] [105], which combines an electrostatic and potential alignment correction. This correction requires an estimate of the dielectric constant ϵ \epsilon ϵ , for which we used the experimental value of 11.9 [129].
Because some of the defects studied in this work are shallow donors and acceptors, and because electrostatic corrections like those developed by Freysoldt et al. [105] and Kumagai et al. [130] assume localized charges, these corrections might be inaccurate. Therefore, for comparison, we also computed the transition levels using the potential alignment correction only, i.e.
E c o r r [ X q ] = q ( V ‾ e s p [ X q ] − V ‾ e s p [ b u l k ] ) , (124) E_{\mathrm{corr}}[X^{q}] = q \left(\overline{V}_{\mathrm{esp}}[X^{q}] - \overline{V}_{\mathrm{esp}}[\mathrm{bulk}]\right), \tag*{(124)} E corr [ X q ] = q ( V esp [ X q ] − V esp [ bulk ] ) , (124)
where V ‾ e s p \overline{V}_{\mathrm{esp}} V esp is the average electrostatic potential in a region far from the defect site. This is equivalent to the full Freysoldt correction with ϵ → ∞ \epsilon\rightarrow\infty ϵ → ∞ . We used pymatgen [109] to perform the corrections.
Construction of Feature Projector Functions for CIDER with PAW
Construction of Feature Smoothing Projectors
We solve for the coefficients D j β D_{j\beta} D j β and C i α C_{i\alpha} C i α in Eqs. (87) and (88) by minimizing the loss function
L = ∑ β w β ∫ d 3 r [ y ^ A , β ( r ) − Δ F A , β ( r ) ] 2 . (125) \mathcal{L} = \sum_{\beta} w_{\beta} \int\mathrm{d}^{3}r \left[\hat{y}_{A,\beta}(\mathbf{r}) - \Delta F_{A,\beta}(\mathbf{r})\right]^2. \tag*{(125)} L = β ∑ w β ∫ d 3 r [ y ^ A , β ( r ) − Δ F A , β ( r ) ] 2 . (125)
For the current implementation, w β = 1 w_{\beta} = 1 w β = 1 , but this could be changed if needed to encourage better smoothness or numerical precision. We start with the following set of functions:
g i ← n l m ( r ) = { Y l m ( r ^ ) P ( 2 n + l , r / r c ) r < r c 0 r ≥ r c (126) g_{i\gets nlm}(\mathbf{r}) =
\begin{cases}
Y_{lm}(\hat{\mathbf{r}})P(2n+l,r/r_c) & r<r_c \\
0 & r\ge r_c
\end{cases} \tag*{(126)} g i ← n l m ( r ) = { Y l m ( r ^ ) P ( 2 n + l , r / r c ) 0 r < r c r ≥ r c (126)
P ( n , x ) = { 1 2 ( 1 + cos π x ) n = 0 Q ( n − 1 , x ) n ∈ O R ( n − 2 , x ) n ∈ E (127) P(n,x)=
\begin{cases}
\frac{1}{2}(1+\cos\pi x) & n=0 \\
Q(n-1,x) & n\in\mathbb{O} \\
R(n-2,x) & n\in\mathbb{E}
\end{cases} \tag*{(127)} P ( n , x ) = ⎩ ⎨ ⎧ 2 1 ( 1 + cos π x ) Q ( n − 1 , x ) R ( n − 2 , x ) n = 0 n ∈ O n ∈ E (127)
Q ( m , x ) = 5 ( sin π x 2 − 1 2 + 1 2 cos π x ) x m e − m x 2 (128) Q(m,x)=5\left(\sin\frac{\pi x}{2}-\frac{1}{2}+\frac{1}{2}\cos\pi x\right)x^{m}e^{-mx^{2}} \tag*{(128)} Q ( m , x ) = 5 ( sin 2 π x − 2 1 + 2 1 cos π x ) x m e − m x 2 (128)
R ( m , x ) = 1 2 ( 1 − cos ( 2 π x ) ) x m e − m x 2 , (129) R(m,x)=\frac{1}{2}(1-\cos(2\pi x))x^{m}e^{-mx^{2}}, \tag*{(129)} R ( m , x ) = 2 1 ( 1 − cos ( 2 π x )) x m e − m x 2 , (129)
where O \mathbb{O} O and E \mathbb{E} E are the sets of positive odd and even integers, respectively. Then we introduce the matrix P \mathbf{P} P
P = ( P 11 P 12 ( P 12 ) T P 22 ) . (130) \mathbf{P}=
\begin{pmatrix}
\mathbf{P}^{11} & \mathbf{P}^{12} \\
(\mathbf{P}^{12})^{\mathsf{T}} & \mathbf{P}^{22}
\end{pmatrix}. \tag*{(130)} P = ( P 11 ( P 12 ) T P 12 P 22 ) . (130)
In the above equation, P 11 \mathbf{P}^{11} P 11 is an N h N a × N h N a N_hN_a\times N_hN_a N h N a × N h N a matrix,
P j β , j ′ β ′ 11 = δ β β ′ ⟨ h j A ∣ h j ′ ⟩ + δ j j ′ δ β β ′ σ 11 , (131) P^{11}_{j\beta,j'\beta'}=\delta_{\beta\beta'}\langle h^A_j|h_{j'}\rangle+\delta_{jj'}\delta_{\beta\beta'}\sigma_{11}, \tag*{(131)} P j β , j ′ β ′ 11 = δ β β ′ ⟨ h j A ∣ h j ′ ⟩ + δ j j ′ δ β β ′ σ 11 , (131)
where N h N_h N h is the number of h j A h^A_j h j A functions and N a N_a N a is the number of control points for kernel interpolation. Similarly, P 22 \mathbf{P}^{22} P 22 is an N g N a × N g N a N_gN_a\times N_gN_a N g N a × N g N a matrix,
P i α , i ′ α ′ 22 = ∑ β w β I β , i α , i ′ α ′ + δ i i ′ δ α α ′ σ 22 ′ (132) P^{22}_{i\alpha,i'\alpha'}=\sum_{\beta}w_\beta I_{\beta,i\alpha,i'\alpha'}+\delta_{ii'}\delta_{\alpha\alpha'}\sigma'_{22} \tag*{(132)} P i α , i ′ α ′ 22 = β ∑ w β I β , i α , i ′ α ′ + δ i i ′ δ α α ′ σ 22 ′ (132)
I β , i α , i ′ α ′ = ∫ d 3 r ϕ i α β ( r ) ϕ i ′ α ′ β ( r ) ( 1 + σ 22 f c u t ( r ) ) (133) I_{\beta,i\alpha,i'\alpha'}=\int\mathrm{d}^{3}r\,\phi_{i\alpha\beta}(\mathbf{r})\phi_{i'\alpha'\beta}(\mathbf{r})(1+\sigma_{22}f_{\mathrm{cut}}(\mathbf{r})) \tag*{(133)} I β , i α , i ′ α ′ = ∫ d 3 r ϕ i α β ( r ) ϕ i ′ α ′ β ( r ) ( 1 + σ 22 f cut ( r )) (133)
ϕ i α β ( r ) = ∫ d 3 r ′ Φ α β ( r ′ − r ) g i A ( r ′ ) , (134) \phi_{i\alpha\beta}(\mathbf{r})=\int\mathrm{d}^{3}r'\,\Phi_{\alpha\beta}(\mathbf{r}'-\mathbf{r})g^A_i(\mathbf{r}'), \tag*{(134)} ϕ i α β ( r ) = ∫ d 3 r ′ Φ α β ( r ′ − r ) g i A ( r ′ ) , (134)
where N g N_g N g is the number of g i A g^A_i g i A functions. The constants σ 11 \sigma_{11} σ 11 , σ 22 \sigma_{22} σ 22 , and σ 22 ′ \sigma'_{22} σ 22 ′ are regularization constants to reduce the magnitude of the feature augmentation correction in the core region, which helps with numerical stability by preventing the pseudo-feature from being unrealistically large in the core.
Lastly, P 12 \mathbf{P}^{12} P 12 is an N h N a × N g N a N_hN_a\times N_gN_a N h N a × N g N a matrix:
P j β , i α 12 = w β ∫ d 3 r h j ( r ) ϕ i α β ( r ) . (135) P^{12}_{j\beta,i\alpha}=w_\beta\int\mathrm{d}^{3}r\,h_j(\mathbf{r})\phi_{i\alpha\beta}(\mathbf{r}). \tag*{(135)} P j β , i α 12 = w β ∫ d 3 r h j ( r ) ϕ i α β ( r ) . (135)
The coefficients can be solved by minimizing the loss Eq. (125) , which results in the linear system
\begin{pmatrix}
\mathbf{D}\\
\mathbf{C}
\end{pmatrix}
=
\begin{pmatrix}
\mathbf{P}^{11} & \mathbf{P}^{12}\\
(\mathbf{P}^{12})^{\mathsf{T}} & \mathbf{P}^{22}
\end{pmatrix}^{-1}
\begin{pmatrix}
\mathbf{b}^{1}\\
\mathbf{b}^{2}
\end{pmatrix}, \tag*{(136)}\end{pmatrix}
where
b j β 1 = w β ∫ d 3 k Δ F A , β ( k ) h j A ( k ) (137) b^1_{j\beta}=w_\beta\int\mathrm{d}^{3}k\,\Delta F_{A,\beta}(\mathbf{k})h^A_j(\mathbf{k}) \tag*{(137)} b j β 1 = w β ∫ d 3 k Δ F A , β ( k ) h j A ( k ) (137)
b i α 2 = ∑ β w β ∫ d 3 k Δ F A , β ( k ) ϕ i α β ( k ) . (138) b^2_{i\alpha}=\sum_{\beta}w_\beta\int\mathrm{d}^{3}k\,\Delta F_{A,\beta}(\mathbf{k})\phi_{i\alpha\beta}(\mathbf{k}). \tag*{(138)} b i α 2 = β ∑ w β ∫ d 3 k Δ F A , β ( k ) ϕ i α β ( k ) . (138)
The matrix P \mathbf{P} P is block-diagonal in the angular momentum channels due to the orthogonality of spherical harmonics, so each angular momentum channel can be solved separately. This makes the whole routine fairly efficient, and the number of operations per SCF cycle is linear-scaling in the number of atoms. The Fourier transforms of the g i A g^A_i g i A and h j A h^A_j h j A functions needed for Eqs. (137) and (138) are obtained using the NUMSBT algorithm [131], and the Δ F A , β ( k ) \Delta F_{A,\beta}(\mathbf{k}) Δ F A , β ( k ) is decomposed into angular momentum channels via the Δ F A , L , β ( k ) \Delta F_{A,L,\beta}(\mathbf{k}) Δ F A , L , β ( k ) term obtained in Eq. (85) . Also, note that there are a larger number of α \alpha α and β \beta β indices in the atomic augmentation regions than on the FFT grid. The above fitting procedure is only used for indices going up to the maximum α \alpha α index on the FFT grid. The higher indices (corresponding to larger values of q α q_\alpha q α that are only relevant in the core regions) are solved by simply projecting onto the h j A ( r ) h^A_j(\mathbf{r}) h j A ( r ) basis, i.e. setting
D = ( P 11 ) − 1 b 1 \mathbf{D}=(\mathbf{P}^{11})^{-1}\mathbf{b}^{1} D = ( P 11 ) − 1 b 1
in the subspace of core region-only α \alpha α indices. This procedure is equivalent to assuming that the larger q α q_\alpha q α contributions to θ α \theta_\alpha θ α and F β F_\beta F β are localized in the core region, which is reasonable because these contributions correspond to high densities and short length-scales for Φ α β ( r ) \Phi_{\alpha\beta}(\mathbf{r}) Φ α β ( r ) (Eq. (19) ).
Construction of the Feature Augmentation Projectors
The feature augmentation projectors used in Eq. (92) are constructed as
p i ← n l m ( r ) = Y l m ( r ^ ) P ( 2 n + l , r / r 0 ) (139) p_{i\gets nlm}(\mathbf{r})=Y_{lm}(\hat{\mathbf{r}})P(2n+l,r/r_0) \tag*{(139)} p i ← n l m ( r ) = Y l m ( r ^ ) P ( 2 n + l , r / r 0 ) (139)
P ( n , x ) = { x n ( x 2 − 1 ) 2 x < 1 0 x ≥ 1 . (140) P(n,x)=
\begin{cases}
x^n(x^2-1)^2 & x<1 \\
0 & x\ge1
\end{cases}. \tag*{(140)} P ( n , x ) = { x n ( x 2 − 1 ) 2 0 x < 1 x ≥ 1 . (140)
The on-site function sets f ~ i A ( r ) \tilde{f}^A_i(\mathbf{r}) f ~ i A ( r ) and p ~ j A ( r ) \tilde{p}^A_j(\mathbf{r}) p ~ j A ( r ) are then computed as
p ~ i ( r ) = p i ( r ) f c u t ( r ) (141) \tilde{p}_i(\mathbf{r})=p_i(\mathbf{r})f_{\mathrm{cut}}(\mathbf{r}) \tag*{(141)} p ~ i ( r ) = p i ( r ) f cut ( r ) (141)
f ~ i ( r ) = ∑ j ( S − 1 ) i j p j ( r ) (142) \tilde{f}_i(\mathbf{r})=\sum_j(\mathbf{S}^{-1})_{ij}p_j(\mathbf{r}) \tag*{(142)} f ~ i ( r ) = j ∑ ( S − 1 ) ij p j ( r ) (142)
S i j = ∫ d 3 r p i ( r ) p j ( r ) f c u t ( r ) (143) S_{ij}=\int\mathrm{d}^{3}r\,p_i(\mathbf{r})p_j(\mathbf{r})f_{\mathrm{cut}}(\mathbf{r}) \tag*{(143)} S ij = ∫ d 3 r p i ( r ) p j ( r ) f cut ( r ) (143)
f c u t ( r ) = { 1 2 ( 1 + cos π r r c ) r < r c 0 r ≥ r c (144) f_{\mathrm{cut}}(\mathbf{r})=
\begin{cases}
\frac{1}{2}\left(1+\cos\frac{\pi r}{r_c}\right) & r<r_c \\
0 & r\ge r_c
\end{cases} \tag*{(144)} f cut ( r ) = { 2 1 ( 1 + cos r c π r ) 0 r < r c r ≥ r c (144)
The construction localizes p ~ j A ( r ) \tilde{p}^A_j(\mathbf{r}) p ~ j A ( r ) inside the core region and satisfies the condition that ⟨ f ~ i A ∣ p ~ j A ⟩ = δ i j \langle\tilde{f}^A_i|\tilde{p}^A_j\rangle=\delta_{ij} ⟨ f ~ i A ∣ p ~ j A ⟩ = δ ij .
Kyle Bystrom; Boris Kozinsky
Table of All GMTKN55 Subset Error Metrics
This section consists of a large table (shown below) with detailed GMTKN55 error metrics for each functional presented in this work. All entries contain the mean absolute deviation (MAD) relative to GMTKN55 reference values, except for the aggregate errors WT2 (WTMAD-2 as defined by Goerigk et al. [1 ] ) and MOM (mean of means, as defined in § 6 A). The “DTR tr,” “DTR val,” and “DTR test” labels denote the train/test partition used to train NL-MGGA-DTR on GMTKN55, while “tr/val” and “test” refer to the train/test partition used for all other functionals. All units are in kcal/mol.
Table 1 (PDF p. 35).
Table 2.
Table 3.
Table 4.
Table 5.
Analysis of GMTKN55 Results with Different Error Averaging Metrics
Results Relative to PBE0
As mentioned in § 6 A, the analysis of functional accuracy can be sensitive to the averaging scheme for aggregate errors across the entire GMTKN55 database. In Figure S1 , we plot the average deviations relative to PBE0 of semilocal and ML functionals using two alternative metrics to the mean of means (MoM, i.e. the mean of the mean absolute deviations for the sub-databases) used in § 6 A: mean absolute deviation (MAD, top) and weighted MAD (specifically, the WTMAD-2 metric used by Goerigk et al. [1 ] ). The general trends for these metrics are similar to those for the MoM; the SL-MGGA is more accurate than the SL-GGA and NL-GGA, and the NL-MGGA is more accurate than the SL-MGGA. However, the transferability issues suffered by the SL-GGA and NL-GGA appear even more severe when using the WTMAD-2 metric, and the NL-GGA is actually less accurate than the SL-GGA overall. The WTMAD-2 metric gives larger weight to sub-databases with smaller reaction energy magnitudes (e.g. noncovalent interactions are weighted more heavily than atomization energies), so this worse performance for WTMAD-2 suggests that the SL-GGA and NL-GGA cannot accurately describe the smaller density variations associated with these reactions. Notably, the SL-MGGA suffers a similar problem on the intermolecular NCI database; as a result, the NL-MGGA is the only functional that is closer to PBE0 than r2 ^{2} 2 SCAN for every sub-database, a testament to its robustness.
Figure S1. Deviations from PBE0 for semilocal and PBE0/CIDER functionals using the MAD (top) and WTMAD-2 (bottom) error metrics.
Results Relative to Reference Values
Figure S2 shows the performance of the semilocal, hybrid, and CIDER functionals on the GMTKN55 database with the three different averaging metrics MoM, MAD, and WTMAD-2 discussed in the previous section. The same dispersion corrections are used as in § 6 D. The results with MoM and MAD are very similar, while the performance of CIDER functionals relative to hybrid DFT is notably worse for the WTMAD-2. This again speaks to the difficulty of capturing small relative energies accurately for CIDER functionals, which will need to be a topic
Figure S2. Errors of different semilocal, hybrid, and CIDER functionals on GMTKN55 with different error metrics.
a ^{a} a This work.
b ^{b} b Borlido et al. [2 ]
of future research. However, it is notable that the SL-MGGA models suffer from this problem more severely than the NL-MGGA-DTR; PW6B95/SL-MGGA has a worse WTMAD-2 than r2SCAN, while PW6B95/NL-MGGA-DTR has a WTMAD-2 that is better than r2SCAN and comparable to B3LYP and PBE0. As shown in Table S1 , the NL-MGGA and NL-MGGA-PBE models also significantly outperform the SL-MGGA models, so the worse performance of SL-MGGA is not an artefact of the different training set used for NL-MGGA-DTR. From this analysis, we conclude that introducing a more flexible feature set like that used for the NL-MGGA and its variants results in models that perform better for a variety of systems and therefore are less sensitive to a particular error metric.
More Data on Band Gaps
Table S2 contains the error metrics for semilocal, PBE0/CIDER, and hybrid functionals for all 453 systems in the database of Borlido et al. [2 ] that do not contain La, Th, or Yb. Note that for several systems, we used SG15 norm-conserving pseudopotentials [3 ] as described in Appendix A11.
Table S2. Error statistics for 453 band gaps from the database published by Borlido et al. [2 ] . The ML functionals use the PBE0(α \alpha α )/CIDER formalism. a ^{a} a This work. b ^{b} b Borlido et al. [2 ] .
Validation Results and Selected Hyperparameters
Table S3 -Table S6 show the final validation results for each functional type, ordered by validation loss (Eq. (119) ) and labeled by internal project code name. Table S7 shows the final hyperparameters for the CIDER functionals presented in the main text. For all of these figures, see Appendix A8 for hyperparameter definitions.
Table S3. SL-GGA validation results according to Eq. (119) , with PBE provided as reference. Loss in kcal/mol.
Table S6. NL-MGGA validation results according to Eq. (119) , with PBE provided as reference. Loss in kcal/mol.
Table S7. Names and hyperparameters for CIDER functionals presented in the main text.
Density Accuracy
The density error for a molecule i i i is measured using numerical integrals as
Δ i = ∑ σ ∑ g w g i ∣ n σ i ( r g ) − n σ i , r e f ( r g i ) ∣ ∑ σ ∑ g w g i n σ i , r e f ( r g i ) (145) \Delta_i = \frac{\sum_{\sigma}\sum_g w_g^i \left|n_\sigma^i(r_g)-n_\sigma^{i,\mathrm{ref}}(r_g^i)\right|}{\sum_{\sigma}\sum_g w_g^i n_\sigma^{i,\mathrm{ref}}(r_g^i)} \tag*{(145)} Δ i = ∑ σ ∑ g w g i n σ i , ref ( r g i ) ∑ σ ∑ g w g i n σ i ( r g ) − n σ i , ref ( r g i ) (145)
where w g i w_g^i w g i and r g i r_g^i r g i are the real-space integration weights and positions for molecule i i i . The aggregate percentage error for a sub-database is
Δ ‾ D B = 100 N D B ∑ i ∈ D B Δ i . (146) \overline{\Delta}_{\mathrm{DB}} = \frac{100}{N_{\mathrm{DB}}}\sum_{i\in\mathrm{DB}}\Delta_i. \tag*{(146)} Δ DB = N DB 100 i ∈ DB ∑ Δ i . (146)
Table S8 shows the density deviations from PBE0 for PBE, r2 ^{2} 2 SCAN, and the ML functionals over several subsets of GMTKN55. The NL-MGGA achieves the best match to PBE0 of the ML functionals, and this improvement is transferable to the test set. r2 ^{2} 2 SCAN, however, matches PBE0 more closely than the ML functionals for most subsets. This suggests there might still be some room for improvement in designing ML functionals with accurate densities. However, NL-MGGA yields smaller deviations from PBE0 densities than PBE does for every subset, so the density distributions get closer to those of PBE0 when the ML exchange is added.
Table S8. Average percentage density deviations from PBE0 (Eq. (146) with PBE0 densities for n i r e f n_i^{\mathrm{ref}} n i ref in Eq. (145) ) for semilocal and ML functionals. The ML functionals use the PBE0/CIDER surrogate hybrid form. Whether each sub-database was used for training or testing is indicated.
Table S4. NL-GGA validation results according to Eq. (119) , with PBE provided as reference. Loss in kcal/mol.
Table S5. SL-MGGA validation results according to Eq. (119) , with PBE provided as reference. Loss in kcal/mol.
Bond lengths and lattice constants
To assess the accuracy of CIDER functionals for structure prediction, we performed two benchmarks. First, we computed the bond lengths of six diatomic molecules in PySCF (Table S9 ) and compared the PBE0/CIDER bond length predictions to those of PBE0. For these calculations, the def2-QZVPPD basis set was used without density fitting, level 4 PySCF grids were used, and the internal nonlocal feature evaluation settings were q max = 3000 q_{\max} = 3000 q m a x = 3000 , β a u x = 1.6 \beta^{\mathrm{aux}} = 1.6 β aux = 1.6 , λ = 1.6 \lambda= 1.6 λ = 1.6 , and l max = 10 l_{\max} = 10 l m a x = 10 . The results are shown in Table S9 and suggest that with increasing model complexity, the bond lengths more closely match PBE0. For reference, the atomization energy predictions for these molecules are shown in Table S10 and demonstrate a similar trend. Note that the CIDER models are trained on these systems, so this benchmark serves only as a basic sanity test for CIDER forces, and not as a test of generalizability.
Table S9. Bond lengths for diatomic molecules in Å. Note that these molecules are contained in the CIDER training set. All ML functionals use the PBE0/CIDER surrogate hybrid form.
Table S10. Atomization energies for diatomic molecules in kcal/mol. Note that these molecules are contained in the CIDER training set. All ML functionals use the PBE0/CIDER surrogate hybrid form.
For the second benchmark, we computed the lattice constants of the materials in the SOL62 database [4 , 5 ] in GPAW (with the same settings as were used for the cohesive energy calculations, c.f. Sections A3 and A10) and compared the lattice constants to their experimental values. We separated the results into the train, validation, and test partitions to evaluate model generalizability. Also, we only computed the lattice constants for the PBE0/NL-MGGA-DTR CIDER functional since it was the most numerically stable. The results are shown in Table S11 . No zero-point vibrational corrections were performed, but the mean absolute lattice constant deviation with and without these corrections for PBE is 0.01 Å, significantly smaller than the typical deviations between the DFT predictions and experiment.
Table S11. Lattice constant predictions for the SOL62 database [4 , 5 ] using functionals including PBE0/NL-MGGA-DTR, in units of Å. The midrule bars separate the training, validation, and test sets, in that order from top to bottom. Except for the mean absolute error reported separately for the train, validation, and test sets, error metrics are for the entire dataset.
The MAE of PBE0/NL-MGGA-DTR compared to experiment is slightly smaller than that of PBE on the test set, but larger than that of r2 ^{2} 2 SCAN and HSE06. Due to unusually large errors on a few alkali and alkaline earth metals (i.e. K, Rb, and Ba), PBE0/NL-MGGA-DTR also has a slightly larger RMSE than PBE. The results suggest that overall, the ML functional can predict reasonable, GGA-accuracy structures for materials outside the training set, but more work is needed to achieve the accuracy of more advanced functionals like r2 ^{2} 2 SCAN and HSE06 for structure optimization. In the future, we will also need to investigate the cause of NL-MGGA-DTR severely underbinding the larger alkali/alkaline earth metals (even compared to HSE06).
More detailed defect transition level data
This section provides more details on the defect transition level data. The bulk lattice constant and band gap, determined by the methods in Section A5, are tabulated in Table S12 . The defect transition levels themselves are shown in Table S13 .
Table S12. Bulk data for silicon for different functionals. The silicon experimental lattice constant was taken from Ref. [4 ] .
Table S13. Transition levels for several defects in silicon for different functionals and correction schemes. “Fr.” is the Freysoldt correction scheme, while “Pot.” is the potential alignment-only scheme. All PBE0(α \alpha α )/CIDER calculations use the NL-MGGA-DTR exchange functional. CIDER refers to PBE0(0.3)/NL-MGGA-DTR. The experimental references are from the following sources: vacancy [6 ] , boron [7 ] , phosphorus [8 ] , copper [9 ] , and sulfur [10 ] [11].
The Cu +/0 Transition Level
As discussed in the main text, the copper +1/0 transition level computed with PBE0(0.3)/NL-MGGA-DTR is not converged with the supercell size and k-point mesh used with HSE06 [12] by Sharan et al. [13]. To further illustrate this point, Table S14 shows the +1/0 energy transition level for copper E ( + / 0 ) E(+/0) E ( + /0 ) for HSE06 and PBE0(0.3)/NL-MGGA-DTR for several k-point meshes. For this comparison, finite-size effects are ignored, so
Table S14. +/0 transition level of the copper substitutional defect in silicon (excluding finite-size effects), computed using 64-atom supercells and different k-point meshes. CIDER refers to PBE0(0.3)/NL-MGGA-DTR.
E ( + / 0 ) = E X 0 − E X 1 − ϵ V B M (147) E(+/0) = E_{X}^{0} - E_{X}^{1} - \epsilon_{\mathrm{VBM}} \tag*{(147)} E ( + /0 ) = E X 0 − E X 1 − ϵ VBM (147)
(with X X X being the defect supercell). For consistency, all calculations were performed with a geometry relaxed with HSE06. The lattice constant was optimized with HSE06 with an 8 × 8 × 8 8 \times8 \times8 8 × 8 × 8 Γ \Gamma Γ -centered k-point mesh, and the defect geometries were optimized with HSE06 with the 2 × 2 × 2 2 \times2 \times2 2 × 2 × 2 Γ \Gamma Γ -centered k-point mesh. No restrictions were placed on the magnetic moment for structural relaxation, but for consistency, all static calculations for computing energy level differences were computed using fixed total magnetic moments of S = 1 / 2 S = 1/2 S = 1/2 for the neutral state and S = 0 S = 0 S = 0 for the +1 state. These are the ground-state spins as determined by Sharan et al. [13], though it is worth noting that for the 512-atom supercell and 2 × 2 × 2 2 \times2 \times2 2 × 2 × 2 Γ \Gamma Γ -centered k-point mesh, the S = 3 / 2 S = 3/2 S = 3/2 spin state is favored over S = 1 / 2 S = 1/2 S = 1/2 . The
HSE06 calculations were performed with VASP [14], while the CIDER calculations were performed with GPAW [15]. The primary takeaway from Table S14 is that neither the CIDER nor the HSE06 calculations are converged with the 2 × 2 × 2 2 \times2 \times2 2 × 2 × 2 k-point meshes. The most significant deviation is between the 2 × 2 × 2 2 \times2 \times2 2 × 2 × 2 MP and 4 × 4 × 4 4 \times4 \times4 4 × 4 × 4 Γ \Gamma Γ -centered mesh for HSE06, for which the difference is 0.15 eV. In addition, PBE0(0.3)/NL-MGGA-DTR and HSE06 are in good agreement for all calculations for a given k-point mesh, with deviations of 0.02-0.04 eV.
Numerical Precision of Nonlocal Feature Implementation
One of the key details in determining the practical usefulness of a functional is its numerical precision, or how accurately the approximate numerical evaluation of the model XC energy matches the hypothetical “exact” evaluation of the model. This section covers a few useful metrics of numerical precision for the NL-MGGA-DTR functional used in the main text.
Comparison of Numerical Feature Integration to Fast Feature Evaluation in PySCF
Section IIE describes how the evaluation of nonlocal density features can be accelerated within all-electron, localized orbital codes for molecular systems. One might be interested in how accurately this approximation reproduces the simple numerical integration approach, which is much slower:
G i ( r 1 ) = ( B i + B 0 2 ) 3 / 2 ∫ d 3 r 2 Φ ( a ( r 2 ) , b i ( r 1 ) , r 12 ) n ( r 2 ) (148) G_i(\mathbf{r}_1) = \left(\frac{B_i+B_0}{2}\right)^{3/2} \int\mathrm{d}^3r_2 \, \Phi\left(a(\mathbf{r}_2), b_i(\mathbf{r}_1), r_{12}\right)n(\mathbf{r}_2) \tag*{(148)} G i ( r 1 ) = ( 2 B i + B 0 ) 3/2 ∫ d 3 r 2 Φ ( a ( r 2 ) , b i ( r 1 ) , r 12 ) n ( r 2 ) (148)
≈ ( B i + B 0 2 ) 3 / 2 ∑ g w g Φ ( a ( r g ) , b i ( r 1 ) , ∣ r 1 − r g ∣ ) n ( r g ) (149) \approx\left(\frac{B_i+B_0}{2}\right)^{3/2} \sum_g w_g\Phi\left(a(\mathbf{r}_g), b_i(\mathbf{r}_1), \lvert\mathbf{r}_1-\mathbf{r}_g\rvert\right)n(\mathbf{r}_g) \tag*{(149)} ≈ ( 2 B i + B 0 ) 3/2 g ∑ w g Φ ( a ( r g ) , b i ( r 1 ) , ∣ r 1 − r g ∣ ) n ( r g ) (149)
Table S15 displays the deviation in atomization energy obtained for some small molecules with the PBE0/NL-MGGA-DTR functional, when the functional is evaluated from simple numerical integration (Eq. (149) , “Numerical”) vs. the auxiliary approximation of Section IIE (“Auxiliary”). The deviations for molecules with first and second-row atoms (the first 8 molecules) are all less than 1 meV, and only 0.3 meV on average. For molecules containing larger atoms (the last 5 molecules, with elements from row 3 or higher), the errors are a bit higher, but still less than 2 meV in all cases. All calculations were performed in PySCF with the def2-QZVPPD basis set, using level 3 grids with the default NWChem pruning for the auxiliary basis approach and level 3 grids without pruning (necessary for numerical precision in the core region) for the numerical integration. See the main text Appendix A10 for details on the CIDER auxiliary expansion settings.
TABLE S15. Atomization energies of small molecules predicted using the PBE0/NL-MGGA-DTR functional with numerical integration or auxiliary expansion used to evaluate the nonlocal features (in eV). MAD is mean absolute deviation, and RMSD is root mean square deviation.
Table S15.
Comparison of Localized-Orbital All-Electron Implementation and PAW Plane-wave Implementation
Having seen that the auxiliary approximation of Section IIE agrees quite accurately with the direct numerical integration of the nonlocal features, we now turn our attention to the PAW plane-wave implementation of the nonlocal CIDER features in Section IIF. Due to the approximations inherent in PAW [16], as well as the differences between plane-wave and Gaussian-type orbital basis sets, results from Gaussian-type orbital and plane-wave calculations are not expected to agree completely, but ideally the deviations between these methods should be no larger when using CIDER functionals than when using standard functionals. In Table S16 , we see that this is indeed the case for PBE0/NL-MGGA-DTR. For molecules with smaller atoms (rows 1 and 2), there is an average deviation between PySCF (Gaussian-type orbital DFT) and GPAW (plane-wave DFT with the PAW method) of 0.02 eV for both PBE and PBE0/NL-MGGA-DTR. The PAW potentials for GPAW were generated with the PBE functional, so it is reassuring that using a CIDER functional instead does not increase the deviation between the all-electron and PAW calculations significantly. For molecules with larger atoms, the disagreement is larger (around 0.1 eV on average), but again no larger when using CIDER functionals than when using PBE. There are two possible causes of the larger deviations seen for the molecules with larger atoms. First, these molecules are simply larger and therefore have more bonds being broken during atomization, so the same error per bond will yield a larger total error. Second, the def2-QZVPPD basis uses effective core potentials for atoms larger than Kr rather than performing all-electron calculations, which adds an additional source of disagreement between GPAW and PySCF. For these calculations, the same PySCF settings were used as in Section S8 A. For GPAW, an energy cutoff of 800 eV and grid spacing of 0.108 Å were used to store the wave functions, and the XC energy was integrated on a 2-times denser grid than the wave functions. 5 Å of vacuum was added on all sides of each system to avoid spurious periodic image interactions. The same CIDER-specific settings were used as in the main text, Appendix A10.
TABLE S16. Atomization energies of small molecules predicted using the PBE and PBE0/NL-MGGA-DTR functionals in the PySCF and GPAW codes (in eV). MAD is mean absolute deviation, and RMSD is root mean square deviation.
Table S16.
Convergence of Total Energy with Respect to Grid Size
Having verified that both implementations of the nonlocal density features are numerically accurate, we now turn to the question of energy convergence with respect to the grid size used to integrate the XC energy. Ideally, an XC functional should be sufficiently smooth that numerical integrals of the functional over coarse grids accurately match more precise numerical integrals using denser grids. Being able to use a coarser grid can significantly decrease the cost of a DFT calculation. As such, we are interested in how quickly CIDER functionals converge with respect to grid size compared to other functionals.
Figure S3 depicts the convergence of total energy of the HF molecule and Kr atom with respect to grid size for different functionals, as computed with PySCF. The calculations were performed with a def2-QZVPPD basis set. The most immediately obvious observation is that PBE (a GGA functional) has negligible numerical integration error compared to the SCAN and r 2 r^{2} r 2 SCAN meta-GGAs and the CIDER functionals, which use the NL-MGGA-DTR model from the main text. The original SCAN functional is known to have significant numerical stability issues, and this is illustrated in Figure S3 by the large integration errors for smaller grid sizes. The r 2 r^{2} r 2 SCAN functional fixes these issues. Reassuringly, the errors for the CIDER exchange functional (HF/CIDER) are not much larger than for r 2 r^{2} r 2 SCAN for most grid sizes for both systems. In practice, it will be more common to use functionals with a fraction of CIDER like PBE0/CIDER. Since these functionals only use a small fraction of the CIDER functional and otherwise rely on GGA-level information, the numerical error is smaller than for the full CIDER functional.
Figure S3. Convergence of the total energy of the HF molecule and Kr atom for different functionals with respect to the PySCF grid level. A larger grid level indicates a larger grid, and the energy for each functional is referenced to the total energy for grid level 5. HF/CIDER corresponds to the CIDER exchange functional with no correlation. The NL-MGGA-DTR model is used for the CIDER functionals. The plots on the left side show the full convergence plots, while the plots on the right provide a zoomed-in view to present a clearer picture of the behavior of the functionals for larger grid sizes.
Figure S4 shows the convergence of total energy of bulk Si and MgO with respect to grid size for different functionals, as computed with GPAW. The calculations used a 520 eV energy cutoff for both systems, an 8 × 8 × 8 8 \times8 \times8 8 × 8 × 8 k-point mesh for Si, and a 4 × 4 × 4 4 \times4 \times4 4 × 4 × 4 k-point mesh for MgO. For these systems, SCAN and r 2 r^{2} r 2 SCAN have similar convergence behavior with respect to grid size (though SCAN is still a bit less stable), and PBE0/CIDER also matches the convergence behavior of these functionals fairly closely. However, HF/CIDER has slower convergence due to the larger fraction of CIDER exchange, especially for MgO. Therefore, improving the numerical precision of CIDER functionals for solid-state systems could be important for future applications.
Figure S4. Convergence of the total energy of bulk Si and MgO for different functionals with respect to the grid spacing setting h h h in GPAW. (Note: the XC integration grid has spacing h / 2 h/2 h /2 .) The energy for each functional is referenced to the total energy for h = 0.06 h = 0.06 h = 0.06 Å. HF/CIDER corresponds to the CIDER exchange functional with no correlation. The NL-MGGA-DTR model is used for the CIDER functionals. The plots on the left side show the full convergence plots, while the plots on the right provide a zoomed-in view to present a clearer picture of the behavior of the functionals for larger grid sizes.
Accuracy of Nyström Approximation for Gaussian Process
In the main text (Eqs. (37) and (38) ), the Nyström approximation was used to reduce the computational complexity of fitting the Gaussian process models. The control points x ~ a \tilde{x}_{a} x ~ a for this method were selected using pivoted Cholesky decomposition as described in Appendix A6. This method requires selecting a tolerance parameter, for which 10 − 5 10^{-5} 1 0 − 5 was chosen in the main text. The Nyström approximation becomes exact if all feature vectors in the training set are included as control points, though this negates the computational efficiency of the approximation.
It is not computationally feasible to evaluate the covariance kernel exactly via Eq. (35) for the full training set, but we can test the effect of changing the number of control points used in the Nyström approximation, which becomes more accurate as more control points are added. To change the number of control points, we test a larger tolerance of 10 − 3 10^{-3} 1 0 − 3 (which decreases the number of control points) and a smaller tolerance of 10 − 8 10^{-8} 1 0 − 8 (which increases the number of control points) in the Cholesky factorization step described in Appendix A6. Table S17 shows the deviations between models trained using these different tolerance parameters, with the NL-MGGA-DTR functional form and training set used as an example. The deviations are quite small, with the 10 − 3 10^{-3} 1 0 − 3 and 10 − 5 10^{-5} 1 0 − 5 tolerances disagreeing by a few meV on average and the 10 − 8 10^{-8} 1 0 − 8 and 10 − 5 10^{-5} 1 0 − 5 tolerances disagreeing by a few tenths of an meV on average.
Table S17. Mean absolute deviations between the NL-MGGA-DTR model trained with a tolerance factor for the Nyström approximation of 10 − 5 10^{-5} 1 0 − 5 (as used in the main text) and models trained with tolerances of 10 − 3 10^{-3} 1 0 − 3 (“large tol”) and 10 − 8 10^{-8} 1 0 − 8 (“small tol”), in meV. SOL62 corresponds to the solid-state cohesive energy database, and all the other datasets constitute the small molecule category of the GMTKN55 database. MoM indicates the mean-of-means error for the 18 sub-databases of GMTKN55 in this table.