betaFeedback

WorksPhysics and Astronomy

Learning Properties of Ordered and Disordered Materials from Multi-fidelity Data

Abstract

Predicting the properties of a material from the arrangement of its atoms is a fundamental goal in materials science. While machine learning has emerged in recent years as a new paradigm to provide rapid predictions of materials properties, their practical utility is limited by the scarcity of high-fidelity data. Here, we develop multi-fidelity graph networks as a universal approach to achieve accurate predictions of materials properties with small data sizes. As a proof of concept, we show that the inclusion of low-fidelity Perdew-Burke-Ernzerhof band gaps greatly enhances the resolution of latent structural features in materials graphs, leading to a 22-45% decrease in the mean absolute errors of experimental band gap predictions. We further demonstrate that learned elemental embeddings in materials graph networks provide a natural approach to model disorder in materials, addressing a fundamental gap in the computational prediction of materials properties.

In silico predictions of the properties of materials can most reliably be carried out using ab initio calculations. However, their high computational expense and poor scalability have limited their application mostly to materials containing <1000< 1000 atoms without site disorder. Further, a rule of thumb is that the more accurate the ab initio method, the higher the computational expense and the poorer the scalability. [1, 2] [3] It is therefore no surprise that supervised machine learning (ML) of ab initio calculations has garnered substantial interest as a means to develop efficient surrogate models for materials property predictions. 4 State-of-the-art ML models encode structural information (e.g., as graphs5,6 or local environmental features7–9) in addition to composition information, allowing them to distinguish between polymorphs that may have vastly different properties.

Most frustratingly, while building ML models from high-accuracy calculations or experiments would yield the greatest value, obtaining sufficient data to reliably train such models is also the most challenging. For example, the number of Perdew-Burke-Ernzerhof (PBE)10 calculations in large, public databases such as the Materials Project11 and Open Quantum Materials Database (OQMD)12 is on the order of 105−10610^{5} - 10^{6}, while the number of more accurate Heyd-Scuseria-Ernzerhof (HSE)13 calculations is at least two orders of magnitude fewer.

Similarly, while Becke, 3-parameter, Lee-Yang-Parr (B3LYP)/PBE calculations are available for millions of molecules,14 “gold standard” coupled-cluster single-, double-, and perturbative triple-excitations (CCSD(T)) calculations are only available for perhaps thousands of small molecules. Even fewer are the number of high-quality experimental data points.15

A potential approach to address this challenge is through multi-fidelity models16 that combine low-fidelity data with high-fidelity data. In the handful of previous works utilizing this approach in ML of materials properties, most are two-fidelity models utilizing a co-kriging17 approach, which assumes an approximately linear relationship between targets of different fidelity. The training of co-kriging models scales as O(N3)O(N^{3}) (where NN is the number of data points), which becomes prohibitively expensive when NN exceeds 10,000. Further, these efforts have been limited to specific properties of single structure prototypes.18,19 Similarly, the transfer learning and Δ\Delta-learning20 are either two-fidelity approaches or non-trivial21 to extend to more than two fidelities. Multi-task neural network models22 can handle multi-fidelity data and scale linearly with the number of data fidelities, but require homogeneous data that have all properties labeled for all the data, which is rarely the case in materials property datasets.

Graph networks are a general, composable deep learning framework that supports both relational reasoning and combinatorial generalization.23 Previously, the current authors have shown that materials graph network (MEGNet) models significantly outperform prior ML models in predicting the properties of both molecules and crystals.5 However, previous graph network and other state-of-the-art models, such as the Crystal Graph Convolutional Neural Networks (CGCNN)6 and SchNet,24 are single fidelity, typically trained on a large PBE-computed dataset, and have not been extended to multi-fidelity datasets with varying sizes. Further, all prior models have been limited to ordered materials only. Here, we develop multi-

Multi-fidelity materials graph networks

Figure 1. Multi-fidelity materials graph networks. a, Representation of a material in a graph network model, with atoms as the nodes, bonds as the edges coupled with a structure-independent global state. The input atomic feature is embedded atomic number of the element. The bond feature vector is the Gaussian expanded distance. The fidelity of each data is encoded as an integer (e.g., 0 for PBE, 1 for GLLB-SC, 2 for HSE, 3 for SCAN and 4 for experiment or Exp). b, A materials graph network model is constructed by stacking graph convolution layers. In each graph convolution layer, sequential updates of atomic, bond and state features are performed using information from connected neighbors in the graph. The output graph in the last layer is then readout and processed in a neural network to arrive at the final prediction.

fidelity graph networks as a generalized framework for materials property prediction across computational methodologies and experiments for both ordered and disordered materials.

Results

Multi-fidelity Graph Networks

Fig. 1 depicts a schematic of the multi-fidelity graph network framework. The starting point is a natural graph representation of a material, where the atoms are the nodes and the bonds between them are the edges. The input atomic attributes are simply the atomic numbers of the elements passed to a trainable elemental embedding matrix to obtain a length-16 elemental embedding vector, and the bond attributes are the Gaussian-expanded distances. The state attribute vector provides a portal for structural-independent features to be incorporated into the model. Here, the data fidelity level (e.g., computational method or experiment) is encoded as an integer and passed to a trainable fidelity embedding matrix to get a length-16 fidelity embedding vector, forming the input state attributes. A graph network model is built by applying a series of graph convolutional layers that sequentially exchange information between atoms, bonds and the state vector. In the final step, the latent features in the output graph are read out and passed into a neural network to arrive at a property prediction. Further details are available in the Methods section.

We have selected the prediction of the band gap (EgE_g) of crystals as the target problem because of its great importance in a broad range of technological applications, including photovoltaics, solar water splitting, etc., as well as the availability of data of multiple fidelities. Another application of the approach to the prediction of multi-fidelity molecular energies is demonstrated in Supplementary § 18. The low-fidelity (low-fi) dataset comprises 52,348 PBE band gaps from the Materials Project, 11 For the demonstration of model transferability, we also show the prediction of the multi-fidelity molecular energy. The high-fidelity (high-fi) computed datasets comprise 2,290 Gritsenko-Leeuwen-Lenthe-Baerends with solid correction (GLLB-SC), 25–27 472 strongly constrained and appropriately normed (SCAN) 28 29 and 6,030 Heyd-Scuseria-Ernzerhof (HSE) 13,30 band gaps. Experimental (Exp) band gaps for 2,703 ordered crystals and 278 disordered crystals 31 are considered as a separate high-fi dataset. The computationally least expensive PBE functional is well-known to systematically underestimate the band gap, 32 and the high-fi functionals correct this to varying degrees. The data within each fidelity was randomly split into 80% training, 10% validation and 10% test data and repeated six times for all models in this work. The statistics (mean and distribution) of the mean absolute errors (MAEs) on the test datasets are reported to provide an accurate assessment of model reliability.

Band Gaps of Ordered Structures

Single-fidelity, or 1-fi, graph network models for the band gaps of ordered crystals were first developed for each fidelity in isolation. The MAE of the 1-fi models (Fig. 2a) is related to the data size as well as the mean absolute deviation (MAD, see Supplementary Dataset 1) within each dataset. The PBE dataset is the largest with a small MAD, and the 1-fi PBE models have the smallest average MAE of 0.27 eV. The average MAEs for the computed 1-fi models increases in the order of PBE << HSE << GLLB-SC << SCAN, in inverse order to the dataset size. The lower MAE of the 1-fi Exp model compared to the 1-fi HSE model despite having a smaller data set size may be attributed to the relatively large fraction of metals (with zero band gap) in that dataset, which leads to smaller MAD.

Multi-fidelity graph network models utilizing the large PBE dataset with data from other fidelities can mitigate this data quality/quantity-performance trade-off. Significantly lower average MAEs are achieved across all high-fi computed and the experimental predictions (Fig. 2a). The 5-fi models, i.e., the models fitted using all available data, substantially improve on the MAE on the high-fi predictions over the 2-fi models, at the expense of a small increase in the MAE of the low-fi PBE predictions. The error distributions broken down in metals and non-metals for the 5-fi models are shown in Extended Data Figure 1.

With the exception of the extremely small set of SCAN data on metals (17 data points), all model errors have a Gaussian-like distribution centered at zero.

Other 2-fi, 3-fi and 4-fi models, with and without PBE, were also explored (Supplementary Dataset 1). The multi-fidelity models without PBE generally have higher MAEs than the multi-fidelity models with PBE, though they typically still outperform the 1-fi models. The 4-fi models that exclude the very small SCAN dataset, i.e., models trained on PBE, GLLB-SC, HSE and Exp data, outperform the 5-fi models across all non-SCAN fidelities, which indicates that the poor quality of the SCAN dataset may have degraded performance. The reduction in average MAE of the 4-fi models over the 1-fi models range from ∼22%\sim22\% for the Exp band gap to ∼45%\sim45\% for the HSE band gap. Further, an increase in the number of fidelities also tends to improve model consistency, i.e., lower the standard deviation in the MAE.

The multi-fidelity graph network models significantly outperform prior ML models in the literature. The best reported GLLB-SC model has a root-mean-squared error (RMSE) of 0.95 eV,33 much higher than the 4-fi RMSE of 0.68 eV on the GLLB-SC predictions. For experimental band gaps, Zhuo et al. 31 reported MAE and RMSE of 0.75 eV and 1.46 eV for a test set of 10 compounds using a support vector regression model; the MAE and RMSE for the 4-fi model on the experimental band gap of these compounds are 0.65 eV and 1.39 eV, respectively. Zhuo et al. 31 also reported an RMSE of 0.45 eV on the entire experimental dataset, but the dataset contains duplicated band gaps for the same composition and thus is an inaccurate metric of model performance.

To compare alternative approaches, we have also constructed baseline 1-fi-stacked models, where a linear model is fitted for each high-fi dataset to the optimized 1-fi PBE model. This is akin to a model stacking approach and can be justified based on the relatively strong correlation between the high-fi computed and PBE band gaps (Extended Data Figure 2).34 The multi-fidelity models outperform the 1-fi-stacked models, with especially large reductions in average MAEs of up to 38% on arguably the most valuable experimental band gap predictions and 44–56% on the GLLB-SC and HSE predictions. Another alternative approach explored is transfer learning, whereby the graph convolution layers from the PBE 1-fi models were fixed and the final output layers were retrained with higher fidelity data. Compared to the 2-fi models, the transfer learning models have somewhat lower model errors on the GLLB-SC and SCAN datasets but much higher errors on the most valuable Exp dataset, as shown in Supplementary Dataset 2. It should be noted that transfer learning is fundamentally a two-fidelity approach and requires a two-step training process. The 4-fi models outperform the transfer learning models on the Exp dataset even further. These results indicate that multi-fidelity graph network architecture is able to capture complex relationships between datasets of different fidelities. The errors of other combinations of fidelities are listed in Supplementary Dataset 1.

To gain insights into the effect of low-fi and high-fi data size on model accuracy, 2-fi models were developed using sampled subsets of each high-fi computed/experimental dataset (Nhigh−fiN_{\mathrm{high-fi}}) together with different quantities of data from the low-fi PBE dataset (NPBEN_{\mathrm{PBE}}). From Fig. 2b-e, it may be observed that adding low-fi PBE data results in a significant decrease in the average MAEs of the high-fi predictions in all cases. The average MAEs of the 2-fi models follow an approximately linear relationship with log⁡10NPBE\log_{10} N_{\mathrm{PBE}}. With the exception of the SCAN 2-fi models, the magnitude of the slope decreases monotonically with an increase in Nhigh−fiN_{\mathrm{high-fi}}, i.e., the largest improvements are observed in the most data-constrained models. The nearly constant slope for the 2-fi SCAN models may be attributed to the extremely small size of the SCAN dataset as well as its strong correlation to the PBE dataset (Extended Data Figure 2).

Latent Space Visualization

We compared the latent structural features extracted from the 1-fi and the 2-fi models trained using 100 experimental data points without and with PBE data, respectively. The t-distributed Stochastic Neighbor Embedding (t-SNE) 35 2D projections of the latent structure

Test mean absolute errors of multi-fidelity graph network model predictions on ordered crystal band gaps

Figure 2. Test mean absolute errors (MAEs) of multi-fidelity graph network model predictions on ordered crystal band gaps. a, Performance of graph network models with different fidelity combinations. The 4-fi models used the PBE, GLLB-SC, HSE and Exp datasets, i.e., the very small SCAN dataset is excluded. All errors were obtained on the corresponding test sets of the fidelity. The error bars show one standard deviation and the dots are the individual model errors. Average MAEs of b, GLLB-SC, c, SCAN, d, HSE, and e, experimental band gaps of 2-fi models trained using sampled datasets for each high-fidelity data and PBE data. For each sub-plot, the error line is lowered with more high-fidelity data. The x-axis is plotted on a log scale and the shaded areas indicate one standard deviation of the MAE. ss indicates the slope for a linear fit of MAE to log⁡10NPBE\log_{10} N_{\mathrm{PBE}}

features (Figure 3a and b) show that the inclusion of the large PBE dataset in the 2-fi model results in superior structural representations that clearly separate structures with large band gap differences.

Effect of including low-fidelity PBE data on latent structural features

Figure 3. Effect of including low-fidelity PBE data on latent structural features. Two dimensional t-distributed Stochastic Neighbor Embedding (complexity = 1000) projection of features for a, 1-fi and b, 2-fi models trained using 100 experimental data points and the entire PBE dataset. The markers are colored according to the experimental band gap. Plots of the experimental band gap difference (ΔEg\Delta E_g) against normalized latent structural feature distance (dFd_F) in arbitrary units (a.u.) for the c, 1-fi and d, 2-fi-PBE models trained on all available experimental data. The dashed lines indicate the envelope of the maximum ΔEg\Delta E_g at each dFd_F. The scattering points are sub-sampled by a factor of 15.

This separation can be further quantified by plotting the band gap difference ΔEg\Delta E_g against the distance in the normalized structural features dFd_F between all 3,651,753 unique pairs of crystals in the experimental data, as shown in Figure 3c and d). The 1-fi model for experimental band gaps has poor resolution, especially for large ΔEg\Delta E_g. A wide dFd_F range from 0.25 to 1 corresponds to a max ΔEg∼10\Delta E_g \sim10 eV, and the correspondence between dFd_F and max ΔEg\Delta E_g is extremely noisy at low values. In contrast, the 2-fi model exhibits an almost linear correspondence between dFd_F and max ΔEg\Delta E_g across the entire range of band gaps. Our conclusion is therefore that the inclusion of a large quantity of low-fidelity PBE data greatly assists in the learning of better latent structural features, which leads to substantially improved high-fidelity predictions. It should be noted, however, that a prerequisite for such improvements is that the low-fidelity data is of sufficient size and quality. For example, the 2-fi models without PBE perform worse than the 2-fi models with PBE (Supplementary Dataset 1).

Band Gaps of Disordered Materials

The multi-fidelity graph network architecture also provides a natural framework to address another major gap in the computational materials property predictions - disordered crystals. The majority of known crystals exhibit site disorder. For example, of the ∼200,000\sim200{,}000 crystals reported in the Inorganic Crystal Structure Database (ICSD), 36 more than 120,000 are disordered crystals. Typically, the properties of disordered crystals are estimated by sampling low energy structures among a combinatorial enumeration of distinct orderings, usually within a supercell.

In the graph network approach, we can use the learned length-16 elemental embeddings WZ\mathbf{W}_Z directly as the node features. In such a scheme, disordered sites can be represented as a linear combination of the elemental embeddings as Wdisordered=∑i=1xiWZi\mathbf{W}_{disordered} = \sum_{i=1} x_i \mathbf{W}_{Z_i}, where xix_i is

the occupancy of species ii in the site and WZi\mathbf{W}_{Z_i} is the element embedding for atomic number ZiZ_i. Using the 4-fi model for the ordered crystals without further retraining, the MAE of the predicted band gaps of the 278 disordered crystals in our experimental dataset is a respectable 0.63±\pm0.14 eV, similar to the MAE of the 1-fi-stacked model on the experimental band gaps of ordered crystals. By retraining with the disordered experimental band gap dataset, the average MAE on the experimental band gaps of disordered crystals decreases to 0.51±\pm0.11 eV, while that of the ordered crystals remains approximately the same (0.37±\pm0.02 eV). The average MAEs for a retrained 1-fi model on the experimental band gaps of disordered and ordered crystals are 0.55±\pm0.13 eV and 0.50±\pm0.07 eV, respectively. Clearly, the multi-fidelity approach improves on the performance on disordered crystals as well as ordered crystals.

To demonstrate the power of the disordered multi-fidelity graph network models, band gap engineering data was extracted from the literature for AlxGa1−xN\mathrm{Al}_x\mathrm{Ga}_{1-x}\mathrm{N}37, ZnxCd1−xSe\mathrm{Zn}_x\mathrm{Cd}_{1-x}\mathrm{Se}38, MgxZn1−xO\mathrm{Mg}_x\mathrm{Zn}_{1-x}\mathrm{O}39, and Lu3(GaxAl1−x)5O12\mathrm{Lu}_3(\mathrm{Ga}_x\mathrm{Al}_{1-x})_5\mathrm{O}_{12}40. The band gaps of Lu3(GaxAl1−x)5O12\mathrm{Lu}_3(\mathrm{Ga}_x\mathrm{Al}_{1-x})_5\mathrm{O}_{12} were not present and only the band gaps of the stoichiometric endpoints for the other systems were present in our experimental dataset. The 4-fi model performs remarkably well, reproducing qualitative trends in all instances and achieving near quantitative accuracy for most systems, as shown in Figure 4. The 4-fi model reproduces the concave relationship between xx and the change in band gap ΔEg\Delta E_g for Lu3(GaxAl1−x)5O12\mathrm{Lu}_3(\mathrm{Ga}_x\mathrm{Al}_{1-x})_5\mathrm{O}_{12} (Figure 4d) reported by Fasoli et al.40 For Zn1−xMgxO\mathrm{Zn}_{1-x}\mathrm{Mg}_x\mathrm{O}, a more pronounced concave relationship is predicted by the 4-fi model compared to the experimental measurements.39 The band gap of ZnO is notoriously poorly estimated by DFT techniques,41 and even experimental measurements range from 3.1 to 3.4 eV across publications.42 An additional proof of concept for BaySr1−yCoxFe1−xO3−δ\mathrm{Ba}_y\mathrm{Sr}_{1-y}\mathrm{Co}_x\mathrm{Fe}_{1-x}\mathrm{O}_{3-\delta} perovskite,43 a highly promising catalyst for the oxygen reduction reaction that exhibits disorder on multiple sites, is given in Extended Data Figure 3.

Performance of disordered multi-fidelity graph network models

Figure 4. Performance of disordered multi-fidelity graph network models. Predicted and experimental band gaps (EgE_g) as a function of composition variable xx in a, AlxGa1−xN\mathrm{Al}_x\mathrm{Ga}_{1-x}\mathrm{N}, b, ZnxCd1−xSe\mathrm{Zn}_x\mathrm{Cd}_{1-x}\mathrm{Se}, and c, MgxZn1−xO\mathrm{Mg}_x\mathrm{Zn}_{1-x}\mathrm{O}. d, Comparison of the change in band gap with respect to Lu3Al5O12\mathrm{Lu}_3\mathrm{Al}_5\mathrm{O}_{12} (ΔEg\Delta E_g) with xx in Lu3(GaxAl1−x)5O12\mathrm{Lu}_3(\mathrm{Ga}_x\mathrm{Al}_{1-x})_5\mathrm{O}_{12}. The error bars indicate one standard deviation.

Discussion

Data quality and quantity constraints are major bottlenecks to materials design. Multifidelity graph networks enable the efficient learning of latent structural features using large quantities of cheap low-fidelity computed data to achieve vastly improved predictions for more costly computational methods and experiments. While crystal band gaps have been selected as the model problem in this work, the multi-fidelity graph network framework is universal and readily applicable to other properties and to molecules. Two examples are provided in Extended Data Figure 4, where a large number of low-fidelity molecule calculations are shown to lead to vast improvements in the high-fidelity energy predictions.

The ability to predict band gaps of disordered materials suggests that the learned elemental embeddings in graph network models encode chemistry in a way that naturally interpolates between elements. In a traditional ML model development, each element is represented by a vector of atomic features, e.g., atomic number, electronegativity, atomic radius, etc. A disordered site, e.g., Al-Ga in AlxGa1−xN\mathrm{Al}_{x}\mathrm{Ga}_{1-x}\mathrm{N}, cannot be naturally represented as a linear combination of such feature vectors. Therefore, this is a unique attribute of our graph network models that is not present in feature-engineered ML models. The interpolation approach bears some similarity to that used in the virtual crystal approximation (VCA).44 A limitation of the VCA is that it may fail in systems that do not exhibit full disorder. In the case of multi-fidelity graph networks, this limitation can be mitigated to a certain extent by applying the highly efficient models to the interpolated disordered crystal as well as ordered crystals at the same composition to arrive at a range of property predictions. Thus, multi-fidelity graph network models provide an alternative approach to in silico materials design for the large class of disordered materials that is extremely difficult to treat with existing ab initio computations or ML techniques.

One potential limitation of the proposed approach is the reliance on large low-fidelity datasets for learning effective latent structural representations. The low-fi dataset needs to reproduce at least general qualitative trends in the target property between different materials for such learning to be effective. For some properties, even low-fi datasets may not be available in sufficiently large quantities for effective learning. Under such instances, a transfer learning approach, i.e., where portions of a model trained on a different property is retrained on the target property, may be more appropriate.

Acknowledgement

This work was primarily supported by the Materials Project, funded by the U.S. Department of Energy, Office of Science, Office of Basic Energy Sciences, Materials Sciences and Engineering Division under contract no. DE-AC02-05-CH11231: Materials Project program KC23MP. The authors also acknowledge support from the National Science Foundation SI2-SSI Program under Award No. 1550423 for the software development portions of the work. C.C. thanks Dr. M. Horton for his assistance with the GLLB-SC data set.

Author contributions

C.C. and S.P.O. conceived the idea and designed the work. C.C. implemented the models and performed the analysis. S.P.O. supervised the project. Y.Z., W.Y. and X.L. helped with the data collection and analysis. C.C. and S.P.O. wrote the manuscript. All authors contributed to the discussion and revision.

Ethics Declaration

Competing Interests

The authors declare that they have no competing financial interests.

Correspondence Correspondence and requests for materials should be addressed to S.P.O. (email: [email protected]).

Figure Legends

Image not extracted

Figure 1 (PDF p. 21). Multi-fidelity materials graph networks. a, Representation of a material in a graph network model, with atoms as the nodes, bonds as the edges coupled with a structure-independent global state. The input atomic feature is embedded atomic number of the element. The bond feature vector is the Gaussian expanded distance. The fidelity of each data is encoded as an integer (e.g., 0 for PBE, 1 for GLLB-SC, 2 for HSE, 3 for SCAN and 4 for experiment or Exp). b, A materials graph network model is constructed by stacking graph convolution layers. In each graph convolution layer, sequential updates of atomic, bond and state features are performed using information from connected neighbors in the graph. The output graph in the last layer is then readout and processed in a neural network to arrive at the final prediction.

Image not extracted

Figure 2 (PDF p. 21). Test mean absolute errors (MAEs) of multi-fidelity graph network model predictions on ordered crystal band gaps. a, Performance of graph network models with different fidelity combinations. The 4-fi models used the PBE, GLLB-SC, HSE and Exp datasets, i.e., the very small SCAN dataset is excluded. All

errors were obtained on the corresponding test sets of the fidelity. The error bars show one standard deviation and the dots are the individual model errors. Average MAEs of b, GLLB-SC, c, SCAN, d, HSE, and e, experimental band gaps of 2-fi models trained using sampled datasets for each high-fidelity data and PBE data. For each plot, the error line is lowered with more high-fidelity data. The x-axis is plotted on a log scale and the shaded areas indicate one standard deviation of the MAE. ss indicates the slope for a linear fit of MAE to log⁡10NPBE\log_{10} N_{\mathrm{PBE}}.

Figure 3. Effect of including low-fidelity PBE data on latent structural features. Two dimensional t-distributed Stochastic Neighbor Embedding (complexity = 1000) projection of features for a, 1-fi and b, 2-fi models trained using 100 experimental data points and the entire PBE dataset. The markers are colored according to the experimental band gap. Plots of the experimental band gap difference (ΔEg\Delta E_g) against normalized latent structural feature distance (dFd_F) in arbitrary units (a.u.) for the c, 1-fi and d, 2-fi-PBE models trained on all available experimental data. The dashed lines indicate the envelope of the maximum ΔEg\Delta E_g at each dFd_F.

Figure 4. Performance of disordered multi-fidelity graph network models. Predicted and experimental band gaps (EgE_g) as a function of composition variable xx in a, AlxGa1−xN\mathrm{Al}_x\mathrm{Ga}_{1-x}\mathrm{N}, b, ZnxCd1−xSe\mathrm{Zn}_x\mathrm{Cd}_{1-x}\mathrm{Se}, and c, MgxZn1−xO\mathrm{Mg}_x\mathrm{Zn}_{1-x}\mathrm{O}. d, Comparison of the change in band gap with respect to Lu3Al5O12\mathrm{Lu}_3\mathrm{Al}_5\mathrm{O}_{12} (ΔEg\Delta E_g) with xx in Lu3(GaxAl1−x)5O12\mathrm{Lu}_3(\mathrm{Ga}_x\mathrm{Al}_{1-x})_5\mathrm{O}_{12}. The error bars indicate one standard deviation.

Methods

Data Collection and Processing

The PBE10 dataset comprising 52,348 crystal structures with band gaps were obtained from Materials Project11 on Jun 1 2019 using the Materials Application Programming Interface in the Python Materials Genomics (pymatgen) library. 45,46 The GLLB-SC band gaps from Castelli et al. 27 were obtained via MPContribs. 47 The total number of GLLB-SC band gaps is 2,290 after filtering out materials that do not have structures in the current Materials Project database and those that failed the graph computations due to abnormally long bond (>5 A˚>5\,\text{\AA}). The GLLB-SC data all have positive band gaps due to the constraints applied in the structure selection in the previous work. 27 The SCAN28 band gaps for 472 nonmagnetic materials were obtained from Borlido et al. 29 The HSE13 band gaps with corresponding Materials Project structures were downloaded from the MaterialGo website. 30 After filtering out ill-converged calculations and those that have a much smaller HSE band gap compared to the PBE band gaps, 6,030 data points remain, of which 2,775 are metallic. Finally, the experimental band gaps were obtained from the work by Zhuo et al. 31 As this data set only contains compositions, the experimental crystal structure for each composition was obtained by looking up the lowest energy polymorph for a given formula in the Materials Project, followed by cross-referencing with the corresponding Inorganic Crystal Structure Database (ICSD) entry. 36 Further, as multiple band gap can be reported for the same composition in this data set, the band gaps for the duplicated entries were averaged. In total, 2,703 ordered (938 binary, 1306 ternary and 459 quaternary) and 278 disordered (41 binary, 132 ternary and 105 quaternary) structure-band gap pairs were obtained. All data sets are publicly available. 48

Materials Graph Networks Construction

In materials graph networks, atoms and bonds are represented as nodes and edges in an undirected graph as (V,E,u)(V,E,u). The atom attributes VV are the atomic numbers Z∈NZ \in\mathbb{N}. Each atom attribute is associated with a row vector WZi∈R16\mathbf{W}_{Z_i} \in\mathbb{R}^{16} in the element embedding matrix WZ=[W0;W1;W2;…;W94]\mathbf{W}_Z = [\mathbf{W}_0;\mathbf{W}_1;\mathbf{W}_2;\ldots;\mathbf{W}_{94}] where W0\mathbf{W}_0 is a dummy vector. The bond attribute is the set of Gaussian-expanded distances. For the kk-th bond in the structure, the attributes are

ek,m=exp⁡(−(dk−μm)2σ2), ∀dk≤Rce_{k,m} = \exp\left(-\frac{(d_k-\mu_m)^2}{\sigma^2}\right),\ \forall d_k \le R_c

where dkd_k is the length of the bond kk formed by atom indices rkr_k and sks_k, RcR_c is the cutoff radius and

μm=mnbf−1μmax⁡\mu_m = \frac{m}{n_{bf}-1}\mu_{\max}

for m={0,1,2,…,nbf−1}m = \{0,1,2,\ldots,n_{bf}-1\} and nbfn_{bf} is the number of bond features. In this work, Rc=5R_c = 5Å, μmax⁡=6\mu_{\max} = 6 Å, and nbf=100n_{bf} = 100. The graphs are constructed using an edge list representation, and the edge set of the graph is represented as E={(ek,rk,sk)}E = \{(\mathbf{e}_k,r_k,s_k)\}. The state attributes u\mathbf{u} are fidelity levels F∈NF \in\mathbb{N}. Similar to atom attributes, fidelity FiF_i is associated with a row vector WFif\mathbf{W}^{f}_{F_i} in the fidelity embedding matrix WF=[W0f;W1f;W2f;W3f;W4f]\mathbf{W}_F = [\mathbf{W}^{f}_0;\mathbf{W}^{f}_1;\mathbf{W}^{f}_2;\mathbf{W}^{f}_3;\mathbf{W}^{f}_4].

Both embedding matrices WZ\mathbf{W}_Z and WF\mathbf{W}_F are trainable in the models, except in disordered models where the elemental embedding matrix WZ\mathbf{W}_Z is fixed to previously obtained values.

In each graph convolution layer, the graph networks are propagated sequentially as follows:

  1. The attributes of each bond kk in the graph are updated as

ek′=ϕe(vsk⊕vrk⊕ek⊕u)\mathbf{e}'_k = \phi_e(\mathbf{v}_{s_k} \oplus\mathbf{v}_{r_k} \oplus\mathbf{e}_k \oplus\mathbf{u})

where ϕe\phi_e is the bond update function, vsk\mathbf{v}_{s_k} and vrk\mathbf{v}_{r_k} are the atomic attributes of the two atoms forming the bond kk, and ⊕\oplus is the concatenation function.

  1. Each atom ii is then updated as

vi′=ϕv(vˉie⊕vi⊕u)\mathbf{v}'_i = \phi_v(\bar{\mathbf{v}}^e_i \oplus\mathbf{v}_i \oplus\mathbf{u})

where ϕv\phi_v is the atomic update function, and vˉie=average⁡k(ek′)\bar{\mathbf{v}}^e_i = \operatorname{average}_k(\mathbf{e}'_k), ∀rk=i\forall r_k = i is the averaged bond attributes from all bonds connected to atom ii.

  1. Finally, the state attributes are updated as

u′=ϕu(u‾e⊕u‾v⊕u)\mathbf{u}' = \phi_u(\overline{\mathbf{u}}^{e} \oplus\overline{\mathbf{u}}^{v} \oplus\mathbf{u})

where ϕu\phi_u is the state update function, and u‾e=average⁡k(ek′)\overline{\mathbf{u}}^{e} = \operatorname{average}_{k}(e'_k) and u‾v=average⁡i(vi′)\overline{\mathbf{u}}^{v} = \operatorname{average}_{i}(v'_i) are the averaged attributes from all atoms and bonds, respectively.

Experimental Setup

The models were constructed using tensorflow. 49 Three graph convolution layers and the Set2Set readout function with two steps are used in the model training. 5 The update functions in each graph convolutional layer were chosen to be multi-layer perceptron models with [64, 64, 32] hidden neurons and shifted softplus function ln⁡(ex+1)−ln⁡2\ln(e^x + 1) - \ln2 as the non-linear activation function.

We split the data into 80%-10%-10% train-validation-test ratios randomly for each fidelity independently and repeated the splitting six times. It should be noted that each structure-band gap data point is considered as a unique data point, and there are instances where the same structure with different fidelity band gaps are present in the training and validation/test data. An alternative data splitting strategy wherein structural overlaps in the training/validation/test sets are disallowed is presented in Supplementary Dataset 3. It was found that such a data splitting strategy results in significantly higher model errors, which indicates that information from multi-fidelities is necessary for the models to learn the relationships between different fidelities.

A learning rate of 10−310^{-3} was used with the Adam optimizer with mean squared error as the loss function. The mean absolute error (MAE) was used as the error metric on the validation and test datasets, and the batch size for the training was set to 128. All models were trained on the corresponding training data for a maximum of 1500 epochs. During the training process, the MAE error metrics were calculated on the validation data set after each epoch.

The model weights were saved after each epoch if the validation MAE reduced. Fitting was deemed to have converged if the validation MAE did not reduce for a consecutive 500 steps. An automatic training recovering mechanism was also implemented by reloading the saved model weights and reduce the learning rate by half when gradient explosion happens. For multi-fidelity model fitting, only the high-fidelity datasets without the PBE data were used in the validation set. The final model performances were evaluated on the test datasets and reported in this work.

Data Availability

Multi-fidelity band gap data and molecular data in the current study are available at https://doi.org/10.6084/m9.figshare.13040330. The data for all figures and extended data figures are available in Source Data.

Code Availability

Model fitting and results plotting codes are available at https://github.com/materialsvirtuallab/megnet/tree/master/multifidelity. Materials graph network is available at https://github.com/materialsvirtuallab/megnet. The specific version of the package can be found at https://doi.org/10.5281/zenodo.407202950.

Methods-Only References

(45) Ong, S. P.; Richards, W. D.; Jain, A.; Hautier, G.; Kocher, M.; Cholia, S.; Gunter, D.; Chevrier, V. L.; Persson, K. A.; Ceder, G. Python Materials Genomics (Pymatgen): A Robust, Open-Source Python Library for Materials Analysis. Computational Materials Science 2013, 68, 314–319.

Supporting Information: Learning Properties of Ordered and Disordered Materials from Multi-fidelity Data

Chi Chen; Yunxing Zuo; Weike Ye; Xiangguo Li; Shyue Ping Ong

Comparison with alternative models

Supplementary Table 1: Average mean absolute errors (MAEs) of 2-fi and 4-fi graph network models trained using alternative approaches. The first column shows the data fidelity combinations in training the models, and the other columns are the average MAEs with standard deviations on the corresponding test data fidelity.

PBE (eV)GLLB-SC (eV)SCAN (eV)HSE (eV)Exp (eV)
Non-overlapping-structure split
PBE/GLLB-SC0.30±0.020.30 \pm0.020.57±0.020.57 \pm0.02---
PBE/SCAN0.33±0.040.33 \pm0.04-0.52±0.050.52 \pm0.05--
PBE/HSE0.33±0.030.33 \pm0.03--0.44±0.030.44 \pm0.03-
PBE/Exp0.31±0.060.31 \pm0.06---0.42±0.040.42 \pm0.04
PBE/GLLB-SC/HSE/Exp0.32±0.020.32 \pm0.020.55±0.020.55 \pm0.02-0.40±0.040.40 \pm0.040.40±0.030.40 \pm0.03
Positive PBE band gap
PBE/GLLB-SC0.35±0.020.35 \pm0.020.45±0.030.45 \pm0.03---
PBE/SCAN0.36±0.010.36 \pm0.01-0.44±0.030.44 \pm0.03--
PBE/HSE0.34±0.020.34 \pm0.02--0.42±0.040.42 \pm0.04-
PBE/Exp0.37±0.020.37 \pm0.02---0.44±0.030.44 \pm0.03
Inverse-data-size-weighted
PBE/GLLB-SC0.49±0.100.49 \pm0.100.61±0.060.61 \pm0.06---
PBE/SCAN0.78±0.320.78 \pm0.32-0.65±0.150.65 \pm0.15--
PBE/HSE0.38±0.070.38 \pm0.07--0.42±0.040.42 \pm0.04-
PBE/Exp0.52±0.030.52 \pm0.03---0.48±0.050.48 \pm0.05
PBE/GLLB-SC/SCAN/HSE/Exp0.38±0.070.38 \pm0.070.53±0.030.53 \pm0.030.43±0.080.43 \pm0.080.44±0.060.44 \pm0.060.42±0.050.42 \pm0.05

Table 1.

Non-overlapping-structure split

The models reported in main text were trained based on a random data split procedure that treated each structure and specific fidelity band gap data point as a unique data point. We also developed 2-fi and 4-fi models where the data split is performed in such a way that a structure can only be present once regardless of fidelity in either the training, validation or test data, shown in Table 1. In all cases, we observe a substantial increase in model errors under the non-overlapping-structure data split method, which indicates that having multifidelity property data for the same structure is critical for the model to learn the relationships between different fidelities.

Positive PBE band gap

We also explored 2-fi models in which structures with zero PBE band gaps were excluded. Compared to the original 2-fi models, these 2-fi models show reduced model errors for the GLLB-SC and SCAN fidelities, but increased model errors for the HSE and Exp fidelity (Table 1). The results are consistent with the data distributions where the GLLB-SC and SCAN data are all or largely non-metals, where HSE and Exp data have considerable fraction of metallic systems.

Inverse-data-size-weighted

Finally, we explored the application of data weights to bias the model performance towards higher accuracy on higher fidelities. Both 2-fi and 5-fi models were developed where the weight for each data point is set as the reciprocal of the corresponding fidelity data size, i.e., the smaller size high-fidelity data sets were given much larger weights than the large PBE data. Somewhat surprisingly, these inverse-data-sized-weighted models exhibit much higher average MAEs (Table 1) relative to corresponding uniformly weighted models. These results suggest that having high data weights on the low-fidelity is necessary to achieve good crystal representation in the deep learning models, which likely help the models on the high-fidelity tasks.

The poorer performance of the inverse-data-weighted models vis-a-vis the uniformly-weighted models is due to a decrease in the quality of the latent representations learned in these models. The PBE data set is the largest and covers a diverse range of structures and chemistries, while the other data sets are much smaller and less diverse. By decreasing the PBE weight relative to the higher fidelities, we decrease the model’s ability to effectively extract effective latent features, which outweighs any potential gain in model performance by biasing the model for higher fidelities. This can be effectively demonstrated by plotting the latent feature distances between materials against the experimental band gap differences for all materials in the experimental structures for 2-fi PBE/Exp models trained on uniform weights and inverse-data-size weights (Fig. 1). The models with uniform data weights tend to place the materials better in the latent space, as shown by the roughly linear relationship between the maximum band gap differences and the feature distances, while for the inverse-size weighted models the materials that have the feature distances dFd_F from 0.5 to 1.0 are almost indistinguishable in terms of their band gap differences δEg\delta E_g.

Material latent feature distances against the predicted experimental band gap differences for PBE/Exp 2-fi models

Figure 1. Material latent feature distances against the predicted experimental band gap differences for PBE/Exp 2-fi models. Models are trained using (a) uniform data weights and (b) data weights proportional to the reciprocal of fidelity data volume.

Transfer learning

We have implemented transfer learning (TL) models for all higher fidelity datasets by taking the low-fi PBE models, fixing the graph convolution layers and retraining the final layers with the higher fidelity data. The TL model MAEs are shown in Table 2. Compared with the 2-fi models, the TL models do achieve somewhat lower MAEs on the GLLB-SC and SCAN datasets, but the errors on HSE and experimental datasets are higher. In particular, the MAE on the experimental dataset for the TL models is 0.47 eV, much higher than the 0.40 eV of the 2-fi PBE/Exp model. Further, the best performing PBE/GLLB-SC/HSE/Expt 4-fi models outperform the TL models even further.

GLLB-SC (eV)SCAN (eV)HSE (eV)Exp (eV)
Transfer learning0.44±0.080.44 \pm0.080.35±0.040.35 \pm0.040.38±0.040.38 \pm0.040.47±0.040.47 \pm0.04
2-fi0.55±0.060.55 \pm0.060.52±0.110.52 \pm0.110.36±0.060.36 \pm0.060.40±0.050.40 \pm0.05
5-fi0.50±0.040.50 \pm0.040.45±0.120.45 \pm0.120.33±0.040.33 \pm0.040.40±0.040.40 \pm0.04

Table S2. Model test MAE comparisons for transfer learning, 2-fi models and 5-fi models. The first column is the model category and the other columns are the average model MAEs with standard deviation on the corresponding test data fidelity.

High-fi sizeLow-fi sizeMAE mean (eV)MAE std (eV)MAE range (eV)
10001.790.29[1.39, 2.29]
10010001.370.14[1.14, 1.57]
10020001.220.05[1.15, 1.29]
10050001.150.12[0.99, 1.31]
100100001.110.10[0.93, 1.25]
100200000.980.14[0.80, 1.16]
100410000.890.08[0.74, 1.00]
50001.130.08[1.02, 1.22]
50010001.040.06[0.94, 1.10]
50020000.990.04[0.93, 1.06]
50050000.930.05[0.87, 1.02]
500100000.840.06[0.75, 0.94]
500200000.750.05[0.67, 0.82]
500410000.720.08[0.60, 0.86]
100000.980.05[0.91, 1.06]
100010000.860.05[0.78, 0.94]
100020000.820.05[0.78, 0.93]
100050000.780.05[0.72, 0.84]
1000100000.730.03[0.67, 0.77]
1000200000.670.03[0.64, 0.73]
1000410000.580.03[0.53, 0.64]
150000.800.05[0.72, 0.88]
150010000.760.04[0.68, 0.82]
150020000.730.03[0.68, 0.78]
150050000.670.04[0.61, 0.73]
1500100000.660.04[0.59, 0.72]
1500200000.620.03[0.58, 0.67]
1500410000.550.04[0.50, 0.60]
206000.680.06[0.60, 0.76]
206010000.690.06[0.61, 0.79]
206020000.700.06[0.59, 0.77]
206050000.640.04[0.60, 0.73]
2060100000.570.04[0.50, 0.62]
2060200000.590.09[0.50, 0.75]
2060410000.550.06[0.48, 0.65]

Table 3. The GLLB-SC test MAEs for two-fidelity PBE/GLLB-SC models with various low-fidelity and high-fidelity data size combinations.

High-fi sizeLow-fi sizeMAE mean (eV)MAE std (eV)MAE range (eV)
10001.150.27[0.81, 1.68]
10010000.860.17[0.56, 1.09]
10020000.740.12[0.57, 0.92]
10050000.750.11[0.64, 0.92]
100100000.730.09[0.57, 0.86]
100200000.720.10[0.55, 0.83]
100410000.680.09[0.59, 0.85]
20000.830.14[0.66, 1.10]
20010000.750.18[0.54, 1.01]
20020000.740.10[0.64, 0.89]
20050000.710.16[0.55, 1.00]
200100000.710.11[0.57, 0.88]
200200000.660.10[0.54, 0.80]
200410000.600.11[0.46, 0.78]
30000.820.19[0.61, 1.19]
30010000.750.11[0.57, 0.92]
30020000.710.14[0.54, 0.90]
30050000.720.13[0.55, 0.94]
300100000.710.11[0.57, 0.87]
300200000.610.13[0.45, 0.82]
300410000.570.14[0.39, 0.75]
40000.760.11[0.61, 0.95]
40010000.720.12[0.54, 0.94]
40020000.670.14[0.50, 0.92]
40050000.650.09[0.51, 0.81]
400100000.650.09[0.54, 0.82]
400200000.630.17[0.49, 0.90]
400410000.530.12[0.34, 0.70]
42400.780.13[0.61, 1.01]
42410000.710.13[0.59, 0.89]
42420000.650.10[0.56, 0.86]
42450000.640.13[0.53, 0.84]
424100000.590.14[0.43, 0.80]
424200000.570.12[0.38, 0.74]
424410000.520.11[0.35, 0.68]

Table 4. The SCAN test MAEs for two-fidelity PBE/SCAN models with various low-fidelity and high-fidelity data size combinations.

High-fi sizeLow-fi sizeMAE mean (eV)MAE std (eV)MAE range (eV)
10001.420.20[1.22, 1.76]
10010000.990.08[0.89, 1.09]
10020000.880.05[0.80, 0.95]
10050000.790.06[0.72, 0.87]
100100000.720.03[0.69, 0.77]
100200000.680.08[0.57, 0.76]
100410000.590.05[0.53, 0.65]
50001.020.07[0.92, 1.12]
50010000.880.06[0.82, 0.97]
50020000.830.05[0.76, 0.90]
50050000.700.05[0.64, 0.80]
500100000.650.06[0.56, 0.71]
500200000.590.05[0.51, 0.64]
500410000.570.07[0.48, 0.65]
100000.920.03[0.88, 0.97]
100010000.790.03[0.75, 0.85]
100020000.780.05[0.70, 0.83]
100050000.670.03[0.64, 0.73]
1000100000.620.05[0.56, 0.71]
1000200000.550.04[0.49, 0.60]
1000410000.470.03[0.45, 0.52]
300000.710.08[0.59, 0.81]
300010000.620.04[0.55, 0.66]
300020000.640.03[0.60, 0.67]
300050000.550.03[0.52, 0.59]
3000100000.530.03[0.47, 0.57]
3000200000.490.04[0.42, 0.56]
3000410000.390.05[0.30, 0.43]
480000.570.03[0.53, 0.61]
480010000.570.05[0.49, 0.64]
480020000.520.04[0.46, 0.55]
480050000.490.01[0.48, 0.51]
4800100000.470.04[0.43, 0.54]
4800200000.440.04[0.40, 0.53]
4800410000.360.06[0.29, 0.46]

Table 5. The HSE test MAEs for two-fidelity PBE/HSE models with various low-fidelity and high-fidelity data size combinations.

High-fi sizeLow-fi sizeMAE mean (eV)MAE std (eV)MAE range (eV)
10000.870.13[0.75, 1.08]
10010000.790.08[0.68, 0.93]
10020000.710.05[0.63, 0.79]
10050000.650.06[0.54, 0.73]
100100000.630.03[0.58, 0.67]
100200000.600.05[0.52, 0.69]
100410000.560.05[0.48, 0.64]
20000.800.08[0.64, 0.88]
20010000.670.05[0.59, 0.76]
20020000.620.05[0.55, 0.71]
20050000.610.05[0.54, 0.70]
200100000.600.04[0.55, 0.68]
200200000.560.05[0.48, 0.61]
200410000.530.02[0.50, 0.55]
50000.710.06[0.59, 0.79]
50010000.640.07[0.52, 0.74]
50020000.600.07[0.49, 0.70]
50050000.570.06[0.50, 0.69]
500100000.540.07[0.44, 0.66]
500200000.520.05[0.44, 0.59]
500410000.510.02[0.47, 0.54]
100000.600.04[0.55, 0.66]
100010000.580.06[0.47, 0.66]
100020000.550.06[0.45, 0.63]
100050000.520.06[0.42, 0.60]
1000100000.520.06[0.44, 0.61]
1000200000.490.05[0.43, 0.57]
1000410000.470.05[0.39, 0.51]
243000.500.06[0.42, 0.60]
243010000.490.05[0.41, 0.56]
243020000.450.05[0.38, 0.52]
243050000.440.03[0.40, 0.48]
2430100000.430.05[0.36, 0.51]
2430200000.450.06[0.37, 0.54]
2430410000.400.05[0.33, 0.48]

Table 6. The Exp test MAEs for two-fidelity PBE/Exp models with various low-fidelity and high-fidelity data size combinations.

Multi-fidelity models for energies of molecules

To establish the universal applicability of the multi-fidelity graph network framework to molecules and other properties beyond band gaps, models were developed using computed molecule energies from the QM9[1] dataset and the QM7b dataset.[2] The procedures for training (data split, hyperparameter optimization, etc.) are similar to those for the crystal band gap models, as outlined in the Methods section of the main manuscript.

The QM9 dataset contains B3LYP/6-31G(2df,p) calculations of 130,462 small molecules consisting of C, H, O, N, F elements. For a subset of 6,095 C7H10O2C_{7}H_{10}O_{2}, the energies, enthalpies and free energies are also calculated using more accurate/expensive G4MP2 level of theory. Extended Data Fig. 4a shows the effect adding the low-fidelity B3LYP data on the prediction average mean absolute error (MAE) of high-fidelity G4MP2 energies. With the addition of the B3LYP data, the 2-fi models reaches chemical accuracy of 1 kcal/mol on the average MAE of the G4MP2 energy predictions with only ∼300 G4MP2 data points. Without B3LYP data, i.e., a 1-fi model, chemical accuracy can only be achieved with ∼ 4000 G4MP2 data points. In other words, the 2-fi models converges to within chemical accuracy with an order of magnitude fewer computationally costly G4MP2 data points.

The QM7b dataset contains HF, MP2, and CCSD(T) energy calculations with the ccpvdz basis set of 7,211 molecules of C, H, O, N, F, Cl, S elements.[2] Extended Data Fig. 4b shows the effect of adding low-fidelity HF and MP2 data on average MAEs of the the multi-fidelity models on the CCSD(T) energy predictions. In all cases, the multi-fidelity models significantly outperform the 1-fi CCSD(T) models, achieving lower MAEs on the high-fidelity CCSD(T) energies with much fewer CCSD(T) data points. A larger data size ratio ss (greater quantity of low-fidelity HF and MP2 calculations) leads to a more rapid decrease in MAE with the addition of CCSD(T) data. Most interestingly, the inclusion of a large quantity of computationally cheap HF data in a 2-fi HF/CCSD(T) model (gray lines in Extended Data Fig. 4b) achieves essentially the same improvements in MAE as a 3-fi HF/MP2/CCSD(T) model (blue lines in Extended Data Fig. 4b) that requires more computationally expensive MP2 calculations. Close to chemical accuracy of 1 kcal/mol can be achieved on the average MAE with about O(102)O(10^{2}) costly CCSD(T) datapoints when 16×16 \times the quantity of HF data is included in a 2-fi model.

References

  1. [1]Ramakrishnan, R.; Dral, P. O.; Rupp, M.; von Lilienfeld, O. A. Quantum Chemistry Structures and Properties of 134 Kilo Molecules. Scientific Data 2014, 1, 1–7.
  2. [2]Zaspel, P.; Huang, B.; Harbrecht, H.; von Lilienfeld, O. A. Boosting Quantum Machine Learning Models with a Multilevel Combination Technique: Pople Diagrams Revisited. Journal of Chemical Theory and Computation 2019, 15, 1546–1559.DOI

Paper details

vlnc:W_v7p4fdqa
Version
v1 · latest
Published
Fresh — added to the Hub within the last 30 days
Added
License
CC BY-SA 4.0
Usage

views · 10

downloads · 0

A view counts once per browser per day, a download each time the PDF is requested.

  • Physics and Astronomy

Not recorded for this paper

View statement
Looking for related papers…
Contents