betaFeedback

WorksPhysics and Astronomy

How to verify the precision of density-functional-theory implementations via reproducible and universal workflows

Abstract

In the past decades many density-functional theory methods and codes adopting periodic boundary conditions have been developed and are now extensively used in condensed matter physics and materials science research. Only in 2016, however, their precision (i.e., to which extent properties computed with different codes agree among each other) was systematically assessed on elemental crystals: a first crucial step to evaluate the reliability of such computations. We discuss here general recommendations for verification studies aiming at further testing precision and transferability of density-functional-theory computational approaches and codes. We illustrate such recommendations using a greatly expanded protocol covering the whole periodic table from Z=1 to 96 and characterizing 10 prototypical cubic compounds for each element: 4 unaries and 6 oxides, spanning a wide range of coordination numbers and oxidation states. The primary outcome is a reference dataset of 960 equations of state cross-checked between two all-electron codes, then used to verify and improve nine pseudopotential-based approaches. Such effort is facilitated by deploying AiiDA common workflows that perform automatic input parameter selection, provide identical input/output interfaces across codes, and ensure full reproducibility. Finally, we discuss the extent to which the current results for total energies can be reused for different goals (e.g., obtaining formation energies).

Key points:

  • Verification efforts are critical to assess the reliability of density-functional theory (DFT) simulations and provide results with properly quantified uncertainties.

  • Developing standard computation protocols to perform verification studies and publishing curated and FAIR reference datasets can significantly facilitate their use to improve codes and computational approaches.

  • The use of fully automated workflows with common interfaces between codes can guarantee uniformity, transferability, and reproducibility of results.

  • A careful description of the numerical and methodological details needed to compare with the reference datasets is essential; we discuss and illustrate this point with a dataset of 960 all-electron equations of state.

  • Reference datasets should always include an explanation of the target property for which they were generated, and a discussion of their limits of applicability.

  • Further extensions of DFT verification efforts are needed to cover more functionals, more computational approaches, and the treatment of magnetic and relativistic (spin-orbit) effects. They should also aim at concurrently delivering optimized protocols that, not only target ultimate precision, but also optimize the computational cost for a target accuracy.

Website summary: Verification efforts of DFT calculations are of crucial importance to evaluate the reliability of simulation results. We discuss general recommendations for performing such studies and illustrate them with an all-electron reference dataset of 960 equations of state covering the whole periodic table (hydrogen to curium). The importance of verification for the improvement of pseudopotential codes is also demonstrated.

The fast improvement of hardware, methods, and tools for density-functional theory (DFT) calculations in periodic boundary conditions has greatly advanced the field of condensed matter physics and computational materials science, paving the way for an effective use of the “materials design process” that accelerates the discovery, development and deployment of new materials thanks to the aid of simulations[1, 2]. Efficient software infrastructures[3, 4, 5, 6, 7, 8, 9, 10, 11, 12, 13] facilitate, nowadays, large high-throughput calculations of a panoply of material properties which are often made available to the public in large repositories[14, 15, 16, 17, 18, 19, 20, 21]. Most datasets aspire to be findable, accessible, interoperable, and reusable (FAIR)[22] in order to accelerate materials discovery, possibly with the aid of machine learning. They are queryable with ad hoc application programming interfaces (APIs) or, for many of them, via a single common API thanks to the recent efforts of the OPTIMADE[23] consortium. However, full integration of different data is often limited by considerations related to uncertainty quantification[1, 24, 25, 26, 27]. In this work, we discuss recommendations on how to quantify to which extent properties (total energies and derived quantities) obtained by different DFT codes agree among each other.

In principle, DFT applies the fundamental laws of quantum physics to predict properties of a material, with no other inputs than the chemical composition and the crystal structure. In reality, the electronic-structure calculations involve a variety of choices to solve the equations prescribed by DFT and introduce several levels of approximation. Those choices, reflected in the resulting data, range from the specific flavor of DFT (e.g., the approach used for the exchange-correlation functional) to the discretization assumptions (e.g., the basis set), to the specific computational parameters needed by the codes. Some approaches are more reliable, and therefore often slower, while others make more substantial approximations in order to gain computational speed and enable the study of systems with more atoms. Furthermore, even when formally the same choices have been made in different codes, these may provide slightly different results due to the details of their implementations. The importance of verifying the precision of codes has been long recognized[28]. Despite this, when considering DFT codes adopting periodic boundary conditions, a first systematic assessment of their precision was performed in 2016, where the consistency of 40 computational approaches was assessed by calculating the equation of state (EOS) (i.e., the energy-versus-volume curve) for a test set of 71 elemental crystals[29, 30]. This so-called “Δ\Delta-project” led to the conclusion that the mainstream codes were in very good agreement with each other, which was not the case a decade before. Despite being already a large project by itself, the “Δ\Delta-project” was only the first step towards a careful verification of DFT calculations, which requires a much larger diversity of structural and chemical variables, as also discussed in the outlook of Ref. 29.

In this Expert Recommendation, we list a set of guiding principles to perform new verification studies of DFT calculations (see Box 1), as well as a recommendation (see Box 2) for users of DFT codes, encouraging them to refer to quantitative sources on the reliability and precision of the codes and computational approaches used in their publications. In order to illustrate these recommendations, we create a curated reference set of highly converged results for the EOS of 960 crystals, using two independent state-of-the-art all-electron (AE) DFT codes (FLEUR[31, 32] and WIEN2k[33, 34]). These 960 crystals cover all elements and a wide variety of structural and chemical environments in the form of four unary compounds and six oxides. The resulting data are shared on the Materials Cloud[19] according to the FAIR[22] principles. A key feature of our work is that the thousands of computations performed are implemented within a reproducible and automatic infrastructure. Specifically, the launching and management of all the DFT calculations is carried out using AiiDA[3, 4, 35]. The choice of code-specific inputs and numerical parameters (called “protocols” in the following) are implemented in the publicly available aiida-common-workflows (ACWF) package[36, 37] together with a number of error handlers to recover automatically from typical failure modes of each code. This setup enables to easily generate new datasets and to extend the current work for the verification of other computational approaches (see also Box 4).

As we discuss later, some choices of numerical parameters (such as the smearing type and size, or the k-point integration mesh) must be performed consistently in order to make correct use of the dataset. The suggestions regarding how to use our reference dataset are summarized in Box 3. One of our recommendations for verification efforts is to develop metrics to quantify discrepancies between codes that depend on physically measurable quantities. We implement this recommendation by defining two new metrics (in addition to the Δ\Delta metric introduced in Ref. 38) to facilitate quantitative comparison of EOS results for pairs of codes or computational approaches, and we discuss their benefits.

Using these metrics, we then compare the EOS results of our reference dataset to the results obtained by a number of pseudopotential codes. The latter are designed to enhance computational efficiency by considering explicitly only “valence” electrons, which contribute to bonding[39, 40, 41]. The codes considered here are: ABINIT[13, 42, 43], BigDFT[44], CASTEP[45], CP2K[46, 47], GPAW[48, 49], Quantum ESPRESSO[50, 51], Siesta[52, 53], the SIRIUS[54, 55] library (via its CP2K interface) and VASP[55, 56]. The numerical basis sets implemented in these codes include plane waves, Gaussians combined with plane waves, Daubechies wavelets, and atomic orbitals. For this reason, we do not label our results simply with the code name, but with a short string also indicating a few additional relevant parameters to better specify the details of the computational approach. We stress that the aim of this study is not to provide a ranking or to evaluate the quality of different codes, but to illustrate with a few examples the value of curated datasets generated following our recommendations. In particular, we illustrate its use to improve existing pseudopotentials and to assess the consistency of results of several computational approaches to compute the EOS within DFT.

Expert Recommendation Box 1, Summary of recommendations to perform verification studies of DFT calculations

  • Quantitatively estimate the precision of DFT computational approaches and implementations with respect to exact numerical results. Provide adequate details of the verification protocols to ensure reproducibility of the results and a correct reuse in data-driven research, e.g., clarifying their range of applicability and specifying which parameters need to be fixed — independent of the approach — to ensure comparable results.

  • Develop fully automated workflows to guarantee uniformity and transferability of parameters between computational approaches. This includes the definition and use of “standard protocols”, i.e., automated selection of numerical parameters — often specific to each computational approach — that can ensure numerically precise results.

  • Publish curated reference datasets from systematic verification studies. Facilitate their use to improve other codes by making the datasets FAIR: findable and accessible on open repositories, interoperable by using standard formats and clear annotations, and reusable by specifying all parameters needed to reproduce the results. See Box 3 for an example.

  • Organize the reference data in appropriate subsets by recognizing the diversity of focus and the non-uniform capabilities of available computational approaches (e.g., if some systems require additional effort to be supported by all codes).

Expert Recommendation Box 2, Summary of recommendations for users of DFT codes

  • When publishing research that makes use of DFT codes, refer as much as possible to quantitative sources that document the precision of the numerical implementation (all-electron vs. pseudopotential, basis-set type and size, . . .).

  • Equally important is a validation statement that refers to the accuracy of the chosen exchange–correlation functional to correctly and accurately address the physics at hand. Note that, however, this is beyond the scope of the current Expert Recommendation focusing on the precision of numerical implementations.

  • Always cite the exact pseudopotentials that are used in published simulations, including the exchange–correlation functional, the library from which they were obtained and the exact library version, together with all the essential numerical parameters of the calculations (e.g., k-point integration mesh and smearing, basis set type and size or plane-wave cutoffs, . . .). Lack of this information results in essentially non-reproducible simulations.

Finally, in our Outlook, we discuss a set of recommendations (summarized in Box 4) on future extensions of verification efforts. On the one hand, we suggest to cover more exchange–correlation functionals, computational approaches, and treatment of magnetic and relativistic (spin-orbit) effects. On the other hand, we highlight how future studies should not only target ultimate precision, but also aim at delivering protocols that optimize the computational cost for a target accuracy. We stress that, in this Expert Recommendation, we limit all discussions to verification efforts: i.e., investigating code precision, that is, how codes reproduce the ideal theoretical results given by DFT (e.g., with a given choice of exchange–correlation functional). We do not discuss validation, i.e., accuracy with respect to the experimental results. While this is also an highly relevant topic (and we briefly mention it in Box 2), it is beyond the scope of this Expert Recommendation.

AE reference dataset for EOS parameters

In this section we discuss our reference dataset of EOS calculations, that we use to illustrate, with a practical example, how to implement the recommendations of Box 1. The results are obtained with the AE codes FLEUR and WIEN2k, using the PBE[57] exchange–correlation functional. The two codes use the linearized augmented plane waves plus local orbitals method, but differ in details of the basis set and some computational setup parameters.

Crystal-structures dataset

We compute the EOS on a dataset of 960 cubic crystal structures. In order to provide a chemically comprehensive dataset, we consider all elements in the periodic table from Z=1Z = 1 (hydrogen) to Z=96Z = 96 (curium). Furthermore, we systematically scan structural diversity and investigate the transferability to more complex chemical environments by examining, for each element, 4 mono-elemental cubic crystals (“unaries dataset”) and six cubic oxides (“oxides dataset”).

Specifically, the unaries dataset considers all elements in the face-centered cubic (FCC), body-centered cubic (BCC), simple cubic (SC) and diamond crystal structure, thus covering a wide range of coordination numbers (12, 8, 6, and 4, respectively); a total of 384 systems. More information about the crystal structures are presented in the Supplementary Information (SI) Table S1.1. The oxides dataset is composed of six cubic oxides for each of the 96 elements X, with chemical formula X2_2O, XO, X2_2O3_3, XO2_2, X2_2O5_5 and XO3_3, thus totaling 576 additional structures, whose crystal structures are detailed in SI Table S1.2. The oxide stoichiometries are chosen such that the formal oxidation state of the element considered varies from +1 to +6. We note that the actual oxidation state is typically different (see the discussion with a Hirshfeld-I[58, 59, 60] analysis in SI Sec. S2), but shows a good correlation with calculated charges being on average about half the formal oxidation state. The X-O distance varies rather systematically over these 6 oxides (see SI Fig. S1.3), typically with the smallest distance for XO3_3 and the largest for X2_2O. This indicates that XO3_3 could be a proxy for systems with very short bond lengths, such as in high-pressure studies. The two datasets of unaries and oxides (jointly called “full dataset”; a total of 960 systems) complement each other in covering chemical and structural variety for each element.

In addition to the criteria above, all structures have been chosen to be cubic and such that forces on all atoms are zero by symmetry. Therefore, the only free parameter is the unit cell volume VV or, equivalently, the lattice parameter. As a consequence, the EOS results can be compared with any code able to compute total energies, with no requirement on the capability of computing forces and stresses.

It is important to note that most structures are not stable in nature (in particular under our constraint of cubic spacegroup symmetry). Still, they can be used to assess that all codes reproduce the same DFT result, with the advantage of providing a consistent set across the whole periodic table.

Computation of EOS parameters and comparison between AE codes

The EOS has been traditionally used to determine the computational parameters and study convergence of DFT calculations. By fitting the DFT energy vs. cell volume to an EOS, it is possible to extract the theoretical predictions of the equilibrium volume V0V_0, the bulk modulus B0B_0, and its derivative with respect to the pressure, B1B_1. The Birch–Murnaghan EOS[61]

E(V)=E0+9V0B016{[(V0V)23−1]3B1+[(V0V)23−1]2[6−4(V0V)23]}(1)E(V) = E_0 + \frac{9V_0B_0}{16} \left\{ \left[ \left(\frac{V_0}{V}\right)^{\frac{2}{3}} - 1 \right]^3 B_1 + \left[ \left(\frac{V_0}{V}\right)^{\frac{2}{3}} - 1 \right]^2 \left[ 6 - 4\left(\frac{V_0}{V}\right)^{\frac{2}{3}} \right] \right\} \tag*{(1)}

was used in the Δ-project[29, 30] and we follow the same approach by performing a fit of E(V)E(V) of Eq. (1) using calculations of the total energy corresponding to 7 equidistant constant volumes between 94% and 106% of a reference central volume V~0\widetilde{V}_0 (for each Structure). We emphasize that the results are quite sensitive to the precise choice of volume range, reference central volume, and even of fitting algorithm, as we discuss in SI Sec. S3. In this work, the reference central volumes V~0\widetilde{V}_0 for each of the 960 crystals have been chosen after an iterative process of performing more and more accurate simulations with the two AE codes considered here, until the difference between the reference central volume and the equilibrium volume of the EOS fit was smaller than the 2% volume spacing between total-energy calculations. These central reference volumes are tabulated in SI Sec. S1 and the corresponding crystal-structure files are available in the data entry associated to this Recommendation[62]. These volumes have no physical significance, but for precise comparison between computational approaches, each of them should use the same reference volumes.

The results obtained with the AE codes FLEUR and WIEN2k constitute our reference data. Figure 1 shows the distributions of the percentage difference between FLEUR and WIEN2k for V0V_0, B0B_0 and B1B_1 with respect to their average, for instance the V0V_0 difference (in %) is given by:

100⋅V0WIEN2k−V0FLEUR(V0WIEN2k+V0FLEUR)/2.(2)100 \cdot\frac{V_0^{\mathrm{WIEN2k}} - V_0^{\mathrm{FLEUR}}}{\left(V_0^{\mathrm{WIEN2k}} + V_0^{\mathrm{FLEUR}}\right)/2}. \tag*{(2)}

Although the histograms do not carry material-specific information, they clearly highlight the agreement between the two AE codes. The relative difference on the equilibrium volume is below 0.3% for all the materials except for 5 oxides (see SI Sec. S4 and the raw data in Ref. 62 for the full dataset). The discrepancies for B0B_0 and B1B_1 are larger; this is not surprising, because they originate from higher derivatives of the EOS curves (see also discussion in SI Sec. S3). We emphasize that these values, obtained after careful convergence of all numerical parameters related to the basis-set choices in the two codes, are of extremely high precision, with a spread that can even be an order of magnitude smaller than the typical discrepancies that we observe between pseudopotential codes (see discussion later).

The complete list of numerical parameters used for the AE calculations is presented in SI Sec. S5. We highlight here that the exact choice of the electronic-state smearing and of the k-point integration mesh, as well as the specific quantity considered as the energy E(V)E(V) (internal energy, or free energy including the entropic smearing contribution as we do here), are of crucial importance for a reliable comparison among codes and must be fully consistent; therefore, we discuss those in detail in section “Using the All-Electron Reference Dataset” and in Box 3.

Average AE dataset: the reference for further studies

In addition to the data for each of the two codes and in order to provide a single comparison reference, we also provide a “reference average all-electron dataset” obtained by averaging the values of V0V_0, B0B_0, and B1B_1 for each of the 960 systems in the full dataset. The corresponding values are in SI Sec. S4 and published according to the FAIR principles in Ref. 62. Considering the very good agreement between the two codes, this average dataset constitutes an excellent reference, and we use this average to compare with the pseudopotential codes in Section “Comparison with Pseudopotential-based Computational Approaches”. In addition, if error bars are desired, the spread between the results of the two AE codes can be used as an estimate of our dataset precision.

Metrics for EOS comparison

In Refs. 29, 30, the “Δ\Delta” metric was used to compare the EOS computed with two different DFT computational approaches aa and bb. There, Δ=Δ(a,b)\Delta= \Delta(a,b) was defined as:

Δ(a,b)=1VM−Vm∫VmVM[Ea(V)−Eb(V)]2 dV,(3)\Delta(a,b) = \sqrt{\frac{1}{V_M - V_m}\int_{V_m}^{V_M} [E_a(V)-E_b(V)]^2\,dV}, \tag*{(3)}

where Ea(V)E_a(V) and Eb(V)E_b(V) are the Birch–Murnaghan fits of the data points obtained from approaches aa and bb respectively, the two EOS have been lined up with respect to their minimum energy, and as discussed earlier the integral spans a ±6%\pm6\% volume range centered at a central volume V~0\tilde{V}_0 (with V~0\tilde{V}_0 values tabulated in SI Sec. S1), i.e., Vm=0.94V~0V_m = 0.94\tilde{V}_0 and VM=1.06V~0V_M = 1.06\tilde{V}_0.

The use of a single metric to compare two EOS curves simplifies the data analysis, since it can be used instead of the difference of the Birch–Murnaghan parameters V0V_0, B0B_0, and B1B_1, as we did in Figure 1. However the value of Δ(a,b)\Delta(a,b), that has the units of energy, has the shortcoming of being too sensitive to the value of the bulk modulus of the material: visually similar discrepancies between two curves result in larger Δ\Delta values for materials with larger B0B_0. This was already recognized in Ref. 63, where a modified metric Δ1\Delta_1 was suggested, renormalized to a reference value of V0V_0 and B0B_0. In addition, the Ea(V)E_a(V) and Eb(V)E_b(V) quantities in Eq. (3) are typically renormalized by the number of atoms in the unit cell, to provide a “Δ\Delta/atom” metric, independent of the choice of the simulation cell size. Since we expand our analysis to two-component oxides, generalizations might be required (e.g., by normalizing instead per formula unit).

Image not extracted

Figure 1. Histograms of the percentage difference between the results of the two all-electron codes (FLEUR and WIEN2k) with respect to their average for the three parameters of the EOS: V0V_0, B0B_0 and B1B_1, for the full dataset of unaries and oxides. Positive values indicate larger values for WIEN2k with respect to FLEUR. Mean and standard deviation (stdev) of the distributions are reported on the top left of each panel. The number near the arrows indicate the number of outliers outside of the xx-axis range. The relative difference on V0V_0 is below 0.1% for 93% of all structures in our dataset; the relative difference on B0B_0 is below 1% for 97% of the structures; and the relative difference on B1B_1 is below 2% for 92% of the structures. The 5 outliers for V0V_0 are NeO3_3 (0.302%), RbO3_3 (0.343%), Cs2_2O5_5 (0.323%), Fr2_2O5_5 (0.645%) and Ra2_2O5_5 (0.333%), corresponding to lattice parameters for FLEUR/WIEN2k of 4.320/4.324Å, 4.783/4.789Å, 6.247/6.254Å, 6.120/6.133Å and 6.238/6.244Å, respectively.

We propose and recommend here two new metrics that we label ε\varepsilon and ν\nu, and we discuss their pros and cons. We first define the following shorthand notation for the integral average of a quantity f(V)f(V) over the volume range [Vm,VM][V_m, V_M]:

⟨f⟩=1VM−Vm∫VmVMf(V) dV.(4)\langle f\rangle= \frac{1}{V_M-V_m}\int_{V_m}^{V_M} f(V)\,\mathrm{d}V. \tag*{(4)}

Using this notation, we can simply write Δ(a,b)=⟨[Ea(V)−Eb(V)]2⟩\Delta(a,b)=\sqrt{\langle[E_a(V)-E_b(V)]^2\rangle}. The first metric ε(a,b)\varepsilon(a,b) that we define is a renormalized dimensionless version of Δ\Delta:

References

  1. [1]Alberi, K. et . al . (2018). The 2019 materials by design roadmap. J. Phys. D: Appl. Phys., 52, 013001. https://doi.org/10.1088/1361-6463/aad926
  2. [2]Marzari, N., Ferretti, A. & Wolverton, C. Electronic-structure methods for materials design. Nat. Mater. 20, 736–749, DOI: 10.1038/s41563-021-01013-3 (2021).DOI
  3. [3]Pizzi, G., Cepellotti, A., Sabatini, R., Marzari, N. & Kozinsky, B. AiiDA: automated interactive infrastructure and database for computational science. Comput. Mater. Sci. 111, 218–230, DOI: 10.1016/j.commatsci.2015.09.013 (2016).DOI
  4. [4]Huber, S. P. et al. AiiDA 1.0, a scalable computational infrastructure for automated reproducible workflows and data provenance. Sci. Data 7, DOI: 10.1038/s41597-020-00638-4 (2020).DOI
  5. [5]Ong, S. P. et al. Python materials genomics (pymatgen): A robust, open-source python library for materials analysis. Comput. Mater. Sci. 68, 314–319, DOI: https://doi.org/10.1016/j.commatsci.2012.10.028 (2013).
  6. [6]Jain, A. et al. Fireworks: a dynamic workflow system designed for high-throughput applications. Concurr. Comput. Pract. Exp. 27, 5037–5059, DOI: 10.1002/cpe.3505 (2015).DOI
  7. [7]Mathew, K. et al. Atomate: A high-level interface to generate, execute, and analyze computational materials science workflows. Comput. Mater. Sci. 139, 140–152, DOI: 10.1016/j.commatsci.2017.07.030 (2017).DOI
  8. [8]Bahn, S. R. & Jacobsen, K. W. An object-oriented scripting interface to a legacy electronic structure code. Comput. Sci. Eng. 4, 56–66, DOI: 10.1109/5992.998641 (2002).DOI
  9. [9]Hjorth Larsen, A. et al. The atomic simulation environment—a Python library for working with atoms. J. Physics: Condens. Matter 29, 273002, DOI: 10.1088/1361-648X/AA680E (2017).DOI
  10. [10]Curtarolo, S. et al. Aflow: An automatic framework for high-throughput materials discovery. Comput. Mater. Sci. 58, 218–226, DOI: https://doi.org/10.1016/j.commatsci.2012.02.005 (2012).
  11. [11]Janssen, J. et al. pyiron: An integrated development environment for computational materials science. Comput. Mater. Sci. 163, 24 – 36, DOI: https://doi.org/10.1016/j.commatsci.2018.07.043 (2019).
  12. [12]Armiento, R. Database-Driven High-Throughput Calculations and Machine Learning Models for Materials Design, 377–395 (Springer International Publishing, Cham, 2020).DOI
  13. [13]Gonze, X. et al. The Abinit project: Impact, environment and recent developments. Comput. Phys. Commun. 248, 107042, DOI: 10.1016/j.cpc.2019.107042 (2020).DOI
  14. [14]Jain, A. et al. Commentary: The Materials Project: A materials genome approach to accelerating materials innovation. APL Mater. 1, 011002, DOI: 10.1063/1.4812323 (2013).DOI
  15. [15]Landis, D. D. et al. The Computational Materials Repository. Comput. Sci. & Eng. 14, 51–57, DOI: 10.1109/MCSE.2012.16 (2012).DOI
  16. [16]Kirklin, S. et al. The open quantum materials database (oqmd): assessing the accuracy of dft formation energies. npj Comput. Mater. 1, 15010, DOI: 10.1038/npjcompumats.2015.10 (2015).DOI
  17. [17]Merkys, A. et al. A posteriori metadata from automated provenance tracking: Integration of AiiDA and TCOD. J. Cheminformatics 9, 56, DOI: 10.1186/s13321-017-0242-y (2017). 1706.08704v3.DOI
  18. [18]Stevanović, V., Lany, S., Zhang, X. & Zunger, A. Correcting density functional theory for accurate predictions of compound enthalpies of formation: Fitted elemental-phase reference energies. Phys. Rev. B 85, 115104, DOI: 10.1103/PhysRevB.85.115104 (2012).DOI
  19. [19]Talirz, L. et al. Materials cloud, a platform for open computational science. Sci. Data 7, DOI: 10.1038/s41597-020-00637-5 (2020).DOI
  20. [20]Draxl, C. & Scheffler, M. Nomad: The fair concept for big data-driven materials science. MRS Bull. 43, 676–682, DOI: 10.1557/mrs.2018.208 (2018).DOI
  21. [21]Curtarolo, S. et . al . (2012). Aflowlib.org: A distributed materials properties repository from high-throughput ab initio calculations. Comput. Mater. Sci., 58, 227–235. https://doi.org/10.1016/j.commatsci.2012.02.002
  22. [22]Wilkinson, M. D. et al. The FAIR guiding principles for scientific data management and stewardship. Sci. Data 3, DOI: 10.1038/sdata.2016.18 (2016).DOI
  23. [23]Andersen, C. W. et al. OPTIMADE, an API for exchanging materials data. Sci. Data 8, DOI: 10.1038/s41597-021-00974-z (2021).DOI
  24. [24]IEEE Standard for System, Software, and Hardware Verification and Validation. IEEE Std 1012-2016 (Revision IEEE Std 1012-2012/ Incorporates IEEE Std 1012-2016/Cor1-2017) 1–260, DOI: 10.1109/IEEESTD.2017.8055462 (2017).DOI
  25. [25]Wang, A. et al. A framework for quantifying uncertainty in dft energy corrections. Sci. Reports 11, 15496, DOI: 10.1038/s41598-021-94550-5 (2021).DOI
  26. [26]Carbogno, C. et al. Numerical quality control for dft-based materials databases. npj Comput. Mater. 8, 69, DOI: 10.1038/s41524-022-00744-4 (2022).DOI
  27. [27]Poncé, S. et al. Verification of first-principles codes: Comparison of total energies, phonon frequencies, electron–phonon coupling and zero-point motion correction to the gap between abinit and qe/yambo. Comput. Mater. Sci. 83, 341–348, DOI: https://doi.org/10.1016/j.commatsci.2013.11.031 (2014).
  28. [28]Pople, J. (n.d.). Nobel Lecture. NobelPrize.org. Retrieved https://www.nobelprize.org/prizes/chemistry/1998/pople/lecture/nobelprize.org/prizes/chemistry/1998/pople/lecture
  29. [29]Lejaeghere, K. et al. Reproducibility in density functional theory calculations of solids. Science 351, aad3000, DOI: 10.1126/science.aad3000 (2016).DOI
  30. [30]71-crystal benchmark results from ref. 29 plus data collected later on. (n.d.). 71-crystal benchmark results from ref. 29 plus data collected later on. <a href="https://doi.org/https://molmod.ugent.be/deltacodesdft" rel="noopener noreferrer" target="_blank">https://doi.org/https://molmod.ugent.be/deltacodesdft</a>
  31. [31]The FLEUR project. (n.d.). The FLEUR project. Retrieved https://www.flapw.de/flapw.de
  32. [32]Wortmann, D. (2023). Fleur. In Zenodo. https://doi.org/10.5281/zenodo.7576163
  33. [33]Blaha, P. et al. Wien2k: An augmented plane wave plus local orbitals program for calculating crystal properties. http://www.wien2k.at/.wien2k.at
  34. [34]Blaha, P. et al. Wien2k: An apw+lo program for calculating the properties of solids. J. Chem. Phys. 152, 074101, DOI: 10.1063/1.5143061 (2020).DOI
  35. [35]Uhrin, M., Huber, S. P., Yu, J., Marzari, N. & Pizzi, G. Workflows in AiiDA: Engineering a high-throughput, event-based engine for robust and modular computational workflows. Comput. Mater. Sci. 187, 110086, DOI: 10.1016/j.commatsci.2020.110086 (2021).DOI
  36. [36]Huber, S. P. et al. Common workflows for computing material properties using different quantum engines. npj Comput. Mater. 7, 136, DOI: 10.1038/s41524-021-00594-6 (2021).DOI
  37. [37]Huber, S. P. (2021). AiiDA common workflows (ACWF) package. https://github.com/aiidateam/aiida-common-workflowsgithub.com/aiidateam/aiida-common-workflows
  38. [39]Grosso, G., & Pastori Parravicini, G. (2013). Solid State Physics.
  39. [40]Martin, R. M. (2013). Electronic Structure: Basic Theory and Practical Methods.
  40. [41]Cohen, M. L. & Louie, S. G. Fundamentals of Condensed Matter Physics (Cambridge University Press, Cambridge, 2016).DOI
  41. [42]Gonze, X. et al. Recent developments in the ABINIT software package. Comput. Phys. Commun. 205, 106–131, DOI: 10.1016/j.cpc.2016.04.003 (2016).DOI
  42. [43]Romero, A. H. et al. ABINIT: Overview and focus on selected capabilities. The J. Chem. Phys. 152, 124102, DOI: 10.1063/1.5144261 (2020).DOI
  43. [44]Ratcliff, L. E. et al. Flexibilities of wavelets as a computational basis set for large-scale electronic structure calculations. The J. Chem. Phys. 152, 194110, DOI: 10.1063/5.0004792 (2020).DOI
  44. [45]Clark, S. J. et al. First principles methods using CASTEP. Zeitschrift für Kristallographie - Cryst. Mater. 220, DOI: 10.1524/zkri.220.5.567.65075 (2005).DOI
  45. [46]The CP2K simulation package. (n.d.). The CP2K simulation package. Retrieved https://www.cp2k.orgcp2k.org
  46. [47]Kühne, T. D. et al. CP2K: An electronic structure and molecular dynamics software package - quickstep: Efficient and accurate electronic structure calculations. The J. Chem. Phys. 152, 194103, DOI: 10.1063/5.0007045 (2020).DOI
  47. [48]Mortensen, J. J., Hansen, L. B. & Jacobsen, K. W. Real-space grid implementation of the projector augmented wave method. Phys. Rev. B 71, 035109, DOI: 10.1103/PhysRevB.71.035109 (2005).DOI
  48. [49]Enkovaara, J. et al. Electronic structure calculations with GPAW: a real-space implementation of the projector augmented-wave method. J. Physics: Condens. Matter 22, 253202 (2010).DOI
  49. [50]Giannozzi, P. et al. QUANTUM ESPRESSO: a modular and open-source software project for quantum simulations of materials. J. Physics: Condens. Matter 21, 395502, DOI: 10.1088/0953-8984/21/39/395502 (2009).DOI
  50. [51]Giannozzi, P. et al. Advanced capabilities for materials modelling with quantum ESPRESSO. J. Physics: Condens. Matter 29, 465901, DOI: 10.1088/1361-648x/aa8f79 (2017).DOI
  51. [52]Soler, J. M. et al. The SIESTA method for ab initio order-N materials simulation. J. Physics: Condens. Matter 14, 2745–2779, DOI: 10.1088/0953-8984/14/11/302 (2002).DOI
  52. [53]García, A. et al. Siesta: Recent developments and applications. The J. Chem. Phys. 152, 204108, DOI: 10.1063/5.0005077 (2020).DOI
  53. [54]The SIRIUS domain-specific library for electronic-structure calculations. (n.d.). The SIRIUS domain-specific library for electronic-structure calculations. <a href="https://doi.org/https://github.com/electronic-structure/SIRIUS" rel="noopener noreferrer" target="_blank">https://doi.org/https://github.com/electronic-structure/SIRIUS</a>
  54. [55]Kresse, G. & Furthmüller, J. Efficient iterative schemes for ab initio total-energy calculations using a plane-wave basis set. Phys. Rev. B 54, 11169–11186, DOI: 10.1103/physrevb.54.11169 (1996).DOI
  55. [56]Kresse, G. & Joubert, D. From ultrasoft pseudopotentials to the projector augmented-wave method. Phys. Rev. B 59, 1758–1775, DOI: 10.1103/physrevb.59.1758 (1999).DOI
  56. [57]Perdew, J. P., Burke, K. & Ernzerhof, M. Generalized gradient approximation made simple. Phys. Rev. Lett. 77, 3865–3868, DOI: 10.1103/physrevlett.77.3865 (1996).DOI
  57. [58]Vanpoucke, D. E. P., Bultinck, P. & Van Driessche, I. Extending Hirshfeld-I to bulk and periodic materials. J. Comput. Chem. 34, 405–417, DOI: 10.1002/jcc.23088 (2013).DOI
  58. [59]Vanpoucke, D. E. P., Van Driessche, I. & Bultinck, P. Reply to ‘Comment on “Extending Hirshfeld-I to bulk and periodic materials” ’. J. Comput. Chem. 34, 422–427, DOI: 10.1002/jcc.23193 (2013).DOI
  59. [60]Bultinck, P., Van Alsenoy, C., Ayers, P. W. & Carbó-Dorca, R. Critical analysis and extension of the hirshfeld atoms in molecules. J. Chem. Phys. 126, 144111, DOI: 10.1063/1.2715563 (2007).DOI
  60. [61]Birch, F. Finite elastic strain of cubic crystals. Phys. Rev. 71, 809–824, DOI: 10.1103/PhysRev.71.809 (1947).DOI
  61. [62]Bosoni, E. et al. How to verify the precision of density-functional-theory implementations via reproducible and universal workflows. Mater. Cloud Arch. DOI: https://doi.org/10.24435/materialscloud:s4-3h (2023). https://doi.org/10.24435/materialscloud:s4-3h.
  62. [63]Jollet, F., Torrent, M. & Holzwarth, N. Generation of projector augmented-wave atomic data: A 71 element validated table in the XML format. Comput. Phys. Commun. 185, 1246–1254, DOI: 10.1016/j.cpc.2013.12.023 (2014).DOI
  63. [64]Glantz, S. A., Slinker, B. K., & Neilands, T. B. (2017). Selecting the “Best” Regression Model, in Primer of Applied Regression and Analysis of Variance, third edition. In McGraw-Hill Education.
  64. [66]dos Santos, F. J. & Marzari, N. Fermi energy determination for advanced smearing techniques. arXiv 2212.07988, DOI: 10.48550/ARXIV.2212.07988 (2022).DOI
  65. [67]Gillan, M. J. Calculation of the vacancy formation energy in aluminium. J. Phys. Cond. Matter 1, 689, DOI: 10.1088/0953-8984/14/005 (1989).DOI
  66. [68]van Setten, M. et al. The PseudoDojo: Training and grading a 85 element optimized norm-conserving pseudopotential table. Comput. Phys. Commun. 226, 39–54, DOI: 10.1016/j.cpc.2018.01.012 (2018).DOI
  67. [69]The PseudoDojo website. (n.d.). The PseudoDojo website. <a href="https://doi.org/http://www.pseudo-dojo.org/" rel="noopener noreferrer" target="_blank">https://doi.org/http://www.pseudo-dojo.org/</a>
  68. [70]Hartwigsen, C., Goedecker, S. & Hutter, J. Relativistic separable dual-space Gaussian pseudopotentials from H to Rn. Phys. Rev. B 58, 3641–3662, DOI: 10.1103/PhysRevB.58.3641 (1998).DOI
  69. [71]Goedecker, S., Teter, M. & Hutter, J. Separable dual-space Gaussian pseudopotentials. Phys. Rev. B 54, 1703 (1996).pubmed.ncbi.nlm.nih.gov/9986014
  70. [72]Krack, M. Pseudopotentials for H to Kr optimized for gradient-corrected exchange-correlation functionals. Theor. Chem. Accounts 114, 145–152 (2005).DOI
  71. [73]VandeVondele, J. & Hutter, J. Gaussian basis sets for accurate calculations on molecular systems in gas and condensed phases. The J. chemical physics 127, 114105 (2007).pubmed.ncbi.nlm.nih.gov/17887826
  72. [74]GPAW atomic PAW setups. (n.d.). GPAW atomic PAW setups. Retrieved https://wiki.fysik.dtu.dk/gpaw/setups/setups.html#atomic-paw-setupswiki.fysik.dtu.dk/gpaw/setups/setups.html#atomic-paw-setups
  73. [75]Prandini, G., Marrazzo, A., Castelli, I. E., Mounet, N. & Marzari, N. Precision and efficiency in solid-state pseudopotential calculations. npj Comput. Mater. 4, DOI: 10.1038/s41524-018-0127-2 (2018).DOI
  74. [76]Prandini, G. et al. A standard solid state pseudopotentials (sssp) library optimized for precision and efficiency. Mater. Cloud Arch. 2023.65 (2022). Version v11, https://doi.org/10.24435/materialscloud:f3-ym.
  75. [77]García, A., Verstraete, M. J., Pouillon, Y. n & Junquera, J. The psml format and library for norm-conserving pseudopotential data curation and interoperability. Comput. Phys. Commun. 227, 51–71, DOI: https://doi.org/10.1016/j.cpc.2018.02.011 (2018).
  76. [78]Topsakal, M. & Wentzcovitch, R. Accurate projected augmented wave (paw) datasets for rare-earth elements (re=la–lu). Comput. Mater. Sci. 95, 263–270, DOI: https://doi.org/10.1016/j.commatsci.2014.07.030 (2014).
  77. [80]Prandini, G. et al. A standard solid state pseudopotentials (sssp) library optimized for precision and efficiency. Mater. Cloud Arch. 2022.159 (2022). Version v8, https://doi.org/10.24435/materialscloud:3v-xt.
  78. [81]Sachs, M. et . al . (2021). Dft-guided crystal structure redetermination and lattice dynamics of the intermetallic actinoid compound uir. Inorg. Chem., 60, 16686–16699.
  79. [82]Dal Corso, A. Pseudopotentials periodic table: From H to Pu. Comput. Mater. Sci. 95, 337–350 (2014).DOI
  80. [84]Levin, I. (2018). NIST Inorganic Crystal Structure Database (ICSD), National Institute of Standards and Technology.
  81. [85]Kokalj, A. Xcrysden—a new program for displaying crystalline structures and electron densities. J. Mol. Graph. Model. 17, 176–179, DOI: https://doi.org/10.1016/S1093-3263(99)00028-5 (1999).
  82. [86]Mulliken, R. S. Electronic structures of molecules xi. electroaffinity, molecular orbitals and dipole moments. J. Chem. Phys. 3, 573–585, DOI: 10.1063/1.1749731 (1935).DOI
  83. [87]Mulliken, R. S. Electronic population analysis on lcao–mo molecular wave functions. i. J. Chem. Phys. 23, 1833–1840, DOI: 10.1063/1.1740588 (1955).DOI
  84. [88]Hirshfeld, F. L. Bonded-atom fragments for describing molecular charge densities. Theor. Chim. Acta 44, 129–138, DOI: 10.1007/BF00549096 (1977).DOI
  85. [89]Bader, R. F. W. A quantum theory of molecular structure and its applications. Chem. Rev. 91, 893–928, DOI: 10.1021/cr00005a013 (1991).DOI
  86. [90]Becke, A. D. A multicenter numerical integration scheme for polyatomic molecules. J. Chem. Phys. 88, 2547–2553, DOI: 10.1063/1.454033 (1988).DOI
  87. [91]Vanpoucke, D. E. P. (n.d.). HIVE-tools. Retrieved https://github.com/DannyVanpoucke/HIVE4-toolsgithub.com/DannyVanpoucke/HIVE4-tools
  88. [92]Lebedev, V. I., & Laikov, D. (1988). A quadrature formula for the sphere of the 131st algebraic order of accuracy. Doklady Math., 59, 477–481.
  89. [93]Bultinck, P., Ayers, P. W., Fias, S., Tiels, K. & Van Alsenoy, C. Uniqueness and basis set dependence of iterative hirshfeld charges. Chem. Phys. Let. 444, 205–208, DOI: https://doi.org/10.1016/j.cplett.2007.07.014 (2007).
  90. [94]Wolffis, J. J., Vanpoucke, D. E. P., Sharma, A., Lawler, K. V. & Forster, P. M. Predicting partial atomic charges in siliceous zeolites. Microporous Mesoporous Mater. 277, 184–196, DOI: 10.1016/j.micromeso.2018.10.028 (2019).DOI
  91. [95]Bueken, B. et . al . (2015). A Flexible Photoactive Titanium Metal–Organic Framework Based on a [Ti₃IV(μ₃-O)(O)₂(COO)₆] Cluster. Angew. Chem. Int. Ed., 54, 13912–13917. https://doi.org/10.1002/anie.201505512
  92. [96]Verstraelen, T., Sukhomlinov, S. V., Van Speybroeck, V., Waroquier, M. & Smirnov, K. S. Computation of charge distribution and electrostatic potential in silicates with the use of chemical potential equalization models. J. Phys. Chem. C 116, 490–504, DOI: 10.1021/jp210129r (2012).DOI

Paper details

Contents