Introduction
Density-functional calculations [1 , 2 ] are now routinely carried out in automated workflows, which produce materials databases and the training data of machine-learned models [3 , 4 , 5 , 6 ] . The results of different codes agree at the level of meV [7 ] in comparisons with matched settings, but whether an individual calculation has reached an acceptable solution is rarely tested. A calculation is declared converged when successive energies agree and the residual of the density or potential is small. Neither criterion is sufficient: a stalled or oscillating iteration can fulfill both, and so can an unstable self-consistent solution or the self-consistent solution of a Hamiltonian that is not the derivative of the reported energy. The common practical safeguard, agreement between different algorithms, fails as well when the algorithms share the same Hamiltonian. For high-throughput data the consequence is not only an energy error: a falsely converged state can carry the wrong magnetic moment or the wrong band gap, and even the wrong sign of the gap, into every quantity and every model derived from it.
Three properties are at stake and must be distinguished. A state is self-consistent if its density and Hamiltonian satisfy the fixed-point condition of the Kohn-Sham equations. It is variationally stable if no descent of the implemented free energy is accessible from it within the tested variational space. A self-consistent state can be a saddle point. The Hamiltonian is consistent with the energy if it is the derivative of the implemented energy functional. Conventional stopping criteria can wrongly suggest any of the three. We use false self-consistency as an umbrella term for a calculation that is reported as converged but fails at least one of them, not only for states that violate the fixed-point equations.
Such failures occur in practice. A PBE+U U U calculation of U O 2 \mathrm{UO}_{2} UO 2 in VASP that reports convergence lies 1.46 eV above the state reached by a descent from its own orbitals (Figure 2 ). Under a static protocol derived from that of the Materials Project [3 ] , CoO converges to a metal 0.79 eV above an insulating state with a gap of 1.40 eV. With a deliberately reintroduced Hamiltonian inconsistency, five independent optimizers agree on a state 7.5 meV too high [Figure 1 c].
Figure 1. The two tests. a) C1, schematic: the free energy A ( θ ) \mathcal{A}(\theta) A ( θ ) of the state induced by a trial Hamiltonian never falls below A min \mathcal{A}_{\min} A m i n (shaded: no admissible state). HM accepts only steps that do not raise A \mathcal{A} A , whereas the energy reported for a mixed density (open circles) is not bounded. A returned state on a plateau or at an unstable stationary point can pass conventional criteria. The descent probe C1 restarts it variationally and exposes a remaining descent. b) C2, schematic: energy of frozen orbitals versus the occupation of one eigenstate. If the Hamiltonian is the derivative of the energy, the slope equals the eigenvalue ϵ n \epsilon_n ϵ n [Eq. (4) ]. An inconsistent Hamiltonian reports a different eigenvalue, and Δ n ≠ 0 \Delta_n \ne0 Δ n = 0 . c) MgO with SCAN0r at the Γ \Gamma Γ point: final energies of the optimizers, all of which report convergence, relative to the minimum of the correct code, and the outcome of C1 (largest decrease, meV) and C2 (largest ∣ Δ n ∣ |\Delta_n| ∣ Δ n ∣ , meV), with thresholds 10 δ = 0.27 10\delta= 0.27 10 δ = 0.27 meV and 5 meV.
Figure 2. False convergence of UO2 _2 2 with PBE+U U U in VASP 6.4.2. Free energy relative to the end of the descent probe C1. Pulay mixing (grey, LMAXMIX=6, after a U U U ramp) reports convergence (dotted line) with a band gap of 0.56 eV. C1, started from the same orbitals (blue, ALGO=All), lowers the free energy by 1.46 eV and opens the gap to 3.08 eV.
Here we show that a variational formulation of the self-consistency problem in the space of Hamiltonians, together with Janak’s theorem, provides two tests that can falsify such a result. They do not depend on the optimizer that produced the state, they distinguish a state from which a descent remains from a Hamiltonian that is inconsistent with the energy, and they apply to other codes without changes. We demonstrate them on controlled failures in our own code, on f f f -electron oxides in VASP, on 22 targeted magnetic oxides computed with a database protocol, and on a census of 366 magnetic oxide structures from the Materials Project, which measures how often a calculation that reports convergence under this protocol retains a variational descent beyond its tolerance. We also state precisely what the tests cannot establish, and we give a recipe by which they can be added to an automated workflow, either as an early warning or as a full correction.
Results
Free energy as a function of the Hamiltonian
In the projector augmented-wave (PAW) method [8 ] , the Kohn-Sham Hamiltonian H ~ [ θ ] = T ^ + H ^ 0 + ∑ α θ α ρ ^ α \widetilde{H}[\theta] = \widehat{T} + \widehat{H}_0 + \sum_\alpha\theta_\alpha\widehat{\rho}_\alpha H [ θ ] = T + H 0 + ∑ α θ α ρ α , where T ^ \widehat{T} T is the kinetic-energy operator and H ^ 0 \widehat{H}_0 H 0 contains the parameter-independent terms, is linear in the parameters θ = ( v ~ ( r ) , v τ ( r ) , { Δ H R , i j } ) \theta= (\widetilde{v}(r), v_\tau(r), \{\Delta H_{R,ij}\}) θ = ( v ( r ) , v τ ( r ) , { Δ H R , ij }) : the plane-wave potential, the potential of the kinetic-energy density and the one-center Hamiltonian of each atom R R R . The operators ρ ^ α \widehat{\rho}_\alpha ρ α yield the conjugate densities ρ \rho ρ . The eigenstates of each trial Hamiltonian, with Fermi occupations for N N N electrons, define a state for which we evaluate the Mermin free energy [9 ]
A = E K S − T S , \mathcal{A} = E_{\mathrm{KS}} - TS, A = E KS − T S ,
A ( θ ) = Ω ( θ ) − θ ⋅ ρ ( θ ) + E p o t [ ρ ( θ ) ] . (1) \mathcal{A}(\theta) = \Omega(\theta) - \theta\cdot\rho(\theta) + E_{\mathrm{pot}}[\rho(\theta)]. \tag*{(1)} A ( θ ) = Ω ( θ ) − θ ⋅ ρ ( θ ) + E pot [ ρ ( θ )] . (1)
Here Ω \Omega Ω is the free energy of noninteracting electrons in H ~ [ θ ] \widetilde{H}[\theta] H [ θ ] at fixed electron number N N N , and E p o t E_{\mathrm{pot}} E pot contains the density-dependent contributions to the energy. Because A \mathcal{A} A is the free energy of an admissible ensemble, every trial state provides an upper bound to A min \mathcal{A}_{\min} A m i n , the minimum of the implemented functional over the states representable with the same basis, PAW data sets, k k k -point sampling and symmetry. It is not a bound to the exact free energy or to states outside this space. This remains true for approximate orbitals from an iterative eigensolver, provided they are orthonormal with respect to the PAW overlap operator. The gradient below assumes in addition that the orbitals are exact eigenstates of H ~ [ θ ] \widetilde{H}[\theta] H [ θ ] , that the occupations are smooth Fermi functions, so that Ω \Omega Ω remains differentiable at degeneracies, and that E p o t E_{\mathrm{pot}} E pot depends on the state only through ρ \rho ρ . The last condition holds for semilocal functionals, meta-GGAs and DFT+U U U , but not for the Fock-type terms of hybrid functionals (Supplementary Note 1, which states the proposition and its assumptions). Since Ω \Omega Ω is concave, with ∂ Ω / ∂ θ = ρ \partial\Omega/\partial\theta= \rho ∂ Ω/ ∂ θ = ρ [10 ] ,
∂ A ∂ θ = χ ( v o u t − θ ) , v o u t = ∂ E p o t ∂ ρ , (2) \frac{\partial A}{\partial\theta} = \chi\left(v_{\mathrm{out}}-\theta\right), \qquad v_{\mathrm{out}} = \frac{\partial E_{\mathrm{pot}}}{\partial\rho}, \tag*{(2)} ∂ θ ∂ A = χ ( v out − θ ) , v out = ∂ ρ ∂ E pot , (2)
where χ = ∂ 2 Ω / ∂ θ 2 \chi= \partial^{2}\Omega/\partial\theta^{2} χ = ∂ 2 Ω/ ∂ θ 2 is the symmetric, negative-semidefinite static response. The residual r = v o u t − θ r = v_{\mathrm{out}}-\theta r = v out − θ therefore satisfies
∂ A ∂ θ ⋅ r = r T χ r ≤ 0 , (3) \frac{\partial A}{\partial\theta}\cdot r = r^{T}\chi r \le0, \tag*{(3)} ∂ θ ∂ A ⋅ r = r T χr ≤ 0 , (3)
so the residual is a non-ascent direction, and a strict descent direction when it has a component outside the null space of χ \chi χ . Besides the gauge mode below, null directions can arise, for example, from linearly dependent projector functions. Because χ \chi χ is semidefinite, r T χ r = 0 r^{T}\chi r = 0 r T χr = 0 implies χ r = 0 \chi r = 0 χr = 0 , so that the whole gradient then vanishes: a residual in the null space of χ \chi χ , such as the gauge mode below, does not change the state to first order. The operator χ \chi χ never has to be constructed. Near the minimum θ ∗ \theta^{*} θ ∗ , r = − ε ( θ − θ ∗ ) r = -\varepsilon(\theta-\theta^{*}) r = − ε ( θ − θ ∗ ) with the dielectric operator ε = 1 − K H x c χ \varepsilon= 1-K_{\mathrm{Hxc}}\chi ε = 1 − K Hxc χ and the kernel K H x c = ∂ v o u t / ∂ ρ K_{\mathrm{Hxc}} = \partial v_{\mathrm{out}}/\partial\rho K Hxc = ∂ v out / ∂ ρ , so that the Newton step is ε − 1 r \varepsilon^{-1}r ε − 1 r . The inverse dielectric operator is thus the natural preconditioner in Hamiltonian space, and the Kerker filter$[11 ] G 2 / ( G 2 + q 0 2 ) G^{2}/(G^{2}+q_{0}^{2}) G 2 / ( G 2 + q 0 2 ) , with G = ∣ G ∣ G = \lvert G\rvert G = ∣ G ∣ and the screening wave vector q 0 q_{0} q 0 , is its Thomas-Fermi model (Supplementary Note 1).
Hamiltonian minimization (HM) minimizes A ( θ ) A(\theta) A ( θ ) by Anderson steps$[12 ] along the preconditioned residual and accepts a step only if A A A does not increase beyond the noise of the eigensolver [Fig. 1a]. Equation (3) concerns the residual itself. A preconditioned Anderson step is not guaranteed to descend, which is why every step is tested on the energy. The energy reported during density mixing, in contrast, is evaluated for a mixed density in the spirit of the Harris-Foulkes functional [13 , 14 ] . Since the mixed density does not represent the state induced by a trial Hamiltonian, the resulting energy is not bounded by A min \mathcal{A}_{\min} A m i n . Related energy functions of the potential have been used to choose the damping of potential mixing [15 , 16 ] , where steps that raise the energy may still be accepted. In the PAW method a shift H ~ → H ~ + c O ^ \widetilde{H} \to\widetilde{H} + c\widehat{O} H → H + c O by the overlap operator leaves the state unchanged. This null direction is shared between the plane-wave and the one-center sectors and must be projected out of the residual. Otherwise, the sectors drift apart and the iteration stalls above the minimum (Supplementary Note 1). The implementation of HM and its validation are summarized in Methods.
Two tests
Both tests apply to the state returned by any optimizer. They are complementary rather than fully independent: C1 probes residual variational descent, whereas C2 probes the consistency between occupation derivatives of the implemented energy and the reported eigenvalues. Both rest on the implemented energy, so an implementation that is wrong but consistent can pass both.
C1: descent probe. The returned state is restarted with a variational optimizer, HM in VESPAW or direct minimization in other codes, and we report Δ A C 1 = A e n d − A r e t \Delta\mathcal{A}_{\mathrm{C1}} = \mathcal{A}_{\mathrm{end}} - \mathcal{A}_{\mathrm{ret}} Δ A C1 = A end − A ret . The reference A r e t \mathcal{A}_{\mathrm{ret}} A ret is the free energy of the state represented by the returned orbitals, evaluated in the first step of the probe, not the energy that a density-mixing code reports for an intermediate mixed density: only the former belongs to an admissible state and obeys the upper bound. A decrease beyond the tolerance demonstrates that the returned state is not the lowest state accessible to the probe, and it provides a tighter upper bound on A min A_{\min} A m i n , provided that both energies refer to the same implemented functional and numerical settings. If the decrease is continuous from the returned state, this is in addition evidence that the state is not a local minimum, within the directions and the resolution tested, either because it is not stationary or because it is an unstable stationary point. A trajectory with finite steps is not a complete stability analysis, and the evidence is strongest when the descent starts in the first steps of the probe. A lower end point alone does not establish this: an optimizer can cross a barrier and leave a metastable local minimum. We therefore also record whether the energy falls steadily, without a rise exceeding the numerical tolerance, from the returned state until the decrease exceeds the threshold. The converse is weaker: if the probe finds no decrease, the returned state is a minimum only within the tolerance, along the directions the probe explores, and for the Hamiltonian of the code, whose errors C1 inherits.
C2: frozen-orbital Janak test. C2 rests not on Hamiltonian minimization but on Janak’s theorem [17 ] : if the Hamiltonian is the functional derivative of the energy E E E , the derivative of E E E with respect to an eigenstate occupation, at fixed orbitals, equals its eigenvalue,
Δ n = ∂ E ∂ f n ∣ ψ − ϵ n = ⟨ ψ n ∣ H ^ E − H ~ [ θ ] ∣ ψ n ⟩ , (4) \Delta_n = \left.\frac{\partial E}{\partial f_n}\right|_{\psi} - \epsilon_n = \langle\psi_n | \hat{H}_E - \widetilde{H}[\theta] | \psi_n \rangle, \tag*{(4)} Δ n = ∂ f n ∂ E ψ − ϵ n = ⟨ ψ n ∣ H ^ E − H [ θ ] ∣ ψ n ⟩ , (4)
where H ^ E \hat{H}_E H ^ E is the functional derivative of the implemented energy and H ~ [ θ ] \widetilde{H}[\theta] H [ θ ] the Hamiltonian whose eigenvalues are reported. The derivative is taken at fixed occupations of all other states and without the entropy term, which would add − T ∂ S / ∂ f n -T \partial S/\partial f_n − T ∂ S / ∂ f n . The orbitals are frozen because relaxing them after the occupation change would include the self-consistent response of the state. At fixed orbitals, Δ n \Delta_n Δ n isolates the derivative of the implemented energy with respect to occupation. Because the orbitals are frozen, the identity holds for each state individually, also within a degenerate set, and for orbital-dependent terms such as Fock exchange and DFT+U U U , provided ϵ n \epsilon_n ϵ n is the generalized Kohn-Sham eigenvalue of the Hamiltonian that the code uses. At a state that is not self-consistent, H ^ E − H ~ [ θ ] \hat{H}_E-\tilde{H}[\theta] H ^ E − H ~ [ θ ] also contains the residual, so that C2 responds to incomplete relaxation as well. The tests are therefore applied in sequence: C1 first, continued to its end point if it lowers the free energy, and C2 at that end point, where a remaining Δ n \Delta_n Δ n indicates a Hamiltonian-energy inconsistency in the tested state rather than incomplete relaxation. The numerical procedure and the pass thresholds (10 δ 10\delta 10 δ for C1, with δ \delta δ the energy tolerance per cell that the run claims to have met, and 5 meV for C2) are given in Methods.
C1 cannot detect a Hamiltonian-energy inconsistency because the probe shares the Hamiltonian, while C2 alone can miss incomplete relaxation in degrees of freedom that do not affect the tested frontier states. C2 is a differential test: it checks the Hamiltonian against the implemented energy, not the energy against the intended functional. Neither test can in general detect a functional that is implemented incorrectly but consistently with its derivative, and passing both does not establish that the state is the global minimum: on a nonconvex landscape, even a genuine local minimum need not be the lowest one.
Controlled failures
Two errors found during the development of VESPAW were reintroduced to test C1 and C2 on MgO with the local hybrid SCAN0r at the Γ \Gamma Γ point (Figure 1 and Table 1), with the default tolerance δ = 10 − 6 \delta=10^{-6} δ = 1 0 − 6 Hartree (0.027 meV). In the first, the double-counting term of the onsite hybrid subtracted the kinetic-energy-density potential of the frozen core. The energy functional is unchanged, but the Hamiltonian is not its derivative. All five optimizers converge to the same wrong state. The descent probe passes: because it shares the Hamiltonian, C1 cannot detect this inconsistency. The Janak test fails by −92 and −190 meV for the lowest unoccupied and highest occupied states, compared with about 1 meV for the correct code.
In the second error, the density residual of the Pulay and Anderson mixers included Fourier components outside the cutoff sphere, which the Kerker filter never mixes. The residual stayed at a floor, the mixers stopped on the change of the energy, and Anderson mixing ended 29 meV above the minimum with an unrelaxed one-center density matrix. Its frontier orbitals are converged, so C2 passes, but C1 lowers the free energy by 29.45 meV in two accepted steps. With both errors removed, all optimizers end within 0.21 meV of the minimum and pass both tests. The two controlled failures are of opposite type, and each is caught by exactly one of the tests.
False convergence in another code
Both tests apply to VASP 6.4.2 [18 , 19 ] without changes to the code (Table 1, procedure in Methods). For MgO, the Janak test passes for five functionals from PBE to the Fock hybrids, showing no Hamiltonian-energy inconsistency in the tested frontier states. For antiferromagnetic NiO, Pulay mixing with the default LMAXMIX=2 passes only the angular components up to L = 2 L = 2 L = 2 of the one-center densities to the mixer and reports convergence with δ = 10 − 7 \delta= 10^{-7} δ = 1 0 − 7 eV. C1 nevertheless lowers the free energy of the returned orbitals by 30 μ \mu μ eV, 300δ \delta δ , and with LMAXMIX=4 it passes.
The decrease is too small to matter physically, but it shows that the probe resolves, in another code, the signature of the residual floor in VESPAW: a consistent state from which a descent remains, not detected by the convergence criterion or the Janak test but detected by the descent probe.
A much larger C1 failure occurs for U O 2 \mathrm{UO}_2 UO 2 (Figure 2 ). PBE+U U U calculations of f f f -electron oxides are known to end in metastable orbital states [20 ] , and U U U is commonly ramped up to steer the iteration. In an antiferromagnetic cell with 12 atoms, ramped from PBE to U − J = 3.96 U-J=3.96 U − J = 3.96 eV with LMAXMIX=6, Pulay mixing reports convergence with tolerances from 10 − 4 10^{-4} 1 0 − 4 to 10 − 7 10^{-7} 1 0 − 7 eV, always at the same state, which is self-consistent according to the mixer. The descent probe, started from the same orbitals, lowers the free energy by 1.44 to 1.46 eV and ends in an insulating state with a gap of 3.08 eV instead of 0.56 eV. The first step of the descent changes the energy by only 0.5 μ \mu μ eV, and the decrease then grows by six orders of magnitude within 11 steps. The returned state therefore suggests a nearly stationary, variationally unstable solution: a mixing scheme can come to rest there, whereas a variational descent departs. Damped electronic dynamics started from a recomputed returned state of the same kind departs in the same way for three time steps, whereas in NiO it decays (Supplementary Note 9). Residual and energy criteria cannot distinguish such a state from a minimum, but C1 can. The Janak test passes within 1 meV at the returned state and within 1.5 meV at the end point of C1: the returned state passes the consistency test, but is not a minimum. The error is not an artifact of the Γ \Gamma Γ point: with a 2 × 2 × 2 2 \times2 \times2 2 × 2 × 2 k k k -point mesh the same protocol reports convergence 0.21 eV above the end point of C1. The probe does not find the ground state. The probes from the three Γ \Gamma Γ -point converged states end in two different minima, 24 meV apart, and a probe from random orbitals reaches a state at least 14 meV lower still.
The error need not be large to be systematic. For ferromagnetic Ce2 _2 2 O3 _3 3 with one 4 f 4f 4 f electron per Ce and 20 atoms, the same protocol converges in 42 iterations with δ = 10 − 7 \delta= 10^{-7} δ = 1 0 − 7 eV. The descent probe lowers the free energy by 17.6 meV, more than 10 5 δ 10^{5}\delta 1 0 5 δ , while the band gap stays at 2.5 eV: the occupied f f f orbitals rotate slowly in a flat part of the landscape that the mixer does not resolve. At the returned state the Janak test fails as well, with 7 meV for the highest occupied state, because the state is not fully relaxed and C2 therefore contains a residual contribution. At the end point of the probe it passes within 1 meV. This illustrates why C2 is interpreted only after C1.
Targeted survey under a database protocol
Can such failures occur under a production database protocol? We took 22 magnetic oxides from the Materials Project [3 ] , among them rocksalt, corundum, rutile, spinel and perovskite oxides and the cathode materials LiCoO2 _2 2 , LiNiO2 _2 2 , LiMn2 _2 2 O4 _4 4 and LiFePO4 _4 4 , and recomputed them in their database structures with its static protocol [21 , 22 ] (Methods). All 22 report convergence. Fifteen pass C1 within 2.1δ \delta δ , and seven fail (Table 2 ).
Table 2. Targeted survey with the Materials Project static protocol in VASP 6.4.2, database structures. N N N : atoms in the cell. δ \delta δ : energy tolerance per cell (meV). it.: iterations to the reported convergence. Δ A C 1 \Delta A_{\mathrm{C1}} Δ A C1 : change of the free energy of the returned orbitals under C1 (meV). M M M : total moment (μ B \mu_B μ B ) before and after C1. Bold: C1 failed (− Δ A C 1 > 10 δ -\Delta A_{\mathrm{C1}} > 10\delta − Δ A C1 > 10 δ ). All runs report convergence.
The failures show predominantly two behaviors. For rocksalt CoO and FeO, the protocol returns metallic states with gaps below 5 meV. C1 lowers their free energies by 0.79 and 0.90 eV at unchanged total moment and ends in insulating states with gaps of 1.40 and 1.38 eV, respectively. For Fe3 _3 3 O4 _4 4 , it lowers the free energy by 0.24 eV and opens the gap from 0.03 to 0.49 eV, and for LiFePO4 _4 4 it lowers it by 46 meV. In these four cases the iteration oscillates by several meV per step and stops when a single energy change happens to fall below the tolerance. In CoO, for example, the last three changes are +3.4, −0.70 and +0.42 meV before a change of +0.06 meV ends the run at δ = 0.1 \delta= 0.1 δ = 0.1 meV. The returned state of CoO is also close to an unstable point: over nine successive iterations of C1, the decrease per iteration grows by a factor of 1.9 to 2.5 per step, from 0.14 to 79 meV. Damped electronic dynamics with three time steps leaves the returned state in the same way and reaches the same end point. In V2 _2 2 O3 _3 3 , Co3 _3 3 O4 _4 4 and FeTiO3 _3 3 , the iteration is still descending steadily, by 0.3 to 4 meV per step, when a single small energy change satisfies the stopping criterion. C1 lowers the free energy by 14, 9.8 and 5.5 meV, 29, 14 and 11δ \delta δ , and opens the gap of V2 _2 2 O3 _3 3 from 0.13 to 0.64 eV.
With experimental structures instead, the same protocol lets these seven pass and LiCoO2 _2 2 fail by 0.37 eV (Supplementary Note 7), so the outcome depends on the starting structure and not only on the material. The 22 systems were chosen as difficult magnetic oxides. The targeted survey therefore shows that such failures occur under a production protocol, but not how often they occur.
Prevalence in a census of database structures
To measure how often a calculation that reports convergence under this protocol retains a variational descent, we computed all 366 experimentally observed magnetic oxides of V, Cr, Mn, Fe, Co, Ni and Cu in the Materials Project with an energy above the hull of at most 25 meV/atom and at most 20 sites in the database cell, excluding the 22 compounds above (Methods, Supplementary Table 14). The population was fixed before any calculation was run. Because the census enumerates the whole population, its rates for this population have no sampling error. This does not make them transferable: they characterize the modified protocol applied to these structures, not the energies stored in the database, and not other classes of materials, protocols or codes.
The 23 entries that contain W are treated separately, a decision taken after the census had revealed their anomaly and before any control was run. With the potential W p v W_{\mathrm{pv}} W pv , 15 of the 21 that report convergence fail C1, but these runs are anomalous: most end with a charge-density residual far above that of any other entry, and in ten the free energy of the returned orbitals lies 0.3 to 28 eV above the reported energy. We repeated the 23 entries with W s v W_{\mathrm{sv}} W sv and otherwise identical settings, having fixed in advance that the rate including W would be the main result only if at least 8 of the 15 failures persisted. The anomalies disappear, all 23 report convergence, and only 3 of the 15 failures persist, while three other entries fail (Supplementary Note 8, Supplementary Table 13 ). The comparison implicates the W p v W_{\mathrm{pv}} W pv potential as used here, but it does not establish the cause, because the two data sets also differ in their valence (12 and 14 electrons) and release. We give the rates below for the 343 entries without W and treat W as a sensitivity analysis: including the W entries, the rate at 10 δ 10\delta 10 δ is 20.0% with W p v W_{\mathrm{pv}} W pv and 17.4% with W s v W_{\mathrm{sv}} W sv .
Supplementary Table 13. Supplementary Table 13: W-containing entries with W_pv (06Nov2007, from PBE.52) and with W_sv (04Sep2015, from PBE.54), otherwise identical settings. rms(c): last charge-density residual printed by VASP before the final iteration. Δ F r e p \Delta F_{\mathrm{rep}} Δ F rep in meV. Bold: fails C1 (> 10 δ 10\delta 10 δ ).
Of the 343 entries, four do not report convergence within 200 iterations, a visible failure that needs no test. Of the 339 that do, 86 (25.4%) fail C1 at 5 δ 5\delta 5 δ , 57 (16.8%) at 10 δ 10\delta 10 δ , 37 (10.9%) at 30 δ 30\delta 30 δ , 25 (7.4%) at 100 δ 100\delta 100 δ and 23 (6.8%) at 300 δ 300\delta 300 δ (Figure 3 a). In 49 of the 57 failures at 10 δ 10\delta 10 δ (14.5% of the 339), the energy falls steadily from the returned state, with no rise larger than δ \delta δ , until the decrease exceeds 10 δ 10\delta 10 δ . In the other eight it first rises, so that the end point shows a lower state but not by itself an instability of the returned one. Along the whole trajectory, 19 of the 57 show a rise larger than δ \delta δ . For 23 entries (6.8%) the reported free energy lies more than 100 meV above the end point of C1, and for nine (2.7%) more than 0.5 eV. Per atom, the error exceeds 25 meV, the width of the stability window used to select these entries, for 12 (3.5%).
Figure 3. Prevalence of false convergence in a census of Materials Project magnetic oxide structures. The 339 calculations without W that report convergence under the modified static protocol. a) Distribution of the decrease of the free energy in C1 in units of the requested tolerance δ \delta δ , with the threshold 10 δ 10\delta 10 δ . Values below 10 − 2 10^{-2} 1 0 − 2 are placed in the lowest bin. b) Fraction of the converged calculations that fail C1 at 10 δ 10\delta 10 δ , by transition-metal element and by the treatment of f f f electrons, with Wilson 95% intervals and counts. Elements overlap in compounds containing more than one. c) Band gap of the returned state and of the end point of C1 for the 56 failures in which C1 converges. The 57th, whose C1 does not converge within 1000 iterations, is omitted. The line marks an unchanged gap.
The failures are not only errors of the energy. Among the 56 failures in which C1 converges, 18 open the band gap by more than 0.2 eV and 12 by more than 0.5 eV. Ten change from a metal (gap below 0.1 eV) to an insulator (gap above 0.5 eV, Figure 3 c), and 19 change the total moment by more than 0.1 μ B \mu_{\mathrm{B}} μ B . Thirty show at least one of these changes. Of the 282 entries that pass, none opens the gap by more than 0.2 eV, and two change the moment. In the largest failure, GdCrTeO6 _6 6 , the reported energy lies 5.8 eV above the end point of C1, the moment changes from 3.25 to 8.00 μ B \mu_{\mathrm{B}} μ B and the gap from 0.06 to 2.71 eV.
The rate depends strongly on the chemistry (Figure 3 b). It is 63% of the compounds containing Co (17 of 27) and 34% of those containing V, but 5–6% of those containing Mn or Ni. It is 50% (13 of 26) for compounds with f f f electrons in the valence (Ce, Eu, Gd, U and Np, without a Hubbard correction on the f f f shell), and 14.1% for the others, including 12.5% for those whose f f f electrons are in the frozen core. It rises from 11.8% for computed cells of up to 10 atoms to 19.0% for cells of 16 to 20 atoms. These rates hold for this protocol and initialization and do not by themselves show that the chemistry causes the failures. All runs start from ferromagnetic moments, whatever the ordering in the database, so that the census tests this initialization and not the database magnetic order: none of the 42 entries labelled antiferromagnetic ends with a net moment below 0.1 μ B \mu_{\mathrm{B}} μ B , and for 12 of them the chemically primitive cell used here is smaller than the database cell and cannot hold that order (a control from the database order follows below). The rate is 14.4% (39 of 271) for entries labelled ferromagnetic, 16.7% (7 of 42) for antiferromagnetic and 42.3% (11 of 26) for ferrimagnetic. These groups overlap and are not independent. Neither the energy history nor the residual reported by the code identifies the failures: two or more sign changes among the last six energy changes occur in 37% of the failures and 34% of the passes, and the median charge-density residual is 0.024 and 0.020, respectively.
Four controls test the protocol (Methods, Supplementary Note 9). First, the starting state: for 25 failures and 25 passes drawn at random, the run was repeated with the tetrahedron method of the original input set and probed with C1 with Gaussian smearing as before. The tetrahedron run minimizes a different free energy, so this control does not test whether it reached a minimum of its own functional. It tests whether the failures depend on the state from which the probe starts. Of the 25 failures, 22 still fail at 10 δ 10\delta 10 δ , and 2 of the 25 passes fail, so the failures are not specific to a start from a run with Gaussian smearing. Second, the magnetic start: the 42 entries labelled antiferromagnetic were repeated from the moments and cell of the database. Of these, 41 report convergence and all 41 end with a net moment below 0.1 μ B 0.1\ \mu_{\mathrm{B}} 0.1 μ B . The end point of C1 lies below that of the ferromagnetic start in 40 of the 42 entries, by a median of 3.6 meV/atom. Four of the 41 fail C1 at 10 δ 10\delta 10 δ (9.8%), compared with 7 of 42 from the ferromagnetic start: two failures become passes, and no pass becomes a failure. Third, the optimizer: for 20 failures drawn at random and for C o O \mathrm{CoO} CoO and F e O \mathrm{FeO} FeO , Pulay mixing started from the end point of C1 reports convergence within two iterations, within 2.6 meV of the free energy of C1 and with a gap changed by at most 0.07 eV, so the end points are not an artifact of direct minimization. Fourth, a plain restart. All 57 failures and 25 randomly drawn passes were computed again and then restarted from their orbitals with unchanged settings. The repeated runs give the same result for 56 of the 57 failures and for all 25 passes. The one exception, N a 3 C o 2 S b O 6 \mathrm{Na}_3\mathrm{Co}_2\mathrm{SbO}_6 Na 3 Co 2 SbO 6 , now gives 7.5 δ 7.5\delta 7.5 δ and passes, so the repeated set contains 56 failures and 26 passes. The restart ends more than 10 δ 10\delta 10 δ below the reported free energy for only 15 of the 56 failures (27%) and for none of the passes. We had fixed 80% in advance as the condition for a restart to replace C1. In 33 of the 41 missed failures the restart ends within 3 δ 3\delta 3 δ of the reported free energy, even where C1 lowers the energy by more than 800 δ 800\delta 800 δ , and two restarts do not converge. Restarting with the same algorithm thus brings the calculation back to almost the same state. The first control also shows the limit of a pass. Started from the run with the tetrahedron method, C1 for C o O \mathrm{CoO} CoO ends in a state with a gap of 0.02 eV and passes, although its free energy lies 0.60 eV above the insulating end point reached from the run with Gaussian smearing. Both end points are evaluated with the same functional and settings of C1.
Cost
C2 requires five frozen-orbital energy evaluations. In the census, C1 takes a median of 14 iterations for a state that passes and 68 for one that fails, compared with 15 for the SCF run. Its median wall time is 0.84 of that of the SCF run it tests (interquartile range 0.67–1.13): 0.80 for states that pass and 2.3 for those that fail, up to 17 times the SCF run. Applying the full probe therefore about doubles the median cost of a static calculation in this census, and it costs most where it is most needed. Two modes of use differ in cost. As an early warning, the probe stops as soon as the decrease exceeds 10 δ 10\delta 10 δ , which takes a median of 7 iterations for the failures, and a state that passes costs the 0.80 given above. As a full correction, the probe continues to the lower end point, at the median cost of 2.3 static runs given above for the failures. Relative to a workflow that relaxes the structure before the static run, the additional cost is smaller. A plain restart is cheaper, a median of 0.22 of the run against 1.39 for C1 in the same set, but it misses 41 of the 56 failures.
Discussion
The examples establish a hierarchy. A small change of the energy does not imply convergence, a small residual does not imply a correct solution, and agreement between independent optimizers does not imply either when they share the Hamiltonian. What the variational formulation adds is a pair of falsifiable statements about a returned state: that a variational restart finds no descent beyond the tolerance (C1), and that, at the end point of C1, the tested frontier states show no inconsistency between the reported Hamiltonian and energy (C2). A failure of C1 shows that a lower state is accessible from the returned state and, if the descent is continuous, it gives evidence that the returned state is not a local minimum within the directions and the resolution tested. A failure of C2 at the end point of C1 indicates a Hamiltonian-energy inconsistency in the tested state. A pass holds relative to the probe, its tolerance and the Hamiltonian space of the code. The first test follows from the upper-bound property of the free energy, with Eqs. (2) and (3) providing the descent direction of HM, and the second from Janak’s theorem at frozen orbitals. Together they separate two distinct ways in which a calculation can falsely appear converged: a remaining descent and an inconsistent Hamiltonian, which no single conventional criterion distinguishes. The mechanisms behind the failures are classified in Table 3.
For databases and the models trained on them, the relevant failure is not the energy alone. In
Table 3. Mechanisms of false convergence in this work. The interpretation of UO2 _2 2 and CoO as close to unstable points rests on the trajectories of C1 and of damped electronic dynamics, in which the residual grows roughly geometrically over the first iterations from the returned state, while that of NiO decays. A quantitative criterion fixed in advance was not met, so the interpretation is qualitative (Supplementary Note 9). The mechanisms are not exclusive, and CoO shows two.
CoO, Fe3 _3 3 O4 _4 4 and V2 _2 2 O3 _3 3 the falsely converged state differs qualitatively from the end point of the probe: a metal instead of an insulator, or a gap that changes by a factor of five or more. Such entries can propagate the artifact into models trained on them. In the census, more than half of the failures with a converged probe change the gap or the moment, and the failures concentrate in compounds of Co and V and in those with f f f electrons in the valence. Because the tests require only a restart from the stored orbitals, they can also be applied retrospectively to existing entries whose wave functions or inputs are available. The census rates refer to the modified protocol described in Methods, with Gaussian smearing and a ferromagnetic start from scratch, and they are not error rates of the stored database entries, and they count only the calculations that report convergence.
Started from runs with the tetrahedron method, 22 of 25 sampled failures persist under the same probe. With the antiferromagnetic order of the database, the rate of those entries falls from 7 of 42 to 4 of 41.
Testing whether a self-consistent solution is a minimum has a long history in quantum chemistry, where the orbital Hessian of Hartree-Fock and Kohn-Sham solutions is examined for negative eigenvalues [23 , 24 ] , and direct minimization reaches minima of the energy rather than fixed points of the mixing [25 , 26 , 27 , 28 , 29 ] . C1 needs no Hessian: it uses the variational optimizer that the code already has and only the energy of the returned state, so it applies to plane-wave PAW calculations with many states and to the output of any mixing scheme. Unlike a Hessian analysis, it does not classify the stationary point. A decrease beyond the tolerance shows a lower accessible state, and only the shape of the trajectory indicates an instability.
For automated workflows, this suggests five practical rules. (i) After a run that reports convergence, restart from its orbitals with a variational optimizer (in VASP, ALGO=All), compare with the free energy of the returned orbitals, evaluated in the first step, and flag the run if the decrease exceeds 10 δ 10\delta 10 δ . For an early warning, stop there, and for a full correction continue to the end point. (ii) For new or modified functionals and code versions, apply the frozen-orbital Janak test at the end point of C1 and flag ∣ Δ n ∣ > 5 |\Delta n| > 5 ∣Δ n ∣ > 5 meV. (iii) In the correction mode, store the end point of C1, not the returned state, when C1 fails, after checking that C1 converged with the settings of the production run. (iv) Do not rely on a tighter tolerance or on agreement between mixing schemes: the UO2 _2 2 calculation reports the same false state from 10 − 4 10^{-4} 1 0 − 4 to 10 − 7 10^{-7} 1 0 − 7 eV. A plain restart with the same settings does not replace C1 either: in the census it misses 41 of 56 failures. (v) Sample competing magnetic and orbital orders separately, since the tests do not address the global minimum.
Three limits are essential. First, passing C1 and C2 does not prove that the state is the global minimum. Competing magnetic or charge orders must still be sampled, and in U O 2 \mathrm{UO_2} U O 2 and L i C o O 2 \mathrm{LiCoO_2} LiCo O 2 the probe ends in different minima from different starting points. Started from a run with the tetrahedron method, C1 for CoO passes a nearly gapless state 0.60 eV above the insulating state of the same functional: a pass excludes a remaining descent, not a false state. Second, an error that changes the functional and its derivative consistently can pass both tests. In our development, a missing factor 1 2 \frac{1}{2} 2 1 in the one-center kinetic-energy density of the meta-GGA passed a relaxed Janak test within 5 meV and was found only by comparison with another code (Supplementary Note 6). Third, C2 probes only the frontier states, and C1 only the Hamiltonian space of the code, including its symmetry: a state that is stable under symmetric variations can be unstable when the symmetry is broken.
In summary, treating the Kohn-Sham Hamiltonian as the variable of an energy minimization, in which the free energy of the induced state is an upper bound and the residual weighted by the response function is its gradient, supplements conventional convergence criteria with falsifiable tests of variational descent and of Hamiltonian-energy consistency. The resulting descent probe, together with a frozen-orbital Janak test, exposes false convergence that conventional criteria and agreement between optimizers can miss. In VASP, it finds a U O 2 \mathrm{UO_2} U O 2 calculation converged 1.46 eV above a lower state and, under a database protocol, CoO and FeO returned as metals 0.8–0.9 eV above insulating states, and it finds a decrease beyond the threshold in 16.8% of the converged calculations in a census of tungsten-free Materials Project structures recomputed with a modified static protocol. The tests require no change of the code to which they are applied.
Methods
Hamiltonian minimization. We implemented HM in the PAW code VESPAW [30 ] , derived from CP-PAW [8 ] . Steps are proposed by an Anderson extrapolation [12 ] from the accepted points ( θ k , r k ) (\theta_k, r_k) ( θ k , r k ) , applied to the preconditioned residual with a damping factor β \beta β , θ ′ = θ 1 + β P r 1 − ∑ k ≥ 2 γ k [ ( θ 1 − θ k ) + β P ( r 1 − r k ) ] \theta' = \theta_1 + \beta P r_1 - \sum_{k \ge2} \gamma_k[(\theta_1 - \theta_k) + \beta P(r_1 - r_k)] θ ′ = θ 1 + β P r 1 − ∑ k ≥ 2 γ k [( θ 1 − θ k ) + β P ( r 1 − r k )] , where the coefficients γ k \gamma_k γ k minimize the norm of the extrapolated residual and P P P is the Kerker preconditioner. If the extrapolated step exceeds the plain damped step β P r 1 \beta P r_1 β P r 1 by more than a factor of ten, the plain step is used. For each trial θ \theta θ , the eigenstates of H ~ [ θ ] \widetilde{H}[\theta] H [ θ ] are determined with the Davidson, RMM-DIIS or conjugate-gradient eigensolver, the Fermi occupations are set for fixed N N N , A ( θ ) \mathcal{A}(\theta) A ( θ ) is evaluated for this state, and the gauge mode H ~ → H ~ + c O ^ \widetilde{H} \to\widetilde{H} + c\widehat{O} H → H + c O is projected out of the residual. A proposal is accepted if A ( θ ′ ) ≤ A b e s t + ϵ \mathcal{A}(\theta') \le\mathcal{A}_{\mathrm{best}} + \epsilon A ( θ ′ ) ≤ A best + ϵ , with a slack ϵ = 10 − 7 \epsilon= 10^{-7} ϵ = 1 0 − 7 Ha that absorbs the noise of the iterative eigensolver. Otherwise the step is halved towards the best point, and the best point is evaluated again after five consecutive rejections. The damping β ∈ [ 0.02 , 0.5 ] \beta\in[0.02, 0.5] β ∈ [ 0.02 , 0.5 ] is halved once per sequence of rejections, at the second consecutive rejection, when the Anderson history is also discarded, and increased by a factor 1.2 after an accepted step and 1.5 after three consecutive accepted steps. The iteration is converged when the mean-square residual of the grid potential is below its threshold and the largest residual of the one-center Hamiltonian below its square root, or after two consecutive accepted full-length steps whose energy gain is below the energy tolerance. In cells with vacuum, θ \theta θ is set equal to v o u t v_{\mathrm{out}} v out where the density is below 10 − 6 a B − 3 10^{-6}a_{\mathrm{B}}^{-3} 1 0 − 6 a B − 3 , and θ \theta θ and the history are shifted along the gauge mode when its offset from v o u t v_{\mathrm{out}} v out exceeds 0.5 Ha. For the local hybrid functionals, the onsite Fock-type correction is applied as an operator constructed from the current state. The eigensolvers return orthonormal eigenstates: the Davidson correction vectors are orthogonalized twice to all current bands and the extended basis is orthogonalized canonically with an eigenvalue cutoff of 10 − 10 10^{-10} 1 0 − 10 , and RMM-DIIS is preceded by eight Davidson iterations. With space-group symmetry, the output density and the one-center density matrices are symmetrized, so that every residual and every Anderson combination keeps θ \theta θ in the symmetric subspace. In parallel runs, all scalar products of the energy test and the Anderson fit are summed over the MPI tasks, so that every task takes the same decisions. The derivation of the gradient, the second derivative and the gauge mode and the effect of the step rules are given in Supplementary Notes 1 and 2.
VESPAW calculations. The VESPAW calculations use scalar-relativistic PAW setups, a cutoff for the auxiliary density of twice that of the wave functions, and the PBE functional [31 ] unless stated otherwise. The controlled failures use MgO with the local hybrid SCAN0r, 25% exact exchange in the onsite terms, at the Γ \Gamma Γ point and the default energy tolerance δ = 10 − 6 \delta= 10^{-6} δ = 1 0 − 6 Ha. For the validation, NiO is antiferromagnetic (AFM-II) in a rhombohedral cell with four atoms, with PBE+U U U (U − J = 6.3 U-J=6.3 U − J = 6.3 eV on Ni 3 d 3d 3 d [32 ] ) or local hybrids with 25% exact exchange. The spin order is prepared by a short friction run with an orbital potential, which is then switched off, and all optimizers start from this state. The reference is Car–Parrinello friction dynamics [33 ] , which minimizes the same functional in the space of orbitals. For bulk semiconductors, metals, antiferromagnetic and mixed-valent oxides and local hybrid functionals, HM with three eigensolvers and friction dynamics reach the same free energy within 0.2 meV. The larger deviations found for direct minimization of the oxides and for HM of a water layer in vacuum all lie above the minimum. Systems, cutoffs, k k k -point meshes and all validation energies are listed in Supplementary Table 1 .
Supplementary Table 1. Converged free energies. DAV: HM with the Davidson eigensolver, in eV. The other columns give the deviation from DAV in µeV. RMM: HM with RMM-DIIS. CG: HM with band-by-band conjugate gradients. DM: direct minimization. MD: Car-Parrinello friction dynamics. E g E_g E g : Kohn-Sham band gap of the DAV calculation on the given k-point mesh. SCAN: evaluated with the orbital kinetic-energy density.
Descent probe C1 in VASP. C1 restarts the final orbitals of the tested run (ISTART=1) with direct minimization (ALGO=All), all other settings unchanged, an energy tolerance of 10 − 7 10^{-7} 1 0 − 7 eV and up to 300 iterations (600 for the f f f -electron oxides). A r e t A_{\mathrm{ret}} A ret is the free energy of the first step of the restart, which evaluates the returned orbitals, with the density they generate and the occupations of the probe, before any update of the orbitals. For the 282 W-free converged entries that pass, A r e t A_{\mathrm{ret}} A ret is within δ \delta δ of the free energy reported by the run in 248 cases and within 10 δ 10\delta 10 δ in all of them (median 0.14 δ 0.14\delta 0.14 δ ), so the first step reproduces the energy of the returned state. Larger differences appear only for failures, where the reported energy can belong to a mixed density (Supplementary Note 7). With the reported free energy as reference, the result at 10 δ 10\delta 10 δ changes for 6 of the 339 entries. A e n d A_{\mathrm{end}} A end is the free energy at the end of the probe. For FeO, where ALGO=All with the default step became unstable, TIME=0.1 was used.
Janak test C2. We evaluate ∂ E / ∂ f n \partial E/\partial f_n ∂ E / ∂ f n for the lowest unoccupied and highest occupied states by the second-order one-sided difference [ − 3 E ( 0 ) + 4 E ( h ) − E ( 2 h ) ] / ( 2 h ) [-3E(0)+4E(h)-E(2h)]/(2h) [ − 3 E ( 0 ) + 4 E ( h ) − E ( 2 h )] / ( 2 h ) with ∣ h ∣ = 0.01 |h|=0.01 ∣ h ∣ = 0.01 electrons, requiring five frozen-orbital energy evaluations: one at the unchanged occupations and two for each state. Any code that returns the energy of given orbitals at given occupations can do this. In VASP, the converged orbitals are read with fixed occupations (ISMEAR=-2, FERWE/FERDO), and a single step recomputes the density and the energy of these orbitals (ALGO=Eigenval). With ALGO=None the stored eigenvalues are reused and the energy is not that of the frozen orbitals. For the f f f -electron oxides, the spin-up frontier states are tested. The MgO tests in VASP use the Γ \Gamma Γ point with PBE, r2 ^{2} 2 SCAN, SCAN, HSE06 and PBE0. NiO in VASP is antiferromagnetic with PBE+U U U in a cell of 64 atoms at the Γ \Gamma Γ point with δ = 10 − 7 \delta=10^{-7} δ = 1 0 − 7 eV, and LiMn2 _2 2 O4 _4 4 uses the cubic cell of 56 atoms with δ = 10 − 4 e V \delta= 10^{-4}\ \mathrm{eV} δ = 1 0 − 4 eV .
Pass thresholds. The pass thresholds are deliberately conservative operational choices. C1 fails if the probe lowers the free energy by more than 10 δ 10\delta 10 δ , where δ \delta δ is the energy tolerance per cell that the run claims to have met. For a tolerance given per atom, δ = N a t δ a t \delta= N_{\mathrm{at}}\delta_{\mathrm{at}} δ = N at δ at . The factor is motivated, not derived: if the energy approached its limit geometrically with a ratio r ≤ 0.9 r \le0.9 r ≤ 0.9 , a last change below δ \delta δ would leave an energy error of at most 9 δ 9\delta 9 δ (Supplementary Note 7). We do not rely on this assumption: all C1 failures reported in Table 1 are at least 300 δ 300\delta 300 δ , except four database cases at 11 11 11 –33 δ 33\delta 33 δ , and increasing the threshold from 10 δ 10\delta 10 δ to 30 δ 30\delta 30 δ would change the classification of only three. For C2, if the Hamiltonian is the functional derivative of the implemented energy, Δ n = 0 \Delta_n = 0 Δ n = 0 is exact, and its numerical evaluation has a truncation error h 2 E ′ ′ ′ / 3 h^2E'''/3 h 2 E ′′′ /3 , of second order in the occupation change. The error budget of C2 in VASP (Supplementary Notes 7 and 9) has five parts. The truncation error is below 0.2 meV: Δ n \Delta_n Δ n of MgO changes by less than this between ∣ h ∣ = 0.005 |h| = 0.005 ∣ h ∣ = 0.005 and 0.02. The scatter of the frozen-orbital energies reaches 1.5 meV for NiO with PBE+U U U , where Δ n \Delta_n Δ n varies with ∣ h ∣ |h| ∣ h ∣ without a trend. A finer integration grid (ADDGRID), a cutoff of 800 eV and a larger smearing change Δ n \Delta_n Δ n by at most 0.2 meV. With SCAN, Δ n \Delta_n Δ n has a systematic offset of 3.0 to 3.6 meV, compared with at most 1.3 meV with r2 ^2 2 SCAN and 0.7 meV with PBE. Its origin is not established, and it limits the resolution of C2 for SCAN. Finally, a residual of the tested state adds to Δ n \Delta_n Δ n : for Ce2 _2 2 O3 _3 3 , Δ H \Delta_H Δ H is 7.3 meV at the returned state and 1.0 meV at the end point of C1, which is why C2 is applied there. We call C2 failed if ∣ Δ n ∣ > 5 |\Delta_n| > 5 ∣ Δ n ∣ > 5 meV. This threshold exceeds the other contributions for the functionals tested, but only by 1.4 meV for SCAN, and a value between 5 and 10 meV should be confirmed with tighter settings before it is read as an inconsistency.
f f f -electron oxides. U O 2 \mathrm{UO}_{2} UO 2 and C e 2 O 3 \mathrm{Ce}_{2}\mathrm{O}_{3} Ce 2 O 3 were computed with VASP 6.4.2 (vasp_gam), PBE PAW potentials, a plane-wave cutoff of 520 eV, PREC=Accurate, LASPH, Gaussian smearing of 0.01 eV and the Γ \Gamma Γ point. U O 2 \mathrm{UO}_{2} UO 2 : fluorite structure with a = 5.47 A ˚ a = 5.47\,\text{\AA} a = 5.47 A ˚ , 12 atoms, collinear type-1 antiferromagnet, PBE+U U U in the form of Dudarev et al. with U = 4.5 U = 4.5 U = 4.5 eV and J = 0.54 J = 0.54 J = 0.54 eV on U 5 f 5f 5 f . C e 2 O 3 \mathrm{Ce}_{2}\mathrm{O}_{3} Ce 2 O 3 : A-type structure, 2 × 2 × 1 2 \times2 \times1 2 × 2 × 1 cell with 20 atoms, ferromagnetic, U = 5 U = 5 U = 5 eV on Ce 4 f 4f 4 f . A calculation with PBE is followed by one with U / 2 U/2 U /2 and then one with full U U U , each restarted from the orbitals of the previous step, with Pulay mixing (ALGO=Normal) and up to 300 iterations. These are minimal test cells, and the energies are not meant as physical results (Supplementary Note 7).
Database protocol. Structures were taken from the Materials Project [3 ] through its application programming interface [34 ] and recomputed with its static protocol as implemented in pymatgen [21 ] (MPStaticSet): PBE+U U U with the U U U values of the database [22 ] , ferromagnetic initial moments, 520 eV cutoff, its k k k -point density, LMAXMIX=4 (6 if the cell contains an element heavier than Ba), ALGO=Fast (Davidson followed by RMM-DIIS with Pulay mixing), and an energy tolerance of 5 × 10 − 5 5 \times10^{-5} 5 × 1 0 − 5 eV per atom, that is δ = N a t × 5 × 10 − 5 \delta= N_{\mathrm{at}} \times5 \times10^{-5} δ = N at × 5 × 1 0 − 5 eV per cell. We changed two settings of this input set: Gaussian smearing of 0.05 eV instead of the tetrahedron method, so that the reported energy is the free energy tested by C1, and a limit of 200 instead of 100 iterations. Each run starts from scratch, from atomic densities and random orbitals, without the output of a preceding relaxation. For the targeted survey, each compound is represented by the database structure of the experimentally known polymorph with the lowest energy above the hull, in its primitive cell. These calculations do not reproduce the energies stored in the database, which come from a different workflow: they test this modified protocol, not the database entries.
Census. The population was defined before any calculation: oxides labelled experimentally observed (theoretical=False) that contain at least one of V, Cr, Mn, Fe, Co, Ni and Cu, with a ferromagnetic, ferrimagnetic or antiferromagnetic ground state in the database (is_magnetic), an energy above the hull of at most 25 meV/atom and at most 20 sites, excluding the 22 compounds of the targeted survey (366 entries, database version 2026.09.28). All entries were computed with the protocol above in VASP 6.4.3 on 12 cores and then probed with C1 (ALGO=All, EDIFF=10^{-7} eV, at most 300 iterations). The other calculations of this work used VASP 6.4.2. The two versions agree for CoO and NiO, with C1 giving − 789.3 -789.3 − 789.3 meV for CoO in both. Six entries that stopped at the start with a symmetry error were computed with SYMPREC=10^{-4}. Runs in which C1 reached 300 iterations were repeated with a limit of 1000. If C1 did not converge within this limit (Na3 _3 3 Co2 _2 2 SbO6 _6 6 among the entries without W), we use the lowest energy along the trajectory, a lower bound on the decrease. Runs that do not report convergence within 200 iterations are counted separately. Four of the W-free runs that report convergence needed more than 100 iterations and all four fail C1. With the original limit of 100 the rate at 10 δ 10\delta 10 δ would be 53 of 335 (15.8%). All entries are computed in the chemically primitive cell and start from the ferromagnetic moments of the protocol, also those whose database ordering is antiferromagnetic or ferrimagnetic. Rates are relative to the calculations that report convergence. The POTCAR files are from PBE.54, except W_pv, which is absent from it and was taken from PBE.52 (06Nov2007). The control uses W_sv from PBE.54. The Wilson 95% intervals of Figure 3 b indicate the statistical uncertainty of a subgroup rate read as an estimate for compounds of that kind.
Controls. The selection rules and random seeds of the controls were fixed in a script before any control was run. The smearing and optimizer controls draw from the W-free failures and passes whose run and C1 converged, that were computed without the change of SYMPREC and whose k k k -point mesh has at least four points. For the smearing control, 25 failures and 25 passes (seed 2026) and CoO, FeO and NiO of the targeted survey were computed with ISMEAR=-5 and otherwise unchanged settings, followed by C1 with Gaussian smearing of 0.05 eV as in the census. For the magnetic start, all 42 W-free converged entries labelled antiferromagnetic were computed in the database cell (larger than the census cell for 12) with the database moments. For the optimizer control, 20 failures (seed 2027), CoO and FeO were restarted from the end point of C1 with Pulay mixing (ALGO=Normal, EDIFF as in the census). An end point counts as stable if this run reports convergence within max ( 10 δ , 1 m e V ) \max(10\delta, 1\ \mathrm{meV}) max ( 10 δ , 1 meV ) of the free energy of C1 and within 0.1 eV of its gap. For the restart control, all 57 W-free failures and 25 passes (seed 2028, same filter) were computed again with the census settings, and from the returned orbitals we ran both a restart with the same INCAR (ISTART=1) and C1. The restart detects an entry if it ends more than 10 δ 10\delta 10 δ below the reported free energy. It would be accepted as a replacement for C1 if it detected at least 80% of the entries that fail C1 in the same calculation and flagged at most two passes. The criteria were fixed before the runs: for the tetrahedron start, at least 15 of the 25 failures and at most 4 of the 25 passes failing at 10 δ 10\delta 10 δ . The step dependence of C2 was tested with ∣ h ∣ = 0.005 |h| = 0.005 ∣ h ∣ = 0.005 , 0.01 and 0.02 on MgO with PBE, SCAN and r2 ^2 2 SCAN, for PBE and SCAN also with ADDGRID, a cutoff of 800 eV and smearing of 0.05 eV, and on NiO with PBE+U U U . Damped electronic dynamics (ALGO=Damped, TIME = 0.02, 0.05 and 0.10) and ALGO=All were started from the returned orbitals of UO2 _2 2 (recomputed), CoO and NiO. The criterion fixed in advance, a growth rate of the residual per unit of n n n TIME that agrees within 30% across the time steps, was not met (Supplementary Note 9). All controls used VASP 6.4.3.
Data availability
The input files of all VESPAW and VASP calculations, the Materials Project identifiers of all entries of the census, the selection and analysis scripts and the per-system results are provided with the Supplementary Information (inputs.zip) and will be archived at Zenodo [DOI to be provided at the proof stage]. VASP POTCAR files are not redistributed for licensing reasons.
Code availability
VESPAW is distributed under the GNU General Public License, version 3, and the version used here, 1.0.2, is archived at Zenodo [30 ] . The scripts that run C1 and C2 on top of VASP and analyse the results are provided with the input files. VASP is a commercial code.
Acknowledgements The author thanks Axel Groß (Ulm University) and M. Saiful Islam (University of Oxford) for fruitful discussions. The author acknowledges financial support from the German Research Foundation (DFG) through Project ID 390874152 (EXC 2154, POLiS Cluster of Excellence). Computer time provided by the state of Baden-Württemberg through bwHPC and the German Research Foundation (DFG) through grant no INST 40/575-1 FUGG (JUSTUS2 cluster) is gratefully acknowledged.
Author Contributions M.S. conceived the work, developed the theory and the code, performed all calculations and wrote the manuscript.
Competing Interests The author declares no competing interests.
Correspondence Correspondence and requests for materials should be addressed to Mohsen Sotoudeh (email mohsen.sotoudeh@kit.edu).
Mohsen Sotoudeh
Correspondence and requests for materials should be addressed to Mohsen Sotoudeh (email mohsen.sotoudeh@kit.edu).
Hartree atomic units are used in the formalism, and results are given in eV. The input files of all calculations are contained in inputs.zip. The calculations in Tables 1, 2 and 6 are also part of the regression suite of VESPAW S1.
Free energy as a function of the Hamiltonian
We minimize the Mermin free energy S2, S3
A [ { ψ n } , { f n } ] = E K S [ { ψ n } , { f n } ] − T S [ { f n } ] , (5) \mathcal{A}[\{\psi_n\}, \{f_n\}] = E_{\mathrm{KS}}[\{\psi_n\}, \{f_n\}] - T S[\{f_n\}], \tag*{(5)} A [{ ψ n } , { f n }] = E KS [{ ψ n } , { f n }] − T S [{ f n }] , (5)
where E K S E_{\mathrm{KS}} E KS is the PAW total energy of the orbitals ψ n \psi_n ψ n with occupations f n ∈ [ 0 , 1 ] f_n \in[0,1] f n ∈ [ 0 , 1 ] and ∑ n f n = N \sum_n f_n = N ∑ n f n = N .
The orbitals are orthonormal with respect to the overlap operator O ^ \hat{O} O ^ of the PAW method. S S S is the entropy of non-interacting electrons. For insulators and T → 0 T \to0 T → 0 , A \mathcal{A} A is the Kohn-Sham energy.
Figure Supplementary Fig. 1. Two routes to self-consistency. (a) Density mixing: the Hamiltonian is built from an input density, and input and output densities are mixed. The energy of the mixed density is not an upper bound and can fall below the minimum (red). (b) Hamiltonian minimization (HM): the parameters θ \theta θ of the PAW Hamiltonian are the variables, and the free energy A ( θ ) \mathcal{A}(\theta) A ( θ ) of the state induced by each trial Hamiltonian is an upper bound. A trial step that raises it is rejected. The energy curves are schematic.
In the PAW method the Hamiltonian, which depends on the state, has the form
H ~ = − 1 2 ∇ 2 + v ~ ( r ) + ∑ R , i j ∣ p ~ R , i ⟩ Δ H R , i j ⟨ p ~ R , j ∣ (6) \tilde{H}=-\frac{1}{2}\nabla^2+\tilde{v}(r)+\sum_{R,ij}\lvert\tilde{p}_{R,i}\rangle\Delta H_{R,ij}\langle\tilde{p}_{R,j}\rvert \tag*{(6)} H ~ = − 2 1 ∇ 2 + v ~ ( r ) + R , ij ∑ ∣ p ~ R , i ⟩ Δ H R , ij ⟨ p ~ R , j ∣ (6)
together with the potential v τ ( r ) v_{\tau}(r) v τ ( r ) of the kinetic-energy density for meta-GGA functionals. v ~ \tilde{v} v ~ is the plane-wave potential with one component for each spin density, and Δ H R \Delta H_R Δ H R is the one-center Hamiltonian of atom R R R . Apart from the kinetic energy of the auxiliary wave functions, the PAW energy depends on the state only through the auxiliary density, the kinetic-energy density and the one-center density matrices D R , i j D_{R,ij} D R , ij . Terms that depend only on D R D_R D R , such as a Hubbard correction in the form of Dudarev et al. S4,S5, are therefore contained in Δ H R \Delta H_R Δ H R .
Free energy as a function of the Hamiltonian
The parameters of the Hamiltonian are collected in a vector
θ = ( v ~ ( r ) , v τ ( r ) , { Δ H R , i j } ) . (7) \theta=\left(\tilde{v}(r),v_{\tau}(r),\{\Delta H_{R,ij}\}\right). \tag*{(7)} θ = ( v ~ ( r ) , v τ ( r ) , { Δ H R , ij } ) . (7)
The Hamiltonian is linear in θ \boldsymbol{\theta} θ , H ~ [ θ ] = T ^ + H ^ 0 + ∑ α θ α ρ ^ α \widetilde{H}[\boldsymbol{\theta}] = \hat{T} + \hat{H}_0 + \sum_\alpha\theta_\alpha\hat{\rho}_\alpha H [ θ ] = T ^ + H ^ 0 + ∑ α θ α ρ ^ α , where ρ ^ α \hat{\rho}_\alpha ρ ^ α are the operators whose expectation values yield the densities ρ \boldsymbol{\rho} ρ conjugate to θ \boldsymbol{\theta} θ , namely the auxiliary density, the kinetic-energy density and the one-center density matrices. T ^ \hat{T} T ^ is the kinetic energy of the auxiliary wave functions, and H ^ 0 \hat{H}_0 H ^ 0 contains terms that do not depend on the state, such as spin-orbit coupling.
Each trial Hamiltonian induces a state: its eigenstates ψ n ( θ ) \psi_n(\boldsymbol{\theta}) ψ n ( θ ) with the Fermi occupations f n ( θ ) f_n(\boldsymbol{\theta}) f n ( θ ) for a fixed number of electrons. We define the free energy as a function of the Hamiltonian by evaluating the Mermin functional for this state,
A ( θ ) = A [ { ψ n ( θ ) } , { f n ( θ ) } ] . (8) \mathcal{A}(\boldsymbol{\theta}) = \mathcal{A}\left[\{\psi_n(\boldsymbol{\theta})\}, \{f_n(\boldsymbol{\theta})\}\right]. \tag*{(8)} A ( θ ) = A [ { ψ n ( θ )} , { f n ( θ )} ] . (8)
The thermodynamic functional is unchanged. Only the space of the search has changed from states to Hamiltonians.
Two statements must be distinguished. (i) For the exact eigenstates of H ~ [ θ ] \widetilde{H}[\boldsymbol{\theta}] H [ θ ] , Eq. (8) defines a function of θ \boldsymbol{\theta} θ alone. It has the trace-log form derived below, and its gradient is given exactly by Eq. (2) of the main text. (ii) An iterative eigensolver returns approximate orbitals ψ ~ n \widetilde{\psi}_n ψ n and occupations f ~ n \widetilde{f}_n f n . The code evaluates the energy of this numerical trial state,
A ~ ( θ ; ψ ~ , f ~ ) = A [ { ψ ~ n } , { f ~ n } ] , (9) \widetilde{\mathcal{A}}(\boldsymbol{\theta};\widetilde{\psi},\widetilde{f}) = \mathcal{A}\left[\{\widetilde{\psi}_n\}, \{\widetilde{f}_n\}\right], \tag*{(9)} A ( θ ; ψ , f ) = A [ { ψ n } , { f n } ] , (9)
which is not exactly the function A ( θ ) \mathcal{A}(\boldsymbol{\theta}) A ( θ ) . As long as the orbitals are orthonormal and the occupations admissible, however, it is the energy of an admissible ensemble and therefore an upper bound of the minimum free energy, before the diagonalization is complete. The acceptance test of Supplementary Note 2.1 uses A ~ \widetilde{\mathcal{A}} A , while the gradient of Eq. (2) of the main text acquires an error of first order in the residuals of the eigenvectors (Supplementary Note 1.2). At self-consistency, θ \theta θ is equal to the parameters v o u t [ θ ] = ∂ E p o t / ∂ ρ v_{\mathrm{out}}[\theta] = \partial E_{\mathrm{pot}}/\partial\rho v out [ θ ] = ∂ E pot / ∂ ρ obtained from the induced state, and the free energy is stationary with respect to variations of θ \theta θ , under the assumptions (a) to (c) given below. This does not mean that the minimum A min A_{\min} A m i n is reached. Here E p o t E_{\mathrm{pot}} E pot is the part of the energy that depends on the densities, i.e. the Hartree energy, the compensation charges, exchange and correlation and the one-center terms.
Written explicitly, A A A has the trace-log form
A ( θ ) = Ω ( θ ) − θ ⋅ ρ ( θ ) + E p o t [ ρ ( θ ) ] (10) A(\theta) = \Omega(\theta) - \theta\cdot\rho(\theta) + E_{\mathrm{pot}}[\rho(\theta)] \tag*{(10)} A ( θ ) = Ω ( θ ) − θ ⋅ ρ ( θ ) + E pot [ ρ ( θ )] (10)
with the free energy of non-interacting electrons
Ω ( θ ) = − k B T ∑ n ln ( 1 + e − ( ϵ n ( θ ) − μ ) / k B T ) + μ N , (11) \Omega(\theta) = -k_{\mathrm{B}}T \sum_n \ln\left(1 + e^{-(\epsilon_n(\theta)-\mu)/k_{\mathrm{B}}T}\right) + \mu N, \tag*{(11)} Ω ( θ ) = − k B T n ∑ ln ( 1 + e − ( ϵ n ( θ ) − μ ) / k B T ) + μ N , (11)
where ϵ n ( θ ) \epsilon_n(\theta) ϵ n ( θ ) are the eigenvalues of H ~ [ θ ] \widetilde{H}[\theta] H [ θ ] relative to O ^ \widehat{O} O and the chemical potential μ \mu μ is fixed by the number of electrons. Ω \Omega Ω is the minimum of ∑ n f n ⟨ ψ n ∣ H ~ [ θ ] ∣ ψ n ⟩ − T S [ f ] \sum_n f_n\langle\psi_n|\widetilde{H}[\theta]|\psi_n\rangle- TS[f] ∑ n f n ⟨ ψ n ∣ H [ θ ] ∣ ψ n ⟩ − T S [ f ] over orthonormal orbitals and occupations (Supplementary Note 1.2). The Fermi occupations with a chemical potential that fixes the average electron number are a grand-canonical construction: Ω \Omega Ω is the grand potential of the noninteracting electrons plus μ N \mu N μ N , the free energy at fixed average N N N in the sense of Mermin, not the free energy of a strictly canonical ensemble.
Gradient, second derivative and gauge mode
Let ρ ^ α \hat{\rho}_{\alpha} ρ ^ α denote the operators whose expectation values yield the densities conjugate to θ \theta θ , that is ∣ r ⟩ ⟨ r ∣ |r\rangle\langle r| ∣ r ⟩ ⟨ r ∣ for the auxiliary density, the corresponding operator for the kinetic-energy density, and ∣ p ~ R , j ⟩ ⟨ p ~ R , i ∣ |\tilde{p}_{R,j}\rangle\langle\tilde{p}_{R,i}| ∣ p ~ R , j ⟩ ⟨ p ~ R , i ∣ for the one-center density matrices. The Hamiltonian is linear in θ \theta θ ,
H ~ [ θ ] = T ^ + H ^ 0 + ∑ α θ α ρ ^ α , (12) \widetilde{H}[\theta] = \widehat{T} + \widehat{H}_0 + \sum_{\alpha} \theta_{\alpha}\widehat{\rho}_{\alpha}, \tag*{(12)} H [ θ ] = T + H 0 + α ∑ θ α ρ α , (12)
where T ^ \widehat{T} T is the kinetic energy of the auxiliary wave functions and H ^ 0 \widehat{H}_0 H 0 contains terms that do not depend on the state, such as spin-orbit coupling. For an ensemble ( ψ , f ) (\psi, f) ( ψ , f ) the densities are
ρ α = ∑ n f n ⟨ ψ n ∣ ρ ^ α ∣ ψ n ⟩ , \rho_{\alpha} = \sum_n f_n \langle\psi_n | \widehat{\rho}_{\alpha} | \psi_n \rangle, ρ α = n ∑ f n ⟨ ψ n ∣ ρ α ∣ ψ n ⟩ ,
and the PAW free energy has the form
A [ ψ , f ] = ∑ n f n ⟨ ψ n ∣ T ^ + H ^ 0 ∣ ψ n ⟩ + E p o t [ ρ ] − T S [ f ] , (13) A[\psi, f] = \sum_n f_n \langle\psi_n | \widehat{T} + \widehat{H}_0 | \psi_n \rangle+ E_{\mathrm{pot}}[\rho] - T S[f], \tag*{(13)} A [ ψ , f ] = n ∑ f n ⟨ ψ n ∣ T + H 0 ∣ ψ n ⟩ + E pot [ ρ ] − T S [ f ] , (13)
where E p o t E_{\mathrm{pot}} E pot contains the Hartree energy, the compensation charges, exchange and correlation, and the one-center terms including a Hubbard correction. The output parameters are v o u t = ∂ E p o t / ∂ ρ v_{\mathrm{out}} = \partial E_{\mathrm{pot}}/\partial\rho v out = ∂ E pot / ∂ ρ .
We define the free energy of non-interacting electrons
Ω ( θ ) = min ψ , f [ ∑ n f n ⟨ ψ n ∣ H ~ [ θ ] ∣ ψ n ⟩ − T S [ f ] ] , (14) \Omega(\theta) = \min_{\psi,f} \left[ \sum_n f_n \langle\psi_n | \widetilde{H}[\theta] | \psi_n \rangle- T S[f] \right], \tag*{(14)} Ω ( θ ) = ψ , f min [ n ∑ f n ⟨ ψ n ∣ H [ θ ] ∣ ψ n ⟩ − T S [ f ] ] , (14)
where the minimum is taken over O ^ \widehat{O} O -orthonormal orbitals and occupations with ∑ n f n = N \sum_n f_n = N ∑ n f n = N . The minimum is attained for the eigenstates ψ n ( θ ) \psi_n(\theta) ψ n ( θ ) with Fermi occupations f n ( θ ) f_n(\theta) f n ( θ ) . Thus
A ( θ ) = Ω ( θ ) − θ ⋅ ρ ( θ ) + E p o t [ ρ ( θ ) ] . (15) A(\theta) = \Omega(\theta) - \theta\cdot\rho(\theta) + E_{\mathrm{pot}}[\rho(\theta)]. \tag*{(15)} A ( θ ) = Ω ( θ ) − θ ⋅ ρ ( θ ) + E pot [ ρ ( θ )] . (15)
The constraints do not depend on θ \theta θ , and the function to be minimized is linear in θ \theta θ . Therefore (i) Ω \Omega Ω is concave, and (ii) the Hellmann-Feynman theorem yields ∂ Ω / ∂ θ = ρ ( θ ) \partial\Omega/\partial\theta= \rho(\theta) ∂ Ω/ ∂ θ = ρ ( θ ) . The response function
χ = ∂ ρ / ∂ θ = ∂ 2 Ω / ∂ θ 2 \chi= \partial\rho/\partial\theta= \partial^2 \Omega/\partial\theta^2 χ = ∂ ρ / ∂ θ = ∂ 2 Ω/ ∂ θ 2
is therefore symmetric and negative semidefinite. It contains the change of the orbitals and, via the Fermi function, the change of the occupations for a fixed number of electrons. The derivative of Eq. (15) is
∂ A ∂ θ = ρ − ρ − χ θ + χ v o u t = χ ( v o u t − θ ) , (16) \frac{\partial A}{\partial\theta} = \rho- \rho- \chi\theta+ \chi v_{\mathrm{out}} = \chi(v_{\mathrm{out}} - \theta), \tag*{(16)} ∂ θ ∂ A = ρ − ρ − χ θ + χ v out = χ ( v out − θ ) , (16)
which is Eq. (2) of the main text. Only the stationarity of the eigenstates enters. For approximate eigenstates, A \mathcal{A} A remains an upper bound, and the gradient acquires an error of first order in the residuals of the eigenvectors.
Let s s s be the direction with ∑ α s α ρ ^ α = O ^ \sum_{\alpha} s_{\alpha}\hat{\rho}_{\alpha}=\hat{O} ∑ α s α ρ ^ α = O ^ , which consists of a unit constant on the grid, contributing 1 ^ \hat{1} 1 ^ , and of Δ O R \Delta O_{R} Δ O R in the one-center part, contributing O ^ − 1 ^ \hat{O}-\hat{1} O ^ − 1 ^ . Then H ~ [ θ + c s ] = H ~ [ θ ] + c O ^ \widetilde{H}[\theta+cs]=\widetilde{H}[\theta]+c\hat{O} H [ θ + cs ] = H [ θ ] + c O ^ . For O ^ \hat{O} O ^ -normalized orbitals this adds c c c to every expectation value. Hence Ω ( θ + c s ) = Ω ( θ ) + c N \Omega(\theta+cs)=\Omega(\theta)+cN Ω ( θ + cs ) = Ω ( θ ) + c N , the minimizing state is unchanged, χ s = 0 \chi s=0 χs = 0 , and s ⋅ ρ = N s\cdot\rho=N s ⋅ ρ = N . Inserting this into Eq. (15) yields
A ( θ + c s ) = A ( θ ) . \mathcal{A}(\theta+cs)=\mathcal{A}(\theta). A ( θ + cs ) = A ( θ ) .
Let θ ∗ \theta^{*} θ ∗ be the self-consistent point and θ = θ ∗ + δ \theta=\theta^{*}+\delta θ = θ ∗ + δ . To first order, v o u t ( θ ) = θ ∗ + K H x c χ δ v_{\mathrm{out}}(\theta)=\theta^{*}+K_{\mathrm{Hxc}}\chi\delta v out ( θ ) = θ ∗ + K Hxc χδ with the kernel K H x c = ∂ 2 E p o t / ∂ ρ 2 K_{\mathrm{Hxc}}=\partial^{2}E_{\mathrm{pot}}/\partial\rho^{2} K Hxc = ∂ 2 E pot / ∂ ρ 2 , so that r = − ϵ δ r=-\epsilon\delta r = − ϵδ with ϵ = 1 − K H x c χ \epsilon=1-K_{\mathrm{Hxc}}\chi ϵ = 1 − K Hxc χ , and
A ( θ ∗ + δ ) − A ( θ ∗ ) = 1 2 δ ⋅ ( − χ + χ K H x c χ ) δ + O ( δ 3 ) . (17) \mathcal{A}(\theta^{*}+\delta)-\mathcal{A}(\theta^{*})=\frac{1}{2}\delta\cdot\left(-\chi+\chi K_{\mathrm{Hxc}}\chi\right)\delta+O(\delta^{3}). \tag*{(17)} A ( θ ∗ + δ ) − A ( θ ∗ ) = 2 1 δ ⋅ ( − χ + χ K Hxc χ ) δ + O ( δ 3 ) . (17)
The second derivative − χ ϵ = − χ + χ K H x c χ -\chi\epsilon=-\chi+\chi K_{\mathrm{Hxc}}\chi − χϵ = − χ + χ K Hxc χ is symmetric and, at a stable minimum, positive semidefinite. The Newton step is − δ = ϵ − 1 r -\delta=\epsilon^{-1}r − δ = ϵ − 1 r , so that the ideal preconditioner for the residual is the inverse dielectric operator, just as for potential mixing. For the Thomas-Fermi model of the long-wavelength response, ϵ − 1 ( G ) = G 2 / ( G 2 + q 0 2 ) \epsilon^{-1}(G)=G^{2}/(G^{2}+q_{0}^{2}) ϵ − 1 ( G ) = G 2 / ( G 2 + q 0 2 ) , which is the Kerker filter. The error of A \mathcal{A} A is quadratic in δ \delta δ and not negative, whereas the Harris-Foulkes energy of density mixing is quadratic in the error of the density, but can have either sign. Proposition and assumptions. Assume that (a) the orbitals are the exact O ^ \hat{O} O ^ -orthonormal eigenstates of H ~ [ θ ] \widetilde{H}[\theta] H [ θ ] in the finite basis, with the eigenvalues defined relative to O ^ \hat{O} O ^ , (b) the occupations are Fermi functions at T > 0 T>0 T > 0 with μ \mu μ fixed by N N N , or the system has a finite gap at T → 0 T \to0 T → 0 , and (c) E p o t E_{\mathrm{pot}} E pot depends on the state only through the densities ρ \rho ρ conjugate to θ \boldsymbol{\theta} θ . Then (i) A ( θ ) ≥ A min A(\boldsymbol{\theta}) \ge A_{\min} A ( θ ) ≥ A m i n , (ii) ∂ A / ∂ θ = χ ( v o u t − θ ) \partial A/\partial\boldsymbol{\theta} = \boldsymbol{\chi}(\mathbf{v}_{\mathrm{out}}-\boldsymbol{\theta}) ∂ A / ∂ θ = χ ( v out − θ ) , (iii) χ \boldsymbol{\chi} χ is symmetric and negative semidefinite, and (iv) r T χ r ≤ 0 \mathbf{r}^{T}\boldsymbol{\chi}\mathbf{r} \le0 r T χ r ≤ 0 for r = v o u t − θ \mathbf{r} = \mathbf{v}_{\mathrm{out}}-\boldsymbol{\theta} r = v out − θ , with equality if and only if χ r = 0 \boldsymbol{\chi}\mathbf{r} = 0 χ r = 0 . Statement (i) needs only an admissible ensemble and holds without (a) and (c), provided the orbitals are O ^ \hat{O} O ^ -orthonormal. Statement (iii) follows because Ω \Omega Ω is a minimum of functions linear in θ \boldsymbol{\theta} θ . For (iv), write − χ = B T B -\boldsymbol{\chi} = B^{T}B − χ = B T B . Then r T χ r = − ∣ B r ∣ 2 \mathbf{r}^{T}\boldsymbol{\chi}\mathbf{r} = -|B\mathbf{r}|^{2} r T χ r = − ∣ B r ∣ 2 vanishes only if B r = 0 B\mathbf{r} = 0 B r = 0 , and therefore χ r = 0 \boldsymbol{\chi}\mathbf{r} = 0 χ r = 0 , so that the whole gradient vanishes. With R ^ = ∑ α r α ρ ^ α \hat{R} = \sum_{\alpha}r_{\alpha}\hat{\rho}_{\alpha} R ^ = ∑ α r α ρ ^ α , this means that R ^ \hat{R} R ^ has no matrix elements between states of different occupation and acts as a constant on the partially occupied states. Such a residual exerts no force on orbitals or occupations, and a vanishing directional derivative along r \mathbf{r} r therefore does establish stationarity in the space of Hamiltonians under (a) to (c). At T > 0 T > 0 T > 0 the function Ω \Omega Ω is a smooth symmetric function of the eigenvalues, so degeneracies cause no difficulty. If (a) fails, the gradient acquires the error of first order in the eigenvector residuals given above. Condition (c) holds for semilocal functionals, meta-GGAs and DFT+U U U . It fails for the onsite Fock-type terms of the local hybrids, where the variables are not complete (Supplementary Note 2.3), and for plane-wave Fock exchange. The descent probe does not require (a) or (c) when it is carried out by direct minimization, as in VASP: it relies only on (i). C1 does require that both energies are evaluated with the same implemented functional and numerical settings.
Gauge redundancy
Different parameters θ \boldsymbol{\theta} θ can describe physically equivalent Hamiltonians. A uniform shift H ~ → H ~ + c O ^ \widetilde{H} \to\widetilde{H} + c\hat{O} H → H + c O ^ of the Hamiltonian shifts all eigenvalues and the chemical potential by c c c , while the orbitals, and for a fixed number of electrons also the occupations, remain unchanged.
In the space of parameters, this shift is the direction
s = ( 1 , 0 , { Δ O R , i j } ) (18) s = \left(1,\,0,\,\{\Delta O_{R,ij}\}\right) \tag*{(18)} s = ( 1 , 0 , { Δ O R , ij } ) (18)
that is, a constant on the plane-wave grid together with the one-center overlap Δ O R \Delta O_R Δ O R . Thus A ( θ + c s ) = A ( θ ) \mathcal{A}(\theta+ c\,s) = \mathcal{A}(\theta) A ( θ + c s ) = A ( θ ) exactly, and χ s = 0 \chi s = 0 χs = 0 . In a norm-conserving formulation the gauge mode is the constant potential alone, which the Kerker filter removes automatically. In the PAW method the same constant-potential gauge acquires a nontrivial representation, shared between the plane-wave and the one-center sectors.
The Kerker filter removes the grid component of s s s , while the one-center part is updated by a different rule. If s s s is not removed, the average of the grid potential and the part of Δ H \Delta H Δ H proportional to Δ O \Delta O Δ O therefore drift apart, and the mismatch acts like a spurious step of the potential at the surface of the augmentation spheres. For bulk Si the iteration then freezes with a finite gauge offset and an energy above the minimum [Figure 3 (b) and Table 6]. In an earlier version of the code the unprojected iteration even converged to an excited state with a negative band gap. We therefore project s s s out of every residual. Keeping the G = 0 G = 0 G = 0 component of the grid residual unfiltered also moves both parts of the shift together, but the projection is the general remedy. The physical search space is thus the quotient H P A W / { c O ^ } \mathcal{H}_{\mathrm{PAW}}/\{c\hat{O}\} H PAW / { c O ^ } of the parameter space by the null direction generated by the overlap operator. The minimization is naturally formulated on this quotient, and the projection of s s s out of every residual is its numerical realization. A fixed gauge convention, for example a vanishing average of the grid potential, would have to constrain the one-center sector consistently as well, which amounts to the same projection. For spin-polarized calculations the shift acts only on the total potential. The G = 0 G = 0 G = 0 component of the spin potential is not a gauge and is not filtered.
Minimization algorithm {#sec:Supplementary Note 2}
Line search and step length
The steps are proposed by an Anderson extrapolation S6 from the accepted points ( θ k , r k ) (\theta_k, r_k) ( θ k , r k ) , which is applied to the preconditioned grid residual with a damping factor β \beta β ,
θ ′ = θ 1 + β P r 1 − ∑ k ≥ 2 γ k [ ( θ 1 − θ k ) + β P ( r 1 − r k ) ] . (19) \theta' = \theta_1 + \beta P r_1 - \sum_{k \ge2} \gamma_k \left[(\theta_1-\theta_k) + \beta P(r_1-r_k)\right]. \tag*{(19)} θ ′ = θ 1 + β P r 1 − k ≥ 2 ∑ γ k [ ( θ 1 − θ k ) + β P ( r 1 − r k ) ] . (19)
The coefficients γ k \gamma_k γ k minimize the norm of the extrapolated residual, and P P P is the preconditioner. If the extrapolated step exceeds the plain damped step by more than a factor of ten, the plain step β P r 1 \beta P r_1 β P r 1 is used instead. A proposal θ ′ \theta' θ ′ is accepted if A ( θ ′ ) ≤ A b e s t + ϵ \mathcal{A}(\theta') \le\mathcal{A}_{\mathrm{best}}+\epsilon A ( θ ′ ) ≤ A best + ϵ . Otherwise the step is halved towards the best point. The value A b e s t \mathcal{A}_{\mathrm{best}} A best may itself result from eigenstates that were not fully converged. Therefore, the best point is evaluated again after five consecutive rejections. The procedure is summarized in Figure 2 .
θ 0 ← v o u t [ ψ s t a r t ] \theta_0 \leftarrow v_{\mathrm{out}}[\psi_{\mathrm{start}}] θ 0 ← v out [ ψ start ] and keep θ 0 \theta_0 θ 0 fixed for a few warm-up steps of the eigensolver
loop
determine the eigenstates of H ~ [ θ ] \widetilde{H}[\theta] H [ θ ] (Davidson, RMM-DIIS or CG) and the Fermi occupations for fixed N N N
evaluate A ( θ ) \mathcal{A}(\theta) A ( θ ) for the output state and construct v o u t v_{\mathrm{out}} v out
r ← v o u t − θ r \leftarrow v_{\mathrm{out}}-\theta r ← v out − θ and project out the gauge mode s s s
if A ( θ ) ≤ A b e s t + ϵ \mathcal{A}(\theta) \le\mathcal{A}_{\mathrm{best}}+\epsilon A ( θ ) ≤ A best + ϵ then
accept, add ( θ , P r ) (\theta,Pr) ( θ , P r ) to the history and adapt β \beta β
if converged then stop
θ ← \theta\leftarrow θ ← Anderson proposal from the accepted points
else
θ ← 1 2 ( θ + θ b e s t ) \theta\leftarrow\frac{1}{2}(\theta+\theta_{\mathrm{best}}) θ ← 2 1 ( θ + θ best ) ; re-evaluate θ b e s t \theta_{\mathrm{best}} θ best after 5 rejections
end if
end loop
Supplementary Figure 2. Hamiltonian minimization (HM).
The damping factor β ∈ [ 0.02 , 0.5 ] \beta\in[0.02, 0.5] β ∈ [ 0.02 , 0.5 ] is adapted during the iteration. Halving β \beta β at every rejection is not a good choice, because each backtracking step already shortens the trial step, so that β \beta β collapses to its minimum within one sequence of rejections. We halve β \beta β only once per sequence, at the second consecutive rejection, and discard the Anderson history at the same time. After an accepted step β \beta β is increased by a factor 1.2, and by a factor 1.5 after three consecutive accepted steps. A faster increase directly after a rejection produced large trial steps, which drove the eigensolver into an excited state. A second rule suppresses a cycle of period two, in which the full Anderson step is rejected and the halved step accepted, so that β \beta β is never reduced: β \beta β is also halved when a rejection follows directly on an accepted backtracking step. The effect of these rules is quantified in Table 2 .
The slack ϵ = 10 − 7 \epsilon= 10^{-7} ϵ = 1 0 − 7 Ha absorbs the noise of the iterative eigensolvers. It is measured from the last accepted energy, so that the accepted energies never rise by more than ϵ \epsilon ϵ , but they can rise by less. Measuring it from the lowest accepted energy removes the possible drift of a stalled iteration, but a single evaluation that happens to fall low then blocks all later steps of a noisy calculation, as we observed for a water molecule in vacuum.
The calculation is converged when the mean-square residual of the grid potential is below a threshold and the largest residual of the one-center Hamiltonian is below its square root, or after two consecutive accepted full-length steps whose energy gain is below the energy tolerance. The grid residual alone is not sufficient, because the one-center part can lag behind. A small energy gain after a backtracking step only reflects a short step and is not counted.
Cells with vacuum
The projection of the gauge mode keeps the uniform offset of θ \theta θ at its starting value. In a slab or a molecule with a large vacuum region, the average of v o u t v_{\mathrm{out}} v out changes by tens of eV during the iteration, because the charge distribution and the dipole change. The energy is not affected, but the preconditioned eigensolvers stall, because they assume eigenvalues on the scale of the kinetic energy (Table 6). Two measures remove this problem. (i) Whenever the gauge offset between θ \theta θ and v o u t v_{\mathrm{out}} v out exceeds 0.5 Ha, θ \theta θ and the Anderson history are shifted along s s s , which leaves A \mathcal{A} A unchanged. A realignment in every step disturbed the eigensolver, which starts from the states of the previous step. (ii) Where the density is below 10 − 6 a B − 3 10^{-6}\,a_{\mathrm{B}}^{-3} 1 0 − 6 a B − 3 , θ \theta θ is set equal to v o u t v_{\mathrm{out}} v out . In empty space the response function vanishes, the energy contains no information about θ \theta θ , and the Anderson extrapolation would let θ \theta θ drift.
Orbital-dependent operators
In the local hybrid functionals PBE0r and PBE-SL0r S7–S9, the Kohn-Sham wave functions are mapped onto local orbitals, and only the onsite terms of the exchange correction are included S7, S10. PBE-SL0r uses PBE, but replaces the onsite PBE exchange of the local orbitals by deorbitalized SCAN (SCAN-L) exchange S11 before a fraction of it is replaced by exact exchange. The fraction of exact exchange controls the localization of holes and polarons in such systems S12, S13. This energy depends on the orbitals through their projection onto local orbitals, and not only on the densities conjugate to θ \theta θ . We therefore apply the Fock-type correction as an operator constructed from the current state. A line search with respect to its matrix elements left a residual that did not decay. The energy of the induced state remains an upper bound, but the variables of the minimization are then not complete, and convergence is not guaranteed. A Hubbard correction depends only on D R D_R D R and is part of Δ H R \Delta H_R Δ H R , and spin-orbit coupling is a fixed one-center operator.
Direct minimization (DM) with respect to the orbitals and the occupations serves as a second variational reference and is described in Supplementary Note 3.
Direct minimization (DM)
As an alternative, A \mathcal{A} A is minimized directly with respect to the orbitals and an auxiliary Hamiltonian H a u x H^{\mathrm{aux}} H aux , whose eigenvalues determine the occupations, as in ensemble density-functional theory S14,S15.
The orbitals follow a preconditioned S16 Polak-Ribière conjugate-gradient direction, and H a u x H^{\mathrm{aux}} H aux the direction H p h y s − H a u x H^{\mathrm{phys}} - H^{\mathrm{aux}} H phys − H aux , which is downhill because the divided differences of the Fermi function are not positive. The same rules for acceptance and backtracking are used as in HM. Because long-wavelength changes of the density are poorly conditioned in the space of orbitals, the orbital direction is computed from the Kerker-filtered Hamiltonian θ = θ c + K ( v o u t − θ c ) \theta= \theta_{\mathrm{c}} + K(v_{\mathrm{out}} - \theta_{\mathrm{c}}) θ = θ c + K ( v out − θ c ) , where θ c \theta_{\mathrm{c}} θ c is moved only after accepted steps. The energy test uses the true functional, so that the filter changes the path but not the minimum.
Implementation and computational details
Both HM and DM have been implemented in the PAW code VESPAW, which has been derived from CP-PAW S17, the code of our earlier work on transition-metal oxides and their interfaces S7,S18,S19. HM is selected by MIXER=’VARIATIONAL’ together with one of the eigensolvers OPTIMIZER=’DAV’, ’RMM’ or ’CG’, and DM by OPTIMIZER=’EDFT’. HM is the default.
The method requires an eigensolver that returns orthonormal eigenstates. We therefore rewrote the construction of the subspace in the block-Davidson method S20, S21. The correction vectors are orthogonalized twice to all current bands, vectors lost by cancellation are dropped, and the extended basis is orthogonalized canonically with an eigenvalue cutoff of 10 − 10 10^{-10} 1 0 − 10 . The previous implementation orthogonalized only to the converged bands and stabilized the nearly singular overlap matrix by a shift, which produced non-orthonormal orbitals. RMM-DIIS S21, S22 is preceded by eight Davidson iterations, because from random orbitals it converged to wrong states, the known failure of residual minimization. The Davidson eigensolver handles two-component spinors, whereas RMM-DIIS, CG and DM are implemented for collinear spin only.
With space-group symmetry only the irreducible k k k -points are treated, and the output density, the one-center density matrices, the forces and the stress are symmetrized. Then every residual and every Anderson combination of residuals is symmetric, so that θ \theta θ remains in the symmetric subspace. At time-reversal-invariant k k k -points the eigensolvers use real super wave functions, which combine two bands. For parallel calculations the k k k -points are distributed over groups of MPI tasks, and within a group the plane-wave coefficients and the real-space grid. The optimizer works on the distributed grid, the one-center parameters are replicated, and all scalar products of the energy test and the Anderson fit are summed over the tasks, so that every task takes the same decisions.
The calculations use the PBE functional S23 unless stated otherwise, scalar-relativistic setups without ZORA corrections, and a cutoff for the auxiliary density of twice that of the wave functions. The partial waves for Si, Al, Mg, O and H have been constructed as nodeless partial waves S24. For Ni we used two projectors per angular momentum s s s , p p p and d d d . The systems, cutoffs and k-point meshes are listed in Table 1. NiO is antiferromagnetic (AFM-II) in a rhombohedral cell with four atoms. PBE+U U U uses U − J = 6.3 U-J=6.3 U − J = 6.3 eV on the Ni-d shell S5, and the local hybrid functionals use an admixture of 25%. For the meta-GGA, the PAW setups are constructed with PBE, and the kinetic-energy density includes the pseudized core contribution in both the plane-wave and the one-center parts. The polar water layer is placed in an 18 Å cell with the dipole correction S25.
Two materials of energy technology serve as demanding test cases. The spinel L i M n 2 O 4 \mathrm{LiMn_2O_4} LiM n 2 O 4 is a cathode material of Li-ion batteries S26, and spinel oxides and chalcogenides are also studied as cathodes and solid electrolytes of Mg batteries S27–S32. In the cubic cell of L i M n 2 O 4 \mathrm{LiMn_2O_4} LiM n 2 O 4 with 56 atoms, half of the 16 Mn ions are Jahn-Teller active M n 3 + \mathrm{Mn^{3+}} M n 3 + and half are M n 4 + \mathrm{Mn^{4+}} M n 4 + . We use the Jahn-Teller distorted structure obtained from a PBE+U U U relaxation, the Γ \Gamma Γ point and 50 Ry. The antiferromagnetic spin order is prepared with an orbital potential, whereas the charge order of M n 3 + \mathrm{Mn^{3+}} M n 3 + and M n 4 + \mathrm{Mn^{4+}} M n 4 + forms without it on the distorted sites. C a M n O 3 \mathrm{CaMnO_3} CaMn O 3 is the electron-poor end member of the manganite series P r 1 − x C a x M n O 3 \mathrm{Pr_{1-x}Ca_xMnO_3} P r 1 − x C a x Mn O 3 , in which the local hybrid functional PBE0r has been parameterized S7, and its oxygen-deficient form is a photocatalyst S33. We remove one oxygen atom from the orthorhombic cell with 20 atoms (C a M n O 2.75 \mathrm{CaMnO_{2.75}} CaMn O 2.75 ) and use 2 × 2 × 2 2 \times2 \times2 2 × 2 × 2 k-points. The two electrons left by the vacancy localize as M n 3 + \mathrm{Mn^{3+}} M n 3 + polarons in the G-type antiferromagnetic host, so that several self-consistent solutions compete. We use 40 Ry. Both materials are calculated with PBE0r and species-dependent exact-exchange fractions, those of Ref. S26 for L i M n 2 O 4 \mathrm{LiMn_2O_4} LiM n 2 O 4 and those of Ref. S7 for C a M n O 2.75 \mathrm{CaMnO_{2.75}} CaMn O 2.75 . The magnetic order is prepared by 300 steps of friction dynamics with an orbital potential, as for NiO, and all optimizers start from this state.
The reference is Car–Parrinello friction dynamics (OPTIMIZER='MD') [34 ] , which minimizes the same functional directly. The spin order of NiO is prepared by a short friction run with an orbital potential, which is switched off afterwards, and all optimizers start from this state. All energies are taken from the regression suite of VESPAW, and the results for HM and friction dynamics have been reproduced with four MPI tasks.
Validation of Hamiltonian minimization
Accuracy
Table 1 compares the converged free energies of all optimizers. The three eigensolvers of HM agree within 4.5 μ e V 4.5\,\mu\mathrm{eV} 4.5 μ eV , except for NiO with PBE, where RMM-DIIS and CG stopped 12 and 22 μ e V 22\,\mu\mathrm{eV} 22 μ eV above Davidson, for MgO, where the Davidson calculation stopped 21 μ e V 21\,\mu\mathrm{eV} 21 μ eV above RMM-DIIS and CG, and for the water layer discussed below. Friction dynamics agrees within 3 μ e V 3\,\mu\mathrm{eV} 3 μ eV for NiO with all four functionals. Where the deviation is larger and positive, as for friction dynamics in Si and Al and for DM, the calculation stopped on its own energy criterion while still approaching the minimum from above. For DM this deviation reaches 0.4 m e V 0.4\,\mathrm{meV} 0.4 meV for NiO and 1.2 m e V 1.2\,\mathrm{meV} 1.2 meV for MgO with the local hybrid, in line with its slow convergence for these systems. The variational principle guarantees that such an error has this sign. The band gaps of the different optimizers agree within 1 m e V 1\,\mathrm{meV} 1 meV , for NiO with PBE within 5 m e V 5\,\mathrm{meV} 5 meV . Without spin-orbit coupling, the calculation with two-component spinors reproduces the collinear energy of Si within 0.3 μ e V 0.3\,\mu\mathrm{eV} 0.3 μ eV . The water layer is the only case in which HM stops noticeably above friction dynamics: by 0.7 m e V 0.7\,\mathrm{meV} 0.7 meV with Davidson and by a further 5.5 5.5 5.5 and 6.0 m e V 6.0\,\mathrm{meV} 6.0 meV with RMM-DIIS and CG, while DM agrees with friction dynamics within 16 μ e V 16\,\mu\mathrm{eV} 16 μ eV . The gradient-corrected exchange-correlation potential in the tail of the density leaves a residual in the vacuum, and HM stops on the change of the energy before this residual has decreased.
Table 1. Table 1: Tests in two codes, grouped by outcome. C1: Δ A C 1 \Delta A_{\mathrm{C1}} Δ A C1 , the change of the free energy of the returned state when it is restarted with a variational optimizer (HM in VESPAW, ALGO=All in VASP). C2: Δ n \Delta n Δ n of Eq. (4) for the lowest unoccupied (L) and highest occupied (H) state. All calculations report convergence. Bold: failed (C1: decrease > 10 δ >10\delta > 10 δ , with δ \delta δ the requested energy tolerance per cell, and C2: ∣ Δ n ∣ > 5 |\Delta n|>5 ∣Δ n ∣ > 5 meV). Energies in meV.
Supplementary Figure 3. (a) Error of the energy during the iteration for a Si supercell with 64 atoms (Γ \Gamma Γ point, T = 300 T = 300 T = 300 K), relative to the lowest energy reached by HM: HM with three eigensolvers, DM, Pulay mixing with the Davidson eigensolver (energy of the mixed density) and Car-Parrinello friction dynamics. The shaded band marks 30 µeV. (b) The PAW gauge mode for bulk Si (6 × 6 × 6 6 \times6 \times6 6 × 6 × 6 k-points): HM with the Davidson eigensolver with and without projecting the gauge direction s s s out of the residuals.
Control of the line search
Fig. 4 shows the number of trial Hamiltonians and of rejected steps. For most systems no step is rejected: near the minimum the Anderson extrapolation of a well-preconditioned residual is a good step, and the energy test only confirms it. Rejections occur where the energy is strongly anharmonic on the scale of one step, for localized d states, whose one-center Hamiltonian responds strongly, and in vacuum, where the grid potential is only weakly coupled to the energy. For these cases Table 2 shows the effect of the rules of Supplementary Note 2.1. Without the rule against the cycle of period two, the Davidson calculation of NiO requires ten times more trial Hamiltonians, half of which are rejected. Halving β \beta β at every rejection costs about 30% more. For fcc Al no step is rejected with any of the rules, also on a 12 3 12^3 1 2 3 mesh without symmetry.
The last row of Table 2 uses the acceptance rule of Herbst and Levitt S35, in which a trial Hamiltonian is also accepted when the energy rises but the residual decreases. All other parts of HM are unchanged, so that this tests the acceptance rule and not their complete algorithm, which chooses the damping from a quadratic model of the energy. For all systems of Table 1 except NiO with PBE and the water layer, and for Si64 _{64} 64 , the iterations are identical with both rules. For NiO the additional acceptances lead to more trial Hamiltonians. For the water layer the rule based on the residual accepts an early step that raises the energy by 6.5 eV, and the calculation stops 6.0 meV above the result of the minimization with the energy test. The energy test prevents the iteration from leaving the region in which the quadratic description of Supplementary Note 1 applies.
Supplementary Figure 4. (a) Trial Hamiltonians of HM with the Davidson eigensolver, accepted (blue) and rejected (red), without the warm-up steps at fixed Hamiltonian. (b) Deviation of the converged energy from HM with Davidson: the other HM eigensolvers (largest of RMM-DIIS and CG, diamonds), friction dynamics (circles, for Al with symmetry HM on the full k-point mesh) and DM (squares). The dashed line marks 30 μ e V \mu\mathrm{eV} μ eV .
Supplementary Table 2. Number of trial Hamiltonians (rejected steps in parentheses) of HM for different rules of the line search. Present: rules of Supplementary Note 2.1. Energy or residual: a step is also accepted if the residual decreases, as in Ref. S35.
All NiO energies agree within 8 μ e V \mu\mathrm{eV} μ eV . a ^{a} a 96 (25) in the calculation of Fig. 4, because the eigensolver noise in the vacuum changes the sequence of rejections. b ^{b} b Stops 6.0 meV above the present rule.
Materials for energy storage and conversion
Table 3 compares the optimizers for L i M n 2 O 4 \mathrm{LiMn}_{2}\mathrm{O}_{4} LiMn 2 O 4 and C a M n O 2.75 \mathrm{CaMnO}_{2.75} CaMnO 2.75 , all started from the same prepared state. The three variants of HM and friction dynamics agree within 0.18 meV for L i M n 2 O 4 \mathrm{LiMn}_{2}\mathrm{O}_{4} LiMn 2 O 4 , that is 3 μ \mu μ eV per atom, and within 0.05 meV for C a M n O 2.75 \mathrm{CaMnO}_{2.75} CaMnO 2.75 . The band gaps agree within 1 meV (1.023 to 1.024 eV and 0.576 to 0.577 eV), and all calculations end in the same arrangement of M n 3 + \mathrm{Mn}^{3+} Mn 3 + and M n 4 + \mathrm{Mn}^{4+} Mn 4 + ions and of the polarons. Friction dynamics stops above HM, as the variational principle requires.
Supplementary Table 3: L i M n 2 O 4 \mathrm{LiMn}_{2}\mathrm{O}_{4} LiMn 2 O 4 (56 atoms, Γ \Gamma Γ , 50 Ry) and C a M n O 2.75 \mathrm{CaMnO}_{2.75} CaMnO 2.75 (19 atoms, 2 3 2^{3} 2 3 k-points, 40 Ry) with PBE0r, started from the same prepared state. N N N : trial Hamiltonians (rejected) for HM, iterations otherwise. Δ E \Delta E Δ E : final energy relative to HM with Davidson. t t t : wall time on one Apple M3 with 8 threads.
Table 3 (PDF p. 57). Supplementary Table 3: L i M n 2 O 4 \mathrm{LiMn}_{2}\mathrm{O}_{4} LiMn 2 O 4 (56 atoms, Γ \Gamma Γ , 50 Ry) and C a M n O 2.75 \mathrm{CaMnO}_{2.75} CaMnO 2.75 (19 atoms, 2 3 2^{3} 2 3 k-points, 40 Ry) with PBE0r, started from the same prepared state. N N N : trial Hamiltonians (rejected) for HM, iterations otherwise. Δ E \Delta E Δ E : final energy relative to HM with Davidson. t t t : wall time on one Apple M3 with 8 threads.
Rejected steps are more frequent than for the systems of Table 1: 22 of 64 trial Hamiltonians for L i M n 2 O 4 \mathrm{LiMn}_{2}\mathrm{O}_{4} LiMn 2 O 4 with Davidson. We attribute this to the strongly anharmonic dependence of the energy on the one-center Hamiltonians of the Mn d shells, whose occupations decide between M n 3 + \mathrm{Mn}^{3+} Mn 3 + and M n 4 + \mathrm{Mn}^{4+} Mn 4 + . With RMM-DIIS, HM requires about half as many trial Hamiltonians and a third of the time of Davidson. For L i M n 2 O 4 \mathrm{LiMn}_{2}\mathrm{O}_{4} LiMn 2 O 4 , the acceptance rule based on the residual reduces the trial Hamiltonians of the Davidson calculation from 64 to 43 at the same energy, whereas for C a M n O 2.75 \mathrm{CaMnO}_{2.75} CaMnO 2.75 it makes no difference.
Pulay mixing reaches the same states within 0.03 meV, once its residual and the change of its energy are both small. It needs 61 and 53 iterations, more than HM with RMM-DIIS and three times its time. In an earlier version of this work, Pulay mixing stopped here on the change of the energy alone, with a density residual frozen at 2.5 × 10 − 4 2.5 \times10^{-4} 2.5 × 1 0 − 4 , 0.35 meV below HM for LiMn2 _2 2 O4 _4 4 . That residual was a floor caused by an error of our implementation, not a property of Pulay mixing (Supplementary Note 7). Friction dynamics is the fastest optimizer here, because it starts from the prepared state, which is itself the result of friction dynamics, and each of its steps is more than ten times cheaper than a Davidson iteration. From random orbitals, it needs twenty times more iterations than HM and three to five times its time (Table 4).
Symmetry
The calculations with irreducible k k k -points agree with those on the full mesh within 10 − 8 10^{-8} 1 0 − 8 eV for Al with 48 symmetry operations, and within 6 × 10 − 8 6 \times10^{-8} 6 × 1 0 − 8 eV for a Si cell with a displaced atom, which has 8 operations including non-lattice translations. The agreement extends to forces and stress. The number of iterations, however, can be larger with symmetry (Supplementary Note 5.6).
Performance
Table 4 compares the optimizers for a Si supercell with 64 atoms. All three eigensolvers of HM are within 30 μ \mu μ eV of the minimum after 18 iterations [Figure 3 (a)]. With RMM-DIIS and with Davidson, HM is faster than Pulay mixing for this cell, which ends 0.09 meV above the minimum. Before the residual floor of Supplementary Note 7 was removed, Pulay mixing stopped on the change of the energy after the same number of iterations, 1.4 meV above the minimum and in less time, because the frozen residual kept the tolerance of the eigensolver loose. DM and friction dynamics require an order of magnitude more iterations. Friction dynamics, although variational, approaches the minimum slowly, and DM is limited by the conditioning in the space of orbitals.
The cost per iteration is dominated by the eigensolver. Table 5 gives the time per Davidson iteration for 8, 64 and 216 atoms and the speed-up on several MPI tasks. For Si216 _{216} 216 , 71% of the time is spent in scalar products between plane-wave vectors and in the application of the projector functions, which are the obvious targets for further optimization.
From random orbitals, HM does not converge for Si216 _{216} 216 : every step of the line search raises the energy. The line search starts from the Hamiltonian of the random orbitals, and for Si216 _{216} 216 this Hamiltonian is far from the minimum (mean-square residual 1.05 H2 ^2 2 compared with 0.33 H2 ^2 2 for Si64 _{64} 64 ), and a longer warm-up at fixed Hamiltonian does not help. Thirty steps of friction dynamics provide a better start. From there, HM with RMM-DIIS (three Davidson iterations first) accepts every trial Hamiltonian and converges in 19 iterations to a residual of 5 × 10 − 9 5 \times10^{-9} 5 × 1 0 − 9 H2 ^2 2 . On a 12-core AMD EPYC 4464P with 24 threads, the friction steps take 11 min and HM takes 48 min, with 120 s per RMM-DIIS iteration compared with about 330 s per Davidson iteration. With Davidson, HM comes within 1 meV of its final energy in 12 iterations, but then stalls 2.8 meV above the minimum reached with RMM-DIIS, at a residual of 1.6 × 10 − 6 1.6 \times10^{-6} 1.6 × 1 0 − 6 H2 ^2 2 (Supplementary Note 5.6).
Supplementary Table 4. Si supercell with 64 atoms, Γ \Gamma Γ point, 25 Ry, T = 300 T = 300 T = 300 K, one task with 8 OpenMP threads. Δ E \Delta E Δ E : final energy relative to the lowest energy of HM. a ^{a} a The first 8 iterations with Davidson. b ^{b} b Energy of the mixed density, not an upper bound.
Supplementary Table 5. Supplementary Table 5: Time per iteration of HM with the Davidson eigensolver for Si supercells (Γ \Gamma Γ point, 25 Ry, T = 300 T = 300 T = 300 K). N b N_b N b : bands. N P W N_{\mathrm{PW}} N PW : plane waves of the wave functions. Upper part: one task with 8 OpenMP threads. Lower part: Si64 _{64} 64 on n n n MPI tasks with one thread each.
Limitations
The minimization requires reasonably converged eigenstates for every trial point, and for RMM-DIIS a few Davidson iterations at the start. For large cells the Hamiltonian of random orbitals is too poor a starting point, and a short run of friction dynamics is needed first, as for Si216 _{216} 216 .
In that cell, HM with Davidson stalled 2.8 meV above the minimum at a residual of 1.6 × 10 − 6 1.6 \times10^{-6} 1.6 × 1 0 − 6 H2 ^2 2 ; there, steps along the residual raise the energy, and repeated rejections reduce the step length to its lower bound. A tighter tolerance of the eigensolver did not remove the stall, and RMM-DIIS from the same start does not show it. With symmetry, the Anderson extrapolation can point uphill after accepted steps: Al with 12 3 12^3 1 2 3 k-points requires 69 iterations (24 rejected) with symmetry instead of 13 on the full mesh, although both reach the same energy within 10 − 6 10^{-6} 1 0 − 6 eV. Orbital-dependent operators, such as those of the local hybrid functionals, are not part of θ \theta θ . In cells with much vacuum the residual of the exchange-correlation potential in the tail of the density limits the accuracy, for the water layer to 6 meV with RMM-DIIS and CG. For spinors only the Davidson eigensolver is available. DM is slow for antiferromagnetic NiO and for large supercells.
Errors revealed by the variational bound
During this work the variational formulation revealed several errors in our implementation of density mixing and of the Davidson eigensolver, which are listed in Table 6. They are errors of an implementation and not intrinsic failures of density mixing, which converges correctly once they are removed. They are reported because each of them produced a result that appeared converged, and because they remained undetected as long as only mixing schemes were used. The most serious one, a Kerker filter that also removed the G = 0 G = 0 G = 0 component of the residual of the auxiliary density, left Si 2.46 eV above the variational minimum with a frozen residual [Fig. 5(a)], and this value had been used as a reference for some time. A variational optimizer cannot stop at such a point.
The energy reported for the intermediate mixed density of correct Pulay mixing undershoots the converged variational state energy by up to 0.62 eV. Because this quantity is not the energy of an admissible Kohn-Sham state, this does not violate the variational principle, and it is not an error. It shows, however, that the energies of a mixing scheme cannot reveal errors of this kind.
Supplementary Table 6. Errors revealed by the variational bound, bulk Si (6 3 6^3 6 3 k-points) unless stated otherwise. Δ E \Delta E Δ E : final energy relative to the converged variational minimum. The first two rows were reproduced with diagnostic switches of the present code.
Figure S5. (a) An implementation error of density mixing in bulk Si (6 × 6 × 6 6 \times6 \times6 6 × 6 × 6 k-points), which is not detected by the convergence criterion: with the G = 0 G = 0 G = 0 component removed by the Kerker filter, Pulay mixing stops 2.5 eV above the minimum with a frozen residual. Correct Pulay mixing, whose mixed-density energy falls below the minimum on the way (open circles), HM and friction dynamics reach it. (b) Wall time for the Si supercell with 64 atoms (Table 4). The shaded bars are HM.
The two tests: details
Thresholds. C1 is reported as Δ A C 1 = A e n d − A r e t \Delta A_{\mathrm{C1}} = A_{\mathrm{end}} - A_{\mathrm{ret}} Δ A C1 = A end − A ret , where A r e t A_{\mathrm{ret}} A ret is the free energy of the returned orbitals, evaluated in the first step of the probe, and A e n d A_{\mathrm{end}} A end that at its end point. In VASP, the energy reported by a density-mixing run can differ from A r e t A_{\mathrm{ret}} A ret in either direction, because it is evaluated for a mixed density: by +0.03 meV for NiO with LMAXMIX=2, −0.09 meV for LiMn2 _2 2 O4 _4 4 and −100 meV for LiCoO2 _2 2 (Supplementary Note 7.2). Only A r e t A_{\mathrm{ret}} A ret belongs to an admissible state and obeys the upper bound. C1 is counted as failed if the probe lowers the free energy by more than 10 δ 10\delta 10 δ , where δ \delta δ is the energy tolerance of the run per cell (10 − 6 10^{-6} 1 0 − 6 Hartree in VESPAW, EDIFF in VASP, and the Materials Project tolerance of 5 × 10 − 5 5 \times10^{-5} 5 × 1 0 − 5 eV per atom gives δ = N a t × 5 × 10 − 5 \delta= N_{\mathrm{at}} \times5 \times10^{-5} δ = N at × 5 × 1 0 − 5 eV). The factor 10 is an operational choice. It is motivated by the following model, which is an assumption about the sequence of energies and not a property of every self-consistency iteration: if E k − E ∞ = c r k E_k - E_\infty= c r^k E k − E ∞ = c r k with 0 < r < 1 0 < r < 1 0 < r < 1 , then E k − E k + 1 = c r k ( 1 − r ) E_k - E_{k+1} = c r^k(1-r) E k − E k + 1 = c r k ( 1 − r ) , and a last change E k − E k + 1 < δ E_k - E_{k+1} < \delta E k − E k + 1 < δ implies E k + 1 − E ∞ = c r k + 1 < δ r / ( 1 − r ) E_{k+1} - E_\infty= c r^{k+1} < \delta r/(1-r) E k + 1 − E ∞ = c r k + 1 < δ r / ( 1 − r ) , that is less than 9 δ 9\delta 9 δ for r ≤ 0.9 r \le0.9 r ≤ 0.9 . Iterations that oscillate, stall or converge sublinearly are not covered by this bound, and these are exactly the cases that C1 is meant to detect. No conclusion of the main text depends on the factor: every C1 failure reported there is at least 300 δ 300\delta 300 δ , except four in the database survey at 11 to 33 δ 33\delta 33 δ (Supplementary Note 7.2). A rise of the energy under C1 is a pass. C2 is counted as failed if ∣ Δ n ∣ > 5 |\Delta_n| > 5 ∣ Δ n ∣ > 5 meV. The derivative in Eq. (4) of the main text is evaluated as [ − 3 E ( 0 ) + 4 E ( h ) − E ( 2 h ) ] / ( 2 h ) [-3E(0) + 4E(h) - E(2h)]/(2h) [ − 3 E ( 0 ) + 4 E ( h ) − E ( 2 h )] / ( 2 h ) with ∣ h ∣ = 0.01 |h| = 0.01 ∣ h ∣ = 0.01 electrons. Its truncation error, h 2 E ′ ′ ′ / 3 h^2 E'''/3 h 2 E ′′′ /3 , is of second order in the occupation change. In VASP, Δ n \Delta_n Δ n for MgO changes by at most 0.2 meV between ∣ h ∣ = 0.005 |h| = 0.005 ∣ h ∣ = 0.005 and 0.02 with PBE and r2 ^2 2 SCAN. With SCAN it is 3.0 to 3.6 meV, independent of ∣ h ∣ |h| ∣ h ∣ , of ADDGRID, of a cutoff of 800 eV and of the smearing, so it is a systematic offset rather than scatter from the integration grid. Its origin is not established. For NiO with PBE+U it varies between − 1.7 -1.7 − 1.7 and 1.1 meV with ∣ h ∣ |h| ∣ h ∣ , without a trend (Tables 8 and 18). At a state that is not self-consistent, Δ n \Delta_n Δ n also contains the difference between the Hamiltonian used for the eigenvalues and that of the current density, so C2 responds to a large residual as well (Supplementary Note 7.1). C2 is therefore interpreted at the end point of C1, where the residual has been removed and a remaining Δ n \Delta_n Δ n measures the inconsistency of the Hamiltonian. A failure of C1 shows that the returned state is not the lowest state accessible to the probe. If the descent is continuous from the returned state, the state is also not a local minimum of the implemented functional, but this does not distinguish a state that is not stationary from an unstable stationary state, and both occur (NiO and UO2 _2 2 below). A lower end point reached after the energy has risen does not establish an instability, since the optimizer may have crossed a barrier from a metastable minimum.
Two further errors were found in the course of this work and are used here to test C1 and C2 of the main text (Table 7). The test system is MgO with SCAN0r at the Γ \Gamma Γ point, for which a calculation takes seconds. In the first error, the double-counting term of the onsite hybrid functional subtracted the potential conjugate to the kinetic-energy density of the frozen core, although this term depends on the valence state only through its cutoff function. The energy functional is unchanged, but the Hamiltonian is not its derivative. In the second error, the density residual of the Pulay and Anderson mixers included the Fourier components in the corners of the FFT box outside the cutoff sphere of the density. The Kerker filter, which is applied in the cutoff sphere, never mixes them, so that the residual stays at a constant floor, 1.7 × 10 − 3 1.7 \times10^{-3} 1.7 × 1 0 − 3 in this cell, while the part inside the sphere decreases to 10 − 12 10^{-12} 1 0 − 12 . The density mixers therefore stopped only on the change of the energy, and the restart rule of the mixers, which reduces the damping whenever the residual does not improve, reduced it in every step.
With the inconsistent Hamiltonian, all optimizers report convergence: Pulay and Anderson mixing, HM with Davidson and RMM-DIIS, and friction dynamics. They agree with each other within 0.25 meV, at a state 7.5 meV above the minimum, whose band gap is 42 meV too small. Agreement between independent optimizers is therefore no proof of convergence when they all follow the same Hamiltonian. HM gives only a weak signal: one rejected step with Davidson and nine with RMM-DIIS. For a charged cell and for NiO the signal is strong, with more than a third of the steps rejected and no convergence. The descent probe C1 passes: because it uses the same Hamiltonian, it cannot detect this inconsistency. The Janak test C2 fails by − 92 -92 − 92 and − 190 -190 − 190 meV for the lowest unoccupied and highest occupied states, compared with about 1 meV for the correct code.
With the residual floor, Anderson mixing stops 29 meV above the minimum, with a one-center density matrix that has not relaxed. Its frontier states are converged, so that C2 passes, but C1 lowers the free energy by 29.45 meV in two accepted steps. Pulay mixing reaches the minimum in spite of the floor. With the floor removed, by measuring the residual on the cutoff sphere and taking the components outside it from the output density, and with a stop that requires both a small residual of the density and of the one-center density matrix and a small change of the energy, all optimizers end within 0.21 meV of the minimum and pass both tests.
Supplementary Table 7. Certificates for MgO with SCAN0r at the Γ \Gamma Γ point. Δ E \Delta E Δ E : final energy relative to HM with the correct code. C1: change of the energy when the final state is restarted with HM. MD: friction dynamics. C2: Δ n \Delta_n Δ n of Eq. (4) of the main text for the lowest unoccupied (L) and highest occupied (H) state. All runs report convergence. Incons. H ^ \hat{H} H ^ : Hamiltonian not the derivative of the energy. Res. floor: residual floor of the density mixers. Energies in meV.
A further error, a factor of two in the one-center kinetic-energy density of the meta-GGA, was invisible to tests of this kind: HM converged without rejected steps, and the relaxed Janak relation between the change of the energy with the electron number and the eigenvalue held within 5 meV.
The error changed the functional itself, energy and Hamiltonian consistently, and it was found by comparing the integral of the kinetic-energy density with the kinetic energy and the band gaps with another code. The tests examine whether a lower state is accessible from the returned state of the implemented functional, not whether the functional is the intended one.
Both tests can be applied to another code without changes. We applied them to VASP 6.4.2 S21,S36.
For C2 the energy of the frozen orbitals is obtained by reading the converged wave functions with fixed occupations and a single step that recomputes the eigenvalues for the new density (ALGO=Eigenval). With ALGO=None the stored eigenvalues are used and the energy is not that of the frozen orbitals. For MgO at the Γ \Gamma Γ point, ∣ Δ n ∣ \lvert\Delta n\rvert ∣ Δ n ∣ is about 1 meV with PBE, r2 ^{2} 2 SCAN, HSE06 and PBE0, and 3 meV with SCAN. For C1 the final state is restarted with direct minimization (ALGO=All). For antiferromagnetic NiO with PBE+U U U (64 atoms, Γ \Gamma Γ point), Pulay mixing with the default LMAXMIX=2, which passes only the angular components up to L = 2 L = 2 L = 2 of the one-center densities to the mixer, reports convergence with an energy tolerance of 10 − 7 10^{-7} 1 0 − 7 eV, but C1 lowers the free energy of the returned orbitals by 30 μ \mu μ eV (56 μ \mu μ eV below the reported energy). With LMAXMIX=4 the decrease is 0.04 μ \mu μ eV. As for the residual floor above, the state is consistent but admits a remaining descent: C2 passes within 1 meV and only C1 detects it.
f f f -electron oxides in VASP
UO2 _2 2 and Ce2 _2 2 O3 _3 3 were computed with VASP 6.4.2 (vasp_gam), PBE PAW potentials (U 06Sep2000, Ce 23Dec2003, O 08Apr2002), a plane-wave cutoff of 520 eV,
Table S8. Janak test C2 in VASP 6.4.2 for MgO at the Γ \Gamma Γ point: Δ n \Delta_{n} Δ n (meV) for the lowest unoccupied (L) and highest occupied (H) state.
PREC=Accurate, LASPH, Gaussian smearing of 0.01 eV and the Γ \Gamma Γ point only. These are minimal test cells, chosen to make the runs cheap, and the converged energies are not meant as physical results. U O 2 \mathrm{UO}_{2} UO 2 : fluorite structure with a = 5.47 a = 5.47 a = 5.47 Å, 12 atoms, collinear type-1 antiferromagnet, PBE+U U U in the form of Dudarev et al. with U = 4.5 U = 4.5 U = 4.5 eV and J = 0.54 J = 0.54 J = 0.54 eV on U 5 f 5f 5 f . C e 2 O 3 \mathrm{Ce}_{2}\mathrm{O}_{3} Ce 2 O 3 : A-type structure, 2 × 2 × 1 2 \times2 \times1 2 × 2 × 1 cell with 20 atoms, ferromagnetic, U = 5 U = 5 U = 5 eV on Ce 4 f 4f 4 f .
Protocol. A calculation with PBE (U = 0 U = 0 U = 0 ) is followed by one with U / 2 U/2 U /2 and then by one with full U U U , each restarted from the orbitals of the previous step (ISTART=1, ICHARG=0), with Pulay mixing (ALGO=Normal) and NELM=300. The full-U U U run uses LMAXMIX=6 or 2 and an energy tolerance EDIFF of 10 − 4 10^{-4} 1 0 − 4 , 10 − 6 10^{-6} 1 0 − 6 or 10 − 7 10^{-7} 1 0 − 7 eV. C1 restarts the final orbitals with ALGO=All, EDIFF=1E-9 and up to 600 iterations. C2 follows Supplementary Note 7 with fixed occupations of the spin-up frontier states.
U O 2 \mathrm{UO}_{2} UO 2 . Pulay mixing with LMAXMIX=6 reports convergence at the same state for all three tolerances, 1.44 to 1.46 eV above the end point of C1 (Table 9). For EDIFF=1E-7 the Janak test gives Δ L = − 0.7 \Delta_{L} = -0.7 Δ L = − 0.7 meV and Δ H = 1.0 \Delta_{H} = 1.0 Δ H = 1.0 meV at the returned state and − 1.0 -1.0 − 1.0 and 1.5 meV at the end point of C1: the Hamiltonian is consistent. The C1 trajectory starts with an energy change of 5 × 10 − 7 5 \times10^{-7} 5 × 1 0 − 7 eV and reaches a decrease of 0.48 eV per iteration after 11 iterations: the returned state is consistent with a nearly stationary but unstable solution, which is not a minimum (damped electronic dynamics in Supplementary Note 9). The end points of C1 are not unique: from the EDIFF=1E-7 state C1 reaches − 113.2394 -113.2394 − 113.2394 eV, from the two looser states and from the unconverged LMAXMIX=2 run − 113.2150 -113.2150 − 113.2150 eV, and a run from random orbitals ends at − 113.2534 -113.2534 − 113.2534 eV. Several f -orbital minima exist, as known for UO2 _2 2 S37. C1 tests for a lower state accessible from the returned one, not for the ground state.
Supplementary Table 9. f -electron oxides in VASP: free energy F F F reported by Pulay mixing at full U U U and after the descent probe C1, band gaps E g E_g E g before and after C1. Energies in eV. a ^{a} a Not converged. b ^{b} b C1 still descending after 600 iterations. c ^{c} c 400 iterations of C1, last change below 2 × 10 − 7 2 \times10^{-7} 2 × 1 0 − 7 eV. d ^{d} d C1 still descending after 300 iterations, so an upper bound.
Ce2 _2 2 O3 _3 3 . Pulay mixing with LMAXMIX=6 converges in 42 iterations. C1 lowers the free energy of the returned orbitals by 17.57 meV (17.85 meV below the reported energy) at almost unchanged gap. The decrease per iteration in the first C1 steps, 6 × 10 − 5 6 \times10^{-5} 6 × 1 0 − 5 eV, is 600 times the tolerance, so a single C1 iteration already flags the state. The unconverged LMAXMIX=2 run reaches the same energy under C1. C1 from the LMAXMIX=6 state converges to 10 − 9 10^{-9} 1 0 − 9 eV after 713 iterations. The Janak test on the Pulay state gives Δ L = 0.3 \Delta_L = 0.3 Δ L = 0.3 meV and Δ H = 7.3 \Delta_H = 7.3 Δ H = 7.3 meV. The frozen-orbital energy at unchanged occupations differs from the SCF energy by 0.28 meV, so the input Hamiltonian and output density of the Pulay state are not yet consistent with each other. At the end point of C1 the test gives −0.5 and 1.0 meV. The example shows why C2 is interpreted at the end point of C1: at the returned state it responds to the residual, not to an inconsistency.
The k k k -point check behaves differently for the two oxides. For UO2 _2 2 with a 2 × 2 × 2 2 \times2 \times2 2 × 2 × 2 mesh, Pulay mixing again reports convergence, 0.21 eV above the end point of C1. For Ce2 _2 2 O3 _3 3 with the same mesh it does not converge in 300 iterations, and C1 lowers the free energy of its last state by 118 meV. This failure is visible to the convergence criterion and needs no test.
Survey of magnetic oxides with a database protocol
To test whether C1 failures occur under a production protocol, 22 magnetic oxides were taken from the Materials Project S38 through its application programming interface S39 and recomputed with its static protocol as implemented in pymatgen S40 (MPStaticSet): PBE+U U U with the U U U values of the database S41, ferromagnetic initial moments, 520 eV cutoff, its k k k -point density, LMAXMIX=4, ALGO=Fast (Davidson followed by RMM-DIIS with Pulay mixing), and an energy tolerance of 5 × 10 − 5 5 \times10^{-5} 5 × 1 0 − 5 eV per atom, that is δ = N a t × 5 × 10 − 5 \delta= N_{\mathrm{at}} \times5 \times10^{-5} δ = N at × 5 × 1 0 − 5 eV per cell. Two settings of this input set are changed: Gaussian smearing of 0.05 eV instead of the tetrahedron method, so that the reported energy is the free energy tested by C1, and a limit of 200 instead of 100 iterations. Each run starts from scratch, without the output of a preceding relaxation. For each compound the entry is the database structure of the experimentally known polymorph with the lowest energy above the hull, in its primitive cell (Table 10). For V2 _2 2 O3 _3 3 and Fe3 _3 3 O4 _4 4 , VASP stops at the default symmetry tolerance with inconsistent Bravais lattices of the database cell. These two were run with SYMPREC=1E-4, which changes only the symmetry analysis. C1 restarts the final orbitals with ALGO=All and a tolerance of 10 − 7 10^{-7} 1 0 − 7 eV. For FeO, where ALGO=All with the default step became unstable, it uses TIME=0.1. These calculations do not reproduce the energies stored in the database, which come from a different workflow. They test this modified protocol, not the database entries.
Supplementary Table 10. Supplementary Table 10: Survey with the Materials Project static protocol in VASP 6.4.2, database structures. N N N : atoms in the cell. δ \delta δ : energy tolerance per cell (meV). it.: iterations to the reported convergence. Δ F C 1 \Delta F_{\mathrm{C1}} Δ F C1 : change of the free energy of the returned orbitals under C1 (meV). M M M : total moment (μ B \mu_{\mathrm{B}} μ B ) before and after C1. Bold: C1 failed (− Δ F C 1 > 10 δ -\Delta F_{\mathrm{C1}} > 10\delta − Δ F C1 > 10 δ ). All runs report convergence.
Fifteen of the 22 systems pass, within 2.1 δ 2.1\delta 2.1 δ of the end point of C1. Seven fail, by two mechanisms. In CoO, FeO, Fe3 _3 3 O4 _4 4 and LiFePO4 _4 4 the RMM-DIIS iteration oscillates by several meV per step and stops when one energy change happens to fall below the tolerance. In CoO, for example, the last three changes are + 3.4 +3.4 + 3.4 , − 0.70 -0.70 − 0.70 and + 0.42 +0.42 + 0.42 meV before a change of + 0.06 +0.06 + 0.06 meV ends the run at δ = 0.1 \delta= 0.1 δ = 0.1 meV. The returned states of CoO and FeO are metallic, with gaps of 0.001 and 0.003 eV. C1 lowers the free energy by 0.79 and 0.90 eV at unchanged moment and ends in insulators with gaps of 1.40 and 1.38 eV. For Fe3 _3 3 O4 _4 4 C1 lowers the free energy by 0.24 eV and opens the gap from 0.03 to 0.49 eV. For LiFePO4 _4 4 it lowers the free energy by 46 meV, 33 δ 33\delta 33 δ , with the gap changing from 4.21 to 4.35 eV. In V2 _2 2 O3 _3 3 , Co3 _3 3 O4 _4 4 and FeTiO3 _3 3 the iteration is still descending steadily, by 0.3 to 4 meV per step, when one step falls below the tolerance. C1 lowers the free energy by 14.4, 9.8 and 5.5 meV, that is 29, 14 and 11 δ 11\delta 11 δ . The gap of V2 _2 2 O3 _3 3 changes from 0.13 to 0.64 eV, those of Co3 _3 3 O4 _4 4 and FeTiO3 _3 3 by less than 0.05 eV. FeTiO3 _3 3 and Co3 _3 3 O4 _4 4 would pass with a factor of 15 instead of 10 in the threshold, and V2 _2 2 O3 _3 3 with a factor of 30, whereas the other four fail by more than 30 δ 30\delta 30 δ . The C1 trajectory of CoO starts like that of UO2 _2 2 , with energy changes of 0.1 to 1 meV in the first steps that grow to 0.08 eV by step 10, and converges in 27 iterations. That of FeO passes through unstable stretches even with TIME=0.1 and returns repeatedly to within 1 meV of its final energy, − 13.1619 -13.1619 − 13.1619 eV, before it converges.
Experimental structures. The same protocol was first applied to experimental room-temperature structures of the same 22 compounds (Table 11). There, the seven systems above pass, and LiCoO2 _2 2 fails, while it passes with the database structure (nonmagnetic, gap 2.40 eV). Whether the iteration is caught depends on the starting point, not only on the material.
With experimental structures, 21 of the 22 systems pass. Their returned states lie within
Supplementary Table 11. Supplementary Table 11: Survey with the Materials Project static protocol in VASP 6.4.2, experimental structures. N N N : atoms in the cell. δ \delta δ : energy tolerance per cell (meV). it.: iterations to the reported convergence. Δ F C 1 \Delta F_{\mathrm{C1}} Δ F C1 : change of the free energy of the returned orbitals under C1 (meV). M M M : total moment (μ B \mu_{\mathrm{B}} μ B ) before and after C1. Bold: C1 failed (− Δ F C 1 > 10 δ -\Delta F_{\mathrm{C1}} > 10\delta − Δ F C1 > 10 δ ). All runs report convergence.
1.2 δ 1.2\delta 1.2 δ of the end point of C1, except SrCoO3 _3 3 (2.3 δ 2.3\delta 2.3 δ ). Measured from the reported energy instead, LiFePO4 _4 4 would lie 4.1 δ 4.1\delta 4.1 δ above the end point, because its reported energy is 5.1 meV above the free energy of its orbitals. LiCoO2 _2 2 fails by 374 meV per formula unit, 1868 δ 1868\delta 1868 δ , and its reported energy lies 273 meV above the end point. Its RMM-DIIS iteration oscillates by several meV per step at a density residual of about 0.3 that does not decrease. The run stops at iteration 36 because one energy change, − 6 × 10 − 6 -6 \times10^{-6} − 6 × 1 0 − 6 eV, happens to fall below the tolerance of 2 × 10 − 4 2 \times10^{-4} 2 × 1 0 − 4 eV. The reported energy, − 21.2353 -21.2353 − 21.2353 eV, is that of a mixed density: the first step of C1, which evaluates the returned orbitals self-consistently, gives − 21.1350 -21.1350 − 21.1350 eV, 100 meV higher, and C1 then descends to − 21.5086 -21.5086 − 21.5086 eV with a total moment of 2.00 μ B \mu_B μ B instead of 4.53 μ B \mu_B μ B . Three follow-up runs (Table 12 ) show that the stop is reproducible and that a tight tolerance exposes the failure as non-convergence.
Table 12. LiCoO2 _2 2 with the Materials Project protocol: follow-up runs. Energies in eV, moments in μ B \mu_B μ B .
The start without moments converges to the low-spin state, which passes C1 and lies 28 meV above the end point of the other probes. Which of the two is the ground state of the functional is not decided by C1. The example shows only that the protocol reported a state that is neither.
Census of Materials Project magnetic oxides
Supplementary Table 14 lists all 366 entries of the census (Methods of the main text). Six entries that stopped at the start of the run with a symmetry error of VASP were computed with SYMPREC = 10 − 4 10^{-4} 1 0 − 4 , and all six report convergence and pass C1.
Entries containing W. The protocol assigns the potential Wp v _{\mathrm{pv}} pv , which is absent from PBE.54, and the version of 06Nov2007 from PBE.52 was used. With this potential, 21 of the 23 W-containing entries report convergence and 15 of these fail C1 at 10 δ 10\delta 10 δ . The runs also behave anomalously: 17 of the 21 end with a charge-density residual rms(c) above 1, and in 10 the free energy of the returned orbitals, evaluated in the first step of C1, lies 0.3 to 28 eV above the reported energy. Outside this group, rms(c) never exceeds 0.43 and the corresponding difference never exceeds 59 meV. The decision to treat these entries separately was taken after this anomaly appeared and before any control was run. The same 23 entries were then repeated with W_sv from PBE.54 and otherwise identical settings, with the rule fixed in advance that the rate including W would be the main result only if at least 8 of the 15 failures persisted. The two data sets are PAW_PBE W_pv 06Nov2007 (12 valence electrons, 5 p 6 5 d 6 6 s 5p^{6}5d^{6}6s 5 p 6 5 d 6 6 s ) and PAW_PBE W_sv 04Sep2015 (14 valence electrons, with 5s in addition), with the same ENMAX of 223.057 eV and the same RCORE of 2.5 a B a_{B} a B (Supplementary Table 13 ). All 23 then report convergence, with rms(c) ≤ 0.04 \le0.04 ≤ 0.04 and differences of at most 1.6 meV. Only 3 of the 15 failures persist, and three entries that passed or aborted with W_pv now fail, giving six failures in total. The comparison implicates W_pv of this release under the protocol settings, but because the release and the valence differ together it does not establish the cause. The W-containing entries are reported as a sensitivity analysis, and the rates of the main text are for the entries without W. Including them changes the rate at 10 δ 10\delta 10 δ from 57 of 339 (16.8%) to 72 of 360 (20.0%) with W_pv, or to 63 of 362 (17.4%) with W_sv.
Threshold, trajectory and starting point. For the 339 calculations without W that report convergence, the number that fail C1 is 86 at 5 δ 5\delta 5 δ , 57 at 10 δ 10\delta 10 δ , 37 at 30 δ 30\delta 30 δ , 25 at 100 δ 100\delta 100 δ and 23 at 300 δ 300\delta 300 δ . For the 57 failures at 10 δ 10\delta 10 δ , the energy falls from the returned state, with no rise larger than δ \delta δ , until the decrease exceeds 10 δ 10\delta 10 δ in 49 cases, and rises first in 8. Along the whole trajectory of C1, the energy of 38 never rises by more than δ \delta δ and that of 19 does. C r 2 N i O 4 \mathrm{Cr}_{2}\mathrm{NiO}_{4} Cr 2 NiO 4 (mp-19303), for example, first descends past 10 δ 10\delta 10 δ , later rises to 4.1 eV above the starting energy, and ends 19 meV below it. For the 8 failures that rise before the threshold is reached, a lower state is established but an instability of the returned state is not. The per-entry values are in c1_descent.csv in the archive. All entries are computed in the chemically primitive cell and start from the ferromagnetic moments of the protocol. For 12 of the 42 W-free converged entries labelled antiferromagnetic, this cell is smaller than the database cell and cannot hold that order. Four converged runs without W needed more than 100 iterations, the limit of the original input set, and all four fail C1. With that limit the rate at 10 δ 10\delta 10 δ would be 53 of 335. Controls with the tetrahedron method, the antiferromagnetic order of the database and Pulay mixing from the end point of C1 are given in Supplementary Note 9.
Supplementary Table 14: Census of the 366 Materials Project magnetic oxides, sorted by − Δ A C 1 / δ -\Delta A_{\mathrm{C1}}/\delta − Δ A C1 / δ , followed by the runs that do not report convergence. MP ID without the prefix mp-. N N N : atoms in the computed cell (chemically primitive, for 30 entries smaller than the database cell, whose number of sites, at most 20, defines the population). Energies in meV per cell. δ \delta δ : the requested tolerance. Δ A C 1 \Delta A_{\mathrm{C1}} Δ A C1 : change of the free energy from the first step to the end of C1. Δ F r e p \Delta F_{\mathrm{rep}} Δ F rep : end of C1 minus the reported free energy. Bold : fails C1 (> 10 δ 10\delta 10 δ ). W W W : contains W and computed with W_pv. These entries are not used for the rates of the main text, and the control is given in Supplementary Table 13. *: C1 not converged in 1000 iterations, lowest energy on the trajectory, lower bound of the decrease, with band gap and moment of the last step.
Table 14.
Table 1.
Table 8 (PDF p. 81).
Controls of the census protocol
The selection rules and random seeds were fixed in the script make_sets.py before any control was run, together with the criteria given below (Methods of the main text). The smearing and optimizer controls draw from the W-free failures and passes whose run and C1 converged, that were computed without the change of SYMPREC and whose k k k -point mesh has at least four points. All controls used VASP 6.4.3.
Starting state from the tetrahedron method. The census replaces the tetrahedron method of the input set by Gaussian smearing of 0.05 eV. Because C1 is run with Gaussian smearing, the control below tests whether the failures depend on the state from which the probe starts. It does not test whether the tetrahedron run reached a minimum of its own free energy, which differs from the Gaussian one. For 25 failures and 25 passes (seed 2026), and for CoO, FeO and NiO of the targeted survey, the run was repeated with ISMEAR=-5 and otherwise unchanged settings, and probed with C1 with Gaussian smearing of 0.05 eV, as in the census (Table 15). All 53 runs report convergence.
The criterion fixed in advance, formulated as independence of the smearing, was met if at least 15 of the 25 failures and at most 4 of the 25 passes fail at 10 δ 10\delta 10 δ . Of the failures, 22 still fail, while Co2 _2 2 GeO4 _4 4 (mp-18758, 10.0 δ 10.0\delta 10.0 δ in the census), BaCu3 _3 3 O4 _4 4 (mp-3988) and Cu2 _2 2 SO5 _5 5 (mp-4386) pass. Of the passes, NpCuPO (mp-1087496) and Ba2 _2 2 Tm(CuO2 _2 2 )4 _4 4 (mp-6710) fail. FeO fails as with Gaussian smearing and NiO passes. CoO passes: the run with the tetrahedron method returns a state with a gap of 0.005 eV, C1 converges in eight iterations to a state with a gap of 0.018 eV and a free energy of − 11.488 -11.488 − 11.488 eV, while C1 from the run with Gaussian smearing ends at − 12.083 -12.083 − 12.083 eV with a gap of 1.40 eV, with the same INCAR of C1, cell and k k k -points. The tetrahedron run thus returns a state 0.60 eV above the insulating state that admits no descent beyond the tolerance: a metastable state of the Gaussian-smearing functional that C1 cannot detect.
Magnetic start. All entries of the census start from ferromagnetic moments. The 42 W-free converged entries labelled antiferromagnetic were repeated from the moments of the database (magmoms of the magnetism record) in the database cell, which is larger than the census cell for 12 of them (Table 16). Of these, 41 report convergence, and all 41 have a net moment below 0.1 μ B \mu_{\mathrm{B}} μ B both at the end of the run and at the end point of C1. LaVO3 _3 3 (mp-19350) does not converge. Local moments were not written (LORBIT not set), so the order is identified by the net moment and by the energy: the end point of C1 lies below that of the ferromagnetic start in 40 of the 42 entries, by a median of 3.6 meV/atom, which excludes a collapse to a non-magnetic state. Four of the 41 fail C1 at 10 δ 10\delta 10 δ (9.8%), compared with 7 of 42 (16.7%) from the ferromagnetic start. Sr2 _2 2 CoMoO6 _6 6 (mp-510546) and K2 _2 2 CoO2 _2 2 (mp-28890) fail from the ferromagnetic start and pass from the antiferromagnetic one, and no pass becomes a failure.
Optimizer. To test whether the end points of C1 are artifacts of direct minimization, 20 failures (seed 2027), CoO and FeO were recomputed with Gaussian smearing, probed with C1, and restarted from the end point of C1 with Pulay mixing (ALGO=Normal, EDIFF as in the census). The criterion fixed in advance was convergence of the Pulay run within max ( 10 δ , 1 m e V ) \max(10\delta, 1\ \mathrm{meV}) max ( 10 δ , 1 meV ) of the free energy of C1 and within 0.1 eV of its gap. All 22 meet it: the Pulay run converges in two iterations, its free energy differs from that of C1 by at most 2.6 meV (largest for mp-1104680, below C1, whose C1 did not converge within 1000 iterations), and the gap changes by at most 0.064 eV. In one entry (mp-1188719) the repeated run with Gaussian smearing did not report convergence within 200 iterations. Its C1 end point is nevertheless stable under Pulay mixing.
Ordinary restart. To test whether a restart of the production algorithm would detect the failures, all 57 W-free failures and 25 passes (seed 2028, same filter) were recomputed with the census settings and restarted from the returned orbitals, once with the same INCAR (ALGO=Fast, the same EDIFF, ISTART=1) and once with C1 (Table 17). The comparison is made within each entry, since the repeated run may return a state that differs from the census one. The criterion fixed in advance: the restart detects an entry if it ends more than 10 δ 10\delta 10 δ below the free energy reported by the run, and it counts as a substitute for C1 if it detects at least 80% of the entries that fail C1 in the same calculation and flags at most two of the passes. All 82 repeated runs report convergence. C1 reproduces 56 of the 57 failures and all 25 passes. Na3 _3 3 Co2 _2 2 SbO6 _6 6 (mp-19087) gives 7.5 δ 7.5\delta 7.5 δ and passes, so the repeated set has 56 failures and 26 passes. The restart detects 15 of the 56 failures (27%) and flags none of the 26 passes, so the criterion is not met. Of the 41 failures it misses, 33 end within 3 δ 3\delta 3 δ of the reported free energy, including entries with a C1 decrease of up to 808 δ 808\delta 808 δ (mp-31514). In two (mp-1188719, mp-22543) the restart does not converge within 200 iterations and its free energy rises. Detection does not follow the size of the descent: the restart detects 8 of the 25 failures with a C1 decrease above 100 δ 100\delta 100 δ and 7 of the 31 below. The restart takes a median of 5.5 iterations for the failures and 2 for the passes and costs a median of 0.22 of the run, compared with 1.39 for C1.
Step of the Janak test. C2 was evaluated with ∣ h ∣ = 0.005 |h| = 0.005 ∣ h ∣ = 0.005 , 0.01 and 0.02 (Table 18). The criterion fixed in advance was a change of less than 1 meV between the smallest and largest step. MgO meets it with all functionals and settings. NiO with PBE+U U U does not: Δ n \Delta_n Δ n varies by up to 1.5 meV, without a trend in ∣ h ∣ |h| ∣ h ∣ , so the variation is scatter rather than truncation error, and all values lie below 1.7 meV, well inside the threshold of 5 meV.
Damped electronic dynamics. To test whether the returned states of UO2 _2 2 and CoO lie close to unstable stationary points, damped electronic dynamics (ALGO=Damped) with TIME = 0.02, 0.05 and 0.10, and direct minimization (ALGO=All), were started from the returned orbitals, with EDIFF = 10^{-9} eV and at most 400 iterations (Table 19). NiO of the targeted survey, which passes C1, is the control. CoO is the returned state of the targeted survey. For UO2 _2 2 the returned state was recomputed with the U U U ramp and settings of Supplementary Note 7.1 (EDIFF = 10^{-7} eV, LMAXMIX=6) in VASP 6.4.3 with PBE.54 potentials. It lies 0.36 eV above the returned state of Table 9, and ALGO=All lowers it by 1.82 eV to −113.238 eV, 1.5 meV above the end point of C1 given there: the scans therefore test a returned state of the same kind, not the same state.
In NiO the residual decays from the first step in all four runs and the energy falls by 12 μ \mu μ eV.
In CoO all four runs leave the returned state, with a residual that grows from the first step, and end at the same state, 0.789 eV lower. In UO2 _2 2 the residual falls in the first three to five steps from 8.7 × 10 − 6 8.7 \times10^{-6} 8.7 × 1 0 − 6 to about 3 × 10 − 6 3 \times10^{-6} 3 × 1 0 − 6 and then grows. The damped runs are 1.55 to 1.79 eV lower after 400 steps and still descending. In the phase of growth, ln of the residual is linear in the step number to within 0.14. The criterion fixed in advance assumed a first-order gradient flow with TIME as the time step: the state counts as unstable if the growth rate per unit of n n n TIME agrees within 30% across the three time steps and NiO decays. NiO decays, but the rates per unit of n n n TIME fall by a factor of 2.1 from the smallest to the largest time step, for both compounds, so the criterion is not met. The growth per step divided by T I M E \sqrt{\mathtt{TIME}} TIME agrees within 9% for each compound. This scaling is expected for a single unstable direction if the damped algorithm is a second-order dynamics in which TIME acts as the square of a time step. This was identified after the runs, and we have not verified that TIME enters the algorithm in this way. We therefore take the scans as qualitative support only: the returned states of UO2 _2 2 and CoO are left by four optimizer settings, with a residual that grows roughly geometrically over the first iterations, whereas the residual of NiO decays under the same treatment.
Supplementary Table 15. Start from the tetrahedron method. C 1 C_1 C 1 decrease in units of δ \delta δ for the census run (Gaussian smearing) and for the run with the tetrahedron method (I S M E A R = − 5 \mathrm{ISMEAR}=-5 ISMEAR = − 5 ), each probed by C 1 C_1 C 1 with Gaussian smearing of 0.05 eV. The first 25 entries fail C 1 C_1 C 1 in the census, the next 25 pass. CoO, FeO and NiO are from the targeted survey. For CoO and FeO, C 1 C_1 C 1 gives 7893 and 8983 in units of δ \delta δ after the run with Gaussian smearing (optimizer control). *C 1 C_1 C 1 not converged within 1000 iterations, so the decrease is a lower bound.
Supplementary Table 16. Magnetic-start control for the 42 W-free converged entries labelled antiferromagnetic. N F N_{\mathrm{F}} N F , N A N_{\mathrm{A}} N A : atoms in the census cell (ferromagnetic start) and in the database cell (database moments). M A M_{\mathrm{A}} M A : net moment of the antiferromagnetic start (μ B \mu_{\mathrm{B}} μ B ). Δ A \Delta_{\mathrm{A}} Δ A : free energy per atom of the end point of C1, antiferromagnetic minus ferromagnetic start (meV). C1 decrease in units of δ \delta δ for both starts. †The run from the database moments does not report convergence within 200 iterations.
Supplementary Table 17. Supplementary Table 17: Ordinary restart. Decrease of the free energy in units of δ \delta δ : C1 in the census, C1 in the repeated run, and the restart with the settings of the run (ISTART=1), measured from the free energy reported by the run. A negative value is a rise. The first 57 entries fail C1 in the census, the last 25 pass. †Restart not converged within 200 iterations.
Supplementary Table 18. Step dependence of the Janak test C2 in VASP 6.4.3: Δ n \Delta_n Δ n (meV) of the lowest unoccupied (L) and highest occupied (H) state for ∣ h ∣ = 0.005 |h| = 0.005 ∣ h ∣ = 0.005 , 0.01 and 0.02. MgO at the Γ \Gamma Γ point, cutoff 520 eV and Gaussian smearing of 0.01 eV unless stated. NiO is antiferromagnetic with PBE+U U U as in Methods.
Supplementary Table 19. Damped electronic dynamics from the returned state. Δ E \Delta E Δ E : change of the energy at the last iteration (eV). n n n : iterations (*400 reached without convergence). rms: residual at the start and its minimum before growth. g g g : growth rate of ln ( r m s ) \ln(\mathrm{rms}) ln ( rms ) per iteration in the phase of approximately geometric growth, also per unit of n n n TIME and divided by T I M E \sqrt{\mathrm{TIME}} TIME . In CoO the residual grows from the first step, so no minimum is given. NiO decays in all runs.