betaFeedback

WorksPhysics and Astronomy

Jackknife-conformal uncertainty for linear ACE potentials: calibrated forces and silicon properties compared with Bayesian and POPS baselines

Abstract

Machine-learned interatomic potentials are misspecified, so Bayesian posterior variances understate their errors. For linear atomic cluster expansion (ACE) potentials we derive calibrated uncertainties from one misspecification-robust parameter covariance, the delete-one-configuration jackknife. Scaled by one conformal quantile from a held-out part of the training set, it gives per-atom force uncertainties close to their target coverage on held-out silicon data and on cracks left out of the fit; screw dislocations left out are under-covered by the smaller of the two models. With one scale fitted to the forces of a held-out test split, the same parameter distribution is propagated to 21 properties of silicon, from elastic constants to defect and surface energies, by implicit differentiation through each relaxation and by committees, and is tested against density functional theory (DFT) without tuning on any property. For a smaller and a larger model the intervals cover most properties, and the larger model's are about 4 times tighter. The Bayesian posterior with one calibrated scale has the wrong shape: it covers fewer properties and has far lower joint evidence. Pointwise optimal parameter sets (POPS), once their hypercube is numerically defined, performs comparably on the properties, but its per-atom force regions are not calibrated to a target (0.95 and 0.84 at a nominal 0.90). Where the jackknife fails is relaxed point defects, whose training cells are all hot snapshots: a bias that every configuration shares is invisible to a jackknife, but it can be measured from the training labels, and as a discrepancy term, devised after the property results were seen, it brings the coverage close to nominal.

Introduction

Machine-learned interatomic potentials are used for forces, in molecular dynamics and in active learning where force errors must be controlled during a simulation, and for the derived properties that a potential is built to predict. Both uses need an uncertainty. The training labels are deterministic density functional theory (DFT) energies, forces and stresses, so the error of a potential is approximation error: the model is misspecified. A Bayesian posterior is then far too narrow. Under misspecification it concentrates at the best-fitting parameters at a rate set by the amount of data, not by the size of the model error, and it collapses to a point as Ndata→∞N_{\mathrm{data}} \to\infty while the prediction error stays finite[1, 2, 3].

The most widely used alternatives for forces are committees and ensembles of models[4, 5, 6] and approximate posteriors on the last layer of a network[7]. They need several fits, or a calibration of their own. Conformal prediction gives intervals with guaranteed coverage under exchangeability[8, 9, 10], and Best et al.[11] combined it with the Bayesian posterior of linear atomic cluster expansion (ACE) potentials, and committees drawn from it, for silicon. A single conformal scale covers on average, however, while force errors vary strongly between local environments and are correlated within a configuration.

In this paper we construct calibrated per-atom force uncertainties for linear ACE potentials[12] from two parts. The shape is a cluster-robust ("sandwich") covariance of the coefficients[13, 14], with each training configuration as one cluster of correlated rows. We use the variant proposed by Bell and McCaffrey[15] and named CR3 by Pustejovsky and Tipton[16] (CV3 in MacKinnon, Nielsen and Webb[17]), the cluster form of the heteroskedasticity-consistent HC3 estimator[18]. CR3 is the delete-one-cluster jackknife: it measures how far the fitted coefficients move when each configuration is left out. It is computed exactly, without refitting, from the PRESS (predicted residual error sum of squares) residuals. These are the residuals each row would have if its cluster were left out of the fit, and they follow from the ordinary residuals by a leverage correction: for one row, division by one minus its leverage; for a cluster, the matrix analogue[19, 20]. The per-atom 3×33 \times3 block of the covariance gives anisotropic force regions.

The scale comes from split conformal prediction[8], which turns the scores of held-out calibration points into a quantile that covers a new point with a guaranteed probability. One quantile serves every atom: the jackknife shape already carries the dependence of the error on the environment. A scale learned from the site descriptor (normalised conformal prediction[21, 22, 23]) makes the coverage more even across environments represented in the training data, but not on environments left out of it, so we report it as a refinement, not as the method. The scores come from a stratified hold-out of the training set, and a fitted transfer exponent carries them to the fit on all the data. The open-source ace-jax package[24] provides the shape, the hold-out scores and the transfer; § 1.1.5 and § 3.2 describe and test the scale.

Per-atom intervals cannot be combined into an interval on a derived property, a quantity of interest (QoI), because a QoI depends on every atom and on relaxed geometries. A QoI's uncertainty has two parts[25]. The aleatoric part is the sampling error of the simulation that computes the QoI, assessed with statistical convergence tests[26, 27] (§ 1.1.3). The epistemic part comes from the uncertainty in the fit. It is most widely estimated with a committee of potentials, an idea long established in statistics[28]. The QoIs here come from relaxations and static calculations, so they have no aleatoric part and their uncertainty is purely epistemic. For a linear model the consistent object is then a distribution over the coefficients θ\theta: one draw gives energies, forces and stresses together, and every QoI follows from it.

We propagate the same CR3 distribution, with one scale fitted to held-out forces, to 21 QoIs of silicon, for potentials fitted to the GAP-18 database[29], the training set of the GAP-18 Gaussian approximation potential. The propagation uses implicit differentiation through every relaxation, including that of the lattice constant, and is cross-checked by committees run through the unmodified benchmark protocol. We test the intervals against DFT, one QoI at a time and jointly, and compare them with two baselines: the posterior with one conformal scale, and pointwise optimal parameter sets (POPS)[3].

Misspecification-aware parameter uncertainty has been propagated to material properties before. Perez et al.[30] propagate POPS hypercube ensembles of a quadratic spectral neighbour analysis potential (SNAP) for tungsten to phase, elastic, defect and surface properties, re-relaxing each member, and use the implicit derivative of a relaxed minimum with respect to the parameters[31] to predict volumes and a vacancy-migration barrier; the DFT values lie inside the ensemble's envelope without a calibration step. We differ in three ways: the shape is the CR3 covariance with one scale calibrated on held-out data, so its size does not depend on which training rows define a box; the intervals are tested against DFT one QoI at a time and jointly, by coverage, the continuous ranked probability score (CRPS) and evidence; and the committees run through an unmodified benchmark protocol, beside implicit differentiation of every relaxation.

We also show where calibrated parameter-space uncertainty fails, and why. A bias that all training configurations of one kind share is invisible to any delete-one jackknife; in the GAP-18 database[29] every defect cell is a hot molecular-dynamics snapshot. We diagnose this bias, measure it from the training labels alone, and add it to the defect QoIs as a discrepancy term, which leaves the fit unchanged.

Section 2 sets out the methodology: the linear ACE fit (2.1–2.2), automatic relevance determination as a check on the prior (2.3), the CR3 shape (2.4), the per-atom force calibration (2.5), the baselines (2.6) and the propagation to QoIs (2.7). Section 3 gives the validation statistics and the results: the per-atom forces first, then the global scale that the propagation uses, then the QoIs. Section 4 discusses them and Section 5 concludes.

Methodology

Linear ACE: model and observations

Each atom ii, at position rir_i with species ZiZ_i and environment xix_i (its neighbours within the cutoff), has a site descriptor ϕ(xi)∈Rp\phi(x_i) \in\mathbb{R}^{p}: the atomic cluster expansion (ACE) product basis[12], in species-major blocks, then pair blocks. The basis, its radial functions and the smoothness prior below are those of ACEpotentials[32]. The energy is linear in the coefficients θ\theta,

E({ri};θ)=∑iϕ(xi)⋅θ+∑iEZi0,(1)E(\{r_i\};\theta) = \sum_i \phi(x_i)\cdot\theta+ \sum_i E^0_{Z_i}, \tag*{(1)}

and so is every observation of a configuration:

bE=∑iϕ(xi),(2)b_E = \sum_i \phi(x_i), \tag*{(2)}
BF,iα=−∂bE∂riα,(3)B_{F,i\alpha} = -\frac{\partial b_E}{\partial r_{i\alpha}}, \tag*{(3)}
BV,αβ=−∂bE∂ϵαβ∣ϵ=0,(4)B_{V,\alpha\beta} = -\left.\frac{\partial b_E}{\partial\epsilon_{\alpha\beta}}\right|_{\epsilon=0}, \tag*{(4)}

with EZ0E^0_Z a per-species reference energy, riαr_{i\alpha} the Cartesian components of rir_i, and ϵ\epsilon a symmetric strain applied to positions and cell. The predictions are bE⋅θb_E\cdot\theta, BFθB_F\theta and the virial W=BVθW=B_V\theta (in eV; the stress is −W/Ω-W/\Omega, with Ω\Omega the cell volume). Each row of bEb_E, BFB_F or BVB_V is called an observable row bb. Each row has a weight ww, which combines a factor 1/N1/\sqrt{N} on the energy and virial rows of an NN-atom configuration, a weight per quantity (energy, force, virial) and a weight per configuration type, and all rows share one noise scale σ\sigma. The weights set the balance between energies, forces and virials, and σ\sigma sets the overall noise level. For an energy row the whitened row and target are ψ=bEw/σ\psi=b_Ew/\sigma and y~=yw/σ\tilde y=yw/\sigma, with y=EDFT−∑iEZi0y=E_{\mathrm{DFT}}-\sum_i E^0_{Z_i} the DFT energy minus the reference energies. Force and virial rows enter in the same way: each of the 3N3N rows of BFB_F and each of the six independent (Voigt) rows of BVB_V is one row bb, with ψ=bw/σ\psi=bw/\sigma and y~=yw/σ\tilde y=yw/\sigma, and its target yy is the matching DFT force or virial component (E0E^0 contributes to neither). Stacked over all configurations, the observable rows bb form the design matrix Φ\Phi, and the ψ\psi its whitened form Ψ\Psi. With least-squares E0E^0, one column per species carries the atom count on the energy rows (a fixed, wide prior).

The two models. We study two members of the linear ACE ladder of the ace-jax silicon accuracy–cost benchmark, fitted to the GAP-18 database[29] (Figure 1). A model onDM has correlation order NN, the largest number of neighbour factors in a basis function (body order N+1N+1, the index used for ARD in Section 2.3), and maximum total polynomial degree M: each one-particle factor tt with radial index ntn_t and angular degree ℓt\ell_t contributes nt+32ℓtn_t+\frac{3}{2}\ell_t, and the sum over the factors of a basis function is at most M. This is the total-degree basis of ACEpotentials, described in detail by Witt et al.[32]. Table 1 lists the silicon QoIs, their DFT (CASTEP) values and the benchmark's tolerances: 10 % of the DFT value for the elastic constants, the bulk modulus, the unstable stacking-fault energies and the dechoesion stress; 5 % for the (100) surface energy; 10 meV/atom for the phase energies; 0.1 eV for the point-defect formation energies; and 0.2 THz for the root-mean-square (RMS) error of the phonon frequencies. The QoI error of a model is the geometric mean of ∣error∣/tolerance|\mathrm{error}|/\mathrm{tolerance} over these QoIs. The ratios span orders of magnitude, so an arithmetic mean would be set by the one or two worst QoIs; the geometric mean, the exponential of the mean log ratio, counts a factor-of-two improvement equally on every QoI. o3d12 (223 coefficients) is the cheaper model, with a test force RMS error (RMSE) of 0.158 eV/Å and a typical QoI error of one tolerance (1.03). o4d20 (5476 coefficients) is the only size of the ladder that meets both of the benchmark's targets, a test force RMSE no larger than GAP-18's 0.0825 eV/Å on this split (0.0816 eV/Å) and a QoI error below 0.5 (0.39); o3d12 is 11× cheaper on a CPU node and 19× cheaper on an NVIDIA A100 GPU. The pair spans the useful range of the ladder: a cheap model whose intervals must be wide to be honest, and an accurate one whose intervals should be narrow. The two columns of Figure 1 also rank the sizes differently: o4d12 has a larger force error than o4d14 but a smaller QoI error (0.76 against 0.86). Force accuracy alone does not predict QoI accuracy, which is why the uncertainty is propagated to the QoIs themselves.

Figure 1. The linear ACE ladder of the ace-jax silicon accuracy–cost benchmark: test force RMSE (left) and QoI error (right; the geometric mean over the silicon QoIs of |error|/tolerance against CASTEP) against cost in LAMMPS, float64: on a CPU node with the ace-jax C++ kernel (top) and on an NVIDIA A100 GPU with the ace-jax CUDA kernel (bottom; hardware in § 3.5). Dashed: the benchmark's targets, GAP-18's force RMSE on this split and a QoI error of 0.5. Circled: the two models studied here.

Prior, posterior and evidence

The prior is θ∼N(0,Λ−1)\theta\sim\mathcal{N}(0,\Lambda^{-1}) with the smoothness prior

Λ=D2σc2,(5)\Lambda= \frac{D^2}{\sigma_c^2}, \tag*{(5)}
D=diag⁡(Γj),(6)D = \operatorname{diag}(\Gamma_j), \tag*{(6)}
Γj=∑t[(ltwl)4+(ntwn)4],(7)\Gamma_j = \sum_t \left[\left(\frac{l_t}{w_l}\right)^4 + \left(\frac{n_t}{w_n}\right)^4\right], \tag*{(7)}

where σc\sigma_c is the prior scale of the coefficients and the sum runs over the one-particle factors tt of basis function jj, with radial index ntn_t and angular degree ltl_t (the algebraic prior of ACEpotentials[32], wl=2/3w_l = 2/3, wn=1w_n = 1), so rough basis functions are damped. The smoothness weights DD do not depend on any hyperparameter; § 1.1.3 and § 1.1.4 use them to scale the coefficients. This is Bayesian linear regression (BLR). The posterior is Gaussian, with mean θ^\hat{\theta}, the maximum a posteriori (MAP) estimate,

A=∑rowsψψT+Λ,(8)A = \sum_{\mathrm{rows}} \psi\psi^{\mathrm{T}} + \Lambda, \tag*{(8)}
θ^=A−1∑rowsψy~,(9)\hat{\theta} = A^{-1}\sum_{\mathrm{rows}}\psi\tilde{y}, \tag*{(9)}
Cov⁡(θ∣D)=A−1,(10)\operatorname{Cov}(\theta\mid D) = A^{-1}, \tag*{(10)}

and the hyperparameters h=(σc,σ)h = (\sigma_c,\sigma) maximise the evidence (for o3d12 the maximum lies on the upper bound σc=104\sigma_c = 10^4 of the search, and the fit is used there; § 3.2.5)

log⁡p(D∣h)=−∥yw∥22σ2+12cTA−1c−12log⁡∣A∣+12log⁡∣Λ∣−nlog⁡σ,(11)\log p(\mathcal{D}\mid h) = -\frac{\lVert y^w\rVert^2}{2\sigma^2}+\frac{1}{2}c^{\mathrm{T}}A^{-1}c-\frac{1}{2}\log|A|+\frac{1}{2}\log|\Lambda|-n\log\sigma, \tag*{(11)}

up to a constant, with D\mathcal{D} the training data, ywy^w the weighted targets (ywy_w), nn the number of rows, and

c=∑rowsψy~=Aθ^.c = \sum_{\mathrm{rows}}\psi\tilde{y}=A\hat{\theta}.

Near-interpolating fits leave the condition number cond⁡(A)\operatorname{cond}(A) beyond 1/ϵmach1/\epsilon_{\mathrm{mach}} (ϵmach\epsilon_{\mathrm{mach}} the float64 unit roundoff), so the evidence and the posterior are computed by QR factorisation. RwR_w is the triangular factor of the weighted design (Φw=QwRw\Phi^w=Q_wR_w, rows bwb^w), accumulated by streamed updating QRs, and

\begin{bmatrix} \Lambda^{1/2}\\ R_w/\sigma \end{bmatrix} =Q_A R_A, \tag*{(12)}\end{bmatrix}
RATRA=A,(13)R_A^{\mathrm{T}}R_A=A, \tag*{(13)}

which works at cond⁡(Φ)\operatorname{cond}(\Phi) instead of cond⁡(Φ)2\operatorname{cond}(\Phi)^2.

Sensitivity to the prior: automatic relevance determination

Automatic relevance determination (ARD) replaces the single σc\sigma_c by one log-precision per body order,

Λ=Ddiag⁡(eak(j))D,(14)\Lambda= D\operatorname{diag}\left(e^{a_{k(j)}}\right)D, \tag*{(14)}

with k(j)k(j) the body order of basis function jj and DD the smoothness weights of § 1.1.2. The BLR fit is ARD at ak=−2log⁡σca_k=-2\log\sigma_c for every kk. The aka_k are fitted by type-II maximum likelihood in the scaled system S=D−1AD−1S=D^{-1}AD^{-1}, whose prior part is diag⁡(eak(j))\operatorname{diag}(e^{a_{k(j)}}). The scaling is by DD, not by Λ1/2\Lambda^{1/2}, because DD does not change with the aka_k: the data part of SS is computed once, and only its diagonal changes during the maximisation. In sequential mode, used here, the noise scale σ\sigma stays at the BLR fit's value and only the aka_k are fitted.

QR evidence. As for the BLR fit (§ 1.1.2), the evidence, its gradient and Hessian, and the posterior are computed from the QR factor

\begin{bmatrix} \operatorname{diag}\left(e^{a_{k(j)}/2}\right)\\ R_wD^{-1}/\sigma \end{bmatrix} =Q_S R_S, \tag*{(15)}\end{bmatrix}
RSTRS=S,(16)R_S^{\mathrm{T}}R_S=S, \tag*{(16)}

whose condition number is cond⁡(S)\sqrt{\operatorname{cond}(S)}. The posterior stores RSTR_S^{\mathrm{T}} as its Cholesky factor, so the CR3 jackknife and the per-atom force variances are unchanged. The aka_k are bounded below at afloor=log⁡(λmax⁡/1022)a_{\mathrm{floor}}=\log(\lambda_{\max}/10^{22}), with λmax⁡\lambda_{\max} the largest eigenvalue of the data part of SS (D−1ΨTΨD−1D^{-1}\Psi^{\mathrm{T}}\Psi D^{-1}) at the starting noise scale, and an endpoint with every aka_k on the floor and the gradient pointing below it is flagged as degenerate and refused.

Neither fit is degenerate. For o3d12 only the two-body precision sits on the floor, and for o4d20 every aka_k is interior. ARD weakens the prior relative to the BLR's single σc\sigma_c on most body orders (on all of them for o3d12; for o4d20 only the 5-body block is held more tightly), yet the means match the BLR fits on the held-out test split, and the test-split scales and the propagated coverage agree.

QoIDFTunittoleranceACE o3d12 valueerrorerror (%)ACE o4d20 valueerrorerror (%)
C11C_{11}153.3GPa10 %130.3−23.0-23.0−15.0-15.0151.3−2.0-2.0−1.3-1.3
C12C_{12}56.3GPa10 %60.7+4.5+4.5+8.0+8.055.2−1.1-1.1−1.9-1.9
C44C_{44}72.2GPa10 %73.4+1.2+1.2+1.7+1.770.6−1.6-1.6−2.2-2.2
bulk modulus BB88.6GPa10 %83.9−4.7-4.7−5.3-5.387.2−1.4-1.4−1.6-1.6
E(hexagonal diamond) −- E(diamond)11.5meV/atom10 meV/atom13.0+1.6+1.6+13.5+13.511.1−0.4-0.4−3.1-3.1
E(bc8) −- E(diamond)163.0meV/atom10 meV/atom161.7−1.3-1.3−0.8-0.8159.9−3.1-3.1−1.9-1.9
E(st12) −- E(diamond)172.5meV/atom10 meV/atom155.4−17.0-17.0−9.9-9.9168.5−4.0-4.0−2.3-2.3
E(β\beta-Sn) −- E(diamond)321.7meV/atom10 meV/atom319.6−2.1-2.1−0.6-0.6320.6−1.1-1.1−0.3-0.3
E(simple hexagonal) −- E(diamond)344.1meV/atom10 meV/atom340.0−4.1-4.1−1.2-1.2341.6−2.5-2.5−0.7-0.7
E(hcp) −- E(diamond)561.4meV/atom10 meV/atom562.7+1.4+1.4+0.2+0.2560.3−1.0-1.0−0.2-0.2
E(bcc) −- E(diamond)572.2meV/atom10 meV/atom541.4−30.8-30.8−5.4-5.4573.1+0.9+0.9+0.2+0.2
E(fcc) −- E(diamond)589.7meV/atom10 meV/atom569.9−19.7-19.7−3.3-3.3592.9+3.3+3.3+0.6+0.6
vacancy formation3.673eV0.1 eV2.792−0.881-0.881−24.0-24.03.562−0.111-0.111−3.0-3.0
tetrahedral interstitial formation3.910eV0.1 eV2.172−1.738-1.738−44.5-44.53.311−0.599-0.599−15.3-15.3
hexagonal interstitial formation3.724eV0.1 eV2.277−1.448-1.448−38.9-38.93.312−0.412-0.412−11.1-11.1
dumbbell interstitial formation3.662eV0.1 eV2.481−1.181-1.181−32.2-32.23.290−0.372-0.372−10.1-10.1
(100) surface energy, relaxed2.173J/m2^25 %2.108−0.064-0.064−3.0-3.02.133−0.040-0.040−1.8-1.8
unstable stacking fault, glide1.742J/m2^210 %1.819+0.077+0.077+4.4+4.41.870+0.128+0.128+7.3+7.3
unstable stacking fault, shuffle1.608J/m2^210 %1.021−0.587-0.587−36.5-36.51.461−0.147-0.147−9.2-9.2
maximum (100) decohesion stress, unrelaxed0.189eV/Å3^310 %0.200+0.011+0.011+5.7+5.70.184−0.005-0.005−2.6-2.6
phonon frequencies, RMS against DFT−-THz0.2 THz0.286−-−-0.153−-−-

Table 1. The silicon QoIs: DFT (CASTEP) reference value, unit, benchmark tolerance, and the value, error (model − DFT) and relative error (error / DFT) of the two ACE models. A tolerance in % is relative to the DFT value. The 10 meV/atom phase-energy tolerance is most of the small hexagonal-diamond value; the relative error is the more telling measure there. The phonon entry is the RMS difference of the phonon frequencies from DFT, so it has no DFT value or error and is not scored for coverage.

modelARD log-precision aka_k (body order kk) 2345ARD floorBLR aaE RMSE (meV/atom) ARDBLRF RMSE (eV/Å) ARDBLRV RMSE (meV/atom) ARDBLRtest-split λF\lambda_F ARDcoverage, 95 % (68 %) ARDBLR
o3d12−24.5∗-24.5^*−20.6-20.6−18.7-18.7−-−24.5-24.5−18.4∗-18.4^*15.8915.900.15880.1589112.9113.25.885.8217 (14)17 (14)
o4d20−12.0-12.0−12.3-12.3−8.7-8.7−6.4-6.4−14.1-14.1−7.8-7.83.383.500.08190.081932.432.66.186.1715 (12)15 (12)

Table 2. ARD against the benchmark BLR fits. ARD log-precision per body order at the endpoint (* on the floor), the floor, and the single log-precision of the BLR prior (−2log⁡σc-2 \log\sigma_c; * on the upper bound σc=104\sigma_c = 10^4 of the evidence search); test-split RMSE (E and V per atom) over the rows the calibration keeps, without the force components that vanish by site symmetry, which is why the force RMSE is slightly above the benchmark's value in Section 2.1; test-split λF\lambda_F; and the propagated CR3 ×\times λF\lambda_F coverage of the 20 scored QoIs.

Every result in this paper, per-atom and QoI, therefore uses one posterior per model: the BLR fit, which is the benchmark model. ARD is reported only as this check. The per-atom calibration of Section 2.5 run on the ARD fits covers 0.87 (o3d12) and 0.90 (o4d20) of the test atoms at a nominal 0.90, against 0.87 and 0.90 on the BLR fits (Section 3.2). Each δi\delta_i stays finite while the model is misspecified.

Comparison. CR3 below is built from the same per-row influence, A−1ψi/riA^{-1}\psi_i/r_i, but asks the opposite question: how far the fit moves when a configuration is removed, not how far it must move to fit a row exactly. For one row the two differ only in the leverage factor, 1/(1−hi)1/(1-h_i) for deletion against 1/hi1/h_i for an exact fit. CR3 is therefore largest for rows the fit leans on, and POPS for rows the fit hardly sees, whose corrections are unbounded as hi→0h_i \to0 (Section 3.7). CR3 also deletes whole configurations, so that the correlated energy, force and virial rows of one configuration move together, and it is a covariance that is calibrated by one scale on held-out data (Section 3.3); POPS has no calibration step. Like any sampling variance, the CR3 covariance shrinks as data are added, which is why the scale is needed: CR3 supplies the shape, and the held-out data the size.

Misspecification-robust shape: the CR3 jackknife

POPS. The closest existing approach for linear potentials is POPS³. For each training row ii, with residual rir_i and leverage hi=ψiTA−1ψih_i = \psi_i^\mathsf{T} A^{-1}\psi_i, the pointwise optimal parameter set is θ^+δi\hat{\theta} + \delta_i, with δi=A−1ψiri/hi\delta_i = A^{-1}\psi_i r_i/h_i the smallest change (in the AA metric) that makes the model reproduce that row exactly. The spread of the δi\delta_i, as an ensemble or as a hypercube in their principal directions, is the uncertainty. It does not shrink as data are added, because

CR3. For a cluster k=1,…,nclk = 1,\ldots,n_{\rm cl} of rows (here: one whole configuration with its energy, force and virial rows) with whitened rows Ψk\Psi_k (the cluster's rows ψT\psi^{\mathsf{T}} stacked), whitened targets y~k\tilde{y}_k and residuals ρk=y~k−Ψkθ^\rho_k = \tilde{y}_k - \Psi_k\hat{\theta}, the cluster hat block and the leverage-corrected (PRESS) score are

Hkk=ΨkA−1ΨkT=WkTWk,(17)H_{kk} = \Psi_k A^{-1}\Psi_k^{\mathsf{T}} = W_k^{\mathsf{T}}W_k, \tag*{(17)}
Wk=RS−TD−1ΨkT,(18)W_k = R_S^{-\mathsf{T}}D^{-1}\Psi_k^{\mathsf{T}}, \tag*{(18)}
g~k=ΨkT(I−Hkk)−1ρk.(19)\tilde{g}_k = \Psi_k^{\mathsf{T}}(I-H_{kk})^{-1}\rho_k. \tag*{(19)}

Each object has a direct reading.

  • HkkH_{kk} is the cluster's block of the hat matrix: it maps the cluster's targets to its own fitted values, so it measures how far the fit is pulled towards that configuration. Because Λ≻0\Lambda\succ0, its eigenvalues lie in [0,1)[0,1). An eigenvalue near 0 means the configuration is predicted by the rest of the data; one near 1 means the fit nearly interpolates it along that direction, and its residual there underestimates the error it would have if it were left out.

  • WkW_k is the cluster's rows expressed in whitened coefficient coordinates, scaled by the smoothness weights (DD) and by the Cholesky factor of SS (§ 1.1.3). In these coordinates the posterior is isotropic, so Hkk=WkTWkH_{kk} = W_k^{\mathsf{T}}W_k is a plain Gram matrix; computing WkW_k costs one triangular solve per cluster.

  • g~k\tilde{g}_k is the cluster's contribution to the gradient of the least-squares objective, ΨkTρk\Psi_k^{\mathsf{T}}\rho_k, evaluated with its leave-cluster-out (PRESS) residuals (I−Hkk)−1ρk(I-H_{kk})^{-1}\rho_k instead of its ordinary residuals. It is the pull of configuration kk on the coefficients as an unseen configuration would exert it.

With the hyperparameters fixed, deleting cluster kk moves the mean by exactly θ^−θ^(−k)=A−1g~k\hat{\theta}-\hat{\theta}_{(-k)} = A^{-1}\tilde{g}_k (the cluster

Figure 2. The CR3 construction on the Lennard-Jones (LJ) example of the interactive regression demo[33] (scene lj-cr3), computed exactly. (a) A degree-5 polynomial, fitted by Bayesian ridge regression with evidence-optimised hyperparameters to 197 nearly noiseless samples (noise standard deviation 0.001, as for DFT labels) of the smooth part of a Lennard-Jones curve, cannot follow the steep repulsive side and the well: the error is approximation error, not noise. The posterior band (orange; σ2A−1\sigma^2 A^{-1} in the demo's unwhitened rows) is barely visible. The CR3 band (blue), scaled by λ\lambda from 50 held-out calibration points, is wide where the fit is wrong: on the steep side near x=0x = 0 and at the right-hand end. (b) Each dot is the change of the fitted curve when one training point is left out, −A−1g~k-A^{-1}\widetilde{g}_k, in the two leading principal curves of the CR3 covariance of the prediction; the closed form reproduces the refit to the precision the basis allows. The points on the steep side and in the well (x<1x < 1) move the fit most. The CR3 covariance is the sum of these shifts' outer products over the 197 training points, so its ellipse is about 197\sqrt{197} times the spread of single dots; the posterior ellipse is smaller. (c) The computation for the real models, with the reading of each quantity: one fit, then per configuration its leverage, its leave-out residuals and its pull, and finally the spread of the pulls.

DFBETA[20] (§ 3.2.4). The spread of these shifts over clusters is the CR3 covariance. The centred meat and the delete-one-cluster (CR3) covariance are

M~=∑k(g~k−gˉ)(g~k−gˉ)T,(20)\widetilde{M} = \sum_k (\widetilde{g}_k - \bar{g})(\widetilde{g}_k - \bar{g})^{\mathsf{T}}, \tag*{(20)}
ΣCR3=A−1M~A−1=D−1Q~Q~TD−1,(21)\Sigma_{\mathrm{CR3}} = A^{-1}\widetilde{M}A^{-1} = D^{-1}\widetilde{Q}\widetilde{Q}^{\mathsf{T}}D^{-1}, \tag*{(21)}
Q~=S−1D−1[q~1−qˉ,…].(22)\widetilde{Q} = S^{-1}D^{-1}[\widetilde{q}_1-\bar{q},\ldots]. \tag*{(22)}

with gˉ=ncl−1∑kg~k\bar{g} = n_{\mathrm{cl}}^{-1}\sum_k \widetilde{g}_k. Centring removes the common offset of the pulls: for the ordinary pulls ΨkTρk\Psi_k^{\mathsf{T}}\rho_k it is the prior's gradient (∑kΨkTρk=Aθ^\sum_k \Psi_k^{\mathsf{T}}\rho_k = A\hat{\theta} at the MAP); the leverage-corrected g~k\tilde{g}_k are centred on their own mean. ΣCR3\Sigma_{\mathrm{CR3}} is a Huber–White sandwich. With one residual variance s2s^2 for every row, the expected uncorrected meat is s2(A−Λ)s^2(A-\Lambda), so the sandwich is s2A−1(A−Λ)A−1s^2A^{-1}(A-\Lambda)A^{-1}, which tends to s2A−1s^2A^{-1} for a weak prior: the homoscedastic special case. It is stored as a factor RR with RRT=Q~Q~TRR^{\mathsf{T}}=\widetilde{Q}\widetilde{Q}^{\mathsf{T}} (rank ≤\le number of clusters), so that

Σ=LLT,(23)\Sigma= LL^{\mathsf{T}}, \tag*{(23)}
L=D−1R∈Rp×r,(24)L = D^{-1}R \in\mathbb{R}^{p \times r}, \tag*{(24)}

with rr the rank. For any observable row bb, σ(b)=λ∥LTb∥\sigma(b) = \lambda\lVert L^{\mathsf{T}}b\rVert, where λ\lambda is a scale calibrated on held-out data (§ 3.2.1). The CR3 covariance measures how much a prediction depends on which configurations were in the fit: large where the fit leans on configurations it reproduces poorly. It is a parameter variance used as a proxy for approximation error, which is why it needs a scale.

Calibrated per-atom forces: the CR3 shape and one conformal quantile

This section describes the per-atom force uncertainty introduced in this work. Every predicted force gets a calibrated uncertainty: a shape times one scale. The shape is the per-atom 3×33 \times3 block of the CR3 covariance, V(x)αβ=uαTRRTuβV(x)_{\alpha\beta} = u_\alpha^{\mathsf{T}}RR^{\mathsf{T}}u_\beta with uα=D−1BF,iαTu_\alpha= D^{-1}B_{F,i\alpha}^{\mathsf{T}} built from the atom's three force rows (α∈{x,y,z}\alpha\in\{x,y,z\}), and v=tr⁡Vv = \operatorname{tr} V. The scale is one conformal quantile of scores on held-out training configurations.

Hold-out. Training configurations are split 80/20 into TfitT_{\mathrm{fit}} and TvalT_{\mathrm{val}}, stratified so that rare, strongly distorted or off-coordinated environments appear in both parts (the ace-jax default). The posterior PfitP_{\mathrm{fit}} on TfitT_{\mathrm{fit}} keeps the prior and noise scale of the full BLR fit (they are not refitted on TfitT_{\mathrm{fit}}), so it differs from the final posterior only in its data. It has its own CR3 shape VfitV_{\mathrm{fit}}, predicts TvalT_{\mathrm{val}}, so both the force error ei=FiDFT−Fifit∈R3e_i = F_i^{\mathrm{DFT}} - F_i^{\mathrm{fit}} \in\mathbb{R}^3 and the shape are out of sample. The anisotropic score is

si=eiT(Vfit(xi)+ϵiI)−1ei,(25)s_i = \sqrt{e_i^{\mathsf{T}}\left(V_{\mathrm{fit}}(x_i) + \epsilon_i I\right)^{-1}e_i}, \tag*{(25)}
ϵi=10−313tr⁡Vfit(xi),(26)\epsilon_i = 10^{-3}\frac{1}{3}\operatorname{tr} V_{\mathrm{fit}}(x_i), \tag*{(26)}

and s/λ∼χ3s/\lambda\sim\chi_3 (the chi distribution with three degrees of freedom) under e∼N(0,λ2V)e \sim\mathcal{N}(0,\lambda^2 V).

Transfer to the full training set. The final posterior is fitted on all of TT, about 25 % more data than PfitP_{\mathrm{fit}}. A second hold-out fit Pfit2P_{\mathrm{fit2}} on a subset of TfitT_{\mathrm{fit}} scores the same TvalT_{\mathrm{val}} atoms. With NN, NfitN_{\mathrm{fit}}, Nfit2N_{\mathrm{fit2}} the numbers of configurations behind the final, first and second fits, and λ1,λ2\lambda_1,\lambda_2 the RMS scores of PfitP_{\mathrm{fit}} and Pfit2P_{\mathrm{fit2}}, β=clip⁡(log⁡(λ1/λ2)log⁡(Nfit/Nfit2),0,12)\beta= \operatorname{clip}\left(\frac{\log(\lambda_1/\lambda_2)}{\log(N_{\mathrm{fit}}/N_{\mathrm{fit2}})},0,\frac{1}{2}\right), and every score is multiplied by ftr=(N/Nfit)βf_{\mathrm{tr}} = (N/N_{\mathrm{fit}})^\beta. β=0\beta= 0 is the variance-dominated limit and 12\frac{1}{2} the bias-dominated one.

One posterior. The final posterior of this stage is the BLR posterior that the QoI propagation uses (§ 2): ace-jax computes the stage at fixed hyperparameters (--ard-prior fixed, ARD at the BLR's single a=−2log⁡σca = -2 \log\sigma_c, § 1.1.3) and the QR solver. Against the CR3 posterior built independently at the BLR fit, its mean differs by 2×10−62 \times10^{-6} (o3d12) and 8×10−108 \times10^{-10} (o4d20) in relative norm, and its CR3 covariance by 4×10−64 \times10^{-6} and 3×10−83 \times10^{-8}.

The scale. The conformal quantile qq is the ⌈(n+1)(1−α)⌉/n\lceil(n+1)(1-\alpha)\rceil/n empirical quantile of the nn scores sis_i of the TvalT_{\mathrm{val}} atoms, at level 1−α1-\alpha (default α=0.1\alpha= 0.1). It is atom-weighted: a quantile that counts each configuration once in total is dominated by the many small, well-predicted cells and under-covers (§ 3.2).

Per-atom outputs. The conformal region for a force error ΔF\Delta F is {ΔF:ΔFT(V+ϵI)−1ΔF≤q2}\{\Delta F : \Delta F^{\mathsf{T}}(V + \epsilon I)^{-1}\Delta F \le q^2\}, the CR3 ellipsoid scaled to the quantile. Under exchangeability of the calibration and target configurations it covers a new atom with probability ≈1−α\approx1-\alpha. The Gaussian reading is the force standard deviation σF,i=(q/χ3−1(1−α))v\sigma_{F,i} = (q/\chi_3^{-1}(1-\alpha))\sqrt{v}, with χ3−1\chi_3^{-1} the χ3\chi_3 quantile function, and covariance σF,i2V/v\sigma_{F,i}^2 V/v.

A learned scale, as a refinement. The CR3 shape is a quadratic form in the atom's force rows, so it already grows with the distance of the environment from the training descriptors. In that it is related to the D-optimality extrapolation grade used to select training data for linear potentials^{34,35}, a leverage of the atom's descriptor against the training set; the CR3 shape weights that leverage by the residuals the fit leaves, configuration by configuration, and the quantile calibrates it to a coverage. A scale that depends on the environment should therefore model the ratio of the error to the shape, the score ss, not novelty itself. We test one learned from the site descriptor (normalised conformal prediction^{21,22}). Let ϕi\phi_i be atom ii's whitened principal components of the normalised site descriptor, from a principal component analysis of training atoms only. On one half of the TvalT_{\mathrm{val}} configurations a ridge regression f(ϕ)=β0+βTϕ+γTϕ2f(\phi) = \beta_0 + \beta^{\mathsf{T}}\phi+ \gamma^{\mathsf{T}}\phi^2 of log si2s_i^2 is fitted (each configuration weighted once, the ridge by five-fold cross-validation over configurations) and clipped to the range it takes on the calibration atoms. On the other half the normalised scores sie−f(ϕi)/2s_i e^{-f(\phi_i)/2} give one quantile qfq_f, and the region is s≤qfef(ϕ)/2s \le q_f e^{f(\phi)/2}. The split is repeated over five seeds. Ho, Ortner and Wang^{23} learn an environment-dependent scale in the same spirit: a neural network of MACE descriptors that supplies a last-layer uncertainty, fitted with a weighted absolute-error loss rather than as a conformal quantile.

Why the propagation does not use this scale. The conformal quantile tempers force outputs atom by atom. It does not define a covariance in θ\theta, and a QoI depends on every atom and on relaxed geometries. Propagation therefore uses one global λ\lambda (§ 3.2.1). § 3.2 tests the per-atom force uncertainties directly.

Baselines

We compare the CR3 propagation with three baselines. Each changes one ingredient, the shape of the covariance or the way its size is set, so that the comparison shows which ingredient matters. The first baseline, B1, is the approach of Best et al.^{11}. It keeps the Bayesian posterior as the shape, Σ=A−1\Sigma= A^{-1}, and corrects its size with one conformal quantile q^\hat q. The quantile is taken from the absolute standardised scores ∣z∣|z| of the held-out test split (§ 3.2.1, here computed with the posterior shape) at the 68.27 % level, so that ±q^σ\pm\hat{q}\sigma is the 1σ1\sigma interval, pooled over energies, forces and virials, with every configuration given equal weight. B1 therefore uses the same held-out data as the global scale of our method. It differs mainly in the shape, so it tests whether the CR3 covariance describes the errors better than the posterior; it also forms its single scale differently (a quantile of ∣z∣|z| pooled over E, F and V, against an RMS over force components).

The second baseline, B2, is POPS3^{3}, as implemented in ace-jax24^{24} by a port of the reference package POPSRegression36^{36}. For each training row ii the correction is δi=Aμ−1ψiwri/hi\delta_i=A_{\mu}^{-1}\psi_i^w r_i/h_i, with leverage hi=ψiwTAμ−1ψiwh_i=\psi_i^{w\mathsf{T}}A_{\mu}^{-1}\psi_i^w. Here ψiw=biwi\psi_i^w=b_iw_i carries the row weights but no noise scale, rir_i is the row's residual at the ridge solution, and Aμ=M+μdiag⁡(Γ2)A_{\mu}=M+\mu\operatorname{diag}(\Gamma^2), with M=∑iψiwψiwTM=\sum_i\psi_i^{w}\psi_i^{w\mathsf{T}} and the ridge parameter μ\mu chosen on a path. Members are drawn uniformly in the hypercube spanned by the principal directions of the δi\delta_i, with bounds from their extreme projections, as in Swinburne and Perez3^{3}. Following the reference package, the box is built only from the rows at or above the median leverage (leverage_percentile = 50, its default; a box from every row is unphysical here, § 3.2.5). The ridge is selected per quantity by the CRPS on a hold-out of the training set, over relative ridges from 10−210^{-2} down to 10−1010^{-10} of the largest eigenvalue, a decade above the relative ridge of about 10−1110^{-11} below which the box is set by roundoff (§ 3.2.5). We use ace-jax24^{24} version 0.5.3, which includes the numerical corrections that this study prompted. B2 has no calibration step, so it tests a misspecification-aware shape without a fitted scale.

The third baseline, B3, is the CR3 covariance with λ=1\lambda=1. It isolates the effect of the scale: the difference between B3 and the headline is entirely the calibration of § 3.2.1.

Fisher et al.37^{37} take the conformal step further. They calibrate joint conformal sets for energies, forces and virials (Bonferroni-corrected boxes, Mahalanobis ellipsoids, and losses under conformal risk control) and propagate them to derived quantities such as elastic constants and vacancy formation energies, where modelling the correlations captures error cancellation. B1 and our CR3 method use one scalar scale instead. We do not use multivariate conformal prediction here; combining it with the CR3 shape is left to future work.

Propagation to QoIs

The distribution propagated is θ∼N(θ^,λ2LLT)\theta\sim\mathcal{N}(\hat{\theta},\lambda^2L L^{\mathsf{T}}); one sample gives energy, forces and stress consistently. Per-atom force ellipsoids are never combined.

The QoIs

The 21 QoIs are those of the silicon test suite of Bartók et al.29^{29}, with their DFT (CASTEP) reference values, run with the tolerances of the ace-jax silicon benchmark (a Python 3 port of that test framework): C11C_{11}, C12C_{12}, C44C_{44}, BB; eight phase energies relative to diamond (compressed hcp excluded as extrapolation); vacancy and three interstitial formation energies (216-atom cells); the relaxed (100) surface energy; the maximum unrelaxed decohesion stress; the glide and shuffle unstable stacking-fault energies; the phonon RMS against DFT. Every protocol starts from the model's own relaxed diamond cell, so no QoI is linear in θ\theta. The phonon RMS is a distance to DFT and has no DFT value: it is left out of coverage (n=20n=20).

Delta method by implicit differentiation

Implicit differentiation of a relaxed minimum with respect to the parameters of an interatomic potential is the method of Maliyev, Grigoriev and Swinburne31^{31}; here it is applied to every relaxation of the benchmark, the lattice constant included. Write the energy of a structure with coordinates xx (positions, and cell where relaxed) as E=bE(x)+θTE0(x)E=b_E(x)+\theta^{\mathsf{T}}E^0(x), with bE(x)b_E(x) its energy row (§ 1.1.1) and E0(x)E^0(x) the reference energies. A QoI Q(θ)Q(\theta) is computed at a relaxed geometry x∗(θ)x^*(\theta) solving ∇xE=0\nabla_xE=0, so

dx∗dθ=−H−1∂∇xE∂θ=−H−1(∂bE∂x)T,(27)\frac{d x^*}{d\theta}=-\mathcal{H}^{-1}\frac{\partial\nabla_xE}{\partial\theta}=-\mathcal{H}^{-1}\left(\frac{\partial b_E}{\partial x}\right)^{\mathsf{T}}, \tag*{(27)}
dQdθ=∂Q∂θ+∂Q∂xdx∗dθ,(28)\frac{dQ}{d\theta}=\frac{\partial Q}{\partial\theta}+\frac{\partial Q}{\partial x}\frac{d x^*}{d\theta}, \tag*{(28)}

with H=∇x2E\mathcal{H}=\nabla_x^2E the Hessian at x∗(θ^)x^*(\hat{\theta}). Then σQ=λ∥LTG∥\sigma_Q=\lambda\lVert L^{\mathsf{T}}G\rVert, with G=dQ/dθG=dQ/d\theta the QoI's row.

  • Lattice constant. The diamond lattice constant a0a_0 makes the diamond-cell energy Edia(a)E_{\mathrm{dia}}(a) stationary, so

da0dθ=−1Eaa∂bE,dia∂a,(29)\frac{d a_0}{d\theta}=-\frac{1}{E_{aa}}\frac{\partial b_{E,\mathrm{dia}}}{\partial a}, \tag*{(29)}
Eaa=d2Ediada2∣a0.(30)E_{aa}=\left.\frac{d^2E_{\mathrm{dia}}}{d a^2}\right|_{a_0}. \tag*{(30)}
  • Energies at a minimum (phase, defect, surface energies). By the envelope theorem the relaxed coordinates add nothing to the derivative of the relaxed energy E∗E^*: it is the energy row at x∗x^* plus the response of the cell the protocol builds from a0a_0,

dE∗dθ=bE(x∗)+dEda0da0dθ,(31)\frac{dE^*}{d\theta}=b_E(x^*)+\frac{dE}{d a_0}\frac{d a_0}{d\theta}, \tag*{(31)}
dEda0=1a0θ⋅∂bE∂ln⁡a=−tr⁡Wa0,(32)\frac{dE}{d a_0}=\frac{1}{a_0}\theta\cdot\frac{\partial b_E}{\partial\ln a}=-\frac{\operatorname{tr}W}{a_0}, \tag*{(32)}

with WW the virial of the relaxed structure. The relaxed geometries are the harness's own, recorded for the mean model. For example, the vacancy formation energy has the row

G=bvac+1a0(θ⋅∂bvac∂ln⁡a)da0dθ−N−1Nbbulk,(33)G=b_{\mathrm{vac}}+\frac{1}{a_0}\left(\theta\cdot\frac{\partial b_{\mathrm{vac}}}{\partial\ln a}\right)\frac{d a_0}{d\theta}-\frac{N-1}{N}b_{\mathrm{bulk}}, \tag*{(33)}

with bvacb_{\mathrm{vac}} and bbulkb_{\mathrm{bulk}} the energy rows of the relaxed vacancy cell and the NN-atom bulk cell (the bulk cell has zero virial at a0a_0, so it has no a0a_0 term).

  • Elastic constants, relaxed-ion. At each applied strain the internal coordinates uu (atomic positions in the strained cell) are relaxed, and the stress τ=−W/Ω\tau=-W/\Omega is linear in θ\theta at fixed geometry, τ=Bτθ\tau=B_{\tau}\theta with Bτ=−BV/ΩB_{\tau}=-B_V/\Omega. Then

dτdθ=Bτ+∂τ∂udu∗dθ+∂τ∂ada0dθ,(34)\frac{d\tau}{d\theta}=B_{\tau}+\frac{\partial\tau}{\partial u}\frac{d u^*}{d\theta}+\frac{\partial\tau}{\partial a}\frac{d a_0}{d\theta}, \tag*{(34)}
du∗dθ=−Huu−1[(∂bE∂u)+∂a∇uE⊗da0dθ],(35)\frac{d u^*}{d\theta}=-H_{uu}^{-1}\left[\left(\frac{\partial b_E}{\partial u}\right)+\partial_a\nabla_u E\otimes\frac{d a_0}{d\theta}\right], \tag*{(35)}

with HuuH_{uu} the Hessian in uu. The elastic constants are linear fits of τ\tau against strain, so their rows are the same fits of these. Stress route: the harness's matscipy protocol (Voigt strain pattern e1+e4e_1+e_4, 5 steps of ±1%\pm1\%, linear fits). Energy route: curvature at the same steps of patterns e1e_1, e4e_4, e1+e2e_1+e_2.

  • Decohesion. Rigid openings of the relaxed bulk: linear in θ\theta at fixed a0a_0 plus the a0a_0 term; the cubic-spline derivative is linear in the energies.

  • Discrete steps (E(V)E(V)-grid argmin, maximum over a grid) are held at their index; γus\gamma_{\mathrm{us}} and phonons are committee-only. So is the bcc phase energy: for both models its E(V)E(V) minimum is not at the grid point of the freely relaxed volume, so the selected point changes with θ\theta and has no derivative. The delta method therefore covers 17 of the 20 scored QoIs.

Rows are checked against finite differences in θ\theta (to ∼10−5\sim10^{-5}) and the values they imply reproduce the harness to ∼10−10\sim10^{-10} (energy QoIs) and 0.04 GPa (elastic constants).

Committees

K=200K=200 draws θk=θ^+Lξk\theta_k=\hat{\theta}+L\xi_k, ξk∼N(0,Ir)\xi_k\sim N(0,I_r) (fixed seeds), are run through the unmodified benchmark harness. Per the protocol, the draws are made once at λ=1\lambda=1 and QoI deviations are rescaled, Qk(λ)=Q^+λ(Qk−Q^)Q_k(\lambda)=\hat{Q}+\lambda(Q_k-\hat{Q}), with QkQ_k member kk's QoI and Q^\hat{Q} the mean model's; a full re-run at λF\lambda_F tests that linearisation. In that re-run a member is rejected, by a rule fixed beforehand that uses no DFT value, if a test fails, if any relaxed interatomic distance falls below 2.0 Å (collapse), or if its diamond cell is Born-unstable. The headline σ\sigma is the delta-method σ\sigma where it agrees with the λ=1\lambda=1 committee to 10 %, else the committee's.

Measuring the bias: a discrepancy term

A bias shared by every training configuration of a kind is invisible to any delete-one jackknife: deleting one configuration does not move the fit along it. The defect formation energies expose such a bias. Every training interstitial, vacancy and divacancy cell is a hot molecular-dynamics snapshot, while the QoIs are relaxed defects.

Measuring the bias from training labels. For each defect class t∈{interstitial,vacancy,divacancy}t\in\{\text{interstitial},\text{vacancy},\text{divacancy}\}, the bulk reference is a per-atom thermal trend eb(η)=β0+β1ηe_b(\eta)=\beta_0+\beta_1\eta, fitted by least squares to the residuals per atom of the diamond bulk cells within 0.6 Å3^3/atom of the defect cells' mean volume, with η=Frms2\eta=F_{\mathrm{rms}}^2 the mean squared DFT force component of a cell (proportional to the thermal mean-square displacement for a harmonic solid). Each training defect cell ii, with nin_i atoms, energy row bib_i and energy label yiy_i, then gives a bulk-subtracted residual ui=yi−bi⋅θ^−nieb(ηi)u_i=y_i-b_i\cdot\hat{\theta}-n_i e_b(\eta_i). Regressing uiu_i on ηi\eta_i and extrapolating to η→0\eta\to0 estimates the model's error for a relaxed defect. The intercept is linear in the labels and in θ^\hat{\theta}, so it is a synthetic observation:

rt∗=∑iaiui=yt∗−dt∗⋅θ^,(36)r_t^*=\sum_i a_i u_i=y_t^*-d_t^*\cdot\hat{\theta}, \tag*{(36)}
a=e1T(XTX)−1XT,(37)a=e_1^{\mathsf{T}}(X^{\mathsf{T}}X)^{-1}X^{\mathsf{T}}, \tag*{(37)}
X=[1,ηi],(38)X=[1,\eta_i], \tag*{(38)}

with XX the regression design (rows [1,ηi][1,\eta_i]), e1=(1,0)Te_1=(1,0)^{\mathsf{T}} selecting the intercept, dt∗d_t^* and yt∗y_t^* the same combination of the defect rows and labels with the bulk-subtracted rows and labels subtracted, and standard error st∗s_t^* from the regression scatter. No QoI value or tolerance enters.

Training-cell size. The training vacancy cells come in two sizes, 63 and 215 atoms, so the vacancy trend need not be shared. We compare four linear-Gaussian models of the vacancy residuals uiu_i by their evidence: one intercept and slope for both sizes; per-size intercepts with a shared slope; a shared intercept with per-size slopes; and per-size intercepts and slopes. The coefficients have Gaussian priors (intercepts N(0,sa2)N(0,s_a^2) with sa=1s_a=1 eV, slopes scaled so that a slope moves uu by about sas_a across the data) and are integrated analytically. Each size has its own noise level, log-uniform on [10−3,10][10^{-3},10] eV and integrated on a grid. The QoI vacancy sits in a 216-site cell, so its bias is the 215-atom intercept, averaged over the four models with their posterior probabilities (equal prior odds). This uses training labels only, as for the interstitials.

Discrepancy term. For the QoIs of defect class tt, the variance becomes σQ2=σQ,method2+rt∗2+st∗2\sigma_Q^2=\sigma_{Q,\mathrm{method}}^2+r_t^{*2}+s_t^{*2}, with σQ,method\sigma_{Q,\mathrm{method}} the σ\sigma of the uncertainty method without the term (interstitial class for the three interstitials; vacancy class for the vacancy). Every other QoI is unchanged. This leaves the pure parameter-space propagation for these QoIs and is reported as a separate, labelled method. The term widens σ\sigma but leaves the prediction at the potential's own value, on purpose: the uncertainty describes the potential that is run. Subtracting rt∗r_t^* from the prediction would correct the potential, a different object and one that no longer has a parameter vector; Table XIV gives the errors of that corrected prediction for reference.

modelQoIDFT − modelDFT − (model + r∗r^*)σ\sigmaσ\sigma with termzz
o3d12vacancy+0.881+0.6670.5450.601−1.62 →\to −1.47
o3d12dumbbell int.+1.181−0.3960.3511.617−3.37 →\to −0.73
o3d12hexagonal int.+1.448−0.1290.5251.663−2.76 →\to −0.87
o3d12tetrahedral int.+1.738+0.1610.8471.791−2.05 →\to −0.97
o4d20vacancy+0.111+0.1540.0840.106−1.33 →\to −1.05
o4d20dumbbell int.+0.372−0.0130.1770.425−2.10 →\to −0.88
o4d20hexagonal int.+0.412+0.0280.1760.424−2.34 →\to −0.97
o4d20tetrahedral int.+0.599+0.2140.2200.444−2.72 →\to −1.35

Table XIV. Defect QoIs, CR3 ×λF\times\lambda_F: error, error of the prediction corrected by r∗r^* (for reference; not used), σ\sigma without and with the discrepancy term (eV), and zz.

Nothing in this analysis is specific to the CR3 propagation. The bias rt∗r_t^* and its standard error come from the training labels and the fitted mean alone, so the same term can be added to the σQ\sigma_Q of any method that shares that mean, the baselines included.

Results

Validation statistics

Section 2 ends with a σ\sigma for every prediction we test: a per-atom force covariance and conformal region (§ 1.1.5), a scaled CR3 covariance in θ\theta (§ 1.1.4), and from it a σQ\sigma_Q for each QoI by the delta method or by committees, with or without the discrepancy term (§ 2 and § 2.0.2.2). The same few statistics judge all of them against held-out data or DFT. Two held-out sets are used, and they play different roles. The per-atom quantile comes from a hold-out of the training set, so the test split is an independent test of it. The propagation's global scale and B1's quantile are fitted on the test split's residuals, as the specification fixed (§ 3.2.1), so their per-atom coverage on the test split is in sample. No DFT QoI value enters any scale.

Sign conventions. An error in Table I and in zQz_Q below is model − DFT. The defect analysis (§ 3.2.6) works with training residuals, DFT−model, and its tables say so.

For the per-atom forces (§ 3.2), the test is the coverage of the conformal regions on the 248 test configurations, overall and per configuration type, together with the RMS of the observed force error over the predicted σF,i\sigma_{F,i}, which is 1 for a calibrated Gaussian reading. For the QoIs (§ 3.2.2–§ 3.2.6), each prediction gives a standardised error zQ=(Q^−QDFT)/σQz_Q = (\hat{Q} - Q_{\mathrm{DFT}})/\sigma_Q, with Q^\hat{Q} the mean model's value. We count how many of the 20 scored QoIs fall inside the Gaussian 68 % and 95 % intervals. Each count kk of nn gives a posterior for the coverage probability, Beta⁡(k+12,n−k+12)\operatorname{Beta}(k + \frac{1}{2}, n - k + \frac{1}{2}) from the Jeffreys prior, and we quote its 95 % credible interval and the posterior probability that the coverage reaches its nominal value. With only 20 QoIs these are broad: 17 of 20 at 95 % gives a credible interval of [0.65, 0.96], with P(coverage≥0.95)=0.04P(\mathrm{coverage} \ge0.95) = 0.04.

Counts treat the QoIs as independent, but they share the same coefficients, so their errors are correlated. § 3.2.3 therefore scores the whole vector of DFT values at once, by the evidence each method's predictive distribution gives it. With Δ\Delta the vector of (QDFT−Q^)/tolQ(Q_{\mathrm{DFT}}-\hat{Q})/\mathrm{tol}_Q over the scored QoIs and CQC_Q its predicted covariance in the same tolerance units (λ2GLLTGT\lambda^2 GLL^TG^T for the delta method, with GG the QoI rows stacked, or the committee covariance),

log⁡p(Δ∣method)=−12ΔT(CQ+ϵI)−1Δ−12log⁡∣2π(CQ+ϵI)∣(39)\log p(\Delta\mid\mathrm{method}) = -\frac{1}{2}\Delta^{T}(C_Q+\epsilon I)^{-1}\Delta- \frac{1}{2}\log\lvert2\pi(C_Q+\epsilon I)\rvert \tag*{(39)}

with a small ridge ϵ=10−6\epsilon= 10^{-6} (a floor of 10−310^{-3} tolerances on σ\sigma; 10−810^{-8} and 10−410^{-4} as sensitivities). Every method's scale is fixed beforehand, on held-out forces or held-out scores, so no method has a free parameter left, and the difference of two methods' log evidence on the same QoIs is a log Bayes factor. The first term rewards predictions whose errors, and their correlations, match the covariance; the second penalises intervals that are wider than they need to be. The evidence therefore weighs calibration and sharpness together, and a method cannot gain by inflating its intervals.

Coverage alone rewards intervals that are too wide. We therefore also report sharpness, the median of σQ/tolQ\sigma_Q/\mathrm{tol}_Q with tolQ\mathrm{tol}_Q the benchmark's tolerance for QoI QQ: a value below 1 means the interval is narrower than the accuracy the benchmark asks for. A useful method has near-nominal coverage at the smallest sharpness. The CRPS combines the two in one number per QoI[38]. For a Gaussian prediction with mean Q^\hat{Q} and standard deviation σQ\sigma_Q it is

CRPS=σQ[z(2Φ(z)−1)+2φ(z)−1π](40)\mathrm{CRPS} = \sigma_Q\left[z(2\Phi(z)-1)+2\varphi(z)-\frac{1}{\sqrt{\pi}}\right] \tag*{(40)}

with zz the standardised error and Φ\Phi, φ\varphi the standard normal distribution and density functions. It is a proper score: it is smallest in expectation for the true predictive distribution, it grows for intervals that are too narrow and for intervals that are too wide, and it tends to the absolute error as σQ→0\sigma_Q \to0. We report its mean over the scored QoIs in tolerance units, CRPS/tol (lower is better); committee methods are scored with the Gaussian of their spread, as for coverage.

Calibrated per-atom forces

Figure 3 and Table III test the per-atom force uncertainty of § 1.1.5 on the 248 test configurations, with the BLR fits whose CR3 posteriors the QoIs use; atoms whose force rows vanish by site symmetry are dropped by the rule of § 3.2.1. At a nominal 90 %, the CR3 shape with one hold-out quantile covers 0.87 of the test atoms for o3d12 and 0.90 for o4d20. o3d12 is below nominal, and the cause is the transfer from the hold-out fit to the full fit. The calibration scores come from the hold-out posterior, fitted on 1782 of the 2227 training configurations, and are carried to the full posterior by (N/Nfit)β(N/N_{\mathrm{fit}})^\beta, with β\beta estimated from a second, smaller hold-out fit (1427 configurations) and clipped to [0, 1/2] (§ 1.1.5). For o4d20 that estimate gave β=0.43\beta= 0.43, a factor 1.101, and the test scores' 90 % coverage is 0.991 times the transferred one. For o3d12 the scale of the smaller fit came out larger (8.56 against 7.03), so βraw=−0.88\beta_{\mathrm{raw}} = -0.88 was clipped to 0 and nothing was transferred; the full fit's test scores are 6.8 % larger at the 90 % quantile, which is the shortfall. We report this as found: the transfer rule was fixed before any test coverage was seen, and changing it now would be tuning on the test split. Where the transfer worked, for o4d20, one quantile reaches the nominal level overall with no environment-dependent scale: the shape carries most of the dependence of the error on the environment. The estimate of β\beta is fragile in one respect. It is taken from RMS scales, which a few atoms with almost no CR3 variance can dominate: in the out-of-distribution refits below, the first group's RMS scale is 3.6×1093.6 \times10^{9} (o4d20) where its 90 % quantile is 17.2, and that refit's β=0.23\beta= 0.23 (factor 1.053) reflects those atoms rather than the data size. An estimate from quantiles would be robust to them; as with the clipping, we keep the rule as it was fixed.

modelconfiguration typeatomsconfigsone quantilelearned scaleλF\lambda_FλV\lambda_V
o3d12amorph3304160.830.806.4–
o3d12vacancy2691210.820.886.04.9
o3d12interstitial2543120.800.867.53.8
o3d12dia2144490.940.914.14.2
o3d12divacancy171280.830.885.8–
o3d12bt1546441.000.943.814.6
o3d12liq57680.960.925.04.4
o3d12surface_11148050.970.935.2–
o3d12surface_111_pandey48050.950.915.4–
o3d12surface_00143230.950.905.5–
o3d12sh367130.990.924.14.4
o3d12surface_11032430.970.974.8–
o3d12RMS deviation from 0.900.0720.044
o4d20amorph3304160.990.934.5–
o4d20vacancy2691210.780.858.25.2
o4d20interstitial2543120.830.898.05.4
o4d20dia2144490.860.906.53.7
o4d20divacancy171280.860.896.9–
o4d20bt1546441.000.952.715.5
o4d20liq57681.000.943.02.5
o4d20surface_11148051.000.924.0–
o4d20surface_111_pandey48051.000.933.9–
o4d20surface_00143230.950.945.0–
o4d20sh367131.000.942.52.2
o4d20surface_11032431.000.993.9–
o4d20RMS deviation from 0.900.0870.041

Table III. Test coverage of the 90 % per-atom force region per GAP-18 configuration type (types with at least 300 test atoms): one hold-out quantile against the learned scale (mean over five hold-out splits), and the RMS deviation of the per-type coverage from nominal; and the test-split scales of the propagation, λF\lambda_F and λV\lambda_V = RMS zz with the CR3 shape over that type's force and virial rows (Section 3.3; "–": no virials).

The coverage is not even across environments, though. The point-defect cells are under-covered and the liquid and surface cells over-covered (Figure 3, bottom). The amorphous cells differ between the models: under-covered for o3d12 (0.83) and over-covered for o4d20 (0.99). The learned scale of § 1.1.5 corrects much of this in distribution: it covers 0.87 and 0.91 overall (ranges over the five hold-out splits 0.86–0.88 and 0.90–0.91), with similar median region sizes (17.2 against 17.1 for o3d12, 15.0 against 16.6 for o4d20, in units of the score), and the RMS deviation of the per-type coverage from 0.90 falls from 0.072 to 0.044 for o3d12 and from 0.087 to 0.041 for o4d20 (Table III; for o3d12 the amorphous cells are the exception). Out of distribution it is less robust, as the next test shows.

Figure 3. Top: binned on predicted σ\sigma (20 equal-count bins), RMS observed ∣ΔF∣|\Delta F| against RMS σ\sigma, for the CR3 shape with one hold-out quantile (the method) and with the learned scale (σF,i\sigma_{F,i} from the 90 % quantile), and for the per-atom σ\sigma of the propagation's methods: CR3 ×λF\times\lambda_F (the headline), B1 (posterior ×q^\times\hat{q}), B2 (POPS, 50 % leverage cut; the box's uniform variance) and B3 (CR3, λ=1\lambda= 1); the per-atom density behind. Points below the diagonal are conservative. Bottom: test coverage of the 90 % region, overall and per GAP-18 configuration type with at least 300 test atoms. Every method's region is the same kind: the ellipsoid of its own 3×\times3 force covariance, eTC−1e≤χ32(0.9)\mathbf{e}^{T}\mathbf{C}^{-1}\mathbf{e} \le\chi^{2}_{3}(0.9) (for the one quantile and the learned scale this is their conformal region). The headline's λF\lambda_F is fitted on the test split itself, so its per-atom coverage here is in sample.

How the quantile weights the calibration atoms matters more than either scale. A quantile that counts each calibration configuration once in total is about 12 % smaller and covers only 0.82 and 0.84, because the many small, well-predicted cells then dominate it; the quantiles here are atom-weighted. The same CR3 shape with the propagation's global λF\lambda_F covers 0.79 and 0.88, below the one quantile for both models and with the same pattern across configuration types, even though λF\lambda_F is fitted on these test configurations (in sample). λF\lambda_F is an RMS over force components, not a quantile of the 3-D score, and the CR3 shape is strongly anisotropic, so errors along its thin directions leave the ellipsoid. All regions in Figure 3 are ellipsoids of the method's own 3×33 \times3 covariance; an isotropic ball ∣ΔF∣≤σ(χ32(0.9)/3)1/2|\Delta F| \le\sigma\left(\chi^2_3(0.9)/3\right)^{1/2} of the same σ=(tr⁡C)1/2\sigma= (\operatorname{tr} C)^{1/2} is larger along those directions and covers more (0.95 and 0.89 for λF\lambda_F); compared that way, λF\lambda_F would look better calibrated than the one quantile, but only because its region is larger. Increasing the regulariser ε\varepsilon of the ellipsoid tenfold changes no coverage by more than 0.012.

An out-of-distribution test asks whether this holds for environments absent from the fit. We refitted both models (BLR, evidence-optimised on the reduced training set) with GAP-18's cracks ([39]) (111) and (110) cleavage, 1-10 front) and screw dislocations removed from training, from the calibration hold-out and from the test file, and scored the 6056 atoms in 36 configurations that were removed

(Table IV). The design follows the big-cell check of an alloy model in the ace-jax development tests[39]. The global scale is fitted on the in-distribution test split, so it is out of sample here, like the conformal ones. Over all held-out atoms one quantile covers 0.86 (o3d12) and 0.93 (o4d20), and the global scale 0.63 and 0.89. The cracks are covered ((111) 0.91 and (110) 0.95 for o3d12; 0.91 and 0.96 for o4d20). The screw dislocations are under-covered for o3d12 (0.77) but not for o4d20 (0.91).

modelmethodID testall held outcrack (110)crack (111)screw dislocation
o3d12one hold-out quantile0.880.86 [0.83, 0.88]0.95 [0.94, 0.96]0.91 [0.90, 0.92]0.77 [0.76, 0.78]
o3d12learned scale0.880.86 [0.83, 0.87]0.94 [0.93, 0.95]0.90 [0.88, 0.90]0.78 [0.76, 0.79]
o3d12global scale (ID test)0.670.63 [0.59, 0.66]0.78 [0.76, 0.80]0.69 [0.67, 0.72]0.50 [0.49, 0.52]
o4d20one hold-out quantile0.900.93 [0.91, 0.94]0.96 [0.96, 0.97]0.91 [0.90, 0.93]0.91 [0.90, 0.93]
o4d20learned scale0.890.87 [0.84, 0.88]0.93 [0.91, 0.94]0.88 [0.84, 0.89]0.84 [0.80, 0.84]
o4d20global scale (ID test)0.860.89 [0.88, 0.91]0.94 [0.93, 0.96]0.88 [0.87, 0.90]0.87 [0.85, 0.89]

Table IV. Out-of-distribution per-atom force coverage at nominal 90 %: BLR refits without cracks and screw dislocations, scored on them (90 % intervals from a bootstrap over configurations). "ID test": the same refit's coverage on the in-distribution test split.

Out of distribution the learned scale gains nothing, and it can lose. For o3d12 it covers 0.86 of the held-out atoms, as one quantile does (0.86), and the two agree to 0.01 on each held-out type (Table IV); for o4d20 it covers 0.87 against one quantile's 0.93, and it falls below nominal on the (111) crack (0.88 against 0.91) and on the screw dislocations (0.84 against 0.91), which one quantile covers. The earlier ARD fit of o4d20 showed the same pattern. The risk is structural: a learned scale narrows the regions of environments whose training counterparts the model reproduces well, mostly near-bulk ones, and atoms at dislocation cores and on crack flanks resemble those in descriptor space but can carry larger errors. One quantile cannot narrow selectively, and it is simpler, which is why it is the method here; the learned scale is a refinement for target environments that are represented in the training data.

Ho, Ortner and Wang[23] reach a more favourable verdict on their learned scale out of distribution, on compositions and exchange-correlation functionals outside the calibration set. Their test is the correlation of the predicted uncertainty with the error; ours is coverage. These are different properties[40]. The two results are not in conflict, since a scale can rank errors better and still be too narrow where the model extrapolates, but they show that a learned scale should be checked for coverage on environments left out of its calibration.

Figure 3 also shows the per-atom forces of the baselines, to which the QoI comparison of § 3.2.2 applies. B3 is about λF\lambda_F times too narrow everywhere. B1, the posterior shape with one scale, under-covers (0.72 for o3d12, 0.60 for o4d20), most in the amorphous, point-defect and surface cells. B2 over-covers for o3d12 (0.95; per type 0.87 (surface_001) to 0.97). For o4d20 it under-covers overall (0.84) and unevenly: per type 0.51 (surface_001) to 0.99, low on the liquid and surface cells.

One global scale for propagation

The per-atom scale of § 3.2 calibrates force outputs atom by atom, but it does not define a covariance in θ\theta (§ 1.1.5). Propagation needs a single scale λ\lambda for the CR3 distribution θ∼N(θ^,λ2LLT)\theta\sim\mathcal{N}(\hat{\theta}, \lambda^2 L L^{\mathsf{T}}). We fit it on the same 248 held-out test configurations as § 3.2, with the same fit and CR3 shape, but over all rows of a type at once. Every row bb gives a standardised score z=(y−y^)/∥LTb∥z = (y - \hat{y})/\lVert L^{\mathsf{T}}b\rVert, with y^=b⋅θ^\hat{y} = b \cdot\hat{\theta} (energies per atom, force components, virial components). The scale for observable type q∈{E,F,V}q \in\{E, F, V\} is λq=RMS⁡(zq)\lambda_q = \operatorname{RMS}(z_q) over that type's rows; λpool\lambda_{\mathrm{pool}} weights E, F and V equally within a configuration and configurations equally. As in § 3.2, rows that vanish by site symmetry (∥LTb∥<10−3\lVert L^{T}b \rVert< 10^{-3} of the median of their kind) are dropped. No QoI, and no DFT QoI value, enters any scale. The headline uses λF\lambda_F: forces are most of the training rows, and they are the quantity calibrated per atom in § 3.2. In Figure 3, λF\lambda_F gives the headline line.

A single scale is a compromise. Table III gives the test-split scales per configuration type. Over the types listed there λF\lambda_F ranges from 3.8 to 7.5 (o3d12) and 2.5 to 8.2 (o4d20), and λV\lambda_V is largest for the bt cells (14.6 and 15.5). The ratios of the global scales (Table V) show the same at the level of observables.

modelλE\lambda_EλF\lambda_FλV\lambda_Vλpool\lambda_{\mathrm{pool}}λV/λF\lambda_V/\lambda_FλE/λF\lambda_E/\lambda_F
o3d124.925.828.035.851.380.84
o4d205.036.178.236.051.330.82

Table V. Test-split scales (CR3 shape).

Whether the observables need different scales is a model comparison. The test configurations are the independent units, as in CR3, because the force components of one configuration are correlated. For configuration cc and observable oo, let mcom_{co} be the mean of z2z^{2} over the block's components (1 energy, 3NN forces, 6VV virial components). We model mco=λo2Xm_{co} = \lambda_o^{2}X with X∼χνo2/νoX \sim\chi^{2}_{\nu_o}/\nu_o, a Gamma distribution with mean λo2\lambda_o^{2}, as the RMS-zz definition of λ\lambda requires. The effective number of degrees of freedom νo\nu_o absorbs the correlation within a block and the heavy tails. It is free for each observable in every model, log-uniform on [0.05,1000][0.05, 1000], and integrated out. The models are the five ways to let E, F and V share scales, each scale log-

uniform on [0.1,1000][0.1, 1000]. A joint Gaussian likelihood of each block under λ2\lambda^{2} times the full CR3 block covariance was also tried and is not usable: for forces it is set by the regularisation of a nearly singular block, that is by the off-diagonal CR3 shape, not by the scale.

scaleso3d12o4d20
one scale-31.6-23.8
E | F = V-32.7-23.7
F | E = V-22.3-19.4
V | E = F+0.0+0.0
E | F | V-4.2-3.9

Table VI. Test-split evidence for shared against per-observable scales: log Bayes factor of each way of sharing scales against the best (configurations as units, equal prior odds).

In both models the evidence favours one scale for energies and forces and a separate, larger one for virials: log Bayes factors of 32 (o3d12) and 24 (o4d20) against one scale for all, and 4.2 and 3.9 against three separate scales. The posterior scales are λV\lambda_V 8.1 [7.2, 9.0], λE=F\lambda_{E=F} 5.0 [4.8, 5.2] for o3d12 and λV\lambda_V 8.3 [7.3, 9.3], λE=F\lambda_{E=F} 5.2 [4.9, 5.5] for o4d20 (mean and 95 % credible interval). With each configuration weighted equally, λE\lambda_E and λF\lambda_F agree; the smaller λE\lambda_E of Table V comes from the component-weighted RMS, in which the largest cells dominate λF\lambda_F. The degree of misspecification therefore depends on the observable, and with λF\lambda_F virials are under-covered on the test split. The QoIs most like virials are the elastic constants, which are derivatives of the stress, and the phase energies, which come from energy–volume curves, so their intervals at λF\lambda_F may be too narrow; § 3.2.2 reports the coverage with λV\lambda_V as a sensitivity.

Coverage and sharpness

With the headline CR3 ×\times λF\lambda_F, 17/20 QoIs of o3d12 and 15/20 of o4d20 lie inside their 95 % intervals (Table VII, Figure 5 top row, Figure 6). Both counts are below the nominal 19, and the coverage posteriors (§ 3.1) give P(coverage≥0.95)=0.04P(\text{coverage} \ge0.95) = 0.04 for o3d12 and 0.00 for o4d20: the headline under-covers at the 95 % level. The misses are not spread across the QoIs. For o3d12 they are the three interstitial formation energies; for o4d20 they are the three interstitial formation energies, the sh phase energy and the (100) surface energy. The interstitial formation energies, missed by both models, are the subject of § 3.2.6: they share a bias that a delete-one jackknife cannot see, not a scale that is too small. Elsewhere the 95 % intervals reach DFT, and the 68 % counts (14 and 12 of 20, against a nominal 13.6) are close to nominal.

modelmethodnn68 %95 %RMS zzmedian σ\sigma/tolCRPS/tol
o3d12CR3 ×λF\times\lambda_F (headline)2014 [0.48, 0.86]17 [0.65, 0.96]1.292.522.79
o3d12B1: posterior ×q^\times\hat{q}2013 [0.43, 0.83]15 [0.54, 0.90]2.281.632.92
o3d12B2: POPS (leverage cut 50 %, the package default)2013 [0.43, 0.83]15 [0.54, 0.90]1.551.752.86
o3d12B3: CR3, λ=1\lambda= 1205 [0.10, 0.46]8 [0.21, 0.62]7.530.433.16
o4d20CR3 ×λF\times\lambda_F (headline)2012 [0.38, 0.79]15 [0.54, 0.90]1.310.640.75
o4d20B1: posterior ×q^\times\hat{q}2011 [0.34, 0.75]12 [0.38, 0.79]2.580.370.82
o4d20B2: POPS (leverage cut 50 %, the package default)2011 [0.34, 0.75]17 [0.65, 0.96]1.690.600.81
o4d20B3: CR3, λ=1\lambda= 1202 [0.02, 0.28]5 [0.10, 0.46]8.110.100.91

Table VII. Coverage of the DFT values (count [95 % credible interval of the coverage probability]) and sharpness, for the four methods of Figures 5 and 6.

Figure 5. Model prediction minus DFT, in units of the benchmark tolerance, with the 95 % interval, for o3d12 (circles) and o4d20 (squares); DFT is the cross at 0 and the shaded band is ± 1 tolerance. The axis is linear within ± 1 and logarithmic beyond, and is shared by all rows. Top row: the headline, CR3 × λF\lambda_F; the larger model's intervals are 3.9× tighter in median σ\sigma/tol while its coverage stays comparable. Below it the baselines of Section 2.6: B1, the posterior scaled by one conformal quantile; B2, POPS with the 50 % leverage cut (the reference package's default); B3, CR3 without calibration (λ\lambda = 1). Each row's legend gives its 95 % coverage and median σ\sigma/tol.

Figure 6. Fraction of the 20 scored QoIs inside the 68 % and 95 % intervals for the four methods of Figure 5 (headline, B1, B2 with the 50 % leverage cut, B3), with 95 % credible intervals from the Jeffreys-prior Beta posterior; dashed lines are the nominal levels.

The intervals follow the accuracy of the model. The median σ/tol\sigma/\text{tol} is 2.52 for o3d12 and 0.64 for o4d20, 3.9×\times narrower: o3d12's intervals are wider than the benchmark tolerance, as they should be for a model whose typical QoI error is about one tolerance (Figure 1), and o4d20's are mostly inside it. So one scale fitted to held-out forces gives similar coverage for both models while the intervals shrink with model quality. The CRPS, which weighs calibration and sharpness in one score, ranks the headline first among the four methods for both models: 2.79 for o3d12 and 0.75 for o4d20, against 2.92 / 0.82 for B1, 2.86 / 0.81 for B2 and 3.16 / 0.91 for B3 (o3d12 / o4d20; Table VII). The margins are small because the mean is dominated by the QoIs with the largest errors, which no width can fix: for o3d12 the three interstitials are 12–17 tolerances off, so its CRPS/tol is near 3 for every method.

Of the baselines (§ 3.2.5), B1 and B3 cover fewer QoIs. POPS (B2) covers 15/20 (o3d12) and 17/20 (o4d20), at widths close to the headline's (median σ/tol\sigma/\text{tol} 1.75 and 0.60). The sensitivity table (Table VIII) shows that the choice of scale matters at the margin. The virial scale λV\lambda_V, larger than λF\lambda_F by about a third, covers 18/20 (o3d12) and 19/20 (o4d20) at the price of wider intervals (median σ/tol\sigma/\text{tol} 3.47 for o3d12 and 0.85 for o4d20); λE\lambda_E and λpool\lambda_{\text{pool}} give about the same as λF\lambda_F. The full committee re-run at λF\lambda_F, keeping only physical members, covers 15/20 for o3d12 and 17/20 for o4d20, against 17/20 and 15/20 for the headline. The differences come mainly from the phase energies, which are strongly nonlinear in θ\theta at $\lambda_F ({xref}Section 3.6`).

modelmethodnn68 %95 %RMS zzmedian σ\sigma/tolCRPS/tol
o3d12CR3 ×λV\times\lambda_V2015 [0.54, 0.90]18 [0.72, 0.98]0.943.472.78
o3d12CR3 ×λE\times\lambda_E2014 [0.48, 0.86]16 [0.59, 0.93]1.532.132.82
o3d12CR3 ×λpool\times\lambda_{\mathrm{pool}}2014 [0.48, 0.86]17 [0.65, 0.96]1.292.532.79
o3d12CR3 at λF\lambda_F, full re-run, physical members2014 [0.48, 0.86]15 [0.54, 0.90]1.304.495.12
o3d12POPS, leverage cut 5 %2013 [0.43, 0.83]15 [0.54, 0.90]1.422.442.87
o4d20CR3 ×λV\times\lambda_V2014 [0.48, 0.86]19 [0.79, 0.99]0.990.850.73
o4d20CR3 ×λE\times\lambda_E2011 [0.34, 0.75]15 [0.54, 0.90]1.610.520.77
o4d20CR3 ×λpool\times\lambda_{\mathrm{pool}}2012 [0.38, 0.79]15 [0.54, 0.90]1.340.630.75
o4d20CR3 at λF\lambda_F, full re-run, physical members2015 [0.54, 0.90]17 [0.65, 0.96]1.061.011.41
o4d20POPS, leverage cut 5 %2014 [0.48, 0.86]18 [0.72, 0.98]1.110.990.84

Table VIII. Sensitivity of the coverage to the scale and to the committee variant: the CR3 shape with the energy, virial or pooled scale (Section 3.3), the full re-run at λF\lambda_F keeping physical members only (Section 3.6), and POPS with the 5 % leverage cut instead of B2's 50 % (Section 3.7).

Figure 4 compares the four methods on both models. On the QoIs the headline has the lowest CRPS/tol for both models and the highest log evidence for o4d20; for o3d12, B2 has the higher evidence (Table IX). The per-atom forces rank the methods differently. B2's box, read as a Gaussian of its covariance, covers 0.95 and 0.84 of the test atoms at 90 % (o3d12, o4d20), against 0.79 and 0.88 for the headline. On the held-out cracks and dislocations the two models disagree. For o3d12 B2 covers 0.89 against the headline's 0.76, and is higher on every held-out type. For o4d20 it covers 0.59 against 0.91: 0.50 and 0.40 on the two cracks (headline 0.95 and 0.89) and 0.77 on the screw dislocations (0.89), with the POPS ridge selected as in distribution. The held-out mean log scores follow the coverages: 1.2 against 0.7 nats for o3d12, 1.0 against 2.5 for o4d20. On the test split B2's mean log score per atom is 2.2 and 4.3 nats, against the headline's 1.6 and 4.4. B1 under-covers the forces in and out of distribution (0.72/0.58 and 0.60/0.46), and B3 covers almost no atoms. In distribution B2 is also uneven per configuration type for o4d20: its lowest per-type test coverage is 0.51, against 0.74 for the headline (o3d12: 0.87 and 0.70). The headline's λF\lambda_F and B1's q^\hat{q} are fitted on the test split, so their test-split force scores are in sample; on the held-out configurations every method is out of sample, with λF\lambda_F, q^\hat{q} and the POPS box refitted on the reduced training set by the same rules. The headline's λF\lambda_F is the RMS over force components of § 3.2.1; the global scale of Table IV is fitted per atom instead (λ2\lambda^2 the mean of ∣ΔF∣2/tr⁡V|\Delta F|^2/\operatorname{tr} V), so its coverages differ.

modelmethodcovariance fromnnlog⁡p\log plog⁡BF\log\mathrm{BF} vs headlined2d^2log⁡det⁡\log\det
o3d12CR3 ×λF\times\lambda_F (headline)committee20-45.0+0.049.63.6
o3d12CR3 ×λF\times\lambda_F (headline)delta17-38.4+0.048.2-2.7
o3d12B1: posterior ×q^\times\hat{q}committee20-59.2-14.280.11.6
o3d12B1: posterior ×q^\times\hat{q}delta17-46.9-8.562.8-0.2
o3d12B2: POPS (leverage cut 50 %)committee20-39.4+5.630.111.9
o3d12B3: CR3, λ=1\lambda= 1committee20-827.5-782.51681.6-63.3
o3d12B3: CR3, λ=1\lambda= 1delta17-803.9-765.61635.8-59.1
o3d12CR3 ×λV\times\lambda_V (sensitivity)committee20-39.3+5.726.115.8
o3d12CR3 ×λV\times\lambda_V (sensitivity)delta17-32.1+6.325.47.6
o3d12CR3 ×λF\times\lambda_F + discrepancy termcommittee20-35.4+9.618.915.2
o3d12CR3 ×λF\times\lambda_F + discrepancy termdelta17-27.6+10.815.78.2
o3d12B1 + discrepancy termcommittee20-41.7+3.332.714.0
o3d12B1 + discrepancy termdelta17-30.3+8.118.111.2
o3d12B2 + discrepancy termcommittee20-38.6+6.418.322.1
o4d20CR3 ×λF\times\lambda_F (headline)committee20-11.3+0.022.8-37.0
o4d20CR3 ×λF\times\lambda_F (headline)delta17-8.7+0.023.5-37.5
o4d20B1: posterior ×q^\times\hat{q}committee20-40.2-29.0100.7-56.9
o4d20B1: posterior ×q^\times\hat{q}delta17-48.9-40.2118.7-52.2
o4d20B2: POPS (leverage cut 50 %)committee20-22.9-11.648.5-39.6
o4d20B3: CR3, λ=1\lambda= 1committee20-398.3-387.0865.9-106.1
o4d20B3: CR3, λ=1\lambda= 1delta17-416.2-407.5896.9-95.7
o4d20CR3 ×λV\times\lambda_V (sensitivity)committee20-11.8-0.512.8-26.1
o4d20CR3 ×λV\times\lambda_V (sensitivity)delta17-8.1+0.513.3-28.3
o4d20CR3 ×λF\times\lambda_F + discrepancy termcommittee20-12.0-0.717.2-30.0
o4d20CR3 ×λF\times\lambda_F + discrepancy termdelta17-7.5+1.113.9-30.0
o4d20B1 + discrepancy termcommittee20-22.8-11.655.4-46.5
o4d20B1 + discrepancy termdelta17-17.6-8.943.9-40.0
o4d20B2 + discrepancy termcommittee20-13.1-1.819.5-30.1

Table IX. Log evidence of the 20 scored DFT values (17 with the delta method) under each method's predictive distribution, in tolerance units (ε=10−6\varepsilon= 10^{-6}), and the log Bayes factor against the headline with the same covariance source. d2d^2 and log⁡det⁡\log\det are the two parts of the evidence: the misfit of the errors, and the width of the covariance.

Figure 4. The headline method, CR3 ×λF\times\lambda_F, and the three baselines: B1, the Bayesian posterior scaled by one conformal quantile q^\hat{q}; B2, POPS with the 50 % leverage cut; B3, CR3 without calibration (λ=1\lambda= 1). One row of panels per model. Left: the 20 scored QOIs, CRPS/tol, coverage of the 95 % intervals, and the log evidence of the DFT values (committee covariance, ϵ=10−6\epsilon= 10^{-6}; Table IX). Right: per-atom forces, coverage of each method's 90 % ellipsoid on the test split, its lowest value over the configuration types of Figure 3 (Table III), and on the cracks and screw dislocations held out of BLR refits (Table IV), and the mean Gaussian log score per test atom. Dashed: nominal coverage. Full colour: the best method on each proper score (CRPS, log evidence, log score); coverage panels are not ranked. On the test split the headline's λF\lambda_F and B1's q^\hat{q} are fitted on the same data, so their test-split force panels are in sample. The log axes are cut at −75-75 (QOIs) and 0 (forces) so that the other methods can be compared; B3's values beyond them are written on the panel.

Joint evidence

The QoIs share one coefficient vector, so their errors are correlated, and Figure 7 shows how. In the CR3 committee for o4d20 the correlations form blocks: the elastic constants with each other and with the β\beta-Sn and sh phase energies, which are both compressed diamond-like structures; and the four point defects with each other (0.3–0.8), most strongly the three interstitials. Most other pairs are only weakly correlated.

Figure 7. Correlation between the QoIs in the CR3 committee for o4d20 (K=200K = 200 members).

tbl:9 compares the methods by their evidence. The headline is far ahead of B1 and B3 for both models: by log Bayes factors of -14.2 (B1) and -782.5 (B3) for o3d12, and -29.0 and -387.0 for o4d20, with the committee covariance. On the scale of Kass and Raftery[41] a log Bayes factor beyond about 3 is strong evidence and beyond 5 very strong. B3 loses through the misfit term, with intervals far too narrow for its errors. B1 loses less, because its overall size is calibrated, but its shape puts the width in the wrong places. POPS (B2) is the close competitor. Without the discrepancy term the evidence prefers it for o3d12 (+5.6) and the headline for o4d20 (-11.6). The o3d12 preference does not survive the discrepancy term of § 3.2.6: with the term added to both, the headline is preferred for both models, strongly for o3d12 (+3.1) and weakly for o4d20 (+1.1). One at a time the headline covers more QoIs for o3d12 (17/20 against 15/20) and POPS for o4d20 (17/20 against 15/20): per-QoI coverage and the joint evidence need not agree. The correlations can also favour a method that misses QoIs one at a time: o4d20's headline misses five QoIs, but three of the five are the interstitials, which its covariance expects to move together, so jointly they cost less than three independent misses.

The virial scale is better than the force scale for o3d12 (+5.7), whose QoI errors are larger relative to its force errors, and about as good for o4d20 (-0.5). The discrepancy term of § 3.2.6 is strongly supported for o3d12 (+9.6 with the committee covariance, +10.8 with the delta method), and roughly neutral for o4d20 (-0.7 and +1.1): o4d20's defect bias is small, so the term's extra width costs about as much as the misses it removes. The delta-method and committee covariances agree, and the ranking of the methods is the same for ε\varepsilon from 10−810^{-8} to 10−410^{-4}.

Delta method, committees and nonlinearity

The two routes to σQ\sigma_Q agree where both apply. At λ=1\lambda= 1, the delta-method σ\sigma is within 10 % of the committee's for all but 1 of the 17 QoIs of o3d12 and 4 of the 17 of o4d20; the ratio ranges from 0.64 to 1.06 for o3d12 and from 0.79 to 1.16 for o4d20. The flagged QoIs (the hcp phase energy for o3d12; the bulk modulus, the β\beta-Sn phase energy, the vacancy, the tetrahedral interstitial for o4d20) are taken from the committee, as Section 2.7.3 prescribes. The implicit differentiation through each relaxation, including that of the lattice constant, is therefore accurate at the size of the CR3 distribution itself. The elastic constants are a stricter check, because they can be computed by two routes, from the stress or from the curvature of the energy. Figure 8 shows that the two give nearly the same covariance ellipses for both models, and that the DFT values lie well outside the 1σ1\sigma ellipses for o3d12, whose C11C_{11} is about 15 % too low, but on or inside them for o4d20 (a 1σ1\sigma ellipse holds only 39 % of a two-dimensional Gaussian).

At λF\lambda_F the distribution is several times wider, and the linear rescaling of the λ=1\lambda= 1 committee, Qk(λ)=Q^+λ(Qk−Q^)Q_k(\lambda) = \hat{Q} + \lambda(Q_k - \hat{Q}), can fail. The full re-run at λF\lambda_F tests it. Of its 200 members, 180 (o3d12) and 198 (o4d20) pass the physicality rule fixed beforehand; the rest collapse (20 and 2), and none fails a test. Outside the phase energies, the σ\sigma of the physical members is 0.63–1.56×\times the rescaled σ\sigma for o3d12 and 0.80–1.49×\times for o4d20: for the elastic constants, defects and surfaces the rescaling is right to within these factors. The phase energies are different: for o3d12 the σ\sigma of the β\beta-Sn, sh and hcp energies is 37, 30 and 19 times the rescaled σ\sigma, and for o4d20 the σ\sigma of the sh, β\beta-Sn and hcp energies is 86, 43 and 24 times it. These are the phases furthest from diamond, whose structures relax into other phases when θ\theta moves far enough, so the phase-energy of an atom at λF\lambda_F is a property of that relaxation path rather than of a Gaussian in θ\theta. The headline keeps the rescaled σ\sigma, and Table VIII reports the full re-run as a sensitivity.

Baselines

B1 and B3 fall short in different ways (Figure 5, Figure 6 and Table VII); B2 does not, once its box is numerically defined. B3, the CR3 shape without calibration, covers only 8/20 (o3d12) and 5/20 (o4d20) of the QoIs at 95 %: the shape alone is several times too narrow, as the test-split scales of Section 3.3 already show. B1, the posterior with one conformal scale, covers 15/20 (o3d12) and 12/20 (o4d20). It has a calibrated overall size but the wrong shape, so it is too wide for some QoIs and too narrow for others, and its joint evidence is far below the headline's (Table IX).

Figure 8. Elastic constants: the 1σ1\sigma covariance ellipses of (C11,C12)(C_{11}, C_{12}) and (C11,C44)(C_{11}, C_{44}) at λF\lambda_{\mathrm{F}} from the delta method, by the stress route (solid) and the energy route (dashed), for o3d12 (top) and o4d20 (bottom); the dot is the mean model's value and the star DFT.

B2, POPS, needed more care. A hypercube built from every training row, instead of the reference package's 50 % cut, gives unphysical members: the bounds are set by a handful of rows with near-zero leverage, whose corrections are unbounded, and the members spread the bulk energy of o3d12 by 2×10122 \times10^{12} meV/atom. Rejecting unphysical members cannot rescue a box of that size.

The box is also numerically delicate on these designs. The o3d12 design matrix has a condition number of 5×10165 \times10^{16}, and its evidence fit puts σc\sigma_c on its upper bound, so the ridge implied by the BLR prior is only 4×10−204 \times10^{-20} of the largest eigenvalue. At ridges this small the box is

not well defined: below a relative ridge of about 10−1110^{-11} it depends on how it is computed. At 10−1410^{-14}, mathematically equivalent evaluations from the same rows, mean and leverage cut give o3d12 bulk-energy spreads of 1.6 (eigendecomposition), 2.1 (a singular value decomposition, SVD, of the rows) and 2.2 meV/atom (the hypercube step of the reference package POPSRegression[36]). We therefore select the ridge from 10−1010^{-10} upwards (o3d12: E 10−910^{-9}, F 10−1010^{-10}, V 10−1010^{-10}; o4d20: E 10−610^{-6}, F 10−810^{-8}, V 10−710^{-7}). The o3d12 selection lands on this floor or one decade above it, so for o3d12 the floor rather than the data largely fixes the box; the o4d20 selection falls inside the grid. At the selected ridges the evaluations agree: the B2 box, the eigendecomposition, the SVD and the reference package give covariances that agree to within 7×10−47 \times10^{-4} (o3d12) and 2×10−42 \times10^{-4} (o4d20), relative, so our implementation and the reference package differ in numerics only, not in method.

Two dependences remain. The size of the box depends strongly on the ridge: for o3d12, with one ridge for all quantities, the bulk-energy spread is 4.54 meV/atom at 10−1010^{-10}, 31 meV/atom at 10−610^{-6}, 263 meV/atom at 10−210^{-2}. It also depends on the coordinates: the reference package takes its axes in the coordinates it is given, here the prior-scaled Γθ\Gamma\theta, and its box there differs from the box in θ\theta (161 against 206 modes, 4.53 against 5.50 meV/atom (o3d12) and 683 against 2766 modes, 0.87 against 1.10 meV/atom (o4d20) in θ\theta against Γθ\Gamma\theta, at the selected ridges). The reference package's default ridge, chosen by evidence, is 4×10−214 \times10^{-21} (o3d12) of the largest eigenvalue, below the floor, so we do not use it.

We ran committees at the 50 % cut (B2) and at 5 %, at the ridges selected per quantity; Table X compares them with the CR3 members. The 5 % cut widens the box, by a factor of 1.4 (o3d12) and 2.4 (o4d20) in bulk-energy spread, but it changes the 95 % coverage by at most one QoI. With a numerically defined box, POPS gives QoI intervals of about the headline's width and coverage without a calibration step. What sets its width is the ridge and the leverage cut instead: over the numerically defined ridges the o3d12 box's bulk-energy spread changes by a factor of about 60 (above).

modelmethodbulk-energy spread (meV/atom)95 %RMS zzmedian σ\sigma/tolCRPS/tol
o3d12B2: POPS, leverage cut 50 %3.8915/201.551.752.86
o3d12POPS, leverage cut 5 %5.6415/201.422.442.87
o3d12CR3 ×λF\times\lambda_F (headline)1.9817/201.292.522.79
o3d12B3: CR3, λ=1\lambda= 10.348/207.530.433.16
o4d20B2: POPS, leverage cut 50 %0.8317/201.690.600.81
o4d20POPS, leverage cut 5 %2.0218/201.110.990.84
o4d20CR3 ×λF\times\lambda_F (headline)0.6915/201.310.640.75
o4d20B3: CR3, λ=1\lambda= 10.115/208.110.100.91

Table X. POPS committees at the two leverage cuts against the CR3 members: spread of the members' bulk (diamond) energy, and QoI coverage and sharpness as in Table VII.

Defect formation energies: diagnosis and the discrepancy term

For both models the relaxed interstitial formation energies are the QoIs that are confidently wrong: all three are too low, and outside the 95 % interval. This holds for every method of § 3.2.5, not only the headline, so the diagnosis below is about the fit, not the uncertainty method; the discrepancy term it leads to (below) applies to any of them. Table XIII shows it for the posterior baseline B1 as well as for CR3: with the term B1 covers 18/20 (o3d12) and 16/20 (o4d20). They are low for every ACE model in the benchmark too, linear and nonlinear (PACE), and the tetrahedral one for GAP-18.

modelmethod95 % without95 % with [credible interval]68 %median σ\sigma/tolCRPS/tol without →\to with
o3d12CR3 ×λF\times\lambda_F17/2020/20 [0.88, 1.00]14 →\to 172.522.79 →\to 2.38
o3d12CR3 ×λV\times\lambda_V18/2020/20 [0.88, 1.00]15 →\to 183.472.78 →\to 2.53
o3d12B1: posterior ×q^\times\hat{q}15/2018/20 [0.72, 0.98]13 →\to 151.662.92 →\to 2.30
o4d20CR3 ×λF\times\lambda_F15/2018/20 [0.72, 0.98]12 →\to 140.640.75 →\to 0.65
o4d20CR3 ×λV\times\lambda_V19/2020/20 [0.88, 1.00]14 →\to 160.850.73 →\to 0.66
o4d20B1: posterior ×q^\times\hat{q}12/2016/20 [0.59, 0.93]11 →\to 120.390.82 →\to 0.64

Table XIII. Coverage of the 20 scored QoIs with the discrepancy term on the four defect QoIs (models with r∗r^*: o3d12, o4d20).

Over the hot cells the model's interstitial error averages near zero, so the CR3 intervals are calibrated there (test-split zz RMS 0.85–1.13 for those cells). The error grows towards the cold limit, and the relaxed defect lies beyond the data along a bias that all interstitial clusters share. With per-atom energy weighting, 0.4–1.6 eV in a 217-atom cell costs only 2–7 meV/atom in the fit.

Why not in parameter space. The interstitial synthetic row is far outside the jackknife's reach: its CR3 ×λFσ\times\lambda_F \sigma is 0.49 eV against r∗=1.58r^* = 1.58 eV (o3d12) and 0.17 eV against r∗=0.38r^* = 0.38 eV (o4d20). The minimum-norm correction that removes r∗r^*, δt=Σdt∗/(dt∗Σdt∗)\delta_t = \Sigma d_t^*/(d_t^*\Sigma d_t^*), with Σ\Sigma the propagated covariance (the POPS construction applied to the synthetic row, in the CR3 or posterior metric), added as Σ+δtδtT\Sigma+ \delta_t\delta_t^\mathsf{T}, carries part of the bias to the relaxed defects, but unevenly: the projected σ\sigma on the three interstitial QoIs is o3d12: 0.2–1.4 of r∗=1.58r^* = 1.58 eV; o4d20: 0.3–0.7 of r∗=0.38r^* = 0.38 eV (both metrics). The hot-snapshot contrast and the relaxed defect's formation row point in different directions in θ\theta, so the transfer depends on the QoI. Coverage at 95 % with λF\lambda_F, over the 17 QoIs the delta method gives, becomes o3d12 14 →\to 14–16 of 17, o4d20 12 →\to 15–16 of 17, short of the observable-space term below.

The term lowers the CRPS of every method (Table XIII), most for o3d12, whose interstitial errors dominate its score. With the term, B1 and CR3 ×λF\times\lambda_F score almost the same (2.30 against 2.38 for o3d12, 0.64 against 0.65 for o4d20, B1 marginally lower). Once the defects carry the term, the remaining difference between the two shapes is small on this score, while CR3 still covers more QoIs.

The interstitial estimator agrees well with the observed interstitial errors. That agreement was seen before the bulk reference was chosen: an alternative, each defect cell's nearest bulk cell in volume and thermal amplitude, was rejected because its standard error is larger and it extrapolates worse to held-out cold snapshots. The choice was therefore not made blind, although the reasons for it use training data only.

The vacancy

The vacancy shows no thermal trend when its 63- and 215-atom training cells are pooled. Table XV compares the size models of the method above by their evidence, from training labels only.

model of the size dependenceo3d12 log BFbiaso4d20 log BFbiasmodel vs model log BFbias
one intercept, one slope−114.6+0.79 ± 0.04−142.7+0.14 ± 0.02−80.1+0.60 ± 0.04
per-size intercepts, one slope−3.9−0.20 ± 0.04−15.1−0.33 ± 0.02−1.0+0.14 ± 0.03
one intercept, per-size slopes−9.0+0.79 ± 0.04−5.4+0.14 ± 0.02−2.6+0.64 ± 0.03
per-size intercepts and slopes+0.0+0.22 ± 0.12+0.0−0.04 ± 0.05+0.0+0.35 ± 0.10
average over models+0.21 ± 0.14−0.04 ± 0.05+0.31 ± 0.14

Table XV. Vacancy size models: log Bayes factor against the best model, and the posterior zero-thermal-amplitude bias of a 215-atom cell (eV), for each model and averaged over the models (sa=1s_a = 1 eV). The last pair of columns applies the same analysis to the energy difference of the two models on the training cells (o4d20 − o3d12); the true difference of the models' own relaxed vacancy QoIs is 0.77 eV (no DFT value involved).

The data reject one trend for both sizes outright, and disfavour a shared intercept: the zero-temperature bias depends on the cell size. The per-size models win, and the vacancy QoI, in a 216-site cell, takes the 215-atom intercept averaged over the models: +0.21±0.14+0.21 \pm0.14 eV (o3d12) and −0.04±0.05-0.04 \pm0.05 eV (o4d20). The 215-atom cells are all hot, so this extrapolation is weakly determined and depends on the prior. For the shared-intercept model, the log Bayes factor and the model average at the other prior scales are, for o3d12, sa=0.3s_a = 0.3 eV: -74.4, average -0.23 eV; sa=3s_a = 3 eV: -3.6, average +0.35 eV; for o4d20, sa=0.3s_a = 0.3 eV: -16.1, average -0.12 eV; sa=3s_a = 3 eV: -3.5, average -0.03 eV.

One check disagrees. Applied to the difference between the two models, where the truth is known without DFT, the shared-intercept model with per-size slopes predicts 0.64 of the true 0.77 eV and the model average only 0.31. Both miss by several standard errors, so neither form captures how the relaxed vacancy differs from the hot cells. The one near-relaxed training cell (63 atoms) gives direct evidence without extrapolation: its bulk-subtracted residual is +0.61 eV (o3d12) and 0.00 eV (o4d20), and relaxing it (o3d12: relaxation gain 0.34 eV against a DFT bound of 0.24 eV, o4d20: relaxation gain 0.18 eV against a DFT bound of 0.18 eV) suggests that o3d12 over-relaxes. Both models relax the harness vacancy to a symmetric structure (vacancy neighbour distances o3d12 3.40–3.40 Å, o4d20 3.47–3.47 Å); the harness gives no DFT structure to compare.

modelextrapolated to Frms→0F_{\rm rms} \to0QoI errorsmean over hot cells
o3d12+1.58±0.06+1.58 \pm0.06+1.18/+1.45/+1.74+1.18 / +1.45 / +1.74−0.46-0.46
o4d20+0.38±0.03+0.38 \pm0.03+0.37/+0.41/+0.60+0.37 / +0.41 / +0.60−0.43-0.43

Table XI. The training residual trend predicts the QoI error. Interstitial cells, residual (DFT − model) minus a temperature-matched bulk reference, extrapolated to Frms→0F_{\rm rms} \to0 (eV), against the QoI errors (DFT − model) of the dumbbell / hexagonal / tetrahedral interstitials, and the mean over the hot training cells.

modelclassestimatorrt∗r_t^* (eV)st∗s_t^* (eV)
o3d12interstitialbulk-trend regression, η→0\eta\to0+1.5770.062
o3d12vacancy215-atom intercept, η→0\eta\to0, averaged over size models+0.2140.137
o4d20interstitialbulk-trend regression, η→0\eta\to0+0.3850.032
o4d20vacancy215-atom intercept, η→0\eta\to0, averaged over size models−0.0420.049

Table XII. The bias rt∗r_t^* and its standard error st∗s_t^* in the headline discrepancy term (§ 2.0.2.2; DFT training labels only). Every row is an extrapolation of the training residuals to zero thermal amplitude (η→0\eta\to0). For the vacancy that extrapolation is also for the size of the QoI cell (215 atoms), averaged over the four size models of Table XV, and s∗s^* is the standard deviation of that average.

The vacancy term is small in both models, and the CR3 σ\sigma alone already covers the vacancy at 95 % (Table XIV).

Discussion

A single parameter covariance, the CR3 jackknife scaled once to held-out forces, gives usable uncertainties for derived properties that the fit never saw. For both models most QoIs fall inside their 95 % intervals (17/20 for o3d12, 15/20 for o4d20), and the intervals scale with the model: the median σ\sigma/tol is 2.52 for o3d12 and 0.64 for the accurate o4d20, whose intervals are mostly inside the benchmark tolerance. The scale is fitted once, on forces of held-out configurations, and never on a QoI. The same CR3 shape serves per-atom forces, with one conformal quantile (§ 3.2), and relaxed QoIs, with one global scale. The second use is what a distribution over θ\theta buys: one draw gives energies, forces, stresses and every QoI together, consistently.

The baselines show which ingredient matters. The CR3 shape without a scale (B3) is far too narrow, so calibration is essential, as expected for a parameter variance that shrinks with the amount of data. The Bayesian posterior with a calibrated scale (B1) has the right overall size but the wrong shape: it under-covers the QoIs and its joint evidence is far lower, because the posterior describes noise in the labels, not the approximation error of a misspecified model. Shanks et al.[40] find the same split between accuracy and uncertainty for argon: MACE committees and seed ensembles predict accurately but cover far fewer errors than their intervals claim, and calibration, sharpness and the ranking of errors need not improve together. POPS (B2) targets that error directly and, with a numerically defined box, performs comparably on these QoIs: similar coverage and widths. The joint evidence is mixed: without the discrepancy term it prefers POPS for o3d12 (log Bayes factor +5.6) and the CR3 propagation for o4d20 (+11.6); with the term on both it prefers the CR3 propagation, by +3.1 (o3d12) and +1.1 (o4d20). The CR3 approach is therefore not more accurate than POPS. Implicit differentiation is no distinction either: Perez et al.[30] combine it with POPS, and either distribution can be propagated by the delta method or by committees. What CR3 adds is a parameter distribution that is calibrated and numerically well posed.

Three properties of the distribution make the difference. First, its size is one scale, fitted to held-out errors and interpretable as such. The size of the POPS box is set instead by the ridge and the leverage cut: across the numerically defined ridges the o3d12 box's bulk-energy spread changes by a factor of about 60, the held-out selection put the o3d12 ridge at the floor, and a box built from every row is unphysical. Second, a covariance is a sum over configurations, and it is formed here through its factor LL, never as an ill-conditioned quadratic form. A hypercube is bounded by the extreme projections of single rows: on the same rows, mean and cut, equivalent evaluations of one box gave bulk-energy spreads from 1.6 to 2.2 meV/atom at a relative ridge of 10−1410^{-14}, and the box exists only above a ridge floor. Third, a covariance transforms consistently under a linear change of parameters, and a box does not: with the axes taken in θ\theta or in the prior-scaled coefficients Γθ\Gamma\theta, the same corrections give different boxes (161 against 206 modes, 4.53 against

Figure 9. As Figure 5, with the headline discrepancy term on the four defect QoIs (CR3 ×λF\times\lambda_F): the bulk-trend regression for the interstitials and the size-model average for the vacancy. Thin black bars: the same QoIs' 95 % intervals without the term. Only the four defect QoIs change.

5.50 meV/atom (o3d12) and 683 against 2766 modes, 0.87 against 1.10 meV/atom (o4d20) in θ\theta against Γθ\Gamma\theta, at the selected ridges). For forces the calibration is explicit. One conformal quantile on the CR3 shape covers 0.87 and 0.90 of the test atoms against a nominal 0.90 (the o3d12 shortfall is the hold-out transfer of § 3.2), and holds out of distribution on the cracks for both models and on the screw dislocations for o4d20, though not for o3d12 (0.77); the global force scale on the same kind of region covers less (0.79 and 0.88), even fitted in sample. The POPS box covers 0.95 (o3d12) and 0.84 (o4d20), the latter below nominal and unevenly (per type 0.51 (surface_001) to 0.99), and nothing in its construction aims at a target coverage. Refitted without the cracks and dislocations, it covers 0.89 of the held-out atoms for o3d12 but 0.59 for o4d20, against 0.76 and 0.91 for CR3 ×λF\times\lambda_F (§ 3.2.2). Two costs of CR3 remain. It is a sampling variance, so it needs held-out data and a scale; POPS needs neither in principle, although here its ridge was chosen on a hold-out of the training set. And like POPS it cannot see a bias that every configuration of a kind shares (§ 3.2.6).

The scale is not unique. The virial scale is about a third larger than the force scale, and using it covers 18/20 QoIs for o3d12 and 19/20 for o4d20 with wider intervals; its joint evidence is higher than the force scale's for o3d12 and about equal for o4d20. Misspecification depends on the observable, and one could choose the scale by the QoI's physics, the virial scale for stress-like QoIs such as elastic constants, or report both. A single scale is also a compromise across configuration types (Figure 5), which a per-type scale in θ\theta could relax. A learned per-atom scale cannot be used for this directly: it rescales force outputs atom by atom and defines no covariance in θ\theta; a θ\theta-space analogue would need a shape that depends on the environment.

Linearisation is adequate at the size of the distribution, but not everywhere at the calibrated size. The delta method agrees with committees run through the unmodified benchmark for almost every QoI at λ=1\lambda= 1, and implicit differentiation through each relaxation, including that of the lattice constant, makes it cheap: one set of rows per QoI. At λF\lambda_F the phase energies of the structures furthest from diamond become strongly nonlinear, because their relaxations change character; for those, committees are the safer route, and their spread is then a property of the relaxation path rather than of a Gaussian.

The clearest failure is the relaxed point defects, missed by every method. Its cause is in the data: every defect cell in GAP-18 is a hot molecular-dynamics snapshot, the model's defect–bulk error drifts with thermal amplitude, and the relaxed defect lies beyond the data along a direction that every snapshot shares. A delete-one jackknife cannot see such a bias, because deleting one snapshot does not move the fit along it, and a rank-one parameter correction built from the snapshots carries it to the relaxed geometry only partly and unevenly. The bias can, however, be measured from the training labels alone, by extrapolating the defect–bulk residual to zero thermal amplitude, and added as a discrepancy term; this raises the coverage to 20/20 (o3d12) and 18/20 (o4d20) without using any DFT QoI value, although the term was devised after the QoI misses had been seen. For the vacancy, Bayesian comparison of models for the training cells' size dependence shows that the bias depends on the cell size; the 215-atom extrapolation it selects is weakly determined, a reminder that such extrapolations depend on how the training data were made. Relaxed or low-temperature defect configurations in the training set would remove the need for the term, and the same diagnostic, a residual trend that extrapolates away from zero, can flag where such data are missing.

Mondrian conformal prediction[8], which calibrates a separate quantile in each group of a partition fixed in advance, is a natural extension for environments absent from training. The ace-jax force uncertainty provides eight such groups of atoms, set by the distortion and coordination of each atom's first shell[24]. On a Cantor-alloy model in the package's development tests[39], those groups with the hold-out transfer met their coverage targets on crack and dislocation cells left out of both training and calibration: 0.90 on the cracks, 0.89 at the crack tips and 0.94 on edge and screw dislocations, at a nominal 0.90. That test did not compare the groups with a single quantile. On silicon here the same groups change the out-of-distribution coverage of the atom-weighted quantile by at most 0.02 and leave the screw dislocations of o3d12 under-covered (0.79, against 0.77 with one quantile). Whether groups resolve the remaining under-coverage on strongly distorted environments, for example with finer distortion bands or a descriptor-novelty axis, is left to future work.

The study has limitations. The calibration assumes that the test split and the QoI structures are exchangeable with the training data, which the defects show is not always true. The hyperparameters are held fixed in the jackknife, so their own uncertainty is not propagated. The locality of the model, which cannot represent interactions beyond its cutoff, appears here only as part of the effective error. And twenty QoIs give coarse coverage estimates: 17 of 20 leaves a 95% credible interval from 0.65 to 0.96 for the coverage probability, so the evidence of § 3.2.3, which uses all the QoIs jointly, is the sharper comparison.

Conclusions

For linear ACE potentials, the delete-one-configuration (CR3) jackknife covariance, scaled once, gives uncertainties close to their target coverage at two levels. Per atom, with one conformal quantile from a held-out part of the training set, the force regions come close to their target coverage on held-out silicon data and on cracks left out of the fit, though for the smaller model screw dislocations left out are under-covered; a scale learned from the site descriptor evens the coverage across environments seen in training but does not improve it on those left out, and for the larger model lowers it. For derived properties, the same covariance with one scale fitted to held-out forces, propagated by implicit differentiation through each relaxation, covers most of 20 silicon QoIs against DFT, with intervals that shrink with the accuracy of the model. It outperforms the Bayesian posterior and the uncalibrated shape, and on these QoIs it performs comparably with POPS once the POPS box is numerically defined. Its advantage over POPS is not accuracy but calibration and numerical robustness: one interpretable scale, fitted to held-out errors, sets the width at both levels; the covariance needs no ridge or leverage cut beyond the fit's own prior, and does not depend on the parameterisation; and the per-atom regions are calibrated to a target coverage. On per-atom forces left out of the fit, POPS is not consistent between the two models: refitted without the cracks and dislocations, it covers 0.89 of the held-out atoms for o3d12, above the 0.76 of the scaled CR3 covariance, but 0.59 for o4d20, where the scaled CR3 covariance covers 0.91. Where the CR3 covariance fails, for relaxed point defects, the cause is a bias shared by all training snapshots of a kind, which no delete-one method can see; it can be measured from the training labels and added as a discrepancy term, which brings the coverage close to nominal. The methods are implemented in ace-jax.

Computational details

The training and test data are the GAP-18 silicon database[29], split once at random into 2227 training and 248 test configurations (seed 0; the split's content hash is c433c98d49bb). Both models are linear ACE potentials fitted with ace-jax[24] at a 5 Å cutoff, with least-squares reference energies, the smoothness prior of § 1.1.2, one shared noise scale and the structural weights of the ace-jax silicon accuracy–cost benchmark. Every calculation is in double precision. The QoIs are computed by a Python 3 port of the silicon test framework of Bartók et al.[29], run unmodified apart from a relaxed-ion force tolerance of 10−310^{-3} eV/Å for the elastic constants. Committees use 200 members drawn with fixed seeds. The fits, committees and analyses ran on workstations of the University of Warwick's Scientific Computing Research Technology Platform, and the cost timings of Figure 1 on a CPU node and an NVIDIA A100 GPU of the Sulis Tier 2 HPC platform.

What was fixed in advance. The specification of the study, committed before any QoI was computed (repository below), fixed the 21 QoIs and their tolerances, the four candidate scales λF\lambda_F, λE\lambda_E, λV\lambda_V and λpool\lambda_{\mathrm{pool}}, the three baselines, the committee size and the 90% rule that picks the delta method or the committee. λF\lambda_F was chosen as the headline before any QoI coverage was seen; the specification lists the four scales but does not record that choice. Decided later, after QoI results had been seen: the POPS leverage cut (the box was first built from every row, which gave unphysical members, § 3.2.5); B2 follows the reference package's default, the 50% cut) and its ridge floor (set by where the box is numerically defined, § 3.2.5), the member-rejection rule of the λF\lambda_F re-run (fixed before that re-run), the discrepancy term and its estimators (§ 2.0.2.2; computed from the training cells only, the test split's defect cells being left out), the per-atom force method (§ 1.1.5, chosen after comparing it with a learned scale on the test split and on the out-of-distribution set), the use of the BLR posterior instead of ARD for that stage (so that one posterior serves forces and QoIs; § 1.1.3 gives both), and Bayesian model comparison in place of the specification's fixed [0.8, 1.25] band on λV/λF\lambda_V/\lambda_F. None of these uses a DFT QoI value as an input, but they were not made blind.

Use of AI tools in the research. The analysis software of this study was written and run with Claude Code (Anthropic; model Claude Opus 5.5): the uncertainty propagation, calibration and validation code in ace-jax and the scripts that produce every table and figure. Claude Code was used to implement the analysis the author specified, to carry out the runs and to draft text. It did not generate or alter any data: every number comes from the stored results of that code. The author checked the code, the results and the text, including the parity tests and finite-difference checks of § 2.0.2. Session transcripts are available on request.

Code and data availability. The uncertainty methods, including the per-atom stage at fixed hyperparameters (--ard-prior fixed), are part of ace-jax version 0.5.3 (MIT licence, https://github.com/ACEsuit/ace-jax); the per-atom fits were run from its development branch before the release, and the parity and transfer analyses of § 3.2 reproduce exactly with the release. The analysis code, the results behind every table and figure, the generator of this paper and the original specification of the study are at https://github.com/keremdogruoglu/si-qoi-uq-paper; the version used here, release v1.0.0, is archived on Zenodo at https://doi.org/10.5281/zenodo.23986930. The GAP-18 database is available from its original publication[29]. The Python 3 port of the silicon test framework used here is the branch py3 of https://github.com/libAtoms/silicon-testing-framework, at commit f23b1df.

Acknowledgements

The author thanks Thomas Swinburne and George Simmons for useful discussions.

Calculations were performed on the Sulis Tier 2 HPC platform hosted by the Scientific Computing Research Technology Platform at the University of Warwick; Sulis is funded by EPSRC Grant EP/T022108/1 and the HPC Midlands+ consortium. We also used the Scientific Computing Research Technology Platform's desktop compute nodes.

Use of AI tools. Claude Code (Anthropic; model Claude Opus 5.5) was used to write and run the analysis software in this work (the propagation, calibration and validation code in the ace-jax repository and the scripts that produce every table and figure), and to help draft and revise the text. All numbers in the paper are generated by that code from the stored results. Claude Code usage for software development was supported by credits from Anthropic's AI for Science program. The author designed the study, reviewed the code, results and text, and takes full responsibility for the correctness of the resulting implementation and for the content of this paper.

References

  1. [1]B. J. K. Kleijn and A. W. van der Vaart, “The Bernstein–von Mises theorem under misspecification,” Electron. J. Stat. 6, 354–381 (2012).DOI
  2. [2]U. K. Müller, “Risk of Bayesian inference in misspecified models, and the sandwich covariance matrix,” Econometrica 81, 1805–1849 (2013).
  3. [3]T. D. Swinburne and D. Perez, “Parameter uncertainties for imperfect surrogate models in the low-noise regime,” Mach. Learn.: Sci. Technol. 6, 015008 (2025).arxiv.org/abs/2402.01810
  4. [4]F. Musil, M. J. Willatt, M. A. Langovoy, and M. Ceriotti, “Fast and accurate uncertainty estimation in chemical machine learning,” J. Chem. Theory Comput. 15, 906–915 (2019).DOI
  5. [5]G. Imbalzano, Y. Zhuang, V. Kapil, K. Rossi, E. A. Engel, F. Grasselli, and M. Ceriotti, “Uncertainty estimation for molecular dynamics and sampling,” J. Chem. Phys. 154, 074102 (2021).DOI
  6. [6]M. Kellner and M. Ceriotti, “Uncertainty quantification by direct propagation of shallow ensembles,” Mach. Learn.: Sci. Technol. 5, 035006 (2024).arxiv.org/abs/2402.16621
  7. [7]F. Bigi, S. Chong, M. Ceriotti, and F. Grasselli, “A prediction rigidity formalism for low-cost uncertainties in trained neural networks,” Mach. Learn.: Sci. Technol. 5, 045018 (2024).arxiv.org/abs/2403.02251
  8. [8]V. Vovk, A. Gammerman, and G. Shafer, Algorithmic Learning in a Random World (Springer, 2005).
  9. [9]R. J. Tibshirani, R. F. Barber, E. J. Candès, and A. Ramdas, “Conformal prediction under covariate shift,” in Adv. Neural Inf. Process. Syst., Vol. 32 (2019) pp. 2530–2540.arxiv.org/abs/1904.06019
  10. [10]R. F. Barber, E. J. Candès, A. Ramdas, and R. J. Tibshirani, “Predictive inference with the jackknife+,” Ann. Stat. 49, 486–507 (2021).arxiv.org/abs/1905.02928
  11. [11]I. R. Best, T. J. Sullivan, and J. R. Kermode, “Uncertainty quantification in atomistic simulations of silicon using interatomic potentials,” J. Chem. Phys. 161 (2024), 10.1063/5.0214590.DOI
  12. [12]R. Drautz, “Atomic cluster expansion for accurate and transferable interatomic potentials,” Phys. Rev. B 99, 014104 (2019).DOI
  13. [13]P. J. Huber, “The behavior of maximum likelihood estimates under nonstandard conditions,” in Proc. Fifth Berkeley Symp. Math. Stat. Probab., Vol. 1 (1967) pp. 221–233.
  14. [14]H. White, “Maximum likelihood estimation of misspecified models,” Econometrica 50, 1–25 (1982).
  15. [15]R. M. Bell and D. F. McCaffrey, “Bias reduction in standard errors for linear regression with multi-stage samples,” Survey Methodology 28, 169–181 (2002).
  16. [16]J. E. Pustejovsky and E. Tipton, “Small-sample methods for cluster-robust variance estimation and hypothesis testing in fixed effects models,” J. Bus. Econ. Stat. 36, 672–683 (2018).
  17. [17]J. G. MacKinnon, M. Ø. Nielsen, and M. D. Webb, “Cluster-robust inference: a guide to empirical practice,” J. Econometrics 232, 272–299 (2023).arxiv.org/abs/2205.03285
  18. [18]J. G. MacKinnon and H. White, “Some heteroskedasticity-consistent covariance matrix estimators with improved finite sample properties,” J. Econometrics 29, 305–325 (1985).
  19. [19]D. M. Allen, “The relationship between variable selection and data augmentation and a method for prediction,” Technometrics 16, 125–127 (1974), PRESS was introduced in Allen’s 1971 technical report.DOI
  20. [20]R. D. Cook and S. Weisberg, Residuals and Influence in Regression (Chapman and Hall, 1982).
  21. [21]H. Papadopoulos, V. Vovk, and A. Gammerman, “Normalized nonconformity measures for regression conformal prediction,” in Proc. IASTED Int. Conf. Artificial Intelligence and Applications (2008) pp. 64–69.
  22. [22]J. Lei, M. G’Sell, A. Rinaldo, R. J. Tibshirani, and L. Wasserman, “Distribution-free predictive inference for regression,” J. Am. Stat. Assoc. 113, 1094–1111 (2018).arxiv.org/abs/1604.04173
  23. [23]C. H. Ho, C. Ortner, and Y. Wang, “Flexible uncertainty calibration for machine-learned interatomic potentials,” npj Comput. Mater. 12, 225 (2026).DOI
  24. [24]ace-jax developers, “ace-jax: atomic cluster expansion potentials in JAX,” version 0.5.3, https://github.com/ACEsuit/ace-jax (2026).
  25. [25]G. Simmons, T. Hudson, T. D. Swinburne, and J. R. Kermode, “Review of uncertainty quantification for machine-learned interatomic potentials,” (2026), in preparation.
  26. [26]A. Grossfield and D. M. Zuckerman, “Quantifying uncertainty and sampling quality in biomolecular simulations,” in Annual Reports in Computational Chemistry, Vol. 5 (Elsevier, 2009).pubmed.ncbi.nlm.nih.gov/20454547
  27. [27]T. Lelièvre, M. Rousset, and G. Stoltz, Free Energy Computations: A Mathematical Perspective (Imperial College Press, 2010).
  28. [28]J. A. Hoeting, D. Madigan, A. E. Raftery, and C. T. Volinsky, “Bayesian model averaging: a tutorial,” Stat. Sci. 14, 382–417 (1999).
  29. [29]A. P. Bartók, J. Kermode, N. Bernstein, and G. Csányi, “Machine learning a general-purpose interatomic potential for silicon,” Phys. Rev. X 8, 041048 (2018).DOI
  30. [30]D. Perez, A. P. A. Subramanyam, I. Maliov, and T. D. Swinburne, “Uncertainty quantification for misspecified machine learned interatomic potentials,” arXiv:2502.07104 (2025).arxiv.org/abs/2502.07104
  31. [31]I. Maliov, P. Grigoriev, and T. D. Swinburne, “Exploring parameter dependence of atomic minima with implicit differentiation,” npj Comput. Mater. 11, 22 (2025).
  32. [32]W. C. Witt, C. van der Oord, E. Gelžnytė, T. Järvinen, A. Ross, J. P. Darby, C. H. Ho, W. J. Baldwin, M. Sachs, J. Kermode, N. Bernstein, G. Csányi, and C. Ortner, “ACEpotentials.jl: a Julia implementation of the atomic cluster expansion,” J. Chem. Phys. 159, 164101 (2023).
  33. [33]kermodegroup, “Interactive regression demo,” https://github.com/kermodegroup/demos, apps/regression-demo.py, scene lj-cr3 (2026).
  34. [36]T. D. Swinburne and D. Perez, “POPSRegression: Bayesian regression for low-noise data with misspecification uncertainty,” Python package, version 0.4.0, https://pypi.org/project/popsregression/ (2026).
  35. [38]T. Gneiting and A. E. Raftery, “Strictly proper scoring rules, prediction, and estimation,” J. Am. Stat. Assoc. 102, 359–378 (2007).
  36. [39]ace-jax developers, “Acceptance tests of the calibrated force uncertainty on a Cantor-alloy model,” https://github.com/ACEsuit/ace-jax/blob/main/bench/defect_uq/results/2026-10-03_rev2_acceptance.md (2026).
  37. [40]B. L. Shanks, G. Simmons, A. P. Bartók, and J. R. Kermode, “Uncertainty quantification design principles for machine learning interatomic potentials: lessons learned from hierarchical Bayesian inference,” arXiv:2206.08744v2 (2026).
  38. [41]R. E. Kass and A. E. Raftery, “Bayes factors,” J. Am. Stat. Assoc. 90, 773–795 (1995).

Paper details

vlnc:W_b8sazjtk
Version
v1 · latest
Published
Fresh — added to the Hub within the last 30 days
Added
License
CC BY 4.0
Usage

views · 2

downloads · 0

A view counts once per browser per day, a download each time the PDF is requested.

  • Physics and Astronomy

Claude Code

View statement
Looking for related papers…
Contents