betaFeedback

WorksPhysics and Astronomy

Materials representations organize transferable supervision for chemical extrapolation

Abstract

Materials discovery often requires chemical extrapolation beyond labeled data. When target compounds contain elements absent from training, which compounds of other elements provide transferable supervision remains unclear. Here we show that, without target-property labels, materials representations rank other elements by the supervision their compounds provide. Engineered descriptors rank them largely by elemental-property similarity, whereas a pretrained crystal-graph encoder ranks them by the geometry of its learned representation. At fixed training-set size, excluding the compounds of the encoder's six top-ranked elements raises formation-energy and bandgap errors by 100-110 meV per atom and 42-85 meV, versus little change for six lower-ranked elements. The rankings hold across properties, regressors and datasets. The encoder's ranking outperforms elemental-property similarity and periodic groups, suggesting that it captures how elements behave within compounds. Materials representations thus organize transferable supervision, which can be identified before model training, informing dataset assessment and label acquisition.

*Corresponding author(s). E-mail(s): [email protected]; [email protected];

Introduction

Machine learning accelerates scientific discovery by enabling rapid property prediction for candidate materials, molecules and reactions, reducing reliance on costly experiments or first-principles calculations [1, 2, 3, 4]. These predictions inform candidate selection in high-throughput screening and guide candidate generation in emerging generative approaches [5, 6, 7, 8]. As these approaches explore new chemistries, their success depends on whether models remain reliable when applied to chemistries not represented in their training data. However, recent studies show that strong performance on conventional random-split benchmarks can coexist with substantial degradation for unfamiliar chemistries [9, 10, 11]. This challenge is especially important for functional properties, for which costly measurements or calculations often produce small datasets with uneven chemical coverage [12].

Existing studies address related aspects of chemical extrapolation from different perspectives. Li et al. showed that unfamiliar compounds in poorly covered regions of materials-representation space generally exhibit larger prediction errors, linking extrapolation difficulty to the coverage of the labeled training data [10]. Influence functions and data-valuation methods can estimate the contributions of individual training samples, while uncertainty-based acquisition methods identify additional samples for labeling [13, 14, 15, 16, 17, 18]. These approaches, however, depend on target-property labels and typically on an already trained model. Chemical similarity, by contrast, relates elements independently of the property being predicted. Elemental properties and periodic-table coordinates are supplied as input features that relate an element absent from labeled training to the elements present in it [19, 20]. Learned elemental embeddings are often interpreted through their agreement with periodic trends [21, 22, 23]. It remains unclear which chemistries in the training data inform predictions for an unfamiliar chemistry, and whether chemical similarity or other relationships between elements can identify them.

Training supervised property models involves three conceptually distinct components: representation mapping, labeled coverage and regressor learning (Figure 1) [24]. Because each labeled compound enters supervised learning as a pair of representation and label, the representation mapping and labeled coverage are often considered together. Conceptually, however, they play distinct roles. A mapping ϕ\phi determines how compounds are organized in representation space, whereas the labeled dataset determines which regions in that space are paired with target-property values and exposed to downstream learning. Because a regressor gg learns representation–property relationships only from these labeled pairs, its predictions for a target chemistry absent from the labeled coverage rely entirely on pairs from other chemistries. Here we ask which chemically defined subsets of the labeled training data expose representation–property relationships most transferable to a given target chemistry, and whether these subsets can be identified from the representation mapping alone, without target-property labels or a trained regressor.

Figure 1. Representation mapping, labeled coverage and regressor learning. a, A representation mapping ϕ\phi, either an engineered descriptor or a trained encoder, maps a material xx in materials space X\mathcal{X} to a point z=ϕ(x)z=\phi(x) in representation space Z\mathcal{Z}, shown schematically with two coordinates z1z_1 and z2z_2. b, Labeled training data (points) cover part of Z\mathcal{Z} (blue region). Compounds from an unfamiliar chemistry may lie outside this labeled coverage (orange region). c, A downstream regressor gg learns from the labeled data, giving predictions y^=g(ϕ(x))\hat{y}=g(\phi(x)). Predictions within the labeled coverage are interpolations, whereas those outside it are extrapolations.

We define target chemistries using the leave-one-element-out (LOEO) protocol, in which compounds containing a held-out element XX are excluded from training and serve as the target set, while the remaining XX-free compounds form the training pool [10, 25]. We then divide the pool into overlapping source subsets, each comprising the compounds that contain a given element YY. Targets and sources are thus both indexed by elements, so that each source subset represents a distinct chemistry and each target–source pair corresponds to an element pair (X,Y)(X,Y). Because both are defined by composition alone, they are identical for every representation and model. We compare two widely used classes of materials representations that differ in how they are constructed. Engineered descriptors are built from predefined elemental properties and operations, whereas pretrained crystal-graph encoders learn representations through training. As representative examples, we use Matminer descriptors and the encoder of the pretrained interatomic potential eSEN-30M-OAM [26, 27, 28, 29, 30, 31, 32]. We evaluate formation energy and bandgap, representing thermodynamic and electronic properties, across JARVIS, MP and OQMD databases [33, 34, 35, 36, 37].

We find that materials representations rank source subsets by the transferable supervision they provide. Excluding top-ranked subsets at fixed training-set size substantially increases extrapolation error, whereas excluding lower-ranked subsets has little effect, and the rankings hold across properties, regressors and datasets. The engineered descriptor (ϕE\phi_{\mathrm{E}}) ranks source subsets largely by elemental-property similarity, whereas the pretrained crystal-graph encoder (ϕPCG\phi_{\mathrm{PCG}}) ranks them by directional alignment, which compares how compounds shift in representation space as the fractions of two elements increase. This alignment identifies useful subsets more effectively than elemental-property similarity or periodic-group membership.

Results

Element fraction tracks representation-space separation and extrapolation difficulty

Compounds containing a given chemical element XX (= H, He, Li, …) can differ substantially in its atomic fraction, fX=NX/Ntotalf_X = N_X/N_{\mathrm{total}}, where NXN_X is the number of XX atoms and NtotalN_{\mathrm{total}} is the total number of atoms in the compound. To examine how this variation relates to representation-space separation and extrapolation performance, we grouped these compounds into fraction intervals BX,kB_{X,k} with centers fX,kcf_{X,k}^{\mathrm{c}} and half-width ww (Figure 2a).

Element fraction resolves representation-space separation and extrapolation difficulty

Figure 2. Element fraction resolves representation-space separation and extrapolation difficulty. a, Fraction-resolved analysis. For each element XX, compounds are divided into XX-free (fX=0f_X = 0) and XX-present (fX>0f_X > 0) sets, and XX-present compounds are assigned to intervals BX,kB_{X,k} of the element fraction fX=NX/Ntotalf_X = N_X/N_{\mathrm{total}}, with center fX,kcf_{X,k}^{c} and half-width ww. b, Top, separation between the representation-space distribution of each fraction interval and that of XX-free compounds, quantified by the normalized Sinkhorn divergence S~ε\widetilde{S}_{\varepsilon} as a function of the interval center fXcf_X^{c}. The divergence is computed from the representations without dimensionality reduction or property labels. Marker shapes distinguish ϕE\phi_E and ϕPCG\phi_{\mathrm{PCG}}; colors distinguish JARVIS, MP and OQMD. Bottom, formation-energy extrapolation performance on JARVIS in a LOEO setting, in which models are trained on samples from the XX-free training pool and evaluated by R2R^2 on XX-present target compounds within each fraction interval. Fixed representations paired with RF, XGB and SVR are compared with scratch-trained ALIGNN, scratch-trained eSEN and transfer-learned eSEN. Intervals have w=0.1w = 0.1, so adjacent intervals overlap. Values are means over eligible held-out elements (64, 72 and 84 for JARVIS, MP and OQMD, respectively) and five sampling repetitions. c, Two-dimensional UMAP projections of JARVIS compounds in ϕPCG\phi_{\mathrm{PCG}} (top) and ϕE\phi_E (bottom). Mg-free compounds (fMg=0f_{\mathrm{Mg}} = 0) are shown in gray, and Mg-present compounds are colored by fMgf_{\mathrm{Mg}}.

We first examined how the representation-space separation between each fraction-interval distribution and the fX=0f_X = 0 reference distribution varies with the interval center fXcf_X^{\mathrm{c}} (Figure 2b). We quantified separation using normalized Sinkhorn divergence (S~ϵ\widetilde{S}_{\epsilon}), obtained by normalizing the Sinkhorn divergence by a reference divergence within the fX=0f_X = 0 distribution. For each fraction interval, we report the mean S~ϵ\widetilde{S}_{\epsilon} across elements. Separation generally increased with fXcf_X^{\mathrm{c}} in both ϕE\phi_{\mathrm{E}} and ϕPCG\phi_{\mathrm{PCG}}, indicating increasing departure from the fX=0f_X = 0 distribution.

We then evaluated formation-energy extrapolation toward different fraction intervals (Figure 2b). Each eligible element was used as a held-out element XX in turn. Models were trained on samples from the XX-free training pool and evaluated on XX-present target compounds within each interval. Besides pairing the ϕE\phi_{\mathrm{E}} and ϕPCG\phi_{\mathrm{PCG}} representations with random forest (RF), extreme gradient boosting (XGB) and radial-basis-function support-vector regression (SVR), we also evaluated scratch-trained ALIGNN, scratch-trained eSEN and transfer-learned eSEN on the JARVIS formation-energy dataset [5, 38, 39, 40]. For each fraction interval, we report the mean R2R^2 across held-out elements. Most models approached the interpolation performance at low fXcf_X^{\mathrm{c}} [10]. As fXcf_X^{\mathrm{c}} increased, performance generally declined and differences among models became more pronounced, revealing different extrapolation behaviors. This trend was observed across JARVIS, MP and OQMD (Figure 2b and Supplementary Figure 1 and Figure 2. Extrapolation difficulty therefore varies among the target compounds of a held-out element, with higher fXf_X generally posing a greater challenge.

Resolving compounds by fXf_X also offers a compositional view of representation overlap.

Li et al. found that regions with little overlap between XX-present and XX-free compounds were concentrated in Uniform Manifold Approximation and Projection (UMAP) projections of learned encoders but dispersed in those of engineered descriptors [10]. Using Mg-present compounds as an example (Figure 2c), low-fMgf_{\mathrm{Mg}} compounds were interspersed with Mg-free compounds in both projections. With increasing fMgf_{\mathrm{Mg}}, Mg-present compounds followed a concentrated progression in ϕPCG\phi_{\mathrm{PCG}} but extended along multiple dispersed branches in ϕE\phi_E, mirroring the reported distribution of low-overlap regions. Together with the rise in separation and extrapolation difficulty with fXf_X (Figure 2b), this suggests that low-overlap regions consist largely of compounds with high fXf_X. The element fraction thus provides a representation-independent coordinate for interpreting overlap between XX-present and XX-free compounds in the representation space.

Directional alignment identifies useful source subsets in pretrained

In the original representation space, compositional trajectories connecting the successive fraction intervals were approximately linear in ϕPCG\phi_{\mathrm{PCG}} (Supplementary Figure 3 and Figure 4), consistent with the concentrated progression of Mg-present compounds in the UMAP projection (Figure 2c). This linearity allows the change in representation with increasing fXf_X to be summarized by a direction vector for each element. We quantify alignment between the directions of different elements and show that it identifies source subsets that provide transferable supervision.

Figure 3. Directional alignment in the pretrained representation identifies useful source subsets. Changes in MAE (Δ\DeltaMAE) after source exclusion for formation-energy (a,c) and bandgap (b,d) prediction on OQMD using the ϕPCG\phi_{\mathrm{PCG}} representation, relative to a baseline trained on samples from the full X-free training pool. A source subset comprises compounds containing a source element Y but not the held-out element X. a,b, Removal of the top two, four or six source subsets from the training pool, ranked by AXYPCGA^{\mathrm{PCG}}_{XY} (filled markers and lines), and of six lower-ranked subsets (ranks 11–16; hollow markers). c,d, Top-six exclusion with source subsets ranked by alignment in the final compound representation (AXYPCGA^{\mathrm{PCG}}_{XY}), elemental-embedding similarity (AXYembdA^{\mathrm{embd}}_{XY}), alignment before message passing (AXYcompA^{\mathrm{comp}}_{XY}) and elemental-property similarity (AXYpropA^{\mathrm{prop}}_{XY}). The shaded column shows exclusion of source subsets whose source elements share the periodic group of X; their number varies with X. All conditions use 10,000 training compounds. Test sets contain target compounds with fX>0.4f_X > 0.4, filtering out compounds that contain any of the six highest-ranked source elements of the ranking measure used; each condition and its baseline share identical test compounds. Each marker is the mean of Δ\DeltaMAE pooled over 61 held-out elements and five sampling repetitions after discarding the lowest and highest 10% of values. Colors and marker shapes distinguish RF, XGB and SVR. Results on JARVIS and MP are shown in Supplementary Figs. 5 and 6, all values in Supplementary Figs. 7–9, and results with unfiltered test sets in Supplementary Figs. 10–12.

Figure 4. Source exclusion with engineered descriptors and element-specific extrapolation error. a, Δ\DeltaMAE after source exclusion for formation-energy (left) and bandgap (right) prediction on OQMD using the fixed ϕE\phi_{\mathrm{E}} representation. Filled markers and lines show removal of the top two, four or six source subsets from the training pool, ranked by elemental-property similarity, AXYpropA^{\mathrm{prop}}_{XY}; hollow markers show exclusion of six lower-ranked subsets (ranks 11–16). The shaded column shows the same-periodic-group control, whose exclusion count varies with XX. Results on JARVIS and MP are shown in Supplementary Figs. 13 and 14, all values in Supplementary Figs. 15–17, and results with unfiltered test sets in Supplementary Figs. 18–20. b, Extrapolation MAE on target compounds for held-out H, N, O, F, S, Se and Te on OQMD, for formation energy (top) and bandgap (bottom) with ϕPCG\phi_{\mathrm{PCG}} (left) and ϕE\phi_{\mathrm{E}} (right). Models were trained on 10,000 compounds sampled from each full training pool without source exclusion. Colors distinguish RF, XGB and SVR. The dashed line (right axis) shows the top-six score ⟨AXY⟩top6\langle A_{XY}\rangle_{\mathrm{top6}}, the mean of the six largest values of AXYPCGA^{\mathrm{PCG}}_{XY} (left) or AXYpropA^{\mathrm{prop}}_{XY} (right). Results on JARVIS and MP are shown in Supplementary Figs. 21 and 22.

For each element XX, we used five fraction intervals with centers fX,kc∈{0.1,0.3,0.5,0.7,0.9}f^c_{X,k} \in\{0.1, 0.3, 0.5, 0.7, 0.9\} and half-width w=0.1w = 0.1. We fitted a line to their centroids in the original representation space to obtain a compositional direction,

μX,k=aX+fX,kcvX+ϵX,k,(1)\mu_{X,k} = \mathbf{a}_X + f^c_{X,k}\mathbf{v}_X + \epsilon_{X,k}, \tag*{(1)}

where μX,k\mu_{X,k} and fX,kcf^c_{X,k} are the representation centroid and the center of interval kk; vX\mathbf{v}_X, aX\mathbf{a}_X and ϵX,k\epsilon_{X,k} are the fitted direction vector, intercept and residual, respectively. Directional alignment between elements XX and YY is defined as the cosine similarity of their fitted direction vectors,

AXY=vXTvY∥vX∥2∥vY∥2.(2)A_{XY} = \frac{\mathbf{v}_X^{\mathsf{T}}\mathbf{v}_Y}{\lVert\mathbf{v}_X\rVert_2\lVert\mathbf{v}_Y\rVert_2}. \tag*{(2)}

Values approaching one indicate similarly directed changes in representation space with increasing XX or YY content. We denote the alignment computed in ϕPCG\phi_{\mathrm{PCG}} by AXYPCGA^{\mathrm{PCG}}_{XY}, which can be interpreted as a geometric similarity between elements in the learned compound representation.

Computing AXYA_{XY} requires no target-property labels, but it is influenced by the collection of compounds used as probes in representation space. To assess this dependence, we ranked, for each element XX, the other elements by AXYPCGA^{\mathrm{PCG}}_{XY} computed separately from JARVIS, MP and OQMD, and compared the rankings between datasets using the Spearman rank correlation coefficient ρ\rho. The rankings were highly consistent, with element-averaged ρ\rho values of 0.92 for JARVIS–MP, 0.83 for JARVIS–OQMD and 0.82 for MP–OQMD.

To test whether AXYPCGA^{\mathrm{PCG}}_{XY} identifies source subsets that provide transferable supervision, we performed a source-exclusion analysis, removing selected source subsets from the training pool and measuring the resulting change in extrapolation mean absolute error (MAE). For each held-out element XX, source subsets were ranked by AXYPCGA^{\mathrm{PCG}}_{XY} computed from OQMD, the largest of the three datasets, and this single ranking was used for both properties and for every regressor and dataset. Starting from a baseline pool of all XX-free compounds, we removed the top two, four or six source subsets or, as a control, six lower-ranked subsets (ranks 11–16). Models were trained on 10,000 compounds sampled from each pool, so that conditions differed in chemical coverage but not in training-set size. Test sets consisted of target compounds with fX>0.4f_X > 0.4, for which extrapolation is more challenging.

MAEMAE increased progressively as the number of excluded top-ranked source subsets rose from two to four to six, whereas ranks 11–16 exclusion produced little change (Figure 3a,Figure 3b). On OQMD, top-six exclusion increased MAEMAE by approximately 100–110 meV atom−1^{-1} for formation energy and 42–85 meV for bandgap across the three regressors, compared with changes of less than 5 meV atom−1^{-1} and 2 meV for ranks 11–16 exclusion. The same pattern held on JARVIS and MP for every regressor and property (Supplementary Figs. 5–12). Ranking by AXYPCGA^{\mathrm{PCG}}_{XY} therefore identifies source subsets whose labeled examples provide transferable supervision for the target compounds. This single ranking, computed from OQMD without target-property labels, remained informative across properties, regressors and datasets, suggesting that the representation mapping largely determines which source subsets are useful.

We next asked whether AXYPCGA^{\mathrm{PCG}}_{XY} simply reflects conventional chemical similarity between elements. We compared it with two such measures, one based on membership in the same periodic group as XX and the other on AXYpropA^{\mathrm{prop}}_{XY}, the cosine similarity between 22-dimensional vectors of elemental properties such as electronegativity and covalent radius. Rankings by AXYPCGA^{\mathrm{PCG}}_{XY} and AXYpropA^{\mathrm{prop}}_{XY} agreed only moderately (ρ=0.46\rho= 0.46). Excluding the top six source subsets ranked by AXYpropA^{\mathrm{prop}}_{XY} increased MAEMAE, but less than excluding the top six ranked by AXYPCGA^{\mathrm{PCG}}_{XY}, and the same-group control, whose exclusion count varies with XX, also produced smaller increases (Figure 3c,Figure 3d). Alignment in the ϕPCG\phi_{\mathrm{PCG}} representation therefore departs from elemental-property similarity and identifies useful source subsets more effectively.

Message passing refines relationships captured by AXYA_{XY}

In ϕPCG\phi_{\mathrm{PCG}}, each atom is first represented by a learned embedding of its element; these atomic features are then updated through message-passing layers and averaged over all atoms to obtain the crystal representation [10, 31, 41, 42]. To determine whether the ranking captured by AXYPCGA^{\mathrm{PCG}}_{XY} is already present before message passing, we examined two measures derived from the elemental embeddings of ϕPCG\phi_{\mathrm{PCG}}. AXYembdA^{\mathrm{embd}}_{XY} is the cosine similarity between the embedding vectors of elements XX and YY. AXYcompA^{\mathrm{comp}}_{XY} is the directional alignment of compositional trajectories fitted to compound representations formed by averaging the element embeddings according to atomic fractions, that is, the representations before message passing.

Rankings by AXYembdA^{\mathrm{embd}}_{XY} were moderately correlated with those by AXYpropA^{\mathrm{prop}}_{XY} (ρ=0.42\rho= 0.42), suggesting that ϕPCG\phi_{\mathrm{PCG}} already departs from conventional chemical similarity at the embedding level. Message passing then substantially changed relationships in embeddings, as rankings by AXYPCGA^{\mathrm{PCG}}_{XY} were only moderately correlated with those by AXYcompA^{\mathrm{comp}}_{XY} (ρ=0.55\rho= 0.55) and AXYembdA^{\mathrm{embd}}_{XY} (ρ=0.48\rho= 0.48). For example, the six source elements ranked highest for Cu by AXYembdA^{\mathrm{embd}}_{XY} (Ni, Au, Pt, Fe, V and Mn) include Fe, V and Mn from groups 5–8, whereas those ranked highest by AXYPCGA^{\mathrm{PCG}}_{XY} (Ag, Au, Ni, Hg, Cd and Pd) all belong to groups 10–12 (Supplementary Table 1).

Excluding the six source subsets ranked highest by AXYembdA^{\mathrm{embd}}_{XY} or AXYcompA^{\mathrm{comp}}_{XY} increased MAE, but less than excluding those ranked highest by AXYPCGA^{\mathrm{PCG}}_{XY}, for every combination of property, regressor and dataset (Figure 3c,Figure 3d and Supplementary Figs. 5 and 6). The learned elemental embeddings thus provide a useful starting point, but the most informative ranking emerges once message passing places each element in its chemical environment.

Besides changing rankings, message passing also affects the linear geometry of the trajectories on which AXYA_{XY} relies. Because message passing makes each atomic feature depend on its local chemical environment, atoms of the same element can have different features, and linear compositional trajectories are not guaranteed by mean pooling alone. In ϕPCG\phi_{\mathrm{PCG}}, trajectories became progressively less linear over the first message-passing layers but remained approximately linear at the fourth layer used here (Supplementary Fig. 4). Li and Walsh found that atomic representations after message passing are still organized by element across foundation interatomic potentials with different architectures and training data [23]. Under mean pooling, such element-dependent organization suggests that the observed linearity is common to encoders extracted from pretrained interatomic potentials.

Elemental-property similarity identifies useful source subsets in ϕE\phi_{\mathrm{E}}

Like ϕPCG\phi_{\mathrm{PCG}}, ϕE\phi_{\mathrm{E}} can be viewed as a set of operations applied to elemental embeddings, but in ϕE\phi_{\mathrm{E}} both are predefined. Most of its descriptors are statistics of 22 elemental properties, such as electronegativity and covalent radius, pooled over the constituent elements or over property differences between Voronoi neighbors [26, 27, 28]. The 22-dimensional property vector of each element therefore acts as a fixed elemental embedding, and neighbor differences take the place of message passing. The similarity between elemental-property vectors, AXYpropA^{\mathrm{prop}}_{XY}, which served as a control for ϕPCG\phi_{\mathrm{PCG}} (Figure 3), is thus the counterpart of the embedding similarity AXYembdA^{\mathrm{embd}}_{XY} in ϕE\phi_{\mathrm{E}}. Of the pooling statistics, only the fraction-weighted means vary linearly with atomic fraction, so trajectories in ϕE\phi_{\mathrm{E}} were much less linear than in ϕPCG\phi_{\mathrm{PCG}} (Supplementary Fig. 3).

We tested whether AXYpropA^{\mathrm{prop}}_{XY} identifies useful source subsets by repeating the exclusion study with ϕE\phi_{\mathrm{E}}. MAE increased as the number of excluded top-ranked source subsets rose, whereas excluding six lower-ranked subsets (ranks 11–16) produced small changes (Figure 4a). On OQMD, top-six exclusion increased MAE by approximately 55–100 meV atom−1^{-1} for formation energy and 46–53 meV for bandgap across the three regressors, whereas exclusion of ranks 11–16 changed MAE by less than 7 meV atom−1^{-1} and 2 meV, respectively. The same graded pattern, with little change for ranks 11–16, was observed across all evaluated regressors, target properties and datasets (Supplementary Figs. 13–20).

In ϕE\phi_E, periodic-group membership was nearly as informative as elemental-property similarity. The same-group control removed two to five source subsets. With RF and XGB, its MAE increases were comparable to those of top-two to top-four exclusion by AXYpropA_{XY}^{\mathrm{prop}}, whereas with SVR they were smaller than those of top-two exclusion (Figure 4a). This agreement is expected, because the periodic table reflects trends in elemental properties, making group membership a coarse proxy for the elemental-property similarity on which ϕE\phi_E is built. In ϕPCG\phi_{\mathrm{PCG}}, by contrast, same-group exclusion produced smaller increases than even top-two exclusion by AXYPCGA_{XY}^{\mathrm{PCG}} for every combination of property, regressor and dataset (Figure 3 and Supplementary Figs. 5–9).

Top-ranked AXYA_{XY} values indicate element-specific extrapolation difficulty

Beyond ranking source subsets for a given held-out element, we asked whether AXYA_{XY} values also indicate which held-out elements are difficult to extrapolate to. For each held-out element XX, we defined the top-six score ⟨AXY⟩top6\langle A_{XY}\rangle_{\mathrm{top6}} as the mean of the six largest AXYA_{XY} values, using AXYPCGA_{XY}^{\mathrm{PCG}} for ϕPCG\phi_{\mathrm{PCG}} and AXYpropA_{XY}^{\mathrm{prop}} for ϕE\phi_E. A low score indicates that even the top-ranked source elements relate only weakly to XX. We compared this score with the extrapolation MAE obtained without source exclusion. Across held-out elements, higher scores were associated with lower MAE for every combination of representation, regressor, property and dataset, with Spearman ρ\rho ranging from −0.74 to −0.29 (Table 1). The two representations gave similar correlations for formation energy (−0.52 to −0.29), whereas for bandgap, correlations were stronger with ϕPCG\phi_{\mathrm{PCG}} (−0.74 to −0.65) than with ϕE\phi_E (−0.58 to −0.42) in every combination of dataset and regressor. The moderate strength of these correlations is expected, because extrapolation error also depends on factors that the score does not capture, such as the labeled coverage within useful source subsets and how effectively the regressor learns from them.

DatasetRegressorFormation energy ϕPCG\phi_{\mathrm{PCG}}Formation energy ϕE\phi_{\mathrm{E}}Bandgap ϕPCG\phi_{\mathrm{PCG}}Bandgap ϕE\phi_{\mathrm{E}}
JARVISRF−0.46-0.46−0.44-0.44−0.65-0.65−0.44-0.44
XGB−0.52-0.52−0.49-0.49−0.70-0.70−0.46-0.46
SVR−0.42-0.42−0.48-0.48−0.74-0.74−0.58-0.58
MPRF−0.40-0.40−0.30-0.30−0.67-0.67−0.42-0.42
XGB−0.44-0.44−0.39-0.39−0.71-0.71−0.47-0.47
SVR−0.33-0.33−0.29-0.29−0.73-0.73−0.56-0.56
OQMDRF−0.39-0.39−0.41-0.41−0.70-0.70−0.45-0.45
XGB−0.45-0.45−0.41-0.41−0.70-0.70−0.51-0.51
SVR−0.38-0.38−0.44-0.44−0.70-0.70−0.56-0.56

Table 1. Spearman rank correlations between top-six scores and chemical extrapolation error. Coefficients are Spearman ρ\rho calculated across held-out elements between the top-six score, ⟨AXY⟩top6\langle A_{XY}\rangle_{\mathrm{top6}}, and extrapolation MAE. The top-six score is the mean of the six largest values of AXYA_{XY} over source elements YY indexing the source subsets, using AXYPCGA^{\mathrm{PCG}}_{XY} for the pretrained crystal-graph encoder (ϕPCG\phi_{\mathrm{PCG}}) and AXYpropA^{\mathrm{prop}}_{XY} for the engineered descriptors (ϕE\phi_{\mathrm{E}}). AXYPCGA^{\mathrm{PCG}}_{XY} was computed from OQMD for all three datasets. MAE was evaluated on filtered test sets of target compounds with fX>0.4f_X > 0.4. Negative coefficients indicate that higher top-six scores are associated with lower error. RF, random forest; XGB, extreme gradient boosting; SVR, support-vector regression.

To illustrate this relationship for individual elements, we examined H, N, O and F, for which previous studies have reported particularly poor extrapolation [10, 25, 43]. In ϕPCG\phi_{\mathrm{PCG}}, these elements had lower ⟨AXY⟩top6\langle A_{XY}\rangle_{\mathrm{top6}} and generally higher extrapolation MAEs than the heavier chalcogens S, Se and Te (Figure 4b and Supplementary Figs. 21 and 22). In ϕE\phi_E, the correspondence was weaker, as elemental-property similarity gave O and F relatively high scores despite their poor extrapolation. For these elements, the element similarity learned by ϕPCG\phi_{\mathrm{PCG}} therefore tracks extrapolation difficulty more closely than elemental-property similarity does. Within H, N, O and F, however, the scores did not follow the order of the errors in either representation, indicating that representation-level measures alone cannot fully account for extrapolation difficulty.

Discussion

Our results show that materials representations can identify which source subsets of a training pool supply transferable supervision for target compounds, without requiring target-property labels. A single ranking for each held-out element remained informative across properties, regressors and datasets, indicating that which subsets are useful is set largely by the representation. These findings clarify the distinct roles that the three components in Figure 1 play in chemical extrapolation. The representation determines which compounds expose representation–property relationships that carry over to the extrapolation, labeled coverage determines whether labels of such compounds are available, and the regressor determines how effectively these relationships are learned. In this sense, a representation organizes supervision.

The two representations organized this supervision differently. In ϕE\phi_{\mathrm{E}}, useful source subsets largely followed elemental-property similarity and periodic-group membership, as expected for descriptors built from predefined elemental properties. In ϕPCG\phi_{\mathrm{PCG}}, the learned element similarity differed substantially from both criteria and identified useful source subsets more effectively than either. Unlike ϕE\phi_{\mathrm{E}}, ϕPCG\phi_{\mathrm{PCG}} receives no elemental properties as input. Its elemental embeddings are learned entirely during pretraining on compounds, and message passing further makes atomic features depend on local chemical environments. The difference may therefore arise because elemental properties characterize elements in isolation, whereas ϕPCG\phi_{\mathrm{PCG}} relates elements by how they behave within compounds.

Because the rankings require no target-property labels, they separate which source subsets can supply supervision from whether labeled data cover those subsets. This separation matters most when labels are scarce, as for many functional materials whose property datasets are small and chemically uneven [12]. Rankings computed from OQMD remained informative when applied to JARVIS and MP, so for a fixed representation they can be estimated once from a large collection of probe compounds and reused. A small labeled dataset could then be assessed by its coverage of the source subsets ranked highly for a target chemistry. Because probe compounds serve only to sample representation space, rankings computed from first-principles databases could also be applied to datasets with experimental labels. Poor extrapolation could likewise be traced to one of two causes. If even the top-ranked source subsets relate only weakly to the target chemistry, which is associated with higher extrapolation error (Table 1), a different representation may be needed. If highly ranked subsets instead lack labeled coverage, labels could be acquired within them, complementing acquisition criteria based on model uncertainty or data diversity [16, 17, 18].

More broadly, our results suggest that the value of labeled data for chemical extrapolation is determined not by the data alone but also by the representation through which they are learned. Which labeled compounds are most informative for a new chemistry can differ between representations, and this can be assessed before any property model is trained. Evaluating materials representations by how they organize supervision, alongside their predictive accuracy, could therefore guide both the choice of representation and the design of datasets for exploring new chemistries.

Methods

Datasets, representations and regression models

We used the JARVIS22, MP21 and OQMD21 datasets (hereafter JARVIS, MP and OQMD) from the Zenodo archive accompanying the study by Li et al. [17]. The engineered representation ϕE\phi_{E} consisted of the 273 Matminer descriptors precomputed for each compound and distributed with the same archive [17, 28]. Of these, 242 are statistics of 22 elemental properties, such as electronegativity, covalent radius and valence-electron counts, with 132 computed over the constituent elements of a compound (fraction-weighted mean, mean absolute deviation, minimum, maximum, range and mode), and 110 from property differences between neighboring atoms identified by Voronoi tessellation [26, 27]. A further six are derived from subsets of these properties, and the remaining 25 describe stoichiometry, charge balance, coordination, bond-length variation, packing efficiency and chemical ordering. The pretrained representation ϕPCG\phi_{\mathrm{PCG}} was obtained from the embedding layer and the first four message-passing layers of the pretrained eSEN-30M-OAM model [30, 31, 32]. For each crystal, we extracted the rotationally invariant scalar (ℓ=0\ell= 0) components of the node features after the fourth message-passing layer and averaged them over all atoms, giving a 128-dimensional representation. The encoder weights were kept fixed throughout this study.

Both representations were standardized feature-wise to zero mean and unit variance. For the geometric analyses, which use no target-property labels, the means and standard deviations were computed from all compounds in each dataset, which served as probes of representation space. For property prediction, they were computed from the sampled training set only.

Each representation was paired with three regressors, RF, XGB and SVR. RF and XGB used the hyperparameters reported by Li et al. [10]. SVR was implemented in cuML with a radial basis function kernel, a regularization parameter C=1.0C = 1.0, an ϵ\epsilon-insensitive loss with ϵ=0.1\epsilon= 0.1 and the kernel coefficient set by the ‘scale’ heuristic [44]. Hyperparameters were not tuned for individual representations, properties or held-out elements.

Fraction-resolved separation and performance

For each element XX, XX-present compounds (fX>0f_X > 0) were grouped into fraction intervals

BX,k={i:fX,kc−w<fX,i≤fX,kc+w},(3)B_{X,k} = \{i : f^{c}_{X,k} - w < f_{X,i} \le f^{c}_{X,k} + w\}, \tag*{(3)}

where fX,kcf^{c}_{X,k} is the center of interval BX,kB_{X,k}, ww is its half-width, and ii indexes compounds. For the analyses in Figure 2b, we used eight overlapping intervals with centers fXc=0.1,0.2,…,0.8f^{c}_{X} = 0.1, 0.2, \ldots, 0.8 and half-width w=0.1w = 0.1. Elements were analyzed only if every interval contained at least 50 compounds, giving 64, 72 and 84 elements for JARVIS, MP and OQMD, respectively.

Representation-space separation was quantified with the debiased Sinkhorn divergence (S~ϵ\widetilde{S}_{\epsilon}) implemented in GeomLoss, using a quadratic cost (p=2p = 2) and a blur of 0.05, which corresponds to ϵ=0.052\epsilon= 0.05^{2} [45]. Let ZX,k={zi:i∈BX,k}Z_{X,k} = \{z_i : i \in B_{X,k}\} denote the representations of the compounds in interval BX,kB_{X,k}, and ZX,0Z_{X,0} those of the XX-free compounds. The divergence was normalized by a reference divergence between two disjoint subsamples of the XX-free compounds, ZX,0(1)Z^{(1)}_{X,0} and ZX,0(2)Z^{(2)}_{X,0}:

S~ϵ(BX,k)=Sϵ(ZX,k,ZX,0)Sϵ(ZX,0(1),ZX,0(2)).(4)\widetilde{S}_{\epsilon}(B_{X,k}) = \frac{S_{\epsilon}(Z_{X,k}, Z_{X,0})}{S_{\epsilon}(Z^{(1)}_{X,0}, Z^{(2)}_{X,0})}. \tag*{(4)}

A value of one therefore corresponds to the separation between two samples of XX-free compounds.

For predictive evaluation, training sets of 50,000, 70,000 and 100,000 compounds were sampled from the XX-free training pools of JARVIS, MP and OQMD, respectively, and models were evaluated on the XX-present target compounds in each interval. For each held-out element, the training set was sampled five times with the test set fixed. R2R^2 was computed within each interval for each repetition and then averaged over held-out elements and repetitions.

For comparison, three end-to-end models were trained on JARVIS formation energies. ALIGNN used the configuration reported by Li et al. [10]. The scratch-trained eSEN model had four message-passing layers. The transfer-learned eSEN model had six message-passing layers, of which the first four were transferred from eSEN-30M-OAM and kept frozen, following the frozen-transfer-learning approach implemented in MLIP-FTL [40].

Source-exclusion analysis

In the source-exclusion analysis (Figure 3 and Figure 4a), the baseline training pool for a held-out element XX consisted of all XX-free compounds. The source subset for element YY comprised the compounds in this pool containing YY, so its membership depended on both XX and YY. Source subsets can overlap because a compound may contain several source elements. Exclusion pools were formed by removing the union of the top two, four or six source subsets ranked for XX by a given measure, or the union of the six subsets ranked 11–16. Rankings by AXYPCGA_{XY}^{\mathrm{PCG}} and AXYcompA_{XY}^{\mathrm{comp}} were computed from OQMD probe compounds and applied to source subsets in all three datasets; rankings by AXYembdA_{XY}^{\mathrm{embd}} and AXYpropA_{XY}^{\mathrm{prop}} do not depend on a compound collection.

In the same-group control, source subsets whose source elements share the periodic group of XX were removed. Five training sets of 10,000 compounds were sampled from each pool, so that conditions differed in chemical coverage but not in training-set size. Held-out elements were required to have at least 100 test compounds. The 32 elements assigned to group 3 (Sc, Y, the lanthanides and the actinides) were not used as held-out elements, because the same-group control would then remove an unusually large number of source subsets. These criteria yielded 60, 61 and 61 held-out elements for JARVIS, MP and OQMD, respectively.

Extrapolation was evaluated on target compounds with fX>0.4f_X > 0.4 using two test sets. The unfiltered set contained all such compounds, and the filtered set excluded compounds containing any of the six source elements ranked highest by the measure under study. Filtering removes a confound introduced by source exclusion. When the source subset for YY is removed from the training pool, target compounds containing both XX and YY require extrapolation to two elements absent from the labeled training data. Changes in their errors cannot then be attributed solely to the loss of transferable supervision from that source subset. For each measure, the same filtered set was used for the top-two, top-four, top-six and ranks 11–16 conditions. The same-group control used the AXYPCGA^{\mathrm{PCG}}_{XY} filter with ϕPCG\phi_{\mathrm{PCG}} and the AXYpropA^{\mathrm{prop}}_{XY} filter with ϕE\phi_E. Because the filter removes only compounds containing the six highest-ranked source elements, the test sets for the ranks 11–16 and same-group controls can retain target compounds that contain an element removed under these controls. Any resulting double extrapolation would, if anything, increase the ΔMAE\Delta\mathrm{MAE} of these controls, making their comparison with top-ranked exclusion conservative. Each exclusion condition was evaluated on the same test compounds as its baseline. For each held-out element and repetition, we computed the change in MAE, ΔMAE\Delta\mathrm{MAE}, relative to the baseline. ΔMAE\Delta\mathrm{MAE} values were pooled across held-out elements and repetitions and summarized by a trimmed mean, which discards the lowest and highest values to limit the influence of extreme values. The conclusions were insensitive to the trimming fraction, and we show results obtained by discarding 10% of values at each end. Results for the filtered test sets are shown in the main text; those for the unfiltered test sets are given in Supplementary Figs. 10–12 and 18–20 and lead to the same conclusions.

Data availability

Data in this study are available at https://huggingface.co/datasets/nims-spin-theory/Axy_extrapolation.

Code availability

Scripts in this study are available at https://github.com/nims-spin-theory/Axy_extrapolation.

Acknowledgements

This research was conducted using the Supermicro ARS-111GL-DNHR-LCC and FUJITSU Server PRIMERGY CX2550 M7 (Miyabi) at Joint Center for Advanced High Performance Computing (JCAHPC) and the NIMS Numerical Materials Simulator.

Funding

This study was partially supported by the MEXT Program: Data Creation and Utilization-Type Material Research and Development Project (Digital Transformation Initiative Center for Magnetic Materials), Grant Number JPMXP1122715503, and by JSPS KAKENHI Grant Numbers 25H01252 and 26K24734.

Author contributions

T.T. conceptualized and designed the project. E.X. performed the calculations and analyzed the data. Both authors contributed to the discussion and drafted the manuscript.

Competing interests

The authors declare no competing interests.

Supplementary information

Supplementary Figure 1–22 and Supplementary Table 1 are provided in a separate Supplementary Information file.

References

References

  1. [1]Morgan, D. & Jacobs, R. Opportunities and Challenges for Machine Learning in Materials Science. Annual Review of Materials Research 50, 71–103 (2020).
  2. [2]Butler, K. T., Davies, D. W., Cartwright, H., Isayev, O. & Walsh, A. Machine learning for molecular and materials science. Nature 559, 547–555 (2018).DOI
  3. [3]Ramprasad, R., Batra, R., Pilania, G., Mannodi-Kanakkithodi, A. & Kim, C. Machine learning in materials informatics: Recent applications and prospects. npj Computational Materials 3, 54 (2017).DOI
  4. [4]Xiao, E. & Tadano, T. High-throughput computational screening of Heusler compounds with phonon considerations for enhanced material discovery. Acta Materialia 297, 121312 (2025).
  5. [5]Xiao, E. & Tadano, T. Accurate screening of functional materials with machine-learning potential and transfer-learned regressions: Heusler alloy benchmark. npj Computational Materials 12, 133 (2026).
  6. [6]Takahara, I., Mizoguchi, T. & Liu, B. Accelerated inorganic materials design with generative AI agents. Cell Reports Physical Science 6 (2025).DOI
  7. [7]Han, X.-Q., Guo, P.-J., Gao, Z.-F., Sun, H. & Lu, Z.-Y. InvDesFlow-AL: Active learning-based workflow for inverse design of functional materials. npj Computational Materials 11, 364 (2025).DOI
  8. [8]Park, H. & Walsh, A. Guiding generative models to uncover diverse and novel crystals via reinforcement learning. Nature Machine Intelligence 8, 1087–1099 (2026).arxiv.org/abs/2511.07158
  9. [9]Tenorio, M., Rahman, M. H., Mannodi-Kanakkithodi, A. & Chapman, J. Out-of-distribution machine learning for materials discovery: Challenges and opportunities. Chemical Physics Reviews 7, 011317 (2026).DOI
  10. [10]Li, K. et al. Probing out-of-distribution generalization in machine learning for materials. Communications Materials 6, 9 (2025).DOI
  11. [11]Omee, S. S., Fu, N., Dong, R., Hu, M. & Hu, J. Structure-based out-of-distribution (OOD) materials property prediction: A benchmark study. npj Computational Materials 10, 144 (2024).arxiv.org/abs/2401.08032
  12. [12]Xu, P., Ji, X., Li, M. & Lu, W. Small data machine learning in materials science. npj Computational Materials 9, 42 (2023).DOI
  13. [13]Koh, P. W. & Liang, P. Understanding black-box predictions via influence functions. Proc. 34th International Conference on Machine Learning 70, 1885–1894 (2017). URL https://proceedings.mlr.press/v70/koh17a.html.arxiv.org/abs/1703.04730
  14. [14]Ghorbani, A. & Zou, J. Data Shapley: Equitable valuation of data for machine learning. Proc. 36th International Conference on Machine Learning 97, 2242–2251 (2019). URL https://proceedings.mlr.press/v97/ghorbani19c.html.arxiv.org/abs/1904.02868
  15. [15]Liu, T., Tho, Z. Y. & Barnard, A. S. Understanding the importance of individual samples and their effects on materials data using explainable artificial intelligence. Digital Discovery 3, 422–435 (2024).
  16. [16]Smith, J. S. et al. The ANI-1ccx and ANI-1x data sets, coupled-cluster and density functional theory properties for molecules. Scientific Data 7, 134 (2020).DOI
  17. [17]Li, K. et al. Exploiting redundancy in large materials datasets for efficient machine learning with less data. Nature Communications 14, 7283 (2023).DOI
  18. [18]Tan, A. R., Urata, S., Goldman, S., Dietschreit, J. C. B. & Gómez-Bombarelli, R. Single-model uncertainty quantification in neural network potentials does not consistently outperform model ensembles. npj Computational Materials 9, 225 (2023).DOI
  19. [19]Mahdavi, H., Honavar, V. & Morgan, D. Beyond training data: How elemental features enhance ML-based formation energy predictions. Digital Discovery 4, 2972–2982 (2025).DOI
  20. [20]Zhao, R. & Li, K. Element priors and target support shape chemical transfer in materials graph networks (2026). arXiv:2609.00915.arxiv.org/abs/2609.00915
  21. [21]Chen, C., Ye, W., Zuo, Y., Zheng, C. & Ong, S. P. Graph Networks as a Universal Machine Learning Framework for Molecules and Crystals. Chemistry of Materials 31, 3564–3572 (2019).DOI
  22. [22]Onwuli, A., Hegde, A. V., Nguyen, K. V. T., Butler, K. T. & Walsh, A. Element similarity in high-dimensional materials representations. Digital Discovery 2, 1558–1564 (2023).arxiv.org/abs/2307.00784
  23. [23]Li, Z. & Walsh, A. Platonic representation of foundation machine learning interatomic potentials. Nature Machine Intelligence 8, 830–840 (2026).
  24. [24]Bengio, Y., Courville, A. & Vincent, P. Representation Learning: A Review and New Perspectives. IEEE Transactions on Pattern Analysis and Machine Intelligence 35, 1798–1828 (2013).DOI
  25. [25]Witman, M. D. & Schindler, P. MatFold: Systematic insights into materials discovery models’ performance through standardized cross-validation protocols. Digital Discovery 4, 625–635 (2025).
  26. [26]Ward, L., Agrawal, A., Choudhary, A. & Wolverton, C. A general-purpose machine learning framework for predicting properties of inorganic materials. npj Computational Materials 2, 16028 (2016).DOI
  27. [27]Ward, L. et al. Including crystal structure attributes in machine learning models of formation energies via Voronoi tessellations. Physical Review B 96, 024104 (2017).DOI
  28. [28]Ward, L. et al. Matminer: An open source toolkit for materials data mining. Computational Materials Science 152, 60–69 (2018).DOI
  29. [29]Matminer Development Team. matminer: Data mining for materials science. Software package. URL https://hackingmaterials.lbl.gov/matminer/. Accessed: 24 September 2026.
  30. [30]Barroso-Luque, L. et al. Open materials 2024 (OMat24) inorganic materials dataset and models (2024). URL https://arxiv.org/abs/2410.12771. arXiv:2410.12771.
  31. [31]Fu, X. et al. Learning Smooth and Expressive Interatomic Potentials for Physical Property Prediction (2025). arXiv:2502.12147.arxiv.org/abs/2502.12147
  32. [32]Meta FAIR Chemistry. OMat24: Meta open materials 2024 models (2024). URL https://huggingface.co/facebook/OMAT24. Accessed: 24 September 2026.
  33. [33]Jain, A. et al. Commentary: The Materials Project: A materials genome approach to accelerating materials innovation. APL Materials 1, 011002 (2013).DOI
  34. [34]Saal, J. E., Kirklin, S., Aykol, M., Meredig, B. & Wolverton, C. Materials Design and Discovery with High-Throughput Density Functional Theory: The Open Quantum Materials Database (OQMD). JOM 65, 1501–1509 (2013).DOI
  35. [35]Kirklin, S. et al. The Open Quantum Materials Database (OQMD): Assessing the accuracy of DFT formation energies. npj Computational Materials 1, 15010 (2015).DOI
  36. [36]Choudhary, K. et al. The joint automated repository for various integrated simulations (JARVIS) for data-driven materials design. npj Computational Materials 6, 173 (2020).DOI
  37. [37]Horton, M. K. et al. Accelerated data-driven materials science with the Materials Project. Nature Materials 24, 1522–1532 (2025).DOI
  38. [38]Choudhary, K. & DeCost, B. Atomistic Line Graph Neural Network for improved materials property predictions. npj Computational Materials 7, 1–8 (2021).DOI
  39. [39]Choudhary, K. & DeCost, B. ALIGNN: Atomistic line graph neural network. GitHub repository (2021). URL https://github.com/usnistgov/alignn. Accessed: 24 September 2026.
  40. [40]Xiao, E. & Tadano, T. MLIP-FTL: Frozen transfer learning implementation to FairChem v1. GitHub repository (2026). URL https://github.com/nims-spin-theory/MLIP_FTL. Accessed: 24 September 2026.
  41. [41]Kim, S. Y., Park, Y. J. & Li, J. Leveraging neural network interatomic potentials for a foundation model of chemistry. npj Computational Materials (2026).
  42. [42]Sugiura, T. & Mizoguchi, T. Achieving robust extrapolation in materials property prediction via decoupled transfer learning. PRX Intelligence (2026). URL https://link.aps.org/doi/10.1103/pbrg-x848.DOI
  43. [43]Vasconcelos, H. C. & Meirelles, M. Interpretable formation-energy structure from composition, density, and band-gap descriptors. Applied Physics A 132, 748 (2026).DOI
  44. [44]Raschka, S., Patterson, J. & Nolet, C. Machine Learning in Python: Main Developments and Technology Trends in Data Science, Machine Learning, and Artificial Intelligence. Information 11, 193 (2020).
  45. [45]Feydy, J. et al. Interpolating between optimal transport and MMD using Sinkhorn divergences. Proc. 22nd International Conference on Artificial Intelligence and Statistics 89, 2681–2690 (2019). URL https://proceedings.mlr.press/v89/feydy19a.html.arxiv.org/abs/1810.08278

Paper details

vlnc:W_ta2chpp4
Version
v1 · latest
Published
Fresh — added to the Hub within the last 30 days
Added
License
CC BY 4.0
Usage

views · 5

downloads · 0

A view counts once per browser per day, a download each time the PDF is requested.

  • Physics and Astronomy

Not recorded for this paper

View statement
Looking for related papers…
Contents