WorksPhysics and Astronomy
ORCHID: Connecting Finite-Temperature Thermodynamic Evidence to Computational Materials Discovery
Abstract
Finite-temperature phase stability is essential to materials discovery, but the evidence needed to describe it remains scattered across decades of literature. ORCHID organizes this evidence around the thermodynamic states that the observations describe, using tiered agentic extraction guided by thermodynamic rules, expert supervision and explicit validation. The collection contains 73,542 values from 1,321 papers, represented with their thermodynamic states, phase identities, provenance, evidence classes and uncertainty, together with assessed phase assemblages and calculated references. A qualified subset is released in ESPEI-compatible form for CALPHAD assessment, and machine-readable access supports both human and AI-assisted reuse. Blind expert evaluation of 48 diagram fields gives 93.8% agreement, increasing to 97.9% after accounting for notation conventions. On a common formation-enthalpy benchmark, eight MLIPs show residual spreads close to their DFT reference, motivating evaluation of finite-temperature phase stability through PhaseForge. One-shot MLIP-PhaseForge descriptions of Cu-Zr recover 0.35-0.56 of observed phase occurrences at assessed field points. ORCHID-assisted refinement through PhaseForge+ recovers 9 of 13 invariant temperatures within 2 sigma and reduces held-out activity error from 5.1 to 1.7 kJ/mol. By supplying interpreted literature evidence directly to gradient-based refinement through PhaseForge+, ORCHID connects evidence recovery to programmatic CALPHAD assessment. With the initial description and fitting datasets prepared, the final Cu-Zr refinement completes in approximately 27 minutes, providing a foundation for continuously updated thermodynamic models and their use in materials discovery.
Introduction
Understanding phase stability—that is, which phases are stable and under which conditions—constitutes the starting point for the exploration of any materials space [1], an idea captured by Hume-Rothery’s dictum that phase diagrams are “the beginning of wisdom, not the end of it” [2]. For materials discovery, this understanding connects a predicted compound to the conditions under which it might be synthesized and retained during operation. While knowledge of the 0 K ground state and its neighborhood helps identify competing phases, evaluating synthesizability requires understanding their relative stability at finite temperature, together with the kinetics and processing pathways through which they can be obtained. Thermodynamic descriptions also provide the driving forces for phase transformations, linking processing conditions to the evolution of microstructure, and, through it, to material properties. This connection places phase stability at the foundation of integrated computational materials engineering (ICME).
Open first-principles resources have made information on ground-state stability increasingly accessible. The Materials Project [3], the Open Quantum Materials Database (OQMD) [4, 5] and AFLOW [6], for example, provide energies and structures from which competing compounds and their stability at 0 K can be examined, while the NOMAD repository [7] provides access to computational materials data contributed by the broader community. The experimental evidence needed to establish phase stability at finite temperature, however, remains scattered across decades of literature, in reports of phase boundaries, invariant reactions, calorimetric measurements and activities. Interpreting these measurements requires knowing the chemistry, temperature, pressure and phase constitution of the states they describe, information that is frequently distributed across a table, its caption, the surrounding text and earlier cited work. Recovering a useful observation therefore requires reconstructing the thermodynamic state to which the reported value refers, together with its basis and source.
Once the states represented by these observations have been established, the observations can be brought together to constrain a thermodynamic description of the system. The CALPHAD method provides a framework for constructing such descriptions by assessing parametric models of the Gibbs energies of the phases against experimental and computed evidence [1, 8]. Equilibrium is then obtained by minimizing the total Gibbs energy subject to conservation of mass at specified temperature and pressure. By incorporating different observations into a common thermodynamic description, this approach also provides a basis for extrapolation to higher-order systems. Open-source tools such as pycalphad [9] and ESPEI [10] support the calculation of phase equilibria and the estimation of model parameters and their uncertainties, respectively. The SGTE unary descriptions [11] and the binary assessments compiled by Hallstedt [12] provide openly reusable models. Thermodynamic database (TDB) files preserve these fitted descriptions, but the underlying evidence often has to be collected again when a model is assessed or refined.
The difficulty of recovering that evidence is partly linguistic and partly thermodynamic. The same phase can appear under different names in papers, assessed diagrams, TDB files and first-principles databases, while interpreting a reported formation enthalpy requires knowing its composition, reference state and normalization basis.
Recent literature-extraction frameworks demonstrate that language models can recover structured materials information [13], and data resources coupled to predictive models illustrate the value of making such information reusable [14]. For thermodynamic evidence, the relations among phases and conditions provide additional constraints on the interpretation of a reported observation. The Gibbs phase rule and the rules governing adjacent phase fields can be used to reject impossible readings and, where sufficient information is available, resolve ambiguous labels and identify phase assemblages. These constraints provide a basis for guiding extraction with physical knowledge, while language models and expert review address cases that require further interpretation.
Commercial CALPHAD databases embody decades of expert assessment of published and, where available, proprietary measurements [15, 16]. The fitted models make this accumulated knowledge available for calculations, but the observations supporting them are not generally released together as an openly accessible, machine-readable dataset. Constructing a new assessment or independently evaluating an existing one may therefore require collecting and interpreting evidence that has already been examined by others. The open computational resources described above demonstrate the value of organizing materials data so that they can support further calculations and model development. An analogous resource for experimental thermodynamic evidence would allow the observations underlying finite-temperature phase stability to be examined, combined and reused across assessments.
In this work, we present ORCHID (Open Repository for CALPHAD Hypotheses, Intelligence, and Discovery), which organizes thermodynamic evidence around the states that the observations describe (Figure 1). Measurements of thermodynamic properties constrain the Gibbs energies of phases, while phase boundaries, invariant reactions and assessed assemblages constrain the conditions under which those phases coexist. Describing these observations in terms of their chemistry, temperature, pressure and phase constitution allows different forms of evidence to be brought together while preserving their sources, evidence classes and uncertainties. Observations that meet the requirements for thermodynamic assessment are released in ESPEI-compatible form [10]. Through programmatic interfaces, researchers and AI agents can retrieve this evidence together with the information needed to interpret it, select observations for fitting, examine disagreements among sources and evaluate model predictions.
Figure 1. Connecting thermodynamic evidence to computational discovery. Literature, assessed diagrams, CALPHAD models and first-principles databases are linked through extraction constrained by thermodynamic rules, source verification and phase identities. The resulting observations retain their states, evidence classes, uncertainties and provenance for model fitting, comparison and automated access. Glyphs are schematic except the downsampled DFT–experiment and citation-network tiles; no publisher page or handbook drawing is reproduced.
The evidence collected in ORCHID also provides a basis for evaluating whether computational models reproduce the competition among phases at finite temperature. For machine-learned interatomic potentials (MLIPs), this extends evaluation beyond agreement with ground-state formation energies to the phase equilibria implied by their predictions. PhaseForge connects these predictions to CALPHAD descriptions, allowing the resulting equilibria to be compared with the observations recorded in ORCHID[17, 18, 19]. Where discrepancies are identified, the ESPEI-ready subset can support refinement through PhaseForge+, using selected observations for fitting and separate evidence for evaluation [20, 21]. Making this process accessible to researchers and AI agents provides a basis for continuously refining thermodynamic descriptions as new evidence becomes available, supporting the connection between materials prediction, synthesis and processing needed for self-driving materials discovery.
ORCHID: connecting thermodynamic evidence to models
The use of a reported value in a thermodynamic assessment requires knowing the state to which that value refers. For a melting point, this requires identifying the participating phases, while a formation enthalpy must be associated with the structure of the compound and its reference state. Recovering this information from the literature is complicated by the different forms in which the evidence is reported, from invariant temperatures in tables to heat-capacity curves in figures. In ORCHID, this evidence is organized around a description of the thermodynamic state that includes the chemical system, coexisting phases, temperature, pressure and phase compositions, together with the reference state of the property (Figure 1).
Organizing evidence by thermodynamic state allows independent measurements, predictions from assessed models and results from first-principles calculations or machine-learned potentials to be compared while retaining the identity of the individual observations. The present release connects these observations through their recorded state attributes; a planned extension will assign permanent state identifiers through an append-only registry. This organization also requires tracking changes in phase constitution within a reported property series. For heat-capacity, heat-content and lattice-parameter series that span a phase transition, the observations were attributed to the phases on the corresponding sides of the transition. Cases that could not be resolved from the source were flagged and excluded from fitting. The evidence class and source remain attached to each observation so that its origin can be examined alongside its thermodynamic interpretation. The present release focuses primarily on binary systems, although the representation extends to higher-order systems.
Comparing observations from different sources requires establishing whether the reported phase names refer to the same phase. Differences in naming among the experimental literature, thermodynamic models and first-principles databases make this correspondence difficult to establish from the names alone. ORCHID therefore uses a phase registry that associates reported names with a common phase identity within a binary system, connecting the observations to the corresponding definitions in TDB models. A structure prototype is assigned when the source information supports that identification unambiguously.
Establishing phase identity also allows evidence from assessed phase diagrams to be considered alongside numerical values from the literature. These diagrams summarize an expert interpretation of the available experiments, identifying the phase equilibria considered most consistent with that evidence. ORCHID captures this information from assessed binary diagrams in a phase-diagram handbook by recording the phases present at representative temperatures and compositions within the phase fields. The resulting assemblages preserve information about phase coexistence in a form that can be used in thermodynamic calculations. They are retained as a separate evidence class so that users can distinguish the assessment from the underlying measurements and decide whether to include it in a fit.
The assessed phase assemblages can be compared with equilibria calculated from thermodynamic models to identify differences in the predicted phase stability of the same system. ORCHID supports this comparison through binary phase diagrams calculated with pycalphad[9] using the SGTE collection of binary assessments compiled by Hallstedt[12]. These finite-temperature descriptions are complemented by binary convex hulls derived from the Materials Project and OQMD, with the calculations linked to the model or database release used. For compounds represented in these first-principles resources, calculated formation energies are paired with formation enthalpies from the experimental literature. Comparison across 965 compounds provides an empirical estimate of the uncertainty of DFT formation energies, which is used to determine their weight in the ESPEI datasets so that their influence in a fit reflects their agreement with experiment.
To support the construction and refinement of CALPHAD models, ORCHID releases observations as ESPEI datasets after checking that they meet the requirements for fitting. This requires a fully specified equilibrium state, phases that can be identified in the CALPHAD model and consistency with the rules of phase equilibria. Approximately 11,000 values, or about one in seven, meet these requirements. For these observations, uncertainties are reported in physical units together with the corresponding weights, making explicit how uncertainty determines their influence in a fit. The datasets also retain their evidence classes and quality flags so that users can select observations and reconsider their uncertainty budgets according to the purpose of the assessment (SI 1, Figure S10).
Figure S10. From a literature value to an ESPEI dataset. One released ESPEI-ready value followed through the release. a, The literature record (literature/systems/MG-PB.json, record ORC-GDBEZR33OB.63d4cf): system, phase, conditions with their bases, property, value, printed uncertainty, reference state, method, reader agreement and source; the colour of the card is its evidence class. b, The ESPEI dataset that contains it, trimmed, with the value highlighted and the orchid_uncertainty block that records the budget of each value and the weight used. c, The weight: ESPEI 0.9.1 takes one weight per dataset, here the experimental reference divided by the median effective uncertainty of the values of the dataset, and data are selected by tag with apply_tags. d, ESPEI datasets per evidence-class tag (experimental 1,118 plus 1,546 invariant datasets, model 504, assessed phase diagram 643, and the three DFT sources). Every line of panel b is rendered from the released file.
These decisions can also be supported through programmatic access to the observations and their associated thermodynamic information. An AI agent can query a chemical system, resolve phase identities across sources and inspect the evidence available for model development or evaluation. Because the returned records include evidence classes, uncertainty budgets, provenance and fitting status, the agent can distinguish observations ready for fitting from those that require further interpretation.
The range of tasks supported by these interfaces depends on the evidence available for the chemistry of interest. The present release brings together approximately 74,000 literature values from 1,300 papers, spanning nearly 1,500 chemical systems and 79 elements (Figure 2). Of these values, 66,551 satisfy the extraction and consistency rules. Coverage is predominantly binary, comprising approximately 58,000 values across nearly 1,000 systems, with additional observations for unary, ternary and higher-order systems. This distribution reflects the literature and assessments used to construct the collection, so an unrepresented system or phase does not imply an absence of experimental evidence. Approximately 850 binary systems in the literature collection also have calculated diagrams, allowing about 50,000 literature values to be examined alongside the corresponding model predictions.
Figure 2. Chemistry, properties and temperatures represented in ORCHID. a, Values by constituent element on a logarithmic scale; grey indicates no records. b, Property groups separated by evidence class. c, Temperatures of 55,516 records with an explicit scalar temperature, including invariant and transition temperatures as reported properties; 23 values above 4,500 K are outside the plotted range. d, A disjoint partition of the temperature metadata separates these scalar records from assumptions, ranges or bounds and expression terms, and other or unresolved cases. Counts describe released records, rather than independent measurements or fitting-eligible observations. The classification rules, binary-system coverage and source history are retained in SI 1.
Reusing this evidence requires preserving the distinctions among measurements, assessed interpretations and model predictions, even when they describe the same thermodynamic state. Figure 2 summarizes these evidence classes and distinguishes explicitly reported temperatures from assumptions, ranges and unresolved conditions. Observations that are incomplete or do not meet the requirements for fitting remain accessible with their status recorded, allowing users to examine the broader evidence when selecting data for an assessment. Figure S11 in SI 1 illustrates how records extracted from the same sentence
Figure S11. Anatomy of a record. a, A table row of an open-access paper and the released record of its first lattice constant; numbered marks pair printed items with the fields that record them (3, the value). Temperature and pressure are not printed and are recorded with their bases (assumed); the row’s other lattice constants are two sibling records. The kind of the cited work [52] is not printed, so the evidence class is unspecified. b, One sentence gives two records: the calculated eutectic temperature (631 °C, model) and the measured value the paper cites (634 °C, experimental); both carry the same verbatim quote and page. JSON excerpts are trimmed (. . . ). Crops are reproduced unchanged from Liao, Wang and Chen [34], an open-access article distributed under a CC BY 4.0 licence.
can retain different evidence classes and how assumed conditions remain distinguishable from those reported by the authors. The Appendix provides a visual guide to the eight connected data products. SI 2 gives their inventory, field definitions and access instructions, while SI 1 describes the extraction, calibration and evaluation procedures used in the analyses.
Physics-guided extraction and evaluation of the evidence
Recovering thermodynamic states from the literature
Recovering thermodynamic evidence requires establishing both what a source reports and the context needed to interpret it. ORCHID combines independent readings of the source with constraints on the phases, compositions and conditions associated with an observation. We assembled the literature by starting from the assessments in the SGTE binary collection [12], following their references to primary studies and including further assessments of the same systems. The resulting corpus contains approximately 1,300 papers published between 1975 and 2025. Extraction focused on tables and running text, leaving numerical values available only in plots for later digitization. Because a measurement may be reproduced in several papers, the records distinguish the reporting work from the original source wherever it could be identified, allowing reprinted evidence to be recognized.
For tables, two vision-language models independently interpreted the page image using instructions that incorporated the rules of phase equilibria. Their readings were compared with the PDF text layer where available, and agreement combined with deterministic checks supported acceptance. Disagreements were referred to a judge model or retained with their unresolved status recorded. Running-text extraction also used paired readings, retaining a value only when the quoted sentence and numerical value were found together on the cited page. Associated phases, methods and citations were retained when supported by that sentence or its immediate context. Since agreement between readers can reproduce an error in the PDF text layer, suspect formation-enthalpy signs were checked against rendered pages. These checks preserved the sign established by the source or recorded that the value was printed without one.
Interpreting a phase diagram requires combining the information printed in the source with the thermodynamic constraints on phase coexistence. A deterministic pass first traces boundaries, calibrates axes and identifies phase fields. Printed labels and the law of adjacent phase regions then constrain the assemblages in neighboring fields, while the phase rule constrains invariant reactions. A field is assigned an assemblage directly when only one satisfies these constraints. Otherwise, the remaining possibilities define the choices presented to the machine readers, with an adjudicator and, finally, a domain expert addressing unresolved cases. Figure 3 illustrates this progression on a synthetic Ni–Ti diagram with known field identities, showing how boundaries and neighboring phases provide information beyond that contained in an isolated label.
Once a phase field is identified, a temperature and composition within its interior define an observation of the phase assemblage. The distance from the boundary provides a measure of how securely the selected point lies within that field. For numerical records, checks similarly require a physically consistent set of phases and agreement between the assigned chemical system and the elements in the reported phase formulas. The thermodynamic state associated with a value is then established by combining this information with the table headers, property units, reference state and conditions reported elsewhere in the source. Corrections to the sign, unit, scale or chemical system require support from the rendered page, and the original representation is retained. Observations with unresolved normalization or reference states remain available for inspection but can be excluded from fitting. The reading rules and correction history are documented in SI 1, preserving the distinction between reported, assumed and inferred information.
Human calibration and consistency across the collection
We evaluated the extraction through controlled tests, expert readings and comparisons with thermodynamic assessments. The controlled tests used synthetic documents generated from SGTE descriptions, providing known numerical values and phase identities that could be compared with the extracted results. These documents were rendered as born-digital PDFs and degraded scans to examine the effect of document quality. The AI readers recovered invariant-reaction tables almost completely from born-digital PDFs and substantially from degraded scans, whereas deterministic tools recovered fewer invariant rows and had difficulty interpreting degraded diagrams. The benchmark also identified a prompt-dependent failure that produced empty tables, leading to a correction of the acceptance logic. The test results and rendering variants are retained in SI 1, Figure S5 and Table S6, allowing subsequent extraction methods to be evaluated under the same controlled conditions.
Figure S5. On synthetic papers whose truth is known, the AI readers recover most tables and text, whereas deterministic routes fail on many tasks. Scores of deterministic routes (top) and AI readers (bottom) on the 20 synthetic papers of the extraction benchmark in four renderings; hatched cells were not run for that rendering. The Gemini reader used the production prompts, and its zero scores for structure labels and property records come from the prompt version that returned placeholder rows (Supplementary Note 1). Claude Opus 5.5 was run on a subset of the documents in two renderings. Headline values are given in Supplementary Table S6.
| Metric | Extractor | Vector PDF | Raster 300 dpi | Degraded scan | Image-only PDF |
| table cells exact | pdfplumber (det.) | 0.76 | – | – | – |
| table cells exact | OCR cells (det.) | 0.45 | 0.45 | 0.16 | 0.17 |
| invariant rows recall | Camelot (det.) | 0.62 | – | – | – |
| invariant rows recall | AI reader | 0.99 | 0.99 | 0.85 | 0.85 |
| property records recall | Camelot (det.) | 0.94 | – | – | – |
| property records recall | AI reader | 0.01 | 0.00 | 0.01 | 0.00 |
| boundary recall | tracer (det.) | 0.58 | 0.54 | 0.74 | 0.74 |
| field recall | tracer (det.) | 0.26 | 0.17 | 0.07 | 0.07 |
| field recall | AI reader | 0.45 | 0.36 | 0.39 | 0.37 |
| structure labels | AI reader | 0.00 | 0.00 | 0.00 | 0.00 |
| text values recall | regex (det.) | 0.97 | 0.86 | 0.74 | 0.74 |
| text values recall | AI reader | 0.53 | 0.34 | 0.31 | 0.27 |
| text values precision | AI reader | 0.99 | 1.00 | 0.98 | 0.96 |
Table S6. Extraction benchmark: baseline scores by rendering. det.: deterministic route; AI reader: Gemini 3.8 Flash with the production prompts, as a single reader; –: not run for that rendering. Scores from the prompt version that returned placeholder rows (Supplementary Note 1).
To evaluate the interpretation of assessed diagrams, a domain expert first reviewed 126 difficult fields with access to the adjudicator’s reasoning, accepting 92 of the 93 proposals for which a phase assemblage had been named. This review was followed by a blind evaluation of 48 fields, one from each of 48 diagrams that the expert had not previously reviewed. Without seeing the machine’s answer, the expert assigned phases that agreed with the extraction in 45 cases (93.8%, Wilson 95% interval 0.83–0.98). Agreement increased to 47 cases (97.9%, interval 0.89–1.00) after accounting for two notation conventions and included all 28 fields identified through printed labels and thermodynamic rules. The remaining discrepancy involved three closely spaced line compounds in an ambiguous drawing, for which the expert’s interpretation was adopted.
For numerical extraction, evaluation addresses both the location of the reported value and the thermodynamic interpretation assigned to it. All approximately 5,700 running-text records retain a passage found verbatim on the cited page that contains the printed number. Source verification establishes where the value was reported, while phase-recovery review and the audits described below examine its assigned property, phase and reference state. These checks evaluate specific components of the extracted records; estimating precision across the complete numerical collection requires an independent human evaluation of a representative sample.
A subsequent recovery pass assigned phase labels to numerical observations whose labels had remained unresolved during extraction. To examine these assignments, a separate assistant context reread a seeded sample of 100 proposals from the 1,268-record batch, recording its interpretation of the source before viewing the proposed answers (Figure 4b). The readings agreed exactly in 84 cases and differed only in notation in another 15, giving agreement on phase identity in 99 of 100 cases (Wilson 95% interval 94.55–99.82%). The remaining assignment was withheld because the source did not establish a unique phase identity. This AI-assisted review measures agreement for the sampled recovery proposals, with record-level intervals that do not account for dependencies within a table. The review used shared model capabilities and may have involved prior exposure to source pages, so it does not constitute an independent human evaluation. Subsequent corrections and release snapshots are tracked separately.
Figure 4. Source-based evaluation of phase interpretation. a, A human expert assigned phases to 48 assessed-diagram fields without seeing the machine answer, agreeing on 45 fields as written and 47 after accounting for notation. b, An answer-key-blinded review in a separate assistant context examined 100 randomly selected proposals from a batch of 1,268 phase recoveries. Exact labels agreed in 84 cases and phase identity in 99, with one unresolved prototype retained as a disagreement. Wilson 95% intervals describe field- or record-level agreement within these separate samples and omit within-source dependence. Panel b uses AI-assisted review, with shared-model capabilities and possible prior exposure to source pages. Sampling, adjudication and the separate evaluation of numerical records are documented in SI 1.
Figure 3. Thermodynamic constraints guide the interpretation of phase diagrams. Synthetic Ni–Ti diagram generated from the SGTE assessment of Tokunaga et al. (2004), with known field identities. a, The source drawing (grey) and the boundary lines extracted from its vector paths (black). b, Enclosed fields. c, Assigned phase assemblages. d, Representative interior points and a drawing-based tie-line. Field names follow the benchmark truth key; narrow fields may not admit an interior point with the required margin. The diagram is synthetic, and no handbook image is reproduced. Geometry thresholds and label mappings are given in SI 1.
To examine errors beyond phase assignment, separate language-model auditors combined targeted searches with seeded random samples, returning to source pages to investigate suspected discrepancies. The audits distinguished errors in numerical values and thermodynamic states from errors in other metadata, identifying systematic problems involving units, phase mappings, compositions and the pairing of experimental observations with DFT entries. Repeated sampling also identified errors introduced during correction, making verification of the revised records part of the audit procedure. The evaluated snapshots and subsequent changes are documented in SI 1, together with sample sizes, error rates and correction details.
A final agent-assisted re-audit of the third-round release covered all six audit scopes, with its results and subsequent corrections reported in SI 1. A source-based review of 200 records identified 189 that met all criteria for direct use in thermodynamic fitting (SI 1). SI 1 also reports a preliminary human review of a frozen sample of 300 records, which provides additional feedback on the reviewed observations without estimating precision across the complete collection.
The consistency of the recovered observations can also be examined by comparing them with equilibria calculated from assessed Gibbs-energy descriptions. For invariant reactions, experimental temperatures are matched to SGTE predictions by chemical system, reaction type and phase identities, giving approximately 3,600 observations across 400 systems. Where several assessments provide a match, their median prediction is used so that each observation enters the comparison once. The temperature differences have a median of 0.1 K and a robust spread of 5.9 K, with 73% lying within 10 K (Figure 5a,Figure 5b). Nearly all of these observations come from papers that also supply the corresponding SGTE assessments, making this comparison a check on the recovery of temperatures and reaction identities. Disagreements identify observations or assessments for closer examination.
Figure 5. Finite-temperature evidence compared with CALPHAD assessments. a, Experimental invariant temperatures against matching SGTE-calculated invariants for 3,616 observations in 401 systems. b, Temperature residuals, with median 0.1 K and robust spread 5.9 K; 73% lie within 10 K. The complete temperature comparison is retained in SI 1, Figure S15. Of the matched observations, 3,599 originate in a paper supplying an SGTE assessment, so agreement is a consistency check with substantial shared-source dependence. c, Assessed assemblage points over one calculated binary diagram, classified by agreement with the available assessments. d, Agreement fractions across systems with at least five comparable point–assessment pairs; dashed line, aggregate agreement; dotted line, median system. Of 30,769 comparisons, 9,281 lack a comparable phase mapping; 17,876 of 21,488 comparable pairs agree. Separately, 8,333 of 9,425 comparable points agree with at least one assessment. These two denominators answer different questions. All scientific panels retain the data and geometry of the source figures; no handbook drawing is reproduced.
The comparison extends to phase assemblages recorded in the handbook, provided that phase identities can be matched to those in the thermodynamic assessments. Because several assessments of a system may incorporate different selections of evidence, agreement is evaluated both for individual point–assessment pairs and for points that match at least one assessment. The first comparison gives agreement in 17,876 of 21,488 comparable pairs, while the second gives agreement for 8,333 of 9,425 points (Figure 5c,Figure 5d). Examining disagreements revealed systematic annotation errors that were corrected before release, as well as unresolved differences between the handbook interpretation and the fitted Gibbs-energy descriptions. Since the handbook diagram and the assessment are independent. dent expert evaluations of the same system, often based on different selections of evidence and made at different times, a disagreement between them does not by itself indicate an error in the extraction, in the experiments or in the assessment, and each point therefore records the outcome of its comparison with every assessment of the system. These differences remain accessible so that the assessed-diagram evidence retains its identity independently of agreement with a model. An independent formation-enthalpy compilation [22] provides an additional check on extraction, units and signs, as documented in SI 1.
Uncertainty and compatibility with CALPHAD assessment
Assigning an observation a weight in a fit requires accounting for variation between independent measurements of the same thermodynamic state. These measurements can differ by more than their reported uncertainties, indicating that the precision stated within a study does not capture all sources of variation between studies. ORCHID estimates this additional variation by grouping measurements of the same property and state obtained using the same class of experimental method in different primary studies, excluding repeated reports of the same measurement. Differences between pairs of measurements provide an estimate of the scatter associated with the method. Giving equal weight to each state prevents frequently studied compounds from dominating the calibration and establishes a minimum uncertainty, or floor, for observations obtained by that method.
For formation enthalpies measured by solution and reaction calorimetry, 80 pairs spanning 47 states give an estimated scatter of approximately kJ (mol of atoms), rounded upward to a floor of kJ (mol of atoms). This estimate includes the variation present in the collected evidence, arising from experimental scatter and any remaining extraction or reference-state errors. Where comparisons are insufficient to estimate a floor, documented literature estimates provide the starting uncertainty [23, 24]. The effective uncertainty used for fitting takes the larger of the reported uncertainty and the method floor, then incorporates reading resolution and applicable contributions from composition, reference state and inconsistency. The reported uncertainty is preserved separately, and SI 1 documents the estimator, sampling requirements and combination rules.
Using these observations in ESPEI requires representing their thermodynamic states in the expected form and mapping their phases to the selected CALPHAD model. We tested this compatibility by evaluating likelihoods against the named TDB files, obtaining finite values for all 8,589 released datasets tested. Another 387 datasets failed the check and were excluded, with the reasons retained. These checks establish compatibility with the fitting workflow, while the adequacy and uniqueness of the thermodynamic model are evaluated during assessment. Physical uncertainties and the basis of the exported weights remain available so that assessors can select evidence classes and revise the weighting policy for a particular fit. Preserving these choices allows the same observations to support both evaluation of computational predictions and refinement of thermodynamic descriptions.
From ground-state energies to finite-temperature MLIP benchmarks
Experimental formation enthalpies provide a common reference for comparing compound energies predicted by first-principles calculations and machine-learned interatomic potentials (MLIPs) (Figure 6). Determining which phases coexist at finite temperature, however, requires the relative Gibbs energies of competing compounds, solid solutions and liquids. Their balance determines the phase boundaries and invariant reactions, providing a test of how the computational description represents competition among phases. ORCHID supplies both formation enthalpies and observations of finite-temperature equilibria, allowing us to examine whether agreement with the former carries over to predictions of the latter (Fig. 7).
Figure 6. Formation-energy comparisons against the same experimental observations. Calculated formation energies from the Materials Project (a), OQMD (b) and eight MLIPs (c–j) are paired with 1,138 experimental formation-enthalpy records representing 455 matched structures in 185 binary systems. All panels use identical axes; the solid line denotes 1:1 agreement and the grey band spans ±10 kJ (mol of atoms). Panel annotations give the robust spread of the calculated-minus-experimental residuals, evaluated using all records, and the number outside the displayed window. The window is −110 to 20 kJ (mol of atoms) on both axes; no observations are excluded from the statistics on this basis. The MLIP energies use the relaxed structures and MP2020 correction convention of the matched benchmark described in SI 1. Similar deviations are visible across models; their correlations with Materials Project residuals are quantified in Fig. 7b.
What formation-energy agreement can distinguish
Comparing formation-energy errors among models requires accounting for differences in chemistry, structure matching and database reference conventions. We evaluated eight potentials on a common subset of 1,138 experimental records representing 455 compounds in 185 binary systems, using relaxed Materials Project structures and the specified MP2020 correction convention. Evaluating the same compounds removes differences in chemical coverage as a source of variation among models. Uncertainty intervals were obtained by resampling chemical systems together with all their records, preserving the dependence among observations within a system. These statistics describe the matched benchmark, while SI 1 reports the full DFT calibration, including the effects of chemical stratum, structure matching and reference conventions on the associated uncertainty weights.
The parity plots show similar deviations from experiment for DFT and the MLIPs (Figure 6). Across the common benchmark, the robust spreads of MLIP formation-enthalpy residuals range from to kJ (mol of atoms), compared with kJ (mol of atoms) for the Materials Project (Fig. 7a). All system-bootstrap intervals for the difference in spread between an MLIP and the Materials Project include zero. The residuals are also strongly correlated with those of the Materials Project, with Pearson correlations of – (Fig. 7b). These spreads exceed the effective calorimetry floor of kJ (mol of atoms) used in this comparison. Within this benchmark, the potentials thus approach the agreement of their DFT reference with experiment, while their similar formation-energy errors provide little basis for distinguishing their ability to predict phase stability.
Evaluating these potentials for materials discovery therefore requires testing the phase stability that follows from their predicted energies. Frameworks such as Matbench Discovery [25] compare formation-energy and K crystal-stability predictions, while observed phase boundaries, invariant reactions and phase constitutions provide additional tests over temperature and composition. ORCHID supplies the experimental states needed for these comparisons, and PhaseForge connects MLIP predictions to the CALPHAD calculations through which their finite-temperature equilibria can be evaluated.
One-shot phase-stability prediction through PhaseForge
We demonstrate this connection for Cu–Zr by using PhaseForge to construct CALPHAD descriptions from eight MLIPs through the same procedure (Figure 9a) [17, 18, 19]. The descriptions combine compound and solid-solution energies, liquid molecular dynamics and SGTE unary references, without fitting to the ORCHID observations used for evaluation. We use “one-shot” to denote this absence of fitting to the benchmark evidence, rather than the number of calculations or the absence of assumptions in the free-energy model. Applying a common construction procedure allows the resulting workflows to be evaluated by comparing their predicted equilibria with the phases recorded at specified temperatures and compositions (Fig. 7c).
Figure 9. Refinement with ORCHID, illustrated for Cu–Zr. a, PhaseForge constructs the starting description from MLIP compound, solid-solution and liquid energies on SGTE unary references. PhaseForge+ combines these with uncertainty-weighted ORCHID observations using the gradient-capable ESPEI development branch. b, Refined diagram (violet), initial MLIP description (dashed) and SGTE assessment (grey), with fitted and held-out literature points and assessed assemblages. Refinement recovers 9 of 13 invariant temperatures within and reduces held-out activity error from 5.1 to 1.7 kJ mol. c, Predicted and observed activities at all 12 withheld Cu and Zr observations, evaluated at their recorded temperatures and compositions with pure liquid reference states. d, Corresponding chemical-potential residuals, ; lines connect the initial and refined predictions for an observation. The sources provide no measurement uncertainties for these points. Holdouts are individual observations within Cu–Zr and may share sources with fitted evidence. The final fit adds assessed-field points after the preregistered evaluation, so its assessed-field recall is in sample. Remaining phase-stability discrepancies are discussed in the text; SI 1 documents construction, fitting and data splits.
For each phase , the recall is the fraction of evidence points containing that phase at which the model also predicts its presence. The prevalence-weighted score is , where is the number of evidence points containing the phase. This score therefore measures the fraction of observed phase occurrences recovered by the model¹⁷. Precision and F1 are reported alongside recall to account for predictions of additional phases absent from the observed assemblage. Points within assessed phase fields and points on boundaries are evaluated separately, with the tolerance rules and bootstrap procedures documented in SI 1. The benchmark thus evaluates the phases predicted at specified thermodynamic states, complementing the comparison of formation energies.
At the 28 assessed Cu–Zr field points, the eight one-shot descriptions obtain weighted recalls of 0.35–0.56, compared with 0.72 for the SGTE assessment and 0.70–0.98 for nine later assessments (Fig. 7c). The overlapping intervals do not resolve an ordering among the eight MLIPs, but examination of the predicted equilibria reveals distinct errors in phase stability. Some descriptions underestimate liquid stability, while others overestimate it. All eight descriptions leave B2 CuZr unstable at the four tested field points where it is observed, despite including this phase in their models. Three other assessed phase classes are absent from the common starting model vocabulary, making their omission a consequence of the construction procedure. Figure 8 separates these errors by state and phase identity, distinguishing missing observed phases from additional predicted phases. These comparisons show how finite-temperature equilibria can reveal differences among workflows with similar aggregate formation-energy errors.
Figure 8. Finite-temperature errors depend on the state and on the phases represented by the model. Eight one-shot MLIP–PhaseForge descriptions are evaluated at the same 28 assessed Cu–Zr field points, ordered by temperature and then composition. a, Number of observed phases missing from the prediction. b, Number of predicted phases absent from the assessed assemblage. c, Fraction of observations of a phase at which that phase is missed. Hatched cells identify phases absent from the model vocabulary; they are retained in the field scores but distinguished from represented phases that are predicted incorrectly. C15b and CuZr2 use the physical-phase grouping of the benchmark. The panels compare state-specific errors within Cu–Zr; the overlapping aggregate-score intervals do not resolve a ranking of the MLIPs. Point coordinates, phase supports and grouping rules are given in SI 1.
Figure 7. Similar formation-energy errors can accompany different finite-temperature failures. a, Robust formation-enthalpy residual spread for eight MLIPs and two DFT references on the same 1,138 records (455 compounds, 185 binary systems), with 95% intervals from 1,000 system-bootstrap resamples. All intervals for the MLIP-minus-MP spread include zero. The experimental band is the effective calorimetry floor used in this analysis. b, GRACE residuals against MP residuals; 25 pairs outside ±40 kJ (mol of atoms)⁻¹ are omitted from the plotted window. The strip gives residual correlations for all models. c, Weighted phase recall at 28 assessed Cu–Zr field points for one-shot PhaseForge descriptions, with point-bootstrap intervals. MLIP point estimates span 0.35–0.56 and their intervals overlap. The grey band represents nine later assessments; the dashed line is the SGTE assessment. The PhaseForge+ star is an in-sample result after refinement with ORCHID and is not a one-shot prediction. The benchmark inputs and construction assumptions are documented in SI 1.
PhaseForge+ established the refinement of MLIP-derived CALPHAD descriptions using thermodynamic evidence [21]. ORCHID extends this capability to the recovery and preparation of that evidence, connecting observations dispersed across the literature to the states, phase identities and uncertainty estimates required for fitting. To our knowledge, this is the first integration of literature-derived thermodynamic evidence at this scale with the evaluation and refinement of MLIP-derived descriptions. The Cu–Zr demonstration shows how this prepared evidence can enter gradient-based refinement directly. This integration addresses both the preparation of the evidence and the optimization of the model, tasks that can otherwise require months of manual work. It points toward a mode of CALPHAD assessment in which descriptions can be assembled and updated as new experimental and computational evidence becomes available.
Using the evidence to refine thermodynamic descriptions
The comparison of predicted and observed phase stability also provides information that can be used to refine a thermodynamic description. This requires a choice of the observations to be included in the optimization and of the uncertainty assigned to them, while preserving separate evidence for evaluation. Since the ORCHID observations are already compatible with ESPEI, their thermodynamic states do not need to be reconstructed from the literature to perform this refinement. The retained uncertainty budgets determine the relative influence of experimental and computed information in the fit.
The approach can be used to revise an existing assessment or refine a description initialized from first-principles or MLIP information. Here, we consider the latter case for Cu–Zr using PhaseForge+, with application to other systems requiring the corresponding state and phase mappings.
For this demonstration, PhaseForge+[21] starts from the GRACE-based description used in the benchmark. ORCHID phase equilibria, activities and formation enthalpies constrain the refinement of liquid and solid-solution interaction parameters, compound energies and a CuZr entropy term [20]. The optimization uses the analytical-gradient ES-PEI implementation introduced by Kunselmann et al. [26], through a development branch [27] distinct from the released version used for the dataset compatibility checks. Experimental formation enthalpies govern compound energies when both experimental and MLIP values are available, while calibration against experiment determines the uncertainties assigned to computed energies. The fit uses a fixed data and calibration snapshot 778cc6bfa, preserving the connection between the reported results and the evidence used to obtain them as the collection is updated.
Refinement improves agreement with both the invariant temperatures and the observations withheld from fitting (Figure 9). 9 of the 13 invariant temperatures fall within twice their uncertainty after refinement, compared with none for the starting MLIP description. The held-out activity error decreases from 5.1 to 1.7 kJ mol, while the held-out phase-equilibria error decreases from 1.7 to 0.11 kJ mol. The corresponding SGTE errors are 2.6 and 0.12 kJ mol, respectively, and SGTE recovers 12 invariant temperatures within . These results demonstrate that uncertainty-weighted observations from ORCHID can bring an MLIP-derived description into substantially closer agreement with experimental thermodynamics.
The evaluation distinguishes agreement with fitted evidence from predictions of withheld observations. The weighted recall of 0.76 at the assessed field points measures agreement within the fitting set, while the activity and phase-equilibrium holdouts evaluate observations excluded from optimization within Cu–Zr. The final model extends the preregistered fit, which recovered 4 of the 13 invariant temperatures, through additional refinement
performed after that initial evaluation; both stages remain in the demonstration record. The remaining discrepancies identify specific targets for further improvement: the CuZr congruent melting temperature is 106 K too high, the eutectoid decomposition temperature is 16 K too low, and a narrow spurious (Zr) field persists. These results establish the improvement achieved for Cu–Zr and provide a basis for extending the evaluation to other chemical systems and phase models.
Traditional CALPHAD assessment often proceeds through staged parameter optimization, in which the assessor fixes some parameters while adjusting others, progressively releasing or revisiting parameter groups as the description develops[8]. This intervention reflects the difficulty of minimizing an error function whose dependence on model parameters is mediated by phase-equilibrium calculations. Changes in the predicted phase assemblage can complicate the optimization, making convergence sensitive to the initial parameters and the sequence in which they are adjusted. Much of the assessor’s thermodynamic intuition therefore enters through the management of the fitting procedure itself. The analytical-gradient ESPEI
implementation[26] provides derivative information to guide parameter updates, enabling the programmatic refinement used here, while ORCHID supplies the interpreted evidence and uncertainty estimates that define the fitting problem.
The complete workflow proceeds programmatically from MLIP predictions to a CALPHAD description constructed by PhaseForge and refined by PhaseForge+ using ORCHID observations. With the initial description and fitting datasets prepared, the final Cu–Zr optimization completed in approximately 27 minutes on an Apple M1 Max Mac Studio (SI 1). In our experience, conventional assessment from literature collection through final fitting can require months of expert work. Bringing evidence preparation and gradient-based optimization into the same workflow makes rapid model assembly and refinement reproducible, with the source snapshot and fitting weights preserved and observations reserved for evaluation.
This integration provides a route toward continuous thermodynamic assessment. When a source is added or corrected, its observations can be connected to the affected phases and models, checked for compatibility and incorporated into an updated fitting and evaluation problem. Decisions about model form, contradictory evidence and missing phases remain guided by scientific criteria and expert judgment, while the workflow executes the corresponding calculations. Retaining the evidence snapshot, software version, model form and fitting choices for each release makes these updates reproducible. New or reinterpreted ORCHID observations can then enter PhaseForge+, and the refined description can be evaluated against its predecessor to establish what the new evidence improves.
Reuse and prospects for materials discovery
Putting continuous refinement into practice requires access to observations together with the information needed to interpret them (Figure 10). ORCHID provides downloadable records and machine-readable metadata, complemented by a Model Context Protocol server through which programs and AI agents can query chemical systems, phase identities and associated observations. Provenance, evidence classes and uncertainty budgets accompany the returned values, allowing an agent to investigate discrepancies and select evidence for fitting or evaluation. Researchers can access the same information directly, so both forms of use draw on a common thermodynamic record. These interfaces support the objectives of FAIR and AI-ready data [28, 29], with formats and usage documented in SI 2.
Figure 10. From ground-state agreement to continuously improving phase-stability models. a, On the matched formation-enthalpy set, the residual spreads of eight MLIPs are similar to that of their Materials Project reference, limiting the ability of a 0 K energy benchmark to distinguish the models. b, One-shot PhaseForge descriptions expose different finite-temperature failures when evaluated against observed thermodynamic states. c, ORCHID makes these states reusable by retaining their chemistry, phases, conditions, uncertainty and provenance in forms that can be consumed by ESPEI and by agents. d, In the Cu–Zr demonstration, PhaseForge+ refines the prepared MLIP-derived description against this evidence in approximately 27 minutes. The refined model recovers 9 of 13 invariant temperatures within 2σ, and its held-out activity and phase-equilibrium errors approach those of the SGTE assessment; the final fit is post hoc and its assessed-field score is in sample. e, The prospective loop extends this connection across chemistry by combining agents and expert judgment with new evidence, finite-temperature benchmarks, refinement and versioned releases. Panels a–d summarize results demonstrated here; panel e describes the path forward, for which cross-system performance remains to be established.
Provenance also guides the design of machine-learning evaluations because thermodynamic observations can share experimental methods, samples or an underlying measurement. These dependencies extend across papers when later authors reproduce previously reported values. The released paper and system folds support tests on sources or chemical systems absent from training, while grouping by the original experimental work and canonical evidence identity provides stricter control of shared observations. Random record-level splits can distribute related observations across training and test sets, giving an apparent performance that does not establish transfer to unseen literature or chemistry. Preserving these relationships allows researchers to construct evaluations that match the intended use of the model. In addition to supporting the assessment of individual systems, the binary records in ORCHID can be used to train models that estimate the thermochemistry of compounds in systems that have not yet been assessed (Figure 11). To illustrate this, we trained gradient-boosting and random-forest models on the formation enthalpies and entropies at 298.15 K of 2,533 intermetallic compounds in 636 binary systems, using elemental properties and the Miedema chemical enthalpy as descriptors. For the compounds included in the training set, the predictions of these models agree closely with the CALPHAD values of the enthalpy and appear to describe the entropy satisfactorily. In their intended application, however, these models are applied to systems that are not part of the training set, and when entire binary systems are withheld from training the behaviour of the two properties becomes quite different. The enthalpy model retains much of its accuracy, with a mean absolute error of 6.8 kJ per mole of atoms that improves on the 9.9 kJ per mole of atoms of the Miedema estimate, whereas the error of the entropy model is only about 10% below that of a prediction equal to the mean of the training data. This behaviour of the entropy model is consistent with the way in which formation entropies are obtained in many assessments, where they are adjusted to reproduce the phase boundaries and are not constrained by thermal measurements. The estimation of formation entropies for ternary and higher-order compounds will therefore require information beyond that contained in the binary assessments.
Figure 11. Binary compound thermochemistry as training data. Formation enthalpy (; a, b) and formation entropy (; c, d) at 298.15 K relative to SER, predicted by models trained on 2,533 () and 2,490 () intermetallic compounds from 636 binary systems in ORCHID-Hulls and compared with the CALPHAD values. a, c, Predictions for the compounds used in training. b, d, Out-of-fold predictions with entire binary systems withheld (five folds grouped by system); the metrics are means over three random fold assignments, and the mean absolute error (MAE) is in the units of each row. Shading shows the density of points on a logarithmic scale, the dashed line marks 1:1 agreement and the grey band ±10 kJ per mole of atoms. Entropy labels fixed by the Neumann–Kopp convention are excluded, and 18 entropy values outside the plotted window are drawn at its edge.
The predominantly binary coverage of ORCHID provides a foundation for extending the collection to higher-order systems. Its 1,234 binaries with calculated diagrams define approximately 9,700 ternary combinations for which all three constituent binaries have assessments. Including literature and handbook coverage increases the number of ternary combinations supported by their constituent binaries to approximately 34,000. Developing thermodynamic descriptions of these systems also requires evidence for genuinely ternary phases and the corresponding model parameters [30, 31]. Once binary and ternary descriptions are established, the lower frequency of genuinely higher-order phases may make further extrapolation more tractable [32]. Extending coverage to higher-order observations, measurements available only in plots and kinetic properties would broaden the range of scientific questions the resource can address.
Self-driving materials discovery requires connecting proposed structures to the phase assemblages that can be obtained and retained under synthesis and processing conditions. Finite-temperature thermodynamics establishes the competition among these phases, while kinetics and processing history determine the pathways through which stable or metastable structures can be reached. ORCHID provides the thermodynamic evidence needed to develop this connection and a foundation for linking it to kinetic and processing data within ICME workflows. By preserving the observations and interpretations underlying model predictions, it allows successive assessments to build on accumulated knowledge and supports the use of that knowledge in automated decisions about materials synthesis and processing.
Conclusions
ORCHID organizes thermodynamic evidence dispersed across the literature around the states described by the observations. Thermodynamic constraints guide its recovery and interpretation, while source verification, expert review and controlled evaluations establish the basis for its use. Retaining phase identities, provenance, evidence classes and uncertainty allows observations from different sources to be brought together without losing their individual meaning. The ESPEI-compatible datasets and programmatic interfaces connect this evidence to CALPHAD assessment, making the same thermodynamic information available to researchers and AI agents.
The computational comparisons demonstrate why finite-temperature phase stability provides an essential complement to formation-energy benchmarks. Eight MLIPs exhibit strongly correlated formation-enthalpy errors with spreads close to their DFT reference, yet their PhaseForge descriptions reveal distinct errors in the competition among phases in Cu–Zr. Using ORCHID evidence, refinement through PhaseForge+ recovers 9 of 13 invariant temperatures within 2σ and reduces the held-out activity error from 5.1 to 1.7 kJ mol. The final gradient-based refinement completed in approximately 27 minutes with the initial description and fitting datasets prepared. This demonstration connects literature-derived evidence to rapid thermodynamic assessment and provides a reproducible procedure for extending the evaluation across chemical systems.
Connecting reusable thermodynamic evidence to programmatic model refinement opens a path toward continuously updated CALPHAD descriptions. New measurements, corrected interpretations and improved computational predictions can enter successive assessments while preserving the evidence and modeling choices behind each revision. ORCHID provides the foundation for this accumulation of knowledge, allowing the work invested in recovering and interpreting a source to support many subsequent assessments. This capability brings thermodynamic model development closer to the pace of computational materials discovery and strengthens the connection between predicted materials, their phase stability and the conditions needed for synthesis and processing.
Data availability
The datasets are released under CC BY 4.0 and described in SI 2, with the copyright status of quoted source passages retained separately. Versioned data are identified through the umbrella record (10.5281/zenodo.23736065) and the individual dataset records listed in SI 2. The data site is https://data.orchid-project.org, and repositories are under https://github.com/orchid-calphad. The analyses and demonstrations retain their data snapshots so that later corrections do not silently change reported benchmark or fitting results.
Code availability
The code is MIT-licensed under https://github.com/orchid-calphad. Extraction, data construction, agent access, synthetic benchmarks and the runnable scientific examples are documented in SI 2. SI 1 records the versions and development branches used for the calculations, including the gradient-capable ESPEI branch used for refinement.
Author contributions
R.A., J.S. and D.S. conceived ORCHID and have worked jointly on all aspects of the project. All authors reviewed and approved the manuscript.
Use of AI tools. AI tools were used extensively in this work, as described in the Methods: language models read the papers and the diagrams under the constraints and checks described there, and coding assistants (Claude Code and Codex) wrote and tested much of the software and drafted text that the authors edited. The authors take responsibility for all content, and the numbers in this paper are generated from the released files by scripts and were checked in the audits described above.
Conflicts of interest
There are no conflicts to declare.
Acknowledgements
This work was supported by the U.S. National Science Foundation under awards OAC-2608825 (CSSI Elements: PhaseForge) and DMR-2534344 (Autonomous Robotic Metallurgist: An NSF Materials Innovation Platform, ARM-MIP). We thank B. Hallstedt and the SGTE for releasing the binary collection under an open licence, and the Materials Project and OQMD teams for their open data. We thank R. Otis and B. Bocklund for pycalphad and ESPEI, on which the calculated diagrams and the ESPEI datasets of ORCHID are built. We also thank C. Kunselman, whose gradient-based optimization for ESPEI (pull request #268, with contributions from B. Bocklund) makes likelihood gradients nearly as inexpensive as the likelihood itself and thereby allows assessments to be refined at a scale that MCMC sampling alone does not reach.
Appendix: ORCHID data products and access
The eight ORCHID datasets serve different scientific roles, but their connection is established by the thermodynamic state rather than by the folders in which they are distributed. As summarized in Figure 12, Values and Assemblages preserve reported observations, while Diagrams, Hulls and Parity provide calculated references for their interpretation. Registry and Bibliography connect these observations to phase identities and sources, and Bench supplies documents with known answers for evaluating extraction.
Figure 12. Eight datasets connected through a common thermodynamic language. Each tile represents one released ORCHID data product and includes a visual summary of its scientific object. These products meet in an explicit thermodynamic state that retains the chemical system, conditions, phase constitution, observation, evidence class, uncertainty and source. Shared phase identities and provenance allow the same state to enter CALPHAD fitting, finite-temperature MLIP benchmarking, machine learning and joint analysis by agents and experts. Counts describe values, calculated diagrams, Materials Project systems, parity pairs, phase identities, assemblage points, papers or synthetic benchmark documents, as indicated in the corresponding tile.
Across these products, record, phase and source identifiers connect observations to their chemistry, conditions, phase constitution, evidence class, uncertainty and provenance. The present release establishes these connections through the recorded state attributes; permanent state identifiers issued by an append-only registry are planned for a subsequent release. This representation supports inspection against the source, export of eligible observations to ESPEI and evaluation of finite-temperature predictions. The file-level inventory, schemas, licences, release locations and programmatic access routes are given in SI 2.
What the datasets contain. The units counted in these products reflect different scientific objects. A literature value is an observation, a parity pair links an observation to a calculated entry, and a diagram or hull describes a chemical system under specified modeling assumptions. Their counts therefore cannot be added to obtain a number of independent measurements.
| Dataset | Scientific object | Principal use |
| Values | A reported property value with its state, source and evidence class | Select observations for assessment; export the eligible subset to ESPEI |
| Assemblages | A phase assemblage at a temperature and composition in an assessed diagram | Test finite-temperature phase predictions or constrain a fit |
| Diagrams | A phase diagram calculated from an identified TDB assessment | Compare reported equilibria with model predictions |
| Hulls | A convex hull and its entries for a chemical system and reference convention | Examine competing phases and calculated energetic stability |
| Parity | A literature formation enthalpy paired with a matched DFT entry | Compare computed and reported energies; calibrate DFT uncertainty |
| Registry | A phase identity linked to reported names and TDB models | Resolve names across sources and map observations to models |
| Bibliography | A source work and typed relations among works | Trace provenance and recognize shared or reprinted evidence |
| Bench | A synthetic document with known numerical and phase information | Evaluate extraction against an answer key |
Table 1.
Following an observation into model evaluation. The archived Cu–Zr demonstration provides a concrete example. At 1573 K and , the activity record attributed to Zaitsev (2003) reports and retains the observation identifier ORC-5OZGTSNZVA.3a2b7e. In the ESPEI activity dataset, the component, temperature and composition are accompanied by the pure-liquid reference state. This observation belongs to the held-out set shown in Figure 9c,Figure 9d, where the initial and refined models predict activities of approximately 0.702 and 0.758, respectively. No printed measurement uncertainty is available for this observation, and none is inferred from its agreement with either model. The retained source and observation identifier connect the numerical comparison to the evidence, while the held-out assignment distinguishes its use in evaluation from the observations used to determine the model parameters. The archived inputs and uncertainty treatment for the demonstration are documented in SI 1.
Choosing evidence for a task. For CALPHAD fitting, begin with the ESPEI-compatible export of Values, check the phase mapping to the intended TDB model and select evidence classes and uncertainty weights appropriate to the assessment. Assessed-diagram constraints from Assemblages can be included as a distinct evidence class. For MLIP evaluation, use matched formation-enthalpy observations from Parity and finite-temperature observations from Values or Assemblages, reserving evaluation evidence before fitting. For extraction-method evaluation, use Bench and its known answers. SI 1 gives the validation and calibration procedures, and SI 2 describes retrieval and evidence selection, including source- and system-based splits for tests of transfer to new literature or chemistry.
Scope and organization
SI 1 documents the extraction, uncertainty calibration and validation underlying the main paper. Supplementary Notes 1–4 retain the extraction history, annotation procedures, audit findings and calculation details. The original Supplementary Figs. S1–S7 and Supplementary Tables S1–S8 are followed by the detailed Methods and Validation sections, their additional figures and tables, the matched MLIP benchmark and the full construction and refinement figure notes. SI 2 describes the repository organization and access conventions.
Counts use the shared manuscript snapshot in numbers_DD.tex, derived from numbers.tex generated by scripts/make_numbers.py from the released files. The exceptions are the window-rule counts in Supplementary Table S3 and the reader-pilot and sign-check counts in Supplementary Note 1, which come from reports of the extraction record store that are not released, and the recovery-trial counts in Supplementary Note 4, which come from a separate calculation audit. The scientific figures are retained from those generated by scripts/make_figures.py, with style definitions in scripts/style.py. The provenance and release status of these supporting records are stated where the results are discussed.
Table S3: Quality gates. Top: the gates that a literature value passes before release, with what each checks and its effect on the values, in the order applied. Bottom: the window rule, which keeps a field read by the first reader only when it is printed in the quote or in an allowed window around it; its counts are from the release fix-up of 22 October 2026 and are recorded in the extraction store, not in a released file.
| Gate | What it checks | Effect | Values |
| reader agreement (tables) | two or more independent readings of a row agree; disagreements go to a judge model or stay not accepted | accepted values | 66,551 |
| quote verification (text) | the reader’s sentence is found verbatim on one page and contains the printed number | every running-text value passes | 5,743 |
| window rule | every published field is printed in its quote or an allowed window | fields outside removed; values flagged | 2,781 |
| page-image sign check | suspect formation-enthalpy signs read on the page image | sign verified / flagged unsigned | 11 / 42 |
| clean-up (Version 1) | duplicates, vocabularies, per-formula units, garbled fields | duplicates removed | 3,550 |
| per-formula values converted (basis read on the page) | 26 | ||
| DFT outliers flagged, unchanged | 42 | ||
| literature-level gates (G1–G12, page checks) | temperatures, units, signs, systems, phase lists, property semantics, evidence class, compositions, duplicates, pressure; changes of unit, sign, scale or system confirmed on the page (Supplementary Note 3) | records corrected | 15,528 |
| records demoted (not accepted) | 9,378 | ||
| further duplicates merged | 3,998 | ||
| states without phases (R3) | an invariant, transition, phase-composition or formation value needs the phases of its state | phases recovered from the quote or page / missing-phase findings at R3 | 280 / 8,944 |
| complete invariant reactions (R3) | phases, class and phase count against the printed reaction; same-page duplicates | phases added / classes filled | 219 / 3,022 |
| flagged: class conflict / phase count / duplicate / temperature | 67 / 303 / 179 / 5 | ||
| unit tokens (R3) | unit prefix of a caption (“k J”); formation values below 0.1 kJ (mol of atoms)⁻¹ | values restored / small values flagged | 11 / 3 |
| system assignment (R3) | system bounded by the printed formulas; composition of a compound from its formula | compositions recomputed / flagged | 419 / 74 |
| ESPEI publication gate | value published, accepted, ESPEI-ready, of the folder’s class and equal to the published numbers | values cut, listed with the reason | validation.json |
| ESPEI checks | check_dataset and load_datasets of ESPEI 0.9.1; likelihood finite against the named TDB | datasets excluded whole | 24 + 387 |
| Field | Accepted windows | Kept | Dropped | Total |
| transition kind | W0 | 4 | 1,327 | 1,331 |
| citation | W0, W1-clean | 383 | 642 | 1,025 |
| phase | W0, W1 | 346 | 396 | 742 |
| method | W0, W1 | 274 | 351 | 625 |
| reaction | W0, W1 | 83 | 102 | 185 |
| temperature | W0 | 0 | 27 | 27 |
Table S3.
Supplementary Note 1: reading the values
This note documents the empty-answer failure detected in one reader, the comparison of the two text readers and the verification of signs against printed pages.
Empty reads under one prompt version. Reader agreement can support acceptance of a value only if both readers return data. Under one version of the extraction prompt, Gemini returned a single placeholder row for tables that Claude transcribed in full. The synthetic benchmark first exposed this failure, and an audit of the production records identified 98 affected tables in 79 papers. Because the sanity filter counted rows rather than values, it had treated these empty reads as votes.
To correct this failure, two subsequent pipeline releases stopped sending the affected prompt to Gemini and changed the filter to count values. Under this revised filter, an answer without values was treated as a failed read requiring a tie-break. All table rows supported by only one visual reader were also sent to an independent second reader using the same crop, prompt and schema. To document the outcome of these second readings, the added field reader_agreement records the resulting support in release v2026.10, with multi_reader for 69,253 values, single_reader for 1,342 and single_reader_text_layer for 2,547. The 400 values in unsettled invariant-table rows have no agreement classification and none is accepted. The affected prompt was thermo-table-v7.1, and the corrections were introduced in pipeline releases v2.1.19q and v2.1.19r. The record-store audit is documented in build/sgte-corpus/audit/v71-empty-rows-20261022/REPORT.md, which is not a released file.
Reader comparison pilot. To evaluate replacing Gemini with Claude as the second text reader, we compared the readers on the same 127 requests. On 94 passages of running text, they agreed in 89 cases on whether the passage contained data. For these passages, Claude returned 553 rows and Gemini 388, with 491 distinct values found by both, 280 found only by Claude and 28 found only by Gemini. About 17 of the values found only by Gemini were errors.
Verification of the returned quotes provided a further comparison of the readers. The quotes were verbatim in 546 of Claude’s 553 rows and 306 of Gemini’s 388, with failures arising mostly from control characters in the PDF text layer. In 32 passages, these characters also led to nine sign errors for Gemini and none for Claude. Gemini, however, itemized samples somewhat better in passages describing experimental conditions. The pilot compares the readers with one another and does not establish an independent human precision estimate for numerical extraction. The comparison used Claude Opus and Gemini Flash on 22 October 2026 and is documented in claude-reads/pilot-20261022/PILOT_RESULTS.md, which is not a released file.
Printed-page sign checks. Loss of a minus sign in the PDF text layer changes the interpretation of a thermochemical value. To check for this error, candidates identified by either of two independent criteria were examined individually on page images. The set contained 144 candidates from 76 papers, with 67 identified by the first criterion, 122 by the second and 45 by both. Of these candidates, 39 had a printed minus sign that the text layer had lost, either by dropping the glyph (22 cases), splitting it across a line break (9), or reading it as the digit 2 (8). The other 105 were printed without a sign, mostly as magnitudes under the convention of the paper.
A sign verified on the page is recorded with flag_sign_pdf_page_verified. Values printed without a sign are retained as printed and flagged unsigned_in_print. In release v2026.10, 11 values have a page-verified sign, including 10 set by the literature-level gates in Supplementary Note 3, and 42 are flagged as unsigned in print. The source is the sign-check record sign_check.json, which is not a released file.
Supplementary Note 2: annotating assessed diagrams
Most fields of a handbook diagram can be named using the rules of phase diagrams alone. For the fields left unresolved by these rules, pairs of vision models select among the permitted assemblages. For these unresolved fields, disagreement between readers leads to successive readings by Gemini 3.1 Pro with Claude Haiku 5.5, two Claude Opus 5.5 readers, and then Claude Fable 5.1 with GPT-6 Astra. Agreement at this last stage is retained only as a pro posal. The rules then propagate names into thin fields, after which an adjudicator (Claude Opus 5.5) examines the whole diagram, its phase table and all reader answers. Based on this information, the adjudicator names the field, identifies a tracing fault or an undetermined region, or passes the field on, recording a reason for the decision.
For low-temperature fields below the range of the drawing, the adjudicator consulted Materials Project and OQMD hulls to identify the next stable phase. Names obtained in this way carry the basis dft_hull_corroborated. Their confidence is high when the two hulls agree and low when they disagree.
These names were evaluated through expert verification, an agent audit and a separate blind expert sample. In the verification step, a domain expert reviewed the 126 hardest fields with the adjudicator’s reason shown and accepted 92 of the 93 named proposals (a median of 41 s per field over the first 20 fields). The accepted proposals included 19 of the 22 fields named with the help of the hulls and all 13 for which the two hulls agreed.
To examine agreement across naming routes, a blind stratified audit of 185 fields used independent adjudicator instances that saw the proposed name but not its basis. This agent audit gave name precisions of 0.87–1.00 across the naming routes and about 0.96 when weighted by the number of fields. The independent expert sample used a different form of blinding, in which the expert named each of 48 fields without seeing the machine’s answer. This sample gave 45 agreements (0.83–0.98), increasing to 47 when two conventions of notation were treated as equivalent (0.89–1.00). These evaluations distinguish verification of a proposed answer from agreement with an independently assigned phase assemblage.
(Sources: open-tdb/docs/reports/HANDBOOK_PHASE_DIAGRAM_PILOT.md, HUMAN_ASSISTED_ANNOTATION_STUDY.md, HULL_HEURISTIC_UNCERTAIN_LOW_T_FIELDS.md; blind sample tier5/blind-sample/score.json.)
Supplementary Note 3: literature-level gates, page checks and audits
The literature-level gates described in § 20 reduced the 77,540 records remaining after clean-up to 73,542 records, of which 65,284 are accepted. Supplementary Table S8 records the actions of the individual gates. To verify these gate actions, an AI agent confirmed changes of unit, sign, scale or chemical system on the rendered PDF page. The ledger of 3,262 checks retains the page, image hash, verdict and a short note of what is printed, without retaining the image or page text. A check of a whole table applies to its rows, giving 3,303 lookup keys for the gates.
After these page checks, adversarial audits searched the release, code and manuscript for classes of error. Their native error rates and the post-correction re-audit rates are reported in the main text (§ 3.2). To retain the findings of these audits, the release change log and test suite record the classes found, corrections applied and permanent tests introduced to guard against recurrence. Among the 15 values sampled from the empty-phase demonstrations of the third audit round, 12 had their phases printed in prose that the deterministic rule could not associate with the value. A later page-checked recovery restored the printed phases of 1,267 records that had been demoted by the placeholder rule; these records were accepted again, although they remain excluded from ESPEI output. Among the 1,687 placeholder records left unchanged by that recovery, a blind sample found a printed phase for 3 of 60 records (5.0%, Wilson 95% interval 1.7–13.7%). The records that still lack a real phase retain the flag state_phases_missing, and none is accepted.
In the final build of the third audit round (24 October 2026), the re-sample of records changed by that round found no wrong value among the split-row compositions (0 of 15) or the phases recovered from quotes (0 of 20), one among the parity pairs (1 of 30), and residuals that predate the round in three other pools (1 of 25, 3 of 25 and 2 of 25). The later corrections are listed in the change log of the release.
(Sources: tools/literature_gates.py; literature/QUALITY_FIXES.md and quality_fixes.json, which list every rule and count; ledger tools/literature_visual_checks.json.)
Supplementary Note 4: details of the methods, comparisons and uses
One-shot phase-diagram score. The one-shot score asks how much of the recorded phase equilibrium a description recovers when it is used without fitting to the benchmark evidence, and it treats each equilibrium recorded by ORCHID as a small classification problem in which every phase is either present or absent.
Each evidence point therefore has a reference assemblage , the phases recorded at its temperature and composition, and a predicted assemblage , the phases with non-zero amount in the pycalphad equilibrium of the description at the same conditions ( Pa), with the names of both mapped to registry identities so that order-disorder partners and composition sets are compared correctly.
For each phase , , , , recall and precision , and the prevalence-weighted recall is then the fraction of the recorded occurrences of phases that the description recovers, so that a phase seen at many points counts in proportion to the evidence for it.
Recall alone does not penalize a phase that is predicted where it is not observed, as CuZr is at many points of the Cu–Zr description from GRACE, and this is the reason that precision and F1, with the same weights, are reported with it.
Points placed inside fields of the assessed diagram are scored at their coordinates; points that lie on a boundary (tie-line ends and invariant vertices from the literature) are scored as the union of phases predicted within ±10 K and ±0.01 in mole fraction and reported separately. Confidence intervals resample whole equilibria or whole source studies. Phases recorded by the handbook but absent from every model’s phase list are reported, and a score restricted to representative points is given alongside.
Machine-learning split. Values reported in a paper share authors, samples and methods, which can make a test on that paper appear more accurate if some of its values were used for training. To avoid this overlap, the split column of the Parquet tables assigns all values of a paper to the same set. The assignment uses the first 8 bytes of sha256("orchid-split-v1:" + paper_id), read as an integer and taken modulo 100. The resulting integers below 80 enter training, those below 90 enter validation, and the remainder enter the test set. This deterministic rule requires no stored list of assignments.
To complement this fixed split with cross-validation, fold_lpo leaves out whole papers, while fold_lso leaves out whole chemical systems. The corresponding sets of five folds are obtained from the SHA-256 of the paper identifier or chemical system, modulo 5. The latter grouping tests prediction for systems absent from the training data.
Clean-up of Version 1. The first release of the literature values (Version 1) contained four kinds of defect, addressed in order of increasing difficulty. Duplicate records were identified by agreement in paper, page, property, value, unit, temperature, phases, quote and all fields distinguishing one state from another. Merging 3,330 groups removed 3,550 records, while retaining the original identifiers in merged_ids. After duplicate removal, reaction and transition labels were mapped to controlled vocabularies. This mapping reduced 187 printed variants in 21,703 records to 16 terms, relabelling 15,971 records while preserving their printed labels. Malformed property codes were treated similarly in 528 records.
After correcting the labels, the DFT comparison identified candidates for the next class of correction, involving the basis or sign of a value. For this comparison, we examined 74 accepted experimental formation enthalpies that had a Materials Project or OQMD match and no flag of suspect magnitude. These candidates differed from every matched DFT value by more than 50 kJ , or had the opposite sign of a DFT value with kJ . A per-atom value was converted from a per-formula-unit basis only when the printed page established the column basis, giving 26 corrections documented by page checks. For these basis conversions, an earlier rule had accepted changes when they improved agreement with DFT. This rule was withdrawn because it used DFT to correct the experimental values subsequently used to calibrate DFT. Unsigned magnitudes were flagged (6), and the remaining 42 values were retained unchanged with dft_outlier_unexplained. Of these, 25 remain in the default subset, while the others are excluded by its point rules.
After the basis and sign checks, the final corrections addressed garbled fields. The 2 entries that did not represent valid page numbers were set to null and read again on the rendered page. In addition to these page-number corrections, one free-text reference state unsupported by the quote was removed. The evidence for these and all other changes is retained in literature/QUALITY_FIXES.md and quality_fixes.json.
Assessed diagrams: annotation layers and points. To extract assemblages from an assessed diagram, the fields must first be identified and named. To identify these fields, a deterministic pass reads the vector drawing, reduces outlines to centrelines and calibrates both axes using the tick marks and printed labels. It then finds the enclosed fields and assigns printed phase labels to the fields containing them. The rules of phase diagrams allow further assignments because adjacent fields differ by exactly one phase, except across an invariant line or the line of a compound, and a phase of a two-phase field vanishes into a neighbouring field. Using these constraints, the solver accepts a name only if a single assemblage is consistent with the whole diagram. For fields unresolved by these constraints, vision-model readers and an adjudicator provide assignments, followed by expert review as described in Supplementary Note 2.
Once the fields have been named, a point deep inside a field provides the assemblage supported by the drawing without relying on the precise position of a drawn boundary. The area centroid is used when it lies inside the field at a safe distance from all drawn lines. Otherwise, the point is placed farthest from all field boundaries, at the pole of inaccessibility, i.e. the centre of the largest inscribed circle. This alternative is needed because the centroid of a curved or crescent-shaped field can lie outside the field or on a boundary. Fields too thin to satisfy the margin are listed as exclusions and provide no point.
Dashed boundaries represent estimates by the evaluator, but they are traced and used as field boundaries in the same way as solid lines. To retain this distinction, a point records whether the boundaries along its isotherm and isopleth are solid, dashed or both (20,782, 1,028 and 1,292 points, in 1,021 diagrams with dashed lines). This boundary information is used to enlarge the reading uncertainty when a bounding line is dashed. The annotation figure in the main text illustrates these steps using a synthetic diagram from Bench, without reproducing a handbook drawing.
For the points placed in these fields, the record contains temperature, mole fraction, pressure (1 atm, assumed) and assemblage, with phase-registry identities where the mapping is unambiguous. To document the assignment, it also gives the naming basis (deterministic, reader consensus, adjudicator, DFT-hull corroborated, expert reviewed), confidence and review state.
The uncertainty budget of a point combines the limitations of reading the drawing and of the handbook method. The reading component is propagated from axis calibration as the maximum calibration residual plus half a raster cell. The method component uses a prior of 10 K and 0.02 in mole fraction, enlarged for dashed bounding lines. Combining the reading and method contributions in this way, the larger component is used as the effective uncertainty. A two-phase point may additionally contain a tie_line block obtained by intersecting its isotherm with the drawn boundaries. Because these endpoints are based on the drawing alone, they are marked as a weak constraint. This tie-line block is omitted, with a recorded reason, when the isotherm does not meet two clean boundaries.
To use these point records in fitting, diagrams whose phases can be named in a TDB file also produce 1,068 ES-PEI ZPF datasets containing 9,877 points. The 13,252 points without such names are excluded. For these datasets, phase compositions are left unknown by default, using null vertices and a hyperplane vertex at the overall composition of the point. An alternative to these unknown compositions supplies the tie-line endpoints. The points are also compared with the equilibria of the SGTE assessments at the nearest grid cell, but the comparisons never alter the records.
In distributing the point records and their comparisons, the released content is restricted to derived phase names, assemblages, presence facts, coordinates, phase metadata and citations. The builder enforces this restriction by failing on output resembling page images, crops or handbook text. The records cite the original evaluation first and the handbook second, identifying the latter as the compilation.
Extraction benchmark. To test reading against exact truth, 20 synthetic documents containing 86 pages were generated from SGTE TDB files using pycalphad 0.11.2, one document per binary system (Bench). The phase diagram in a document is drawn using a seeded choice of line width, colour, labelled or unlabelled fields, K or °C, at.% or mole fraction, insets, experimental-style markers and a shifted second assessment. The document also contains phase-fraction and property plots, tables of invariant reactions, solid phases, formation properties and liquid mixing properties, and running text stating values in varied phrasing. Layouts and units are varied across the tables.
To test the effect of document quality, the documents are rendered as born-digital vector PDFs, 300-dpi rasters, degraded 150-dpi scans and image-only PDFs. Scan degradation includes contrast loss, blur, small rotations, noise and JPEG compression. Before testing extraction, an oracle wrote the known truth in the extractor format to validate the scorer. For this oracle, all 1,528 rate metrics (precision, recall, F1, accuracy, exact match) equal 1.0.
Supplementary Table S6 compares deterministic extraction routes with the production AI reader, Gemini 3.8 Flash using the production prompts as a single reader. On clean renderings, the AI reader recovers almost all invariant-table rows, with lower recall on degraded renderings. Its running-text values are precise, but recall is limited by the paragraph filter of the production pipeline. In contrast, Tesseract drops the reaction arrow and therefore never recovers invariant rows, while automatic axis detection fails on many degraded diagrams.
This benchmark exposed the empty reads discussed in Supplementary Note 1. In that run, the AI reader returned placeholder rows, giving scores near zero for property records and structure labels. In contrast to these placeholder outputs, Claude recovered, on the same requests, all property records from vector PDFs (1.00) and 0.86 from degraded scans. Together, these benchmark runs involved 525 calls at a list-price cost of US$88.46. Supplementary Figure S5 reports the results across tasks, metrics and renderings.
Sign checks and the DFT comparison. Values that are printed without a sign could bias the DFT comparison if they were taken as positive, but after the clean-up only 42 of them remain, and including them changes the Materials Project MAE of the experimental set from 9.6
to 9.6 kJ (mol of atoms), while the Pearson coefficient stays at 0.94, so that the choice of including or excluding them does not affect the comparison.
DFT parity in detail. The experimental formation enthalpies are individual extractions from the papers of the corpus and not values taken from a compilation, and values reprinted from earlier work are linked to their original source where it could be resolved. The DFT parity compares each experimental formation enthalpy in the default subset with the structure-matched entries of the Materials Project and OQMD, and the mean absolute error (MAE) is 9.6 kJ (mol of atoms) for the Materials Project (1,070 pairs) and 8.3 kJ (mol of atoms) for OQMD (1,107 pairs). The point rules left out, for the Materials Project and OQMD respectively, 45 and 50 pairs whose reference state is not the solid standard reference, 115 and 118 whose literature value has a gate failure, 7 and 12 measured away from the stoichiometry of the compound, 3 and 3 whose composition conflicts with the label, and 1 and 10 whose named structure is absent from the database.
The OQMD value corresponds to 0.086 eV atom over pairs that are mostly intermetallic (0.079 eV atom for the intermetallics alone), which may be compared with the 0.069 eV atom that Kiirklin et al. reported for 278 binary intermetallics and the 0.096 eV atom for 1,670 compounds dominated by oxides and halides, obtained with the same fitted elemental references1. The closeness of the overall values is therefore a coincidence of populations, and some of our measurements are among those to which those references were fitted.
When the median of the values of a compound is taken first, the MAEs become 9.8 (MP) and 8.3 kJ (mol of atoms) (OQMD). By chemistry, intermetallics agree better (MAE 8.3 and 7.6) than borides, carbides, nitrides and chalcogenides (14.9 and 11.5). By method, calorimetric values agree better than EMF and vapour-pressure values (MP 9.0 versus 12.3, OQMD 7.6 versus 9.6). Adding the model values to the experimental set (3,106 MP and 3,352 OQMD pairs) gives MAEs of 8.5 and 7.9 kJ (mol of atoms).
Behind these averages lie the 74 formation enthalpies with large residuals, which the clean-up described above sorted into three classes, namely 26 per-formula values that had been labelled per atom (such as CuO, CuO, CaO and LiO, which are printed in kJ mol of compound), 6 magnitudes printed without a sign, and 42 values with no deterministic explanation, which are kept and flagged (25 of them are in the default subset, and the others are excluded by its point rules). They include printed values that disagree with both DFT sources (e.g. Ca–Si compounds from one effusion study, with residuals above 150 kJ (mol of atoms)), compounds for which the two DFT sources disagree with each other (e.g. ReSi), and values whose sign is printed opposite to that of DFT (e.g. AlBi). Removing these values would lower the MAE to 9.2 (MP) and 7.9 kJ (mol of atoms) (OQMD), but they are kept because no rule justifies a change. DFT calibration. As for the experimental values, DFT uncertainty is described by a robust , here obtained from residuals against experiment. For the Materials Project, the resulting is smallest for magnetic 3d intermetallics and largest for compounds containing p-block or anion elements. By comparison, the OQMD strata differ less and do not follow the same ordering (Supplementary Table S5). This difference between the two databases reflects their different treatment of energies, through the MP2020 corrections for anions and for the mixing of GGA and GGA+U calculations in the Materials Project and through elemental reference energies fitted to experimental formation enthalpies in OQMD, and part of the present experimental set is among the data to which the OQMD references were fitted. The literature DFT values have a lower overall (7.2 kJ (mol of atoms), compared with 8.6 and 7.5 kJ (mol of atoms)), but within each chemistry stratum they are not systematically closer to experiment (Supplementary Table S5), so that their lower reflects mainly a larger share of magnetic 3d intermetallics and a smaller share of p-block and anion compounds rather than more accurate calculations.
The residual distributions have heavy tails, with RMSE values 1.6 and 1.7 times the robust for the Materials Project and OQMD, respectively. A Gaussian likelihood based on this assigns too little probability to large residuals, allowing outliers to influence a fit too strongly. To limit this influence, a wider or a heavy-tailed likelihood may therefore be preferable when fitting DFT data. The estimate treats pairs as individual observations, while the confidence interval resamples whole chemical systems. It also inherits errors in the machine-extracted experimental values used as the reference.
Cross-check with Kim et al. The compilation of experimental formation enthalpies by Kim et al. includes intermetallic and inorganic compounds [22]. Because several ORCHID values reproduce the same calorimetric measurements, agreement with this compilation primarily tests extraction, units and signs, rather than the measurements themselves. There are 1,436 accepted, signed experimental formation enthalpies in kJ (mol of atoms) for 606 formulas available for comparison. The unit conversion requires care because the file of Kim et al. does not state the unit, but the magnitudes identify the values as eV/atom. After conversion, compounds were matched by reduced formula, using the median when either source contained multiple values.
The sources overlap in 160 compounds, with a mean absolute difference of 3.8 and a median of 1.2 kJ (mol of atoms). All 160 compounds agree in sign. The 19 compounds differing by more than 10 kJ (mol of atoms), including GeHf, TaB and AlIr, are listed with their record identifiers in kim2017_crosscheck.json for review.
Invariant temperatures against the SGTE assessments. Literature invariant temperatures can also be compared with the calculated invariants in Diagrams using only released files, as shown in the invariant figure of the main text. Because the assessments were fitted to many of these measurements, this comparison mainly tests extraction and phase identification. For this comparison, a match requires agreement in chemical system, reaction class and the set of three phase-registry identities. The reaction class distinguishes a eutectic, in which the liquid lies between the other phases, from a peritectic, in which it lies at one end, and from reactions without liquid. Phase identities are obtained from the paper and TDB names in Registry. When this procedure matches several assessments, their median invariant temperature is used to avoid favouring an individual assessment.
Of 10,207 accepted binary eutectic, peritectic, eutectoid and peritectoid temperatures, 8,770 map to three identities, and 3,616 experimental values in 401 systems match a calculated invariant. The difference between the calculated and the literature temperature has a median of 0.1 K and a robust of 5.9 K, and 73% of the values lie within 10 K (mean absolute difference 12.9 K). Model values agree more closely (3,380 values, robust 2.5 K), as is to be expected when a paper reports the assessment that the TDB file encodes, and 3,599 of the 3,616 experimental values (100%) come from a paper that is itself a source of one of the SGTE assessments of its system.
Using the ESPEI datasets. The use for which ORCHID was built is the refinement of CALPHAD models, and it supplies the data needed at each step of an ESPEI fit2. In the parameter-generation step, ESPEI fits end-member and interaction parameters to non-equilibrium thermochemical data. Such data are held in the folders experimental/, dft/mp/, dft/oqmd/ and dft/literature/, the DFT folders with a calibrated . Bayesian refinement by Markov chain Monte Carlo then fits all parameters to phase-equilibrium (ZPF), thermochemical and activity data. For this step, the folders experimental/ and invariants/ supply measured thermochemical and phase-equilibrium data, while Assemblages supplies ZPF points from assessed diagrams. Thermochemical weights follow the effective uncertainty of the observation; ZPF data retain the driving-force tolerance of the equilibrium likelihood, without converting their temperature uncertainty into a thermochemical weight.
In practice, a fit for one binary system can proceed as follows. The SGTE TDB of the system, taken from the ORCHID catalog, is used as the starting model, with a phase-models file built from its phases, for which the phase registry gives the sublattices and constituents. The ORCHID datasets of the system are then loaded with load_datasets, after which apply_tags is used to choose the evidence, for example to disable model, to use one DFT source and to down-weight single_reader. For these fits, the regenerated bundle should be loaded from a clean directory, retaining the exported weight of each record; the split manifest UNCERTAINTY_SPLITS.json identifies the parent files that must be replaced rather than loaded alongside their children. These weights already include the conversion from effective uncertainty, so any additional tag or property weight is a separate modeling choice that should be recorded with the release and ESPEI version. The parent and source-study links remain available for sensitivity analyses of shared systematic effects, although covariance between those observations is not quantified by the exported weights. The previously reported Cu–Zr fitting demonstrations retain their archived input manifests and weighting choices; their fitted parameters and scores have not been recomputed under this export revision.
At the last step, ESPEI is run with an mcmc block whose input_db is the starting TDB. This route was tested on Cu–Zr by refining six parameters of the 1994 assessment and, in a pre-registered fallback, seven parameters including a further liquid term. Although the fit to ORCHID activities improved by about half, the invariant temperatures could not be kept within the pre-registered gate. The main-text Cu–Zr example therefore used the gradient route described below, while the unsuccessful attempt is documented in orchid-examples.
The gradient route used for this refinement connects the data to the MLIP-based workflows of our group. One of these, PhaseForge, computes end-member and special-quasirandom-structure energies with machine-learning interatomic potentials in the Alloy Theoretic Automated Toolkit and builds CALPHAD descriptions from them for high-throughput phase-diagram prediction [17, 18, 19]. PhaseForge+ constructs physically informed CALPHAD models from MLIPs and tunes them against experimental observations by gradient-informed optimization [20, 21]. The experimental phase-equilibrium and thermochemical data of ORCHID are the observations that this kind of tuning needs. The Pt–W example of PhaseForge+ encodes phase-equilibrium points read from a handbook diagram as ESPEI ZPF data with one phase composition for each point, and it also uses __HYPERPLANE__ vertices, the encoding that Assemblages uses for points inside two-phase fields, whose phase compositions are left unspecified [21]. Assemblages supplies 1,068 ZPF datasets in this encoding (9,877 points), so that they can be used without conversion. To fit these data by gradients, PhaseForge+ relies on a branch of ESPEI, not yet merged, that adds analytic derivatives of the ZPF driving forces with respect to the model parameters [27]. The release is validated with ESPEI 0.9.1, whose dataset format that branch also reads, and the Cu–Zr example of the main text loaded the released datasets with the branch.
Alongside these fitting workflows, the labelled phase-stability observations, including assessed assemblages, invariants and calculated diagrams, also support training of phase-stability classifiers and Bayesian active learning of phase diagrams [33]. Beyond the ESPEI files, the DFT calibration offers a template for weighting computed (DFT or MLIP) energies against experiment. In turn, the phase registry gives the structure types, prototypes and TDB definitions that are needed to choose which phases and end-members to compute. Handbook phases that have no counterpart in the SGTE assessments of a system, which are found by the cross-check of Assemblages, are concrete targets for such calculations.
Recovery of failed diagram calculations. The recovery review began with 54 failed calculations. An initial review identified nine recovery candidates and one entry unsuitable for the binary calculation, leaving 44 cases for further investigation. Of these remaining cases, 23 underwent numerical recovery trials, while 21 required clarification of the source model or additional support in the calculation engine. Those trials examined 57 candidate variants through 888 separately solved equilibrium checks retained in the review reports, and 0 candidates were admitted at that stage because several failed denser checks or represented only fragments of a diagram.
The final review extended the checks of the initial candidates and resolved a constituent-declaration inconsistency in the Mg–Y description against the COST 507 source and database implementations. For that exact source file, the calculation reader removes an extra vacancy constituent group that conflicts with the two declared sublattices, while retaining the original file and every numerical parameter. Four descriptions, Cu–O (1992), Al–Ti (1992), Cu–Nb (1990) and Mg–Y (1998), passed 114 checks across their sampled two-phase regions and were included in the release. A subsequent audit of the entries marked successful identified two empty outputs, Ga–Se (2008) and Fe–Ti (2023), with neither mapped boundaries nor an equilibrium grid. Repeating the calculations with the original initialization and with interior starting points did not recover a usable diagram, so both entries were retained as failed calculations, with the evidence recorded in phase-diagrams/EMPTY_OUTPUT_REVIEW.json. The release therefore contains 2,722 calculated diagrams and 52 entries retained explicitly as failed calculations, with the decisions and verification records in phase-diagrams/RECOVERY_REVIEW.json. These checks test consistency between mapped boundaries and separate equilibrium solutions from the same thermodynamic engine; they do not independently validate the source assessments or every boundary point. The calculation tool rejects empty boundary outputs and provides explicit options for interior starting points and the phase exclusions specified by the source TDB, and the recovery work does not replace literature values.
Supplementary Figures
Figure S4. Each phase receives one identity that gathers the names used for it across assessments and papers. a, Structure assignment of the 10,430 phases by tier (Supplementary Table S4). b, The 20 most frequent crystalline structure types, labelled by prototype, Strukturbericht designation and Pearson symbol. c, The names collected for one identity, (): the same TDB phase name in 13 SGTE assessments of Ni–Ti, the printed label in 1 paper, and the mapping used in extraction.
Supplementary Tables
Supplementary Tables
Counts are of field instances, one per observation, before the clean-up and the literature-level gates. W0: inside the quote; W1: the preceding sentence of the same paragraph; W1-clean: W1 with no intervening citation. In the release, 2,781 values carry fields removed by this rule.
| Gate | Checks | Corrected | Demoted | Flagged | Merged |
| G1 | temperatures | 2,933 | 17 | 26 | 0 |
| G2 | energies, entropies, heat capacities, activities | 477 | 42 | 93 | 0 |
| G3 | signs | 41 | 0 | 45 | 0 |
| G4 | system assignment | 327 | 3 | 67 | 0 |
| G5 | property semantics | 392 | 189 | 42 | 0 |
| G6 | invariant phase lists | 933 | 0 | 376 | 0 |
| G7 | duplicates | 0 | 0 | 509 | 7,981 |
| G8 | evidence class | 53 | 0 | 734 | 0 |
| G9 | compositions | 4 | 0 | 269 | 0 |
| G10 | reader tier / agreement | 0 | 0 | 0 | 0 |
| G11 | CALPHAD plausibility fields | 0 | 0 | 0 | 0 |
| G12 | pressure basis | 0 | 0 | 0 | 0 |
| page | page checks (visual) | 2,663 | 88 | 1,155 | 0 |
Table S8. Literature-level gates: actions by gate. Corrected: field changed with the stored value kept; demoted: not accepted; flagged: flag or not ESPEI-ready only; merged: duplicates merged into one record. A record can receive several actions. Source: literature/quality_fixes.json (gates).
Methods
Turning the literature into a record of this kind requires answering, for every value, a sequence of questions: where it is printed, what exactly it says, which equilibrium state it describes, how it was obtained, and whether it is consistent with what is already known about the system. The first of these questions are answered while a value is being read, through the corpus from which it was taken, the way it was located on the page and the agreement required between independent readers. The remaining ones, in contrast, can only be answered by placing the value in context, against calculated phase diagrams, first-principles energies and the rules of phase equilibria, and it is in the ESPEI datasets that all of these answers finally come together.
Corpus
A corpus for CALPHAD data is most naturally built around the assessments themselves, since each assessment already identifies, through its references, the experimental work on which its model rests. The corpus used here is therefore anchored in the assessments behind the SGTE collection of binary databases [12], and it extends to the primary sources these assessments cite and to further assessments of the same systems. This extension was carried out in three steps, starting from the source paper named in the header of each SGTE binary TDB file, continuing with the works cited in these papers, and completing the set with further assessments of each system found through bounded title queries of the Crossref database. The papers themselves were read from lawfully held copies, and, since their copyright remains with the publishers, they are not redistributed, although every value points to the published version through its DOI. The values themselves were read from 1,321 papers published between 1975 and 2025. Because many of these values are reprints, each is attributed both to the paper in which it was read and to the primary source in which it first appeared, so that Values refers to 4,882 works in all. The bibliography, Bibliography, in turn, places these works within a wider map of 9,570 papers, among them 3,578 works cited as the original source of extracted values, and assessments that were identified but not yet read.
Locating values
Within each paper, values appear in two forms, in tables, where they are organized but often abbreviated, and in running text, where they are embedded in sentences that also state their context. Tables were therefore located first and classified as invariant-reaction tables or as general tables for thermochemical, phase-composition and crystallographic data, and the running text was then read passage by passage. Values shown only in figures were not read in Version 1, since recovering them requires the digitization of plots rather than the reading of printed numbers, a different task that is left to a later release. Whatever its form, every value that was read keeps a locator, namely the paper, page, table label and row, or the passage and sentence in which it is printed, so that it can always be found again on the page.
Dual-reader extraction
Finding a value on the page, however, is not the same as reading it correctly. To obtain independent readings of a table, the PDF text layer, where available, was compared with the outputs of two vision-language models reading the page image. Gemini 3.8 Flash was the first reader and Claude Opus 5.5 the second, with Gemini Pro used to confirm a reading or resolve disagreement (Figure S2 in the ESI¹). Both models received instructions encoding the rules of phase equilibria, including the number of phases required by the phase rule for an invariant reaction. Agreement between these independent readings is the basis for accepting a value, since two readers that work separately are unlikely to make the same mistake. A row was therefore accepted only when the readers agreed and the deterministic checks described below were passed, while disagreements were either settled by a judge model (198 values) or left flagged as not accepted. Whichever path a value followed is recorded in the field reader_tier (Table S9), so that a user can select values by the strength of the agreement behind them. The running text called for a stricter form of the same principle, since a value in a sentence has no row or column to anchor it. Two readers therefore read each passage, and each returned every value together with the verbatim sentence that states it and the page on which it appears, so that the claim could be checked against the printed text itself. In both tables and text, finally, the phase labels used by the authors were mapped to the phases of a CALPHAD model only when the mapping was unambiguous, and otherwise kept as printed.

Figure S2. Two independent readers agree before a value is released. a, One table row (outlined) as read independently by the PDF text layer and two vision–language models (Gemini 3.8 Flash and Claude Opus 5.5); because the three readings agree, the row is accepted and released as three records, one for each lattice constant (, , ). b, One sentence as read by two text readers, both of which return the value, its unit, the reaction and the verbatim sentence; the sentence is found verbatim on the cited page with the printed value inside it, so the value is released, as class model because the paper states that it was calculated, while the second value in the sentence (634 °C, cited from Kononenko and Golubev) becomes a separate record of class experimental. Crops are reproduced unchanged from Liao, Wang and Chen3 (CC BY 4.0).
The principle of agreement rests on an assumption that is easy to overlook, namely that each reader returns what it sees, so that an empty answer means an empty table. During the preparation of this release, this assumption failed for one reader under one version of its instructions, which returned a single empty row for tables that the other reader transcribed in full. The synthetic benchmark described in the Validation exposed this behaviour, and an audit of the production records showed that, because such empty readings had been counted as votes, a number of rows were in effect supported by a single reader. To correct this failure, empty readings were excluded from agreement and affected rows were sent to an independent second reader. The resulting field reader_agreement records the number of independent readers supporting a table value. Values supported by a single reader remain in the release, visibly flagged (Supplementary Note 1 in the ESI†, Figure S3 in the ESI†). At list prices, the language-model calls behind this release would cost about US$8,600, of which about US$4,000 went to reading the literature and the handbook diagrams, roughly 6 US cents for each accepted value; this is a lower bound and a counterfactual one, since some sessions left no usage record and part of the work ran under subscriptions. Supervising and checking the agents took more than 100 hours of one author’s active time.
Figure S3. Nearly all released values rest on two or more independent readings. Flows of the 73,542 released values from the kind of record they were located in (left), through the number of independent readers behind them (centre, field reader_agreement), to whether they are accepted (right). Two or more readers: 69,253 values (94%). One visual reader whose numbers are all found in the PDF text layer: 2,547. One reader: 1,342, including running-text values read by one text reader, which are released only with a verbatim quote verified on the page. Row not settled: 400 invariant-table rows still awaiting a second reader, none of them accepted. Values removed before release (failed quotes, fields outside their window, the 3,550 duplicates removed in the clean-up and the 3,998 merged by the gates) are not drawn. Node heights are proportional to counts, except that small nodes are drawn at a minimum height for legibility.
Verification against the printed page
Agreement between readers shows that they saw the same thing, but not that it is printed where they claim. For this reason, a value read from running text is kept only when its quoted sentence is found verbatim on a single page of the PDF with the printed number inside it, and any further detail that a reader attached to the value, such as a phase, a method or a citation, is kept only when it is printed in the same sentence or immediately around it, the exact rules being listed in Table S3 in the ESI†. Details that fail this test are removed rather than guessed, and the removal is recorded with each value.
Figure S8. From sources to datasets. Six sources pass through processing and checks into eight datasets that serve four uses. Literature tables and running-text passages are read by independent readers; values pass reader agreement, verbatim-quote verification, the window rule for field placement, page-image sign checks, the Version 1 clean-up and the literature-level gates. The SGTE binary TDB files give calculated phase diagrams and, with the literature labels and the Materials Project and OQMD entries (arrow from lane 2), the phase registry. The bibliography (Bibliography) links the papers of the corpus, the cited works, the TDB files and the handbook diagrams. Handbook diagrams are traced and named by rules, paired readers, an adjudicator and an expert; only derived data are released. Synthetic papers generated from TDB files test the readers against exact truth. Rows are coloured by source. All datasets are served as files, web pages and through an MCP server.
| reader_tier | values |
| Flash and Claude agree (preliminary_flash+claude) | 59,435 |
| confirmed by Gemini Pro (confirmed_pro) | 8,053 |
| first reader, second reader’s verdict recorded (preliminary_flash) | 5,856 |
| judge decision (preliminary_flash+claude+fable_judge) | 198 |
| total | 73,542 |
Table S9. Reader agreement tiers of the literature values.
Evidence classes
A value that has passed these tests still says nothing, by itself, about how it was obtained, and yet the difference between a measurement, a calculation and an expert’s evaluation is precisely what a user needs to know before including the value in a fit. Each value is therefore assigned exactly one evidence class, namely experimental, model (calculated with a CALPHAD or other thermodynamic model), first-principles (DFT) or unspecified, while the assemblages taken from expert-drawn phase diagrams form a class of their own in Assemblages. The class is decided from what the paper prints, in the column or label of a table, in a caption or in the text, and, for a value reprinted from another work, from that work itself, whose title, abstract or text usually states whether it reports a measurement or a calculation. Where neither source settles the question, a cited work published before 1980 is taken to be experimental and any other value remains unspecified, and in every case the field origin records which of these rules made the decision, so that the classification itself can be checked (Table S10).
Printed-page sign checks
Reader agreement and quote verification can both miss errors already present in the PDF text layer. In this text layer, a minus sign may be dropped, separated by a line break or interpreted as a digit. To check for these errors, formation enthalpies with suspect signs were examined on images of the printed pages. The record retains either the sign confirmed on the page or a flag indicating that no sign is printed, so that a sign is never supplied without printed evidence (Supplementary Note 1 in the ESI†).
| Class | Definition | Rule (recorded in origin) | Values |
| experimental | measured | the paper’s column, label, caption or text names a measurement; for a cited value, the cited work’s title, abstract or full text does; cited works published before 1980 are taken as experimental | 36,288 |
| model | calculated from a CALPHAD or other thermodynamic model | the printed source names a calculation or an assessment; linked to a TDB file only when unambiguous (12,833 values) | 26,153 |
| dft | first-principles calculation | printed as DFT or ab initio; never compared with DFT | 6,150 |
| unspecified | origin not settled | no printed or resolvable kind of source | 4,951 |
| assessed_phase_diagram | read from an expert-drawn handbook diagram | Assemblages only; the original evaluation is cited first, the handbook second | 23,129 |
Table S10. Evidence classes. Every value has exactly one class; classes are never mixed or averaged. Counts: literature values (Values) and assessed points (Assemblages).
Units and conditions
The same principle governs the conversion of values to common units, which is necessary if values from different papers are to be compared, but which must not obscure what the authors actually printed. Temperatures are therefore converted to kelvin and compositions to mole fractions, while the printed string and the original basis are kept with each value. Formation and mixing quantities are expressed per mole of atoms only when that basis is printed or can be established from the paper, and are otherwise left unconverted, with their basis marked as unstated. This distinction also applies to pressure, which is recorded as an assumed one atmosphere for all but 5 values because it is almost never printed. The basis field identifies the assumption, and it is withdrawn when the page specifies another pressure. Lattice lengths printed in nm or pm are converted to Å, with the stored value retained. Other quantities retain their printed units, including 0 Gibbs energies that are coefficients of in expressions of the form and therefore have units of J mol K.
Uncertainties are treated in the same way. Only 5,852 of 73,542 values have a printed uncertainty. Where present, the reported uncertainty is retained in the primary uncertainty field, with documented conversions to the record unit and preservation of the original information. Estimates of scatter between studies and of uncertainty due to composition differences are stored separately in the uncertainty budget, together with their derivation. The distinction allows the reported precision to be separated from the effective uncertainty used to weight ESPEI datasets. To retain the experimental context, samples, purity, heat treatment and measurement methods are stored as quoted items with page references. These items are attached only to values from the paper’s own experiments, preventing the transfer of one study’s conditions to another study’s data.
Calculated phase diagrams
After extraction of a value and its conditions, an assessed CALPHAD description provides a reference for interpreting the result within its binary system. For this purpose, the release includes 2,722 phase diagrams calculated with pycalphad[9] from the SGTE collection of binary assessments[12]. The diagrams were mapped using BinaryStrategy in pycalphad 0.11.2 at 101,325 Pa, with composition steps of 0.02 and temperature steps of 10 K. The requested temperature range was 300–2,500 K, extended to 300–4,000 K when a refractory element was present. This mapping provides phase boundaries as zero-phase-fraction lines together with the invariant equilibria. A grid of stable phases, calculated by direct equilibrium calculations at 0.01 in composition and 10 K, was added so that the stable phase assembly at any point of the diagram can be looked up rather than recomputed. The SHA-256 checksum associates a diagram with its TDB file, and the accompanying citation identifies the assessment. The calculated invariants are therefore model values whose reliability is limited by that assessment and the resolution of the calculation grid.
Agreement with the CALPHAD model
The predictions of an assessment provide a second description of a reaction that can be compared with the literature record. Although this comparison is useful for detecting inconsistencies, model values do not constitute a standard for establishing that literature values are correct. When a value read from a paper and the corresponding value calculated from an assessment agree closely, the agreement supports both, whereas a large discrepancy points to an error in the reading, in the paper or in the model, or to a reaction that the model does not reproduce. The comparison was performed for three-phase invariants whose phases could be mapped to a linked TDB file and for pure-element transitions using the SGTE unary data. Agreement required a temperature difference of at most 25 K and, where printed compositions could be mapped, phase-composition differences of at most 0.05 in mole fraction. This comparison does not yet include congruent and polymorphic transitions of compounds. Across these comparisons with 438 TDB files, 6,476 values were evaluated, of which 5,691 agree and 785 disagree. For these evaluated values, the plausibility field records the model and the temperature and composition differences, while the extracted value remains unchanged. Agreement is reported separately for experimental values, of which 2,551 of 3,006 agree, since a model value checked against the TDB file of its own paper tests the self-consistency of the record rather than its agreement with independent evidence.
Convex hulls
A second, independent reference is provided by first-principles energies, from which binary convex hulls at 0 K were built using pinned snapshots of the Materials Project (release 2026.04.13) [3] and OQMD (v1.8, February 2026 release) [4, 5]. These energies are calculated without reference to the extracted literature values and thus provide an independent comparison with those values and the assessments. However, both databases use experimental references in their corrections or fitting, through the fitted Materials Project corrections (MP2020) [35] and the OQMD fitted elemental reference energies (the standard, or fit-partial, set of Kirklin et al. [5]), which are used as stored by each database. To make the stability information accessible, a hull is stored as a sequence of composition intervals from which stable phases and relative chemical potentials can be read directly. In this representation, polymorphs on or very near the hull are retained and flagged. The two databases, however, use different energy references and corrections, and they are therefore never averaged or compared with one another directly. Their stability information is also treated differently, since the Materials Project hulls reproduce the stability reported by the database itself for all but one system, whereas the stability of OQMD entries, which could not be reproduced from its stored values, is derived from the hulls.
DFT comparison and calibrated uncertainty
Each source can, however, be compared with experiment, since many of the compounds on the hulls also appear in the literature with measured formation enthalpies. To pair a literature value with a DFT entry, their reduced compositions must be identical. The formula is taken from the printed phase label, and records naming more than one formula are set aside. For a given source, the match uses the lowest entry in the space group identified by the record through a Strukturbericht designator or prototype. If no structure is identified, the lowest-energy entry at that composition is used. Restricting the comparison to accepted experimental values whose sign and magnitude raise no doubt then gives, for each database, a set of pairs whose differences measure how far DFT formation energies can be trusted. The differences are analysed separately by database and by groups of chemically similar compounds. This analysis gives robust uncertainty estimates of 8.6 kJ (mol of atoms) for the Materials Project and 7.5 kJ (mol of atoms) for OQMD, together with a small systematic bias that is recorded but not applied. The ESPEI datasets use the calibrated uncertainty to weight first-principles data (Figure S14). Source-verified repeated extractions or later reports of the same DFT result contribute once to the calibration and fitting inputs, while the literature records retain both identifiers and the reported values through an explicit canonical evidence link. The same identity links alternate unit representations of one observation. For experimental values reported alongside DFT calculations, the evidence class follows the description of the measurement; when the original measurement conditions remain unresolved, both representations are retained in the literature release and excluded from calibration and fitting.
Figure S14. Calibrated DFT uncertainty and the resulting ESPEI weights. a, Robust of DFT minus experiment with 95% intervals from 1,000 bootstrap resamples of whole chemical systems for each stratum (chemistry and DFT match quality) of the Materials Project and OQMD, and for literature DFT values by chemistry; numbers give the pairs per stratum. The match-quality stratum of a compound is assigned separately in each database, according to its own polymorphs and convex hull, so that the same compound can fall in different strata and the numbers of pairs per stratum differ between the databases. Top axis: the ESPEI weight, 500 J mol divided by . Strata with fewer than 30 pairs fall back to the source-level in the datasets. b, Cumulative distribution of the per-value weights written to the DFT datasets (leave-system-out ). Stratum values: Table S5 in the ESI.
:::`
Data clean-up
The two references also serve a more practical purpose, since a literature value that disagrees with both of them by a wide margin often reveals an error of reading or of reporting rather than a genuine discrepancy. The clean-up of Version 1 used this and simpler criteria to address four kinds of defect: duplicated records, inconsistent labels for reactions and properties, formation enthalpies printed per formula unit but labelled per atom, and fields garbled during reading. In every case, the clean-up merges, relabels, converts or flags a record but never invents a value, and the original is kept wherever a value changed, and each change is listed with its evidence in the release (QUALITY_FIXES.md; counts in Supplementary Note 4 in the ESI†).
Literature-level consistency gates
The clean-up treats records one at a time, but many errors only become visible when a value is compared with its table, its paper, the physics of its quantity and the calculated diagrams of its system, and for this reason a second set of gates was applied to the whole release (tools/literature_gates.py). These gates correct a record only when the correction is deterministic and evidenced, by a printed header, caption or heading or by the internal consistency of a whole table, and for changes of unit, sign, scale or chemical system the correction was in addition confirmed on the rendered page of the PDF. Of the 77,540 records that entered the gates, 15,528 received a correction. Most corrections concerned the conditions of a value, namely 2,933 temperatures (137 in tables printed in degrees Celsius but stored as kelvin, and 2,790 stored at 298 K that denote 298.15 K) and 327 assignments to a chemical system, while others concerned its meaning, namely 916 phase lists of invariant reactions and 286 formation quantities that had been recorded as mixing quantities, and the remaining 108 values and 211 units were read again on the page. A record that could not be corrected in this way was demoted or flagged instead, with the reason in gate_failures, and the gates merged a further 3,998 duplicates that differ only in the spelling of a phase label, leaving 73,542 records, of which 65,284 were accepted at that stage (76,600 before the gates). The current release contains 66,551 accepted records. The confirmations on the page were made by AI agents that inspected the rendered page, not by a person, and each of them is recorded in a ledger of 3,262 entries (tools/literature_visual_checks.json) that gives the page, the hash of the page image, the verdict and a short note of what is printed, without any page image or text. No printed value or quote is altered, and the stored value is kept beside every correction. Three rules introduced in the third round determine which states can be released as accepted. A value of an invariant, a transition, a phase composition or a formation quantity requires the phases of its state, recovered from the verbatim quote (264 records) or read on the page (16); labels such as “unknown”, “other” and “?” do not satisfy this requirement. The R3 gate recorded 8,944 missing-phase findings at that stage; values without identified phases were not accepted. A later page-checked recovery restored printed phases for 1,267 records demoted only by the placeholder rule and accepted them again, while retaining their exclusion from ESPEI output. For invariant reactions, the phases are checked against the printed reaction, a missing class is filled only for binaries written in a deciding form and a printed class is never overwritten. The count of phases is also checked against the phase rule, and a second record of the same invariant on one page is flagged (303 and 179 records). In turn, every element in the printed formulas of the phases provides a lower bound on the system of the state, so that compound data cannot be narrowed to a unary system. Finally, a formation value below 0.1 kJ (mol of atoms), or one whose unit prefix is printed separately from the unit, is accepted only after a check on the page.
Phase registry
Combining values from different sources raises the question, already noted in the introduction, of whether two of them refer to the same phase. The registry answers it by giving each phase of a binary system a single identity, built from the phases declared in the TDB files of the SGTE collection, the phase mappings made during extraction and the labels printed with the literature values, and by collecting under that identity every name the phase carries, each with the source in which it appears (Figure S9). Structure types (prototype, Strukturbericht designation, Pearson symbol and space group) were assigned with the same caution, through three kinds of evidence of decreasing directness. The most direct is a standard phase name that encodes a single structure, such as L12_FCC, accepted when the model’s stoichiometry, sublattices and DFT data do not contradict it. Where the name is not informative, a stoichiometric compound can take its structure from DFT, but only when it matches exactly one entry on the hull of both the Materials Project and OQMD, with the same space group and no other phase at that composition. The crystallographic table that accompanies the SGTE collection [12] provides the third kind of evidence, accepted when all of its rows for a phase give the same structure and nothing in the first two contradicts it. A structure type is thus assigned only when the evidence makes it certain (the first two kinds) or when it is stated by the assessors themselves (the third), and every other phase is placed in a review queue together with its candidate structures and the reasons why none was assigned. For each phase, the registry also keeps the TDB models that describe it, with their definitions copied verbatim, its stability range in the calculated diagrams, and examples of the papers that model it or report values for it.
Figure S9. The phase registry (Registry). a, The identity phase:al-sc:al3sc-l12 (AlSc, L1), read from the released file phases/systems/AL-SC.json: its names in five namespaces (paper labels with their DOIs, the TDB phase name with its files, the corpus identifier, and the Materials Project and OQMD entries), its structure type with the kind of evidence that assigned it, and its TDB model with the verbatim PHASE and CONSTITUENT statements. b, Phases by the way their structure type was assigned: standard name (Tier A, 6,228), DFT-supported (Tier B, 154), assessment-declared (Tier C, 1,248), liquid (1,304), gas (231), review queue (1,236) and unresolved (29), of 10,430 phases.
Assessed phase assemblages
Not all of the evidence about a system takes the form of printed values, since much of it has been condensed by experts into evaluated phase diagrams, drawn from the experimental literature without an underlying thermodynamic model. Assemblages draws on the binary diagrams of the Okamoto Desk Handbook: Phase Diagrams for Binary Alloys (2nd ed., ASM International, 2010), and because such a diagram is the outcome of an evaluation rather than a measurement or a calculation, the assemblages read from it form an evidence class of their own. Reading such diagrams relies, above all, on the rules of phase equilibria. A deterministic pass first traces drawn lines, calibrates the axes and identifies the fields, and printed labels and the law of adjacent phase regions then name as many fields as the diagram allows, a name being accepted only when a single assemblage is consistent with the whole diagram (main-paper Fig. 3). Machine readers are consulted only for the fields that the rules leave open, and even then they choose among the assemblages that the rules permit, while an adjudicator, and finally an expert, settle the cases that the readers cannot (Supplementary Note 2 in the ESI†).
Uncertainty budget
To weight evidence in a fit, the uncertainty must describe the reliability of the value rather than only the statistical precision usually reported for a measurement. For example, Knudsen-effusion mass spectrometry values for CuZr and CuZr report uncertainties of 0.08 to 0.26 kJ (mol of atoms), yet differ from calorimetric values by 3.7 to 8.0 kJ (mol of atoms). The comparisons below show that differences of this magnitude between independent studies are common. An uncertainty budget is therefore retained for all values, preserving the reported uncertainty alongside additional contributions. The reported interval and its stated coverage remain distinct from the effective uncertainty used in fitting, so that a calculated estimate does not replace the uncertainty attributed to the source.
These are the resolution of the printed number, a systematic contribution that depends on the class of method used, and, where they apply, terms for a measurement made at a historical formula, for the scatter of values found on the page to be mutually inconsistent, and for a conversion to another reference state. To calibrate the systematic contribution, we compared independent measurements of the same state obtained by the same class of method in different primary studies. As in the DFT calibration, the scatter across these measurements determines the floor, rather than the precision claimed within an individual study. The scatter was estimated robustly as 1.4826 times the median absolute difference divided by , with a bootstrap over states providing the 95% interval. Classes with fewer than 20 pairs or fewer than 10 states received a documented literature prior [23, 24]. If such a class still contained at least 5 independent pairs from 3 states, the larger of this prior and its own robust scatter was used (11 classes). In applying these rules, no floor was permitted below the default ESPEI uncertainty. For formation enthalpies, the calibrated floors are 3.2 kJ (mol of atoms) for solution and reaction calorimetry (80 pairs; 95% interval 2.2–4.6), 4.6 for EMF measurements and 3.5 when the method is not printed. For invariant temperatures and phase compositions, the floors are 11 K and 0.011 in mole fraction (Table S11). These floors are larger than the agreement usually claimed for good calorimetry, because they also absorb the errors of extraction and of reference state that remain in Version 1, and they are therefore conservative.
| Quantity | Method class | Floor | Pairs (states) | 95% interval |
| formation enthalpy | method not printed | 3.5 kJ (mol of atoms) | 290 (98) | 2.41–4.3 |
| formation enthalpy | solution / reaction calorimetry | 3.2 kJ (mol of atoms) | 80 (47) | 2.2–4.6 |
| formation enthalpy | experimental, method not named† | 4.1 kJ (mol of atoms) | 29 (9) | 1.99–9.33 |
| formation enthalpy | Knudsen effusion / KEMS | 4.2 kJ (mol of atoms) | 20 (15) | 1.15–7.65 |
| formation enthalpy | EMF† | 4.6 kJ (mol of atoms) | 17 (8) | 1.89–14.8 |
| formation enthalpy | combustion calorimetry† | 1 kJ (mol of atoms) | 15 (4) | 0.0881–5.33 |
| Gibbs energy of formation | method not printed | 1.2 kJ (mol of atoms) | 37 (17) | 0.66–2.1 |
| invariant temperature | method not printed | 11 K | 4,912 (1,680) | 9.44–10.5 |
| transition temperature | method not printed | 16 K | 274 (53) | 10.5–26.2 |
| transition temperature | thermal analysis† | 5 K | 9 (4) | 2.1–58.2 |
| phase composition | method not printed | 0.011 mole fraction | 3,014 (1,426) | 0.0105–0.0115 |
| lattice parameter | method not printed | 0.22 % of value | 586 (192) | 0.18–0.25 |
| lattice parameter | X-ray / neutron diffraction | 0.13 % of value | 30 (25) | 0.09–0.39 |
| lattice parameter | experimental, method not named† | 0.13 % of value | 9 (9) | 0.05–0.48 |
| cell or molar volume | method not printed† | 0.54 % of value | 43 (5) | 0.37–1.97 |
| absolute entropy | method not printed | 0.7 J K mol | 77 (20) | 0.285–1.17 |
| absolute entropy | drop / heat-content calorimetry† | 1 J K mol | 19 (9) | 0.0734–1.93 |
| partial Gibbs energy | method not printed† | 1.4 kJ mol | 13 (13) | 1.09–1.7 |
| formation enthalpy | model-calculated (against experiment) | 3.9 kJ (mol of atoms) | 2,045 (361) | 3.11–4.55 |
| invariant temperature | model-calculated (against experiment) | 4 K | 14,146 (5,377) | 3.11–4.3 |
| phase composition | model-calculated (against experiment) | 0.005 mole fraction | 16,730 (8,821) | 0.00445–0.00445 |
Table S11. Table S11: Method floors of the uncertainty budget calibrated from the release. Floor: robust scatter between independent measurements of the same state made with the same class of method in different primary studies (1.4826 times the median absolute difference divided by , rounded); pairs (states) and the bootstrap 95% interval of the scatter. Bottom: model-calculated values against experimental values of the same state. The other 46 classes, with fewer than 20 pairs or fewer than 10 states, take a documented prior [23, 24]. Every value carries the budget; 47,252 values take a calibrated floor and 12,904 a prior. Source: literature/UNCERTAINTY_CALIBRATION.json.
The effective uncertainty combines the larger of the printed uncertainty and the floor with the other contributions. This value determines the default ESPEI weights, while the retained components allow users to adopt a different combination rule.
ESPEI datasets
To convert the extracted evidence into ESPEI datasets, the preceding checks must establish the state, its mapping to the phases of a model, and the consistency and uncertainty of the value. These conditions are recorded for every value in two fields. The first, espei_ready, marks a value that can be used in a fit, because its state is complete, its phases can be mapped unambiguously to those of a CALPHAD model and are physically possible at its temperature, and it is consistent with the rules of phase equilibria. The field espei_certified further identifies ready values whose states are reproduced by the pycalphad equilibrium of the corresponding assessment within 25 K and 0.05 in composition. Ready values that disagree with this calculation, or lack a corresponding calculation, remain available with the appropriate flag.
The influence of these values on a fit depends on how ESPEI represents uncertainty. ESPEI expresses the uncertainty used in a fit through weights relative to default experimental uncertainties. These default uncertainties in ESPEI 0.9.1 are 500 J mol for enthalpies and 0.2 J K mol for entropies and heat capacities [10]. The weight of an observation is therefore the experimental reference divided by its effective uncertainty. When a literature dataset contains values with unequal effective uncertainties, ORCHID exports them as separate fitting records so that a common weight does not replace those differences; values with equal uncertainties can remain grouped. To allow this weight to be recomputed under another policy, the uncertainty budgets are retained in physical units, together with the basis of their components. These bases include uncertainties printed by the authors, method floors calibrated against independent measurements, and comparisons with experiment for DFT and model values. The records retain their links to the original dataset and source study because separation for weighting does not establish statistical independence, and the likelihood used here does not represent covariance between observations from a shared source.
The uncertainty budgets also allow the choice of evidences to be reconsidered. First-principles formation energies are included only when the compound can be assigned to a single endmember of a specific TDB phase model, and receive weights based on the calibrated DFT uncertainty. Model-calculated values instead use a floor obtained by comparison with experimental values of the same state (3.9 kJ (mol of atoms) for formation enthalpies). The resulting weights are choices that users are expected to revisit. Every dataset is tagged with its evidence class and its quality flags, so that what enters a fit, and with what weight, is decided by the user rather than fixed by the release (Figure S10).
Validation
A user who is to decide what enters a fit needs, above all, an estimate of how far each kind of value can be trusted. The validation described in this section provides such estimates in three steps, moving from the reading itself, through the consistency of the extracted values with independent references, to the derived datasets built from them.
Quote verification
The most direct test of a reading, that the value is printed where the record says it is, was applied to every value taken from running text, and all 5,743 of them carry a quoted passage found verbatim on the cited page with the printed number inside it. This test establishes where a value comes from, but not whether it was interpreted correctly, and the remaining checks in this section address that second question.
Reader comparison pilot
A first indication of how often a reading goes wrong came from a pilot in which the two text readers answered the same 127 requests. Claude Opus found more values than Gemini Flash and quoted them verbatim far more reliably (546 of 553 rows, against 306 of 388), and the errors of both readers concentrated in passages whose PDF text layer is corrupted by control characters, the same passages in which Flash also lost minus signs. These differences are the reason the release relies on two readers together with verification against the page rather than on either reader alone, although a comparison between readers is not, in itself, an estimate of precision (Supplementary Note 1 in the ESI†).
Extraction benchmark
A controlled estimate of extraction precision can be obtained from a reference with known generating values, which is the purpose of the synthetic papers in Bench. Because every table, diagram and sentence in them was generated from an SGTE database before being rendered as a born-digital PDF, a clean image or a degraded scan, any extraction tool can be scored against the truth, and in this release the benchmark served both to compare the readers and to expose the failure described in the Methods, where one reader returned empty tables under one prompt version. Its scores, summarized in Table S6 in the ESI† and Figure S5 in the ESI†, show that the AI readers recover invariant-reaction tables almost completely from born-digital PDFs and largely from degraded scans, while deterministic tools recover far fewer invariant rows even from born-digital PDFs and fail on most degraded diagrams.
Scope of the numerical human-review sample
Synthetic papers cannot reproduce every difficulty of the published literature, from corrupted text layers to tables that continue across pages. A stratified random sample of accepted values was frozen before the corrections described below for independent reviewers to check against the printed pages field by field, with overlapping items intended to measure reviewer agreement. The sample holds 300 values from the export of 22 October 2026, of which 288 remain in release v2026.10 (260 of them accepted), the other 12 having been merged with duplicates. This sampling record is retained as part of the audit history, but the review was not completed and no aggregate human precision estimate is reported for the numerical collection. The preliminary responses described below do not constitute a completed evaluation of the sample, and subsequent source adjudication was AI-assisted. Because the sample predates the corrections, it also does not establish the precision of later releases.
Consistency with independent references
Verification of a reading establishes fidelity to the source, while comparison with thermodynamic references can reveal a source error or an incorrect assignment of the system or state. We therefore compare the eligible records across the release with first-principles formation energies, invariant reactions calculated from the SGTE assessments and an independent compilation of experimental formation enthalpies. The independence of these references varies, since an assessment may have used the same measurements and a compilation may reproduce them, as detailed below.
Formation enthalpies offer the broadest of these comparisons, since many of the compounds reported in the literature also appear in the Materials Project and OQMD. Over the default subset of accepted experimental values, DFT and experiment agree with a mean absolute error of 9.6 kJ (mol of atoms) for the Materials Project and 8.3 kJ (mol of atoms) for OQMD, and with Spearman rank correlations of 0.86 and 0.91 (Figure S13, Table S12), values consistent with earlier assessments of DFT accuracy1. The residuals are, however, far from Gaussian, with a robust spread of about 7.5–8.6 kJ (mol of atoms) and a tail of large disagreements, of which 74 were examined, 26 being per-formula values labelled per atom and 6 unsigned magnitudes, which were explained and corrected or flagged, while 42 have no deterministic explanation and are flagged dft_outlier_unexplained. 25 of them remaining in the default subset, with the flag, and the others being excluded by its point rules (Supplementary Note 4 in the ESI4). The comparison is not entirely independent of experiment, since the Materials Project corrects its energies, and OQMD fits its elemental references, against measured enthalpies, but it is independent of the literature values and of the reading.
| Source | Subset | MAE | Median | RMSE | Spearman | Pearson | Band | |
| MP | default | 1,070 | 9.6 | 6.0 | 14.0 | 0.86 | 0.94 | 0.66 |
| MP | per-compound median | 455 | 9.8 | 6.1 | 14.1 | 0.87 | 0.93 | 0.65 |
| MP | without unexplained outliers | 1,053 | 9.2 | 5.9 | 13.2 | 0.86 | 0.95 | 0.67 |
| MP | calorimetry only | 295 | 9.0 | 5.7 | 13.3 | 0.89 | 0.96 | 0.66 |
| OQMD | default | 1,107 | 8.3 | 5.3 | 12.4 | 0.91 | 0.95 | 0.71 |
| OQMD | per-compound median | 481 | 8.3 | 5.2 | 12.3 | 0.92 | 0.95 | 0.71 |
| OQMD | without unexplained outliers | 1,085 | 7.9 | 5.1 | 11.6 | 0.91 | 0.96 | 0.72 |
| OQMD | calorimetry only | 303 | 7.6 | 4.9 | 11.7 | 0.94 | 0.97 | 0.74 |
Table S12. Literature formation enthalpies against DFT (default subset unless stated). MAE, median and RMSE in kJ (mol of atoms); band: fraction within kJ (mol of atoms).

Figure S13. DFT against experiment for formation enthalpies. Default subset: accepted experimental values, signed in print, without uncorrected sign loss or magnitude flags, each paired with the DFT entry of the structure that the paper names (the lowest-energy entry of that space group), or with the lowest-energy entry at the same composition when no structure is printed. Pairs are left out when the named structure is absent from the DFT database, when the measured composition is more than 0.02 from that of the compound, when the reference state is not the solid standard reference of 0 K DFT, and when the literature value has a gate failure. a, Materials Project (n = 1,070; MAE 9.6 kJ (mol of atoms)). b, OQMD (n = 1,107; MAE 8.3 kJ (mol of atoms)). Hexagonal bins on a log colour scale; solid line, ; dotted lines, kJ (mol of atoms). c, Residuals (DFT minus experiment) of both sources; dashed lines, medians (bias 2.0 and 2.0 kJ (mol of atoms), recorded but not applied); robust (1.4826 times the median absolute deviation) 8.6 and 7.5 kJ (mol of atoms); 1 (MP) and 1 (OQMD) residuals beyond kJ (mol of atoms) are not shown.
Phase equilibria provide a second and more demanding comparison, since an invariant reaction fixes a temperature, a set of phases and their compositions at once. When the experimental invariant temperatures in the release are matched, by system, reaction type and phase set, to the invariants calculated from the SGTE assessments, the two agree with a median offset of 0.1 K and a robust spread of 5.9 K, 73% of them within 10 K (main-paper Figure S15). Since the assessments were themselves fitted to many of these measurements, close agreement mainly confirms that the values were read and assigned correctly, while the small number of large disagreements points either to assessments that depart from the data or to values that require a second look.
Figure S15. Literature invariant temperatures against SGTE-calculated invariants. Accepted experimental temperatures of binary eutectic, peritectic, eutectoid and peritectoid reactions, matched by system, reaction class and phase-registry identities (, 401 systems). Eutectic, peritectic, eutectoid and peritectoid values number 2,014, 1,135, 281 and 186. a, Calculated against literature temperature; hexagonal bins on a log colour scale; line, ; grey band, K. b, Residuals (calculated minus literature); dashed line, median (0.1 K); solid lines, median robust (5.9 K); 86 residuals beyond K are not shown. Where several assessments of a system match, the median over assessments is used, each distinct invariant counted once. The comparison is in sample: 3,599 of the 3,616 values come from a paper that is the source of one of the SGTE assessments of its system.
Of the three references, the most direct is an independent compilation of experimental formation enthalpies [22] because its values were gathered by other authors, often from the same original sources, so that a disagreement points to the reading or to the compilation rather than to any model. For the 160 compounds present in both, the two agree with a median difference of 1.2 kJ (mol of atoms) and a mean absolute difference of 3.8 kJ (mol of atoms), with every sign in agreement, while the 19 compounds that differ by more than 10 kJ (mol of atoms) are listed in the release so that the source of each discrepancy can be traced.
Independent audits and corrections
To determine whether individual discrepancies reflected systematic errors, separate audits searched the whole dataset for recurring classes of error. These audits found several, among them tables in which temperatures printed in degrees Celsius had been stored as kelvin, values assigned to the wrong chemical system, formation quantities recorded as mixing quantities, and an export step that labelled thermochemical data as liquid by default. Each class was then either corrected, when the printed page settled the value unambiguously, which was verified by inspecting the page itself, or the affected values were withdrawn from the fit-ready subset and flagged, and a second audit confirmed the result (Supplementary Note 3 in the ESI[1]). The release records the checks passed and corrections received by each value so that users can select evidence according to the requirements of their analysis.
Derived datasets
Checks on derived datasets address both the source values and the transformations used to prepare them for their intended use. For the ESPEI datasets, the relevant test is whether ESPEI can use them, and each was therefore evaluated against the TDB file it names, with every one of the 8,589 datasets in the release yielding a finite likelihood, while the 387 that did not were excluded with the reason recorded.
For the phase registry, the corresponding test is whether the identities it asserts hold up when different sources are compared, and building the registry was, in effect, such a test, since every disagreement between sources was recorded rather than resolved silently. The recorded conflicts (CONFLICTS.md) identify uncertainty in phase identity, including names used for different structures in different systems, disagreement between Materials Project and OQMD ground states, names unsupported by DFT, and families of polymorphs. In all such cases no structure type was assigned, while the verbatim TDB definitions were checked against the files from which they were copied. Structure types assigned from the assessors’ own tables (the third kind of evidence) rest on their statement rather than on an independent check, and are marked accordingly.
The phase assemblages read from handbook diagrams required the most extensive validation, because their names were assigned partly by machine readers. To assess the naming procedure, a blind audit of 185 fields used independent model instances that saw the assigned name but not its basis. Name precision ranged from 0.87 to 1.00 across naming routes, and was about 0.96 after weighting by the number of fields. Auditors from the same model family as some readers may, however, judge those readers too favourably. The assessed assemblages agree with SGTE equilibria in 17,876 of 21,488 comparable point–assessment pairs. Considering agreement at the point level, 8,333 of 9,425 comparable points agree with at least one assessment. The disagreements identify differences between handbook evaluations and assessments, and also exposed two systematic annotation faults corrected before release. The expert, finally, reviewed the 126 most difficult fields with the adjudicator’s reasoning in view and accepted 92 of the 93 named proposals, a figure that measures verification rather than independent agreement. An independent estimate comes from a blind sample of 48 fields, one per diagram and drawn from diagrams the expert had not reviewed, in which the expert named each field without seeing the machine’s answer. In this blind sample, the expert and machine names agreed for 45 of 48 fields (93.8%, 95% interval 0.83–0.98). Agreement increased to 47 fields (97.9%, 0.89–1.00) when two notation differences were recognized, namely repetition of the same phase and a prefix unresolved by the drawing. Within this sample, all 28 fields named using printed labels and phase-equilibrium rules agreed. The one substantive disagreement concerns a field bounded by three closely spaced, separately labelled line compounds, where the drawing itself leaves the assignment ambiguous, and the expert’s answer has been adopted.
Adversarial audit and native error rates
The adversarial audits searched for error classes beyond those anticipated by the existing checks. For this reason, before the release, we also searched for errors deliberately, using six independent auditors, language-model agents that could read the release, the code and the printed pages but could not change them, each instructed to find errors rather than to confirm quality within a single scope. Four of these scopes concerned the data themselves (elements and compositions; phases and their mapping to CALPHAD models; units, conditions and uncertainties; and DFT energies with their pairing to experiment), while the other two concerned how the data are presented, tracing every figure and every number and claim in this paper back to the release. Each auditor combined targeted scans for whole classes of error with a random sample of records drawn with a recorded seed, and every suspected error was checked against the printed page of its source before it was counted. The sampled fractions therefore estimate the native error rate of the release, that is, the rate of errors that remained after all the gates and corrections described above (Figure S12, Table S13). For elements and compositions, phases, and the pairing of experimental and DFT energies, the native rates of wrong values or states lay between 3 and 4 per cent. The rate was highest, 10.9 per cent (6.4–18.1 per cent), among thermochemical records, because thermochemical tables mix bases, reference states and property types more often than tables of phase equilibria do, and it is these tables that the uncertainty budget described above treats most conservatively. Random samples of this size cannot, however, reveal errors that are rare among records but systematic in their cause, and it was the targeted scans that found these, among them compositions divided by one hundred twice when a header announced atomic per cent over values that were already fractions, polymorphs matched to the wrong DFT structure, intermediate phases labelled ε mapped to the bcc solution, absolute entropies recorded as entropies of formation, and lower-order data filed under the system of the paper rather than that of the state. Each of these classes was corrected from the printed pages and incorporated into a regression test that subsequent releases must pass. The native rates document the audited snapshot before correction, providing a baseline for comparison with subsequent audits. When the same protocol was applied to new random samples of the corrected release, the rate of wrong values or states was 1.82 per cent for elements and compositions, 0.33 per cent for phases, 9.09 per cent for units, conditions and uncertainties, 2.73 per cent for the pairing of DFT with experiment, 0.0 per cent for figures and 1.0 per cent for the claims of this paper (Table S13), and every rate is equal to or lower than in the first round, although for most scopes the intervals overlap, because the samples are small. The incomplete human-review record (§ 21.4) contains 240 submitted responses covering 210 distinct records in the frozen 300-record sample. These preliminary responses do not establish a sampling-adjusted estimate of population accuracy, and the subsequent source adjudication was AI-assisted.
| Audit scope | Sample; seed | Wrong value or state | All errors | Systematic classes | Post-fix |
| Elements & compositions (A1) | accepted literature records; 110, seed 20261024 | 4/110 (3.6%; 1.4–9.0) | 17/110 (15.5%; 9.9–23.4) | 8: at.% divided twice; header names wrong element; binary filed under ternary | 1.82% all 4.55% |
| Phases & mapping (A2) | registry, Assemblages and literature phase labels; 300, seed 20261024 | pooled 9/300 (3.0%; 1.6–5.6) registry: 5/100 (5.0%) Assemblages labels: 2/100 (2.0%) lit. model_phase: 2/100 (2.0%) | same (no metadata errors) | 8: ε→BCC_A2; βPtZr (B2)→PTZR_B33; first sublattice dropped | 0.33% all 0.33% |
| Units, conditions & uncertainties (A3) | accepted thermochemical records; 110, seed 20261027 | 12/110 (10.9%; 6.4–18.1) | 20/110 (18.2%; 12.1–26.4) | 10: H(T)−H(298) stored at 298 K; per-formula ΔH as per-atom; unsigned ΔH (Co17Y2) | 9.09% all 23.64% |
| DFT pairing & hulls (A4) | default-subset DFT–experiment pairs; 110, seed 424242 | 4/110 (3.6%; 1.4–9.0) | same (no metadata errors) | 6: wrong polymorph (B2→Cm); per-formula/per-atom slip; off-stoich. vs stoich. DFT | 2.73% all 4.55% |
| Figures (A5) | parity pairs and invariant matches behind figures; 200, seed 20261024 | 0/200 (0.0%; 0.0–1.9) | same (no metadata errors) | 10: duplicate invariant rows; page no. parsed as year; dataset numbering differs | 0.0% all 0.0% |
| Paper claims (A6) | literature records vs record-level claims; 100, seed 20261024 | 3/100 (3.0%; 1.0–8.5) | same (no metadata errors) | 8: superseded rule in prose; hand-typed counts; units break Usage claim | 1.0% all 6.0% |
Table S13. Table S13: Adversarial audits of release v2026.10: samples, native error rates and systematic classes. Wrong value or state and all in-scope errors (including metadata): records with an error / records sampled (rate; Wilson 95% interval in per cent). Systematic classes: number found by the targeted scans and three examples. Post-fix: the same protocol applied to the corrected release (orchid-data 1e5f4ce7d), as the rate of wrong values or states and, below it, of all in-scope errors (§ 3.2).
Figure S12. Errors identified by adversarial audits. Seeded samples across six scopes distinguish wrong values or states from all in-scope errors, with Wilson 95% intervals. Initial rates refer to the audited release snapshot 0d9f7c4c3; post-fix rates refer to 1e5f4ce7d. Targeted scans identified additional systematic classes. These are agent-assisted audits of specified snapshots, not human precision estimates or a final error rate for every later revision. Full denominators, correction history and subsequent resampling are retained in SI 1.
The re-audit itself found new classes of error, and some of them had been introduced by the fixes: the rule that moved lower-order states out of higher-order systems had trusted a composition that was already corrupt and narrowed about 290 compound records to a unary or smaller system, the floor of the uncertainty had been scaled by the number of atoms for formulas that were already normalised, and the rule that set temperatures printed as 0 K to 298.15 K had been applied to the DFT rows of papers whose own values were labelled experimental. The post-fix rates are therefore not those of a release free of error, and for units, conditions and uncertainties the rate of all in-scope errors (23.64 per cent) cannot be compared with that of the first round (18.2 per cent), because the re-audit also counted labels that had lost their basis and temperatures that were missing.
We corrected these classes in a third round, and because the re-audit had shown that a fix can introduce errors, we sampled 155 records again in 7 pools of records that the round had changed, rendering and viewing every item. This caught 12 errors that the first build of the round had introduced, among them 11 of 25 sampled invariant reactions, where derived classes had overwritten printed ones, and they were corrected before release; the counts for each pool, with the residuals that predate the round, are in Supplementary Note 3 in the ESI.
A final AI-assisted re-audit of the third-round release used new seeded samples checked against the printed pages. It found wrong values or states in 3 of 110 literature values (2.7%, Wilson 95% interval 0.9–7.7%), 1 of 300 phase-registry records (0.33%), 5 of 110 records checked for thermochemical units, temperatures and reference states (4.5%), and 4 of 110 experimental–DFT parity pairs (3.6%); no error was found in 200 figure checks, whereas 7 of 100 record-level claims in the paper failed the stricter claim protocol (7.0%). Separate samples found errors in 2 of 50 invariant records and 2 of 30 recovered-phase records. Although none of the 30 sampled C4 system reassignments had the wrong system, 7 contained a wrong value or state, concentrated in three tables. Corrections made on 27–28 October followed this audit. In a subsequent AI-assisted review of the fit-facing release, 189 of 200 ESPEI-ready records were fit-safe (94.5%, Wilson 95% interval 90.4–96.9%). The Y11 review used a frozen, stratified sample from the data snapshot 7ac9ca9f18, with rare evidence categories deliberately oversampled. The value, composition, phase, property and evidence class passed in all 200 records. Two temperatures conflicted with the source pages, and nine other records lacked a source-supported molar basis: six DFT mixing-enthalpy records retained an unstated basis, while three experimental formation enthalpies used a normalization inferred from magnitude. These nine cases were precautionary fitting exclusions, not demonstrated errors in the printed numerical values. The strict composite criterion required all seven fields, including temperature and fitting-compatible unit normalization, to pass. The 94.5% proportion describes the reviewed sample; applying the population stratum weights gives a 97.59% point estimate, for which no confidence interval was reported. The audit proposed two temperature corrections and nine fitting-exclusion flags for subsequent integration; its score characterizes the frozen pre-correction snapshot.
Matched MLIP benchmark and one-shot construction
The energy benchmark uses the identical 1,138 records, 455 compounds and 185 binary systems for all eight MLIPs and both DFT references. The primary variant relaxes Materials Project structures and applies the specified MP2020 correction convention. Robust residual spreads use 1.4826 times the median absolute deviation, with 1,000 cluster-bootstrap replicates resampling whole chemical systems. The matched-set results are distinct from the source-level calibration of all available DFT pairs.
The one-shot Cu–Zr models use a common PhaseForge construction for GRACE-2L-OMAT, MACE-MPA0, MACE-OMAT-0, ORB-v3 MPA, ORB-v3 OMat, SevenNet-Omni, MatterSim 5M and CHGNet 0.3.0. Each model completed 21 structure relaxations and five liquid runs. The liquid schedules comprised 2,000, 3,000 and 10,000 steps; all 40 liquid runs passed the recorded diffusion check. Some special quasirandom structures did not meet the force convergence threshold within 1,000 FIRE steps and entered the construction at their last energies; relaxed-away structures were flagged rather than dropped. These choices are part of the tested workflow and must accompany interpretation of the phase-stability results.
The figure and reported field scores use a pinned evidence snapshot, with 28 assessed field points and 287 boundary points. A later corrected release contains 276 boundary points, while the field points and field scores are unchanged. Field confidence intervals use 2,000 point-bootstrap replicates with seed zero. The overlapping intervals do not resolve the ranking of the eight models; a paired significance test against SGTE was not performed. The final PhaseForge+ field score is in sample and is kept distinct from one-shot predictions and held-out activity errors.
Runtime of the final Cu–Zr refinement
The final run, runs/final2/F10b_s3, records 1,618.98 s (26.98 min) and 409 objective evaluations on a Mac Studio (Mac13,1, Apple M1 Max, macOS 14.3). The optimizer script fit_pr268.py uses analytical ESPEI gradients, first with conjugate gradients and then with scaled L-BFGS-B. This run used starting seed 3, the prepared CuZr-PhaseForge-MLIP-B2b.tdb description and the data/a4d_F10 fitting datasets together with the MLIP anchors. The timer starts after loading the datasets and constructing the residual context and covers optimization and final run-output preparation. It excludes literature extraction and review, MLIP relaxation and molecular dynamics, construction of the starting description, preparation of the fitting datasets, and exploration of alternative starts. The reported time therefore measures the final refinement run, rather than the development of the complete demonstration. The earlier tuning.wall_s entry in the aggregate results file refers to a different fit. The timing provenance and source checksum are retained in figures-DD/data/cuzr_final_runtime.json.
Full construction and refinement figure notes
The following legends retain the detailed methodological qualifications of the source manuscript. The figures themselves appear in the main paper.
Figure 3 (PDF p. 46). Annotating an assessed diagram, shown on a synthetic benchmark diagram. The Ni–Ti diagram of the synthetic paper syn–NiTi–04Tok–v0 (Bench), calculated from the SGTE assessment of Tokunaga et al. (2004) and drawn as in a paper; no handbook drawing is reproduced. Field names are exactly those of the benchmark’s truth key, which uses conventional labels for the TDB phases: (Ni) = FCC_A1, (Ti) = HCP_A3, (Ti) = BCC_A2, NiTi = NITI_B2, NiTi = NI3TI_D024, NiTi = NITI2, L = LIQUID. a, Lines traced automatically from the vector PDF (black) over the drawn lines (grey); closed traces smaller than 0.03 in mole fraction and 60 K, which are plot markers, are removed. b, Fields enclosed by the drawn lines (truth field map); a field name can label more than one enclosed region, so that regions and names are counted separately. Temperatures are shown up to 2,100 K; fields are clipped to this range before their points are placed. c, Field names; names of fields too narrow for a label are placed outside with a leader line, and names that would overlap are not printed. d, One point per field: the area centroid when it lies at least 0.03 (axis fraction) from every line and the frame, otherwise the interior point farthest from all field boundaries (the pole of inaccessibility, i.e. the centre of the largest inscribed circle), since the centroid of a curved or crescent-shaped field can lie outside the field or on a boundary; fields too thin for half that margin give no point. Dashed: the tie-line on the isotherm through the point of the largest two-phase field, between the two boundaries it meets; it is drawing-based and a weak constraint. Point coordinates and margins: scripts/data/fig_annotation_points.json.
Figure 7 (PDF p. 46). MLIPs: saturated on energies, discriminating on phase diagrams. a, Robust standard deviation (1.4826 the median absolute deviation) of the formation enthalpy minus experiment for the two DFT databases and 8 universal machine-learning interatomic potentials (MLIPs), all evaluated on the identical set of 1,138 records (455 compounds, 185 binary systems); MLIP energies are relaxations of the MP structures with the MP2020 correction (Methods). Bars are 95% confidence intervals from a cluster bootstrap that resamples chemical systems with all their records (1,000 replicates). Violet shades separate OMat24-trained models from models with an MP-compatible head. The green band is the interval of Materials Project; the blue band on the left is the experimental noise floor, the calibrated between-study of solution and reaction calorimetry for formation enthalpies (3.2 kJ (mol of atoms), from 80 pairs of independent measurements of 47 states; ESI). The MLIPs span = 8.6–9.7 kJ (mol of atoms) against 8.8 for MP, about 2.8 times the noise floor (PBE ceiling). The column gives with its cluster-bootstrap interval; every interval includes zero. b, Residual of GRACE-2L-OMAT against the residual of MP for the same records (hexagonal bins, logarithmic density; dashed line ; 25 of 1,138 pairs lie beyond 40 kJ (mol of atoms) and are not drawn). The strip lists the Pearson correlation of each MLIP’s residuals with those of MP ( = 0.961–0.997).
| Model | Recall [95% interval] | Precision | F1 |
| MACE-MPA-0 | 0.56 [0.42, 0.69] | 0.53 | 0.51 |
| ORB-v3 MPA | 0.54 [0.40, 0.67] | 0.50 | 0.42 |
| ORB-v3 OMat | 0.48 [0.36, 0.60] | 0.45 | 0.41 |
| MACE-OMAT-0 | 0.46 [0.32, 0.60] | 0.60 | 0.46 |
| SevenNet-Omni | 0.46 [0.34, 0.59] | 0.41 | 0.40 |
| MatterSim 5M | 0.44 [0.31, 0.58] | 0.41 | 0.36 |
| GRACE-2L-OMAT | 0.43 [0.30, 0.55] | 0.41 | 0.33 |
| CHGNet 0.3.0 | 0.35 [0.22, 0.49] | 0.44 | 0.34 |
| SGTE assessment | 0.72 [0.59, 0.84] | 0.65 | 0.66 |
| PhaseForge+ (in sample) | 0.76 [0.64, 0.87] | 0.65 | 0.69 |
Table S14. One-shot Cu–Zr field scores on the pinned evidence. Intervals are 95% point-bootstrap intervals; precision and F1 use the corresponding phase-prevalence weights.
c, Prevalence-weighted recall of the phase assemblages at the 28 assessed field points of Cu–Zr (the fraction of the phases present at those points that a description also predicts, each phase weighted by its prevalence; Methods and ESI) for one-shot descriptions built by PhaseForge from each MLIP alone, with no fit to experiment, against the of panel a (same axis). Vertical bars are point-bootstrap 95% intervals. GRACE-2L-OMAT reaches 0.43. The grey band spans the 9 later CALPHAD assessments in the collection (0.70–0.98), with a tick per assessment, and the dashed line is the SGTE assessment of Zeng et al. [40] (0.72). The star is the PhaseForge+ description tuned with ORCHID data (0.76), drawn at the of its starting MLIP; it is in-sample, because the tuning used Cu–Zr data that include these points.
Figure 9. Constructing a Cu–Zr description from MLIP energies and ORCHID data. a, Workflow. PhaseForge[17, 18, 19] builds a starting description (TDB, MLIP only) from GRACE-2L-OMAT energies[36] of the six compounds, of special quasirandom structures (SQS) generated with m_scraps[37, 38] instead of ATAT, and of the liquid from molecular dynamics, on SGTE unary data. ORCHID supplies phase equilibria, liquid activities and experimental formation enthalpies, each with its uncertainty, and its DFT-versus-experiment calibration gives the MLIP energies an uncertainty of 9.7 kJ (mol of atoms), so that experiment governs where it exists (bars drawn to scale against a typical experimental uncertainty); the fit used the data and the calibration of release 778cc6bfa of orchid-data, whose calibration has since changed slightly (release v2026.10 gives 9.3 kJ (mol of atoms)). PhaseForge+[20, 21] then tunes the interaction parameters of the liquid and the solid solutions (constant and linear in ), the compound energies and an entropy term for CuZr by gradient-based maximization of the likelihood, with analytic gradients (Jansson derivatives for the phase equilibria) from the gradient-capable ESPEI of Kunselman et al.[26, 27] (pull request #268, not yet merged). The badge counts the 13 invariant temperatures within twice their uncertainty (). b, Phase diagram tuned with ORCHID data (PhaseForge+, violet lines, fields coloured by phase), compared with the same model before tuning (PhaseForge from MLIP energies alone, dashed) and with the SGTE assessment of Zeng et al. [40]; CuZr–94Zen–LB, one of the 11 Cu–Zr assessments of the SGTE collection; grey). Sym-
bols are ORCHID phase-equilibrium points (filled, fitted; open, held out) and the points of Assemblages placed in the fields of the assessed diagram (orange; the 7 of the 28 released Cu–Zr points whose phases map to this assessment). Tuning brings 9 of the 13 invariant temperatures within twice their uncertainty (MLIP alone 0, SGTE 12), lowers the liquid-activity error from 5.1 to 1.7 kJ mol on held-out isotherms (SGTE 2.6) and the held-out phase-equilibrium error from 1.7 kJ mol to 0.11 (SGTE 0.12). It still places the congruent melting of CuZr 106 K too high and the eutectoid decomposition of CuZr 16 K too low, and shows a narrow spurious (Zr) field near between 1220 and 1290 K. Besides the points drawn, the fit used 177 further points inside the fields of the same assessed diagram, which are released with ORCHID-Assemblages as a dense grid; the model is post hoc relative to the pre-registered fit (4 of 13 invariants), which is reported in the demonstration repository.
Supporting records, audits and DFT calibration
The figures and calibration below support the main-paper evaluation and MLIP comparison (§ 4 of the main paper). The final six-scope re-audit, the measured precision of the ESPEI-ready subset and the preliminary human review are reported in the Validation section; none of them is used as a precision estimate for the whole numerical collection. The DFT calibration covers the default experimental subset and is distinct from the matched set used to compare eight MLIPs.
Representation of the source evidence
Figure S11 illustrates how a printed table row and sentence are represented as records, retaining the distinction between the reported values, their evidence classes and assumed conditions.
Audit snapshots and correction history
The audit rates refer to the specific snapshots identified in the caption and in the Validation section above. They distinguish wrong values or states from other metadata errors and are not independent human precision estimates.
Experimental calibration of formation energies
For the default subset of accepted experimental observations, the comparison with DFT gives mean absolute errors of 9.6 kJ (mol of atoms) for the Materials Project and 8.3 kJ (mol of atoms) for OQMD, with Spearman correlations of 0.86 and 0.91, respectively (Figure S13). The residual distributions have robust spreads of 8.6 and 7.5 kJ (mol of atoms) and tails containing both possible source errors and disagreements that remain unexplained after inspection. The comparison is independent of the extraction, but not wholly independent of experiment, since the databases use measured information in their energy corrections or elemental references [5, 35]. The resulting uncertainty therefore describes agreement under the stated matching and reference conventions, not an intrinsic error bound applicable to every DFT calculation.
The residual distributions provide a basis for placing computed energies and experimental evidence on a common uncertainty scale for fitting. The spread depends on the chemical stratum and the quality of the structure match, which are accounted for in the calibration. Within this calibration, the system under consideration is excluded when estimating the uncertainty assigned to its first-principles observations, thereby avoiding the use of the same residuals for calibration and subsequent weighting. Figure S14 connects these spreads to the resulting ESPEI weights, showing how a computed energy can remain useful without receiving the precision of a well-characterized experimental observation. The small systematic biases are recorded but not applied, and strata with insufficient data fall back to the source-level estimate. The calibrated weights thus express a documented comparison with experiment while leaving the choice of evidence and weighting policy open to the assessor.
Additional coverage and state-resolved benchmark results
The following figures document the source history, coverage and collection-wide comparison with calculated invariant temperatures.
Temperature-coverage classification
The temperature panels use the same 73,542-record Parquet collection as the manuscript inventory. An explicit scalar record has a finite positive temperature with a printed, row-temperature, stated-room-temperature or caption-ambient basis, or represents a scalar invariant or transition temperature. The latter temperatures are themselves reported properties and may have no separate temperature-basis field. Ranges, bounds, expression terms and assumed conditions are excluded from the scalar set. This gives 55,516 explicit scalar records, 4,856 assumed-temperature records, 1,582 range, bound or expression records, and 11,588 other or unresolved records. The temperature histogram uses 150 K bins; 23 scalar values above 4,500 K are omitted from the displayed range but retained in the counts. This partition describes metadata availability, not independent measurement counts, acceptance status or fitting eligibility. The complete predicate, grouped counts and input hash are retained in the figure-generation script and its JSON sidecar.
Field-wise and phase-wise failures
The failure maps use the cached results and pinned evidence described in § 22. No new equilibrium calculations or changes to the evidence snapshot enter these maps. For an assessed phase set and predicted set , the two field maps show and , respectively. The phase map gives the number of false negatives divided by the number of assessed field points containing that phase. Supports, in the plotted phase order, are 2, 3, 4, 5, 5, 4, 7, 3, 1, 11, 2, 2 and 5. They are overlapping phase-presence counts across 28 points, not independent sample sizes. All three unrepresented classes, , and , are retained in the benchmark, with hatching distinguishing the missing model vocabulary from errors among represented phases. C15b combines the and labels by structural identity under the benchmark’s physical mapping; groups the specified polymorph family. The existing strict mapping remains available in the benchmark outputs. Point coordinates below follow the temperature-then-composition order of the main figure; full-precision coordinates and observation identifiers are retained in figures-DD/data/mlip_field_coordinates.csv.
Predictions at withheld activity observations
The activity panels evaluate the original MLIP-only and final refined TDB files against all 12 activity observations in the demonstration’s holdout directory. The calculation uses the same ESPEI equilibrium residual implementation as the reported benchmark, including the allowed phase set, recorded conditions and pure-liquid reference states. The predicted activity is recovered from the residual through . It therefore represents the equilibrium prediction used in the reported score, rather than imposing a metastable liq-
uid where the model predicts another equilibrium. Re-evaluation reproduces the cached RMSE values of 5.09299 and 1.70697 kJ mol for the initial and final descriptions. No printed activity uncertainties are available, and no measurement error bars or model confidence intervals are inferred from the residual spread. The split withholds observations within the same system and can share a primary study with fitting data; it is not an independent source- or system-level validation. Input file hashes, observation identifiers and individual predictions accompany the generated figure.
Figure S16. Breadth and history of the literature evidence. a, Values by constituent element (logarithmic scale; grey indicates none). b, Values for the 977 binary systems, with elements ordered by atomic number. c, Publication years of the reporting papers, their extracted values and the oldest cited original for values with such a reference. Counts describe the released collection and its source history, not an exhaustive survey of all published thermodynamic measurements.
Vector (PDF)| Point | T (K) | x(Zr) | Observed assemblage |
| 1 | 922.6 | 0.2438 | Cu51Zr14 + Cu8Zr3 |
| 2 | 928.6 | 0.5393 | Cu10Zr7 + CuZr2 |
| 3 | 981.6 | 0.8333 | CuZr2 + (αZr) |
| 4 | 993.6 | 0.5148 | Cu10Zr7 + Cu5Zr8 |
| 5 | 1021.1 | 0.3425 | Cu10Zr7 + Cu8Zr3 |
| 6 | 1074.1 | 0.0896 | C15b + (Cu) |
| 7 | 1091.6 | 0.1988 | C15b + Cu51Zr14 |
| 8 | 1096.1 | 0.4552 | Cu10Zr7 + CuZr (B2) |
| 9 | 1101.1 | 0.5564 | Cu5Zr8 + CuZr (B2) |
| 10 | 1145.1 | 0.8193 | (βZr) + CuZr2 |
| 11 | 1175.6 | 0.3731 | Cu10Zr7 + Cu2Zr |
| 12 | 1176.1 | 0.6345 | Cu5Zr8 + CuZr2 |
| 13 | 1189.6 | 0.3029 | Cu2Zr + Cu8Zr3 |
| 14 | 1204.1 | 0.4802 | CuZr (B2) + Liquid |
Table a. Assessed field coordinates
| Point | T (K) | x(Zr) | Observed assemblage |
| 15 | 1209.1 | 0.5854 | Cu5Zr8 + Liquid |
| 16 | 1209.6 | 0.6410 | Cu5Zr8 + CuZr2 |
| 17 | 1210.6 | 0.5163 | CuZr (B2) + Liquid |
| 18 | 1225.6 | 0.3691 | Cu24Zr13 + Liquid |
| 19 | 1241.1 | 0.8133 | (βZr) + CuZr2 |
| 20 | 1244.1 | 0.3115 | Cu24Zr13 + Cu8Zr3 |
| 21 | 1246.1 | 0.6285 | CuZr2 + Liquid |
| 22 | 1286.1 | 0.3029 | Cu8Zr3 + Liquid |
| 23 | 1291.1 | 0.1367 | C15b + Liquid |
| 24 | 1303.6 | 0.0255 | (Cu) + Liquid |
| 25 | 1338.6 | 0.2534 | Cu51Zr14 + Liquid |
| 26 | 1340.1 | 0.1757 | Cu51Zr14 + Liquid |
| 27 | 1492.1 | 0.9841 | (βZr) |
| 28 | 1733.6 | 0.4943 | Liquid |
Table b. Assessed field coordinates
Figure S17. Coordinates and assessed assemblages of the field benchmark. Point indices match the main-paper failure maps. Temperatures and compositions are rounded only for this display; evaluation uses the full-precision coordinates. Phase labels follow the physical-phase mapping specified above.
Automated source-verification pilot
Protocol for automated source verification of a frozen literature snapshot
The preceding audits identify defects within their stated scopes, while a source review of complete records must also check whether a correctly transcribed value has been associated with the correct thermodynamic state. To address this distinction, we froze a separate working-tree export on 27 October 2026 and specified an automated audit of the accepted literature records. This snapshot contains 73,542 records in total, of which 65,310 are accepted records in the three source kinds covered by the audit. These counts describe the audit snapshot and are distinct from the manuscript’s earlier release population. The source files, sampling code and record identifiers are retained with their hashes so that the audit can be repeated against the same evidence. The sampling protocol was fixed before source judgments were obtained.
Within this population, we drew 300 records without replacement using seed 20261027, stratifying by evidence class and source kind. The 11 occupied strata received a proportional allocation with a minimum of 15 records, capped at the size of a stratum, which includes all three records in the smallest stratum. The resulting sample contains records from 229 papers. We retain the inclusion probability and inverse-probability weight of every selected record, and inaccessible or ambiguous source pages remain unresolved rather than being replaced. Records from the same paper can describe related observations, so the number of selected records does not represent a count of independent experiments.
For source verification, the protocol assigns an initial reader to reconstruct the observation from its original page and surrounding context without seeing the stored extraction. A separate verifier checks the frozen record against the source before either judgment is disclosed to the other reader. Adjudication then examines the original evidence and both judgments, including the association of the property and value with its units, composition, phases, temperature, pressure and provenance. It also distinguishes printed conditions from assumptions and measurement conditions from processing history. Deterministic thermodynamic checks provide additional evidence about consistency, while agreement among readers alone is insufficient to establish that an extraction is supported. The model identities and original judgments are retained, and any use of the same model family must accompany the reported outcomes.
For the sampled records, a demonstrated defect in a required field constitutes failure, full source support of the applicable fields constitutes support, and the remaining cases are unresolved. For a completed audit, population fractions would be estimated with the sampling weights, retaining unresolved records in the denominator. If the weighted supported and unresolved fractions are denoted by and , respectively, the corresponding bounds on correctness are , conditional on the validity of the audit judgments. Finite-population uncertainty is calculated within the sampling strata, with conservative intervals for sparse outcomes. This uncertainty does not account for shared mistakes by the auditors. Separate known-answer challenges are specified to assess error detection and false alarms, and a future page-level completeness study would be required to estimate missed observations. The automated audit is therefore reported separately from the preliminary human review and from any subsequent corrections to the release.
Pilot outcomes and limitations of automated source verification
The protocol above provides a means of checking the association of a numerical value with its source context, although its use also requires an assessment of the auditors themselves. Verified source documents were recovered for all 300 sampled records. The initial reconstruction used GPT-5.5 (gpt-5.5-2026-04-23), while a separate GPT-5.4 call (gpt-5.4-2026-03-05) checked the stored record before GPT-5.5 adjudicated the two outputs against the original pages. These are separate review passes within one model family. The intended second provider could not be used because its account lacked credit, and the subsequent run stopped at the OpenAI project’s enforced spending limit. Thus, only 46 records completed all three stages, with initial labels of 30 supported, 14 erroneous and two unresolved; 254 records remain pending. Because completion is neither random nor independent of record complexity, these outcomes do not provide an estimate of population accuracy.
To examine the distinction between numerical transcription and the thermodynamic state to which it belongs, we also drew 18 records from the full 300-record sample using seed 20261028 for an additional source review by three agents. The numerical values and their direct unit conversions were supported in all 18 records, but six contained errors in state association, phase labels or provenance (Figure S18). These included a sample identifier interpreted as a phase, the term “congruent” retained as a phase name, a binary observation assigned the chemistry of a broader table, a calculated invariant classified as DFT, and an earlier cited measurement marked as the reporting paper’s own work. Another record assigned a composition reported at 878 K to an equilibrium state at 910.7 K after losing the qualifying footnote. The remaining 12 records were consistent with the source within their explicitly retained assumptions and unknowns. These descriptive counts do not establish the completeness of the states or the correctness of an inferred molar basis, and they are not a population precision estimate.

Figure S18. Automated source audit and additional agent review. The initial 300-record audit remains incomplete, so panel a shows unweighted execution counts rather than extraction accuracy. Panel b distinguishes numerical transcription from the asserted state and provenance in a separate random subset of 18 records. All 18 numerical values were supported, while six records contained a demonstrated state or provenance defect. Transparent unknowns and explicitly labeled assumptions were retained as limitations rather than counted as errors. These are AI-assisted source judgments, not independent human reference labels; no population precision is inferred from either panel.
The additional source reviews also expose limitations of the automated judgments. In two targeted checks, an initial error label was overturned because the judge had selected a different row sharing the same temperature or had treated a valid lower-value-plus-upper-bound representation as a lost range. In a third check, the source confirmed that a Sc–Cr calculation had been assigned to a six-element system and incorrectly attributed to an experimental reference. Likewise, a synthetic challenge containing six correct records and six deliberate corruptions detected all six introduced defects, but one correct record was flagged because a general pure-solid reference was judged insufficiently specific for a Cu activity. This challenge is small and deliberately simple; its outcome cannot calibrate error rates on the historical literature. The original labels, subsequent reviews and supporting evidence are retained separately so that disagreements remain inspectable.
These results support the use of multiple agents to identify defects and organize source review, while also showing why neither agreement between readers nor a correct numerical value is sufficient to establish extraction reliability. The 18-record review was additional AI assessment rather than human validation; one reviewer had already seen the initial judgment for one of its records. This automated evaluation of the 300-record sample therefore remained incomplete; the preliminary human review of the same frozen sample and the later measurement of the fitting subset are reported separately in this section. The frozen export has SHA256 8de3831eed304f1f845e368c4f1cb0d28050bb8b372aea79bc40552c811ace7f. Its 65,310 accepted records must not be conflated with the earlier manuscript inventory of 68,266 or with later revisions of the live release. No corrections were applied to the frozen data during this audit.
The additional phase-recovery review in main-paper Figure 4b concerns a proposed correction batch rather than the complete numerical collection. A sample of 100 proposals was drawn with seed 20261027 from 1,268 phase-only proposals, spanning 36 page groups and one continuation page. A reviewer in a separate assistant context completed all page readings and froze the answers before opening the proposal key. The comparison gives 84 exact unordered-label matches, 15 notation-only differences and one unresolved phase-identity case. Wilson 95% intervals are 75.58–89.90% for exact agreement and 94.55–99.82% for phase identity. The latter retains the unresolved record in the denominator.
The unresolved Ni–Mo row prints the same composition label for bcc-?SQSI6 and fcc-?SQSI6 prototypes. It was withheld because a phase-only correction must preserve that distinction. The remaining notation differences concern parentheses, solid-state suffixes and liquid abbreviations. The review establishes agreement for the proposed phase labels under these comparison rules, while temperature, composition, property interpretation and units require their own checks. It is AI-assisted review with shared-model and source dependence, and possible prior exposure during earlier source inspections. The intervals assume independent field or record outcomes and do not describe source-cluster uncertainty. No aggregate human precision estimate is inferred.
Diagram-extraction demonstration and phase-recovery review
The synthetic Ni–Ti example in main-paper Fig. 3 illustrates the extraction procedure. It uses a calculated diagram with known phase identities, so that tracing and phase assignment can be examined against a defined reference. …
Scope of this document
This SI describes how the ORCHID evidence is organized, retrieved and selected for analysis. The dataset inventory is followed by access routes, evidence-selection rules and machine-learning splits, with licensing and release information at the end. SI 1 documents extraction, calibration and validation. Counts use the same snapshot as the main paper, and features awaiting release are identified explicitly.
A guide to the ORCHID datasets
The evidence assembled and checked through the procedures described in SI 1 is organized into eight datasets, which are released together as version v2026.10. Within this common release, the datasets are deposited separately so that users can retrieve and cite them independently. An umbrella record (10.5281/zenodo.23736065) links the eight individual data deposits on Zenodo, giving 9 records including the umbrella and 10 when the runnable examples deposit (10.5281/zenodo.23736043) is also counted. Of these deposits, seven contain the Values (10.5281/zenodo.23735569), Diagrams (10.5281/zenodo.23735859), Hulls (10.5281/zenodo.23735891), Parity (10.5281/zenodo.23735943), Registry (10.5281/zenodo.23735970), Assemblages (10.5281/zenodo.23735996) and Bibliography (10.5281/zenodo.23736021) datasets. The eighth deposit contains the ESPEI datasets derived from Values (10.5281/zenodo.23735788), which are deposited separately so that users can retrieve and use them independently. Within each deposit, the files are packaged in a single archive that preserves their repository paths and includes a manifest of SHA-256 checksums. In addition to the archived deposits, these seven datasets and the ESPEI export are available at https://data.orchid-project.org, where they are served from the orchid-data repository[39]. The eighth named dataset, Bench, contains the synthetic extraction benchmark and is maintained separately in orchid-bench, alongside its generator and scorer, with which it is archived in the repository’s software release (10.5281/zenodo.23748414). The deposits are connected through shared phase identities, source relations and descriptions of thermodynamic states. As summarized in the Appendix of the main paper (Fig. 12), Values and Assemblages provide observations of thermodynamic states, while Diagrams, Hulls and Parity supply calculated references against which those observations can be interpreted. The connection among these products is maintained through Registry and Bibliography, which preserve phase identities and source relations, and Bench provides documents with known answers for evaluating extraction. Their shared identifiers and state representation allow the same evidence to support CALPHAD assessment, finite-temperature benchmarks, machine learning and agent-based analysis. Table R1 gives the corresponding file-level inventory, while Fig. 2 shows the coverage of the literature values on which the release is built.
| Dataset | Files | Content |
| 1 ORCHID-Values (Values) | literature/ (index.json, systems/<SYS>.json, orchid-literature.jsonl.gz, .csv.gz, papers.json, contexts.json, SCHEMA.md, UNCERTAINTY_CALIBRATION.*); Parquet literature_values, papers, experimental_context; espei/ (own record) | 73,542 values; 1,474 systems; 1,321 reporting papers; 4,882 papers and cited works; 1,504 context items; 8,589 ESPEI datasets, of which 6,453 hold literature and DFT values (12,513 values, 690 systems) |
| 2 ORCHID-Diagrams (Diagrams) | phase-diagrams/index.json, phase-diagrams/sgte/*.json; Parquet phase_diagram_index, phase_diagram_invariants | 2,774 TDB files, 2,722 calculated (1,234 systems); 15,336 distinct invariants |
| 3 ORCHID-Hulls (Hulls) | hulls/dft-mp/, hulls/dft-oqmd/, hulls/calphad-sgte/; Parquet hull_vertices | MP: 3,828 systems, 12,829 vertices; OQMD: 3,916 systems, 15,314 vertices; SGTE at 298.15 K: 2,695 assessments |
| 4 ORCHID-Parity (Parity) | literature/dft_parity.json, kim2017_crosscheck.*; Parquet dft_parity | 7,646 pairs (MP 3,679; OQMD 3,967) from 4,067 records |
| 5 ORCHID-Registry (Registry) | phases/ (registry.json, systems/<SYS>.json, registry.jsonld, context.jsonld, registry.csv, registry.parquet, structure_types.*, review_queue.json, CONFLICTS.md) | 10,430 phases in 1,241 binary systems; 43,269 names; 22,556 TDB models; 1,563 structure types |
| 6 ORCHID-Assemblages (Assemblages) | assessed-phase-assemblages/ (records.jsonl.gz, .csv, .parquet, exclusions.*, espei-zpf/, espei-zpf-with-tie-lines/, summary.json) | 23,129 points from 2,178 diagrams; 1,068 ESPEI ZPF datasets (9,877 points) |
| 7 ORCHID-Bibliography (Bibliography) | bibliography/ (papers.jsonl, edges.jsonl, .csv, .parquet, orchid-bibliography.bib, index.json, SCHEMA.md) | 9,570 papers (7,813 with a DOI); 60,235 typed relations |
| 8 ORCHID-Bench (Bench) | orchid-bench: docs/<doc>/ (four renderings), truth/<doc>.json, outputs/, results/ | 20 synthetic papers, 86 pages, four renderings each |
Table R1. File-level overview of the eight ORCHID datasets (release v2026.10). All datasets except Bench are in the orchid-data repository and on Zenodo (the ESPEI datasets of Values as a record of their own); Bench is in the orchid-bench repository.
Within these datasets, the JSON files provide the reference representation of the records, from which the Parquet tables are derived. The Parquet tables add only a split column to this representation, with its value determined from the paper identifier according to the hash rule in Supplementary Note 4 in SI 1. To support the use of these files, the release includes checksum manifests, citation files and field descriptions (Table R1), so that users can verify, cite and interpret the data without consulting the code that produced them.
Literature values (Values)
The connection between the literature evidence and its representation in ORCHID can be seen in Figure S11 in SI 1, where released records are shown alongside the printed table row and sentence from which they were extracted. These records are grouped according to the chemical system to which they refer and stored in systems/<SYS>.json, where <SYS> lists the elements in alphabetical order, in upper-case letters and joined by hyphens, as in AL-NI. Within this file, the records are retained together with the associated papers, contexts, tables or passages and links to TDB files. To allow analyses across chemical systems, the complete collection is also provided in a single JSON dump (orchid-literature.jsonl.gz) and in CSV form, in which nested fields are encoded as JSON strings.
The literature dataset contains 73,542 values, of which 66,551 are accepted, and cover 96 property codes organized into 11 property groups (Table R3, Figure S1 in SI 1). Most of these values concern binary systems, with 58,050 values in 977 systems, although the collection also includes 11,853 values in 365 ternary systems, 2,770 in 73 unary systems and 869 in 59 systems of higher order. Across these systems, the collection covers 79 elements (main-paper Fig. 2a,b). The values collected across these systems were extracted from invariant-reaction tables, which contributed 26,693 values, from general tables, which contributed 41,106, and from running text, which contributed 5,743. The fields used to represent these values are listed in Table S2 in SI 1 and the Parquet columns are documented in SCHEMA.md, while Table S1 in SI 1 gives the most frequent property codes together with their units and ESPEI outputs.

Figure S1. Half of the literature values are temperatures of invariant reactions and phase transitions, and nine in ten were read from tables. a, Values by property group, stacked by evidence class. b, Values by record kind (general tables, invariant-reaction tables, running text). c, The 15 most frequent property codes. Colours: evidence class, as in every figure (blue experimental, vermilion model, green DFT, grey unspecified). values.
| Code | Quantity | Unit | ESPEI output | Values |
| INVARIANT_TEMPERATURE | invariant-reaction temperature | K | ZPF | 26,411 |
| TRANSITION_TEMPERATURE | transition temperature | K | ZPF | 10,430 |
| HM_FORM | enthalpy of formation | kJ/mol-atom | HM_FORM, HMR | 9,137 |
| PHASE_COMPOSITION | phase composition at equilibrium | – | ZPF | 4,841 |
| LATTICE_A | lattice parameter | Å | – | 4,226 |
| LATTICE_C | lattice parameter | Å | – | 2,781 |
| CPM | molar heat capacity | J/mol (basis unstated)/K | CPM | 2,322 |
| HM_MIX | enthalpy of mixing | kJ/mol (basis unstated) | HMR | 1,345 |
| LATTICE_B | lattice parameter | Å | – | 1,322 |
| SM | molar entropy | J/mol-atom/K | SM, SMR | 1,053 |
| PHASE_ASSEMBLAGE | phases present at a state | – | – | 939 |
| HEAT_CONTENT | heat content | J/g at. | – | 889 |
| SM_FORM | entropy of formation | J/mol-atom/K | SM_FORM, SMR | 825 |
| PARTIAL_ENTHALPY | partial molar enthalpy | kJ/mol (basis unstated) | – | 820 |
| GM_FORM | Gibbs energy of formation | kJ/mol-atom | GMR | 792 |
| GIBBS_ENERGY | Gibbs energy | J/mol | – | 595 |
| GIBBS_ENERGY_EXPRESSION_TERM | model expression | J/mol | – | 555 |
| EMF | electromotive force | mV | – | 429 |
| ENTROPY | heat capacity / entropy / enthalpy | J/(mol K) | – | 327 |
| GM_MIX | Gibbs energy of mixing | kJ/mol (basis unstated) | GMR | 256 |
Table S1. Table S1: Property vocabulary: the 20 most frequent codes. Unit: the most frequent unit of the values of each code after normalization, where kJ/mol (basis unstated) marks values whose per-atom or per-formula basis is not printed and which are therefore not converted. Lattice lengths are in Å where a unit was printed, and the 639 lengths printed without a unit are left unconverted; other units are as read. A coefficient of a printed expression in , such as the or of or a term of a polynomial for , is stored under a code that ends in _EXPRESSION_TERM (838 records, 555 of them for the Gibbs energy), together with the printed expression, the term and its range of temperature, and is not exported to ESPEI. ESPEI output: the names of the ESPEI datasets that carry values of the code (ZPF: phase-equilibrium data); –: not exported. All 96 codes are listed in literature/SCHEMA.md.
| Field | Meaning |
id, pid | stable ORCHID id; persistent locator (paper/page/table or passage/row) |
system, elements | chemical system |
source_system | system of the source table or paper, which differs from system when the state belongs to a lower-order subsystem |
record_kind | invariant_table, general_table or running_text |
property, property_group | controlled quantity code (e.g. INVARIANT_TEMPERATURE, HM_FORM, CPM, LATTICE_A) and its group |
value, unit, value_printed, unit_printed | normalized and printed value and unit |
value_min, value_max, value_expression | printed range; printed expression (e.g. model parameters) |
uncertainty_value, uncertainty_basis | source-reported uncertainty in the record unit (printed, printed_range), or an explicitly identified placeholder (stated_default, reading_resolution); scatter and composition-offset estimates are stored separately in the budget |
sigma_printed, sigma_reading, sigma_method, sigma_composition, sigma_scatter, sigma_conversion, sigma_effective, sigma_method_basis, uncertainty_budget | uncertainty budget: reported and estimated components kept separate; reported coverage and its conversion to a convention are documented; sigma_effective determines the fitting weight (Methods, uncertainty budget) |
uncertainty_budget, sigma_x_*, sigma_T_* | uncertainty budget with its components kept separate; uncertainties of composition and temperature |
temperature_K, temperature_basis, pressure_Pa, pressure_basis | state variables and how they were established |
phases, model_phases | printed phase labels; mapped CALPHAD model phases |
compositions | list of {phase, element, value, basis, source}; mole fractions |
overall_composition | composition of the alloy or sample, where given (mole fraction) |
reaction, reaction_type, transition_kind | reaction and its controlled class; transition_kind_printed keeps the label |
reference_state_json | reference state of the quantity, where printed |
structure_* | Pearson symbol, prototype, Strukturbericht designation and space group, where printed |
flag_state_fully_specified, specification_gaps | whether the record states every variable that its equilibrium state requires, and which are missing |
evidence_class, origin, origin_measured_in | evidence class, the rule that decided it, and the work in which the value was obtained |
accepted, reader_tier, readers, reader_agreement | acceptance, which readings agreed, reader identities, and the number of independent visual readers (tables) |
flag-*, review_flags | quality flags (SI 2, section “Choosing what to trust”) |
method, conditions_kind | measurement method when printed with the value; where the conditions apply |
paper_id, paper_doi, paper_citation, cited_dois | source paper and cited works |
quote, quote_page, artifact_caption, table_row | verbatim sentence (running text) or table locator |
model_tdb, model_tdb_sha256 | linked TDB file of a model value (path, SHA-256 and catalog entry), only where the link is unambiguous (12,833 values) |
LATTICE_* units | lengths in Å for 7,690 of the 8,329 lattice constants; the 639 without a printed unit are unconverted |
unit = kJ/mol (basis unstated) | formation values whose per-atom basis could not be established (2,255 values); flag_basis_inferred_from_magnitude marks the 2,117 taken as per atom from their magnitude alone |
plausibility_*, calphad_check | comparison with a calculated phase diagram, where one exists |
flag_espi_ready, flag_espi_certified | usable in ESPI; usable and reproduced by the CALPHAD calculation |
gate_failures, corrections_json | literature-level gates: demotions and flags with reasons; corrections with the stored value |
tool_release, split, fold_iso, fold_lpo | extraction tool release; recommended machine-learning split (by paper); five folds by chemical system and by paper |
Table S2. Main fields of a literature value. Parquet column names (the JSON nests some of them); the release has 165 columns in all, documented in literature/SCHEMA.md.
| Property group | Experimental | Model | DFT | Unspecified | Total |
| invariant reaction | 12,066 | 12,822 | 27 | 1,496 | 26,411 |
| formation | 2,930 | 5,007 | 2,438 | 458 | 10,833 |
| phase transition | 7,294 | 2,844 | 6 | 310 | 10,454 |
| crystal structure | 3,301 | 1,330 | 3,065 | 2,161 | 9,857 |
| phase equilibrium | 4,785 | 706 | 0 | 289 | 5,780 |
| heat capacity / entropy / enthalpy | 2,599 | 2,014 | 139 | 161 | 4,913 |
| mixing / partial | 1,576 | 499 | 431 | 20 | 2,526 |
| activity / EMF / Gibbs energy | 1,062 | 614 | 7 | 21 | 1,704 |
| model expression | 604 | 227 | 4 | 19 | 854 |
| other property | 69 | 86 | 33 | 16 | 204 |
| transformation enthalpy | 2 | 4 | 0 | 0 | 6 |
| total | 36,288 | 26,153 | 6,150 | 4,951 | 73,542 |
| accepted | 31,541 | 23,988 | 6,133 | 4,889 | 66,551 |
Table R3. Literature values by property group and evidence class (release v2026.10).
To interpret these values, it is also necessary to know how the conditions to which they refer were established. The records therefore distinguish conditions printed in the source from those assumed or inferred, as in the assumption of a pressure of one atmosphere for 73,537 of the 73,542 values and the inference of a per-atom basis solely from the magnitude of a formation value for 2,117 values. Alongside this information, the uncertainty is retained as a budget in which the contribution printed in the source is kept separate from the systematic contribution of the measurement method. The contributions retained in this budget are combined according to the uncertainty model described in SI 1 to obtain the effective uncertainty. Alongside the uncertainty budget, the records retain infor-

Figure R1. The ORCHID datasets are connected through a common representation of thermodynamic state. The eight products retain different forms of evidence and calculated reference data, while shared record, phase and source identifiers connect them to the state that they describe. The same organization supports thermodynamic assessment, finite-temperature benchmarking, machine learning and inspection by people or agents; Table R1 gives the corresponding file-level inventory. The diagram contains the headings THE ORCHID DATA LANDSCAPE, DATA PRODUCTS, THERMODYNAMIC STATE, and WHAT THE DATA ENABLE. Data products: Values — 73,542 properties + conditions; Assemblages — 23,129 finite-T phase states; Diagrams — 2,722 calculated equilibria; Hulls — 3,828 MP systems, 0 K stability landscapes; Parity — 7,646 calculation ↔ experiment; Registry — 10,430 phase identities + aliases; Bibliography — 9,570 papers + typed relations; Bench — 20 synthetic documents + exact truth. Thermodynamic state: the common scientific unit; chemistry — system + composition; conditions — T, P and ranges; constitution — phases + identities; observation — property + value; status — evidence + uncertainty; source — paper + location; shared identifiers connect values, phases, models and sources; JSON • CSV • Parquet • ESPEI; versioned + citable + machine-readable. What the data enable: CALPHAD + ESPEI — fit and refine models; MLIP benchmarks — test finite-T stability; Machine learning — paper- and system-held-out splits; Agents + experts — inspect evidence and provenance; Web • API/MCP • Zenodo — human and programmatic access. Scientific roles are summarized here; Table R1 gives the file inventory, schemas and release locations.
mation on ESPEI readiness and the comparison with the CALPHAD model to support subsequent fitting and evaluation, together with the chemical system, source paper and fold assignments needed for machine learning. The names and meanings of these fields are given in Table S2 in SI 1, together with the units used for lattice lengths and the links connecting model values to the corresponding TDB files in the ORCHID catalog.
From these records, ESPEI datasets are derived and made available in the folder espei/ (Table R2). These datasets follow the ESPEI 0.9.1 format and retain the source in the reference field, together with a bibtex key that links to its entry in references.bib. Alongside this source information, the datasets retain the ORCHID observation identifiers of the values, the agreement between readers and the caveats, together with an orchid_uncertainty block that records the uncertainty budget of each value and the weight used. Before a dataset is written, the values it contains are checked against those in the published literature release. For an observation to be retained, it must be published and accepted in the literature release and belong to the evidence class of the folder, while its numerical value, temperature and phase compositions must match those in the release. Values that do not meet these requirements are excluded, and the reason for excluding them is recorded. The results of these checks are recorded in validation.json and VALIDATION.md, which identify the dropped values and excluded datasets and report the outcome of check_dataset and load_datasets in ESPEI 0.9.1, while the datasets themselves are listed in index.json and index.csv. The basis of the uncertainty assigned to the different folders and outputs is documented in UNCERTAINTY_SUMMARY.md, while dft/CALIBRATION.json and .md describe the DFT calibration and report the leave-system-out for the individual systems. The datasets also retain these qualifications through their tags, with 9 carrying single_reader and 558 carrying no_printed_uncertainty.
| Folder | Content | Datasets | Values | Systems |
| experimental/ | experimental thermochemical and phase-composition (ZPF) data | 1,118 | 2,387 | 185 |
| invariants/ | ZPF data from experimental invariant-reaction tables | 1,546 | 5,061 | 464 |
| model/ | model-calculated values (TDB named where known) | 504 | 782 | 82 |
| dft/literature/ | DFT formation enthalpies printed in papers | 783 | 1,781 | 91 |
| dft/mp/ | Materials Project on-hull formation energies | 1,251 | 1,251 | 381 |
| dft/oqmd/ | OQMD on-hull formation energies | 1,251 | 1,251 | 381 |
Table R2. ESPEI datasets in espei/. Folders are never mixed: each holds one evidence class or one DFT source.
Calculated phase diagrams (Diagrams)
The reported evidence preserved in the literature records can be considered alongside the calculated phase diagrams, which describe the predictions of the assessments. The association of these predictions with their underlying models is preserved by organizing the diagrams according to the TDB file used in the calculation, rather than the chemical system. This association is recorded in phase-diagrams/sgte/<TDB>.json, where the TDB file is identified by its name and SHA-256 checksum, together with the components, composition variable (x_component) and temperature range of the calculation. Alongside this description of the calculation, the file records the phases and their boundaries (regions), together with the invariants and their temperatures and phase compositions, and the grid of stable phases. To allow these diagram files to be located by chemical system, phase-diagrams/index.json lists them together with the citation and DOI of the corresponding assessment. Phase diagrams were calculated from 2,722 of the 2,774 TDB files, covering 1,234 chemical systems, while the remaining 52 files did not yield a calculated diagram and their status is recorded.
In calculating these diagrams, the mapper can reach the same invariant from different starting points, so that the 24,849 detections it returned were consolidated into 15,336 distinct invariants. In this consolidation, two detections are treated as the same invariant when they involve the same phases, their temperatures differ by no more than 0.1 K and their phase compositions agree to within 0.01. The distinct invariants obtained through this consolidation are stored in the Parquet tables alongside rows for the diagrams, with n_mapper_hits recording how many times an invariant was detected. The JSON files, in turn, retain all detections returned by the mapper, including those consolidated into the same invariant. The coverage of these calculated diagrams overlaps with that of the literature values, with a calculated diagram available for 851 of the 977 binary systems represented in the literature collection. For these systems, the availability of calculated diagrams allows a direct comparison between the 50,038 literature values and the predictions of the corresponding assessments. An example of this comparison is shown for Cu–Zr in Figure S7 in SI 1, where the literature values are considered alongside the predictions of the assessment of Zeng et al. [40][40], one of the 11 Cu–Zr assessments in the SGTE collection.
Figure S7. Most invariant compositions printed in the literature fall on the invariant lines of an assessed Cu–Zr diagram, and the largest exceptions are reactions that this assessment places differently. Phase fields are coloured by assemblage as in Figure 9b (single-phase fields in the colour of the phase, two-phase fields as light tints), with dark-grey phase boundaries, vertical lines for the stoichiometric compounds and black invariant equilibria, calculated with pycalphad 0.11.2 from the Cu–Zr assessment of Zeng et al. [40], one of the 11 Cu–Zr assessments in the SGTE binary collection and the reference assessment of the Cu–Zr example in the main text. Symbols: printed phase compositions of accepted literature invariant reactions, by evidence class (number of points in the legend); compositions filled from stoichiometry are not shown. The points near 1,375–1,410 K at record the congruent melting of CuZr, printed above the calculated liquidus, and those near 885 K record the decomposition of CuZr, which the calculated diagram does not show.
Convex hulls (Hulls)
The calculated phase diagrams are complemented by DFT convex hulls, which identify the compounds predicted to be stable at 0 K. These hull data are organized by binary system, so that the data for a given system within a layer are stored in a separate JSON document. Within this organization, the Materials Project layer covers 3,828 binary systems involving 88 elements, of which 2,084 systems contain a stable intermediate compound. For 3,827 of these systems, the stability flags in this layer agree with those reported by the Materials Project. The corresponding layer derived from OQMD covers 3,916 binary systems involving 89 elements, with a stable intermediate compound present in 2,256 of these systems. To allow these DFT predictions to be compared with those of the assessments, a third layer was calculated from the SGTE models using pycalphad[9] at 298.15 K, covering 2,695 assessments of 1,223 systems.
To make these comparisons, it is necessary to retain the reference conventions used by the source databases. The catalog for a given layer therefore records the source snapshot and tolerances, together with the elemental reference and the energy corrections applied by the provider. The elemental reference recorded in this catalog is dft_elemental_ground_state_0K for the Materials Project and dft_fitted_elemental_reference_0K for OQMD.
The stable phases are identified by the vertices of these hulls, while nearby entries are also retained in the released layers to allow comparison. These nearby entries are retained when they lie within 0.05 eV atom of the hull, together with the hull vertices themselves. The hull vertices are collected in the Parquet table hull_vertices, which contains 28,143 vertices, including elemental endpoints, of which 12,829 come from the Materials Project and 15,314 from OQMD.
DFT versus experiment (Parity)
To determine how much weight these DFT energies should receive in a fit, formation enthalpies reported in the literature are paired with entries for the same compound in the Materials Project and OQMD. In making this match, when the paper identifies the crystal structure of the compound, only the DFT entry for that structure is considered. For these matched measurements, a departure from the stoichiometry of the compound is accounted for through an additional contribution to the uncertainty associated with composition. Among the 9,137 formation-enthalpy records considered for matching, 4,067 matched at least one DFT entry, comprising 1,364 experimental, 2,543 model and 160 unspecified values. For records that could not be matched, the principal reasons for exclusion were an unstated per-atom basis (2,154), a phase label without a multi-element formula (436), or a value that was itself obtained from DFT (1,545).
A match alone does not establish that a pair is suitable for calibrating the DFT uncertainty. Calibration therefore uses a default subset of accepted experimental values that are not unsigned in print and have no flag of a suspect magnitude. For a pair to enter this subset, the measured composition must lie within 0.02 of that of the compound, and the structure identified in the record must be present in the DFT database. The reference state must also be the solid standard reference of 0 K DFT, excluding pairs with a liquid or gas elemental reference or a reference element outside the compound. Finally, the record must have no literature gate failure. These restrictions leave 1,070 Materials Project and 1,107 OQMD pairs.
To account for differences within these populations, the pairs are stratified by DFT match quality and chemistry (all-metal, p-block or anion element, magnetic 3d metal). Within a stratum, a median bias and robust are calculated, with 95% intervals obtained by resampling whole chemical systems. The applied to a system is recomputed without the pairs from that system, while strata with fewer than 30 pairs use the source-level value (Table S5 in SI 1). The points, matching rules, exclusions and statistics are recorded in dft_parity.json, and the calibration in espei/dft/CALIBRATION.json. For this comparison, energies are converted using 1 eV atom = 96.485 kJ (mol of atoms), and residuals are defined as DFT minus experiment.
| Source | Chemistry | Match | 95% CI | Bias | Weight | ||
| MP | all-metal | on hull, single | 216 | 9.2 | 6.7–11.1 | 5.0 | 0.054 |
| MP | all-metal | polymorphs | 78 | 6.0 | 3.6–10.8 | 4.8 | 0.083 |
| MP | all-metal | above hull | 113 | 7.3 | 4.6–9.6 | 6.3 | 0.069 |
| MP | magnetic 3d | on hull, single | 68 | 5.2 | 3.5–7.6 | 1.2 | 0.096 |
| MP | magnetic 3d | polymorphs | 65 | 5.0 | 2.4–10.7 | 0.1 | 0.100 |
| MP | magnetic 3d | above hull | 32 | 6.0 | 1.8–8.8 | 5.3 | 0.083 |
| MP | p-block/anion | on hull, single | 115 | 6.5 | 3.7–16.3 | 1.6 | 0.077 |
| MP | p-block/anion | polymorphs | 317 | 10.9 | 6.7–17.0 | −2.1 | 0.046 |
| MP | p-block/anion | above hull | 66 | 15.1 | 8.3–23.3 | 7.8 | 0.033 |
| OQMD | all-metal | on hull, single | 36 | 9.1 | 4.9–12.3 | 5.2 | 0.055 |
| OQMD | all-metal | polymorphs | 277 | 7.1 | 4.9–9.6 | 1.9 | 0.071 |
| OQMD | all-metal | above hull | 102 | 6.9 | 4.6–9.6 | 4.9 | 0.073 |
| OQMD | magnetic 3d | on hull, single | 15 | 3.0 | 0.8–7.1 | 1.0 | 0.165 |
| OQMD | magnetic 3d | polymorphs | 110 | 4.8 | 3.6–6.9 | 0.1 | 0.104 |
| OQMD | magnetic 3d | above hull | 60 | 7.5 | 2.6–17.2 | 2.5 | 0.067 |
| OQMD | p-block/anion | on hull, single | 26 | 3.3 | 1.7–17.0 | 2.6 | 0.151 |
| OQMD | p-block/anion | polymorphs | 410 | 8.1 | 6.3–10.9 | 0.7 | 0.062 |
| OQMD | p-block/anion | above hull | 71 | 13.7 | 5.7–15.6 | 9.5 | 0.037 |
| literature DFT | all-metal | – | 230 | 8.4 | 5.9–11.4 | 2.0 | 0.060 |
| literature DFT | magnetic 3d | – | 105 | 3.9 | 2.0–5.2 | 0.6 | 0.127 |
| literature DFT | p-block/anion | – | 123 | 10.0 | 6.0–14.8 | 1.1 | 0.050 |
Table S5. DFT calibration by stratum. Residuals are DFT minus experiment in kJ (mol of atoms) over the default subset, with literature DFT values compared with the median experimental value of the same compound. : robust (1.4826 times the median absolute deviation), with its 95% interval from 1,000 bootstrap resamples of whole chemical systems. Match: the hull status of the paired DFT entry, on the hull and the only entry at its composition (on hull, single), one of several entries at that composition (polymorphs), or above the hull. Bias: median residual, recorded but not applied. Weight: the ESPEI weight, 500 J mol/, for the full stratum; the released datasets use the leave-system-out . Source: espei/dft/CALIBRATION.json.
Phase registry (Registry)
Comparison across sources depends on recognizing when different names refer to the same phase. The registry assigns the identity phase:<elements>:<slug> to a phase and associates it with the names used in TDB files, papers, the extraction corpus and the Materials Project and OQMD. The resulting 43,269 names comprise 22,556 TDB names, 8,368 paper labels, 10,916 corpus labels, 677 Materials Project names and 677 OQMD names. Through this identity, the record links to the TDB models and their verbatim definitions, stability in calculated diagrams (8,549 phases), literature values (4,974 phases) and example references (9,903 phases, 3,070 distinct DOIs). Structure types are assigned on the basis of a standard name (Tier A), DFT support (Tier B), or the declaration of an assessment (Tier C). When none of these applies, the record retains the candidate structures and the reasons for review (Table S4 in SI 1).
Table S4: Structure assignment in the phase registry. Each phase is given a crystal structure only when a rule establishes it, and the tiers are listed in decreasing order of certainty; phases for which no rule is decisive are kept in a review queue with their candidates.
| Status | Rule | Phases |
| tier A, standard name | the TDB name encodes one structure; stoichiometry, sublattices and DFT do not contradict it, including 23 FCC_A1 descriptions with an interstitial sublattice, (M)(X,Va), that model B1 or C1 compounds | 6,228 |
| tier B, DFT-supported | unique on-hull match in both Materials Project and OQMD, same space group | 154 |
| tier C, assessment-declared | one consistent structure in the SGTE crystallographic table for this name and system or, in one case (NiTi, B19′), in a comment of the TDB file | 1,248 |
| liquid; gas | by definition | 1,304; 231 |
| review queue | candidates listed, none certain | 1,236 |
| unresolved | no candidate | 29 |
| total | 10,430 |
Table S4.
The compact index phases/registry.json points to the full records in phases/systems/<SYS>.json. To make these identities available as linked data, registry.jsonld and context.jsonld express the registry as a schema.org DefinedTermSet in the manner of SKOS. The release also lists the 1,563 available structure types (711 used), phases awaiting a structure in review_queue.json, and unresolved disagreements in CONFLICTS.md.
Assessed phase assemblages (Assemblages)
An assessed phase assemblage represents the phases drawn by an expert at a point deep inside a named field of a handbook diagram. To describe this point, the record specifies temperature, composition and their uncertainty budgets, together with the placement method, either the area centroid or, for a nonconvex field, its pole of inaccessibility. To retain the distinction between drawn and estimated boundaries, the record states whether the bounding lines are solid, dashed or both. There are 20,782, 1,028 and 1,292 points in these categories, including 1,021 diagrams with dashed lines. The release contains 23,129 points from 2,178 diagrams, placed at 21,616 centroids and 1,513 poles of inaccessibility. Of these points, 18,490 lie in two-phase fields and 11,961 have an optional tie-line block. Their records retain the handbook label, canonical phase name and phase-registry identity where available, together with the basis of the naming decision (Table R4). The cross-check against SGTE assessments (main-paper Figure 5), fields for machine learning and source citations are retained as well, citing the original evaluation before the handbook.
| Basis | Points |
| deterministic adjacency | 8,035 |
| reader consensus | 6,793 |
| deterministic printed label | 3,205 |
| adjudicator | 2,803 |
| deterministic phase rules | 2,186 |
| expert reviewed | 87 |
| dft hull corroborated | 20 |
Table R4. Table R4: Assessed phase-assemblage points by naming basis. Deterministic printed label: a phase label printed inside the field. Deterministic adjacency and deterministic phase rules: names propagated by the solver from neighbouring fields, using the rule that adjacent fields differ by one phase and the constraints of invariant and compound lines. Reader consensus: agreement of the paired vision-model readers. Adjudicator: name assigned by the adjudicating model with a recorded reason. Expert reviewed: name confirmed or corrected by the domain expert. DFT hull corroborated: low-temperature field named with the help of the Materials Project and OQMD hulls (Supplementary Note 2 of SI 1).
To use these assemblages in thermodynamic fitting, the points are also distributed as 1,068 ESPEI ZPF datasets containing 9,877 points. The 13,252 points whose phases have no TDB name are listed separately. A further 643 datasets map the same points to the phases of the SGTE assessments of the system. Among these mapped datasets are those from a denser grid inside the fields of the Cu–Zr diagram, used in the final fit of main-paper Figure 9. The underlying assemblage records are available as JSON, CSV and Parquet, accompanied by exclusions and their reasons for fields and diagrams in which no point could be placed. As in the other datasets, the release contains only derived data and includes no page image, crop or text from the handbook.
Papers and their relations (Bibliography)
The bibliography places the extracted values within the literature from which they were obtained and the literature that remains to be read. Its relations allow a value to be traced back to the original report and forward to the assessments and diagrams that used it. The file bibliography/papers.jsonl contains records for 9,570 papers, 7,813 of them with a DOI. A record gives the Crossref metadata, the roles of the paper and the number of values it reports or for which it is cited as the source. The file edges.jsonl connects these papers to TDB files and handbook diagrams through 60,235 typed relations, retaining the provenance of the relations. The collection includes 1,427 corpus papers and assessments that were read, 3,578 original sources of extracted values, 1,886 TDB sources, 2,283 evaluations reproduced in the handbook and 2,806 papers identified but not yet read. Although a paper can have several of these roles, the collection also includes works to which no extracted value is yet attributed, so it is larger than the 4,882 papers and cited works associated with Values. The records are also distributed as CSV, Parquet, JSON-LD, Bib-TeX (orchid-bibliography.bib) and CSL-JSON. These records can also be accessed by chemical system through the files in systems/.
Synthetic extraction benchmark (Bench)
To evaluate extraction against known answers, the synthetic benchmark groups its files by document. A document is supplied as a vector PDF, 300-dpi page images, degraded 150-dpi scans and an image-only PDF, together with the truth key truth/<doc>.json (schema orchid-synthetic-paper-truth/v2). The truth key identifies the source TDB by SHA-256 and records the page transforms of the renderings. For figures, it specifies the axes, boundaries, invariant lines and plotted series. For the tables in the same documents, it identifies the semantic field and exact value of the cells, and for running text it records the stated values and their positions. The outputs to be compared with this truth are stored in outputs/<extractor>/<rendering>/. The corresponding scores, ledger of reader calls and RESULTS.md are stored in results/. Within this benchmark, the documents identify themselves as synthetic, and the three fictitious references cited in a document are explicitly identified as such.
Access and reuse
Related resources
ORCHID complements existing resources in three broad groups. The first consists of compilations of experimental values for particular properties, such as the formation enthalpies of intermetallic and other inorganic compounds collected by Kim et al. [22], the magnetic transition temperatures extracted automatically from the literature by Court and Cole [41] and extended by Kelly and Cole [42], and the properties and observed phases of multi-principal element alloys curated by Borg et al. [43]. The second consists of high-throughput DFT databases, such as the Materials Project [3] and OQMD [4, 5], which provide formation energies at 0 K for very large numbers of compounds, and the third of the CALPHAD resources themselves, the TDB collections and fitting software discussed in the introduction. What none of these provides is the phase-equilibrium and thermochemical evidence behind CALPHAD models, recorded as equilibrium states with their provenance, and Table R5 sets out how ORCHID differs from the most closely related of them in this respect.
| Resource | Open licence | CALPHAD-ready conditions | Per-value provenance | ML splits | Agent access |
| ORCHID (this work) | yes (CC BY 4.0) | yes (states; ESPEI datasets) | yes (DOI, page, quote or row) | yes (by paper; system and paper folds) | MCP server; OpenAPI |
| ThermoML archive (NIST TRC) [44, 54] | partial† (NIST open data; files from a cooperation between NIST TRC and five journals (about 2003–2019)) | partial (state variables per property; mainly fluids; not ESPEI format) | partial (one file per article) | no | no (bulk download) |
| MPDS / PAULING FILE (incl. former ASM APD) [46, 48] | partial† (subscription; open subset under CC BY) | partial (redrawn diagrams with phase labels, phase lists; no ZPF data or digitized equilibria) | yes (per entry, to publication; per diagram for the former APD†) | no | API (no MCP) |
| SGTE binary collection [12] | yes (CC BY 4.0) | no (model parameters, not observations) | partial (each TDB cites its assessment) | no | no |
| Materials Project [3, 49] | yes (CC BY 4.0) | partial (0 K DFT energies) | – (calculation ids) | no | MCP server [50]; API |
| OQMD [4, 51] | yes (CC BY 4.0) | partial (0 K DFT energies) | – (calculation ids) | no | API (REST, OPTIMADE) [52]; no MCP |
| ESPEI-datasets [53] | yes (CC BY 4.0) | yes (ESPEI JSON) | partial (reference key per dataset) | no | no |
Table R5.
Obtaining the data
The choice of an access route depends on whether the analysis requires complete records, a tabular view or queries for selected systems and phases. The data deposits described above are archived on Zenodo and available with documentation on the project’s data site; the synthetic benchmark is archived with its software release. JSON files preserve complete records, while CSV files provide a flat view for spreadsheets and inspection. Parquet tables include fixed training and test splits and are also available as configurations of a Hugging Face dataset. Programs and AI agents can, in addition, query the data directly through a Model Context Protocol server [55], whose tools answer questions such as which values are known for a system or how a phase is named across sources, and every answer returns with the provenance of the values it contains (Table R6).
| Aspect | How ORCHID meets it | Where (file / endpoint) |
| Formats and schemas | documented JSON, CSV and Parquet; a schema per dataset; controlled vocabularies for property codes, evidence classes and reaction types | parquet/, */SCHEMA.md, literature/QUALITY_FIXES.md |
| Machine learning | Parquet with by-paper splits (training and test data never share a paper) and leave-system-out and leave-paper-out folds; Hugging Face dataset card with 12 configurations | parquet/literature_values.parquet (split, fold_lso, fold_lpo), hf/README.md, DATASET_CARD.md |
| Metadata | Croissant [57] and JSON-LD metadata; DataCite metadata through Zenodo; citation files; datasheet [58] | data/croissant.json (orchid-project.org), phases/context.jsonld, zenodo/*.metadata.json, CITATION.cff, DATASET_CARD.md |
| Discoverability | crawlable static pages with schema.org JSON-LD, sitemap, llms.txt; stable identifiers (ORC-, phase:) and DOIs | systems/, phases/pages/, sitemap.xml, robots.txt, llms.txt, llms-full.txt, zenodo/dois.json |
| Access for programs and agents | MCP server with 15 tools; OpenAPI specification; Python package installable from GitHub (pip install git+https://github.com/orchid-calphad/orchid-mcp; 10.5281/zenodo.23748397) | orchid-mcp, openapi/openapi.yaml |
| Per-value provenance and trust | DOI, page, verbatim quote, cited original where resolved, readers and their agreement, evidence class, tool release, uncertainty budget | literature/systems/<SYS>.json, paper_doi, quote, readers, reader_agreement, sigma_effective |
| Quality labels | flags, review states, adversarial audit rates, extraction benchmark | flag_*, review_state, literature/QUALITY_FIXES.md, orchid-bench/results/ |
| Reuse | ESPEI-validated datasets with , weights and tags; calibrated DFT | espei/, espei/dft/CALIBRATION.json, assessed-phase-assemblages/espei-zpf/ |
| Licensing | CC BY 4.0 for ORCHID’s data, which permits use for model training with attribution; quotes are short excerpts kept for verification under the publishers’ copyright; third-party licenses stated; restricted sources are never redistributed | LICENSE, LICENSES.md, NOTICE.md |
| Reproducibility | deterministic builders with --check; versioned DOIs; changelog; status “Version 1, machine-extracted” | tools/*.py --check (orchid-project.org), CHANGELOG.md, zenodo/ |
| Field or flag | Values | Recommended use |
| accepted | 66,551 | restrict analyses to accepted values |
| evidence_class | model, unspecified | do not treat as measurements |
| assessed_phase_diagram points | 23,129 | keep apart from measured and model values |
| reader_agreement = single_reader | 1,342 | exclude or down-weight |
| unit kJ/mol (basis unstated) | 2,255 | exclude from per-atom analyses |
| flag_basis_inferred_from_magnitude | 2,117 | may be excluded from per-atom analyses |
| flag_unsigned_in_print | 42 | exclude from signed comparisons |
| flag_source_disputed | 512 | check before use |
| flag_magnitude_suspect | 499 | check before use |
| flag_system_mismatch | 424 | check before use |
| flag_dft_outlier_unexplained | 130 | check before use |
| sigma_effective (sigma_method_basis) | all | weights; the basis states whether the method floor was calibrated or is a prior |
Table R6. AI readiness of ORCHID. Each aspect follows the ESIP AI-readiness checklist [29] and the FAIR principles [28]. Paths are relative to the orchid-data repository unless another repository is named.
For those who work with files, temperatures are given in K, pressures in Pa, formation enthalpies in kJ (mol of atoms) or, where the basis is not printed, in kJ mol with the basis unstated, and compositions as mole fractions, with the exceptions, such as DFT energies in eV atom, lattice lengths in Å, Gibbs energies that keep their printed units and 572 formation values whose unit could not be normalized, which keep the unit as read and are never ESPEI-ready, listed in Table S1 in SI 1.
The Parquet tables can be loaded directly with pandas, DuckDB or the Hugging Face datasets library [56] (12 configurations: literature_values, papers, experimental_context, dft_parity, phase_diagram_invariants, phase_diagram_index, hull_vertices, phases, structure_types, assessed_phase_assemblages, bibliography_papers, bibliography_edges). For those who prefer to browse, every chemical system, compound and binary phase has a static page at https://data.orchid-project.org, and https://orchid-project.org provides viewers for the data, the phase diagrams, the DFT comparison and the hulls. For programs and AI agents, the orchid-mcp server exposes 15 tools (search_systems, get_system_data, get_invariant_reactions, get_formation_enthalpy, get_phase_diagram, list_tdb_files, get_tdb, get_paper, search_papers, get_paper_relations, get_assessment_evidence, compare_to_dft, get_phase, search_phases, get_structure_type; Table S7 in SI 1, with a recorded exchange in Figure S6 in SI 1), an OpenAPI specification describes the static file endpoints for programs that do not use MCP, and CROISSANT metadata [57] and llms.txt files describe the data for machines.

Figure S6. An AI agent receives each value with its full state and its source. The orchid-mcp server was run locally against the released files, and a client listed its tools and called two of them; requests are shown as sent and responses are trimmed for print. The invariant reaction arrives with its phases, temperature, evidence class, citation, page and the verbatim sentence it was read from, and the phase query returns the registry identity with the names used for it in the SGTE assessments and in the literature. The full exchange is released with the examples repository (orchid-examples, quickstart/mcp_exchange.json).
| Route | Purpose |
| MCP tools | |
search_systems | find chemical systems by elements or text; counts by evidence class |
get_system_data | literature values of a system, filtered by property group, code or evidence class |
get_invariant_reactions | literature and calculated invariant reactions of a system, side by side |
get_formation_enthalpy | formation enthalpies of a compound: literature, Materials Project, OQMD |
get_phase_diagram | calculated binary diagrams: TDB file, SHA-256, citation, phases, invariants |
compare_to_dft | literature formation enthalpies against both DFT sources, with residuals |
list_tdb_files, get_tdb | TDB catalog entries and files |
get_paper | bibliographic record of a paper, its values and experimental context |
search_papers | search Bibliography by text, system, role or year |
get_paper_relations | typed relations of a paper: what it cites, which assessments and TDB files used it |
get_assessment_evidence | the data papers behind the assessments of a system |
get_phase, search_phases | phase-registry identities, names, models, structure and stability |
get_structure_type | structure types by prototype, Strukturbericht, Pearson symbol or space group |
Static endpoints (OpenAPI, openapi/openapi.yaml) | |
/literature/... | index, per-system files, papers, contexts, DFT parity, full dumps (JSONL, CSV) |
/phase-diagrams/... | index and one file per calculated diagram |
/hulls/{layer}/... | catalogs and per-system hulls (MP, OQMD, SGTE at 298.15 K), entries tiers |
/phases/... | registry (JSON, JSON-LD, CSV), per-system records, structure types |
/catalog.json, /databases/... | TDB catalog and bundled TDB files |
/croissant.json | Croissant metadata |
| Other routes | |
| Parquet, Hugging Face | parquet/*.parquet; dataset card with 12 configurations |
| web pages | per-system and per-phase pages at https://data.orchid-project.org; llms.txt |
| Zenodo | 9 records, one for each dataset except Bench and one for the ESPEI datasets, each with its own DOI, and an umbrella record |
| examples | orchid-examples: runnable workflows (Cu–Zr with PhaseForge+ and with ESPEI, the one-shot score, the MLIP benchmark, a quick start) |
| contributions | issue forms and validated files for corrections and new data (CONTRIBUTING.md) |
Table S7. Programmatic access. The 15 tools, 4 resources and 1 prompt of the orchid-mcp server (tool names as returned by tools/list; Supplementary Fig. S6), the static file endpoints described by its OpenAPI specification, and the other routes to the data. Every answer carries the citation, DOI, page, quote or table locator, evidence class and flags of each value, with the source information needed to report a discrepancy. Direct links to pre-filled issues are part of the contribution workflow under development in SI 2.
These routes return the same versioned records, so the choice of interface can be separated from the later choice of evidence for a particular analysis (Figure R2).
Figure R2. Access and evidence selection are separate operations. Complete archives, tabular files, web pages and agent tools expose the same state, evidence, provenance and identity fields. A user can then select records according to acceptance, evidence class, completeness of the state and the requirements of the intended calculation, while the records left out of a working subset remain available with the reasons for their exclusion.
Selecting evidence for an analysis
Once the data have been obtained, their use depends on selecting evidence appropriate to the intended analysis. The release makes this selection explicit by retaining the information needed to assess a value before it enters a fit. The records specify the evidence class, number of agreeing independent readers, checks passed and corrections received.
Beyond these checks, the records distinguish readiness for ESPEI from certification by agreement with a CALPHAD calculation. A user who needs the most conservative subset can therefore restrict a fit to certified experimental values, while one who wants the widest coverage can include single-reader or model values, knowing exactly what each choice admits. The following coverage and evaluation details inform these selections. The numerical literature values were extracted by machine, with page verification performed by AI agents; no independent human precision estimate is reported for the numerical collection. The completed expert evaluation of assessed-diagram fields reported in SI 1 concerns a separate population and task. The coverage also excludes values reported only in figures, and pressure is almost never printed, so the one atmosphere recorded for all but 5 values is an assumption.
To implement evidence selection, Table R7 gives the filters and their counts. The accepted values provide the default subset for analysis. Within this subset, model values retain their status as calculations, while unspecified values retain an unresolved evidence class. Assessed points should be kept separate from both, while the weight assigned to a value should follow its effective uncertainty. Interpretation also requires considering how the sources were selected and read. In particular, the corpus consists of the papers that could be read, rather than all papers on a system. Within that corpus, an experimental value cited from older work retains the citing paper’s interpretation of that work. For tables covering several systems, values are assigned to the system identified by their row.
Such errors can be reported through the issue trackers of the repositories, and identifiers (ORC-…, phase:…) are stable across releases.
Contribution workflow under development. A contribution branch that is not yet merged adds links from data pages and MCP responses to pre-filled issues in the orchid-data repository. These links are intended to preserve the record identifier, source and page when a user or agent reports a discrepancy. The same branch provides a pull-request workflow for corrections, new values and assessments, with automated release-rule checks followed by source-page review by a maintainer. Under this proposed workflow, accepted contributions would enter a subsequent release with contributor credit and the original value retained in the record history. These mechanisms remain prospective until the branch is merged and the corresponding release is verified.
Community calibration and planned educational use
The retained source information also provides a basis for community contributions to the coverage and calibration of the collection. Using the released identifiers and provenance, a contributor can report a discrepancy in a specific record together with the source needed to examine it, while proposed additions can be evaluated against the existing requirements for state assignment and consistency. An educational use is also being prepared in which students identify the phases at points of real diagrams, with known-answer questions used to calibrate their reliabilTable R5: ORCHID and related resources. Open licence: terms that allow reuse and redistribution of the data. CALPHAD-ready conditions: each datum carries its thermodynamic state (phases, temperature, pressure, compositions) in a form usable for fitting. Per-value provenance: each value points to its source publication (ORCHID: DOI, page and quote or table row). ML splits: official training and test splits. Agent access: an official Model Context Protocol (MCP) server or, failing that, an API. –: not applicable (computed data). Cells were checked against each resource’s public documentation on 23 October 2026 12;[44, 45, 46, 47, 48, 49, 50, 51, 52, 53].
†Not fully verified: the reuse terms of publisher-contributed ThermoML files, the scope of the open MPDS subset, the granularity of the former ASM Alloy Phase Diagram Database (APD, now ASM MPDS) citations; the Materials Project and OQMD licence statements were read from the legacy terms page and an archived download page.
ity before uncertain responses are considered for inclusion. The first classroom use is planned for November 2026 and is therefore a prospective contribution mechanism, not a completed validation result. Repository mechanisms that remain under development are documented separately rather than assumed to be operating in the released workflow.
Machine learning
The provenance retained for evidence selection also determines how the records can be divided into training and test sets for machine-learning models of thermodynamic properties. Such divisions must account for correlations among observations from the same chemical system and paper. To support these evaluations, the records retain their chemical system and source paper together with two fixed assignments to five folds. One assignment groups all values of a chemical system, while the other groups all values of a paper. These groupings allow a model to be tested on chemistry or sources absent from its training data. Within these folds, composition and structure can be used to learn formation enthalpies, invariant temperatures and phase stability. For these learning tasks, the evidence classes allow training to be restricted to experiment, and the uncertainty budgets allow observations to be weighted according to their reliability. For a single fixed division, the column split assigns all values of a paper to the training, validation or test set. The resulting sets contain 58,315, 6,754 and 8,473 values from 1,053, 123 and 145 papers, respectively, using the hash rule in Supplementary Note 4 in SI 1. However, a measurement may be reprinted in several papers. The strictest leakage control therefore groups values by the work in which they were first measured, recorded in origin_measured_in. Alongside these property-prediction tasks, for the training and testing of extraction itself, the quotes and table locators of the records can be paired with their structured fields, and new extractors can be tested on Bench, whose answers are known exactly.
Licence and citation
All of these uses rest on the licence, and the derived data are released under CC BY 4.0. Within these terms, each quote is a single short excerpt, kept so that the value can be checked against its source, and it remains under the copyright of the original publisher. Apart from such quotes, the source papers, page images and full texts are not redistributed. For Assemblages, only derived facts are released (phase names, assemblages, coordinates, phase metadata and citations), and the pages, images and text of the handbook are not. Users are asked to cite this paper, the dataset DOI and the original source of a value (paper_doi, cited_dois, and, for Assemblages, the original evaluation data and then the handbook). Third-party data keep their licences and citations, which applies to the SGTE binary collection [12], the Materials Project [3]
and OQMD[4, 5], all released under CC BY 4.0, to the compilation of Kim et al. 22, whose files are released under the MIT licence, and to the bibliographic metadata from Crossref, which carry no restriction (CC0). The dataset card, in turn, follows the datasheet questions of Gebru et al. 58.
Data availability
The data described in this paper are release v2026.10 of ORCHID, released under CC BY 4.0 (third-party data under the licences stated in the licence section and in LICENSES.md). The release is archived on Zenodo in 9 records, comprising an umbrella record (10.5281/zenodo.23736065) and eight individual data deposits: literature values (10.5281/zenodo.23735569), ESPEI datasets (10.5281/zenodo.23735788), calculated phase diagrams (10.5281/zenodo.23735859), convex hulls (10.5281/zenodo.23735891), DFT versus experiment (10.5281/zenodo.23735943), phase registry (10.5281/zenodo.23735970), assessed phase assemblages (10.5281/zenodo.23735996) and bibliography (10.5281/zenodo.23736021). The synthetic extraction benchmark, Bench, is archived with the software release of the orchid-bench repository (10.5281/zenodo.23748414). The same files are served, with documentation, at https://data.orchid-project.org from the orchid-data repository at https://github.com/orchid-calphad, and the Parquet tables are mirrored as a Hugging Face dataset. The GitHub repositories and the data site have been public since 28 October 2026.
Code availability
The code is MIT-licensed at https://github.com/orchid-calphad. The orchid-project.org repository holds the builders of the released files, namely
export_literature_data.py, literature_quality.py, literature_gates.py, phase_rule.py, uncertainty_budget.py and ml_folds.py (Values, clean-up, gates, uncertainty budget and folds), build_espei_bundle.py, espei_constraints.py and espei_evaluate.py (ESPEI datasets, DFT calibration and likelihood evaluation), compute_binary_diagrams.py and add_phase_grid.py (Diagrams), build_dft_parity.py and kim2017_crosscheck.py (Parity), build_phase_registry.py (Registry), build_assessed_assemblages.py and _dashed_boundaries.py (Assemblages), build_bibliography.py (Bibliography), calc_invariants.py and build_parquet.py, build_static_pages.py, build_precision_sample.py and score_precision_sample.py. The builders are deterministic, and each has a --check mode that verifies the released files. The orchid-data repository holds the data, and orchid-mcp holds the MCP server and its OpenAPI specification. Phase diagrams were calculated with pycalphad 0.11.2 under Python 3.13, and the ESPEI[10] datasets were validated with ESPEI 0.9.1. The extraction pipeline, the hull builders and the phase-diagram annotator are in orchid-pipeline, and the synthetic extraction benchmark, with its generator, baselines and scorer, is in orchid-bench. Each release is archived on Zenodo (10.5281/zenodo.23748335). The counts and figures of this paper are generated from the released files by scripts/make_numbers.py and scripts/make_figures.py, which are distributed with the manuscript source.
References
- [1]R. Arróyave, Journal of Phase Equilibria and Diffusion, 2022, 43, 606–628.
- [2]R. Schmid-Fetzer, Journal of Phase Equilibria and Diffusion, 2014, 35, 735–760.
- [3]A. Jain, S. P. Ong, G. Hautier, W. Chen, W. D. Richards, S. Dacek, S. Cholia, D. Gunter, D. Skinner, G. Ceder and K. A. Persson, APL Materials, 2013, 1, 011002.
- [4]J. E. Saal, S. Kirklin, M. Aykol, B. Meredig and C. Wolverton, JOM, 2013, 65, 1501–1509.
- [5]S. Kirklin, J. E. Saal, B. Meredig, A. Thompson, J. W. Doak, M. Aykol, S. Rühl and C. Wolverton, npj Computational Materials, 2015, 1, 15010.
- [6]S. Curtarolo et al., Computational Materials Science, 2012, 58, 227–235.
- [7]C. Draxl and M. Scheffler, Journal of Physics: Materials, 2019, 2, 036001.
- [8]H. L. Lukas, S. G. Fries and B. Sundman, Computational Thermodynamics: The Calphad Method, Cambridge University Press, Cambridge, 2007.
- [9]R. Otis and Z.-K. Liu, Journal of Open Research Software, 2017, 5, 1.
- [10]B. Bocklund, R. Otis, A. Egorov, A. Obaied, I. Roslyakova and Z.-K. Liu, MRS Communications, 2019, 9, 618–627.
- [11]A. T. Dinsdale, Calphad, 1991, 15, 317–425.
- [12]B. Hallstedt, Calphad, 2025, 89, 102833.
- [13]A. Roy, E. Grisan, J. Buckeridge and C. Gattoni, Digital Discovery, 2026, 5, 1794–1808.
- [14]S. L. Hödl, L. Hermans, P. F. J. Dankloff, A. Piruska, W. T. S. Huck and W. E. Robinson, Digital Discovery, 2025, 4, 1176–1187.
- [15]Thermo-Calc Software, Assessment of Thermodynamic Data, https://thermocalc.com/about-us/methodology/the-calphad-methodology/assessment-of-thermodynamic-data/, Accessed 27 October 2026.
- [16]Thermo-Calc Software, Why Choose Thermo-Calc?, https://thermocalc.com/about-us/why-choose-thermo-calc/, Accessed 27 October 2026.
- [17]S. Zhu, D. Saritürk and R. Arróyave, npj Computational Materials, 2025, 11, 340.
- [18]S. Zhu, D. Saritürk and R. Arróyave, Acta Materialia, 2025, 286, 120747.
- [19]D. Saritürk, S. Zhu and R. Arróyave, PhaseForge, Zenodo, 2025, https://github.com/dogusariturk/PhaseForge, Software, version v1.0.0.
- [20]C. Kunselman, S. Zhu, D. Saritürk and R. Arróyave, Journal of Phase Equilibria and Diffusion, 2025.
- [21]D. Saritürk, C. Kunselman and S. Zhu, PhaseForge-Plus, Zenodo, 2025, https://github.com/dogusariturk/PhaseForgePlus, Software, version v0.1.0.
- [22]G. Kim, S. V. Meschel, P. Nash and W. Chen, Scientific Data, 2017, 4, 170162.
- [23]O. Kubaschewski, C. B. Alcock and P. J. Spencer, Materials Thermochemistry, Pergamon Press, Oxford, 6th edn., 1993.
- [24]J. Drowart, C. Chatillon, J. Hastie and D. Bonnell, Pure and Applied Chemistry, 2005, 77, 683–737.
- [25]J. Riebesell, R. E. A. Goodall, P. Benner, Y. Chiang, B. Deng, G. Ceder, M. Asta, A. A. Lee, A. Jain and K. A. Persson, Nature Machine Intelligence, 2025, 7, 836–847.
- [26]C. Kunselman, B. Bocklund, R. Otis and R. Arróyave, Acta Materialia, 2025, 300, 121453.
- [27]C. Kunselman, Add Gradient Capability (ESPEI pull request #268, branch Gradient-based_update), GitHub pull request to PhasesResearchLab/ESPEI, opened 24 April 2025, open (not merged) as of 23 October 2026, 2025, https://github.com/PhasesResearchLab/ESPEI/pull/268, Accessed 23 October 2026.
- [28]M. D. Wilkinson, M. Dumontier, I. J. Aalbersberg, G. Appleton, M. Axton, A. Baak et al., Scientific Data, 2016, 3, 160018.
- [29]ESIP Data Readiness Cluster, Checklist to Examine AI-readiness for Open Environmental Datasets, figshare, 2022, Version 0.2.DOI
- [30]J. Dshemuchadse and W. Steurer, Acta Crystallographica Section A: Foundations and Advances, 2015, 71, 335–345.
- [31]P. Omprakash, Beyond the Binary Hull: Higher-Order Thermodynamic Stabilization in Inorganic Ternary Compounds, 2026, Preprint, submitted 15 September 2026.arxiv.org/abs/2609.17098
- [32]W. Steurer and J. Dshemuchadse, Intermetallics: Structures, Properties, and Statistics, Oxford University Press, Oxford, 2016.DOI
- [33]M. Fan, Y. Wang, G. Vazquez, R. Zhou, I. Karaman, R. Arróyave and X. Qian, Digital Discovery, 2026, 5, 2478–2490.
- [34]J. Liao, H. Wang and T.-Y. Chen, Materials, 2018, 11, 2396.
- [35]A. Wang, R. Kingsbury, M. McDermott, M. Horton, A. Jain, S. P. Ong, S. Dwaraknath and K. A. Persson, Scientific Reports, 2021, 11, 15496.
- [36]A. Bochkarev, Y. Lysogorskiy and R. Drautz, Physical Review X, 2024, 14, 021036.
- [37]R. Arróyave, m_scraps: multi-sublattice short-range-order and cluster-correlation targeting, 2026, https://github.com/rarroyave/m_scraps, MIT licence; extends SCRAPS / Hybrid-Cuckoo-Search (Singh et al. 2021). Draft from CITATION.cff, commit 776e698.DOI
- [38]R. Singh, A. Sharma, P. Singh, G. Balasubramanian and D. D. Johnson, Nature Computational Science, 2021, 1, 54–61.
- [39]R. Arróyave, J. Saal and D. Shin, ORCHID data, GitHub repository orchid-data, 2026, https://data.orchid-project.org, Release v2026.10.
- [40]K. J. Zeng, M. Hämäläinen and H. L. Lukas, Journal of Phase Equilibria, 1994, 15, 577–586.
- [41]C. J. Court and J. M. Cole, Scientific Data, 2018, 5, 180111.
- [42]C. R. Kelly and J. M. Cole, Scientific Data, 2025, 12, 2000.
- [43]C. K. H. Borg, C. Frey, J. Moh, T. M. Pollock, S. Gorsse, D. B. Miracle, O. N. Senkov, B. Meredig and J. E. Saal, Scientific Data, 2020, 7, 430.
- [44]D. Riccardi, A. Bazyleva, E. Paulechka, V. Diky, J. W. Magee, A. F. Kazakov, S. A. Townsend and C. D. Muzny, ThermoML/Data Archive, NIST Public Data Repository, 2021, https://data.nist.gov/od/id/mds2-2422, Version 1.2.6. Accessed 23 October 2026.DOI
- [45]National Institute of Standards and Technology, Copyright, Fair Use, and Licensing Statements for SRD, Data, Software, and Technical Series Publications, 2025, https://www.nist.gov/open/copyright-fair-use-and-licensing-statements-srd-data-software-and-technical-series-publications, Updated 24 June 2025. Accessed 23 October 2026.
- [46]Materials Platform for Data Science, MPDS API Documentation, 2026, https://developer.mpds.io/, Accessed 23 October 2026.
- [47]E. Blokhin, MPDS PAULING FILE: Big Data and Machine Learning, Psi-k forum, General announcements, 2017, https://psi-k.net/general-announcements/mpdspauling-file-big-data-and-machine-learning/, Announces the open CC BY 4.0 subset of the PAULING FILE. Accessed 23 October 2026.
- [48]ASM International, ASM Alloy Phase Diagram Database (now transitioned to the ASM Materials Platform for Data Science), 2026, https://www.asminternational.org/materials-resources/online-databases/, Accessed 23 October 2026.
- [49]The Materials Project, Materials Project Terms of Use, 2026, https://legacy.materialsproject.org/terms, Legacy website. Accessed 23 October 2026.
- [50]The Materials Project, MP + LLMs (MCP), Materials Project Documentation, 2026, https://docs.materialsproject.org/services/ml-and-ai-applications/mcp, Accessed 23 October 2026.
- [51]OQMD Development Team, OQMD Download, 2025, https://oqmd.org/download/, Live page returned HTTP 502 on 23 October 2026; licence statement (CC BY 4.0) read from the archived copy of 10 November 2025, https://web.archive.org/web/2025110402317/https://www.oqmd.org/download/.
- [52]OQMD Development Team, OQMD RESTful API, qmypy v1.2.0 documentation, 2019, https://static.oqmd.org/static/docs/restful.html, Includes the OPTiMaDe RESTful specification. Accessed 23 October 2026.
- [53]Phases Research Lab, ESPEI-datasets, GitHub repository, 2026, https://github.com/PhasesResearchLab/ESPEI-datasets. Licensed under CC BY 4.0. Accessed 23 October 2026.
- [54]M. Frenkel, R. D. Chirico, V. V. Diky, Q. Dong, S. Frenkel, P. R. Français, D. L. Embry, T. L. Teague, K. N. Marsh and R. C. Wilhoit, Journal of Chemical & Engineering Data, 2003, 48, 2–13.
- [55]Anthropic, Model Context Protocol specification, https://modelcontextprotocol.io, 2024, Accessed 28 October 2026.
- [56]Q. Lhoest, A. Villanova del Moral, Y. Jernite, A. Thakur, P. von Platen et al., Proceedings of the 2021 Conference on Empirical Methods in Natural Language Processing: System Demonstrations, 2021, pp. 175–184.
- [57]M. Akhtar, O. Benjelloun, C. Conforti, P. Gijsbers, J. Giner-Miguelez, N. Jain et al., Proceedings of the Eighth Workshop on Data Management for End-to-End Machine Learning (DEEM ’24), 2024, pp. 1–6.
- [58]T. Gebru, J. Morgenstern, B. Vecchione, J. W. Vaughan, H. Wallach, H. Daumé III and K. Crawford, Communications of the ACM, 2021, 64, 86–92.
























