WorksPhysics and Astronomy
No Value Is Better Than a Wrong Value: Finding and Handling Erroneous Data in Literature-Mined Datasets for AI
Abstract
Machine learning across materials science, chemistry, energy, and medicine increasingly relies on literature-extracted datasets due to a lack of curated databases. However, automated extraction introduces errors: text recognition misreads digits, units and notations are confused, values are misattributed, and LLMs hallucinate numbers. Drawing on our work building an electro-optic property database, we present a taxonomy of errors organized by their origin and signatures, alongside a layered framework for error detection. The framework combines schema and provenance checks, physical admissibility (bounds, symmetry, and governing equations), cross-source statistical consensus, model auditing, and suspicion-ranked review. In a production run, our verification gate halted 402 of 816 candidate values with typed reasons. Yet, an audit against curated references showed that 30 of 112 accepted values were defective; standard symmetry rules alone caught 21 of these. On measured BaTiO3 and KNbO3 tensors, constitutive identities caught 8 of 10 single-value errors and isolated the corrupted entries. In a controlled benchmark, the combined framework caught 81% of errors at 85% precision, including errors in single-report materials, bringing model accuracy on held-out data to within 3% of a model trained on clean data. Uncleaned data severely impairs discovery: 89% of the twenty largest values were errors, causing AI searches to prioritize false positives. Once bad numbers are controlled, missing measurement conditions emerge as the main limit, as distinct records become indistinguishable without full context. We outline directions for the field, including shared physics rule libraries and self-reporting databases.
Introduction
Machine learning has changed how materials, molecules, and devices are discovered, but the limit on what it can do is set more and more by the data it learns from, not by the algorithm [1, 2]. Where large, curated databases exist, progress has been dramatic. High-throughput first-principles repositories such as the Materials Project [3], OQMD [4], and AFLOW [5] made formation energies, band structures, and elastic constants available at a scale no single laboratory could produce. Most of the knowledge a researcher actually needs, though, is still held in unstructured publications: the measured value of a functional property, the conditions under which it was measured, the synthesis route that produced the sample, and the device-level figure of merit [6, 7]. For a large and growing number of properties no database exists at all. Anyone who wants to train a model on them must first build the dataset, and in practice that means reading the literature [8, 9, 10, 11].
Automated literature mining has become practical. Rule-based toolkits such as ChemDataExtractor [8] have been used to build databases of phase-transition temperatures [9] and battery materials [10]. More recently, large language models (LLMs) have extracted structured records from scientific text with precision approaching that of expert curators [12, 13]. What these tools cannot do is make every extracted value correct. A PDF is a page-layout format, not a data format. Subscripts collapse into the main line, table cells come apart from their headers, and scanned papers from before the 1990s reach the extractor as images of text. Units, sign conventions, tensor notations, and measurement conditions vary between papers. Some values are wrong in the original publication. An LLM that reads the text well can still attach a number to the wrong label, material, or condition, and it can produce a value that was never in the text [14].
The need for such a strategy is not specific to materials science. The cheminformatics community learned years ago that errors in public databases reach published models: incorrect chemical structures [15, 16, 17] and inconsistent bioactivity values combined from different sources [18, 19] measurably degrade structure–activity models. The thermophysical-property community built dynamic data evaluation around the same problem [20, 21].
In medicine, a meta-analysis combines the numbers reported by many clinical trials into one overall answer. The numbers are copied by hand from the trial reports, copying mistakes are common, and they can be large enough to change the overall answer [22, 23]. Recent audits of widely used structure databases for metal–organic frameworks found high error rates in supposedly computation-ready structures [24, 25].
In machine learning more broadly, label errors in benchmark test sets change which model is ranked best [26]. Work on data is often valued less than work on models, and small data problems left early in a project then grow into larger failures later, which [27] call “data cascades”.
This Perspective asks how erroneous data should be handled when AI models are trained on datasets built from the literature. We draw on our experience of building an AI-ready database of electro-optic (Pockels) coefficients from the literature [28, 29], where most of the categories of error discussed below were met in practice. The argument is meant to be general: the same failure modes appear wherever numbers are harvested from publications, whether in battery research, medicine, or mechanics. Figure 1 summarises the approach.
The paper is a hybrid: part perspective, part quantitative demonstration. We make four contributions.
A taxonomy of errors in literature-mined data, organised by where in the pipeline each error arises and by the signature it leaves in the data, illustrated with errors we caught in the electro-optic campaign (§ 3).
A layered detection framework combining schema and provenance checks, physical admissibility and governing equations, cross-source statistical consensus, model-based auditing, and targeted human review. For each layer we state what it can detect, what it cannot, and what it needs as input (§ 4).
Four demonstrations, two of them on real production data. We tally what the verification gate of our electro-optic pipeline stopped in its latest run and audit what it accepted against a curated reference; we show that constitutive identities on measured BaTiO and KNbO tensors expose realistic single-value errors by many standard deviations; and in a controlled benchmark with known ground truth we measure the detection rate of each layer by error type and by source redundancy (§ 5).
A quantitative account of the consequences for AI modelling: accuracy, tolerance to errors, and above all inverse design (searching for the material, or the conditions, that maximise a predicted property), which actively seeks out erroneous values (§ 6). The paper closes with directions for the field (§ 7).
Because the paper draws on several fields, each symbol and field-specific quantity is explained where it first appears, and all of them are collected in § 9.
[
every node/.style={font=\footnotesize, align=center},
src/.style={ddevidence, text width=27mm, minimum height=9mm},
lay/.style={ddmethod, text width=40mm, minimum height=9mm},
outc/.style={ddresult, text width=24mm, minimum height=9mm},
node distance=2.2mm]
\node[src] (s1) {Source document\\typos, cited values};
\node[src, below=of s1] (s2) {Conversion\\OCR, layout, scans};
\node[src, below=of s2] (s3) {Extraction\\label, unit, material};
\node[src, below=of s3] (s4) {Context\\lost conditions};
\node[src, below=of s4] (s5) {Harmonisation\\conventions, duplicates};
\node[lay, right=12mm of s1] (l0) {L0\enspace Schema, units, provenance};
\node[lay, below=of l0] (l1) {L1\enspace Physics: bounds, symmetry,\\governing equations};
\node[lay, below=of l1] (l2) {L2\enspace Cross-source consensus};
\node[lay, below=of l2] (l3) {L3\enspace Model-based audit};
\node[lay, below=of l3] (l4) {L4\enspace Targeted human review};
\node[outc, right=12mm of l0] (o1) {Accept\\with weight};
\node[outc, below=of o1] (o2) {Repair\\verified rescale};
\node[outc, below=of o2] (o3) {Hold\\typed reason};
\node[outc, below=of o3] (o4) {Down-weight\\in training};
\node[ddcontrol, text width=24mm, minimum height=9mm, below=of o4] (o5) {Reject\\logged, not deleted};
\node[above=1.5mm of s1, font=\small\bfseries] {Where errors arise};
\node[above=1.5mm of l0, font=\small\bfseries] {Layered detection};
\node[above=1.5mm of o1, font=\small\bfseries] {What is done};
\draw[ddflow] ($(s3.east)+(0.5mm,0)$) -- ($(l2.west)-(0.5mm,0)$);
\draw[ddflow] ($(l2.east)+(0.5mm,0)$) -- ($(o3.west)-(0.5mm,0)$);
\draw[ddreturn] (l4.south) -- ++(0,-4mm) -| ($(l0.west)+(-4mm,0)$) -- (l0.west);
\node[font=\scriptsize\itshape, anchor=north] at ($(l4.south)+(0,-4.6mm)$) {review decisions fed back as rules};Figure 1. Overview of the approach. Errors enter a literature-mined dataset at five points (left; § 3). They are caught by five detection layers of increasing cost (centre; § 4), and each value leaves with one of five outcomes (right). No value is silently deleted: every hold, repair, and rejection is recorded with its reason, so the database can be audited and the decision reversed.
AI-Ready Data Collection from the Literature
Why the literature, and why now
Three conditions make literature mining necessary rather than merely convenient. First, many functional properties are expensive or impossible to compute at scale. Electro-optic, piezoelectric, and nonlinear-optical responses need linear-response calculations with phonon and strain contributions. Ionic conductivity, cycle life, and toxicity depend on processing and microstructure, which no bulk calculation represents. The measured value in a paper is therefore often the only value that exists. Second, a property value means something only together with its context: the material and its form (bulk crystal, thin film, nanostructure), the tensor component, the mechanical boundary condition, the temperature, the frequency or wavelength, and the method. That context is written in prose and table captions, not in a schema. Third, measured data on rare or new materials are found only in papers. A composition that performs unusually well has often been reported once, in a single paper, and appears in no database.
The expected payoff is a database that is AI-ready, which means more than a table of numbers. Each value is bound to its material, component, units, and conditions. Each value carries its provenance (paper, page, table cell, raw snippet). Each value has a confidence that later stages can use [30, 31]. The rest of this paper is about the last property: knowing which values can be trusted, and how far.
Case study: electro-optic materials
Electro-optic materials serve here as a worked example; the approach does not depend on them. An electro-optic material changes how it bends light when a voltage is applied. That simple effect is how electrical signals are written onto light: it drives the optical modulators that carry data through fibre networks and data centres, and, at cryogenic temperatures, it can link superconducting quantum processors to optical fibre [78, 74, 75, 76, 77] (Figure 3a). Our motivating application is therefore the discovery of materials with a large linear electro-optic (Pockels) response, which allows lower drive voltages and less energy per bit. In the Pockels effect an applied electric field changes the refractive index of a crystal in proportion to the field. The change is described by the Pockels (electro-optic) tensor , defined by , where is the refractive index, the indices label the crystal axes, and is usually given in pm/V. It is a rank-three polar tensor whose allowed non-zero components are fixed by crystal symmetry [32].
Its value depends on whether the crystal is mechanically clamped or free, on wavelength and temperature, and, in thin films, on strain and domain structure. No curated database of measured Pockels coefficients exists. The values are scattered across five decades of optics, ferroelectrics, and thin-film literature, written in at least three notations: the full-tensor form ; the compressed Voigt form , in which the symmetric index pair is replaced by one index from 1 to 6 (, , , , , ); and the alternative symbol for the same quantity. A superscript (constant strain, or clamped) or (constant stress, or unclamped) states the mechanical boundary condition, and it is often dropped. These and all other symbols are collected in § 9.
We built a pipeline to recover these values [28, 29] (Figure 2). After an acquisition step (Stage 0) finds the papers, four stages process them: (i) layout-aware extraction that recovers tables and text together with the subject material and the measurement conditions, using an LLM where contextual reasoning cannot be avoided; (ii) a three-tier verification gate (accepted, review, rejected) that encodes symmetry and provenance rules; (iii) cross-paper consensus with a robust rule based on the median and the median absolute deviation (MAD; § 4.3); and (iv) noise-robust modelling, audited by leave-one-material-out generalisation, in which each material in turn is left out of training and then predicted. The pipeline follows one rule: no value is better than a wrong value. Over the campaign, staged runs over 49 distinct publications produced a 448-record consensus database across 22 inorganic materials, and a forward surrogate (a machine-learning model that predicts the property from descriptors of the material and the measurement conditions) trained on it generalised to nine of thirteen held-out materials [29].
[x=1mm, y=1mm,
every node/.append style={execute at begin node={\hyphenpenalty=10000\relax}},
st/.style={ddmethod, text width=26mm, minimum height=19mm, inner sep=3pt},
outp/.style={ddresult, text width=26mm, minimum height=12mm, inner sep=3pt, font=\scriptsize},
hd/.style={font=\footnotesize\bfseries, text=DDNavy}]
% stages
\node[st] (s0) at (14,40) {\textbf{Stage 0}\\Acquisition\\[0.6mm]{\scriptsize search and rank papers from bibliographic services}};
\node[st] (s1) at (47,40) {\textbf{Stage 1}\\Extraction\\[0.6mm]{\scriptsize layout-aware parsing; LLM (GPT-4o) reads values and conditions}};
\node[st, ddcontrol, text width=26mm, minimum height=19mm] (s2) at (80,40)
{\textbf{Stage 2}\\Verification gate\\[0.6mm]{\scriptsize symmetry, unit, label, and provenance rules}};
\node[st] (s3) at (113,40) {\textbf{Stage 3}\\Consensus\\[0.6mm]{\scriptsize median and MAD across papers (\cref{eq:mad})}};
\node[st] (s4) at (146,40) {\textbf{Stage 4}\\Modelling\\[0.6mm]{\scriptsize noise-robust surrogate, leave-one-material-out audit}};
\foreach \a/\b in {s0/s1, s1/s2, s2/s3, s3/s4} \draw[ddflow] (\a) -- (\b);
% outputs
\node[outp] (o0) at (14,17) {49 PDFs\\(3 out of scope)};
\node[outp] (o1) at (47,17) {816 candidate values from 43 papers};
\node[outp, text width=26mm] (o2) at (80,17)
{{\color{DDGreen!70!black}\textbf{414} accepted}\\{\color{DDGold!60!black}\textbf{215} held}\enspace{\color{DDRed!80!black}\textbf{187} rejected}};
\node[outp] (o3) at (113,17) {448 consensus records, 22 materials};
\node[outp] (o4) at (146,17) {predicts 9 of 13 held-out materials};
\foreach \i in {0,...,4} \draw[ddflow, draw=DDTeal!70] (s\i.south) -- (o\i.north);
% gold standard
\node[ddevidence, text width=58mm, minimum height=9mm, font=\scriptsize] (gold) at (63.5,1)
{Certified gold standard: 303 values from 16 papers, read by hand\\
$\Rightarrow$ precision 0.51, recall 0.48 of the production run};
\draw[ddreturn, dashed] (gold.north -| o1.south) -- (o1.south);
\draw[ddreturn, dashed] (gold.north -| o2.south) -- (o2.south);
\node[hd, anchor=west] at (0,53.5) {Each stage passes on only what it can support};
\node[font=\scriptsize\itshape, text=DDInk, anchor=west] at (95,1)
{held and rejected values are logged with a reason, never deleted};Figure 2. The electro-optic data pipeline used as our case study [28, 29]. Top row: the five stages. Middle row: what each stage produced. The counts for Stages 0–2 come from the latest production run, which is analysed in sec:gate,sec:audit; the consensus database and the leave-one-material-out result in Stages 3–4 summarise the whole campaign. The certified gold standard (bottom) is read by hand, independently of the pipeline, and measures how good the extracted values are.
The latest production run of this pipeline [29] is the data set analysed in sec:gate,sec:audit. Forty-nine PDFs were processed with a general-purpose LLM (GPT-4o) at a cost of under ten US cents per paper [29]. Three papers were withheld at detection as out of scope (hybrid or organic perovskites, or a multi-material survey), and 43 papers yielded 816 candidate values. Of these, 414 were accepted, 215 were held for review, and 187 were rejected, each with a typed reason. Two references let us measure how good the result is. The first is a hand-curated set of tensor values from benchmark papers [33, 34, 35]. The second is a certified gold standard of 303 values from 16 papers, read by hand and independent of any extraction pipeline [29]. Against the gold standard, the production configuration reaches a precision of 0.51 and a recall of 0.48 at the level of individual target values. A diagnostic pass on an earlier run showed that numeric precision was much higher (0.74) and that nearly half of the false positives were the right number stored in the wrong slot: the wrong tensor label, boundary condition, or contribution tier. These internal benchmarks are the first measured error rates for our extractor, and § 5 builds on them.
The same need in other fields
The electro-optic case is typical, not special. tab:fields lists domains in which the property of interest is mostly reported in the literature, a comprehensive database is missing or incomplete, and the reporting conventions invite the same classes of error. What these fields share is the reason this paper exists. Literature mining lets them build datasets that would otherwise not exist. They then inherit error processes that the literature and the extractor introduce together, and an unguarded model cannot tell those errors from real physics.
Figure 3 gives one schematic for each field of tab:fields. In electro-optics, a voltage across a crystal shifts the phase of light, and the size of the shift is set by the Pockels coefficient. In energy storage, ions move between two electrodes while electrons flow through the external circuit, and the measured capacity is limited by how much charge the electrode material can hold. In drug delivery, nanoparticles carry a drug to target cells, and the reported numbers describe how much drug they hold and how fast they release it. In mechanics, a tensile test pulls a sample until it yields and breaks. In superconductivity, the electrical resistance falls to zero below a critical temperature. In thermophysics, the pressure, temperature, and heat of a phase change are tied together by thermodynamics. In every panel the same pattern holds: a number in a paper only means something together with its conditions, and a law of physics limits which numbers are possible.
[x=1mm, y=1mm, font=\scriptsize,
every node/.append style={execute at begin node={\hyphenpenalty=10000\relax}},
frame/.style={draw=DDRule, rounded corners=1.5mm, line width=0.5pt, fill=white},
ttl/.style={anchor=north west, font=\footnotesize\bfseries, text=DDNavy},
chk/.style={anchor=south west, text width=49mm, align=left, text=DDInk, font=\scriptsize},
lab/.style={text=DDInk, align=center, font=\scriptsize}]
%% ===================== (a) electro-optics ================================
\begin{scope}[shift={(0,62)}]
\draw[frame] (0,0) rectangle (53,58);
\node[ttl] at (1.5,57) {(a) Electro-optics};
\fill[DDTeal!12] (12,26) rectangle (40,38);
\draw[DDTeal] (12,26) rectangle (40,38);
\fill[DDGold!65] (12,38) rectangle (40,40.2);
\fill[DDGold!65] (12,23.8) rectangle (40,26);
\node[lab, text=DDGold!40!black] at (26,43) {$+V$};
\foreach \x in {16,22,...,36} \draw[-{Stealth[length=1.1mm]}, DDOrange!85] (\x,37.4) -- (\x,33.2);
\node[lab, text=DDTeal!70!black] at (26,29) {crystal, $r_{ij}$};
\draw[DDRed!80, line width=0.7pt, domain=2:11, samples=60, smooth] plot (\x,{32+2.2*sin(\x*120)});
\draw[DDPurple, line width=0.7pt, domain=41:51, samples=60, smooth] plot (\x,{32+2.2*sin(\x*120-80)});
\node[lab] at (6.5,20) {light in};
\node[lab] at (46,20) {phase\\shifted};
\node[chk] at (1.5,1.5) {\textbf{Mined:} Pockels coefficient $r_{ij}$ (pm/V), clamped or unclamped.\\
\textbf{Check:} symmetry allows only some $r_{ij}$; clamped and unclamped values obey \cref{eq:identities}.};
\end{scope}
%% ===================== (b) batteries =====================================
\begin{scope}[shift={(56,62)}]
\draw[frame] (0,0) rectangle (53,58);
\node[ttl] at (1.5,57) {(b) Energy storage};
\fill[DDGray!35] (8,20) rectangle (18,42); \draw[DDGray] (8,20) rectangle (18,42);
\fill[DDNavy!15] (35,20) rectangle (45,42); \draw[DDNavy!60] (35,20) rectangle (45,42);
\draw[DDMeta, dashed] (26.5,20) -- (26.5,42);
\node[lab] at (13,17) {anode};
\node[lab] at (40,17) {cathode};
\node[lab] at (26.5,17) {separator};
\foreach \y in {26,31,36} {
\node[circle, fill=DDTeal!70, inner sep=0pt, minimum size=1.8mm] at (22,\y) {};
\draw[-{Stealth[length=1.1mm]}, DDTeal] (23.5,\y) -- (31,\y);
}
\node[lab, text=DDTeal!70!black] at (26.5,40) {Li$^{+}$};
\draw[DDInk, line width=0.5pt] (13,42) -- (13,47) -- (40,47) -- (40,42);
\draw[-{Stealth[length=1.3mm]}, DDOrange!85, line width=0.6pt] (22,47) -- (31,47);
\node[lab, anchor=south] at (26.5,47.4) {e$^{-}$ through the load};
\node[chk] at (1.5,1.5) {\textbf{Mined:} specific capacity (mAh\,g$^{-1}$), cycle life, conductivity.\\
\textbf{Check:} capacity cannot exceed the Faraday limit $nF/(3.6M)$ of the active material.};
\end{scope}
%% ===================== (c) drug delivery =================================
\begin{scope}[shift={(112,62)}]
\draw[frame] (0,0) rectangle (53,58);
\node[ttl] at (1.5,57) {(c) Drug delivery};
% nanoparticle
\fill[DDPurple!12] (15,32) circle (7mm); \draw[DDPurple!70] (15,32) circle (7mm);
\foreach \a in {0,45,...,315} {
\draw[DDPurple!70] ($(15,32)+(\a:7mm)$) -- ($(15,32)+(\a:9.5mm)$);
\fill[DDPurple!70] ($(15,32)+(\a:9.9mm)$) circle (0.5mm);
}
\foreach \p in {(13,34),(17,35),(15,30),(12,30),(18,31),(15,33.5)}
\fill[DDRed!75] \p circle (0.7mm);
\node[lab] at (15,18) {nanoparticle\\carrying drug};
% cell membrane
\draw[DDGreen!60!black, line width=0.9pt] (38,16) .. controls (34,26) and (34,38) .. (38,48);
\foreach \y in {24,32,40} \draw[DDGreen!60!black, line width=0.6pt] (35.2,\y) -- (32.5,\y) -- (31.8,\y+1) (32.5,\y) -- (31.8,\y-1);
\node[lab, text=DDGreen!50!black] at (45,32) {target\\cell};
\draw[-{Stealth[length=1.3mm]}, DDInk] (25.6,32) -- (30.6,32);
\node[chk] at (1.5,1.5) {\textbf{Mined:} encapsulation efficiency, release, IC$_{50}$ (nM or \textmu M).\\
\textbf{Check:} efficiencies lie in 0--100\%; $\mathrm{SEM}=\mathrm{SD}/\sqrt{n}$.};
\end{scope}
%% ===================== (d) mechanics =====================================
\begin{scope}[shift={(0,0)}]
\draw[frame] (0,0) rectangle (53,58);
\node[ttl] at (1.5,57) {(d) Mechanics};
\draw[-{Stealth[length=1.3mm]}, DDInk] (8,18) -- (49,18) node[below left, lab] {strain};
\draw[-{Stealth[length=1.3mm]}, DDInk] (8,18) -- (8,50) node[below right, lab] {stress};
\draw[DDNavy, line width=0.8pt] (8,18) -- (14,38) .. controls (18,42) and (30,47) .. (36,46) .. controls (40,45) and (43,41) .. (45,36);
\fill[DDOrange] (14,38) circle (0.8mm);
\node[lab, anchor=west, text=DDOrange!60!black] at (15,36) {yield};
\fill[DDRed!75] (36,46) circle (0.8mm);
\node[lab, anchor=south, text=DDRed!70!black] at (36,46.8) {ultimate};
\draw[DDTeal!80, dashed, line width=0.6pt] (36,46) .. controls (40,48) and (43,49) .. (46,49.5);
\node[lab, anchor=west, text=DDTeal!70!black] at (46.3,49.5) {true};
\node[chk] at (1.5,1.5) {\textbf{Mined:} yield and ultimate strength, hardness.\\
\textbf{Check:} ultimate $\ge$ yield; engineering and true stress must not be mixed.};
\end{scope}
%% ===================== (e) superconductors ===============================
\begin{scope}[shift={(56,0)}]
\draw[frame] (0,0) rectangle (53,58);
\node[ttl] at (1.5,57) {(e) Superconductors};
\draw[-{Stealth[length=1.3mm]}, DDInk] (8,18) -- (49,18) node[below left, lab] {temperature $T$};
\draw[-{Stealth[length=1.3mm]}, DDInk] (8,18) -- (8,50) node[below right, lab] {resistance};
\draw[DDNavy, line width=0.8pt] (8,18.2) -- (22,18.2) .. controls (26,18.2) and (27,42) .. (31,43) -- (47,46);
\foreach \x/\c/\t in {23/DDTeal/zero, 26.2/DDGold/mid, 30/DDRed/onset} {
\draw[\c, dashed] (\x,18) -- (\x,48);
}
\node[lab, anchor=south, text=DDTeal!70!black] at (20.5,48) {zero};
\node[lab, anchor=south, text=DDGold!50!black] at (26.2,48) {mid};
\node[lab, anchor=south, text=DDRed!70!black] at (32.5,48) {onset};
\node[chk] at (1.5,1.5) {\textbf{Mined:} critical temperature $T_c$, critical field.\\
\textbf{Check:} the three criteria give different $T_c$; the criterion and pressure must be recorded.};
\end{scope}
%% ===================== (f) thermophysics =================================
\begin{scope}[shift={(112,0)}]
\draw[frame] (0,0) rectangle (53,58);
\node[ttl] at (1.5,57) {(f) Thermophysics};
\draw[-{Stealth[length=1.3mm]}, DDInk] (8,18) -- (49,18) node[below left, lab] {temperature $T$};
\draw[-{Stealth[length=1.3mm]}, DDInk] (8,18) -- (8,50) node[below right, lab] {pressure $P$};
\draw[DDNavy, line width=0.8pt] (12,20) .. controls (25,23) and (35,32) .. (44,45);
\fill[DDNavy] (44,45) circle (0.8mm);
\node[lab, anchor=west] at (38,48) {critical point};
\node[lab, text=DDTeal!70!black] at (18,40) {liquid};
\node[lab, text=DDOrange!60!black] at (42,23) {vapour};
\draw[DDRed!75, line width=0.6pt] (26.6,25.3) -- (38.6,36.6);
\fill[DDRed!75] (32.6,31.45) circle (0.6mm);
\node[lab, text=DDRed!70!black, anchor=north west] at (36.5,32.5) {slope $\mathrm{d}P/\mathrm{d}T$};
\node[chk] at (1.5,1.5) {\textbf{Mined:} vapour pressure, density, latent heat.\\
\textbf{Check:} Clapeyron, $\mathrm{d}P/\mathrm{d}T = L/(T\Delta v)$, ties the slope to the latent heat.};
\end{scope}Figure 3. Six fields in which AI models rely on values mined from papers (tab:fields). Each panel shows the physics behind the mined quantity and one check that exposes a wrong value without any comparison data. (a) In an electro-optic crystal an applied voltage changes the refractive index and so shifts the phase of light passing through it (Pockels effect; [32, 78]). (b) In a lithium-ion cell, lithium ions move through the electrolyte while electrons flow through the external circuit; the charge the active material can store sets an upper limit on the specific capacity (Faraday’s law; [83]). (c) A drug-carrying nanoparticle with surface ligands binds to receptors on a target cell [85]; encapsulation and release fractions are bounded by 0 and 100%. (d) A tensile test gives yield and ultimate strengths; engineering stress (force over original area) and true stress (force over current area) differ at large strain [37]. (e) The superconducting transition has a width, so the onset, midpoint, and zero-resistance criteria give different values of [38]. (f) On a phase diagram the Clapeyron equation links the slope of the liquid–vapour line to the latent heat and the volume change [90].
| DDBand DDNavyField | DDNavyTypical targets | DDNavyContext that must travel with the value | DDNavyCharacteristic hazards |
| Electro-optics, piezo- and ferroelectrics | ,,,, | Tensor component, clamped or unclamped, wavelength, temperature, strain, film or bulk | Voigt versus full-tensor factors of two,versusnotation, symmetry-forbidden components, e-11C/N versus pm/V |
| Energy storage | Capacity, Coulombic efficiency, cycle life, ionic conductivity, voltage | Loading, C-rate, electrolyte, temperature, cell format, reference electrode | mAh gversus mAh cm, active-mass versus electrode-mass normalisation, S cmversus mS cm, values read from plots |
| Medicine and drug delivery | IC,, encapsulation efficiency, release half-life, dose | Assay, cell line, species, dose basis, formulation, time point | nM versus M, pICversus IC, SD versus SEM, mg kgversus mg m, arm-level versus patient-level values |
| Mechanics and structural alloys | Yield and ultimate strength, hardness, fatigue life, toughness | Processing, grain size, strain rate, temperature, test standard | MPa versus GPa versus ksi, engineering versus true stress, HV versus GPa hardness, compression versus tension |
| Superconductors and quantum materials | , gap, critical field, coherence time | Doping, pressure, onset versus midpoint versus zero-resistance criterion | Criterion ambiguity, pressure-dependent values reported without the pressure |
| Chemistry and thermophysics | Vapour pressure, density, solubility, reaction yield | Temperature, pressure, purity, phase | Thermodynamic inconsistency, unit and basis confusion (mole versus mass) |
Sources of Error in Literature-Mined Data
An error in a mined value is any difference between the stored record and what the source actually established. Errors should be classified by the signature they leave, not only by their cause, because the signature decides which detector can see them. We group errors by the pipeline stage at which they arise (Figure 1, left) and illustrate most of them with errors met in the electro-optic campaign (tab:taxonomy; [29]). tab:record first shows what a single record has to carry, and so how many places there are for an error to enter.
A taxonomy by pipeline stage
| DDBand DDNavyField | DDNavyExample | DDNavyHow it can go wrong |
| Value and unit | 105 pm/V | OCR digit, lost decimal point, unit or scale slip |
| Property and label | Pockels coefficient,(in Voigt notation) | Label swap (stored as), symmetry-forbidden component |
| Boundary condition | Unclamped (, constant stress) | Superscript dropped; clamped and unclamped mixed |
| Material and phase | BaTiO, tetragonal, point group | Comparison material attributed to the subject; wrong phase |
| Sample form | Bulk single crystal | Film and bulk values mixed |
| Conditions | Temperature, wavelength, strain, as stated in the paper | Condition lost or misparsed (``5 kV/cm'' read as 5 K) |
| Uncertainty | Reported standard uncertainty | Index or subscript stored as uncertainty |
| Origin | Measured in this paper, not cited | Cited value counted as a new measurement |
| Provenance | Paper, table, cell, and verbatim snippet | Missing; the value cannot be checked or traced |
Source-level errors
Some values are already wrong in the published paper, before any software reads it: typographical errors, values miscopied from earlier work, figures whose axes disagree with the text, and, rarely, fabricated data [39]. A related and much more common hazard is the cited value: a paper quotes a well-known constant from earlier literature in order to compare with its own result. If the cited value is extracted as a new measurement, it is double-counted, and it is often attached to the citing paper’s own material and conditions.
Conversion errors
Turning a PDF into text damages information in well-documented ways [40, 41]. Characters are confused by optical character recognition (1 with 7, 3 with 8, 0 with O, 5 with 6). Decimal points and minus signs are lost. Subscripts and superscripts collapse into the main line, so becomes and becomes . Two-column layouts are read across columns, table rows break apart, and scans of older papers may give no usable text at all. In our own early pipeline, a text-normalisation step meant to repair spacing instead broke words and formulas apart, turning the journal name “PHYSICAL REVIEW” into “PHYS ICAL REVI EW” and the formula BaTiO into “Ba Ti O”, and every later stage worked on the damaged text [29].
Extraction and interpretation errors
Even on clean text, a number has to be bound to the right label, unit, material, and component. Rule-based extractors bind by proximity, so they attach numbers to whatever label is nearby. We found a tensor subscript (1133) stored as the uncertainty of the preceding value, and phonon mode indices (“modes 4–6”) stored as a Pockels coefficient because the word “Pockels” was in the same sentence [29]. LLMs bind by meaning but make errors of their own. They swap closely related quantities (an electro-optic coefficient labelled as a piezoelectric coefficient , which instead gives the strain produced by an electric field), assign a comparison material’s value to the paper’s subject material, and occasionally produce a plausible value that is not in the source at all [14].
Context errors
The number can be right while the metadata attached to it is wrong. A coefficient measured just above a phase transition, at cryogenic temperature, or in a strained film may be ten or a hundred times its room-temperature bulk value. If the condition is lost, a correct measurement becomes a wrong data point. We found a temperature of “5 K” that had been parsed from the phrase “5 kV/cm” [29]. In a later stage we found 51 strain-resolved records whose strain labels had been dropped during harmonisation [42]; the values were intact, and what had been lost was the information that made them usable. In the latest corpus, after the extraction was revised, the number of records with an exact strain rose from 32 to 48 [29].
Recording conditions in more detail brings new hazards of its own. Once the extractor was asked to keep conditions as the paper words them, and not only as bare numbers, the step that turns those words back into numbers made new mistakes. It turned “below 50 K” into exactly 50 K, “partially relaxed” into a strain of exactly 0%, and “0.88% biaxial compressive” into a positive (tensile) strain, and it filed an electrical state, “partially poled”, in the strain column [29].
The most important form of context loss cannot be seen by looking at one record at a time. Many electro-optic records sit in groups that share the material, tensor component, source paper, and every recorded condition, yet carry different values (§ 4.3). Each value may be a correct measurement taken from the paper. What is missing is the condition that explains why the values differ (for example the crystal phase, an applied field, or the lattice constant of the film), because it was never recorded. A model therefore sees identical inputs paired with different answers, and it cannot learn from them.
Harmonisation errors
Merging values from many papers requires common units and notations, and conversion is itself a source of error. Unit conversions can fail silently: in our first pipeline the unit label was changed while the number stayed the same. Notation conventions differ by exact factors. In Voigt notation (§ 2.2) the shear piezoelectric coefficient equals twice the full-tensor component , because the Voigt shear strain counts the two equal tensor components and together. A value moved between the two conventions without the factor is therefore wrong by exactly two.
Symmetry folding brings a subtler hazard. Crystal symmetry makes some components equal; in a tetragonal crystal, for example, the and axes are equivalent, so . Mapping such symmetry-equivalent labels to one canonical label (here to ) is right for scoring, but in our corpus it also made 12 records count the same measurement twice, because a paper listed both equivalent components [29].
A record can also disagree with itself. In one elastic stiffness constant the stored text field read 783 GPa while the numeric field correctly held 7.83 GPa, and for five KNbO stiffness constants the numeric field was a factor of 100 too small after conversion from × 1011 N m−2 [34, 29]. Duplicates arise when the same paper is collected under two filenames or when one measurement is quoted in many later papers.
| DDBand DDNavyError class | DDNavyObserved instance (electro-optic campaign) | DDNavySignature in the data | DDNavyDetected by |
| Unit or scale | KNbOpiezoelectricreported in e-11C/N stored as pm/V; stiffness a factor of 100 too small after conversion from e11N m^-2 | Decade-quantised shift; often outside physical bounds | L0, L1, L2 (repairable) |
| OCR digit and decimal | Formula and word fragments (``Ba Ti O''); in one table, ``rS131'' read as ``rS31'' and ``rS113'' as ``rS411'' | Arbitrary multiplicative shift, sometimes decade-quantised | L2, L3; L1 if coupled |
| Label or component | Pockelsstored as piezoelectric; tensor subscript 1133 stored as an uncertainty; mode indices stored as a coefficient | Value typical of a different quantity; violates a governing equation | L1, L3 |
| Symmetry-forbidden | ,,,for zinc-blende InP ;andforKNbO;forBaTiO | Non-zero component forbidden by the point group | L1 (deterministic) |
| Number look-alike | A CdTe ``r_{41}=\SI{110}{pm/V}'' that was the crystal direction | Order-of-magnitude outlier against chemical neighbours | L1 bounds, L2, L3 |
| Material misattribution | Relaxor 0.88 Pb(ZnNb)O--0.12 PbTiOvalues assigned to PbTiO; every record of a batch stamped with a default material | Plausible magnitude, wrong identity; often in range | L2, L3; L0 provenance |
| Context loss | ``5 kV/cm'' parsed as 5 K; ``below 50 K'' stored as 50 K; 51 strain-resolved records with strain labels dropped; records indistinguishable by their recorded conditions | Correct number, wrong or missing condition; identical inputs with different values | L0 completeness, L2 conflict (), L3 |
| Notation convention | used for(same quantity, different symbol); Voigtversus | Exact factor (1 or 2) or a false ``unknown label'' | L0 symbol map, L1 identities |
| Cited value | GaAsquoted for comparison in a paper on another material | Duplicate of a canonical value, attached to the wrong paper | L0 provenance, L2 duplicates |
| Hallucination | No confirmed case; three values held because the number was not verbatim in its source snippet | Plausible, in range, unsupported by any snippet | L0 snippet check, L3, L4 |
Random versus systematic errors
For detection, the most useful distinction is whether an error is random or systematic. A random error, such as a single OCR slip or a single mislabelled cell, is independent of the errors in other papers. When several independent sources report the same quantity, a random error lies away from the cluster of correct values and is outvoted. A systematic error repeats: a unit convention misread identically in every paper that uses it, or a wrong value propagated through a chain of citations. Such errors can outnumber the correct values locally and pull the consensus towards themselves. No purely statistical detector can separate a systematic error that has become the majority from the truth; that requires physics (§ 4.2) or provenance (§ 4.1).
Two further regularities help. First, many errors are multiplicative: unit prefixes, lost decimal points, and notation factors multiply the value by a constant. In logarithmic space they become additive shifts, and many of those shifts are quantised: close to an integer number of decades, or for the Voigt factor of two. Second, many errors are structured by provenance. They cluster by paper, by extraction route (text, table, or scanned image), and by publication era, so a provenance tag is a strong prior on error probability.
What to expect in other fields
The same five stages produce errors specific to each field.
In energy storage, specific capacity is normalised by active mass in one paper and by electrode mass or area in another, cycle life depends on the end-of-life criterion, and many values are read off plots, which adds digitisation error [10].
In medicine and drug delivery, potencies such as the half-maximal inhibitory concentration IC are reported in nM, μM, or as negative logarithms (pIC); dispersion is reported as the standard deviation (SD) or the standard error of the mean (SEM), which differ by a factor for a sample of subjects [89]; and doses are normalised by body mass or body surface area. Combining assays from different sources adds variance of the size of the effects being modelled [18, 19], extraction errors in meta-analyses are frequent [22, 23], and automating the extraction speeds up systematic reviews but does not by itself guarantee correctness [43, 44].
In mechanics, engineering and true stress differ increasingly at large strain, hardness is reported on several scales, and imperial units persist in older alloy data [37].
In structural and chemical databases, missing hydrogens, disorder, and residual solvent produce structures that are chemically impossible although they parse as valid files [24, 45].
The details differ from field to field. The signatures do not, and the detectors of the next section are built around signatures.
How to Detect Erroneous Data
No single detector sees every error class in tab:taxonomy. We therefore organise detection as a stack of layers (Figure 1, centre), ordered by cost and by what they need as input. The cheap, deterministic layers go first and need no data beyond the record itself. The statistical and model-based layers need many records. Human review, the most expensive layer, is kept for the short list of values the other layers could not settle. Throughout, a record is a value with its label, units, material, conditions, and provenance, and we work in logarithmic space, , wherever the quantity spans decades, so that multiplicative errors become additive shifts. A shift of is one decade, a factor of ten in . All symbols are collected in tab:symbols (§ 9). tab:layers summarises the five layers before each is described in turn.
| DDBand DDNavyLayer | DDNavyWhat it checks | DDNavyRule or score | DDNavyNeeds as input | DDNavyCatches | DDNavyMisses |
| L0 Schema and provenance | Label, unit dimension, required fields, verbatim snippet, own versus cited value | Controlled vocabulary; snippet match | One record and its source text | Malformed records, hallucinated values, cited values | Well-formed wrong numbers |
| L1 Physics | Hard bounds, crystal symmetry, governing equations | , ; | One record, or several values from one paper | Impossible values, forbidden components, unit slips, even in values reported once | Errors smaller than the measurement uncertainty |
| L2 Consensus | Agreement between independent reports of the same quantity | Robust, ;;, | Three or more reports | Random errors, decade shifts (repairable), mixed conditions | Copied errors that form a majority; values reported once or twice |
| L3 Model audit | Agreement with what similar materials predict | Suspicion, | A trained model and descriptors | Outliers among chemical neighbours, including singletons | Cannot tell an error from a real anomaly |
| L4 Review | The short list the other layers cannot settle | Combined score, | A person and the stored provenance | Whatever the evidence in the source decides | Limited by budget |
Layer 0: schema, units, and provenance
The first layer checks that a record is well formed before asking whether it is right. Is the label in the controlled vocabulary for its property family? Do the units have the right dimension (a Pockels coefficient must be a length per voltage)? Are the required context fields present, and is the value supported by a verbatim snippet in the source? The snippet check is the cheapest defence against hallucination: a value that cannot be found in the source text is held, however plausible it looks. Symbol maps belong here too. Reading as , and mapping clamped and unclamped superscripts to a boundary-condition field, recovered a complete five-component KTiOPO tensor that a literal reader would have discarded [29]. Layer 0 also records whether a value is the paper’s own result or a cited one, and it removes byte-identical duplicate documents. Finally, it checks that a record agrees with itself: the value string, the parsed number, and the converted number must describe the same quantity. In our corpus this simple test found stored values that differed from their own source text by a factor of 100.
Layer 1: physics
Physics is the only layer that can reject a value without comparing it with other data. That makes it the only protection for values reported once, and the only protection against systematic errors that have become the majority. We distinguish four strengths of physical constraint.
Hard admissibility bounds
Many quantities have strict limits: a refractive index and a static relative permittivity cannot be below one, the elastic stiffness matrix of a mechanically stable crystal must be positive definite (all its eigenvalues positive) [84], the Poisson ratio (lateral contraction per unit axial extension) of an isotropic stable solid lies in , and an efficiency cannot exceed its thermodynamic limit. Below these are softer plausibility ranges set by the known range of the property family. A value outside a hard bound is rejected; a value outside a plausibility range is held.
Symmetry
Neumann’s principle requires every physical property tensor to be invariant under the point group of the crystal, the set of rotations and reflections that leave the crystal unchanged [32]. The allowed non-zero components are therefore known in advance. Point groups are written here in the standard Hermann–Mauguin notation: is the cubic zinc-blende class (GaAs, InP, CdTe), the tetragonal class of BaTiO at room temperature, the orthorhombic class of KNbO, and the trigonal class of LiNbO. For the zinc-blende class the only independent linear electro-optic component is (Voigt notation, § 2.2). A reported for InP is not a surprising measurement; it is an extraction error, and it can be rejected deterministically. The same logic applies to piezoelectric, elastic, and dielectric tensors and to any quantity constrained by symmetry.
Two cautions come from our own corpus. First, the rule has to be applied to every crystal class, not only the one that motivated it: a gate that enforced the zinc-blende rule alone let forbidden labels for , , and crystals through (§ 5.2). Second, the allowed set depends on the phase and on the axis setting, not on the material name alone. BaTiO appears in our corpus as tetragonal crystals, rhombohedral first-principles models, and monoclinic films. For crystals, the allowed shear-type components are or depending on which axis the mirror plane is perpendicular to. A symmetry violation under an unstated setting should be held for review, not rejected.
Governing equations and constitutive identities
The strongest physical check uses relations that connect several independently reported quantities. Let \vect{v}=(v_1,\dots,v_K) be a set of values reported in one paper, or assembled for one material under matched conditions, and let g(\vect{v})=0 be a physical identity that the true values satisfy exactly. Because measured values carry uncertainty, evaluated on the reported values is not exactly zero, and the question is whether it is further from zero than the uncertainties allow. The normalised physics residual answers it:
where is the reported standard uncertainty of and is the resulting standard uncertainty of , obtained in practice by Monte Carlo propagation: every input is redrawn many times from its uncertainty distribution, and the spread of the resulting values of is taken as [92]. A record set with , a residual more than three standard uncertainties from zero, contains at least one wrong value. When one value appears in several identities, the pattern of violated identities points to which value it is.
Electro-optic crystals, our case study, offer a set of such identities. A crystal with a Pockels effect lacks a centre of symmetry, and every such crystal is also piezoelectric: an electric field deforms it, and a deformation creates an electric polarisation [32]. Because of this electromechanical coupling, its electro-optic, dielectric, piezoelectric, and elastic constants are not independent. A crystal can be measured either free to deform (unclamped, at constant stress, superscript ) or held rigid (clamped, at constant strain, superscript ). The two sets of electro-optic and dielectric tensors differ because a free crystal also deforms piezoelectrically, and that deformation changes its optical and dielectric response. The difference is fixed by the piezoelectric and elasto-optic tensors,
and the piezoelectric strain and stress tensors are related through the compliance, [32]. Here is the piezoelectric strain coefficient (strain per unit electric field, in pm/V), the piezoelectric stress coefficient (stress per unit field, in C/m), the elasto-optic tensor (change of per unit strain), the relative permittivity (dielectric constant), the permittivity of vacuum, and the elastic compliance, the inverse of the elastic stiffness . The superscript means that these quantities are taken at constant electric field. Indices run over the three crystal axes, and a repeated index is summed over. We test these identities on real data in § 5.3.
Similar relations exist in every field (tab:equations). In chemistry, charge and mass must balance. In batteries, the Nernst equation ties an electrode potential to concentration [91], and stored energy must equal the integral of voltage over capacity. In thermophysics, the Gibbs–Duhem equation links the chemical potentials of the components of a mixture, and the Clapeyron equation ties the slope of a phase boundary to the latent heat [90, 20]. In optics and electronics, the Kramers–Kronig relations make the real and imaginary parts of a response function (for example the refractive index and the absorption) determine each other [88]. In transport, Onsager reciprocity requires the matrix of coupled transport coefficients to be symmetric [79]. tab:equations collects these and other relations, with the error each one exposes.
| DDBand DDNavyField | DDNavyRelation | DDNavyEquation | DDNavyA violation reveals | DDNavyType |
| Electro-optics | Clamped and unclamped | Boundary condition swapped; wrong,, or | hard | |
| Piezoelectricity | Strain and stress coefficients | Unit slip inor; Voigt factor of two | hard | |
| Dielectrics, optics | Index and permittivity | at optical frequency;, | Mixed static and optical values; impossible value | hard |
| Optics | Kramers--Kronig | Index and absorption from different samples or wrong units | hard | |
| Crystals | Symmetry (Neumann) | Tensor invariant under the point group | Forbidden component, wrong label | hard |
| Elasticity | Mechanical stability | Stiffness matrixpositive definite; | Mislabelled or rescaled constants | hard |
| Elasticity | Voigt--Reuss bounds | , | Polycrystal modulus inconsistent with single-crystal data | hard |
| Batteries | Faraday limit | mAh g(in g mol) | Wrong mass basis or material | hard |
| Batteries | Energy and efficiency | ; Coulombic efficiency | Inconsistent energy, voltage, or capacity | hard |
| Electrochemistry | Nernst | Wrong reference electrode or temperature | hard | |
| Thermodynamics | Clapeyron | Inconsistent vapour pressure and latent heat | hard | |
| Thermodynamics | Gibbs--Duhem | at fixed, | Inconsistent activity or mixture data | hard |
| Transport | Onsager reciprocity | Wrong coupled coefficient (e.g. Seebeck--Peltier) | hard | |
| Medicine | Summary statistics | ; | SD and SEM swapped; nM and M confused | hard |
| Chemistry | Conservation | Mass and charge balance; fractions in | Wrong stoichiometry or normalisation | hard |
| Alloys | Vegard's law | Wrong composition or phase | soft | |
| Mechanics | Hall--Petch | Grain size or strength in wrong units | soft | |
| Transport | Arrhenius | Temperature in C read as K; outlier point | soft | |
| Nonlinear optics | Miller's rule | Order-of-magnitude outlier | soft |
Empirical laws as soft constraints
Many relations hold approximately. Miller’s rule states that a second-order nonlinear optical coefficient is roughly proportional to the product of the linear susceptibilities involved [69]. The Moss relation links a high refractive index to a small band gap [46]. Vegard’s law makes the lattice parameter of an alloy vary linearly with composition [86], the Hall–Petch relation makes yield strength rise as the inverse square root of grain size [80, 81], and Arrhenius behaviour makes the logarithm of conductivity fall linearly with inverse temperature [87]. Their scatter is too large to reject a value, but large enough deviations justify review. Such a law should be used with a threshold calibrated on its known scatter, never as a hard filter, because the genuinely anomalous material is exactly what a discovery campaign is looking for.
Layer 2: cross-source statistical consensus
When the same quantity (same material, component, boundary condition, and matched conditions) has been reported times, the correct values lie close together around the true value, spread only by the usual differences between laboratories, while random errors land far from this cluster (Figure 4). The mean and the standard deviation are poor tools for finding the outliers, because the outliers themselves pull them. For the log-values of a group we therefore use the median and the median absolute deviation (MAD). These barely change when a few values are wildly wrong: more than half of the values in the group would have to be wrong before the median could be pulled far from the cluster [47, 48, 49]:
where is the robust score of record (Figure 4). The constant makes read like an ordinary -score (a distance in standard deviations) when the scatter is normally distributed. The floor encodes the irreducible scatter between laboratories. Without it, a tight cluster of identical values (a common result of citation copying) makes every legitimate value look like an outlier. Values with are flagged. Four refinements make consensus much more useful in practice.
[x=1mm, y=1mm, font=\scriptsize,
every node/.append style={execute at begin node={\hyphenpenalty=10000\relax}},
ok/.style={circle, fill=DDGreen!70, draw=DDGreen!50!black, inner sep=0pt, minimum size=2.6mm},
bad/.style={circle, fill=DDRed!75, draw=DDRed!50!black, inner sep=0pt, minimum size=2.6mm},
dec/.style={circle, fill=DDOrange!80, draw=DDOrange!50!black, inner sep=0pt, minimum size=2.6mm},
lab/.style={align=center, text=DDInk}]
% axis: y = log10 v, 1 decade = 80 mm, x = 10 + 80 (y - 1.3)
% accepted band |z| <= 3.5 with MAD = 0.03: |y - median| <= 0.156
\fill[DDGreen!10] (21.5,4) rectangle (46.5,30);
\node[lab, text=DDGreen!50!black, anchor=east] at (20.5,26) {accepted\\band,\\$|z_i|\le3.5$};
\draw[DDInk, line width=0.6pt, -{Stealth[length=2mm]}] (4,4) -- (146,4);
\foreach \x/\t in {26/1.5, 66/2.0, 106/2.5} {
\draw[DDInk] (\x,4) -- (\x,2.8);
\node[anchor=north, text=DDInk] at (\x,2.6) {\t};
}
\node[anchor=north, text=DDInk] at (134,2.6) {$y=\log_{10}|v|$};
% median and mean
\draw[DDNavy, line width=0.9pt] (34.8,4) -- (34.8,26);
\node[lab, text=DDNavy, anchor=east] at (34.3,24) {median $\tilde y$};
\draw[DDGray!80!black, line width=0.8pt, dashed] (47.6,4) -- (47.6,24);
\node[lab, text=DDGray!30!black, anchor=west] at (48.3,24) {mean, pulled by the two errors};
% correct reports
\foreach \x/\h in {30/8, 32.4/12, 34/8, 34.8/16, 35.6/12, 37.2/8, 38.8/12}
\node[ok] at (\x,\h) {};
% random error
\node[bad] at (70,8) {};
\node[lab, anchor=south, text=DDRed!70!black] at (70,10.5)
{random error,\\flagged ($|z_i|\gg3.5$)};
% decade shift
\node[dec] at (114.8,8) {};
\node[lab, anchor=south, text=DDOrange!60!black] at (114.8,10.5)
{unit slip, $\Delta y\approx1$\\(a factor of ten)};
\draw[-{Stealth[length=2mm]}, DDOrange!85, line width=0.7pt, dashed]
(114.8,19) .. controls (104,46) and (56,46) .. (40,29);
\node[lab, text=DDOrange!60!black, anchor=south] at (80,40.5)
{proposed repair: $y_i-1$, stored as a hypothesis};Figure 4. Robust consensus in one group of reports of the same quantity (schematic). Correct values (green) cluster around the true value, spread only by the differences between laboratories. The median (solid line) stays inside the cluster, whereas the mean (dashed) is pulled towards the two errors. The robust score of eq:mad measures each report’s distance from the median in units of the median absolute deviation. A report far outside the band is flagged (red); one that sits almost exactly a whole number of decades away (orange) is most likely a unit or decimal slip, and the shifted value is stored as a proposed repair, never written over the raw value.
Decade-shift repair
If a flagged value lies close to a whole number of decades from the median, that is, for a non-zero integer and a small tolerance (0.12 decades in § 11), the error is almost certainly a unit or decimal slip, and the value can be repaired as rather than discarded. The same test with a shift of in place of catches Voigt-factor errors. A repair is a hypothesis, so it is recorded as a repair and never written over the raw value.
Consistency between sources
Metrology has long faced the problem of combining measurements that do not agree within their stated uncertainties. The Birge ratio [50] and the scale factor used by the Particle Data Group [51], with , measure whether a group of reports is mutually consistent. Here is the reported uncertainty of report and the mean weighted by . For consistent reports ; indicates an outlier or an unrecognised difference in conditions. Fundamental-constant evaluations use the same idea when weighting discrepant inputs [52]. Applied to literature-mined groups, a large is a request to check whether the group actually mixes two conditions (for example clamped and unclamped) before any value is called wrong.
Within-source conflicts and the indistinguishable fraction
Consensus across papers has a counterpart within a single paper. When two records share every recorded feature (material, component, boundary condition, source, and conditions) but disagree in value, at least one of them is mislabelled, or a distinguishing condition was never captured. Let be the set of groups of records with identical feature vectors, the number of records in group , and the total number of records. The share of records in groups whose values disagree,
measures how much of a database a model cannot learn from, whatever its algorithm. Here the indicator equals 1 when the condition in brackets holds and 0 otherwise, and is a tolerance of about 10%. In words, is the fraction of records that have a “look-alike”: another record with the same material, component, source, and recorded conditions, but a value that differs by more than . A model receives the same input for look-alike records and is asked for different outputs, so it cannot fit them all. needs no reference data, so it can be computed for any literature-mined database. It should always be reported together with the list of fields used to define a group, because adding a field can split a group and lower . tab:phi gives for the 232 accepted electro-optic records of our latest production run under four choices of fields.
| DDBand DDNavyFields used to define a group (besides material, component, source) | DDNavy |
| Every recorded condition and provenance field | 0.39 |
| Temperature, strain, wavelength, and boundary condition | 0.47 |
| Feature set of our modelling study | 0.54 |
| Temperature and strain only | 0.72 |
Multivariate and distributional checks
Some errors look normal in each property separately and abnormal only jointly, for example a refractive index that does not match the reported optical permittivity, although the two must satisfy at optical frequencies. Density- and isolation-based outlier detectors [53, 54, 55] can find them in the joint space. Digit-frequency tests such as Benford’s law [56], the observation that the leading digits of many natural data sets follow a fixed, non-uniform distribution, are sometimes proposed for screening fabricated data; at the scale of a single material they have little power, and we do not recommend them as a filter.
Layer 3: model-based auditing
A model trained on the whole database learns the dominant, mostly correct, relation between descriptors (numerical features of each material and measurement, such as composition, structure, and conditions) and property. Values the model cannot reproduce are then candidate errors. To avoid a model simply memorising its own training labels, each record is scored with an out-of-fold prediction : the database is split into folds, and record is predicted by a model trained on the other folds, so the model never saw it:
and flagged when exceeds a threshold. The denominator, 1.4826 times the median absolute residual, is a robust estimate of the standard deviation of the residuals, so measures how many typical residuals record lies from its prediction. The model borrows strength from chemically similar materials, which is why this layer can flag a value reported once, where consensus has nothing to compare it with. It is the regression form of the idea behind confident learning and related label-error detectors [57, 58, 59, 60]. Ensembles, sets of models trained independently, add a second signal: where members disagree strongly with each other, the label or the model is unreliable [61]. Influence functions and data valuation [62, 63] rank records by how much each one changes the fitted model, which is the right order in which to check them.
The limitation is equally clear. A model-based detector cannot tell an error from a real anomaly, and a material that is truly exceptional is by definition poorly predicted by its neighbours. Layer 3 must therefore never delete a value. It ranks values for review and, if review is not possible, down-weights them in training (§ 6).
Layer 4: targeted human review
The cost of review must scale with the number of errors, not with the size of the database. The layers above make this possible because each produces a score. Dividing each score by its flagging threshold ( for physics, for consensus, for the model) and taking the largest gives one combined suspicion score,
so that means at least one layer flags record . Reviewing records in decreasing order of turns an impossible audit of ten thousand records into a feasible audit of a few hundred. With provenance attached (paper, page, table cell, raw snippet), each check takes seconds. The reviewer’s decisions are fed back as labels, so that the rules of Layers 0–1 improve and the same error is not reviewed twice. We quantify the efficiency of this ranking in § 5.4.
What to do with a flagged value
Detection is only half of the problem. A flagged value has five possible outcomes (Figure 1, right). It is with a confidence weight if the flag is explained, for example by a genuine change of condition. It is if the error is identified and the correction verified, such as a decade shift or a Voigt factor. It is with a typed reason if it cannot be decided. It is if it must be used before review. It is if it violates a hard physical constraint. It is never deleted without a trace. A database that stores only the values it accepted cannot be audited, and it cannot tell a later acquisition step what is missing. The complete procedure is stated in alg:protocol (§ 10).
Domain-specific examples
Electro-optic materials
Layer 0 maps to and the combined coefficient (a single number that mixes several tensor components, as measured in some interferometric set-ups) to an effective coefficient, and it separates intrinsic tensor components from device-level effective values. Layer 1 applies point-group selection rules and (1). Layer 2 groups by material, component, boundary condition, and temperature range, because clamped and unclamped values can legitimately differ by a factor of two or more [33]. Layer 3 audits against composition descriptors. In an earlier run of our pipeline, a CdTe entry read “r_{41}=\SI{110}{pm/V}”, far above any reported CdTe value; the same paper reports r_{41}=\SI{15.5}{pm/V} [72]. Tracing the entry to its source showed that “110” was the direction along which the samples were cleaved [29].
Batteries and energy storage
Physical checks include the theoretical capacity from Faraday’s law (a reported specific capacity above the theoretical limit of the active material has an error in normalisation or in material identity; [83]), Coulombic efficiency (charge delivered on discharge divided by charge stored on charge) at or below 100% in steady cycling, energy density equal to the integral of voltage over capacity, and Arrhenius consistency of conductivities reported at several temperatures. Consensus works well for widely reproduced systems such as LiFePO or graphite. For new electrolytes that are rarely reproduced, Layers 1 and 3 carry the load.
Medicine and drug delivery
Physical and logical checks include dose conversion between mg kg and mg m, encapsulation efficiency and drug loading between 0 and 100%, release fractions that do not decrease with time, and the internal consistency of summary statistics. For example, the relation between SD and SEM identifies which one was reported [89], and a mean that falls outside its reported range reveals an error. Consensus between sources must respect the variability between assays, which is known to be large for potency data [18, 19].
Mechanics
Ultimate tensile strength cannot be below yield strength. Elongation and reduction of area must be consistent. Hardness and strength are linked by approximate empirical conversions that serve as soft constraints. Elastic constants must give a positive-definite stiffness matrix. The modulus of a polycrystal must lie between the Voigt and Reuss bounds, the averages of the single-crystal constants computed under uniform strain and under uniform stress [82] (the Voigt bound is unrelated to the Voigt index notation of § 2.2).
Demonstrations
We test the framework of § 4 in four ways, moving from real data to controlled data. First, we look at what the verification gate caught in our latest production run. Second, we audit the values the gate accepted against a curated reference, to see which errors survive the gate and which reference-free checks would have caught them. Third, we show that governing equations expose single-value errors in real tensor data. Fourth, we measure detection rates by error type in a benchmark where every error is known.
What the gate caught in a production run
Of the 816 candidate values extracted in the latest production run (§ 2.2; [29]), 402 did not pass the verification gate. tab:reasons groups the typed reasons attached to them. A value can carry more than one reason, so the columns count every reason a value touches.
| DDBand DDNavyReason class | DDNavyHeld | DDNavyRejected | DDNavyTotal |
| Corrupted, missing, or unrecognised tensor label | 95 | 100 | 195 |
| Out-of-scope property family (e.g. phonon frequencies) | 0 | 137 | 137 |
| Cited or comparison value, not the paper's own result | 54 | 32 | 86 |
| Missing condition key (value from a series) | 27 | 0 | 27 |
| Family, label, or unit mismatch | 25 | 2 | 27 |
| Implausible magnitude for the stated context | 19 | 3 | 22 |
| Symmetry-forbidden component (zinc-blende rule) | 0 | 20 | 20 |
| OCR corruption, or value not verbatim in the source | 5 | 12 | 17 |
| Not a measurement (equation number, imaginary mode) | 0 | 4 | 4 |
Three observations matter for the rest of the paper. Label problems dominate: nearly half of the stopped values carry a corrupted, incomplete, or unrecognised label. Many of these are held under a strict rule that refuses any label it cannot positively recognise, so the count mixes real errors with unusual notation. Out-of-scope values are not errors; they simply fall outside this database. Cited values are therefore the second largest class of genuine hazard (86 values); a recall-first extractor would have double-counted each of them under the citing paper’s material and conditions. Finally, the one physics rule that was implemented, zinc-blende symmetry, stopped 20 values on its own, among them the components , , , and extracted for zinc-blende InP from a paper on GaInAsP/InP modulators [71]. The next subsection shows what the same rule misses when it is not extended to other crystal classes.
Auditing accepted values against a curated reference
A gate reports what it stopped, not what it let through. To measure what gets through, we compared every accepted value from the three benchmark papers in the corpus (BaTiO, KNbO, LiNbO; [33, 34, 35]) with our curated reference values for those papers. We then applied five checks that need no reference data: Neumann’s principle for every crystal class (not only zinc-blende), label well-formedness, agreement between a record’s text and numeric fields, a plausibility range for diagonal stiffness constants, and within-source conflict (eq:phi). We added one constitutive identity that the extracted values allow, the shear relation , which follows from with the shear stiffness at constant field (the factor 2 again counts the two equal shear components). tab:audit gives the result.
| DDBand DDNavyOutcome against the reference | DDNavyRecords | DDNavyFlagged |
| Matches the reference (value within 2%, component, boundary) | 64 | 5 |
| Value within 2--5% of the reference | 4 | 1 |
| Right number, wrong component label | 11 | 4 |
| Right number, wrong boundary condition (clamped or unclamped) | 2 | 2 |
| Decade or unit shift | 4 | 2 |
| Component absent from the reference | 17 | 9 |
| Value not found in the reference | 10 | 6 |
| Total | 112 | 29 |
Only 64 of the 112 accepted values (57%) match the reference in every field. Seventeen are demonstrably wrong against it: eleven carry the right number under the wrong component label, two carry the unclamped label on a clamped value, and four are KNbO piezoelectric coefficients reported in units of × 10−11 C/N but stored as pm/V. Since 1 pm/V equals × 10−12 C/N, the stored values are ten times too small. A further 13 records are wrong by construction, whatever the reference contains. Some carry labels that are forbidden or meaningless for their crystal class ( for KNbO, stiffness labels such as that orthorhombic symmetry forbids, for BaTiO, and , whose index 4 cannot occur in full-tensor notation). Others are five KNbO stiffness constants whose numeric field is a factor of 100 below their own source text. In total, 30 of the 112 accepted records are demonstrably defective.
The reference-free checks flag 29 records. Twenty-four of these do not match the reference, and the checks catch 21 of the 30 defective records. The remaining five flags fall on correct values: four are the correct partners in a within-source conflict, where the flag marks the group for review, and one is a correct number whose text field was wrong by a factor of 100. The shear identity is especially instructive. From the extracted e_{131}=\SI{5.16}{C/m^2} and c^{E}_{1313}=\SI{25}{GPa} it implies d_{131}=\SI{103}{pm/V}. The extracted value, 10.73 pm/V, is smaller by a factor of 9.6. This is a decade-quantised violation, so the check both detects the unit error and proposes its repair. This audit covers three dense legacy tensor papers, one of them scanned. It is a stress test, not a random sample of the corpus, and the defect rate of a typical modern paper is likely lower.
Constitutive identities expose extraction errors in real tensor data
Complete, internally consistent sets of electromechanical and electro-optic constants exist for a few benchmark ferroelectrics. We use the clamped and unclamped dielectric and electro-optic tensors, the piezoelectric strain and stress tensors, the elasto-optic tensor, and the elastic stiffness of tetragonal BaTiO (point group ) [33] and of orthorhombic KNbO () [34], all as they appear in our curated reference set. From these we evaluate sixteen identities of the forms in (1) and . Each identity has a normalised residual (eq:zphys), with propagated from the reported uncertainties by Monte Carlo sampling: every input is redrawn times from a normal distribution with its reported uncertainty, and is the standard deviation of the resulting values of .
| BaTiO | 8.0 | 8.0 | 3.2 | ||
| BaTiO | 105.0 | 105.2 | 11.6 | ||
| BaTiO | 1300 | 1294 | 185.4 | ||
| BaTiO | 4400 | 4378 | 485.9 | ||
| BaTiO | 129.0 | 129.4 | 8.9 | ||
| BaTiO | 90.0 | 93.9 | 24.6 | ||
| BaTiO | 12.3 | ||||
| BaTiO | 564.0 | 560.7 | 57.0 | ||
| KNbO | 33.5 | 33.7 | 3.4 | ||
| KNbO | 6.0 | 6.3 | 1.6 | ||
| KNbO | 63.3 | 63.5 | 3.2 | ||
| KNbO | 120.0 | 120.5 | 23.4 | ||
| KNbO | 450.0 | 448.9 | 48.5 | ||
| KNbO | 154.0 | 154.1 | 7.7 | ||
| KNbO | 985.0 | 986.1 | 31.0 | ||
| KNbO | 44.0 | 44.3 | 3.5 |
Table 1. Constitutive identities evaluated on literature tensor data for BaTiO [33] and KNbO [34]. “Reported” is the value in the source; “Implied” is the value computed from the other reported tensors through (1) or ; is the propagated uncertainty of the difference. Units: pm/V for and (with in Voigt notation), dimensionless for .
All sixteen identities hold, with (Table 1). This is expected, because these data sets were derived by self-consistent least-squares fitting [33, 34], which makes them a clean reference. The small values also show that treating the reported uncertainties as independent overestimates . We keep that conservative choice: it makes the test harder to trigger, so any violation it reports is robust.
We then applied realistic single-value errors of the kinds catalogued in tab:taxonomy, one at a time, and re-evaluated all identities (tab:injected). Eight of the ten injected errors violate at least one identity at . The strongest violations reach –19 for decade and unit slips, and a clamped value stored in the unclamped field gives . The lost sign on and the Voigt factor of two are caught through a different identity from the one we first expected, because the same tensor element enters several relations. That redundancy is useful: the set of violated identities points to the corrupted value. A wrong in KNbO violates exactly the four identities that contain ; a wrong violates only the identity for .
| DDBand DDNavyInjected error | DDNavyCrystal | DDNavyValue | DDNavyMax | DDNavyFired |
| Decade shift (dropped decimal point) | KNbO | 17.9 | 4 | |
| Unit: e-11C/N read as pm/V | KNbO | 15.2 | 2 | |
| Voigtstored as full-tensor | BaTiO | 10.0 | 2 | |
| Clamped value stored as unclamped | KNbO | 8.9 | 1 | |
| Sign lost | BaTiO | 12.4 | 2 | |
| OCR digit, 9 read as 3 (90 stored as 30) | BaTiO | 5.4 | 3 | |
| OCR digit, 7 read as 1 (780 stored as 180) | KNbO | 19.2 | 1 | |
| Stiffness unit slip (factor of ten) | BaTiO | 17.0 | 2 | |
| Adjacent table cell read (value in) | KNbO | 1.9 | 0 | |
| Small transcription slip (+10%) | BaTiO | 0.9 | 0 |
A controlled benchmark with known ground truth
Real literature data has no labels saying which values are wrong, so the recall of a detector cannot be measured on it directly. We therefore built a synthetic “literature” in which every error is known, designed to reproduce the statistical structure that matters for detection (§ 11). Four hundred materials are described by six descriptors, and their true property (in ) is a smooth nonlinear function of the descriptors plus a material-specific term of 0.12 decades that no descriptor explains. The number of papers per material is heavy-tailed, as in the real literature: just over half of the materials are reported once, and about 6% are reported more than ten times (up to sixty). Each report carries between-laboratory and within-laboratory scatter (together about 10%). In 40% of reports, two auxiliary quantities and are also reported, related to the target by an exact identity, (that is, ). This is the analogue of reporting , , and together with in (1). Errors from seven classes are then injected so that a fraction of the records (the error rate) are wrong: unit or scale shifts, OCR digit and decimal corruption, label swaps, misattribution to another material (often a chemically similar one), context loss, hallucinated values drawn from the distribution of the property, and one wrong value copied into several reports of the same material, as through a chain of citations. We report means over ten random corpora (about 1300 records each), for unless stated otherwise.
Figure 5. Recall by error class and detection layer in the controlled benchmark (, mean of ten corpora). No single layer dominates: the physics checks are precise but cover few records, consensus needs redundancy, and the model audit has high recall at lower precision. Their union reaches 81% recall at 85% precision. The remaining errors are concentrated in the classes that produce plausible values.
Each layer sees different errors
Figure 5 gives the recall of each detector by error class. The bounds check is perfectly precise but catches only the 38% of unit errors that leave the plausible range. The identity check is also almost perfectly precise (precision 0.998); it is limited by coverage, not by power, catching 82% of the errors in records that report the auxiliary quantities, which is 33% of all errors. Consensus catches 58% of all errors with precision 0.98. The model-based audit has the highest single-layer recall (0.72), but its precision is lower (0.84), and it is the only layer that flags a noticeable fraction of clean records (2.5%). The layered combination catches 81% of errors with precision 0.85. It catches every unit or scale error, 96–98% of label swaps and context errors, and 78% of copied errors. The classes that remain hard are exactly the ones that produce plausible values: misattribution to a similar material (55%), OCR digit corruption (57%), and hallucination (68%).
Figure 6. (a) Recall as a function of how often the material is reported (; bars are one standard deviation over ten corpora). Consensus cannot act below three reports, while the model audit, which borrows strength from similar materials, keeps the layered detector effective on the long tail. (b) Fraction of all errors found when records are reviewed in order of the combined suspicion score, compared with random review.
Redundancy decides what consensus can do
Figure 6(a) splits recall by the number of reports of the material. Consensus is blind for materials reported once or twice, and it reaches its best performance (86%) only for materials with more than ten reports. The model-based audit is roughly flat across redundancy because it borrows strength from chemically similar materials. As a result the layered detector keeps 75–79% recall on the singletons and pairs that consensus cannot touch. This is the quantitative form of the long-tail problem: the materials reported least often are the most likely discoveries, and they are the materials for which physics and models, not voting, must supply the checks.
Review effort scales with the number of errors
Figure 6(b) orders all records by the combined suspicion score (§ 4.5) and plots the fraction of errors found against the fraction of the database reviewed. With errors at 15% of the records, reviewing the top 10% of the ranked list finds 64% of all errors, and the top 15% finds 83%; random review would find 10% and 15%. For a database of ten thousand records, the ranked list turns a review of every record into a review of about fifteen hundred, most of which are actual errors.
Repair
Of the values flagged by consensus as decade-quantised shifts, 92% were restored to within 0.1 decades of the truth by the repair rule of § 4.3. The remaining 8% were other errors that happened to fall near an integer shift, which is why a repair is recorded as a hypothesis and not written over the raw value.
Impact on AI Modelling
Why does an error rate of a few percent matter to a model trained on thousands of records? Because errors in extracted data are neither small nor symmetric, and because the most valuable uses of a materials model, ranking and inverse design, are exactly the uses that errors distort most.
Why a few errors dominate training
Under a squared-error loss, a value that is off by three decades contributes a squared residual of 9 (in units). With the scatter between and within laboratories of our benchmark, a clean record contributes about . One unit error therefore adds as much to the loss as roughly five thousand correct records, and its pull on the gradient that drives training, which is proportional to the residual, equals that of about seventy. On a linear scale the imbalance is larger still. Flexible models make this worse: over-parameterised learners can fit arbitrary labels exactly [64], so with enough capacity an erroneous value is memorised rather than averaged away, and it distorts predictions for its chemical neighbours. Robust losses such as Huber and absolute error [65, 66] limit the pull of each record, but they cannot tell a wrong value from an unusual correct one.
Accuracy: what cleaning recovers
We trained a gradient-boosted surrogate [66] on the benchmark corpora under six strategies and evaluated it on held-out materials against their true property values (Figure 7a; § 11). With no errors, all strategies reach a mean absolute error (MAE) of 0.13–0.14 decades. The naive model, trained with a squared loss on everything extracted, degrades steadily: its MAE rises by 32% at (0.175 against 0.132 decades) and by 47% at . A robust loss alone recovers about half of the loss at (0.153) and almost none at (0.190). Aggregating to one median value per material, a common practice, gives almost no protection (0.171), because most materials are reported once and their errors pass through unchanged. Layered cleaning with repair of decade shifts recovers nearly all of the gap: at it gives 0.136 decades, within about 3% of the error-free model, and it stays within 0.01 decades of the error-free model up to . Soft down-weighting by the suspicion score, which removes nothing, performs almost as well (0.140) and is the safer choice when exceptional values are expected.
Cleaning has a price, and it should be stated. With no errors present, the model-based layer still flags 4.5% of the (clean) records, and removing them raises the MAE slightly, from 0.132 to 0.139 decades. Precision is lowest when errors are rarest (0.55 at , against 0.85 at ). A detector tuned for a noisy corpus therefore needs to be re-tuned as the extractor improves, which is one more reason to route model-based flags to review or to down-weighting rather than to deletion.
The same effects appear in the real electro-optic corpus, measured with the modelling code of our pipeline [29]. When the consensus layer was skipped, every record entered training with nearly the same weight, and the surrogate’s skill fell from 0.456 to 0.373, where skill is one minus the ratio of the model’s cross-validated MAE to that of a baseline that always predicts the weighted mean. The weighted coefficient of determination fell from 0.626 to 0.569 on otherwise identical data. When the model-based audit was run on the 232 accepted electro-optic records, it flagged 20. About half of these were legitimate values, mostly electronic or ionic contributions to a Pockels coefficient that were being modelled together with total coefficients; the rest were worth checking against the source. This is the behaviour the benchmark predicts for Layer 3: a useful triage list, with about 50% precision on its own, that must not be used to delete values.
Evaluation: errors in test data hide the problem
Errors damage the measurement of a model as well as its training. If the test records are drawn from the same uncleaned database, their errors enter the metric. In our benchmark at , the naive and cleaned models have true held-out MAEs of 0.175 and 0.136 decades. Measured against the uncleaned test records, the apparent MAEs are 0.319 and 0.269: the metric nearly doubles, the apparent improvement from cleaning shrinks from 22% to 16%, and the variation between corpora grows three- to fivefold. This is the regression analogue of the finding that label errors in test sets change which model is ranked best [26]. It also means that a leave-one-material-out audit, such as the one we use for the electro-optic surrogate [29], is only as trustworthy as the held-out labels. In any audit that claims generalisation, the test set should be cleaned first.
Discovery: inverse design seeks out errors
The most consequential effect concerns inverse design, and it follows from simple selection. Screening and generative design look for the extremes of a property, and the largest errors are, by construction, among the most extreme values. Figure 7(b) shows the result. When errors make up 15% of the records, 89% of the twenty largest reported values are errors, and at the share is already 39%. After layered cleaning it falls to 1% at . A discovery pipeline that ranks by the raw extracted value, or that trains a surrogate and then maximises it, will therefore put errors at the top of its candidate list, and it will do so with confidence, because the surrogate has learned those values. This is the data-side version of the “optimiser’s curse” [70]: optimisation turns a random error into a systematic bias. In our inverse-design study for cryogenic Pockels materials, the surrogate was allowed to use only the condition variables (such as strain and temperature) that the data could support, and the verification step was run on the database as well as on the candidates; that step exposed a recoverable extraction defect that mattered more than any single ranked candidate [42].
Figure 7. Consequences for modelling in the controlled benchmark (means over ten corpora). (a) Mean absolute error on held-out materials, measured against the true values, for six training strategies. Layered cleaning with decade repair stays close to the error-free model up to , whereas a robust loss alone or per-material median aggregation recovers only part of the loss. (b) Share of errors among the twenty largest reported values, the records an inverse-design or screening step would select first. Errors are enriched far above their base rate (dashed line), because the largest errors are the largest values.
The binding constraint is missing context
The benchmark treats an error as a wrong number. In our real corpus the largest obstacle to learning is different. Between 39% and 72% of the accepted electro-optic records, depending on which recorded fields define a group, share every recorded feature with another record that has a different value (eq:phi). No loss function, weighting scheme, or detector can recover what distinguishes them, because it was never extracted. A related analysis found that an extraction improvement that added records to one BiFeO paper reduced the usable information from that paper to zero, because the new values arrived without the conditions that tell them apart [42]. Record counts cannot reveal this, and can. We therefore recommend reporting , with its feature set, together with precision against a gold standard, as the two quality figures of any literature-mined database. When is large, the most valuable next step is to recover conditions, or to generate condition-resolved data by calculation, not to mine more papers.
Uncertainty and trust
Two further effects are harder to quantify but matter for trust. First, a model trained on data with errors reports uncertainties calibrated against labels that include those errors, so its confidence is wrong in both directions: too wide where errors add apparent scatter, and too narrow where a copied error has produced false agreement. Ensembles [61] partly capture the first effect and not the second. Second, the confidence weights, corroboration counts, and typed reasons recorded by the detection layers are themselves useful inputs to the model. They can serve as sample weights during training, as features that let the model learn how reliable each source is, and as the admissible-domain constraint of an inverse-design objective [29]. A database that carries its own quality information lets the model decide how much to trust each record, instead of assuming that every record is equally correct.
Future Directions
The layered approach turns error handling from something done after the fact into part of how the database is built. Several directions would make it stronger and more widely used.
Measured error rates for real extractors
The field lacks a basic number: how often does each extractor make each class of error, on which kinds of source? We have taken a first step, with a certified gold standard of 303 values from 16 papers and a reference audit of accepted values (sec:case,sec:audit; [29]). The measurements are sobering: precision against the gold standard is about one half, and in three benchmark papers about a quarter of the values that passed a strict gate were still defective. Much larger hand-audited samples, stratified by publication era, document type (table, text, scan), and field, would make it possible to calibrate detector thresholds and to replace synthetic benchmarks like ours with real ones. Publishing such audit sets alongside extraction benchmarks would do for literature mining what curated test sets did for machine learning.
Encoding physics as reusable, machine-readable rules
Symmetry selection rules, constitutive identities, and conservation laws are known, but each project writes them again, often with different thresholds; our own early pipeline had three inconsistent copies of its physical bounds. Shared rule libraries, organised by property family and expressed so that both code and paper can cite them, would give every project Layer 1 at no cost. The same library could constrain LLM extraction directly: an extractor that knows is forbidden for crystals should never propose it.
Extraction that verifies itself
LLM extractors can be asked to return, with each value, the verbatim span that supports it, the reason for the label they chose, and the alternatives they rejected. Checking the span against the source (Layer 0) and re-extracting in a second independent pass make hallucination and label swaps detectable at the point of extraction [12]. Disagreement between two extractors, or between an extractor and a physics rule, is a cheap and informative review signal.
Detecting systematic errors
Consensus fails when an error has been copied through the literature. Tracing citation graphs, so that a value quoted by ten later papers counts as one source and not ten, and weighting groups of reports by the independence of their measurements, would remove the main blind spot of statistical detection. The same provenance graph also exposes clusters of papers that share an unusual convention.
Databases that store uncertainty and provenance, not only values
An AI-ready database should store, for each quantity, the individual reports, their provenance, their quality flags, the consensus value, and a measure of how consistent the reports are with each other [50, 51]. Held and rejected values should be stored with typed reasons. Versioned releases, with the record of each change, allow a model to be retrained on a corrected database and let downstream users see what changed and why [30, 31].
Generating the missing conditions
Where is large, the missing information was usually never written down, because the authors had no reason to tabulate it, and further mining cannot recover it. First-principles calculations can generate the condition axis (strain, temperature, field) at the density a model needs, with provenance recorded by construction. How to weight dense computed data with a systematic offset against sparse measurements is an open question; it probably requires treating provenance as a level of a hierarchical model rather than as one more feature.
Closing the loop with acquisition
Detection results say where the data are weakest: which materials have a single unconfirmed report, which groups disagree, which conditions are missing. Coupled to uncertainty-guided acquisition [67, 68], the pipeline could look for the paper, measurement, or calculation that would best confirm or refute a suspect value. The same discipline applies at the corpus boundary: in a first screen of the papers our acquisition stage retrieved, only about one in ten reported an electro-optic coefficient at all [29].
Upstream: better reporting
Many of the errors in tab:taxonomy would disappear if property values were published in machine-readable form, with units, tensor conventions, and conditions stated explicitly. Journals already ask for data availability. Asking for structured property tables in supplementary information, and checking them against simple physical rules at submission, as has been done for thermophysical data [21], would reduce errors where they begin.
Conclusion
Literature mining is becoming the main route to AI-ready data for properties that are measured but not collected, and it brings errors that no extractor will remove completely. We have argued that these errors should be treated as an expected property of the data, not a failure to be hidden, and handled with a layered approach. Schema and provenance checks catch malformed and unsupported records. Physics, through bounds, symmetry, and governing equations, catches errors even in values reported once. Statistical consensus catches random errors wherever the literature is redundant. Model-based auditing extends coverage to the long tail. Targeted human review, ordered by suspicion, settles what is left, with effort that grows with the number of errors and not with the size of the database.
Four demonstrations support this approach. In a production run of our electro-optic pipeline, the gate stopped 402 of 816 candidate values with typed reasons. Yet an audit against a curated reference showed that about a quarter of the values it accepted from three benchmark papers were still defective, and simple reference-free checks caught 70% of them. On real BaTiO and KNbO tensor data, constitutive identities caught eight of ten realistic single-value extraction errors and pointed to the corrupted value. In a controlled benchmark, the combined layers found about four fifths of injected errors at 85% precision, kept their power for materials reported only once, and restored model accuracy to within a few percent of error-free training. The benchmark also exposed the reason all this matters for discovery: without cleaning, the extreme values that inverse design selects first are almost all errors. The real corpus added a lesson the benchmark could not: once wrong numbers are under control, missing context becomes the binding constraint, and it has to be measured () and repaired at the source.
The principle that governs our electro-optic pipeline also applies here: no value is better than a wrong value. A database that knows which of its values to doubt is worth more to AI-driven science than a larger database that treats every value as correct.
Notation and glossary
This appendix collects the symbols used in the paper, so that a reader from any of the fields discussed can look up a quantity without searching the text. tab:symbols lists the symbols of the detection framework and the benchmark. tab:glossary lists the physical quantities and field-specific terms, most of which appear in tab:fields,tab:taxonomy.
| DDBand DDNavySymbol | DDNavyMeaning | DDNavyWhere |
| , | A reported value (of record), in the units of its property | |
| , | Logarithm of a value,, in decades | |
| A shift in;is a factor of ten in | ||
| \vect{v}, | A set ofrelated valuesentering one identity | |
| g(\vect{v}) | A physical identity, equal to zero for the true values | |
| , | Standard uncertainty of value, and ofafter propagation | |
| Normalised physics residual, | ||
| Number of reports in a consensus group | ||
| Median of thein a group | ||
| MAD | Median absolute deviation, | |
| Lower limit on the MAD: the irreducible scatter between laboratories | ||
| Robust consensus score of record | ||
| , | Integer number of decades in a decade-shift repair, and its tolerance | |
| , | Birge ratio and chi-squared statistic of a group of reports | |
| , | Uncertainty-weighted mean of a group; reported uncertainty of report | |
| Indistinguishable fraction: share of records in groups with identical features but different values | ||
| ,, | Set of groups with identical feature vectors; one group; its size | |
| Total number of records | ||
| Tolerance within which two values count as equal (about 10%) | ||
| Indicator: 1 if the condition holds, otherwise 0 | ||
| Out-of-fold prediction ofby a model that did not see record | ||
| Model-based suspicion score of record | ||
| Combined suspicion score, the largest threshold-normalised score | ||
| ,, | Flagging thresholds for the physics, consensus, and model scores | |
| Error rate: fraction of records that are wrong | ||
| Number of materials in a benchmark corpus (400) | ||
| \vect{x}, | Descriptor vector of a material and its components | |
| , | True property of a material (in) and its unexplained material-specific part | |
| , | Number of reports of material, and the heavy-tailed random number that sets it | |
| , | Auxiliary quantities with(analogue of a constitutive identity) | |
| Normal distribution with meanand standard deviation | ||
| MAE | Mean absolute error, here in decades | |
| Coefficient of determination of a model's predictions |
| DDBand DDNavySymbol or term | DDNavyMeaning | DDNavyTypical unit |
| Electro-optic, dielectric, piezoelectric, and elastic tensors | ||
| ;, | Linear electro-optic (Pockels) coefficient,; unclamped and clamped values | pm/V |
| , | The same coefficient in Voigt notation;is an alternative symbol | pm/V |
| Combined or effective coefficient that mixes several components | pm/V | |
| Componentof the applied electric field | V/m | |
| Refractive index (in and ) | -- | |
| ;, | Relative permittivity (dielectric constant); unclamped and clamped values | -- |
| Permittivity of vacuum, 8.854e-12F/m | F/m | |
| , | Piezoelectric strain coefficient: strain per unit electric field | pm/Ve-12C/N |
| , | Piezoelectric stress coefficient: stress per unit electric field | C/m |
| Elasto-optic tensor: change ofper unit strain | -- | |
| , | Elastic stiffness at constant field (full and Voigt notation) | GPa |
| Elastic compliance, the inverse of the stiffness | GPa | |
| Voigt notation | Compression of a symmetric index pair to one index:,,,,, | -- |
| ,,, | Crystal point groups in Hermann--Mauguin notation (cubic zinc blende; tetragonal; orthorhombic; trigonal) | -- |
| Poisson ratio | -- | |
| Energy storage | ||
| Specific capacity | Charge stored per unit mass (or area) of electrode | mAh g |
| Coulombic efficiency | Charge delivered on discharge divided by charge stored on charge | % |
| C-rate | Charge or discharge current relative to capacity; 1C empties the cell in one hour | h |
| Ionic conductivity | Ease of ion transport through an electrolyte | S cm |
| Medicine and drug delivery | ||
| IC, pIC | Concentration giving half-maximal inhibition; pICIC(molar) | nM, M |
| Inhibition constant: dissociation constant of an inhibitor from its target | nM | |
| SD, SEM | Standard deviation; standard error of the mean, | -- |
| Dose basis | Dose per body mass (mg kg) or per body surface area (mg m) | -- |
| Mechanics and other fields | ||
| Engineering and true stress | Force per original area, and per current area | MPa |
| HV | Vickers hardness number | HV, GPa |
| ksi | Thousand pounds-force per square inch (6.895MPa) | -- |
| Superconducting critical temperature | K | |
| Governing equations of | ||
| ,,, | Theoretical specific capacity; electrons transferred per formula unit; Faraday constant (96485C/mol); molar mass | mAh g |
| ,,, | Electrode potential, its standard value, reaction quotient, and charge number (Nernst equation) | V |
| , | Latent heat and volume change of a phase transition (Clapeyron equation) | J/mol, m/mol |
| , | Mole fraction and chemical potential of component(Gibbs--Duhem) | --, J/mol |
| Coupled transport coefficients (Onsager) | -- | |
| , | Extinction coefficient at angular frequency; principal value of the integral (Kramers--Kronig) | -- |
| ,;, | Bulk and shear modulus; subscripts for the Voigt and Reuss bounds | GPa |
| ,, | Yield strength, grain size, and Hall--Petch coefficient | MPa, m |
| , | Lattice parameter and alloy composition (Vegard's law) | , -- |
| ,, | Activation energy, Boltzmann constant, and prefactor (Arrhenius) | eV |
| , | Linear and second-order susceptibilities (Miller's rule) | --, pm/V |
A protocol for handling erroneous data
The layered approach of § 4 is written below as a procedure, so that a new project can adopt it directly and the paper and the implementation describe the same order of operations.
Benchmark details
Corpus
Each corpus contains materials with descriptors \vect{x}\in[0,1]^6 drawn uniformly. The true property is
in units, where are the components of \vect{x} and is a material-specific term, drawn from a normal distribution with mean 0 and standard deviation 0.12, that the descriptors do not explain. The number of reports of material is , capped at sixty, where rounds down and is drawn from a Lomax (Pareto type II) distribution with shape 1.3, a heavy-tailed distribution in which most draws are small and a few are very large. This gives a mean of about 3.5 reports, with 54% of materials reported once. Each report carries a laboratory offset and a measurement error . With probability 0.4 a report also gives the auxiliary quantities and , with a smooth function of \vect{x} and , each with its own measurement error.
Error injection
Records are corrupted until a fraction of target values are wrong. The error classes and their relative frequencies are: unit or scale shifts of or decades (0.18); OCR corruption (0.14), in which a digit of the three-significant-figure value is replaced by a visually confusable digit or the decimal point is dropped; label swap, in which the auxiliary is stored as the target (0.14); misattribution to another material, half the time one of its five nearest neighbours in descriptor space (0.14); context loss, a shift of to decades mimicking a value measured near a transition (0.12); hallucination, a value drawn from the distribution of true values across materials with 0.1-decade jitter (0.12); and citation copying, in which one wrong value is copied into two to four reports of a material with at least four reports (0.16). Corruptions smaller than 0.02 decades are discarded, because they cannot be told apart from measurement error.
Detectors
Bounds: . Identity: decades. Consensus: in groups with at least three reports, with . Decade repair: flagged values within 0.12 of a non-zero integer shift from the group median. Model audit: five-fold out-of-fold predictions from a gradient-boosted regressor with Huber loss, a loss that is quadratic for small residuals and linear for large ones [65] (250 trees, depth 3, learning rate 0.06, subsample 0.8), flagging . The layered detector is the union of all four, and the suspicion score is the maximum of the threshold-normalised scores.
Models
In each corpus, 20% of materials are held out. Surrogates are gradient-boosted regressors (300 trees, depth 3, learning rate 0.05, subsample 0.8), trained on the records of the remaining materials under six strategies: error-free records only (reference); all records with squared loss; all records with Huber loss; one median per material; records that survive layered cleaning, with verified repairs applied; and all records with repairs applied and weight . MAE is computed against the true of the held-out materials. All results are means over ten random seeds for each error rate.
Details of the real-data audits
Gate reasons (tab:reasons)
Every held or rejected record carries a free-text list of reasons written by the verification gate. Reasons were mapped to the nine classes of tab:reasons by keyword, and a record is counted once in each class it touches. One held record (a non-canonical unit string) matched no class.
Reference audit (tab:audit)
All accepted records whose source is one of the three benchmark papers were compared with the curated reference values for the same paper. Tensor labels were converted to full-index form. Voigt and full-tensor notations, index-order variants, and the symmetry equalities and (for and ) were treated as equivalent. A value matches if it lies within 2% (or 0.006 in absolute terms for small dimensionless values) of a reference value with the same component and a compatible clamped or unclamped label. Decade shifts were tested for factors of to with a 5% tolerance. For the reference-free checks, allowed tensor components were computed by averaging basis tensors over each point group (, , , , ), with the group chosen from material and reported phase, and records in films or low-symmetry phases were skipped. A label was malformed if an index was out of range for its notation. The text and numeric fields of a record had to agree within 2%. Diagonal stiffness constants had to lie between 20 and 800 GPa. A within-source conflict was flagged when records sharing paper, family, component, and boundary differed by more than 10%.
Indistinguishable fraction
(eq:phi) was computed on the 232 accepted electro-optic records of the latest production run with , grouping by material, canonical component, and source paper together with the listed condition fields; records with a missing field share the empty value for that field.
References
- [1]Himanen, L., Geurts, A., Foster, A. S., & Rinke, P. (2019). Data-driven materials science: Status, challenges, and perspectives. Advanced Science, 6(21), 1900808. https://doi.org/10.1002/advs.201900808
- [2]Morgan, D., & Jacobs, R. (2020). Opportunities and challenges for machine learning in materials science. Annual Review of Materials Research, 50, 71–103. https://doi.org/10.1146/annurev-matsci-070218-010015
- [3]Jain, A., Ong, S. P., Hautier, G., Chen, W., Richards, W. D., Dacek, S., Cholia, S., Gunter, D., Skinner, D., Ceder, G., & Persson, K. A. (2013). Commentary: The Materials Project: A materials genome approach to accelerating materials innovation. APL Materials, 1(1), 011002. https://doi.org/10.1063/1.4812323
- [4]Saal, J. E., Kirklin, S., Aykol, M., Meredig, B., & Wolverton, C. (2013). Materials design and discovery with high-throughput density functional theory: The Open Quantum Materials Database (OQMD). JOM, 65(11), 1501–1509. https://doi.org/10.1007/s11837-013-0755-4
- [5]Curtarolo, S., Setyawan, W., Hart, G. L. W., Jahnatek, M., Chepulskii, R. V., Taylor, R. H., Wang, S., Xue, J., Yang, K., Levy, O., Mehl, M. J., Stokes, H. T., Demchenko, D. O., & Morgan, D. (2012). AFLOW: An automatic framework for high-throughput materials discovery. Computational Materials Science, 58, 218–226. https://doi.org/10.1016/j.commatsci.2012.02.005
- [6]Olivetti, E. A., Cole, J. M., Kim, E., Kononova, O., Ceder, G., Han, T. Y.-J., & Hiszpanski, A. M. (2020). Data-driven materials research enabled by natural language processing and information extraction. Applied Physics Reviews, 7(4), 041317. https://doi.org/10.1063/5.0021106
- [7]Kononova, O., He, T., Huo, H., Trewartha, A., Olivetti, E. A., & Ceder, G. (2021). Opportunities and challenges of text mining in materials research. iScience, 24(3), 102155. https://doi.org/10.1016/j.isci.2021.102155
- [8]Swain, M. C., & Cole, J. M. (2016). ChemDataExtractor: A toolkit for automated extraction of chemical information from the scientific literature. Journal of Chemical Information and Modeling, 56(10), 1894–1904. https://doi.org/10.1021/acs.jcim.6b00207
- [9]Court, C. J., & Cole, J. M. (2018). Auto-generated materials database of Curie and Néel temperatures via semi-supervised relationship extraction. Scientific Data, 5, 180111. https://doi.org/10.1038/sdata.2018.111
- [10]Huang, S., & Cole, J. M. (2020). A database of battery materials auto-generated using ChemDataExtractor. Scientific Data, 7, 260. https://doi.org/10.1038/s41597-020-00602-2
- [11]Kim, E., Huang, K., Saunders, A., McCallum, A., Ceder, G., & Olivetti, E. (2017). Materials synthesis insights from scientific literature via text extraction and machine learning. Chemistry of Materials, 29(21), 9436–9444. https://doi.org/10.1021/acs.chemmater.7b03500
- [12]Polak, M. P., & Morgan, D. (2024). Extracting accurate materials data from research papers with conversational language models and prompt engineering. Nature Communications, 15, 1569. https://doi.org/10.1038/s41467-024-45914-8
- [13]Dagdelen, J., Dunn, A., Lee, S., Walker, N., Rosen, A. S., Ceder, G., Persson, K. A., & Jain, A. (2024). Structured information extraction from scientific text with large language models. Nature Communications, 15, 1418. https://doi.org/10.1038/s41467-024-45563-x
- [14]Ji, Z., Lee, N., Frieske, R., Yu, T., Su, D., Xu, Y., Ishii, E., Bang, Y. J., Madotto, A., & Fung, P. (2023). Survey of hallucination in natural language generation. ACM Computing Surveys, 55(12), 248. https://doi.org/10.1145/3571730
- [15]Young, D., Martin, T., Venkatapathy, R., & Harten, P. (2008). Are the chemical structures in your QSAR correct?. QSAR & Combinatorial Science, 27, 1337–1345. https://doi.org/10.1002/qsar.200810084
- [16]Fourches, D., Muratov, E., & Tropsha, A. (2010). Trust, but verify: On the importance of chemical structure curation in cheminformatics and QSAR modeling research. Journal of Chemical Information and Modeling, 50(7), 1189–1204. https://doi.org/10.1021/ci100176x
- [17]Fourches, D., Muratov, E., & Tropsha, A. (2016). Trust, but verify II: A practical guide to chemogenomics data curation. Journal of Chemical Information and Modeling, 56(7), 1243–1252. https://doi.org/10.1021/acs.jcim.6b00129
- [18]Kramer, C., Kalliokoski, T., Gedeck, P., & Vulpetti, A. (2012). The experimental uncertainty of heterogeneous public $K_ i$ data. Journal of Medicinal Chemistry, 55(11), 5165–5173. https://doi.org/10.1021/jm300131x
- [19]Landrum, G. A., & Riniker, S. (2024). Combining IC$_50$ or $K_ i$ values from different sources is a source of significant noise. Journal of Chemical Information and Modeling, 64(5), 1560–1567. https://doi.org/10.1021/acs.jcim.4c00049
- [20]Frenkel, M., Chirico, R. D., Diky, V., Yan, X., Dong, Q., & Muzny, C. (2005). ThermoData Engine (TDE): Software implementation of the dynamic data evaluation concept. Journal of Chemical Information and Modeling, 45(4), 816–838. https://doi.org/10.1021/ci050067b
- [21]Chirico, R. D., Frenkel, M., Magee, J. W., Diky, V., Muzny, C. D., Kazakov, A. F., Kroenlein, K., Abdulagatov, I., Hardin, G. R., Acree, J. , W. E., et al. (2013). Improvement of quality in publication of experimental thermophysical property data: Challenges, assessment tools, global implementation, and online support. Journal of Chemical & Engineering Data, 58(10), 2699–2716. https://doi.org/10.1021/je400569s
- [22]Gøtzsche, P. C., Hróbjartsson, A., Marić, K., & Tendal, B. (2007). Data extraction errors in meta-analyses that use standardized mean differences. JAMA, 298(4), 430–437. https://doi.org/10.1001/jama.298.4.430
- [23]Mathes, T., Klaßen, P., & Pieper, D. (2017). Frequency of data extraction errors and methods to increase data extraction quality: a methodological review. BMC Medical Research Methodology, 17, 152. https://doi.org/10.1186/s12874-017-0431-4
- [24]White, A. J., Gibaldi, M., Burner, J., Mayo, R. A., & Woo, T. K. (2025). High structural error rates in ``computation-ready'' MOF databases discovered by checking metal oxidation states. Journal of the American Chemical Society, 147(21), 17579–17583. https://doi.org/10.1021/jacs.5c04914
- [25]Gibaldi, M., Kapeliukha, A., White, A., Luo, J., Mayo, R. A., Burner, J., & Woo, T. K. (2025). MOSAEC-DB: a comprehensive database of experimental metal--organic frameworks with verified chemical accuracy suitable for molecular simulations. Chemical Science, 16(9), 4085–4100. https://doi.org/10.1039/d4sc07438f
- [26]Northcutt, C. G., Athalye, A., & Mueller, J. (2021). Pervasive label errors in test sets destabilize machine learning benchmarks. Proceedings of the Neural Information Processing Systems Track on Datasets and Benchmarks, 1.
- [27]Sambasivan, N., Kapania, S., Highfill, H., Akrong, D., Paritosh, P., & Aroyo, L. M. (2021). ``Everyone wants to do the model work, not the data work'': Data cascades in high-stakes AI. Proceedings of the 2021 CHI Conference on Human Factors in Computing Systems, 1–15. https://doi.org/10.1145/3411764.3445518
- [28]Datta, D. (2026). An autonomous scientific knowledge generation framework for AI-driven scientific discovery. In arXiv preprint arXiv:2607.09806 [cs.DL]. https://doi.org/10.48550/arXiv.2607.09806
- [29]Datta, D. (2026). From Autonomous Knowledge Generation to Generalizable Property Prediction: A Precision-First Pipeline for Electro-Optic Materials Discovery. In arXiv (submitted).
- [30]Wilkinson, M. D., Dumontier, M., Aalbersberg, I. J., Appleton, G., Axton, M., Baak, A., Blomberg, N., Boiten, J.-W., da Silva Santos, L. B., Bourne, P. E., et al. (2016). The FAIR guiding principles for scientific data management and stewardship. Scientific Data, 3, 160018. https://doi.org/10.1038/sdata.2016.18
- [31]Gebru, T., Morgenstern, J., Vecchione, B., Vaughan, J. W., Wallach, H., Daumé III, H., & Crawford, K. (2021). Datasheets for datasets. Communications of the ACM, 64(12), 86–92. https://doi.org/10.1145/3458723
- [32]Nye, J. F. (1985). Physical Properties of Crystals: Their Representation by Tensors and Matrices. Oxford University Press.
- [33]Zgonik, M., Bernasconi, P., Duelli, M., Schlesser, R., Günter, P., Garrett, M. H., Rytz, D., Zhu, Y., & Wu, X. (1994). Dielectric, elastic, piezoelectric, electro-optic, and elasto-optic tensors of BaTiO$_3$ crystals. Physical Review B, 50(9), 5941–5949. https://doi.org/10.1103/PhysRevB.50.5941
- [34]Zgonik, M., Schlesser, R., Biaggio, I., Voit, E., Tscherry, J., & Günter, P. (1993). Materials constants of KNbO$_3$ relevant for electro- and acousto-optics. Journal of Applied Physics, 74(2), 1287–1297. https://doi.org/10.1063/1.354934
- [35]Jazbinšek, M., & Zgonik, M. (2002). Material tensor parameters of LiNbO$_3$ relevant for electro- and elasto-optics. Applied Physics B, 74, 407–414. https://doi.org/10.1007/s003400200818
- [37]Borg, C. K. H., Frey, C., Moh, J., Pollock, T. M., Gorsse, S., Miracle, D. B., Senkov, O. N., Meredig, B., & Saal, J. E. (2020). Expanded dataset of mechanical properties and observed phases of multi-principal element alloys. Scientific Data, 7, 430. https://doi.org/10.1038/s41597-020-00768-9
- [38]Stanev, V., Oses, C., Kusne, A. G., Rodriguez, E., Paglione, J., Curtarolo, S., & Takeuchi, I. (2018). Machine learning modeling of superconducting critical temperature. Npj Computational Materials, 4, 29. https://doi.org/10.1038/s41524-018-0085-8
- [39]Carlisle, J. B. (2017). Data fabrication and other reasons for non-random sampling in 5087 randomised, controlled trials in anaesthetic and general medical journals. Anaesthesia, 72(8), 944–952. https://doi.org/10.1111/anae.13938
- [40]Blecher, L., Cucurull, G., Scialom, T., & Stojnic, R. (2023). Nougat: Neural optical understanding for academic documents. In arXiv preprint arXiv:2308.13418. https://doi.org/10.48550/arXiv.2308.13418
- [41]Lopez, P. (2009). GROBID: Combining automatic bibliographic data recognition and term extraction for scholarship publications. Research and Advanced Technology for Digital Libraries (ECDL 2009), 5714, 473–474. https://doi.org/10.1007/978-3-642-04346-8_62
- [42]Datta, D. (2026). Inverting the forward model: Joint composition-and-condition optimization for cryogenic Pockels response, and the data density the inversion requires. In In preparation, Dibakar Datta Laboratory, New Jersey Institute of Technology.
- [43]Marshall, I. J., & Wallace, B. C. (2019). Toward systematic review automation: a practical guide to using machine learning tools in research synthesis. Systematic Reviews, 8, 163. https://doi.org/10.1186/s13643-019-1074-9
- [44]Jonnalagadda, S. R., Goyal, P., & Huffman, M. D. (2015). Automating data extraction in systematic reviews: a systematic review. Systematic Reviews, 4, 78. https://doi.org/10.1186/s13643-015-0066-7
- [45]Chung, Y. G., Haldoupis, E., Bucior, B. J., Haranczyk, M., Lee, S., Zhang, H., Vogiatzis, K. D., Milisavljevic, M., Ling, S., Camp, J. S., Slater, B., Siepmann, J. I., Sholl, D. S., & Snurr, R. Q. (2019). Advances, updates, and analytics for the computation-ready, experimental metal--organic framework database: CoRE MOF 2019. Journal of Chemical & Engineering Data, 64(12), 5985–5998. https://doi.org/10.1021/acs.jced.9b00835
- [46]Moss, T. S. (1985). Relations between the refractive index and energy gap of semiconductors. Physica Status Solidi (b), 131(2), 415–427. https://doi.org/10.1002/pssb.2221310202
- [47]Iglewicz, B., & Hoaglin, D. C. (1993). How to Detect and Handle Outliers (Vol. 16). ASQC Quality Press.
- [48]Leys, C., Ley, C., Klein, O., Bernard, P., & Licata, L. (2013). Detecting outliers: Do not use standard deviation around the mean, use absolute deviation around the median. Journal of Experimental Social Psychology, 49(4), 764–766. https://doi.org/10.1016/j.jesp.2013.03.013
- [49]Rousseeuw, P. J., & Croux, C. (1993). Alternatives to the median absolute deviation. Journal of the American Statistical Association, 88(424), 1273–1283. https://doi.org/10.1080/01621459.1993.10476408
- [50]Birge, R. T. (1932). The calculation of errors by the method of least squares. Physical Review, 40(2), 207–227. https://doi.org/10.1103/PhysRev.40.207
- [52]Tiesinga, E., Mohr, P. J., Newell, D. B., & Taylor, B. N. (2021). CODATA recommended values of the fundamental physical constants: 2018. Reviews of Modern Physics, 93(2), 025010. https://doi.org/10.1103/RevModPhys.93.025010
- [53]Breunig, M. M., Kriegel, H.-P., Ng, R. T., & Sander, J. (2000). LOF: Identifying density-based local outliers. Proceedings of the 2000 ACM SIGMOD International Conference on Management of Data, 93–104. https://doi.org/10.1145/342009.335388
- [54]Liu, F. T., Ting, K. M., & Zhou, Z.-H. (2008). Isolation forest. 2008 Eighth IEEE International Conference on Data Mining (ICDM), 413–422. https://doi.org/10.1109/ICDM.2008.17
- [55]Chandola, V., Banerjee, A., & Kumar, V. (2009). Anomaly detection: A survey. ACM Computing Surveys, 41(3), 15. https://doi.org/10.1145/1541880.1541882
- [56]Benford, F. (1938). The law of anomalous numbers. Proceedings of the American Philosophical Society, 78(4), 551–572. https://www.jstor.org/stable/984802jstor.org/stable/984802
- [57]Northcutt, C. G., Jiang, L., & Chuang, I. L. (2021). Confident learning: Estimating uncertainty in dataset labels. Journal of Artificial Intelligence Research, 70, 1373–1411. https://doi.org/10.1613/jair.1.12125
- [58]Brodley, C. E., & Friedl, M. A. (1999). Identifying mislabeled training data. Journal of Artificial Intelligence Research, 11, 131–167. https://doi.org/10.1613/jair.606
- [59]Frénay, B., & Verleysen, M. (2014). Classification in the presence of label noise: A survey. IEEE Transactions on Neural Networks and Learning Systems, 25(5), 845–869. https://doi.org/10.1109/TNNLS.2013.2292894
- [60]Song, H., Kim, M., Park, D., Shin, Y., & Lee, J.-G. (2023). Learning from noisy labels with deep neural networks: A survey. IEEE Transactions on Neural Networks and Learning Systems, 34(11), 8135–8153. https://doi.org/10.1109/TNNLS.2022.3152527
- [61]Lakshminarayanan, B., Pritzel, A., & Blundell, C. (2017). Simple and scalable predictive uncertainty estimation using deep ensembles. Advances in Neural Information Processing Systems, 30, 6402–6413.
- [62]Koh, P. W., & Liang, P. (2017). Understanding black-box predictions via influence functions. Proceedings of the 34th International Conference on Machine Learning, 70, 1885–1894.
- [63]Ghorbani, A., & Zou, J. (2019). Data Shapley: Equitable valuation of data for machine learning. Proceedings of the 36th International Conference on Machine Learning, 97, 2242–2251.
- [64]Zhang, C., Bengio, S., Hardt, M., Recht, B., & Vinyals, O. (2017). Understanding deep learning requires rethinking generalization. 5th International Conference on Learning Representations (ICLR).
- [65]Huber, P. J. (1964). Robust estimation of a location parameter. The Annals of Mathematical Statistics, 35(1), 73–101. https://doi.org/10.1214/aoms/1177703732
- [66]Friedman, J. H. (2001). Greedy function approximation: A gradient boosting machine. The Annals of Statistics, 29(5), 1189–1232. https://doi.org/10.1214/aos/1013203451
- [67]Lookman, T., Balachandran, P. V., Xue, D., & Yuan, R. (2019). Active learning in materials science with emphasis on adaptive sampling using uncertainties for targeted design. Npj Computational Materials, 5, 21. https://doi.org/10.1038/s41524-019-0153-8
- [68]Settles, B. (2009). Active Learning Literature Survey. In University of Wisconsin--Madison, Department of Computer Sciences (Number 1648). http://digital.library.wisc.edu/1793/60660digital.library.wisc.edu/1793/60660
- [69]Miller, R. C. (1964). Optical second harmonic generation in piezoelectric crystals. Applied Physics Letters, 5(1), 17–19. https://doi.org/10.1063/1.1754022
- [70]Smith, J. E., & Winkler, R. L. (2006). The optimizer's curse: Skepticism and postdecision surprise in decision analysis. Management Science, 52(3), 311–322. https://doi.org/10.1287/mnsc.1050.0451
- [71]Krauser, J., Albrecht, P., Bornholdt, C., Döldissen, W., Niggebrügge, U., Nolting, H.-P., & Schlak, M. (1986). Electro-optic modulators in GaInAsP/InP. Fiber Optic Sources and Detectors, 587, 180. https://doi.org/10.1117/12.951218
- [72]Jeon, Y., & Kang, H. S. (2007). Electro-Optic Coefficient Measurements for Zn$_x$Cd$_1-x$Te Single Crystals at 1550 nm Wavelength. Optical Review, 14(6), 373–375. https://doi.org/10.1007/s10043-007-0373-x
- [74]Lauk, N., Sinclair, N., Barzanjeh, S., Covey, J. P., Saffman, M., Spiropulu, M., & Simon, C. (2020). Perspectives on quantum transduction. Quantum Science and Technology, 5(2), 020501. https://doi.org/10.1088/2058-9565/ab788a
- [75]Han, X., Fu, W., Zou, C.-L., Jiang, L., & Tang, H. X. (2021). Microwave-optical quantum frequency conversion. Optica, 8(8), 1050. https://doi.org/10.1364/OPTICA.425414
- [76]Holzgrafe, J., Sinclair, N., Zhu, D., Shams-Ansari, A., Colangelo, M., Hu, Y., Zhang, M., Berggren, K. K., & Lončar, M. (2020). Cavity electro-optics in thin-film lithium niobate for efficient microwave-to-optical transduction. Optica, 7(12), 1714. https://doi.org/10.1364/OPTICA.397513
- [77]Sahu, R., Hease, W., Rueda, A., Arnold, G., Qiu, L., & Fink, J. M. (2022). Quantum-enabled operation of a microwave-optical interface. Nature Communications, 13, 1276. https://doi.org/10.1038/s41467-022-28924-2
- [78]Yariv, A., & Yeh, P. (1984). Optical Waves in Crystals: Propagation and Control of Laser Radiation. Wiley.
- [79]Onsager, L. (1931). Reciprocal relations in irreversible processes. I. Physical Review, 37(4), 405–426. https://doi.org/10.1103/PhysRev.37.405
- [80]Hall, E. O. (1951). The deformation and ageing of mild steel: III discussion of results. Proceedings of the Physical Society. Section B, 64(9), 747–753. https://doi.org/10.1088/0370-1301/64/9/303
- [81]Cordero, Z. C., Knight, B. E., & Schuh, C. A. (2016). Six decades of the Hall--Petch effect -- a survey of grain-size strengthening studies on pure metals. International Materials Reviews, 61(8), 495–512. https://doi.org/10.1080/09506608.2016.1191808
- [82]Hill, R. (1952). The elastic behaviour of a crystalline aggregate. Proceedings of the Physical Society. Section A, 65(5), 349–354. https://doi.org/10.1088/0370-1298/65/5/307
- [83]Goodenough, J. B., & Park, K.-S. (2013). The Li-ion rechargeable battery: A perspective. Journal of the American Chemical Society, 135(4), 1167–1176. https://doi.org/10.1021/ja3091438
- [84]Mouhat, F., & Coudert, F.-X. (2014). Necessary and sufficient elastic stability conditions in various crystal systems. Physical Review B, 90(22), 224104. https://doi.org/10.1103/PhysRevB.90.224104
- [85]Mitchell, M. J., Billingsley, M. M., Haley, R. M., Wechsler, M. E., Peppas, N. A., & Langer, R. (2021). Engineering precision nanoparticles for drug delivery. Nature Reviews Drug Discovery, 20(2), 101–124. https://doi.org/10.1038/s41573-020-0090-8
- [86]Denton, A. R., & Ashcroft, N. W. (1991). Vegard's law. Physical Review A, 43(6), 3161–3164. https://doi.org/10.1103/PhysRevA.43.3161
- [87]Laidler, K. J. (1984). The development of the Arrhenius equation. Journal of Chemical Education, 61(6), 494. https://doi.org/10.1021/ed061p494
- [88]Lucarini, V., Saarinen, J. J., Peiponen, K.-E., & Vartiainen, E. M. (2005). Kramers--Kronig Relations in Optical Materials Research (Vol. 110). Springer. https://doi.org/10.1007/b138913
- [89]Altman, D. G., & Bland, J. M. (2005). Standard deviations and standard errors. BMJ, 331(7521), 903. https://doi.org/10.1136/bmj.331.7521.903
- [90]Callen, H. B. (1985). Thermodynamics and an Introduction to Thermostatistics. Wiley.
- [91]Bard, A. J., & Faulkner, L. R. (2001). Electrochemical Methods: Fundamentals and Applications. Wiley.
- [92]Joint Committee for Guides in Metrology. (2008). Evaluation of measurement data -- Supplement 1 to the ``Guide to the expression of uncertainty in measurement'' -- Propagation of distributions using a Monte Carlo method. In BIPM (Numbers JCGM 101:2008).