betaFeedback

WorksComputer Science

Confidence-Based Materials Data Quality Assessment via Domain Knowledge and Graph-Structured Learning

Abstract

The rapid development of materials machine learning relies on large-scale, heterogeneous datasets, yet model reliability is often compromised by imperfect data from diverse experiments, simulations, and database integration. Real-world materials databases may contain measurement errors, failed calculations, missing or duplicate records, and hidden violations of physicochemical constraints. Such problems are difficult to identify because corrupted samples may remain statistically plausible rather than appearing as explicit outliers. In this paper, we propose a confidence-based materials data quality assessment framework that integrates domain knowledge with graph-structured learning. Data quality is formulated as a continuous confidence estimation problem rather than a binary anomaly identification task, enabling sample-level assessment and historical-reference-based batch comparison. A nine-dimensional feature-quality prior and duplicate-aware sample initialization are calibrated using materials domain rules and statistical evidence. Confidence is further refined through sample-feature bipartite graph propagation and weakly supervised graph attention, while a Memory-GAT module provides historical reference scoring. Experiments on 23 ODDS benchmark datasets show that the generic module achieves an average AUC-ROC of 0.866. In five repeated mixed-contamination experiments on mp_20, rule calibration improves HistGradientBoostingRegressor test R2R^2 from 0.538±0.0080.538\pm0.008 to 0.616±0.0140.616\pm0.014. Graph refinement increases average precision from 0.943±0.0040.943\pm0.004 to 0.953\pm0.004,whiletherule−calibratedconfigurationhasthehighestmeantest, while the rule-calibrated configuration has the highest mean test R^2amongtheevaluatedscreeningconfigurations.FiverepeatedMACEsubsetexperimentsfurthershowanincreasein among the evaluated screening configurations. Five repeated MACE subset experiments further show an increase in R^2from from 0.590\pm0.053to to 0.880\pm0.027$ after rule-based screening.

Introduction

Data-driven machine learning has emerged as a fundamental paradigm in materials science, enabling large-scale exploration of composition-structure-property relationships and accelerating the transition from empirical trial-and-error toward data-driven materials design [1]. However, model performance is determined not only by algorithmic architectures but also by the reliability, representativeness, and scientific consistency of the training data. Low-quality data can distort learned structure-property relationships, reduce model generalization, and introduce unreliable predictions into subsequent materials screening and design workflows. Therefore, systematic data quality assessment and continuous quality control are essential for developing reliable and trustworthy artificial intelligence (AI) systems in materials science.

Despite the success of several world-class materials databases, including Materials Project [2], OQMD [3], AFLOWlib [4], NOMAD [5], and the Materials Data Facility [6], maintaining consistent and reliable data quality remains challenging due to the diversity of computational workflows, experimental measurements, and curation procedures involved. Nevertheless, records in these databases may contain unconverged density-functional theory calculations [7], failed structural relaxations, unreasonable crystal structures, inconsistent units or field semantics, missing key properties, duplicate entries, and violations of chemical or physical constraints. Importantly, many of these issues cannot be detected by conventional statistical criteria because corrupted samples may remain within physically plausible distributions while violating underlying scientific constraints.

Existing data quality assessment and anomaly detection methods [8], including multidimensional data quality frameworks [9], distance- and density-based detectors, ensemble outlier methods [10], reconstruction-error models, and deep representation-learning approaches for tabular anomaly detection [11], have provided valuable tools for identifying corrupted records in generic datasets. However, their direct application to materials science data remains challenging due to three fundamental limitations. First, many approaches focus on a single quality dimension or binary anomaly labels, rather than constructing a unified and interpretable quality representation across samples, features, and datasets [12]. Second, purely statistical methods are sensitive to distribution shapes and feature scales, and their performance can be degraded in sparse and high-dimensional materials descriptor spaces [13]. Third, many low-quality materials records arise from violations of physical constraints, chemical relationships, structural consistency, or experimental conditions rather than simple numerical deviations.

To incorporate scientific knowledge into data quality evaluation, recent studies have explored knowledge-assisted approaches that combine domain rules with statistical analysis [14]. Knowledge-assisted methods demonstrate that expert rules can improve scientific consistency evaluation and provide inspectable explanations for low-confidence records [15]. However, these approaches still lack a unified framework that integrates domain knowledge, statistical evidence, and relational structures among data records for continuous, interpretable, and comparable quality assessment. In addition, materials science data are often submitted in batches, sources, or projects, making relative confidence scores within a single dataset insufficient for cross-batch quality comparison and acceptance [16].

This paper proposes a confidence-based materials data quality assessment framework via domain knowledge and graph-structured learning, where data quality is formulated as a continuous confidence estimation problem rather than a binary anomaly detection task. The framework first constructs a domain-informed quality prior and then refines confidence through heterogeneous graph learning, including global sample-feature consistency propagation, local neighborhood refinement, and historical reference-based quality assessment. An overview of the proposed framework is presented in Figure 1. The quality concepts summarize the assessment goals; the numerical indicators and calibration rules are specified in § 3.1–§ 3.2.

Overview of the proposed confidence-based materials data quality assessment framework

Figure 1. Overview of the proposed confidence-based materials data quality assessment framework. The five stages connect materials data, quality priors, domain evidence, graph refinement and assessment outputs.

The main contributions of this work are summarized as follows:

  1. We formulate materials data quality assessment as a continuous confidence estimation problem, enabling interpretable sample and feature assessment beyond binary anomaly identification;

  1. We develop a domain-informed quality prior and hierarchical calibration strategy that integrates physicochemical constraints with statistical evidence;

  1. We introduce graph-based confidence refinement at the sample–feature and sample-neighborhood levels, together with historical reference scoring, and evaluate their contributions separately from rule calibration;

  1. We evaluate the framework through generic anomaly benchmarks, repeated controlled-contamination and downstream prediction experiments, coefficient sensitivity analysis, and an interpretable materials-data case study.

Related work

Materials data quality assessment

Data quality assessment has evolved from single-dimensional statistical checks to multidimensional reliability evaluation frameworks. The “fitness for use” perspective emphasizes that data quality should be evaluated according to accuracy, completeness, consistency, accessibility, and task relevance [17]. With the increasing adoption of machine learning in scientific discovery, data quality has become closely associated with model reliability, uncertainty, and downstream generalization capability [18]. However, most existing data quality frameworks were originally developed for information systems or industrial data governance, where quality dimensions mainly focus on structural integrity, operational consistency, and data management requirements rather than scientific validity [19].

In materials science, large-scale data infrastructures have greatly reduced the cost of accessing crystal structures, calculated properties, and experimental records. However, their primary roles are storage, sharing, retrieval, and high-throughput data aggregation [20]. Automated quality assessment for scientific reliability remains comparatively limited. Data governance for materials machine learning must simultaneously handle statistical outliers, semantic mismatches, unreasonable structures, failed calculations, and violations of domain rules. Therefore, materials data quality assessment should evaluate not only statistical abnormality, but also scientific plausibility and suitability for downstream learning tasks [21].

Scientific data quality assessment and anomaly identification

Although anomaly detection provides an effective mechanism for identifying suspicious records, conventional approaches usually define anomalies according to statistical rarity rather than scientific reliability. Classical anomaly detection approaches include distance-based, density-based [22], statistical, clustering-based [23], and one-class classification methods. In materials data, anomaly detection methods generally target three aspects. The first concerns crystal-structure plausibility, including interatomic distances, lattice parameters, space groups, and coordination environments. The second focuses on property reliability, including formation energy, band gap, energy above hull, and calculation convergence. The third addresses multi-source data consistency, including missing values, duplicate entries, field mismatches, unit inconsistency, and abnormal property relationships.

The main limitation of purely data-driven anomaly detection is that it measures deviation from the current data distribution, but cannot directly determine whether a sample obeys materials-science rules [15]. For example, negative band gaps, out-of-range space-group identifiers, and inconsistencies between lattice matrices and lattice constants should be treated as low-quality evidence even when they are not extreme in a low-dimensional statistical projection. These limitations indicate that reliable materials data assessment requires a unified framework that integrates statistical evidence, domain constraints, and relational information among data entities [24].

Domain knowledge integration

Domain knowledge integration aims to convert expert rules, physical constraints, and empirical relationships into explicit quality indicators. Knowledge-assisted approaches have shown that materials rules can improve quality evaluation and provide explanations that domain experts can audit. Representative methods, such as DKA-DAD [15] and DADMmdk [25], introduce materials rules into anomaly detection and demonstrate the value of expert knowledge in identifying scientifically inconsistent records.

However, two limitations remain. First, rules are often triggered in isolation, rather than fused with statistical evidence in a unified scoring mechanism. Second, rule evidence is difficult to propagate between samples and features, so local defects may be handled as isolated events instead of being incorporated into a globally consistent quality judgment. These limitations motivate the development of methods that can integrate rule-based evidence, statistical signals, and relational structures into a unified quality representation.

Graph learning and confidence quantification

Graph-based methods provide a natural way to capture dependencies among samples, features, and reference data [26] in structured datasets. Graph propagation approaches enable confidence information to spread through relational structures, while graph attention mechanisms learn adaptive importance among neighboring entities. Personalized PageRank introduces restart-based propagation and is suitable for spreading confidence over a sample–feature bipartite graph [27]. Graph attention networks [28] allow each node to assign adaptive weights to its neighbors and can suppress the influence of abnormal neighbors during representation learning. In a data-quality setting, graph propagation provides global consistency, while graph attention provides local anomaly isolation [29].

Because data quality is influenced by interactions among samples, features, and historical references, graph-based methods provide a promising representation for modeling quality propagation. Confidence-based learning extends discrete error identification to probabilistic quality estimation [30]. This idea provides useful inspiration for modeling data quality as continuous confidence rather than binary normal/anomalous labels. This perspective motivates extending confidence modeling from individual prediction reliability to structured scientific data quality assessment. Moreover, historical quality anchors provide a reference for batch-level comparison. The Memory-GAT module uses these anchors to relate current batch scores to a common historical reference.

Overall, existing studies have advanced materials data quality control from statistical screening toward knowledge-assisted evaluation. However, a unified framework that simultaneously considers domain constraints, statistical evidence, relational structures, and historical-reference batch comparison remains insufficiently explored. Addressing this gap requires a continuous and interpretable quality representation capable of integrating heterogeneous quality signals across materials datasets.

Method

The framework assigns confidence scores to samples and features, then updates them using statistical checks, materials knowledge and graph relationships. These scores express data reliability on a scale from zero to one. They are used to rank records and identify fields that need inspection; they are not estimates of prediction uncertainty. The workflow consists of prior construction, evidence calibration, sample–feature propagation, local attention refinement and historical-reference scoring (Figure 2).

Workflow of the proposed materials data quality assessment framework

Figure 2. Workflow of the proposed materials data quality assessment framework. The diagram summarizes prior construction, evidence calibration, graph refinement and quality outputs. The feature indicators are defined in § 3.1. The schematic calibration factors are implemented using the minimum cell penalty for each sample and the mean cell penalty for each feature, as specified in Eq. (2).

Let the dataset be X={xi}i=1nX = \{x_i\}_{i=1}^{n}, where xi∈Rdx_i \in\mathbb{R}^{d} denotes the ii-th sample, and nn and dd denote the numbers of samples and features, respectively. Sample–feature confidence scores at successive stages are denoted by subscripts: CinitC_{\mathrm{init}} for initial confidence, CcalC_{\mathrm{cal}} for calibrated confidence, CgraphC_{\mathrm{graph}} for graph-propagated confidence, and CfinalC_{\mathrm{final}} for final confidence. Feature confidence is denoted analogously by ZinitZ_{\mathrm{init}}, ZcalZ_{\mathrm{cal}}, and ZgraphZ_{\mathrm{graph}}. A lower confidence score indicates a higher likelihood of quality defects.

Nine-dimensional quality prior

The initial feature confidence combines nine indicators derived from metadata and numerical data. The indicators translate broad data-quality concepts [31] into quantities that can be calculated for each field. Let qj,kq_{j,k} be the score of feature jj on indicator kk, with weight vkv_k. The initial feature and sample scores are

Zinit,j=∑k=19vkqj,k,Cinit,i={0.2,exact duplicate numerical row,1,otherwise.(1)Z_{\mathrm{init},j} = \sum_{k=1}^{9} v_k q_{j,k}, \qquad C_{\mathrm{init},i} = \begin{cases} 0.2, & \text{exact duplicate numerical row},\\ 1, & \text{otherwise.} \end{cases} \tag*{(1)}

The feature weights sum to one and are fixed at (0.05, 0.15, 0.15, 0.08, 0.20, 0.05, 0.10, 0.10, 0.12) for documentation availability, statistical-property matching, numeric compatibility, metadata coupling, non-missing rate, uniqueness, distributional skewness, numerical regularity and descriptor-variation relevance, respectively. Every record in an exact duplicate group receives the lower initial sample score. The weighted feature indicators and the sample scores are then updated by the evidence checks.

The weights are fixed throughout the main experiments. Appendix § 10.§ 3 defines each indicator and explains which become constant after preprocessing. The last indicator measures the correlation between a feature and the first principal component of the input descriptors. § 5.3 examines sensitivity to the weights and penalty coefficients.

SymbolMeaning
qj,vkq_j, v_kFeature indicator kk and its normalized weight
Cinit,ZinitC_{\mathrm{init}}, Z_{\mathrm{init}}Initial sample confidence accounting for duplicates, and weighted feature prior
Ccal,ZcalC_{\mathrm{cal}}, Z_{\mathrm{cal}}Sample and feature confidence after evidence calibration
aij,Bija_{ij}, B_{ij}Cell penalty and sample–feature edge reliability
P,αP, \alphaRow-stochastic transition and restart coefficient
Cgraph,ZgraphC_{\mathrm{graph}}, Z_{\mathrm{graph}}Scores after bipartite propagation
ri,Cfinal,ir_i, C_{\mathrm{final},i}Rule-trained GAT risk and final sample confidence
mi,QDm_i, Q_DHistorical-reference sample score and batch mean

Table 1. Main notation used in the confidence-assessment workflow.

Hierarchical materials-knowledge calibration

A record can fall within a common numerical range while violating a materials constraint. Calibration therefore uses three types of evidence: individual field ranges, relationships between fields and deviations from the batch distribution [32]. In the controlled benchmark, these checks cover configured value ranges, agreement between lattice representations, and standardized-score/IQR checks. Property targets, identifiers and injected anomaly labels are excluded from scoring.

Evidence is mapped to a cell penalty aija_{ij}: severe evidence gives 0.3, mild evidence gives 0.7, and unaffected cells give 1.0. Severe evidence takes precedence, so overlapping checks do not multiply the same cell repeatedly. The calibrated confidence scores are

Ccal,i=Cinit,imin⁡jaij,Zcal,j=Zinit,j1n∑iaij.(2)C_{\mathrm{cal},i}=C_{\mathrm{init},i}\min_j a_{ij}, \qquad Z_{\mathrm{cal},j}=Z_{\mathrm{init},j}\frac{1}{n}\sum_i a_{ij}. \tag*{(2)}

Here Si=min⁡jaijS_i=\min_j a_{ij} and Tj=n−1∑iaijT_j=n^{-1}\sum_i a_{ij} are the sample and feature factors used in the workflow. The evidence-channel factors shown schematically in Figure 2 are implemented by these aggregate penalties: the minimum cell penalty for a sample and the mean cell penalty for a feature. The severe and mild coefficients remain fixed in the main experiments and are varied in the sensitivity analysis.

The sample-feature reliability matrix uses Bij=1B_{ij}=1, 0.5 and 0.1 for unaffected, mildly flagged and severely flagged cells. Table 2 specifies the implemented checks and their screening ranges.

EvidenceEvidence contentInterpretation
Univariate boundaryBand gap [0,10] eV; energy above hull [0,0.5] eV/atom; space group [1,230]; lattice lengths [2, 30] Å; angles [30,150] degrees; fractional coordinates [0,1]. Outside a range is severe.Configured plausibility checks; coordinate wrapping must be declared.
Multivariate relationshipRelative mismatch between a lattice length and its lattice-matrix row norm exceeds 0.05; all four involved cells receive severe evidence.Inconsistency between stored representations.
Global distributionPopulation-standardized ∣z∣>5|z| > 5: severe; ∣z∣>3|z| > 3 or outside the Tukey 1.5 IQR interval: mild unless severe. Zero standard deviation or IQR is replaced by one.Batch-level deviation evidence.

Table 2. Representative quality-evidence types and their effects on confidence.

Sample-feature bipartite graph propagation

Calibration first scores each record and field from the checks they trigger. A bipartite graph then links sample nodes to feature nodes [33], using BijB_{ij} as the weight between sample ii and feature jj. Each sample receives information from its associated features, and each feature receives information from the samples containing it.

This exchange allows the confidence of a record to reflect both its own checks and the reliability of its fields. In turn, feature confidence reflects the records in which that feature appears [34]. Let

Hcal=[CcalZcal]H_{\mathrm{cal}} = \begin{bmatrix} C_{\mathrm{cal}}\\ Z_{\mathrm{cal}} \end{bmatrix}

collect the calibrated sample and feature scores. The propagation matrix is

P=[0Ds−1BDf−1BT0],(3)P = \begin{bmatrix} 0 & D_s^{-1}B\\ D_f^{-1}B^{\mathsf{T}} & 0 \end{bmatrix}, \tag*{(3)}

where Ds=diag⁡(B1d)D_s=\operatorname{diag}(B\mathbf{1}_d) and Df=diag⁡(BT1n)D_f=\operatorname{diag}(B^{\mathsf{T}}\mathbf{1}_n) are the separate sample and feature degree matrices. Thus each outgoing block is row-normalized. The confidence propagation process is then written as

H(t+1)=αHcal+(1−α)PH(t).(4)H^{(t+1)}=\alpha H_{\mathrm{cal}}+(1-\alpha)PH^{(t)}. \tag*{(4)}

Here, α=0.35\alpha=0.35 controls how much calibrated confidence is retained at each iteration [35]. The first term preserves the original evidence, and the second averages the scores of neighboring nodes using the weights in PP. Starting from H(0)=HcalH^{(0)}=H_{\mathrm{cal}}, iteration stops when the largest score change is below 10−810^{-8}, or after 400 iterations. The resulting sample and feature scores are CgraphC_{\mathrm{graph}} and ZgraphZ_{\mathrm{graph}}. Appendix § 10.§ 4 establishes convergence and checks the implementation against independent calculations.

Graph attention local refinement

Bipartite propagation combines information across the sample–feature graph. Local refinement then uses a sample KNN graph to compare records with similar descriptors. Attention learns how strongly each neighboring record contributes to the risk score.

For each sample, the normalized raw feature vector is first weighted by graph-propagated feature confidence and then concatenated with the sample’s own graph-propagated confidence:

Ui=[x~i⊙Zgraph∥Cgraph,i](5)U_i = [\tilde{x}_i \odot Z_{\mathrm{graph}} \mathbin{\|} C_{\mathrm{graph},i}] \tag*{(5)}

where x~i\tilde{x}_i denotes the normalized feature vector, ⊙\odot denotes elementwise multiplication, and ∥\mathbin{\|} denotes vector concatenation. This operation reduces the influence of low-confidence features and preserves more information from high-confidence features. A sample KNN graph is then constructed from UiU_i, and graph attention learns adaptive neighbor weights [36]. The KNN graph has eight incoming neighbors and a self-loop per sample. The two-layer GAT uses four first-layer heads with 16 outputs each, ELU and dropout 0.05, followed by a sigmoid risk output. It is trained for 50 full-batch epochs with Adam (learning rate 10−310^{-3}, weight decay 10−510^{-5}, gradient clipping one). The training target is the rule-derived risk ti=1−Ccal,it_i = 1 - C_{\mathrm{cal},i}:

LGAT=1n∑i(ri−ti)2.(6)\mathcal{L}_{\mathrm{GAT}} = \frac{1}{n}\sum_i (r_i - t_i)^2. \tag*{(6)}

Neither injected anomaly labels nor downstream property targets enter this loss. The model learns to reproduce this rule-derived risk using the sample and its neighbors. Let ri∈[0,1]r_i \in[0, 1] be the risk score output by the GAT, where a larger value indicates a more suspicious sample. The final sample confidence is updated as

Cfinal,i=clip⁡(Cgraph,i(1−λri),0,1),(7)C_{\mathrm{final},i} = \operatorname{clip}\left(C_{\mathrm{graph},i}(1-\lambda r_i), 0, 1\right), \tag*{(7)}

where λ=0.5\lambda= 0.5 controls the impact of GAT refinement. If the GAT estimates high risk, the confidence is further decreased from the graph-propagated value. If the risk is low, the original confidence is largely retained. A matched GAT configuration uses CcalC_{\mathrm{cal}}, ZcalZ_{\mathrm{cal}} in place of the propagated scores, with the same objective and training budget. These comparisons distinguish the contributions of local attention and bipartite propagation.

Historical-reference Memory-GAT

Traditional anomaly scores are dataset-dependent. Historical quality memory provides a reference for comparing newly arrived batches with previously assessed samples. The memory stores descriptors, confidence, rule evidence and sample fingerprints. Historical samples covering a range of confidence scores are selected as frozen anchors [37]. Their reference scores are obtained through rule calibration and graph propagation. The anchor descriptors and scores remain fixed while the attention model is fitted for the current batch.

The memory graph uses standardized descriptors and transformed reference confidence scores. Each current node receives messages from eight historical neighbors, six current neighbors and itself. Historical nodes receive only their own self-loop messages. A two-layer attention encoder with a sigmoid output is trained to reproduce the historical reference scores and the current rule/propagation scores. A smoothness term encourages connected nodes to have similar outputs. Appendix § 10.§ 5 gives the inputs, loss and training settings. Injected anomaly labels are excluded from training.

For current-sample scores mim_i, the batch score is

QD=1∣D∣∑i∈Dmi.(8)Q_D = \frac{1}{|D|}\sum_{i\in D} m_i. \tag*{(8)}

The mean QDQ_D summarizes the quality of the current batch relative to the historical anchors. § 5.5 compares its ordering of controlled batches with batch-only scores and a distance-weighted historical KNN reference.

Quality representation and applications

The framework outputs sample confidence Cfinal,iC_{\mathrm{final},i}, feature confidence Zgraph,jZ_{\mathrm{graph},j}, and the corresponding rule-trigger evidence. Sample confidence supports quality ranking and helps users prioritize low-confidence records for inspection. Feature confidence summarizes the quality of individual fields and helps identify columns requiring further checks. Rule-trigger evidence specifies the affected fields and the checks triggered by each record, providing a basis for verification against the original data source.

When historical quality memory is available, the framework uses historical anchors to calculate a reference-based score mim_i for each current sample. The dataset-level quality score QDQ_D is the mean of these scores over the current batch, supporting comparison of overall quality across batches. Appendix § 10.§ 1 presents the complete assessment workflow.

Experimental setup

The experiments assess anomaly identification, materials data screening, the contribution of each module, interpretation of flagged records and comparison of batches. They include 23 ODDS datasets [38], controlled contamination of mp_20 with downstream prediction, ablation and sensitivity tests, an alloy-data case study and historical-reference batch scoring.

Generic anomaly identification benchmark

The ODDS experiment tests anomaly identification without materials-specific rules. It retains the feature prior, statistical checks and graph refinement.

The method is evaluated on 23 ODDS datasets using AUC-ROC. The suite includes datasets that could be retrieved and processed consistently. Datasets with unavailable download links and exceptionally large datasets with very few features were excluded.

The baselines cover statistical methods, neighborhood-based detectors, representation-learning approaches, and large-language-model-based anomaly detection [38].

Materials data cleaning and downstream validation

The mp_20 dataset contains 45,229 records. Five independently sampled 16,000-record subsets (seeds 0–4) are deduplicated by material identifier and grouped by exact composition label and nine unrounded lattice-matrix entries. Grouped splits yield approximately 12,000 outer-training and 4,000 clean test records; the outer-training partition is further divided into 10,000 fitting and 2,000 calibration records. Feature eligibility and median imputation are determined from outer training, excluding identifiers, property targets and auxiliary property fields. The main runs retain 43 numerical descriptors.

Corruption is applied only after splitting. Half of each fitting or calibration batch is contaminated, evenly across statistical outliers, configured boundary violations, lattice-representation inconsistency and mixed perturbations. The same records receive zero-mean target noise with standard deviation 2.8 times the clean batch target standard deviation. The mixed-input perturbation scale is 0.6σ0.6\sigma. This setup tests screening when both descriptors and target values contain errors. Appendix § 10.§ 6 gives the corruption procedure.

Scoring uses input descriptors and the evidence derived from them. The formation-energy target is excluded from prior construction, scoring, graph construction and GAT training. For local GAT refinement, fitting and calibration batches are each standardized using their own column means and standard deviations; a local GAT is trained separately on each batch using its rule-derived risk targets. For each method, the threshold that maximizes F1 on the synthetically labeled calibration batch is then applied to the independently scored fitting batch. The clean test data are used only for downstream evaluation.

HistGradientBoostingRegressor [39] uses 220 iterations, learning rate 0.055 and L2 regularization 0.02. All methods share the same clean test records within each seed. Isolation Forest (50 estimators, contamination 0.1), PCA (95% variance) and random retention provide comparisons. For each seed, the random control retains the same number of fitting records as the corresponding screening method. In the tables, the label rules and full identify the method whose retained count is matched; random selection itself uses no confidence scores.

MACE [40] is evaluated on five grouped 4,000-record subsets with approximately 2,500 fitting, 500 clean validation and 1,000 clean test crystals before shared structural eligibility filtering. Thresholds use separately corrupted copies of the calibration partition. Every method shares the eligible clean test set; the clean validation loss selects the model, and test predictions are computed once after training. The repeated subset study comprises five splits and seven conditions, totaling 35 MACE training runs. MACE uses random initialization, two interactions, hidden irreps 128x0e+128x1o, radius 4 Å, float32, batch size 32, learning rate 0.005 and a common ten-epoch budget. Only energy is trained; forces have zero loss weight and stage two is disabled. Appendix § 10.§ 6 specifies structural filtering, remaining settings and evaluation procedure.

Ablation and robustness analysis

The ablation compares six configurations: statistical checks, boundary checks, complete rules, rules with propagation, rules with GAT, and rules with both graph stages. This separates the effects of relationship checks, sample–feature propagation and local attention. Historical memory is tested in the batch-comparison experiment.

Robustness crosses contamination fractions 10%, 30% and 50% with mixed-descriptor scales 0.1σ0.1\sigma, 0.3σ0.3\sigma and 0.6σ0.6\sigma, while holding other corruption mechanisms and target-noise multipliers fixed. Parameter sensitivity includes nine severe/mild penalty combinations, equal feature weights, and individual weight perturbation or omission. All configurations are repeated over seeds 0–4; the full weight results are in Appendix § 10.7. Additional diagnostics use a fixed threshold or a prespecified 50% deletion budget; these are distinguished from the main results, which use thresholds selected on the calibration partition.

Materials data governance case study

To demonstrate interpretability in a realistic materials data governance scenario, this study uses an alloy mechanical-property dataset collected from public literature [41, 42, 43]. The primary case-study table contains 185 records and 50 fields, comprising one alloy identifier, 14 elemental-composition variables, one testing-temperature variable, two mechanical-property variables—ultimate tensile strength (UTS) and fracture elongation (FE)—and 32 composition-derived physicochemical, thermodynamic, and elastic descriptors. A larger source-literature reference table containing 315 records and 61 fields is used as an auxiliary source for statistical comparison and fitting-based verification. It comprises one sample identifier, the same 14 elemental-composition variables, 12 processing-history variables describing homogenization, deformation, rolling, annealing, and aging, the two mechanical-property variables, and the same 32 material descriptors. Detailed definitions and categories of these fields are provided in Appendix § 10.10.

The case study examines whether each flagged record can be linked to a specific quality concern. It combines range checks, relationships between fields, distribution checks, missingness and differences between reported and model-predicted properties. The audit also records whether processing-history information is available.

Samples outside configured screening ranges or with property differences at matched recorded composition and temperature are treated as review candidates. Processing-history availability is included in the review evidence. Samples identified primarily through statistical or global distribution evidence are further examined using KNN and Random Forest regressors trained under five-fold cross-validation [44]. The regressors predict ultimate tensile strength (UTS) and fracture elongation (FE), and large fitting residuals are used as additional evidence of potential inconsistency [45]. The goal is to evaluate whether the proposed method can generate auditable and interpretable quality assessments for public materials datasets.

Historical-reference batch comparison

Mutually disjoint mp_20 samples form three historical batches of 800 records at 0%, 10% and 30% contamination. Historical samples are sorted by their rule/propagation confidence scores, and 800 anchors are selected at evenly spaced positions in this ordering. Four current batches contain 900 records each at 0%, 10%, 30% and 50% contamination. All preprocessing for the 18 shared descriptors is fitted on history only.

The experiment is repeated over five seeds and compares batch-only scores, distance-weighted historical KNN and Memory-GAT. Mean scores and Spearman correlation with decreasing contamination assess controlled batch ordering. The experiment evaluates historical reference scoring across disjoint batches from a common materials source. Current contamination labels are used for evaluation.

Evaluation metrics and parameter settings

Anomaly ranking is evaluated by AUC-ROC and average precision (AP). Screening is evaluated by F1 at the calibration-selected threshold, with recall also reported for each corruption mechanism. Downstream prediction is evaluated by the coefficient of determination (R2R^2) and mean absolute error (MAE). Higher AUC-ROC, AP, F1, recall and R2R^2, and lower MAE, indicate better performance. For batch comparison, Spearman correlation measures agreement between batch confidence and decreasing contamination, so a positive correlation indicates higher scores for cleaner batches.

The ODDS benchmark retains its original seed-42 results. The materials experiments are repeated with seeds 0–4 and report mean ± sample standard deviation. Within each seed, methods share the data split and corruption realization. Differences in R2R^2 are calculated within each seed and summarized with 95% Student-tt intervals over the five runs.

Default severe/mild penalties are 0.3/0.7, edge weights are 0.1/0.5/1.0, restart is 0.35, and the GAT gate is 0.5. These manual settings remain fixed except in declared sensitivity experiments. KNN size is eight, the local GAT has two layers and hidden width 64, and historical memory retains 800 anchors. Full training and runtime settings are reported in the method and appendix.

Results and discussion

Generalization evaluation on ODDS benchmarks

Table 3 summarizes the average AUC-ROC and overall ranking on the ODDS benchmarks. Detailed per-dataset results are provided in Appendix § 10.11. With all materials-domain rules disabled, the proposed method achieves an average AUC-ROC of 0.866 on 23 ODDS datasets, showing that the generic configuration can rank anomalies without materials-specific rules.

MethodAverage AUC-ROCRank
AnoLLM-135M0.8921
ICL0.8842
DTE0.8823
KNN0.8784
AnoLLM-360M0.8675
Our method0.8666
NeuTraL0.8637
SLAD0.8488
RCA0.8439
IForest0.83910
PCA0.81411
DeepSVDD0.81212
ECOD0.78113
REPEN0.75614
GOAD0.72915

Table 3. Average AUC-ROC and ranking on ODDS benchmarks.

Note. AUC-ROC is the area under the receiver operating characteristic curve; the average is over 23 datasets. Rank orders methods by decreasing average AUC-ROC (1 is highest).

The ODDS results establish the anomaly-ranking performance of the generic configuration. The following materials experiments examine the contributions of rule calibration and graph refinement.

Materials data quality assessment and downstream model validation

The downstream experiments test whether screening improves models trained on contaminated materials data. Table 4 reports MACE results, and Table 5 compares screening methods and graph modules using HGBR. All methods are evaluated on the same clean test records within each seed.

Table 4: MACE downstream validation on five repeated grouped subsets (mean ± SD). F1 and retained counts follow the common structural eligibility check. MAE is in eV/atom. Matched random retains the same number of structures as complete rules.

ConditionF1R2R^2MAERetained
Clean reference–0.9387±0.01580.9387 \pm0.01580.1488±0.01950.1488 \pm0.01952256±222256 \pm22
No cleaning0.0000±0.00000.0000 \pm0.00000.5896±0.05270.5896 \pm0.05270.4908±0.02480.4908 \pm0.02482256±222256 \pm22
Complete rules0.9622±0.00390.9622 \pm0.00390.8801±0.02680.8801 \pm0.02680.2121±0.03160.2121 \pm0.03161158±81158 \pm8
Rules + propagation + GAT0.9451±0.01190.9451 \pm0.01190.8150±0.04130.8150 \pm0.04130.2790±0.03280.2790 \pm0.03281185±491185 \pm49
Isolation Forest0.7002±0.09160.7002 \pm0.09160.6589±0.06020.6589 \pm0.06020.4335±0.02570.4335 \pm0.02571350±3941350 \pm394
PCA0.6429±0.02560.6429 \pm0.02560.5740±0.05760.5740 \pm0.05760.4963±0.03040.4963 \pm0.03041466±1461466 \pm146
Matched random0.4761±0.01240.4761 \pm0.01240.2720±0.14680.2720 \pm0.14680.6444±0.05410.6444 \pm0.05411158±81158 \pm8

Table 4.

Table 5: Independent module ablation under 50% mixed contamination (five-seed mean ± SD). Retention is out of approximately 10,000 fitting records; MAE is in eV/atom. Random controls match the retained counts of complete rules (rules) or the configuration with both graph stages (full).

ConfigurationAPF1R2R^2MAERetained
Clean reference––0.6566±0.00880.6566 \pm0.00880.4222±0.01050.4222 \pm0.010510000±010000 \pm0
No cleaning––0.5379±0.00770.5379 \pm0.00770.5154±0.01100.5154 \pm0.011010000±010000 \pm0
Statistical0.8010±0.00890.8010 \pm0.00890.8687±0.00640.8687 \pm0.00640.5682±0.01550.5682 \pm0.01550.4791±0.01490.4791 \pm0.01493638±853638 \pm85
Boundary0.8534±0.00730.8534 \pm0.00730.8695±0.00640.8695 \pm0.00640.5785±0.01780.5785 \pm0.01780.4714±0.01490.4714 \pm0.01493563±813563 \pm81
Complete rules0.9427±0.00370.9427 \pm0.00370.9705±0.00190.9705 \pm0.00190.6160±0.01400.6160 \pm0.01400.4495±0.01320.4495 \pm0.01324696±214696 \pm21
Rules + propagation0.9024±0.00540.9024 \pm0.00540.7715±0.27760.7715 \pm0.27760.5828±0.04330.5828 \pm0.04330.4760±0.03130.4760 \pm0.03136039±18176039 \pm1817
Rules + GAT0.9516±0.00390.9516 \pm0.00390.9653±0.00550.9653 \pm0.00550.6137±0.01450.6137 \pm0.01450.4508±0.01360.4508 \pm0.01364755±844755 \pm84
Rules + propagation + GAT0.9530±0.00410.9530 \pm0.00410.9634±0.00170.9634 \pm0.00170.6107±0.01430.6107 \pm0.01430.4521±0.01350.4521 \pm0.01354707±724707 \pm72
Isolation Forest0.8687±0.02330.8687 \pm0.02330.7731±0.01940.7731 \pm0.01940.5403±0.01980.5403 \pm0.01980.5028±0.01800.5028 \pm0.01805264±9175264 \pm917
PCA0.8144±0.00420.8144 \pm0.00420.7177±0.01340.7177 \pm0.01340.5067±0.01830.5067 \pm0.01830.5260±0.01410.5260 \pm0.01414748±5594748 \pm559
Matched random (rules)––0.4773±0.01220.4773 \pm0.01220.5490±0.01330.5490 \pm0.01334696±214696 \pm21
Matched random (full)––0.4749±0.01360.4749 \pm0.01360.5492±0.01130.5492 \pm0.01134707±724707 \pm72

Table 5.

Note. F1 is the harmonic mean of anomaly-detection precision and recall. R2R^2 is the coefficient of determination on clean test structures; MAE is the mean absolute prediction error in eV/atom. Retained is the number of fitting structures kept after screening and the shared structural check. Values are mean ± sample standard deviation over five seeds; – indicates a metric not reported for that condition.

On the shared eligible fitting set of 2256 ± 22 structures, clean-reference MACE achieves R2=0.9387±0.0158R^2 = 0.9387 \pm0.0158, compared with 0.5896 ± 0.0527 without cleaning. Rule calibration retains 1158 ± 8 structures and recovers R2R^2 to 0.8801 ± 0.0268. Isolation Forest and PCA obtain 0.6589 ± 0.0602 and 0.5740 ± 0.0576, respectively. The rule-versus-no-cleaning paired gain is 0.2906, with a descriptive 95% interval [0.2098, 0.3714]. The full graph configuration yields 0.8150 ± 0.0413; its difference from rules is −0.0652 [−0.1321, 0.0018]. Rule-based screening therefore recovers a substantial part of the prediction performance lost to corruption under the shared ten-epoch budget. Test sets contain 990–994 eligible clean crystals per seed.

Note. AP is average precision for anomaly ranking; F1 measures screening at the calibration-selected threshold. R2R^2 and MAE describe HGBR predictions on the clean test set. Retained counts fitting records kept for training. Values are mean ± sample standard deviation over five seeds; – indicates a metric not reported for that condition.

The HGBR results show that contamination reduces clean-reference R2R^2 from 0.6566 ± 0.0088 to 0.5379 ± 0.0077. Statistical and boundary calibration recover part of the loss, while complete rules achieve 0.6160 ± 0.0140. The paired gain over no cleaning is 0.0781 [0.0683, 0.0880]. Matched-count random retention yields markedly lower R2R^2, showing that evidence-based sample selection drives the recovery.

The full graph configuration achieves the highest AP, increasing it from 0.9427 ± 0.0037 for complete rules to 0.9530 ± 0.0041. Among the screening configurations, complete rules achieve the highest F1 (0.9705 ± 0.0019) and downstream R2R^2 (0.6160 ± 0.0140), compared with 0.9634 ± 0.0017 and 0.6107 ± 0.0143 for the full graph configuration. The paired R2R^2 difference is −0.0053 [−0.0121, 0.0016]. Propagation alone gives AP 0.9024 ± 0.0054 and more variable retained counts. Local attention therefore improves anomaly ranking in this benchmark, while rule calibration gives the highest mean F1 and test R2R^2 among the screening configurations.

Figure 3. Repeated ablation under mixed contamination. Rules, bipartite propagation and local GAT are evaluated separately for anomaly ranking and downstream prediction.

Contamination robustness and parameter sensitivity

Table 6 compares screening F1 at three contamination fractions and three mixed-input perturbation scales. Only the mixed-descriptor perturbation changes; the other corruption mechanisms and target noise remain fixed. F1 is higher at larger contamination fractions for both configurations and changes little across the tested perturbation scales.

ConfigurationContamination0.1σ0.1\sigma0.3σ0.3\sigma0.6σ0.6\sigma
Rules10%0.782 ± 0.0100.782 ± 0.0100.782 ± 0.010
Rules30%0.931 ± 0.0020.931 ± 0.0020.931 ± 0.002
Rules50%0.970 ± 0.0020.971 ± 0.0020.970 ± 0.002
Full graph10%0.733 ± 0.0160.728 ± 0.0210.728 ± 0.021
Full graph30%0.905 ± 0.0300.911 ± 0.0180.910 ± 0.019
Full graph50%0.962 ± 0.0010.963 ± 0.0020.963 ± 0.002

Table 6. Screening F1 across contamination fractions and mixed-descriptor scales (five-seed mean ± SD).

Note. Contamination is the fraction of records with injected errors. Column headings give the mixed-descriptor perturbation scale relative to the descriptor standard deviation σ\sigma. Entries are F1 scores (higher is better), reported as mean ± sample standard deviation over five seeds. Full graph uses both propagation and GAT.

Table 7 reports recall separately for the four corruption mechanisms. Complete rules detect almost all injected records at the calibration-selected thresholds. The F1 results in Table 5 also account for clean records incorrectly removed during screening.

MechanismStatisticalRulesRules + propagationFull graph
Statistical outlier0.9992 ± 0.00061.0000 ± 0.00000.6373 ± 0.50060.9685 ± 0.0268
Boundary violation0.9979 ± 0.00181.0000 ± 0.00000.9832 ± 0.03710.9990 ± 0.0010
Representation mismatch0.9803 ± 0.00321.0000 ± 0.00000.6000 ± 0.54770.9995 ± 0.0007
Mixed perturbation0.9706 ± 0.00680.9998 ± 0.00040.7053 ± 0.40730.9997 ± 0.0004

Table 7. Recall by injected mechanism under internal calibration (mean ± SD; approximately 1,250 contaminated fitting records per mechanism per seed).

Note. Entries are recall: the fraction of injected records of each mechanism identified as anomalous. Statistical uses distribution checks only; Rules uses the complete rule set; Full graph adds propagation and GAT. Values are mean ± sample standard deviation over five seeds.

Appendix § 10.§ 2 reports the score distributions at each stage. The main comparison uses AP, calibration-selected F1 and downstream prediction to assess ranking and screening performance.

Table 8 varies the severe and mild penalties for the propagation configuration. AP ranges from 0.8991 to 0.9052. F1 and downstream R2R^2 vary more because they also depend on applying the calibration threshold to a separately scored fitting batch. Appendix § 10.7 reports the corresponding tests with equal feature weights and changes to individual weights.

Severe/mildAPF1R2R^2
s0.2/m0.60.9052±0.00520.9052 \pm0.00520.7423±0.32560.7423 \pm0.32560.5794±0.04910.5794 \pm0.0491
s0.2/m0.70.9046±0.00520.9046 \pm0.00520.7619±0.27660.7619 \pm0.27660.5803±0.03850.5803 \pm0.0385
s0.2/m0.80.9038±0.00530.9038 \pm0.00530.7903±0.22200.7903 \pm0.22200.5761±0.03670.5761 \pm0.0367
s0.3/m0.60.9034±0.00530.9034 \pm0.00530.7498±0.33920.7498 \pm0.33920.5855±0.04380.5855 \pm0.0438
s0.3/m0.70.9024±0.00540.9024 \pm0.00540.7715±0.27760.7715 \pm0.27760.5828±0.04330.5828 \pm0.0433
s0.3/m0.80.9014±0.00550.9014 \pm0.00550.7960±0.21790.7960 \pm0.21790.5781±0.03510.5781 \pm0.0351
s0.4/m0.60.9010±0.00560.9010 \pm0.00560.7662±0.42650.7662 \pm0.42650.5939±0.03880.5939 \pm0.0388
s0.4/m0.70.9000±0.00550.9000 \pm0.00550.8318±0.27970.8318 \pm0.27970.5970±0.03530.5970 \pm0.0353
s0.4/m0.80.8991±0.00560.8991 \pm0.00560.8164±0.21110.8164 \pm0.21110.5828±0.04170.5828 \pm0.0417

Table 8. Penalty sensitivity of the static-propagation configuration with independent threshold calibration (five-seed mean ±\pm SD).

Note. Severe/mild gives the cell penalty coefficients: for example, s0.2/m0.6 means 0.2 for severe evidence and 0.6 for mild evidence. AP is average precision; F1 measures threshold-based screening; R2R^2 evaluates HGBR prediction on clean test data. Values are mean ±\pm sample standard deviation over five seeds.

Interpretable materials data governance case study

To enhance interpretability in a realistic materials data governance scenario, this study conducts a case analysis on alloy data collected from public literature [41]. A curated structured analysis table supports rule aggregation and consistency checking [42] [43]. Record-level auditing identifies missing FE in 68 records and missing UTS in 12 records of the 185-record table. These are field-specific counts and can overlap within the same record. The processing-history fields are absent from this table. Two FE values exceed the configured screening threshold. Separately, 15 composition/temperature group–target combinations meet the property-spread criteria (Appendix § 10.9); these group-level events are distinct from the record-level missingness counts. Table 9 retains representative evidence patterns and distinguishes their interpretation.

Evidence typeRepresentative sampleMain evidenceInterpretation
Unusual recorded valueTi67_{67}Al12_{12}V7_7Cr7_7Nb7_7 (750 °C, FE = 130%)Recorded FE = 130% triggers the configured value-range check.Source/context check required
Property spread with missing contextCo19.60_{19.60}Cr19.60_{19.60}Fe19.60_{19.60}Ni19.60_{19.60}Mn19.60_{19.60}Ti2.00_{2.00} (25 °C)For the same recorded composition and temperature, UTS ranges from 630 to 2220 MPa and FE from 3% to 85%. Processing and testing details are needed to interpret this spread.Source/context check required
Missing field with property spreadAl13.33_{13.33}Co13.33_{13.33}Cr6.67_{6.67}Fe33.33_{33.33}Ni33.33_{33.33} (25 °C)At matched recorded composition and testing temperature, UTS ranges from 766 to 1013 MPa, while some corresponding FE values are missing, providing directly observable missingness and a property difference requiring additional context.Missingness observed; context check required
Statistical discrepancy with fitting evidenceOriginal literature No. 1 (UTS = 1072 MPa, FE = 55%)The reported FE differs from its neighborhood prediction. KNN predicts FE ≈\approx 11.85%, while Random Forest also suggests a clear deviation.Model-discrepancy candidate

Table 9. Representative review candidates and evidence in the public-literature alloy dataset.

Note. UTS is ultimate tensile strength (MPa); FE is fracture elongation (%). Evidence identifies the observed value, missing field or prediction discrepancy; Interpretation states the follow-up review suggested by that evidence.

Engineering elongation has no universal upper bound of 100%, and its interpretation depends on measurement definition and gauge length. Likewise, processing can alter properties at the same nominal composition and testing temperature. These two candidate types therefore require source and experimental-context checks; fitting residuals help prioritize that review.

For example, in the original literature reference table, sample No.1 reports FE = 55, whereas the KNN-predicted FE is approximately 11.85. Sample No.108 reports UTS = 710 MPa, whereas the KNN-predicted UTS is approximately 1088.69 MPa. These differences direct attention to records whose source values or processing conditions warrant inspection.

Historical-reference batch comparison

The historical experiment tests whether batch scores follow the known contamination order. Table 10 compares within-batch anchors, historical KNN and trained Memory-GAT over five seeds. All three mean scores decrease with contamination, expressing a consistent average quality trend on their respective score scales.

ContaminationWithin-batch anchorHistorical KNNMemory-GAT
0%0.8637 ± 0.00540.8410 ± 0.00300.5912 ± 0.0663
10%0.8079 ± 0.00400.8266 ± 0.00340.5872 ± 0.0635
30%0.7294 ± 0.00220.7912 ± 0.00260.5776 ± 0.0583
50%0.6515 ± 0.00240.7581 ± 0.00200.5657 ± 0.0520

Table 10. Controlled cross-batch historical comparison (five-seed mean ± SD).

Note. Contamination is the injected-error fraction in the current batch. Within-batch anchor is the mean rule/propagation score; Historical KNN averages historical reference scores by inverse-distance weights; Memory-GAT reports QDQ_D. Larger values indicate higher assessed quality within each scoring method. Values are mean ± sample standard deviation over five seeds.

The within-batch and historical KNN scores preserve the four-batch ordering in all five runs, with mean Spearman correlation 1.00. Memory-GAT preserves the order in four runs and obtains mean correlation 0.96. Historical KNN averages reference scores using descriptor distances, while Memory-GAT learns scores from the reference nodes and their graph connections. The mean scores of both historical methods track the increasing contamination levels.

Computational cost

Dense reliability storage and each bipartite propagation iteration cost O(nd)O(nd); exact KNN construction has O(n2d)O(n^{2}d) distance cost. A GAT layer requires O(nFH+EH)O(nFH+EH) operations for input/hidden widths FF, HH and EE edges. Measured with four CPU threads, the 15,999-record workflow takes 7.480±0.176 s and peaks at 699.1±0.2 MiB process RSS; this includes both local GAT comparisons. The 800-anchor Memory-GAT call for 900 current samples takes 0.216±0.002 s with peak process RSS 390.1±0.3 MiB, excluding upstream scoring and anchor selection. Appendix § 10.8 reports scaling and environment details.

Scope of the evaluation. The materials benchmark uses controlled input and target errors, with synthetic calibration labels for threshold selection. MACE results compare methods under the same ten-epoch budget, and the historical experiment compares disjoint batches from the same materials source. These settings define the conditions represented by the reported results.

Conclusion

This work presents a confidence-based framework for materials data quality assessment that integrates domain knowledge, statistical evidence, and graph-structured optimization. Rather than treating data quality as a binary anomaly identification problem, the proposed framework formulates quality evaluation as a continuous confidence-modeling problem and establishes an interpretable workflow consisting of nine-dimensional prior construction, knowledge-guided calibration, heterogeneous graph-based confidence propagation, local refinement, and historical-reference batch comparison.

Experiments show that the framework supports both generic anomaly ranking and materials-specific quality assessment. With materials rules disabled, it achieves an average AUC-ROC of 0.866 on 23 ODDS datasets. In the materials data cleaning experiment, the identification and removal of low-quality records lead to substantial recovery of downstream model performance, as validated by both conventional machine learning prediction and MACE-based downstream modeling, with rule-based screening increasing mean MACE R2R^{2} from 0.590 ± 0.053 to 0.880 ± 0.027 in the repeated subset study. Graph refinement improves mean anomaly-ranking AP from 0.943 ± 0.004 to 0.953 ± 0.004, while rule calibration gives the highest mean screening F1 and test R2R^{2} among the evaluated screening configurations. Historical reference scoring tracks batch-quality changes as contamination increases. Together, these results connect continuous confidence, inspectable evidence and task-dependent choice of scoring configuration within one assessment workflow.

Future work will extend graph construction and reference retrieval through approximate search, expand rule coverage using knowledge graphs and literature mining [46], and evaluate historical quality anchors across additional materials sources.

Code and data availability

The code and data supporting this study is available at https://github.com/materialsCnicCas/materials-data-qua

Acknowledgements

This work was supported by the National Key Research and Development Program of China (Grant No. 2025YFE0102600) and the Strategic Priority Research Program of Chinese Academy of Sciences (Grant No.XDA0410404).

Declarations

The authors declare no competing interests.

Supplementary methods and benchmark results

Complete data quality assessment workflow

Algorithm A.1: End-to-end confidence-based materials data quality assessment.

Input: dataset XX; metadata MM; feature-quality weights vv; rule sets Ru\mathcal{R}_u, Rm\mathcal{R}_m, and Rg\mathcal{R}_g; coefficients α\alpha and λ\lambda; historical quality memory H\mathcal{H}.

Output: sample confidence CfinalC_{\mathrm{final}}; feature confidence ZgraphZ_{\mathrm{graph}}; rule-trigger evidence E\mathcal{E}; memory scores mm; dataset-level score QDQ_D.

  1. Compute feature indicators qj,kq_{j,k} and initialize Zinit,j=∑kvkqj,kZ_{\mathrm{init},j} = \sum_k v_kq_{j,k}; set Cinit,i=0.2C_{\mathrm{init},i} = 0.2 for exact duplicate-group members and 11 otherwise.

  2. Generate univariate boundary, multivariate relationship, and global distribution evidence: Eu=Ru(X)\mathcal{E}_u = \mathcal{R}_u(X), Em=Rm(X)\mathcal{E}_m = \mathcal{R}_m(X), and Eg=Rg(X)\mathcal{E}_g = \mathcal{R}_g(X).

  3. Combine the evidence as E=Eu∪Em∪Eg\mathcal{E} = \mathcal{E}_u \cup\mathcal{E}_m \cup\mathcal{E}_g, and derive the penalty coefficients SiS_i, TjT_j, and edge-reliability matrix BB.

  4. Calibrate confidence using Ccal,i=Cinit,iSiC_{\mathrm{cal},i} = C_{\mathrm{init},i}S_i and Zcal,j=Zinit,jTjZ_{\mathrm{cal},j} = Z_{\mathrm{init},j}T_j.

  5. Compute DsD_s and DfD_f and construct P=[0Ds−1BDf−1BT0]P = \begin{bmatrix} 0 & D_s^{-1}B \\ D_f^{-1}B^{\mathsf{T}} & 0 \end{bmatrix}; set H(0)=Hcal=[Ccal;Zcal]H^{(0)} = H_{\mathrm{cal}} = [C_{\mathrm{cal}}; Z_{\mathrm{cal}}].

  6. Iterate H(t+1)=αHcal+(1−α)PH(t)H^{(t+1)} = \alpha H_{\mathrm{cal}} + (1-\alpha)PH^{(t)} until convergence.

  7. Split the converged state into CgraphC_{\mathrm{graph}} and ZgraphZ_{\mathrm{graph}}.

  8. Construct Ui=[x~i⊙Zgraph∥Cgraph,i]U_i = [\tilde{x}_i \odot Z_{\mathrm{graph}} \mathbin{\|} C_{\mathrm{graph},i}], build the sample KNN graph, and train GAT against 1−Ccal,i1-C_{\mathrm{cal},i} to obtain rir_i.

  9. Refine sample confidence as Cfinal,i=clip⁡(Cgraph,i(1−λri),0,1)C_{\mathrm{final},i} = \operatorname{clip}(C_{\mathrm{graph},i}(1-\lambda r_i),0,1).

  10. Select frozen anchors covering the confidence-score range from H\mathcal{H} and combine them with the current samples to construct the memory graph.

  11. Apply Memory-GAT to obtain mim_i and compute QD=∣D∣−1∑i∈DmiQ_D = |D|^{-1}\sum_{i\in D}m_i.

  12. return CfinalC_{\mathrm{final}}, ZgraphZ_{\mathrm{graph}}, E\mathcal{E}, mm, and QDQ_D.

Confidence distributions across stages

Table A.1 reports the mean confidence of clean and injected records, together with their difference. These statistics describe how the score distributions change during calibration and graph processing. A mean difference depends on score scale: multiplying all scores by a positive constant changes the difference without changing their ranking. It is therefore considered alongside AP, F1 and downstream prediction, rather than used alone to compare stages. In a separate 12,000-record diagnostic, a fixed cutoff of 0.4 retains all records after propagation. The main screening experiment instead selects a threshold for each scoring configuration on the calibration partition.

ConfigurationCleanInjectedGap
Complete rules0.8878 ± 0.00640.3000 ± 0.00000.5878 ± 0.0064
Rules + propagation0.7709 ± 0.00350.5686 ± 0.00130.2023 ± 0.0023
Rules + GAT0.7477 ± 0.01010.2238 ± 0.00290.5239 ± 0.0121
Full graph0.6483 ± 0.00740.4215 ± 0.00530.2268 ± 0.0116

Table A.1. Confidence for clean and injected groups under the controlled benchmark (five-seed mean ± SD).

Note. Clean and Injected are the mean confidence scores of clean and contaminated records, respectively. Gap is Clean minus Injected on the corresponding score scale. Values are mean ± sample standard deviation over five seeds. Full graph includes both propagation and GAT.

Operational definitions of the feature prior

The nine indicators are calculated for each scoring batch using the weights in § 3.1. In the repeated benchmark, each field name also serves as its metadata description; the declared type is float, and the range is either configured or taken from the batch minimum and maximum. Several indicators are consequently constant: median imputation makes the non-missing fraction one, all retained fields are numerical, and the supplied metadata yield the same documentation scores. The feature uniqueness indicator is set to one because duplicate records are handled separately by the sample prior. The formulas below specify each indicator. For feature jj, mjm_j, MjM_j, μj\mu_j, γj\gamma_j and IQRj\mathrm{IQR}_j denote the minimum, maximum, mean, pandas sample skewness and interquartile range. The configured range is [lj,uj][l_j,u_j], and δ=10−9\delta= 10^{-9} prevents division by zero.

Name / interpretationFormula in this reference experimentOperational meaning
provenance / documentation availabilitymin⁡(1,0.4I[description]+0.3I[range]+0.3I[type])\min(1,0.4I[\mathrm{description}] + 0.3I[\mathrm{range}] + 0.3I[\mathrm{type}])Documentation completeness from available metadata.
Name / interpretationFormulaOperational meaning
dataset_characteristics / range–center agreementIf an endpoint lies outside the configured range, R=max⁡(0,1−(∣m−l∣+∣M−u∣)/(u−l+δ))R = \max(0, 1 - (|m-l| + |M-u|)/(u-l+\delta)); otherwise R=1R=1. Set C=max⁡(0,1−∣μ−(l+u)/2∣/(u−l+δ))C=\max(0,1-|\mu-(l+u)/2|/(u-l+\delta)) and q=(R+C)/2q=(R+C)/2.Measures range agreement and distance from the configured range center.
uniformity / numeric compatibilityFraction of nonmissing observations convertible to a number; zero for an empty field.Numerical conversion rate; constant for the numerical fields used here.
metadata_coupling / lexical matchdifflib.SequenceMatcher ratio between lowercased field name and description.Lexical agreement proxy for field documentation.
non_missing_cells / observed-cell fractionFraction nonmissing at scoring time.Adapted completeness; equals one after imputation here.
non_duplicate_rows / duplicate handlingq=1q=1; exact duplicate detection acts through CinitC_{\mathrm{init}} instead.Fixed at one; sample duplicates are handled by the sample prior.
un_skewness / shape regularityWith s=∣γ∣s=|\gamma|: 1−0.2s1-0.2s for s≤0.5s\le0.5; 0.9−0.4(s−0.5)0.9-0.4(s-0.5) for 0.5<s≤10.5<s\le1; 0.5−0.3(s−1)0.5-0.3(s-1) for 1<s≤21<s\le2; max⁡(0,0.2−0.1(s−2))\max(0,0.2-0.1(s-2)) otherwise.Piecewise shape score, with a discontinuity at s=1s=1 in the implementation.
accuracy / numeric regularityStart at one. Multiply by max⁡(0.5,1−1.5(o−0.10))\max(0.5,1-1.5(o-0.10)) only when Tukey-outlier fraction o>0.10o>0.10 and IQR >0>0. Multiply by 0.8 if the standard deviation of decimal digit counts exceeds three, or 0.9 if it exceeds two.Numerical regularity proxy using precision and outlier frequency.
target_correlation / principal-variation relevancemin⁡(1,max⁡(0.2,1.2∣corr⁡(xj,PC1)∣))\min(1,\max(0.2,1.2|\operatorname{corr}(x_j,\mathrm{PC1})|)) using unstandardized numerical PCA; 0.5 for undefined correlation.Descriptor-only relevance to the principal variation of the batch.

Table A.2. Feature indicators: implementation names and operational definitions.

All nine feature scores are saved for each seed using these fixed operational formulas.

Convergence and implementation verification

All entries of the static BB are positive, hence the separate degree matrices have nonzero diagonals and the block transition PP is row-stochastic. For F(H)=αHcal+(1−α)PHF(H)=\alpha H_{\mathrm{cal}}+(1-\alpha)PH,

∥F(H)−F(G)∥∞≤(1−α)∥H−G∥∞.(9)\lVert F(H)-F(G)\rVert_{\infty}\le(1-\alpha)\lVert H-G\rVert_{\infty}. \tag*{(9)}

For 0<α≤10<\alpha\le1, the map has the unique fixed point H∗=α[I−(1−α)P]−1HcalH^*=\alpha[I-(1-\alpha)P]^{-1}H_{\mathrm{cal}} and converges geometrically. Every update is a convex combination, preserving [0,1][0,1]. An independent sparse solution of a small system agrees within 2.6×10−92.6\times10^{-9}; an independently assembled 12,000-row sparse transition agrees within 1.1×10−131.1\times10^{-13}.

Memory-GAT inputs and training

Memory-GAT uses 18 descriptors: band gap, energy above hull, space group, three lattice lengths, three angles and nine lattice-matrix entries. Missing values and standardization are handled using medians, means and standard deviations fitted on the historical data. The historical input is [standardize⁡H(x),2qH−1][\operatorname{standardize}_{\mathcal{H}}(x),2q_{\mathcal{H}}-1], where qHq_{\mathcal{H}} is the stored reference confidence. The current input is [standardize⁡H(x),2a−1][\operatorname{standardize}_{\mathcal{H}}(x),2a-1], where aa is the current rule/propagation score. These 19-dimensional vectors are also used to retrieve Euclidean nearest neighbors. The encoder uses hidden width 24, two first-layer attention heads and a sigmoid MLP output. Each current node receives messages from eight historical neighbors, six current neighbors and itself; historical nodes receive only self-loop messages. The loss is

Lmemory=4∣H∣∑u∈H(q^u−qu)2+1∣C∣∑i∈C(q^i−ai)2+0.1∣E∣∑(u,v)∈E(q^u−q^v)2.(10)\mathcal{L}_{\mathrm{memory}}=\frac{4}{|\mathcal{H}|}\sum_{u\in\mathcal{H}}(\widehat{q}_u-q_u)^2+\frac{1}{|\mathcal{C}|}\sum_{i\in\mathcal{C}}(\widehat{q}_i-a_i)^2 +\frac{0.1}{|\mathcal{E}|}\sum_{(u,v)\in\mathcal{E}}(\widehat{q}_u-\widehat{q}_v)^2. \tag*{(10)}

The first two terms fit historical and current reference scores; the third penalizes differences between connected nodes. Adam uses learning rate 10−310^{-3}, weight decay 10−510^{-5}, dropout 0.05 and gradient clipping at one. Training runs for at most 30 epochs and stops after eight epochs without a lower training loss; the parameters with the lowest loss are restored. Historical KNN uses eight historical neighbors with inverse-distance weights.

Controlled corruption and MACE settings

Within each corrupted batch, 50% of rows are sampled and divided into four equal groups. Statistical corruption perturbs up to eight generic descriptors by independent Gaussian offsets at 4σ4\sigma; boundary corruption sets up to five configured fields 35% of their range below or above the endpoints. Relationship corruption replaces lattice lengths with 2 or 30 Å according to their stored row norm. Mixed corruption perturbs up to 12 descriptors at the declared scale, shifts the first eight descriptors by 1.8σ1.8\sigma, may set a negative band gap, and modifies one lattice length. Every selected row receives target noise at 2.8σy2.8\sigma_y. The same recipe is applied independently to fitting and calibration batches. Corruption labels are used for metrics and calibration-threshold selection, not scorer inputs or training targets.

For MACE, corrupted numerical lattice and coordinate fields are exported as actual ASE structures, and energy per atom is multiplied by atom count. All conditions use the same structural filter, which requires finite coordinates, absolute cell determinant at least 11 Å3^3, condition number at most 10410^4, minimum periodic pair distance at least 1 Å, and at most 120 periodic neighbors within 4 Å. A fitting record is eligible only if both its original and corrupted structures pass these checks. Validation and test records are checked using their clean structures. Random retention is matched to the screened sample count after structural filtering.

The MACE 0.3.15 environment uses Python 3.10.20, PyTorch 2.11.0 with CUDA 12.8 and NVIDIA RTX 5090 GPUs. Additional settings are max_L=1, atomic-number vocabulary 1–94, validation batch size 64, Adam with AMSGrad, gradient clipping 10 and ReduceLROnPlateau (factor 0.5, patience 10). Energy loss weight is one and force loss weight zero; stage two is disabled. Clean validation selects the exported checkpoint, with no test file passed to training. All 35 runs are evaluated on common clean test records within each seed. Every condition uses the same ten-epoch training budget.

Additional sensitivity results

Table A.3 reports the propagation configuration with thresholds selected on the calibration partition. Each indicator weight is multiplied by 0, 0.8 or 1.2, after which all weights are normalized to sum to one. Thus, x0.0 removes an indicator, and x0.8/x1.2 reduce/increase its weight by 20% before normalization. The uniform setting gives all nine indicators equal weight. All other parameters remain fixed. Changing a constant indicator can still change the overall score scale through weight normalization.

Table A.3: Complete feature-weight sensitivity results. Setting identifies the indicator and its weight multiplier (x0.0 removes it); uniform gives equal weights. AP is average precision, F1 measures screening, and R2R^2 evaluates HGBR prediction on clean test data. Values are mean ±\pm sample standard deviation over five seeds.

SettingAPF1R2R^2
accuracy_x0.00.9032 ± 0.00560.7851 ± 0.26890.5857 ± 0.0425
accuracy_x0.80.9025 ± 0.00540.7726 ± 0.27660.5828 ± 0.0430
accuracy_x1.20.9023 ± 0.00530.7683 ± 0.28070.5858 ± 0.0371
dataset_characteristics_x0.00.9067 ± 0.00490.6881 ± 0.26920.5656 ± 0.0392
dataset_characteristics_x0.80.9040 ± 0.00520.7657 ± 0.28160.5828 ± 0.0414
dataset_characteristics_x1.20.9009 ± 0.00550.7837 ± 0.27260.5866 ± 0.0427
default0.9024 ± 0.00540.7715 ± 0.27760.5828 ± 0.0433
metadata_coupling_x0.00.9003 ± 0.00560.7827 ± 0.27140.5849 ± 0.0402
metadata_coupling_x0.80.9020 ± 0.00540.7720 ± 0.27760.5836 ± 0.0434
metadata_coupling_x1.20.9028 ± 0.00530.7688 ± 0.27980.5826 ± 0.0425
non_duplicate_rows_x0.00.9011 ± 0.00550.7813 ± 0.27150.5835 ± 0.0402
non_duplicate_rows_x0.80.9021 ± 0.00540.7716 ± 0.27770.5851 ± 0.0413
non_duplicate_rows_x1.20.9027 ± 0.00540.7713 ± 0.27770.5823 ± 0.0429
non_missing_cells_x0.00.8980 ± 0.00580.8380 ± 0.26570.5976 ± 0.0357
non_missing_cells_x0.80.9014 ± 0.00550.7811 ± 0.27130.5839 ± 0.0376
non_missing_cells_x1.20.9034 ± 0.00530.7675 ± 0.28090.5827 ± 0.0420
provenance_x0.00.9011 ± 0.00550.7813 ± 0.27150.5835 ± 0.0402
provenance_x0.80.9021 ± 0.00540.7716 ± 0.27770.5851 ± 0.0413
provenance_x1.20.9027 ± 0.00540.7713 ± 0.27770.5823 ± 0.0429
target_correlation_x0.00.9038 ± 0.00520.8434 ± 0.25500.5977 ± 0.0330
target_correlation_x0.80.9027 ± 0.00530.7797 ± 0.27030.5848 ± 0.0414
target_correlation_x1.20.9021 ± 0.00550.7686 ± 0.28250.5817 ± 0.0422
un_skewness_x0.00.9052 ± 0.00510.7435 ± 0.32760.5803 ± 0.0496
SettingAPF1R2R^2
un_skewness_x0.80.9031 ± 0.00540.7649 ± 0.28700.5821 ± 0.0414
un_skewness_x1.20.9017 ± 0.00540.7817 ± 0.26850.5834 ± 0.0415
uniform0.9047 ± 0.00510.7615 ± 0.27860.5807 ± 0.0396
uniformity_x0.00.8988 ± 0.00560.7900 ± 0.26890.5863 ± 0.0443
uniformity_x0.80.9016 ± 0.00550.7810 ± 0.27130.5835 ± 0.0394
uniformity_x1.20.9032 ± 0.00530.7679 ± 0.28050.5823 ± 0.0418

Table A.3.

Computational scaling

Local timing uses Python 3.9.7, PyTorch 2.7.1 CPU and scikit-learn 1.6.1 on an AMD Ryzen 5 9500F, with four threads and three fresh-process runs per scale. Total scoring includes both 50-epoch local GAT comparisons and supporting steps, but not reading or corruption generation. Peak resident set size (RSS) measures the memory of the entire process, including the Python runtime and data preparation. The static iteration converges in 38–39 iterations.

RecordsFieldsScoring (s)Peak RSS (MiB)
99943–460.787 ± 0.022457.9 ± 27.7
3,999431.968 ± 0.004468.0 ± 12.0
8,000433.684 ± 0.026529.5 ± 0.7
15,999437.480 ± 0.176699.1 ± 0.2

Table A.4. Local scoring cost (mean ± SD over three fresh processes).

Note. Records and Fields give the numbers of samples and numerical descriptors. Scoring is elapsed scoring time in seconds. Peak RSS is the maximum resident set size of the entire process, in mebibytes (MiB). Values are mean ± sample standard deviation over three independent process runs.

Historical scoring uses 900 current records and 200/400/800 anchors. Mean scorer-call times are 0.205/0.210/0.216 s, with process RSS 388.4/389.0/390.1 MiB. These include descriptor preparation, retrieval, attention training and inference, excluding upstream rule scoring and anchor selection. Dense propagation uses O(nd)O(nd) storage and per-iteration arithmetic. Exact sample KNN uses O(n2d)O(n^2d) arithmetic; historical/current retrieval has worst-case O(NM(d+1))/O(N2(d+1))O(NM(d+1))/O(N^2(d+1)) distance cost even when KD-tree pruning is used. The timing measurements use exact retrieval on one machine.

Alloy audit criteria

Missingness counts are computed from the original supplied workbooks before imputation. The 185-record table lacks all 12 processing-history columns listed in Table A.5; the 315-record source table contains them. Candidate groups use the 14 element fractions rounded to four decimals and, in the 185-record table, recorded testing temperature. Missing/nonnumeric fractions are replaced by zero for group-key construction. Each property is examined separately within a group. With at least two observations, the group is flagged for that property when UTS spread is at least 100 MPa and relative spread at least 0.15, or FE spread is at least five percentage points and relative spread at least 0.20. Relative spread is the maximum minus the minimum, divided by the absolute median with a lower bound of 10−610^{-6} on the denominator. These flags prioritize source review using the recorded composition, temperature and property fields.

Field definitions for the alloy case-study datasets

The two alloy tables differ primarily in their representation of experimental conditions. The 185-record case-study table retains a single testing-temperature field, whereas the 315-record source-literature table provides a more detailed processing history. Both tables contain the same elemental-composition variables, mechanical-property variables, and 32 composition-derived material descriptors.

AvailabilityCategoryFieldsDefinition
BothSample identifierAlloy_Name (185-record table); No. (315-record table)Alloy-composition label or numerical record identifier used to distinguish individual observations.
AvailabilityCategoryFieldsDefinition
BothElemental compositionCo, Cr, Fe, Mn, Ni, Al, Ti, V, Cu, W, Mo, Zr, Nb, TaContents of the 14 constituent elements. The values describe the nominal elemental fractions recorded for each alloy, typically in atomic percent.
185 onlyTesting conditiontemperatureTemperature at which the reported UTS and FE values were measured.
315 onlyHomogenizationHom-T, Hom-tHomogenization temperature and holding time, respectively.
315 onlyDeformationDef-T, Def-rDeformation temperature and deformation or reduction ratio, respectively.
315 onlyFirst annealing stageAnnI-T, AnnI-tTemperature and holding time of the first annealing treatment.
315 onlyRollingRol-T, Rol-rateRolling temperature and rolling-reduction rate, respectively.
315 onlySecond annealing stageAnnII-T, AnnII-tTemperature and holding time of the second annealing treatment.
315 onlyAgingAge-T, Age-tAging temperature and aging duration, respectively.
BothMechanical propertiesUTS, FEUltimate tensile strength and fracture elongation, which characterize alloy strength and ductility, respectively.
BothAtomic and periodic descriptorsZ, Per, Gro, M, VECComposition-weighted atomic number, periodic-table period, group number, atomic mass, and valence-electron concentration.
BothElectronic and chemical descriptorsAff, Ion, χ\chiComposition-weighted electron affinity, first ionization energy, and electronegativity.
BothBasic physical and thermal descriptorsρ\rho, Tm, Tb, k/κ\kappa, CmComposition-weighted density, melting point, boiling point, thermal conductivity, and molar heat capacity. The thermal-conductivity field is named k in the 185-record table and κ\kappa in the 315-record table.
BothPhase-change thermodynamic descriptorsΔHv\Delta H_{\mathrm{v}}, ΔHf\Delta H_{\mathrm{f}}, ΔSf\Delta S_{\mathrm{f}}Composition-weighted enthalpy of vaporization, enthalpy of fusion, and entropy of fusion.
BothElectronic, transport, and structural descriptorsED, D, a, RElectron-density descriptor, diffusion-related coefficient, lattice parameter, and atomic-radius descriptor supplied by the source dataset.
BothElastic descriptorsE, K, G, C11, C12, C44Composition-weighted Young’s modulus, bulk modulus, shear modulus, and the three independent elastic stiffness constants commonly used for cubic materials.
BothVibrational descriptorTdComposition-weighted Debye temperature, characterizing the characteristic lattice-vibration scale.
BothAlloy-level mismatch and thermodynamic descriptorsΔR\Delta R, Δχ\Delta\chi, ΔSmix\Delta S_{\mathrm{mix}}, ΔHmix\Delta H_{\mathrm{mix}}, Ω\OmegaAtomic-radius mismatch, electronegativity mismatch, configurational entropy of mixing, enthalpy of mixing, and the thermodynamic Ω\Omega parameter combining melting temperature, mixing entropy, and mixing enthalpy.

Table A.5. Field categories and definitions for the 185-record case-study table and the 315-record source-literature reference table.

Note: The field names and grouping follow the uploaded source tables. The units and numerical conventions are retained from the corresponding literature data. The physicochemical descriptors are primarily composition-weighted elemental properties, whereas ΔR\Delta R, Δχ\Delta\chi, ΔSmix\Delta S_{\mathrm{mix}}, ΔHmix\Delta H_{\mathrm{mix}}, and Ω\Omega are alloy-level quantities derived from the overall composition.

Detailed ODDS benchmark results

DatasetClassical methodsDeep-learning-based methodsAnoLLMOur method
IForestPCAKNNECODDeepSVDDRCASLADGOADNeuTraLICLDTEREPEN135M360M
Annthyroid0.9220.8390.8110.7900.7420.7180.7610.5720.8130.8420.9770.7360.9270.9310.924
Arrhythmia0.8270.7960.7860.8110.7650.7860.7840.6810.7600.7850.7710.6840.8250.8220.790
BreastW0.9940.9880.9920.9920.9740.9870.9860.9940.9830.9920.9820.9550.9920.9930.993
Cardio0.9480.9660.9210.9350.8420.9480.8400.5240.8590.8940.9200.8290.9400.8730.926
ForestCover0.8700.9450.9850.9210.5330.9440.8570.2780.8980.9770.9780.9020.8810.8350.902
Glass0.8020.7130.8490.6930.8240.7190.7900.5740.9330.8870.7990.7550.8190.7970.760
Ionosphere0.8910.8940.9600.7340.9630.9160.9600.9500.9560.9700.9640.5450.9090.9240.924
LetterRecognition0.6310.5290.8650.5670.7760.7160.9080.8110.9290.9590.8720.5970.9670.8670.766
Lymphography0.6730.8260.8600.8300.8990.9190.9640.8170.8470.8270.9090.8080.9680.9930.953
Mammography0.8810.9000.8720.9060.8570.8730.7400.7560.6900.7820.8640.8630.9150.8760.884
Musk0.9711.0001.0000.9561.0001.0001.0001.0001.0001.0001.0000.7221.0001.0001.000
Optdigits0.8240.5740.9440.6060.7530.8000.7350.8470.9790.9550.8880.6070.9830.9390.687
Pendigits0.9710.9420.9990.9280.8380.9670.9320.2200.9630.9710.9820.9220.9710.9640.969
Pima0.7200.7110.7410.5870.5930.7040.5840.6650.7630.6970.6620.6880.6630.6540.745
Satellite0.8070.6620.8740.5820.8000.7340.8640.7950.8590.8540.7890.7400.9020.8770.832
Satimage-20.9930.9790.9990.9650.9610.9980.9970.9940.8610.9970.9880.9981.0000.9990.995
Shuttle0.9960.9940.9990.9930.9970.9960.9980.9900.9970.9990.9980.9921.0001.0000.993
Speech0.4780.4710.4860.4710.5630.4720.5110.5500.6090.5820.5110.5310.4700.4700.615
Thyroid0.9890.9840.9760.9780.8690.9690.9480.6890.8860.9870.9900.9040.9750.9830.985
Vertebral0.4460.4940.4060.4740.4780.4780.4830.5160.5450.5430.5700.2470.5650.4080.403
Vowels0.7790.6440.9750.5970.8950.8910.9690.9250.9880.9790.9830.7530.9820.9380.946
WBC0.9470.9490.9470.9070.9040.9460.9280.6630.7650.9080.9130.7700.9640.9520.944
Wine0.9300.9270.9520.7290.8540.8990.9640.9610.9600.9440.9740.8350.9090.8510.973
Average0.8390.8140.8780.7810.8120.8430.8480.7290.8630.8840.8820.7560.8920.8670.866
Rank101141312981572314156

Table A.6. Detailed ODDS benchmark comparison in terms of AUC-ROC. Note. All entries are AUC-ROC (area under the receiver operating characteristic curve); higher values indicate better anomaly ranking. IForest, PCA and KNN denote Isolation Forest, principal component analysis and nearest-neighbor detection. AnoLLM headings 135M and 360M identify model sizes in millions of parameters. Other column names identify the baseline models described in the benchmark comparison.

References

  1. [1]Lauri Himanen, Amber Geurts, Adam Stuart Foster, and Patrick Rinke. Data-driven materials science: status, challenges, and perspectives. Advanced Science, 6(21):1900808, 2019.DOI
  2. [2]Anubhav Jain, Shyue Ping Ong, Geoffroy Hautier, Wei Chen, William Davidson Richards, Stephen Dacek, Shreyas Cholia, Dan Gunter, David Skinner, Gerbrand Ceder, et al. Commentary: The materials project: A materials genome approach to accelerating materials innovation. APL materials, 1(1), 2013.DOI
  3. [3]Scott Kirklin, James E Saal, Bryce Meredig, Alex Thompson, Jeff W Doak, Muratahan Aykol, Stephan Rühl, and Chris Wolverton. The open quantum materials database (oqmd): assessing the accuracy of dft formation energies. npj Computational Materials, 1(1):1–15, 2015.DOI
  4. [4]Camilo E Calderon, Jose J Plata, Cormac Toher, Corey Osenes, Ohad Levy, Marco Fornari, Amir Natan, Michael J Mehl, Gus Hart, Marco Buongiorno Nardelli, et al. The aflow standard for high-throughput materials science calculations. Computational Materials Science, 108:233–238, 2015.arxiv.org/abs/1506.00303
  5. [5]Claudia Draxl and Matthias Scheffler. Nomad: The fair concept for big data-driven materials science. Mrs Bulletin, 43(9):676–682, 2018.arxiv.org/abs/1805.05039
  6. [6]Ben Blaizsik, Kyle Chard, Jim Pruyne, Rachana Ananthakrishnan, Steven Tuecke, and Ian Foster. The materials data facility: data services to advance materials science research. Jom, 68(8):2045–2052, 2016.DOI
  7. [7]Quratul Ain, Haider Tawfiq Naeem, Mubashar Ali, Junaid Munir, Zunaira Bibi, Hamid M Ghaithan, Abdullah Ahmed Ali Ahmed, and Saif MH Qaid. A precise prediction of structure stability and hydrogen storage capability of kcdh3 perovskite hydride using density functional theory calculations. Journal of Energy Storage, 100:113734, 2024.DOI
  8. [8]Durgesh Samariya and Amit Thakkar. A comprehensive survey of anomaly detection algorithms. Annals of Data Science, 10(3):829–850, 2023.
  9. [9]André M Carvalho, Sónia Soares, João Montenegro, and Lígia Conceição. Data quality: revisiting dimensions towards new framework development. Procedia Computer Science, 253:247–256, 2025.DOI
  10. [10]Farshad Rahmati, Reza Heydari Gharaei, and Hossein Nezamabadi-pour. Ardod: adaptive radius density-based outlier detection. Evolutionary Intelligence, 17(5):3707–3722, 2024.DOI
  11. [11]Anthony Yu-Tung Wang, Ryan J Murdock, Steven K Kauwe, Anton O Oliynyk, Aleksander Gurlo, Jakoah Brgoch, Kristin A Persson, and Taylor D Sparks. Machine learning for materials scientists: an introductory guide toward best practices. Chemistry of Materials, 32(12):4954–4965, 2020.DOI
  12. [12]Jonathan Schmidt, Mário RG Marques, Silvana Botti, and Miguel AL Marques. Recent advances and applications of machine learning in solid-state materials science. npj computational materials, 5(1):83, 2019.DOI
  13. [13]Aasim Ayaz Wani. Advancing material stability prediction: leveraging machine learning and high-dimensional data for improved accuracy. Materials Sciences and Applications, 16(02):79–105, 2025.
  14. [14]Ruoyu Li, Qing Li, Yu Zhang, Dan Zhao, Yong Jiang, and Yong Yang. Interpreting unsupervised anomaly detection in security via rule extraction. Advances in Neural Information Processing Systems, 36:62224–62243, 2023.DOI
  15. [15]Yue Liu, Shuchang Ma, Zhengwei Yang, Duo Wu, Xiaoming Avdeev, and Siqi Shi. Domain knowledge-assisted materials data anomaly detection towards constructing high-performance machine learning models. Journal of Materiomics, 11(6):101066, 2025.DOI
  16. [16]Guangbo Yu, Xiaoqiang Zhao, and Jixiang Zhao. Cognitive-cycle learning framework for small-sample industrial process quality prediction. Measurement Science and Technology, 37(10):105107, 2026.DOI
  17. [17]Sandeep Rangineni, Amit Bhanushali, Manoj Suryadevara, Srinivas Venkata, and Kiran Peddireddy. A review on enhancing data quality for optimal data analytics performance. International Journal of Computer Sciences and Engineering, 11(10):51–58, 2023.DOI
  18. [18]Maria Priestley, Fionntán O’donnell, and Elena Simperl. A survey of data quality requirements that matter in ml development pipelines. ACM Journal of Data and Information Quality, 15(2):1–39, 2023.DOI
  19. [19]S Kokott, A Marek, F Merz, P Karpov, C Carkovic, M Rossi, M Rampp, V Blum, and M Scheffler. Towards efficient and accurate input for data-driven materials science from large-scale all-electron density functional theory (dft) simulations. Modelling and Simulation in Materials Science and Engineering, 32(6):28–31, 2024.
  20. [20]Ran Wang, Cheng Xu, Runshi Dong, Zhenghui Luo, Rong Zheng, and Xiaotong Zhang. A secured big-data sharing platform for materials genome engineering: State-of-the-art, challenges and architecture. Future Generation Computer Systems, 142:59–74, 2023.DOI
  21. [21]Rees Chang, Yu-Xiong Wang, and Elif Ertekin. Towards overcoming data scarcity in materials science: unifying models and datasets with a mixture of experts framework. npj Computational Materials, 8(1):242, 2022.arxiv.org/abs/2207.13880
  22. [22]Fatih Aydın. Boundary-aware local density-based outlier detection. Information Sciences, 647:119520, 2023.DOI
  23. [23]Braulio V Sánchez Vinces, Erich Schubert, Arthur Zimek, and Robson LF Cordeiro. A comparative evaluation of clustering-based outlier detection. Data Mining and Knowledge Discovery, 39(2):13, 2025.DOI
  24. [24]Chenlong Feng, Chao Liu, and Dongxiang Jiang. Unsupervised anomaly detection using graph neural networks integrated with physical-statistical feature fusion and local-global learning. Renewable Energy, 206:309–323, 2023.DOI
  25. [25]SHI Siqi, SUN Shiyu, MA Shuchang, ZOU Xinxin, QIAN Quan, and LIU Yue. Detection method on data accuracy incorporating materials domain knowledge. Journal of Inorganic Materials, 37:1311–1320, 2022.DOI
  26. [26]Shun Zhu, Xichen Yang, Yan Zhang, Tianshu Wang, Zhongyuan Mao, Tianyin Li, Zhuoyuan Sun, and Xiaobo Shen. Graph-driven domain co-adaptation for cross-domain image quality assessment. In Proceedings of the AAAI Conference on Artificial Intelligence, volume 40, pages 13970–13978, 2026.
  27. [27]Mingji Yang, Hanzhi Wang, Zhewei Wei, Sibo Wang, and Ji-Rong Wen. Efficient algorithms for personalized pagerank computation: A survey. IEEE Transactions on Knowledge and Data Engineering, 36(9):4582–4602, 2024.DOI
  28. [28]Aristidis G Vrahatis, Konstantinos Lazaros, and Sotiris Kotsiantis. Graph attention networks: a comprehensive review of methods and applications. Future Internet, 16(9):318, 2024.DOI
  29. [29]Xiaoxiao Ma, Ruikun Li, Fanzhen Liu, Kaize Ding, Jian Yang, and Jia Wu. Graph anomaly detection with few labels: A data-centric approach. In Proceedings of the 30th ACM SIGKDD Conference on Knowledge Discovery and Data Mining, pages 2153–2164, 2024.DOI
  30. [30]Xuchen Guo. Sample-based error rate assessment through predictive confidence in biomedical applications. PhD thesis, St. Francis Xavier University, 2026.
  31. [31]Theodoros Alexakis, Evgenia Adamopoulou, Nikolaos Peppes, Emmanouil Daskalakis, and Georgios Ntouskas. Evaluating data quality: Comparative insights on standards, methodologies, and modern software tools. Electronics, 14(15):3038, 2025.DOI
  32. [32]WEI Xiao, WANG Xiaoxin, CHEN Yongqi, and ZHANG Huiran. Constructing a material-domain knowledge graph based on natural language processing. Journal of Shanghai University, 28(3):386, 2022.
  33. [33]Chunyu Wei, Wenji Hu, Xingjia Hao, Yunhai Wang, Yueguo Chen, Bing Bai, and Fei Wang. Graph evidential learning for anomaly detection. In Proceedings of the 31st ACM SIGKDD Conference on Knowledge Discovery and Data Mining V. 2, pages 3122–3133, 2025.DOI
  34. [34]Jiaxing Zhang, Xiaoou Liu, Dongsheng Luo, and Hua Wei. Is your explanation reliable: Confidence-aware explanation on graph neural networks. In Proceedings of the 31st ACM SIGKDD Conference on Knowledge Discovery and Data Mining V. 2, pages 3740–3751, 2025.DOI
  35. [35]Meihao Liao, Rong-Hua Li, Qiangqiang Dai, Hongyang Chen, Hongchao Qin, and Guowen Wang. Efficient personalized pagerank computation: The power of variance-reduced monte carlo approaches. Proceedings of the ACM on Management of Data, 1(2):1–26, 2023.DOI
  36. [36]Lin Kaibiao, Jinpo Chen, Chen Ruicong, Yang Fan, Zhang Yang, Lin Min, and Lu Ping. Adaptive neighbor graph aggregated graph attention network for heterogeneous graph embedding. ACM Transactions on Knowledge Discovery from Data, 18(1):1–21, 2023.DOI
  37. [37]Xuan-Thuy Vo and Kang-Hyun Jo. A review on anchor assignment and sampling heuristics in deep learning-based object detection. Neurocomputing, 506:96–116, 2022. ISSN 0925-2312.DOI
  38. [38]Che-Ping Tsai, Ganyu Teng, Phillip Wallis, and Wei Ding. Anollm: Large language models for tabular anomaly detection. In The Thirteenth International Conference on Learning Representations, 2025.DOI
  39. [39]Mohan Raparthi, Dharmesh Dhabliya, Tanu Kumari, Rajkamal Upadhyaya, and Aditi Sharma. Implementation and performance comparison of gradient boosting algorithms for tabular data. Deep Learning and Visual Artificial Intelligence: Proceedings of ICDLAI 2024, page 461, 2024.DOI
  40. [40]Dávid Péter Kovács, Ilyes Batatia, Eszter Sára Arany, and Gábor Csányi. Evaluation of the mace force field architecture: From medicinal chemistry to materials science. The Journal of Chemical Physics, 159(4), 2023.DOI
  41. [41]Kang Xu, Zhengming Sun, Jian Tu, Wenwang Wu, and Huihui Yang. Intelligent design of fe–cr–ni–al/ti multi-principal element alloys based on machine learning. Journal of Materials Research and Technology, 35:6864–6873, 2025.DOI
  42. [42]Bowen Wang, Bingheng Lu, Lijuan Zhang, Jianxun Zhang, Bobo Li, Qianyu Ji, Peng Luo, and Qian Liu. Rapid in situ alloying of cocrfemnni high-entropy alloy from elemental feedstock toward high-throughput synthesis via laser powder bed fusion. Frontiers of Mechanical Engineering, 18(1):11, 2023.DOI
  43. [43]Xinsheng Chen, Yuzheng Liang, Qipeng Wang, Kewei Dong, Yang Yang, Shuai Feng, Xueqi Zhang, Yingchao Cai, Yong Peng, Kehong Wang, et al. Introduction of nanoprecipitation into supersaturated cocrfemnin0. 5 alloy prepared by laser-powder bed fusion via microstructure tailoring to achieve excellent properties. Scripta Materialia, 235:115636, 2023.DOI
  44. [44]Paidipati Dinesh, AS Vickram, and P Kalyanasundaram. Medical image prediction for diagnosis of breast cancer disease comparing the machine learning algorithms: Svm, knn, logistic regression, random forest and decision tree to measure accuracy. In AIP Conference Proceedings, volume 2853, page 020140. AIP Publishing LLC, 2024.
  45. [45]HF Li, JL Wei, SP Yang, and YQ Liu. The relationship between fracture toughness and tensile property in high-strength steels. Engineering Failure Analysis, 131:105854, 2022.DOI
  46. [46]Yang Yuan, Sihan Liang, Ji Zhang, Yangang Wang, and Zongguo Wang. A deep learning framework for scientific chart data extraction and reconstruction. Communications Engineering, 5(1):147, 2026.DOI

Paper details

vlnc:W_nes8bh87
Version
v1 · latest
Published
Fresh — added to the Hub within the last 30 days
Added
License
CC BY 4.0
Usage

views · 4

downloads · 0

A view counts once per browser per day, a download each time the PDF is requested.

  • Computer Science

Not recorded for this paper

View statement
Looking for related papers…
Contents