WorksPhysics and Astronomy
A Foundation Model for Non-Destructive Defect Identification from Vibrational Spectra
Abstract
Defects are ubiquitous in solids and strongly influence materials' mechanical and functional properties. However, non-destructive characterization and quantification of defects, especially when multiple types coexist, remain a long-standing challenge. Here we introduce DefectNet, a foundation machine learning model that predicts the chemical identity and concentration of substitutional point defects with multiple coexisting elements directly from vibrational spectra, specifically phonon density-of-states (PDoS). Trained on over 16,000 simulated spectra from 2,000 semiconductors, DefectNet employs a tailored attention mechanism to identify up to six distinct defect elements at concentrations ranging from 0.2% to 25%. The model generalizes well to unseen crystals across 56 elements and can be fine-tuned on experimental data. Validation using inelastic scattering measurements of SiGe alloys and MgB superconductor demonstrates its accuracy and transferability. Our work establishes vibrational spectroscopy as a viable, non-destructive probe for point defect quantification in bulk materials, and highlights the promise of foundation models in data-driven defect engineering.
Introduction
Defects play a pivotal role in materials science, profoundly influencing physical properties and ultimately determining device performance across a wide range of technological applications[1]. The atomic-scale defects are central to the functionality of materials, including solar cells[2], thermoelectrics[3, 4], electrode materials in batteries[5], catalysts6, and quantum materials7–9. Historically, the study of defects in semiconductors has received considerable attention since the 1950s, due to their critical impact on electronic and optical properties. For instance, controlled doping in silicon enables tunable carrier concentrations10, which are instrumental to transistor operation and microelectronics. Similarly, defect engineering in ultrawide bandgap semiconductors enhances their potential for next-generation power electronics11. More recently, precise defect control, including doping, has emerged as a key enabler for quantum materials. In high-temperature superconductors, optimal superconductivity is achieved only within a range of hole-type doping concentrations12, 13. In flat-band systems, tuning the Fermi level via doping can unlock strongly correlated phases14, 15. Topological materials further highlight this sensitivity. In topological insulators and Weyl semimetals, quantum states such as the quantum anomalous Hall effect16,17 or large Berry curvature-induced responses18,19 only manifest when the Fermi level is precisely tuned through defect or dopant engineering. The importance of point-defect engineering in functional materials, such as semiconductors and quantum materials, creates an urgent need for characterization methods that can accurately quantify the types and concentrations of defects.
However, despite the variety of defect characterization techniques, significant limitations remain in terms of sensitivity, selectivity, quantifiability, and destructiveness. For sensitivity, methods such as diffraction and diffuse scattering primarily respond to large-scale microstructures and inhomogeneities, but not to isolated point defects. Regarding selectivity, positron annihilation spectroscopy (PAS) is highly sensitive only to vacancy-type defects20. Techniques like deep-level transient spectroscopy (DLTS)21, nuclear magnetic resonance (NMR)22 and electron paramagnetic resonance (EPR)23 offer exceptional sensitivity to low-concentration defects, but are only selective to electrically active trap states (for DLTS), specific isotopes (for NMR), or without chemical specificity (EPR). As to quantification, optical probes such as photoluminescence (PL)24, cathodoluminescence (CL)25, Raman and infrared (IR) spectroscopy26, along with X-ray probes like energy-dispersive X-ray spectroscopy (EDS), X-ray absorption spectroscopy (XAS) and X-ray photoelectron spectroscopy (XPS)27 are responsive to point defects, but often struggle to accurately quantify their concentrations, particularly in the presence of multiple coexisting defect species. Regarding destructiveness, imaging methods such as transmission electron microscopy (TEM)28 and atom probe tomography (APT)29 provide direct 2D and 3D visualization of defects, but require invasive sample preparation and are inherently destructive. Given these constraints, it would be highly desirable to develop characterization techniques that can accurately determine both the chemical identity and concentration of point defects with high sensitivity, broad chemical selectivity, and in a non-destructive manner.
In this work, we demonstrate that vibrational spectroscopy that measures PDoS, when integrated with machine learning, offers a promising route toward non-destructive characterization and quantification of point defects in bulk solids. We introduce DefectNet, a foundational machine learning model designed to infer both the chemical identity and concentration of point defects in solid-state materials from vibrational spectra. DefectNet takes as input the PDoS of pristine and defective crystals and outputs the defect chemical species and concentrations for up to six types of substitutional defects that coexist. To accommodate prior knowledge, DefectNet is flexibly designed to integrate initial guesses of candidate defect species, whether provided by the user or generated by an automated recommender. DefectNet addresses three key challenges. First, conventional atom-by-atom representations are inefficient across diverse materials with defects. We overcome this with the impurity-averaged configuration (IAC) representation, an encoding inspired by impurity average procedures in quantum field theory30–32. Second, at low concentrations, defective and pristine phonon spectra can appear nearly identical. To resolve this, we incorporate a spectral attention mechanism that enhances sensitivity to subtle perturbations. Third, the high cost of density functional theory (DFT)-based phonon simulations limits high-throughput data generation. We mitigate this by using machine-learning interatomic potentials (MLIPs)33–36 for efficient, high-fidelity vibrational spectra in broad chemical spaces. Trained on a dataset comprised of 16,000 simulated PDoS spectra derived from 2,000 distinct parent semiconductor materials, DefectNet accurately predicts defect types and concentrations as low as 0.2%. We further demonstrate that DefectNet can be fine-tuned using experimental inelastic neutron scattering (INS) data to recover Ge concentrations in SiGe alloys, as well as Al doping in superconductors. DefectNet marks an initial step toward general-purpose, non-destructive defect characterization and quantification from vibrational spectra, enabling systematic characterization and guiding rational defect engineering through high-throughput simulation and machine learning.
Results
Defect representation with impurity-averaged configuration
We start by introducing the defect representations in our model. Inspired by the fact that defects are normally modeled as perturbations on the crystal structure, we split atomic embeddings into for crystalline sites and for defect sites. Here, is the atom index, represents the defect type, and indicates that all these embeddings are learnable features. These features can be learned by performing an ensemble average on multiple atomic features. For example, for feature vector , which may include physical quantities such as charge number , mass number , spin , and radius , the atomic embedding can be expressed as
i.e., all input features of atom are optimized for their embedding mixture by an ensemble embedding layer Embed37. Similarly, denotes the feature vector of a defect atom of type at location , with the same embedding procedure as . Subsequently, to define the complete system with global geometry, we can define a generalized density operator with these atomic embeddings in Fourier space:
For the pristine part , it captures the full structure information of the parent material. In our approach, we reduce it to the chemical formula and pristine PDoS, which shows surprisingly good performance even with significant dimensionality dimension compression. For the defect part , we introduce the Impurity-Averaged Configuration (IAC), a stochastic embedding of ensemble defect contributions, to provide a machine-readable defect representation. Rather than working with deterministic positions , we treat them as random variables drawn from a learnable distribution , which encodes structural correlations such as short-range order (SRO) or spatial clustering. This yields a generalized form:
Here denotes a tensor product, and represents elemental multiplication. The probability model can be explicitly parametrized via pairwise correlation functions, learned via deep neural networks, or sampled from known experimental distributions. With the IAC defined by Eq. (3), one can readily embed the general defect-related information through averaging all the detailed positional information, given the appropriate probability distribution . Since the feature embedding is introduced in general, the IAC can be applied to represent multiple configurations of correlated defects, including positional, charge, and spin correlations, which can even be further connected to general observables.
Assuming randomly and independently distributed defects, the IAC representation reduces to the lowest order, which is only dependent on defect density for uncorrelated defects:
where is the density of type- defects and is the average feature embedding for all atoms of that defect type. Now if we focus on the first column of each feature embedding , i.e. gives full weight of the first trivial atomic feature in Embed, the resulting IAC descriptor turns out to be a concatenated vector , which gives us exactly the dopant concentration of each defect in a given material. We note that the IAC representation above is inspired by the impurity average technique for defects in quantum field theory. Details regarding the physical motivation are discussed in Supplementary Information 1.
Model architecture of DefectNet
Equipped with the reduced IAC defect representation (Eq. (4)), we now introduce the workflow of the DefectNet model, shown in Figure 1. The goal is to identify point-defect chemical species and concentrations directly from PDoS non-destructively. Data Generation: We construct a dataset of 16,000 doped supercells derived from 2,000 pristine materials38, covering a wide range of binary, ternary, quaternary, and quinary semiconductors (Figure 2b). Substitutional defects are selected from the first 56 elements of the periodic table, excluding noble gases (Figure 2a). These defects are introduced individually or in combination to reflect realistic co-doping scenarios, and may shed light for future defect engineering in a high-dimensional design space. We employ a machine-learned recommender to guide substitutional element selection (more details in Methods)39. The recommender serves a dual purpose: it guides the training data generation, and when turned on, it guesses the potential defect species in a supercell. We perform high-throughput structural relaxation and phonon calculations on both pristine crystals and doped supercells using the MACE-MP-0 foundational MLLIP35 under the frozen phonon formalism. We also apply Gaussian smearing to the simulated PDoS curves to mimic the experimental resolution function. Input for DefectNet: The model accepts four inputs: the parent pristine crystal composition, the PDoS of the pristine crystal, the PDoS of the doped system, and an initial guess of likely defect chemical species. This guess can be provided manually by human intuition or prior knowledge, or generated automatically through a defect recommender, which is an ML-based probabilistic model to suggest the most likely substitutional elements39. While our training data are simulated, the framework is designed to work with experimental spectra such as those from INS through a fine-tuning process. Model architecture and defect attention: Each input is processed through a separate channel. The composition and PDoS curves are fed through multi-channel convolution layers to generate a latent embedding . The defect recommendation is embedded into a query vector , which is then combined with in a multi-head attention module40. This allows the model to focus on spectral features that are most informative for identifying specific defects, which is especially important at low concentrations or when multiple dopants are present. The resulting features are passed through multi-layer perceptrons (MLPs), and a final hard-masking step filters the predictions according to the candidate defects. Output: Finally, the output features are hard-masked based on the initial defect guess, eliminating concentrations of dopants not included in the guess. This yields DefectNet’s predicted defect concentrations constrained to the set of initially assumed defect species. As a result, missed and incomplete guesses may prevent the model from recovering certain dopants. More details on the model architecture are discussed in Methods and Supplementary Information 2.
Figure 1. DefectNet workflow for predicting defect chemical elements and concentrations from phonon spectra. Starting from a pristine parent material, doped supercells with substitutional dopants are generated and processed through high-throughput PDoS calculations using machine learning interatomic potential (MLIP). An initial dopant guess, either manually provided or provided by a dopant recommender, is encoded along with the PDoS and parent material composition as input to DefectNet. It first employs multi-channel convolutions to extract spectral features, followed by a multi-head attention mechanism that integrates contextual relationships between defects and PDoS features. A final multilayer perceptron (MLP) outputs predicted defect chemical identities and concentrations, represented as an impurity-averaged configuration (IAC).
Figure 2. a. Histogram of dopant elements across all generated doped supercells, indicating chemical diversity. b. Composition breakdown of parent compounds, ranging from binary to quinary systems, with ternary compounds most common. c. Phonon density-of-states (PDoS) comparisons between pristine and , doped systems for two prototypical semiconductors: Si and AlN, with spectral shifts and broadening peaks introduced by light (H) and heavy (Sb) dopants. d. UMAP dimensionality reduction of the PDoS dataset used for DefectNet training, where a subset of materials is colored according to defect mass (blue: lighter than Ne; red: heavier than Br). The UMAP projection reveals clustering based on defect mass, demonstrating a level of feature separability in PDoS space and supporting its suitability for machine learning.
Defect signatures in vibrational spectra
Before applying DefectNet, we first demonstrate that vibrational spectra can indeed encode defect profiles, which offers an intuitive insight of DefectNet. Based on the quantum many-body theory of neutron scattering32,41, the defective and pristine PDoS can be expressed in the following form under certain assumptions for analytical convenience (see Supplementary Information 1):
where is the doped PDoS, represents the pristine PDoS, and is the prefactor depending on the microscopic properties with dependence.
Eq. (5) establishes a direct but nonlinear relationship between the defect concentration and the PDoS under the given approximation, which poses a challenge for conventional methods to be deconvoluted to extract defect information. To demonstrate the feasibility of machine learning on defect features from PDoS, we analyze simulated spectra of doped crystals using spectral comparison and unsupervised learning (Figure 2). Figure 2c compares the PDoS of pristine, 0.3%, and 1.2% doped silicon (Si) and aluminum nitride (AlN). Lighter defects like hydrogen (H) introduce high-frequency modes, while heavier defects like antimony (Sb) cause redshifts. These subtle but systematic shifts demonstrate the feasibility of machine learning to resolve defect chemical species from vibrational spectra. The meV-level energy shift due to the defects lies within the feasibility range of the actual experiment of INS. We further apply Uniform Manifold Approximation and Projection (UMAP)42 for dimensionality reduction, which projects high-dimensional inputs into a lower-dimensional space while maintaining relative distances between nearby points, to the 16,000 spectra. Figure 2d shows the resulting 2D embeddings clustered by defect mass, further confirming that defect-specific vibrational signatures are encoded and exhibit partial separability, even in unsupervised settings.
DefectNet for prediction of defect identify and concentration
Now that we have established the learnability of defect signatures from PDoS with the IAC representation, we proceed to train DefectNet and demonstrate its generalization capabilities. We start by evaluating the performance of DefectNet on simulated PDoS for prototypical binary (SiC and AlAs) and ternary (AgGaS2 and InCuSe2) semiconductors.
SiC and AlAs are semiconductors with large bandgaps valued for high-power electronics and heterostructure design, respectively43–45. Figure 3a shows the PDoS of pristine and doped configurations alongside predicted and true dopant concentrations. Despite low doping levels (), DefectNet largely captures subtle vibrational shifts and reliably recovers the correct dopant concentrations. We next apply DefectNet to more chemically complex ternary semiconductors AgGaS2 and InCuSe2 (Figure 3b). AgGaS2 is used in infrared nonlinear optics46, 47, while InCuSe2 is a promising material in thin-film photovoltaics48, 49. These materials contain multiple inequivalent atomic sites and more diverse vibrational modes. Still, DefectNet tracks PDoS changes and is capable of inferring dopant concentrations, showing its robustness across a broad range of structural and chemical complexities.
Figure 3. Representative predictions of DefectNet on prototypical semiconductors. a. Results for binary semiconductors SiC and AlAs; b. Results for ternary semiconductors AgGaS and InCuSe. For each material, we show the atomic structure of the parent crystal (top left), a bar plot comparing predicted and true defect concentrations (top right), and the target phonon density-of-states (PDoS) before and after doping (bottom).
To further evaluate DefectNet’s generalizability, we test its performance on the full defect dataset containing multiple coexisting dopants, including “distractor” defects that appear in the initial guess but do not actually exist in the ground truth. This setting evaluates the robustness of DefectNet to detect true dopants while suppressing false-positive candidates. Figure 4 summarizes these results by comparing predicted defect concentrations (colored dots) and ground-truth values (black dots), grouped by quartiles of prediction mean-square error (MSE). In the in-distribution case (Figure 4a), where pristine parent materials are seen in training data but their defect information is unknown, DefectNet achieves high-fidelity predictions across a wide range of defect chemical species and concentrations. Even with distractor defects included in the input, DefectNet can eliminate the distractors and identify the true dopants. Figure 4b shows the more demanding out-of-distribution case, where even pristine materials are not seen during training. Although the prediction accuracy is slightly reduced, DefectNet continues to capture the key dopant signatures and assigns near-zero concentration values to most distractors, reflecting strong model generalizability. Additional tests are shown in Supplementary Information 3.
Figure 4. Evaluating DefectNet predictions on diverse materials and defect configurations. a. In-distribution, known-parent-material, unknown-defect predictions, where training and testing sets are generated by randomly splitting phonon density-of-states (PDoS) curves from the same set of pristine parent materials. b. Out-of-distribution, unseen-parent-material, unknown-defect predictions, where the entire pristine parent materials are excluded from training and used only for testing. In both cases, each panel compares the predicted (colored dots) and ground-truth (black dots) defect concentrations with multiple coexisting defects. Red colored chemical elements indicate distractor species introduced only during testing to evaluate model robustness. The leftmost plots display the distribution of mean squared error (MSE) in predicted defect concentrations, grouped into quartiles based on overall model performance sorted by error (top to bottom: low to high).
While DefectNet performs well on synthetic PDoS data generated by MLIPs, these spectra differ from those obtained via DFT or experiments. Discrepancies in force-field accuracy, measurement noise, and uncontrolled defect environments can shift the data distribution, making defect signatures especially subtle and hard to resolve in cases such as lightly doped semiconductors. To effectively apply DefectNet to real experimental data, additional steps, such as fine-tuning on the target dataset, are necessary to bridge this domain gap and ensure direct applicability to experimental data.
Fine-tuning DefectNet on experimental data
To establish the practical utility of DefectNet, we fine-tune and test it on experimental data. In particular, we test DefectNet on thermoelectric SiGe alloy50 as well as MgB2 superconductor. Here, we highlight the case of SiGe, since it poses additional challenge due to the structural disorders even in the undoped system. DefectNet addresses this challenge by relying only on the vibrational spectra and chemical formula, making it suitable to study non-crystalline systems and thereby significantly expands its applicability. We construct a training dataset of 100 amorphous Si supercells sampled from the Si-GAP-18 database51 using quenching simulations. These supercells
span a wide range of disorders, from low-energy, near-crystalline states to highly disordered configurations. As the disorder increases, the PDoS broadens and the characteristic 60 meV optical phonon peak becomes suppressed (Figure 5a), providing parent materials for model fine-tuning. Subsequently, we simulate SiGe alloys by randomly substituting Ge into amorphous Si supercells, spanning various levels of disorder, at concentrations ranging from 0% to 25%. Given access to an existing database of amorphous Si, this approach is more efficient than direct quenching of an alloy system while still capturing meaningful spectral variations across a broad structural space. Vibrational properties are computed using MatterSim, a high-accuracy MLIP36. To match experimental observables, we compute the generalized phonon density-of-states (GPDoS), which represents a neutron-weighted average over atomic species:
Figure 5. Fine-tuning DefectNet for experimental validation on SiGe alloy. a. Distribution of excess potential energies for disordered Si structures, indicating varied degrees of disorder. Three representative atomic configurations are shown alongside their computed PDoS, ranging from near-crystalline (top) to highly amorphous (bottom). b. Workflow of fine-tuning DefectNet: a pre-trained DefectNet (red box) is adapted to through transfer learning to obtain the fine-tuned DefectNet with updated weights (red box with blue fine-tuned weights). The scatter plot compares predicted and truth Ge fraction with a low RMSE=0.019. c. Experimental generalized phonon density-of-states (GPDoS) for pure Si and alloys with , , and , extracted from Ref. 50. d. Comparison of predicted Ge fractions from fine-tuned DefectNet against experimentally reported values for the same set of samples.
where is the partial PDoS for atom , is the atomic fraction, is the neutron scattering cross-section52, and is the atomic mass. This results in 500 unique GPDoS data used to fine-tune DefectNet.
After fine-tuning, DefectNet achieves a test set root squared mean squared error (RMSE) of 0.019, indicating strong predictive performance (Figure 5b). We then apply the fine-tuned DefectNet model to experimental GPDoS data for SiGe alloy50, with nominal alloying ratio of , , and (Figure 5c). DefectNet predicts Ge concentrations of 7%, 13%, and 22%, respectively, which closely track the experimental trends (Figure 5d). Given the inherent difficulty in precisely quantifying defects in amorphous materials, our results demonstrate strong predictive power of DefectNet when applying to experimental data. For the aluminum (Al)-doped multiband superconductor MgB53, the fine-tuned DefectNet reproduces the experimental trend of dopant concentration up to 25%. More details are discussed in Supplementary Information 4.
Discussion
In this work, we present DefectNet, a foundational machine learning model that enables non-destructive identification and quantification of point defects directly from vibrational spectra. Capable of resolving up to six coexisting substitutional defect species at concentrations as low as 0.2%, DefectNet operates on PDoS data without requiring detailed atomic structures. The model is built upon a physically motivated IAC representation and enhanced with a spectral attention mechanism that boosts sensitivity to subtle vibrational shifts. When fine-tuned on experimental INS data, DefectNet can recover dopant concentrations in both crystalline and amorphous systems, as demonstrated in SiGe alloys and MgB superconductors. These results not only establish the transferability of DefectNet to real-world materials, but also position INS as a powerful and element-sensitive platform for non-destructive defect reconstruction.
While promising, several challenges remain. First, the model’s sensitivity decreases at very low defect concentrations, where vibrational signatures become subtle and hard to distinguish from noise. Second, the current version is limited to substitutional doping; extending DefectNet to handle multiple point defect types, such as interstitials, vacancies, Frenkel pairs, or defect clusters, would significantly expand its scope. Third, despite strong generalization from simulated data, fine-tuning on experimental datasets remains essential. A model directly and robustly applicable to raw experimental spectra without retraining remains an aspirational but formidable goal. Finally, while INS offers exceptional resolution and chemical sensitivity, its limited accessibility constrains widespread deployment. Building a parallel DefectNet variant for more accessible techniques such as Raman spectroscopy would greatly enhance usability and integration in diverse laboratory environments.
Looking forward, DefectNet represents a step toward a unified, data-driven paradigm in defect science. Its architecture is naturally compatible with multimodal spectroscopic inputs and opens the door to inverse design of materials with targeted defect profiles. By combining physically informed representations, high-throughput simulation, scalable learning, and experimental fine-tuning, DefectNet charts a path toward automated, interpretable, and non-destructive defect engineering across the full complexity of real materials.
Methods
Dopant recommender for DefectNet
To determine viable dopant substitutions for each semiconductor, we employ a two-stage recommendation system. For each parent compound, we first consult the probabilistic substitutional doping model proposed by Hautier et al.39, which ranks element pairs by chemical compatibility and historical substitution trends. In cases where the Hautier model failed, often due to invalid oxidation states, we apply a fallback strategy based on Shannon ionic radius matching54,55 to identify plausible dopants. Only dopants that passed either filter are used in subsequent sampling. This recommendation framework ensures physically reasonable, chemically consistent dopant-host combinations across all DefectNet training data. Both dopant recommenders are implemented in the Pymatgen interface56.
Dataset generation for universal DefectNet
To generate a high-quality dataset for training the universal DefectNet, we randomly select 2,000 semiconductors from an established semiconductor database38. Each structure is processed through the dopant recommender to identify -type and -type doping candidates. The host crystals are subsequently expanded into supercells containing between 433 and 500 atoms, with the size adaptively adjusted to ensure dilute doping concentrations with a lower bound of . Dopants are substituted into the host lattice, and each doped structure is relaxed, until all atomic forces converged below 0.01 eV/Å. Following structural relaxation, vibrational properties are evaluated using the finite-displacement method to compute PDoS, forming the spectral inputs to DefectNet. All calculations for this pre-training step are performed using the MACE-MP0 MLIP35 on the Atomic Simulation Environment (ASE)57 and Pymatgen56 interfaces.
Additionally, we note that DefectNet imposes no strict constraint on the source of the vibrational spectroscopy. In practice, the doped sample’s spectrum is tractable from measurements like INS as a real-world input. For the pristine counterpart, users are free to choose the most credible or convenient source available: this may be first-principle DFT calculations, a machine-learned surrogate model58,59, a machine learning interatomic potential (MLIP), or even experimental data collected under the same lab conditions as the doped sample to ensure maximal consistency.
Methods of machine learning
We use scikit-learn60 and UMAP42 package for dimensional reduction on PDoS data. The PyTorch61 package is used for DefectNet training. DefectNet is a hybrid convolutional-attention neural network predicting dopant species and concentrations from vibrational spectra. The spectral features are first processed by two blocks of multi-channel 1D convolutional layers, each followed by a ReLU activation. The convolved representation and initial dopant guess vector are then projected to value () and query () embeddings and passed through a multi-head self-attention module. The attention mechanism follows the standard scaled dot-product self-attention formulation40:
where , , and are the query, key, and value matrices, respectively, and is the dimensionality of the key vectors. In the case of self-attention used in DefectNet, we have as both are derived from the same input. To enhance representational capacity, DefectNet employs multi-head self-attention:
where each attention head is computed as:
and , , , and are learnable weight matrices specific to each head. Hard masking is applied to suppress attention weights for zero dopant concentrations, ensuring that attention is only computed over the user’s initial guessed dopants.
Data and code availability
The data used in this study are numerically generated using our code implementing MLIP, and the code used in this study is available at https://github.com/vipandyc/MLforDefect.
Acknowledgments
The authors thank KA Nelson and N Andrejevic for helpful discussions, and thank S Wilson for providing the experimental data on SiGe alloy, originally published in Ref. 50. M.C. acknowledges the support of U.S. Department of Energy (DOE), Office of Science (SC), Basic Energy Sciences (BES), award No. DE-SC0021940. C.F. acknowledges support from DOE BES award No. DE-SC0020148. B.Y. National Science Foundation (NSF) Designing Materials to Revolutionize and Engineer our Future (DMREF) Program with award No. DMR-2118448. A.C. acknowledges support from NSF ITE-2345084. D.L.A. and Y.C. were supported by the Scientific User Facilities Division, Office of Basic Energy Sciences, U.S. Department of Energy, under Contract No. DE-AC0500OR22725 with UT Battelle, LLC. A portion of computational simulation results were obtained using the Lonestar6 computing system at the Texas Advanced Computing Center. M.L. acknowledges the support from the Class of 1947 Career Development Chair and the support from R. Wachnik.
A Foundation Model for Non-Destructive Defect Identification from Vibrational Spectra: Supplementary Information
Mouyang Cheng; Chu-Liang Fu; Bowen Yu; Eunbi Rha; Abhijatmedhi Chotrattanapituk; Douglas L Abernathy; Yongqiang Cheng; Mingda Li
Defect Representation Physics
Defect representation with impurity average
We start by introducing our theoretical framework of the defect representation. To represent crystalline systems with defects, we begin by defining a generalized atomic density field in real space for an ideal, pristine crystal:
where runs over all atoms in the material, and is the corresponding spatial position. is constructed as the atomic feature vector, which could involve multiple features where , , , , and , are one-hot, charge, mass, spin, and radius features at the specific positions and finally can be expressed as:
which make all input physical features of each atom optimized for their embedding mixture by an ensemble embedding layer . Suppose we limit ourselves to thinking only about the local point-like perturbation introduced by the defects. We model defects as perturbations of these feature vectors or as contributions to off-lattice positions, thus
where denotes the feature vector of a defect of type at location , with same embedding types as represented before. To reduce the complexity under the thermodynamic limit with an infinite number of lattice sites/atoms and change from the linear vector to embedding, we shift to the momentum space with the embedding form:
Now we introduce the impurity average. For an impurity-position-dependent physical quantity and its thermodynamic ensemble average , the impurity average can be carried out by a joint probability that performs weighted average of the impurity spatial distribution,
In particular, assuming that the positions of each impurity are distributed independently and each impurity is distributed uniformly inside the solid, then we have
where is the system volume and is the number of defects assuming one type of defects exist. Notice that the probability here is assumed to be uniform, but can be generalized in other cases. The first equality comes from the assumption of independence between pairs of defects. The second equality comes from the uniformity of defect spatial distribution.
While the atomic density with defect depends on the specific position of the impurities, we can perform the impurity average with the thermodynamic ensemble average with the joint spatial probability of the defect configuration , which finally motivates us to define our Impurity-Averaged Configuration (IAC) for defects:
Here denotes a tensor product, represents elemental multiplication, and is the expectation value. The probability model contains all the spatial distribution relations of defects. This can be explicitly parametrized via pairwise correlation functions, learned via deep neural networks or sampled from known experimental distributions.
With the IAC defined above, one can readily embed the general defect-related information through averaging all the detailed positional information, given the appropriate probability distribution . Since the feature embedding is introduced in general, the IAC can be applied to represent multiple configurations of correlated defects, including positional, charge, and spin correlations, which can even be further connected to general observables. Assuming the defects are distributed randomly and independently, the IAC representation reduces to the lowest order, which is only dependent on defect concentrations:
where is the defect density of type- defects normalized from the number of defects and is the expectation value of the feature embedding for that defect type . Now if we focus on the first trivial atomic feature of each feature embedding , the resulting IAC descriptor turns out to be a concatenated vector , which gives us the dopant concentration of each defect in a given material.
Inelastic scattering for the phonon dispersion
To link the vibrational spectra and the IAC representation, we further define the retarded density-density response function and adapt the inelastic neutron scattering for the nuclear scattering to see how the phonon density of state can be considered as one of the features for representing the defects. The derivation is based on and adapted to the neutron inelastic scattering derivation[1, 2, 3], but with the defects in the signal. The neutron inelastic scattering is utilized to probe the nuclear density-density response function to understand the phonon dispersion. The nuclear density-density response function can be expressed as,
in which is the density-density response function in “interaction picture” in imaginary time , where the interaction picture means the average is , referring to the full materials Hamiltonian . Here , is Bosonic Matsubara frequency, while denotes the imaginary-time ordering operator. is the inverse temperature, while is the Boltzmann constant. represents the nuclear density function. We expand the density function in the lowest order with the embedded defect in a similar embedding form:
Here is the displacement for the pristine atomic density. is the displacement of the th defect with type. All the “hat” operators are in interaction picture w.r.t. the non-interacting part of the materials Hamiltonian, i.e. . This embedding and approximation means we treat the effect of defects or embedded inhomogeneities at the operator level, by modifying the expression of the embedded density operator to account for deviations from the ideal crystal structure. However, we assume that these modifications do not alter the underlying lattice dynamics. Therefore, all correlation functions are assumed to be evaluated with respect to the non-interacting Hamiltonian . As a result, when we consider the embedded density correlation, it will have three correlation contributions, the pristine crystal’s contribution , pristine-defect interaction contribution , defect-defect contribution , while
For , it generates an embedding of the density response function for the pristine part, and the connection between this response function and the inelastic neutron scattering is standard now[2]. To simplify the discussion, we take the low-temperature limit to ignore the Debye-Waller factor. We have the following equation
For and , cases are a little bit different, as we require the impurity average to conduct the averaging among the whole summation. But what we need to deal with is the momentum in exponential form from the density expansion and the displacement operator. So we may have the following results to simplify the expression
The term comes from the expansion of the displacement operator, and we have placed the impurity average with the independent uniform distribution in the final equality. Here we have assumed the corresponding displacement operator on the defect is expanded in a similar perfect crystal’s plane-wave basis, but with "distorted" mass, frequency, and (impurity) positions, and then we have
While this is an approximation—since defect modes may deviate from bulk phonons—the leading effects are captured by using defect-specific mass and effective frequencies, and finally by performing an ensemble average over the defect positions. This yields a defect-sensitive yet analytically tractable phonon response framework. Conducting such a calculation will lead to three channels with different defect number densities and the averaged feature factors as the contribution. Then for and , we have
in which is assumed the general free-phonon propagator, which is connected to Matsubara-domain phonon propagator as , is the phonon dispersion or phonon frequency corresponding to the wavevector . here refers to the averaged embedded feature of under the embedded form. In the actual physical expression of the neutron scattering, it should be reduced to the neutron scattering length. Here and are the ensemble averaged mass and frequency for the type defects after the impurity average. The impurity averaged frequency may bring subtlety to the response function and the final phonon density of state. However, the overall contribution is explicitly related to the defect concentration and its associated defect feature embeddings. We emphasize that this expression is a formal approximation, not a physically exact description of defect vibrations. It serves to provide a structured mathematical form that supports reasoning and feature construction in our ML model, rather than modeling defect dynamics with full physical accuracy. The possible actual subtle non-linear contribution should be modelled through ML.
Finally, notice that the imaginary part of corresponds to the spectral function, whose momentum integration yields the phonon density of states (PDoS) :
and for the sole defect contribution, we should have the embedding
where is the defective PDoS, represents the pristine PDoS, is the prefactor with dependence. and are the corresponding embedded retarded response function of and , respectively. Thus, there exists a direct link from the defect embeddings – which is defined in IAC – and the measured PDoS . This connection confirms the intuition that PDoS is inherently sensitive to the type and distribution of dopants present in the material.
Training Data and Architecture for DefectNet
Statistics of training data
In this section, we analyze the elemental composition and dopant relationships within the DefectNet training dataset. Elemental distributions of pristine (undoped) and doped crystals are illustrated in Figure S1. In pristine crystals, oxygen (O), sulfur (S), and selenium (Se) emerge as the most abundant elements, whereas doped crystals predominantly contain oxygen (O), fluorine (F), and chlorine (Cl). Notably, among the 56 elements considered, all five noble gases (He, Ne, Ar, Kr, Xe) are absent from the dataset, consistent with their limited presence in stable crystalline materials.
Figure S1. Elemental distributions in the DefectNet training dataset. a. Elemental composition of pristine (undoped) crystals. b. Elemental composition of doped crystals.
In Figure S2, we demonstrate a heatmap visualization of defect element frequencies in doped crystalline materials among the DefectNet dataset. Each cell quantifies the frequency with which a given host element (rows) is replaced by a different dopant element (columns), visualized on a logarithmic scale to accommodate the wide dynamic range of substitution events. The diagonal entries representing self-substitution are naturally zero. Notably, several substitution trends emerge from the off-diagonal structure. Halogen elements such as F, Cl, and alkaline earth metals such as Na, K, and Ca exhibit broad substitution profiles, consistent with their high mobility and prevalence in defect chemistry. Sparse regions of the matrix highlight chemically unfavorable or structurally constrained substitutions, often involving large size or charge mismatches heavy post-transition metals.
Figure S2. Heatmap visualization of defect element frequencies in doped crystalline materials among the DefectNet dataset. Rows represent 56 original host elements, and columns represent 56 defect-introduced elements (part of chemical elements are omitted in the label).
Architecture of DefectNet
Here we illustrate the detailed neural network architecture of the proposed DefectNet. The DefectNet model is implemented in PyTorch and follows a modular architecture consisting of four components: (1) a spectral encoder based on 1D convolutions, (2) a dopant embedding module, (3) a multi-head attention mechanism, and (4) a
dopant-masked module, which will be discussed the next subsection. This section details the first three components, as depicted in Figure S3.
Figure S3. Detailed architecture of the DefectNet model with annotated tensor dimensions. The model takes as input three aligned one-dimensional signals of length 100: the pristine PDoS, the doped PDoS, and a vector encoding the parent crystal composition. These are concatenated into a 3-channel input tensor of shape , where is the batch size. The spectral encoder processes this input through multiple 1D convolutional layers and ReLU activations, yielding a latent feature sequence of shape , denoted as the value matrix . Simultaneously, the initial dopant guess is represented as a binary vector of length 56, which is embedded in shape to form the query matrix . The attention output is then passed through a multi-layer perceptron to predict dopant concentrations, while a masking step is applied to ensure only the initially guessed dopants receive non-zero predictions. All tensor dimensions shown reflect batch-wise shapes.
Each training instance consists of three aligned one-dimensional inputs of fixed length for the normalized PDoS, including the PDoS of the pristine host material and the PDoS after doping. Besides, we input a zero-padded vector encoding the host composition as well. These inputs are concatenated along a channel axis and processed through a sequence of one-dimensional convolutional layers with ReLU activations. The convolutional stack extracts local spectral features and transforms the input into a latent representation with channels, corresponding to an embedding dimension of 128. The result is a sequence of 100 embedded spectral tokens capturing both local and global vibrational characteristics.
In parallel, the initial guess of dopant candidates is provided as a 56-dimensional binary vector, indicating which dopants are considered for a given sample. This binary vector is projected into the same 128-dimensional latent space through a fully connected layer, resulting in a single dopant embedding vector. This embedding serves as a global query for the attention mechanism, allowing the model to modulate its spectral interpretation based on the specific dopant context.
To model the interaction between the dopant candidates and vibrational features, DefectNet employs a multi-head self-attention mechanism[4]. The dopant embedding acts as the query , while the spectral features act as both the key and value matrices. The attention follows the standard scaled dot-product formulation:
where , . To capture diverse spectral-dopant relationships, DefectNet uses multi-head attention:
where each attention head is computed as:
and , , , and are learnable weight matrices specific to each head. This attention mechanism enables the model to identify which parts of the vibrational spectrum are most informative for determining the presence and identity of dopants. By aligning spectral patterns with dopant-specific context, the model can resolve subtle or overlapping features arising from co-doping or low-concentration defects.
The output of the attention block is a 128-dimensional feature vector that summarizes dopant-aware spectral information. This vector is passed through a multi-layer perceptron consisting of two hidden layers and a final regression layer that outputs predicted concentration values for all 56 possible dopants.
Dopant recommender and hard masking
To ensure both user-friendly deployment of DefectNet and physical consistency with dopant selection, the model’s output is element-wise masked using the original dopant vector. There are two typical use cases for dopant selection. In the first case, users have prior knowledge of the dopant species of interest. They can apply a hard masking operation to enforce predictions only on the specified elements. In the second case, users may be uncertain about the exact dopants present in the material system. In this scenario, a dopant recommender can be used to construct an appropriate hard masking, thereby restricting the predictions to a physically plausible set of dopant elements.
The dopant recommender, which evaluates the compatibility between candidate dopant elements and the host material, has been described in the main text. The hard masking mechanism, as illustrated in Figure S4, imposes an explicit constraint whereby only the initially selected dopants are allowed to receive non-zero predictions. Specifically, the mean squared error loss is computed only over those dopant elements selected in the initial guess. This loss masking reinforces the effect of the architectural masking by ensuring that the model is only penalized for predicting concentrations on plausible dopant species, which further improves training stability, prevents overfitting to distractor classes, and aligns optimization with the physical priors embedded in the dopant recommender.
Figure S4. Hard masking procedure in DefectNet. This masking is employed to enforce that only selected dopants receive non-zero predictions. Such a design allows flexibility and alignment with the user’s initial guess on dopant species.
Additional Results for DefectNet Prediction
In this section, we present additional prediction results of the pre-trained DefectNet model.
Prediction within the full dataset
In Figure S5 and Figure S6, the full dataset comprising parent crystals and their corresponding doped variants is randomly partitioned into distinct training and testing subsets. This setup constitutes an in-distribution scenario typical of standard machine learning validation protocols, where spectral features and dopant interactions present in the test set closely resemble those encountered during training. Within this evaluation framework, DefectNet demonstrates robust and highly accurate predictive capabilities, effectively capturing complex spectral signatures across diverse dopant configurations.
Figure S5. Additional prediction results of DefectNet on defects, with all PDoS curves randomly mixed and split to the testing set. Each panel compares the predicted (colored) and ground-truth (black) dopant concentrations across the full dataset with multiple coexisting dopants. Additionally, distractor dopant candidates (elements colored red) exist during the testing process. The leftmost plots display the distribution of mean squared error (MSE) in predicted defect concentrations, grouped into quartiles based on overall model performance sorted by error (top to bottom: low to high).
Figure S6. Additional prediction results of DefectNet on defects, with all PDoS curves randomly mixed and split to the testing set. Continued from Figure S5.
Prediction extrapolating parent crystals beyond the dataset
DefectNet achieves state-of-the-art predictive performance even when evaluated under extrapolative conditions that rigorously test its generalization capabilities. In Figure S7 and Figure S8, a subset of parent crystals is completely excluded from the training process, ensuring that no such undoped structures are present during model training. Despite the absence of prior exposure to these parent compounds, DefectNet continues to deliver highly accurate predictions.
Figure S7. Additional prediction results of DefectNet on defects, with a fraction of parent crystals held out entirely from training to test DefectNet’s ability to extrapolate to unseen materials. Each panel compares the predicted (colored) and ground-truth (black) dopant concentrations across the full dataset with multiple coexisting dopants. Additionally, distractor dopant candidates (elements colored red) exist during the testing process. The leftmost plots display the distribution of mean squared error (MSE) in predicted defect concentrations, grouped into quartiles based on overall model performance sorted by error (top to bottom: low to high).
Figure S8. Additional prediction results of DefectNet on defects, with a fraction of parent crystals held out entirely from training to test DefectNet’s ability to extrapolate to unseen materials. Continued from Figure S7.
Validation on Experimental Data
Experimental fine-tuning on SiGe alloy
As discussed in the main text, we have demonstrated the feasibility of applying DefectNet to SiGe alloys. Here we show additional structural statistics of the training dataset, which consists of a diverse set of disordered bulk silicon configurations used for fine-tuning DefectNet. As shown in Figure S9, structures with higher excess energy exhibit increasingly disordered local environments, as evidenced by the progressive loss of long-range order in their radial distribution functions . The dataset spans a broad range of structural motifs, from nearly crystalline to amorphous-like, enabling comprehensive sampling of the disordered configuration space. Ge atoms are subsequently doped into these host structures to construct the full training set for DefectNet.
Figure S9. Statistics for the training data used for fine-tuning DefectNet. Left panel: distribution of excess energies of the sampled bulk Si structures. Right panel: radial distribution functions for three representative energy levels, indicating increasing structural disorder at higher excess energies.
Experimental fine-tuning on Al-doped MgB
To evaluate the transferability of our defect learning framework beyond simulated datasets, in addition to the SiGe alloy presented in the main text, we also fine-tune DefectNet on experimental data for the Al-doped superconductor MgAlB, as is shown in Figure S10. This system is a well-known electron-doped superconductor where Al substitution modifies both electronic and vibrational properties, providing a stringent test for spectral-based ML prediction for superconducting systems other than semiconductors. We collect experimental generalized phonon density of states (GPDoS) for various Al concentrations () as reported in Ref. [5] and reproduced in Figure S10(a). The DefectNet model is fine-tuned on simulated spectra of pristine and Al-doped MgB using the MatterSim v1.0.0-1M MLIP. After fine-tuning, the model achieved a root-mean-square error (RMSE) of 0.008 on synthetic data (Figure S10(b)).
Figure S10. Fine-tuning DefectNet and validation on experimental Al-doped MgB. a. Experimental generalized phonon density of states (GPDoS) curves for bulk MgB and MgAlB with , 0.25, and 0.50, as is extracted from Ref. [5]. b. Scattered plot for predicted versus ground-truth Al concentration . c. Comparison between DefectNet-predicted Al fractions and the lab reported ground-truth for the three MgAlB samples.
Predictions for dopant concentration showed reliable agreement with experimental labels, as shown in Figure S10(c). However, at a higher doping level (), prediction performance degraded significantly. This breakdown likely originates from the increasing complexity and nonlinearity of the experimental signal at high dopant levels, which diverges from the cleaner and more idealized training spectra generated from simulations. Moreover, the discrepancy between clean DFT-generated spectra, MLIP-based approximations, and actual measured spectra introduces a domain shift that is not fully captured by the current model. These findings underscore both the promise and current limitations of purely spectrum-driven approaches for defect inference in complex materials and highlight opportunities for incorporating domain adaptation techniques or hybrid physical priors to improve robustness.
References
References
References
- [1]Bruus, H. & Flensberg, K. Many-body quantum theory in condensed matter physics: an introduction (Oxford university press, 2004).
- [2]Coleman, P. Introduction to many-body physics (Cambridge University Press, 2015).DOI
- [3]Fu, C. et al. Anomalous neutron nuclear-magnetic interference spectroscopy. arXiv preprint arXiv:2403.13990 (2024).arxiv.org/abs/2403.13990
- [4]Vaswani, A. et al. Attention is all you need. Adv. neural information processing systems 30 (2017).arxiv.org/abs/1706.03762
- [5]Yokoo, T. et al. Evidence of electron–phonon interaction in Al-substituted Mg₁₋ₓAlₓB₂. J. superconductivity 17, 199–203 (2004).DOI




