WorksPhysics and Astronomy
Machine Learning the Self-Energy Operator for Real Materials
Abstract
Machine learning of in solids has largely focused on prediction of scalar quasiparticle energy corrections. Here, we introduce a phase-aware many-body transformer that predicts the full -resolved complex self-energy matrix. As a proof of principle, we apply the model to predict quasiparticle energies before and after diagonalization of the quasiparticle Hamiltonian, obtaining a test set mean absolute error (MAE) of 29 meV for a dataset of 1745 2D materials in the static COHSEX approximation and 60 meV for a more limited dataset of 433 materials calculated within a plasmon-pole model. COHSEX pretraining followed by finetuning reduces this MAE to 42 meV. The predicted matrices also accurately reproduce QP wavefunctions.
main.pdf
Machine learning (ML) has become a powerful tool for accelerating first-principles calculations of materials’ ground-state properties [1], ranging from neural-network interatomic potentials that can reach the accuracy of density functional theory (DFT) at a small fraction of its computational cost [2, 3] to graph neural networks that can predict materials properties and independent-particle Hamiltonians directly from the atomic structure [4, 5, 6]. Most of these approaches are built on DFT, the workhorse of ground-state electronic structure and the basis of large-scale materials databases [7]. However, Kohn-Sham (KS) eigenvalues [8, 9] belong, in principle, to an auxiliary system of noninteracting particles and do not describe the quasiparticle (QP) energies measured in photoemission and tunneling spectroscopy. Within first-principles many-body perturbation theory, the approximation is the standard approach for predicting QP properties of solids [10, 11]. Within the approximation, the electron correlation is encoded in the nonlocal self-energy operator , where is the single-particle Green’s function and is the screened Coulomb interaction. calculations are, however, typically orders of magnitude more expensive than DFT [12].
The high computational cost of has motivated the development of ML surrogates for QP energies and self-energy corrections. Early machine-learning approaches to focused on accelerating intermediate quantities such as the dielectric screening [13, 14]. Most subsequent work has directly targeted QP energy corrections, corresponding to the diagonal self-energy in the Kohn-Sham (KS) basis. These approaches have been used to transfer accuracy across configurations and system sizes for individual materials [15, 16, 17], as well as across chemical space for molecules [18, 19] and solids [20–22]. More recently, machine learning has been extended to the off-diagonal self-energy matrix: Venturella and co-workers demonstrated transferable prediction across chemically diverse molecules [23, 24], while applications to solids have so far been limited to model Hamiltonians [25] and -point only models for simple solids like diamond and silicon [26]. A -resolved prediction of the complex self-energy matrix across chemically diverse solids has not yet been demonstrated.
Going beyond a diagonal one-shot correction, the self-energy matrix elements are needed to obtain QP wavefunctions [27], enable quasiparticle self-consistent (QSGW) calculations [28, 29], and provide the QP basis required for downstream treatments of electron-phonon interactions [30] and nonequilibrium dynamics [31–33]. At the mean-field level, the analogous step from learning eigenvalues to learning the full Hamiltonian is already well established [5, 6] [34, 35, 36, 37, 38]. However, directly learning for solids in the Bloch basis poses two additional challenges. First, is complex-valued and gauge-covariant. Under the arbitrary phase freedom of each Bloch state, , the matrix elements corotate as
so a faithful surrogate must retain the relative interband phase. Second, small errors in individual matrix elements can be amplified by diagonalization. Accurate Hamiltonian learning therefore demands substantially greater element-wise fidelity than regression of scalar QP energies.
In this Letter, we overcome these challenges and demonstrate the -resolved prediction of the complex self-energy matrix across chemically distinct 2D materials. In our previous work, we introduced MB-Former [22], a transformer-based framework for predicting many-body corrections from mean-field electronic states in which attention [39] uses relationships among mean-field states to predict their many-body renormalization. Here, we extend MBFormer from diagonal QP corrections to the full self-energy matrix. The key advance is a phase-aware autoencoder that compresses both the amplitudes and phases of the complex KS wavefunctions, retaining the relative interband phase required to predict off-diagonal matrix elements. Additionally, we improve the accuracy, transferability, and efficiency of the model by introducing 1) joint training of the autoencoder and transformer, 2) a common physical-momentum representation of the wavefunctions, 3) gauge augmentation, and 4) factorized attention over momentum and band-pair degrees of freedom. We demonstrate this approach for both the static Coulomb-hole screened exchange (COHSEX) [10, 11] and the dynamically screened generalized plasmon-pole (GPP) [11, 12] self energies. The resulting models achieve state-of-the-art QP-energy mean absolute errors (MAE) of 29 meV for COHSEX and 60 meV for GPP, retain their accuracy upon diagonalization, and reproduce QP wavefunctions. Joint training further reduces the QP error and makes many-body information more accessible in the wavefunction latent space.
We start by building two GW datasets with the BerkeleyGW code [12] on top of plane-wave DFT from Quantum ESPRESSO [40]. First, we calculate static COHSEX self-energy matrices for 1745 2D semiconductors whose structures are taken from the C2DB database [41, 42]. Because COHSEX retains the matrix structure of the GW self energy while being substantially cheaper to compute, it provides a practical setting for developing the model and testing its transferability across materials. Then, we apply the same framework to GPP self energy matrices for 433 hexagonal materials to assess its performance with dynamical screening. For each dataset, 90% of the materials are used for training and 10% are withheld for testing. Details of dataset generation are given in the Supplemental Information (SI).
Fig. 1 summarizes the model architecture. We start with an explicit KS DFT calculation. Then, we construct a common, phase-aware representation of each KS state. For each KS Bloch state, we start by mapping the planewave coefficients to a physical momentum and resample it with a fixed Gaussian kernel onto a common grid of momenta (P grid in Fig. 1b). This gives the coefficient array a consistent physical meaning across materials, so that its coordinates correspond to the same physical momenta rather than material-dependent reciprocal-lattice indices. Having established this common momentum-space representation, we next decompose the coefficients into amplitude and phase components, allowing us to go beyond the amplitude-only representation used in our previous work [21, 22]. We write the coefficients as and encode and decode the amplitude and phase on separate branches (Fig. 1c). The phase branch predicts rather than directly, avoiding the artificial discontinuity between phases near 0 and . Both branches are convolutional networks with ConvNeXt-type residual blocks over the inplane momenta. Each state is then compressed into separate amplitude and phase latents, and , with a combined dimensionality of 280, corresponding to a compression of nearly two orders of magnitude.
The compressed states are then fed into a factorized many-body transformer (Fig. 1d) whose tokens are band pairs with , since the remaining elements follow from Hermiticity. We construct the inputs to mirror the inputs of an explicit calculation. For every point, the token concatenates the state vectors and with the exchange-correlation matrix element , pair-wise overlap descriptors (see SI), and charge densities (Figs. 1a,d). Each state vector combines the latents and with additional physical descriptors of the state (see SI). Because the additional pairwise descriptors in connect the same two KS states as , they carry the same relative interband phase and provide a gauge-covariant reference for the complex output.
During training, we apply exact gauge augmentation. Random phases are applied to the KS wave-functions (Fig. 1c), and the label and the pair features (Figs. 1d,e) are rotated according to Eq. (1). While the autoencoder has a learned gauge-covariance, the transformer is not explicitly gauge-equivariant. Instead, the augmentation teaches the network the required covariance [43].
The large number of band-pair tokens makes full attention over all combinations computationally expensive. We reduce this cost and guide the band coupling by taking advantage of the physical structure of the self energy. In the KS basis, the self energy has the form
where runs over intermediate bands, over momentum transfers, is the vector over of plane-wave matrix elements , and is the screened Coulomb interaction. Because the sums over bands and over momenta are separate, we factorize attention along the -point and band-pair axes (Fig. 1d). For the present system size of 25 points and 105 band-pair tokens per point, this reduces the attention cost by a factor of roughly 20. Eq. (2) also couples and through the product . Each transformer layer therefore ends with a gated triangle update [44], which adds to each token a learned map of (Fig. 1d), where and are learned projections of the pair tokens at the same , and runs over the bands included in the model so that each pair receives information from all pairs and , in analogy to the contraction over intermediate bands in Eq. (2). A lightweight head then maps each token to , with . Diagonalizing the QP Hamiltonian then yields updated QP energies and wavefunctions (Fig. 1f).
Finally, the autoencoder and the transformer are trained jointly using the composite loss (Fig. 1e)
Figure 1. Architecture. a, Inputs from DFT. b, The plane-wave coefficients of each KS state are resampled from the G grid () to a grid of physical momenta (i.e. the P grid of dimension ) and split into amplitude and phase. c, The phase-aware autoencoder compresses them separately into the latents and and reconstructs the amplitude and the phase. d, In MBFormer, the state vector combines , , a radial encoding of , the element decomposition, , and the band indices. The pair tokens () combine , , , overlaps, and density. Three layers of -axis attention, -axis attention, and a triangle update, followed by an MLP head and a Hermitian mirror, give . e, and are optimized jointly. and denote the random gauge transformation of Eq. (1) applied during training. f, Diagonalizing yields the QP energies and wavefunctions.
where is the error of the complex matrix elements of and is the error of the reconstructed amplitude and phase on the P grid. Because the transformer operates directly on the autoencoder latent, the self energy loss also updates the encoder, allowing the representation to adapt to the many-body prediction task rather than reconstruction alone. Fig. 2a shows the predicted complex self-energy matrix elements against the calculated ones for the static COHSEX dataset. The model reaches on the training set and on the test set, demonstrating high-fidelity prediction of the self energy matrix across materials. The QP energies obtained from this matrix are also accurate (Fig. 2b): the one-shot energies from its diagonal, , and the eigenvalues of the full QP Hamiltonian after diagonalizing, , both have a mean absolute error (MAE) of 29 meV. The near equality of these two errors shows that diagonalization does not degrade the prediction. The inset of Fig. 2b shows the contribution of the off-diagonal self energy by plotting for each state, which the model reproduces with and an MAE of 4 meV, compared with a mean magnitude of 21 meV. Diagonalization also yields the QP wavefunctions, whose overlaps with the KS states are predicted with (see SI). The predicted self energy also reproduces the QP dispersion across the Brillouin zone.
Figures 2c,d show the band structures of the two test materials whose corrections have the largest variation across band index and , respectively. In FeZrCl the lowest valence band moves 1.2 eV more than the valence bands above it, and in CuCl the correction of a single band changes by up to 1 eV along . The model reproduces both bandstructure with MAE of 38 and 12 meV, respectively. The corrections are predicted on the input grid and interpolated to a dense path. More examples, including the material NaCNO with the largest MAE=174meV, are given in the SI. Once the DFT calculation is finished, the model predicts the self energy matrix of a material in about 3 s on one GPU node, when the first principle calculation takes on average 42 min on 256 CPUs
Figure 3 and 3b show the corresponding results for the GPP dataset. Despite the smaller dataset, the model still attains and 0.9988 for the training and test sets, respectively, with corresponding MAEs of 59 meV for the one-shot energies and 60 meV for the energies. The larger errors compared with the static COHSEX prediction are consistent with the reduced amount of training data. As shown in Fig. 3c for the static COHSEX dataset, the test-set MAE decreases monotonically with the number of training materials, from 92 meV for 174 training materials to 30 meV for 1571. We therefore expect the GPP prediction accuracy to approach the static COHSEX level once more labels are generated. If the model learns transferable structure in the mapping from mean-field states to the self energy, rather than correlations specific to the static COHSEX labels, pretraining on COHSEX should improve its prediction of the GPP self energy. Indeed, we find that fine-tuning the static COHSEX model on the GPP training set for only 20 epochs reduces the MAEs to 41/42 meV (Fig. 3d), about 30% lower than training from scratch. This setting reflects the practical situation in which inexpensive static COHSEX labels are available for all materials and the more expensive dynamically screened labels only for some of them.

Figure 3. Prediction of GPP self energy matrices and data efficiency. a, Same as Fig. 2a for the GPP self-energy matrix elements from the GPP dataset. b, Same as Fig. 2b for the GPP test set. c, Test-set MAE vs. training epoch for static COHSEX models trained on subsets of 174 to 1571 training materials and evaluated on the fixed set of 174 test materials. d, Same as b for a model pretrained on the static COHSEX dataset and fine-tuned on the GPP training set for 20 epochs.

Figure 2. Prediction of static COHSEX self energy matrices. a, Predicted vs. calculated static COHSEX self energy matrix elements (real and imaginary parts as separate points) for the training and test sets. b, Test-set QP energies from the diagonal elements () and from diagonalization of the QP Hamiltonian (). Inset: predicted vs. calculated . c, d, Band structures of the test materials FeZrCl (c) and CuCl (d) from DFT (black), calculated (blue, GW), and predicted (orange dashed, GW-ML).
Figure 4. Ablation studies and joint versus separate training. a, Test set MSE of the static COHSEX self-energy matrix vs. epoch for the full model (Base), without the wavefunction input (w/o wfn), and with the transformer layers replaced by a MLP (w/o attention). b, Same as a without the charge density, , the KS energies, the overlap descriptors, or the chemical element labels. c, Same as Fig. 2b with the autoencoder pretrained on reconstruction alone and frozen (separate training). d, Reconstruction fidelity of the test states on the grid for joint and separate training.
To identify the essential ingredients of the model, we perform ablation studies on the static COHSEX dataset, in which one input or component is removed and the model is retrained with otherwise identical settings (Figs. 4a,b). We first test the two core components of the model: the wavefunction input and the transformer attention that models the many-body interaction. Removing all wavefunction inputs and the whole autoencoder structure increases the MSE of the test-set self-energy matrix by a factor of 5.5 (Fig. 4a). Replacing the transformer layers by a multilayer perceptron, which removes all communication between band pairs and points, increases the error by a factor of 2.5. Both the wavefunction information and the transformer attention between band pairs and points are therefore indispensable. Among the other physical inputs (Fig. 4b), removing , which then has to be learned as part of , has the largest effect (factor 2.6), followed by the KS energies (1.8) and the density (1.5), while removing the overlap descriptors or the chemical element labels increases the error by 14% and 9%, respectively.
Next, using the static COHSEX dataset, we examine the effect of joint training on the accuracy of the predictions. When the autoencoder is first trained on wavefunction reconstruction alone and then frozen before training the transformer, the model still predicts the self energy matrix elements with high fidelity . However, the QP-level MAEs increase to 33 meV for and (Fig. 4c), compared with 29 meV for joint training (Fig. 2b). This difference does not arise from poor wavefunction reconstruction: the frozen autoencoder reconstructs the wavefunctions of test materials with a median fidelity of 0.86, slightly higher than the 0.84 of a jointly trained autoencoder (Fig. 4d). Joint training instead lets the self energy loss reshape the latent space toward features relevant to the prediction, at the cost of only a slight reduction in reconstruction fidelity.
To identify what joint training changes in the representation itself, we probe the compressed latent directly (Fig. 5). First, a linear ridge probe predicts the many-body targets more accurately from the jointly trained latent than from the separately trained one: and the QP correction have and 0.72, in the joint case compared with 0.61 and 0.51 in the separately trained case (Fig. 5a). To test whether joint training also produces a more informative representation of the underlying KS states, we apply the same linear probe to properties of the KS states themselves. These are also predicted more accurately from the joint latent: (0.80 vs. 0.67) and the magnitude of the momentum matrix element (0.35 vs. 0.22). Additionally, we find that the relevant information is more concentrated in the leading directions of the jointly trained representation. Figure 5b shows the of as a function of the number of leading principal components used by the probe. The two leading directions of the joint latent already carry as much information about as the entire separate latent, and the joint curve stays above the separate one as more components are included. Finally, joint training produces a clearer separation between valence and conduction states in the two leading principal components of the latents of each material (Figs. 5c,d).
Figure 5. Joint training reshapes the latent into a more physically structured code. a, Test-set ridge-probe for , , , and ; error bars are 95% bootstrap intervals over test materials. b, for vs. the number of leading principal components retained; the full latents reach 0.81 (joint) and 0.61 (separate). c,d, Latents of the states of a representative test monolayer projected onto their two leading principal components and colored by occupation, for joint (c) and separate (d) training. The valence-conduction silhouette score is 0.32 vs. 0.07 (test median 0.31 vs. 0.08; joint larger in 88% of test materials).
This is a physically relevant distinction because the self energy correction differs systematically between occupied and empty states. Together, these results show that the self energy objective reorganizes the wavefunction representation so that information relevant to the downstream task is more accessible and more compactly encoded than in a latent trained for reconstruction alone.
In summary, we have demonstrated the k-resolved prediction of the complex self energy matrix across chemically and structurally diverse 2D semiconductors. Our model achieves a new state-of-the-art for the cross-material prediction of quasiparticle energies in solids, and it is the first to do it with the full self energy matrix. The approach combines a phase-aware representation of the complex KS wavefunctions on a common physical-momentum grid with gauge augmentation and a factorized transformer that reflects the band and momentum structure of the self energy. Joint end-to-end training further allows the many-body objective itself to shape the wavefunction representation, producing a latent space in which information relevant to both the KS states and the self energy is more directly accessible. The predicted matrices retain their accuracy upon diagonalization and yield QP wavefunctions as well as energies,
Acknowledgments
opening a route to accelerating calculations that require the full basis, including self-consistent , perturbation theory for electron-phonon interactions [30], and nonequilibrium many-body dynamics. We show that the framework carries over from static COHSEX to dynamically screened GPP self energies without redesign and that pretraining on inexpensive static COHSEX labels improves the dynamical model with minimal additional training. More broadly, these results move machine learning of beyond scalar corrections toward learning the many-body operator itself, giving access to both quasiparticle energies and wavefunctions in real materials.
This work was primarily supported by the U.S. Department of Energy, Office of Science, Basic Energy Sciences under Award No. DE-SC0026655. Development of the BerkeleyGW code was supported by Center for Computational Study of Excited-State Phenomena in Energy Materials (C2SEPEM) at the Lawrence Berkeley National Laboratory, funded by the U.S. Department of Energy, Office of Science, Basic Energy Sciences, Materials Sciences and Engineering Division, under Contract No. DE-AC02-05CH11231. The calculations used resources of the National Energy Research Scientific Computing (NERSC), a Department of Energy, Office of Science User Facility, operated under contract number DE-AC02-05CH11231, under award numbers BES-ERCAP-0031507 and BES-ERCAP-0027380. An award of computer time was provided by the U.S. Department of Energy’s (DOE) Innovative and Novel Computational Impact on Theory and Experiment (INCITE) Program. This research used supporting resources at the Argonne and the Oak Ridge Leadership Computing Facilities. The Argonne Leadership Computing Facility at Argonne National Laboratory is supported by the Office of Science of the U.S. DOE under Contract No. DE-AC02-06CH11357. The Oak Ridge Leadership Computing Facility at the Oak Ridge National Laboratory is supported by the Office of Science of the U.S. DOE under Contract No. DE-AC05-00OR22725. This material is based upon research conducted by D.Y.Q. as an Alfred P. Sloan Research Fellow under Grant No. FG-2026-79515.
Data and code availability. The numerical data underlying the figures and the trained model checkpoints and inference code used to generate the results reported in this work will be deposited in the Materials Data Facility. Other data are available from the authors upon reasonable request.
References
Supplemental Information for Machine Learning the GW Self Energy Operator for Real Materials
Xian Xu; Bowen Hou; Chengyan Zhang; Diana Y. Qiu

Figure 1 (PDF p. 9). Errors and band structures of static COHSEX test materials. a, Distribution of the MAE for each of the 174 test materials for the model shown in Fig. 2 of the main text. The median MAE is 19 meV, and the mean MAE is 29 meV, with NaCNO having the largest MAE (174 meV). The test-set MAE for the smallest band gap on the grid is 59 meV for this model, 120 meV for the GPP model shown in Fig. 3b of the main text, and 81 meV for the finetuned model of Fig. 3d of the main text. b to g, Band structures for a range of test materials with different MAEs at the DFT level (black) calculated level (blue), and the predicted (GW-ML) level (orange dashed). g is the bandstructure of the material with the largest MAE in our dataset. As in Figs. 2c,d of the main text, the energies of the four highest valence and four lowest conduction bands are interpolated from the grid to the fixed path using the dual grid interpolation scheme implemented in BerkeleyGW [1]

Figure 2 (PDF p. 10). Off-diagonal self energy matrix elements. Same as Fig. 2a of the main text for the off-diagonal elements with only. The test set MAE is 5.0 meV per real or imaginary part.
Note 1: Datasets and computational parameters
The static COHSEX dataset contains 1745 2D semiconductors with structures obtained from C2DB [2, 3], with 2 to 14 atoms per unit cell and PBE [4] band gaps of 0.20 to 8.05 eV. The GPP dataset contains 433 hexagonal 2D semiconductors with 2 to 6 atoms per unit cell and PBE band gaps of 0.22 to 7.44 eV. For each dataset, 90% (10%) of the materials are randomly selected for training (test), resulting in 1571 (174) static COHSEX and 389 (44) GPP materials.
All DFT calculations are performed using Quantum ESPRESSO [5] with the PBE functional and scalar relativistic optimized norm-conserving Vanderbilt pseudopotentials [6], without spin polarization or spin-orbit coupling. The cutoffs of the wavefunctions and the charge density are 50 and 200 Ry, the cell height is 12 Å, and the grid is without symmetry reduction. All calculations are performed using BerkeleyGW [1] with slab Coulomb truncation [7]. The energy cutoff for the planewave components of the screened Coulomb interaction is 20 Ry for the static COHSEX dataset and (25 Ry for the generalized plasmon-pole model (GPP) [8]). Stochastic pseudobands [9] are used to accelerate the sums over empty bands, and the nonuniform neck subsampling scheme [10] is used to converge the screening at small . On 256 MPI tasks with two OpenMP threads each, the calculation takes on average 42 min for the static COHSEX dataset and 87 min for the GPP dataset. For the 411 materials contained in both datasets, it takes 10 min for static COHSEX and 88 min for GPP(1).
The target is the self energy matrix in the basis of the six highest valence and eight lowest conduction bands (resulting in a matrix of size ) for the static COHSEX dataset and of five valence and four conduction bands (resulting in a matrix of size ) for the GPP dataset. The static COHSEX self energy is Hermitian by construction. The GPP self energy depends on the energy at which it is evaluated, so its off-diagonal elements are evaluated at the Kohn-Sham (KS) energies of both states and symmetrized,
analogous to the procedure in quasiparticle self-consistent [11, 12].
Note 2: The P grid
The plane-wave coefficients are naturally stored on a reciprocal-lattice grid indexed by . Across different crystals, however, the same grid index corresponds to different physical momenta because the reciprocal-lattice vectors depend on the lattice geometry and lattice constants. To give the coefficient array a common physical meaning across materials, we instead represent each coefficient by its Cartesian momentum , which we refer to as the P grid. Each coefficient is placed at this physical momentum and distributed over nearby nodes of a fixed Cartesian momentum grid using a Gaussian kernel with a broadening of two thirds of the node spacing ( in plane and out of plane). We use nodes.
Because this resampling interpolates the original coefficients onto a fixed grid, we test how much information is lost in the transformation. We reconstruct the original coefficients from the P-grid representation by solving the inverse resampling problem using least squares with Tikhonov regularization. The reconstructed states have a median fidelity of 0.990, showing that the fixed-grid representation retains nearly all of the information in the original plane-wave coefficients.
The resulting representation is illustrated in Fig. 3. On the original G grid, states from materials with different lattice geometries or lattice constants occupy different index patterns even when their physical momentum content is similar. On the P grid, each node corresponds to the same physical momentum in every material. In particular, the primitive-cell and supercell representations of the same state in hexagonal boron nitride (hBN) map onto the same field on the P grid, demonstrating that the representation depends on physical momentum rather than the choice of reciprocal-lattice indexing.

Figure 3 (PDF p. 13). The same states on the G grid and on the P grid. Coefficients of one KS state at : a, the highest valence states of hexagonal boron nitride (hBN), which has a hexagonal lattice; b, the highest valence state of SnO, which has a square lattice; c, the sixth highest valence state of TaSI, which has a hexagonal lattice with a much larger lattice constant than hBN; d, the same state as in a from a supercell of hBN. Top, the G grid at . Middle, the same coefficients at their momenta , on the P grid (white lines). Bottom, the P grid after the Gaussian broadening.
Note 3: Input features
Besides the latent, each state carries its KS energy, its band indices, a radial encoding, and an element decomposition.

Figure 4 (PDF p. 14). State inputs. Two states of the test material SiS at : the lowest valence state v and the lowest conduction state c. a, Radial encoding: the weight collected in each of the 48 shells of , one bin per shell. v has its weight at smaller momenta than c. b, c, summed along z, with the Si (orange) and S (green) atoms. d, Element shares of v and c: c lives more on Si than v.

Figure 5 (PDF p. 14). Pair inputs for the states v and c of Fig. 4. a, Overlaps resolved in momentum for and , and the magnitude for , one bin per shell of . For , they give the momentum distribution of the state on the P grid and add up to one; the zigzag follows the number of nodes of the P grid in each shell. b, c, The ground state valence density and its gradient in the plane of the Si atoms (orange); the S atoms (green, open circles) lie above and below this plane.
The radial encoding describes how the weight of a state is distributed over the momentum . The weights on the G grid are collected in 48 smooth Gaussian shells, evenly spaced in (Fig. 4a), and a perceptron with two layers maps the 48 shell weights to 24 features. The positions and widths of the shells are trainable but stay close to these initial values.
The element decomposition describes on which atoms a state lives. Each point of the grid in real space is shared among the nearby atoms with Gaussian weights of width 1.2 bohr, and is summed with these weights for each atom and then over the atoms of each element (Fig. 4b to d). All the elements’ shares are mapped linearly to 16 features.
Each band pair carries the exchange-correlation matrix element , momentum-resolved overlap descriptors, and charge-density overlaps, which all transform like under Eq. (1) of the main text (Fig. 5). The overlap in momentum shell is defined as , the sum of over the nodes of the P grid in this shell. The 26 shells are evenly spaced in (Fig. 5a). Similarly, the charge density overlap is resolved over the regions of real space with given values of the ground state valence density (10 bins) and of its reduced gradient (6 bins) (Fig. 5b,c). The token is a linear map of the normalized , , these pair-dependent quantities, and five energy features: and, for both states, the energy relative to the valence band maximum and to the conduction band minimum of the window.
Note 4: Quasiparticle wavefunctions
Diagonalizing writes each QP state as a combination of the KS states , with overlaps . In the static COHSEX test set the QP states differ markedly from the KS states. The predicted agree with the calculated ones with and an MAE of 0.009 (Figure 6), compared with and 0.027 when the KS states are taken as the QP states.

Figure 6. Accuracy of the quasiparticle (QP) states. Absolute values of the overlaps between the KS states and the QP states of , predicted vs. calculated, for the 174 static COHSEX test materials, from MBFormer (orange) and with the KS states taken as the QP states (blue).
Note 5: Autoencoder architecture
Our autoencoder is deterministic and contains no variational step. We treat the inplane nodes of the P grid as an image and the 36 out-of-plane nodes as channels. Each encoder consists of three convolution blocks (Figure 7) with 96, 192, and 384 channels, which progressively reduce the inplane grid to , followed by a linear layer that maps the result to the latent space. The amplitude and phase encoders produce the latents and , each with 140 dimensions. The decoders mirror this structure with transposed convolutions. To keep the reconstructed amplitude nonnegative, the amplitude decoder ends with a softplus activation, while the phase decoder outputs two numbers per node that are normalized onto the unit circle to give . Both branches are trained with the mean squared error of the reconstructed amplitudes and of the real and imaginary parts of the coefficients, multiplied by a weight .

Figure 7. Convolution block of the autoencoder. Each encoder and decoder has three such blocks, from the inplane nodes of the P grid to , and a linear map to connect the latent. The nodes enter as channels.
Transformer architecture and training details
We represent each band pair with at each k point as one token, since the remaining elements follow from Hermiticity. This gives 105 tokens per k point for static COHSEX and 45 for GPP. Each token is built by concatenating the state vectors and with the pair features described in Note 3, and is then projected linearly to a dimension of 640. The transformer contains three factorized layers. Within each layer, attention is first applied along the k axis and then along the band-pair axis, after which a triangle update refines the tokens. All attention blocks use 8 heads with root mean square normalization [13] and a SwiGLU feedforward network [14] with a hidden dimension of 2048. The triangle update couples the band pairs at the same k point through a sum over an intermediate band ,
where and are gated linear projections of the tokens with 64 channels (the product is taken channel by channel), LN denotes layer normalization, and is a linear layer. This form is motivated by the band sum in of Eq. (2) of the main text. Finally, a small output head maps each token to the real and imaginary parts of . For we use the mean absolute error of the complex matrix elements of , and the total loss is averaged over the materials in each training step. During training, we multiply every state by a random phase at each step and corotate , , and both sets of overlaps according to Eq. (1) of the main text. We use this gauge augmentation because imposing exact equivariance throughout the complex MBFormer architecture would require restricting every transformer operation. Our previous work showed that phase augmentation can impose an approximate equivariance even without an explicitly equivariant architecture [15]. We optimize all weight matrices of the linear layers with Muon [16] and the remaining parameters with Adam [17]. No weight decay is used. The learning rate follows a cosine decay from to over 200 epochs. Dropout with a rate of 0.1 is applied to the attention weights, the attention outputs, and the hidden layer of the SwiGLU network. All reported results use the model after the final training epoch.
References
- [1]Jack Deslippe, Georgy Samsonidze, David A. Strubbe, Manish Jain, Marvin L. Cohen, and Steven G. Louie, “BerkeleyGW: A massively parallel computer package for the calculation of the quasiparticle and optical properties of materials and nanostructures,” Comput. Phys. Commun. 183, 1269–1289 (2012).DOI
- [2]Sten Haastrup, Mikkel Strange, Mohnish Pandey, Thorsten Deilmann, Per S. Schmidt, Nicki F. Hinsche, Morten N. Gjerding, et al., “The Computational 2D Materials Database: high-throughput modeling and discovery of atomically thin crystals,” 2D Mater. 5, 042002 (2018).DOI
- [3]Morten Niklas Gjerding, Alireza Taghizadeh, Asbjørn Rasmussen, Sajid Ali, Fabian Bertoldo, Thorsten Deilmann, et al., “Recent progress of the Computational 2D Materials Database (C2DB),” 2D Mater. 8, 044002 (2021).
- [4]John P. Perdew, Kieron Burke, and Matthias Ernzerhof, “Generalized gradient approximation made simple,” Phys. Rev. Lett. 77, 3865 (1996).pubmed.ncbi.nlm.nih.gov/10062328
- [5]Paolo Giannozzi et al., “QUANTUM ESPRESSO: a modular and open-source software project for quantum simulations of materials,” J. Phys.: Condens. Matter 21, 395502 (2009).DOI
- [6]D. R. Hamann, “Optimized norm-conserving Vanderbilt pseudopotentials,” Phys. Rev. B 88, 085117 (2013).DOI
- [7]Sohrab Ismail-Beigi, “Truncation of periodic image interactions for confined systems,” Phys. Rev. B 73, 233103 (2006).
- [8]Mark S. Hybertsen and Steven G. Louie, “Electron correlation in semiconductors and insulators: Band gaps and quasiparticle energies,” Phys. Rev. B 34, 5390–5413 (1986).pubmed.ncbi.nlm.nih.gov/9940372
- [9]Aaron R. Altman, Sudipta Kundu, and Felipe H. da Jornada, “Mixed stochastic-deterministic approach for many-body perturbation theory calculations,” Phys. Rev. Lett. 132, 086401 (2024).DOI
- [10]Felipe H. da Jornada, Diana Y. Qiu, and Steven G. Louie, “Nonuniform sampling schemes of the brillouin zone for many-electron perturbation-theory calculations in reduced dimensionality,” Phys. Rev. B 95, 035109 (2017).DOI
- [11]Sergey V. Faleev, Mark van Schilfgaarde, and Takao Kotani, “All-electron self-consistent GW approximation: Application to Si, MnO, and NiO,” Phys. Rev. Lett. 93, 126406 (2004).pubmed.ncbi.nlm.nih.gov/15447292
- [12]Mark van Schilfgaarde, Takao Kotani, and Sergey Faleev, “Quasiparticle self-consistent GW theory,” Phys. Rev. Lett. 96, 226402 (2006).
- [13]Biao Zhang and Rico Sennrich, “Root mean square layer normalization,” (2019), arXiv:1910.07467.arxiv.org/abs/1910.07467
- [14]Noam Shazeer, “GLU variants improve transformer,” (2020), arXiv:2002.05202.arxiv.org/abs/2002.05202
- [15]Chengyan Zhang, Xian Xu, Bowen Hou, and Diana Y. Qiu, “U(1) Gauge-Equivariant Representation Learning of Bloch States,” arXiv preprint arXiv:2609.30845 (2026), arXiv:2609.30845 [physics.comp-ph].arxiv.org/abs/2609.30845
- [16]Keller Jordan, Yuchen Jin, Vlado Boza, Jiacheng You, Franz Cesista, Laker Newhouse, and Jeremy Bernstein, “Muon: An optimizer for hidden layers in neural networks,” https://kellerjordan.github.io/posts/muon/ (2024).
- [17]Diederik P. Kingma and Jimmy Ba, “Adam: A method for stochastic optimization,” (2014), arXiv:1412.6980.arxiv.org/abs/1412.6980


