WorksPhysics and Astronomy
Hot-Ham: an accurate and efficient E(3)-equivariant machine-learning electronic structures calculation framework
Abstract
The combinations of machine learning with ab initio methods have attracted much attention for their potential to resolve the accuracy-efficiency dilemma and facilitate calculations for large-scale systems. Recently, equivariant message passing neural networks (MPNNs) that explicitly incorporate symmetry constraints have demonstrated promise for interatomic potential and density functional theory (DFT) Hamiltonian predictions. However, the high-order tensors used to represent node and edge information are coupled through the Clebsch-Gordan tensor product (CGTP), leading to steep increases in computational complexity and seriously hindering the performance of equivariant MPNNs. Here, we develop High-order Tensor machine-learning Hamiltonian (Hot-Ham), an E(3) equivariant MPNN framework that combines two advanced technologies local coordinate transformation and Gaunt tensor product (GTP) to efficiently model DFT Hamiltonians. These two innovations significantly reduce the complexity of tensor products from O(L^6) to O(L^3) or O(L^2 log^2 L) for the max tensor order L, and enhance the performance of MPNNs. Benchmarks on several public datasets demonstrate its state-of-the-art accuracy with relatively few parameters, and the applications to multilayer twisted moiré systems, heterostructures and allotropes showcase its generalization ability and high efficiency. Our Hot-Ham method provides a new perspective for developing efficient equivariant neural networks and would be a promising approach for investigating the electronic properties of large-scale materials systems.
Introduction
Density Functional Theory (DFT) has become one of the most popular tools for studying the structure, properties, and reactions of materials at the atomic level. While DFT provides a quantum mechanical framework for electronic structure calculations, solving the Kohn-Sham equations requires significant computational resources, restricting its application to systems with a limited number of atoms. In contrast, empirical tight-binding such as Slater-Koster method[1] can typically be evaluated orders of magnitude faster than first-principles methods due to their simple mathematical form, hence enabling large-scale electronic calculations. However, empirical tight-binding often comes at the cost of reduced accuracy compared to DFT methods and limited transferability across different systems.
Emerging machine learning (ML) techniques have been increasingly applied to model electronic structures[2, 3, 4, 5, 6, 8, 9, 10, 11, 12, 13, 14] [7], showing great promise in providing a way to address this accuracy-efficiency dilemma. Among the various approaches, graph neutral networks[15, 16, 17, 18] (GNNs) have become the dominant choice to characterize the graph structures. Schütt et al.[5] proposed a GNN model SchNorb to predict Hamiltonian as a linear combination of spherical harmonics, however, it does not incorporate the symmetry as a priori knowledge. An important property of Hamiltonian is its equivariant transformation under the Euclidean group on the three-dimension space ( group), which contains translations, rotations, and inversion. Models that explicitly incorporate equivariance constraints are more data-efficient and generalizable, making them promising for complex prediction tasks. PhiSNet[6], QHNET7, and ACEHamiltonians[8] achieve physical correctness under translation and rotation transformations, but all of them would be struggling with periodic materials due to a lack of parity symmetry[12]. The N-center representation equivariant framework proposed by Nigam[9] achieved E(3)-equivariant Hamiltonian using Gaussian process regression (GPR). However, GPR usually has limited generalization ability and is more computationally expensive compared with GNNs such as message-passing neural networks[19] (MPNNs). Combining E(3) equivariance constraints with MPNNs, Gong et al[11] and Zhong et al[12] proposed DeepH-E3 and HamGNN respectively and showcased excellent accuracy in Hamiltonian predictions across diverse materials.
Despite the remarkable success of E(3)-equivariant neural networks (E(3)-ENNs), these models face significant limitations due to the high computational complexity of equivariant operations that employ Clebsch-Gordan tensor product (CGTP) of irreducible representations (irreps). The CGTP couples tensors with orders up to to produce new features, leading to a computational complexity of for the full operation. This steeply increasing complexity hinders the application of E(3)-ENNs for predicting physical quantities that require high-order tensors. To address this issue, Passaro and Zitnick[20] introduced local coordinate transformation, simplifying CGTP to SO(2) convolution thus reducing the complexity to . Recent E(3)-equivariant Hamiltonian frameworks DeepH-2[13] and DeePTB[14] have adopted this approach and show state-of-the-art accuracy in Hamiltonian predictions. However, SO(2) convolution is only used for equivariant convolutions, a special case of CGTPs. As there are no specific local coordinate transformations for arbitrary equivariant features, this method is not suitable for equivariant feature interactions (e.g. HamGNN) and equivariant many-body interactions (e.g. MACE[21]). Xin et al[22] proposed another fast and accurate approach for spherical harmonics products, and Luo et al[23] also proposed a similar tensor product named Gaunt tensor products (GTPs), which are more efficient and general as compared with SO(2) convolution. Nonetheless, since GTPs inherently exclude antisymmetric parity, they are hardly utilized in E(3)-ENNs, especially for the Hamiltonian predictions where antisymmetric tensors are indispensable.
In this work, we propose High-order Tensor machine learning Hamiltonian (Hot-Ham), an E(3)-equivariant message passing neural network framework designed for efficient Hamiltonian representation. By combining local coordinate transformations and GTPs, our framework not only effectively leverages their strengths but also compensates for their limitations, reducing tensor products from to or and establishing an efficient E(3)-ENN for electronic structure calculation.
We benchmark Hot-Ham on several public datasets and compare its performance with other Hamiltonian models. Our results show that Hot-Ham achieves superior performance, while maintaining significantly smaller parameter size (within 2M) compared to most models. To demonstrate the generalization ability and efficiency, we train Hot-Ham on non-twisted MoS, graphene/hexagonal boron nitride (h-BN) heterostructures and phosphorus allotropes, and test it on various structures out of datasets. The accuracy shows that our model is able to accurately predict electronic structure even though the crystal structure is not included in the training set, with computational costs much lower than DFT. The accuracy, generalization, and efficiency of Hot-Ham significantly mitigate the accuracy-efficiency trade-off dilemma of DFT, opening up new possibilities for large-scale electronic structure calculations and novel functional material discovery.
Results
Equivariant Hamiltonian
The properties of physical systems generally exhibit symmetries under transformations of a set of atoms, leading to the concept of equivariance naturally. Formally, a function is equivariant for and with respect to group if for , where and are group representations on and . ENNs are guaranteed to preserve equivariance under a change of coordinates because they are composed of equivariant functions.
In GNNs, the structure is represented as a graph, where each node corresponds to an atom, and each edge represents a connection between atoms within a cutoff radius . Given a set of atoms, the two key problems of designing ENNs are how to encode the atom positions and chemical elements as equivariant features for nodes and edges, and how to relate these features to equivariant Hamiltonians.
Traditionally, DFT Hamiltonians can be calculated on various bases, such as plane-wave basis[24, 25, 26], linear combination of atomic orbitals (LCAO) basis[27, 28, 29, 30], and so on. Specifically, the LCAO basis has the form , where is the atom index, and are radial function and spherical harmonics respectively with projectors’ multiplicity , angular momentum quantum number and magnetic quantum number . On the one hand, this form restricts the interaction calculations within a cutoff distance, which is consistent with the finite receptive field of ENNs. On the other hand, spherical harmonics, which is also referred to as spherical tensors, satisfy rotation and inversion equivariance ( group), and have been widely utilized to represent equivariant features inside ENNs[21, 31, 32, 33]. Therefore, among various kinds of bases, LCAO-basis Hamiltonians are suitable learning targets for ML Hamiltonian model. According to Wigner-Eckart theorem, the LCAO-basis Hamiltonian element between atoms can be decomposed into several learnable spherical tensors :
Here is the channel index that is determined by the and of two atomic orbitals, and is the parity index. And is the Clebsch-Gordan coefficient. The atomic orbitals basis for th atom has a parity of . So the parities of , and thus , are determined by the two atomic orbitals basis as . represents equivariant feature of atom for onsite block if , or edge for hopping block if . Hence, the aim of ENNs is to predict high-order tensor features that comprise a direct sum of irreps of the for all nodes and edges.
Chemical elements and interatomic distance are invariant physical quantities and are used to construct 0-order tensors, i.e., scalars. The relative positions of atoms are mapped into spherical harmonics to serve as high-order tensors, which automatically ensure the invariance of ENNs under translation. After a series of equivariant operations, the final equivariant features are used to reconstruct Hamiltonians in a block-wise manner through (1). It should be noted that the equivariant operations have to be carefully designed, not only to preserve equivariance but also to achieve high efficiency for the neural networks. One of the main equivariant operations is the E3linear transformations. The linear operations mix the features between channels in the same irreps. For the common feature representation , which represents the direct sum of spherical tensors up to order , the linear transformations can be conducted in a complexity of . The other main equivariant operations are tensor products. These operations couple tensors and in different irreps through the Clebsch-Gordan coefficient, and yield new tensors that is E(3)-equivariant:
However, when applying (2) to features like , the complete tensor products will lead to an complexity, which seriously hinders the application of ENNs to Hamiltonian predictions, where high-order tensors are essential to represent atom orbitals and their interactions, thus demanding new approaches to achieve efficient tensor products
Efficient tensor product operations
SO(2) convolution is one of the approaches to accelerating the tensor product operations. In the field of ENNs, filters can be defined as spherical harmonic acting on relative positions . For convolution operation, i.e. tensor product between feature and filter , the complexity can be reduced through local coordinate transformation. By rotating the embeddings’ primary axis to align with the edge vectors, the filter becomes sparse: if . This eliminates the summation over , reducing (2) to a 2D matrix multiplication. By rearranging the Clebsch-Gordan coefficients and relevant weights, the tensor products between rotated features like and filters with orders up to are simplified to 2D matrix multiplications, which accounts for cost. Additionally, an operation is needed for each irrep among to rotate align with the edge vector or rotate back, accounting for two extra cost.
GTP is another more efficient, and general approach. Gaunt coefficients[34] are defined as the integrals of three spherical harmonics products, and are related to Clebsch-Gordan coefficients via constant factors that are independent on magnetic quantum number: . Luo et al[23] propose new perspectives that GTPs calculate the coefficients of spherical functions based on spherical harmonics:
for which is a square-integrable spherical function, and appropriate base transformations will be helpful to parallel these operations. An illustration is presented in Figure 1a. By changing the spherical harmonics bases into 2D Fourier bases, spherical function will be simplified into 2D convolution, which can be accelerated via 2D FFT in . Finally, the coefficients of are converted back to spherical harmonic representations. These base transformations account for cost. And we call it GTP(2D-FB) in this paper to distinguish it from the method GTP(sphere-grid) proposed by Xie et al.[35] through spherical convolution, as shown in Figure 1b. Features are first projected on a sphere grid through an inverse FFT. Following an element-wise producting, a FFT is performed to convert back to spherical harmonics. The two FFT accounts for an complexity. Therefore, GTP(sphere-grid) becomes theoretically the fastest method among the four kinds of tensor products. Similarly, the coefficients of both GTP methods can be further sparse via local coordinate transformation in the case of convolution. However, it should be noted that there is a difference between GTP and CGTP, because vanish when non-negative integer , thus leading to the lack of antisymmetric tensors in GTP. Nonetheless, GTPs still adhere to E(3) equivariance requirements.
Figure 1. Illustration of Gaunt tensor product (GTP) method and comparison of tensor product efficiency. a GTP (2D-FB) method. Spherical harmonics features are transformed into 2D Fourier bases representation. After FFT convolution, they are transformed back into spherical harmonic representation. b GTP (sphere-grid) method. Spherical harmonics features are projected into sphere grid via inverse FFT, followed by element-wise product, and finally converted back to spherical harmonic representation. c Comparison of efficiency for CGTP (implemented by e3nn, labeled as E3TP), SO(2) convolution, GTP (2D-FB), and GTP (sphere-grid).
We conduct experiments on the convolution operations using these four tensor products to demonstrate their efficiency, as shown in Figure 1c. The origin CGTP implemented by the e3nn package is unexpectedly the most time-consuming, while GTPs, especially GTP(sphere-grid), achieve the highest efficiency. These results, as well as the fact that GTP is suitable for various tensor products not only for convolution, indicate that GTP is a promising operation to achieve high-efficiency equivariant MPNNs.
Equivariant message passing neural network
The Hot-Ham model architecture is illustrated in Figure 2. Atomic number , interatomic distance and direction information are embedded to generate initial node features and edge features . Features are structured as a direct sum of symmetric tensors and antisymmetric tensors . Since only symmetric tensors can be generated by GTP, we need to introduce antisymmetric tensors at least once through CGTP. Given that including antisymmetric tensors in intermediate layers would result in about double computational costs, we choose to perform CGTP in the last layer through SO(2) convolution. Node and edge symmetric tensor features are encoded and aggregated to update iteratively in each GTP convolution layer, then are extended to include antisymmetric tensor features in the last SO(2) convolution layer. Finally, these features are transformed into Hamiltonian matrix block through Wigner-Eckart theorem in the Readout layer.
Figure 2. The architecture of Hot-Ham. a The overall architecture of Hot-Ham. Atom species and relative positions are embedded into initial node and edge features. After several iterative updates in convolution layers, the final node and edge features are used to construct Hamiltonians. b the embedding layer. The scalar information including atom species , and interatomic distance is embedded through the ScalarEmd block into weight , and then is used to weight the spherical harmonics functions . c The convolution layer. Node and edge features are concatenated to serve as messages. After the gate (activation function), messages are used to generate new node and edge features simultaneously.
Feature initialization
As shown in Figure 2b, the edge features are defined as weighted spherical harmonics through a linear transformation, and node features are the mean of edge features:
Here are neighbor atoms of atom . are weights that contain information about the species of two connected atoms and their interatomic distance. Specifically, and are mapped by one-hot encoding and then learnable multi-layer perceptron (MLP) into two species vectors, and is expanded by radial bases functions (RBF) such as Gaussian, Bessel, and Chebyshev basis. Subsequently, these species vectors and distance expansions are concatenated and transformed by MLP into .
Convolution
In convolution layers, local environment information including connected node features and edge feature are used to encode edge messages through the tensor products between the weighted concatenations of features and weighted filters :
Here and are distance-dependent weight functions, and denotes concatenation. To introduce nonlinearity without breaking E(3) equivariance, a gate (activation function) must be carefully designed. In Hot-Ham, we use a gate like ref. 36, where scalar and pseudoscalar components are nonlinear by SiLU and tanh respectively, while others are scaled by learnable scalars that are nonlinear by sigmoid. Messages are then used to update edge and node features simultaneously. New edge features are updated with the residual net like architecture[32]:
For node features, messages are first propagated to each node along edges and then aggregated by mean function. Similarly, the old node features are combined with the messages to form the new node features:
Layer normalization
Normalization is a crucial technique in deep learning models used to stabilize and enhance training. Layer normalization is one of the most common methods applied in ENNs due to its independence of batch size, which is typically small in Hamiltonian model training. To preserve equivariance, feature components belonging to different irreps are normalized separately. For scalar features , we use a layer normalization like ref. [11, 31]:
where represents atom index for or edge index for , and are mean and standard deviation of features with degree . is a small number used to avoid numerical instability when is close to zero. and are learnable parameters for affine transformation. While for pseudoscalar or tensor with , we simply subtract the mean:
Readout
The Hamiltonian matrix block between atom with orbital angular momentum and atom with orbital angular momentum takes the form of a direct product: , a matrix with parity . After several layers of convolution iteration, the final features have to be passed through a linear layer to rearrange into a form , and then transform to through (1). Incorporating the prior Hermitian property for Hamiltonian will be beneficial to reducing loss. To enforce the Hamiltonian to be Hermitian, we define the final Hamiltonian matrix block as: .
Performance of Hot-Ham
To benchmark the accuracy of our models, we apply Hot-Ham to public datasets[11], including monolayer graphene, monolayer MoS, and bilayer graphene, as reported in ref. [10]. For each dataset, we train two models using GTP(2D-FB) and GTP(sphere-grid) respectively in the GTP convolution layers and compare the results with other models. To further demonstrate the generalization and efficiency of our approach, we also apply Hot-Ham to study the electronic structure of multilayer twisted MoS, graphene/h-BN heterostructures, and phosphorus allotropes.
Benchmark accuracy
We compare the mean absolute error (MAE) of Hamiltonian matrix elements and the number of training parameters for Hot-Ham with other models, including DeepH-E3, HamGNN, DeepH-2, and DeePTB. The results for the monolayer graphene and monolayer MoS datasets, which are commonly used to evaluate the accuracy of -equivariant Hamiltonian frameworks, are shown in Table 1. The results for the bilayer graphene dataset are provided in the Supplementary information. Both GTP implementations share the same tensor product paths, so they are expected to achieve comparable accuracy. This is confirmed by the results of Hot-Ham. Our models demonstrate state-of-the-art accuracy across all systems, achieving 0.08, 0.12 and 0.15 meV MAEs for monolayer graphene, monolayer MoS and bilayer graphene respectively. Notably, compared to the models achieving the best performance in each system, our models require significantly fewer parameters.
Table 1. Comparison of the results by DeepH-E3, HamGNN, DeepH-2, DeePTB, and Hot-Ham on Hamiltonians of monolayer graphene and monolayer MoS₂. (MAEs are in unit of meV. Parentheses indicate the number of parameters. For each model we only show their best performance found in previous work.)
Multilayer twisted MoS
2D multilayer materials have garnered significant attention due to their unique electronic, optical, and mechanical properties, which arise from their tunable interlayer coupling under different twist angles and complex stacking configurations. The twist angles between layers can lead to a range of fascinating physical phenomena, such as unconventional superconductivity and nonlinear optical effects[37]. Additionally, the stacking configuration of each adjacent layer can be independently modulated, as the weak interlayer coupling, dominated by van der Waals interactions, allows for considerable flexibility[38]. However, fabricating multilayer materials with precise controlling twist angles is not an easy task for experiments, and performing large-scale electronic structure calculations is impracticable for DFT. In the following, we demonstrate that Hot-Ham offers a general and efficient approach to studying electronic structures by investigating multilayer twisted MoS.
Through training on bilayer and trilayer non-twisted MoS, our models achieve an accuracy of 0.20 meV. To demonstrate the generalization ability, We applied the trained model to multilayer twisted MoS systems, including bilayer, trilayer, and double bilayer structures. For trilayer MoS, we rotated the second layer while keeping the third layer aligned with the first. In the case of double bilayer MoS, we rotated the upper two layers, as schematized with one of the rotation angles in Figure 3c. The MAEs are shown in Figure 3a. Despite the training set not containing any four-layer structures, our model still predicts Hamiltonians with an accuracy comparable to those of trilayer. The MAEs reach relatively large values at twist angle, the maximum angle typically used for modeling twist systems, but their values of 1.2-1.4meV are still sufficient to accurately describe the electronic structure. The MAEs decrease gradually as the twist angle decreases, approaching the values of non-twisted systems. This can be attributed to the smaller deviation from the training set as the twist angle decreases. In addition to the powerful generalization ability, our model also exhibits high efficiency. The wall time of DFT calculations and our model’s inference are displayed in Figure 3b. The CPU and GPU calculations are performed in one node equipped with Intel Xeon Gold 6140 processors and one NVIDIA GeForce RTX 4090 respectively. The inference time of CPU and GPU are close. The CPU inference further expands the scale to , in a time cost that increases linearly with a slope less than DFT calculations when . We can easily derive band structures by diagonalizing the predicted Hamiltonians. Figure 3d illustrates the band structure near the Fermi level for double bilayer MoS with a twisted angle of . The results of our model exhibit a high degree of agreement with that obtained from DFT calculations.
Figure 3. Application of Hot-Ham to multilayer MoS₂. a Hamiltonian MAE for bilayer, trilayer, and double bilayer under different twist angles. b The wall time for DFT calculations and our model’s inference at CPU and GPU. c structure of double bilayer MoS₂ containing 1524 atoms with twist angle . d band structure for the structure in c predicted by DFT (OpenMX) and Hot-Ham.
Heterostructures
By stacking 2D materials with different properties in an incommensurate manner, heterostructures exhibit novel physical effects that are significantly distinct from the intrinsic properties of individual materials. This provides a new degree of freedom to manipulate properties and band structures, thereby enriching the properties of 2D material systems. Although the lattice mismatches introduced by incommensurability and the possible twist angles make the heterostructures rather complex and rich, our model, trained on graphene/h-BN systems for example, will show its capability to accurately describe the electronic structures.
Similarly, our model is trained on the non-twisted graphene/h-BN but with a mismatch of 1.79%[39, 40] corresponding to the difference of lattice constants between graphene and h-BN. These structures are generated from three stacking configurations[41], one of which is and another two are Bernal arrangements ( and ). Figure 4b demonstrates the good agreement on band structure for a Bernal arrangement configuration (Figure 4a). To test the generalization ability of Hot-Ham on unseen configurations, we generate 100 heterostructures with various mismatch and twisted angles within 500 atoms by VASPKIT package[42] for band structure calculations. As shown in Figure 4c, the predicted band eigenvalues from -1.5 eV to 1.5 eV for all the 100 structures achieve 5.46 meV MAE, while the MAE for Hamiltonians is only 0.49 meV. This accuracy is sufficient to capture significant features, such as the relatively large band gap induced by the inversion symmetry breaking, as shown in the insert of Figure 4b. Finally, we test our model on a larger system that contains 1022 atoms (Figure 4d). The predicted band structure (Figure 4e) and density of state (DOS, Figure 4f) agree well with results obtained from OpenMX. These again indicate that Hot-Ham can serve as a robust and powerful approach to exploring the electronic structures of a wide variety of configurations, even though trained on a limited dataset.
Figure 4. a supercell cell of a Bernal arrangement graphene/h-BN heterostructure. b the band structure of a. c comparison of band eigenvalues within eV for the 100 heterostructures with various mismatch and twisted angles. The MAE of their Hamiltonians is also displayed. d structure of graphene/h-BN containing 1022 atoms (518 C atoms, 252 B/N atoms) with mismatch=0.4% and twist angle . e and f are band structure and DOS for structure d predicted by DFT (OpenMX) and Hot-Ham.
Phosphorus allotropes
In addition to 2D materials, we also evaluated Hot-Ham’s performance on bulk systems using phosphorus allotropes as an example. We have collected 10 allotropes from Materials Project[43], which contains 2D and bulk systems. Among these structures, the violet phosphorus that has been experimentally synthesized recently is found to be the most stable allotrope at ambient pressure[44] and is thought as a promising semiconducting material for photonic and electronic applications. Therefore, we use the violet phosphorus to demonstrate Hot-Ham’s generalization ability, while the remaining 9 allotropes are used for training.
. Our model achieves an accuracy of 0.68meV MAE for the Hamiltonian matrix on the test set. Then we verify the accuracy on violet phosphorus. The violet phosphorus shown in Figure 5a has 84 atoms in unit cell, featuring tubular strands arranged perpendicularly. The MAE of Hamiltonian matrix is only 0.89meV, enabling our model to accurately reproduce the band structure in Figure 5b. This example demonstrates that Hot-Ham can effectively represent features of atoms and bonds in diverse chemical environments, showcasing its potential for applications in high-throughput material discovery, such as crystal structure searches[45] and functional materials predictions.
Figure 5. Predicted electronic structure on phosphorus allotropes. a crystal structure of violet phosphorus. b band structure calculated by DFT (OpenMX) and Hot-Ham for the structure that is not in the training dataset.
Discussion
In this work, we develop Hot-Ham, an -equivariant message passing neural network that combines local coordinate transformation and GTP to efficiently model DFT Hamiltonians from material structures. Through a local coordinate transformation, convolution significantly reduces the CGTP computational cost from to for the convolution operations. While GTP provides a more efficient and general approach to implement tensor products, either through convolving in 2D Fourier bases with or by spherical convolution with , however, at the expense of the absence of antisymmetric tensors. Our method not only retains the advantage of high efficiency of GTP, but also compensates for the antisymmetric tensors through convolution, achieving an accurate, generalizable, and efficient -equivariant framework for electronic structure calculations for large systems. The comparison with other -ENN models demonstrate Hot-Ham’s state-of-the-art accuracy with a significantly smaller parameter size compared to most models. And the experiments on , heterostructures, and phosphorus allotropes show Hot-Ham’s good generalization ability to structures out of training sets. In the future, it should be very valuable to expand the application scope of Hot-Ham to overcome challenges faced by traditional ab initio methods. For example, one can derive forces from predicted Hamiltonians, allowing for the explicit incorporation of electronic effects in molecular dynamics simulations. In addition, by utilizing automatic differentiation techniques, one would be able to investigate the electron-phonon interactions for large-scale systems. Furthermore, the prediction of orthogonal-base Hamiltonians to interface with the linear scaling quantum transport methodologies[46, 47] is also a possible direction. In summary, our Hot-Ham method is a promising framework with great potential for large-scale electronic calculations, which is fundamentally important for designing electronic devices.
Materials and Method
Dataset
The Hamiltonians and overlap matrixes are all calculated by OpenMX using Perdew-Burke-Ernzerhof[48] (PBE) functional and normconserving pseudopotential. The PAO employed for B, C, N, P, S, Mo are B7.0-s2p2d1, C6.0-s2p2d1, N6.0-s2p2d1, P7.0-s2p2d1, S7.0-s2p2d1, Mo7.0-s3p2d2 respectively. We use supercell and 6.50Å interlayer spacing for multilayer MoS, supercell and 3.22Å interlayer spacing for graphene/h-BN, and we enlarge the phosphorus allotropes to obtain supercells with 32 to 64 atoms. In addition to applying 300K random perturbations by Phonopy package, we also performed random interlayer slides for multilayer MoS and graphene/h-BN up to their lattice constants. For each system, the dataset is splitted for training, testing, and validation by 3:1:1. For structures used for band structure predictions, the twisted multilayer MoS were modeled by ASE package[49]. The graphene/h-BN heterostructures with no more than 500 atoms generated by the VASPKIT package are all within 1.79% mismatch, and the 1022 atoms twisted heterostructure was also modeled by the ASE package.
Details of neural network and training
In our model, we use the Chebyshev basis to expand the interatomic distance :
Where is the cutoff distance, is the th order Chebyshev polynomial of the first kind, and is the cutoff function:
The features passed to gate have the form:
Then it is nonlinear as:
The target of the neural network is to minimize the following loss function:
Where is the number of structures, is the dimension of th Hamiltonian, and is the total number of matrix elements. Parameters were optimized by the AdamW[50, 51] algorithm with an initial learning rate of 0.02. To accelerate convergence, we use the ReduceLROnPlateau method to schedule the learning rate: the learning rate will be reduced by a factor of 0.9 if accuracy is not improved within 50 epochs. We trained 4000 epochs in graphene/h-BN, and 3000 epochs in the remaining systems. All models were trained on one NVIDIA 4090 GPU. More hyperparameters are provided in Supplementary information.
Acknowledgments
We gratefully acknowledge the financial support from the National Natural Science Foundation of China (grant number. 12125404, T2495231, 123B2049), the Basic Research Program of Jiangsu (Grant BK20233001, BK20241253), the Jiangsu Funding Program for Excellent Postdoctoral Talent (2024ZB002, 2024ZB075), the Postdoctoral Fellowship Program of CPSF (Grant GZC20240695), the AI & AI for Science program of Nanjing University, and the Fundamental Research Funds for the Central Universities. The calculations were carried out using supercomputers at the High Performance Computing Center of Collaborative Innovation Center of Advanced Microstructures, the high-performance supercomputing center of Nanjing University.
Figures and table
References
- [1]Slater, J. C. & Koster, G. F. Simplified LCAO Method for the Periodic Potential Problem. Phys. Rev. 94, 1498–1524 (1954).
- [2]Wang, Z. et al. Machine learning method for tight-binding Hamiltonian parameterization from ab-initio band structure. npj Comput Mater 7, 11 (2021).DOI
- [3]Gu, Q., Zhang, L. & Feng, J. Neural network representation of electronic structure from ab initio molecular dynamics. Science Bulletin 67, 29–37 (2022).
- [4]Gu, Q. et al. Deep learning tight-binding approach for large-scale electronic simulations at finite temperatures with ab initio accuracy. Nat Commun 15, 6772 (2024).DOI
- [5]Schütt, K. T., Gastegger, M., Tkatchenko, A., Müller, K.-R. & Maurer, R. J. Unifying machine learning and quantum chemistry with a deep neural network for molecular wavefunctions. Nat Commun 10, 5024 (2019).
- [6]Unke, O. T. & Bogoieski, M. SE(3)-equivariant prediction of molecular wavefunctions and electronic densities. 7. Yu, H., Xu, Z., Qian, X., Qian, X. & Ji, S. Efficient and Equivariant Graph Networks for Predicting Quantum Hamiltonian. in Proceedings of the 40th International Conference on Machine Learning 40412–40424 (PMLR, 2023).
- [8]Zhang, L. et al. Equivariant analytical mapping of first principles Hamiltonians to accurate and transferable materials models. npj Comput Mater 8, 158 (2022).DOI
- [9]Nigam, J., Willatt, M. J. & Ceriotti, M. Equivariant representations for molecular Hamiltonians and N -center atomic-scale properties. The Journal of Chemical Physics 156, 014115 (2022).DOI
- [10]Li, H. et al. Deep-Learning Density Functional Theory Hamiltonian for Efficient ab initio Electronic-Structure Calculation. Nat Comput Sci 2, 367–377 (2022).
- [11]Gong, X. et al. General framework for E(3)-equivariant neural network representation of density functional theory Hamiltonian. Nat Commun 14, 2848 (2023).
- [12]Zhong, Y., Yu, H., Su, M., Gong, X. & Xiang, H. Transferable equivariant graph neural networks for the Hamiltonians of molecules and solids. npj Comput Mater 9, 182 (2023).
- [13]Wang, Y. et al. DeepH-2: Enhancing deep-learning electronic structure via an equivariant local-coordinate transformer. Preprint at http://arxiv.org/abs/2401.17015 (2024).
- [14]Zhouyin, Z., Gan, Z., Pandey, S. K., Zhang, L. & Gu, Q. Learning local equivariant representations for quantum operators. Preprint at http://arxiv.org/abs/2407.06053 (2024).
- [15]Scarselli, F., Gori, M., Tsoi, A. C., Hagenbuchner, M. & Monfardini, G. The Graph Neural Network Model. IEEE Transactions on Neural Networks 20, 61–80 (2009).DOI
- [16]16.Battaglia, P. W. et al. Relational inductive biases, deep learning, and graph networks. Preprint at https://doi.org/10.48550/arXiv.1806.01261 (2018).
- [17]Kipf, T. N. & Welling, M. Semi-Supervised Classification with Graph Convolutional Networks. Preprint at https://doi.org/10.48550/arXiv.1609.02907 (2017).
- [18]Zhou, J. et al. Graph Neural Networks: A Review of Methods and Applications. Preprint at https://doi.org/10.48550/arXiv.1812.08434 (2021).
- [19]Gilmer, J., Schoenholz, S., Riley, P. F., Vinyals, O. & Dahl, G. E. Neural Message Passing for Quantum Chemistry. in (2017).arxiv.org/abs/1704.01212
- [20]Passaro, S. & Zitnick, C. L. Reducing SO(3) Convolutions to SO(2) for Efficient Equivariant GNNs. Preprint at http://arxiv.org/abs/2302.03655 (2023).
- [21]Batatia, I., Kovács, D. P., Simm, G. N. C., Ortner, C. & Csányi, G. MACE: Higher Order Equivariant Message Passing Neural Networks for Fast and Accurate Force Fields. Preprint at http://arxiv.org/abs/2206.07697 (2023).
- [22]Xin, H. et al. Fast and accurate spherical harmonics products. ACM Trans. Graph. 40, 1–14 (2021).
- [23]Luo, S., Chen, T. & Krishnapriyan, A. S. Enabling Efficient Equivariant Operations in the Fourier Basis via Gaunt Tensor Products. Preprint at http://arxiv.org/abs/2401.10216 (2024).
- [24]Projector augmented-wave method | Phys. Rev. B. https://journals.aps.org/prb/abstract/10.1103/PhysRevB.50.17953.
- [25]Kresse, G. & Furthmüller, J. Efficient iterative schemes for ab initio total-energy calculations using a plane-wave basis set. Phys. Rev. B 54, 11169–11186 (1996).
- [26]Kresse, G. & Furthmüller, J. Efficiency of ab-initio total energy calculations for metals and semiconductors using a plane-wave basis set. Computational Materials Science 6, 15–50 (1996).
- [27]Ozaki, T. & Kino, H. Numerical atomic basis orbitals from H to Kr. Phys. Rev. B 69, 195113 (2004).DOI
- [28]Ozaki, T. Variationally optimized atomic orbitals for large-scale electronic structures. Phys. Rev. B 67, 155108 (2003).
- [29]Li, P. et al. Large-scale ab initio simulations based on systematically improvable atomic basis. Computational Materials Science 112, 503–517 (2016).
- [30]Lin, P., Ren, X., Liu, X. & He, L. Ab initio electronic structure calculations based on numerical atomic orbitals: Basic formalisms and recent progresses. WIREs Computational Molecular Science 14, e1687 (2024).
- [31]Liao, Y.-L. & Smidt, T. Equiformer: Equivariant Graph Attention Transformer for 3D Atomistic Graphs. Preprint at http://arxiv.org/abs/2206.11990 (2023).
- [32]Batzner, S. et al. E(3)-equivariant graph neural networks for data-efficient and accurate interatomic potentials. Nat Commun 13, 2453 (2022).
- [33]Musaelian, A. et al. Learning Local Equivariant Representations for Large-Scale Atomistic Dynamics. Preprint at http://arxiv.org/abs/2204.05249 (2022).
- [34]Group Theory and Its Application to the Quantum Mechanics of Atomic Spectra | American Journal of Physics | AIP Publishing. https://pubs.aip.org/aapt/ajp/article-abstract/28/4/408/1036645/Group-Theory-and-Its-Application-to-the-Quantum?redirectedFrom=fulltext.
- [35]Xie, Y., Daigavane, A., Kotak, M. & Smidt, T. The Price of Freedom: Exploring Tradeoffs between Expressivity and Computational Efficiency in Equivariant Tensor Products.
- [37]Liao, M. Precise control of the interlayer twist angle in large scale MoS2 homostructures.
- [38]Zhang, S. et al. Domino-like stacking order switching in twisted monolayer–multilayer graphene. Nat. Mater. 21, 621–626 (2022).
- [39]Moon, P. & Koshino, M. Electronic properties of graphene/hexagonal-boron-nitride moiré superlattice. PHYSICAL REVIEW B (2014).
- [40]Wang, E. et al. Gaps induced by inversion symmetry breaking and second-generation Dirac cones in graphene/hexagonal boron nitride. Nature Phys 12, 1111–1115 (2016).
- [41]Wang, J. Electrical properties and applications of graphene, hexagonal boron nitride (h-BN), and graphene/h-BN heterostructures. Materials Today Physics (2017).
- [42]Wang, V., Xu, N., Liu, J.-C., Tang, G. & Geng, W.-T. VASPKIT: A user-friendly interface facilitating high-throughput computing and analysis using VASP code. Computer Physics Communications 267, 108033 (2021).
- [43]Jain, A. et al. Commentary: The Materials Project: A materials genome approach to accelerating materials innovation. APL Materials 1, 011002 (2013).
- [44]Zhang, L. et al. Structure and Properties of Violet Phosphorus and Its Phosphorene Exfoliation. Angew Chem Int Ed 59, 1074–1080 (2020).
- [45]Wang, J. et al. MAGUS: machine learning and graph theory assisted universal structure searcher. National Science Review 10, nwad128 (2023).
- [46]Fan, Z. et al. Linear scaling quantum transport methodologies. Physics Reports 903, 1–69 (2021).
- [47]Li, Y., Zhan, Z., Kuang, X., Li, Y. & Yuan, S. TBPLaS: A tight-binding package for large-scale simulation. Computer Physics Communications 285, 108632 (2023).
- [48]Perdew, J. P., Burke, K. & Ernzerhof, M. Generalized Gradient Approximation Made Simple. Phys. Rev. Lett. 77, 3865–3868 (1996).
- [49]Larsen, A. H. et al. The atomic simulation environment—a Python library for working with atoms. J. Phys.: Condens. Matter 29, 273002 (2017).
- [50]Loshchilov, I. & Hutter, F. Decoupled Weight Decay Regularization. Preprint at https://doi.org/10.48550/arXiv.1711.05101 (2019).
- [51]Reddi, S. J., Kale, S. & Kumar, S. On the Convergence of Adam and Beyond. Preprint at https://doi.org/10.48550/arXiv.1904.09237 (2019).




