WorksPhysics and Astronomy
Renormalization-Inspired Effective Field Neural Networks for Scalable Modeling of Classical and Quantum Many-Body Systems
Abstract
We introduce Effective Field Neural Networks (EFNNs), a new architecture based on continued functions – mathematical tools used in renormalization to handle divergent perturbative series. Our key insight is that neural networks can implement these continued functions directly, providing a principled approach to many-body interactions. Testing on three systems (a classical 3-spin infinite- range model, a continuous classical Heisenberg spin system, and a quantum double exchange model), we find that EFNN outperforms standard deep networks, ResNet, and DenseNet. Most striking is EFNN's generalization: trained on lattices, it accurately predicts behavior on systems up to with no additional training – and the accuracy improves with system size, with a computational time speed-up of compared to ED for lattice. This demonstrates that EFNN captures the underlying physics rather than merely fitting data, making it valuable beyond many-body problems to any field where renormalization ideas apply.
The collective behaviors of many interacting particles—such as spins, molecules, and atoms—give rise to some of the most intriguing phenomena in condensed matter physics. However, theoretical and computational studies of many-body problems often confront the curse of dimensionality and one must devise strategies to circumvent it. Machine learning has emerged as a powerful tool for extracting effective features from high-dimensional datasets and tackling long-standing problems in condensed matter physics. Numerous machine learning techniques have been applied to a wide range of classical and quantum many-body systems, including classifying phases of matter with supervised [1, 2, 3] and unsupervised machine learning [4, 5, 6, 7, 8], representing many-body quantum states [9–13], accelerating Monte Carlo simulations [14–17], fitting high-dimensional potential energy surfaces [18–22] and searching for spin glass ground states [23].
Standard deep neural networks (DNNs) are ineffective in solving many-body problems unless augmented with physical knowledge. For example, an classical two-dimensional Ising model requires highly complex DNN structures [24]. Consequently, extensive research has focused on encoding physics into neural network architectures to enhance their effectiveness in physical applications, including incorporating physical laws and constraints [25–40]. These physics-encoded neural networks have found applications in computing effective Hamiltonians [41–43], property prediction [36, 39, 41, 43–49], structure search [37, 38], etc. These methods, although often outperforming standard DNNs, tend to rely on intuitive and simplified imitations of physical principles, and still remain elusive in revealing the underlying many-body interactions.
In this work, we propose Effective Field Neural Networks (EFNNs), a new architecture inspired by field theory. The core motivation is to decompose many-body interactions into single quasi-particle representations governed by an emergent effective field. Unlike standard DNNs where adjacent layers rely only on previous layers, EFNNs incorporate initial features at every layer, forming a self-similar structure trained via recursive self-refining that captures many-body interactions through renormalization principles. Unlike approaches such as FermiNet [50], which directly parameterizes high-dimensional wave-functions without explicit separation of quasi-particles and fields, our method does not presuppose a fixed ansatz for the wavefunction. Instead, the quasi-particle and effective field representations are learned through a recursive self-refining process, mirroring the renormalization group (RG) framework in field theory, while leveraging the expressive power of deep neural networks. This recursive approach progressively refines the accuracy of both the quasi-particle and effective field descriptions.
We first illustrate the EFNN architecture by reformulating the classical 2D Ising model within the EFNN framework. The Hamiltonian of a classical 2D Ising model is defined as , where denotes a summation over pairs of nearest-neighbor sites, represents the spin at site , is the interaction strength between two spins, and is the collection of all spins. We define the effective field on spin as (sum of nearest neighbors), and the quasi-particle as , yielding total energy . As illustrated schematically in Fig. 1(a), the interacting spins are first mapped to an effective field, then the effective field element-wise multiplied with the interacting spins is mapped to independent spins. Finally, a summation is performed over the independent spins, and the total energy is obtained. We reformulate this as a neural network structure (Fig. 1(b)), where represents the interacting spins. The effective field layer and the quasi-particle layer constitute a single field-particle (FP) layer. The evaluation of relies on both the effective layer and the interacting spin layer , therefore a connection is required from to , as well as from to , distinguishing this architecture from a standard DNN. Finally, a summation is performed over to obtain the energy .
Figure 1. Energy evaluation of a 2D Ising model reformulated as a neural network. (a) Calculation of the effective field by summing the interacting spins. Each interacting spin is multiplied by its corresponding effective field to obtain independent spin values, and the total energy is determined by summing up these independent spins. (b) Neural network representation of the energy evaluation process. represents the element-wise multiplication.
Figure 2. Performance of EFNNs on a classical 3-spin infinite range model and classical Heisenberg spin system. (a) Architecture of the EFNN. (b) Performance of EFNNs, DenseNets, ResNets, DNNs on 3-spin infinite range model. (c) Performance of EFNNs, DenseNets, ResNets, DNNs on classical Heisenberg spin system.
In the classical 2D Ising model, both the effective field and quasi-particles can be exactly calculated, which, however, becomes infeasible for more complicated many-body interactions. Following the spirit of renormalization from field theory, we can recursively refine the evaluations of the effective field and the quasi-particles. This recursive structure, when combined with the multiplicative interactions and self-similar connectivity pattern, implements a continued function—a mathematical tool from renormalization theory for handling divergent perturbative series (see Supplementary Material [51] for the complete derivation). As shown in Fig. 2(a), we can extend the FP layer number in Fig. 1(b) to obtain the deep Effective Field Neural Networks. The evaluation procedure is detailed as
where the function maps the previous quasi-particle layer (for ) or the initial layer (for ) to the effective field layer . The input layer is first processed by , then element-wise multiplied by to produce the quasi-particle layer . Each FP layer consists of and . The final quasi-particle layer generates the output energy through the function , which sums all elements of the last layer. The functions are nonlinear (specifically, we use hyperbolic tangent functions) to implement renormalization by ensuring finite values, while are nonlinear to enhance expressiveness. Linear functions may suffice only for simpler models like the classical 2D Ising model with nearest-neighbor interactions.
We present three case studies to demonstrate the capability of EFNNs to capture many-body interactions. The first case study focuses on the classical 3-spin infinite-range model in 1D, with Hamiltonian:
where the spins take binary values , are constants, and the system comprises 15 lattice sites. We train EFNNs to evaluate in Eq. (1), and compare its performance with those of DenseNets, ResNets, and DNNs.
We generate 16000 training and 4000 test samples using Eq. (1). The nonlinear mappings and use sequential linear layers with tanh activations. The network is trained with the Adam optimizer and a decaying learning rate. Performance is evaluated by the relative error, defined as the root mean square error (RMSE) on the test set divided by the absolute value of the mean of . We vary the number of network layers from 1 to 3 and the number of neurons from 15 to 150. The performance of EFNNs, DenseNets, ResNets, and DNNs is presented in Fig. 2(b). The relative error decreases with more neurons for all networks, but EFNNs significantly outperform others as layers increase, with the relative error decreasing below for EFNNs with 2 and 3 layers when the number of neurons . The only exception is the ResNet with 3 layers and 150 neurons. In most cases, DenseNets, ResNets, and DNNs perform worse than 2-layer EFNNs.
Figure 3. Symmetrization: Each of the three channels of (, , ) undergoes convolution and element-wise multiplication within the same channel. The resulting products are summed to form the symmetrization layer . Subsequently, is passed through a sequence of layers—convolution, tanh activation, and another convolution—to produce for or for .
To test the capability of EFNNs beyond discrete systems, our second case study examines a classical Heisenberg spin system, using the same three-body interaction model in Eq. (1) with continuous spins, where takes continuous values in rather than discrete values 0, 1. We use the RMSE of the total energy as the performance metric, with results shown in Fig. 2(c), since the average value of Eq. (1) is 0. Compared to the discrete case, the configuration space is now continuous, yet EFNNs with 2 and 3 layers still outperform all other networks. Fig. 2(c) also shows the general trend that the error decreases with increasing number of neurons. The most accurate result is achieved by the 3-layer EFNN with 150 neurons, where the error of the total energy reduces to . These results demonstrate that EFNNs effectively capture many-body interactions through the renormalization procedure of their recursive continued-function structure; failing to incorporate this structure leads to inaccurate results. We show in the Supplementary Material [51] that DNNs completely lack the structure of a continued function, while DenseNets and ResNets misrepresent a continued function.
We proceed to our third, more challenging case study: evaluating the Monte Carlo energy of the quantum double exchange model [52–54] at finite temperature. The Hamiltonian of an lattice reads
where denotes nearest neighbors, is the electron annihilation operator with spin at site , are the Pauli matrices, and is a 3D classical unit vector representing the local spin at site , the classical spin interacts with the electrons through an on-site coupling, and is the collection of all classical spins . We set , , , , .
At finite temperature , the Monte Carlo energy determines the acceptance rate in Monte Carlo simulations, where are the eigenvalues of with chemical potential in Eq. (2). Computing the exact value of requires diagonalization of Eq. (2) at each Monte Carlo update step, incurring computational complexity per step. With millions of Monte Carlo steps required for convergence, this approach becomes computationally prohibitive. An intuitive solution is to approximate using a perturbative expansion, yielding an effective model with RKKY-type interactions [55–57]. This effective model with 2-body interactions has proven sufficiently accurate to replace exact diagonalization in Determinant Quantum Monte Carlo (DQMC) simulations [58]. The effective model including up to 4-body interactions takes the form
Here, the dot products include all pairs of spins on the lattice, including self-interactions. The perturbative approach faces a fundamental dilemma: truncating at low orders fails to capture the full physics, while including higher-order terms leads to numerical divergence due to the factorial growth in the number of terms. EFNN addresses this challenge through renormalization.
The spin interactions in Eq. (3) are exclusively expressed as dot products to satisfy the symmetry of Eq. (2). Inspired by this symmetry, we incorporate symmetrization layers into the EFNN architecture, depicted in Fig. 3. The symmetrization is achieved by summing dot products of each channel of after convolution:
Here, each is convolved into . Supplementary Material [51] details the convolution process and proves that remains invariant under transformations. As shown in Fig. 4(a), in the first FP layer, and are initialized as
Figure 4. Relative error of EFNNs. (a) Architecture of EFNN incorporating symmetrization layers , . (b) Performance of EFNNs, DenseNets, ResNets, DNNs and effective model on a test set with lattice size , for channel numbers . (c) Generalization performance of networks trained on lattices () when applied to larger systems.
For subsequent layers , the effective field and quasi-particle layers are
The mappings and consist of a convolutional layer, a tanh activation, and another convolutional layer. For the mapping , the final quasi-particle layer is convolved into a single-channel matrix, and an element-wise summation is performed to produce the scalar energy prediction , see Supplementary Material [51]. The layer dimensions are as follows: input layer , intermediate layers , , for , and output . For comparison, we also incorporate symmetrization layers and final summation into DenseNet, ResNet, and DNN architectures, detailed in Supplementary Material [51].
We generate a total of data points and split them into 80% for training and 20% for testing. These data are obtained by exact diagonalization of the Hamiltonian in Eq. (2) with chemical potential. For the effective model in Eq. (3), we perform linear regression to obtain the coefficients. While the full effective model would require 12758826 parameters, we reduce this to 213201 by restricting interactions within each tile. The EFNNs, DenseNets, ResNets, and DNNs with symmetrization are trained using networks with channels and layers, with parameter counts ranging from 200 to 30000.
with channels and layers, with parameter counts ranging from 200 to 30000. As shown in Fig. 4(b), the effective model achieves a relative error of approximately , whereas the neural networks achieve relative errors on the order of —approximately six times better using significantly fewer parameters. Among the neural networks, EFNNs with 1, 2, and 3 layers consistently outperform all other architectures, with some overlap observed between 1-layer EFNNs and 3-layer DNNs, as well as 1-, 2-, and 3-layer DenseNets at . For EFNNs, the error systematically decreases as increases and as the number of layers increases from 1 to 3. This superior performance with fewer parameters shows that the EFNNs possess inherent renormalization capabilities beyond simple parameter fitting.
We next test the generalization capability of these neural networks trained on lattices by evaluating their performance on systems up to . We select networks with channels as a representative configuration and evaluate EFNN, DenseNet, ResNet, and DNN architectures across this range of lattice sizes. The convolution operations with padding under periodic boundary conditions (PBC) effectively capture interactions between localized and neighboring spins. As shown in Fig. 4(c), all neural network architectures show improved performance on larger lattices, with relative errors decreasing as the system size increases from to . The EFNNs exhibit the best extrapolation capability and renormalization ability: the 3-layer EFNN achieves the lowest errors across all lattice sizes, reaching a remarkable relative error of approximately at . The 2-layer EFNN mostly outperforms the 1-layer EFNN, while among non-EFNN architectures, only the 3-layer DNN has comparable performance to the 1-layer EFNN. All other configurations consistently produce higher errors, confirming EFNN’s superior ability to capture scale-invariant features. Notably, as demonstrated in Supplementary Material [51], for , EFNN achieves speed-up compared to ED.
We give some discussions and conclude this paper. In all three case studies—the classical 3-spin infinite-range model, continuous classical Heisenberg system, and quantum double exchange model—EFNNs consistently outperform DenseNets, ResNets, DNNs, and the effective model, achieving the lowest errors. Particularly striking is EFNN’s ability to generalize: trained on lattices, it accurately predicted energies for systems as large as , while DenseNets, ResNets, and standard DNNs struggle with this extrapolation task. This superior performance isn’t accidental—it stems from EFNN’s mathematical foundation in continued functions. While conventional architectures like DenseNet and ResNet were designed through trial and error with various architectural innovations, EFNN’s structure directly implements continued functions, a mathematical framework that naturally captures many-body interactions up to infinite order. More importantly, for a perturbative expansion like Eq. (3), EFNN selects the most relevant terms via training, and expresses these terms in nonlinear continued function structure. Fig. 5 compares these structures, demonstrating that only EFNN (Fig. 5(a)) possesses the correct continued function structure. The Supplementary Material [51] provides a detailed mathematical explanation of why EFNN implements continued functions while ResNet and DenseNet do not. A special case of continued functions is the Padé approximant, which takes the form of a continued fraction and is ubiquitously used throughout physics and numerical methods for its rapid convergence, strong extrapolation capability, and compact representation [59]. Continued functions generalize continued fractions by replacing the inversion operation with generic nonlinear functions. Here, we use tanh activation functions instead of inverses, which makes EFNN trainable through standard backpropagation—something impossible with traditional Padé approximants due to gradient singularities. This simple substitution preserves the mathematical power of continued functions while enabling practical neural network training. Notably, EFNN actually implements self-similar approximation [60], a powerful mathematical framework that has been successfully applied across diverse fields including high energy physics [61–65], condensed matter physics [66, 67], quantum mechanics [60, 64, 68–72], quantum field theory [60, 73–75], and applied mathematics [66, 76–85]. Previously, applying self-similar approximation required tedious manual calculations to obtain order by order matching; EFNN changes this by automating the process through neural network optimization. EFNN’s self-similar structure also naturally comes from renormalization group equation, as detailed in the Supplementary Material [51].
Figure 5. Comparison of neural network structures. (a) In an EFNN, the initial layer is recursively integrated into every quasi-particle layer. After a mapping, is multiplied with subsequent layers, forming a continued-function representation (see the equation above the figure). (b) A typical architecture of ResNet, only one skip connection starts from and summation is used, this is not a correct continued-function structure. (c) Removing the connections transforms an EFNN into a standard DNN, with no continued-function structure. (d) A typical architecture of DenseNet, is concatenation operation, this is not a correct continued-function structure.
In conclusion, we have developed EFNN, a neural network architecture inspired by field theory that captures high-order interactions in both classical and quantum models. The key to EFNN’s success lies in how it recursively refines effective fields and constructs quasi-particles to encode complex many-body effects. This partitioning into field and quasi-particle layers provides clear physical interpretation while maintaining computational efficiency. As we showed, the connections from the true particle layer to the quasi-particle layers create a continued function structure with renormalization properties similar to Padé approximants. What EFNN actually implements is self-similar approximation—a powerful mathematical technique that has been widely used in physics but previously required manual calculations for each problem. Once properly trained, this architecture learns effective representations that accurately predict energies even for systems with complicated many-body interactions. Our tests on three diverse systems—the classical 3-spin finite-range model, continuous classical Heisenberg spin system, and quantum double exchange model—demonstrate that EFNN consistently outperforms DenseNet, ResNet, and standard DNNs. Most remarkably, EFNNs trained on small systems improve accuracy on larger ones, opening possibilities for training on expensive DFT data from small lattices then inferring on much larger systems. Looking ahead, we plan to develop theoretical error bounds and apply EFNN to other challenging problems in physics.
DATA AVAILABILITY
The data that support the findings of this study are openly available in Zenodo at Ref. [86].
**Acknowledgments—**This work is supported by the Hong Kong Research Grants Council (16306220) and National Key R&D Program of China (2021YFA1401500). Y.Z. is supported by the Max Planck Partner lab on quantum materials from Max Planck Institute Chemical Physics of Solids.
References
Supplemental Information of Effective Field Neural Networks
supplementary_v2.pdf
In this supplementary information, we present the complete workflow of EFNN with symmetrization for the quantum double exchange model in Section I. We introduce the basics of Padé approximation, continued fractions, and continued functions in Section II, and demonstrate that EFNN implements a continued function, while ResNet, DenseNet and DNN are not continued functions. In Section III, we derive the structure of EFNN from the renormalization group (RG) equation. In Section IV, we show that the convolution layer ensures the system’s invariance under the point group O(3). In Section V, we present the adapted architectures of ResNet, DenseNet, and DNN for energy fitting of the quantum double exchange model. In Section VI, we break down the details of DQMC, and demonstrate how EFNN solves the computational bottleneck problem in DQMC for quantum double exchange model. We report the speed-up of EFNN against ED. In Section VII, we report the distributions of EFNN’s predictions, and the ground truth data, for quantum double exchange model, demonstrating that the accuracy of EFNN.
Computational workflow of EFNNs with symmetrization
Figure 1 (PDF p. 9). Computational workflow of EFNNs with symmetrization
In this section, we present the complete workflow of EFNN with symmetrization for the quantum double exchange model. Fig. 1(a) illustrates the symmetrization process: each of the three channels of (, , ) undergoes convolution and element-wise multiplication within the same channel. The resulting products are summed to form the symmetrization layer . Subsequently, is passed through a sequence of layers—convolution, tanh activation, and convolution—to produce for or for . Fig. 1(a) is Fig. 3 in the main text. Fig. 1(b) shows the generation of quasi-particle layers: the transformed symmetrization layer is element-wise multiplied with the effective field layer , yielding the quasi-particle layer for . Fig. 1(c) depicts the transformation to effective field layers: each quasi-particle layer is processed through convolution, tanh activation, and convolution to generate the effective field layer for . Fig. 1(d) illustrates the energy evaluation: the final quasi-particle layer is transformed into a single-channel matrix, and an element-wise summation of this matrix yields the scalar energy prediction . These visualizations are generated after training the model with channels and layers, using the 576th sample from the test set as input .
Mathematical details of DenseNet, ResNet, DNN
In this section, we describe the mathematical structures of EFNN, ResNet, DenseNet, and DNN.
We denote the input spins as .
Figure 2 (PDF p. 10). Comparison of structures of (a) EFNN, (b) ResNet, (c) DNN, and (d) DenseNet
Why EFNN is a continued function
In Fig. 2(a), the functions , , , , all have the forms of a composition of a linear mapping linear mapping. We denote
The quasi-particle layers in EFNN form a nested structure that implements a continued function. For instance, the third quasi-particle layer is expressed as:
where denotes the Hadamard (element-wise) product.
To illustrate why this structure constitutes a continued function, we draw an analogy with Padé approximants, which can be expressed as continued fractions—a special case of continued functions.
In physics and applied mathematics, perturbative series naturally express physical interactions to arbitrary order. However, these series have a fundamental dilemma: truncating at low order fails to capture the complete physics, while including higher-order terms typically leads to divergence. Renormalization provides the solution to this problem, with Padé approximants being the most widely used numerical scheme for transforming divergent series into convergent expressions. To match a perturbative series with a Padé approximant , one equates truncated with the rational function , then deals with the algebraic equation
is the order of the polynomial. The approximation accuracy typically improves as . Examples of continued fraction:
Exponential function:
Natural logarithm:
The Padé approximant for in Eq. (7) has everyday application throughout numerical analysis, since this is how matrix exponential is computed in various programming languages [1], for example, the expm in Python, Mathematica, and MATLAB. In Eq. (8), the perturbative series diverges for , however, the Padé approximant always converges and captures well the value of . The continued fraction can be naturally generalized, by replacing the inverse operation with other nonlinear functions. Examples of continued function [2]:
,
,
.
To see why EFNN is a continued function, we first turn to a simpler case by requiring . We let , then in Eq. (3) becomes:
Since is actually a vector, and rational functions always introduce fictitious poles, we would like to replace the inverse function with tanh, using Eq. (2). Then the neural network implementation with tanh then gives us:
And we have fully demonstrated that is a continued function.
Note that in Eq. (10), is vector-valued, each component of , which is a finite value, contains infinite number of perturbation terms, as shown in Eq. (11).
Why ResNet, DenseNet, and DNN are not continued function
For DNN, as shown in Fig. 2(c), the mapping , does not contain any skip connections, as required by continued function. Therefore it is apparent that DNN is not a continued function.
For ResNet,
is element-wise addition in Fig. 2(b). Then
is nonlinear. Clearly Eq. (14) is not a correct continued function. The correct form would be:
There are too many redundant terms in Eq. (14). The expression in Eq. (15) has the structure of Fig. 2(a), with replaced by , and set to identity.
For DenseNet, as shown in Fig. 2(d), is vector concatenation: for , , . We take nonlinear functions , the subsequent layer are:
The explicit expressions for , , read
The expression in Eq. (21) is not a correct continued function. The layer concatenates previous layers, a summation of ’s components gives energy, such complicated function cannot be interpreted, and is not properly renormalized.
From renormalization group equation to EFNN
In the previous section, we have explained that EFNN is a continued function. In this section, we show that the EFNN structure can be derived from a renormalization group (RG) equation, as all continued functions can be.
We follow the arguments in Refs. [3, 4, 5], to derive the EFNN structure from RG equation. The derivation proceeds in 4 major steps: Step 1, we transform the perturbative series using an algebraic prefactor, and obtain exponential renormalization through RG equation. Step 2, we apply nested renormalization to create the exponential self-similar approximant with hierarchical structure. Step 3, we replace unbounded functions with bounded tanh functions while preserving asymptotic properties. Step 4, we generalize from scalar to vector inputs using Hadamard products, yielding the final EFNN architecture.
Step 1: using algebraic transform to renormalize perturbative series through RG equation
We first use scalar , target function is , we know its perturbative series up to -th order:
We apply an algebraic transform to the above polynomials:
We embed the discrete functions into , with , since continuous variable is easier to deal with. is interpreted as time. We let , and further denote as , to show that the initial value of the flow is , while the variable in physical space is . We assume that satisfies the self-similar equation:
because we hope to find the fixed point solution :
to guarantee the best convergence when using variable [6], i.e., in the physical space. does not depend on , because it is already a fixed point solution. does not depend on initial value , because we hope that is an attractor. Eq. (32) leads to the Lie equation:
the velocity is given by
Lie equation is exactly RG equation, when is scaled as :
We can’t obtain the fixed point solution exactly, and need to compute the so called -th order quasi-fixed point solution. We use the Euler scheme to approximate the velocity at :
We assume that starting from , it takes time to reach the -th order quasi-fixed point , integrating the Lie equation Eq. (37), we obtain:
The dummy variable is in physical space, starting with for lower limit . We choose the renormalization map from physical space to function space as algebraic:
then
We multiply the above equation by to scale back, and obtain a renormalized value for :
Starting from Eq. (44), by tuning values of , , , we can arrive at several continued function approximants, including Padé. Here, we let , and obtain
This is to say, this one-step renormalization is a map:
We repeat the renormalization process for the polynomial prefactor until only the affine term is left:
Step 2: using nested renormalization to obtain exponential self-similar approximant
We use the exponential renormalization to renormalize the polynomial in the exponential part in Eq. (47), and always leave a linear term as a prefactor, we obtain the exponential self-similar approximant:
is an integer depending on . One often performs a perturbation analysis for the exponential self-similar approximant with the perturbative series to obtain the unknown coefficients , , , . Note that one may also include the linear term in the exponential, we have the freedom of not taking this approach, and we have numerically tested that this approach does not work well.
Step 3: further renormalization for exponential self-similar approximant
Since linear functions and exponential functions are not bounded, to achieve better convergence, we need to further renormalize the nested exponential in Eq. (48). Since we do not use order by order matching to determine the unknown coefficients, we can set with a given integer. We take as an example. To further renormalize, we proceed in 5 substeps. We denote the exponential continued function as
First, we scale and shift the exp function in :
Second, we shift the origin of in :
Third, renormalize exp by tanh. We know that , when is small, and is bounded on . Therefore we replace by :
Fourth, renormalize the linear prefactors before tanh. Since , when is small, we perform the following renormalizations:
Then
Fifth, rescale , :
Step 4: generalization to vector case
The above derivations take . For a spin system, is vector-valued. We generalize , let , and transform scalar product to Hadamard product , and let , , , , . For , , we let , , , . Then
The above is just the in Eq. (10). We need one more step to obtain the energy, i.e., summing up all components of to get a real scalar as the predicted energy.
Symmetrization
In the main text, the symmetrization layer is given by
We choose to use 2 different matrices as the filters for the 2 convolutional operations in , and rewrite as
For , the convolution filter is , is the output channel number. The -th element of the convolution result is
where we assume a proper padding scheme is chosen in the above equation. Then
the -th element of the symmetrization layer reads
We let
Figure 3 (PDF p. 17). (a) ResNet, (b) DenseNet, and (c) DNN adapted for Monte Carlo energy fitting of quantum double exchange model. is element-wise addition, while is concatenation.
Then
For a matrix ,
As a result, under any operation , the value remains invariant.
ResNet, DenseNet, DNN adapted for Monte Carlo energy fitting of quantum double exchange model
In this section, we present the structures of ResNet, DenseNet, and DNN, adapted for Monte Carlo energy fitting of quantum double exchange model.
As shown in Fig. 3(a) and Fig. 3(b), for ResNet and DenseNet, we add 2 symmetrization layers and to obtain invariant results. For DNN in Fig. 3(c), we use 1 symmetrization layer , as no skip connection is used for DNN. How the nonlinear layers are computed, how the symmetrization is performed, and how the final energy is computed are similar to Fig. 3 in the main text.
DQMC and the speed-up by EFNN
In this section, we explain the computational bottleneck problem in Determinant Quantum Monte Carlo (DQMC), and report the computational speed-up achieved by EFNN.
Outline of DQMC computation in 2D quantum double exchange model
For this quantum double exchange model, the DQMC computation has 3 major steps [7, 8].
Converting the Hamiltonian to quadratic form
The 2D quantum double exchange model is a lattice system describing electronic hopping interactions and on-site coupling between itinerant electrons and localized classical spins. In this framework, the classical spins act as a random field. The Hamiltonian reads (with chemical potential part)
is the hopping amplitude, is the on-site interaction strength, is a classical spin vector. , , are Pauli matrices. , denote spin components and .
We first need to expand the site index to double index, the double index read , , denoting a lattice point on a square. Now a creation operator reads or .
We group the creation operators into a row vector. The row vector has two parts,
the first part is a row of creation operators with spin , the explicit expression reads
The creation operators in Eq. (69) are arranged in row-major order. Similarly, we arrange creation operators in row-major order:
The Hamiltonian in Eq. (67) is expressed in a quadratic form:
Hermitian matrix ,
and are hopping matrices in and directions. For , the -th element is , the -th element is , the rest are 0. The boundary condition is PBC, the , indices are understood as , . For hopping matrix , the -th element and the -th element are , the rest are 0. The boundary condition is PBC, the , indices are understood as , .
Matrix contains all the components along the diagonal:
Matrix contains all the components along the diagonal:
Matrix contains all the components along the diagonal:
Convert tr to det
In this step, with the quadratic form in Eq. (71), the trace is converted to det, following the DQMC scheme:
In this step, the trace of a matrix is converted to the determinant of a matrix, this is the quantum-to-classical mapping in DQMC.
More precisely, the partition function is
since all the classical continuous spins have norm 1. Eq. (76) can replace Eq. (77), only differing by a constant factor when the data points are dense and even enough.
Classical Monte Carlo using determinants
We use the determinants in Eq. (76) as statistical weight for Monte Carlo sampling. Determinants in Eq. (76) have the nice property that they are all strictly positive, because they can be written as
are the eigenvalues of Hermitian matrix . In each Monte Carlo step, we perform update on one component of one classical spin vector, among such 3D vectors in . Evaluating Eq. (78) requires Exact Diagonalization (ED), for such matrix, one evaluation requires complexity . Note that evaluating the det in left hand side of Eq. (78) has exactly the same complexity. There are scalar components in in total, a sweep to update every component has complexity . In order to obtain convergent results, we typically need 500 effective samples, then the total computational cost is
As increases, with computational complexity in Eq. (79), the computation becomes prohibitive for classical Monte Carlo very quickly. We use EFNN to address the computational bottleneck problem by training EFNN using exact value of Eq. (78), given . We make one more transformation, and define a Monte Carlo energy ,
then
is formally a free energy. We use ED (Exact Diagonalization) to generate data as training set to EFNN. ED has the same complexity as determinant computation.
| 10 | 15 | 20 | 25 | 30 | 35 | 40 | |
| EFNN time/s | |||||||
| ED time/s |
Table 1. Average computational time for EFNN inference, , 3 layers, and ED, quantum double exchange model
Figure 4 (PDF p. 20). Average computational time for EFNN inference, , 3 layers, and ED. The y-axis has log scale.
We report the average computation times of EFNN and ED in Table I, and make a plot in Fig. 4, for comparison. While the EFNN model was trained on a supercomputer, this benchmark compares the inference time for a single value given a configuration against the corresponding ED calculation, both performed on an Alienware m15 R12 laptop. As shown in Table I and the green line in Fig. 4, the EFNN inference time remains approximately constant at order when the lattice size grows from to . Conversely, as shown by the red line, the ED computational time scales polynomially with , rapidly reaching approximately for . This scaling behavior makes ED computationally prohibitive for Monte Carlo sampling. Notably, for , the EFNN achieves a speedup of approximately .
Details of inference error and data distribution, quantum double exchange model
In this section, we report the error of EFNN prediction, plot the distributions of the test data (ground truth) and the EFNN predictions, for the quantum double exchange model.
We report the absolute value of mean of for test set, RMSE of prediction by EFNN, and relative error in Table II, with , layer number , in quantum double exchange model. The relative error is defined as
| 10 | 15 | 20 | 25 | 30 | 35 | 40 | |
| 51.6508 | 116.1850 | 206.5729 | 322.7573 | 464.7841 | 632.5962 | 826.2887 | |
| RMSE | 0.0584 | 0.0989 | 0.1340 | 0.1714 | 0.2118 | 0.2549 | 0.3003 |
| relative error |
Table II. Performance metrics for EFNN, , 3 layers, quantum double exchange model
As shown in Table II, even though the model is trained on a lattice under boundary condition PBC, when applied to larger lattices, the RMSE increases only marginally. Moreover, the increases very quickly with respect to . As a result, the difference of the prediction value by EFNN and the ground truth value of , becomes smaller and smaller perturbations as increases, demonstrating the accuracy of EFNN.
We present the histograms of test set data (, ground truth value of ) and prediction results computed by EFNN (, predicted value of ) for the quantum double exchange model in Fig. 5, for . The blue histograms are for ground truth data , and green histograms are for predictions . Note that as lattice size increases, the model trained on lattice still correctly predicts for larger lattices, up to , as shown by the overall histogram shapes, and the mean values of and . Note that as increases, the absolute value of mean increases.
Figure 5 (PDF p. 22). EFNN’s prediction results of quantum double exchange model, with and 3 layers: histogram of and , for , 15, 20, 25, 30, 35, 40.
References
- [1]A. H. Al-Mohy and N. J. Higham, A New Scaling and Squaring Algorithm for the Matrix Exponential, SIAM Journal on Matrix Analysis and Applications 31, 970 (2010).DOI
- [2]C. M. Bender and S. A. Orszag, Advanced Mathematical Methods for Scientists and Engineers I, 1st ed. (Springer New York, New York, NY, 1999).DOI
- [3]V. Yukalov and E. Yukalova, Higher Orders of Self-Similar Approximations for Thermodynamic Potentials, Physica A: Statistical Mechanics and its Applications 206, 553 (1994).DOI
- [4]V. Yukalov and S. Gluzman, Self-Similar Bootstrap of Divergent Series, Physical Review E 55, 6552 (1997).DOI
- [5]V. Yukalov and S. Gluzman, Self-Similar Exponential Approximants, Physical Review E 58, 1359 (1998).DOI
- [6]V. Yukalov and E. Yukalova, Self-Similar Structures and Fractal Transforms in Approximation Theory, Chaos, Solitons & Fractals 14, 839 (2002).DOI
- [7]Z. Bai, Lecture notes on determinant quantum monte carlo (dqmc), University of California, Davis, Department of Computer Science (2009), accessed: 2023-10-27.
- [8]E. LOH and J. GUBERNATIS, Chapter 4 - stable numerical simulations of models of interacting electrons in condensed-matter physics, in Electronic Phase Transitions, Modern Problems in Condensed Matter Sciences, Vol. 32, edited by W. HANKE and Y. KOPAEV (Elsevier, 1992) pp. 177–235.