WorksPhysics and Astronomy
Foundation Tensor Networks Generalize Across Lattice Hamiltonians
Abstract
Neural quantum states and tensor networks have emerged as leading techniques for simulating quantum many-body systems. However, disordered systems remain extremely challenging in both frameworks as they require computation of the ground state for every realization of the disorder. Recently, neural quantum states have been used to construct foundation models that generate the ground state directly from the Hamiltonian. Still, they are hindered by the need to perform extensive sampling to estimate observables. To address these difficulties, we introduce the foundation tensor network (FTN), which uses a neural network to generate the tensor network representation of the ground-state wavefunction from the Hamiltonian. To accelerate training, we develop a technique, called GridBP, based on iterated one-dimensional belief propagation that reduces the cost of computing the energy expectation value to just two tensor contractions. We use FTNs to calculate the critical parameters of the glass transition in the quantum Edwards-Anderson model via finite-size scaling analysis. To facilitate rapid averaging, we use a moment-generating function to evaluate the order parameter, lowering the cost to a single tensor contraction. We obtain high-fidelity estimates for the critical parameters by averaging over 500,000 disorder realizations.
Introduction
Examining zero-temperature critical phenomena in disordered systems is among the most theoretically interesting problems in quantum statistical mechanics due to the complex phase behavior of these systems arising from quantum, rather than thermal, fluctuations. Spin glasses are archetypal systems in the study of disordered and glassy critical phenomena [1]. In these systems, quenched disorder that persists over extremely long time scales generates a glassy phase where spins are frozen in a random but ordered state that is neither ferromagnetic nor antiferromagnetic [2, 3]. Here, a central problem is to understand the physics of how spin glasses exit the glassy phase into other disordered phases. The Edwards-Anderson transverse-field quantum Ising (EA-TFQIM) model, examined in this paper, is a paradigmatic disordered system that undergoes a transition from a glassy to a disordered paramagnetic phase as the strength of the transverse field increases [4].
Simulating quantum spin glasses requires gathering statistics over many ground states generated by different realizations of the disorder, making it one of the most challenging problems in computational physics. The direct approach of exactly diagonalizing [5] the Hamiltonian is intractable for systems of even moderate size, making Monte Carlo methods the dominant technique [6, 7, 8, 9, 10]. One such approach [6] converts the system from a -dimensional quantum system to a -dimensional classical system by employing a Trotter decomposition in imaginary time [11]. To understand quantum phase transitions, classical Monte Carlo methods must approach the limit of infinite inverse temperature, which imposes increasing computational costs.
Variational Monte Carlo [12, 13] (VMC), meanwhile, optimizes a trial wavefunction by drawing samples from the corresponding Born rule distribution. VMC incurs both sampling and optimization costs to converge the variational ansatz to the ground state. Applying these methods to quantum spin glasses is computationally prohibitive as they require two layers of averaging: disorder and quantum. One must first find the ground state for each realization of disorder and then draw samples from that wavefunction to estimate observables. Neural network parametrization of the variational wavefunction within VMC defines the emerging framework of neural quantum states (NQS) [14, 15]. Recent work shows that NQS can act as foundation models for a range of Hamiltonians [16], reducing the need to optimize the trial wavefunction for each realization of the disorder. However, one must still perform extensive sampling to obtain accurate estimates of observables from an NQS.
Tensor networks define a prominent alternative to Monte Carlo for quantum many-body simulations [17]. They encode the wavefunction as a tensor network state (TNS), a contraction of tensor cores arranged in a network that mirrors the topology of the system they represent. The central advantage is that observables can be computed deterministically as tensor contractions. The TNS representation of a ground state can be obtained by directly optimizing the tensor cores to minimize the variational energy [18], cooling the state through imaginary-time evolution [19], or evaluating the partition function for the -dimensional classical system [20]. Tensor networks have proven effective for computing properties of lattice systems, such as the quantum Ising and Heisenberg models [21], among a range of other applications [22, 23, 24, 25, 26, 27]. Provided that contraction is feasible [28] for a given lattice, the primary cost of tensor networks lies in finding the ground state. Recent work has also combined tensor networks with both VMC and NQS [29, 30]. methods. However, these approaches, although highly accurate and expressive, still employ Monte Carlo sampling to estimate observables and suffer from the usual sampling costs.
In this paper, we introduce a methodological framework called foundation tensor networks (FTNs) that combines the strengths of NQS and TNS. FTNs learn a foundation model for the tensor network cores that generalizes across Hamiltonians. In contrast to other hybridizations of neural and tensor networks, the wavefunction returned by the FTN is fundamentally a TNS. We use a transformer [31] to directly generate the tensor network representation of the ground state from a graph embedding of the Hamiltonian. In disordered systems, this circumvents the need to find the ground state for each sample, amortizing the optimization work. To demonstrate FTNs, we train a foundation model to produce the ground states of the EA-TFQIM as a function of the transverse field and all site-site couplings. We use this to examine the transition from the low-field glassy phase to the high-field disordered phase. Directly averaging the one-shot ground states across realizations of the disorder enables us to compute the critical parameters via finite-size scaling analysis [32]. We obtain estimates that match and refine results in the literature, which have significant uncertainty, demonstrating that FTNs are a powerful new tool for exploring critical phenomena.
The Quantum Edwards-Anderson Model
We study the EA-TFQIM on a square two-dimensional lattice with sites and open boundary conditions. Its Hamiltonian takes the form
where and are Pauli operators acting on spin , is the coupling between sites and , and is the transverse field. The first sum runs over unordered nearest-neighbor pairs. The couplings are i.i.d. Gaussian random variables with unit variance and zero mean, . All units in this study are nondimensionalized in terms of a natural coupling scale that coincides with the width of the coupling distribution.
Each set of random couplings generates a different ground-state wavefunction that minimizes the Rayleigh quotient
At zero temperature, the EA-TFQIM exhibits a quantum phase transition mediated by the transverse field. At the system transitions from a glassy phase to a paramagnetic phase. The order parameter for the transition is the replica overlap
where denotes a quantum average taken with respect to the ground state with the couplings .
Amortized Disorder Sampling
To accelerate averaging over the disorder, we learn a mapping between the parameters of the Hamiltonian and a representation of the ground-state wavefunction. Then we can average over the disorder by directly generating wavefunctions from the disorder samples via the learned mapping. The process of learning the mapping over the entire distribution of Hamiltonians amortizes the computational effort compared to optimizing a trial state for each sample individually. We call this amortized disorder sampling. A transformer provides an expressive ansatz to learn the mapping, and the projected entangled pair state (PEPS) enables us to efficiently represent the wavefunction and measure observables without an additional step of sampling from the Born distribution. Here, we develop a method to perform amortized disorder sampling using FTNs.
We first expand the wavefunction in the computational basis
where is the spin index, is the coefficient tensor, and is the tensor product computational basis state. Next, we encode the coefficient tensor as a projected entangled pair state (PEPS) [33] of the form
where are the tensor cores, and is the tensor trace that sums over virtual indices of connected cores.
Instead of optimizing the PEPS directly, we use a transformer-based neural network to learn the mapping between Hamiltonian parameters and tensor cores. We embed information about each site, including local couplings, the transverse field, and position, into a series of tokens . After passing the tokens through the generator , a series of transformer blocks, we obtain a set of latent states representing the approximate ground state of the input Hamiltonian: , where are the parameters. The readout, , a multilayer perceptron, then converts each token into a tensor core .
We construct the generator from a series of transformer blocks [31] of the form
where , , , and denote layer normalization [34], the identity map, a multi-layer perceptron (MLP), and multi-head self-attention (MSA), respectively. The MSA layer takes the form
where is the number of attention heads, and are learnable matrices indexed for each attention head , and denotes vector concatenation. To form the generator, we compose these transformer blocks
where is the number of layers. The combination of the transformer and tensor network forms an FTN, which we denote as . We train the generator and readout to minimize the average energy over a collection of Hamiltonians [16]
Once trained, the FTN enables us to compute quantities like the overlap parameter in Eq. (3) directly by evaluating the neural network and contracting the resulting PEPS.
Grid Belief Propagation
An essential ingredient in the feasibility of FTNs is fast training. At each training step, we must evaluate the energy and its gradient with respect to the parameters . Both require one or more contractions, which are significantly more expensive than performing inference on the transformer. This makes the contraction step the computationally limiting factor. Thus, we attempt to minimize the number of contraction steps. There are several ways to compute the energy expectation value with a PEPS. One is to form a projected entangled pair operator (PEPO) representation of the Hamiltonian [21]. Another is to use linearity and evaluate the Hamiltonian term by term. The former requires enlarging the bond dimension, which is undesirable due to the high cost of PEPS contraction [35], and the latter scales quadratically with the number of sites. Inspired by belief propagation algorithms for tensor networks [36, 37], we derive a method called grid belief propagation (GridBP) that enables us to compute the energy and its gradient, via automatic differentiation, with only two contractions and no increase in the bond dimension.
Consider a general product of states, and , and operators, , of the form
By defining messages with a recursion relation of the form
where and , we can isolate a particular operator at position via
We can exploit this relation to isolate pairs of sites in a PEPS contraction and obtain the two-body environment tensor
which is sufficient to compute the energy contribution of any two-body term. Here is the bilayer network, cf. Figure 1 (a), and is the core bra-ket pair at site . We first isolate a column at :
Figure 1. An illustration of the GridBP method for computing the two-body environment tensor. We select a pair of cores in the bilayer network (a) and isolate their column using a first round of message passing left and right between columns (b). Then, from the MPS-MPO-MPS product, we isolate their rows via a second round of message passing up and down the network (c). Finally, we differentiate the resulting network with respect to and to obtain the two-body environment tensor.
where is the matrix product operator (MPO) for column and and are matrix product states (MPS), cf. Figure 1 (b). Next, we isolate a pair of rows in the MPS-MPO-MPS product at sites and in column , Figure 1 (c),
and are the row MPOs. and are the column matrix product states. Finally, we take derivatives with respect to the cores to generate the two-body environment tensor
where and . The one-body environment follows similarly. By iterating both recursion relations, we obtain all one-body and two-body environments along all columns. Performing the same procedure on the transposed network generates the two-body environments along all rows. Because the messages are cached in each stage, this procedure has the same cost as two contractions of the bilayer PEPS.
Replica Moment-Generating Functions
Computing the overlap parameter (3) requires computing correlations over all pairs of sites. Naively, this would necessitate contractions. To avoid this, we use an approach based on replica moment-generating functions (MGFs). To handle the quadratic dependence on the correlation matrix, we introduce a replica wavefunction and a Pauli-Z operator . This reduces the per-sample overlap parameter to
where denotes the average taken with respect to the replica wavefunction, and . Then its MGF is
where are the MGF parameters, and is a vector of replica Pauli-Z products. The overlap parameter reduces to
We can transform this into a form that requires a single contraction using the Hessian-vector product trick:
where is the all-ones vector. This only costs one evaluation of under automatic differentiation.
Training and Validation
To train the FTN, we perform transfer learning across system sizes from to . At each stage, the FTN is exposed only to systems of the new size. We use a single architecture for all sizes, with four transformer blocks of embedding width 64, eight attention heads, and an MLP with a single hidden layer of width
256 and GELU activation in each transformer block, and a readout MLP with two hidden layers of width 128 and tanh activations. At each size, we optimize the multi-Hamiltonian objective in Eq. (9) using 250,000 steps of Adam [38] with a cosine annealing [39] schedule from to .
For each step, we draw a new set of fields from a uniform distribution and disorder samples from the normal distribution mentioned above with a batch size of four. We compute the gradient with respect to the parameters via automatic differentiation of the energy computed by GridBP. In all cases, we use a bond dimension of and a bond dimension cutoff of during contraction. To lower computational costs and avoid instabilities in the SVD during automatic differentiation [18], we use randomized QRbased rounding [40]. We find that each successive size benefits significantly from the last and that only a few steps of additional training are needed to bring the error into an acceptable range, cf. Figure 2. This demonstrates the ability of the FTN to generalize across system sizes. We compare FTN results with exact diagonalization results at and with DMRG results for . We performed the entire training run on a single Nvidia A100 GPU in approximately 52 hours.
Figure 2. Convergence of energy and overlap parameter error during training. “Steps” denotes the total number of steps accumulated across all training phases. Dotted vertical lines denote transfer learning to a new system size.
We validate the FTN model by comparing it to brute-force DMRG [41], as well as imaginary-time evolution using simple update (SU) [21]. We run SU with a decreasing time-step schedule from 0.1 to and values of and matching those of the FTN. At each time-step size, convergence is determined by an absolute change in energy per site below for three consecutive checks, performed every 100 evolution steps. Similarly, we run DMRG sweeps at a maximum bond dimension of , with convergence determined by an absolute change in total energy below between successive sweeps. For the FTN, we compare the ground states generated directly from inference and after ten additional fine-tuning steps of energy optimization at a learning rate of . We measure the time required for each algorithm to generate its approximate ground state. We find that the FTN without post-inference optimization is as accurate as or more accurate than SU in almost all cases, cf. Figure 3(a),(Figure 3b), and is also faster, cf. Figure 3(c). Moreover, we observe that post-inference optimization of the FTN output does not significantly improve results, suggesting that the PEPS output by the FTN for each disorder realization is already near-optimal for its bond dimension.
Figure 3. Comparison of FTNs (with and without post-inference optimization), DMRG, and simple update (SU). Mean and maximum errors are denoted by closed and open markers, respectively. (a) Energy error vs. system size. (b) Overlap parameter (3) error vs. system size. (c) Wall time to compute the approximate ground state.
The Glass Transition
To study the critical phenomena of the EA-TFQIM, we use our FTN model to compute over 500,000 ground
| Parameter | Estimate | 95% CI |
| 2.121 | [2.115, 2.126] | |
| 1.128 | [1.104, 1.151] | |
| 1.918 | [1.914, 1.921] |
Table 1. Critical parameters of the glass transition obtained from finite-size scaling analysis.
states for different realizations of the disorder. This was possible using eight A6000 GPUs in roughly 24 hours since our combined methodology is able to generate a PEPS, fine-tune it, and measure its overlap parameter (Eq. 3) in under three seconds per sample. Indeed, we compute each overlap parameter using the replica MGF method, applied to the FTN output after post-inference optimization.
The transition is characterized by three critical parameters: , the critical field at which the transition occurs; , the scaling exponent of the correlation length; and , the scaling exponent of the order parameter. We use finite-size scaling (FSS) analysis to determine the critical parameters, cf. Figure 4. FSS posits that the rescaled curves should collapse to a single curve when plotted against . A widely used FSS ansatz for the curve is
Figure 4. Collapse of the rescaled overlap parameter to a single curve under finite-size scaling.
under the transformation
where , , and are coefficients. We fit the curves using nonlinear least squares from 128 random starts and estimate the 95% confidence intervals (CIs) by drawing 1024 bootstrapped samples with replacement.
Our results, reported in Table I, are in close agreement with other studies of the EA-TFQIM [6, 7, 42]. The large number (7168) of samples per point enables us to obtain narrow CIs quantifying uncertainty due to disorder under our model assumptions.
Acknowledgments
This work was supported by the U.S. Department of Energy, Office of Science, Accelerated Research in Quantum Computing Centers, Quantum Utility through Advanced Computational Quantum Algorithms, grant no. DE-SC0025572.
References
- [1]T. Castellani and A. Cavagna, Spin-glass theory for pedestrians, J. Stat. Mech. 2005, P05012 (2005).DOI
- [2]K. Binder and A. P. Young, Spin glasses: Experimental facts, theoretical concepts, and open questions, Rev. Mod. Phys. 58, 801 (1986).DOI
- [3]S. F. Edwards and P. W. Anderson, Theory of spin glasses, J. Phys. 5, 965 (1975).
- [4]H. R. Rieger and A. P. Young, Zero-temperature quantum phase transition of a two-dimensional ising spin glass, Phys. Rev. Lett. 72, 4141 (1994).DOI
- [5]H. Q. Lin, Exact diagonalization of quantum-spin models, Phys. Rev. B Condens. Matter 42, 6561 (1990).
- [6]I. González-Adalid Pemartín, V. Martín-Mayor, and G. Parisi, The quantum transition of the two-dimensional ising spin glass, Nature (2024).
- [7]L. Brodoloni and S. Pilati, Zero-temperature monte carlo simulations of two-dimensional quantum spin glasses guided by neural network states, Phys. Rev. E 110, 065305 (2024).DOI
- [8]P. Lunts, M. S. Albergo, and M. Lindsey, Non-hertz-millis scaling of the antiferromagnetic quantum critical metal via scalable hybrid monte carlo, Nat. Commun. 14, 2547 (2023).DOI
- [9]A. A. Patel, P. Lunts, and S. Sachdev, Localization of overdamped bosonic modes and transport in strange metals, Proc. Natl. Acad. Sci. U. S. A. 121, e2402052121 (2024).DOI
- [10]A. A. Patel, P. Lunts, and M. S. Albergo, Strange metals and planckian transport in a gapless phase from spatially random interactions, Phys. Rev. X 15, 031064 (2025).DOI
- [11]M. Suzuki, Relationship between d-dimensional quantal spin systems and (d+1)-dimensional ising systems: Equivalence, critical exponents and systematic approximants of the partition function and spin correlations, Prog. Theor. Phys. 56, 1454 (1976).DOI
- [12]S. Sorella, Wave function optimization in the variational monte carlo method, Phys. Rev. B Condens. Matter Mater. Phys. 71, 10.1103/physrevb.71.241103 (2005).
- [13]A. W. Sandvik and G. Vidal, Variational quantum monte carlo simulations with tensor-network states, Phys. Rev. Lett. 99, 220602 (2007).DOI
- [14]G. Carleo and M. Troyer, Solving the quantum many-body problem with artificial neural networks, Science 355, 602 (2017).DOI
- [15]H. Lange, A. Van de Walle, A. Abedinia, and A. Bohrdt, From architectures to applications: a review of neural quantum states, Quantum Sci. Technol. 9, 040501 (2024).
- [16]R. Rende, L. L. Viteritti, F. Becca, A. Scardicchio, A. Laio, and G. Carleo, Foundation neural-networks quantum states as a unified ansatz for multiple hamiltonians, Nat. Commun. 16, 7213 (2025).DOI
- [17]R. Orús, Tensor networks for complex quantum systems, Nature Reviews Physics 1, 538 (2019).DOI
- [18]H.-J. Liao, J.-G. Liu, L. Wang, and T. Xiang, Differentiable programming tensor networks, Phys. Rev. X. 9, 10.1103/physrevx.9.031041 (2019).DOI
- [19]G. Vidal, Efficient simulation of one-dimensional quantum many-body systems, Phys. Rev. Lett. 93, 040502 (2004).pubmed.ncbi.nlm.nih.gov/15323740
- [20]Z. Y. Xie, J. Chen, M. P. Qin, J. W. Zhu, L. P. Yang, and T. Xiang, Coarse-graining renormalization by higher-order singular value decomposition, Phys. Rev. B Condens. Matter Mater. Phys. 86, 045139 (2012).DOI
- [21]M. Lubasch, J. I. Cirac, and M.-C. Baňuls, Algorithms for finite projected entangled pair states, Phys. Rev. B Condens. Matter Mater. Phys. 90, 10.1103/physrevb.90.064425 (2014).DOI
- [22]R. T. Grimm, A. J. Staat, and J. D. Eaves, The integral decimation method for quantum dynamics and statistical mechanics, Quantum 10, 2064 (2026), 2506.11341v4.DOI
- [23]R. T. Grimm and J. D. Eaves, Direct numerical solutions to stochastic differential equations with multiplicative noise, Phys. Rev. Lett. 132, 267101 (2024).DOI
- [24]R. T. Grimm and J. D. Eaves, Accurate numerical simulations of open quantum systems using spectral tensor trains, J. Chem. Phys. 161, 234111 (2024).arxiv.org/abs/2407.11327
- [25]A. Strathearn, P. Kirton, D. Kilda, J. Keeling, and B. W. Lovett, Efficient non-markovian quantum dynamics using time-evolving matrix product operators, Nat. Commun. 9, 3322 (2018).DOI
- [26]G. Park, N. Ng, D. R. Reichman, and G. K.-L. Chan, Tensor network influence functionals in the continuous-time limit: Connections to quantum embedding, bath discretization, and higher-order time propagation, Phys. Rev. B 110, 045104 (2024).DOI
- [27]G. K.-L. Chan and S. Sharma, The density matrix renormalization group in quantum chemistry, Annu. Rev. Phys. Chem. 62, 465 (2011).DOI
- [28]N. Schuch, M. M. Wolf, F. Verstraete, and J. I. Cirac, Computational complexity of projected entangled pair states, Phys. Rev. Lett. 98, 140506 (2007).pubmed.ncbi.nlm.nih.gov/17501258
- [29]S.-J. Du, A. Chen, and G. K.-L. Chan, Neuralized fermionic tensor networks for quantum many-body systems, Phys. Rev. B 113, 10.1103/x8v1-qf14 (2026).DOI
- [30]Z. Chen, L. Newhouse, E. Chen, D. Luo, and M. Soljačić, ANTN: Bridging autoregressive neural networks and tensor networks for quantum many-body simulation, arXiv [quant-ph] (2023), arXiv:2304.01996 [quant-ph].arxiv.org/abs/2304.01996
- [31]A. Vaswani, N. Shazeer, N. Parmar, J. Uszkoreit, L. Jones, A. N. Gomez, L. Kaiser, and I. Polosukhin, Attention is all you need, Adv. Neural Inf. Process. Syst. 30 (2017).arxiv.org/abs/1706.03762
- [32]J. Houdayer and A. Hartmann, Low-temperature behavior of two-dimensional gaussian ising spin glasses, Phys. Rev. B Condens. Matter Mater. Phys. 70, 014418 (2004).DOI
- [33]J. I. Cirac, D. Pérez-García, N. Schuch, and F. Verstraete, Matrix product states and projected entangled pair states: Concepts, symmetries, theorems, Rev. Mod. Phys. 93, 045003 (2021).DOI
- [34]J. L. Ba, J. R. Kiros, and G. E. Hinton, Layer normalization, arXiv [stat.ML] 10.48550/arXiv.1607.06450 (2016), arXiv:1607.06450 [stat.ML].
- [35]M. Lubasch, J. I. Cirac, and M.-C. Baňuls, Unifying projected entangled pair state contractions, New J. Phys. 16, 033014 (2014).DOI
- [36]R. Alkabetz and I. Arad, Tensor networks contraction and the belief propagation algorithm, Phys. Rev. Res. 3, 023073 (2021).DOI
- [37]G. Evenbly, N. Pancotti, A. Milsted, J. Gray, and G. K.-L. Chan, Loop series expansions for tensor networks, Phys. Rev. Res. 8, 013245 (2026).arxiv.org/abs/2409.03108
- [38]D. P. Kingma and J. Ba, Adam: A method for stochas- tic optimization, arXiv [cs.LG] (2014), arXiv:1412.6980 [cs.LG].arxiv.org/abs/1412.6980
- [39]I. Loshchilov and F. Hutter, SGDR: Stochastic gradi- ent descent with warm restarts, arXiv [cs.LG] (2016), arXiv:1608.03983 [cs.LG].arxiv.org/abs/1608.03983
- [40]H. Al Daas, G. Ballard, P. Cezeaux, E. Hallman, A. Międlar, M. Pasha, T. W. Reid, and A. K. Saibaba, Randomized algorithms for rounding in the tensor-train format, SIAM J. Sci. Comput. 45, A74 (2023).
- [41]S. R. White, Density matrix formulation for quantum renormalization groups, Phys. Rev. Lett. 69, 2863 (1992).pubmed.ncbi.nlm.nih.gov/10046608
- [42]A. D. King, J. Raymond, T. Lanting, R. Harris, A. Zucca, F. Altomare, A. J. Berkley, K. Boothby, S. Ejtmae, C. Enderud, E. Hoskinson, S. Huang, E. Ladizinsky, A. J. R. MacDonald, G. Marsden, R. Molavi, T. Oh, G. Poulin-Lamarre, M. Reis, C. Rich, Y. Sato, N. Tsai, M. Volkmann, J. D. Whittaker, Y. Yao, W. Sandvik, and M. H. Amin, Quantum critical dynamics in a 5,000-qubit programmable spin glass, Nature 617, 61 (2023).DOI



