betaFeedback

WorksPhysics and Astronomy

Sampling the XY model with an attention-based neural network

Abstract

We present a transformer-based variational autoregressive network for sampling lattice systems with continuous degrees of freedom and apply it to the two-dimensional XY model near the Berezinskii-Kosterlitz-Thouless transition. The architecture is combined with physics-informed ingredients: locality-informed attention and approximate probabilities. The resulting sampler significantly outperforms existing neural samplers. We train the model on lattices up to 96×9696\times96, verify with a mode-dropping estimator that no parts of the configuration space are missed, and reproduce the energy, magnetization and helicity modulus obtained with the traditional Markov Chain Monte Carlo algorithm.

Introduction

Markov Chain Monte Carlo (MCMC) remains the method of choice for computing observables in statistical physics and lattice field theory. Its efficiency, however, is limited by autocorrelations between consecutive configurations, which grow rapidly in the vicinity of phase transitions (critical slowing down). For some systems this problem can be alleviated by cluster algorithms [1, 2], but such algorithms are model-specific and are not available for many systems of interest, e.g., frustrated magnets, spin glasses or gauge theories. In addition, MCMC does not give direct access to the partition function and hence to the free energy and entropy, which require dedicated techniques such as thermodynamic integration or the Wang–Landau algorithm [3].

In recent years, generative neural networks have been proposed as an alternative approach. A neural sampler is a model that provides both independent configurations and the exact, normalized probability qθq_\theta with which each of them was generated. The network is trained to approximate the Boltzmann distribution pp, typically by minimizing the reverse Kullback–Leibler divergence, which does not require any previously generated training data. Knowledge of qθq_\theta allows one to remove the remaining bias between qθq_\theta and pp, either by Neural Importance Sampling (NIS) or by using the network as an independent proposal in the Metropolis–Hastings algorithm (Neural Markov Chain Monte Carlo, NCMC) [4]. Both approaches are asymptotically exact, and also give a direct estimate of the free energy. Two families of architectures dominate this field: Variational Autoregressive Networks (VAN) [5], which factorize the probability into a product of conditional distributions of consecutive spins, and normalizing flows (NF) [6], which transform a simple prior distribution by a sequence of invertible maps.

For discrete spin systems, autoregressive networks have advanced considerably. Hierarchical Autoregressive Networks (HAN) [7, 8] and RiGCS [9] exploit the locality of interactions to reduce the number of spins on which each conditional distribution depends, physics-informed approximate probabilities [10, 11] inject an analytical approximation of the conditionals directly into the network, and transformer-based architectures [12, 13, 14] replace convolutional or dense layers with the attention mechanism [15]. Systems with continuous degrees of freedom are usually treated with normalizing flows, but the focus has mostly been on lattice field theories [6, 16, 17, 18, 19, 20].

Email addresses: [email protected] (Piotr Białas), [email protected] (Christian D. Morales), [email protected] (Tomasz Stebel), [email protected] (Dawid Zapolski)

The two-dimensional XY model is the natural benchmark for samplers of systems with continuous degrees of freedom: it is the simplest model with a continuous O(2)O(2) symmetry, it exhibits the Berezinskii–Kosterlitz–Thouless (BKT) transition [21, 22] driven by the unbinding of vortices, and it shares its compact, U(1)U(1)-valued variables with lattice gauge theories. Existing neural samplers for this model include: Continuous-mixture Autoregressive Networks (CAN) [23], normalizing flows [24], and flows trained with adversarial learning (AdvNF) [25]. As we show in this work, their sampling quality degrades quickly with the system size.

In this paper, we follow an approach similar to CAN and sample the XY model with an autoregressive architecture, spin by spin. The probability distribution is parameterized by a linear combination of simple distributions that are easy to sample from. The parameters of this combination are given by the output of the neural architecture. Two crucial differences between our approach and CAN are: (i) instead of a convolutional network, we use a transformer architecture; and (ii) we use von Mises functions that are more physically justified than the beta distributions used in [23]. We combine this architecture with three physics-motivated ingredients: locality-informed attention [14], which restricts the attention span according to the geometry of the lattice and the Hammersley–Clifford theorem [26]; approximate probabilities [11] in the lowest (t1t^1) order, which for the XY model stay within the von Mises family and can therefore be combined analytically with the network output; and averaging the model probability over a group of lattice symmetries, applied after training.

The paper is organized as follows. In § 2, we review neural samplers, asymptotically unbiased estimators, and diagnostics of mode collapse, and we describe the XY model and the tVAN architecture together with its physics-informed modifications. § 3 presents the numerical results: a comparison with other neural samplers, the effect of approximate probabilities and gauge fixing, scaling with the lattice size, physical observables, post-training symmetrization, and tVAN as a normalizing flow prior. We summarize in § 4. Derivations of the t1t^1 approximate probabilities, the proof that symmetrization does not decrease the ESS, and a study of hyperparameters are collected in the Appendices.

Method

Neural Samplers

Let us consider system of NN particles, with internal continuous degrees of freedom sis_i. The probability of finding a system in state s=(s1,s2,…,sN)\mathbf{s}=(s_1,s_2,\ldots,s_N) is given by the Boltzmann distribution:

p(s)=1Ze−βE(s),where Z=∫ds e−βE(s).(1)p(\mathbf{s}) = \frac{1}{Z}e^{-\beta E(\mathbf{s})}, \quad\text{where } Z = \int d\mathbf{s}\,e^{-\beta E(\mathbf{s})}. \tag*{(1)}

In the above expressions, E(s)E(\mathbf{s}) is the energy of a configuration, β\beta is the inverse temperature, and the integration ∫ds\int d\mathbf{s} is performed over the whole configuration space (in the case of discrete degrees of freedom, the integral is replaced by a sum).

The main idea of a neural sampler is to generate configurations from some distribution qθ(s)q_\theta(\mathbf{s}) that approximates well the (target) distribution p(s)p(\mathbf{s}). In practice, the trained distribution qθ(s)q_\theta(\mathbf{s}) will not reproduce p(s)p(\mathbf{s}) exactly, which leads to a bias in observables. The availability of qθ(s)q_\theta(\mathbf{s}) allows one to use importance sampling or the Metropolis-Hastings algorithm to correct the remaining bias between qθ(s)q_\theta(\mathbf{s}) and p(s)p(\mathbf{s}). For this reason, the chosen architecture in this work provides not only configurations but also their normalized probability density.

The free energy can be calculated from the partition function:

F=−1βlog⁡Z.(2)F = -\frac{1}{\beta}\log Z. \tag*{(2)}

The model is trained by minimizing the reverse Kullback-Leibler divergence:

DKL(qθ∥p)=∫ds qθ(s)log⁡qθ(s)p(s)≡⟨log⁡qθ(s)p(s)⟩qθ.(3)D_{\mathrm{KL}}(q_\theta\Vert p)=\int d\mathbf{s}\,q_\theta(\mathbf{s})\log\frac{q_\theta(\mathbf{s})}{p(\mathbf{s})}\equiv\left\langle\log\frac{q_\theta(\mathbf{s})}{p(\mathbf{s})}\right\rangle_{q_\theta}. \tag*{(3)}

Substituting the Boltzmann distribution, we get:

DKL(qθ∥p)=β(Fq−F),(4)D_{\mathrm{KL}}(q_\theta\Vert p)=\beta\left(F_{q}-F\right), \tag*{(4)}

where

Fq=1β⟨log⁡qθ(s)+βE(s)⟩qθ(5)F_q = \frac{1}{\beta}\left\langle\log q_\theta(\mathbf{s})+\beta E(\mathbf{s})\right\rangle_{q_\theta} \tag*{(5)}

is the variational free energy. Since FF is independent of θ\theta, minimizing FqF_q is in fact minimizing DKLD_{\mathrm{KL}}. Since DKL(qθ∥p)≥0D_{\mathrm{KL}}(q_\theta\Vert p) \ge0 it means that Fq≥FF_q \ge F.

Minimization of FqF_q can be done through the REINFORCE algorithm[5]. Let us define the signal:

Sθ(s)=log⁡qθ(s)+βE(s),(6)S_\theta(\mathbf{s}) = \log q_\theta(\mathbf{s})+\beta E(\mathbf{s}), \tag*{(6)}

then the gradient of the loss function is written as:

∇θFq=1β⟨(Sθ(s)−⟨Sθ(s)⟩qθ)∇θlog⁡qθ(s)⟩qθ.(7)\nabla_\theta F_q = \frac{1}{\beta}\left\langle\left(S_\theta(\mathbf{s})-\left\langle S_\theta(\mathbf{s})\right\rangle_{q_\theta}\right)\nabla_\theta\log q_\theta(\mathbf{s})\right\rangle_{q_\theta}. \tag*{(7)}

At each optimizer step, a batch of configurations of size 212=40962^{12}=4096, unless stated otherwise, is generated from qθq_\theta. Energy and log-probabilities of each configuration are then used to calculate the gradient of FqF_q, after which the model’s weights are updated.

Autoregressive representation

Let us consider a system of NN continuous spins:

s=(s1,s2,…,sN).(8)\mathbf{s}=(s_1,s_2,\ldots,s_N). \tag*{(8)}

We can define the joint probability density by factorizing it into a product of conditional densities:

qθ(s1,s2,…,sN)=q(s1)∏i=2Nqθ(si∣s<i),(9)q_\theta(s_1,s_2,\ldots,s_N)=q(s_1)\prod_{i=2}^{N}q_\theta(s_i\mid s_{<i}), \tag*{(9)}

where

s<i≡(s1,s2,…,si−1).(10)s_{<i}\equiv(s_1,s_2,\ldots,s_{i-1}). \tag*{(10)}

The selected model needs to parametrize the conditional densities qθ(si∣s<i)q_\theta(s_i\mid s_{<i}), hence it needs to satisfy the autoregressive property - output ii may depend only on inputs jj that satisfy j<ij<i. Configurations are produced using ancestral sampling. First, spin s1s_1 is drawn from a uniform distribution on the circle. Then the procedure is iterative: after s1,…,si−1s_1,\ldots,s_{i-1} are generated, the model evaluates qθ(si∣s<i)q_\theta(s_i\mid s_{<i}) and draws the next spin from this density. The process continues until all of the NN spins are generated.

Asymptotically unbiased observables

The VAN algorithm described above does not guarantee correct sampling from pp. It only provides a variational upper bound on thermodynamic quantities such as the free energy. For this reason, the variational estimate alone is never sufficient, and it must be supplemented by one of the asymptotically exact procedures [4]: Neural Markov Chain Monte Carlo (NMCMC) or Neural Importance Sampling (NIS).

NMCMC introduces an accept/reject step: a configuration proposed by the network, snext\mathbf{s}_{\mathrm{next}}, is accepted with probability

pacc=min⁡(1,qθ(sprevious)p(snext)qθ(snext)p(sprevious)),(11)p_{\mathrm{acc}}=\min\left(1,\frac{q_\theta(\mathbf{s}_{\mathrm{previous}})p(\mathbf{s}_{\mathrm{next}})}{q_\theta(\mathbf{s}_{\mathrm{next}})p(\mathbf{s}_{\mathrm{previous}})}\right), \tag*{(11)}

which is the Metropolis–Hastings rule for an independent proposal distribution.

The NIS compensates for the mismatch between pp and qθq_\theta by assigning each configuration an importance weight:

w(s)=e−βE(s)qθ(s).(12)w(\mathbf{s})=\frac{e^{-\beta E(\mathbf{s})}}{q_\theta(\mathbf{s})}. \tag*{(12)}

The expectation value of an observable OO in the target distribution can then be rewritten as an expectation value in the distribution generated by the network,

⟨O⟩p=∫ds p(s) O(s)=∫ds qθ(s) w(s) O(s)∫ds qθ(s) w(s).(13)\langle O\rangle_p = \int ds\,p(s)\,O(s) = \frac{\int ds\,q_\theta(s)\,w(s)\,O(s)}{\int ds\,q_\theta(s)\,w(s)}. \tag*{(13)}

In practice, integrals over the whole configuration space are estimated on a set of NstatN_{\mathrm{stat}} configurations si\mathbf{s}_i drawn independently from the distribution qθ(s)q_\theta(s):

⟨O⟩p≈∑i=1Nstatw(si) O(si)∑i=1Nstatw(si),si∼qθ.(14)\langle O\rangle_p \approx\frac{\sum_{i=1}^{N_{\mathrm{stat}}}w(\mathbf{s}_i)\,O(\mathbf{s}_i)}{\sum_{i=1}^{N_{\mathrm{stat}}}w(\mathbf{s}_i)}, \qquad\mathbf{s}_i \sim q_\theta. \tag*{(14)}

When Nstat→∞N_{\mathrm{stat}}\to\infty this estimator converges to the correct value for any qθq_\theta, provided only that qθ(s)>0q_\theta(s)>0 wherever p(s)>0p(s)>0, so that no region of the configuration space is inaccessible to the network. In this sense NIS, exactly like NMCMC, turns an imperfectly trained network into an asymptotically exact algorithm; the quality of the training determines the statistical error, not the correctness of the result.

The weights give direct access to the partition function itself,

ZNIS=∫ds qθ(s) w(s)≈1Nstat∑i=1Nstatw(si),si∼qθ(s),(15)Z_{\mathrm{NIS}} = \int ds\,q_\theta(s)\,w(s) \approx\frac{1}{N_{\mathrm{stat}}}\sum_{i=1}^{N_{\mathrm{stat}}}w(\mathbf{s}_i), \qquad\mathbf{s}_i \sim q_\theta(s), \tag*{(15)}

and hence to the free energy

FNIS=−1βlog⁡ZNIS.(16)F_{\mathrm{NIS}}=-\frac{1}{\beta}\log Z_{\mathrm{NIS}}. \tag*{(16)}

The difference with respect to the variational estimate FqF_q of Eq. (5) is essential. The estimate (15) is unbiased, and although taking its logarithm introduces a bias, it is of order 1/Nstat1/N_{\mathrm{stat}} and can be made arbitrarily small by increasing the statistics. The gap between FqF_q and FF, on the contrary, is a property of the trained network and does not decrease with the number of generated configurations.

The price to be paid is that the variance of the weights (12) may be very large. If qθq_\theta differs appreciably from pp, a few configurations with huge weights dominate the sums in Eq. (14) and the effective statistics is much smaller than NstatN_{\mathrm{stat}}. This is quantified by the effective sample size,

ESS=(∑i=1Nstatw(si))2Nstat∑i=1Nstatw(si)2∈(0,1],(17)\mathrm{ESS}=\frac{\left(\sum_{i=1}^{N_{\mathrm{stat}}}w(\mathbf{s}_i)\right)^2}{N_{\mathrm{stat}}\sum_{i=1}^{N_{\mathrm{stat}}}w(\mathbf{s}_i)^2}\in(0,1], \tag*{(17)}

which equals one when qθ=pq_\theta=p and tends to zero when the network distribution and the Boltzmann distribution differ strongly. The ESS is therefore both a measure of the quality of the trained model and the quantity which has to be monitored and reported together with every result: it plays, for NIS, the role that the inverse integrated autocorrelation time plays for NMCMC.

It should be stressed that a large ESS is a necessary but insufficient condition for correctness. If the network never generates configurations belonging to part of the configuration space, the weights cannot repair this, since reweighting can only redistribute the probability among configurations that have actually been generated. The same is true for the NMCMC method. Such mode collapse is particularly dangerous in systems where the structure of modes is not well known, for example, spin glasses. In practice, even if formally all configurations have qθ>0q_\theta>0, the mode collapse can still occur with finite NstatN_{\mathrm{stat}}.

In general, there is no method to detect mode collapse other than by comparing results with other methods. Assuming that one has access to samples generated from distribution pp, it is possible to measure mode collapse. In [27], it was proposed to use the following estimate of free energy in comparison to (16):

FMC=−1βlog⁡(ZMC)(18)F_{\mathrm{MC}}=-\frac{1}{\beta}\log\left(Z_{\mathrm{MC}}\right) \tag*{(18)}

where

ZMC−1=⟨qθ(s)e−βE(s)⟩p=1Nstat∑i=1Nstatqθ(si)e−βE(si),si∼p(19)Z_{MC}^{-1}=\left\langle\frac{q_\theta(\mathbf{s})}{e^{-\beta E(\mathbf{s})}}\right\rangle_p=\frac{1}{N_{\mathrm{stat}}}\sum_{i=1}^{N_{\mathrm{stat}}}\frac{q_\theta(\mathbf{s}_i)}{e^{-\beta E(\mathbf{s}_i)}},\qquad\mathbf{s}_i\sim p \tag*{(19)}

It can be shown that [27]:

FNIS≥F≥FMC,(20)F_{NIS}\ge F\ge F_{MC}, \tag*{(20)}

hence the difference between FNISF_{NIS} and FMCF_{MC} is a measure of mode seeking. Following [27], we define the mode-dropping estimator:

W≡ZNISZMC=eβ(FMC−FNIS).(21)\mathcal{W}\equiv\frac{Z_{NIS}}{Z_{MC}}=e^{\beta(F_{MC}-F_{NIS})}. \tag*{(21)}

A value of W\mathcal{W} below one indicates missing modes.

In this contribution, we shall use samples generated using an in-house Wolff MCMC code which is an implementation of [2]. This script performs 10410^4 thermalization cluster updates and stores 10610^6 configurations that are separated by nsepn_{\mathrm{sep}} cluster updates. The nsepn_{\mathrm{sep}} was chosen so that consecutive stored samples have τint≤1.02\tau_{\mathrm{int}}\le1.02. This means nsep=70,80,90n_{\mathrm{sep}}=70,80,90 for L=16,32,64L=16,32,64; nsep=90n_{\mathrm{sep}}=90 for L=96L=96 at β=1.0\beta=1.0 or 1.121.12; nsep=190n_{\mathrm{sep}}=190 for L=96L=96 at β=1.2\beta=1.2. Our script was checked against [28] at β=1.1199\beta=1.1199 and lattice sizes up to L=128L=128 and the observables agree within statistical error.

XY model

In this work we consider a 2D XY model defined on an L×LL\times L lattice, implying N=L2N=L^2. Each lattice site contains a two dimensional spin vector S⃗i\vec{S}_i of length one, and the energy of the configuration is defined as

E(s)=−J∑⟨i,j⟩S⃗i⋅S⃗j,(22)E(\mathbf{s})=-J\sum_{\langle i,j\rangle}\vec{S}_i\cdot\vec{S}_j, \tag*{(22)}

where only the nearest-neighbor interactions are included. Spins at each lattice site can be represented as two-component vectors1

S⃗i=(cos⁡si,sin⁡si),si∈[0,2π).(23)\vec{S}_i=(\cos s_i,\sin s_i),\qquad s_i\in[0,2\pi). \tag*{(23)}

We can rewrite Eq. (22) as

E(s)=−J∑⟨i,j⟩cos⁡(si−sj).(24)E(\mathbf{s})=-J\sum_{\langle i,j\rangle}\cos(s_i-s_j). \tag*{(24)}

In this work, we assume that the couplings are constant J=1J=1, and the periodic boundary conditions are imposed. In the thermodynamic limit, the model exhibits the famous BKT transition [21, 22] around an inverse temperature of β=1.12\beta=1.12.

Since the energy depends only on the angle differences, the model itself is invariant under global rotation by α\alpha, meaning

si⟶(si+α) mod 2π,E(s1+α,…,sN+α)=E(s1,…,sN).(25)s_i\longrightarrow(s_i+\alpha)\bmod2\pi,\qquad E(s_1+\alpha,\ldots,s_N+\alpha)=E(s_1,\ldots,s_N). \tag*{(25)}

This is a global gauge symmetry, and we can use it to fix the first spin to s1=0s_1=0 during generation. The contribution of q(s1)q(s_1) to the probability density qθq_\theta is retained and equals that of a uniformly drawn spin:

q(s1)=12π(26)q(s_1)=\frac{1}{2\pi} \tag*{(26)}

This allows us to rewrite the logarithm of Eq. (9) as

log⁡qθ(s)=−log⁡2π+∑i=2Nlog⁡qθ(si∣s<i)(27)\log q_\theta(\mathbf{s})=-\log2\pi+\sum_{i=2}^{N}\log q_\theta(s_i\mid s_{<i}) \tag*{(27)}

In what follows, we refer to sis_i as spins for simplicity, although they formally represent angular variables. Since there is a one-to-one correspondence between sis_i and the spin vectors S⃗i\vec{S}_i, this terminology should not cause any ambiguity.

Transformer architecture

An autoregressive transformer [15] with causally masked attention outputs a sequence of elements by predicting each of them from the previous ones. Usually, in this type of model, elements are treated as tokens from a finite vocabulary, and the output at a given position is a categorical distribution over the values from said vocabulary. The tVAN [13] that was previously introduced for discrete spin systems, specifically the Ising and Edwards–Anderson models, uses this exact idea: the spins or patches of spins are treated as tokens in the model’s vocabulary, and the network assigns a probability to each existing token.

This approach cannot be used for an XY model directly without binning the spins, since each lattice site carries a continuous value si∈[0,2π)s_i \in[0, 2\pi) and every qθ(si∣s<i)q_\theta(s_i \mid s_{<i}) is a probability density rather than a finite set of probabilities. In order to utilize tVAN’s approach, we need to adapt the interface through which the model ingests spin values and outputs spin probability densities.

At the input, the spin needs to be represented by some form of a continuous value. A simple scalar representation of the angle introduces a continuity problem at the 0→2π0 \to2\pi; two physically close spins would have numerically distant representations. Therefore, every spin is represented as a pair (cos⁡si,sin⁡si)(\cos s_i, \sin s_i). The two components are mapped to the embedding space, and a learned positional embedding is used to provide the network location of spins on different lattice sites.

The embedded sequence is processed by transformer blocks. Each block consists of multi-head self-attention, a position-wise feed-forward network, and residual connections with layer normalization. A causal attention mask ensures that the network uses only previously generated spins to construct qθ(si∣s<i)q_\theta(s_i \mid s_{<i}). Attention used in this work additionally restricts the context by following the geometry of the lattice and will be described in the following section.

At the output, the transformer’s result determines the conditional density qθ(si∣s<i)q_\theta(s_i \mid s_{<i}). In this work we parametrize it with a mixture of von Mises distributions,

qθ(si∣s<i)=∑j=1Kωij VM(si;μij,κij),(28)q_\theta(s_i \mid s_{<i}) = \sum_{j=1}^{K} \omega_{ij}\,\mathrm{VM}(s_i;\mu_{ij},\kappa_{ij}), \tag*{(28)}

where

VM(s;μ,κ)=eκcos⁡(s−μ)2πI0(κ),κ≥0(29)\mathrm{VM}(s;\mu,\kappa) = \frac{e^{\kappa\cos(s-\mu)}}{2\pi I_0(\kappa)}, \qquad\kappa\ge0 \tag*{(29)}

and I0I_0 is a modified Bessel function of the first kind.

The choice of von Mises distribution is dictated by the fact that it closely resembles the Boltzmann distribution of the energy of the XY model. In principle, any distribution can be used for this purpose, but choosing von Mises allows for efficient qθq_\theta representation even for small values of KK. To sample from the von Mises distribution, we utilize inverse-CDF sampling [29] instead of a standard rejection algorithm, this gives every sample the same cost, without introducing rejections, and makes sampling more efficient on the GPU.

The transformer produces 3K3K real-number outputs at each site that we will label as aij,bij,cija_{ij}, b_{ij}, c_{ij}, which are connected to von Mises parameters (28) by:

ωij=softmax⁡(aij),μij=bij,κij=softplus⁡(cij).(30)\omega_{ij} = \operatorname{softmax}(a_{ij}), \qquad\mu_{ij} = b_{ij}, \qquad\kappa_{ij} = \operatorname{softplus}(c_{ij}). \tag*{(30)}

Locality-informed attention

Causal attention allows the model to generate an output that is dependent only on previously generated tokens. Usually this implies that the model’s output uses all of the preceding tokens, but in the case of the XY model, this is not strictly necessary. Since Eq. (22) contains only nearest-neighbor interactions, one can show that restricting the attention span of the model while considering that the lattice’s geometry does not reduce tVAN’s quality. Formally, this is a consequence of the Hammersley-Clifford theorem [26] and was used to propose Hierarchical Autoregressive Networks (HAN) [7, 8]. In the context of the Ising model and transformer-based architectures, it was first used in [14].

We can rewrite our conditional density as follows

p(si∣s<i)=p(si∣sBi)(31)p(s_i \mid s_{<i}) = p(s_i \mid s_{B_i}) \tag*{(31)}

where

Bi={j<i∣j≤L or i−j≤L},(32)B_i = \{j < i \mid j \le L \text{ or } i-j \le L\}, \tag*{(32)}

which consists of the most recently generated LL spins and the entire first row - see Figure 1 where darker circles represent B21B_{21} in a 6×66 \times6 system.

Figure 1. Schematic view of spin-by-spin configuration generation, for L=6L = 6. Solid circles represent already generated spins, and the red spin (s21s_{21}) is about to be generated. Locality-informed attention means that the conditional probability of a spin depends only on two rows of previously generated spins (darker circles). The approximate probabilities depend on two spins: top and left (for spin s21s_{21} this means s21,top=s15s_{21,top}=s_{15} and s21,left=s20s_{21,left}=s_{20}).

Approximate probabilities

The approach of using probability priors to inject physics awareness into the model was developed in [11] (see also [10] for a similar approach to physically inspired improvements). It introduces a simple approximation to the conditional distribution directly into the neural sampler, which now leaves the network to learn only the remaining difference.

In this work, we consider priors depending only on two nearest-neighbor spins - this is the lowest-order t1t^1 approximation considered in [11]. Higher-order approximations were considered, but t2t^2 no longer stays in the von Mises family. The derivation is given in Appendix § 6; here we present the final result.

The conditional density with approximate probabilities (AP) can be expressed as

qθ(si∣s<i)=∑j=1Kω~ij VM(si;μ~ij,κ~ij),(33)q_\theta(s_i \mid s_{<i}) = \sum_{j=1}^{K} \widetilde{\omega}_{ij}\,\mathrm{VM}(s_i;\widetilde{\mu}_{ij},\widetilde{\kappa}_{ij}), \tag*{(33)}

where

ω~ij=ωijI0(κ~ij)I0(κij)(∑l=1KωilI0(κ~il)I0(κil))−1,(34)\widetilde{\omega}_{ij} = \omega_{ij}\frac{I_0(\widetilde{\kappa}_{ij})}{I_0(\kappa_{ij})}\left(\sum_{l=1}^{K}\omega_{il}\frac{I_0(\widetilde{\kappa}_{il})}{I_0(\kappa_{il})}\right)^{-1}, \tag*{(34)}
κ~ij=A~ij2+B~ij2,μ~ij=atan2⁡(B~ij,A~ij),(35)\widetilde{\kappa}_{ij} = \sqrt{\widetilde{A}_{ij}^{2}+\widetilde{B}_{ij}^{2}}, \qquad\widetilde{\mu}_{ij} = \operatorname{atan2}(\widetilde{B}_{ij},\widetilde{A}_{ij}), \tag*{(35)}
A~ij=κijcos⁡(μij)+κiAPcos⁡(μiAP),B~ij=κijsin⁡(μij)+κiAPsin⁡(μiAP),(36)\begin{aligned} \widetilde{A}_{ij} &= \kappa_{ij}\cos(\mu_{ij})+\kappa_i^{\mathrm{AP}}\cos(\mu_i^{\mathrm{AP}}),\\ \widetilde{B}_{ij} &= \kappa_{ij}\sin(\mu_{ij})+\kappa_i^{\mathrm{AP}}\sin(\mu_i^{\mathrm{AP}}), \tag*{(36)} \end{aligned}
κiAP=βAi2+Bi2,μiAP=atan2⁡(Bi,Ai).(37)\kappa_i^{\mathrm{AP}} = \beta\sqrt{A_i^2+B_i^2}, \qquad\mu_i^{\mathrm{AP}} = \operatorname{atan2}(B_i,A_i). \tag*{(37)}

In the above expressions ωij\omega_{ij}, μij\mu_{ij}, κij\kappa_{ij} are obtained from the output of the transformer according to Eq. (30), whereas AiA_i and BiB_i are directly computed from two neighboring, already fixed spins

Ai=cos⁡(si,left)+cos⁡(si,top),Bi=sin⁡(si,left)+sin⁡(si,top),(38)A_i = \cos(s_{i,left})+\cos(s_{i,top}), \qquad B_i = \sin(s_{i,left})+\sin(s_{i,top}), \tag*{(38)}

where we used the notation si,tops_{i,\mathrm{top}} and si,lefts_{i,\mathrm{left}} since spins are generated in the raster order (see Figure 1).

Results

All presented results in this section were obtained with the same transformer architecture: a single transformer layer, an embedding dimension of 128, and 4 attention heads. We also use K=4K = 4, which means that for each spin the output of the transformer is 12 real numbers. For the optimizer, we used AdamW with β1=0.9\beta_1 = 0.9, β2=0.95\beta_2 = 0.95 and a constant learning rate 0.001. In Appendix C we discuss the dependence of training efficiency on the number of layers and KK.

Comparison with existing neural samplers

We start by comparing tVAN with two neural samplers that have already been used for continuous lattice systems: Continuous-mixture Autoregressive Network (CAN) introduced in Ref. [23] and a normalizing-flow sampler [24].

CAN was introduced as a model for continuous spin systems and was applied to the two-dimensional XY model, using PixelCNN as its autoregressive representation and utilizing a mixture of Beta distributions to represent the autoregressive conditional. In this comparison, we used the implementation of Ref. [23]: 3 masked convolutional layers with 32 channels and kernel of size 3×33 \times3. The conditional density is constructed as a mixture of 50 Beta distributions.

An approach using normalizing flows [30] represents a probability density through an invertible transformation of some prior distribution. We use the NeuMC package [24] to construct a model with 16 circular-spline[31] coupling layers. We used splines with nine knots. Each layer contains four convolution layers with a 3×33 \times3 kernel and 16 channels with LeakyReLU activation functions in between. The prior distribution was uniform on [0,2π)[0, 2\pi) for each spin ss. We minimized the reverse Kullback–Leibler divergence using the path-gradient estimator [32] and optimized the parameters with Adam at a learning rate of 5×10−45 \times10^{-4}.

Figure 2 shows a comparison of ESS across all three models at a lattice size of L=16L = 16 and an inverse temperature of β=1.12\beta= 1.12. The horizontal axis denotes optimizer steps, and the vertical axis represents ESS on a logarithmic scale. The training of the CAN architecture does not increase the ESS significantly, as ESS is volatile and sits at around 10−310^{-3} – 10−210^{-2}, whereas NF plateaus at 10−210^{-2}, and tVAN rapidly approaches 1 and plateaus. For bigger system sizes, CAN and NF fail to train.

Figure 2. ESS comparison of CAN, NF, and tVAN through optimizer steps.

A stronger baseline can be found in the work of [25], which uses Adversarial Learning for Normalizing Flows (AdvNF). This study provides an average acceptance rate (AR) of the Markov chain (11) as an evaluation metric for the sampler. At size L=32L = 32, AdvNF achieves the best AR of 7.2% when averaged over 10 temperatures T∈[0.85,1.25]T \in[0.85, 1.25]. In comparison, tVAN reaches AR of 80.48% at the same L=32L = 32 directly at β=1.12\beta= 1.12.

Physics-informed improvements

In this section, we will compare how injecting probability priors and using gauge fixing affect training efficiency. This comparison consists of four different trained models at L=32L = 32 and β=1.12\beta= 1.12, covering all combinations of gauge fixing and AP switched on and off. For readability, the ESS was computed as a 500-step mean.

Figure 3 shows all four of the training curves for each of the model types. From the graph we can clearly see that the AP contribution to training speed and quality is significant. AP increases the ESS by a factor of 1.5–1.8 after 20k steps. On the other hand, global gauge-fixing yields no visible ESS advantage.

Figure 3. Effect of gauge fixing and AP on ESS through optimizer steps.

Results for different lattice sizes

We now explore the results of tVAN on different lattice sizes. The lattice sizes span from L=16L = 16 to L=96L = 96. Models are trained at β=1.12\beta= 1.12, utilizing spin-fixing and enabling AP throughout all of the runs. For each lattice size, we trained the model for 2×1042 \times10^4 steps. After training, 10610^6 configurations are collected from the model and used to measure ESS. To calculate FMC (18) we used 10610^6 Wolff configurations.

Table 1 shows ESS and W\mathcal{W} throughout the lattice sizes. We can see that at L=16L = 16 tVAN reaches ESS of 0.976, which decreases to 0.834 at L=32L = 32. The degradation becomes considerably stronger as the lattice size increases with the model reaching an ESS of 0.160 at L=64L = 64 and 0.074 at L=96L = 96. On the other hand, across all of the entries W\mathcal{W} remains consistent with 1, which indicates that this drop in ESS is not related to a problem with missing modes. This is important, as it shows that NIS/NMCMC can compensate for the difference between qθq_\theta and pp.

LLβ\betaESSW\mathcal{W}
161.120.9759350.99997 ±\pm 0.00022
321.120.8336631.00029 ±\pm 0.00062
641.120.1599950.99961 ±\pm 0.00238
961.120.0743160.99828 ±\pm 0.00556

Table 1. ESS and mode-dropping estimator comparison using 10610^6 tVAN samples and 10610^6 Wolff configurations.

Physical observables

While ESS provides a good metric for estimating the model’s training and sampling performance, and W\mathcal{W} estimates the quality of the sampler in terms of mode collapse, we have not yet established if the model is capturing the underlying system’s physics. To assess this, we compare the model’s predictions with MCMC on a 96×9696 \times96 lattice at inverse temperatures of β∈{1.0,1.12,1.2}\beta\in\{1.0, 1.12, 1.2\}. As observables, we compare the mean energy ⟨E⟩\langle E\rangle and the mean absolute magnetization

∣m∣=1N⟨∣∑isi∣⟩p(39)|m| = \frac{1}{N}\left\langle\left|\sum_i s_i\right| \right\rangle_p \tag*{(39)}

where NN is the number of lattice sites.

We also define the helicity modulus Υ\Upsilon [33]

Υ=−1L2⟨E(s)/2⟩p−βL2⟨(Sx2(s)+Sy2(s))/2⟩p(40)\Upsilon= -\frac{1}{L^2}\langle E(s)/2\rangle_p-\beta L^2\left\langle\left(S_x^2(s)+S_y^2(s)\right)/2\right\rangle_p \tag*{(40)}

where

Sx(s)=1L2∑⟨i,j⟩xsin⁡(si−sj)andSy(s)=1L2∑⟨i,j⟩ysin⁡(si−sj).(41)S_x(s) = \frac{1}{L^2}\sum_{\langle i,j\rangle_x}\sin(s_i-s_j) \qquad\text{and} \qquad S_y(s) = \frac{1}{L^2}\sum_{\langle i,j\rangle_y}\sin(s_i-s_j). \tag*{(41)}

In the above expressions, ∑⟨i,j⟩x\sum_{\langle i,j\rangle_x} (∑⟨i,j⟩y\sum_{\langle i,j\rangle_y}) means that the sum is over all links in the horizontal (vertical) direction. Definition (40) is equivalent to the one from Ref. [33] because our system is symmetric with respect to interchanging the directions x↔yx \leftrightarrow y. We verified this by explicitly comparing our results for Υ\Upsilon with those from Ref. [33].

In addition to physical observables, we also consider the mode-dropping estimator (21).

Figure 4 provides this comparison for three β\beta at L=96L = 96. All mean values and uncertainties are divided by the MCMC mean, so that all observables can be shown on a single plot. For calculating each observable, we use 10610^6 tVAN and 10610^6 Wolff configurations. Errors were calculated using the bootstrap method.

Figure 4. Energy (EE), absolute magnetization (∣m∣|m|), helicity modulus Υ\Upsilon, and W\mathcal{W} at lattice size L=96L = 96 across β∈{1.0,1.12,1.2}\beta\in\{1.0, 1.12, 1.2\}.

For all three inverse temperatures β\beta, we can clearly see that tVAN observables reproduce MCMC results for all three observables up to statistical fluctuations.

Model symmetrization

Up to now, the results we have presented have been obtained using configurations and their probabilities qθq_\theta. The autoregressive factorization explicitly breaks many lattice symmetries that the system possesses due to the periodic boundary conditions. Ref. [5] suggested, in the context of VAN, using the averaged probability qθq_\theta during training so that the model is exposed to flipped spin configurations s→−ss \to-s. This approach can be trivially extended to any system symmetry and has often been used in our previous works [34, 35, 36]. Here we will simplify the method and apply symmetrization only after training. This means that the training is unaltered compared to previous sections; however, at the generation stage, when one assesses the sampler’s accuracy or calculates physical observables, the probability density is averaged. In Appendix § 7, we show a proof that such symmetrization leads to an ESS no smaller than the original one.

For a given configuration s\mathbf{s}, we define the averaged probability:

qˉθ(s)=1∣G∣∑g∈Gqθ(gs),(42)\bar{q}_\theta(\mathbf{s}) = \frac{1}{|G|}\sum_{g\in G}q_\theta(g\mathbf{s}), \tag*{(42)}

where GG is a finite (with ∣G∣|G| elements) group of symmetry transformations, i.e. p(gs)=p(s)p(g\mathbf{s}) = p(\mathbf{s}), g∈Gg \in G. In what follows, we shall use groups of square lattice symmetries with periodic boundary conditions. These include the translations TnT_n, where the lattice is shifted by kL/nkL/n, k=1,2,…,n−1k = 1, 2, \ldots, n - 1 sites in both directions; the rotations of the lattice C4C_4; the reflection of the lattice along the horizontal axis M2M_2; and finally, D4D_4, the dihedral group of order 8 (a group of a square’s symmetries).

It is worth noting that in the VAN approach, in particular the gradient estimator (7), the configuration s\mathbf{s} is first generated from the model spin by spin, and then the probability of this configuration qθ(s)q_\theta(\mathbf{s}) is calculated on the whole configuration. The difference between the forward pass through the network for generating configurations and the evaluation of their probabilities lies in the gradient tracing: the former is done without gradients, while the latter needs them, which significantly slows down the pass. For these reasons, using (42) during training for large ∣G∣|G| is numerically costly. On the other hand, post-training generation does not require gradient tracing; hence, symmetrization is less expensive.

The symmetrization requires substituting qθ→qˉθq_\theta\to\bar{q}_\theta in all definitions we introduced in previous sections, in particular, the definitions of FqF_q and ESS. The average ⟨…⟩qˉθ\langle\ldots\rangle_{\bar{q}_\theta} requires sampling from qˉθ\bar{q}_\theta. This, in practice, can be done by first sampling configurations from qθq_\theta and then randomly applying a given transformation g∈Gg \in G, with the probability of applying it equal to 1/∣G∣1/|G|. When the observable OO is invariant under GG, the procedure of randomly applying group transformations doesn’t change the result, hence

⟨O⟩qˉθ=⟨O⟩qθ.(43)\langle O\rangle_{\bar{q}_\theta} = \langle O\rangle_{q_\theta}. \tag*{(43)}

In the cases we will consider in this manuscript, this condition is satisfied.2

Table 2 shows the effect of symmetrization on the sampling. A model was trained at L=96L = 96 and β=1.12\beta= 1.12, and then 10610^6 configurations were generated. II is a model that has not been symmetrized (as considered in the previous sections).

Group∣G∣|G|FqF_qESSESS Gain vs. IICost vs. II
II1-2.3168308(1)0.066 ± 0.0131.01.0
T2T_24-2.31686946(9)0.384 ± 0.0046.1 ± 1.21.2
T4T_416-2.31688522(8)0.542 ± 0.0038.6 ± 1.71.7
M2M_22-2.3168561(1)0.182 ± 0.0312.9 ± 0.71.1
C4C_44-2.31687830(8)0.381 ± 0.0376.0 ± 1.31.2
D4D_48-2.31688866(7)0.470 ± 0.0537.4 ± 1.61.3
T4×C4T_4 \times C_464-2.31690352(5)0.808 ± 0.00112.8 ± 2.64.0

Table 2. Symmetry comparison for L=96L = 96 and β=1.12\beta= 1.12, evaluated using 10610^6 samples.

We first notice that symmetrization, even with very small groups, dramatically improves ESS; the gain ranges from a factor of 3 to 6 for group sizes 2≤∣G∣≤42 \le|G| \le4, at a cost increase of only 10–20%. For example, applying a group of 4 lattice rotations C4C_4 increases the ESS from 0.066 to 0.38. In general, the larger the group is, the more gain in ESS it provides, but this comes at the cost of increasing the simulation expense – for a group with 64 elements, the ESS reached a value of 0.81, but the cost increased 4 times compared to no symmetrization.

In Figure 5, we show the helicity modulus Υ\Upsilon obtained after symmetrization with different groups. Parameters were chosen as above. All the results after symmetrization are consistent with the Wolff MCMC result. The reduction of uncertainties (obtained using the bootstrap method) is due to increased effective statistics and is consistent with the ESS values.

Figure 5. Helicity modulus Υ\Upsilon for L=96L = 96, β=1.12\beta= 1.12 obtained from 10610^6 samples and different symmetrization groups.

The results presented in this section were obtained with gauge fixing; however, we explicitly checked that without using it the gain from symmetrization is comparable. The reason for this is that lattice symmetries are already broken by the use of the conditional probability decomposition (9); hence, the gauge fixing does not change this.

tVAN as NF prior

In section 3.1 we used NF with a simple prior distribution: uncorrelated variables drawn from a uniform distribution on a circle. This is the standard setup for NFs; however, it does not lead to a high-quality sampler. Here we test an alternative approach, where, instead of a simple prior, we use the tVAN sampler. Since much of the system’s structure is learned by tVAN, the NF only has to correct the remaining difference between the proposal and target distributions, instead of learning it from the ground up. For a trivial NF, which is the identity transformation, we restore tVAN.

In Tab 3 we show the comparison of a tVAN and an NF with a tVAN prior for L=32L = 32. Adding the NF increases the ESS only marginally. For larger system sizes, the difference is even smaller, or the NF even deteriorates the quality of sampling. We attribute this behavior to the low performance of the used NF architecture, as demonstrated in section 3.1.

ModelESS
tVAN0.826
tVAN+NF0.844

Table 3. Comparison at L=32L = 32 and β=1.12\beta= 1.12 using 10610^6 samples: pure tVAN vs. NF with a tVAN prior.

Summary

In this work we have extended the transformer-based Variational Autoregressive Network (tVAN) from discrete to continuous spin systems and applied it to the two-dimensional XY model in the vicinity of the BKT transition. Each spin enters the network as a pair (cos⁡si,sin⁡si)(\cos s_i,\sin s_i), and the conditional densities are parametrized by mixtures of von Mises distributions. This choice provides normalized densities on the circle that are easy to sample from and closely resemble the local Boltzmann factor of the model, so that K=4K = 4 mixture components and a single transformer layer already suffice. The architecture was supplemented with locality-informed attention, which restricts the attention span to the spins on which the exact conditional distribution depends, and with t1t^1 approximate probabilities, which for the XY model remain within the von Mises family and can be combined with the network output analytically.

The resulting sampler outperforms existing neural samplers for the XY model by a large margin. At L=16L = 16, where CAN and a normalizing flow reach an effective sample size of order 10−210^{-2} or lower, tVAN reaches an ESS close to one, and at L=32L = 32 its Metropolis–Hastings acceptance rate of about 80% should be compared with the 7.2% reported for AdvNF. Approximate probabilities substantially improve training, while fixing the global O(2)O(2) rotation does not lead to a visible gain. We trained the model on lattices up to 96×9696 \times96. The ESS decreases with the system size, from 0.98 at L=16L = 16 to 0.07 at L=96L = 96, but the mode-dropping estimator remains consistent with one for all studied sizes, showing that the decrease is not caused by missing parts of the configuration space and can be compensated by NIS or NMCMC. Indeed, the energy, magnetization and helicity modulus at L=96L = 96 agree with the results of the Wolff algorithm within statistical uncertainties for all three studied temperatures.

We have also shown that averaging the model probability over a group of lattice symmetries after training provably does not decrease the ESS, and in practice increases it dramatically. At L=96L = 96, averaging over four lattice rotations raises the ESS from 0.07 to 0.38 at the cost of a 20% increase in the generation time, and averaging over 64 transformations yields an ESS of 0.81. Since the cost grows roughly linearly with the size of the group, small groups offer the best gain per unit of computational time. As no training is involved, this procedure can be applied to any autoregressive sampler at negligible implementation cost. On the other hand, using tVAN as a prior for a normalizing flow improves the ESS only marginally, which we attribute to the limited expressivity of the flow architecture.

A natural next step is the application of the method to lattice gauge theories. Given the progress in NF-based samplers for lattice field theories, the combination of the tVAN prior with such architectures seems like a promising route.

Acknowledgments

We gratefully acknowledge Polish high-performance computing infrastructure PLGrid (HPC Center: ACK Cyfronet AGH) for providing computer facilities and support within computational grant no. PLG/2025/018811. T.S. and D.Z. kindly acknowledge the support of the Polish National Science Center (NCN) Grant No. 2021/43/D/ST2/03375.

t1^{1} approximate probabilities for the XY model

The conditional density is written as

p(si∣s<i)=∫ds>i e−βE(s)∫ds≥i e−βE(s)(44)p(s_i \mid s_{<i}) = \frac{\int ds_{>i}\, e^{-\beta E(s)}}{\int ds_{\geq i}\, e^{-\beta E(s)}} \tag*{(44)}

where ds>i=dsi+1dsi+2…dsNds_{>i} = ds_{i+1}ds_{i+2}\ldots ds_N. Since the denominator does not depend on sis_i,

p(si∣s<i)∝∫ds>i e−βE(s)(45)p(s_i \mid s_{<i}) \propto\int ds_{>i}\, e^{-\beta E(s)} \tag*{(45)}

Decomposing the energy into an internal part E(s<i)E(s_{<i}), which contains interactions only between already generated spins, and E(s≥i)E(s_{\geq i}), which contains the remaining contribution to EE, we get

p(si∣s<i)∝e−βE(s<i)∫ds>i e−βE(s≥i)(46)p(s_i \mid s_{<i}) \propto e^{-\beta E(s_{<i})} \int ds_{>i}\, e^{-\beta E(s_{\geq i})} \tag*{(46)}

The factor of E(s<i)E(s_{<i}) does not depend on sis_i and can be absorbed into a normalization constant,

p(si∣s<i)∝∫ds>i e−βE(s≥i)(47)p(s_i \mid s_{<i}) \propto\int ds_{>i}\, e^{-\beta E(s_{\geq i})} \tag*{(47)}

For the t1^{1} approximation, we consider interactions between sis_i and its already generated closest neighbors, denoted as si,lefts_{i,\mathrm{left}} and si,tops_{i,\mathrm{top}}, (see Figure 1):

E(si)=−(cos⁡(si−si,left)+cos⁡(si−si,top))(48)E(s_i) = -\left(\cos(s_i - s_{i,\mathrm{left}}) + \cos(s_i - s_{i,\mathrm{top}})\right) \tag*{(48)}

The t1^{1} approximation means E(s≥i)≈E(si)E(s_{\geq i}) \approx E(s_i), substituting that into Eq. (A.4), we get

p(si∣s<i)≈p~(si∣s<i)∝∫ds>i eβ(cos⁡(si−si,left)+cos⁡(si−si,top))(49)p(s_i \mid s_{<i}) \approx\tilde{p}(s_i \mid s_{<i}) \propto\int ds_{>i}\, e^{\beta(\cos(s_i-s_{i,\mathrm{left}})+\cos(s_i-s_{i,\mathrm{top}}))} \tag*{(49)}

Since the integration happens over the future spins s>is_{>i}, p~\tilde{p} can be rewritten as

p~(si∣s<i)∝eβ(cos⁡(si−si,left)+cos⁡(si−si,top)).(50)\tilde{p}(s_i \mid s_{<i}) \propto e^{\beta(\cos(s_i-s_{i,left})+\cos(s_i-s_{i,top}))}. \tag*{(50)}

This is because the integrand does not depend on the integration variables; hence, the integral is a constant that is absorbed in normalization.

The sum of cosines could be transformed as follows

cos⁡(si−si,left)+cos⁡(si−si,top)=Aicos⁡(si)+Bisin⁡(si)(51)\cos(s_i-s_{i,left})+\cos(s_i-s_{i,top})=A_i\cos(s_i)+B_i\sin(s_i) \tag*{(51)}

where

Ai=cos⁡(si,left)+cos⁡(si,top),Bi=sin⁡(si,left)+sin⁡(si,top)(52)A_i=\cos(s_{i,left})+\cos(s_{i,top}),\qquad B_i=\sin(s_{i,left})+\sin(s_{i,top}) \tag*{(52)}

Defining

Ri=Ai2+Bi2,μiAP=atan2⁡(Bi,Ai)(53)R_i=\sqrt{A_i^2+B_i^2},\qquad\mu_i^{AP}=\operatorname{atan2}(B_i,A_i) \tag*{(53)}

yields

Aicos⁡(si)+Bisin⁡(si)=Ricos⁡(si−μiAP)(54)A_i\cos(s_i)+B_i\sin(s_i)=R_i\cos\left(s_i-\mu_i^{AP}\right) \tag*{(54)}

By substituting the result into Eq. (A.7), we can see that the t1t^1 approximation is itself a von Mises distribution; with normalization, we have

p~i(1)(si∣s<i)=eκiAPcos⁡(si−μiAP)2πI0(κiAP),κiAP=βRi.(55)\tilde{p}_i^{(1)}(s_i\mid s_{<i})=\frac{e^{\kappa_i^{AP}\cos(s_i-\mu_i^{AP})}}{2\pi I_0(\kappa_i^{AP})},\qquad\kappa_i^{AP}=\beta R_i. \tag*{(55)}

Distribution (28) is a mixture of von Mises distributions and, before applying t1t^1, has the form of

qθ(0)(si∣s<i)=∑j=1Kωijeκijcos⁡(si−μij)2πI0(κij).(56)q_\theta^{(0)}(s_i\mid s_{<i})=\sum_{j=1}^{K}\omega_{ij}\frac{e^{\kappa_{ij}\cos(s_i-\mu_{ij})}}{2\pi I_0(\kappa_{ij})}. \tag*{(56)}

Multiplying qθ(0)q_\theta^{(0)} by the t1t^1 factor (A.12) yields us

qθ(si∣s<i)∝∑j=1Kωijeκijcos⁡(si−μij)2πI0(κij)eκiAPcos⁡(si−μiAP)2πI0(κiAP).(57)q_\theta(s_i\mid s_{<i})\propto\sum_{j=1}^{K}\omega_{ij}\frac{e^{\kappa_{ij}\cos(s_i-\mu_{ij})}}{2\pi I_0(\kappa_{ij})}\frac{e^{\kappa_i^{AP}\cos(s_i-\mu_i^{AP})}}{2\pi I_0(\kappa_i^{AP})}. \tag*{(57)}

By performing the same trigonometric operations on the exponents, we get

κijcos⁡(si−μij)+κiAPcos⁡(si−μiAP)=A~ijcos⁡(si)+B~ijsin⁡(si),(58)\kappa_{ij}\cos(s_i-\mu_{ij})+\kappa_i^{AP}\cos\left(s_i-\mu_i^{AP}\right)=\tilde{A}_{ij}\cos(s_i)+\tilde{B}_{ij}\sin(s_i), \tag*{(58)}

where

A~ij=κijcos⁡(μij)+κiAPcos⁡(μiAP),\tilde{A}_{ij}=\kappa_{ij}\cos(\mu_{ij})+\kappa_i^{AP}\cos(\mu_i^{AP}),
B~ij=κijsin⁡(μij)+κiAPsin⁡(μiAP).(59)\tilde{B}_{ij}=\kappa_{ij}\sin(\mu_{ij})+\kappa_i^{AP}\sin(\mu_i^{AP}). \tag*{(59)}

Defining

κ~ij=A~ij2+B~ij2,μ~ij=atan2⁡(B~ij,A~ij),(60)\tilde{\kappa}_{ij}=\sqrt{\tilde{A}_{ij}^{2}+\tilde{B}_{ij}^{2}},\qquad\tilde{\mu}_{ij}=\operatorname{atan2}(\tilde{B}_{ij},\tilde{A}_{ij}), \tag*{(60)}

we get

A~ijcos⁡(si)+B~ijsin⁡(si)=κ~ijcos⁡(si−μ~ij).(61)\tilde{A}_{ij}\cos(s_i)+\tilde{B}_{ij}\sin(s_i)=\tilde{\kappa}_{ij}\cos(s_i-\tilde{\mu}_{ij}). \tag*{(61)}

Substituting into Eq. (A.14), we get

qθ(si∣s<i)∝∑j=1KωijI0(κ~ij)2πI0(κij)I0(κiAP)eκ~ijcos⁡(si−μ~ij)2πI0(κ~ij)(62)q_\theta(s_i\mid s_{<i})\propto\sum_{j=1}^{K}\omega_{ij}\frac{I_0(\tilde{\kappa}_{ij})}{2\pi I_0(\kappa_{ij})I_0(\kappa_i^{AP})}\frac{e^{\tilde{\kappa}_{ij}\cos(s_i-\tilde{\mu}_{ij})}}{2\pi I_0(\tilde{\kappa}_{ij})} \tag*{(62)}

Since 2πI0(κiAP)2\pi I_0(\kappa_i^{\mathrm{AP}}) is shared across all of the mixture components, it can be absorbed into the normalizing constant

qθ(si∣s<i)∝∑j=1KωijI0(κ~ij)I0(κij)VM⁡(si;μ~ij,κ~ij)(63)q_\theta(s_i \mid s_{<i}) \propto\sum_{j=1}^{K} \omega_{ij} \frac{I_0(\tilde{\kappa}_{ij})}{I_0(\kappa_{ij})} \operatorname{VM}(s_i; \tilde{\mu}_{ij}, \tilde{\kappa}_{ij}) \tag*{(63)}

Mixture weights are then normalized, and we get

ω~ij=ωijI0(κ~ij)I0(κij)∑l=1KωilI0(κ~il)I0(κil).(64)\tilde{\omega}_{ij} = \frac{\omega_{ij} \dfrac{I_0(\tilde{\kappa}_{ij})}{I_0(\kappa_{ij})}}{\sum_{l=1}^{K} \omega_{il} \dfrac{I_0(\tilde{\kappa}_{il})}{I_0(\kappa_{il})}}. \tag*{(64)}

Finally, the conditional density is expressed as (33):

qθ(si∣s<i)=∑j=1Kω~ijVM⁡(si;μ~ij,κ~ij).(65)q_\theta(s_i \mid s_{<i}) = \sum_{j=1}^{K} \tilde{\omega}_{ij} \operatorname{VM}(s_i; \tilde{\mu}_{ij}, \tilde{\kappa}_{ij}). \tag*{(65)}

Proof that ESSqˉ_{\bar{q}} ≥\ge ESSq_q

We consider a group of transformations GG that do not change the energy: E(gs)=E(s)E(gs) = E(s), g∈Gg \in G. It follows that the target probability pp is also invariant under the action of GG. The distribution qθq_\theta, on the other hand, does not generally respect those symmetries. In section 3.5 we have introduced a symmetrized distribution

qˉθ(s)=1∣G∣∑g∈Gqθ(gs).(66)\bar{q}_\theta(s) = \frac{1}{|G|}\sum_{g \in G} q_\theta(gs). \tag*{(66)}

By integrating both sides, we get

∫qˉθ(s) ds=1∣G∣∑g∈G∫qθ(gs) ds(67)\int\bar{q}_\theta(s) \, \mathrm{d}s = \frac{1}{|G|}\sum_{g \in G}\int q_\theta(gs) \, \mathrm{d}s \tag*{(67)}

Changing variables gx=ygx = y, we get

∫qˉθ(s) ds=1∣G∣∑g∈G∫qθ(y) dy=1∣G∣∑g∈G1=1(68)\int\bar{q}_\theta(s) \, \mathrm{d}s = \frac{1}{|G|}\sum_{g \in G}\int q_\theta(y) \, \mathrm{d}y = \frac{1}{|G|}\sum_{g \in G}1 = 1 \tag*{(68)}

meaning that qˉθ\bar{q}_\theta is a normalized probability density. In the derivation, we have used the fact that the integration measure is invariant under the group transformations.

It is also easy to prove that qˉθ(s)\bar{q}_\theta(s) is invariant under the action of the group GG. By definition

qˉθ(hs)=1∣G∣∑g∈Gqθ(hgs),h∈G.(69)\bar{q}_\theta(hs) = \frac{1}{|G|}\sum_{g \in G}q_\theta(hgs), \quad h \in G. \tag*{(69)}

Since g↦hgg \mapsto hg is just a permutation of GG, qˉθ(hs)=qˉθ(s)\bar{q}_\theta(hs) = \bar{q}_\theta(s), h∈Gh \in G. This means that the "symmetrized" weights

wˉ(s)=Zp(s)qˉθ(s)=e−βE(s)qˉθ(s)(70)\bar{w}(s) = Z\frac{p(s)}{\bar{q}_\theta(s)} = \frac{e^{-\beta E(s)}}{\bar{q}_\theta(s)} \tag*{(70)}

are invariant under transformations in GG.

By definition

ESS⁡qˉθ=⟨w⟩qˉθ2⟨w2⟩qˉθ(71)\operatorname{ESS}_{\bar{q}_\theta} = \frac{\langle w\rangle_{\bar{q}_\theta}^{2}}{\langle w^{2}\rangle_{\bar{q}_\theta}} \tag*{(71)}

where unsymmetrized weights were defined in Eq. (12). We can write

⟨w2⟩qθ=(Z∫qθ(s)p(s)qθ(s) ds)2=(Z∫p(s) ds)2=Z2⟨w2⟩qθ=Z2∫qθ(s)p2(s)qθ2(s) ds=Z2∫p2(s)qθ(s) ds=Z2Iq(72)\left\langle w^2\right\rangle_{q_\theta}=\left(Z\int q_\theta(\mathbf{s})\frac{p(\mathbf{s})}{q_\theta(\mathbf{s})}\,\mathrm{d}s\right)^2=\left(Z\int p(\mathbf{s})\,\mathrm{d}s\right)^2=Z^2 \\ \left\langle w^2\right\rangle_{q_\theta}=Z^2\int q_\theta(\mathbf{s})\frac{p^2(\mathbf{s})}{q_\theta^2(\mathbf{s})}\,\mathrm{d}s=Z^2\int\frac{p^2(\mathbf{s})}{q_\theta(\mathbf{s})}\,\mathrm{d}s=Z^2I_q \tag*{(72)}

Substituting into (B.6), we get

ESS⁡q=1IqandESS⁡qˉ=1Iqˉ(73)\operatorname{ESS}_q=\frac{1}{I_q}\qquad\text{and}\qquad\operatorname{ESS}_{\bar q}=\frac{1}{I_{\bar q}} \tag*{(73)}

where IqˉI_{\bar q} has the same form as IqI_q with qˉθ\bar q_\theta substituted in place of qθq_\theta.

For y>0y>0, the function φ(y)=1y\varphi(y)=\frac{1}{y} is a convex function, so Jensen’s inequality

φ(∑iλiyi)≤∑iλiφ(yi),λi≥0,∑iλi=1(74)\varphi\left(\sum_i\lambda_i y_i\right)\leq\sum_i\lambda_i\varphi(y_i),\qquad\lambda_i\geq0,\qquad\sum_i\lambda_i=1 \tag*{(74)}

translates to

1∑iλiyi≤∑iλi1yi.(75)\frac{1}{\sum_i\lambda_i y_i}\leq\sum_i\lambda_i\frac{1}{y_i}. \tag*{(75)}

Substituting yi=q(si)y_i=q(s_i), we obtain

1∑iλiq(si)≤∑iλi1q(si).(76)\frac{1}{\sum_i\lambda_iq(s_i)}\leq\sum_i\lambda_i\frac{1}{q(s_i)}. \tag*{(76)}

Now set

sg=gs,g∈G,λg=1∣G∣(77)s_g=gs,\qquad g\in G,\qquad\lambda_g=\frac{1}{|G|} \tag*{(77)}

to obtain

∫dsp2(s)∣G∣−1∑gq(gs)≤∣G∣−1∑g∫dsp2(s)q(gs)(78)\int\mathrm{d}s\frac{p^2(s)}{|G|^{-1}\sum_gq(gs)}\leq|G|^{-1}\sum_g\int\mathrm{d}s\frac{p^2(s)}{q(gs)} \tag*{(78)}

as the multiplication by p2(s)>0p^2(s)>0 and integration do not change the inequality. The left-hand side of this inequality is

∫dsp2(s)qˉ(s)=Iqˉ(79)\int\mathrm{d}s\frac{p^2(s)}{\bar q(s)}=I_{\bar q} \tag*{(79)}

by definition. The right-hand side

∣G∣−1∑g∫dsp2(s)q(gs)=∣G∣−1∑g∫dsp2(gs)q(gs)=∣G∣−1∑g∫dsp2(s)q(s)=∣G∣−1∑gIq=Iq,(80)|G|^{-1}\sum_g\int\mathrm{d}s\frac{p^2(s)}{q(gs)}=|G|^{-1}\sum_g\int\mathrm{d}s\frac{p^2(gs)}{q(gs)}=|G|^{-1}\sum_g\int\mathrm{d}s\frac{p^2(s)}{q(s)}=|G|^{-1}\sum_g I_q=I_q, \tag*{(80)}

where we have used the fact that p(gs)=p(s)p(gs)=p(s) and the invariance of the measure under the group transformations. This proves that Iq≥IqˉI_q\geq I_{\bar q}, hence

ESS⁡q≤ESS⁡qˉ.(81)\operatorname{ESS}_q\leq\operatorname{ESS}_{\bar q}. \tag*{(81)}

Hyperparameters

The main text works with results obtained on a simple model composed of a single transformer layer and K=4K=4 components for the von Mises mixture. To examine the dependence on model expressivity, we additionally performed tests with different number of mixture components, namely K=8K=8, 1212, and two transformer layers, while the remaining setup was left unchanged.

We analyze the model’s ESS through training steps as well as training time (see Fig C.6). The runs were performed at L=96L=96 with inverse temperature β=1.12\beta=1.12 on a single A100 (40GB) GPU. We do not observe statistically significant differences between training for different KK. The 2-layer transformer reaches a higher ESS when trained for 20k steps, but at the expense of a longer training time. We therefore conclude that K=4K=4 and one transformer layer is a reasonable choice.

Figure C.6. ESS comparison of single-layer models with K=4K = 4, 88, 1212 mixture components and a two-layer model with K=4K = 4 mixture size across steps (left panel) and training time (right panel).

References

  1. [1]R. H. Swendsen, J.-S. Wang, Nonuniversal critical dynamics in Monte Carlo simulations, Phys. Rev. Lett. 58 (1987) 86–88. URL: https://link.aps.org/doi/10.1103/PhysRevLett.58.86. doi:10.1103/PhysRevLett.58.86.DOI
  2. [2]U. Wolff, Collective Monte Carlo updating for spin systems, Phys. Rev. Lett. 62 (1989) 361–364. URL: https://link.aps.org/doi/10.1103/PhysRevLett.62.361. doi:10.1103/PhysRevLett.62.361.DOI
  3. [3]F. Wang, D. P. Landau, Efficient, multiple-range random walk algorithm to calculate the density of states, Phys. Rev. Lett. 86 (2001) 2050–2053. URL: https://link.aps.org/doi/10.1103/PhysRevLett.86.2050. doi:10.1103/PhysRevLett.86.2050.DOI
  4. [4]K. A. Nicoli, S. Nakajima, N. Strodthoff, W. Samek, K.-R. Müller, P. Kessel, Asymptotically unbiased estimation of physical observables with neural samplers, Phys. Rev. E 101 (2020) 023304. arXiv:1910.13496.DOI
  5. [5]D. Wu, L. Wang, P. Zhang, Solving Statistical Mechanics Using Variational Autoregressive Networks, Phys. Rev. Lett. 122 (2019) 080602. doi:10.1103/PhysRevLett.122.080602. arXiv:1809.10606.DOI
  6. [6]M. S. Albergo, G. Kanwar, P. E. Shanahan, Flow-based generative models for Markov chain Monte Carlo in lattice field theory, Phys. Rev. D 100 (2019) 034515. URL: https://link.aps.org/doi/10.1103/PhysRevD.100.034515. doi:10.1103/PhysRevD.100.034515.DOI
  7. [7]P. Białas, P. Korcyl, T. Stebel, Hierarchical autoregressive neural networks for statistical systems, Comput. Phys. Commun. 281 (2022) 108502. doi:10.1016/j.cpc.2022.108502. arXiv:2203.10989.DOI
  8. [8]P. Białas, V. Chahar, P. Korcyl, T. Stebel, M. Winiarski, D. Zapolski, Hierarchical autoregressive neural networks in three-dimensional statistical system, Comput. Phys. Commun. 318 (2026) 109892. doi:10.1016/j.cpc.2025.109892. arXiv:2503.08610.DOI
  9. [9]A. Singha, E. Cellini, K. A. Nicoli, K. Jansen, S. Kühn, S. Nakajima, Multilevel Generative Samplers for Investigating Critical Phenomena, in: International Conference on Learning Representations, 2025. arXiv:2503.08918.arxiv.org/abs/2503.08918
  10. [10]I. Biazzo, T. The autoregressive neural network architecture of the Boltzmann distribution of pairwise interacting spins systems, Communications Physics 6 (2023). URL: http://dx.doi.org/10.1038/s42005-023-01416-5. doi:10.1038/s42005-023-01416-5.
  11. [11]P. Białas, P. Korcyl, T. Stebel, D. Zapolski, Variational autoregressive networks with probability priors, 2026. URL: https://arxiv.org/abs/2605.16020. arXiv:2605.16020.
  12. [12]S. Bunaiyan, C. Delacour, S. Chowdhury, K. Lee, K. Y. Camsari, IsingFormer: Augmenting Parallel Tempering With Learned Proposals, arXiv e-prints (2025) arXiv:2509.23043. doi:10.48550/arXiv.2509.23043. arXiv:2509.23043.
  13. [13]P. Białas, P. Korcyl, T. Stebel, A. Stefański, D. Zapolski, Sampling two-dimensional spin systems with transformers (2026). arXiv:2604.27738.
  14. [14]L. Zhong, W. Duan, J. Liu, P. Zhang, Y. Tang, Scalable physics-inspired transformers for spin glasses, 2026. URL: https://arxiv.org/abs/2606.22984. arXiv:2606.22984.
  15. [15]A. Vaswani, N. Shazeer, N. Parmar, J. Uszkoreit, L. Jones, A. N. Gomez, Ł. Kaiser, I. Polosukhin, Attention is all you need, in: I. Guyon, U. V. Luxburg, S. Bengio, H. Wallach, R. Fergus, S. Vish-wanathan, R. Garnett (Eds.), Advances in Neural Information Processing Systems, volume 30, Curran Associates, Inc., 2017. URL: https://proceedings.neurips.cc/paper_files/paper/2017/file/3f5ee243547dee91fbd053c1c4ca845aa-Paper.pdf.arxiv.org/abs/1706.03762
  16. [16]K. A. Nicoli, C. J. Anders, L. Funcke, T. Hartung, K. Jansen, P. Kessel, S. Nakajima, P. Stornati, Estimation of thermodynamic observables in lattice field theories with deep generative models, Phys. Rev. Lett. 126 (2021) 032001. URL: https://link.aps.org/doi/10.1103/PhysRevLett.126.032001. doi:10.1103/PhysRevLett.126.032001.DOI
  17. [17]M. Gerdes, P. de Haan, C. Rainone, R. Bondesan, M. C. N. Cheng, Learning lattice quantum field theories with equivariant continuous flows, SciPost Phys. 15 (2023) 238. doi:10.21468/SciPostPhys.15.6.238. arXiv:2207.00283.DOI
  18. [18]P. Białas, P. Korcyl, T. Stebel, Training normalizing flows with computationally intensive target probability distributions (2023). arXiv:2308.13294.DOI
  19. [19]K. Cranmer, G. Kanwar, S. Racanière, D. J. Rezende, P. E. Shanahan, Advances in machine-learning-based sampling motivated by lattice quantum chromodynamics, Nature Rev. Phys. 5 (2023) 526–535. doi:10.1038/s42254-023-00616-w. arXiv:2309.01156.DOI
  20. [20]A. Singha, J. Kauffmann, E. Cellini, K. Jansen, S. Nakajima, Scalable Generative Sampling and Multilevel Estimation for Lattice Field Theories Near Criticality (2026). arXiv:2604.10209.DOI
  21. [21]V. L. Berezinskii, Destruction of long-range order in one-dimensional and two-dimensional systems having a continuous symmetry group. I. Classical systems, Sov. Phys. JETP 32 (1971) 493–500. Russian original: Zh. Eksp. Teor. Fiz. 59, 907 (1970).
  22. [22]J. M. Kosterlitz, D. J. Thouless, Ordering, metastability and phase transitions in two-dimensional systems, J. Phys. C: Solid State Phys. 6 (1973) 1181–1203. doi:10.1088/0022-3719/6/7/010.DOI
  23. [23]L. Wang, Y. Jiang, L. He, K. Zhou, Continuous-mixture autoregressive networks learning the kosterlitz-thouless transition, Chinese Physics Letters 39 (2022) 120502.DOI
  24. [24]P. Białas, P. Korcyl, T. Stebel, D. Zapolski, NeuMC — a package for neural sampling for lattice field theories, SoftwareX 31 (2025) 102253. URL: http://dx.doi.org/10.1016/j.softx.2025.102253. doi:10.1016/j.softx.2025.102253.
  25. [25]V. Kanaujia, M. S. Scheurer, V. Arora, AdvNF: Reducing mode collapse in conditional normalising flows using adversarial learning, SciPost Physics 16 (2024). URL: http://dx.doi.org/10.21468/SciPostPhys.16.5.132. doi:10.21468/scipostphys.16.5.132.
  26. [26]P. Clifford, Markov random fields in statistics, in: Disorder in Physical Systems. A Volume in Honour of John M. Hammersley, Clarendon Press, 1990.
  27. [27]K. A. Nicoli, C. J. Anders, T. Hartung, K. Jansen, P. Kessel, S. Nakajima, Detecting and mitigating mode-collapse for flow-based sampling of lattice field theories, Phys. Rev. D 108 (2023) 114501. doi:10.1103/PhysRevD.108.114501.DOI
  28. [28]M. Hasenbusch, The two-dimensional XY model at the transition temperature: a high-precision Monte Carlo study, Journal of Physics A: Mathematical and General 38 (2005) 5869–5883. URL: http://dx.doi.org/10.1088/0305-4470/38/26/003. doi:10.1088/0305-4470/38/26/003.
  29. [29]L. Devroye, Non-Uniform Random Variate Generation, Springer New York, 1986. URL: http://dx.doi.org/10.1007/978-1-4613-8643-8. doi:10.1007/978-1-4613-8643-8.
  30. [30]I. Kobzyev, S. J. D. Prince, M. A. Brubaker, Normalizing Flows: An Introduction and Review of Current Methods, arXiv e-prints (2019) arXiv:1908.09257. doi:10.48550/arXiv.1908.09257. arXiv:1908.09257.
  31. [31]D. J. Rezende, G. Papamakarios, S. Racanière, M. S. Albergo, G. Kanwar, P. E. Shanahan, K. Cranmer, Normalizing flows on tori and spheres, 2020. URL: https://arxiv.org/abs/2002.02428. arXiv:2002.02428.
  32. [32]L. Vaitl, K. A. Nicoli, S. Nakajima, P. Kessel, Gradients should stay on path: better estimators of the reverse-and forward KL divergence for normalizing flows, Machine Learning: Science and Technology 3 (2022) 045006.
  33. [33]P. Minnhagen, B. J. Kim, Direct evidence of the discontinuous character of the Kosterlitz-Thouless jump, Phys. Rev. B 67 (2003) 172509. doi:10.1103/PhysRevB.67.172509. arXiv:cond-mat/0304226.DOI
  34. [34]P. Białas, P. Korcyl, T. Stebel, Analysis of autocorrelation times in neural Markov chain Monte Carlo simulations, Phys. Rev. E 107 (2023) 015303. doi:10.1103/PhysRevE.107.015303. arXiv:2111.10189.
  35. [35]P. Białas, P. Czarnota, P. Korcyl, T. Stebel, Simulating first-order phase transition with hierarchical autoregressive networks, Phys. Rev. E 107 (2023) 054127. doi:10.1103/PhysRevE.107.054127. arXiv:2212.04955.
  36. [36]P. Białas, P. Korcyl, T. Stebel, Mutual information of spin systems from autoregressive neural networks, Phys. Rev. E 108 (2023) 044140. doi:10.1103/PhysRevE.108.044140. arXiv:2304.13412.

Paper details

vlnc:W_8vb6yccs
Version
v1 · latest
Published
Fresh — added to the Hub within the last 30 days
Added
License
CC BY 4.0
Usage

views · 3

downloads · 0

A view counts once per browser per day, a download each time the PDF is requested.

  • Physics and Astronomy

Not recorded for this paper

View statement
Looking for related papers…
Contents