Introduction
Molecular dynamics (MD) is a central tool for studying the structure and dynamics of molecular systems in physics, chemistry, and biology. A recurring goal is to compute the free-energy landscape over a set of informative collective variables (CVs), defined as the negative logarithm of their marginal density under the equilibrium distribution, multiplied by the thermal energy k B T k_{\mathrm{B}}T k B T . This landscape quantifies the relative stability of metastable states and the barriers separating them.
Given samples from this marginal, learning it would be a standard estimation problem. In MD, however, the microscopic potential is known, but the relevant configurations must be generated by simulation. Brute-force MD is often computationally prohibitive: barrier-crossing transitions between metastable states can occur on microsecond-to-millisecond timescales, whereas integration requires femtosecond timesteps [23 ] . Free-energy learning is therefore coupled to a sampling problem, and the model must accelerate the simulation while it is itself being learned.
We introduce VESPER (Variational Extended Space Parallel Ensemble Regression), which learns a neural free energy G θ G_\theta G θ while using it to bias the simulations that produce its own training data. VESPER minimizes a single variational objective: a positively weighted sum of components, one per tempering level, each using a distinct temperature-boosting factor to flatten the effective free-energy landscape. Each component is convex in the free energy and is minimized by the physical free energy at every such level. The sum therefore has the same minimizer up to an additive constant, and every level contributes its gradient to one model without reweighting to the physical ensemble. VESPER couples biased microscopic dynamics, persistent auxiliary CV walkers, and free-energy learning through one shared model. Through the CV map s ( x ) s(x) s ( x ) , the same G θ G_\theta G θ sets both the MD bias and the tempered auxiliary potential, with no separate free-energy recursion. We also establish conditional convergence of the coupled sampling and learning dynamics. These contributions build on established variational connections: individual components extend the pseudo-spherical family to negative exponents and admit Rényi variational forms [8 , 20 , 60 ] , and their gradients recover the instantaneous update of well-tempered variationally enhanced sampling (VES), up to scale, when its target is held fixed during differentiation [9 , 58 , 59 ] .
The gradient of each component is the difference between two averages of ∇ θ G θ \nabla_\theta G_\theta ∇ θ G θ : one over MD configurations evolving under the microscopic potential plus a bias built from G θ G_\theta G θ , the other over persistent auxiliary walkers evolving directly in CV space under a tempered G θ G_\theta G θ . At the optimum both populations sample the same flattened distribution, and the walkers interact with the molecular system only through the shared parameters, so no adiabatic limit is needed to identify the free energy, unlike in temperature-accelerated MD and driven adiabatic free-energy dynamics [1 , 40 ] . Replica exchange can additionally carry configurations discovered under strongly flattened biases down the ladder, facilitating barrier crossing along the CVs.
Our main contributions are:
A single objective with a common minimizer. We establish that the losses associated with all tempering levels are convex in the free-energy function and share the physical free energy as their minimizer, up to an additive constant. Their positively weighted sum therefore defines one objective for one model, with every level contributing its native gradient without reweighting to the physical ensemble (Proposition 1 ).
Online learning from coupled sampling populations. Biased MD and persistent auxiliary CV walkers generate the configurations needed for learning as the model evolves, without a pre-existing dataset or an adiabatic limit between microscopic and auxiliary coordinates. We prove conditional convergence of the coupled overdamped population dynamics without replica exchange under log-Sobolev and Polyak–Łojasiewicz assumptions, controlling both objective error and sampling lag (Section A.6).
Replica exchange on the learned free energy. The Metropolis acceptance probability for exchange between adjacent levels depends only on learned free-energy differences in lower-dimensional CV space, avoiding additional evaluations of the microscopic potential. At a fixed model, the exchange kernel preserves the product of the sampling laws across levels and therefore leaves the population gradient unchanged.
Controlled tests of tempering and replica exchange. On the Müller–Brown potential, the tempering ladder balances basin accuracy, measured by KL divergence, with landscape reconstruction, measured by RMSE. On four-well landscapes spanning two to one hundred dimensions, a fixed ladder remains competitive without dimension-specific selection of α \alpha α , with replica exchange improving accuracy and convergence in the most challenging high-dimensional case (§ 3.1 ).
Molecular free-energy reconstruction and high-dimensional exploration. On alanine dipeptide, VESPER achieves competitive reconstruction accuracy and convergence relative to the tested enhanced-sampling baselines. On alanine tetrapeptide, agreement between Boltzmann marginals learned with three and six CVs demonstrates dimensional self-consistency. For methane dehydrogenation on Ni(511) with a machine-learned interatomic potential, VESPER learns a three-CV landscape whose two-CV Boltzmann marginals and reactant-to-saddle free-energy difference along a three-CV path are consistent with the lower-dimensional reference. On chignolin, biasing seventeen dihedral CVs produces repeated RMSD-defined unfolding and refolding transitions, demonstrating effective exploration of a high-dimensional molecular CV space (§ 3 ).
Methodology
Collective variables and free energy
Let x ∈ X x \in\mathcal{X} x ∈ X denote a molecular configuration, U ( x ) U(x) U ( x ) its microscopic potential energy, and ϱ ( x ) = Z − 1 e − β U ( x ) \varrho(x) = Z^{-1}e^{-\beta U(x)} ϱ ( x ) = Z − 1 e − β U ( x ) its equilibrium density at inverse temperature β = ( k B T ) − 1 \beta= (k_{\mathrm{B}}T)^{-1} β = ( k B T ) − 1 . A collective-variable map s : X → S s : \mathcal{X} \to\mathcal{S} s : X → S reduces the d d d -dimensional configuration space to a D D D -dimensional CV domain, typically with D ≪ d D \ll d D ≪ d . The equilibrium CV density and its associated free energy are
p ( s ) = ∫ X δ ( s − s ( x ) ) ϱ ( x ) d x , G ⋆ ( s ) = − β − 1 log p ( s ) . (1) p(s) = \int_{\mathcal{X}} \delta(s-s(x))\varrho(x)\,\mathrm{d}x,\qquad G_{\star}(s) = -\beta^{-1}\log p(s). \tag*{(1)} p ( s ) = ∫ X δ ( s − s ( x )) ϱ ( x ) d x , G ⋆ ( s ) = − β − 1 log p ( s ) . (1)
The microscopic potential supplies the forces for simulation, whereas the marginal p ( s ) p(s) p ( s ) must be inferred from configurations generated by the simulation. VESPER represents G ⋆ G_{\star} G ⋆ by a differentiable model G θ G_{\theta} G θ and uses it to bias the molecular dynamics that generate its training configurations.
The variational objective
Fix an auxiliary inverse temperature 0 < β ^ < β 0 < \hat{\beta} < \beta 0 < β ^ < β and set α = β ^ / β ∈ ( 0 , 1 ) \alpha= \hat{\beta}/\beta\in(0,1) α = β ^ / β ∈ ( 0 , 1 ) . We define the functional
L β ^ [ G ] = 1 β ^ log ∫ S e − β ^ G ( s ) d s + 1 β − β ^ log ∫ X e ( β − β ^ ) G ( s ( x ) ) ϱ ( x ) d x . (2) \mathcal{L}_{\hat{\beta}}[G] = \frac{1}{\hat{\beta}}\log\int_{\mathcal{S}}e^{-\hat{\beta}G(s)}\,\mathrm{d}s+\frac{1}{\beta-\hat{\beta}}\log\int_{\mathcal{X}}e^{(\beta-\hat{\beta})G(s(x))}\varrho(x)\,\mathrm{d}x. \tag*{(2)} L β ^ [ G ] = β ^ 1 log ∫ S e − β ^ G ( s ) d s + β − β ^ 1 log ∫ X e ( β − β ^ ) G ( s ( x )) ϱ ( x ) d x . (2)
The two terms are the log-partition functions of a tempered version of the model on S \mathcal{S} S and of the equilibrium density tilted by the model. They define the auxiliary CV law
P G , β ^ ( s ) = e − β ^ G ( s ) A G , A G = ∫ S e − β ^ G ( s ) d s (3) P_{G,\hat{\beta}}(s)=\frac{e^{-\hat{\beta}G(s)}}{A_G},\qquad A_G=\int_{\mathcal{S}}e^{-\hat{\beta}G(s)}\,\mathrm{d}s \tag*{(3)} P G , β ^ ( s ) = A G e − β ^ G ( s ) , A G = ∫ S e − β ^ G ( s ) d s (3)
and the biased microscopic law together with its CV marginal,
ν G , β ^ ( x ) = e ( β − β ^ ) G ( s ( x ) ) ϱ ( x ) B G , Q G , β ^ ( s ) = p ( s ) e ( β − β ^ ) G ( s ) B G , B G = ∫ X e ( β − β ^ ) G ( s ( x ) ) ϱ ( x ) d x . (4) \nu_{G,\hat{\beta}}(x)=\frac{e^{(\beta-\hat{\beta})G(s(x))}\varrho(x)}{B_G},\qquad Q_{G,\hat{\beta}}(s)=\frac{p(s)e^{(\beta-\hat{\beta})G(s)}}{B_G},\qquad B_G=\int_{\mathcal{X}}e^{(\beta-\hat{\beta})G(s(x))}\varrho(x)\,\mathrm{d}x. \tag*{(4)} ν G , β ^ ( x ) = B G e ( β − β ^ ) G ( s ( x )) ϱ ( x ) , Q G , β ^ ( s ) = B G p ( s ) e ( β − β ^ ) G ( s ) , B G = ∫ X e ( β − β ^ ) G ( s ( x )) ϱ ( x ) d x . (4)
The first variation of equation (2) in a direction h h h is E Q G , β ^ h − E P G , β ^ h \mathbb{E}_{Q_{G,\hat{\beta}}}h-\mathbb{E}_{P_{G,\hat{\beta}}}h E Q G , β ^ h − E P G , β ^ h , so G G G is stationary exactly when the two CV laws coincide:
e − β ^ G A G = p e ( β − β ^ ) G B G ⟺ e − β G ∝ p ⟺ G = G ⋆ + C . (5) \frac{e^{-\hat{\beta}G}}{A_G}=\frac{p e^{(\beta-\hat{\beta})G}}{B_G}\Longleftrightarrow e^{-\beta G}\propto p\Longleftrightarrow G=G_\star+C. \tag*{(5)} A G e − β ^ G = B G p e ( β − β ^ ) G ⟺ e − β G ∝ p ⟺ G = G ⋆ + C . (5)
The tempering exponent cancels: for every α \alpha α , the stationary point is the physical free energy. The second variation, β ^ Var P G , β ^ ( h ) + ( β − β ^ ) Var Q G , β ^ ( h ) \hat{\beta}\operatorname{Var}_{P_{G,\hat{\beta}}}(h)+(\beta-\hat{\beta})\operatorname{Var}_{Q_{G,\hat{\beta}}}(h) β ^ Var P G , β ^ ( h ) + ( β − β ^ ) Var Q G , β ^ ( h ) , is nonnegative, so L β ^ \mathcal{L}_{\hat{\beta}} L β ^ is convex and its minimizer set is { G ⋆ + C : C ∈ R } \{G_\star+C:C\in\mathbb{R}\} { G ⋆ + C : C ∈ R } (Proposition S1). At the minimizer,
P G ⋆ , β ^ = Q G ⋆ , β ^ = π α , π α ∝ e − α β G ⋆ ∝ p α , (6) P_{G_\star,\hat{\beta}}=Q_{G_\star,\hat{\beta}}=\pi_\alpha,\qquad\pi_\alpha\propto e^{-\alpha\beta G_\star}\propto p^\alpha, \tag*{(6)} P G ⋆ , β ^ = Q G ⋆ , β ^ = π α , π α ∝ e − α β G ⋆ ∝ p α , (6)
so both populations explore the same flattened landscape, with barriers reduced by the factor α \alpha α . Near the minimizer, L β ^ [ G ] − L β ^ [ G ⋆ ] ≈ β 2 Var π α ( G − G ⋆ ) \mathcal{L}_{\hat{\beta}}[G]-\mathcal{L}_{\hat{\beta}}[G_\star]\approx\frac{\beta}{2}\operatorname{Var}_{\pi_\alpha}(G-G_\star) L β ^ [ G ] − L β ^ [ G ⋆ ] ≈ 2 β Var π α ( G − G ⋆ ) : α \alpha α does not change the target, only where errors are weighted, with smaller α \alpha α giving more weight to high free-energy regions.
For a parametric model G θ G_\theta G θ , the chain rule gives
∇ θ L β ^ [ G θ ] = E ν G θ , β ^ [ ∇ θ G θ ( s ( x ) ) ] − E P G θ , β ^ [ ∇ θ G θ ( s ) ] . (7) \nabla_\theta\mathcal{L}_{\hat{\beta}}[G_\theta]=\mathbb{E}_{\nu_{G_\theta,\hat{\beta}}}\left[\nabla_\theta G_\theta(s(x))\right]-\mathbb{E}_{P_{G_\theta,\hat{\beta}}}\left[\nabla_\theta G_\theta(s)\right]. \tag*{(7)} ∇ θ L β ^ [ G θ ] = E ν G θ , β ^ [ ∇ θ G θ ( s ( x )) ] − E P G θ , β ^ [ ∇ θ G θ ( s ) ] . (7)
Neither expectation requires knowing p p p . The first is over configurations sampled by MD biased with G θ G_\theta G θ , and the second over walkers sampling the tempered model in CV space (§ 2.4 ).
The combined objective
Choose 0 ≤ β ^ 1 < ⋯ < β ^ K ≤ β 0\leq\hat{\beta}_1<\cdots<\hat{\beta}_K\leq\beta 0 ≤ β ^ 1 < ⋯ < β ^ K ≤ β , set α k = β ^ k / β \alpha_k=\hat{\beta}_k/\beta α k = β ^ k / β , and write P G , k P_{G,k} P G , k , ν G , k \nu_{G,k} ν G , k , Q G , k Q_{G,k} Q G , k for the laws of level k k k . The component L β ^ k \mathcal{L}_{\hat{\beta}_k} L β ^ k is given by equation (2) for 0 < β ^ k < β 0<\hat{\beta}_k<\beta 0 < β ^ k < β . At β ^ k = β \hat{\beta}_k=\beta β ^ k = β and, on compact S \mathcal{S} S , at β ^ k = 0 \hat{\beta}_k=0 β ^ k = 0 , define the endpoint objectives directly under the endpoint conditions in Section A.2:
L β [ G ] = 1 β log ∫ S e − β G ( s ) d s + E p [ G ] , L 0 [ G ] = − E u [ G ] + 1 β log E p [ e β G ] . (8) \mathcal{L}_{\beta}[G] = \frac{1}{\beta} \log\int_{\mathcal{S}} e^{-\beta G(s)} \, \mathrm{d}s + \mathbb{E}_{p}[G], \qquad\mathcal{L}_{0}[G] = -\mathbb{E}_{u}[G] + \frac{1}{\beta} \log\mathbb{E}_{p}[e^{\beta G}]. \tag*{(8)} L β [ G ] = β 1 log ∫ S e − β G ( s ) d s + E p [ G ] , L 0 [ G ] = − E u [ G ] + β 1 log E p [ e β G ] . (8)
Here u u u is the uniform law on S \mathcal{S} S , and L 0 \mathcal{L}_{0} L 0 is taken after subtracting the constant β ^ − 1 log ∣ S ∣ \hat{\beta}^{-1}\log|\mathcal{S}| β ^ − 1 log ∣ S ∣ from equation (2) . The first is the maximum-likelihood objective, β − 1 K L ( p ∥ q G ) \beta^{-1}\mathrm{KL}(p \mathbin\Vert q_G) β − 1 KL ( p ∥ q G ) up to a constant with q G ∝ e − β G q_G \propto e^{-\beta G} q G ∝ e − β G : its physical walkers are unbiased. The second is the fully flattened level, whose auxiliary target is P G , 0 = u P_{G,0}=u P G , 0 = u . These definitions agree with the corresponding interior limits, renormalized at zero, under the additional moment conditions in the proof of Proposition S2.
Proposition 1 (VESPER objective and minimizer). Let c k > 0 c_k > 0 c k > 0 with ∑ k = 1 K c k = 1 \sum_{k=1}^{K} c_k = 1 ∑ k = 1 K c k = 1 , and define
L V E S P E R [ G ] = ∑ k = 1 K c k L β ^ k [ G ] , (9) \mathcal{L}_{\mathrm{VESPER}}[G] = \sum_{k=1}^{K} c_k \mathcal{L}_{\hat{\beta}_k}[G], \tag*{(9)} L VESPER [ G ] = k = 1 ∑ K c k L β ^ k [ G ] , (9) with components given by equation (2) , or by equation (8) if β ^ 1 = 0 \hat{\beta}_1 = 0 β ^ 1 = 0 or β ^ K = β \hat{\beta}_K = \beta β ^ K = β . Under the integrability assumptions of Section A.4, L V E S P E R \mathcal{L}_{\mathrm{VESPER}} L VESPER is convex in G G G , invariant under addition of a constant, and has minimizer set { G ⋆ + C : C ∈ R } \{G_{\star} + C : C \in\mathbb{R}\} { G ⋆ + C : C ∈ R } . In a realizable model family, every minimizer therefore represents G ⋆ G_{\star} G ⋆ up to a constant. For a differentiable model G θ G_{\theta} G θ ,
∇ θ L V E S P E R [ G θ ] = ∑ k = 1 K c k { E ν G θ , k [ ∇ θ G θ ( s ( x ) ) ] − E P G θ , k [ ∇ θ G θ ( s ) ] } . (10) \nabla_{\theta}\mathcal{L}_{\mathrm{VESPER}}[G_{\theta}] = \sum_{k=1}^{K} c_k \left\{ \mathbb{E}_{\nu_{G_{\theta},k}}[\nabla_{\theta}G_{\theta}(s(x))] - \mathbb{E}_{P_{G_{\theta},k}}[\nabla_{\theta}G_{\theta}(s)] \right\}. \tag*{(10)} ∇ θ L VESPER [ G θ ] = k = 1 ∑ K c k { E ν G θ , k [ ∇ θ G θ ( s ( x ))] − E P G θ , k [ ∇ θ G θ ( s )] } . (10)
Every level, including strongly flattened ones, thus trains the same model directly, without reweighting between levels. The weights c k c_k c k and exponents α k \alpha_k α k leave the target unchanged but set how errors are weighted: near the minimizer, L V E S P E R [ G ] − L V E S P E R [ G ⋆ ] ≈ β 2 ∑ k c k Var π α k ( G − G ⋆ ) \mathcal{L}_{\mathrm{VESPER}}[G]-\mathcal{L}_{\mathrm{VESPER}}[G_{\star}] \approx\frac{\beta}{2}\sum_k c_k \operatorname{Var}_{\pi_{\alpha_k}}(G-G_{\star}) L VESPER [ G ] − L VESPER [ G ⋆ ] ≈ 2 β ∑ k c k Var π α k ( G − G ⋆ ) .
Coupled sampling and optimization
We estimate the expectations in equation (10) with two populations at each level, all sharing G θ G_{\theta} G θ (Figure 1 ). At level k k k , n k n_k n k physical walkers evolve under the biased potential U ( x ) − ( 1 − α k ) G θ ( s ( x ) ) U(x)-(1-\alpha_k)G_{\theta}(s(x)) U ( x ) − ( 1 − α k ) G θ ( s ( x )) , and N k N_k N k auxiliary walkers evolve directly in CV space under the tempered potential α k G θ ( s ) \alpha_kG_{\theta}(s) α k G θ ( s ) :
Figure 1. VESPER couples sampling and learning. Each tempering level k k k carries two walker populations, and all levels share one model G θ G_\theta G θ . Left: physical walkers evolve in configuration space under U ( x ) − ( 1 − α k ) G θ ( s ( x ) ) U(x)-(1-\alpha_k)G_\theta(s(x)) U ( x ) − ( 1 − α k ) G θ ( s ( x )) and are mapped to CV space by s ( x ) s(x) s ( x ) . Replica exchange swaps configurations between adjacent levels. Right: auxiliary walkers evolve directly in CV space under α k G θ ( s ) \alpha_kG_\theta(s) α k G θ ( s ) . The two populations supply the averages in equation (10) , whose c k c_k c k -weighted difference updates θ \theta θ . The updated model then sets the forces on both populations. At convergence, both populations at level k k k sample the same flattened landscape α k G ⋆ \alpha_kG_\star α k G ⋆ . During learning, the instantaneous target for the physical CVs is Q G θ , k ∝ p e ( β − β ^ k ) G θ Q_{G_\theta,k} \propto p e^{(\beta-\hat{\beta}_k)G_\theta} Q G θ , k ∝ p e ( β − β ^ k ) G θ .
m s s ¨ t i , k + γ s s ˙ t i , k = − α k ∇ s G θ t ( s t i , k ) + 2 γ s β − 1 η t i , k , m x x ¨ t j , k + γ x x ˙ t j , k = − ∇ x U ( x t j , k ) + ( 1 − α k ) ∇ x G θ t ( s ( x t j , k ) ) + 2 γ x β − 1 η ~ t j , k . (11) \begin{aligned}
m_s\ddot{s}^{i,k}_t+\gamma_s\dot{s}^{i,k}_t&=-\alpha_k\nabla_sG_{\theta_t}(s^{i,k}_t)+\sqrt{2\gamma_s\beta^{-1}}\,\eta^{i,k}_t,\\
m_x\ddot{x}^{j,k}_t+\gamma_x\dot{x}^{j,k}_t&=-\nabla_xU(x^{j,k}_t)+(1-\alpha_k)\nabla_xG_{\theta_t}(s(x^{j,k}_t))+\sqrt{2\gamma_x\beta^{-1}}\,\tilde{\eta}^{j,k}_t. \tag*{(11)}
\end{aligned} m s s ¨ t i , k + γ s s ˙ t i , k m x x ¨ t j , k + γ x x ˙ t j , k = − α k ∇ s G θ t ( s t i , k ) + 2 γ s β − 1 η t i , k , = − ∇ x U ( x t j , k ) + ( 1 − α k ) ∇ x G θ t ( s ( x t j , k )) + 2 γ x β − 1 η ~ t j , k . (11)
where m s , m x m_s,m_x m s , m x are masses, γ s , γ x \gamma_s,\gamma_x γ s , γ x friction coefficients, and η t i , k , η ~ t j , k \eta^{i,k}_t,\tilde{\eta}^{j,k}_t η t i , k , η ~ t j , k independent white noises. The bias force on the molecular system is ∇ x G θ ( s ( x ) ) = J s ( x ) ⊤ ∇ s G θ ( s ( x ) ) \nabla_xG_{\theta}(s(x))=J_s(x)^{\top}\nabla_sG_{\theta}(s(x)) ∇ x G θ ( s ( x )) = J s ( x ) ⊤ ∇ s G θ ( s ( x )) , with J s = ∂ s / ∂ x ∈ R D × d J_s=\partial s/\partial x\in\mathbb{R}^{D\times d} J s = ∂ s / ∂ x ∈ R D × d . For fixed θ \theta θ , these dynamics have configurational invariant laws P G θ , k P_{G_{\theta},k} P G θ , k and ν G θ , k \nu_{G_{\theta},k} ν G θ , k (Section A.3), so their walker averages estimate the two expectations in equation (10) . The parameters follow the corresponding gradient flow with γ θ − 1 \gamma_{\theta}^{-1} γ θ − 1 the learning rate,
γ θ θ ˙ t = ∑ k = 1 K c k [ 1 N k ∑ i = 1 N k ∇ θ G θ t ( s t i , k ) − 1 n k ∑ j = 1 n k ∇ θ G θ t ( s ( x t j , k ) ) ] . (12) \gamma_{\theta}\dot{\theta}_t=\sum_{k=1}^{K}c_k\left[\frac{1}{N_k}\sum_{i=1}^{N_k}\nabla_{\theta}G_{\theta_t}(s^{i,k}_t)-\frac{1}{n_k}\sum_{j=1}^{n_k}\nabla_{\theta}G_{\theta_t}(s(x^{j,k}_t))\right]. \tag*{(12)} γ θ θ ˙ t = k = 1 ∑ K c k [ N k 1 i = 1 ∑ N k ∇ θ G θ t ( s t i , k ) − n k 1 j = 1 ∑ n k ∇ θ G θ t ( s ( x t j , k )) ] . (12)
Three features of this construction matter in practice.
Timescales. Since θ \theta θ moves, the walkers track moving targets, and the averages in equation (12) approximate the gradient when θ \theta θ evolves more slowly than the walkers relax. Section A.6 makes
this quantitative for the overdamped dynamics: objective error and sampling lag decay together when the learning rate is below a threshold set by the log-Sobolev constants of the sampling laws and a bound on the parameter gradients, so no adiabatic limit is needed.
Section A.6 derives a conditional convergence bound for the coupled overdamped population dynamics without replica exchange. For 0 < α k < 1 0 < \alpha_k < 1 0 < α k < 1 , define V t = F ( θ t ) − F ∗ + H t V_t = F(\theta_t) - F_* + H_t V t = F ( θ t ) − F ∗ + H t , where F = β V E S P E R F = \beta\mathrm{VESPER} F = β VESPER and H t H_t H t is the weighted KL sampling lag defined in equation (35) . Under assumptions (C1)–(C4), for 0 < η < a / B 2 0 < \eta< a/B^2 0 < η < a / B 2 ,
V t ≤ e − κ t V 0 , κ = 2 min { η μ , a − η B 2 } . (13) V_t \le e^{-\kappa t}V_0,\qquad\kappa= 2\min\{\eta\mu, a-\eta B^2\}. \tag*{(13)} V t ≤ e − κ t V 0 , κ = 2 min { η μ , a − η B 2 } . (13)
Thus objective error and sampling lag decay together. Under (C1)–(C3), the same learning-rate restriction gives the time-averaged stationarity bound
1 T ∫ 0 T ∥ ∇ F ( θ t ) ∥ 2 d t ≤ V 0 η T . (14) \frac{1}{T}\int_0^T \lVert\nabla F(\theta_t)\rVert^2\,\mathrm{d}t \le\frac{V_0}{\eta T}. \tag*{(14)} T 1 ∫ 0 T ∥ ∇ F ( θ t ) ∥ 2 d t ≤ η T V 0 . (14)
In the implementation, equation (12) is discretized: sampling epochs alternate with optimizer steps, and batches may be reused across several steps with within-batch importance weights (Algorithm 1, Section B.7). When equilibrium configurations from ϱ \varrho ϱ are available, the physical average can instead be computed from them with self-normalized weights (Section A.9).
Replica exchange across tempering levels
The common-minimizer property does not require the levels to communicate. Sampling, however, benefits from it: strongly flattened levels cross barriers readily, while weakly flattened ones, including the physical level, may remain trapped. We therefore couple adjacent levels by replica exchange [2 , 18 , 55 ] , performed between parameter updates with G θ G_\theta G θ held fixed. For a symmetric swap proposal between levels i i i and j j j , the Metropolis acceptance probability for physical walkers is
A x ( x i , x j ) = min { 1 , exp [ ( β ^ i − β ^ j ) ( G θ ( s ( x i ) ) − G θ ( s ( x j ) ) ) ] } . (15) A_x(x^i,x^j)=\min\left\{1,\exp\left[(\hat{\beta}_i-\hat{\beta}_j)\left(G_\theta(s(x^i))-G_\theta(s(x^j))\right)\right]\right\}. \tag*{(15)} A x ( x i , x j ) = min { 1 , exp [ ( β ^ i − β ^ j ) ( G θ ( s ( x i )) − G θ ( s ( x j )) ) ] } . (15)
and A s ( s i , s j ) A_s(s^i,s^j) A s ( s i , s j ) is identical with s ( x i ) , s ( x j ) s(x^i),s(x^j) s ( x i ) , s ( x j ) replaced by s i , s j s^i,s^j s i , s j . Each swap preserves the fixed-model target product law ∏ k ν G θ , k \prod_k \nu_{G_\theta,k} ∏ k ν G θ , k or ∏ k P G θ , k \prod_k P_{G_\theta,k} ∏ k P G θ , k , and hence the population-gradient identity (Section A.5). All physical replicas share the same potential U U U and thermostat temperature, so the microscopic energy and Maxwell factors cancel from the exchange ratio. Both acceptance ratios therefore depend only on the learned free-energy differences in CV space. Given the CV values, their evaluation requires only two model values and no additional microscopic-energy evaluation. The tempering-level spacing is controlled by fluctuations of G θ G_\theta G θ , which can depend on the system and choice of CVs. Physical and auxiliary swaps are proposed independently.
Related work
Adaptive-bias enhanced sampling. Umbrella sampling, adaptive biasing force, metadynamics and its well-tempered variant, and on-the-fly probability enhanced sampling accelerate MD with restraining or adaptively built CV-dependent biases during the simulation [6 , 13 , 14 , 29 , 36 , 57 ] . Neural representations of the free energy or bias have been trained from mean forces, histograms, or both [22 , 50 , 51 , 53 , 63 ] . Reinforced dynamics couples such training to an uncertainty-gated bias [61 ] . VESPER couples biased MD and persistent auxiliary CV walkers through one physical free-energy model, with all tempering levels contributing to an objective with a common minimizer (Proposition 1 ). Its closest variational connection is to variationally enhanced sampling (VES) [58 ] , in its well-tempered [59 ] and neural [9 ] forms. With bias V θ = − ( 1 − α ) G θ V_\theta=-(1-\alpha)G_\theta V θ = − ( 1 − α ) G θ and target t θ ∝ e − α β G θ t_\theta\propto e^{-\alpha\beta G_\theta} t θ ∝ e − α β G θ held fixed during differentiation, the VES gradient equals ( 1 − α ) ∇ θ L β ^ [ G θ ] (1-\alpha)\nabla_\theta\mathcal{L}_{\hat{\beta}}[G_\theta] ( 1 − α ) ∇ θ L β ^ [ G θ ] (Section D.1). VESPER gives this instantaneous update an explicit objective in the physical free energy, with the current model directly determining both the molecular bias and the auxiliary target, without a separate recursive free-energy estimate. Neural VES also evaluates target averages by numerical integration or Monte Carlo. Here, persistent auxiliary dynamics form part of the coupled learning system (Section A.6).
Replica ladders with CV biases. Replica exchange improves exploration by swapping configurations between replicas with different temperatures or Hamiltonians [2 , 18 , 55 ] . Bias-exchange metadynamics assigns different CVs to different replicas [46 ] . Replica exchange with collective-variable tempering uses a ladder of well-tempered biases of increasing strength, which is the closest analogue of VESPER’s physical population [21 ] . VESPER uses the ladder for joint free-energy learning as well as configuration exchange: every level supplies its native gradient to one shared model, and the common-minimizer property allows these contributions to be combined without reweighting between levels (Proposition 1 ).
Extended-variable methods. Temperature-accelerated MD, driven adiabatic free-energy dynamics, and related schemes couple the CVs to hot auxiliary variables through a stiff spring. They recover the free energy in the limit of adiabatic separation between auxiliary and molecular motion [1 , 4 , 11 , 40 , 48 ] , possibly followed by mean-force reconstruction [41 ] . VESPER identifies the physical free energy through its variational minimizer, without requiring an adiabatic limit between auxiliary and molecular coordinates. Its auxiliary walkers explore the tempered model and communicate with the molecular system through shared parameters, without a spring coupling between the two populations.
Energy-based models and tempered objectives. VESPER connects tempered energy-based learning to molecular simulation through the prescribed CV map s ( x ) s(x) s ( x ) . Biased microscopic dynamics generate the tilted CV law, while persistent auxiliary walkers sample the tempered model directly in CV space [24 , 56 ] . Both populations train the same physical free-energy model without a pre-existing equilibrium dataset (Section A.3). With q = β G q=\beta G q = β G and γ = α − 1 \gamma=\alpha-1 γ = α − 1 , the single-level objective takes the form of the pseudo-spherical contrastive divergence (PS-CD) objective [60 ] , built on the γ \gamma γ -cross-entropy [20 ] , continued to negative γ \gamma γ , and admits a Rényi variational representation [8 ] . At α = 1 \alpha=1 α = 1 it reduces to maximum likelihood. For 0 < α < 1 0<\alpha<1 0 < α < 1 , VESPER uses the negative branch γ = α − 1 ∈ ( − 1 , 0 ) \gamma=\alpha-1\in(-1,0) γ = α − 1 ∈ ( − 1 , 0 ) . The factor q θ γ q_\theta^\gamma q θ γ relatively upweights observations in regions of low model density and high model free energy. At the physical solution, both populations have CV densities proportional to ρ α \rho^\alpha ρ α , scaling CV free-energy barriers by α \alpha α and increasing the relative weight of rare regions (Section D). PS-CD estimates its positive term from a fixed dataset and its tempered model. term by importance weighting model samples [60 ] . VESPER realizes both tilted laws through coupled dynamics, extending this variational principle to simultaneous molecular sampling and free-energy reconstruction (Section D). Parallel tempering has also been used to train restricted Boltzmann machines [12 , 15 ] , with only the untempered chain entering the gradient. In VESPER every level contributes.
Learned samplers. Boltzmann generators and non-equilibrium transport samplers learn maps or flows that sample a known energy [3 , 16 , 30 , 45 ] , and annealed importance sampling and Jarzynski-type estimators weight paths between distributions [10 , 32 , 44 ] . These methods target the full configurational distribution. VESPER instead targets a CV free energy through directly simulated tilted populations.
Numerical results
We evaluate VESPER on analytic landscapes and molecular systems to test free-energy reconstruction accuracy, convergence as the number of CVs increases, and exploration of conformational and chemical changes in complex MD systems. The analytic benchmarks provide exact references and controlled comparisons of tempering and replica exchange, while the molecular applications test the method in realistic MD. Complete simulation settings and numerical comparisons are given in the appendices. Molecular simulation times are per replica. Aggregate MD and population budgets are specified in Table 2 and Section C.2.
Table 2. Replicas K K K , simulated time per replica at the readout, and aggregate MD per run (K K K times the time per replica).
Analytic landscapes: tempering and increasing dimension
The Müller–Brown benchmark [42 ] tests how tempering balances accuracy in low-energy basins with reconstruction over a broader region of the landscape, including high-free-energy regions of low probability mass (Figure 2 A–C). Moderate tempering gives the lowest basin-sensitive KL divergence, while stronger tempering gives the lowest full-domain RMSE. The fixed exchange ladder balances these objectives, reaching a KL divergence of 0.017 ± 0.013 0.017 \pm0.013 0.017 ± 0.013 while keeping its mean RMSE within 1 % 1\% 1% of the lowest tested value. Here the RMSE includes extrapolation beyond the simulation box into a high-energy rim, so it measures broad-domain reconstruction rather than barrier accuracy alone (Section C.1).
Figure 2. Analytic landscapes. (A) Full-evaluation-domain RMSE in k B T k_\mathrm{B}T k B T and (B) dimensionless D K L ( ρ ∥ ρ θ ) D_\mathrm{KL}(\rho\parallel\rho_\theta) D KL ( ρ ∥ ρ θ ) against reduced time on Müller–Brown. Curves show means for single levels and a five-level ladder with (L-RE) and without (L-NRE) exchange, with 1,000 walkers per population. (C) Reference and learned surfaces at t = 80 t = 80 t = 80 on a common 0–20 k B T k_\mathrm{B}T k B T scale. Red and blue dots are physical and auxiliary walkers. (D–F) Four-well landscape in d = 2 d = 2 d = 2 –100 coordinates, with fixed hidden width and primary walker budget: (D) final SWA full-coordinate NRMSE, (E) D K L ( ρ ∥ ρ θ ) D_\mathrm{KL}(\rho\parallel\rho_\theta) D KL ( ρ ∥ ρ θ ) , and (F) reduced time to sustained EMA NRMSE below 0.25. Points and error bars show means and sample SDs. Open circles in (D,E) mark values above the plotted range or non-finite divergences. Open markers in (F) indicate at least one run without a sustained crossing.
We then increase the dimension of a rotated four-well landscape from two to one hundred, keeping the hidden width, walker budget and update schedule fixed (Figure 2 D–F). The same tempering ladder remains competitive across the tested dimensions without selecting a different tempering strength for each problem. At one hundred dimensions, the exchange ladder gives the lowest mean normalized RMSE among the primary comparisons: 0.32 ± 0.21 0.32 \pm0.21 0.32 ± 0.21 , compared with 0.42 ± 0.05 0.42 \pm0.05 0.42 ± 0.05 for the best tested single level. These results support using a shared ladder across dimensions, with its strongest advantages appearing in the more demanding exploration problem. The landscape, averaging conventions and larger-population controls are specified in Section C.2.
Alanine dipeptide: comparison with established methods
Alanine dipeptide (ADP) in explicit water tests whether VESPER can reconstruct a molecular free-energy surface with accuracy and convergence comparable to established enhanced-sampling methods. With the two backbone dihedrals (ϕ , ψ \phi,\psi ϕ , ψ ) as CVs, tempered VESPER explores the periodic landscape and recovers the reference surface [63 ] , while the unbiased control remains concentrated in its initial basin (Figure 3 ). At 10 ns per replica, the exchange ladder reaches an RMSE of 2.02 ± 0.24 k J m o l − 1 2.02 \pm0.24\ \mathrm{kJ\,mol^{-1}} 2.02 ± 0.24 kJ mo l − 1 , comparable to 2.04 ± \pm ± 0.51 kJ mol− 1 ^{-1} − 1 for SIREN-based combined force-frequency sampling (S-CFF), the strongest tested baseline at that horizon. A single tempered level also performs well. The ladders make only a small change to the endpoint error on this relatively simple low-dimensional landscape.
Figure 3. Alanine dipeptide. (A) RMSE against the reference surface [63 ] over 0–10 ns per replica, averaged over three independent runs. VESPER curves are solid and baselines dashed. Gray shading marks errors above the chemical accuracy of 1 kcal mol− 1 ^{-1} − 1 (4.184 kJ mol− 1 ^{-1} − 1 ). L-RE, L-NRE: four-level ladders with and without exchange; OPES: on-the-fly probability enhanced sampling [29 ] ; MetaD: metadynamics [36 ] ; VES: neural variationally enhanced sampling [9 , 58 ] ; TAMD: temperature-accelerated MD [5 , 40 ] ; CFF: combined force-frequency sampling [51 ] ; S-CFF: its SIREN variant [63 ] . (B) Reference (top left), and VESPER surfaces for α = 1 \alpha= 1 α = 1 (top right), a single level at α = 0.01 \alpha= 0.01 α = 0.01 (bottom left) and the exchange ladder (bottom right), on a common scale. Red and blue dots are physical and auxiliary walkers. (C) ADP with ϕ \phi ϕ , ψ \psi ψ labeled [28 ] . (D) ϕ \phi ϕ of the physical walkers over 1–10 ns, colored by the learned free energy.
The VESPER mean error curves first fall below 1 kcal mol− 1 ^{-1} − 1 and remain there through 10 ns at approximately 2.0 ns for a single level and 1.8–1.9 ns for the ladders. Metadynamics and S-CFF reach this threshold earlier, at 1.3–1.4 ns. VESPER therefore combines comparable late-time accuracy with rapid practical convergence, within about 2 ns, on this benchmark. Full baseline comparisons and scoring conventions are given in Sections B.1 and B.9 and Table 3 .
Table 3. Values reported in the main text for the molecular systems. ADP: 10 ns per replica, SWA8 readout, three independent runs. ATP: 10 ns, SWA10, three independent runs, mean surfaces for the plane comparisons. Methane: 7.5 ns, EMA readout, three independent runs. Its 1.12 ± 0.25 eV entry is the route value on the mean surface with the sample SD evaluated at fixed route points. Chignolin: event counts use the first 25 ns per replica and the unbiased-level occupancy uses the full 26.2 ns trajectory, four independent runs. RMSEs are mean ± sample SD over runs.
Alanine tetrapeptide: consistency as the number of CVs increases
Alanine tetrapeptide (ATP) tests whether VESPER retains a consistent molecular description when the number of biased backbone dihedrals increases from three to six. The two models differ only in their input layer, and their two-dimensional Boltzmann marginals retain similar broad structure (Figure 4 ). Both six-CV ladder variants also produce more consistent learned shapes between 5 and 10 ns than the selected single-level model: the correlations with their own 10 ns estimates are 0.913 ± 0.012 with exchange and 0.941 ± 0.028 without, compared with 0.804 ± 0.179 for the single level. This suggests that multiple learning levels can stabilize reconstruction over the observed interval in the higher-dimensional six-CV model. Without an independent converged six-CV reference, these comparisons establish dimensional and temporal consistency.
Figure 4. Alanine tetrapeptide: consistency across CV dimension. Two-dimensional Boltzmann marginals of (A) the three-CV and (B) the six-CV model, each from the mean surface of the four-level ladder with exchange. Columns: ( ϕ 1 , ϕ 2 ) (\phi_1,\phi_2) ( ϕ 1 , ϕ 2 ) , ( ϕ 1 , ϕ 3 ) (\phi_1,\phi_3) ( ϕ 1 , ϕ 3 ) , ( ϕ 2 , ϕ 3 ) (\phi_2,\phi_3) ( ϕ 2 , ϕ 3 ) . (C) ATP with ϕ \phi ϕ bonds in blue and ψ \psi ψ bonds in red.
Catalytic methane dehydrogenation on Ni(511)
We next test VESPER on a reactive free-energy landscape described by a machine-learned interatomic potential. For methane on Ni(511) at 700 K, a single tempered level learns a joint surface over the C–Ni and C–H coordination numbers and the carbon atom’s lateral position. Its two-CV Boltzmann marginals recover the physisorbed and dissociated basins and the transition region of the lower-dimensional reference [33 ] (Figure 5 ).
Figure 5. Methane on Ni(511): three-CV free energy. (A–C) Two-dimensional Boltzmann marginals in the planes (C–Ni, C–H coordination), (S 1 S_1 S 1 , C–H coordination) and (S 1 S_1 S 1 , C–Ni coordination). White line: bounded path on the mean surface obtained by a string-method calculation [17 ] followed by constrained gradient-flow refinement, from reactant (1) through the saddle (3) to product (5). (D) Trajectory frames corresponding to the labeled points.
The differentiable neural representation also supports post hoc analysis of the joint three-CV landscape using its gradients. A string-method path calculation [17 ] , followed by constrained gradient-flow refinement, connects reactant and product through a fitted saddle. The bounded reactant-to-saddle free-energy difference is 1.12 ± 0.25 eV, consistent with the reference. Including the lateral position directly in the model locates the fitted saddle near the step-edge atoms. Whereas Jin et al. [33 ] obtained an additional spatial free-energy description through post hoc Spectral-ABF analysis of existing trajectories, VESPER learns the joint three-CV surface during sampling. The route, averaging and projection conventions are specified in Section B.5.
Chignolin: exploration with seventeen CVs
Finally, we test whether one shared neural free energy can support exploration in the high-dimensional space of chignolin’s seventeen backbone dihedrals. For CLN025 in explicit water, a twelve-level ladder continuously trains the joint model during replica exchange, with every level contributing to learning (Figure 6 ). The representation is not restricted to a sum of pairwise terms, and the target is not prescribed as a folded/unfolded mixture, extending earlier work on this system [52 ] .
Figure 6. Chignolin with seventeen dihedral CVs (four independent runs, twelve levels from α = 1 \alpha= 1 α = 1 to 1 / 18 1/18 1/18 ). (A) Mean number of levels holding folded (C α C_{\alpha} C α RMSD below 0.20 nm), intermediate (labeled Partial) or unfolded (above 0.40 nm) configurations. Shading shows the minimum-to-maximum range across runs. (B) Mean folded fraction at each tempering level. In A and B, occupancies are averaged in 0.2 ns bins and across the four runs. Both panels display the first 25 ns per replica. (C) Free-energy projection at the end of the run from unbiased-level auxiliary walkers reconstructed on a fixed backbone, in C α C_{\alpha} C α radius of gyration and RMSD (dots: MD frames, red, and auxiliary walkers, blue, from all levels). (D) Dihedral marginal in (ψ 3 \psi_3 ψ 3 , ψ 4 \psi_4 ψ 4 ) from the same unbiased-level auxiliary walkers. Contours every 5 kJ mol− 1 ^{-1} − 1 .
Starting from folded configurations, VESPER explores RMSD-defined folded and unfolded regions. Exploration broadens along the ladder while the unbiased level remains predominantly folded. Auxiliary-sample projections place the free-energy minimum in the folded region and represent a broader range of conformations also visited by MD. VESPER thus learns a joint free energy while exploring conformational changes with seventeen CVs.
Discussion and conclusion
We introduced VESPER, which casts free-energy learning and enhanced sampling as a single optimization problem. The free energy is parameterized by a neural network G θ G_\theta G θ that biases the MD, drives tempered auxiliary walkers in CV space, and is trained on the difference between its mean parameter gradients over the auxiliary walkers and the biased MD configurations. This difference estimates the gradient of one variational objective, so the simulation generates its own training data as the model improves. Working in an extended space of molecular configurations and auxiliary walkers, VESPER minimizes an explicit objective whose minimizer is the physical free energy itself. The free energy is therefore learned directly, rather than recovered from a bias optimized toward a fixed or recursively updated target, and no adiabatic separation between auxiliary and molecular coordinates is required. Because the objective is convex and every tempering level shares
the same global minimizer, all levels train one model without reweighting between them. Strongly flattened levels that cross barriers readily therefore work alongside weakly tempered levels, and when the ladder includes the physical level, α = 1 \alpha= 1 α = 1 , the ladder objective bounds the KL divergence of the model from the true CV distribution. Replica exchange moves configurations along the ladder using only learned free-energy differences in CV space, with no additional evaluations of the microscopic potential. The coupled sampling and learning dynamics also carry a conditional convergence guarantee.
VESPER is effective across a wide range of problems. The analytic landscapes provide exact references for controlled tests. On the Müller–Brown potential, the exchange ladder balances accuracy in the low-energy basins with reconstruction of the broader landscape. On a four-well landscape whose dimension grows from two to one hundred, one fixed ladder stays competitive without retuning and shows its clearest advantage at one hundred dimensions. Alanine dipeptide is a standard molecular benchmark with a reference surface for direct comparison. Here VESPER matches or improves on the accuracy of established enhanced-sampling methods, reaching the endpoint error of the best tested baseline, and converges within 2 ns. Alanine tetrapeptide tests whether the learned description stays consistent as more CVs are biased, and the Boltzmann marginals learned with three and six CVs agree. Methane dehydrogenation on Ni(511) extends the tests from conformational to chemical change, using a machine-learned interatomic potential. VESPER learns a reactive three-CV landscape whose two-CV marginals recover the basins and transition region of the reference. Because the model is a differentiable neural network, a string-method calculation with gradient-flow refinement then maps the reaction path directly on the joint three-CV surface learned during sampling. Chignolin tests exploration in a realistic high-dimensional CV space. While biasing seventeen backbone dihedrals, one shared model drives repeated unfolding and refolding. These results connect online free-energy learning to enhanced sampling across conformational and chemical changes. Molecular comparisons retain their per-replica cost: a K K K -level ladder uses K K K times the aggregate MD of a single replica at the same horizon.
VESPER directly accelerates barrier crossing along the chosen CVs. Orthogonal slow motions can still limit exploration by the physical walkers. Our convergence analysis assumes overdamped dynamics and a Polyak–Łojasiewicz condition and excludes exchange, batch reuse and practical optimizers. The high-dimensional molecular results establish self-consistency and exploration without converged free-energy references. The stability controls for batch reuse (Section B.7) and per-system hyperparameters. Natural extensions are joint CV learning, adaptive level weights, and combining online sampling with existing trajectories (Section A.9).
Reproducibility statement
Proofs, protocols, scoring conventions and the algorithm are given in the appendices. VESPER is integrated into PySAGES [62 ] , available at https://github.com/SSAGESLabs/PySAGES . All code, run configurations and analysis scripts will be released publicly at https://github.com/JayLKaP/vesper . Generative AI tools assisted with code implementation, experiment execution, figure preparation, literature review, and manuscript editing under author supervision. The authors reviewed the AI-assisted work and take full responsibility for the final content.
Proofs
We retain the notation of the main text, with G G G in energy units and α = β ^ / β \alpha= \hat{\beta}/\beta α = β ^ / β . When considering a single tempering level, we write P G P_G P G , ν G \nu_G ν G , and Q G Q_G Q G for P G , β ^ P_{G,\hat{\beta}} P G , β ^ , ν G , β ^ \nu_{G,\hat{\beta}} ν G , β ^ , and Q G , β ^ Q_{G,\hat{\beta}} Q G , β ^ , respectively.
Convexity and the physical free-energy minimizer
Let p p p be the physical CV density in equation (1) . We work on its support S S S , where p > 0 p > 0 p > 0 almost everywhere, and for the chosen 0 < α < 1 0 < \alpha< 1 0 < α < 1 assume that
C α = ∫ S p ( s ) α d s < ∞ . (16) C_\alpha= \int_S p(s)^\alpha\,\mathrm{d}s < \infty. \tag*{(16)} C α = ∫ S p ( s ) α d s < ∞. (16)
We consider finite-valued measurable free energies G G G for which the two normalizers
A G = ∫ S e − β ^ G ( s ) d s , B G = ∫ S p ( s ) e ( β − β ^ ) G ( s ) d s (17) A_G = \int_S e^{-\hat{\beta}G(s)} \,\mathrm{d}s,\qquad B_G = \int_S p(s)e^{(\beta-\hat{\beta})G(s)} \,\mathrm{d}s \tag*{(17)} A G = ∫ S e − β ^ G ( s ) d s , B G = ∫ S p ( s ) e ( β − β ^ ) G ( s ) d s (17)
are finite and positive.
The auxiliary density, biased microscopic density and its CV marginal are
P G ( s ) = e − β ^ G ( s ) A G , ν G ( x ) = e ( β − β ^ ) G ( s ( x ) ) ϱ ( x ) B G , (18) P_G(s) = \frac{e^{-\hat{\beta}G(s)}}{A_G},\qquad\nu_G(x) = \frac{e^{(\beta-\hat{\beta})G(s(x))}\varrho(x)}{B_G}, \tag*{(18)} P G ( s ) = A G e − β ^ G ( s ) , ν G ( x ) = B G e ( β − β ^ ) G ( s ( x )) ϱ ( x ) , (18)
Q G ( s ) = p ( s ) e ( β − β ^ ) G ( s ) B G . (19) Q_G(s) = \frac{p(s)e^{(\beta-\hat{\beta})G(s)}}{B_G}. \tag*{(19)} Q G ( s ) = B G p ( s ) e ( β − β ^ ) G ( s ) . (19)
Proposition S1 (Convexity and the free-energy minimizer). The functional in equation (2) is convex in G G G , invariant under addition of a constant, and has minimizer set { G ⋆ + C } \{G_\star+ C\} { G ⋆ + C } . Its minimum is
L β ^ [ G ⋆ ] = log C α β α ( 1 − α ) . (20) \mathcal{L}_{\hat{\beta}}[G_\star] = \frac{\log C_\alpha}{\beta\alpha(1-\alpha)}. \tag*{(20)} L β ^ [ G ⋆ ] = β α ( 1 − α ) log C α . (20) At this minimum, P G ⋆ = Q G ⋆ = p α / C α P_{G_\star} = Q_{G_\star} = p^\alpha/C_\alpha P G ⋆ = Q G ⋆ = p α / C α .
Proof. Each term of equation (2) is convex by Hölder’s inequality. Adding a constant c c c to G G G changes the two terms by − c -c − c and c c c , so their sum is unchanged. To identify the minimizer, differentiate in a bounded direction h h h :
D L β ^ [ G ] ( h ) = E Q G h − E P G h , (21) D\mathcal{L}_{\hat{\beta}}[G](h) = \mathbb{E}_{Q_G}h - \mathbb{E}_{P_G}h, \tag*{(21)} D L β ^ [ G ] ( h ) = E Q G h − E P G h , (21)
D 2 L β ^ [ G ] ( h , h ) = β ^ Var P G ( h ) + ( β − β ^ ) Var Q G ( h ) ≥ 0. (22) D^2\mathcal{L}_{\hat{\beta}}[G](h,h) = \hat{\beta}\operatorname{Var}_{P_G}(h) + (\beta-\hat{\beta})\operatorname{Var}_{Q_G}(h) \ge0. \tag*{(22)} D 2 L β ^ [ G ] ( h , h ) = β ^ Var P G ( h ) + ( β − β ^ ) Var Q G ( h ) ≥ 0. (22)
The first variation vanishes for every such h h h exactly when P G = Q G P_G = Q_G P G = Q G . Using equation (18) and equation (19) , this condition becomes
e − β ^ G ( s ) A G = p ( s ) e ( β − β ^ ) G ( s ) B G ⟺ G ( s ) = − β − 1 log p ( s ) + C . \frac{e^{-\hat{\beta}G(s)}}{A_G} = \frac{p(s)e^{(\beta-\hat{\beta})G(s)}}{B_G}
\quad\Longleftrightarrow\quad
G(s) = -\beta^{-1}\log p(s) + C. A G e − β ^ G ( s ) = B G p ( s ) e ( β − β ^ ) G ( s ) ⟺ G ( s ) = − β − 1 log p ( s ) + C .
To see that this candidate attains the minimum, apply Hölder’s inequality once more:
C α = ∫ S ( e − β ^ G ) 1 − α ( p e ( β − β ^ ) G ) α d s ≤ A G 1 − α B G α . C_\alpha= \int_S \left(e^{-\hat{\beta}G}\right)^{1-\alpha}\left(pe^{(\beta-\hat{\beta})G}\right)^\alpha\,\mathrm{d}s \le A_G^{1-\alpha}B_G^\alpha. C α = ∫ S ( e − β ^ G ) 1 − α ( p e ( β − β ^ ) G ) α d s ≤ A G 1 − α B G α .
Taking logarithms gives the lower bound in equation (20) . Equality holds precisely when the two normalized factors coincide, hence precisely at G = G ⋆ + C G = G_\star+ C G = G ⋆ + C . This candidate is admissible by equation (16) , and substitution gives P G ⋆ = Q G ⋆ = p α / C α P_{G_\star} = Q_{G_\star} = p^\alpha/C_\alpha P G ⋆ = Q G ⋆ = p α / C α . Finally, applying the chain rule to equation (21) gives the parameter gradient in equation (7) , provided differentiation under the integrals is justified.
KL control and endpoint limits
Proposition S2 (Loss and KL control). Let π α = p α / C α \pi_\alpha= p^\alpha/C_\alpha π α = p α / C α . Under the assumptions of Proposition S1 and integrability of G G G and log p \log p log p under π α \pi_\alpha π α ,
L β ^ [ G ] − L β ^ [ G ⋆ ] = KL ( π α ∥ P G ) β ^ + KL ( π α ∥ Q G ) β − β ^ . (23) \mathcal{L}_{\widehat{\beta}}[G] - \mathcal{L}_{\widehat{\beta}}[G_\star] = \frac{\operatorname{KL}(\pi_\alpha\Vert P_G)}{\widehat{\beta}} + \frac{\operatorname{KL}(\pi_\alpha\Vert Q_G)}{\beta-\widehat{\beta}}. \tag*{(23)} L β [ G ] − L β [ G ⋆ ] = β KL ( π α ∥ P G ) + β − β KL ( π α ∥ Q G ) . (23) In particular,
KL ( π α ∥ P G ) ≤ β ^ ( L β ^ [ G ] − L β ^ [ G ⋆ ] ) , KL ( π α ∥ Q G ) ≤ ( β − β ^ ) ( L β ^ [ G ] − L β ^ [ G ⋆ ] ) . (24) \operatorname{KL}(\pi_\alpha\Vert P_G) \le\widehat{\beta}\left(\mathcal{L}_{\widehat{\beta}}[G] - \mathcal{L}_{\widehat{\beta}}[G_\star]\right), \qquad\operatorname{KL}(\pi_\alpha\Vert Q_G) \le(\beta-\widehat{\beta})\left(\mathcal{L}_{\widehat{\beta}}[G] - \mathcal{L}_{\widehat{\beta}}[G_\star]\right). \tag*{(24)} KL ( π α ∥ P G ) ≤ β ( L β [ G ] − L β [ G ⋆ ] ) , KL ( π α ∥ Q G ) ≤ ( β − β ) ( L β [ G ] − L β [ G ⋆ ] ) . (24) At β ^ = β \widehat{\beta}=\beta β = β , assume Z G = ∫ e − β G d s < ∞ Z_G = \int e^{-\beta G}\,\mathrm{d}s < \infty Z G = ∫ e − β G d s < ∞ , E p ∣ G ∣ < ∞ \mathbb{E}_p|G| < \infty E p ∣ G ∣ < ∞ , and E p ∣ log p ∣ < ∞ \mathbb{E}_p|\log p| < \infty E p ∣ log p ∣ < ∞ . With q G = e − β G / Z G q_G=e^{-\beta G}/Z_G q G = e − β G / Z G , the extension in equation (8) satisfies
L β [ G ] − L β [ G ⋆ ] = β − 1 KL ( p ∥ q G ) . (25) \mathcal{L}_{\beta}[G] - \mathcal{L}_{\beta}[G_\star] = \beta^{-1}\operatorname{KL}(p\Vert q_G). \tag*{(25)} L β [ G ] − L β [ G ⋆ ] = β − 1 KL ( p ∥ q G ) . (25) At β ^ = 0 \widehat{\beta}=0 β = 0 on compact S S S , assume E u ∣ G ∣ < ∞ \mathbb{E}_u|G| < \infty E u ∣ G ∣ < ∞ , E u ∣ log p ∣ < ∞ \mathbb{E}_u|\log p| < \infty E u ∣ log p ∣ < ∞ , and 0 < B G , 0 : = E p e β G < ∞ 0 < B_{G,0}:=\mathbb{E}_p e^{\beta G} < \infty 0 < B G , 0 := E p e β G < ∞ . With Q G , 0 = p e β G / B G , 0 Q_{G,0}=pe^{\beta G}/B_{G,0} Q G , 0 = p e β G / B G , 0 , the zero-level objective satisfies
L 0 [ G ] − L 0 [ G ⋆ ] = β − 1 KL ( u ∥ Q G , 0 ) . (26) \mathcal{L}_{0}[G] - \mathcal{L}_{0}[G_\star] = \beta^{-1}\operatorname{KL}(u\Vert Q_{G,0}). \tag*{(26)} L 0 [ G ] − L 0 [ G ⋆ ] = β − 1 KL ( u ∥ Q G , 0 ) . (26)
Proof. Substituting the expressions for P G P_G P G and Q G Q_G Q G into the two divergences gives
KL ( π α ∥ P G ) = E π α log π α + β ^ E π α G + log A G , \operatorname{KL}(\pi_\alpha\Vert P_G) = \mathbb{E}_{\pi_\alpha}\log\pi_\alpha+ \widehat{\beta}\mathbb{E}_{\pi_\alpha}G + \log A_G, KL ( π α ∥ P G ) = E π α log π α + β E π α G + log A G ,
KL ( π α ∥ Q G ) = E π α log π α − E π α log p − ( β − β ^ ) E π α G + log B G . \operatorname{KL}(\pi_\alpha\Vert Q_G) = \mathbb{E}_{\pi_\alpha}\log\pi_\alpha- \mathbb{E}_{\pi_\alpha}\log p - (\beta-\widehat{\beta})\mathbb{E}_{\pi_\alpha}G + \log B_G. KL ( π α ∥ Q G ) = E π α log π α − E π α log p − ( β − β ) E π α G + log B G .
Dividing by β ^ \widehat{\beta} β and β − β ^ \beta-\widehat{\beta} β − β , respectively, and adding cancels the expectations of G G G . Since log π α = α log p − log C α \log\pi_\alpha= \alpha\log p - \log C_\alpha log π α = α log p − log C α , the remaining terms are L β ^ [ G ] − log C α / [ β α ( 1 − α ) ] \mathcal{L}_{\widehat{\beta}}[G] - \log C_\alpha/[\beta\alpha(1-\alpha)] L β [ G ] − log C α / [ β α ( 1 − α )] . This proves equation (23) , and nonnegativity of either divergence gives equation (24) .
At the physical endpoint, the definition in equation (8) gives directly
KL ( p ∥ q G ) = E p log p + β E p G + log Z G = β L β [ G ] + E p log p . \operatorname{KL}(p\Vert q_G) = \mathbb{E}_p\log p + \beta\mathbb{E}_pG + \log Z_G = \beta\mathcal{L}_{\beta}[G] + \mathbb{E}_p\log p. KL ( p ∥ q G ) = E p log p + β E p G + log Z G = β L β [ G ] + E p log p .
Since q G ⋆ = p q_{G_\star}=p q G ⋆ = p , subtracting the same identity at G ⋆ G_\star G ⋆ proves equation (25) .
To recover this endpoint as a limit of the interior objectives, assume in addition that ∫ e − β ^ 0 G d s < ∞ \int e^{-\widehat{\beta}_0G}\,\mathrm{d}s < \infty ∫ e − β 0 G d s < ∞ for some 0 < β ^ 0 < β 0 < \widehat{\beta}_0 < \beta 0 < β 0 < β and E p e ε 0 G < ∞ \mathbb{E}_p e^{\varepsilon_0G} < \infty E p e ε 0 G < ∞ for some ε 0 > 0 \varepsilon_0 > 0 ε 0 > 0 . The bound e − β ^ G ≤ e − β ^ 0 G + e − β G e^{-\widehat{\beta}G} \le e^{-\widehat{\beta}_0G} + e^{-\beta G} e − β G ≤ e − β 0 G + e − β G then gives β ^ − 1 log A G → β − 1 log Z G \widehat{\beta}^{-1}\log A_G \to\beta^{-1}\log Z_G β − 1 log A G → β − 1 log Z G by dominated convergence. Writing ε = β − β ^ ↓ 0 \varepsilon=\beta-\widehat{\beta}\downarrow0 ε = β − β ↓ 0 , the difference quotient ( e ε G − 1 ) / ε (e^{\varepsilon G}-1)/\varepsilon ( e εG − 1 ) / ε decreases to G G G and lies between the integrable functions G G G and ( e ε 0 G − 1 ) / ε 0 (e^{\varepsilon_0G}-1)/\varepsilon_0 ( e ε 0 G − 1 ) / ε 0 . Thus E p e ε G = 1 + ε E p G + o ( ε ) \mathbb{E}_p e^{\varepsilon G}=1+\varepsilon\mathbb{E}_pG+o(\varepsilon) E p e εG = 1 + ε E p G + o ( ε ) , so the second term of equation (2) , ε − 1 log B G = ε − 1 log E p e ε G \varepsilon^{-1}\log B_G=\varepsilon^{-1}\log\mathbb{E}_p e^{\varepsilon G} ε − 1 log B G = ε − 1 log E p e εG , tends to E p G \mathbb{E}_pG E p G as required.
At the zero endpoint, direct expansion gives
KL ( u ∥ Q G , 0 ) = β L 0 [ G ] − E u log p − log ∣ S ∣ . \operatorname{KL}(u\Vert Q_{G,0}) = \beta\mathcal{L}_{0}[G] - \mathbb{E}_u\log p - \log|S|. KL ( u ∥ Q G , 0 ) = β L 0 [ G ] − E u log p − log ∣ S ∣.
Since β L 0 [ G ⋆ ] = E u log p + log ∣ S ∣ \beta\mathcal{L}_{0}[G_\star]=\mathbb{E}_u\log p+\log|S| β L 0 [ G ⋆ ] = E u log p + log ∣ S ∣ , subtraction proves equation (26) . Equality holds exactly when Q G , 0 = u Q_{G,0}=u Q G , 0 = u , or G = G ⋆ + C G=G_\star+C G = G ⋆ + C almost everywhere. For the renormalized limit β ^ ↓ 0 \widehat{\beta}\downarrow0 β ↓ 0 , additionally assume ∫ e − β ^ 0 G d s < ∞ \int e^{-\widehat{\beta}_0G}\,\mathrm{d}s < \infty ∫ e − β 0 G d s < ∞ for some 0 < β ^ 0 < β 0 < \widehat{\beta}_0 < \beta 0 < β 0 < β . The same difference-quotient argument under u u u gives β ^ − 1 log E u e − β ^ G → − E u G \widehat{\beta}^{-1}\log\mathbb{E}_u e^{-\widehat{\beta}G}\to-\mathbb{E}_uG β − 1 log E u e − β G → − E u G , while dominated convergence gives E p e ( β − β ^ ) G → B G , 0 \mathbb{E}_p e^{(\beta-\widehat{\beta})G}\to B_{G,0} E p e ( β − β ) G → B G , 0 . These are the two terms of the renormalized limit.
For a probability density ρ \rho ρ on S S S , define its entropy relative to the CV reference measure by Ent d s ( ρ ) = ∫ S ρ log ρ d s \operatorname{Ent}_{\mathrm{d}s}(\rho)=\int_S \rho\log\rho\,\mathrm{d}s Ent d s ( ρ ) = ∫ S ρ log ρ d s . Let ν \nu ν be a probability law on configuration space. Define
J ( G , ρ , ν ) = E ν [ G ∘ s ] − E ρ G − Ent d s ( ρ ) β ^ − KL ( ν ∥ ϱ ) β − β ^ . (27) \mathcal{J}(G,\rho,\nu)=\mathbb{E}_{\nu}[G\circ s]-\mathbb{E}_{\rho}G-\frac{\operatorname{Ent}_{\mathrm{d}s}(\rho)}{\hat{\beta}}-\frac{\operatorname{KL}(\nu\Vert\varrho)}{\beta-\hat{\beta}}. \tag*{(27)} J ( G , ρ , ν ) = E ν [ G ∘ s ] − E ρ G − β ^ Ent d s ( ρ ) − β − β ^ KL ( ν ∥ ϱ ) . (27)
Proposition S3 (Dual formulation and stationary sampling laws). For 0 < β ^ < β 0<\hat{\beta}<\beta 0 < β ^ < β and admissible G G G ,
L β ^ [ G ] = sup ρ , ν J ( G , ρ , ν ) , min G L β ^ [ G ] = min G sup ρ , ν J ( G , ρ , ν ) . (28) \mathcal{L}_{\hat{\beta}}[G]=\sup_{\rho,\nu}\mathcal{J}(G,\rho,\nu),\qquad\min_G\mathcal{L}_{\hat{\beta}}[G]=\min_G\sup_{\rho,\nu}\mathcal{J}(G,\rho,\nu). \tag*{(28)} L β ^ [ G ] = ρ , ν sup J ( G , ρ , ν ) , G min L β ^ [ G ] = G min ρ , ν sup J ( G , ρ , ν ) . (28) The maximizing laws are P G P_G P G and ν G \nu_G ν G , and the pushforward of ν G \nu_G ν G is Q G Q_G Q G . Suppose the potentials and CV map are smooth, the Langevin equations are well posed, and the boundaries are periodic, reflecting, or confining with sufficient decay. The fixed-model dynamics in equation (11) then preserve the configurational laws P G P_G P G and ν G \nu_G ν G .
Proof. The dual identity follows by expanding the relative entropies:
J ( G , ρ , ν ) = L β ^ [ G ] − KL ( ρ ∥ P G ) β ^ − KL ( ν ∥ ν G ) β − β ^ . (29) \mathcal{J}(G,\rho,\nu)=\mathcal{L}_{\hat{\beta}}[G]-\frac{\operatorname{KL}(\rho\Vert P_G)}{\hat{\beta}}-\frac{\operatorname{KL}(\nu\Vert\nu_G)}{\beta-\hat{\beta}}. \tag*{(29)} J ( G , ρ , ν ) = L β ^ [ G ] − β ^ KL ( ρ ∥ P G ) − β − β ^ KL ( ν ∥ ν G ) . (29)
The two KL terms are nonnegative and vanish at ρ = P G \rho=P_G ρ = P G and ν = ν G \nu=\nu_G ν = ν G , proving equation (28) . The expansion is immediate when all terms are finite, and the Gibbs variational formula gives the same supremum otherwise.
For fixed ρ \rho ρ and ν \nu ν , differentiating J \mathcal{J} J with respect to θ \theta θ , under the integrals, gives E ν ∇ θ G θ ∘ s − E ρ ∇ θ G θ \mathbb{E}_{\nu}\nabla_{\theta}G_{\theta}\circ s-\mathbb{E}_{\rho}\nabla_{\theta}G_{\theta} E ν ∇ θ G θ ∘ s − E ρ ∇ θ G θ . At the maximizing laws this is ∇ θ L β ^ [ G θ ] \nabla_{\theta}\mathcal{L}_{\hat{\beta}}[G_{\theta}] ∇ θ L β ^ [ G θ ] . Taking the negative derivative at each level, weighting by c k c_k c k , and replacing the expectations by walker averages gives the parameter update in equation (12) . Meanwhile, Wasserstein ascent in ρ \rho ρ and ν \nu ν , with mobilities α / γ s \alpha/\gamma_s α / γ s and ( 1 − α ) / γ x (1-\alpha)/\gamma_x ( 1 − α ) / γ x , gives
∂ t ρ = γ s − 1 ∇ s ⋅ ( α ρ ∇ s G + β − 1 ∇ s ρ ) , (30) \partial_t\rho=\gamma_s^{-1}\nabla_s\cdot\left(\alpha\rho\nabla_sG+\beta^{-1}\nabla_s\rho\right), \tag*{(30)} ∂ t ρ = γ s − 1 ∇ s ⋅ ( α ρ ∇ s G + β − 1 ∇ s ρ ) , (30)
∂ t ν = γ x − 1 ∇ x ⋅ ( ν ∇ x [ U − ( 1 − α ) G ∘ s ] + β − 1 ∇ x ν ) . (31) \partial_t\nu=\gamma_x^{-1}\nabla_x\cdot\left(\nu\nabla_x\left[U-(1-\alpha)G\circ s\right]+\beta^{-1}\nabla_x\nu\right). \tag*{(31)} ∂ t ν = γ x − 1 ∇ x ⋅ ( ν ∇ x [ U − ( 1 − α ) G ∘ s ] + β − 1 ∇ x ν ) . (31)
Their particle realizations are the overdamped sampling equations. Adding momenta and Langevin thermostats yields the inertial realization in equation (11) with the same configurational targets.
The CV marginal of ν G \nu_G ν G follows from the definition of p p p : for any bounded test function h h h ,
∫ X h ( s ( x ) ) ν G ( x ) d x = 1 B G ∫ S h ( s ) p ( s ) e ( β − β ^ ) G ( s ) d s . \int_{\mathcal{X}}h(s(x))\nu_G(x)\,\mathrm{d}x=\frac{1}{B_G}\int_S h(s)p(s)e^{(\beta-\hat{\beta})G(s)}\,\mathrm{d}s. ∫ X h ( s ( x )) ν G ( x ) d x = B G 1 ∫ S h ( s ) p ( s ) e ( β − β ^ ) G ( s ) d s .
Thus s # ν G = Q G s_{\#}\nu_G=Q_G s # ν G = Q G . To check invariance, note that the potentials α G \alpha G α G and U − ( 1 − α ) G ∘ s U-(1-\alpha)G\circ s U − ( 1 − α ) G ∘ s at inverse temperature β \beta β have Gibbs factors e − β G ^ e^{-\beta\hat{G}} e − β G ^ and e − β U + ( β − β ^ ) G ∘ s e^{-\beta U+(\beta-\hat{\beta})G\circ s} e − β U + ( β − β ^ ) G ∘ s . These make the probability currents in equation (30) and equation (31) vanish. For inertial dynamics, the corresponding phase-space laws include Maxwell momentum factors: Hamiltonian transport preserves energy and phase volume, while friction and noise preserve the Maxwell factors. Under ergodicity, time averages therefore recover the required expectations.
The sampling laws also hold at β ^ = β \hat{\beta}=\beta β ^ = β , although the dual functional in equation (27) is singular there. The auxiliary potential is then G G G , with invariant density P G , β ∝ e − β G P_{G,\beta}\propto e^{-\beta G} P G , β ∝ e − β G , and the unbiased physical dynamics preserve ϱ \varrho ϱ .
The common-minimizer ladder
Proposition S4 (Weighted excess loss and level gradients). Assume Proposition S1 at every interior level and the moment and normalizer conditions of Proposition S2 at an endpoint level. For c k > 0 c_k > 0 c k > 0 with ∑ k c k = 1 \sum_k c_k = 1 ∑ k c k = 1 , the ladder functional is convex and its minimizer set is { G ⋆ + C } \{G_\star+ C\} { G ⋆ + C } . Its parameter gradient is given by equation 10. These statements imply Proposition 1 .
Proof. For each level, Propositions S1 and S2 give
Δ k [ G ] : = L β ^ k [ G ] − L β ^ k [ G ⋆ ] ≥ 0 , Δ k [ G ] = 0 ⟺ G = G ⋆ + C . \Delta_k[G] := \mathcal{L}_{\hat{\beta}_k}[G] - \mathcal{L}_{\hat{\beta}_k}[G_\star] \ge0,\qquad\Delta_k[G] = 0 \Longleftrightarrow G = G_\star+ C. Δ k [ G ] := L β ^ k [ G ] − L β ^ k [ G ⋆ ] ≥ 0 , Δ k [ G ] = 0 ⟺ G = G ⋆ + C .
This includes both endpoints by equation 25 and equation 26. The excess of the combined objective is ∑ k c k Δ k [ G ] \sum_k c_k\Delta_k[G] ∑ k c k Δ k [ G ] . Since every c k c_k c k is positive, this sum vanishes exactly when every component vanishes, proving the common-minimizer claim. Each component is convex and invariant under constant shifts, so the weighted sum has both properties as well. At either endpoint, these properties follow directly from equation 8: the log-partition term is convex, the expectation is linear, and their shifts cancel. If the model family contains G ⋆ + C G_\star+ C G ⋆ + C , it attains the same minimum.
Finally, differentiating the finite sum gives
∇ θ L V E S P E R [ G θ ] = ∑ k c k ∇ θ L β ^ k [ G θ ] . \nabla_\theta\mathcal{L}_{\mathrm{VESPER}}[G_\theta] = \sum_k c_k \nabla_\theta\mathcal{L}_{\hat{\beta}_k}[G_\theta]. ∇ θ L VESPER [ G θ ] = k ∑ c k ∇ θ L β ^ k [ G θ ] .
Substitution of equation 7 proves equation 10. At the physical endpoint, differentiating equation 8 gives E p ∇ θ G θ − E P G θ , β ∇ θ G θ \mathbb{E}_p\nabla_\theta G_\theta- \mathbb{E}_{P_{G_\theta,\beta}}\nabla_\theta G_\theta E p ∇ θ G θ − E P G θ , β ∇ θ G θ , assuming differentiation under the integral is justified. This has the same form, since ν G , K = ϱ \nu_{G,K} = \varrho ν G , K = ϱ and s # ϱ = p s_\#\varrho= p s # ϱ = p . At the zero endpoint the gradient is E Q G θ , 0 ∇ θ G θ − E u ∇ θ G θ \mathbb{E}_{Q_{G_\theta,0}}\nabla_\theta G_\theta- \mathbb{E}_u\nabla_\theta G_\theta E Q G θ , 0 ∇ θ G θ − E u ∇ θ G θ , again assuming justified differentiation, with P G θ , 0 = u P_{G_\theta,0} = u P G θ , 0 = u and s # ν G θ , 0 = Q G θ , 0 s_\#\nu_{G_\theta,0} = Q_{G_\theta,0} s # ν G θ , 0 = Q G θ , 0 . Thus each level contributes expectations under its own sampling laws, with coefficient c k c_k c k .
Replica-exchange acceptance
Proposition S5 (Exchange preserves the level product laws). Fix G G G for which all level distributions are normalizable, and consider replicas of the same physical system at inverse temperature β \beta β , with auxiliary inverse temperatures β ^ k \hat{\beta}_k β ^ k . A symmetric proposal to swap two physical configurations, accepted with A x A_x A x in equation 15, is reversible with respect to ∏ k ν G , k \prod_k \nu_{G,k} ∏ k ν G , k . The analogous auxiliary swap with acceptance A s A_s A s is reversible with respect to ∏ k P G , k \prod_k P_{G,k} ∏ k P G , k .
Proof. For an auxiliary swap between levels i i i and j j j , the ratio of the swapped and original product densities is
P G , i ( s j ) P G , j ( s i ) P G , i ( s i ) P G , j ( s j ) = exp [ ( β ^ i − β ^ j ) ( G ( s i ) − G ( s j ) ) ] . \frac{P_{G,i}(s^j)P_{G,j}(s^i)}{P_{G,i}(s^i)P_{G,j}(s^j)}
= \exp\left[(\hat{\beta}_i-\hat{\beta}_j)\left(G(s^i)-G(s^j)\right)\right]. P G , i ( s i ) P G , j ( s j ) P G , i ( s j ) P G , j ( s i ) = exp [ ( β ^ i − β ^ j ) ( G ( s i ) − G ( s j ) ) ] .
For a physical swap, write G i = G ( s ( x i ) ) G_i = G(s(x^i)) G i = G ( s ( x i )) and G j = G ( s ( x j ) ) G_j = G(s(x^j)) G j = G ( s ( x j )) . The common physical factors ϱ ( x i ) ϱ ( x j ) \varrho(x^i)\varrho(x^j) ϱ ( x i ) ϱ ( x j ) and the normalizers cancel, leaving
log ν G , i ( x j ) ν G , j ( x i ) ν G , i ( x i ) ν G , j ( x j ) = ( β − β ^ i ) G j + ( β − β ^ j ) G i − ( β − β ^ i ) G i − ( β − β ^ j ) G j = ( β ^ i − β ^ j ) ( G i − G j ) . \log\frac{\nu_{G,i}(x^j)\nu_{G,j}(x^i)}{\nu_{G,i}(x^i)\nu_{G,j}(x^j)}
= (\beta-\hat{\beta}_i)G_j + (\beta-\hat{\beta}_j)G_i
- (\beta-\hat{\beta}_i)G_i - (\beta-\hat{\beta}_j)G_j
= (\hat{\beta}_i-\hat{\beta}_j)(G_i-G_j). log ν G , i ( x i ) ν G , j ( x j ) ν G , i ( x j ) ν G , j ( x i ) = ( β − β ^ i ) G j + ( β − β ^ j ) G i − ( β − β ^ i ) G i − ( β − β ^ j ) G j = ( β ^ i − β ^ j ) ( G i − G j ) .
Thus the two log ratios are
ℓ s = ( β ^ i − β ^ j ) [ G ( s i ) − G ( s j ) ] , ℓ x = ( β ^ i − β ^ j ) [ G i − G j ] . (32) \ell_s = (\hat{\beta}_i-\hat{\beta}_j)[G(s^i)-G(s^j)],\qquad\ell_x = (\hat{\beta}_i-\hat{\beta}_j)[G_i-G_j]. \tag*{(32)} ℓ s = ( β ^ i − β ^ j ) [ G ( s i ) − G ( s j )] , ℓ x = ( β ^ i − β ^ j ) [ G i − G j ] . (32)
In either case, let T T T denote the swap and Π \Pi Π the product density. Since T 2 z = z T^2z=z T 2 z = z , the Metropolis acceptance A ( z ) = min { 1 , Π ( T z ) / Π ( z ) } A(z)=\min\{1,\Pi(Tz)/\Pi(z)\} A ( z ) = min { 1 , Π ( T z ) /Π ( z )} satisfies Π ( z ) A ( z ) = min { Π ( z ) , Π ( T z ) } = Π ( T z ) A ( T z ) \Pi(z)A(z)=\min\{\Pi(z),\Pi(Tz)\}=\Pi(Tz)A(Tz) Π ( z ) A ( z ) = min { Π ( z ) , Π ( T z )} = Π ( T z ) A ( T z ) . Together with the symmetric proposal, this proves detailed balance. With equal masses and a common thermostat, the Maxwell factors also cancel if momenta are exchanged. Leaving momenta in place preserves the same independent Maxwell law. Composing pair-exchange kernels therefore preserves Π \Pi Π . A joint symmetric swap of both populations instead uses the sum ℓ x + ℓ s \ell_x+\ell_s ℓ x + ℓ s as its log acceptance ratio.
Convergence of the coupled population dynamics
We now consider the overdamped population dynamics with continuous parameter descent and no replica exchange. The aim is to control the objective error together with the lag of the sampling distributions behind their changing targets.
First suppose that all levels satisfy 0 < α k < 1 0<\alpha_k<1 0 < α k < 1 , with c k > 0 c_k>0 c k > 0 and ∑ k c k = 1 \sum_k c_k=1 ∑ k c k = 1 . We treat the physical endpoint below. It is convenient to use dimensionless quantities,
g θ = β G θ , F ( θ ) = β L V E S P E R [ G θ ] , f θ ( s ) = ∇ θ g θ ( s ) , g_\theta=\beta G_\theta,\qquad F(\theta)=\beta\mathcal{L}_{\mathrm{VESPER}}[G_\theta],\qquad f_\theta(s)=\nabla_\theta g_\theta(s), g θ = β G θ , F ( θ ) = β L VESPER [ G θ ] , f θ ( s ) = ∇ θ g θ ( s ) ,
so the instantaneous targets are
ρ k θ ( s ) = e − α k g θ ( s ) A k ( θ ) , ν k θ ( x ) = ϱ ( x ) e ( 1 − α k ) g θ ( s ( x ) ) B k ( θ ) . (33) \rho_k^\theta(s)=\frac{e^{-\alpha_k g_\theta(s)}}{A_k(\theta)},\qquad\nu_k^\theta(x)=\frac{\varrho(x)e^{(1-\alpha_k)g_\theta(s(x))}}{B_k(\theta)}. \tag*{(33)} ρ k θ ( s ) = A k ( θ ) e − α k g θ ( s ) , ν k θ ( x ) = B k ( θ ) ϱ ( x ) e ( 1 − α k ) g θ ( s ( x )) . (33)
Let ρ k , t \rho_{k,t} ρ k , t and ν k , t \nu_{k,t} ν k , t denote the evolving auxiliary and physical laws. Suppressing the time subscript inside expectations, their equations are
∂ t ρ k = d s , k ∇ s ⋅ ( ρ k ∇ s log ρ k ρ k θ t ) , d s , k = ( β γ s , k ) − 1 , ∂ t ν k = d x , k ∇ x ⋅ ( ν k ∇ x log ν k ν k θ t ) , d x , k = ( β γ x , k ) − 1 , θ ˙ t = − η ∑ k c k ( E ν k [ f θ t ∘ s ] − E ρ k f θ t ) . (34) \begin{aligned}
\partial_t\rho_k&=d_{s,k}\nabla_s\cdot\left(\rho_k\nabla_s\log\frac{\rho_k}{\rho_k^{\theta_t}}\right),\qquad d_{s,k}=(\beta\gamma_{s,k})^{-1},\\
\partial_t\nu_k&=d_{x,k}\nabla_x\cdot\left(\nu_k\nabla_x\log\frac{\nu_k}{\nu_k^{\theta_t}}\right),\qquad d_{x,k}=(\beta\gamma_{x,k})^{-1},\\
\dot{\theta}_t&=-\eta\sum_k c_k\left(\mathbb{E}_{\nu_k}[f_{\theta_t}\circ s]-\mathbb{E}_{\rho_k}f_{\theta_t}\right). \tag*{(34)}
\end{aligned} ∂ t ρ k ∂ t ν k θ ˙ t = d s , k ∇ s ⋅ ( ρ k ∇ s log ρ k θ t ρ k ) , d s , k = ( β γ s , k ) − 1 , = d x , k ∇ x ⋅ ( ν k ∇ x log ν k θ t ν k ) , d x , k = ( β γ x , k ) − 1 , = − η k ∑ c k ( E ν k [ f θ t ∘ s ] − E ρ k f θ t ) . (34)
Here η = ( β γ θ ) − 1 \eta=(\beta\gamma_\theta)^{-1} η = ( β γ θ ) − 1 matches the normalization in equation (12) . The physical drift is available directly from ∇ x log ν k θ = − β ∇ x U + ( 1 − α k ) ∇ x ( g θ ∘ s ) \nabla_x\log\nu_k^\theta=-\beta\nabla_xU+(1-\alpha_k)\nabla_x(g_\theta\circ s) ∇ x log ν k θ = − β ∇ x U + ( 1 − α k ) ∇ x ( g θ ∘ s ) .
Measure the sampling lag by
H t = ∑ k c k { KL ( ρ k , t ∥ ρ k θ t ) α k + KL ( ν k , t ∥ ν k θ t ) 1 − α k } . (35) H_t=\sum_k c_k\left\{\frac{\operatorname{KL}(\rho_{k,t}\Vert\rho_k^{\theta_t})}{\alpha_k}+\frac{\operatorname{KL}(\nu_{k,t}\Vert\nu_k^{\theta_t})}{1-\alpha_k}\right\}. \tag*{(35)} H t = k ∑ c k { α k KL ( ρ k , t ∥ ρ k θ t ) + 1 − α k KL ( ν k , t ∥ ν k θ t ) } . (35)
These weights will cancel the tempering factors when we differentiate the relative entropies.
Let F ∗ = min θ F ( θ ) F_*=\min_\theta F(\theta) F ∗ = min θ F ( θ ) be finite and attained. We make the following assumptions on an open neighborhood O \mathcal{O} O of the initial sublevel { θ : F ( θ ) ≤ F ( θ 0 ) + H 0 } \{\theta:F(\theta)\le F(\theta_0)+H_0\} { θ : F ( θ ) ≤ F ( θ 0 ) + H 0 } :
(C1) The normalizers are finite, g θ g_\theta g θ is differentiable in θ \theta θ and smooth enough in space for equation (34) , and sup θ ∈ O , s ∥ f θ ( s ) ∥ ≤ B < ∞ \sup_{\theta\in\mathcal{O},s}\|f_\theta(s)\|\le B<\infty sup θ ∈ O , s ∥ f θ ( s ) ∥ ≤ B < ∞ . Differentiation under the integrals is justified.
(C2) The evolution is well posed from initial laws with finite relative entropy. Entropy differentiation and integration by parts are valid, with periodic or zero-flux boundaries, or sufficient decay on unbounded domains.
(C3) The instantaneous laws satisfy uniform log-Sobolev inequalities on their respective spaces:
I ( ζ ∥ P k θ ) ≥ 2 λ s , k KL ( ζ ∥ P k θ ) , I ( ζ ∥ ν k θ ) ≥ 2 λ x , k KL ( ζ ∥ ν k θ ) , I(\zeta\Vert P_k^\theta) \ge2\lambda_{s,k}\operatorname{KL}(\zeta\Vert P_k^\theta), \qquad I(\zeta\Vert\nu_k^\theta) \ge2\lambda_{x,k}\operatorname{KL}(\zeta\Vert\nu_k^\theta), I ( ζ ∥ P k θ ) ≥ 2 λ s , k KL ( ζ ∥ P k θ ) , I ( ζ ∥ ν k θ ) ≥ 2 λ x , k KL ( ζ ∥ ν k θ ) ,
where I ( ζ ∥ π ) = ∫ ∥ ∇ log ( ζ / π ) ∥ 2 d ζ I(\zeta\Vert\pi)=\int\lVert\nabla\log(\zeta/\pi)\rVert^2\,\mathrm{d}\zeta I ( ζ ∥ π ) = ∫ ∥ ∇ log ( ζ / π ) ∥ 2 d ζ . With constants positive and uniform on O \mathcal{O} O , set a = min k { d s , k λ s , k , d x , k λ x , k } > 0 a=\min_k\{d_{s,k}\lambda_{s,k},d_{x,k}\lambda_{x,k}\}>0 a = min k { d s , k λ s , k , d x , k λ x , k } > 0 .
(C4) The objective satisfies the Polyak–Łojasiewicz inequality ∥ ∇ F ( θ ) ∥ 2 ≥ 2 μ ( F ( θ ) − F ∗ ) \lVert\nabla F(\theta)\rVert^2 \ge2\mu(F(\theta)-F_*) ∥ ∇ F ( θ ) ∥ 2 ≥ 2 μ ( F ( θ ) − F ∗ ) on O \mathcal{O} O , for some μ > 0 \mu>0 μ > 0 .
In (C3), the physical inequality is required for the full microscopic law. Uniform constants follow, for example, from the Holley–Stroock perturbation lemma when the model gives a uniformly bounded tilt of a reference law satisfying a log-Sobolev inequality [25 ] .
Theorem S6 (Convergence of the two-population ladder). Under (C1)–(C4), for 0 < η < a / B 2 0<\eta<a/B^2 0 < η < a / B 2 , the evolution in equation (34) obeys
F ( θ t ) − F ∗ + H t ≤ e − κ t [ F ( θ 0 ) − F ∗ + H 0 ] , κ = 2 min { η μ , a − η B 2 } . (36) F(\theta_t)-F_*+H_t\le e^{-\kappa t}[F(\theta_0)-F_*+H_0], \qquad\kappa=2\min\{\eta\mu,a-\eta B^2\}. \tag*{(36)} F ( θ t ) − F ∗ + H t ≤ e − κ t [ F ( θ 0 ) − F ∗ + H 0 ] , κ = 2 min { η μ , a − η B 2 } . (36) Thus the objective error and the weighted sampling lag decay together.
Proof. Let e e e denote the error in the population gradient, and write
H s , k = KL ( ρ k ∥ P k θ t ) , H x , k = KL ( ν k ∥ ν k θ t ) , H_{s,k}=\operatorname{KL}(\rho_k\Vert P_k^{\theta_t}), \qquad H_{x,k}=\operatorname{KL}(\nu_k\Vert\nu_k^{\theta_t}), H s , k = KL ( ρ k ∥ P k θ t ) , H x , k = KL ( ν k ∥ ν k θ t ) ,
e k = ( E ν k − E ν k θ t ) [ f θ t ∘ s ] − ( E ρ k − E P k θ t ) f θ t , e = ∑ k c k e k . e_k=(\mathbb{E}_{\nu_k}-\mathbb{E}_{\nu_k^{\theta_t}})[f_{\theta_t}\circ s]-(\mathbb{E}_{\rho_k}-\mathbb{E}_{P_k^{\theta_t}})f_{\theta_t}, \qquad e=\sum_k c_k e_k. e k = ( E ν k − E ν k θ t ) [ f θ t ∘ s ] − ( E ρ k − E P k θ t ) f θ t , e = k ∑ c k e k .
By equation (10) , the parameter equation is θ ˙ = − η ( ∇ F + e ) \dot{\theta}=-\eta(\nabla F+e) θ ˙ = − η ( ∇ F + e ) .
To differentiate H H H , first note that normalization of the target densities gives
∇ θ log P k θ = − α k ( f θ − E P k θ f θ ) , ∇ θ log ν k θ = ( 1 − α k ) ( f θ ∘ s − E ν k θ [ f θ ∘ s ] ) . \nabla_\theta\log P_k^\theta=-\alpha_k(f_\theta-\mathbb{E}_{P_k^\theta}f_\theta), \qquad\nabla_\theta\log\nu_k^\theta=(1-\alpha_k)(f_\theta\circ s-\mathbb{E}_{\nu_k^\theta}[f_\theta\circ s]). ∇ θ log P k θ = − α k ( f θ − E P k θ f θ ) , ∇ θ log ν k θ = ( 1 − α k ) ( f θ ∘ s − E ν k θ [ f θ ∘ s ]) .
Integration by parts in equation (34) gives
H ˙ s , k = − d s , k I ( ρ k ∥ P k θ t ) + α k θ ˙ T ( E ρ k − E P k θ t ) f θ t , (37) \dot{H}_{s,k}=-d_{s,k}I(\rho_k\Vert P_k^{\theta_t})+\alpha_k\dot{\theta}^{\mathsf T}(\mathbb{E}_{\rho_k}-\mathbb{E}_{P_k^{\theta_t}})f_{\theta_t}, \tag*{(37)} H ˙ s , k = − d s , k I ( ρ k ∥ P k θ t ) + α k θ ˙ T ( E ρ k − E P k θ t ) f θ t , (37)
H ˙ x , k = − d x , k I ( ν k ∥ ν k θ t ) − ( 1 − α k ) θ ˙ T ( E ν k − E ν k θ t ) [ f θ t ∘ s ] . (38) \dot{H}_{x,k}=-d_{x,k}I(\nu_k\Vert\nu_k^{\theta_t})-(1-\alpha_k)\dot{\theta}^{\mathsf T}(\mathbb{E}_{\nu_k}-\mathbb{E}_{\nu_k^{\theta_t}})[f_{\theta_t}\circ s]. \tag*{(38)} H ˙ x , k = − d x , k I ( ν k ∥ ν k θ t ) − ( 1 − α k ) θ ˙ T ( E ν k − E ν k θ t ) [ f θ t ∘ s ] . (38)
Multiplying by c k / α k c_k/\alpha_k c k / α k and c k / ( 1 − α k ) c_k/(1-\alpha_k) c k / ( 1 − α k ) , respectively, and summing over levels combines the moving-target terms into − θ ˙ T e -\dot{\theta}^{\mathsf T}e − θ ˙ T e . The log-Sobolev inequalities in (C3) therefore give
H ˙ ≤ − 2 a H − θ ˙ T e . (39) \dot{H}\le-2aH-\dot{\theta}^{\mathsf T}e. \tag*{(39)} H ˙ ≤ − 2 a H − θ ˙ T e . (39)
The gradient error is controlled by the same entropies. For a vector-valued function f f f with ∥ f ∥ ≤ B \lVert f\rVert\le B ∥ f ∥ ≤ B , applying Pinsker’s inequality to unit scalar projections gives ∥ ( E ζ − E π ) f ∥ ≤ 2 B KL ( ζ ∥ π ) \lVert(\mathbb{E}_\zeta-\mathbb{E}_\pi)f\rVert\le\sqrt{2}B\sqrt{\operatorname{KL}(\zeta\Vert\pi)} ∥( E ζ − E π ) f ∥ ≤ 2 B KL ( ζ ∥ π ) . Applying this to both populations and then using weighted Cauchy–Schwarz yields
∥ e ∥ 2 ≤ 2 B 2 [ ∑ k c k ( H s , k + H x , k ) ] 2 \lVert e\rVert^2\le2B^2\left[\sum_k c_k\left(\sqrt{H_{s,k}}+\sqrt{H_{x,k}}\right)\right]^2 ∥ e ∥ 2 ≤ 2 B 2 [ k ∑ c k ( H s , k + H x , k ) ] 2
≤ 2 B 2 [ ∑ k c k { α k + ( 1 − α k ) } ] [ ∑ k c k { H s , k α k + H x , k 1 − α k } ] = 2 B 2 H . (40) \le2B^2\left[\sum_k c_k\{\alpha_k+(1-\alpha_k)\}\right]\left[\sum_k c_k\left\{\frac{H_{s,k}}{\alpha_k}+\frac{H_{x,k}}{1-\alpha_k}\right\}\right]=2B^2H. \tag*{(40)} ≤ 2 B 2 [ k ∑ c k { α k + ( 1 − α k )} ] [ k ∑ c k { α k H s , k + 1 − α k H x , k } ] = 2 B 2 H . (40)
Now let V = F − F ∗ + H V = F - F_* + H V = F − F ∗ + H . Combining equation (39) with the parameter equation cancels the mixed gradient terms:
V ˙ ≤ ( ∇ F − e ) ⊤ θ ˙ − 2 a H = − η ( ∇ F − e ) ⊤ ( ∇ F + e ) − 2 a H = − η ∥ ∇ F ∥ 2 + η ∥ e ∥ 2 − 2 a H ≤ − η ∥ ∇ F ∥ 2 − 2 ( a − η B 2 ) H ≤ − κ V . \begin{aligned}
\dot{V} \le(\nabla F - e)^\top\dot{\theta} - 2aH \\
&= -\eta(\nabla F - e)^\top(\nabla F + e) - 2aH \\
&= -\eta\lVert\nabla F\rVert^2 + \eta\lVert e\rVert^2 - 2aH \\
&\le-\eta\lVert\nabla F\rVert^2 - 2(a - \eta B^2)H \le-\kappa V.
\end{aligned} V ˙ ≤ ( ∇ F − e ) ⊤ θ ˙ − 2 a H = − η ( ∇ F − e ) ⊤ ( ∇ F + e ) − 2 a H = − η ∥ ∇ F ∥ 2 + η ∥ e ∥ 2 − 2 a H ≤ − η ∥ ∇ F ∥ 2 − 2 ( a − η B 2 ) H ≤ − κV .
The last inequality uses (C4), and Gronwall’s inequality gives equation (36) . This estimate also gives F ( θ t ) ≤ F ∗ + V t ≤ F ( θ 0 ) + H 0 F(\theta_t) \le F_* + V_t \le F(\theta_0) + H_0 F ( θ t ) ≤ F ∗ + V t ≤ F ( θ 0 ) + H 0 . Applying the argument up to a possible first exit from O \mathcal{O} O therefore rules out such an exit and proves the estimate for all time.
The physical endpoint. The same argument covers a ladder with α K = 1 \alpha_K = 1 α K = 1 , where the last physical replica has the fixed target ν K θ = ϱ \nu_K^\theta= \varrho ν K θ = ϱ . The gradient formula in equation (10) still holds under the endpoint moment conditions of Proposition S2, as shown in Proposition S4. To adapt the entropy H H H , keep the terms for k < K k < K k < K and replace the last pair by
c K { KL ( ρ K ∥ P K θ ) + w KL ( ν K ∥ ϱ ) } , w > 0. c_K\{\operatorname{KL}(\rho_K\Vert P_K^\theta) + w\operatorname{KL}(\nu_K\Vert\varrho)\}, \qquad w > 0. c K { KL ( ρ K ∥ P K θ ) + w KL ( ν K ∥ ϱ )} , w > 0.
Write H ′ H' H ′ for this modified entropy and use (C1)–(C4) on an open neighborhood of { θ : F ( θ ) ≤ F ( θ 0 ) + H 0 ′ } \{\theta: F(\theta) \le F(\theta_0) + H'_0\} { θ : F ( θ ) ≤ F ( θ 0 ) + H 0 ′ } , with the physical log-Sobolev inequality at level K K K imposed on ϱ \varrho ϱ . Since ϱ \varrho ϱ does not depend on θ \theta θ , its relative entropy has no moving-target term. Thus, with e x , K = ( E ν K − E ϱ ) [ f θ ∘ s ] e_{x,K} = (\mathbb{E}_{\nu_K} - \mathbb{E}_{\varrho})[f_\theta\circ s] e x , K = ( E ν K − E ϱ ) [ f θ ∘ s ] , the entropy estimate becomes
H ˙ ′ ≤ − 2 a H ′ − θ ˙ ⊤ ( e − c K e x , K ) . \dot{H}' \le-2aH' - \dot{\theta}^{\top}(e - c_K e_{x,K}). H ˙ ′ ≤ − 2 a H ′ − θ ˙ ⊤ ( e − c K e x , K ) .
Pinsker’s inequality and the same weighted Cauchy–Schwarz estimate give
∥ e ∥ 2 ≤ 2 B 2 ( 1 + c K / w ) H ′ , c K 2 ∥ e x , K ∥ 2 ≤ 2 B 2 ( c K / w ) H ′ . \lVert e\rVert^2 \le2B^2(1 + c_K/w)H', \qquad c_K^2\lVert e_{x,K}\rVert^2 \le2B^2(c_K/w)H'. ∥ e ∥ 2 ≤ 2 B 2 ( 1 + c K / w ) H ′ , c K 2 ∥ e x , K ∥ 2 ≤ 2 B 2 ( c K / w ) H ′ .
For V ′ = F − F ∗ + H ′ V' = F - F_* + H' V ′ = F − F ∗ + H ′ , the mixed terms therefore cancel as before, except for the contribution from the last physical replica:
V ˙ ′ ≤ − η ∥ ∇ F ∥ 2 + η ∥ e ∥ 2 − η c K ( ∇ F + e ) ⊤ e x , K − 2 a H ′ ≤ − η 2 ∥ ∇ F ∥ 2 − [ 2 a − η B 2 ( 3 + 5 c K / w ) ] H ′ . \begin{aligned}
\dot{V}' \le-\eta\lVert\nabla F\rVert^2 + \eta\lVert e\rVert^2 - \eta c_K(\nabla F + e)^\top e_{x,K} - 2aH' \\
&\le-\frac{\eta}{2}\lVert\nabla F\rVert^2 - [2a - \eta B^2(3 + 5c_K/w)]H'.
\end{aligned} V ˙ ′ ≤ − η ∥ ∇ F ∥ 2 + η ∥ e ∥ 2 − η c K ( ∇ F + e ) ⊤ e x , K − 2 a H ′ ≤ − 2 η ∥ ∇ F ∥ 2 − [ 2 a − η B 2 ( 3 + 5 c K / w )] H ′ .
Here we used Young’s inequality in the form ∣ c K ( ∇ F + e ) ⊤ e x , K ∣ ≤ 1 2 ∥ ∇ F ∥ 2 + 1 2 ∥ e ∥ 2 + c K 2 ∥ e x , K ∥ 2 |c_K(\nabla F + e)^\top e_{x,K}| \le\frac{1}{2}\lVert\nabla F\rVert^2 + \frac{1}{2}\lVert e\rVert^2 + c_K^2\lVert e_{x,K}\rVert^2 ∣ c K ( ∇ F + e ) ⊤ e x , K ∣ ≤ 2 1 ∥ ∇ F ∥ 2 + 2 1 ∥ e ∥ 2 + c K 2 ∥ e x , K ∥ 2 . For 0 < η < 2 a / [ B 2 ( 3 + 5 c K / w ) ] 0 < \eta< 2a/[B^2(3 + 5c_K/w)] 0 < η < 2 a / [ B 2 ( 3 + 5 c K / w )] , (C4) then gives
V t ′ ≤ e − κ ′ t V 0 ′ , κ ′ = min { η μ , 2 a − η B 2 ( 3 + 5 c K / w ) } . V'_t \le e^{-\kappa't}V'_0, \qquad\kappa' = \min\{\eta\mu, 2a - \eta B^2(3 + 5c_K/w)\}. V t ′ ≤ e − κ ′ t V 0 ′ , κ ′ = min { η μ , 2 a − η B 2 ( 3 + 5 c K / w )} .
As in the proof above, F ( θ t ) ≤ F ( θ 0 ) + H 0 ′ F(\theta_t) \le F(\theta_0) + H'_0 F ( θ t ) ≤ F ( θ 0 ) + H 0 ′ keeps the trajectory in the region where the assumptions hold. Without (C4), integrating the bound on V ˙ ′ \dot{V}' V ˙ ′ gives the time-averaged estimate in equation (41) , with 2 V 0 ′ / ( η T ) 2V'_0/(\eta T) 2 V 0 ′ / ( η T ) on the right.
Corollary S7 (Time-averaged stationarity). Under (C1)–(C3) and 0 < η < a / B 2 0 < \eta< a/B^2 0 < η < a / B 2 , for every T > 0 T > 0 T > 0 ,
1 T ∫ 0 T ∥ ∇ F ( θ t ) ∥ 2 d t ≤ F ( θ 0 ) − F ∗ + H 0 η T . (41) \frac{1}{T}\int_0^T \lVert\nabla F(\theta_t)\rVert^2\,\mathrm{d}t \le\frac{F(\theta_0) - F_* + H_0}{\eta T}. \tag*{(41)} T 1 ∫ 0 T ∥ ∇ F ( θ t ) ∥ 2 d t ≤ η T F ( θ 0 ) − F ∗ + H 0 . (41)
Proof. The proof above gives V ˙ ≤ − η ∥ ∇ F ∥ 2 − 2 ( a − η B 2 ) H \dot{V} \le-\eta\lVert\nabla F\rVert^2 - 2(a - \eta B^2)H V ˙ ≤ − η ∥ ∇ F ∥ 2 − 2 ( a − η B 2 ) H before invoking (C4). Integrating over [ 0 , T ] [0,T] [ 0 , T ] and using V T ≥ 0 V_T \ge0 V T ≥ 0 proves the bound.
General reference measures
The same variational argument applies to a general reference measure. Let r r r be sigma-finite and let the probability measure p p p have density f = d p / d r > 0 f = \mathrm{d}p/\mathrm{d}r > 0 f = d p / d r > 0 r r r -almost everywhere. For r = d s r = \mathrm{d}s r = d s , this f f f is the physical CV density p ( s ) p(s) p ( s ) . Writing g = β G g = \beta G g = β G , define
ℓ α [ g ] = 1 α log ∫ e − α g d r + 1 1 − α log ∫ e ( 1 − α ) g d p . (42) \ell_\alpha[g] = \frac{1}{\alpha}\log\int e^{-\alpha g}\,\mathrm{d}r + \frac{1}{1-\alpha}\log\int e^{(1-\alpha)g}\,\mathrm{d}p. \tag*{(42)} ℓ α [ g ] = α 1 log ∫ e − α g d r + 1 − α 1 log ∫ e ( 1 − α ) g d p . (42)
For the physical Lebesgue reference, ℓ α [ β G ] = β L ^ β [ G ] \ell_\alpha[\beta G] = \beta\widehat{L}_\beta[G] ℓ α [ β G ] = β L β [ G ] . When Z g = ∫ e − g d r < ∞ Z_g = \int e^{-g}\,\mathrm{d}r < \infty Z g = ∫ e − g d r < ∞ , the normalized residual model is
q g ( d y ) = Z g − 1 e − g ( y ) r ( d y ) . (43) q_g(\mathrm{d}y) = Z_g^{-1}e^{-g(y)}r(\mathrm{d}y). \tag*{(43)} q g ( d y ) = Z g − 1 e − g ( y ) r ( d y ) . (43)
Proposition S8 (Reference-measure minimizer). For 0 < α < 1 0 < \alpha< 1 0 < α < 1 with M ( α ) = ∫ f α d r < ∞ M(\alpha) = \int f^\alpha\,\mathrm{d}r < \infty M ( α ) = ∫ f α d r < ∞ , the functional in equation (42) , restricted to g g g for which both integrals are finite, is minimized exactly at g ⋆ = − log f + C g_\star= -\log f + C g ⋆ = − log f + C . At the minimum, the two tilted laws coincide with f α d r / M ( α ) f^\alpha\mathrm{d}r/M(\alpha) f α d r / M ( α ) .
Proof. As in the proof of Proposition S1, Hölder’s inequality gives
M ( α ) = ∫ ( e − α g ) 1 − α ( f e ( 1 − α ) g ) α d r ≤ ( ∫ e − α g d r ) 1 − α ( ∫ e ( 1 − α ) g d p ) α . M(\alpha) = \int\left(e^{-\alpha g}\right)^{1-\alpha}\left(fe^{(1-\alpha)g}\right)^\alpha\,\mathrm{d}r \le\left(\int e^{-\alpha g}\,\mathrm{d}r\right)^{1-\alpha}\left(\int e^{(1-\alpha)g}\,\mathrm{d}p\right)^\alpha. M ( α ) = ∫ ( e − α g ) 1 − α ( f e ( 1 − α ) g ) α d r ≤ ( ∫ e − α g d r ) 1 − α ( ∫ e ( 1 − α ) g d p ) α .
Equality requires the two factors to be proportional, which is equivalent to g = − log f + C g = -\log f + C g = − log f + C . At this choice, both normalized densities relative to r r r equal f α / M ( α ) f^\alpha/M(\alpha) f α / M ( α ) .
Positive-weight effective sample size
Proposition S9 (Population normalized ESS). At the population optimum, the fixed-data weights are proportional to f α − 1 f^{\alpha-1} f α − 1 . If M ( 2 α − 1 ) < ∞ M(2\alpha- 1) < \infty M ( 2 α − 1 ) < ∞ , their population normalized effective sample size is
( E p w ) 2 E p w 2 = M ( α ) 2 M ( 2 α − 1 ) . (44) \frac{(\mathbb{E}_p w)^2}{\mathbb{E}_p w^2} = \frac{M(\alpha)^2}{M(2\alpha- 1)}. \tag*{(44)} E p w 2 ( E p w ) 2 = M ( 2 α − 1 ) M ( α ) 2 . (44) For independent y j ∼ p y_j \sim p y j ∼ p with w j = w ( y j ) w_j = w(y_j) w j = w ( y j ) , the empirical quantity ( ∑ j w j ) 2 / [ n ∑ j w j 2 ] (\sum_j w_j)^2/[n\sum_j w_j^2] ( ∑ j w j ) 2 / [ n ∑ j w j 2 ] converges to this value almost surely. At α = 1 / 2 \alpha= 1/2 α = 1/2 , the denominator is M ( 0 ) = r ( supp p ) M(0) = r(\operatorname{supp}p) M ( 0 ) = r ( supp p ) .
Proof. The scale of the weights cancels in the ratio, so take w = f α − 1 w = f^{\alpha-1} w = f α − 1 . Since d p = f d r \mathrm{d}p = f\mathrm{d}r d p = f d r , we have E p w = ∫ f α d r = M ( α ) \mathbb{E}_p w = \int f^\alpha\,\mathrm{d}r = M(\alpha) E p w = ∫ f α d r = M ( α ) and E p w 2 = ∫ f 2 α − 1 d r = M ( 2 α − 1 ) \mathbb{E}_p w^2 = \int f^{2\alpha-1}\,\mathrm{d}r = M(2\alpha- 1) E p w 2 = ∫ f 2 α − 1 d r = M ( 2 α − 1 ) , proving equation (44) . The assumed second moment allows the strong law of large numbers to be applied to both empirical moments, giving the almost-sure limit.
Learning from existing data
When equilibrium configurations x j ⋆ ∼ ϱ x_j^\star\sim\varrho x j ⋆ ∼ ϱ are already available, their CV values y j = s ( x j ⋆ ) y_j = s(x_j^\star) y j = s ( x j ⋆ ) can replace the physical walkers. The physical expectation in equation (7) is then estimated using normalized importance weights
w ˉ j ( θ ) = e ( β − β ^ ) G θ ( y j ) ∑ ℓ e ( β − β ^ ) G θ ( y ℓ ) . (45) \bar{w}_j(\theta) = \frac{e^{(\beta-\hat{\beta})G_\theta(y_j)}}{\sum_\ell e^{(\beta-\hat{\beta})G_\theta(y_\ell)}}. \tag*{(45)} w ˉ j ( θ ) = ∑ ℓ e ( β − β ^ ) G θ ( y ℓ ) e ( β − β ^ ) G θ ( y j ) . (45)
This leads to the following weighted-data consistency result.
Proposition S10 (Weighted-data gradient). Fix θ \theta θ and let x j ∗ x_j^* x j ∗ be independent samples from ϱ \varrho ϱ , with y j = s ( x j ∗ ) y_j=s(x_j^*) y j = s ( x j ∗ ) . Suppose B G θ < ∞ B_{G_\theta}<\infty B G θ < ∞ and E p [ e ( β − β ^ ) G θ ∥ ∇ θ G θ ∥ ] < ∞ \mathbb{E}_p[e^{(\beta-\hat{\beta})G_\theta}\lVert\nabla_\theta G_\theta\rVert]<\infty E p [ e ( β − β ^ ) G θ ∥ ∇ θ G θ ∥] < ∞ . Then the weights in equation (45) satisfy
∑ j = 1 n w ˉ j ( θ ) ∇ θ G θ ( y j ) ⟶ E ν G θ ∇ θ G θ ∘ s almost surely as n → ∞ . (46) \sum_{j=1}^{n}\bar{w}_j(\theta)\nabla_\theta G_\theta(y_j)\longrightarrow\mathbb{E}_{\nu_{G_\theta}}\nabla_\theta G_\theta\circ s \quad\text{almost surely as } n\to\infty. \tag*{(46)} j = 1 ∑ n w ˉ j ( θ ) ∇ θ G θ ( y j ) ⟶ E ν G θ ∇ θ G θ ∘ s almost surely as n → ∞. (46) Moreover, the corresponding weighted population objective has minimizer G ⋆ + C G_\star+C G ⋆ + C .
Proof. Using the CV marginal in equation (19) , the required expectation can be written as
E ν G θ ∇ θ G θ ∘ s = E p [ e ( β − β ^ ) G θ ∇ θ G θ ] E p [ e ( β − β ^ ) G θ ] \mathbb{E}_{\nu_{G_\theta}}\nabla_\theta G_\theta\circ s =
\frac{\mathbb{E}_p[e^{(\beta-\hat{\beta})G_\theta}\nabla_\theta G_\theta]}
{\mathbb{E}_p[e^{(\beta-\hat{\beta})G_\theta}]} E ν G θ ∇ θ G θ ∘ s = E p [ e ( β − β ^ ) G θ ] E p [ e ( β − β ^ ) G θ ∇ θ G θ ]
The assumed moments allow the strong law of large numbers to be applied to both sample averages. Their ratio converges almost surely because the limiting denominator is positive, proving equation (46) . Together with the auxiliary expectation, this yields the gradient in equation (7) of the original population objective, whose minimizer is G ⋆ + C G_\star+C G ⋆ + C by Proposition S1. The exponential weighting is essential: using unweighted samples from p p p would instead give the functional stationarity condition P G = p P_G=p P G = p , equivalently G = G ⋆ / α + C G=G_\star/\alpha+C G = G ⋆ / α + C .
The auxiliary walkers still sample P G θ P_{G_\theta} P G θ , so the resulting update is a weighted-data version of the same learning rule. With g = β G g=\beta G g = β G and γ = α − 1 \gamma=\alpha-1 γ = α − 1 , the objective coincides with the PS-CD/γ \gamma γ -cross-entropy objective continued to negative γ \gamma γ [20 , 60 ] , with the Rényi variational representation described in Section D [8 ] .
The same ratio argument also yields within-population sample-reuse weights. If a batch was generated at G 0 G_0 G 0 , then the current-to-generating density ratios are proportional to
w ( s ) = e − β ^ ( G − G 0 ) , w ( x ) = e ( β − β ^ ) ( G − G 0 ) ∘ s . (47) w^{(s)}=e^{-\hat{\beta}(G-G_0)},\qquad w^{(x)}=e^{(\beta-\hat{\beta})(G-G_0)\circ s}. \tag*{(47)} w ( s ) = e − β ^ ( G − G 0 ) , w ( x ) = e ( β − β ^ ) ( G − G 0 ) ∘ s . (47)
Normalization within each pool removes the unknown partition functions:
E P G 0 [ w ( s ) f ] = A G A G 0 E P G f , E ν G 0 [ w ( x ) f ∘ s ] = B G B G 0 E ν G f ∘ s . \mathbb{E}_{P_{G_0}}[w^{(s)}f]=\frac{A_G}{A_{G_0}}\mathbb{E}_{P_G}f,\qquad
\mathbb{E}_{\nu_{G_0}}[w^{(x)}f\circ s]=\frac{B_G}{B_{G_0}}\mathbb{E}_{\nu_G}f\circ s. E P G 0 [ w ( s ) f ] = A G 0 A G E P G f , E ν G 0 [ w ( x ) f ∘ s ] = B G 0 B G E ν G f ∘ s .
Hence, if G G G and G 0 G_0 G 0 are admissible, f f f is integrable under the current law, and the batch satisfies a law of large numbers under its generating law, the corresponding self-normalized estimators are consistent. Their weight concentration is monitored by the ESS criterion described in Section E.
The single-level update also admits a scalar-objective interpretation in self-consistent well-tempered VES. For V θ = − ( 1 − α ) G θ V_\theta=-(1-\alpha)G_\theta V θ = − ( 1 − α ) G θ and the instantaneous target t θ = P G θ t_\theta=P_{G_\theta} t θ = P G θ , differentiating the VES functional while holding t t t fixed gives
∇ θ Ω [ V θ ; t ] ∣ t = t θ = ( 1 − α ) ∇ θ L β ^ [ G θ ] . (48) \left.\nabla_\theta\Omega[V_\theta;t]\right|_{t=t_\theta}=(1-\alpha)\nabla_\theta\mathcal{L}_{\hat{\beta}}[G_\theta]. \tag*{(48)} ∇ θ Ω [ V θ ; t ] ∣ t = t θ = ( 1 − α ) ∇ θ L β ^ [ G θ ] . (48)
The target convention and the recursive neural VES update are detailed in Section D.1 [9 , 58 , 59 ] .
Physical systems: models and protocols
This appendix gives the model, force field, collective-variable and network settings behind § 3 , together with the trust region, readout and gauge conventions used for every number reported there.
Unless stated otherwise, reported estimates are means ± \pm ± sample standard deviations across independent runs. Errors are computed for each run before averaging. Free-energy maps formed by averaging model predictions before marginalization are identified explicitly. Molecular times are per physical replica, and aggregate MD is summed over replicas.
Alanine dipeptide
System and CVs. Explicit-solvent alanine dipeptide, AMBER99SB [27 ] with TIP3P water [34 ] and particle-mesh Ewald electrostatics, 298.15 K, 2 fs timestep, bonds to hydrogen constrained. The CVs are the backbone dihedrals ϕ \phi ϕ , ψ \psi ψ , both periodic.
Model and training. G θ G_\theta G θ is a SIREN [54 ] with one hidden layer of width 32, ω 0 = 16 \omega_0 = 16 ω 0 = 16 and a periodic input embedding. Each replica carries 10 4 10^4 1 0 4 auxiliary walkers with γ s = 10 \gamma_s = 10 γ s = 10 and γ x = 100 \gamma_x = 100 γ x = 100 . Training uses AdamW [38 ] at learning rate 3 × 10 − 4 3 \times10^{-4} 3 × 1 0 − 4 , up to 80 inner iterations per epoch, and epochs of 1000 MD steps, 50 per checkpoint (100 ps). The single replica uses α = 0.01 \alpha= 0.01 α = 0.01 . Both four-level ladders use α = ( 0.1 , 0.05 , 0.02 , 0.01 ) \alpha= (0.1, 0.05, 0.02, 0.01) α = ( 0.1 , 0.05 , 0.02 , 0.01 ) with equal loss weights. L-RE attempts exchange of physical configurations between adjacent levels every epoch. L-NRE uses the same ladder without exchange. The single replica starts from the same configurations as the OPES runs.
Reference and analysis. The reference surface is the free energy from 100 ns of adaptive-biasing-force sampling by Zubieta Rico et al. [63 ] , on a 100 × 100 100 \times100 100 × 100 periodic grid. Figure 3 reports mean-aligned RMSE over this full grid. OPES and TAMD are scored only on cells where their native free-energy estimators are finite. VESPER predictions use parameter averages over the trailing eight available checkpoints, or all available checkpoints at earlier times. RMSE curves average the individual-run errors. The displayed surfaces average the mean-aligned predictions and shift the resulting minimum to zero. Physical points overlaid on the maps are drawn from the last nanosecond and auxiliary points from the final saved populations, balanced across runs and ladder levels.
Alanine tetrapeptide
System and CVs. Gas-phase Ace-( A l a ) 3 (\mathrm{Ala})_3 ( Ala ) 3 -NMe, AMBER99SB [27 ] without cutoff, 298.15 K, 2 fs timestep, bonds to hydrogen constrained. The three-CV model biases ( ϕ 1 , ϕ 2 , ϕ 3 ) (\phi_1, \phi_2, \phi_3) ( ϕ 1 , ϕ 2 , ϕ 3 ) and the six-CV model adds ψ 1 \psi_1 ψ 1 , ψ 2 \psi_2 ψ 2 and ψ 3 \psi_3 ψ 3 .
Model and training. G θ G_\theta G θ is a SIREN with hidden widths (14, 4), ω 0 = 16 \omega_0 = 16 ω 0 = 16 and a periodic input embedding, so the two models differ only in the input layer (163 and 247 parameters). Each replica carries 2 × 10 4 2 \times10^4 2 × 1 0 4 auxiliary walkers with γ s = γ x = 100 \gamma_s = \gamma_x = 100 γ s = γ x = 100 . Training uses AdamW at 3 × 10 − 4 3 \times10^{-4} 3 × 1 0 − 4 , up to 80 inner iterations, and epochs of 1000 steps, 100 per checkpoint (200 ps). The single replica uses α = 0.01 \alpha= 0.01 α = 0.01 , and the ladder arms are those of ADP. Both models train at a constant step size throughout. Each model is read as the parameter-SWA10 average over checkpoints from 8.2–10.0 ns.
Readout and analysis. Temporal correlations compare trailing ten-checkpoint parameter averages ending at 5 and 10 ns, using windows of 3.2–5.0 and 8.2–10.0 ns, respectively. Pearson correlations are calculated on 262,144 evaluation points pooled from four scrambled Sobol sets spanning the full six-dimensional CV domain. Reported values are the mean and sample standard deviation across three independent runs.
Figure 4 shows physical Boltzmann marginals at 298.15 K. The full physical free-energy predictions from the three independent runs of the four-level exchange ladder are averaged before Boltzmann integration and marginalization. The retained ( ϕ 1 , ϕ 2 ) (\phi_1,\phi_2) ( ϕ 1 , ϕ 2 ) , ( ϕ 1 , ϕ 3 ) (\phi_1,\phi_3) ( ϕ 1 , ϕ 3 ) and ( ϕ 2 , ϕ 3 ) (\phi_2,\phi_3) ( ϕ 2 , ϕ 3 ) planes are complete 64 × 64 64 \times64 64 × 64 grids. Six-CV maps integrate omitted ψ \psi ψ coordinates on 32-point axes and the remaining omitted ϕ \phi ϕ on its 64-point grid, while three-CV maps integrate the omitted ϕ \phi ϕ on a 256-point grid. Display surfaces are shifted to zero minimum and use a common 0–120 kJ mol− 1 ^{-1} − 1 scale with 10 kJ mol− 1 ^{-1} − 1 contours. The three-to-six-CV squared-Pearson correlations over all plane cells are 0.951, 0.895 and 0.890.
Chignolin
System and CVs. CLN025 [26 ] in explicit solvent, ff14SB [39 ] with TIP3P [34 ] , 340 K, 2 fs timestep, with its seventeen backbone dihedrals as CVs. Structural deviation is measured by Cα \alpha α root-mean-square deviation (RMSD) after Kabsch superposition [35 ] .
Model and training. G θ G_\theta G θ is a SIREN with hidden widths (48, 24), ω 0 = 1 \omega_0 = 1 ω 0 = 1 and a periodic input embedding (2,881 parameters). Twelve replicas span a geometric ladder from α = 1 \alpha= 1 α = 1 to α = 1 / 18 \alpha= 1/18 α = 1/18 , every level exchanging physical configurations with its neighbors every epoch. Each carries 10 5 10^5 1 0 5 auxiliary walkers. Training uses AdamW at learning rate 10 − 4 10^{-4} 1 0 − 4 with a gradient clip of 10 k B T 10 k_B T 10 k B T , up to 100 inner iterations and epochs of 5000 steps, 20 per checkpoint (200 ps), at constant step throughout. Four independent runs extend to 26.2 ns per replica, and the readout is the final checkpoint.
Readout and analysis. Frames are saved every 5 ps, 5,240 per level and run. Full-run occupancy fractions and exchange acceptance pool the four runs over 26.2 ns. Every reported unfolding and refolding event falls within the first 25 ns. Folded and unfolded are defined as Cα \alpha α RMSD below 0.20 nm and above 0.40 nm. An unfolding or refolding event requires the new state to persist for 20 ps and is counted along each configuration’s identity. This dwell criterion is not applied to the framewise occupancy estimates.
The time-resolved occupancies in Figure 6 A,B use non-overlapping 0.2 ns bins, each containing 40 saved frames per run. At each saved time, configurations are assigned to their current tempering levels before RMSD classification. For panel B, the within-bin folded fractions at each level are averaged equally across the four runs. For panel A, each run’s per-level fractions are summed to give the time-averaged numbers of levels in each state. The lines show their mean and the bands their minimum-to-maximum range across runs. Both panels display bins in the first 25 ns. The reported 99.3% unbiased-level occupancy uses the full 26.2 ns trajectory.
The learned surfaces are estimated from 25,000 auxiliary walkers of the unbiased level drawn at random from each run at the readout (100,000 in all), and the MD frames shown in Figure 6 C come from the last nanosecond. For the radius-of-gyration and RMSD map, each walker’s dihedrals are placed on a backbone with the bond lengths, bond angles, peptide ω \omega ω and Pro4 ϕ \phi ϕ of the native structure, and F = − k B T ln p F = -k_B T \ln p F = − k B T ln p (k B T = 2.83 k_B T = 2.83 k B T = 2.83 kJ mol− 1 ^{-1} − 1 ), with p p p an adaptive Gaussian kernel density estimate. The dihedral maps are product von Mises kernel density estimates, compared after mean alignment with the same estimate on the unbiased level’s MD frames where the MD free energy is below 10 kJ mol− 1 ^{-1} − 1 . Over ( ϕ 2 , ψ 2 ) (\phi_2,\psi_2) ( ϕ 2 , ψ 2 ) , ( ϕ 3 , ψ 3 ) (\phi_3,\psi_3) ( ϕ 3 , ψ 3 ) and ( ϕ 5 , ψ 5 ) (\phi_5,\psi_5) ( ϕ 5 , ψ 5 ) , the learned minimum lies 0.42, 0.07 and 0.19 rad from the MD minimum, at points where the MD free energy is 1.3, 0.2 and 0.5 kJ mol− 1 ^{-1} − 1 above its minimum, with RMS differences of 1.9, 2.2 and 2.0 kJ mol− 1 ^{-1} − 1 . Accepted exchanges permute configurations between levels, so every configuration keeps its identity (Section A.5).
Catalytic methane dehydrogenation
System and CVs. Methane on stepped Ni(511) at 700 K uses an Allegro machine-learned interatomic potential [43 ] , deployed through the NequIP toolchain [7 ] and driven by OpenMM [19 ] . The CVs are the two coordination numbers of Jin et al. [33 ] and the periodic fractional coordinate S 1 S_1 S 1 of the carbon atom along the first surface lattice vector. The system has 85 atoms, 25 held fixed, in a triclinic cell. A soft restraint keeps the system in the CV ranges: S 1 ∈ [ 0 , 1 ] S_1 \in[0, 1] S 1 ∈ [ 0 , 1 ] , C N i ∈ [ 0.05 , 3.02 ] \mathrm{CNi} \in[0.05, 3.02] CNi ∈ [ 0.05 , 3.02 ] , and C H ∈ [ 2.95 , 3.98 ] \mathrm{CH} \in[2.95, 3.98] CH ∈ [ 2.95 , 3.98 ] .
Model and training. G θ G_\theta G θ is a SIREN with hidden widths ( 14 , 4 ) (14, 4) ( 14 , 4 ) and ω 0 = 16 \omega_0 = 16 ω 0 = 16 . One replica at α = 0.05 \alpha= 0.05 α = 0.05 carries 6.5 × 10 4 6.5 \times10^{4} 6.5 × 1 0 4 auxiliary walkers, using AdamW at learning rate 10 − 4 10^{-4} 1 0 − 4 , with up to 100 inner iterations and epochs of 5000 steps, 20 per checkpoint, at a 0.25 fs timestep, so one checkpoint is 25 ps. Training uses a gradient clip of 50 k B T k_{\mathrm{B}}T k B T , γ s = 100 \gamma_s = 100 γ s = 100 , γ x = 10 \gamma_x = 10 γ x = 10 , Langevin friction 9.82 p s − 1 9.82\ \mathrm{ps}^{-1} 9.82 ps − 1 and a constant step. The reweighted inner step uses the full trust region of Section B.7. Three independent runs extend to checkpoint 300. Each checkpoint-300 physical free-energy prediction uses deployed parameter EMA decay 0.9.
Catalysis free-energy projections
With S 1 S_1 S 1 the periodic fractional coordinate and y y y , z z z the C–Ni and C–H coordination numbers, the two-dimensional free energies shown in Figure 5 are Boltzmann marginals of the equal-weight mean of the three gauge-aligned checkpoint-300 EMA predictions,
F R ( q R ) = − k B T log ∫ D O ( q R ) exp [ − G m e a n ( q R , q O ) / ( k B T ) ] d q O , F_R(q_R) = -k_{\mathrm{B}}T \log\int_{D_O(q_R)} \exp[-G_{\mathrm{mean}}(q_R,q_O)/(k_{\mathrm{B}}T)]\,\mathrm{d}q_O, F R ( q R ) = − k B T log ∫ D O ( q R ) exp [ − G mean ( q R , q O ) / ( k B T )] d q O ,
at T = 700 K T = 700\ \mathrm{K} T = 700 K , with ordinary measure in the original CV coordinates and integration box D = S 1 [ 0 , 1 ] × y [ 0.02 , 3 ] × z [ 2.90 , 3.94 ] D = S_1[0, 1] \times y[0.02, 3] \times z[2.90, 3.94] D = S 1 [ 0 , 1 ] × y [ 0.02 , 3 ] × z [ 2.90 , 3.94 ] , while the visible route uses S 1 [ 0.02 , 0.98 ] × y [ 0.10 , 2.95 ] × z [ 2.97 , 3.94 ] S_1[0.02, 0.98] \times y[0.10, 2.95] \times z[2.97, 3.94] S 1 [ 0.02 , 0.98 ] × y [ 0.10 , 2.95 ] × z [ 2.97 , 3.94 ] . Each map is shifted to its own minimum for display. The route is initialized by a string-method calculation on the differentiable mean surface and refined by constrained gradient flow, resolving contacts with the route bounds. This yields a locally resolved route within the stated box, but global path optimality is not established. The physisorbed representative is constrained at the lower C–Ni route bound, and the fitted saddle is at S 1 ≈ 0.65 S_1 \approx0.65 S 1 ≈ 0.65 , near the step-edge atoms at S 1 ≈ 0.70 S_1 \approx0.70 S 1 ≈ 0.70 . The route’s G ( 3 ) − G ( 1 ) = 1.12 e V G(3) - G(1) = 1.12\ \mathrm{eV} G ( 3 ) − G ( 1 ) = 1.12 eV is a bounded surface difference. Its 0.25 eV uncertainty is the sample standard deviation of the individual model differences evaluated at the same reactant and saddle points on the mean-surface route.
Ladder configuration by system
Table 1 lists the levels of every molecular system. Each level is one physical trajectory. The auxiliary walkers enter only the training loss. The single-level runs of ADP and ATP use α = 0.01 \alpha= 0.01 α = 0.01 , and ADP also has a single-level run at α = 1 \alpha= 1 α = 1 .
Table 1. Levels per molecular system: tempering exponents α \alpha α , number of levels K K K , and walkers per level.
Reweighted inner steps and the step-size controller
Within an epoch the walker samples generated under G θ 0 G_{\theta_0} G θ 0 are frozen while several parameter steps are taken and reweighted by self-normalized importance sampling at every step (§ 2.4 ). A trust region bounds the log-weights: a soft threshold δ = 5 \delta= 5 δ = 5 on their 95th percentile, a hard threshold δ h a r d = 8 \delta_{\mathrm{hard}} = 8 δ hard = 8 on their maximum, and a floor of 0.2 on the effective sample size of the pool. Exceeding the hard threshold or falling below the floor ends the epoch and forces fresh trajectory data. Exceeding the soft threshold multiplies the parameter step by 1 / q 2 1/q^2 1/ q 2 with q = ∥ log w ∥ 95 / δ q = \lVert\log w \rVert_{95}/\delta q = ∥ log w ∥ 95 / δ . It is used on ADP, ATP and catalysis. Chignolin trains on fresh samples only.
On ADP, the inner-iteration budget and learning rate are governed by a sample-based controller rather than a predefined schedule. The controller latches a decay phase when cumulative relative parameter change across checkpoints exceeds a budget of 160%, then multiplies the learning rate by 0.95 and the inner-iteration count by 0.7 per checkpoint, flooring the count at 5. Once the count reaches its floor and the learning rate is below 5% of its initial value, a final phase continues the decay to a floor of 0.3% and deploys an exponential moving average (EMA) of parameters with decay 0.99 per epoch. ATP uses constant-step training.
Readout, optimizer settings and RMSE gauge
Models use system-specific parameter readouts. ATP predictions use a uniform average of the last ten checkpoints, and ADP uses the last eight available checkpoints. At the reported horizon these ADP parameters have already undergone EMA smoothing. These are the Polyak–Ruppert or stochastic-weight-averaging (SWA) readouts [31 , 47 , 49 ] . Catalysis uses the deployed EMA readout with decay 0.9 at checkpoint 300, and chignolin reports its final checkpoint (26.2 ns per replica). AdamW [38 ] uses decoupled weight decay w = 0.01 w = 0.01 w = 0.01 in every run, and the peptide runs apply the cautious mask of Liang et al. [37 ] , which zeroes coordinates whose Adam step is not descent-aligned with the current gradient and rescales the rest.
RMSE is reported after removing the additive constant. Mean alignment , used for ADP, removes the mean difference between the learned and reference surfaces on the scored region. Flat-surface baselines, where reported, use the same region and gauge. ATP dimensional-consistency comparisons use complete retained planes, squared-Pearson correlations and mean-aligned RMSEs after physical Boltzmann marginalization.
Baseline settings
The ADP and ATP comparisons in § 3.2 and § 3.3 use the Hamiltonians and CVs specified above and a 2 fs timestep. The no-exchange control retains the ladder’s architecture and population sizes. In ADP and ATP, auxiliary moves swap whole populations and were accepted in at most 0.5% of epochs over 10 ns. Most accepted exchanges transport physical configurations. ADP VESPER single-replica, OPES and TAMD runs share initial configurations. Means and sample standard deviations use the trajectories that reach the reported horizon.
OPES uses bias factor 100 and barrier parameter 40 kJ mol− 1 ^{-1} − 1 [29 ] . Metadynamics deposits 1.2 kJ mol− 1 ^{-1} − 1 hills every 500 steps [36 ] , and TAMD uses UFEDMM [5 ] . Readouts are the OPES kernel density, the negative metadynamics hill sum, and the TAMD auxiliary-variable histogram at its own temperature. CFF [51 ] and its SIREN variant (S-CFF) [63 ] use their saved native free-energy functions. ADP errors use the full-grid convention for methods with finite estimates and the native finite-estimator-cell convention for OPES and TAMD.
Neural VES. We test this method using the released code [9 ] . The bias factor is 100, with 50 grid bins per periodic CV and ReLU widths 48–24–12 for ADP. Adam starts at 10 − 3 10^{-3} 1 0 − 3 . Parameter and target updates occur every 500 MD steps with the released system-specific learning-rate controllers. Errors use the native free-energy exports.
Compute
Table 2 lists the simulated time per replica at the readout and the resulting aggregate MD per run. Several runs continued beyond the readout, and the table counts MD up to it. The ADP baselines are OPES, metadynamics, TAMD, neural VES, CFF and S-CFF. VESPER, CFF and S-CFF ran on NVIDIA H100, H200, L40S and RTX PRO 6000 GPUs, and the other baselines on CPUs, so wall-clock times are not comparable across methods. For scale, one chignolin run takes about 100 h on one H100 GPU, which runs all twelve replicas, and one methane run takes about 140–180 GPU-hours, mostly on L40S GPUs.
Reported molecular results
Table 3 collects the values quoted in § 3.2 to § 3.5 , with the readout behind each.
Controlled landscapes
Müller–Brown protocol
Figure 2 A–C. The ground-truth energy is 0.1 times the standard Müller–Brown potential at β = 1 \beta= 1 β = 1 on the physical box [ − 1.5 , 1 ] × [ − 0.5 , 2 ] [-1.5, 1] \times[-0.5, 2] [ − 1.5 , 1 ] × [ − 0.5 , 2 ] , with the identity CV map. Seven arms share every setting except the tempering and, for the ladders, exchange: single levels at α = 1 , 0.5 , 0.25 , 0.1 , 0.05 \alpha= 1, 0.5, 0.25, 0.1, 0.05 α = 1 , 0.5 , 0.25 , 0.1 , 0.05 with 1,000 physical and 1,000 auxiliary walkers, and a five-level ladder (α = 1 , 0.5 , 0.25 , 0.1 , 0.05 \alpha= 1, 0.5, 0.25, 0.1, 0.05 α = 1 , 0.5 , 0.25 , 0.1 , 0.05 ; 200 walkers per level and population; normalized level weights 0.2) with and without exchange of physical and auxiliary walkers (every 100 steps, 200 attempted swaps). All physical walkers start within ± 0.025 \pm0.025 ± 0.025 of the deepest minimum, near ( − 0.558 , 1.442 ) (-0.558, 1.442) ( − 0.558 , 1.442 ) , and the auxiliary walkers start uniformly over the box. Every arm of a run receives the same initial arrays and initial model. The network has width 32 and is trained with Adam at 3 × 10 − 4 3 \times10^{-4} 3 × 1 0 − 4 . The dynamics use reduced timesteps Δ t x = Δ t s = 0.002 \Delta t_x = \Delta t_s = 0.002 Δ t x = Δ t s = 0.002 , unit frictions, force clips of 100 and 500, and reflecting boundaries. 40,000 steps give t = 80 t = 80 t = 80 . Curves show the causal trailing-half parameter average: at each plotted time T T T the raw parameters saved in [ T / 2 , T ] [T/2, T] [ T /2 , T ] are averaged, so no point uses a future state.
Errors are evaluated on a 241 × 241 241 \times241 241 × 241 grid over [ − 1.5 , 1.5 ] × [ − 0.5 , 2.5 ] [-1.5, 1.5] \times[-0.5, 2.5] [ − 1.5 , 1.5 ] × [ − 0.5 , 2.5 ] , which extends beyond the simulation box so that the high-energy rim is scored. The RMSE removes the additive constant and weights every cell equally. Because the rim energies are large it is dominated by the barrier and rim regions, which is why its scale is 237 237 237 –252 k B T 252\,k_{\mathrm{B}}T 252 k B T and why the most strongly tempered level is best on it. D K L ( ρ ∥ ρ θ ) D_{\mathrm{KL}}(\rho\mathbin{\|} \rho_\theta) D KL ( ρ ∥ ρ θ ) normalizes ρ ∝ e − F \rho\propto e^{-F} ρ ∝ e − F and ρ θ ∝ e − G θ \rho_\theta\propto e^{-G_\theta} ρ θ ∝ e − G θ on the same grid and is dominated by the basins. At t = 80 t = 80 t = 80 (mean ± \pm ± sample SD over three independent runs) the RMSE is 237.7 ± \pm ± 2.5 (α = 0.05 \alpha= 0.05 α = 0.05 ), 238.8 ± \pm ± 1.4 (L-NRE), 239.0 ± \pm ± 1.1 (α = 0.1 \alpha= 0.1 α = 0.1 ), 240.0 ± \pm ± 1.2 (L-RE), 243.1 ± \pm ± 2.0 (α = 1 \alpha= 1 α = 1 ), 243.7 ± \pm ± 0.4 (α = 0.25 \alpha= 0.25 α = 0.25 ) and 243.9 ± \pm ± 0.5 (α = 0.5 \alpha= 0.5 α = 0.5 ). The divergence is 0.010 ± \pm ± 0.008 (α = 0.5 \alpha= 0.5 α = 0.5 ), 0.010 ± \pm ± 0.007 (α = 0.25 \alpha= 0.25 α = 0.25 ), 0.017 ± \pm ± 0.013 (L-RE), 0.019 ± \pm ± 0.013 (L-NRE), 0.049 ± \pm ± 0.034 (α = 0.1 \alpha= 0.1 α = 0.1 ), 0.102 ± \pm ± 0.138 (α = 0.05 \alpha= 0.05 α = 0.05 ) and 0.82 ± \pm ± 1.14 (α = 1 \alpha= 1 α = 1 ). The physical walkers of the α = 1 \alpha= 1 α = 1 arm remain concentrated in the starting basin (355 of the 360 overlaid positions from t = 40 t = 40 t = 40 –80 80 80 lie nearest the deepest minimum), so its surface is unconstrained outside it. The maps in Figure 2 C are means of the final trailing-half averages after aligning the additive constants, each shifted to its own minimum. The walker overlays pool 40 physical and 40 auxiliary positions per run from each of t = 40 t = 40 t = 40 , 60 and 80.
Tempering and population sweeps. Separate sweeps at t = 320 t = 320 t = 320 (160,000 steps) use four independent runs, uniform starts and a network of width 32. The tempering sweep is α ∈ { 0.05 , 0.1 , 0.25 , 0.5 , 1 } \alpha\in\{0.05, 0.1, 0.25, 0.5, 1\} α ∈ { 0.05 , 0.1 , 0.25 , 0.5 , 1 } at 1,000 walkers per population. The population sweep changes the physical and auxiliary walker counts together through 1, 10, 100, 1000, 10000 at α = 0.25 \alpha= 0.25 α = 0.25 and fixed integration and update counts, so a larger population also means proportionally more sampling work. Errors here are Boltzmann-weighted, mean–gauge FES RMSEs of trailing parameter averages, evaluated on fixed reference samples drawn from a 1200 × 1200 1200 \times1200 1200 × 1200 inverse-CDF discretization of the Müller–Brown density (Table 4 ). Any α ∈ [ 0.05 , 0.5 ] \alpha\in[0.05, 0.5] α ∈ [ 0.05 , 0.5 ] reaches 0.09–0.11 k B T k_{\mathrm{B}}T k B T against 1.28 k B T k_{\mathrm{B}}T k B T for a flat surface, while α = 1 \alpha= 1 α = 1 remains at 0.60 k B T k_{\mathrm{B}}T k B T . The error falls from 1.95 k B T k_{\mathrm{B}}T k B T with one walker to 0.070 k B T k_{\mathrm{B}}T k B T with 10 4 10^4 1 0 4 .
Table 4. Müller–Brown errors at t = 320 t = 320 t = 320 : Boltzmann-weighted FES RMSE in k B T k_{\mathrm{B}}T k B T of the parameter average, mean ± \pm ± sample SD over four independent runs, against the tempering exponent at 1000 walkers per population and against the walker count at α = 0.25 \alpha= 0.25 α = 0.25 . A flat surface scores 1.281.
Four-well dimensional scaling
Landscape. The reference landscape has four symmetric Gaussian components whose centers lie in a fixed two-dimensional plane involving all d d d coordinates. Up to an additive constant, its thermal-unit form is
G ⋆ ( s ) k B T = − log ∑ a , b ∈ { − 1 , 1 } exp [ − ∥ s − δ ( a v 1 + b v 2 ) ∥ 2 2 σ w 2 ] , δ = 1.6 , σ w = 0.3. \frac{G^{\star}(s)}{k_{\mathrm{B}}T} = -\log\sum_{a,b\in\{-1,1\}} \exp\left[-\frac{\left\lVert s-\delta(av_1+bv_2)\right\rVert^2}{2\sigma_w^2}\right], \qquad\delta=1.6,\qquad\sigma_w=0.3. k B T G ⋆ ( s ) = − log a , b ∈ { − 1 , 1 } ∑ exp [ − 2 σ w 2 ∥ s − δ ( a v 1 + b v 2 ) ∥ 2 ] , δ = 1.6 , σ w = 0.3.
Here v 1 = ( 1 , … , 1 ) / d v_1=(1,\ldots,1)/\sqrt{d} v 1 = ( 1 , … , 1 ) / d , and v 2 v_2 v 2 is obtained by orthogonalizing the alternating-sign vector against v 1 v_1 v 1 and normalizing it. The remaining directions are harmonic, with dimensionless curvature 1 / σ w 2 1/\sigma_w^2 1/ σ w 2 . The adjacent-well barriers are approximately 13.5 k B T 13.5\,k_{\mathrm{B}}T 13.5 k B T . The CV map is the identity, and the dynamics use reflecting boundaries on [ − 3 , 3 ] d [-3,3]^d [ − 3 , 3 ] d . The two-dimensional core is identical at every d d d . The reference contains 40,000 i.i.d. draws from the analytic teacher Boltzmann distribution, and errors use all d d d coordinates.
Protocol. d ∈ { 2 , 5 , 10 , 25 , 50 , 100 } d\in\{2,5,10,25,50,100\} d ∈ { 2 , 5 , 10 , 25 , 50 , 100 } . The same three-hidden-layer network of width 1024 is used at every d d d . A separate perfect-information regression of the exact free energy into this network (i.i.d. Boltzmann samples, exact energies, 160,000 Adam updates of batch 1,000) establishes that it can represent the four-well plane at d = 100 d=100 d = 100 (widths 128, 256 and 512 do not learn the d = 100 d=100 d = 100 plane, while width 1024 reaches a plane error of 0.03–0.13 k B T k_{\mathrm{B}}T k B T against 1.29 for a flat surface at every d d d ). The primary arms use 1,000 physical and 1,000 auxiliary walkers in total, both started in the deepest well, 160,000 overdamped steps at Δ t = 0.005 \Delta t=0.005 Δ t = 0.005 (reduced time 800), Adam at 3 × 10 − 5 3\times10^{-5} 3 × 1 0 − 5 , a bias-corrected parameter EMA with time constant 2,000 steps for the convergence readout and a trailing-half parameter average for the endpoint. Arms: single levels at α = 1 \alpha=1 α = 1 , 0.5, 0.25, 0.1, 0.05, and four-level ladders (α = 0.5 \alpha=0.5 α = 0.5 , 0.25, 0.1, 0.05; 250 walkers per level and population) with and without symmetric exchange of physical and auxiliary walkers. As a control on population rather than temperature, the same ladders were also run with 1,000 walkers per level and population. Three independent runs were specified per arm. The unbiased d = 2 d=2 d = 2 arm uses its two completed runs, and all other arms use three.
Readouts. NRMSE = = = RMSE/σ \sigma σ , where the RMSE is the teacher-Boltzmann-weighted full-coordinate error after removing the additive constant and σ \sigma σ is the corresponding standard deviation of the reference free energy. The endpoint NRMSE uses the trailing-half SWA readout, while threshold time uses the bias-corrected EMA readout. The reduced threshold time is when the EMA NRMSE first stays below 0.25 to the end of the run. D K L ( ρ ∥ ρ θ ) D_{\mathrm{KL}}(\rho\|\rho_{\theta}) D KL ( ρ ∥ ρ θ ) uses the same teacher-Boltzmann reference samples. Endpoint summaries are means ± \pm ± sample SD (Table 5 and Table 6 ). For threshold-time summaries, a run that never reaches the sustained threshold is capped at the budget of 800.
Table 5. Four-well dimensional scaling: mean ± \pm ± sample SD of the full-coordinate NRMSE and D K L ( ρ ∥ ρ θ ) D_{\mathrm{KL}}(\rho\parallel\rho_\theta) D KL ( ρ ∥ ρ θ ) at the final SWA readout. Each best single level is selected by the lowest mean of the corresponding metric among the tested levels at that dimension. All primary arms use the same total walker budget.
Table 6. Larger-population controls for the four-well landscape: mean ± \pm ± sample SD with 1,000 walkers per level and population, four times the primary ladder budget.
Observations. The same exchange ladder is within a factor of 1.62 of the best tested single level on mean NRMSE across all dimensions. Its mean NRMSE ranges from 0.19 to 0.42, with lower mean error than the best single level at d = 50 d = 50 d = 50 and 100. At d = 100 d = 100 d = 100 , the primary exchange and no-exchange ladders reach 0.32 ± 0.21 0.32 \pm0.21 0.32 ± 0.21 and 0.40 ± 0.19 0.40 \pm0.19 0.40 ± 0.19 , respectively, while the best single level reaches 0.42 ± 0.05 0.42 \pm0.05 0.42 ± 0.05 . The larger-population controls reach 0.39 ± 0.20 0.39 \pm0.20 0.39 ± 0.20 with exchange and 0.33 ± 0.11 0.33 \pm0.11 0.33 ± 0.11 without (Table 6 ). These descriptive comparisons support one fixed ladder across the tested dimensions. They do not establish uniform superiority or an exchange advantage at every dimension.
Connections to VES, PS-CD, and Rényi objectives
The following identities connect the VESPER objective to well-tempered VES, pseudo-spherical contrastive divergence, and a Rényi variational principle.
Well-tempered VES and an explicit shared-energy objective
Let p p p be the physical CV density and t t t the VES target. The fixed-target functional of Valsson & Parrinello [58 ] is
Ω [ V ; t ] = β − 1 log E p e − β V + ∫ t ( s ) V ( s ) d s . \Omega[V;t] = \beta^{-1}\log\mathbb{E}_{p}e^{-\beta V} + \int t(s)V(s)\,\mathrm{d}s. Ω [ V ; t ] = β − 1 log E p e − β V + ∫ t ( s ) V ( s ) d s .
Differentiating with t t t held fixed gives
∇ θ Ω [ V θ ; t ] = − E b i a s e d ∇ θ V θ + E t ∇ θ V θ . \nabla_\theta\Omega[V_\theta;t] = -\mathbb{E}_{\mathrm{biased}}\nabla_\theta V_\theta+ \mathbb{E}_{t}\nabla_\theta V_\theta. ∇ θ Ω [ V θ ; t ] = − E biased ∇ θ V θ + E t ∇ θ V θ .
Set V θ = − ( 1 − α ) G θ V_\theta= -(1-\alpha)G_\theta V θ = − ( 1 − α ) G θ and t θ = P G θ t_\theta= P G_\theta t θ = P G θ , with 0 < α < 1 0 < \alpha< 1 0 < α < 1 . The biased physical law is then ν G θ \nu_{G_\theta} ν G θ , and substitution yields
∇ θ Ω [ V θ ; t ] ∣ t = t θ = ( 1 − α ) { E ν G θ ∇ θ G θ ∘ s − E P G θ ∇ θ G θ } = ( 1 − α ) ∇ θ L ? β [ G θ ] . \left.\nabla_\theta\Omega[V_\theta;t]\right|_{t=t_\theta} = (1-\alpha)\left\{\mathbb{E}_{\nu_{G_\theta}}\nabla_\theta G_\theta\circ s - \mathbb{E}_{P G_\theta}\nabla_\theta G_\theta\right\} = (1-\alpha)\nabla_\theta\mathcal{L}^{\beta}_{\mathrm{?}}[G_\theta]. ∇ θ Ω [ V θ ; t ] ∣ t = t θ = ( 1 − α ) { E ν G θ ∇ θ G θ ∘ s − E P G θ ∇ θ G θ } = ( 1 − α ) ∇ θ L ? β [ G θ ] .
This proves equation (48) for every admissible model for which differentiation under the integrals is valid. The factor 1 − α 1-\alpha 1 − α can be absorbed into the step size for ordinary gradient descent with the same parameterization and sample estimate.
VESPER thus gives the instantaneous self-consistent update an explicit scalar objective, convex in the free energy. This point requires holding the VES target fixed during differentiation. Taking the total derivative of Ω [ V θ ; t θ ] \Omega[V_\theta;t_\theta] Ω [ V θ ; t θ ] also differentiates the target and adds
∫ V θ ( s ) ∇ θ t θ ( s ) d s = α β ( 1 − α ) Cov t θ ( G θ , ∇ θ G θ ) , \int V_\theta(s)\nabla_\theta t_\theta(s)\,\mathrm{d}s=\alpha\beta(1-\alpha)\operatorname{Cov}_{t_\theta}(G_\theta,\nabla_\theta G_\theta), ∫ V θ ( s ) ∇ θ t θ ( s ) d s = α β ( 1 − α ) Cov t θ ( G θ , ∇ θ G θ ) ,
where we used ∇ θ t θ = − α β t θ ( ∇ θ G θ − E t θ ∇ θ G θ ) \nabla_\theta t_\theta=-\alpha\beta t_\theta(\nabla_\theta G_\theta-\mathbb{E}_{t_\theta}\nabla_\theta G_\theta) ∇ θ t θ = − α β t θ ( ∇ θ G θ − E t θ ∇ θ G θ ) . In contrast, differentiating L ^ β \widehat{\mathcal{L}}_\beta L β directly gives the desired two-population update, with its minimum and KL control established in Propositions S1 and S2.
The distinction also appears in how the target evolves. Well-tempered VES sets t ∝ p α t\propto p^\alpha t ∝ p α [59 ] . In neural VES, the free-energy relation F ^ = − V − β − 1 log t \widehat{F}=-V-\beta^{-1}\log t F = − V − β − 1 log t and the previous target t n − 1 ∝ e − α β F ^ n − 1 t_{n-1}\propto e^{-\alpha\beta\widehat{F}_{n-1}} t n − 1 ∝ e − α β F n − 1 give the recursive update [9 ]
F ^ n = − V n + α F ^ n − 1 + C n , t n ∝ e − α β F ^ n . (49) \widehat{F}_n=-V_n+\alpha\widehat{F}_{n-1}+C_n,\qquad t_n\propto e^{-\alpha\beta\widehat{F}_n}. \tag*{(49)} F n = − V n + α F n − 1 + C n , t n ∝ e − α β F n . (49)
At a fixed point, ( 1 − α ) F ^ = − V + C (1-\alpha)\widehat{F}=-V+C ( 1 − α ) F = − V + C , so V = − ( 1 − α ) G V=-(1-\alpha)G V = − ( 1 − α ) G gives F ^ = G + c o n s t \widehat{F}=G+\mathrm{const} F = G + const . During learning, VESPER instead defines the target directly from the current G θ G_\theta G θ and represents it with persistent auxiliary walkers. One model therefore determines both the molecular bias and the auxiliary potential, without a separate recursion for the free-energy estimate. The walker laws track these changing targets through the coupled dynamics analyzed in Section A.6.
This formulation extends directly to the combined objective: every L ^ β k \widehat{\mathcal{L}}_{\beta_k} L β k has the same physical minimizer, so all levels train one G θ G_\theta G θ through their native gradients (Proposition 1 ). In VES notation, the interior-level update is reproduced by weighting the corresponding fixed-target gradients by c k / ( 1 − α k ) c_k/(1-\alpha_k) c k / ( 1 − α k ) . The rescaled gradient has the finite physical-endpoint limit
lim α ↑ 1 1 1 − α ∇ θ Ω [ − ( 1 − α ) G θ ; t ] ∣ t = P G θ = E p ∇ θ G θ − E P G θ , β ∇ θ G θ , \lim_{\alpha\uparrow1}\frac{1}{1-\alpha}\nabla_\theta\Omega[-(1-\alpha)G_\theta;t]\big|_{t=P_{G_\theta}}=\mathbb{E}_p\nabla_\theta G_\theta-\mathbb{E}_{P_{G_\theta,\beta}}\nabla_\theta G_\theta, α ↑ 1 lim 1 − α 1 ∇ θ Ω [ − ( 1 − α ) G θ ; t ] t = P G θ = E p ∇ θ G θ − E P G θ , β ∇ θ G θ ,
whenever the two expectations converge. VESPER includes this endpoint through equation (8) .
PS-CD and direct sampling of the two tilted populations
With g = β G g=\beta G g = β G , the dimensionless objective ℓ α \ell_\alpha ℓ α is given by equation (42) . Let q g = e − g / Z g q_g=e^{-g}/Z_g q g = e − g / Z g be the model density relative to a reference measure r r r , with Z g < ∞ Z_g<\infty Z g < ∞ . Substituting log q g = − g − log Z g \log q_g=-g-\log Z_g log q g = − g − log Z g and γ = α − 1 \gamma=\alpha-1 γ = α − 1 gives
ℓ α [ g ] = 1 1 + γ log ∫ q g 1 + γ d r − 1 γ log ∫ q g γ d p = L γ P S [ g ] . (50) \ell_\alpha[g]=\frac{1}{1+\gamma}\log\int q_g^{1+\gamma}\,\mathrm{d}r-\frac{1}{\gamma}\log\int q_g^\gamma\,\mathrm{d}p=\mathcal{L}^{\mathrm{PS}}_\gamma[g]. \tag*{(50)} ℓ α [ g ] = 1 + γ 1 log ∫ q g 1 + γ d r − γ 1 log ∫ q g γ d p = L γ PS [ g ] . (50)
The first term contributes − log Z g -\log Z_g − log Z g and the second + log Z g +\log Z_g + log Z g , so the normalizer cancels. This is the PS-CD objective [60 ] , including its negative-γ \gamma γ extension, whenever both integrals are finite. For VESPER, 0 < α < 1 0<\alpha<1 0 < α < 1 gives γ = α − 1 ∈ ( − 1 , 0 ) \gamma=\alpha-1\in(-1,0) γ = α − 1 ∈ ( − 1 , 0 ) . The physical free-energy minimizer follows from Proposition S1.
For the Lebesgue reference, write q θ = q G θ = e − β G θ / Z G θ q_\theta=q_{G_\theta}=e^{-\beta G_\theta}/Z_{G_\theta} q θ = q G θ = e − β G θ / Z G θ . The two populations in the gradient are exactly the laws used in the main text:
P G θ ( s ) = q θ ( s ) α ∫ q θ ( u ) α d u , P_{G_\theta}(s)=\frac{q_\theta(s)^\alpha}{\int q_\theta(u)^\alpha\,\mathrm{d}u}, P G θ ( s ) = ∫ q θ ( u ) α d u q θ ( s ) α ,
Q G θ ( s ) = p ( s ) q θ ( s ) α − 1 ∫ p ( u ) q θ ( u ) α − 1 d u . (51) Q_{G_\theta}(s)=\frac{p(s)q_\theta(s)^{\alpha-1}}{\int p(u)q_\theta(u)^{\alpha-1}\,\mathrm{d}u}. \tag*{(51)} Q G θ ( s ) = ∫ p ( u ) q θ ( u ) α − 1 d u p ( s ) q θ ( s ) α − 1 . (51)
Differentiating the two log integrals gives ∇ θ ℓ α [ g θ ] = E Q G θ ∇ θ g θ − E P G θ ∇ θ g θ \nabla_\theta\ell_\alpha[g_\theta] = \mathbb{E}_{Q_{G_\theta}}\nabla_\theta g_\theta- \mathbb{E}_{P_{G_\theta}}\nabla_\theta g_\theta ∇ θ ℓ α [ g θ ] = E Q G θ ∇ θ g θ − E P G θ ∇ θ g θ , recovering equation (7) after division by β \beta β . At the physical endpoint, P G θ = q θ P_{G_\theta} = q_\theta P G θ = q θ and Q G θ = p Q_{G_\theta} = p Q G θ = p , recovering the maximum-likelihood gradient.
The computational difference is how these expectations are obtained. PS-CD uses a fixed dataset for the positive term and reweights samples from q θ q_\theta q θ to estimate the tempered model expectation [60 ] . VESPER directly evolves auxiliary walkers in α G θ \alpha G_\theta α G θ , removing that temperature-change importance weighting from the fresh-sample gradient. In molecular applications, it also generates Q G θ Q_{G_\theta} Q G θ online through the biased microscopic law ν G θ \nu_{G_\theta} ν G θ . This makes the same variational principle usable when equilibrium configurations are not available in advance: sampling supplies new configurations while their gradients update the free-energy model (Section A.3).
Rényi variational identity and its sampling realization
For a probability reference r r r and 0 < α < 1 0 < \alpha< 1 0 < α < 1 , set h = − g h = -g h = − g in equation (42) . Then
− ℓ α [ g ] = 1 α − 1 log ∫ e ( α − 1 ) h d p − 1 α log ∫ e α h d r -\ell_\alpha[g] = \frac{1}{\alpha- 1} \log\int e^{(\alpha-1)h}\,\mathrm{d}p - \frac{1}{\alpha} \log\int e^{\alpha h}\,\mathrm{d}r − ℓ α [ g ] = α − 1 1 log ∫ e ( α − 1 ) h d p − α 1 log ∫ e α h d r
which is the variational expression of Birrell et al. [8 Theorem 3.1] . With f = d p / d r > 0 f = \mathrm{d}p/\mathrm{d}r > 0 f = d p / d r > 0 r r r -almost everywhere and M ( α ) = ∫ f α d r M(\alpha) = \int f^\alpha\,\mathrm{d}r M ( α ) = ∫ f α d r , Proposition S8 gives g ⋆ = − log f + C g_\star= -\log f + C g ⋆ = − log f + C . Substituting this minimizer into the two integrals yields
inf g ℓ α [ g ] = log M ( α ) α ( 1 − α ) = − 1 α D α ( p ∥ r ) , D α ( p ∥ r ) = log M ( α ) α − 1 . \inf_g \ell_\alpha[g] = \frac{\log M(\alpha)}{\alpha(1-\alpha)} = -\frac{1}{\alpha}D_\alpha(p\mathbin{\|}r), \qquad D_\alpha(p\mathbin{\|}r) = \frac{\log M(\alpha)}{\alpha- 1}. g inf ℓ α [ g ] = α ( 1 − α ) log M ( α ) = − α 1 D α ( p ∥ r ) , D α ( p ∥ r ) = α − 1 log M ( α ) .
The factor 1 / α 1/\alpha 1/ α accounts for the divergence convention in that reference. If f f f vanishes on a set of positive r r r -measure, the infimum need not be attained by a finite free energy.
The value of this identity for VESPER is that its optimizer is also the quantity needed to drive sampling. For r = d s r = \mathrm{d}s r = d s , g ⋆ = β G ⋆ g_\star= \beta G_\star g ⋆ = β G ⋆ up to a constant. VESPER uses G θ G_\theta G θ to bias microscopic dynamics and to evolve the auxiliary walkers, obtaining both gradient expectations without evaluating the unknown CV density p p p or first collecting equilibrium data (Section A.3). The KL decomposition in Proposition S2 connects the objective excess to the discrepancy of P G P_G P G and Q G Q_G Q G from their common tempered target, and Proposition 1 lets several orders train the same physical free energy.
For a reference of finite mass R R R , normalization to r / R r/R r / R subtracts ( log R ) / α (\log R)/\alpha ( log R ) / α from ℓ α \ell_\alpha ℓ α and leaves its gradients unchanged. On infinite-volume CV domains we instead use the exponential integrals directly, with the integrability conditions of Sections A.1 and A.7. This preserves the same free-energy minimizer without introducing an artificial uniform probability reference.
Algorithm and implementation details
VESPER is implemented in PySAGES [62 ] . We alternate molecular and auxiliary sampling with parameter updates, retaining the auxiliary population between epochs. Neural models provide both parameter derivatives for learning and spatial derivatives for biasing. When several parameter updates reuse one batch, within-population importance weights account for the change from its generating model. The pseudocode and weight formulas are given below. Model architectures, integration parameters, optimizer settings, exchange intervals, and readouts are specified for each experiment in Section B.
Algorithm 1 describes physical sampling with an energy-unit model G θ G_\theta G θ and level ratios α k = β ^ k / β \alpha_k = \hat{\beta}_k/\beta α k = β ^ k / β . Each epoch generates physical and auxiliary samples, optionally exchanges configurations (predominantly physical configurations in the reported ADP and ATP runs, Section B), and updates the shared model.
Algorithm 1 Physical VESPER ladder
Input: potential U U U , CV map s s s , inverse temperature β \beta β ; levels 0 ≤ β ^ 1 < ⋯ < β ^ K ≤ β 0 \le\hat{\beta}_1 < \cdots< \hat{\beta}_K \le\beta 0 ≤ β ^ 1 < ⋯ < β ^ K ≤ β ; weights c k > 0 c_k > 0 c k > 0 , ∑ k c k = 1 \sum_k c_k = 1 ∑ k c k = 1 ; model G θ G_\theta G θ ; sampler, optimizer, exchange, and readout settings.
Initialize θ \theta θ and the physical and auxiliary populations on every level.
repeat
Set θ 0 ← θ \theta_0 \leftarrow\theta θ 0 ← θ . Advance physical walkers in U − ( 1 − α k ) G θ 0 ∘ s U - (1-\alpha_k)G_{\theta_0}\circ s U − ( 1 − α k ) G θ 0 ∘ s and auxiliary walkers in α k G θ 0 \alpha_kG_{\theta_0} α k G θ 0 , at inverse temperature β \beta β .
If exchange is enabled, attempt adjacent-level swaps using equation (15) .
Save batches { x k j } j = 1 n k \{x_k^j\}_{j=1}^{n_k} { x k j } j = 1 n k and { s k i } i = 1 N k \{s_k^i\}_{i=1}^{N_k} { s k i } i = 1 N k , together with their generating model G θ 0 G_{\theta_0} G θ 0 .
for each parameter update allowed by the protocol do
for k = 1 , … , K k=1,\ldots,K k = 1 , … , K do
if sample reweighting is enabled then
Δ G ← G θ − G θ 0 \Delta G \leftarrow G_\theta-G_{\theta_0} Δ G ← G θ − G θ 0 .
w j ( x , k ) ← e ( β − β ^ k ) Δ G ( s ( x k j ) ) w_j^{(x,k)} \leftarrow e^{(\beta-\hat{\beta}_k)\Delta G(s(x_k^j))} w j ( x , k ) ← e ( β − β ^ k ) Δ G ( s ( x k j )) , w i ( s , k ) ← e − β ^ k Δ G ( s k i ) w_i^{(s,k)} \leftarrow e^{-\hat{\beta}_k\Delta G(s_k^i)} w i ( s , k ) ← e − β ^ k Δ G ( s k i ) .
Normalize the weights separately in each pool.
else
w ˉ j ( x , k ) ← 1 / n k \bar{w}_j^{(x,k)} \leftarrow1/n_k w ˉ j ( x , k ) ← 1/ n k , w ˉ i ( s , k ) ← 1 / N k \bar{w}_i^{(s,k)} \leftarrow1/N_k w ˉ i ( s , k ) ← 1/ N k .
end if
h k ← ∑ j w ˉ j ( x , k ) ∇ θ G θ ( s ( x k j ) ) − ∑ i w ˉ i ( s , k ) ∇ θ G θ ( s k i ) h_k \leftarrow\sum_j \bar{w}_j^{(x,k)}\nabla_\theta G_\theta(s(x_k^j))-\sum_i \bar{w}_i^{(s,k)}\nabla_\theta G_\theta(s_k^i) h k ← ∑ j w ˉ j ( x , k ) ∇ θ G θ ( s ( x k j )) − ∑ i w ˉ i ( s , k ) ∇ θ G θ ( s k i ) .
end for
If an enabled sample-reuse threshold is exceeded, end the inner loop and generate fresh samples.
Update θ \theta θ with the configured optimizer using ∑ k c k h k \sum_k c_kh_k ∑ k c k h k .
end for
Save model checkpoints and sampling diagnostics.
until the stopping condition or simulation budget is reached
Output: the checkpoint or averaged readout selected by the protocol.
Sample reuse. The weights of equation (47) supply the two density ratios for a batch generated at G θ 0 G_{\theta_0} G θ 0 . Reweighted estimators also appear in [10 ] and [29 ] . The molecular implementation can monitor the 95th percentile and maximum of ∣ log w ∣ |\log w| ∣ log w ∣ , with soft and hard thresholds 5 and 8, and the normalized ESS ( ∑ w ) 2 / [ n ∑ w 2 ] (\sum w)^2/[n\sum w^2] ( ∑ w ) 2 / [ n ∑ w 2 ] , with floor 0.2. Activation, step reduction, and refresh rules are specified per system in Section B.7.
Existing-data updates. For equilibrium observations x j ∗ ∼ ϱ x_j^* \sim\varrho x j ∗ ∼ ϱ , replace physical sampling and its reuse weights by equation (45) .
Checkpoint and averaged readouts. A trailing weight-space average produces a network with parameters
θ ˉ = 1 ∣ W ∣ ∑ k ∈ W θ k . (52) \bar{\theta}=\frac{1}{|W|}\sum_{k\in W}\theta_k. \tag*{(52)} θ ˉ = ∣ W ∣ 1 k ∈ W ∑ θ k . (52)
The checkpoint set W W W , stride, and window are specified per experiment [31 , 47 ] . Fixed-checkpoint and last-iterate readouts use their corresponding θ k \theta_k θ k .