Introduction
We consider the problem of sampling from a Boltzmann distribution,
π ( x ) = e − U ( x ) Z , Z = ∫ e − U ( x ) d x , \pi(x) = \frac{e^{-U(x)}}{Z}, \qquad Z = \int e^{-U(x)}\,dx, π ( x ) = Z e − U ( x ) , Z = ∫ e − U ( x ) d x ,
where U ( x ) U(x) U ( x ) is an energy function and Z Z Z is the normalizing constant. We assume no samples from π \pi π are available and have only black-box access to U ( x ) U(x) U ( x ) and its gradient ∇ U ( x ) \nabla U(x) ∇ U ( x ) . Many systems in physics, chemistry, and biology are naturally described by Boltzmann distributions. For these systems, samples from π \pi π are useful because they allow us to estimate expectations E π [ f ( X ) ] \mathbb{E}_{\pi}[f(X)] E π [ f ( X )] . For appropriate choices of f f f , these expectations answer important scientific questions such as how likely a protein is to fold [26 , 21 ] , how likely a drug is to bind to a particular protein [5 ] , or, for a material, what its physical properties are and which of its phases is most stable [7 , 14 ] .
Classical approaches to Boltzmann sampling include Markov chain Monte Carlo (MCMC; [24 ] ) and molecular dynamics (MD; [7 ] ). These methods start from an initial state x 0 x_0 x 0 and repeatedly update it with small, local changes computed from U U U or ∇ U \nabla U ∇ U , producing a sequence of states x 0 → x 1 → x 2 → ⋯ x_0 \rightarrow x_1 \rightarrow x_2 \rightarrow\cdots x 0 → x 1 → x 2 → ⋯ . Langevin dynamics is a canonical example of such a classical approach [19 ] . It updates x t x_t x t with a small step along − ∇ U ( x t ) -\nabla U(x_t) − ∇ U ( x t ) , toward more likely states, plus Gaussian noise. Under suitable conditions, the distribution of x t x_t x t approaches π \pi π as t → ∞ t \rightarrow\infty t → ∞ .
In practice, however, MCMC methods like Langevin dynamics take a long time to converge to π \pi π . The reason is that moving from one likely region to another may require passing through unlikely states, and such transitions are rare [30 ] . Moreover, evaluating U U U or ∇ U \nabla U ∇ U at every step can be expensive. In computational chemistry, for example, evaluating the energy of a particular state requires an expensive quantum-mechanical calculation, such as one based on density functional theory [18 , 27 ] .
These costs motivate amortized samplers : generative models trained to approximate the Boltzmann distribution. Once trained, these models can produce independent samples cheaply, without evaluating the energy or its gradient. Although training requires such evaluations, their cost is spread, or amortized, across the samples the model later produces. By contrast, obtaining representative samples from π \pi π with MCMC or MD can be computationally expensive, as both may require long simulations involving repeated energy or gradient evaluations.
In this work, we introduce WAsserstein SAmpling via Bridge Interpolants (WASABI) , an amortized sampler for Boltzmann distributions. WASABI is trained by minimizing the Kullback–Leibler (KL) divergence K L ( q ∥ π ) \mathrm{KL}(q\Vert\pi) KL ( q ∥ π ) between its sampling distribution q q q and the target π \pi π . The idea is to perform gradient descent on this divergence directly in the space of distributions, a procedure known as the Wasserstein gradient flow (WGF; [15 ] ). To implement this flow, WASABI represents q q q with a diffusion model, a powerful class of generative models that have been successfully applied to images, video, and protein structures [11 , 29 , 12 , 32 ] . WASABI repeatedly updates the diffusion model so that its endpoint distribution q q q takes small steps along the Wasserstein gradient flow toward π \pi π .
Figure 1 illustrates the idea. WASABI trains a diffusion model to follow Wasserstein gradient flow toward π \pi π . At each step n n n , WASABI seeks to translate a Wasserstein descent step of its current distribution q n q_n q n into an update of the diffusion model, yielding a new distribution q n + 1 q_{n+1} q n + 1 . Directly implementing this update would require the gradient of the model’s log density (the model score), which is generally unavailable for diffusion samplers. We show how to perform this update using only samples drawn from q n q_n q n and target energy gradients ∇ U \nabla U ∇ U , without evaluating the model score. Repeating this procedure yields a diffusion model trained to approximate π \pi π , from which samples can be generated without further energy or gradient evaluations.
Figure 1. Overview of WASABI . The endpoint distributions evolve from an initialization q 0 q_0 q 0 , distinct from the shared diffusion prior p 0 p_0 p 0 . Top: we optimize a sequence of sampling distributions q n q_n q n by evolve them toward the target π \pi π through Wasserstein gradient descent of the KL divergence. Bottom: the sampling distribution q n q_n q n is parameterized by a diffusion model with a prior p 0 p_0 p 0 for all n n n ; WASABI translates Wasserstein descent into updates of the diffusion model by updating its denoising representation m ˉ t ( n ) \bar{m}_t^{(n)} m ˉ t ( n ) , gradually moving it toward the target across iterations n n n . The final panel shows the ideal diffusion sampler, with denoising representation m ˉ t ∗ \bar{m}_t^* m ˉ t ∗ associated with π \pi π .
We summarize our contributions as follows:
gradient flow of the KL divergence to π \pi π [15 ] . Discretizing this flow gives an update that WASABI applies repeatedly to its distribution; in its exact form, the update has π \pi π as its fixed point.
A tractable training algorithm. We derive a tractable algorithm for applying Wasserstein descent-induced updates to the diffusion model that avoids evaluating the diffusion model score. Each step updates the diffusion model through regression, for which we construct unbiased, empirically low-variance estimators of the training targets.
Accurate molecular sampling with fewer energy evaluations. On alanine dipeptide, WASABI reproduces the dominant conformational basins and achieves lower Jensen–Shannon divergence than the best reported baselines [4 ] , using an estimated 30% of their energy/force evaluations.
Background
Diffusion models and bridge interpolants. Diffusion models [11 , 29 ] and related developments [22 , 3 , 10 ] provide tools for generative modeling and sampling through continuous-time dynamics. To sample a target distribution p 1 p_1 p 1 , we simulate the following SDE driven by a function family m t : R d → R d m_t : \mathbb{R}^d \to\mathbb{R}^d m t : R d → R d :
d X t = m t ( X t ) − X t 1 − t d t + B d W t , X 0 ∼ p 0 , 0 ≤ t < 1 , (1) \mathrm{d}X_t = \frac{m_t(X_t)-X_t}{1-t}\,\mathrm{d}t + B\,\mathrm{d}W_t, \qquad X_0 \sim p_0,\qquad0 \le t < 1, \tag*{(1)} d X t = 1 − t m t ( X t ) − X t d t + B d W t , X 0 ∼ p 0 , 0 ≤ t < 1 , (1)
where p 0 p_0 p 0 is the source distribution which is easy to sample from, B > 0 B > 0 B > 0 is the diffusion coefficient, and W t W_t W t is standard Brownian motion. The choice of m t m_t m t that generates the same endpoint distribution p 1 p_1 p 1 need not be unique. We consider a canonical representation of p 1 p_1 p 1 induced by a bridge interpolant :
Y t = ( 1 − t ) X ~ 0 + t X ~ 1 + h t Z , 0 ≤ t ≤ 1 , (2) Y_t = (1-t)\widetilde{X}_0 + t\widetilde{X}_1 + h_t Z,\qquad0 \le t \le1, \tag*{(2)} Y t = ( 1 − t ) X 0 + t X 1 + h t Z , 0 ≤ t ≤ 1 , (2)
where X ~ 0 ∼ p 0 \widetilde{X}_0 \sim p_0 X 0 ∼ p 0 , X ~ 1 ∼ p 1 \widetilde{X}_1 \sim p_1 X 1 ∼ p 1 are source and endpoint samples and Z ∼ N ( 0 , I d ) Z \sim\mathcal{N}(0,I_d) Z ∼ N ( 0 , I d ) is the noise, all drawn independently; h t = B t ( 1 − t ) h_t = B\sqrt{t(1-t)} h t = B t ( 1 − t ) is the bridge noise standard deviation. The associated bridge conditional mean defines the denoising representation below.
Definition 2.1 (Denoising representation of p 1 p_1 p 1 ). Under the bridge setup in Eq. (2) , we call the family
m ‾ t ( y ) = E [ X ~ 1 ∣ Y t = y ] , 0 ≤ t < 1 , \overline{m}_t(y) = \mathbb{E}[\widetilde{X}_1 \mid Y_t = y],\qquad0 \le t < 1, m t ( y ) = E [ X 1 ∣ Y t = y ] , 0 ≤ t < 1 , the denoising representation of p 1 p_1 p 1 relative to p 0 p_0 p 0 and B B B .
Under suitable regularity conditions, choosing m t = m ‾ t m_t = \overline{m}_t m t = m t in Eq. (1) reproduces the bridge marginals (X t = d Y t X_t \overset{d}{=} Y_t X t = d Y t ) and yields endpoint distribution p 1 p_1 p 1 as t ↑ 1 t \uparrow1 t ↑ 1 .
When endpoint samples X ~ 1 ∼ p 1 \widetilde{X}_1 \sim p_1 X 1 ∼ p 1 are available, a neural denoiser m ω ( y , t ) m_\omega(y,t) m ω ( y , t ) is trained using the denoising regression objective, whose population minimizer is m ‾ t ( y ) \overline{m}_t(y) m t ( y ) :
min ω E t , X ~ 0 , X ~ 1 , Z [ ∥ m ω ( Y t , t ) − X ~ 1 ∥ 2 ] . (3) \min_\omega\mathbb{E}_{t,\widetilde{X}_0,\widetilde{X}_1,Z}\left[\left\|m_\omega(Y_t,t)-\widetilde{X}_1\right\|^2\right]. \tag*{(3)} ω min E t , X 0 , X 1 , Z [ m ω ( Y t , t ) − X 1 2 ] . (3)
Wasserstein gradient flows. Let P 2 ( R d ) \mathcal{P}_2(\mathbb{R}^d) P 2 ( R d ) denote the probability measures on R d \mathbb{R}^d R d with finite second moment. In Euclidean space, the gradient describes how a function changes when its input is displaced. Here, we consider a functional F : P 2 ( R d ) → R \mathcal{F}:\mathcal{P}_2(\mathbb{R}^d)\to\mathbb{R} F : P 2 ( R d ) → R that assigns a scalar objective value to a probability distribution, e.g. a divergence to some target distribution. Its gradient likewise describes how the objective value F \mathcal{F} F changes when that distribution is displaced. We displace a distribution q q q in the direction of a vector field v v v by moving each point x x x to x + ϵ v ( x ) x+\epsilon v(x) x + ϵ v ( x ) , where ϵ > 0 \epsilon>0 ϵ > 0 is a small step size, which gives q ϵ ν = ( id + ϵ v ) # q q^\nu_\epsilon=(\operatorname{id}+\epsilon v)_\#q q ϵ ν = ( id + ϵ v ) # q , where id \operatorname{id} id is the identity map and # \# # denotes the pushforward [28 ] .3 This notion of displacement allows us to define the Wasserstein gradient of F \mathcal{F} F at q q q , denoted ∇ W 2 F [ q ] \nabla_{W_2}\mathcal{F}[q] ∇ W 2 F [ q ] , as the vector field satisfying
d d ϵ F ( q ϵ ν ) ∣ ϵ = 0 = E X ∼ q [ ∇ W 2 F [ q ] ( X ) ⋅ v ( X ) ] (4) \left.\frac{\mathrm{d}}{\mathrm{d}\epsilon}\mathcal{F}(q^\nu_\epsilon)\right|_{\epsilon=0}
=
\mathbb{E}_{X\sim q}\left[\nabla_{W_2}\mathcal{F}[q](X)\cdot v(X)\right] \tag*{(4)} d ϵ d F ( q ϵ ν ) ϵ = 0 = E X ∼ q [ ∇ W 2 F [ q ] ( X ) ⋅ v ( X ) ] (4)
for every sufficiently regular velocity field v v v [15 ] . This identity expresses the directional derivative of F \mathcal{F} F as an L 2 ( q ) L^2(q) L 2 ( q ) inner product; consequently, the negative Wasserstein gradient gives the direction of steepest descent among velocity fields v v v with the same L 2 ( q ) L^2(q) L 2 ( q ) -norm E q ∥ v ∥ 2 \mathbb{E}_q\|v\|^2 E q ∥ v ∥ 2 .
We say that a sufficiently regular curve ( q τ ) τ ≥ 0 (q_\tau)_{\tau\ge0} ( q τ ) τ ≥ 0 follows 2-Wasserstein gradient flow if, to first order,
q τ + ϵ = ( id − ϵ ∇ W 2 F [ q τ ] ) # q τ + o ( ϵ ) , (5) q_{\tau+\epsilon} = \left(\operatorname{id} - \epsilon\nabla W_2 \mathcal{F}[q_\tau]\right)_{\#}q_\tau+ o(\epsilon), \tag*{(5)} q τ + ϵ = ( id − ϵ ∇ W 2 F [ q τ ] ) # q τ + o ( ϵ ) , (5)
where the expansion is understood weakly, that is, after integration against smooth test functions. Equivalently, when q τ q_\tau q τ admits a density, its evolution satisfies the continuity equation
∂ τ q τ = ∇ ⋅ ( q τ ∇ W 2 F [ q τ ] ) . (6) \partial_\tau q_\tau= \nabla\cdot\left(q_\tau\nabla W_2 \mathcal{F}[q_\tau]\right). \tag*{(6)} ∂ τ q τ = ∇ ⋅ ( q τ ∇ W 2 F [ q τ ] ) . (6)
For sampling the Boltzmann target π ( x ) ∝ e − U ( x ) \pi(x) \propto e^{-U(x)} π ( x ) ∝ e − U ( x ) , consider the reverse KL objective F [ q ] = KL ( q ∥ π ) \mathcal{F}[q] = \operatorname{KL}(q \mathbin\Vert\pi) F [ q ] = KL ( q ∥ π ) . Its Wasserstein gradient is
∇ W 2 F [ q ] ( x ) = ∇ log q ( x ) − ∇ log π ( x ) = ∇ log q ( x ) + ∇ U ( x ) . (7) \nabla W_2 \mathcal{F}[q](x) = \nabla\log q(x) - \nabla\log\pi(x) = \nabla\log q(x) + \nabla U(x). \tag*{(7)} ∇ W 2 F [ q ] ( x ) = ∇ log q ( x ) − ∇ log π ( x ) = ∇ log q ( x ) + ∇ U ( x ) . (7)
Substituting this gradient into Eq. (6) yields the Fokker–Planck equation associated with the following overdamped Langevin dynamics,
d Z τ = − ∇ U ( Z τ ) d τ + 2 d W τ , (8) dZ_\tau= -\nabla U(Z_\tau)\,d\tau+ \sqrt{2}\,dW_\tau, \tag*{(8)} d Z τ = − ∇ U ( Z τ ) d τ + 2 d W τ , (8)
where W τ W_\tau W τ is standard Brownian motion and π \pi π is the stationary density. Thus, the distribution of the Langevin process follows the 2-Wasserstein gradient flow of the reverse KL [15 ] .
Stein’s identity. Let r r r be a smooth positive density with score s r = ∇ log r s_r = \nabla\log r s r = ∇ log r . For any sufficiently regular scalar test function ξ : R d → R \xi: \mathbb{R}^d \to\mathbb{R} ξ : R d → R , Stein’s identity gives
E X ∼ r [ ξ ( X ) s r ( X ) ] = − E X ∼ r [ ∇ ξ ( X ) ] , (9) \mathbb{E}_{X \sim r}[\xi(X)s_r(X)] = -\mathbb{E}_{X \sim r}[\nabla\xi(X)], \tag*{(9)} E X ∼ r [ ξ ( X ) s r ( X )] = − E X ∼ r [ ∇ ξ ( X )] , (9)
provided the expectations are finite and the integration-by-parts boundary term vanishes. This identity transfers a derivative of the density to the test function, allowing score-weighted expectations to be expressed without evaluating the score.
Method
Our goal is to learn a diffusion sampler whose endpoint distribution q q q approximates the Boltzmann target π ∝ e − U \pi\propto e^{-U} π ∝ e − U by minimizing F [ q ] = KL ( q ∥ π ) \mathcal{F}[q] = \operatorname{KL}(q \mathbin\Vert\pi) F [ q ] = KL ( q ∥ π ) . The key idea is to parameterize the sampling distribution through a diffusion model’s denoising representation and translate the prescribed distributional evolution into an update of this representation. This yields a fixed-point algorithm with regression updates using generated samples from the current model and target energy gradients.
In Sec. 3.1, we describe the model setup and derive the denoiser update induced by Wasserstein descent. In Sec. 3.2, we construct estimators of the mean and covariance contributions to the denoiser update that avoid evaluating the unknown score of the model’s distribution. Finally, Sec. 3.3 combines these ingredients into the WASABI training algorithm.
Model Setup and Denoiser Updates
We consider the denoising representation m ˉ t \bar{m}_t m ˉ t of q q q given a source p 0 p_0 p 0 and a diffusion coefficient B B B (Def. 2.1). WASABI progressively optimizes KL ( q ∥ π ) \operatorname{KL}(q \mathbin\Vert\pi) KL ( q ∥ π ) by translating Wasserstein descent steps of the endpoint distribution q q q into updates of the denoiser m ˉ t \bar{m}_t m ˉ t . We index these update iterations by n = 1 , 2 , … n = 1, 2, \ldots n = 1 , 2 , … , writing m ˉ t ( n ) \bar{m}_t^{(n)} m ˉ t ( n ) for the current denoiser and q n q_n q n for the endpoint distribution.
At iteration n n n , the negative Wasserstein gradient (Eq. (7) ) specifies the descent direction for q n q_n q n , with
∇ W 2 F [ q n ] = ∇ log q n − ∇ log π = ∇ log q n + ∇ U (10) \nabla W_2 \mathcal{F}[q_n] = \nabla\log q_n - \nabla\log\pi= \nabla\log q_n + \nabla U \tag*{(10)} ∇ W 2 F [ q n ] = ∇ log q n − ∇ log π = ∇ log q n + ∇ U (10)
For a step size η > 0 \eta> 0 η > 0 , the desired endpoint distribution is updated as
q n , η = ( id − η ∇ W 2 F [ q n ] ) # q n . (11) q_{n,\eta} = \left(\operatorname{id} - \eta\nabla W_2 \mathcal{F}[q_n]\right)_{\#}q_n. \tag*{(11)} q n , η = ( id − η ∇ W 2 F [ q n ] ) # q n . (11)
For the same source p 0 p_0 p 0 and diffusion coefficient B B B , the transported distribution q n , η q_{n,\eta} q n , η induces a new denoising representation m ˉ t , η ( n ) \bar{m}_{t,\eta}^{(n)} m ˉ t , η ( n ) . In the following theorem, we characterize how the denoiser changes under a small endpoint transport step by computing its derivative with respect to η \eta η at η = 0 \eta= 0 η = 0 .
Theorem 3.1 (Denoiser change induced by Wasserstein descent). Let q n , η = ( id − η ∇ W 2 F [ q n ] ) # q n q_{n,\eta} = (\operatorname{id} - \eta\nabla_{W_2}\mathcal{F}[q_n])_{\#}q_n q n , η = ( id − η ∇ W 2 F [ q n ] ) # q n , and let ( m ˉ t , η ( n ) ) t (\bar{m}^{(n)}_{t,\eta})_t ( m ˉ t , η ( n ) ) t be its denoising representation, using the same initial distribution p 0 p_0 p 0 and noise scale B B B , so that m ˉ t , 0 ( n ) = m ˉ t ( n ) \bar{m}^{(n)}_{t,0} = \bar{m}^{(n)}_t m ˉ t , 0 ( n ) = m ˉ t ( n ) . Let h t = B t ( 1 − t ) h_t = B\sqrt{t(1-t)} h t = B t ( 1 − t ) and Y t = ( 1 − t ) X ~ 0 + t X ~ 1 + h t Z Y_t = (1-t)\widetilde{X}_0 + t\widetilde{X}_1 + h_t Z Y t = ( 1 − t ) X 0 + t X 1 + h t Z where X ~ 0 ∼ p 0 \widetilde{X}_0 \sim p_0 X 0 ∼ p 0 , X ~ 1 ∼ q n \widetilde{X}_1 \sim q_n X 1 ∼ q n , and Z ∼ N ( 0 , I ) Z \sim\mathcal{N}(0,I) Z ∼ N ( 0 , I ) are independent. Assume the conditional moments below are finite and differentiation under the conditional integrals is valid. Then, for 0 < t < 1 0 < t < 1 0 < t < 1 ,
∂ ∂ η m ˉ t , η ( n ) ( y ) ∣ η = 0 = − E [ ∇ W 2 F [ q n ] ( X ~ 1 ) ∣ Y t = y ] + t h t Cov ( X ~ 1 , − Z ⋅ ∇ W 2 F [ q n ] ( X ~ 1 ) ∣ Y t = y ) . (12) \left.\frac{\partial}{\partial\eta}\bar{m}^{(n)}_{t,\eta}(y)\right|_{\eta=0}
= -\mathbb{E}\left[\nabla_{W_2}\mathcal{F}[q_n](\widetilde{X}_1)\mid Y_t=y\right]
+ \frac{t}{h_t}\operatorname{Cov}\left(\widetilde{X}_1,-Z\cdot\nabla_{W_2}\mathcal{F}[q_n](\widetilde{X}_1)\mid Y_t=y\right). \tag*{(12)} ∂ η ∂ m ˉ t , η ( n ) ( y ) η = 0 = − E [ ∇ W 2 F [ q n ] ( X 1 ) ∣ Y t = y ] + h t t Cov ( X 1 , − Z ⋅ ∇ W 2 F [ q n ] ( X 1 ) ∣ Y t = y ) . (12) Moreover, if q n = π q_n = \pi q n = π , then ∇ W 2 F [ π ] = 0 \nabla_{W_2}\mathcal{F}[\pi] = 0 ∇ W 2 F [ π ] = 0 , and hence M t [ π ] = C t [ π ] = 0 M_t[\pi] = C_t[\pi] = 0 M t [ π ] = C t [ π ] = 0 .
Transporting the endpoints changes the denoiser in two ways. The mean term M t [ q n ] ( y ) M_t[q_n](y) M t [ q n ] ( y ) captures the average movement while keeping each endpoint’s original contribution (under q n q_n q n -induced process) to the average fixed. But moving an endpoint also changes how likely the bridge is to produce the same observation Y t = y Y_t = y Y t = y . Endpoints that become more likely given this observation contribute more to the conditional average, and those that become less likely contribute less. The covariance term C t [ q n ] ( y ) C_t[q_n](y) C t [ q n ] ( y ) accounts for how these changes in the endpoints’ relative contributions shift the conditional average. Together, they give the full first-order change of the denoiser.
Thm. 3.1 motivates the WASABI update operator for the denoiser. With a time-dependent step size η t > 0 \eta_t > 0 η t > 0 and a weight λ t \lambda_t λ t controlling the covariance contribution, we define
T [ m ˉ ( n ) ] t ( y ) : = m ˉ t ( n ) ( y ) + η t ( M t [ q n ] ( y ) + λ t C t [ q n ] ( y ) ) . (13) \mathcal{T}[\bar{m}^{(n)}]_t(y) := \bar{m}^{(n)}_t(y) + \eta_t\left(M_t[q_n](y) + \lambda_t C_t[q_n](y)\right). \tag*{(13)} T [ m ˉ ( n ) ] t ( y ) := m ˉ t ( n ) ( y ) + η t ( M t [ q n ] ( y ) + λ t C t [ q n ] ( y ) ) . (13)
With λ t = 1 \lambda_t = 1 λ t = 1 and a common step size η t = η \eta_t = \eta η t = η , this update agrees, to first order, with the denoiser induced by the desired transport in Eq. (11) . Allowing η t \eta_t η t and λ t \lambda_t λ t to vary with bridge time t t t modifies this direction and its step size. Since both contributions vanish at q n = π q_n = \pi q n = π , the update leaves the denoiser of π \pi π unchanged for any finite λ t \lambda_t λ t ; hence, π \pi π is a fixed point of the ideal WASABI update.
Estimating the denoiser update
The WASABI update direction is now specified (Eq. (13) ), but its mean and covariance contributions (Eq. (12) ) involve the unknown endpoint score ∇ log q n \nabla\log q_n ∇ log q n through the Wasserstein gradient (Eq. (10) ). We derive score-free estimators for both contributions, assuming the bridge’s source and endpoint are drawn independently. Apps. A.3 and A.5 extend these estimators to general couplings.
Mean contribution. The first component of the denoiser update in Eq. (13) can be written as
M t [ q n ] ( y ) = E [ − ∇ U ( X ~ 1 ) − ∇ log q n ( X ~ 1 ) ∣ Y t = y ] . (14) M_t[q_n](y) = \mathbb{E}\left[-\nabla U(\widetilde{X}_1)-\nabla\log q_n(\widetilde{X}_1)\mid Y_t=y\right]. \tag*{(14)} M t [ q n ] ( y ) = E [ − ∇ U ( X 1 ) − ∇ log q n ( X 1 ) ∣ Y t = y ] . (14)
where we substitute the Wasserstein gradient from Eq. (10) .
The target score − ∇ U -\nabla U − ∇ U is available through energy gradients. Using Stein’s identity (Eq. (9) ), we obtain the following score-free expression (App. A.3 gives the derivation):
M t [ q n ] ( y ) = E [ − ∇ U ( X ~ 1 ) + t h t Z ∣ Y t = y ] . (15) M_t[q_n](y) = \mathbb{E}\left[-\nabla U(\widetilde{X}_1) + \frac{t}{h_t}Z\mid Y_t=y\right]. \tag*{(15)} M t [ q n ] ( y ) = E [ − ∇ U ( X 1 ) + h t t Z ∣ Y t = y ] . (15)
This expression provides a computable regression target: draw independent X ~ 0 ∼ p 0 \widetilde{X}_0 \sim p_0 X 0 ∼ p 0 , X ~ 1 ∼ q n \widetilde{X}_1 \sim q_n X 1 ∼ q n , and Z ∼ N ( 0 , I d ) Z \sim\mathcal{N}(0,I_d) Z ∼ N ( 0 , I d ) , construct Y t = ( 1 − t ) X ~ 0 + t X ~ 1 + h t Z Y_t = (1-t)\widetilde{X}_0 + t\widetilde{X}_1 + h_tZ Y t = ( 1 − t ) X 0 + t X 1 + h t Z , and compute − ∇ U ( X ~ 1 ) + ( t / h t ) Z -\nabla U(\widetilde{X}_1) + (t/h_t)Z − ∇ U ( X 1 ) + ( t / h t ) Z . Regressing this target on Y t Y_t Y t with squared-error loss its conditional mean, M t [ q n ] ( Y t ) M_t[q_n](Y_t) M t [ q n ] ( Y t ) , without evaluating the unknown endpoint score ∇ log q n ( X ~ 1 ) \nabla\log q_n(\widetilde{X}_1) ∇ log q n ( X 1 ) .
Covariance contribution. To learn the covariance contribution C t [ q n ] ( y ) C_t[q_n](y) C t [ q n ] ( y ) in Eq. (12) by regression, we need not evaluate it pointwise. In Sec. 3.3 we show that expanding the regression loss requires only the expected inner product between C t [ q n ] ( y ) C_t[q_n](y) C t [ q n ] ( y ) and a learned correction network f θ f_\theta f θ . To estimate this inner product, here we derive an estimator for a general smooth vector test field φ : R d → R d \varphi:\mathbb{R}^d \to\mathbb{R}^d φ : R d → R d ; in § 3.3 we take φ = f θ ( ⋅ , t ) \varphi=f_\theta(\cdot,t) φ = f θ ( ⋅ , t ) . Write s π = − ∇ U s_\pi=-\nabla U s π = − ∇ U and define F φ ( y , x ) : = φ ( y ) ⋅ ( x − m ˉ t ( n ) ( y ) ) F_\varphi(y,x):=\varphi(y)\cdot(x-\bar{m}^{(n)}_t(y)) F φ ( y , x ) := φ ( y ) ⋅ ( x − m ˉ t ( n ) ( y )) , the scalar projection of the centered endpoint onto φ ( y ) \varphi(y) φ ( y ) . Stein’s identity yields the following score-free expression:
E [ φ ( Y t ) ⋅ C t [ q n ] ( Y t ) ] = t h t E [ φ ( Y t ) ⋅ Z + F φ ( Y t , X ~ 1 ) ( Z ⋅ s π ( X ~ 1 ) + t h t ( ∥ Z ∥ 2 − d ) ) ] . (16) \mathbb{E}[\varphi(Y_t)\cdot C_t[q_n](Y_t)] = \frac{t}{h_t}\mathbb{E}\left[\varphi(Y_t)\cdot Z+F_\varphi(Y_t,\widetilde{X}_1)\left(Z\cdot s_\pi(\widetilde{X}_1)+\frac{t}{h_t}(\lVert Z\rVert^2-d)\right)\right]. \tag*{(16)} E [ φ ( Y t ) ⋅ C t [ q n ] ( Y t )] = h t t E [ φ ( Y t ) ⋅ Z + F φ ( Y t , X 1 ) ( Z ⋅ s π ( X 1 ) + h t t (∥ Z ∥ 2 − d ) ) ] . (16)
The derivation is provided in App. A.4.
Although this expression removes the unknown score, direct Monte Carlo estimates had high variance in our early experiments. We seek an alternative representation of the same expectation by changing how bridge noise is sampled. Completing the square, with a t ( x ) : = h t 2 t s π ( x ) a_t(x):=\frac{h_t}{2t}s_\pi(x) a t ( x ) := 2 t h t s π ( x ) , gives
Z ⋅ s π ( x ) + t h t ( ∥ Z ∥ 2 − d ) = t h t ( ∥ Z + a t ( x ) ∥ 2 − d − ∥ a t ( x ) ∥ 2 ) . Z\cdot s_\pi(x)+\frac{t}{h_t}(\lVert Z\rVert^2-d)=\frac{t}{h_t}\left(\lVert Z+a_t(x)\rVert^2-d-\lVert a_t(x)\rVert^2\right). Z ⋅ s π ( x ) + h t t (∥ Z ∥ 2 − d ) = h t t ( ∥ Z + a t ( x ) ∥ 2 − d − ∥ a t ( x ) ∥ 2 ) .
Completing the square isolates the nonnegative factor ∥ Z + a t ( x ) ∥ 2 \lVert Z+a_t(x)\rVert^2 ∥ Z + a t ( x ) ∥ 2 . We use this factor to reweight the standard Gaussian noise distribution: noise values with larger factors are sampled more often. Normalizing these weights gives the tilted distribution in the following theorem.
Theorem 3.2 (Tilted-noise covariance estimator). Fix 0 < t < 1 0<t<1 0 < t < 1 and independent bridge endpoints X ~ 0 ∼ p 0 \widetilde{X}_0\sim p_0 X 0 ∼ p 0 , X ~ 1 ∼ q n \widetilde{X}_1\sim q_n X 1 ∼ q n . Let ϕ \phi ϕ be the standard Gaussian density on R d \mathbb{R}^d R d , and let φ : R d → R d \varphi:\mathbb{R}^d\to\mathbb{R}^d φ : R d → R d be a smooth test field. Write a t ( x ) = h t s π ( x ) / ( 2 t ) a_t(x)=h_ts_\pi(x)/(2t) a t ( x ) = h t s π ( x ) / ( 2 t ) and F φ ( y , x ) = φ ( y ) ⋅ ( x − m ˉ t ( n ) ( y ) ) F_\varphi(y,x)=\varphi(y)\cdot(x-\bar{m}^{(n)}_t(y)) F φ ( y , x ) = φ ( y ) ⋅ ( x − m ˉ t ( n ) ( y )) . Then
ϕ ~ t ( z ∣ x ) = ∥ z + a t ( x ) ∥ 2 d + ∥ a t ( x ) ∥ 2 ϕ ( z ) (17) \widetilde{\phi}_t(z\mid x)=\frac{\lVert z+a_t(x)\rVert^2}{d+\lVert a_t(x)\rVert^2}\phi(z) \tag*{(17)} ϕ t ( z ∣ x ) = d + ∥ a t ( x ) ∥ 2 ∥ z + a t ( x ) ∥ 2 ϕ ( z ) (17) is a probability density. Conditional on ( X ~ 0 , X ~ 1 ) (\widetilde{X}_0,\widetilde{X}_1) ( X 0 , X 1 ) , let Z Z Z and Z ~ \widetilde{Z} Z have respective distributions ϕ \phi ϕ and ϕ ~ t ( ⋅ ∣ X ~ 1 ) \widetilde{\phi}_t(\cdot\mid\widetilde{X}_1) ϕ t ( ⋅ ∣ X 1 ) , with any coupling. Set Y ~ t = ( 1 − t ) X ~ 0 + t X ~ 1 + h t Z ~ \widetilde{Y}_t=(1-t)\widetilde{X}_0+t\widetilde{X}_1+h_t\widetilde{Z} Y t = ( 1 − t ) X 0 + t X 1 + h t Z . The estimator
C ^ t [ φ ] = t h t φ ( Y t ) ⋅ Z + t 2 h t 2 ( d + ∥ a t ( X ~ 1 ) ∥ 2 ) [ F φ ( Y ~ t , X ~ 1 ) − F φ ( Y t , X ~ 1 ) ] (18) \widehat{C}_t[\varphi]=\frac{t}{h_t}\varphi(Y_t)\cdot Z+\frac{t^2}{h_t^2}\left(d+\lVert a_t(\widetilde{X}_1)\rVert^2\right)\left[F_\varphi(\widetilde{Y}_t,\widetilde{X}_1)-F_\varphi(Y_t,\widetilde{X}_1)\right] \tag*{(18)} C t [ φ ] = h t t φ ( Y t ) ⋅ Z + h t 2 t 2 ( d + ∥ a t ( X 1 ) ∥ 2 ) [ F φ ( Y t , X 1 ) − F φ ( Y t , X 1 ) ] (18) is an unbiased estimator of E [ φ ( Y t ) ⋅ C t [ q n ] ( Y t ) ] \mathbb{E}[\varphi(Y_t)\cdot C_t[q_n](Y_t)] E [ φ ( Y t ) ⋅ C t [ q n ] ( Y t )] , provided the relevant expectations are finite, the integration-by-parts boundary terms vanish, and m ˉ t ( n ) ( y ) = E [ X ~ 1 ∣ Y t = y ] \bar{m}^{(n)}_t(y)=\mathbb{E}[\widetilde{X}_1\mid Y_t=y] m ˉ t ( n ) ( y ) = E [ X 1 ∣ Y t = y ] .
The proof is given in App. A.4; Thm. A.8 provides a procedure for sampling the tilted noise.
The tilted distribution (Eq. (17) ) replaces weighting ordinary Gaussian samples by sampling noise values in proportion to their weights and multiplying by the known normalization constant d + ∥ a t ( x ) ∥ 2 d+\lVert a_t(x)\rVert^2 d + ∥ a t ( x ) ∥ 2 . The estimator compares evaluations under the original and tilted noise distributions. The two noise draws may be coupled so that the estimator uses paired evaluations at Y t Y_t Y t and Y ~ t \widetilde{Y}_t Y t . Its expectation is unchanged by the coupling, but its variance can depend on that choice.
The WASABI Algorithm
We now describe the WASABI algorithm, which realizes Wasserstein descent through learned denoiser updates and canonicalizes the updated sampler by refitting its denoising representation.
At a high level, WASABI starts from an endpoint sample set D 0 \mathcal{D}_0 D 0 drawn from an initial distribution q 0 q_0 q 0 ; in practice, it can be obtained from an uninformative Gaussian or a high temperature MD simulation. We use D 0 \mathcal{D}_0 D 0 to fit an initial denoiser m ω 0 m_{\omega_0} m ω 0 via Eq. (3) . At iteration n n n , we learn the direction of the WASABI denoiser update in Eq. (13) by training a correction network f θ f_\theta f θ while keeping the current denoiser m ω n m_{\omega_n} m ω n fixed. Applying the learned correction gives the new distribution q n + 1 q_{n+1} q n + 1 by sampling from the diffusion SDE (Eq. (1) ) with m t ( y ) = m ~ t ( n + 1 ) ( y ) : = m ω n ( y , t ) + η t f θ ( y , t ) m_t(y)=\widetilde{m}^{(n+1)}_t(y):=m_{\omega_n}(y,t)+\eta_t f_\theta(y,t) m t ( y ) = m t ( n + 1 ) ( y ) := m ω n ( y , t ) + η t f θ ( y , t ) . This corrected function m ~ ( n + 1 ) \widetilde{m}^{(n+1)} m ( n + 1 ) need not be the denoising representation of q n + 1 q_{n+1} q n + 1 . We therefore canonicalize it by refitting the denoiser m ω n + 1 m_{\omega_{n+1}} m ω n + 1 to new samples from q n + 1 q_{n+1} q n + 1 , preparing it for the next iteration. After N N N outer iterations, we return the diffusion sampler driven by m ω N m_{\omega_N} m ω N .
Alg. 1 summarizes the procedure. We derive the training objectives and practical update rules below.
Algorithm 1 WASABI
Inputs: Energy U U U ; source p 0 p_0 p 0 ; diffusion coefficient B B B ; initial endpoint sample set D 0 D_0 D 0 ; iterations N N N ; schedules η t \eta_t η t , λ t \lambda_t λ t .
Fit initial denoiser: fit m ω 0 m_{\omega_0} m ω 0 to D 0 D_0 D 0 using Eq. (3) .
for n = 0 , … , N − 1 n = 0, \ldots, N - 1 n = 0 , … , N − 1 do // Wasserstein-induced update
Fit correction: freeze m ω n m_{\omega_n} m ω n and minimize L n ( θ ) \mathcal{L}_n(\theta) L n ( θ ) in Eq. (21) to obtain θ n \theta_n θ n .
Update sampler: m ~ t ( n + 1 ) ( y ) ← m ω n ( y , t ) + η t f θ n ( y , t ) \widetilde{m}_t^{(n+1)}(y) \leftarrow m_{\omega_n}(y,t) + \eta_t f_{\theta_n}(y,t) m t ( n + 1 ) ( y ) ← m ω n ( y , t ) + η t f θ n ( y , t ) . // Canonicalization
Refresh sample set: generate D n + 1 D_{n+1} D n + 1 by simulating Eq. (1) with m ~ ( n + 1 ) \widetilde{m}^{(n+1)} m ( n + 1 ) .
Fit denoiser: ω n + 1 ≈ arg min ω E [ ∥ m ω ( Y t , t ) − X ~ 1 ∥ 2 ] \omega_{n+1} \approx\arg\min_\omega\mathbb{E}[\lVert m_\omega(Y_t,t) - \widetilde{X}_1\rVert^2] ω n + 1 ≈ arg min ω E [∥ m ω ( Y t , t ) − X 1 ∥ 2 ] , using X ~ 1 ∼ D n + 1 \widetilde{X}_1 \sim D_{n+1} X 1 ∼ D n + 1 .
end for
return Diffusion sampler driven by m ω N m_{\omega_N} m ω N .
Learning the Wasserstein-induced update. Below we state the population objectives for the ideal representation m ˉ ( n ) \bar{m}^{(n)} m ˉ ( n ) ; implementation uses its fitted approximation m ω n m_{\omega_n} m ω n .
To approximate the first-order denoiser change in Thm. 3.1 , we train the correction network to fit the update direction prescribed by Eq. (13) :
r t ( n ) ( y ) : = η t − 1 ( T [ m ˉ ( n ) ] t ( y ) − m ˉ t ( n ) ( y ) ) = M t [ q n ] ( y ) + λ t C t [ q n ] ( y ) . (19) r_t^{(n)}(y) := \eta_t^{-1}\left(\mathcal{T}[\bar{m}^{(n)}]_t(y) - \bar{m}_t^{(n)}(y)\right) = \mathcal{M}_t[q_n](y) + \lambda_t\mathcal{C}_t[q_n](y). \tag*{(19)} r t ( n ) ( y ) := η t − 1 ( T [ m ˉ ( n ) ] t ( y ) − m ˉ t ( n ) ( y ) ) = M t [ q n ] ( y ) + λ t C t [ q n ] ( y ) . (19)
Fitting this direction amounts to the ideal regression problem
min θ 1 2 E [ ∥ f θ ( Y t , t ) − r t ( n ) ( Y t ) ∥ 2 ] . (20) \min_\theta\frac{1}{2}\mathbb{E}\left[\left\lVert f_\theta(Y_t,t) - r_t^{(n)}(Y_t)\right\rVert^2\right]. \tag*{(20)} θ min 2 1 E [ f θ ( Y t , t ) − r t ( n ) ( Y t ) 2 ] . (20)
Here the expectation is over the sampled bridge time t t t and independent X ~ 0 ∼ p 0 \widetilde{X}_0 \sim p_0 X 0 ∼ p 0 , X ~ 1 ∼ q n \widetilde{X}_1 \sim q_n X 1 ∼ q n , and Z ∼ N ( 0 , I d ) Z \sim\mathcal{N}(0,I_d) Z ∼ N ( 0 , I d ) , with Y t Y_t Y t constructed as in the background.
We do not need to evaluate r t ( n ) ( Y t ) r_t^{(n)}(Y_t) r t ( n ) ( Y t ) itself to train the correction. Expanding the squared-error objective in Eq. (20) and dropping the term independent of θ \theta θ , we replace its mean and covariance inner-product terms using the estimators in Eqs. (15) and (18) . This gives the following computable objective with the same population minimizers (App. § 9 .§ 6 provides the derivation):
L n ( θ ) = E [ 1 2 ∥ f θ ( Y t , t ) ∥ 2 − f θ ( Y t , t ) ⋅ ( − ∇ U ( X ~ 1 ) + t h t Z ) − λ t C ^ t [ f θ ( ⋅ , t ) ] ] . (21) \mathcal{L}_n(\theta) = \mathbb{E}\left[\frac{1}{2}\lVert f_\theta(Y_t,t)\rVert^2 - f_\theta(Y_t,t)\cdot\left(-\nabla U(\widetilde{X}_1) + \frac{t}{h_t}Z\right) - \lambda_t\widehat{\mathcal{C}}_t[f_\theta(\cdot,t)]\right]. \tag*{(21)} L n ( θ ) = E [ 2 1 ∥ f θ ( Y t , t ) ∥ 2 − f θ ( Y t , t ) ⋅ ( − ∇ U ( X 1 ) + h t t Z ) − λ t C t [ f θ ( ⋅ , t )] ] . (21)
This expectation additionally includes the tilted noise Z ~ \widetilde{Z} Z , whose conditional distribution given the source and endpoint samples is specified in Thm. 3.2 .
We minimize this objective with minibatch estimates, keeping m ω n m_{\omega_n} m ω n fixed inside the covariance estimator. With the fitted parameters θ n \theta_n θ n , we form m ~ t ( n + 1 ) ( y ) = m ω n ( y , t ) + η t f θ n ( y , t ) \widetilde{m}_t^{(n+1)}(y) = m_{\omega_n}(y,t) + \eta_t f_{\theta_n}(y,t) m t ( n + 1 ) ( y ) = m ω n ( y , t ) + η t f θ n ( y , t ) .
Refitting the denoising representation. With m ~ ( n + 1 ) \widetilde{m}^{(n+1)} m ( n + 1 ) , the corrected sampler generates endpoint distribution q n + 1 q_{n+1} q n + 1 via Eq. (1) , but m ~ ( n + 1 ) \widetilde{m}^{(n+1)} m ( n + 1 ) needs not equal the denoising representation of q n + 1 q_{n+1} q n + 1 , which is required for the next update. We therefore canonicalize the corrected sampler by refitting the denoiser. To do so, we simulate samples D n + 1 D_{n+1} D n + 1 from q n + 1 q_{n+1} q n + 1 , then fit m ω n + 1 m_{\omega_{n+1}} m ω n + 1 using the denoising regression objective in Eq. (3) .
Replay and annealing. In practice, we maintain the endpoint sample set as a replay buffer, retaining samples across iterations and partially refreshing it with samples from the updated sampler. Batches from this buffer are used to fit both the denoiser and its correction. Because retained samples may come from earlier samplers, this reuse approximates the population iteration written in terms of q n q_n q n . For difficult targets, we also use annealing: U β n = β n U U_{\beta_n} = \beta_n U U β n = β n U , with β n \beta_n β n increasing toward
At each stage, the target score − β n ∇ U -\beta_n\nabla U − β n ∇ U replaces − ∇ U -\nabla U − ∇ U in both estimators, allowing the sampler to adapt through a sequence of tempered targets. Alg. 2 gives the extended procedure with replay, optional alignment, and annealing.
Related work
We focus on diffusion-based methods that learn amortized samplers for unnormalized targets using access to the target energy and/or its gradients, without requiring a pre-generated target dataset. PIS [33 ] formulates sampling as stochastic optimal control and trains a controlled diffusion with a path-space objective. DDS [31 ] constructs a noising process from the target to a tractable reference and learns the corresponding time reversal. iDEM [1 ] estimates scores of noised target distributions from energy evaluations and iteratively refines a diffusion model using its own samples.
More recent methods use bridge-based regression to iteratively refine the sampler. Adjoint Sampling (AS; [8 ] ) uses reciprocal adjoint matching to regress diffusion drifts while reusing endpoint energy gradients across reference bridges. Adjoint Schrödinger Bridge Sampler (ASBS; [23 ] ) extends this construction to general source distributions through a Schrödinger bridge formulation with a learned terminal corrector. Bridge Matching Sampler (BMS; [4 ] ) introduces a generalized fixed-point matching framework that recovers AS and ASBS, with an independent-coupling construction yielding explicit regression targets without a separate corrector. Flow Sampling [9 ] similarly learns denoising drifts from source-conditioned probability paths using regression targets constructed from generated endpoints and endpoint energy gradients. WASABI likewise learns through regression using generated samples and target energy gradients. However, whereas the preceding methods derive regression targets from stochastic-control optimality or bridge-based matching identities, WASABI derives its updates from the Wasserstein gradient flow.
Finally, several works, like WASABI, train diffusion samplers through a prescribed evolution of the sampling distribution. PITA [2 ] progressively trains a sequence of diffusion models along a temperature-annealing path from high temperature to the target temperature, using Sequential Monte Carlo (SMC)-based inference-time annealing to generate samples for successive model refitting. WASABI instead learns Wasserstein-induced updates through regression without SMC reweighting. Tilt Matching similarly derives regression updates to a vector-field representation of the model, but follows an exponentially tilted path induced by a reward rather than WASABI’s Wasserstein descent of the KL divergence to the target. Closely related concurrent work, Newton Matching [20 ] , optimizes the same KL objective under Fisher–Rao geometry and focuses on theory and algorithms without empirical evaluation. In comparison, WASABI uses Wasserstein geometry and is evaluated on sampling benchmarks involving particle systems and alanine dipeptide.
Experiments
Following previous literature, we evaluate WASABI on three benchmark Boltzmann distributions: the four-particle double well (DW-4), a 13-particle Lennard–Jones cluster (LJ-13), and alanine dipeptide (ALA2) [17 , 25 ] . We assess how well it reproduces each target’s energy and structure, and compare its performance to existing diffusion samplers.
Our comparison methods are Adjoint Sampling (AS; [8 ] ), Adjoint Schrödinger Bridge Sampler (ASBS; [23 ] ), and Bridge Matching Sampler (BMS; [4 ] ), including damped variants where reported. For LJ-13 and ALA2, baseline results are taken from [4 ] ; the DW-4 methods are trained under matched settings. Experiment details are given in Appendix § 10 .
Double-well system (DW-4)
We first consider DW-4, a two-dimensional system of four particles with pairwise double-well interactions (Appendix § 10 .§ 2 ). Each pairwise term is minimized at two distances, r i j ≈ 2.51 r_{ij} \approx2.51 r ij ≈ 2.51 and 5.49 5.49 5.49 , favoring competing structural arrangements. Interactions among all four particles can shift the peaks of the pair-distance distribution, although we observe them near these minima. WASABI reproduces both peaks of the pair-distance distribution (Figure 2 ). Under matched training settings, it achieves the lowest mean energy W 2 W_2 W 2 , while ASBS achieves the lowest configuration W 2 W_2 W 2 , indicating that the methods differ in how accurately they reproduce energies and particle arrangements (Table 1 ).
Figure 2. DW-4. (Left) Pooled pairwise distances { r i j } i < j \{r_{ij}\}_{i<j} { r ij } i < j from 10,000 reference and 10,000 WASABI samples; dashed lines mark the pair-potential minima. (Right) Matched-training comparison using samples pooled over rounds 625–1,000. W 2 W_2 W 2 compares particle configurations (Config.) and unfiltered potential energies (Energy).
Table 1. DW-4 at 1,000 rounds. 16 × 4 EGNN; matched training settings. Mean ± SD over six seeds. †Five seeds; one divergent run excluded (energy W 2 W_2 W 2 = 2,449).
Lennard–Jones cluster (LJ-13)
We next consider 13 identical particles in three dimensions with Lennard–Jones interactions and quadratic confinement (Appendix § 10 .§ 2 ). The pair interaction strongly repels nearby particles and favors a separation of r i j = 1 r_{ij} = 1 r ij = 1 , while the quadratic term penalizes dispersed clusters.
We train a center and a correction PaiNN, each with four layers and 64 channels, from a Gaussian prior with scale σ 0 = 1.75 \sigma_0 = 1.75 σ 0 = 1.75 . We evaluate three seeds after 7,000 outer iterations, using 128 sampling steps. WASABI closely matches the pooled pair-distance distribution, although its energy density is more concentrated near the peak and underrepresents the low-energy shoulder (Figure 3 ). Its configuration W 2 W_2 W 2 is close to ASBS, and its mean energy W 2 W_2 W 2 is lower than AS and ASBS, while BMS performs better on both metrics (Table 2 ).
Figure 3. LJ-13. (Left) Energy and pooled pair-distance { r i j } i < j \{r_{ij}\}_{i<j} { r ij } i < j densities: WASABI (green, three-seed mean) and reference (dashed). (Right) Configuration W 2 W_2 W 2 compares particle arrangements after matching and alignment; energy W 2 W_2 W 2 compares energy distributions. Results are mean ± SD across three seeds; baselines are quoted from Blessing et al. (2026, Table 2 ).
Table 2. LJ-13 comparison.
Alanine dipeptide (ALA2)
We finally evaluate WASABI on alanine dipeptide (ALA2). Its energy function includes terms describing bond stretching and angle bending, as well as preferred molecular conformations, electrostatic interactions, and the surrounding solvent (details in Appendix § 10 .§ 3 ). As in prior work [4 ] , we compare the sampled energy and backbone-angle distributions with molecular-dynamics reference samples.
Throughout our ALA2 experiments, we use molecular-dynamics configurations sampled at 1,000 K as the prior. During training, each sampled prior configuration is Kabsch-aligned to its paired endpoint from the replay buffer. The resulting coupling and corresponding mean and covariance estimators are derived in Appendices § 9 .§ 2 –§ 9 .§ 5 . Generating these samples on an Apple M5 Max chip takes approximately five minutes using one CPU thread. For the main results and hyperparameter sweeps, we train for 4,000 outer iterations, annealing the target from 1,000 K to 300 K over the first 1,500 iterations. We maintain a ring replay buffer of 4,096 endpoints and replace 256 endpoints with fresh samples at each outer iteration.
Using these settings, WASABI achieves a Jensen–Shannon divergence of 0.0254 ± 0.0033, using the same evaluation procedure as ASBS and BMS ([23 , 4 ] ), with one million generated samples per seed and one million reference configurations. WASABI trains for 4,000 outer iterations at an estimated 31.7 TFLOPs per iteration. BMS reports 5,000 outer iterations for ALA2 ([4 ] ); under the inferred eight-GPU configuration, we estimate costs of 112.9 and 190.2 TFLOPs per iteration for its undamped and damped variants, respectively (Table 3 ). Figure 4 and Table 3 summarize our results.
Figure 4. ALA2 at 300 K. (Left) Ramachandran plots comparing 20k WASABI and 20k REMD samples; gray bins are unobserved. (Right) Jensen–Shannon divergence and estimated computational cost. WASABI results are mean ± SD across three seeds; baselines are quoted from Blessing et al. [4 ] . Training cost is in TFLOPs per outer iteration, energy/force calls in millions, and D J S D_{\mathrm{JS}} D JS in nats; — denotes unreported values.
Table 3. Accuracy and computational cost. a ^{a} a Inferred eight-GPU setting based on the released codebase. b ^{b} b 3.13M calls: 2M prior production + 0.1M equilibration + 1.03M training (4,000 outer steps; bank of 4,096; 256 fresh/step).
Influence of hyperparameters. We examine the effects of covariance weight, update step size, and replay-buffer settings in Appendix § 10 .§ 4 . Among the settings tested, WASABI performs best on ALA2 with covariance weights below 0.5. To limit the computational cost of the covariance-weight and step-size ablations, which comprised 96 training runs requiring approximately 8–10 GPU-hours each, we used a replay-buffer capacity of 4,096 endpoints and refreshed 256 endpoints per outer iteration. Low Jensen–Shannon divergence can coexist with occasional underrepresentation of a smaller backbone-angle mode. Increasing buffer capacity to 8,192 and the refresh count to 512 mitigates this behavior in the evaluated runs.
Conclusion
In this paper, we introduced WASABI, a method for training diffusion samplers for Boltzmann distributions using Wasserstein gradient flow. We derived a practical training algorithm that avoids computing the diffusion model’s intractable log-density gradient. On alanine dipeptide, WASABI achieves lower Jensen–Shannon divergence than the best reported ASB/BMS variant while using an estimated 3.3 times fewer energy/force evaluations. Under matched training settings on DW-4, WASABI also achieves the lowest mean energy W 2 W_2 W 2 .
Future work can investigate scaling WASABI to larger molecular systems, where complex energy landscapes make accurate sampling particularly challenging. Another promising direction is to extend WASABI to transferable sampling, where a single model learns to sample from a family of Boltzmann distributions rather than a fixed target distribution. This setting arises naturally when the potential energy U ( x ; c ) U(x;c) U ( x ; c ) depends on additional parameters c c c , such as chemical composition [6 ] or molecular identity [16 ] . By conditioning the diffusion sampler on these parameters, one could train a model that generalizes to previously unseen conditions at test time without retraining.
More broadly, WASABI shows how the distributional evolution underlying classical sampling methods can be translated into a practical learning rule for diffusion models. Beyond Boltzmann sampling, one promising direction is to extend this approach to fine-tuning pretrained diffusion models toward reward-tilted distributions.
Limitations WASABI uses first-order denoiser updates, and finite model fitting and replay-buffer reuse introduce further approximations to the ideal population iteration. On LJ-13, WASABI under-represents the low-energy shoulder and trails BMS on both configuration and energy Wasserstein distances. On alanine dipeptide, low Jensen–Shannon divergence can coexist with underrepresentation of a smaller conformational mode; increasing replay capacity and refresh count mitigates this behavior in the evaluated runs but does not ensure complete mode coverage.
Acknowledgements
DF was partially supported by Soledad Villar’s NSF CAREER award (NSF CAREER 2339682). DB was supported by NSF grants IIS-2127869 and DMS-2311108, Office of Naval Research (ONR) grant N000142412243, and the Simons Foundation. This work was conducted under the U.S. Department of Energy National Nuclear Security Administration’s Office of Defense Nuclear Nonproliferation Research and Development (NA-22) Steel Thread Venture. This research also utilized resources provided by the Los Alamos National Laboratory Institutional Computing Program, which is supported by the U.S. Department of Energy National Nuclear Security Administration under Contract No. 89233218CNA000001. We are deeply grateful to the members of the Blei Lab for their invaluable feedback on early drafts of this manuscript.
References
Method
Denoiser Response to Endpoint Transport
Proposition A.1 (Denoiser response to endpoint transport). Let ( X 0 , X 1 ) ∼ γ n (X_0, X_1) \sim\gamma_n ( X 0 , X 1 ) ∼ γ n have marginals p 0 , q n p_0, q_n p 0 , q n , and let v : R d → R d v : \mathbb{R}^d \to\mathbb{R}^d v : R d → R d be measurable, with E [ ∥ X 0 ∥ 2 + ∥ X 1 ∥ 2 + ∥ v ( X 1 ) ∥ 2 ] < ∞ \mathbb{E}[\lVert X_0\rVert^2 + \lVert X_1\rVert^2 + \lVert v(X_1)\rVert^2] < \infty E [∥ X 0 ∥ 2 + ∥ X 1 ∥ 2 + ∥ v ( X 1 ) ∥ 2 ] < ∞ . Define X 1 ϵ = X 1 + ϵ v ( X 1 ) X_1^\epsilon= X_1 + \epsilon v(X_1) X 1 ϵ = X 1 + ϵ v ( X 1 ) , whose distribution is q n , ϵ = ( id + ϵ v ) # q n q_{n,\epsilon} = (\operatorname{id} + \epsilon v)_\#q_n q n , ϵ = ( id + ϵ v ) # q n .
For fixed t ∈ ( 0 , 1 ) t \in(0, 1) t ∈ ( 0 , 1 ) , let h t = B t ( 1 − t ) h_t = B\sqrt{t(1-t)} h t = B t ( 1 − t ) , B > 0 B > 0 B > 0 , and let Z ∼ N ( 0 , I d ) Z \sim\mathcal{N}(0, I_d) Z ∼ N ( 0 , I d ) be independent of ( X 0 , X 1 ) (X_0, X_1) ( X 0 , X 1 ) . Define Y t ϵ = ( 1 − t ) X 0 + t X 1 ϵ + h t Z Y_t^\epsilon= (1-t)X_0 + tX_1^\epsilon+ h_tZ Y t ϵ = ( 1 − t ) X 0 + t X 1 ϵ + h t Z and m ˉ t , ϵ ( n ) ( y ) = E [ X 1 ϵ ∣ Y t ϵ = y ] \bar{m}_{t,\epsilon}^{(n)}(y) = \mathbb{E}[X_1^\epsilon\mid Y_t^\epsilon= y] m ˉ t , ϵ ( n ) ( y ) = E [ X 1 ϵ ∣ Y t ϵ = y ] . Then,
∂ ϵ m ˉ t , ϵ ( n ) ( y ) ∣ ϵ = 0 = E [ v ( X 1 ) ∣ Y t = y ] + t h t Cov ( X 1 , Z ⋅ v ( X 1 ) ∣ Y t = y ) , (22) \left.\partial_\epsilon\bar{m}_{t,\epsilon}^{(n)}(y)\right|_{\epsilon=0}
= \mathbb{E}[v(X_1) \mid Y_t = y] + \frac{t}{h_t}\operatorname{Cov}(X_1, Z \cdot v(X_1) \mid Y_t = y), \tag*{(22)} ∂ ϵ m ˉ t , ϵ ( n ) ( y ) ϵ = 0 = E [ v ( X 1 ) ∣ Y t = y ] + h t t Cov ( X 1 , Z ⋅ v ( X 1 ) ∣ Y t = y ) , (22) where Y t = Y t 0 Y_t = Y_t^0 Y t = Y t 0 .
Proof. Fix t ∈ ( 0 , 1 ) t \in(0, 1) t ∈ ( 0 , 1 ) and y ∈ R d y \in\mathbb{R}^d y ∈ R d . Conditional on X 0 = x 0 X_0 = x_0 X 0 = x 0 and X 1 = x 1 X_1 = x_1 X 1 = x 1 , the perturbed interpolant has Gaussian density,
K ϵ ( y ∣ x 0 , x 1 ) = 1 ( 2 π h t 2 ) d / 2 exp ( − ∥ y − ( 1 − t ) x 0 − t [ x 1 + ϵ v ( x 1 ) ] ∥ 2 2 h t 2 ) . K_\epsilon(y \mid x_0, x_1) = \frac{1}{(2\pi h_t^2)^{d/2}}\exp\left(-\frac{\lVert y - (1-t)x_0 - t[x_1 + \epsilon v(x_1)]\rVert^2}{2h_t^2}\right). K ϵ ( y ∣ x 0 , x 1 ) = ( 2 π h t 2 ) d /2 1 exp ( − 2 h t 2 ∥ y − ( 1 − t ) x 0 − t [ x 1 + ϵ v ( x 1 )] ∥ 2 ) .
Consequently,
m ˉ t , ϵ ( n ) ( y ) = E [ X 1 ϵ ∣ Y t ϵ = y ] = ∫ [ x 1 + ϵ v ( x 1 ) ] K ϵ ( y ∣ x 0 , x 1 ) d γ n ( x 0 , x 1 ) ∫ K ϵ ( y ∣ x 0 , x 1 ) d γ n ( x 0 , x 1 ) = : N ( ϵ ) D ( ϵ ) . \bar{m}_{t,\epsilon}^{(n)}(y) = \mathbb{E}[X_1^\epsilon\mid Y_t^\epsilon= y]
= \frac{\int[x_1 + \epsilon v(x_1)]K_\epsilon(y \mid x_0, x_1)\,d\gamma_n(x_0,x_1)}{\int K_\epsilon(y \mid x_0, x_1)\,d\gamma_n(x_0,x_1)}
=: \frac{N(\epsilon)}{D(\epsilon)}. m ˉ t , ϵ ( n ) ( y ) = E [ X 1 ϵ ∣ Y t ϵ = y ] = ∫ K ϵ ( y ∣ x 0 , x 1 ) d γ n ( x 0 , x 1 ) ∫ [ x 1 + ϵ v ( x 1 )] K ϵ ( y ∣ x 0 , x 1 ) d γ n ( x 0 , x 1 ) =: D ( ϵ ) N ( ϵ ) .
The denominator is strictly positive because the Gaussian kernel is positive.
To differentiate under these integrals, we establish integrable bounds for their derivatives. Differentiating the Gaussian gives
∂ ϵ K ϵ = t h t 2 [ y − ( 1 − t ) x 0 − t ( x 1 + ϵ v ( x 1 ) ) ] ⋅ v ( x 1 ) K ϵ . \partial_\epsilon K_\epsilon= \frac{t}{h_t^2}\left[y - (1-t)x_0 - t(x_1 + \epsilon v(x_1))\right]\cdot v(x_1)\,K_\epsilon. ∂ ϵ K ϵ = h t 2 t [ y − ( 1 − t ) x 0 − t ( x 1 + ϵ v ( x 1 )) ] ⋅ v ( x 1 ) K ϵ .
By the product rule,
∂ ϵ ( [ x 1 + ϵ v ( x 1 ) ] K ϵ ) = v ( x 1 ) K ϵ + [ x 1 + ϵ v ( x 1 ) ] ∂ ϵ K ϵ = v ( x 1 ) K ϵ + t h t 2 [ x 1 + ϵ v ( x 1 ) ] × ( [ y − ( 1 − t ) x 0 − t ( x 1 + ϵ v ( x 1 ) ) ] ⋅ v ( x 1 ) ) K ϵ . \begin{aligned}
\partial_\epsilon\left([x_1+\epsilon v(x_1)]K_\epsilon\right)
&=v(x_1)K_\epsilon+[x_1+\epsilon v(x_1)]\partial_\epsilon K_\epsilon\\
&=v(x_1)K_\epsilon+\frac{t}{h_t^2}[x_1+\epsilon v(x_1)]\\
&\qquad\times\left([y-(1-t)x_0-t(x_1+\epsilon v(x_1))]\cdot v(x_1)\right)K_\epsilon.
\end{aligned} ∂ ϵ ( [ x 1 + ϵ v ( x 1 )] K ϵ ) = v ( x 1 ) K ϵ + [ x 1 + ϵ v ( x 1 )] ∂ ϵ K ϵ = v ( x 1 ) K ϵ + h t 2 t [ x 1 + ϵ v ( x 1 )] × ( [ y − ( 1 − t ) x 0 − t ( x 1 + ϵ v ( x 1 ))] ⋅ v ( x 1 ) ) K ϵ .
Taking norms, applying Cauchy–Schwarz to the inner product, and using ∣ ϵ ∣ ≤ 1 |\epsilon|\le1 ∣ ϵ ∣ ≤ 1 , we obtain
∥ ∂ ϵ ( [ x 1 + ϵ v ( x 1 ) ] K ϵ ) ∥ ≤ ∥ v ( x 1 ) ∥ K ϵ + t h t 2 ( ∥ x 1 ∥ + ∥ v ( x 1 ) ∥ ) ∥ v ( x 1 ) ∥ × ∥ y − ( 1 − t ) x 0 − t ( x 1 + ϵ v ( x 1 ) ) ∥ K ϵ ≤ C t ( ∥ v ( x 1 ) ∥ + ∥ x 1 ∥ ∥ v ( x 1 ) ∥ + ∥ v ( x 1 ) ∥ 2 ) . \begin{aligned}
\left\|\partial_\epsilon\left([x_1+\epsilon v(x_1)]K_\epsilon\right)\right\|
&\le\|v(x_1)\|K_\epsilon+\frac{t}{h_t^2}\left(\|x_1\|+\|v(x_1)\|\right)\|v(x_1)\|\\
&\qquad\times\|y-(1-t)x_0-t(x_1+\epsilon v(x_1))\|K_\epsilon\\
&\le C_t\left(\|v(x_1)\|+\|x_1\|\|v(x_1)\|+\|v(x_1)\|^2\right).
\end{aligned} ∥ ∂ ϵ ( [ x 1 + ϵ v ( x 1 )] K ϵ ) ∥ ≤ ∥ v ( x 1 ) ∥ K ϵ + h t 2 t ( ∥ x 1 ∥ + ∥ v ( x 1 ) ∥ ) ∥ v ( x 1 ) ∥ × ∥ y − ( 1 − t ) x 0 − t ( x 1 + ϵ v ( x 1 )) ∥ K ϵ ≤ C t ( ∥ v ( x 1 ) ∥ + ∥ x 1 ∥∥ v ( x 1 ) ∥ + ∥ v ( x 1 ) ∥ 2 ) .
The last inequality follows because a Gaussian density is bounded, as is its product with the distance from its mean. Applying a b ≤ ( a 2 + b 2 ) / 2 ab\le(a^2+b^2)/2 ab ≤ ( a 2 + b 2 ) /2 gives
∥ v ( x 1 ) ∥ ≤ 1 + ∥ v ( x 1 ) ∥ 2 2 , ∥ x 1 ∥ ∥ v ( x 1 ) ∥ ≤ ∥ x 1 ∥ 2 + ∥ v ( x 1 ) ∥ 2 2 . \|v(x_1)\|\le\frac{1+\|v(x_1)\|^2}{2},\qquad
\|x_1\|\|v(x_1)\|\le\frac{\|x_1\|^2+\|v(x_1)\|^2}{2}. ∥ v ( x 1 ) ∥ ≤ 2 1 + ∥ v ( x 1 ) ∥ 2 , ∥ x 1 ∥∥ v ( x 1 ) ∥ ≤ 2 ∥ x 1 ∥ 2 + ∥ v ( x 1 ) ∥ 2 .
Thus, after enlarging C t C_t C t ,
∥ ∂ ϵ ( [ x 1 + ϵ v ( x 1 ) ] K ϵ ) ∥ ≤ C t ( 1 + ∥ x 1 ∥ 2 + ∥ v ( x 1 ) ∥ 2 ) . \left\|\partial_\epsilon\left([x_1+\epsilon v(x_1)]K_\epsilon\right)\right\|
\le C_t\left(1+\|x_1\|^2+\|v(x_1)\|^2\right). ∥ ∂ ϵ ( [ x 1 + ϵ v ( x 1 )] K ϵ ) ∥ ≤ C t ( 1 + ∥ x 1 ∥ 2 + ∥ v ( x 1 ) ∥ 2 ) .
Similarly, ∣ ∂ ϵ K ϵ ∣ ≤ C t ∥ v ( x 1 ) ∥ |\partial_\epsilon K_\epsilon|\le C_t\|v(x_1)\| ∣ ∂ ϵ K ϵ ∣ ≤ C t ∥ v ( x 1 ) ∥ . These bounds are integrable under the assumed second moments, so dominated convergence justifies differentiation under both integrals.
Evaluating the derivatives at ϵ = 0 \epsilon=0 ϵ = 0 , and abbreviating the kernel arguments and d γ n = d γ n ( x 0 , x 1 ) d\gamma_n=d\gamma_n(x_0,x_1) d γ n = d γ n ( x 0 , x 1 ) , gives
N ′ ( 0 ) = ∫ v ( x 1 ) K 0 d γ n + t h t 2 ∫ x 1 ( [ y − ( 1 − t ) x 0 − t x 1 ] ⋅ v ( x 1 ) ) K 0 d γ n , D ′ ( 0 ) = t h t 2 ∫ ( [ y − ( 1 − t ) x 0 − t x 1 ] ⋅ v ( x 1 ) ) K 0 d γ n . \begin{aligned}
N'(0)
&=\int v(x_1)K_0\,d\gamma_n\\
&\qquad+\frac{t}{h_t^2}\int x_1\left([y-(1-t)x_0-tx_1]\cdot v(x_1)\right)K_0\,d\gamma_n,\\
D'(0)
&=\frac{t}{h_t^2}\int\left([y-(1-t)x_0-tx_1]\cdot v(x_1)\right)K_0\,d\gamma_n.
\end{aligned} N ′ ( 0 ) D ′ ( 0 ) = ∫ v ( x 1 ) K 0 d γ n + h t 2 t ∫ x 1 ( [ y − ( 1 − t ) x 0 − t x 1 ] ⋅ v ( x 1 ) ) K 0 d γ n , = h t 2 t ∫ ( [ y − ( 1 − t ) x 0 − t x 1 ] ⋅ v ( x 1 ) ) K 0 d γ n .
The conditional distribution of the endpoints given Y t = y Y_t=y Y t = y is
γ n ( d x 0 , d x 1 ∣ Y t = y ) = K 0 ( y ∣ x 0 , x 1 ) D ( 0 ) d γ n ( x 0 , x 1 ) . \gamma_n(dx_0,dx_1\mid Y_t=y)=\frac{K_0(y\mid x_0,x_1)}{D(0)}\,d\gamma_n(x_0,x_1). γ n ( d x 0 , d x 1 ∣ Y t = y ) = D ( 0 ) K 0 ( y ∣ x 0 , x 1 ) d γ n ( x 0 , x 1 ) .
Under this conditioning, y − ( 1 − t ) X 0 − t X 1 = h t Z y-(1-t)X_0-tX_1=h_tZ y − ( 1 − t ) X 0 − t X 1 = h t Z . Therefore,
N ′ ( 0 ) D ( 0 ) = E [ v ( X 1 ) ∣ Y t = y ] + t h t E [ X 1 ( Z ⋅ v ( X 1 ) ) ∣ Y t = y ] , \frac{N'(0)}{D(0)}
=\mathbb{E}[v(X_1)\mid Y_t=y]+\frac{t}{h_t}\mathbb{E}[X_1(Z\cdot v(X_1))\mid Y_t=y], D ( 0 ) N ′ ( 0 ) = E [ v ( X 1 ) ∣ Y t = y ] + h t t E [ X 1 ( Z ⋅ v ( X 1 )) ∣ Y t = y ] ,
D ′ ( 0 ) D ( 0 ) = t h t E [ Z ⋅ v ( X 1 ) ∣ Y t = y ] . \frac{D'(0)}{D(0)}
=\frac{t}{h_t}\mathbb{E}[Z\cdot v(X_1)\mid Y_t=y]. D ( 0 ) D ′ ( 0 ) = h t t E [ Z ⋅ v ( X 1 ) ∣ Y t = y ] .
Finally, applying the quotient rule and using N ( 0 ) / D ( 0 ) = m ˉ t ( n ) ( y ) N(0)/D(0)=\bar m_t^{(n)}(y) N ( 0 ) / D ( 0 ) = m ˉ t ( n ) ( y ) ,
∂ ϵ m ˉ t , ϵ ( n ) ( y ) ∣ 0 = N ′ ( 0 ) D ( 0 ) − m ˉ t ( n ) ( y ) D ′ ( 0 ) D ( 0 ) = E [ v ( X 1 ) ∣ Y t = y ] + t h t ( E [ X 1 ( Z ⋅ v ( X 1 ) ) ∣ Y t = y ] − E [ X 1 ∣ Y t = y ] E [ Z ⋅ v ( X 1 ) ∣ Y t = y ] ) = E [ v ( X 1 ) ∣ Y t = y ] + t h t Cov ( X 1 , Z ⋅ v ( X 1 ) ∣ Y t = y ) , \begin{aligned}
\left.\partial_\epsilon\bar m_{t,\epsilon}^{(n)}(y)\right|_0
&=\frac{N'(0)}{D(0)}-\bar m_t^{(n)}(y)\frac{D'(0)}{D(0)}\\
&=\mathbb{E}[v(X_1)\mid Y_t=y]\\
&\qquad+\frac{t}{h_t}\bigl(\mathbb{E}[X_1(Z\cdot v(X_1))\mid Y_t=y]\\
&\qquad\qquad-\mathbb{E}[X_1\mid Y_t=y]\mathbb{E}[Z\cdot v(X_1)\mid Y_t=y]\bigr)\\
&=\mathbb{E}[v(X_1)\mid Y_t=y]+\frac{t}{h_t}\operatorname{Cov}(X_1,Z\cdot v(X_1)\mid Y_t=y),
\end{aligned} ∂ ϵ m ˉ t , ϵ ( n ) ( y ) 0 = D ( 0 ) N ′ ( 0 ) − m ˉ t ( n ) ( y ) D ( 0 ) D ′ ( 0 ) = E [ v ( X 1 ) ∣ Y t = y ] + h t t ( E [ X 1 ( Z ⋅ v ( X 1 )) ∣ Y t = y ] − E [ X 1 ∣ Y t = y ] E [ Z ⋅ v ( X 1 ) ∣ Y t = y ] ) = E [ v ( X 1 ) ∣ Y t = y ] + h t t Cov ( X 1 , Z ⋅ v ( X 1 ) ∣ Y t = y ) ,
which proves the claim.
Taking γ n = p 0 ⊗ q n \gamma_n=p_0\otimes q_n γ n = p 0 ⊗ q n and v = − ∇ W 2 F [ q n ] v=-\nabla W_2F[q_n] v = − ∇ W 2 F [ q n ] recovers the main-text theorem whenever the stated moment condition holds.
Kabsch Alignment and the Induced Coupling
Let g = ( g 1 , … , g N ) g = (g_1,\ldots,g_N) g = ( g 1 , … , g N ) and x = ( x 1 , … , x N ) x = (x_1,\ldots,x_N) x = ( x 1 , … , x N ) be centered configurations in R 3 N \mathbb{R}^{3N} R 3 N . Kabsch alignment finds a rotation
R ⋆ ( g , x ) ∈ arg min R ∈ S O ( 3 ) 1 2 ∑ i = 1 N ∥ R g i − x i ∥ 2 R^\star(g,x) \in\underset{R\in SO(3)}{\operatorname{arg\,min}}\ \frac{1}{2}\sum_{i=1}^{N}\lVert Rg_i-x_i\rVert^2 R ⋆ ( g , x ) ∈ R ∈ S O ( 3 ) arg min 2 1 i = 1 ∑ N ∥ R g i − x i ∥ 2
where S O ( 3 ) = { R ∈ R 3 × 3 : R T R = I , det R = 1 } SO(3)=\{R\in\mathbb{R}^{3\times3}:R^{\mathsf{T}}R=I,\det R=1\} S O ( 3 ) = { R ∈ R 3 × 3 : R T R = I , det R = 1 } . We define the aligned configuration by A ( g , x ) = ( R ⋆ ( g , x ) g i ) i = 1 N A(g,x)=(R^\star(g,x)g_i)_{i=1}^{N} A ( g , x ) = ( R ⋆ ( g , x ) g i ) i = 1 N .
To construct the aligned coupling, we independently draw G ∼ p 0 G\sim p_0 G ∼ p 0 and X 1 ∼ q n X_1\sim q_n X 1 ∼ q n , then set X 0 = A ( G , X 1 ) X_0=A(G,X_1) X 0 = A ( G , X 1 ) . In our ALA2 experiments, p 0 p_0 p 0 is the empirical distribution of configurations obtained from high-temperature molecular dynamics. The resulting pair ( X 0 , X 1 ) (X_0,X_1) ( X 0 , X 1 ) defines the coupling γ n \gamma_n γ n , and the bridge observation is
Y t = ( 1 − t ) X 0 + t X 1 + h t Z , Z ∼ N ( 0 , I ) . Y_t=(1-t)X_0+tX_1+h_tZ,\qquad Z\sim\mathcal{N}(0,I). Y t = ( 1 − t ) X 0 + t X 1 + h t Z , Z ∼ N ( 0 , I ) .
with Z Z Z independent of G , X 1 G,X_1 G , X 1 . Here p 0 p_0 p 0 describes the unaligned seed G G G ; the aligned sample X 0 X_0 X 0 need not have the same marginal distribution.
In the endpoint-transport derivative, the already aligned source X 0 X_0 X 0 is held fixed while X 1 X_1 X 1 is perturbed; alignment is not recomputed during that infinitesimal perturbation.
To derive the Jacobian D x A ( g , x ) D_xA(g,x) D x A ( g , x ) , we first review basic facts about the Lie algebra of S O ( 3 ) SO(3) S O ( 3 ) .
Lemma A.2 (Derivative of a rotation). Let R : ( − δ , δ ) → S O ( 3 ) R:(-\delta,\delta)\to SO(3) R : ( − δ , δ ) → S O ( 3 ) be differentiable. Then there exists Ω ∈ T I S O ( 3 ) \Omega\in T_I SO(3) Ω ∈ T I S O ( 3 ) such that
R ′ ( 0 ) = Ω R ( 0 ) , T I S O ( 3 ) = { Ω ∈ R 3 × 3 : Ω T = − Ω } . R'(0)=\Omega R(0),\qquad T_I SO(3)=\{\Omega\in\mathbb{R}^{3\times3}:\Omega^{\mathsf{T}}=-\Omega\}. R ′ ( 0 ) = Ω R ( 0 ) , T I S O ( 3 ) = { Ω ∈ R 3 × 3 : Ω T = − Ω } . Proof. Differentiating R ( ϵ ) R ( ϵ ) T = I R(\epsilon)R(\epsilon)^{\mathsf{T}}=I R ( ϵ ) R ( ϵ ) T = I at ϵ = 0 \epsilon=0 ϵ = 0 gives
R ′ ( 0 ) R ( 0 ) T + R ( 0 ) R ′ ( 0 ) T = 0. R'(0)R(0)^{\mathsf{T}}+R(0)R'(0)^{\mathsf{T}}=0. R ′ ( 0 ) R ( 0 ) T + R ( 0 ) R ′ ( 0 ) T = 0. Thus Ω : = R ′ ( 0 ) R ( 0 ) T \Omega:=R'(0)R(0)^{\mathsf{T}} Ω := R ′ ( 0 ) R ( 0 ) T is skew-symmetric and belongs to T I S O ( 3 ) T_I SO(3) T I S O ( 3 ) . Multiplying by R ( 0 ) R(0) R ( 0 ) yields R ′ ( 0 ) = Ω R ( 0 ) R'(0)=\Omega R(0) R ′ ( 0 ) = Ω R ( 0 ) .
Every Ω ∈ T I S O ( 3 ) \Omega\in T_I SO(3) Ω ∈ T I S O ( 3 ) can be written uniquely as
Ω = [ ω ] × : = ( 0 − ω 3 ω 2 ω 3 0 − ω 1 − ω 2 ω 1 0 ) , ω ∈ R 3 . \Omega=[\omega]_\times:=\begin{pmatrix}0&-\omega_3&\omega_2\\\omega_3&0&-\omega_1\\-\omega_2&\omega_1&0\end{pmatrix},\qquad\omega\in\mathbb{R}^3. Ω = [ ω ] × := 0 ω 3 − ω 2 − ω 3 0 ω 1 ω 2 − ω 1 0 , ω ∈ R 3 . Direct multiplication shows that Ω a = ω × a \Omega a=\omega\times a Ω a = ω × a . Consequently, for a i ( ϵ ) = R ( ϵ ) g i a_i(\epsilon)=R(\epsilon)g_i a i ( ϵ ) = R ( ϵ ) g i , the preceding lemma gives
a i ′ ( 0 ) = Ω R ( 0 ) g i = ω × a i ( 0 ) . a_i'(0)=\Omega R(0)g_i=\omega\times a_i(0). a i ′ ( 0 ) = Ω R ( 0 ) g i = ω × a i ( 0 ) .
Lemma A.3 (Optimality condition for alignment). Let R ⋆ R^\star R ⋆ minimize 1 2 ∑ i ∥ R g i − x i ∥ 2 \frac{1}{2}\sum_i\lVert Rg_i-x_i\rVert^2 2 1 ∑ i ∥ R g i − x i ∥ 2 over S O ( 3 ) SO(3) S O ( 3 ) , and set a i = R ⋆ g i a_i=R^\star g_i a i = R ⋆ g i . Then
∑ i a i × x i = 0. \sum_i a_i\times x_i=0. i ∑ a i × x i = 0. Proof. Fix u ∈ R 3 u\in\mathbb{R}^3 u ∈ R 3 and consider the rotated positions b i ( s ) = exp ( s [ u ] × ) a i b_i(s)=\exp(s[u]_\times)a_i b i ( s ) = exp ( s [ u ] × ) a i , where exp \exp exp denotes the matrix exponential. These satisfy b i ( 0 ) = a i b_i(0)=a_i b i ( 0 ) = a i and b i ′ ( 0 ) = u × a i b_i'(0)=u\times a_i b i ′ ( 0 ) = u × a i .
Since a a a is optimally aligned, E ( s ) : = 1 2 ∑ i ∥ b i ( s ) − x i ∥ 2 E(s):=\frac{1}{2}\sum_i\lVert b_i(s)-x_i\rVert^2 E ( s ) := 2 1 ∑ i ∥ b i ( s ) − x i ∥ 2 has a minimum at s = 0 s=0 s = 0 . Therefore,
0 = E ′ ( 0 ) = ∑ i ( a i − x i ) ⋅ ( u × a i ) 0=E'(0)=\sum_i(a_i-x_i)\cdot(u\times a_i) 0 = E ′ ( 0 ) = i ∑ ( a i − x i ) ⋅ ( u × a i ) = − ∑ i x i ⋅ ( u × a i ) =-\sum_i x_i\cdot(u\times a_i) = − i ∑ x i ⋅ ( u × a i ) = − u ⋅ ∑ i a i × x i . =-u\cdot\sum_i a_i\times x_i. = − u ⋅ i ∑ a i × x i .
As this holds for every u ∈ R 3 u \in\mathbb{R}^{3} u ∈ R 3 , we conclude that
∑ i a i × x i = 0. \sum_{i} a_i \times x_i = 0. i ∑ a i × x i = 0.
Theorem A.4 (Jacobian of Kabsch alignment). Fix centered configurations g , x g, x g , x , and suppose the optimal rotation R ⋆ ( g , ⋅ ) R^\star(g,\cdot) R ⋆ ( g , ⋅ ) is unique and differentiable in a neighborhood of x x x . Write a i = R ⋆ ( g , x ) g i a_i = R^\star(g,x)g_i a i = R ⋆ ( g , x ) g i , and assume that
H ≔ ∑ i = 1 N [ ( a i T x i ) I − a i x i T ] H \coloneqq\sum_{i=1}^{N}\left[(a_i^{\mathsf T}x_i)I-a_ix_i^{\mathsf T}\right] H : = i = 1 ∑ N [ ( a i T x i ) I − a i x i T ] is invertible. Then, for any centered perturbation δ x \delta x δ x ,
[ D x A ( g , x ) δ x ] i = ( H − 1 ∑ j = 1 N a j × δ x j ) × a i . [D_xA(g,x)\,\delta x]_i=\left(H^{-1}\sum_{j=1}^{N}a_j\times\delta x_j\right)\times a_i. [ D x A ( g , x ) δ x ] i = ( H − 1 j = 1 ∑ N a j × δ x j ) × a i . Equivalently, defining T ω = ( ω × a i ) i = 1 N T\omega=(\omega\times a_i)_{i=1}^{N} T ω = ( ω × a i ) i = 1 N , we have D x A ( g , x ) = T H − 1 T T D_xA(g,x)=TH^{-1}T^{\mathsf T} D x A ( g , x ) = T H − 1 T T .
Proof. Let R ( ϵ ) = R ⋆ ( g , x + ϵ δ x ) R(\epsilon)=R^\star(g,x+\epsilon\delta x) R ( ϵ ) = R ⋆ ( g , x + ϵδ x ) and a i ( ϵ ) = R ( ϵ ) g i a_i(\epsilon)=R(\epsilon)g_i a i ( ϵ ) = R ( ϵ ) g i . By the rotation derivative lemma, there exists ω ∈ R 3 \omega\in\mathbb{R}^{3} ω ∈ R 3 such that
a i ′ ( 0 ) = ω × a i . a_i'(0)=\omega\times a_i. a i ′ ( 0 ) = ω × a i .
For each sufficiently small ϵ \epsilon ϵ , the configuration a ( ϵ ) a(\epsilon) a ( ϵ ) is optimally aligned to x + ϵ δ x x+\epsilon\delta x x + ϵδ x . The optimality condition therefore gives
∑ i a i ( ϵ ) × ( x i + ϵ δ x i ) = 0. \sum_i a_i(\epsilon)\times(x_i+\epsilon\delta x_i)=0. i ∑ a i ( ϵ ) × ( x i + ϵδ x i ) = 0.
Differentiating at ϵ = 0 \epsilon=0 ϵ = 0 yields
0 = ∑ i [ ( ω × a i ) × x i + a i × δ x i ] . 0=\sum_i\left[(\omega\times a_i)\times x_i+a_i\times\delta x_i\right]. 0 = i ∑ [ ( ω × a i ) × x i + a i × δ x i ] .
Using the BAC–CAB identity Jackson [13 ] ,
( ω × a i ) × x i = a i ( x i T ω ) − ( a i T x i ) ω = [ a i x i T − ( a i T x i ) I ] ω . \begin{aligned}
(\omega\times a_i)\times x_i=a_i(x_i^{\mathsf T}\omega)-(a_i^{\mathsf T}x_i)\omega\\
&=[a_ix_i^{\mathsf T}-(a_i^{\mathsf T}x_i)I]\omega.
\end{aligned} ( ω × a i ) × x i = a i ( x i T ω ) − ( a i T x i ) ω = [ a i x i T − ( a i T x i ) I ] ω .
Substituting and rearranging, we obtain
H ω = ∑ i a i × δ x i . H\omega=\sum_i a_i\times\delta x_i. H ω = i ∑ a i × δ x i .
Since H H H is invertible,
ω = H − 1 ∑ i a i × δ x i . \omega=H^{-1}\sum_i a_i\times\delta x_i. ω = H − 1 i ∑ a i × δ x i .
Substituting into [ D x A ( g , x ) δ x ] i = a i ′ ( 0 ) = ω × a i [D_xA(g,x)\,\delta x]_i=a_i'(0)=\omega\times a_i [ D x A ( g , x ) δ x ] i = a i ′ ( 0 ) = ω × a i proves the stated formula. Finally, the scalar triple-product identity gives
⟨ T ω , δ x ⟩ = ∑ i ( ω × a i ) ⋅ δ x i = ω ⋅ ∑ i a i × δ x i . \langle T\omega,\delta x\rangle=\sum_i(\omega\times a_i)\cdot\delta x_i=\omega\cdot\sum_i a_i\times\delta x_i. ⟨ T ω , δ x ⟩ = i ∑ ( ω × a i ) ⋅ δ x i = ω ⋅ i ∑ a i × δ x i .
Thus T T δ x = ∑ i a i × δ x i T^{\mathsf T}\delta x=\sum_i a_i\times\delta x_i T T δ x = ∑ i a i × δ x i , and we conclude that
D x A ( g , x ) = T H − 1 T T . D_xA(g,x)=TH^{-1}T^{\mathsf T}. D x A ( g , x ) = T H − 1 T T .
Corollary A.5 (Positivity of the bridge Jacobian). Under the assumptions of the preceding theorem, H ≻ 0 H\succ0 H ≻ 0 and D x A ( g , x ) = T H − 1 T T ⪰ 0 D_xA(g,x)=TH^{-1}T^{\mathsf T}\succeq0 D x A ( g , x ) = T H − 1 T T ⪰ 0 . Consequently, for 0 < t < 1 0<t<1 0 < t < 1 ,
J t = t I + ( 1 − t ) D x A ( g , x ) ⪰ t I ≻ 0. J_t=tI+(1-t)D_xA(g,x)\succeq tI\succ0. J t = t I + ( 1 − t ) D x A ( g , x ) ⪰ t I ≻ 0.
Proof. The optimality condition implies that ∑ i a i x i ⊤ \sum_i a_i x_i^\top ∑ i a i x i ⊤ is symmetric, so H H H is symmetric. For any u ∈ R 3 u \in\mathbb{R}^3 u ∈ R 3 , let Ω v = u × v \Omega v = u \times v Ω v = u × v and consider E ( s ) = 1 2 ∑ i ∥ e s Ω a i − x i ∥ 2 E(s) = \frac{1}{2}\sum_i \lVert e^{s\Omega}a_i - x_i\rVert^2 E ( s ) = 2 1 ∑ i ∥ e s Ω a i − x i ∥ 2 . The alignment minimum at s = 0 s=0 s = 0 gives
0 ≤ E ′ ′ ( 0 ) = ∑ i [ ( a i ⊤ x i ) ∥ u ∥ 2 − ( u ⊤ a i ) ( x i ⊤ u ) ] = u ⊤ H u . 0 \le E''(0) = \sum_i \left[(a_i^\top x_i)\lVert u\rVert^2 - (u^\top a_i)(x_i^\top u)\right] = u^\top H u. 0 ≤ E ′′ ( 0 ) = i ∑ [ ( a i ⊤ x i ) ∥ u ∥ 2 − ( u ⊤ a i ) ( x i ⊤ u ) ] = u ⊤ H u .
Thus H ⪰ 0 H \succeq0 H ⪰ 0 ; its assumed invertibility gives H ≻ 0 H \succ0 H ≻ 0 . The remaining claims follow from D x A = T H − 1 T ⊤ D_xA = TH^{-1}T^\top D x A = T H − 1 T ⊤ .
Stein’s Identity and the Mean Estimator
Proposition A.6 (Mean estimator under the aligned coupling). Let G ∼ p 0 G \sim p_0 G ∼ p 0 , X 1 ∼ q n X_1 \sim q_n X 1 ∼ q n , and Z ∼ N ( 0 , I ) Z \sim\mathcal{N}(0,I) Z ∼ N ( 0 , I ) be independent. For fixed t ∈ ( 0 , 1 ) t \in(0,1) t ∈ ( 0 , 1 ) and h t > 0 h_t > 0 h t > 0 , define
Y t = ( 1 − t ) A ( G , X 1 ) + t X 1 + h t Z , J t = t I + ( 1 − t ) D X 1 A ( G , X 1 ) . Y_t = (1-t)A(G,X_1) + tX_1 + h_tZ,\qquad J_t = tI + (1-t)D_{X_1}A(G,X_1). Y t = ( 1 − t ) A ( G , X 1 ) + t X 1 + h t Z , J t = t I + ( 1 − t ) D X 1 A ( G , X 1 ) . Assume q n q_n q n has a differentiable positive density, A ( G , ⋅ ) A(G,\cdot) A ( G , ⋅ ) is differentiable almost surely, and the integrability and vanishing-boundary conditions required for the following applications of Stein’s identity hold. Then
M t [ q n ] ( y ) : = E [ ∇ log π ( X 1 ) − ∇ log q n ( X 1 ) ∣ Y t = y ] M_t[q_n](y) := \mathbb{E}[\nabla\log\pi(X_1) - \nabla\log q_n(X_1)\mid Y_t=y] M t [ q n ] ( y ) := E [ ∇ log π ( X 1 ) − ∇ log q n ( X 1 ) ∣ Y t = y ] = E [ ∇ log π ( X 1 ) + J t ⊤ Z h t ∣ Y t = y ] = \mathbb{E}\left[\left.\nabla\log\pi(X_1) + \frac{J_t^\top Z}{h_t}\right|Y_t=y\right] = E [ ∇ log π ( X 1 ) + h t J t ⊤ Z Y t = y ] for almost every y y y under the distribution of Y t Y_t Y t .
Proof. Let φ : R d → R \varphi: \mathbb{R}^d \to\mathbb{R} φ : R d → R be a smooth, compactly supported test function. Holding G G G and Z Z Z fixed, we apply the vector Stein identity (Eq. (9) ) in X 1 X_1 X 1 to
f ( x ) = φ ( ( 1 − t ) A ( G , x ) + t x + h t Z ) . f(x) = \varphi((1-t)A(G,x) + tx + h_tZ). f ( x ) = φ (( 1 − t ) A ( G , x ) + t x + h t Z ) .
Applying Eq. (9) with r = q n r=q_n r = q n gives
E X 1 [ f ( X 1 ) ∇ log q n ( X 1 ) + ∇ x f ( X 1 ) ] = 0. \mathbb{E}_{X_1}[f(X_1)\nabla\log q_n(X_1) + \nabla_x f(X_1)] = 0. E X 1 [ f ( X 1 ) ∇ log q n ( X 1 ) + ∇ x f ( X 1 )] = 0.
By the chain rule,
∇ x f ( X 1 ) = [ t I + ( 1 − t ) D X 1 A ( G , X 1 ) ] ⊤ ⏟ J t ⊤ ∇ φ ( Y t ) . \nabla_x f(X_1) = \underbrace{[tI + (1-t)D_{X_1}A(G,X_1)]^\top}_{J_t^\top}\nabla\varphi(Y_t). ∇ x f ( X 1 ) = J t ⊤ [ t I + ( 1 − t ) D X 1 A ( G , X 1 ) ] ⊤ ∇ φ ( Y t ) .
Substituting and averaging over G , Z G,Z G , Z , we obtain
E [ φ ( Y t ) ∇ log q n ( X 1 ) + J t ⊤ ∇ φ ( Y t ) ] = 0. \mathbb{E}[\varphi(Y_t)\nabla\log q_n(X_1) + J_t^\top\nabla\varphi(Y_t)] = 0. E [ φ ( Y t ) ∇ log q n ( X 1 ) + J t ⊤ ∇ φ ( Y t )] = 0.
For standard Gaussian noise, whose score is − Z -Z − Z , the general Stein identity in Eq. (9) specializes to
E [ Z ξ ( Z ) ] = E [ ∇ ξ ( Z ) ] (23) \mathbb{E}[Z\xi(Z)] = \mathbb{E}[\nabla\xi(Z)] \tag*{(23)} E [ Z ξ ( Z )] = E [ ∇ ξ ( Z )] (23)
for sufficiently regular scalar test functions ξ \xi ξ satisfying the integrability and vanishing-boundary conditions.
Next, applying the Gaussian vector Stein identity (Eq. (23) ) in Z Z Z , with G , X 1 G,X_1 G , X 1 fixed, gives
E Z [ Z φ ( Y t ) ∣ G , X 1 ] = h t E Z [ ∇ φ ( Y t ) ∣ G , X 1 ] . \mathbb{E}_Z[Z\varphi(Y_t)\mid G,X_1] = h_t\mathbb{E}_Z[\nabla\varphi(Y_t)\mid G,X_1]. E Z [ Z φ ( Y t ) ∣ G , X 1 ] = h t E Z [ ∇ φ ( Y t ) ∣ G , X 1 ] .
Since J t J_t J t does not depend on Z Z Z , multiplying by J t ⊤ / h t J_t^\top/h_t J t ⊤ / h t and averaging yields
E [ J t ⊤ ∇ φ ( Y t ) ] = 1 h t E [ φ ( Y t ) J t ⊤ Z ] . \mathbb{E}[J_t^\top\nabla\varphi(Y_t)] = \frac{1}{h_t}\mathbb{E}[\varphi(Y_t)J_t^\top Z]. E [ J t ⊤ ∇ φ ( Y t )] = h t 1 E [ φ ( Y t ) J t ⊤ Z ] .
Combining the two identities,
E [ φ ( Y t ) ( ∇ log q n ( X 1 ) + J t ⊤ Z h t ) ] = 0. \mathbb{E}\left[\varphi(Y_t)\left(\nabla\log q_n(X_1) + \frac{J_t^\top Z}{h_t}\right)\right] = 0. E [ φ ( Y t ) ( ∇ log q n ( X 1 ) + h t J t ⊤ Z ) ] = 0.
As this holds for every such φ \varphi φ , we conclude that
E [ ∇ log q n ( X 1 ) ∣ Y t = y ] = − 1 h t E [ J t ⊤ Z ∣ Y t = y ] . \mathbb{E}[\nabla\log q_n(X_1)\mid Y_t=y] = -\frac{1}{h_t}\mathbb{E}[J_t^\top Z\mid Y_t=y]. E [ ∇ log q n ( X 1 ) ∣ Y t = y ] = − h t 1 E [ J t ⊤ Z ∣ Y t = y ] .
Substitution into the definition of M t [ q n ] M_t[q_n] M t [ q n ] proves the result.
For independent endpoints, A ( G , x ) = G A(G,x)=G A ( G , x ) = G , so J t = t I J_t=tI J t = t I , recovering the estimator ∇ log π ( X 1 ) + t Z / h t \nabla\log\pi(X_1)+tZ/h_t ∇ log π ( X 1 ) + tZ / h t . For Kabsch alignment, the preceding Jacobian theorem gives J t = t I + ( 1 − t ) T H − 1 T ⊤ J_t=tI+(1-t)TH^{-1}T^\top J t = t I + ( 1 − t ) T H − 1 T ⊤ .
Covariance Estimators under Independent Coupling
Fix t ∈ ( 0 , 1 ) t \in(0,1) t ∈ ( 0 , 1 ) with h t > 0 h_t > 0 h t > 0 . Suppose X 1 ∼ q n X_1 \sim q_n X 1 ∼ q n , X 0 ∼ p 0 X_0 \sim p_0 X 0 ∼ p 0 and Z ∼ N ( 0 , I d ) Z \sim\mathcal{N}(0,I_d) Z ∼ N ( 0 , I d ) are independent, and write s q n = ∇ log q n s_{q_n} = \nabla\log q_n s q n = ∇ log q n and s π = ∇ log π s_\pi= \nabla\log\pi s π = ∇ log π . Let ϕ \phi ϕ be the standard Gaussian density on R d \mathbb{R}^d R d , and set
Y = t X 1 + ( 1 − t ) X 0 + h t Z , m ˉ ( y ) = E [ X 1 ∣ Y = y ] , (24) Y = tX_1 + (1-t)X_0 + h_t Z, \qquad\bar{m}(y) = \mathbb{E}[X_1 \mid Y=y], \tag*{(24)} Y = t X 1 + ( 1 − t ) X 0 + h t Z , m ˉ ( y ) = E [ X 1 ∣ Y = y ] , (24)
F ( y , x ) = f θ ( y ) ⋅ ( x − m ˉ ( y ) ) , a ( x ) = h t 2 t s π ( x ) . (25) F(y,x) = f_\theta(y) \cdot(x-\bar{m}(y)), \qquad a(x) = \frac{h_t}{2t}s_\pi(x). \tag*{(25)} F ( y , x ) = f θ ( y ) ⋅ ( x − m ˉ ( y )) , a ( x ) = 2 t h t s π ( x ) . (25)
Here f θ : R d → R d f_\theta: \mathbb{R}^d \to\mathbb{R}^d f θ : R d → R d is the vector field being fitted. Using the main-text definition of C t [ q n ] \mathcal{C}_t[q_n] C t [ q n ] , we estimate its expected inner product with f θ f_\theta f θ :
C t [ f θ ] : = E [ f θ ( Y ) ⋅ C t [ q n ] ( Y ) ] = t h t E [ f θ ( Y ) ⋅ Cov ( X 1 , Z ⋅ ( s π ( X 1 ) − s q n ( X 1 ) ) ∣ Y ) ] . (26) \begin{aligned}
\mathcal{C}_t[f_\theta] := \mathbb{E}[f_\theta(Y) \cdot\mathcal{C}_t[q_n](Y)] \\
&= \frac{t}{h_t}\mathbb{E}\left[f_\theta(Y) \cdot\operatorname{Cov}(X_1,Z \cdot(s_\pi(X_1)-s_{q_n}(X_1)) \mid Y)\right]. \tag*{(26)}
\end{aligned} C t [ f θ ] := E [ f θ ( Y ) ⋅ C t [ q n ] ( Y )] = h t t E [ f θ ( Y ) ⋅ Cov ( X 1 , Z ⋅ ( s π ( X 1 ) − s q n ( X 1 )) ∣ Y ) ] . (26)
All expectations below average over the sampled variables unless conditioning is shown. We assume sufficient differentiability, absolute integrability of the expectations involved, and vanishing boundary terms in integration by parts. The mean m m m and the covariance are defined under the original Gaussian bridge law.
Theorem A.7 (Unbiased covariance estimator). The function,
ϕ ~ ( z ∣ x ) = ∥ z + a ( x ) ∥ 2 d + ∥ a ( x ) ∥ 2 ϕ ( z ) , (27) \widetilde{\phi}(z \mid x) = \frac{\lVert z+a(x)\rVert^2}{d+\lVert a(x)\rVert^2}\phi(z), \tag*{(27)} ϕ ( z ∣ x ) = d + ∥ a ( x ) ∥ 2 ∥ z + a ( x ) ∥ 2 ϕ ( z ) , (27) is a probability density. For any coupling of ( Z , Z ~ ) (Z,\widetilde{Z}) ( Z , Z ) with conditional marginals ϕ \phi ϕ and ϕ ~ ( ⋅ ∣ X 1 ) \widetilde{\phi}(\cdot\mid X_1) ϕ ( ⋅ ∣ X 1 ) given ( X 0 , X 1 ) (X_0,X_1) ( X 0 , X 1 ) , set Y ~ = t X 1 + ( 1 − t ) X 0 + h t Z ~ \widetilde{Y}=tX_1+(1-t)X_0+h_t\widetilde{Z} Y = t X 1 + ( 1 − t ) X 0 + h t Z . Then,
C ^ t [ f θ ] = t h t f θ ( Y ) ⋅ Z + t 2 h t 2 ( d + ∥ a ( X 1 ) ∥ 2 ) [ F ( Y ~ , X 1 ) − F ( Y , X 1 ) ] (28) \widehat{\mathcal{C}}_t[f_\theta] = \frac{t}{h_t}f_\theta(Y)\cdot Z + \frac{t^2}{h_t^2}(d+\lVert a(X_1)\rVert^2)[F(\widetilde{Y},X_1)-F(Y,X_1)] \tag*{(28)} C t [ f θ ] = h t t f θ ( Y ) ⋅ Z + h t 2 t 2 ( d + ∥ a ( X 1 ) ∥ 2 ) [ F ( Y , X 1 ) − F ( Y , X 1 )] (28) satisfies E [ C ^ t [ f θ ] ] = C t [ f θ ] \mathbb{E}[\widehat{\mathcal{C}}_t[f_\theta]]=\mathcal{C}_t[f_\theta] E [ C t [ f θ ]] = C t [ f θ ] .
Proof. Expanding out the covariance term we get
C t [ f θ ] = t h t E [ f θ ( Y ) ⋅ ( X 1 − m ˉ ( Y ) ) ( Z ⋅ ( s π ( X 1 ) − s q n ( X 1 ) ) ) ] F ( Y , X 1 ) = t h t E [ F Z ⋅ s π ( X 1 ) ] − t h t E [ F Z ⋅ s q n ( X 1 ) ] . (29) \begin{aligned}
\mathcal{C}_t[f_\theta] = \frac{t}{h_t}\mathbb{E}\left[f_\theta(Y)\cdot(X_1-\bar{m}(Y))\left(Z\cdot(s_\pi(X_1)-s_{q_n}(X_1))\right)\right]_{F(Y,X_1)} \\
&= \frac{t}{h_t}\mathbb{E}[FZ\cdot s_\pi(X_1)]-\frac{t}{h_t}\mathbb{E}[FZ\cdot s_{q_n}(X_1)]. \tag*{(29)}
\end{aligned} C t [ f θ ] = h t t E [ f θ ( Y ) ⋅ ( X 1 − m ˉ ( Y )) ( Z ⋅ ( s π ( X 1 ) − s q n ( X 1 )) ) ] F ( Y , X 1 ) = h t t E [ F Z ⋅ s π ( X 1 )] − h t t E [ F Z ⋅ s q n ( X 1 )] . (29)
For the term involving the unknown endpoint score, hold X 0 X_0 X 0 , Z Z Z fixed and apply the vector Stein identity (Eq. (9) ) with r = q n r=q_n r = q n and scalar test function ξ ( x ) = F ( t x + ( 1 − t ) X 0 + h t Z , x ) \xi(x)=F(tx+(1-t)X_0+h_tZ,x) ξ ( x ) = F ( t x + ( 1 − t ) X 0 + h t Z , x ) . Taking the dot product with the fixed vector Z Z Z and averaging over X 0 , Z X_0,Z X 0 , Z gives the first equality below. Here ∇ X 1 \nabla_{X_1} ∇ X 1 includes the dependence through Y Y Y , while ∂ X 1 F ( Y , X 1 ) \partial_{X_1}F(Y,X_1) ∂ X 1 F ( Y , X 1 ) holds Y Y Y fixed.
E [ F ( Y , X 1 ) Z ⋅ s q n ( X 1 ) ] = − E [ ⟨ ∇ X 1 F ( Y , X 1 ) , Z ⟩ ] (30) \mathbb{E}[F(Y,X_1)Z\cdot s_{q_n}(X_1)] = -\mathbb{E}[\langle\nabla_{X_1}F(Y,X_1),Z\rangle] \tag*{(30)} E [ F ( Y , X 1 ) Z ⋅ s q n ( X 1 )] = − E [⟨ ∇ X 1 F ( Y , X 1 ) , Z ⟩] (30)
= − E [ ⟨ ∂ X 1 F ( Y , X 1 ) + t ∂ Y F ( Y , X 1 ) , Z ⟩ ] (31) = -\mathbb{E}[\langle\partial_{X_1}F(Y,X_1)+t\partial_YF(Y,X_1),Z\rangle] \tag*{(31)} = − E [⟨ ∂ X 1 F ( Y , X 1 ) + t ∂ Y F ( Y , X 1 ) , Z ⟩] (31)
= − E [ ⟨ f θ ( Y ) + t ∂ Y [ f θ ( Y ) ⋅ ( X 1 − m ˉ ( Y ) ) ] , Z ⟩ ] (32) = -\mathbb{E}[\langle f_\theta(Y)+t\partial_Y[f_\theta(Y)\cdot(X_1-\bar{m}(Y))],Z\rangle] \tag*{(32)} = − E [⟨ f θ ( Y ) + t ∂ Y [ f θ ( Y ) ⋅ ( X 1 − m ˉ ( Y ))] , Z ⟩] (32)
= − E [ ⟨ f θ ( Y ) , Z ⟩ + t ⟨ ∇ Y [ f θ ( Y ) ⋅ ( X 1 − m ˉ ( Y ) ) ] , Z ⟩ ] (33) = -\mathbb{E}[\langle f_\theta(Y),Z\rangle
+ t\langle\nabla_Y[f_\theta(Y)\cdot(X_1-\bar{m}(Y))],Z\rangle] \tag*{(33)} = − E [⟨ f θ ( Y ) , Z ⟩ + t ⟨ ∇ Y [ f θ ( Y ) ⋅ ( X 1 − m ˉ ( Y ))] , Z ⟩] (33)
= − E [ ⟨ f θ ( Y ) , Z ⟩ (34) = -\mathbb{E}[\langle f_\theta(Y),Z\rangle \tag*{(34)} = − E [⟨ f θ ( Y ) , Z ⟩ (34)
⟨ ∇ Z [ F ( Y , X 1 ) ] , Z ⟩ ] using ∇ Z F = h t ∇ Y F (35) \langle\nabla_Z[F(Y,X_1)],Z\rangle] \quad\text{using } \nabla_ZF=h_t\nabla_YF \tag*{(35)} ⟨ ∇ Z [ F ( Y , X 1 )] , Z ⟩] using ∇ Z F = h t ∇ Y F (35)
= − E [ ⟨ f θ ( Y ) , Z ⟩ + t h t ⟨ ∇ Z [ F ( Y , X 1 ) ] , Z ⟩ ] (36) = -\mathbb{E}[\langle f_\theta(Y),Z\rangle
+ \frac{t}{h_t}\langle\nabla_Z[F(Y,X_1)],Z\rangle] \tag*{(36)} = − E [⟨ f θ ( Y ) , Z ⟩ + h t t ⟨ ∇ Z [ F ( Y , X 1 )] , Z ⟩] (36)
For the next step, hold X 0 X_0 X 0 , X 1 X_1 X 1 fixed and write F ( Z ) F(Z) F ( Z ) for F ( Y ( t , X 1 , X 0 , Z ) , X 1 ) F(Y(t,X_1,X_0,Z),X_1) F ( Y ( t , X 1 , X 0 , Z ) , X 1 ) . Apply the Gaussian vector Stein identity (Eq. (23) ) with scalar test function ξ ( z ) = z i F ( z ) \xi(z)=z_iF(z) ξ ( z ) = z i F ( z ) , and take its i i i th component to obtain
E [ Z i 2 F ( Z ) ] = E [ ∇ Z i [ Z i F ( Z ) ] ] (37) \mathbb{E}[Z_i^2F(Z)]=\mathbb{E}[\nabla_{Z_i}[Z_iF(Z)]] \tag*{(37)} E [ Z i 2 F ( Z )] = E [ ∇ Z i [ Z i F ( Z )]] (37)
= E [ F ( Z ) + Z i ∇ Z i F ( Z ) ] (38) =\mathbb{E}[F(Z)+Z_i\nabla_{Z_i}F(Z)] \tag*{(38)} = E [ F ( Z ) + Z i ∇ Z i F ( Z )] (38)
Rearranging and summing over i = 1 , … , d i=1,\ldots,d i = 1 , … , d yields
E [ ⟨ Z , ∇ Z F ⟩ ] = E [ ( ∥ Z ∥ 2 − d ) F ( Z ) ] (39) \mathbb{E}[\langle Z,\nabla_ZF\rangle]=\mathbb{E}[(\lVert Z\rVert^2-d)F(Z)] \tag*{(39)} E [⟨ Z , ∇ Z F ⟩] = E [(∥ Z ∥ 2 − d ) F ( Z )] (39)
Substituting this above we get
E [ F ( Y , X 1 ) Z ⋅ s q n ( X 1 ) ] = − E [ ⟨ f θ ( Y ) , Z ⟩ (40) \mathbb{E}[F(Y,X_1)Z\cdot s_{q_n}(X_1)]=-\mathbb{E}[\langle f_\theta(Y),Z\rangle \tag*{(40)} E [ F ( Y , X 1 ) Z ⋅ s q n ( X 1 )] = − E [⟨ f θ ( Y ) , Z ⟩ (40)
+ t h t ( ∥ Z ∥ 2 − d ) F ( Y , X 1 ) ] (41) +\frac{t}{h_t}(\lVert Z\rVert^2-d)F(Y,X_1)] \tag*{(41)} + h t t (∥ Z ∥ 2 − d ) F ( Y , X 1 )] (41)
Therefore, combining with the target-score term gives
C t [ f θ ] = t h t E [ f θ ( Y ) ⋅ Z ] + t h t E [ F ( Y , X 1 ) ( Z ⋅ s π ( X 1 ) + t h t ( ∥ Z ∥ 2 − d ) ) ] . \mathcal{C}_t[f_\theta]=\frac{t}{h_t}\mathbb{E}[f_\theta(Y)\cdot Z]
+\frac{t}{h_t}\mathbb{E}\left[F(Y,X_1)\left(Z\cdot s_\pi(X_1)+\frac{t}{h_t}(\lVert Z\rVert^2-d)\right)\right]. C t [ f θ ] = h t t E [ f θ ( Y ) ⋅ Z ] + h t t E [ F ( Y , X 1 ) ( Z ⋅ s π ( X 1 ) + h t t (∥ Z ∥ 2 − d ) ) ] .
For t > 0 t>0 t > 0 and h t > 0 h_t>0 h t > 0 , write s = s π ( X 1 ) s=s_\pi(X_1) s = s π ( X 1 ) . Completing the square gives
Z ⋅ s + t h t ( ∥ Z ∥ 2 − d ) = t h t ∥ Z + h t 2 t s ∥ 2 − h t 4 t ∥ s ∥ 2 − t h t d (42) Z\cdot s+\frac{t}{h_t}(\lVert Z\rVert^2-d)=\frac{t}{h_t}\left\lVert Z+\frac{h_t}{2t}s\right\rVert^2-\frac{h_t}{4t}\lVert s\rVert^2-\frac{t}{h_t}d \tag*{(42)} Z ⋅ s + h t t (∥ Z ∥ 2 − d ) = h t t Z + 2 t h t s 2 − 4 t h t ∥ s ∥ 2 − h t t d (42)
Substituting this above gives
C t [ f θ ] = t h t E [ f θ ( Y ) ⋅ Z ] + t 2 h t 2 E [ F ( Y , X 1 ) ∥ Z + h t 2 t s π ( X 1 ) ∥ 2 ] − 1 4 E [ F ( Y , X 1 ) ∥ s π ( X 1 ) ∥ 2 ] − t 2 d h t 2 E [ F ( Y , X 1 ) ] . (43) \mathcal{C}_t[f_\theta]=\frac{t}{h_t}\mathbb{E}[f_\theta(Y)\cdot Z]
+\frac{t^2}{h_t^2}\mathbb{E}\left[F(Y,X_1)\left\lVert Z+\frac{h_t}{2t}s_\pi(X_1)\right\rVert^2\right]
-\frac{1}{4}\mathbb{E}[F(Y,X_1)\lVert s_\pi(X_1)\rVert^2]-\frac{t^2d}{h_t^2}\mathbb{E}[F(Y,X_1)]. \tag*{(43)} C t [ f θ ] = h t t E [ f θ ( Y ) ⋅ Z ] + h t 2 t 2 E [ F ( Y , X 1 ) Z + 2 t h t s π ( X 1 ) 2 ] − 4 1 E [ F ( Y , X 1 ) ∥ s π ( X 1 ) ∥ 2 ] − h t 2 t 2 d E [ F ( Y , X 1 )] . (43)
Change of measure step. Let ϕ \phi ϕ be the standard Gaussian density on R d \mathbb{R}^d R d . For a fixed vector a a a ,
E Z ∼ ϕ [ ∥ Z + a ∥ 2 ] = d + ∥ a ∥ 2 (44) \mathbb{E}_{Z\sim\phi}\left[\lVert Z+a\rVert^2\right]=d+\lVert a\rVert^2 \tag*{(44)} E Z ∼ ϕ [ ∥ Z + a ∥ 2 ] = d + ∥ a ∥ 2 (44)
Now fix X 0 = x 0 X_0=x_0 X 0 = x 0 and X 1 = x 1 X_1=x_1 X 1 = x 1 , and set
a ( x 1 ) = h t 2 t s π ( x 1 ) , ϕ ~ ( z ∣ x 1 ) = ∥ z + a ( x 1 ) ∥ 2 d + ∥ a ( x 1 ) ∥ 2 ϕ ( z ) . (45) a(x_1)=\frac{h_t}{2t}s_\pi(x_1),\qquad\widetilde{\phi}(z\mid x_1)=\frac{\lVert z+a(x_1)\rVert^2}{d+\lVert a(x_1)\rVert^2}\phi(z). \tag*{(45)} a ( x 1 ) = 2 t h t s π ( x 1 ) , ϕ ( z ∣ x 1 ) = d + ∥ a ( x 1 ) ∥ 2 ∥ z + a ( x 1 ) ∥ 2 ϕ ( z ) . (45)
The preceding Gaussian identity shows that ϕ ~ ( ⋅ ∣ x 1 ) \widetilde{\phi}(\cdot\mid x_1) ϕ ( ⋅ ∣ x 1 ) is a normalized probability density. Draw Z ~ ∣ X 1 ∼ ϕ ~ ( ⋅ ∣ X 1 ) \widetilde{Z}\mid X_1\sim\widetilde{\phi}(\cdot\mid X_1) Z ∣ X 1 ∼ ϕ ( ⋅ ∣ X 1 ) and define
Y ~ = t X 1 + ( 1 − t ) X 0 + h t Z ~ . (46) \widetilde{Y}=tX_1+(1-t)X_0+h_t\widetilde{Z}. \tag*{(46)} Y = t X 1 + ( 1 − t ) X 0 + h t Z . (46)
The endpoint variables retain their original independent laws X 1 ∼ q n X_1\sim q_n X 1 ∼ q n and X 0 ∼ p 0 X_0\sim p_0 X 0 ∼ p 0 ; only the conditional noise law changes. The change of measure and the law of iterated expectation give
E [ F ( Y , X 1 ) ∥ Z + h t 2 t s π ( X 1 ) ∥ 2 ] = E X 0 , X 1 [ ( d + ∥ a ( X 1 ) ∥ 2 ) E Z ~ ∼ ϕ ~ ( ⋅ ∣ X 1 ) [ F ( Y ~ , X 1 ) ] ] = E [ ( d + h t 2 4 t 2 ∥ s π ( X 1 ) ∥ 2 ) F ( Y ~ , X 1 ) ] . (47) \begin{aligned}
\mathbb{E}\left[F(Y,X_1)\left\lVert Z+\frac{h_t}{2t}s_\pi(X_1)\right\rVert^2\right]
=\mathbb{E}_{X_0,X_1}\left[(d+\lVert a(X_1)\rVert^2)\mathbb{E}_{\widetilde{Z}\sim\widetilde{\phi}(\cdot\mid X_1)}[F(\widetilde{Y},X_1)]\right] \\
&=\mathbb{E}\left[\left(d+\frac{h_t^2}{4t^2}\lVert s_\pi(X_1)\rVert^2\right)F(\widetilde{Y},X_1)\right]. \tag*{(47)}
\end{aligned} E [ F ( Y , X 1 ) Z + 2 t h t s π ( X 1 ) 2 ] = E X 0 , X 1 [ ( d + ∥ a ( X 1 ) ∥ 2 ) E Z ∼ ϕ ( ⋅ ∣ X 1 ) [ F ( Y , X 1 )] ] = E [ ( d + 4 t 2 h t 2 ∥ s π ( X 1 ) ∥ 2 ) F ( Y , X 1 ) ] . (47)
The two subtracted terms from completing the square combine as
− 1 4 E [ F ( Y , X 1 ) ∥ s π ( X 1 ) ∥ 2 ] − t 2 d h t 2 E [ F ( Y , X 1 ) ] = − t 2 h t 2 E [ ( d + ∥ a ( X 1 ) ∥ 2 ) F ( Y , X 1 ) ] . (48) -\frac{1}{4}\mathbb{E}\left[F(Y,X_1)\lVert s_{\pi}(X_1)\rVert^2\right]-\frac{t^2d}{h_t^2}\mathbb{E}[F(Y,X_1)]
= -\frac{t^2}{h_t^2}\mathbb{E}\left[(d+\lVert a(X_1)\rVert^2)F(Y,X_1)\right]. \tag*{(48)} − 4 1 E [ F ( Y , X 1 ) ∥ s π ( X 1 ) ∥ 2 ] − h t 2 t 2 d E [ F ( Y , X 1 )] = − h t 2 t 2 E [ ( d + ∥ a ( X 1 ) ∥ 2 ) F ( Y , X 1 ) ] . (48)
Substituting both identities into the covariance expression therefore yields
C t [ f θ ] = t h t E [ f θ ( Y ) ⋅ Z ] + t 2 h t 2 E [ ( d + ∥ a ( X 1 ) ∥ 2 ) F ( Y ~ , X 1 ) ] − t 2 h t 2 E [ ( d + ∥ a ( X 1 ) ∥ 2 ) F ( Y , X 1 ) ] = t h t E [ f θ ( Y ) ⋅ Z ] + t 2 h t 2 E [ ( d + ∥ a ( X 1 ) ∥ 2 ) ( F ( Y ~ , X 1 ) − F ( Y , X 1 ) ) ] . (49) \begin{aligned}
C_t[f_{\theta}] = \frac{t}{h_t}\mathbb{E}[f_{\theta}(Y)\cdot Z]
+\frac{t^2}{h_t^2}\mathbb{E}\left[(d+\lVert a(X_1)\rVert^2)F(\widetilde{Y},X_1)\right]-\frac{t^2}{h_t^2}\mathbb{E}\left[(d+\lVert a(X_1)\rVert^2)F(Y,X_1)\right] \\
&= \frac{t}{h_t}\mathbb{E}[f_{\theta}(Y)\cdot Z]
+\frac{t^2}{h_t^2}\mathbb{E}\left[(d+\lVert a(X_1)\rVert^2)\left(F(\widetilde{Y},X_1)-F(Y,X_1)\right)\right]. \tag*{(49)}
\end{aligned} C t [ f θ ] = h t t E [ f θ ( Y ) ⋅ Z ] + h t 2 t 2 E [ ( d + ∥ a ( X 1 ) ∥ 2 ) F ( Y , X 1 ) ] − h t 2 t 2 E [ ( d + ∥ a ( X 1 ) ∥ 2 ) F ( Y , X 1 ) ] = h t t E [ f θ ( Y ) ⋅ Z ] + h t 2 t 2 E [ ( d + ∥ a ( X 1 ) ∥ 2 ) ( F ( Y , X 1 ) − F ( Y , X 1 ) ) ] . (49)
This is exactly E [ C ^ t [ f θ ] ] \mathbb{E}[\widehat{C}_t[f_{\theta}]] E [ C t [ f θ ]] for the estimator in Eq. (28) . Only the conditional noise marginals enter the two expectations, so the identity holds for any coupling with those marginals. □ \square □
Their coupling affects variance but not the expectation. It remains to draw from the tilted density. The following lemma gives an exact sampler.
Lemma A.8 (Sampling the tilted density). Fix a ∈ R d a \in\mathbb{R}^d a ∈ R d . Draw independent G ∼ N ( 0 , I d ) G \sim\mathcal{N}(0,I_d) G ∼ N ( 0 , I d ) and H 1 , H 2 ∼ N ( 0 , 1 ) H_1,H_2 \sim\mathcal{N}(0,1) H 1 , H 2 ∼ N ( 0 , 1 ) . Independently choose
V = { G , with probability ∥ a ∥ 2 d + ∥ a ∥ 2 , 1 + H 1 2 + H 2 2 ∥ G ∥ 2 G , otherwise , (50) V =
\begin{cases}
G, & \text{with probability } \dfrac{\lVert a\rVert^2}{d+\lVert a\rVert^2},\\
\sqrt{1+\dfrac{H_1^2+H_2^2}{\lVert G\rVert^2}}\,G, & \text{otherwise},
\end{cases} \tag*{(50)} V = ⎩ ⎨ ⎧ G , 1 + ∥ G ∥ 2 H 1 2 + H 2 2 G , with probability d + ∥ a ∥ 2 ∥ a ∥ 2 , otherwise , (50) then set
Z ~ = { V , with probability 1 2 + V ⋅ a ∥ V ∥ 2 + ∥ a ∥ 2 , − V , otherwise . (51) \widetilde{Z} =
\begin{cases}
V, & \text{with probability } \dfrac{1}{2}+\dfrac{V\cdot a}{\lVert V\rVert^2+\lVert a\rVert^2},\\
-V, & \text{otherwise}.
\end{cases} \tag*{(51)} Z = ⎩ ⎨ ⎧ V , − V , with probability 2 1 + ∥ V ∥ 2 + ∥ a ∥ 2 V ⋅ a , otherwise . (51) Then Z ~ \widetilde{Z} Z has density φ ~ ( z ∣ a ) = ∥ z + a ∥ 2 φ ( z ) / ( d + ∥ a ∥ 2 ) \widetilde{\varphi}(z\mid a)=\lVert z+a\rVert^2\varphi(z)/(d+\lVert a\rVert^2) φ ( z ∣ a ) = ∥ z + a ∥ 2 φ ( z ) / ( d + ∥ a ∥ 2 ) .
Proof. Start by choosing a pair { z , − z } \{z,-z\} { z , − z } using the symmetrized density below. Then conditional on { z , − z } \{z,-z\} { z , − z } choose one of these, this corresponds to:
P ( choose z ∣ ± z ) = φ ~ ( z ) φ ~ ( z ) + φ ~ ( − z ) (52) P(\text{choose }z\mid\pm z)=\frac{\widetilde{\varphi}(z)}{\widetilde{\varphi}(z)+\widetilde{\varphi}(-z)} \tag*{(52)} P ( choose z ∣ ± z ) = φ ( z ) + φ ( − z ) φ ( z ) (52)
= ∥ z + a ∥ 2 ∥ z + a ∥ 2 + ∥ z − a ∥ 2 (53) =\frac{\lVert z+a\rVert^2}{\lVert z+a\rVert^2+\lVert z-a\rVert^2} \tag*{(53)} = ∥ z + a ∥ 2 + ∥ z − a ∥ 2 ∥ z + a ∥ 2 (53)
= 1 2 + z ⋅ a ∥ z ∥ 2 + ∥ a ∥ 2 (54) =\frac{1}{2}+\frac{z\cdot a}{\lVert z\rVert^2+\lVert a\rVert^2} \tag*{(54)} = 2 1 + ∥ z ∥ 2 + ∥ a ∥ 2 z ⋅ a (54)
1 2 ( φ ~ ( z ∣ a ) + φ ~ ( − z ∣ a ) ) = ∥ z ∥ 2 + ∥ a ∥ 2 d + ∥ a ∥ 2 φ ( z ) (55) \frac{1}{2}\left(\widetilde{\varphi}(z\mid a)+\widetilde{\varphi}(-z\mid a)\right)
=\frac{\lVert z\rVert^2+\lVert a\rVert^2}{d+\lVert a\rVert^2}\varphi(z) \tag*{(55)} 2 1 ( φ ( z ∣ a ) + φ ( − z ∣ a ) ) = d + ∥ a ∥ 2 ∥ z ∥ 2 + ∥ a ∥ 2 φ ( z ) (55)
Now consider the symmetrized density s ( z ∣ a ) = 1 2 ( φ ~ ( z ∣ a ) + φ ~ ( − z ∣ a ) ) s(z\mid a)=\frac{1}{2}\left(\widetilde{\varphi}(z\mid a)+\widetilde{\varphi}(-z\mid a)\right) s ( z ∣ a ) = 2 1 ( φ ( z ∣ a ) + φ ( − z ∣ a ) ) . Note that
s ( z ∣ a ) P ( choose z ∣ ± z ) = φ ~ ( z ∣ a ) φ ~ ( z ∣ a ) + φ ~ ( − z ∣ a ) ⋅ 1 2 ( φ ~ ( z ∣ a ) + φ ~ ( − z ∣ a ) ) = 1 2 φ ~ ( z ∣ a ) . (56) s(z\mid a)P(\text{choose }z\mid\pm z)
=\frac{\widetilde{\varphi}(z\mid a)}{\widetilde{\varphi}(z\mid a)+\widetilde{\varphi}(-z\mid a)}
\cdot\frac{1}{2}\left(\widetilde{\varphi}(z\mid a)+\widetilde{\varphi}(-z\mid a)\right)
=\frac{1}{2}\widetilde{\varphi}(z\mid a). \tag*{(56)} s ( z ∣ a ) P ( choose z ∣ ± z ) = φ ( z ∣ a ) + φ ( − z ∣ a ) φ ( z ∣ a ) ⋅ 2 1 ( φ ( z ∣ a ) + φ ( − z ∣ a ) ) = 2 1 φ ( z ∣ a ) . (56)
But we can also output z z z by first drawing − z -z − z and choosing its opposite. Adding both contributions, the output density at z z z is
s ( z ∣ a ) P ( choose z ∣ ± z ) + s ( − z ∣ a ) P ( choose z ∣ ± z ) = 1 2 ϕ ~ ( z ∣ a ) + 1 2 ϕ ~ ( z ∣ a ) = ϕ ~ ( z ∣ a ) . (57) s(z \mid a)P(\text{choose }z\mid\pm z)+s(-z\mid a)P(\text{choose }z\mid\pm z)
=\frac{1}{2}\widetilde{\phi}(z\mid a)+\frac{1}{2}\widetilde{\phi}(z\mid a)=\widetilde{\phi}(z\mid a). \tag*{(57)} s ( z ∣ a ) P ( choose z ∣ ± z ) + s ( − z ∣ a ) P ( choose z ∣ ± z ) = 2 1 ϕ ( z ∣ a ) + 2 1 ϕ ( z ∣ a ) = ϕ ( z ∣ a ) . (57)
Thus, it suffices to sample from the symmetrized density and then choose the sign with the probability above. For fixed a a a , write g ( z ) = s ( z ∣ a ) g(z)=s(z\mid a) g ( z ) = s ( z ∣ a ) , so
g ( z ) = ∥ z ∥ 2 + ∥ a ∥ 2 d + ∥ a ∥ 2 ϕ ( z ) (58) g(z)=\frac{\lVert z\rVert^2+\lVert a\rVert^2}{d+\lVert a\rVert^2}\phi(z) \tag*{(58)} g ( z ) = d + ∥ a ∥ 2 ∥ z ∥ 2 + ∥ a ∥ 2 ϕ ( z ) (58)
= ∥ a ∥ 2 ∥ a ∥ 2 + d ϕ ( z ) + d ∥ a ∥ 2 + d ∥ z ∥ 2 d ϕ ( z ) . (59) =\frac{\lVert a\rVert^2}{\lVert a\rVert^2+d}\phi(z)+\frac{d}{\lVert a\rVert^2+d}\frac{\lVert z\rVert^2}{d}\phi(z). \tag*{(59)} = ∥ a ∥ 2 + d ∥ a ∥ 2 ϕ ( z ) + ∥ a ∥ 2 + d d d ∥ z ∥ 2 ϕ ( z ) . (59)
This is just a mixture of two densities ϕ ( z ) \phi(z) ϕ ( z ) and d − 1 ∥ z ∥ 2 ϕ ( z ) d^{-1}\lVert z\rVert^2\phi(z) d − 1 ∥ z ∥ 2 ϕ ( z ) . The second is normalized since E ϕ [ ∥ Z ∥ 2 ] = d \mathbb{E}_{\phi}[\lVert Z\rVert^2]=d E ϕ [∥ Z ∥ 2 ] = d . Multiplying the Gaussian density by ∥ z ∥ 2 \lVert z\rVert^2 ∥ z ∥ 2 changes its radial density from one proportional to r d − 1 e − r 2 / 2 r^{d-1}e^{-r^2/2} r d − 1 e − r 2 /2 to one proportional to r d + 1 e − r 2 / 2 r^{d+1}e^{-r^2/2} r d + 1 e − r 2 /2 . This is just the Gaussian radius in d + 2 d+2 d + 2 dimensions.
Recall that sampling from a Gaussian may be done by sampling the radius with density proportional to r d − 1 e − r 2 / 2 r^{d-1}e^{-r^2/2} r d − 1 e − r 2 /2 and independently drawing a direction from U ( S d − 1 ) \mathcal{U}(S^{d-1}) U ( S d − 1 ) , namely
G = R U , U ∼ U ( S d − 1 ) , R ⊥ U , (60) G=RU,\qquad U\sim\mathcal{U}(S^{d-1}),\qquad R\perp U, \tag*{(60)} G = R U , U ∼ U ( S d − 1 ) , R ⊥ U , (60)
p R ( r ) ∝ r d − 1 e − r 2 / 2 , r > 0. (61) p_R(r)\propto r^{d-1}e^{-r^2/2},\qquad r>0. \tag*{(61)} p R ( r ) ∝ r d − 1 e − r 2 /2 , r > 0. (61)
Equivalently, given G ∼ N ( 0 , I d ) G\sim\mathcal{N}(0,I_d) G ∼ N ( 0 , I d ) , take R = ∥ G ∥ R=\lVert G\rVert R = ∥ G ∥ and U = G / ∥ G ∥ U=G/\lVert G\rVert U = G / ∥ G ∥ . The Gaussian density depends only on the radius, and the volume element in polar coordinates contributes the factor r d − 1 r^{d-1} r d − 1 . This is why the direction is uniform and independent of the radius.
Multiplying the Gaussian density by ∥ z ∥ 2 \lVert z\rVert^2 ∥ z ∥ 2 changes only its radial density. So to sample from a density proportional to ∥ z ∥ 2 ϕ ( z ) \lVert z\rVert^2\phi(z) ∥ z ∥ 2 ϕ ( z ) , draw two independent one dimensional standard normals H 1 , H 2 H_1,H_2 H 1 , H 2 , also independent of G G G , and do
R ~ = G 1 2 + ⋯ + G d 2 + H 1 2 + H 2 2 , H 1 , H 2 ∼ N ( 0 , 1 ) , (62) \widetilde{R}=\sqrt{G_1^2+\cdots+G_d^2+H_1^2+H_2^2},\qquad H_1,H_2\sim\mathcal{N}(0,1), \tag*{(62)} R = G 1 2 + ⋯ + G d 2 + H 1 2 + H 2 2 , H 1 , H 2 ∼ N ( 0 , 1 ) , (62)
G ~ = R ~ U = 1 + H 1 2 + H 2 2 ∥ G ∥ 2 G . (63) \widetilde{G}=\widetilde{R}U=\sqrt{1+\frac{H_1^2+H_2^2}{\lVert G\rVert^2}}G. \tag*{(63)} G = R U = 1 + ∥ G ∥ 2 H 1 2 + H 2 2 G . (63)
Now R ~ 2 \widetilde{R}^2 R 2 is the sum of d + 2 d+2 d + 2 independent squared standard normals, so R ~ \widetilde{R} R has the required radial density. Also, U U U is independent of ( R , H 1 , H 2 ) (R,H_1,H_2) ( R , H 1 , H 2 ) and hence of R ~ \widetilde{R} R . Thus G ~ \widetilde{G} G has density d − 1 ∥ z ∥ 2 ϕ ( z ) d^{-1}\lVert z\rVert^2\phi(z) d − 1 ∥ z ∥ 2 ϕ ( z ) . It is still a vector in R d \mathbb{R}^d R d ; the extra two scalars change only its length.
Thus (50) draws V V V from s ( ⋅ ∣ a ) s(\cdot\mid a) s ( ⋅ ∣ a ) . The sign selection in (51) then gives a draw from ϕ ~ ( ⋅ ∣ a ) \widetilde{\phi}(\cdot\mid a) ϕ ( ⋅ ∣ a ) , as shown above.
For the estimator, take a = a ( X 1 ) a=a(X_1) a = a ( X 1 ) . Using Z = G Z=G Z = G for the original Gaussian noise and this Z ~ \widetilde{Z} Z for the tilted noise gives a coupling with the required marginals. We can therefore substitute Y = t X 1 + ( 1 − t ) X 0 + h t G Y=tX_1+(1-t)X_0+h_tG Y = t X 1 + ( 1 − t ) X 0 + h t G and Y ~ = t X 1 + ( 1 − t ) X 0 + h t Z ~ \widetilde{Y}=tX_1+(1-t)X_0+h_t\widetilde{Z} Y = t X 1 + ( 1 − t ) X 0 + h t Z into (28) . The construction is exact; it does not by itself guarantee lower variance.
Covariance Estimator under a General Coupling
Stein’s identity under a general coupling. Let G ∼ p 0 G\sim p_0 G ∼ p 0 , X 1 ∼ q n X_1\sim q_n X 1 ∼ q n , and Z ∼ N ( 0 , I d ) Z\sim\mathcal{N}(0,I_d) Z ∼ N ( 0 , I d ) be independent. For fixed t ∈ ( 0 , 1 ) t\in(0,1) t ∈ ( 0 , 1 ) and h t > 0 h_t>0 h t > 0 , define
Y t = ( 1 − t ) A ( G , X 1 ) + t X 1 + h t Z , J t = t I + ( 1 − t ) D X 1 A ( G , X 1 ) . Y_t=(1-t)A(G,X_1)+tX_1+h_tZ,\qquad J_t=tI+(1-t)D_{X_1}A(G,X_1). Y t = ( 1 − t ) A ( G , X 1 ) + t X 1 + h t Z , J t = t I + ( 1 − t ) D X 1 A ( G , X 1 ) .
Write s q n = ∇ log q n s_{q_n}=\nabla\log q_n s q n = ∇ log q n and F ( y , x ) = f θ ( y ) ⋅ ( x − m ˉ ( y ) ) F(y,x)=f_\theta(y)\cdot(x-\bar{m}(y)) F ( y , x ) = f θ ( y ) ⋅ ( x − m ˉ ( y )) , where m ˉ ( y ) = E [ X 1 ∣ Y t = y ] \bar{m}(y)=\mathbb{E}[X_1\mid Y_t=y] m ˉ ( y ) = E [ X 1 ∣ Y t = y ] . We assume sufficient smoothness and integrability for the following integrations by parts, with vanishing boundary terms.
Holding G G G , Z Z Z fixed, we apply Stein’s identity in X 1 X_1 X 1 to the vector field F ( Y t , X 1 ) Z F(Y_t,X_1)Z F ( Y t , X 1 ) Z . The chain rule gives
∇ X 1 F ( Y t , X 1 ) = f θ ( Y t ) + J t ⊤ ∇ y F ( Y t , X 1 ) , \nabla_{X_1}F(Y_t,X_1)=f_\theta(Y_t)+J_t^{\top}\nabla_yF(Y_t,X_1), ∇ X 1 F ( Y t , X 1 ) = f θ ( Y t ) + J t ⊤ ∇ y F ( Y t , X 1 ) ,
where the first term comes from differentiating the explicit X 1 X_1 X 1 in F F F , and the second accounts for its dependence through Y t Y_t Y t . Therefore,
E [ F ( Y t , X 1 ) Z ⋅ s q n ( X 1 ) ] = − E [ Z ⋅ ∇ X 1 F ( Y t , X 1 ) ] = − E [ f θ ( Y t ) ⋅ Z ] − E [ ∇ y F ( Y t , X 1 ) ⋅ J t Z ] = − E [ f θ ( Y t ) ⋅ Z ] − 1 h t E [ ∇ Z F ( Y t , X 1 ) ⋅ J t Z ] , \begin{aligned}
\mathbb{E}[F(Y_t,X_1)Z\cdot s_{q_n}(X_1)]&=-\mathbb{E}[Z\cdot\nabla_{X_1}F(Y_t,X_1)]\\
&=-\mathbb{E}[f_\theta(Y_t)\cdot Z]-\mathbb{E}[\nabla_yF(Y_t,X_1)\cdot J_tZ]\\
&=-\mathbb{E}[f_\theta(Y_t)\cdot Z]-\frac{1}{h_t}\mathbb{E}[\nabla_ZF(Y_t,X_1)\cdot J_tZ],
\end{aligned} E [ F ( Y t , X 1 ) Z ⋅ s q n ( X 1 )] = − E [ Z ⋅ ∇ X 1 F ( Y t , X 1 )] = − E [ f θ ( Y t ) ⋅ Z ] − E [ ∇ y F ( Y t , X 1 ) ⋅ J t Z ] = − E [ f θ ( Y t ) ⋅ Z ] − h t 1 E [ ∇ Z F ( Y t , X 1 ) ⋅ J t Z ] ,
using ∇ Z F ( Y t , X 1 ) = h t ∇ y F ( Y t , X 1 ) \nabla_ZF(Y_t,X_1)=h_t\nabla_yF(Y_t,X_1) ∇ Z F ( Y t , X 1 ) = h t ∇ y F ( Y t , X 1 ) .
Next, hold G G G , X 1 X_1 X 1 fixed and apply Gaussian Stein’s identity to F ( Y t , X 1 ) J t Z F(Y_t,X_1)J_tZ F ( Y t , X 1 ) J t Z . Since J t J_t J t does not depend on Z Z Z , the product rule yields
E [ F ( Y t , X 1 ) Z ⊤ J t Z ] = E [ ∇ Z ⋅ ( F ( Y t , X 1 ) J t Z ) ] = E [ ∇ Z F ( Y t , X 1 ) ⋅ J t Z + F ( Y t , X 1 ) tr J t ] . \begin{aligned}
\mathbb{E}[F(Y_t,X_1)Z^{\top}J_tZ]&=\mathbb{E}[\nabla_Z\cdot(F(Y_t,X_1)J_tZ)]\\
&=\mathbb{E}[\nabla_ZF(Y_t,X_1)\cdot J_tZ+F(Y_t,X_1)\operatorname{tr}J_t].
\end{aligned} E [ F ( Y t , X 1 ) Z ⊤ J t Z ] = E [ ∇ Z ⋅ ( F ( Y t , X 1 ) J t Z )] = E [ ∇ Z F ( Y t , X 1 ) ⋅ J t Z + F ( Y t , X 1 ) tr J t ] .
Rearranging and substituting into the preceding expression, we obtain
E [ F ( Y t , X 1 ) Z ⋅ s q n ( X 1 ) ] = − E [ f θ ( Y t ) ⋅ Z ] − 1 h t E [ F ( Y t , X 1 ) ( Z ⊤ J t Z − tr J t ) ] . (64) \mathbb{E}[F(Y_t,X_1)Z\cdot s_{q_n}(X_1)]=-\mathbb{E}[f_\theta(Y_t)\cdot Z]-\frac{1}{h_t}\mathbb{E}[F(Y_t,X_1)(Z^{\top}J_tZ-\operatorname{tr}J_t)]. \tag*{(64)} E [ F ( Y t , X 1 ) Z ⋅ s q n ( X 1 )] = − E [ f θ ( Y t ) ⋅ Z ] − h t 1 E [ F ( Y t , X 1 ) ( Z ⊤ J t Z − tr J t )] . (64)
For independent coupling, A ( G , x ) = G A(G,x)=G A ( G , x ) = G and J t = t I J_t=tI J t = t I , recovering the factor t ( ∥ Z ∥ 2 − d ) t(\lVert Z\rVert^2-d) t (∥ Z ∥ 2 − d ) from the preceding proof.
Completing the square and changing measure. We retain the covariance convention from the main text:
C t [ f θ ] : = t h t E [ F ( Y t , X 1 ) Z ⋅ ( s π ( X 1 ) − s q n ( X 1 ) ) ] . (65) C_t[f_\theta]:=\frac{t}{h_t}\mathbb{E}[F(Y_t,X_1)Z\cdot(s_\pi(X_1)-s_{q_n}(X_1))]. \tag*{(65)} C t [ f θ ] := h t t E [ F ( Y t , X 1 ) Z ⋅ ( s π ( X 1 ) − s q n ( X 1 ))] . (65)
The following construction requires the symmetric part S t = ( J t + J t ⊤ ) / 2 S_t=(J_t+J_t^{\top})/2 S t = ( J t + J t ⊤ ) /2 to be positive definite almost surely, in addition to the integration-by-parts and integrability assumptions above. This holds for independent coupling and for nondegenerate Kabsch alignment (Corollary A.5). The mean m m m remains the conditional mean under the original Gaussian bridge.
Theorem A.9 (Covariance estimator under a general coupling). Define
a = h t 2 S t − 1 s π ( X 1 ) , κ = tr S t + a ⊤ S t a . a=\frac{h_t}{2}S_t^{-1}s_\pi(X_1),\qquad\kappa=\operatorname{tr}S_t+a^{\top}S_ta. a = 2 h t S t − 1 s π ( X 1 ) , κ = tr S t + a ⊤ S t a . Let ϕ \phi ϕ denote the standard Gaussian density. Conditional on G G G , X 1 X_1 X 1 , the function,
ϕ ~ ( z ∣ G , X 1 ) = ( z + a ) ⊤ S t ( z + a ) κ ϕ ( z ) , (66) \widetilde{\phi}(z\mid G,X_1)=\frac{(z+a)^{\top}S_t(z+a)}{\kappa}\phi(z), \tag*{(66)} ϕ ( z ∣ G , X 1 ) = κ ( z + a ) ⊤ S t ( z + a ) ϕ ( z ) , (66) is a probability density. For any coupling of Z , Z ~ Z,\widetilde{Z} Z , Z with conditional marginals ϕ , ϕ ~ \phi,\widetilde{\phi} ϕ , ϕ given G G G , X 1 X_1 X 1 , set
Y ~ t = ( 1 − t ) A ( G , X 1 ) + t X 1 + h t Z ~ . \widetilde{Y}_t=(1-t)A(G,X_1)+tX_1+h_t\widetilde{Z}. Y t = ( 1 − t ) A ( G , X 1 ) + t X 1 + h t Z . Then,
C ^ t [ f θ ] = t h t f θ ( Y t ) ⋅ Z + t κ h t 2 [ F ( Y ~ t , X 1 ) − F ( Y t , X 1 ) ] (67) \widehat{C}_t[f_\theta]=\frac{t}{h_t}f_\theta(Y_t)\cdot Z+\frac{t\kappa}{h_t^2}[F(\widetilde{Y}_t,X_1)-F(Y_t,X_1)] \tag*{(67)} C t [ f θ ] = h t t f θ ( Y t ) ⋅ Z + h t 2 t κ [ F ( Y t , X 1 ) − F ( Y t , X 1 )] (67) satisfies E [ C ^ t [ f θ ] ] = C t [ f θ ] \mathbb{E}[\widehat{C}_t[f_\theta]]=C_t[f_\theta] E [ C t [ f θ ]] = C t [ f θ ] .
Proof. Substituting Eq. (64) into Eq. (65) , and using Z ⊤ J t Z = Z ⊤ S t Z Z^{\top}J_tZ=Z^{\top}S_tZ Z ⊤ J t Z = Z ⊤ S t Z and tr J t = tr S t \operatorname{tr}J_t=\operatorname{tr}S_t tr J t = tr S t , gives
C t [ f θ ] = t h t E [ f θ ( Y t ) ⋅ Z ] + t h t 2 E [ F ( Y t , X 1 ) ( Z ⊤ S t Z + h t Z ⋅ s π ( X 1 ) − tr S t ) ] . (68) \begin{aligned}
C_t[f_\theta]&=\frac{t}{h_t}\mathbb{E}[f_\theta(Y_t)\cdot Z]\\
&\quad+\frac{t}{h_t^2}\mathbb{E}[F(Y_t,X_1)(Z^{\top}S_tZ+h_tZ\cdot s_\pi(X_1)-\operatorname{tr}S_t)]. \tag*{(68)}
\end{aligned} C t [ f θ ] = h t t E [ f θ ( Y t ) ⋅ Z ] + h t 2 t E [ F ( Y t , X 1 ) ( Z ⊤ S t Z + h t Z ⋅ s π ( X 1 ) − tr S t )] . (68)
Since 2 S t a = h t s π ( X 1 ) 2S_ta=h_ts_{\pi}(X_1) 2 S t a = h t s π ( X 1 ) , completing the square yields,
Z T S t Z + h t Z ⋅ s π ( X 1 ) − tr S t = ( Z + a ) T S t ( Z + a ) − κ . Z^{\mathsf T}S_tZ+h_tZ\cdot s_{\pi}(X_1)-\operatorname{tr}S_t=(Z+a)^{\mathsf T}S_t(Z+a)-\kappa. Z T S t Z + h t Z ⋅ s π ( X 1 ) − tr S t = ( Z + a ) T S t ( Z + a ) − κ .
Conditional on G G G , X 1 X_1 X 1 , the matrix S t S_t S t and vector a a a are fixed, and
E Z [ ( Z + a ) T S t ( Z + a ) ] = tr S t + a T S t a = κ . \mathbb{E}_Z[(Z+a)^{\mathsf T}S_t(Z+a)]=\operatorname{tr}S_t+a^{\mathsf T}S_ta=\kappa. E Z [( Z + a ) T S t ( Z + a )] = tr S t + a T S t a = κ .
Together with S t ≻ 0 S_t\succ0 S t ≻ 0 , this establishes that Eq. (66) is a probability density. Changing measure conditional on G G G , X 1 X_1 X 1 , then averaging, gives
E [ F ( Y t , X 1 ) ( Z + a ) T S t ( Z + a ) ] = E [ κ F ( Y ~ t , X 1 ) ] . \mathbb{E}\left[F(Y_t,X_1)(Z+a)^{\mathsf T}S_t(Z+a)\right]=\mathbb{E}\left[\kappa F(\widetilde{Y}_t,X_1)\right]. E [ F ( Y t , X 1 ) ( Z + a ) T S t ( Z + a ) ] = E [ κ F ( Y t , X 1 ) ] .
Combining these identities,
C t [ f θ ] = t h t E [ f θ ( Y t ) ⋅ Z ] + t h t 2 E [ κ ( F ( Y ~ t , X 1 ) − F ( Y t , X 1 ) ) ] , C_t[f_\theta]=\frac{t}{h_t}\mathbb{E}[f_\theta(Y_t)\cdot Z]
+\frac{t}{h_t^2}\mathbb{E}\left[\kappa\left(F(\widetilde{Y}_t,X_1)-F(Y_t,X_1)\right)\right], C t [ f θ ] = h t t E [ f θ ( Y t ) ⋅ Z ] + h t 2 t E [ κ ( F ( Y t , X 1 ) − F ( Y t , X 1 ) ) ] ,
which is the expectation of Eq. (67) .
Lemma A.10 (Sampling a quadratic Gaussian tilt). Fix S ≻ 0 S\succ0 S ≻ 0 and a ∈ R d a\in\mathbb{R}^d a ∈ R d , and write S = Q diag ( λ 1 , … , λ d ) Q T S=Q\operatorname{diag}(\lambda_1,\ldots,\lambda_d)Q^{\mathsf T} S = Q diag ( λ 1 , … , λ d ) Q T , b = Q T a b=Q^{\mathsf T}a b = Q T a , and κ = tr S + a T S a \kappa=\operatorname{tr}S+a^{\mathsf T}Sa κ = tr S + a T S a . Choose an index I = i I=i I = i with probability w i = λ i ( 1 + b i 2 ) / κ w_i=\lambda_i(1+b_i^2)/\kappa w i = λ i ( 1 + b i 2 ) / κ . Conditional on I = i I=i I = i , draw U i U_i U i from the one-dimensional density,
( u + b i ) 2 1 + b i 2 ϕ 1 ( u ) , \frac{(u+b_i)^2}{1+b_i^2}\phi_1(u), 1 + b i 2 ( u + b i ) 2 ϕ 1 ( u ) , using Lemma A.8 in dimension one. Draw the remaining coordinates independently from N ( 0 , 1 ) \mathcal{N}(0,1) N ( 0 , 1 ) and independently of U i U_i U i . Then Z ~ = Q U \widetilde{Z}=QU Z = Q U has density κ − 1 ( z + a ) T S ( z + a ) ϕ ( z ) \kappa^{-1}(z+a)^{\mathsf T}S(z+a)\phi(z) κ − 1 ( z + a ) T S ( z + a ) ϕ ( z ) .
Proof. The weights sum to one because ∑ i λ i ( 1 + b i 2 ) = κ \sum_i\lambda_i(1+b_i^2)=\kappa ∑ i λ i ( 1 + b i 2 ) = κ . The mixture density of U U U is
∑ i w i ( u i + b i ) 2 1 + b i 2 ∏ j ≠ i ϕ 1 ( u j ) = ∑ i λ i ( u i + b i ) 2 κ ϕ ( u ) . \sum_i w_i\frac{(u_i+b_i)^2}{1+b_i^2}\prod_{j\ne i}\phi_1(u_j)
=\frac{\sum_i\lambda_i(u_i+b_i)^2}{\kappa}\phi(u). i ∑ w i 1 + b i 2 ( u i + b i ) 2 j = i ∏ ϕ 1 ( u j ) = κ ∑ i λ i ( u i + b i ) 2 ϕ ( u ) .
The orthogonal transformation z = Q u z=Qu z = Q u preserves the standard Gaussian density and turns the numerator into ( z + a ) T S ( z + a ) (z+a)^{\mathsf T}S(z+a) ( z + a ) T S ( z + a ) .
For independent coupling, S t = t I S_t=tI S t = t I , so a = h t s π ( X 1 ) / ( 2 t ) a=h_ts_{\pi}(X_1)/(2t) a = h t s π ( X 1 ) / ( 2 t ) and κ = t ( d + ∥ a ∥ 2 ) \kappa=t(d+\lVert a\rVert^2) κ = t ( d + ∥ a ∥ 2 ) , recovering Eq. (28) . For Kabsch alignment, S t = J t = t I + ( 1 − t ) T H − 1 T T ≻ 0 S_t=J_t=tI+(1-t)TH^{-1}T^{\mathsf T}\succ0 S t = J t = t I + ( 1 − t ) T H − 1 T T ≻ 0 . The Stein identity above also holds for other differentiable couplings, but the positive-definiteness condition is required for this particular completed-square tilt.
Training Objective Derivation
Fix an outer iteration n n n and hold the denoiser m ˉ ( n ) \bar{m}^{(n)} m ˉ ( n ) fixed. Write r t ( n ) = M t [ q n ] + λ t C t [ q n ] r_t^{(n)}=\mathcal{M}_t[q_n]+\lambda_t\mathcal{C}_t[q_n] r t ( n ) = M t [ q n ] + λ t C t [ q n ] as in Eq. (19) . Assume the relevant second moments and estimator expectations are finite, and that the score-free identities hold, including the conditional-mean assumption on m ˉ ( n ) \bar{m}^{(n)} m ˉ ( n ) . Expanding the ideal regression loss gives
J n ( θ ) : = 1 2 E [ ∥ f θ ( Y t , t ) − r t ( n ) ( Y t ) ∥ 2 ] = 1 2 E [ ∥ f θ ( Y t , t ) ∥ 2 ] − E [ f θ ( Y t , t ) ⋅ M t [ q n ] ( Y t ) ] − E [ λ t f θ ( Y t , t ) ⋅ C t [ q n ] ( Y t ) ] + K n , (69) \begin{aligned}
J_n(\theta)&:=\frac{1}{2}\mathbb{E}\left[\left\lVert f_\theta(Y_t,t)-r_t^{(n)}(Y_t)\right\rVert^2\right]\\
&=\frac{1}{2}\mathbb{E}\left[\lVert f_\theta(Y_t,t)\rVert^2\right]-\mathbb{E}\left[f_\theta(Y_t,t)\cdot\mathcal{M}_t[q_n](Y_t)\right]\\
&\quad-\mathbb{E}\left[\lambda_tf_\theta(Y_t,t)\cdot\mathcal{C}_t[q_n](Y_t)\right]+K_n, \tag*{(69)}
\end{aligned} J n ( θ ) := 2 1 E [ f θ ( Y t , t ) − r t ( n ) ( Y t ) 2 ] = 2 1 E [ ∥ f θ ( Y t , t ) ∥ 2 ] − E [ f θ ( Y t , t ) ⋅ M t [ q n ] ( Y t ) ] − E [ λ t f θ ( Y t , t ) ⋅ C t [ q n ] ( Y t ) ] + K n , (69)
where K n = 1 2 E [ ∥ r t ( n ) ( Y t ) ∥ 2 ] K_n=\frac{1}{2}\mathbb{E}\left[\lVert r_t^{(n)}(Y_t)\rVert^2\right] K n = 2 1 E [ ∥ r t ( n ) ( Y t ) ∥ 2 ] is independent of θ \theta θ . The expectation averages over bridge time, source and endpoint samples, and the original Gaussian noise, as specified in § 3.3 .
Set A t = − ∇ U ( X ~ 1 ) + ( t / h t ) Z A_t=-\nabla U(\widetilde{X}_1)+(t/h_t)Z A t = − ∇ U ( X 1 ) + ( t / h t ) Z . By Eq. (15) and the tower property,
E [ f θ ( Y t , t ) ⋅ M t [ q n ] ( Y t ) ] = E [ f θ ( Y t , t ) ⋅ A t ] . \mathbb{E}\left[f_\theta(Y_t,t)\cdot\mathcal{M}_t[q_n](Y_t)\right]=\mathbb{E}\left[f_\theta(Y_t,t)\cdot A_t\right]. E [ f θ ( Y t , t ) ⋅ M t [ q n ] ( Y t ) ] = E [ f θ ( Y t , t ) ⋅ A t ] .
For each fixed t t t , applying Thm. 3.2 with φ = f θ ( ⋅ , t ) \varphi= f_\theta(\cdot,t) φ = f θ ( ⋅ , t ) gives
E [ λ t f θ ( Y t , t ) ⋅ C t [ q n ] ( Y t ) ] = E [ λ t C ^ t [ f θ ( ⋅ , t ) ] ] \mathbb{E}[\lambda_t f_\theta(Y_t,t) \cdot C_t[q_n](Y_t)] = \mathbb{E}[\lambda_t \widehat{C}_t[f_\theta(\cdot,t)]] E [ λ t f θ ( Y t , t ) ⋅ C t [ q n ] ( Y t )] = E [ λ t C t [ f θ ( ⋅ , t )]]
where the right-hand expectation also includes the tilted noise. Substituting these two identities into Eq. (69) yields J n ( θ ) = L n ( θ ) + K n J_n(\theta) = L_n(\theta) + K_n J n ( θ ) = L n ( θ ) + K n , with L n L_n L n given in Eq. (21) . Thus the two population objectives have the same minimizers; this equivalence does not remove finite-sample or denoiser-approximation errors.
Extended WASABI Algorithm
Algorithm 2 WASABI with replay, optional alignment, and annealing
Inputs: Energy U U U ; sampleable prior p 0 p_0 p 0 ; K K K initial endpoints; refresh count R ≤ K R \le K R ≤ K ; noise scale B B B ; iterations N N N ; schedules η t \eta_t η t , λ t \lambda_t λ t , β n \beta_n β n ; coupling map A A A .
Initialize denoiser parameters α \alpha α and correction parameters θ \theta θ .
Initialize the ring buffer D 0 = { ( x ( k ) , ∇ U ( x ( k ) ) ) } k = 1 K \mathcal{D}_0 = \left\{ \left(x^{(k)}, \nabla U\left(x^{(k)}\right)\right) \right\}_{k=1}^{K} D 0 = { ( x ( k ) , ∇ U ( x ( k ) ) ) } k = 1 K .
Set A ( g , x ) = g A(g,x)=g A ( g , x ) = g without alignment and β n = 1 \beta_n=1 β n = 1 without annealing.
for n = 0 , … , N − 1 n=0,\ldots,N-1 n = 0 , … , N − 1 do
Set target score: s π n ( x ) = − β n ∇ U ( x ) s_{\pi_n}(x)=-\beta_n\nabla U(x) s π n ( x ) = − β n ∇ U ( x ) .
Fit denoiser: sample minibatches of ( X 1 , ∇ U ( X 1 ) ) ∼ D n (X_1,\nabla U(X_1))\sim\mathcal{D}_n ( X 1 , ∇ U ( X 1 )) ∼ D n , independent G ∼ p 0 G\sim p_0 G ∼ p 0 , bridge times t t t , and Z ∼ N ( 0 , I ) Z\sim\mathcal{N}(0,I) Z ∼ N ( 0 , I ) . Construct
X 0 = A ( G , X 1 ) , h t = B t ( 1 − t ) , X_0=A(G,X_1),\qquad h_t=B\sqrt{t(1-t)}, X 0 = A ( G , X 1 ) , h t = B t ( 1 − t ) , Y t = ( 1 − t ) X 0 + t X 1 + h t Z , Y_t=(1-t)X_0+tX_1+h_tZ, Y t = ( 1 − t ) X 0 + t X 1 + h t Z , and fit
α n ≈ arg min α E D n [ ∥ m α ( Y t , t ) − X 1 ∥ 2 ] . \alpha_n\approx\underset{\alpha}{\operatorname{arg\,min}}\ \mathbb{E}_{\mathcal{D}_n}\left[\lVert m_\alpha(Y_t,t)-X_1\rVert^2\right]. α n ≈ α arg min E D n [ ∥ m α ( Y t , t ) − X 1 ∥ 2 ] . Fit correction: freeze α n \alpha_n α n ; construct training pairs as above and compute
J t = t I + ( 1 − t ) D X 1 A ( G , X 1 ) . J_t=tI+(1-t)D_{X_1}A(G,X_1). J t = t I + ( 1 − t ) D X 1 A ( G , X 1 ) . Using paired Gaussian and tilted noise, fit
θ n ≈ arg min θ E D n [ 1 2 ∥ f θ ( Y t , t ) ∥ 2 − λ t C ^ t , n [ f θ ( ⋅ , t ) ] \theta_n\approx\underset{\theta}{\operatorname{arg\,min}}\ \mathbb{E}_{\mathcal{D}_n}[\frac{1}{2}\lVert f_\theta(Y_t,t)\rVert^2-\lambda_t\widehat{C}_{t,n}[f_\theta(\cdot,t)] θ n ≈ θ arg min E D n [ 2 1 ∥ f θ ( Y t , t ) ∥ 2 − λ t C t , n [ f θ ( ⋅ , t )] − f θ ( Y t , t ) ⋅ ( s π n ( X 1 ) + J t T Z h t ) ] . \qquad-f_\theta(Y_t,t)\cdot(s_{\pi_n}(X_1)+\frac{J_t^{\mathsf{T}}Z}{h_t})]. − f θ ( Y t , t ) ⋅ ( s π n ( X 1 ) + h t J t T Z )] . Update sampler: m ~ t ( n + 1 ) ( y ) ← m α n ( y , t ) + η t f θ n ( y , t ) \widetilde{m}_t^{(n+1)}(y)\leftarrow m_{\alpha_n}(y,t)+\eta_t f_{\theta_n}(y,t) m t ( n + 1 ) ( y ) ← m α n ( y , t ) + η t f θ n ( y , t ) .
Generate fresh endpoints: simulate the updated sampler to obtain R R R configurations x x x and evaluate their gradients ∇ U ( x ) \nabla U(x) ∇ U ( x ) .
Refresh buffer: replace the oldest R R R entries of D n \mathcal{D}_n D n with the fresh pairs ( x , ∇ U ( x ) ) (x,\nabla U(x)) ( x , ∇ U ( x )) to obtain D n + 1 \mathcal{D}_{n+1} D n + 1 .
end for
return The sampler driven by m ~ ( N ) \widetilde{m}^{(N)} m ( N ) .
Here C ^ t , n \widehat{C}_{t,n} C t , n denotes the general-coupling covariance estimator from Appendix § 9 .§ 5 , evaluated using the current target score s π n s_{\pi_n} s π n and the frozen denoiser m α n m_{\alpha_n} m α n . The subscript D n \mathcal{D}_n D n indicates that endpoints are drawn uniformly from the replay buffer; the expectations also average over prior configurations, bridge times, and noise.
The prior p 0 p_0 p 0 need only support sampling: prior configurations may be generated on demand or drawn from a stored sample bank, without evaluating the prior density or score. In the ALA2 experiments, we use a bank of configurations generated by a molecular dynamics simulation of the ALA2 Hamiltonian (Appendix § 10 .§ 3 ) at 1,000 K. This prior bank is distinct from the replay buffer, which stores generated endpoints together with their energy gradients and is partially refreshed at each outer iteration. When annealing, the cached gradients are rescaled by the current factor β n \beta_n β n .
Retaining endpoints from earlier samplers makes replay-buffer training an approximation to the population iteration. This reuse follows prior work [23 , 4 ] , and empirically we find that it produces samples that closely match the target distributions defined by the potentials in § 5 .
Experiment Details
DW-4 Evaluation
We compare ASBS, damped ASBS, BMS, damped BMS, and WASABI using a 16 × 4 16 \times4 16 × 4 EGNN under matched training settings. Training stops at 1,000 outer rounds for every method. For each seed, we pool 128 fresh endpoints from each of 16 checkpoints, saved every 25 rounds from 625 through 1,000, giving 2,048 endpoints. Each run is evaluated against one random 2,048-frame subsample of the public test split. Configuration W 2 W_2 W 2 uses the released assignment convention [4 ] . Energy W 2 W_2 W 2 uses every endpoint, without an energy cutoff.
Table 1 reports means and standard deviations across six seeds, except for damped BMS, where seed 1 is excluded from both metrics after divergence in this window (energy W 2 = 2,449 W_2 = 2{,}449 W 2 = 2 , 449 ). The retained five seeds give configuration W 2 = 1.48 ± 0.21 W_2 = 1.48 \pm0.21 W 2 = 1.48 ± 0.21 and energy W 2 = 2.72 ± 0.58 W_2 = 2.72 \pm0.58 W 2 = 2.72 ± 0.58 . Including all six seeds gives 1.46 ± \pm ± 0.20 and 410 ± \pm ± 999, respectively.
The pair-distance plot in Figure 2 pools all six pairwise distances from 10,000 WASABI and 10,000 reference configurations. It is a separate illustrative sample set, not the pooled checkpoint samples used for the table. The dashed lines mark the minima of the pairwise potential.
DW-4 and LJ-13 potentials
For both systems, r i j = ∥ x i − x j ∥ r_{ij} = \lVert x_i - x_j \rVert r ij = ∥ x i − x j ∥ denotes the distance between particles i i i and j j j .
Double-well system (DW-4). The four particles lie in two dimensions and interact through
U D W − 4 ( x ) = ∑ i < j [ 0.9 ( r i j − 4 ) 4 − 4 ( r i j − 4 ) 2 ] . (70) U_{\mathrm{DW-4}}(x) = \sum_{i<j} \left[0.9(r_{ij}-4)^4 - 4(r_{ij}-4)^2\right]. \tag*{(70)} U DW − 4 ( x ) = i < j ∑ [ 0.9 ( r ij − 4 ) 4 − 4 ( r ij − 4 ) 2 ] . (70)
Lennard–Jones cluster (LJ-13). The 13 particles lie in three dimensions. Their pairwise interactions are supplemented by quadratic confinement around the center of mass:
U L J − 13 ( x ) = 2 ∑ i < j ( r i j − 12 − 2 r i j − 6 ) + 1 2 ∑ i = 1 13 ∥ x i − x C O M ∥ 2 , x C O M = 1 13 ∑ i = 1 13 x i . (71) U_{\mathrm{LJ-13}}(x) = 2\sum_{i<j}\left(r_{ij}^{-12} - 2r_{ij}^{-6}\right) + \frac{1}{2}\sum_{i=1}^{13}\lVert x_i-x_{\mathrm{COM}}\rVert^2,\qquad x_{\mathrm{COM}} = \frac{1}{13}\sum_{i=1}^{13}x_i. \tag*{(71)} U LJ − 13 ( x ) = 2 i < j ∑ ( r ij − 12 − 2 r ij − 6 ) + 2 1 i = 1 ∑ 13 ∥ x i − x COM ∥ 2 , x COM = 13 1 i = 1 ∑ 13 x i . (71)
Alanine-dipeptide potential and training setup
Following the alanine-dipeptide benchmark in [4 ] , we use ff96 parameters with OBC1 implicit solvent and an additional chirality restraint. Let x = ( r 1 , … , r n ) ∈ ( R 3 ) n x = (r_1,\ldots,r_n) \in(\mathbb{R}^3)^n x = ( r 1 , … , r n ) ∈ ( R 3 ) n denote the atomic positions. The configurational potential is
U ( x ) = U f f 96 ( x ) + U O B C 1 ( x ) + U c h i r a l i t y ( x ) , U f f 96 ( x ) = U b o n d ( x ) + U a n g l e ( x ) + U t o r s i o n ( x ) + U L J ( x ) + U C o u l o m b ( x ) , U O B C 1 ( x ) = U G B ( x ) + U S A ( x ) . (72) \begin{aligned}
U(x) &= U_{\mathrm{ff96}}(x) + U_{\mathrm{OBC1}}(x) + U_{\mathrm{chirality}}(x),\\
U_{\mathrm{ff96}}(x) &= U_{\mathrm{bond}}(x) + U_{\mathrm{angle}}(x) + U_{\mathrm{torsion}}(x) + U_{\mathrm{LJ}}(x) + U_{\mathrm{Coulomb}}(x),\\
U_{\mathrm{OBC1}}(x) &= U_{\mathrm{GB}}(x) + U_{\mathrm{SA}}(x). \tag*{(72)}
\end{aligned} U ( x ) U ff96 ( x ) U OBC1 ( x ) = U ff96 ( x ) + U OBC1 ( x ) + U chirality ( x ) , = U bond ( x ) + U angle ( x ) + U torsion ( x ) + U LJ ( x ) + U Coulomb ( x ) , = U GB ( x ) + U SA ( x ) . (72)
The nonzero bonded coefficients of this Hamiltonian define the index sets
E = { e = ( i , j ) : k e b ≠ 0 } , A = { a = ( i , j , k ) : k a a ≠ 0 } , T = { τ = ( i , j , k , l ) : V τ n ≠ 0 for some n } . (73) \begin{aligned}
E &= \{e=(i,j): k^b_e \ne0\},\\
A &= \{a=(i,j,k): k^a_a \ne0\},\\
T &= \{\tau=(i,j,k,l): V_{\tau n} \ne0\text{ for some }n\}. \tag*{(73)}
\end{aligned} E A T = { e = ( i , j ) : k e b = 0 } , = { a = ( i , j , k ) : k a a = 0 } , = { τ = ( i , j , k , l ) : V τ n = 0 for some n } . (73)
Here j j j is the central atom of an angle triple, and n n n indexes the Fourier components of a dihedral. For this alanine-dipeptide topology, A A A coincides with the triples formed by pairs of bonds in E E E sharing a central atom.
Table 4. Energy terms in the alanine-dipeptide Hamiltonian. The formulas display functional forms; ff96 exclusions and special 1–4 parameters are included in the starred nonbonded sums. Coefficients include any thermal rescaling used to define U U U .
Here r i j = ∥ r i − r j ∥ r_{ij}=\lVert r_{i}-r_{j}\rVert r ij = ∥ r i − r j ∥ , ρ i j = σ i j / r i j \rho_{ij}=\sigma_{ij}/r_{ij} ρ ij = σ ij / r ij , and Δ ϵ = ϵ i n − 1 − ϵ o u t − 1 \Delta\epsilon=\epsilon_{\mathrm{in}}^{-1}-\epsilon_{\mathrm{out}}^{-1} Δ ϵ = ϵ in − 1 − ϵ out − 1 . The generalized-Born pair distance is
f i j G B = [ r i j 2 + R i R j exp ( − r i j 2 4 R i R j ) ] 1 / 2 . f_{ij}^{\mathrm{GB}}=\left[r_{ij}^{2}+R_{i}R_{j}\exp\left(-\frac{r_{ij}^{2}}{4R_{i}R_{j}}\right)\right]^{1/2}. f ij GB = [ r ij 2 + R i R j exp ( − 4 R i R j r ij 2 ) ] 1/2 .
The R i R_{i} R i are OBC1 effective Born radii, a i a_{i} a i are atomic radii, and a s a_{\mathrm{s}} a s is the solvent probe radius. For each restrained dihedral s s s , d s = ∣ wrap ( χ s − χ s 0 ) ∣ d_{s}=|\operatorname{wrap}(\chi_{s}-\chi_{s}^{0})| d s = ∣ wrap ( χ s − χ s 0 ) ∣ is its wrapped angular distance from the preferred chirality, with tolerance τ \tau τ .
Bond–angle split readout. The sets E E E and A A A defined by the nonzero bond and angle terms index the bond lengths and angle cosines used in our readout. We define the local bonded-coordinate map
c ( x ) = { ℓ i j ( x ) } ( i , j ) ∈ E , { cos α i j k ( x ) } ( i , j , k ) ∈ A , c(x)=\{\ell_{ij}(x)\}_{(i,j)\in E},\{\cos\alpha_{ijk}(x)\}_{(i,j,k)\in A}, c ( x ) = { ℓ ij ( x ) } ( i , j ) ∈ E , { cos α ij k ( x ) } ( i , j , k ) ∈ A ,
ℓ i j ( x ) = ∥ r i − r j ∥ , \ell_{ij}(x)=\lVert r_{i}-r_{j}\rVert, ℓ ij ( x ) = ∥ r i − r j ∥ ,
cos α i j k ( x ) = ( r i − r j ) ⋅ ( r k − r j ) ∥ r i − r j ∥ ∥ r k − r j ∥ . (74) \cos\alpha_{ijk}(x)=\frac{(r_{i}-r_{j})\cdot(r_{k}-r_{j})}{\lVert r_{i}-r_{j}\rVert\lVert r_{k}-r_{j}\rVert}. \tag*{(74)} cos α ij k ( x ) = ∥ r i − r j ∥ ∥ r k − r j ∥ ( r i − r j ) ⋅ ( r k − r j ) . (74)
All Jacobians below are taken in the orthonormal zero-centroid coordinate basis used by the model. Thus c ( x ) c(x) c ( x ) uses the known connectivity and current positions, but neither evaluates the Hamiltonian nor uses its force constants or preferred bond lengths. With J ( x ) = D c ( x ) J(x)=Dc(x) J ( x ) = D c ( x ) , a small displacement satisfies c ( x + ε v ) = c ( x ) + ε J ( x ) v + O ( ε 2 ) c(x+\varepsilon v)=c(x)+\varepsilon J(x)v+O(\varepsilon^{2}) c ( x + ε v ) = c ( x ) + ε J ( x ) v + O ( ε 2 ) . Directions in ker J ( x ) \ker J(x) ker J ( x ) leave the selected bond lengths and angles unchanged to first order.
To distinguish these motions in the corrector, we use the regularized map
P δ ( x ) = J ( x ) T ( J ( x ) J ( x ) T + δ I ) − 1 J ( x ) , δ = 10 − 4 , (75) P_{\delta}(x)=J(x)^{\mathsf{T}}\left(J(x)J(x)^{\mathsf{T}}+\delta I\right)^{-1}J(x),\qquad\delta=10^{-4}, \tag*{(75)} P δ ( x ) = J ( x ) T ( J ( x ) J ( x ) T + δ I ) − 1 J ( x ) , δ = 1 0 − 4 , (75)
and combine the two PaiNN vector heads as
f θ ( x , t ) = σ s t i f f P δ ( x ) H s t i f f ( x , t ) + σ s o f t ( I − P δ ( x ) ) H s o f t ( x , t ) . (76) f_{\theta}(x,t)=\sigma_{\mathrm{stiff}}P_{\delta}(x)H_{\mathrm{stiff}}(x,t)+\sigma_{\mathrm{soft}}\left(I-P_{\delta}(x)\right)H_{\mathrm{soft}}(x,t). \tag*{(76)} f θ ( x , t ) = σ stiff P δ ( x ) H stiff ( x , t ) + σ soft ( I − P δ ( x ) ) H soft ( x , t ) . (76)
We call this the Bond–Angle Split Heads (BASH) readout. The reported ALA2 runs use a separate PaiNN backbone for each head, trained through their sum. The split is evaluated at the current noisy configuration. Because P δ P_\delta P δ is regularized, this is not an exact orthogonal decomposition and does not fix bond lengths or angles. In particular, c ( x ) c(x) c ( x ) does not encode handedness; the separate chirality restraint acts through the target energy and score.
Training setup and prior preparation. The reported sweeps share a 4,096-endpoint replay buffer and 256 fresh samples per outer iteration (6.25%). We linearly interpolate inverse temperature from 1,000 K to 300 K over the first 1,500 of 4,000 outer iterations:
β n = 1 − α n k B ( 1,000 K ) + α n k B ( 300 K ) , α n = min { n 1,500 , 1 } , (77) \beta_n = \frac{1-\alpha_n}{k_{\mathrm{B}}(1{,}000\ \mathrm{K})}+\frac{\alpha_n}{k_{\mathrm{B}}(300\ \mathrm{K})}, \qquad\alpha_n=\min\left\{\frac{n}{1{,}500},1\right\}, \tag*{(77)} β n = k B ( 1 , 000 K ) 1 − α n + k B ( 300 K ) α n , α n = min { 1 , 500 n , 1 } , (77)
where n ≥ 0 n \ge0 n ≥ 0 is the outer-iteration index and β n = ( k B T n ) − 1 \beta_n=(k_{\mathrm{B}}T_n)^{-1} β n = ( k B T n ) − 1 . The target remains at 300 K thereafter. Replay samples are reweighted as the temperature changes. The fixed prior bank is separate from this evolving replay buffer. It contains 20,000 configurations from the 1,000 K state of a seven-temperature CUDA REMD run; the full run, including all temperatures and equilibration, took approximately 62.5 minutes. An independent chain supplies the 20,000-frame reference in Figure 4 . A separate local benchmark generated 25,000 configurations at 1,000 K in approximately five minutes on an Apple M5 Max using OpenMM’s CPU platform with one thread. This timing includes system setup, minimization, 100,000 equilibration steps and 2,500,000 production steps at 1 fs, recording every 100 steps; plotting is excluded. The 20k-frame local-prior budget in Table 3 uses 2,000,000 production steps and the same equilibration length.
We compared this locally generated 20k-frame bank with the 1,000 K states of two independent CUDA REMD chains, using 20,000 frames per bank and the same 50 × 50 angle grid. Local MD versus the two REMD banks gave D J S = 0.0309 D_{\mathrm{JS}} = 0.0309 D JS = 0.0309 and 0.0307, compared with 0.0296 between the two REMD banks. The discrepancy between the local MD and REMD backbone-angle distributions is comparable to the observed chain-to-chain sampling variability. Based on this agreement, we do not expect replacing the REMD-derived prior with the locally generated high-temperature MD prior to materially affect the reported results. The reported training runs used the REMD-derived bank.
Ramachandran maps. The maps in Figure 4 show relative free energy Δ F / ( k B T ) = − log ( p / p max ) \Delta F/(k_{\mathrm{B}}T)=-\log(p/p_{\max}) Δ F / ( k B T ) = − log ( p / p m a x ) , where p p p is the histogram-bin probability and p max p_{\max} p m a x is its maximum within each panel. Free-energy zeros are therefore set separately, and gray bins contain no observed samples. The positive-ϕ \phi ϕ region contains 10 of 20,000 WASABI samples (0.05%) and 78 of 20,000 reference frames (0.39%).
Figure 4 compares 20,000 WASABI samples with 20,000 frames from an independent REMD simulation. We show seed 1 for s η = 0.10 s_\eta=0.10 s η = 0.10 , λ c o v = 0.25 \lambda_{\mathrm{cov}}=0.25 λ cov = 0.25 , which had the lowest JS divergence against this reference among the three seeds.
Alanine-dipeptide ablations and computational cost
Influence of hyperparameters. We separately vary the covariance weight and update step scale, holding s η = 1 s_{\eta}=1 s η = 1 fixed in the covariance sweep and λ c o v = 0.25 \lambda_{\mathrm{cov}}=0.25 λ cov = 0.25 fixed in the step-scale sweep. Each sweep includes 16 settings evaluated over three training seeds, for 96 runs in total (Figure 5 (a,c), Table 5 ). Covariance weights below 0.5 generally give lower JS divergence, with the lowest mean at λ c o v = 0.4 \lambda_{\mathrm{cov}}=0.4 λ cov = 0.4 . Performance varies less across the tested step scales, with the lowest mean at s η = 0.10 s_{\eta}=0.10 s η = 0.10 .
Figure 5. ALA2 ablations and replay comparison. (a,c) JS mean ± \pm ± SD over three seeds; gray dots show individual scores and dashed lines published BMS/ASBS means [4 ] . (b,d) Matched seeds with smaller and larger replay settings; dashed curves show REMD. Arrows highlight mitigation of seed 2’s central-peak underrepresentation. Both capacity and refresh increase. The sweeps and replay comparison use different checkpoints and sample counts; evaluation protocols are described below.
Table 5. Covariance-weight and step-scale sweeps. Values are mean ± \pm ± SD over three seeds; bold marks the lowest mean in each sweep.
Replay-buffer size and mode coverage. A low JS divergence can coexist with underrepresentation of a smaller peak in the backbone-angle distribution. Figure 5 (b,d) illustrates this behavior in the central peak of the ψ \psi ψ distribution. Increasing the replay capacity from 4,096 to 8,192 and the number of fresh endpoints per iteration from 256 to 512 improves coverage in the highlighted seed, although peak populations remain variable across the five runs evaluated with the larger replay setting. Both replay capacity and refresh count are changed in this comparison.
Evaluation. Following Blessing et al. [4 ] , we compute JS divergence in nats using a 50 × \times × 50 grid of backbone angles. The hyperparameter sweeps use 20,000 generated samples per seed at outer iteration 4,000, evaluated against the one-million-frame validation set released with FAB (reference dataset). Values are reported as mean ± \pm ± standard deviation over three seeds. Evaluating the same selected checkpoints with 1,024,000 generated samples per seed reduces JS from 0.0312 ± \pm ± 0.0023 to 0.0254 ± \pm ± 0.0033 (Table 3 ), consistent with reduced sampling noise in the empirical histograms. The replay comparison uses matched seeds 0–2 at iteration 2,500, with s η = 0.20 s_{\eta}=0.20 s η = 0.20 , λ c o v = 0.25 \lambda_{\mathrm{cov}}=0.25 λ cov = 0.25 , 4,096 generated samples per seed, and an independent 20,000-frame REMD reference. The plotted ψ \psi ψ marginal uses unsmoothed 5° bins.
Computational cost. We count matrix multiplications in PaiNN’s message, update, and readout blocks, including backpropagation, with each multiply-add counted as two FLOPs. Applying a linear map from d i n d_{\mathrm{in}} d in to d o u t d_{\mathrm{out}} d out channels to K K K feature vectors therefore costs approximately
2 K d i n d o u t FLOPs . 2K d_{\mathrm{in}}d_{\mathrm{out}}\ \text{FLOPs}. 2 K d in d out FLOPs .
We sum these costs using the tensor dimensions in the implementation. Other operations, including activations, geometric calculations, optimizer updates, and molecular-energy evaluations, are excluded.
For ALA2, one evaluation of our four-layer, 64-channel PaiNN costs 76.6 MFLOPs per molecule. Sampling evaluates two correction networks and one denoiser, totaling 229.7 MFLOPs per integration step. Each outer iteration performs 100 correction updates and 200 denoiser updates with batch size 64, then generates 256 fresh endpoints using 256 integration steps. Including the antithetic and tilted queries used during training gives 31.7 TFLOPs per outer iteration.4
For BMS, we assume eight GPUs based on the paper’s 16,384-sample replay buffer and the released configuration’s 2,048 samples per GPU [4 ] . This implies a global training batch of 512 and 2,048 fresh endpoints per outer iteration. Under these settings, we estimate 112.9 TFLOPs per outer iteration without damping and 190.2 TFLOPs with damping. The paper does not explicitly report the GPU count. We do not estimate ASBS training FLOPs because its corrector and adjoint calculations require separate accounting.
Molecular energy and force evaluations. Table 3 reports estimated molecular-potential calls in millions per run, including force evaluations. One endpoint scored for both energy and force counts once. Neural sampler evaluations and optimizer updates are not molecular-potential calls. Diagnostic and evaluation samples, minimization, and additional energy-only reporting queries are excluded from these nominal force-call budgets.
WASABI scores the 4,096 initial replay endpoints and 256 fresh endpoints at each of 4,000 outer iterations, totaling 4,096 + 4,000 × 256 = 1,028,096 4{,}096 + 4{,}000 \times256 = 1{,}028{,}096 4 , 096 + 4 , 000 × 256 = 1 , 028 , 096 calls. Inner updates reuse cached scores; annealing rescales cached physical forces and reweights cached energies without new potential evaluations. Charging each run for a local 20k-frame hot bank adds 2,000,000 production and 100,000 equilibration MD steps, yielding 3,128,096 ≈ 3.13 × 10 6 3{,}128{,}096 \approx3.13 \times10^{6} 3 , 128 , 096 ≈ 3.13 × 1 0 6 calls. This is the budget for the local-prior procedure, rather than a measured end-to-end cost of the reported REMD-prior training runs.
For ASBS and BMS, we use the paper’s 5,000 ALA2 outer iterations and the inferred global refresh of 2,048 endpoints, yielding 5,000 × 2,048 = 10.24 × 10 6 5{,}000 \times2{,}048 = 10.24 \times10^{6} 5 , 000 × 2 , 048 = 10.24 × 1 0 6 calls. The released BMS configuration initializes replay with the same number of endpoints as a refresh, counted in its first outer iteration. ASBS’s released upstream implementation scores fresh adjoint-buffer endpoints and reuses cached terminal costs; corrector fitting uses the analytic reference-diffusion score and does not query the molecular potential. Damping adds neural evaluations, but no additional molecular-potential calls under the same refresh schedule. These counts estimate the published setting, rather than measure its unreleased execution logs; the released BMS default of 7,000 iterations would instead imply 14.336 × 10 6 14.336 \times10^{6} 14.336 × 1 0 6 calls.
The REMD row uses seven temperatures, 20,000 production cycles and 1,000 equilibration cycles, with 500 MD steps per cycle at 1 fs. Across replicas, this gives 7 × ( 20,000 + 1,000 ) × 500 = 73.5 × 10 6 7 \times(20{,}000 + 1{,}000) \times500 = 73.5 \times10^{6} 7 × ( 20 , 000 + 1 , 000 ) × 500 = 73.5 × 1 0 6 force evaluations. There are additionally 147,000 energy queries for exchange decisions and initial minimization work. This row records the cost of the reference protocol; it is not an accuracy-matched comparison with trained samplers, and no self-comparison JS score is reported.