Introduction
Stochastic differential equations (SDEs) provide a fundamental framework for describing stochastic processes, with applications across physics, finance, and biology. A prominent example is molecular dynamics ([34 , 20 ] ), where Langevin equations describe the evolution of the atomic coordinates and momenta under interatomic forces and random fluctuations. Classical approaches, such as Euler-Maruyama ([29 ] ) and Langevin splitting schemes ([33 ] ), approximate the dynamics through a sequence of short-time updates and provide systematic control over discretization error. However, such methods are inherently sequential, with computational cost scaling with the number of integration steps. In molecular dynamics, femtosecond-scale time steps (10 − 15 10^{-15} 1 0 − 15 seconds) are typically required to maintain numerical stability and resolve fast motions such as bond vibrations, while conformational changes and transitions between metastable states can occur over orders-of-magnitude longer timescales, e.g. 10 − 6 10^{-6} 1 0 − 6 seconds. Reaching these slow phenomena can therefore require an enormous number of sequential force evaluations, making long-horizon molecular simulation computationally expensive.
Recently, numerous approaches were developed to learn surrogates that can accelerate molecular simulation by predicting the evolution over larger time intervals. Generative modeling approaches ([27 , 51 ] ) learn conditional distributions of future molecular configurations from simulated trajectories. Other approaches ([4 ] ) learn direct deterministic updates of the positions and momenta. However, both formulations learn these transitions from pairs of states extracted from reference trajectories. Generating these pairs still requires numerical integration with time steps typically of the order of 10 − 15 10^{-15} 1 0 − 15 s, incurring substantial computational cost to produce the training data. Moreover, equilibrium samples can also be obtained using methods such as sequential Monte Carlo [41 ] or parallel tempering [21 ] . Yet, their sampling transitions do not necessarily follow the physical dynamics of interest, so the resulting data may not provide the time-dependent pairs required by supervised methods.
The underlying SDE, however, provides a different source of supervision. Although the finite-time transition kernel is generally unavailable in closed form, the drift and diffusion specify the dynamics locally. This raises the possibility of learning finite-time transitions from the governing SDE itself, without first generating reference trajectories. Recent approaches in flow-based generative modeling [16 , 52 , 36 , 3 ] use consistency relations to connect the underlying vector fields with the finite-time maps [53 , 13 , 6 ] , enabling accelerated generative sampling. Hamiltonian Flow Maps [49 ] apply this consistency principle to molecular dynamics, learning Hamiltonian evolution from independent states and instantaneous force fields without trajectory supervision. For stochastic dynamics, we seek an analogous formulation at the level of conditional distributions, connecting their finite-time evolution to the known drift and diffusion of the SDE.
In this work, we present Kolmogorov-Chapman Maps (KC-maps), a trajectory-free method for learning finite-time transitions of SDEs. Our method uses uncorrelated samples of the initial state together with the known vector field and diffusion, without requiring trajectory supervision or a prescribed probability path. We derive a distributional consistency objective from the Kolmogorov–Chapman equation, enforcing agreement between transitions composed over different time scales; for example, two successive 5 fs transitions should agree in distribution with a single 10 fs transition. To enable tractable density evaluation and satisfy SDE boundary conditions, we parameterize transitions densities using a conditional normalizing flow with a Gaussian reference kernel. The parameterization is constrained to recover the appropriate short-time behavior of the underlying SDE, while additional design choices enable learning and sampling over substantially longer time intervals.
The formulation of KC-maps applies to general SDEs. Here, we focus on molecular Langevin dynamics because of both its computational difficulty and its importance to applications such as drug discovery. We first conduct an empirical study on a small molecular system to identify important design choices required for stable and accurate learning, including the training objective, design of the normalizing flow, and stabilization techniques for sampling over long time intervals. We then apply KC-maps to larger scale molecular systems and highlight the following main results:
Using only equilibrium samples and force-field evaluations, without trajectory supervision, we learn transition distributions over intervals of up to 100 fs for alanine dipeptide (AL2), tetrapeptide (AL4), and hexapeptide (AL6), replacing up to 100 sequential MD integration steps with a single learned transition.
We demonstrate higher simulation throughput and faster exploration of conformational modes. On AL6, our method achieves approximately 2.3× aggregate simulation throughput (ns per GPU-hour) of MD on a single L40s GPU and reaches three of the four TICA modes while MD baseline still misses two modes after an hour. Additionally, our method is able to recover all four modes of AL6 with longer simulations despite the training data only covering three of the four modes.
Beyond equilibrium sampling, KC-maps captures molecular kinetics, with mean first passage times (MFPT), mode autocorrelations, and transition probabilities closely matching ground-truth MD.
Background
Stochastic Differential Equations. Stochastic differential equations (SDEs) describe the stochastic process X t X_t X t via the differential equation for their time-increments d X t dX_t d X t , i.e.
d X t = v t ( X t ) d t + g t d W t , (1) dX_t = v_t(X_t)\,dt + g_t\,dW_t, \tag*{(1)} d X t = v t ( X t ) d t + g t d W t , (1)
where g t ∈ R d × d g_t \in\mathbb{R}^{d \times d} g t ∈ R d × d is a state-independent diffusion matrix, and W t W_t W t is standard Brownian motion in R d \mathbb{R}^d R d . We will refer to the solution of the Cauchy problem of this SDE as a transition kernel , i.e. given the initial condition X s = x X_s = x X s = x , we denote the density of the distribution of X t X_t X t as p s , t ( y ∣ x ) : = p ( X t = y ∣ X s = x ) p_{s,t}(y \mid x) := p(X_t = y \mid X_s = x) p s , t ( y ∣ x ) := p ( X t = y ∣ X s = x ) .
Figure 1. A single KC-maps rollout explores the landscape of metastable alanine tetrapeptide conformers. Starting from a single conformer X 0 X_0 X 0 (left), KC-maps is rolled out for 200 ns. Orange arrows trace the first visit to each newly discovered basin. Representative conformers from each basin are shown on the right, colored by basin identity, along with their backbone torsion angles ( Φ 2 , Φ 3 ) (\Phi_2, \Phi_3) ( Φ 2 , Φ 3 ) .
Molecular Dynamics. The formalism of SDEs is especially useful for describing the evolution of molecular systems. Their state is described by the position X t X_t X t and momentum P t P_t P t and its time-evolution is modeled by the second-order dynamics, which we will write in the matrix form
\begin{bmatrix} dX_t \\ dP_t \end{bmatrix}
=
\begin{bmatrix} M^{-1}P_t \\ v_t(X_t)-\gamma P_t \end{bmatrix}dt
+
\begin{bmatrix} 0 & 0 \\ 0 & g_t \end{bmatrix}dW_t, \tag*{(2)}\end{bmatrix}
where v t v_t v t is the force field driving the system, γ \gamma γ is the friction coefficient, M M M is the diagonal mass matrix, and g t g_t g t is the diffusion coefficient that depends on the temperature of the system. For v t ( X ) = − ∇ U ( X ) v_t(X)=-\nabla U(X) v t ( X ) = − ∇ U ( X ) where U ( X ) U(X) U ( X ) is a scalar potential and g t = 2 γ k B T M g_t=\sqrt{2\gamma k_B T M} g t = 2 γ k B T M , the stationary distribution of X t X_t X t is the Boltzmann distribution, i.e. the equilibrium of the molecular system. Thus, in practice, simulating this SDE for long enough time one can sample from the Boltzmann distribution describing different states — conformations — of the system.
Normalizing Flows. Generative modeling of the empirical data has two principal components: maximizing the model likelihood on the observed data and generating new samples under the learned density model. Normalizing flows satisfy both criteria by parameterizing bijective maps with tractable Jacobians. The building block of normalizing flows is the affine coupling layer Ψ k \Psi_k Ψ k , which transforms the state x ( k ) = [ x 1 ( k ) , x 2 ( k ) ] x^{(k)}=[x_1^{(k)},x_2^{(k)}] x ( k ) = [ x 1 ( k ) , x 2 ( k ) ] into x ( k + 1 ) = Ψ k ( x ( k ) ) x^{(k+1)}=\Psi_k(x^{(k)}) x ( k + 1 ) = Ψ k ( x ( k ) ) as
x 1 ( k + 1 ) = x 1 ( k ) , (3) x_1^{(k+1)}=x_1^{(k)}, \tag*{(3)} x 1 ( k + 1 ) = x 1 ( k ) , (3)
x 2 ( k + 1 ) = x 2 ( k ) ⊙ exp ( a k ( x 1 ( k ) ; θ , c ) ) + b k ( x 1 ( k ) ; θ , c ) , (4) x_2^{(k+1)}=x_2^{(k)}\odot\exp\left(a_k(x_1^{(k)};\theta,c)\right)+b_k(x_1^{(k)};\theta,c), \tag*{(4)} x 2 ( k + 1 ) = x 2 ( k ) ⊙ exp ( a k ( x 1 ( k ) ; θ , c ) ) + b k ( x 1 ( k ) ; θ , c ) , (4)
where a k ( ⋅ ; θ , c ) : R d 1 → R d 2 a_k(\cdot;\theta,c):\mathbb{R}^{d_1}\to\mathbb{R}^{d_2} a k ( ⋅ ; θ , c ) : R d 1 → R d 2 and b k ( ⋅ ; θ , c ) : R d 1 → R d 2 b_k(\cdot;\theta,c):\mathbb{R}^{d_1}\to\mathbb{R}^{d_2} b k ( ⋅ ; θ , c ) : R d 1 → R d 2 are functions parameterized by the neural networks with parameters θ \theta θ and, optionally, taking a condition c c c as an input. This layer is invertible, i.e. one can find [ x 1 ( k ) , x 2 ( k ) ] [x_1^{(k)},x_2^{(k)}] [ x 1 ( k ) , x 2 ( k ) ] from [ x 1 ( k + 1 ) , x 2 ( k + 1 ) ] [x_1^{(k+1)},x_2^{(k+1)}] [ x 1 ( k + 1 ) , x 2 ( k + 1 ) ] , and the Jacobian of this transformation is available in a closed-form without additional computations
log det ∣ ∂ Ψ k ( x ( k ) ) ∂ x ( k ) ∣ = log [ I 0 ∂ x 2 ( k + 1 ) ∂ x 1 ( k ) I exp ( a k ( x 1 ( k ) ; θ , c ) ) ] = ∑ i a k ( x 1 ( k ) ; θ , c ) i . (5) \log\det\left|\frac{\partial\Psi_k(x^{(k)})}{\partial x^{(k)}}\right|
=
\log\left[
\begin{array}{cc}
I & 0 \\
\frac{\partial x_2^{(k+1)}}{\partial x_1^{(k)}} & I\exp\left(a_k(x_1^{(k)};\theta,c)\right)
\end{array}
\right]
=
\sum_i a_k(x_1^{(k)};\theta,c)_i. \tag*{(5)} log det ∂ x ( k ) ∂ Ψ k ( x ( k ) ) = log [ I ∂ x 1 ( k ) ∂ x 2 ( k + 1 ) 0 I exp ( a k ( x 1 ( k ) ; θ , c ) ) ] = i ∑ a k ( x 1 ( k ) ; θ , c ) i . (5)
Finally, normalizing flows parameterize a density p ( x ∣ θ , c ) p(x\mid\theta,c) p ( x ∣ θ , c ) as a push-forward map of some simple density p r e f ( x ) p^{\mathrm{ref}}(x) p ref ( x ) through the composition of affine layers Ψ = Ψ K ∘ … ∘ Ψ 1 \Psi=\Psi_K\circ\ldots\circ\Psi_1 Ψ = Ψ K ∘ … ∘ Ψ 1
log p ( x ∣ θ , c ) = log p r e f ( Ψ − 1 ( x ; θ , c ) ) − ∑ k log det ∣ ∂ Ψ k ( x ( k ) ) ∂ x ( k ) ∣ . (6) \log p(x\mid\theta,c)=\log p^{\mathrm{ref}}\left(\Psi^{-1}(x;\theta,c)\right)-\sum_k\log\det\left|\frac{\partial\Psi_k(x^{(k)})}{\partial x^{(k)}}\right|. \tag*{(6)} log p ( x ∣ θ , c ) = log p ref ( Ψ − 1 ( x ; θ , c ) ) − k ∑ log det ∂ x ( k ) ∂ Ψ k ( x ( k ) ) . (6)
Kolmogorov-Chapman Maps
Our method is designed around the classical definition of a diffusion process as a Markov process with continuous trajectories and infinitesimal first and second moments prescribed by the drift and diffusion coefficient [32 , 12 ] . The process defined this way is a weak solution of the corresponding SDE [54 ] . Therefore, our parametric model has to satisfy two conditions: (i) parameterize a valid Markov process, i.e. satisfy the Kolmogorov–Chapman equation, which we enforce through training (ii) the short-time behavior has to match the corresponding SDE, i.e. commit no jumps and have same first two moments, which we incorporate directly into our parameterization. We now proceed to formulating this statement formally.
For the SDE defined in Eq. (1) , the infinitesimal change of the density p s , t ( y ∣ x ) p_{s,t}(y \mid x) p s , t ( y ∣ x ) is defined by the following boundary conditions. ∀ ε > 0 \forall\varepsilon> 0 ∀ ε > 0 , we have
lim τ → 0 1 τ ∫ ∥ y − x ∥ > ε d y p s , s + τ ( y ∣ x ) = 0 , (7) \lim_{\tau\to0} \frac{1}{\tau} \int_{\lVert y-x\rVert> \varepsilon} \mathrm{d}y \, p_{s,s+\tau}(y \mid x) = 0, \tag*{(7)} τ → 0 lim τ 1 ∫ ∥ y − x ∥ > ε d y p s , s + τ ( y ∣ x ) = 0 , (7)
(delta function with no jumps)
lim τ → 0 1 τ ∫ ∥ y − x ∥ ≤ ε d y p s , s + τ ( y ∣ x ) ( y − x ) = v s ( x ) , (8) \lim_{\tau\to0} \frac{1}{\tau} \int_{\lVert y-x\rVert\le\varepsilon} \mathrm{d}y \, p_{s,s+\tau}(y \mid x)(y-x) = v_s(x), \tag*{(8)} τ → 0 lim τ 1 ∫ ∥ y − x ∥ ≤ ε d y p s , s + τ ( y ∣ x ) ( y − x ) = v s ( x ) , (8)
(matching first moment)
lim τ → 0 1 τ ∫ ∥ y − x ∥ ≤ ε d y p s , s + τ ( y ∣ x ) ( y − x ) ( y − x ) ⊤ = g s g s ⊤ , (9) \lim_{\tau\to0} \frac{1}{\tau} \int_{\lVert y-x\rVert\le\varepsilon} \mathrm{d}y \, p_{s,s+\tau}(y \mid x)(y-x)(y-x)^{\top} = g_s g_s^{\top}, \tag*{(9)} τ → 0 lim τ 1 ∫ ∥ y − x ∥ ≤ ε d y p s , s + τ ( y ∣ x ) ( y − x ) ( y − x ) ⊤ = g s g s ⊤ , (9)
(matching second moment)
i.e. for t → s t \to s t → s , the density converges to the delta-function and makes no jumps, the change of the first moment equals the vector field and the change of the second moment equals the diffusion coefficient. Furthermore, any Markov transition kernel satisfies the Kolmogorov–Chapman equation
p s , t ( y ∣ x ) = ∫ d x ′ p r , t ( y ∣ x ′ ) p s , r ( x ′ ∣ x ) . (10) p_{s,t}(y \mid x) = \int\mathrm{d}x' \, p_{r,t}(y \mid x')p_{s,r}(x' \mid x). \tag*{(10)} p s , t ( y ∣ x ) = ∫ d x ′ p r , t ( y ∣ x ′ ) p s , r ( x ′ ∣ x ) . (10)
This equation is the stochastic analog of the semi-group consistency condition in the ODEs, which is used in the Flow Maps literature ([6 , 59 ] ).
These two conditions define the SDE Eq. (1) and vice versa, the weak solution of an SDE satisfies both these conditions. The first statement we prove in Section A.1 using the classical result of probability theory ([54 ] ), the second statement is a textbook statement.
Proposition 1 (Identification of the transition kernel). Assume that g s g_s g s is continuous in s s s and that v s ( x ) v_s(x) v s ( x ) is continuous, locally Lipschitz in x x x , and of at most linear growth, uniformly for s s s in compacts. Let p s , t ( y ∣ x ) p_{s,t}(y \mid x) p s , t ( y ∣ x ) be a measurable family of nonnegative, normalized conditional densities. Suppose it satisfies the boundary conditions in Eqs. (7) to (9) , with limits locally uniform in ( s , x ) (s,x) ( s , x ) , and the Kolmogorov–Chapman equation in Eq. (10) . Then p s , t ( y ∣ x ) p_{s,t}(y \mid x) p s , t ( y ∣ x ) coincides (almost everywhere) with the transition density of the SDE in Eq. (1) . Conversely, the SDE’s transition kernel satisfies both the boundary conditions and Eq. (10) .
Kolmogorov–Chapman Constraints (Our Objective)
We parameterize our transition kernel q s , t ( y ∣ x ; θ ) q_{s,t}(y \mid x;\theta) q s , t ( y ∣ x ; θ ) such that the boundary conditions are satisfied (see the following section; Section B.2 details the case for learning the drift instead), which leaves the Kolmogorov–Chapman equation as the sole optimization objective. The most straightforward way to train the model would be to minimize the discrepancy between both sides of the Kolmogorov–Chapman equation for the parameterized model, i.e.
L K C ( θ ) = D K L ( ∫ d x ′ q r , t ( y ∣ x ′ ; θ ) q s , r ( x ′ ∣ x ; θ ) ∥ q s , t ( y ∣ x ; θ ) ) , ∀ x , t , s , r . (11) \mathcal{L}_{\mathrm{KC}}(\theta) = D_{\mathrm{KL}}\left(\int\mathrm{d}x' \, q_{r,t}(y \mid x';\theta)q_{s,r}(x' \mid x;\theta) \,\middle\|\, q_{s,t}(y \mid x;\theta)\right), \ \forall x,t,s,r. \tag*{(11)} L KC ( θ ) = D KL ( ∫ d x ′ q r , t ( y ∣ x ′ ; θ ) q s , r ( x ′ ∣ x ; θ ) q s , t ( y ∣ x ; θ ) ) , ∀ x , t , s , r . (11)
However, as written, this function does not have an unbiased estimator of the gradient. Therefore, we propose to use the “stop-gradient trick” ubiquitously employed in the generative modeling literature ([13 , 6 ] ). Setting η = stopgrad ( θ ) \eta= \operatorname{stopgrad}(\theta) η = stopgrad ( θ ) as the parameters in the first argument of the KL-divergence, we get (see Section B.1)
L K C ( θ ) = − E y [ log q s , t ( y ∣ x ; θ ) ] , where y ∼ q r , t ( y ∣ x ′ ; η ) , and x ′ ∼ q s , r ( x ′ ∣ x ; η ) . (12) \mathcal{L}_{\mathrm{KC}}(\theta) = -\mathbb{E}_{y}\left[\log q_{s,t}(y \mid x;\theta)\right], \ \text{where } y \sim q_{r,t}(y \mid x';\eta), \text{ and } x' \sim q_{s,r}(x' \mid x;\eta). \tag*{(12)} L KC ( θ ) = − E y [ log q s , t ( y ∣ x ; θ ) ] , where y ∼ q r , t ( y ∣ x ′ ; η ) , and x ′ ∼ q s , r ( x ′ ∣ x ; η ) . (12)
The objective in Eq. (11) , along with the parameterization that satisfies the boundary conditions, is globally minimized by the transition kernel of the SDE Eq. (1) , as shown by a classical result in stochastic processes Section A.2.
Probability-flow objective. An alternative objective follows from the forward Kolmogorov equation written as a probability-flow ODE. Using the model’s transition score, we define the velocity
v P F ( y ) = v t ( y ) − 1 2 g t g t ⊤ ∇ y log q s , t ( y ∣ x ; θ ) . (13) v_{\mathrm{PF}}(y) = v_t(y) - \frac{1}{2} g_t g_t^{\top} \nabla_y \log q_{s,t}(y \mid x;\theta). \tag*{(13)} v PF ( y ) = v t ( y ) − 2 1 g t g t ⊤ ∇ y log q s , t ( y ∣ x ; θ ) . (13)
For a generated sample y θ y_\theta y θ , define the average velocity u θ = ( y θ − x ) / τ u_\theta= (y_\theta- x)/\tau u θ = ( y θ − x ) / τ . Since d y θ d τ = u θ + τ ∂ u θ ∂ τ \frac{\mathrm{d}y_\theta}{\mathrm{d}\tau} = u_\theta+ \tau\frac{\partial u_\theta}{\partial\tau} d τ d y θ = u θ + τ ∂ τ ∂ u θ , using the same "stop-gradient trick" and matching the sample velocity to v P F v_{\mathrm{PF}} v PF gives
L P F ( θ ) = E y [ ∥ u θ − stopgrad ( v P F ( y θ ) − τ d u θ d τ ) ∥ 2 ] , y ∼ q s , t ( y ∣ x ; θ ) and ∀ x , s < t . (14) \mathcal{L}_{\mathrm{PF}}(\theta) = \mathbb{E}_y \left[\left\|u_\theta- \operatorname{stopgrad}\left(v_{\mathrm{PF}}(y_\theta) - \tau\frac{\mathrm{d}u_\theta}{\mathrm{d}\tau}\right)\right\|^2\right], \qquad y \sim q_{s,t}(y \mid x;\theta) \text{ and } \forall x,\ s < t. \tag*{(14)} L PF ( θ ) = E y [ u θ − stopgrad ( v PF ( y θ ) − τ d τ d u θ ) 2 ] , y ∼ q s , t ( y ∣ x ; θ ) and ∀ x , s < t . (14)
However, we use the KC-KL objective in Eq. (12) as our primary training objective and compare it with the probability-flow objective in our experiments.
Boundary Constraints (Our Parameterization)
We parameterize our transition kernel q s , t ( y ∣ x ; θ ) q_{s,t}(y \mid x;\theta) q s , t ( y ∣ x ; θ ) as a normalizing flow with time-scaled prior and time-scaled affine coupling layers. This guarantees the boundary constraints for t → s t \to s t → s and puts no restrictions on larger time scales. For simplicity, we consider the autonomous SDE with the drift v ( x ) v(x) v ( x ) and diffusion coefficient g g ⊤ gg^\top g g ⊤ , then our model is
y = x + τ v ( x ) + Ψ ( τ ⋅ g z ; x , θ ) , z ∼ N ( z ∣ 0 , I ) , (15) y = x + \tau v(x) + \Psi(\sqrt{\tau} \cdot gz; x,\theta), \qquad z \sim\mathcal{N}(z \mid0,I), \tag*{(15)} y = x + τ v ( x ) + Ψ ( τ ⋅ g z ; x , θ ) , z ∼ N ( z ∣ 0 , I ) , (15)
where τ = t − s \tau= t - s τ = t − s is the new time variable for the autonomous system and Ψ ( ⋅ ; x , θ ) \Psi(\cdot; x,\theta) Ψ ( ⋅ ; x , θ ) is the normalizing flow forward map. Importantly, we add time-gates to the definition of the affine coupling layers, which then become defined as
x 1 ( k + 1 ) = x 1 ( k ) , (16) x_1^{(k+1)} = x_1^{(k)}, \tag*{(16)} x 1 ( k + 1 ) = x 1 ( k ) , (16)
x 2 ( k + 1 ) = x 2 ( k ) ⊙ exp ( ( ω τ ) n a 1 + ( ω τ ) n a ⋅ a k ( x 1 ( k ) ; x , θ ) ) + ( ω τ ) n b 1 + ( ω τ ) n b ⋅ b k ( x 1 ( k ) ; x , θ ) , (17) x_2^{(k+1)} = x_2^{(k)} \odot\exp\left(\frac{(\omega\tau)^{n_a}}{1+(\omega\tau)^{n_a}} \cdot a_k(x_1^{(k)};x,\theta)\right) + \frac{(\omega\tau)^{n_b}}{1+(\omega\tau)^{n_b}} \cdot b_k(x_1^{(k)};x,\theta), \tag*{(17)} x 2 ( k + 1 ) = x 2 ( k ) ⊙ exp ( 1 + ( ω τ ) n a ( ω τ ) n a ⋅ a k ( x 1 ( k ) ; x , θ ) ) + 1 + ( ω τ ) n b ( ω τ ) n b ⋅ b k ( x 1 ( k ) ; x , θ ) , (17)
where ω \omega ω , n a n_a n a and n b n_b n b are hyperparameters of the gates. The key property of the gates is that they behave as ( ω τ ) n (\omega\tau)^n ( ω τ ) n for small τ \tau τ , which controls the boundary conditions and converge to 1 1 1 for large τ \tau τ , which implies full expressivity of the model at large time-scales.
Proposition 2. (Time-gates satisfy the boundary constraints) Let q s , t ( y ∣ x ; θ ) q_{s,t}(y \mid x;\theta) q s , t ( y ∣ x ; θ ) be the density of the random variable y y y defined in Eq. (15) , and the map Ψ = Ψ K ∘ … ∘ Ψ 1 \Psi= \Psi_K \circ\ldots\circ\Psi_1 Ψ = Ψ K ∘ … ∘ Ψ 1 consists of the composition of affine coupling layers with time-gates of powers n a n_a n a and n b n_b n b . Assuming that a k a_k a k and b k b_k b k are uniformly bounded over their inputs and sufficiently small τ > 0 \tau> 0 τ > 0 , for
n a > 1 2 , and n b > 1 , (18) n_a > \frac{1}{2}, \qquad\text{and } n_b > 1, \tag*{(18)} n a > 2 1 , and n b > 1 , (18) q s , t ( ⋅ ∣ x ; θ ) q_{s,t}(\cdot\mid x;\theta) q s , t ( ⋅ ∣ x ; θ ) satisfies the boundary conditions in Eqs. (7) to (9) . See proof in Section A.3.
Second-order dynamics. For second-order Langevin dynamics, we use unit masses in the following expressions for simplicity and extend the parameterization to the joint source state x = ( X , P ) x = (X,P) x = ( X , P ) ,
y = x + m ( x , τ ) + Ψ ( L ( τ ) z ; x , θ ) , z ∼ N ( z ∣ 0 , I ) , (19) y = x + m(x,\tau) + \Psi(L(\tau)z;x,\theta), \qquad z \sim\mathcal{N}(z \mid0,I), \tag*{(19)} y = x + m ( x , τ ) + Ψ ( L ( τ ) z ; x , θ ) , z ∼ N ( z ∣ 0 , I ) , (19)
where m ( x , τ ) m(x,\tau) m ( x , τ ) and L ( τ ) L ( τ ) ⊤ L(\tau)L(\tau)^\top L ( τ ) L ( τ ) ⊤ are mean displacement and covariance of the Gaussian reference obtained holding the vector field at v ( X ) v(X) v ( X ) . The expressions for m m m and L L L are derived in Section B.3.2. Moreover, we additionally scale the position shift with a free-flight term ℓ i ( τ ) \ell_i(\tau) ℓ i ( τ ) where b k , i = ℓ i ( τ ) b ~ k , i b_{k,i} = \ell_i(\tau)\tilde{b}_{k,i} b k , i = ℓ i ( τ ) b ~ k , i , where the factor scales like O ( τ ) O(\tau) O ( τ ) at small τ \tau τ . We provide more details in Section B.3.3.
The coupling layers in Eq. (17) alternate between position and momentum updates. We use ( n a , n b ) = ( 3 , 3 ) (n_a,n_b) = (3,3) ( n a , n b ) = ( 3 , 3 ) for position updates, and ( n a , n b ) = ( 2 , 3 ) (n_a,n_b) = (2,3) ( n a , n b ) = ( 2 , 3 ) for momentum updates. This choice of gate orders preserves the boundary constraints for the second-order SDE (see Section B.3.3).
Practical Design Choices
We use the design choices listed below to improve training stability and accuracy of learned transitions.
Stabilizing the Gaussian reference. To stabilize sampling over longer intervals, we use a capped duration h ( τ ) h(\tau) h ( τ ) to limit transport, force contributions, and position noise in the Gaussian reference. We set h ( τ ) = τ h(\tau) = \tau h ( τ ) = τ for τ ≤ τ 0 \tau\leq\tau_0 τ ≤ τ 0 and h ( τ ) = τ 0 + δ tanh ( ( τ − τ 0 ) / δ ) h(\tau) = \tau_0 + \delta\tanh((\tau- \tau_0)/\delta) h ( τ ) = τ 0 + δ tanh (( τ − τ 0 ) / δ ) otherwise, so the duration saturates at τ 0 + δ \tau_0 + \delta τ 0 + δ . The reference is unchanged near τ = 0 \tau= 0 τ = 0 , preserving its boundary conditions. We give the exact rescaling in Section C.1.
Furthermore, two-step targets in Eq. (11) are formed from shorter transitions, motivating a more accurate short-time reference. For source position X X X and momentum P P P , we evaluate the vector field at ν ( X m i d ) \nu(X_{\mathrm{mid}}) ν ( X mid ) , where X m i d = X + 1 2 h ( τ ) P X_{\mathrm{mid}} = X + \frac{1}{2}h(\tau)P X mid = X + 2 1 h ( τ ) P (see details in Section B.3). We ablate duration capping and midpoint evaluation separately in Table 1 a.
Table 1. Table 1: Ablation study on AL2. NLL on samples from MD simulation and energy E \mathcal{E} E -W 2 W_2 W 2 (k B T k_{\mathrm{B}}T k B T ) over 64 source states with 256 samples per state compared to 256 independent MD trajectories with a 0.1 fs timestep. Results are averaged over τ = 10 , 20 , … , 100 \tau= 10, 20, \ldots, 100 τ = 10 , 20 , … , 100 fs, with lower values being better. Each subtable compares the full model with variants that change only the indicated component, keeping all other settings fixed. All models were trained with a reduced compute budget of 250K steps. Best value in each column within each subtable is bold .
Time prior. During training, we sample times from a mixture τ ∼ w LogUniform [ T min , T max ] + ( 1 − w ) Uniform [ T min , T max ] \tau\sim w\,\operatorname{LogUniform}[T_{\min}, T_{\max}] + (1-w)\,\operatorname{Uniform}[T_{\min}, T_{\max}] τ ∼ w LogUniform [ T m i n , T m a x ] + ( 1 − w ) Uniform [ T m i n , T m a x ] , where T min T_{\min} T m i n and T max T_{\max} T m a x are the minimum and maximum sampled times, with 0 < T min < T max 0 < T_{\min} < T_{\max} 0 < T m i n < T m a x and w ∈ [ 0 , 1 ] w \in[0,1] w ∈ [ 0 , 1 ] . The uniform component upweights longer intervals, while T max T_{\max} T m a x is gradually increased early in training so the model first learns shorter transitions. We compare this mixture against either distribution alone in Table 1 b.
Input features. Alongside the source state and elapsed time τ \tau τ , we condition the flow on the Gaussian reference mean displacement and covariance. Flow inputs are normalized by the reference marginal standard deviations for predicting affine scale and shift. We provide more details in Section C.1 and separately ablate mean and covariance features and input normalization in Table 1 c.
Normalizing flow design. The fixed rate ω \omega ω controls the time τ \tau τ at which the flow corrections take effect, without changing their prescribed short-time orders. We set this rate at ω = 333 p s − 1 \omega= 333\ \mathrm{ps}^{-1} ω = 333 ps − 1 from empirical analysis. For second-order dynamics, we also compare our gate choices with a lower-order alternative, labeled “Lower-order gates,” which uses n a = 1 n_a = 1 n a = 1 for both log scales and n b = 2 n_b = 2 n b = 2 for both shifts, as shown in Table 1 d. Finally, we compare the displacement parameterization used here with standard and whitened flows on AL2. We briefly describe these alternatives and their corresponding gates in Section E.7, with results in Figure A8 .
Figure A8. Flow parameterization on AL2. Comparison of standard, displacement, and whitened flows, each trained for 500K steps with batch size 512 and evaluated using EMA parameters with decay 0.999. Lines and shaded bands show the mean and standard deviation across training seeds: 42, 43, and 44 for all three parameterizations. Left: NLL per degree of freedom (DoF) on molecular dynamics samples. Right: E \mathcal{E} E -W 2 \mathcal{W}_2 W 2 in units of k B T k_{\mathrm{B}}T k B T , averaged over source configurations.
Equivariance. To account for global translation and rotation symmetries, we center positions at the center of mass and apply the flow in an orthonormal frame determined by the source configuration ([23 , 22 ] ). We rotate the output back to the original orientation. The frame construction and treatment of the center-of-mass constraint are detailed in Section C.1.
Stabilizing training. To stabilize learning from model-generated targets, we use η = EMA ( θ ) \eta= \operatorname{EMA}(\theta) η = EMA ( θ ) in L K C \mathcal{L}_{\mathrm{KC}} L KC . We compare different EMA decay rates, including η = stopgrad ( θ ) \eta= \operatorname{stopgrad}(\theta) η = stopgrad ( θ ) , in Figure A7 .
Figure A7. Target EMA decay on AL2. Models differ only in ema_decay for the two-step target η = EMA ( θ ) \eta= \operatorname{EMA}(\theta) η = EMA ( θ ) in Eq. (12) . All models are trained for 500K steps with batch size 512. We report NLL on MD transition samples at τ ∈ { 10 , 20 , 50 , 100 } \tau\in\{10, 20, 50, 100\} τ ∈ { 10 , 20 , 50 , 100 } fs. A decay of zero corresponds to η = stopgrad ( θ ) \eta= \operatorname{stopgrad}(\theta) η = stopgrad ( θ ) .
Detailed-Balance Loss. The Kolmogorov–Chapman objective constrains how the kernel composes across τ \tau τ but does not ensure that stationary distribution is π \pi π . Since π \pi π is known, we add a detailed-balance objective as done in Klein et al. [27 ] , which holds for underdamped Langevin dynamics up to momentum reversal R ( X , P ) = ( X , − P ) R(X,P) = (X,-P) R ( X , P ) = ( X , − P ) , and implies that π \pi π is the stationary distribution of the kernel:
L = L K C + λ L D B , L D B = D K L ( q s , t ( y ∣ x ) π ( x ) ∥ q s , t ( R x ∣ R y ) π ( y ) ) . (20) \mathcal{L} = \mathcal{L}_{\mathrm{KC}} + \lambda\mathcal{L}_{\mathrm{DB}}, \qquad\mathcal{L}_{\mathrm{DB}} = D_{\mathrm{KL}}\left(q_{s,t}(y \mid x)\pi(x)\,\middle\|\,q_{s,t}(Rx \mid Ry)\pi(y)\right). \tag*{(20)} L = L KC + λ L DB , L DB = D KL ( q s , t ( y ∣ x ) π ( x ) ∥ q s , t ( R x ∣ R y ) π ( y ) ) . (20)
We optimize L D B \mathcal{L}_{DB} L D B with a low-variance gradient estimator following Tucker et al. [57 ] . The detailed-balance loss is only applied towards the end of training with more details in Section D.3.
Inference-time stabilization. Following the use of inference-time corrections in HFM Ripken et al. [49 ] , we apply additional corrections to limit accumulated errors during long simulations. We describe these corrections in Section C.3 and summarize a simulation step in Algorithm 2.
Experiments
Baselines
Throughout our experiments, we primarily consider Langevin dynamics as a baseline and other trajectory-free methods at the target distribution in our comparisons. Long reference simulations provide the transition samples and equilibrium statistics used for evaluation. For training, we assume we have access to the unpaired initial states x ∼ π ( x ) x \sim\pi(x) x ∼ π ( x ) . We match training FLOPs between KC-KL and PF-ODE on Lennard-Jones systems and between our method and HFM on AL4 and AL6 (see Table A4 ). For AL2, we use the released HFM checkpoint Ripken et al. [49 ] .
Table A4. Training compute on AL4 and AL6. Parameter counts, completed training updates and estimated FLOPs for the loss and parameter gradients. FLOP totals follow the accounting described above.
Table 2 : Performance of KC-maps on LJ-13 and LJ-55. Published baseline results are taken from Potaptchik et al. [47 ] . Metrics are the 2-Wasserstein distance between energy distributions (E \mathcal{E} E -W 2 \mathcal{W}_2 W 2 ) and the geometric 2-Wasserstein distance (Geo-W 2 \mathcal{W}_2 W 2 ). In all settings, the target temperature is T = 1 T = 1 T = 1 ; best values are bolded and the second best values are <u>underlined</u>. Dashes indicate unavailable results.
Table 2. Performance of KC-maps on LJ-13 and LJ-55. Published baseline results are taken from Potaptchik et al. [47 ] . Metrics are the 2-Wasserstein distance between energy distributions (E \mathcal{E} E -W 2 \mathcal{W}_2 W 2 ) and the geometric 2-Wasserstein distance (Geo-W 2 \mathcal{W}_2 W 2 ). In all settings, the target temperature is T = 1 T = 1 T = 1 ; best values are bolded and the second best values are underlined. Dashes indicate unavailable results.
Lennard-Jones
To demonstrate our method to learn the map for the first-order SDE, we consider the Lennard-Jones 13-particle (LJ-13) and 55-particle (LJ-55) systems. Building on the setup in Akhound-Sadegh et al. [2 ] , we train our flow to model transitions of particle positions at the target temperature T = 1 T = 1 T = 1 and diffusion coefficient D = g 2 2 = 0.02 D = \frac{g^2}{2} = 0.02 D = 2 g 2 = 0.02 . We compare two different training settings: (i) configurations sampled at the target temperature for the initial state during training, and (ii) configurations sampled from a higher temperature T S = 4 T_S = 4 T S = 4 as the initial states, motivated by what is done in Akhound-Sadegh et al. [2 ] ; Potaptchik et al. [47 ] . In both settings, the vector field and diffusion correspond to the SDE at the target temperature. This comparison demonstrates that our method can learn the target dynamics without requiring training configurations from the target distribution.
We evaluate 2-Wasserstein distances on 1D energy histograms (E \mathcal{E} E -W 2 \mathcal{W}_2 W 2 ) and 3D coordinates (Geo-W 2 \mathcal{W}_2 W 2 ). We compare Metropolis–Hastings sampling Hastings [15 ] with our learned kernel against MALA (Roberts & Tweedie [50 ] ) and results from Tilt Matching Potaptchik et al. [47 ] . Results are reported in Table 2 , with sampling and evaluation details in Section D.4.
Molecular Systems
We evaluate our second-order parameterization on three molecular systems of increasing size: alanine dipeptide (AL2), tetrapeptide (AL4), and hexapeptide (AL6). We train the flow to model transitions of the joint state of positions and momenta under Langevin dynamics. Training uses uncorrelated equilibrium configurations as initial states and the molecular force field, without requiring paired states from MD. We generate the trajectories by sequentially applying the learned transition kernel. with step size τ \tau τ , and we compare the resulting equilibrium distribution against HFM Ripken et al., [49 ] and molecular dynamics simulated using BAOAB scheme Leimkuhler & Matthews, [33 ] .
For the peptide experiments, we train using L K C \mathcal{L}_{\mathrm{KC}} L KC with the detailed-balance loss L D B \mathcal{L}_{\mathrm{DB}} L DB . We use L K C \mathcal{L}_{\mathrm{KC}} L KC since we found training with L P F \mathcal{L}_{\mathrm{PF}} L PF unstable, even with adaptive loss weighting [13 , 59 ] . The LJ-55 results in Table 2 also show substantially larger errors, especially with energy distribution, when training with L P F \mathcal{L}_{\mathrm{PF}} L PF .
Equilibrium. We compare generated samples with long MD references using 2-Wasserstein distances in energy (E \mathcal{E} E -W 2 W_2 W 2 ), torsion angles (T \mathcal{T} T -W 2 W_2 W 2 ), and first two component projections from time-lagged independent component analysis (TICA-W 2 W_2 W 2 ), as done in [55 , 56 ] .
In Table 3 , we evaluate all methods under an equal budget of one GPU hour. For each method, we vary the number of parallel chains and, for the learned models, the step size τ \tau τ , and report the
Table 3. Equilibrium sampling of alanine peptides per method with a fixed budget of one GPU-hour on single L40S, scored against each molecule’s long MD. W 2 W_2 W 2 metrics are computed between 10 4 10^4 1 0 4 reference frames and 10 4 10^4 1 0 4 frames drawn from the run after a 10 % burn-in, and report the mean ± \pm ± std. over 3 draws. Best of HFM and KC-maps in bold .
configuration with the best average across all three metrics. Sampling and evaluation details are provided in Section D.4.
At matched GPU-hour budgets, KC-maps achieves the lowest TICA-W 2 W_2 W 2 on all three systems while remaining competitive on energy and torsion metrics. We report performance across τ \tau τ with and without the detailed-balance loss in Figure A1 . Longer rollouts of KC-maps show broad mode coverage and closely match the reference energy distribution (Figure 2 ). For AL6, in particular, these rollouts reach modes that are absent in the training data, demonstrating exploration beyond the training conformations (see Figure A2 ).
Figure 2. Equilibrium sampling of alanine peptides. Density panels show the reference MD (16 parallel chains × \times × 2 μ \mu μ s per chain) beside KC-maps (Ours). Potential energy histograms of KC-maps against each system’s MD reference. (a) AL2: Ramachandran and TICA densities, energy histogram, and TIC0 against simulated MD trajectory (top) and ours (bottom). (b) AL4: TICA densities and energy histogram. (c) AL6: TICA densities and energy histogram. For each molecule, samples generated from KC-maps were run with varying compute budgets (1 GPU-hour for AL2; 22 GPU-hour for AL4; 40 GPU-hours for AL6).
Figure A1. Equilibrium results across step size. Equilibrium metrics for KC-maps with and without detailed-balance loss against reference long MD.
Figure A2. AL6 sampling beyond the training distribution. TICA projections of reference long MD, training data, and KC-maps samples. The KC-maps samples are pooled from 16 chains of approximately 400 ns each. The lower-left mode is absent from the training configurations but visited by KC-maps.
Furthermore, we evaluate exploration efficiency on AL4 and AL6 through aggregate simulated time and number of modes discovered within one GPU-hour. As shown in Figure 3 , KC-maps’s throughput increases with the number of chains on a single GPU, exceeding MD and HFM at larger chain counts. Within the same compute budget, KC-maps also discovers the modes faster and samples more modes than either baseline.
Figure 3. One GPU-hour on single L40S. (a) TICA densities of the reference (16 chains; d t = 1 f s dt = 1\mathrm{fs} d t = 1 fs ; 2 μ s 2\,\mu\mathrm{s} 2 μ s each), MD (16 parallel chains; d t = 1 f s dt = 1\mathrm{fs} d t = 1 fs ; one GPU-hour), HFM (16 chains; τ = 5 f s \tau= 5\mathrm{fs} τ = 5 fs ; one GPU-hour) and KC-maps (16 chains; τ = 30 f s \tau= 30\mathrm{fs} τ = 30 fs ; one GPU-hour). KC-maps consistently visits most if not all of the modes within the same GPU-hour. (b) Left: simulated aggregate time per GPU-hour against the number of parallel chains for KC-maps (τ = 30 f s \tau= 30\mathrm{fs} τ = 30 fs for AL4 and AL6), MD and HFM (7 f s 7\mathrm{fs} 7 fs for AL4, 5 f s 5\mathrm{fs} 5 fs for AL6). Right: metastable TICA modes discovered against GPU time. Dotted: modes of the reference MD.
Kinetics. Here, we assess the kinetics of KC-maps on AL2. We measure mean first-passage time (MFPT) between modes and the autocorrelation of mode occupancy. As shown in Figure 4 , KC-maps closely matches MD for MFPT between α \alpha α , β \beta β , and ppII, as well as their autocorrelation curves for these modes.
Figure 4. AL2 kinetics. (a) The four metastable modes (α \alpha α , β \beta β , ppII, α L \alpha_L α L ) on the reference MD Ramachandran plot, with their populations. (b) Mean first-passage time between modes. (c) Autocorrelation C ( t ) C(t) C ( t ) of mode occupancy. MD: 16 chains of 2 μ \mu μ s each; KC-maps: 16 chains at τ = 30 \tau= 30 τ = 30 fs run for one GPU-hour on L40S. Both metrics are measured on trajectories with frames saved every 1 ps.
We observe the largest errors when involving α L \alpha_L α L : KC-maps overestimates MFPT into this mode, and its autocorrelation decays faster at long lags. We believe these errors to stem from the limited support for that mode in the training data. Namely, α L \alpha_L α L accounts for only 0.7% of training configurations, compared with 2.2% in the reference MD. We define the kinetic metrics in Section D.4 and provide additional results on transition probabilities in Figure A6 .
Figure A6. AL2 transition probabilities. P i j ( t ) P_{ij}(t) P ij ( t ) from mode i i i (rows) to mode j j j (columns) against the lag t t t . Shaded bands are 95% bootstrap intervals over chains.
Related Work
Boltzmann generators ([42 , 26 , 55 , 56 ] ) directly sample uncorrelated equilibrium configurations without integrating MCMC. However, equilibrium distributions alone do not capture dynamical properties such as relaxation times and transition rates [10 ] , motivating surrogates that model time-coarsened MD transitions.
Hamiltonian Flow Maps [49 ] learn deterministic updates from Hamiltonian equations using uncorrelated states and forces, and apply a separate thermostat update during simulation as done in the OBABO scheme (Bussi & Parrinello, 2007). Similarly, we leverage local physical information, but learn the stochastic transition kernel of Langevin dynamics.
Consistency models [53 , 38 ] and Flow Map methods [5 , 13 , 6 ] learn finite-time maps of generative ODEs. Stochastic Flow Maps (SFMs) [48 , 19 ] extend this framework to conditional distributions, while subsequent works learn pathwise SDE solution maps conditioned on Brownian motion [40 , 44 ] . Our method falls within the SFM framework, directly parameterizing the transition kernel and enforcing its composition through the Kolmogorov–Chapman equation.
Lastly, similar works like Neural Stochastic Flows (NSF) [25 ] also parameterize the SDE transition kernel with a conditional normalizing flows. However, their objective combines likelihood on observed transition pairs with a variational upper bound on bidirectional KL regularization evaluated through a learned bridge model. KC-maps learns from unpaired states and the known SDE drift and diffusion. Unlike NSF, our parameterization preserves the prescribed infinitesimal dynamics, while the consistency objective uses transitions generated by composing the model itself. This allows us to learn finite-time transition distributions without observed future states.
Conclusion
Our work introduces KC-maps, a trajectory-free approach to learning transition kernels of SDEs. Our method combines a parameterization that preserves the infinitesimal dynamics with a training objective based on the Kolmogorov–Chapman consistency, using uncorrelated initial states and the known drift and diffusion. On molecular systems, we learn transitions of up to 100 fs and demonstrate improved equilibrium sampling efficiency compared with classical molecular dynamics, combining higher simulation throughput with broader conformational coverage at matched GPU-hour budgets.
A limitation of our method is its dependence on training-data coverage. In particular, the method’s kinetics suffers in performance in modes that are sparsely represented in the training set. Also, our method is trained separately for each molecular system and does not currently support transfer to unseen systems. Future work could address these limitations by amortizing across molecular systems, scaling to larger systems, and learning longer time horizons more accurately. We believe training an amortized model across systems may help improve performance in sparsely covered modes through shared structural and dynamical information across systems. Finally, our Lennard-Jones results motivate learning a map by leveraging higher-temperature data of molecular systems, without requiring equilibrium samples at the target temperature.
Acknowledgments
M.H. acknowledges the support of the Natural Sciences and Engineering Research Council of Canada (NSERC), [funding reference number RGPIN-2026-05517]. M.S. was supported by the IVADO 2025 Postdoctoral Research Funding Program. This project was undertaken thanks to funding from IVADO and the Canada First Research Excellence Fund.
Reproducibility Statement
We provide code at https://github.com/majhas/kc-maps . We also fully detail the training implementation and evaluation setup with details on hyperparameters in Section C. Proofs of theoretical results are in Section A and derivations in Section B.
Proofs
Identification of the transition kernel
Here we show the proof of a classical result for the identification of the kernel [54 ] .
Proof of Proposition 1. Fix s < t s < t s < t and an initial state x x x . Let τ = t − s \tau= t - s τ = t − s , set t j = s + j τ / n t_j = s + j\tau/n t j = s + j τ / n , and define the Markov chain
X s ( n ) = x , X t j + 1 ( n ) ∣ X t j ( n ) ∼ p t j , t j + 1 ( ⋅ ∣ X t j ( n ) ) , j = 0 , … , n − 1. X_s^{(n)} = x,\qquad X_{t_{j+1}}^{(n)} \mid X_{t_j}^{(n)} \sim p_{t_j,t_{j+1}}(\mathord{\cdot} \mid X_{t_j}^{(n)}),\qquad j = 0,\ldots,n-1. X s ( n ) = x , X t j + 1 ( n ) ∣ X t j ( n ) ∼ p t j , t j + 1 ( ⋅ ∣ X t j ( n ) ) , j = 0 , … , n − 1.
By repeated application of Kolmogorov–Chapman, X t ( n ) X_t^{(n)} X t ( n ) has density p s , t ( ⋅ ∣ x ) p_{s,t}(\mathord{\cdot} \mid x) p s , t ( ⋅ ∣ x ) for every n n n .
The boundary conditions in Eqs. (7) to (9) are exactly the assumption of the diffusion approximation [54 ] . Its remaining hypothesis, well-posedness of the martingale problem for ( v , g g T ) (v,gg^{\mathsf T}) ( v , g g T ) , follows from the assumptions on v v v and g g g in Proposition 1: they give a pathwise unique, non-explosive solution of Eq. (1) , hence uniqueness in law, which is equivalent to well-posedness [54 ] . Therefore, this gives weak convergence of the linearly interpolated chains ( X r ( n ) ) r ∈ [ s , t ] (X_r^{(n)})_{r\in[s,t]} ( X r ( n ) ) r ∈ [ s , t ] to ( X r ) r ∈ [ s , t ] (X_r)_{r\in[s,t]} ( X r ) r ∈ [ s , t ] , where ( X r ) r ∈ [ s , t ] (X_r)_{r\in[s,t]} ( X r ) r ∈ [ s , t ] solves the SDE in Eq. (1) with initial state X s = x X_s = x X s = x . Note that to apply the diffusion approximation to time-dependent systems, we can simply apply the result to ( r , X r ) (r,X_r) ( r , X r ) , treating time as part of the state. We first establish convergence within some bounded region where the boundary condition errors vanish uniformly as n → ∞ n \to\infty n → ∞ . Second, since the SDE has continuous trajectories defined through [ s , t ] [s,t] [ s , t ] , the probability of leaving the bounded region tends to zero as the region extends to the full state space. Therefore, the localization argument gives convergence over the full interval.
Weak convergence thus gives, for every bounded continuous f f f ,
∫ d y f ( y ) p s , t ( y ∣ x ) = E [ f ( X t ( n ) ) ] ⟶ E [ f ( X t ) ∣ X s = x ] . \int dy\, f(y)\,p_{s,t}(y \mid x) = \mathbb{E}\left[f\left(X_t^{(n)}\right)\right] \longrightarrow\mathbb{E}\left[f(X_t)\mid X_s=x\right]. ∫ d y f ( y ) p s , t ( y ∣ x ) = E [ f ( X t ( n ) ) ] ⟶ E [ f ( X t ) ∣ X s = x ] .
The left-hand side is independent of n n n , so p s , t ( ⋅ ∣ x ) p_{s,t}(\mathord{\cdot} \mid x) p s , t ( ⋅ ∣ x ) is the SDE’s transition density, up to equality almost everywhere. Since s s s , t t t , and x x x were arbitrary, this identifies the entire transition kernel.
Conversely, the SDE’s transition kernel satisfies the boundary conditions Eqs. (7) to (9) , and its Markov property gives Kolmogorov–Chapman consistency. For the proof, we refer the reader to a textbook [24 ] .
Global minimizers of the KC divergence
Proof of global minimizer. Here, L K C \mathcal{L}_{\mathrm{KC}} L KC denotes the KL divergence in Eq. (11) , averaged over the sampled source states and time triples. KL divergence is nonnegative, so L K C ( θ ) ≥ 0 \mathcal{L}_{\mathrm{KC}}(\theta) \ge0 L KC ( θ ) ≥ 0 . The SDE’s transition kernel p p p satisfies Kolmogorov–Chapman consistency and belongs to the model class. When
q s , t ( y ∣ x ; θ ) = ∫ d x ′ q r , t ( y ∣ x ′ ; θ ) q s , r ( x ′ ∣ x ; θ ) , q_{s,t}(y \mid x;\theta) = \int dx'\,q_{r,t}(y \mid x';\theta)q_{s,r}(x' \mid x;\theta), q s , t ( y ∣ x ; θ ) = ∫ d x ′ q r , t ( y ∣ x ′ ; θ ) q s , r ( x ′ ∣ x ; θ ) ,
the parameterized kernel achieves L K C = 0 \mathcal{L}_{\mathrm{KC}} = 0 L KC = 0 , giving min θ L K C ( θ ) = 0 \min_\theta\mathcal{L}_{\mathrm{KC}}(\theta) = 0 min θ L KC ( θ ) = 0 .
Now suppose L K C ( θ ) = 0 \mathcal{L}_{\mathrm{KC}}(\theta) = 0 L KC ( θ ) = 0 . The nonnegative KL integrand in Eq. (11) must vanish almost everywhere under the joint sampling distribution. Continuity and full support imply that it vanishes for every initial state and time triple s < r < t s < r < t s < r < t .
Since the target and learned transition kernels match, zero KL divergence gives
q s , t ( y ∣ x ; θ ) = ∫ d x ′ q r , t ( y ∣ x ′ ; θ ) q s , r ( x ′ ∣ x ; θ ) (21) q_{s,t}(y \mid x;\theta) = \int dx'\,q_{r,t}(y \mid x';\theta)q_{s,r}(x' \mid x;\theta) \tag*{(21)} q s , t ( y ∣ x ; θ ) = ∫ d x ′ q r , t ( y ∣ x ′ ; θ ) q s , r ( x ′ ∣ x ; θ ) (21)
for every x x x and s < r < t s < r < t s < r < t , with equality almost everywhere in y y y . Thus, the learned kernel satisfies Kolmogorov–Chapman consistency. By assumption, it also satisfies the boundary conditions with limits locally uniform in ( s , x ) (s,x) ( s , x ) . Applying Proposition 1 yields
q s , t ( ⋅ ∣ x ; θ ) = p s , t ( ⋅ ∣ x ) , ∀ x , s < t . (22) q_{s,t}(\mathord{\cdot} \mid x;\theta) = p_{s,t}(\mathord{\cdot} \mid x),\qquad\forall x,\ s < t. \tag*{(22)} q s , t ( ⋅ ∣ x ; θ ) = p s , t ( ⋅ ∣ x ) , ∀ x , s < t . (22)
Proof of Proposition 2
We use the following notation:
u : = τ g z u := \sqrt{\tau}gz u := τ g z with z ∼ N ( 0 , I ) z \sim\mathcal{N}(0,I) z ∼ N ( 0 , I ) ;
κ n ( τ ) : = ( ω τ ) n 1 + ( ω τ ) n \kappa_n(\tau) := \frac{(\omega\tau)^n}{1+(\omega\tau)^n} κ n ( τ ) := 1 + ( ω τ ) n ( ω τ ) n ;
a ˉ : = max k sup ∥ a k ∥ ∞ \bar{a} := \max_k \sup\lVert a_k\rVert_\infty a ˉ := max k sup ∥ a k ∥ ∞ and b ˉ : = max k sup ∥ b k ∥ \bar{b} := \max_k \sup\lVert b_k\rVert b ˉ := max k sup ∥ b k ∥ denote the uniform bounds on the conditioners (over x 1 ( k ) x_1^{(k)} x 1 ( k ) and x x x );
C C C denote constants depending only on K K K , ω \omega ω , a ˉ \bar{a} a ˉ , b ˉ \bar{b} b ˉ .
Lemma 1 (Gated layers are small perturbations of the identity). For all τ ∈ ( 0 , 1 ] \tau\in(0,1] τ ∈ ( 0 , 1 ] and all u u u ,
∥ Ψ ( u ) − u ∥ ≤ C ( τ n a ∥ u ∥ + τ n b ) . (23) \lVert\Psi(u)-u\rVert\le C(\tau^{n_a}\lVert u\rVert+\tau^{n_b}). \tag*{(23)} ∥ Ψ ( u ) − u ∥ ≤ C ( τ n a ∥ u ∥ + τ n b ) . (23)
Proof. Consider one layer Eq. (17) with input x = ( x 1 , x 2 ) x=(x_1,x_2) x = ( x 1 , x 2 )
∥ Ψ k ( x ) − x ∥ = ∥ x 2 ⊙ ( e κ n a ( τ ) a k − 1 ) + κ n b ( τ ) b k ∥ ≤ ∥ x 2 ∥ κ n a ( τ ) a ˉ e a ˉ + κ n b ( τ ) b ˉ , (24) \lVert\Psi_k(x)-x\rVert=\left\lVert x_2\odot\left(e^{\kappa_{n_a}(\tau)a_k}-1\right)+\kappa_{n_b}(\tau)b_k\right\rVert\le\lVert x_2\rVert\kappa_{n_a}(\tau)\bar{a}e^{\bar{a}}+\kappa_{n_b}(\tau)\bar{b}, \tag*{(24)} ∥ Ψ k ( x ) − x ∥ = x 2 ⊙ ( e κ n a ( τ ) a k − 1 ) + κ n b ( τ ) b k ≤ ∥ x 2 ∥ κ n a ( τ ) a ˉ e a ˉ + κ n b ( τ ) b ˉ , (24)
where we use ∣ e c − 1 ∣ ≤ ∣ c ∣ e ∣ c ∣ |e^c-1|\le|c|e^{|c|} ∣ e c − 1∣ ≤ ∣ c ∣ e ∣ c ∣ . Then we have
∥ Ψ k ( x ) − x ∥ ≤ ∥ x 2 ∥ κ n a ( τ ) a ˉ e a ˉ + κ n b ( τ ) b ˉ ≤ ∥ x ∥ ( ω τ ) n a a ˉ e a ˉ + ( ω τ ) n b b ˉ (25) \lVert\Psi_k(x)-x\rVert\le\lVert x_2\rVert\kappa_{n_a}(\tau)\bar{a}e^{\bar{a}}+\kappa_{n_b}(\tau)\bar{b}\le\lVert x\rVert(\omega\tau)^{n_a}\bar{a}e^{\bar{a}}+(\omega\tau)^{n_b}\bar{b} \tag*{(25)} ∥ Ψ k ( x ) − x ∥ ≤ ∥ x 2 ∥ κ n a ( τ ) a ˉ e a ˉ + κ n b ( τ ) b ˉ ≤ ∥ x ∥ ( ω τ ) n a a ˉ e a ˉ + ( ω τ ) n b b ˉ (25)
For x ( 0 ) = u x^{(0)}=u x ( 0 ) = u and x ( k ) = Ψ k ( x ( k − 1 ) ) x^{(k)}=\Psi_k(x^{(k-1)}) x ( k ) = Ψ k ( x ( k − 1 ) ) , we have
∥ x ( k ) ∥ = ∥ Ψ k ( x ( k − 1 ) ) − x ( k − 1 ) + x ( k − 1 ) ∥ ≤ ∥ Ψ k ( x ( k − 1 ) ) − x ( k − 1 ) ∥ + ∥ x ( k − 1 ) ∥ (26) \lVert x^{(k)}\rVert=\left\lVert\Psi_k(x^{(k-1)})-x^{(k-1)}+x^{(k-1)}\right\rVert\le\lVert\Psi_k(x^{(k-1)})-x^{(k-1)}\rVert+\lVert x^{(k-1)}\rVert \tag*{(26)} ∥ x ( k ) ∥ = Ψ k ( x ( k − 1 ) ) − x ( k − 1 ) + x ( k − 1 ) ≤ ∥ Ψ k ( x ( k − 1 ) ) − x ( k − 1 ) ∥ + ∥ x ( k − 1 ) ∥ (26)
≤ ∥ x ( k − 1 ) ∥ ( ( ω τ ) n a a ˉ e a ˉ + 1 ) + ( ω τ ) n b b ˉ . (27) \le\lVert x^{(k-1)}\rVert\left((\omega\tau)^{n_a}\bar{a}e^{\bar{a}}+1\right)+(\omega\tau)^{n_b}\bar{b}. \tag*{(27)} ≤ ∥ x ( k − 1 ) ∥ ( ( ω τ ) n a a ˉ e a ˉ + 1 ) + ( ω τ ) n b b ˉ . (27)
So the norm is multiplied by ( ( ω τ ) n a a ˉ e a ˉ + 1 ) ((\omega\tau)^{n_a}\bar{a}e^{\bar{a}}+1) (( ω τ ) n a a ˉ e a ˉ + 1 ) and added to ( ω τ ) n b b ˉ (\omega\tau)^{n_b}\bar{b} ( ω τ ) n b b ˉ . Thus, we have
∥ x ( k ) ∥ ≤ ( ( ω τ ) n a a ˉ e a ˉ + 1 ) k [ ∥ x ( 0 ) ∥ + k ( ω τ ) n b b ˉ ] (28) \lVert x^{(k)}\rVert\le\left((\omega\tau)^{n_a}\bar{a}e^{\bar{a}}+1\right)^k\left[\lVert x^{(0)}\rVert+k(\omega\tau)^{n_b}\bar{b}\right] \tag*{(28)} ∥ x ( k ) ∥ ≤ ( ( ω τ ) n a a ˉ e a ˉ + 1 ) k [ ∥ x ( 0 ) ∥ + k ( ω τ ) n b b ˉ ] (28)
≤ ( ( ω τ ) n a a ˉ e a ˉ + 1 ) K [ ∥ x ( 0 ) ∥ + K ( ω τ ) n b b ˉ ] . (29) \le\left((\omega\tau)^{n_a}\bar{a}e^{\bar{a}}+1\right)^K\left[\lVert x^{(0)}\rVert+K(\omega\tau)^{n_b}\bar{b}\right]. \tag*{(29)} ≤ ( ( ω τ ) n a a ˉ e a ˉ + 1 ) K [ ∥ x ( 0 ) ∥ + K ( ω τ ) n b b ˉ ] . (29)
Therefore
∥ Ψ ( u ) − u ∥ ≤ ∑ k = 1 K ∥ Ψ k ( x ( k − 1 ) ) − x ( k − 1 ) ∥ ≤ ∑ k = 1 K ( ∥ x ( k − 1 ) ∥ ( ω τ ) n a a ˉ e a ˉ + ( ω τ ) n b b ˉ ) (30) \lVert\Psi(u)-u\rVert\le\sum_{k=1}^{K}\lVert\Psi_k(x^{(k-1)})-x^{(k-1)}\rVert\le\sum_{k=1}^{K}\left(\lVert x^{(k-1)}\rVert(\omega\tau)^{n_a}\bar{a}e^{\bar{a}}+(\omega\tau)^{n_b}\bar{b}\right) \tag*{(30)} ∥ Ψ ( u ) − u ∥ ≤ k = 1 ∑ K ∥ Ψ k ( x ( k − 1 ) ) − x ( k − 1 ) ∥ ≤ k = 1 ∑ K ( ∥ x ( k − 1 ) ∥ ( ω τ ) n a a ˉ e a ˉ + ( ω τ ) n b b ˉ ) (30)
≤ K ( ( ( ω τ ) n a a ˉ e a ˉ + 1 ) K [ ∥ u ∥ + K ( ω τ ) n b b ˉ ] ( ω τ ) n a a ˉ e a ˉ + K ( ω τ ) n b b ˉ ) (31) \le K\left(((\omega\tau)^{n_a}\bar{a}e^{\bar{a}}+1)^K\left[\lVert u\rVert+K(\omega\tau)^{n_b}\bar{b}\right](\omega\tau)^{n_a}\bar{a}e^{\bar{a}}+K(\omega\tau)^{n_b}\bar{b}\right) \tag*{(31)} ≤ K ( (( ω τ ) n a a ˉ e a ˉ + 1 ) K [ ∥ u ∥ + K ( ω τ ) n b b ˉ ] ( ω τ ) n a a ˉ e a ˉ + K ( ω τ ) n b b ˉ ) (31)
≤ K a ˉ e a ˉ ω n a ( ω n a a ˉ e a ˉ + 1 ) K [ τ n a ∥ u ∥ + K ( ω τ ) n b b ˉ ] + K 2 ( ω τ ) n b b ˉ (32) \le K\bar{a}e^{\bar{a}}\omega^{n_a}\left(\omega^{n_a}\bar{a}e^{\bar{a}}+1\right)^K\left[\tau^{n_a}\lVert u\rVert+K(\omega\tau)^{n_b}\bar{b}\right]+K^2(\omega\tau)^{n_b}\bar{b} \tag*{(32)} ≤ K a ˉ e a ˉ ω n a ( ω n a a ˉ e a ˉ + 1 ) K [ τ n a ∥ u ∥ + K ( ω τ ) n b b ˉ ] + K 2 ( ω τ ) n b b ˉ (32)
≤ C [ τ n a ∥ u ∥ + τ n b ] , (33) \le C[\tau^{n_a}\lVert u\rVert+\tau^{n_b}], \tag*{(33)} ≤ C [ τ n a ∥ u ∥ + τ n b ] , (33)
for some constant C C C .
Proof of Proposition 2. From our parameterization Eq. (15) we have
y − x = τ v ( x ) + u + Ψ ( u ) − u . (34) y-x=\tau v(x)+u+\Psi(u)-u. \tag*{(34)} y − x = τ v ( x ) + u + Ψ ( u ) − u . (34)
From Lemma 1, we have
∥ Ψ ( u ) − u ∥ ≤ C ( τ n a ∥ u ∥ + τ n b ) , (35) \lVert\Psi(u)-u\rVert\le C(\tau^{n_a}\lVert u\rVert+\tau^{n_b}), \tag*{(35)} ∥ Ψ ( u ) − u ∥ ≤ C ( τ n a ∥ u ∥ + τ n b ) , (35)
and since u u u is Gaussian, we can write E ∥ u ∥ m = c m τ m / 2 \mathbb{E}\lVert u\rVert^m=c_m\tau^{m/2} E ∥ u ∥ m = c m τ m /2 with c m c_m c m being a constant depending only on g g g , m m m and number of dimensions, so we have
E ∥ Ψ ( u ) − u ∥ m ≤ C ( τ m n a E ∥ u ∥ m + τ m n b ) = C ( τ m ( n a + 1 / 2 ) + τ m n b ) = O ( τ m β ) , (36) \mathbb{E}\lVert\Psi(u)-u\rVert^m\le C\left(\tau^{mn_a}\mathbb{E}\lVert u\rVert^m+\tau^{mn_b}\right)=C\left(\tau^{m(n_a+1/2)}+\tau^{mn_b}\right)=O(\tau^{m\beta}), \tag*{(36)} E ∥ Ψ ( u ) − u ∥ m ≤ C ( τ m n a E ∥ u ∥ m + τ m n b ) = C ( τ m ( n a + 1/2 ) + τ m n b ) = O ( τ m β ) , (36)
where we denote β : = min { n a + 1 2 , n b } \beta:=\min\{n_a+\frac{1}{2},n_b\} β := min { n a + 2 1 , n b } , and note that by the condition n a > 1 2 n_a>\frac{1}{2} n a > 2 1 and n b > 1 n_b>1 n b > 1 , we have β > 1 \beta>1 β > 1 .
The first moment is then
E [ y − x ] = τ v ( x ) + E u + E [ Ψ ( u ) − u ] = τ v ( x ) + O ( τ β ) . (37) \mathbb{E}[y-x]=\tau v(x)+\mathbb{E}u+\mathbb{E}[\Psi(u)-u]=\tau v(x)+O(\tau^\beta). \tag*{(37)} E [ y − x ] = τ v ( x ) + E u + E [ Ψ ( u ) − u ] = τ v ( x ) + O ( τ β ) . (37)
The second moment we write down using r : = Ψ ( u ) − u r := \Psi(u) - u r := Ψ ( u ) − u . That is
E [ ( y − x ) ( y − x ) T ] = τ 2 v v T + τ g g T + E [ r r T ] + τ ( v E [ u + r ] T + E [ u + r ] v T ) (38) \mathbb{E}[(y-x)(y-x)^{\mathsf{T}}] = \tau^2 vv^{\mathsf{T}} + \tau gg^{\mathsf{T}} + \mathbb{E}[rr^{\mathsf{T}}] + \tau\left(v\mathbb{E}[u+r]^{\mathsf{T}} + \mathbb{E}[u+r]v^{\mathsf{T}}\right) \tag*{(38)} E [( y − x ) ( y − x ) T ] = τ 2 v v T + τ g g T + E [ r r T ] + τ ( v E [ u + r ] T + E [ u + r ] v T ) (38)
+ E [ u r T ] + E [ r u T ] . (39) + \mathbb{E}[ur^{\mathsf{T}}] + \mathbb{E}[ru^{\mathsf{T}}]. \tag*{(39)} + E [ u r T ] + E [ r u T ] . (39)
Furthermore, using E u = 0 \mathbb{E}u = 0 E u = 0 and Cauchy-Schwarz, we have
∥ E [ u r T ] ∥ ≤ E ∥ u ∥ 2 E ∥ r ∥ 2 = O ( τ 1 / 2 + β ) (40) \lVert\mathbb{E}[ur^{\mathsf{T}}]\rVert\le\sqrt{\mathbb{E}\lVert u\rVert^2\mathbb{E}\lVert r\rVert^2} = O(\tau^{1/2+\beta}) \tag*{(40)} ∥ E [ u r T ]∥ ≤ E ∥ u ∥ 2 E ∥ r ∥ 2 = O ( τ 1/2 + β ) (40)
∥ E [ r r T ] ∥ ≤ E ∥ r ∥ 2 = O ( τ 2 β ) (41) \lVert\mathbb{E}[rr^{\mathsf{T}}]\rVert\le\mathbb{E}\lVert r\rVert^2 = O(\tau^{2\beta}) \tag*{(41)} ∥ E [ r r T ]∥ ≤ E ∥ r ∥ 2 = O ( τ 2 β ) (41)
∥ v E [ u + r ] T ∥ = ∥ v E r T ∥ ≤ E ∥ v ∥ 2 E ∥ r ∥ 2 = O ( τ β ) . (42) \left\lVert v\mathbb{E}[u+r]^{\mathsf{T}}\right\rVert= \lVert v\mathbb{E}r^{\mathsf{T}}\rVert\le\sqrt{\mathbb{E}\lVert v\rVert^2\mathbb{E}\lVert r\rVert^2} = O(\tau^\beta). \tag*{(42)} v E [ u + r ] T = ∥ v E r T ∥ ≤ E ∥ v ∥ 2 E ∥ r ∥ 2 = O ( τ β ) . (42)
Since β > 1 \beta> 1 β > 1 , we, finally, have
E [ ( y − x ) ( y − x ) T ] = τ g g T + o ( τ ) . (43) \mathbb{E}[(y-x)(y-x)^{\mathsf{T}}] = \tau gg^{\mathsf{T}} + o(\tau). \tag*{(43)} E [( y − x ) ( y − x ) T ] = τ g g T + o ( τ ) . (43)
To prove the no jump condition in Eq. (7) , we can use Markov’s inequality i.e., for every ϵ > 0 \epsilon> 0 ϵ > 0 ,
P ( ∥ y − x ∥ > ϵ ) ≤ E ∥ y − x ∥ 4 ϵ 4 ≤ 1 ϵ 4 27 ( τ 4 ∥ v ( x ) ∥ 4 + E ∥ u ∥ 4 + E ∥ r ∥ 4 ) = O ( τ 2 ) , (44) \mathbb{P}(\lVert y-x\rVert> \epsilon) \le\frac{\mathbb{E}\lVert y-x\rVert^4}{\epsilon^4} \le\frac{1}{\epsilon^4}27(\tau^4\lVert v(x)\rVert^4 + \mathbb{E}\lVert u\rVert^4 + \mathbb{E}\lVert r\rVert^4) = O(\tau^2), \tag*{(44)} P (∥ y − x ∥ > ϵ ) ≤ ϵ 4 E ∥ y − x ∥ 4 ≤ ϵ 4 1 27 ( τ 4 ∥ v ( x ) ∥ 4 + E ∥ u ∥ 4 + E ∥ r ∥ 4 ) = O ( τ 2 ) , (44)
where we use ( a + b + c ) 4 ≤ 27 ( a 4 + b 4 + c 4 ) (a+b+c)^4 \le27(a^4+b^4+c^4) ( a + b + c ) 4 ≤ 27 ( a 4 + b 4 + c 4 ) . Thus, we show P ( ∥ y − x ∥ > ϵ ) / τ → 0 \mathbb{P}(\lVert y-x\rVert> \epsilon)/\tau\to0 P (∥ y − x ∥ > ϵ ) / τ → 0 . Finally, we show that the truncation does not matter for the moments k ∈ { 1 , 2 } k \in\{1,2\} k ∈ { 1 , 2 } , by Cauchy–Schwarz,
E [ ∥ y − x ∥ k 1 { ∥ y − x ∥ > ϵ } ] ≤ E ∥ y − x ∥ 2 k P ( ∥ y − x ∥ > ϵ ) = O ( τ k / 2 ) O ( τ ) = o ( τ ) . \mathbb{E}\left[\lVert y-x\rVert^k\mathbf{1}\{\lVert y-x\rVert> \epsilon\}\right] \le\sqrt{\mathbb{E}\lVert y-x\rVert^{2k}}\sqrt{\mathbb{P}(\lVert y-x\rVert> \epsilon)} = O(\tau^{k/2})O(\tau) = o(\tau). E [ ∥ y − x ∥ k 1 {∥ y − x ∥ > ϵ } ] ≤ E ∥ y − x ∥ 2 k P (∥ y − x ∥ > ϵ ) = O ( τ k /2 ) O ( τ ) = o ( τ ) .
Derivations
Kolmogorov–Chapman KL loss derivation
Fix a source state x x x and times s < r < t s<r<t s < r < t , and hold η \eta η fixed when optimizing θ \theta θ . The two-step target density is
q s , r , t ( y ∣ x ; η ) = ∫ d x ′ q r , t ( y ∣ x ′ ; η ) q s , r ( x ′ ∣ x ; η ) . (45) q_{s,r,t}(y\mid x;\eta) = \int dx' q_{r,t}(y\mid x';\eta)q_{s,r}(x'\mid x;\eta). \tag*{(45)} q s , r , t ( y ∣ x ; η ) = ∫ d x ′ q r , t ( y ∣ x ′ ; η ) q s , r ( x ′ ∣ x ; η ) . (45)
We expand the KL divergence as
D K L ( q s , r , t ( ⋅ ∣ x ; η ) ∥ q s , t ( ⋅ ∣ x ; θ ) ) = ∫ d y q s , r , t ( y ∣ x ; η ) log q s , r , t ( y ∣ x ; η ) q s , t ( y ∣ x ; θ ) = − ∫ d y q s , r , t ( y ∣ x ; η ) log q s , t ( y ∣ x ; θ ) + C = − ∫ d x ′ d y q s , r ( x ′ ∣ x ; η ) q r , t ( y ∣ x ′ ; η ) log q s , t ( y ∣ x ; θ ) + C = E x ′ ∼ q s , r ( ⋅ ∣ x ; η ) y ∼ q r , t ( ⋅ ∣ x ′ ; η ) [ − log q s , t ( y ∣ x ; θ ) ] + C , (46) \begin{aligned}
D_{\mathrm{KL}}(q_{s,r,t}(\cdot\mid x;\eta)\mathbin{\|}q_{s,t}(\cdot\mid x;\theta))
&= \int dy\,q_{s,r,t}(y\mid x;\eta)\log\frac{q_{s,r,t}(y\mid x;\eta)}{q_{s,t}(y\mid x;\theta)}\\
&= -\int dy\,q_{s,r,t}(y\mid x;\eta)\log q_{s,t}(y\mid x;\theta)+C\\
&= -\int dx'\,dy\,q_{s,r}(x'\mid x;\eta)q_{r,t}(y\mid x';\eta)\log q_{s,t}(y\mid x;\theta)+C\\
&= \mathbb{E}_{x'\sim q_{s,r}(\cdot\mid x;\eta)}\atop y\sim q_{r,t}(\cdot\mid x';\eta)[-\log q_{s,t}(y\mid x;\theta)]+C, \tag*{(46)}
\end{aligned} D KL ( q s , r , t ( ⋅ ∣ x ; η ) ∥ q s , t ( ⋅ ∣ x ; θ )) = ∫ d y q s , r , t ( y ∣ x ; η ) log q s , t ( y ∣ x ; θ ) q s , r , t ( y ∣ x ; η ) = − ∫ d y q s , r , t ( y ∣ x ; η ) log q s , t ( y ∣ x ; θ ) + C = − ∫ d x ′ d y q s , r ( x ′ ∣ x ; η ) q r , t ( y ∣ x ′ ; η ) log q s , t ( y ∣ x ; θ ) + C y ∼ q r , t ( ⋅ ∣ x ′ ; η ) [ − log q s , t ( y ∣ x ; θ )] + C , = E x ′ ∼ q s , r ( ⋅ ∣ x ; η ) (46)
where
C = ∫ d y q s , r , t ( y ∣ x ; η ) log q s , r , t ( y ∣ x ; η ) (47) C = \int dy\,q_{s,r,t}(y\mid x;\eta)\log q_{s,r,t}(y\mid x;\eta) \tag*{(47)} C = ∫ d y q s , r , t ( y ∣ x ; η ) log q s , r , t ( y ∣ x ; η ) (47)
is the negative entropy of the fixed target and is independent of θ \theta θ . The two objectives therefore have the same minimizers,
arg min θ D K L ( q s , r , t ( ⋅ ∣ x ; η ) ∥ q s , t ( ⋅ ∣ x ; θ ) ) = arg min θ E x ′ ∼ q s , r ( ⋅ ∣ x ; η ) y ∼ q r , t ( ⋅ ∣ x ′ ; η ) [ − log q s , t ( y ∣ x ; θ ) ] . (48) \begin{aligned}
\mathop{\arg\min}_{\theta}D_{\mathrm{KL}}(q_{s,r,t}(\cdot\mid x;\eta)\mathbin{\|}q_{s,t}(\cdot\mid x;\theta))
&= \mathop{\arg\min}_{\theta}\mathbb{E}_{x'\sim q_{s,r}(\cdot\mid x;\eta)}\atop y\sim q_{r,t}(\cdot\mid x';\eta)[-\log q_{s,t}(y\mid x;\theta)]. \tag*{(48)}
\end{aligned} arg min θ D KL ( q s , r , t ( ⋅ ∣ x ; η ) ∥ q s , t ( ⋅ ∣ x ; θ )) y ∼ q r , t ( ⋅ ∣ x ′ ; η ) [ − log q s , t ( y ∣ x ; θ )] . = arg min θ E x ′ ∼ q s , r ( ⋅ ∣ x ; η ) (48)
Thus, we sample two-step transitions and evaluate their likelihood under the direct transition model without evaluating the composed density. Taking the expectation over the sampled source states and times gives Eq. (12) . With η = θ ⋆ \eta=\theta^\star η = θ ⋆ , θ ⋆ \theta^\star θ ⋆ minimizes K L ( q s , r , t ( ⋅ ∣ x ; θ ⋆ ) ∥ q s , t ( ⋅ ∣ x ; θ ) ) KL(q_{s,r,t}(\cdot\mid x;\theta^\star)\mathbin{\|}q_{s,t}(\cdot\mid x;\theta)) K L ( q s , r , t ( ⋅ ∣ x ; θ ⋆ ) ∥ q s , t ( ⋅ ∣ x ; θ )) over θ \theta θ . If the minimum is zero for every x x x and s < r < t s<r<t s < r < t , then q s , t ( ⋅ ∣ x ; θ ⋆ ) q_{s,t}(\cdot\mid x;\theta^\star) q s , t ( ⋅ ∣ x ; θ ⋆ ) satisfies the Kolmogorov–Chapman equation Eq. (10) .
Learning the boundary conditions
In § 3.2 , we enforce the boundary conditions through the parameterization. Alternatively, we can retain the locality and diffusion conditions in the parameterization and learn the infinitesimal drift through a training objective. We consider two approaches: direct regression to the known vector field and likelihood training on short-time samples generated from the SDE’s drift and diffusion. We also describe a conditional regression formulation for generative modeling, where the marginal vector field is unavailable but conditional training targets can be constructed.
Drift regression. Instead of adding the known displacement τ v ( x ) \tau v(x) τ v ( x ) as in Eq. (15) , we apply the flow as follows,
y θ = Ψ ( x + τ g z ; x , θ ) , z ∼ N ( 0 , I ) . (49) y_\theta= \Psi\left(x+\sqrt{\tau}gz;x,\theta\right), \qquad z \sim\mathcal{N}(0,I). \tag*{(49)} y θ = Ψ ( x + τ g z ; x , θ ) , z ∼ N ( 0 , I ) . (49)
We use the coupling layers in Eq. (17) with n a = n b = 1 n_a=n_b=1 n a = n b = 1 , allowing the flow to introduce a displacement proportional to τ \tau τ while preserving the locality and diffusion conditions.
Assume log-scale and shift are uniformly bounded over their inputs for sufficiently small τ \tau τ , and jointly continuous in their inputs and τ \tau τ at τ = 0 \tau=0 τ = 0 . The model’s infinitesimal drift is then
v θ ( x ) : = ∂ ∂ τ E z [ y θ − x ] ∣ τ = 0 = ∂ Ψ ( x ; x , θ ) ∂ τ ∣ τ = 0 , (50) v_\theta(x) := \left.\frac{\partial}{\partial\tau}\mathbb{E}_z[y_\theta-x]\right|_{\tau=0} = \left.\frac{\partial\Psi(x;x,\theta)}{\partial\tau}\right|_{\tau=0}, \tag*{(50)} v θ ( x ) := ∂ τ ∂ E z [ y θ − x ] τ = 0 = ∂ τ ∂ Ψ ( x ; x , θ ) τ = 0 , (50)
where the derivative of Ψ \Psi Ψ holds its input and source state fixed.
Since every coupling layer is the identity at τ = 0 \tau=0 τ = 0 , differentiating their composition gives
v θ ( x ) = ∑ k = 1 K ∂ Ψ k ( x ; x , θ ) ∂ τ ∣ τ = 0 . (51) v_\theta(x)=\sum_{k=1}^{K}\left.\frac{\partial\Psi_k(x;x,\theta)}{\partial\tau}\right|_{\tau=0}. \tag*{(51)} v θ ( x ) = k = 1 ∑ K ∂ τ ∂ Ψ k ( x ; x , θ ) τ = 0 . (51)
For n a = n b = 1 n_a=n_b=1 n a = n b = 1 , both gates have derivative ω \omega ω at τ = 0 \tau=0 τ = 0 . A layer that transforms the second block therefore contributes
∂ Ψ k ( x ; x , θ ) ∂ τ ∣ τ = 0 = ω ( 0 x 2 ⊙ a k ( x 1 ; x , θ ) + b k ( x 1 ; x , θ ) ) , (52) \left.\frac{\partial\Psi_k(x;x,\theta)}{\partial\tau}\right|_{\tau=0}
=\omega\begin{pmatrix}0\\x_2\odot a_k(x_1;x,\theta)+b_k(x_1;x,\theta)\end{pmatrix}, \tag*{(52)} ∂ τ ∂ Ψ k ( x ; x , θ ) τ = 0 = ω ( 0 x 2 ⊙ a k ( x 1 ; x , θ ) + b k ( x 1 ; x , θ ) ) , (52)
where x = [ x 1 , x 2 ] x=[x_1,x_2] x = [ x 1 , x 2 ] and the conditioners are evaluated at τ = 0 \tau=0 τ = 0 . For layers transforming the first block, the block roles are exchanged. Thus, we can evaluate the infinitesimal drift directly from the scale and shift conditioners. Expanding the gated transformations gives E [ y θ − x ] = τ v θ ( x ) + o ( τ ) \mathbb{E}[y_\theta-x]=\tau v_\theta(x)+o(\tau) E [ y θ − x ] = τ v θ ( x ) + o ( τ ) . By Lemma 1 with n a = n b = 1 n_a=n_b=1 n a = n b = 1 , the flow correction is at most C τ ( ∥ x + τ g z ∥ + 1 ) C\tau(\|x+\sqrt{\tau}gz\|+1) C τ ( ∥ x + τ g z ∥ + 1 ) , so β = 1 \beta=1 β = 1 in Section A.3. The diffusion and locality estimates there need only β > 1 2 \beta>\frac{1}{2} β > 2 1 .
We learn this drift by minimizing
L d r i f t ( θ ) = E x ∼ π [ ∥ v θ ( x ) − v ( x ) ∥ 2 ] , (53) \mathcal{L}_{\mathrm{drift}}(\theta)=\mathbb{E}_{x\sim\pi}\left[\|v_\theta(x)-v(x)\|^2\right], \tag*{(53)} L drift ( θ ) = E x ∼ π [ ∥ v θ ( x ) − v ( x ) ∥ 2 ] , (53)
where π \pi π is the distribution of source states used for training.
For generative modeling, we can sample states and vector-field targets from prescribed conditional probability paths, following conditional flow matching (Lipman et al., [36 ] ). When the conditional targets average to the marginal vector field given the state and time, this regression objective differs from regression against the marginal field only by a constant independent of θ \theta θ .
Short-time likelihood. Alternatively, we can supervise the model using short-time Euler–Maruyama transitions. For a fixed small elapsed time τ > 0 \tau>0 τ > 0 , we sample
y = x + τ v ( x ) + τ g z , z ∼ N ( 0 , I ) . (54) y=x+\tau v(x)+\sqrt{\tau}gz,\qquad z\sim\mathcal{N}(0,I). \tag*{(54)} y = x + τ v ( x ) + τ g z , z ∼ N ( 0 , I ) . (54)
We maximize the likelihood of these samples under the model’s transition density at elapsed time τ \tau τ ,
L E M ( θ ) = E x ∼ π , z ∼ N ( 0 , I ) [ − log q 0 , τ ( y ∣ x ; θ ) ] . (55) \mathcal{L}_{\mathrm{EM}}(\theta)=\mathbb{E}_{x\sim\pi,z\sim\mathcal{N}(0,I)}[-\log q_{0,\tau}(y\mid x;\theta)]. \tag*{(55)} L EM ( θ ) = E x ∼ π , z ∼ N ( 0 , I ) [ − log q 0 , τ ( y ∣ x ; θ )] . (55)
The target has mean displacement τ v ( x ) \tau v(x) τ v ( x ) and covariance τ g g T \tau gg^{\mathsf{T}} τ g g T , giving the required infinitesimal moments as τ → 0 \tau\to0 τ → 0 . At a fixed τ \tau τ , this objective fits the Euler–Maruyama approximation and does not by itself guarantee the boundary conditions, which concern the limit τ → 0 \tau\to0 τ → 0 .
Gaussian reference kernel
We derive the Gaussian reference kernels for the first- and second-order autonomous parameterizations in Eqs. (15) and (19) . The source state x x x denotes the position in the first-order case and the joint position–momentum state in the second-order case. By time homogeneity, we take the source time to be zero and write p τ ( y ∣ x ) p_\tau(y \mid x) p τ ( y ∣ x ) for the exact transition density over elapsed time τ \tau τ .
Throughout these derivations, Ψ \Psi Ψ is set to the identity, so the model reduces to the Gaussian reference density q τ r e f ( y ∣ x ) q_\tau^{\mathrm{ref}}(y \mid x) q τ ref ( y ∣ x ) with mean μ = x + m \mu= x+m μ = x + m and covariance Σ = L L ⊤ \Sigma= LL^{\top} Σ = L L ⊤ . All expectations and covariances are conditional on the source state, held fixed in time derivatives. We assume a smooth vector field with bounded derivatives and at most linear growth, and constant isotropic noise with scalar coefficient g g g , to justify the expansions and integrations by parts below.
First-order Gaussian reference
Let p τ ( y ∣ x ) p_\tau(y \mid x) p τ ( y ∣ x ) denote the conditional density of X τ X_\tau X τ given X 0 = x X_0 = x X 0 = x , evaluated at y y y . Conditioning on X 0 = x X_0 = x X 0 = x , we have the SDE
d X τ = v ( X τ ) d τ + g d W τ , (56) dX_\tau= v(X_\tau)\,d\tau+ g\,dW_\tau, \tag*{(56)} d X τ = v ( X τ ) d τ + g d W τ , (56)
and its corresponding Fokker–Planck equation
∂ p τ ( y ∣ x ) ∂ τ = − ⟨ ∇ y , p τ ( y ∣ x ) v ( y ) ⟩ + g 2 2 Δ y p τ ( y ∣ x ) , (57) \frac{\partial p_\tau(y \mid x)}{\partial\tau}
= -\langle\nabla_y, p_\tau(y \mid x)v(y)\rangle+ \frac{g^2}{2}\Delta_y p_\tau(y \mid x), \tag*{(57)} ∂ τ ∂ p τ ( y ∣ x ) = − ⟨ ∇ y , p τ ( y ∣ x ) v ( y )⟩ + 2 g 2 Δ y p τ ( y ∣ x ) , (57)
with p 0 ( y ∣ x ) = δ ( y − x ) p_0(y \mid x) = \delta(y-x) p 0 ( y ∣ x ) = δ ( y − x ) .
To obtain a Gaussian reference with a closed-form sampler, we approximate the vector field about the source position. Its Taylor expansion is
v ( y ) = v ( x ) + ∂ v ( x ) ∂ x ( y − x ) + O ( ∥ y − x ∥ 2 ) . (58) v(y) = v(x) + \frac{\partial v(x)}{\partial x}(y-x) + O(\lVert y-x\rVert^2). \tag*{(58)} v ( y ) = v ( x ) + ∂ x ∂ v ( x ) ( y − x ) + O (∥ y − x ∥ 2 ) . (58)
Retaining only the zeroth-order term gives the reference SDE
d y τ = v ( x ) d τ + g d W τ , y 0 = x . (59) dy_\tau= v(x)\,d\tau+ g\,dW_\tau,\qquad y_0=x. \tag*{(59)} d y τ = v ( x ) d τ + g d W τ , y 0 = x . (59)
Its transition density satisfies
∂ q τ r e f ( y ∣ x ) ∂ τ = − ⟨ ∇ y , q τ r e f ( y ∣ x ) v ( x ) ⟩ + g 2 2 Δ y q τ r e f ( y ∣ x ) , (60) \frac{\partial q_\tau^{\mathrm{ref}}(y \mid x)}{\partial\tau}
= -\langle\nabla_y, q_\tau^{\mathrm{ref}}(y \mid x)v(x)\rangle+ \frac{g^2}{2}\Delta_y q_\tau^{\mathrm{ref}}(y \mid x), \tag*{(60)} ∂ τ ∂ q τ ref ( y ∣ x ) = − ⟨ ∇ y , q τ ref ( y ∣ x ) v ( x )⟩ + 2 g 2 Δ y q τ ref ( y ∣ x ) , (60)
with q 0 r e f ( y ∣ x ) = δ ( y − x ) q_0^{\mathrm{ref}}(y \mid x) = \delta(y-x) q 0 ref ( y ∣ x ) = δ ( y − x ) . The reference SDE has constant drift and diffusion, so its transition kernel is Gaussian. We derive its mean and covariance below.
Mean and covariance. Starting with the first moment, we have
μ ( x , τ ) = ∫ d y y q τ r e f ( y ∣ x ) . (61) \mu(x,\tau) = \int dy\, y q_\tau^{\mathrm{ref}}(y \mid x). \tag*{(61)} μ ( x , τ ) = ∫ d y y q τ ref ( y ∣ x ) . (61)
Differentiating with respect to τ \tau τ and integrating by parts gives
∂ μ ∂ τ = ∫ d y y ∂ q τ r e f ( y ∣ x ) ∂ τ = − ∫ d y y ⟨ ∇ y , q τ r e f ( y ∣ x ) v ( x ) ⟩ + g 2 2 ∫ d y y Δ y q τ r e f ( y ∣ x ) = ∫ d y q τ r e f ( y ∣ x ) v ( x ) = v ( x ) . (62) \frac{\partial\mu}{\partial\tau}
= \int dy\, y \frac{\partial q_\tau^{\mathrm{ref}}(y \mid x)}{\partial\tau}
= -\int dy\, y\langle\nabla_y, q_\tau^{\mathrm{ref}}(y \mid x)v(x)\rangle
+ \frac{g^2}{2}\int dy\, y\Delta_y q_\tau^{\mathrm{ref}}(y \mid x)
= \int dy\, q_\tau^{\mathrm{ref}}(y \mid x)v(x) = v(x). \tag*{(62)} ∂ τ ∂ μ = ∫ d y y ∂ τ ∂ q τ ref ( y ∣ x ) = − ∫ d y y ⟨ ∇ y , q τ ref ( y ∣ x ) v ( x )⟩ + 2 g 2 ∫ d y y Δ y q τ ref ( y ∣ x ) = ∫ d y q τ ref ( y ∣ x ) v ( x ) = v ( x ) . (62)
Using the initial condition μ ( x , 0 ) = x \mu(x,0)=x μ ( x , 0 ) = x , we obtain
μ ( x , τ ) = x + τ v ( x ) . (63) \mu(x,\tau) = x+\tau v(x). \tag*{(63)} μ ( x , τ ) = x + τ v ( x ) . (63)
Similarly, differentiating the second moment and integrating by parts gives
∂ ∂ τ E [ y y ⊤ ] = ∫ d y y y ⊤ ∂ q τ r e f ( y ∣ x ) ∂ τ = v ( x ) μ ⊤ + μ v ( x ) ⊤ + g 2 I d . (64) \frac{\partial}{\partial\tau}\mathbb{E}[yy^{\top}]
= \int dy\, yy^{\top}\frac{\partial q_\tau^{\mathrm{ref}}(y \mid x)}{\partial\tau}
= v(x)\mu^{\top}+\mu v(x)^{\top}+g^2 I_d. \tag*{(64)} ∂ τ ∂ E [ y y ⊤ ] = ∫ d y y y ⊤ ∂ τ ∂ q τ ref ( y ∣ x ) = v ( x ) μ ⊤ + μv ( x ) ⊤ + g 2 I d . (64)
Since Σ = E [ y y ⊤ ] − μ μ ⊤ \Sigma= \mathbb{E}[yy^{\top}]-\mu\mu^{\top} Σ = E [ y y ⊤ ] − μ μ ⊤ , its derivative is
∂ Σ ∂ τ = ∂ ∂ τ E [ y y ⊤ ] − ∂ μ ∂ τ μ ⊤ − μ ( ∂ μ ∂ τ ) ⊤ = g 2 I d . (65) \frac{\partial\Sigma}{\partial\tau}
= \frac{\partial}{\partial\tau}\mathbb{E}[yy^{\top}]
- \frac{\partial\mu}{\partial\tau}\mu^{\top}
- \mu\left(\frac{\partial\mu}{\partial\tau}\right)^{\top}
= g^2 I_d. \tag*{(65)} ∂ τ ∂ Σ = ∂ τ ∂ E [ y y ⊤ ] − ∂ τ ∂ μ μ ⊤ − μ ( ∂ τ ∂ μ ) ⊤ = g 2 I d . (65)
Using the initial condition Σ ( 0 ) = 0 \Sigma(0)=0 Σ ( 0 ) = 0 , we obtain
Σ ( τ ) = τ g 2 I d . (66) \Sigma(\tau)=\tau g^2 I_d. \tag*{(66)} Σ ( τ ) = τ g 2 I d . (66)
The reference density and sampler are
q T r e f ( y ∣ x ) = N ( y ∣ μ ( x , τ ) , Σ ( τ ) ) , (67) q_T^{\mathrm{ref}}(y\mid x)=\mathcal{N}\left(y\mid\mu(x,\tau),\Sigma(\tau)\right), \tag*{(67)} q T ref ( y ∣ x ) = N ( y ∣ μ ( x , τ ) , Σ ( τ ) ) , (67)
y = x + τ v ( x ) + u , u = τ g z , z ∼ N ( 0 , I d ) . (68) y=x+\tau v(x)+u,\qquad u=\sqrt{\tau}\,gz,\qquad z\sim\mathcal{N}(0,I_d). \tag*{(68)} y = x + τ v ( x ) + u , u = τ g z , z ∼ N ( 0 , I d ) . (68)
For nonzero g g g , the reference covariance is positive definite whenever τ > 0 \tau>0 τ > 0 .
Second-order Gaussian reference
We start from the second-order Langevin dynamics
d X τ = P τ d τ , dX_\tau=P_\tau\,d\tau, d X τ = P τ d τ ,
d P τ = ( v ( X τ ) − γ P τ ) d τ + g d W τ , (69) dP_\tau=\left(v(X_\tau)-\gamma P_\tau\right)d\tau+g\,dW_\tau, \tag*{(69)} d P τ = ( v ( X τ ) − γ P τ ) d τ + g d W τ , (69)
where γ > 0 \gamma>0 γ > 0 is the friction coefficient. For simplicity, we work in mass-weighted coordinates M 1 / 2 X M^{1/2}X M 1/2 X and M − 1 / 2 P M^{-1/2}P M − 1/2 P , where the dynamics have unit mass, vector field M − 1 / 2 v M^{-1/2}v M − 1/2 v and isotropic noise 2 γ k B T \sqrt{2\gamma k_{\mathrm{B}}T} 2 γ k B T ; the general case follows by rescaling the position and momentum rows of μ \mu μ and L L L by M − 1 / 2 M^{-1/2} M − 1/2 and M 1 / 2 M^{1/2} M 1/2 . The dynamics are conditioned on the source state x = ( X , P ) ∈ R 2 d x=(X,P)\in\mathbb{R}^{2d} x = ( X , P ) ∈ R 2 d , where X , P ∈ R d X,P\in\mathbb{R}^{d} X , P ∈ R d , with X 0 = X X_0=X X 0 = X and P 0 = P P_0=P P 0 = P .
As in the first-order case, we hold the vector field fixed at the source position, v ( X ) v(X) v ( X ) , while keeping the momentum damping term unchanged. Let y ( τ ) = ( y x ( τ ) , y p ( τ ) ) y(\tau)=(y_x(\tau),y_p(\tau)) y ( τ ) = ( y x ( τ ) , y p ( τ )) denote the reference process, which satisfies
d y x ( τ ) = y p ( τ ) d τ , dy_x(\tau)=y_p(\tau)\,d\tau, d y x ( τ ) = y p ( τ ) d τ ,
d y p ( τ ) = ( v ( X ) − γ y p ( τ ) ) d τ + g d W τ , (70) dy_p(\tau)=\left(v(X)-\gamma y_p(\tau)\right)d\tau+g\,dW_\tau, \tag*{(70)} d y p ( τ ) = ( v ( X ) − γ y p ( τ ) ) d τ + g d W τ , (70)
with y ( 0 ) = x y(0)=x y ( 0 ) = x . Here, W τ W_\tau W τ is standard Brownian motion in R d \mathbb{R}^{d} R d , and g g g is a scalar noise coefficient. Its density q T r e f ( y ∣ x ) q_T^{\mathrm{ref}}(y\mid x) q T ref ( y ∣ x ) , evaluated at y = ( y x , y p ) y=(y_x,y_p) y = ( y x , y p ) , satisfies
∂ q T r e f ( y ∣ x ) ∂ τ = − ⟨ ∇ y x , y p q T r e f ( y ∣ x ) ⟩ − ⟨ ∇ y p , ( v ( X ) − γ y p ) q T r e f ( y ∣ x ) ⟩ + g 2 2 Δ y p q T r e f ( y ∣ x ) , (71) \frac{\partial q_T^{\mathrm{ref}}(y\mid x)}{\partial\tau}
=-\left\langle\nabla_{y_x},y_pq_T^{\mathrm{ref}}(y\mid x)\right\rangle
-\left\langle\nabla_{y_p},\left(v(X)-\gamma y_p\right)q_T^{\mathrm{ref}}(y\mid x)\right\rangle
+\frac{g^2}{2}\Delta_{y_p}q_T^{\mathrm{ref}}(y\mid x), \tag*{(71)} ∂ τ ∂ q T ref ( y ∣ x ) = − ⟨ ∇ y x , y p q T ref ( y ∣ x ) ⟩ − ⟨ ∇ y p , ( v ( X ) − γ y p ) q T ref ( y ∣ x ) ⟩ + 2 g 2 Δ y p q T ref ( y ∣ x ) , (71)
with q 0 r e f ( y ∣ x ) = δ ( y x − X ) δ ( y p − P ) q_0^{\mathrm{ref}}(y\mid x)=\delta(y_x-X)\delta(y_p-P) q 0 ref ( y ∣ x ) = δ ( y x − X ) δ ( y p − P ) . The reference dynamics are affine in the joint state and driven by Gaussian noise, so their transition kernel is Gaussian. Now, we derive its mean and covariance.
Mean. Starting with the first moment, we have
μ ( x , τ ) = ( μ x ( x , τ ) μ p ( x , τ ) ) = ∫ d y y q T r e f ( y ∣ x ) . (72) \mu(x,\tau)=
\begin{pmatrix}
\mu_x(x,\tau)\\
\mu_p(x,\tau)
\end{pmatrix}
=\int dy\,y\,q_T^{\mathrm{ref}}(y\mid x). \tag*{(72)} μ ( x , τ ) = ( μ x ( x , τ ) μ p ( x , τ ) ) = ∫ d y y q T ref ( y ∣ x ) . (72)
Differentiating with respect to τ \tau τ , substituting Eq. (71) , and integrating by parts gives
∂ μ ∂ τ = ∫ d y y ∂ q T r e f ( y ∣ x ) ∂ τ = − ∫ d y y ⟨ ∇ y x , y p q T r e f ( y ∣ x ) ⟩ − ∫ d y y ⟨ ∇ y p , ( v ( X ) − γ y p ) q T r e f ( y ∣ x ) ⟩ + g 2 2 ∫ d y y Δ y p q T r e f ( y ∣ x ) = ∫ d y ( y p v ( X ) − γ y p ) q T r e f ( y ∣ x ) = ( μ p v ( X ) − γ μ p ) . (73) \frac{\partial\mu}{\partial\tau}
=\int dy\,y\,\frac{\partial q_T^{\mathrm{ref}}(y\mid x)}{\partial\tau}
=-\int dy\,y\left\langle\nabla_{y_x},y_pq_T^{\mathrm{ref}}(y\mid x)\right\rangle
-\int dy\,y\left\langle\nabla_{y_p},\left(v(X)-\gamma y_p\right)q_T^{\mathrm{ref}}(y\mid x)\right\rangle
+\frac{g^2}{2}\int dy\,y\,\Delta_{y_p}q_T^{\mathrm{ref}}(y\mid x)
=\int dy
\begin{pmatrix}
y_p\\
v(X)-\gamma y_p
\end{pmatrix}
q_T^{\mathrm{ref}}(y\mid x)
=
\begin{pmatrix}
\mu_p\\
v(X)-\gamma\mu_p
\end{pmatrix}. \tag*{(73)} ∂ τ ∂ μ = ∫ d y y ∂ τ ∂ q T ref ( y ∣ x ) = − ∫ d y y ⟨ ∇ y x , y p q T ref ( y ∣ x ) ⟩ − ∫ d y y ⟨ ∇ y p , ( v ( X ) − γ y p ) q T ref ( y ∣ x ) ⟩ + 2 g 2 ∫ d y y Δ y p q T ref ( y ∣ x ) = ∫ d y ( y p v ( X ) − γ y p ) q T ref ( y ∣ x ) = ( μ p v ( X ) − γ μ p ) . (73)
Thus, the position and momentum means satisfy
d μ x d τ = μ p , \frac{d\mu_x}{d\tau}=\mu_p, d τ d μ x = μ p ,
d μ p d τ + γ μ p = v ( X ) , (74) \frac{d\mu_p}{d\tau}+\gamma\mu_p=v(X), \tag*{(74)} d τ d μ p + γ μ p = v ( X ) , (74)
with μ x ( 0 ) = X \mu_x(0)=X μ x ( 0 ) = X and μ p ( 0 ) = P \mu_p(0)=P μ p ( 0 ) = P . Multiplying the momentum equation by e γ τ e^{\gamma\tau} e γ τ and integrating from 0 0 0 to τ \tau τ gives
d d τ ( e γ τ μ p ( τ ) ) = e γ τ v ( X ) , \frac{d}{d\tau}\left(e^{\gamma\tau}\mu_p(\tau)\right)=e^{\gamma\tau}v(X), d τ d ( e γ τ μ p ( τ ) ) = e γ τ v ( X ) ,
e γ τ μ p ( τ ) − P = v ( X ) ∫ 0 τ d r e γ r , e^{\gamma\tau}\mu_p(\tau)-P=v(X)\int_0^\tau\mathrm{d}r\,e^{\gamma r}, e γ τ μ p ( τ ) − P = v ( X ) ∫ 0 τ d r e γ r ,
μ p ( τ ) = e − γ τ P + 1 − e − γ τ γ v ( X ) . (75) \mu_p(\tau)=e^{-\gamma\tau}P+\frac{1-e^{-\gamma\tau}}{\gamma}v(X). \tag*{(75)} μ p ( τ ) = e − γ τ P + γ 1 − e − γ τ v ( X ) . (75)
For position, integrating its mean equation gives
μ x ( τ ) − X = ∫ 0 τ d r μ p ( r ) = P ∫ 0 τ d r e − γ r + v ( X ) γ ∫ 0 τ d r ( 1 − e − γ r ) = 1 − e − γ τ γ P + γ τ − 1 + e − γ τ γ 2 v ( X ) . (76) \begin{aligned}
\mu_x(\tau)-X=\int_0^\tau\mathrm{d}r\,\mu_p(r) \\
&=P\int_0^\tau\mathrm{d}r\,e^{-\gamma r}+\frac{v(X)}{\gamma}\int_0^\tau\mathrm{d}r\,(1-e^{-\gamma r}) \\
&=\frac{1-e^{-\gamma\tau}}{\gamma}P+\frac{\gamma\tau-1+e^{-\gamma\tau}}{\gamma^2}v(X). \tag*{(76)}
\end{aligned} μ x ( τ ) − X = ∫ 0 τ d r μ p ( r ) = P ∫ 0 τ d r e − γ r + γ v ( X ) ∫ 0 τ d r ( 1 − e − γ r ) = γ 1 − e − γ τ P + γ 2 γ τ − 1 + e − γ τ v ( X ) . (76)
Using the functions
ϕ 1 ( τ ) = 1 − e − γ τ γ , ϕ 2 ( τ ) = τ − ϕ 1 ( τ ) γ , (77) \phi_1(\tau)=\frac{1-e^{-\gamma\tau}}{\gamma},\qquad\phi_2(\tau)=\frac{\tau-\phi_1(\tau)}{\gamma}, \tag*{(77)} ϕ 1 ( τ ) = γ 1 − e − γ τ , ϕ 2 ( τ ) = γ τ − ϕ 1 ( τ ) , (77)
we can write the mean as μ = x + m \mu=x+m μ = x + m , with
m = ( ϕ 1 ( τ ) P + ϕ 2 ( τ ) v ( X ) ϕ 1 ( τ ) ( v ( X ) − γ P ) ) . (78) m=\begin{pmatrix}
\phi_1(\tau)P+\phi_2(\tau)v(X)\\
\phi_1(\tau)(v(X)-\gamma P)
\end{pmatrix}. \tag*{(78)} m = ( ϕ 1 ( τ ) P + ϕ 2 ( τ ) v ( X ) ϕ 1 ( τ ) ( v ( X ) − γ P ) ) . (78)
Covariance. To construct the Gaussian sampler, we need the position variance, the momentum variance, and their cross-covariance. We derive these from the reference process after subtracting its mean:
X ~ = y x − μ x , P ~ = y p − μ p . (79) \widetilde{X}=y_x-\mu_x,\qquad\widetilde{P}=y_p-\mu_p. \tag*{(79)} X = y x − μ x , P = y p − μ p . (79)
Both start at zero because the source state is fixed. Subtracting the mean equations from the reference dynamics gives
d X ~ = P ~ d τ , \mathrm{d}\widetilde{X}=\widetilde{P}\,\mathrm{d}\tau, d X = P d τ ,
d P ~ = − γ P ~ d τ + g d W τ , (80) \mathrm{d}\widetilde{P}=-\gamma\widetilde{P}\,\mathrm{d}\tau+g\,\mathrm{d}W_\tau, \tag*{(80)} d P = − γ P d τ + g d W τ , (80)
where we note that the vector field cancels.
For each coordinate j = 1 , … , d j=1,\ldots,d j = 1 , … , d , define
S x x = E [ X ~ j 2 ] , S x p = E [ X ~ j P ~ j ] , S p p = E [ P ~ j 2 ] . (81) S_{xx}=\mathbb{E}[\widetilde{X}_j^2],\qquad S_{xp}=\mathbb{E}[\widetilde{X}_j\widetilde{P}_j],\qquad S_{pp}=\mathbb{E}[\widetilde{P}_j^2]. \tag*{(81)} S xx = E [ X j 2 ] , S x p = E [ X j P j ] , S pp = E [ P j 2 ] . (81)
These quantities do not depend on j j j and are initially zero. We write W τ , j W_{\tau,j} W τ , j for the j j j th component of the Brownian motion.
Starting with S p p S_{pp} S pp , the product rule gives
d ( P ~ j 2 ) = 2 P ~ j d P ~ j + ( d P ~ j ) 2 . (82) \mathrm{d}(\widetilde{P}_j^2)=2\widetilde{P}_j\,\mathrm{d}\widetilde{P}_j+(\mathrm{d}\widetilde{P}_j)^2. \tag*{(82)} d ( P j 2 ) = 2 P j d P j + ( d P j ) 2 . (82)
Using the Itô identities ( d W τ , j ) 2 = d τ (\mathrm{d}W_{\tau,j})^2=\mathrm{d}\tau ( d W τ , j ) 2 = d τ and d τ d W τ , j = ( d τ ) 2 = 0 \mathrm{d}\tau\,\mathrm{d}W_{\tau,j}=(\mathrm{d}\tau)^2=0 d τ d W τ , j = ( d τ ) 2 = 0 , we obtain
d ( P ~ j 2 ) = − 2 γ P ~ j 2 d τ + 2 g P ~ j d W τ , j + g 2 d τ . (83) \mathrm{d}(\widetilde{P}_j^2)=-2\gamma\widetilde{P}_j^2\,\mathrm{d}\tau+2g\widetilde{P}_j\,\mathrm{d}W_{\tau,j}+g^2\,\mathrm{d}\tau. \tag*{(83)} d ( P j 2 ) = − 2 γ P j 2 d τ + 2 g P j d W τ , j + g 2 d τ . (83)
The stochastic integral has zero expectation, so
d S p p d τ + 2 γ S p p = g 2 . (84) \frac{\mathrm{d}S_{pp}}{\mathrm{d}\tau}+2\gamma S_{pp}=g^2. \tag*{(84)} d τ d S pp + 2 γ S pp = g 2 . (84)
Multiplying by e 2 γ τ e^{2\gamma\tau} e 2 γ τ , integrating, and using S p p ( 0 ) = 0 S_{pp}(0)=0 S pp ( 0 ) = 0 gives
d d τ ( e 2 γ τ S p p ( τ ) ) = g 2 e 2 γ τ , \frac{d}{d\tau}\left(e^{2\gamma\tau}S_{pp}(\tau)\right)=g^2e^{2\gamma\tau}, d τ d ( e 2 γ τ S pp ( τ ) ) = g 2 e 2 γ τ ,
S p p ( τ ) = g 2 2 γ ( 1 − e − 2 γ τ ) . (85) S_{pp}(\tau)=\frac{g^2}{2\gamma}(1-e^{-2\gamma\tau}). \tag*{(85)} S pp ( τ ) = 2 γ g 2 ( 1 − e − 2 γ τ ) . (85)
We repeat for S x p S_{xp} S x p ,
d ( X ~ j P ~ j ) = X ~ j d P ~ j + P ~ j d X ~ j + d X ~ j d P ~ j . (86) d(\widetilde{X}_j\widetilde{P}_j)=\widetilde{X}_j\,d\widetilde{P}_j+\widetilde{P}_j\,d\widetilde{X}_j+d\widetilde{X}_j\,d\widetilde{P}_j. \tag*{(86)} d ( X j P j ) = X j d P j + P j d X j + d X j d P j . (86)
The Itô identities above give d X ~ j d P ~ j = 0 d\widetilde{X}_j\,d\widetilde{P}_j=0 d X j d P j = 0 . Substituting the centered dynamics gives
d ( X ~ j P ~ j ) = ( − γ X ~ j P ~ j + P ~ j 2 ) d τ + g X ~ j d W τ , j . (87) d(\widetilde{X}_j\widetilde{P}_j)=(-\gamma\widetilde{X}_j\widetilde{P}_j+\widetilde{P}_j^2)d\tau+g\widetilde{X}_j\,dW_{\tau,j}. \tag*{(87)} d ( X j P j ) = ( − γ X j P j + P j 2 ) d τ + g X j d W τ , j . (87)
Taking expectations,
d S x p d τ + γ S x p = S p p , (88) \frac{dS_{xp}}{d\tau}+\gamma S_{xp}=S_{pp}, \tag*{(88)} d τ d S x p + γ S x p = S pp , (88)
and solving with the expression for S p p S_{pp} S pp and the initial condition S x p ( 0 ) = 0 S_{xp}(0)=0 S x p ( 0 ) = 0 gives
d d τ ( e γ τ S x p ( τ ) ) = g 2 2 γ ( e γ τ − e − γ τ ) , e γ τ S x p ( τ ) = g 2 2 γ 2 ( e γ τ + e − γ τ − 2 ) , S x p ( τ ) = g 2 2 γ 2 ( 1 − e − γ τ ) 2 . (89) \begin{aligned}
\frac{d}{d\tau}\left(e^{\gamma\tau}S_{xp}(\tau)\right)&=\frac{g^2}{2\gamma}\left(e^{\gamma\tau}-e^{-\gamma\tau}\right),\\
e^{\gamma\tau}S_{xp}(\tau)&=\frac{g^2}{2\gamma^2}\left(e^{\gamma\tau}+e^{-\gamma\tau}-2\right),\\
S_{xp}(\tau)&=\frac{g^2}{2\gamma^2}\left(1-e^{-\gamma\tau}\right)^2. \tag*{(89)}
\end{aligned} d τ d ( e γ τ S x p ( τ ) ) e γ τ S x p ( τ ) S x p ( τ ) = 2 γ g 2 ( e γ τ − e − γ τ ) , = 2 γ 2 g 2 ( e γ τ + e − γ τ − 2 ) , = 2 γ 2 g 2 ( 1 − e − γ τ ) 2 . (89)
Finally, for S x x S_{xx} S xx ,
d ( X ~ j 2 ) = 2 X ~ j d X ~ j + ( d X ~ j ) 2 = 2 X ~ j P ~ j d τ . (90) d(\widetilde{X}_j^2)=2\widetilde{X}_j\,d\widetilde{X}_j+(d\widetilde{X}_j)^2=2\widetilde{X}_j\widetilde{P}_j\,d\tau. \tag*{(90)} d ( X j 2 ) = 2 X j d X j + ( d X j ) 2 = 2 X j P j d τ . (90)
Taking expectations and integrating gives
d S x x d τ = 2 S x p , \frac{dS_{xx}}{d\tau}=2S_{xp}, d τ d S xx = 2 S x p ,
S x x ( τ ) = g 2 γ 2 ∫ 0 τ d r ( 1 − e − γ r ) 2 = g 2 γ 2 [ τ − 2 ( 1 − e − γ τ ) γ + 1 − e − 2 γ τ 2 γ ] . (91) \begin{aligned}
S_{xx}(\tau)&=\frac{g^2}{\gamma^2}\int_0^\tau dr\left(1-e^{-\gamma r}\right)^2\\
&=\frac{g^2}{\gamma^2}\left[\tau-\frac{2(1-e^{-\gamma\tau})}{\gamma}+\frac{1-e^{-2\gamma\tau}}{2\gamma}\right]. \tag*{(91)}
\end{aligned} S xx ( τ ) = γ 2 g 2 ∫ 0 τ d r ( 1 − e − γ r ) 2 = γ 2 g 2 [ τ − γ 2 ( 1 − e − γ τ ) + 2 γ 1 − e − 2 γ τ ] . (91)
The same covariance structure holds for each coordinate, with zero covariance between different coordinates. Thus, the joint covariance is
Σ ( τ ) = ( S x x I d S x p I d S x p I d S p p I d ) . (92) \Sigma(\tau)=\begin{pmatrix}
S_{xx}I_d & S_{xp}I_d\\
S_{xp}I_d & S_{pp}I_d
\end{pmatrix}. \tag*{(92)} Σ ( τ ) = ( S xx I d S x p I d S x p I d S pp I d ) . (92)
Reference kernel and sampler. The covariance factor satisfying L L T = Σ LL^\mathsf{T}=\Sigma L L T = Σ is
L ( τ ) = ( S x x I d 0 S x p S x x I d S p p − S x p 2 S x x I d ) . (93) L(\tau)=\begin{pmatrix}
\sqrt{S_{xx}}I_d & 0\\
\frac{S_{xp}}{\sqrt{S_{xx}}}I_d & \sqrt{S_{pp}-\frac{S_{xp}^2}{S_{xx}}}I_d
\end{pmatrix}. \tag*{(93)} L ( τ ) = ( S xx I d S xx S x p I d 0 S pp − S xx S x p 2 I d ) . (93)
For g > 0 g>0 g > 0 and τ > 0 \tau>0 τ > 0 , this is the Cholesky factor with positive diagonal. At τ = 0 \tau=0 τ = 0 , we define L ( 0 ) = 0 L(0)=0 L ( 0 ) = 0 by continuity.
The reference kernel and sampler are therefore
q T r e f ( y ∣ x ) = N ( y ∣ μ ( x , τ ) , Σ ( τ ) ) , (94) q_T^{\mathrm{ref}}(y\mid x)=\mathcal{N}\left(y\mid\mu(x,\tau),\Sigma(\tau)\right), \tag*{(94)} q T ref ( y ∣ x ) = N ( y ∣ μ ( x , τ ) , Σ ( τ ) ) , (94)
y x = μ x + S x x z x , y_x=\mu_x+\sqrt{S_{xx}}\,z_x, y x = μ x + S xx z x ,
y p = μ p + S x p S x x z x + S p p − S x p 2 S x x z p , (95) y_p=\mu_p+\frac{S_{xp}}{\sqrt{S_{xx}}}\,z_x+\sqrt{S_{pp}-\frac{S_{xp}^2}{S_{xx}}}\,z_p, \tag*{(95)} y p = μ p + S xx S x p z x + S pp − S xx S x p 2 z p , (95)
where z x , z p ∼ N ( 0 , I d ) z_x,z_p\sim\mathcal{N}(0,I_d) z x , z p ∼ N ( 0 , I d ) are independent. The shared z x z_x z x produces the position–momentum correlation derived above.
Midpoint reference and leading corrections
For the duration schedule h ( τ ) h(\tau) h ( τ ) of § 3.3 , h ( τ ) = τ h(\tau)=\tau h ( τ ) = τ for τ ≤ τ 0 \tau\le\tau_0 τ ≤ τ 0 , so the schedule does not affect the expansions below. We retain the smoothness and moment assumptions stated above. Evaluating the vector field at X m i d = X + 1 2 h ( τ ) P = X + τ 2 P X_{\mathrm{mid}}=X+\frac{1}{2}h(\tau)P=X+\frac{\tau}{2}P X mid = X + 2 1 h ( τ ) P = X + 2 τ P for τ ≤ τ 0 \tau\le\tau_0 τ ≤ τ 0 gives the reference mean
μ x = X + ϕ 1 ( τ ) P + ϕ 2 ( τ ) v ( X m i d ) , μ p = e − γ τ P + ϕ 1 ( τ ) v ( X m i d ) . (96) \begin{aligned}
\mu_x &= X+\phi_1(\tau)P+\phi_2(\tau)v(X_{\mathrm{mid}}),\\
\mu_p &= e^{-\gamma\tau}P+\phi_1(\tau)v(X_{\mathrm{mid}}). \tag*{(96)}
\end{aligned} μ x μ p = X + ϕ 1 ( τ ) P + ϕ 2 ( τ ) v ( X mid ) , = e − γ τ P + ϕ 1 ( τ ) v ( X mid ) . (96)
Its covariance remains that of Eq. (92) . We denote its blocks by Σ x x = S x x I d \Sigma_{xx}=S_{xx}I_d Σ xx = S xx I d , Σ x p = S x p I d \Sigma_{xp}=S_{xp}I_d Σ x p = S x p I d , and Σ p p = S p p I d \Sigma_{pp}=S_{pp}I_d Σ pp = S pp I d . Throughout this subsection, μ \mu μ denotes the midpoint reference mean.
Mean corrections. Let μ x ⋆ = E [ X τ ] \mu_x^\star=\mathbb{E}[X_\tau] μ x ⋆ = E [ X τ ] and μ p ⋆ = E [ P τ ] \mu_p^\star=\mathbb{E}[P_\tau] μ p ⋆ = E [ P τ ] denote the exact conditional means. Taking expectations of the SDE gives
∂ μ x ⋆ ∂ τ = μ p ⋆ , ∂ μ p ⋆ ∂ τ = E [ v ( X τ ) ] − γ μ p ⋆ , μ x ⋆ ( 0 ) = X , μ p ⋆ ( 0 ) = P . (97) \begin{aligned}
\frac{\partial\mu_x^\star}{\partial\tau} &= \mu_p^\star,\\
\frac{\partial\mu_p^\star}{\partial\tau} &= \mathbb{E}[v(X_\tau)]-\gamma\mu_p^\star,\qquad\mu_x^\star(0)=X,\quad\mu_p^\star(0)=P. \tag*{(97)}
\end{aligned} ∂ τ ∂ μ x ⋆ ∂ τ ∂ μ p ⋆ = μ p ⋆ , = E [ v ( X τ )] − γ μ p ⋆ , μ x ⋆ ( 0 ) = X , μ p ⋆ ( 0 ) = P . (97)
Since position has no direct diffusion term, differentiating the expected vector field at the source gives
∂ ∂ τ E [ v ( X τ ) ] ∣ τ = 0 = ∂ v ( X ) ∂ X P , (98) \left.\frac{\partial}{\partial\tau}\mathbb{E}[v(X_\tau)]\right|_{\tau=0}
=\frac{\partial v(X)}{\partial X}P, \tag*{(98)} ∂ τ ∂ E [ v ( X τ )] τ = 0 = ∂ X ∂ v ( X ) P , (98)
∂ 2 ∂ τ 2 E [ v ( X τ ) ] ∣ τ = 0 = ∂ 2 v ( X ) ∂ X 2 [ P , P ] + ∂ v ( X ) ∂ X ( v ( X ) − γ P ) . (99) \left.\frac{\partial^2}{\partial\tau^2}\mathbb{E}[v(X_\tau)]\right|_{\tau=0}
=\frac{\partial^2v(X)}{\partial X^2}[P,P]+\frac{\partial v(X)}{\partial X}\left(v(X)-\gamma P\right). \tag*{(99)} ∂ τ 2 ∂ 2 E [ v ( X τ )] τ = 0 = ∂ X 2 ∂ 2 v ( X ) [ P , P ] + ∂ X ∂ v ( X ) ( v ( X ) − γ P ) . (99)
Here, ∂ 2 v ( X ) ∂ X 2 [ P , P ] \frac{\partial^2v(X)}{\partial X^2}[P,P] ∂ X 2 ∂ 2 v ( X ) [ P , P ] is the second spatial derivative applied to ( P , P ) (P,P) ( P , P ) . The midpoint vector field has the expansion
v ( X m i d ) = v ( X ) + τ 2 ∂ v ( X ) ∂ X P + τ 2 8 ∂ 2 v ( X ) ∂ X 2 [ P , P ] + O ( τ 3 ) . (100) v(X_{\mathrm{mid}})=v(X)+\frac{\tau}{2}\frac{\partial v(X)}{\partial X}P+\frac{\tau^2}{8}\frac{\partial^2v(X)}{\partial X^2}[P,P]+O(\tau^3). \tag*{(100)} v ( X mid ) = v ( X ) + 2 τ ∂ X ∂ v ( X ) P + 8 τ 2 ∂ X 2 ∂ 2 v ( X ) [ P , P ] + O ( τ 3 ) . (100)
Taylor expanding the exact mean equations and subtracting Eq. (96) yields
μ x ⋆ − μ x = − τ 3 12 ∂ v ( X ) ∂ X P + O ( τ 4 ) , (101) \mu_x^\star-\mu_x=-\frac{\tau^3}{12}\frac{\partial v(X)}{\partial X}P+O(\tau^4), \tag*{(101)} μ x ⋆ − μ x = − 12 τ 3 ∂ X ∂ v ( X ) P + O ( τ 4 ) , (101)
μ p ⋆ − μ p = τ 3 [ 1 24 ∂ 2 v ( X ) ∂ X 2 [ P , P ] + 1 6 ∂ v ( X ) ∂ X v ( X ) − γ 12 ∂ v ( X ) ∂ X P ] + O ( τ 4 ) . (102) \mu_p^\star-\mu_p=\tau^3\left[\frac{1}{24}\frac{\partial^2v(X)}{\partial X^2}[P,P]+\frac{1}{6}\frac{\partial v(X)}{\partial X}v(X)-\frac{\gamma}{12}\frac{\partial v(X)}{\partial X}P\right]+O(\tau^4). \tag*{(102)} μ p ⋆ − μ p = τ 3 [ 24 1 ∂ X 2 ∂ 2 v ( X ) [ P , P ] + 6 1 ∂ X ∂ v ( X ) v ( X ) − 12 γ ∂ X ∂ v ( X ) P ] + O ( τ 4 ) . (102)
Thus, both mean corrections start at order τ 3 \tau^3 τ 3 . Evaluating the vector field at the source instead leaves a momentum mean correction τ 2 2 ∂ v ( X ) ∂ X P + O ( τ 3 ) \frac{\tau^2}{2}\frac{\partial v(X)}{\partial X}P+O(\tau^3) 2 τ 2 ∂ X ∂ v ( X ) P + O ( τ 3 ) . The midpoint removes this term while preserving the reference’s infinitesimal drift. Since the covariance is unchanged and the reference remains Gaussian, its diffusion and locality conditions are also preserved.
Covariance corrections. The Itô product rule gives the exact conditional covariance equations
∂ ∂ τ Cov ( X τ ) = Cov ( X τ , P τ ) + Cov ( P τ , X τ ) , (103) \frac{\partial}{\partial\tau}\operatorname{Cov}(X_\tau)
=\operatorname{Cov}(X_\tau,P_\tau)+\operatorname{Cov}(P_\tau,X_\tau), \tag*{(103)} ∂ τ ∂ Cov ( X τ ) = Cov ( X τ , P τ ) + Cov ( P τ , X τ ) , (103)
∂ ∂ τ Cov ( X τ , P τ ) = Cov ( P τ ) − γ Cov ( X τ , P τ ) + Cov ( X τ , v ( X τ ) ) , (104) \frac{\partial}{\partial\tau}\operatorname{Cov}(X_\tau,P_\tau)
=\operatorname{Cov}(P_\tau)-\gamma\operatorname{Cov}(X_\tau,P_\tau)+\operatorname{Cov}(X_\tau,v(X_\tau)), \tag*{(104)} ∂ τ ∂ Cov ( X τ , P τ ) = Cov ( P τ ) − γ Cov ( X τ , P τ ) + Cov ( X τ , v ( X τ )) , (104)
∂ ∂ τ Cov ( P τ ) = − 2 γ Cov ( P τ ) + g 2 I d + Cov ( v ( X τ ) , P τ ) + Cov ( P τ , v ( X τ ) ) . (105) \frac{\partial}{\partial\tau}\operatorname{Cov}(P_\tau)
=-2\gamma\operatorname{Cov}(P_\tau)+g^2I_d+\operatorname{Cov}(v(X_\tau),P_\tau)+\operatorname{Cov}(P_\tau,v(X_\tau)). \tag*{(105)} ∂ τ ∂ Cov ( P τ ) = − 2 γ Cov ( P τ ) + g 2 I d + Cov ( v ( X τ ) , P τ ) + Cov ( P τ , v ( X τ )) . (105)
All covariance blocks vanish at τ = 0 \tau=0 τ = 0 . The reference captures the leading terms
Σ p p = g 2 τ I d + O ( τ 2 ) , \Sigma_{pp}=g^2\tau I_d+O(\tau^2), Σ pp = g 2 τ I d + O ( τ 2 ) ,
Σ x p = g 2 τ 2 2 I d + O ( τ 3 ) , \Sigma_{xp}=\frac{g^2\tau^2}{2}I_d+O(\tau^3), Σ x p = 2 g 2 τ 2 I d + O ( τ 3 ) ,
Σ x x = g 2 τ 3 3 I d + O ( τ 4 ) . (106) \Sigma_{xx}=\frac{g^2\tau^3}{3}I_d+O(\tau^4). \tag*{(106)} Σ xx = 3 g 2 τ 3 I d + O ( τ 4 ) . (106)
Expanding the vector field about X X X gives
Cov ( v ( X τ ) , P τ ) = g 2 τ 2 2 ∂ v ( X ) ∂ X + O ( τ 3 ) , (107) \operatorname{Cov}(v(X_\tau),P_\tau)=\frac{g^2\tau^2}{2}\frac{\partial v(X)}{\partial X}+O(\tau^3), \tag*{(107)} Cov ( v ( X τ ) , P τ ) = 2 g 2 τ 2 ∂ X ∂ v ( X ) + O ( τ 3 ) , (107)
Cov ( X τ , v ( X τ ) ) = g 2 τ 3 3 ( ∂ v ( X ) ∂ X ) T + O ( τ 4 ) . (108) \operatorname{Cov}(X_\tau,v(X_\tau))=\frac{g^2\tau^3}{3}\left(\frac{\partial v(X)}{\partial X}\right)^{\mathsf T}+O(\tau^4). \tag*{(108)} Cov ( X τ , v ( X τ )) = 3 g 2 τ 3 ( ∂ X ∂ v ( X ) ) T + O ( τ 4 ) . (108)
Subtracting the reference covariance equations and integrating the leading terms therefore yields
Cov ( P τ ) − Σ p p = g 2 τ 3 6 [ ∂ v ( X ) ∂ X + ( ∂ v ( X ) ∂ X ) T ] + O ( τ 4 ) , (109) \operatorname{Cov}(P_\tau)-\Sigma_{pp}=\frac{g^2\tau^3}{6}\left[\frac{\partial v(X)}{\partial X}+\left(\frac{\partial v(X)}{\partial X}\right)^{\mathsf T}\right]+O(\tau^4), \tag*{(109)} Cov ( P τ ) − Σ pp = 6 g 2 τ 3 [ ∂ X ∂ v ( X ) + ( ∂ X ∂ v ( X ) ) T ] + O ( τ 4 ) , (109)
Cov ( X τ , P τ ) − Σ x p = g 2 τ 4 24 [ ∂ v ( X ) ∂ X + 3 ( ∂ v ( X ) ∂ X ) T ] + O ( τ 5 ) , (110) \operatorname{Cov}(X_\tau,P_\tau)-\Sigma_{xp}=\frac{g^2\tau^4}{24}\left[\frac{\partial v(X)}{\partial X}+3\left(\frac{\partial v(X)}{\partial X}\right)^{\mathsf T}\right]+O(\tau^5), \tag*{(110)} Cov ( X τ , P τ ) − Σ x p = 24 g 2 τ 4 [ ∂ X ∂ v ( X ) + 3 ( ∂ X ∂ v ( X ) ) T ] + O ( τ 5 ) , (110)
Cov ( X τ ) − Σ x x = g 2 τ 5 30 [ ∂ v ( X ) ∂ X + ( ∂ v ( X ) ∂ X ) T ] + O ( τ 6 ) . (111) \operatorname{Cov}(X_\tau)-\Sigma_{xx}=\frac{g^2\tau^5}{30}\left[\frac{\partial v(X)}{\partial X}+\left(\frac{\partial v(X)}{\partial X}\right)^{\mathsf T}\right]+O(\tau^6). \tag*{(111)} Cov ( X τ ) − Σ xx = 30 g 2 τ 5 [ ∂ X ∂ v ( X ) + ( ∂ X ∂ v ( X ) ) T ] + O ( τ 6 ) . (111)
These corrections are the same for the source and midpoint references because their covariances are identical.
Free-flight term. For atom i i i , we scale the position shift as b k , i = ℓ i ( τ ) b ~ k , i b_{k,i}=\ell_i(\tau)\widetilde{b}_{k,i} b k , i = ℓ i ( τ ) b k , i , where b ~ k , i \widetilde{b}_{k,i} b k , i is the bounded, dimensionless network output. In physical units,
ℓ i ( τ ) = v t h , i τ 1 + ( v t h , i τ / ℓ max ) 2 , (112) \ell_i(\tau)=\frac{v_{\mathrm{th},i}\tau}{\sqrt{1+(v_{\mathrm{th},i}\tau/\ell_{\max})^2}}, \tag*{(112)} ℓ i ( τ ) = 1 + ( v th , i τ / ℓ m a x ) 2 v th , i τ , (112)
with thermal velocity v t h , i = k B T / M i v_{\mathrm{th},i}=\sqrt{k_{\mathrm B}T/M_i} v th , i = k B T / M i and ℓ max = 0.01 n m \ell_{\max}=0.01\ \mathrm{nm} ℓ m a x = 0.01 nm . This factor follows the ballistic free-flight displacement at small elapsed times and saturates to ℓ max \ell_{\max} ℓ m a x . Since ℓ i ( τ ) = v t h , i τ + O ( τ 3 ) \ell_i(\tau)=v_{\mathrm{th},i}\tau+O(\tau^3) ℓ i ( τ ) = v th , i τ + O ( τ 3 ) , it contributes one additional power of τ \tau τ to the position shift, giving O ( τ 4 ) O(\tau^4) O ( τ 4 ) when combined with the cubic gate. In dimensionless coordinates, we divide ℓ i \ell_i ℓ i by the position normalization scale.
Gate orders. We select gate orders empirically within the family that preserves the boundary conditions, using ( n a , n b ) = ( 2 , 3 ) (n_a,n_b)=(2,3) ( n a , n b ) = ( 2 , 3 ) for momentum updates and ( 3 , 3 ) (3,3) ( 3 , 3 ) for position updates. The leading mean and covariance corrections in Eqs. (102) and (111) provide a starting point for these choices. Both mean corrections are O ( τ 3 ) O(\tau^3) O ( τ 3 ) . The momentum and position covariance corrections are O ( τ 3 ) O(\tau^3) O ( τ 3 ) and O ( τ 5 ) O(\tau^5) O ( τ 5 ) , respectively, compared with reference variances of O ( τ ) O(\tau) O ( τ ) and O ( τ 3 ) O(\tau^3) O ( τ 3 ) . Thus, both covariance corrections are of relative order O ( τ 2 ) O(\tau^2) O ( τ 2 ) . Comparing these powers suggests log-scale gates of order O ( τ 2 ) O(\tau^2) O ( τ 2 ) and shifts of order O ( τ 3 ) O(\tau^3) O ( τ 3 ) .
For momentum, the quadratic log-scale gate and cubic shift follow these orders. For position, the cubic log-scale gate suppresses scale updates by one additional power of τ \tau τ . The cubic shift gate also includes the O ( τ ) O(\tau) O ( τ ) free-flight factor in Eq. (112) , giving an effective shift of order O ( τ 4 ) O(\tau^4) O ( τ 4 ) , one order higher than the leading mean correction. Our empirical choices therefore suppress position updates more strongly at small τ \tau τ . This comparison motivates the gate powers without requiring the learned flow to match every leading mean and covariance correction. We establish boundary preservation below.
Boundary preservation. Both second-order references satisfy the boundary conditions and have E ∥ L ( τ ) z ∥ 2 = O ( τ ) \mathbb{E}\lVert L(\tau)z\rVert^2=O(\tau) E ∥ L ( τ ) z ∥ 2 = O ( τ ) . Under the boundedness assumptions of Proposition 2 , define the flow correction as
ξ τ = Ψ ( L ( τ ) z ; x , θ ) − L ( τ ) z . (113) \xi_\tau=\Psi(L(\tau)z;x,\theta)-L(\tau)z. \tag*{(113)} ξ τ = Ψ ( L ( τ ) z ; x , θ ) − L ( τ ) z . (113)
With n a ≥ 2 n_a\ge2 n a ≥ 2 for both blocks and effective shift powers of at least two, the perturbation bound in Eq. (23) gives
( E ∥ ξ τ ∥ 2 ) 1 / 2 ≤ C [ τ 2 ( E ∥ L ( τ ) z ∥ 2 ) 1 / 2 + τ 2 ] = O ( τ 2 ) = o ( τ ) . (114) \left(\mathbb{E}\lVert\xi_\tau\rVert^2\right)^{1/2}\le C\left[\tau^2\left(\mathbb{E}\lVert L(\tau)z\rVert^2\right)^{1/2}+\tau^2\right]=O(\tau^2)=o(\tau). \tag*{(114)} ( E ∥ ξ τ ∥ 2 ) 1/2 ≤ C [ τ 2 ( E ∥ L ( τ ) z ∥ 2 ) 1/2 + τ 2 ] = O ( τ 2 ) = o ( τ ) . (114)
The correction therefore leaves the infinitesimal drift and diffusion unchanged. More generally, min { n a + 1 2 , n b } > 1 \min\{n_a+\frac{1}{2},n_b\}>1 min { n a + 2 1 , n b } > 1 suffices, as in Proposition 2 , which also covers the lower-order gates of Table 1 d. For every ϵ > 0 \epsilon>0 ϵ > 0 , locality follows from
Pr ( ∥ y − x ∥ > ϵ ) ≤ Pr ( ∥ m + L ( τ ) z ∥ > ϵ / 2 ) + 4 E ∥ ξ τ ∥ 2 ϵ 2 = o ( τ ) . (115) \Pr(\lVert y-x\rVert>\epsilon)\le\Pr(\lVert m+L(\tau)z\rVert>\epsilon/2)+\frac{4\mathbb{E}\lVert\xi_\tau\rVert^2}{\epsilon^2}=o(\tau). \tag*{(115)} Pr (∥ y − x ∥ > ϵ ) ≤ Pr (∥ m + L ( τ ) z ∥ > ϵ /2 ) + ϵ 2 4 E ∥ ξ τ ∥ 2 = o ( τ ) . (115)
Implementation Details
Model Details
Equivariance. We center positions at the center of mass and keep positions and momenta in the center-of-mass-free subspace, where densities are evaluated. The learned flow acts on this subspace. We propagate the center-of-mass position and total momentum separately using the exact force-free Langevin transition for the total mass. For peptides, the source configuration defines a rotation R = [ e 1 , e 2 , e 3 ] R=[e_1,e_2,e_3] R = [ e 1 , e 2 , e 3 ] from the N, Cα _\alpha α and C atoms of the first complete residue,
e 1 = X N − X C α ∥ X N − X C α ∥ , e 2 ∝ ( X C − X C α ) − ⟨ X C − X C α , e 1 ⟩ e 1 , e 3 = e 1 × e 2 . (116) e_1=\frac{X_{\mathrm{N}}-X_{\mathrm{C}_\alpha}}{\lVert X_{\mathrm{N}}-X_{\mathrm{C}_\alpha}\rVert},\qquad e_2\propto(X_{\mathrm{C}}-X_{\mathrm{C}_\alpha})-\langle X_{\mathrm{C}}-X_{\mathrm{C}_\alpha},e_1\rangle e_1,\qquad e_3=e_1\times e_2. \tag*{(116)} e 1 = ∥ X N − X C α ∥ X N − X C α , e 2 ∝ ( X C − X C α ) − ⟨ X C − X C α , e 1 ⟩ e 1 , e 3 = e 1 × e 2 . (116)
The flow acts on inputs rotated by R ⊤ R^\top R ⊤ , and its outputs are rotated back by R R R . Since R R R depends only on the source and is orthogonal, the log-determinant is unchanged and the kernel is rotation equivariant. This does not extend to LJ, so instead we apply a random rotation augmentation to each training example.
Gaussian reference duration schedule. For τ > τ 0 \tau>\tau_0 τ > τ 0 , the schedule h ( τ ) h(\tau) h ( τ ) of § 3.3 rescales the transport and vector-field terms of the midpoint mean Eq. (96) and the position rows of the Cholesky factor Eq. (93) , while remaining terms keep the full elapsed time:
μ x = X + h ( τ ) τ ϕ 1 ( τ ) P + ( h ( τ ) τ ) 2 ϕ 2 ( τ ) v ( X m i d ) , μ p = e − γ τ P + h ( τ ) τ ϕ 1 ( τ ) v ( X m i d ) , L h ( τ ) = diag ( h ( τ ) τ I d , I d ) L ( τ ) . (117) \begin{aligned}
\mu_x&=X+\frac{h(\tau)}{\tau}\phi_1(\tau)P+\left(\frac{h(\tau)}{\tau}\right)^2\phi_2(\tau)v(X_{\mathrm{mid}}),\\
\mu_p&=e^{-\gamma\tau}P+\frac{h(\tau)}{\tau}\phi_1(\tau)v(X_{\mathrm{mid}}),\\
L_{h(\tau)}&=\operatorname{diag}\left(\frac{h(\tau)}{\tau}I_d,I_d\right)L(\tau). \tag*{(117)}
\end{aligned} μ x μ p L h ( τ ) = X + τ h ( τ ) ϕ 1 ( τ ) P + ( τ h ( τ ) ) 2 ϕ 2 ( τ ) v ( X mid ) , = e − γ τ P + τ h ( τ ) ϕ 1 ( τ ) v ( X mid ) , = diag ( τ h ( τ ) I d , I d ) L ( τ ) . (117)
with X m i d = X + 1 2 h ( τ ) P X_{\mathrm{mid}}=X+\frac{1}{2}h(\tau)P X mid = X + 2 1 h ( τ ) P . For τ ≤ τ 0 \tau\le\tau_0 τ ≤ τ 0 , h ( τ ) / τ = 1 h(\tau)/\tau=1 h ( τ ) / τ = 1 , resulting in the reference of Section B.3.3.
Reference forces. The midpoint force is projected to remove its net force and its torque about the source positions, so that the reference mean satisfies the same conservation constraints as the flow.
Coupling layers. The flow has K = 8 K=8 K = 8 coupling blocks that alternate between position and momentum updates, starting with positions, without permutations. Each block projects its affine update onto the allowed subspace:
x 2 ( k + 1 ) = x 2 ( k ) + Π k [ ( exp ( ( ω τ ) n a 1 + ( ω τ ) n a ⋅ a k ) − 1 ) ⊙ x 2 ( k ) + ( ω τ ) n b 1 + ( ω τ ) n b ⋅ b k ] . (118) x_2^{(k+1)}=x_2^{(k)}+\Pi_k\left[\left(\exp\left(\frac{(\omega\tau)^{n_a}}{1+(\omega\tau)^{n_a}}\cdot a_k\right)-1\right)\odot x_2^{(k)}+\frac{(\omega\tau)^{n_b}}{1+(\omega\tau)^{n_b}}\cdot b_k\right]. \tag*{(118)} x 2 ( k + 1 ) = x 2 ( k ) + Π k [ ( exp ( 1 + ( ω τ ) n a ( ω τ ) n a ⋅ a k ) − 1 ) ⊙ x 2 ( k ) + 1 + ( ω τ ) n b ( ω τ ) n b ⋅ b k ] . (118)
In position blocks, Π k \Pi_k Π k removes changes to the center of mass. In momentum blocks, it removes changes to total momentum and angular momentum about the source positions. We evaluate the log-determinant on this constrained subspace.
We bound the learned scales and shifts in the standardized units of Section C.2. Position blocks use
a k = tanh α k , b ~ k , i = 4 tanh β k , i , b k , i = ℓ i ( τ ) X s t d b ~ k , i . (119) a_k=\tanh\alpha_k,\qquad\widetilde{b}_{k,i}=4\tanh\beta_{k,i},\qquad b_{k,i}=\frac{\ell_i(\tau)}{X_{\mathrm{std}}}\widetilde{b}_{k,i}. \tag*{(119)} a k = tanh α k , b k , i = 4 tanh β k , i , b k , i = X std ℓ i ( τ ) b k , i . (119)
where ℓ i ( τ ) \ell_i(\tau) ℓ i ( τ ) is the free-flight factor in Eq. (112) . Momentum blocks use
a k = 3 tanh α k , b k = 32 tanh β k . (120) a_k=3\tanh\alpha_k,\qquad b_k=32\tanh\beta_k. \tag*{(120)} a k = 3 tanh α k , b k = 32 tanh β k . (120)
These bounds satisfy the assumption in Proposition 2 .
Angular-momentum map. We observed that the learned model could introduce excess spin that accumulated over successive steps and destabilized long rollouts. For a rotationally invariant potential, internal forces exert no net torque, so angular momentum evolves through friction and thermal noise. We use the Gaussian reference to approximate this evolution and constrain the coupling layers to preserve its angular momentum about the source positions. A final momentum transformation transfers this angular-momentum value to the proposed geometry.
Let P ′ P' P ′ denote the momenta after the coupling layers, before this transformation. Define
Λ = ∑ i X t , i × P i ′ , Δ L = Λ − ∑ i X t + τ , i × P i ′ . (121) \Lambda=\sum_i X_{t,i}\times P'_i,\qquad\Delta L=\Lambda-\sum_i X_{t+\tau,i}\times P'_i. \tag*{(121)} Λ = i ∑ X t , i × P i ′ , Δ L = Λ − i ∑ X t + τ , i × P i ′ . (121)
Let I t + τ I_{t+\tau} I t + τ denote the inertia tensor at the proposed positions. Keeping these positions fixed, we add a rigid rotation:
T ( P ′ ) i = P i ′ + M i [ ( I t + τ − 1 Δ L ) × X t + τ , i ] . (122) T(P')_i=P'_i+M_i\left[(I_{t+\tau}^{-1}\Delta L)\times X_{t+\tau,i}\right]. \tag*{(122)} T ( P ′ ) i = P i ′ + M i [ ( I t + τ − 1 Δ L ) × X t + τ , i ] . (122)
This gives ∑ i X t + τ , i × T ( P ′ ) i = Λ \sum_i X_{t+\tau,i} \times T(P')_i = \Lambda ∑ i X t + τ , i × T ( P ′ ) i = Λ , retaining the reference angular-momentum distribution, including its friction and noise.
For fixed source and proposed positions, T T T is linear in the momenta. To express its determinant, define the matrix
I m i x ( a , b ) = ∑ i M i [ ( a i ⋅ b i ) Id − b i a i T ] , (123) I_{\mathrm{mix}}(a,b)=\sum_i M_i\left[(a_i\cdot b_i)\operatorname{Id}-b_i a_i^{\mathsf{T}}\right], \tag*{(123)} I mix ( a , b ) = i ∑ M i [ ( a i ⋅ b i ) Id − b i a i T ] , (123)
so that I t + τ = I m i x ( X t + τ , X t + τ ) I_{t+\tau}=I_{\mathrm{mix}}(X_{t+\tau},X_{t+\tau}) I t + τ = I mix ( X t + τ , X t + τ ) . When I t + τ I_{t+\tau} I t + τ and I m i x ( X t , X t + τ ) I_{\mathrm{mix}}(X_t,X_{t+\tau}) I mix ( X t , X t + τ ) are nonsingular, the map is invertible and
log ∣ det T ∣ = log ∣ det I m i x ( X t , X t + τ ) ∣ − log ∣ det I t + τ ∣ . (124) \log\lvert\det T\rvert=\log\lvert\det I_{\mathrm{mix}}(X_t,X_{t+\tau})\rvert-\log\lvert\det I_{t+\tau}\rvert. \tag*{(124)} log ∣ det T ∣ = log ∣ det I mix ( X t , X t + τ )∣ − log ∣ det I t + τ ∣ . (124)
Extending T T T to the joint state by leaving positions unchanged, the model becomes
y = T ( x + m ( x , τ ) + Ψ ( L ( τ ) z ; x , θ ) ) . (125) y=T\left(x+m(x,\tau)+\Psi(L(\tau)z;x,\theta)\right). \tag*{(125)} y = T ( x + m ( x , τ ) + Ψ ( L ( τ ) z ; x , θ ) ) . (125)
We include the log-determinant of T T T when evaluating the model density.
Conditioner network. Per-atom tokens of width 384 combine the source and mean embeddings with embeddings of atom type, residue type, residue index, chain length and atom name, and pass through one cross-attention layer over these chemical embeddings and two self-attention layers (Vaswani et al., [58 ] ). A per-atom conditioning vector is computed from the τ \tau τ embedding, the covariance features, the speed and impulse expansions, and the angular-momentum features by a linear map and two residual SwiGLU layers, where τ \tau τ enters through a sinusoidal embedding (Peebles & Xie, 2023). These conditioning features are detailed in the paragraph below. Each coupling block adds to the tokens a linear embedding of the variables it conditions on, divided by the reference widths without the duration schedule, S x x / M i \sqrt{S_{xx}/M_i} S xx / M i and M i S p p \sqrt{M_iS_{pp}} M i S pp , and applies two self-attention and SwiGLU layers with six heads, hidden width 768 and AdaLN-Zero modulation (Peebles & Xie, 2023) by the conditioning vector. A zero-initialized linear head outputs ( α k , β k ) (\alpha_k,\beta_k) ( α k , β k ) for every atom, so the flow is the identity at initialization.
Conditioning features. Besides the source state and τ \tau τ embedding, each particle i i i receives features of the Gaussian reference. For first-order dynamics, the reference has mean displacement m ( x , τ ) = h ( τ ) v ( x ) m(x,\tau)=h(\tau)v(x) m ( x , τ ) = h ( τ ) v ( x ) and width σ ( τ ) = g τ \sigma(\tau)=g\sqrt{\tau} σ ( τ ) = g τ ,
f r e f , i = ( m i ( x , τ ) , log σ ( τ ) ) . (126) f_{\mathrm{ref},i}=\left(m_i(x,\tau),\log\sigma(\tau)\right). \tag*{(126)} f ref , i = ( m i ( x , τ ) , log σ ( τ ) ) . (126)
For second-order dynamics, with m ( x , τ ) m(x,\tau) m ( x , τ ) from Eq. (78) and the covariance blocks of Eq. (92) for atom i i i ,
f r e f , i = ( m i ( x , τ ) , log σ x , i , log σ p , i , ρ x p , i ) , (127) f_{\mathrm{ref},i}=\left(m_i(x,\tau),\log\sigma_{x,i},\log\sigma_{p,i},\rho_{xp,i}\right), \tag*{(127)} f ref , i = ( m i ( x , τ ) , log σ x , i , log σ p , i , ρ x p , i ) , (127)
where, with S x x S_{xx} S xx , S x p S_{xp} S x p and S p p S_{pp} S pp from Section B.3.2 and atom mass M i M_i M i ,
σ x , i = h ( τ ) τ S x x / M i , σ p , i = M i S p p , ρ x p , i = S x p S x x S p p . (128) \sigma_{x,i}=\frac{h(\tau)}{\tau}\sqrt{S_{xx}/M_i},\qquad
\sigma_{p,i}=\sqrt{M_iS_{pp}},\qquad
\rho_{xp,i}=\frac{S_{xp}}{\sqrt{S_{xx}S_{pp}}}. \tag*{(128)} σ x , i = τ h ( τ ) S xx / M i , σ p , i = M i S pp , ρ x p , i = S xx S pp S x p . (128)
Both log widths are min–max normalized to [ − 1 , 1 ] [-1,1] [ − 1 , 1 ] in standardized units, using their smallest value over atoms at τ = 0.01 \tau=0.01 τ = 0.01 fs and their largest at T max T_{\max} T m a x , with τ \tau τ floored at 10 − 6 10^{-6} 1 0 − 6 ps. The mean enters with the source state through four per-atom vectors, X ^ i \hat{X}_i X ^ i , P ^ i \hat{P}_i P ^ i , m x , i / ( v t h , i τ ) m_{x,i}/(v_{\mathrm{th},i}\tau) m x , i / ( v th , i τ ) and m p , i T s t d / ( P s t d , i τ ) m_{p,i}T_{\mathrm{std}}/(P_{\mathrm{std},i}\tau) m p , i T std / ( P std , i τ ) , with each vector u u u mapped to ( u asinh ( ∥ u ∥ / ∥ u ∥ ) , log ( 1 + ∥ u ∥ ) ) (u\operatorname{asinh}(\lVert u\rVert/\lVert u\rVert),\log(1+\lVert u\rVert)) ( u asinh (∥ u ∥ / ∥ u ∥) , log ( 1 + ∥ u ∥)) , which keeps its direction and log-scales its magnitude, embedded by an MLP, and the four embeddings are added. Following Ripken et al. [49 ] , each atom also receives a Gaussian basis expansion of its speed ∥ P ^ i ∥ \lVert\hat{P}_i\rVert ∥ P ^ i ∥ (8 functions on [ 0 , 6 ] [0,6] [ 0 , 6 ] ), and likewise of its reference impulse h ( τ ) τ ϕ 1 ( τ ) ∥ v i ( X m i d ) ∥ / P s t d , i \frac{h(\tau)}{\tau}\phi_1(\tau)\lVert v_i(X_{\mathrm{mid}})\rVert/P_{\mathrm{std},i} τ h ( τ ) ϕ 1 ( τ ) ∥ v i ( X mid )∥ / P std , i (16 functions on [ 0 , 3.5 ] [0,3.5] [ 0 , 3.5 ] ), together with the cosine between the two. All atoms also share two angular-momentum features: Λ \Lambda Λ itself and its deviation from the damped source value e − γ τ ∑ i X t , i × P t , i e^{-\gamma\tau}\sum_i X_{t,i}\times P_{t,i} e − γ τ ∑ i X t , i × P t , i , each divided by its thermal standard deviation and passed into asinh, which is linear near zero and logarithmic for large values.
Training Details
Standardization. After centering molecular positions at the center of mass, we standardize the state and elapsed time as
X ^ i = X i X s t d , P ^ i = P i P s t d , i , τ ^ = τ T s t d . (129) \hat{X}_i=\frac{X_i}{X_{\mathrm{std}}},\qquad
\hat{P}_i=\frac{P_i}{P_{\mathrm{std},i}},\qquad
\hat{\tau}=\frac{\tau}{T_{\mathrm{std}}}. \tag*{(129)} X ^ i = X std X i , P ^ i = P std , i P i , τ ^ = T std τ . (129)
The position scale is the root mean square over training configurations, atoms and Cartesian coordinates, X s t d 2 = ( 3 N ) − 1 E t r a i n [ ∑ i ∥ X i ∥ 2 ] X_{\mathrm{std}}^2=(3N)^{-1}\mathbb{E}_{\mathrm{train}}[\sum_i\lVert X_i\rVert^2] X std 2 = ( 3 N ) − 1 E train [ ∑ i ∥ X i ∥ 2 ] , where N N N is the number of atoms. We use P s t d , i = M i k B T P_{\mathrm{std},i}=\sqrt{M_i k_{\mathrm{B}}T} P std , i = M i k B T at the system temperature and T s t d = 10 T_{\mathrm{std}}=10 T std = 10 fs. These scales remain fixed during training and inference, including momentum augmentation. Generated states are converted back to physical units using the same scales.
Table A1. Standardization scales. Position scales are fitted to the training split. Momentum scales are listed by element at 300 K for AL2/AL4 and 310 K for AL6.
Momentum Augmentation. As done in Ripken et al. [49 ] , we expose the model to a wider range of source momenta. In particular, we sample P i ∼ N ( 0 , M i k B T ) P_i\sim\mathcal{N}(0,M_i k_{\mathrm{B}}T) P i ∼ N ( 0 , M i k B T ) for each atom i i i , then enforce zero total momentum. For 75% of training examples, we draw T ∼ N ( 500 K , ( 150 K ) 2 ) T\sim\mathcal{N}(500\mathrm{K},(150\mathrm{K})^2) T ∼ N ( 500 K , ( 150 K ) 2 ) , and we clamp the temperature below at 50 K. The remaining examples use the system temperature. This augmentation changes the source momentum distribution while the target Langevin dynamics remain at the system temperature.
Algorithm 1 Training KC-maps with L K C \mathcal{L}_{\mathrm{KC}} L KC for Peptides
Require: Kernel q s , t ( y ∣ x ; θ ) q_{s,t}(y\mid x;\theta) q s , t ( y ∣ x ; θ ) , target parameters η \eta η , configurations D \mathcal{D} D , time prior p ( τ ) p(\tau) p ( τ )
repeat
X ∼ D , T ∼ N ( 500 K , ( 150 K ) 2 ) , P ∼ N ( 0 , M k B T ) , x ← ( X , P ) X\sim\mathcal{D},\quad T\sim\mathcal{N}(500\mathrm{K},(150\mathrm{K})^2),\quad P\sim\mathcal{N}(0,M k_{\mathrm{B}}T),\quad x\leftarrow(X,P) X ∼ D , T ∼ N ( 500 K , ( 150 K ) 2 ) , P ∼ N ( 0 , M k B T ) , x ← ( X , P )
τ ∼ p ( τ ) , r ∼ T min / 2 + ( τ − T min ) Beta ( 2 , 2 ) \tau\sim p(\tau),\quad r\sim T_{\min}/2+(\tau-T_{\min})\operatorname{Beta}(2,2) τ ∼ p ( τ ) , r ∼ T m i n /2 + ( τ − T m i n ) Beta ( 2 , 2 )
x ′ ∼ q 0 , r ( x ′ ∣ x ; η ) , y ∼ q r , τ ( y ∣ x ′ ; η ) x'\sim q_{0,r}(x'\mid x;\eta),\quad y\sim q_{r,\tau}(y\mid x';\eta) x ′ ∼ q 0 , r ( x ′ ∣ x ; η ) , y ∼ q r , τ ( y ∣ x ′ ; η )
L K C ← − log q 0 , τ ( y ∣ x ; θ ) \mathcal{L}_{\mathrm{KC}}\leftarrow-\log q_{0,\tau}(y\mid x;\theta) L KC ← − log q 0 , τ ( y ∣ x ; θ )
θ ← ADAMW ( θ , ∇ θ L K C ) , η ← EMA ( θ ) \theta\leftarrow\operatorname{ADAMW}(\theta,\nabla_\theta\mathcal{L}_{\mathrm{KC}}),\quad\eta\leftarrow\operatorname{EMA}(\theta) θ ← ADAMW ( θ , ∇ θ L KC ) , η ← EMA ( θ )
until converged
Inference Details
We apply the following corrections around to our map’s proposal to prevent repeated transitions from accumulating errors during long molecular simulations. Algorithm 2 summarizes one step.
Force capping. While rare, the model can generate a bad configuration which could result in a large force in the subsequent step. To prevent large forces increments, and ultimately prevent non-finite values, we measure the force increment from the reference Gaussian kernel in thermal-momentum units as h ( τ ) τ ϕ 1 ( τ ) ∥ v i ( X m i d ) ∥ / M i k B T \frac{h(\tau)}{\tau}\phi_1(\tau)\lVert v_i(X_{\mathrm{mid}})\rVert/\sqrt{M_i k_{\mathrm{B}}T} τ h ( τ ) ϕ 1 ( τ ) ∥ v i ( X mid )∥ / M i k B T , where h ( τ ) h(\tau) h ( τ ) is the duration schedule of § 3.3 and ϕ 1 ( τ ) = ( 1 − e − γ τ ) / γ \phi_1(\tau)=(1-e^{-\gamma\tau})/\gamma ϕ 1 ( τ ) = ( 1 − e − γ τ ) / γ , and we rescale the forces according to a specified cap. The rescaled forces enter the reference mean and conditioning features.
Momentum correction. Following Ripken et al. [49 ] , we retain the proposed positions and minimally adjust the proposed momenta to match an internal kinetic-energy target K t a r g e t K_{\mathrm{target}} K target . We construct this target to account for the friction and thermal noise of the Langevin dynamics.
To adjust angular momentum and internal kinetic energy separately, we decompose the momenta into rigid rotational and internal components. For positions X i X_i X i , define the angular momentum and inertia tensor as
L = ∑ i X i × P i , I = ∑ i M i ( ∥ X i ∥ 2 Id − X i X i T ) . (130) L=\sum_i X_i\times P_i,\qquad I=\sum_i M_i\left(\lVert X_i\rVert^2\operatorname{Id}-X_iX_i^{\mathsf{T}}\right). \tag*{(130)} L = i ∑ X i × P i , I = i ∑ M i ( ∥ X i ∥ 2 Id − X i X i T ) . (130)
The decomposition is
P i = M i ( I − 1 L × X i ) + P i i n t . (131) P_i=M_i\left(I^{-1}L\times X_i\right)+P_i^{\mathrm{int}}. \tag*{(131)} P i = M i ( I − 1 L × X i ) + P i int . (131)
The internal momenta carry no angular momentum, ∑ i X i × P i i n t = 0 \sum_i X_i \times P_i^{\mathrm{int}} = 0 ∑ i X i × P i int = 0 . Consequently, for any angular velocity Ω \Omega Ω ,
∑ i P i i n t ⋅ [ M i ( Ω × X i ) ] M i = Ω ⋅ ∑ i X i × P i i n t = 0. (132) \sum_i \frac{P_i^{\mathrm{int}} \cdot[M_i(\Omega\times X_i)]}{M_i} = \Omega\cdot\sum_i X_i \times P_i^{\mathrm{int}} = 0. \tag*{(132)} i ∑ M i P i int ⋅ [ M i ( Ω × X i )] = Ω ⋅ i ∑ X i × P i int = 0. (132)
Therefore, the two components are orthogonal in the mass-weighted inner product, and their kinetic energies add:
K t o t = K r o t ( L , I ) + K i n t , K r o t ( L , I ) = L ⊤ I − 1 L 2 k B T , K i n t = ∑ i ∥ P i i n t ∥ 2 2 M i k B T . (133) K^{\mathrm{tot}} = K_{\mathrm{rot}}(L,I) + K^{\mathrm{int}}, \qquad K_{\mathrm{rot}}(L,I) = \frac{L^\top I^{-1}L}{2k_{\mathrm{B}}T}, \qquad K^{\mathrm{int}} = \sum_i \frac{\lVert P_i^{\mathrm{int}}\rVert^2}{2M_i k_{\mathrm{B}}T}. \tag*{(133)} K tot = K rot ( L , I ) + K int , K rot ( L , I ) = 2 k B T L ⊤ I − 1 L , K int = i ∑ 2 M i k B T ∥ P i int ∥ 2 . (133)
This separation allows us to set the internal kinetic energy by rescaling the internal component while keeping the rigid component. We evaluate these quantities at the source state and the proposed state, denoted by subscripts t t t and t + τ t+\tau t + τ . Below, U U U denotes potential energy and K K K denotes internal kinetic energy, both in units of k B T k_{\mathrm{B}}T k B T .
The angular momentum needs no correction, since the map T T T of Section C.1 is part of the model and already sets the angular momentum that the flow keeps about the source positions.
For the internal kinetic-energy target, we need to account for friction and thermal noise. As a result, we use the internal-energy law induced by an Ornstein–Uhlenbeck momentum update at fixed positions. This is the closed-form law of the stochastic velocity-rescaling thermostat [9 ] :
K b a t h = 1 2 ( a 2 K t + 1 − a 2 Z ) 2 + 1 2 ( 1 − a 2 ) Y , a = e − γ τ , (134) K_{\mathrm{bath}} = \frac{1}{2}\left(a\sqrt{2K_t} + \sqrt{1-a^2}Z\right)^2 + \frac{1}{2}(1-a^2)Y, \qquad a = e^{-\gamma\tau}, \tag*{(134)} K bath = 2 1 ( a 2 K t + 1 − a 2 Z ) 2 + 2 1 ( 1 − a 2 ) Y , a = e − γ τ , (134)
where Z ∼ N ( 0 , 1 ) Z \sim\mathcal{N}(0,1) Z ∼ N ( 0 , 1 ) and Y ∼ χ n − 1 2 Y \sim\chi^2_{n-1} Y ∼ χ n − 1 2 are independent, and n = 3 N − 6 n=3N-6 n = 3 N − 6 is the number of internal degrees of freedom. Then, we account for the change in potential energy and the change in rotational energy at fixed source angular momentum:
K t a r g e t = U t − U t + τ + K b a t h + Δ K r o t , (135) K_{\mathrm{target}} = U_t-U_{t+\tau}+K_{\mathrm{bath}}+\Delta K_{\mathrm{rot}}, \tag*{(135)} K target = U t − U t + τ + K bath + Δ K rot , (135)
Δ K r o t = K r o t ( L t , I t ) − K r o t ( L t , I t + τ ) . (136) \Delta K_{\mathrm{rot}} = K_{\mathrm{rot}}(L_t,I_t)-K_{\mathrm{rot}}(L_t,I_{t+\tau}). \tag*{(136)} Δ K rot = K rot ( L t , I t ) − K rot ( L t , I t + τ ) . (136)
Here, Δ K r o t \Delta K_{\mathrm{rot}} Δ K rot represents rotational energy released into internal motion as the inertia tensor changes. When γ = 0 \gamma=0 γ = 0 , we have K b a t h = K t K_{\mathrm{bath}}=K_t K bath = K t where if angular momentum is also conserved, this target enforces conservation of total energy. Unlike Ripken et al. [49 ] , who conserve angular momentum and total energy and apply a thermostat separately, we incorporate the Langevin thermostat into the target construction.
With the target specified, we choose the smallest change to the proposed momenta in the mass-weighted norm:
min P i ′ ∑ i ∥ P i ′ − P t + τ , i ∥ 2 2 M i s.t. ∑ i X t + τ , i × P i ′ = L t + τ , K i n t ( P ′ ) = K t a r g e t . (137) \min_{P_i'} \sum_i \frac{\lVert P_i'-P_{t+\tau,i}\rVert^2}{2M_i}
\quad\text{s.t.} \quad
\sum_i X_{t+\tau,i}\times P_i'=L_{t+\tau}, \qquad K^{\mathrm{int}}(P')=K_{\mathrm{target}}. \tag*{(137)} P i ′ min i ∑ 2 M i ∥ P i ′ − P t + τ , i ∥ 2 s.t. i ∑ X t + τ , i × P i ′ = L t + τ , K int ( P ′ ) = K target . (137)
Orthogonality of the rigid and internal components gives the closed-form correction
P i ′ = M i ( I t + τ − 1 L t + τ × X t + τ , i ) + K t a r g e t K t + τ P t + τ , i i n t . (138) P_i' = M_i\left(I_{t+\tau}^{-1}L_{t+\tau}\times X_{t+\tau,i}\right) + \sqrt{\frac{K_{\mathrm{target}}}{K_{t+\tau}}}P_{t+\tau,i}^{\mathrm{int}}. \tag*{(138)} P i ′ = M i ( I t + τ − 1 L t + τ × X t + τ , i ) + K t + τ K target P t + τ , i int . (138)
Here L t + τ = ∑ i X t + τ , i × P t + τ , i L_{t+\tau}=\sum_i X_{t+\tau,i}\times P_{t+\tau,i} L t + τ = ∑ i X t + τ , i × P t + τ , i , so the first term keeps the angular momentum, while the second sets the internal kinetic energy. The positions and total momentum remain unchanged.
For numerical stability, both kinetic energies in the rescaling ratio are floored at 0.1 0.1 0.1 .
Adaptive proposal retries. We redraw a proposal if it is non-finite, ∣ ( U t + τ + K t + τ ) − ( U t + K t ) ∣ > H t h r e s h o l d \left|(U_{t+\tau}+K_{t+\tau})-(U_t+K_t)\right|>H_{\mathrm{threshold}} ∣ ( U t + τ + K t + τ ) − ( U t + K t ) ∣ > H threshold , K t a r g e t < 0.1 K_{\mathrm{target}}<0.1 K target < 0.1 , or det T < 0.05 \det T<0.05 det T < 0.05 . Each attempt uses fresh proposal noise and its own heat-bath draw to compute K t a r g e t K_{\mathrm{target}} K target . After N r e t r y N_{\mathrm{retry}} N retry unsuccessful attempts at the same step size, we halve τ \tau τ , down to a minimum of 1 f s 1\mathrm{fs} 1 fs . If all attempts fail, we use the last proposal at the minimum step size. Only in this case, if K t a r g e t < 0.1 K_{\mathrm{target}}<0.1 K target < 0.1 , we replace it with max ( K b a t h + Δ K r o t , 0.1 ) \max(K_{\mathrm{bath}}+\Delta K_{\mathrm{rot}},0.1) max ( K bath + Δ K rot , 0.1 ) . A chain stops if the committed state is non-finite. The momentum correction uses the duration and heat-bath draw of the committed proposal, and the chain clock advances by that duration. The next step starts again at the maximum step size.
Algorithm 2 One inference step
Require: Kernel parameters θ \theta θ , current state x t = ( X t , P t ) x_t=(X_t,P_t) x t = ( X t , P t ) , step size τ \tau τ , z ∼ N ( 0 , I ) z\sim\mathcal{N}(0,I) z ∼ N ( 0 , I )
v ( X m i d ) ← FORCECAPFILTER v ( X m i d ) v(X_{\mathrm{mid}}) \leftarrow\operatorname{FORCECAPFILTER}_v(X_{\mathrm{mid}}) v ( X mid ) ← FORCECAPFILTER v ( X mid )
X t + τ , P t + τ ← T ( x t + m ( x t , τ ) + Ψ ( L ( τ ) z ; x t , θ ) ) X_{t+\tau},P_{t+\tau} \leftarrow T\left(x_t+m(x_t,\tau)+\Psi(L(\tau)z;x_t,\theta)\right) X t + τ , P t + τ ← T ( x t + m ( x t , τ ) + Ψ ( L ( τ ) z ; x t , θ ) )
( X ′ , P ′ , τ u s e d , K b a t h ) ← REDRAWFILTER ( x t , X t + τ , P t + τ , τ ) (X',P',\tau_{\mathrm{used}},K_{\mathrm{bath}}) \leftarrow\operatorname{REDRAWFILTER}(x_t,X_{t+\tau},P_{t+\tau},\tau) ( X ′ , P ′ , τ used , K bath ) ← REDRAWFILTER ( x t , X t + τ , P t + τ , τ )
P ′ ← MOMENTUMCORRECTIONFILTER ( x t , X ′ , P ′ , τ u s e d , K b a t h ) P' \leftarrow\operatorname{MOMENTUMCORRECTIONFILTER}(x_t,X',P',\tau_{\mathrm{used}},K_{\mathrm{bath}}) P ′ ← MOMENTUMCORRECTIONFILTER ( x t , X ′ , P ′ , τ used , K bath )
return ( X ′ , P ′ , τ u s e d ) (X',P',\tau_{\mathrm{used}}) ( X ′ , P ′ , τ used )
Experimental Setup
Lennard-Jones systems
Potential. We use the confined Lennard-Jones systems studied by Köhler et al. [30 ] ; Akhound-Sadegh et al. [2 ] . For N N N particles, let d i j = ∥ x i − x j ∥ d_{ij}=\lVert x_i-x_j\rVert d ij = ∥ x i − x j ∥ and x ˉ = N − 1 ∑ i x i \bar{x}=N^{-1}\sum_i x_i x ˉ = N − 1 ∑ i x i . The potential is
E ( x ) = ϵ ∑ i < j [ ( r m d i j ) 12 − 2 ( r m d i j ) 6 ] + c 2 ∑ i ∥ x i − x ˉ ∥ 2 (139) E(x)=\epsilon\sum_{i<j}\left[\left(\frac{r_m}{d_{ij}}\right)^{12}-2\left(\frac{r_m}{d_{ij}}\right)^6\right]+\frac{c}{2}\sum_i\lVert x_i-\bar{x}\rVert^2 \tag*{(139)} E ( x ) = ϵ i < j ∑ [ ( d ij r m ) 12 − 2 ( d ij r m ) 6 ] + 2 c i ∑ ∥ x i − x ˉ ∥ 2 (139)
with ϵ = 2 \epsilon=2 ϵ = 2 , r m = 1 r_m=1 r m = 1 and c = 1 c=1 c = 1 . We learn overdamped Langevin transitions with drift D ∇ log π ( x ) D\nabla\log\pi(x) D ∇ log π ( x ) and diffusion 2 D \sqrt{2D} 2 D , where π ( x ) ∝ exp ( − E ( x ) / k B T ) \pi(x)\propto\exp(-E(x)/k_BT) π ( x ) ∝ exp ( − E ( x ) / k B T ) , k B T = 1 k_BT=1 k B T = 1 and D = 0.02 D=0.02 D = 0.02 . Positions and noise are restricted to the zero-center-of-mass subspace, with 3 ( N − 1 ) 3(N-1) 3 ( N − 1 ) degrees of freedom.
Data. At source temperature T S = 1 T_S=1 T S = 1 , we use the public splits of Akhound-Sadegh et al. [1 ] , with 100k/10k/10k training/validation/test configurations for LJ-13 and 10k/10k/10k for LJ-55. At T S = 4 T_S=4 T S = 4 , we generate 200k configurations per system using the NUTS (Hoffman & Gelman, [17 ] ) implementation as done in Akhound-Sadegh et al. [2 ] , with energy E / 4 E/4 E /4 . In both settings, the target dynamics and evaluation data remain at T = 1 T=1 T = 1 . Training draws source configurations independently and uniformly with replacement, with a random rotation for each draw.
Model. The Gaussian reference has mean x + D h ( τ ) ∇ log π ( x ) x+D\,h(\tau)\nabla\log\pi(x) x + D h ( τ ) ∇ log π ( x ) and covariance 2 D τ I 2D\tau I 2 D τ I on the center-of-mass-free subspace, using the duration cap in § 3.3 with τ 0 = δ = 0.01 \tau_0=\delta=0.01 τ 0 = δ = 0.01 . The flow has 9 coupling blocks, each updating one Cartesian coordinate across all particles conditioned on the other two coordinates. Each coupling network uses one transformer layer with AdaLN-Zero conditioning and a SwiGLU feed-forward layer. The context includes the source geometry, time embedding of τ \tau τ , the reference mean and variance, and force on the source geometry.
Alanine peptides
Data. For training, AL2 uses the 1 μ \mu μ s trajectory of Köhler et al. [31 ] . AL4 and AL6 use the datasets of Tan et al. [55 ] . Table A2 lists the force fields (Lindorff-Larsen et al., 2010; Maier et al., 2015) and OBC solvent models [43 ] . We randomly split the available configurations 80/10/10 with seed 42, following the split procedure of Ripken et al. [49 ] for AL2. For AL4 and AL6, we combine the original SBG splits before reshuffling and doing the random splits. During training, we draw positions independently and uniformly from the training split, and do not use trajectory order or paired transitions. Source momenta are sampled independently using the augmentation in Section C.2. Energies and forces are implemented in JAX [7 ] and checked against OpenMM [11 ] .
Table A2. Peptide data and physical settings. Training uses random configurations from these datasets. The learned dynamics use the listed force fields, temperatures and friction coefficients. The source trajectories use a 1 fs integration step with flexible bonds.
Training
Objective. Our main models use the Kolmogorov–Chapman objective in Eq. (12) with two transitions from the EMA target kernel (Algorithm 1). We split each sampled elapsed time τ \tau τ into two durations using a rescaled Beta(2, 2) distribution, with each step lasting at least T min / 2 T_{\min}/2 T m i n /2 . The minimum elapsed time T min T_{\min} T m i n is listed in Table A3 . For numerical stability, we remove the parameter-independent terms from the NLL, divide by the number of unconstrained degrees of freedom and cap each sample’s NLL at 10. Additionally, we train with the detailed-balance objective in Eq. (20) in the last 15% of training steps over 25% of the batch, with λ \lambda λ increasing linearly from 0 to 0.2 over 5K steps for all molecular systems. Source momenta for the detailed-balance objective are sampled at the target temperature. We omit this objective in the ablations.
Table A3. Training hyperparameters. LJ times are in reduced units. For LJ, T max T_{\max} T m a x is given for T S = 1 / T S = 4 T_S = 1 / T_S = 4 T S = 1/ T S = 4 .
Optimization. We use AdamW [37 ] with β = ( 0.9 , 0.95 ) \beta= (0.9, 0.95) β = ( 0.9 , 0.95 ) for AL2 and AL4 and β 2 = 0.9025 \beta_2 = 0.9025 β 2 = 0.9025 for AL6, ϵ = 10 − 8 \epsilon= 10^{-8} ϵ = 1 0 − 8 , weight decay 10 − 8 10^{-8} 1 0 − 8 and a global gradient-norm limit of 1. The learning rate warms up linearly to 3 × 10 − 4 3 \times10^{-4} 3 × 1 0 − 4 and then decays to 3 × 10 − 6 3 \times10^{-6} 3 × 1 0 − 6 with a cosine schedule. Elapsed times follow the mixture prior in § 3.3 , with T max T_{\max} T m a x increased log-linearly during early training. Table A3 lists the settings for each system. The LJ PF-ODE comparison uses the same architecture, with T max = 0.05 T_{\max} = 0.05 T m a x = 0.05 annealed from 0.008. Its training budget matches the FLOPs of KC-KL at T S = 1 T_S = 1 T S = 1 , resulting in 275k updates for LJ-13 and 545k for LJ-55. We train on a heterogeneous cluster consisting of L40s and H100s.
HFM training. For AL4 and AL6, we train HFM using the public implementation of Ripken et al. [49 ] , with the same data splits and force fields as our method. The model has six transformer layers, 256 channels and eight attention heads. We use a global batch size of 512 and seed 42. AdamW uses β = ( 0.9 , 0.95 ) \beta= (0.9, 0.95) β = ( 0.9 , 0.95 ) , no weight decay and gradient clipping at norm 5. The learning rate increases from 10 − 6 10^{-6} 1 0 − 6 to 10 − 4 10^{-4} 1 0 − 4 over the first 1% of updates, then follows cosine decay to 10 − 8 10^{-8} 1 0 − 8 . The maximum elapsed time is 10 fs, with 75% of training samples assigned zero elapsed time and the remainder drawn using HFM’s improved time sampler. The momentum-augmentation temperature has mean 300 K for AL4 and 310 K for AL6, with standard deviation 150 K. We evaluate the final EMA parameters.
Training compute. We set the HFM update counts to match the estimated FLOPs of our method on each system (Table A4 ). We estimate FLOPs using JAX 0.9.2 CPU cost analysis of the loss and its parameter gradients, multiplied by the number of training updates. This includes model evaluations, teacher evaluations, Jacobian–vector products and force computations within the loss.
Evaluation
Lennard-Jones. We run 5000 parallel Metropolis–Hastings chains (Hastings, [15 ] ) for 5000 steps, using learned proposals at τ = 0.05 \tau= 0.05 τ = 0.05 for LJ-13 and τ = 0.02 \tau= 0.02 τ = 0.02 for LJ-55. Each step computes the Metropolis-Hastings acceptance ratio to accept or reject the proposal. All chains start from the same energy-minimized configuration, and we evaluate their final states. MALA uses the same initialization and number of chains. We sweep physical step sizes h ∈ { 10 − 4 , 3 × 10 − 4 , 10 − 3 , 3 × 10 − 3 , 10 − 2 , 0.02 , 0.05 } h \in\{10^{-4}, 3 \times10^{-4}, 10^{-3}, 3 \times10^{-3}, 10^{-2}, 0.02, 0.05\} h ∈ { 1 0 − 4 , 3 × 1 0 − 4 , 1 0 − 3 , 3 × 1 0 − 3 , 1 0 − 2 , 0.02 , 0.05 } , with 5000 steps of sampling on one L40S per setting. We select h h h using validation Geo-W 2 W_2 W 2 , giving 3 × 10 − 4 3 \times10^{-4} 3 × 1 0 − 4 for LJ-13 and 10 − 4 10^{-4} 1 0 − 4 for LJ-55.
Energy W 2 W_2 W 2 compares the 5000 endpoint energies with the 10k test energies. Geo-W 2 W_2 W 2 compares the endpoints with 5000 test configurations as done in (Klein et al., [28 ] ). Published baseline values are taken from Potaptchik et al. [47 ] .
Molecular reference trajectories. Training uses samples drawn independently from the datasets provided in (Köhler et al., [31 ] ) for AL2 and (Tan et al., [55 ] ) for AL4 and AL6, whereas evaluation compares generated trajectories with long MD references. The original AL2 data were generated with the legacy LangevinIntegrator in an older OpenMM release (Eastman et al., [11 ] ). To reduce discretization bias in evaluation and ensure accurate coverage of all modes, we generate new AL2, AL4, and AL6 references with OpenMM 8.5.1 using LangevinMiddleIntegrator to serve as ground truth (Leimkuhler & Matthews, [33 ] ), keeping the settings in Table A2 and a 1 fs integration step. The references for all systems were run with 16 independent chains where each chain was simulated for 2 μ \mu μ s, with frames saved every 1 ps and the first 1 ns discarded (burn-in). Each run took 300 GPU-hours, 312 GPU-hours, and 520 GPU-hours for AL2, AL4, and AL6, respectively.
Molecular sampling. For the results reported in Table 3 , we run each setting for one GPU-hour on a single L40S with 4 CPU cores, excluding compilation and initialization. All methods start from the same configuration for each molecule, with independently sampled momenta at the target temperature.
MD: OpenMM LangevinMiddle integrator with a 1 fs step, running 1, 4 or 16 chains.
HFM: the sampler of Ripken et al. [49 ] , including its energy and momentum corrections. We use τ ∈ { 5 , 7 } \tau\in\{5, 7\} τ ∈ { 5 , 7 } fs with { 1 , 16 , 64 } \{1, 16, 64\} { 1 , 16 , 64 } chains for AL2 and AL4, and τ ∈ { 3 , 5 } \tau\in\{3, 5\} τ ∈ { 3 , 5 } fs with { 1 , 4 , 16 } \{1, 4, 16\} { 1 , 4 , 16 } chains for AL6. AL2 uses the released checkpoint. AL4 and AL6 use the EMA checkpoints described in Section D.3.
KC-maps: τ ∈ { 5 , 10 , 20 , 30 , 40 , 50 , 60 , 70 } \tau\in\{5, 10, 20, 30, 40, 50, 60, 70\} τ ∈ { 5 , 10 , 20 , 30 , 40 , 50 , 60 , 70 } fs with { 8 , 16 , 32 } \{8, 16, 32\} { 8 , 16 , 32 } chains for all three molecular systems. We apply the corrections in Section C.3, with a dimensionless force cap of 7, N r e t r y = 5 N_{\mathrm{retry}} = 5 N retry = 5 attempts per step size and F t h r e s h o l d = 40 F_{\mathrm{threshold}} = 40 F threshold = 40 for energies in units of k B T k_B T k B T . Failed proposals trigger step-size halving down to 1 fs.
For each method, Table 3 reports the configuration with the best average across the three metrics: MD with 16 chains for all systems; HFM with τ = 7 \tau= 7 τ = 7 fs and 16 / 64 chains for AL2 / AL4, and τ = 5 \tau= 5 τ = 5 fs with 1 chain for AL6; KC-maps with τ = 30 \tau= 30 τ = 30 fs and 16 chains for AL2 and AL6, and τ = 40 \tau= 40 τ = 40 fs and 8 chains for AL4.
Equilibrium Metrics. For the molecular equilibrium metrics, we discard the first 10% of each chain’s simulated time. We pool the remaining frames and draw 10 4 10^4 1 0 4 without replacement, weighted by the simulated interval each frame represents. We compare these with 10 4 10^4 1 0 4 uniformly drawn reference frames using exact optimal transport, and report the mean and standard deviation over three subsampling draws.
Following Tan et al. [56 ] , we compare the generated and reference configurations using empirical Wasserstein-2 distances. For samples { x i } i = 1 n \{x_i\}_{i=1}^n { x i } i = 1 n and { y j } j = 1 m \{y_j\}_{j=1}^m { y j } j = 1 m , define the empirical distributions μ = 1 n ∑ i = 1 n δ x i \mu= \frac{1}{n}\sum_{i=1}^n \delta_{x_i} μ = n 1 ∑ i = 1 n δ x i and ν = 1 m ∑ j = 1 m δ y j \nu= \frac{1}{m}\sum_{j=1}^m \delta_{y_j} ν = m 1 ∑ j = 1 m δ y j . We compute
W 2 , c ( μ , ν ) = [ min π ∈ Π ( μ , ν ) ∑ i = 1 n ∑ j = 1 m π i j c ( x i , y j ) 2 ] 1 / 2 , (140) W_{2,c}(\mu,\nu) = \left[\min_{\pi\in\Pi(\mu,\nu)} \sum_{i=1}^{n}\sum_{j=1}^{m}\pi_{ij}c(x_i,y_j)^2\right]^{1/2}, \tag*{(140)} W 2 , c ( μ , ν ) = [ π ∈ Π ( μ , ν ) min i = 1 ∑ n j = 1 ∑ m π ij c ( x i , y j ) 2 ] 1/2 , (140)
where Π ( μ , ν ) \Pi(\mu,\nu) Π ( μ , ν ) contains couplings with marginals μ \mu μ and ν \nu ν . The squared cost c ( x , y ) 2 c(x,y)^2 c ( x , y ) 2 determines which properties of the configurations are compared.
For E \mathcal{E} E -W2 _2 2 , let E ( x ) \mathcal{E}(x) E ( x ) denote the potential energy in units of k B T k_BT k B T . We use
c E ( x , y ) 2 = ∣ E ( x ) − E ( y ) ∣ 2 . (141) c_{\mathcal{E}}(x,y)^2 = \lvert\mathcal{E}(x)-\mathcal{E}(y)\rvert^2. \tag*{(141)} c E ( x , y ) 2 = ∣ E ( x ) − E ( y ) ∣ 2 . (141)
For T \mathcal{T} T -W2 _2 2 , let Dihedrals( x ) (x) ( x ) collect all backbone ( ϕ , ψ ) (\phi,\psi) ( ϕ , ψ ) angles in radians. Its length is D = 2 , 6 , 10 D=2,6,10 D = 2 , 6 , 10 for AL2, AL4 and AL6, respectively. We account for angle periodicity using
c T ( x , y ) 2 = ∑ ℓ = 1 D [ ( Dihedrals ( x ) ℓ − Dihedrals ( y ) ℓ + π ) mod 2 π − π ] 2 . (142) c_{\mathcal{T}}(x,y)^2 = \sum_{\ell=1}^{D}\left[(\operatorname{Dihedrals}(x)_{\ell}-\operatorname{Dihedrals}(y)_{\ell}+\pi)\mathbin{\operatorname{mod}}2\pi-\pi\right]^2. \tag*{(142)} c T ( x , y ) 2 = ℓ = 1 ∑ D [ ( Dihedrals ( x ) ℓ − Dihedrals ( y ) ℓ + π ) mod 2 π − π ] 2 . (142)
Each term is the squared shortest angular difference, so angles separated by 2 π 2\pi 2 π are equivalent.
For TICA-W2 _2 2 , we compare configurations along slowly varying coordinates identified by time-lagged independent component analysis (Pérez-Hernández et al. [46 ] ). Let x t x_t x t denote the mean-centered molecular feature time series, comprising pairwise distances among carbon and nitrogen atoms and the sine and cosine of backbone angles, with the reference feature mean subtracted. For a trajectory of M M M frames and lag ℓ \ell ℓ , the instantaneous and time-lagged covariance estimates with uniform frame weights are
C ^ 00 = 1 M − ℓ ∑ t = 1 M − ℓ x t x t T , C ^ 0 ℓ = 1 M − ℓ ∑ t = 1 M − ℓ x t x t + ℓ T . (143) \widehat{C}_{00}=\frac{1}{M-\ell}\sum_{t=1}^{M-\ell}x_tx_t^{\mathsf{T}},\qquad\widehat{C}_{0\ell}=\frac{1}{M-\ell}\sum_{t=1}^{M-\ell}x_tx_{t+\ell}^{\mathsf{T}}. \tag*{(143)} C 00 = M − ℓ 1 t = 1 ∑ M − ℓ x t x t T , C 0 ℓ = M − ℓ 1 t = 1 ∑ M − ℓ x t x t + ℓ T . (143)
TICA finds directions with high autocorrelation at lag ℓ \ell ℓ by solving the generalized eigenvalue problem
C ^ 0 ℓ w j = λ j C ^ 00 w j (144) \widehat{C}_{0\ell}w_j=\lambda_j\widehat{C}_{00}w_j \tag*{(144)} C 0 ℓ w j = λ j C 00 w j (144)
where λ j \lambda_j λ j measures the autocorrelation of the projected component at lag ℓ \ell ℓ , and w j w_j w j defines the corresponding direction, normalized so that w j T C ^ 00 w j = 1 w_j^{\mathsf{T}}\widehat{C}_{00}w_j=1 w j T C 00 w j = 1 . We retain the first two components and use the squared transport cost
c T I C A ( x , y ) 2 = ∑ j = 1 2 λ j 2 ( w j T x − w j T y ) 2 . (145) c_{\mathrm{TICA}}(x,y)^2=\sum_{j=1}^{2}\lambda_j^2(w_j^{\mathsf{T}}x-w_j^{\mathsf{T}}y)^2. \tag*{(145)} c TICA ( x , y ) 2 = j = 1 ∑ 2 λ j 2 ( w j T x − w j T y ) 2 . (145)
In practice, we fit the Koopman-reweighted covariance estimates using deeptime (Hoffmann et al. [18 ] ). Reference trajectories have frames space at 1 ps. We use a lag of 100 frames (100 ps) for AL2 and 1000 frames (1 ns) for AL4 and AL6. The TICA projections are fitted on the long MD reference trajectories consisting of 16 chains of 2 μ \mu μ s of simulation per chain. We apply that same projection to every method’s samples and the reference configurations.
Mode discovery. For the discovery curves in Figure 3 , we use the TICA projections fitted on each system’s 32 μ \mu μ s MD reference with a 1 ns lag. We divide the TICA plane into four regions using splits at TICA0 = − 3 =-3 = − 3 and TICA1 = − 2 =-2 = − 2 for AL4, and TICA0 = − 2 =-2 = − 2 and TICA1 = − 2 =-2 = − 2 for AL6. A mode counts as discovered once a single chain remains in its region for 20 ps of simulated time. We record the GPU time of the frame that completes this stay. We include burn-in frames from the one-hour runs, but exclude the startup segments of KC-maps.
Kinetic metrics. We measure the mean first-passage time (MFPT) between modes and the autocorrelation of mode occupancy, both from trajectories saved every 1 ps.
Mode labels. We obtain the modes using the reference MD free energy F ( ϕ , ψ ) = − k B T ln p ( ϕ , ψ ) F(\phi,\psi)=-k_{\mathrm{B}}T\ln p(\phi,\psi) F ( ϕ , ψ ) = − k B T ln p ( ϕ , ψ ) , obtained by a watershed on a 120 × 120 120 \times120 120 × 120 grid. Modes separated by saddles below 0.5 k B T 0.5\,k_{\mathrm{B}}T 0.5 k B T or holding under 0.5 % 0.5\% 0.5% of the population are merged. A mode’s core is the region of its basin with F ≤ F min + 2 k B T F \le F_{\min}+2\,k_{\mathrm{B}}T F ≤ F m i n + 2 k B T . A frame inside a mode’s core takes that mode’s label. A frame outside all cores keeps the label of the last core visited, so brief excursions do not count as transitions. Frames before the first core visit are unlabelled.
Mean first-passage time. For distinct modes S S S and D D D , let s k s_k s k be the time of the k k k -th entry into S S S from another mode (the first core visit is not an entry). The first-passage time from this entry is the time until the trajectory first enters the core of D D D ,
T k S → D = inf { u > 0 : X s k + u ∈ D } , τ S → D = E [ T k S → D ] . (146) T_k^{S\to D}=\inf\{u>0:X_{s_k+u}\in D\}, \qquad\tau_{S\to D}=\mathbb{E}\left[T_k^{S\to D}\right]. \tag*{(146)} T k S → D = inf { u > 0 : X s k + u ∈ D } , τ S → D = E [ T k S → D ] . (146)
If the trajectory ends before reaching D D D , we only know that T k S → D T_k^{S\to D} T k S → D exceeds the remaining time C k C_k C k . To keep these censored passages, we model passage times as exponential and use the maximum-likelihood estimate
τ ^ S → D = ∑ k min ( T k S → D , C k ) ∑ k 1 { T k S → D ≤ C k } , (147) \widehat{\tau}_{S\to D}=\frac{\sum_k\min\left(T_k^{S\to D},C_k\right)}{\sum_k\mathbf{1}\left\{T_k^{S\to D}\le C_k\right\}}, \tag*{(147)} τ S → D = ∑ k 1 { T k S → D ≤ C k } ∑ k min ( T k S → D , C k ) , (147)
the total observed waiting time divided by the number of completed passages. Without censoring, this is the sample mean of the passage times.
Autocorrelation. For each method, let π A \pi_A π A be the fraction of its labelled frames in mode A A A . The autocorrelation of mode occupancy is
C A ( t ) = P A A ( t ) − π A 1 − π A , (148) C_A(t)=\frac{P_{AA}(t)-\pi_A}{1-\pi_A}, \tag*{(148)} C A ( t ) = 1 − π A P AA ( t ) − π A , (148)
where P A A ( t ) P_{AA}(t) P AA ( t ) is the fraction of frames labelled A A A that are labelled A A A again a lag t t t later. C A C_A C A decays from 1 to 0 as the dynamics forget which mode they started in.
Transition probabilities. For modes i i i and j j j , P i j ( t ) P_{ij}(t) P ij ( t ) is the fraction of frames labelled i i i that are labelled j j j a lag t t t later, pooled over chains. As t → ∞ t\to\infty t → ∞ , P i j ( t ) P_{ij}(t) P ij ( t ) converges to the population of mode j j j , so it tests both the relaxation time scales and the stationary distribution.
Additional Experiments
Equilibrium Results across step size
In Figure A1 , we show the performance of KC-maps across τ ∈ [ 5 , 70 ] \tau\in[5, 70] τ ∈ [ 5 , 70 ] fs on equilibrium sampling against the baseline molecular dynamics run on a single GPU-hour. We compare both methods to the reference molecular dynamics simulation (16 chains × \times × 2 μ \mu μ s).
Moreover, we demonstrate the ability of our method to extrapolate beyond its training distribution and recover modes not seen during training. Namely, for the AL6 system, Figure A2 shows the reference distribution from long MD, the training data distribution on the train split from Tan et al., [55 ] , which has the missing bottom left mode, and our samples from long simulation from our method (16 chains × \times × 400 ns per chain for a total of 40 GPU-hours).
Throughput across step sizes
We show the throughput measured in simulated time across τ \tau τ for our method against a single chain of MD. We see in smaller systems, the method lags behind in simulated time for a single chain below τ = 50 \tau= 50 τ = 50 fs. However, this gap closes as system sizes grow larger e.g. KC-maps exceeds MD throughput for single chain at τ = 40 \tau= 40 τ = 40 fs for AL6. Moreover, as seen in Figure 3 , KC-maps better utilizes GPU resources when simulating parallel chains in a single GPU, highlighting the advantage of deep learning based approaches for accelerated molecular dynamics.
Figure A3. Aggregate physical time simulated across chains across τ \tau τ for n c h a i n s ∈ 1 , 16 , 32 n_{\mathrm{chains}} \in1, 16, 32 n chains ∈ 1 , 16 , 32 . All runs on a single L40S GPU.
Metropolis–Hastings acceptance across step sizes
We show the Metropolis–Hastings acceptance rates across different τ \tau τ . We observe strong acceptances at small τ \tau τ for all systems. We evaluate each system using 10K sources with average across 16 proposals per source across τ ∈ [ 5 , 100 ] \tau\in[5, 100] τ ∈ [ 5 , 100 ] fs for AL2 and τ ∈ [ 5 , 85 ] \tau\in[5, 85] τ ∈ [ 5 , 85 ] fs AL4 and AL6. For AL2, acceptance remains above 20% for the evaluated step sizes τ < 80 \tau< 80 τ < 80 fs. For AL4 and AL6, the acceptance rates considerably drop and are practically unusable for simulation for larger τ \tau τ . The force-field and simulation settings vary across systems (Table A2 ) and may also affect acceptance rates.
Figure A4. Acceptance rates across step size for KC-maps.
Comparison to MLE on Trajectory Data
We compare our method under same settings, i.e., model parameterization, training steps, and with L D B \mathcal{L}_{DB} L D B loss, but instead train using MLE on trajectory data from MD. In Figure A5 , we see our method is competitive to MLE directly from data, showcasing that we do not need trajectory data to learn accelerated dynamics for τ \tau τ up to 70 fs.
Figure A5. Equilibrium accuracy on AL2 across step sizes: trajectory MLE vs. L K C \mathcal{L}_{KC} L K C . 2 2 2 -Wasserstein distances to reference MD for 16 chains run for one hour on a single L40S GPU at each τ \tau τ . Error bars are the standard deviation over three subsamples of the trajectory. The dashed line is MD with 16 chains for one hour.
AL2 transition probabilities
Figure A6 reports the transition probabilities P i j ( t ) P_{ij}(t) P ij ( t ) of KC-maps and reference long MD between the four AL2 modes (Section D.4). We assign each frame a model label as described in Section D.4, discarding the first 10% of each trajectory for KC-maps and the first 1 ns for each MD trajectory.
We evaluate the lags on a logarithmically spaced grid from 1 ps to 5 ns. At each lag t t t , we pool all pairs of frames separated by t t t within the same trajectory across 16 trajectories. We estimate P i j ( t ) P_{ij}(t) P ij ( t ) as the number of pairs starting in mode i i i and ending in mode j j j , divided by the number starting in mode i i i .
For each method, we estimate uncertainty by bootstrapping trajectories ([14 ] ). We draw 16 trajectories with replacement, pool their transition counts, and recompute P i j ( t ) P_{ij}(t) P ij ( t ) . We repeat this procedure 2000 times. At each lag, the 2.5th and 97.5th percentiles of these estimates define the 95% confidence interval.
Target EMA decay
In Figure A7 , the stop-gradient target (ema_decay = 0) has the highest NLL at every elapsed time, decays between 0.99 and 0.9995 perform similarly, and the slowest decay, 0.99975, degrading in performance.
Flow parameterizations
We compare three coordinate choices for the affine coupling flow, using the same Gaussian reference μ ( x , τ ) + L ( τ ) z \mu(x,\tau) + L(\tau)z μ ( x , τ ) + L ( τ ) z , where z ∼ N ( 0 , I ) z \sim\mathcal{N}(0, I) z ∼ N ( 0 , I ) , μ ( x , τ ) = x + m ( x , τ ) \mu(x,\tau) = x + m(x,\tau) μ ( x , τ ) = x + m ( x , τ ) , and Σ ( τ ) = L ( τ ) L ( τ ) T \Sigma(\tau) = L(\tau)L(\tau)^{\mathsf{T}} Σ ( τ ) = L ( τ ) L ( τ ) T . Writing Ψ \Psi Ψ for the coupling map in the chosen coordinates, the three parameterizations are
Standard: y = Ψ ( x + m ( x , τ ) + L ( τ ) z ; x , θ ) , Displacement: y = x + m ( x , τ ) + Ψ ( L ( τ ) z ; x , θ ) , Whitened: y = x + m ( x , τ ) + L ( τ ) Ψ ( z ; x , θ ) . (149) \begin{aligned}
\text{Standard:}\quad& y = \Psi(x + m(x,\tau) + L(\tau)z; x,\theta),\\
\text{Displacement:}\quad& y = x + m(x,\tau) + \Psi(L(\tau)z; x,\theta),\\
\text{Whitened:}\quad& y = x + m(x,\tau) + L(\tau)\Psi(z; x,\theta). \tag*{(149)}
\end{aligned} Standard: Displacement: Whitened: y = Ψ ( x + m ( x , τ ) + L ( τ ) z ; x , θ ) , y = x + m ( x , τ ) + Ψ ( L ( τ ) z ; x , θ ) , y = x + m ( x , τ ) + L ( τ ) Ψ ( z ; x , θ ) . (149)
The standard flow transforms the full joint state, the displacement flow transforms residuals about the reference mean, and the whitened flow transforms standard Gaussian coordinates. All three recover the Gaussian reference when Ψ \Psi Ψ is the identity.
We adapt the gates to each coordinate choice. Using κ n ( τ ) = ( ω τ ) n 1 + ( ω τ ) n \kappa_n(\tau) = \frac{(\omega\tau)^n}{1 + (\omega\tau)^n} κ n ( τ ) = 1 + ( ω τ ) n ( ω τ ) n , the gates are
A x A p B x B p Standard κ 3 κ 2 κ 3 κ 2 Displacement κ 2 κ 2 κ 3 κ 2 Whitened κ 2 κ 2 κ 3 κ 3 (150) \begin{array}{c|cccc}
& A_x & A_p & B_x & B_p \\
\hline
\text{Standard} & \kappa_3 & \kappa_2 & \kappa_3 & \kappa_2 \\
\text{Displacement} & \kappa_2 & \kappa_2 & \kappa_3 & \kappa_2 \\
\text{Whitened} & \kappa_2 & \kappa_2 & \sqrt{\kappa_3} & \sqrt{\kappa_3} \tag*{(150)}
\end{array} Standard Displacement Whitened A x κ 3 κ 2 κ 2 A p κ 2 κ 2 κ 2 B x κ 3 κ 3 κ 3 B p κ 2 κ 2 κ 3 (150)
where A A A gates the log scale and B B B gates the shift in the corresponding coordinates. For bounded coupling outputs, these choices give physical corrections of at most O ( τ 3 ) O(\tau^3) O ( τ 3 ) in position and O ( τ 2 ) O(\tau^2) O ( τ 2 ) in momentum in all three representations. These gate orders differ from the main model (see Section B.3.3).
In Figure A8 , displacement and whitened flows perform similarly at short elapsed times, while the displacement flow has lower mean NLL and E \mathcal{E} E -W 2 \mathcal{W}_2 W 2 at longer elapsed times. Both improve on the standard flow across the evaluated step sizes.