Introduction
Rare events—nucleation of a new phase, switching between metastable states, extreme fluctuations of currents or densities—often have dramatic consequences for the behavior of stochastic systems. Yet because they occur with probabilities that are exponentially small on the natural time scales of the dynamics, they are inaccessible to direct simulation or standard perturbation theory. Large deviation theory provides the natural mathematical framework for analyzing such events: it characterizes exponentially rare fluctuations through variational principles that transform probabilistic questions into deterministic optimization problems. These notes, adapted from lectures given at the Les Houches school “Large Deviations: Theory and Applications” (July 2024), develop this theory for stochastic dynamical systems with an emphasis on the computational structures that emerge.
The unifying thread of these notes is methodological: in each setting we derive the Large Deviation Principle (LDP) from a multiscale expansion technique, namely the WKB (Wentzel–Kramers–Brillouin) ansatz. We apply this formal asymptotic expansion to the appropriate backward equation to obtain the LDP. This approach is simple and transparent, revealing the Hamiltonian structure that underlies all large deviation problems in a unified way. This Hamiltonian structure reflects the Gärtner–Ellis theorem at the infinitesimal level: the Hamiltonian is a cumulant generating function, and the Lagrangian—obtained via Legendre transform—is the corresponding rate function. The WKB approach is also generic, applying with minimal modification across limits taken in noise, spatial discretization, and time. While these formal derivations are not rigorous, the resulting statements can be established rigorously using the theory of viscosity solutions or probabilistic methods; we refer to the foundational texts [1 , 2 , 3 , 4 , 5 ] for complete proofs.
The notes are organized around two fundamental asymptotic regimes—small noise and long times—and their interplay.
The first two sections develop the LDP at fixed, finite time. § 2 treats stochastic differential equations (SDEs) in the small-noise limit (ε → 0 \varepsilon\to0 ε → 0 ). We present the succession from the Law of Large Numbers to the Central Limit Theorem to the Large Deviation Principle, showing how each provides a progressively richer characterization of the stochastic dynamics: typical behavior, Gaussian fluctuations, and rare large deviations, respectively. The LDP transforms the evaluation of rare-event probabilities into a variational problem with natural Hamiltonian structure. § 3 extends this framework to Markov jump processes (MJPs) in the large-volume limit (N → ∞ N \to\infty N → ∞ ), where the small parameter ε = 1 / N \varepsilon= 1/N ε = 1/ N arises as a continuum limit for spatial discretization. We illustrate the theory through the Schlögl model of chemical bistability and discrete diffusion on a ring.
The next two sections address the long-time limit (T → ∞ T \to\infty T → ∞ ). § 4 develops the quasipotential formalism, which characterizes metastability and transition rates between stable states. The quasipotential emerges naturally in the combined limit of small noise (or large volume) and long time, and provides a principled generalization of potential landscapes to nonequilibrium systems. We illustrate the formalism through both single-compartment and spatially extended versions of the Schlögl model. § 5 treats the long-time limit with O ( 1 ) \mathcal{O}(1) O ( 1 ) noise, where the LDP characterizes fluctuations of time-averaged observables. We show how this regime connects to spectral methods via tilted generators, and how it enables systematic reduction of multiscale systems through averaging over fast degrees of freedom.
Finally, § 6 surveys numerical methods for computing large deviation quantities across all these settings. We discuss both classical approaches based on minimum action methods and modern machine-learning techniques, illustrating the latter through applications to entropy production in active matter systems.
Reading guide: Readers seeking a first pass through the material may focus on § 2 and § 4 , which develop the core theory for SDEs and the quasipotential, returning to Markov jump processes (§ 3 ) and tilted generators (§ 5 ) afterward. § 6 on computational methods can be read independently once § 4 is understood. Appendices § 8 and § 9 provide background on stochastic differential equations and Markov jump processes; readers new to the subject should consult them before reading the main text.
Small-noise limit: Stochastic differential equations
In this section, we present the theory of large deviations for stochastic differential equations (SDEs) in the limit of small noise. We establish the basic definitions, introduce the backward Kolmogorov equation, and show how the Large Deviation Principle emerges naturally from a WKB analysis, leading to a Hamiltonian formulation that extends to general diffusion processes.
Consider a general stochastic differential equation (SDE)
d X t ε = b ( t , X t ε ) d t + 2 ε σ ( t , X t ε ) d W t , (1) \mathrm{d}X_t^\varepsilon= b(t,X_t^\varepsilon)\,\mathrm{d}t + \sqrt{2\varepsilon}\,\sigma(t,X_t^\varepsilon)\,\mathrm{d}W_t, \tag*{(1)} d X t ε = b ( t , X t ε ) d t + 2 ε σ ( t , X t ε ) d W t , (1)
with initial condition X t = 0 ε = x 0 X_{t=0}^\varepsilon= x_0 X t = 0 ε = x 0 . Here X t ε ∈ R d X_t^\varepsilon\in\mathbb{R}^d X t ε ∈ R d denotes the state variables, b : R × R d → R d b:\mathbb{R}\times\mathbb{R}^d \to\mathbb{R}^d b : R × R d → R d is the deterministic drift, σ : R × R d → R d × d \sigma:\mathbb{R}\times\mathbb{R}^d \to\mathbb{R}^{d\times d} σ : R × R d → R d × d is the volatility, and W t W_t W t is a d d d -dimensional Wiener process. The parameter ε > 0 \varepsilon> 0 ε > 0 controls the noise strength, and we are interested in the limit ε ↓ 0 \varepsilon\downarrow0 ε ↓ 0 . The noise covariance matrix is D ( t , x ) = ( σ σ T ) ( t , x ) D(t,x)=(\sigma\sigma^{\mathsf{T}})(t,x) D ( t , x ) = ( σ σ T ) ( t , x ) , which we assume positive-definite (and hence invertible). For a review of basic properties of SDEs we refer the reader to Appendix § 8 .
The small-noise limit: three levels of description
The limit ε → 0 \varepsilon\to0 ε → 0 admits three progressively more information-rich levels of description, corresponding to the Law of Large Numbers, Central Limit Theorem, and Large Deviation Principle.
Law of Large Numbers (LLN)
In the absence of noise (ε = 0 \varepsilon=0 ε = 0 ), the dynamics becomes deterministic. Let X t 0 X_t^0 X t 0 denote the solution of the ordinary differential equation
X ˙ t 0 = b ( t , X t 0 ) , X 0 = x 0 . (2) \dot{X}_t^0=b(t,X_t^0), \qquad X_0=x_0. \tag*{(2)} X ˙ t 0 = b ( t , X t 0 ) , X 0 = x 0 . (2)
The Law of Large Numbers states that
lim ε ↓ 0 E x 0 [ sup 0 ≤ t ≤ T ∣ X t ε − X t ∣ ] = 0 , ∀ T > 0 , (3) \lim_{\varepsilon\downarrow0}\mathbb{E}^{x_0}\left[\sup_{0\leq t\leq T}\left|X_t^\varepsilon-X_t\right|\right]=0,\qquad\forall\,T>0, \tag*{(3)} ε ↓ 0 lim E x 0 [ 0 ≤ t ≤ T sup ∣ X t ε − X t ∣ ] = 0 , ∀ T > 0 , (3)
where E x 0 [ ⋅ ] \mathbb{E}^{x_0}[\cdot] E x 0 [ ⋅ ] denotes the conditional expectation over realizations of the stochastic process starting from X t = 0 ε = x 0 X_{t=0}^\varepsilon=x_0 X t = 0 ε = x 0 . Equivalently, for any δ > 0 \delta>0 δ > 0 ,
lim ε ↓ 0 P x 0 [ sup 0 ≤ t ≤ T ∣ X t ε − X t ∣ > δ ] = 0. (4) \lim_{\varepsilon\downarrow0}\mathbb{P}^{x_0}\left[\sup_{0\leq t\leq T}\left|X_t^\varepsilon-X_t\right|>\delta\right]=0. \tag*{(4)} ε ↓ 0 lim P x 0 [ 0 ≤ t ≤ T sup ∣ X t ε − X t ∣ > δ ] = 0. (4)
Thus, typical trajectories of the SDE converge uniformly to the deterministic trajectory as ε ↓ 0 \varepsilon\downarrow0 ε ↓ 0 .
Central Limit Theorem (CLT)
The CLT characterizes the distribution of fluctuations around the deterministic trajectory on the natural scale ε \sqrt{\varepsilon} ε . Define the rescaled deviation
η t ε = X t ε − X t ε . (5) \eta_t^\varepsilon=\frac{X_t^\varepsilon-X_t}{\sqrt{\varepsilon}}. \tag*{(5)} η t ε = ε X t ε − X t . (5)
Differentiating and expanding b ( t , X t + ε η t ε ) b(t,X_t+\sqrt{\varepsilon}\eta_t^\varepsilon) b ( t , X t + ε η t ε ) to first order in ε \sqrt{\varepsilon} ε , we find that in the limit ε ↓ 0 \varepsilon\downarrow0 ε ↓ 0 , the rescaled fluctuation η t = lim ε ↓ 0 η t ε \eta_t=\lim_{\varepsilon\downarrow0}\eta_t^\varepsilon η t = lim ε ↓ 0 η t ε satisfies
d η t = ∇ b ( t , X t ) η t d t + 2 σ ( t , X t ) d W t . (6) \mathrm{d}\eta_t=\nabla b(t,X_t)\eta_t\,\mathrm{d}t+\sqrt{2}\sigma(t,X_t)\,\mathrm{d}W_t. \tag*{(6)} d η t = ∇ b ( t , X t ) η t d t + 2 σ ( t , X t ) d W t . (6)
Since this equation is linear in η t \eta_t η t and driven by Gaussian noise, the variables η t \eta_t η t are Gaussian. This tells us that X t ε = X t + ε η t + o ( ε ) X_t^\varepsilon=X_t+\sqrt{\varepsilon}\,\eta_t+o(\sqrt{\varepsilon}) X t ε = X t + ε η t + o ( ε ) . Large deviations go beyond this Gaussian description by characterizing fluctuations of order O ( 1 ) \mathcal{O}(1) O ( 1 ) away from the deterministic path.
Large Deviation Principle (LDP)
To probe fluctuations beyond the CLT scale, consider the exponentially weighted expectation
A ε = E x 0 [ e f ( X T ε ) / ε ] , (7) A^\varepsilon=\mathbb{E}^{x_0}\left[e^{f(X_T^\varepsilon)/\varepsilon}\right], \tag*{(7)} A ε = E x 0 [ e f ( X T ε ) / ε ] , (7)
for some function f : R d → R f:\mathbb{R}^d\to\mathbb{R} f : R d → R . The factor 1 / ε 1/\varepsilon 1/ ε in the exponent ensures that A ε A^\varepsilon A ε is dominated by realizations where X t ε X_t^\varepsilon X t ε deviates by O ( 1 ) \mathcal{O}(1) O ( 1 ) from the deterministic trajectory—these are the large deviations we seek to characterize. Our aim is to show that:
lim ϵ ↓ 0 ϵ log A ϵ = sup { x t } t ∈ [ 0 , T ] [ f ( x T ) − ∫ 0 T L ( t , x t , x ˙ t ) d t ] (8) \lim_{\epsilon\downarrow0} \epsilon\log A^\epsilon= \sup_{\{x_t\}_{t\in[0,T]}} \left[f(x_T)-\int_0^T L(t,x_t,\dot{x}_t)\,\mathrm{d}t\right] \tag*{(8)} ϵ ↓ 0 lim ϵ log A ϵ = { x t } t ∈ [ 0 , T ] sup [ f ( x T ) − ∫ 0 T L ( t , x t , x ˙ t ) d t ] (8)
where the supremum is over paths with x 0 x_0 x 0 fixed and the Lagrangian is
L ( t , x , x ˙ ) = 1 4 ( x ˙ − b ( t , x ) ) ⋅ D − 1 ( t , x ) ( x ˙ − b ( t , x ) ) , (9) L(t,x,\dot{x})=\frac{1}{4}(\dot{x}-b(t,x))\cdot D^{-1}(t,x)(\dot{x}-b(t,x)), \tag*{(9)} L ( t , x , x ˙ ) = 4 1 ( x ˙ − b ( t , x )) ⋅ D − 1 ( t , x ) ( x ˙ − b ( t , x )) , (9)
which measures the “cost” of deviating from the deterministic flow. We derive this result systematically in the following subsections; a complementary heuristic derivation based on path integrals is given in Appendix § 8 .8.
Remark 2.1. A direct Monte Carlo estimate of A ϵ A^\epsilon A ϵ is extremely inefficient for small ϵ \epsilon ϵ : the CLT tells us that typical sampled paths remain within O ( ϵ ) O(\sqrt{\epsilon}) O ( ϵ ) of the deterministic trajectory and contribute negligibly to the exponentially weighted average. The LDP (8) provides a principled solution. Once we identify the optimal path x t ∗ x_t^* x t ∗ that achieves the supremum, we can design importance sampling schemes that preferentially generate paths near x t ∗ x_t^* x t ∗ , transforming an exponentially hard sampling problem into a deterministic optimization followed by efficient local sampling. We return to computational methods in § 6 .
Remark 2.2 (Connection to classical large deviation theory). The result (8) is an instance of Varadhan’s lemma [6 ] applied to the Freidlin–Wentzell LDP for trajectories. The rate function for paths is the action S T [ x ] = ∫ 0 T L ( t , x t , x ˙ t ) d t S_T[x]=\int_0^T L(t,x_t,\dot{x}_t)\,\mathrm{d}t S T [ x ] = ∫ 0 T L ( t , x t , x ˙ t ) d t ; Varadhan’s lemma converts this to the variational formula for exponential expectations. The Hamiltonian H ( x , θ ) H(x,\theta) H ( x , θ ) plays the role of a local cumulant generating function, with the Lagrangian obtained via Legendre transform—this is the infinitesimal version of the Gärtner–Ellis theorem [7 , 8 ] , which itself builds on the classical result by Cramér [9 ] . For Brownian motion specifically, the trajectory LDP is Schilder’s theorem [10 ] ; the extension to SDEs with drift is due to Freidlin and Wentzell [1 ] .
Derivation of the LDP: WKB analysis of the backward equation
To establish the LDP (8) , we must compute A ϵ = E x 0 [ e f ( X T ϵ ) / ϵ ] A^\epsilon=\mathbb{E}^{x_0}[e^{f(X_T^\epsilon)/\epsilon}] A ϵ = E x 0 [ e f ( X T ϵ ) / ϵ ] in the limit ϵ ↓ 0 \epsilon\downarrow0 ϵ ↓ 0 . The key tool is the backward Kolmogorov equation (BKE), which governs conditional expectations of the stochastic process.
The generator and backward equation. The infinitesimal generator L L L of a diffusion process (1) gives a deterministic equation for the evolution of expectations. Let ϕ \phi ϕ be an arbitrary compactly supported smooth test function, then its expectation evolves as,
( L ϕ ) ( t , x ) = b ( t , x ) ⋅ ∇ ϕ ( x ) + ϵ D ( t , x ) : ∇ ∇ ϕ ( x ) , (10) (L\phi)(t,x)=b(t,x)\cdot\nabla\phi(x)+\epsilon D(t,x):\nabla\nabla\phi(x), \tag*{(10)} ( L ϕ ) ( t , x ) = b ( t , x ) ⋅ ∇ ϕ ( x ) + ϵD ( t , x ) : ∇∇ ϕ ( x ) , (10)
where D : ∇ ∇ ϕ = ∑ i , j D i j ∂ x i ∂ x j ϕ D:\nabla\nabla\phi=\sum_{i,j}D_{ij}\partial_{x_i}\partial_{x_j}\phi D : ∇∇ ϕ = ∑ i , j D ij ∂ x i ∂ x j ϕ . The derivation from the Euler–Maruyama discretization is given in Appendix § 8 . We can, of course, use the generator to compute conditional expectations. The partial differential equation known as the backward Kolmogorov equation formalizes this: if u ϵ ( t , x ) u^\epsilon(t,x) u ϵ ( t , x ) satisfies
∂ t u ϵ + L u ϵ = 0 , u ϵ ( T , x ) = F ( x ) , (11) \partial_t u^\epsilon+Lu^\epsilon=0,\qquad u^\epsilon(T,x)=F(x), \tag*{(11)} ∂ t u ϵ + L u ϵ = 0 , u ϵ ( T , x ) = F ( x ) , (11)
then u ϵ ( 0 , x ) = E x [ F ( X T ϵ ) ] u^\epsilon(0,x)=\mathbb{E}^{x}[F(X_T^\epsilon)] u ϵ ( 0 , x ) = E x [ F ( X T ϵ )] . The proof follows from Itô’s formula and the martingale property of stochastic integrals; see Appendix § 8 . The BKE evolves backward in time: given terminal data F ( x ) F(x) F ( x ) at time T T T , the solution u ϵ ( t , x ) u^\epsilon(t,x) u ϵ ( t , x ) at earlier times gives the expected value of F ( X T ϵ ) F(X_T^\epsilon) F ( X T ϵ ) conditioned on X t ϵ = x X_t^\epsilon=x X t ϵ = x .
WKB Ansatz The Wentzel-Kramers-Brillouin (WKB) ansatz takes its name from efforts to solve the Schrödinger equation in settings where ℏ \hbar ℏ can be treated as a small parameter, though the approximation was first used by Liouville and Green [11 ] . The formal expansion yields accurate results when there is a separation of scales between the wavelength of solutions and the length scales of the environment. The WKB approach relies on an asymptotic expansion in which the ansatz supposes that the leading order term in the solution can be expressed as u ϵ ( t , x ) = e v ϵ ( t , x ) / ϵ u^\epsilon(t,x)=e^{v^\epsilon(t,x)/\epsilon} u ϵ ( t , x ) = e v ϵ ( t , x ) / ϵ . We directly employ the ansatz in this form below.
Application to the LDP. To compute A ϵ = E x 0 [ e f ( X T ϵ ) / ϵ ] A^\epsilon=\mathbb{E}^{x_0}[e^{f(X_T^\epsilon)/\epsilon}] A ϵ = E x 0 [ e f ( X T ϵ ) / ϵ ] , we solve the BKE with terminal condition u ϵ ( T , x ) = e f ( x ) / ϵ u^\epsilon(T,x)=e^{f(x)/\epsilon} u ϵ ( T , x ) = e f ( x ) / ϵ , so that A ϵ = u ϵ ( 0 , x 0 ) A^\epsilon=u^\epsilon(0,x_0) A ϵ = u ϵ ( 0 , x 0 ) . The exponential form of the terminal condition suggests the WKB ansatz
u ϵ ( t , x ) = e v ϵ ( t , x ) / ϵ . (12) u^\epsilon(t,x)=e^{v^\epsilon(t,x)/\epsilon}. \tag*{(12)} u ϵ ( t , x ) = e v ϵ ( t , x ) / ϵ . (12)
Substituting into the BKE ∂ t u ϵ + b ⋅ ∇ u ϵ + ϵ D : ∇ ∇ u ϵ = 0 \partial_tu^\epsilon+b\cdot\nabla u^\epsilon+\epsilon D:\nabla\nabla u^\epsilon=0 ∂ t u ϵ + b ⋅ ∇ u ϵ + ϵD : ∇∇ u ϵ = 0 and dividing by u ϵ / ϵ u^\epsilon/\epsilon u ϵ / ϵ :
∂ t v ϵ + b ⋅ ∇ v ϵ + ϵ D : ∇ ∇ v ϵ + ∇ v ϵ ⋅ D ∇ v ϵ = 0. (13) \partial_tv^\epsilon+b\cdot\nabla v^\epsilon+\epsilon D:\nabla\nabla v^\epsilon+\nabla v^\epsilon\cdot D\nabla v^\epsilon=0. \tag*{(13)} ∂ t v ϵ + b ⋅ ∇ v ϵ + ϵD : ∇∇ v ϵ + ∇ v ϵ ⋅ D ∇ v ϵ = 0. (13)
In the limit ϵ ↓ 0 \epsilon\downarrow0 ϵ ↓ 0 , the viscous term ϵ 2 D : ∇ ∇ v ϵ \frac{\epsilon}{2}D:\nabla\nabla v^\epsilon 2 ϵ D : ∇∇ v ϵ becomes negligible, yielding the Hamilton–Jacobi equation
∂ t v + H ( t , x , ∇ v ) = 0 , v ( T , x ) = f ( x ) , (14) \partial_tv+H(t,x,\nabla v)=0,\qquad v(T,x)=f(x), \tag*{(14)} ∂ t v + H ( t , x , ∇ v ) = 0 , v ( T , x ) = f ( x ) , (14)
with Hamiltonian
H ( t , x , θ ) = b ( t , x ) ⋅ θ + θ ⋅ D ( t , x ) θ . (15) H(t,x,\theta)=b(t,x)\cdot\theta+\theta\cdot D(t,x)\theta. \tag*{(15)} H ( t , x , θ ) = b ( t , x ) ⋅ θ + θ ⋅ D ( t , x ) θ . (15)
Since v = lim ϵ ↓ 0 v ϵ v=\lim_{\epsilon\downarrow0}v^\epsilon v = lim ϵ ↓ 0 v ϵ and v ϵ ( 0 , x 0 ) = ϵ log A ϵ v^\epsilon(0,x_0)=\epsilon\log A^\epsilon v ϵ ( 0 , x 0 ) = ϵ log A ϵ by construction, we obtain the large deviation principle:
lim ϵ ↓ 0 ϵ log A ϵ = v ( 0 , x 0 ) , (16) \lim_{\epsilon\downarrow0}\epsilon\log A^\epsilon=v(0,x_0), \tag*{(16)} ϵ ↓ 0 lim ϵ log A ϵ = v ( 0 , x 0 ) , (16)
where v ( 0 , x 0 ) v(0,x_0) v ( 0 , x 0 ) is determined by solving the Hamilton–Jacobi equation (14) .
Remark 2.3. The formal WKB expansion is valid when characteristics of the Hamilton–Jacobi equation (14) do not cross. When they do cross—corresponding to caustics in the variational problem—the classical solution ceases to exist and ∇ v \nabla v ∇ v becomes multi-valued. The physically relevant continuation is then the viscosity solution, which can be characterized as the limit lim ϵ → 0 v ϵ \lim_{\epsilon\to0}v^\epsilon lim ϵ → 0 v ϵ of solutions to the regularized equation. For our purposes, we work in regimes where characteristics remain well-separated; the general theory is developed in [1 , 12 , 13 ] .
Method of characteristics and Hamilton's equations
The Hamilton–Jacobi equation (14) is a first-order nonlinear PDE that can be solved by the method of characteristics. Along a characteristic curve x t x_t x t , we set θ t = ∇ v ( t , x t ) \theta_t=\nabla v(t,x_t) θ t = ∇ v ( t , x t ) , and the pair ( x t , θ t ) (x_t,\theta_t) ( x t , θ t ) satisfies Hamilton’s equations:
x ˙ t = ∂ θ H = b ( t , x t ) + 2 D ( t , x t ) θ t , θ ˙ t = − ∂ x H = − [ ∇ b ( t , x t ) ] T θ t − ∇ x ( θ t ⋅ D ( t , x t ) θ t ) , (17) \begin{aligned}
\dot{x}_t&=\partial_\theta H=b(t,x_t)+2D(t,x_t)\theta_t,\\
\dot{\theta}_t&=-\partial_xH=-[\nabla b(t,x_t)]^\mathsf{T}\theta_t-\nabla_x(\theta_t\cdot D(t,x_t)\theta_t), \tag*{(17)}
\end{aligned} x ˙ t θ ˙ t = ∂ θ H = b ( t , x t ) + 2 D ( t , x t ) θ t , = − ∂ x H = − [ ∇ b ( t , x t ) ] T θ t − ∇ x ( θ t ⋅ D ( t , x t ) θ t ) , (17)
where [ ∇ x ( θ ⋅ D θ ) ] k = ∑ i , j θ i ( ∂ x k D i j ) θ j [\nabla_x(\theta\cdot D\theta)]_k=\sum_{i,j}\theta_i(\partial_{x_k}D_{ij})\theta_j [ ∇ x ( θ ⋅ D θ ) ] k = ∑ i , j θ i ( ∂ x k D ij ) θ j . Setting θ = 0 \theta=0 θ = 0 recovers the deterministic dynamics: x ˙ = b ( x ) \dot{x}=b(x) x ˙ = b ( x ) . When D D D is independent of x x x , the equation for θ ˙ t \dot{\theta}_t θ ˙ t simplifies to θ ˙ t = − [ ∇ b ] T θ t \dot{\theta}_t=-[\nabla b]^\mathsf{T}\theta_t θ ˙ t = − [ ∇ b ] T θ t . The boundary conditions are
x t = 0 = x 0 , θ t = T = ∇ f ( x T ) . (18) x_{t=0}=x_0,\qquad\theta_{t=T}=\nabla f(x_T). \tag*{(18)} x t = 0 = x 0 , θ t = T = ∇ f ( x T ) . (18)
The path x t x_t x t solving (17) –(18) is called the instanton (or optimal fluctuation path); it determines the most likely trajectory for the rare event specified by the boundary conditions. This path achieves the supremum in the variational principle (8) .
To extract the LDP, we compute the evolution of v v v along characteristics. Using θ t = ∇ v ( t , x t ) \theta_t=\nabla v(t,x_t) θ t = ∇ v ( t , x t ) and the Hamilton–Jacobi equation:
d d t v ( t , x t ) = ∂ t v + x ˙ t ⋅ ∇ v = − H + ( b + 2 D θ t ) ⋅ θ t = θ t ⋅ D θ t , (19) \frac{\mathrm{d}}{\mathrm{d}t}v(t,x_t)=\partial_t v+\dot{x}_t\cdot\nabla v=-H+(b+2D\theta_t)\cdot\theta_t=\theta_t\cdot D\theta_t, \tag*{(19)} d t d v ( t , x t ) = ∂ t v + x ˙ t ⋅ ∇ v = − H + ( b + 2 D θ t ) ⋅ θ t = θ t ⋅ D θ t , (19)
where all the functions are evaluated at ( t , x t ) (t,x_t) ( t , x t ) . Integrating from 0 0 0 to T T T :
v ( T , x T ) − v ( 0 , x 0 ) = ∫ 0 T θ t ⋅ D ( t , x t ) θ t d t . (20) v(T,x_T)-v(0,x_0)=\int_0^T\theta_t\cdot D(t,x_t)\theta_t\,\mathrm{d}t. \tag*{(20)} v ( T , x T ) − v ( 0 , x 0 ) = ∫ 0 T θ t ⋅ D ( t , x t ) θ t d t . (20)
Since v ( T , x T ) = f ( x T ) v(T,x_T)=f(x_T) v ( T , x T ) = f ( x T ) and lim ϵ ↓ 0 ϵ log A ϵ = v ( 0 , x 0 ) \lim_{\epsilon\downarrow0}\epsilon\log A^\epsilon=v(0,x_0) lim ϵ ↓ 0 ϵ log A ϵ = v ( 0 , x 0 ) , we obtain the LDP:
lim ϵ ↓ 0 ϵ log A ϵ = f ( x T ) − ∫ 0 T θ t ⋅ D ( t , x t ) θ t d t , (21) \lim_{\epsilon\downarrow0}\epsilon\log A^\epsilon=f(x_T)-\int_0^T\theta_t\cdot D(t,x_t)\theta_t\,\mathrm{d}t, \tag*{(21)} ϵ ↓ 0 lim ϵ log A ϵ = f ( x T ) − ∫ 0 T θ t ⋅ D ( t , x t ) θ t d t , (21)
where ( x t , θ t ) (x_t,\theta_t) ( x t , θ t ) is the solution of Hamilton’s equations (17) with boundary conditions (18) .
Physical interpretation and Lagrangian formulation
From x ˙ t = b + 2 D θ t \dot{x}_t=b+2D\theta_t x ˙ t = b + 2 D θ t , we see that θ t = 1 2 D − 1 ( t , x t ) ( x ˙ t − b ( t , x t ) ) \theta_t=\frac{1}{2}D^{-1}(t,x_t)(\dot{x}_t-b(t,x_t)) θ t = 2 1 D − 1 ( t , x t ) ( x ˙ t − b ( t , x t )) represents the “force” required to deviate from the deterministic drift. The optimal θ t \theta_t θ t can be roughly interpreted as the most likely noise realization that produces the most likely path achieving a given rare event. The Hamiltonian formalism, combined with Laplace’s principle, tells us that for small ϵ \epsilon ϵ , knowledge of this maximum-likelihood path captures the leading-order behavior of rare-event probabilities.
The LDP admits an equivalent Lagrangian formulation. The Lagrangian is obtained from the Hamiltonian via the Legendre transform:
L ( t , x , x ˙ ) = sup θ [ x ˙ ⋅ θ − H ( t , x , θ ) ] = 1 4 ( x ˙ − b ( t , x ) ) ⋅ D − 1 ( t , x ) ( x ˙ − b ( t , x ) ) . (22) L(t,x,\dot{x})=\sup_{\theta}\left[\dot{x}\cdot\theta-H(t,x,\theta)\right]=\frac{1}{4}(\dot{x}-b(t,x))\cdot D^{-1}(t,x)(\dot{x}-b(t,x)). \tag*{(22)} L ( t , x , x ˙ ) = θ sup [ x ˙ ⋅ θ − H ( t , x , θ ) ] = 4 1 ( x ˙ − b ( t , x )) ⋅ D − 1 ( t , x ) ( x ˙ − b ( t , x )) . (22)
This confirms (9) . Using the involution property of the Legendre transform we also know that
H ( t , x , θ ) = sup x ˙ [ x ˙ ⋅ θ − L ( t , x , x ˙ ) ] = θ ⋅ D ( t , x ) θ (23) H(t,x,\theta)=\sup_{\dot{x}}\left[\dot{x}\cdot\theta-L(t,x,\dot{x})\right]=\theta\cdot D(t,x)\theta \tag*{(23)} H ( t , x , θ ) = x ˙ sup [ x ˙ ⋅ θ − L ( t , x , x ˙ ) ] = θ ⋅ D ( t , x ) θ (23)
Therefore the variational principle (8) can then be written as
lim ϵ ↓ 0 ϵ log A ϵ = sup { x t } t ∈ [ 0 , T ] { f ( x T ) − ∫ 0 T L ( t , x t , x ˙ t ) d t } . (24) \lim_{\epsilon\downarrow0}\epsilon\log A^\epsilon=\sup_{\{x_t\}_{t\in[0,T]}}\left\{f(x_T)-\int_0^T L(t,x_t,\dot{x}_t)\,\mathrm{d}t\right\}. \tag*{(24)} ϵ ↓ 0 lim ϵ log A ϵ = { x t } t ∈ [ 0 , T ] sup { f ( x T ) − ∫ 0 T L ( t , x t , x ˙ t ) d t } . (24)
and the Euler–Lagrange equations reproduce Hamilton’s equations (17) (see Appendix § 10 ).
More generally, the probability that X T ε X_T^\varepsilon X T ε lies in a set B ⊂ R d B \subset\mathbb{R}^d B ⊂ R d satisfies
lim ε ↓ 0 ε log P x 0 [ X T ε ∈ B ] = − inf x T ∈ B ∫ 0 T L ( t , x t , x ˙ t ) d t , (25) \lim_{\varepsilon\downarrow0}\varepsilon\log\mathbb{P}^{x_0}\left[X_T^\varepsilon\in B\right]=-\inf_{x_T\in B}\int_0^T L(t,x_t,\dot{x}_t)\,\mathrm{d}t, \tag*{(25)} ε ↓ 0 lim ε log P x 0 [ X T ε ∈ B ] = − x T ∈ B inf ∫ 0 T L ( t , x t , x ˙ t ) d t , (25)
where the infimum is over optimal paths ending in B B B .
Example: Ornstein–Uhlenbeck process
Consider the Ornstein–Uhlenbeck process with σ = 1 \sigma=1 σ = 1 :
d X t ε = − X t ε d t + 2 ε d W t , X t = 0 = 0 , (27) \mathrm{d}X_t^\varepsilon=-X_t^\varepsilon\,\mathrm{d}t+\sqrt{2\varepsilon}\,\mathrm{d}W_t,\qquad X_{t=0}=0, \tag*{(27)} d X t ε = − X t ε d t + 2 ε d W t , X t = 0 = 0 , (27)
and the observable A ε = E 0 [ exp ( λ X T ε / ε ) ] A^\varepsilon=\mathbb{E}^0[\exp(\lambda X_T^\varepsilon/\varepsilon)] A ε = E 0 [ exp ( λ X T ε / ε )] for λ ∈ R \lambda\in\mathbb{R} λ ∈ R .
Exact solution. This process can be solved exactly. Setting Y t = e t X t ε Y_t=e^{t}X_t^\varepsilon Y t = e t X t ε and applying Itô’s formula, we find d Y t = ε e t d W t \mathrm{d}Y_t=\sqrt{\varepsilon}e^t\,\mathrm{d}W_t d Y t = ε e t d W t , which integrates to
X t ε = 2 ε e − t ∫ 0 t e s d W s . (28) X_t^\varepsilon=\sqrt{2\varepsilon}e^{-t}\int_0^t e^s\,\mathrm{d}W_s. \tag*{(28)} X t ε = 2 ε e − t ∫ 0 t e s d W s . (28)
Thus X t ε ∼ N ( 0 , ε ( 1 − e − 2 t ) ) X_t^\varepsilon\sim\mathcal{N}(0,\varepsilon(1-e^{-2t})) X t ε ∼ N ( 0 , ε ( 1 − e − 2 t )) , and a direct Gaussian integration yields
lim ε ↓ 0 ε log A ε = λ 2 2 ( 1 − e − 2 T ) . (29) \lim_{\varepsilon\downarrow0}\varepsilon\log A^\varepsilon=\frac{\lambda^2}{2}(1-e^{-2T}). \tag*{(29)} ε ↓ 0 lim ε log A ε = 2 λ 2 ( 1 − e − 2 T ) . (29)
Hamiltonian approach. With b ( x ) = − x b(x)=-x b ( x ) = − x , D = 1 D=1 D = 1 , and f ( x ) = λ x f(x)=\lambda x f ( x ) = λ x , Hamilton’s equations (17) simplify to
x ˙ t = − x t + 2 θ t , θ ˙ t = θ t , (30) \dot{x}_t=-x_t+2\theta_t,\qquad\dot{\theta}_t=\theta_t, \tag*{(30)} x ˙ t = − x t + 2 θ t , θ ˙ t = θ t , (30)
with boundary conditions x 0 = 0 x_0=0 x 0 = 0 and θ T = λ \theta_T=\lambda θ T = λ . The second equation gives θ t = λ e t − T \theta_t=\lambda e^{t-T} θ t = λ e t − T . Substituting into the first and solving with $x_0=0:
x t = λ e − T ( e t − e − t ) . (31) x_t=\lambda e^{-T}(e^t-e^{-t}). \tag*{(31)} x t = λ e − T ( e t − e − t ) . (31)
Evaluating the LDP (21) with D = 1 D=1 D = 1 :
lim ε ↓ 0 ε log A ε = λ x T − ∫ 0 T ∣ θ t ∣ 2 d t = λ 2 ( 1 − e − 2 T ) − λ 2 2 ( 1 − e − 2 T ) = λ 2 2 ( 1 − e − 2 T ) , (32) \begin{aligned}
\lim_{\varepsilon\downarrow0}\varepsilon\log A^\varepsilon
&=\lambda x_T-\int_0^T|\theta_t|^2\,\mathrm{d}t\\
&=\lambda^2(1-e^{-2T})-\frac{\lambda^2}{2}(1-e^{-2T})=\frac{\lambda^2}{2}(1-e^{-2T}), \tag*{(32)}
\end{aligned} ε ↓ 0 lim ε log A ε = λ x T − ∫ 0 T ∣ θ t ∣ 2 d t = λ 2 ( 1 − e − 2 T ) − 2 λ 2 ( 1 − e − 2 T ) = 2 λ 2 ( 1 − e − 2 T ) , (32)
in agreement with (29) . Note that the optimal path x t x_t x t differs qualitatively from typical trajectories, which fluctuate around zero on scale ε \sqrt{\varepsilon} ε .
Large-volume limit: Markov jump processes
We now extend the large deviation framework to Markov jump processes (MJPs), which describe systems evolving through discrete transitions between states. Let X t ∈ Ω ⊂ R d X_t \in\Omega\subset\mathbb{R}^d X t ∈ Ω ⊂ R d (e.g., Ω = Z d \Omega= \mathbb{Z}^d Ω = Z d ). The dynamics is specified by R R R reaction channels with rates λ j ( X ) ≥ 0 \lambda_j(X) \ge0 λ j ( X ) ≥ 0 and displacement vectors ν j ∈ R d \nu_j \in\mathbb{R}^d ν j ∈ R d for j = 1 , … , R j = 1,\ldots,R j = 1 , … , R . In an infinitesimal time Δ t \Delta t Δ t :
X t + Δ t ∣ X t = { X t + ν j , with probability λ j ( X t ) Δ t + o ( Δ t ) , X t , with probability 1 − ∑ j = 1 R λ j ( X t ) Δ t + o ( Δ t ) . (33) X_{t+\Delta t}\mid X_t =
\begin{cases}
X_t+\nu_j, & \text{with probability } \lambda_j(X_t)\Delta t+o(\Delta t),\\
X_t, & \text{with probability } 1-\sum_{j=1}^{R}\lambda_j(X_t)\Delta t+o(\Delta t).
\end{cases} \tag*{(33)} X t + Δ t ∣ X t = { X t + ν j , X t , with probability λ j ( X t ) Δ t + o ( Δ t ) , with probability 1 − ∑ j = 1 R λ j ( X t ) Δ t + o ( Δ t ) . (33)
The generator of this process, analogous to (10) , is
( L ϕ ) ( x ) = lim Δ t ↓ 0 1 Δ t E X t = x [ ϕ ( X t + Δ t ) − ϕ ( X t ) ] = ∑ j = 1 R λ j ( x ) [ ϕ ( x + ν j ) − ϕ ( x ) ] . (34) (\mathcal{L}\phi)(x)=\lim_{\Delta t\downarrow0}\frac{1}{\Delta t}\mathbb{E}_{X_t=x}\left[\phi(X_{t+\Delta t})-\phi(X_t)\right]=\sum_{j=1}^{R}\lambda_j(x)\left[\phi(x+\nu_j)-\phi(x)\right]. \tag*{(34)} ( L ϕ ) ( x ) = Δ t ↓ 0 lim Δ t 1 E X t = x [ ϕ ( X t + Δ t ) − ϕ ( X t ) ] = j = 1 ∑ R λ j ( x ) [ ϕ ( x + ν j ) − ϕ ( x ) ] . (34)
See Appendix § 9 for a derivation and further discussion of MJP fundamentals.
Interpretation and numerical simulation. The definition above has a transparent interpretation in terms of the Gillespie algorithm [16 ] , which simulates MJPs exactly. Given the current state X t X_t X t , the process waits for an exponentially distributed time τ ∼ Exp ( Λ ( X t ) ) \tau\sim\operatorname{Exp}(\Lambda(X_t)) τ ∼ Exp ( Λ ( X t )) , where Λ ( X t ) = ∑ j λ j ( X t ) \Lambda(X_t)=\sum_j\lambda_j(X_t) Λ ( X t ) = ∑ j λ j ( X t ) is the total transition rate. When the jump occurs, reaction j j j is selected with probability λ j ( X t ) / Λ ( X t ) \lambda_j(X_t)/\Lambda(X_t) λ j ( X t ) /Λ ( X t ) , and the state updates to X t + τ = X t + ν j X_{t+\tau}=X_t+\nu_j X t + τ = X t + ν j . This cycle then repeats from the new state. The memoryless property of the exponential distribution ensures that the process is Markov: the future evolution depends only on the current state, not on how long the system has been in that state.
Examples
The Schlögl model. This paradigmatic model of chemical bistability [17 ] involves species A A A , B B B , and S S S with reactions
S ⇌ k 0 k 1 B , (35) S \underset{k_1}{\overset{k_0}{\rightleftharpoons}} B, \tag*{(35)} S k 1 ⇌ k 0 B , (35)
A + 2 S ⇌ k 3 k 2 3 S . (36) A+2S \underset{k_2}{\overset{k_3}{\rightleftharpoons}} 3S. \tag*{(36)} A + 2 S k 2 ⇌ k 3 3 S . (36)
In a well-mixed volume V V V with fixed concentrations c A c_A c A and c B c_B c B , the state variable is N ∈ N N \in\mathbb{N} N ∈ N , the number of S S S molecules. The transitions N → N ± 1 N \to N \pm1 N → N ± 1 occur with rates
λ + ( N ) = k 1 V c B + k 3 c A N ( N − 1 ) / V , (37) \lambda_+(N)=k_1Vc_B+k_3c_A N(N-1)/V, \tag*{(37)} λ + ( N ) = k 1 V c B + k 3 c A N ( N − 1 ) / V , (37)
λ − ( N ) = k 0 N + k 2 N ( N − 1 ) ( N − 2 ) / V 2 . (38) \lambda_-(N)=k_0N+k_2N(N-1)(N-2)/V^2. \tag*{(38)} λ − ( N ) = k 0 N + k 2 N ( N − 1 ) ( N − 2 ) / V 2 . (38)
These rates are extensive (proportional to system size). Introducing the intensive variable x = N / V x=N/V x = N / V and the small parameter ϵ = 1 / V \epsilon=1/V ϵ = 1/ V , we obtain
λ + ϵ ( x ) = 1 ϵ [ k 1 c B + k 3 c A x ( x − ϵ ) ] , (39) \lambda_+^\epsilon(x)=\frac{1}{\epsilon}\left[k_1c_B+k_3c_Ax(x-\epsilon)\right], \tag*{(39)} λ + ϵ ( x ) = ϵ 1 [ k 1 c B + k 3 c A x ( x − ϵ ) ] , (39)
λ − ϵ ( x ) = 1 ϵ [ k 0 x + k 2 x ( x − ϵ ) ( x − 2 ϵ ) ] . (40) \lambda_-^\epsilon(x)=\frac{1}{\epsilon}\left[k_0x+k_2x(x-\epsilon)(x-2\epsilon)\right]. \tag*{(40)} λ − ϵ ( x ) = ϵ 1 [ k 0 x + k 2 x ( x − ϵ ) ( x − 2 ϵ ) ] . (40)
The limit ϵ → 0 \epsilon\to0 ϵ → 0 corresponds to the thermodynamic limit at fixed density.
Discrete diffusion on a ring. Consider M M M compartments with n i n_i n i particles in compartment i i i , where ∑ i n i = N T \sum_i n_i = N_T ∑ i n i = N T is conserved. Particles hop between adjacent compartments at rate α n i \alpha n_i α n i :
n i → n i − 1 , n i ± 1 → n i ± 1 + 1 , rate α n i . (41) n_i \to n_i - 1,\qquad n_{i\pm1} \to n_{i\pm1} + 1,\qquad\text{rate }\alpha n_i. \tag*{(41)} n i → n i − 1 , n i ± 1 → n i ± 1 + 1 , rate α n i . (41)
The displacement vectors are ν i R = − e i + e i + 1 \nu_i^R=-e_i+e_{i+1} ν i R = − e i + e i + 1 and ν i L = − e i + e i − 1 \nu_i^L=-e_i+e_{i-1} ν i L = − e i + e i − 1 , where e i e_i e i is the i i i th unit vector. With periodic boundary conditions and intensive variables x i = n i / N T x_i=n_i/N_T x i = n i / N T , the small parameter is ε = 1 / N T \varepsilon=1/N_T ε = 1/ N T .
Rescaled generator and backward Kolmogorov equation
The rates in both examples are extensive: proportional to system size 1 / ε 1/\varepsilon 1/ ε . This is generic for well-mixed reaction systems, where the number of reactive events per unit time scales with the volume. Passing to intensive variables, the rates take the form ε − 1 [ λ j ( x ) + O ( ε ) ] \varepsilon^{-1}[\lambda_j(x)+O(\varepsilon)] ε − 1 [ λ j ( x ) + O ( ε )] , where the leading-order rates λ j ( x ) \lambda_j(x) λ j ( x ) depend only on concentrations. The rescaled generator is
( L ε ϕ ) ( x ) = ∑ j = 1 R 1 ε [ λ j ( x ) + O ( ε ) ] [ ϕ ( x + ε ν j ) − ϕ ( x ) ] . (42) (\mathcal{L}^{\varepsilon}\phi)(x)=\sum_{j=1}^{R}\frac{1}{\varepsilon}[\lambda_j(x)+O(\varepsilon)][\phi(x+\varepsilon\nu_j)-\phi(x)]. \tag*{(42)} ( L ε ϕ ) ( x ) = j = 1 ∑ R ε 1 [ λ j ( x ) + O ( ε )] [ ϕ ( x + ε ν j ) − ϕ ( x )] . (42)
For the Schlögl model, expanding (39) –(40) :
λ + ( x ) = k 1 c B + k 3 c A x 2 , (43) \lambda_+(x)=k_1c_B+k_3c_Ax^2, \tag*{(43)} λ + ( x ) = k 1 c B + k 3 c A x 2 , (43)
λ − ( x ) = k 0 x + k 2 x 3 . (44) \lambda_-(x)=k_0x+k_2x^3. \tag*{(44)} λ − ( x ) = k 0 x + k 2 x 3 . (44)
For discrete diffusion: λ i , L ( x ) = λ i , R ( x ) = α x i \lambda_{i,L}(x)=\lambda_{i,R}(x)=\alpha x_i λ i , L ( x ) = λ i , R ( x ) = α x i .
As in the diffusive case (§ 2.2 ), the generator determines the evolution of conditional expectations via the backward Kolmogorov equation: if u ε ( t , x ) u^\varepsilon(t,x) u ε ( t , x ) solves
∂ t u ε + L ε u ε = 0 , u ε ( T , x ) = F ( x ) , (45) \partial_tu^\varepsilon+\mathcal{L}^\varepsilon u^\varepsilon=0,\qquad u^\varepsilon(T,x)=F(x), \tag*{(45)} ∂ t u ε + L ε u ε = 0 , u ε ( T , x ) = F ( x ) , (45)
then u ε ( 0 , x ) = E x [ F ( X T ε ) ] u^\varepsilon(0,x)=\mathbb{E}^x[F(X_T^\varepsilon)] u ε ( 0 , x ) = E x [ F ( X T ε )] . The proof is analogous to the SDE case; see Appendix § 9 . We now use the BKE to derive the Law of Large Numbers and the Large Deviation Principle.
Law of Large Numbers and the law of mass action
Setting u ( t , x ) = lim ε ↓ 0 u ε ( t , x ) u(t,x)=\lim_{\varepsilon\downarrow0}u^\varepsilon(t,x) u ( t , x ) = lim ε ↓ 0 u ε ( t , x ) in the BKE (45) and expanding ϕ ( x + ε ν j ) = ϕ ( x ) + ε ν j ⋅ ∇ ϕ + O ( ε 2 ) \phi(x+\varepsilon\nu_j)=\phi(x)+\varepsilon\nu_j\cdot\nabla\phi+O(\varepsilon^2) ϕ ( x + ε ν j ) = ϕ ( x ) + ε ν j ⋅ ∇ ϕ + O ( ε 2 ) , we obtain
∂ t u + ∑ j = 1 R λ j ( x ) ν j ⋅ ∇ u = 0. (46) \partial_tu+\sum_{j=1}^{R}\lambda_j(x)\nu_j\cdot\nabla u=0. \tag*{(46)} ∂ t u + j = 1 ∑ R λ j ( x ) ν j ⋅ ∇ u = 0. (46)
The characteristics of this first-order PDE satisfy the law of mass action:
x ˙ t = ∑ j = 1 R λ j ( x t ) ν j . (47) \dot{x}_t=\sum_{j=1}^{R}\lambda_j(x_t)\nu_j. \tag*{(47)} x ˙ t = j = 1 ∑ R λ j ( x t ) ν j . (47)
For the Schlögl model: x ˙ = k 1 c B + k 3 c A x 2 − k 0 x − k 2 x 3 \dot{x}=k_1c_B+k_3c_Ax^2-k_0x-k_2x^3 x ˙ = k 1 c B + k 3 c A x 2 − k 0 x − k 2 x 3 . For discrete diffusion: x ˙ i = α ( x i + 1 + x i − 1 − 2 x i ) \dot{x}_i=\alpha(x_{i+1}+x_{i-1}-2x_i) x ˙ i = α ( x i + 1 + x i − 1 − 2 x i ) , which is the discrete diffusion equation.
The convergence of MJPs to deterministic dynamics in the large-volume limit is established rigorously in [18 , 19 ] ; see also [20 ] for interacting particle systems.
Derivation of the LDP: WKB analysis for jump processes
To derive the LDP, we apply the WKB ansatz to the BKE [45 ] . Consider u ε ( t , x ) = E x [ e f ( X T − t ε ) / ε ] u^\varepsilon(t,x)=\mathbb{E}^x[e^{f(X^\varepsilon_{T-t})/\varepsilon}] u ε ( t , x ) = E x [ e f ( X T − t ε ) / ε ] with terminal condition u ε ( 0 , x ) = e f ( x ) / ε u^\varepsilon(0,x)=e^{f(x)/\varepsilon} u ε ( 0 , x ) = e f ( x ) / ε . Setting u ε = e v ε / ε u^\varepsilon=e^{v^\varepsilon/\varepsilon} u ε = e v ε / ε and substituting into the BKE, we must evaluate L ε e v ε / ε L^\varepsilon e^{v^\varepsilon/\varepsilon} L ε e v ε / ε . Using the rescaled generator and expanding v ε ( x + ε ν j ) v^\varepsilon(x+\varepsilon\nu_j) v ε ( x + ε ν j ) around x x x :
v ε ( x + ε ν j ) = v ε ( x ) + ε ν j ⋅ ∇ v ε ( x ) + O ( ε 2 ) , (48) v^\varepsilon(x+\varepsilon\nu_j)=v^\varepsilon(x)+\varepsilon\,\nu_j\cdot\nabla v^\varepsilon(x)+O(\varepsilon^2), \tag*{(48)} v ε ( x + ε ν j ) = v ε ( x ) + ε ν j ⋅ ∇ v ε ( x ) + O ( ε 2 ) , (48)
we obtain
e v ε ( x + ε ν j ) / ε = e v ε ( x ) / ε e ν j ⋅ ∇ v ε ( x ) + O ( ε ) . (49) e^{v^\varepsilon(x+\varepsilon\nu_j)/\varepsilon}=e^{v^\varepsilon(x)/\varepsilon}\,e^{\nu_j\cdot\nabla v^\varepsilon(x)+O(\varepsilon)}. \tag*{(49)} e v ε ( x + ε ν j ) / ε = e v ε ( x ) / ε e ν j ⋅ ∇ v ε ( x ) + O ( ε ) . (49)
Therefore,
L ε e v ε / ε = ∑ j = 1 R 1 ε [ λ j ( x ) + O ( ε ) ] [ e v ε ( x + ε ν j ) / ε − e v ε ( x ) / ε ] = 1 ε e v ε / ε ∑ j = 1 R [ λ j ( x ) + O ( ε ) ] [ e ν j ⋅ ∇ v ε − 1 ] . (50) L^\varepsilon e^{v^\varepsilon/\varepsilon}=\sum_{j=1}^{R}\frac{1}{\varepsilon}[\lambda_j(x)+O(\varepsilon)]\left[e^{v^\varepsilon(x+\varepsilon\nu_j)/\varepsilon}-e^{v^\varepsilon(x)/\varepsilon}\right]=\frac{1}{\varepsilon}e^{v^\varepsilon/\varepsilon}\sum_{j=1}^{R}[\lambda_j(x)+O(\varepsilon)]\left[e^{\nu_j\cdot\nabla v^\varepsilon}-1\right]. \tag*{(50)} L ε e v ε / ε = j = 1 ∑ R ε 1 [ λ j ( x ) + O ( ε )] [ e v ε ( x + ε ν j ) / ε − e v ε ( x ) / ε ] = ε 1 e v ε / ε j = 1 ∑ R [ λ j ( x ) + O ( ε )] [ e ν j ⋅ ∇ v ε − 1 ] . (50)
Substituting into the BKE ∂ t u ε + L ε u ε = 0 \partial_tu^\varepsilon+L^\varepsilon u^\varepsilon=0 ∂ t u ε + L ε u ε = 0 , dividing by e v ε / ε / ε e^{v^\varepsilon/\varepsilon}/\varepsilon e v ε / ε / ε , and taking ε → 0 \varepsilon\to0 ε → 0 , we obtain the Hamilton–Jacobi equation:
∂ t v + H ( x , ∇ v ) = 0 , v ( T , x ) = f ( x ) . (51) \partial_t v+H(x,\nabla v)=0,\qquad v(T,x)=f(x). \tag*{(51)} ∂ t v + H ( x , ∇ v ) = 0 , v ( T , x ) = f ( x ) . (51)
with Hamiltonian
H ( x , θ ) = ∑ j = 1 R λ j ( x ) [ e ν j ⋅ θ − 1 ] . (52) H(x,\theta)=\sum_{j=1}^{R}\lambda_j(x)\left[e^{\nu_j\cdot\theta}-1\right]. \tag*{(52)} H ( x , θ ) = j = 1 ∑ R λ j ( x ) [ e ν j ⋅ θ − 1 ] . (52)
The associated Hamilton’s equations are
x ˙ t = ∂ θ H = ∑ j = 1 R λ j ( x ) e ν j ⋅ θ ν j , θ ˙ t = − ∂ x H = − ∑ j = 1 R ∇ λ j ( x ) [ e ν j ⋅ θ − 1 ] . (53) \begin{aligned}
\dot{x}_t&=\partial_\theta H=\sum_{j=1}^{R}\lambda_j(x)e^{\nu_j\cdot\theta}\nu_j,\\
\dot{\theta}_t&=-\partial_xH=-\sum_{j=1}^{R}\nabla\lambda_j(x)\left[e^{\nu_j\cdot\theta}-1\right]. \tag*{(53)}
\end{aligned} x ˙ t θ ˙ t = ∂ θ H = j = 1 ∑ R λ j ( x ) e ν j ⋅ θ ν j , = − ∂ x H = − j = 1 ∑ R ∇ λ j ( x ) [ e ν j ⋅ θ − 1 ] . (53)
Setting θ = 0 \theta=0 θ = 0 in the first equation recovers the law of mass action [47 ] .
Note that the Hamiltonian (51) differs fundamentally from the diffusive case [15 ] . Expanding for small θ \theta θ : H ( x , θ ) ≈ ∑ j λ j ( x ) [ ν j ⋅ θ + 1 2 ( ν j ⋅ θ ) 2 ] H(x,\theta)\approx\sum_j\lambda_j(x)[\nu_j\cdot\theta+\frac{1}{2}(\nu_j\cdot\theta)^2] H ( x , θ ) ≈ ∑ j λ j ( x ) [ ν j ⋅ θ + 2 1 ( ν j ⋅ θ ) 2 ] , which resembles [15 ] but with state-dependent diffusion. The non-quadratic nature of (51) means that large deviations are not captured by a Gaussian approximation.
Remark 3.1. As in the diffusive case (Remark 2.2), the Hamiltonian (52) plays the role of a cumulant generating function: for a single jump with displacement ν \nu ν occurring at rate λ \lambda λ , the term λ ( e ν ⋅ θ − 1 ) \lambda(e^{\nu\cdot\theta}-1) λ ( e ν ⋅ θ − 1 ) is precisely the CGF of a Poisson process. The Lagrangian obtained via Legendre transform gives the rate function, following the Gärtner–Ellis structure.
Schlögl model. With ν ± = ± 1 \nu_\pm=\pm1 ν ± = ± 1 :
H ( x , θ ) = ( k 1 c B + k 3 c A x 2 ) ( e θ − 1 ) + ( k 0 x + k 2 x 3 ) ( e − θ − 1 ) . (54) H(x,\theta)=(k_1c_B+k_3c_Ax^2)(e^\theta-1)+(k_0x+k_2x^3)(e^{-\theta}-1). \tag*{(54)} H ( x , θ ) = ( k 1 c B + k 3 c A x 2 ) ( e θ − 1 ) + ( k 0 x + k 2 x 3 ) ( e − θ − 1 ) . (54)
For appropriate parameter values, the deterministic dynamics x ˙ = ∂ θ H ∣ θ = 0 \dot{x}=\left.\partial_\theta H\right|_{\theta=0} x ˙ = ∂ θ H ∣ θ = 0 has two stable fixed points x ± ∗ x_\pm^* x ± ∗ and one unstable fixed point x 0 ∗ x_0^* x 0 ∗ (Figure 1 ).
Figure 1. Deterministic dynamics of the Schlögl model. Top: drift b ( x ) = k 1 c B + k 3 c A x 2 − k 0 x − k 2 x 3 b(x) = k_1c_B + k_3c_A x^2 - k_0x - k_2x^3 b ( x ) = k 1 c B + k 3 c A x 2 − k 0 x − k 2 x 3 . Bottom: potential U ( x ) = − ∫ b ( x ) d x U(x) = -\int b(x)\,\mathrm{d}x U ( x ) = − ∫ b ( x ) d x . Parameters: k 1 c B = 0.8 k_1c_B = 0.8 k 1 c B = 0.8 , k 0 = 2.9 k_0 = 2.9 k 0 = 2.9 , k 3 c A = 3.1 k_3c_A = 3.1 k 3 c A = 3.1 , k 2 = 1 k_2 = 1 k 2 = 1 . Black dots: stable fixed points; orange square: unstable fixed point. Arrows indicate the direction of relaxation.
Discrete diffusion. The Hamiltonian is
H ( x , θ ) = α ∑ i = 1 M x i [ e θ i + 1 − θ i + e θ i − 1 − θ i − 2 ] . (55) H(x,\theta) = \alpha\sum_{i=1}^{M} x_i \left[e^{\theta_{i+1}-\theta_i} + e^{\theta_{i-1}-\theta_i} - 2\right]. \tag*{(55)} H ( x , θ ) = α i = 1 ∑ M x i [ e θ i + 1 − θ i + e θ i − 1 − θ i − 2 ] . (55)
Setting θ = 0 \theta= 0 θ = 0 recovers the discrete diffusion equation.
Continuum limit of discrete diffusion. Dividing [ 0 , 1 ] [0,1] [ 0 , 1 ] into M M M boxes of width Δ s = 1 / M \Delta s = 1/M Δ s = 1/ M , we identify s = i Δ s s = i\Delta s s = i Δ s and interpret x i ≈ x ( s ) Δ s x_i \approx x(s)\Delta s x i ≈ x ( s ) Δ s as the fraction in compartment i i i , where x ( s ) x(s) x ( s ) is a density satisfying ∫ 0 1 x ( s ) d s = 1 \int_0^1 x(s)\,\mathrm{d}s = 1 ∫ 0 1 x ( s ) d s = 1 . Similarly θ i → θ ( s ) \theta_i \to\theta(s) θ i → θ ( s ) . The physical diffusion coefficient is D = α ( Δ s ) 2 D = \alpha(\Delta s)^2 D = α ( Δ s ) 2 , which we hold fixed as Δ s → 0 \Delta s \to0 Δ s → 0 . Expanding:
e θ i ± 1 − θ i − 1 = ± Δ s ∂ s θ + ( Δ s ) 2 2 [ ∂ s 2 θ + ∣ ∂ s θ ∣ 2 ] + O ( ( Δ s ) 3 ) . (56) e^{\theta_{i\pm1}-\theta_i} - 1 = \pm\Delta s\,\partial_s\theta+ \frac{(\Delta s)^2}{2}\left[\partial_s^2\theta+ |\partial_s\theta|^2\right] + O((\Delta s)^3). \tag*{(56)} e θ i ± 1 − θ i − 1 = ± Δ s ∂ s θ + 2 ( Δ s ) 2 [ ∂ s 2 θ + ∣ ∂ s θ ∣ 2 ] + O (( Δ s ) 3 ) . (56)
Substituting into (55) and using ∑ i x i [ … ] = ∑ i x ( s ) Δ s [ … ] → ∫ x ( s ) [ … ] d s \sum_i x_i[\ldots] = \sum_i x(s)\Delta s[\ldots] \to\int x(s)[\ldots]\,\mathrm{d}s ∑ i x i [ … ] = ∑ i x ( s ) Δ s [ … ] → ∫ x ( s ) [ … ] d s :
H [ x , θ ] = D ∫ 0 1 d s x ( s ) [ ∂ s 2 θ + ∣ ∂ s θ ∣ 2 ] . (57) H[x,\theta] = D\int_0^1 \mathrm{d}s\,x(s)\left[\partial_s^2\theta+ |\partial_s\theta|^2\right]. \tag*{(57)} H [ x , θ ] = D ∫ 0 1 d s x ( s ) [ ∂ s 2 θ + ∣ ∂ s θ ∣ 2 ] . (57)
The resulting Hamilton equations are PDEs for the fields x ( s , t ) x(s,t) x ( s , t ) and θ ( s , t ) \theta(s,t) θ ( s , t ) .
Long-time asymptotics I: The quasipotential
The quasipotential is a fundamental quantity for analyzing the long-time behavior of stochastic systems in the small-noise limit. It generalizes the notion of a potential landscape to nonequilibrium settings where detailed balance may not hold.
Motivation: stationary distributions and effective dynamics in the small noise limit
We consider autonomous systems—SDEs or MJPs with time-independent generators—in the small-noise or large-volume limit ε → 0 \varepsilon\to0 ε → 0 . Our goal is to characterize:
The stationary distribution as ε → 0 \varepsilon\to0 ε → 0 ;
When the deterministic dynamics from the LLN has multiple stable fixed points, the effective dynamics over these metastable states.
The key object for both tasks is the quasipotential V ( x , y ) V(x,y) V ( x , y ) , which measures the “cost” of a fluctuation-driven transition from x x x to y y y . It is defined via a variational principle in § 4.3 and generalizes the notion of a potential landscape to systems that may not satisfy detailed balance.
The quasipotential determines the stationary distribution as follows. For a system with a single stable fixed point x ∗ x_* x ∗ , the stationary distribution satisfies
ρ ∗ ( x ) ≃ e − V ( x ) / ε . (58) \rho_*(x) \simeq e^{-V(x)/\varepsilon}. \tag*{(58)} ρ ∗ ( x ) ≃ e − V ( x ) / ε . (58)
Here and throughout, f ≃ g f \simeq g f ≃ g means lim ε → 0 ε log ( f / g ) = 0 \lim_{\varepsilon\to0} \varepsilon\log(f/g) = 0 lim ε → 0 ε log ( f / g ) = 0 , i.e., f f f and g g g have the same exponential scaling. In (58) , V ( x ) = V ( x ∗ , x ) V(x) = V(x_*,x) V ( x ) = V ( x ∗ , x ) . For systems with multiple stable fixed points x 1 , … , x n x_1,\ldots,x_n x 1 , … , x n , a single-argument function V ( x ) V(x) V ( x ) can be constructed by patching together the functions V ( x i , x ) V(x_i,x) V ( x i , x ) computed from each fixed point. The resulting V ( x ) V(x) V ( x ) is the viscosity solution of the stationary Hamilton–Jacobi equation (see § 4.4 ), and the stationary distribution still satisfies ρ ∗ ( x ) ≃ e − V ( x ) / ε \rho_*(x) \simeq e^{-V(x)/\varepsilon} ρ ∗ ( x ) ≃ e − V ( x ) / ε .
The quasipotential also determines transition rates between metastable states. When the deterministic dynamics has multiple stable fixed points, the stochastic system exhibits metastability : it spends long times near each fixed point, with rare noise-driven transitions between them. The transition rate from x i x_i x i to an adjacent fixed point x j x_j x j is
k i j ≃ e − V ( x i , x j ) / ε . (59) k_{ij} \simeq e^{-V(x_i,x_j)/\varepsilon}. \tag*{(59)} k ij ≃ e − V ( x i , x j ) / ε . (59)
For nonequilibrium systems, detailed balance is broken:
ρ ∗ ( x i ) k i j ≠ ρ ∗ ( x j ) k j i . (60) \rho_*(x_i)k_{ij} \ne\rho_*(x_j)k_{ji}. \tag*{(60)} ρ ∗ ( x i ) k ij = ρ ∗ ( x j ) k j i . (60)
This asymmetry—equivalently, V ( x i , x j ) ≠ V ( x j , x i ) V(x_i,x_j) \ne V(x_j,x_i) V ( x i , x j ) = V ( x j , x i ) in general—is a hallmark of systems that do not satisfy detailed balance and cannot be captured by the single-argument V ( x ) V(x) V ( x ) alone.
Order of limits. The quasipotential formalism arises from taking the limit ε → 0 \varepsilon\to0 ε → 0 first (yielding the Hamiltonian formalism of § 2 and § 3 ), followed by T → ∞ T \to\infty T → ∞ . These limits do not generally commute for multistable systems; see § 5.3 for a detailed discussion. The quasipotential characterizes the system’s behavior on timescales T ∼ e c / ε T \sim e^{c/\varepsilon} T ∼ e c / ε for appropriate constants c > 0 c > 0 c > 0 , during which transitions between metastable states become relevant. The systematic analysis of metastable transitions, including the hierarchy of timescales and the construction of cycles describing the sequence of most likely transitions, is developed in detail in [1 ] .
The stationary Hamilton–Jacobi equation
To characterize the long-time behavior of the stochastic system, we seek the stationary distribution ρ ∗ ( x ) \rho_*(x) ρ ∗ ( x ) . For SDEs, this satisfies the stationary forward Kolmogorov equation
0 = − ∇ ⋅ ( b ρ ∗ ) + ε ∇ ⋅ ( D ∇ ρ ∗ ) = − ∇ ⋅ J , (61) 0 = -\nabla\cdot(b\rho_*) + \varepsilon\nabla\cdot(D\nabla\rho_*) = -\nabla\cdot J, \tag*{(61)} 0 = − ∇ ⋅ ( b ρ ∗ ) + ε ∇ ⋅ ( D ∇ ρ ∗ ) = − ∇ ⋅ J , (61)
where J = b ρ ∗ − ϵ D ∇ ρ ∗ J=b\rho_*-\epsilon D\nabla\rho_* J = b ρ ∗ − ϵD ∇ ρ ∗ is the probability current. Substituting the WKB ansatz ρ ∗ ( x ) = w ( x ) e − V ( x ) / ϵ \rho_*(x)=w(x)e^{-V(x)/\epsilon} ρ ∗ ( x ) = w ( x ) e − V ( x ) / ϵ and collecting terms at leading order O ( ϵ − 1 ) O(\epsilon^{-1}) O ( ϵ − 1 ) , we obtain the stationary Hamilton–Jacobi equation :
H ( x , ∇ V ) = 0. (62) H(x,\nabla V)=0. \tag*{(62)} H ( x , ∇ V ) = 0. (62)
An analogous derivation starting from the stationary master equation yields the same structure for MJPs, with the Hamiltonian (51) .
A key feature of this equation is that it generically admits multiple solution branches : the equation H ( x , θ ) = 0 H(x,\theta)=0 H ( x , θ ) = 0 has several solutions θ ( x ) \theta(x) θ ( x ) at each point x x x . In one dimension, SDEs with quadratic Hamiltonian (15) yield two branches; in higher dimensions and for MJPs, the structure can be more complex.
One branch always corresponds to θ = 0 \theta=0 θ = 0 , which recovers the deterministic dynamics from the LLN. The other branches, with θ ≠ 0 \theta\ne0 θ = 0 , correspond to fluctuation-driven motion against the deterministic flow.
These branches encode essential physics: each corresponds to optimal paths from a different stable fixed point, and the associated costs determine transition rates via (59) .
The HJE (62) alone does not specify which branch to select at each point. To resolve this ambiguity, we define the quasipotential via a variational principle that explicitly specifies both endpoints.
Variational definition of the quasipotential
We now give a precise definition of the quasipotential that specifies both endpoints and thereby resolves the branch ambiguity.
Define the action of a path { x t } t ∈ [ 0 , T ] \{x_t\}_{t\in[0,T]} { x t } t ∈ [ 0 , T ] as
S T [ x ] = ∫ 0 T L ( x t , x ˙ t ) d t , (63) S_T[x]=\int_0^T L(x_t,\dot{x}_t)\,\mathrm{d}t, \tag*{(63)} S T [ x ] = ∫ 0 T L ( x t , x ˙ t ) d t , (63)
where L ( x , x ˙ ) = sup θ [ x ˙ ⋅ θ − H ( x , θ ) ] L(x,\dot{x})=\sup_\theta[\dot{x}\cdot\theta-H(x,\theta)] L ( x , x ˙ ) = sup θ [ x ˙ ⋅ θ − H ( x , θ )] is the Lagrangian obtained from the Hamiltonian via the Legendre transform. For SDEs with Hamiltonian (15) , this gives
L ( x , x ˙ ) = 1 4 ( x ˙ − b ( x ) ) ⋅ D − 1 ( x ) ( x ˙ − b ( x ) ) , (64) L(x,\dot{x})=\frac{1}{4}(\dot{x}-b(x))\cdot D^{-1}(x)(\dot{x}-b(x)), \tag*{(64)} L ( x , x ˙ ) = 4 1 ( x ˙ − b ( x )) ⋅ D − 1 ( x ) ( x ˙ − b ( x )) , (64)
measuring the “cost” of deviating from the deterministic flow. For MJPs, the Lagrangian is obtained from (51) and has a more complex, non-quadratic form.
The minimum cost to travel from x a x_a x a to x b x_b x b in time T T T is
V T ( x a , x b ) = inf ( x t ) t ∈ [ 0 , T ] : x 0 = x a , x T = x b S T [ x ] ≥ 0. (65) V_T(x_a,x_b)=\inf_{\substack{(x_t)_{t\in[0,T]}:\\ x_0=x_a,x_T=x_b}} S_T[x]\ge0. \tag*{(65)} V T ( x a , x b ) = ( x t ) t ∈ [ 0 , T ] : x 0 = x a , x T = x b inf S T [ x ] ≥ 0. (65)
The non-negativity follows from convexity of the Hamiltonian: x ˙ ⋅ θ − H ( x , θ ) ≥ − H ( x , 0 ) = 0 \dot{x}\cdot\theta-H(x,\theta)\ge-H(x,0)=0 x ˙ ⋅ θ − H ( x , θ ) ≥ − H ( x , 0 ) = 0 , with equality if and only if θ = 0 \theta=0 θ = 0 (i.e., along the deterministic flow).
The quasipotential is defined as the minimum cost over all possible transit times:
V ( x , y ) = inf T > 0 V T ( x , y ) . (66) V(x,y)=\inf_{T>0}V_T(x,y). \tag*{(66)} V ( x , y ) = T > 0 inf V T ( x , y ) . (66)
This definition is unambiguous: V ( x , y ) V(x,y) V ( x , y ) is the cost of the optimal fluctuation-driven path from x x x to y y y , minimized over all possible durations.
Basic properties. The quasipotential satisfies:
Non-negativity: V ( x , y ) ≥ 0 V(x,y) \ge0 V ( x , y ) ≥ 0 , with equality if and only if y y y can be reached from x x x along the deterministic flow.
Identity: V ( x , x ) = 0 V(x,x) = 0 V ( x , x ) = 0 .
Triangle inequality: V ( x , z ) ≤ V ( x , y ) + V ( y , z ) V(x,z) \le V(x,y) + V(y,z) V ( x , z ) ≤ V ( x , y ) + V ( y , z ) .
Asymmetry (generically): V ( x , y ) ≠ V ( y , x ) V(x,y) \ne V(y,x) V ( x , y ) = V ( y , x ) for nonequilibrium systems.
Optimal paths and the zero-energy surface
We now show that for transitions between stable fixed points, the variational definition (66) recovers the stationary Hamilton–Jacobi equation (62) along optimal paths.
Optimal paths between fixed points lie on H = 0 H = 0 H = 0 . Along any trajectory satisfying Hamilton’s equations, the energy E = H ( x t , θ t ) E = H(x_t,\theta_t) E = H ( x t , θ t ) is conserved. For a trajectory with constant energy E ≥ 0 E \ge0 E ≥ 0 , the action becomes
S T = ∫ 0 T [ x ˙ t ⋅ θ t − H ] d t = ∫ γ θ ⋅ d x − E T , (67) S_T = \int_{0}^{T} [\dot{x}_t \cdot\theta_t - H] \,\mathrm{d}t = \int_{\gamma} \theta\cdot\mathrm{d}x - ET, \tag*{(67)} S T = ∫ 0 T [ x ˙ t ⋅ θ t − H ] d t = ∫ γ θ ⋅ d x − E T , (67)
where ∫ γ θ ⋅ d x \int_{\gamma} \theta\cdot\mathrm{d}x ∫ γ θ ⋅ d x is a line integral depending only on the geometric path γ \gamma γ , not on how it is traversed in time.
For paths connecting stable fixed points (where θ → 0 \theta\to0 θ → 0 at the endpoints), the transit time T → ∞ T \to\infty T → ∞ for E = 0 E = 0 E = 0 , since the velocity x ˙ = ∂ θ H → 0 \dot{x} = \partial_{\theta}H \to0 x ˙ = ∂ θ H → 0 as θ → 0 \theta\to0 θ → 0 . For E > 0 E > 0 E > 0 , the term − E T -ET − E T penalizes long transit times, but paths with E > 0 E > 0 E > 0 yield larger total action than the E = 0 E = 0 E = 0 paths; see [1 ] for a rigorous treatment.
Thus, for transitions between stable fixed points, the infimum in (66) is achieved along paths on the zero-energy surface H ( x , θ ) = 0 H(x,\theta) = 0 H ( x , θ ) = 0 :
V ( x i , x j ) = ∫ γ θ ⋅ d x , (68) V(x_i,x_j) = \int_{\gamma} \theta\cdot\mathrm{d}x, \tag*{(68)} V ( x i , x j ) = ∫ γ θ ⋅ d x , (68)
where γ \gamma γ is the optimal path from x i x_i x i to x j x_j x j with H ( x , θ ) = 0 H(x,\theta) = 0 H ( x , θ ) = 0 throughout.
Recovery of the Hamilton–Jacobi equation. Along the optimal path from a stable fixed point x ∗ x_* x ∗ to x x x , the momentum satisfies θ = ∇ x V ( x ∗ , x ) \theta= \nabla_x V(x_*,x) θ = ∇ x V ( x ∗ , x ) . The constraint H ( x , θ ) = 0 H(x,\theta) = 0 H ( x , θ ) = 0 along this path shows that the two-point quasipotential satisfies the stationary HJE (62) with respect to its second argument.
Note that the HJE specifies ∇ V \nabla V ∇ V , not V V V itself. For a system with multiple stable fixed points, the single-argument function V ( x ) V(x) V ( x ) appearing in the stationary distribution satisfies ∇ V ( x ) = ∇ x V ( x i , x ) \nabla V(x) = \nabla_x V(x_i,x) ∇ V ( x ) = ∇ x V ( x i , x ) in the basin of x i x_i x i —this selects the appropriate branch. At basin boundaries, ∇ V \nabla V ∇ V is discontinuous as the branch changes, but V V V itself remains continuous. The global function V ( x ) V(x) V ( x ) is the viscosity solution of (62) .
Remark 4.1. Constructing a single-argument function V ( x ) V(x) V ( x ) such that ρ ∗ ( x ) ≍ e − V ( x ) / ϵ \rho_*(x) \asymp e^{-V(x)/\epsilon} ρ ∗ ( x ) ≍ e − V ( x ) / ϵ from the two-point quasipotentials V ( x i , x ) V(x_i,x) V ( x i , x ) requires adding appropriate constants when patching solutions from different basins. These constants are determined by the condition that ρ ∗ ( x i ) \rho_*(x_i) ρ ∗ ( x i ) must be the stationary distribution of the effective Markov jump process over fixed points with rates (59) . Recall that this quasipotential formalism arises from taking ϵ → 0 \epsilon\to0 ϵ → 0 first, then T → ∞ T \to\infty T → ∞ —the opposite order from the tilted generator approach in Section 5. See [1 , 21 ] for details.
Gradient systems and detailed balance
Before analyzing general nonequilibrium systems, we examine the important special case of gradient systems, where the quasipotential takes a particularly simple form.
Consider the SDE
d X t ε = − D ∇ U ( X t ε ) d t + 2 ε σ d W t , (69) dX_t^\varepsilon= -D\nabla U(X_t^\varepsilon)\,dt + \sqrt{2\varepsilon}\,\sigma\,dW_t, \tag*{(69)} d X t ε = − D ∇ U ( X t ε ) d t + 2 ε σ d W t , (69)
where D = σ σ T D = \sigma\sigma^T D = σ σ T and U ( x ) U(x) U ( x ) is a potential. This system satisfies detailed balance : the probability current vanishes at stationarity, and the stationary distribution is the Boltzmann distribution
ρ ∗ ( x ) ∝ e − U ( x ) / ε . (70) \rho_*(x) \propto e^{-U(x)/\varepsilon}. \tag*{(70)} ρ ∗ ( x ) ∝ e − U ( x ) / ε . (70)
Quasipotential equals the potential. For gradient systems, the two-point quasipotential is simply the potential difference:
V ( x , y ) = U ( y ) − U ( x ) . (71) V(x,y) = U(y) - U(x). \tag*{(71)} V ( x , y ) = U ( y ) − U ( x ) . (71)
To verify this, note that the Hamiltonian for this system is
H ( x , θ ) = − D ∇ U ( x ) ⋅ θ + θ T D θ . (72) H(x,\theta) = -D\nabla U(x)\cdot\theta+ \theta^T D\theta. \tag*{(72)} H ( x , θ ) = − D ∇ U ( x ) ⋅ θ + θ T D θ . (72)
The zero-energy surface H ( x , θ ) = 0 H(x,\theta) = 0 H ( x , θ ) = 0 has two branches: θ = 0 \theta= 0 θ = 0 (relaxation) and θ = ∇ U \theta= \nabla U θ = ∇ U (fluctuation). Integrating along the fluctuation branch from x x x to y y y :
V ( x , y ) = ∫ γ θ ⋅ d x = ∫ γ ∇ U ⋅ d x = U ( y ) − U ( x ) . (73) V(x,y) = \int_\gamma\theta\cdot dx = \int_\gamma\nabla U\cdot dx = U(y) - U(x). \tag*{(73)} V ( x , y ) = ∫ γ θ ⋅ d x = ∫ γ ∇ U ⋅ d x = U ( y ) − U ( x ) . (73)
Time-reversal symmetry. The simplicity of (71) reflects a deeper structure: for gradient systems, the optimal fluctuation path from x x x to y y y is the time-reversal of the deterministic relaxation path from y y y to x x x . This follows from Hamilton’s equations: on the fluctuation branch θ = ∇ U \theta= \nabla U θ = ∇ U , we have
x ˙ = ∂ θ H = D ∇ U ( x ) , (74) \dot{x} = \partial_\theta H = D\nabla U(x), \tag*{(74)} x ˙ = ∂ θ H = D ∇ U ( x ) , (74)
which is the time-reversal of the relaxation dynamics x ˙ = − D ∇ U ( x ) \dot{x} = -D\nabla U(x) x ˙ = − D ∇ U ( x ) (up to a rescaling of time). The fluctuation path retraces the relaxation path in reverse.
Symmetric barriers. As a consequence, the quasipotential is symmetric. For two minima x ± x_\pm x ± separated by a saddle x s x_s x s :
V ( x − , x s ) = U ( x s ) − U ( x − ) , V ( x + , x s ) = U ( x s ) − U ( x + ) . (75) V(x_-,x_s) = U(x_s) - U(x_-), \qquad V(x_+,x_s) = U(x_s) - U(x_+). \tag*{(75)} V ( x − , x s ) = U ( x s ) − U ( x − ) , V ( x + , x s ) = U ( x s ) − U ( x + ) . (75)
The ratio of transition rates satisfies
k − + k + − = e − [ V ( x − , x + ) − V ( x + , x − ) ] / ε = e − [ U ( x + ) − U ( x − ) ] / ε , (76) \frac{k_{-+}}{k_{+-}} = e^{-[V(x_-,x_+)-V(x_+,x_-)]/\varepsilon} = e^{-[U(x_+)-U(x_-)]/\varepsilon}, \tag*{(76)} k +− k −+ = e − [ V ( x − , x + ) − V ( x + , x − )] / ε = e − [ U ( x + ) − U ( x − )] / ε , (76)
consistent with detailed balance: ρ ∗ ( x − ) k − + = ρ ∗ ( x + ) k + − \rho_*(x_-)k_{-+} = \rho_*(x_+)k_{+-} ρ ∗ ( x − ) k −+ = ρ ∗ ( x + ) k +− .
Example: double-well potential. Consider the one-dimensional double-well potential
U ( x ) = 1 4 ( x 2 − 1 ) 2 , (77) U(x) = \frac{1}{4}(x^2 - 1)^2, \tag*{(77)} U ( x ) = 4 1 ( x 2 − 1 ) 2 , (77)
with minima at x ± = ± 1 x_\pm= \pm1 x ± = ± 1 and a saddle at x s = 0 x_s = 0 x s = 0 . The quasipotential barriers are
V ( x − , x s ) = V ( x + , x s ) = U ( 0 ) − U ( ± 1 ) = 1 4 , (78) V(x_-,x_s) = V(x_+,x_s) = U(0) - U(\pm1) = \frac{1}{4}, \tag*{(78)} V ( x − , x s ) = V ( x + , x s ) = U ( 0 ) − U ( ± 1 ) = 4 1 , (78)
and the transition rates are equal: k − + = k + − ≃ e − 1 / ( 4 ε ) k_{-+} = k_{+-} \simeq e^{-1/(4\varepsilon)} k −+ = k +− ≃ e − 1/ ( 4 ε ) .
Non-gradient systems
For systems that do not satisfy detailed balance, the quasipotential framework remains valid but its structure is richer. We now discuss the key features that distinguish non-gradient systems from the equilibrium case.
Broken time-reversal symmetry. For gradient systems, the optimal fluctuation path from x x x to y y y is the time-reversal of the relaxation path from y y y to x x x . This symmetry is broken for non-gradient systems: the forward and backward optimal paths are generically distinct, reflecting the nonequilibrium nature of the dynamics.
Asymmetric quasipotential. As a consequence, the two-point quasipotential is asymmetric:
V ( x , y ) ≠ V ( y , x ) . (79) V(x,y) \ne V(y,x). \tag*{(79)} V ( x , y ) = V ( y , x ) . (79)
The cost of transitioning from x x x to y y y differs from the reverse. This asymmetry has physical consequences: the ratio of forward and backward transition rates,
k i j k j i = e − [ V ( x i , x j ) − V ( x j , x i ) ] / ϵ , (80) \frac{k_{ij}}{k_{ji}} = e^{-[V(x_i,x_j)-V(x_j,x_i)]/\epsilon}, \tag*{(80)} k j i k ij = e − [ V ( x i , x j ) − V ( x j , x i )] / ϵ , (80)
is not determined by an equilibrium free energy difference, but by the full nonequilibrium dynamics. Detailed balance is violated:
ρ ∗ ( x i ) k i j ≠ ρ ∗ ( x j ) k j i . (81) \rho_*(x_i)k_{ij} \ne\rho_*(x_j)k_{ji}. \tag*{(81)} ρ ∗ ( x i ) k ij = ρ ∗ ( x j ) k j i . (81)
Non-trivial branch structure. For gradient systems, the global quasipotential V ( x ) = U ( x ) − U ( x ∗ ) V(x)=U(x)-U(x_*) V ( x ) = U ( x ) − U ( x ∗ ) corresponds to a single branch of the HJE (the fluctuation branch θ = ∇ U \theta=\nabla U θ = ∇ U ). For non-gradient systems, the fluctuation branches emanating from different stable fixed points are genuinely distinct and must be explicitly patched together at basin boundaries, as described in § 4.4 .
Limiting Markov chain on attractors. When the deterministic dynamics has multiple stable fixed points { x 1 , … , x n } \{x_1,\ldots,x_n\} { x 1 , … , x n } , the quasipotential induces a hierarchy of timescales. On times of order e V i j / ϵ e^{V_{ij}/\epsilon} e V ij / ϵ , where V i j = V ( x i , x j ) V_{ij}=V(x_i,x_j) V ij = V ( x i , x j ) is the cost of transition from x i x_i x i to x j x_j x j , transitions between basins become relevant. The long-time dynamics can be approximated by a Markov jump process on the discrete set of attractors, with transition rates given by the Arrhenius formula (59).
This reduced Markov chain inherits the asymmetry of the quasipotential: its transition matrix is not reversible unless the original system satisfies detailed balance. In particular, the stationary distribution of this chain determines the constants needed to patch the single-argument quasipotential V ( x ) V(x) V ( x ) , as discussed in § 4.4 . The construction of this limiting Markov chain, and the identification of the hierarchy of metastable states through cycles of transitions, is developed systematically in [1 , 21 ] .
Example: the Schlögl model
The Schlögl model introduced in § 3 provides a concrete illustration of the quasipotential formalism for non-gradient systems.
Scalar case
Recall that the Hamiltonian for the Schlögl model is
H ( x , θ ) = λ + ( x ) ( e θ − 1 ) + λ − ( x ) ( e − θ − 1 ) , (82) H(x,\theta)=\lambda_{+}(x)(e^{\theta}-1)+\lambda_{-}(x)(e^{-\theta}-1), \tag*{(82)} H ( x , θ ) = λ + ( x ) ( e θ − 1 ) + λ − ( x ) ( e − θ − 1 ) , (82)
with λ + ( x ) = k 1 c B + k 3 c A x 2 \lambda_{+}(x)=k_{1}c_{B}+k_{3}c_{A}x^{2} λ + ( x ) = k 1 c B + k 3 c A x 2 and λ − ( x ) = k 0 x + k 2 x 3 \lambda_{-}(x)=k_{0}x+k_{2}x^{3} λ − ( x ) = k 0 x + k 2 x 3 . The deterministic drift is
b ( x ) = λ + ( x ) − λ − ( x ) = k 1 c B + k 3 c A x 2 − k 0 x − k 2 x 3 . (83) b(x)=\lambda_{+}(x)-\lambda_{-}(x)=k_{1}c_{B}+k_{3}c_{A}x^{2}-k_{0}x-k_{2}x^{3}. \tag*{(83)} b ( x ) = λ + ( x ) − λ − ( x ) = k 1 c B + k 3 c A x 2 − k 0 x − k 2 x 3 . (83)
For appropriate parameter values, the deterministic dynamics has two stable fixed points x ± ∗ x_{\pm}^{*} x ± ∗ and one unstable fixed point x 0 ∗ x_{0}^{*} x 0 ∗ (Figure 1 ).
Graphical construction. In one dimension, the quasipotential has a simple graphical interpretation in the ( x , θ ) (x,\theta) ( x , θ ) phase plane. The zero-energy surface H ( x , θ ) = 0 H(x,\theta)=0 H ( x , θ ) = 0 consists of curves connecting the fixed points. For the Schlögl model, solving H ( x , θ ) = 0 H(x,\theta)=0 H ( x , θ ) = 0 yields a transcendental equation with two branches: the relaxation branch θ = 0 \theta=0 θ = 0 (lying along the x x x -axis) and a fluctuation branch θ ( x ) ≠ 0 \theta(x)\ne0 θ ( x ) = 0 .
Figure 2 (a) shows this phase portrait. Starting from a stable fixed point x − x_{-} x − , the quasipotential V ( x − , x ) V(x_{-},x) V ( x − , x ) is computed by integrating along the fluctuation branch:
Figure 2. (a) Phase portrait for the bistable Schlögl model showing the Hamiltonian contours H ( x , θ ) H(x,\theta) H ( x , θ ) . Black disks indicate stable fixed points x − x_{-} x − (left) and x + x_{+} x + (right); the orange square marks the unstable fixed point at x s x_s x s . White lines represent the zero-energy surface H ( x , θ ) = 0 H(x,\theta)=0 H ( x , θ ) = 0 : the dashed horizontal line (θ = 0 \theta=0 θ = 0 ) corresponds to deterministic relaxation, while the solid curve (θ ≠ 0 \theta\ne0 θ = 0 ) represents optimal fluctuation paths. Arrows indicate the direction of motion along each branch. The quasipotential V ( x − , x ) V(x_{-},x) V ( x − , x ) for x ≤ x s x\le x_s x ≤ x s equals the area between the two branches from x − x_{-} x − to x x x . (b) The global quasipotential V ( x ) V(x) V ( x ) , obtained by patching V ( x − , x ) V(x_{-},x) V ( x − , x ) and V ( x + , x ) V(x_{+},x) V ( x + , x ) with appropriate constants. Adapted from [22 ] .
V ( x − , x ) = ∫ x − x θ ( x ′ ) d x ′ . (84) V(x_{-},x)=\int_{x_{-}}^{x}\theta(x')\,\mathrm{d}x'. \tag*{(84)} V ( x − , x ) = ∫ x − x θ ( x ′ ) d x ′ . (84)
Geometrically, this integral equals the area between the two branches in the phase portrait. The global quasipotential, shown in Figure 2 (b), is obtained by computing V ( x − , x ) V(x_{-},x) V ( x − , x ) and V ( x + , x ) V(x_{+},x) V ( x + , x ) from each stable fixed point and patching them together with appropriate constants as described in § 4.4 .
Quasipotential versus naïve potential. One might attempt to construct a potential via
U ( x ) = − ∫ x b ( x ′ ) d x ′ = − k 1 x − k 3 3 x 3 + k 0 2 x 2 + k 2 4 x 4 , (85) U(x)=-\int^{x}b(x')\,\mathrm{d}x'=-k_{1}x-\frac{k_{3}}{3}x^{3}+\frac{k_{0}}{2}x^{2}+\frac{k_{2}}{4}x^{4}, \tag*{(85)} U ( x ) = − ∫ x b ( x ′ ) d x ′ = − k 1 x − 3 k 3 x 3 + 2 k 0 x 2 + 4 k 2 x 4 , (85)
and naïvely expect the stationary distribution to be ρ ( x ) ∝ e − U ( x ) / ε \rho(x)\propto e^{-U(x)/\varepsilon} ρ ( x ) ∝ e − U ( x ) / ε . However, this is incorrect: the Schlögl model is a Markov jump process, not a gradient diffusion, and the Hamiltonian is not quadratic in θ \theta θ . The true quasipotential V ( x ) V(x) V ( x ) , obtained by solving H ( x , V ′ ( x ) ) = 0 H(x,V'(x))=0 H ( x , V ′ ( x )) = 0 and patching appropriately, differs from U ( x ) U(x) U ( x ) .
Figure 3 shows that as the system size Ω = 1 / ε \Omega=1/\varepsilon Ω = 1/ ε increases, the rescaled stationary distribution converges to e − V ( x ) / ε e^{-V(x)/\varepsilon} e − V ( x ) / ε , not e − U ( x ) / ε e^{-U(x)/\varepsilon} e − U ( x ) / ε . Strikingly, the relative stability of the two minima can be reversed: the naïve potential U ( x ) U(x) U ( x ) may predict that x + x_{+} x + is more stable, while the quasipotential correctly identifies x − x_{-} x − as the globally stable state.
Figure 3. Rescaled logarithm of the stationary distribution for the Schlögl model as the system size Ω = 1 / ε \Omega= 1/\varepsilon Ω = 1/ ε increases, showing convergence to the quasipotential V ( x ) V(x) V ( x ) rather than the naïve potential U ( x ) U(x) U ( x ) . For the parameters shown, x − x_{-} x − is the stable phase according to the quasipotential, while the naïve potential incorrectly predicts x + x_{+} x + as stable. Parameters: k 1 = 0.8 k_1 = 0.8 k 1 = 0.8 , k 0 = 2.9 k_0 = 2.9 k 0 = 2.9 , k 3 = 3.1 k_3 = 3.1 k 3 = 3.1 , k 2 = 1 k_2 = 1 k 2 = 1 . Reproduced from [22 ] .
This discrepancy arises because the naïve potential U ( x ) U(x) U ( x ) is constructed from the deterministic drift alone, ignoring the state-dependent fluctuation structure encoded in the full Hamiltonian. For gradient diffusions, the drift and fluctuations are related by detailed balance, and U U U coincides with the quasipotential. For the Schlögl model—and for non-gradient systems generally—this relationship breaks down.
Spatially extended systems
The Schlögl model can be extended to a spatially distributed system [23 ] by dividing a domain into L L L compartments. Within each compartment i i i , the Schlögl reactions occur locally; particles also hop between adjacent compartments at rate α \alpha α . The Hamiltonian is
H = H R + H D , (86) H = H^{R} + H^{D}, \tag*{(86)} H = H R + H D , (86)
where
H R = ∑ i = 1 L [ λ + ( x i ) ( e θ i − 1 ) + λ − ( x i ) ( e − θ i − 1 ) ] , (87) H^{R} = \sum_{i=1}^{L} \left[\lambda_{+}(x_i)(e^{\theta_i} - 1) + \lambda_{-}(x_i)(e^{-\theta_i} - 1)\right], \tag*{(87)} H R = i = 1 ∑ L [ λ + ( x i ) ( e θ i − 1 ) + λ − ( x i ) ( e − θ i − 1 ) ] , (87)
H D = α ∑ i = 1 L x i [ e θ i + 1 − θ i + e θ i − 1 − θ i − 2 ] . (88) H^{D} = \alpha\sum_{i=1}^{L} x_i \left[e^{\theta_{i+1}-\theta_i} + e^{\theta_{i-1}-\theta_i} - 2\right]. \tag*{(88)} H D = α i = 1 ∑ L x i [ e θ i + 1 − θ i + e θ i − 1 − θ i − 2 ] . (88)
For two compartments (L = 2 L = 2 L = 2 ), the phase portrait becomes two-dimensional (Figure 4 ). The forward and backward minimum-action paths connecting the metastable states are distinct, confirming nonequilibrium behavior. The asymmetry in the action along these paths determines which state is globally stable.
Figure 4. Optimal transition paths in the bistable Schlögl model with two reactive compartments coupled by particle exchange at rate α \alpha α . (a) Flow lines of the deterministic dynamics with stable fixed points (black dots) and unstable fixed points (black crosses). The solid blue and orange lines show the minimum-action paths between metastable states; the dashed red line is the heteroclinic orbit. The divergence between these paths indicates nonequilibrium behavior. (b) Action accumulated along both minimum-action paths, demonstrating that the lower-left state is more stable under stochastic fluctuations. Reproduced from [22 ] .
For larger systems, the diffusion rate α \alpha α controls a nonequilibrium phase transition [23 ] . At a critical value α c \alpha_c α c , the relative stability of the high-density (x + x_{+} x + ) and low-density (x − x_{-} x − ) phases reverses (Figure 5 ). This transition cannot be predicted from the deterministic dynamics alone—it emerges from the interplay of reactions and diffusion in the stochastic system.
Figure 5. Phase diagram of the spatially extended Schlögl model as a function of the hopping rate α \alpha α . For α < α c ≈ 5.0 \alpha< \alpha_{\mathrm{c}} \approx5.0 α < α c ≈ 5.0 , the high-density phase x + x_{+} x + is stable; for α > α c \alpha> \alpha_{\mathrm{c}} α > α c , the low-density phase x − x_{-} x − becomes stable. This nonequilibrium phase transition depends on the full stochastic dynamics, not just the deterministic flow. Parameters: k 1 = 0.8 k_{1}=0.8 k 1 = 0.8 , k 0 = 2.9 k_{0}=2.9 k 0 = 2.9 , k 3 = 3.1 k_{3}=3.1 k 3 = 3.1 , k 2 = 1 k_{2}=1 k 2 = 1 , L = 40 L=40 L = 40 . Reproduced from [22 ] .
Figure 6 shows the optimal transition paths for L = 40 L = 40 L = 40 compartments. The transitions x − → x + x_{-} \to x_{+} x − → x + and x + → x − x_{+} \to x_{-} x + → x − proceed via qualitatively different mechanisms: one involves nucleation of a localized domain, while the other occurs more uniformly. This asymmetry is a hallmark of nonequilibrium phase coexistence.
Figure 6. Optimal transition paths in the spatially extended Schlögl model with L = 40 L=40 L = 40 compartments. Top row: density profiles ρ ^ ( s , x ) \hat{\rho}(s,x) ρ ^ ( s , x ) along the transition path (parameterized by s s s ). Middle row: conjugate momentum θ ^ ( s , x ) \hat{\theta}(s,x) θ ^ ( s , x ) . Bottom row: Lagrangian along the trajectory. (a) Transition from x − x_{-} x − to x + x_{+} x + . (b) Transition from x + x_{+} x + to x − x_{-} x − . Note the pronounced asymmetry between forward and backward paths. (c) Heteroclinic orbit connecting x − x_{-} x − and x + x_{+} x + . Parameters: k 1 = 0.8 k_{1}=0.8 k 1 = 0.8 , k 0 = 2.9 k_{0}=2.9 k 0 = 2.9 , k 3 = 3.1 k_{3}=3.1 k 3 = 3.1 , k 2 = 1 k_{2}=1 k 2 = 1 , α = 12 \alpha=12 α = 12 . Reproduced from [22 ] .
Pitfall: adding noise to the deterministic equations
It is tempting to capture fluctuations by adding small noise to the deterministic equations from the LLN. For the continuum Schlögl model, this procedure yields the Allen–Cahn/Ginzburg–Landau equation
∂ t x = α ∂ s 2 x + λ + ( x ) − λ − ( x ) , (89) \partial_t x = \alpha\partial_s^2 x + \lambda_+(x) - \lambda_-(x), \tag*{(89)} ∂ t x = α ∂ s 2 x + λ + ( x ) − λ − ( x ) , (89)
which is a gradient flow for the energy
E [ x ] = ∫ 0 1 [ α 2 ∣ ∂ s x ∣ 2 + U ( x ) ] d s . (90) E[x] = \int_0^1 \left[\frac{\alpha}{2}|\partial_s x|^2 + U(x)\right]\,\mathrm{d}s. \tag*{(90)} E [ x ] = ∫ 0 1 [ 2 α ∣ ∂ s x ∣ 2 + U ( x ) ] d s . (90)
If this were the correct description, the stationary distribution would be ρ [ x ] ∝ e − E [ x ] / ϵ \rho[x] \propto e^{-E[x]/\epsilon} ρ [ x ] ∝ e − E [ x ] / ϵ , implying that the relative stability of phases depends only on U ( x ) U(x) U ( x ) , not on α \alpha α .
However, the full stochastic analysis reveals an α \alpha α -dependent phase transition (Figure 5 ). The naïve approach fails because it ignores the non-Gaussian fluctuation structure encoded in the full Hamiltonian (51) . This is a cautionary tale: adding noise to deterministic equations does not generally reproduce the correct large-deviation behavior.
Long-time asymptotics II: Finite noise and tilted generators
The preceding sections analyzed large deviations in the small-noise limit ϵ → 0 \epsilon\to0 ϵ → 0 , either at fixed time T T T or in the limit as T → ∞ T \to\infty T → ∞ taken a posteriori . We now consider the complementary regime: fixed noise strength with T → ∞ T \to\infty T → ∞ . This long-time limit characterizes time-averaged observables and connects to spectral properties of the generator.
The tilted generator
Consider the diffusion process
d X t = b ( X t ) d t + 2 σ ( X t ) d W t , (91) \mathrm{d}X_t = b(X_t)\,\mathrm{d}t + \sqrt{2}\sigma(X_t)\,\mathrm{d}W_t, \tag*{(91)} d X t = b ( X t ) d t + 2 σ ( X t ) d W t , (91)
and the exponentially weighted expectation
I T = E [ e ∫ 0 T f ( X t ) d t ] , (92) I_T = \mathbb{E}\left[e^{\int_0^T f(X_t)\,\mathrm{d}t}\right], \tag*{(92)} I T = E [ e ∫ 0 T f ( X t ) d t ] , (92)
where f : R d → R f:\mathbb{R}^d \to\mathbb{R} f : R d → R is an observable. We seek the behavior of I T I_T I T as T → ∞ T \to\infty T → ∞ .
Define the joint process ( X t , A t ) (X_t,A_t) ( X t , A t ) where A ˙ t = f ( X t ) \dot{A}_t=f(X_t) A ˙ t = f ( X t ) with A 0 = 0 A_0=0 A 0 = 0 , so that A T = ∫ 0 T f ( X t ) d t A_T=\int_0^T f(X_t)\,\mathrm{d}t A T = ∫ 0 T f ( X t ) d t and I t = E [ e A T ] I_t=\mathbb{E}[e^{A_T}] I t = E [ e A T ] . Let ρ ( t , x , a ) \rho(t,x,a) ρ ( t , x , a ) be the joint probability density function of ( X t , A t ) (X_t,A_t) ( X t , A t ) ; it evolves according to the forward Kolmogorov equation
∂ t ρ = − ∇ ⋅ ( b ρ ) + D : ∇ ∇ ρ − f ( x ) ∂ a ρ , (93) \partial_t\rho=-\nabla\cdot(b\rho)+D:\nabla\nabla\rho-f(x)\,\partial_a\rho, \tag*{(93)} ∂ t ρ = − ∇ ⋅ ( b ρ ) + D : ∇∇ ρ − f ( x ) ∂ a ρ , (93)
where D = σ σ T D=\sigma\sigma^{\mathsf{T}} D = σ σ T . The e a e^a e a -weighted marginal ρ ^ ( t , x ) = ∫ − ∞ ∞ e a ρ ( t , x , a ) d a \hat{\rho}(t,x)=\int_{-\infty}^{\infty}e^a\rho(t,x,a)\,\mathrm{d}a ρ ^ ( t , x ) = ∫ − ∞ ∞ e a ρ ( t , x , a ) d a satisfies
∂ t ρ ^ = − ∇ ⋅ ( b ρ ^ ) + D : ∇ ∇ ρ ^ + f ( x ) ρ ^ , (94) \partial_t\hat{\rho}=-\nabla\cdot(b\hat{\rho})+D:\nabla\nabla\hat{\rho}+f(x)\hat{\rho}, \tag*{(94)} ∂ t ρ ^ = − ∇ ⋅ ( b ρ ^ ) + D : ∇∇ ρ ^ + f ( x ) ρ ^ , (94)
and I T = ∫ R d ρ ^ ( T , x ) d x I_T=\int_{\mathbb{R}^d}\hat{\rho}(T,x)\,\mathrm{d}x I T = ∫ R d ρ ^ ( T , x ) d x .
Note that ρ ^ \hat{\rho} ρ ^ does not preserve mass: ∫ R d ρ ^ ( t , x ) d x = E [ e A t ] = I t \int_{\mathbb{R}^d}\hat{\rho}(t,x)\,\mathrm{d}x=\mathbb{E}[e^{A_t}]=I_t ∫ R d ρ ^ ( t , x ) d x = E [ e A t ] = I t , which grows or decays depending on the sign of f f f . The normalized density ρ ~ ( t , x ) = ρ ^ ( t , x ) / ∫ R d ρ ^ ( t , y ) d y \tilde{\rho}(t,x)=\hat{\rho}(t,x)/\int_{\mathbb{R}^d}\hat{\rho}(t,y)\,\mathrm{d}y ρ ~ ( t , x ) = ρ ^ ( t , x ) / ∫ R d ρ ^ ( t , y ) d y satisfies
∂ t ρ ~ = − ∇ ⋅ ( b ρ ~ ) + D : ∇ ∇ ρ ~ + f ( x ) ρ ~ − λ ( t ) ρ ~ t , (95) \partial_t\tilde{\rho}=-\nabla\cdot(b\tilde{\rho})+D:\nabla\nabla\tilde{\rho}+f(x)\tilde{\rho}-\lambda(t)\tilde{\rho}_t, \tag*{(95)} ∂ t ρ ~ = − ∇ ⋅ ( b ρ ~ ) + D : ∇∇ ρ ~ + f ( x ) ρ ~ − λ ( t ) ρ ~ t , (95)
where λ ( t ) = ∫ R d f ( x ) ρ ~ ( t , x ) d x \lambda(t) = \int_{\mathbb{R}^d} f(x)\tilde{\rho}(t,x)\,dx λ ( t ) = ∫ R d f ( x ) ρ ~ ( t , x ) d x . A direct calculation shows that
∫ f ( x ) ρ ~ ( t , x ) d x = ∫ R d f ( x ) ρ ^ ( t , x ) d x ∫ R d ρ ^ ( t , x ) d x = E [ f ( X t ) e A t ] E [ e A t ] = E [ f ( X t ) e ∫ 0 t f ( X s ) d s ] E [ e ∫ 0 t f ( X s ) d s ] = d d t log I t (96) \int f(x)\tilde{\rho}(t,x)\,dx = \frac{\int_{\mathbb{R}^d} f(x)\hat{\rho}(t,x)\,dx}{\int_{\mathbb{R}^d} \hat{\rho}(t,x)\,dx} = \frac{\mathbb{E}[f(X_t)e^{A_t}]}{\mathbb{E}[e^{A_t}]} = \frac{\mathbb{E}\left[f(X_t)e^{\int_0^t f(X_s)\,ds}\right]}{\mathbb{E}\left[e^{\int_0^t f(X_s)\,ds}\right]} = \frac{d}{dt}\log I_t \tag*{(96)} ∫ f ( x ) ρ ~ ( t , x ) d x = ∫ R d ρ ^ ( t , x ) d x ∫ R d f ( x ) ρ ^ ( t , x ) d x = E [ e A t ] E [ f ( X t ) e A t ] = E [ e ∫ 0 t f ( X s ) d s ] E [ f ( X t ) e ∫ 0 t f ( X s ) d s ] = d t d log I t (96)
Thus λ ( t ) = d d t log I t \lambda(t) = \frac{d}{dt}\log I_t λ ( t ) = d t d log I t and so I T = e ∫ 0 T λ t d t I_T = e^{\int_0^T \lambda_t\,dt} I T = e ∫ 0 T λ t d t .
In the long-time limit, assuming convergence to a stationary distribution ρ ~ ( t , x ) → ρ ~ ∗ ( x ) \tilde{\rho}(t,x) \to\tilde{\rho}_*(x) ρ ~ ( t , x ) → ρ ~ ∗ ( x ) , we have λ ( t ) → λ ∗ \lambda(t) \to\lambda_* λ ( t ) → λ ∗ and
lim T → ∞ 1 T log I T = λ ∗ . (97) \lim_{T\to\infty}\frac{1}{T}\log I_T = \lambda_*. \tag*{(97)} T → ∞ lim T 1 log I T = λ ∗ . (97)
The quantity λ ∗ \lambda_* λ ∗ is the principal eigenvalue (the eigenvalue with largest real part) of the tilted generator
L f ρ ~ = − ∇ ⋅ ( b ρ ~ ) + D : ∇ ∇ ρ ~ + f ( x ) ρ ~ = λ ∗ ρ ~ . (98) L_f\tilde{\rho} = -\nabla\cdot(b\tilde{\rho}) + D:\nabla\nabla\tilde{\rho} + f(x)\tilde{\rho} = \lambda_*\tilde{\rho}. \tag*{(98)} L f ρ ~ = − ∇ ⋅ ( b ρ ~ ) + D : ∇∇ ρ ~ + f ( x ) ρ ~ = λ ∗ ρ ~ . (98)
When f ( x ) = s ϕ ( x ) f(x) = s\phi(x) f ( x ) = s ϕ ( x ) , the quantity λ ∗ ( s ) \lambda_*(s) λ ∗ ( s ) is the scaled cumulant generating function (SCGF) of the time-averaged observable ϕ ˉ = 1 T ∫ 0 T ϕ ( X t ) d t \bar{\phi} = \frac{1}{T}\int_0^T \phi(X_t)\,dt ϕ ˉ = T 1 ∫ 0 T ϕ ( X t ) d t . The Legendre transform of λ ∗ ≡ λ ∗ ( s ) \lambda_* \equiv\lambda_*(s) λ ∗ ≡ λ ∗ ( s ) gives the large-deviation rate function for ϕ ˉ \bar{\phi} ϕ ˉ . The spectral approach to large deviations via tilted generators originates in the work of Donsker and Varadhan [24 ] ; see [25 ] for a modern treatment in the context of nonequilibrium processes.
Remark 5.1 (Gärtner–Ellis theorem and its limitations). The Legendre relationship between Hamiltonian and Lagrangian encountered in Sections 2 and 3 is the infinitesimal version of the Gärtner–Ellis theorem. Here, this structure appears globally: the SCGF λ ∗ ( s ) \lambda_*(s) λ ∗ ( s ) is the cumulant generating function for the time-averaged observable, and the rate function is obtained via Legendre transform, I ( a ) = sup s [ s a − λ ∗ ( s ) ] I(a) = \sup_s[sa - \lambda_*(s)] I ( a ) = sup s [ s a − λ ∗ ( s )] [7 , 8 ] . This approach succeeds when λ ∗ ( s ) \lambda_*(s) λ ∗ ( s ) is everywhere differentiable; the rate function is then convex. If λ ∗ ( s ) \lambda_*(s) λ ∗ ( s ) has a non-differentiable point, the Legendre transform yields only a lower bound, and the true rate function may be non-convex. Such singularities arise when the system exhibits qualitatively different mechanisms for achieving different fluctuation magnitudes. See [2 , 26 , 27 ] for comprehensive discussions.
Population dynamics interpretation
Equation (95) admits an interpretation as a birth-death process on an ensemble of trajectories. The term f ( x ) ρ ~ − λ t ρ ~ f(x)\tilde{\rho} - \lambda_t\tilde{\rho} f ( x ) ρ ~ − λ t ρ ~ acts as a source/sink: trajectories with f ( X t ) > λ ( t ) f(X_t) > \lambda(t) f ( X t ) > λ ( t ) are “born” (replicated), while those with f ( X t ) < λ ( t ) f(X_t) < \lambda(t) f ( X t ) < λ ( t ) “die” (are removed). The constraint λ ( t ) = ∫ R d f ( x ) ρ ~ ( t , x ) d x \lambda(t) = \int_{\mathbb{R}^d} f(x)\tilde{\rho}(t,x)\,dx λ ( t ) = ∫ R d f ( x ) ρ ~ ( t , x ) d x maintains normalization.
This interpretation underlies population dynamics or cloning algorithms for computing large deviations [28 , 29 ] . The algorithm maintains an ensemble of N c N_c N c trajectories (“clones”), each evolving independently according to the original dynamics. At regular intervals Δ t \Delta t Δ t , each clone at position x ( i ) x^{(i)} x ( i ) is assigned a weight w i = e f ( x ( i ) ) Δ t w_i = e^{f(x^{(i)})\Delta t} w i = e f ( x ( i ) ) Δ t . The ensemble is then resampled with replacement according to these weights: clones in regions where f ( x ) f(x) f ( x ) is large are duplicated, while those where f ( x ) f(x) f ( x ) is small are eliminated. The total weight W = ∑ i w i W = \sum_i w_i W = ∑ i w i fluctuates; its time-averaged growth rate converges to λ ∗ \lambda_* λ ∗ as N c → ∞ N_c \to\infty N c → ∞ and Δ t → 0 \Delta t \to0 Δ t → 0 . The resampled ensemble approximates the tilted distribution ρ ~ ∗ \tilde{\rho}_* ρ ~ ∗ , enabling direct measurement of observables conditioned on the rare event.
Combined small-noise and long-time limits
Consider now the combined limit T → ∞ T \to\infty T → ∞ and ε → 0 \varepsilon\to0 ε → 0 for the process
d X t = b ( X t ) d t + 2 ε σ ( X t ) d W t , (99) \mathrm{d}X_t = b(X_t)\,\mathrm{d}t + \sqrt{2\varepsilon}\,\sigma(X_t)\,\mathrm{d}W_t, \tag*{(99)} d X t = b ( X t ) d t + 2 ε σ ( X t ) d W t , (99)
with
I T = E [ e 1 ε ∫ 0 T f ( X t ) d t ] . (100) I_T = \mathbb{E}\left[e^{\frac{1}{\varepsilon}\int_0^T f(X_t)\,\mathrm{d}t}\right]. \tag*{(100)} I T = E [ e ε 1 ∫ 0 T f ( X t ) d t ] . (100)
The tilted eigenvalue problem becomes
− ∇ ⋅ ( b ρ ~ ∗ ) + ε D : ∇ ∇ ρ ~ ∗ + 1 ε f ( x ) ρ ~ ∗ = λ ∗ ε ρ ~ ∗ , (101) -\nabla\cdot(b\widetilde{\rho}_*) + \varepsilon D : \nabla\nabla\widetilde{\rho}_* + \frac{1}{\varepsilon} f(x)\widetilde{\rho}_* = \frac{\lambda_*}{\varepsilon}\widetilde{\rho}_*, \tag*{(101)} − ∇ ⋅ ( b ρ ∗ ) + ε D : ∇∇ ρ ∗ + ε 1 f ( x ) ρ ∗ = ε λ ∗ ρ ∗ , (101)
and the SCGF is
lim ε → 0 lim T → ∞ ε T log I T = λ ∗ . (102) \lim_{\varepsilon\to0}\lim_{T \to\infty}\frac{\varepsilon}{T}\log I_T = \lambda_*. \tag*{(102)} ε → 0 lim T → ∞ lim T ε log I T = λ ∗ . (102)
The WKB ansatz ρ ~ ∗ = e − ϕ ∗ / ε \widetilde{\rho}_* = e^{-\phi_*/\varepsilon} ρ ∗ = e − ϕ ∗ / ε yields the stationary Hamilton–Jacobi equation
H ( x , ∇ ϕ ∗ ) = λ ∗ , (103) H(x,\nabla\phi_*) = \lambda_*, \tag*{(103)} H ( x , ∇ ϕ ∗ ) = λ ∗ , (103)
with Hamiltonian H ( x , θ ) = b ( x ) ⋅ θ + θ ⋅ D ( x ) θ + f ( x ) H(x,\theta) = b(x)\cdot\theta+ \theta\cdot D(x)\theta+ f(x) H ( x , θ ) = b ( x ) ⋅ θ + θ ⋅ D ( x ) θ + f ( x ) .
Remark 5.2 (Non-commutativity of limits). We stress that the limits ε → 0 \varepsilon\to0 ε → 0 and T → ∞ T \to\infty T → ∞ of the scaled cumulant generating function
ε T log E [ e 1 ε ∫ 0 T f ( X t ) d t ] (104) \frac{\varepsilon}{T}\log\mathbb{E}\left[e^{\frac{1}{\varepsilon}\int_0^T f(X_t)\,\mathrm{d}t}\right] \tag*{(104)} T ε log E [ e ε 1 ∫ 0 T f ( X t ) d t ] (104)
do not generally commute. For systems with a unique attractor, the two limits typically agree. However, for multistable systems the order matters: taking T → ∞ T \to\infty T → ∞ first allows the system to explore transitions between metastable states (with rates exponentially small in 1 / ε 1/\varepsilon 1/ ε ), while taking ε → 0 \varepsilon\to0 ε → 0 first traps trajectories in a single basin, precluding such transitions. More precisely, the invariant measure at finite ε \varepsilon ε assigns weight to all attractors, but the ε → 0 \varepsilon\to0 ε → 0 limit of this measure depends on how ε \varepsilon ε and 1 / T 1/T 1/ T approach zero together [1 ] .
Separation of time scales
A powerful application of the long-time limit is to systems with well-separated time scales [30 ] . Consider
d X t ε = a ( X t ε , Y t ε ) d t , (105) \mathrm{d}X_t^\varepsilon= a(X_t^\varepsilon,Y_t^\varepsilon)\,\mathrm{d}t, \tag*{(105)} d X t ε = a ( X t ε , Y t ε ) d t , (105)
d Y t ε = 1 ε b ( X t ε , Y t ε ) d t + σ ε d W t , (106) \mathrm{d}Y_t^\varepsilon= \frac{1}{\varepsilon}b(X_t^\varepsilon,Y_t^\varepsilon)\,\mathrm{d}t + \frac{\sigma}{\sqrt{\varepsilon}}\,\mathrm{d}W_t, \tag*{(106)} d Y t ε = ε 1 b ( X t ε , Y t ε ) d t + ε σ d W t , (106)
where X t ε X_t^\varepsilon X t ε is a slow variable driven by the fast variable Y t ε Y_t^\varepsilon Y t ε , and for simplicity we take σ ∈ R \sigma\in\mathbb{R} σ ∈ R constant. The generator of this process is
L ε = a ( x , y ) ⋅ ∇ x + 1 ε ( b ( x , y ) ⋅ ∇ y + σ 2 2 Δ y ) . (107) L^\varepsilon= a(x,y)\cdot\nabla_x + \frac{1}{\varepsilon}\left(b(x,y)\cdot\nabla_y + \frac{\sigma^2}{2}\Delta_y\right). \tag*{(107)} L ε = a ( x , y ) ⋅ ∇ x + ε 1 ( b ( x , y ) ⋅ ∇ y + 2 σ 2 Δ y ) . (107)
We seek the LDP for E x , y [ e f ( X T ε ) / ε ] \mathbb{E}^{x,y}[e^{f(X_T^\varepsilon)/\varepsilon}] E x , y [ e f ( X T ε ) / ε ] . The corresponding backward Kolmogorov equation is ∂ t u ε + L ε u ε = 0 \partial_t u^\varepsilon+ L^\varepsilon u^\varepsilon= 0 ∂ t u ε + L ε u ε = 0 with terminal condition u ε ( T , x , y ) = e f ( x ) / ε u^\varepsilon(T,x,y) = e^{f(x)/\varepsilon} u ε ( T , x , y ) = e f ( x ) / ε . Since the terminal condition depends only on x x x , we use the two-scale WKB ansatz
u ε ( t , x , y ) = w ( t , x , y ) e u ( t , x ) / ε , (108) u^\varepsilon(t,x,y) = w(t,x,y)e^{u(t,x)/\varepsilon}, \tag*{(108)} u ε ( t , x , y ) = w ( t , x , y ) e u ( t , x ) / ε , (108)
where the exponent u u u depends only on the slow variable. Substituting into the BKE and collecting powers of ϵ \epsilon ϵ , we find at O ( ϵ − 1 ) O(\epsilon^{-1}) O ( ϵ − 1 ) :
a ( x , y ) ⋅ ( ∇ x u ) w + b ( x , y ) ⋅ ∇ y w + σ 2 2 Δ y w = − ( ∂ t u ) w . (109) a(x,y)\cdot(\nabla_x u)w+b(x,y)\cdot\nabla_y w+\frac{\sigma^2}{2}\Delta_y w=-(\partial_t u)w. \tag*{(109)} a ( x , y ) ⋅ ( ∇ x u ) w + b ( x , y ) ⋅ ∇ y w + 2 σ 2 Δ y w = − ( ∂ t u ) w . (109)
In equation (109) , the quantities x x x and ∇ x u \nabla_x u ∇ x u enter as parameters, and the equation determines w w w as a function of y y y . For a solution w ( y ) w(y) w ( y ) to exist, the remaining quantity ∂ t u \partial_t u ∂ t u cannot be arbitrary—there must be a solvability condition. The structure becomes transparent upon denoting θ = ∇ x u \theta=\nabla_xu θ = ∇ x u and defining the tilted generator
L θ w ≡ a ( x , y ) ⋅ θ w + b ( x , y ) ⋅ ∇ y w + σ 2 2 Δ y w , (110) \mathcal{L}_\theta w\equiv a(x,y)\cdot\theta w+b(x,y)\cdot\nabla_yw+\frac{\sigma^2}{2}\Delta_yw, \tag*{(110)} L θ w ≡ a ( x , y ) ⋅ θ w + b ( x , y ) ⋅ ∇ y w + 2 σ 2 Δ y w , (110)
so that equation (109) becomes the eigenvalue problem L θ w = λ w \mathcal{L}_\theta w=\lambda w L θ w = λ w with λ = − ∂ t u \lambda=-\partial_tu λ = − ∂ t u . That is, the solvability condition is that λ \lambda λ must belong to the spectrum of L θ \mathcal{L}_\theta L θ . For ergodic fast dynamics, the relevant eigenvalue is the principal one λ ( x , θ ) \lambda(x,\theta) λ ( x , θ ) —the eigenvalue with largest real part—which is real, simple, and has a positive eigenfunction w > 0 w>0 w > 0 . This positivity ensures the WKB ansatz u ϵ = w e u / ϵ u^\epsilon=we^{u/\epsilon} u ϵ = w e u / ϵ remains well-defined. The solvability condition − ∂ t u = λ ( x , ∇ x u ) -\partial_tu=\lambda(x,\nabla_xu) − ∂ t u = λ ( x , ∇ x u ) then yields the Hamilton–Jacobi equation for the slow variable:
∂ t u + H e f f ( x , ∇ x u ) = 0 , H e f f ( x , θ ) = λ ( x , θ ) . (111) \partial_tu+H_{\mathrm{eff}}(x,\nabla_xu)=0,\qquad H_{\mathrm{eff}}(x,\theta)=\lambda(x,\theta). \tag*{(111)} ∂ t u + H eff ( x , ∇ x u ) = 0 , H eff ( x , θ ) = λ ( x , θ ) . (111)
The interpretation is that the fast process generates an effective Hamiltonian for the slow variable: the principal eigenvalue λ ( x , θ ) \lambda(x,\theta) λ ( x , θ ) encodes a long-time average over the fast dynamics of Y Y Y , tilted by the momentum θ \theta θ . Large deviations of X X X then follow from a small-noise analysis governed by this effective Hamiltonian.
Example. Consider
d X t ϵ = ∣ Y t ϵ ∣ 2 d t − V ′ ( X t ϵ ) d t , (112) dX_t^\epsilon=|Y_t^\epsilon|^2\,dt-V'(X_t^\epsilon)\,dt, \tag*{(112)} d X t ϵ = ∣ Y t ϵ ∣ 2 d t − V ′ ( X t ϵ ) d t , (112)
d Y t ϵ = − γ ( X t ϵ ) ϵ Y t ϵ d t + σ ϵ d W t , (113) dY_t^\epsilon=-\frac{\gamma(X_t^\epsilon)}{\epsilon}Y_t^\epsilon\,dt+\frac{\sigma}{\sqrt{\epsilon}}\,dW_t, \tag*{(113)} d Y t ϵ = − ϵ γ ( X t ϵ ) Y t ϵ d t + ϵ σ d W t , (113)
where V ( x ) V(x) V ( x ) is some potential, γ ( x ) > 0 \gamma(x)>0 γ ( x ) > 0 , and σ ∈ R \sigma\in\mathbb{R} σ ∈ R . For fixed x x x , the fast variable Y Y Y is an Ornstein–Uhlenbeck process with stationary variance σ 2 / ( 2 γ ( x ) ) \sigma^2/(2\gamma(x)) σ 2 / ( 2 γ ( x )) .
For this system, a ( x , y ) = y 2 − V ′ ( x ) a(x,y)=y^2-V'(x) a ( x , y ) = y 2 − V ′ ( x ) and b ( x , y ) = − γ ( x ) y b(x,y)=-\gamma(x)y b ( x , y ) = − γ ( x ) y . The tilted generator (110) becomes
L θ w = ( y 2 − V ′ ( x ) ) θ w − γ ( x ) y ∂ y w + σ 2 2 ∂ y 2 w . (114) \mathcal{L}_\theta w=(y^2-V'(x))\theta w-\gamma(x)y\partial_yw+\frac{\sigma^2}{2}\partial_y^2w. \tag*{(114)} L θ w = ( y 2 − V ′ ( x )) θ w − γ ( x ) y ∂ y w + 2 σ 2 ∂ y 2 w . (114)
We seek solutions of L θ w = λ w \mathcal{L}_\theta w=\lambda w L θ w = λ w using the Gaussian ansatz w = e B y 2 / 2 w=e^{By^2/2} w = e B y 2 /2 . Substituting and using ∂ y w = B y w \partial_yw=Byw ∂ y w = B y w and ∂ y 2 w = ( B + B 2 y 2 ) w \partial_y^2w=(B+B^2y^2)w ∂ y 2 w = ( B + B 2 y 2 ) w :
( y 2 − V ′ ( x ) ) θ − γ ( x ) B y 2 + σ 2 2 ( B + B 2 y 2 ) = λ . (115) (y^2-V'(x))\theta-\gamma(x)By^2+\frac{\sigma^2}{2}(B+B^2y^2)=\lambda. \tag*{(115)} ( y 2 − V ′ ( x )) θ − γ ( x ) B y 2 + 2 σ 2 ( B + B 2 y 2 ) = λ . (115)
Collecting powers of y 2 y^2 y 2 :
y 2 : θ − γ ( x ) B + σ 2 2 B 2 = 0 , (116) y^2:\quad\theta-\gamma(x)B+\frac{\sigma^2}{2}B^2=0, \tag*{(116)} y 2 : θ − γ ( x ) B + 2 σ 2 B 2 = 0 , (116)
y 0 : − V ′ ( x ) θ + σ 2 2 B = λ . (117) y^0:\quad-V'(x)\theta+\frac{\sigma^2}{2}B=\lambda. \tag*{(117)} y 0 : − V ′ ( x ) θ + 2 σ 2 B = λ . (117)
The first equation determines B B B . Solving the quadratic:
B = γ ( x ) ± γ ( x ) 2 − 2 σ 2 θ σ 2 . (118) B = \frac{\gamma(x) \pm\sqrt{\gamma(x)^2 - 2\sigma^2\theta}}{\sigma^2}. \tag*{(118)} B = σ 2 γ ( x ) ± γ ( x ) 2 − 2 σ 2 θ . (118)
Since H e f f H_{\mathrm{eff}} H eff is a cumulant generating function of the fast dynamics, it must be convex in θ \theta θ by Jensen’s inequality, which selects the minus sign. Substituting into the second equation yields the principal eigenvalue:
H e f f ( x , θ ) = λ ( x , θ ) = − V ′ ( x ) θ + 1 2 [ γ ( x ) − γ ( x ) 2 − 2 σ 2 θ ] . (119) H_{\mathrm{eff}}(x,\theta) = \lambda(x,\theta) = -V'(x)\theta+ \frac{1}{2}\left[\gamma(x) - \sqrt{\gamma(x)^2 - 2\sigma^2\theta}\right]. \tag*{(119)} H eff ( x , θ ) = λ ( x , θ ) = − V ′ ( x ) θ + 2 1 [ γ ( x ) − γ ( x ) 2 − 2 σ 2 θ ] . (119)
At the LLN level, the deterministic dynamics is x ˙ = ∂ θ H e f f ∣ θ = 0 = − V ′ ( x ) + σ 2 / ( 2 γ ( x ) ) \dot{x} = \left.\partial_\theta H_{\mathrm{eff}}\right|_{\theta=0} = -V'(x) + \sigma^2/(2\gamma(x)) x ˙ = ∂ θ H eff ∣ θ = 0 = − V ′ ( x ) + σ 2 / ( 2 γ ( x )) , corresponding to a “thermal” correction from the fast fluctuations. For the full LDP, the non-quadratic structure of H e f f H_{\mathrm{eff}} H eff captures rare events where the slow variable is driven by atypical fluctuations of the fast variable.
Figure 7 illustrates a two-dimensional example where the optimal transition path differs significantly from a simple time-reversal of the relaxation path, with the noise-driven instanton achieving only about 20% of the action of the naive reversed path.
Figure 7. Optimal paths in a two-dimensional system with time-scale separation. (a) Phase portrait showing fixed points (dots) and the flow under the effective LLN dynamics. Solid curve: minimum-action path (instanton) between fixed points. Dotted curve: time-reversed relaxation path. Dashed curve: quadratic approximation to the Hamiltonian. (b) Cumulative action along each path. The instanton achieves approximately 20% of the action required by the time-reversed path. Reproduced from [22 ] .
Computational methods
The variational problems arising from large deviation theory require numerical solution for all but the simplest models. This section surveys computational approaches, from classical minimum action methods to modern machine-learning techniques and sampling algorithms. Key references include [31 ] for the minimum action method, [32 ] for its geometric version, and [33 , 34 ] for the string method.
Minimum action methods
The computation of instantons and quasipotentials reduces to finding paths that extremize the action functional (8) . Two equivalent formulations are available: the Lagrangian formulation, which directly minimizes the action
S T [ x ] = ∫ 0 T L ( x t , x ˙ t ) d t , (120) S_T[\mathbf{x}] = \int_0^T L(x_t,\dot{x}_t)\,\mathrm{d}t, \tag*{(120)} S T [ x ] = ∫ 0 T L ( x t , x ˙ t ) d t , (120)
and the Hamiltonian formulation, which seeks saddle points of
S T [ x , θ ] = ∫ 0 T [ x ˙ t ⋅ θ t − H ( x t , θ t ) ] d t . (121) S_T[\mathbf{x},\theta] = \int_0^T [\dot{x}_t\cdot\theta_t-H(x_t,\theta_t)]\,\mathrm{d}t. \tag*{(121)} S T [ x , θ ] = ∫ 0 T [ x ˙ t ⋅ θ t − H ( x t , θ t )] d t . (121)
Since H H H is convex in θ \theta θ , this is a min-max problem: minimization over x x x and maximization over θ \theta θ . Maximizing over θ \theta θ at fixed x x x recovers the Lagrangian via Legendre transform, L ( x , x ˙ ) = max θ [ x ˙ ⋅ θ − H ( x , θ ) ] L(x,\dot{x})=\max_{\theta}[\dot{x}\cdot\theta-H(x,\theta)] L ( x , x ˙ ) = max θ [ x ˙ ⋅ θ − H ( x , θ )] . The Lagrangian formulation requires explicitly computing this transform, which assumes invertibility of the diffusion matrix D D D for SDEs. The Hamiltonian formulation works directly with H H H , which is available from the generator without inversion, making it preferable when the diffusion is degenerate.
The minimum action method (MAM)
The minimum action method [31 ] discretizes the path x t x_t x t as { x 0 , x 1 , … , x N } \{x_0,x_1,\ldots,x_N\} { x 0 , x 1 , … , x N } at times t k = k Δ t t_k=k\Delta t t k = k Δ t , with Δ t = T / N \Delta t=T/N Δ t = T / N . The action is approximated by
S N = ∑ k = 0 N − 1 L ( x k + x k + 1 2 , x k + 1 − x k Δ t ) Δ t , (122) S_N=\sum_{k=0}^{N-1}L\left(\frac{x_k+x_{k+1}}{2},\frac{x_{k+1}-x_k}{\Delta t}\right)\Delta t, \tag*{(122)} S N = k = 0 ∑ N − 1 L ( 2 x k + x k + 1 , Δ t x k + 1 − x k ) Δ t , (122)
with boundary conditions x 0 = x a x_0=x_a x 0 = x a and x N = x b x_N=x_b x N = x b held fixed. The discretized action S N S_N S N is minimized over the interior points { x 1 , … , x N − 1 } \{x_1,\ldots,x_{N-1}\} { x 1 , … , x N − 1 } by gradient descent in a fictitious time τ \tau τ :
∂ x k ∂ τ = − δ S N δ x k . (123) \frac{\partial x_k}{\partial\tau}=-\frac{\delta S_N}{\delta x_k}. \tag*{(123)} ∂ τ ∂ x k = − δ x k δ S N . (123)
As τ → ∞ \tau\to\infty τ → ∞ , the path converges to a solution of the Euler–Lagrange equations. In practice, quasi-Newton methods such as L-BFGS significantly accelerate convergence over simple gradient descent.
When the diffusion matrix D D D is not invertible and the Lagrangian cannot be computed explicitly, one works instead with the Hamiltonian formulation (121) ; see e.g. [22 , 35 ] . Both x x x and θ \theta θ are discretized, and the saddle point is found by simultaneous gradient descent in x x x and ascent in θ \theta θ . This gradient descent-ascent approach is developed systematically in § 6.1.2 .
The MAM is straightforward to implement for fixed transition time T T T , but cannot directly handle the limit T → ∞ T\to\infty T → ∞ relevant for quasipotential calculations, since discretization points cluster near fixed points where the velocity vanishes. The geometric methods of § 6.1.3 and § 6.1.4 address this limitation.
Gradient descent-ascent for the min-max problem
A robust approach to solving the min-max problem (121) directly in the Hamiltonian formulation is gradient descent-ascent (GDA) [22 ] . The problem is
inf x S T [ x ] = min x max θ ∫ 0 T [ x ˙ t ⋅ θ t − H ( x t , θ t ) ] d t , (124) \inf_x S_T[x]=\min_x\max_\theta\int_0^T[\dot{x}_t\cdot\theta_t-H(x_t,\theta_t)]\,\mathrm{d}t, \tag*{(124)} x inf S T [ x ] = x min θ max ∫ 0 T [ x ˙ t ⋅ θ t − H ( x t , θ t )] d t , (124)
where the optimization is over trajectories ( x t , θ t ) t ∈ [ 0 , T ] (x_t,\theta_t)_{t\in[0,T]} ( x t , θ t ) t ∈ [ 0 , T ] subject to boundary conditions x 0 = x a x_0=x_a x 0 = x a and x T = x b x_T=x_b x T = x b . The functional is strictly concave in θ \theta θ (by convexity of H H H ), so the min-max can be solved by alternating infinitesimal steps of steepest descent in x x x and ascent in θ \theta θ . This leads to the GDA evolution equations in a fictitious optimization time τ \tau τ :
∂ τ x = α ( ∂ t θ + ∂ x H ) , α ∂ τ θ = ∂ t x − ∂ θ H , (125) \partial_{\tau} x = \alpha(\partial_t \theta+ \partial_x H), \qquad\alpha\partial_{\tau}\theta= \partial_t x - \partial_{\theta} H, \tag*{(125)} ∂ τ x = α ( ∂ t θ + ∂ x H ) , α ∂ τ θ = ∂ t x − ∂ θ H , (125)
where α > 0 \alpha> 0 α > 0 is a parameter controlling the relative timescales. The fixed points of this system satisfy Hamilton’s equations (17) , but the boundary conditions x t = 0 = x a x_{t=0}=x_a x t = 0 = x a and x t = T = x b x_{t=T}=x_b x t = T = x b must be imposed to select the instanton connecting the two states. This is again a two-point boundary value problem, but now with position specified at both endpoints rather than the mixed conditions (18) arising in the LDP; the momentum θ \theta θ is determined implicitly.
Since the system (125) is hyperbolic, and for such equations boundary conditions can only be imposed where characteristics enter the domain, we introduce the characteristic variables (not to be confused with the WKB ansatz of § 2.2 )
u = x + α θ , v = x − α θ , (126) u = x + \alpha\theta, \qquad v = x - \alpha\theta, \tag*{(126)} u = x + α θ , v = x − α θ , (126)
which satisfy
∂ τ u = ∂ t u + f ( u , v ) , (127) \partial_{\tau}u = \partial_tu + f(u,v), \tag*{(127)} ∂ τ u = ∂ t u + f ( u , v ) , (127)
∂ τ v = − ∂ t v + g ( u , v ) , (128) \partial_{\tau}v = -\partial_tv + g(u,v), \tag*{(128)} ∂ τ v = − ∂ t v + g ( u , v ) , (128)
where
f ( u , v ) = α ∂ x H − ∂ θ H , g ( u , v ) = α ∂ x H + ∂ θ H . (129) f(u,v) = \alpha\partial_xH - \partial_{\theta}H, \qquad g(u,v) = \alpha\partial_xH + \partial_{\theta}H. \tag*{(129)} f ( u , v ) = α ∂ x H − ∂ θ H , g ( u , v ) = α ∂ x H + ∂ θ H . (129)
The variable u u u propagates backward in physical time t t t (characteristics travel from t = T t = T t = T to t = 0 t = 0 t = 0 ), while v v v propagates forward (characteristics travel from t = 0 t = 0 t = 0 to t = T t = T t = T ). Thus boundary conditions are naturally imposed at the upstream end of each characteristic: u u u at t = T t = T t = T and v v v at t = 0 t = 0 t = 0 . Using x = ( u + v ) / 2 x = (u + v)/2 x = ( u + v ) /2 and θ = ( u − v ) / ( 2 α ) \theta= (u - v)/(2\alpha) θ = ( u − v ) / ( 2 α ) , the constraints x t = 0 = x a x_{t=0}=x_a x t = 0 = x a and x t = T = x b x_{t=T}=x_b x t = T = x b become
v ( τ , t = 0 ) = 2 x a − u ( τ , t = 0 ) , u ( τ , t = T ) = 2 x b − v ( τ , t = T ) . (130) v(\tau, t = 0) = 2x_a - u(\tau, t = 0), \qquad u(\tau, t = T) = 2x_b - v(\tau, t = T). \tag*{(130)} v ( τ , t = 0 ) = 2 x a − u ( τ , t = 0 ) , u ( τ , t = T ) = 2 x b − v ( τ , t = T ) . (130)
This suggests a natural algorithm based on Strang splitting: at each optimization step, first update u u u by propagating from t = T t = T t = T to t = 0 t = 0 t = 0 , then update v v v by propagating from t = 0 t = 0 t = 0 to t = T t = T t = T .
Algorithm 1 gives the complete discretized scheme. The path is discretized on M + 1 M + 1 M + 1 points with Δ t = T / M \Delta t = T/M Δ t = T / M , and an implicit upwind scheme ensures numerical stability.
Algorithm 1 Action minimization by gradient descent-ascent
Require: Number of points M M M ; initial path { x i 0 } i ∈ I \{x_i^0\}_{i\in\mathcal{I}} { x i 0 } i ∈ I with x 0 0 = x a x_0^0=x_a x 0 0 = x a and x M 0 = x b x_M^0=x_b x M 0 = x b ; functions f ( u , v ) f(u,v) f ( u , v ) and g ( u , v ) g(u,v) g ( u , v ) ; final time T > 0 T>0 T > 0 ; step sizes Δ τ > 0 \Delta\tau>0 Δ τ > 0 , α > 0 \alpha>0 α > 0 .
Initialize: For all i ∈ I = { 0 , … , M } i\in\mathcal{I}=\{0,\ldots,M\} i ∈ I = { 0 , … , M } , set θ i 0 = 0 \theta_i^0=0 θ i 0 = 0 , u i 0 = x i 0 + α θ i 0 u_i^0=x_i^0+\alpha\theta_i^0 u i 0 = x i 0 + α θ i 0 , v i 0 = x i 0 − α θ i 0 v_i^0=x_i^0-\alpha\theta_i^0 v i 0 = x i 0 − α θ i 0 ; set Δ t = T / M \Delta t=T/M Δ t = T / M .
for n ≥ 0 n\geq0 n ≥ 0 do
Update u u u (backward sweep): Solve { u i n + 1 } i ∈ I \{u_i^{n+1}\}_{i\in\mathcal{I}} { u i n + 1 } i ∈ I sequentially from i = M i=M i = M to i = 0 i=0 i = 0 :
u M n + 1 = − v M n + 2 x b , u_M^{n+1}=-v_M^n+2x_b, u M n + 1 = − v M n + 2 x b ,
u i n + 1 − u i n Δ τ = u i + 1 n + 1 − u i n + 1 Δ t + f ( u i + 1 n , v i + 1 n ) , i = M − 1 , … , 0. \frac{u_i^{n+1}-u_i^n}{\Delta\tau}=\frac{u_{i+1}^{n+1}-u_i^{n+1}}{\Delta t}+f\left(u_{i+1}^n,v_{i+1}^n\right),\quad i=M-1,\ldots,0. Δ τ u i n + 1 − u i n = Δ t u i + 1 n + 1 − u i n + 1 + f ( u i + 1 n , v i + 1 n ) , i = M − 1 , … , 0.
Update v v v (forward sweep): Solve { v i n + 1 } i ∈ I \{v_i^{n+1}\}_{i\in\mathcal{I}} { v i n + 1 } i ∈ I sequentially from i = 0 i=0 i = 0 to i = M i=M i = M :
v 0 n + 1 = − u 0 n + 1 + 2 x a , v_0^{n+1}=-u_0^{n+1}+2x_a, v 0 n + 1 = − u 0 n + 1 + 2 x a ,
v i n + 1 − v i n Δ τ = v i n + 1 − v i − 1 n + 1 Δ t + g ( u i − 1 n + 1 , v i − 1 n ) , i = 1 , … , M . \frac{v_i^{n+1}-v_i^n}{\Delta\tau}=\frac{v_i^{n+1}-v_{i-1}^{n+1}}{\Delta t}+g\left(u_{i-1}^{n+1},v_{i-1}^n\right),\quad i=1,\ldots,M. Δ τ v i n + 1 − v i n = Δ t v i n + 1 − v i − 1 n + 1 + g ( u i − 1 n + 1 , v i − 1 n ) , i = 1 , … , M .
Recover fields: Compute x i n + 1 = 1 2 ( u i n + 1 + v i n + 1 ) x_i^{n+1}=\frac{1}{2}\left(u_i^{n+1}+v_i^{n+1}\right) x i n + 1 = 2 1 ( u i n + 1 + v i n + 1 ) and θ i n + 1 = 1 2 α ( u i n + 1 − v i n + 1 ) \theta_i^{n+1}=\frac{1}{2\alpha}\left(u_i^{n+1}-v_i^{n+1}\right) θ i n + 1 = 2 α 1 ( u i n + 1 − v i n + 1 ) .
end for
The implicit upwind discretization is unconditionally stable and allows large time steps Δ τ \Delta\tau Δ τ , though the physical time step Δ t \Delta t Δ t must still resolve the instanton dynamics. The reaction terms f f f and g g g are evaluated at upwind grid points, which improves stability for spatially extended fields.
The geometric minimum action method (gMAM)
For quasipotential calculations, optimal paths connect fixed points and have infinite duration (T → ∞ T\to\infty T → ∞ ). The geometric minimum action method [32 ] addresses this by exploiting the Maupertuis principle (68) : for paths on the zero-energy surface H ( x t , θ t ) = 0 H(x_t,\theta_t)=0 H ( x t , θ t ) = 0 , the action becomes the line integral
S [ x ] = ∫ γ θ ⋅ d x , (131) S[x]=\int_{\gamma}\theta\cdot dx, \tag*{(131)} S [ x ] = ∫ γ θ ⋅ d x , (131)
which depends only on the geometric curve γ \gamma γ , not on its time parametrization. This reparametrization invariance allows treating infinite time intervals with finitely many discretization points.
The path is parametrized by normalized arc-length s ∈ [ 0 , 1 ] s\in[0,1] s ∈ [ 0 , 1 ] with constant speed ∥ x ′ ( s ) ∥ D = L \lVert x'(s)\rVert_D=L ∥ x ′ ( s ) ∥ D = L , where the prime denotes differentiation with respect to s s s and ∥ v ∥ D 2 = v ⋅ D − 1 v \lVert v\rVert_D^2=v\cdot D^{-1}v ∥ v ∥ D 2 = v ⋅ D − 1 v is the norm induced by the diffusion matrix. The constraint H ( x , θ ) = 0 H(x,\theta)=0 H ( x , θ ) = 0 determines θ \theta θ as a function of ( x , x ′ ) (x,x') ( x , x ′ ) :
x ′ = μ ∇ θ H ( x , θ ) , (132) x'=\mu\nabla_{\theta}H(x,\theta), \tag*{(132)} x ′ = μ ∇ θ H ( x , θ ) , (132)
where μ ( s ) = d t / d s \mu(s)=dt/ds μ ( s ) = d t / d s is the time reparametrization factor. Enforcing the arc-length constraint gives
μ = ∥ x ′ ∥ D 2 ⟨ x ′ , ∇ θ H ⟩ D , (133) \mu=\frac{\lVert x'\rVert_D^2}{\langle x',\nabla_{\theta}H\rangle_D}, \tag*{(133)} μ = ⟨ x ′ , ∇ θ H ⟩ D ∥ x ′ ∥ D 2 , (133)
where ⟨ u , v ⟩ D = u ⋅ D − 1 v \langle u,v\rangle_D=u\cdot D^{-1}v ⟨ u , v ⟩ D = u ⋅ D − 1 v .
The gMAM algorithm iteratively updates the path:
Solve H ( x k , θ k ) = 0 H(x^{k},\theta^{k})=0 H ( x k , θ k ) = 0 for θ k \theta^{k} θ k given x k x^{k} x k ;
Compute the descent direction d k = θ k + μ k ∇ x H ( x k , θ k ) d^{k}=\theta^{k}+\mu^{k}\nabla_{x}H(x^{k},\theta^{k}) d k = θ k + μ k ∇ x H ( x k , θ k ) ;
Update x k + 1 = x k + Δ τ ( μ k ) − 1 d k x^{k+1}=x^{k}+\Delta\tau(\mu^{k})^{-1}d^{k} x k + 1 = x k + Δ τ ( μ k ) − 1 d k ;
Redistribute points to maintain equal arc-length spacing.
The redistribution step prevents clustering of discretization points near fixed points where the velocity vanishes. The preconditioning by μ − 1 \mu^{-1} μ − 1 ensures proper convergence where μ → ∞ \mu\to\infty μ → ∞ as ∥ b ( x ) ∥ → 0 \lVert b(x)\rVert\to0 ∥ b ( x )∥ → 0 .
For further analytical and numerical approaches to computing the quasipotential in non-gradient systems, see [21 , 36 , 37 ] .
Geometric GDA for quasipotential calculations
The GDA approach of Section § 6.1.2 can be extended to compute the quasipotential by incorporating the geometric formulation [22 ] . The problem becomes
V x a ( x b ) = inf T > 0 inf x S T [ x ] = min λ ≥ 0 min x ^ max θ ^ ∫ 0 1 [ x ^ ′ ⋅ θ ^ − λ − 1 H ( x ^ , θ ^ ) ] d s , (134) V_{x_a}(x_b)=\inf_{T>0}\inf_{x}S_T[x]=\min_{\lambda\ge0}\min_{\hat{x}}\max_{\hat{\theta}}\int_{0}^{1}\left[\hat{x}'\cdot\hat{\theta}-\lambda^{-1}H(\hat{x},\hat{\theta})\right]\,ds, \tag*{(134)} V x a ( x b ) = T > 0 inf x inf S T [ x ] = λ ≥ 0 min x ^ min θ ^ max ∫ 0 1 [ x ^ ′ ⋅ θ ^ − λ − 1 H ( x ^ , θ ^ ) ] d s , (134)
where we reparametrize by s ∈ [ 0 , 1 ] s\in[0,1] s ∈ [ 0 , 1 ] with d t / d s = λ − 1 ( s ) dt/ds=\lambda^{-1}(s) d t / d s = λ − 1 ( s ) , and x ^ ( s ) = x ( t ( s ) ) \hat{x}(s)=x(t(s)) x ^ ( s ) = x ( t ( s )) , θ ^ ( s ) = θ ( t ( s ) ) \hat{\theta}(s)=\theta(t(s)) θ ^ ( s ) = θ ( t ( s )) . By imposing ∣ x ^ ′ ∣ = c o n s t |\hat{x}'|=\mathrm{const} ∣ x ^ ′ ∣ = const (normalized arc-length), the discretization points remain uniformly distributed along the path regardless of physical time.
The GDA equations for the geometric formulation are
∂ τ x ^ = α λ ∂ s θ ^ + α ∂ x ^ H , α ∂ τ θ ^ = λ ∂ s x ^ − ∂ θ ^ H , (135) \partial_{\tau}\hat{x}=\alpha\lambda\partial_{s}\hat{\theta}+\alpha\partial_{\hat{x}}H,\qquad\alpha\partial_{\tau}\hat{\theta}=\lambda\partial_{s}\hat{x}-\partial_{\hat{\theta}}H, \tag*{(135)} ∂ τ x ^ = α λ ∂ s θ ^ + α ∂ x ^ H , α ∂ τ θ ^ = λ ∂ s x ^ − ∂ θ ^ H , (135)
where λ \lambda λ is determined by the constraint that the path lies on the zero-energy surface H = 0 H=0 H = 0 at convergence:
λ = ∣ ⟨ ∂ θ H , ∂ s x ^ ⟩ ∣ + max ( 0 , ∣ ⟨ ∂ θ H , ∂ s x ^ ⟩ ∣ 2 − 4 H ∣ ∂ s x ^ ∣ 2 ) 2 ∣ ∂ s x ^ ∣ 2 . (136) \lambda=\frac{\left|\langle\partial_{\theta}H,\partial_{s}\hat{x}\rangle\right|+\sqrt{\max\left(0,\left|\langle\partial_{\theta}H,\partial_{s}\hat{x}\rangle\right|^{2}-4H|\partial_{s}\hat{x}|^{2}\right)}}{2|\partial_{s}\hat{x}|^{2}}. \tag*{(136)} λ = 2∣ ∂ s x ^ ∣ 2 ∣ ⟨ ∂ θ H , ∂ s x ^ ⟩ ∣ + max ( 0 , ∣ ⟨ ∂ θ H , ∂ s x ^ ⟩ ∣ 2 − 4 H ∣ ∂ s x ^ ∣ 2 ) . (136)
At convergence, when H ( x ^ , θ ^ ) = 0 H(\hat{x},\hat{\theta})=0 H ( x ^ , θ ^ ) = 0 , the geometric Hamilton equations are satisfied:
λ x ^ ′ = ∂ θ H , λ θ ^ ′ = − ∂ x H . (137) \lambda\hat{x}'=\partial_{\theta}H,\qquad\lambda\hat{\theta}'=-\partial_{x}H. \tag*{(137)} λ x ^ ′ = ∂ θ H , λ θ ^ ′ = − ∂ x H . (137)
Algorithm 2 gives the complete scheme for computing the quasipotential. The key modifications compared to Algorithm 1 are the inclusion of the reparametrization coefficient λ \lambda λ and the redistribution step.
Algorithm 2 Geometric action minimization by gradient descent-ascent
Require: Number of points M M M ; stable fixed points x a x_a x a and x b x_b x b of the noiseless dynamics where H ( x a , b , 0 ) = ∂ θ H ( x a , b , 0 ) = 0 H(x_{a,b},0)=\partial_\theta H(x_{a,b},0)=0 H ( x a , b , 0 ) = ∂ θ H ( x a , b , 0 ) = 0 ; initial path { x ^ i 0 } i ∈ I \{\hat{x}_i^0\}_{i\in\mathcal{I}} { x ^ i 0 } i ∈ I with x ^ 0 0 = x a \hat{x}_0^0=x_a x ^ 0 0 = x a , x ^ M 0 = x b \hat{x}_M^0=x_b x ^ M 0 = x b , and ∣ x ^ i + 1 0 − x ^ i 0 ∣ \lvert\hat{x}_{i+1}^0-\hat{x}_i^0\rvert ∣ x ^ i + 1 0 − x ^ i 0 ∣ constant in i i i ; functions f ( u , v ) f(u,v) f ( u , v ) , g ( u , v ) g(u,v) g ( u , v ) , and λ ( u , v ) \lambda(u,v) λ ( u , v ) ; step sizes Δ τ > 0 \Delta\tau>0 Δ τ > 0 , α > 0 \alpha>0 α > 0 .
Initialize: For all i ∈ I = { 0 , … , M } i\in\mathcal{I}=\{0,\ldots,M\} i ∈ I = { 0 , … , M } , set θ ^ i 0 = 0 \hat{\theta}_i^0=0 θ ^ i 0 = 0 , u i 0 = x ^ i 0 + α θ ^ i 0 u_i^0=\hat{x}_i^0+\alpha\hat{\theta}_i^0 u i 0 = x ^ i 0 + α θ ^ i 0 , v i 0 = x ^ i 0 − α θ ^ i 0 v_i^0=\hat{x}_i^0-\alpha\hat{\theta}_i^0 v i 0 = x ^ i 0 − α θ ^ i 0 ; set Δ s = 1 / M \Delta s=1/M Δ s = 1/ M .
for n ≥ 0 n\ge0 n ≥ 0 do
Update u u u (backward sweep): Solve { u i n + 1 } i ∈ I \{u_i^{n+1}\}_{i\in\mathcal{I}} { u i n + 1 } i ∈ I sequentially from i = M i=M i = M to i = 0 i=0 i = 0 :
u M n + 1 = − v M n + 2 x b , u_M^{n+1}=-v_M^n+2x_b, u M n + 1 = − v M n + 2 x b ,
u i n + 1 − u i n Δ τ = λ i ( u i n , v i n ) u i + 1 n + 1 − u i n + 1 Δ s + f ( u i n , v i n ) , i = M − 1 , … , 0. \frac{u_i^{n+1}-u_i^n}{\Delta\tau}
=\lambda_i(u_i^n,v_i^n)\frac{u_{i+1}^{n+1}-u_i^{n+1}}{\Delta s}+f(u_i^n,v_i^n),\qquad i=M-1,\ldots,0. Δ τ u i n + 1 − u i n = λ i ( u i n , v i n ) Δ s u i + 1 n + 1 − u i n + 1 + f ( u i n , v i n ) , i = M − 1 , … , 0.
Update v v v (forward sweep): Solve { v i n + 1 } i ∈ I \{v_i^{n+1}\}_{i\in\mathcal{I}} { v i n + 1 } i ∈ I sequentially from i = 0 i=0 i = 0 to i = M i=M i = M :
v 0 n + 1 = − u 0 n + 1 + 2 x a , v_0^{n+1}=-u_0^{n+1}+2x_a, v 0 n + 1 = − u 0 n + 1 + 2 x a ,
v i n + 1 − v i n Δ τ = − λ i ( u i n + 1 , v i n ) v i n + 1 − v i − 1 n + 1 Δ s + g ( u i n + 1 , v i n ) , i = 1 , … , M . \frac{v_i^{n+1}-v_i^n}{\Delta\tau}
=-\lambda_i(u_i^{n+1},v_i^n)\frac{v_i^{n+1}-v_{i-1}^{n+1}}{\Delta s}+g(u_i^{n+1},v_i^n),\qquad i=1,\ldots,M. Δ τ v i n + 1 − v i n = − λ i ( u i n + 1 , v i n ) Δ s v i n + 1 − v i − 1 n + 1 + g ( u i n + 1 , v i n ) , i = 1 , … , M .
Recover fields: Compute x ˉ i n + 1 = 1 2 ( u i n + 1 + v i n + 1 ) \bar{x}_i^{n+1}=\frac{1}{2}(u_i^{n+1}+v_i^{n+1}) x ˉ i n + 1 = 2 1 ( u i n + 1 + v i n + 1 ) and θ ˉ i n + 1 = 1 2 α ( u i n + 1 − v i n + 1 ) \bar{\theta}_i^{n+1}=\frac{1}{2\alpha}(u_i^{n+1}-v_i^{n+1}) θ ˉ i n + 1 = 2 α 1 ( u i n + 1 − v i n + 1 ) .
Redistribute: Interpolate { ( x ˉ i n + 1 , θ ˉ i n + 1 ) } i ∈ I \{(\bar{x}_i^{n+1},\bar{\theta}_i^{n+1})\}_{i\in\mathcal{I}} {( x ˉ i n + 1 , θ ˉ i n + 1 ) } i ∈ I onto a path { ( x ^ i n + 1 , θ ^ i n + 1 ) } i ∈ I \{(\hat{x}_i^{n+1},\hat{\theta}_i^{n+1})\}_{i\in\mathcal{I}} {( x ^ i n + 1 , θ ^ i n + 1 ) } i ∈ I such that ∣ x ^ i + 1 n + 1 − x ^ i n + 1 ∣ \lvert\hat{x}_{i+1}^{n+1}-\hat{x}_i^{n+1}\rvert ∣ x ^ i + 1 n + 1 − x ^ i n + 1 ∣ is constant in i i i (as in the string method).
Update characteristic variables: Set u i n + 1 = x ^ i n + 1 + α θ ^ i n + 1 u_i^{n+1}=\hat{x}_i^{n+1}+\alpha\hat{\theta}_i^{n+1} u i n + 1 = x ^ i n + 1 + α θ ^ i n + 1 and v i n + 1 = x ^ i n + 1 − α θ ^ i n + 1 v_i^{n+1}=\hat{x}_i^{n+1}-\alpha\hat{\theta}_i^{n+1} v i n + 1 = x ^ i n + 1 − α θ ^ i n + 1 .
end for
The redistribution step (line 6) prevents clustering of discretization points near fixed points where the velocity vanishes, while the coefficient λ \lambda λ from (136) automatically handles the transition from finite to infinite physical time by adjusting the speed along the path.
Both GDA algorithms require only evaluating the Hamiltonian H ( x , θ ) H(x,\theta) H ( x , θ ) and its first derivatives ∂ x H \partial_xH ∂ x H and ∂ θ H \partial_\theta H ∂ θ H —the same information needed to solve Hamilton’s equations directly. This makes them applicable to systems with non-Gaussian noise (such as chemical reaction networks with Poisson fluctuations) where the Lagrangian is not available in closed form.
The string method for gradient systems
For gradient systems satisfying detailed balance (§ 4.5 ), the optimal fluctuation path is the time-reversal of the deterministic relaxation path. As a result, instantons are heteroclinic orbits of the gradient flow x ˙ = ± ∇ U ( x ) \dot{x}=\pm\nabla U(x) x ˙ = ± ∇ U ( x ) , connecting two minima through the lowest saddle point. The string method [33 , 34 ] exploits this simplification.
In continuous time, the string evolves according to
∂ τ x = − ∇ U ( x ) + ν t ^ , (138) \partial_\tau x=-\nabla U(x)+\nu\hat{t}, \tag*{(138)} ∂ τ x = − ∇ U ( x ) + ν t ^ , (138)
where τ \tau τ is a fictitious relaxation time, t ^ = x ′ / ∥ x ′ ∥ \hat{t}=x'/\lVert x'\rVert t ^ = x ′ / ∥ x ′ ∥ is the unit tangent vector, and ν \nu ν is a Lagrange multiplier enforcing the parametrization constraint. The algorithm iteratively updates the path:
Evolve each image: x ~ k + 1 = x k − Δ τ ∇ U ( x k ) \tilde{x}^{k+1}=x^k-\Delta\tau\nabla U(x^k) x ~ k + 1 = x k − Δ τ ∇ U ( x k ) ;
Redistribute points along the updated curve to maintain equal arc-length spacing.
The redistribution step implicitly enforces the constraint without explicitly computing ν \nu ν . While restricted to gradient systems, the string method is computationally simpler than the gMAM and is widely used in chemical physics for computing minimum energy paths and transition states.
Neural network approaches
The methods described above require discretizing the path in time. For spatially extended systems governed by stochastic PDEs, the path must also be discretized in space, leading to high-dimensional optimization problems that are not only computationally costly but also technically challenging to implement: the underlying PDEs are typically stiff, requiring implicit time-stepping and careful treatment of boundary conditions. Neural network methods offer a compelling alternative by parametrizing the solution with neural networks and leveraging automatic differentiation for gradient computation [22 , 38 ] .
Deep MAM and deep gMAM
Deep MAM and deep gMAM [22 , 38 ] adopt a physics-informed neural network (PINN) approach [39 ] : rather than discretizing the path on a mesh, they parameterize it by a neural network ansatz satisfying the boundary conditions. For a transition from state x a x_a x a to state x b x_b x b , one constructs
x t ( ω ) = ( 1 − t / T ) x a + ( t / T ) x b + ( t / T ) ( 1 − t / T ) N ω ( t ) , (139) x_t(\omega) = (1-t/T)x_a + (t/T)x_b + (t/T)(1-t/T)\mathcal{N}_{\omega}(t), \tag*{(139)} x t ( ω ) = ( 1 − t / T ) x a + ( t / T ) x b + ( t / T ) ( 1 − t / T ) N ω ( t ) , (139)
where N ω \mathcal{N}_{\omega} N ω is a neural network with parameters ω \omega ω . This ansatz automatically satisfies x 0 ( ω ) = x a x_0(\omega)=x_a x 0 ( ω ) = x a and x T ( ω ) = x b x_T(\omega)=x_b x T ( ω ) = x b . For field theories, x t x_t x t , x a x_a x a , x b x_b x b , and N ω \mathcal{N}_{\omega} N ω are all functions of the spatial variable.
The action S [ x ( ω ) ] S[x(\omega)] S [ x ( ω )] becomes a loss function over the network parameters, minimized by stochastic gradient descent. Algorithm 3 summarizes the procedure for finite transition time T T T .
Algorithm 3 Deep MAM: Action minimization via neural network
Require: Boundary states x a x_a x a , x b x_b x b ; final time T > 0 T>0 T > 0 ; neural network N ω \mathcal{N}_{\omega} N ω with initial parameters ω 0 \omega^0 ω 0 ; number of collocation points N c N_c N c ; learning rate η \eta η ; number of iterations N i t e r N_{\mathrm{iter}} N iter .
Define path ansatz: x t ( ω ) = ( 1 − t / T ) x a + ( t / T ) x b + ( t / T ) ( 1 − t / T ) N ω ( t ) x_t(\omega)=(1-t/T)x_a+(t/T)x_b+(t/T)(1-t/T)\mathcal{N}_{\omega}(t) x t ( ω ) = ( 1 − t / T ) x a + ( t / T ) x b + ( t / T ) ( 1 − t / T ) N ω ( t )
for n = 0 , … , N i t e r − 1 n=0,\ldots,N_{\mathrm{iter}}-1 n = 0 , … , N iter − 1 do
Sample collocation points: Draw { t j } j = 1 N c \{t_j\}_{j=1}^{N_c} { t j } j = 1 N c uniformly from [ 0 , T ] [0,T] [ 0 , T ]
Evaluate loss: L ( ω n ) = T N c ∑ j = 1 N c L ( x t j ( ω n ) , x ˙ t j ( ω n ) ) \mathcal{L}(\omega^n)=\frac{T}{N_c}\sum_{j=1}^{N_c}\mathcal{L}(x_{t_j}(\omega^n),\dot{x}_{t_j}(\omega^n)) L ( ω n ) = N c T ∑ j = 1 N c L ( x t j ( ω n ) , x ˙ t j ( ω n ))
Update parameters: ω n + 1 = ω n − η ∇ ω L ( ω n ) \omega^{n+1}=\omega^n-\eta\nabla_{\omega}\mathcal{L}(\omega^n) ω n + 1 = ω n − η ∇ ω L ( ω n )
end for
return Optimized path x t ( ω N i t e r ) x_t(\omega^{N_{\mathrm{iter}}}) x t ( ω N iter )
The gradients ∇ ω L \nabla_{\omega}\mathcal{L} ∇ ω L are computed via automatic differentiation, which also provides x ˙ t = ∂ t x t ( ω ) \dot{x}_t=\partial_t x_t(\omega) x ˙ t = ∂ t x t ( ω ) at the collocation points. Space-time collocation points are randomly sampled at each optimization step, avoiding explicit grid discretization and the associated stability constraints.
For quasipotential calculations with T → ∞ T\to\infty T → ∞ , deep gMAM reparametrizes the path by arc-length s ∈ [ 0 , 1 ] s\in[0,1] s ∈ [ 0 , 1 ] and minimizes the geometric action. This requires additional considerations: enforcing the arc-length constraint ∣ x ′ ( s ) ∣ = c o n s t \lvert x'(s)\rvert=\mathrm{const} ∣ x ′ ( s )∣ = const and the zero-energy condition H ( x , θ ) = 0 H(x,\theta)=0 H ( x , θ ) = 0 , typically via penalty terms or augmented Lagrangian methods. See [22 ] for details.
Physics-informed neural networks
More broadly, physics-informed neural networks (PINNs) [39 ] can solve the Hamilton–Jacobi equation (14) or the stationary HJE (62) directly. A neural network v ω ( t , x ) v_\omega(t,x) v ω ( t , x ) is trained to minimize the residual
L ( ω ) = ∫ ∣ ∂ t v ω + H ( x , ∇ v ω ) ∣ 2 d x d t + boundary terms , (140) \mathcal{L}(\omega)=\int\left|\partial_t v_\omega+H(x,\nabla v_\omega)\right|^2\,\mathrm{d}x\,\mathrm{d}t+\text{boundary terms}, \tag*{(140)} L ( ω ) = ∫ ∣ ∂ t v ω + H ( x , ∇ v ω ) ∣ 2 d x d t + boundary terms , (140)
where the gradients are computed via automatic differentiation. This is useful for high-dimensional quasipotential calculations. For spatially extended systems where x x x is itself a field, direct solution of the Hamilton–Jacobi equation becomes considerably more involved; score-based methods that learn the probability flow provide an alternative for computing probability currents and entropy production rates [40 , 41 ] .
Sampling methods for finite-noise, long-time limits
The methods above compute instantons in the small-noise limit ε → 0 \varepsilon\to0 ε → 0 . For the long-time limit at finite noise (§ 5 ), sampling-based approaches are needed. Genealogical algorithms [28 , 29 ] provide an efficient framework, and knowledge of the instanton can significantly improve their performance.
Cloning algorithms
The cloning algorithm maintains N c N_c N c copies of the system, each evolving independently according to the original stochastic dynamics. At regular intervals Δ t \Delta t Δ t , each clone at position x ( i ) x^{(i)} x ( i ) is assigned a weight w i = e f ( x ( i ) ) Δ t w_i=e^{f(x^{(i)})\Delta t} w i = e f ( x ( i ) ) Δ t , where f ( x ) f(x) f ( x ) defines the tilted generator (98) . The ensemble is resampled with replacement according to these weights: clones in regions where f ( x ) f(x) f ( x ) is large are duplicated, while those where f ( x ) f(x) f ( x ) is small are eliminated.
The total weight’s growth rate converges to the principal eigenvalue λ ∗ \lambda_* λ ∗ as N c → ∞ N_c \to\infty N c → ∞ :
λ ∗ = lim T → ∞ 1 T log E [ ∏ k W ‾ k ] , (141) \lambda_*=\lim_{T\to\infty}\frac{1}{T}\log\mathbb{E}\left[\prod_k \overline{W}_k\right], \tag*{(141)} λ ∗ = T → ∞ lim T 1 log E [ k ∏ W k ] , (141)
where W ‾ k \overline{W}_k W k is the average weight at the k k k th resampling step. The resampled ensemble samples the tilted distribution ρ ~ ∗ \widetilde{\rho}_* ρ ∗ , enabling measurement of observables conditioned on the rare event.
Instanton-guided importance sampling
Knowledge of the instanton can dramatically improve sampling efficiency through importance sampling. Given the instanton ( x t ∗ , θ t ∗ ) (x_t^*,\theta_t^*) ( x t ∗ , θ t ∗ ) , one biases the dynamics by adding a drift:
d X t = b ( X t ) d t + 2 D θ t ∗ d t + 2 ε σ d W t . (142) \mathrm{d}X_t=b(X_t)\,\mathrm{d}t+2D\theta_t^*\,\mathrm{d}t+\sqrt{2\varepsilon}\,\sigma\,\mathrm{d}W_t. \tag*{(142)} d X t = b ( X t ) d t + 2 D θ t ∗ d t + 2 ε σ d W t . (142)
This keeps the original drift b ( X t ) b(X_t) b ( X t ) responsive to the actual position while biasing toward the instanton. Samples are then reweighted using the Girsanov formula (Appendix § 8 .8) to correct for the modified dynamics, yielding unbiased estimates with reduced variance.
While this improves over direct sampling, using the instanton momentum θ t ∗ \theta_t^* θ t ∗ as a time-dependent control does not achieve optimal variance reduction. Provably optimal schemes require adaptive controls that depend on the current position, as developed in [42 ] .
Neural Optimal Control
Similar approaches to instanton guidance can be employed within the Donsker–Varadhan framework, as well. For a dynamical observable of the form, A T = ∫ 0 T f ( X t ) d t + ∫ 0 T g ( X t ) ∘ d X t A_T=\int_0^T f(X_t)\,dt+\int_0^T g(X_t)\circ dX_t A T = ∫ 0 T f ( X t ) d t + ∫ 0 T g ( X t ) ∘ d X t , where X t X_t X t evolves according to (1) , we can compute the scaled-cumulant generating function via an optimal importance sampling. As above, the scaled cumulant generating function
λ ∗ ( s ) = lim T → ∞ 1 T log E [ e s A T ] . (143) \lambda_*(s)=\lim_{T\to\infty}\frac{1}{T}\log\mathbb{E}\left[e^{sA_T}\right]. \tag*{(143)} λ ∗ ( s ) = T → ∞ lim T 1 log E [ e s A T ] . (143)
As shown in [43 ] , this problem can be reformulated as a variational optimal control problem in which one optimizes an optimal control drift, f f f leading to a driven equation of motion,
d X t = b ( X t ) d t + f ( X t ) d t + 2 ε σ d W t . (144) dX_t=b(X_t)\,dt+f(X_t)\,dt+\sqrt{2\varepsilon}\,\sigma\,dW_t. \tag*{(144)} d X t = b ( X t ) d t + f ( X t ) d t + 2 ε σ d W t . (144)
The objective is obtained by constructing the optimal importance sampler and subsequently applying Jensen’s inequality to obtain,
λ ∗ ( s ) = sup f lim T → ∞ [ s E f [ A T ] − T − 1 D K L ( P f ∥ P ) ] , (145) \lambda_*(s)=\sup_f\lim_{T\to\infty}\left[s\mathbb{E}_f[A_T]-T^{-1}D_{\mathrm{KL}}\left(\mathbb{P}^f\middle\|\mathbb{P}\right)\right], \tag*{(145)} λ ∗ ( s ) = f sup T → ∞ lim [ s E f [ A T ] − T − 1 D KL ( P f P ) ] , (145)
where D K L D_{\mathrm{KL}} D KL denotes the Kullback–Leibler divergence between the path measures of the controlled and uncontrolled processes. As developed in Refs. [43 , 44 ] , a neural network ansatz for f f f yields a powerful and flexible ansatz for the control function. Directly maximizing (145) via direct backpropagation or an adjoint method converges quickly and is robust even near dynamical phase transitions with appropriate sampling strategies.
Acknowledgments
We thank the organizers of the Les Houches Summer School “Large Deviations: Theory and Applications” for the opportunity to present these lectures. W.D.P acknowledges funding from the Marie-Skłodowska-Curie grant No. 101104945.
Background on stochastic differential equations
This appendix provides a self-contained introduction to the key results of stochastic calculus needed in these notes. We emphasize intuition and practical understanding over mathematical rigor; for complete treatments, we refer to [45 , 46 , 47 ] .
Throughout this appendix, we use uppercase letters X t X_t X t and X ~ k \widetilde{X}_k X k to denote stochastic processes (continuous and discrete, respectively), and lowercase letters x t x_t x t and x k x_k x k for dummy integration variables representing paths.
Our starting point is the stochastic differential equation (SDE)
d X t = b ( t , X t ) d t + σ ( t , X t ) d W t , (146) dX_t=b(t,X_t)\,dt+\sigma(t,X_t)\,dW_t, \tag*{(146)} d X t = b ( t , X t ) d t + σ ( t , X t ) d W t , (146)
where X t ∈ R d X_t\in\mathbb{R}^d X t ∈ R d , the drift b : R × R d → R d b:\mathbb{R}\times\mathbb{R}^d\to\mathbb{R}^d b : R × R d → R d and volatility σ : R × R d → R d × d \sigma:\mathbb{R}\times\mathbb{R}^d\to\mathbb{R}^{d\times d} σ : R × R d → R d × d are given functions, and W t W_t W t is a d d d -dimensional Wiener process (Brownian motion). We denote the diffusion matrix by D ( t , x ) = σ ( t , x ) σ ( t , x ) T D(t,x)=\sigma(t,x)\sigma(t,x)^{\mathsf{T}} D ( t , x ) = σ ( t , x ) σ ( t , x ) T .
Euler–Maruyama discretization
The SDE (146) is defined as the continuum limit of a discrete-time process. To solve it numerically on a time interval [ 0 , T ] [0,T] [ 0 , T ] , divide time into N N N steps of width Δ t = T / N \Delta t = T/N Δ t = T / N , with t k = k Δ t t_k = k\Delta t t k = k Δ t . The Euler–Maruyama scheme defines the discrete approximation X ~ k ≈ X t k \widetilde{X}_k \approx X_{t_k} X k ≈ X t k by the recursion
X ~ k + 1 = X ~ k + b ( t k , X ~ k ) Δ t + σ ( t k , X ~ k ) Δ t ξ k , (147) \widetilde{X}_{k+1}=\widetilde{X}_k+b(t_k,\widetilde{X}_k)\Delta t+\sigma(t_k,\widetilde{X}_k)\sqrt{\Delta t}\,\xi_k, \tag*{(147)} X k + 1 = X k + b ( t k , X k ) Δ t + σ ( t k , X k ) Δ t ξ k , (147)
where ξ k ∼ N ( 0 , 1 ) \xi_k \sim\mathcal{N}(0,1) ξ k ∼ N ( 0 , 1 ) are independent standard Gaussian random vectors. The noise term Δ t ξ k \sqrt{\Delta t}\,\xi_k Δ t ξ k represents the Wiener increment
Δ W k ≡ W t k + 1 − W t k = d Δ t ξ k , (148) \Delta W_k \equiv W_{t_{k+1}}-W_{t_k}\mathrel{\overset{d}{=}}\sqrt{\Delta t}\,\xi_k, \tag*{(148)} Δ W k ≡ W t k + 1 − W t k = d Δ t ξ k , (148)
where = d \overset{d}{=} = d denotes equality in distribution. This makes explicit that Brownian motion has increments of order Δ t \sqrt{\Delta t} Δ t , not Δ t \Delta t Δ t . The key properties are:
E [ Δ W k ] = 0 , E [ Δ W k Δ W k T ] = Δ t 1 , E [ Δ W k Δ W j T ] = 0 for k ≠ j . (149) \mathbb{E}[\Delta W_k]=0,\qquad\mathbb{E}[\Delta W_k\Delta W_k^{\mathsf{T}}]=\Delta t\,\mathbf{1},\qquad\mathbb{E}[\Delta W_k\Delta W_j^{\mathsf{T}}]=0\ \text{for } k\ne j. \tag*{(149)} E [ Δ W k ] = 0 , E [ Δ W k Δ W k T ] = Δ t 1 , E [ Δ W k Δ W j T ] = 0 for k = j . (149)
As Δ t → 0 \Delta t\to0 Δ t → 0 , the discrete process converges to the continuous solution X t X_t X t , which is almost surely continuous but nowhere differentiable (Hölder continuous with exponent 1 / 2 1/2 1/2 ). The Δ t \sqrt{\Delta t} Δ t scaling of increments—rather than Δ t \Delta t Δ t —is the origin of the non-standard rules of stochastic calculus.
The Markov property
The Euler–Maruyama scheme (147) makes manifest that the process is Markov : given the current state X ~ k \widetilde{X}_k X k , the next state X ~ k + 1 \widetilde{X}_{k+1} X k + 1 depends only on X ~ k \widetilde{X}_k X k and the fresh noise ξ k \xi_k ξ k , which is independent of all previous states and increments. The history of how the process reached X ~ k \widetilde{X}_k X k is irrelevant for predicting its future.
This property carries over to the continuum limit: for any times s < t s<t s < t and any measurable set A ⊂ R d A\subset\mathbb{R}^d A ⊂ R d ,
P ( X t ∈ A ∣ ( X r ) r ≤ s ) = P ( X t ∈ A ∣ X s ) . (150) \mathbb{P}(X_t\in A\mid(X_r)_{r\le s})=\mathbb{P}(X_t\in A\mid X_s). \tag*{(150)} P ( X t ∈ A ∣ ( X r ) r ≤ s ) = P ( X t ∈ A ∣ X s ) . (150)
where ( X r ) r ≤ s (X_r)_{r\le s} ( X r ) r ≤ s denotes the trajectory up to time s s s . In words, conditioning on the entire trajectory up to time s s s provides no more information about X t X_t X t than conditioning on X s X_s X s alone. The Markov property underlies all the results that follow, particularly the backward and forward Kolmogorov equations.
The generator
The generator L \mathcal{L} L of the diffusion process encodes the infinitesimal evolution of expected values. For any smooth test function ϕ : R d → R \phi:\mathbb{R}^d\to\mathbb{R} ϕ : R d → R , it is defined by
( L ϕ ) ( x ) = lim Δ t ↓ 0 1 Δ t E X t = x [ ϕ ( X t + Δ t ) − ϕ ( x ) ] . (151) (\mathcal{L}\phi)(x)=\lim_{\Delta t\downarrow0}\frac{1}{\Delta t}\mathbb{E}^{X_t=x}\left[\phi(X_{t+\Delta t})-\phi(x)\right]. \tag*{(151)} ( L ϕ ) ( x ) = Δ t ↓ 0 lim Δ t 1 E X t = x [ ϕ ( X t + Δ t ) − ϕ ( x ) ] . (151)
Derivation from Euler–Maruyama. Using (147) and Taylor-expanding ϕ ( X ~ k + 1 ) \phi(\widetilde{X}_{k+1}) ϕ ( X k + 1 ) around X ~ k = x \widetilde{X}_k=x X k = x :
ϕ ( X ~ k + 1 ) = ϕ ( x + b Δ t + σ Δ t ξ ) = ϕ ( x ) + ∇ ϕ ⋅ ( b Δ t + σ Δ t ξ ) + 1 2 ∑ i , j ∂ i ∂ j ϕ ( σ Δ t ξ ) i ( σ Δ t ξ ) j + O ( Δ t 3 / 2 ) . (152) \begin{aligned}
\phi(\widetilde{X}_{k+1})&=\phi\left(x+b\Delta t+\sigma\sqrt{\Delta t}\,\xi\right)\\
&=\phi(x)+\nabla\phi\cdot\left(b\Delta t+\sigma\sqrt{\Delta t}\,\xi\right)\\
&\quad+\frac{1}{2}\sum_{i,j}\partial_i\partial_j\phi\left(\sigma\sqrt{\Delta t}\,\xi\right)_i\left(\sigma\sqrt{\Delta t}\,\xi\right)_j+O(\Delta t^{3/2}). \tag*{(152)}
\end{aligned} ϕ ( X k + 1 ) = ϕ ( x + b Δ t + σ Δ t ξ ) = ϕ ( x ) + ∇ ϕ ⋅ ( b Δ t + σ Δ t ξ ) + 2 1 i , j ∑ ∂ i ∂ j ϕ ( σ Δ t ξ ) i ( σ Δ t ξ ) j + O ( Δ t 3/2 ) . (152)
Since ξ ∼ N ( 0 , 1 ) \xi\sim\mathcal{N}(0,1) ξ ∼ N ( 0 , 1 ) , we have E [ ξ ] = 0 \mathbb{E}[\xi] = 0 E [ ξ ] = 0 and E [ ξ i ξ j ] = δ i j \mathbb{E}[\xi_i\xi_j] = \delta_{ij} E [ ξ i ξ j ] = δ ij . The cross term
( σ Δ t ξ ) i ( σ Δ t ξ ) j = Δ t ( σ ξ ) i ( σ ξ ) j (\sigma\sqrt{\Delta t}\,\xi)_i(\sigma\sqrt{\Delta t}\,\xi)_j = \Delta t(\sigma\xi)_i(\sigma\xi)_j ( σ Δ t ξ ) i ( σ Δ t ξ ) j = Δ t ( σ ξ ) i ( σ ξ ) j
contributes at order Δ t \Delta t Δ t :
E [ ( σ ξ ) i ( σ ξ ) j ] = ∑ k , l σ i k σ j l E [ ξ k ξ l ] = ∑ k σ i k σ j k = ( σ σ T ) i j = D i j . (153) \mathbb{E}[(\sigma\xi)_i(\sigma\xi)_j] = \sum_{k,l}\sigma_{ik}\sigma_{jl}\mathbb{E}[\xi_k\xi_l] = \sum_k\sigma_{ik}\sigma_{jk} = (\sigma\sigma^{\mathsf{T}})_{ij} = D_{ij}. \tag*{(153)} E [( σ ξ ) i ( σ ξ ) j ] = k , l ∑ σ ik σ j l E [ ξ k ξ l ] = k ∑ σ ik σ j k = ( σ σ T ) ij = D ij . (153)
Thus:
E X ~ k = x [ ϕ ( X ~ k + 1 ) − ϕ ( x ) ] = b ⋅ ∇ ϕ Δ t + 1 2 D : ∇ ∇ ϕ Δ t + O ( Δ t 3 / 2 ) . (154) \mathbb{E}_{\tilde{X}_k=x}\left[\phi(\tilde{X}_{k+1})-\phi(x)\right] = b\cdot\nabla\phi\,\Delta t + \frac{1}{2}D:\nabla\nabla\phi\,\Delta t + O(\Delta t^{3/2}). \tag*{(154)} E X ~ k = x [ ϕ ( X ~ k + 1 ) − ϕ ( x ) ] = b ⋅ ∇ ϕ Δ t + 2 1 D : ∇∇ ϕ Δ t + O ( Δ t 3/2 ) . (154)
Dividing by Δ t \Delta t Δ t and taking Δ t → 0 \Delta t\to0 Δ t → 0 yields the generator:
( L ϕ ) ( x ) = b ( t , x ) ⋅ ∇ ϕ ( x ) + 1 2 D ( t , x ) : ∇ ∇ ϕ ( x ) , (155) (\mathcal{L}\phi)(x) = b(t,x)\cdot\nabla\phi(x) + \frac{1}{2}D(t,x):\nabla\nabla\phi(x), \tag*{(155)} ( L ϕ ) ( x ) = b ( t , x ) ⋅ ∇ ϕ ( x ) + 2 1 D ( t , x ) : ∇∇ ϕ ( x ) , (155)
where D : ∇ ∇ ϕ = ∑ i , j D i j ∂ i ∂ j ϕ D:\nabla\nabla\phi= \sum_{i,j}D_{ij}\partial_i\partial_j\phi D : ∇∇ ϕ = ∑ i , j D ij ∂ i ∂ j ϕ . The second-order term arises because the noise enters at order Δ t \sqrt{\Delta t} Δ t , so its square contributes at order Δ t \Delta t Δ t .
Itô's formula is the stochastic analogue of the chain rule. For a smooth function ϕ ( t , x ) \phi(t,x) ϕ ( t , x ) evaluated along a trajectory X t X_t X t :
d ϕ ( t , X t ) = ( ∂ t ϕ + L ϕ ) d t + ∇ ϕ ⋅ σ d W t . (156) d\phi(t,X_t) = (\partial_t\phi+\mathcal{L}\phi)\,dt+\nabla\phi\cdot\sigma\,dW_t. \tag*{(156)} d ϕ ( t , X t ) = ( ∂ t ϕ + L ϕ ) d t + ∇ ϕ ⋅ σ d W t . (156)
Derivation: From the Euler–Maruyama scheme, the increment over a small time step is X t + Δ t − X t ≈ b Δ t + σ Δ t ξ X_{t+\Delta t}-X_t\approx b\Delta t+\sigma\sqrt{\Delta t}\,\xi X t + Δ t − X t ≈ b Δ t + σ Δ t ξ where ξ ∼ N ( 0 , 1 ) \xi\sim\mathcal{N}(0,1) ξ ∼ N ( 0 , 1 ) . Taylor-expanding ϕ ( t + Δ t , X t + Δ t ) \phi(t+\Delta t,X_{t+\Delta t}) ϕ ( t + Δ t , X t + Δ t ) :
ϕ ( t + Δ t , X t + Δ t ) − ϕ ( t , X t ) = ∂ t ϕ Δ t + ∇ ϕ ⋅ ( b Δ t + σ Δ t ξ ) + 1 2 ∑ i , j ∂ x i ∂ x j ϕ ( σ Δ t ξ ) i ( σ Δ t ξ ) j + O ( Δ t 3 / 2 ) . (157) \begin{aligned}
\phi(t+\Delta t,X_{t+\Delta t})-\phi(t,X_t)
={}&\partial_t\phi\,\Delta t+\nabla\phi\cdot\left(b\Delta t+\sigma\sqrt{\Delta t}\,\xi\right)\\
&+\frac{1}{2}\sum_{i,j}\partial_{x_i}\partial_{x_j}\phi\,(\sigma\sqrt{\Delta t}\,\xi)_i(\sigma\sqrt{\Delta t}\,\xi)_j+O(\Delta t^{3/2}). \tag*{(157)}
\end{aligned} ϕ ( t + Δ t , X t + Δ t ) − ϕ ( t , X t ) = ∂ t ϕ Δ t + ∇ ϕ ⋅ ( b Δ t + σ Δ t ξ ) + 2 1 i , j ∑ ∂ x i ∂ x j ϕ ( σ Δ t ξ ) i ( σ Δ t ξ ) j + O ( Δ t 3/2 ) . (157)
The key observation is that
( σ Δ t ξ ) i ( σ Δ t ξ ) j = Δ t ( σ ξ ) i ( σ ξ ) j = D i j Δ t + O ( Δ t ) , (158) (\sigma\sqrt{\Delta t}\,\xi)_i(\sigma\sqrt{\Delta t}\,\xi)_j = \Delta t(\sigma\xi)_i(\sigma\xi)_j = D_{ij}\Delta t+O(\Delta t), \tag*{(158)} ( σ Δ t ξ ) i ( σ Δ t ξ ) j = Δ t ( σ ξ ) i ( σ ξ ) j = D ij Δ t + O ( Δ t ) , (158)
where the second equality holds in the mean-square sense: while ( σ ξ ) i ( σ ξ ) j (\sigma\xi)_i(\sigma\xi)_j ( σ ξ ) i ( σ ξ ) j fluctuates, its deviation from D i j D_{ij} D ij averages to zero and contributes only at higher order. Thus:
ϕ ( t + Δ t , X t + Δ t ) − ϕ ( t , X t ) = ∂ t ϕ Δ t + b ⋅ ∇ ϕ Δ t + 1 2 D : ∇ ∇ ϕ Δ t + ∇ ϕ ⋅ σ Δ t ξ + O ( Δ t 3 / 2 ) . (159) \begin{aligned}
\phi(t+\Delta t,X_{t+\Delta t})-\phi(t,X_t)
={}&\partial_t\phi\,\Delta t+b\cdot\nabla\phi\,\Delta t+\frac{1}{2}D:\nabla\nabla\phi\,\Delta t\\
&+\nabla\phi\cdot\sigma\sqrt{\Delta t}\,\xi+O(\Delta t^{3/2}). \tag*{(159)}
\end{aligned} ϕ ( t + Δ t , X t + Δ t ) − ϕ ( t , X t ) = ∂ t ϕ Δ t + b ⋅ ∇ ϕ Δ t + 2 1 D : ∇∇ ϕ Δ t + ∇ ϕ ⋅ σ Δ t ξ + O ( Δ t 3/2 ) . (159)
Taking Δ t → 0 \Delta t\to0 Δ t → 0 and writing Δ t ξ → d W t \sqrt{\Delta t}\,\xi\to dW_t Δ t ξ → d W t yields (156) . The “extra” term 1 2 D : ∇ ∇ ϕ \frac{1}{2}D:\nabla\nabla\phi 2 1 D : ∇∇ ϕ compared to ordinary calculus arises because ( d W t ) 2 = d t (dW_t)^2=dt ( d W t ) 2 = d t rather than zero—this is the essence of Itô calculus. In differential notation:
( d W t ) 2 = d t , d W t d t = 0 , ( d t ) 2 = 0. (160) (dW_t)^2=dt,\qquad dW_t\,dt=0,\qquad(dt)^2=0. \tag*{(160)} ( d W t ) 2 = d t , d W t d t = 0 , ( d t ) 2 = 0. (160)
These rules follow directly from the Δ t \sqrt{\Delta t} Δ t scaling of Wiener increments.
Martingale property and Itô isometry
Stochastic integrals ∫ 0 T g ( t , X t ) ⋅ d W t \int_0^T g(t,X_t)\cdot dW_t ∫ 0 T g ( t , X t ) ⋅ d W t satisfy two fundamental properties. The first is the martingale property:
E [ ∫ 0 T g ( t , X t ) ⋅ d W t ] = 0. (161) \mathbb{E}\left[\int_0^T g(t,X_t)\cdot dW_t\right]=0. \tag*{(161)} E [ ∫ 0 T g ( t , X t ) ⋅ d W t ] = 0. (161)
The second is the Itô isometry:
E [ ∣ ∫ 0 T g ( t , X t ) ⋅ d W t ∣ 2 ] = ∫ 0 T E [ ∣ g ( t , X t ) ∣ 2 ] d t . (162) \mathbb{E}\left[\left|\int_0^T g(t,X_t)\cdot dW_t\right|^2\right]=\int_0^T\mathbb{E}\left[|g(t,X_t)|^2\right]\,dt. \tag*{(162)} E ∫ 0 T g ( t , X t ) ⋅ d W t 2 = ∫ 0 T E [ ∣ g ( t , X t ) ∣ 2 ] d t . (162)
The term “isometry” reflects that stochastic integration preserves the L 2 L^2 L 2 norm: the variance of the stochastic integral equals the expected integral of ∣ g ∣ 2 |g|^2 ∣ g ∣ 2 .
Derivation: Discretizing ∫ 0 T g ( t , X t ) ⋅ d W t ≈ ∑ k g k ⋅ Δ t ξ k \int_0^T g(t,X_t)\cdot dW_t\approx\sum_k g_k\cdot\sqrt{\Delta t}\,\xi_k ∫ 0 T g ( t , X t ) ⋅ d W t ≈ ∑ k g k ⋅ Δ t ξ k where g k = g ( t k , X t k ) g_k=g(t_k,X_{t_k}) g k = g ( t k , X t k ) . Since g k g_k g k depends only on the trajectory up to time t k t_k t k , it is independent of the current and future noise increments ξ j \xi_j ξ j for j ≥ k j\ge k j ≥ k . In particular, E [ ξ k ] = 0 \mathbb{E}[\xi_k]=0 E [ ξ k ] = 0 implies:
E [ ∑ k g k ⋅ Δ t ξ k ] = Δ t ∑ k E [ g k ⋅ ξ k ] = Δ t ∑ k E [ g k ] ⋅ E [ ξ k ] = 0. (163) \mathbb{E}\left[\sum_k g_k\cdot\sqrt{\Delta t}\,\xi_k\right]=\sqrt{\Delta t}\sum_k\mathbb{E}[g_k\cdot\xi_k]=\sqrt{\Delta t}\sum_k\mathbb{E}[g_k]\cdot\mathbb{E}[\xi_k]=0. \tag*{(163)} E [ k ∑ g k ⋅ Δ t ξ k ] = Δ t k ∑ E [ g k ⋅ ξ k ] = Δ t k ∑ E [ g k ] ⋅ E [ ξ k ] = 0. (163)
For the second moment:
E [ ∣ ∑ k g k ⋅ Δ t ξ k ∣ 2 ] = Δ t ∑ k , j E [ ( g k ⋅ ξ k ) ( g j ⋅ ξ j ) ] . (164) \mathbb{E}\left[\left|\sum_k g_k\cdot\sqrt{\Delta t}\,\xi_k\right|^2\right]=\Delta t\sum_{k,j}\mathbb{E}[(g_k\cdot\xi_k)(g_j\cdot\xi_j)]. \tag*{(164)} E k ∑ g k ⋅ Δ t ξ k 2 = Δ t k , j ∑ E [( g k ⋅ ξ k ) ( g j ⋅ ξ j )] . (164)
For k < j k<j k < j , the product g k ⋅ ξ k g_k\cdot\xi_k g k ⋅ ξ k depends on the trajectory up to time t k + 1 t_{k+1} t k + 1 , and is therefore independent of ξ j \xi_j ξ j . Since E [ ξ j ] = 0 \mathbb{E}[\xi_j]=0 E [ ξ j ] = 0 :
E [ ( g k ⋅ ξ k ) ( g j ⋅ ξ j ) ] = E [ ( g k ⋅ ξ k ) g j ] ⋅ E [ ξ j ] = 0. (165) \mathbb{E}[(g_k\cdot\xi_k)(g_j\cdot\xi_j)]=\mathbb{E}[(g_k\cdot\xi_k)g_j]\cdot\mathbb{E}[\xi_j]=0. \tag*{(165)} E [( g k ⋅ ξ k ) ( g j ⋅ ξ j )] = E [( g k ⋅ ξ k ) g j ] ⋅ E [ ξ j ] = 0. (165)
By symmetry, the same holds for k > j k>j k > j . For k = j k=j k = j , using the independence of g k g_k g k and ξ k \xi_k ξ k together with E [ ξ k , i ξ k , l ] = δ i l \mathbb{E}[\xi_{k,i}\xi_{k,l}]=\delta_{il} E [ ξ k , i ξ k , l ] = δ i l :
E [ ( g k ⋅ ξ k ) 2 ] = E [ ∑ i , l g k , i g k , l ξ k , i ξ k , l ] = E [ ∑ i g k , i 2 ] = E [ ∣ g k ∣ 2 ] . (166) \mathbb{E}[(g_k\cdot\xi_k)^2]=\mathbb{E}\left[\sum_{i,l}g_{k,i}g_{k,l}\,\xi_{k,i}\xi_{k,l}\right]=\mathbb{E}\left[\sum_i g_{k,i}^2\right]=\mathbb{E}[|g_k|^2]. \tag*{(166)} E [( g k ⋅ ξ k ) 2 ] = E i , l ∑ g k , i g k , l ξ k , i ξ k , l = E [ i ∑ g k , i 2 ] = E [ ∣ g k ∣ 2 ] . (166)
Therefore:
E [ ∣ ∑ k g k Δ t ξ k ∣ 2 ] = Δ t ∑ k E [ ∣ g k ∣ 2 ] → ∫ 0 T E [ ∣ g ( t , X t ) ∣ 2 ] d t as Δ t → 0. (167) \mathbb{E}\left[\left|\sum_k g_k\sqrt{\Delta t}\,\xi_k\right|^2\right]=\Delta t\sum_k\mathbb{E}[|g_k|^2]\to\int_0^T\mathbb{E}\left[|g(t,X_t)|^2\right]\,dt\quad\text{as }\Delta t\to0. \tag*{(167)} E k ∑ g k Δ t ξ k 2 = Δ t k ∑ E [ ∣ g k ∣ 2 ] → ∫ 0 T E [ ∣ g ( t , X t ) ∣ 2 ] d t as Δ t → 0. (167)
Backward Kolmogorov equation
With Itô’s formula and the martingale property in hand, we can now derive the fundamental equations governing expectations over trajectories. The backward Kolmogorov equation (BKE) governs the evolution of conditional expectations. Let u ( t , x ) = E t X t = x [ F ( X T ) ] u(t,x)=\mathbb{E}_{t}^{X_t=x}[F(X_T)] u ( t , x ) = E t X t = x [ F ( X T )] for some terminal function F F F . Then u u u satisfies
∂ t u + L u = 0 , u ( T , x ) = F ( x ) . (168) \partial_tu+\mathcal{L}u=0,\qquad u(T,x)=F(x). \tag*{(168)} ∂ t u + L u = 0 , u ( T , x ) = F ( x ) . (168)
Derivation: By the Markov property and the tower rule for conditional expectations:
u ( t , x ) = E X t = x [ F ( X T ) ] = E X t = x [ E X t + Δ t [ F ( X T ) ] ] = E X t = x [ u ( t + Δ t , X t + Δ t ) ] . (169) u(t,x)=\mathbb{E}^{X_t=x}[F(X_T)]=\mathbb{E}^{X_t=x}\left[\mathbb{E}^{X_{t+\Delta t}}[F(X_T)]\right]=\mathbb{E}^{X_t=x}[u(t+\Delta t,X_{t+\Delta t})]. \tag*{(169)} u ( t , x ) = E X t = x [ F ( X T )] = E X t = x [ E X t + Δ t [ F ( X T )] ] = E X t = x [ u ( t + Δ t , X t + Δ t )] . (169)
Using the definition of the generator:
u ( t , x ) = u ( t + Δ t , x ) + L u ( t + Δ t , x ) Δ t + O ( Δ t 2 ) . (170) u(t,x)=u(t+\Delta t,x)+\mathcal{L}u(t+\Delta t,x)\Delta t+O(\Delta t^2). \tag*{(170)} u ( t , x ) = u ( t + Δ t , x ) + L u ( t + Δ t , x ) Δ t + O ( Δ t 2 ) . (170)
Rearranging and taking Δ t → 0 \Delta t\to0 Δ t → 0 :
u ( t + Δ t , x ) − u ( t , x ) Δ t = − L u + O ( Δ t ) ⇒ ∂ t u = − L u . (171) \frac{u(t+\Delta t,x)-u(t,x)}{\Delta t}=-\mathcal{L}u+O(\Delta t)\quad\Rightarrow\quad\partial_tu=-\mathcal{L}u. \tag*{(171)} Δ t u ( t + Δ t , x ) − u ( t , x ) = − L u + O ( Δ t ) ⇒ ∂ t u = − L u . (171)
The BKE evolves backward in time: given terminal data F ( x ) F(x) F ( x ) at time T T T , it determines u ( t , x ) u(t,x) u ( t , x ) for t < T t<T t < T .
Remark A.1. An alternative proof applies Itô’s formula directly to u ( t , X t ) u(t,X_t) u ( t , X t ) :
d u = [ ∂ t u + L u ] d t + ∇ u ⋅ σ d W t . (172) du=[\partial_tu+\mathcal{L}u]\,dt+\nabla u\cdot\sigma\,dW_t. \tag*{(172)} d u = [ ∂ t u + L u ] d t + ∇ u ⋅ σ d W t . (172)
Integrating from t t t to T T T and taking expectations, the stochastic integral vanishes by the martingale property (161) :
E X t = x [ u ( T , X T ) ] − u ( t , x ) = E X t = x [ ∫ t T [ ∂ t u + L u ] d s ] . (173) \mathbb{E}^{X_t=x}[u(T,X_T)]-u(t,x)=\mathbb{E}^{X_t=x}\left[\int_t^T[\partial_tu+\mathcal{L}u]\,ds\right]. \tag*{(173)} E X t = x [ u ( T , X T )] − u ( t , x ) = E X t = x [ ∫ t T [ ∂ t u + L u ] d s ] . (173)
Since u ( T , X T ) = F ( X T ) u(T,X_T)=F(X_T) u ( T , X T ) = F ( X T ) and u ( t , x ) = E X t = x [ F ( X T ) ] u(t,x)=\mathbb{E}^{X_t=x}[F(X_T)] u ( t , x ) = E X t = x [ F ( X T )] by definition, the left-hand side vanishes. For this to hold for all t t t and x x x , we must have ∂ t u + L u = 0 \partial_tu+\mathcal{L}u=0 ∂ t u + L u = 0 .
Forward Kolmogorov equation (Fokker–Planck)
The probability density function (PDF) ρ ( t , x ) \rho(t,x) ρ ( t , x ) characterizes the distribution of X t X_t X t : for any measurable set A ⊆ R d A\subseteq\mathbb{R}^d A ⊆ R d ,
P ( X t ∈ A ) = ∫ A ρ ( t , x ) d x . (174) \mathbb{P}(X_t\in A)=\int_A\rho(t,x)\,dx. \tag*{(174)} P ( X t ∈ A ) = ∫ A ρ ( t , x ) d x . (174)
Equivalently, ρ ( t , x ) d x \rho(t,x)\,dx ρ ( t , x ) d x is the probability of finding the process in an infinitesimal volume d x dx d x around x x x at time t t t .
The forward Kolmogorov equation , also called the Fokker–Planck equation , governs the evolution of ρ ( t , x ) \rho(t,x) ρ ( t , x ) :
∂ t ρ = L ∗ ρ = − ∇ ⋅ ( b ρ ) + 1 2 ∇ ∇ : ( D ρ ) , (175) \partial_t\rho=\mathcal{L}^*\rho=-\nabla\cdot(b\rho)+\frac{1}{2}\nabla\nabla:(D\rho), \tag*{(175)} ∂ t ρ = L ∗ ρ = − ∇ ⋅ ( b ρ ) + 2 1 ∇∇ : ( D ρ ) , (175)
where L ∗ \mathcal{L}^* L ∗ is the adjoint of L \mathcal{L} L , and ∇ ∇ : ( D ρ ) = ∑ i , j ∂ x i ∂ x j ( D i j ρ ) \nabla\nabla:(D\rho)=\sum_{i,j}\partial_{x_i}\partial_{x_j}(D_{ij}\rho) ∇∇ : ( D ρ ) = ∑ i , j ∂ x i ∂ x j ( D ij ρ ) .
Derivation: For any test function ϕ \phi ϕ , the expected value E [ ϕ ( X t ) ] = ∫ ϕ ( x ) ρ ( t , x ) d x \mathbb{E}[\phi(X_t)]=\int\phi(x)\rho(t,x)\,dx E [ ϕ ( X t )] = ∫ ϕ ( x ) ρ ( t , x ) d x satisfies
d d t ∫ ϕ ρ d x = lim Δ t → 0 1 Δ t E [ ϕ ( X t + Δ t ) − ϕ ( X t ) ] = ∫ ( L ϕ ) ρ d x . (176) \frac{d}{dt}\int\phi\,\rho\,dx=\lim_{\Delta t\to0}\frac{1}{\Delta t}\mathbb{E}[\phi(X_{t+\Delta t})-\phi(X_t)]=\int(\mathcal{L}\phi)\rho\,dx. \tag*{(176)} d t d ∫ ϕ ρ d x = Δ t → 0 lim Δ t 1 E [ ϕ ( X t + Δ t ) − ϕ ( X t )] = ∫ ( L ϕ ) ρ d x . (176)
Integrating by parts (assuming boundary terms vanish):
∫ ϕ ∂ t ρ d x = ∫ ϕ ( L ∗ ρ ) d x , (177) \int\phi\,\partial_t\rho\,dx=\int\phi(\mathcal{L}^*\rho)\,dx, \tag*{(177)} ∫ ϕ ∂ t ρ d x = ∫ ϕ ( L ∗ ρ ) d x , (177)
which holds for all ϕ \phi ϕ , yielding (175) . The Fokker–Planck equation evolves forward in time from an initial density ρ ( 0 , x ) \rho(0,x) ρ ( 0 , x ) .
Path integral representation and Girsanov formula
The Euler–Maruyama discretization leads naturally to a path integral representation of expectations over trajectories. Consider for simplicity the SDE:
d X t = b ( t , X t ) d t + d W t , (178) dX_t = b(t,X_t)\,dt + dW_t, \tag*{(178)} d X t = b ( t , X t ) d t + d W t , (178)
where for simplicity we take σ = 1 \sigma= 1 σ = 1 . From (147) , the update is
X ~ k + 1 = X ~ k + b ( t k , X ~ k ) Δ t + Δ t ξ k . (179) \widetilde{X}_{k+1} = \widetilde{X}_k + b(t_k,\widetilde{X}_k)\Delta t + \sqrt{\Delta t}\,\xi_k. \tag*{(179)} X k + 1 = X k + b ( t k , X k ) Δ t + Δ t ξ k . (179)
The conditional distribution is
X ~ k + 1 ∣ X ~ k ∼ N ( X ~ k + b ( t k , X ~ k ) Δ t , Δ t 1 ) , (180) \widetilde{X}_{k+1}\mid\widetilde{X}_k \sim\mathcal{N}\left(\widetilde{X}_k+b(t_k,\widetilde{X}_k)\Delta t,\Delta t\,1\right), \tag*{(180)} X k + 1 ∣ X k ∼ N ( X k + b ( t k , X k ) Δ t , Δ t 1 ) , (180)
with density proportional to exp ( − 1 2 Δ t ∣ x k + 1 − x k − b ( t k , x k ) Δ t ∣ 2 ) \exp\left(-\frac{1}{2\Delta t}|x_{k+1}-x_k-b(t_k,x_k)\Delta t|^2\right) exp ( − 2Δ t 1 ∣ x k + 1 − x k − b ( t k , x k ) Δ t ∣ 2 ) . The joint probability density function of a discretized path ( x 0 , x 1 , … , x K ) (x_0,x_1,\ldots,x_K) ( x 0 , x 1 , … , x K ) is therefore
P ( x 1 , … , x K ∣ x 0 ) ∝ exp ( − 1 2 Δ t ∑ k = 0 K − 1 ∣ x k + 1 − x k − b ( t k , x k ) Δ t ∣ 2 ) . (181) P(x_1,\ldots,x_K\mid x_0)\propto\exp\left(-\frac{1}{2\Delta t}\sum_{k=0}^{K-1}|x_{k+1}-x_k-b(t_k,x_k)\Delta t|^2\right). \tag*{(181)} P ( x 1 , … , x K ∣ x 0 ) ∝ exp ( − 2Δ t 1 k = 0 ∑ K − 1 ∣ x k + 1 − x k − b ( t k , x k ) Δ t ∣ 2 ) . (181)
To interpret this, compare with pure Brownian motion (b = 0 b=0 b = 0 ), for which
P 0 ( x 1 , … , x K ∣ x 0 ) ∝ exp ( − 1 2 Δ t ∑ k = 0 K − 1 ∣ x k + 1 − x k ∣ 2 ) . (182) P_0(x_1,\ldots,x_K\mid x_0)\propto\exp\left(-\frac{1}{2\Delta t}\sum_{k=0}^{K-1}|x_{k+1}-x_k|^2\right). \tag*{(182)} P 0 ( x 1 , … , x K ∣ x 0 ) ∝ exp ( − 2Δ t 1 k = 0 ∑ K − 1 ∣ x k + 1 − x k ∣ 2 ) . (182)
The ratio of path probabilities is
P ( x 1 , … , x K ∣ x 0 ) P 0 ( x 1 , … , x K ∣ x 0 ) = exp ( ∑ k = 0 K − 1 b ( t k , x k ) ⋅ ( x k + 1 − x k ) − 1 2 ∑ k = 0 K − 1 ∣ b ( t k , x k ) ∣ 2 Δ t ) . (183) \frac{P(x_1,\ldots,x_K\mid x_0)}{P_0(x_1,\ldots,x_K\mid x_0)}
=\exp\left(\sum_{k=0}^{K-1}b(t_k,x_k)\cdot(x_{k+1}-x_k)-\frac{1}{2}\sum_{k=0}^{K-1}|b(t_k,x_k)|^2\Delta t\right). \tag*{(183)} P 0 ( x 1 , … , x K ∣ x 0 ) P ( x 1 , … , x K ∣ x 0 ) = exp ( k = 0 ∑ K − 1 b ( t k , x k ) ⋅ ( x k + 1 − x k ) − 2 1 k = 0 ∑ K − 1 ∣ b ( t k , x k ) ∣ 2 Δ t ) . (183)
In the continuum limit Δ t → 0 \Delta t\to0 Δ t → 0 , recognizing ( x k + 1 − x k ) → d x t (x_{k+1}-x_k)\to dx_t ( x k + 1 − x k ) → d x t formally and Δ t ∑ k → ∫ d t \Delta t\sum_k\to\int dt Δ t ∑ k → ∫ d t , this becomes the Girsanov formula :
d P d P 0 [ ( x t ) t ∈ [ 0 , T ] ] = exp ( ∫ 0 T b ( t , x t ) ⋅ d x t − 1 2 ∫ 0 T ∣ b ( t , x t ) ∣ 2 d t ) , (184) \frac{d\mathbb{P}}{d\mathbb{P}_0}[(x_t)_{t\in[0,T]}]
=\exp\left(\int_0^T b(t,x_t)\cdot dx_t-\frac{1}{2}\int_0^T|b(t,x_t)|^2dt\right), \tag*{(184)} d P 0 d P [( x t ) t ∈ [ 0 , T ] ] = exp ( ∫ 0 T b ( t , x t ) ⋅ d x t − 2 1 ∫ 0 T ∣ b ( t , x t ) ∣ 2 d t ) , (184)
which gives the relative density of the path measure P \mathbb{P} P (for the SDE with drift b b b ) with respect to the Wiener measure P 0 \mathbb{P}_0 P 0 . Here ( x t ) t ∈ [ 0 , T ] (x_t)_{t\in[0,T]} ( x t ) t ∈ [ 0 , T ] denotes a continuous path starting at x 0 x_0 x 0 , and d x t dx_t d x t its infinitesimal increment. For a rigorous treatment, see [45 , 46 ] .
Using (184) , expectations under the SDE with drift b b b can be expressed in terms of expectations under pure Brownian motion:
E x 0 [ F [ ( X t ) t ∈ [ 0 , T ] ] ] = E 0 [ F [ ( x 0 + W t ) t ∈ [ 0 , T ] ] M T ] \mathbb{E}^{x_0}\left[F[(X_t)_{t\in[0,T]}]\right]
=\mathbb{E}_0\left[F[(x_0+W_t)_{t\in[0,T]}]M_T\right] E x 0 [ F [( X t ) t ∈ [ 0 , T ] ] ] = E 0 [ F [( x 0 + W t ) t ∈ [ 0 , T ] ] M T ]
with
M T = exp ( ∫ 0 T b ( t , x 0 + W t ) ⋅ d W t − 1 2 ∫ 0 T ∣ b ( t , x 0 + W t ) ∣ 2 d t ) , (185) M_T=\exp\left(\int_0^T b(t,x_0+W_t)\cdot dW_t-\frac{1}{2}\int_0^T|b(t,x_0+W_t)|^2dt\right), \tag*{(185)} M T = exp ( ∫ 0 T b ( t , x 0 + W t ) ⋅ d W t − 2 1 ∫ 0 T ∣ b ( t , x 0 + W t ) ∣ 2 d t ) , (185)
where E 0 \mathbb{E}_0 E 0 denotes expectation with respect to standard Brownian motion W t W_t W t starting at 0 0 0 . This exact formula has a formal path integral representation :
E x 0 [ F [ ( X t ) t ∈ [ 0 , T ] ] ] ∝ ∫ D [ x t ] F [ ( x t ) t ∈ [ 0 , T ] ] exp ( − 1 2 ∫ 0 T ∣ x ˙ t − b ( t , x t ) ∣ 2 d t ) , (186) \mathbb{E}^{x_0}\left[F[(X_t)_{t\in[0,T]}]\right]\propto\int\mathcal{D}[x_t]\,F[(x_t)_{t\in[0,T]}]\exp\left(-\frac{1}{2}\int_0^T|\dot{x}_t-b(t,x_t)|^2dt\right), \tag*{(186)} E x 0 [ F [( X t ) t ∈ [ 0 , T ] ] ] ∝ ∫ D [ x t ] F [( x t ) t ∈ [ 0 , T ] ] exp ( − 2 1 ∫ 0 T ∣ x ˙ t − b ( t , x t ) ∣ 2 d t ) , (186)
where the integral is formally taken over continuous paths starting at x 0 x_0 x 0 , and the Lagrangian L ( t , x , x ˙ ) = 1 2 ∣ x ˙ − b ( t , x ) ∣ 2 L(t,x,\dot{x})=\frac{1}{2}|\dot{x}-b(t,x)|^2 L ( t , x , x ˙ ) = 2 1 ∣ x ˙ − b ( t , x ) ∣ 2 measures the “cost” of deviating from the deterministic flow.
Heuristic derivation of the LDP
The path integral representation provides a direct route to the Large Deviation Principle. Consider now the SDE with small noise:
d X t ε = b ( t , X t ε ) d t + ε d W t . (187) dX_t^{\varepsilon}=b(t,X_t^{\varepsilon})\,dt+\sqrt{\varepsilon}\,dW_t. \tag*{(187)} d X t ε = b ( t , X t ε ) d t + ε d W t . (187)
Repeating the discretization argument, the path probability becomes
P ( x 1 , … , x K ∣ x 0 ) ∝ exp ( − 1 2 ε Δ t ∑ k = 0 K − 1 ∣ x k + 1 − x k − b ( t k , x k ) Δ t ∣ 2 ) . (188) P(x_1,\ldots,x_K\mid x_0)\propto\exp\left(-\frac{1}{2\varepsilon\Delta t}\sum_{k=0}^{K-1}\left|x_{k+1}-x_k-b(t_k,x_k)\Delta t\right|^2\right). \tag*{(188)} P ( x 1 , … , x K ∣ x 0 ) ∝ exp ( − 2 ε Δ t 1 k = 0 ∑ K − 1 ∣ x k + 1 − x k − b ( t k , x k ) Δ t ∣ 2 ) . (188)
leading to the path integral
E X 0 [ e f ( X T ε ) / ε ] ∝ ∫ D [ x t ] exp [ 1 ε ( f ( x T ) − 1 2 ∫ 0 T ∣ x ˙ t − b ( t , x t ) ∣ 2 d t ) ] . (189) \mathbb{E}^{X^0}\left[e^{f(X_T^\varepsilon)/\varepsilon}\right]\propto\int\mathcal{D}[x_t]\exp\left[\frac{1}{\varepsilon}\left(f(x_T)-\frac{1}{2}\int_0^T\left|\dot{x}_t-b(t,x_t)\right|^2\,dt\right)\right]. \tag*{(189)} E X 0 [ e f ( X T ε ) / ε ] ∝ ∫ D [ x t ] exp [ ε 1 ( f ( x T ) − 2 1 ∫ 0 T ∣ x ˙ t − b ( t , x t ) ∣ 2 d t ) ] . (189)
As ε → 0 \varepsilon\to0 ε → 0 , the factor 1 / ε 1/\varepsilon 1/ ε in the exponent becomes large. By Laplace’s method (saddle-point approximation), the integral is dominated by paths that maximize the exponent:
lim ε → 0 ε log E X 0 [ e f ( X T ε ) / ε ] = sup { x t } : x 0 f i x e d [ f ( x T ) − 1 2 ∫ 0 T ∣ x ˙ t − b ( t , x t ) ∣ 2 d t ] . (190) \lim_{\varepsilon\to0}\varepsilon\log\mathbb{E}^{X^0}\left[e^{f(X_T^\varepsilon)/\varepsilon}\right]=\sup_{\{x_t\}:x_0\ \mathrm{fixed}}\left[f(x_T)-\frac{1}{2}\int_0^T\left|\dot{x}_t-b(t,x_t)\right|^2\,dt\right]. \tag*{(190)} ε → 0 lim ε log E X 0 [ e f ( X T ε ) / ε ] = { x t } : x 0 fixed sup [ f ( x T ) − 2 1 ∫ 0 T ∣ x ˙ t − b ( t , x t ) ∣ 2 d t ] . (190)
This recovers the LDP of § 2 , with Lagrangian L ( t , x , x ˙ ) = 1 2 ∣ x ˙ − b ( t , x ) ∣ 2 L(t,x,\dot{x})=\frac{1}{2}\left|\dot{x}-b(t,x)\right|^2 L ( t , x , x ˙ ) = 2 1 ∣ x ˙ − b ( t , x ) ∣ 2 . The optimal path—the one achieving the supremum—is the most probable way to realize the rare event, and corresponds to a specific “optimal noise realization.”
While this path-integral derivation is heuristic, it provides valuable intuition: rare events occur when the noise conspires to push the system along an atypical trajectory, and the probability of such a conspiracy is controlled by the action ∫ L d t \int L\,dt ∫ L d t .
Background on Markov jump processes
While the main text focuses on diffusion processes, many systems—particularly in chemical kinetics and population dynamics—are better described by Markov jump processes (MJPs). This appendix summarizes the key properties that parallel the SDE theory developed in Appendix A.
Definition via transition rates
An MJP on state space Ω ⊆ R d \Omega\subseteq\mathbb{R}^d Ω ⊆ R d is specified by R R R reaction channels with rates λ j ( x ) ≥ 0 \lambda_j(x)\ge0 λ j ( x ) ≥ 0 and displacement vectors ν j ∈ R d \nu_j\in\mathbb{R}^d ν j ∈ R d , j = 1 , … , R j=1,\ldots,R j = 1 , … , R . In an infinitesimal time Δ t \Delta t Δ t , the process jumps from state x x x to state x + ν j x+\nu_j x + ν j with probability λ j ( x ) Δ t + o ( Δ t ) \lambda_j(x)\Delta t+o(\Delta t) λ j ( x ) Δ t + o ( Δ t ) , and remains at x x x with probability 1 − Λ ( x ) Δ t + o ( Δ t ) 1-\Lambda(x)\Delta t+o(\Delta t) 1 − Λ ( x ) Δ t + o ( Δ t ) , where Λ ( x ) = ∑ j = 1 R λ j ( x ) \Lambda(x)=\sum_{j=1}^{R}\lambda_j(x) Λ ( x ) = ∑ j = 1 R λ j ( x ) is the total rate.
This definition has a transparent interpretation in terms of the Gillespie algorithm [16 ] , which simulates MJPs exactly. Given the current state X t X_t X t , the process waits for an exponentially distributed time τ ∼ Exp ( Λ ( X t ) ) \tau\sim\operatorname{Exp}(\Lambda(X_t)) τ ∼ Exp ( Λ ( X t )) . When the jump occurs, reaction j j j is selected with probability λ j ( X t ) / Λ ( X t ) \lambda_j(X_t)/\Lambda(X_t) λ j ( X t ) /Λ ( X t ) , and the state updates to X t + τ = X t + ν j X_{t+\tau}=X_t+\nu_j X t + τ = X t + ν j . This cycle then repeats from the new state. The memoryless property of the exponential distribution ensures that the process is Markov.
The generator
The generator of an MJP, analogous to (151) for diffusions, is
( L ϕ ) ( x ) = lim Δ t ↓ 0 1 Δ t E X t = x [ ϕ ( X t + Δ t ) − ϕ ( X t ) ] = ∑ j = 1 R λ j ( x ) [ ϕ ( x + ν j ) − ϕ ( x ) ] . (191) (L\phi)(x) = \lim_{\Delta t \downarrow0} \frac{1}{\Delta t}\mathbb{E}^{X_t=x}\left[\phi(X_{t+\Delta t})-\phi(X_t)\right] = \sum_{j=1}^{R}\lambda_j(x)\left[\phi(x+\nu_j)-\phi(x)\right]. \tag*{(191)} ( L ϕ ) ( x ) = Δ t ↓ 0 lim Δ t 1 E X t = x [ ϕ ( X t + Δ t ) − ϕ ( X t ) ] = j = 1 ∑ R λ j ( x ) [ ϕ ( x + ν j ) − ϕ ( x ) ] . (191)
This follows directly from the transition probabilities: in time Δ t \Delta t Δ t , the expected change in ϕ \phi ϕ is
E X t = x [ ϕ ( X t + Δ t ) − ϕ ( x ) ] = ∑ j = 1 R λ j ( x ) Δ t [ ϕ ( x + ν j ) − ϕ ( x ) ] + o ( Δ t ) . (192) \mathbb{E}^{X_t=x}\left[\phi(X_{t+\Delta t})-\phi(x)\right] = \sum_{j=1}^{R}\lambda_j(x)\Delta t\left[\phi(x+\nu_j)-\phi(x)\right] + o(\Delta t). \tag*{(192)} E X t = x [ ϕ ( X t + Δ t ) − ϕ ( x ) ] = j = 1 ∑ R λ j ( x ) Δ t [ ϕ ( x + ν j ) − ϕ ( x ) ] + o ( Δ t ) . (192)
Backward Kolmogorov equation
The BKE holds for MJPs with the same interpretation as for SDEs: if u ( t , x ) u(t,x) u ( t , x ) satisfies
∂ t u + L u = 0 , u ( T , x ) = F ( x ) , (193) \partial_t u + Lu = 0,\qquad u(T,x)=F(x), \tag*{(193)} ∂ t u + Lu = 0 , u ( T , x ) = F ( x ) , (193)
then u ( t , x ) = E X t = x [ F ( X T ) ] u(t,x)=\mathbb{E}^{X_t=x}[F(X_T)] u ( t , x ) = E X t = x [ F ( X T )] .
The proof parallels the diffusive case. Define the process M t = u ( t , X t ) M_t=u(t,X_t) M t = u ( t , X t ) . Between jumps, X t X_t X t is constant, so d M t = ∂ t u d t dM_t=\partial_tu\,dt d M t = ∂ t u d t . At a jump time τ \tau τ where X τ − = x X_{\tau-}=x X τ − = x and X τ = x + ν j X_\tau=x+\nu_j X τ = x + ν j , the process changes by u ( τ , x + ν j ) − u ( τ , x ) u(\tau,x+\nu_j)-u(\tau,x) u ( τ , x + ν j ) − u ( τ , x ) . The expected rate of change due to jumps is precisely ( L u ) ( t , x ) (Lu)(t,x) ( Lu ) ( t , x ) . Combining these contributions:
E [ d M t ∣ X t = x ] = ( ∂ t u + L u ) d t = 0 , (194) \mathbb{E}[dM_t\mid X_t=x]=(\partial_tu+Lu)\,dt=0, \tag*{(194)} E [ d M t ∣ X t = x ] = ( ∂ t u + Lu ) d t = 0 , (194)
where the last equality uses the BKE. Thus M t M_t M t is a martingale, and taking expectations yields u ( 0 , x 0 ) = E [ M 0 ] = E [ M T ] = E [ F ( X T ) ] u(0,x_0)=\mathbb{E}[M_0]=\mathbb{E}[M_T]=\mathbb{E}[F(X_T)] u ( 0 , x 0 ) = E [ M 0 ] = E [ M T ] = E [ F ( X T )] .
Forward Kolmogorov equation (Master equation)
The probability mass function p ( t , x ) = P ( X t = x ) p(t,x)=\mathbb{P}(X_t=x) p ( t , x ) = P ( X t = x ) evolves according to the master equation :
∂ t p ( t , x ) = ∑ j = 1 R [ λ j ( x − ν j ) p ( t , x − ν j ) − λ j ( x ) p ( t , x ) ] . (195) \partial_t p(t,x)=\sum_{j=1}^{R}\left[\lambda_j(x-\nu_j)p(t,x-\nu_j)-\lambda_j(x)p(t,x)\right]. \tag*{(195)} ∂ t p ( t , x ) = j = 1 ∑ R [ λ j ( x − ν j ) p ( t , x − ν j ) − λ j ( x ) p ( t , x ) ] . (195)
This is the adjoint of the generator, describing probability flow into and out of each state: the first term represents inflow from states x − ν j x-\nu_j x − ν j that can jump to x x x , while the second term represents outflow from x x x to other states.
Variational problems with terminal payoff
This appendix derives the Euler–Lagrange and Hamilton equations for variational problems with fixed initial position and a terminal payoff function, yielding the boundary conditions used throughout the main text.
Euler–Lagrange equations
Consider the action
S T [ x ] = ∫ 0 T L ( t , x t , x ˙ t ) d t − f ( x T ) , (196) S_T[x]=\int_0^T L(t,x_t,\dot{x}_t)\,dt-f(x_T), \tag*{(196)} S T [ x ] = ∫ 0 T L ( t , x t , x ˙ t ) d t − f ( x T ) , (196)
to be minimized over paths with fixed initial condition x t = 0 = x 0 x_{t=0}=x_0 x t = 0 = x 0 and free final condition x t = T x_{t=T} x t = T . Let x t x_t x t be an extremum and x ~ t = x t + ϵ δ x t \tilde{x}_t=x_t+\epsilon\,\delta x_t x ~ t = x t + ϵ δ x t a perturbation with δ x 0 = 0 \delta x_0=0 δ x 0 = 0 . The stationarity condition d d ϵ S T [ x ~ ] ∣ ϵ = 0 = 0 \left.\frac{d}{d\epsilon}S_T[\tilde{x}]\right|_{\epsilon=0}=0 d ϵ d S T [ x ~ ] ϵ = 0 = 0 gives
0 = ∫ 0 T ( ∂ L ∂ x t δ x t + ∂ L ∂ x ˙ t δ x ˙ t ) d t − ∇ f ( x T ) ⋅ δ x T . (197) 0=\int_0^T\left(\frac{\partial L}{\partial x_t}\delta x_t+\frac{\partial L}{\partial\dot{x}_t}\delta\dot{x}_t\right)\,\mathrm{d}t-\nabla f(x_T)\cdot\delta x_T. \tag*{(197)} 0 = ∫ 0 T ( ∂ x t ∂ L δ x t + ∂ x ˙ t ∂ L δ x ˙ t ) d t − ∇ f ( x T ) ⋅ δ x T . (197)
Integrating by parts and using δ x 0 = 0 \delta x_0=0 δ x 0 = 0 :
0 = ∫ 0 T ( ∂ L ∂ x t − d d t ∂ L ∂ x ˙ t ) δ x t d t + ( ∂ L ∂ x ˙ t ∣ T − ∇ f ( x T ) ) ⋅ δ x T . (198) 0=\int_0^T\left(\frac{\partial L}{\partial x_t}-\frac{\mathrm{d}}{\mathrm{d}t}\frac{\partial L}{\partial\dot{x}_t}\right)\delta x_t\,\mathrm{d}t+\left(\left.\frac{\partial L}{\partial\dot{x}_t}\right|_T-\nabla f(x_T)\right)\cdot\delta x_T. \tag*{(198)} 0 = ∫ 0 T ( ∂ x t ∂ L − d t d ∂ x ˙ t ∂ L ) δ x t d t + ( ∂ x ˙ t ∂ L T − ∇ f ( x T ) ) ⋅ δ x T . (198)
Since δ x \delta x δ x is arbitrary, we obtain the Euler–Lagrange equation
d d t ∂ L ∂ x ˙ t = ∂ L ∂ x t . (199) \frac{\mathrm{d}}{\mathrm{d}t}\frac{\partial L}{\partial\dot{x}_t}=\frac{\partial L}{\partial x_t}. \tag*{(199)} d t d ∂ x ˙ t ∂ L = ∂ x t ∂ L . (199)
with boundary conditions
x t = 0 = x 0 , (200) x_{t=0}=x_0, \tag*{(200)} x t = 0 = x 0 , (200)
∂ L ∂ x ˙ ∣ T = ∇ f ( x t = T ) . (201) \left.\frac{\partial L}{\partial\dot{x}}\right|_T=\nabla f(x_{t=T}). \tag*{(201)} ∂ x ˙ ∂ L T = ∇ f ( x t = T ) . (201)
Hamilton's equations
The Hamiltonian formulation follows from the Legendre transform. Define the conjugate momentum
θ t = ∂ L ∂ x ˙ t , (202) \theta_t=\frac{\partial L}{\partial\dot{x}_t}, \tag*{(202)} θ t = ∂ x ˙ t ∂ L , (202)
and the Hamiltonian
H ( t , x t , θ t ) = θ t ⋅ x ˙ t − L ( t , x t , x ˙ t ) , (203) H(t,x_t,\theta_t)=\theta_t\cdot\dot{x}_t-L(t,x_t,\dot{x}_t), \tag*{(203)} H ( t , x t , θ t ) = θ t ⋅ x ˙ t − L ( t , x t , x ˙ t ) , (203)
where x ˙ t ( t , x t , θ t ) \dot{x}_t(t,x_t,\theta_t) x ˙ t ( t , x t , θ t ) is obtained by inverting the momentum relation (202) . Taking differentials of (203) :
d H = x ˙ t ⋅ d θ t + θ t ⋅ d x ˙ t − ∂ L ∂ t d t − ∂ L ∂ x t ⋅ d x t − ∂ L ∂ x ˙ t ⋅ d x ˙ t . (204) \mathrm{d}H=\dot{x}_t\cdot\mathrm{d}\theta_t+\theta_t\cdot\mathrm{d}\dot{x}_t-\frac{\partial L}{\partial t}\,\mathrm{d}t-\frac{\partial L}{\partial x_t}\cdot\mathrm{d}x_t-\frac{\partial L}{\partial\dot{x}_t}\cdot\mathrm{d}\dot{x}_t. \tag*{(204)} d H = x ˙ t ⋅ d θ t + θ t ⋅ d x ˙ t − ∂ t ∂ L d t − ∂ x t ∂ L ⋅ d x t − ∂ x ˙ t ∂ L ⋅ d x ˙ t . (204)
Using θ t = ∂ L / ∂ x ˙ t \theta_t=\partial L/\partial\dot{x}_t θ t = ∂ L / ∂ x ˙ t , the d x ˙ t \mathrm{d}\dot{x}_t d x ˙ t terms cancel, yielding
d H = x ˙ t ⋅ d θ t − ∂ L ∂ t d t − ∂ L ∂ x t ⋅ d x t (205) \mathrm{d}H=\dot{x}_t\cdot\mathrm{d}\theta_t-\frac{\partial L}{\partial t}\,\mathrm{d}t-\frac{\partial L}{\partial x_t}\cdot\mathrm{d}x_t \tag*{(205)} d H = x ˙ t ⋅ d θ t − ∂ t ∂ L d t − ∂ x t ∂ L ⋅ d x t (205)
Comparing with d H = ∂ H ∂ x t ⋅ d x t + ∂ H ∂ θ t ⋅ d θ t + ∂ H ∂ t d t \mathrm{d}H=\frac{\partial H}{\partial x_t}\cdot\mathrm{d}x_t+\frac{\partial H}{\partial\theta_t}\cdot\mathrm{d}\theta_t+\frac{\partial H}{\partial t}\,\mathrm{d}t d H = ∂ x t ∂ H ⋅ d x t + ∂ θ t ∂ H ⋅ d θ t + ∂ t ∂ H d t , we identify
∂ H ∂ θ t = x ˙ t , ∂ H ∂ x t = − ∂ L ∂ x t , ∂ H ∂ t = − ∂ L ∂ t . (206) \frac{\partial H}{\partial\theta_t}=\dot{x}_t,\qquad\frac{\partial H}{\partial x_t}=-\frac{\partial L}{\partial x_t},\qquad\frac{\partial H}{\partial t}=-\frac{\partial L}{\partial t}. \tag*{(206)} ∂ θ t ∂ H = x ˙ t , ∂ x t ∂ H = − ∂ x t ∂ L , ∂ t ∂ H = − ∂ t ∂ L . (206)
The Euler–Lagrange equation (199) becomes θ ˙ t = ∂ L / ∂ x t = − ∂ H / ∂ x t \dot{\theta}_t=\partial L/\partial x_t=-\partial H/\partial x_t θ ˙ t = ∂ L / ∂ x t = − ∂ H / ∂ x t , yielding Hamilton’s equations:
x ˙ t = ∂ H ∂ θ t , θ ˙ t = − ∂ H ∂ x t . (207) \dot{x}_t=\frac{\partial H}{\partial\theta_t},\qquad\dot{\theta}_t=-\frac{\partial H}{\partial x_t}. \tag*{(207)} x ˙ t = ∂ θ t ∂ H , θ ˙ t = − ∂ x t ∂ H . (207)
The boundary conditions (200) –(201) become:
x t = 0 = x 0 , θ t = T = ∇ f ( x T ) . (208) x_{t=0}=x_0,\qquad\theta_{t=T}=\nabla f(x_T). \tag*{(208)} x t = 0 = x 0 , θ t = T = ∇ f ( x T ) . (208)
These are the boundary condition (18) in the main text: position is fixed at the initial time, while momentum is determined by the terminal payoff at the final time.
References