betaFeedback

WorksComputer Science

TORNADO: Adaptive Latent Stochastic Transport for Calibrated Probabilistic PDE Forecasting

Abstract

Probabilistic neural surrogates for PDEs enable ensemble forecasting, but often exhibit miscalibration when used for uncertainty quantification, especially in compute-constrained, few-step inference regimes. Existing generative approaches typically rely on fixed global stochasticity, which cannot adapt predictive uncertainty to the local dynamical state. We propose TORNADO, a latent probabilistic forecasting framework that formulates next-step prediction as stochastic transport between VAE encoder posteriors. Unlike existing approaches with globally prescribed stochasticity, TORNADO learns a SDE with state-dependent diffusion, enabling the predictive uncertainty to adapt to the local latent dynamics. The model is trained using analytic conditional endpoint targets derived from a Gaussian interpolant, enabling simulation-free learning. An additional energy distance regularization term improves distributional alignment. Empirically, TORNADO improves uncertainty calibration over representative generative baselines, including stochastic interpolants, diffusion models, and flow matching, while maintaining comparable predictive accuracy. Our results show that adapting stochasticity to the latent state, rather than prescribing it globally, is an effective mechanism for improving calibration in practical neural PDE forecasting settings. Project page: https://tornado-sde.github.io.

Introduction

Neural surrogates for partial differential equations (PDEs) approximate solution operators, enabling fast data-driven prediction of high-dimensional spatiotemporal fields [25] (Lu et al. 2021). Often, solution forecasting is performed in a reduced latent space through an encode–evolve–decode pipeline, which lowers computational cost [49, 35, 52]. However, their reliability can deteriorate under distribution shift [28], they may fail to respect known physical constraints, and small one-step errors can compound in autoregressive rollout [31, 38].

These effects are particularly pronounced in chaotic systems, where small perturbations to a state can produce exponentially diverging trajectories. Deterministic models trained with squared-error losses therefore often degrade over rollout, since pointwise accuracy does not ensure that the forecast preserves the statistical structure of the dynamics [23]. This motivates a conditional probabilistic formulation: given the current state, the model should represent a distribution over plausible future states rather than a single trajectory [8]. Probabilistic forecasting also provides a basis for uncertainty quantification [5], since predictive spread can be used to assess forecast confidence. Crucially, probabilistic forecasts are only informative when their uncertainty is well calibrated, i.e., when predictive spread matches actual forecast uncertainty. Generative models provide a natural route to modeling high-dimensional forecast distributions, motivating the use of diffusion models, flow matching, and stochastic interpolants for probabilistic PDE forecasting and operator learning [46, 34, 54, 19]. However, sampling along a generative bridge requires repeated network evaluations, with computational cost increasing with the number of sampling steps. This cost is particularly restrictive in autoregressive ensemble forecasting, motivating few-step methods.

Additionally, sample diversity is not the same as calibrated uncertainty. In conditional forecasting, generative time-steppers may become underdispersed, with samples collapsing toward a conditional mean instead of covering the range of plausible futures, especially in few-step settings [10, 17, 3]. This often results in overconfident generative surrogates, which can be particularly harmful for complex and chaotic systems [29]. Classical stochastic interpolants (SI) inject randomness through a prescribed inference-time diffusion coefficient, but this stochasticity is state-agnostic. Since physical predictability varies across states and regimes, calibrated uncertainty requires adaptive stochasticity.

We introduce TORNADO (laTent pOsteRioR traNsport with ADaptive diffusiOn), a framework for calibrated probabilistic forecasting in latent-space PDE surrogates. Given consecutive PDE states, TORNADO models the one-step transition between their variational autoencoder (VAE) encoder posteriors using a learned Itô SDE. Our central contribution is to equip this stochastic transport with state-dependent diffusion. In contrast to fixed-schedule generative baselines, this learned diffusion acts as an adaptive uncertainty mechanism, allowing predictive spread to vary across states and dynamical regimes, and enabling calibrated, heteroscedastic forecasts. To instantiate this idea, we introduce the following key methodological contributions:

  1. Posterior-to-posterior latent forecasting. We formulate PDE stepping as an explicit distribution-matching problem between the VAE encoder posteriors at consecutive times. We theoretically justify this as a representation-consistent target, and empirically find that it mitigates the overconfident mean-matching behavior often observed in few-step generative time-steppers.

  1. Analytic local endpoint supervision. We introduce a simulation-free local supervision objective for the endpoint transport without differentiating through an SDE solver. This is achieved by defining a Gaussian interpolant over an auxiliary time τ\tau, from which we derive analytic conditional endpoint targets for local supervision.

  1. Adaptive, state-dependent diffusion. We learn both drift and diffusion fields for the latent SDE by matching the conditional endpoint mean and variance, augmented with an energy-distance term. This turns the diffusion field into a learned state-dependent uncertainty mechanism, enabling heteroscedastic probabilistic predictions that adapt across trajectories.

Background

Problem setting

We focus on time-dependent PDEs defined on a spatial domain Ω⊂Rd\Omega\subset\mathbb{R}^{d} over a time interval [0,T][0,T]. Each instance is defined by an initial condition u0(x)\mathbf{u}^{0}(x), boundary conditions b∈L2(∂Ω×[0,T],Rdb)\mathbf{b} \in L^{2}(\partial\Omega\times[0,T],\mathbb{R}^{d_b}), a forcing term f∈L2(Ω×[0,T],Rdf)\mathbf{f} \in L^{2}(\Omega\times[0,T],\mathbb{R}^{d_f}), and/or PDE coefficients c\mathbf{c}. The governing system is:

N[u;c,f](x,t)=0,(1)\mathcal{N}[\mathbf{u};\mathbf{c},\mathbf{f}](x,t)=0, \tag*{(1)}
B[u;b](x,t)=0,(2)\mathcal{B}[\mathbf{u};\mathbf{b}](x,t)=0, \tag*{(2)}
u(x,0)=u0(x),(3)\mathbf{u}(x,0)=\mathbf{u}^{0}(x), \tag*{(3)}

where N\mathcal{N} is a (possibly nonlinear) differential operator, B\mathcal{B} encodes the boundary conditions, and u:[0,T]×Ω→Rdu\mathbf{u} : [0,T] \times\Omega\to\mathbb{R}^{d_u} denotes the solution field. We denote u(x,t)=ut\mathbf{u}(x,t)=u_t for simplicity, where the subscript tt denotes time step indices separated by some Δt\Delta t. Given a trained model G\mathcal{G} as a next-step forecaster: given the current state utu_t, it defines a conditional distribution over the next state. At inference, we evaluate it as a probabilistic predictor by drawing multiple samples {u^t+1(k)}k=1K∼G(⋅∣ut)\{\hat{u}_{t+1}^{(k)}\}_{k=1}^{K} \sim\mathcal{G}(\cdot\mid u_t), yielding an ensemble of candidates (and, under rollout, an ensemble of trajectories).

Diffusion, Flows, and Stochastic Interpolants

Transport-based generative models (e.g., normalizing flows and diffusion models) generate samples from a complex target density ρ1\rho_1 by continuously transporting samples drawn from a simple prior density ρ0\rho_0. This transport is defined by an ordinary or stochastic differential equation (ODE/SDE). SI provide a common view of these methods: training amounts to learning the vector field (or velocity) that reproduces a prescribed probability path between arbitrary ρ0\rho_0 and ρ1\rho_1 Albergo et al. [1].

Both flow matching (FM) and SI learn a velocity field fθf_\theta. In FM, sampling proceeds by integrating the corresponding ODE, whereas in SI one may instead use an SDE realization in which the injected noise is prescribed by an auxiliary-time diffusion schedule g(τ)g(\tau). For example, for the Euler-Maruyama SDE solver, a step is computed as:

zτ+Δτ=zτ+Δτfθ(zτ,τ)+Δτ g(τ)⊙ξ,(4)z_{\tau+\Delta\tau} = z_\tau+ \Delta\tau f_\theta(z_\tau,\tau) + \sqrt{\Delta\tau}\,g(\tau) \odot\xi, \tag*{(4)}

with ξ∼N(0,I)\xi\sim\mathcal{N}(0,I). A key limitation of classical SI for PDE forecasting is the use of this global time-dependent noise schedule g(τ)g(\tau). Since g(τ)g(\tau) is only a function of τ\tau, the injected stochasticity is state-agnostic. In the few-step regime targeted here, this makes it difficult to adapt predictive spread to the local dynamics. We therefore learn a state-dependent diffusion amplitude so that uncertainty can adapt directly to the local regime.

Method

TORNADO parameterizes a latent Itô SDE over an auxiliary time τ∈[0,1]\tau\in[0,1] with learned drift and state-dependent diffusion. To train this stochastic step model, we need a tractable supervision signal for both drift and diffusion, avoiding the cost of pathwise differentiation through the SDE solver. We first describe the inference procedure to make explicit how the learned SDE generates the next latent state (§ 3.1). We then justify the encoder posterior of the next snapshot as the supervision target for the latent predictor (§ 3.2). To obtain such supervision, we exploit the Gaussianity of the encoder posteriors and introduce an auxiliary-time Gaussian interpolant (§ 3.3). The resulting analytic conditional targets make training simulation-free and also support an energy-distance regularizer on sampled endpoint updates (§ 3.4).

Inference by SDE integration

Building on the SDE formulation of SI, we generate the next latent state by integrating a learned Itô SDE over an auxiliary interpolation time τ∈[0,1]\tau\in[0,1]:

dzτ=fθ(zτ,τ) dτ+gθ′(zτ,τ) dWτ,(5)\mathrm{d}z_\tau= f_\theta(z_\tau,\tau)\,\mathrm{d}\tau+ g_\theta^{\prime}(z_\tau,\tau)\,\mathrm{d}W_\tau, \tag*{(5)}

where WτW_\tau is a standard Wiener process. The drift fθf_\theta and the state-dependent diffusion amplitude gθ′g_\theta^{\prime} are parameterized by neural networks. Unlike classical SI that prescribe a global diffusion schedule, TORNADO learns both drift and diffusion directly from data, with endpoint supervision derived from consecutive latent posteriors.

The auxiliary time τ\tau distinguishes this formulation from simpler one-step probabilistic predictors. We show in Appendix § 16.§ 2 that deterministic (μ,σ)(\mu,\sigma) regression, MC-Dropout, and direct moment matching produce miscalibrated samples and faster error accumulation on our benchmarks.

At inference time, generating u^t+1\hat{u}_{t+1} from utu_t proceeds as follows.

Inference procedure for generating the next latent state

Figure 1.

We encode utu_t into the posterior qϕ(zt∣ut)q_\phi(z_t \mid u_t) and draw an initial sample ztz_t. Independent realizations of the learned SDE produce an ensemble {z^t+1(k)}k=1K\{\hat{z}_{t+1}^{(k)}\}_{k=1}^{K}, which is decoded to physical space as {u^t+1(k)}k=1K\{\hat{u}_{t+1}^{(k)}\}_{k=1}^{K}. For multi-step forecasting, each ensemble member is propagated autoregressively as the next initial condition, with one SDE realization per member at each step. This yields a predictive ensemble over future states.

Inference pipeline of TORNADO

Figure 1 (PDF p. 4). Inference pipeline of TORNADO. The encoder EϕE_{\phi} maps the current PDE state utu_t to a latent posterior qϕ(zt∣ut)q_{\phi}(z_t \mid u_t), from which a latent sample ztz_t is drawn. Starting from the encoded latent state, we draw KK independent realizations of the learned latent SDE over the auxiliary time τ∈[0,1]\tau\in[0,1], using drift fθf_{\theta} and diagonal diffusion gθ′g_{\theta'} predicted by PΘP_{\Theta}. The decoder DϕD_{\phi} maps the resulting latent ensemble to a predictive distribution over ut+1u_{t+1}, yielding a mean u^t+1\widehat{u}_{t+1} and a 95% confidence interval CI95(u^t+1)\mathrm{CI}_{95}(\widehat{u}_{t+1}).

We integrate the SDE numerically via Euler–Maruyama (EM) for NN steps with step Δτ=1/N\Delta\tau= 1/N:

z~n+1(k)=z~n(k)+Δτfθ(z~n(k),τn)+Δτ gθ′(z~n(k),τn)⊙ξn,(6)\widetilde{z}_{n+1}^{(k)} = \widetilde{z}_{n}^{(k)} + \Delta\tau f_{\theta}(\widetilde{z}_{n}^{(k)}, \tau_n) + \sqrt{\Delta\tau}\, g_{\theta'}(\widetilde{z}_{n}^{(k)}, \tau_n) \odot\xi_n, \tag*{(6)}

for n=0,…,N−1n = 0,\ldots,N-1, with ξn∼N(0,I)\xi_n \sim\mathcal{N}(0,I). We refer to the number of integration steps NN as the number of forward evaluations (NFE). The inference procedure is summarized in Algorithm 2 (Appendix § 9) and illustrated in Figure 1. We study the impact of integrating with higher order solvers in Appendix § 15.§ 4.

Objective: latent posterior transport

A common setting for neural PDE surrogates is to learn the time evolution model in a reduced latent space, where PDE states are first compressed into a lower-dimensional representation. Concretely, we use a VAE, where the encoder EϕE_{\phi} maps PDE solution fields u∈Rduu \in\mathbb{R}^{d_u} to a diagonal Gaussian posterior

qϕ(z∣u)=N(μϕ(u),σϕ2(u)),z∈Rdz, dz≪du,q_{\phi}(z \mid u) = \mathcal{N}\left(\mu_{\phi}(u), \sigma_{\phi}^{2}(u)\right), \quad z \in\mathbb{R}^{d_z},\ d_z \ll d_u,

and the decoder DϕD_{\phi} defines a conditional likelihood pϕ(u∣z)p_{\phi}(u \mid z). Apart from allowing for reduced compute with the compressed representations, the VAE turns each individual snapshot uu into a local continuous distribution qϕ(z∣u)q_{\phi}(z \mid u), from which we can sample different latent representations of the same field. Given a pair of consecutive time frames (ut,ut+1)(u_t,u_{t+1}), encoding both elements yields two local posteriors qϕ(zt∣ut)q_{\phi}(z_t \mid u_t) and qϕ(zt+1∣ut+1)q_{\phi}(z_{t+1} \mid u_{t+1}).

Encoder posterior as target. Since zt+1z_{t+1} is unobserved, the only supervision available for the next latent state comes from the encoder posterior qϕ(zt+1∣ut+1)q_{\phi}(z_{t+1} \mid u_{t+1}). Combined with the data dynamics, this induces a target distribution over zt+1z_{t+1} given utu_t, obtained by averaging the encoder posterior over likely next states:

p~(zt+1∣ut):=Eut+1∼pdata(⋅∣ut)[qϕ(zt+1∣ut+1)].\widetilde{p}(z_{t+1} \mid u_t) := \mathbb{E}_{u_{t+1} \sim p_{\mathrm{data}}(\cdot\mid u_t)}\left[q_{\phi}(z_{t+1} \mid u_{t+1})\right].

The following result shows that this encoder-induced distribution is the natural latent target for a probabilistic step model. Let pθ(zt+1∣ut)p_{\theta}(z_{t+1} \mid u_t) denote a candidate conditional latent step model.

L(θ):=E(ut,ut+1)∼pdata[KL⁡(qϕ(z∣ut+1) ∥ pθ(z∣ut))].(7)\mathcal{L}(\theta) := \mathbb{E}_{(u_t,u_{t+1})\sim p_{\mathrm{data}}}\left[\operatorname{KL}\left(q_\phi(z \mid u_{t+1}) \,\|\, p_\theta(z \mid u_t)\right)\right]. \tag*{(7)}

We use a forward-KL reference objective because, when averaged over possible next states, its minimizer is exactly the encoder-averaged distribution p~(⋅∣ut)\widetilde{p}(\cdot\mid u_t):

Proposition 3.1. The objective in Eq. (7) is minimized when

pθ(zt+1∣ut)=p~(zt+1∣ut),p_\theta(z_{t+1} \mid u_t) = \widetilde{p}(z_{t+1} \mid u_t),

i.e., when the step model matches the encoder-induced latent target.

This identifies p~(zt+1∣ut)\widetilde{p}(z_{t+1} \mid u_t) as the appropriate latent supervision target. The proof of Proposition 3.1 is provided in Appendix § 8. This ties the calibration of latent probabilistic predictors to the calibration of the underlying VAE posterior, a dependence we study empirically in Appendix § 15.§ 2. We next construct a tractable local supervision scheme for the drift and diffusion dynamics.

Gaussian interpolant and conditional endpoint targets

To obtain training samples, we construct an interpolant between independent endpoint samples z0∼qϕ(zt∣ut)z_0 \sim q_\phi(z_t \mid u_t) and z1∼qϕ(zt+1∣ut+1)z_1 \sim q_\phi(z_{t+1} \mid u_{t+1}) similarly to SI:

zτ=(1−τ)z0+τz1+γ(τ) ξ,ξ∼N(0,I),(8)z_\tau= (1-\tau)z_0 + \tau z_1 + \sqrt{\gamma(\tau)}\,\xi,\qquad\xi\sim\mathcal{N}(0,I), \tag*{(8)}

where γ(τ)=σγτ(1−τ)\gamma(\tau) = \sigma_\gamma\tau(1-\tau) and σγ\sigma_\gamma is a tunable hyperparameter. We study its effect empirically in Appendix § 15.7 and analyze limiting regimes in Appendix § 7.§ 3. Overall, σγ>0\sigma_\gamma> 0 is needed to obtain well-conditioned supervision, while large values can degrade calibration.

Learning the diffusion from endpoint pairs alone is underdetermined, as many drift–diffusion pairs can induce similar endpoint distributions, while supervising diffusion through full SDE trajectories would require differentiating through the solver at substantial cost. We therefore approximate the remaining evolution to the endpoint with a local one-step transition conditioned on the bridge state (zτ,τ)(z_\tau,\tau):

z^1∣zτ=zτ+(1−τ)fθ(zτ,τ)+1−τ gθ′(zτ,τ)⊙ϵ,ϵ∼N(0,I).(9)\widehat{z}_1 \mid z_\tau= z_\tau+ (1-\tau)f_\theta(z_\tau,\tau) + \sqrt{1-\tau}\,g'_\theta(z_\tau,\tau)\odot\epsilon,\qquad\epsilon\sim\mathcal{N}(0,I). \tag*{(9)}

Here, the drift determines the remaining conditional mean displacement to the endpoint via (1−τ)fθ(zτ,τ)(1-\tau)f_\theta(z_\tau,\tau), while the diffusion determines the corresponding spread via (1−τ)gθ′(zτ,τ)2(1-\tau)g'_\theta(z_\tau,\tau)^2. The natural supervision targets are therefore the conditional endpoint mean and covariance of z1∣zτz_1 \mid z_\tau.

Ldrift=Eτ,zτ[∥(1−τ)fθ(zτ,τ)−(μtarget−zτ)∥22],Lvar=Eτ,zτ[∥(1−τ)gθ′(zτ,τ)2−Σtarget∥22].(10)\begin{aligned} \mathcal{L}_{\mathrm{drift}} &= \mathbb{E}_{\tau,z_\tau}\left[\left\|(1-\tau)f_\theta(z_\tau,\tau)-(\mu_{\mathrm{target}}-z_\tau)\right\|_2^2\right],\\ \mathcal{L}_{\mathrm{var}} &= \mathbb{E}_{\tau,z_\tau}\left[\left\|(1-\tau)g'_\theta(z_\tau,\tau)^2-\Sigma_{\mathrm{target}}\right\|_2^2\right]. \tag*{(10)} \end{aligned}

This construction provides local endpoint supervision rather than a constraint on the full continuous-time marginal path. Accordingly, SDE integration is used as a finite-step sampler. We set (μtarget,Σtarget)=(μ1∣τ,Σ1∣τ)(\mu_{\mathrm{target}},\Sigma_{\mathrm{target}})=(\mu_{1\mid\tau},\Sigma_{1\mid\tau}), whose conditional endpoint moments admit the closed-form expression below:

Proposition 3.2 (Conditional endpoint law under the Gaussian interpolant). Under the above setup, let μ1∣τ\mu_{1\mid\tau} and Σ1∣τ\Sigma_{1\mid\tau} denote the conditional mean and covariance of z1z_1 given zτz_\tau. Then the conditional moments are given explicitly by:

μ1∣τ=μ1+τΣ1Στ−1(zτ−μτ),\mu_{1\mid\tau} = \mu_1 + \tau\Sigma_1\Sigma_\tau^{-1}(z_\tau-\mu_\tau),
Σ1∣τ=Σ1−τ2Σ1Στ−1Σ1,\Sigma_{1\mid\tau} = \Sigma_1-\tau^2\Sigma_1\Sigma_\tau^{-1}\Sigma_1,

where μτ=(1−τ)μ0+τμ1\mu_\tau=(1-\tau)\mu_0+\tau\mu_1 and Στ=(1−τ)2Σ0+τ2Σ1+γ(τ)I\Sigma_\tau=(1-\tau)^2\Sigma_0+\tau^2\Sigma_1+\gamma(\tau)I.

Proposition 3.2 follows from standard Gaussian conditioning; derivation provided in Appendix § 7.§ 2.

Distributional regularization via energy distance (ED)

The drift and variance losses match the conditional endpoint moments directly, but they supervise the two components separately. We additionally use ED as a sample-level regularizer, comparing samples z^1(k)\hat{z}_1^{(k)} from the one-step transition (9) and samples ztarget(ℓ)z_{\mathrm{target}}^{(\ell)} from the target conditional Gaussian N(μtarget,Σtarget)\mathcal{N}(\mu_{\mathrm{target}}, \Sigma_{\mathrm{target}}).

LED=Eτ,zτ[ED⁡({z^1(k)}k=1K,{ztarget(ℓ)}ℓ=1L)],(11)\mathcal{L}_{\mathrm{ED}} = \mathbb{E}_{\tau,z_\tau}\left[\operatorname{ED}\left(\{\hat{z}_1^{(k)}\}_{k=1}^{K}, \{z_{\mathrm{target}}^{(\ell)}\}_{\ell=1}^{L}\right)\right], \tag*{(11)}

with ED⁡(P,Q)=2E∥X−Y∥−E∥X−X′∥−E∥Y−Y′∥\operatorname{ED}(P,Q) = 2\mathbb{E}\lVert X-Y\rVert- \mathbb{E}\lVert X-X'\rVert- \mathbb{E}\lVert Y-Y'\rVert, X,X′∼PX,X'\sim P, Y,Y′∼QY,Y'\sim Q i.i.d.

Our final objective is

L=Ldrift+λvarLvar+λEDLED,(12)\mathcal{L} = \mathcal{L}_{\mathrm{drift}} + \lambda_{\mathrm{var}}\mathcal{L}_{\mathrm{var}} + \lambda_{\mathrm{ED}}\mathcal{L}_{\mathrm{ED}}, \tag*{(12)}

where λvar\lambda_{\mathrm{var}} and λED\lambda_{\mathrm{ED}} are ramped up from 0 after a drift warm-up stage. The training regime is summarized in Algorithm 1 (Appendix § 9). A study of the impact of each loss component and their weights is carried out in Appendix § 15.§ 1.

Experiments

We evaluate TORNADO’s ability to produce calibrated predictive distributions under compute-constrained, few-step inference. We compare against standard generative baselines on four PDE datasets using four metrics computed in physical space capturing mean accuracy, distributional quality, calibration, and spread. Our empirical goal is to test whether learning state-dependent diffusion improves calibration at fixed inference budget while remaining competitive in predictive accuracy.

Experimental setting

Datasets Synthetic data. We use APEBench ([26]) with default settings to generate 2D Kolmogorov flow, Kuramoto–Sivashinsky (KS), Burgers, and Gray–Scott reaction–diffusion (GS) datasets on a 160×160160 \times160 grid, with 1024 training and 256 test trajectories each (GS: 256/64). Real-world data. We use measured flame imaging from the Combustion scenario of RealPDEBench ([20]) and ERA5 atmospheric reanalysis ([15]) obtained through WeatherBench ([47]). The combustion dataset contains 30 recorded trajectories of 2001 128×128128 \times128 frames, each at a unique (NH3\mathrm{NH}_3 blend, equivalence ratio) condition; 26 trajectories are used for training and 4 for testing on unseen conditions. For ERA5, we use 6-hourly 121×240121 \times240 fields at 1.5° resolution: 2-m temperature (T2mT_{2m}), 10-m zonal and meridional wind (U10U_{10}, V10V_{10}), and 500 hPa geopotential (Z500Z_{500}), with calendar-year splits of 2010–2015 for training, 2016 for validation, and 2017–2018 for testing. Further details are provided in Appendix § 11.

Baselines As a generative framework, TORNADO’s direct counterparts are diffusion, flow matching, and stochastic interpolants. We use those three as baselines to ensure that differences in performance can be attributed to the transport and uncertainty mechanisms rather than to model scale or task-specific architectural optimization. All baselines share the same trained VAE, conditioning, latent-processor backbone, and inference budget (measured in NFE), differing in auxiliary-time construction, training objective, and sampler.

  • Diffusion (DDIM). Discrete noising with N=100N=100 steps, trained to predict Gaussian noise and sampled with DDIM ([51]).

  • Flow Matching (FM). Noise-to-data conditional FM ([37]) with a linear interpolant and Euler sampling.

  • Stochastic Interpolants (SI). ([1]) Continuous-time adaptation of Zhou et al. [54], sampled with EM.

Implementation details and comparisons with direct one-step probabilistic baselines appear in Appendices § 12 and § 16.§ 2, respectively.

Metrics. The experiments characterize probabilistic surrogates along three axes: (i) accuracy of the central prediction, (ii) quality of the predictive distribution, and (iii) calibration, separating reliability from trivial over/under-dispersion. Accordingly, we report Mean Squared Error (MSE) for mean accuracy, Continuous Ranked Probability Score (CRPS) as a strictly proper score for distributional quality, Root Mean Squared Calibration Error (RMSCE) to quantify calibration, and Spread–Skill ratio (SSR) as a complementary diagnostic to detect over/under-dispersion. We also report ∣log⁡(SSR)∣|\log(\mathrm{SSR})| so that lower values indicate better performance, consistent with the directionality of the other metrics, for clarity. All metrics are computed in physical space and averaged over spatial grid points, channels, batch elements, and over the reported prediction horizon (except on ERA5, where metrics use normalized units and cos⁡(lat)\cos(\mathrm{lat}) weighting). Further details in Appendix § 13.

Architecture and experimental setting. VAE: All methods operate in the same latent space given by a Deep Compression Autoencoder (DCAE) ([6]) trained with a standard β\beta-VAE objective ([16]). We train one VAE instance for each dataset. Processor: TORNADO and all baselines share a Transformer trunk PΘP_{\Theta} (DiT-style ([45]). The trunk’s output branches into a drift head fθf_{\theta} and a diffusion head gθ′g_{\theta'}. The drift head consists of a final per-token projection followed by an unpatchify operation. The diffusion head, instantiated only for TORNADO, uses the same decoder preceded by an additional transformer block to parameterize the state-dependent diffusion amplitude; it is detached (stopgrad) from the shared trunk by default, so its gradients do not increase the drift model’s representation capacity. More details are in Appendix § 14.

Training protocol. We adopt one-step teacher-forced training as a controlled simplification shared by all methods; this isolates differences in probabilistic transport and calibration without conflating them with rollout-training effects. This choice is made for comparability rather than necessity: TORNADO is not limited to teacher forcing, and we additionally report an autoregressive extension in Appendix § 16.§ 1. All methods are trained on randomly sampled consecutive pairs (ut,ut+1)(u_t, u_{t+1}). For methods that sample an interpolation time, we draw τ∼U(τmin⁡,1−τmin⁡)\tau\sim U(\tau_{\min},1-\tau_{\min}) with τmin⁡=10−3\tau_{\min}=10^{-3} to avoid numerical issues near the endpoints.

Evaluation protocol and compute budget. We evaluate probabilistic forecasts under autoregressive (AR) rollout with a fixed ensemble size K=16K=16 and focus on compute-constrained sampling with small NFE. Unless stated otherwise, we report results at NFE=5. We choose NFE=5 as a representative low-step regime: it reflects the compute constraints of AR ensemble forecasting, while giving all iterative baselines enough integration steps for a meaningful comparison. Additional KK and NFE sweeps are deferred to Appendix § 15.8 and Appendix § 16.7 respectively. Rollout horizons are specified per experiment to account for dataset-dependent trajectory lengths and error accumulation.

Performance of predictive distributions

Table 1 reports interval metrics over rollout steps 1–20, except ERA5 (1–40), across the four synthetic PDE and two real-world datasets at fixed compute budget (NFE=5). Across all four synthetic datasets, TORNADO achieves the lowest calibration error at fixed budget, with RMSCE reductions ranging from roughly 2× to over 5× relative to the baselines on representative cases such as Kolmogorov and Burgers. It also yields substantially better spread–skill agreement, with ∣log⁡(SSR)∣|\log(\mathrm{SSR})| reduced by around 7× on KS and Gray–Scott relative to SI, and even more relative to DDIM and FM. These gains come while maintaining competitive MSE and CRPS in the few-step regime targeted.

Flow matching is consistently strong in mean accuracy, but it is systematically miscalibrated. Its large ∣log⁡(SSR)∣|\log(\mathrm{SSR})| values, together with large RMSCE, indicate under-dispersed predictive ensembles, i.e., the model concentrates too much probability mass around the ensemble mean. This aligns with prior observations that flow-matching-style training can behave primarily as a conditional mean matcher in low-step forecasting regimes, rather than faithfully representing the full predictive distribution. By contrast, the remaining baselines do not exhibit a uniform behavior across scenarios. DDIM and SI show mixed behavior across PDEs, suggesting that fixed schedules/noise parameterizations interact strongly with the dataset’s intrinsic stochasticity and error growth.

TORNADO achieves the lowest RMSCE on both real-world datasets. On Combustion, it also leads in MSE and CRPS. SI’s slightly better SSR should be interpreted in light of its higher predictive MSE. On ERA5, TORNADO nearly matches SI’s leading MSE and CRPS while improving RMSCE by approximately 10% and achieving the best spread–skill agreement.

Calibration diagnostics

RMSCE quantifies calibration error and ∣log⁡SSR∣|\log\mathrm{SSR}| spread–skill mismatch, but neither reveals whether ensembles are over- or underdispersed. We therefore examine probability integral transform (PIT) histograms. At each spatial location, the predictive ensemble’s empirical CDF evaluated at the ground truth gives the fraction of predictions at or below it. For calibrated forecasts, the ground truth falls below each predicted percentile with the expected frequency, yielding uniformly distributed PIT values. Bell- and U-shaped distributions indicate excessive and insufficient spread, respectively. Figure 2 shows PIT histograms pooled over spatial locations, trajectories, and rollout steps 1–20 on Kolmogorov flow (K=16K=16, NFE=5NFE=5). Despite similar RMSCE, SI is bell-shaped while DDIM is U-shaped, with FM showing an even stronger concentration at the extremes. TORNADO is closest to uniform, with only moderate excess mass at the boundaries. More details in Appendix § 16.§ 4

PIT histograms for TORNADO, FM, DDIM, and SI

Figure 2. PIT histograms on Kolmogorov flow NFE=5NFE=5, K=16K=16, 20 rollout steps

Analysis over long rollouts

Figure 3 shows per time-step rollout metrics up to T=50T = 50 (NFE=5). MSE exhibits a clear elbow around t≈20t \approx20, after which errors grow more rapidly. We still report MSE because it marks the onset of error amplification, but beyond that point it becomes a limited proxy for forecast quality in chaotic regimes. We report in Appendix § 16.§ 3 a complementary long-horizon evaluation on the Kolmogorov Flow dataset based on total enstrophy, a more meaningful statistic once pointwise metrics become less informative, and find that TORNADO remains closest to the ground-truth distribution. Calibration metrics tell a different story. Before the elbow, TORNADO retains the strongest uncertainty quality, with lower RMSCE and CRPS, and SSR closer to 1. Later, SI appears more competitive because its initial overspread is partially offset by the accumulated MSE, bringing SSR closer to 1. This should not be interpreted as SI learning a better uncertainty model at long horizons. Once the rollout leaves the stable regime, calibration metrics become increasingly influenced by compounding forecast error. Figure 4 illustrates one rollout trajectory over 50 steps on the Kolmogorov dataset. As forecast error accumulates, TORNADO widens its predicted 95% confidence interval accordingly, keeping it of comparable magnitude to the realized error over rollout.

Kolmogorov flow rollout metrics vs. horizon $t$ at fixed compute

Figure 3. Kolmogorov flow rollout metrics vs. horizon tt at fixed compute (NFE=5): MSE, CRPS, RMSCE, and SSR. Lower is better for all metrics except SSR (ideal value 1).

Rollout visualization for $T = 50$ on the Kolmogorov Flow dataset

Figure 4. Rollout visualization for T=50T = 50 on the Kolmogorov Flow dataset. Rows top to bottom: ground truth trajectory, TORNADO mean prediction, pixel-wise mean prediction error, pixel-wise predictive ensemble 95% confidence interval.

Ablation on adaptive diffusion

To isolate the effect of learning a state-dependent diffusion term gg, we perform an inference ablation. With the same trained drift and same NFE=5, we compare the distributional quality of the ensembles produced by performing the rollout with different integration techniques:

  1. Euler: deterministic (g=0g = 0),

  2. EM - Constant amplitude (g=gˉθ′g = \bar{g}_{\theta'}), where gˉθ′\bar{g}_{\theta'} is the empirical average value of gθ′(zτ,τ)g_{\theta'}(z_\tau, \tau) over the validation set,

  3. EM - Fixed schedule (g(τ)=γ(τ)g(\tau) = \sqrt{\gamma(\tau)}), where γ(τ)=σscaleτ(1−τ)\gamma(\tau) = \sigma_{\mathrm{scale}}\tau(1-\tau) as in the interpolant,

  4. EM - TORNADO (g=gθ′(zτ,τ)g = g_{\theta'}(z_\tau, \tau)).

MethodMSE ↓\downarrowCRPS ↓\downarrowRMSCE ↓\downarrow∣log⁡(SSR)∣↓\lvert\log(\mathrm{SSR})\rvert\downarrow
Euler0.4390.3510.2869.674
Constant Amplitude2.7010.8100.0850.602
Fixed Schedule0.4880.2910.0670.361
TORNADO0.3710.2410.0390.264

Table 2. Learned state-dependent adaptive diffusion (TORNADO) vs. other integration methods for a same drift, matched NFE=5, averaged over rollout time T=33T = 33 for the Kolmogorov Flow dataset.

Table 2 reports metrics on the Kolmogorov Flow test set for a shared learned drift. The deterministic Euler method achieves competitive mean accuracy but the lack of stochasticity yields under-dispersed, overconfident ensembles with poor calibration. Injecting diffusion with a constant amplitude or with a fixed schedule remains suboptimal: they cannot adapt the noise level to the local state, leading to mismatched dispersion and degraded distributional scores. In contrast, learning a state-dependent adaptive diffusion term allows for more calibrated predictive distributions, with spread better aligned to forecast error and stable mean accuracy over the rollout.

Additional ablations and experiments: The Appendix provides additional ablations on diffusion loss components and weights (Ap. I.1), the impact of VAE calibration (Ap. I.2), posterior quality and its effect on rollouts (Ap. I.3), higher-order solvers (Ap. I.4), gradient stopping at the diffusion head (Ap. I.5), compute time (Ap. I.6), the impact of σγ\sigma_\gamma (Ap. I.7), and ensemble size KK (Ap. I.8). Additional experiments cover autoregressive training (Ap. J.1), comparisons with one-step baselines (Ap. J.2), long-horizon statistical evaluation (Ap. J.3), calibration and sharpness beyond RMSCE (Ap. J.4), error versus variance allocation (Ap. J.5), ERA5 per-variable results (Ap. J.6), and the impact of NFE (Ap. J.7).

Conclusion

TORNADO addresses calibrated probabilistic PDE forecasting in the compute-constrained, few-step regime, where fixed stochasticity can inadequately capture dynamical variability. Its learned state-dependent diffusion adapts predictive uncertainty to the local regime, consistently improving calibration over representative generative baselines at matched inference cost while maintaining competitive accuracy. Controlled ablations show that learned state-dependent diffusion improves calibration over constant or globally scheduled alternatives, and TORNADO also outperforms standard stochastic interpolants with fixed diffusion in calibration. These results highlight adaptive stochasticity as a key ingredient for reliable probabilistic neural PDE surrogates.

Limitations The quality of the learned uncertainty is tied to the latent supervision target induced by the VAE posterior. Appendix § 15.§ 2 shows that better latent calibration improves reliability, motivating more expressive, better-calibrated representations. Additionally, our objective supervises local endpoint moments rather than explicitly enforcing consistency with the interpolant’s full continuous-time marginal evolution. While our NFE and solver ablations support its empirical effectiveness, further characterizing how local supervision and numerical discretization jointly shape the resulting endpoint distribution remains an interesting direction for future work.

Acknowledgements

We acknowledge the financial support provided by DL4CLIM (ANR-19-CHIA-0018-01), DEEPNUM (ANR-21-CE23-0017-02), PHLUSIM (ANR-23-CE23-0025-02), MELISSA (ANR-24-CE23-7140-01), PEPR-AR (ANR-23-PEIA0008, ANR, FRANCE 2030), and cluster IA PostGenAI@Paris (ANR-23-IACL-0007).

This project was provided with computer and storage resources by GENCI at IDRIS thanks to the grants 2025-AD011015481, 2025-A0191016890, 2026-A0211016890 on the supercomputer Jean Zay’s V100/A100/H100 partitions.

Theoretical Analysis of Latent Posterior Transport

Background: Gaussian conditioning identity

We use the standard conditioning formula for jointly Gaussian random variables. If (X,Y)(X,Y) are jointly Gaussian with means (μX,μY)(\mu_X,\mu_Y) and covariances Var⁡(X)\operatorname{Var}(X), Var⁡(Y)\operatorname{Var}(Y), Cov⁡(Y,X)\operatorname{Cov}(Y,X), then

Y∣X=x∼N(μY+Cov⁡(Y,X)Var⁡(X)−1(x−μX),Var⁡(Y)−Cov⁡(Y,X)Var⁡(X)−1Cov⁡(X,Y)).Y \mid X=x \sim\mathcal{N}\left(\mu_Y+\operatorname{Cov}(Y,X)\operatorname{Var}(X)^{-1}(x-\mu_X),\operatorname{Var}(Y)-\operatorname{Cov}(Y,X)\operatorname{Var}(X)^{-1}\operatorname{Cov}(X,Y)\right).

This identity is all we need to derive the closed-form conditional targets in Proposition A.1 (Similar formulas also appear in Gaussian Schrödinger-bridge references Bunne et al. [4]).

Main Results

Let qϕ(zt∣ut)=N(μ0,Σ0)q_\phi(z_t \mid u_t)=\mathcal{N}(\mu_0,\Sigma_0) and qϕ(zt+1∣ut+1)=N(μ1,Σ1)q_\phi(z_{t+1} \mid u_{t+1})=\mathcal{N}(\mu_1,\Sigma_1) denote the diagonal Gaussian posteriors of consecutive PDE states, with Σ0,Σ1≻0\Sigma_0,\Sigma_1 \succ0. Define the stochastic interpolant:

zτ=(1−τ)z0+τz1+γ(τ)ξ,τ∈(0,1),z_\tau=(1-\tau)z_0+\tau z_1+\sqrt{\gamma(\tau)}\xi,\qquad\tau\in(0,1),

where z0∼N(μ0,Σ0)z_0\sim\mathcal{N}(\mu_0,\Sigma_0), z1∼N(μ1,Σ1)z_1\sim\mathcal{N}(\mu_1,\Sigma_1), and ξ∼N(0,I)\xi\sim\mathcal{N}(0,I) are mutually independent, and γ(τ)=σγτ(1−τ)\gamma(\tau)=\sigma_\gamma\tau(1-\tau) for some σγ>0\sigma_\gamma>0. We sample z0∼qϕ(zt∣ut)z_0\sim q_\phi(z_t\mid u_t) and z1∼qϕ(zt+1∣ut+1)z_1\sim q_\phi(z_{t+1}\mid u_{t+1}) independently, hence Cov⁡(z0,z1)=0\operatorname{Cov}(z_0,z_1)=0 conditional on (ut,ut+1)(u_t,u_{t+1}). The goal of this section is to justify the use as training objectives the terms described in Eq. (10).

Assumption (diagonal covariances). Σ0,Σ1\Sigma_0,\Sigma_1 are diagonal (VAE posteriors) and γ(τ)I\gamma(\tau)I is isotropic, hence all covariances below are diagonal and operations are element-wise.

Proposition A.1 (Conditional endpoint law under the Gaussian interpolant). Under the above setup, let μ1∣τ\mu_{1\mid\tau} and Σ1∣τ\Sigma_{1\mid\tau} denote the conditional mean and covariance of z1z_1 given zτz_\tau. Then the conditional moments are given explicitly by:

μ1∣τ=μ1+τΣ1Στ−1(zτ−μτ),\mu_{1\mid\tau}=\mu_1+\tau\Sigma_1\Sigma_\tau^{-1}(z_\tau-\mu_\tau),
Σ1∣τ=Σ1−τ2Σ1Στ−1Σ1,\Sigma_{1\mid\tau}=\Sigma_1-\tau^2\Sigma_1\Sigma_\tau^{-1}\Sigma_1,

where μτ=(1−τ)μ0+τμ1\mu_\tau=(1-\tau)\mu_0+\tau\mu_1 and Στ=(1−τ)2Σ0+τ2Σ1+γ(τ)I\Sigma_\tau=(1-\tau)^2\Sigma_0+\tau^2\Sigma_1+\gamma(\tau)I.

Proof. Since (z0,z1,ξ)(z_0,z_1,\xi) are mutually independent Gaussians, the interpolant zτz_\tau is Gaussian with:

E[zτ]=E[(1−τ)z0+τz1+γ(τ)ξ]=(1−τ)E[z0]+τE[z1]+γ(τ)E[ξ]=(1−τ)μ0+τμ1,\mathbb{E}[z_\tau]=\mathbb{E}\left[(1-\tau)z_0+\tau z_1+\sqrt{\gamma(\tau)}\xi\right]=(1-\tau)\mathbb{E}[z_0]+\tau\mathbb{E}[z_1]+\sqrt{\gamma(\tau)}\mathbb{E}[\xi]=(1-\tau)\mu_0+\tau\mu_1,
Var⁡(zτ)=Var⁡((1−τ)z0+τz1+γ(τ)ξ)=(1−τ)2Var⁡(z0)+τ2Var⁡(z1)+γ(τ)Var⁡(ξ)\operatorname{Var}(z_\tau)=\operatorname{Var}\left((1-\tau)z_0+\tau z_1+\sqrt{\gamma(\tau)}\xi\right)=(1-\tau)^2\operatorname{Var}(z_0)+\tau^2\operatorname{Var}(z_1)+\gamma(\tau)\operatorname{Var}(\xi)
=(1−τ)2Σ0+τ2Σ1+γ(τ)I=:Στ.=(1-\tau)^2\Sigma_0+\tau^2\Sigma_1+\gamma(\tau)I=:\Sigma_\tau.

Since the joint distribution of (zτ,z1)(z_\tau,z_1) is Gaussian, by linearity of covariance we have that:

Cov⁡(z1,zτ)=Cov⁡(z1,(1−τ)z0+τz1+γ(τ)ξ)\operatorname{Cov}(z_1,z_\tau)=\operatorname{Cov}\left(z_1,(1-\tau)z_0+\tau z_1+\sqrt{\gamma(\tau)}\xi\right)
=(1−τ)Cov⁡(z1,z0)+τCov⁡(z1,z1)+γ(τ)Cov⁡(z1,ξ)=τΣ1.=(1-\tau)\operatorname{Cov}(z_1,z_0)+\tau\operatorname{Cov}(z_1,z_1)+\sqrt{\gamma(\tau)}\operatorname{Cov}(z_1,\xi)=\tau\Sigma_1.

By the standard Gaussian conditioning formula, for jointly Gaussian (X,Y)(X,Y) with Y∣X∼N(μY∣X,ΣY∣X)Y\mid X\sim\mathcal{N}(\mu_{Y\mid X},\Sigma_{Y\mid X}):

μY∣X=μY+Cov⁡(Y,X)Var⁡(X)−1(X−μX),ΣY∣X=ΣY−Cov⁡(Y,X)Var⁡(X)−1Cov⁡(X,Y).\mu_{Y\mid X}=\mu_Y+\operatorname{Cov}(Y,X)\operatorname{Var}(X)^{-1}(X-\mu_X),\qquad\Sigma_{Y\mid X}=\Sigma_Y-\operatorname{Cov}(Y,X)\operatorname{Var}(X)^{-1}\operatorname{Cov}(X,Y).

Applying this with Y=z1Y=z_1 and X=zτX=z_\tau:

μ1∣τ=μ1+τΣ1Στ−1(zτ−μτ),\mu_{1\mid\tau}=\mu_1+\tau\Sigma_1\Sigma_\tau^{-1}(z_\tau-\mu_\tau),
Σ1∣τ=Σ1−τΣ1Στ−1τΣ1=Σ1−τ2Σ1Στ−1Σ1.(13)\Sigma_{1\mid\tau}=\Sigma_1-\tau\Sigma_1\Sigma_\tau^{-1}\tau\Sigma_1=\Sigma_1-\tau^2\Sigma_1\Sigma_\tau^{-1}\Sigma_1. \tag*{(13)}

By construction, the VAE posteriors have diagonal covariance, and γ(τ)ξ\sqrt{\gamma(\tau)}\xi has covariance γ(τ)I\gamma(\tau)I, hence diagonal. Since sums, products, and inverses of diagonal matrices are diagonal, all covariance matrices in the above expressions are diagonal, and thus products and inverses are element-wise. □

Define s1∣τ:=diag⁡(Σ1∣τ)∈Rds_{1\mid\tau}:=\operatorname{diag}(\Sigma_{1\mid\tau})\in\mathbb{R}^{d} and let Δτ:=1−τ\Delta_{\tau}:=1-\tau.

Remark A.2 (Moment targets for the one-step kernel). Conditioned on zτz_{\tau}, the one-step update has mean zτ+Δτfθ(zτ,τ)z_{\tau}+\Delta_{\tau}f_{\theta}(z_{\tau},\tau) and covariance Δτgθ′(zτ,τ)2\Delta_{\tau}g_{\theta}^{\prime}(z_{\tau},\tau)^{2}. Thus the squared-error objectives in Eq. (10) are minimized when these moments match (μ1∣τ,Σ1∣τ)(\mu_{1\mid\tau},\Sigma_{1\mid\tau}), i.e., f∗(zτ,τ)=(μ1∣τ−zτ)/Δτf^{*}(z_{\tau},\tau)=(\mu_{1\mid\tau}-z_{\tau})/\Delta_{\tau} and g∗(zτ,τ)2=s1∣τ/Δτg^{*}(z_{\tau},\tau)^{2}=s_{1\mid\tau}/\Delta_{\tau}. The minimizer of Lvar\mathcal{L}_{\mathrm{var}} is unique in g2g^{2}, while gg is only identifiable up to sign (coordinate-wise). Uniqueness in gg can be enforced by parameterizing g≥0g\geq0 (e.g., g=softplus⁡(g~)g=\operatorname{softplus}(\widetilde{g})).

Remark A.3 (Connection to Gaussian conditioning). The conditional moments in Proposition A.1 follow from the standard jointly-Gaussian conditioning identity, here the required moments (μτ,Στ,Cov⁡(z1,zτ))(\mu_{\tau},\Sigma_{\tau},\operatorname{Cov}(z_{1},z_{\tau})) are induced directly by the interpolant construction.

Role of the Interpolant Noise Scale

The hyperparameter σγ\sigma_{\gamma} in γ(τ)=σγτ(1−τ)\gamma(\tau)=\sigma_{\gamma}\tau(1-\tau) controls the informativeness of the stochastic interpolant zτz_{\tau} about the endpoint z1z_{1}. Under the setup of Subsection A.2, with Σ0,Σ1≻0\Sigma_{0},\Sigma_{1}\succ0, recall that

zτ=(1−τ)z0+τz1+γ(τ)ξ,τ∈(0,1),z_{\tau}=(1-\tau)z_{0}+\tau z_{1}+\sqrt{\gamma(\tau)}\xi,\qquad\tau\in(0,1),

where z0∼N(μ0,Σ0)z_{0}\sim\mathcal{N}(\mu_{0},\Sigma_{0}), z1∼N(μ1,Σ1)z_{1}\sim\mathcal{N}(\mu_{1},\Sigma_{1}), and ξ∼N(0,I)\xi\sim\mathcal{N}(0,I) are mutually independent.

From Proposition A.1, the marginal variance of the interpolant is

Στ=(1−τ)2Σ0+τ2Σ1+σγτ(1−τ)I,\Sigma_{\tau}=(1-\tau)^{2}\Sigma_{0}+\tau^{2}\Sigma_{1}+\sigma_{\gamma}\tau(1-\tau)I,

and the conditional variance of the endpoint given the interpolant is

Σ1∣τ=Σ1−τ2Σ1Στ−1Σ1.\Sigma_{1\mid\tau}=\Sigma_{1}-\tau^{2}\Sigma_{1}\Sigma_{\tau}^{-1}\Sigma_{1}.

We now analyze how σγ\sigma_{\gamma} affects the conditional variance Σ1∣τ\Sigma_{1\mid\tau} for a fixed τ∈(0,1)\tau\in(0,1).

Corollary A.4 (Effect of σγ\sigma_{\gamma} and τ\tau on conditional endpoint uncertainty). Recall Σ1∣τ=Σ1−τ2Σ1Στ−1Σ1\Sigma_{1\mid\tau}=\Sigma_{1}-\tau^{2}\Sigma_{1}\Sigma_{\tau}^{-1}\Sigma_{1} with Στ=(1−τ)2Σ0+τ2Σ1+σγτ(1−τ)I\Sigma_{\tau}=(1-\tau)^{2}\Sigma_{0}+\tau^{2}\Sigma_{1}+\sigma_{\gamma}\tau(1-\tau)I. Assume the diagonal setting, so all operations are element-wise.

  1. Varying σγ\sigma_{\gamma} (fixed τ∈(0,1)\tau\in(0,1)). The map σγ↦Σ1∣τ(σγ)\sigma_{\gamma}\mapsto\Sigma_{1\mid\tau}(\sigma_{\gamma}) is monotonically increasing in PSD order. Moreover,

lim⁡σγ→0+Σ1∣τ=Σ1−τ2Σ12(1−τ)2Σ0+τ2Σ1,lim⁡σγ→+∞Σ1∣τ=Σ1.\lim_{\sigma_{\gamma}\to0^{+}}\Sigma_{1\mid\tau} =\Sigma_{1}-\frac{\tau^{2}\Sigma_{1}^{2}}{(1-\tau)^{2}\Sigma_{0}+\tau^{2}\Sigma_{1}}, \qquad \lim_{\sigma_{\gamma}\to+\infty}\Sigma_{1\mid\tau}=\Sigma_{1}.
  1. Varying τ\tau (fixed σγ>0\sigma_{\gamma}>0).

lim⁡τ→0+Σ1∣τ=Σ1,lim⁡τ→1−Σ1∣τ=0.\lim_{\tau\to0^{+}}\Sigma_{1\mid\tau}=\Sigma_{1}, \qquad \lim_{\tau\to1^{-}}\Sigma_{1\mid\tau}=0.

Proof. Since all covariance matrices are diagonal, products and inverses are element-wise. We work coordinate-wise without loss of generality.

1) Varying σγ\sigma_{\gamma} (fixed τ∈(0,1)\tau\in(0,1)). We have

∂Στ∂σγ=τ(1−τ)I≻0for τ∈(0,1).\frac{\partial\Sigma_{\tau}}{\partial\sigma_{\gamma}}=\tau(1-\tau)I\succ0\qquad\text{for }\tau\in(0,1).

Thus Στ\Sigma_{\tau} is strictly increasing in σγ\sigma_{\gamma}, which implies Στ−1\Sigma_{\tau}^{-1} is strictly decreasing. Since

Σ1∣τ=Σ1−τ2Σ1Στ−1Σ1,\Sigma_{1\mid\tau}=\Sigma_{1}-\tau^{2}\Sigma_{1}\Sigma_{\tau}^{-1}\Sigma_{1},

and the subtracted term τ2Σ1Στ−1Σ1\tau^{2}\Sigma_{1}\Sigma_{\tau}^{-1}\Sigma_{1} decreases as Στ−1\Sigma_{\tau}^{-1} decreases, we conclude that Σ1∣τ\Sigma_{1\mid\tau} is increasing in σγ\sigma_{\gamma}.

As σγ→0+\sigma_{\gamma}\to0^{+}, we have Στ→(1−τ)2Σ0+τ2Σ1\Sigma_{\tau}\to(1-\tau)^{2}\Sigma_{0}+\tau^{2}\Sigma_{1}. Direct substitution yields the result.

As σγ→+∞\sigma_{\gamma}\to+\infty, for each coordinate ii, Στ,i=(1−τ)2Σ0,i+τ2Σ1,i+σγτ(1−τ)→+∞\Sigma_{\tau,i}=(1-\tau)^{2}\Sigma_{0,i}+\tau^{2}\Sigma_{1,i}+\sigma_{\gamma}\tau(1-\tau)\to+\infty, so Στ,i−1→0\Sigma_{\tau,i}^{-1}\to0 and hence τ2Σ1,i2Στ,i−1→0\tau^{2}\Sigma_{1,i}^{2}\Sigma_{\tau,i}^{-1}\to0. Therefore Σ1∣τ,i→Σ1,i\Sigma_{1\mid\tau,i}\to\Sigma_{1,i}, i.e. Σ1∣τ→Σ1\Sigma_{1\mid\tau}\to\Sigma_{1}.

:::

  1. Varying τ\tau (fixed σγ>0\sigma_\gamma> 0). Rewrite the conditional variance as:

Σ1∣τ=Σ1−τ2Σ12(1−τ)2Σ0+τ2Σ1+σγτ(1−τ)I\Sigma_{1\mid\tau} = \Sigma_1 - \tau^2 \frac{\Sigma_1^2}{(1-\tau)^2\Sigma_0+\tau^2\Sigma_1+\sigma_\gamma\tau(1-\tau)I}

As τ→0+\tau\to0^+, the denominator remains bounded away from 00 (it converges to Σ0≻0\Sigma_0 \succ0) while the prefactor τ2→0+\tau^2 \to0^+, so the second term vanishes and Σ1∣τ→Σ1\Sigma_{1\mid\tau} \to\Sigma_1.

Rewrite the conditional variance as:

Σ1∣τ=Σ1−τ2Σ12Στ=Σ1Στ−τ2Σ12Στ=Σ1((1−τ)2Σ0+σγτ(1−τ)I)Στ.\Sigma_{1\mid\tau} = \Sigma_1 - \frac{\tau^2\Sigma_1^2}{\Sigma_\tau} = \frac{\Sigma_1\Sigma_\tau-\tau^2\Sigma_1^2}{\Sigma_\tau} = \frac{\Sigma_1\left((1-\tau)^2\Sigma_0+\sigma_\gamma\tau(1-\tau)I\right)}{\Sigma_\tau}.

The numerator contains the factor (1−τ)(1-\tau), so it vanishes as τ→1−\tau\to1^-. The denominator approaches Σ1≻0\Sigma_1 \succ0. Therefore Σ1∣τ→0\Sigma_{1\mid\tau} \to0.

Posterior-as-target objective: a principled justification

This section provides the formal justification for Proposition 3.1 in the main text. Intuitively, because latent states are unobserved, the only available supervision signal for zt+1z_{t+1} is the encoder posterior qϕ(zt+1∣ut+1)q_\phi(z_{t+1}\mid u_{t+1}). When combined with the data dynamics over ut+1u_{t+1} given utu_t, this induces an encoder-averaged (or aggregated) target distribution over the next latent state conditioned on utu_t. We show that minimizing the expected KL objective in Eq. (7) learns this induced target.

Setup. Let pdata(ut,ut+1)p_{\mathrm{data}}(u_t,u_{t+1}) denote the (empirical) joint distribution of consecutive PDE states, and let qϕ(z∣u)q_\phi(z\mid u) be a fixed encoder density with respect to a base measure μ(dz)\mu(dz) on Rd\mathbb{R}^d. Define the induced joint distribution over (ut,zt+1)(u_t,z_{t+1}) by composing the data dynamics with the encoder kernel:

p~(ut,zt+1):=∫pdata(ut,ut+1)qϕ(zt+1∣ut+1) dut+1.(14)\widetilde{p}(u_t,z_{t+1}) := \int p_{\mathrm{data}}(u_t,u_{t+1})q_\phi(z_{t+1}\mid u_{t+1})\,du_{t+1}. \tag*{(14)}

Equivalently, for pdatap_{\mathrm{data}}-almost every utu_t, the induced conditional density is

p~(zt+1∣ut)=∫pdata(ut+1∣ut)qϕ(zt+1∣ut+1) dut+1.(15)\widetilde{p}(z_{t+1}\mid u_t) = \int p_{\mathrm{data}}(u_{t+1}\mid u_t)q_\phi(z_{t+1}\mid u_{t+1})\,du_{t+1}. \tag*{(15)}

Training objective. We train a conditional step model p(zt+1∣ut)p(z_{t+1}\mid u_t) by minimizing the expected KL divergence

L(p):=E(ut,ut+1)∼pdata[KL⁡(qϕ(z∣ut+1)∥p(z∣ut))].(16)\mathcal{L}(p) := \mathbb{E}_{(u_t,u_{t+1})\sim p_{\mathrm{data}}}\left[\operatorname{KL}\left(q_\phi(z\mid u_{t+1})\mathbin{\|}p(z\mid u_t)\right)\right]. \tag*{(16)}

For clarity, Eq. (16) is the same expected KL objective as Eq. (7). Expanding the KL and dropping terms independent of pp yields a cross-entropy term −Eut∫p~(z∣ut)log⁡p(z∣ut)μ(dz)-\mathbb{E}_{u_t}\int\widetilde{p}(z\mid u_t)\log p(z\mid u_t)\mu(dz), where p~\widetilde{p} is defined in Eq. (15).

Technical conditions. Assume L(p)<∞\mathcal{L}(p)<\infty and restrict to conditional densities p(⋅∣ut)p(\cdot\mid u_t) w.r.t. μ\mu such that p(z∣ut)>0p(z\mid u_t)>0 whenever p~(z∣ut)>0\widetilde{p}(z\mid u_t)>0 (to avoid infinite KL terms). These conditions are standard for KL-based projection arguments.

Proposition B.1 (KL projection onto the encoder-induced target). The functional L(p)\mathcal{L}(p) in Eq. (16) is minimized by

p∗(z∣ut)=p~(z∣ut)p^*(z\mid u_t)=\widetilde{p}(z\mid u_t)

for pdatap_{\mathrm{data}}-almost every utu_t, uniquely up to (pdata×μ)(p_{\mathrm{data}}\times\mu)-null sets.

Proof. Expand the KL divergence:

KL⁡(qϕ(z∣ut+1)∥p(z∣ut))=Ez∼qϕ(⋅∣ut+1)[log⁡qϕ(z∣ut+1)−log⁡p(z∣ut)].\operatorname{KL}\left(q_\phi(z\mid u_{t+1})\mathbin{\|}p(z\mid u_t)\right)=\mathbb{E}_{z\sim q_\phi(\cdot\mid u_{t+1})}\left[\log q_\phi(z\mid u_{t+1})-\log p(z\mid u_t)\right].

Substituting into Eq. (16) yields

L(p)=E(ut,ut+1)∼pdataEz∼qϕ(⋅∣ut+1)[log⁡qϕ(z∣ut+1)]−E(ut,ut+1)∼pdataEz∼qϕ(⋅∣ut+1)[log⁡p(z∣ut)].(17)\mathcal{L}(p)=\mathbb{E}_{(u_t,u_{t+1})\sim p_{\mathrm{data}}}\mathbb{E}_{z\sim q_\phi(\cdot\mid u_{t+1})}\left[\log q_\phi(z\mid u_{t+1})\right]-\mathbb{E}_{(u_t,u_{t+1})\sim p_{\mathrm{data}}}\mathbb{E}_{z\sim q_\phi(\cdot\mid u_{t+1})}\left[\log p(z\mid u_t)\right]. \tag*{(17)}

Using the definition of the induced conditional p~(⋅∣ut)\widetilde{p}(\cdot\mid u_t) in Eq. (15) and Fubini/Tonelli to exchange integrals (justified under L(p)<∞\mathcal{L}(p) < \infty), we obtain

E(ut,ut+1)∼pdataEz∼qϕ(⋅∣ut+1)[log⁡p(z∣ut)]=Eut∼pdata[∫p~(z∣ut)log⁡p(z∣ut) μ(dz)].\mathbb{E}_{(u_t,u_{t+1})\sim p_{\mathrm{data}}}\mathbb{E}_{z\sim q_{\phi}(\cdot\mid u_{t+1})}[\log p(z\mid u_t)] = \mathbb{E}_{u_t\sim p_{\mathrm{data}}}\left[\int\widetilde{p}(z\mid u_t)\log p(z\mid u_t)\,\mu(dz)\right].

Therefore,

L(p)=−Eut∼pdata[∫p~(z∣ut)log⁡p(z∣ut) μ(dz)]+const.\mathcal{L}(p) = -\mathbb{E}_{u_t\sim p_{\mathrm{data}}}\left[\int\widetilde{p}(z\mid u_t)\log p(z\mid u_t)\,\mu(dz)\right] + \mathrm{const}.

Using the identity

∫p~(z∣ut)log⁡p(z∣ut) μ(dz)=−KL⁡(p~(⋅∣ut)∥p(⋅∣ut))+∫p~(z∣ut)log⁡p~(z∣ut) μ(dz)⏟independent of p,\int\widetilde{p}(z\mid u_t)\log p(z\mid u_t)\,\mu(dz) = -\operatorname{KL}\left(\widetilde{p}(\cdot\mid u_t)\mathbin{\|}p(\cdot\mid u_t)\right) + \underbrace{\int\widetilde{p}(z\mid u_t)\log\widetilde{p}(z\mid u_t)\,\mu(dz)}_{\text{independent of }p},

we conclude that minimizing L(p)\mathcal{L}(p) is equivalent to minimizing Eut∼pdata[KL⁡(p~(⋅∣ut)∥p(⋅∣ut))]\mathbb{E}_{u_t\sim p_{\mathrm{data}}}\left[\operatorname{KL}\left(\widetilde{p}(\cdot\mid u_t)\mathbin{\|}p(\cdot\mid u_t)\right)\right]. This is achieved iff p(⋅∣ut)=p~(⋅∣ut)p(\cdot\mid u_t)=\widetilde{p}(\cdot\mid u_t) for pdatap_{\mathrm{data}}-almost every utu_t, with uniqueness up to (pdata×μ)(p_{\mathrm{data}}\times\mu)-null sets.

Algorithms

Algorithm 1TORNADO training on one-step posterior pairs
Require:Consecutive states (ut,ut+1)(u_t,u_{t+1}), posteriors qϕq_{\phi}, parameters Θ=(θ,θ′)\Theta=(\theta,\theta')
1:Sample z0∼qϕ(zt∣ut)z_0\sim q_{\phi}(z_t\mid u_t) and z1∼qϕ(zt+1∣ut+1)z_1\sim q_{\phi}(z_{t+1}\mid u_{t+1})
2:Sample τ∼U(τmin⁡,1−τmin⁡)\tau\sim\mathcal{U}(\tau_{\min},1-\tau_{\min}) and ξ∼N(0,I)\xi\sim\mathcal{N}(0,I)
3:Form zτ=(1−τ)z0+τz1+γ(τ) ξz_{\tau}=(1-\tau)z_0+\tau z_1+\sqrt{\gamma(\tau)}\,\xi
4:Compute (μtarget,Σtarget)(\mu_{\mathrm{target}},\Sigma_{\mathrm{target}}) using Proposition (3.2)
5:Compute Ldrift\mathcal{L}_{\mathrm{drift}} and Lvar\mathcal{L}_{\mathrm{var}} using (10)
6:Draw {z^(k)}k=1K\{\widehat{z}^{(k)}\}_{k=1}^{K} from (9)
7:Draw {z^1,target(ℓ)}ℓ=1L\{\widehat{z}^{(\ell)}_{1,\mathrm{target}}\}_{\ell=1}^{L} from N(μtarget,Σtarget)\mathcal{N}(\mu_{\mathrm{target}},\Sigma_{\mathrm{target}})
8:Compute LED\mathcal{L}_{\mathrm{ED}} using (11) and total loss (12)
9:Update Θ\Theta by gradient descent (drift warm-up; ramp λvar,λED\lambda_{\mathrm{var}},\lambda_{\mathrm{ED}})

Table 1 (PDF p. 19).

Algorithm 2TORNADO inference (one-step forecast and rollout)
Require:Current state utu_t, (Eϕ,Dϕ)(E_{\phi},D_{\phi}), learned (fθ,gθ′)(f_{\theta},g_{\theta'}), steps NN, samples KK
1:Sample z0∼qϕ(zt∣ut)z_0\sim q_{\phi}(z_t\mid u_t). Generate KK copies {z0(k)}k=1K\{z_0^{(k)}\}_{k=1}^{K}
2:Δτ←1/N\Delta\tau\leftarrow1/N
3:for k=1k=1 to KK do
4:z←z0(k)z\leftarrow z_0^{(k)}
5:for n=0n=0 to N−1N-1 do
6:τn←nΔτ\tau_n\leftarrow n\Delta\tau, ϵn∼N(0,I)\epsilon_n\sim\mathcal{N}(0,I)
7:z←z+Δτfθ(z,τn)+Δτ gθ′(z,τn)⊙ϵnz\leftarrow z+\Delta\tau f_{\theta}(z,\tau_n)+\sqrt{\Delta\tau}\,g_{\theta'}(z,\tau_n)\odot\epsilon_n
8:end for
9:z^t+1(k)←z\widehat{z}_{t+1}^{(k)}\leftarrow z
10:end for
11:Decode {u^t+1(k)}k=1K←Dϕ({z^t+1(k)}k=1K)\{\widehat{u}_{t+1}^{(k)}\}_{k=1}^{K}\leftarrow D_{\phi}(\{\widehat{z}_{t+1}^{(k)}\}_{k=1}^{K})
12:(Optional) Autoregressive rollout: set z0(k)←z^t+1(k)z_0^{(k)}\leftarrow\widehat{z}_{t+1}^{(k)} and repeat

Table 2 (PDF p. 19).

Related Work

Generative models for PDE surrogates. Generative models learn (possibly conditional) distributions over high-dimensional data, enabling sampling and uncertainty-aware prediction. While they have driven major advances in vision and language, there is increasing interest in using them as neural surrogates for spatiotemporal physical systems governed by PDEs. Conditional diffusion models Ho et al. [18]; Song et al. [51] learn to sample from a target distribution by iteratively denoising a sequence of corrupted variables, typically predefined by a fixed forward noising process. Conditioning can be imposed on context such as past states, parameters, or forcings, producing conditional samplers for PDE solution fields. In PDE surrogate modeling for forward processes, diffusion models are commonly used in two ways. (i) Refinement / correction: a diffusion model acts as a learned stochastic corrector that corrects trajectories produced by a deterministic surrogate by denoising, improving long-horizon stability under autoregressive rollout Lippe et al. [38]; Pedersen et al. [44]; Serrano et al. [49]. (ii) Direct generative surrogate: the diffusion model itself predicts future solution states by sampling ut+Δtu_{t+\Delta t} conditioned on history, parameters, or observations Huang et al. [21]; Jacobsen et al. [22]; Shysheya et al. [50]; Kohl et al. [27]; Price et al. [46]. Concurrently, Conditional flow matching Lipman et al. [37] learns a time-dependent vector field vθ(x,t∣c)v_\theta(x,t \mid c) whose induced ODE transports a simple base distribution (usually a standard Gaussian) to the data distribution, enabling fast sampling via probability flow ODE integration. This provides a diffusion-adjacent alternative with deterministic dynamics and has recently been explored for PDE surrogate generation Baldan et al. [2]; Li et al. [34]; Koupaï et al. [29], while providing a simulation free alternative to Continuous Normalizing flows Chen et al. [7] without the need to integrate the adjoint for backpropagation. Rout et al. [48] combine learned initial-state perturbations with deterministic flow-matching propagation. FuXi-ENS (Zhong et al. [53]) uses a VAE-based model to generate flow-dependent Gaussian perturbations at each autoregressive forecast step, followed by a deterministic forecasting network. Both approaches learn ensemble variability through perturbation generation, whereas TORNADO learns the diffusion amplitude within the latent stochastic transport.

Finally, stochastic interpolants (SIs) Albergo et al. [1] unify diffusion- and flow-based perspectives by learning transports along stochastic paths connecting two endpoint distributions. In practice, SIs learn a drift field (often via conditional regression of an interpolant) and generate samples by integrating an Itô SDE whose diffusion schedule is specified by the user. This formulation is particularly natural for PDE surrogates that map one state distribution to the next (“data-to-data”), and has been adopted in recent work on generative physical forecasting Zhou et al. [54]; Hou et al. [19]. Zhou et al. [54] benchmarks SIs against diffusion and flow baselines on multiple physics and climate datasets, while Hou et al. [19] proposes spline-based probability paths for continuous-time forecasting and a simulation-free alternative to Neural ODE training. Extensions imposing physical structure or constraints on SI-based surrogates have also been explored for turbulent flows Mücke and Sanderse [43].

Uncertainty quantification in neural PDE solvers. Quantifying predictive uncertainty in neural PDE surrogates remains challenging, and existing approaches can be broadly categorized by their methodological strategy.

Bayesian and ensemble methods. Classical strategies adapt Bayesian inference to neural operators. The Bayesian Neural Operator (BNO) Magnani et al. [40] employs a Laplace approximation of the weight posterior, while deep ensembles Lakshminarayanan et al. [32] remain a strong baseline albeit computationally expensive. DiverseNO Mouli et al. [42] reduces this cost for Fourier Neural Operators (FNO) Li et al. [36] by generating multiple outputs from a single forward pass through the final Fourier layer. DINOZAUR Matveev et al. [41] takes a different approach, replacing the dense FNO multiplier with a heat-kernel-inspired diffusion parameterization that requires only one learnable time parameter per channel; by placing priors on these parameters and performing variational inference, it yields spatially correlated uncertainty estimates with drastically reduced parameter count. BARNN Coscia et al. [9] extends Bayesian treatment to autoregressive architectures by treating both network weights and hidden states as stochastic.

Latent-space and bottleneck methods. An alternative strategy incorporates uncertainty within a compressed representation. IB-UQ Guo et al. [14] uses an information bottleneck to recover predictive means and variances, while LE-PDE-UQ Wu et al. [52] introduces an encode-processdecode architecture where uncertainty is modeled in the latent processor and decoded via a separate head. These methods reduce the dimensionality of the uncertainty estimation problem but typically rely on parametric assumptions (e.g., Gaussian outputs).

Post-hoc calibration. Rather than modifying the model architecture, Gopakumar et al. [12] applies conformal prediction to physics residual errors, providing distribution-free coverage guarantees. However, this requires a held-out calibration set and does not improve the underlying predictive distribution.

Proper scoring approaches. The Probabilistic Neural Operator (PNO) Bülte et al. [5] provides theoretical guarantees for training neural operators using the Energy Score (ES), directly optimizing a strictly proper scoring rule. This aligns with our evaluation methodology, though PNO operates in physical space and does not exploit temporal structure.

Most existing methods operate in physical space, incurring costs that scale with the spatial resolution of the PDE discretization. Moreover, few approaches explicitly leverage the sequential structure of autoregressive forecasting, where consecutive states are often similar.

Dataset details

Synthetic data

The synthetic PDE trajectory data were generated with APEBench Koehler et al. [26], a benchmark suite to evaluate autoregressive neural PDE emulators. All datasets were generated on 2D spatial grids of size [160,160][160, 160], with 1024 trajectories for train and 256 for test (except Gray-Scott, where it is 256/64) using the suite defaults. Concretely:

  • Kolmogorov Flow: 2D incompressible Navier-Stokes on a periodic square domain of extent 2π2\pi, with viscosity ν=10−2\nu= 10^{-2} (Re= 100 in the scenario convention) and linear drag −0.1-0.1, driven by a sinusoidal forcing of amplitude 1.0 at injection mode k=4k = 4; integrated with Δt=0.1\Delta t = 0.1 using 20 solver substeps (Δtsolver=0.005\Delta t_{\mathrm{solver}} = 0.005) and 500 warmup steps. We generate 100 steps train and test.

  • Kuramoto-Sivashinsky (KS): parameterized through the suite difficulty coefficients with integrated diffusion and hyper-diffusion (γ2,γ4)=(−1.2,−15.0)(\gamma_{2},\gamma_{4}) = (-1.2,-15.0) and a gradient term δ=−6.0\delta= -6.0; 500 warmup steps are used. We generate 50 steps train and test.

  • Burgers: 2D Burgers single-channel scenario on a unit domain (extent 1.0) with outer step Δt=0.1\Delta t = 0.1 (one substep), convection coefficient −0.125-0.125 and viscosity 3×10−43 \times10^{-4} (no external forcing). We generate 25 steps train and test.

  • Gray-Scott (GS): two-channel reaction–diffusion system on a unit domain (extent 1.0) with feed/kill rates (F,k)=(0.04,0.06)(F,k) = (0.04, 0.06) and diffusivities (D1,D2)=(2×10−5,1×10−5)(D_{1},D_{2}) = (2 \times10^{-5}, 1 \times10^{-5}), integrated with Δt=10.0\Delta t = 10.0 using 10 solver substeps (Δtsolver=1.0\Delta t_{\mathrm{solver}} = 1.0). We generate 100 steps train and test.

Real-world data

In order to test whether the calibration gains transfer to observational or experimentally measured systems, where measurement noise, partial observation, and model mismatch differ from clean simulator data, we carry out the main experiment also on two real world datasets. We use measured flame imaging from the Combustion scenario of RealPDEBench (Hu et al. [20]) and ERA5 atmospheric reanalysis (Hersbach et al., [15]) obtained through WeatherBench (Rasp et al., [47]).

RealPDEBench

RealPDEBench (Hu et al., [20]) is a benchmark for scientific ML that integrates real-world measurements with paired numerical simulations. We focus on one of their proposed tasks, real-world training, which consists of training and evaluating on real, observational, noisy data.

We use the Combustion dataset, consisting of cross-sectional data of 3D swirl-stabilized NH3_3/CH4_4/air flames, measured as real-world OH* chemiluminescence intensity II. Table 3 compiles its specifications. It contains 30 trajectories of 2001 128 × 128 frames, each at a distinct combination of NH3_3 blend and equivalence ratio. The train–test split is 26/4 by trajectory, so no test condition is seen during training: the model must generalize to an unseen (blend, ϕ\phi) pair, whose individual values it has only encountered in other combinations. All models are conditioned on the pair (NH3\mathrm{NH}_3 blend, ϕ\phi) through adaLN.

Trajectories30 (one per operating condition)
Frames per trajectory2001
Spatial resolution128×128128 \times128
Channels1 (OH* chemiluminescence intensity)
ConditioningNH3\mathrm{NH}_3 blend ∈{0,20,40,60,80}\in\{0, 20, 40, 60, 80\}%; ϕ∈{0.75,0.85,0.9,0.95,1.0,1.05,1.1,1.15,1.2,1.25,1.3}\phi\in\{0.75, 0.85, 0.9, 0.95, 1.0, 1.05, 1.1, 1.15, 1.2, 1.25, 1.3\}
Split (by trajectory)26 train / 4 test
Held-out conditions (NH3\mathrm{NH}_3, ϕ\phi)(0%, 1.10), (20%, 1.05), (40%, 1.10), (80%, 1.15)
Normalisationlog⁡(1+x)\log(1+x) followed by a scalar zz-score, fitted on training trajectories

Table 3. RealPDEBench Combustion, experimental subset: dataset specifications.

The convention proposed by RealPDEBench is to evaluate forecasts on short spatiotemporal windows sampled from the long trajectories. We use non-overlapping 21-frame windows: the first frame is the initial condition and the model predicts the remaining 20 autoregressively. Each held-out trajectory gives 95 windows, 380 in total.

WeatherBench ERA5

ERA5 (Hersbach et al., [15]) is the ECMWF atmospheric reanalysis: a gridded state estimate produced by assimilating observations into a numerical weather prediction model. We use the 1.5° WeatherBench-2 distribution (Rasp et al., [47]). Table 4 compiles its specifications. Where Combustion is a small, noisy, single-channel measurement, ERA5 is a large four-channel state on a periodic sphere, so together they probe the same calibration question in two very different regimes.

Grid121×240121 \times240 (latitude ×\times longitude), poles included, longitude periodic
Cadence6-hourly
Channels4 (T2mT_{2m}, U10U_{10}, V10V_{10}, Z500Z_{500})
Frames8764 train / 1464 validation / 2920 test
Split (by year)2010–2015 / 2016 / 2017–2018
Encoder padding121→128121 \to128 latitudes: circular in longitude, replicate at the poles; cropped back after decoding
Latent grid16×30×816 \times30 \times8
Conditioning[sin, cos] of day-of-year and time-of-day (4 features)
Normalisationper-channel zz-score, fitted on the training years

Table 4. WeatherBench-2 ERA5 at 1.5°: dataset specifications.

Splits are by whole calendar year: 2010–2015 for training, 2016 for validation and 2017–2018 for testing. We take a subset of GraphCast’s fields (Lam et al., [33]): three surface variables and one atmospheric variable. This is not a full atmospheric state and we claim no complete forecasting system, but the four are heterogeneous by design: a thermodynamic scalar, a vector pair whose components must stay correlated, and the circulation field that is the standard verification variable for medium-range skill. Because they differ by two orders of magnitude, each channel is zz-scored with training-year statistics. The same applies at evaluation, aggregate metrics are computed in normalized units and weighted by cos⁡(latitude)\cos(\mathrm{latitude}). Conditioning is on four date features, [sin, cos] of day-of-year and of time-of-day, injected through adaLN.

Evaluation uses non-overlapping 41-frame windows: the first frame is the initial condition and the model predicts the remaining 40 autoregressively, a 10-day forecast at 6-hourly resolution. Each test year gives 35 windows, 70 in total.

Baselines

Conditional Diffusion Models We use a standard conditional diffusion model that defines a discrete noising process indexed by τ∈{0,…,N−1}\tau\in\{0,\ldots,N-1\}, with N=100N=100 noising steps. The target z1=Eϕ(ut+1)z_1=E_{\phi}(u_{t+1}) is noised using a linear schedule as

zτ=αˉτz1+1−αˉτϵ,ϵ∼N(0,I),z_{\tau}=\sqrt{\bar{\alpha}_{\tau}}z_1+\sqrt{1-\bar{\alpha}_{\tau}}\epsilon,\quad\epsilon\sim\mathcal{N}(0,I),

with ατ=1−βτ\alpha_{\tau}=1-\beta_{\tau} and αˉτ=∏j=0ταj\bar{\alpha}_{\tau}=\prod_{j=0}^{\tau}\alpha_j. The network is trained to predict ϵ\epsilon by minimizing:

E[∥ϵ−ϵθ([z0,zτ],τ)∥22].\mathbb{E}\left[\left\|\epsilon-\epsilon_{\theta}([z_0,z_{\tau}],\tau)\right\|_2^2\right].

At inference, we sample following the protocol from Denoising Diffusion Implicit Models (DDIM) Song et al. [51] non-Markovian deterministic sampling where we can control the number of denoising steps to compare NFEs.

Conditional Flow Matching (FM) Lipman et al. [37] We implement a standard noise-to-data Conditional Flow Matching baseline defining a linear interpolator in continuous interpolation time τ∈[τmin⁡,1−τmin⁡]\tau\in[\tau_{\min},1-\tau_{\min}]:

zτ=(1−τ)z1+τϵ,ϵ∼N(0,I).z_{\tau}=(1-\tau)z_1+\tau\epsilon,\quad\epsilon\sim\mathcal{N}(0,I).

We sample τ∼Unif⁡(0,1)\tau\sim\operatorname{Unif}(0,1) and supervise the velocity target v∗=z˙τ=ϵ−z1v^*=\dot{z}_{\tau}=\epsilon-z_1 and minimize:

E[∥v∗−fθ([z0,zτ],τ)∥22].\mathbb{E}\left[\left\|v^*-f_{\theta}([z_0,z_{\tau}],\tau)\right\|_2^2\right].

At inference, we start from a sample zτ=1∼N(0,I)z_{\tau=1}\sim\mathcal{N}(0,I) and integrate the auxiliary-time ODE from τ=1→0\tau=1\to0 (prediction for next-step posterior sample z1z_1) with the Euler method, where we can choose the number of integrator steps (NFE).

Stochastic Interpolant (SI) Albergo et al. [1] We follow the implementation from Zhou et al. [54] but changing the definition of the interpolation time to be continuous in τ∈[τmin⁡,1−τmin⁡]\tau\in[\tau_{\min},1-\tau_{\min}], like in the FM baseline. We define the stochastic interpolant between the two latent posterior distributions of consecutive timesteps as

I(τ)=(1−τ)z0+τz1+γ(τ)Wτ,I(\tau)=(1-\tau)z_0+\tau z_1+\gamma(\tau)W_{\tau},

with z0∼qϕ(zt∣ut)z_0\sim q_{\phi}(z_t|u_t), z1∼qϕ(zt+1∣ut+1)z_1\sim q_{\phi}(z_{t+1}|u_{t+1}), γ(τ)=1−τ\gamma(\tau)=1-\tau, and WτW_{\tau} a standard Wiener process, so that Wτ∼N(0,τI)W_{\tau}\sim\mathcal{N}(0,\tau I). The stochastic perturbation therefore has scale (1−τ)τ(1-\tau)\sqrt{\tau} and vanishes at both endpoints, respecting the SI boundary conditions. By Itô’s lemma, I(τ)I(\tau) satisfies

dI(τ)=[z1−z0+γ˙(τ)Wτ]dτ+γ(τ) dWτ,\mathrm{d}I(\tau)=\left[z_1-z_0+\dot{\gamma}(\tau)W_{\tau}\right]\mathrm{d}\tau+\gamma(\tau)\,\mathrm{d}W_{\tau},

and since WτW_{\tau} is not differentiable, no chain-rule term in τ\sqrt{\tau} arises. We train the model to match the drift b(τ,z0,z1)=z1−z0+γ˙(τ)Wτb(\tau,z_0,z_1)=z_1-z_0+\dot{\gamma}(\tau)W_{\tau} by minimizing Eq. 17.

Lθ=Eτ,z0,z1,Wτ[∥vθ(I(τ),τ)−b(τ,z0,z1)∥22].(18)\mathcal{L}_{\theta}=\mathbb{E}_{\tau,z_0,z_1,W_{\tau}}\left[\left\|v_{\theta}(I(\tau),\tau)-b(\tau,z_0,z_1)\right\|_2^2\right]. \tag*{(18)}

At inference, similarly to FM, we integrate the drift with tuneable NFE, using Euler–Maruyama to solve dz=vθ dτ+g(τ) dWτ\mathrm{d}z=v_{\theta}\,\mathrm{d}\tau+g(\tau)\,\mathrm{d}W_{\tau} with diffusion schedule g(τ)=γ(τ)g(\tau)=\gamma(\tau). For a general gg the drift requires the correction 12(g2−γ2)∇log⁡ρτ\frac{1}{2}(g^2-\gamma^2)\nabla\log\rho_{\tau}; with g=γg=\gamma this term vanishes.

Metrics

Mean Square Error (MSE): we compute the mean square error of the ensemble mean with respect to the ground truth as a measure of accuracy:

MSE⁡:=E[∥uˉt+1−ut+1∥22].\operatorname{MSE}:=\mathbb{E}\left[\|\bar{u}_{t+1}-u_{t+1}\|_2^2\right].

Continuous Ranked Probability Score (CRPS): it is a widely used strictly proper scoring rule Gneiting and Raftery [11] which evaluates the full predictive distribution beyond first and second moments. We report the empirical CRPS computed per grid entry from the ensemble,

CRPS⁡(F^,y)=EX∼F^∣X−y∣−12EX,X′∼F^∣X−X′∣,\operatorname{CRPS}(\hat{F},y)=\mathbb{E}_{X\sim\hat{F}}|X-y|-\frac{1}{2}\mathbb{E}_{X,X'\sim\hat{F}}|X-X'|,

and averaged over all entries. Here F^\hat{F} denotes the empirical distribution supported on {u^t+1(k)}k=1K\{\hat{u}_{t+1}^{(k)}\}_{k=1}^{K} and y=ut+1y=u_{t+1}.

Root Mean Square Calibration Error (RMSCE): calibration refers to how much the predicted uncertainty of a model corresponds to the actual uncertainty of the prediction Guo et al. [13]. We quantify calibration computing probability integral transform (PIT) values computed per grid entry. For a calibrated prediction, the PIT should be distributed uniformly on [0,1][0,1].

Let pmp_m be M=50M=50 evenly-spaced points in [0,1][0,1] and let G^(p)\widehat{G}(p) be the empirical CDF of PIT values. We report the root mean-squared calibration error:

RMSCE⁡PIT:=(1M∑m=1M(G^(pm)−pm)2)1/2.\operatorname{RMSCE}_{\mathrm{PIT}} := \left(\frac{1}{M}\sum_{m=1}^{M}\left(\widehat{G}(p_m)-p_m\right)^2\right)^{1/2}.

Spread-Skill Ratio (SSR): Because RMSCE measures coverage at a finite set of nominal levels and does not directly penalize overly wide intervals, we additionally report SSR as a sharpness/spread diagnostic:

SSR⁡:=E[Var⁡(u^t+1(k))]MSE⁡(uˉt+1,ut+1).\operatorname{SSR} := \frac{\sqrt{\mathbb{E}\left[\operatorname{Var}\left(\hat{u}_{t+1}^{(k)}\right)\right]}}{\sqrt{\operatorname{MSE}\left(\bar{u}_{t+1},u_{t+1}\right)}}.

Ideally, SSR ≈1\approx1, values >1>1 indicate overspread and values <1<1, underspread. We report ∣log⁡(SSR⁡)∣|\log(\operatorname{SSR})| so that lower is better, as in the other metrics, for clarity.

Implementation details

Model sizes.

TypeParams
DiT (Diffusion Transformer)15.9M
Convolutional VAE (encoder–decoder)20.5M

Table 1 (PDF p. 24).

VAE

The autoencoder uses a Deep Compression Autoencoder (DCAE) style backbone Chen et al. [6] trained with a standard β\beta-VAE objective Higgins et al. [16]. Defined with latent dimensionality 4 channels and an 8×8\times spatial downsampling (from 160×160160\times160 to 20×2020\times20 for APEBench, 16×1616\times16 for Combustion and ERA5 uses 8 latent channels, giving 16×30×816\times30\times8).

DiT

The latent stepper is a Diffusion Transformer Peebles and Xie [45] operating on the 20×2020\times20 latent grid (one token per spatial site, i.e., 1×11\times1 patch projection). For Real Combustion and ERA5 it operates on the 16×1616\times16 and 16×3016\times30 grids above, with all other DiT hyperparameters unchanged. We use 12 transformer blocks with hidden width 256 and 16 self-attention heads; conditioning is injected through a 256-dimensional embedding (used to condition on τ\tau). Inputs are the current zτz_\tau and start-point sample z0z_0 concatenated as [z0,zτ][z_0,z_\tau].

When training TORNADO, we implement a separate diffusion-prediction branch and stop gradients from this branch into the shared transformer trunk; additionally, the diffusion branch includes an extra transformer block before its output projection.

Optimization and schedules. (DiT)

We train the dynamics model with Adam at learning rate 10−410^{-4} and a step-wise exponential decay schedule (StepLR with step size 10 and multiplicative factor 0.99).

DiT training We use K=L=16K=L=16 for the ED samples, λED\lambda_{\mathrm{ED}} and λvar\lambda_{\mathrm{var}} are ramped from 0 to (0.1, 1) respectively during 200 epochs after a 1000 epoch warmup. TORNADO and baselines are trained matched on number of epochs (5000).

Additional ablations

Ablation on diffusion losses

Here, we test the importance of each of the diffusion loss terms. First, we train several instances of TORNADO with combinations of λvar∈{0.0,1.0}\lambda_{\mathrm{var}} \in\{0.0, 1.0\} and λED∈{0.0,0.1,1.0}\lambda_{\mathrm{ED}} \in\{0.0, 0.1, 1.0\} for the diffusion loss terms in Equation (12). We report on Table 5 the averaged inference results at rollout horizon T=10T = 10 for each of the trained instances at matched NFE=5 for the Kolmogorov Flow data set.

Loss settingMSE ↓\downarrowCRPS ↓\downarrowRMSCE ↓\downarrow∣log⁡(SSR)∣↓\lvert\log(\mathrm{SSR})\rvert\downarrow
λED=0,λvar=1\lambda_{\mathrm{ED}} = 0, \lambda_{\mathrm{var}} = 10.02450.07660.10040.6240
λED=0.1,λvar=0\lambda_{\mathrm{ED}} = 0.1, \lambda_{\mathrm{var}} = 00.02440.07440.04200.1520
λED=1,λvar=0\lambda_{\mathrm{ED}} = 1, \lambda_{\mathrm{var}} = 00.03880.09290.05830.1273
λED=0.1,λvar=1\lambda_{\mathrm{ED}} = 0.1, \lambda_{\mathrm{var}} = 10.01960.06620.03090.2335

Table 5. Diffusion-loss ablation on Kolmogorov Flow at NFE=5. Lower is better.

Moment matching alone (λED=0.0,λvar=1.0\lambda_{\mathrm{ED}} = 0.0, \lambda_{\mathrm{var}} = 1.0) produces predictions that are not well calibrated due to under-dispersion, whereas supervising on strong ED only (λED=1.0,λvar=0.0\lambda_{\mathrm{ED}} = 1.0, \lambda_{\mathrm{var}} = 0.0) degrades the mean accuracy of the ensemble. We find that combining both terms achieves the best overall performance under this trade-off, supporting the use of ED as a stabilizing regularizer for learning diffusion while retaining accurate mean dynamics.

To further investigate the sensitivity to the ED regularization, we fix λvar=1\lambda_{\mathrm{var}} = 1 and train the model on a finer grid of λED\lambda_{\mathrm{ED}} values. Figure 5 presents the averaged results with a rollout time of T=33T = 33 for models trained with λvar=1\lambda_{\mathrm{var}} = 1 and λED∈[0,1]\lambda_{\mathrm{ED}} \in[0, 1].

Hyperparameter sensitivity sweep plots for $\lambda_{ED}$

Figure 5. Hyperparameter sensitivity sweep for λED∈[0,1.0]\lambda_{ED} \in[0, 1.0]. The method exhibits a stability plateau for λED∈[0.1,0.5]\lambda_{ED} \in[0.1, 0.5] where calibration is minimized without degrading accuracy.

The results reveal two distinct performance regimes separated by a sharp break between λED=0.05\lambda_{\mathrm{ED}} = 0.05 and 0.10.1 consistently across all metrics. Performance on MSE and CRPS plateaus after this point showing worsening results for λED=1.0\lambda_{\mathrm{ED}} = 1.0 indicating the onset of over-regularization. Gains in RMSCE and SSR performance start flattening after the λED=0.1\lambda_{\mathrm{ED}} = 0.1 point. This study suggests that, while superior performance could be obtained with λED=0.5\lambda_{\mathrm{ED}} = 0.5, our choice of λED=0.1\lambda_{\mathrm{ED}} = 0.1 is not a narrowly tuned optimum, but rather the critical entry point into the effective regularization regime.

Impact of VAE calibration

To explore how the calibration of the VAE posterior affects the calibration of TORNADO predictions, we trained four VAE instances with β∈{10−7,10−6,10−5,10−4}\beta\in\{10^{-7}, 10^{-6}, 10^{-5}, 10^{-4}\} and one TORNADO model per VAE under identical settings for the Kolmogorov dataset. We test the average RMSCE TORNADO’s one step predictive ensemble of size K=16K = 16. We compare it to the average VAE reconstruction RMSCE. To obtain the predictive ensemble we encode each frame and sample the posterior K=16K = 16 times and decoding each sample. Results are in Table 6.

β\betaVAE RMSCE ↓\downarrowTORNADO 1-step RMSCE ↓\downarrow
10−710^{-7}0.2840.098
10−610^{-6}0.1000.073
10−510^{-5}0.1690.148
10−410^{-4}0.2180.201

Table 6. Calibration results for the VAE’s reconstruction task and TORNADO’s one-step prediction task averaged over the validation set for different values of β\beta

Indeed, the gains depend on the quality of the VAE posterior, and an appropriate trade-off must be achieved. Two failure modes can be identified. When β\beta is too small, the posterior collapses toward a Dirac distribution (autoencoder regime), leaving TORNADO with no meaningful variance target to match. Nevertheless, the learned stochasticity still improves calibration compared to the raw VAE posterior. Conversely, when β\beta is too large, the posterior becomes overly smooth, inducing over-dispersed targets and degrading calibration.

Interestingly, in all regimes, TORNADO manages to improve the one step forecast ensemble calibration with respect to the underlying VAE reconstruction calibration baseline, suggesting that the latter does not represent a lower bound for the calibration of TORNADO.

Importantly, however, we observe that the VAE setting yielding the best-calibrated latent posterior also leads to the best-calibrated predictor. This suggests that TORNADO effectively inherits and refines the uncertainty structure of the latent representation when the latter is well balanced.

Impact of posterior quality on rollouts

We repeat the sweep of Appendix § 15.§ 2 with independently trained VAEs and TORNADO models for the same values of β\beta, and evaluate 20-step rollouts on Kolmogorov flow (NFE=5, K=16K=16). Table 7 reports the reconstruction error and KL divergence of each VAE, and the rollout MSE, RMSCE and sharpness of the corresponding TORNADO model.

β\betaRecon. vRMSE ↓\downarrowKL (nats/sample)Rollout MSE ↓\downarrowRollout RMSCE ↓\downarrowSharpness ↓\downarrow
10−710^{-7}0.001811031.90.02160.05140.0907
10−610^{-6}0.00245376.70.20600.03930.4301
10−510^{-5}0.00842004.00.21770.02870.3467
10−410^{-4}0.0357635.42.07230.03881.7371

Table 7. VAE regularization sweep on Kolmogorov flow: latent statistics of each VAE and 20-step rollout metrics of the TORNADO model trained on it (NFE=5, K=16K=16).

As expected, increasing β\beta raises the reconstruction error and lowers the KL divergence. At β=10−7\beta= 10^{-7}, TORNADO attains the lowest rollout MSE, but its ensembles are the narrowest and its calibration error exceeds that obtained for β∈{10−6,10−5}\beta\in\{10^{-6}, 10^{-5}\}, consistent with the near-deterministic posterior discussed in Appendix § 15.§ 2. At β=10−4\beta= 10^{-4}, the rollout MSE increases by an order of magnitude and the ensemble spread by a factor of four relative to β=10−6\beta= 10^{-6}, while RMSCE remains low. The low calibration error is then a consequence of the inflated spread, so RMSCE must be read together with accuracy and sharpness (Appendix § 16.§ 4).

The range β∈[10−6,10−5]\beta\in[10^{-6}, 10^{-5}] combines low reconstruction error, low calibration error and moderate spread. This agrees with Appendix § 15.§ 2, where the best one-step calibration is obtained at β=10−6\beta= 10^{-6}: both sweeps identify the same regime. In practice, we recommend a logarithmic sweep over β\beta, retaining the range in which reconstruction remains accurate and calibration and sharpness are preserved. All methods in the main text use β=10−6\beta= 10^{-6}; this value is not tuned per method.

Comparison of higher order solvers

To discern the impact of the choice of ODE or SDE solver on the final results, we test TORNADO and FM with higher-order solvers: Heun (for FM) and a derivative-free Milstein scheme ([24]) (for TORNADO), alongside their standard counterparts Euler and EM. Results2 for Kolmogorov at T=20T = 20 are in Table 8.

MethodMSE ↓\downarrowCRPS ↓\downarrowRMSCE ↓\downarrow∣log⁡(SSR)∣↓\lvert\log(\mathrm{SSR}) \rvert\downarrow
TORNADO-EM0.1550.1670.0530.376
TORNADO-Milstein0.1550.1670.0530.376
FM-Euler0.1070.1630.2031.500
FM-Heun0.1060.1610.1951.404

Table 8. Results for Kolmogorov at T=20T = 20 with different ODE/SDE solvers for TORNADO (Euler-Maruyama and Milstein) and FM (Euler and Heun).

Higher-order solvers do not change the conclusions: FM-Heun improves slightly over FM-Euler yet remains worse than TORNADO-EM, and Milstein yields identical results to EM. This is consistent with a smooth learned diffusion field, where higher-order stochastic corrections are negligible.

Ablation on gradient stopping

To motivate the design choice of stopping the diffusion head’s gradients from propagating through the shared trunk, we report on Table 9 the average performance at NFE=5 and T=10T = 10 for models trained on the Kolmogorov Flow with and without adding a detach (gradient stopping) to the diffusion head’s input.

SettingMSE ↓\downarrowCRPS ↓\downarrowRMSCE ↓\downarrow∣log⁡(SSR)∣↓\lvert\log(\mathrm{SSR}) \rvert\downarrow
w/ detach0.01950.06620.03090.2334
w/o detach0.02030.06720.03040.2296

Table 9. Gradient-stop ablation (detach diffusion head) on Kolmogorov Flow at NFE=5, averaged over rollout horizon T=10T = 10. Lower is better.

Detaching the diffusion head gradients from the shared trunk has negligible effect on both accuracy and calibration at NFE=5 on Kolmogorov Flow, suggesting that the drift learning is largely unaffected by whether diffusion gradients backpropagate into the trunk. We therefore treat the diffusion head as a modular add-on and keep gradient stopping as a conservative default to reduce potential gradient interference, without affecting performance. We leverage this property by only training the diffusion head once the drift learning has converged, reducing the training cost significantly by limiting the sample based ED computations.

Computing time comparison

Admittedly, the implementation of TORNADO entails the addition of an extra component to the shared trunk model used by the baselines. Namely, we require the use of an extra diffusion head that models the diffusion term of our SDE. On Table 10 we report the average physical time step transition time (t→t+1t \to t + 1) in function of the number of integrator steps (NFE). The time is measured after a GPU warmup of 50 steps, averaged over 200 steps and 3 repetitions. To compare with other experiments, the measurement is taken for a batch size of 32 and an ensemble size of 16 with the Kolmogorov Flow data set. Measurements are taken on a NVIDIA H100 80GB HBM3.

MethodNFE=2NFE=5NFE=10
DDIM555.001387.702773.38
FM555.161387.802775.52
SI554.701387.572775.01
TORNADO602.411505.953011.94

Table 10. Compute cost (ms per forecast step) at batch size B=32B = 32 and ensemble size K=16K = 16. Lower is better.

Table 1: Interval evaluation at NFE=5 over each dataset’s horizon: steps 1–20, except ERA5 (1–40). Mean ± std over the test set, except Combustion, where ± is the spread across the four held-out conditions. ERA5 aggregates its four variables in normalised, cos(lat)-weighted units. Best and second-best per dataset and metric are highlighted (green/blue).

DatasetModelMSE (10−310^{-3}) (↓)CRPS (10−210^{-2}) (↓)RMSCE (10−210^{-2}) (↓)∣log⁡(SSR)∣|\log(\mathrm{SSR})| (↓)
KolmogorovDDIM178 ± 418.84 ± 0.149.52 ± 0.050.628 ± 0.005
FM107 ± 316.33 ± 0.1520.33 ± 0.031.494 ± 0.006
SI220 ± 522.43 ± 0.1410.56 ± 0.060.653 ± 0.005
TORNADO120 ± 214.53 ± 0.123.95 ± 0.060.268 ± 0.006
KSDDIM33.88 ± 0.1411.12 ± 0.0215.051 ± 0.0150.8773 ± 0.0014
FM9.138 ± 0.0155.152 ± 0.0055.483 ± 0.0100.2984 ± 0.0007
SI9.604 ± 0.0095.220 ± 0.0033.378 ± 0.0090.1417 ± 0.0003
TORNADO7.824 ± 0.0124.760 ± 0.0042.509 ± 0.0070.0209 ± 0.0007
BurgersDDIM0.789 ± 0.0061.523 ± 0.00411.00 ± 0.040.575 ± 0.003
FM0.611 ± 0.0041.383 ± 0.00410.80 ± 0.030.647 ± 0.002
SI0.676 ± 0.0041.496 ± 0.0027.31 ± 0.030.409 ± 0.002
TORNADO0.764 ± 0.0071.462 ± 0.0052.55 ± 0.020.003 ± 0.005
GSDDIM0.710 ± 0.0061.226 ± 0.0047.943 ± 0.0090.5121 ± 0.0018
FM0.5176 ± 0.00081.0692 ± 0.00088.250 ± 0.0090.5598 ± 0.0011
SI0.5238 ± 0.00151.0695 ± 0.00124.40 ± 0.020.2683 ± 0.0007
TORNADO0.4163 ± 0.00120.9489 ± 0.00152.476 ± 0.0160.0372 ± 0.0014
CombustionDDIM1.67 ± 1.510.812 ± 0.62713.18 ± 0.952.395 ± 0.712
FM1.65 ± 1.400.852 ± 0.59013.63 ± 1.121.332 ± 0.241
SI1.88 ± 1.200.954 ± 0.4309.52 ± 1.320.526 ± 0.113
TORNADO1.47 ± 1.260.775 ± 0.5128.91 ± 1.800.549 ± 0.219
ERA5DDIM288.5 ± 22.823.37 ± 1.0515.39 ± 1.110.650 ± 0.041
FM292.5 ± 24.523.19 ± 1.1212.91 ± 1.320.576 ± 0.042
SI265.0 ± 20.221.13 ± 0.916.39 ± 0.860.266 ± 0.033
TORNADO266.0 ± 20.821.16 ± 0.895.77 ± 0.750.254 ± 0.033

Table 1. Interval evaluation at NFE=5NFE=5 over each dataset’s horizon: steps 1–20, except ERA5 (1–40). Mean ± std over the test set, except Combustion, where ± is the spread across the four held-out conditions. ERA5 aggregates its four variables in normalised, cos(lat)-weighted units. Best and second-best per dataset and metric are highlighted (green/blue).

Overall, the addition of the diffusion head incurs on a compute time around 8.5% higher than that of the drift model. We deem this added cost acceptable in trade-off for the improved calibration of the predictions.

Effect of σγ\sigma_\gamma on calibration

The stochastic interpolant in Eq. (8) injects Gaussian noise with variance γ(τ)=σγτ(1−τ)\gamma(\tau) = \sigma_\gamma\tau(1 - \tau). This hyperparameter affects both the sampled intermediate states zτz_\tau and the closed-form conditional targets μ1∣τ\mu_{1|\tau} and Σ1∣τ\Sigma_{1|\tau} used for supervision. In particular, it controls how informative zτz_\tau is about the endpoint z1z_1 through the conditional covariance Σ1∣τ\Sigma_{1|\tau}.

Theoretical observation (see Appendix § 7.§ 3, Corollary A.4). For fixed τ∈(0,1)\tau\in(0, 1), increasing σγ\sigma_\gamma monotonically increases the conditional endpoint covariance Σ1∣τ\Sigma_{1|\tau}, interpolating from a low-noise regime to a noise-dominated regime where Σ1∣τ→Σ1\Sigma_{1|\tau} \to\Sigma_1 as σγ→∞\sigma_\gamma\to\infty. Moreover, for any fixed σγ>0\sigma_\gamma> 0, we have Σ1∣τ→Σ1\Sigma_{1|\tau} \to\Sigma_1 as τ→0+\tau\to0^+ and Σ1∣τ→0\Sigma_{1|\tau} \to0 as τ→1−\tau\to1^-. This suggests a direct mechanism by which σγ\sigma_\gamma influences predictive dispersion: increasing σγ\sigma_\gamma increases Σ1∣τ\Sigma_{1|\tau}, reducing the contraction of uncertainty about the endpoint induced by conditioning on zτz_\tau. Motivated by this, we empirically evaluate calibration and dispersion as σγ\sigma_\gamma varies.

Figure 6 reports averaged RMSCE and ∣log⁡(SSR)∣|\log(\mathrm{SSR})| over 10 rollout steps, spatial grid, and batch elements for predictions of models trained with σγ∈{0,0.1,0.5,1,10.0}\sigma_\gamma\in\{0, 0.1, 0.5, 1, 10.0\}. The effect of σγ\sigma_\gamma is asymmetrical across PDEs; however, a consistent trend can be observed: at σγ=0\sigma_\gamma= 0, models exhibit poor calibration and under-dispersion. At larger values of σγ\sigma_\gamma, while calibration improves, SSR indicates over-spread predictions. Overall, the experimental findings are consistent with the theoretical results and moderate noise levels provide the best trade-off, we chose σγ=0.1\sigma_\gamma= 0.1 in all experiments.

RMSCE and $|\log(\mathrm{SSR})|$ for the Kolmogorov Flow, Burgers and KS datasets

Figure 6. RMSCE and ∣log⁡(SSR)∣|\log(\mathrm{SSR})| at σγ∈{0,0.1,0.5,1,10.0}\sigma_\gamma\in\{0, 0.1, 0.5, 1, 10.0\} for the Kolmogorov Flow, Burgers and KS datasets averaged over 10 rollout steps.

Ensemble size KK

We explore the impact of the inference ensemble size KK choice. In Figure 7 we sweep K∈{4,8,16,32,64}K \in\{4, 8, 16, 32, 64\} on Kolmogorov flow at NFE=5 over 20 autoregressive steps, reusing the trained models and the forecasting protocol unchanged, only the number of sampled members varies. Increasing the ensemble size improves TORNADO’s calibration, with RMSCE falling from about 0.11 at K=4K = 4 to 0.045 at K=64K = 64. TORNADO has the lowest RMSCE from K=8K = 8 onwards, with most of the improvement occurring by K=16K = 16, showing it is a reasonable choice in the trade-off between performance and cost. Its CRPS also decreases, approaching FM’s at larger ensemble sizes. The spread–skill ratio moves towards one but remains below it, while the 95% intervals widen moderately. DDIM improves more modestly, and FM retains the narrowest intervals and highest calibration error. SI’s CRPS improves with larger ensembles, but its calibration worsens as its intervals widen and its spread–skill ratio rises further above one.

Calibration, predictive skill, and spread across ensemble sizes

Figure 7. Calibration, predictive skill, and spread across ensemble sizes. Lower RMSCE and CRPS are better; the target spread–skill ratio is one.

Additional experiments

Extension to Auto-Regressive training

While the main paper adopts one-step teacher-forced training as a controlled simplification, TORNADO is not restricted to this setting. We additionally train autoregressive variants that unroll the latent dynamics over horizons H∈{2,4}H \in\{2, 4\} during training. Starting from the posterior of the current frame, the model is recursively rolled forward and re-anchored at each step using the encoder posterior of the predicted state, while the same local drift, variance, and energy-distance objectives are applied at every step with a decaying weight over the rollout horizon. This extends TORNADO beyond teacher forcing while preserving its local conditional supervision. As shown in Table 11, the autoregressive variants retain the gains of the base method and further improve calibration, with H=4H = 4 also giving the strongest overall performance.

Method / training regimeMSE ↓CRPS ↓RMSCE ↓∣log⁡(SSR)∣\lvert\log(\mathrm{SSR})\rvert ↓
TORNADO (TF)0.1200.1450.03950.268
TORNADO-AR (H=2H = 2)0.1390.1570.03680.225
TORNADO-AR (H=4H = 4)0.1110.1410.03180.180

Table 11. Autoregressive extension of TORNADO on the Kolmogorov Flow dataset. Extending training beyond teacher forcing preserves the benefits of the base method and further improves calibration, with H=4H = 4 achieving the best overall performance.

Comparison to one-step baselines

We additionally consider three simple latent baselines that bypass auxiliary-time transport and predict the next latent state directly in a single step. MC Dropout predicts the next latent state with a standard MSE loss and relies on stochastic dropout at inference to produce ensemble uncertainty. Gaussian NLL takes the current posterior statistics (μt\mu_t, σt\sigma_t) as input and predicts the mean and diagonal standard deviation of a Gaussian next-state distribution, trained by Gaussian negative log-likelihood against a sample from the next VAE posterior. VAE Matching uses the same parameterization, but is trained to match the mean and variance of the next VAE posterior directly through a Gaussian KL. Unlike TORNADO, these baselines do not introduce an auxiliary-time stochastic transport between consecutive latent posteriors. The results for the Kolmogorov dataset averaged on a 20 step rollout are shown in Table 12.

MethodMSE ↓CRPS ↓RMSCE ↓|log(SSR)| ↓
TORNADO (NFE=5)0.1200.1450.0390.268
NLL0.3260.2430.1922.344
VAE matching0.6870.3600.1992.628
MC Dropout0.3210.2500.0750.447

Table 12. Comparison with direct one-step latent baselines on the Kolmogorov Flow dataset.

Across all three baselines, TORNADO yields better calibrated predictive distributions and more stable autoregressive rollouts, showing that direct one-step latent uncertainty prediction is not sufficient to match the benefits of adaptive stochastic transport. We do not consider approaches such as a deep ensemble comparison because they would introduce substantially different training, storage, and parameter budgets from our controlled comparison.

Long-horizon statistical evaluation via total enstrophy

At long rollout horizons, pointwise trajectory agreement becomes increasingly difficult to interpret in chaotic systems. Following the long-horizon statistical evaluation protocol of Zighed et al. [55], we therefore compare the distributions of a coarse scalar statistic over fixed rollout windows. For the Kolmogorov flow, total enstrophy is a natural choice, as it measures the global intensity of vorticity fluctuations and remains meaningful beyond the horizon where pointwise comparisons become difficult to interpret. We define the total enstrophy of a snapshot as

Z(ωt)=∑i,j,cωt(i,j,c)2.Z(\omega_t) = \sum_{i,j,c} \omega_t(i,j,c)^2 .

For each window of size 20 in T=100T = 100 rollouts, we compare the empirical distributions of predicted and ground-truth enstrophy values over the test set using the 1D Wasserstein distance.

Figure 8 reports the 1D Wasserstein distance between the predicted and ground-truth total-enstrophy distributions across 20-step rollout windows. TORNADO achieves the smallest distance in all windows, while SI is generally the next closest method and FM and DDIM exhibit larger statistical drift, particularly at later horizons. Under this coarse statistical evaluation, TORNADO is therefore the most consistent method over long rollouts on the Kolmogorov dataset.

Wasserstein distance between predicted and ground-truth total-enstrophy distributions

Figure 8. 1D Wasserstein distance between the predicted and ground-truth total-enstrophy distributions over the rollout windows of length 20, aggregated over the test set for Kolmogorov Flow over 100 step trajectories. Lower is better.

While the Wasserstein analysis quantifies distributional agreement aggregated over the test set, it does not reveal how each method’s predictive band tracks the ground truth along an individual rollout. Calibrated probabilistic forecasting requires not only that the marginal distribution of physical statistics match in aggregate, but that the predictive band actually contain the realized trajectory at the rate implied by its nominal coverage. As a qualitative complement, in Figure 9 we examine the evolution of normalized total enstrophy Z(t)/Z(0)Z(t)/Z(0) along one illustrative ground-truth trajectory together with the predictive ensemble mean and standard deviation for each method. TORNADO tracks the evolution of this coarse statistic closely over the first half of the rollout, and as its mean prediction begins to drift, the ensemble spread widens accordingly. FM also follows the early dynamics well, but remains comparatively underdispersed once it departs from the ground-truth trajectory. SI exhibits the opposite behavior, with a broader spread already present from the start.

Normalized total enstrophy over two representative Kolmogorov flow rollouts

Figure 9. Normalized total enstrophy over two representative Kolmogorov flow rollouts. Red curves show the ground truth; gray curves and shaded regions show the predictive ensemble mean and one standard deviation for each method.

DDIM departs from the ground-truth statistics earlier and remains less well aligned throughout the rollout.

Calibration beyond RMSCE

RMSCE and ∣log⁡(SSR)∣|\log(\mathrm{SSR})| collapse some aspects of calibration into one scalar. On one hand, good calibration and SSR can be achieved at low accuracy by widening the predictive intervals. While these metrics were reported alongside MSE and CRPS, which constitute a measure of accuracy and sharpness, the analysis can be made more explicit by looking at sharpness measures, like the width of the predictive ensemble. On the other hand, by collapsing the RMSCE into a scalar and taking the log absolute value of the SSR, we lose the information on whether miscalibration comes from underspreading (U-shaped PIT, SSR <1< 1) or overspreading (bell-shaped PIT, SSR >1> 1).

Absolute sharpness. SSR measures spread relative to error: a value near one indicates that the two match, but does not imply narrow intervals. We therefore report mean predictive interval width alongside calibration error for Kolmogorov flow with NFE=5, K=16K=16, and 20 rollout steps. Table 13 reports 95% interval widths and coverage, while Figure 10 plots pooled RMSCE against mean 90% interval width.

Figure 10. 20 step pooled calibration error versus mean 90% predictive interval width on Kolmogorov flow.

MethodRMSCE ↓PI width@95%cov.@95%SSR → 1CRPS ↓
DDIM0.0950.5210.6320.5330.188
FM0.2030.1560.2840.2250.163
SI0.1062.0370.9901.9210.224
TORNADO0.0390.6240.8000.7650.145

Table 13. Calibration, sharpness, and predictive skill on Kolmogorov flow.

TORNADO has the lowest calibration error, and its 95% intervals are 3.3×3.3 \times narrower than SI’s. Its calibration advantage therefore does not come from wider intervals. FM has the narrowest intervals, but they cover only 28.4% of observations at the nominal 95% level, and its calibration error is the largest. TORNADO’s coverage is higher at 80.0%, though still below the 95% target. Interval width must therefore be read alongside coverage. Figure 10 shows no clear trade-off between calibration and sharpness: TORNADO lies toward the lower-left corner, while the baselines vary across both axes.

Per-level calibration. We assess calibration using the rank of the ground truth among the KK predictive samples, normalised to [0,1][0, 1], as an empirical probability integral transform (PIT). For a calibrated ensemble, these ranks are uniformly distributed over the available rank positions. We also examine observed coverage G^(p)\hat{G}(p) at each nominal level pp. The RMSCE defined in Appendix § 13 summarizes the root-mean-square deviation of this coverage curve from the diagonal.

RMSCE measures the size of calibration errors but not their direction. A U-shaped rank histogram indicates insufficient ensemble spread, while a center-peaked histogram indicates excessive spread. The corresponding coverage curves show whether predictive intervals under-cover or over-cover at each nominal level.

Figure 11 and Figure 12 show this difference for SI and DDIM. Their RMSCE values are similar, but SI’s histogram is centre-peaked and its PIT CDF lies below the diagonal at low quantiles and above it at high quantiles, consistent with excessive spread. DDIM’s histogram is U-shaped and its PIT CDF exhibits the opposite pattern. TORNADO has the flattest histogram and the coverage curve closest to the diagonal across the evaluated levels. Its lower RMSCE thus reflects better calibration across nominal levels, rather than an improvement confined to a few levels.

Figure 11. Reliability diagrams on Kolmogorov flow (NFE=5, K=16K=16), one-step (left) and over the 20-step rollout (right). Curves above the diagonal over-cover, below under-cover.

PIT histograms on Kolmogorov flow

Figure 12. PIT histograms on Kolmogorov flow NFE=5, K=16K=16, 20 rollout steps

Figures 13 and 14 extend this analysis to Combustion and ERA5. On both datasets, FM and DDIM exhibit pronounced boundary peaks, consistent with insufficient ensemble spread, whereas TORNADO and SI have substantially flatter histograms. On Combustion, both retain an excess of low PIT values, indicating that observations fall below their predictive ensembles more often than expected. On ERA5, both show moderate excess mass at the boundaries, suggesting residual underdispersion. Unlike on Kolmogorov flow, SI does not exhibit a pronounced centre peak, and its rank profile is similar to TORNADO’s. These results show that the direction and severity of miscalibration depend on the dataset, while TORNADO maintains comparatively balanced rank profiles across all three settings.

Error vs Variance allocation

To assess how each model allocates uncertainty relative to its own errors, we compute pointwise MSE and ensemble variance in latent space over the first 10 rollout steps. For each method, spatial points are grouped into low-, medium-, and high-error terciles based on that model’s MSE. As shown in Table 14, both methods assign higher variance to higher-error regions. However, TORNADO shows a tighter correspondence between variance and error across terciles. SI remains globally over-dispersed, especially in low-error regions, whereas TORNADO keeps the variance-MSE ratio much better aligned with its errors. This suggests that learned diffusion improves calibration primarily through better magnitude control.

MSE tercile (method specific)TORNADOSI
MSEVarVar/MSEMSEVarVar/MSE
low error0.00040.006115.00.00080.0897111.4
mid error0.00400.00812.00.00800.118014.8
high error0.06280.02150.340.11090.29982.70

Table 14. Within-method tercile analysis of latent-space error and variance for TORNADO and SI.

PIT histograms on Combustion

Figure 13. PIT histograms on Combustion NFE=5, K=16K=16, 20 rollout steps

PIT histograms on ERA5

Figure 14. PIT histograms on ERA5 NFE=5, K=16K=16, 20 rollout steps

ERA5 per-variable results

Table 1 aggregates the four ERA5 variables into a single score. Table 15 reports the same metrics per variable, computed from the same rollouts. Metrics in Table 1 are computed in normalised units with cos⁡(latitude)\cos(\mathrm{latitude}) weighting. MSE and CRPS are averaged over the four variables, so they equal the means of Table 15, whereas RMSCE and SSR are computed after pooling variables (one rank histogram over all variables, and spread and error pooled across variables), so they are not averages of the per-variable scores.

VariableModelMSE (10−310^{-3}) (↓\downarrow)CRPS (10−210^{-2}) (↓\downarrow)RMSCE (10−210^{-2}) (↓\downarrow)∣log⁡(SSR)∣\lvert\log(\mathrm{SSR})\rvert (↓\downarrow)
T2mT_{2m}DDIM13.116±2.63413.116 \pm2.6345.60±0.445.60 \pm0.4417.72±2.1617.72 \pm2.160.787±0.0510.787 \pm0.051
T2mT_{2m}FM13.367±2.56913.367 \pm2.5695.52±0.435.52 \pm0.4315.91±2.2215.91 \pm2.220.637±0.0460.637 \pm0.046
T2mT_{2m}SI11.604±2.21411.604 \pm2.2144.90±0.384.90 \pm0.388.44±1.488.44 \pm1.480.292±0.0450.292 \pm0.045
T2mT_{2m}TORNADO11.885±2.21211.885 \pm2.2124.94±0.354.94 \pm0.357.53±1.217.53 \pm1.210.265±0.0420.265 \pm0.042
U10U_{10}DDIM451.1±40.7451.1 \pm40.735.06±1.6735.06 \pm1.6712.35±0.5112.35 \pm0.510.655±0.0440.655 \pm0.044
U10U_{10}FM460.3±48.3460.3 \pm48.335.08±1.8935.08 \pm1.8911.00±0.6511.00 \pm0.650.578±0.0480.578 \pm0.048
U10U_{10}SI417.4±37.0417.4 \pm37.032.16±1.5132.16 \pm1.515.71±0.635.71 \pm0.630.268±0.0350.268 \pm0.035
U10U_{10}TORNADO416.7±38.7416.7 \pm38.732.14±1.5132.14 \pm1.515.48±0.555.48 \pm0.550.255±0.0390.255 \pm0.039
V10V_{10}DDIM641.8±55.1641.8 \pm55.141.68±1.8341.68 \pm1.8312.04±0.4912.04 \pm0.490.632±0.0440.632 \pm0.044
V10V_{10}FM645.3±53.3645.3 \pm53.341.39±1.8041.39 \pm1.8010.67±0.4710.67 \pm0.470.564±0.0410.564 \pm0.041
V10V_{10}SI587.5±46.9587.5 \pm46.938.16±1.5638.16 \pm1.565.63±0.495.63 \pm0.490.263±0.0360.263 \pm0.036
V10V_{10}TORNADO591.4±47.1591.4 \pm47.138.25±1.5038.25 \pm1.505.44±0.395.44 \pm0.390.255±0.0330.255 \pm0.033
Z500Z_{500}DDIM48.052±8.05748.052 \pm8.05711.13±1.0611.13 \pm1.0624.07±3.4624.07 \pm3.460.798±0.0690.798 \pm0.069
Z500Z_{500}FM51.090±9.93951.090 \pm9.93910.76±1.1510.76 \pm1.1518.76±4.1518.76 \pm4.150.669±0.0810.669 \pm0.081
Z500Z_{500}SI43.264±6.44943.264 \pm6.4499.29±0.719.29 \pm0.719.16±3.089.16 \pm3.080.250±0.0550.250 \pm0.055
Z500Z_{500}TORNADO43.940±7.25543.940 \pm7.2559.30±0.779.30 \pm0.777.73±2.577.73 \pm2.570.219±0.0590.219 \pm0.059

Table 15. ERA5 per-variable results on the held-out years (NFE=5, K=16K=16, steps 1–40, 70 test trajectories). Metrics are in normalised, cos⁡(lat)\cos(\mathrm{lat})-weighted units; mean ±\pm std over the test set. Best and second-best per variable and metric are highlighted (green/blue).

The pattern is identical on every field: SI and TORNADO achieve similar MSE and CRPS on all four fields, while TORNADO attains the lowest calibration error and the ∣log⁡(SSR)∣\lvert\log(\mathrm{SSR})\rvert closest to zero on all four. The aggregate is therefore not driven by a single atmospheric quantity.

NFE effect

Here we show similar plots as in 4.4 for NFE=2NFE = 2 and 10. Increased integration steps benefits baselines while TORNADO is comparatively NFE-insensitive in calibration. In Figure 17 we show the T=50T = 50 time-averaged metrics as a function of NFE. TORNADO remains stable across the tested range, with MSE and CRPS varying only mildly and calibration strongest at intermediate budgets. This supports treating NFE as a finite sampler budget in the compute-constrained regime rather than as a setting that must be increased for reliable performance.

NFE sensitivity on Kolmogorov Flow

Figure 17. NFE sensitivity on Kolmogorov Flow. We report rollout MSE, CRPS, RMSCE, and SSR as a function of NFE used by method. Metrics are averaged over the same rollout horizon T=50T = 50.

Kolmogorov flow rollout metrics versus horizon t at fixed compute for NFE=2

Figure 15. Kolmogorov flow rollout metrics vs. horizon tt at fixed compute (NFE=2NFE=2): MSE, CRPS, RMSCE, and SSR.

Kolmogorov flow rollout metrics versus horizon t at fixed compute for NFE=10

Figure 16. Kolmogorov flow rollout metrics vs. horizon tt at fixed compute (NFE=10NFE=10): MSE, CRPS, RMSCE, and SSR.

Visualizations

Visualization of rollout predictions at NFE=5 for methods TORNADO, FM, DDIM and SI alongside the ground truth trajectory for the Kolmogorov Flow data set

Figure 18. Visualization of rollout predictions at NFE=5 for methods TORNADO, FM, DDIM and SI alongside the ground truth trajectory for the Kolmogorov Flow data set.

Visualization of the 95% confidence interval and RMSE of the predictive ensemble distribution

Figure 19. Visualization of the 95% confidence interval of the predictive ensemble distribution at NFE=5 for methods TORNADO, FM, DDIM and SI alongside the RMSE of the mean of the predictive ensemble distribution with respect to the ground truth for the Kolmogorov Flow data set.

Visualization of rollout predictions at NFE=5 for methods TORNADO, FM, DDIM and SI alongside the ground truth trajectory for the Burgers data set.

Figure 20. Visualization of rollout predictions at NFE=5 for methods TORNADO, FM, DDIM and SI alongside the ground truth trajectory for the Burgers data set.

Visualization of the 95% confidence interval and RMSE for methods TORNADO, FM, DDIM and SI on the Burgers data set

Figure 21. Visualization of the 95% confidence interval of the predictive ensemble distribution at NFE=5 for methods TORNADO, FM, DDIM and SI alongside the RMSE of the mean of the predictive ensemble distribution with respect to the ground truth for the Burgers data set.

Visualization of rollout predictions at NFE=5 for methods TORNADO, FM, DDIM and SI alongside the ground truth trajectory for the KS data set.

Figure 22. Visualization of rollout predictions at NFE=5 for methods TORNADO, FM, DDIM and SI alongside the ground truth trajectory for the KS data set.

Visualization of the predictive ensemble distribution and RMSE for TORNADO, FM, DDIM and SI

Figure 23. Visualization of the 95% confidence interval of the predictive ensemble distribution at NFE=5 for methods TORNADO, FM, DDIM and SI alongside the RMSE of the mean of the predictive ensemble distribution with respect to the ground truth for the KS data set.

Visualization of rollout predictions at NFE=5 for methods TORNADO, FM, DDIM and SI alongside the ground truth trajectory for the GS data set.

Figure 24. Visualization of rollout predictions at NFE=5 for methods TORNADO, FM, DDIM and SI alongside the ground truth trajectory for the GS data set.

Visualization of the 95% confidence interval of the predictive ensemble distribution at NFE=5 for methods TORNADO, FM, DDIM and SI alongside the RMSE of the mean of the predictive ensemble distribution with respect to the ground truth for the GS data set.

Figure 25. Visualization of the 95% confidence interval of the predictive ensemble distribution at NFE=5 for methods TORNADO, FM, DDIM and SI alongside the RMSE of the mean of the predictive ensemble distribution with respect to the ground truth for the GS data set.

Visualization of rollout predictions at NFE=5 for methods TORNADO, FM, DDIM and SI alongside the ground truth trajectory for the RealPDEBench Combustion data set.

Figure 26. Visualization of rollout predictions at NFE=5 for methods TORNADO, FM, DDIM and SI alongside the ground truth trajectory for the RealPDEBench Combustion data set.

Visualization of the 95% confidence interval of the predictive ensemble distribution at NFE=5 for methods TORNADO, FM, DDIM and SI alongside the RMSE of the mean of the predictive ensemble distribution with respect to the ground truth for the RealPDEBench Combustion data set

Figure 27. Visualization of the 95% confidence interval of the predictive ensemble distribution at NFE=5 for methods TORNADO, FM, DDIM and SI alongside the RMSE of the mean of the predictive ensemble distribution with respect to the ground truth for the RealPDEBench Combustion data set.

Visualization of rollout predictions at NFE=5 for methods TORNADO, FM, DDIM and SI alongside the ground truth trajectory for the $T_{2m}$ variable WeatherBench ERA5 data set.

Figure 28. Visualization of rollout predictions at NFE=5 for methods TORNADO, FM, DDIM and SI alongside the ground truth trajectory for the T2mT_{2m} variable WeatherBench ERA5 data set.

Visualization of the 95% confidence interval of the predictive ensemble distribution at NFE=5 for methods TORNADO, FM, DDIM and SI alongside the RMSE of the mean of the predictive ensemble distribution with respect to the ground truth for the $T_{2m}$ variable WeatherBench ERA5 data set.

Figure 29. Visualization of the 95% confidence interval of the predictive ensemble distribution at NFE=5 for methods TORNADO, FM, DDIM and SI alongside the RMSE of the mean of the predictive ensemble distribution with respect to the ground truth for the T2mT_{2m} variable WeatherBench ERA5 data set.

Visualization of rollout predictions at NFE=5 for methods TORNADO, FM, DDIM and SI alongside the ground truth trajectory for the $U_{10}$ variable WeatherBench ERA5 data set.

Figure 30. Visualization of rollout predictions at NFE=5 for methods TORNADO, FM, DDIM and SI alongside the ground truth trajectory for the U10U_{10} variable WeatherBench ERA5 data set.

Visualization of the 95% confidence interval of the predictive ensemble distribution at NFE=5 for methods TORNADO, FM, DDIM and SI alongside the RMSE of the mean of the predictive ensemble distribution with respect to the ground truth for the $U_{10}$ variable WeatherBench ERA5 data set.

Figure 31. Visualization of the 95% confidence interval of the predictive ensemble distribution at NFE=5 for methods TORNADO, FM, DDIM and SI alongside the RMSE of the mean of the predictive ensemble distribution with respect to the ground truth for the U10U_{10} variable WeatherBench ERA5 data set.

Visualization of rollout predictions at NFE=5 for methods TORNADO, FM, DDIM and SI alongside the ground truth trajectory

Figure 32. Visualization of rollout predictions at NFE=5 for methods TORNADO, FM, DDIM and SI alongside the ground truth trajectory for the V10_{10} variable WeatherBench ERA5 data set.

Visualization of the 95% confidence interval of the predictive ensemble distribution at NFE=5 for methods TORNADO, FM, DDIM and SI alongside the RMSE of the mean of the predictive ensemble distribution

Figure 33. Visualization of the 95% confidence interval of the predictive ensemble distribution at NFE=5 for methods TORNADO, FM, DDIM and SI alongside the RMSE of the mean of the predictive ensemble distribution with respect to the ground truth for the V10_{10} variable WeatherBench ERA5 data set.

Visualization of rollout predictions at NFE=5 for methods TORNADO, FM, DDIM and SI alongside the ground truth trajectory for the Z500 variable WeatherBench ERA5 data set

Figure 34. Visualization of rollout predictions at NFE=5 for methods TORNADO, FM, DDIM and SI alongside the ground truth trajectory for the Z500Z_{500} variable WeatherBench ERA5 data set.

Visualization of the 95% confidence interval of the predictive ensemble distribution at NFE=5 for methods TORNADO, FM, DDIM and SI alongside the RMSE of the mean of the predictive ensemble distribution with respect to the ground truth for the Z500 variable WeatherBench ERA5 data set

Figure 35. Visualization of the 95% confidence interval of the predictive ensemble distribution at NFE=5 for methods TORNADO, FM, DDIM and SI alongside the RMSE of the mean of the predictive ensemble distribution with respect to the ground truth for the Z500Z_{500} variable WeatherBench ERA5 data set.

References

  1. [1]Michael S. Albergo, Nicholas M. Boffi, and Eric Vanden-Eijnden. Stochastic interpolants: A unifying framework for flows and diffusions, 2023. URL https://arxiv.org/abs/2303.08797.
  2. [2]Giacomo Baldan, Qiang Liu, Alberto Guardone, and Nils Thurey. Flow Matching Meets PDEs: A Unified Framework for Physics-Constrained Generation, 2025. URL https://arxiv.org/abs/2506.08604.
  3. [3]Valentin De Bortoli, Alexandre Galashov, J Swaroop Guntupalli, Guangyao Zhou, Kevin Patrick Murphy, Arthur Gretton, and Arnaud Doucet. Distributional diffusion models with scoring rules. In Forty-second International Conference on Machine Learning, 2025. URL https://openreview.net/forum?id=N82967FcVK.arxiv.org/abs/2502.02483
  4. [4]Charlotte Bunne, Ya-Ping Hsieh, Marco Cuturi, and Andreas Krause. The schrödinger bridge between gaussian measures has a closed form, 2023. URL https://arxiv.org/abs/2202.05722.
  5. [5]Christopher Bülte, Philipp Scholl, and Gitta Kutyniok. Probabilistic neural operators for functional uncertainty quantification, 2025. URL https://arxiv.org/abs/2502.12902.
  6. [6]Junyu Chen, Han Cai, Junsong Chen, Enze Xie, Shang Yang, Haotian Tang, Muyang Li, and Song Han. Deep compression autoencoder for efficient high-resolution diffusion models. In The Thirteenth International Conference on Learning Representations, 2025. URL https://openreview.net/forum?id=wH8XXUU0UZU.arxiv.org/abs/2410.10733
  7. [7]Ricky T. Q. Chen, Yulia Rubanova, Jesse Bettencourt, and David Duvenaud. Neural ordinary differential equations, 2019. URL https://arxiv.org/abs/1806.07366.
  8. [8]Yifan Chen, Mark Goldstein, Mengjian Hua, Michael S. Albergo, Nicholas M. Boffi, and Eric Vanden-Eijnden. Probabilistic Forecasting with Stochastic Interpolants and Föllmer Processes, 2024. URL https://arxiv.org/abs/2403.13724.
  9. [9]Dario Coscia, Max Welling, Nicola Demo, and Gianluigi Rozza. BARNN: A Bayesian autoregressive and recurrent neural network. arXiv preprint arXiv:2501.18665, 2025.arxiv.org/abs/2501.18665
  10. [10]Prafulla Dhariwal and Alexander Quinn Nichol. Diffusion models beat GANs on image synthesis. In A. Beygelzimer, Y. Dauphin, P. Liang, and J. Wortman Vaughan, editors, Advances in Neural Information Processing Systems, 2021. URL https://openreview.net/forum?id=AAWuCvzaVt.
  11. [11]Tilmann Gneiting and Adrian E Raftery. Strictly proper scoring rules, prediction, and estimation. Journal of the American Statistical Association, 102(477):359–378, March 2007. ISSN 1537-274X. doi: 10.1198/016214506000001437. URL http://dx.doi.org/10.1198/016214506000001437.
  12. [12]Vignesh Gopakumar, Ander Gray, Lorenzo Zanisi, Timothy Nunn, Daniel Giles, Matt Kusner, Stanislas Pamela, and Marc Peter Deisenroth. Calibrated physics-informed uncertainty quantification. In Forty-second International Conference on Machine Learning, 2025. URL https://openreview.net/forum?id=Z2uLBBck2X.arxiv.org/abs/2502.04406
  13. [13]Chuan Guo, Geoff Pleiss, Yu Sun, and Kilian Q. Weinberger. On calibration of modern neural networks, 2017. URL https://arxiv.org/abs/1706.04599.
  14. [14]Ling Guo, Hao Wu, Yan Wang, Wenwen Zhou, and Tao Zhou. Ib-uq: Information bottleneck based uncertainty quantification for neural function regression and neural operator learning. Journal of Computational Physics, 510:113089, 2024.arxiv.org/abs/2302.03271
  15. [15]Hans Hersbach, Bill Bell, Paul Berrisford, Shoji Hirahara, Andrés Horányi, Joaquín Muñoz-Sabater, Julien Nicolas, Carole Peubey, Raluca Radu, Dinand Schepers, Adrian Simmons, Cornel Soci, Saleh Abdalla, Xavier Abellan, Gianpaolo Balsamo, Peter Bechtold, Gionata Bివati, Jean Bidlot, Massimo Bonavita, Giovanna De Chiara, Per Dahlgren, Dick Dee, Michail Diamantakis, Rossana Dragani, Johannes Flemming, Richard Forbes, Manuel Fuentes, Alan Geer, Leo Haimberger, Sean Healy, Robin J. Hogan, Elías Hólm, Marta Janisková, Sarah Keeley, Patrick Laloyaux, Philippe Lopez, Cristina Lupu, Gabor Radnoti, Patricia de Rosnay, Iryna Rozum, Freja Vamborg, Sebastien Villaume, and Jean-Noël Thépaut. The era5 global reanalysis. Quarterly Journal of the Royal Meteorological Society, 146(730):1999–2049, 2020. doi: https://doi.org/10.1002/qj.3803. URL https://rmets.onlinelibrary.wiley.com/doi/abs/10.1002/qj.3803.
  16. [16]Irina Higgins, Loic Matthey, Arka Pal, Christopher Burgess, Xavier Glorot, Matthew Botvinick, Shakir Mohamed, and Alexander Lerchner. β-VAE: Learning basic visual concepts with a constrained variational framework. In International Conference on Learning Representations, 2017. URL https://openreview.net/forum?id=Sy2fzU9gl.
  17. [17]Jonathan Ho and Tim Salimans. Classifier-free diffusion guidance. In NeurIPS 2021 Workshop on Deep Generative Models and Downstream Applications, 2021. URL https://openreview.net/forum?id=qw8AKxfYbI.arxiv.org/abs/2207.12598
  18. [18]Jonathan Ho, Ajay Jain, and Pieter Abbeel. Denoising diffusion probabilistic models, 2020. URL https://arxiv.org/abs/2006.11239.
  19. [19]Xianglong Hou, Xinquan Huang, and Paris Perdikaris. Cfo: Learning continuous-time pde dynamics via flow-matched neural operators, 2025. URL https://arxiv.org/abs/2512.05297.
  20. [20]Peiyan Hu, Haodong Feng, Hongyuan Liu, Tongtong Yan, Wenhao Deng, Tianrun Gao, Rong Zheng, Haoren Zheng, Chenglei Yu, Chunraui Wang, Kaiwen Li, Zhi-Ming Ma, Dezhi Zhou, Xingcai Lu, Dixia Fan, and Tailin Wu. Realpdebench: A benchmark for complex physical systems with real-world data. In The Fourteenth International Conference on Learning Representations, 2026. URL https://openreview.net/forum?id=y3OHMco1Tr. Oral Presentation.arxiv.org/abs/2601.01829
  21. [21]Jiahe Huang, Guandao Yang, Zichen Wang, and Jeong Joon Park. Diffusionpde: Generative pde-solving under partial observation, 2024. URL https://arxiv.org/abs/2406.17763.
  22. [22]Christian Jacobsen, Yilin Zhuang, and Karthik Duraisamy. Cocogen: Physically-consistent and conditioned score-based generative models for forward and inverse problems, 2024. URL https://arxiv.org/abs/2312.10527.
  23. [23]Ruoxi Jiang, Peter Y. Lu, Elena Orlova, and Rebecca Willett. Training neural operators to preserve invariant measures of chaotic attractors, 2024. URL https://arxiv.org/abs/2306.01187.
  24. [24]Peter E Kloeden and Eckhard Platen. Numerical solution of stochastic differential equations. Stochastic Modelling and Applied Probability. Springer, Berlin, Germany, December 2010.
  25. [25]Dmitrii Kochkov, Jamie A. Smith, Ayya Alieva, Qing Wang, Michael P. Brenner, and Stephan Hoyer. Machine learning accelerated computational fluid dynamics. Proceedings of the National Academy of Sciences, 2021.DOI
  26. [26]Felix Koehler, Simon Niedermayr, Rüdiger Westermann, and Nils Thuerey. APEBench: A Benchmark for Autoregressive Neural Emulators of PDEs. In A. Globerson, L. Mackey, D. Belgrave, A. Fan, U. Paquet, J. Tomczak, and C. Zhang, editors, Advances in Neural Information Processing Systems, volume 37, pages 120252–120310. Curran Associates, Inc., 2024. doi: 10.52202/079017-3822. URL https://proceedings.neurips.cc/paper_files/paper/2024/file/d9875ebcf74bccdc5076acab0dbee62c-Paper-Datasets_and_Benchmarks_Track.pdf.DOI
  27. [27]Georg Kohl, Li-Wei Chen, and Nils Thuerey. Benchmarking autoregressive conditional diffusion models for turbulent flow simulation, 2024. URL https://arxiv.org/abs/2309.01745.
  28. [28]Armand Kassaï Koupai, Jorge Mifsut Benet, Yuan Yin, Jean-Noël Vittaut, and Patrick Gallinari. Geps: Boosting generalization in parametric pde neural solvers through adaptive conditioning. Advances in Neural Information Processing Systems, 38, 2024.
  29. [29]Armand Kassaï Koupai, Lise Le Boudec, Louis Serrano, and Patrick Gallinari. ENMA: Tokenwise autoregression for continuous neural PDE operators. In The Thirty-ninth Annual Conference on Neural Information Processing Systems, 2025. URL https://openreview.net/forum?id=3CYXSMFv55.DOI
  30. [31]Nikola B. Kovachki, Samuel Lanthaler, and Andrew M. Stuart. Chapter 9 - operator learning: Algorithms and analysis. In Siddhartha Mishra and Alex Townsend, editors, Numerical Analysis Meets Machine Learning, volume 25 of Handbook of Numerical Analysis, pages 419–467. Elsevier, 2024. doi: https://doi.org/10.1016/bs.hna.2024.05.009. URL https://www.sciencedirect.com/science/article/pii/S1570865924000097.
  31. [32]Balaji Lakshminarayanan, Alexander Pritzel, and Charles Blundell. Simple and scalable predictive uncertainty estimation using deep ensembles, 2017. URL https://arxiv.org/abs/1612.01474.
  32. [33]Remi Lam, Alvaro Sanchez-Gonzalez, Matthew Wilson, Peter Wirnsberger, Meire Fortunato, Ferran Alet, Suman Ravuri, Timo Ewalds, Zach Eaton-Rosen, Weihua Hu, Alexander Merose, Stephan Hoyer, George Holland, Oriol Vinyals, Jacklynn Stott, Alexander Pritzel, Shakir Mohamed, and Peter Battaglia. Graphcast: Learning skillful medium-range global weather forecasting, 2023. URL https://arxiv.org/abs/2212.12794.
  33. [34]Zijie Li, Saurabh Patil, Francis Ogoke, Dule Shu, Wilson Zhen, Michael Schneier, John R. Buchanan, and Amir Barati Farimani. Latent neural pde solver: A reduced-order modeling framework for partial differential equations. Journal of Computational Physics, 524:113705, 2025a. ISSN 0021-9991. doi: https://doi.org/10.1016/j.jcp.2024.113705. URL https://www.sciencedirect.com/science/article/pii/S0021999124009537.
  34. [35]Zijie Li, Anthony Zhou, and Amir Barati Farimani. Generative latent neural pde solver using flow matching, 2025b. URL https://arxiv.org/abs/2503.22600.
  35. [36]Zongyi Li, Nikola Kovachki, Kamyar Azizzadenesheli, Burigide Liu, Kaushik Bhattacharya, Andrew Stuart, and Anima Anandkumar. Fourier neural operator for parametric partial differential equations. arXiv preprint arXiv:2010.08895, 2020.arxiv.org/abs/2010.08895
  36. [37]Yaron Lipman, Ricky T. Q. Chen, Heli Ben-Hamu, Maximilian Nickel, and Matthew Le. Flow matching for generative modeling. In The Eleventh International Conference on Learning Representations, 2023. URL https://openreview.net/forum?id=PqvMRDCJT9t.arxiv.org/abs/2210.02747
  37. [38]Phillip Lippe, Bastiaan S. Veeling, Paris Perdikaris, Richard E. Turner, and Johannes Brandstetter. Pde-refiner: Achieving accurate long rollouts with neural pde solvers, 2023. URL https://arxiv.org/abs/2308.05732.
  38. [40]Emilia Magnani, Nicholas Krämer, Runa Eschenhagen, Lorenzo Rosasco, and Philipp Hennig. Approximate bayesian neural operators: Uncertainty quantification for parametric pdes. arXiv preprint arXiv:2208.01565, 2022.arxiv.org/abs/2208.01565
  39. [41]Albert Matveev, Sanmitra Ghosh, Aamal Hussain, James-Michael Leahy, and Michalis Michaelides. Light-weight diffusion multiplier and uncertainty quantification for fourier neural operators. arXiv preprint arXiv:2508.00643, 2025.arxiv.org/abs/2508.00643
  40. [42]S Chandra Mouli, Danielle C Maddix, Shima Alizadeh, Gaurav Gupta, Andrew Stuart, Michael W Mahoney, and Yuyang Wang. Using uncertainty quantification to characterize and improve out-of-domain learning for PDEs. arXiv preprint arXiv:2403.10642, 2024.arxiv.org/abs/2403.10642
  41. [43]Nikolaj T. Mücke and Benjamin Sanderse. Physics-aware generative models for turbulent fluid flows through energy-consistent stochastic interpolants, 2025. URL https://arxiv.org/abs/2504.05852.
  42. [44]Chris Pedersen, Laure Zanna, and Joan Bruna. Thermalizer: Stable autoregressive neural emulation of spatiotemporal chaos, 2025. URL https://arxiv.org/abs/2503.18731.
  43. [45]William Peebles and Saining Xie. Scalable diffusion models with transformers, 2023. URL https://arxiv.org/abs/2212.09748.
  44. [46]Ilan Price, Alvaro Sanchez-Gonzalez, Ferran Alet, Tom R. Andersson, Andrew El-Kadi, Dominic Masters, Timo Ewalds, Jacklynn Stott, Shakir Mohamed, Peter Battaglia, Remi Lam, and Matthew Le. Probabilistic weather forecasting with machine learning. Nature, 637(8044):84–90, December 2024. ISSN 1476-4687. doi: 10.1038/s41586-024-08252-9. URL http://dx.doi.org/10.1038/s41586-024-08252-9.
  45. [47]Stephan Rasp, Stephan Hoyer, Alex Merose, Ian Langmore, Peter Battaglia, Tyler Russell, Alvaro Sanchez, Vivian Yang, Rob Carver, Shreya Agrawal, Matthew Chantry, Zied Ben Bouallegue, Peter Dueben, Carla Bromberg, Jared Sisk, Luke Barrington, Aaron Bell, and Fei Sha. Weatherbench 2: A benchmark for the next generation of data-driven global weather models. arXiv, 2023.
  46. [48]Siddharth Rout, Eldad Haber, and Stephane Gaudreault. Fast and flexible probabilistic forecasting of dynamical systems using flow matching and physical perturbation, 2026. URL https://arxiv.org/abs/2508.01101.
  47. [49]Louis Serrano, Thomas X Wang, Etienne Le Naour, Jean-Noël Vittaut, and Patrick Gallinari. AROMA: Preserving spatial structure for latent PDE modeling with local neural fields. In The Thirty-eighth Annual Conference on Neural Information Processing Systems, 2024. URL https://openreview.net/forum?id=Aj8RKCGwJE.arxiv.org/abs/2406.02176
  48. [50]Aliaksandra Shysheya, Cristiana Diaconu, Federico Bergamin, Paris Perdikaris, José Miguel Hernández-Lobato, Richard E. Turner, and Emile Mathieu. On conditional diffusion models for PDE simulations. In The Thirty-eighth Annual Conference on Neural Information Processing Systems, 2024. URL https://openreview.net/forum?id=nQ18EjyMzh.arxiv.org/abs/2410.16415
  49. [51]Jiaming Song, Chenlin Meng, and Stefano Ermon. Denoising diffusion implicit models. arXiv:2010.02502, October 2020. URL https://arxiv.org/abs/2010.02502.
  50. [52]Tailin Wu, Willie Neiswanger, Hongtao Zheng, Stefano Ermon, and Jure Leskovec. Uncertainty quantification for word and inverse problems of pdes via latent global evolution. In Proceedings of the AAAI Conference on Artificial Intelligence, volume 38, pages 320–328, 2024.
  51. [53]Xiaohui Zhong, Lei Chen, Hao Li, Roberto Buizza, Jun Liu, Jie Feng, Zijian Zhu, Xu Fan, Kan Dai, Jing-jia Luo, Jie Wu, and Bo Lu. Fuxi-ens: A machine learning model for efficient and accurate ensemble weather prediction. Science Advances, 11(44), October 2025. ISSN 2375-2548. doi: 10.1126/sciadv.adu2854. URL http://dx.doi.org/10.1126/sciadv.adu2854.
  52. [54]Anthony Zhou, Alexander Wikner, Amaury Lancelin, Pedram Hassanzadeh, and Amir Barati Farimani. Reframing generative models for physical systems using stochastic interpolants, 2025. URL https://arxiv.org/abs/2509.26282.
  53. [55]Ismaël Zig​​hed, Nicolas Thome, Patrick Gallinari, and Taraneh Sayadi. Leveraging scale separation and stochastic closure for data-driven prediction of chaotic dynamics. Data-Centric Engineering, 7:e12, 2026. doi: 10.1017/dce.2026.10045.DOI

Paper details

vlnc:W_gpxrqrwv
Version
v1 · latest
Published
Fresh — added to the Hub within the last 30 days
Added
License
CC BY 4.0
Usage

views · 0

downloads · 0

A view counts once per browser per day, a download each time the PDF is requested.

  • Computer Science

Not recorded for this paper

View statement
Looking for related papers…
Contents