betaFeedback

WorksPhysics and Astronomy

Lazy Diffusion: Mitigating spectral collapse in generative diffusion-based stable autoregressive emulation of turbulent flows

Abstract

Turbulent flows posses broadband, power-law spectra in which multiscale interactions couple high-wavenumber fluctuations to large-scale dynamics. Although diffusion-based generative models offer a principled probabilistic forecasting framework, we show that standard DDPMs induce a fundamental spectral collapse: a Fourier-space analysis of the forward SDE reveals a closed-form, mode-wise signal-to-noise ratio (SNR) that decays monotonically in wavenumber, ∣k∣|k| for spectra S(k) ⁣∝ ⁣∣k∣−λS(k)\!\propto\!|k|^{-λ}, rendering high-wavenumber modes indistinguishable from noise and producing an intrinsic spectral bias. We reinterpret the noise schedule as a spectral regularizer and introduce power-law schedules β(τ) ⁣∝ ⁣τγβ(τ)\!\propto\!τ^γ that preserve fine-scale structure deeper into diffusion time, along with Lazy Diffusion, a one-step distillation method that leverages the learned score geometry to bypass long reverse-time trajectories and prevent high-kk degradation. Applied to high-Reynolds-number 2D Kolmogorov turbulence and 1/12∘1/12^\circ Gulf of Mexico ocean reanalysis, these methods resolve spectral collapse, stabilize long-horizon autoregression, and restore physically realistic inertial-range scaling. Together, they show that naïve Gaussian scheduling is structurally incompatible with power-law physics and that physics-aware diffusion processes can yield accurate, efficient, and fully probabilistic surrogates for multiscale dynamical systems.

Introduction

Accurate emulation of turbulent dynamics is essential across the natural sciences and engineering, including aerospace and ocean engineering, combustion, and atmospheric and ocean modeling. While physics-based numerical simulations face prohibitive computational costs due to the need to resolve a broad range of spatiotemporal scales, data-driven surrogates encounter a fundamentally different barrier: the difficulty of reproducing multi-scale interactions and maintaining numerical stability over long forecasting horizons. In realistic settings, initial conditions observed from turbulent systems are noisy and often partial states can only be observed. This makes precise deterministic prediction infeasible beyond a few Lyapunov time scales. This limitation motivates a shift toward probabilistic forecasting, where the objective is not to learn a point map xt↦xt+Δtx_t \mapsto x_{t+\Delta t} but instead the conditional distribution p(xt+Δt∣xt)p(x_{t+\Delta t} \mid x_t) [1, 2, 3]. Such distributional models naturally account for uncertainty arising both from imperfect or noisy initial conditions and from model inadequacies introduced by the data-driven approximation, enabling the generation of diverse yet physically plausible trajectories.

Despite these conceptual advantages, applications of generative or otherwise probabilistic models to long-term turbulent emulation remain limited. Most existing deep learning surrogates target short-term dynamics [4, 5, 6, 7], and extending them to long-range autoregressive prediction exposes structural instabilities. A central difficulty arises from spectral bias: during training, gradients preferentially fit low-wavenumber structures, systematically under-representing high-wavenumber dynamics [8]. In systems governed by power-law energy spectra, such misrepresentation of high-wavenumber dynamics is consequential. High-wavenumber errors rapidly interact with and contaminate large-scale modes during autoregressive inference, triggering a progressive out-of-distribution (OOD) drift of the predicted states. As the predicted state departs from the training manifold, surrogates produce unphysical dynamics and ultimately become unstable [9, 8, 10].

Recent hybrid approaches have attempted to address these issues. The G-LED framework [11] combines a transformer-based latent dynamical model with a diffusion model for subgrid-scale reconstruction, achieving strong performance on both the Kuramoto–Sivashinsky system and turbulent Navier–Stokes flows. The Thermalizer framework [9] applies a U-Net for full-state autoregression while using a diffusion model trained on the original data distribution to resample states when predictions drift OOD. Our prior work, the FouRKS framework [8], proposed stabilizing long-range deterministic emulation by enforcing spectral fidelity via Fourier-space losses during inference as a form of fine tuning. While effective individually, these approaches rely on composite architectures, auxiliary generative components, or explicit post-hoc correction mechanisms.

In this work, we pursue a fully generative diffusion-based [12, 13, 14] surrogate capable of stable autoregressive prediction for both high-Reynolds-number

Systems and Datasets

In this section, we introduce the two fully turbulent flow systems, System 1 (forced two-dimensional Kolmogorov turbulence) and System 2 (regional ocean surface reanalysis), from which all training, validation, and test data are derived.

System 1

We considered two systems. The first is the 2D forced Kolmogorov flow on a β\beta-plane at Re=10,000Re = 10{,}000. The governing equations are

∂ω∂t+u⋅∇ω+βv=1Re∇2ω+sin⁡(4x)+sin⁡(4y),ω=∇2ψ,u=(−∂yψ,∂xψ),\begin{aligned} \frac{\partial\omega}{\partial t} + \mathbf{u} \cdot\nabla\omega+ \beta v &= \frac{1}{Re}\nabla^{2}\omega+ \sin(4x) + \sin(4y), \\ \omega&= \nabla^{2}\psi,\qquad\mathbf{u} = (-\partial_y\psi,\partial_x\psi), \end{aligned}

where β=20\beta= 20 is the Coriolis parameter, ω\omega is the vorticity, u\mathbf{u} is the velocity vector derived from the stream function ψ\psi, and forcing has both zonal and meridional wavenumber 4. We integrate this system with a doubly periodic pseudo-spectral solver at 256×256256 \times256 resolution and ΔtDNS=5×10−3\Delta t_{DNS} = 5 \times10^{-3} until steady-state turbulence is reached. From this simulation, we extract samples of ω(t)\omega(t) at intervals of 20ΔtDNS20\Delta t_{\mathrm{DNS}}. The vorticity fields are normalized to zero mean and unit variance for training.

System 2

The second system we analyze consists of surface ocean circulation dynamics over the Gulf of Mexico. Specifically, we use zonal surface velocity (SSU), and meridional surface velocity (SSV) fields extracted from the GLORYS global ocean reanalysis dataset [18], provided at 112∘\frac{1}{12}^{\circ} horizontal resolution and daily temporal resolution. This reanalysis product combines satellite and in situ observations with the NEMO ocean circulation model [19] through a data assimilation framework. For training, we employ 7000 temporal samples of SSU and SSV spanning the period from 1993 to 2012. Model evaluation is performed autoregressively, beginning from an initial state in 2013 and propagated forward for a duration of 1000 days. These velocities are normalized to zero mean and unit variance.

Methods

Here, we detail the architectural design of our generative autoregressive models, the training protocols employed, and the proposed lazy-diffusion retraining procedure. Finally, we describe several additional engineering considerations and stabilization strategies that were essential for ensuring robust long-horizon autoregressive performance in the diffusion models. A schematic of the architecture design and other components is shown in Figure 1.

Schematic of the diffusion model and the lazy diffusion distillation

Figure 1. Schematic of the diffusion model and the lazy diffusion distillation.

Score-Based Continuous Diffusion for Conditional Forecasting

Our objective is to learn a conditional generative model for the one-step forecasting distribution

p(X(t+Δt)∣X(t)),p\left(X(t+\Delta t) \mid X(t)\right),

which can be sampled autoregressively to produce long-horizon trajectories. To achieve this, we adopt the continuous score-based diffusion framework, in which a neural network learns the score of progressively noised data and uses it to integrate a reverse-time stochastic differential equation (SDE) that generates physically plausible future states conditioned on the current state.

Forward Noising Process (Continuous-Time Limit of DDPM)

Denoising Diffusion Probabilistic Models (DDPMs) introduce noise through a discrete Markov chain:

xτ=1−βτxτ−1+βτ ϵ,ϵ∼N(0,I),x_{\tau}=\sqrt{1-\beta_{\tau}}x_{\tau-1}+\sqrt{\beta_{\tau}}\,\epsilon,\qquad\epsilon\sim\mathcal{N}(0,I),

with noise schedule {βτ}τ=1τf\{\beta_\tau\}_{\tau=1}^{\tau_f}. This recursion has the closed form

xτ=αˉτx0+1−αˉτϵ,αˉτ=∏s=1τ(1−βs),x_\tau= \sqrt{\bar{\alpha}_\tau}x_0 + \sqrt{1-\bar{\alpha}_\tau}\epsilon,\qquad\bar{\alpha}_\tau= \prod_{s=1}^{\tau}(1-\beta_s),

which clarifies that αˉτ\bar{\alpha}_\tau controls the remaining signal and 1−αˉτ1-\bar{\alpha}_\tau controls the accumulated noise.

To obtain a differentiable-time formulation suitable for continuous score-based models, we examine the infinitesimal limit. Using the Taylor expansion

1−βτ=1−12βτ+O(βτ2),\sqrt{1-\beta_\tau} = 1-\frac{1}{2}\beta_\tau+ O(\beta_\tau^2),

the one-step increment becomes

xτ−xτ−1=−12βτxτ−1+βτϵ.x_\tau-x_{\tau-1}=-\frac{1}{2}\beta_\tau x_{\tau-1}+\sqrt{\beta_\tau}\epsilon.

Letting βτ=β(τ) dτ\beta_\tau=\beta(\tau)\,d\tau and taking dτ→0d\tau\to0 yields the forward Itô SDE:

dX=f(X,τ) dτ+g(τ) dWτ,f(X,τ)=−12β(τ)X,g(τ)=β(τ).dX=f(X,\tau)\,d\tau+g(\tau)\,dW_\tau,\qquad f(X,\tau)=-\frac{1}{2}\beta(\tau)X,\qquad g(\tau)=\sqrt{\beta(\tau)}.

This SDE defines a continuous noising process that maps data at τ=0\tau=0 to an approximately Gaussian distribution at τ=τf\tau=\tau_f, providing a mathematically coherent forward diffusion formulation.

Training: Learning the Conditional Score

Score-based diffusion models do not directly learn the conditional density

p(X(t+Δt)∣X(t))p(X(t+\Delta t)\mid X(t))

itself. Instead, they learn the score of the intermediate noised distributions:

∇xlog⁡pτ(x∣X(t)).\nabla_x\log p_\tau(x\mid X(t)).

Using the forward SDE, one can sample xτx_\tau in closed form via

xτ=αˉτx0+1−αˉτϵ,ϵ∼N(0,I),x_\tau=\sqrt{\bar{\alpha}_\tau}x_0+\sqrt{1-\bar{\alpha}_\tau}\epsilon,\qquad\epsilon\sim\mathcal{N}(0,I),

where x0=X(t+Δt)x_0=X(t+\Delta t) is the target future state conditioned on the known present state X(t)X(t).

The score network sθ(x,τ,X(t))s_\theta(x,\tau,X(t)) (θ\theta are the parameters of the neural network) is trained using denoising score matching:

L(θ)=Eτ,x0,ϵ[∥sθ(xτ,τ,X(t))+ϵ1−αˉτ∥22].\mathcal{L}(\theta)=\mathbb{E}_{\tau,x_0,\epsilon}\left[\left\|s_\theta(x_\tau,\tau,X(t))+\frac{\epsilon}{\sqrt{1-\bar{\alpha}_\tau}}\right\|_2^2\right].

This loss arises from the identity

∇xlog⁡pτ(xτ∣X(t))=−ϵ1−αˉτ,\nabla_x\log p_\tau(x_\tau\mid X(t))=-\frac{\epsilon}{\sqrt{1-\bar{\alpha}_\tau}},

and therefore encourages the network to estimate the true score of the noised conditional distribution at every diffusion time.

Thus, training consists of sampling a timestep τ\tau, generating xτx_\tau, and asking the model to predict the noise responsible for the corruption of x0x_0. The conditioning variable (the present state X(t)X(t)) is concatenated or encoded in the model input so that learning is performed over the entire conditional distribution.

Sampling: Reverse-Time SDE for Conditional Generation

Once the conditional score is learned, generation proceeds by integrating the reverse-time SDE corresponding to the forward diffusion. Anderson [20] showed that for a forward SDE

dX=f(X,τ)dτ+g(τ)dWτdX = f(X,\tau)d\tau+ g(\tau)dW_\tau

the reverse process obeys

dX=[f(X,τ)−g(τ)2∇xlog⁡pτ(x∣X(t))]dτ+g(τ)dW‾τdX = \left[f(X,\tau) - g(\tau)^2\nabla_x \log p_\tau(x \mid X(t))\right]d\tau+ g(\tau)d\overline{W}_\tau

where W‾τ\overline{W}_\tau is a reverse-time Wiener process.

Replacing the true score with its neural approximation sθs_\theta yields the practical sampling equation:

dX=[f(X,τ)−g(τ)2sθ(X,τ,X(t))]dτ+g(τ)dW‾τdX = \left[f(X,\tau) - g(\tau)^2s_\theta(X,\tau,X(t))\right]d\tau+ g(\tau)d\overline{W}_\tau

Integrating this SDE from τ=τf\tau=\tau_f back to τ=0\tau=0 transforms Gaussian noise into a sample from the learned conditional distribution:

X(t+Δt)∼pθ(⋅∣X(t)).X(t+\Delta t) \sim p_\theta(\cdot\mid X(t)).

Repeating this process autoregressively produces a full model rollout:

Xn+1=X^(tn+1)∼pθ(⋅∣Xn),X_{n+1} = \hat{X}(t_{n+1}) \sim p_\theta(\cdot\mid X_n),

enabling long-horizon simulation of System 1 and System 2.

Power law noise schedulers

As will be demonstrated in detail in § 4, the standard DDPM forward process when paired with a white-noise Gaussian corruption kernel, induces a rapid, wavenumber-dependent degradation of the signal-to-noise ratio, with high-wavenumber modes collapsing exponentially earlier than their low-wavenumber counterparts. Although more sophisticated kernels or adaptive scheduling mechanisms could potentially alleviate this effect, we focus on a simple and effective modification that is compatible with the continuous-time diffusion framework: a power-law noise schedule,

β(τ)=βmin⁡+(βmax⁡−βmin⁡)τγ.\beta(\tau) = \beta_{\min} + (\beta_{\max} - \beta_{\min})\tau^\gamma.

Increasing the exponent γ\gamma concentrates the noise injection near the end of the forward diffusion trajectory. For example, increasing γ\gamma from 1 to 5 shifts three-quarters of the total noise addition into the final ∼20%\sim20\% of diffusion time, thereby delaying corruption of high-wavenumber components. This preserves fine-scale structure deeper into the forward process and provides the reverse-time sampler with more effective refinement capacity for the highest resolved modes.

To assess the influence of this scheduling parameter, we conduct a sweep over γ∈[1.0,7.0]\gamma\in[1.0, 7.0] and evaluate both the baseline score models and their lazily re-trained variants. As shown later in the Results section, power-law scheduling plays a critical role in mitigating spectral bias, improving high-wavenumber fidelity, and enhancing long-horizon stability in autoregressive emulation for both dynamical systems studied.

Lazy Diffusion: Distilling the Score Model Into a Single-Step Conditional Predictor

A key advantage of the score-based diffusion framework is that the network is trained on samples drawn from the entire forward diffusion trajectory. Thus, it learns the structure of the conditional data distribution at multiple corruption levels, enabling robust denoising across a wide range of noise intensities. For conditional forecasting, however, the eventual sampling objective is considerably simpler: given the present state X(t)X(t), the model must generate a single draw from the conditional distribution

p(X(t+Δt)∣X(t)).p(X(t + \Delta t) \mid X(t)).

This raises the question of whether multi-step numerical integration of the reverse-time SDE is necessary, or whether this mapping can be achieved directly.

Limitations of Direct One-Step Denoisers

Training a neural network to map a corrupted sample xτ∗x_{\tau}^{*} directly to the clean sample x0x_{0} in one step is generally unstable. Without guidance from the diffusion process, the network must implicitly infer the geometry of the conditional data manifold and simultaneously remove noise and reconstruct fine-scale structure. In systems exhibiting broadband spectral content, such as turbulent flows and ocean circulation, this is a highly ill-posed regression problem prone to over-smoothing, spectral collapse, or mode failure.

Insight From the Conditional Score

The continuous-time diffusion formulation provides the structural information necessary to overcome these limitations. The pretrained score network

sθ(x,τ,X(t))≈∇xlog⁡pτ(x∣X(t))s_{\theta}(x, \tau, X(t)) \approx\nabla_{x} \log p_{\tau}(x \mid X(t))

encodes the gradient of the log-density of the conditional distribution at each diffusion time τ\tau. This gradient characterizes the local geometry of the manifold of likely states (see § 7.2). For many diffusion formulations (e.g., variance-exploding SDEs), this score field directs corrupted samples toward regions of higher probability density, thereby providing a principled direction of denoising at each noise level.

Crucially, this means that the pretrained score network already contains information describing how to transition from moderately corrupted samples toward the clean data distribution. The task of producing a one-step estimator can therefore be formulated not as learning this structure from scratch, but as distilling it into a direct mapping.

Lazy Re-Training: A Distillation Procedure

We propose lazy diffusion, a second-stage training procedure that refines the pretrained score model into a single-step conditional predictor. Let τ∗\tau^* denote an intermediate noise level at which the forward process yields a moderately corrupted sample. Using the closed-form forward diffusion process,

xτ∗=αˉτ∗x0+1−αˉτ∗ ϵ,ϵ∼N(0,I),x_{\tau^*} = \sqrt{\bar{\alpha}_{\tau^*}}x_0 + \sqrt{1-\bar{\alpha}_{\tau^*}}\,\epsilon,\qquad\epsilon\sim\mathcal{N}(0,I),

we construct supervised training pairs (xτ∗,x0)(x_{\tau^*},x_0).

A new neural network FϕF_\phi is initialized from the pretrained score model parameters and optimized to approximate the clean state via

Llazy(ϕ)=Ex0,ϵ[∥Fϕ(xτ∗,X(t))−x0∥2].\mathcal{L}_{\mathrm{lazy}}(\phi) = \mathbb{E}_{x_0,\epsilon}\left[\left\|F_\phi(x_{\tau^*},X(t))-x_0\right\|^2\right].

This distillation process does not require the model to infer the full reverse-time denoising trajectory. Instead, it leverages the fact that the score model already represents the local log-density geometry of the conditional distribution at τ∗\tau^*, allowing the distilled model to focus solely on aggregating this information into a single-step reconstruction of x0x_0.

Sampling With Lazy Diffusion

After lazy re-training, sampling from the conditional distribution reduces to two steps. First, we generate a corrupted sample at the chosen noise level τ∗\tau^*:

xτ∗=αˉτ∗x0+1−αˉτ∗ ϵ,ϵ∼N(0,I).x_{\tau^*} = \sqrt{\bar{\alpha}_{\tau^*}}x_0 + \sqrt{1-\bar{\alpha}_{\tau^*}}\,\epsilon,\qquad\epsilon\sim\mathcal{N}(0,I).

Second, we apply the distilled predictor to obtain the model’s estimate of the future state:

X^(t+Δt)=Fϕ(xτ∗,X(t)).\widehat{X}(t+\Delta t) = F_\phi(x_{\tau^*},X(t)).

This produces a single-step approximation to the conditional forecast distribution without requiring numerical integration of the reverse SDE. In practice, lazy diffusion yields substantial computational savings while maintaining high-wavenumber fidelity and long-horizon stability in both System 1 and System 2.

Addressing Distribution Shift

A fundamental challenge in autoregressive emulation arises from the distribution shift that inevitably develops once model predictions are recursively fed back as inputs. During training, the model learns the conditional density

p(X(t+Δt)∣X(t)),p(X(t + \Delta t) \mid X(t)),

where the conditioning state is always drawn from the true data manifold. During inference, however, the conditioning state rapidly departs from this manifold. Due to spectral bias in the learned dynamics, the earliest and largest errors occur at high wavenumbers; these errors then propagate into the conditioning input at the next timestep:

p(X(t+Δt)∣X(t)+ϵ(t)),p(X(t + \Delta t) \mid X(t) + \epsilon(t)),

where ϵ(t)\epsilon(t) denotes accumulated autoregressive prediction error. This mismatch places the model outside the distribution encountered during training, leading to accelerated error growth and eventual dynamical instability.

To mitigate this phenomenon, we introduce controlled perturbations to the conditioning states during training, chosen to match the magnitude and variability of prediction errors expected during autoregressive rollout. Consider a model prediction

X^(t+Δt)=fθ(X(t))\widehat{X}(t + \Delta t) = f_{\theta}(X(t))

with instantaneous error

ϵ(t)=X^(t+Δt)−X(t+Δt).\epsilon(t) = \widehat{X}(t + \Delta t) - X(t + \Delta t).

If accumulated errors are modeled as isotropic Gaussian perturbations,

ϵ∼N(0,σϵ2I),\epsilon\sim\mathcal{N}(0, \sigma_{\epsilon}^{2} I),

then the mean squared error satisfies

MSE=E[∥X^−X∥2]=σϵ2.\mathrm{MSE} = \mathbb{E}\left[\left\lVert\widehat{X} - X \right\rVert^{2}\right] = \sigma_{\epsilon}^{2}.

A naive stabilization strategy would therefore set the conditioning noise standard deviation to ση=MSE\sigma_{\eta} = \sqrt{\mathrm{MSE}}. However, matching only the expected error amplitude is insufficient, since autoregressive degradation—particularly that produced by high-wavenumber attenuation—varies significantly across time and spatial scales.

Instead, we adopt a coverage-based criterion: conditioning noise should be chosen such that approximately half of the training perturbations lie within the expected error magnitude, and half exceed it, thereby improving robustness to out-of-distribution inputs. Formally, we require

P(∥η∥≤σϵ)≈0.5,η∼N(0,ση2I).P\left(\left\lVert\eta\right\rVert\leq\sigma_{\epsilon}\right) \approx0.5, \qquad\eta\sim\mathcal{N}(0, \sigma_{\eta}^{2} I).

For one-dimensional marginals,

P(∣ηi∣≤σϵ)=2Φ(σϵση)−1P(\lvert\eta_i\rvert\le\sigma_\epsilon) = 2\Phi\left(\frac{\sigma_\epsilon}{\sigma_\eta}\right) - 1

and setting this probability to 0.5 yields

Φ(σϵση)=0.75⟹σϵση=Φ−1(0.75)≈0.674,\Phi\left(\frac{\sigma_\epsilon}{\sigma_\eta}\right) = 0.75 \quad\Longrightarrow\quad\frac{\sigma_\epsilon}{\sigma_\eta} = \Phi^{-1}(0.75) \approx0.674,

so that

ση≈1.48 σϵ≈1.5MSE.\sigma_\eta\approx1.48\,\sigma_\epsilon\approx1.5\sqrt{\mathrm{MSE}}.

This inflation factor ensures that conditioning samples span both the in-distribution regime (perturbations within expected predictive error) and a controlled degree of out-of-distribution deviation (perturbations exceeding the error scale), which is essential for stabilizing autoregressive trajectories.

During training, the perturbed conditioning input is thus constructed as

Xtcond=X(t)+η,η∼N(0,ση2I),X_t^{\mathrm{cond}} = X(t) + \eta,\qquad\eta\sim\mathcal{N}(0,\sigma_\eta^2 I),

with ση\sigma_\eta determined from the coverage criterion above. This approach directly targets the distribution shift induced by spectral bias, improves robustness of the conditional model to off-manifold inputs, and significantly enhances long-horizon stability during autoregressive emulation.

Network Architecture for the Conditional Score Model

The conditional score network used throughout this study is based on a U-Net backbone augmented with sinusoidal diffusion-time embeddings and linear-time self-attention modules. This architecture is chosen for its ability to capture both local multi-scale structure and long-range spatial correlations, which are essential for accurately representing turbulent and oceanic dynamical fields.

General Structure. The network follows an encoder–decoder U-Net design with symmetric skip connections. Each resolution level contains two residual blocks, each consisting of a convolutional layer with Group Normalization and SiLU activation. Time information is injected at every block through a sinusoidal positional encoding passed through a two-layer MLP with a 4×4\times expansion of the hidden dimension. Self-attention layers are inserted at selected spatial resolutions to facilitate global information exchange without incurring prohibitive computational cost. The final projection head consists of

GroupNorm→SiLU→Conv1×1→SiLU→Conv1×1,\mathrm{GroupNorm} \rightarrow\mathrm{SiLU} \rightarrow\mathrm{Conv}_{1\times1} \rightarrow\mathrm{SiLU} \rightarrow\mathrm{Conv}_{1\times1},

mapping the latent representation to the appropriate number of score channels.

System 1: Turbulent Flow. For the two-dimensional Kolmogorov turbulence dataset, we employ five resolution levels with channel multipliers [2,4,4,8,16][2, 4, 4, 8, 16], applied to a base of 16 channels. This results in channel widths

[16,32,64,64,128,256][16, 32, 64, 64, 128, 256]

across successive downsampling stages. Self-attention is applied at the 32×3232 \times32, 16×1616 \times16, and 8×88 \times8 resolutions, which correspond to the scales most relevant for capturing the nonlinear energy cascade and the long-range coupling characteristic of turbulent flows. Each attention layer uses 32 heads with 64-dimensional head representations. Residual blocks employ Group Normalization with the number of groups

min⁡(channels4,32),\min\left(\frac{\text{channels}}{4}, 32\right),

ensuring stable training across varying feature-map widths.

System 2: Ocean Reanalysis. The surface ocean circulation fields are defined on a non-square grid of 192×320192 \times320 with six input channels (two channels for predicted increments and four channels for two-timestep conditioning). To accommodate this geometry and maintain computational tractability, we use four resolution levels with channel multipliers [2,4,4,8][2, 4, 4, 8] applied to a 32-channel base width. Self-attention is introduced only at the 24×1224 \times12 and 12×612 \times6 resolutions, where the spatial dimensionality is sufficiently reduced to allow efficient global mixing. The model predicts zonal and meridional surface velocities (U,V)(U, V), and a land–sea mask is applied during loss computation to exclude land points, ensuring that learning is focused exclusively on dynamically active ocean regions.

This architecture provides a unified backbone capable of representing the multiscale, anisotropic structure of both turbulent flows and ocean circulation. The combination of hierarchical convolutional processing, scale-selective self-attention, and diffusion-time modulation enables the score model to capture fine-scale statistical structure while maintaining stability across the full range of noise levels used during training.

Training and Evaluation Protocol

System 1: Models for the turbulent flow dataset are trained using a batch size of 2 with the AdamW optimizer and a cosine learning-rate schedule decaying from 10−410^{-4} to 10−510^{-5} over 20,000 optimization steps. Gradient norms are clipped at 0.25 to ensure numerical stability when learning high-wavenumber structure. Conditioning noise is introduced with standard deviation ση=0.01\sigma_\eta= 0.01, selected based on the single-step prediction error estimated from an initial pilot model. At each training iteration, four diffusion times are sampled independently from a uniform distribution,

τ∼U(0.001,1.0),\tau\sim\mathcal{U}(0.001, 1.0),

which encourages learning of the score function across the full range of noise levels.

System 2: For the ocean reanalysis dataset, we train using the 1/12∘1/12^\circ GLORYS fields defined on a 192×320192 \times320 grid over the Gulf of Mexico, using data from 1993–2012. The model conditions on two preceding timesteps and predicts the increments (ΔU,ΔV)(\Delta U, \Delta V) for the zonal and meridional surface velocities. Training uses a batch size of 1 with the AdamW optimizer and cosine decay of the learning rate from 10−410^{-4} to 10−610^{-6} over 200,000 steps. Gradient clipping is applied at a threshold of 0.5. Conditioning noise is set to ση=0.001\sigma_\eta= 0.001, derived from the error magnitude of the first autoregressive prediction step. Two diffusion times are sampled per iteration from

τ∼U(0.001,1.0),\tau\sim U(0.001, 1.0),

to span the full diffusion-time domain.

Lazy Diffusion Retraining. Lazy diffusion distillation is performed by fixing a single intermediate noise level, τ∗=0.5\tau^* = 0.5, and re-training the model for 20,000 additional iterations using the root-mean-squared error between the predicted and clean fields as the sole loss term. No auxiliary regularization or score-matching terms are employed. All other hyperparameters—including optimizer settings, gradient clipping, and conditioning noise—are inherited from the corresponding base score model. This stage converts the multi-step denoising process into a single-step conditional predictor while preserving high-wavenumber fidelity.

Inference and Evaluation. For score-based models, inference uses Euler–Maruyama integration of the reverse-time SDE with 1000 discretization steps, unless otherwise specified. Lazy diffusion models perform prediction in a single network evaluation. Autoregressive evaluation consists of 2000-step rollouts for System 1 and 1000-step rollouts for System 2. For Kolmogorov turbulence, spectral error is computed across all resolved wavenumbers kk; for the ocean dataset, errors are evaluated across the corresponding zonal and meridional wavenumber bands. These long-horizon rollouts allow assessment of stability, spectral fidelity, and preservation of physically relevant statistics.

Results

In this section, we first discuss the theoretical analysis of spectral collapse in diffusion models trained on turbulent energy spectra. We then provide a mitigative solution and implement it at scale for diffusion-based autoregressive emulation of Systems 1 and 2.

Theoretical spectral analysis of forward diffusion for turbulent energy spectra

A central property of both dynamical systems considered in this study, forced two-dimensional turbulence and regional ocean surface dynamics, is the presence of scale invariance within their inertial ranges. Empirically, this manifests as an isotropic power-law decay of the power spectral density (PSD),

S0(k)∝∣k∣−λ,λ>0,S_0(k) \propto\lvert k\rvert^{-\lambda}, \qquad\lambda> 0,

for wavenumbers kk lying in the inertial subrange. This power-law structure has significant implications for the behavior of diffusion-based generative models when used for autoregressive emulation. In particular, the standard forward noising process in Denoising Diffusion Probabilistic Models (DDPMs) attenuates high-wavenumber dynamics more aggressively than low-wavenumber dynamics, leading to a wavenumber-dependent collapse of the signal-to-noise ratio (SNR), which we call spectral collapse in this paper. We now derive this phenomenon in detail.

Let X0∈RdX_0 \in\mathbb{R}^d denote an initial physical state (e.g., a vorticity field), and consider the standard DDPM forward diffusion dynamics:

Xτ=αˉτX0+1−αˉτ ε,ε∼N(0,σ2Id),(1)X_\tau= \sqrt{\bar{\alpha}_\tau}X_0 + \sqrt{1-\bar{\alpha}_\tau}\,\varepsilon,\qquad\varepsilon\sim\mathcal{N}(0,\sigma^2 I_d), \tag*{(1)}

where

αˉτ=∏s=1ταs,0<αs<1.\bar{\alpha}_\tau= \prod_{s=1}^{\tau} \alpha_s,\qquad0 < \alpha_s < 1.

This representation follows from the closed-form reparameterization of the multi-step Gaussian noising [21].

The scalar SNR at diffusion time τ\tau is defined as

SNR⁡(τ)=E[∥αˉτX0∥2]E[∥1−αˉτ ε∥2]=αˉτ(1−αˉτ)σ2,(2)\operatorname{SNR}(\tau) = \frac{\mathbb{E}\left[\left\|\sqrt{\bar{\alpha}_\tau}X_0\right\|^2\right]}{\mathbb{E}\left[\left\|\sqrt{1-\bar{\alpha}_\tau}\,\varepsilon\right\|^2\right]} = \frac{\bar{\alpha}_\tau}{(1-\bar{\alpha}_\tau)\sigma^2}, \tag*{(2)}

where we used E∥X0∥2=E∥ε∥2=d\mathbb{E}\|X_0\|^2 = \mathbb{E}\|\varepsilon\|^2 = d under unit variance normalization.

Let X^(k)=F[X](k)\widehat{X}(k) = \mathcal{F}[X](k) denote the unitary discrete Fourier transform. Because the forward process, given by Eq. (1) is linear and convolution-free (additive white noise), it acts independently on each Fourier coefficient. Applying F\mathcal{F} to Eq. (1) gives:

X^τ(k)=αˉτ X^0(k)+1−αˉτ ε^(k),(3)\widehat{X}_\tau(k) = \sqrt{\bar{\alpha}_\tau}\,\widehat{X}_0(k) + \sqrt{1-\bar{\alpha}_\tau}\,\widehat{\varepsilon}(k), \tag*{(3)}

where independence and zero-mean of X0X_0 and ε\varepsilon are mode-wise preserved.

Assume that the initial field has an isotropic power-law energy spectrum,

S0(k):=E[∣X^0(k)∣2]=A∣k∣−λ,A>0, λ>0,(4)S_0(k) := \mathbb{E}\left[\left|\widehat{X}_0(k)\right|^2\right] = A|k|^{-\lambda},\qquad A>0,\ \lambda>0, \tag*{(4)}

consistent with turbulent inertial-range scaling. Moreover, since ε\varepsilon is white Gaussian noise,

E[∣ε^(k)∣2]=σ2for all k.\mathbb{E}\left[\left|\widehat{\varepsilon}(k)\right|^2\right] = \sigma^2 \qquad\text{for all } k.

Taking the expectation of the squared magnitude of Eq. (3), and using the independence of X^0(k)\widehat{X}_0(k) and ε^(k)\widehat{\varepsilon}(k), yields:

Sτ(k):=E[∣X^τ(k)∣2](5)S_\tau(k) := \mathbb{E}\left[\left|\widehat{X}_\tau(k)\right|^2\right] \tag*{(5)}
=αˉτE[∣X^0(k)∣2]+(1−αˉτ)E[∣ε^(k)∣2](6)= \bar{\alpha}_\tau\mathbb{E}\left[\left|\widehat{X}_0(k)\right|^2\right] + (1-\bar{\alpha}_\tau)\mathbb{E}\left[\left|\widehat{\varepsilon}(k)\right|^2\right] \tag*{(6)}
=αˉτA∣k∣−λ+(1−αˉτ)σ2.(7)= \bar{\alpha}_\tau A|k|^{-\lambda} + (1-\bar{\alpha}_\tau)\sigma^2. \tag*{(7)}

The cross-term vanishes because each component has zero mean and the pair is independent.

Equation (7) shows explicitly that the noisy spectrum at time τ\tau is a convex combination of a power-law signal and flat (white) noise.

We now compute the spectral SNR, which quantifies the relative strength of the physical signal and injected noise at each wavenumber:

SNR⁡(k,τ):=αˉτA∣k∣−λ(1−αˉτ)σ2.(8)\operatorname{SNR}(k,\tau) := \frac{\bar{\alpha}_{\tau} A |k|^{-\lambda}}{(1-\bar{\alpha}_{\tau})\sigma^{2}}. \tag*{(8)}

For fixed τ\tau, this SNR decays as a power law in ∣k∣|k|. Differentiating Eq. (8) with respect to ∣k∣|k| gives

∂∂∣k∣SNR⁡(k,τ)=αˉτA(1−αˉτ)σ2∂∂∣k∣∣k∣−λ(9)\frac{\partial}{\partial|k|}\operatorname{SNR}(k,\tau) = \frac{\bar{\alpha}_{\tau} A}{(1-\bar{\alpha}_{\tau})\sigma^{2}} \frac{\partial}{\partial|k|}|k|^{-\lambda} \tag*{(9)}
=−αˉτAλ(1−αˉτ)σ2∣k∣−λ−1.(10)= -\frac{\bar{\alpha}_{\tau} A\lambda}{(1-\bar{\alpha}_{\tau})\sigma^{2}} |k|^{-\lambda-1}. \tag*{(10)}

Since all constants are positive and ∣k∣−λ−1>0|k|^{-\lambda-1} > 0, we obtain the strict inequality

∂∂∣k∣SNR⁡(k,τ)<0,∀k≠0.\frac{\partial}{\partial|k|}\operatorname{SNR}(k,\tau) < 0,\qquad\forall k \ne0.

Equation (8) combined with Eq. (10) reveals a key limitation of standard diffusion noising: the spectral SNR decays polynomially in ∣k∣|k|, with exponent λ>0\lambda> 0 determined by the physics of the underlying turbulent cascade. Thus, high-wavenumber dynamics (large ∣k∣|k|) lose signal power far more rapidly than low-wavenumber components as τ\tau increases. This is shown in both Figure 2 (a) and (b), where the lines corresponding to γ=1\gamma= 1 (standard DDPM with linear noise schedule) for the high-wavenumber component of the signal decay far more quickly than those in mid wavenumbers or low wavenumbers. As discussed in our proposed power law noise schedulers in § 3.2.2, increasing the value of γ\gamma improves the preservation of the signal in the high wavenumbers late into the forward diffusion process (discussed further in § 4.2).

Signal to Noise Ratio for low, medium, and high k frequency bands

Figure 2. Signal to Noise Ratio for low, medium, and high kk frequency bands for a power law adopting signal using different power values γ\gamma. SNR =0= 0 represents the white noise floor of the DDPM process.

Once SNR⁡(k,τ)\operatorname{SNR}(k,\tau) drops below the learnability threshold of the score model (parameterized by a U-Net in our case), the corresponding Fourier mode becomes effectively indistinguishable from noise. This occurs at much earlier diffusion times for high-wavenumber modes, causing the diffusion model to learn an increasingly biased representation of the data: large-scale structures remain visible throughout the diffusion horizon, while inertial-range and dissipation-range features are overwhelmed by noise.

This stratified decay demonstrates that the forward diffusion process erases information in a wavenumber-dependent manner: large-scale modes survive nearly intact, inertial-range modes degrade gradually, and dissipation-range modes vanish almost immediately. Because the score network is trained to denoise samples across all diffusion times, it receives virtually no usable supervision for high-kk modes at moderate or large τ\tau. As a result, the learned score field is intrinsically biased toward low-wavenumber reconstruction. Figure 2(a) makes this mechanism explicit: the diffusion process itself imposes a spectral preference that inevitably leads to high-wavenumber underfitting and long-term instability in autoregressive emulation.

This phenomenon is the root of the spectral bias in diffusion models when applied to turbulent systems with power-law spectra. It directly affects the model’s ability to reconstruct fine-scale corrections during sampling and consequently limits the fidelity and stability of long-horizon autoregressive emulation.

Biased loss weighting at the high wavenumbers

The wavenumber-stratified collapse of SNR⁡(k,τ)\operatorname{SNR}(k,\tau) shown in Figure 2(a) and (b) reveals that, under a standard DDPM noise schedule, the forward diffusion process progressively eliminates learnable information in a manner that is fundamentally incompatible with the physics of turbulent flows. In particular, while low-wavenumber (large-scale) modes retain coherent signal until the final diffusion timestep, mid-kk inertial-range modes lose usable SNR halfway through the process, and high-kk modes collapse almost immediately. This implies that, for a large portion of the diffusion horizon, all high-wavenumber Fourier components that reach the score network during training are effectively indistinguishable from white noise.

This observation has a direct and profound consequence on the DDPM training objective. Recall that the score network is trained via the denoising objective

L(θ)=Eτ,X0,ε[∥sθ(Xτ,τ)+ε1−αˉτ∥2].\mathcal{L}(\theta)=\mathbb{E}_{\tau,X_0,\varepsilon}\left[\left\|s_\theta(X_\tau,\tau)+\frac{\varepsilon}{\sqrt{1-\bar{\alpha}_\tau}}\right\|^2\right].

Because XτX_\tau is distributed according to the forward process Xτ=αˉτX0+1−αˉτεX_\tau=\sqrt{\bar{\alpha}_\tau}X_0+\sqrt{1-\bar{\alpha}_\tau}\varepsilon, the target that the network must predict is precisely the noise field ε\varepsilon rescaled by (1−αˉτ)−1/2(1-\bar{\alpha}_\tau)^{-1/2}. At diffusion times where the SNR for a given wavenumber shell has collapsed to the noise floor (as is the case for high-kk modes at very small τ\tau), the score model is asked to fit a target that contains pure noise in that shell. Thus, the score-matching loss forces the model to devote capacity to predicting an irreducible Gaussian random variable at high kk, even though those modes in the true data distribution contain meaningful turbulent structure at τ=0\tau=0.

To see this relationship explicitly, we analyze the loss in spectral space. Parseval’s theorem allows the pixel-space objective to be rewritten as an integral over the Fourier domain:

L^(θ)∝Eτ,X0,ε[∫R2∣s^θ(k,τ)+ε^(k)1−αˉτ∣2dk].\widehat{\mathcal{L}}(\theta)\propto\mathbb{E}_{\tau,X_0,\varepsilon}\left[\int_{\mathbb{R}^2}\left|\widehat{s}_\theta(\mathbf{k},\tau)+\frac{\widehat{\varepsilon}(\mathbf{k})}{\sqrt{1-\bar{\alpha}_\tau}}\right|^2\mathrm{d}\mathbf{k}\right].

In two spatial dimensions, the wavenumber measure becomes dk=∣k∣ d∣k∣ dϕ\mathrm{d}k = |k|\,\mathrm{d}|k|\,\mathrm{d}\phi, and for isotropic turbulence, the angular dependence integrates out. The loss reduces to

L^(θ)∝Eτ,X0,ϵ[∫0kmax⁡∣S^θ(∣k∣,τ)+ϵ^(∣k∣)1−αˉτ∣2∣k∣ d∣k∣]\widehat{\mathcal{L}}(\theta) \propto\mathbb{E}_{\tau,X_0,\epsilon}\left[\int_{0}^{k_{\max}}\left|\widehat{S}_{\theta}(|k|,\tau)+\frac{\widehat{\epsilon}(|k|)}{\sqrt{1-\bar{\alpha}_{\tau}}}\right|^2 |k|\,\mathrm{d}|k|\right]

The key geometric factor ∣k∣|k| arises because the number of Fourier modes in a wavenumber shell [k,k+dk][k,k+\mathrm{d}k] grows linearly with kk. As a result, the contribution of each shell to the loss scales as the shell area in Fourier space.

Combining this observation with the turbulent power spectrum

S0(k)∝∣k∣−λ,λ>1,S_0(k) \propto|k|^{-\lambda}, \qquad\lambda> 1,

reveals a deep structural misalignment between the DDPM objective and the physics of turbulence. The integrand for low-wavenumber shells decays as ∣k∣1−λ|k|^{1-\lambda}, which is integrable and small for realistic turbulent spectra. In contrast, the high-wavenumber contribution grows proportionally to ∣k∣|k|, since the noise spectrum is flat and the shell geometry dominates. Thus, the spectral form of the loss disproportionately weights precisely the Fourier bands that the forward process has already corrupted into white noise (high kk), while assigning extremely small weight to the dynamically important, energy-containing large scales (low kk).

The consequence is that the network effectively expends the majority of its representational capacity attempting to fit irreducible Gaussian noise in the high-wavenumber shells, rather than reconstructing the physically meaningful turbulent structures carried by the low-kk and mid-kk modes. This is the spectral analogue of the SNR collapse illustrated in Figure 2(a) and (b): the forward process with a linear noise schedule destroys high-wavenumber signal long before the network ever sees it, and the DDPM loss then forces the network to waste capacity on predicting those already-destroyed modes.

This wavenumber-dependent imbalance directly motivates the two mitigation strategies we develop in this work. First, we introduce power schedules, which delay the injection of noise until very late in the diffusion trajectory. By keeping αˉτ\bar{\alpha}_{\tau} near unity for the majority of diffusion time, the SNR for all wavenumber bands, including the high-kk shells, remains above the noise floor for a significantly longer duration as shown in both Figure 2(a) and (b) for power schedules, γ≫1\gamma\gg1. This preserves high-wavenumber signal deeper into the forward process and provides the score network with meaningful supervision for those modes.

Second, we develop a lazy diffusion distillation (see § 3.3) procedure that bypasses most of the forward diffusion entirely. By exploiting the unimodal structure of the conditional distributions encountered in autoregressive prediction, we train a model to perform a single-shot denoising step. This approach sidesteps the early high-kk SNR collapse by eliminating the long diffusion trajectory in which high-wavenumber information is destroyed. Together, these two mitigation strategies directly counteract the structural biases introduced by the DDPM forward process and restore meaningful high-wavenumber learning essential for accurate turbulent emulation.

Autoregressive emulation with power-law schedules and lazy diffusion

Using the theoretical insights derived in § 4.1 and § 4.2, we now evaluate three autoregressive emulation strategies at scale using data from Systems 1 and 2: (i) a standard DDPM with a linear noise schedule, (ii) a continuous score-based diffusion model using power-law noise schedules with exponents γ∈[2.5,7.0]\gamma\in[2.5, 7.0], and (iii) our proposed lazy diffusion, a distilled single-step score model designed to bypass the instability of long diffusion trajectories.

In System 1, Figure 3 shows that although all models generate superficially plausible turbulence at early times, their long-term behavior diverges sharply. The ground truth maintains four coherent jets aligned with the meridional and zonal forcing at wavenumber 4, a hallmark feature of statistically stationary Kolmogorov flow. This jet structure is preserved only by the γ=5.0\gamma= 5.0 model across thousands of autoregressive steps. The linear-schedule DDPM instead exhibits periodic jet collapse and reformation, consistent with the theoretical prediction that early SNR collapse eliminates high-wavenumber structure and disrupts the energy balance between forcing and dissipation. The lazy diffusion model, despite using only a single denoising step, retains substantially more flow coherence than DDPM, reflecting the benefits of bypassing the high-kk signal destruction inherent to long forward-diffusion trajectories.

Autoregressive evolution of vorticity fields in System 1

Figure 3. Autoregressive evolution of vorticity fields in System 1. Snapshots shown at t={1,100,500,1000,2000}t = \{1, 100, 500, 1000, 2000\} timesteps (t=20ΔtDNSt = 20\Delta t_{\mathrm{DNS}}) for the ground truth, linear-schedule DDPM, the power-schedule model with γ=5.0\gamma= 5.0, and its lazy-diffusion variant.

Standard accuracy metrics obscure these differences. As shown in Figure 4, relative RMSE remains similar across models during the 2000-step rollout: DDPM achieves 1.366, the γ=5.0\gamma= 5.0 model 1.292, and lazy diffusion 1.274. This highlights a key limitation: RMSE penalizes magnitude errors but is blind to structural drift and spectral mode misallocation.

Image not extracted

Figure 4. Quantitative evaluation metrics over 2000-timestep autoregressive rollouts for linear (γ=1\gamma= 1) DDPM, γ=5.0\gamma= 5.0 power schedule, and Lazy retraining (γ=5.0\gamma= 5.0). Row 1 shows Relative RMSE at each timestep. Row 2 Shows the total histogram of of vorticity values over all autoregression times. Row 3 shows the latitudinal spectrum averaged over the rollout. Row 4 shows the turbulent kinetic energy (TKE) spectra averaged over the whole rollout.

More physically meaningful metrics expose the differences. The vorticity distributions show that γ=5.0\gamma= 5.0 closely matches the ground truth, while DDPM displays excess probability mass at extreme vorticity values and a rightward shift, corresponding to unphysical coherent structures. Quantitatively, the Earth Mover’s Distance increases from 0.142 for γ=5.0\gamma= 5.0 to 0.342 for DDPM, while KL divergence increases from 0.023 to 0.075.

Spectral analysis pinpoints the mechanism behind these failures: as seen in the latitudinal spectra of Figure 4, DDPM severely overestimates power for wavenumbers k>10k > 10, with deviations exceeding an order of magnitude at high kk. In contrast, the γ=5.0\gamma= 5.0 model reproduces spectral slopes and amplitudes accurately up to the highest resolved scales. The relative spectral error for γ=5.0\gamma= 5.0 is 10.2%, compared to 96.3% for DDPM. Lazy diffusion exhibits mild degradation (22.7%) but compensates with three orders of magnitude lower inference cost.

These findings are systematically reinforced by the power-schedule sweep in Figure 5. Increasing γ\gamma from the linear schedule (γ=1\gamma= 1) progressively improves spectral fidelity, peaking at γ=5.0\gamma= 5.0. The improvement stems from delayed noise injection: the linear schedule reaches α(τ)=0.5\alpha(\tau) = 0.5 at τ≈0.3\tau\approx0.3, whereas γ=5.0\gamma= 5.0 delays this transition until τ≈0.75\tau\approx0.75, preserving high-wavenumber information for a much larger portion of the diffusion trajectory and providing significantly more reverse-diffusion steps for fine-scale refinement.

Power schedule analysis and spectral performance

Figure 5. Power schedule analysis and spectral performance. (a) Relative spectral error as a function of power exponent γ\gamma, showing optimal performance at γ=5.0\gamma= 5.0 and catastrophic failure above γ=6.0\gamma= 6.0. Orange triangles indicate lazy diffusion retraining. (b) Signal-to-noise ratio evolution, α(τ)\alpha(\tau), for different power schedules, with higher γ\gamma delaying noise injection. (c) Time-averaged latitudinal spectra of vorticity. (d) TKE spectra for all models demonstrate that appropriate power scheduling recovers high-wavenumber dynamics.

For γ>5.5\gamma> 5.5, a sharp training instability emerges. At γ=6.0\gamma= 6.0 and γ=7.0\gamma= 7.0, spectral error jumps by two orders of magnitude to 1470% and 1476%, respectively. These failures occur because excessively aggressive schedules push nearly all noise injection to the terminal portion of the forward process, where the discrete DDPM formulation cannot resolve the resulting large noise jumps. Training becomes numerically unstable without impractically small timestep discretization.

Lazy diffusion improves the performance of most base models, regardless of γ\gamma, as indicated by the orange triangles in Figure 5(a). However, the best lazy variant arises from the best base model: γ=5.0\gamma= 5.0. Lazy diffusion, therefore provides a computationally efficient alternative that inherits from but cannot exceed the stability region of the underlying score model.

Turning to System 2, similar trends emerge. As shown in Figure 6–Figure 8, models with the power-law noise schedule accurately preserve both zonal and meridional ocean velocities across 1000-step rollouts, whereas the linear DDPM systematically exhibits spectral energy deficits at intermediate and small scales. The γ=5.0\gamma= 5.0 model and its lazy variant reproduce the correct latitudinal, zonal, and meridional spectral slopes of the ocean reanalysis, while DDPM displays rapid drift toward overly dissipative dynamics.

Autoregressive evolution of meridional velocity from the GLORYS reanalysis dataset

Figure 6. Autoregressive evolution of meridional (VV) velocity from the GLORYS reanalysis dataset. Snapshots shown at t={10,500,1000}t = \{10, 500, 1000\} timesteps (t=1t = 1 day) for ground truth, DDPM, and the highest performing γ\gamma (=5.0= 5.0) and lazy retrain. Color scale shows normalized vorticity.

Quantitative evaluation metrics over autoregressive rollouts

Figure 8. Quantitative evaluation metrics over 1000-timestep autoregressive rollouts for linear (γ=1\gamma= 1) DDPM, γ=5.0\gamma= 5.0 power schedule, and Lazy retraining (γ=1\gamma= 1). Row 1 Shows the total histogram of Zonal velocity over all autoregression times. Row 2 Shows the total histogram of Meridional values over all autoregression times. Row 3 shows the Latitudinal spectrum averaged over the rollout. Rows 3 and 4 shows the Zonal and Meridional velocity spectra averaged over all autoregression times.

Across both systems, the spectral improvements obtained through power-law scheduling persist at all dynamically relevant scales. Both the latitudinal and TKE spectra show that γ=5.0\gamma= 5.0 and its lazy-diffusion counterpart recover the correct power-law behavior through the inertial range and maintain high-wavenumber structure. DDPM, by contrast, exhibits systematic energy loss beginning at intermediate wavenumbers, producing excessive dissipation in physical space.

Together, these results show that power-law noise schedules and lazy diffusion address the precise theoretical failure modes identified earlier: they counteract the wavenumber-dependent SNR collapse, redistribute training emphasis away from irreducible noise, preserve high-kk information deep into the diffusion horizon, and produce stable, structure-preserving, computationally efficient surrogates for turbulent flow.

Autoregressive evolution of zonal velocity from the GLORYS reanalysis dataset

Figure 7. Autoregressive evolution of zonal velocity (UU) from the GLORYS reanalysis dataset. Snapshots shown at t={10,500,1000,2000}t = \{10, 500, 1000, 2000\} timesteps (t=1t = 1 day) for ground truth, DDPM, and the highest performing γ\gamma (=5.0= 5.0) and lazy retrain. Color scale shows normalized vorticity.

Conclusion

This work establishes a rigorous theoretical and practical foundation for fully probabilistic diffusion-based emulation of multiscale dynamical systems. By deriving a Fourier-space characterization of the DDPM forward process, we demonstrated that standard Gaussian noise schedules induce a wavenumber-dependent spectral collapse: the signal-to-noise ratio decays monotonically in ∣k∣|k| for power-law spectra, eliminating high-wavenumber information early in diffusion time. This analysis reveals that conventional diffusion models are structurally ill-suited for turbulent systems, whose inertial-range dynamics require persistent access to fine-scale structure. Our results show that this collapse is not merely a numerical artifact but a fundamental property of the DDPM formulation when applied to power-law-distributed physical data.

Building on these insights, we introduced power-law noise schedules that reinterpret the corruption process as a spectral regularizer. By delaying noise injection, these schedules preserve high-wavenumber coherence deeper into the forward trajectory, thereby enabling the score network to access learnable gradients across the full spectral range. The empirical results on both 2D Kolmogorov turbulence and Gulf of Mexico ocean circulation show that an optimal exponent (γ≈5\gamma\approx5) yields substantial improvements in spectral fidelity, inertial-range scaling, and long-horizon stability. These improvements persist despite minimal architectural modifications, highlighting that the primary barrier to stable generative emulation lies in the physics–model mismatch introduced by the noise schedule. The associated lazy diffusion distillation further demonstrates that, once the score field is properly learned, a one-step generator can replicate the spectral and structural fidelity of long reverse-SDE trajectories at drastically reduced computational cost.

Despite these advances, several limitations remain. First, power-law scheduling introduces sensitivity to discretization: values of γ\gamma that are too large cause near-singular noise injection late in the diffusion trajectory, leading to instability during training. This suggests that theoretical alignment between the continuous-time SDE and the discrete-time parameterization is crucial for preventing numerical artifacts. Second, the lazy diffusion framework, although computationally efficient, inherits the biases and failure modes of the base score model; it cannot compensate for deficiencies in the learned geometry of the conditional distribution. Moreover, the current models operate at moderate resolution and two-dimensional domains, leaving open questions regarding scalability to fully three-dimensional turbulence, anisotropic spectra, and coupled multiphysics systems.

Future work may address these limitations along several axes. From a theoretical standpoint, developing spectrally adaptive noise schedules that respond dynamically to local energy content could provide a principled mechanism for stabilizing diffusion on nonstationary or inhomogeneous flows. Another promising direction lies in unifying diffusion with operator-learning frameworks, enabling models to generalize across domains, resolutions, or physical regimes. On the computational side, scaling lazy diffusion to multi-GPU and distributed environments could enable real-time probabilistic forecasting at resolutions approaching operational numerical weather prediction. Finally, extending the framework to joint state–parameter estimation or uncertainty propagation in hybrid physics–ML systems may further broaden the impact of diffusion-based generative modeling on oceanography, climate science, and beyond.

Overall, this work demonstrates that diffusion models, when redesigned to respect the spectral geometry of turbulence, can overcome long-standing stability challenges in autoregressive emulation. By coupling analytical understanding with algorithmic innovations, we provide a general blueprint for building physically aware, uncertainty-preserving generative surrogates capable of scaling to increasingly realistic dynamical systems.

Acknowledgement

AC designed the research. AS conducted the research and developed the computational codes. Both the authors analyzed the results and prepared the manuscript. Both AS and AC were supported by the National Science Foundation (grant no. 2425667). Computational resources were provided by NSF AC-CESS MTH240019, MTH250006 and NCAR CISL UCSC0008, and UCSC0009. The computational codes can be found in both the Zenodo link: 10.5281/zenodo.17807024 and on GitHub:https://github.com/TACS-UCSC/LazyDiffusion.

Appendix

Detailed analysis of power-law sweeps for System 1

Here, we we show the snapshots of evolution of System 1 for different values of γ\gamma used during the forward diffusion process.

Autoregressive evolution of vorticity fields for 2D Kolmogorov flow

Figure 9. Autoregressive evolution of vorticity fields for 2D Kolmogorov flow. Snapshots shown at t={1,100,500,1000,2000}t = \{1, 100, 500, 1000, 2000\} timesteps for ground truth, DDPM, a sweep of γ\gamma values with their lazy retrains.

Why the score points toward a mode.

Let p0p_0 denote the data density and consider a VE forward process whose marginal at diffusion time τ\tau equals a Gaussian smoothing

pτ(x)=(p0∗N(0,σ2(τ)I))(x),σ(τ)>0.p_\tau(x) = (p_0 * \mathcal{N}(0, \sigma^2(\tau)I))(x), \qquad\sigma(\tau) > 0.

Differentiating under the integral (Dominated Convergence Theorem) gives

∇xlog⁡pτ(x)=mσ(x)−xσ2,\nabla_x \log p_\tau(x) = \frac{m_\sigma(x)-x}{\sigma^2},

where

mσ(x)=∫yp0(y)ϕσ(x−y) dy∫p0(y)ϕσ(x−y) dym_\sigma(x) = \frac{\int y p_0(y)\phi_\sigma(x-y)\,dy}{\int p_0(y)\phi_\sigma(x-y)\,dy}

is the Gaussian-kernel weighted mean. Thus the score direction is

dir⁡(∇xlog⁡pτ(x))=dir⁡(mσ(x)−x).\operatorname{dir}(\nabla_x \log p_\tau(x)) = \operatorname{dir}(m_\sigma(x)-x).

Points with mσ(x)=xm_\sigma(x)=x are stationary points of the smoothed density pτp_\tau. If pτp_\tau is locally unimodal near xx (or p0p_0 is a well-separated mixture at scale σ(τ)\sigma(\tau)), the gradient-ascent flow of log⁡pτ\log p_{\tau} from xx converges to the mode in its attraction basin.

This property means that once a model has learned the score at a given noise level, moving directly toward the mode from xτ∗x_{\tau*} becomes a much simpler task; the model only needs to learn an additional discretization, not the full mapping.

In practice, we found that introducing the extra training instability caused by a Variance Exploding SDE was unnecessary. Models trained using a VPSDE still exhibit stable training, likely because for normalized data the lower variance makes it easier to learn the score, which more than offsets the difficulty of learning a relatively small drift at each noise level.

References

  1. [1]Martin Ehrendorfer. Predicting the uncertainty of numerical weather forecasts: A review. METEOROLOGISCHE ZEITSCHRIFT-BERLIN-, 6:147–183, 1997.
  2. [2]Roberto Buizza. The value of probabilistic prediction. Atmospheric Science Letters, 9(2):36–42, 2008.DOI
  3. [3]Ashesh Chattopadhyay, Jaideep Pathak, Ebrahim Nabizadeh, Wahid Bhimji, and Pedram Hassanzadeh. Long-term stability and generalization of observationally-constrained stochastic data-driven models for geophysical turbulence. Environmental Data Science, 2:e1, 2023.arxiv.org/abs/2205.04601
  4. [4]Georg Kohl, Li-Wei Chen, and Nils Thuerey. Benchmarking autoregressive conditional diffusion models for turbulent flow simulation. arXiv preprint arXiv:2309.01745, 2023.arxiv.org/abs/2309.01745
  5. [5]Jaideep Pathak, Shashank Subramanian, Peter Harrington, Sanjeev Raja, Ashesh Chattopadhyay, Morteza Mardani, Thorsten Kurth, David Hall, Zongyi Li, Kamyar Azizzadenesheli, Pedram Hassanzadeh, Karthik Kashinath, and Animashree Anandkumar. FourCastNet: A global data-driven high-resolution weather model using adaptive Fourier neural operators. arXiv preprint arXiv:2202.11214, 2022.arxiv.org/abs/2202.11214
  6. [6]Remi Lam, Alvaro Sanchez-Gonzalez, Matthew Willson, Peter Wirnsberger, Meire Fortunato, Ferran Ale, Suman Ravuri, Timo Ewalds, Zach Eaton-Rosen, Weihua Hu, Alexander Merose, Stephan Hoyer, George Holland, Oriol Vinyals, Jacklynn Stott, Alexander Pritzel, Shakir Mohamed, and Peter Battaglia. Learning skillful medium-range global weather forecasting. Science, 0(0):eadi2336, 2023.DOI
  7. [7]Kaifeng Bi, Lingxi Xie, Hengheng Zhang, Xin Chen, Xiaotao Gu, and Qi Tian. Accurate medium-range global weather forecasting with 3d neural networks. Nature, pages 1–6, 2023.DOI
  8. [20]Brian DO Anderson. Reverse-time diffusion equation models. Stochastic Processes and their Applications, 12(3):313–326, 1982.DOI
  9. [21]Jonathan Ho, Ajay Jain, and Pieter Abbeel. Denoising diffusion probabilistic models. CoRR, abs/2006.11239, 2020.arxiv.org/abs/2006.11239

Paper details

vlnc:W_g5gqpftb
Version
v1 · latest
Published
Fresh — added to the Hub within the last 30 days
Added
License
CC BY 4.0
Usage

views · 0

downloads · 0

A view counts once per browser per day, a download each time the PDF is requested.

  • Physics and Astronomy

Not recorded for this paper

View statement
Looking for related papers…
Contents