betaFeedback

WorksComputer Science

Dynamical State-Space Diffusion for Spatiotemporal Prediction

Abstract

Diffusion models for temporal prediction typically corrupt each future frame with independent noise, leaving temporal dependence entirely to the denoiser. We propose Dynamical State-Space Diffusion (DSSD), which learns the temporal structure of the corruption process itself. DSSD builds a temporal kernel from the damped oscillatory modes of a stable diagonal state-space model, whose learned decay rates and frequencies let the corruption capture multiple timescales and oscillatory correlations, while the denoiser architecture remains unchanged. Running this model on a finite clip, however, requires the state before the first frame, which is unobserved; starting from a zero state leaves the early frames with less noise than the diffusion schedule prescribes. We avoid this by using the Cholesky factor of the model's stationary covariance as the temporal operator. We prove that this operator is causal, exactly invertible, keeps each frame's noise variance equal to that of standard diffusion, and reduces to the identity at the clean end of the process, and that it is exactly the Kalman innovations form of the state-space model. The denoiser is trained to predict the temporally correlated noise, and sampling removes and reapplies the temporal mixing at each step. On scientific-field and robot-video datasets, DSSD outperforms diffusion baselines that share its denoiser, and the evolution of its learned modes during training shows that the temporal operator adapts to each dataset.

Introduction

Diffusion models generate data by learning to reverse a forward process that gradually corrupts clean samples with Gaussian noise. This denoising view has led to strong image and video generators [18, 36, 31, 19, 4], and it has also become attractive for temporal prediction [40, 10, 29, 37]. In forecasting, a model must represent not one future but a distribution over plausible futures conditioned on an observed history, making diffusion a natural fit for uncertain spatiotemporal signals such as video, weather, and multivariate time series.

Most diffusion frameworks for temporal prediction use a forward process that adds temporally independent noise to each future frame, relying on the denoising network to learn temporal dependencies [40, 10]. However, neighboring states are often temporally coupled through motion, transport, or oscillation. This motivates incorporating temporal structure into the forward process itself, exposing the denoiser to temporally structured signal and noise.

Dynamical Diffusion (DyDiff) [16] incorporates temporal dependence into diffusion by applying an exponential moving average (EMA) operator to both the clean sequence and the diffusion noise. This introduces temporal correlations while preserving a closed-form Gaussian forward process and an invertible temporal operator for sampling. However, at each diffusion step, the EMA operator imposes a single exponential decay across frames. Its decay rate follows a predefined schedule rather than being learned from data, limiting the operator to one memory scale and preventing it from expressing oscillatory correlations.

We propose Dynamical State-Space Diffusion (DSSD), a novel diffusion algorithm that incorporates a learnable state-space operator into forward corruption and reverse sampling. Inspired by S4D [15], the model combines multiple damped oscillatory modes whose decay rates, frequencies, and mixture weights are learned jointly with the denoiser. As illustrated in Figure 1, DSSD uses these learned dynamics to causally mix observed and future latents and to introduce temporal correlations into the diffusion noise. The state-space parameters are shared across sequences and spatial locations, while the denoiser architecture remains unchanged.

DSSD forward process showing observed frame latents $\mathbf{o}$, future frame latents $\mathbf{x}$, learned temporal dynamics, independent noise $\boldsymbol{\epsilon}$, temporal noise mixing, forward corruption at $t$, mixed future latents $\tilde{\mathbf{x}}_t$, correlated noise $\tilde{\boldsymbol{\epsilon}}_t$, and noisy mixed future latents $z_t$

Figure 1. DSSD forward process. Learned state-space dynamics causally mix observed and future latents and separately correlate noise within the future window. The mixed future latents and correlated noise are combined at diffusion step tt to form ztz_t.

Constructing a finite-window temporal operator from this learned kernel requires care at the start of the prediction window. Each output depends on both current and past noise inputs, including those preceding the window. Initializing the state-space model at zero omits the contribution of these earlier inputs, leaving the early frames with less noise variance than prescribed. This violates the requirement that every future frame receive the same noise variance at a given diffusion step. DSSD avoids this problem by using the Cholesky factor of the kernel’s stationary finite-window covariance, yielding a causal and exactly invertible operator with unit marginal noise variance. We use corresponding factors to transform the signal and noise, train the denoiser to predict the resulting correlated noise, and adapt each sampling step to undo and reapply the temporal transform.

Our contributions are summarized as follows:

  • We introduce DSSD, a dynamics-aware diffusion framework that uses a learnable diagonal state-space operator to define the temporal coordinates of the forward and reverse processes, rather than placing all temporal modeling burden on the denoiser.

  • We construct a stationary-covariance Cholesky operator that avoids finite-window boundary effects, is causal and exactly invertible, and preserves unit marginal noise variance. This is equivalent to a Kalman innovations realization, with DyDiff’s EMA operator recovered as a single-mode case.

  • Experiments on scientific-field (SEVIR, Turbulence) and robot-video (BAIR, RoboNet) datasets show that DSSD outperforms diffusion baselines sharing the same denoiser. Analysis of mode evolution during training shows that DSSD’s temporal operator adapts to each dataset.

Related Work

Diffusion models for temporal prediction. Conditional diffusion models have been applied to video prediction [40, 19], precipitation nowcasting [10], and probabilistic time-series forecasting [29, 37, 25]. These methods usually model temporal dependence through the conditioning scheme and the denoiser, for example by masking past frames, generating autoregressively, or adding temporal attention and convolution layers. The forward process remains standard, where each future time step receives independent noise. DSSD instead changes the temporal geometry of the forward process and leaves the denoiser unchanged, so it applies to existing diffusion algorithms.

Structured and learned forward processes. For video generation, temporally correlated noise priors share noise components across frames [11] or warp noise along optical flow [6]; these methods structure the noise but do not apply a matched temporal transform to the clean signal. Blurring and inverse heat dissipation apply a fixed linear operator to the signal, but along the spatial frequency axis rather than the prediction horizon [20, 30]. Other works learn part of the forward process, such as the noise schedule [24] or a data-dependent transformation of the corruption [3]. DYffusion [33] replaces diffusion steps with a learned temporal interpolator, so its reverse process advances physical time, whereas DSSD keeps Gaussian diffusion and changes only its covariance. DyDiff [16] applies a fixed EMA operator to both signal and noise, limiting its ability to represent multi-scale or oscillatory temporal correlations. DSSD learns this operator as a state-space model jointly with the denoiser, keeps the forward process causal, invertible and variance preserving, and recovers DyDiff as an exact special case.

State-space models. Structured state-space models parameterize linear recurrences with stable, structured transition matrices and scale to long sequences [14 b], and selective variants make these dynamics input dependent [13]. In diffusion models, SSMs have mainly served as network layers, either as attention-free backbones (Yan et al., 2024) or as temporal layers of time-series denoisers [25]. Linear-Gaussian state-space models also underlie probabilistic forecasting, where they describe how the data are generated [28]. In contrast, DSSD uses an SSM to define the temporal covariance of the forward process, whose stationary innovations factor gives the diffusion forward operator.

Preliminaries

Conditional diffusion for temporal prediction

Given NN observed frames of a spatiotemporal signal, we predict the next MM. Frames are encoded by a pretrained autoencoder [31] into observed latents o∈RN×do \in\mathbb{R}^{N \times d} and future latents x∈RM×dx \in\mathbb{R}^{M \times d}, where dd is the dimension of a flattened latent frame. We model p(x∣o)p(x \mid o) by diffusing only the future while conditioning the denoiser on the clean observations.

Diffusion models [18] corrupt xx with Gaussian noise. For a variance-preserving schedule αt∈(0,1)\alpha_t \in(0,1), t=1,…,Tt = 1,\ldots,T, whose product αˉt=∏s=1tαs\bar{\alpha}_t = \prod_{s=1}^{t} \alpha_s decreases from nearly 1 to 0,

zt=αˉtx+1−αˉt ϵ,ϵ∼N(0,I).(1)z_t = \sqrt{\bar{\alpha}_t}x + \sqrt{1-\bar{\alpha}_t}\,\epsilon,\qquad\epsilon\sim\mathcal{N}(0,I). \tag*{(1)}

A denoiser fθf_\theta is trained to predict the added noise by minimizing Ex,o,t,ϵ∥fθ(zt,o,t)−ϵ∥22\mathbb{E}_{x,o,t,\epsilon}\lVert f_\theta(z_t,o,t)-\epsilon\rVert_2^2, with tt sampled uniformly. The added noise is independent across future frames, while the denoising network learns temporal dependencies from the data.

Futures are generated from Gaussian noise by repeated denoising, for example with the deterministic DDIM step [36] zt′=αˉt′ x^+1−αˉt′ ϵ^z_{t'} = \sqrt{\bar{\alpha}_{t'}}\,\hat{x}+\sqrt{1-\bar{\alpha}_{t'}}\,\hat{\epsilon} for t′<tt' < t, where ϵ^=fθ(zt,o,t)\hat{\epsilon}=f_\theta(z_t,o,t) and x^=(zt−1−αˉt ϵ^)/αˉt\hat{x}=(z_t-\sqrt{1-\bar{\alpha}_t}\,\hat{\epsilon})/\sqrt{\bar{\alpha}_t}. Throughout, temporal matrices act along the frame axis and apply the same transformation independently to each latent coordinate.

State-space models

Structured state-space models (SSMs) [14 b] describe a sequence through a hidden state that summarizes its past. For the continuous-time linear system h(τ)=Ah(τ)+b e(τ)h(\tau)=Ah(\tau)+b\,e(\tau), y(τ)=cTh(τ)y(\tau)=c^\mathsf{T}h(\tau), zero-order-hold discretization with step size Δ>0\Delta>0 gives the recurrence

hk=Aˉhk−1+bˉek,yk=cThk,Aˉ=eΔA,(2)h_k=\bar{A}h_{k-1}+\bar{b}e_k,\qquad y_k=c^\mathsf{T}h_k,\qquad\bar{A}=e^{\Delta A}, \tag*{(2)}

with bˉ\bar{b} the discretized input vector. From a zero initial state, unrolling gives a causal convolution yk=∑ℓ≥0Kˉℓek−ℓy_k=\sum_{\ell\geq0}\bar{K}_{\ell}e_{k-\ell} whose kernel Kˉℓ=cTAˉℓbˉ\bar{K}_{\ell}=c^\mathsf{T}\bar{A}^{\ell}\bar{b} weights the input ℓ\ell steps in the past.

S4D [15] makes AA diagonal. Taking PP eigenvalues together with their complex conjugates, A=diag⁡(λ1,…,λP,λ1∗,…,λP∗)A = \operatorname{diag}(\lambda_1,\ldots,\lambda_P,\lambda_1^*,\ldots,\lambda_P^*) with Re⁡λn<0\operatorname{Re}\lambda_n < 0, and conjugate coefficients for conjugate states, the kernel becomes a real-valued sum of damped oscillatory modes,

Kˉℓ=∑n=1PrneℓΔRe⁡λncos⁡(ℓΔIm⁡λn+ψn),(3)\bar{K}_{\ell} = \sum_{n=1}^{P} r_n e^{\ell\Delta\operatorname{Re}\lambda_n}\cos(\ell\Delta\operatorname{Im}\lambda_n+\psi_n), \tag*{(3)}

where the amplitude rn≥0r_n \ge0 and phase ψn\psi_n are determined by bˉ\bar{b} and cc. Each mode is a damped oscillation with decay rate −Re⁡λn-\operatorname{Re}\lambda_n and frequency Im⁡λn\operatorname{Im}\lambda_n: one zero-frequency mode gives a geometric decay, and several modes combine decay rates and oscillations in one kernel.

When driven by i.i.d. zero-mean, unit-variance inputs from the infinite past, a stable SSM has a stationary output whose covariance depends only on the lag δ≥0\delta\ge0,

Cov⁡(yk,yk+δ)=∑ℓ≥0KˉℓKˉℓ+δ.(4)\operatorname{Cov}(y_k,y_{k+\delta}) = \sum_{\ell\ge0} \bar{K}_{\ell}\bar{K}_{\ell+\delta}. \tag*{(4)}

Dynamical State-Space Diffusion

We first define how a temporal operator enters the forward process, then build it from a learned SSM kernel through its stationary covariance and Cholesky factor, and finally describe training, sampling and the scope of the guarantees.

Dynamics-aware forward process

Following Guo et al. [16], we couple the strength of temporal mixing to the noise level through a dynamics schedule γˉt=1−η(1−αˉt)∈[1−η,1]\bar{\gamma}_t = 1 - \eta(1-\bar{\alpha}_t) \in[1-\eta,1] with η∈[0,1]\eta\in[0,1], so that γˉt→1\bar{\gamma}_t \to1 (no mixing) as the data become clean and mixing is strongest at high noise. Let Wγ(L)∈RL×LW_{\gamma}^{(L)} \in\mathbb{R}^{L\times L} denote a temporal operator on LL frames at dynamics level γ∈(0,1]\gamma\in(0,1], and partition the operator on the full window as

Wγ(N+M)=[Wγoo0WγfoWγff].(5)W_{\gamma}^{(N+M)} = \begin{bmatrix} W_{\gamma}^{oo} & 0 \\ W_{\gamma}^{fo} & W_{\gamma}^{ff} \end{bmatrix}. \tag*{(5)}

The blocks act on the NN observed and MM future frames of § 3.1: WγfoW_{\gamma}^{fo} carries the observed history into the future window, WγffW_{\gamma}^{ff} mixes future frames with one another, and the zero block keeps the observations free of any influence from the future. DSSD replaces equation (1) by

zt=αˉt(Wγˉtfoo+Wγˉtffx)+1−αˉtWγˉt(M)ϵ,ϵ∼N(0,I).(6)z_t = \sqrt{\bar{\alpha}_t}\left(W_{\bar{\gamma}_t}^{fo}o + W_{\bar{\gamma}_t}^{ff}x\right) + \sqrt{1-\bar{\alpha}_t}W_{\bar{\gamma}_t}^{(M)}\epsilon,\qquad\epsilon\sim\mathcal{N}(0,I). \tag*{(6)}

The clean future is mixed together with the observed history, so the first future frame inherits the temporal state of the observations, whereas the noise is mixed within the future window alone, without an observed prefix. For fixed dynamics parameters, q(zt∣o,x)q(z_t \mid o,x) is Gaussian with mean αˉt(Wfoo+Wffx)\sqrt{\bar{\alpha}_t}(W^{fo}o+W^{ff}x) and covariance (1−αˉt)W(M)W(M)⊤(1-\bar{\alpha}_t)W^{(M)}W^{(M)\top}, so training inputs are still sampled in closed form, and W=IW=I recovers equation (1). As in DDIM [36], the process is specified by these marginals and a coupling in which all diffusion steps share the same (x,ϵ)(x,\epsilon); a Markov forward chain is not required.

We require the operator to be (i) causal, mixing a frame only with itself and preceding frames, i.e., lower triangular; (ii) invertible, so that clean frames and noise can be recovered during sampling; (iii) of unit per-frame variance, diag⁡(WW⊤)=1\operatorname{diag}(WW^\top)=1, so that every frame receives noise of variance 1−αˉt1-\bar{\alpha}_t exactly as in equation (1) and the actual noise level given to the denoiser describes every frame; and (iv) the identity at γ=1\gamma=1, so that the clean end of the process is unchanged. These are design requirements for DSSD, rather than necessary conditions for diffusion models in general.

State-space temporal kernel

DSSD describes the temporal correlation of the mixed frames with a diagonal SSM (§ 3.2) whose discretization step is set by the dynamics level. For PP modes with learnable decay rates an>0a_n > 0 and frequencies ωn≥0\omega_n \ge0, we set

λn=−an2+iωn,Δγ=−log⁡(1−γ).(7)\lambda_n = -\frac{a_n}{2} + i\omega_n,\qquad\Delta_{\gamma} = -\log(1-\gamma). \tag*{(7)}

For 0<γ<10 < \gamma< 1, each discrete pole pn=eΔγλnp_n = e^{\Delta_\gamma\lambda_n} has modulus (1−γ)an/2<1(1-\gamma)^{a_n/2} < 1. Figure 2(c) illustrates these stable poles and their conjugate pairs. As γ→1\gamma\to1 the step Δγ\Delta_\gamma grows without bound and every mode forgets immediately, whereas at high noise the step is smallest and memory is longest. The step is determined by the dynamics schedule, while the learned rates and frequencies control individual modes; an=1a_n = 1 reproduces the EMA decay 1−γ\sqrt{1-\gamma} of DyDiff. We use amplitudes r=softmax⁡(ρ)r = \operatorname{softmax}(\rho) and set every phase ψn\psi_n in equation 3 to zero, giving positive real modal coefficients, and normalize the resulting kernel to unit ℓ2\ell_2 norm,

Figure 2. Noise correlations and state-space modes in DSSD. (a) Illustrative noise correlation matrices for DPM, DyDiff, and DSSD. (b) Corresponding noise correlations by lag: DyDiff’s EMA decays exponentially, whereas DSSD can express oscillatory correlations. (c) An illustrative six-mode DSSD configuration shows discrete poles pn=eΔγλnp_n = e^{\Delta_\gamma\lambda_n} and their conjugates inside the unit circle. Filled and hollow markers denote conjugate partners.

u~ℓ=∑n=1PrneℓΔγRe⁡λncos⁡(ℓΔγIm⁡λn),uℓ=u~ℓ/(∑ℓ′≥0u~ℓ′2)1/2.(8)\tilde{u}_{\ell} = \sum_{n=1}^{P} r_n e^{\ell\Delta_\gamma\operatorname{Re}\lambda_n} \cos\left(\ell\Delta_\gamma\operatorname{Im}\lambda_n\right), \qquad u_{\ell} = \tilde{u}_{\ell}\Big/\left(\sum_{\ell'\geq0}\tilde{u}_{\ell'}^2\right)^{1/2}. \tag*{(8)}

This zero-phase choice fixes u~0=∑nrn=1\tilde{u}_0 = \sum_n r_n = 1, which keeps the normalization bounded away from zero. We use this kernel for memory of past inputs and add a separate current-input term. The resulting impulse response is

w0=γ,wk=1−γ uk−1 (k≥1),∑k≥0wk2=1.(9)w_0 = \sqrt{\gamma}, \qquad w_k = \sqrt{1-\gamma}\,u_{k-1}\ (k\geq1), \qquad\sum_{k\geq0}w_k^2 = 1. \tag*{(9)}

Thus the SSM kernel is delayed by one frame: w0w_0 weights the current input and wkw_k weights an input kk frames in the past. For independent unit-variance inputs, γ\gamma is the variance contributed by the current input, while the kernel distributes the remaining variance 1−γ1-\gamma over the past. At γ=1\gamma= 1, we define w=(1,0,…)w = (1,0,\ldots) by its limit. The dynamics parameters ϕ={log⁡an,ωn,ρn}n=1P\phi= \{\log a_n,\omega_n,\rho_n\}_{n=1}^{P} comprise only 3P3P scalars, shared across frames, pixels, samples and diffusion steps. Learning these parameters jointly with the denoiser allows the noise correlations to adapt to the data’s temporal structure beyond a fixed exponential decay.

Causal innovations operator

Applying the impulse response equation 9 as a convolution over a finite window requires inputs preceding that window, or their accumulated state. Starting from a zero state discards the memory of earlier inputs, so the early frames have variance below one and (iii) fails: the first frame has variance w02=γw_0^2 = \gamma. For example, at γ=0.5\gamma= 0.5 it receives only half of the prescribed noise variance. DSSD instead assumes that the SSM has reached its stationary regime. By equation 4, driving the impulse response equation 9 with i.i.d. unit-variance inputs yields a stationary unit-variance process, and DSSD uses the causal factor of the covariance of LL consecutive outputs,

κδ(γ)=∑k≥0wkwk+δ,[Jγ(L)]ij=κ∣i−j∣(γ),Wγ(L)=chol⁡(Jγ(L)),(10)\kappa_\delta(\gamma) = \sum_{k\geq0} w_k w_{k+\delta}, \qquad[J_\gamma^{(L)}]_{ij} = \kappa_{|i-j|}(\gamma), \qquad W_\gamma^{(L)} = \operatorname{chol}\left(J_\gamma^{(L)}\right), \tag*{(10)}

where chol returns the unique lower-triangular factor with positive diagonal satisfying WWT=J‾WW^\mathsf{T} = \overline{J}, and all infinite sums admit closed forms (Appendix A.2). For Gaussian inputs, row jj of WW expresses output jj as the part predictable from preceding outputs plus an independent innovation, whose scale is [W]jj[W]_{jj}. Thus uu defines the memory kernel, ww includes the current-input term, JJ records stationary correlations, and WW realizes them on a finite window. Figure 2(a–Figure 2b) compares these noise correlations with those of standard diffusion and DyDiff.

Proposition 1 (Admissibility). For every γ∈(0,1]\gamma\in(0,1], window length LL and parameters ϕ\phi: (a) Jγ(L)J_{\gamma}^{(L)} is positive definite, so Wγ(L)W_{\gamma}^{(L)} exists and is causal and invertible; (b) diag⁡(Wγ(L)Wγ(L)T)=1\operatorname{diag}\left(W_{\gamma}^{(L)}W_{\gamma}^{(L)\mathsf{T}}\right)=1; (c) [Wγ(L)]jj≥γ\left[W_{\gamma}^{(L)}\right]_{jj}\ge\sqrt{\gamma} for all jj, hence [Wγˉt(L)]jj≥1−η\left[W_{\bar{\gamma}_t}^{(L)}\right]_{jj}\ge\sqrt{1-\eta} at every diffusion step; (d) the leading N×NN\times N block of Wγ(N+M)W_{\gamma}^{(N+M)} equals Wγ(N)W_{\gamma}^{(N)}, and WγfoWγfoT+WγffWγffT=Wγ(M)Wγ(M)TW_{\gamma}^{\mathrm{fo}}W_{\gamma}^{\mathrm{fo}\mathsf{T}}+W_{\gamma}^{\mathrm{ff}}W_{\gamma}^{\mathrm{ff}\mathsf{T}}=W_{\gamma}^{(M)}W_{\gamma}^{(M)\mathsf{T}}; and (e) Wγ(L)→W1(L)=IW_{\gamma}^{(L)}\to W_{1}^{(L)}=I as γ→1\gamma\to1.

Properties (a), (b) and (e) establish requirements (i)–(iv), and (c) bounds from below every divisor in the triangular solves used during sampling. The schedule-wide bound is strictly positive when η<1\eta<1; for η=1\eta=1, invertibility requires αˉt>0\bar{\alpha}_t>0. These bounds do not guarantee a well-conditioned inverse, which we check numerically (Appendix § 10.10). The first part of (d) makes the operator consistent with the observed history, so the signal branch transforms oo and continues into the future; the second part is used in § 4.5. All proofs are given in Appendix § 10.

The Cholesky factor also has a state-space realization. It is an innovations representation of the stationary process defined by ww, rather than the original S4D convolution matrix.

Proposition 2 (State-space recurrence). For γ∈(0,1)\gamma\in(0,1), Wγ(L)W_{\gamma}^{(L)} is the finite-window Kalman innovations realization ([21]) of the SSM of § 4.2, started at its stationary state covariance and zero prior state mean: y=Wγ(L)ey=W_{\gamma}^{(L)}e is computed by a linear recurrence with a 2P2P-dimensional state (Appendix § 10.§ 4) whose innovation scales satisfy σj=[Wγ(L)]jj\sigma_j=\left[W_{\gamma}^{(L)}\right]_{jj}, σ0=1\sigma_0=1 and σj+1≤σj\sigma_{j+1}\le\sigma_j.

The state is the best estimate of the SSM state given the frames read so far, and the innovation scale shrinks as the state becomes better determined. In this view the two branches of equation (6) use different histories: the signal branch runs the recurrence with the observed latents as inputs and continues with the future latents, whereas the noise branch restarts from the stationary initialization. For long windows, this recurrence reduces the cost of applying the operator to linear in LL (Appendix § 10.§ 4).

Proposition 3 (EMA and richer correlations). Let γ∈(0,1)\gamma\in(0,1). With a single mode, a1=1a_1=1 and ω1=0\omega_1=0, Wγ(L)W_{\gamma}^{(L)} is the EMA operator of DyDiff ([16]), y0=e0y_0=e_0 and yj=1−γ yj−1+γ ejy_j=\sqrt{1-\gamma}\,y_{j-1}+\sqrt{\gamma}\,e_j, whose autocorrelation κδ(γ)=(1−γ)δ/2\kappa_\delta(\gamma)=(1-\gamma)^{\delta/2} is positive at every lag δ\delta. In contrast, the DSSD family admits a single oscillating mode with κ1(γ)<0\kappa_1(\gamma)<0.

At each fixed dynamics level, the unrestricted DSSD covariance family therefore includes the EMA covariance (Corollary 1) and correlations beyond it: an oscillating mode can make neighboring frames anticorrelated, a pattern that a monotonically decaying memory cannot produce (Appendix § 10.§ 5).

Training and sampling

Training. The denoiser parameters θ\theta and the dynamics parameters ϕ\phi are trained jointly by regressing the mixed noise,

L(θ,ϕ)=Ex,o,t,ϵ∥fθ(zt,o,t)−Wγˉt(M)ϵ∥22(11)\mathcal{L}(\theta,\phi)=\mathbb{E}_{x,o,t,\epsilon}\left\lVert f_\theta(z_t,o,t)-W_{\bar{\gamma}_t}^{(M)}\epsilon\right\rVert_2^2 \tag*{(11)}

with ztz_t given by equation (6) and tt sampled uniformly. For a fixed operator, the minimizer is the posterior mean of the mixed noise. Because the noise enters ztz_t only through Wγˉt(M)ϵW_{\bar{\gamma}_t}^{(M)}\epsilon, the score of q(zt∣o)q(z_t\mid o) is a fixed linear transform of this posterior mean (Appendix § 10.8), so the correlated target requires no whitening. Since every coordinate of the target has unit variance for every ϕ\phi, learning ϕ\phi cannot lower the loss by shrinking the noise. Because ϕ\phi enters both the corrupted input and the target, learning ϕ\phi selects the temporal coordinates in which the denoiser operates rather than identifying the physical dynamics of the data. Initialization and numerical checks are given in Appendices A.9 and A.10. Algorithm 1 summarizes joint training of the denoiser and dynamics parameters.

Sampling. We use deterministic DDIM steps that undo the mixing of the current step and reapply that of the next one. Given the prediction ϵ^c=fθ(zt,o,t)\hat{\epsilon}_c=f_\theta(z_t,o,t) at step tt and the next step t′<tt'<t,

x^=(Wγˉt′ff)−1(1αˉt(zt−1−αˉtϵ^c)−Wγˉt′foo),ϵ^=(Wγˉt′(M))−1ϵ^c,\hat{x}=\left(W^{ff}_{\bar{\gamma}_{t'}}\right)^{-1}\left(\frac{1}{\sqrt{\bar{\alpha}_t}}\left(z_t-\sqrt{1-\bar{\alpha}_t}\hat{\epsilon}_c\right)-W^{fo}_{\bar{\gamma}_{t'}}o\right),\qquad\hat{\epsilon}=\left(W^{(M)}_{\bar{\gamma}_{t'}}\right)^{-1}\hat{\epsilon}_c,
zt′=αˉt′(Wγˉt′foo+Wγˉt′ffx^)+1−αˉt′Wγˉt′(M)ϵ^.(12)z_{t'}=\sqrt{\bar{\alpha}_{t'}}\left(W^{fo}_{\bar{\gamma}_{t'}}o+W^{ff}_{\bar{\gamma}_{t'}}\hat{x}\right)+\sqrt{1-\bar{\alpha}_{t'}}W^{(M)}_{\bar{\gamma}_{t'}}\hat{\epsilon}. \tag*{(12)}

Sampling starts from zT=WγˉT(M)ξz_T=W^{(M)}_{\bar{\gamma}_T}\xi with ξ∼N(0,I)\xi\sim\mathcal{N}(0,I), and the last step uses αˉ=γˉ=1\bar{\alpha}=\bar{\gamma}=1, where the operator is the identity and x^\hat{x} is returned. Both inverses are triangular solves whose positive divisors satisfy Proposition 1(c), and with W=IW=I the update reduces to the standard DDIM step. Algorithm 2 gives the complete sampling procedure, including initialization and the clean endpoint. The following result justifies this update.

Proposition 4 (Sampler consistency). Fix the dynamics parameters ϕ\phi, assume finite second moments of the data, and let fθf_\theta be the unrestricted minimizer of equation 11. For 0<αˉt<10<\bar{\alpha}_t<1, x^=E[x∣zt,o]\hat{x}=\mathbb{E}[x\mid z_t,o], ϵ^=E[ϵ∣zt,o]\hat{\epsilon}=\mathbb{E}[\epsilon\mid z_t,o], and equation 12 returns E[zt′∣zt,o]\mathbb{E}[z_{t'}\mid z_t,o] under the shared (x,ϵ)(x,\epsilon) coupling. On an interior time interval where αˉ\bar{\alpha} is twice continuously differentiable and 0<αˉ<10<\bar{\alpha}<1, zt′=zt+(t′−t)vt(zt)+O((t′−t)2)z_{t'}=z_t+(t'-t)v_t(z_t)+O((t'-t)^2) for a velocity field vtv_t whose continuity equation is solved by the marginals q(zt∣o)q(z_t\mid o).

Under the flow regularity conditions in Appendix A.6, equation 12 is a first-order discretization of a flow that transports the marginals of the forward process [1], as DDIM is for standard diffusion. With the ideal denoiser, its last step returns the posterior mean of the future.

What the temporal operator changes

Since every operator is invertible, one may ask whether DSSD merely reparameterizes standard diffusion. It does not: the mixing varies with the diffusion step and differs between the signal and noise branches of equation 6. Let Ft=(Wγˉt(M))−1WγˉtffF_t=\left(W^{(M)}_{\bar{\gamma}_t}\right)^{-1}W^{ff}_{\bar{\gamma}_t}, and let ztstd=αˉtx+1−αˉtϵz_t^{\mathrm{std}}=\sqrt{\bar{\alpha}_t}x+\sqrt{1-\bar{\alpha}_t}\epsilon be the standard corruption equation 1 at the same noise level.

Proposition 5 (Information ordering). For every tt, FtFtT⪯IF_tF_t^{\mathsf{T}}\preceq I. Consequently, given oo, a variable with the same joint distribution with xx as ztstdz_t^{\mathrm{std}} can be generated from ztz_t and independent noise without access to xx, and, for data with finite second moments,

min⁡gE[∥x−g(ztstd,o)∥2]≤min⁡gE[∥x−g(zt,o)∥2].(13)\min_g\mathbb{E}\left[\lVert x-g(z_t^{\mathrm{std}},o)\rVert^2\right]\leq\min_g\mathbb{E}\left[\lVert x-g(z_t,o)\rVert^2\right]. \tag*{(13)}

Temporal mixing therefore creates no information about the future at any single noise level; for the EMA operator, Ft=diag⁡(γˉt,1,…,1)F_t=\operatorname{diag}(\sqrt{\bar{\gamma}_t},1,\ldots,1) attenuates only the direction of the first future frame (Appendix A.7). DSSD instead changes the geometry of the regression problem that a finite denoiser solves across noise levels, and the path along which samples are generated. These changes may help a finite denoiser and sampler, but the proposition does not establish such a benefit; whether they help is tested empirically in Section 5.

Experiments

Proposition 5 rules out an information advantage from temporal mixing at a fixed noise level. We therefore test whether learning the temporal operator improves finite-step forecasting over standard diffusion DPM and DyDiff. We evaluate scientific fields from SEVIR [39] and Turbulence [41], and robot videos from BAIR [9] and RoboNet [8].

Experimental setup

Within each dataset, all methods use the same frozen framewise VAE and Stable Video Diffusion-inspired video U-Net [4]. We train for 50,000 optimizer updates and use a shared 50-step DDIM sampling grid [36]. Task windows and training settings are detailed in Appendix C, and example forecasts for all four datasets in Appendix D.1. Metrics include the continuous ranked probability score (CRPS), critical success index (CSI) and Fréchet video distance (FVD); all are defined in Appendix § 12.§ 3. In Tables 1 and 2, bold and underline mark the best and second-best scores for each dataset and metric.

DatasetMethodCRPS ↓Raww8avgw8maxCSI w5 ↑Temporal variogram ↓Energy score ↓RMSE ↓
SEVIRDPM<u>4.174</u><u>2.737</u><u>6.599</u><u>0.5518</u><u>2.339</u><u>10.862</u><u>15.420</u>
SEVIRDyDiff4.6423.1697.9550.39114.49213.02425.540
SEVIRDSSD (Ours)3.9752.4215.9950.57072.12910.26314.585
TurbulenceDPM0.01148<u>0.00751</u><u>0.01065</u>0.9641<u>0.00262</u><u>0.01920</u><u>0.02600</u>
TurbulenceDyDiff<u>0.01100</u>0.007520.01123<u>0.9674</u>0.002600.017910.02427
TurbulenceDSSD (Ours)0.007560.004400.007570.97790.001470.011780.01582

Table 1. Scientific-field prediction results.

DatasetMethodCRPS ↓RMSE ↓Energy ↓Vario. ↓PSNR ↑SSIM ↑LPIPS ↓FVD ↓
BAIRDPM17.78842.90730.17611.07815.0360.59380.2246<u>1329.2</u>
BAIRDyDiff<u>17.691</u><u>42.815</u><u>30.092</u>11.412<u>15.149</u><u>0.5999</u><u>0.2220</u>1417.4
BAIRDSSD (Ours)16.82542.36729.6618.54815.7170.62050.2007766.0
RoboNetDPM8.782<u>23.523</u><u>15.130</u><u>5.939</u><u>22.680</u>0.78890.1370<u>659.2</u>
RoboNetDyDiff<u>8.383</u>23.73514.7056.50422.6300.78650.1377770.6
RoboNetDSSD (Ours)8.10823.48514.5484.40922.6930.78580.1379430.2

Table 2. Robot-video prediction results.

Scientific-field prediction

DSSD has the lowest scores on all three CRPS variants, temporal variogram, energy score, and RMSE, and the highest CSI on both datasets. Its raw CRPS is 3.975 versus 4.174 for the strongest SEVIR baseline (DPM), and 0.00756 versus 0.01100 for the strongest Turbulence baseline (DyDiff).

Robot-video prediction

On BAIR, DSSD outperforms both DPM and DyDiff across all eight reported metrics. On RoboNet, DSSD leads six of eight metrics, while DPM achieves the best SSIM and LPIPS. On both datasets, DSSD shows substantial improvements in FVD and temporal variogram scores, whereas RMSE gains are modest.

Prediction quality across horizons

Figure 3 compares prediction quality across horizons on BAIR. Panel (a) shows an example predicted sequence and absolute-error maps for each method. In this example, DSSD produces more accurate predictions throughout the sequence, with particularly smaller absolute errors than DyDiff at intermediate time steps. Panel (b) reports frame-wise CRPS(w8max⁡w8_{\max}) and temporal variogram scores, averaged over all BAIR test windows at each prediction horizon. DSSD maintains lower scores than both baselines throughout the evaluated horizons, with its advantage persisting into the later frames. These results indicate sustained improvements in probabilistic prediction and closer agreement with ground-truth changes between consecutive frames.

Figure 3. BAIR forecast quality across future frames. (a) Example predictions and absolute-error maps. (b) Test-set frame-wise pooled CRPS and temporal variogram scores; lower is better.

Analysis of learned temporal modes

The learned operator is compact, with only 3P=183P = 18 parameters (§ 4.2). Figure 4 tracks these modes from 10,000 to 50,000 updates through the centered residue logits ∣ρn−ρˉ∣|\rho_n - \bar{\rho}|, their L2L_2 norm (the residue-logit norm), and the normalized entropy of τ=softmax⁡(ρ)\boldsymbol{\tau} = \operatorname{softmax}(\rho) (Appendix § 12.§ 4). The residue-logit norm measures logit spread around the mean, whereas lower entropy indicates more concentrated residue weights across modes. On SEVIR, the residue-logit norm increases from 1.53 to 4.11 while entropy decreases. Entropy also decreases on BAIR and RoboNet: SEVIR and RoboNet concentrate residue weight on the first mode, and BAIR on the second (Appendix § 13.§ 3). On Turbulence, the residue shares remain nearly uniform, yet the modes continue to evolve: at γ=0.5\gamma= 0.5, the first mode’s pole radius decreases from 0.937 to 0.889, indicating faster decay and shorter temporal memory. These observations indicate dataset-dependent adaptation of both mode weights and timescales.

Figure 4. DSSD mode-parameter diagnostics. (a) SEVIR centered residue logits and their norm. (b) Normalized residue entropy on four datasets.

Conclusion

We introduce DSSD, the novel diffusion framework for spatiotemporal prediction to make the coupling between spatial denoising and temporal dynamics jointly learnable. Its forward process adds noise while mixing frames through a learnable state-space operator, and its reverse process combines learned denoising with inversion of the temporal mixing. Joint training of the state-space dynamics and denoiser allows this coupled process to adapt to the data without changing the denoiser architecture. We establish a theoretical foundation through a stationary-covariance construction that ensures causal, invertible temporal mixing with the prescribed noise variance, together with a consistency analysis of the sampling update. Experiments on scientific fields and robot videos show improved probabilistic forecasts and temporal consistency, while analysis of the learned modes demonstrates dataset-dependent adaptation. We expect this framework to help accelerate the development of dynamics-aware video foundation models at higher resolutions and larger scales. Its learnable temporal dynamics may also support online adaptation to changing temporal patterns in streaming data, with potential applications in robotics.

AI Use Statement

We used OpenAI Codex and Anthropic Claude to assist with implementing training, sampling, evaluation, and experiment-management code based on our specifications. These tools also assisted with typesetting and editing proofs, writing numerical verification scripts, providing feedback on experimental design, drafting and editing the manuscript, searching and summarizing literature, and formatting references.

The theoretical model, operator construction, mathematical claims, and proofs were developed by the authors. AI assistance with proofs was limited to presentation and numerical checks; it did not supply proof arguments. AI tools were not used to develop the conceptual framework, propose or refine hypotheses, generate synthetic datasets, or conduct qualitative and thematic data analysis. Translation was not applicable.

We reviewed all AI-assisted work, checked the proofs against our derivations, and verified the algebra through independent numerical implementations. We tested the code and checked citations against primary sources. The authors take responsibility for the final text, claims, code, and other artifacts.

Ethics Statement

This work involves no human subjects, crowdsourced annotation, or personally identifiable information; no institutional review was required.

Reproducibility Statement

Section 4 specifies the method and theoretical assumptions; Appendix A provides complete proofs, and Appendix A.10 details initialization and numerical implementation. Appendix C specifies the data processing, model and optimization settings, evaluation protocols, and compute resources used in our experiments.

We use publicly accessible datasets—SEVIR, BAIR, and RoboNet—and a simulation-based benchmark, Turbulence. Third-party datasets are used in accordance with their terms and are not redistributed.

We will release model weights, training and sampling code, and configurations for the models and datasets used in this paper. The release will include environment specifications and instructions for data preparation and reproducing the experiments. Run records and numerical tests support verification of the experimental settings and theoretical identities.

Mathematical Details and Proofs

We first derive the stationary covariance, then prove the operator properties and its state-space realization, and finally analyze regression and sampling. Throughout, the dynamics parameters ϕ\phi are fixed unless joint optimization is explicitly discussed; an>0a_n > 0, ωn≥0\omega_n \ge0 and rn>0r_n > 0 with ∑nrn=1\sum_n r_n = 1. For γ∈(0,1)\gamma\in(0,1), we omit γ\gamma and ϕ\phi when no confusion arises and write ν=1−γ\nu= \sqrt{1-\gamma}, Δ=Δγ\Delta= \Delta_\gamma, Rn=∣eΔλn∣=νanR_n = |e^{\Delta\lambda_n}| = \nu^{a_n} and ϑn=Δωn\vartheta_n = \Delta\omega_n, so that u~ℓ=∑nrnRnℓcos⁡(ℓϑn)\tilde{u}_\ell= \sum_n r_n R_n^\ell\cos(\ell\vartheta_n) in equation (8). As in the main text, matrices act on the time axis and are shared by all coordinates of each frame. The endpoint γ=1\gamma= 1 is defined by w=(1,0,…)\mathbf{w} = (1,0,\ldots) and W=I\mathbf{W} = I; expressions involving Δ\Delta are evaluated only for γ<1\gamma< 1. We distinguish the SSM memory kernel uℓu_\ell, the full impulse response wkw_k, its stationary covariance JJ, and the finite-window innovations factor WW.

Notation

Table 3 collects the symbols used in § 3 and § 4 and in this appendix, grouped in the order in which they appear. Bold lowercase letters denote vectors or stacked latent frames, bold uppercase letters denote matrices, and all temporal matrices act along the frame axis, identically for every latent coordinate. A few auxiliary symbols local to a single proof step, such as yjprey_j^{\mathrm{pre}} in Appendix § 10.§ 3, are defined where they appear.

SymbolMeaningDefined in
Data and diffusion
N,MN, MNumbers of observed and future frames§ 3.1
LLLength of the window on which a temporal operator acts; L∈{N,M,N+M}L \in\{N, M, N + M\}§ 4.1
ddDimension of a flattened latent frame§ 3.1
o∈RN×d\mathbf{o} \in\mathbb{R}^{N \times d}Observed latents (clean conditioning history)§ 3.1
x∈RM×d\mathbf{x} \in\mathbb{R}^{M \times d}Future latents to be predicted§ 3.1
T,tT, tNumber of diffusion steps; current step t∈{1,…,T}t \in\{1,\ldots,T\}Eq. (1)
t′t'Next, less noisy step of a sampling update, t′<tt' < t§ 3.1
αt,αˉt\alpha_t, \bar{\alpha}_tNoise schedule and signal level αˉt=∏s≤tαs\bar{\alpha}_t = \prod_{s \le t} \alpha_sEq. (1)
ϵ\epsilonWhite Gaussian noise, N(0,I)\mathcal{N}(0,I), before temporal mixingEq. (1)
ztz_tNoisy future latents at step ttEq. (1), Eq. (6)
fθ,θf_\theta, \thetaDenoiser and its parameters§ 3.1
p(x∣o)p(\mathbf{x} \mid\mathbf{o})Conditional distribution of the future given the history§ 3.1
SymbolMeaningDefined in
q(zt∣o)q(z_t \mid o)Marginal of the forward process at step tt§4.4
State-space models (generic)
τ\tauContinuous time§3.2
h,e,yh, e, yState, scalar input and scalar output of an SSM(2)
A,b,c\mathbf{A}, \mathbf{b}, \mathbf{c}State, input and output parameters§3.2
Δ,A‾,b‾\Delta, \overline{\mathbf{A}}, \overline{\mathbf{b}}Step size and zero-order-hold discretization, A‾=eΔA\overline{\mathbf{A}}=e^{\Delta\mathbf{A}}(2)
K‾ℓ\overline{K}_{\ell}Convolution kernel of the discretized SSM; weight of the input ℓ\ell steps in the past(3)
PPNumber of complex-conjugate mode pairs§3.2
λn\lambda_nContinuous-time eigenvalue of mode nn, Re⁡λn<0\operatorname{Re}\lambda_n<0; in DSSD, −an/2+iωn-a_n/2+i\omega_n§3.2, (7)
rn,ψnr_n,\psi_nAmplitude and phase of mode nn(3)
Dynamics schedule and learned modes
η\etaMixing strength, η∈[0,1]\eta\in[0,1]§4.1
γ‾t\overline{\gamma}_tDynamics schedule, 1−η(1−α‾t)∈[1−η,1]1-\eta(1-\overline{\alpha}_t)\in[1-\eta,1]; 1 means no mixing§4.1
γ\gammaGeneric dynamics level, γ∈(0,1]\gamma\in(0,1]§4.1
an,ωna_n,\omega_nLearned decay rate (>0>0) and frequency (≥0\geq0) of mode nn(7)
ρn\rho_nLearned residue logit of mode nn; r=softmax⁡(ρ)r=\operatorname{softmax}(\rho)§4.2
ϕ\phiDynamics parameters {log⁡an,ωn,ρn}n=1P\{\log a_n,\omega_n,\rho_n\}_{n=1}^{P}, 3P3P scalars§4.2
Δγ\Delta_\gammaDiscretization step tied to the dynamics level, −log⁡(1−γ)-\log(1-\gamma)(7)
pnp_nDiscrete pole of mode nn, eΔγλne^{\Delta_\gamma\lambda_n}, with modulus (1−γ)an/2(1-\gamma)^{a_n/2}§4.2
Kernel, covariance and operator
u~ℓ,uℓ\widetilde{u}_{\ell},u_{\ell}Memory kernel before and after normalization to unit ℓ2\ell_2 norm(8)
wkw_kImpulse response: w0=γw_0=\sqrt{\gamma} weights the current input, wk=1−γ uk−1w_k=\sqrt{1-\gamma}\,u_{k-1} the input kk frames earlier(9)
κδ(γ)\kappa_{\delta}(\gamma)Stationary autocorrelation at lag δ\delta, ∑kwkwk+δ\sum_k w_k w_{k+\delta}; κ0=1\kappa_0=1(10)
Jγ(L)J_{\gamma}^{(L)}Stationary covariance of LL consecutive outputs, [J]ij=κ∣i−j∣[J]_{ij}=\kappa_{\lvert i-j\rvert}(10)
chol⁡\operatorname{chol}Lower-triangular Cholesky factor with positive diagonal(10)
Wγ(L)W_{\gamma}^{(L)}Temporal operator, chol⁡(Jγ(L))\operatorname{chol}(J_{\gamma}^{(L)})(10)
W∞∞,Wfo,WffW^{\infty\infty},W^{fo},W^{ff}Blocks of Wγ(N+M)W_{\gamma}^{(N+M)}: history on history, history into future, future on future(5)
Wγ(M)W_{\gamma}^{(M)}Operator applied to the noise, on the future window alone(6)
Innovations recurrence (Proposition 2)
e,eje,e_jInput sequence of the recurrence, y=Wγ(L)ey=W_{\gamma}^{(L)}eProp. 2
y,yjy,y_jOutput sequence of the operatorProp. 2
σj\sigma_jInnovation scale, [Wγ(L)]jj[W_{\gamma}^{(L)}]_{jj}Prop. 2
sjs_jPre-input SSM state, containing only inputs before frame jj(20)
s^j\hat{s}_jState estimate given y0,…,yj−1y_0,\ldots,y_{j-1}(22)
Σ\SigmaStationary state covarianceApp. A.4
SjS_jConditional state covariance given y0,…,yj−1y_0,\ldots,y_{j-1}(21)
kjk_jKalman gain(21)
DDDirect current-input term of the realization, γ\sqrt{\gamma}(20)
Training and sampling
L(θ,ϕ)\mathcal{L}(\theta,\phi)Mixed-noise regression lossEq. (11)
ϵ^c\hat{\epsilon}_cDenoiser prediction of the mixed noise, fθ(zt,o,t)f_{\theta}(z_t,o,t)Eq. (12)
x^,ϵ^\hat{x},\hat{\epsilon}Estimates of the clean future and of the white noise at step ttEq. (12)
ξ\xiWhite Gaussian noise that initializes sampling, zT=WγˉT(M)ξz_T=W_{\bar{\gamma}_T}^{(M)}\xi§4.4
vtv_tVelocity field transported by the samplerProp. 4
Analysis (Proposition 5)
ztstdz_t^{\mathrm{std}}Standard corruption Eq. (1) at the same noise level§4.5
FtF_t(Wγˉt(M))−1Wγˉtff(W_{\bar{\gamma}_t}^{(M)})^{-1}W_{\bar{\gamma}_t}^{ff}, future signal in whitened noise coordinates§4.5

Table 3. Notation.

SymbolMeaningDefined in
ggEstimator of xxEq. (13)
Appendix-only notation
ν\nu1−γ\sqrt{1-\gamma}App. A
RnR_n, ϑn\vartheta_nPole modulus νan\nu^{a_n} and angle Δγωn\Delta_\gamma\omega_n of mode nnApp. A
ZZNormalizer of the kernel, (∑ℓu~ℓ2)1/2≥1\left(\sum_\ell\tilde{u}_\ell^2\right)^{1/2} \ge1App. A.2
VδV_\deltaUnnormalized kernel autocorrelation, ∑ℓu~ℓu~ℓ+δ\sum_\ell\tilde{u}_\ell\tilde{u}_{\ell+\delta}App. A.2
C(R,ϑ,ψ)C(R,\vartheta,\psi)Closed form of ∑ℓRℓcos⁡(ℓϑ+ψ)\sum_\ell R^\ell\cos(\ell\vartheta+\psi)Eq. (14)
ϵj\epsilon_jScalar i.i.d. N(0,1)\mathcal{N}(0,1) inputs of the stationary process, j∈Zj \in\mathbb{Z}App. A.3
E\mathbf{E}Matrix of the EMA recurrenceCor. 1
W~\widetilde{\mathbf{W}}Lower-triangular matrix of the unrolled recurrenceApp. A.4
βt\beta_tNoise variance 1−αˉt1-\bar{\alpha}_tApp. A.6
BtB_tNoise operator Wγˉt(M)\mathbf{W}_{\bar{\gamma}_t}^{(M)}App. A.6
mtm_t, MtM_t, CtC_tCoefficients of zt=mt+Mtx+Ctϵz_t=m_t+M_tx+C_t\epsilonEq. (23)
f⋆f^\starUnrestricted minimizer of Eq. (11)App. A.6
χ\chiSmooth, compactly supported test functionApp. A.6
UU(Wγˉt(M))−1Wγˉtf∘\left(W_{\bar{\gamma}_t}^{(M)}\right)^{-1}W_{\bar{\gamma}_t}^{f\circ}App. A.7
ζ\zetaAuxiliary white noise independent of all other variablesApp. A.7
μ(x)\mu(x)Mean of q(zt∣o,x)q(z_t\mid o,x)App. A.8
Shorthand in Algorithms 1 and 2
HH, GG, BBWγf∘W_\gamma^{f\circ}, WγffW_\gamma^{ff} and Wγ(M)W_\gamma^{(M)} returned by BuildOperators(γ)(\gamma)App. B
EE, DDFrozen encoder and decoder of the autoencoderApp. B
NbN_bMinibatch sizeAlg. 1
x~i\tilde{x}_i, ϵ~i\tilde{\epsilon}_iMixed signal and mixed noise of training example iiAlg. 1
tS>⋯>t0t_S>\cdots>t_0Sampling grid of SS denoising steps, t0=0t_0=0Alg. 2
Indices
jjFrame (row) index within a window
kkDelay of an input in the impulse response
ℓ\ellDelay of an input in the memory kernel
δ\deltaLag between two frames
nn, mmMode indices

Table 1 (PDF p. 15).

Closed-form sums

For 0≤R<10 \le R < 1 and real ϑ,ψ\vartheta,\psi, the real part of the geometric series ∑ℓ≥0Rℓei(ℓϑ+ψ)=eiψ/(1−Reiϑ)\sum_{\ell\geq0}R^\ell e^{i(\ell\vartheta+\psi)}=e^{i\psi}/(1-Re^{i\vartheta}) gives

C(R,ϑ,ψ)=∑ℓ≥0Rℓcos⁡(ℓϑ+ψ)=cos⁡ψ−Rcos⁡(ψ−ϑ)(1−R)2+4Rsin⁡2(ϑ/2)(14)C(R,\vartheta,\psi)=\sum_{\ell\geq0}R^\ell\cos(\ell\vartheta+\psi)=\frac{\cos\psi-R\cos(\psi-\vartheta)}{(1-R)^2+4R\sin^2(\vartheta/2)} \tag*{(14)}

where the denominator equals 1−2Rcos⁡ϑ+R21-2R\cos\vartheta+R^2 and is written in a form that avoids cancellation. Using cos⁡xcos⁡y=12[cos⁡(x−y)+cos⁡(x+y)]\cos x\cos y=\frac{1}{2}[\cos(x-y)+\cos(x+y)] with x=ℓϑnx=\ell\vartheta_n and y=(ℓ+δ)ϑmy=(\ell+\delta)\vartheta_m,

Z2=∑ℓ≥0u~ℓ2=12∑n,mrnrm[C(RnRm,ϑn−ϑm,0)+C(RnRm,ϑn+ϑm,0)],(15)Z^2=\sum_{\ell\geq0}\tilde{u}_\ell^2=\frac{1}{2}\sum_{n,m}r_nr_m\left[C(R_nR_m,\vartheta_n-\vartheta_m,0)+C(R_nR_m,\vartheta_n+\vartheta_m,0)\right], \tag*{(15)}
Vδ=∑ℓ≥0u~ℓu~ℓ+δ=12∑n,mrnrmRmδ[C(RnRm,ϑn−ϑm,−δϑm)+C(RnRm,ϑn+ϑm,δϑm)].(16)V_\delta=\sum_{\ell\geq0}\tilde{u}_\ell\tilde{u}_{\ell+\delta}=\frac{1}{2}\sum_{n,m}r_nr_mR_m^\delta\left[C(R_nR_m,\vartheta_n-\vartheta_m,-\delta\vartheta_m)+C(R_nR_m,\vartheta_n+\vartheta_m,\delta\vartheta_m)\right]. \tag*{(16)}

For δ≥1\delta\geq1, the definition of κδ\kappa_\delta in equation 10 splits into the term w0wδw_0w_\delta and the terms with k≥1k\geq1, and ∑k≥1wkwk+δ=(1−γ)∑ℓ≥0uℓuℓ+δ\sum_{k\geq1}w_kw_{k+\delta}=(1-\gamma)\sum_{\ell\geq0}u_\ell u_{\ell+\delta}. Since uℓ=u~ℓ/Zu_\ell=\tilde{u}_\ell/Z, this gives

κ0=1,κδ=γ(1−γ) uδ−1+(1−γ)VδZ2(δ≥1).(17)\kappa_0=1,\qquad\kappa_\delta=\sqrt{\gamma(1-\gamma)}\,u_{\delta-1}+(1-\gamma)\frac{V_\delta}{Z^2}\quad(\delta\geq1). \tag*{(17)}

Because the amplitudes are positive and sum to one, u~0=1\tilde{u}_0=1 and Z≥1Z\geq1. As γ→1\gamma\to1, every Rn→0R_n\to0, so u~ℓ→0\tilde{u}_\ell\to0 for ℓ>0\ell>0 while u~0=1\tilde{u}_0=1, giving the endpoint w=(1,0,0,…)w=(1,0,0,\ldots).

Admissibility (Proposition 1)

Proof of Proposition 1. Let (εj)j∈Z(\varepsilon_j)_{j\in\mathbb{Z}} be i.i.d. N(0,1)\mathcal{N}(0,1) and yj=∑k≥0wkεj−ky_j=\sum_{k\geq0}w_k\varepsilon_{j-k}. Since ∑kwk2=1\sum_k w_k^2=1, the series converges in mean square, (yj)(y_j) is a stationary Gaussian process, and Cov⁡(yi,yj)=κ∣i−j∣\operatorname{Cov}(y_i,y_j)=\kappa_{\lvert i-j\rvert}; hence J(L)J^{(L)} is the covariance of (y0,…,yL−1)(y_0,\ldots,y_{L-1}).

(a) For 0≤j<L0\leq j<L, split yj=∑k=0jwj−kεk+yjprey_j=\sum_{k=0}^{j}w_{j-k}\varepsilon_k+y_j^{\mathrm{pre}}, where yjprey_j^{\mathrm{pre}} depends only on (εk)k<0(\varepsilon_k)_{k<0}. The first term equals Q(ε0,…,εL−1)⊤Q(\varepsilon_0,\ldots,\varepsilon_{L-1})^\top with QQ lower triangular and diagonal entries w0=γ>0w_0=\sqrt{\gamma}>0, and it is independent of yprey^{\mathrm{pre}}. Thus J(L)=QQ⊤+Cov⁡(ypre)⪰QQ⊤≻0J^{(L)}=QQ^\top+\operatorname{Cov}(y^{\mathrm{pre}})\succeq QQ^\top\succ0. A positive definite matrix has a unique Cholesky factor, which is lower triangular with a positive diagonal and therefore causal and invertible.

(b) diag⁡(WW⊤)=diag⁡(J)=κ0=∑kwk2=1\operatorname{diag}(WW^\top)=\operatorname{diag}(J)=\kappa_0=\sum_k w_k^2=1.

(c) A conditional variance of a Gaussian vector depends only on its covariance, so we may compute it for (y0,…,yL−1)⊤=We(y_0,\ldots,y_{L-1})^\top=We with e∼N(0,I)e\sim\mathcal{N}(0,I). Because WW is lower triangular with a positive diagonal, (e0,…,ej−1)(e_0,\ldots,e_{j-1}) is an invertible linear function of (y0,…,yj−1)(y_0,\ldots,y_{j-1}), and eje_j is independent of it. Hence Var⁡(yj∣y0,…,yj−1)=[W]jj2\operatorname{Var}(y_j\mid y_0,\ldots,y_{j-1})=[W]^2_{jj}. For the process itself, y0,…,yj−1y_0,\ldots,y_{j-1} are functions of (εk)k≤j−1(\varepsilon_k)_{k\leq j-1}, whereas yj=w0εj+yjpasty_j=w_0\varepsilon_j+y_j^{\mathrm{past}} with yjpasty_j^{\mathrm{past}} a function of (εk)k≤j−1(\varepsilon_k)_{k\leq j-1} and εj\varepsilon_j independent of the past. For jointly Gaussian variables, conditioning on more information cannot increase the conditional variance, so [W]jj2=Var⁡(yj∣y<j)≥Var⁡(yj∣(εk)k≤j−1)=w02=γ[W]^2_{jj}=\operatorname{Var}(y_j\mid y_{<j})\geq\operatorname{Var}(y_j\mid(\varepsilon_k)_{k\leq j-1})=w_0^2=\gamma. Finally, γˉt=1−η(1−αˉt)≥1−η\bar{\gamma}_t=1-\eta(1-\bar{\alpha}_t)\geq1-\eta. This gives a positive uniform lower bound if η<1\eta<1. For η=1\eta=1, each finite step with αˉt>0\bar{\alpha}_t>0 remains admissible, but the zero-signal endpoint would have γˉT=0\bar{\gamma}_T=0 and lies outside the construction.

(d) The leading N×NN\times N block of the Toeplitz matrix J(N+M)J^{(N+M)} is J(N)J^{(N)}. In the block form equation 5, W∞oW∞o⊤=J(N)W^{\infty o}W^{\infty o\top}=J^{(N)} and W∞oW^{\infty o} is lower triangular with a positive diagonal, so W∞o=W(N)W^{\infty o}=W^{(N)} by uniqueness. The trailing M×MM\times M block of J(N+M)J^{(N+M)} is likewise J(M)J^{(M)}; multiplying out the block form yields

WfoWfo⊤+WffWff⊤=J(M)=W(M)W(M)⊤.(18)W^{fo}W^{fo\top}+W^{ff}W^{ff\top}=J^{(M)}=W^{(M)}W^{(M)\top}. \tag*{(18)}

Moreover, the first row of W(M)W^{(M)} is (1,0,…,0)(1,0,\ldots,0) because κ0=1\kappa_0=1, so the noise branch of equation 6 leaves its first frame unchanged.

(e) At γ=1\gamma=1, w=(1,0,…)w=(1,0,\ldots) by Appendix A.2, so J(L)=IJ^{(L)}=I and W(L)=IW^{(L)}=I. As γ→1\gamma\to1, both coefficients in equation 17 tend to zero, while ∣uδ−1∣≤1\lvert u_{\delta-1}\rvert\leq1 and ∣Vδ/Z2∣=∣∑ℓuℓuℓ+δ∣≤1\lvert V_\delta/Z^2\rvert=\lvert\sum_\ell u^\ell u^{\ell+\delta}\rvert\leq1 by the unit norm of uu and the Cauchy–Schwarz inequality. Hence κδ→0\kappa_\delta\to0 for every δ≥1\delta\geq1, so Jγ(L)→IJ^{(L)}_\gamma\to I, and continuity of the Cholesky factor on positive definite matrices gives Wγ(L)→IW^{(L)}_\gamma\to I.

Corollary 1 (EMA special case). If P=1P=1, a1=1a_1=1 and w1=0w_1=0, then for every γ∈(0,1]\gamma\in(0,1] the operator Wγ(L)W^{(L)}_\gamma is the matrix of the recurrence y0=v0y_0=v_0, yj=νyj−1+γεjy_j=\nu y_{j-1}+\sqrt{\gamma}\varepsilon_j, i.e., the exponential moving average operator of DyDiff [16].

Proof. At γ=1\gamma=1 both matrices are the identity. For γ<1\gamma<1, R1=νR_1=\nu, u~ℓ=νℓ\tilde{u}_\ell=\nu^\ell, Z2=1/(1−ν2)=1/γZ^2=1/(1-\nu^2)=1/\gamma, uℓ=γνℓu_\ell=\sqrt{\gamma}\nu^\ell and wk=γνkw_k=\sqrt{\gamma}\nu^k, so κδ=γ∑kν2k+δ=νδ\kappa_\delta=\gamma\sum_k\nu^{2k+\delta}=\nu^\delta. Let EE be the matrix of the recurrence, with Ej0=νjE_{j0}=\nu^j and Ejk=γνj−kE_{jk}=\sqrt{\gamma}\nu^{j-k} for 1≤k≤j1\leq k\leq j. For i≥ji\geq j,

(EE⊤)ij=νi+j+γ∑k=1jνi+j−2k=νi+j+νi−j(1−ν2j)=νi−j=κi−j,(EE^\top)_{ij}=\nu^{i+j}+\gamma\sum_{k=1}^{j}\nu^{i+j-2k}=\nu^{i+j}+\nu^{i-j}(1-\nu^{2j})=\nu^{i-j}=\kappa_{i-j},

using γ=1−ν2\gamma=1-\nu^2. Since EE is lower triangular with a positive diagonal, E=chol⁡(J(L))E=\operatorname{chol}(J^{(L)}).

State-space recurrence (Proposition 2)

State-space realization. Unlike the post-input state hkh_k in equation 2, we use a pre-input state sjs_j that contains only inputs before frame jj. This convention implements the one-frame delay in equation 9, with a separate direct term for the current input. For γ∈(0,1)\gamma\in(0,1), define the 2P2P-dimensional real system

Aˉ=blockdiag⁡n(Rn[cos⁡ϑn−sin⁡ϑnsin⁡ϑncos⁡ϑn]),D=γ,(19)\bar{A}=\operatorname{blockdiag}_{n}\left(R_n\begin{bmatrix}\cos\vartheta_n&-\sin\vartheta_n\\ \sin\vartheta_n&\cos\vartheta_n\end{bmatrix}\right),\qquad D=\sqrt{\gamma}, \tag*{(19)}
b=(1,0,…,1,0)T,c=1−γZ(r1,0,…,rP,0)T.b=(1,0,\ldots,1,0)^{\mathsf{T}},\qquad c=\frac{\sqrt{1-\gamma}}{Z}(r_1,0,\ldots,r_P,0)^{\mathsf{T}}.

Each block rotates (1,0)T(1,0)^{\mathsf{T}} by ℓϑn\ell\vartheta_n and scales it by RnℓR_n^\ell, so cTAˉℓb=1−γ uℓ=wℓ+1c^{\mathsf{T}}\bar{A}^{\ell}b=\sqrt{1-\gamma}\,u_\ell=w_{\ell+1}. Hence, with sj=∑k≥1Aˉk−1b εj−ks_j=\sum_{k\geq1}\bar{A}^{k-1}b\,\varepsilon_{j-k}, the process of Appendix A.3 satisfies

sj+1=Aˉsj+bεj,yj=cTsj+Dεj.(20)s_{j+1}=\bar{A}s_j+b\varepsilon_j,\qquad y_j=c^{\mathsf{T}}s_j+D\varepsilon_j. \tag*{(20)}

Because the spectral radius max⁡nRn\max_n R_n is below one, sjs_j is stationary with covariance Σ\Sigma solving Σ=AˉΣAˉT+bbT\Sigma=\bar{A}\Sigma\bar{A}^{\mathsf{T}}+bb^{\mathsf{T}}, and sjs_j is independent of (εk)k≥j(\varepsilon_k)_{k\geq j}.

Innovations. Let s^j=E[sj∣y0,…,yj−1]\hat{s}_j=\mathbb{E}[s_j\mid y_0,\ldots,y_{j-1}] and Sj=Cov⁡(sj∣y0,…,yj−1)S_j=\operatorname{Cov}(s_j\mid y_0,\ldots,y_{j-1}), so s^0=0\hat{s}_0=0 and S0=ΣS_0=\Sigma. Conditional on y<jy_{<j}, the pair (sj+1,yj)(s_{j+1},y_j) is Gaussian with means (Aˉs^j,cTs^j)(\bar{A}\hat{s}_j,c^{\mathsf{T}}\hat{s}_j), variance σj2=Var⁡(yj∣y<j)=cTSjc+D2\sigma_j^2=\operatorname{Var}(y_j\mid y_{<j})=c^{\mathsf{T}}S_jc+D^2 and cross-covariance Cov⁡(sj+1,yj∣y<j)=AˉSjc+bD\operatorname{Cov}(s_{j+1},y_j\mid y_{<j})=\bar{A}S_jc+bD, because εj\varepsilon_j is independent of sjs_j and of y<jy_{<j}. Gaussian conditioning gives the Kalman recursion

kj=AˉSjc+bDσj2,s^j+1=Aˉs^j+kj(yj−cTs^j),Sj+1=AˉSjAˉT+bbT−σj2kjkjT.(21)k_j=\frac{\bar{A}S_jc+bD}{\sigma_j^2},\qquad\hat{s}_{j+1}=\bar{A}\hat{s}_j+k_j(y_j-c^{\mathsf{T}}\hat{s}_j),\qquad S_{j+1}=\bar{A}S_j\bar{A}^{\mathsf{T}}+bb^{\mathsf{T}}-\sigma_j^2k_jk_j^{\mathsf{T}}. \tag*{(21)}

Let ej=(yj−cTs^j)/σje_j=(y_j-c^{\mathsf{T}}\hat{s}_j)/\sigma_j. By the defining property of conditional expectation, eje_j is uncorrelated with every function of y<jy_{<j}, and each eke_k with k<jk<j is such a function. The eje_j are therefore uncorrelated, jointly Gaussian and of unit variance, hence i.i.d. N(0,1)\mathcal{N}(0,1). Substituting yj=cTs^j+σjejy_j=c^{\mathsf{T}}\hat{s}_j+\sigma_je_j into equation 21 gives the recurrence of Proposition 2,

yj=cTs^j+σjej,s^j+1=Aˉs^j+σjkjej,s^0=0,(22)y_j=c^{\mathsf{T}}\hat{s}_j+\sigma_je_j,\qquad\hat{s}_{j+1}=\bar{A}\hat{s}_j+\sigma_jk_je_j,\qquad\hat{s}_0=0, \tag*{(22)}

whose input sequence is e=(e0,…,eL−1)e=(e_0,\ldots,e_{L-1}). Both sjs_j and its estimate s^j\hat{s}_j precede the current output; the latter is initialized at zero mean with uncertainty S0=ΣS_0=\Sigma, not at a known zero physical state.

Proof of Proposition 2. Unrolling equation 22 gives (y0,…,yL−1)T=W~(e0,…,eL−1)T(y_0,\ldots,y_{L-1})^{\mathsf{T}}=\widetilde{W}(e_0,\ldots,e_{L-1})^{\mathsf{T}} for a lower-triangular W~\widetilde{W} with diagonal entries σj≥D=γ>0\sigma_j\geq D=\sqrt{\gamma}>0. Then J(L)=Cov⁡(y)=W~W~TJ^{(L)}=\operatorname{Cov}(y)=\widetilde{W}\widetilde{W}^{\mathsf{T}}, and uniqueness of the Cholesky factor gives W~=W(L)\widetilde{W}=W^{(L)}, in particular σj=[W(L)]jj\sigma_j=[W^{(L)}]_{jj}. Since the recurrence is linear and its coefficients do not depend on the input, it computes W~(L)e\widetilde{W}^{(L)}e for any input sequence ee. Furthermore, σ02=cTΣc+D2=Var⁡(y0)=1\sigma_0^2=c^{\mathsf{T}}\Sigma c+D^2=\operatorname{Var}(y_0)=1, and by stationarity σj2=Var⁡(y0∣y−j,…,y−1)\sigma_j^2=\operatorname{Var}(y_0\mid y_{-j},\ldots,y_{-1}), which cannot increase as the conditioning set grows.

In the setting of Corollary 1, y0y_0 determines the state, so S1=0S_1=0 and the recurrence closes on the previous output. The gains depend only on (γ,φ,j)(\gamma,\varphi,j) and cost O(LP3)O(LP^3) per dynamics level, after which applying the recurrence to dd-dimensional frames costs O(LPd)O(LPd), compared with O(L3+L2d)O(L^3+L^2d) for equation (10).

Strictly richer correlations

Proof of Proposition 3. The EMA special case follows from Corollary 1, whose proof gives κδ=νδ=(1−γ)δ/2>0\kappa_\delta=\nu^\delta=(1-\gamma)^{\delta/2}>0. To exhibit a correlation outside that family, fix γ∈(0,1)\gamma\in(0,1), take P=1P=1 and ω1=π/Λ\omega_1=\pi/\Lambda, so that ϑ1=π\vartheta_1=\pi, and write R=R1∈(0,1)R=R_1\in(0,1). Then u~ℓ=Rℓcos⁡(ℓπ)=(−R)ℓ\tilde{u}_\ell=R^\ell\cos(\ell\pi)=(-R)^\ell, Z2=∑ℓR2ℓ=1/(1−R2)Z^2=\sum_\ell R^{2\ell}=1/(1-R^2) and uℓ=1−R2(−R)ℓu_\ell=\sqrt{1-R^2}(-R)^\ell, so

∑ℓ≥0uℓuℓ+1=(1−R2)∑ℓ≥0(−R)2ℓ+1=−R(1−R2)∑ℓ≥0R2ℓ=−R.\sum_{\ell\geq0}u_\ell u_{\ell+1}=(1-R^2)\sum_{\ell\geq0}(-R)^{2\ell+1}=-R(1-R^2)\sum_{\ell\geq0}R^{2\ell}=-R.

By equation (17) with V1/Z2=∑ℓuℓuℓ+1V_1/Z^2=\sum_\ell u_\ell u_{\ell+1} and u0=1−R2u_0=\sqrt{1-R^2},

κ1=γ(1−γ)1−R2−(1−γ)R=1−γ(γ1−R2−1−γR),\kappa_1=\sqrt{\gamma(1-\gamma)}\sqrt{1-R^2}-(1-\gamma)R=\sqrt{1-\gamma}\left(\sqrt{\gamma}\sqrt{1-R^2}-\sqrt{1-\gamma}R\right),

which is negative if and only if (1−γ)R2>γ(1−R2)(1-\gamma)R^2>\gamma(1-R^2), that is, R2>γR^2>\gamma. Since R2=(1−γ)a1R^2=(1-\gamma)^{a_1} and log⁡(1−γ)<0\log(1-\gamma)<0, this holds exactly when a1<log⁡γ/log⁡(1−γ)a_1<\log\gamma/\log(1-\gamma), and such a1>0a_1>0 exists because the right-hand side is positive.

For example, γ=0.5\gamma= 0.5 and a1=0.5a_1 = 0.5 give κ1≈−0.150\kappa_1 \approx-0.150. The required frequency ω1=π/Δγ\omega_1 = \pi/\Delta_\gamma lies within the clipping range [0,2π][0, 2\pi] of Appendix § 10.10 whenever Δγ≥12\Delta_\gamma\ge\frac{1}{2}, i.e., γ≥1−e−1/2≈0.39\gamma\ge1-e^{-1/2} \approx0.39, which covers every γˉt\bar{\gamma}_t when η≤e−1/2\eta\le e^{-1/2}, including η=0.5\eta= 0.5.

Sampler consistency (Proposition 4)

Write βt=1−αˉt\beta_t = 1-\bar{\alpha}_t, Bt=Wγˉt(M)B_t = W_{\bar{\gamma}_t}^{(M)}, mt=αˉtWγˉtfoom_t = \sqrt{\bar{\alpha}_t}W_{\bar{\gamma}_t}^{fo}o, Mt=αˉtWγˉtffM_t = \sqrt{\bar{\alpha}_t}W_{\bar{\gamma}_t}^{ff} and Ct=βtBtC_t = \sqrt{\beta_t}B_t, so that equation 6 reads

zt=mt+Mtx+Ctϵ,(23)z_t = m_t + M_t x + C_t \epsilon, \tag*{(23)}

with the same (x,ϵ)(x,\epsilon) for every tt. For 0<αˉt<10 < \bar{\alpha}_t < 1, MtM_t and CtC_t are invertible by Proposition 1(a).

Proof of Proposition 4. Posterior means. The minimizer of equation 11 at step tt is f⋆=E[Btϵ∣zt,o]f^\star= \mathbb{E}[B_t\epsilon\mid z_t,o]. Taking the conditional expectation of equation 23 given (zt,o)(z_t,o), and using E[zt∣zt,o]=zt\mathbb{E}[z_t \mid z_t,o] = z_t, yields zt=mt+MtE[x∣zt,o]+βtf⋆z_t = m_t + M_t\mathbb{E}[x \mid z_t,o] + \sqrt{\beta_t}f^\star. Solving for the first expectation gives exactly x^\hat{x} in equation 12 with ϵ^c=f⋆\hat{\epsilon}_c = f^\star, and E[ϵ∣zt,o]=Bt−1f⋆=ϵ^\mathbb{E}[\epsilon\mid z_t,o] = B_t^{-1}f^\star= \hat{\epsilon}. By linearity of equation 23 at step t′t', E[zt′∣zt,o]=mt′+Mt′x^+Ct′ϵ^\mathbb{E}[z_{t'} \mid z_t,o] = m_{t'} + M_{t'}\hat{x} + C_{t'}\hat{\epsilon}, which is the update equation 12.

Transport. The entries of Jγ(L)J_{\gamma}^{(L)} are smooth in γ∈(0,1)\gamma\in(0,1) by Appendix § 10.§ 2, the Cholesky factor is smooth on positive definite matrices, and γˉt\bar{\gamma}_t is affine in αˉt\bar{\alpha}_t; hence mtm_t, MtM_t and CtC_t are twice continuously differentiable on interior time intervals for η>0\eta> 0. For η=0\eta= 0, the operator is identically II and the same conclusion follows directly. Define vt(z)=m˙t+M˙tE[x∣zt=z,o]+C˙tE[ϵ∣zt=z,o]v_t(z) = \dot{m}_t + \dot{M}_t\mathbb{E}[x \mid z_t = z,o] + \dot{C}_t\mathbb{E}[\epsilon\mid z_t = z,o]. For a smooth, compactly supported test function χ\chi and data with finite second moments,

ddtE[χ(zt)∣o]=E[∇χ(zt)⊤(m˙t+M˙tx+C˙tϵ)∣o]=E[∇χ(zt)⊤vt(zt)∣o].\frac{d}{dt}\mathbb{E}[\chi(z_t)\mid o] = \mathbb{E}\left[\nabla\chi(z_t)^\top(\dot{m}_t+\dot{M}_t x+\dot{C}_t\epsilon)\mid o\right] = \mathbb{E}\left[\nabla\chi(z_t)^\top v_t(z_t)\mid o\right].

by the tower property, which is the weak form of the continuity equation ∂tqt+∇⋅(qtvt)=0\partial_t q_t + \nabla\cdot(q_t v_t) = 0 for qt=q(zt∣o)q_t = q(z_t\mid o).

Consistency. By the first part, zt=mt+Mtx^+Ctϵ^z_t = m_t+M_t\hat{x}+C_t\hat{\epsilon}, so zt′−zt=(mt′−mt)+(Mt′−Mt)x^+(Ct′−Ct)ϵ^=(t′−t)vt(zt)+O((t′−t)2)z_{t'}-z_t = (m_{t'}-m_t)+(M_{t'}-M_t)\hat{x}+(C_{t'}-C_t)\hat{\epsilon} = (t'-t)v_t(z_t)+O((t'-t)^2) by Taylor’s theorem, where the remainder is uniform wherever the posterior means are bounded.

Assuming the velocity admits a unique flow and the continuity equation has a unique solution in the considered class, integrating z˙=vt(z)\dot{z}=v_t(z) backward from the exact terminal marginal reproduces every marginal qtq_t on the interior interval, and equation 12 is its first-order discretization. Extension to a degenerate endpoint requires separate control of the flow limit. At the last step, αˉt′=γˉt′=1\bar{\alpha}_{t'}=\bar{\gamma}_{t'}=1 gives mt′=0m_{t'}=0, Mt′=IM_{t'}=I and Ct′=0C_{t'}=0, so the update returns x^=E[x∣zt,o]\hat{x}=\mathbb{E}[x\mid z_t,o]. Two approximations remain, and both are shared with DDIM. First, by the law of total covariance, Cov⁡(zt′∣o)\operatorname{Cov}(z_{t'}\mid o) exceeds the covariance of E[zt′∣zt,o]\mathbb{E}[z_{t'}\mid z_t,o] by E[Cov⁡(zt′∣zt,o)∣o]⪰0\mathbb{E}[\operatorname{Cov}(z_{t'}\mid z_t,o)\mid o]\succeq0, so a large step omits conditional variance. Second, the initialization BTξB_T\xi matches qTq_T for arbitrary data distributions at αˉT=0\bar{\alpha}_T=0, provided γˉT>0\bar{\gamma}_T>0, since then mT=0m_T=0 and MT=0M_T=0. At αˉT>0\bar{\alpha}_T>0 it is generally approximate; special data distributions can still yield equality. These statements concern the ideal denoiser under the forward marginals, not errors along a learned sampler’s trajectories.

Information ordering (Proposition 5)

Proof of Proposition 5. Fix tt, write B=Wγˉt(M)B = W_{\bar{\gamma}_t}^{(M)}, F=B−1WffF = B^{-1}W^{ff} and U=B−1WfoU = B^{-1}W^{fo}, and let β=1−αˉt\beta= 1-\bar{\alpha}_t. Multiplying equation 18 by B−1B^{-1} on the left and by B−⊤B^{-\top} on the right gives FF⊤+UU⊤=IFF^\top+UU^\top=I, hence FF⊤=I−UU⊤⪯IFF^\top=I-UU^\top\preceq I.

Given oo, the map zt↦z~t=B−1(zt−αˉtWfoo)=αˉtFx+β ϵz_t\mapsto\tilde{z}_t=B^{-1}(z_t-\sqrt{\bar{\alpha}_t}W^{fo}o)=\sqrt{\bar{\alpha}_t}Fx+\sqrt{\beta}\,\epsilon is invertible, so estimators based on (zt,o)(z_t,o) and on (z~t,o)(\tilde{z}_t,o) achieve the same errors. Let ζ∼N(0,I)\zeta\sim\mathcal{N}(0,I) be independent of everything else and define z~t′=Fztstd+(I−FF⊤)ζ\tilde{z}_t^{\prime}=Fz_t^{\mathrm{std}}+\sqrt{(I-FF^\top)}\zeta. Conditional on (x,o)(x,o), z~t′\tilde{z}_t^{\prime} is Gaussian with mean αˉtFx\sqrt{\bar{\alpha}_t}Fx and covariance βFF⊤+β(I−FF⊤)=βI\beta FF^\top+\beta(I-FF^\top)=\beta I, which is the conditional law of z~t\tilde{z}_t. Hence (x,o,z~t′)(x,o,\tilde{z}_t^{\prime}) and (x,o,z~t)(x,o,\tilde{z}_t) have the same joint distribution, and z~t′\tilde{z}_t^{\prime} is computed from ztstdz_t^{\mathrm{std}} and without xx. For any estimator gg with finite risk, g(z~t′,o)g(\tilde{z}_t^{\prime},o) is a function of (ztstd,ζ,o)(z_t^{\mathrm{std}},\zeta,o), so E∥x−g(z~t,o)∥2=E∥x−g(z~t′,o)∥2≥E∥x−g(ztstd,ζ,o)∥2=E∥x−E[x∣ztstd,ζ,o]∥2\mathbb{E}\lVert x-g(\tilde{z}_t,o)\rVert^2 = \mathbb{E}\lVert x-g(\tilde{z}_t^{\prime},o)\rVert^2 \ge\mathbb{E}\lVert x-g(z_t^{\mathrm{std}},\zeta,o)\rVert^2 = \mathbb{E}\lVert x-\mathbb{E}[x\mid z_t^{\mathrm{std}},\zeta,o]\rVert^2, where the inequality holds because the conditional mean minimizes the squared error for data with finite second moments, and the last equality uses the independence of ζ\zeta. Minimizing over gg yields equation (13).

For the EMA operator of Corollary 1 with N≥1N \ge1, every future column of W(N+M)W^{(N+M)} has global index at least one, so its entries are γ νj−k\sqrt{\gamma}\,\nu^{j-k}, whereas W(M)W^{(M)} has first column (νj)j(\nu^j)_j and the same entries elsewhere. Hence Wff=W(M)diag⁡(γ,1,…,1)W^{\mathrm{ff}} = W^{(M)} \operatorname{diag}(\sqrt{\gamma},1,\ldots,1) and F=diag⁡(γ,1,…,1)F = \operatorname{diag}(\sqrt{\gamma},1,\ldots,1): only the first future frame is attenuated. For a general DSSD operator, I−FFT=UUTI - FF^{\mathsf{T}} = UU^{\mathsf{T}} has rank at most min⁡(N,M)\min(N,M).

Colored-noise regression and the score

Fix tt and ϕ\phi, and write β=1−αˉt\beta= 1-\bar{\alpha}_t, B=Wγt(M)B = W^{(M)}_{\gamma_t} and μ(x)=αˉt (Wfoo+Wffx)\mu(x) = \sqrt{\bar{\alpha}_t}\,(W^{\mathrm{fo}}o + W^{\mathrm{ff}}x). For each tt, the minimizer of equation (11) over unrestricted functions is f⋆(z,o,t)=E[Bϵ∣zt=z,o]f^\star(z,o,t) = \mathbb{E}[B\epsilon\mid z_t = z,o]. The marginal is the mixture q(z∣o)=∫p(x∣o) N(z;μ(x),βBBT) dxq(z \mid o) = \int p(x \mid o)\,\mathcal{N}(z;\mu(x),\beta BB^{\mathsf{T}})\,dx. Differentiating under the integral, which is valid under standard integrability conditions,

∇zlog⁡q(z∣o)=E[−(βBBT)−1(z−μ(x))∣zt=z,o]=−β−1/2(BBT)−1f⋆(z,o,t),(24)\nabla_z \log q(z \mid o) = \mathbb{E}\left[-(\beta BB^{\mathsf{T}})^{-1}(z-\mu(x)) \mid z_t=z,o\right] = -\beta^{-1/2}(BB^{\mathsf{T}})^{-1}f^\star(z,o,t), \tag*{(24)}

since zt−μ(x)=β Bϵz_t-\mu(x)=\sqrt{\beta}\,B\epsilon. For any fixed operator, the colored target therefore determines the score, and the correlation between target coordinates requires no whitening of equation (11). Whitening would regress on B−1B^{-1} times the target, that is, on ϵ\epsilon itself; since conditional expectation commutes with the fixed matrix BB, the unrestricted minimizers of the two losses are related by E[ϵ∣zt,o]=B−1f⋆\mathbb{E}[\epsilon\mid z_t,o] = B^{-1}f^\star, so they carry the same information, and the unwhitened minimizer is the one that enters the sampling update equation (12) directly. The two losses do weight the coordinates of the residual differently, which can matter for a finite denoiser; this is part of the regression geometry discussed in § 4.5. This statement concerns a fixed ϕ\phi; the value of ϕ\phi that minimizes equation (11) need not minimize a forecasting error.

Nonzero-frequency initialization

For ℓ≥0\ell\ge0, ∂u~ℓ/∂ωn=−rnRℓℓΔsin⁡(ℓΔωn)\partial\widetilde{u}_{\ell}/\partial\omega_n = -r_nR_{\ell}^{\ell}\Delta\sin(\ell\Delta\omega_n), which vanishes for every ℓ\ell and γ\gamma when ωn=0\omega_n=0. The normalized kernel, the autocorrelation, the operator equation (10) and the loss equation (11) depend on ωn\omega_n only through (u~ℓ)ℓ≥0(\widetilde{u}_{\ell})_{\ell\ge0}, so ∂L/∂ωn=0\partial\mathcal{L}/\partial\omega_n=0 at ωn=0\omega_n=0. With a zero gradient, first-order optimizers such as Adam leave both the parameter and its moment estimates unchanged, so a mode initialized at zero frequency remains non-oscillatory throughout training. This motivates the nonzero initial frequencies of Appendix § 10.10.

Stable parameterization and optimization

Bounded modes. Decay rates and frequencies are clipped to log⁡an∈[−9,3]\log a_n \in[-9,3] and ωn∈[0,2π]\omega_n \in[0,2\pi], and amplitudes are kept positive by the softmax. The closed-form sums and the Cholesky factorization are computed in double precision.

Initialization. Decay rates are log-spaced in [0.12,3][0.12,3], frequencies are set to ωn=2π(n−12)/P\omega_n = 2\pi(n-\frac{1}{2})/P, and amplitudes are uniform. Nonzero initial frequencies are necessary by the argument of Appendix § 10.9.

Operator certificates. Proposition 1 holds exactly, but finite-precision inverses can still amplify errors when a learned operator is ill conditioned. Before training and at every evaluation, we therefore check the operator on a fixed grid of 33 dynamics levels taken from the schedule, at window lengths MM and N+MN+M: the reconstruction error of the full and history-conditioned inverses, unit variance, causality, the identity at γ=1\gamma=1, and a condition number below 10310^3. The checks are run in single and double precision, and training stops if any of them fails. These checks certify only the tested dynamics levels and checkpoints, not every possible parameter value or every point between grid levels.

Training and Sampling Algorithms

Algorithms 1 and 2 summarize the mixed-noise training objective and deterministic sampler. The pretrained encoder E\mathcal{E} and decoder D\mathcal{D} are frozen. All temporal matrices act independently on each latent coordinate.

Shared operator construction. Both algorithms use BuildOperators(γ\gamma), with the current dynamics parameters ϕ\phi and window lengths NN, MM understood. It returns three matrices: HH, which maps the observed history into the future; GG, which mixes future signal frames; and BB, which mixes future noise. The routine constructs the normalized SSM kernel and impulse response in equation 8 and equation 9, computes their stationary covariance matrices for lengths N+MN+M and MM using Appendix A.2, and takes their lower-triangular Cholesky factors as in equation 10. The blocks H=WγfoH = W^{fo}_{\gamma} and G=WγffG = W^{ff}_{\gamma} come from the length-N+MN+M factor, while B=Wγ(M)B = W^{(M)}_{\gamma}. In general, G≠BG \ne B: signal mixing includes the observed history, whereas noise mixing uses a separate future window. At γ=1\gamma= 1, the routine returns (0,IM,IM)(0,I_M,I_M). Parameter bounds and double-precision covariance calculations follow Appendix A.10. The construction is differentiable in ϕ\phi during training.

Algorithm 1Training of DSSD
Require:Training sequences; frozen encoder E\mathcal{E}; window lengths NN, MM; batch size NbN_b; schedule {αˉt}t=1T\{\bar{\alpha}_t\}_{t=1}^{T}; mixing strength η\eta; optimizer and update budget
Ensure:Trained parameters (θ,ϕ)(\theta,\phi)
1:Initialize (θ,ϕ)(\theta,\phi) as in Appendix A.10
2:while update budget remains do
3:Sample a minibatch {(Oi,Xi)}i=1Nb\{(O_i,X_i)\}_{i=1}^{N_b}
4:oi←E(Oi)\mathbf{o}_i \gets\mathcal{E}(O_i); xi←E(Xi)\mathbf{x}_i \gets\mathcal{E}(X_i)
5:for i=1,…,Nbi=1,\ldots,N_b do
6:Sample ti∼Uniform⁡{1,…,T}t_i \sim\operatorname{Uniform}\{1,\ldots,T\}
7:Sample ϵi∼N(0,I)\epsilon_i \sim\mathcal{N}(0,I)
8:γi←1−η(1−αˉti)\gamma_i \gets1-\eta(1-\bar{\alpha}_{t_i})
9:(Hi,Gi,Bi)←BuildOperators⁡(γi)(H_i,G_i,B_i) \gets\operatorname{BuildOperators}(\gamma_i)
10:x~i←Hioi+Gixi\tilde{\mathbf{x}}_i \gets H_i\mathbf{o}_i+G_i\mathbf{x}_i
11:ϵ~i←Biϵi\tilde{\epsilon}_i \gets B_i\epsilon_i
12:zi←αˉtix~i+1−αˉtiϵ~i\mathbf{z}_i \gets\sqrt{\bar{\alpha}_{t_i}}\tilde{\mathbf{x}}_i+\sqrt{1-\bar{\alpha}_{t_i}}\tilde{\epsilon}_i
13:ϵ^c,i←fθ(zi,oi,ti)\hat{\epsilon}_{c,i} \gets f_\theta(\mathbf{z}_i,\mathbf{o}_i,t_i)
14:end for
15:L←Nb−1∑i=1Nb∥ϵ^c,i−ϵ~i∥2\mathcal{L} \gets N_b^{-1}\sum_{i=1}^{N_b}\lVert\hat{\epsilon}_{c,i}-\tilde{\epsilon}_i\rVert^2
16:Compute ∇θ,ϕL\nabla_{\theta,\phi}\mathcal{L} through zi\mathbf{z}_i and ϵ~i\tilde{\epsilon}_i
17:Update (θ,ϕ)(\theta,\phi)
18:end while
19:return (θ,ϕ)(\theta,\phi)

Table 1 (PDF p. 20).

Algorithm 2Sampling of DSSD
Require:Observed frames OO; frozen encoder E\mathcal{E} and decoder D\mathcal{D}; trained (θ,ϕ)(\theta,\phi); prediction length MM; schedule αˉ\bar{\alpha} and mixing strength η\eta; grid T≥tS>⋯>t1>t0=0T \ge t_S > \cdots> t_1 > t_0=0
Ensure:Predicted future frames X^\hat{X}
1:o←E(O)\mathbf{o}\gets\mathcal{E}(O); N←N\gets number of frames in o\mathbf{o}
2:Set αˉ0←1\bar{\alpha}_0\gets1 and γˉ0←1\bar{\gamma}_0\gets1
3:for j=0,…,Sj=0,\ldots,S do
4:γˉtj←1−η(1−αˉtj)\bar{\gamma}_{t_j}\gets1-\eta(1-\bar{\alpha}_{t_j})
5:(Htj,Gtj,Btj)←BuildOperators⁡(γˉtj)(H_{t_j},G_{t_j},B_{t_j})\gets\operatorname{BuildOperators}(\bar{\gamma}_{t_j})
6:end for
7:Draw ξ∼N(0,I)\xi\sim\mathcal{N}(0,I) of shape M×dM\times d
8:ztS←BtSξ\mathbf{z}_{t_S}\gets B_{t_S}\xi
9:for j=S,…,1j=S,\ldots,1 do
10:t←tjt\gets t_j; t′←tj−1t'\gets t_{j-1}
11:ϵ^c←fθ(zt,o,t)\hat{\epsilon}_c\gets f_\theta(\mathbf{z}_t,\mathbf{o},t)
12:Solve Gtx^=(zt−1−αˉtϵ^c)/αˉt−HtoG_t\hat{\mathbf{x}}=(\mathbf{z}_t-\sqrt{1-\bar{\alpha}_t}\hat{\epsilon}_c)/\sqrt{\bar{\alpha}_t}-H_t\mathbf{o} for x^\hat{\mathbf{x}}
13:Solve Btϵ^=ϵ^cB_t\hat{\epsilon}=\hat{\epsilon}_c for ϵ^\hat{\epsilon}
14:zt′←αˉt′(Ht′o+Gt′x^)\mathbf{z}_{t'}\gets\sqrt{\bar{\alpha}_{t'}}(H_{t'}\mathbf{o}+G_{t'}\hat{\mathbf{x}})
15:zt′←zt′+1−αˉt′Bt′ϵ^\mathbf{z}_{t'}\gets\mathbf{z}_{t'}+\sqrt{1-\bar{\alpha}_{t'}}B_{t'}\hat{\epsilon}
16:end for
17:return X^←D(z0)\hat{X}\gets\mathcal{D}(\mathbf{z}_0)

Table 2 (PDF p. 20).

Training details. The noise target is not detached from the dynamics parameters. The loss is written as a batch average of squared norms; the implementation additionally averages over latent coordinates. Operator checks are performed before training and at evaluation checkpoints as described in Appendix A.10.

Sampling details. All model-evaluation steps satisfy 0<αˉtj<10 < \bar{\alpha}_{t_j} < 1 and γˉtj>0\bar{\gamma}_{t_j} > 0. The parameters remain fixed, so the operators can be precomputed; no gradients or new noise draws are needed inside the sampling loop. At t′=0t'=0, (H0,G0,B0)=(0,IM,IM)(H_0,G_0,B_0)=(0,I_M,I_M), and the update returns z0=x^\mathbf{z}_0=\hat{\mathbf{x}}. The initial colored Gaussian is generated from the terminal forward marginal at nonzero αˉtS\bar{\alpha}_{t_S}, as discussed in Appendix A.6. We use steps 1,…,T1,\ldots,T and the clean endpoint 00; the implementation uses array indices 0,…,T−10,\ldots,T-1 and a clean-endpoint sentinel −1-1.

Experimental Details

Data and forecasting tasks

Table 4 specifies the input and prediction windows. All models forecast future frames from observed frames. We map SEVIR and RGB inputs from [0,255][0, 255] to [−1,1][-1, 1] and divide both Turbulence velocity components by four, without clipping.

DatasetInputResolutionN→MN \to MInput transform
SEVIRVIL, 1 channel1282128^27→67 \to6x↦x/127.5−1x \mapsto x/127.5 - 1
TurbulenceVelocity, 2 channels64264^24→114 \to11x↦x/4x \mapsto x/4
BAIRRGB video64264^21→151 \to15x↦x/127.5−1x \mapsto x/127.5 - 1
RoboNetRGB video64264^22→102 \to10x↦x/127.5−1x \mapsto x/127.5 - 1

Table 4. Forecasting tasks. NN and MM are the numbers of observed and future frames.

Architecture, optimization, and sampling

Within each dataset, all methods use the same pretrained, frozen, framewise VAE ([23]), including its EMA weights. The VAE reduces each spatial dimension by a factor of four, outputs three latent channels, and uses a latent scale factor of 0.18215. During training, latent frames are sampled from the VAE posterior; evaluation encodes observations using its posterior mode. The U-Net takes clean observed latents and noisy future latents as conditioning and prediction inputs, respectively. We use a video U-Net [32] inspired by Stable Video Diffusion [4], configured identically for all three methods.

ComponentSetting
Video U-Net64 base channels; channel multipliers (1,2,4,4)(1, 2, 4, 4); two residual blocks per level.
Attention and time mixingAttention at downsampling factors (4,2,1)(4, 2, 1); four heads; transformer depth 1; dropout 0; temporal convolution kernel (3,1,1)(3, 1, 1) and learned spatial/temporal mixing.
Diffusion1,000 timesteps; cosine schedule ([26]) with s=0.008s = 0.008; ϵ\epsilon prediction.
OptimizationAdam ([22]), (β1,β2)=(0.9,0.999)(\beta_1, \beta_2) = (0.9, 0.999), ϵAdam=10−8\epsilon_{\mathrm{Adam}} = 10^{-8}, weight decay 0; effective batch size 16; FP32. No denoiser-weight EMA.
Sampling50 deterministic DDIM steps on the same released_uniform timestep grid.

Table 5. Shared model, optimization, and sampling settings.

MethodTemporal operatorLearning rate
DPMIdentity (no temporal mixing).10−410^{-4}.
DyDiffFixed EMA, η=0.5\eta= 0.5.10−410^{-4}.
DSSDLearned stationary SSM operator, state size 12 (six conjugate pairs), η=0.5\eta= 0.510−410^{-4}.

Table 6. Method-specific temporal operator and learning-rate schedule.

The DSSD operator parameters form a separate optimizer group; the learning-rate schedule applies to both groups. DSSD’s stationary covariance and Cholesky factor are computed in double precision, with the parameter bounds and numerical operator checks detailed in Appendix § 10.10.

DiT backbone. For the SEVIR backbone ablation (Appendix § 13.§ 2), we replace the U-Net with a DiT-S/4 ([27]): 12 blocks of width 384 with six attention heads, MLP ratio 4 and no dropout. Each latent frame is split spatially into 4×44 \times4 patches, giving 64 tokens per frame and 832 tokens for the 13 frames, and attention runs jointly over all frames with fixed sine–cosine spatial and temporal position embeddings. The inputs are the same as for the U-Net, the timestep enters through adaLN-Zero, and only the outputs at future-frame positions are used. The DiT has 32,529,840 parameters; all other settings, including the method-specific learning rates, match the U-Net runs on SEVIR.

Compute. Each model is trained on a single NVIDIA A100 80GB GPU, using the PCIe variant for SEVIR and Turbulence and the SXM4 variant for BAIR and RoboNet. Within each dataset, the three methods share the same GPU type and batch configuration. DSSD adds 18 trainable parameters for its six conjugate mode pairs to the 60,982,185-parameter denoiser shared by all methods, whereas DyDiff’s fixed EMA operator adds none. Table 7 reports the training cost of the first 50,000 updates. Relative to the faster baseline, DSSD’s median time per update is 1.4% higher on SEVIR and 2.0% higher on BAIR, is the lowest on Turbulence, and is 4.9% higher on RoboNet, computed from the values in Table 7. All three methods reserve the same GPU memory on every dataset. Learning the temporal operator therefore adds little training cost.

MethodSEVIRTurbulenceBAIRRoboNet
s/upd.TotalMem.s/upd.TotalMem.s/upd.TotalMem.s/upd.TotalMem.
DPM2.2231.848.81.0614.818.40.9821.511.50.8217.49.2
DyDiff2.1931.348.81.0715.018.40.9821.911.50.8317.89.2
DSSD (Ours)2.2231.948.81.0214.818.41.0022.711.50.8617.79.2

Table 7. Training cost over the first 50,000 updates on one A100 80GB GPU. s/upd.: median time per update (s); Total: wall-clock time (h); Mem.: reserved GPU memory (GiB).

Evaluation metrics

Evaluation protocol. The number KK of generated futures per input follows the evaluation protocol of DyDiff [16], which adopts the prevailing convention of each domain. For SEVIR and Turbulence, the K=8K = 8 generated futures form a probabilistic ensemble, as in Chen et al. (2024). The finite-ensemble CRPS estimator below accounts for ensemble size, and K=8K = 8 keeps sampling tractable on the 128×128128 \times128 SEVIR test set. For BAIR and RoboNet, we follow the video-prediction protocol of Gupta et al. (2023) and Wu et al. (2024), which selects among K=100K = 100 generated videos per input. Because this selection improves with KK, we retain K=100K = 100 for comparability. Within each dataset, all methods share the same KK and 50-step DDIM sampler, so method rankings are comparable. Scores are not comparable between the two dataset groups.

CRPS. For target yy and generated futures x(1),…,x(K)x^{(1)}, \ldots, x^{(K)}, the finite-ensemble probability-weighted-moment CRPS [12] (Zamo & Naveau 2018) at a pixel is

CRPS^(y,x(1:K))=1K∑k=1K∣x(k)−y∣−1K(K−1)∑1≤i<j≤K∣x(i)−x(j)∣.\widehat{\mathrm{CRPS}}(y,x^{(1:K)}) = \frac{1}{K}\sum_{k=1}^{K}\left|x^{(k)}-y\right|-\frac{1}{K(K-1)}\sum_{1\leq i<j\leq K}\left|x^{(i)}-x^{(j)}\right|.

Unpooled CRPS averages this score over pixels, channels, and forecast times, and is reported on all four datasets. On SEVIR and Turbulence, we also report pooled variants: we pool the target and each generated future spatially, using either 8×88 \times8 average or maximum pooling with stride 2, then compute the same CRPS. The subscripts in w8avgw_{8\mathrm{avg}} and w8maxw_{8\mathrm{max}} identify these two pooling operations. To retain compatibility with the reference evaluation, the pooled Turbulence variants include only pooling windows whose target velocities are all nonnegative; unpooled CRPS uses the full field. The per-horizon BAIR curves in Figure 3 use frame-wise CRPS after 8×88 \times8 spatial max pooling, averaged over the full BAIR test set.

CSI. CSI is reported on SEVIR and Turbulence. We threshold the fields before applying a 5×55 \times5 maximum neighborhood operator with stride 2. SEVIR uses raw VIL thresholds 16, 74, 133, 160, 181, 219; Turbulence uses threshold zero for each velocity component. Each CSI is TP/(TP + FP + FN), and the reported score is macro-averaged over defined threshold–horizon–channel cells; cells without an observed or predicted event are excluded. We follow the implemented maximum-neighborhood convention, which differs from an average-pooling description of CSI in the DyDiff paper [16].

Temporal variogram score. The temporal variogram score [35] averages, over lags 1–4, the squared discrepancies between the target’s absolute temporal increments raised to power 1/21/2 and the ensemble mean of the corresponding powered absolute increments. The per-horizon BAIR curves use only consecutive future-frame pairs and therefore start at the second future frame, because the first has no preceding future frame.

Energy score. Let dist⁡(a,b)=∥a−b∥2/MCHW\operatorname{dist}(a,b)=\lVert a-b\rVert_{2}/\sqrt{MCHW} denote the root-mean-square distance over a complete M×C×H×WM \times C \times H \times W future video. We report the energy score [12]

ES⁡(y,x(1:K))=1K∑k=1Kdist⁡(x(k),y)−12K2∑i=1K∑j=1Kdist⁡(x(i),x(j)).\operatorname{ES}(y,x^{(1:K)})=\frac{1}{K}\sum_{k=1}^{K}\operatorname{dist}(x^{(k)},y)-\frac{1}{2K^{2}}\sum_{i=1}^{K}\sum_{j=1}^{K}\operatorname{dist}(x^{(i)},x^{(j)}).

RMSE. RMSE is defined differently for the two dataset groups. On SEVIR and Turbulence, it is the RMSE of the ensemble mean of the KK generated futures. On BAIR and RoboNet, it is the square root of the mean per-input minimum full-video MSE, so a generated future is selected for each input; the BAIR entries are computed as square roots of the corresponding MSE scores.

PSNR, SSIM, and LPIPS. These metrics are reported on BAIR and RoboNet. For each of the K=100K=100 generated videos, we average framewise PSNR, SSIM (Wang et al., 2004), and VGG-based LPIPS (Zhang et al., 2018) over future time. We then report the maximum PSNR and SSIM and the minimum LPIPS for each input, selecting independently for each metric, so the selected video can differ across the three metrics. PSNR and SSIM are evaluated on the [0,255][0,255] RGB scale; LPIPS uses RGB in [−1,1][-1,1].

FVD. Unlike the selection-based metrics above, FVD [38] compares the distribution of all 100 generated videos per input with the real videos. We prepend the observed frames to every generated future and to its real counterpart, then compute the Fréchet distance between I3D [5] feature distributions. LPIPS and FVD are computed offline from saved predictions. Their implementation follows the stated aggregation protocol, but exact numerical identity with an unspecified upstream feature extractor is not assumed.

Learned-mode diagnostics

Each temporal mode corresponds to one complex-conjugate pole pair, so the six residue weights index six modes. Figure 4(a) reports the absolute centered residue logits ∣ρn−ρˉ∣\lvert\rho_n-\bar{\rho}\rvert and their L2L_2 norm, the residue-logit norm, on SEVIR. Figure 7 additionally shows the residue shares rn=softmax⁡(ρ)nr_n=\operatorname{softmax}(\rho)_n for all four datasets. These quantities are computed from checkpoints saved every 10,000 updates from 10,000 to 50,000 updates. Centering removes the common logit shift that leaves the shares unchanged. The residue-logit norm measures the spread of the logits around their mean; absolute centered logits alone do not indicate whether a mode gains or loses weight. The shares directly show how relative weight is redistributed.

Figure 7. Mode-resolved residue diagnostics on the four datasets, measured every 10k training steps (10k–50k) from saved DSSD checkpoints. (a) Softmax residue shares of the six temporal modes, each corresponding to a conjugate pair. (b) Absolute centered residue logits of each mode (bars) and their total L2L_2 norm (diamonds).

Figure 4(b) reports normalized residue entropy, −∑n=16rnlog⁡rn/log⁡6-\sum_{n=1}^{6}r_n\log r_n/\log6, computed from logged training values over the same update range. Values near one indicate nearly uniform shares, while lower values indicate greater concentration. These shares are mixture weights, not independent mode energies; their entropy does not measure pole decay or frequency. As a separate check on the near-constant Turbulence entropy, we reconstruct the discrete pole of its first mode at fixed γ=0.5\gamma=0.5: its radius is 0.937 at 10,000 updates and 0.889 at 50,000 updates, indicating faster decay at that dynamics level despite little redistribution of residue weight. These parameter changes do not establish physical mode identification or a causal contribution to forecasting quality.

Additional Results

Qualitative forecasts

Figures 5 and 6 show one test sequence per dataset. We select the sequence on which DSSD most improves on the better baseline in per-sequence 8×88 \times8 average-pooled CRPS. For SEVIR, we take the highest-ranked sequence whose last two observations and target each have at most 5% saturated pixels and a standard deviation of at least 30 VIL units, because nearly saturated fields are hard to read. Each method row shows the generated future with the lowest full-video MAE among its KK samples, and the error rows show that sample’s absolute error.

BackboneMethodCRPS ↓\downarrow RawCRPS ↓\downarrow w8avgw8_{\mathrm{avg}}CRPS ↓\downarrow w8maxw8_{\mathrm{max}}CSI w5↑w5 \uparrowTemporal variogram ↓\downarrowEnergy score ↓\downarrowRMSE ↓\downarrow
U-NetDPM<u>4.174</u><u>2.737</u><u>6.599</u><u>0.5518</u><u>2.339</u><u>10.862</u><u>15.420</u>
U-NetDyDiff4.6423.1697.9550.39114.49213.02425.540
U-NetDSSD (Ours)3.9752.4215.9950.57072.12910.26314.585
DiTDPM5.4253.8627.4530.43143.559<u>10.437</u><u>23.689</u>
DiTDyDiff<u>5.260</u><u>3.789</u><u>7.327</u><u>0.4383</u><u>3.326</u><u>10.777</u><u>24.905</u>
DiTDSSD (Ours)5.2113.7067.1880.51182.78410.30223.375

Table 8. Backbone ablation on SEVIR. Results with a U-Net and a DiT denoiser. Bold: best; <u>underline:</u> second best within each backbone.

In all four examples, errors concentrate in moving structures: storm cells in SEVIR, shear regions in Turbulence, and the robot arm in BAIR and RoboNet. The baselines’ errors spread more with lead time. This is most visible from t+5t+5 to t+9t+9 in Turbulence, and against DyDiff in the robot videos. DSSD’s errors stay more localized, although they also grow at the final RoboNet frame.

Backbone ablation on SEVIR

Table 8 compares the three methods with either a video U-Net or a Diffusion Transformer (DiT) denoiser (Peebles & Xie [27]), described in Appendix § 12.§ 2. Within each backbone and metric, bold and underlined values indicate the best and second-best methods, respectively.

DSSD obtains the best score on all seven reported metrics within both backbone groups. With DiT, it achieves a higher CSI w5 than DyDiff, the strongest baseline on this metric (0.5118 vs. 0.4383), and a lower temporal variogram score (2.784 vs. 3.326). Its raw CRPS is also lower (5.211 vs. 5.260). Thus, the favorable within-backbone ranking observed with the U-Net also holds for the transformer denoiser on SEVIR.

Mode-resolved temporal adaptation

Each temporal mode corresponds to one complex-conjugate pair of poles, so the six residue weights index six modes. Figure 7 tracks how the size and share of each mode change during training, revealing which modes the model comes to rely on and which it suppresses. Concretely, it shows the softmax residue shares rn=softmax⁡(ρ)nr_n = \operatorname{softmax}(\rho)_n at the saved 10,000–50,000-update checkpoints, together with ∣ρn−ρˉ∣\lvert\rho_n - \bar{\rho} \rvert for each mode and the L2L_2 norm of the centered six-vector. Centering removes the common logit shift, which leaves the residue shares unchanged.

The trajectories differ across datasets. SEVIR’s first-mode share increases from 0.255 to 0.664, BAIR’s second-mode share from 0.329 to 0.744, and RoboNet’s first-mode share from 0.234 to 0.667. Turbulence retains similar shares across the five checkpoints, and its residue-logit norm remains near 0.7. These observations explain the differing entropy trends in Figure 4; they measure residue reweighting, not independent mode energies. In particular, nearly fixed Turbulence residue shares do not imply fixed poles, as the pole-radius comparison in § 5.5 illustrates. Mode indices are tracked within each run and do not identify matching physical modes across datasets.

Multivariate time-series forecasting

We evaluate multivariate forecasting on six datasets distributed through GluonTS (Alexandrov et al. [2]), following TimeGrad’s dataset and prediction length setting (Salinas et al. [34]; Rasul et al. [29]). The datasets are Exchange, Solar, Electricity, Traffic, Taxi, and Wikipedia. To hold the representation and denoiser fixed across methods, DPM, DyDiff, and DSSD use the same temporal 1D dilated-convolution network: a pointwise channel projection followed by eight residual blocks with 128 hidden channels, which jointly processes observed values and noisy future values.

Figure 5. Example scientific-field forecasts and absolute errors on (a) SEVIR and (b) Turbulence. Turbulence shows the vv velocity component. Errors above the color-bar maximum saturate.

Example robot-video forecasts and RGB-mean absolute errors for BAIR, showing Ground Truth, DPM, DyDiff, and DSSD (Ours), with DPM error, DyDiff error, and DSSD (Ours) error; RGB-mean absolute error (pixel units) color bar

(a) (a) BAIR

Example robot-video forecasts and RGB-mean absolute errors for RoboNet, showing Ground Truth, DPM, DyDiff, and DSSD (Ours), with DPM error, DyDiff error, and DSSD (Ours) error; RGB-mean absolute error (pixel units) color bar

(b) (b) RoboNet

Figure 6. Example robot-video forecasts and RGB-mean absolute errors on (a) BAIR and (b) RoboNet. Errors above the color-bar maximum saturate.

Each dataset is one multivariate sequence with DD channels. We compute a mean and standard deviation for each channel from the training sequence and apply the same normalization to test data. For each GluonTS test window, we forecast its future MM points from the preceding NN points. Table 9 gives these dimensions and window lengths.

DatasetDDNNMMMethodCRPS ↓\downarrowRMSE ↓\downarrowMAE ↓\downarrowEnergy ↓\downarrowVariogram ↓\downarrow
Exchange83030DPM<u>0.37573</u><u>0.68591</u><u>0.56123</u>0.594040.78559
DyDiff0.376400.760330.64038<u>0.54593</u><u>0.60261</u>
DSSD (Ours)0.298960.613150.510800.466100.44125
Solar1372424DPM<u>1.43163</u><u>2.31897</u>1.950401.895472.25310
DyDiff1.482592.349641.97462<u>1.92896</u>2.37242
DSSD (Ours)1.416532.31678<u>1.95240</u>1.88563<u>2.26805</u>
Electricity3702424DPM<u>7.12656</u><u>7.08183</u><u>5.72460</u><u>6.72375</u>8.66073
DyDiff7.171277.107745.745846.74831<u>8.67150</u>
DSSD (Ours)7.044367.056995.702726.695638.87129
Traffic9632424DPM<u>7.12199</u><u>10.90963</u>9.67041<u>8.47188</u><u>6.88153</u>
DyDiff7.1484010.911339.666988.477186.85649
DSSD (Ours)7.0837710.90212<u>9.66832</u>8.461177.08459
Taxi1,2142424DPM<u>4.40064</u><u>5.75505</u>4.91774<u>4.65397</u><u>3.57764</u>
DyDiff4.410675.75627<u>4.91771</u>4.656383.57039
DSSD (Ours)4.375685.750234.914154.646673.68216
Wikipedia9,5353030DPM10.3560416.9527811.2046212.660449.40709
DyDiff10.4186216.7049611.0316212.54705<u>9.47266</u>
DSSD (Ours)<u>10.41440</u><u>16.93813</u><u>11.19473</u><u>12.65975</u>9.52240

Table 9. Multivariate time-series test results at 50,000 updates. Bold and underlined values are best and second best within each dataset. All scores use training-set normalization; lower is better. DD is the number of jointly modeled series; NN and MM are observed and predicted time steps.

All methods are trained from scratch for 50,000 optimizer updates with batch size 16, Adam at a constant learning rate of 10−510^{-5}, a 1,000-step cosine diffusion schedule, and seed 42. DyDiff uses the fixed EMA operator with η=0.5\eta= 0.5. For DSSD, we use three conjugate SSM mode pairs (state size six), compared with six pairs in the video experiments. The time-series representation has no spatial latent feature maps, so we use fewer temporal modes as a modest capacity choice. All methods share the same denoiser. We use 50-step deterministic DDIM with a linearly spaced timestep grid.

Table 9 reports channel- and time-averaged finite-ensemble PWM CRPS, ensemble-mean RMSE and MAE, energy score, and temporal variogram score. Metric definitions follow Appendix C.3.

At 50,000 updates, DSSD has the lowest CRPS, RMSE, and energy score on five of six datasets, with DPM leading CRPS on Wikipedia and DyDiff leading its RMSE and energy score. The temporal variogram score is mixed: DSSD is best on Exchange and second on Solar, but has the highest score of the three methods on Electricity, Traffic, Taxi, and Wikipedia.

References

  1. [1]Michael Albergo, Nicholas M. Boffi, and Eric Vanden-Eijnden. Stochastic interpolants: A unifying framework for flows and diffusions. Journal of Machine Learning Research, 26(209):1–80, 2025.
  2. [2]Alexander Alexandrov, Konstantinos Benidis, Michael Bohlke-Schneider, Valentin Flunkert, Jan Gasthaus, Tim Januschowski, Danielle C. Maddix, Syama Rangapuram, David Salinas, Jasper Schulz, Lorenzo Stella, Ali Caner Türkmen, and Yuyang Wang. GluonTS: Probabilistic and neural time series modeling in Python. Journal of Machine Learning Research, 21(116):1–6, 2020.arxiv.org/abs/1906.05264
  3. [3]Grigory Bartosh, Dmitry Vetrov, and Christian A. Naesseth. Neural Flow Diffusion Models: Learnable forward process for improved diffusion modelling. In Advances in Neural Information Processing Systems (NeurIPS), 2024.arxiv.org/abs/2404.12940
  4. [4]Andreas Blattmann, Tim Dockhorn, Sumith Kulal, Daniel Mendelevitch, Maciej Kilian, Dominik Lorenz, Yam Levi, Zion English, Vikram Voleti, Adam Letts, Varun Jampani, and Robin Rombach. Stable Video Diffusion: Scaling latent video diffusion models to large datasets. arXiv preprint arXiv:2311.15127, 2023.arxiv.org/abs/2311.15127
  5. [5]João Carreira and Andrew Zisserman. Quo vadis, action recognition? A new model and the Kinetics dataset. In IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), 2017.arxiv.org/abs/1705.07750
  6. [6]Pascal Chang, Jingwei Tang, Markus Gross, and Vinicius C. Azevedo. How I warped your noise: a temporally-correlated noise prior for diffusion models. In International Conference on Learning Representations (ICLR), 2024.arxiv.org/abs/2504.03072
  7. [8]Sudeep Dasari, Frederik Ebert, Stephen Tian, Suraj Nair, Bernadette Bucher, Karl Schmeckpeper, Siddharth Singh, Sergey Levine, and Chelsea Finn. RoboNet: Large-scale multi-robot learning. In Conference on Robot Learning (CoRL), volume 100 of PMLR, 2019.arxiv.org/abs/1910.11215
  8. [9]Frederik Ebert, Chelsea Finn, Alex X. Lee, and Sergey Levine. Self-supervised visual planning with temporal skip connections. In Conference on Robot Learning (CoRL), volume 78 of PMLR, 2017.arxiv.org/abs/1710.05268
  9. [10]Zhihan Gao, Xingjian Shi, Boran Han, Hao Wang, Xiaoyong Jin, Danielle Maddix, Yi Zhu, Mu Li, and Yuyang Wang. PreDiff: Precipitation nowcasting with latent diffusion models. In Advances in Neural Information Processing Systems (NeurIPS), 2023.arxiv.org/abs/2307.10422
  10. [11]Songwei Ge, Seungjun Nah, Guilin Liu, Tyler Poon, Andrew Tao, Bryan Catanzaro, David Jacobs, Jia-Bin Huang, Ming-Yu Liu, and Yogesh Balaji. Preserve your own correlation: A noise prior for video diffusion models. In IEEE/CVF International Conference on Computer Vision (ICCV), 2023.arxiv.org/abs/2305.10474
  11. [12]Tilmann Gneiting and Adrian E. Raftery. Strictly proper scoring rules, prediction, and estimation. Journal of the American Statistical Association, 102(477):359–378, 2007.
  12. [13]Albert Gu and Tri Dao. Mamba: Linear-time sequence modeling with selective state spaces. In Conference on Language Modeling (COLM), 2024.arxiv.org/abs/2312.00752
  13. [14]Albert Gu, Karan Goel, and Christopher Ré. Efficiently modeling long sequences with structured state spaces. In International Conference on Learning Representations (ICLR), 2022a.arxiv.org/abs/2111.00396
  14. [15]Albert Gu, Ankit Gupta, Karan Goel, and Christopher Ré. On the parameterization and initialization of diagonal state space models. In Advances in Neural Information Processing Systems (NeurIPS), 2022b.arxiv.org/abs/2206.11893
  15. [16]Xingzhuo Guo, Yu Zhang, Baixu Chen, Haoran Xu, Jianmin Wang, and Mingsheng Long. Dynamical Diffusion: Learning temporal dynamics with diffusion models. In International Conference on Learning Representations (ICLR), 2025.arxiv.org/abs/2503.00951
  16. [18]Jonathan Ho, Ajay Jain, and Pieter Abbeel. Denoising diffusion probabilistic models. In Advances in Neural Information Processing Systems (NeurIPS), 2020.arxiv.org/abs/2006.11239
  17. [19]Jonathan Ho, Tim Salimans, Alexey Gritsenko, William Chan, Mohammad Norouzi, and David J. Fleet. Video diffusion models. In Advances in Neural Information Processing Systems (NeurIPS), 2022.
  18. [20]Emiel Hoogeboom and Tim Salimans. Blurring diffusion models. In International Conference on Learning Representations (ICLR), 2023.arxiv.org/abs/2209.05557
  19. [21]Thomas Kailath, Ali H. Sayed, and Babak Hassibi. Linear Estimation. Prentice Hall, 2000.
  20. [22]Diederik P. Kingma and Jimmy Ba. Adam: A method for stochastic optimization. In International Conference on Learning Representations (ICLR), 2015.arxiv.org/abs/1412.6980
  21. [23]Diederik P. Kingma and Max Welling. Auto-encoding variational Bayes. In International Conference on Learning Representations (ICLR), 2014.arxiv.org/abs/1312.6114
  22. [24]Diederik P. Kingma, Tim Salimans, Ben Poole, and Jonathan Ho. Variational diffusion models. In Advances in Neural Information Processing Systems (NeurIPS), 2021.arxiv.org/abs/2107.00630
  23. [25]Juan Miguel Lopez Alcaraz and Nils Strodthoff. Diffusion-based time series imputation and forecasting with structured state space models. Transactions on Machine Learning Research (TMLR), 2023.arxiv.org/abs/2208.09399
  24. [26]Alexander Quinn Nichol and Prafulla Dhariwal. Improved denoising diffusion probabilistic models. In International Conference on Machine Learning (ICML), 2021.arxiv.org/abs/2102.09672
  25. [27]William Peebles and Saining Xie. Scalable diffusion models with transformers. In IEEE/CVF International Conference on Computer Vision (ICCV), 2023.arxiv.org/abs/2212.09748
  26. [28]Syama Sundar Rangapuram, Matthias W. Seeger, Jan Gasthaus, Lorenzo Stella, Yuyang Wang, and Tim Januschowski. Deep state space models for time series forecasting. In Advances in Neural Information Processing Systems (NeurIPS), 2018.
  27. [29]Kashif Rasul, Calvin Seward, Ingmar Schuster, and Roland Vollgraf. Autoregressive denoising diffusion models for multivariate probabilistic time series forecasting. In International Conference on Machine Learning (ICML), 2021.arxiv.org/abs/2101.12072
  28. [30]Severi Rissanen, Markus Heinonen, and Arno Solin. Generative modelling with inverse heat dissipation. In International Conference on Learning Representations (ICLR), 2023.arxiv.org/abs/2206.13397
  29. [31]Robin Rombach, Andreas Blattmann, Dominik Lorenz, Patrick Esser, and Björn Ommer. High-resolution image synthesis with latent diffusion models. In IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), 2022.arxiv.org/abs/2112.10752
  30. [32]Olaf Ronneberger, Philipp Fischer, and Thomas Brox. U-Net: Convolutional networks for biomedical image segmentation. In Medical Image Computing and Computer-Assisted Intervention (MICCAI), 2015.arxiv.org/abs/1505.04597
  31. [33]Salva Rühling Cachay, Bo Zhao, Hailey Joren, and Rose Yu. DYffusion: A dynamics-informed diffusion model for spatiotemporal forecasting. In Advances in Neural Information Processing Systems (NeurIPS), 2023.arxiv.org/abs/2306.01984
  32. [34]David Salinas, Michael Bohlke-Schneider, Laurent Callot, Roberto Medico, and Jan Gasthaus. High-dimensional multivariate forecasting with low-rank Gaussian copula processes. In Advances in Neural Information Processing Systems (NeurIPS), 2019.arxiv.org/abs/1910.03002
  33. [35]Michael Scheuerer and Thomas M. Hamill. Variogram-based proper scoring rules for probabilistic forecasts of multivariate quantities. Monthly Weather Review, 143(4):1321–1334, 2015.
  34. [36]Jiaming Song, Chenlin Meng, and Stefano Ermon. Denoising diffusion implicit models. In International Conference on Learning Representations (ICLR), 2021.arxiv.org/abs/2010.02502
  35. [37]Yusuke Tashiro, Jiaming Song, Yang Song, and Stefano Ermon. CSDI: Conditional score-based diffusion models for probabilistic time series imputation. In Advances in Neural Information Processing Systems (NeurIPS), 2021.arxiv.org/abs/2107.03502
  36. [38]Thomas Unterthiner, Sjoerd van Steenkiste, Karol Kurach, Raphaël Marinier, Marcin Michalski, and Sylvain Gelly. Towards accurate generative models of video: A new metric & challenges. arXiv preprint arXiv:1812.01717, 2018.arxiv.org/abs/1812.01717
  37. [39]Mark Veillette, Siddharth Samsi, and Chris Mattioli. SEVIR: A storm event imagery dataset for deep learning applications in radar and satellite meteorology. In Advances in Neural Information Processing Systems (NeurIPS), 2020.
  38. [40]Vikram Voleti, Alexia Jolicoeur-Martineau, and Christopher Pal. MCVD: Masked conditional video diffusion for prediction, generation, and interpolation. In Advances in Neural Information Processing Systems (NeurIPS), 2022.
  39. [41]Rui Wang, Karthik Kashinath, Mustafa Mustafa, Adrian Albert, and Rose Yu. Towards physics-informed deep learning for turbulent flow prediction. In ACM SIGKDD International Conference on Knowledge Discovery and Data Mining (KDD), 2020.arxiv.org/abs/1911.08655

Paper details

vlnc:W_eyzzhwsw
Version
v1 · latest
Published
Fresh — added to the Hub within the last 30 days
Added
License
CC BY 4.0
Usage

views · 0

downloads · 0

A view counts once per browser per day, a download each time the PDF is requested.

  • Computer Science

Codex, Anthropic Claude

View statement
Looking for related papers…
Contents