Introduction
Suppose we observe a turbulent flow on a coarse grid and want to predict what it will look like some time later. The grid resolves the large scales, but the unobserved fine (subgrid) scales also influence the dynamics. In 1969, Lorenz argued that in a flow with many scales, a fine-scale error propagates to ever larger ones. Two states that differ only in smallest details become as different as two random states, in finite time (Lorenz, [19 ] ; Palmer et al., [26 ] ). Weather forecasting is a classic example of this “butterfly effect” (Lorenz, [19 ] ). The same coarse observation X 0 X_0 X 0 is therefore consistent with many futures X T X_T X T , one for each subgrid realization, and the forecast is the conditional distribution P ( X T ∣ X 0 ) P(X_T \mid X_0) P ( X T ∣ X 0 ) (Chorin et al., [11 ] ; 2002).
A modern approach is to learn P ( X T ∣ X 0 ) P(X_T \mid X_0) P ( X T ∣ X 0 ) with conditional generative models such as diffusion models (Song et al., [33 ] ) and stochastic interpolants (SIs) (Albergo & Vanden-Eijnden, [3 ] ; Albergo et al., [2 ] )1 trained on pairs of initial and final observed states. These models generate samples of future weather and turbulent flows (Price et al., [28 ] ; Li et al., [18 ] ; Chen et al., [9 ] ; Kohl et al., [17 ] ) which resemble real physical states, but do they actually sample from the right conditional distribution (Theis et al., [35 ] )? Testing this is a challenging problem. One reason is that the conditional distribution describes all futures consistent with a given observation but available data usually only contain one trajectory per initial state.
Here we address this challenge. We design an experiment that lets us show that common generative forecasters can make substantial distributional mistakes. We then study a way to reduce these mistakes by better using the available data: forecasting over shorter time intervals and composing the results. Practical forecasters already do this, but mostly as a matter of convenience. Our goal is to understand when and why it helps, and when it can actually hurt.
The experiment works as follows. For five spatio–temporal systems with different dynamical characteristics we first generate coarse states and then complete each coarse state with many independently drawn fine-scale components (§ 2 ). The full states are identical under coarse graining, but the dynamics (computed by a numerical solver) evolve them into different coarse futures. For each coarse initial state this yields a reference ensemble from the conditional distribution, whose statistical properties can be studied empirically. We test a standard conditional SI and a conditional diffusion model trained directly on initial–final pairs ( X 0 , X T ) (X_0, X_T) ( X 0 , X T ) , and find that they can be substantially wrong. Figure 2 shows an example for Euler flow (second and third rows). Their samples have visually correct local statistics, but the global structure is completely wrong.
Figure 2. Direct forecasters can sample the wrong conditional distribution. One held-out Euler observation (top left). Each row shows three samples of the vorticity at time T T T , followed by the conditional mean and the pointwise standard deviation over 10,000 samples. Reference: the simulator, run from 10,000 fine-scale completions of the observation. Direct SI and EDM: generative forecasters trained on pairs ( X 0 , X T ) (X_0, X_T) ( X 0 , X T ) . Cascaded SI: the cascaded forecaster of Section 3 at the validation-selected depth S ⋆ = 10 S^\star= 10 S ⋆ = 10 , discussed in Section 5.
Diffusion models and SIs generate samples via stochastic dynamics in an auxiliary “diffusion” time. This dynamics is in general very different from the physical dynamics of the system over the forecasting period. A forecaster trained only on pairs ( X 0 , X T ) (X_0, X_T) ( X 0 , X T ) has to learn in one fell swoop everything the physical dynamics does between 0 0 0 and T T T . But the physical dynamics is a composition of many short steps. As intermediate observations at times 0 < t < T 0 < t < T 0 < t < T are often available ([31 , 30 ] ), it is natural to ask whether they can be leveraged to improve conditional forecasting. Instead of learning P ( X T ∣ X 0 ) P(X_T \mid X_0) P ( X T ∣ X 0 ) directly, one can learn the conditional transitions P ( X t i + 1 ∣ X t i ) P(X_{t_{i+1}} \mid X_{t_i}) P ( X t i + 1 ∣ X t i ) over S S S shorter intervals and compose them into a cascade of depth S S S . In a cascade of SIs, the generative path is then forced to agree with the physical evolution at some intermediate times. In the Euler example, such a cascaded SI approach is much better than direct and it effectively reproduces the conditional distribution (last row of Figure 2 ).
In practice, the depth of a cascade is set by a fixed time step: GenCast rolls out a twelve-hour transition over fifteen days ([28 ] ), and conditional diffusion models and SIs for fluid simulation use the time step of the training data ([9 , 17 , 23 ] ). Cascading can, however, also discard information even with exact transitions. Only the full state evolves by composing short steps. After coarse graining, the unresolved fine scales carry information from one interval to the next, so that the coarse process is not Markov. To our knowledge, there is no analysis of how cascading affects the learned distribution. The questions we ask here are: is it better to forecast directly or with a cascade, what properties of the physical system determine the answer, and how deep should a cascade be?
We propose a solvable model to answer these questions (§ 3 ). The optimal cascade depth results from the interplay between easier learning and memory loss, and depends on the specific system under study. If the conditional distribution is easy to learn over the full horizon, cascading can be detrimental. This is the case for a scalar field passively advected by a steady flow, whose conditional distribution stays Gaussian at every horizon, with a mean affine in the observation and a covariance that does not depend on it (§ 5 ).
Our main contributions are the following.
An experimental design that enables proper evaluation of the conditional distribution. New datasets for five physical systems, with different simulated futures for each observed initial state, allow us to compare learned and true conditional distributions on every observation (§ 2 and § 4 ). To our knowledge, these are the first datasets designed for this notoriously challenging problem; we release them as an openly accessible benchmark. We find that diffusion models and SIs trained directly over the forecast horizon sample substantially wrong conditional distributions in four of the five systems, despite producing visually plausible samples.
A theoretical explanation. For smooth dynamics, we show that, through nonlinear interactions, the conditional mean leaves a fixed approximation class as the horizon grows, and that, under partial observation, a fine cascade of exact transitions loses the conditional spread of the forecast. We construct a stylized model of a rotating system driven by a hidden Ornstein–Uhlenbeck process where we compute the error of a simple SI exactly and prove that it is minimized at a finite depth S ∗ ≥ 2 S^* \ge2 S ∗ ≥ 2 ; with the rotation switched off, every cascade is worse than the exact direct forecast (§ 3 ).
An experimental test. At a fixed number of supervised transitions, cascading reduces the error of the direct forecasters by factors between 2.3 and 5.7, with the smallest error at an intermediate depth, for nonlinear flows and linear waves alike. For the passive scalar, every added stage increases the error, consistent with the analysis (§ 5 ).
Conditional forecasting from coarse observations
We consider a physical system whose full state Y t Y_t Y t evolves according to Y t = G t ( Y 0 ) Y_t = G_t(Y_0) Y t = G t ( Y 0 ) . A forecaster sees only the coarse state X t = H ( Y t ) X_t = \mathcal{H}(Y_t) X t = H ( Y t ) . At the initial time, we write Y 0 = ( X 0 , Z 0 ) Y_0 = (X_0, Z_0) Y 0 = ( X 0 , Z 0 ) , where Z 0 Z_0 Z 0 contains the fine-scale components. Since the observation X 0 = x 0 X_0 = x_0 X 0 = x 0 generally does not determine Z 0 Z_0 Z 0 , different full initial states consistent with the same observation can lead to different observed futures. The forecasting task is therefore to learn the conditional distribution
P ( X T ∣ X 0 = x 0 ) (1) P(X_T \mid X_0 = x_0) \tag*{(1)} P ( X T ∣ X 0 = x 0 ) (1)
for every observation x 0 x_0 x 0 . In our systems, Y t Y_t Y t consists of one or two fields on a two-dimensional periodic domain, simulated on grids ranging from 128 2 128^2 12 8 2 to 1024 2 1024^2 102 4 2 points. The observation map H \mathcal{H} H keeps only their Fourier modes with 0 < ∣ k ∣ ≤ 8 0 < |k| \le8 0 < ∣ k ∣ ≤ 8 (196 real coordinates per field), while the unresolved initial modes extend to higher wavenumbers that depend on the system (Appendix § 11 .§ 1 ).
Why trajectory data are not enough. A generative forecaster learns an approximation P ^ ( X T ∣ X 0 = x 0 ) \widehat{P}(X_T \mid X_0 = x_0) P ( X T ∣ X 0 = x 0 ) from which it can draw samples. To measure its accuracy, we would like to compare the learned and the true conditional distributions for each observation, using some metric D D D on probability distributions, and then average over observations,
E ( P ^ ) = E x 0 ∼ P ( X 0 ) [ D ( P ( X T ∣ X 0 = x 0 ) , P ^ ( X T ∣ X 0 = x 0 ) ) ] . (2) \mathcal{E}(\widehat{P}) = \mathbb{E}_{x_0 \sim P(X_0)}\left[D\left(P(X_T \mid X_0 = x_0), \widehat{P}(X_T \mid X_0 = x_0)\right)\right]. \tag*{(2)} E ( P ) = E x 0 ∼ P ( X 0 ) [ D ( P ( X T ∣ X 0 = x 0 ) , P ( X T ∣ X 0 = x 0 ) ) ] . (2)
Estimating E \mathcal{E} E requires many samples from P ( X T ∣ X 0 = x 0 ) P(X_T \mid X_0 = x_0) P ( X T ∣ X 0 = x 0 ) for each of many observations x 0 x_0 x 0 . But data usually contain only one future per observed initial state, with all observed states being different. Proper scoring rules can rank forecasters on such data [13 ] , but they cannot determine how far they are from the true conditional. Averaging final spectra or other statistics over initial conditions only tests the marginal distribution P ( X T ) P(X_T) P ( X T ) . Comparing the joint distributions of ( X 0 , X T ) (X_0, X_T) ( X 0 , X T ) could in principle work because they determine the conditionals. The issue is that a forecaster that generates the “correct” forecast for a slightly perturbed observation would have a nearly correct joint distribution, but the forecasts could be completely wrong if they are sensitive to the initial state (which is often the interesting regime).
Hierarchical data generation. When the system can be simulated, this problem can be addressed by sampling the initial data hierarchically. For each conditioning ensemble e e e , we first sample the coarse scales x 0 ( e ) ∼ P ( X 0 ) x_0^{(e)} \sim P(X_0) x 0 ( e ) ∼ P ( X 0 ) , and then many different fine scales for this coarse state,
z 0 ( e , i ) ∼ P ( Z 0 ∣ X 0 = x 0 ( e ) ) , Y 0 ( e , i ) = ( x 0 ( e ) , z 0 ( e , i ) ) , i = 1 , … , n . (3) z_0^{(e,i)} \sim P(Z_0 \mid X_0 = x_0^{(e)}), \qquad Y_0^{(e,i)} = \left(x_0^{(e)}, z_0^{(e,i)}\right), \qquad i = 1,\ldots,n. \tag*{(3)} z 0 ( e , i ) ∼ P ( Z 0 ∣ X 0 = x 0 ( e ) ) , Y 0 ( e , i ) = ( x 0 ( e ) , z 0 ( e , i ) ) , i = 1 , … , n . (3)
A numerical solver sees the full initial state, coarse and fine, and evolves it into (generally) diverging future states, because the fine scales influence the evolution of the coarse ones. But the forecaster only sees the coarse scales, which are the same for every member of the ensemble. The observed endpoints X T ( e , i ) = H ( G T ( Y 0 ( e , i ) ) ) X_T^{(e,i)} = \mathcal{H}(G_T(Y_0^{(e,i)})) X T ( e , i ) = H ( G T ( Y 0 ( e , i ) )) are thus independent samples from P ( X T ∣ X 0 = x 0 ( e ) ) P(X_T \mid X_0 = x_0^{(e)}) P ( X T ∣ X 0 = x 0 ( e ) ) , and they form a reference ensemble against which any forecast from x 0 ( e ) x_0^{(e)} x 0 ( e ) can be compared (Figure 1 ). Each conditioning ensemble belongs as a whole to the training, validation, or test split, so every test observation is new to the forecaster.
Figure 1. Hierarchical data generation. Many unobserved fine initial states z 0 ( i ) z_0^{(i)} z 0 ( i ) are combined with the same observed coarse state x 0 x_0 x 0 . The solver evolves each full initial state Y 0 ( i ) Y_0^{(i)} Y 0 ( i ) , and observing the result at time T T T gives a reference ensemble x T ( 1 ) , x T ( 2 ) , … x_T^{(1)}, x_T^{(2)}, \ldots x T ( 1 ) , x T ( 2 ) , … for P ( X T ∣ X 0 = x 0 ) P(X_T \mid X_0 = x_0) P ( X T ∣ X 0 = x 0 ) .
Direct forecasters can sample the wrong distribution. Using our reference ensembles, we can check whether a learned forecaster samples the right conditional distribution. This leads to our first major finding. Figure 2 shows results for a two-dimensional Euler flow, where the forecaster observes a coarse-grained initial vorticity. We trained two strong generative forecasters on pairs ( X 0 , X T ) (X_0, X_T) ( X 0 , X T ) from the training ensembles: a conditional SI [9 ] , which we call Direct SI, and a conditional elucidated diffusion model (EDM) [16 ] . We then drew 10,000 samples from each model for a held-out observation. We find that both models generate plausible individual samples. However, a closer look reveals that the large-scale structure of their forecasts is wrong. Indeed, the reference mean has a distinct large-scale geometry, and the reference standard deviation is strongly heterogeneous, while the means of the two forecasters are “washed out”, and their standard deviations are nearly uniform. The same happens for waves in heterogeneous media (Appendix § 11 .§ 5 ).
Direct and cascaded forecasts
As in most forecasting datasets, our datasets contain observations at intermediate times 0 < t < T 0 < t < T 0 < t < T . A direct forecaster does not use them: it learns P ( X T ∣ X 0 ) P(X_T \mid X_0) P ( X T ∣ X 0 ) from the pairs ( X 0 , X T ) (X_0, X_T) ( X 0 , X T ) alone. A cascaded forecaster divides the forecast horizon into S S S intervals, with t i = i T / S t_i = iT/S t i = i T / S , and learns the conditional distributions K i ( x , ⋅ ) = P ( X t i + 1 ∈ ⋅ ∣ X t i = x ) K_i(x,\cdot) = P(X_{t_{i+1}} \in\cdot\mid X_{t_i} = x) K i ( x , ⋅ ) = P ( X t i + 1 ∈ ⋅ ∣ X t i = x ) of the transitions between consecutive observed states, for i = 0 , … , S − 1 i = 0,\ldots,S-1 i = 0 , … , S − 1 . Let K ^ i \widehat{K}_i K i denote the learned approximation of K i K_i K i . A cascaded forecast from x 0 x_0 x 0 applies these approximations in turn: starting from X ^ 0 = x 0 \widehat{X}_0 = x_0 X 0 = x 0 , it samples
X ^ t i + 1 ∼ K ^ i ( X ^ t i , ⋅ ) , i = 0 , … , S − 1. (4) \widehat{X}_{t_{i+1}} \sim\widehat{K}_i(\widehat{X}_{t_i},\cdot), \qquad i = 0,\ldots,S-1. \tag*{(4)} X t i + 1 ∼ K i ( X t i , ⋅ ) , i = 0 , … , S − 1. (4)
Each transition is thus conditioned only on the state generated by the previous one, and the forecast X ^ T \widehat{X}_T X T is distributed according to the composition K ^ 0 K ^ 1 ⋯ K ^ S − 1 \widehat{K}_0\widehat{K}_1\cdots\widehat{K}_{S-1} K 0 K 1 ⋯ K S − 1 of the learned transitions. We implement both forecasters with stochastic interpolants (Section 4). The direct one is the Direct SI of Figure 2 , and the cascaded one is Cascaded SI. The two coincide for S = 1 S = 1 S = 1 .
Consider first an ideal setting, in which the model class of the forecaster contains the true conditional distributions, training reaches the population optimum, and sampling is exact. In this setting, Direct SI samples P ( X T ∣ X 0 ) P(X_T \mid X_0) P ( X T ∣ X 0 ) exactly, even though the state is only partially observed, since (2.1) is a well-defined conditional distribution. Cascaded SI need not be exact even in this ideal setting, because the composition K 0 K 1 ⋯ K S − 1 K_0K_1 \cdots K_{S-1} K 0 K 1 ⋯ K S − 1 of the exact transitions generally differs from P ( X T ∣ X 0 ) P(X_T \mid X_0) P ( X T ∣ X 0 ) . In practice, however, the transitions are learned from finite data with a model of finite capacity, which is why shorter transitions can help. We analyze the two effects in turn and then show how they trade off.
Why can shorter transitions be easier to learn? Over a long horizon, the future observation can depend on the initial one in a nonlinear or oscillatory way, which a model of finite capacity may not be able to represent. Over a short horizon this dependence may be simpler. We make this precise for the conditional mean m t ( x ) = E [ X t ∣ X 0 = x ] m_t(x) = \mathbb{E}[X_t \mid X_0 = x] m t ( x ) = E [ X t ∣ X 0 = x ] , since an error in the conditional mean implies an error in the distribution (Proposition C.7). We model limited capacity by a fixed class F \mathcal{F} F of functions of the observation, such as the polynomials of a given degree. For smooth dynamics, the distance from m t m_t m t to F \mathcal{F} F grows like c n t n / n ! c_n t^n/n! c n t n / n ! for short horizons, where t n t^n t n is the first power in the expansion of m t m_t m t whose coefficient lies outside F \mathcal{F} F (Theorem C.1 ). For quadratic dynamics, such as Galerkin truncations of the Euler equations, and quadratic F \mathcal{F} F , this coefficient is cubic in the observation and appears at order t 2 t^2 t 2 (Theorem C.2 ). Repeated nonlinear interactions then create a dependence that a quadratic map cannot represent.
It can however be represented by a composition of short maps from the same class. When the full state is observed, composing S S S fitted polynomial maps of a fixed degree over intervals of length δ = T / S \delta= T/S δ = T / S converges to the true map over [ 0 , T ] [0,T] [ 0 , T ] as δ → 0 \delta\to0 δ → 0 , whereas a single fitted polynomial keeps a positive error (Theorem C.5 ).
Why can too many short transitions hurt? Even when the short transitions are exact, they can incur a memory loss. Along a true physical trajectory, the fine scales at time t i t_i t i are shaped by the unobserved initial state and by the evolution up to t i t_i t i , and they continue to act on the coarse scales after t i t_i t i . The earlier observation X 0 X_0 X 0 therefore has more information about the fine scales at time t i t_i t i than the current observation X t i X_{t_i} X t i and the distribution of X t i + 1 X_{t_{i+1}} X t i + 1 given X t i X_{t_i} X t i changes when we also condition on X 0 X_0 X 0 . In other words, the observed process is not Markov. A cascade replaces the actual fine scales, at each of its S S S stages, by a fresh draw from those compatible with the current observation. This loss of memory under coarse graining is classical in optimal prediction and in the Mori–Zwanzig formalism, where the history of the observed variables enters their reduced dynamics as a memory term [11 , 12 , 21 ] .
An exact transition maps the distribution of X t i X_{t_i} X t i to that of X t i + 1 X_{t_{i+1}} X t i + 1 , so a cascade of exact transitions, averaged over X 0 X_0 X 0 , produces the correct distribution of the state at every time of the partition. What can be wrong is the dependence of the forecast on X 0 X_0 X 0 (Appendix § 12 .§ 6 ). This cannot be detected by pooling statistics over initial conditions, such as energy spectra (§ 2 ). For smooth deterministic dynamics, a cascade of exact transitions converges, as the transitions shrink, to a deterministic flow driven by the conditional mean velocity of the observed state, whose error is at least the conditional standard deviation of the true forecast (Theorem C.10 ). To see why, consider the size of the terms. Over a transition of length δ \delta δ , the unobserved components push the observed state by an amount of order δ \delta δ . Along a true trajectory, these pushes persist and add up to a spread of order T T T , whereas the cascade draws them anew at each stage, so that their fluctuations average out and only their mean survives. Retaining the initial observation X 0 X_0 X 0 at every stage eliminates this memory error for exact transitions (Proposition C.11 ). However, the resulting transitions may be harder to learn, as we show both theoretically and experimentally in the rotation–OU model (Appendices § 12 .8–§ 12 .9).
Why can an intermediate depth be best? The two effects imply a trade-off. To analyze it, we construct a model in which the target distribution, the distribution generated by a specific SI at every depth, and the distance between them can all be computed. The model has two parameters that can be varied independently: ω \omega ω sets how complicated the dependence of the forecast on the observation is, and ρ \rho ρ sets how much of the uncertainty is coupled to the observed state.
We consider a two-dimensional observed state Y t ∈ R 2 Y_t \in\mathbb{R}^2 Y t ∈ R 2 that rotates with an angular velocity controlled by an observed variable c ∼ Unif [ − 1 , 1 ] c \sim\operatorname{Unif}[-1,1] c ∼ Unif [ − 1 , 1 ] , and that is driven in addition by two sources of uncertainty: an unobserved Ornstein–Uhlenbeck (OU) process H t ∈ R 2 H_t \in\mathbb{R}^2 H t ∈ R 2 , which is coloured over time and plays the role of the unobserved fine scales, and independent white noise. With independent $Y_0 \sim N ( 0 , s 0 2 I 2 ) \mathcal{N}(0,s_0^2 I_2) N ( 0 , s 0 2 I 2 ) and H 0 ∼ N ( 0 , I 2 ) H_0 \sim\mathcal{N}(0,I_2) H 0 ∼ N ( 0 , I 2 ) and independent Brownian motions B B B and W W W , we consider
d H t = − κ H t d t + 2 κ d B t , d Y t = ( ω c J Y t + β ρ R ( ω c t ) H t ) d t + σ ρ d W t . (5) \begin{aligned}
dH_t &= -\kappa H_t\,dt + \sqrt{2\kappa}\,dB_t,\\
dY_t &= (\omega_c JY_t + \beta_\rho R(\omega_c t)H_t)\,dt + \sigma_\rho\,dW_t. \tag*{(5)}
\end{aligned} d H t d Y t = − κ H t d t + 2 κ d B t , = ( ω c J Y t + β ρ R ( ω c t ) H t ) d t + σ ρ d W t . (5)
where J = ( 0 − 1 1 0 ) J = \begin{pmatrix} 0 & -1 \\ 1 & 0 \end{pmatrix} J = ( 0 1 − 1 0 ) and R ( θ ) = e θ J R(\theta)=e^{\theta J} R ( θ ) = e θ J is the rotation by θ \theta θ . Without the OU forcing and the white noise, the rotation drift alone would carry Y 0 = y Y_0=y Y 0 = y to Y T = R ( ω c T ) y Y_T=R(\omega_cT)y Y T = R ( ω c T ) y . As ω \omega ω increases, the map c ↦ R ( ω c T ) y c \mapsto R(\omega_cT)y c ↦ R ( ω c T ) y becomes more oscillatory and harder to approximate. The hidden process H t H_t H t is stationary and forgets its value at rate κ \kappa κ . Its values therefore remain correlated across consecutive transitions, and this correlation is discarded by a cascade conditioned only on ( c , Y t i ) (c,Y_{t_i}) ( c , Y t i ) .
The strengths β ρ \beta_\rho β ρ and σ ρ \sigma_\rho σ ρ are chosen so that, for fixed q > 0 q>0 q > 0 and 0 ≤ ρ < 1 0\leq\rho<1 0 ≤ ρ < 1 , the hidden OU process contributes ρ q I 2 \rho q I_2 ρq I 2 to the covariance at time T T T and the white noise contributes ( 1 − ρ ) q I 2 (1-\rho)q I_2 ( 1 − ρ ) q I 2 (Appendix § 12 .7). Expressing the dynamics in the rotating coordinates R ( − ω c t ) Y t R(-\omega_ct)Y_t R ( − ω c t ) Y t then gives
Y T ∣ ( Y 0 = y , c ) ∼ N ( R ( ω c T ) y , q I 2 ) = : K T O U ( y , c ) . (6) Y_T \mid(Y_0=y,c) \sim\mathcal{N}(R(\omega_cT)y,qI_2) =: K_T^{\mathrm{OU}}(y,c). \tag*{(6)} Y T ∣ ( Y 0 = y , c ) ∼ N ( R ( ω c T ) y , q I 2 ) =: K T OU ( y , c ) . (6)
The target conditional distribution is thus the same for every ρ \rho ρ : changing ρ \rho ρ changes how uncertainty is carried through time, but not the conditional distribution at time T T T . This lets us study how hidden-state memory affects cascading without also changing the difficulty of the direct forecasting problem.
Over one transition of length δ = T / S \delta=T/S δ = T / S , the exact conditional distribution is Gaussian, Y t i + 1 ∣ ( Y t i = y , c ) ∼ N ( r i R ( ω c δ ) y , q i I 2 ) Y_{t_{i+1}}\mid(Y_{t_i}=y,c)\sim\mathcal{N}(r_iR(\omega_c\delta)y,q_iI_2) Y t i + 1 ∣ ( Y t i = y , c ) ∼ N ( r i R ( ω c δ ) y , q i I 2 ) , with scalars r i r_i r i and q i q_i q i determined by the model. We compare Direct SI and Cascaded SI for an SI whose only approximation is in the conditional mean. It keeps the exact Gaussian form of each transition, with r i r_i r i and q i q_i q i known, and approximates the rotation R ( ω c δ ) y R(\omega_c\delta)y R ( ω c δ ) y by a polynomial p ( c , y ) p(c,y) p ( c , y ) of total degree at most two, which allows us to compute the approximation error explicitly. For this family, minimizing the SI training loss amounts to fitting p p p by least squares, and exact sampling generates N ( r i p ( c , y ) , q i I 2 ) \mathcal{N}(r_i p(c,y),q_iI_2) N ( r i p ( c , y ) , q i I 2 ) (Proposition C.8). Direct SI fits the full rotation with one such polynomial; Cascaded SI fits the rotation over one transition and composes S S S fitted transitions. Since y y y is centered and isotropic, the best fit has the form M ( c ) y M(c)y M ( c ) y with M M M affine in c c c , so the conditional mean of the cascade, A S M ( c ) S y A_S M(c)^S y A S M ( c ) S y with A S = ∏ i r i A_S=\prod_i r_i A S = ∏ i r i , is a polynomial of degree S S S in c c c .
A cascade can therefore represent a more complicated dependence on c c c , even though each individual transition uses the same quadratic approximation class. In our experiments, we follow the same idea. Rather than increasing the capacity of the neural network, we learn shorter transitions and compose them.
We measure the error of the generated distribution K ^ S ( y , c ) \widehat{K}_S(y,c) K S ( y , c ) by (2.2) with the squared Wasserstein-2 metric D = W 2 2 D=W_2^2 D = W 2 2 , that is, by E S ( ρ ) = E Y 0 , c W 2 2 ( K T O U ( Y 0 , c ) , K ^ S ( Y 0 , c ) ) \mathcal{E}_S(\rho)=\mathbb{E}_{Y_0,c}W_2^2(K_T^{\mathrm{OU}}(Y_0,c),\widehat{K}_S(Y_0,c)) E S ( ρ ) = E Y 0 , c W 2 2 ( K T OU ( Y 0 , c ) , K S ( Y 0 , c )) .
For Direct SI, the error is entirely due to approximation, E 1 ( ρ ) = 2 s 0 2 [ 1 − J 0 ( ω T ) 2 − 3 J 1 ( ω T ) 2 ] > 0 \mathcal{E}_1(\rho)=2s_0^2[1-J_0(\omega T)^2-3J_1(\omega T)^2]>0 E 1 ( ρ ) = 2 s 0 2 [ 1 − J 0 ( ω T ) 2 − 3 J 1 ( ω T ) 2 ] > 0 , with j 0 ( ν ) = sin ν / ν j_0(\nu)=\sin\nu/\nu j 0 ( ν ) = sin ν / ν and j 1 ( ν ) = ( sin ν − ν cos ν ) / ν 2 j_1(\nu)=(\sin\nu-\nu\cos\nu)/\nu^2 j 1 ( ν ) = ( sin ν − ν cos ν ) / ν 2 , and it does not depend on ρ \rho ρ . As S → ∞ S\to\infty S → ∞ , the fitted rotation consists of the exact, and only the memory error remains. We denote its limit by C ρ , ∞ C_{\rho,\infty} C ρ , ∞ (Eq. C.56). C ρ , ∞ C_{\rho,\infty} C ρ , ∞ is a consequence of the parameter ρ \rho ρ , the fraction of the endpoint variance carried by the persistent hidden process. It vanishes at ρ = 0 \rho=0 ρ = 0 and grows like ρ 2 \rho^2 ρ 2 for small ρ \rho ρ .
Theorem 3.1. Fix T , κ , q , s 0 > 0 T,\kappa,q,s_0>0 T , κ , q , s 0 > 0 and ω ≠ 0 \omega\ne0 ω = 0 in (3.2). For the SI model defined above, which approximates the transition mean with a quadratic function:
If ρ = 0 \rho=0 ρ = 0 , then E S ( 0 ) = O ( S − 2 ) \mathcal{E}_S(0)=O(S^{-2}) E S ( 0 ) = O ( S − 2 ) as S → ∞ S\to\infty S → ∞ .
For every fixed 0 < ρ < 1 0<\rho<1 0 < ρ < 1 , there is a Λ ρ > 0 \Lambda_\rho>0 Λ ρ > 0 such that E S ( ρ ) = C ρ , ∞ − Λ ρ T / S + O ( S − 2 ) \mathcal{E}_S(\rho)=C_{\rho,\infty}-\Lambda_\rho T/S+O(S^{-2}) E S ( ρ ) = C ρ , ∞ − Λ ρ T / S + O ( S − 2 ) .
There is a ρ ∗ > 0 \rho_*>0 ρ ∗ > 0 such that, for each 0 < ρ < ρ ∗ 0<\rho<\rho_* 0 < ρ < ρ ∗ , a global minimizer over positive integer depths exists at a finite S ∗ ≥ 2 S^*\geq2 S ∗ ≥ 2 , and E S ∗ ( ρ ) < min { E 1 ( ρ ) , C ρ , ∞ } \mathcal{E}_{S^*}(\rho)<\min\{\mathcal{E}_1(\rho),C_{\rho,\infty}\} E S ∗ ( ρ ) < min { E 1 ( ρ ) , C ρ , ∞ } .
The theorem shows that these two effects can produce a finite optimal cascade depth: without hidden forcing, deep cascades become exact, whereas with it, the asymptotic error is nonzero. We further prove that, as ρ ↓ 0 \rho\downarrow0 ρ ↓ 0 , ρ 2 S ∗ ( ρ ) \rho^2 S^*(\rho) ρ 2 S ∗ ( ρ ) converges to a constant proportional to ω 4 \omega^4 ω 4 (Theorem C.13): the weaker the contribution of the hidden process and the faster the rotation, the deeper the optimal cascade. Without the rotation (ω = 0 \omega=0 ω = 0 ), the quadratic fit is exact, and so is Direct SI; for ρ > 0 \rho>0 ρ > 0 , any cascade performs worse, increasingly so with depth in our evaluations (Appendix § 12 .7).
Connection between analysis and experiment. This model shares two key features with the PDE experiments: the dependence of the forecast on the conditioning variable becomes more complicated as the horizon grows, and an unobserved component carries information across transitions. Its dynamics are linear in ( Y t , H t ) (Y_t, H_t) ( Y t , H t ) for fixed c c c , so its behavior requires neither nonlinearity nor chaos. It suggests three things: cascading helps when the forecast depends on the observation in a complicated way; very deep cascades lose accuracy when unobserved components act on the observed ones strongly and persistently, and the stronger they act, the shallower the best cascade; and cascading does not help when the direct forecast is simple, as at ω = 0 \omega= 0 ω = 0 . In the next section we experimentally study how these two effects play out across five PDE systems.
Experimental design
The experimental dynamical systems. We select five physical systems to investigate conditional forecasting under different physical dynamics and forms of unobserved initial uncertainty (Figure 3 and Table 1 ). They comprise nonlinear fluid dynamics, wave propagation, and linear transport. In particular, we include both nonlinear and linear dynamics to investigate whether the benefits of cascading extend beyond systems with nonlinear interactions or chaotic sensitivity to initial conditions. Appendix § 11 .§ 1 gives the equations, initial distributions, numerical solvers, and convergence checks.
Table 1. Properties of the five systems relevant for forecasting. “Independent” means independent of x 0 x_0 x 0 . The nonlinear systems are expected to be chaotic, but we do not show it formally or experimentally. Cascading helps when S ⋆ > 1 S^\star> 1 S ⋆ > 1 .
Figure 3. Experimental systems. Each row shows one observed initial state and four simulated futures from its conditioning ensemble; the futures differ only in their unobserved initial fine scales. The fields shown are the observed vorticity, buoyancy, displacement, displacement, and concentration.
Euler (euler). The vorticity of an inviscid fluid is transported by a velocity field that is determined by the vorticity itself. The resulting nonlinear interactions couple spatial scales, so the unobserved fine-scale vorticity influences the observed coarse evolution.
Surface quasi-geostrophic flow (sqg). The surface buoyancy of a rapidly rotating, stratified fluid is transported in the same way (Held et al., [14 ] ), but the velocity depends more strongly on fine-scale variations of the buoyancy than in Euler flow. The unobserved scales therefore have a stronger influence on the flow that transports the observed ones.
Heterogeneous waves (het-waves). Damped linear waves propagate in a medium with a fixed, spatially varying wave speed, which scatters them. The state comprises the displacement and the velocity. The observed initial displacement consists of the coarse scales of randomly placed shapes on a random background; the unobserved fine-scale displacement is confined to the interiors of the shapes, so its distribution depends on the observation, and the unobserved initial velocity consists of three Gaussian kicks at random locations. The forecaster observes the coarse displacement and a velocity proxy computed from two consecutive displacement snapshots, which is zero at t = 0 t = 0 t = 0 .
Nonlinear Klein–Gordon (nonlinear-kg). A scalar field undergoes dispersive wave propagation with a cubic restoring force. As in the wave system, the state comprises the field and its time derivative, but the forecaster observes the coarse scales of both. The cubic term couples Fourier modes, so the fine scales influence the coarse ones through a local interaction instead of transport.
Passive scalar (passive-scalar). A scalar is advected by a steady cellular flow and diffuses. Its evolution is linear, and its unobserved fine scales are Gaussian and independent of the observation. The conditional distribution of the future is therefore Gaussian, with a mean that depends affinely on the observation and a covariance that does not depend on it (Proposition C.4 ). Among our systems, this is the simplest conditional distribution to learn.
Data. For each system, we generate 80 training, 10 validation, and 10 test conditioning ensembles, with distinct observed initial states. Each training ensemble contains 200 trajectories stored at 41 equally spaced times to accommodate different cascade depths. An important feature of our benchmark is that the test datasets are large: each test observation comes with 10,000 reference futures, which allows statistically meaningful comparisons of conditional distributions.
Forecasters. Direct SI learns a single conditional stochastic interpolant from X 0 X_0 X 0 to X T X_T X T (Chen et al., [9 ] ). It connects the two states by a stochastic path in an auxiliary time τ ∈ [ 0 , 1 ] \tau\in[0, 1] τ ∈ [ 0 , 1 ] , learns the drift of this path by regression, and generates a forecast by integrating the resulting stochastic differential equation from X 0 X_0 X 0 . Cascaded SI learns the transitions from X t i X_{t_i} X t i to X t i + 1 X_{t_{i+1}} X t i + 1 in the same way, with one network shared by all transitions and conditioned on their physical start and end times. EDM (Karras et al., [16 ] ), a strong diffusion-model baseline, is trained on the same endpoint pairs as Direct SI. For each system, all methods use the same U-Net, with channel widths 64, 128, and 256 and circular convolutions, acting on a 32 × 32 32 \times32 32 × 32 reconstruction of the observed modes. Hyperparameters, checkpoints, and the cascade depth S ⋆ ∈ { 1 , 2 , 5 , 10 , 20 , 40 } S^\star\in\{1, 2, 5, 10, 20, 40\} S ⋆ ∈ { 1 , 2 , 5 , 10 , 20 , 40 } are selected on the validation ensembles. by the conditional energy distance [34 ] , and the number of sampler steps is calibrated on validation data. The test ensembles are used only for the final evaluation (Appendix § 11 .§ 2 ).
A fixed number of training constraints. A single trajectory provides one endpoint pair for Direct SI or EDM, but S S S transition pairs for a cascade of depth S S S . To compare learning the forecast horizon directly with learning it through shorter transitions, we therefore fix the number of supervised pairs at B = 16,000 B = 16{,}000 B = 16 , 000 : Direct SI and EDM use one endpoint pair from each of the 200 trajectories of every training ensemble, and Cascaded SI at depth S S S uses all S S S transitions of 200/S S S trajectories per ensemble. Every run uses 30,000 optimizer updates. A deeper cascade receives no additional pairs or updates, it is trained on fewer distinct trajectories, and its forecasts require S S S sequential sampler integrations. We emphasize that this is a harsh constraint. In this way we equalize the number of training constraints but not the variety of the training data, whereas in reality the intermediate observations are often simply available. Our goal is to ensure that we focus on the tradeoff between approximation and Markovianity. On Euler, increasing the number of distinct training observations by a factor of 10 at a fixed number of supervised pairs improves Direct SI, but its csFID-10k remains nearly three times that of Cascaded SI. Cascading also helps when both forecasters are trained on a single trajectory per initial observation (Appendices B.6.1–B.6.2).
Evaluation. For each test observation x 0 ( e ) x_0^{(e)} x 0 ( e ) , we draw N = 10,000 N = 10{,}000 N = 10 , 000 samples from the forecaster and compare them with the N N N reference samples of the simulator. Because a forecast can reproduce realistic local patterns at the wrong locations (Figure 2 ), we use spatial FID ([24 ] ). Unlike the standard FID ([15 ] ), the features of spatial FID are sensitive to the spatial layout of the images. We verify that this makes the metric more sensitive to spatial misalignment on our data: for example, it penalizes a simple flip of the Euler fields much more strongly than standard FID (Appendix B.4). With ( m e , C e ) (m_e, C_e) ( m e , C e ) and ( m ^ e , C ^ e ) (\hat{m}_e, \hat{C}_e) ( m ^ e , C ^ e ) the means and covariances of ϕ \phi ϕ over the reference and the generated samples, we compute for each test observation the Fréchet distance
d e = ∥ m e − m ^ e ∥ 2 + tr [ C e + C ^ e − 2 ( C e 1 / 2 C ^ e C e 1 / 2 ) 1 / 2 ] . (7) d_e = \lVert m_e - \hat{m}_e \rVert^2 + \operatorname{tr}\left[C_e + \hat{C}_e - 2\left(C_e^{1/2}\hat{C}_e C_e^{1/2}\right)^{1/2}\right]. \tag*{(7)} d e = ∥ m e − m ^ e ∥ 2 + tr [ C e + C ^ e − 2 ( C e 1/2 C ^ e C e 1/2 ) 1/2 ] . (7)
an instance of D D D in (2) . We report its average over the ten test observations, which we call csFID-10k, with the standard error of the mean (SEM) over the test observations, for one training seed. For the two wave systems, the features are computed from the displacement (Appendices B.3–B.4).
Results
Cascading recovers the conditional distribution. The last row of Figure 2 shows the cascaded forecast for the same Euler observation as the other rows. The cascaded forecast correctly captures the spatial structure of the reference conditional mean and the regions of high and low standard deviation, which the direct forecasts fail to reproduce. The same holds for heterogeneous waves, whose direct forecasts are far too variable (Appendix B.5). Table 2 quantifies the improvement over all test observations. On Euler, Cascaded SI at the validation-selected depth S ∗ = 10 S^* = 10 S ∗ = 10 reduces csFID-10k by a factor of about 3.5 relative to both Direct SI and EDM, and it is better than both on each of the ten test observations. We make the following observations:
Table 2. Conditional forecasting error. Test csFID-10k (lower is better), mean ± SEM over the ten test observations, for one training seed. S ∗ S^* S ∗ is selected on validation data; for the passive scalar, S ∗ = 1 S^* = 1 S ∗ = 1 , so Direct SI and Cascaded SI are the same fitted model. †The dagger signifies that the best value of one hyperparameter lay at the edge of the searched range (Appendix B.2).
Cascading helps in all three nonlinear systems. The improvement on Euler extends to SQG and Klein–Gordon, by factors of 2.3 and 5.7 relative to the better direct forecaster.
Neither nonlinearity nor chaos is necessary for cascading to help. The heterogeneous-wave dynamics are linear and nonchaotic, and the conditional mean is affine in the observation, yet Cascaded SI reduces csFID-10k by more than a factor of three relative to both Direct SI and EDM. The rotation model of Section 3 is linear as well.
For the passive scalar, cascading is detrimental. Validation selects S ∗ = 1 S^* = 1 S ∗ = 1 , and every added stage increases the error, from 8.7 at S ∗ = 1 S^* = 1 S ∗ = 1 to 75 at S = 40 S = 40 S = 40 (Figure 4 ); EDM achieves the lowest score. This is what the rotation model does at ω = 0 \omega= 0 ω = 0 : the direct forecast is simple, and a cascade has little to gain. The passive scalar is linear like the waves, so linearity alone does not decide the outcome. Its conditional covariance does not depend on the observation, whereas that of the waves does, through the masked fine scales; the contrast suggests that this dependence is what the direct forecasters cannot capture.
Figure 4. Intermediate-depth cascades improve conditional forecasting in four of five systems. Test csFID-10k versus cascade depth S S S (values in Table 7 ). S = 1 S = 1 S = 1 is Direct SI; outlined markers: validation-selected S ⋆ S^{\star} S ⋆ ; dashed lines: EDM; error bars: ± 1 \pm1 ± 1 SEM over the ten test observations.
Whether cascading is beneficial therefore cannot be explained simply by the underlying physical dynamics. The results point instead to the difficulty of learning the conditional forecast distribution over the full horizon, which depends on the dynamics, the observation, and the distribution of the unobserved scales. This difficulty is reflected in the generalization behavior of the forecasters. In the four systems where cascading helps, continued training improves Direct SI’s fit to the training observations but worsens its forecasts on unseen observations. Cascaded SI achieves substantially lower test errors, whereas on the passive scalar both direct forecasters generalize well (Appendix § 11 .6.3).
The best depth is finite and > 1 > 1 > 1 . Figure 4 shows the test error at each cascade depth S S S , with the model, checkpoint, and sampler resolution selected on validation data separately for each depth. For the other four systems, the best test performance occurs at an intermediate depth: S = 5 S = 5 S = 5 for heterogeneous waves and SQG, S = 10 S = 10 S = 10 for Euler, and S = 20 S = 20 S = 20 for Klein–Gordon (for SQG, validation selects S ⋆ = 10 S^{\star} = 10 S ⋆ = 10 ). Shorter transitions are thus not always better, and the preferred depth depends on the forecasting problem and horizon (Appendix § 11 .6.4). Under the fixed budget of 16,000 supervised transitions and 30,000 optimizer updates, however, deeper cascades are also trained on fewer distinct trajectories, integrate their samplers more often, and compound learning errors, so the deterioration at large depths need not be caused by memory loss alone. Indeed, additionally conditioning every transition on X 0 X_0 X 0 reduces the forecasting error on the passive scalar but not on Euler (Appendix § 11 .6.5). In both systems, the error still increases at large depths, consistent with our analysis of learned anchored transitions in Appendix § 12 .8.
The optimal depths seem to agree with the theory. Among other differences, the fine scales drive the velocity more strongly in SQG than in Euler flow, and on SQG we find that a shallower cascade is better, S = 5 S = 5 S = 5 against S = 10 S = 10 S = 10 ; the optimal gain is similarly smaller (2.3 against 3.5). In the rotation model, a stronger contribution of the hidden process yields a shallower optimum, although we have not measured the memory of the two systems. The wave system illustrates the relevance of how the intermediate states in the cascades are structured. Composing deterministic next-state regressors that use the displacement alone, but not the velocity, reduces the correlation with the reference conditional mean from 0.73 after one transition to 0.11 after five; including the velocity proxy, initialized at zero as in the benchmark, improves it from 0.65 to 0.87 (Table 4 in Appendix § 11 .1.3).
Table 4. Wave state diagnostic. Mean correlation with the true conditional endpoint mean over ten validation conditions. Both augmented columns use the same fitted model. The benchmark initializes the proxy to zero and subsequently carries generated values.
Conclusion
We studied why generative forecasters can sample the wrong distribution of future states given coarse observations of a physical system, and when cascading transitions over shorter time intervals helps. In our experiments, cascaded SIs recovered conditional distributions when direct forecasters failed to do so, for both linear and nonlinear dynamics, but cascading was detrimental for a scalar passively advected and diffused by a prescribed flow. Our analytical model explains both effects by the trade-off between the approximation error of long transitions and the memory error of repeated conditioning on a coarse-grained state.
Open questions. The theory does not predict the depth selected in the PDE experiments; this will require further work, for example measuring how strongly and persistently the unobserved scales drive the observed ones. Cascading increases inference cost, and its benefit may depend on model capacity.
Code and Data Availability
We have released the benchmark datasets, our code, and trained model checkpoints.
Acknowledgments
We gratefully acknowledge support from the European Research Council Consolidator Grant 101232533 (PhaseShift). We also acknowledge computing resources provided by sciCORE, the scientific computing center of the University of Basel, and by the Swiss National Supercomputing Centre (CSCS).
References
Related work
Generative models for probabilistic forecasting. Recent work has used conditional diffusion models [5 , 4 ] and stochastic interpolants [2 ] to represent distributions over future states rather than produce a single deterministic forecast. Chen et al. [9 ] formulate forecasting directly as sampling from the conditional distribution of a future state given the current state, using stochastic interpolants to define a generative path between them. Related diffusion-based approaches have been developed for weather forecasting, including SEEDS [18 ] and GenCast [28 ] , and for complex physical systems and turbulent flows [17 , 23 ] . Our work considers the same probabilistic forecasting objective, but focuses on how the physical time interval used for learning affects the accuracy of the resulting conditional distribution.
Using intermediate times and composing forecasts. Several forecasting methods make use of intermediate physical states or generate trajectories by repeatedly applying learned transitions. GenCast, for example, learns a conditional distribution over a fixed physical time step and samples successive states autoregressively [28 ] . Conditional diffusion models have similarly been applied autoregressively to turbulent-flow simulation [17 ] . Deterministic weather models have compared step lengths: Pangu-Weather composes models trained for several lead times so as to keep the number of steps small, since errors accumulate with every step [6 ] , and Stormer averages forecasts made with different step lengths [25 ] . For conditional diffusion models of PDEs, Shysheya et al. [32 ] find that sampling a trajectory autoregressively forecasts better than sampling it all at once. DYffusion [31 ] is particularly related to our setting, since it uses intermediate physical times to build a dynamics-informed diffusion process. These works show that intermediate temporal information can be useful for forecasting, and they evaluate forecasts against single realized futures. We instead compare learning P ( X T ∣ X 0 ) P(X_T \mid X_0) P ( X T ∣ X 0 ) directly with learning and composing transitions over intervals of length T / S T/S T / S , at a fixed final time T T T and against reference ensembles of the conditional distribution. We vary S S S while keeping the number of supervised training pairs fixed. This exposes a non-monotone dependence on cascade depth: making the transitions shorter can improve the forecast, but dividing the interval increasingly finely need not continue to help, and for a passive scalar every added stage makes the forecast worse.
Iterated and direct multi-step forecasts. In time-series analysis, the choice between iterating a one-step model and estimating a separate model for each horizon has long been studied [22 , 10 ] . For linear autoregressions, it is a trade-off between estimation variance and bias: iterated forecasts are more efficient when the one-step model is correctly specified, while direct forecasts are more robust to misspecification. The two effects in our analysis play analogous roles: a transition over a short interval is easier to approximate than one over the full horizon, which is the approximation effect, and a one-step model of the observed process is misspecified whenever this process is not Markov, which is the memory effect. We study these effects for generative forecasts of the full conditional distribution of high-dimensional physical fields, made with models of limited capacity, rather than for point forecasts with linear models.
Partial observation and memory. The second effect in our analysis is specific to forecasting from incomplete observations. Projection of a dynamical system onto a subset of its variables generally produces dynamics with memory: unresolved variables can retain information about the past that is not contained in the current observation. This phenomenon is classical in optimal prediction and the Mori–Zwanzig formalism [11 , 12 ] . It has a direct consequence for cascaded probabilistic forecasts. Even if each short transition P ( X t i + 1 ∣ X t i ) P(X_{t_{i+1}} \mid X_{t_i}) P ( X t i + 1 ∣ X t i ) were known exactly, their composition need not equal P ( X T ∣ X 0 ) P(X_T \mid X_0) P ( X T ∣ X 0 ) when the observed process is not Markov. Our theoretical analysis makes this loss of information explicit and studies its competition with the approximation error of learning longer transitions. In the rotation–OU model, these two effects produce a finite optimal cascade depth.
Evaluating conditional forecast distributions. Evaluating generative models is difficult because realistic individual samples do not by themselves imply that the correct distribution has been learned [35 ] . Probabilistic forecasting methods therefore use ensemble-based scores, distributional statistics, and diagnostics of forecast uncertainty [18 , 28 , 17 ] . Our benchmark is designed specifically for conditional evaluation under partial observation. For each observed initial state x 0 x_0 x 0 , we generate many full initial states that share x 0 x_0 x 0 but differ in their hidden components, and evolve each of them with the physical simulator. The resulting reference ensemble gives direct samples from P ( X T ∣ X 0 = x 0 ) P(X_T \mid X_0 = x_0) P ( X T ∣ X 0 = x 0 ) . This allows us to distinguish a model that generates plausible states from one that recovers the conditional distribution associated with the particular observation supplied to the forecaster.
Experiments
Dataset construction and numerical solvers
For each observed initial state, we simulate several trajectories by varying the hidden initial conditions. We call this set of trajectories an ensemble . All trajectories in an ensemble therefore start from the same observation but can reach different final states (Figure 3 ).
The benchmark uses 80 ensembles for training, 10 for validation, and 10 for testing, with different observed initial states in each split. Each test ensemble provides 10,000 simulated endpoints for comparison with the model’s predictions.
The model observes only Fourier modes 0 < ∣ k ∣ ≤ 8 0 < |k| \le8 0 < ∣ k ∣ ≤ 8 , represented by 196 real coordinates per scalar field. Table 3 summarizes the simulation settings and observed variables.
Table 3. Benchmark dataset specifications.
Table 3 : Benchmark dataset specifications. Parentheses give the number of real coordinates observed by the model. Each benchmark trajectory contains 41 equally spaced snapshots. Horizons are reported in physical time.
Two-dimensional Euler
On the unit periodic square, vorticity evolves according to
∂ t ω + u ⋅ ∇ ω = 0 , u = ∇ ⊥ Δ − 1 ω . (8) \partial_t \omega+ u \cdot\nabla\omega= 0, \qquad u = \nabla^\perp\Delta^{-1}\omega. \tag*{(8)} ∂ t ω + u ⋅ ∇ ω = 0 , u = ∇ ⊥ Δ − 1 ω . (8)
Here ∇ ⊥ = ( − ∂ y , ∂ x ) \nabla^\perp= (-\partial_y, \partial_x) ∇ ⊥ = ( − ∂ y , ∂ x ) . The initial field has independently drawn low and high Fourier bands. Filtered Gaussian white noise with spectral amplitude proportional to ∣ k ∣ − 2 |k|^{-2} ∣ k ∣ − 2 on 1 < ∣ k ∣ ≤ 8 1 < |k| \le8 1 < ∣ k ∣ ≤ 8 determines the condition; an independent draw with amplitude proportional to ∣ k ∣ − 3 |k|^{-3} ∣ k ∣ − 3 on 8 < ∣ k ∣ ≤ 64 8 < |k| \le64 8 < ∣ k ∣ ≤ 64 is added for each member. The bands are normalized separately to velocity RMS values 1 and 0.2.
We use the double-precision azeban pseudospectral solver Rohner & Mishra [29 ] , with a 1024 2 1024^2 102 4 2 grid, 1536 2 1536^2 153 6 2 padding, smooth-cutoff spectral viscosity (s = 1 s = 1 s = 1 , ϵ = 0.4 \epsilon= 0.4 ϵ = 0.4 ), and adaptive SSP–RK3 integration at CFL 0.5. The 41 states are archived every 0.025 through T = 1 T = 1 T = 1 . A comparison of 1024 2 1024^2 102 4 2 and 2048 2 2048^2 204 8 2 simulations for one initial-condition ensemble supports distributional convergence of the observed endpoint states at the benchmark resolution.
Surface quasi-geostrophic flow
On the periodic square [ 0 , 2 π ) 2 [0, 2\pi)^2 [ 0 , 2 π ) 2 , surface buoyancy obeys
∂ t b + J ( ψ , b ) = 0 , ψ ^ ( k ) = b ^ ( k ) ∣ k ∣ ( k ≠ 0 ) . (9) \partial_t b + J(\psi,b) = 0, \qquad\widehat{\psi}(k) = \frac{\widehat{b}(k)}{|k|} \quad(k \ne0). \tag*{(9)} ∂ t b + J ( ψ , b ) = 0 , ψ ( k ) = ∣ k ∣ b ( k ) ( k = 0 ) . (9)
Here J ( f , g ) = ∂ x f ∂ y g − ∂ y f ∂ x g J(f,g) = \partial_x f\,\partial_y g - \partial_y f\,\partial_x g J ( f , g ) = ∂ x f ∂ y g − ∂ y f ∂ x g . Each condition contains one to four signed elliptical vortices, with uniform centers and orientations, amplitude magnitudes in [ 0.7 , 1.2 ] [0.7, 1.2] [ 0.7 , 1.2 ] , aspect ratios in [ 1.5 , 4 ] [1.5, 4] [ 1.5 , 4 ] , and widths in [ 2 π / 9 , 2 π / 6 ] [2\pi/9, 2\pi/6] [ 2 π /9 , 2 π /6 ] . Centers are separated by at least 1.2. Projection onto 0 < ∣ k ∣ ≤ 8 0 < |k| \le8 0 < ∣ k ∣ ≤ 8 defines the observed initial state. Each ensemble member adds a real Gaussian Fourier field on 8 < ∣ k ∣ ≤ 32 8 < |k| \le32 8 < ∣ k ∣ ≤ 32 , with coefficient variance proportional to ∣ k ∣ − 5 / 3 |k|^{-5/3} ∣ k ∣ − 5/3 , normalized to spatial RMS 0.10.
We use pyqg.SQGModel (Abernathey et al. [1 ] ) (version 0.7.3.dev23, commit 9abad33), with f 0 = N = H = 1 f_0 = N = H = 1 f 0 = N = H = 1 , β = 0 \beta= 0 β = 0 , and no linear drag. The solver uses a 256 2 256^2 25 6 2 Fourier grid, third-order Adams–Bashforth stepping with Δ t = 0.005 \Delta t = 0.005 Δ t = 0.005 , and its built-in high-wavenumber filter, which dissipates buoyancy variance. We archive 41 states every 0.5 through T = 20 T = 20 T = 20 and rescale forecast time to [ 0 , 1 ] [0, 1] [ 0 , 1 ] . A comparison of 256 2 256^2 25 6 2 and 512 2 512^2 51 2 2 simulations, with the time step halved at the finer resolution, supports distributional convergence of the observed endpoint states.
Heterogeneous waves
Displacement u u u and velocity v v v evolve on the unit periodic square:
∂ t u = v , ∂ t v = ∇ ⋅ ( c 2 ∇ u ) − 0.2 v , c 2 ( x , y ) = 1 + 0.5 sin ( 2 π x ) sin ( 2 π y ) . (10) \partial_t u = v,\qquad\partial_t v = \nabla\cdot(c^2\nabla u) - 0.2v,\qquad c^2(x,y) = 1 + 0.5\sin(2\pi x)\sin(2\pi y). \tag*{(10)} ∂ t u = v , ∂ t v = ∇ ⋅ ( c 2 ∇ u ) − 0.2 v , c 2 ( x , y ) = 1 + 0.5 sin ( 2 π x ) sin ( 2 π y ) . (10)
The medium is fixed. Each condition combines an anisotropic Gaussian background (correlation lengths 0.06 and 0.03, scale 0.2) with 18 random boxes, triangles, or disks. Shapes overwrite earlier values where they overlap; centers and orientations are uniform, sizes lie in [ 0.08 , 0.20 ] [0.08, 0.20] [ 0.08 , 0.20 ] , and amplitudes in [ − 3 , 3 ] [-3, 3] [ − 3 , 3 ] . Smoothing with Gaussian width 2 / 128 2/128 2/128 and projection onto 0 < ∣ k ∣ ≤ 8 0 < |k| \le8 0 < ∣ k ∣ ≤ 8 define the shared initial displacement.
Each ensemble member adds masked Gaussian noise projected onto 8 < ∣ k ∣ ≤ 16 8 < |k| \le16 8 < ∣ k ∣ ≤ 16 . The mask depends on the condition’s shape interiors; before masking, the band filter gives expected spatial mean square 3.3 2 3.3^2 3. 3 2 . Initial velocity consists of three Gaussian kicks with uniform centers, width 0.03, and independent strengths N ( 0 , 165 2 ) \mathcal{N}(0, 165^2) N ( 0 , 16 5 2 ) , projected onto ∣ k ∣ > 8 |k| > 8 ∣ k ∣ > 8 . Both perturbations therefore leave the observed initial state unchanged.
A double-precision PyTorch solver uses centered spatial differences and RK4 on a 512 2 512^2 51 2 2 grid with Δ t = 2.5 × 10 − 4 \Delta t = 2.5 \times10^{-4} Δ t = 2.5 × 1 0 − 4 . We archive 41 states every 0.025 through T = 1 T = 1 T = 1 . The observed state combines coarse displacement with the velocity proxy
v ^ t n = P 8 u t n − u t n − 1 0.025 ( n > 0 ) , v ^ 0 = 0 , (11) \hat{v}_{t_n} = P_8\frac{u_{t_n} - u_{t_{n-1}}}{0.025}\quad(n > 0),\qquad\hat{v}_0 = 0, \tag*{(11)} v ^ t n = P 8 0.025 u t n − u t n − 1 ( n > 0 ) , v ^ 0 = 0 , (11)
where P 8 P_8 P 8 extracts the 196 real observed Fourier coordinates. Training targets use this proxy; during generation, the model predicts both channels and passes them to the next segment. An independent py-pde implementation (Zwicker [36 ] ) verifies the spatial operator. Time-step and grid-refinement checks support numerical accuracy of the observed states.
Wave state diagnostic. We compare deterministic regressors using displacement alone or the augmented state ( u , v ^ ) (u,\hat{v}) ( u , v ^ ) . They have nearly equal parameter counts and use mean-squared error, 8,000 training updates, and learning rates 3 × 10 − 4 3 \times10^{-4} 3 × 1 0 − 4 and 10 − 3 10^{-3} 1 0 − 3 , with checkpoint selection on eight development conditions. Each depth receives B = 16 , 000 B = 16,000 B = 16 , 000 supervised transitions. We recursively compose the predictions and measure their correlation with the simulated conditional endpoint mean on ten validation conditions (Table 4 ). The supplied initial proxy uses a forward difference and therefore gives additional initial information. Setting that proxy to zero at inference tests the same fitted augmented model without this information.
Table 4 : Wave state diagnostic. Mean correlation with the true conditional endpoint mean over ten validation conditions. Both augmented columns use the same fitted model. The benchmark initializes the proxy to zero and subsequently carries generated values.
Nonlinear Klein–Gordon
On the periodic square [ 0 , 2 π ) 2 [0, 2\pi)^2 [ 0 , 2 π ) 2 , displacement and velocity obey the defocusing nonlinear Klein–Gordon equation:
∂ t u = v , ∂ t v = Δ u − u − 64 u 3 . (12) \partial_t u = v,\qquad\partial_t v = \Delta u - u - 64u^3. \tag*{(12)} ∂ t u = v , ∂ t v = Δ u − u − 64 u 3 . (12)
Each condition consists of two travelling Gaussian wave packets, projected onto 0 < ∣ k ∣ ≤ 8 0 < |k| \le8 0 < ∣ k ∣ ≤ 8 in both fields. Centers, directions, and phases are uniform; amplitudes lie in [ 0.5 , 1 ] [0.5, 1] [ 0.5 , 1 ] , carrier wavenumber magnitudes in [ 3 , 5 ] [3, 5] [ 3 , 5 ] , and the envelope width is 0.8 0.8 0.8 . Each ensemble member adds hidden displacement and velocity on 8 < ∣ k ∣ ≤ 24 8 < |k| \le24 8 < ∣ k ∣ ≤ 24 . Their Fourier coefficients are proportional to g k 1 + ∣ k ∣ 2 \frac{g_k}{\sqrt{1+|k|^2}} 1 + ∣ k ∣ 2 g k and g k ′ g'_k g k ′ , respectively, with independent complex Gaussian draws and reality enforced. Both hidden fields are scaled by the same member-specific factor to give displacement RMS 0.20 0.20 0.20 . We observe 196 real Fourier coordinates of u u u and 196 of v / 4 v/4 v /4 .
Dedalus 3.0.5 (Burns et al. [8 ] ) uses a 128 2 128^2 12 8 2 Fourier grid, factor-two dealiasing for the cubic term, and the third-order IMEX scheme RK443 with Δ t = 0.0025 \Delta t = 0.0025 Δ t = 0.0025 . States are archived every 0.1 0.1 0.1 ; retaining every third state gives 41 snapshots through T = 12 T = 12 T = 12 . Spatial and time-step refinement checks support numerical convergence.
Passive scalar
On the unit periodic square, concentration evolves under a fixed flow:
∂ t q + v ⋅ ∇ q = κ Δ q , (13) \partial_t q + v \cdot\nabla q = \kappa\Delta q, \tag*{(13)} ∂ t q + v ⋅ ∇ q = κ Δ q , (13)
v ( x , y ) = ( V sin ( 2 π K x ) cos ( 2 π K y ) , − V cos ( 2 π K x ) sin ( 2 π K y ) ) , (14) v(x,y) = \left(V\sin(2\pi Kx)\cos(2\pi Ky), -V\cos(2\pi Kx)\sin(2\pi Ky)\right), \tag*{(14)} v ( x , y ) = ( V sin ( 2 π K x ) cos ( 2 π K y ) , − V cos ( 2 π K x ) sin ( 2 π K y ) ) , (14)
with K = 2 K = 2 K = 2 , V = 0.5 V = 0.5 V = 0.5 , and κ = 6.184154 × 10 − 6 \kappa= 6.184154 \times10^{-6} κ = 6.184154 × 1 0 − 6 . Each condition is drawn on a 256 2 256^2 25 6 2 generator grid from Gaussian white noise filtered with amplitude ∣ k ∣ − 2 |k|^{-2} ∣ k ∣ − 2 on 1 < ∣ k ∣ ≤ 8 1 < |k| \le8 1 < ∣ k ∣ ≤ 8 and normalized to spatial RMS 1. Each ensemble member adds independent Gaussian noise filtered with amplitude ∣ k ∣ − 3 |k|^{-3} ∣ k ∣ − 3 on 8 < ∣ k ∣ ≤ 64 8 < |k| \le64 8 < ∣ k ∣ ≤ 64 . A fixed multiplier gives expected spatial mean square 1.14491, without normalizing individual members. The hidden field is therefore Gaussian, and linear evolution preserves a Gaussian conditional endpoint distribution.
A double-precision Fourier–Galerkin solver uses integrating-factor RK4 on a 512 2 512^2 51 2 2 grid with 768 2 768^2 76 8 2 padding and Δ t = 0.005 / 6 \Delta t = 0.005/6 Δ t = 0.005/6 . We archive states every 0.005 0.005 0.005 through T = 1 T = 1 T = 1 and retain every fifth state for the 41-snapshot benchmark. Independent simulations of five trajectories with Dedalus (Burns et al. [8 ] ) agree with the observed dataset states as the Dedalus time step is refined.
Training, selection, and numerical sampling
Supervision and architecture. The matched-budget comparison uses 80 canonical training conditions and 200 candidate trajectories per condition. At depth S ∈ { 1 , 2 , 5 , 10 , 20 , 40 } S \in\{1, 2, 5, 10, 20, 40\} S ∈ { 1 , 2 , 5 , 10 , 20 , 40 } , 200 / S 200/S 200/ S trajectories supply all S S S consecutive transitions, keeping the budget at B = 16,000 B = 16{,}000 B = 16 , 000 . Direct SI is the S = 1 S = 1 S = 1 member of this family. Normalization is fitted on training data and shared across depths and methods.
The U-Net acts on 32 × 32 32 \times32 32 × 32 reconstructions of the observed modes, with channel widths 64, 128, 256, circular 3 × 3 3 \times3 3 × 3 convolutions, SiLU activations, two average-pooling stages, and skip connections. Its output is projected back onto the observed modes. SI receives the interpolated state, conditioning state, auxiliary time, and physical segment start/end times. Euler uses 1,515,265 parameters. Passive scalar and waves also receive spatial-coordinate features because their flow or medium is spatially inhomogeneous. The wave model predicts both displacement and velocity-proxy channels.
Objectives and samplers. For a normalized pair ( a , b ) (a,b) ( a , b ) , draw τ ∼ Unif [ 0 , 1 ] \tau\sim\operatorname{Unif}[0,1] τ ∼ Unif [ 0 , 1 ] and z ∼ N ( 0 , I ) z \sim\mathcal{N}(0,I) z ∼ N ( 0 , I ) , and set
I τ = ( 1 − τ ) a + τ b + σ ( 1 − τ ) τ z , R τ = b − a − σ τ z . (15) I_\tau= (1-\tau)a + \tau b + \sigma(1-\tau)\sqrt{\tau}\,z,\qquad R_\tau= b-a-\sigma\sqrt{\tau}\,z. \tag*{(15)} I τ = ( 1 − τ ) a + τ b + σ ( 1 − τ ) τ z , R τ = b − a − σ τ z . (15)
The drift regresses R τ R_\tau R τ from ( I τ , τ , a , t i , t i + 1 ) (I_\tau,\tau,a,t_i,t_{i+1}) ( I τ , τ , a , t i , t i + 1 ) using the sum of 1 2 ∥ b θ ∥ 2 − b θ ⋅ R τ \frac{1}{2}\|b_\theta\|^2-b_\theta\cdot R_\tau 2 1 ∥ b θ ∥ 2 − b θ ⋅ R τ for antithetic draws z , − z z,-z z , − z . Euler–Maruyama samples
d Z τ = b θ ( Z τ , τ , a , t i , t i + 1 ) d τ + σ ( 1 − τ ) d W τ , Z 0 = a . (16) dZ_\tau= b_\theta(Z_\tau,\tau,a,t_i,t_{i+1})\,d\tau+ \sigma(1-\tau)\,dW_\tau,\qquad Z_0=a. \tag*{(16)} d Z τ = b θ ( Z τ , τ , a , t i , t i + 1 ) d τ + σ ( 1 − τ ) d W τ , Z 0 = a . (16)
At the next physical segment, a a a is replaced by the generated endpoint. SI gradient norms are clipped at 1.
EDM uses the conditional denoising objective and preconditioning of Karras et al. [16 ] , with σ d a t a = 1 \sigma_{\mathrm{data}} = 1 σ data = 1 , P s t d = 1.2 P_{\mathrm{std}} = 1.2 P std = 1.2 , and a searched log-noise location P m e a n P_{\mathrm{mean}} P mean . Its deterministic Heun sampler uses σ min = 0.002 \sigma_{\min} = 0.002 σ m i n = 0.002 , σ max = 80 \sigma_{\max} = 80 σ m a x = 80 , and schedule exponent 7; n n n numerical steps require 2 n − 1 2n-1 2 n − 1 network evaluations. EDM uses no gradient clipping or learning-rate ramp-up.
Optimization and selection. Models train for 30,000 Adam updates with batch size 256, constant learning rate, no weight decay, ( β 1 , β 2 ) = ( 0.9 , 0.999 ) (\beta_1, \beta_2) = (0.9, 0.999) ( β 1 , β 2 ) = ( 0.9 , 0.999 ) , and ϵ = 10 − 8 \epsilon= 10^{-8} ϵ = 1 0 − 8 . Training uses float32, without mixed precision or TF32. Exponential-moving-average weights have a 500,000-example half-life and ramp-up ratio 0.05 and are used for validation and test. Checkpoints are scored every 1,000 updates and at updates 100, 200, 300, 500, and 750.
Checkpoint and hyperparameter selection minimize the mean validation energy-distance U-statistic [34 ] ; S ⋆ S^\star S ⋆ minimizes it across depths. For generated states A = { a i } i = 1 n A = \{a_i\}_{i=1}^{n} A = { a i } i = 1 n and true states R = { r j } j = 1 m R = \{r_j\}_{j=1}^{m} R = { r j } j = 1 m , we use
E D ^ U ( A , R ) = 2 n m ∑ i , j ∥ a i − r j ∥ − 1 n ( n − 1 ) ∑ i ≠ i ′ ∥ a i − a i ′ ∥ − 1 m ( m − 1 ) ∑ j ≠ j ′ ∥ r j − r j ′ ∥ . (17) \widehat{\mathrm{ED}}_U(A,R) = \frac{2}{nm}\sum_{i,j}\lVert a_i-r_j\rVert- \frac{1}{n(n-1)}\sum_{i\ne i'}\lVert a_i-a_{i'}\rVert- \frac{1}{m(m-1)}\sum_{j\ne j'}\lVert r_j-r_{j'}\rVert. \tag*{(17)} ED U ( A , R ) = nm 2 i , j ∑ ∥ a i − r j ∥ − n ( n − 1 ) 1 i = i ′ ∑ ∥ a i − a i ′ ∥ − m ( m − 1 ) 1 j = j ′ ∑ ∥ r j − r j ′ ∥ . (17)
For sampler-resolution calibration, we instead use the corresponding V-statistic
E D ^ V ( A , R ) = 2 n m ∑ i , j ∥ a i − r j ∥ − 1 n 2 ∑ i , i ′ ∥ a i − a i ′ ∥ − 1 m 2 ∑ j , j ′ ∥ r j − r j ′ ∥ , (18) \widehat{\mathrm{ED}}_V(A,R) = \frac{2}{nm}\sum_{i,j}\lVert a_i-r_j\rVert- \frac{1}{n^2}\sum_{i,i'}\lVert a_i-a_{i'}\rVert- \frac{1}{m^2}\sum_{j,j'}\lVert r_j-r_{j'}\rVert, \tag*{(18)} ED V ( A , R ) = nm 2 i , j ∑ ∥ a i − r j ∥ − n 2 1 i , i ′ ∑ ∥ a i − a i ′ ∥ − m 2 1 j , j ′ ∑ ∥ r j − r j ′ ∥ , (18)
which includes self-pairs and is more stable for relative convergence checks near zero.
Selection uses 100 true and 100 generated states per condition in normalized model coordinates, with float64 accumulation. Exact ties select the earlier checkpoint. One training seed is used; sampling random numbers are shared across configurations within each condition.
Initial learning rates are { 10 − 4 , 3 × 10 − 4 , 10 − 3 } \{10^{-4}, 3 \times10^{-4}, 10^{-3}\} { 1 0 − 4 , 3 × 1 0 − 4 , 1 0 − 3 } , SI noise scales are { 0.25 , 0.5 , 1 , 2 , 4 , 8 , 16 , 32 } \{0.25, 0.5, 1, 2, 4, 8, 16, 32\} { 0.25 , 0.5 , 1 , 2 , 4 , 8 , 16 , 32 } , and EDM uses P m e a n ∈ { − 3.2 , − 2.8 , … , − 0.4 } P_{\mathrm{mean}} \in\{-3.2, -2.8, \ldots, -0.4\} P mean ∈ { − 3.2 , − 2.8 , … , − 0.4 } . One boundary-extension round is allowed. At the lower or upper edge, learning rates extend by { 3 × 10 − 5 , 10 − 5 } \{3 \times10^{-5}, 10^{-5}\} { 3 × 1 0 − 5 , 1 0 − 5 } or { 3 × 10 − 3 , 10 − 2 } \{3 \times10^{-3}, 10^{-2}\} { 3 × 1 0 − 3 , 1 0 − 2 } , SI noise by { 0.125 , 0.0625 } \{0.125, 0.0625\} { 0.125 , 0.0625 } or { 64 , 128 } \{64, 128\} { 64 , 128 } , and P m e a n P_{\mathrm{mean}} P mean by { − 3.6 , − 4 } \{-3.6, -4\} { − 3.6 , − 4 } or { 0 , 0.4 } \{0, 0.4\} { 0 , 0.4 } . Unresolved search boundaries retain a flag.
Sampler resolution. For each depth and method, a fixed probe is evaluated using E D ^ V \widehat{\mathrm{ED}}_V ED V on the step ladder 20, 40, 80, 160, 320, and 640. The upper resolution of the first doubling with relative score change below 5% is retained. Probes use the final checkpoint at learning rate 3 × 10 − 4 3 \times10^{-4} 3 × 1 0 − 4 , with SI noise 8 or EDM P m e a n = − 1.6 P_{\mathrm{mean}} = -1.6 P mean = − 1.6 . The selected model is checked again at twice its assigned resolution. A failed check permits one resolution escalation and validation reselection; a repeated failure is flagged. Sampler calibration is therefore separate from model selection by E D ^ U \widehat{\mathrm{ED}}_U ED U .
To verify that the direct baselines are not limited by sampler resolution, we reevaluate the selected Direct SI and EDM checkpoints using the same number of network evaluations (NFE) as Cascaded SI. As shown in Table 5 , EDM performance remains essentially unchanged. Direct SI improves slightly on waves and NLKG (1–2%), but remains substantially worse than Cascaded SI. On Euler and SQG, increasing the sampler resolution does not improve performance.
Table 5. Matched inference cost. Test csFID-10k for Direct SI and EDM at the original 40 sampling steps and at approximately the same number of network evaluations (NFE) as Cascaded SI. Standard errors across the ten test conditions are omitted for readability. The 40-step values were re-generated for this comparison from the Table 2 checkpoints and agree with Table 2 to within 0.24 csFID (different GPU hardware). Cascaded SI values are those of Table 2 .
Selected configurations. Table 6 reports the frozen selections for all five systems. Sampler steps are reported per segment; network function evaluations (NFE) count the full forecast.
Table 6. Selected configurations and sampling cost.
Hyperparameter boundary checks. Three selected configurations initially reached the boundary of their hyperparameter search range. We therefore extended the searches for nonlinear Klein–Gordon Direct SI and heterogeneous-wave EDM using validation data only.
For Direct SI, extending the noise range to σ = 256 , 512 \sigma= 256, 512 σ = 256 , 512 and testing higher learning rates did not improve the selected configuration (E D U = 10.651 \mathrm{EDU} = 10.651 EDU = 10.651 ). For wave EDM, the best additional configuration reached E D U = 10.678 \mathrm{EDU} = 10.678 EDU = 10.678 , compared with 10.685 for the original selection. This difference is smaller than the observed across-seed standard deviation (0.028), so we retained the original configuration under our predefined extension criterion.
The nonlinear Klein–Gordon EDM search remains unresolved. Validation errors varied little across the original 50 configurations, and additional training modifications did not improve performance.
Conditional distributional evaluation
csFID-10k reconstructs unnormalized coarse fields on a 128 × 128 128 \times128 128 × 128 grid. Signed values are clipped symmetrically at the training-fitted 0.9999 quantile of absolute amplitude, linearly mapped to uint8, and replicated into three identical channels. There is no colormap or per-image normalization. For heterogeneous waves and nonlinear Klein–Gordon, csFID evaluates displacement only, although the forecasting state also includes a velocity channel. Images undergo one bicubic resize to 299 × 299 299 \times299 299 × 299 in clean-fid 0.1.35 [27 ] . The first seven channels of Inception’s mixed_6/conv 17 × 17 17 \times17 17 × 17 activation map are flattened in spatial–channel order, giving 2023 features without spatial pooling. Pooled cFID uses the same preprocessing and Inception weights with 2048-dimensional pool3 features.
For each condition, the Fréchet distance is computed between the feature means and covariances of 10,000 true and 10,000 generated endpoints. The conditional score averages these distances with equal condition weights. Reported uncertainty is the sample standard deviation of condition-level scores divided by the square root of the number of conditions. For the main comparison, this is SEM over ten test conditions for one training seed. Secondary state-space diagnostics use fixed subsets of 200 true and 200 generated endpoints unless otherwise stated.
Numerical values from the depth sweep presented in § 5 are reported in Table 7 .
Table 7. Conditional forecasting performance across cascade depths. Test csFID-10k, mean ± SEM across ten conditioning ensembles for one training seed. Bold indicates the lowest SI score within each system; EDM is shown for comparison. For SQG at S = 20 S = 20 S = 20 , one nonfinite generated endpoint was excluded from one conditioning ensemble, leaving 9,999 generated samples for that score.
Validation of the spatial feature metric
The conditioning field fixes spatial registration: realistic structures at incorrect locations can still constitute an incorrect conditional forecast. We therefore use spatial FID (sFID) [24 ] , which retains intermediate spatial Inception features, alongside pooled FID as a diagnostic. The representations have 2023 and 2048 coordinates, respectively; all image preprocessing and Inception weights are shared. Appendix § 11 .§ 3 specifies their construction and conditional aggregation.
Agreement with state-space diagnostics. On Euler, pooled cFID separates Direct SI and EDM more strongly than csFID does (Table 8 ). The paired EDM-minus-Direct difference over the ten test conditions is −9.35 ± 1.31 for cFID and −0.94 ± 0.90 for csFID (mean ± SEM). The closer csFID scores agree with the similar sliced Wasserstein distances, conditional-mean errors, and spread ratios of these two endpoint models. Cascaded SI has the lowest cFID, csFID, sliced Wasserstein distance, and conditional-mean error.
Table 8. Euler metric comparison. Mean ± SEM over ten test conditions. Feature metrics use 10,000 true and generated endpoints per condition. State-space diagnostics use 200 of each in normalized Fourier coordinates: sliced W 2 W_2 W 2 uses 100 fixed random projections, mean error is the Euclidean distance between ensemble means, and spread is the ratio of mean pairwise distances (generated/true, excluding self-pairs). Lower is better except for spread, whose target is one.
Sensitivity to spatial rearrangement. We split each true 10,000-member endpoint bank into disjoint sets A A A , B B B of 5000 samples and compare d ( A , B ) d(A, B) d ( A , B ) with d ( F ( A ) , B ) d(F(A), B) d ( F ( A ) , B ) . Transformations act exactly on Fourier coefficients, before image construction. A translation of d d d pixels shifts both coordinates by d d d on the 128 × 128 128 \times128 128 × 128 reconstruction. Euler averages ten resplits within each condition; passive scalar uses one split. Table 9 reports ratios of across-condition means to the corresponding truth-to-truth mean. On Euler, spatial features respond more strongly relative to this baseline. Pooled features also respond strongly on passive scalar, so their sensitivity to spatial rearrangement is dataset-dependent. These probes measure sensitivity, not a calibrated distance between spatial transformations. Resplits reuse the same bank and are not independent ensemble replicates.
Table 9. Response to spatial perturbations. Each entry is the ratio of the perturbed score to the truth-to-truth score, using means over ten conditioning states. Passive scalar has only two distinct outputs because Cascaded SI selects S ∗ = 1 S^* = 1 S ∗ = 1 .
Observed initial state x 0 x_0 x 0
Every row forecasts the same observation.
Mean and standard deviation over 10,000 samples.
Sample-size stability. Recomputing both metrics on nested ensembles of N = 2000 N = 2000 N = 2000 , 5000, 10000 preserves the ordering of the mean scores on Euler and passive scalar. On Euler, the EDM/Direct-SI score ratios are 0.995, 0.988, 0.984 for csFID and 0.706, 0.646, 0.615 for cFID. Thus the smaller separation under spatial features persists throughout this range.
Additional qualitative conditional forecasts
Figure 5 shows the comparison of direct and cascaded forecasts for one test condition of the heterogeneous-wave system, in the layout of Figure 2 . Figure 6 shows an example test condition for SQG, nonlinear Klein–Gordon, and passive scalar. For SQG and nonlinear Klein–Gordon, Cascaded SI outperforms Direct SI and EDM by a large amount. For passive scalar, validation selects S ⋆ = 1 S^\star= 1 S ⋆ = 1 , so Direct SI and Cascaded SI coincide.
Figure 5. Heterogeneous waves: direct and cascaded forecasts of one held-out observation. The displayed field is the displacement; the forecasters also predict a velocity proxy. Cascaded SI uses the validation-selected depth S ⋆ = 5 S^\star= 5 S ⋆ = 5 . Means and standard deviations are estimated from 10,000 samples.
Figure 6. Additional conditional endpoint distributions. Test conditions for SQG, nonlinear Klein–Gordon, and passive scalar. For each method, three endpoint samples are shown together with the conditional mean and pointwise standard deviation estimated from 10,000 generated endpoints; reference statistics are computed from 10,000 simulator-generated endpoints from the same conditioning state.
Additional quantitative experiments
Effect of training-condition diversity.
We conduct an additional experiment on Euler at T = 1 T = 1 T = 1 . Keeping the training budget fixed at 16,000 supervised pairs, we compare Direct SI trained on 80 initial conditions with 200 trajectories each against 800 initial conditions with 20 trajectories each. As a reference, we include Cascaded SI with
S = 10 S = 10 S = 10 , trained on 80 initial conditions with 20 trajectories each and 10 transitions per trajectory. The cascade depth is fixed rather than reoptimized for this experiment.
Architecture and optimization follow the main benchmark (U-Net, Adam, batch size 256, 30,000 updates, and the same EMA and checkpoint schedule). For each configuration, we search learning rates { 3 × 10 − 5 , 10 − 4 , 3 × 10 − 4 , 10 − 3 , 3 × 10 − 3 } \{3 \times10^{-5}, 10^{-4}, 3 \times10^{-4}, 10^{-3}, 3 \times10^{-3}\} { 3 × 1 0 − 5 , 1 0 − 4 , 3 × 1 0 − 4 , 1 0 − 3 , 3 × 1 0 − 3 } and SI noise scales { 0.125 , 0.25 , 0.5 , 0.75 , 1 , 1.5 , 2 , 3 , 4 , 6 } \{0.125, 0.25, 0.5, 0.75, 1, 1.5, 2, 3, 4, 6\} { 0.125 , 0.25 , 0.5 , 0.75 , 1 , 1.5 , 2 , 3 , 4 , 6 } , with one boundary extension to { 9 , 13.5 } \{9, 13.5\} { 9 , 13.5 } . This gives 60 hyperparameter configurations per training allocation. Checkpoints are selected by biased validation energy distance E D V \mathrm{ED}_{\mathrm{V}} ED V , using 100 generated samples against the full reference ensemble per validation condition, and choosing the earliest checkpoint within 0.5% of the minimum. Sampling uses 80 steps for Direct SI and 160 per segment for Cascaded SI, with convergence verified by doubling. Hyperparameters are selected using seed 0, and only the selected configuration is retrained with seed 1. Test evaluation uses the same csFID-10k metric, image adapter, and reference ensembles as Table 2 .
Table 10. Effect of training-condition diversity on Euler. Results are averaged over two training seeds within each of ten test conditions and reported as mean ± \pm ± SEM across conditions.
Increasing the number of distinct initial conditions by a factor of ten improves Direct SI by 18%, with a paired csFID reduction of 10.8 ± 2.3 10.8 \pm2.3 10.8 ± 2.3 across the ten test conditions. Nevertheless, Cascaded SI achieves approximately three times lower csFID, despite using ten times fewer original initial conditions and physical trajectories (1,600 versus 16,000). Its depth S = 10 S = 10 S = 10 was not reoptimized for this experiment, so a different depth might yield even better results.
Training with one trajectory per initial condition
To investigate whether the advantage of cascading depends on having multiple training trajectories per initial condition, we conduct an additional experiment on Euler with 16,000 distinct training initial conditions and exactly one trajectory each. Both Direct SI and Cascaded SI use the same 16,000 trajectories and receive 16,000 supervised pairs, matching the training-data budget of the main benchmark. Direct SI uses one full-horizon endpoint pair per trajectory, while Cascaded SI uses one short transition per trajectory, balanced across the S S S segments.
We follow the main benchmark protocol for architecture, optimization, hyperparameter and checkpoint selection, and sampler calibration. Normalization is fitted on the new training data, while validation and test conditions and reference ensembles remain unchanged. We evaluate depths S ∈ { 1 , 2 , 5 , 10 } S \in\{1, 2, 5, 10\} S ∈ { 1 , 2 , 5 , 10 } , selecting S ⋆ = 10 S^{\star} = 10 S ⋆ = 10 on validation.
As shown in Table 11 , Cascaded SI achieves roughly three times lower csFID-10k than Direct SI and outperforms it on all ten test conditions. Notably, it achieves a score comparable to the main benchmark (14.70 versus 16.32).
Table 11. Euler forecasting performance when training with one trajectory per initial condition. Test csFID-10k (mean ± \pm ± SEM across ten test conditions for one training seed). Bold indicates the validation-selected depth.
Training versus test conditions
We evaluated the models in Table 2 on all 80 training conditions and 10 test conditions, using the validation-selected checkpoint and the final checkpoint after 30,000 updates. We used the original samplers and unbiased energy distance (E D U \mathrm{ED}_{U} ED U ), with 100 generated and 100 reference samples per condition.
Table 12. Training and test performance. Results are for the original Table 2 models. Small negative values can occur with the unbiased E D U \mathrm{ED}_{U} ED U metric.
On Euler, SQG, waves, and NLKG, Direct SI fits its training conditions increasingly well but generalizes poorly to unseen conditions. Cascaded SI achieves substantially lower training loss, which remain relatively stable during further training. EDM shows a similar generalization gap on Euler, SQG, and waves, but does not fit its training conditions as well on NLKG. On passive scalar, both direct methods generalize well.
These results explain the early checkpoint selection of the direct models reported in Table 6 : continued training improves their fit to the training conditions while validation performance deteriorates. The test evaluations at 30k were diagnostic only and did not affect model selection.
Forecast horizon and cascade depth
We study how the benefit of cascading changes with the forecasting horizon on Euler. We consider T ∈ { 0.2 , 0.4 , 0.6 , 0.8 , 1.0 } T \in\{0.2, 0.4, 0.6, 0.8, 1.0\} T ∈ { 0.2 , 0.4 , 0.6 , 0.8 , 1.0 } and, at each horizon, all cascade depths S S S that divide the 40 T 40T 40 T stored time steps, giving 31 ( T , S ) (T, S) ( T , S ) configurations.
We use the same Euler data, network architecture, validation-based selection criterion, and csFID-10k evaluation as in the main benchmark. For this experiment, the hyperparameter search is wider: learning rates { 3 × 10 − 5 , 10 − 4 , 3 × 10 − 4 , 10 − 3 , 3 × 10 − 3 } \{3 \times10^{-5}, 10^{-4}, 3 \times10^{-4}, 10^{-3}, 3 \times10^{-3}\} { 3 × 1 0 − 5 , 1 0 − 4 , 3 × 1 0 − 4 , 1 0 − 3 , 3 × 1 0 − 3 } and SI noise scales from 0.5 to 64. As in the benchmark protocol, if the best configuration lies on a boundary of this grid, the search is extended once in that direction (learning rate 10 − 5 10^{-5} 1 0 − 5 or 10 − 2 10^{-2} 1 0 − 2 ; noise scale 0.25 and 0.125, or 128). Each configuration is evaluated with three training seeds, with hyperparameters selected by mean validation E D U \mathrm{ED}_{\mathrm{U}} ED U across seeds. Training runs for 30,000 updates, with one extension to 60,000 if validation error is still improving (14 of 31 configurations). We use 15,840 supervised pairs instead of 16,000 so that the pair budget is divisible by every tested depth. The sampler-resolution ladder is unchanged, with additional convergence checks.
Cascading improves performance at every tested horizon. The advantage increases from approximately twofold at T = 0.2 T = 0.2 T = 0.2 to three- to fourfold at longer horizons. The selected depth generally increases with T T T , from S ∗ = 4 S^* = 4 S ∗ = 4 to S ∗ = 10 S^* = 10 S ∗ = 10 , while the segment duration T / S ∗ T/S^* T / S ∗ remains between 0.05 and 0.1. The depth sweeps in Figure 7 also show that performance deteriorates when too many segments are used. The minima are relatively broad.
Figure 7. Test csFID-10k is shown for five horizons, with Direct SI corresponding to S = 1 S = 1 S = 1 . The black circles mark the validation-selected S ∗ S^* S ∗ . The bottom-right panel compares Direct SI with Cascaded SI at S ∗ ( T ) S^*(T) S ∗ ( T ) across horizons. Values are means across three training seeds and ten test conditions. Error bars show SEM across conditions after averaging over seeds. Plotted numerical values are given in Table 13 .
Table 13. Euler forecasting performance across horizons. Test csFID-10k (mean ± \pm ± SEM across ten test conditions, each averaged over three training seeds).
Absolute csFID values need not increase monotonically with T T T (see performance of direct SI in Figure 7 ), since each horizon has its own reference ensembles, so csFID-10k compares methods at a fixed T T T , not one method across T T T .
For Direct SI at T = 0.4 T = 0.4 T = 0.4 , the selected learning rate (10 − 5 10^{-5} 1 0 − 5 ) lies at the boundary of searched hyperparameters so a wider search might improve this baseline. However, a large improvement is unlikely since validation performance is nearly flat in the learning rate (less than 2% spread among the ten best configurations, which cover 10 − 5 10^{-5} 1 0 − 5 to 10 − 3 10^{-3} 1 0 − 3 ), while the selected cascade is about six times better on the same criterion.
Anchoring the initial observation.
To test whether retaining the initial observation mitigates the deterioration of Cascaded SI at large depths, we conduct the Euler and passive-scalar depth sweeps with an anchored cascade. The unanchored transition conditions on the generated state X t i X_{t_i} X t i while the anchored transition additionally conditions on the original observation X 0 X_0 X 0 at every segment. We use the same training data and U-Net architecture as in the corresponding benchmark, with an additional conditioning input for the anchored model. Hyperparameters, checkpoints, and sampler resolutions are selected separately using the benchmark validation procedure. All curves are evaluated on the same ten test conditions and conditional reference ensembles, with csFID-10k computed from 10,000 samples per condition.
The results differ substantially across systems (Figure 8 ). On euler, anchoring gives no improvement in csFID at any tested depth. Validation selects S ⋆ = 10 S^\star= 10 S ⋆ = 10 for both variants. In contrast, on passive-scalar, anchoring improves all tested depths S ≥ 2 S \ge2 S ≥ 2 on all ten test conditions. At S = 10 S = 10 S = 10 , csFID decreases from 41.14 ± 2.36 41.14 \pm2.36 41.14 ± 2.36 to 26.00 ± 1.96 26.00 \pm1.96 26.00 ± 1.96 , and at S = 40 S = 40 S = 40 from 75.21 ± 4.74 75.21 \pm4.74 75.21 ± 4.74 to 50.15 ± \pm ± 3.85. The anchored curve nevertheless increases with depth, and validation selects S ⋆ = 1 S^{\star} = 1 S ⋆ = 1 for both variants.
Figure 8. Retaining the initial observation. Test csFID-10k against cascade depth for Euler (left) and passive scalar (right). The anchored cascade receives the original observation X 0 X_0 X 0 at every segment, in addition to the generated current state. Error bars are SEM across ten test conditions; markers indicate the depth selected by validation for each method. Direct SI and EDM are shown as references.
Table 14. Anchored and unanchored depth sweeps. Test csFID-10k (mean ± \pm ± SEM over ten conditions). The anchored S = 1 S = 1 S = 1 models were not test-scored. The first two data columns are for euler and the last two for passive-scalar.
To understand why anchoring helps on the passive-scalar dataset but not on the Euler dataset, we provide a theoretical analysis of learned anchored transitions that studies the distinction between available information and learnable dependence in the rotation–OU model in Appendix C.8 and Appendix C.9.
Sensitivity to the unresolved-scale prior
To test whether the advantage of Cascaded SI depends on independently sampled observed and unresolved initial states, we constructed an additional SQG dataset with a dependent fine-scale prior. We kept the observed initial states, SQG dynamics, and total hidden-state amplitude unchanged, but modified how the unresolved Fourier modes are sampled at t = 0 t = 0 t = 0 .
Specifically, we constructed a Markovian prior across spatial scales. Starting from the observed coarse state X 0 X_0 X 0 (∣ k ∣ ≤ 8 |k| \le8 ∣ k ∣ ≤ 8 ), we generated two bands of hidden modes. In the first band, H 1 H_1 H 1 (8 < ∣ k ∣ ≤ 16 8 < |k| \le16 8 < ∣ k ∣ ≤ 16 ), random fluctuations have larger local variance where the observed coarse field has strong gradients. In the second band, H 2 H_2 H 2 (16 < ∣ k ∣ ≤ 32 16 < |k| \le32 16 < ∣ k ∣ ≤ 32 ), the local variance depends on the fluctuations in H 1 H_1 H 1 , rather than directly on X 0 X_0 X 0 . The construction therefore satisfies
p ( H 2 ∣ H 1 , X 0 ) = p ( H 2 ∣ H 1 ) . p(H_2 \mid H_1, X_0) = p(H_2 \mid H_1). p ( H 2 ∣ H 1 , X 0 ) = p ( H 2 ∣ H 1 ) .
Thus, the unresolved scales are statistically dependent on the observed state, with dependence transmitted across successive frequency bands. We used a modulation floor of 0.5 and retained the original hidden-state RMS of 0.1. The dependent prior is used only at initialization; subsequent evolution follows the original SQG equations.
We repeated the benchmark protocol, independently tuning Direct SI and Cascaded SI on the new dataset, with depths S ∈ { 1 , 2 , 5 , 10 , 20 } S \in\{1, 2, 5, 10, 20\} S ∈ { 1 , 2 , 5 , 10 , 20 } . On the ten test conditions, Cascaded SI achieves a csFID-10k of 53.18 ± \pm ± 15.63 at the validation-selected depth S ⋆ = 5 S^{\star} = 5 S ⋆ = 5 , compared with 141.93 ± \pm ± 23.46 for Direct SI (mean ± \pm ± SEM across test conditions; one training seed). Cascaded SI performs better on all ten conditions.
How cascading changes forecast error
A forecast over a fixed horizon T T T can be learned directly, in a single transition, or through a cascade of shorter transitions. The experiments in the main text show that shorter transitions can substantially improve the conditional forecast, but that the improvement can stop as the cascade becomes deeper. In this section, we study two mechanisms that can produce this behavior. First, a fixed approximation class may represent a short transition well while failing to represent the dependence that develops over a longer interval. Composing several simple maps can then represent dependence that a single map in the same class cannot. Second, under partial observation, repeatedly conditioning only on the current observed state can discard information carried by the unresolved state. We study these two effects separately before combining them in the rotation model from the main text, in which a hidden Ornstein–Uhlenbeck process drives the dynamics.
To keep the analysis explicit, we use forecast classes built from polynomial functions of the observed state. These provide a tractable setting in which we can calculate when the dependence of the forecast on its input leaves a fixed function class and how composing shorter fitted transitions changes the resulting error. We then study exact short conditional kernels to isolate the effect of repeated conditioning under partial observation. Finally, we analyze both effects together in the rotation–OU model, giving an example in which an intermediate cascade depth is optimal. The memory effect that appears under partial observation is classical in optimal prediction Chorin et al. [12 ] ; Ma et al. [21 ] .
Notation. In this appendix, the full state is denoted by Z t Z_t Z t and its unobserved initial component by η \eta η ; in the main text, these are Y t Y_t Y t and Z 0 Z_0 Z 0 .
When a fixed approximation class becomes insufficient
Our first goal is to understand how approximation error appears when the conditional forecast is restricted to a fixed class of functions. We begin with a general model of a partially observed dynamical system and identify the first term in the short-time expansion of the forecast that cannot be represented by the chosen class.
Let Z t ∈ R d Z_t \in\mathbb{R}^{d} Z t ∈ R d denote the full physical state. We assume that it evolves according to the smooth autonomous system
Z ˙ t = V ( Z t ) , \dot{Z}_t = V(Z_t), Z ˙ t = V ( Z t ) ,
where V : R d → R d V:\mathbb{R}^{d} \to\mathbb{R}^{d} V : R d → R d . We do not observe the full state Z t Z_t Z t . Instead, we observe
X t = P Z t ∈ R m , X_t = PZ_t \in\mathbb{R}^{m}, X t = P Z t ∈ R m ,
where P : R d → R m P:\mathbb{R}^{d} \to\mathbb{R}^{m} P : R d → R m is a fixed observation map.
Let χ : R m → R p \chi:\mathbb{R}^{m} \to\mathbb{R}^{p} χ : R m → R p specify the quantity that we wish to forecast from the observed state. For example, χ \chi χ may select a single observed Fourier coefficient, a collection of observed coefficients, or the entire observed state, in which case χ = I d \chi= \mathrm{Id} χ = Id . For each forecast horizon t t t , define
( T t χ ) ( x ) = E [ χ ( X t ) ∣ X 0 = x ] . (19) (T_t\chi)(x) = \mathbb{E}[\chi(X_t) \mid X_0 = x]. \tag*{(19)} ( T t χ ) ( x ) = E [ χ ( X t ) ∣ X 0 = x ] . (19)
Thus, ( T t χ ) ( x ) (T_t\chi)(x) ( T t χ ) ( x ) is the conditional mean of the future quantity χ ( X t ) \chi(X_t) χ ( X t ) given the observed initial state X 0 = x X_0 = x X 0 = x .
Because only part of the physical state is observed, the operators T t T_t T t need not satisfy the semigroup identity
T t + s χ = T t ( T s χ ) . T_{t+s}\chi= T_t(T_s\chi). T t + s χ = T t ( T s χ ) .
Indeed, the unresolved variables may carry information from one time interval to the next that cannot be recovered from the current observed state alone. This issue will become important when we study cascades under partial observation below.
We first examine how the conditional forecast changes over a short physical time interval. For any smooth function g : R d → R p g:\mathbb{R}^{d} \to\mathbb{R}^{p} g : R d → R p of the full state, the chain rule gives
d d t g ( Z t ) = D g ( Z t ) V ( Z t ) . \frac{\mathrm{d}}{\mathrm{d}t}g(Z_t) = Dg(Z_t)V(Z_t). d t d g ( Z t ) = D g ( Z t ) V ( Z t ) .
We therefore define the derivative of g g g along the full dynamics by
( L V g ) ( z ) = D g ( z ) V ( z ) . (\mathcal{L}_V g)(z) = Dg(z)V(z). ( L V g ) ( z ) = D g ( z ) V ( z ) .
Higher-order derivatives along the dynamics are defined recursively by
L V 0 g = g , L V j + 1 g = L V ( L V j g ) . \mathcal{L}_V^0 g = g, \qquad\mathcal{L}_V^{j+1}g = \mathcal{L}_V(\mathcal{L}_V^j g). L V 0 g = g , L V j + 1 g = L V ( L V j g ) .
It follows that
d j d t j g ( Z t ) ∣ t = 0 = L V j g ( Z 0 ) . \left.\frac{\mathrm{d}^j}{\mathrm{d}t^j}g(Z_t)\right|_{t=0} = \mathcal{L}_V^j g(Z_0). d t j d j g ( Z t ) t = 0 = L V j g ( Z 0 ) .
We apply this identity to the full-state function g = χ ∘ P g = \chi\circ P g = χ ∘ P . After conditioning on the observed initial state, define
a j , χ ( x ) = E [ L V j ( χ ∘ P ) ( Z 0 ) ∣ X 0 = x ] . (20) a_{j,\chi}(x) = \mathbb{E}\left[\mathcal{L}_V^j(\chi\circ P)(Z_0) \mid X_0 = x\right]. \tag*{(20)} a j , χ ( x ) = E [ L V j ( χ ∘ P ) ( Z 0 ) ∣ X 0 = x ] . (20)
Each a j , χ a_{j,\chi} a j , χ is a function of the observed initial state x x x . However, the derivative inside the conditional expectation is computed using the full state Z 0 Z_0 Z 0 and the full dynamics V V V . The conditional expectation averages this derivative over the unresolved components of Z 0 Z_0 Z 0 .
The theorem below gives precise conditions under which the functions a j , χ a_{j,\chi} a j , χ are the coefficients in the short-time expansion of T t χ T_t\chi T t χ . It also controls the error made by truncating that expansion.
Let
μ = Law ( X 0 ) \mu= \operatorname{Law}(X_0) μ = Law ( X 0 )
denote the distribution of the observed initial state. Choose a closed linear space
F ⊂ L 2 ( μ ; R p ) \mathcal{F} \subset L^2(\mu;\mathbb{R}^p) F ⊂ L 2 ( μ ; R p )
representing the fixed class of forecasting functions under consideration. The class F \mathcal{F} F is chosen before selecting the forecast horizon and before fitting a predictor to future targets. Since F \mathcal{F} F is closed, let Π F \Pi_{\mathcal{F}} Π F denote the orthogonal projection from L 2 ( μ ; R p ) L^2(\mu;\mathbb{R}^p) L 2 ( μ ; R p ) onto F \mathcal{F} F .
We define n ⋆ n_{\star} n ⋆ to be the first order for which the corresponding coefficient cannot be represented by F \mathcal{F} F :
n ⋆ = min { j ≥ 0 : a j , χ ∉ F } . n_{\star} = \min\{j \ge0 : a_{j,\chi} \notin\mathcal{F}\}. n ⋆ = min { j ≥ 0 : a j , χ ∈ / F } .
If every coefficient a j , χ a_{j,\chi} a j , χ belongs to F \mathcal{F} F , we set
n ⋆ = ∞ . n_{\star} = \infty. n ⋆ = ∞.
In particular, if n ⋆ = n < ∞ n_{\star}=n<\infty n ⋆ = n < ∞ , then
a 0 , χ , … , a n − 1 , χ ∈ F , a n , χ ∉ F . a_{0,\chi},\ldots,a_{n-1,\chi}\in\mathcal{F}, \qquad a_{n,\chi}\notin\mathcal{F}. a 0 , χ , … , a n − 1 , χ ∈ F , a n , χ ∈ / F .
Thus, a n , χ a_{n,\chi} a n , χ is the first coefficient in the short-time expansion that cannot be represented by the chosen approximation class.
Theorem C.1. Let t 0 > 0 t_0>0 t 0 > 0 and suppose that n = n ⋆ < ∞ n=n_{\star}<\infty n = n ⋆ < ∞ . Assume that, almost surely, the trajectory Z t Z_t Z t is defined on [ 0 , t 0 ] [0,t_0] [ 0 , t 0 ] and the map
t ↦ χ ( P Z t ) t\mapsto\chi(PZ_t) t ↦ χ ( P Z t ) is ( n + 1 ) (n+1) ( n + 1 ) times continuously differentiable on this interval. Assume also that
E [ ∣ L V j ( χ ∘ P ) ( Z 0 ) ∣ 2 ] < ∞ for every j = 0 , … , n . \mathbb{E}\left[\left|L_V^j(\chi\circ P)(Z_0)\right|^2\right]<\infty\qquad\text{for every }j=0,\ldots,n. E [ L V j ( χ ∘ P ) ( Z 0 ) 2 ] < ∞ for every j = 0 , … , n . Define
M n + 1 : = sup 0 ≤ u ≤ t 0 ( E [ ∣ L V n + 1 ( χ ∘ P ) ( Z u ) ∣ 2 ] ) 1 / 2 , M_{n+1}:=\sup_{0\le u\le t_0}\left(\mathbb{E}\left[\left|L_V^{n+1}(\chi\circ P)(Z_u)\right|^2\right]\right)^{1/2}, M n + 1 := 0 ≤ u ≤ t 0 sup ( E [ L V n + 1 ( χ ∘ P ) ( Z u ) 2 ] ) 1/2 , and assume that M n + 1 < ∞ M_{n+1}<\infty M n + 1 < ∞ . Finally, let
c n = ∥ ( I − Π F ) a n , χ ∥ L 2 ( μ ) . c_n=\left\|(I-\Pi_{\mathcal{F}})a_{n,\chi}\right\|_{L^2(\mu)}. c n = ∥ ( I − Π F ) a n , χ ∥ L 2 ( μ ) . Then c n > 0 c_n>0 c n > 0 and, for every t ∈ [ 0 , t 0 ] t\in[0,t_0] t ∈ [ 0 , t 0 ] ,
∣ dist L 2 ( μ ) ( T t χ , F ) − c n t n n ! ∣ ≤ M n + 1 t n + 1 ( n + 1 ) ! . (21) \left|\operatorname{dist}_{L^2(\mu)}(T_t\chi,\mathcal{F})-\frac{c_nt^n}{n!}\right|\le\frac{M_{n+1}t^{n+1}}{(n+1)!}. \tag*{(21)} dist L 2 ( μ ) ( T t χ , F ) − n ! c n t n ≤ ( n + 1 )! M n + 1 t n + 1 . (21)
Proof of Theorem C.1. Taylor’s integral formula along each trajectory gives, with g = χ ∘ P g=\chi\circ P g = χ ∘ P ,
g ( Z t ) = ∑ j = 0 n t j j ! L V j g ( Z 0 ) + 1 n ! ∫ 0 t ( t − u ) n L V n + 1 g ( Z u ) d u . g(Z_t)=\sum_{j=0}^{n}\frac{t^j}{j!}L_V^jg(Z_0)+\frac{1}{n!}\int_0^t(t-u)^nL_V^{n+1}g(Z_u)\,\mathrm{d}u. g ( Z t ) = j = 0 ∑ n j ! t j L V j g ( Z 0 ) + n ! 1 ∫ 0 t ( t − u ) n L V n + 1 g ( Z u ) d u .
Condition on X 0 X_0 X 0 and use Minkowski’s inequality and the L 2 L^2 L 2 contraction of conditional expectation. The conditional remainder has norm at most M n + 1 t n + 1 / ( n + 1 ) ! M_{n+1}t^{n+1}/(n+1)! M n + 1 t n + 1 / ( n + 1 )! . Applying I − Π F I-\Pi_{\mathcal{F}} I − Π F removes all terms before n n n and does not increase the remainder. The reverse triangle inequality proves (C.3). Since F \mathcal{F} F is closed and a n , χ ∉ F a_{n,\chi}\notin\mathcal{F} a n , χ ∈ / F , its distance c n c_n c n is positive.
A quadratic dynamical model and a fixed forecast class
We now apply the general result above to a concrete class of dynamical systems. We consider quadratic dynamics, forecast the entire observed state, and use quadratic functions of the initial observation as our fixed forecast class. In the notation of the previous subsection, this means that
χ = Id and F = P 2 . \chi= \operatorname{Id} \qquad\text{and} \qquad\mathcal{F} = \mathcal{P}_2. χ = Id and F = P 2 .
Our goal is to show that, even when the full dynamics are quadratic, the conditional mean can develop a nonzero cubic dependence on the observed initial state.
This model isolates the nonlinear mechanism relevant to the active-scalar systems in our benchmark. Finite-dimensional Galerkin versions of Euler, for example, contain quadratic interactions between modes. These interactions can make the observed forecast depend on increasingly complicated combinations of the initial variables, and can transfer information between observed and unresolved modes. The result below is general for quadratic systems satisfying the stated assumptions. We also give Lorenz–96 as a standard additional example, although it is not part of the benchmark.
Let the full physical state satisfy
Z ˙ t = V ( Z t ) , V ( z ) = Q ( z ) + A z + f , Z t ∈ R d , (22) \dot{Z}_t = V(Z_t), \qquad V(z) = Q(z) + Az + f, \qquad Z_t \in\mathbb{R}^d, \tag*{(22)} Z ˙ t = V ( Z t ) , V ( z ) = Q ( z ) + A z + f , Z t ∈ R d , (22)
where Q Q Q is homogeneous and quadratic, A z Az A z is linear, and f f f is constant. We assume that the quadratic term satisfies
z ⊤ Q ( z ) = 0 for every z ∈ R d . (23) z^\top Q(z) = 0 \qquad\text{for every } z \in\mathbb{R}^d. \tag*{(23)} z ⊤ Q ( z ) = 0 for every z ∈ R d . (23)
This means that the quadratic term alone does not change ∣ Z t ∣ 2 |Z_t|^2 ∣ Z t ∣ 2 . Indeed,
d d t ∣ Z t ∣ 2 = 2 Z t ⊤ Q ( Z t ) = 0 \frac{\mathrm{d}}{\mathrm{d}t}|Z_t|^2 = 2Z_t^\top Q(Z_t) = 0 d t d ∣ Z t ∣ 2 = 2 Z t ⊤ Q ( Z t ) = 0
when the dynamics contains only Q Q Q . This property holds in the examples that motivate the analysis: the quadratic term conserves enstrophy in a Galerkin truncation of two-dimensional Euler and the squared norm in Lorenz–96. The linear term A z Az A z and the forcing term f f f need not preserve this quantity. We use Eq. (C.5) both to control the growth of the trajectories and to prove that the cubic term identified below is nonzero.
As in the previous subsection, we observe
X t = P Z t ∈ R m . X_t = PZ_t \in\mathbb{R}^m. X t = P Z t ∈ R m .
We now assume that the rows of P P P are orthonormal:
P P ⊤ = I m . PP^\top= I_m. P P ⊤ = I m .
Define
Then
P E = I m and ∣ E x ∣ = ∣ x ∣ for every x ∈ R m . PE = I_m \qquad\text{and} \qquad|Ex| = |x| \quad\text{for every } x \in\mathbb{R}^m. P E = I m and ∣ E x ∣ = ∣ x ∣ for every x ∈ R m .
Thus, E E E maps an observed state x x x back into the full state space while preserving its observed coordinates and its norm.
We assume that the initial state can be written as
Z 0 = E X 0 + η , P η = 0 , E ∣ Z 0 ∣ 8 < ∞ . (24) Z_0 = EX_0 + \eta, \qquad P\eta= 0, \qquad\mathbb{E}|Z_0|^8 < \infty. \tag*{(24)} Z 0 = E X 0 + η , P η = 0 , E ∣ Z 0 ∣ 8 < ∞. (24)
Here, E X 0 EX_0 E X 0 is the observed part of the initial state and η \eta η is the unresolved part. The condition P η = 0 P\eta= 0 P η = 0 means that η \eta η is invisible in the observation X 0 = P Z 0 X_0 = PZ_0 X 0 = P Z 0 . We assume that X 0 X_0 X 0 and η \eta η are independent. This matches the coarse–fine initialization used in the Euler benchmark, where the observed and hidden Fourier modes are sampled independently. Independence is assumed only at the initial time; the dynamics may couple the two parts at later times. The eighth-moment condition is used to control the remainder in the short-time expansion.
We keep the notation
μ = Law ( X 0 ) \mu= \operatorname{Law}(X_0) μ = Law ( X 0 )
for the distribution of the observed initial state. We assume that μ \mu μ has a density that is strictly positive on a nonempty open set. This ensures that a polynomial which is nonzero as a function cannot vanish μ \mu μ -almost everywhere.
Let Φ t ( z ) \Phi_t(z) Φ t ( z ) denote the full state at time t t t when the initial state is z z z . Given an observed initial state X 0 = x X_0=x X 0 = x , define
K t ( x , d x ′ ) = Law ( P Φ t ( Z 0 ) ∣ X 0 = x ) . (25) K_t(x,dx')=\operatorname{Law}(P\Phi_t(Z_0)\mid X_0=x). \tag*{(25)} K t ( x , d x ′ ) = Law ( P Φ t ( Z 0 ) ∣ X 0 = x ) . (25)
Thus, K t ( x , ⋅ ) K_t(x,\cdot) K t ( x , ⋅ ) is the conditional distribution of the future observed state X t X_t X t . Its conditional mean is
m t ( x ) = ∫ x ′ K t ( x , d x ′ ) = E [ X t ∣ X 0 = x ] . (26) m_t(x)=\int x'K_t(x,dx')=\mathbb{E}[X_t\mid X_0=x]. \tag*{(26)} m t ( x ) = ∫ x ′ K t ( x , d x ′ ) = E [ X t ∣ X 0 = x ] . (26)
Under squared error, m t m_t m t is the best possible forecast of X t X_t X t based only on X 0 X_0 X 0 . Since it predicts X t X_t X t from X 0 X_0 X 0 over the full interval, without using intermediate physical states, it is the direct conditional mean forecast in this analysis.
We now restrict this direct forecast to a fixed class of quadratic functions. Let P 2 \mathcal{P}_2 P 2 be the space of vector-valued polynomials p : R m → R m p:\mathbb{R}^m\to\mathbb{R}^m p : R m → R m of total degree at most 2, and let Π 2 \Pi_2 Π 2 denote the orthogonal projection onto P 2 \mathcal{P}_2 P 2 in L 2 ( μ ) L^2(\mu) L 2 ( μ ) . Define
e 2 ( t ) = inf p ∈ P 2 ∥ m t − p ∥ L 2 ( μ ) = ∥ m t − Π 2 m t ∥ L 2 ( μ ) . (27) e_2(t)=\inf_{p\in\mathcal{P}_2}\|m_t-p\|_{L^2(\mu)}=\|m_t-\Pi_2m_t\|_{L^2(\mu)}. \tag*{(27)} e 2 ( t ) = p ∈ P 2 inf ∥ m t − p ∥ L 2 ( μ ) = ∥ m t − Π 2 m t ∥ L 2 ( μ ) . (27)
Thus, e 2 ( t ) e_2(t) e 2 ( t ) measures how well a quadratic function of X 0 X_0 X 0 can represent m t m_t m t when the whole physical interval is predicted in a single transition.
This is the quantity that appears in Theorem C.1 when
χ = Id and F = P 2 . \chi=\operatorname{Id}\qquad\text{and}\qquad\mathcal{F}=\mathcal{P}_2. χ = Id and F = P 2 .
Indeed,
( T t Id ) ( x ) = E [ X t ∣ X 0 = x ] = m t ( x ) , (\mathcal{T}_t\operatorname{Id})(x)=\mathbb{E}[X_t\mid X_0=x]=m_t(x), ( T t Id ) ( x ) = E [ X t ∣ X 0 = x ] = m t ( x ) ,
and hence
e 2 ( t ) = dist L 2 ( μ ) ( T t Id , P 2 ) . e_2(t)=\operatorname{dist}_{L^2(\mu)}(\mathcal{T}_t\operatorname{Id},\mathcal{P}_2). e 2 ( t ) = dist L 2 ( μ ) ( T t Id , P 2 ) .
The square e 2 ( t ) 2 e_2(t)^2 e 2 ( t ) 2 also has a direct interpretation in terms of prediction error. For any quadratic predictor p ∈ P 2 p\in\mathcal{P}_2 p ∈ P 2 ,
E [ ∣ X t − p ( X 0 ) ∣ 2 ] = E [ ∣ X t − m t ( X 0 ) ∣ 2 ] + ∥ m t − p ∥ L 2 ( μ ) 2 . \mathbb{E}[|X_t-p(X_0)|^2]=\mathbb{E}[|X_t-m_t(X_0)|^2]+\|m_t-p\|_{L^2(\mu)}^2. E [ ∣ X t − p ( X 0 ) ∣ 2 ] = E [ ∣ X t − m t ( X 0 ) ∣ 2 ] + ∥ m t − p ∥ L 2 ( μ ) 2 .
Taking the infimum over p ∈ P 2 p\in\mathcal{P}_2 p ∈ P 2 gives
inf p ∈ P 2 E [ ∣ X t − p ( X 0 ) ∣ 2 ] = E [ ∣ X t − m t ( X 0 ) ∣ 2 ] + e 2 ( t ) 2 . \inf_{p\in\mathcal{P}_2}\mathbb{E}[|X_t-p(X_0)|^2]=\mathbb{E}[|X_t-m_t(X_0)|^2]+e_2(t)^2. p ∈ P 2 inf E [ ∣ X t − p ( X 0 ) ∣ 2 ] = E [ ∣ X t − m t ( X 0 ) ∣ 2 ] + e 2 ( t ) 2 .
The first term is the prediction error that remains because X 0 X_0 X 0 does not fully determine X t X_t X t . The second term is the additional error caused by restricting the conditional mean forecast to quadratic functions.
The next subsection identifies the first part of the conditional mean that lies outside P 2 \mathcal{P}_2 P 2 . We then apply Theorem C.1 to obtain an explicit short-time estimate for e 2 ( t ) e_2(t) e 2 ( t ) .
The cubic term that quadratic forecasts cannot represent
We now identify the first coefficient in the short-time expansion of the direct conditional mean m t m_t m t that can lie outside P 2 \mathcal{P}_2 P 2 . This will allow us to apply Theorem C.1 .
For χ = Id \chi=\operatorname{Id} χ = Id , the zeroth-order coefficient is
a 0 , Id ( x ) = x , a_{0,\operatorname{Id}}(x)=x, a 0 , Id ( x ) = x ,
which belongs to P 2 \mathcal{P}_2 P 2 . The first-order coefficient is
a 1 , Id ( x ) = E [ P V ( Z 0 ) ∣ X 0 = x ] . a_{1,\operatorname{Id}}(x)=\mathbb{E}[PV(Z_0)\mid X_0=x]. a 1 , Id ( x ) = E [ P V ( Z 0 ) ∣ X 0 = x ] .
Since V V V is quadratic and Z 0 = E x + η Z_0=Ex+\eta Z 0 = E x + η , this is a polynomial of degree at most two in x x x . Here we use the independence of X 0 X_0 X 0 and η \eta η , which ensures that the moments of η \eta η appearing in the conditional expectation do not depend on x x x . Therefore,
a 1 , Id ∈ P 2 . a_{1,\operatorname{Id}}\in\mathcal{P}_2. a 1 , Id ∈ P 2 .
The second-order coefficient is
a 2 , I d ( x ) = E [ P D V ( Z 0 ) V ( Z 0 ) ∣ X 0 = x ] . a_{2,\mathrm{Id}}(x)=\mathbb{E}[P\,DV(Z_0)V(Z_0)\mid X_0=x]. a 2 , Id ( x ) = E [ P D V ( Z 0 ) V ( Z 0 ) ∣ X 0 = x ] .
Unlike the previous coefficients, it can contain terms of degree three in x x x . Its degree-three part comes from applying the quadratic term Q Q Q twice and is given by
C 3 ( x ) = P D Q ( E x ) Q ( E x ) . (28) C_3(x)=P\,DQ(Ex)Q(Ex). \tag*{(28)} C 3 ( x ) = P D Q ( E x ) Q ( E x ) . (28)
Here, D Q ( E x ) DQ(Ex) D Q ( E x ) is the derivative of Q Q Q evaluated at E x Ex E x , and the final application of P P P keeps only the observed coordinates. All the remaining terms in a 2 , I d a_{2,\mathrm{Id}} a 2 , Id have degree at most two. Therefore,
a 2 , I d = C 3 + b 2 for some b 2 ∈ P 2 , a_{2,\mathrm{Id}}=C_3+b_2 \qquad\text{for some } b_2\in\mathcal{P}_2, a 2 , Id = C 3 + b 2 for some b 2 ∈ P 2 ,
and hence
( I − Π 2 ) a 2 , I d = ( I − Π 2 ) C 3 . (I-\Pi_2)a_{2,\mathrm{Id}}=(I-\Pi_2)C_3. ( I − Π 2 ) a 2 , Id = ( I − Π 2 ) C 3 .
We measure the part of C 3 C_3 C 3 that cannot be represented by a quadratic function by
c 3 = ∥ ( I − Π 2 ) C 3 ∥ L 2 ( μ ) . (29) c_3=\left\|(I-\Pi_2)C_3\right\|_{L^2(\mu)}. \tag*{(29)} c 3 = ∥ ( I − Π 2 ) C 3 ∥ L 2 ( μ ) . (29)
Since the zeroth- and first-order coefficients belong to P 2 \mathcal{P}_2 P 2 , proving that c 3 > 0 c_3>0 c 3 > 0 will show that n ⋆ = 2 n_\star=2 n ⋆ = 2 . The subscript in c 3 c_3 c 3 refers to the fact that C 3 C_3 C 3 has polynomial degree three, even though it appears at second order in physical time.
Theorem C.2. Under the assumptions of the previous subsection, suppose that Q ( E x ) Q(Ex) Q ( E x ) is not identically zero as a function of x x x . Then
n ⋆ = 2 and c 3 > 0. n_\star=2 \qquad\text{and} \qquad c_3>0. n ⋆ = 2 and c 3 > 0. Moreover, for every fixed t 0 > 0 t_0>0 t 0 > 0 , there is a finite and explicitly bounded constant M 3 M_3 M 3 such that, for every t ∈ [ 0 , t 0 ] t\in[0,t_0] t ∈ [ 0 , t 0 ] ,
∣ e 2 ( t ) − c 3 t 2 2 ∣ ≤ M 3 t 3 6 . (30) \left|e_2(t)-\frac{c_3t^2}{2}\right|\leq\frac{M_3t^3}{6}. \tag*{(30)} e 2 ( t ) − 2 c 3 t 2 ≤ 6 M 3 t 3 . (30) Consequently,
e 2 ( t ) 2 = c 3 2 4 t 4 + O ( t 5 ) . (31) e_2(t)^2=\frac{c_3^2}{4}t^4+O(t^5). \tag*{(31)} e 2 ( t ) 2 = 4 c 3 2 t 4 + O ( t 5 ) . (31) The coefficient c 3 c_3 c 3 depends on the quadratic dynamics Q Q Q , the observation map P P P , and the distribution μ \mu μ of the observed initial state. It does not depend on the distribution of the independent unresolved initial state η \eta η . The unresolved variables can still affect the lower-degree terms and the remainder.
The condition Q ( E x ) ≢ 0 Q(Ex)\not\equiv0 Q ( E x ) ≡ 0 has a simple meaning: an initial state containing only observed modes must activate at least one quadratic interaction. To see why this implies c 3 > 0 c_3>0 c 3 > 0 , differentiate
z ⊤ Q ( z ) = 0 z^\top Q(z)=0 z ⊤ Q ( z ) = 0
in the direction Q ( z ) Q(z) Q ( z ) . This gives
∣ Q ( z ) ∣ 2 + z ⊤ D Q ( z ) Q ( z ) = 0. \lvert Q(z)\rvert^2+z^\top DQ(z)Q(z)=0. ∣ Q ( z ) ∣ 2 + z ⊤ D Q ( z ) Q ( z ) = 0.
Setting z = E x z=Ex z = E x and using E = P ⊤ E=P^\top E = P ⊤ gives
x ⊤ C 3 ( x ) = − ∣ Q ( E x ) ∣ 2 . (32) x^\top C_3(x)=-\lvert Q(Ex)\rvert^2. \tag*{(32)} x ⊤ C 3 ( x ) = − ∣ Q ( E x ) ∣ 2 . (32)
If Q ( E x ) Q(Ex) Q ( E x ) is nonzero for some x x x , the right-hand side is strictly negative at that point. Therefore, C 3 C_3 C 3 is not identically zero.
Finally, C 3 C_3 C 3 is a homogeneous polynomial of degree three, whereas every element of P 2 \mathcal{P}_2 P 2 has degree at most two. Since μ \mu μ has a density that is positive on a nonempty open set, a nonzero cubic polynomial cannot agree with a quadratic polynomial μ \mu μ -almost everywhere. It follows that
Notice that the condition Q ( E x ) ≢ 0 Q(Ex)\not\equiv0 Q ( E x ) ≡ 0 does not require
P Q ( E x ) ≢ 0. PQ(Ex)\not\equiv0. P Q ( E x ) ≡ 0.
The first quadratic interaction may therefore lie entirely in the unresolved coordinates. Identity (C.14) shows that, after the dynamics acts again, this interaction still produces a nonzero term in the observed forecast.
The conclusion n ⋆ = 2 n_{\star}=2 n ⋆ = 2 applies to the forecast of the entire observed state. For a scalar quantity χ ( x ) = u ⊤ x \chi(x)=u^{\top}x χ ( x ) = u ⊤ x , the corresponding term is u ⊤ C 3 ( x ) u^{\top}C_3(x) u ⊤ C 3 ( x ) . Therefore, the same conclusion holds only if
u ⊤ C 3 ≠ 0. u^{\top}C_3\ne0. u ⊤ C 3 = 0.
Thus, some individual coordinates, including conserved or uncoupled quantities, may not contain this term.
For a general initial distribution, evaluating c 3 c_3 c 3 requires an L 2 ( μ ) L^2(\mu) L 2 ( μ ) projection. When the observed initial state is Gaussian, this projection can be computed explicitly; see Corollary C.3 below. The argument below also shows that the conclusion is unchanged by invertible affine changes of the observed coordinates.
Corollary C.3 (An explicit coefficient for Gaussian conditioning states). If X 0 ∼ N ( a , Σ ) X_0\sim\mathcal{N}(a,\Sigma) X 0 ∼ N ( a , Σ ) with Σ > 0 \Sigma>0 Σ > 0 , write
[ C 3 ( Σ 1 / 2 z ) ] k = ∑ i , j , ℓ = 1 m T k i j ℓ z i z j z ℓ , (33) [C_3(\Sigma^{1/2}z)]_k=\sum_{i,j,\ell=1}^{m}T_{kij\ell}z_i z_j z_\ell, \tag*{(33)} [ C 3 ( Σ 1/2 z ) ] k = i , j , ℓ = 1 ∑ m T k ij ℓ z i z j z ℓ , (33) with T T T symmetric in its last three indices. Then
c 3 2 = 6 ∑ k , i , j , ℓ T k i j ℓ 2 , e 2 ( t ) 2 = 3 2 ∥ T ∥ F 2 t 4 + O ( t 5 ) . (34) c_3^2=6\sum_{k,i,j,\ell}T_{kij\ell}^2,\qquad e_2(t)^2=\frac{3}{2}\lVert T\rVert_F^2t^4+O(t^5). \tag*{(34)} c 3 2 = 6 k , i , j , ℓ ∑ T k ij ℓ 2 , e 2 ( t ) 2 = 2 3 ∥ T ∥ F 2 t 4 + O ( t 5 ) . (34) For a fixed unit output direction u u u , the corresponding coefficient is obtained by replacing T k i j ℓ T_{kij\ell} T k ij ℓ by ∑ k u k T k i j ℓ \sum_k u_kT_{kij\ell} ∑ k u k T k ij ℓ .
Proof of Corollary C.3. Write X 0 = a + Σ 1 / 2 Z X_0=a+\Sigma^{1/2}Z X 0 = a + Σ 1/2 Z , where Z ∼ N ( 0 , I m ) Z\sim\mathcal{N}(0,I_m) Z ∼ N ( 0 , I m ) . An invertible affine coordinate change preserves polynomial degree. The shift by a a a changes C 3 C_3 C 3 only by terms of degree at most two after its leading homogeneous part is extracted. For standard Gaussian inputs, the part of Z i Z j Z ℓ Z_iZ_jZ_\ell Z i Z j Z ℓ orthogonal to degrees at most two is
: Z i Z j Z ℓ : = Z i Z j Z ℓ − δ i j Z ℓ − δ i ℓ Z j − δ j ℓ Z i . : Z_iZ_jZ_\ell:= Z_iZ_jZ_\ell-\delta_{ij}Z_\ell-\delta_{i\ell}Z_j-\delta_{j\ell}Z_i. : Z i Z j Z ℓ := Z i Z j Z ℓ − δ ij Z ℓ − δ i ℓ Z j − δ j ℓ Z i .
Their covariance is the sum of the six matchings between the three indices in the first factor and those in the second factor. One direct derivation differentiates the identity
E [ e s ⊤ Z − ∣ s ∣ 2 / 2 e t ⊤ Z − ∣ t ∣ 2 / 2 ] = e s ⊤ t \mathbb{E}\left[e^{s^{\top}Z-\lvert s\rvert^2/2}e^{t^{\top}Z-\lvert t\rvert^2/2}\right]=e^{s^{\top}t} E [ e s ⊤ Z − ∣ s ∣ 2 /2 e t ⊤ Z − ∣ t ∣ 2 /2 ] = e s ⊤ t
three times in s s s and three times in t t t , at zero. Contracting against the symmetric tensor T T T gives
E ∣ ∑ i , j , ℓ T k i j ℓ : Z i Z j Z ℓ : ∣ 2 = 6 ∑ i , j , ℓ T k i j ℓ 2 . \mathbb{E}\left|\sum_{i,j,\ell}T_{kij\ell}:Z_iZ_jZ_\ell:\right|^2=6\sum_{i,j,\ell}T_{kij\ell}^2. E i , j , ℓ ∑ T k ij ℓ : Z i Z j Z ℓ : 2 = 6 i , j , ℓ ∑ T k ij ℓ 2 .
Summation in k k k proves the first identity; Theorem C.2 gives the stated leading term in e 2 ( t ) 2 e_2(t)^2 e 2 ( t ) 2 . Contracting the output index gives the directional version. □ \square □
Affine changes of observed coordinates.
Let B B B be invertible, μ ~ = ( B ⋅ + b ) # μ \widetilde{\mu}=(B\cdot+b)_{\#}\mu μ = ( B ⋅ + b ) # μ , and f ~ ( u ) = f ( B − 1 ( u − b ) ) \widetilde{f}(u)=f(B^{-1}(u-b)) f ( u ) = f ( B − 1 ( u − b )) . Composition with B − 1 ( ⋅ − b ) B^{-1}(\cdot-b) B − 1 ( ⋅ − b ) is an isometry from L 2 ( μ ) L^2(\mu) L 2 ( μ ) onto L 2 ( μ ~ ) L^2(\widetilde{\mu}) L 2 ( μ ) and a bijection of degree-r r r polynomial spaces. Taking the infimum over those spaces gives exact equality of the approximation errors, including their leading coefficients. If outputs are additionally transformed to C f + d Cf+d C f + d , with C C C invertible and r ≥ 0 r\ge0 r ≥ 0 , candidates correspond as p ↦ C p + d p\mapsto Cp+d p ↦ C p + d . The inequalities
σ min ( C ) ∣ f − p ∣ ≤ ∣ C ( f − p ) ∣ ≤ σ max ( C ) ∣ f − p ∣ \sigma_{\min}(C)\lvert f-p\rvert\le\lvert C(f-p)\rvert\le\sigma_{\max}(C)\lvert f-p\rvert σ m i n ( C ) ∣ f − p ∣ ≤ ∣ C ( f − p )∣ ≤ σ m a x ( C ) ∣ f − p ∣
give the bounds after integration and minimization. An isometric output transformation gives equality. Thus membership of the vector coefficients, and their first escape index, is unchanged. This argument transports the input law and output metric consistently; arbitrary rescaling while holding a different law fixed is a different comparison.
Examples and comparisons.
The results above apply differently to the systems considered in the paper and to related examples:
Galerkin Euler. The quadratic vorticity interaction conserves enstrophy and therefore satisfies Eq. (C.5). Any observed subspace for which Q ( E x ) ≢ 0 Q(Ex) \not\equiv0 Q ( E x ) ≡ 0 has the nonzero cubic term described above.
Surface quasi-geostrophic flow. A finite Galerkin truncation of SQG also has quadratic transport. The conclusion of Theorem C.2 applies when its assumptions hold, in particular when Q ∘ E ≢ 0 Q \circ E \not\equiv0 Q ∘ E ≡ 0 for the chosen observed modes.
Lorenz–96. The quadratic part conserves the squared Euclidean norm, while damping and forcing appear in the lower-order terms. For d ≥ 4 d \ge4 d ≥ 4 , the cubic term is nonzero when the full state is observed. Under partial observation, the result applies whenever Q ( E x ) ≢ 0 Q(Ex) \not\equiv0 Q ( E x ) ≡ 0 . Lorenz–96 is included here as a standard quadratic comparison rather than as one of our benchmark systems.
Conditional rotation. After including the conditioning variable in the state, the vector field is
( 0 , ω c J y ) , (0,\omega c J y), ( 0 , ω c J y ) ,
which is quadratic and preserves the norm of y y y . Its finite-time conditional mean is
R ( ω t c ) y , R(\omega t c)y, R ( ω t c ) y ,
which has nonpolynomial dependence on the conditioning variable c c c . This example shows explicitly how repeated quadratic interactions can produce a much more complicated finite-time forecast.
Prescribed passive scalar. This benchmark provides a linear comparison. Its evolution is linear in the initial scalar field, so it does not create the same higher-degree dependence. Under independent coarse–fine initialization, its conditional mean remains affine at every forecast horizon.
The heterogeneous-wave benchmark is also linear in the full physical state, although partial observation of that state still makes the forecast probabilistic. The nonlinear Klein–Gordon benchmark contains a cubic nonlinearity and is therefore not covered by the quadratic theorem. The general first-escape result in Theorem C.1 can still be applied whenever its regularity and moment assumptions hold.
The passive-scalar statement is made precise by the following proposition.
Proposition C.4. Suppose that the full state evolves linearly according to
Z t = U t Z 0 , Z_t = U_t Z_0, Z t = U t Z 0 , where U t : R d → R d U_t : \mathbb{R}^{d} \to\mathbb{R}^{d} U t : R d → R d is the linear solution operator at time t t t . Assume that X 0 X_0 X 0 and η \eta η are independent. Then
m t ( x ) = P U t E x + P U t E [ η ] m_t(x) = P U_t E x + P U_t \mathbb{E}[\eta] m t ( x ) = P U t E x + P U t E [ η ] for every forecast horizon t t t . Hence both the affine and quadratic approximation errors are zero. If η \eta η is Gaussian, then the conditional distribution of the future observed state is Gaussian, with an affine mean and a covariance that does not depend on x x x .
The result follows by substituting Z 0 = E X 0 + η Z_0 = EX_0 + \eta Z 0 = E X 0 + η into Z t = U t Z 0 Z_t = U_t Z_0 Z t = U t Z 0 and conditioning on X 0 = x X_0 = x X 0 = x ; see the proof below.
Proof of the cubic estimate and the linear comparison.
Under (C.5), the radial derivative of a solution of (C.4) is bounded above by ∥ A ∥ o p ∣ Z ∣ + ∣ f ∣ \lVert A\rVert_{\mathrm{op}}\lvert Z\rvert+\lvert f\rvert ∥ A ∥ op ∣ Z ∣ + ∣ f ∣ . Applying the same argument to the time-reversed equation gives, for every finite real t t t ,
∣ Φ t ( z ) ∣ ≤ e a ∣ t ∣ ( ∣ z ∣ + b ∣ t ∣ ) , a = ∥ A ∥ o p , b = ∣ f ∣ . (35) \lvert\Phi_t(z)\rvert\le e^{a\lvert t\rvert}(\lvert z\rvert+b\lvert t\rvert), \qquad a=\lVert A\rVert_{\mathrm{op}},\quad b=\lvert f\rvert. \tag*{(35)} ∣ Φ t ( z )∣ ≤ e a ∣ t ∣ (∣ z ∣ + b ∣ t ∣) , a = ∥ A ∥ op , b = ∣ f ∣ . (35)
One can justify the radial inequality at zero by differentiating ∣ Z ∣ 2 + ε \sqrt{|Z|^2+\varepsilon} ∣ Z ∣ 2 + ε and sending ε \varepsilon ε to zero. Local existence and continuation for a polynomial vector field then give global existence. For A = f = 0 A=f=0 A = f = 0 , the norm is exactly conserved.
Set A = D V V \mathcal{A}=DV\,V A = D V V and B = D A V \mathcal{B}=D\mathcal{A}\,V B = D A V . They are polynomials of degree at most three and four. Along a trajectory they are the second and third time derivatives of the state. Therefore
P Φ t ( z ) = P z + t P V ( z ) + 1 2 t 2 P A ( z ) + ∫ 0 t 1 2 ( t − s ) 2 P B ( Φ s ( z ) ) d s . (36) P\Phi_t(z)=Pz+tPV(z)+\frac{1}{2}t^2P\mathcal{A}(z)+\int_0^t\frac{1}{2}(t-s)^2P\mathcal{B}(\Phi_s(z))\,\mathrm{d}s. \tag*{(36)} P Φ t ( z ) = P z + tP V ( z ) + 2 1 t 2 P A ( z ) + ∫ 0 t 2 1 ( t − s ) 2 P B ( Φ s ( z )) d s . (36)
For a fixed t 0 > 0 t_0>0 t 0 > 0 , the following is one explicit permissible bound:
K 3 = sup z ∈ R d ∣ P B ( z ) ∣ ( 1 + ∣ z ∣ ) 4 < ∞ , G = e a t 0 ( 1 + b t 0 ) , K_3=\sup_{z\in\mathbb{R}^d}\frac{|P\mathcal{B}(z)|}{(1+|z|)^4}<\infty,\qquad G=e^{at_0}(1+bt_0), K 3 = z ∈ R d sup ( 1 + ∣ z ∣ ) 4 ∣ P B ( z ) ∣ < ∞ , G = e a t 0 ( 1 + b t 0 ) ,
M 3 = K 3 G 4 [ E ( 1 + ∣ Z 0 ∣ ) 8 ] 1 / 2 . (37) M_3=K_3G^4\left[\mathbb{E}(1+|Z_0|)^8\right]^{1/2}. \tag*{(37)} M 3 = K 3 G 4 [ E ( 1 + ∣ Z 0 ∣ ) 8 ] 1/2 . (37)
All quantities are determined by the polynomial coefficients, P P P , the initial eighth moment, and t 0 t_0 t 0 . Indeed, 1 + ∣ Φ s ( z ) ∣ ≤ G ( 1 + ∣ z ∣ ) 1+|\Phi_s(z)|\leq G(1+|z|) 1 + ∣ Φ s ( z ) ∣ ≤ G ( 1 + ∣ z ∣ ) on this interval. Minkowski’s inequality in (C.18) bounds the remainder in L 2 L^2 L 2 by M 3 t 3 / 6 M_3t^3/6 M 3 t 3 /6 . Conditional expectation is an L 2 L^2 L 2 contraction, so
m t ( x ) = x + t a 1 ( x ) + 1 2 t 2 a 2 ( x ) + E t ( x ) , a 1 ( x ) = E [ P V ( Z 0 ) ∣ X 0 = x ] , a 2 ( x ) = E [ P A ( Z 0 ) ∣ X 0 = x ] , (38) \begin{aligned}
m_t(x)&=x+ta_1(x)+\frac{1}{2}t^2a_2(x)+\mathcal{E}_t(x),\\
a_1(x)&=\mathbb{E}[PV(Z_0)\mid X_0=x],\\
a_2(x)&=\mathbb{E}[P\mathcal{A}(Z_0)\mid X_0=x], \tag*{(38)}
\end{aligned} m t ( x ) a 1 ( x ) a 2 ( x ) = x + t a 1 ( x ) + 2 1 t 2 a 2 ( x ) + E t ( x ) , = E [ P V ( Z 0 ) ∣ X 0 = x ] , = E [ P A ( Z 0 ) ∣ X 0 = x ] , (38)
with ∥ E t ∥ L 2 ( μ ) ≤ M 3 t 3 / 6 \|\mathcal{E}_t\|_{L^2(\mu)}\leq M_3t^3/6 ∥ E t ∥ L 2 ( μ ) ≤ M 3 t 3 /6 .
Proof of Theorem C.2. Independence in (C.6) implies a 1 ∈ P 2 a_1\in\mathcal{P}_2 a 1 ∈ P 2 . The homogeneous cubic part of D V ( z ) V ( z ) DV(z)V(z) D V ( z ) V ( z ) is D Q ( z ) Q ( z ) DQ(z)Q(z) D Q ( z ) Q ( z ) ; all remaining terms have degree at most two. Substituting z = E x + η z=Ex+\eta z = E x + η , every term with at least one hidden factor has at most two visible factors. Its expectation is therefore a polynomial of degree at most two. Thus
a 2 ( x ) = C 3 ( x ) + p 2 ( x ) , p 2 ∈ P 2 . a_2(x)=C_3(x)+p_2(x),\qquad p_2\in\mathcal{P}_2. a 2 ( x ) = C 3 ( x ) + p 2 ( x ) , p 2 ∈ P 2 .
Differentiate z ⊤ Q ( z ) = 0 z^\top Q(z)=0 z ⊤ Q ( z ) = 0 in direction Q ( z ) Q(z) Q ( z ) : ∣ Q ( z ) ∣ 2 + z ⊤ D Q ( z ) Q ( z ) = 0 |Q(z)|^2+z^\top DQ(z)Q(z)=0 ∣ Q ( z ) ∣ 2 + z ⊤ D Q ( z ) Q ( z ) = 0 . At z = E x z=Ex z = E x this is (C.14), because E = P ⊤ E=P^\top E = P ⊤ . Hence Q ∘ E ≢ 0 Q\circ E\not\equiv0 Q ∘ E ≡ 0 implies C 3 ≢ 0 C_3\not\equiv0 C 3 ≡ 0 . A nonzero homogeneous cubic cannot agree with a quadratic polynomial on an open set. Positivity of the density there and closedness of the finite-dimensional polynomial subspace imply c 3 > 0 c_3>0 c 3 > 0 .
Applying I − Π 2 I-\Pi_2 I − Π 2 to (C.20) leaves
( I − Π 2 ) m t = 1 2 t 2 ( I − Π 2 ) C 3 + ( I − Π 2 ) E t . (I-\Pi_2)m_t=\frac{1}{2}t^2(I-\Pi_2)C_3+(I-\Pi_2)\mathcal{E}_t. ( I − Π 2 ) m t = 2 1 t 2 ( I − Π 2 ) C 3 + ( I − Π 2 ) E t .
The projection is a contraction; the reverse triangle inequality proves (C.12). Squaring proves (C.13). For example e 2 ( t ) ≥ c 3 t 2 / 4 e_2(t)\geq c_3t^2/4 e 2 ( t ) ≥ c 3 t 2 /4 whenever M 3 t ≤ 3 c 3 / 2 M_3t\leq3c_3/2 M 3 t ≤ 3 c 3 /2 .
Proof of Proposition C.4. Substitute Z 0 = E X 0 + η Z_0=EX_0+\eta Z 0 = E X 0 + η into Z t = U t Z 0 Z_t=U_tZ_0 Z t = U t Z 0 and condition on X 0 = x X_0=x X 0 = x . Independence gives E [ η ∣ X 0 = x ] = E η \mathbb{E}[\eta\mid X_0=x]=\mathbb{E}\eta E [ η ∣ X 0 = x ] = E η . If η \eta η is Gaussian, P U t η PU_t\eta P U t η is Gaussian with covariance P U t Cov ( η ) U t ⊤ P ⊤ PU_t\operatorname{Cov}(\eta)U_t^\top P^\top P U t Cov ( η ) U t ⊤ P ⊤ , which is independent of x x x .
Verification for the quadratic examples.
For a finite Fourier–Galerkin truncation of two-dimensional Euler, the vorticity field obeys ∂ t ω = − u ⋅ ∇ ω \partial_t\omega=-u\cdot\nabla\omega ∂ t ω = − u ⋅ ∇ ω and ∇ ⋅ u = 0 \nabla\cdot u=0 ∇ ⋅ u = 0 . Consequently,
d d t 1 2 ∥ ω ∥ L 2 2 = − 1 2 ∫ u ⋅ ∇ ( ω 2 ) d x = 0. \frac{\mathrm{d}}{\mathrm{d}t}\frac{1}{2}\|\omega\|_{L^2}^2=-\frac{1}{2}\int u\cdot\nabla(\omega^2)\,\mathrm{d}x=0. d t d 2 1 ∥ ω ∥ L 2 2 = − 2 1 ∫ u ⋅ ∇ ( ω 2 ) d x = 0.
In orthonormal real Fourier coordinates, the Galerkin vector field is homogeneous quadratic and satisfies z ⊤ Q ( z ) = 0 z^\top Q(z)=0 z ⊤ Q ( z ) = 0 . The condition Q ∘ E ≢ 0 Q\circ E\not\equiv0 Q ∘ E ≡ 0 still needs to be checked for the chosen observed subspace. For example, observed modes ( 1 , 1 ) (1,1) ( 1 , 1 ) and ( 2 , 0 ) (2,0) ( 2 , 0 ) interact to produce mode ( 3 , 1 ) (3,1) ( 3 , 1 ) with a nonzero coefficient in a truncation that retains this mode. The output of this first interaction may itself be unresolved: Theorem C.2 only requires it to be nonzero in the full Galerkin state.
For Lorenz–96 with cyclic indices i = 1 , … , d i = 1,\ldots,d i = 1 , … , d , d ≥ 4 d \ge4 d ≥ 4 , write
V i ( z ) = ( z i + 1 − z i − 2 ) z i − 1 − z i + F , Q i ( z ) = ( z i + 1 − z i − 2 ) z i − 1 . V_i(z) = (z_{i+1} - z_{i-2})z_{i-1} - z_i + F,\qquad Q_i(z) = (z_{i+1} - z_{i-2})z_{i-1}. V i ( z ) = ( z i + 1 − z i − 2 ) z i − 1 − z i + F , Q i ( z ) = ( z i + 1 − z i − 2 ) z i − 1 .
Relabeling indices gives
∑ i z i Q i ( z ) = ∑ i z i − 1 z i z i + 1 − ∑ i z i − 2 z i − 1 z i = 0. \sum_i z_i Q_i(z) = \sum_i z_{i-1}z_i z_{i+1} - \sum_i z_{i-2}z_{i-1}z_i = 0. i ∑ z i Q i ( z ) = i ∑ z i − 1 z i z i + 1 − i ∑ z i − 2 z i − 1 z i = 0.
Since Q ≢ 0 Q \not\equiv0 Q ≡ 0 , the certificate (C.14) proves that its full-state cubic term is nonzero. A particular coarse projection requires the additional check Q ∘ E ≢ 0 Q \circ E \not\equiv0 Q ∘ E ≡ 0 . The damping and forcing have lower degree and cannot cancel that term. For the conditional rotation example, Q ( c , y ) = ( 0 , ω J y ) Q(c,y) = (0,\omega J y) Q ( c , y ) = ( 0 , ω J y ) is likewise quadratic and satisfies ( c , y ) ⊤ Q ( c , y ) = 0 (c,y)^\top Q(c,y) = 0 ( c , y ) ⊤ Q ( c , y ) = 0 .
Why a cascade can reduce approximation error
We now isolate the first effect in the cascading trade-off: shorter transitions can reduce approximation error. The previous subsection showed that a single quadratic function may fail to represent the conditional mean over the full forecast horizon. We now show that the same quadratic class can give a more accurate endpoint forecast when it is fitted over a shorter interval and then composed repeatedly.
Partial observation creates a separate difficulty because each stage of a cascade may discard information carried by unresolved variables. We set that issue aside in this subsection and assume that the full state is observed. Thus there is no unresolved component, and the exact forecast over an interval of length h h h is the deterministic flow map
z ⟼ Φ h ( z ) . z \longmapsto\Phi_h(z). z ⟼ Φ h ( z ) .
Let
B R = { z ∈ R d : ∣ z ∣ ≤ R } , B_R = \{z \in\mathbb{R}^d : |z| \le R\}, B R = { z ∈ R d : ∣ z ∣ ≤ R } ,
and let ν \nu ν be a probability distribution on B R B_R B R whose density, relative to the uniform distribution, is bounded above and below by positive constants. We use ν \nu ν to measure approximation errors in this subsection.
Let Π 2 ν \Pi_2^\nu Π 2 ν denote the orthogonal projection in L 2 ( ν ) L^2(\nu) L 2 ( ν ) onto the space P 2 ( R d ; R d ) \mathcal{P}_2(\mathbb{R}^d;\mathbb{R}^d) P 2 ( R d ; R d ) of quadratic functions from the full state space to itself. For a physical transition of length h h h , define
F h ( 2 ) = Π 2 ν Φ h . F_h^{(2)} = \Pi_2^\nu\Phi_h. F h ( 2 ) = Π 2 ν Φ h .
Thus, F h ( 2 ) F_h^{(2)} F h ( 2 ) is the best quadratic approximation of the exact flow map Φ h \Phi_h Φ h under the distribution ν \nu ν . Its approximation error is
e 2 f u l l ( h ) = ∥ ( I − Π 2 ν ) Φ h ∥ L 2 ( ν ) . e_2^{\mathrm{full}}(h) = \left\| (I - \Pi_2^\nu)\Phi_h \right\|_{L^2(\nu)}. e 2 full ( h ) = ∥ ( I − Π 2 ν ) Φ h ∥ L 2 ( ν ) .
The superscript “full” distinguishes this error from e 2 ( t ) e_2(t) e 2 ( t ) , which was defined earlier for a forecast based only on the observed state.
Fix a final forecast horizon T T T . A direct quadratic forecast uses the single map
A cascade of depth S S S divides the same physical horizon into intervals of length
δ = T S \delta= \frac{T}{S} δ = S T
and repeatedly applies the short-transition approximation F δ ( 2 ) F_\delta^{(2)} F δ ( 2 ) . Its endpoint map is therefore
( F δ ( 2 ) ) ∘ S . \left(F_\delta^{(2)}\right)^{\circ S}. ( F δ ( 2 ) ) ∘ S .
The following theorem compares these two forecasts. It shows that a single quadratic map over the full horizon has a nonzero approximation error, while the endpoint error obtained by composing the fitted short-transition map converges to zero as the transition length decreases.
Theorem C.5. Suppose first that
and that Q Q Q satisfies Eq. (C.5). Then, for every T > 0 T > 0 T > 0 ,
e 2 f u l l ( T ) > 0. e^{\mathrm{full}}_2(T) > 0. e 2 full ( T ) > 0. Therefore, the exact flow map Φ T \Phi_T Φ T cannot be represented exactly by a quadratic function. There are constants D 2 , Λ 2 < ∞ D_2, \Lambda_2 < \infty D 2 , Λ 2 < ∞ , independent of S S S , such that, for all sufficiently large S S S ,
sup z ∈ B R ∣ ( F δ ( 2 ) ) ∘ S ( z ) − Φ T ( z ) ∣ ≤ D 2 e Λ 2 T − 1 Λ 2 δ . (39) \sup_{z \in B_R} \left| \left(F_\delta^{(2)}\right)^{\circ S}(z) - \Phi_T(z) \right| \le D_2 \frac{e^{\Lambda_2 T}-1}{\Lambda_2}\delta. \tag*{(39)} z ∈ B R sup ( F δ ( 2 ) ) ∘ S ( z ) − Φ T ( z ) ≤ D 2 Λ 2 e Λ 2 T − 1 δ . (39) When Λ 2 = 0 \Lambda_2 = 0 Λ 2 = 0 , the quotient is understood to be T T T . Consequently, for sufficiently large S S S , the cascade of short quadratic fits has smaller endpoint approximation error than every direct quadratic function.
For the more general dynamics
V ( z ) = Q ( z ) + A z + f , V(z) = Q(z) + Az + f, V ( z ) = Q ( z ) + A z + f , the same convergence bound holds on a forward-invariant ball. If Q ≠ 0 Q \ne0 Q = 0 , the direct approximation error is positive for every sufficiently small nonzero T T T .
The bound in Eq. (C.21) tends to zero as the transition length δ \delta δ tends to zero. Thus, composing short quadratic fits can approach the exact endpoint map even though no single quadratic function can represent that map exactly.
The same result holds for every fixed polynomial degree r ≥ 2 r \ge2 r ≥ 2 , with an endpoint-error bound of order δ r − 1 \delta^{r-1} δ r − 1 . The proof of this more general statement, together with a finite-sample result for quadratic fitting, is given below in the proof of Theorem C.5.
Proof of Theorem C.5.
Write A n = L V n Id A_n = \mathcal{L}_V^n \operatorname{Id} A n = L V n Id , with A 0 ( z ) = z A_0(z) = z A 0 ( z ) = z . Since deg V ≤ 2 \deg V \le2 deg V ≤ 2 , deg A n ≤ n + 1 \deg A_n \le n + 1 deg A n ≤ n + 1 . The homogeneous component of degree n + 1 n+1 n + 1 is L Q n Id \mathcal{L}_Q^n \operatorname{Id} L Q n Id .
Lemma C.6. If Q ≠ 0 Q \ne0 Q = 0 is homogeneous quadratic and norm preserving, then L Q n Id ≠ 0 \mathcal{L}_Q^n \operatorname{Id} \ne0 L Q n Id = 0 for every integer n ≥ 1 n \ge1 n ≥ 1 .
Proof. Otherwise the n n n th time derivative of every Q Q Q -trajectory would vanish identically. Each trajectory would be a polynomial in time of degree at most n − 1 n-1 n − 1 . It exists for all real time and has constant norm. Every bounded real polynomial is constant, so all trajectories would be constant. Their derivative at zero would imply Q = 0 Q = 0 Q = 0 , a contradiction.
On a forward-invariant ball, Taylor’s formula gives
Φ t = ∑ n = 0 r t n n ! A n + E t , sup B R ∣ E t ∣ ≤ t r + 1 ( r + 1 ) ! sup B R ∣ A r + 1 ∣ . \Phi_t = \sum_{n=0}^{r} \frac{t^n}{n!} A_n + \mathcal{E}_t,\qquad\sup_{B_R} |\mathcal{E}_t| \le\frac{t^{r+1}}{(r+1)!}\sup_{B_R}|A_{r+1}|. Φ t = n = 0 ∑ r n ! t n A n + E t , B R sup ∣ E t ∣ ≤ ( r + 1 )! t r + 1 B R sup ∣ A r + 1 ∣.
For n < r n < r n < r , A n ∈ P r A_n \in\mathcal{P}_r A n ∈ P r . Lemma C.6 shows that A r A_r A r has a nonzero homogeneous component of degree r + 1 r+1 r + 1 . Consequently
∣ e r f u l l ( t ) − c r t r r ! ∣ ≤ t r + 1 ( r + 1 ) ! sup B R ∣ A r + 1 ∣ , c r = ∥ ( I − Π r ν ) A r ∥ L 2 ( ν ) > 0. (40) \left|e_r^{\mathrm{full}}(t) - \frac{c_r t^r}{r!}\right| \le\frac{t^{r+1}}{(r+1)!}\sup_{B_R}|A_{r+1}|,\qquad c_r = \|(I-\Pi_r^\nu)A_r\|_{L^2(\nu)} > 0. \tag*{(40)} e r full ( t ) − r ! c r t r ≤ ( r + 1 )! t r + 1 B R sup ∣ A r + 1 ∣ , c r = ∥ ( I − Π r ν ) A r ∥ L 2 ( ν ) > 0. (40)
This establishes the small-time claim for V = Q + A z + f V = Q + Az + f V = Q + A z + f , including Lorenz–96. It uses a finite Taylor remainder, not convergence of an infinite Lie series at the terminal horizon.
For the pure field V = Q V = Q V = Q , the direct gap is positive at every T > 0 T > 0 T > 0 . Indeed, if e r f u l l ( T ) = 0 e_r^{\mathrm{full}}(T) = 0 e r full ( T ) = 0 , the finite-dimensional space is closed, so Φ T = p \Phi_T = p Φ T = p for a polynomial p p p on B R B_R B R , first almost everywhere and then everywhere by continuity and the density assumption. Norm conservation implies the polynomial identity ∣ p ( z ) ∣ 2 = ∣ z ∣ 2 |p(z)|^2 = |z|^2 ∣ p ( z ) ∣ 2 = ∣ z ∣ 2 on all of R d \mathbb{R}^d R d . If p p p had degree greater than one, its leading squared norm could not vanish. Hence p p p is affine. But Φ T ( 0 ) = 0 \Phi_T(0) = 0 Φ T ( 0 ) = 0 and D Φ T ( 0 ) = I D\Phi_T(0) = I D Φ T ( 0 ) = I , so p = Id p = \operatorname{Id} p = Id . The second variational equation at the equilibrium zero is
D 2 Φ T ( 0 ) [ a , b ] = T D 2 Q ( 0 ) [ a , b ] ≠ 0 , D^2\Phi_T(0)[a,b] = T D^2Q(0)[a,b] \ne0, D 2 Φ T ( 0 ) [ a , b ] = T D 2 Q ( 0 ) [ a , b ] = 0 ,
which contradicts p = Id p = \operatorname{Id} p = Id .
Convergence of the fitted short-transition rule.
Finite-dimensional norm equivalence gives a constant C Π , r C_{\Pi,r} C Π , r with
∥ p ∥ C 1 ( B 2 R ) ≤ C Π , r ∥ p ∥ L 2 ( ν ) , p ∈ P r . (41) \|p\|_{C^1(B_{2R})} \le C_{\Pi,r}\|p\|_{L^2(\nu)}, \qquad p \in\mathcal{P}_r. \tag*{(41)} ∥ p ∥ C 1 ( B 2 R ) ≤ C Π , r ∥ p ∥ L 2 ( ν ) , p ∈ P r . (41)
Here the norm on the right is a genuine norm on polynomial coefficients. Let M r = sup B R ∣ A r ∣ M_r = \sup_{B_R} |A_r| M r = sup B R ∣ A r ∣ . The Taylor polynomial
T r − 1 , δ = ∑ n = 0 r − 1 δ n n ! A n T_{r-1,\delta} = \sum_{n=0}^{r-1} \frac{\delta^n}{n!} A_n T r − 1 , δ = n = 0 ∑ r − 1 n ! δ n A n
belongs to P r \mathcal{P}_r P r . The best L 2 ( ν ) L^2(\nu) L 2 ( ν ) approximation satisfies
F δ ( r ) = T r − 1 , δ + Π ν r ( Φ δ − T r − 1 , δ ) , ∥ Φ δ − T r − 1 , δ ∥ L 2 ( ν ) ≤ M r δ r / r ! . F_\delta^{(r)} = T_{r-1,\delta} + \Pi_\nu^r(\Phi_\delta- T_{r-1,\delta}), \qquad\|\Phi_\delta- T_{r-1,\delta}\|_{L^2(\nu)} \le M_r \delta^r/r!. F δ ( r ) = T r − 1 , δ + Π ν r ( Φ δ − T r − 1 , δ ) , ∥ Φ δ − T r − 1 , δ ∥ L 2 ( ν ) ≤ M r δ r / r ! .
For 0 < δ ≤ 1 0 < \delta\le1 0 < δ ≤ 1 , permissible constants are
D r = ( 1 + C Π , r ) M r / r ! , D_r = (1 + C_{\Pi,r})M_r/r!, D r = ( 1 + C Π , r ) M r / r ! ,
Λ r = ∑ n = 1 r − 1 1 n ! sup B 2 R ∥ D A n ∥ o p + C Π , r M r / r ! . (42) \Lambda_r = \sum_{n=1}^{r-1} \frac{1}{n!}\sup_{B_{2R}}\|D A_n\|_{\mathrm{op}} + C_{\Pi,r}M_r/r!. \tag*{(42)} Λ r = n = 1 ∑ r − 1 n ! 1 B 2 R sup ∥ D A n ∥ op + C Π , r M r / r ! . (42)
They yield
sup B R ∣ F δ ( r ) − Φ δ ∣ ≤ D r δ r , Lip ( F δ ( r ) ∣ B 2 R ) ≤ 1 + Λ r δ . \sup_{B_R}|F_\delta^{(r)} - \Phi_\delta| \le D_r\delta^r, \qquad\operatorname{Lip}(F_\delta^{(r)}|_{B_{2R}}) \le1 + \Lambda_r\delta. B R sup ∣ F δ ( r ) − Φ δ ∣ ≤ D r δ r , Lip ( F δ ( r ) ∣ B 2 R ) ≤ 1 + Λ r δ .
Let z i = Φ i δ ( z 0 ) z_i = \Phi_{i\delta}(z_0) z i = Φ i δ ( z 0 ) and z ^ i = ( F δ ( r ) ) ∘ i ( z 0 ) \widehat{z}_i = (F_\delta^{(r)})^{\circ i}(z_0) z i = ( F δ ( r ) ) ∘ i ( z 0 ) , starting in B R B_R B R . As long as z ^ i ∈ B 2 R \widehat{z}_i \in B_{2R} z i ∈ B 2 R ,
∣ z ^ i + 1 − z i + 1 ∣ ≤ ( 1 + Λ r δ ) ∣ z ^ i − z i ∣ + D r δ r . |\widehat{z}_{i+1} - z_{i+1}| \le(1 + \Lambda_r\delta)|\widehat{z}_i - z_i| + D_r\delta^r. ∣ z i + 1 − z i + 1 ∣ ≤ ( 1 + Λ r δ ) ∣ z i − z i ∣ + D r δ r .
Summing the recursion gives the bound in Theorem C.5 , with r = 2 r = 2 r = 2 ; the same argument gives order δ r − 1 \delta^{r-1} δ r − 1 for any fixed r ≥ 2 r \ge2 r ≥ 2 . If its right-hand side is at most R / 2 R/2 R /2 , every iterate lies in B 3 R / 2 B_{3R/2} B 3 R /2 , so induction justifies all the estimates. This condition holds for sufficiently large S S S because r ≥ 2 r \ge2 r ≥ 2 . Combining with the positive direct gaps above proves Theorem C.5 .
Fitting error and a finite-sample bound.
For r = 2 r = 2 r = 2 , let F ^ δ ∈ P 2 \widehat{F}_\delta\in\mathcal{P}_2 F δ ∈ P 2 and η δ = ∥ F ^ δ − F δ ( 2 ) ∥ L 2 ( ν ) \eta_\delta= \|\widehat{F}_\delta- F_\delta^{(2)}\|_{L^2(\nu)} η δ = ∥ F δ − F δ ( 2 ) ∥ L 2 ( ν ) . If Lip ( F ^ δ ∣ B 2 R ) ≤ 1 + Λ ^ δ \operatorname{Lip}(\widehat{F}_\delta|_{B_{2R}}) \le1 + \widehat{\Lambda}\delta Lip ( F δ ∣ B 2 R ) ≤ 1 + Λ δ , the same proof gives
sup B R ∣ F ^ δ S − Φ T ∣ ≤ e Λ ^ T − 1 Λ ^ ( D 2 δ + C Π , 2 η δ δ ) , (43) \sup_{B_R}|\widehat{F}_\delta^S - \Phi_T| \le\frac{e^{\widehat{\Lambda}T} - 1}{\widehat{\Lambda}}\left(D_2\delta+ \frac{C_{\Pi,2}\eta_\delta}{\delta}\right), \tag*{(43)} B R sup ∣ F δ S − Φ T ∣ ≤ Λ e Λ T − 1 ( D 2 δ + δ C Π , 2 η δ ) , (43)
provided the right-hand side is at most R / 2 R/2 R /2 . For example, η δ ≤ C δ 2 \eta_\delta\le C\delta^2 η δ ≤ C δ 2 implies the required stability with a constant Λ ^ \widehat{\Lambda} Λ independent of δ ≤ 1 \delta\le1 δ ≤ 1 , by (C.23). A small fitting error alone is therefore not enough to justify (C.25): we also need to control the Lipschitz constant of the fitted map.
We next bound η δ \eta_\delta η δ when the map is fitted from finitely many observations. Let p = ( d + 2 2 ) p = \binom{d+2}{2} p = ( 2 d + 2 ) , and let φ 1 , … , φ p \varphi_1,\ldots,\varphi_p φ 1 , … , φ p be an orthonormal basis of the scalar quadratic polynomials in L 2 ( ν ) L^2(\nu) L 2 ( ν ) . Write φ = ( φ 1 , … , φ p ) ⊤ \varphi= (\varphi_1,\ldots,\varphi_p)^\top φ = ( φ 1 , … , φ p ) ⊤ and set κ φ = sup B R ∣ φ ∣ 2 \kappa_\varphi= \sup_{B_R}|\varphi|^2 κ φ = sup B R ∣ φ ∣ 2 . Suppose that we observe n n n independent pairs
Z i ∼ ν , U i = Φ δ ( Z i ) + ζ i , E [ ζ i ∣ Z i ] = 0 , E [ ∣ ζ i ∣ 2 ∣ Z i ] ≤ v o b s 2 . Z_i \sim\nu, \qquad U_i = \Phi_\delta(Z_i) + \zeta_i, \qquad\mathbb{E}[\zeta_i \mid Z_i] = 0, \qquad\mathbb{E}[|\zeta_i|^2 \mid Z_i] \le v_{\mathrm{obs}}^2. Z i ∼ ν , U i = Φ δ ( Z i ) + ζ i , E [ ζ i ∣ Z i ] = 0 , E [ ∣ ζ i ∣ 2 ∣ Z i ] ≤ v obs 2 .
For every ϵ ∈ ( 0 , 1 ) \epsilon\in(0,1) ϵ ∈ ( 0 , 1 ) , with probability at least 1 − 4 κ φ p / n − ϵ 1 - 4\kappa_\varphi p/n - \epsilon 1 − 4 κ φ p / n − ϵ , the empirical Gram matrix is invertible and least squares satisfies
η δ ≤ 2 κ φ ( D 2 2 δ 4 + v o b s 2 ) n ϵ . (44) \eta_\delta\le2\sqrt{\frac{\kappa_\varphi(D_2^2\delta^4 + v_{\mathrm{obs}}^2)}{n\epsilon}}. \tag*{(44)} η δ ≤ 2 n ϵ κ φ ( D 2 2 δ 4 + v obs 2 ) . (44)
To verify this, let G n = n − 1 ∑ i φ i φ i ⊤ G_n = n^{-1}\sum_i \varphi_i\varphi_i^\top G n = n − 1 ∑ i φ i φ i ⊤ . Independence gives E ∥ G n − I ∥ F 2 ≤ κ φ p / n \mathbb{E}\|G_n - I\|_F^2 \le\kappa_\varphi p/n E ∥ G n − I ∥ F 2 ≤ κ φ p / n , so Markov’s inequality bounds the failure of ∥ G n − I ∥ o p ≤ 1 / 2 \|G_n - I\|_{\mathrm{op}} \le1/2 ∥ G n − I ∥ op ≤ 1/2 by 4 κ φ p / n 4\kappa_\varphi p/n 4 κ φ p / n . The mean-zero matrix
Z n = 1 n ∑ i [ Φ δ ( Z i ) − F δ ( 2 ) ( Z i ) + ζ i ] φ i ⊤ Z_n = \frac{1}{n}\sum_i [\Phi_\delta(Z_i) - F_\delta^{(2)}(Z_i) + \zeta_i]\varphi_i^\top Z n = n 1 i ∑ [ Φ δ ( Z i ) − F δ ( 2 ) ( Z i ) + ζ i ] φ i ⊤
has second moment at most κ ϕ ( D 4 2 δ 4 + v o b s 2 ) / n \kappa_{\phi}(D_{4}^{2}\delta^{4}+v_{\mathrm{obs}}^{2})/n κ ϕ ( D 4 2 δ 4 + v obs 2 ) / n . On the Gram event the coefficient error is Z n G n − 1 Z_{n}G_{n}^{-1} Z n G n − 1 of norm at most 2 ∥ Z n ∥ F 2\|Z_{n}\|_{F} 2∥ Z n ∥ F . Orthonormality identifies its Frobenius norm with the L 2 ( ν ) L^{2}(\nu) L 2 ( ν ) norm of the prediction error. A second Markov inequality proves (C.26).
In the noiseless case the fitting error is proportional to δ 2 / n \delta^{2}/\sqrt{n} δ 2 / n at fixed confidence. With fixed observation noise, the propagated contribution can instead grow like v o b s / ( δ n ) v_{\mathrm{obs}}/(\delta\sqrt{n}) v obs / ( δ n ) . This result concerns independent full-state regression pairs. Several transitions drawn from one trajectory do not constitute independent pairs, and physical hidden-state uncertainty is not identified with this observation-noise model.
Connection to the stochastic-interpolant learner
The previous subsection compared direct and cascaded forecasts at the level of their endpoint maps. The experiments in the main text implement these forecasts using stochastic interpolants. We now connect the approximation results above to a specific stochastic-interpolant learner. In the deterministic full-state setting of Theorem C.5, this gives an explicit setting in which Cascaded SI has smaller endpoint approximation error than Direct SI.
We begin by describing the stochastic-interpolant training problem. Fix a physical transition length h h h . Given the initial observation X 0 = x X_{0}=x X 0 = x , let
C = X h , C ∣ X 0 = x ∼ K h ( x , ⋅ ) , C=X_{h}, \qquad C\mid X_{0}=x\sim K_{h}(x,\cdot), C = X h , C ∣ X 0 = x ∼ K h ( x , ⋅ ) ,
denote the future state that we wish to generate. The physical transition length h h h is different from the interpolation variable τ ∈ [ 0 , 1 ] \tau\in[0,1] τ ∈ [ 0 , 1 ] used by the stochastic interpolant. We use the following Wiener stochastic interpolant:
Y τ = ( 1 − τ ) x + τ C + σ b ( 1 − τ ) τ ξ , R τ = C − x − σ b τ ξ , (45) Y_{\tau}=(1-\tau)x+\tau C+\sigma_{b}(1-\tau)\sqrt{\tau}\,\xi,\qquad R_{\tau}=C-x-\sigma_{b}\sqrt{\tau}\,\xi, \tag*{(45)} Y τ = ( 1 − τ ) x + τ C + σ b ( 1 − τ ) τ ξ , R τ = C − x − σ b τ ξ , (45)
where ξ ∼ N ( 0 , I ) \xi\sim\mathcal{N}(0,I) ξ ∼ N ( 0 , I ) is independent of ( X 0 , C ) (X_{0},C) ( X 0 , C ) and σ b > 0 \sigma_{b}>0 σ b > 0 controls the amount of noise. At τ = 0 \tau=0 τ = 0 , the interpolant starts from x x x , while it approaches the endpoint C C C as τ \tau τ approaches 1.
The pair ( Y τ , R τ ) (Y_{\tau},R_{\tau}) ( Y τ , R τ ) defines the regression problem used to train the stochastic interpolant. Let π \pi π denote the distribution from which the interpolation time τ \tau τ is sampled during training, independently of ( X 0 , C , ξ ) (X_{0},C,\xi) ( X 0 , C , ξ ) . For a candidate drift b b b , define the training objective by
J ( b ) = 1 2 E [ ∣ b ( Y τ , τ ; X 0 ) − R τ ∣ 2 ] , (46) \mathcal{J}(b)=\frac{1}{2}\mathbb{E}\left[\left|b(Y_{\tau},\tau;X_{0})-R_{\tau}\right|^{2}\right], \tag*{(46)} J ( b ) = 2 1 E [ ∣ b ( Y τ , τ ; X 0 ) − R τ ∣ 2 ] , (46)
where the expectation is taken over X 0 X_{0} X 0 , C C C , ξ \xi ξ , and τ ∼ π \tau\sim\pi τ ∼ π . We assume that the expectations appearing below are finite.
Under this objective, the optimal drift is
b ⋆ ( y , τ ; x ) = E [ R τ ∣ Y τ = y , X 0 = x ] . b^{\star}(y,\tau;x)=\mathbb{E}[R_{\tau}\mid Y_{\tau}=y,X_{0}=x]. b ⋆ ( y , τ ; x ) = E [ R τ ∣ Y τ = y , X 0 = x ] .
Since
R τ = C − Y τ 1 − τ , R_{\tau}=\frac{C-Y_{\tau}}{1-\tau}, R τ = 1 − τ C − Y τ ,
the optimal drift can also be written as
b ⋆ ( y , τ ; x ) = E [ C ∣ Y τ = y , X 0 = x ] − y 1 − τ . (47) b^{\star}(y,\tau;x)=\frac{\mathbb{E}[C\mid Y_{\tau}=y,X_{0}=x]-y}{1-\tau}. \tag*{(47)} b ⋆ ( y , τ ; x ) = 1 − τ E [ C ∣ Y τ = y , X 0 = x ] − y . (47)
Moreover, for any candidate drift b b b ,
J ( b ) − J ( b ⋆ ) = 1 2 E [ ∣ b ( Y τ , τ ; X 0 ) − b ⋆ ( Y τ , τ ; X 0 ) ∣ 2 ] . (48) \mathcal{J}(b)-\mathcal{J}(b^{\star})=\frac{1}{2}\mathbb{E}\left[\left|b(Y_{\tau},\tau;X_{0})-b^{\star}(Y_{\tau},\tau;X_{0})\right|^{2}\right]. \tag*{(48)} J ( b ) − J ( b ⋆ ) = 2 1 E [ ∣ b ( Y τ , τ ; X 0 ) − b ⋆ ( Y τ , τ ; X 0 ) ∣ 2 ] . (48)
Thus, the difference from the minimum training loss is exactly one half of the mean-square error between the candidate drift b b b and the optimal drift b ⋆ b^{\star} b ⋆ .
The identities above follow directly from the stated interpolant. For τ < 1 \tau<1 τ < 1 ,
C − Y τ = ( 1 − τ ) ( C − x − σ b τ ξ ) = ( 1 − τ ) R τ . C-Y_{\tau}=(1-\tau)(C-x-\sigma_{b}\sqrt{\tau}\,\xi)=(1-\tau)R_{\tau}. C − Y τ = ( 1 − τ ) ( C − x − σ b τ ξ ) = ( 1 − τ ) R τ .
Conditional expectation gives (C.29). Since b ⋆ b^\star b ⋆ is the conditional expectation of R τ R_\tau R τ given ( Y τ , τ , X 0 ) (Y_\tau,\tau,X_0) ( Y τ , τ , X 0 ) , the cross term in
∣ b − R τ ∣ 2 = ∣ b − b ⋆ ∣ 2 + ∣ b ⋆ − R τ ∣ 2 + 2 ( b − b ⋆ ) ⊤ ( b ⋆ − R τ ) |b-R_\tau|^2=|b-b^\star|^2+|b^\star-R_\tau|^2+2(b-b^\star)^\top(b^\star-R_\tau) ∣ b − R τ ∣ 2 = ∣ b − b ⋆ ∣ 2 + ∣ b ⋆ − R τ ∣ 2 + 2 ( b − b ⋆ ) ⊤ ( b ⋆ − R τ )
has zero expectation. This proves (C.30).
Before relating this objective to Direct SI and Cascaded SI, we connect the approximation of the conditional mean to the error in the generated endpoint distribution.
Proposition C.7. Let K ^ h ( x , ⋅ ) \widehat{K}_h(x,\cdot) K h ( x , ⋅ ) be any approximation of the true conditional distribution K h ( x , ⋅ ) K_h(x,\cdot) K h ( x , ⋅ ) . Define the mean of this approximate distribution by
m ^ h ( x ) = ∫ c K ^ h ( x , d c ) . \widehat{m}_h(x)=\int c\,\widehat{K}_h(x,\mathrm{d}c). m h ( x ) = ∫ c K h ( x , d c ) . Assume that
∫ R m ∫ R m ∣ c ∣ 2 [ K h ( x , d c ) + K ^ h ( x , d c ) ] μ ( d x ) < ∞ . \int_{\mathbb{R}^m}\int_{\mathbb{R}^m}|c|^2\left[K_h(x,\mathrm{d}c)+\widehat{K}_h(x,\mathrm{d}c)\right]\mu(\mathrm{d}x)<\infty. ∫ R m ∫ R m ∣ c ∣ 2 [ K h ( x , d c ) + K h ( x , d c ) ] μ ( d x ) < ∞. Then
∫ W 2 2 ( K h ( x , ⋅ ) , K ^ h ( x , ⋅ ) ) μ ( d x ) ≥ ∥ m h − m ^ h ∥ L 2 ( μ ) 2 . (49) \int W_2^2\left(K_h(x,\cdot),\widehat{K}_h(x,\cdot)\right)\mu(\mathrm{d}x)\geq\|m_h-\widehat{m}_h\|_{L^2(\mu)}^2. \tag*{(49)} ∫ W 2 2 ( K h ( x , ⋅ ) , K h ( x , ⋅ ) ) μ ( d x ) ≥ ∥ m h − m h ∥ L 2 ( μ ) 2 . (49)
Proof of Proposition C.7. For each x x x , let ( C , C ^ ) (C,\widehat{C}) ( C , C ) be any coupling of K h ( x , ⋅ ) K_h(x,\cdot) K h ( x , ⋅ ) and K ^ h ( x , ⋅ ) \widehat{K}_h(x,\cdot) K h ( x , ⋅ ) . Jensen’s inequality gives E ∣ C − C ^ ∣ 2 ≥ ∣ m h ( x ) − m ^ h ( x ) ∣ 2 \mathbb{E}|C-\widehat{C}|^2\geq|m_h(x)-\widehat{m}_h(x)|^2 E ∣ C − C ∣ 2 ≥ ∣ m h ( x ) − m h ( x ) ∣ 2 . Take the infimum over couplings and integrate over x x x .
Proposition C.7 shows that an error in the conditional mean necessarily produces an error in the generated conditional distribution. In particular, if m ^ h ∈ P 2 \widehat{m}_h\in\mathcal{P}_2 m h ∈ P 2 , then
∫ W 2 2 ( K h ( x , ⋅ ) , K ^ h ( x , ⋅ ) ) μ ( d x ) ≥ e 2 ( h ) 2 . \int W_2^2\left(K_h(x,\cdot),\widehat{K}_h(x,\cdot)\right)\mu(\mathrm{d}x)\geq e_2(h)^2. ∫ W 2 2 ( K h ( x , ⋅ ) , K h ( x , ⋅ ) ) μ ( d x ) ≥ e 2 ( h ) 2 .
Therefore, the approximation error identified in Theorem C.2 is also a lower bound on the endpoint distribution error of any learner whose conditional mean is restricted to be quadratic.
This first result concerns only the endpoint distributions. It does not yet connect the approximation error e 2 ( h ) e_2(h) e 2 ( h ) to the stochastic-interpolant training objective J \mathcal{J} J . To obtain such a relation, we consider a particular family of conditional distributions for which both the optimal drift and the generated endpoint distribution can be calculated exactly.
Proposition C.8. Suppose that
C ∣ X 0 = x ∼ N ( m h ( x ) , q h I ) , q h ≥ 0. C\mid X_0=x\sim\mathcal{N}(m_h(x),q_hI),\qquad q_h\geq0. C ∣ X 0 = x ∼ N ( m h ( x ) , q h I ) , q h ≥ 0. Assume that the interpolation-time law π \pi π is supported on [ 0 , τ ⋆ ] [0,\tau_\star] [ 0 , τ ⋆ ] for some τ ⋆ < 1 \tau_\star<1 τ ⋆ < 1 . Consider the family of stochastic-interpolant drifts b h , p b_{h,p} b h , p obtained by keeping the exact Gaussian dependence on y y y and τ \tau τ and replacing the true conditional mean m h ( x ) m_h(x) m h ( x ) with a candidate function p ( x ) p(x) p ( x ) . There is a constant w h > 0 w_h>0 w h > 0 such that
J ( b h , p ) − J ( b h , m h ) = 1 2 w h ∥ p − m h ∥ L 2 ( μ ) 2 . (50) \mathcal{J}(b_{h,p})-\mathcal{J}(b_{h,m_h})=\frac{1}{2}w_h\|p-m_h\|_{L^2(\mu)}^2. \tag*{(50)} J ( b h , p ) − J ( b h , m h ) = 2 1 w h ∥ p − m h ∥ L 2 ( μ ) 2 . (50) When b h , p b_{h,p} b h , p is integrated exactly, the generated endpoint distribution is
N ( p ( x ) , q h I ) . \mathcal{N}(p(x),q_hI). N ( p ( x ) , q h I ) . Consequently,
W 2 2 ( N ( m h ( x ) , q h I ) , N ( p ( x ) , q h I ) ) = ∣ m h ( x ) − p ( x ) ∣ 2 . (51) W_2^2\left(\mathcal{N}(m_h(x),q_hI),\mathcal{N}(p(x),q_hI)\right)=|m_h(x)-p(x)|^2. \tag*{(51)} W 2 2 ( N ( m h ( x ) , q h I ) , N ( p ( x ) , q h I ) ) = ∣ m h ( x ) − p ( x ) ∣ 2 . (51) Thus, for this family, approximating the conditional mean controls both the stochastic-interpolant training loss and the endpoint distribution error. In particular, choosing p = Π 2 m h p=\Pi_2m_h p = Π 2 m h gives
J ( b h , Π 2 m h ) − J ( b h , m h ) = 1 2 w h e 2 ( h ) 2 , \mathcal{J}(b_{h,\Pi_2m_h})-\mathcal{J}(b_{h,m_h})=\frac{1}{2}w_he_2(h)^2, J ( b h , Π 2 m h ) − J ( b h , m h ) = 2 1 w h e 2 ( h ) 2 , and
∫ W 2 2 ( N ( m h ( x ) , q h I ) , N ( Π 2 m h ( x ) , q h I ) ) μ ( d x ) = e 2 ( h ) 2 . \int W_2^2\left(\mathcal{N}(m_h(x),q_hI),\mathcal{N}(\Pi_2m_h(x),q_hI)\right)\mu(\mathrm{d}x)=e_2(h)^2. ∫ W 2 2 ( N ( m h ( x ) , q h I ) , N ( Π 2 m h ( x ) , q h I ) ) μ ( d x ) = e 2 ( h ) 2 .
We can now connect this result to the direct and cascaded forecasts from the previous subsection. There, the full state is observed and the dynamics are deterministic. In that setting, the exact endpoint is
C = Φ h ( x ) , C = \Phi_h(x), C = Φ h ( x ) ,
and hence
m h = Φ h . m_h = \Phi_h. m h = Φ h .
There is no conditional randomness in the endpoint. This is the case q h = 0 q_h = 0 q h = 0 in Proposition C.8; the zero covariance simply means that the endpoint distribution is concentrated at Φ h ( x ) \Phi_h(x) Φ h ( x ) .
Since w h > 0 w_h > 0 w h > 0 , the best quadratic function p p p minimizes
∥ p − Φ h ∥ L 2 ( ν ) 2 . \lVert p - \Phi_h \rVert_{L^2(\nu)}^2. ∥ p − Φ h ∥ L 2 ( ν ) 2 .
The optimal quadratic function is therefore precisely
p = Π 2 ν Φ h = F h ( 2 ) . p = \Pi^\nu_2 \Phi_h = F_h^{(2)}. p = Π 2 ν Φ h = F h ( 2 ) .
Moreover, the stochastic interpolant associated with p p p produces the deterministic endpoint p ( x ) p(x) p ( x ) . The fitted quadratic maps studied in Theorem C.5 are therefore exactly the endpoint maps produced by this stochastic-interpolant learner.
We can now state the resulting comparison between Direct SI and Cascaded SI.
Corollary C.9. Under the assumptions of Theorem C.5 , consider the stochastic-interpolant learner in Proposition C.8 with q h = 0 q_h = 0 q h = 0 and with p ∈ P 2 ( R d ; R d ) p \in\mathcal{P}_2(\mathbb{R}^d;\mathbb{R}^d) p ∈ P 2 ( R d ; R d ) . The endpoint map produced by Direct SI is
If the horizon is divided into S S S intervals of length
δ = T S , \delta= \frac{T}{S}, δ = S T , the endpoint map produced by Cascaded SI is
( F δ ( 2 ) ) ∘ S . \left(F_\delta^{(2)}\right)^{\circ S}. ( F δ ( 2 ) ) ∘ S . For all sufficiently large S S S , Cascaded SI has smaller endpoint approximation error than every Direct SI whose endpoint map is quadratic.
Corollary C.9 shows, in this specified full-state setting, that the approximation benefit proved in Theorem C.5 carries directly to the corresponding stochastic-interpolant learners. This result isolates approximation error. It does not yet include the information loss that can occur when the state passed between stages of a cascade is only partially observed.
Proof of Proposition C.8.
Suppose C ∣ x ∼ N ( m ( x ) , q I ) C \mid x \sim\mathcal{N}(m(x),qI) C ∣ x ∼ N ( m ( x ) , q I ) , q ≥ 0 q \ge0 q ≥ 0 . Define
D τ = τ q + σ b 2 ( 1 − τ ) 2 , K τ = q / D τ , H τ = σ b 2 ( 1 − τ ) / D τ , D_\tau= \tau q + \sigma_b^2(1-\tau)^2,\qquad K_\tau= q/D_\tau,\qquad H_\tau= \sigma_b^2(1-\tau)/D_\tau, D τ = τ q + σ b 2 ( 1 − τ ) 2 , K τ = q / D τ , H τ = σ b 2 ( 1 − τ ) / D τ ,
and, for a candidate p ( x ) p(x) p ( x ) , set
b p ( y , τ ; x ) = K τ − 1 1 − τ y − K τ x + H τ p ( x ) . (52) b_p(y,\tau;x) = \frac{K_\tau-1}{1-\tau}y - K_\tau x + H_\tau p(x). \tag*{(52)} b p ( y , τ ; x ) = 1 − τ K τ − 1 y − K τ x + H τ p ( x ) . (52)
At q = 0 q=0 q = 0 , this is ( p ( x ) − y ) / ( 1 − τ ) (p(x)-y)/(1-\tau) ( p ( x ) − y ) / ( 1 − τ ) . For q > 0 q>0 q > 0 , Gaussian conditioning proves b m = b ⋆ b_m=b^\star b m = b ⋆ , since
E [ C ∣ y , x ] = m ( x ) + K τ [ y − ( 1 − τ ) x − τ m ( x ) ] . \mathbb{E}[C\mid y,x] = m(x) + K_\tau\left[y-(1-\tau)x-\tau m(x)\right]. E [ C ∣ y , x ] = m ( x ) + K τ [ y − ( 1 − τ ) x − τ m ( x ) ] .
The same formula holds directly for q = 0 q=0 q = 0 . Thus b p − b m = H τ ( p − m ) b_p-b_m=H_\tau(p-m) b p − b m = H τ ( p − m ) .
Take an interpolation-time law π \pi π independent of ( x , C , ξ ) (x,C,\xi) ( x , C , ξ ) with 0 < w q : = ∫ H τ 2 π ( d τ ) < ∞ 0<w_q:=\int H_\tau^2\pi(\mathrm{d}\tau)<\infty 0 < w q := ∫ H τ 2 π ( d τ ) < ∞ and finite regression losses. One common law supported in [ 0 , τ ∗ ] [0,\tau_\ast] [ 0 , τ ∗ ] , τ ∗ < 1 \tau_\ast<1 τ ∗ < 1 , suffices for all q ≥ 0 q\ge0 q ≥ 0 . The analytic drift formula is specified on all of [ 0 , 1 ) [0,1) [ 0 , 1 ) ; only p p p is fitted. Equation (C.32) follows with w h = w q h w_h=w_{q_h} w h = w q h .
The affine SDE
d Y ‾ τ = b p ( Y ‾ τ , τ ; x ) d τ + σ b ( 1 − τ ) d W τ , Y ‾ 0 = x , (53) \mathrm{d}\overline{Y}_{\tau}=b_p(\overline{Y}_{\tau},\tau;x)\,\mathrm{d}\tau+\sigma_b(1-\tau)\,\mathrm{d}W_{\tau},\qquad\overline{Y}_0=x, \tag*{(53)} d Y τ = b p ( Y τ , τ ; x ) d τ + σ b ( 1 − τ ) d W τ , Y 0 = x , (53)
has, on every compact subinterval of [ 0 , 1 ) [0,1) [ 0 , 1 ) , a unique Gaussian solution. Its mean and covariance are
( 1 − τ ) x + τ p ( x ) , [ τ 2 q + σ b 2 τ ( 1 − τ ) 2 ] I . (1-\tau)x+\tau p(x),\qquad[\tau^2q+\sigma_b^2\tau(1-\tau)^2]I. ( 1 − τ ) x + τ p ( x ) , [ τ 2 q + σ b 2 τ ( 1 − τ ) 2 ] I .
Substitution into the linear mean and covariance equations verifies both expressions and their initial conditions. Taking τ ↑ 1 \tau\uparrow1 τ ↑ 1 gives the endpoint law N ( p ( x ) , q I ) \mathcal{N}(p(x),qI) N ( p ( x ) , q I ) . Equal-covariance Gaussians are translates: the translation coupling has cost ∣ p − m ∣ 2 \lvert p-m\rvert^2 ∣ p − m ∣ 2 , and Jensen’s inequality gives the matching lower bound. This proves Proposition C.8. When q > 0 q>0 q > 0 , the corresponding conditional KL is ∣ p ( x ) − m ( x ) ∣ 2 / ( 2 q ) \lvert p(x)-m(x)\rvert^2/(2q) ∣ p ( x ) − m ( x ) ∣ 2 / ( 2 q ) ; when q = 0 q=0 q = 0 , differing point masses have infinite KL and Wasserstein distance remains the appropriate finite metric.
What repeated conditioning in a cascade forgets
Corollary C.9 shows that a cascade of shorter transitions can reduce approximation error when the full state is observed. We now return to the partially observed setting. Here cascading introduces a second effect: at the end of each stage, the next transition is conditioned only on the current observed state. Information carried by the unresolved variables, and by their relation to earlier observations, may therefore be lost.
We first study this effect using exact short-transition distributions. There is therefore no learning error in the analysis below. Any difference between the direct conditional law and the cascade is caused only by repeated conditioning on the observed state.
Exact short-transition distributions.
We extend the notation introduced above by defining, for 0 ≤ s < t ≤ T 0\le s<t\le T 0 ≤ s < t ≤ T ,
K s , t ( x , ⋅ ) = Law ( X t ∣ X s = x ) , μ t = Law ( X t ) . K_{s,t}(x,\cdot)=\operatorname{Law}(X_t\mid X_s=x),\qquad\mu_t=\operatorname{Law}(X_t). K s , t ( x , ⋅ ) = Law ( X t ∣ X s = x ) , μ t = Law ( X t ) .
Thus,
K 0 , h = K h and μ 0 = μ . K_{0,h}=K_h\qquad\text{and}\qquad\mu_0=\mu. K 0 , h = K h and μ 0 = μ .
The kernel K s , t ( x , ⋅ ) K_{s,t}(x,\cdot) K s , t ( x , ⋅ ) is the exact conditional distribution of the observed state at time t t t , given only the observed state X s = x X_s=x X s = x .
For two conditional distributions K K K and L L L , we use the notation
D μ ( K , L ) = [ ∫ W 2 2 ( K ( x , ⋅ ) , L ( x , ⋅ ) ) μ ( d x ) ] 1 / 2 . \mathcal{D}_{\mu}(K,L)=\left[\int W_2^2(K(x,\cdot),L(x,\cdot))\,\mu(\mathrm{d}x)\right]^{1/2}. D μ ( K , L ) = [ ∫ W 2 2 ( K ( x , ⋅ ) , L ( x , ⋅ )) μ ( d x ) ] 1/2 .
Thus, D μ ( K , L ) \mathcal{D}_{\mu}(K,L) D μ ( K , L ) is their average conditional Wasserstein distance when the input state has distribution μ \mu μ .
Consider a partition
0 = t 0 < t 1 < ⋯ < t S = T , 0=t_0<t_1<\cdots<t_S=T, 0 = t 0 < t 1 < ⋯ < t S = T ,
and write
K i = K t i , t i + 1 , i = 0 , … , S − 1. K_i=K_{t_i,t_{i+1}},\qquad i=0,\ldots,S-1. K i = K t i , t i + 1 , i = 0 , … , S − 1.
The cascade built from the exact short-transition distributions is
K ( S ) = K 0 K 1 ⋯ K S − 1 . (54) K^{(S)}=K_0K_1\cdots K_{S-1}. \tag*{(54)} K ( S ) = K 0 K 1 ⋯ K S − 1 . (54)
Starting from X 0 = x X_0=x X 0 = x , this composition first samples from K 0 ( x , ⋅ ) K_0(x,\cdot) K 0 ( x , ⋅ ) and then repeatedly samples the next observed state conditioned only on the current one. We measure the difference between this exact-kernel cascade and the direct conditional distribution K 0 , T K_{0,T} K 0 , T by
C S = D μ 0 ( K 0 , T , K ( S ) ) . (55) C_S=\mathcal{D}_{\mu_0}\left(K_{0,T},K^{(S)}\right). \tag*{(55)} C S = D μ 0 ( K 0 , T , K ( S ) ) . (55)
Because all the kernels in K ( S ) K^{(S)} K ( S ) are exact, C S C_S C S contains no learning or approximation error. It measures only the error caused by replacing the true conditional dependence from X 0 X_0 X 0 to X T X_T X T with repeated conditioning on the current observed state.
The short-transition kernels always satisfy
μ s K s , t = μ t . \mu_s K_{s,t} = \mu_t. μ s K s , t = μ t .
Consequently, their composition produces the correct unconditional distribution μ t i \mu_{t_i} μ t i at every time in the partition. However, it need not produce the correct distribution conditional on X 0 = x X_0 = x X 0 = x . In other words, all one-time marginals can be correct even when the dependence between the initial and final observations is wrong [7 ] .
The following theorem makes this loss of information precise. As the cascade becomes finer, the composition of the exact observed-state conditional transitions approaches a deterministic evolution driven by the conditional mean observed velocity. Consequently, the memory error C S C_S C S approaches the error made by replacing the true observed trajectory with this deterministic evolution.
The result does not require the quadratic dynamics considered in the previous subsections. It applies to any smooth partially observed deterministic system satisfying the conditions below.
Theorem C.10. Suppose that the full state evolves according to
Z ˙ t = V ( Z t ) \dot{Z}_t = V(Z_t) Z ˙ t = V ( Z t ) on a compact invariant set D ⊂ R d D \subset\mathbb{R}^d D ⊂ R d , and that V V V is twice continuously differentiable on a neighborhood of D D D . As before, let
X t = P Z t . X_t = PZ_t. X t = P Z t . Define the conditional mean of the observed velocity by
v ˉ ( t , x ) = E [ P V ( Z t ) ∣ X t = x ] . \bar{v}(t,x) = \mathbb{E}[PV(Z_t) \mid X_t = x]. v ˉ ( t , x ) = E [ P V ( Z t ) ∣ X t = x ] . Assume that v ˉ \bar{v} v ˉ has a bounded extension to [ 0 , T ] × R m [0,T] \times\mathbb{R}^m [ 0 , T ] × R m that is Lipschitz continuous in x x x with constant L L L and Lipschitz continuous in t t t with constant H H H . Let ψ s , t ( x ) \psi_{s,t}(x) ψ s , t ( x ) denote the solution at time t t t of
d d t ψ s , t ( x ) = v ˉ ( t , ψ s , t ( x ) ) , ψ s , s ( x ) = x . \frac{\mathrm{d}}{\mathrm{d}t}\psi_{s,t}(x) = \bar{v}(t,\psi_{s,t}(x)), \qquad\psi_{s,s}(x) = x. d t d ψ s , t ( x ) = v ˉ ( t , ψ s , t ( x )) , ψ s , s ( x ) = x . Define
M = sup z ∈ D ∣ P V ( z ) ∣ , A 0 = 1 2 sup z ∈ D ∣ P D V ( z ) V ( z ) ∣ , M = \sup_{z \in D} |PV(z)|, \qquad A_0 = \frac{1}{2}\sup_{z \in D}|PDV(z)V(z)|, M = z ∈ D sup ∣ P V ( z ) ∣ , A 0 = 2 1 z ∈ D sup ∣ P D V ( z ) V ( z ) ∣ , and
B 0 = A 0 + 1 2 ( H + L M ) . B_0 = A_0 + \frac{1}{2}(H + LM). B 0 = A 0 + 2 1 ( H + L M ) . For the equally spaced partition with
δ = T S , \delta= \frac{T}{S}, δ = S T , let
r S = e L T ( 2 M T δ + B 0 T δ ) . (56) r_S = e^{LT}\left(2M\sqrt{T\delta} + B_0T\delta\right). \tag*{(56)} r S = e L T ( 2 M T δ + B 0 T δ ) . (56) Then
D μ 0 ( K ( S ) , δ ψ 0 , T ( ⋅ ) ) ≤ r S , (57) \mathcal{D}_{\mu_0}\left(K^{(S)},\delta_{\psi_{0,T}(\cdot)}\right) \le r_S, \tag*{(57)} D μ 0 ( K ( S ) , δ ψ 0 , T ( ⋅ ) ) ≤ r S , (57) where δ ψ 0 , T ( ⋅ ) ( x , ⋅ ) \delta_{\psi_{0,T}(\cdot)}(x,\cdot) δ ψ 0 , T ( ⋅ ) ( x , ⋅ ) is the point mass at ψ 0 , T ( x ) \psi_{0,T}(x) ψ 0 , T ( x ) . Moreover,
∣ C S − Γ T ∣ ≤ r S , Γ T 2 = E [ ∣ X T − ψ 0 , T ( X 0 ) ∣ 2 ] . (58) |C_S-\Gamma_T| \le r_S, \qquad\Gamma_T^2 = \mathbb{E}\left[|X_T-\psi_{0,T}(X_0)|^2\right]. \tag*{(58)} ∣ C S − Γ T ∣ ≤ r S , Γ T 2 = E [ ∣ X T − ψ 0 , T ( X 0 ) ∣ 2 ] . (58) The deterministic flow generated by v ˉ \bar{v} v ˉ has the correct one-time marginals:
( ψ 0 , t ) # μ 0 = μ t for every t ∈ [ 0 , T ] . (\psi_{0,t})_{\#}\mu_0 = \mu_t \qquad\text{for every } t \in[0,T]. ( ψ 0 , t ) # μ 0 = μ t for every t ∈ [ 0 , T ] . Finally, define
v 0 = E [ tr Cov ( P V ( Z 0 ) ∣ X 0 ) ] . v_0 = \mathbb{E}\left[\operatorname{tr}\operatorname{Cov}(PV(Z_0)\mid X_0)\right]. v 0 = E [ tr Cov ( P V ( Z 0 ) ∣ X 0 ) ] . Then
C S ≥ ( T v 0 − A 0 T 2 − r S ) + , (59) C_S \ge\left(T\sqrt{v_0} - A_0T^2 - r_S\right)_+, \tag*{(59)} C S ≥ ( T v 0 − A 0 T 2 − r S ) + , (59) where ( a ) + = max { a , 0 } (a)_+ = \max\{a,0\} ( a ) + = max { a , 0 } .
The main conclusion follows from Eq. (C.40). Since δ = T / S \delta= T/S δ = T / S , we have r S → 0 r_S \to0 r S → 0 as S → ∞ S \to\infty S → ∞ , and therefore
C S ⟶ Γ T . C_S \longrightarrow\Gamma_T. C S ⟶ Γ T .
Thus, as the cascade becomes increasingly fine, the error caused by discarding unresolved information approaches the error of replacing the true observed trajectory by the deterministic conditional-mean flow
x ⟼ ψ 0 , T ( x ) . x \longmapsto\psi_{0,T}(x). x ⟼ ψ 0 , T ( x ) .
Equation (C.39) explains why. The composition of increasingly short observed-state conditional transitions converges to the point mass
δ ψ 0 , T ( x ) . \delta_{\psi_{0,T}(x)}. δ ψ 0 , T ( x ) .
Conditional on the initial observation X 0 = x X_0 = x X 0 = x , the infinitely fine exact-kernel cascade therefore becomes deterministic. It follows the average observed velocity v ˉ ( t , x ) \bar{v}(t,x) v ˉ ( t , x ) and loses the variability caused by the unresolved state.
The unresolved variables persist along the true physical trajectory. Their effects over different physical intervals are therefore correlated. By contrast, after every stage of the cascade, the exact observed-state kernel retains only the new observed state. It forgets which unresolved state produced that transition and averages again over all unresolved states compatible with the current observation.
For a smooth deterministic system, the unresolved contribution to one short increment has size of order δ \delta δ . After S S S such independently reconditioned transitions, the accumulated variance is of order
S δ 2 = T δ , S\delta^2 = T\delta, S δ 2 = T δ ,
which converges to zero as δ → 0 \delta\to0 δ → 0 . This is why the random effect of the unresolved variables disappears and only the conditional mean velocity remains.
Importantly, the limiting deterministic flow still has the correct one-time distributions:
( ψ 0 , t ) # μ 0 = μ t . (\psi_{0,t})_{\#}\mu_0 = \mu_t. ( ψ 0 , t ) # μ 0 = μ t .
It can therefore reproduce the distribution of X t X_t X t at every individual time while failing to preserve the correct dependence between X 0 X_0 X 0 and X T X_T X T . The error Γ T \Gamma_T Γ T measures this lost dependence.
The quantity v 0 v_0 v 0 measures how much the observed velocity can vary among full states with the same initial observation. If v 0 > 0 v_0 > 0 v 0 > 0 , the unresolved state affects the initial observed velocity. The lower bound in Eq. (C.41) then shows that, for sufficiently small fixed T > 0 T > 0 T > 0 and sufficiently fine cascades, the memory error remains strictly positive. Thus, arbitrarily short transitions do not in general recover the correct conditional endpoint distribution.
Proof of Theorem C.10. The extension of v ˉ \bar{v} v ˉ can be assumed to have norm at most M M M : project any given extension onto the closed Euclidean ball of radius M M M . This changes no conditional velocity on its support and does not increase its spatial or temporal Lipschitz constants.
Along a physical trajectory, ∣ X t + δ − X t ∣ ≤ M δ |X_{t+\delta} - X_t| \le M\delta ∣ X t + δ − X t ∣ ≤ M δ . Taylor’s formula followed by conditioning gives, for μ t i \mu_{t_i} μ t i -almost every x x x ,
∫ ( x ′ − x ) K i ( x , d x ′ ) = δ v ˉ ( t i , x ) + r i ( x ) , ∣ r i ( x ) ∣ ≤ A 0 δ 2 , ∫ ∣ x ′ − x ∣ 2 K i ( x , d x ′ ) ≤ M 2 δ 2 . (60) \begin{aligned}
\int(x' - x)K_i(x,dx') &= \delta\bar{v}(t_i,x) + r_i(x), \qquad|r_i(x)| \le A_0\delta^2,\\
\int|x' - x|^2K_i(x,dx') &\le M^2\delta^2. \tag*{(60)}
\end{aligned} ∫ ( x ′ − x ) K i ( x , d x ′ ) ∫ ∣ x ′ − x ∣ 2 K i ( x , d x ′ ) = δ v ˉ ( t i , x ) + r i ( x ) , ∣ r i ( x ) ∣ ≤ A 0 δ 2 , ≤ M 2 δ 2 . (60)
Construct a Markov chain X ~ i \widetilde{X}_i X i with these kernels, starting from μ 0 \mu_0 μ 0 . Its marginal is μ t i \mu_{t_i} μ t i at every step, so all almost-everywhere bounds hold at the states it visits. It has the recursion
X ~ i + 1 = X ~ i + δ v ˉ ( t i , X ~ i ) + r i ( X ~ i ) + ξ i + 1 , \widetilde{X}_{i+1} = \widetilde{X}_i + \delta\bar{v}(t_i,\widetilde{X}_i) + r_i(\widetilde{X}_i) + \xi_{i+1}, X i + 1 = X i + δ v ˉ ( t i , X i ) + r i ( X i ) + ξ i + 1 ,
where ξ i + 1 \xi_{i+1} ξ i + 1 is a martingale difference with conditional second moment at most M 2 δ 2 M^2\delta^2 M 2 δ 2 .
For x i = ψ 0 , t i ( X ~ 0 ) x_i = \psi_{0,t_i}(\widetilde{X}_0) x i = ψ 0 , t i ( X 0 ) , the deterministic flow has the one-step remainder bound
∣ x i + 1 − x i − δ v ˉ ( t i , x i ) ∣ ≤ 1 2 ( H + L M ) δ 2 . |x_{i+1} - x_i - \delta\bar{v}(t_i,x_i)| \le\frac{1}{2}(H + LM)\delta^2. ∣ x i + 1 − x i − δ v ˉ ( t i , x i ) ∣ ≤ 2 1 ( H + L M ) δ 2 .
Sum the two recursions and apply discrete Gronwall to the running maximum. The result is
max i ≤ S ∣ X ~ i − x i ∣ ≤ e L T [ max i ≤ S ∣ ∑ j = 1 i ξ j ∣ + B 0 T δ ] . \max_{i\le S} \lvert\widetilde{X}_i-x_i\rvert\le e^{LT}\left[\max_{i\le S}\left\lvert\sum_{j=1}^{i}\xi_j\right\rvert+B_0T\delta\right]. i ≤ S max ∣ X i − x i ∣ ≤ e L T [ i ≤ S max j = 1 ∑ i ξ j + B 0 T δ ] .
Doob’s L 2 L^2 L 2 inequality and orthogonality of the increments give
∥ max i ≤ S ∣ ∑ j = 1 i ξ j ∣ ∥ L 2 ≤ 2 M S δ = 2 M T δ . \left\|\max_{i\le S}\left\lvert\sum_{j=1}^{i}\xi_j\right\rvert\right\|_{L^2}\le2M\sqrt{S}\delta=2M\sqrt{T\delta}. i ≤ S max j = 1 ∑ i ξ j L 2 ≤ 2 M S δ = 2 M T δ .
Distance to a point mass is the second moment of displacement from that point. The coupling uses the same initial state for the chain and the deterministic flow, so it proves (C.39). The chain endpoint has law μ T \mu_T μ T for every S S S ; passing to the limit proves ( ψ 0 , T ) # μ 0 = μ T (\psi_{0,T})_{\#}\mu_0=\mu_T ( ψ 0 , T ) # μ 0 = μ T . Apply the same argument on [ 0 , t ] [0,t] [ 0 , t ] for each earlier time.
The reverse triangle inequality gives (C.40). Also
Γ T 2 = E tr Cov ( X T ∣ X 0 ) + E ∣ m T ( X 0 ) − ψ 0 , T ( X 0 ) ∣ 2 . (61) \Gamma_T^2=\mathbb{E}\operatorname{tr}\operatorname{Cov}(X_T\mid X_0)+\mathbb{E}\lvert m_T(X_0)-\psi_{0,T}(X_0)\rvert^2. \tag*{(61)} Γ T 2 = E tr Cov ( X T ∣ X 0 ) + E ∣ m T ( X 0 ) − ψ 0 , T ( X 0 ) ∣ 2 . (61)
Finally X T = X 0 + T P V ( Z 0 ) + ε T X_T=X_0+TPV(Z_0)+\varepsilon_T X T = X 0 + T P V ( Z 0 ) + ε T , with ∥ ε T ∥ L 2 ≤ A 0 T 2 \|\varepsilon_T\|_{L^2}\le A_0T^2 ∥ ε T ∥ L 2 ≤ A 0 T 2 . Conditional centering is an orthogonal projection in L 2 L^2 L 2 , so
E tr Cov ( X T ∣ X 0 ) ≥ ( T v 0 − A 0 T 2 ) + . \sqrt{\mathbb{E}\operatorname{tr}\operatorname{Cov}(X_T\mid X_0)}\ge\left(T\sqrt{v_0}-A_0T^2\right)_+. E tr Cov ( X T ∣ X 0 ) ≥ ( T v 0 − A 0 T 2 ) + .
Combine with (C.43) and (C.40) to obtain (C.41).
Retaining the initial observation.
The theorem also identifies a way to prevent this particular loss of information. Instead of conditioning each short transition only on the current observed state, one can retain the initial observation throughout the cascade.
For a fixed initial observation a a a , define
K i a ( x , ⋅ ) = Law ( X t i + 1 ∣ X t i = x , X 0 = a ) K_i^a(x,\cdot)=\operatorname{Law}(X_{t_{i+1}}\mid X_{t_i}=x,X_0=a) K i a ( x , ⋅ ) = Law ( X t i + 1 ∣ X t i = x , X 0 = a )
and
μ i a = Law ( X t i ∣ X 0 = a ) . \mu_i^a=\operatorname{Law}(X_{t_i}\mid X_0=a). μ i a = Law ( X t i ∣ X 0 = a ) .
The kernel K i a K_i^a K i a therefore receives both the current observed state x x x and the initial observation a a a .
Proposition C.11. For any Euclidean-valued process and any finite partition, the exact kernels defined above satisfy
δ a K 0 a K 1 a ⋯ K S − 1 a = Law ( X T ∣ X 0 = a ) (62) \delta_aK_0^aK_1^a\cdots K_{S-1}^a=\operatorname{Law}(X_T\mid X_0=a) \tag*{(62)} δ a K 0 a K 1 a ⋯ K S − 1 a = Law ( X T ∣ X 0 = a ) (62) for μ 0 \mu_0 μ 0 -almost every a a a .
Now let K ^ i a \widehat{K}_i^a K i a be approximations of K i a K_i^a K i a . Suppose that
W 2 ( K ^ i a ( x , ⋅ ) , K ^ i a ( x ′ , ⋅ ) ) ≤ L i ∣ x − x ′ ∣ W_2\left(\widehat{K}_i^a(x,\cdot),\widehat{K}_i^a(x',\cdot)\right)\le L_i\lvert x-x'\rvert W 2 ( K i a ( x , ⋅ ) , K i a ( x ′ , ⋅ ) ) ≤ L i ∣ x − x ′ ∣
for every a a a , and define
ε i 2 = E [ W 2 2 ( K i X 0 ( X t i , ⋅ ) , K ^ i X 0 ( X t i , ⋅ ) ) ] . \varepsilon_i^2=\mathbb{E}\left[W_2^2\left(K_i^{X_0}(X_{t_i},\cdot),\widehat{K}_i^{X_0}(X_{t_i},\cdot)\right)\right]. ε i 2 = E [ W 2 2 ( K i X 0 ( X t i , ⋅ ) , K i X 0 ( X t i , ⋅ ) ) ] .
If the required second moments and the quantity on the right-hand side below are finite, then
[ E a ∼ μ 0 W 2 2 ( Law ( X T ∣ X 0 = a ) , δ a K ^ 0 a K ^ 1 a ⋯ K ^ S − 1 a ) ] 1 / 2 ≤ ∑ i = 0 S − 1 ε i ∏ j > i L j . (63) \left[\mathbb{E}_{a\sim\mu_0}W_2^2\left(\operatorname{Law}(X_T\mid X_0=a),\delta_a\widehat{K}_0^a\widehat{K}_1^a\cdots\widehat{K}_{S-1}^a\right)\right]^{1/2}
\le\sum_{i=0}^{S-1}\varepsilon_i\prod_{j>i}L_j. \tag*{(63)} [ E a ∼ μ 0 W 2 2 ( Law ( X T ∣ X 0 = a ) , δ a K 0 a K 1 a ⋯ K S − 1 a ) ] 1/2 ≤ i = 0 ∑ S − 1 ε i j > i ∏ L j . (63)
Proof of Proposition C.11. For μ 0 \mu_0 μ 0 -almost every a a a , the tower property gives
μ i a K i a = μ i + 1 a . \mu_i^aK_i^a=\mu_{i+1}^a. μ i a K i a = μ i + 1 a .
Starting from μ 0 a = δ a \mu_0^a=\delta_a μ 0 a = δ a proves (C.44) by induction.
For the learned bound, compare two compositions at a time, keeping the first i i i kernels exact and all subsequent kernels learned. The input to the replacement at step i i i has law μ i a \mu_i^a μ i a . Its conditional W 2 W_2 W 2 distance changes by at most
e i ( a ) = [ ∫ W 2 2 ( K i a ( x , ⋅ ) , K ^ i a ( x , ⋅ ) ) μ i a ( d x ) ] 1 / 2 . e_i(a)=\left[\int W_2^2\left(K_i^a(x,\cdot),\widehat{K}_i^a(x,\cdot)\right)\mu_i^a(\mathrm{d}x)\right]^{1/2}. e i ( a ) = [ ∫ W 2 2 ( K i a ( x , ⋅ ) , K i a ( x , ⋅ ) ) μ i a ( d x ) ] 1/2 .
Each later learned kernel enlarges this distance by at most its Wasserstein Lipschitz constant. Sum over the replacement steps, then apply Minkowski’s inequality over a a a . Since E a [ e i ( a ) 2 ] = ε i 2 \mathbb{E}_a[e_i(a)^2]=\varepsilon_i^2 E a [ e i ( a ) 2 ] = ε i 2 , this gives (C.45).
The first part of Proposition C.11 shows that retaining X 0 X_0 X 0 removes this endpoint memory error when the short-transition kernels are exact. This does not mean that the resulting process reproduces the full joint law of the physical trajectory. It guarantees only the correct conditional law at the final time.
This anchored construction is a separate variant of the cascade. The Cascaded SI evaluated in the main benchmark conditions each stage on the generated current state, as described in § 4 ; the result above shows what changes theoretically if the initial observation is retained as an additional conditioning variable.
Adding learning error.
We now consider the additional error caused by learning the short transitions. There are three conditional endpoint distributions to distinguish:
K 0 , T , K ( S ) = K 0 K 1 ⋯ K S − 1 , K ^ 0 K ^ 1 ⋯ K ^ S − 1 . K_{0,T},\qquad K^{(S)}=K_0K_1\cdots K_{S-1},\qquad\widehat{K}_0\widehat{K}_1\cdots\widehat{K}_{S-1}. K 0 , T , K ( S ) = K 0 K 1 ⋯ K S − 1 , K 0 K 1 ⋯ K S − 1 .
The first is the true transition from time 0 0 0 to time T T T . The second is the cascade of exact short transitions, each conditioned only on the current observed state. The third is the learned cascade, in which each exact transition K i K_i K i is replaced by an approximation K ^ i \widehat{K}_i K i .
The distance between the first two distributions is the memory error C S C_S C S studied above. We now bound the distance between the second and third distributions.
For each segment, define
ε i 2 = ∫ W 2 2 ( K i ( x , ⋅ ) , K ^ i ( x , ⋅ ) ) μ t i ( d x ) . \varepsilon_i^2=\int W_2^2\left(K_i(x,\cdot),\widehat{K}_i(x,\cdot)\right)\mu_{t_i}(\mathrm{d}x). ε i 2 = ∫ W 2 2 ( K i ( x , ⋅ ) , K i ( x , ⋅ ) ) μ t i ( d x ) .
Thus, ε i \varepsilon_i ε i measures how well the learned transition K ^ i \widehat{K}_i K i approximates the exact transition K i K_i K i , averaged over the states that occur at time t i t_i t i .
To control how this error affects later stages, assume that
W 2 ( K ^ i ( x , ⋅ ) , K ^ i ( x ′ , ⋅ ) ) ≤ L i ∣ x − x ′ ∣ . W_2\left(\widehat{K}_i(x,\cdot),\widehat{K}_i(x',\cdot)\right)\le L_i|x-x'|. W 2 ( K i ( x , ⋅ ) , K i ( x ′ , ⋅ ) ) ≤ L i ∣ x − x ′ ∣.
The constant L i L_i L i measures how much the i i i th learned transition can enlarge a difference between two input states. An error introduced at step i i i passes through all later learned transitions and can therefore be enlarged by the factor
∏ j > i L j . \prod_{j>i}L_j. j > i ∏ L j .
This leads to the accumulated learning-error bound
B S = ∑ i = 0 S − 1 ε i ∏ j > i L j . B_S=\sum_{i=0}^{S-1}\varepsilon_i\prod_{j>i}L_j. B S = i = 0 ∑ S − 1 ε i j > i ∏ L j .
where the product is equal to one when i = S − 1 i=S-1 i = S − 1 . In particular,
D μ 0 ( K ( S ) , K ^ 0 K ^ 1 ⋯ K ^ S − 1 ) ≤ B S . \mathcal{D}_{\mu_0}\left(K^{(S)},\widehat{K}_0\widehat{K}_1\cdots\widehat{K}_{S-1}\right)\le B_S. D μ 0 ( K ( S ) , K 0 K 1 ⋯ K S − 1 ) ≤ B S .
Thus, B S B_S B S bounds how much replacing the exact short transitions with their learned approximations changes the final conditional distribution.
The total endpoint distribution error of the learned cascade is
E S = D μ 0 ( K 0 , T , K ^ 0 K ^ 1 ⋯ K ^ S − 1 ) . E_S = \mathcal{D}_{\mu_0}\left(K_{0,T},\widehat{K}_0\widehat{K}_1\cdots\widehat{K}_{S-1}\right). E S = D μ 0 ( K 0 , T , K 0 K 1 ⋯ K S − 1 ) .
Recall that
C S = D μ 0 ( K 0 , T , K ( S ) ) C_S = \mathcal{D}_{\mu_0}\left(K_{0,T},K^{(S)}\right) C S = D μ 0 ( K 0 , T , K ( S ) )
is the error caused by repeatedly conditioning only on the current observed state, even when all short transitions are exact. The triangle inequality and the reverse triangle inequality give
∣ E S − C S ∣ ≤ B S . \lvert E_S-C_S\rvert\le B_S. ∣ E S − C S ∣ ≤ B S .
Equivalently,
( C S − B S ) + ≤ E S ≤ C S + B S . (64) (C_S-B_S)_+ \le E_S \le C_S+B_S. \tag*{(64)} ( C S − B S ) + ≤ E S ≤ C S + B S . (64)
These inequalities do not say that the total error is the sum of a memory error and a learning error. The learned approximations may move the endpoint distribution either closer to or farther from the true distribution. The result says only that learning can change the exact-kernel cascade error C S C_S C S by at most B S B_S B S .
The bounds also do not by themselves show that Cascaded SI is better than Direct SI. The quantity e 2 ( T ) e_2(T) e 2 ( T ) is a lower bound on the endpoint distribution error of any direct forecast whose conditional mean is quadratic, whereas
E S ≤ C S + B S E_S \le C_S+B_S E S ≤ C S + B S
is an upper bound on the error of a learned cascade. Therefore, the condition
C S + B S < e 2 ( T ) C_S+B_S < e_2(T) C S + B S < e 2 ( T )
would be sufficient to prove that the cascade is better than every direct forecast with a quadratic conditional mean. Whether this condition holds depends on the system and on the cascade depth S S S . In particular, these bounds alone do not show that the endpoint error is minimized at a finite depth S > 1 S>1 S > 1 .
The results above therefore isolate the two mechanisms in the cascading trade-off. Shorter transitions can reduce approximation error, but repeated conditioning on a partial observation can discard information carried by the unresolved state. The bounds keep these effects separate; they do not determine where their balance is optimal. We next turn to the rotation–OU model, where the endpoint error of a specified learner can be computed explicitly and a finite optimal cascade depth can be proved.
Proof of the learned-kernel bound.
A kernel satisfying the W 2 W_2 W 2 Lipschitz bound with constant L i L_i L i sends input laws at distance d d d to output laws at distance at most L i d L_i d L i d : couple the inputs, couple the conditional outputs, and take infima. Its compositions have the product Lipschitz bound.
Compare the exact and learned products through hybrid products with an exact prefix and a learned suffix. At replacement index i i i , the common prefix, averaged over X 0 X_0 X 0 , has law μ t i \mu_{t_i} μ t i . Couple that prefix identically, use an optimal coupling of K i ( x , ⋅ ) K_i(x,\cdot) K i ( x , ⋅ ) and K ^ i ( x , ⋅ ) \widehat{K}_i(x,\cdot) K i ( x , ⋅ ) , and propagate through the learned suffix. The integrated conditional cost of this replacement is at most ϵ i ∏ j > i L j \epsilon_i\prod_{j>i}L_j ϵ i ∏ j > i L j . Triangle inequalities over the hybrids give
D μ 0 ( K ( S ) , K ^ 0 ⋯ K ^ S − 1 ) ≤ B S . \mathcal{D}_{\mu_0}\left(K^{(S)},\widehat{K}_0\cdots\widehat{K}_{S-1}\right)\le B_S. D μ 0 ( K ( S ) , K 0 ⋯ K S − 1 ) ≤ B S .
The triangle and reverse triangle inequalities with K 0 , T K_{0,T} K 0 , T prove (C.46). In particular, when Theorem C.10 applies, ∣ E S − Γ T ∣ ≤ B S + r S \lvert E_S-\Gamma_T\rvert\le B_S+r_S ∣ E S − Γ T ∣ ≤ B S + r S . Neither B S B_S B S nor C S C_S C S is presumed monotone. A mean-only fitting error cannot replace ϵ i \epsilon_i ϵ i , which measures an entire conditional distribution.
Direct and cascaded forecasts in the rotation–OU model: an explicit trade-off
The previous results identify two opposing effects of cascading. Shorter transitions can be easier to approximate, but repeated conditioning on the observed state can discard information carried by unresolved variables. Those general results do not yet show whether these effects can produce an optimal intermediate cascade depth.
We now exhibit this behavior in a model for which the generated endpoint distribution can be calculated explicitly. This is the model introduced in § 3 . We proceed in three steps. First, we construct a family of forecasting problems with the same target endpoint distribution but different amounts of hidden memory. Second, we compute the endpoint distribution generated by Direct SI and Cascaded SI for a specified quadratic learner. Finally, we compare direct forecasting, fine cascades, and an intermediate cascade.
A fixed endpoint task with tunable hidden memory.
Let
c ∼ Unif [ − 1 , 1 ] , Y 0 ∼ N ( 0 , s 0 2 I 2 ) , c \sim\operatorname{Unif}[-1,1], \qquad Y_0 \sim\mathcal{N}(0,s_0^2 I_2), c ∼ Unif [ − 1 , 1 ] , Y 0 ∼ N ( 0 , s 0 2 I 2 ) ,
independently, and let the hidden state H t ∈ R 2 H_t \in\mathbb{R}^2 H t ∈ R 2 and observed state Y t ∈ R 2 Y_t \in\mathbb{R}^2 Y t ∈ R 2 satisfy
d H t = − κ H t d t + 2 κ d B t , H 0 ∼ N ( 0 , I 2 ) , d Y t = ω c J Y t d t + β ρ R ( ω c t ) H t d t + σ ρ d W t , (65) \begin{aligned}
\mathrm{d}H_t &= -\kappa H_t\,\mathrm{d}t + \sqrt{2\kappa}\,\mathrm{d}B_t, \qquad H_0 \sim\mathcal{N}(0,I_2),\\
\mathrm{d}Y_t &= \omega c JY_t\,\mathrm{d}t + \beta_\rho R(\omega ct)H_t\,\mathrm{d}t + \sigma_\rho\,\mathrm{d}W_t, \tag*{(65)}
\end{aligned} d H t d Y t = − κ H t d t + 2 κ d B t , H 0 ∼ N ( 0 , I 2 ) , = ω c J Y t d t + β ρ R ( ω c t ) H t d t + σ ρ d W t , (65)
where
J = ( 0 − 1 1 0 ) , R ( θ ) = e θ J , J=\begin{pmatrix}0 & -1 \\ 1 & 0\end{pmatrix}, \qquad R(\theta)=e^{\theta J}, J = ( 0 1 − 1 0 ) , R ( θ ) = e θ J ,
and all initial variables and Brownian motions are independent. The conditioning variable c c c is retained throughout the forecast. By contrast, H t H_t H t is unresolved: every stage is conditioned on the current value of Y t Y_t Y t and on c c c , but not on H t H_t H t .
For fixed T T T , q q q , κ \kappa κ , s 0 > 0 s_0>0 s 0 > 0 , define
V H ( t ) = 2 [ t κ − 1 − e − κ t κ 2 ] , β ρ 2 = ρ q V H ( T ) , σ ρ 2 = ( 1 − ρ ) q T , (66) V_H(t)=2\left[\frac{t}{\kappa}-\frac{1-e^{-\kappa t}}{\kappa^2}\right], \qquad\beta_\rho^2=\frac{\rho q}{V_H(T)}, \qquad\sigma_\rho^2=\frac{(1-\rho)q}{T}, \tag*{(66)} V H ( t ) = 2 [ κ t − κ 2 1 − e − κ t ] , β ρ 2 = V H ( T ) ρq , σ ρ 2 = T ( 1 − ρ ) q , (66)
where ρ ∈ [ 0 , 1 ] \rho\in[0,1] ρ ∈ [ 0 , 1 ] . Thus, the integrated OU process contributes ρ q I 2 \rho q I_2 ρq I 2 to the terminal covariance, while the white noise contributes ( 1 − ρ ) q I 2 (1-\rho)qI_2 ( 1 − ρ ) q I 2 .
To remove the deterministic rotation, introduce
Z t = R ( − ω c t ) Y t . Z_t=R(-\omega ct)Y_t. Z t = R ( − ω c t ) Y t .
The dynamics of Z t Z_t Z t contain no rotational drift, and
Z T ∣ ( Y 0 = y , c ) ∼ N ( y , q I 2 ) . Z_T \mid(Y_0=y,c) \sim\mathcal{N}(y,qI_2). Z T ∣ ( Y 0 = y , c ) ∼ N ( y , q I 2 ) .
Transforming back to the original coordinates gives
Y T ∣ ( Y 0 = y , c ) ∼ N ( R ( ω c T ) y , q I 2 ) (67) Y_T \mid(Y_0=y,c) \sim\mathcal{N}(R(\omega cT)y,qI_2) \tag*{(67)} Y T ∣ ( Y 0 = y , c ) ∼ N ( R ( ω c T ) y , q I 2 ) (67)
for every ρ ∈ [ 0 , 1 ] \rho\in[0,1] ρ ∈ [ 0 , 1 ] . Consequently, changing ρ \rho ρ changes the amount of information carried through time by the hidden state without changing the target conditional endpoint distribution.
Exact short transitions.
Although the endpoint forecasting task is independent of ρ \rho ρ , the short conditional transitions are not. For 0 ≤ s ≤ t 0 \le s \le t 0 ≤ s ≤ t , set
I κ ( s , t ) = 2 s κ − ( 1 − e − κ s ) ( 1 + e − κ ( t − s ) ) κ 2 , v s = s 0 2 + β ρ 2 V H ( s ) + σ ρ 2 s , b s , t = s 0 2 + β ρ 2 I κ ( s , t ) + σ ρ 2 s , r s , t = b s , t v s . (68) \begin{aligned}
I_\kappa(s,t) &= \frac{2s}{\kappa}-\frac{(1-e^{-\kappa s})(1+e^{-\kappa(t-s)})}{\kappa^2},\\
v_s &= s_0^2+\beta_\rho^2V_H(s)+\sigma_\rho^2s,\\
b_{s,t} &= s_0^2+\beta_\rho^2I_\kappa(s,t)+\sigma_\rho^2s,\qquad r_{s,t}=\frac{b_{s,t}}{v_s}. \tag*{(68)}
\end{aligned} I κ ( s , t ) v s b s , t = κ 2 s − κ 2 ( 1 − e − κ s ) ( 1 + e − κ ( t − s ) ) , = s 0 2 + β ρ 2 V H ( s ) + σ ρ 2 s , = s 0 2 + β ρ 2 I κ ( s , t ) + σ ρ 2 s , r s , t = v s b s , t . (68)
Consider the equal partition t i = i δ t_i=i\delta t i = i δ , where δ = T / S \delta=T/S δ = T / S , and write
r i = r t i , t i + 1 , q i = v t i + 1 − r i 2 v t i . r_i=r_{t_i,t_{i+1}}, \qquad q_i=v_{t_{i+1}}-r_i^2v_{t_i}. r i = r t i , t i + 1 , q i = v t i + 1 − r i 2 v t i .
Gaussian conditioning gives
Y t i + 1 ∣ ( Y t i = y , c ) ∼ N ( r i R ( ω c δ ) y , q i I 2 ) . (69) Y_{t_{i+1}}\mid(Y_{t_i}=y,c)\sim\mathcal{N}(r_iR(\omega c\delta)y,q_iI_2). \tag*{(69)} Y t i + 1 ∣ ( Y t i = y , c ) ∼ N ( r i R ( ω cδ ) y , q i I 2 ) . (69)
When ρ > 0 \rho> 0 ρ > 0 , composing these exact short transitions does not recover Eq. (C.49). At every reconditioning step, the cascade retains Y t i Y_{t_i} Y t i and c c c but forgets the unresolved state H t i H_{t_i} H t i . A cascade can therefore incur endpoint error even before its short transitions are approximated.
A fixed quadratic learner for Direct SI and Cascaded SI.
We next introduce approximation error while keeping the comparison fully specified. We take 0 ≤ ρ < 1 0 \le\rho< 1 0 ≤ ρ < 1 , so that every q i q_i q i is positive, and use the Gaussian stochastic-interpolant family of Proposition C.8. The exact scalar coefficients r i r_i r i and q i q_i q i are supplied to the learner. The only fitted quantity is a shared function p ( c , y ) p(c,y) p ( c , y ) of total degree at most two, used to approximate the rotation R ( ω c δ ) y R(\omega c\delta)y R ( ω cδ ) y in the conditional mean.
Set
ν = ω δ , j 0 ( ν ) = sin ν ν , j 1 ( ν ) = sin ν − ν cos ν ν 2 . \nu= \omega\delta,\qquad j_0(\nu)=\frac{\sin\nu}{\nu},\qquad j_1(\nu)=\frac{\sin\nu-\nu\cos\nu}{\nu^2}. ν = ω δ , j 0 ( ν ) = ν sin ν , j 1 ( ν ) = ν 2 sin ν − ν cos ν .
with j 0 ( 0 ) = 1 j_0(0)=1 j 0 ( 0 ) = 1 and j 1 ( 0 ) = 0 j_1(0)=0 j 1 ( 0 ) = 0 . Minimizing the expected stochastic-interpolant training objective over the quadratic function class gives, at every stage,
p ν ( c , y ) = M ν ( c ) y , M ν ( c ) = j 0 ( ν ) I 2 + 3 j 1 ( ν ) c J . (70) p_\nu(c,y)=M_\nu(c)y,\qquad M_\nu(c)=j_0(\nu)I_2+3j_1(\nu)cJ. \tag*{(70)} p ν ( c , y ) = M ν ( c ) y , M ν ( c ) = j 0 ( ν ) I 2 + 3 j 1 ( ν ) c J . (70)
Centered Gaussian symmetry eliminates terms that are nonlinear in y y y . The fitted map in Eq. (C.52) is therefore the L 2 L^2 L 2 projection of the rotation onto functions that are linear in y y y and affine in c c c .
The role of the cascade is now explicit. Direct SI uses ν = ω T \nu=\omega T ν = ω T and must approximate the full rotation in one transition. Cascaded SI uses ν = ω T / S \nu=\omega T/S ν = ω T / S and composes S S S simpler fitted rotations.
Exact endpoint distribution error.
Define
d ν ( c ) = j 0 ( ν ) 2 + 9 j 1 ( ν ) 2 c 2 , A S = ∏ i = 0 S − 1 r i , d_\nu(c)=j_0(\nu)^2+9j_1(\nu)^2c^2,\qquad A_S=\prod_{i=0}^{S-1}r_i, d ν ( c ) = j 0 ( ν ) 2 + 9 j 1 ( ν ) 2 c 2 , A S = i = 0 ∏ S − 1 r i ,
and let
q ^ 0 ( c ) = 0 , q ^ i + 1 ( c ) = r i 2 d ν ( c ) q ^ i ( c ) + q i . (71) \widehat q_0(c)=0,\qquad\widehat q_{i+1}(c)=r_i^2d_\nu(c)\widehat q_i(c)+q_i. \tag*{(71)} q 0 ( c ) = 0 , q i + 1 ( c ) = r i 2 d ν ( c ) q i ( c ) + q i . (71)
After S S S learned transitions, the generated conditional distribution is Gaussian, with mean A S M ν ( c ) S y A_SM_\nu(c)^Sy A S M ν ( c ) S y and covariance q ^ S ( c ) I 2 \widehat q_S(c)I_2 q S ( c ) I 2 .
We measure its discrepancy from the target through the average squared conditional Wasserstein error
E S ( ρ ) = 1 2 ∫ − 1 1 [ s 0 2 ∥ R ( ω T c ) − A S M ν ( c ) S ∥ F 2 + 2 ( q − q ^ S ( c ) ) 2 ] d c . (72) \mathcal{E}_S(\rho)=\frac{1}{2}\int_{-1}^{1}\left[s_0^2\left\|R(\omega Tc)-A_SM_\nu(c)^S\right\|_F^2+2\left(\sqrt{q}-\sqrt{\widehat q_S(c)}\right)^2\right]\,dc. \tag*{(72)} E S ( ρ ) = 2 1 ∫ − 1 1 [ s 0 2 R ( ω T c ) − A S M ν ( c ) S F 2 + 2 ( q − q S ( c ) ) 2 ] d c . (72)
The first term measures the error in the conditional mean, while the second measures the error in the conditional covariance. Equation (C.54) gives the complete endpoint error of the specified learner. It is not obtained by adding a separately defined approximation error and memory error.
Direct forecasting and the infinitely deep cascade.
Before comparing all values of S S S , it is useful to identify the two extremes. For Direct SI, S = 1 S=1 S = 1 . Writing Ω = ω T \Omega=\omega T Ω = ω T , we obtain
E 1 ( ρ ) = 2 s 0 2 [ 1 − j 0 ( Ω ) 2 − 3 j 1 ( Ω ) 2 ] > 0 (73) \mathcal{E}_1(\rho)=2s_0^2\left[1-j_0(\Omega)^2-3j_1(\Omega)^2\right]>0 \tag*{(73)} E 1 ( ρ ) = 2 s 0 2 [ 1 − j 0 ( Ω ) 2 − 3 j 1 ( Ω ) 2 ] > 0 (73)
whenever ω ≠ 0 \omega\ne0 ω = 0 . The generated covariance is exact, but one quadratic map cannot reproduce the full dependence of R ( ω T c ) y R(\omega Tc)y R ( ω T c ) y on c c c . Since the endpoint task is independent of ρ \rho ρ , this direct approximation error is also independent of ρ \rho ρ .
At the opposite extreme, let S → ∞ S\to\infty S → ∞ and define
a ρ ( t ) = β ρ 2 ( 1 − e − κ t ) κ v t , A ∞ = exp ( ∫ 0 T a ρ ( t ) d t ) , a_\rho(t)=\frac{\beta_\rho^2(1-e^{-\kappa t})}{\kappa v_t},\qquad A_\infty=\exp\left(\int_0^T a_\rho(t)\,dt\right), a ρ ( t ) = κ v t β ρ 2 ( 1 − e − κ t ) , A ∞ = exp ( ∫ 0 T a ρ ( t ) d t ) ,
q ∞ = v T − s 0 2 A ∞ 2 , C ρ , ∞ = 2 s 0 2 ( A ∞ − 1 ) 2 + 2 ( q − q ∞ ) 2 . (74) q_\infty=v_T-s_0^2A_\infty^2,\qquad C_{\rho,\infty}=2s_0^2(A_\infty-1)^2+2\left(\sqrt q-\sqrt{q_\infty}\right)^2. \tag*{(74)} q ∞ = v T − s 0 2 A ∞ 2 , C ρ , ∞ = 2 s 0 2 ( A ∞ − 1 ) 2 + 2 ( q − q ∞ ) 2 . (74)
If ρ = 0 \rho=0 ρ = 0 , there is no persistent hidden input and arbitrarily short transitions recover the endpoint distribution. If 0 < ρ < 1 0<\rho<1 0 < ρ < 1 , however, C ρ , ∞ > 0 C_{\rho,\infty}>0 C ρ , ∞ > 0 : an increasingly deep cascade repeatedly forgets information carried by the hidden input before the OU state.
The comparison therefore contains two competing effects. Direct SI has a fixed approximation error from fitting the full rotation in one transition. Increasing the cascade depth reduces this error by replacing that transition with simpler rotations. But in the presence of persistent hidden memory, a sufficiently deep cascade approaches a nonzero error caused by repeated reconditioning on the observed state. The following theorem shows that, when the hidden-memory contribution is sufficiently weak but nonzero, neither extreme is optimal.
Theorem C.12 (An intermediate cascade can be optimal). Fix T T T , κ \kappa κ , q q q , s 0 > 0 s_0>0 s 0 > 0 and ω ≠ 0 \omega\ne0 ω = 0 . For the centered rotation–OU model and the quadratic stochastic-interpolant learner defined above, Eq. (C.54) has the following properties.
No persistent hidden memory. If ρ = 0 \rho=0 ρ = 0 , then
E S ( 0 ) = O ( S − 2 ) as S → ∞ . \mathcal{E}_S(0)=O(S^{-2}) \qquad\text{as } S\to\infty. E S ( 0 ) = O ( S − 2 ) as S → ∞. Thus, in the absence of the persistent hidden OU contribution, progressively finer cascades recover the target endpoint distribution.
Persistent hidden memory. For every fixed 0 < ρ < 1 0<\rho<1 0 < ρ < 1 , there is a constant Λ ρ > 0 \Lambda_\rho>0 Λ ρ > 0 such that
E S ( ρ ) = C ρ , ∞ − Λ ρ T S + O ( S − 2 ) . (75) \mathcal{E}_S(\rho)=C_{\rho,\infty}-\Lambda_\rho\frac{T}{S}+O(S^{-2}). \tag*{(75)} E S ( ρ ) = C ρ , ∞ − Λ ρ S T + O ( S − 2 ) . (75) Thus, once the cascade becomes sufficiently deep, increasing S S S moves the endpoint error upward toward the nonzero limit C ρ , ∞ C_{\rho,\infty} C ρ , ∞ .
Intermediate optimum. There exists ρ ∗ > 0 \rho_*>0 ρ ∗ > 0 such that, for every 0 < ρ < ρ ∗ 0<\rho<\rho_* 0 < ρ < ρ ∗ , the minimum over positive integers S S S is attained at a finite S ⋆ ≥ 2 S^\star\ge2 S ⋆ ≥ 2 , and
E S ⋆ ( ρ ) < min { E 1 ( ρ ) , C ρ , ∞ } . (76) \mathcal{E}_{S^\star}(\rho)<\min\{E_1(\rho),C_{\rho,\infty}\}. \tag*{(76)} E S ⋆ ( ρ ) < min { E 1 ( ρ ) , C ρ , ∞ } . (76) Hence the best finite cascade is strictly better than both Direct SI and the infinitely deep cascade limit.
The theorem makes the cascading trade-off explicit. Dividing the horizon into shorter transitions eventually reduces the error from approximating the rotation, although not necessarily at every step from S = 1 S=1 S = 1 : at ρ = 0 \rho=0 ρ = 0 , s 0 2 = q = T = 1 s_0^2=q=T=1 s 0 2 = q = T = 1 , and ω = 2 π \omega=2\pi ω = 2 π , for instance, the error rises from 1.85 at S = 1 S=1 S = 1 to 4.27 at S = 4 S=4 S = 4 before decaying. Once the transitions become too short, however, the benefit of fitting simpler rotations is outweighed by the information lost through repeated conditioning without H t H_t H t . The minimum can therefore occur at an intermediate cascade depth.
Without rotation, cascading only loses memory. Theorem C.12 assumes ω ≠ 0 \omega\ne0 ω = 0 . If ω = 0 \omega=0 ω = 0 , then ν = 0 \nu=0 ν = 0 , M 0 ( c ) = I 2 M_0(c)=I_2 M 0 ( c ) = I 2 , and d 0 ( c ) = 1 d_0(c)=1 d 0 ( c ) = 1 , so every transition mean is fitted exactly. For Direct SI, r 0 = 1 r_0=1 r 0 = 1 and q ^ 1 = q 0 = q \widehat{q}_1=q_0=q q 1 = q 0 = q , so E 1 ( ρ ) = 0 E_1(\rho)=0 E 1 ( ρ ) = 0 : the direct forecast is exact. For i ≥ 1 i\ge1 i ≥ 1 , Eq. (C.50) gives
r i − 1 = β ρ 2 κ 2 v t i ( 1 − e − κ t i ) ( 1 − e − κ δ ) , r_i-1=\frac{\beta_\rho^2}{\kappa^2v_{t_i}}\left(1-e^{-\kappa t_i}\right)\left(1-e^{-\kappa\delta}\right), r i − 1 = κ 2 v t i β ρ 2 ( 1 − e − κ t i ) ( 1 − e − κ δ ) ,
which is positive when ρ > 0 \rho>0 ρ > 0 . Given Y t i Y_{t_i} Y t i alone, a transition cannot tell which part of the state was produced by the persistent hidden process and which part is the initial state, and it extrapolates a fraction of the whole state. Hence A S > 1 A_S>1 A S > 1 for every S ≥ 2 S\ge2 S ≥ 2 , and Eq. (C.54) gives
E S ( ρ ) ≥ 2 s 0 2 ( A S − 1 ) 2 > 0 = E 1 ( ρ ) . \mathcal{E}_S(\rho)\ge2s_0^2(A_S-1)^2>0=E_1(\rho). E S ( ρ ) ≥ 2 s 0 2 ( A S − 1 ) 2 > 0 = E 1 ( ρ ) .
When the conditional mean lies in the approximation class, cascading thus has nothing to gain, and every cascade is worse than the direct forecast. In numerical evaluations of Eq. (C.54), E S ( ρ ) \mathcal{E}_S(\rho) E S ( ρ ) moreover increases with S S S , as does the error of the passive-scalar forecasts (Figure 4 ), whose conditional mean is affine in the observation (Proposition C.4 ).
How the optimal cascade depth depends on memory
The previous theorem establishes the existence of an intermediate optimum. We now determine how its location changes as the hidden-memory contribution becomes small. Define
u ( t ) = q ( 1 − e − κ t ) κ V H ( T ) ( s 0 2 + q t / T ) , U = ∫ 0 T u ( t ) d t , D = 1 2 [ u ( T ) + κ U ] , u(t)=\frac{q(1-e^{-\kappa t})}{\kappa V_H(T)(s_0^2+qt/T)},\qquad U=\int_0^T u(t)\,\mathrm{d}t,\qquad D=\frac{1}{2}\left[u(T)+\kappa U\right], u ( t ) = κ V H ( T ) ( s 0 2 + q t / T ) q ( 1 − e − κ t ) , U = ∫ 0 T u ( t ) d t , D = 2 1 [ u ( T ) + κ U ] ,
λ 2 = 4 s 0 2 ( 1 + s 0 2 q ) U D , B f i t = ω 4 T 2 ( 4 s 0 2 + q ) 90 . (77) \lambda_2=4s_0^2\left(1+\frac{s_0^2}{q}\right)UD,\qquad B_{\mathrm{fit}}=\frac{\omega^4T^2(4s_0^2+q)}{90}. \tag*{(77)} λ 2 = 4 s 0 2 ( 1 + q s 0 2 ) U D , B fit = 90 ω 4 T 2 ( 4 s 0 2 + q ) . (77)
Both λ 2 \lambda_2 λ 2 and B f i t B_{\mathrm{fit}} B fit are strictly positive.
The origin of the scaling can be seen by writing δ = T / S \delta=T/S δ = T / S . For small ρ \rho ρ and small δ \delta δ , the endpoint error satisfies
E S ( ρ ) = C ρ , ∞ − λ 2 ρ 2 δ + B f i t δ 2 + O ( ρ 3 δ + ρ δ 2 + δ 3 ) . (78) \mathcal{E}_S(\rho)=\mathcal{C}_{\rho,\infty}-\lambda_2\rho^2\delta+B_{\mathrm{fit}}\delta^2+O(\rho^3\delta+\rho\delta^2+\delta^3). \tag*{(78)} E S ( ρ ) = C ρ , ∞ − λ 2 ρ 2 δ + B fit δ 2 + O ( ρ 3 δ + ρ δ 2 + δ 3 ) . (78)
The positive term B f i t δ 2 B_{\mathrm{fit}}\delta^2 B fit δ 2 is the remaining error from approximating the short rotation. The negative term − λ 2 ρ 2 δ -\lambda_2\rho^2\delta − λ 2 ρ 2 δ describes the improvement obtained by keeping a finite transition length instead of taking the infinitely deep reconditioning limit. Balancing these two leading terms gives δ ⋆ ≍ ρ 2 \delta^\star\asymp\rho^2 δ ⋆ ≍ ρ 2 , or equivalently S ⋆ ≍ ρ − 2 S^\star\asymp\rho^{-2} S ⋆ ≍ ρ − 2 . The following theorem makes this statement precise.
Theorem C.13 (The optimal cascade depth grows as memory weakens). Under the assumptions of Theorem C.12, every global minimizer over positive integers satisfies
ρ 2 S ⋆ ( ρ ) ⟶ 2 B f i t T λ 2 as ρ ↓ 0. (79) \rho^2 S^\star(\rho)\longrightarrow\frac{2B_{\mathrm{fit}}T}{\lambda_2}\qquad\text{as }\rho\downarrow0. \tag*{(79)} ρ 2 S ⋆ ( ρ ) ⟶ λ 2 2 B fit T as ρ ↓ 0. (79) Moreover,
min S ≥ 1 E S ( ρ ) = C ρ , ∞ − λ 2 2 4 B f i t ρ 4 + o ( ρ 4 ) . (80) \min_{S\ge1}\mathcal{E}_S(\rho)=\mathcal{C}_{\rho,\infty}-\frac{\lambda_2^2}{4B_{\mathrm{fit}}}\rho^4+o(\rho^4). \tag*{(80)} S ≥ 1 min E S ( ρ ) = C ρ , ∞ − 4 B fit λ 2 2 ρ 4 + o ( ρ 4 ) . (80) Thus, weaker hidden memory permits a deeper useful cascade:
S ⋆ ( ρ ) ≍ ρ − 2 . S^\star(\rho)\asymp\rho^{-2}. S ⋆ ( ρ ) ≍ ρ − 2 .
The rotation–OU model therefore exhibits the same qualitative trade-off that motivates the empirical depth study in the main paper: the direct transition can have substantial approximation error, a moderate cascade can reduce it, and a sufficiently deep cascade can lose part of that improvement through repeated conditioning on an incomplete state. The theorem is a statement about this specified Gaussian process and quadratic SI learner. It does not claim that hidden OU-like memory is the mechanism responsible for the depth curves of the PDE benchmarks, nor does it predict the value of S ⋆ S^\star S ⋆ selected in those experiments.
The conditioning calculations, the uniform expansion in Eq. (C.60), and the proofs of Theorems C.12 –C.13 are given in Section C.10.
Learning anchored transitions in the rotation–OU model
Conditioning every transition on Y 0 Y_0 Y 0 removes the memory error when the transition kernels are exact (Proposition C.11 ). However, it also changes the conditional mean that must be learned. To examine this change in the same model, let
m i a n c ( c , y , a ) = E [ Y t i + 1 ∣ Y t i = y , Y 0 = a , c ] . m_i^{\mathrm{anc}}(c,y,a)=\mathbb{E}[Y_{t_{i+1}}\mid Y_{t_i}=y,Y_0=a,c]. m i anc ( c , y , a ) = E [ Y t i + 1 ∣ Y t i = y , Y 0 = a , c ] .
At each stage, approximate this mean by its population least-squares projection onto the vector-valued polynomials of total degree at most two in ( c , y , a ) (c,y,a) ( c , y , a ) , under the true joint law of ( c , Y t i , Y 0 ) (c,Y_{t_i},Y_0) ( c , Y t i , Y 0 ) . As above, supply the exact conditional covariance and sample the resulting Gaussian kernel exactly. Denote the endpoint law by K ^ S a n c ( a , c ) \widehat K_S^{\mathrm{anc}}(a,c) K S anc ( a , c ) and set
E S a n c ( ρ , ω ) = E Y 0 , c W 2 2 ( K T O U ( Y 0 , c ) , K ^ S a n c ( Y 0 , c ) ) . \mathcal{E}_S^{\mathrm{anc}}(\rho,\omega)=\mathbb{E}_{Y_0,c}W_2^2\left(K_T^{\mathrm{OU}}(Y_0,c),\widehat K_S^{\mathrm{anc}}(Y_0,c)\right). E S anc ( ρ , ω ) = E Y 0 , c W 2 2 ( K T OU ( Y 0 , c ) , K S anc ( Y 0 , c ) ) .
The polynomial degree is unchanged, but the learner now receives the initial observation as an additional input.
Theorem C.14 (Anchoring with quadratic transition means). Fix T , κ , q , s 0 > 0 T,\kappa,q,s_0>0 T , κ , q , s 0 > 0 and 0 < ρ < 1 0<\rho<1 0 < ρ < 1 in (3.2). Define
w t = 2 β ρ 2 [ t κ − 1 − e − κ t κ 2 ] + σ ρ 2 t , w_t=2\beta_\rho^2\left[\frac{t}{\kappa}-\frac{1-e^{-\kappa t}}{\kappa^2}\right]+\sigma_\rho^2t, w t = 2 β ρ 2 [ κ t − κ 2 1 − e − κ t ] + σ ρ 2 t , k t = β ρ 2 ( 1 − e − κ t ) κ , λ t = k t w t ( t > 0 ) , k_t=\frac{\beta_\rho^2(1-e^{-\kappa t})}{\kappa},\qquad\lambda_t=\frac{k_t}{w_t}\quad(t>0), k t = κ β ρ 2 ( 1 − e − κ t ) , λ t = w t k t ( t > 0 ) , with λ 0 = β ρ 2 / σ ρ 2 \lambda_0=\beta_\rho^2/\sigma_\rho^2 λ 0 = β ρ 2 / σ ρ 2 . Thus w T = q w_T=q w T = q . Let z u n z_{\mathrm{un}} z un solve
z ˙ u n ( t ) = 2 k t s 0 2 + w t z u n ( t ) + σ ρ 2 , z u n ( 0 ) = 0 ; \dot z_{\mathrm{un}}(t)=\frac{2k_t}{s_0^2+w_t}z_{\mathrm{un}}(t)+\sigma_\rho^2,\qquad z_{\mathrm{un}}(0)=0; z ˙ un ( t ) = s 0 2 + w t 2 k t z un ( t ) + σ ρ 2 , z un ( 0 ) = 0 ; this is the conditional variance per coordinate of the unanchored infinite-depth forecast.
Composing the exact anchored kernels recovers K O U ( a , c ) K^{\mathrm{OU}}(a,c) K OU ( a , c ) at every depth. If ω = 0 \omega=0 ω = 0 , their means are affine, so the quadratic anchored forecast is also exact at every depth.
For every fixed ω ≠ 0 \omega\ne0 ω = 0 , the quadratic anchored forecasts converge in averaged conditional W 2 W_2 W 2 as S → ∞ S\to\infty S → ∞ to the Gaussian law N ( F T ( c ) a , z a n c ( T ) I 2 ) \mathcal{N}(F_T(c)a,z_{\mathrm{anc}}(T)I_2) N ( F T ( c ) a , z anc ( T ) I 2 ) , whose variance satisfies z u n ( T ) < z a n c ( T ) < q z_{\mathrm{un}}(T)<z_{\mathrm{anc}}(T)<q z un ( T ) < z anc ( T ) < q .
Consequently, the limiting error satisfies
C ρ , ∞ a n c ( ω ) : = lim S → ∞ E S a n c ( ρ , ω ) ≥ 2 ( q − z a n c ( T ) ) 2 > 0. (81) \mathcal{C}^{\mathrm{anc}}_{\rho,\infty}(\omega):=\lim_{S\to\infty}\mathcal{E}^{\mathrm{anc}}_S(\rho,\omega)\ge2\left(\sqrt{q}-\sqrt{z_{\mathrm{anc}}(T)}\right)^2>0. \tag*{(81)} C ρ , ∞ anc ( ω ) := S → ∞ lim E S anc ( ρ , ω ) ≥ 2 ( q − z anc ( T ) ) 2 > 0. (81)
Let D t D_t D t quantify the accumulation and propagation of the leading-order approximation error in the conditional mean, through
D ˙ t = λ t ( D t + t 2 ) , D 0 = 0. \dot{D}_t=\lambda_t(D_t+t^2),\qquad D_0=0. D ˙ t = λ t ( D t + t 2 ) , D 0 = 0.
With the other parameters fixed, as ω → 0 \omega\to0 ω → 0 ,
C ρ , ∞ a n c ( ω ) = 2 s 0 2 D T 2 45 ω 4 + O ( ω 6 ) . (82) \mathcal{C}^{\mathrm{anc}}_{\rho,\infty}(\omega)=\frac{2s_0^2D_T^2}{45}\omega^4+O(\omega^6). \tag*{(82)} C ρ , ∞ anc ( ω ) = 45 2 s 0 2 D T 2 ω 4 + O ( ω 6 ) . (82)
In particular, for all sufficiently small ∣ ω ∣ |\omega| ∣ ω ∣ , this error is strictly smaller than the unanchored limiting error C ρ , ∞ \mathcal{C}_{\rho,\infty} C ρ , ∞ .
The proof is given in Section C.10.
Thus retaining the initial observation removes the error of exact reconditioning, but it introduces a dependence on R ( ω c t ) Y 0 R(\omega c t)Y_0 R ( ω c t ) Y 0 into the mean of each short transition. For nonzero rotation, the quadratic class cannot represent this dependence even in the infinitesimal conditional mean. The remaining infinite-depth error is therefore an approximation error. Anchoring reduces the limiting variance error for every nonzero rotation and the total limiting error for sufficiently small rotation.
Learning the anchor dependence in the rotation–OU model
The previous analysis shows that retaining the initial observation removes memory error for exact transitions, but can increase the approximation difficulty for learned transitions. We test this representation effect by training three Wiener stochastic interpolants on the rotation–OU model: (A) unanchored SI, conditioned on the current state Y t i Y_{t_i} Y t i ; (B) raw-anchored SI, additionally conditioned on Y 0 Y_0 Y 0 ; and (C) rotated-anchor SI, supplied with R ( ω c t i + 1 ) Y 0 R(\omega c t_{i+1})Y_0 R ( ω c t i + 1 ) Y 0 instead of Y 0 Y_0 Y 0 . All three receive the observed parameter c c c and relevant times. The anchor in (C) is an invertible, known transformation of the anchor in (B), so the two contain identical information; the network in (C) need not learn this rotation. To match input dimensions, model (A) receives a zero-filled anchor channel.
We use T = 1 T=1 T = 1 , S = 8 S=8 S = 8 , κ = 1 \kappa=1 κ = 1 , β = 0.5 \beta=0.5 β = 0.5 , σ = 0.1 \sigma=0.1 σ = 0.1 , Y 0 ∼ N ( 0 , 0.25 2 I 2 ) Y_0\sim\mathcal{N}(0,0.25^2I_2) Y 0 ∼ N ( 0 , 0.2 5 2 I 2 ) , c ∼ Unif [ − 1 , 1 ] c\sim\operatorname{Unif}[-1,1] c ∼ Unif [ − 1 , 1 ] , and ω ∈ { 0 , 12 } \omega\in\{0,12\} ω ∈ { 0 , 12 } . Each model is a four-hidden-layer MLP of width 128, trained on 16,000 transition pairs for 30,000 updates. For each of three paired training seeds, the SI training-noise scale is selected from { 0.5 , 1 , 2 } \{0.5,1,2\} { 0.5 , 1 , 2 } by validation energy distance; checkpoints are selected separately. The sampling-step doubling check changes mean energy distance by less than 0.001.
For each of 100 held-out pairs ( c ( i ) , y 0 ( i ) ) (c^{(i)},y_0^{(i)}) ( c ( i ) , y 0 ( i ) ) , we generate 4,000 forecast endpoints and estimate their mean m ^ i \hat{m}_i m ^ i and covariance Σ ^ i \hat{\Sigma}_i Σ ^ i . As the exact endpoint law is Gaussian, we report the conditional Gaussian moment-matched distance
1 100 ∑ i = 1 100 W 2 2 ( N ( m ^ i , Σ ^ i ) , N ( R ( ω c ( i ) T ) y 0 ( i ) , σ T 2 I 2 ) ) . (83) \frac{1}{100}\sum_{i=1}^{100}W_2^2\left(\mathcal{N}(\hat{m}_i,\hat{\Sigma}_i),\mathcal{N}\left(R(\omega c^{(i)}T)y_0^{(i)},\sigma_T^2I_2\right)\right). \tag*{(83)} 100 1 i = 1 ∑ 100 W 2 2 ( N ( m ^ i , Σ ^ i ) , N ( R ( ω c ( i ) T ) y 0 ( i ) , σ T 2 I 2 ) ) . (83)
Distances are evaluated per conditioning pair, in original physical coordinates, before averaging. This metric compares fitted Gaussian moments, not necessarily the full generated endpoint laws.
At ω = 0 \omega=0 ω = 0 , raw anchoring strongly improves the forecast (Table 15 ). At ω = 12 \omega=12 ω = 12 , the selected raw-anchored model performs almost identically to the unanchored model, whereas pre-rotating the same information substantially reduces the error.
Table 15. Rotation–OU anchoring ablation. Conditional Gaussian moment-matched W 2 2 W_2^2 W 2 2 averaged across 100 held-out conditioning pairs. Mean ± standard error over three training seeds.
Proofs for the rotation–OU model
Exact OU conditioning.
Conditionally on c c c , rotate the process in (C.47) by Z t = R ( − ω c t ) Y t Z_t=R(-\omega ct)Y_t Z t = R ( − ω c t ) Y t . The visible rotation cancels and
Z t = Y 0 + β ρ ∫ 0 t H u d u + σ ρ W ~ t . (84) Z_t=Y_0+\beta_\rho\int_0^t H_u\,\mathrm{d}u+\sigma_\rho\widetilde{W}_t. \tag*{(84)} Z t = Y 0 + β ρ ∫ 0 t H u d u + σ ρ W t . (84)
The rotated Brownian motion has the standard law and is independent of H H H , Y 0 Y_0 Y 0 conditionally on c c c . This entire conditional law is independent of c c c and ω \omega ω .
Since H H H is stationary with covariance E [ H s H t ⊤ ] = e − κ ∣ t − s ∣ I 2 \mathbb{E}[H_sH_t^\top]=e^{-\kappa|t-s|}I_2 E [ H s H t ⊤ ] = e − κ ∣ t − s ∣ I 2 , for 0 ≤ s ≤ t 0\le s\le t 0 ≤ s ≤ t define
I κ ( s , t ) = ∫ 0 s ∫ 0 t e − κ ∣ r − u ∣ d u d r = 2 s κ − ( 1 − e − κ s ) ( 1 + e − κ ( t − s ) ) κ 2 , I_\kappa(s,t)=\int_0^s\int_0^t e^{-\kappa|r-u|}\,\mathrm{d}u\,\mathrm{d}r=\frac{2s}{\kappa}-\frac{(1-e^{-\kappa s})(1+e^{-\kappa(t-s)})}{\kappa^2}, I κ ( s , t ) = ∫ 0 s ∫ 0 t e − κ ∣ r − u ∣ d u d r = κ 2 s − κ 2 ( 1 − e − κ s ) ( 1 + e − κ ( t − s ) ) ,
v s = s 0 2 + β ρ 2 V H ( s ) + σ ρ 2 s , v_s=s_0^2+\beta_\rho^2V_H(s)+\sigma_\rho^2s, v s = s 0 2 + β ρ 2 V H ( s ) + σ ρ 2 s ,
b s , t = s 0 2 + β ρ 2 I κ ( s , t ) + σ ρ 2 s , r s , t = b s , t / v s . (85) b_{s,t}=s_0^2+\beta_\rho^2I_\kappa(s,t)+\sigma_\rho^2s,\qquad r_{s,t}=b_{s,t}/v_s. \tag*{(85)} b s , t = s 0 2 + β ρ 2 I κ ( s , t ) + σ ρ 2 s , r s , t = b s , t / v s . (85)
Then Cov ( Z s ) = v s I 2 \operatorname{Cov}(Z_s)=v_sI_2 Cov ( Z s ) = v s I 2 and Cov ( Z s , Z t ) = b s , t I 2 \operatorname{Cov}(Z_s,Z_t)=b_{s,t}I_2 Cov ( Z s , Z t ) = b s , t I 2 . Gaussian conditioning gives
Z t ∣ Z s = z ∼ N ( r s , t z , [ v t − r s , t 2 v s ] I 2 ) . (86) Z_t\mid Z_s=z\sim\mathcal{N}\left(r_{s,t}z,[v_t-r_{s,t}^2v_s]I_2\right). \tag*{(86)} Z t ∣ Z s = z ∼ N ( r s , t z , [ v t − r s , t 2 v s ] I 2 ) . (86)
At s = 0 s=0 s = 0 , t = T t=T t = T , r 0 , T = 1 r_{0,T}=1 r 0 , T = 1 and v T − s 0 2 = β ρ 2 V H ( T ) + σ ρ 2 T = q v_T-s_0^2=\beta_\rho^2V_H(T)+\sigma_\rho^2T=q v T − s 0 2 = β ρ 2 V H ( T ) + σ ρ 2 T = q . This proves the endpoint law (C.49).
For a partition write
r i = r t i , t i + 1 , q i = v t i + 1 − r i 2 v t i , A S = ∏ i = 0 S − 1 r i , q ~ S = v T − s 0 2 A S 2 . r_i=r_{t_i,t_{i+1}},\qquad q_i=v_{t_{i+1}}-r_i^2v_{t_i},\qquad A_S=\prod_{i=0}^{S-1}r_i,\qquad\widetilde{q}_S=v_T-s_0^2A_S^2. r i = r t i , t i + 1 , q i = v t i + 1 − r i 2 v t i , A S = i = 0 ∏ S − 1 r i , q S = v T − s 0 2 A S 2 .
Composition of (C.68) gives
Z ~ T ∣ Y 0 = y ∼ N ( A S y , q ~ S I 2 ) . (87) \widetilde{Z}_T\mid Y_0=y\sim\mathcal{N}(A_Sy,\widetilde{q}_SI_2). \tag*{(87)} Z T ∣ Y 0 = y ∼ N ( A S y , q S I 2 ) . (87)
To verify the covariance, start w 0 = 0 w_0=0 w 0 = 0 and use w i + 1 = r i 2 w i + q i w_{i+1}=r_i^2w_i+q_i w i + 1 = r i 2 w i + q i . Induction yields w i = v t i − s 0 2 ∏ j < i r j 2 w_i=v_{t_i}-s_0^2\prod_{j<i}r_j^2 w i = v t i − s 0 2 ∏ j < i r j 2 , so q ~ S ≥ 0 \widetilde{q}_S\ge0 q S ≥ 0 .
The average squared conditional Wasserstein cost is
C ρ , S = 2 s 0 2 ( A S − 1 ) 2 + 2 ( q − q ~ S ) 2 . (88) C_{\rho,S}=2s_0^2(A_S-1)^2+2(\sqrt{q}-\sqrt{\widetilde{q}_S})^2. \tag*{(88)} C ρ , S = 2 s 0 2 ( A S − 1 ) 2 + 2 ( q − q S ) 2 . (88)
This is a squared distance, unlike the root distance C S C_S C S in (C.37). To derive it, the conditional mean discrepancy in the co-rotating frame is ( A S − 1 ) y (A_S-1)y ( A S − 1 ) y , whose squared norm has expectation 2 s 0 2 ( A S − 1 ) 2 2s_0^2(A_S-1)^2 2 s 0 2 ( A S − 1 ) 2 . Isotropic Gaussian covariances contribute 2 ( q − q ~ S ) 2 2(\sqrt{q}-\sqrt{\widetilde{q}_S})^2 2 ( q − q S ) 2 . Terminal rotation is an isometry, so the result is independent of ω \omega ω .
For strict monotonicity in ρ \rho ρ , compute
D s , t : = I κ ( s , t ) − V H ( s ) = ( 1 − e − κ s ) ( 1 − e − κ ( t − s ) ) κ 2 . D_{s,t}:=I_\kappa(s,t)-V_H(s)=\frac{(1-e^{-\kappa s})(1-e^{-\kappa(t-s)})}{\kappa^2}. D s , t := I κ ( s , t ) − V H ( s ) = κ 2 ( 1 − e − κ s ) ( 1 − e − κ ( t − s ) ) .
Then
r s , t = 1 + ρ q D s , t / V H ( T ) s 0 2 + q s / T + ρ q [ V H ( s ) / V H ( T ) − s / T ] . (89) r_{s,t}=1+\frac{\rho qD_{s,t}/V_H(T)}{s_0^2+qs/T+\rho q[V_H(s)/V_H(T)-s/T]}. \tag*{(89)} r s , t = 1 + s 0 2 + q s / T + ρq [ V H ( s ) / V H ( T ) − s / T ] ρq D s , t / V H ( T ) . (89)
Its derivative in ρ \rho ρ is ( q D s , t / V H ( T ) ) ( s 0 2 + q s / T ) (qD_{s,t}/V_H(T))(s_0^2+qs/T) ( q D s , t / V H ( T )) ( s 0 2 + q s / T ) divided by the squared denominator, strictly positive for 0 < s < t 0<s<t 0 < s < t . The first partition interval has r 0 = 1 r_0=1 r 0 = 1 , and each later one has r i > 1 r_i>1 r i > 1 at ρ > 0 \rho>0 ρ > 0 . Hence A S A_S A S increases strictly from one and q ^ S \widehat{q}_S q S decreases from q q q for S ≥ 2 S\ge2 S ≥ 2 . Both terms of (C.70) increase strictly as a function of ρ \rho ρ . At S = 1 S=1 S = 1 or ρ = 0 \rho=0 ρ = 0 the cost is zero. Continuity covers the endpoint ρ = 1 \rho=1 ρ = 1 . This proves monotonicity in the memory parameter for a fixed partition; it does not assert monotonicity under every change of cascade depth.
Optimal quadratic fit and endpoint law.
Keep the centered initial law, take 0 ≤ ρ < 1 0\le\rho<1 0 ≤ ρ < 1 , and use equal intervals of length δ = T / S \delta=T/S δ = T / S . Set Ω = ω T \Omega=\omega T Ω = ω T and ν = ω δ \nu=\omega\delta ν = ω δ . The marginal Y t i ∼ N ( 0 , v t i I 2 ) Y_{t_i}\sim\mathcal{N}(0,v_{t_i}I_2) Y t i ∼ N ( 0 , v t i I 2 ) is independent of c c c , and
Y t i + 1 ∣ ( Y t i = y , c ) ∼ N ( r i R ( ν c ) y , q i I 2 ) . Y_{t_{i+1}}\mid(Y_{t_i}=y,c)\sim\mathcal{N}(r_iR(\nu c)y,q_iI_2). Y t i + 1 ∣ ( Y t i = y , c ) ∼ N ( r i R ( ν c ) y , q i I 2 ) .
Here q i > 0 q_i>0 q i > 0 because the physical white-noise increment has positive variance for ρ < 1 \rho<1 ρ < 1 . The parameter c c c is retained as a conditioning input; it is not interpolated.
Supply the known r i , q i r_i,q_i r i , q i and fit a shared polynomial p ( c , y ) p(c,y) p ( c , y ) of total degree at most two, with candidate local mean r i p ( c , y ) r_ip(c,y) r i p ( c , y ) . The Gaussian SI reduction multiplies each stage’s mean-squared loss by a positive scalar depending on q i q_i q i . Centered Gaussian symmetry in y y y shows that only terms linear in y y y can correlate with the target. Their coefficients can have degree at most one in c c c . Therefore the unique function that minimizes the expected squared error fitted at every stage is
p ν ( c , y ) = M ν ( c ) y , M ν ( c ) = j 0 ( ν ) I 2 + 3 j 1 ( ν ) c J . (90) p_\nu(c,y)=M_\nu(c)y,\qquad M_\nu(c)=j_0(\nu)I_2+3j_1(\nu)cJ. \tag*{(90)} p ν ( c , y ) = M ν ( c ) y , M ν ( c ) = j 0 ( ν ) I 2 + 3 j 1 ( ν ) c J . (90)
Indeed, E c cos ( ν c ) = j 0 ( ν ) \mathbb{E}_c\cos(\nu c)=j_0(\nu) E c cos ( ν c ) = j 0 ( ν ) , while orthogonal projection of sin ( ν c ) \sin(\nu c) sin ( ν c ) onto the span of c c c has coefficient 3 E c [ c sin ( ν c ) ] = 3 j 1 ( ν ) 3\mathbb{E}_c[c\sin(\nu c)]=3j_1(\nu) 3 E c [ c sin ( ν c )] = 3 j 1 ( ν ) . Every positive weighted average of the stage losses has this same minimizer: after restricting to the linear-in-y y y part, the only additional stage factors are scalar r i 2 v t i r_i^2v_{t_i} r i 2 v t i . Thus one shared fit suffices.
Let d ν ( c ) = j 0 ( ν ) 2 + 9 j 1 ( ν ) 2 c 2 d_\nu(c)=j_0(\nu)^2+9j_1(\nu)^2c^2 d ν ( c ) = j 0 ( ν ) 2 + 9 j 1 ( ν ) 2 c 2 , so M ν M ν T = d ν I 2 M_\nu M_\nu^\mathsf{T}=d_\nu I_2 M ν M ν T = d ν I 2 . The exact learned Gaussian kernels compose to a Gaussian endpoint conditional on ( y , c ) (y,c) ( y , c ) , with mean A S M ν ( c ) S y A_SM_\nu(c)^Sy A S M ν ( c ) S y and scalar covariance q ^ S ( c ) I 2 \widehat{q}_S(c)I_2 q S ( c ) I 2 , where
q ^ 0 ( c ) = 0 , q ^ i + 1 ( c ) = r i 2 d ν ( c ) q ^ i ( c ) + q i . (91) \widehat{q}_0(c)=0,\qquad\widehat{q}_{i+1}(c)=r_i^2d_\nu(c)\widehat{q}_i(c)+q_i. \tag*{(91)} q 0 ( c ) = 0 , q i + 1 ( c ) = r i 2 d ν ( c ) q i ( c ) + q i . (91)
Define
d ρ = 1 2 [ a ρ ( T ) + ∫ 0 T ( κ a ρ ( t ) + a ρ ( t ) 2 ) d t ] , d_\rho=\frac{1}{2}\left[a_\rho(T)+\int_0^T\left(\kappa a_\rho(t)+a_\rho(t)^2\right)\,\mathrm{d}t\right], d ρ = 2 1 [ a ρ ( T ) + ∫ 0 T ( κ a ρ ( t ) + a ρ ( t ) 2 ) d t ] ,
C ( A ) = 2 s 0 2 ( A − 1 ) 2 + 2 ( q − v T − s 0 2 A 2 ) 2 . C(A)=2s_0^2(A-1)^2+2\left(\sqrt{q}-\sqrt{v_T-s_0^2A^2}\right)^2. C ( A ) = 2 s 0 2 ( A − 1 ) 2 + 2 ( q − v T − s 0 2 A 2 ) 2 .
Then C ( A ∞ ) = C ρ , ∞ C(A_\infty)=C_{\rho,\infty} C ( A ∞ ) = C ρ , ∞ .
Proof of Theorem C.12. At ρ = 0 \rho=0 ρ = 0 , r i = 1 r_i=1 r i = 1 and q i = q / S q_i=q/S q i = q / S . Uniformly in c ∈ [ − 1 , 1 ] c\in[-1,1] c ∈ [ − 1 , 1 ] ,
M ω δ ( c ) − R ( ω δ c ) = O ( δ 2 ) , ∥ M ω δ ( c ) ∥ o p ≤ 1 + C δ 2 . M_{\omega\delta}(c)-R(\omega\delta c)=O(\delta^2),\qquad\lVert M_{\omega\delta}(c)\rVert_{\mathrm{op}}\le1+C\delta^2. M ω δ ( c ) − R ( ω δ c ) = O ( δ 2 ) , ∥ M ω δ ( c ) ∥ op ≤ 1 + C δ 2 .
A telescoping product gives M ω δ S − R ( Ω c ) = O ( δ ) M_{\omega\delta}^S-R(\Omega c)=O(\delta) M ω δ S − R ( Ω c ) = O ( δ ) . Also d ω δ = 1 + O ( δ 2 ) d_{\omega\delta}=1+O(\delta^2) d ω δ = 1 + O ( δ 2 ) , so
q ^ S ( c ) = q S ∑ j = 0 S − 1 d ω δ ( c ) j = q + O ( δ ) . \widehat{q}_S(c)=\frac{q}{S}\sum_{j=0}^{S-1}d_{\omega\delta}(c)^j=q+O(\delta). q S ( c ) = S q j = 0 ∑ S − 1 d ω δ ( c ) j = q + O ( δ ) .
Since q > 0 q>0 q > 0 , both squared errors in (C.54) are O ( δ 2 ) O(\delta^2) O ( δ 2 ) . Together with (C.55), this proves part (i).
Next fix 0 < ρ < 1 0<\rho<1 0 < ρ < 1 . Equation (C.71) is equivalently
r i = 1 + a ρ ( t i ) 1 − e − κ δ κ . r_i=1+a_\rho(t_i)\frac{1-e^{-\kappa\delta}}{\kappa}. r i = 1 + a ρ ( t i ) κ 1 − e − κ δ .
All following Taylor estimates are uniform in t i ∈ [ 0 , T ] t_i\in[0,T] t i ∈ [ 0 , T ] and c ∈ [ − 1 , 1 ] c\in[-1,1] c ∈ [ − 1 , 1 ] , with constants for the fixed physical parameters. Expanding the logarithm gives
log r i = a ρ ( t i ) δ − 1 2 [ κ a ρ ( t i ) + a ρ ( t i ) 2 ] δ 2 + O ( δ 3 ) . \log r_i=a_\rho(t_i)\delta-\frac{1}{2}\left[\kappa a_\rho(t_i)+a_\rho(t_i)^2\right]\delta^2+O(\delta^3). log r i = a ρ ( t i ) δ − 2 1 [ κ a ρ ( t i ) + a ρ ( t i ) 2 ] δ 2 + O ( δ 3 ) .
The left Riemann sum for the smooth function a ρ a_\rho a ρ , with a ρ ( 0 ) = 0 a_\rho(0)=0 a ρ ( 0 ) = 0 , yields
log A S = log A ∞ − d ρ δ + O ( δ 2 ) , A S = A ∞ ( 1 − d ρ δ ) + O ( δ 2 ) . (92) \log A_S=\log A_\infty-d_\rho\delta+O(\delta^2),\qquad A_S=A_\infty(1-d_\rho\delta)+O(\delta^2). \tag*{(92)} log A S = log A ∞ − d ρ δ + O ( δ 2 ) , A S = A ∞ ( 1 − d ρ δ ) + O ( δ 2 ) . (92)
Here d ρ > 0 d_\rho>0 d ρ > 0 and A ∞ > 1 A_\infty>1 A ∞ > 1 .
To justify differentiating the covariance cost, note that v t ′ = 2 a ρ ( t ) v t + σ ρ 2 v_t'=2a_\rho(t)v_t+\sigma_\rho^2 v t ′ = 2 a ρ ( t ) v t + σ ρ 2 . An integrating factor gives
q ∞ = σ ρ 2 A ∞ 2 ∫ 0 T exp ( − 2 ∫ 0 t a ρ ( s ) d s ) d t > 0. (93) q_\infty=\sigma_\rho^2 A_\infty^2\int_0^T\exp\left(-2\int_0^t a_\rho(s)\,\mathrm{d}s\right)\mathrm{d}t>0. \tag*{(93)} q ∞ = σ ρ 2 A ∞ 2 ∫ 0 T exp ( − 2 ∫ 0 t a ρ ( s ) d s ) d t > 0. (93)
Therefore C C C is smooth near A ∞ A_\infty A ∞ , and
C ′ ( A ) = 4 s 0 2 ( A − 1 ) + 4 s 0 2 A ( q v T − s 0 2 A 2 − 1 ) > 0 (94) C'(A)=4s_0^2(A-1)+4s_0^2A\left(\frac{\sqrt{q}}{\sqrt{v_T-s_0^2A^2}}-1\right)>0 \tag*{(94)} C ′ ( A ) = 4 s 0 2 ( A − 1 ) + 4 s 0 2 A ( v T − s 0 2 A 2 q − 1 ) > 0 (94)
for 1 < A ≤ A ∞ 1<A\leq A_\infty 1 < A ≤ A ∞ . It follows that the exact-kernel squared cost is
C ρ , S = C ρ , ∞ − Λ ρ δ + O ( δ 2 ) , Λ ρ = A ∞ d ρ C ′ ( A ∞ ) > 0. (95) C_{\rho,S}=C_{\rho,\infty}-\Lambda_\rho\delta+O(\delta^2),\qquad\Lambda_\rho=A_\infty d_\rho C'(A_\infty)>0. \tag*{(95)} C ρ , S = C ρ , ∞ − Λ ρ δ + O ( δ 2 ) , Λ ρ = A ∞ d ρ C ′ ( A ∞ ) > 0. (95)
It remains to show that fitting the polynomial rotation does not change this first-order coefficient. Identifying u I + v J uI+vJ u I + v J with u + i v u+iv u + i v , Taylor expansion near one gives
log ( j 0 ( ω δ ) + 3 i j 1 ( ω δ ) c ) = i ω δ c + 1 2 ω 2 δ 2 ( c 2 − 1 / 3 ) + O ( δ 3 ) . \log\bigl(j_0(\omega\delta)+3ij_1(\omega\delta)c\bigr)=i\omega\delta c+\frac{1}{2}\omega^2\delta^2(c^2-1/3)+O(\delta^3). log ( j 0 ( ω δ ) + 3 i j 1 ( ω δ ) c ) = iω δ c + 2 1 ω 2 δ 2 ( c 2 − 1/3 ) + O ( δ 3 ) .
Multiplying by S = T / δ S=T/\delta S = T / δ and exponentiating gives
M ω δ ( c ) S = R ( Ω c ) [ I + δ α ( c ) I + O ( δ 2 ) ] , α ( c ) = 1 2 ω 2 T ( c 2 − 1 / 3 ) , M_{\omega\delta}(c)^S=R(\Omega c)[I+\delta\alpha(c)I+O(\delta^2)],\qquad\alpha(c)=\frac{1}{2}\omega^2T(c^2-1/3), M ω δ ( c ) S = R ( Ω c ) [ I + δ α ( c ) I + O ( δ 2 )] , α ( c ) = 2 1 ω 2 T ( c 2 − 1/3 ) ,
d ω δ ( c ) = 1 + ω 2 δ 2 ( c 2 − 1 / 3 ) + O ( δ 4 ) , E c α ( c ) = 0. (96) d_{\omega\delta}(c)=1+\omega^2\delta^2(c^2-1/3)+O(\delta^4),\qquad\mathbb{E}_c\alpha(c)=0. \tag*{(96)} d ω δ ( c ) = 1 + ω 2 δ 2 ( c 2 − 1/3 ) + O ( δ 4 ) , E c α ( c ) = 0. (96)
The contribution to the mean risk linear in δ \delta δ is a scalar multiple of E c α ( c ) \mathbb{E}_c\alpha(c) E c α ( c ) , so it vanishes.
For the covariance, define nonnegative weights w i = q i ∏ j = i + 1 S − 1 r j 2 w_i=q_i\prod_{j=i+1}^{S-1}r_j^2 w i = q i ∏ j = i + 1 S − 1 r j 2 . They satisfy ∑ i w i = q ~ S ≤ q \sum_iw_i=\widetilde q_S\leq q ∑ i w i = q S ≤ q , and (C.53) gives
q ~ S ( c ) = ∑ i w i d ω δ ( c ) S − 1 − i = q ~ S + ω 2 δ 2 ( c 2 − 1 / 3 ) ∑ i ( S − 1 − i ) w i + O ( δ 2 ) . \widetilde q_S(c)=\sum_iw_i d_{\omega\delta}(c)^{S-1-i}=\widetilde q_S+\omega^2\delta^2(c^2-1/3)\sum_i(S-1-i)w_i+O(\delta^2). q S ( c ) = i ∑ w i d ω δ ( c ) S − 1 − i = q S + ω 2 δ 2 ( c 2 − 1/3 ) i ∑ ( S − 1 − i ) w i + O ( δ 2 ) .
The displayed correction is O ( δ ) O(\delta) O ( δ ) and has mean zero in c c c . The remainder is uniform, since S δ = T S\delta=T S δ = T and ∑ i w i ≤ q \sum_iw_i\leq q ∑ i w i ≤ q . As q ~ S → q ∞ > 0 \widetilde q_S\to q_\infty>0 q S → q ∞ > 0 , the function w ↦ 2 ( q − w ) 2 w\mapsto2(\sqrt q-\sqrt w)^2 w ↦ 2 ( q − w ) 2 has bounded second derivative in a neighborhood of all these covariances for large S S S . Taylor expansion and the zero-mean correction therefore give an expected change O ( δ 2 ) O(\delta^2) O ( δ 2 ) . The same conclusion for the mean term was just proved. Consequently
E S ( ρ ) = C ρ , S + O ( δ 2 ) . (97) \mathcal{E}_S(\rho)=C_{\rho,S}+O(\delta^2). \tag*{(97)} E S ( ρ ) = C ρ , S + O ( δ 2 ) . (97)
Combining (C.77) and (C.79) proves the second claim. This step is a calculation from the fitted endpoint law, not an assumption that learning and memory errors add.
Finally a ρ = O ( ρ ) a_\rho=O(\rho) a ρ = O ( ρ ) uniformly on [ 0 , T ] [0,T] [ 0 , T ] as ρ ↓ 0 \rho\downarrow0 ρ ↓ 0 , since v t ≥ s 0 2 v_t\geq s_0^2 v t ≥ s 0 2 . Thus A ∞ = 1 + O ( ρ ) A_\infty=1+O(\rho) A ∞ = 1 + O ( ρ ) and C ρ , ∞ = O ( ρ 2 ) → 0 C_{\rho,\infty}=O(\rho^2)\to0 C ρ , ∞ = O ( ρ 2 ) → 0 . The strictly positive direct risk (C.55) is independent of ρ \rho ρ . Choose ρ ∗ > 0 \rho_*>0 ρ ∗ > 0 so that C ρ , ∞ < ϵ 1 C_{\rho,\infty}<\epsilon_1 C ρ , ∞ < ϵ 1 for 0 < ρ < ρ ∗ 0<\rho<\rho_* 0 < ρ < ρ ∗ . Part (ii) supplies a sufficiently large finite S 0 S_0 S 0 with
E S 0 ( ρ ) < C ρ , ∞ < E 1 ( ρ ) . \mathcal{E}_{S_0}(\rho)<C_{\rho,\infty}<\mathcal{E}_1(\rho). E S 0 ( ρ ) < C ρ , ∞ < E 1 ( ρ ) .
Because E S ( ρ ) → C ρ , ∞ \mathcal{E}_S(\rho)\to C_{\rho,\infty} E S ( ρ ) → C ρ , ∞ , every sufficiently large index has risk strictly larger than E S 0 ( ρ ) \mathcal{E}_{S_0}(\rho) E S 0 ( ρ ) . The minimum is therefore attained in a finite set of depths and cannot occur at one. This proves the final claim. □
Joint expansion and optimal cascade depth.
The fixed-ρ \rho ρ remainder above is insufficient to optimize jointly over S S S and ρ \rho ρ . We first justify a uniform second-order expansion directly from the finite recurrences that define the learned endpoint law.
Lemma C.15 (Expansion of a smooth affine recurrence). Let θ \theta θ range over a compact parameter set. Suppose the scalar real or complex functions F ( t , h , θ ) F(t,h,\theta) F ( t , h , θ ) and G ( t , h , θ ) G(t,h,\theta) G ( t , h , θ ) are smooth near [ 0 , T ] × { 0 } [0,T]\times\{0\} [ 0 , T ] × { 0 } , uniformly in θ \theta θ , with F ( t , 0 , θ ) = 1 F(t,0,\theta)=1 F ( t , 0 , θ ) = 1 and G ( t , 0 , θ ) = 0 G(t,0,\theta)=0 G ( t , 0 , θ ) = 0 . For h = T / S h=T/S h = T / S , let
y i + 1 = F ( i h , h , θ ) y i + G ( i h , h , θ ) , y 0 = y i n i t ( θ ) , y_{i+1}=F(ih,h,\theta)y_i+G(ih,h,\theta),\qquad y_0=y_{\mathrm{init}}(\theta), y i + 1 = F ( ih , h , θ ) y i + G ( ih , h , θ ) , y 0 = y init ( θ ) , where the initial value is smooth. Then smooth functions y [ j ] ( t , θ ) y^{[j]}(t,\theta) y [ j ] ( t , θ ) , j = 0 , 1 , 2 j=0,1,2 j = 0 , 1 , 2 , exist such that
y S = y [ 0 ] ( T , θ ) + h y [ 1 ] ( T , θ ) + h 2 y [ 2 ] ( T , θ ) + O ( h 3 ) , y_S=y^{[0]}(T,\theta)+hy^{[1]}(T,\theta)+h^2y^{[2]}(T,\theta)+O(h^3), y S = y [ 0 ] ( T , θ ) + h y [ 1 ] ( T , θ ) + h 2 y [ 2 ] ( T , θ ) + O ( h 3 ) , with a remainder uniform in θ \theta θ .
Proof. Expand F = 1 + h f 1 + h 2 f 2 + h 3 f 3 + O ( h 4 ) F=1+hf_1+h^2f_2+h^3f_3+O(h^4) F = 1 + h f 1 + h 2 f 2 + h 3 f 3 + O ( h 4 ) and G = h g 1 + h 2 g 2 + h 3 g 3 + O ( h 4 ) G=hg_1+h^2g_2+h^3g_3+O(h^4) G = h g 1 + h 2 g 2 + h 3 g 3 + O ( h 4 ) , uniformly on the compact set. Writing y j = y [ j ] y_j=y^{[j]} y j = y [ j ] , choose the functions by the linear equations
y 0 ′ = f 1 y 0 + g 1 , y 1 ′ = f 1 y 1 + f 2 y 0 + g 2 − 1 2 y 0 ′ ′ , y 2 ′ = f 1 y 2 + f 2 y 1 + f 3 y 0 + g 3 − 1 2 y 1 ′ ′ − 1 6 y 0 ′ ′ ′ , \begin{aligned}
y_0'&=f_1y_0+g_1,\\
y_1'&=f_1y_1+f_2y_0+g_2-\tfrac{1}{2}y_0'',\\
y_2'&=f_1y_2+f_2y_1+f_3y_0+g_3-\tfrac{1}{2}y_1''-\tfrac{1}{6}y_0''',
\end{aligned} y 0 ′ y 1 ′ y 2 ′ = f 1 y 0 + g 1 , = f 1 y 1 + f 2 y 0 + g 2 − 2 1 y 0 ′′ , = f 1 y 2 + f 2 y 1 + f 3 y 0 + g 3 − 2 1 y 1 ′′ − 6 1 y 0 ′′′ , with y 0 ( 0 ) = y i n i t y_0(0)=y_{\mathrm{init}} y 0 ( 0 ) = y init and y 1 ( 0 ) = y 2 ( 0 ) = 0 y_1(0)=y_2(0)=0 y 1 ( 0 ) = y 2 ( 0 ) = 0 . Successive solution gives smooth functions with bounded derivatives on the compact set. Taylor expansion shows that y 0 ( t ) + h y 1 ( t ) + h 2 y 2 ( t ) y_0(t)+hy_1(t)+h^2y_2(t) y 0 ( t ) + h y 1 ( t ) + h 2 y 2 ( t ) satisfies one step of the recurrence with defect O ( h 4 ) O(h^4) O ( h 4 ) . Since ∣ F ∣ ≤ 1 + C h \lvert F\rvert\leq1+Ch ∣ F ∣ ≤ 1 + C h , the terminal defect is bounded by
C h 4 ∑ i = 0 S − 1 ( 1 + C h ) i = O ( h 3 ) , Ch^4\sum_{i=0}^{S-1}(1+Ch)^i=O(h^3), C h 4 i = 0 ∑ S − 1 ( 1 + C h ) i = O ( h 3 ) , uniformly in θ \theta θ .
Lemma C.16 (Joint small-memory and fine-cascade expansion). For some fixed 0 < ρ 0 < 1 0<\rho_0<1 0 < ρ 0 < 1 and all sufficiently large S S S , uniformly for 0 ≤ ρ ≤ ρ 0 0\leq\rho\leq\rho_0 0 ≤ ρ ≤ ρ 0 and h = T / S h=T/S h = T / S ,
E S ( ρ ) = C ρ , ∞ − λ 2 ρ 2 h + B f i t h 2 + O ( ρ 3 h + ρ h 2 + h 3 ) , (98) \mathcal{E}_S(\rho)=C_{\rho,\infty}-\lambda_2\rho^2h+B_{\mathrm{fit}}h^2+O(\rho^3h+\rho h^2+h^3), \tag*{(98)} E S ( ρ ) = C ρ , ∞ − λ 2 ρ 2 h + B fit h 2 + O ( ρ 3 h + ρ h 2 + h 3 ) , (98) with the strictly positive constants in (C.59).
Proof. Write a = a ρ a=a_\rho a = a ρ and
r ( t , h ) = 1 + a ( t ) ( 1 − e − κ h ) / κ . r(t,h)=1+a(t)(1-e^{-\kappa h})/\kappa. r ( t , h ) = 1 + a ( t ) ( 1 − e − κh ) / κ . The complex mean multiplier obeys the recurrence in Lemma C.15 with
F = r ( t , h ) [ j 0 ( ω h ) + 3 i j 1 ( ω h ) c ] , G = 0 , y i n i t = 1. F=r(t,h)[j_0(\omega h)+3ij_1(\omega h)c],\qquad G=0,\qquad y_{\mathrm{init}}=1. F = r ( t , h ) [ j 0 ( ω h ) + 3 i j 1 ( ω h ) c ] , G = 0 , y init = 1. The covariance obeys it with
F = r ( t , h ) 2 d ω h ( c ) , G = v t + h − r ( t , h ) 2 v t , y i n i t = 0. F=r(t,h)^2d_{\omega h}(c),\qquad G=v_{t+h}-r(t,h)^2v_t,\qquad y_{\mathrm{init}}=0. F = r ( t , h ) 2 d ω h ( c ) , G = v t + h − r ( t , h ) 2 v t , y init = 0. These coefficients are smooth jointly in ( t , h , ρ , c ) (t,h,\rho,c) ( t , h , ρ , c ) , with removable singularities of j 0 , j 1 j_0,j_1 j 0 , j 1 filled in at zero. Their leading endpoint values are A ∞ e i Ω c A_\infty e^{i\Omega c} A ∞ e i Ω c and q ∞ q_\infty q ∞ , respectively, because the leading differential equations are y ′ = ( a + i ω c ) y y'=(a+i\omega c)y y ′ = ( a + iω c ) y and w ′ = 2 a w + σ ρ 2 w'=2aw+\sigma_\rho^2 w ′ = 2 a w + σ ρ 2 , with the stated initial values.
On the compact set ρ ∈ [ 0 , ρ 0 ] \rho\in[0,\rho_0] ρ ∈ [ 0 , ρ 0 ] , q ∞ q_\infty q ∞ is bounded below by a positive constant by (C.75). Thus the square-root cost is smooth uniformly near the limiting covariances. Applying the lemma to both recurrences, substituting in (C.54), and integrating over c c c yields
E S ( ρ ) = C ρ , ∞ − Λ ρ h + B ( ρ ) h 2 + O ( h 3 ) , (99) \mathcal{E}_S(\rho)=C_{\rho,\infty}-\Lambda_\rho h+B(\rho)h^2+O(h^3), \tag*{(99)} E S ( ρ ) = C ρ , ∞ − Λ ρ h + B ( ρ ) h 2 + O ( h 3 ) , (99) where B B B is smooth and the remainder is uniform. The already calculated first-order coefficient is − Λ ρ -\Lambda_\rho − Λ ρ for ρ > 0 \rho>0 ρ > 0 , and continuity extends that identity to zero. This deduction uses the endpoint recurrences, not an assumed decomposition of squared errors.
To calculate B ( 0 ) B(0) B ( 0 ) , use r i = 1 r_i=1 r i = 1 , q i = q / S q_i=q/S q i = q / S , and set
α ( c ) = ω 2 T ( c 2 − 1 / 3 ) / 2. \alpha(c)=\omega^2T(c^2-1/3)/2. α ( c ) = ω 2 T ( c 2 − 1/3 ) /2.
Uniformly in c c c ,
M ω h ( c ) S = R ( Ω c ) [ I + h α ( c ) I + O ( h 2 ) ] , M_{\omega h}(c)^S = R(\Omega c)[I+h\alpha(c)I+O(h^2)], M ω h ( c ) S = R ( Ω c ) [ I + h α ( c ) I + O ( h 2 )] ,
q ^ S ( c ) = q S ∑ j = 0 S − 1 d ω h ( c ) j = q [ 1 + h α ( c ) ] + O ( h 2 ) . \widehat q_S(c)=\frac{q}{S}\sum_{j=0}^{S-1}d_{\omega h}(c)^j=q[1+h\alpha(c)]+O(h^2). q S ( c ) = S q j = 0 ∑ S − 1 d ω h ( c ) j = q [ 1 + h α ( c )] + O ( h 2 ) .
The conditional mean contributes 2 s 0 2 h 2 α ( c ) 2 + O ( h 3 ) 2s_0^2h^2\alpha(c)^2+O(h^3) 2 s 0 2 h 2 α ( c ) 2 + O ( h 3 ) to the risk. The covariance contributes ( q / 2 ) h 2 α ( c ) 2 + O ( h 3 ) (q/2)h^2\alpha(c)^2+O(h^3) ( q /2 ) h 2 α ( c ) 2 + O ( h 3 ) . Since E c ( c 2 − 1 / 3 ) 2 = 4 / 45 \mathbb{E}_c(c^2-1/3)^2=4/45 E c ( c 2 − 1/3 ) 2 = 4/45 ,
B ( 0 ) = ( 2 s 0 2 + q / 2 ) E c α ( c ) 2 = ω 4 T 2 ( 4 s 0 2 + q ) 90 = B f i t . B(0)=(2s_0^2+q/2)\mathbb{E}_c\alpha(c)^2=\frac{\omega^4T^2(4s_0^2+q)}{90}=B_{\mathrm{fit}}. B ( 0 ) = ( 2 s 0 2 + q /2 ) E c α ( c ) 2 = 90 ω 4 T 2 ( 4 s 0 2 + q ) = B fit .
Smoothness gives B ( ρ ) = B f i t + O ( ρ ) B(\rho)=B_{\mathrm{fit}}+O(\rho) B ( ρ ) = B fit + O ( ρ ) .
Finally, uniformly for t ∈ [ 0 , T ] t\in[0,T] t ∈ [ 0 , T ] , a ρ ( t ) = ρ u ( t ) + O ( ρ 2 ) a_\rho(t)=\rho u(t)+O(\rho^2) a ρ ( t ) = ρ u ( t ) + O ( ρ 2 ) , so
A ∞ = 1 + ρ U + O ( ρ 2 ) , d ρ = ρ D + O ( ρ 2 ) . A_\infty=1+\rho U+O(\rho^2),\qquad d_\rho=\rho D+O(\rho^2). A ∞ = 1 + ρ U + O ( ρ 2 ) , d ρ = ρ D + O ( ρ 2 ) .
Writing A = 1 + ϵ A=1+\epsilon A = 1 + ϵ in the definition of C ( A ) C(A) C ( A ) above gives
C ( 1 + ϵ ) = 2 s 0 2 ( 1 + s 0 2 / q ) ϵ 2 + O ( ϵ 3 ) . C(1+\epsilon)=2s_0^2(1+s_0^2/q)\epsilon^2+O(\epsilon^3). C ( 1 + ϵ ) = 2 s 0 2 ( 1 + s 0 2 / q ) ϵ 2 + O ( ϵ 3 ) .
Therefore
Λ ρ = A ∞ d ρ C ′ ( A ∞ ) = 4 s 0 2 ( 1 + s 0 2 / q ) U D ρ 2 + O ( ρ 3 ) = λ 2 ρ 2 + O ( ρ 3 ) . \Lambda_\rho=A_\infty d_\rho C'(A_\infty)=4s_0^2(1+s_0^2/q)UD\,\rho^2+O(\rho^3)=\lambda_2\rho^2+O(\rho^3). Λ ρ = A ∞ d ρ C ′ ( A ∞ ) = 4 s 0 2 ( 1 + s 0 2 / q ) U D ρ 2 + O ( ρ 3 ) = λ 2 ρ 2 + O ( ρ 3 ) .
Substitution in (C.81) proves (C.60). Positivity follows from u ( t ) > 0 u(t)>0 u ( t ) > 0 for t > 0 t>0 t > 0 and ω ≠ 0 \omega\ne0 ω = 0 .
Proof of Theorem C.13. Theorem C.12 gives a finite global minimizer for every sufficiently small positive ρ \rho ρ . Let
x ⋆ = λ 2 2 B f i t > 0 x_\star=\frac{\lambda_2}{2B_{\mathrm{fit}}}>0 x ⋆ = 2 B fit λ 2 > 0
and choose an integer S ρ S_\rho S ρ nearest to T / ( x ⋆ ρ 2 ) T/(x_\star\rho^2) T / ( x ⋆ ρ 2 ) . Then T / S ρ = x ⋆ ρ 2 + O ( ρ 4 ) T/S_\rho=x_\star\rho^2+O(\rho^4) T / S ρ = x ⋆ ρ 2 + O ( ρ 4 ) , and (C.60) gives
E S ρ ( ρ ) − C ρ , ∞ = − λ 2 2 4 B f i t ρ 4 + o ( ρ 4 ) . (100) \mathcal{E}_{S_\rho}(\rho)-C_{\rho,\infty}=-\frac{\lambda_2^2}{4B_{\mathrm{fit}}}\rho^4+o(\rho^4). \tag*{(100)} E S ρ ( ρ ) − C ρ , ∞ = − 4 B fit λ 2 2 ρ 4 + o ( ρ 4 ) . (100)
In particular this candidate has strictly smaller risk than the infinitely deep cascade limit.
Every sequence of global minimizers satisfies S ⋆ ( ρ ) → ∞ S^\star(\rho)\to\infty S ⋆ ( ρ ) → ∞ . Otherwise a subsequence has one fixed finite index S S S , while the candidate risk tends to zero and E S ( ρ ) → E S ( 0 ) > 0 \mathcal{E}_S(\rho)\to\mathcal{E}_S(0)>0 E S ( ρ ) → E S ( 0 ) > 0 . The last strict inequality follows already from the mean term:
( j 0 ( ω T / S ) + 3 i j 1 ( ω T / S ) c ) S \left(j_0(\omega T/S)+3ij_1(\omega T/S)c\right)^S ( j 0 ( ω T / S ) + 3 i j 1 ( ω T / S ) c ) S
is a finite polynomial in c c c and cannot equal e i Ω c e^{i\Omega c} e i Ω c on an interval for Ω ≠ 0 \Omega\ne0 Ω = 0 .
Put h ⋆ = T / S ⋆ h^\star=T/S^\star h ⋆ = T / S ⋆ . The uniform expansion and h ⋆ → 0 h^\star\to0 h ⋆ → 0 imply, for sufficiently small ρ \rho ρ ,
E S ⋆ ( ρ ) − C ρ , ∞ ≥ − C ρ 2 h ⋆ + 1 2 B f i t ( h ⋆ ) 2 . \mathcal{E}_{S^\star}(\rho)-C_{\rho,\infty}\ge-C\rho^2h^\star+\frac{1}{2}B_{\mathrm{fit}}(h^\star)^2. E S ⋆ ( ρ ) − C ρ , ∞ ≥ − C ρ 2 h ⋆ + 2 1 B fit ( h ⋆ ) 2 .
The left side is negative by (C.82), so h ⋆ ≤ C ′ ρ 2 h^\star\le C'\rho^2 h ⋆ ≤ C ′ ρ 2 . For bounded x = h / ρ 2 x=h/\rho^2 x = h / ρ 2 , the same expansion is uniformly
E S ( ρ ) − C ρ , ∞ ρ 4 = − λ 2 x + B f i t x 2 + o ( 1 ) . \frac{\mathcal{E}_S(\rho)-C_{\rho,\infty}}{\rho^4}=-\lambda_2x+B_{\mathrm{fit}}x^2+o(1). ρ 4 E S ( ρ ) − C ρ , ∞ = − λ 2 x + B fit x 2 + o ( 1 ) .
The quadratic has the unique minimum at x ⋆ x_\star x ⋆ . Comparison with (C.82) forces every minimizing sequence to satisfy
h ⋆ ρ 2 → x ⋆ , \frac{h^\star}{\rho^2}\to x_\star, ρ 2 h ⋆ → x ⋆ ,
proving the cascade-depth limit (C.61). Evaluation of that minimum gives (C.62).
Proof of the anchoring comparison
Proof of Theorem C.14. Write R t ( c ) = R ( ω c t ) R_t(c)=R(\omega ct) R t ( c ) = R ( ω c t ) and suppress c c c when unambiguous. Conditionally on c c c , the rotating coordinates satisfy
R t − 1 Y t = Y 0 + U t , U t = β ρ ∫ 0 t H r d r + σ ρ W ~ t . R_t^{-1}Y_t=Y_0+U_t,\qquad U_t=\beta_\rho\int_0^t H_r\,\mathrm{d}r+\sigma_\rho\widetilde W_t. R t − 1 Y t = Y 0 + U t , U t = β ρ ∫ 0 t H r d r + σ ρ W t .
Here U U U is centered Gaussian and independent of Y 0 Y_0 Y 0 , with covariance w t I 2 w_tI_2 w t I 2 . For s > 0 s>0 s > 0 and h > 0 h>0 h > 0 , the anchored conditional is Gaussian with mean m s , h a n c m_{s,h}^{\mathrm{anc}} m s , h anc and covariance η s , h I 2 \eta_{s,h}I_2 η s , h I 2 ,
α s , h = 1 + λ s 1 − e − κ h κ , \alpha_{s,h}=1+\lambda_s\frac{1-e^{-\kappa h}}{\kappa}, α s , h = 1 + λ s κ 1 − e − κh ,
η s , h = w s + h − α s , h 2 w s , \eta_{s,h}=w_{s+h}-\alpha_{s,h}^2w_s, η s , h = w s + h − α s , h 2 w s ,
m s , h a n c ( c , y , a ) = α s , h R h y + ( 1 − α s , h ) R s + h a . (101) m_{s,h}^{\mathrm{anc}}(c,y,a)=\alpha_{s,h}R_hy+(1-\alpha_{s,h})R_{s+h}a. \tag*{(101)} m s , h anc ( c , y , a ) = α s , h R h y + ( 1 − α s , h ) R s + h a . (101)
The first kernel is N ( R h a , w h I 2 ) \mathcal{N}(R_ha,w_hI_2) N ( R h a , w h I 2 ) . If an exact anchored cascade has mean R s a R_sa R s a and covariance w s I 2 w_sI_2 w s I 2 , the next kernel has mean α s , h R h R s a + ( 1 − α s , h ) R s + h a = R s + h a \alpha_{s,h}R_hR_sa+(1-\alpha_{s,h})R_{s+h}a=R_{s+h}a α s , h R h R s a + ( 1 − α s , h ) R s + h a = R s + h a and covariance ( α s , h 2 w s + η s , h ) I 2 = w s + h I 2 (\alpha_{s,h}^2w_s+\eta_{s,h})I_2=w_{s+h}I_2 ( α s , h 2 w s + η s , h ) I 2 = w s + h I 2 . Induction proves the first claim. When ω = 0 \omega=0 ω = 0 , all these means are affine in ( y , a ) (y,a) ( y , a ) .
We next identify the limit of the quadratic fits. Let Π t \Pi_t Π t denote least-squares projection under the true law of ( c , Y t , Y 0 ) (c,Y_t,Y_0) ( c , Y t , Y 0 ) . Centering, rotational symmetry, and the symmetry under simultaneous reflection of the vectors and replacement of c c c by − c -c − c give
Π t [ y − R t a ] = P t ( c ) y + Q t ( c ) a , P t ( c ) = u t I 2 + ζ t c J , Q t ( c ) = d t I 2 + e t c J . (102) \Pi_t[y-R_ta]=P_t(c)y+Q_t(c)a,\qquad P_t(c)=u_tI_2+\zeta_tcJ,\qquad Q_t(c)=d_tI_2+e_tcJ. \tag*{(102)} Π t [ y − R t a ] = P t ( c ) y + Q t ( c ) a , P t ( c ) = u t I 2 + ζ t c J , Q t ( c ) = d t I 2 + e t c J . (102)
Set θ = ω t \theta=\omega t θ = ω t and
j 0 = j 0 ( θ ) , j 1 = j 1 ( θ ) , m 2 = E c [ c 2 cos ( θ c ) ] . j_0=j_0(\theta),\qquad j_1=j_1(\theta),\qquad m_2=\mathbb{E}_c[c^2\cos(\theta c)]. j 0 = j 0 ( θ ) , j 1 = j 1 ( θ ) , m 2 = E c [ c 2 cos ( θ c )] .
Eliminating the coefficients multiplying a a a in the normal equations gives
d t = − j 0 u t + j 1 ζ t , e t = − 3 j 1 u t − 3 m 2 ζ t , d_t=-j_0u_t+j_1\zeta_t,\qquad e_t=-3j_1u_t-3m_2\zeta_t, d t = − j 0 u t + j 1 ζ t , e t = − 3 j 1 u t − 3 m 2 ζ t ,
\begin{pmatrix}
w_t+s_0^2h_0 & s_0^2h_1\\
s_0^2h_1 & w_t/3+s_0^2h_2
\end{pmatrix}
\begin{pmatrix}
u_t\\
\zeta_t
\end{pmatrix}
=
\begin{pmatrix}
w_t\\
0
\end{pmatrix}, \tag*{(103)}\end{pmatrix}
h 0 = 1 − j 0 2 − 3 j 1 2 , h 1 = j 1 ( j 0 − 3 m 2 ) , h 2 = 1 3 − j 1 2 − 3 m 2 2 . h_0=1-j_0^2-3j_1^2,\qquad h_1=j_1(j_0-3m_2),\qquad h_2=\frac{1}{3}-j_1^2-3m_2^2. h 0 = 1 − j 0 2 − 3 j 1 2 , h 1 = j 1 ( j 0 − 3 m 2 ) , h 2 = 3 1 − j 1 2 − 3 m 2 2 .
Indeed, in complex notation, let P P P be the L 2 ( [ − 1 , 1 ] , d c / 2 ) L^2([-1,1],\mathrm{d}c/2) L 2 ([ − 1 , 1 ] , d c /2 ) projection onto the real span of 1 1 1 and i c ic i c . Then ( h j k ) j , k = 0 1 (h_{jk})_{j,k=0}^1 ( h j k ) j , k = 0 1 , with entries h 00 = h 0 h_{00}=h_0 h 00 = h 0 , h 01 = h 1 h_{01}=h_1 h 01 = h 1 , and h 11 = h 2 h_{11}=h_2 h 11 = h 2 , is the real Gram matrix of
( I − P ) e i θ c , ( I − P ) ( i c e i θ c ) . (I-P)e^{i\theta c},\qquad(I-P)(ice^{i\theta c}). ( I − P ) e i θ c , ( I − P ) ( i c e i θ c ) .
Thus h 1 2 ≤ h 0 h 2 h_1^2\le h_0h_2 h 1 2 ≤ h 0 h 2 . Also 0 < h 0 < 1 0<h_0<1 0 < h 0 < 1 when θ ≠ 0 \theta\ne0 θ = 0 : e i θ c e^{i\theta c} e i θ c is not affine in c c c , and j 0 ( θ ) j_0(\theta) j 0 ( θ ) and j 1 ( θ ) j_1(\theta) j 1 ( θ ) cannot both vanish. Solving (C.85) yields
u t = w t w t + s 0 2 χ t , χ t = h 0 − s 0 2 h 1 2 w t / 3 + s 0 2 h 2 . u_t=\frac{w_t}{w_t+s_0^2\chi_t},\qquad\chi_t=h_0-\frac{s_0^2h_1^2}{w_t/3+s_0^2h_2}. u t = w t + s 0 2 χ t w t , χ t = h 0 − w t /3 + s 0 2 h 2 s 0 2 h 1 2 .
The Gram inequality gives 0 < h 0 w t / ( w t + 3 s 0 2 h 2 ) ≤ χ t ≤ h 0 < 1 0<h_0w_t/(w_t+3s_0^2h_2)\le\chi_t\le h_0<1 0 < h 0 w t / ( w t + 3 s 0 2 h 2 ) ≤ χ t ≤ h 0 < 1 . Hence
w t s 0 2 + w t < u t < 1 ( t > 0 , ω ≠ 0 ) . (104) \frac{w_t}{s_0^2+w_t}<u_t<1\qquad(t>0,\ \omega\ne0). \tag*{(104)} s 0 2 + w t w t < u t < 1 ( t > 0 , ω = 0 ) . (104)
Expanding (C.83) and projecting its mean gives
Π t m t , h a n c ( c , y , a ) = y + h [ ω c J y + λ t ( P t ( c ) y + Q t ( c ) a ) ] + O ( h 2 ) ( ∣ y ∣ + ∣ a ∣ ) , \Pi_tm_{t,h}^{\mathrm{anc}}(c,y,a)=y+h\left[\omega cJy+\lambda_t\left(P_t(c)y+Q_t(c)a\right)\right]+O(h^2)(|y|+|a|), Π t m t , h anc ( c , y , a ) = y + h [ ω c J y + λ t ( P t ( c ) y + Q t ( c ) a ) ] + O ( h 2 ) ( ∣ y ∣ + ∣ a ∣ ) ,
η t , h = σ ρ 2 h + O ( h 2 ) . \eta_{t,h}=\sigma_\rho^2h+O(h^2). η t , h = σ ρ 2 h + O ( h 2 ) .
The coefficient bounds in these expansions are uniform for h ≤ t ≤ T − h h\le t\le T-h h ≤ t ≤ T − h . To check the endpoint t ↓ 0 t\downarrow0 t ↓ 0 , note that w t ≥ σ ρ 2 t w_t\ge\sigma_\rho^2t w t ≥ σ ρ 2 t , while the two residual functions defining the Gram matrix have norms O ( t 2 ) O(t^2) O ( t 2 ) and O ( t ) O(t) O ( t ) . After elimination of the coefficients multiplying a a a , the normal-equation matrix is bounded below by ( w t / 3 ) I 2 (w_t/3)I_2 ( w t /3 ) I 2 . The remainder in its right-hand side is O ( h 2 ) ( w t + t ) O(h^2)(w_t+t) O ( h 2 ) ( w t + t ) , so solving for the coefficients preserves the uniform O ( h 2 ) O(h^2) O ( h 2 ) remainder. The coefficients extend smoothly to $P_0=I_2, Q 0 = − I 2 Q_0=-I_2 Q 0 = − I 2 . The first fitted kernel separately has mean a + h ω c J a + O ( h 2 ) ∣ a ∣ a+h\omega cJa+O(h^2)|a| a + hω c J a + O ( h 2 ) ∣ a ∣ and covariance ( σ ρ 2 h + O ( h 2 ) ) I 2 (\sigma_\rho^2h+O(h^2))I_2 ( σ ρ 2 h + O ( h 2 )) I 2 .
The Gaussian mean and variance recursions are therefore consistent Euler recursions, with uniformly bounded coefficients, for
F ˙ t ( c ) = [ ω c J + λ t P t ( c ) ] F t ( c ) + λ t Q t ( c ) , z ˙ a n c ( t ) = 2 λ t u t z a n c ( t ) + σ ρ 2 , (105) \dot{F}_t(c)=[\omega cJ+\lambda_tP_t(c)]F_t(c)+\lambda_tQ_t(c),
\qquad
\dot{z}_{\mathrm{anc}}(t)=2\lambda_tu_tz_{\mathrm{anc}}(t)+\sigma_\rho^2, \tag*{(105)} F ˙ t ( c ) = [ ω c J + λ t P t ( c )] F t ( c ) + λ t Q t ( c ) , z ˙ anc ( t ) = 2 λ t u t z anc ( t ) + σ ρ 2 , (105)
with F 0 ( c ) = I 2 F_0(c)=I_2 F 0 ( c ) = I 2 and z a n c ( 0 ) = 0 z_{\mathrm{anc}}(0)=0 z anc ( 0 ) = 0 . Discrete Gronwall gives convergence uniformly in c c c . In the variance recursion, the coefficient of c J cJ c J contributes only at order h 2 h^2 h 2 ; thus the limiting variance is independent of c c c . The limiting risk is
C ρ , ∞ a n c ( ω ) = s 0 2 E c ∥ F T ( c ) − R T ( c ) ∥ F 2 + 2 ( q − z a n c ( T ) ) 2 . (106) \mathcal{C}^{\mathrm{anc}}_{\rho,\infty}(\omega)
=s_0^2\mathbb{E}_{c}\lVert F_T(c)-R_T(c)\rVert_F^2
+2\left(\sqrt{q}-\sqrt{z_{\mathrm{anc}}(T)}\right)^2. \tag*{(106)} C ρ , ∞ anc ( ω ) = s 0 2 E c ∥ F T ( c ) − R T ( c ) ∥ F 2 + 2 ( q − z anc ( T ) ) 2 . (106)
Since w ˙ t = 2 k t + σ ρ 2 = 2 λ t u t w t + σ ρ 2 \dot{w}_t=2k_t+\sigma_\rho^2=2\lambda_tu_tw_t+\sigma_\rho^2 w ˙ t = 2 k t + σ ρ 2 = 2 λ t u t w t + σ ρ 2 , the strict coefficient inequalities
k t s 0 2 + w t < λ t u t < λ t \frac{k_t}{s_0^2+w_t}<\lambda_tu_t<\lambda_t s 0 2 + w t k t < λ t u t < λ t
and the variation-of-constants formula give z u n ( T ) < z a n c ( T ) < w T = q z_{\mathrm{un}}(T)<z_{\mathrm{anc}}(T)<w_T=q z un ( T ) < z anc ( T ) < w T = q . Equation (C.88) then proves (C.63).
Finally, as ω → 0 \omega\to0 ω → 0 , the normal equations give, uniformly for t ∈ [ 0 , T ] t\in[0,T] t ∈ [ 0 , T ] ,
u t = 1 + O ( ω 4 ) , ζ t = O ( ω 3 ) , d t = − 1 + ω 2 t 2 6 + O ( ω 4 ) , e t = − ω t + O ( ω 3 ) . u_t=1+O(\omega^4),\qquad
\zeta_t=O(\omega^3),\qquad
d_t=-1+\frac{\omega^2t^2}{6}+O(\omega^4),\qquad
e_t=-\omega t+O(\omega^3). u t = 1 + O ( ω 4 ) , ζ t = O ( ω 3 ) , d t = − 1 + 6 ω 2 t 2 + O ( ω 4 ) , e t = − ω t + O ( ω 3 ) .
These estimates also hold at t = 0 t=0 t = 0 because w t ≥ σ ρ 2 t w_t\ge\sigma_\rho^2t w t ≥ σ ρ 2 t . Expanding R t R_t R t therefore gives
P t ( c ) R t ( c ) + Q t ( c ) = − ω 2 t 2 2 ( c 2 − 1 3 ) I 2 + O ( ω 3 ) . P_t(c)R_t(c)+Q_t(c)
=-\frac{\omega^2t^2}{2}\left(c^2-\frac{1}{3}\right)I_2+O(\omega^3). P t ( c ) R t ( c ) + Q t ( c ) = − 2 ω 2 t 2 ( c 2 − 3 1 ) I 2 + O ( ω 3 ) .
Subtracting R ˙ t = ω c J R t \dot{R}_t=\omega cJR_t R ˙ t = ω c J R t from the first equation in (C.87) and using variation of constants yields
F t ( c ) − R t ( c ) = − ω 2 D t 2 ( c 2 − 1 3 ) I 2 + O ( ω 3 ) . F_t(c)-R_t(c)
=-\frac{\omega^2D_t}{2}\left(c^2-\frac{1}{3}\right)I_2+O(\omega^3). F t ( c ) − R t ( c ) = − 2 ω 2 D t ( c 2 − 3 1 ) I 2 + O ( ω 3 ) .
The variance equation gives z a n c ( T ) = q + O ( ω 4 ) z_{\mathrm{anc}}(T)=q+O(\omega^4) z anc ( T ) = q + O ( ω 4 ) , so its contribution to (C.88) is O ( ω 8 ) O(\omega^8) O ( ω 8 ) . Using E c ( c 2 − 1 / 3 ) 2 = 4 / 45 \mathbb{E}_c(c^2-1/3)^2=4/45 E c ( c 2 − 1/3 ) 2 = 4/45 and ∥ I 2 ∥ F 2 = 2 \lVert I_2\rVert_F^2=2 ∥ I 2 ∥ F 2 = 2 gives the leading term 2 s 0 2 D T 2 ω 4 / 45 2s_0^2D_T^2\omega^4/45 2 s 0 2 D T 2 ω 4 /45 . The coefficients are smooth in ω \omega ω , and reflection symmetry makes the risk even in ω \omega ω , giving the O ( ω 6 ) O(\omega^6) O ( ω 6 ) remainder. The unanchored limit C ρ , ∞ > 0 \mathcal{C}_{\rho,\infty}>0 C ρ , ∞ > 0 is independent of ω \omega ω , whereas the anchored limit tends to zero. This proves the final comparison.