Introduction
When the governing equations are unknown, incomplete, or too costly to solve, data-driven methods seek predictive and interpretable models directly from time-resolved observations. Working with partial observations or stochastic systems complicates this task as the measured variables need not satisfy a closed deterministic evolution law.
A deterministic and linear description is available at the level of observables. For a time-homogeneous Markov process, the fixed-lag solution operator of the Kolmogorov backward equation is
( T τ f ) ( x ) : = E [ f ( X t + τ ) ∣ X t = x ] . (T_\tau f)(x) := \mathbb{E}[f(X_{t+\tau}) \mid X_t = x]. ( T τ f ) ( x ) := E [ f ( X t + τ ) ∣ X t = x ] . It remains linear and deterministic when the trajectories are random, and for deterministic
Department of Mathematics, Graduate School of Science, Kyoto University, Kyoto 606-8502, Japan (xu.yuanchao.3a@kyoto-u.ac.jp, ishikawa.isao.5s@kyoto-u.ac.jp). † Department of Computing and Mathematical Sciences, California Institute of Technology, Pasadena, CA 91125, USA (clairev@caltech.edu). ‡ Laboratoire Hubert Curien, UMR 5516, CNRS, IOGS, INRIA, Université Jean Monnet, 18 Rue du Professeur Benoît Lauras, 42000 Saint-Étienne, France (ben.gao@univ-st-etienne.fr). Funding: Y.X. and I.I. acknowledge support from JST CREST Grant No. JPMJCR24Q1 and No. JPMJCR24Q6, including the AIP Challenge Program. C.V. acknowledges support from the DoD Vannevar Bush Faculty Fellowship N00014-22-1-2790. B.G. acknowledges the Auvergne Rhône-Alpes region, Grenoble Alpes Métropole and BPIFRance, for project I-Démo Région GreenAI public grant inside plan France 2030.
dynamics it reduces to the Koopman operator [30 , 31 ] . We refer to T τ T_{\tau} T τ τ \tau τ
Most data-driven approximations of this evolution are formed after an observable space has been chosen. Extended dynamic mode decomposition (EDMD) estimates the Galerkin compression of the operator on a fixed dictionary [32 , 67 ] . The dictionary may also be learned [38 , 43 , 51 , 62 ] , for instance by minimizing single-transition spectral residuals [72 ] , or it may be replaced by a reproducing-kernel space [29 , 35 ] , for which sharp spectral error rates are available [33 ] . Variational principles guide the choice of the approximation [44 , 48 , 49 , 69 ] , residual and invariance criteria assess its accuracy [18 , 19 , 25 ] , and the resulting a posteriori error bounds have been minimized to select dictionary and kernel parameters [19 ] . Several of these ideas have been carried over to stochastic dynamics. Residual DMD has been extended to account for the conditional variance of single transitions [17 ] . Invariant representations have been learned with an objective whose maximizer is the leading invariant subspace [34 ] , and the Koopman semigroup has been approximated from data through a stochastic Taylor expansion [71 ] .
In most of these methods, the approximation of the dynamics is tied to a chosen set of coordinates and each candidate observable space requires its own fit of the dynamics. Operator-learning methods instead approximate maps between function spaces without first fixing a reduced representation [10 , 13 , 36 , 37 , 39 , 42 , 59 , 70 ] . For overviews of the operator-theoretic formulation and its relation to spectral decomposition, model reduction, and data-driven learning, see [9 , 46 , 47 ] .
We combine these two viewpoints in a two-stage method, which we call the backward-solution-operator (BSO) method. In the first stage, the solution operator is approximated as a map on observables, before any reduced coordinates are fixed. In the second stage, the estimator is projected onto an observable space that is either chosen in advance or learned by balancing approximate invariance against the representation of the observables of interest. Approximating the operator before fixing the coordinates has three consequences. First, the same estimator can be evaluated on any candidate observable space, so the space can be searched without refitting the dynamics. Second, for stochastic dynamics, an estimator of T τ T_{\tau} T τ T τ f T_{\tau}f T τ f f f f m m m ν o p \nu_{\mathrm{op}} ν op ν o p < m \nu_{\mathrm{op}} < m ν op < m
The main theoretical contribution of this work is an error analysis of the entire two-stage method, from the approximation of T τ T_{\tau} T τ
when measured by the exact objective, up to the learning error, the optimization tolerance, and the sampling factor (Theorem 4.2 ). The computed eigenvalues and spectral subspaces converge whenever the observable spaces approach a finite-dimensional invariant subspace, the learning error on these spaces tends to zero, and the sampling error stays bounded away from one (Theorem 4.3 ). If the corresponding spectral cluster is isolated, the invariance defect of a subspace is locally equivalent to its distance from the invariant subspace (Theorem 4.4 ). The variational objective therefore bounds this distance, provided that the reconstruction observables resolve the subspace and lie sufficiently close to it and that the value of the objective is small relative to its reconstruction weight.
The forecast error is bounded by the one-step error of the space in use and by the part of the observable that this space does not represent (Theorem 4.5); no invariant subspace is required. For an invariant measure and reduced models that preserve constants and are stable on mean-zero observables, a bound that is uniform in the horizon is given in Appendix D.§ 3 .
For deterministic dynamics, a learning rate for Fourier neural operators (FNOs) provides one explicit bound for the first stage (Corollary 4.6 ); it can be inserted into the spectral and forecast estimates above.
The analysis does not depend on how the estimator is constructed, and stochastic dynamics enter only through its learning error. For stochastic dynamics, the estimator has to approximate conditional means, for instance by regression or through a learned transition kernel (Remark 3.1); no explicit learning rate is given for that case.
The rest of the paper is organized as follows. § 2 gives the operator and spectral setting. § 3 presents the method, its computational realization, and its relation to EDMD. § 4 establishes the error analysis, and § 5 reports the numerical experiments. § 6 concludes. Proofs and supplementary material are collected in the appendices.
Preliminaries
Notation. Unless stated otherwise, observables belong to H : = L 2 ( Ω , ρ ; R ) \mathcal{H} := L^2(\Omega,\rho;\mathbb{R}) H := L 2 ( Ω , ρ ; R ) Ω \Omega Ω ρ \rho ρ Ω \Omega Ω ⟨ ⋅ , ⋅ ⟩ \langle\cdot,\cdot\rangle ⟨ ⋅ , ⋅ ⟩ ∥ ⋅ ∥ \|\cdot\| ∥ ⋅ ∥ L 2 ( ρ ) L^2(\rho) L 2 ( ρ ) X \mathcal{X} X Y \mathcal{Y} Y B ( X , Y ) \mathcal{B}(\mathcal{X},\mathcal{Y}) B ( X , Y ) X \mathcal{X} X Y \mathcal{Y} Y ∥ ⋅ ∥ \|\cdot\| ∥ ⋅ ∥ B ( X ) : = B ( X , X ) \mathcal{B}(\mathcal{X}) := \mathcal{B}(\mathcal{X},\mathcal{X}) B ( X ) := B ( X , X ) V V V H \mathcal{H} H P V P_V P V V V V
The complexification of a real linear space W W W W + i W W+iW W + iW H \mathcal{H} H
For vectors and matrices, T \mathsf{T} T ∗ * ∗ † \dagger † ∥ ⋅ ∥ 2 \|\cdot\|_2 ∥ ⋅ ∥ 2 λ max \lambda_{\max} λ m a x λ min \lambda_{\min} λ m i n ∗ ^* ∗ L 2 ( ρ ) L^2(\rho) L 2 ( ρ ) ρ \rho ρ χ \chi χ V V V ⋆ \star ⋆ E ⋆ E_\star E ⋆
Kolmogorov backward equation
Let ( X t ) t ≥ 0 (X_t)_{t\ge0} ( X t ) t ≥ 0 Ω \Omega Ω ( T t ) t ≥ 0 (T_t)_{t\ge0} ( T t ) t ≥ 0
( T t f ) ( x ) : = E [ f ( X t ) ∣ X 0 = x ] . (T_t f)(x) := \mathbb{E}[f(X_t) \mid X_0=x]. ( T t f ) ( x ) := E [ f ( X t ) ∣ X 0 = x ] . Time homogeneity gives ( T τ f ) ( x ) = E [ f ( X s + τ ) ∣ X s = x ] (T_\tau f)(x) = \mathbb{E}[f(X_{s+\tau}) \mid X_s=x] ( T τ f ) ( x ) = E [ f ( X s + τ ) ∣ X s = x ] s ≥ 0 s\ge0 s ≥ 0 T τ T_\tau T τ [53 ] and is also called the stochastic Koopman operator [20 , 50 , 71 ] . If ρ \rho ρ T τ T_\tau T τ L 2 ( ρ ) L^2(\rho) L 2 ( ρ ) T τ 1 = 1 T_\tau1=1 T τ 1 = 1 T τ T_\tau T τ L 2 ( ρ ) L^2(\rho) L 2 ( ρ ) L 2 ( ρ ) L^2(\rho) L 2 ( ρ ) L \mathcal{L} L dom ( L ) \operatorname{dom}(\mathcal{L}) dom ( L ) T t = e t L T_t=e^{t\mathcal{L}} T t = e t L f ∈ dom ( L ) f\in\operatorname{dom}(\mathcal{L}) f ∈ dom ( L ) u ( t , ⋅ ) = T t f u(t,\cdot)=T_t f u ( t , ⋅ ) = T t f [22 , 53 ]
∂ t u = L u , u ( 0 , ⋅ ) = f . \partial_t u = \mathcal{L}u,\qquad u(0,\cdot)=f. ∂ t u = L u , u ( 0 , ⋅ ) = f . For an Itô diffusion d X t = b ( X t ) d t + σ ( X t ) d W t dX_t=b(X_t)\,dt+\sigma(X_t)\,dW_t d X t = b ( X t ) d t + σ ( X t ) d W t b b b σ \sigma σ W t W_t W t [53 ] gives, for smooth f f f
L f = b ⋅ ∇ f + 1 2 tr ( σ σ ⊤ ∇ 2 f ) , \mathcal{L}f=b\cdot\nabla f+\frac{1}{2}\operatorname{tr}(\sigma\sigma^\top\nabla^2f), L f = b ⋅ ∇ f + 2 1 tr ( σ σ ⊤ ∇ 2 f ) , while for x ˙ = F ( x ) \dot{x}=F(x) x ˙ = F ( x ) L f = F ⋅ ∇ f \mathcal{L}f=F\cdot\nabla f L f = F ⋅ ∇ f Φ t \Phi^t Φ t X t = Φ t ( X 0 ) X_t=\Phi^t(X_0) X t = Φ t ( X 0 ) T t f = f ∘ Φ t T_t f=f\circ\Phi^t T t f = f ∘ Φ t τ > 0 \tau>0 τ > 0
T : = T τ , M T : = ∥ T ∥ B ( H ) . T:=T_\tau,\qquad M_T:=\lVert T\rVert_{\mathcal{B}(\mathcal{H})}. T := T τ , M T := ∥ T ∥ B ( H ) . Invariant subspaces and spectral projections
The targets of the spectral computation are finite-dimensional T T T dist ( V , W ) : = ∥ P V − P W ∥ \operatorname{dist}(V,W):=\lVert P_V-P_W\rVert dist ( V , W ) := ∥ P V − P W ∥ V V V W W W H \mathcal{H} H V V V W W W dist ( V , W ) = ∥ ( I − P V ) P W ∥ = ∥ ( I − P W ) P V ∥ \operatorname{dist}(V,W)=\lVert(I-P_V)P_W\rVert=\lVert(I-P_W)P_V\rVert dist ( V , W ) = ∥( I − P V ) P W ∥ = ∥( I − P W ) P V ∥ V V V W W W [6 , 61 ] , [28 Chap. I, §6.8] .
A subspace E ⊂ H E\subset\mathcal{H} E ⊂ H T T T T E ⊂ E TE\subset E T E ⊂ E E ⋆ E_\star E ⋆ m m m T T T P ⋆ : = P E ⋆ P_\star:=P_{E_\star} P ⋆ := P E ⋆ T ∣ E ⋆ T|_{E_\star} T ∣ E ⋆ T T T Λ ⋆ ⊂ spec ( T ) \Lambda_\star\subset\operatorname{spec}(T) Λ ⋆ ⊂ spec ( T ) spec ( T ) ∖ Λ ⋆ \operatorname{spec}(T)\setminus\Lambda_\star spec ( T ) ∖ Λ ⋆ Γ \Gamma Γ
Π : = 1 2 π i ∮ Γ ( z I − T ) − 1 d z \Pi:=\frac{1}{2\pi i}\oint_\Gamma(zI-T)^{-1}\,dz Π := 2 π i 1 ∮ Γ ( z I − T ) − 1 d z has finite rank m m m T T T Λ ⋆ \Lambda_\star Λ ⋆ Ran Π \operatorname{Ran}\Pi Ran Π E ⋆ = { u ∈ Ran Π : u ‾ = u } E_\star=\{u\in\operatorname{Ran}\Pi:\overline{u}=u\} E ⋆ = { u ∈ Ran Π : u = u } m m m T T T Ran Π = E ⋆ + i E ⋆ \operatorname{Ran}\Pi=E_\star+iE_\star Ran Π = E ⋆ + i E ⋆ spec ( T ∣ E ⋆ ) = Λ ⋆ \operatorname{spec}(T|_{E_\star})=\Lambda_\star spec ( T ∣ E ⋆ ) = Λ ⋆ [28 Chap. III, §6.4] . The complexification Ran Π \operatorname{Ran}\Pi Ran Π Λ ⋆ \Lambda_\star Λ ⋆ E ⋆ E_\star E ⋆ Λ ⋆ \Lambda_\star Λ ⋆ L \mathcal{L} L T τ = e τ L T_\tau=e^{\tau\mathcal{L}} T τ = e τ L L \mathcal{L} L [28 , 61 ] . The eigenvalues μ \mu μ T τ T_\tau T τ λ = τ − 1 log μ \lambda=\tau^{-1}\log\mu λ = τ − 1 log μ μ ≠ 0 \mu\ne0 μ = 0
Approximation and Galerkin projection of the solution operator
The BSO method has two stages. First, an estimator T ^ \widehat{T} T T T T V V V ψ 1 , … , ψ m \psi_1,\ldots,\psi_m ψ 1 , … , ψ m V V V x 1 , … , x n x_1,\ldots,x_n x 1 , … , x n Ω \Omega Ω
( Ψ ) k j : = ψ j ( x k ) , ( Ψ T ^ ) k j : = ( T ^ ψ j ) ( x k ) , Ψ , Ψ T ^ ∈ R n × m . (\mathbf{\Psi})_{kj}:=\psi_j(x_k),\qquad(\mathbf{\Psi}_{\widehat{T}})_{kj}:=(\widehat{T}\psi_j)(x_k),\qquad\mathbf{\Psi},\mathbf{\Psi}_{\widehat{T}}\in\mathbb{R}^{n\times m}. ( Ψ ) k j := ψ j ( x k ) , ( Ψ T ) k j := ( T ψ j ) ( x k ) , Ψ , Ψ T ∈ R n × m . where boldface distinguishes data matrices from bases. If Ψ \mathbf{\Psi} Ψ
K ^ : = Ψ † Ψ T ^ = ( Ψ ⊤ Ψ ) − 1 Ψ ⊤ Ψ T ^ . (1) \widehat{K}:=\mathbf{\Psi}^{\dagger}\mathbf{\Psi}_{\widehat{T}}=(\mathbf{\Psi}^{\top}\mathbf{\Psi})^{-1}\mathbf{\Psi}^{\top}\mathbf{\Psi}_{\widehat{T}}. \tag*{(1)} K := Ψ † Ψ T = ( Ψ ⊤ Ψ ) − 1 Ψ ⊤ Ψ T . (1) This is the EDMD matrix with the observed time-advanced values replaced by the values ( T ^ ψ j ) ( x k ) (\widehat{T}\psi_j)(x_k) ( T ψ j ) ( x k ) K ^ \widehat{K} K (1) into a matrix of inner products. For a linear estimator, this changes K ^ \widehat{K} K
Learning the solution operator
Let X X X t ≥ 0 t\ge0 t ≥ 0 ρ \rho ρ X + : = X t + τ X^+:=X_{t+\tau} X + := X t + τ T T T f f f
( T f ) ( X ) = E [ f ( X + ) ∣ X ] almost surely . (Tf)(X)=\mathbb{E}\left[f(X^+)\mid X\right]\quad\text{almost surely}. ( T f ) ( X ) = E [ f ( X + ) ∣ X ] almost surely . so that the time-lagged values of every observable are governed by T T T D o p \mathcal{D}_{\mathrm{op}} D op n o p n_{\mathrm{op}} n op f ( X k ) f(X_k) f ( X k ) f ( X k + ) f(X_k^+) f ( X k + ) f f f t 1 , … , t n o p t_1,\ldots,t_{n_{\mathrm{op}}} t 1 , … , t n op X k : = X t k X_k:=X_{t_k} X k := X t k X k + : = X t k + τ X_k^+:=X_{t_k+\tau} X k + := X t k + τ ( y t k , y t k + τ ) (y_{t_k},y_{t_k+\tau}) ( y t k , y t k + τ ) § 5 , they are observables f k f_k f k T f k Tf_k T f k T ^ \widehat{T} T T T T T ^ \widehat{T} T T T T § 13 ).
One technical point concerns the functions on which T ^ \widehat{T} T T ^ \widehat{T} T ρ \rho ρ H \mathcal{H} H F ~ \widetilde{\mathcal{F}} F Ω → R \Omega\to\mathbb{R} Ω → R ρ \rho ρ T ^ \widehat{T} T T ^ : F ~ → F ~ \widehat{T}:\widetilde{\mathcal{F}}\to\widetilde{\mathcal{F}} T : F → F Remark 3.2 . The space F ~ \widetilde{\mathcal{F}} F 1 1 1 T ^ \widehat{T} T F \mathcal{F} F ρ \rho ρ § 7 .
Remark 3.1. In the experiments of § 5 , T ^ \widehat{T} T ( f , T f ) (f,Tf) ( f , T f ) T T T f f f Remark 3.2. In the forecasting examples, the data are time series of observed variables y t = h ( X t ) y_t=h(X_t) y t = h ( X t ) Y Y Y h h h Φ ^ : Y → Y \widehat{\Phi}:Y\to Y Φ : Y → Y ( y t , y t + τ ) (y_t,y_{t+\tau}) ( y t , y t + τ ) y t + τ = Φ ^ ( y t ) y_{t+\tau}=\widehat{\Phi}(y_t) y t + τ = Φ ( y t ) T T T y y y Φ \Phi Φ Φ = Φ τ \Phi=\Phi^\tau Φ = Φ τ Ω = Y \Omega=Y Ω = Y ρ \rho ρ Y Y Y T f = f ∘ Φ Tf=f\circ\Phi T f = f ∘ Φ T ^ f : = f ∘ Φ ^ \widehat{T}f:=f\circ\widehat{\Phi} T f := f ∘ Φ f f f Φ ^ \widehat{\Phi} Φ f ∘ Φ ^ f\circ\widehat{\Phi} f ∘ Φ f ( y t + τ ) f(y_{t+\tau}) f ( y t + τ ) y t y_t y t f f f f f f f ( y t + τ ) f(y_{t+\tau}) f ( y t + τ ) y t y_t y t § 13 ). Other operator-learning and regression methods can be used in the same way.
Candidate spaces and the variational objective
Throughout, m ≥ 1 m\ge1 m ≥ 1 E ⋆ E_\star E ⋆ Θ \Theta Θ θ ∈ Θ \theta\in\Theta θ ∈ Θ ϕ θ = ( ϕ θ , 1 , … , ϕ θ , m ) \phi_\theta=(\phi_{\theta,1},\ldots,\phi_{\theta,m}) ϕ θ = ( ϕ θ , 1 , … , ϕ θ , m ) F \mathcal{F} F ρ \rho ρ V θ V_\theta V θ Ω \Omega Ω ϕ θ : Ω → R m \phi_\theta:\Omega\to\mathbb{R}^m ϕ θ : Ω → R m m m m θ \theta θ ϕ θ , j \phi_{\theta,j} ϕ θ , j θ \theta θ Θ \Theta Θ Θ p \Theta_p Θ p p ∈ N p\in\mathbb{N} p ∈ N Remark F.1 ; the level p p p Δ Θ p \Delta\Theta_p Δ Θ p η Θ p \eta\Theta_p η Θ p Θ \Theta Θ Θ = Θ p \Theta=\Theta_p Θ = Θ p
The observables to be represented, called reconstruction observables, form a finite set G : = { g 1 , … , g q } ⊂ F \mathcal{G}:=\{g_1,\ldots,g_q\}\subset\mathcal{F} G := { g 1 , … , g q } ⊂ F m m m V ⊂ H V\subset\mathcal{H} V ⊂ H
I ( V ) : = ∥ ( I − P V ) T ∣ V ∥ B ( V , H ) , R ( V ) : = max 1 ≤ j ≤ q ∥ ( I − P V ) g j ∥ . \mathcal{I}(V):=\|(I-P_V)T|_V\|_{\mathcal{B}(V,\mathcal{H})},\qquad\mathcal{R}(V):=\max_{1\le j\le q}\|(I-P_V)g_j\|. I ( V ) := ∥ ( I − P V ) T ∣ V ∥ B ( V , H ) , R ( V ) := 1 ≤ j ≤ q max ∥ ( I − P V ) g j ∥. and, for a weight α > 0 \alpha>0 α > 0
J ( V ) : = I ( V ) 2 + α R ( V ) 2 . \mathcal{J}(V):=\mathcal{I}(V)^2+\alpha\mathcal{R}(V)^2. J ( V ) := I ( V ) 2 + α R ( V ) 2 . The invariance defect I ( V ) \mathcal{I}(V) I ( V ) V V V T T T T ^ V \widehat{T}V T V V V V R ( V ) \mathcal{R}(V) R ( V ) V V V α \alpha α
For an m m m V ⊂ F V \subset\mathcal{F} V ⊂ F ∥ ⋅ ∥ \lVert\cdot\rVert ∥ ⋅ ∥ V θ V_\theta V θ V V V
Δ ( V ) : = ∥ T ^ ∣ V − T ∣ V ∥ B ( V , H ) , (2) \Delta(V) := \lVert\widehat{T}|_V - T|_V \rVert_{\mathcal{B}(V,\mathcal{H})}, \tag*{(2)} Δ ( V ) := ∥ T ∣ V − T ∣ V ∥ B ( V , H ) , (2) where V V V H \mathcal{H} H § 7 ); it is controlled by the accuracy of T ^ \widehat{T} T V V V
∥ T ^ ∣ V − T ^ ′ ∣ V ∥ B ( V , H ) ≤ ∥ T ^ ∣ V − T ∣ V ∥ + ∥ T ∣ V − T ^ ′ ∣ V ∥ = Δ ( V ) + Δ ′ ( V ) , (3) \lVert\widehat{T}|_V-\widehat{T}'|_V\rVert_{\mathcal{B}(V,\mathcal{H})}
\leq\lVert\widehat{T}|_V-T|_V\rVert+\lVert T|_V-\widehat{T}'|_V\rVert
=\Delta(V)+\Delta'(V), \tag*{(3)} ∥ T ∣ V − T ′ ∣ V ∥ B ( V , H ) ≤ ∥ T ∣ V − T ∣ V ∥ + ∥ T ∣ V − T ′ ∣ V ∥ = Δ ( V ) + Δ ′ ( V ) , (3) and each of the two computed Galerkin operators satisfies the bound (7) of Proposition 4.1 with respect to the same exact compression P V T ∣ V P_VT|_V P V T ∣ V T ^ V \widehat{T}V T V § 4 therefore hold for either choice of the basis, and by (3) the two choices give nearby operators whenever both learning errors are small.
Remark 3.3. Let X ∼ ρ X \sim\rho X ∼ ρ X + X^+ X + τ \tau τ X X X φ ∈ H \varphi\in\mathcal{H} φ ∈ H E ∣ φ ( X + ) ∣ 2 < ∞ \mathbb{E}|\varphi(X^+)|^2 < \infty E ∣ φ ( X + ) ∣ 2 < ∞ μ ∈ C \mu\in\mathbb{C} μ ∈ C
E ∣ φ ( X + ) − μ φ ( X ) ∣ 2 = ∥ T φ − μ φ ∥ 2 + E ∣ φ ( X + ) − ( T φ ) ( X ) ∣ 2 , \mathbb{E}|\varphi(X^+) - \mu\varphi(X)|^2
= \lVert T\varphi-\mu\varphi\rVert^2
+ \mathbb{E}|\varphi(X^+) - (T\varphi)(X)|^2, E ∣ φ ( X + ) − μ φ ( X ) ∣ 2 = ∥ T φ − μ φ ∥ 2 + E ∣ φ ( X + ) − ( T φ ) ( X ) ∣ 2 , because φ ( X + ) − ( T φ ) ( X ) \varphi(X^+) - (T\varphi)(X) φ ( X + ) − ( T φ ) ( X ) X X X ( T φ − μ φ ) ( X ) (T\varphi-\mu\varphi)(X) ( T φ − μ φ ) ( X ) [1 , 3 ] .
Computational realization
Let x 1 , … , x n x_1,\ldots,x_n x 1 , … , x n (1) , either drawn independently from ρ \rho ρ ρ \rho ρ n n n n o p n_{\mathrm{op}} n op T ^ \widehat{T} T [2 ] . The empirical inner product and seminorm are
⟨ u , v ⟩ n : = ∑ k = 1 n w k u ( x k ) ‾ v ( x k ) , ∥ u ∥ n : = ⟨ u , u ⟩ n 1 / 2 , u , v ∈ F , \langle u,v\rangle_n := \sum_{k=1}^{n} w_k \overline{u(x_k)}v(x_k), \qquad\|u\|_n := \langle u,u\rangle_n^{1/2}, \qquad u,v\in\mathcal{F}, ⟨ u , v ⟩ n := k = 1 ∑ n w k u ( x k ) v ( x k ) , ∥ u ∥ n := ⟨ u , u ⟩ n 1/2 , u , v ∈ F , with the weights w k = 1 / n w_k=1/n w k = 1/ n x k x_k x k ρ \rho ρ w k w_k w k (1) replaced by the corresponding weighted least-squares matrix; the sampling error η \eta η
Let V = V θ V=V_\theta V = V θ θ ∈ Θ \theta\in\Theta θ ∈ Θ [ ⟨ ϕ θ , i , ϕ θ , j ⟩ n ] i , j = 1 m \left[\langle\phi_{\theta,i},\phi_{\theta,j}\rangle_n\right]_{i,j=1}^m [ ⟨ ϕ θ , i , ϕ θ , j ⟩ n ] i , j = 1 m ϕ θ \phi_\theta ϕ θ n ≥ m n\ge m n ≥ m Ψ ^ = ( ψ ^ 1 , … , ψ ^ m ) \widehat{\Psi}=(\widehat{\psi}_1,\ldots,\widehat{\psi}_m) Ψ = ( ψ 1 , … , ψ m ) V V V ϕ θ \phi_\theta ϕ θ ⟨ ⋅ , ⋅ ⟩ n \langle\cdot,\cdot\rangle_n ⟨ ⋅ , ⋅ ⟩ n Ψ ^ \widehat{\Psi} Ψ V V V V V V ⟨ ⋅ , ⋅ ⟩ n \langle\cdot,\cdot\rangle_n ⟨ ⋅ , ⋅ ⟩ n P ^ V \widehat{P}_V P V ⟨ ⋅ , ⋅ ⟩ n \langle\cdot,\cdot\rangle_n ⟨ ⋅ , ⋅ ⟩ n V V V F \mathcal{F} F K ^ \widehat{K} K (1) for the computational basis. Since Ψ ^ T Ψ ^ = n I \widehat{\Psi}^{\mathsf T}\widehat{\Psi}=nI Ψ T Ψ = n I
K ^ = [ ⟨ ψ ^ i , T ^ ψ ^ j ⟩ n ] i , j = 1 m , L ^ = [ ⟨ T ^ ψ ^ i , T ^ ψ ^ j ⟩ n ] i , j = 1 m . \widehat{K}=\left[\langle\widehat{\psi}_i,\widehat{T}\widehat{\psi}_j\rangle_n\right]_{i,j=1}^{m}, \qquad\widehat{L}=\left[\langle\widehat{T}\widehat{\psi}_i,\widehat{T}\widehat{\psi}_j\rangle_n\right]_{i,j=1}^{m}. K = [ ⟨ ψ i , T ψ j ⟩ n ] i , j = 1 m , L = [ ⟨ T ψ i , T ψ j ⟩ n ] i , j = 1 m . The matrix L ^ \widehat{L} L [18 ] , with the values of the estimator in place of the observed values. The matrix K ^ \widehat{K} K Ψ ^ \widehat{\Psi} Ψ
K ^ : = P ^ V T ^ ∣ V ∈ B ( V ) , K ^ ψ ^ j = ∑ i = 1 m K ^ i j ψ ^ i . (4) \widehat{\mathcal{K}}:=\widehat{P}_V\widehat{T}|_V\in\mathcal{B}(V), \qquad\widehat{\mathcal{K}}\widehat{\psi}_j=\sum_{i=1}^{m}\widehat{K}_{ij}\widehat{\psi}_i. \tag*{(4)} K := P V T ∣ V ∈ B ( V ) , K ψ j = i = 1 ∑ m K ij ψ i . (4) The eigenvalues of K ^ \widehat{K} K K ^ \widehat{\mathcal{K}} K c c c K ^ \widehat{K} K ∑ j c j ψ ^ j \sum_j c_j\widehat{\psi}_j ∑ j c j ψ j K ^ \widehat{\mathcal{K}} K V = V θ V=V_\theta V = V θ
J ^ ( θ ) : = λ max ( L ^ − K ^ T K ^ ) + α max 1 ≤ j ≤ q ∥ ( I − P ^ V ) g j ∥ n 2 . (5) \widehat{J}(\theta):=\lambda_{\max}\left(\widehat{L}-\widehat{K}^{\mathsf T}\widehat{K}\right)+\alpha\max_{1\le j\le q}\left\|(I-\widehat{P}_V)g_j\right\|_n^2. \tag*{(5)} J ( θ ) := λ m a x ( L − K T K ) + α 1 ≤ j ≤ q max ( I − P V ) g j n 2 . (5) When ⟨ ⋅ , ⋅ ⟩ n \langle\cdot,\cdot\rangle_n ⟨ ⋅ , ⋅ ⟩ n V + T ^ V V+\widehat{T}V V + T V V V V T ^ \widehat{T} T ⟨ ⋅ , ⋅ ⟩ n \langle\cdot,\cdot\rangle_n ⟨ ⋅ , ⋅ ⟩ n J ^ ( θ ) \widehat{J}(\theta) J ( θ ) J ( V θ ) J(V_\theta) J ( V θ ) T T T T ^ \widehat{T} T
Algorithm 3.1 summarizes the procedure. Training the estimator T ^ \widehat{T} T
Relation to EDMD
Consider a fixed space V V V X k X_k X k X k + X_k^+ X k + ( X k , X k + ) (X_k,X_k^+) ( X k , X k + ) X k X_k X k x k x_k x k T ^ \widehat{T} T (1) by the matrix
Algorithm 3.1 Fixed-lag solution-operator approximation and Galerkin projection
Input Data D o p D_{\mathrm{op}} D op T ^ \widehat{T} T x 1 , … , x n x_1,\ldots,x_n x 1 , … , x n { φ θ } θ ∈ Θ \{\varphi_\theta\}_{\theta\in\Theta} { φ θ } θ ∈ Θ G \mathcal{G} G α > 0 \alpha>0 α > 0 ζ ≥ 0 \zeta\geq0 ζ ≥ 0 f ∈ span G f\in\operatorname{span}\mathcal{G} f ∈ span G N N N
Output Space V ^ \widehat{V} V Ψ ^ \widehat{\Psi} Ψ K ^ \widehat{K} K
Construct T ^ \widehat{T} T D o p D_{\mathrm{op}} D op
if the dictionary is fixed in advance then
Set V ^ \widehat{V} V Ψ ^ \widehat{\Psi} Ψ ⟨ ⋅ , ⋅ ⟩ n \langle\cdot,\cdot\rangle_n ⟨ ⋅ , ⋅ ⟩ n
else
Compute θ ^ ∈ Θ \widehat{\theta}\in\Theta θ ∈ Θ J ^ ( θ ^ ) ≤ inf θ ∈ Θ J ^ ( θ ) + ζ \widehat{\mathcal{J}}(\widehat{\theta})\leq\inf_{\theta\in\Theta}\widehat{\mathcal{J}}(\theta)+\zeta J ( θ ) ≤ inf θ ∈ Θ J ( θ ) + ζ V ^ ← V θ ^ \widehat{V}\leftarrow V_{\widehat{\theta}} V ← V θ Ψ ^ \widehat{\Psi} Ψ
end if
Evaluate T ^ ψ j \widehat{T}\psi_j T ψ j j = 1 , … , m j=1,\ldots,m j = 1 , … , m K ^ ← [ ⟨ ψ ^ i , T ^ ψ j ⟩ n ] i , j = 1 m \widehat{K}\leftarrow\left[\langle\widehat{\psi}_i,\widehat{T}\psi_j\rangle_n\right]_{i,j=1}^m K ← [ ⟨ ψ i , T ψ j ⟩ n ] i , j = 1 m (4) ).
Compute the required eigenvalues, eigenvectors, or Schur vectors of K ^ \widehat{K} K
if f f f N N N
Compute c ^ : = ( ⟨ ψ ^ j , f ⟩ n ) j = 1 m \widehat{c}:=\left(\langle\widehat{\psi}_j,f\rangle_n\right)_{j=1}^m c := ( ⟨ ψ j , f ⟩ n ) j = 1 m P ^ V ^ f \widehat{P}_{\widehat{V}}f P V f ∑ j ( K ^ N c ^ ) j ψ ^ j \sum_j(\widehat{K}^N\widehat{c})_j\widehat{\psi}_j ∑ j ( K N c ) j ψ j
end if
of observed values ψ j ( X k + ) \psi_j(X_k^+) ψ j ( X k + ) [67 ] , which converges to the Galerkin matrix of T T T [32 ] . For a fixed T ^ \widehat{T} T T ^ \widehat{T} T T T T Δ ( V ) \Delta(V) Δ ( V ) [38 ] . The distinction here is that T ^ \widehat{T} T [17 ] .
Rank constraints with few transition pairs. Let Ψ X , Ψ X + ∈ R n o p × m \Psi_X,\Psi_{X^+}\in\mathbb{R}^{n_{\mathrm{op}}\times m} Ψ X , Ψ X + ∈ R n op × m ψ j ( X k ) \psi_j(X_k) ψ j ( X k ) ψ j ( X k + ) \psi_j(X_k^+) ψ j ( X k + ) ψ 1 , … , ψ m \psi_1,\ldots,\psi_m ψ 1 , … , ψ m V V V K E K^{\mathrm{E}} K E G K E = A E GK^{\mathrm{E}}=A^{\mathrm{E}} G K E = A E
G : = 1 n o p Ψ X T Ψ X , A E : = 1 n o p Ψ X T Ψ X + . G:=\frac{1}{n_{\mathrm{op}}}\Psi_X^{\mathsf{T}}\Psi_X,\qquad A^{\mathrm{E}}:=\frac{1}{n_{\mathrm{op}}}\Psi_X^{\mathsf{T}}\Psi_{X^+}. G := n op 1 Ψ X T Ψ X , A E := n op 1 Ψ X T Ψ X + . If n o p < m n_{\mathrm{op}}<m n op < m G G G K E = G † A E K^{\mathrm{E}}=G^\dagger A^{\mathrm{E}} K E = G † A E rank ( K E ) ≤ rank ( A E ) ≤ n o p \operatorname{rank}(K^{\mathrm{E}})\leq\operatorname{rank}(A^{\mathrm{E}})\leq n_{\mathrm{op}} rank ( K E ) ≤ rank ( A E ) ≤ n op G G G n o p ≥ m n_{\mathrm{op}}\geq m n op ≥ m n o p n_{\mathrm{op}} n op T ^ \widehat{T} T K ^ \widehat{K} K x k x_k x k X k + X_k^+ X k + Φ ^ \widehat{\Phi} Φ n ≥ m n\geq m n ≥ m K ^ \widehat{K} K n o p < m n_{\mathrm{op}}<m n op < m n o p n_{\mathrm{op}} n op m × m m \times m m × m T ^ \widehat{T} T K ^ \widehat{K} K [16 ] for the regime of fewer snapshots than dictionary functions within residual DMD.
Error analysis for spectral approximation and forecasting
Throughout this section, V ⊂ F V \subset\mathcal{F} V ⊂ F m m m Ψ ^ = ( ψ ^ 1 , … , ψ ^ m ) \widehat{\Psi}=(\widehat{\psi}_1,\ldots,\widehat{\psi}_m) Ψ = ( ψ 1 , … , ψ m ) ⟨ ⋅ , ⋅ ⟩ n \langle\cdot,\cdot\rangle_n ⟨ ⋅ , ⋅ ⟩ n K ^ = P ^ V T ^ ∣ V \widehat{K}=\widehat{P}_V\widehat{T}|_V K = P V T ∣ V (4) ), and K ^ \widehat{K} K ρ \rho ρ ρ \rho ρ V V V χ \chi χ
The analysis compares K ^ \widehat{K} K T T T Δ ( V ) \Delta(V) Δ ( V ) (2) ), the error of the first stage on the unit ball of V V V T ^ \widehat{T} T V V V η V \eta_V η V I ( V ) \mathcal{I}(V) I ( V ) R ( V ) \mathcal{R}(V) R ( V )
The section is organized as follows. Subsection 4.1 defines the sampling error and gives the basic estimate: one application of K ^ \widehat{K} K V V V T T T ( I ( V ) + Δ ( V ) ) / ( 1 − η V ) (\mathcal{I}(V)+\Delta(V))/(1-\eta_V) ( I ( V ) + Δ ( V )) / ( 1 − η V ) V V V T T T E ⋆ E_\star E ⋆ J \mathcal{J} J I \mathcal{I} I R \mathcal{R} R Theorem 4.4 relates I ( V ) \mathcal{I}(V) I ( V ) V V V E ⋆ E_\star E ⋆ § 7 and B; supplementary results are collected in Appendix D.
One-step error
The Galerkin matrix K ^ \widehat{K} K ⟨ ⋅ , ⋅ ⟩ n \langle\cdot,\cdot\rangle_n ⟨ ⋅ , ⋅ ⟩ n L 2 ( ρ ) L^2(\rho) L 2 ( ρ ) W ⊂ F W \subset\mathcal{F} W ⊂ F ∥ ⋅ ∥ \|\cdot\| ∥ ⋅ ∥ ρ \rho ρ η \eta η W W W
η ( W ) : = sup { ∣ ⟨ u , v ⟩ n − ⟨ u , v ⟩ ∣ : u , v ∈ W , ∥ u ∥ = ∥ v ∥ = 1 } = ∥ G ^ W − I ∥ 2 . (6) \eta(W) := \sup\{|\langle u,v\rangle_n-\langle u,v\rangle|:u,v\in W,\ \|u\|=\|v\|=1\}=\|\widehat{G}_W-I\|_2. \tag*{(6)} η ( W ) := sup { ∣ ⟨ u , v ⟩ n − ⟨ u , v ⟩ ∣ : u , v ∈ W , ∥ u ∥ = ∥ v ∥ = 1 } = ∥ G W − I ∥ 2 . (6) where G ^ W : = [ ⟨ χ i W , χ j W ⟩ n ] i , j = 1 dim W \widehat{G}_W := [\langle\chi_i^W,\chi_j^W\rangle_n]_{i,j=1}^{\dim W} G W := [⟨ χ i W , χ j W ⟩ n ] i , j = 1 d i m W ρ \rho ρ χ W \chi^W χ W W W W ⟨ u , v ⟩ n − ⟨ u , v ⟩ = a ⊤ ( G ^ W − I ) b \langle u,v\rangle_n-\langle u,v\rangle=a^\top(\widehat{G}_W-I)b ⟨ u , v ⟩ n − ⟨ u , v ⟩ = a ⊤ ( G W − I ) b u , v ∈ W u,v\in W u , v ∈ W a a a b b b V V V V V V
W V : = V + T ^ V + span G , η V : = η ( W V ) , κ ( η ) : = ( 1 + η 1 − η ) 1 / 2 . W_V:=V+\widehat{T}V+\operatorname{span}\mathcal{G},\qquad\eta_V:=\eta(W_V),\qquad\kappa(\eta):=\left(\frac{1+\eta}{1-\eta}\right)^{1/2}. W V := V + T V + span G , η V := η ( W V ) , κ ( η ) := ( 1 − η 1 + η ) 1/2 . We assume η V < 1 \eta_V<1 η V < 1 n ≥ dim W V n\geq\dim W_V n ≥ dim W V dim W V ≤ 2 m + q \dim W_V\leq2m+q dim W V ≤ 2 m + q ρ \rho ρ n − 1 / 2 n^{-1/2} n − 1/2 [14 , 15 , 65 ] and for ergodic trajectories with a spectral or pseudo-spectral gap by [52 ] (Appendix B).
Proposition 4.1 (One-step error bound). Let η : = η V < 1 \eta:=\eta_V<1 η := η V < 1
∥ K ^ − P V T ∣ V ∥ ≤ Δ ( V ) + η I ( V ) 1 − η , (7) \left\|\widehat{K}-P_VT|_V\right\|\leq\frac{\Delta(V)+\eta\,\mathcal{I}(V)}{1-\eta}, \tag*{(7)} K − P V T ∣ V ≤ 1 − η Δ ( V ) + η I ( V ) , (7) and
∥ ( T − K ^ ) v ∥ ≤ e ( V ) ∥ v ∥ , v ∈ V , where e ( V ) : = I ( V ) + Δ ( V ) 1 − η . (8) \|(T-\widehat{K})v\|\leq e(V)\|v\|,\qquad v\in V,\qquad\text{where}\quad e(V):=\frac{\mathcal{I}(V)+\Delta(V)}{1-\eta}. \tag*{(8)} ∥ ( T − K ) v ∥ ≤ e ( V ) ∥ v ∥ , v ∈ V , where e ( V ) := 1 − η I ( V ) + Δ ( V ) . (8) The proposition holds for every space and every estimator and does not bound I ( V ) \mathcal{I}(V) I ( V ) Δ ( V ) \Delta(V) Δ ( V ) Theorem 4.2 and the learning error of an FNO in Corollary 4.6 . The estimate decomposes the one-step error into two parts: the invariance defect I ( V ) \mathcal{I}(V) I ( V ) P V T ∣ V P_VT|_V P V T ∣ V T T T V V V (7) . The sampling error only inflates these terms, by a factor that tends to one as η → 0 \eta\to0 η → 0 P ^ V \widehat{P}_V P V V V V Theorem 4.3 uses (7) , and the forecast bound of Theorem 4.5 uses (8) .
Data reuse. The deterministic bounds of this section hold for any sample points, whether or not these coincide with the states at which the data for T ^ \widehat{T} T η V \eta_V η V T ^ \widehat{T} T V V V η Θ \eta_{\Theta} η Θ Theorem 4.2 ; when the estimator is nonlinear, it also means uniform control over the spaces generated by the computational bases (Remark 3.2 ). The case in which this uniform control reduces to a bound for a fixed space is described in Appendix B; for neural dictionaries with trainable hidden weights, the uniform control is a hypothesis that is not verified here.
Selection of the observable space
When the space is not fixed in advance, Algorithm 3.1 selects it variationally, by minimizing the computed objective (5) over the candidates, all of which are evaluated on the same sample. The computed objective differs from J \mathcal{J} J T ^ \widehat{T} T T T T L 2 ( ρ ) L^2(\rho) L 2 ( ρ ) J \mathcal{J} J θ ^ ∈ Θ \widehat{\theta} \in\Theta θ ∈ Θ
J ^ ( θ ^ ) ≤ inf θ ∈ Θ J ^ ( θ ) + ζ , ζ ≥ 0 , (9) \widehat{\mathcal{J}}(\widehat{\theta}) \leq\inf_{\theta\in\Theta}\widehat{\mathcal{J}}(\theta) + \zeta,\qquad\zeta\geq0, \tag*{(9)} J ( θ ) ≤ θ ∈ Θ inf J ( θ ) + ζ , ζ ≥ 0 , (9) and write V ^ : = V θ ^ \widehat{V} := V_{\widehat{\theta}} V := V θ θ ^ \widehat{\theta} θ V ^ \widehat{V} V
Δ Θ : = sup θ ∈ Θ Δ ( V θ ) , η Θ : = sup θ ∈ Θ η V θ . \Delta_{\Theta} := \sup_{\theta\in\Theta}\Delta(V_{\theta}),\qquad\eta_{\Theta} := \sup_{\theta\in\Theta}\eta_{V_{\theta}}. Δ Θ := θ ∈ Θ sup Δ ( V θ ) , η Θ := θ ∈ Θ sup η V θ . Theorem 4.2 (Quasi-optimality of the variational selection). If η Θ < 1 \eta_{\Theta}<1 η Θ < 1
J ( V ^ ) 1 / 2 ≤ ε Θ : = κ ( η Θ ) 2 ( inf θ ∈ Θ J ( V θ ) 1 / 2 + 2 Δ Θ + ζ 1 / 2 ) . (10) \mathcal{J}(\widehat{V})^{1/2} \leq\varepsilon_{\Theta} := \kappa(\eta_{\Theta})^2\left(\inf_{\theta\in\Theta}\mathcal{J}(V_{\theta})^{1/2}+2\Delta_{\Theta}+\zeta^{1/2}\right). \tag*{(10)} J ( V ) 1/2 ≤ ε Θ := κ ( η Θ ) 2 ( θ ∈ Θ inf J ( V θ ) 1/2 + 2 Δ Θ + ζ 1/2 ) . (10) The selected space is thus nearly as good as the best candidate; with exact inner products, it is as good up to 2 Δ Θ + ζ 1 / 2 2\Delta_{\Theta}+\zeta^{1/2} 2 Δ Θ + ζ 1/2 (10) separates the four error sources of the selection step, namely the approximation capacity of the candidate family, inf θ J ( V θ ) \inf_{\theta}\mathcal{J}(V_{\theta}) inf θ J ( V θ ) Δ Θ \Delta_{\Theta} Δ Θ ζ \zeta ζ κ ( η Θ ) 2 = ( 1 + η Θ ) / ( 1 − η Θ ) \kappa(\eta_{\Theta})^2=(1+\eta_{\Theta})/(1-\eta_{\Theta}) κ ( η Θ ) 2 = ( 1 + η Θ ) / ( 1 − η Θ ) I ( V ^ ) ≤ ε Θ \mathcal{I}(\widehat{V})\leq\varepsilon_{\Theta} I ( V ) ≤ ε Θ R ( V ^ ) ≤ ε Θ / α 1 / 2 \mathcal{R}(\widehat{V})\leq\varepsilon_{\Theta}/\alpha^{1/2} R ( V ) ≤ ε Θ / α 1/2 (8) of the selected space, e ( V ^ ) ≤ ( ε Θ + Δ Θ ) / ( 1 − η Θ ) e(\widehat{V})\leq(\varepsilon_{\Theta}+\Delta_{\Theta})/(1-\eta_{\Theta}) e ( V ) ≤ ( ε Θ + Δ Θ ) / ( 1 − η Θ ) η V ^ ≤ η Θ \eta_{\widehat{V}}\leq\eta_{\Theta} η V ≤ η Θ Δ ( V ^ ) ≤ Δ Θ \Delta(\widehat{V})\leq\Delta_{\Theta} Δ ( V ) ≤ Δ Θ J ( V ^ ) 1 / 2 ≤ κ ( η Θ ) J ^ ( θ ^ ) 1 / 2 + Δ Θ \mathcal{J}(\widehat{V})^{1/2}\leq\kappa(\eta_{\Theta})\widehat{\mathcal{J}}(\widehat{\theta})^{1/2}+\Delta_{\Theta} J ( V ) 1/2 ≤ κ ( η Θ ) J ( θ ) 1/2 + Δ Θ
The objective is Lipschitz continuous with respect to dist \operatorname{dist} dist 4 M T 2 + 2 α max j ∥ g j ∥ 2 4M_T^2+2\alpha\max_j\lVert g_j\rVert^2 4 M T 2 + 2 α max j ∥ g j ∥ 2 I \mathcal{I} I R \mathcal{R} R 2 M T 2M_T 2 M T max j ∥ g j ∥ \max_j\lVert g_j\rVert max j ∥ g j ∥ M T M_T M T max j ∥ g j ∥ \max_j\lVert g_j\rVert max j ∥ g j ∥ V θ V_{\theta} V θ θ ∈ Θ p \theta\in\Theta_p θ ∈ Θ p p → ∞ p\to\infty p → ∞ m m m J \mathcal{J} J Δ Θ p \Delta_{\Theta_p} Δ Θ p η Θ p \eta_{\Theta_p} η Θ p ζ \zeta ζ p p p (10) shows that J ( V ^ ) \mathcal{J}(\widehat{V}) J ( V ) V ^ \widehat{V} V p p p T T T E ⋆ E_\star E ⋆ E ⋆ E_\star E ⋆ dist ( V ^ , E ⋆ ) → 0 \operatorname{dist}(\widehat{V},E_\star)\to0 dist ( V , E ⋆ ) → 0 Lemma D.1 , and Theorem 4.3 gives the convergence of the computed eigenvalues and spectral subspaces. The choice of the level p p p
Spectral approximation
Let E ⋆ E_\star E ⋆ m m m T T T T ∣ E ⋆ T|_{E_\star} T ∣ E ⋆ V V V E ⋆ E_\star E ⋆ E ⋆ E_\star E ⋆
Theorem 4.3 (Spectral error bound). Let η : = η V < 1 \eta:= \eta_V < 1 η := η V < 1 d : = dist ( V , E ⋆ ) < 1 d := \operatorname{dist}(V,E_\star) < 1 d := dist ( V , E ⋆ ) < 1 Q : E ⋆ → V Q:E_\star\to V Q : E ⋆ → V ∥ Q − ι ∥ B ( E ⋆ , H ) ≤ 2 d \lVert Q-\iota\rVert_{\mathcal{B}(E_\star,\mathcal{H})} \le2d ∥ Q − ι ∥ B ( E ⋆ , H ) ≤ 2 d ι : E ⋆ → H \iota:E_\star\to\mathcal{H} ι : E ⋆ → H Q Q Q H \mathcal{H} H
∥ Q ∗ K ^ Q − T ∣ E ⋆ ∥ ≤ Δ ( V ) + 4 M T d 1 − η . (11) \lVert Q^*\widehat{K}Q-T|_{E_\star}\rVert\le\frac{\Delta(V)+4M_Td}{1-\eta}. \tag*{(11)} ∥ Q ∗ K Q − T ∣ E ⋆ ∥ ≤ 1 − η Δ ( V ) + 4 M T d . (11) Consequently, consider a sequence of m m m V k V_k V k k k k K ^ k \widehat{K}_k K k K ^ k \widehat{K}_k K k Δ ( V k ) → 0 \Delta(V_k) \to0 Δ ( V k ) → 0 dist ( V k , E ⋆ ) → 0 \operatorname{dist}(V_k,E_\star) \to0 dist ( V k , E ⋆ ) → 0 sup k η V k < 1 \sup_k\eta_{V_k}<1 sup k η V k < 1 K ^ k \widehat{K}_k K k T ∣ E ⋆ T|_{E_\star} T ∣ E ⋆ K ^ k \widehat{K}_k K k T ∣ E ⋆ T|_{E_\star} T ∣ E ⋆
The operator Q ∗ K ^ Q = Q − 1 K ^ Q Q^*\widehat{K}Q=Q^{-1}\widehat{K}Q Q ∗ K Q = Q − 1 K Q E ⋆ E_\star E ⋆ K ^ \widehat{K} K (11) bounds its distance from T ∣ E ⋆ T|_{E_\star} T ∣ E ⋆ E ⋆ E_\star E ⋆
The theorem concerns spaces V V V E ⋆ E_\star E ⋆ V V V V V V
We turn to the second question. The distance dist ( V , E ⋆ ) \operatorname{dist}(V,E_\star) dist ( V , E ⋆ ) Theorem 4.3 is not known in practice, whereas the invariance defect is controlled by the selection step (Theorem 4.2 ). The next theorem shows that the defect and the distance are equivalent near E ⋆ E_\star E ⋆ E ⋆ E_\star E ⋆ H = E ⋆ ⊕ E ⋆ ⊥ \mathcal{H}=E_\star\oplus E_\star^\perp H = E ⋆ ⊕ E ⋆ ⊥ T T T A ⋆ : = T ∣ E ⋆ A_\star:=T|_{E_\star} A ⋆ := T ∣ E ⋆ D ⋆ : = ( I − P ⋆ ) T ∣ E ⋆ ⊥ D_\star:=(I-P_\star)T|_{E_\star^\perp} D ⋆ := ( I − P ⋆ ) T ∣ E ⋆ ⊥ B ⋆ : = P ⋆ T ∣ E ⋆ ⊥ B_\star:=P_\star T|_{E_\star^\perp} B ⋆ := P ⋆ T ∣ E ⋆ ⊥
δ ⋆ : = inf { ∥ D ⋆ Z − Z A ⋆ ∥ : Z ∈ B ( E ⋆ , E ⋆ ⊥ ) , ∥ Z ∥ = 1 } , r ⋆ : = min { 1 , δ ⋆ 2 ∥ B ⋆ ∥ } . \delta_\star:=\inf\{\lVert D_\star Z-ZA_\star\rVert: Z\in\mathcal{B}(E_\star,E_\star^\perp),\ \lVert Z\rVert=1\},\qquad r_\star:=\min\left\{1,\frac{\delta_\star}{2\lVert B_\star\rVert}\right\}. δ ⋆ := inf {∥ D ⋆ Z − Z A ⋆ ∥ : Z ∈ B ( E ⋆ , E ⋆ ⊥ ) , ∥ Z ∥ = 1 } , r ⋆ := min { 1 , 2 ∥ B ⋆ ∥ δ ⋆ } . with r ⋆ : = 1 r_\star:=1 r ⋆ := 1 B ⋆ = 0 B_\star=0 B ⋆ = 0 δ ⋆ \delta_\star δ ⋆ A ⋆ A_\star A ⋆ D ⋆ D_\star D ⋆ [60 , 61 ] , and r ⋆ r_\star r ⋆ E ⋆ E_\star E ⋆ E ⋆ E_\star E ⋆ T T T m m m ({xref}Appendix D.1`).
Theorem 4.4 (Invariance defect and distance). Let δ ⋆ > 0 \delta_\star>0 δ ⋆ > 0 V V V m m m dist ( V , E ⋆ ) ≤ r ⋆ / 2 \operatorname{dist}(V,E_\star)\le r_\star/\sqrt{2} dist ( V , E ⋆ ) ≤ r ⋆ / 2
δ ⋆ 4 dist ( V , E ⋆ ) ≤ I ( V ) ≤ 2 M T dist ( V , E ⋆ ) . (12) \frac{\delta_\star}{4}\operatorname{dist}(V,E_\star)\le\mathcal{I}(V)\le2M_T\operatorname{dist}(V,E_\star). \tag*{(12)} 4 δ ⋆ dist ( V , E ⋆ ) ≤ I ( V ) ≤ 2 M T dist ( V , E ⋆ ) . (12) The upper bound holds for every m m m V V V δ ⋆ > 0 \delta_\star>0 δ ⋆ > 0
The theorem is a variant of Stewart’s perturbation theorem for invariant subspaces [60 , 61 ] , in which the separation is measured at the target subspace E ⋆ E_\star E ⋆ V V V E ⋆ E_\star E ⋆ ε Θ / α 1 / 2 \varepsilon_\Theta/\alpha^{1/2} ε Θ / α 1/2 dist ( V ^ , E ⋆ ) ≤ 4 ε Θ / δ ⋆ \operatorname{dist}(\widehat{V},E_\star) \leq4\varepsilon_\Theta/\delta_\star dist ( V , E ⋆ ) ≤ 4 ε Θ / δ ⋆ Theorem 4.3 bounds its spectral error in terms of the quantities of Theorem 4.2 . The precise conditions and the bias caused by the reconstruction term are given in Appendix D.§ 1 (Corollary D.2 ).
Forecast bounds
For an observable f ∈ span G f \in\operatorname{span}\mathcal{G} f ∈ span G Algorithm 3.1 returns the Galerkin forecast
f ( N ) : = K ^ N P ^ V f = ∑ j = 1 m ( K ^ N c ^ ) j ψ ^ j , c ^ : = ( ⟨ ψ ^ j , f ⟩ ) j = 1 m f^{(N)} := \widehat{K}^{N}\widehat{P}_{V}f = \sum_{j=1}^{m}(\widehat{K}^{N}\widehat{c})_j\widehat{\psi}_j,\qquad\widehat{c} := \left(\langle\widehat{\psi}_j,f\rangle\right)_{j=1}^{m} f ( N ) := K N P V f = j = 1 ∑ m ( K N c ) j ψ j , c := ( ⟨ ψ j , f ⟩ ) j = 1 m of T N f = T N τ f T^Nf = T_{N\tau}f T N f = T N τ f N N N f ∈ W V f \in W_V f ∈ W V η V \eta_V η V M : = max { M T , ∥ K ^ ∥ } M := \max\{M_T,\lVert\widehat{K}\rVert\} M := max { M T , ∥ K ∥} ∥ K ^ ∥ \lVert\widehat{K}\rVert ∥ K ∥ K ^ \widehat{K} K V V V L 2 ( ρ ) L^2(\rho) L 2 ( ρ )
Theorem 4.5 (Forecast error bound). Let η : = η V < 1 \eta:= \eta_V < 1 η := η V < 1 e ( V ) e(V) e ( V ) Proposition 4.1 . Then, for every N ≥ 1 N \geq1 N ≥ 1
∥ T N f − f ( N ) ∥ ≤ M N 1 − η ∥ ( I − P V ) f ∥ + N M N − 1 e ( V ) ∥ P V f ∥ . (13) \lVert T^Nf-f^{(N)}\rVert\leq\frac{M^N}{1-\eta}\lVert(I-P_V)f\rVert+ NM^{N-1}e(V)\lVert P_Vf\rVert. \tag*{(13)} ∥ T N f − f ( N ) ∥ ≤ 1 − η M N ∥( I − P V ) f ∥ + N M N − 1 e ( V ) ∥ P V f ∥ . (13) If, moreover, V = V ^ V=\widehat{V} V = V Theorem 4.2 , η Θ < 1 \eta_\Theta< 1 η Θ < 1 f ∈ G f \in\mathcal{G} f ∈ G
∥ T N f − f ( N ) ∥ ≤ 1 1 − η Θ ( M N ε Θ α 1 / 2 + N M N − 1 ( ε Θ + Δ Θ ) ∥ f ∥ ) . (14) \lVert T^Nf-f^{(N)}\rVert\leq\frac{1}{1-\eta_\Theta}\left(M^N\frac{\varepsilon_\Theta}{\alpha^{1/2}}+NM^{N-1}(\varepsilon_\Theta+\Delta_\Theta)\lVert f\rVert\right). \tag*{(14)} ∥ T N f − f ( N ) ∥ ≤ 1 − η Θ 1 ( M N α 1/2 ε Θ + N M N − 1 ( ε Θ + Δ Θ ) ∥ f ∥ ) . (14) For N = 1 N=1 N = 1 f ∈ V f \in V f ∈ V (13) is the one-step error (8) ; the general case follows by applying Proposition 4.1 at each of the N N N f f f V V V N N N M ≤ 1 M \leq1 M ≤ 1 N N N (14) expresses the forecast error through M M M Theorem 4.2 .
If ρ \rho ρ V V V M N M^N M N N M N − 1 NM^{N-1} N M N − 1 N N N Corollary D.4 ). This result, a form whose constraints can be imposed from the sample, and the effect of the corresponding modifications of K ^ \widehat{K} K § 3 .
A learning rate for Fourier neural operators
The preceding results involve the accuracy of the first stage only through the learning error Δ \Delta Δ Δ \Delta Δ Δ \Delta Δ T ^ f = f ∘ Φ ^ \widehat{T}f=f\circ\widehat{\Phi} T f = f ∘ Φ Φ ^ \widehat{\Phi} Φ Φ \Phi Φ ( T ^ − T ) v = v ∘ Φ ^ − v ∘ Φ (\widehat{T}-T)v=v\circ\widehat{\Phi}-v\circ\Phi ( T − T ) v = v ∘ Φ − v ∘ Φ dist Ω \operatorname{dist}_{\Omega} dist Ω Ω = Y \Omega=Y Ω = Y Lip ( V ) \operatorname{Lip}(V) Lip ( V ) v ∈ V v\in V v ∈ V ∥ v ∥ = 1 \lVert v\rVert=1 ∥ v ∥ = 1 Ω 0 \Omega_{0} Ω 0 Φ \Phi Φ Φ ^ \widehat{\Phi} Φ
Δ ( V ) ≤ Lip ( V ) ∥ dist Ω ( Φ ^ , Φ ) ∥ L 2 ( ρ ) , (15) \Delta(V)\leq\operatorname{Lip}(V)\left\lVert\operatorname{dist}_{\Omega}(\widehat{\Phi},\Phi)\right\rVert_{L^{2}(\rho)}, \tag*{(15)} Δ ( V ) ≤ Lip ( V ) dist Ω ( Φ , Φ ) L 2 ( ρ ) , (15) where dist Ω ( Φ ^ , Φ ) \operatorname{dist}_{\Omega}(\widehat{\Phi},\Phi) dist Ω ( Φ , Φ ) y ↦ dist Ω ( Φ ^ ( y ) , Φ ( y ) ) y\mapsto\operatorname{dist}_{\Omega}(\widehat{\Phi}(y),\Phi(y)) y ↦ dist Ω ( Φ ( y ) , Φ ( y )) V V V Ω 0 \Omega_{0} Ω 0 p p p ρ \rho ρ p 3 p^{3} p 3 V V V [11 ] . In that setting, Φ \Phi Φ d x d_x d x T d x \mathbb{T}^{d_x} T d x a > d x / 2 a>d_x/2 a > d x /2 dist Ω \operatorname{dist}_{\Omega} dist Ω L 2 L^{2} L 2 Φ ^ \widehat{\Phi} Φ n o p n_{\mathrm{op}} n op ( u i , Φ ( u i ) ) (u_i,\Phi(u_i)) ( u i , Φ ( u i )) u i ∼ ρ u_i\sim\rho u i ∼ ρ [11 Theorem 3.6] , there are constants C 0 > 0 C_0>0 C 0 > 0 n 0 n_0 n 0 n o p ≥ n 0 n_{\mathrm{op}}\geq n_0 n op ≥ n 0 1 − n o p − 1 1-n_{\mathrm{op}}^{-1} 1 − n op − 1
∥ dist Ω ( Φ ^ , Φ ) ∥ L 2 ( ρ ) 2 ≤ C 0 n o p − 1 / ( 6 + 2 d x / a ) . \left\lVert\operatorname{dist}_{\Omega}(\widehat{\Phi},\Phi)\right\rVert_{L^{2}(\rho)}^{2}\leq C_0 n_{\mathrm{op}}^{-1/(6+2d_x/a)}. dist Ω ( Φ , Φ ) L 2 ( ρ ) 2 ≤ C 0 n op − 1/ ( 6 + 2 d x / a ) . The precise assumptions are stated in Appendix F. Inserting this rate in (15) gives the following bound.
Corollary 4.6 (Learning error for an FNO). In the setting of Appendix F, for n o p ≥ n 0 n_{\mathrm{op}}\geq n_0 n op ≥ n 0 1 − n o p − 1 1-n_{\mathrm{op}}^{-1} 1 − n op − 1
Δ ( V ) ≤ Lip ( V ) C 0 1 / 2 n o p − 1 / ( 12 + 4 d x / a ) \Delta(V)\leq\operatorname{Lip}(V)C_0^{1/2}n_{\mathrm{op}}^{-1/(12+4d_x/a)} Δ ( V ) ≤ Lip ( V ) C 0 1/2 n op − 1/ ( 12 + 4 d x / a ) simultaneously for all m m m V ⊂ F ~ V\subset\widetilde{\mathcal{F}} V ⊂ F Theorem 4.2 satisfies
Δ Θ ≤ Lip Θ C 0 1 / 2 n o p − 1 / ( 12 + 4 d x / a ) , Lip Θ : = sup θ ∈ Θ Lip ( V θ ) . \Delta_{\Theta}\leq\operatorname{Lip}_{\Theta}C_0^{1/2}n_{\mathrm{op}}^{-1/(12+4d_x/a)},\qquad\operatorname{Lip}_{\Theta}:=\sup_{\theta\in\Theta}\operatorname{Lip}(V_{\theta}). Δ Θ ≤ Lip Θ C 0 1/2 n op − 1/ ( 12 + 4 d x / a ) , Lip Θ := θ ∈ Θ sup Lip ( V θ ) . With this bound, the learning error in the one-step, spectral, and forecast estimates of this section is expressed through the number n o p n_{\mathrm{op}} n op
Numerical experiments
This section applies the BSO method to two spectral benchmarks, the Ornstein–Uhlenbeck and triple-well diffusions, and to four forecasting problems involving Lorenz–63 [40 ] , Lorenz–96 [41 ] , global sea-surface temperature (SST), and resting-state electroencephalography (EEG). A further experiment treats the case in which the number of observed transition pairs is smaller than the dimension of the Galerkin matrix (subsection 5.3). Throughout this section, BSO denotes the two-stage method of § 3 with an FNO [39 ] as the operator estimator. In the spectral examples, T ^ \widehat{T} T T τ T_{\tau} T τ Remark 3.2 ), whereas in the forecasting examples it is the composition estimator of Remark 3.1. The experiments are not designed to test the bounds of § 4 quantitatively, and several lie outside their hypotheses. Implementation details are collected in Appendix G.
Spectral approximation
Ornstein–Uhlenbeck process
We consider
d X t = − X t d t + d W t , L f = − x f ′ + 1 2 f ′ ′ . \mathrm{d}X_t = -X_t\,\mathrm{d}t + \mathrm{d}W_t,\qquad\mathcal{L}f = -xf' + \frac{1}{2}f''. d X t = − X t d t + d W t , L f = − x f ′ + 2 1 f ′′ . This benchmark has an analytic spectrum, against which the computed rates and modes can be checked. At lag τ = 0.15 \tau= 0.15 τ = 0.15 ( f , T τ f ) (f,T_\tau f) ( f , T τ f ) f f f Figure 1 compares the recovered rates with the exact generator spectrum λ n = − n \lambda_n = -n λ n = − n n = 0 , 1 , 2 , … n = 0,1,2,\ldots n = 0 , 1 , 2 , … [53 ] .
Figure 1. O U OU O U τ = 0.15 \tau= 0.15 τ = 0.15 ∣ x ∣ ≤ 3 |x| \le3 ∣ x ∣ ≤ 3
Triple-well diffusion
The second benchmark is the triple-well reflected diffusion of [71 ] ,
d X t = − ∇ U ( X t ) d t + 1.09 d W t , \mathrm{d}X_t = -\nabla U(X_t)\,\mathrm{d}t + 1.09\,\mathrm{d}W_t, d X t = − ∇ U ( X t ) d t + 1.09 d W t , with the three-hole potential U U U [45 ] specified in Appendix G. The operator is learned at the lag τ = 0.30 \tau= 0.30 τ = 0.30 Figure 2 are those at the lag 3 τ = 0.9 3\tau= 0.9 3 τ = 0.9 Figure 2 shows the multipliers and rates together with the left–right and upper–lower metastable partitions. The Galerkin space is larger than the invariant subspace spanned by these modes, so the remaining computed eigenvalues are not expected to be accurate (subsection 4.3).
Figure 2. Triple-well spectral approximation at the lag 3 τ = 0.9 3\tau= 0.9 3 τ = 0.9 λ = ( 3 τ ) − 1 log μ \lambda= (3\tau)^{-1}\log\mu λ = ( 3 τ ) − 1 log μ μ \mu μ
Forecasting
Chaotic systems
We compare autonomous BSO rollouts with autoregressive rollouts of the same fixed FNO. Lorenz–63 [40 ] and Lorenz–96 [41 ] are given by
x ˙ = 10 ( y − x ) , y ˙ = x ( 28 − z ) − y , z ˙ = x y − 8 3 z . \dot{x} = 10(y-x), \qquad\dot{y} = x(28-z)-y, \qquad\dot{z} = xy-\frac{8}{3}z. x ˙ = 10 ( y − x ) , y ˙ = x ( 28 − z ) − y , z ˙ = x y − 3 8 z . and
x ˙ i = ( x i + 1 − x i − 2 ) x i − 1 − x i + 8 , i = 1 , … , 20. \dot{x}_i = (x_{i+1}-x_{i-2})x_{i-1}-x_i+8, \qquad i=1,\ldots,20. x ˙ i = ( x i + 1 − x i − 2 ) x i − 1 − x i + 8 , i = 1 , … , 20. with periodic indices. In both systems the FNO acts on a smooth curve sampled at 20 sensor sites that encodes the state. Let y o , j , k y_{o,j,k} y o , j , k y ^ o , j , k \widehat{y}_{o,j,k} y o , j , k o ∈ { 1 , … , 64 } o \in\{1,\ldots,64\} o ∈ { 1 , … , 64 } j ∈ { 1 , … , 20 } j \in\{1,\ldots,20\} j ∈ { 1 , … , 20 } k k k
E k L o r : = ( ∑ o , j ∣ y ^ o , j , k − y o , j , k ∣ 2 ) 1 / 2 ( ∑ o , j ∣ y o , j , k ∣ 2 ) 1 / 2 . E_k^{\mathrm{Lor}} := \frac{\left(\sum_{o,j}\lvert\widehat{y}_{o,j,k}-y_{o,j,k}\rvert^2\right)^{1/2}}{\left(\sum_{o,j}\lvert y_{o,j,k}\rvert^2\right)^{1/2}}. E k Lor := ( ∑ o , j ∣ y o , j , k ∣ 2 ) 1/2 ( ∑ o , j ∣ y o , j , k − y o , j , k ∣ 2 ) 1/2 . For Lorenz-63 (Figure 3 ), the two methods have similar errors at short leads: the autoregressive FNO has lower error for leads up to about t = 1 t = 1 t = 1
Figure 3. Pooled relative L 2 L^2 L 2
Global sea-surface temperature
The SST experiment uses global fields on a 32 × 64 32 \times64 32 × 64 [26 ] as distributed in the WeatherBench 2 archive [55 ] , while the remaining ocean fields are taken from the NCEP Global Ocean Data Assimilation System (GODAS) [4 ] . The 11 input fields are listed in Table 2 of Appendix G. The model is selected using 1993–2016 for fitting and 2017–2019 for chronological validation, then refitted on 1993–2019 and evaluated on 2020–2022. All 11 fields initialize the learned models, but only SST is verified.
Table 2. Input fields of the SST experiment on the 32 × 64 grid. WB2: WeatherBench 2 archive of ERA5. Units are those of the fields as used, before standardization. Only SST is verified.
In this experiment the reduced matrix is not the plain Galerkin compression of § 3 . Instead, it is fitted by ridge regression to a blended one-day target, which for a weight β = 0.4 \beta= 0.4 β = 0.4 1 − β 1-\beta 1 − β β \beta β β = 0 \beta= 0 β = 0 β = 1 \beta= 1 β = 1 Remark 3.4 describes the bias and the fluctuation of the cross matrix of such a blended target, for independent data and without regularization. The fitted matrix is then subjected to stability adjustments (Remark D.5); the settings are given in Appendix G.
Forecast skill (Figure 4 , top) is evaluated through day 90 over 96 held-out origins using the area-weighted root-mean-square error (RMSE), the mean absolute error (MAE), and the centered anomaly correlation coefficient (ACC). The baselines are persistence, calendar-day climatology, and damped persistence,
Figure 4. Global SST evaluation. Top: forecast skill of BSO (fixed dictionary), FNO, SFNO, and the three baselines over 96 held-out 2020–2022 origins, showing area-weighted RMSE and MAE through day 30 and RMSE and centered anomaly ACC through day 90. The SFNO curve ends at the first lead time for which scores are not available for all 96 origins. Bottom: area-weighted SST RMSE in the 540-day numerical-stability test over 48 origins, shown over the full and a zoomed logarithmic range.
S S T D P ( t + ℓ ) : = S S T c l i m ( t + ℓ ) + 0.985 ℓ ( S S T ( t ) − S S T c l i m ( t ) ) , \mathrm{SST}_{\mathrm{DP}}(t+\ell) := \mathrm{SST}_{\mathrm{clim}}(t+\ell) + 0.985^{\ell}\left(\mathrm{SST}(t)-\mathrm{SST}_{\mathrm{clim}}(t)\right), SST DP ( t + ℓ ) := SST clim ( t + ℓ ) + 0.98 5 ℓ ( SST ( t ) − SST clim ( t ) ) , where ℓ \ell ℓ S S T c l i m \mathrm{SST}_{\mathrm{clim}} SST clim Figure 4 shows.
A separate 540-day test (Figure 4 , bottom) assesses numerical stability rather than forecast skill. Over 48 origins, the BSO rollout remains finite through day 540 with SST RMSE 0.66 K, the FNO reaches 816 K, and the spherical Fourier neural operator (SFNO) [7 ] first becomes nonfinite at day 171. No clipping or replacement is used. These values do not rank seasonal-to-annual forecast skill; they show that the BSO rollout remains numerically stable over the test period, whereas the autoregressive rollouts of the neural operators do not.
Resting-state EEG rollout
The final experiment uses subject S029, run R02, from the PhysioNet EEG Motor Movement/Imagery Dataset [24 , 54 , 57 , 58 ] . Evaluation consists of one autonomous 40-step rollout from 15 to 19 seconds, with no observations supplied after launch. With y ℓ y_\ell y ℓ y ^ ℓ \hat{y}_\ell y ^ ℓ ℓ \ell ℓ k k k
E k E E G : = ( ∑ ℓ = 1 k ∥ y ^ ℓ − y ℓ ∥ 2 2 ) 1 / 2 ( ∑ ℓ = 1 k ∥ y ℓ ∥ 2 2 ) 1 / 2 . E_k^{\mathrm{EEG}} := \frac{\left(\sum_{\ell=1}^{k}\lVert\hat{y}_\ell-y_\ell\rVert_2^2\right)^{1/2}}{\left(\sum_{\ell=1}^{k}\lVert y_\ell\rVert_2^2\right)^{1/2}}. E k EEG := ( ∑ ℓ = 1 k ∥ y ℓ ∥ 2 2 ) 1/2 ( ∑ ℓ = 1 k ∥ y ^ ℓ − y ℓ ∥ 2 2 ) 1/2 . The comparators are direct autoregressive FNO, EDMD with dictionary learning (EDMD-DL) [38 ] , Gaussian-kernel EDMD [68 ] , and an affine vector-autoregressive model in principal component analysis (PCA) coordinates (PCA–VAR). The direct FNO and the FNO used inside BSO have different fixed hyperparameters. This comparison therefore does not isolate the effect of the Galerkin step. The reduced matrix of BSO is ridge-regularized (Appendix G); the effect of such a modification on the bounds is described in Remark D.5.
BSO has the lowest cumulative error through step 40, E 40 E E G E_{40}^{\mathrm{EEG}} E 40 EEG Table 1 and Figure 5 ), which is 14.2% below that of the direct FNO. This is a descriptive result for one subject, one run, and one forecast window; resting EEG is only approximately autonomous and no uncertainty interval is available.
Table 1. EEG experiment: relative L 2 L^2 L 2 E 40 E E G E^{\mathrm{EEG}}_{40} E 40 EEG
Figure 5. Cumulative relative L 2 L^2 L 2
Small-data comparison between BSO and rank-deficient EDMD
This experiment illustrates the finite-sample distinction described in subsection 3.4. The system is Lorenz–96 with 40 sites, and the dictionary is linear in the state, so that m = 40 m=40 m = 40
The BSO Galerkin matrix is constructed as follows. A neural operator is fitted to the η o p \eta_{\mathrm{op}} η op ( x k , w k ) (x_k,w_k) ( x k , w k ) k = 1 , … , n k=1,\ldots,n k = 1 , … , n x k x_k x k w k w_k w k
integration nodes for the Galerkin inner product. With the data matrices Ψ \Psi Ψ Ψ ^ T ^ \widehat{\Psi}_{\widehat{T}} Ψ T (1) evaluated at the nodes x k x_k x k w : = ( w 1 , … , w n ) w := (w_1,\ldots,w_n) w := ( w 1 , … , w n )
K ^ : = ( Ψ T diag ( w ) Ψ ) − 1 Ψ T diag ( w ) Ψ ^ T ^ , \widehat{K} := \left(\Psi^{\mathsf{T}} \operatorname{diag}(w)\Psi\right)^{-1}\Psi^{\mathsf{T}} \operatorname{diag}(w)\widehat{\Psi}_{\widehat{T}}, K := ( Ψ T diag ( w ) Ψ ) − 1 Ψ T diag ( w ) Ψ T , which reduces to (1) for equal weights. It is the matrix, in the basis ψ 1 , … , ψ m \psi_1,\ldots,\psi_m ψ 1 , … , ψ m X k + X_k^+ X k + n o p n_{\mathrm{op}} n op Ψ T diag ( w ) Ψ \Psi^{\mathsf{T}}\operatorname{diag}(w)\Psi Ψ T diag ( w ) Ψ n o p n_{\mathrm{op}} n op
In Figure 6 , for n o p < m n_{\mathrm{op}} < m n op < m L 2 L^2 L 2 m m m n o p n_{\mathrm{op}} n op m m m L 2 L^2 L 2
Figure 6. One-step prediction error on held-out transition pairs (relative L 2 L^2 L 2 n o p n_{\mathrm{op}} n op m = 40 m=40 m = 40 n o p n_{\mathrm{op}} n op m = 40 m=40 m = 40 10 − 4 10^{-4} 1 0 − 4 min { n o p , m } \min\{n_{\mathrm{op}},m\} min { n op , m } n n n n o p n_{\mathrm{op}} n op
Conclusion
We developed a two-stage Galerkin framework, the BSO method, in which a fixed-lag solution operator of the Kolmogorov backward equation is first estimated from data and then projected onto an observable space that is fixed in advance or selected variationally. Since the operator is approximated before the coordinates are fixed, the same estimator can be reused across candidate spaces. The invariance of a candidate space is assessed through conditional means rather than single transitions, and the Galerkin matrix is defined even when fewer transition pairs than basis functions are observed.
The error analysis traces the learning error of the estimator, the sampling error of the Galerkin inner products, and the invariance and reconstruction properties of the observable space to the computed eigenvalues, invariant subspaces, and forecasts. The sampling error enters only through a factor that tends to one. Near an isolated spectral cluster, the invariance defect is locally equivalent to the distance from the invariant subspace. The forecast bound is uniform in the horizon when the measure is invariant and the reduced model preserves constants and is stable on mean-zero observables. For deterministic dynamics, a learning rate for FNOs gives one explicit bound for the first stage.
The numerical examples apply this framework to spectral and forecasting problems. We compare the Ornstein–Uhlenbeck and triple-well spectra to analytic and finite-difference references. The Lorenz, SST, and EEG rollouts are compared with autoregressive neural operators and standard baselines, showing long-rollout stability. A small-data experiment demonstrates that BSO can result in reduced prediction error than EDMD when fewer transition pairs than basis functions are observed.
Three limitations indicate directions for further work. First, uniform control of a general nonlinear operator estimator over the candidate families is method dependent and is stated here as a hypothesis. Second, the FNO rate uses exact independent input–output pairs of a deterministic map and an exact empirical risk minimizer, so that dependent trajectory data, noisy pairs, the optimization error of training, and stochastic dynamics are not covered by an explicit rate. Third, partial observations may identify only a compressed solution operator. A quantitative analysis of the regime with fewer transition pairs than basis functions is also left to future work.
Declarations. Generative AI tools were used to assist with the code used in the numerical experiments, check computations, and edit the language and organization of the manuscript text. Authors assume responsibility for all content.
Acknowledgments. We thank Zongyi Li, Zongren Zou, and Houman Owhadi for helpful discussions.
Function-space conventions and the learning error
Point evaluations are well defined on the space F \mathcal{F} F L 2 ( ρ ) L^2(\rho) L 2 ( ρ ) T T T F \mathcal{F} F ρ \rho ρ T ^ \widehat{T} T H \mathcal{H} H f ∘ Φ ^ f \circ\widehat{\Phi} f ∘ Φ F \mathcal{F} F ρ \rho ρ V ⊂ F V \subset\mathcal{F} V ⊂ F ∥ ⋅ ∥ \lVert\cdot\rVert ∥ ⋅ ∥ T ^ ∣ V : V → F \widehat{T}|_V: V \to\mathcal{F} T ∣ V : V → F B ( V , H ) B(V,\mathcal{H}) B ( V , H ) ρ \rho ρ Δ ( V ) \Delta(V) Δ ( V ) (2) is understood in this sense. For a nonlinear estimator (Remark 3.2 ), the first part of the following lemma applies to the basis that defines T ^ ∣ V \widehat{T}|_V T ∣ V
Lemma A.1. Let Ψ = ( ψ 1 , … , ψ m ) \Psi=(\psi_1,\ldots,\psi_m) Ψ = ( ψ 1 , … , ψ m ) V V V G Ψ : = [ ⟨ ψ i , ψ j ⟩ ] i , j = 1 m G_\Psi:= [\langle\psi_i,\psi_j\rangle]_{i,j=1}^m G Ψ := [⟨ ψ i , ψ j ⟩ ] i , j = 1 m
Δ ( V ) ≤ λ min ( G Ψ ) − 1 / 2 ( ∑ j = 1 m ∥ T ^ ψ j − T ψ j ∥ 2 ) 1 / 2 . (16) \Delta(V) \le\lambda_{\min}(G_\Psi)^{-1/2}\left(\sum_{j=1}^{m}\lVert\widehat{T}\psi_j-T\psi_j\rVert^2\right)^{1/2}. \tag*{(16)} Δ ( V ) ≤ λ m i n ( G Ψ ) − 1/2 ( j = 1 ∑ m ∥ T ψ j − T ψ j ∥ 2 ) 1/2 . (16) Moreover, if ⟨ ⋅ , ⋅ ⟩ n \langle\cdot,\cdot\rangle_n ⟨ ⋅ , ⋅ ⟩ n V + T ^ V V+\widehat{T}V V + T V K ^ \widehat{K} K L ^ \widehat{L} L
sup v ∈ V ∖ { 0 } ∥ ( I − P ^ V ) T ^ v ∥ n 2 ∥ v ∥ n 2 = λ max ( L ^ − K ^ ⊤ K ^ ) . (17) \sup_{v\in V\setminus\{0\}}\frac{\lVert(I-\widehat{P}_V)\widehat{T}v\rVert_n^2}{\lVert v\rVert_n^2}
=\lambda_{\max}\left(\widehat{L}-\widehat{K}^{\top}\widehat{K}\right). \tag*{(17)} v ∈ V ∖ { 0 } sup ∥ v ∥ n 2 ∥( I − P V ) T v ∥ n 2 = λ m a x ( L − K ⊤ K ) . (17) Proof. Let S : C m → V S:\mathbb{C}^m\to V S : C m → V S c : = ∑ j c j ψ j Sc:=\sum_j c_j\psi_j S c := ∑ j c j ψ j S ∗ S = G Ψ S^*S=G_\Psi S ∗ S = G Ψ S G Ψ − 1 / 2 SG_\Psi^{-1/2} S G Ψ − 1/2 V V V E : = T ^ ∣ V − T ∣ V E:=\widehat{T}|_V-T|_V E := T ∣ V − T ∣ V ∥ ⋅ ∥ H S \lVert\cdot\rVert_{\mathrm{HS}} ∥ ⋅ ∥ HS Δ ( V ) = ∥ E ∥ ≤ ∥ E S G Ψ − 1 / 2 ∥ H S ≤ ∥ E S ∥ H S ∥ G Ψ − 1 / 2 ∥ 2 \Delta(V)=\lVert E\rVert\le\lVert ESG_\Psi^{-1/2}\rVert_{\mathrm{HS}}\le\lVert ES\rVert_{\mathrm{HS}}\lVert G_\Psi^{-1/2}\rVert_2 Δ ( V ) = ∥ E ∥ ≤ ∥ E S G Ψ − 1/2 ∥ HS ≤ ∥ E S ∥ HS ∥ G Ψ − 1/2 ∥ 2 ∥ E S ∥ H S 2 = ∑ j ∥ T ^ ψ j − T ψ j ∥ 2 \lVert ES\rVert_{\mathrm{HS}}^2=\sum_j\lVert\widehat{T}\psi_j-T\psi_j\rVert^2 ∥ E S ∥ HS 2 = ∑ j ∥ T ψ j − T ψ j ∥ 2 S ^ \widehat{S} S Ψ ^ \widehat{\Psi} Ψ ⟨ ⋅ , ⋅ ⟩ n \langle\cdot,\cdot\rangle_n ⟨ ⋅ , ⋅ ⟩ n S ^ \widehat{S} S C m \mathbb{C}^m C m ( V , ⟨ ⋅ , ⋅ ⟩ n ) (V,\langle\cdot,\cdot\rangle_n) ( V , ⟨ ⋅ , ⋅ ⟩ n ) P ^ V = S ^ S ^ ∗ \widehat{P}_V=\widehat{S}\widehat{S}^* P V = S S ∗ K ^ = S ^ ∗ T ^ S ^ \widehat{K}=\widehat{S}^*\widehat{T}\widehat{S} K = S ∗ T S L ^ = ( T ^ S ^ ) ∗ T ^ S ^ \widehat{L}=(\widehat{T}\widehat{S})^*\widehat{T}\widehat{S} L = ( T S ) ∗ T S [18 ] . Hence the left-hand side of (A.2) equals λ max ( ( T ^ S ^ ) ∗ ( I − P ^ V ) T ^ S ^ ) = λ max ( L ^ − K ^ ⊤ K ^ ) \lambda_{\max}((\widehat{T}\widehat{S})^*(I-\widehat{P}_V)\widehat{T}\widehat{S})=\lambda_{\max}(\widehat{L}-\widehat{K}^{\top}\widehat{K}) λ m a x (( T S ) ∗ ( I − P V ) T S ) = λ m a x ( L − K ⊤ K )
Sampling error and data reuse
Lemma B.1. Let V ⊂ W V\subset W V ⊂ W F \mathcal{F} F V ≠ { 0 } V\ne\{0\} V = { 0 } ∥ ⋅ ∥ \lVert\cdot\rVert ∥ ⋅ ∥ W W W η : = η ( W ) < 1 \eta:=\eta(W)<1 η := η ( W ) < 1 u ∈ W u\in W u ∈ W
( 1 − η ) ∥ u ∥ 2 ≤ ∥ u ∥ n 2 ≤ ( 1 + η ) ∥ u ∥ 2 , (18) (1-\eta)\lVert u\rVert^2\le\lVert u\rVert_n^2\le(1+\eta)\lVert u\rVert^2, \tag*{(18)} ( 1 − η ) ∥ u ∥ 2 ≤ ∥ u ∥ n 2 ≤ ( 1 + η ) ∥ u ∥ 2 , (18) ∥ ( P ^ V − P V ) u ∥ ≤ η 1 − η ∥ ( I − P V ) u ∥ , (19) \lVert(\widehat{P}_V-P_V)u\rVert\le\frac{\eta}{1-\eta}\lVert(I-P_V)u\rVert, \tag*{(19)} ∥( P V − P V ) u ∥ ≤ 1 − η η ∥( I − P V ) u ∥ , (19) ( 1 − η ) ∥ ( I − P V ) u ∥ 2 ≤ ∥ ( I − P ^ V ) u ∥ n 2 ≤ ( 1 + η ) ∥ ( I − P V ) u ∥ 2 . (20) (1-\eta)\lVert(I-P_V)u\rVert^2\le\lVert(I-\widehat{P}_V)u\rVert_n^2\le(1+\eta)\lVert(I-P_V)u\rVert^2. \tag*{(20)} ( 1 − η ) ∥( I − P V ) u ∥ 2 ≤ ∥( I − P V ) u ∥ n 2 ≤ ( 1 + η ) ∥( I − P V ) u ∥ 2 . (20) Proof. Let χ W \chi^W χ W ρ \rho ρ W W W u = ∑ i a i χ i W u=\sum_i a_i\chi_i^W u = ∑ i a i χ i W ∥ u ∥ n 2 − ∥ u ∥ 2 = a ∗ ( G ^ W − I ) a \lVert u\rVert_n^2-\lVert u\rVert^2=a^*(\widehat{G}_W-I)a ∥ u ∥ n 2 − ∥ u ∥ 2 = a ∗ ( G W − I ) a ∣ ⟨ u 1 , u 2 ⟩ n − ⟨ u 1 , u 2 ⟩ ∣ ≤ η ∥ u 1 ∥ ∥ u 2 ∥ \lvert\langle u_1,u_2\rangle_n-\langle u_1,u_2\rangle\rvert\le\eta\lVert u_1\rVert\lVert u_2\rVert ∣⟨ u 1 , u 2 ⟩ n − ⟨ u 1 , u 2 ⟩∣ ≤ η ∥ u 1 ∥ ∥ u 2 ∥ u 1 , u 2 ∈ W u_1,u_2\in W u 1 , u 2 ∈ W ⟨ ⋅ , ⋅ ⟩ n \langle\cdot,\cdot\rangle_n ⟨ ⋅ , ⋅ ⟩ n W W W P ^ V \widehat{P}_V P V v : = P V u − P ^ V u ∈ V v:=P_Vu-\widehat{P}_Vu\in V v := P V u − P V u ∈ V v v v ⟨ ⋅ , ⋅ ⟩ n \langle\cdot,\cdot\rangle_n ⟨ ⋅ , ⋅ ⟩ n u − P ^ V u u-\widehat{P}_Vu u − P V u u − P V u u-P_Vu u − P V u W W W
∥ v ∥ n 2 = − ⟨ v , u − P V u ⟩ n = − ( ⟨ v , u − P V u ⟩ n − ⟨ v , u − P V u ⟩ ) ≤ η ∥ v ∥ ∥ u − P V u ∥ , \lVert v\rVert_n^2=-\langle v,u-P_Vu\rangle_n=-\bigl(\langle v,u-P_Vu\rangle_n-\langle v,u-P_Vu\rangle\bigr)\le\eta\lVert v\rVert\lVert u-P_Vu\rVert, ∥ v ∥ n 2 = − ⟨ v , u − P V u ⟩ n = − ( ⟨ v , u − P V u ⟩ n − ⟨ v , u − P V u ⟩ ) ≤ η ∥ v ∥ ∥ u − P V u ∥ , @page 23/38 starts_mid=1 ends_mid=1 inner products in L 2 ( ρ ) L^2(\rho) L 2 ( ρ ) T T T F \mathcal{F} F ρ \rho ρ T ^ \widehat{T} T H \mathcal{H} H f ∘ Φ ^ f \circ\widehat{\Phi} f ∘ Φ F \mathcal{F} F ρ \rho ρ V ⊂ F V \subset\mathcal{F} V ⊂ F ∥ ⋅ ∥ \lVert\cdot\rVert ∥ ⋅ ∥ T ^ ∣ V : V → F \widehat{T}|_V: V \to\mathcal{F} T ∣ V : V → F B ( V , H ) B(V,\mathcal{H}) B ( V , H ) ρ \rho ρ Δ ( V ) \Delta(V) Δ ( V ) (2) is understood in this sense. For a nonlinear estimator (Remark 3.2 ), the first part of the following lemma applies to the basis that defines T ^ ∣ V \widehat{T}|_V T ∣ V
Lemma A.1. Let Ψ = ( ψ 1 , … , ψ m ) \Psi=(\psi_1,\ldots,\psi_m) Ψ = ( ψ 1 , … , ψ m ) V V V G Ψ : = [ ⟨ ψ i , ψ j ⟩ ] i , j = 1 m G_\Psi:= [\langle\psi_i,\psi_j\rangle]_{i,j=1}^m G Ψ := [⟨ ψ i , ψ j ⟩ ] i , j = 1 m
Δ ( V ) ≤ λ min ( G Ψ ) − 1 / 2 ( ∑ j = 1 m ∥ T ^ ψ j − T ψ j ∥ 2 ) 1 / 2 . (21) \Delta(V) \le\lambda_{\min}(G_\Psi)^{-1/2}\left(\sum_{j=1}^{m}\lVert\widehat{T}\psi_j-T\psi_j\rVert^2\right)^{1/2}. \tag*{(21)} Δ ( V ) ≤ λ m i n ( G Ψ ) − 1/2 ( j = 1 ∑ m ∥ T ψ j − T ψ j ∥ 2 ) 1/2 . (21) Moreover, if ⟨ ⋅ , ⋅ ⟩ n \langle\cdot,\cdot\rangle_n ⟨ ⋅ , ⋅ ⟩ n V + T ^ V V+\widehat{T}V V + T V K ^ \widehat{K} K L ^ \widehat{L} L
sup v ∈ V ∖ { 0 } ∥ ( I − P ^ V ) T ^ v ∥ n 2 ∥ v ∥ n 2 = λ max ( L ^ − K ^ ⊤ K ^ ) . (22) \sup_{v\in V\setminus\{0\}}\frac{\lVert(I-\widehat{P}_V)\widehat{T}v\rVert_n^2}{\lVert v\rVert_n^2}
=\lambda_{\max}\left(\widehat{L}-\widehat{K}^{\top}\widehat{K}\right). \tag*{(22)} v ∈ V ∖ { 0 } sup ∥ v ∥ n 2 ∥( I − P V ) T v ∥ n 2 = λ m a x ( L − K ⊤ K ) . (22) Proof. Let S : C m → V S:\mathbb{C}^m\to V S : C m → V S c : = ∑ j c j ψ j Sc:=\sum_j c_j\psi_j S c := ∑ j c j ψ j S ∗ S = G Ψ S^*S=G_\Psi S ∗ S = G Ψ S G Ψ − 1 / 2 SG_\Psi^{-1/2} S G Ψ − 1/2 V V V E : = T ^ ∣ V − T ∣ V E:=\widehat{T}|_V-T|_V E := T ∣ V − T ∣ V ∥ ⋅ ∥ H S \lVert\cdot\rVert_{\mathrm{HS}} ∥ ⋅ ∥ HS Δ ( V ) = ∥ E ∥ ≤ ∥ E S G Ψ − 1 / 2 ∥ H S ≤ ∥ E S ∥ H S ∥ G Ψ − 1 / 2 ∥ 2 \Delta(V)=\lVert E\rVert\le\lVert ESG_\Psi^{-1/2}\rVert_{\mathrm{HS}}\le\lVert ES\rVert_{\mathrm{HS}}\lVert G_\Psi^{-1/2}\rVert_2 Δ ( V ) = ∥ E ∥ ≤ ∥ E S G Ψ − 1/2 ∥ HS ≤ ∥ E S ∥ HS ∥ G Ψ − 1/2 ∥ 2 ∥ E S ∥ H S 2 = ∑ j ∥ T ^ ψ j − T ψ j ∥ 2 \lVert ES\rVert_{\mathrm{HS}}^2=\sum_j\lVert\widehat{T}\psi_j-T\psi_j\rVert^2 ∥ E S ∥ HS 2 = ∑ j ∥ T ψ j − T ψ j ∥ 2 S ^ \widehat{S} S Ψ ^ \widehat{\Psi} Ψ ⟨ ⋅ , ⋅ ⟩ n \langle\cdot,\cdot\rangle_n ⟨ ⋅ , ⋅ ⟩ n S ^ \widehat{S} S C m \mathbb{C}^m C m ( V , ⟨ ⋅ , ⋅ ⟩ n ) (V,\langle\cdot,\cdot\rangle_n) ( V , ⟨ ⋅ , ⋅ ⟩ n ) P ^ V = S ^ S ^ ∗ \widehat{P}_V=\widehat{S}\widehat{S}^* P V = S S ∗ K ^ = S ^ ∗ T ^ S ^ \widehat{K}=\widehat{S}^*\widehat{T}\widehat{S} K = S ∗ T S L ^ = ( T ^ S ^ ) ∗ T ^ S ^ \widehat{L}=(\widehat{T}\widehat{S})^*\widehat{T}\widehat{S} L = ( T S ) ∗ T S [18 ] . Hence the left-hand side of (A.2) equals λ max ( ( T ^ S ^ ) ∗ ( I − P ^ V ) T ^ S ^ ) = λ max ( L ^ − K ^ ⊤ K ^ ) \lambda_{\max}((\widehat{T}\widehat{S})^*(I-\widehat{P}_V)\widehat{T}\widehat{S})=\lambda_{\max}(\widehat{L}-\widehat{K}^{\top}\widehat{K}) λ m a x (( T S ) ∗ ( I − P V ) T S ) = λ m a x ( L − K ⊤ K )
Sampling error and data reuse
Lemma B.1. Let V ⊂ W V\subset W V ⊂ W F \mathcal{F} F V ≠ { 0 } V\ne\{0\} V = { 0 } ∥ ⋅ ∥ \lVert\cdot\rVert ∥ ⋅ ∥ W W W η : = η ( W ) < 1 \eta:=\eta(W)<1 η := η ( W ) < 1 u ∈ W u\in W u ∈ W
( 1 − η ) ∥ u ∥ 2 ≤ ∥ u ∥ n 2 ≤ ( 1 + η ) ∥ u ∥ 2 , (23) (1-\eta)\lVert u\rVert^2\le\lVert u\rVert_n^2\le(1+\eta)\lVert u\rVert^2, \tag*{(23)} ( 1 − η ) ∥ u ∥ 2 ≤ ∥ u ∥ n 2 ≤ ( 1 + η ) ∥ u ∥ 2 , (23) ∥ ( P ^ V − P V ) u ∥ ≤ η 1 − η ∥ ( I − P V ) u ∥ , (24) \lVert(\widehat{P}_V-P_V)u\rVert\le\frac{\eta}{1-\eta}\lVert(I-P_V)u\rVert, \tag*{(24)} ∥( P V − P V ) u ∥ ≤ 1 − η η ∥( I − P V ) u ∥ , (24) ( 1 − η ) ∥ ( I − P V ) u ∥ 2 ≤ ∥ ( I − P ^ V ) u ∥ n 2 ≤ ( 1 + η ) ∥ ( I − P V ) u ∥ 2 . (25) (1-\eta)\lVert(I-P_V)u\rVert^2\le\lVert(I-\widehat{P}_V)u\rVert_n^2\le(1+\eta)\lVert(I-P_V)u\rVert^2. \tag*{(25)} ( 1 − η ) ∥( I − P V ) u ∥ 2 ≤ ∥( I − P V ) u ∥ n 2 ≤ ( 1 + η ) ∥( I − P V ) u ∥ 2 . (25) Proof. Let χ W \chi^W χ W ρ \rho ρ W W W u = ∑ i a i χ i W u=\sum_i a_i\chi_i^W u = ∑ i a i χ i W ∥ u ∥ n 2 − ∥ u ∥ 2 = a ∗ ( G ^ W − I ) a \lVert u\rVert_n^2-\lVert u\rVert^2=a^*(\widehat{G}_W-I)a ∥ u ∥ n 2 − ∥ u ∥ 2 = a ∗ ( G W − I ) a ∣ ⟨ u 1 , u 2 ⟩ n − ⟨ u 1 , u 2 ⟩ ∣ ≤ η ∥ u 1 ∥ ∥ u 2 ∥ \lvert\langle u_1,u_2\rangle_n-\langle u_1,u_2\rangle\rvert\le\eta\lVert u_1\rVert\lVert u_2\rVert ∣⟨ u 1 , u 2 ⟩ n − ⟨ u 1 , u 2 ⟩∣ ≤ η ∥ u 1 ∥ ∥ u 2 ∥ u 1 , u 2 ∈ W u_1,u_2\in W u 1 , u 2 ∈ W ⟨ ⋅ , ⋅ ⟩ n \langle\cdot,\cdot\rangle_n ⟨ ⋅ , ⋅ ⟩ n W W W P ^ V \widehat{P}_V P V v : = P V u − P ^ V u ∈ V v:=P_Vu-\widehat{P}_Vu\in V v := P V u − P V u ∈ V v v v ⟨ ⋅ , ⋅ ⟩ n \langle\cdot,\cdot\rangle_n ⟨ ⋅ , ⋅ ⟩ n u − P ^ V u u-\widehat{P}_Vu u − P V u u − P V u u-P_Vu u − P V u W W W
∥ v ∥ n 2 = − ⟨ v , u − P V u ⟩ n = − ( ⟨ v , u − P V u ⟩ n − ⟨ v , u − P V u ⟩ ) ≤ η ∥ v ∥ ∥ u − P V u ∥ , \lVert v\rVert_n^2=-\langle v,u-P_Vu\rangle_n=-\bigl(\langle v,u-P_Vu\rangle_n-\langle v,u-P_Vu\rangle\bigr)\le\eta\lVert v\rVert\lVert u-P_Vu\rVert, ∥ v ∥ n 2 = − ⟨ v , u − P V u ⟩ n = − ( ⟨ v , u − P V u ⟩ n − ⟨ v , u − P V u ⟩ ) ≤ η ∥ v ∥ ∥ u − P V u ∥ , @page 23/38 starts_mid=1 ends_mid=1 inner products in L 2 ( ρ ) L^2(\rho) L 2 ( ρ ) T T T F \mathcal{F} F ρ \rho ρ T ^ \widehat{T} T H \mathcal{H} H f ∘ Φ ^ f \circ\widehat{\Phi} f ∘ Φ F \mathcal{F} F ρ \rho ρ V ⊂ F V \subset\mathcal{F} V ⊂ F ∥ ⋅ ∥ \lVert\cdot\rVert ∥ ⋅ ∥ T ^ ∣ V : V → F \widehat{T}|_V: V \to\mathcal{F} T ∣ V : V → F B ( V , H ) B(V,\mathcal{H}) B ( V , H ) ρ \rho ρ Δ ( V ) \Delta(V) Δ ( V ) (2) is understood in this sense. For a nonlinear estimator (Remark 3.2 ), the first part of the following lemma applies to the basis that defines T ^ ∣ V \widehat{T}|_V T ∣ V
Lemma A.1. Let Ψ = ( ψ 1 , … , ψ m ) \Psi=(\psi_1,\ldots,\psi_m) Ψ = ( ψ 1 , … , ψ m ) V V V G Ψ : = [ ⟨ ψ i , ψ j ⟩ ] i , j = 1 m G_\Psi:= [\langle\psi_i,\psi_j\rangle]_{i,j=1}^m G Ψ := [⟨ ψ i , ψ j ⟩ ] i , j = 1 m
Δ ( V ) ≤ λ min ( G Ψ ) − 1 / 2 ( ∑ j = 1 m ∥ T ^ ψ j − T ψ j ∥ 2 ) 1 / 2 . (26) \Delta(V) \le\lambda_{\min}(G_\Psi)^{-1/2}\left(\sum_{j=1}^{m}\lVert\widehat{T}\psi_j-T\psi_j\rVert^2\right)^{1/2}. \tag*{(26)} Δ ( V ) ≤ λ m i n ( G Ψ ) − 1/2 ( j = 1 ∑ m ∥ T ψ j − T ψ j ∥ 2 ) 1/2 . (26) Moreover, if ⟨ ⋅ , ⋅ ⟩ n \langle\cdot,\cdot\rangle_n ⟨ ⋅ , ⋅ ⟩ n V + T ^ V V+\widehat{T}V V + T V K ^ \widehat{K} K L ^ \widehat{L} L
sup v ∈ V ∖ { 0 } ∥ ( I − P ^ V ) T ^ v ∥ n 2 ∥ v ∥ n 2 = λ max ( L ^ − K ^ ⊤ K ^ ) . (27) \sup_{v\in V\setminus\{0\}}\frac{\lVert(I-\widehat{P}_V)\widehat{T}v\rVert_n^2}{\lVert v\rVert_n^2}
=\lambda_{\max}\left(\widehat{L}-\widehat{K}^{\top}\widehat{K}\right). \tag*{(27)} v ∈ V ∖ { 0 } sup ∥ v ∥ n 2 ∥( I − P V ) T v ∥ n 2 = λ m a x ( L − K ⊤ K ) . (27) Proof. Let S : C m → V S:\mathbb{C}^m\to V S : C m → V S c : = ∑ j c j ψ j Sc:=\sum_j c_j\psi_j S c := ∑ j c j ψ j S ∗ S = G Ψ S^*S=G_\Psi S ∗ S = G Ψ S G Ψ − 1 / 2 SG_\Psi^{-1/2} S G Ψ − 1/2 V V V E : = T ^ ∣ V − T ∣ V E:=\widehat{T}|_V-T|_V E := T ∣ V − T ∣ V ∥ ⋅ ∥ H S \lVert\cdot\rVert_{\mathrm{HS}} ∥ ⋅ ∥ HS Δ ( V ) = ∥ E ∥ ≤ ∥ E S G Ψ − 1 / 2 ∥ H S ≤ ∥ E S ∥ H S ∥ G Ψ − 1 / 2 ∥ 2 \Delta(V)=\lVert E\rVert\le\lVert ESG_\Psi^{-1/2}\rVert_{\mathrm{HS}}\le\lVert ES\rVert_{\mathrm{HS}}\lVert G_\Psi^{-1/2}\rVert_2 Δ ( V ) = ∥ E ∥ ≤ ∥ E S G Ψ − 1/2 ∥ HS ≤ ∥ E S ∥ HS ∥ G Ψ − 1/2 ∥ 2 ∥ E S ∥ H S 2 = ∑ j ∥ T ^ ψ j − T ψ j ∥ 2 \lVert ES\rVert_{\mathrm{HS}}^2=\sum_j\lVert\widehat{T}\psi_j-T\psi_j\rVert^2 ∥ E S ∥ HS 2 = ∑ j ∥ T ψ j − T ψ j ∥ 2 S ^ \widehat{S} S Ψ ^ \widehat{\Psi} Ψ ⟨ ⋅ , ⋅ ⟩ n \langle\cdot,\cdot\rangle_n ⟨ ⋅ , ⋅ ⟩ n S ^ \widehat{S} S C m \mathbb{C}^m C m ( V , ⟨ ⋅ , ⋅ ⟩ n ) (V,\langle\cdot,\cdot\rangle_n) ( V , ⟨ ⋅ , ⋅ ⟩ n ) P ^ V = S ^ S ^ ∗ \widehat{P}_V=\widehat{S}\widehat{S}^* P V = S S ∗ K ^ = S ^ ∗ T ^ S ^ \widehat{K}=\widehat{S}^*\widehat{T}\widehat{S} K = S ∗ T S L ^ = ( T ^ S ^ ) ∗ T ^ S ^ \widehat{L}=(\widehat{T}\widehat{S})^*\widehat{T}\widehat{S} L = ( T S ) ∗ T S [18 ] . Hence the left-hand side of (A.2) equals λ max ( ( T ^ S ^ ) ∗ ( I − P ^ V ) T ^ S ^ ) = λ max ( L ^ − K ^ ⊤ K ^ ) \lambda_{\max}((\widehat{T}\widehat{S})^*(I-\widehat{P}_V)\widehat{T}\widehat{S})=\lambda_{\max}(\widehat{L}-\widehat{K}^{\top}\widehat{K}) λ m a x (( T S ) ∗ ( I − P V ) T S ) = λ m a x ( L − K ⊤ K )
Sampling error and data reuse
Lemma B.1. Let V ⊂ W V\subset W V ⊂ W F \mathcal{F} F V ≠ { 0 } V\ne\{0\} V = { 0 } ∥ ⋅ ∥ \lVert\cdot\rVert ∥ ⋅ ∥ W W W η : = η ( W ) < 1 \eta:=\eta(W)<1 η := η ( W ) < 1 u ∈ W u\in W u ∈ W
( 1 − η ) ∥ u ∥ 2 ≤ ∥ u ∥ n 2 ≤ ( 1 + η ) ∥ u ∥ 2 , (28) (1-\eta)\lVert u\rVert^2\le\lVert u\rVert_n^2\le(1+\eta)\lVert u\rVert^2, \tag*{(28)} ( 1 − η ) ∥ u ∥ 2 ≤ ∥ u ∥ n 2 ≤ ( 1 + η ) ∥ u ∥ 2 , (28) ∥ ( P ^ V − P V ) u ∥ ≤ η 1 − η ∥ ( I − P V ) u ∥ , (29) \lVert(\widehat{P}_V-P_V)u\rVert\le\frac{\eta}{1-\eta}\lVert(I-P_V)u\rVert, \tag*{(29)} ∥( P V − P V ) u ∥ ≤ 1 − η η ∥( I − P V ) u ∥ , (29) ( 1 − η ) ∥ ( I − P V ) u ∥ 2 ≤ ∥ ( I − P ^ V ) u ∥ n 2 ≤ ( 1 + η ) ∥ ( I − P V ) u ∥ 2 . (30) (1-\eta)\lVert(I-P_V)u\rVert^2\le\lVert(I-\widehat{P}_V)u\rVert_n^2\le(1+\eta)\lVert(I-P_V)u\rVert^2. \tag*{(30)} ( 1 − η ) ∥( I − P V ) u ∥ 2 ≤ ∥( I − P V ) u ∥ n 2 ≤ ( 1 + η ) ∥( I − P V ) u ∥ 2 . (30) Proof. Let χ W \chi^W χ W ρ \rho ρ W W W u = ∑ i a i χ i W u=\sum_i a_i\chi_i^W u = ∑ i a i χ i W ∥ u ∥ n 2 − ∥ u ∥ 2 = a ∗ ( G ^ W − I ) a \lVert u\rVert_n^2-\lVert u\rVert^2=a^*(\widehat{G}_W-I)a ∥ u ∥ n 2 − ∥ u ∥ 2 = a ∗ ( G W − I ) a ∣ ⟨ u 1 , u 2 ⟩ n − ⟨ u 1 , u 2 ⟩ ∣ ≤ η ∥ u 1 ∥ ∥ u 2 ∥ \lvert\langle u_1,u_2\rangle_n-\langle u_1,u_2\rangle\rvert\le\eta\lVert u_1\rVert\lVert u_2\rVert ∣⟨ u 1 , u 2 ⟩ n − ⟨ u 1 , u 2 ⟩∣ ≤ η ∥ u 1 ∥ ∥ u 2 ∥ u 1 , u 2 ∈ W u_1,u_2\in W u 1 , u 2 ∈ W ⟨ ⋅ , ⋅ ⟩ n \langle\cdot,\cdot\rangle_n ⟨ ⋅ , ⋅ ⟩ n W W W P ^ V \widehat{P}_V P V v : = P V u − P ^ V u ∈ V v:=P_Vu-\widehat{P}_Vu\in V v := P V u − P V u ∈ V v v v ⟨ ⋅ , ⋅ ⟩ n \langle\cdot,\cdot\rangle_n ⟨ ⋅ , ⋅ ⟩ n u − P ^ V u u-\widehat{P}_Vu u − P V u u − P V u u-P_Vu u − P V u W W W
∥ v ∥ n 2 = − ⟨ v , u − P V u ⟩ n = − ( ⟨ v , u − P V u ⟩ n − ⟨ v , u − P V u ⟩ ) ≤ η ∥ v ∥ ∥ u − P V u ∥ , (31) \lVert v\rVert_n^2=-\langle v,u-P_Vu\rangle_n=-\bigl(\langle v,u-P_Vu\rangle_n-\langle v,u-P_Vu\rangle\bigr)\le\eta\lVert v\rVert\lVert u-P_Vu\rVert, \tag*{(31)} ∥ v ∥ n 2 = − ⟨ v , u − P V u ⟩ n = − ( ⟨ v , u − P V u ⟩ n − ⟨ v , u − P V u ⟩ ) ≤ η ∥ v ∥ ∥ u − P V u ∥ , (31) and (B.1) gives ( 1 − η ) ∥ v ∥ 2 ≤ ∥ v ∥ n 2 (1-\eta)\|v\|^2 \le\|v\|_n^2 ( 1 − η ) ∥ v ∥ 2 ≤ ∥ v ∥ n 2 ∥ ( I − P ^ V ) u ∥ n 2 = min w ∈ V ∥ u − w ∥ n 2 \|(I-\widehat{P}_V)u\|_n^2=\min_{w\in V}\|u-w\|_n^2 ∥ ( I − P V ) u ∥ n 2 = min w ∈ V ∥ u − w ∥ n 2 ∥ ( I − P V ) u ∥ 2 = min w ∈ V ∥ u − w ∥ 2 \|(I-P_V)u\|^2=\min_{w\in V}\|u-w\|^2 ∥ ( I − P V ) u ∥ 2 = min w ∈ V ∥ u − w ∥ 2 u − w u-w u − w
Lemma B.2. Let V = V θ V=V_\theta V = V θ η : = η V < 1 \eta:=\eta_V<1 η := η V < 1
J ( V ) 1 / 2 ≤ κ ( η ) J ^ ( θ ) 1 / 2 + Δ ( V ) , J ^ ( θ ) 1 / 2 ≤ κ ( η ) ( J ( V ) 1 / 2 + Δ ( V ) ) . (32) \mathcal{J}(V)^{1/2}\le\kappa(\eta)\widehat{\mathcal{J}}(\theta)^{1/2}+\Delta(V),\qquad\widehat{\mathcal{J}}(\theta)^{1/2}\le\kappa(\eta)\left(\mathcal{J}(V)^{1/2}+\Delta(V)\right). \tag*{(32)} J ( V ) 1/2 ≤ κ ( η ) J ( θ ) 1/2 + Δ ( V ) , J ( θ ) 1/2 ≤ κ ( η ) ( J ( V ) 1/2 + Δ ( V ) ) . (32) Proof. Let I T ^ ( V ) : = ∥ ( I − P V ) T ^ ∣ V ∥ B ( V , H ) I_{\widehat{T}}(V):=\|(I-P_V)\widehat{T}|_V\|_{B(V,\mathcal{H})} I T ( V ) := ∥ ( I − P V ) T ∣ V ∥ B ( V , H ) J T ^ ( V ) : = I T ^ ( V ) 2 + α R ( V ) 2 \mathcal{J}_{\widehat{T}}(V):=I_{\widehat{T}}(V)^2+\alpha R(V)^2 J T ( V ) := I T ( V ) 2 + α R ( V ) 2 η < 1 \eta<1 η < 1 ⟨ ⋅ , ⋅ ⟩ n \langle\cdot,\cdot\rangle_n ⟨ ⋅ , ⋅ ⟩ n W V ⊃ V + T ^ V W_V\supset V+\widehat{T}V W V ⊃ V + T V J ^ ( θ ) \widehat{\mathcal{J}}(\theta) J ( θ ) sup v ≠ 0 ∥ ( I − P ^ V ) T ^ v ∥ n 2 / ∥ v ∥ n 2 \sup_{v\ne0}\|(I-\widehat{P}_V)\widehat{T}v\|_n^2/\|v\|_n^2 sup v = 0 ∥ ( I − P V ) T v ∥ n 2 /∥ v ∥ n 2 ( 1 ∓ η ) ∥ ( I − P V ) T ^ v ∥ 2 (1\mp\eta)\|(I-P_V)\widehat{T}v\|^2 ( 1 ∓ η ) ∥ ( I − P V ) T v ∥ 2 ( 1 ∓ η ) ∥ v ∥ 2 (1\mp\eta)\|v\|^2 ( 1 ∓ η ) ∥ v ∥ 2 κ ( η ) − 2 I T ^ ( V ) 2 \kappa(\eta)^{-2}I_{\widehat{T}}(V)^2 κ ( η ) − 2 I T ( V ) 2 κ ( η ) 2 I T ^ ( V ) 2 \kappa(\eta)^2I_{\widehat{T}}(V)^2 κ ( η ) 2 I T ( V ) 2 ( 1 − η ) R ( V ) 2 ≤ max j ∥ ( I − P ^ V ) g j ∥ n 2 ≤ ( 1 + η ) R ( V ) 2 (1-\eta)R(V)^2\le\max_j\|(I-\widehat{P}_V)g_j\|_n^2\le(1+\eta)R(V)^2 ( 1 − η ) R ( V ) 2 ≤ max j ∥ ( I − P V ) g j ∥ n 2 ≤ ( 1 + η ) R ( V ) 2 κ ( η ) − 2 J T ^ ( V ) ≤ J ^ ( θ ) ≤ κ ( η ) 2 J T ^ ( V ) \kappa(\eta)^{-2}\mathcal{J}_{\widehat{T}}(V)\le\widehat{\mathcal{J}}(\theta)\le\kappa(\eta)^2\mathcal{J}_{\widehat{T}}(V) κ ( η ) − 2 J T ( V ) ≤ J ( θ ) ≤ κ ( η ) 2 J T ( V ) ∣ I T ^ ( V ) − I ( V ) ∣ ≤ ∥ ( I − P V ) ( T ^ ∣ V − T ∣ V ) ∥ ≤ Δ ( V ) |I_{\widehat{T}}(V)-I(V)|\le\|(I-P_V)(\widehat{T}|_V-T|_V)\|\le\Delta(V) ∣ I T ( V ) − I ( V ) ∣ ≤ ∥ ( I − P V ) ( T ∣ V − T ∣ V ) ∥ ≤ Δ ( V ) t ↦ ( t 2 + α R ( V ) 2 ) 1 / 2 t\mapsto(t^2+\alpha R(V)^2)^{1/2} t ↦ ( t 2 + α R ( V ) 2 ) 1/2 ∣ J T ^ ( V ) 1 / 2 − J ( V ) 1 / 2 ∣ ≤ Δ ( V ) |\mathcal{J}_{\widehat{T}}(V)^{1/2}-\mathcal{J}(V)^{1/2}|\le\Delta(V) ∣ J T ( V ) 1/2 − J ( V ) 1/2 ∣ ≤ Δ ( V )
Concentration. For a fixed space W W W ρ \rho ρ w k = 1 / n w_k=1/n w k = 1/ n G ^ W − I \widehat{G}_W-I G W − I n n n [65 Theorem 1.4] then gives, for ν ∈ ( 0 , 1 ) \nu\in(0,1) ν ∈ ( 0 , 1 ) 1 − ν 1-\nu 1 − ν
η ( W ) ≤ ( 2 c W log ( 2 dim W / ν ) n ) 1 / 2 + 2 c W log ( 2 dim W / ν ) 3 n . \eta(W)\le\left(\frac{2c_W\log(2\dim W/\nu)}{n}\right)^{1/2}+\frac{2c_W\log(2\dim W/\nu)}{3n}. η ( W ) ≤ ( n 2 c W log ( 2 dim W / ν ) ) 1/2 + 3 n 2 c W log ( 2 dim W / ν ) . where c W c_W c W ∥ χ W ( x ) ∥ 2 2 \|\chi_W(x)\|_2^2 ∥ χ W ( x ) ∥ 2 2 ρ \rho ρ x x x [14 , 15 ] . For stationary ergodic trajectories and bounded χ W \chi_W χ W [52 ] , give η ( W ) = O ( n − 1 / 2 ) \eta(W)=O(n^{-1/2}) η ( W ) = O ( n − 1/2 ) W W W
Uniform control. The quantities η Θ \eta_\Theta η Θ Δ Θ \Delta_\Theta Δ Θ Theorem 4.2 reduce to quantities for a fixed space when the estimator is linear and all candidate spaces lie in a fixed finite-dimensional space U ⊂ F U\subset\mathcal{F} U ⊂ F ∥ ⋅ ∥ \|\cdot\| ∥ ⋅ ∥ W V θ ⊂ W U : = U + T ^ U + span G W_{V_\theta}\subset W_U:=U+\widehat{T}U+\operatorname{span}\mathcal{G} W V θ ⊂ W U := U + T U + span G θ ∈ Θ \theta\in\Theta θ ∈ Θ η \eta η Δ \Delta Δ η Θ ≤ η ( W U ) \eta_\Theta\le\eta(W_U) η Θ ≤ η ( W U ) Δ Θ ≤ Δ ( U ) \Delta_\Theta\le\Delta(U) Δ Θ ≤ Δ ( U ) η \eta η W V W_V W V
Appendix C. Proofs of the main results
Proof of Proposition 4.1
Proof. For v ∈ V v\in V v ∈ V
K ^ v − P V T v = P V ( T ^ − T ) v + ( P ^ V − P V ) T ^ v . \widehat{K}v-P_VTv=P_V(\widehat{T}-T)v+(\widehat{P}_V-P_V)\widehat{T}v. K v − P V T v = P V ( T − T ) v + ( P V − P V ) T v . The first term has norm at most Δ ( V ) ∥ v ∥ \Delta(V)\|v\| Δ ( V ) ∥ v ∥ T ^ v \widehat{T}v T v W V W_V W V ∥ ( I − P V ) T ^ v ∥ ≤ ∥ ( I − P V ) T v ∥ + ∥ ( T ^ − T ) v ∥ ≤ ( I ( V ) + Δ ( V ) ) ∥ v ∥ \|(I-P_V)\widehat{T}v\|\le\|(I-P_V)Tv\|+\|(\widehat{T}-T)v\|\le(\mathcal{I}(V)+\Delta(V))\|v\| ∥ ( I − P V ) T v ∥ ≤ ∥ ( I − P V ) T v ∥ + ∥ ( T − T ) v ∥ ≤ ( I ( V ) + Δ ( V )) ∥ v ∥ η ( 1 − η ) − 1 ( I ( V ) + Δ ( V ) ) ∥ v ∥ \eta(1-\eta)^{-1}(\mathcal{I}(V)+\Delta(V))\|v\| η ( 1 − η ) − 1 ( I ( V ) + Δ ( V )) ∥ v ∥ (7) , and ( T − K ^ ) v = ( I − P V ) T v + ( P V T − K ^ ) v (T-\widehat{K})v=(I-P_V)Tv+(P_VT-\widehat{K})v ( T − K ) v = ( I − P V ) T v + ( P V T − K ) v (8) , because I ( V ) + ( Δ ( V ) + η I ( V ) ) / ( 1 − η ) = ( I ( V ) + Δ ( V ) ) / ( 1 − η ) \mathcal{I}(V)+(\Delta(V)+\eta\mathcal{I}(V))/(1-\eta)=(\mathcal{I}(V)+\Delta(V))/(1-\eta) I ( V ) + ( Δ ( V ) + η I ( V )) / ( 1 − η ) = ( I ( V ) + Δ ( V )) / ( 1 − η )
Proof of Theorem 4.2
Proof. Let θ ∈ Θ \theta\in\Theta θ ∈ Θ κ : = κ ( η Θ ) ≥ 1 \kappa:= \kappa(\eta_{\Theta}) \ge1 κ := κ ( η Θ ) ≥ 1 η V ^ ≤ η Θ \eta_{\widehat{V}} \le\eta_{\Theta} η V ≤ η Θ Δ ( V ^ ) ≤ Δ Θ \Delta(\widehat{V}) \le\Delta_{\Theta} Δ ( V ) ≤ Δ Θ θ = θ ^ \theta= \widehat{\theta} θ = θ J ( V ^ ) 1 / 2 ≤ κ J ^ ( θ ^ ) 1 / 2 + Δ Θ \mathcal{J}(\widehat{V})^{1/2} \le\kappa\widehat{\mathcal{J}}(\widehat{\theta})^{1/2} + \Delta_{\Theta} J ( V ) 1/2 ≤ κ J ( θ ) 1/2 + Δ Θ J ^ ( θ ^ ) 1 / 2 ≤ J ^ ( θ ) 1 / 2 + ζ 1 / 2 ≤ κ ( J ( V θ ) 1 / 2 + Δ Θ ) + ζ 1 / 2 \widehat{\mathcal{J}}(\widehat{\theta})^{1/2} \le\widehat{\mathcal{J}}(\theta)^{1/2} + \zeta^{1/2} \le\kappa(\mathcal{J}(V_{\theta})^{1/2} + \Delta_{\Theta}) + \zeta^{1/2} J ( θ ) 1/2 ≤ J ( θ ) 1/2 + ζ 1/2 ≤ κ ( J ( V θ ) 1/2 + Δ Θ ) + ζ 1/2 J ( V ^ ) 1 / 2 ≤ κ 2 ( J ( V θ ) 1 / 2 + Δ Θ ) + κ ζ 1 / 2 + Δ Θ \mathcal{J}(\widehat{V})^{1/2} \le\kappa^2(\mathcal{J}(V_{\theta})^{1/2} + \Delta_{\Theta}) + \kappa\zeta^{1/2} + \Delta_{\Theta} J ( V ) 1/2 ≤ κ 2 ( J ( V θ ) 1/2 + Δ Θ ) + κ ζ 1/2 + Δ Θ
Proof of Theorem 4.3
Proof. Let R : = P V ι : E ⋆ → V R := P_{V}\iota: E_{\star} \to V R := P V ι : E ⋆ → V u ∈ E ⋆ u \in E_{\star} u ∈ E ⋆ ∥ R u − u ∥ = ∥ ( I − P V ) u ∥ ≤ d ∥ u ∥ \|Ru-u\| = \|(I-P_{V})u\| \le d\|u\| ∥ R u − u ∥ = ∥ ( I − P V ) u ∥ ≤ d ∥ u ∥ R ∗ R = I − ι ∗ ( I − P V ) ι R^{*}R = I-\iota^{*}(I-P_{V})\iota R ∗ R = I − ι ∗ ( I − P V ) ι ( 1 − d 2 ) I ≤ R ∗ R ≤ I (1-d^2)I \le R^{*}R \le I ( 1 − d 2 ) I ≤ R ∗ R ≤ I ∥ ( I − P V ) ι ∥ = d \|(I-P_{V})\iota\| = d ∥ ( I − P V ) ι ∥ = d R R R [ ( 1 − d 2 ) 1 / 2 , 1 ] [(1-d^2)^{1/2},1] [( 1 − d 2 ) 1/2 , 1 ] Q : = R ( R ∗ R ) − 1 / 2 Q := R(R^{*}R)^{-1/2} Q := R ( R ∗ R ) − 1/2 E ⋆ E_{\star} E ⋆ V V V
∥ Q − ι ∥ ≤ ∥ R ( ( R ∗ R ) − 1 / 2 − I ) ∥ + ∥ R − ι ∥ ≤ 1 − ( 1 − d 2 ) 1 / 2 + d ≤ d 2 + d ≤ 2 d . \|Q-\iota\| \le\|R((R^{*}R)^{-1/2}-I)\|+\|R-\iota\| \le1-(1-d^2)^{1/2}+d \le d^2+d \le2d. ∥ Q − ι ∥ ≤ ∥ R (( R ∗ R ) − 1/2 − I ) ∥ + ∥ R − ι ∥ ≤ 1 − ( 1 − d 2 ) 1/2 + d ≤ d 2 + d ≤ 2 d . since ∥ R ( ( R ∗ R ) − 1 / 2 − I ) ∥ = ∥ I − ( R ∗ R ) 1 / 2 ∥ \|R((R^{*}R)^{-1/2}-I)\|=\|I-(R^{*}R)^{1/2}\| ∥ R (( R ∗ R ) − 1/2 − I ) ∥ = ∥ I − ( R ∗ R ) 1/2 ∥ Q Q Q H \mathcal{H} H Q ∗ P V = Q ∗ Q^{*}P_{V}=Q^{*} Q ∗ P V = Q ∗ T ∣ E ⋆ = ι ∗ T ι T|_{E_{\star}}=\iota^{*}T\iota T ∣ E ⋆ = ι ∗ T ι
Q ∗ K ^ Q − T ∣ E ⋆ = Q ∗ ( K ^ − P V T ∣ V ) Q + ( Q ∗ T Q − ι ∗ T ι ) . Q^{*}\widehat{K}Q-T|_{E_{\star}}=Q^{*}(\widehat{K}-P_{V}T|_{V})Q+(Q^{*}TQ-\iota^{*}T\iota). Q ∗ K Q − T ∣ E ⋆ = Q ∗ ( K − P V T ∣ V ) Q + ( Q ∗ T Q − ι ∗ T ι ) . By (4.2) and the upper bound I ( V ) ≤ 2 M T d \mathcal{I}(V)\le2M_{T}d I ( V ) ≤ 2 M T d V V V ∥ K ^ − P V T ∣ V ∥ ≤ ( Δ ( V ) + 2 η M T d ) / ( 1 − η ) \|\widehat{K}-P_{V}T|_{V}\|\le(\Delta(V)+2\eta M_{T}d)/(1-\eta) ∥ K − P V T ∣ V ∥ ≤ ( Δ ( V ) + 2 η M T d ) / ( 1 − η ) 2 M T ∥ Q − ι ∥ ≤ 4 M T d 2M_{T}\|Q-\iota\|\le4M_{T}d 2 M T ∥ Q − ι ∥ ≤ 4 M T d 2 η + 4 ( 1 − η ) ≤ 4 2\eta+4(1-\eta)\le4 2 η + 4 ( 1 − η ) ≤ 4
For the convergence statement, recall that the eigenvalues of K ^ \widehat{K} K K ^ \widehat{K} K Q ∗ K ^ Q = Q − 1 K ^ Q Q^{*}\widehat{K}Q=Q^{-1}\widehat{K}Q Q ∗ K Q = Q − 1 K Q k k k dist ( V k , E ⋆ ) < 1 \operatorname{dist}(V_{k},E_{\star})<1 dist ( V k , E ⋆ ) < 1 Q k Q_{k} Q k ∥ Q k − ι ∥ → 0 \|Q_{k}-\iota\|\to0 ∥ Q k − ι ∥ → 0 sup k η V k < 1 \sup_{k}\eta_{V_{k}}<1 sup k η V k < 1 A k : = Q k ∗ K ^ k Q k → T ∣ E ⋆ A_{k}:=Q_{k}^{*}\widehat{K}_{k}Q_{k}\to T|_{E_{\star}} A k := Q k ∗ K k Q k → T ∣ E ⋆ B ( E ⋆ ) B(E_{\star}) B ( E ⋆ ) E ⋆ E_{\star} E ⋆ A k A_{k} A k T ∣ E ⋆ T|_{E_{\star}} T ∣ E ⋆ A k A_{k} A k spec ( T ∣ E ⋆ ) \operatorname{spec}(T|_{E_{\star}}) spec ( T ∣ E ⋆ ) T ∣ E ⋆ T|_{E_{\star}} T ∣ E ⋆ [28 Chap. II, §5.1] , [12 ] ; hence their ranges F k F_{k} F k dist \operatorname{dist} dist F F F K ^ k \widehat{K}_{k} K k Q k F k Q_{k}F_{k} Q k F k ∥ P Q k F k − P F k ∥ ≤ 2 ∥ Q k − ι ∥ → 0 \|P_{Q_{k}F_{k}}-P_{F_{k}}\|\le2\|Q_{k}-\iota\|\to0 ∥ P Q k F k − P F k ∥ ≤ 2∥ Q k − ι ∥ → 0 Q k Q_{k} Q k dist ( Q k F k , F ) → 0 \operatorname{dist}(Q_{k}F_{k},F)\to0 dist ( Q k F k , F ) → 0
Proof of Theorem 4.4
Proof. Let d : = dist ( V , E ⋆ ) d:=\operatorname{dist}(V,E_{\star}) d := dist ( V , E ⋆ ) T P V = T P ⋆ P V + T ( I − P ⋆ ) P V TP_{V}=TP_{\star}P_{V}+T(I-P_{\star})P_{V} T P V = T P ⋆ P V + T ( I − P ⋆ ) P V T P ⋆ = P ⋆ T P ⋆ TP_{\star}=P_{\star}TP_{\star} T P ⋆ = P ⋆ T P ⋆
I ( V ) = ∥ ( I − P V ) T P V ∥ ≤ ∥ ( I − P V ) P ⋆ ∥ M T + M T ∥ ( I − P ⋆ ) P V ∥ = 2 M T d . \mathcal{I}(V)=\|(I-P_{V})TP_{V}\|\le\|(I-P_{V})P_{\star}\|M_{T}+M_{T}\|(I-P_{\star})P_{V}\|=2M_{T}d. I ( V ) = ∥ ( I − P V ) T P V ∥ ≤ ∥ ( I − P V ) P ⋆ ∥ M T + M T ∥ ( I − P ⋆ ) P V ∥ = 2 M T d . For the lower bound, d ≤ r ⋆ / 2 < 1 d\le r_{\star}/\sqrt{2}<1 d ≤ r ⋆ / 2 < 1 P ⋆ P_{\star} P ⋆ V V V E ⋆ E_{\star} E ⋆ V = { u + Z u : u ∈ E ⋆ } V=\{u+Zu:u\in E_{\star}\} V = { u + Z u : u ∈ E ⋆ } Z ∈ B ( E ⋆ , E ⋆ ⊥ ) Z\in B(E_{\star},E_{\star}^{\perp}) Z ∈ B ( E ⋆ , E ⋆ ⊥ ) d = ∥ ( I − P ⋆ ) P V ∥ = ∥ Z ∥ ( 1 + ∥ Z ∥ 2 ) − 1 / 2 d=\|(I-P_{\star})P_{V}\|=\|Z\|(1+\|Z\|^{2})^{-1/2} d = ∥ ( I − P ⋆ ) P V ∥ = ∥ Z ∥ ( 1 + ∥ Z ∥ 2 ) − 1/2 ∥ ( I − P ⋆ ) v ∥ 2 / ∥ v ∥ 2 = ∥ Z u ∥ 2 / ( ∥ u ∥ 2 + ∥ Z u ∥ 2 ) \|(I-P_{\star})v\|^{2}/\|v\|^{2}=\|Zu\|^{2}/(\|u\|^{2}+\|Zu\|^{2}) ∥ ( I − P ⋆ ) v ∥ 2 /∥ v ∥ 2 = ∥ Z u ∥ 2 / ( ∥ u ∥ 2 + ∥ Z u ∥ 2 ) v = u + Z u v=u+Zu v = u + Z u t : = ∥ Z ∥ = d ( 1 − d 2 ) − 1 / 2 t:=\|Z\|=d(1-d^{2})^{-1/2} t := ∥ Z ∥ = d ( 1 − d 2 ) − 1/2 d ≤ t ≤ 2 d ≤ r ⋆ ≤ 1 d\le t\le\sqrt{2}d\le r_{\star}\le1 d ≤ t ≤ 2 d ≤ r ⋆ ≤ 1 v = u + Z u v=u+Zu v = u + Z u E ⋆ E_{\star} E ⋆
T v = u ′ + Z u ′ + Ric ( Z ) u , u ′ : = ( A ⋆ + B ⋆ Z ) u ∈ E ⋆ , Ric ( Z ) : = D ⋆ Z − Z A ⋆ − Z B ⋆ Z , Tv=u'+Zu'+\operatorname{Ric}(Z)u,\qquad u':=(A_{\star}+B_{\star}Z)u\in E_{\star},\qquad\operatorname{Ric}(Z):=D_{\star}Z-ZA_{\star}-ZB_{\star}Z, T v = u ′ + Z u ′ + Ric ( Z ) u , u ′ := ( A ⋆ + B ⋆ Z ) u ∈ E ⋆ , Ric ( Z ) := D ⋆ Z − Z A ⋆ − Z B ⋆ Z , so that ( I − P V ) T v = ( I − P V ) Ric ( Z ) u (I-P_V)Tv=(I-P_V)\operatorname{Ric}(Z)u ( I − P V ) T v = ( I − P V ) Ric ( Z ) u Ric ( Z ) u ∈ E ⊥ \operatorname{Ric}(Z)u\in E^\perp Ric ( Z ) u ∈ E ⊥ ∥ v ∥ ≤ ( 1 + t 2 ) 1 / 2 ∥ u ∥ \|v\|\le(1+t^2)^{1/2}\|u\| ∥ v ∥ ≤ ( 1 + t 2 ) 1/2 ∥ u ∥ w ∈ E ⊥ w\in E^\perp w ∈ E ⊥ u ~ ∈ E ⋆ \widetilde{u}\in E_\star u ∈ E ⋆ ∥ w ∥ ≤ ∥ w − Z u ~ ∥ + t ∥ u ~ ∥ ≤ ( 1 + t 2 ) 1 / 2 ∥ w − u ~ − Z u ~ ∥ \|w\|\le\|w-Z\widetilde{u}\|+t\|\widetilde{u}\|\le(1+t^2)^{1/2}\|w-\widetilde{u}-Z\widetilde{u}\| ∥ w ∥ ≤ ∥ w − Z u ∥ + t ∥ u ∥ ≤ ( 1 + t 2 ) 1/2 ∥ w − u − Z u ∥ ∥ ( I − P V ) w ∥ ≥ ( 1 + t 2 ) − 1 / 2 ∥ w ∥ \|(I-P_V)w\|\ge(1+t^2)^{-1/2}\|w\| ∥ ( I − P V ) w ∥ ≥ ( 1 + t 2 ) − 1/2 ∥ w ∥ I ( V ) ≥ ∥ Ric ( Z ) ∥ / ( 1 + t 2 ) \mathcal{I}(V)\ge\|\operatorname{Ric}(Z)\|/(1+t^2) I ( V ) ≥ ∥ Ric ( Z ) ∥/ ( 1 + t 2 ) ∥ D ⋆ Z − Z A ⋆ ∥ ≥ δ ⋆ t \|D_\star Z-ZA_\star\|\ge\delta_\star t ∥ D ⋆ Z − Z A ⋆ ∥ ≥ δ ⋆ t ∥ Z B ⋆ Z ∥ ≤ ∥ B ⋆ ∥ t 2 ≤ δ ⋆ t / 2 \|ZB_\star Z\|\le\|B_\star\|t^2\le\delta_\star t/2 ∥ Z B ⋆ Z ∥ ≤ ∥ B ⋆ ∥ t 2 ≤ δ ⋆ t /2 t ≤ r ⋆ t\le r_\star t ≤ r ⋆ ∥ Ric ( Z ) ∥ ≥ δ ⋆ t / 2 \|\operatorname{Ric}(Z)\|\ge\delta_\star t/2 ∥ Ric ( Z ) ∥ ≥ δ ⋆ t /2 1 + t 2 ≤ 2 1+t^2\le2 1 + t 2 ≤ 2 I ( V ) ≥ δ ⋆ t / 4 ≥ δ ⋆ d / 4 \mathcal{I}(V)\ge\delta_\star t/4\ge\delta_\star d/4 I ( V ) ≥ δ ⋆ t /4 ≥ δ ⋆ d /4
Proof of Theorem 4.5
Proof. Write P : = P V P:=P_V P := P V P ^ : = P ^ V \widehat{P}:=\widehat{P}_V P := P V
T N f − f ( N ) = T N ( I − P ) f + ∑ k = 0 N − 1 T N − 1 − k ( T − K ^ ) K ^ k P f + K ^ N ( P − P ^ ) f . T^Nf-f^{(N)}=T^N(I-P)f+\sum_{k=0}^{N-1}T^{N-1-k}(T-\widehat{K})\widehat{K}^kPf+\widehat{K}^N(P-\widehat{P})f. T N f − f ( N ) = T N ( I − P ) f + k = 0 ∑ N − 1 T N − 1 − k ( T − K ) K k P f + K N ( P − P ) f . The first term is bounded by M T N ∥ ( I − P ) f ∥ M_T^N\|(I-P)f\| M T N ∥ ( I − P ) f ∥ K ^ k P f ∈ V \widehat{K}^kPf\in V K k P f ∈ V (8) bounds the k k k M T N − 1 − k e ( V ) ∥ K ^ ∥ k ∥ P f ∥ M_T^{N-1-k}e(V)\|\widehat{K}\|^k\|Pf\| M T N − 1 − k e ( V ) ∥ K ∥ k ∥ P f ∥ f ∈ W V f\in W_V f ∈ W V ∥ K ^ ∥ N η ( 1 − η ) − 1 ∥ ( I − P ) f ∥ \|\widehat{K}\|^N\eta(1-\eta)^{-1}\|(I-P)f\| ∥ K ∥ N η ( 1 − η ) − 1 ∥ ( I − P ) f ∥ M T N − 1 − k ∥ K ^ ∥ k ≤ M T N − 1 M_T^{N-1-k}\|\widehat{K}\|^k\le M_T^{N-1} M T N − 1 − k ∥ K ∥ k ≤ M T N − 1 M T N + ∥ K ^ ∥ N η / ( 1 − η ) ≤ M T N / ( 1 − η ) M_T^N+\|\widehat{K}\|^N\eta/(1-\eta)\le M_T^N/(1-\eta) M T N + ∥ K ∥ N η / ( 1 − η ) ≤ M T N / ( 1 − η ) (13) .
For the selected space V ^ \widehat{V} V f ∈ G f\in\mathcal{G} f ∈ G Theorem 4.2 gives ∥ ( I − P V ^ ) f ∥ ≤ R ( V ^ ) ≤ ( J ( V ^ ) / α ) 1 / 2 ≤ ϵ Θ / α 1 / 2 \|(I-P_{\widehat{V}})f\|\le\mathcal{R}(\widehat{V})\le(\mathcal{J}(\widehat{V})/\alpha)^{1/2}\le\epsilon\Theta/\alpha^{1/2} ∥ ( I − P V ) f ∥ ≤ R ( V ) ≤ ( J ( V ) / α ) 1/2 ≤ ϵ Θ/ α 1/2 I ( V ^ ) ≤ J ( V ^ ) 1 / 2 ≤ ϵ Θ \mathcal{I}(\widehat{V})\le\mathcal{J}(\widehat{V})^{1/2}\le\epsilon\Theta I ( V ) ≤ J ( V ) 1/2 ≤ ϵ Θ e ( V ^ ) ≤ ( ϵ Θ + Δ Θ ) / ( 1 − η Θ ) e(\widehat{V})\le(\epsilon\Theta+\Delta\Theta)/(1-\eta\Theta) e ( V ) ≤ ( ϵ Θ + ΔΘ ) / ( 1 − η Θ ) η V ^ ≤ η Θ \eta_{\widehat{V}}\le\eta_\Theta η V ≤ η Θ ∥ P V ^ f ∥ ≤ ∥ f ∥ \|P_{\widehat{V}}f\|\le\|f\| ∥ P V f ∥ ≤ ∥ f ∥ (13) gives (14) .
Appendix D. Supplements to the error analysis
Identification of the target subspace
Separation. If E ⋆ E_\star E ⋆ Λ ⋆ \Lambda_\star Λ ⋆ δ ⋆ > 0 \delta_\star>0 δ ⋆ > 0 spec ( A ⋆ ) = Λ ⋆ \operatorname{spec}(A_\star)=\Lambda_\star spec ( A ⋆ ) = Λ ⋆ spec ( D ⋆ ) = spec ( T ) ∖ Λ ⋆ \operatorname{spec}(D_\star)=\operatorname{spec}(T)\setminus\Lambda_\star spec ( D ⋆ ) = spec ( T ) ∖ Λ ⋆ T T T Ran ( I − Π ) \operatorname{Ran}(I-\Pi) Ran ( I − Π ) [28 Chap. III, §6.4] and is similar to D ⋆ D_\star D ⋆ I − P ⋆ I-P_\star I − P ⋆ Z ↦ D ⋆ Z − Z A ⋆ Z\mapsto D_\star Z-ZA_\star Z ↦ D ⋆ Z − Z A ⋆ [5 , 56 ] . If T T T Λ ⋆ \Lambda_\star Λ ⋆ m m m B ⋆ = 0 B_\star=0 B ⋆ = 0 r ⋆ = 1 r_\star=1 r ⋆ = 1 δ ⋆ \delta_\star δ ⋆ Λ ⋆ \Lambda_\star Λ ⋆ [5 ] ; in general, δ ⋆ \delta_\star δ ⋆ T T T
Resolution. Let
s ⋆ : = min { ( ∑ j = 1 q ⟨ u , g j ⟩ 2 ) 1 / 2 : u ∈ E ⋆ , ∥ u ∥ = 1 } . s_\star:=\min\left\{\left(\sum_{j=1}^{q}\langle u,g_j\rangle^2\right)^{1/2}:u\in E_\star,\ \|u\|=1\right\}. s ⋆ := min ⎩ ⎨ ⎧ ( j = 1 ∑ q ⟨ u , g j ⟩ 2 ) 1/2 : u ∈ E ⋆ , ∥ u ∥ = 1 ⎭ ⎬ ⎫ . it measures how well the reconstruction observables resolve E ⋆ E_\star E ⋆ s ⋆ > 0 s_\star>0 s ⋆ > 0 E ⋆ E_\star E ⋆ E ⋆ E_\star E ⋆
Lemma D.1 (Localization by the reconstruction term). For every m m m V ⊂ H V\subset\mathcal{H} V ⊂ H
s ⋆ dist ( V , E ⋆ ) ≤ q ( R ( V ) + R ( E ⋆ ) ) . (33) s_\star\operatorname{dist}(V,E_\star)\le\sqrt{q}\left(\mathcal{R}(V)+\mathcal{R}(E_\star)\right). \tag*{(33)} s ⋆ dist ( V , E ⋆ ) ≤ q ( R ( V ) + R ( E ⋆ ) ) . (33) In particular, if G ⊂ E ⋆ \mathcal{G}\subset E_\star G ⊂ E ⋆ s ⋆ > 0 s_\star>0 s ⋆ > 0 dist ( V , E ⋆ ) ≤ ( q / s ⋆ ) ( J ( V ) / α ) 1 / 2 \operatorname{dist}(V,E_\star)\le(\sqrt{q}/s_\star)(\mathcal{J}(V)/\alpha)^{1/2} dist ( V , E ⋆ ) ≤ ( q / s ⋆ ) ( J ( V ) / α ) 1/2
Proof. Let d : = dist ( V , E ⋆ ) d:=\operatorname{dist}(V,E_\star) d := dist ( V , E ⋆ ) d = 0 d=0 d = 0 E ⋆ E_\star E ⋆ V V V d = ∥ ( I − P V ) P ⋆ ∥ d=\|(I-P_V)P_\star\| d = ∥ ( I − P V ) P ⋆ ∥ u ∈ E ⋆ u\in E_\star u ∈ E ⋆ ∥ ( I − P V ) u ∥ = \|(I-P_V)u\|= ∥ ( I − P V ) u ∥ = w : = ( I − P V ) u / d w := (I-P_V)u/d w := ( I − P V ) u / d w ⊥ V w \perp V w ⊥ V ∥ w ∥ = 1 \|w\|=1 ∥ w ∥ = 1 ⟨ w , u ⟩ = ∥ ( I − P V ) u ∥ 2 / d = d \langle w,u\rangle=\|(I-P_V)u\|^2/d=d ⟨ w , u ⟩ = ∥ ( I − P V ) u ∥ 2 / d = d d ≤ ∥ P ⋆ w ∥ d\leq\|P_\star w\| d ≤ ∥ P ⋆ w ∥ s ⋆ s_\star s ⋆ s ⋆ ∥ P ⋆ w ∥ ≤ ( ∑ j ⟨ P ⋆ w , g j ⟩ 2 ) 1 / 2 s_\star\|P_\star w\|\leq\left(\sum_j\langle P_\star w,g_j\rangle^2\right)^{1/2} s ⋆ ∥ P ⋆ w ∥ ≤ ( ∑ j ⟨ P ⋆ w , g j ⟩ 2 ) 1/2 w ⊥ V w\perp V w ⊥ V
⟨ P ⋆ w , g j ⟩ = ⟨ w , ( I − P V ) g j ⟩ − ⟨ w , ( I − P ⋆ ) g j ⟩ , j = 1 , … , q . \langle P_\star w,g_j\rangle=\langle w,(I-P_V)g_j\rangle-\langle w,(I-P_\star)g_j\rangle,\qquad j=1,\ldots,q. ⟨ P ⋆ w , g j ⟩ = ⟨ w , ( I − P V ) g j ⟩ − ⟨ w , ( I − P ⋆ ) g j ⟩ , j = 1 , … , q . Hence ∣ ⟨ P ⋆ w , g j ⟩ ∣ ≤ R ( V ) + R ( E ⋆ ) |\langle P_\star w,g_j\rangle|\leq\mathcal{R}(V)+\mathcal{R}(E_\star) ∣ ⟨ P ⋆ w , g j ⟩ ∣ ≤ R ( V ) + R ( E ⋆ ) s ⋆ d ≤ s ⋆ ∥ P ⋆ w ∥ ≤ q ( R ( V ) + R ( E ⋆ ) ) s_\star d\leq s_\star\|P_\star w\|\leq\sqrt{q}\,(\mathcal{R}(V)+\mathcal{R}(E_\star)) s ⋆ d ≤ s ⋆ ∥ P ⋆ w ∥ ≤ q ( R ( V ) + R ( E ⋆ )) G ⊂ E ⋆ G\subset E_\star G ⊂ E ⋆ R ( E ⋆ ) = 0 \mathcal{R}(E_\star)=0 R ( E ⋆ ) = 0 R ( V ) ≤ ( J ( V ) / α ) 1 / 2 \mathcal{R}(V)\leq(\mathcal{J}(V)/\alpha)^{1/2} R ( V ) ≤ ( J ( V ) / α ) 1/2
If G ⊂ E ⋆ G\subset E_\star G ⊂ E ⋆ s ⋆ > 0 s_\star>0 s ⋆ > 0 E ⋆ E_\star E ⋆
Corollary D.2 (Identification from the objective). Let δ ⋆ > 0 \delta_\star>0 δ ⋆ > 0 s ⋆ > 0 s_\star>0 s ⋆ > 0 V V V m m m
q s ⋆ ( ( J ( V ) α ) 1 / 2 + R ( E ⋆ ) ) ≤ r ⋆ 2 . (34) \frac{\sqrt{q}}{s_\star}\left(\left(\frac{\mathcal{J}(V)}{\alpha}\right)^{1/2}+\mathcal{R}(E_\star)\right)\leq\frac{r_\star}{\sqrt{2}}. \tag*{(34)} s ⋆ q ( ( α J ( V ) ) 1/2 + R ( E ⋆ ) ) ≤ 2 r ⋆ . (34) Then
dist ( V , E ⋆ ) ≤ 4 I ( V ) δ ⋆ ≤ 4 J ( V ) 1 / 2 δ ⋆ . (35) \operatorname{dist}(V,E_\star)\leq\frac{4\mathcal{I}(V)}{\delta_\star}\leq\frac{4\mathcal{J}(V)^{1/2}}{\delta_\star}. \tag*{(35)} dist ( V , E ⋆ ) ≤ δ ⋆ 4 I ( V ) ≤ δ ⋆ 4 J ( V ) 1/2 . (35) In particular, if η Θ < 1 \eta_\Theta<1 η Θ < 1 J ( V ) 1 / 2 \mathcal{J}(V)^{1/2} J ( V ) 1/2 ϵ Θ \epsilon_\Theta ϵ Θ V ^ \widehat{V} V Theorem 4.2 satisfies dist ( V ^ , E ⋆ ) ≤ 4 ϵ Θ / δ ⋆ \operatorname{dist}(\widehat{V},E_\star)\leq4\epsilon_\Theta/\delta_\star dist ( V , E ⋆ ) ≤ 4 ϵ Θ / δ ⋆ Q Q Q Theorem 4.3 for V = V ^ V=\widehat{V} V = V
∥ Q ⋆ K ^ Q − T ∣ E ⋆ ∥ ≤ Δ Θ + 16 M T ϵ Θ / δ ⋆ 1 − η Θ . (36) \|Q^\star\widehat{K}Q-T|_{E_\star}\|\leq\frac{\Delta_\Theta+16M_T\epsilon_\Theta/\delta_\star}{1-\eta_\Theta}. \tag*{(36)} ∥ Q ⋆ K Q − T ∣ E ⋆ ∥ ≤ 1 − η Θ Δ Θ + 16 M T ϵ Θ / δ ⋆ . (36) Proof. By (D.1) and R ( V ) ≤ ( J ( V ) / α ) 1 / 2 \mathcal{R}(V)\leq(\mathcal{J}(V)/\alpha)^{1/2} R ( V ) ≤ ( J ( V ) / α ) 1/2 dist ( V , E ⋆ ) ≤ r ⋆ / 2 \operatorname{dist}(V,E_\star)\leq r_\star/\sqrt{2} dist ( V , E ⋆ ) ≤ r ⋆ / 2 (12) together with I ( V ) ≤ J ( V ) 1 / 2 \mathcal{I}(V)\leq\mathcal{J}(V)^{1/2} I ( V ) ≤ J ( V ) 1/2 J ( V ^ ) 1 / 2 ≤ ϵ Θ \mathcal{J}(\widehat{V})^{1/2}\leq\epsilon_\Theta J ( V ) 1/2 ≤ ϵ Θ Theorem 4.2 , so that (D.2) holds for V = V ^ V=\widehat{V} V = V dist ( V ^ , E ⋆ ) ≤ 4 ϵ Θ / δ ⋆ \operatorname{dist}(\widehat{V},E_\star)\leq4\epsilon_\Theta/\delta_\star dist ( V , E ⋆ ) ≤ 4 ϵ Θ / δ ⋆ dist ( V ^ , E ⋆ ) ≤ r ⋆ / 2 < 1 \operatorname{dist}(\widehat{V},E_\star)\leq r_\star/\sqrt{2}<1 dist ( V , E ⋆ ) ≤ r ⋆ / 2 < 1 η V ^ ≤ η Θ \eta_{\widehat{V}}\leq\eta_\Theta η V ≤ η Θ Δ ( V ^ ) ≤ Δ Θ \Delta(\widehat{V})\leq\Delta_\Theta Δ ( V ) ≤ Δ Θ (11) for V = V ^ V=\widehat{V} V = V
Since ϵ Θ ≥ 2 Δ Θ + ζ 1 / 2 \epsilon_\Theta\geq2\Delta_\Theta+\zeta^{1/2} ϵ Θ ≥ 2 Δ Θ + ζ 1/2 α − 1 / 2 ( Δ Θ + ζ 1 / 2 ) \alpha^{-1/2}(\Delta_\Theta+\zeta^{1/2}) α − 1/2 ( Δ Θ + ζ 1/2 ) α \alpha α E ⋆ E_\star E ⋆ G G G
Bias. If G ⊄ E ⋆ G\not\subset E_\star G ⊂ E ⋆ J \mathcal{J} J E ⋆ E_\star E ⋆ R ( E ⋆ ) \mathcal{R}(E_\star) R ( E ⋆ ) α → 0 \alpha\to0 α → 0 ρ \rho ρ u u u T T T ∥ u ∥ = 1 \|u\|=1 ∥ u ∥ = 1 E ⋆ = span { u } E_\star=\operatorname{span}\{u\} E ⋆ = span { u } G = { 2 ⋅ 1 + u } G=\{2\cdot\mathbf{1}+u\} G = { 2 ⋅ 1 + u } J ( E ⋆ ) = 4 α > α = J ( span { 1 } ) \mathcal{J}(E_\star)=4\alpha>\alpha=\mathcal{J}(\operatorname{span}\{\mathbf{1}\}) J ( E ⋆ ) = 4 α > α = J ( span { 1 }) span { 1 } \operatorname{span}\{\mathbf{1}\} span { 1 } R ( E ⋆ ) \mathcal{R}(E_\star) R ( E ⋆ ) s ⋆ r ⋆ s_\star r_\star s ⋆ r ⋆
Residual bound for computed eigenpairs
Individual computed eigenpairs can be checked a posteriori by a residual of the form used in residual DMD [18 ] , computed from the matrices K ^ \widehat{K} K L ^ \widehat{L} L
Proposition D.3. Let η : = η V < 1 \eta:= \eta_V < 1 η := η V < 1 μ ∈ C \mu\in\mathbb{C} μ ∈ C c ∈ C m ∖ { 0 } c \in\mathbb{C}^m \setminus\{0\} c ∈ C m ∖ { 0 } φ : = ∑ j c j ψ ^ j \varphi:= \sum_j c_j \widehat{\psi}_j φ := ∑ j c j ψ j
res ^ ( μ , c ) : = ∥ c ∥ 2 − 1 ( c ∗ L ^ c − 2 Re ( μ c ∗ K ^ c ) + ∣ μ ∣ 2 ∥ c ∥ 2 2 ) 1 / 2 . \widehat{\operatorname{res}}(\mu,c) := \lVert c\rVert_2^{-1}\left(c^* \widehat{L}c - 2\operatorname{Re}(\mu c^* \widehat{K}c) + |\mu|^2\lVert c\rVert_2^2\right)^{1/2}. res ( μ , c ) := ∥ c ∥ 2 − 1 ( c ∗ L c − 2 Re ( μ c ∗ K c ) + ∣ μ ∣ 2 ∥ c ∥ 2 2 ) 1/2 . Then
∥ T φ − μ φ ∥ ∥ φ ∥ ≤ Δ ( V ) + κ ( η ) res ^ ( μ , c ) . (37) \frac{\lVert T\varphi-\mu\varphi\rVert}{\lVert\varphi\rVert} \leq\Delta(V) + \kappa(\eta)\widehat{\operatorname{res}}(\mu,c). \tag*{(37)} ∥ φ ∥ ∥ T φ − μ φ ∥ ≤ Δ ( V ) + κ ( η ) res ( μ , c ) . (37) Proof. Since Ψ ^ \widehat{\Psi} Ψ ⟨ ⋅ , ⋅ ⟩ n \langle\cdot,\cdot\rangle_n ⟨ ⋅ , ⋅ ⟩ n ∥ φ ∥ n = ∥ c ∥ 2 \lVert\varphi\rVert_n=\lVert c\rVert_2 ∥ φ ∥ n = ∥ c ∥ 2 ∥ T ^ φ − μ φ ∥ n 2 = c ∗ L ^ c − 2 Re ( μ c ∗ K ^ c ) + ∣ μ ∣ 2 ∥ c ∥ 2 2 \lVert\widehat{T}\varphi-\mu\varphi\rVert_n^2=c^*\widehat{L}c-2\operatorname{Re}(\mu c^*\widehat{K}c)+|\mu|^2\lVert c\rVert_2^2 ∥ T φ − μ φ ∥ n 2 = c ∗ L c − 2 Re ( μ c ∗ K c ) + ∣ μ ∣ 2 ∥ c ∥ 2 2 [18 ] with the values of the estimator in place of observed values; thus res ^ ( μ , c ) = ∥ T ^ φ − μ φ ∥ n / ∥ φ ∥ n \widehat{\operatorname{res}}(\mu,c)=\lVert\widehat{T}\varphi-\mu\varphi\rVert_n/\lVert\varphi\rVert_n res ( μ , c ) = ∥ T φ − μ φ ∥ n / ∥ φ ∥ n T ^ φ − μ φ \widehat{T}\varphi-\mu\varphi T φ − μ φ φ \varphi φ W V W_V W V ∥ T ^ φ − μ φ ∥ ≤ κ ( η ) res ^ ( μ , c ) ∥ φ ∥ \lVert\widehat{T}\varphi-\mu\varphi\rVert\leq\kappa(\eta)\widehat{\operatorname{res}}(\mu,c)\lVert\varphi\rVert ∥ T φ − μ φ ∥ ≤ κ ( η ) res ( μ , c ) ∥ φ ∥ ∥ T φ − μ φ ∥ ≤ ∥ ( T − T ^ ) φ ∥ + ∥ T ^ φ − μ φ ∥ \lVert T\varphi-\mu\varphi\rVert\leq\lVert(T-\widehat{T})\varphi\rVert+\lVert\widehat{T}\varphi-\mu\varphi\rVert ∥ T φ − μ φ ∥ ≤ ∥( T − T ) φ ∥ + ∥ T φ − μ φ ∥
The quantity res ^ ( μ , c ) = ∥ T ^ φ − μ φ ∥ n / ∥ φ ∥ n \widehat{\operatorname{res}}(\mu,c)=\lVert\widehat{T}\varphi-\mu\varphi\rVert_n/\lVert\varphi\rVert_n res ( μ , c ) = ∥ T φ − μ φ ∥ n / ∥ φ ∥ n K ^ c = μ ^ c \widehat{K}c=\widehat{\mu}c K c = μ c
res ^ ( μ ^ , c ) 2 = c ∗ L ^ c ∥ c ∥ 2 2 − ∣ μ ^ ∣ 2 . \widehat{\operatorname{res}}(\widehat{\mu},c)^2 = \frac{c^*\widehat{L}c}{\lVert c\rVert_2^2}-|\widehat{\mu}|^2. res ( μ , c ) 2 = ∥ c ∥ 2 2 c ∗ L c − ∣ μ ∣ 2 . For normal T T T μ \mu μ spec ( T ) \operatorname{spec}(T) spec ( T ) [64 ] . As in residual DMD [18 ] , one may retain only the computed eigenpairs whose right-hand side is below a prescribed tolerance; in contrast with the single-transition residuals of Remark 3.3, res ^ \widehat{\operatorname{res}} res spec ( T τ ) ⊂ [ 0 , 1 ] \operatorname{spec}(T_\tau)\subset[0,1] spec ( T τ ) ⊂ [ 0 , 1 ] L \mathcal{L} L
Forecast bounds
The norm of the computed Galerkin operator satisfies ∥ K ^ ∥ ≤ κ ( η V ) ∥ K ^ ∥ 2 \lVert\widehat{K}\rVert\leq\kappa(\eta_V)\lVert\widehat{K}\rVert_2 ∥ K ∥ ≤ κ ( η V ) ∥ K ∥ 2 ∥ v ∥ n = ∥ c ∥ 2 \lVert v\rVert_n=\lVert c\rVert_2 ∥ v ∥ n = ∥ c ∥ 2 v = ∑ j c j ψ ^ j v=\sum_j c_j\widehat{\psi}_j v = ∑ j c j ψ j (13) also applies with the compressed operator T τ o b s T_\tau^{\mathrm{obs}} T τ obs § 13 in place of T T T
Suppose that ρ \rho ρ T T T T 1 = 1 T1=1 T 1 = 1 1 ∈ V 1\in V 1 ∈ V V 0 : = { v ∈ V : ⟨ 1 , v ⟩ = 0 } V_0:=\{v\in V:\langle1,v\rangle=0\} V 0 := { v ∈ V : ⟨ 1 , v ⟩ = 0 } f 0 : = f − ⟨ 1 , f ⟩ 1 f_0:=f-\langle1,f\rangle1 f 0 := f − ⟨ 1 , f ⟩ 1 f f f
Corollary D.4 (Forecast bound uniform in the horizon). In the setting of Theorem 4.5, with ρ \rho ρ 1 ∈ V 1\in V 1 ∈ V K ^ 1 = 1 \widehat{K}1=1 K 1 = 1 K ^ V 0 ⊂ V 0 \widehat{K}V_0\subset V_0 K V 0 ⊂ V 0 ∥ K ^ k ∣ V 0 ∥ ≤ C K r K k \lVert\widehat{K}^k|_{V_0}\rVert\leq C_K r_K^k ∥ K k ∣ V 0 ∥ ≤ C K r K k k ≥ 0 k\geq0 k ≥ 0 C K ≥ 1 C_K\geq1 C K ≥ 1 0 ≤ r K < 1 0\leq r_K<1 0 ≤ r K < 1 N ≥ 1 N\geq1 N ≥ 1
∥ T N f − f ( N ) ∥ ≤ C K ( ∥ ( I − P V ) f 0 ∥ 1 − η V + e ( V ) ∥ P V f 0 ∥ 1 − r K ) . (38) \lVert T^Nf-f^{(N)}\rVert\leq C_K\left(\frac{\lVert(I-P_V)f_0\rVert}{1-\eta_V}+\frac{e(V)\lVert P_Vf_0\rVert}{1-r_K}\right). \tag*{(38)} ∥ T N f − f ( N ) ∥ ≤ C K ( 1 − η V ∥( I − P V ) f 0 ∥ + 1 − r K e ( V ) ∥ P V f 0 ∥ ) . (38) Proof. Write P : = P V P:=P_V P := P V P ^ : = P ^ V \widehat{P}:=\widehat{P}_V P := P V T 1 = 1 T1=1 T 1 = 1 K ^ 1 = 1 \widehat{K}1=1 K 1 = 1 P 1 = P ^ 1 = 1 P1=\widehat{P}1=1 P 1 = P 1 = 1
T N f − f ( N ) = T N ( I − P ) f 0 + ∑ k = 0 N − 1 T N − 1 − k ( T − K ^ ) K ^ k P f 0 − K ^ N ( P ^ − P ) f 0 . T^Nf-f^{(N)}=T^N(I-P)f_0+\sum_{k=0}^{N-1}T^{N-1-k}(T-\widehat{K})\widehat{K}^kPf_0-\widehat{K}^N(\widehat{P}-P)f_0. T N f − f ( N ) = T N ( I − P ) f 0 + k = 0 ∑ N − 1 T N − 1 − k ( T − K ) K k P f 0 − K N ( P − P ) f 0 . Because 1 ∈ V 1\in V 1 ∈ V P f 0 ∈ V 0 Pf_0\in V_0 P f 0 ∈ V 0 K ^ k P f 0 ∈ V 0 \widehat{K}^kPf_0\in V_0 K k P f 0 ∈ V 0 C K r K k ∥ P f 0 ∥ C_Kr_K^k\lVert Pf_0\rVert C K r K k ∥ P f 0 ∥ ( T − K ^ ) K ^ k P f 0 (T-\widehat{K})\widehat{K}^kPf_0 ( T − K ) K k P f 0 e ( V ) C K r K k ∥ P f 0 ∥ e(V)C_Kr_K^k\lVert Pf_0\rVert e ( V ) C K r K k ∥ P f 0 ∥ (8) . Since V = span { 1 } ⊕ V 0 V=\operatorname{span}\{1\}\oplus V_0 V = span { 1 } ⊕ V 0 K ^ \widehat{K} K ∥ K ^ N ∥ ≤ C K \lVert\widehat{K}^N\rVert\leq C_K ∥ K N ∥ ≤ C K C K η V ( 1 − η V ) − 1 ∥ ( I − P ) f 0 ∥ C_K\eta_V(1-\eta_V)^{-1}\lVert(I-P)f_0\rVert C K η V ( 1 − η V ) − 1 ∥( I − P ) f 0 ∥ T T T ∥ ( I − P ) f 0 ∥ \lVert(I-P)f_0\rVert ∥( I − P ) f 0 ∥ C K e ( V ) ( 1 − r K ) − 1 ∥ P f 0 ∥ C_Ke(V)(1-r_K)^{-1}\lVert Pf_0\rVert C K e ( V ) ( 1 − r K ) − 1 ∥ P f 0 ∥ 1 + C K η V / ( 1 − η V ) ≤ C K / ( 1 − η V ) 1+C_K\eta_V/(1-\eta_V)\leq C_K/(1-\eta_V) 1 + C K η V / ( 1 − η V ) ≤ C K / ( 1 − η V )
The decay condition holds, with a suitable C K C_K C K r K r_K r K K ^ \widehat{K} K V 0 V_0 V 0 K ^ 1 = 1 \widehat{K}1=1 K 1 = 1 1 ∈ V 1 \in V 1 ∈ V T ^ 1 = 1 \widehat{T}1=1 T 1 = 1 K ^ V 0 ⊂ V 0 \widehat{K}V_0 \subset V_0 K V 0 ⊂ V 0 P V T ∣ V P_VT|_V P V T ∣ V K ^ \widehat{K} K ρ \rho ρ V 0 , n : = { v ∈ V : ⟨ 1 , v ⟩ n = 0 } V_{0,n}:=\{v\in V:\langle1,v\rangle_n=0\} V 0 , n := { v ∈ V : ⟨ 1 , v ⟩ n = 0 } ψ ^ 1 = 1 \widehat{\psi}_1=1 ψ 1 = 1 K ^ 1 = 1 \widehat{K}1=1 K 1 = 1 K ^ \widehat{K} K V 0 , n V_{0,n} V 0 , n V 0 V_0 V 0 C K C_K C K 2 κ ( η V ) C K 2\kappa(\eta_V)C_K 2 κ ( η V ) C K v ∈ V v\in V v ∈ V V 0 , n V_{0,n} V 0 , n κ ( η V ) ∥ v ∥ \kappa(\eta_V)\lVert v\rVert κ ( η V ) ∥ v ∥ ( T − K ^ ) 1 = 0 (T-\widehat{K})1=0 ( T − K ) 1 = 0 K ^ k P V f 0 \widehat{K}^kP_Vf_0 K k P V f 0 L 2 ( ρ ) L^2(\rho) L 2 ( ρ ) K ^ \widehat{K} K κ ( η V ) \kappa(\eta_V) κ ( η V )
Remark D.5. A modification K ^ ↦ K ^ ′ \widehat{K}\mapsto\widehat{K}' K ↦ K ′ (8) , (11) , and the forecast bounds through the additional term ∥ K ^ ′ − K ^ ∥ \lVert\widehat{K}'-\widehat{K}\rVert ∥ K ′ − K ∥ ∥ K ^ ∥ \lVert\widehat{K}\rVert ∥ K ∥ ∥ K ^ ′ ∥ \lVert\widehat{K}'\rVert ∥ K ′ ∥ M M M K ^ \widehat{K} K K ^ \widehat{K} K κ ( η V ) ∥ K ^ − K ^ T ∥ 2 / 2 \kappa(\eta_V)\lVert\widehat{K}-\widehat{K}^{\mathsf T}\rVert_2/2 κ ( η V ) ∥ K − K T ∥ 2 /2 Δ ( V ) \Delta(V) Δ ( V ) η V \eta_V η V T T T L 2 ( ρ ) L^2(\rho) L 2 ( ρ ) ( I − P ^ V ) f (I-\widehat{P}_V)f ( I − P V ) f U N U_N U N N M N − 1 e ( V ) ∥ P ^ V f ∥ + ∥ ( T N − U N ) ( I − P ^ V ) f ∥ NM^{N-1}e(V)\lVert\widehat{P}_Vf\rVert+\lVert(T^N-U_N)(I-\widehat{P}_V)f\rVert N M N − 1 e ( V ) ∥ P V f ∥ + ∥( T N − U N ) ( I − P V ) f ∥
For measure-preserving deterministic dynamics, T T T K ^ \widehat{K} K N → ∞ N\to\infty N → ∞ ∥ f 0 ∥ \lVert f_0\rVert ∥ f 0 ∥ f f f Figure 3 are consistent with this.
Partial observations
Let y t : = h ( X t ) y_t:=h(X_t) y t := h ( X t ) H Y ⊂ H \mathcal{H}_Y\subset\mathcal{H} H Y ⊂ H f ∘ h f\circ h f ∘ h X t ∼ ρ X_t\sim\rho X t ∼ ρ φ ∈ H Y \varphi\in\mathcal{H}_Y φ ∈ H Y
E [ φ ( X t + τ ) ∣ y t ] = ( P H Y T τ φ ) ( X t ) . \mathbb{E}[\varphi(X_{t+\tau})\mid y_t]=(P_{\mathcal{H}_Y}T_\tau\varphi)(X_t). E [ φ ( X t + τ ) ∣ y t ] = ( P H Y T τ φ ) ( X t ) . Thus partially observed lagged data identify the compressed operator
T τ o b s : = P H Y T τ ∣ H Y . T_\tau^{\mathrm{obs}}:=P_{\mathcal{H}_Y}T_\tau|_{\mathcal{H}_Y}. T τ obs := P H Y T τ ∣ H Y . If a T τ T_\tau T τ E E E H Y \mathcal{H}_Y H Y T τ o b s ∣ E = T τ ∣ E T_\tau^{\mathrm{obs}}|_E=T_\tau|_E T τ obs ∣ E = T τ ∣ E φ ∈ H Y \varphi\in\mathcal{H}_Y φ ∈ H Y μ ∈ C \mu\in\mathbb{C} μ ∈ C
∥ T τ φ − μ φ ∥ 2 = ∥ T τ o b s φ − μ φ ∥ 2 + ∥ ( I − P H Y ) T τ φ ∥ 2 . \lVert T_\tau\varphi-\mu\varphi\rVert^2=\lVert T_\tau^{\mathrm{obs}}\varphi-\mu\varphi\rVert^2+\lVert(I-P_{\mathcal{H}_Y})T_\tau\varphi\rVert^2. ∥ T τ φ − μ φ ∥ 2 = ∥ T τ obs φ − μ φ ∥ 2 + ∥( I − P H Y ) T τ φ ∥ 2 . The powers of T τ o b s T_\tau^{\mathrm{obs}} T τ obs τ \tau τ [2 , 21 , 23 , 27 , 63 ] .
A map Φ ^ : Y → Y \widehat{\Phi}:Y\to Y Φ : Y → Y ( y t , y t + τ ) (y_t,y_{t+\tau}) ( y t , y t + τ ) y ↦ E [ y t + τ ∣ y t = y ] y\mapsto\mathbb{E}[y_{t+\tau}\mid y_t=y] y ↦ E [ y t + τ ∣ y t = y ] Φ ^ ∘ h \widehat{\Phi}\circ h Φ ∘ h T τ o b s T_\tau^{\mathrm{obs}} T τ obs h h h f f f Y Y Y f ( E [ y t + τ ∣ y t ] ) ≠ E [ f ( y t + τ ) ∣ y t ] f(\mathbb{E}[y_{t+\tau}\mid y_t]) \ne\mathbb{E}[f(y_{t+\tau})\mid y_t] f ( E [ y t + τ ∣ y t ]) = E [ f ( y t + τ ) ∣ y t ] f ∘ Φ ^ ∘ h f \circ\widehat{\Phi} \circ h f ∘ Φ ∘ h T τ o b s ( f ∘ h ) T_{\tau}^{\mathrm{obs}}(f \circ h) T τ obs ( f ∘ h ) T τ o b s ( f ∘ h ) T_{\tau}^{\mathrm{obs}}(f \circ h) T τ obs ( f ∘ h ) y t + τ y_{t+\tau} y t + τ y t y_t y t f ( y t + τ ) f(y_{t+\tau}) f ( y t + τ ) y t y_t y t
Appendix F. Learning error for composition estimators and the FNO setting
Composition estimators. Let T ^ f = f ∘ Φ ^ \widehat{T}f=f\circ\widehat{\Phi} T f = f ∘ Φ dist Ω \operatorname{dist}_{\Omega} dist Ω Ω = Y \Omega=Y Ω = Y Ω 0 ⊂ Ω \Omega_0\subset\Omega Ω 0 ⊂ Ω Φ ( y ) \Phi(y) Φ ( y ) Φ ^ ( y ) \widehat{\Phi}(y) Φ ( y ) ρ \rho ρ y y y Lip ( ⋅ ) \operatorname{Lip}(\cdot) Lip ( ⋅ ) Ω 0 \Omega_0 Ω 0 dist Ω \operatorname{dist}_{\Omega} dist Ω m m m V ⊂ F V\subset\mathcal{F} V ⊂ F ρ \rho ρ χ \chi χ
Lip ( V ) : = Lip ( χ ) , χ : y ↦ ( χ 1 ( y ) , … , χ m ( y ) ) T ∈ R m . \operatorname{Lip}(V):=\operatorname{Lip}(\chi),\qquad\chi:y\mapsto\left(\chi_1(y),\ldots,\chi_m(y)\right)^{\mathsf{T}}\in\mathbb{R}^{m}. Lip ( V ) := Lip ( χ ) , χ : y ↦ ( χ 1 ( y ) , … , χ m ( y ) ) T ∈ R m . This quantity does not depend on the choice of χ \chi χ Ω 0 \Omega_0 Ω 0 v ∈ V v\in V v ∈ V ∥ v ∥ = 1 \lVert v\rVert=1 ∥ v ∥ = 1 ∣ v ( Φ ^ ( y ) ) − v ( Φ ( y ) ) ∣ ≤ Lip ( V ) ∥ v ∥ dist Ω ( Φ ^ ( y ) , Φ ( y ) ) \lvert v(\widehat{\Phi}(y))-v(\Phi(y))\rvert\le\operatorname{Lip}(V)\lVert v\rVert\operatorname{dist}_{\Omega}(\widehat{\Phi}(y),\Phi(y)) ∣ v ( Φ ( y )) − v ( Φ ( y ))∣ ≤ Lip ( V ) ∥ v ∥ dist Ω ( Φ ( y ) , Φ ( y )) v ∈ V v\in V v ∈ V L 2 ( ρ ) L^2(\rho) L 2 ( ρ ) y t + τ y_{t+\tau} y t + τ y t y_t y t f ∘ Φ ^ f\circ\widehat{\Phi} f ∘ Φ f f f Ω 0 \Omega_0 Ω 0 ρ \rho ρ y y y [66 ] gives (4.10) with dist Ω ( Φ ^ , Φ ) \operatorname{dist}_{\Omega}(\widehat{\Phi},\Phi) dist Ω ( Φ , Φ )
FNO setting. For Corollary 4.6 , take
Ω = L 2 ( T d x ; R d o u t ) , B a = { u ∈ H a ( T d x ; R d o u t ) : ∥ u ∥ H a ≤ c a } , a > d x / 2. \Omega=L^2(\mathbb{T}^{d_x};\mathbb{R}^{d_{\mathrm{out}}}),\qquad\mathcal{B}_a=\left\{u\in H^a(\mathbb{T}^{d_x};\mathbb{R}^{d_{\mathrm{out}}}):\lVert u\rVert_{H^a}\le c_a\right\},\qquad a>d_x/2. Ω = L 2 ( T d x ; R d out ) , B a = { u ∈ H a ( T d x ; R d out ) : ∥ u ∥ H a ≤ c a } , a > d x /2. for a fixed radius c a > 0 c_a>0 c a > 0 d o u t d_{\mathrm{out}} d out L 2 L^2 L 2 dist Ω \operatorname{dist}_{\Omega} dist Ω ∥ dist Ω ( Φ ^ , Φ ) ∥ L 2 ( ρ ) 2 = ∫ ∥ Φ ^ ( u ) − Φ ( u ) ∥ L 2 2 d ρ ( u ) \lVert\operatorname{dist}_{\Omega}(\widehat{\Phi},\Phi)\rVert_{L^2(\rho)}^2=\int\lVert\widehat{\Phi}(u)-\Phi(u)\rVert_{L^2}^2\,\mathrm{d}\rho(u) ∥ dist Ω ( Φ , Φ ) ∥ L 2 ( ρ ) 2 = ∫ ∥ Φ ( u ) − Φ ( u ) ∥ L 2 2 d ρ ( u ) [11 Theorem 3.6] . Assume that Φ \Phi Φ S p o l y \mathcal{S}^{\mathrm{poly}} S poly [11 ] , that is, it satisfies [11 Condition 3.3] on B a \mathcal{B}_a B a supp ρ ⊂ B a \operatorname{supp}\rho\subset\mathcal{B}_a supp ρ ⊂ B a [11 Theorem 3.6] applies to the corresponding FNO class. The estimator Φ ^ \widehat{\Phi} Φ ( u i , Φ ( u i ) ) (u_i,\Phi(u_i)) ( u i , Φ ( u i )) u i u_i u i ρ \rho ρ T f = f ∘ Φ ^ T_f=f\circ\widehat{\Phi} T f = f ∘ Φ L 2 ( ρ ) L^2(\rho) L 2 ( ρ ) B a \mathcal{B}_a B a ρ \rho ρ Φ \Phi Φ ρ \rho ρ Φ \Phi Φ ρ \rho ρ Ω 0 ⊂ Ω \Omega_0\subset\Omega Ω 0 ⊂ Ω Φ ( supp ρ ) \Phi(\operatorname{supp}\rho) Φ ( supp ρ ) supp ρ \operatorname{supp}\rho supp ρ Lip ( V θ ) < ∞ \operatorname{Lip}(V_\theta)<\infty Lip ( V θ ) < ∞ Φ ^ \widehat{\Phi} Φ
Remark F.1 (Choice of the level). A larger level of the candidate family is not always better, because the learning and sampling errors in Theorem 4.2 must be controlled uniformly over richer families, and a sufficiently rich family can make J ^ \widehat{J} J V V V Δ ( V θ ) ≤ ( m ( 1 + η V θ ) ) 1 / 2 max j ∥ T ^ φ ^ j − T φ ^ j ∥ \Delta(V_\theta)\le\left(m(1+\eta_{V_\theta})\right)^{1/2}\max_j\lVert\widehat{T}\widehat{\varphi}_j-T\widehat{\varphi}_j\rVert Δ ( V θ ) ≤ ( m ( 1 + η V θ ) ) 1/2 max j ∥ T φ j − T φ j ∥ V θ V_\theta V θ Δ θ \Delta_\theta Δ θ T ^ \widehat{T} T G ϕ θ G_{\phi_\theta} G ϕ θ ϕ θ \phi_\theta ϕ θ L 2 ( ρ ) L^2(\rho) L 2 ( ρ ) Lip ( V θ ) ≤ Lip ( ϕ θ ) λ min ( G ϕ θ ) − 1 / 2 \operatorname{Lip}(V_\theta) \le\operatorname{Lip}(\phi_\theta)\lambda_{\min}(G_{\phi_\theta})^{-1/2} Lip ( V θ ) ≤ Lip ( ϕ θ ) λ m i n ( G ϕ θ ) − 1/2 p p p Ω 0 \Omega_0 Ω 0 ρ \rho ρ Lip ( V θ ) \operatorname{Lip}(V_\theta) Lip ( V θ ) p 3 p^3 p 3 [8 ] . In the setting of Corollary 4.6 , Δ Θ p \Delta_{\Theta_p} Δ Θ p p = p ( n o p ) p=p(n_{\mathrm{op}}) p = p ( n op ) Lip Θ p n o p − 1 / ( 12 + 4 d x / a ) → 0 \operatorname{Lip}_{\Theta_p}n_{\mathrm{op}}^{-1/(12+4d_x/a)} \to0 Lip Θ p n op − 1/ ( 12 + 4 d x / a ) → 0 n o p → ∞ n_{\mathrm{op}}\to\infty n op → ∞
A posteriori estimate. For a general approximation, the learning error can be estimated from held-out data. Conditionally on T ^ \widehat{T} T X ∼ ρ X\sim\rho X ∼ ρ X + X^+ X + f ∈ F ~ f\in\widetilde{\mathcal{F}} f ∈ F E ∣ f ( X + ) ∣ 2 < ∞ \mathbb{E}|f(X^+)|^2<\infty E ∣ f ( X + ) ∣ 2 < ∞
E ∣ ( T ^ f ) ( X ) − f ( X + ) ∣ 2 = ∥ ( T ^ − T ) f ∥ 2 + E ∣ f ( X + ) − ( T f ) ( X ) ∣ 2 . \mathbb{E}|(\widehat{T}f)(X)-f(X^+)|^2=\|(\widehat{T}-T)f\|^2+\mathbb{E}|f(X^+)-(Tf)(X)|^2. E ∣ ( T f ) ( X ) − f ( X + ) ∣ 2 = ∥ ( T − T ) f ∥ 2 + E ∣ f ( X + ) − ( T f ) ( X ) ∣ 2 . Summed over the computational basis Ψ ^ \widehat{\Psi} Ψ V V V T ^ \widehat{T} T ∑ j ∥ ( T ^ − T ) ψ ^ j ∥ 2 + ∑ j E ∣ ψ ^ j ( X + ) − ( T ψ ^ j ) ( X ) ∣ 2 \sum_j\|(\widehat{T}-T)\widehat{\psi}_j\|^2+\sum_j\mathbb{E}|\widehat{\psi}_j(X^+)-(T\widehat{\psi}_j)(X)|^2 ∑ j ∥ ( T − T ) ψ j ∥ 2 + ∑ j E ∣ ψ j ( X + ) − ( T ψ j ) ( X ) ∣ 2 Δ ( V ) 2 / ( 1 + η V ) \Delta(V)^2/(1+\eta_V) Δ ( V ) 2 / ( 1 + η V ) Δ ( V ) 2 \Delta(V)^2 Δ ( V ) 2
Appendix G. Implementation details of the numerical experiments
Ornstein–Uhlenbeck process. The backward equation is discretized on 128 points in [ − 5 , 5 ] [-5,5] [ − 5 , 5 ] τ = 0.15 \tau=0.15 τ = 0.15
Triple-well diffusion. The benchmark follows [71 ] ; the potential is the three-hole potential of [45 ] ,
U ( x , y ) : = 3 e − x 2 − ( y − 1 / 3 ) 2 − 3 e − x 2 − ( y − 5 / 3 ) 2 − 5 e − ( x − 1 ) 2 − y 2 − 5 e − ( x + 1 ) 2 − y 2 + 0.2 x 4 + 0.2 ( y − 1 / 3 ) 4 . U(x,y):=3e^{-x^2-(y-1/3)^2}-3e^{-x^2-(y-5/3)^2}-5e^{-(x-1)^2-y^2}-5e^{-(x+1)^2-y^2}+0.2x^4+0.2(y-1/3)^4. U ( x , y ) := 3 e − x 2 − ( y − 1/3 ) 2 − 3 e − x 2 − ( y − 5/3 ) 2 − 5 e − ( x − 1 ) 2 − y 2 − 5 e − ( x + 1 ) 2 − y 2 + 0.2 x 4 + 0.2 ( y − 1/3 ) 4 . The scalar fields are represented on a 64 × 64 64\times64 64 × 64 [ − 2 , 2 ] × [ − 1 , 2 ] [-2,2]\times[-1,2] [ − 2 , 2 ] × [ − 1 , 2 ] 16 × 16 16\times16 16 × 16 128 × 128 128\times128 128 × 128
Lorenz systems. The first 12000 sampled times are used for fitting and the last 3000 for evaluation. Lorenz–63 uses τ = 0.1 \tau=0.1 τ = 0.1 τ = 0.02 \tau=0.02 τ = 0.02
Sea-surface temperature. The 11 input fields are listed in Table 2 . The ERA5 fields are daily means of the 6-hourly data on the WeatherBench 2 product conservatively regridded to the 32 × 64 grid. The wind stress is computed at each 6-hourly step by the bulk formula ρ a C D ∣ u 10 ∣ u 10 \rho_a C_D \lvert\mathbf{u}_{10} \rvert\mathbf{u}_{10} ρ a C D ∣ u 10 ∣ u 10 u 10 \mathbf{u}_{10} u 10 ρ a = 1.225 \rho_a = 1.225 ρ a = 1.225 − 3 ^{-3} − 3 C D = 1.3 × 10 − 3 C_D = 1.3 \times10^{-3} C D = 1.3 × 1 0 − 3 − 1 ^{-1} − 1 − 1 ^{-1} − 1
SST is centered by a training-only leap-aligned calendar-day climatology, and each auxiliary field is deseasonalized at every grid point by regressing on a constant and six annual harmonics (13 regression columns); the harmonics are not appended to the state. The reduction is channelwise. Cosine-latitude-weighted empirical orthogonal functions (EOFs) of the standardized anomalies are computed on the fitting window only, SST anomalies are represented by 64 EOF coefficients scaled by 0.5 \sqrt{0.5} 0.5 0.05 \sqrt{0.05} 0.05 64 + 4 × 10 = 104 64 + 4 \times10 = 104 64 + 4 × 10 = 104 z ∈ R 96 \mathbf{z} \in\mathbb{R}^{96} z ∈ R 96
z t a r g e t = 0.6 z o b s e r v e d + 0.4 z F N O , \mathbf{z}_{\mathrm{target}} = 0.6\mathbf{z}_{\mathrm{observed}} + 0.4\mathbf{z}_{\mathrm{FNO}}, z target = 0.6 z observed + 0.4 z FNO , where z o b s e r v e d \mathbf{z}_{\mathrm{observed}} z observed z F N O \mathbf{z}_{\mathrm{FNO}} z FNO 10 − 2 10^{-2} 1 0 − 2 δ 0 \delta_0 δ 0 c r e s ξ 0 ℓ c_{\mathrm{res}\xi_0}^{\ell} c res ξ 0 ℓ ℓ \ell ℓ c r e s = 0.905 c_{\mathrm{res}} = 0.905 c res = 0.905
The FNO and SFNO use 11 channels, four layers, width 96, spectral modes (12, 24), and are trained for 180 epochs (no early stopping) with AdamW, initial learning rate 5 × 10 − 4 5 \times10^{-4} 5 × 1 0 − 4 10 − 5 10^{-5} 1 0 − 5 L 2 L^2 L 2
EEG. The first 60 seconds of 64-channel EEG sampled at 160 Hz are common-average referenced, causally bandpass filtered to 8–30 Hz, and standardized using statistics from 1 to 15 seconds, after a one-second filter warm-up. The state is a 64 × 160 64 \times160 64 × 160 τ = 0.1 \tau= 0.1 τ = 0.1 25 × 25 25 \times25 25 × 25
