Introduction
Data-driven inference of governing equations, from equation discovery with SINDy [1 ] and WSINDy [2 ] to parameter estimation, has become a central tool across the quantitative sciences [1 , 3 , 4 , 5 ] . For deterministic ordinary differential equations (ODEs), weak-form estimation has proven especially powerful [2 , 6 , 7 , 8 ] . Rather than estimating derivatives pointwise from noisy data, the weak-form approach relies on multiplying the governing equation by a smooth, compactly supported test function and integrating by parts, thus converting differentiation of the data into differentiation of the (known) smooth test function 1 . The weak-form estimation of nonlinear dynamics (WENDy) framework [8 , 9 ] exploits this to recover model parameters by linear regression, resulting in inference that is ultra-fast and robust to additive and multiplicative measurement noise.
The extension of the WENDy framework to stochastic differential equations (SDEs) has, to our knowledge, not previously been attempted. Many systems of scientific interest are not merely observed with noise but driven by it. Ion channels in neurons, climate oscillators, ecological populations near extinction, and molecular reaction networks are among the examples in which randomness enters into the dynamics themselves, and not just into the measurement process. For such systems, SDEs and stochastic partial differential equations (SPDEs) are more faithful models than ODEs and PDEs. Recovering the drift and diffusion of an SDE from sampled trajectories is the stochastic analogue of WENDy’s goal and the objective of this paper. SPDEs are left for future work.
Inference for discretely observed diffusions is difficult because the transition density of the SDE is almost never available in closed form, so the exact likelihood of a discretely sampled path cannot be evaluated directly (see, e.g., [10 ] and the references therein). A substantial body of work addresses this challenge. Some approaches use simulated-likelihood or data-augmentation schemes to account for the unobserved portions of the trajectory between observations [11 , 12 ] . These methods, however, can be computationally demanding, as they require repeated simulation or sampling of the latent paths, and data-augmentation schemes can suffer from poor MCMC mixing due to strong dependence between the latent paths and the diffusion parameters. Other approaches construct closed-form approximations to the transition density [13 ] . Although these expansions can be highly accurate, practical implementations rely on finite truncations, whose accuracy and computational cost depend on the order of the expansion and the complexity of the underlying model. Finally, exact-simulation methods eliminate discretization error entirely [14 , 15 ] . However, they typically require first transforming the SDE to one with unit diffusion coefficient (the Lamperti transform). Because this transformation generally exists only for scalar diffusions, and for a restricted class of multivariate ones, these methods do not extend to general multivariate systems. Despite their different approaches, these methods are generally developed for settings in which the state is observed exactly at discrete times, without additional measurement noise.
When the state is observed with measurement error, the unknown parameters must be estimated within a state-space formulation that couples them to a nonlinear filtering problem. This is handled through sequential Monte Carlo [16 , 17 , 18 ] , particle MCMC [19 ] , iterated filtering [20 ] , and ensemble Kalman–Bucy schemes [21 ] , jointly inferring the drift and diffusion parameters together with the latent path and yielding consistent, and in some cases unbiased, estimators even in high-dimensional or partially observed regimes. However, the performance varies with the method and the model class; for example, ensemble Kalman–Bucy schemes are exact only for linear Gaussian dynamics and become approximations without convergence guarantees once the drift or diffusion is nonlinear. A parallel line of work instead parameterizes the drift and diffusion by neural networks [22 , 23 , 24 ] ; such models are flexible and accommodate irregular sampling, but they require large training sets and are not directly interpretable as governing equations. In all but trivial cases, these methods are also highly computationally intensive.
Within the governing equation discovery setting, sparse-regression and related methods have been adapted to learn the form of stochastic dynamics (and consequently the associated parameters), typically by fitting drift and diffusion fields to empirical estimates of conditional moments or to Fokker–Planck statistics [25 , 26 , 27 ] . Weak-form ideas have recently entered this setting as well, for mean-field equations estimated from interacting-particle data [28 ] and for mean-field Fokker–Planck equations learned from sparse ecological experiments [29 ] . A weak-collocation approach integrates the Fokker–Planck equation against Gaussian kernels placed in state space and evaluates the resulting integrals as sums over observed samples. This has been done for aggregate snapshot data [30 ] and, through ergodic averaging, for a single stationary trajectory split into segments [31 ] . In all of these weak-form methods, the weak form acts on the density of the process, so the stochastic-integral term is removed by the averaging over samples before any regression is performed. In exchange, the temporal structure of the increments is not used. The strong-form approaches, in turn, rely on finite differencing of the sampled path, which amplifies measurement noise and makes parameter inference more challenging. Many of these approaches are also based on empirical estimates of conditional moments or densities, obtained from single-step increments, by binning, or by smoothing. Binning generally discards the within-bin dynamics, and the recovered parameters show bias that grows with the noise level and with coarser sampling [32 , 33 , 34 ] . In contrast, WAIVES retains the weak form in the time variable, acting on a single trajectory. This preserves both the temporal information in the increments and the computational efficiency and speed of compactly supported test functions, the same properties behind the weak form’s success for deterministic systems. However, this formulation introduces a difficulty of its own, which we address next.
Applied naively to SDEs, weak-form regression in time fails. When the governing SDE is integrated against a test function in time, the resulting regression carries a stochastic-integral error that is correlated with its own regressors, because both are functionals of the same Brownian path over the test-function support. This is the endogeneity problem of classical regression [35 ] , and it makes the weak-form least-squares estimator biased and inconsistent, regardless of how many test functions are used or how long the trajectory grows. In fact, as we show in § 3 , for the Ornstein–Uhlenbeck process the least-squares estimate converges to zero regardless of the record length.
In this work, we propose to address the endogeneity issue directly, using instrumental variables (IV) [35 , 36 ] . IVs are not new to system identification, where they have long been used to obtain consistent estimates of dynamical models from noisy data [37 , 38 , 39 ] , with recent extensions to nonlinear continuous-time models through auxiliary-model instruments [40 ] or sample-splitting of the record [41 ] . Both of these, however, address deterministic output-error models, where the only disturbance is measurement noise, and thus do not carry over to SDEs. Once the driving noise enters the dynamics itself, no noise-free auxiliary trajectory and no subsample independent of the Brownian path exist. Instrumental variable ideas have also appeared against measurement noise alone. For example, in strong-form stochastic SINDy, whose single-step regression is consistent in the absence of measurement noise, the drift term candidates evaluated one time step earlier can serve as instruments against that noise [27 ] . That construction, however, does not address the endogeneity of the weak form, which is present even for perfectly observed paths, as we show in the next section. The instruments we construct below do.
In principle, an instrument can be any quantity that is correlated with the regressor, but uncorrelated with the error. The difficulty lies in finding an instrument for which both properties hold strongly. We divide the record into short sub-intervals (windows) and construct instruments from weak-form integrals over nearby windows disjoint from the target window on which the regressors are built. Because Brownian increments over disjoint intervals are mutually independent, a weak-form integral computed on one window (against a test function supported on that window) is uncorrelated with the stochastic-integral error of a non-overlapping window, while remaining correlated with the regressors whenever the two windows lie within the memory of the process. The resulting estimator, which we call WAIVES (Weak-form Asymptotically-unbiased Instrumental-Variable Estimation for SDEs), is shown to be consistent, asymptotically unbiased (with the bias vanishing in the joint limit of many windows and fine sampling), asymptotically normal, and it requires neither a path likelihood nor a Radon–Nikodym derivative. WAIVES thus belongs to the moment-based tradition for diffusions, alongside martingale estimating functions [42 ] and generator-based generalized method of moments (GMM) [43 ] , which derive their moment conditions from the model’s transition or generator structure and presuppose exactly observed states. WAIVES builds them from the test functions and the noisy observations alone.
Because it avoids the finite-difference estimates on which strong-form methods rely, it inherits the weak-form noise robustness and solver-free efficiency, while still handling diffusions that are observed only at discrete times, with noise, and possibly through a linear sensor map. We demonstrate this on the nine example systems of Table 1 , from scalar diffusions to a 120-dimensional Lorenz-96 model, and on two real data sets, the 167-year Atlantic Multidecadal Oscillation index (a North Atlantic sea-surface-temperature record) and an optical-tweezer calibration record.
Table 1. The nine example systems, each chosen to probe one challenge for a weak-form SDE estimator. For each system the last column reports the mean,(std) over M = 1000 M=1000 M = 1000 replications of one representative drift parameter (the one most sensitive to the listed challenge), and, below it, the diffusion estimate for the same component. Diffusion entries use the lag-one quadratic-variation form, except the slope form for the Langevin at γ = 0.5 \gamma=0.5 γ = 0.5 . The Cox–Ingersoll–Ross entry lists the drift rate − κ ^ -\widehat{\kappa} − κ above ν ^ \widehat{\nu} ν , and the FitzHugh–Nagumo ratio parameter a ^ \widehat{a} a is given as its mode with a 95% Fieller interval. Full drift and diffusion estimates for all nine systems appear in Table 2 and Table 3 . Although each row is designed to stress a different failure mode, the same estimator under the same default rules recovers every reported parameter with negligible bias.
The weak form for SDEs
Model
We consider the state-space model in which a latent process obeys an Itô SDE and is observed discretely
under additive Gaussian error,
State: d X t = f ( X t , θ ) d t + σ ( X t , ν ) d W t , Obs.: Y t n = X t n + η n , η n ∼ i i d N ( 0 , R ) , \begin{aligned}
\text{State:}\quad&\mathrm{d}X_t = f(X_t,\theta)\,\mathrm{d}t + \sigma(X_t,\nu)\,\mathrm{d}W_t,\\
\text{Obs.:}\quad&Y_{t_n} = X_{t_n} + \eta_n,\qquad\eta_n \overset{\mathrm{iid}}{\sim} \mathcal{N}(0,R),
\end{aligned} State: Obs.: d X t = f ( X t , θ ) d t + σ ( X t , ν ) d W t , Y t n = X t n + η n , η n ∼ iid N ( 0 , R ) ,
where { W t } \{W_t\} { W t } is a standard Brownian motion in R d \mathbb{R}^d R d and the state X t ∈ R d X_t \in\mathbb{R}^d X t ∈ R d . The observations are taken at the discrete instants t n = n Δ t t_n = n\Delta t t n = n Δ t for n = 0 , … , N n = 0,\ldots,N n = 0 , … , N , so that the record spans [ 0 , T ] [0,T] [ 0 , T ] with T = N Δ t T = N\Delta t T = N Δ t . The observation errors η n ∈ R d y \eta_n \in\mathbb{R}^{d_y} η n ∈ R d y have covariance R ∈ R d y × d y R \in\mathbb{R}^{d_y \times d_y} R ∈ R d y × d y , which is assumed unknown. Throughout we assume d y = d d_y = d d y = d , but Remark 2 treats the more general linear observation model Y t n = C X t n + η n Y_{t_n} = C X_{t_n} + \eta_n Y t n = C X t n + η n with d y ≥ d d_y \ge d d y ≥ d . The errors { η n } \{\eta_n\} { η n } are assumed to be independent of { W t } \{W_t\} { W t } , and the drift is assumed to be linear in the unknown parameters:
f ( x , θ ) = ∑ j = 1 p θ j g j ( x ) , (1) f(x,\theta) = \sum_{j=1}^{p} \theta_j g_j(x), \tag*{(1)} f ( x , θ ) = j = 1 ∑ p θ j g j ( x ) , (1)
with known drift term candidates { g j } j = 1 p \{g_j\}_{j=1}^{p} { g j } j = 1 p such as monomials or trigonometric functions, and the unknown parameter vector θ = ( θ 1 , … , θ p ) T ∈ R p \theta= (\theta_1,\ldots,\theta_p)^{\mathsf{T}} \in\mathbb{R}^p θ = ( θ 1 , … , θ p ) T ∈ R p .
The diffusion is parameterized in the same way. We take σ = σ ( X t , ν ) \sigma= \sigma(X_t,\nu) σ = σ ( X t , ν ) and, under the diagonal-noise assumption used throughout (each state component driven by its own independent Brownian motion), expand the squared diffusion of component i i i in known diffusion term candidates { h l } l = 1 q \{h_l\}_{l=1}^{q} { h l } l = 1 q ,
σ i 2 ( x , ν ) = ∑ l = 1 q ν i l h l ( x ) , (2) \sigma_i^2(x,\nu) = \sum_{l=1}^{q} \nu_{il} h_l(x), \tag*{(2)} σ i 2 ( x , ν ) = l = 1 ∑ q ν i l h l ( x ) , (2)
where ν i = ( ν i 1 , … , ν i q ) T ∈ R q \nu_i = (\nu_{i1},\ldots,\nu_{iq})^{\mathsf{T}} \in\mathbb{R}^q ν i = ( ν i 1 , … , ν i q ) T ∈ R q collects the unknown diffusion parameters of component i i i and ν \nu ν denotes their collection, so that learning the SDE means recovering both θ \theta θ (drift) and ν \nu ν (diffusion). The general non-diagonal case estimates the diffusion matrix D = σ σ T D = \sigma\sigma^{\mathsf{T}} D = σ σ T ; we use the diagonal form for clarity. The estimation of ν \nu ν is developed in § 4 .
For the noise-free ODE x ˙ = f ( x , θ ) \dot{x} = f(x,\theta) x ˙ = f ( x , θ ) , multiplying by a smooth, compactly supported test function of time ϕ k \phi_k ϕ k with supp ϕ k ⊂ ( a k , b k ) \operatorname{supp}\phi_k \subset(a_k,b_k) supp ϕ k ⊂ ( a k , b k ) and integrating by parts moves the derivative onto ϕ k \phi_k ϕ k . We call the support interval ( a k , b k ) (a_k,b_k) ( a k , b k ) , a sub-interval of the record [ 0 , T ] [0,T] [ 0 , T ] , the window of the test function. Because ϕ k \phi_k ϕ k vanishes at the window endpoints, the boundary terms drop and one obtains a linear system b = A θ \mathbf{b} = A\theta b = A θ with
b k = − ∫ a k b k x ( t ) ϕ k ′ ( t ) d t , A k j = ∫ a k b k g j ( x ( t ) ) ϕ k ( t ) d t . (3) b_k = -\int_{a_k}^{b_k} x(t)\phi_k'(t)\,\mathrm{d}t,\qquad A_{kj} = \int_{a_k}^{b_k} g_j(x(t))\phi_k(t)\,\mathrm{d}t. \tag*{(3)} b k = − ∫ a k b k x ( t ) ϕ k ′ ( t ) d t , A k j = ∫ a k b k g j ( x ( t )) ϕ k ( t ) d t . (3)
Because ϕ k \phi_k ϕ k and ϕ k ′ \phi_k' ϕ k ′ vanish outside the window, the same integrals may equally be taken over the whole record [ 0 , T ] [0,T] [ 0 , T ] . Stacking K K K such test functions, each supported on its own window, yields the WENDy regression [8 ] . Its integrals are computed from the observations Y t n Y_{t_n} Y t n by quadrature, so their only error is the measurement noise η n \eta_n η n , which is independent of the latent path x ( t ) x(t) x ( t ) . WENDy accounts for the error covariance this induces, treating b = A θ + ϵ \mathbf{b} = A\boldsymbol{\theta} + \boldsymbol{\epsilon} b = A θ + ϵ as an errors-in-variables regression solved by iteratively reweighted generalized least squares. As the noise is exogenous to the dynamics, this yields a consistent parameter estimator for ODE data with additive measurement noise.
The stochastic weak form
Here and below we abbreviate σ ( X t ) = σ ( X t , ν ) \sigma(X_t) = \sigma(X_t,\nu) σ ( X t ) = σ ( X t , ν ) . Integrating the SDE Eq. (1) against ϕ k \phi_k ϕ k over its support, applying semimartingale integration by parts to ∫ ϕ k d X t \int\phi_k\,\mathrm{d}X_t ∫ ϕ k d X t , and using the vanishing of the boundary terms, we obtain
− ∫ a k b k X t ϕ k ′ ( t ) d t ⏟ b k = ∑ j = 1 p θ j ∫ a k b k ϕ k ( t ) g j ( X t ) d t ⏟ A k j + ∫ a k b k ϕ k ( t ) σ ( X t ) d W t ⏟ ϵ k . (4) -\underbrace{\int_{a_k}^{b_k} X_t\phi_k'(t)\,\mathrm{d}t}_{b_k}
=
\sum_{j=1}^{p}\theta_j\underbrace{\int_{a_k}^{b_k}\phi_k(t)g_j(X_t)\,\mathrm{d}t}_{A_{kj}}
+
\underbrace{\int_{a_k}^{b_k}\phi_k(t)\sigma(X_t)\,\mathrm{d}W_t}_{\epsilon_k}. \tag*{(4)} − b k ∫ a k b k X t ϕ k ′ ( t ) d t = j = 1 ∑ p θ j A k j ∫ a k b k ϕ k ( t ) g j ( X t ) d t + ϵ k ∫ a k b k ϕ k ( t ) σ ( X t ) d W t . (4)
This is the ODE weak form Eq. (3) with one new term, the residual ϵ k = ∫ a k b k ϕ k σ ( X t ) d W t \epsilon_k = \int_{a_k}^{b_k}\phi_k\sigma(X_t)\,\mathrm{d}W_t ϵ k = ∫ a k b k ϕ k σ ( X t ) d W t , a stochastic integral, hence a martingale increment, with mean zero E [ ϵ k ] = 0 \mathbb{E}[\epsilon_k] = 0 E [ ϵ k ] = 0 . with mean zero. As it depends on the Brownian path, this residual is correlated with the regressors, the design-matrix entries A k j A_{kj} A k j of Eq. (4) . The correlation arises because, within the window, the path is built from the same Brownian increments that enter the residual. For a k ≤ s < t ≤ b k a_k \le s < t \le b_k a k ≤ s < t ≤ b k , the state X t X_t X t depends on the increment d W s \mathrm{d}W_s d W s , so E [ g j ( X t ) d W s ] \mathbb{E}[g_j(X_t)\,\mathrm{d}W_s] E [ g j ( X t ) d W s ] does not in general vanish. As a result, these contributions may accumulate over the entire window, and result in a nonzero covariance between A k j A_{kj} A k j and ϵ k \epsilon_k ϵ k .
Inconsistency and bias of ordinary least squares
We now quantify this covariance. The regressor A k j A_{kj} A k j and the residual ϵ k \epsilon_k ϵ k in Eq. (4) are both functionals of the same Brownian path { W s : s ∈ [ a , b ] } \{W_s:s\in[a,b]\} { W s : s ∈ [ a , b ]} over the window, so their product does not in general average to zero:
E [ A k j ϵ k ] = E [ ∫ a k b k ϕ k g j ( X t ) d t ∫ a k b k ϕ k σ ( X t ) d W t ] ≠ 0. (5) \mathbb{E}[A_{kj}\epsilon_k]
=
\mathbb{E}\left[
\int_{a_k}^{b_k}\phi_k g_j(X_t)\,\mathrm{d}t
\int_{a_k}^{b_k}\phi_k\sigma(X_t)\,\mathrm{d}W_t
\right]\ne0. \tag*{(5)} E [ A k j ϵ k ] = E [ ∫ a k b k ϕ k g j ( X t ) d t ∫ a k b k ϕ k σ ( X t ) d W t ] = 0. (5)
This covariance between regressors and error, termed endogeneity [35 ] , violates the orthogonality condition E [ A T ϵ ] = 0 \mathbb{E}[A^\mathsf{T}\boldsymbol{\epsilon}]=0 E [ A T ϵ ] = 0 and renders the ordinary least squares (OLS) estimator θ ^ O L S = ( A T A ) − 1 A T b \hat{\boldsymbol{\theta}}_{\mathrm{OLS}}=(A^\mathsf{T}A)^{-1}A^\mathsf{T}\mathbf{b} θ ^ OLS = ( A T A ) − 1 A T b biased and inconsistent [44 ] . The mechanism is most transparent for the stationary Ornstein–Uhlenbeck (OU) process
d X t = − κ X t d t + σ d W t , κ > 0 , \mathrm{d}X_t=-\kappa X_t\,\mathrm{d}t+\sigma\,\mathrm{d}W_t,\qquad\kappa>0, d X t = − κ X t d t + σ d W t , κ > 0 ,
whose state admits the moving-average representation X t = σ ∫ − ∞ t e − κ ( t − s ) d W s X_t=\sigma\int_{-\infty}^{t}e^{-\kappa(t-s)}\,\mathrm{d}W_s X t = σ ∫ − ∞ t e − κ ( t − s ) d W s [45 ] . With the drift term candidate g ( x ) = x g(x)=x g ( x ) = x , the regressor A = ∫ a b ϕ ( t ) X t d t A=\int_a^b\phi(t)X_t\,\mathrm{d}t A = ∫ a b ϕ ( t ) X t d t contains every increment d W s \mathrm{d}W_s d W s with s ∈ ( a , b ) s\in(a,b) s ∈ ( a , b ) , each of which also enters the residual ϵ = σ ∫ a b ϕ d W t \epsilon=\sigma\int_a^b\phi\,\mathrm{d}W_t ϵ = σ ∫ a b ϕ d W t , so that
E [ A ϵ ] = σ 2 ∫ a < s < t < b ϕ ( t ) ϕ ( s ) e − κ ( t − s ) d s d t > 0. \mathbb{E}[A\epsilon]
=
\sigma^2\int_{a<s<t<b}\phi(t)\phi(s)e^{-\kappa(t-s)}\,\mathrm{d}s\,\mathrm{d}t>0. E [ A ϵ ] = σ 2 ∫ a < s < t < b ϕ ( t ) ϕ ( s ) e − κ ( t − s ) d s d t > 0.
Positivity is immediate for the nonnegative bump test functions used here, but holds for any nonzero test function (Appendix A.1).
On each window, the weak-form response is b = ∫ a b ϕ ( t ) d X t = − ∫ a b ϕ ′ ( t ) X t d t \mathbf{b}=\int_a^b\phi(t)\,\mathrm{d}X_t=-\int_a^b\phi'(t)X_t\,\mathrm{d}t b = ∫ a b ϕ ( t ) d X t = − ∫ a b ϕ ′ ( t ) X t d t , and substituting the dynamics gives b = − κ A + ϵ \mathbf{b}=-\kappa A+\boldsymbol{\epsilon} b = − κ A + ϵ . Each window contributes one row ( A k , b k , ϵ k ) (A_k,b_k,\epsilon_k) ( A k , b k , ϵ k ) of Eq. (4) ; with the window span held fixed, a longer record adds more windows. As the number of windows grows, the least-squares estimator converges in probability to the population regression coefficient, obtained by replacing sample averages with expectations:
θ O L S p o p : = arg min θ ∈ R E [ ( b − θ A ) 2 ] = E [ A b ] E [ A 2 ] = − κ + E [ A ϵ ] E [ A 2 ] . \theta_{\mathrm{OLS}}^{\mathrm{pop}}
:=
\underset{\theta\in\mathbb{R}}{\operatorname{arg\,min}}\ \mathbb{E}\left[(b-\theta A)^2\right]
=
\frac{\mathbb{E}[Ab]}{\mathbb{E}[A^2]}
=
-\kappa+\frac{\mathbb{E}[A\epsilon]}{\mathbb{E}[A^2]}. θ OLS pop := θ ∈ R arg min E [ ( b − θ A ) 2 ] = E [ A 2 ] E [ A b ] = − κ + E [ A 2 ] E [ A ϵ ] .
The positive covariance E [ A ϵ ] \mathbb{E}[A\epsilon] E [ A ϵ ] thus attenuates the estimated drift toward zero, away from its true value − κ -\kappa − κ . For OU, which is time-reversible, the attenuation is total. As we will see in Proposition 1 below, E [ A ϵ ] = κ E [ A 2 ] \mathbb{E}[A\epsilon]=\kappa\,\mathbb{E}[A^2] E [ A ϵ ] = κ E [ A 2 ] , so the population least-squares coefficient is exactly zero.
The same decomposition holds for a general set of drift term candidates. Writing a k T \mathbf{a}_k^\mathsf{T} a k T for the k k k -th row of A A A , the pairs ( a k , ϵ k ) (\mathbf{a}_k,\epsilon_k) ( a k , ϵ k ) are identically distributed across windows under stationarity, and the least-squares estimator converges to θ O L S p o p = θ + E [ a a T ] − 1 E [ a ϵ ] \boldsymbol{\theta}_{\mathrm{OLS}}^{\mathrm{pop}}=\boldsymbol{\theta}+\mathbb{E}[\mathbf{a}\mathbf{a}^\mathsf{T}]^{-1}\mathbb{E}[\mathbf{a}\epsilon] θ OLS pop = θ + E [ a a T ] − 1 E [ a ϵ ] , provided E [ a a T ] \mathbb{E}[\mathbf{a}\mathbf{a}^\mathsf{T}] E [ a a T ] is invertible [44 ] . The bias is a fixed vector of per-window expectations, independent of the number of windows, so it does not diminish asymptotically.
Proposition 1. Let X X X be a stationary Itô diffusion d X t = f ( X t ) d t + σ ( X t ) d W t \mathrm{d}X_t=f(X_t)\,\mathrm{d}t+\sigma(X_t)\,\mathrm{d}W_t d X t = f ( X t ) d t + σ ( X t ) d W t in R d \mathbb{R}^d R d that is time-reversible, that is, ( X 0 , X u ) (X_0,X_u) ( X 0 , X u ) and ( X u , X 0 ) (X_u,X_0) ( X u , X 0 ) have the same law for every u u u . Let g 1 , … , g p g_1,\ldots,g_p g 1 , … , g p be drift term candidates, and assume E ∥ X 0 ∥ 2 \mathbb{E}\|X_0\|^2 E ∥ X 0 ∥ 2 , E ∥ f ( X 0 ) ∥ 2 \mathbb{E}\|f(X_0)\|^2 E ∥ f ( X 0 ) ∥ 2 , E ∥ σ ( X 0 ) ∥ 2 \mathbb{E}\|\sigma(X_0)\|^2 E ∥ σ ( X 0 ) ∥ 2 , and E [ g j ( X 0 ) 2 ] \mathbb{E}[g_j(X_0)^2] E [ g j ( X 0 ) 2 ] are finite, and that each cross-covariance C j i ( u ) = Cov ( g j ( X u ) , X 0 ( i ) ) C_{ji}(u)=\operatorname{Cov}(g_j(X_u),X_0^{(i)}) C j i ( u ) = Cov ( g j ( X u ) , X 0 ( i ) ) is continuously differentiable for u ≠ 0 u\ne0 u = 0 with bounded derivative. Let ϕ \phi ϕ be any bounded test function supported on [ a , b ] [a,b] [ a , b ] , and let a j = ∫ a b ϕ ( t ) g j ( X t ) d t a_j=\int_a^b\phi(t)g_j(X_t)\,\mathrm{d}t a j = ∫ a b ϕ ( t ) g j ( X t ) d t and b i = ∫ a b ϕ ( t ) d X t ( i ) b_i=\int_a^b\phi(t)\,\mathrm{d}X_t^{(i)} b i = ∫ a b ϕ ( t ) d X t ( i ) . Then Cov ( a j , b i ) = 0 \operatorname{Cov}(a_j,b_i)=0 Cov ( a j , b i ) = 0 for all j j j and i i i , and, whenever the population Gram matrix of the drift term candidates is invertible, the population regression coefficient of b i b_i b i on the drift term candidates, which is the probability limit of the least-squares estimator as the number of windows grows, is the zero vector, for every window and every test function.
The full proof is given in Appendix A.1. Briefly, off the diagonal, stationarity gives E [ g j ( X t ) d X s ( i ) ] = − C j i ′ ( t − s ) d s \mathbb{E}[g_j(X_t)\,\mathrm{d}X_s^{(i)}]=-C'_{ji}(t-s)\,\mathrm{d}s E [ g j ( X t ) d X s ( i ) ] = − C j i ′ ( t − s ) d s , and reversibility makes C j i C_{ji} C j i an even function of the lag, implying that the integrand ϕ ( t ) ϕ ( s ) C j i ′ ( t − s ) \phi(t)\phi(s)C'_{ji}(t-s) ϕ ( t ) ϕ ( s ) C j i ′ ( t − s ) is odd and integrates to zero over the symmetric window square. Also, note that every stationary ergodic scalar diffusion is reversible, so for scalar systems (like OU and double-well) the entire drift is always erased. § 8.1 demonstrates this drift cancellation also on a real climate record.
Instrumental variables from disjoint windows
The inconsistency of least squares discussed above stems from structural endogeneity, and consequently, the failure of the orthogonality condition E [ A T ϵ ] = 0 \mathbb{E}[\mathbf{A}^{\mathsf{T}}\boldsymbol{\epsilon}]=0 E [ A T ϵ ] = 0 . A possible remedy is to impose orthogonality with respect to an instrumental variable instead. Writing the full weak-form system as b = A θ + ϵ \mathbf{b}=\mathbf{A}\boldsymbol{\theta}+\boldsymbol{\epsilon} b = A θ + ϵ , an instrument is a matrix Z \mathbf{Z} Z with one row per moment condition, like A \mathbf{A} A , and p p p columns, satisfying:
(IV1) Relevance. E [ Z T A ] \mathbb{E}[\mathbf{Z}^{\mathsf{T}}\mathbf{A}] E [ Z T A ] has full column rank (the instrument is correlated with the regressors). When Z \mathbf{Z} Z has as many columns as A \mathbf{A} A , full column rank makes E [ Z T A ] \mathbb{E}[\mathbf{Z}^{\mathsf{T}}\mathbf{A}] E [ Z T A ] square and invertible, and the estimator below is well defined.
(IV2) Exogeneity. E [ Z T ϵ ] = 0 \mathbb{E}[\mathbf{Z}^{\mathsf{T}}\boldsymbol{\epsilon}]=0 E [ Z T ϵ ] = 0 (the instrument is uncorrelated with the residual).
Given a valid instrument, the IV estimator is defined by the empirical moment condition Z T ( b − A θ ) = 0 \mathbf{Z}^{\mathsf{T}}(\mathbf{b}-\mathbf{A}\boldsymbol{\theta})=0 Z T ( b − A θ ) = 0 , that is,
θ ^ I V = ( Z T A ) − 1 Z T b . (6) \widehat{\boldsymbol{\theta}}_{\mathrm{IV}}=(\mathbf{Z}^{\mathsf{T}}\mathbf{A})^{-1}\mathbf{Z}^{\mathsf{T}}\mathbf{b}. \tag*{(6)} θ IV = ( Z T A ) − 1 Z T b . (6)
We make the condition (IV2) hold by building the instrument from a test function supported away from the target window. We tile the observation span into N w N_w N w pairs of windows, each pair holding a target window [ a k , a k + T w ] [a_k,a_k+T_w] [ a k , a k + T w ] preceded by an instrument window [ a k ′ , a k ′ + T w ] [a'_k,a'_k+T_w] [ a k ′ , a k ′ + T w ] with a k ′ + T w ≤ a k a'_k+T_w\le a_k a k ′ + T w ≤ a k , so distinct windows overlap at most at the single boundary point a k ′ + T w = a k a'_k+T_w=a_k a k ′ + T w = a k (Figure 1 ). Evaluating M ϕ M_\phi M ϕ test functions per pair (Remark 3) then gives M ϕ N w M_\phi N_w M ϕ N w stacked moment rows. Asymptotic statements are in the number of window pairs, which we denote K = N w K=N_w K = N w .
Figure 1. Window layout for one moment pair. Each target window [ a k , a k + T w ] [a_k,a_k+T_w] [ a k , a k + T w ] of span T w = L Δ t T_w=L\Delta t T w = L Δ t carries a test function ϕ k \phi_k ϕ k , and its instrument is built from the n l a g s n_{\mathrm{lags}} n lags preceding blocks of the same length, carrying ψ k \psi_k ψ k and separated by a gap δ \delta δ . These window pairs tile the record with stride 2 L Δ t + δ 2L\Delta t+\delta 2 L Δ t + δ . The sketch shows two instrument windows (depth two), although the default used throughout is depth one.
Definition 1. For each window pair k k k , let ϕ k \phi_k ϕ k be a target test function supported on [ a k , a k + T w ] [a_k,a_k+T_w] [ a k , a k + T w ] and let ψ k \psi_k ψ k be an instrument test function supported on [ a k ′ , a k ′ + T w ) [a'_k,a'_k+T_w) [ a k ′ , a k ′ + T w ) . Define
Z k j : = ∫ a k ′ a k ′ + T w ψ k ( t ) g j ( X t ) d t . (7) Z_{kj}:=\int_{a'_k}^{a'_k+T_w}\psi_k(t)g_j(X_t)\,\mathrm{d}t. \tag*{(7)} Z k j := ∫ a k ′ a k ′ + T w ψ k ( t ) g j ( X t ) d t . (7)
Exogeneity holds because the residual ϵ k \epsilon_k ϵ k is a stochastic integral of the Brownian increments over ( a k , a k + T w ] (a_k,a_k+T_w] ( a k , a k + T w ] , while Z k j Z_{kj} Z k j is measurable with respect to the increments up to time a k ′ + T w ≤ a k a'_k+T_w\le a_k a k ′ + T w ≤ a k . Past and future increments are independent, so E [ Z k j ϵ k ] = 0 \mathbb{E}[Z_{kj}\epsilon_k]=0 E [ Z k j ϵ k ] = 0 , giving (IV2). Relevance (IV1) holds provided the instrument window lies within the system’s memory, so that the state X X X on [ a k ′ , a k ′ + T w ] [a'_k,a'_k+T_w] [ a k ′ , a k ′ + T w ] is informative about g j ( X ) g_j(X) g j ( X ) on [ a k , a k + T w ] [a_k,a_k+T_w] [ a k , a k + T w ] .
Observable quantities
States are observed via Y t n = X t n + η n Y_{t_n}=X_{t_n}+\eta_n Y t n = X t n + η n . Each integral is evaluated as a grid sum of step Δ t \Delta t Δ t over the observation instants in its window, I k = { n : t n ∈ [ a k , a k + T w ] } \mathcal{I}_k=\{n:t_n\in[a_k,a_k+T_w]\} I k = { n : t n ∈ [ a k , a k + T w ]} for the target and I k ′ = { m : t m ∈ [ a k ′ , a k ′ + T w ] } \mathcal{I}'_k=\{m:t_m\in[a'_k,a'_k+T_w]\} I k ′ = { m : t m ∈ [ a k ′ , a k ′ + T w ]} for the instrument. Because every test function and its derivative vanish at the window endpoints, this sum coincides with both the left-endpoint Riemann sum and the trapezoidal rule. Each resulting quantity decomposes into its exact value plus an observation-noise perturbation:
b k = − Δ t ∑ n ∈ I k ϕ k ′ ( t n ) Y t n ≈ b k X − Δ t ∑ n ∈ I k ϕ k ′ ( t n ) η n , (8) b_k=-\Delta t\sum_{n\in\mathcal{I}_k}\phi'_k(t_n)Y_{t_n}\approx b_k^X-\Delta t\sum_{n\in\mathcal{I}_k}\phi'_k(t_n)\eta_n, \tag*{(8)} b k = − Δ t n ∈ I k ∑ ϕ k ′ ( t n ) Y t n ≈ b k X − Δ t n ∈ I k ∑ ϕ k ′ ( t n ) η n , (8)
A k j = Δ t ∑ n ∈ I k ϕ k ( t n ) g j ( Y t n ) ≈ A k j X + Δ t ∑ n ∈ I k ϕ k ( t n ) g j ′ ( X t n ) η n . (9) \begin{aligned}
A_{kj}=\Delta t\sum_{n\in\mathcal{I}_k}\phi_k(t_n)g_j(Y_{t_n}) \\
&\approx A_{kj}^X+\Delta t\sum_{n\in\mathcal{I}_k}\phi_k(t_n)g'_j(X_{t_n})\eta_n. \tag*{(9)}
\end{aligned} A k j = Δ t n ∈ I k ∑ ϕ k ( t n ) g j ( Y t n ) ≈ A k j X + Δ t n ∈ I k ∑ ϕ k ( t n ) g j ′ ( X t n ) η n . (9)
Z k j = Δ t ∑ m ∈ I k ′ ψ k ( t m ) g j ( Y t m ) ≈ Z k j X + Δ t ∑ m ∈ I k ′ ψ k ( t m ) g j ′ ( X t m ) η m . (10) \begin{aligned}
Z_{kj}=\Delta t\sum_{m\in\mathcal{I}'_k}\psi_k(t_m)g_j(Y_{t_m}) \\
&\approx Z_{kj}^X+\Delta t\sum_{m\in\mathcal{I}'_k}\psi_k(t_m)g'_j(X_{t_m})\eta_m. \tag*{(10)}
\end{aligned} Z k j = Δ t m ∈ I k ′ ∑ ψ k ( t m ) g j ( Y t m ) ≈ Z k j X + Δ t m ∈ I k ′ ∑ ψ k ( t m ) g j ′ ( X t m ) η m . (10)
The approximations in Eq. (8) –Eq. (10) contain two remainders. The first is the O ( Δ t ) O(\Delta t) O ( Δ t ) quadrature error, which vanishes as Δ t → 0 \Delta t\to0 Δ t → 0 . The second, present only in A A A and Z Z Z when the drift term candidate g j g_j g j is nonlinear, is the second-order Taylor remainder of g j ( X + η ) g_j(X+\eta) g j ( X + η ) , which has nonzero mean of order ∥ R ∥ \lVert R\rVert ∥ R ∥ , proportional to the curvature g j ′ ′ g_j'' g j ′′ (removed by the debiasing of § 6 ; the higher-order terms are absorbed in the remainder ρ k \rho_k ρ k below). The response b is linear in the data, so δ b k \delta b_k δ b k has mean zero. Using the identity b k X = A k X θ + ϵ k b_k^X=A_k^X\theta+\epsilon_k b k X = A k X θ + ϵ k , the observable regression is b k = A k θ + ϵ k t o t b_k=A_k\theta+\epsilon_k^{\mathrm{tot}} b k = A k θ + ϵ k tot , with the observed A k A_k A k instead of A k X A_k^X A k X on the right, and with total residual ϵ k t o t = ϵ k + δ b k − δ A k θ + ρ k \epsilon_k^{\mathrm{tot}}=\epsilon_k+\delta b_k-\delta A_k\theta+\rho_k ϵ k tot = ϵ k + δ b k − δ A k θ + ρ k . Here ρ k \rho_k ρ k collects the O ( Δ t ) O(\Delta t) O ( Δ t ) quadrature and O ( ∥ R ∥ 2 ) O(\lVert R\rVert^2) O (∥ R ∥ 2 ) Taylor remainders of Eq. (8) –Eq. (10) . The growth conditions of the consistency theorem (Theorem 1) ensure ρ k \rho_k ρ k is asymptotically negligible.
Consistency and asymptotic normality
The IV estimator Eq. (6) has the exact error decomposition
θ ^ I V − θ = ( 1 K Z T A ) − 1 ( 1 K Z T ϵ t o t ) . (11) \hat{\theta}_{\mathrm{IV}}-\theta=\left(\frac{1}{K}Z^{\mathsf{T}}A\right)^{-1}\left(\frac{1}{K}Z^{\mathsf{T}}\epsilon^{\mathrm{tot}}\right). \tag*{(11)} θ ^ IV − θ = ( K 1 Z T A ) − 1 ( K 1 Z T ϵ tot ) . (11)
Under (IV1)–(IV2) and E [ Z k ϵ k t o t ] = 0 \mathbb{E}[Z_k\epsilon_k^{\mathrm{tot}}]=0 E [ Z k ϵ k tot ] = 0 , the second factor vanishes in probability and the first converges to an invertible limit, and thus θ ^ I V → P θ \hat{\theta}_{\mathrm{IV}}\xrightarrow{\mathbb{P}}\theta θ ^ IV P θ as K → ∞ K\to\infty K → ∞ with Δ t → 0 \Delta t\to0 Δ t → 0 . Note that consistency requires no rate coupling between the two, unlike asymptotic normality, which requires K Δ t → 0 \sqrt{K}\,\Delta t\to0 K Δ t → 0 (Theorem 2). Because each instrument is built strictly before its target window starts, the time-ordered summands Z k ϵ k t o t Z_k\epsilon_k^{\mathrm{tot}} Z k ϵ k tot form a martingale-difference sequence (the instruments themselves are dependent across k k k ; see Appendix § 11 ), and the martingale central limit theorem gives
K ( θ ^ I V − θ ) → d N ( 0 , G ) , (12) \sqrt{K}\left(\hat{\theta}_{\mathrm{IV}}-\theta\right)\xrightarrow{d}\mathcal{N}(0,G), \tag*{(12)} K ( θ ^ IV − θ ) d N ( 0 , G ) , (12)
with covariance G \mathbf{G} G determined by Z 1 Z_1 Z 1 , A 1 A_1 A 1 , and ϵ 1 t o t \epsilon_1^{\mathrm{tot}} ϵ 1 tot . The precise statements follow, with proofs in Appendix § 11 .
Proposition 2. Under model Eq. (1)–Eq. (2) with { η n } ⊥ ⊥ { W t } \{\eta_n\}\perp\!\!\!\perp\{W_t\} { η n } ⊥ ⊥ { W t } , the process-noise residual satisfies E [ Z k j ϵ k ] = 0 \mathbb{E}[Z_{kj}\epsilon_k]=0 E [ Z k j ϵ k ] = 0 exactly. The observation-noise perturbations δ b k \delta b_k δ b k and δ A k j \delta A_{kj} δ A k j are orthogonal to Z k j Z_{kj} Z k j to first order in the errors, so the total residual satisfies E [ Z k j ϵ k t o t ] = E [ Z k j ρ k ] \mathbb{E}[Z_{kj}\epsilon_k^{\mathrm{tot}}]=\mathbb{E}[Z_{kj}\rho_k] E [ Z k j ϵ k tot ] = E [ Z k j ρ k ] . After the debiasing this remainder is O ( Δ t ) + O ( ∥ R ∥ 2 ) O(\Delta t)+O(\lVert R\rVert^2) O ( Δ t ) + O (∥ R ∥ 2 ) , and the rate conditions of the consistency theorem make it negligible.
Proposition 3. Suppose { X t } \{X_t\} { X t } is stationary and β \beta β -mixing with mixing coefficient β ( u ) → 0 \beta(u)\to0 β ( u ) → 0 , u u u denoting the time separation, where
β ( u ) = E sup B ∈ σ ( X s : s ≥ u ) ∣ P ( B ∣ σ ( X s : s ≤ 0 ) ) − P ( B ) ∣ \beta(u)=\mathbb{E}\sup_{B\in\sigma(X_s:s\geq u)}\left|\mathbb{P}\left(B\mid\sigma(X_s:s\leq0)\right)-\mathbb{P}(B)\right| β ( u ) = E B ∈ σ ( X s : s ≥ u ) sup ∣ P ( B ∣ σ ( X s : s ≤ 0 ) ) − P ( B ) ∣
measures the dependence between the past up to time 0 and the future from time u u u on, that the active drift term candidates have finite moments of some order ν > 2 \nu>2 ν > 2 under the stationary law, and that the gap between instrument and target windows is δ ≥ 0 \delta\geq0 δ ≥ 0 . Write Γ j j ′ ( u ) = Cov ( g j ( X 0 ) , g j ′ ( X u ) ) \Gamma_{jj'}(u)=\operatorname{Cov}(g_j(X_0),g_{j'}(X_u)) Γ j j ′ ( u ) = Cov ( g j ( X 0 ) , g j ′ ( X u )) , and let Γ ( u ) = [ Γ j j ′ ( u ) ] j , j ′ = 1 p \Gamma(u)=[\Gamma_{jj'}(u)]_{j,j'=1}^{p} Γ ( u ) = [ Γ j j ′ ( u ) ] j , j ′ = 1 p be the p × p p\times p p × p matrix of these covariances. Suppose further that the per-pair cross-moment matrix Q \mathbf{Q} Q defined below is nonsingular; a sufficient condition, under which the proof establishes positive-definiteness, is uniform positivity of the cross-moment kernel: with g ˉ = E [ g ( X 0 ) ] \bar{g}=\mathbb{E}[g(X_0)] g ˉ = E [ g ( X 0 )] , there is m > 0 m>0 m > 0 with
ζ T [ g ˉ g ˉ T + Γ ( u ) ] ζ ≥ m ∥ ζ ∥ 2 \zeta^{\mathsf{T}}\left[\bar{g}\bar{g}^{\mathsf{T}}+\Gamma(u)\right]\zeta\geq m\lVert\zeta\rVert^2 ζ T [ g ˉ g ˉ T + Γ ( u ) ] ζ ≥ m ∥ ζ ∥ 2
for every ζ ∈ R p \zeta\in\mathbb{R}^p ζ ∈ R p and every separation 0 ≤ u ≤ δ + 2 T w 0\leq u\leq\delta+2T_w 0 ≤ u ≤ δ + 2 T w . (The centered matrix Γ ( u ) \Gamma(u) Γ ( u ) alone is singular whenever the model contains a constant term, as every model in this paper does, so the condition must be placed on the uncentered kernel.) Let Q ∈ R p × p \mathbf{Q}\in\mathbb{R}^{p\times p} Q ∈ R p × p be the per-pair cross-moment matrix with entries Q j j ′ = E [ Z k j A k j ′ ] Q_{jj'}=\mathbb{E}[Z_{kj}A_{kj'}] Q j j ′ = E [ Z k j A k j ′ ] , the same for every pair k k k by stationarity, so that Q = E [ Z k A k T ] \mathbf{Q}=\mathbb{E}[Z_kA_k^{\mathsf{T}}] Q = E [ Z k A k T ] . Then Q \mathbf{Q} Q is positive definite, and E [ Z T A ] \mathbb{E}[Z^{\mathsf{T}}A] E [ Z T A ] , a positive multiple of Q \mathbf{Q} Q , has full column rank (IV1).
The following results establish consistency and asymptotic normality, and show that plugging in the estimated noise covariance and the estimated GMM weight leaves the first-order theory intact. All limits are in the number of window pairs K K K , with the conditions on Δ t \Delta t Δ t and on the observation noise stated in each result. Throughout, the drift term candidates are assumed to admit exact Gaussian deconvolution, as the polynomial and trigonometric terms used in this paper do; general terms corrected by the regularized Gauss–Hermite rule of Appendix § 13 incur an additional O ( ∥ R ∥ 2 ) O(\lVert R\rVert^2) O (∥ R ∥ 2 ) approximation error, which must satisfy K ∥ R ∥ 2 → 0 \sqrt{K}\lVert R\rVert^2\to0 K ∥ R ∥ 2 → 0 for the central limit theorem to carry over.
Theorem 1. Under Propositions 2–3, with ∑ u ≥ 1 β ( u ) 1 − 2 / ν < ∞ \sum_{u\geq1}\beta(u)^{1-2/\nu}<\infty ∑ u ≥ 1 β ( u ) 1 − 2/ ν < ∞ for the moment order ν > 2 \nu>2 ν > 2 of Proposition 3 (exponential mixing, as in Remark 5, suffices) and with K → ∞ K\to\infty K → ∞ and Δ t → 0 \Delta t\to0 Δ t → 0 such that K Δ t 2 → 0 K\Delta t^2\to0 K Δ t 2 → 0 (equivalently K Δ t → 0 \sqrt{K}\Delta t\to0 K Δ t → 0 , i.e. T Δ t → 0 \sqrt{T}\Delta t\to0 T Δ t → 0 at fixed window geometry, since K ≃ T / ( 2 T w + δ ) K\simeq T/(2T_w+\delta) K ≃ T / ( 2 T w + δ ) ; this single rate serves both results below, and is sharp only for the CLT, since for consistency alone Δ t → 0 \Delta t\to0 Δ t → 0 suffices for the O ( Δ t ) O(\Delta t) O ( Δ t ) quadrature remainder) and ∥ R ∥ o p Δ t / ( σ min 2 T w 2 ) → 0 \lVert R\rVert_{\mathrm{op}}\Delta t/(\sigma_{\min}^2T_w^2)\to0 ∥ R ∥ op Δ t / ( σ m i n 2 T w 2 ) → 0 , with T w = L Δ t T_w=L\Delta t T w = L Δ t the window span, ∥ R ∥ o p \lVert R\rVert_{\mathrm{op}} ∥ R ∥ op the spectral norm of the observation-noise covariance, and σ min 2 \sigma_{\min}^2 σ m i n 2 the smallest eigenvalue of σ T \sigma^{\mathsf{T}} σ T over the support of the stationary law (controlling the observation-noise term, cf. Eq. (16) ), θ ^ I V → P θ \hat{\theta}_{\mathrm{IV}}\xrightarrow{\mathbb{P}}\theta θ ^ IV P θ .
Remark 1. The window width enters the two rate conditions in competing directions. The observation-noise condition ∥ R ∥ o p Δ t / ( σ min 2 T w 2 ) → 0 \lVert R\rVert_{\mathrm{op}}\Delta t/(\sigma_{\min}^2T_w^2)\to0 ∥ R ∥ op Δ t / ( σ m i n 2 T w 2 ) → 0 improves as the span grows, since wider windows suppress measurement noise quadratically (Eq. (16) ), whereas relevance (Proposition 3) requires T w T_w T w to stay within the characteristic time τ \tau τ , favoring short windows. The two are reconciled by fixing the physical span inside the memory, T w ≲ τ T_w\lesssim\tau T w ≲ τ , as the design rule of Appendix § 12 does, and refining the sampling; as Δ t → 0 \Delta t\to0 Δ t → 0 at fixed T w T_w T w the window holds L = T w / Δ t L=T_w/\Delta t L = T w /Δ t samples, the noise ratio vanishes as Δ t / T w \Delta t/T_w Δ t / T w , and relevance is untouched, so consistency holds along every such refinement.
Theorem 2. Under the same conditions, with Q = E [ Z 1 A 1 T ] \mathbf{Q}=\mathbb{E}[Z_1A_1^{\mathsf{T}}] Q = E [ Z 1 A 1 T ] and Σ = Var ( Z 1 ϵ 1 t o t ) \boldsymbol{\Sigma}=\operatorname{Var}(Z_1\epsilon_1^{\mathrm{tot}}) Σ = Var ( Z 1 ϵ 1 tot ) finite and positive definite,
K ( θ ^ I V − θ ) → d N ( 0 , G ) , G = Q − 1 Σ Q − T . (13) \sqrt{K}\left(\hat{\theta}_{\mathrm{IV}}-\theta\right)\xrightarrow{d}\mathcal{N}(0,G),\qquad\mathbf{G}=\mathbf{Q}^{-1}\boldsymbol{\Sigma}\mathbf{Q}^{-\mathsf{T}}. \tag*{(13)} K ( θ ^ IV − θ ) d N ( 0 , G ) , G = Q − 1 Σ Q − T . (13)
The over-identified GMM estimator with weight W ^ = Σ − 1 \widehat{W} = \Sigma^{-1} W = Σ − 1 attains the minimal covariance ( Q T Σ − 1 Q ) − 1 (Q^{\mathsf{T}}\Sigma^{-1}Q)^{-1} ( Q T Σ − 1 Q ) − 1 among GMM estimators based on these moment conditions.
The theorems above concern the estimator with known R R R and a fixed weight. The implemented estimator plugs in the variogram R ^ \widehat{R} R , debiases the drift term candidates with the same R ^ \widehat{R} R , and estimates the GMM weight from first-step residuals. The next results close this gap.
Proposition 4. Assume the conditions of the consistency theorem, including the noise rate ∥ R ∥ o p Δ t / ( σ min 2 T w 2 ) → 0 \lVert R\rVert_{\mathrm{op}}\Delta t/(\sigma_{\min}^{2}T_{w}^{2}) \to0 ∥ R ∥ op Δ t / ( σ m i n 2 T w 2 ) → 0 , and let R ^ \widehat{R} R be the multilag variogram estimator of § 4 computed from the same record of length N N N . Write θ ^ ( R ^ ) \widehat{\theta}(\widehat{R}) θ ( R ) for the Algorithm 1 estimator in which R ^ \widehat{R} R enters the response subtraction and the debiasing, and suppose the debiasing map R ↦ g ~ R R \mapsto\widetilde{g}_{R} R ↦ g R is continuously differentiable in R R R near the truth, as it is for polynomial and trigonometric terms. Then, at fixed window span T w T_{w} T w , as in the consistency theorem, θ ^ ( R ^ ) \widehat{\theta}(\widehat{R}) θ ( R ) is consistent, ∥ θ ^ ( R ^ ) − θ ^ ( R ) ∥ = O p ( ∥ R ^ − R ∥ ) = O p ( N − 1 / 2 ) \lVert\widehat{\theta}(\widehat{R})-\widehat{\theta}(R)\rVert= O_{p}(\lVert\widehat{R}-R\rVert)=O_{p}(N^{-1/2}) ∥ θ ( R ) − θ ( R )∥ = O p (∥ R − R ∥) = O p ( N − 1/2 ) , and K ( θ ^ ( R ^ ) − θ ) → d N ( 0 , G R ^ ) \sqrt{K}(\widehat{\theta}(\widehat{R})-\theta)\xrightarrow{d}\mathcal{N}(0,G_{\widehat{R}}) K ( θ ( R ) − θ ) d N ( 0 , G R ) at the same rate K \sqrt{K} K as with R R R known, with G R ^ = G + G p l G_{\widehat{R}}=G+G_{\mathrm{pl}} G R = G + G pl , where G p l G_{\mathrm{pl}} G pl collects the variance of R ^ \widehat{R} R propagated through the debiased columns and its covariance with the moment error, both of relative order K / N ≍ 1 / L K/N\asymp1/L K / N ≍ 1/ L . When R R R enters no drift column, as for the affine drift of § 8.1 and § 8.2 , θ ^ ( R ^ ) = θ ^ ( R ) \widehat{\theta}(\widehat{R})=\widehat{\theta}(R) θ ( R ) = θ ( R ) exactly. Estimating the weight leaves the first-order theory unchanged whenever W ^ → p Σ − 1 \widehat{W}\xrightarrow{p}\Sigma^{-1} W p Σ − 1 , which holds for the first-step residual plug-in [36 ] .
Remark 2. The estimator applies verbatim to the observation model
Y t n = C X t n + η n , η n ∼ i i d N ( 0 , R ) , C ∈ R d y × d , (14) Y_{t_n}=CX_{t_n}+\eta_n,\qquad\eta_n \mathrel{\overset{\mathrm{iid}}{\sim}}\mathcal{N}(0,R),\qquad C\in\mathbb{R}^{d_y\times d}, \tag*{(14)} Y t n = C X t n + η n , η n ∼ iid N ( 0 , R ) , C ∈ R d y × d , (14)
with C C C known, d y ≥ d d_y\ge d d y ≥ d and rank C = d \operatorname{rank}C=d rank C = d . The deconvolution Y ^ t n = ( C T C ) − 1 C T Y t n \widehat{Y}_{t_n}=(C^{\mathsf{T}}C)^{-1}C^{\mathsf{T}}Y_{t_n} Y t n = ( C T C ) − 1 C T Y t n recovers the state up to additive Gaussian error with covariance
R e f f = ( C T C ) − 1 C T R C ( C T C ) − 1 , (15) R_{\mathrm{eff}}=(C^{\mathsf{T}}C)^{-1}C^{\mathsf{T}}RC(C^{\mathsf{T}}C)^{-1}, \tag*{(15)} R eff = ( C T C ) − 1 C T R C ( C T C ) − 1 , (15)
where the feasible estimator uses Y ^ \widehat{Y} Y and R ^ e f f \widehat{R}_{\mathrm{eff}} R eff in place of R ^ \widehat{R} R . Note that for d y < d d_y<d d y < d the state is not identified pointwise.
Quadratic suppression of observation noise
The relative size of measurement-noise and process-noise contributions to the response is governed by the window length L L L , the number of observation steps in each window, whose support span is T w = L Δ t T_w=L\Delta t T w = L Δ t . The instrument and target windows are designed to share the same length L L L . Let ∥ f ∥ L 2 2 = ∫ a k a k + T w f ( t ) 2 d t \lVert f\rVert_{L^2}^{2}=\int_{a_k}^{a_k+T_w}f(t)^2\,\mathrm{d}t ∥ f ∥ L 2 2 = ∫ a k a k + T w f ( t ) 2 d t . For σ \sigma σ roughly constant per window, the Itô isometry gives Var [ ε k ] ≈ σ 2 ∥ ϕ k ∥ L 2 2 \operatorname{Var}[\varepsilon_k]\approx\sigma^2\lVert\phi_k\rVert_{L^2}^{2} Var [ ε k ] ≈ σ 2 ∥ ϕ k ∥ L 2 2 . Similarly, the noise sum gives Var [ δ b k ] ≈ R Δ t ∥ ϕ k ′ ∥ L 2 2 \operatorname{Var}[\delta b_k]\approx R\Delta t\lVert\phi'_k\rVert_{L^2}^{2} Var [ δ b k ] ≈ R Δ t ∥ ϕ k ′ ∥ L 2 2 .
Each ϕ k \phi_k ϕ k is a fixed reference bump stretched to its window, so ∥ ϕ k ∥ L 2 2 ∝ T w \lVert\phi_k\rVert_{L^2}^{2}\propto T_w ∥ ϕ k ∥ L 2 2 ∝ T w and ∥ ϕ k ′ ∥ L 2 2 ∝ T w − 1 \lVert\phi'_k\rVert_{L^2}^{2}\propto T_w^{-1} ∥ ϕ k ′ ∥ L 2 2 ∝ T w − 1 , and hence
Var [ δ b k ] Var [ ε k ] ≈ R Δ t σ 2 T w 2 . (16) \frac{\operatorname{Var}[\delta b_k]}{\operatorname{Var}[\varepsilon_k]}\approx\frac{R\Delta t}{\sigma^2T_w^2}. \tag*{(16)} Var [ ε k ] Var [ δ b k ] ≈ σ 2 T w 2 R Δ t . (16)
Wider windows suppress observation noise quadratically but weaken the instrument. Choosing L L L is the main practical decision in WAIVES; § 7 does it empirically.
Joint diffusion estimation
The same weak-form construction can recover the diffusion in addition to the drift. Writing the diagonal ansatz σ i 2 ( x ) = ∑ l ν i , l h l ( x ) \sigma_i^2(x)=\sum_l\nu_{i,l}h_l(x) σ i 2 ( x ) = ∑ l ν i , l h l ( x ) for diffusion term candidates { h l } \{h_l\} { h l } , the quadratic variation of X i X^i X i , the i i i th component of the state, is absolutely continuous, with d [ X i , X i ] t = σ i 2 ( X t ) d t \mathrm{d}[X^i,X^i]_t=\sigma_i^2(X_t)\,\mathrm{d}t d [ X i , X i ] t = σ i 2 ( X t ) d t . For each component i i i and window k k k , testing against ϕ k \phi_k ϕ k and discretizing gives
b k , i σ = ∑ n ∈ I k ϕ ~ k ( t n ) [ ( e n i ) 2 − 2 R ^ i i ] , A k , l σ = Δ t ∑ n ∈ I k ϕ k ( t n ) h ~ l ( Y n ) . (17) \begin{aligned}
b_{k,i}^{\sigma}&=\sum_{n\in\mathcal{I}_k}\widetilde{\phi}_k(t_n)\left[(e_n^i)^2-2\widehat{R}_{ii}\right],\\
A_{k,l}^{\sigma}&=\Delta t\sum_{n\in\mathcal{I}_k}\phi_k(t_n)\widetilde{h}_l(Y_n). \tag*{(17)}
\end{aligned} b k , i σ A k , l σ = n ∈ I k ∑ ϕ k ( t n ) [ ( e n i ) 2 − 2 R ii ] , = Δ t n ∈ I k ∑ ϕ k ( t n ) h l ( Y n ) . (17)
Here, e n i = Δ Y n i − 1 2 [ f ^ i ( Y n ) + f ^ i ( Y n + 1 ) ] Δ t e_n^i=\Delta Y_n^i-\frac{1}{2}[\widehat{f}_i(Y_n)+\widehat{f}_i(Y_{n+1})]\Delta t e n i = Δ Y n i − 2 1 [ f i ( Y n ) + f i ( Y n + 1 )] Δ t , with Δ Y n i = Y t n + 1 i − Y t n i \Delta Y_n^i=Y_{t_{n+1}}^i-Y_{t_n}^i Δ Y n i = Y t n + 1 i − Y t n i the lag-one increment, ϕ ~ k ( t n ) = 1 2 ( ϕ k ( t n ) + ϕ k ( t n + 1 ) ) \widetilde{\phi}_k(t_n)=\frac{1}{2}(\phi_k(t_n)+\phi_k(t_{n+1})) ϕ k ( t n ) = 2 1 ( ϕ k ( t n ) + ϕ k ( t n + 1 )) is the trapezoidal step weight, f ^ \widehat{f} f is the drift estimate, and h ~ l \widetilde{h}_l h l is the diffusion term candidate debiased for observation noise as described in § 6 . Stacking the rows over the window pairs and test functions gives, for each component i i i , the linear system b i σ = A σ ν i + ε i σ b_i^\sigma=A^\sigma\nu_i+\varepsilon_i^\sigma b i σ = A σ ν i + ε i σ , solved for ν i \nu_i ν i by the same disjoint-window IV/GMM step as the drift, with instruments Z k , l σ = Δ t ∑ m ∈ I k ψ k ( t m ) h ~ l ( Y m ) Z_{k,l}^\sigma=\Delta t\sum_{m\in\mathcal{I}_k}\psi_k(t_m)\widetilde{h}_l(Y_m) Z k , l σ = Δ t ∑ m ∈ I k ψ k ( t m ) h l ( Y m ) on the instrument window (§ 4.5 ).
Differencing the observed series injects a deterministic measurement-noise variance 2 R ^ i i ∑ n ϕ ~ k ( t n ) 2\widehat{R}_{ii}\sum_n\widetilde{\phi}_k(t_n) 2 R ii ∑ n ϕ k ( t n ) into the raw quadratic variation, which we subtract directly. Additionally, the increment Δ Y n i \Delta Y_n^i Δ Y n i contains the drift ∫ t n t n + 1 f i ( X t ) d t \int_{t_n}^{t_{n+1}}f_i(X_t)\,\mathrm{d}t ∫ t n t n + 1 f i ( X t ) d t , which e n i e_n^i e n i removes by subtracting its trapezoidal approximation 1 2 [ f ^ i ( Y n ) + f ^ i ( Y n + 1 ) ] Δ t \frac{1}{2}[\widehat{f}_i(Y_n)+\widehat{f}_i(Y_{n+1})]\Delta t 2 1 [ f i ( Y n ) + f i ( Y n + 1 )] Δ t . The trapezoidal step also cancels the leading bias that a left-endpoint drift subtraction would leave, a cross term between the Brownian increment and the drift’s change within the step, of relative size f ′ Δ t f'\Delta t f ′ Δ t , and it retains the O ( Δ t 2 ) O(\Delta t^2) O ( Δ t 2 ) deterministic correction, without which the ∣ J f f ∣ 2 Δ t 4 \lvert J_f f\rvert^2\Delta t^4 ∣ J f f ∣ 2 Δ t 4 bias of an uncorrected increment can swamp the σ 2 Δ t \sigma^2\Delta t σ 2 Δ t signal in strongly driven systems (§ 4.5 ).
The noise covariance R ^ \widehat{R} R entering Eq. (17) , and a second, independent diffusion estimate, both come from one further construction: a multilag variogram of the observed increments, where the lag is the number of steps k k k an increment spans. For k = 1 , … , n ν k=1,\ldots,n_\nu k = 1 , … , n ν lags, let
V k = 1 N − k ∑ n ( Y t n + k − Y t n ) ( Y t n + k − Y t n ) T . (18) V_k=\frac{1}{N-k}\sum_n(Y_{t_{n+k}}-Y_{t_n})(Y_{t_{n+k}}-Y_{t_n})^{\mathsf{T}}. \tag*{(18)} V k = N − k 1 n ∑ ( Y t n + k − Y t n ) ( Y t n + k − Y t n ) T . (18)
whose expectation satisfies, for k Δ t k\Delta t k Δ t small,
E [ ( V k ) i j ] = 2 R i j + E π [ ( σ σ T ) i j ] ( k Δ t ) + c 2 , i j ( k Δ t ) 2 + O ( ( k Δ t ) 3 ) , (19) \mathbb{E}[(V_k)_{ij}] = 2R_{ij} + \mathbb{E}_{\pi}[(\sigma\sigma^{\mathsf T})_{ij}](k\Delta t) + c_{2,ij}(k\Delta t)^2 + O((k\Delta t)^3), \tag*{(19)} E [( V k ) ij ] = 2 R ij + E π [( σ σ T ) ij ] ( k Δ t ) + c 2 , ij ( k Δ t ) 2 + O (( k Δ t ) 3 ) , (19)
where c 2 , i j c_{2,ij} c 2 , ij is a drift-dependent stationary constant (identified for the OU case in Section 4.6). Stationarity of X X X is assumed throughout, so each expectation is affine in ( k , k 2 ) (k,k^2) ( k , k 2 ) . For each pair ( i , j ) (i,j) ( i , j ) , the n v n_v n v numbers ( V 1 ) i j , … , ( V n v ) i j (V_1)_{ij},\ldots,(V_{n_v})_{ij} ( V 1 ) ij , … , ( V n v ) ij are then regressed on the design ( 1 , k , k 2 ) (1,k,k^2) ( 1 , k , k 2 ) by ordinary least squares, ( V ~ k ) i j = c ~ i j + d ~ i j k + e ~ i j k 2 (\widetilde V_k)_{ij}=\widetilde c_{ij}+\widetilde d_{ij}k+\widetilde e_{ij}k^2 ( V k ) ij = c ij + d ij k + e ij k 2 . The intercepts of that regression give
R ^ = Π ⪰ 0 ( 1 2 c ~ ) , c ~ = ( c ~ i j ) , (20) \widehat R=\Pi_{\succeq0}\left(\frac{1}{2}\widetilde c\right),\qquad\widetilde c=(\widetilde c_{ij}), \tag*{(20)} R = Π ⪰ 0 ( 2 1 c ) , c = ( c ij ) , (20)
with Π ⪰ 0 \Pi_{\succeq0} Π ⪰ 0 the Frobenius-norm projection onto the positive-semidefinite cone, computed by zeroing negative eigenvalues. This projection is unique, and the fit is well posed for any n v ≥ 3 n_v\ge3 n v ≥ 3 . We use n v = 6 n_v=6 n v = 6 , and show that sensitivity to n v ∈ { 3 , … , 10 } n_v\in\{3,\ldots,10\} n v ∈ { 3 , … , 10 } is negligible (Appendix § 13 ).
Reading R ^ \widehat R R off the intercept requires no knowledge of the drift, and in particular of c 2 , i j c_{2,ij} c 2 , ij , whose value is absorbed by the quadratic regressor. Off-diagonal entries follow from the polarization identity, applied to the scalar variograms of the sums and differences Y i ± Y j Y^i\pm Y^j Y i ± Y j (Appendix § 13 ). Under the observation model Eq. (14) of Remark 2, the same construction, run on the deconvolved observations, returns R ^ e f f \widehat R_{\mathrm{eff}} R eff for the R e f f R_{\mathrm{eff}} R eff of Eq. (15) directly.
The slope d ~ i j \widetilde d_{ij} d ij of the same fit identifies the stationary mean E π [ ( σ σ T ) i j ] Δ t \mathbb{E}_{\pi}[(\sigma\sigma^{\mathsf T})_{ij}]\Delta t E π [( σ σ T ) ij ] Δ t of Eq. (19) , the diffusion matrix itself when σ \sigma σ is constant, and so supplies a second, independent estimate of that mean, ( σ σ T ^ ) i j = d ~ i j / Δ t (\widehat{\sigma\sigma^{\mathsf T}})_{ij}=\widetilde d_{ij}/\Delta t ( σ σ T ) ij = d ij /Δ t . We refer to this estimator as the multilag estimator. Its per-window, weak-form version replaces the lag-one response of Eq. (17) by
b k , i σ , ν = ∑ j = 2 j max w j S k , i ( j ) ; S k , i ( j ) = ∑ n ∈ I k ϕ k ( t n ) ( Y n + j i − Y n i ) 2 (21) b_{k,i}^{\sigma,\nu}=\sum_{j=2}^{j_{\max}}w_jS_{k,i}(j);\qquad S_{k,i}(j)=\sum_{n\in I_k}\phi_k(t_n)(Y^i_{n+j}-Y^i_n)^2 \tag*{(21)} b k , i σ , ν = j = 2 ∑ j m a x w j S k , i ( j ) ; S k , i ( j ) = n ∈ I k ∑ ϕ k ( t n ) ( Y n + j i − Y n i ) 2 (21)
with ( w j ) (w_j) ( w j ) the fixed weights that read the slope off a ( 1 , j , j 2 ) (1,j,j^2) ( 1 , j , j 2 ) fit across lags j = 2 , … , j max j=2,\ldots,j_{\max} j = 2 , … , j m a x . Every lag uses the same anchors n ∈ I k n\in I_k n ∈ I k , so the total test-function weight does not depend on j j j and the slope carries no lag-dependent truncation, while increments anchored near the end of the window reach up to j max j_{\max} j m a x steps past it. The design A σ A^\sigma A σ is unchanged, and the derivation and defaults are given in Appendix § 13 . The multilag estimator needs no noise subtraction and no drift correction, and it is insensitive to additive distortions of the increment confined to lags below the first fitted one, j min Δ t j_{\min}\Delta t j m i n Δ t (§ 4.6 ). Its variance is comparable to the lag-one estimator’s at moderate noise and larger at very low noise. The multilag estimator can also serve as an internal model check, since under Eq. (2) the two diffusion estimators agree up to finite-lag, discretization, and sampling error.
Both diffusion estimators are governed by the per-component noise-to-signal ratio ϱ i = 2 R i i / ( σ i 2 Δ t ) \varrho_i=2R_{ii}/(\sigma_i^2\Delta t) ϱ i = 2 R ii / ( σ i 2 Δ t ) of the increment variance. (When every component of a system shares a common scale, as in all our examples, we drop the subscript and write a single ϱ \varrho ϱ .) As we will see, the multilag fit recovers R ^ \widehat R R accurately in all our regimes, because the quadratic coefficient absorbs the drift bias even for the chaotic flow at ϱ ≈ 50 \varrho\approx50 ϱ ≈ 50 . We also show that the diffusion estimators are accurate to within Monte Carlo error in regimes with ϱ ≲ 20 \varrho\lesssim20 ϱ ≲ 20 . Beyond that noise-to-signal ratio, however, their spread and their bias grow with ϱ \varrho ϱ , and for chaotic systems such as Lorenz-63 at ϱ ≈ 50 \varrho\approx50 ϱ ≈ 50 the spread can reach 20% per component and the y , z y,z y , z estimates are biased high (§ 7 ). Sub-sampling from a finer grid, when available, lowers ϱ \varrho ϱ and can restore precision, provided the coarsened step still resolves the drift. Several of the systems of § 7 illustrate performance on thinned grids.
Derivation of the quadratic-variation system
For component i i i of Eq. (1), d X t i = f i ( X t ) d t + σ i ( X t ) d W t i \mathrm{d}X_t^i=f_i(X_t)\,\mathrm{d}t+\sigma_i(X_t)\,\mathrm{d}W_t^i d X t i = f i ( X t ) d t + σ i ( X t ) d W t i , the quadratic variation is d [ X i , X i ] t = ∑ l ν i l h l ( X t ) d t \mathrm{d}[X^i,X^i]_t=\sum_l\nu_{il}h_l(X_t)\,\mathrm{d}t d [ X i , X i ] t = ∑ l ν i l h l ( X t ) d t , and testing against ϕ k \phi_k ϕ k gives
∫ a k b k ϕ k ( t ) d [ X i , X i ] t = ∑ l ν i l ∫ a k b k ϕ k ( t ) h l ( X t ) d t . (22) \int_{a_k}^{b_k}\phi_k(t)\,\mathrm{d}[X^i,X^i]_t=\sum_l\nu_{il}\int_{a_k}^{b_k}\phi_k(t)h_l(X_t)\,\mathrm{d}t. \tag*{(22)} ∫ a k b k ϕ k ( t ) d [ X i , X i ] t = l ∑ ν i l ∫ a k b k ϕ k ( t ) h l ( X t ) d t . (22)
The right-hand side is ∑ l ν i l A k , l σ \sum_l\nu_{il}A^\sigma_{k,l} ∑ l ν i l A k , l σ up to the O ( Δ t ) O(\Delta t) O ( Δ t ) quadrature error, with h ~ l ( Y n ) \widetilde h_l(Y_n) h l ( Y n ) in place of h l ( X t n ) h_l(X_{t_n}) h l ( X t n ) to remove the observation-noise blur. The left-hand side is the limit of ∑ n ∈ I k ϕ ‾ k ( t n ) ( Δ X n i ) 2 \sum_{n\in I_k}\overline\phi_k(t_n)(\Delta X_n^i)^2 ∑ n ∈ I k ϕ k ( t n ) ( Δ X n i ) 2 , the increment over [ t n , t n + 1 ] [t_n,t_{n+1}] [ t n , t n + 1 ] carrying the step average ϕ ‾ k ( t n ) \overline\phi_k(t_n) ϕ k ( t n ) of the test function. Three corrections turn ( Δ X n i ) 2 (\Delta X_n^i)^2 ( Δ X n i ) 2 into the observable ( e n i ) 2 (e_n^i)^2 ( e n i ) 2 . The increment contains the drift, which the trapezoidal step 1 2 [ f i ( X t n ) + f i ( X t n + 1 ) ] Δ t \frac{1}{2}[f_i(X_{t_n})+f_i(X_{t_{n+1}})]\Delta t 2 1 [ f i ( X t n ) + f i ( X t n + 1 )] Δ t removes to O ( Δ t 2 ) O(\Delta t^2) O ( Δ t 2 ) in mean, leaving the martingale part with conditional second moment σ i 2 ( X t n ) Δ t + O ( Δ t 2 ) \sigma_i^2(X_{t_n})\Delta t+O(\Delta t^2) σ i 2 ( X t n ) Δ t + O ( Δ t 2 ) (the left-endpoint rule would leave a cross term between the Brownian increment and the within-step change of the drift, of mean f i ′ σ i 2 Δ t 2 f_i'\sigma_i^2\Delta t^2 f i ′ σ i 2 Δ t 2 , that is, of relative size f i ′ Δ t f_i'\Delta t f i ′ Δ t , which the trapezoidal step cancels, as well as the deterministic term 1 4 ∣ J f i ∣ 2 Δ t 4 \frac{1}{4}|Jf_i|^2\Delta t^4 4 1 ∣ J f i ∣ 2 Δ t 4 above). The observations replace X X X by Y = X + η Y=X+\eta Y = X + η , so e n i e_n^i e n i carries η n + 1 i ( 1 − 1 2 f i ′ Δ t ) − η n i ( 1 + 1 2 f i ′ Δ t ) \eta_{n+1}^i(1-\frac{1}{2}f_i'\Delta t)-\eta_n^i(1+\frac{1}{2}f_i'\Delta t) η n + 1 i ( 1 − 2 1 f i ′ Δ t ) − η n i ( 1 + 2 1 f i ′ Δ t ) to first order, of variance 2 R i i + O ( R Δ t 2 ) 2R_{ii}+O(R\Delta t^2) 2 R ii + O ( R Δ t 2 ) , the term 2 R ^ i i 2\widehat R_{ii} 2 R ii subtracted in Eq. (17) ; the symmetric weights cancel the cross term with η n i \eta_n^i η n i , which a left-endpoint drift would leave as a bias of + 2 Δ t f i ′ R i i +2\Delta t f_i'R_{ii} + 2Δ t f i ′ R ii per step. The cross terms between the martingale part and the observation errors have mean zero, and the drift-estimation error contributes O ( Δ t ∥ θ ^ − θ ∥ ) O(\Delta t\|\widehat\theta-\theta\|) O ( Δ t ∥ θ − θ ∥ ) . Collecting terms, E [ ( e n i ) 2 ∣ X t n ] = σ i 2 ( X t n ) Δ t + 2 R i i + O ( Δ t 2 ) + O ( R Δ t 2 ) \mathbb{E}[(e_n^i)^2\mid X_{t_n}]=\sigma_i^2(X_{t_n})\Delta t+2R_{ii}+O(\Delta t^2)+O(R\Delta t^2) E [( e n i ) 2 ∣ X t n ] = σ i 2 ( X t n ) Δ t + 2 R ii + O ( Δ t 2 ) + O ( R Δ t 2 ) , and summing with the weights ϕ ‾ k ( t n ) \overline\phi_k(t_n) ϕ k ( t n ) yields
b k , i σ = ∑ l ν i l A k , l σ + ε k , i σ + ρ k , i σ , (23) b_{k,i}^{\sigma}=\sum_l\nu_{il}A_{k,l}^{\sigma}+\varepsilon_{k,i}^{\sigma}+\rho_{k,i}^{\sigma}, \tag*{(23)} b k , i σ = l ∑ ν i l A k , l σ + ε k , i σ + ρ k , i σ , (23)
with ε k , i σ \varepsilon_{k,i}^{\sigma} ε k , i σ a sum of mean-zero terms, martingale differences and a one-dependent noise part, and ρ k , i σ \rho_{k,i}^{\sigma} ρ k , i σ the O ( Δ t ) O(\Delta t) O ( Δ t ) remainders, the error of R ^ \widehat{R} R and the debiasing remainder. Stacked over window pairs and test functions this is the system Eq. (17) . For a constant diffusion term candidate the regressor A σ A^\sigma A σ is deterministic; for a state-dependent term such as h ( x ) = x h(x)=x h ( x ) = x of the Cox–Ingersoll–Ross model it is built from the same increments as ϵ k , i σ \epsilon^\sigma_{k,i} ϵ k , i σ , the endogeneity of the drift case, and the disjoint-window instrument Z k , l σ Z^\sigma_{k,l} Z k , l σ restores exogeneity by Proposition 2. The subtraction of 2 R ^ i i 2\widehat{R}_{ii} 2 R ii is where the observation noise enters at order one, the variance of ( e i n ) 2 − 2 R ^ i i (e_i^n)^2-2\widehat{R}_{ii} ( e i n ) 2 − 2 R ii being of order ( σ i 2 Δ t + 2 R i i ) 2 (\sigma_i^2\Delta t+2R_{ii})^2 ( σ i 2 Δ t + 2 R ii ) 2 against a signal σ i 2 Δ t \sigma_i^2\Delta t σ i 2 Δ t , which is why the precision of the lag-one form is governed by ϱ i = 2 R i i / ( σ i 2 Δ t ) \varrho_i=2R_{ii}/(\sigma_i^2\Delta t) ϱ i = 2 R ii / ( σ i 2 Δ t ) .
The multilag slope estimator
The slope response Eq. (21) regresses the test-function-weighted squared j j j -increments on ( 1 , j , j 2 ) (1,j,j^2) ( 1 , j , j 2 ) and keeps the linear coefficient. Its properties follow from the small-time expansion of the variance of an observed increment. For a stationary diffusion observed with the noise of Eq. (2), and for a lag u = j Δ t u=j\Delta t u = j Δ t ,
E [ ( Y t + u i − Y t i ) 2 ] = 2 R i i + σ ‾ i 2 u + c 2 , i u 2 + c 3 , i u 3 + O ( u 4 ) , (24) \mathbb{E}\left[(Y^i_{t+u}-Y^i_t)^2\right]=2R_{ii}+\overline{\sigma}_i^2u+c_{2,i}u^2+c_{3,i}u^3+O(u^4), \tag*{(24)} E [ ( Y t + u i − Y t i ) 2 ] = 2 R ii + σ i 2 u + c 2 , i u 2 + c 3 , i u 3 + O ( u 4 ) , (24)
where σ ‾ i 2 = E [ σ i 2 ( X t ) ] \overline{\sigma}_i^2=\mathbb{E}[\sigma_i^2(X_t)] σ i 2 = E [ σ i 2 ( X t )] is the stationary mean of the i i i th diagonal diffusion coefficient of § 4 , the quantity the slope estimates, and c 2 , i c_{2,i} c 2 , i and c 3 , i c_{3,i} c 3 , i are stationary moments of the drift and its derivatives. The constant term is the only place where the observation noise enters, since the two errors in the increment are independent of the path and of each other. For the OU model the expansion is exact, E [ ( Y t + u − Y t ) 2 ] = 2 R + 2 v ( 1 − e − κ u ) \mathbb{E}[(Y_{t+u}-Y_t)^2]=2R+2v(1-e^{-\kappa u}) E [( Y t + u − Y t ) 2 ] = 2 R + 2 v ( 1 − e − κ u ) with v = σ 2 / 2 κ v=\sigma^2/2\kappa v = σ 2 /2 κ , so c 2 = − σ 2 κ / 2 c_2=-\sigma^2\kappa/2 c 2 = − σ 2 κ /2 and c 3 = σ 2 κ 2 / 6 c_3=\sigma^2\kappa^2/6 c 3 = σ 2 κ 2 /6 . Observation noise enters Eq. (24) only through the u u u -independent level, which the intercept absorbs, and so does any additive distortion whose correlation vanishes beyond the first fitted lag, Cov ( η t + u , η t ) = 0 \operatorname{Cov}(\eta_{t+u},\eta_t)=0 Cov ( η t + u , η t ) = 0 for u ≥ j min Δ t u\ge j_{\min}\Delta t u ≥ j m i n Δ t . An error correlation c ( u ) c(u) c ( u ) of longer range adds 2 R i i [ 1 − c ( u ) ] 2R_{ii}[1-c(u)] 2 R ii [ 1 − c ( u )] , which varies across the fitted lags and enters the slope, so the immunity is to distortions confined below j min Δ t j_{\min}\Delta t j m i n Δ t , a condition that § 8.2 checks against the Einstein value rather than assumes.
The fit on ( 1 , j , j 2 ) (1,j,j^2) ( 1 , j , j 2 ) is exact for the first three terms of Eq. (24) , so the intercept returns 2 R i i 2R_{ii} 2 R ii , the linear coefficient returns σ ‾ i 2 Δ t \overline{\sigma}_i^2\Delta t σ i 2 Δ t , and the quadratic coefficient absorbs c 2 , i u 2 c_{2,i}u^2 c 2 , i u 2 , the drift contamination. The only bias of the slope is therefore the part of the cubic term c 3 , i u 3 c_{3,i}u^3 c 3 , i u 3 that the fit assigns to the linear coefficient. Regressing j 3 j^3 j 3 on ( 1 , j , j 2 ) (1,j,j^2) ( 1 , j , j 2 ) over j ∈ { 2 , … , j max } j\in\{2,\ldots,j_{\max}\} j ∈ { 2 , … , j m a x } gives that part in closed form, and the relative bias of σ ‾ i 2 \overline{\sigma}_i^2 σ i 2 is c ( c 3 , i / σ ‾ i 2 ) ( j max Δ t ) 2 c(c_{3,i}/\overline{\sigma}_i^2)(j_{\max}\Delta t)^2 c ( c 3 , i / σ i 2 ) ( j m a x Δ t ) 2 with a design constant c c c of order one. For OU this is c ( κ j max Δ t ) 2 / 6 c(\kappa j_{\max}\Delta t)^2/6 c ( κ j m a x Δ t ) 2 /6 , about 3 × 10 − 3 3\times10^{-3} 3 × 1 0 − 3 at the defaults, and the rule j max Δ t ≲ 0.1 τ j_{\max}\Delta t\lesssim0.1\tau j m a x Δ t ≲ 0.1 τ keeps it negligible in general, while j max ≤ L / 2 j_{\max}\le L/2 j m a x ≤ L /2 keeps the increments anchored in the window from reaching far beyond it. For the double-well benchmark the two drift term candidates can be written explicitly. The noise adds only the constant 2 R 2R 2 R to Eq. (24) , so c 2 c_2 c 2 and c 3 c_3 c 3 are those of the path itself, and for a reversible scalar diffusion with constant σ \sigma σ they are stationary averages of the drift, c 2 = − E [ f ( X ) 2 ] c_2=-\mathbb{E}[f(X)^2] c 2 = − E [ f ( X ) 2 ] and c 3 = − 1 3 E [ f ( X ) ( f f ′ + 1 2 σ 2 f ′ ′ ) ( X ) ] c_3=-\frac{1}{3}\mathbb{E}\left[f(X)\left(ff'+\frac{1}{2}\sigma^2f''\right)(X)\right] c 3 = − 3 1 E [ f ( X ) ( f f ′ + 2 1 σ 2 f ′′ ) ( X ) ] . On the double-well stationary law both equal 0.195 0.195 0.195 , so at j = j max j=j_{\max} j = j m a x the quadratic term is ∣ c 2 ∣ j max Δ t / σ 2 ≈ 6 % |c_2|j_{\max}\Delta t/\sigma^2\approx6\% ∣ c 2 ∣ j m a x Δ t / σ 2 ≈ 6% of the linear signal and the cubic term 0.5%; the fit removes the first exactly and leaves a slope bias of 0.5% from the second.
Per window the same expansion holds conditionally on the path, with σ i 2 ( X t n ) \sigma_i^2(X_{t_n}) σ i 2 ( X t n ) at each anchor in place of σ ‾ i 2 \overline{\sigma}_i^2 σ i 2 , so the linear coefficient of S k , i ( j ) S_{k,i}(j) S k , i ( j ) is Δ t ∑ n ∈ Z k ϕ k ( A k , l σ , σ i 2 ( X t n ) ) = ∑ l ν l , i A k , l σ \Delta t\sum_{n\in\mathcal{Z}_k}\phi_k(A^\sigma_{k,l},\sigma_i^2(X_{t_n}))=\sum_l\nu_{l,i}A^\sigma_{k,l} Δ t ∑ n ∈ Z k ϕ k ( A k , l σ , σ i 2 ( X t n )) = ∑ l ν l , i A k , l σ up to the quadratic remainder. The slope response therefore obeys the moment equation Eq. (23) and recovers the state-dependent coefficients ν \nu ν , while the pooled fit Eq. (18) gives only the stationary mean of σ ‾ i 2 \overline{\sigma}_i^2 σ i 2 . Since increments anchored near the end of the target window reach up to j max j_{\max} j m a x steps into the next instrument window, consecutive pair scores ξ k = Z k ϵ k \xi_k=Z_k\epsilon_k ξ k = Z k ϵ k are correlated at lag one. This does not affect exogeneity within a pair, because the instrument still precedes its own target. It changes only the variance. For a stationary sequence with correlation at lag one the central limit theorem still holds, with Σ \Sigma Σ replaced by the long-run covariance Σ + Σ 1 + Σ 1 ⊤ \Sigma+\Sigma_1+\Sigma_1^\top Σ + Σ 1 + Σ 1 ⊤ , Σ 1 = Cov ( ξ k , ξ k + 1 ) \Sigma_1=\operatorname{Cov}(\xi_k,\xi_{k+1}) Σ 1 = Cov ( ξ k , ξ k + 1 ) , and this long-run covariance is exactly what the Newey–West plug-in of Appendix D estimates.
The slope form carries a variance cost, since a slope fitted through j max − 1 j_{\max}-1 j m a x − 1 correlated points is noisier than a single lag-one level. The lag-one estimator, however, must subtract 2 R ^ 2\widehat{R} 2 R , and that subtraction is itself noisy when 2 R i i 2R_{ii} 2 R ii is large against σ ‾ i 2 Δ t \overline{\sigma}_i^2\Delta t σ i 2 Δ t . The two are therefore about equally precise on the noisy synthetic benchmarks, while the slope estimate is the noisier one on the low-noise optical-trap record (Table 3 , § 8.2 ). What the slope provides is immunity to the short-lag distortions of a real detector, which is why § 8.2 uses it.
Table 3. Diffusion recovery at the operative window L = 20 L=20 L = 20 and the main-analysis noise level (using the fixed variance R R R per system of Table 5). Entries are the estimator average and standard deviation in parentheses, taken over M = 1000 M=1000 M = 1000 single-trajectory estimates (M = 200 M=200 M = 200 for Lorenz-96), each using its trajectory’s own R ^ \hat{R} R ^ , at depth one. QV denotes the lag-one quadratic-variation system Eq. (17) , and slope denotes the multilag slope Eq. (21) , replaced by the two-lag slope for the Langevin p p p component. True values appear in the second column. System abbreviations follow Table 1 .
Generalized method of moments
The estimator Eq. (6) is exactly identified. It uses one instrument column per regressor, so the number of moment conditions equals the number of unknowns, and the solution exists and is unique whenever Z T A Z^\mathsf{T}A Z T A is invertible. One can instead use more instruments than parameters, so that Z Z Z has more columns than A A A and the moment vector m ( θ ) = Z T ( b − A θ ) m(\theta)=Z^\mathsf{T}(b-A\theta) m ( θ ) = Z T ( b − A θ ) has more conditions than parameters. This can happen when we create multiple disjoint instrument windows per target window, or when more drift term candidates enter on the instrument side than on the target side. The system is then over-identified , and the efficient combination of moment conditions is the generalized method of moments (GMM) [36 ] :
θ ^ G M M = ( A T Z W Z T A ) − 1 A T Z W Z T b . (25) \widehat{\theta}_{\mathrm{GMM}}=(A^\mathsf{T}ZWZ^\mathsf{T}A)^{-1}A^\mathsf{T}ZWZ^\mathsf{T}b. \tag*{(25)} θ GMM = ( A T Z W Z T A ) − 1 A T Z W Z T b . (25)
The optimal weight matrix
W ^ = ( 1 K ∑ k Z k ϵ ^ k ϵ ^ k T Z k T ) − 1 (26) \widehat{W}=\left(\frac{1}{K}\sum_k Z_k\widehat{\epsilon}_k\widehat{\epsilon}_k^{\mathsf{T}}Z_k^{\mathsf{T}}\right)^{-1} \tag*{(26)} W = ( K 1 k ∑ Z k ϵ k ϵ k T Z k T ) − 1 (26)
is estimated from first-step residuals ϵ ^ k = b k − A k θ ^ 2 S L S \widehat{\epsilon}_k=b_k-A_k\widehat{\theta}_{2\mathrm{SLS}} ϵ k = b k − A k θ 2 SLS , obtained from the two-stage least-squares estimator
θ ^ 2 S L S = ( A T P Z A ) − 1 A T P Z b ; P Z = Z ( Z T Z ) − 1 Z T . (27) \widehat{\theta}_{2\mathrm{SLS}}=(A^{\mathsf{T}}P_ZA)^{-1}A^{\mathsf{T}}P_Zb;\qquad P_Z=Z(Z^{\mathsf{T}}Z)^{-1}Z^{\mathsf{T}}. \tag*{(27)} θ 2 SLS = ( A T P Z A ) − 1 A T P Z b ; P Z = Z ( Z T Z ) − 1 Z T . (27)
The resulting estimator has the smallest asymptotic variance among GMM estimators based on the given moment conditions [36 ] . At the default depth n l a g s = 1 n_{\mathrm{lags}}=1 n lags = 1 , meaning one past instrument block per target window (Remark 3), the system is exactly identified provided the instrument window carries the same drift term candidates and test-function bank as the target (Eq. (25) then reduces to Eq. (6) ). The two-step estimator Eq. (25) operates whenever the depth rule of Remark 3 raises n l a g s n_{\mathrm{lags}} n lags to two.
Remark 3. For the test functions in both roles, ϕ k \phi_k ϕ k on the target window and ψ k \psi_k ψ k on the instrument window, we use the compactly supported bump functions of Messenger and Bortz [2 ] , w p ( t ) ∝ ( t − a k ) p ( a k + T w − t ) p 1 [ a k , a k + T w ] ( t ) w_p(t) \propto(t-a_k)^p(a_k+T_w-t)^p\mathbf{1}_{[a_k,a_k+T_w]}(t) w p ( t ) ∝ ( t − a k ) p ( a k + T w − t ) p 1 [ a k , a k + T w ] ( t ) . We evaluate a bank of M ϕ = 3 M_\phi=3 M ϕ = 3 test functions per window pair, with smoothness exponents p ∈ { 4 , 6 , 8 } p \in\{4,6,8\} p ∈ { 4 , 6 , 8 } ; exponents up to p = 16 p=16 p = 16 perform comparably. To satisfy exogeneity we tile the observation grid into consecutive disjoint blocks of L L L observations (span L Δ t L\Delta t L Δ t ) and pair them sequentially. With n l a g s = 1 n_{\mathrm{lags}}=1 n lags = 1 the instrument is integrated over one block and the target over the next; with n l a g s = 2 n_{\mathrm{lags}}=2 n lags = 2 the instrument spans the two preceding blocks. At n l a g s = 1 n_{\mathrm{lags}}=1 n lags = 1 the moment system is exactly identified, and when relevance (IV1) is marginal the estimator inherits the usual weak-instrument bias toward the least-squares limit [46 ] , which for the reversible systems of Proposition 1 is zero. Doubling the instrument set makes the system over-identified and brings in the GMM weighting. We use n l a g s = 1 n_{\mathrm{lags}}=1 n lags = 1 by default, and raise the depth to two only when the diagnostics defined in § 7 flag a weak or contaminated first stage. Instrument and target are further separated by a gap of δ ≥ 0 \delta\ge0 δ ≥ 0 , an integer multiple of Δ t \Delta t Δ t . A positive gap is not required for exogeneity, which holds even at δ = 0 \delta=0 δ = 0 , the setting used on the short climate record of § 8.1 . Rather, it buffers the instrument against short-range correlation in the measurement noise. The guiding principle is to take δ \delta δ beyond the noise correlation length while keeping it far below τ \tau τ . A gap of a few steps covers short-range noise correlation at negligible variance cost; longer noise correlation calls for a longer gap. We use δ = 4 Δ t \delta=4\Delta t δ = 4Δ t on the benchmarks, lengthened to 8 Δ t 8\Delta t 8Δ t on the low-pass-filtered trap record, following the design rule of Appendix § 12 (see Figure 1 ). The test-function banks are illustrated in Figure 2 .
Figure 2. Test-function banks (shown on a rescaled window). Top: the powers bank ϕ P ( t ) ∝ [ ( t − a ) ( b − t ) ] P \phi_P(t) \propto[(t-a)(b-t)]^P ϕ P ( t ) ∝ [( t − a ) ( b − t ) ] P for P ∈ { 4 , … , 10 } P \in\{4,\ldots,10\} P ∈ { 4 , … , 10 } , with larger P P P concentrating the bump toward the window center. Bottom: the centered bank, one compactly supported bump per interior node, for two smoothness settings P ∈ { 8 , 12 } P \in\{8,12\} P ∈ { 8 , 12 } . Both banks supply the M ϕ M_\phi M ϕ test functions per window pair (Remark 3). The main results use the powers bank with P ∈ { 4 , 6 , 8 } P \in\{4,6,8\} P ∈ { 4 , 6 , 8 } .
The WAIVES algorithm
Algorithm 1 assembles the preceding constructions into a complete procedure for estimating drift and diffusion parameters θ \theta θ and ν \nu ν from the data { Y t n } n = 0 N \{Y_{t_n}\}_{n=0}^{N} { Y t n } n = 0 N , which carry both process and measurement noise.
Note that a nonlinear term evaluated at a noisy observation is biased even for symmetric noise, since E [ g ( Y ) ∣ X ] \mathbb{E}[g(Y)\mid X] E [ g ( Y ) ∣ X ] represents g g g averaged over the noise density. This averaging of g g g results in a bias (the classical errors-in-variables bias of nonlinear regression) proportional to R R R and to the curvature of g g g (for instance E [ Y 2 ∣ X ] = X 2 + R \mathbb{E}[Y^2\mid X]=X^2+R E [ Y 2 ∣ X ] = X 2 + R ). The debiased term is constructed score [47 , 48 , 49 ] , computed via the deconvolution g ~ = e − 1 2 R ^ ∂ 2 g \widetilde{g}=e^{-\frac{1}{2}\widehat{R}\partial^2}g g = e − 2 1 R ∂ 2 g , resulting in adjustment to monomials (Y 2 − R ^ Y^2-\widehat{R} Y 2 − R , Y 3 − 3 R ^ Y Y^3-3\widehat{R}Y Y 3 − 3 R Y , and so on). General terms, including trigonometric ones, are handled by a regularized Gauss–Hermite rule (Appendix § 13 ).
Numerical experiments
In this section we test the performance of WAIVES on nine systems. Table 1 summarizes each system’s governing equations and the difficulty it probes. Each system is described below, and the simulation protocol is given in Appendix § 13 . Example trajectories at two noise levels of every system are shown in Figure 3 .
Figure 3. Representative latent trajectory (blue) and noisy observations (orange) for the nine systems of Table 1 , at the main-analysis noise level (left) and at the stress level (right). The two levels, a fixed noise variance R R R per system and a noise standard deviation of 15% of the component’s range, are listed in Table 5 . Kuramoto is shown as sin θ 1 \sin\theta_1 sin θ 1 with the noise applied to the phase.
To test the performance of the WAIVES estimator over replications, we simulate M = 1000 M=1000 M = 1000 independent trajectories for each model by Euler–Maruyama on a grid finer than the observation step (Appendix § 13 ), producing M M M sets of observations { Y t n } n = 0 N \{Y_{t_n}\}_{n=0}^{N} { Y t n } n = 0 N , with added i.i.d. Gaussian measurement noise with covariance R R R . R R R is fixed per
Algorithm 1 WAIVES, weak-form IV/GMM estimation for SDEs
Require: Observations { Y t n } n = 0 N \{Y_{t_n}\}_{n=0}^{N} { Y t n } n = 0 N ; drift term candidates { g j } j = 1 p \{g_j\}_{j=1}^{p} { g j } j = 1 p and diffusion term candidates { h l } l = 1 q \{h_l\}_{l=1}^{q} { h l } l = 1 q ; target test functions { ϕ k } \{\phi_k\} { ϕ k } and instrument test functions { ψ k } \{\psi_k\} { ψ k } , shared by the drift system Eq. (8) –Eq. (10) and the diffusion system Eq. (17) .
Noise covariance. Estimate R ^ \widehat{R} R from the multilag variogram of Section 4 (default n v = 6 n_{\mathrm{v}}=6 n v = 6 ), then project onto the positive-semidefinite cone.
Weak-form system from raw Y t n Y_{t_n} Y t n . Build b k b_k b k , A k j A_{kj} A k j over [ a k , a k + T w ] [a_k,a_k+T_w] [ a k , a k + T w ] and Z k j Z_{kj} Z k j over the disjoint window [ a k ′ , a k ′ + T w ] [a'_k,a'_k+T_w] [ a k ′ , a k ′ + T w ] via Eq. (8) –Eq. (10) ; debias polynomial drift term candidates for observation noise (rules in the text below).
Drift estimate. If exactly identified, compute θ ^ I V \widehat{\theta}_{\mathrm{IV}} θ IV by Eq. (6) . If over-identified, run GMM, (i) θ ^ 2 S L S \widehat{\theta}_{\mathrm{2SLS}} θ 2SLS by Eq. (27) ; (ii) residuals ε ^ k = b k − A k θ ^ 2 S L S \widehat{\varepsilon}_k=b_k-A_k\widehat{\theta}_{\mathrm{2SLS}} ε k = b k − A k θ 2SLS ; (iii) W ^ \widehat{W} W by Eq. (26) ; (iv) θ ^ G M M \widehat{\theta}_{\mathrm{GMM}} θ GMM by Eq. (25) .
Iterative reweighting. Recompute residuals, update the covariance, re-solve step 3; repeat to convergence (at most 10 iterations). In the exactly identified configuration used for the main-text tables the solution does not depend on the weighting, so this step leaves the estimate unchanged; it is active in the over-identified runs.
Diffusion estimate. With θ ^ \widehat{\theta} θ fixed, build the quadratic-variation system Eq. (17) on the same windows and instruments, using the multilag slope response Eq. (21) whenever the two responses disagree beyond their standard errors or the variogram intercept of Section 4 is fitted negative, and solve for the diffusion coefficients by the same IV/GMM steps.
Ensure: Drift parameters estimate θ ^ \widehat{\theta} θ and diffusion parameters estimate ν ^ \widehat{\nu} ν .
system rather than set relative to the signal, so relative to each observed component’s stationary standard deviation the main analyses span noise levels from 0.1% to 30% (Table 5 ). Measurements lengths differ across systems, each spanning a multiple of the system’s characteristic time. Every experiment shares the bank and depth rule of Remark 3 and the gap δ = 4 Δ t \delta=4\Delta t δ = 4Δ t (Appendix § 13 ). Reported estimates are by default based on the exactly identified IV (n l a g s = 1 n_{\mathrm{lags}}=1 n lags = 1 ), except where the depth rule raises n l a g s n_{\mathrm{lags}} n lags to two. In the latter case the moment system is over-identified, and the two-step GMM estimator Eq. (25) is used instead (Section 5).
Table 5. Observation-noise levels for each system. Each row lists the observed component, the fixed noise variance R R R of the main analyses, the component’s stationary standard deviation (SD, computed from one long simulation after burn-in), and R \sqrt{R} R as a percentage of that SD. The last two columns give the stress level values of Table 6 and Table 7 : the variance R s R_s R s , fixed per component as the square of 15% of the component’s range (where the range is taken over one reference record), and R s \sqrt{R_s} R s as a percentage of the same SD. The Kuramoto row is for sin θ 1 \sin\theta_1 sin θ 1 with the noise applied to the phase. The observed Lorenz-63 value is per sensor. FHN stands for FitzHugh–Nagumo, CIR for Cox–Ingersoll–Ross, and Langevin for underdamped Langevin.
It is important to note that every drift and diffusion estimate is a single-trajectory estimate, and each of the M M M trajectories yields one estimate from its own record. The tables report the average and standard deviation of the M M M estimates, as estimator mean and standard error, respectively. Single-trajectory uncertainty is quantified by Wald intervals built on the plug-in GMM covariance (Appendix § 14 ), with a Newey–West long-run correction across window pairs. On OU, the double-well, Lorenz-63 and FitzHugh–Nagumo these intervals attain near-nominal coverage at the operative record lengths, and on OU and the double-well also at records one-eighth as long; bootstrap intervals do not improve on them (Appendix § 14 ).
When the support span T w = L Δ t T_w=L\Delta t T w = L Δ t exceeds the characteristic time τ \tau τ of the dynamics (the local relaxation time for the dissipative models, and for the chaotic models the 1 / e 1/e 1/ e decorrelation time, computed here from long simulated trajectories and in applications estimated from the record’s autocorrelation), the state decorrelates across the disjoint windows, relevance (IV1) degrades, and the variance of θ ^ \widehat{\theta} θ increases sharply. For this reason, we compare L = 100 L=100 L = 100 , 20, and 5 throughout (the full sweep is given in Figure 4 ). Panel (c) of Figure 4 shows the standard deviation rising as the gap δ \delta δ grows, underscoring the need for the disjoint windows to stay close enough to remain mutually informative. However, even though δ = 0 \delta=0 δ = 0 would be acceptable (Section 5), in practice a small but positive δ \delta δ can guard against short-range correlation in the measurement noise, with a relatively low effect on variance.
Figure 4. Window and gap diagnostics , one representative parameter per system (M = 1000 M = 1000 M = 1000 ). (a) Estimator standard deviation versus span T w = L Δ t T_w = L\Delta t T w = L Δ t . Each curve turns sharply upward as the span approaches the system’s characteristic time τ \tau τ (dashed vertical lines, values in Appendix § 12 ). (b) The companion first-stage Cragg–Donald F F F , which rises as the window shrinks. The dashed line marks the F = 10 F = 10 F = 10 threshold. (c) Standard deviation versus the instrument–target gap δ \delta δ at L = 20 L = 20 L = 20 . Accuracy degrades as the gap grows and the instrument leaves the system’s memory (the dashed line marks the default δ = 4 Δ t \delta= 4\Delta t δ = 4Δ t ). FitzHugh–Nagumo is shown at depth two (two instrument windows per target window), as selected by the depth rule, since its depth-one first stage falls below F = 10 F = 10 F = 10 once the gap exceeds 6 Δ t 6\Delta t 6Δ t . Together the panels depict the operating logic behind the design rule of Appendix § 12 .
Instrument strength is monitored by the first-stage Cragg–Donald F F F [50 ] (Appendix § 12 ), with F ≥ 10 F\geq10 F ≥ 10 treated as trustworthy relevance [46 , 51 ] ; below 10 at depth one (n l a g s = 1 n_{\mathrm{lags}}=1 n lags = 1 ) the depth rule of Remark 3 is applied. This rule picks the depth from the data itself, a step the theorems do not cover, so we test its effect directly in the coverage study (Appendix § 14 ). As F F F measures strength only, we also recompute the estimate at depth two. Under valid instruments both depths are consistent for the same parameters and agree to within their standard errors; a large disagreement indicates a residual correlation between the instrument and the error and calls for instruments beyond the contamination’s range (Section 8.2; Appendix § 12 ).
The nine systems
The nine systems follow in the order of Table 1 , each with its simulation settings. Together they stress the features that challenge a weak-form estimator, namely linear and bistable dynamics, chaos, slow–fast excitability, a state-dependent diffusion, a high-dimensional system, non-polynomial drift term candidates, degenerate (hypoelliptic) noise, and a non-identity observation map, and none of them requires any change to the estimator. All use the same disjoint-window IV/GMM procedure, recover drift and diffusion jointly, and, unless noted, inherit the settings above; the reported figures are estimator averages (std) over M = 1000 M=1000 M = 1000 replications at L = 20 L=20 L = 20 , and Table 2 and Table 3 collect every parameter.
Table | Drift recovery for all nine systems at the main-analysis noise level (using the fixed variance R R R per system of Table 5 ). Entries are the estimator average and standard deviation in parentheses, taken over M = 1000 M = 1000 M = 1000 single-trajectory estimates (M = 200 M = 200 M = 200 for Lorenz-96), each using its trajectory’s own R ^ \hat{R} R ^ , at raw IV depth one, with gap δ = 4 Δ t \delta= 4\Delta t δ = 4Δ t (depth-two comparisons appear in Table 6 ). The columns compare the window lengths L = 100 L = 100 L = 100 , 20 20 20 (the operative window), and 5 5 5 , with true values in the last column. The FitzHugh–Nagumo ratio parameters a a a , b b b are reported as the Marsaglia–Hinkley mode with a 95% Fieller interval (unbnd” means the interval is unbounded, indicating non-identifiability). The last row of each block gives the average first-stage F F F , with the percentage of replications whose own F F F falls below the weak-instrument threshold 10 in parentheses. Where the average F F F falls below 1, the first stage carries less signal than noise and we regard the inference as unreliable. These cells read n.i. (not identified). Note that at depth one the exactly identified estimator becomes heavy-tailed, and at higher depths the estimates collapse toward the least-squares limit, so no stable summary of the replications exists (Appendix § 14 ). System abbreviations follow Table 1 . Stress-level recovery appears in Table 6 .
Table 2.
Ornstein–Uhlenbeck
The scalar OU process d X t = κ ( μ − X t ) d t + σ d W t dX_t=\kappa(\mu-X_t)\,dt+\sigma\,dW_t d X t = κ ( μ − X t ) d t + σ d W t with κ = 1.5 , μ = 2 , σ = 0.8 \kappa=1.5,\mu=2,\sigma=0.8 κ = 1.5 , μ = 2 , σ = 0.8 is simulated at Δ t = 0.01 \Delta t=0.01 Δ t = 0.01 , N = 200 , 000 N=200,000 N = 200 , 000 (T = 2000 T=2000 T = 2000 ), R = 0.02 R=0.02 R = 0.02 . The drift term candidates are g ( x ) = [ 1 , x ] g(x)=[1,x] g ( x ) = [ 1 , x ] , so the true drift vector is [ κ μ , − κ ] = [ 3 , − 1.5 ] [\kappa\mu,-\kappa]=[3,-1.5] [ κ μ , − κ ] = [ 3 , − 1.5 ] , the diffusion term candidate is h ( x ) = [ 1 ] h(x)=[1] h ( x ) = [ 1 ] , and the characteristic time is the relaxation time τ = 1 / κ ≈ 0.67 s \tau=1/\kappa\approx0.67\,\mathrm{s} τ = 1/ κ ≈ 0.67 s . Table 2 shows unbiased recovery, with the smallest standard deviations at the intermediate and short windows. At L = 20 L=20 L = 20 and δ = 4 Δ t \delta=4\Delta t δ = 4Δ t this record gives K ≈ 4545 K\approx4545 K ≈ 4545 window pairs, K Δ t ≈ 0.67 \sqrt{K}\Delta t\approx0.67 K Δ t ≈ 0.67 , outside the rate condition of Theorem 2; the quadrature bias that this permits is nonetheless negligible here, the mean of κ ^ \widehat{\kappa} κ lying within 0.003 of the truth against a standard deviation of 0.13. Table 3 recovers σ \sigma σ within Monte Carlo error. This linear-Gaussian case admits an exact likelihood through the Kalman filter, so it calibrates the price of the weak-form route. A Kalman maximum-likelihood fit [52 ] of the same trajectories attains the Cramér–Rao bound, with standard deviations about three times smaller than WAIVES (Appendix § 15 ); the gap is the cost of never forming a likelihood. The computational comparison, by contrast, favors WAIVES, at 13 ms per trajectory for κ \kappa κ , μ \mu μ , and σ \sigma σ against 0.08–0.28 s for the maximum-likelihood fits, a factor of 6 to 22 (Appendix § 15 ).
Double-well (bistable)
d X t = ( α X t − β X t 3 ) d t + σ d W t dX_t=(\alpha X_t-\beta X_t^3)\,dt+\sigma\,dW_t d X t = ( α X t − β X t 3 ) d t + σ d W t with α = β = 1 \alpha=\beta=1 α = β = 1 , σ = 0.5 \sigma=0.5 σ = 0.5 has stable equilibria at ± 1 \pm1 ± 1 and a near-well relaxation time τ = 1 / ( 2 α ) = 0.5 s \tau=1/(2\alpha)=0.5\,\mathrm{s} τ = 1/ ( 2 α ) = 0.5 s ; the record’s overall autocorrelation time, ≈ 13 s \approx13\,\mathrm{s} ≈ 13 s , instead reflects rare well hopping, and it is the local time that governs instrument relevance. The diffusion term candidate is again h ( x ) = [ 1 ] h(x)=[1] h ( x ) = [ 1 ] . We use Δ t = 0.01 \Delta t=0.01 Δ t = 0.01 , N = 200 , 000 N=200,000 N = 200 , 000 , R = 0.02 R=0.02 R = 0.02 , drift term candidates g ( x ) = [ 1 , x , x 2 , x 3 ] g(x)=[1,x,x^2,x^3] g ( x ) = [ 1 , x , x 2 , x 3 ] , with the observation-noise debiasing of § 6 applied to the quadratic and cubic columns. Table 2 shows the bias flattening once the window exceeds the relaxation scale (L = 100 L=100 L = 100 corresponds to 1.0 s, twice τ \tau τ ), and Table 3 recovers σ \sigma σ within error. A strong-form baseline regressing raw Euler increments on the same drift term candidates [1 , 25 ] is corrupted by an order of magnitude at this noise level (α ^ = 11.1 \widehat{\alpha}=11.1 α = 11.1 against 1) while WAIVES stays unbiased (Appendix § 15 ), so the correction is measured against an established equation-discovery method, not only against weak-form least squares.
Stochastic Lorenz-63
d x = a ( y − x ) d t + σ d W 1 dx = a(y-x)\,dt+\sigma\,dW_1 d x = a ( y − x ) d t + σ d W 1 , d y = ( x ( b − z ) − y ) d t + σ d W 2 dy = (x(b-z)-y)\,dt+\sigma\,dW_2 d y = ( x ( b − z ) − y ) d t + σ d W 2 , d z = ( x y − c z ) d t + σ d W 3 dz = (xy-cz)\,dt+\sigma\,dW_3 d z = ( x y − cz ) d t + σ d W 3 , with a = 10 a = 10 a = 10 , b = 28 b = 28 b = 28 , c = 8 / 3 c = 8/3 c = 8/3 , σ = 1 \sigma= 1 σ = 1 , is chaotic with Lyapunov time 1 / λ max ≈ 1.1 1/\lambda_{\max} \approx1.1 1/ λ m a x ≈ 1.1 s; the operative characteristic time is the measured 1 / e 1/e 1/ e decorrelation τ ≈ 0.14 \tau\approx0.14 τ ≈ 0.14 s, far shorter because the driving noise erases memory along the attractor. The diffusion term candidate is h ( x ) = [ 1 ] h(x) = [1] h ( x ) = [ 1 ] per component. We use Δ t = 0.002 \Delta t = 0.002 Δ t = 0.002 , N = 200,000 N = 200{,}000 N = 200 , 000 (T = 400 T = 400 T = 400 ), R = 0.05 R = 0.05 R = 0.05 per component, degree-2 monomial drift term candidates (10 per component). Table 2 shows the intermediate window L = 20 L = 20 L = 20 is by far the most reliable. Over M = 1000 M = 1000 M = 1000 trajectories b ^ = 28.027 \hat{b} = 28.027 b ^ = 28.027 is within 0.1% of its true value, while a ^ = 10.028 \hat{a} = 10.028 a ^ = 10.028 and c ^ = 2.721 \hat{c} = 2.721 c ^ = 2.721 differ from truth by 0.3 and 2.0% (about three and five Monte Carlo standard errors), small biases consistent with finite-Δ t \Delta t Δ t discretization at the chaotic timescales. Here a a a is read from the y y y term of the x x x -equation; the x x x term of the same equation returns 10.093 (0.377), the same within errors, and carries a +0.9% finite-Δ t \Delta t Δ t offset that the y y y term is free of. The agreement is an internal consistency check available whenever a parameter enters more than one term. The long window’s variance increases sharply once T w T_w T w passes τ \tau τ , as the distribution of the c c c estimate shows (Figure 5 ).
Figure 5. Distribution of the Lorenz-63 c ^ \widehat{c} c across the M = 1000 M = 1000 M = 1000 replications, at three window lengths (clipped to [ − 10 , 20 ] [-10,20] [ − 10 , 20 ] for display). At L = 100 L = 100 L = 100 (span 0.2 s 0.2\,\mathrm{s} 0.2 s , a substantial fraction of the noise-shortened decorrelation time) the estimate is essentially uninformative, with a tail-dominated sample standard deviation of ≈ 306 \approx306 ≈ 306 . At L = 20 L = 20 L = 20 and L = 5 L = 5 L = 5 the distribution concentrates tightly on the true value c = 8 / 3 c = 8/3 c = 8/3 . Once the window exceeds the correlation time, the first stage collapses (average F < 1 F < 1 F < 1 ) and the exactly identified estimator becomes heavy-tailed, so that its average over the replications is governed by a few extreme draws. Shortening the window in such cases restores a tightly concentrated estimator.
Stochastic FitzHugh–Nagumo
d V t = ( V t − V t 3 / 3 − W t + I ) d t + σ V d B 1 , t dV_t = (V_t - V_t^3/3 - W_t + I)\,dt+\sigma_V\,dB_{1,t} d V t = ( V t − V t 3 /3 − W t + I ) d t + σ V d B 1 , t , d W t = ϵ ( V t + a − b W t ) d t + σ W d B 2 , t dW_t = \epsilon(V_t+a-bW_t)\,dt+\sigma_W\,dB_{2,t} d W t = ϵ ( V t + a − b W t ) d t + σ W d B 2 , t , with I = 0.5 I = 0.5 I = 0.5 , a = 0.7 a = 0.7 a = 0.7 , b = 0.8 b = 0.8 b = 0.8 , ϵ = 0.08 \epsilon= 0.08 ϵ = 0.08 , σ V = σ W = 0.3 \sigma_V = \sigma_W = 0.3 σ V = σ W = 0.3 , simulated at Δ t = 0.05 \Delta t = 0.05 Δ t = 0.05 , N = 100,000 N = 100{,}000 N = 100 , 000 (T = 5000 T = 5000 T = 5000 ), R = 0.05 R = 0.05 R = 0.05 ; drift term candidates { 1 , v , w , v 2 , v w , w 2 , v 3 } \{1,v,w,v^2,vw,w^2,v^3\} { 1 , v , w , v 2 , v w , w 2 , v 3 } , diffusion term candidate h = [ 1 ] h = [1] h = [ 1 ] per component, and a measured characteristic time τ ≈ 6 \tau\approx6 τ ≈ 6 s set by the excitable orbit. The slow equation’s first stage clears the F F F threshold at the operative window (Table 2 ), and the two depths agree within uncertainty, so the default depth one applies here as well. Table 2 shows the short and intermediate windows recovering all four drift parameters, while the longest window destabilizes the slow ( a , b ) (a,b) ( a , b ) recovery. The W W W -equation is parameterized as ϵ ( V + a − b W ) \epsilon(V+a-bW) ϵ ( V + a − bW ) , so a , b a,b a , b are ratios of estimated coefficients with a near-zero denominator ϵ ≈ 0.08 \epsilon\approx0.08 ϵ ≈ 0.08 . A naive per-trajectory ratio has no finite moments, so we report the Marsaglia–Hinkley ratio mode [53 , 54 ] with a Fieller [55 ] 95% interval, both built from the IV sampling covariance (Figure 6 ). At L = 20 L = 20 L = 20 and L = 5 L = 5 L = 5 the mode lies at the true value with a bounded interval; at L = 100 L = 100 L = 100 the Fieller interval is unbounded, a correct report of non-identifiability. Figure 6 shows the Marsaglia–Hinkley density and ensemble Fieller intervals behind the a , b a,b a , b entries of Table 2 , at all three window lengths.
Figure 6. Ratio-of-normals inference for the FHN slow parameters a , b a,b a , b (M = 1000 M = 1000 M = 1000 ). The Marsaglia–Hinkley density (black) matches the Monte Carlo sampling distribution of the ratio estimator (histogram) in every panel. A ratio of jointly normal estimates is itself skewed and heavy-tailed (the gray text gives the mass beyond the axis endpoints). At L = 20 L = 20 L = 20 and L = 5 L = 5 L = 5 the mode (vertical orange line) lies close to the true value (dashed) with a tight bounded ensemble-level Fieller interval (green band); at L = 100 L = 100 L = 100 the denominator ϵ ^ \hat{\epsilon} ϵ ^ is indistinguishable from zero, and the density develops Cauchy-like tails (the tail mass halves with each doubling of the threshold as 1 / r 2 1/r^2 1/ r 2 decay predicts), and the Fieller interval is unbounded, indicating non-identifiability.
Cox–Ingersoll–Ross (state-dependent diffusion)
d X t = κ ( μ − X t ) d t + ν X t d W t dX_t = \kappa(\mu-X_t)\,dt+\nu\sqrt{X_t}\,dW_t d X t = κ ( μ − X t ) d t + ν X t d W t , with ( κ , μ , ν ) = ( 2 , 1.5 , 1 ) (\kappa,\mu,\nu) = (2,1.5,1) ( κ , μ , ν ) = ( 2 , 1.5 , 1 ) , Δ t = 0.01 \Delta t = 0.01 Δ t = 0.01 , T = 100 T = 100 T = 100 , R = 0.01 R = 0.01 R = 0.01 , for which the diffusion σ 2 ( x ) = ν 2 x \sigma^2(x) = \nu^2x σ 2 ( x ) = ν 2 x depends on the state itself. With the Feller condition 2 κ μ = 6 ≫ ν 2 2\kappa\mu= 6 \gg\nu^2 2 κ μ = 6 ≫ ν 2 satisfied, the affine drift term candidates g ( x ) = [ 1 , x ] g(x) = [1,x] g ( x ) = [ 1 , x ] , and the linear diffusion term candidate h ( x ) = x h(x) = x h ( x ) = x , the drift coefficients ( κ μ , − κ ) (\kappa\mu,-\kappa) ( κ μ , − κ ) are recovered as 3.14 (1.09) and −2.10 (0.75) against truth (3, −2), and the diffusion coefficient as ν ^ = 1.00 \hat{\nu} = 1.00 ν ^ = 1.00 (0.04) against the true value of 1. The multilag estimator Eq. (18) , applied to each trajectory on its own, returns R ^ = 0.0100 \widehat{R} = 0.0100 R = 0.0100 against R = 0.01 R = 0.01 R = 0.01 on average. The diffusion side thus handles non-constant σ \sigma σ through its diffusion term candidates just as the drift side handles non-constant f f f . At L = 100 L = 100 L = 100 (T w = 1.0 T_w = 1.0 T w = 1.0 , twice 1 / κ 1/\kappa 1/ κ ) a few diverging records inflate the spread (Table 2 ).
Lorenz-96 (high dimensional)
The cyclic system d x i = [ ( x i + 1 − x i − 2 ) x i − 1 − x i + F ] d t + σ d W i \mathrm{d}x_i = [(x_{i+1}-x_{i-2})x_{i-1}-x_i+F]\,\mathrm{d}t + \sigma\,\mathrm{d}W_i d x i = [( x i + 1 − x i − 2 ) x i − 1 − x i + F ] d t + σ d W i with forcing F = 8 F = 8 F = 8 and σ = 0.5 \sigma= 0.5 σ = 0.5 (Δ t = 10 − 3 \Delta t = 10^{-3} Δ t = 1 0 − 3 , T = 10 T = 10 T = 10 , R = 10 − 3 R = 10^{-3} R = 1 0 − 3 ), is run at d = 40 d = 40 d = 40 and d = 120 d = 120 d = 120 . The pooled per-component drift term candidates are [ 1 , x i , x i − 1 x i + 1 , x i − 1 x i − 2 ] [1, x_i, x_{i-1}x_{i+1}, x_{i-1}x_{i-2}] [ 1 , x i , x i − 1 x i + 1 , x i − 1 x i − 2 ] under cyclic indexing, and
the diffusion term candidate is h = [ 1 ] h = [1] h = [ 1 ] . The forcing and the three structural coefficients are recovered tightly at both dimensions (F ^ = 7.99 \widehat{F} = 7.99 F = 7.99 (0.04) with structural terms within 0.2 % 0.2\% 0.2% at d = 120 d = 120 d = 120 ) along with σ ^ = 0.50 \widehat{\sigma} = 0.50 σ = 0.50 and R ^ = 1.00 × 10 − 3 \widehat{R} = 1.00 \times10^{-3} R = 1.00 × 1 0 − 3 at both dimensions; at d = 120 d = 120 d = 120 the learned coefficients are more accurate, because the larger system supplies more pooled windows.
Stochastic Kuramoto network (non-polynomial drift term candidates)
A network of N = 50 N = 50 N = 50 phase oscillators, d θ i = [ ω + K N ∑ j sin ( θ j − θ i ) ] d t + σ d W i \mathrm{d}\theta_i = \left[\omega+ \frac{K}{N}\sum_j \sin(\theta_j-\theta_i)\right]\mathrm{d}t + \sigma\,\mathrm{d}W_i d θ i = [ ω + N K ∑ j sin ( θ j − θ i ) ] d t + σ d W i , with ω = 0.5 \omega= 0.5 ω = 0.5 , σ = 0.3 \sigma= 0.3 σ = 0.3 , coupling K = 2.0 K = 2.0 K = 2.0 or K = 0.8 K = 0.8 K = 0.8 , Δ t = 0.005 \Delta t = 0.005 Δ t = 0.005 , T = 50 T = 50 T = 50 , and R = 10 − 3 R = 10^{-3} R = 1 0 − 3 ; in this example only, N N N counts oscillators and K K K denotes the coupling, not the moment-system quantities of § 4 . The per-oscillator drift term candidates are [ 1 , S ‾ s cos θ i − S ‾ c sin θ i ] [1,\overline{S}_s\cos\theta_i-\overline{S}_c\sin\theta_i] [ 1 , S s cos θ i − S c sin θ i ] , the mean-field coupling expanded through the order-parameter components S ‾ s = 1 N ∑ j sin θ j \overline{S}_s = \frac{1}{N}\sum_j\sin\theta_j S s = N 1 ∑ j sin θ j and S ‾ c = 1 N ∑ j cos θ j \overline{S}_c = \frac{1}{N}\sum_j\cos\theta_j S c = N 1 ∑ j cos θ j ; the diffusion term candidate is h = [ 1 ] h = [1] h = [ 1 ] . Across both the coherent and the near-incoherent regime, the natural frequency and coupling are recovered, ω ^ = 0.50 \widehat{\omega}=0.50 ω = 0.50 (0.01), with K ^ = 2.01 \widehat{K}=2.01 K = 2.01 (0.10) and 0.80 (0.06) respectively, σ ^ ≈ 0.30 \widehat{\sigma}\approx0.30 σ ≈ 0.30 and R ^ = 1.00 × 10 − 3 \widehat{R}=1.00\times10^{-3} R = 1.00 × 1 0 − 3 in both regimes, confirming that exogeneity and relevance are properties of the windows, not of the basis.
Underdamped Langevin (hypoelliptic)
d q = p d t \mathrm{d}q = p\,\mathrm{d}t d q = p d t , d p = [ q − q 3 − γ p ] d t + σ d W \mathrm{d}p = [q-q^3-\gamma p]\,\mathrm{d}t+\sigma\,\mathrm{d}W d p = [ q − q 3 − γ p ] d t + σ d W , run at ( γ , σ ) = ( 0.5 , 1 ) (\gamma,\sigma)=(0.5,1) ( γ , σ ) = ( 0.5 , 1 ) and ( 2 , 2 ) (2,2) ( 2 , 2 ) with Δ t = 0.005 \Delta t=0.005 Δ t = 0.005 , T = 100 T=100 T = 100 , R = 10 − 3 R=10^{-3} R = 1 0 − 3 , the hardest case of the nine. The q q q -equation carries no noise (σ q = 0 \sigma_q = 0 σ q = 0 ), so the diffusion matrix is rank-deficient and the system is only hypoelliptic. It also carries two separated timescales, the fast conservative oscillation and the slow damping time 1 / γ 1/\gamma 1/ γ , and it is run at the common window L = 20 L = 20 L = 20 . The estimates are insensitive to this choice; at L = 50 L = 50 L = 50 the friction estimate is c p = − 0.51 c_p = -0.51 c p = − 0.51 (0.25), against − 0.51 -0.51 − 0.51 (0.23) at L = 20 L = 20 L = 20 (γ = 0.5 \gamma= 0.5 γ = 0.5 ), the kinematic coefficient is more precise at the longer window (0.012 against 0.028), and the p p p -equation terms are slightly more precise at the shorter one. The drift term candidates are [ p ] [p] [ p ] for the q q q -equation and [ q , q 3 , p ] [q, q^3, p] [ q , q 3 , p ] for the p p p -equation, with diffusion term candidate h = [ 1 ] h = [1] h = [ 1 ] per component. The kinematic constraint d q = p d t \mathrm{d}q = p\,\mathrm{d}t d q = p d t is recovered with coefficient 1.00 (0.03), the potential and friction terms of the p p p -equation are recovered (c q 3 = − 1.01 c_{q^3} = -1.01 c q 3 = − 1.01 (0.22) and c p = − 0.51 c_p = -0.51 c p = − 0.51 (0.23) at γ = 0.5 \gamma= 0.5 γ = 0.5 , and c p = − 2.01 c_p = -2.01 c p = − 2.01 (0.53) at γ = 2 \gamma= 2 γ = 2 , against truth − 1 -1 − 1 and − γ -\gamma − γ ), and σ ^ p \hat{\sigma}_p σ ^ p matches the truth under both diffusion forms, the lag-one quadratic variation Eq. (17) giving 1.00 (0.01) at γ = 0.5 \gamma= 0.5 γ = 0.5 , σ = 1 \sigma= 1 σ = 1 and 2.00 (0.02) at γ = 2 \gamma= 2 γ = 2 , σ = 2 \sigma= 2 σ = 2 , and the two-lag slope 1.01 (0.01) and 1.99 (0.02). The multilag estimator Eq. (18) returns R ^ = ( 1.00 , 1.00 ) × 10 − 3 \hat{R} = (1.00, 1.00) \times10^{-3} R ^ = ( 1.00 , 1.00 ) × 1 0 − 3 for ( q , p ) (q,p) ( q , p ) at γ = 0.5 \gamma= 0.5 γ = 0.5 and ( 1.00 , 1.01 ) × 10 − 3 (1.00, 1.01) \times10^{-3} ( 1.00 , 1.01 ) × 1 0 − 3 at γ = 2 \gamma= 2 γ = 2 , against 10 − 3 10^{-3} 1 0 − 3 . The diffusion of p p p uses the lag-one and lag-two variograms, in which R R R cancels. These are IV estimates on the full set of drift term candidates. The degenerate σ q \sigma_q σ q is returned not as zero but as a small, predictable finite-Δ t \Delta t Δ t artifact ≈ 3 E [ p 2 ] Δ t \approx\sqrt{3\,\mathbb{E}[p^2]\,\Delta t} ≈ 3 E [ p 2 ] Δ t , which correctly flags the component as near-degenerate instead of silently mis-estimating it.
Lorenz-63 through a non-identity observation map
This is the case of Remark 2, and the synthetic counterpart of the optical-trap analysis of § 8.2 . The drift and diffusion are those of the stochastic Lorenz-63 system above (σ = ( 0.5 , 0.4 , 0.55 ) \boldsymbol{\sigma} = (0.5, 0.4, 0.55) σ = ( 0.5 , 0.4 , 0.55 ) ), but the state is observed through
C = 1 2 ( 1 1 0 0 1 1 0 0 1 ) , (28) C = \frac{1}{2}
\begin{pmatrix}
1 & 1 & 0 \\
0 & 1 & 1 \\
0 & 0 & 1
\end{pmatrix}, \tag*{(28)} C = 2 1 1 0 0 1 1 0 0 1 1 , (28)
taken from [21 ] , with per-sensor noise variance R y = 10 − 4 R_y = 10^{-4} R y = 1 0 − 4 . Here d y = d = 3 d_y = d = 3 d y = d = 3 and C C C is invertible but far from the identity. The deconvolution X ^ t n = ( C T C ) − 1 C T Y t n \hat{X}_{t_n} = (C^\mathsf{T}C)^{-1}C^\mathsf{T}Y_{t_n} X ^ t n = ( C T C ) − 1 C T Y t n of Remark 2 amplifies the sensor noise (∥ C − 1 ∥ ≈ 3 \lVert C^{-1}\rVert\approx3 ∥ C − 1 ∥ ≈ 3 on the worst component), so the effective per-state covariance R e f f = ( C T C ) − 1 C T R ( C T C ) − 1 R_{\mathrm{eff}} = (C^\mathsf{T}C)^{-1}C^\mathsf{T}R(C^\mathsf{T}C)^{-1} R eff = ( C T C ) − 1 C T R ( C T C ) − 1 has diagonal ( 1.2 , 0.8 , 0.4 ) × 10 − 3 (1.2, 0.8, 0.4) \times10^{-3} ( 1.2 , 0.8 , 0.4 ) × 1 0 − 3 , an order of magnitude above the per-sensor level. Sampling and term candidates follow the Lorenz-63 settings above (Δ t = 10 − 3 \Delta t = 10^{-3} Δ t = 1 0 − 3 , N = 10 4 N = 10^4 N = 1 0 4 estimation points, n s u b = 10 n_{\mathrm{sub}} = 10 n sub = 10 , meaning the estimation grid retains every ten simulated step, the degree-2 monomial drift term candidates and diffusion term candidate h = [ 1 ] h = [1] h = [ 1 ] ), all corrections on, and the standard settings L = 20 L = 20 L = 20 , δ = 4 Δ t \delta= 4\Delta t δ = 4Δ t , n l a g s = 1 n_{\mathrm{lags}} = 1 n lags = 1 .
Over M = 1000 M = 1000 M = 1000 trajectories (all of which converged) the IV estimates are − 10.02 -10.02 − 10.02 (0.83) and 10.00 10.00 10.00 (0.53) for x x x and y y y in x ˙ \dot{x} x ˙ (truth − 10 -10 − 10 , 10 10 10 ); 28.01 (0.67), − 1.01 -1.01 − 1.01 (0.43) and − 1.001 -1.001 − 1.001 (0.019) for x x x , y y y and x z xz x z in y ˙ \dot{y} y ˙ (truth 28, − 1 -1 − 1 , − 1 -1 − 1 ); and − 2.65 -2.65 − 2.65 (0.63) and 0.998 (0.078) for z z z and x y xy x y in z ˙ \dot{z} z ˙ (truth − 2.667 -2.667 − 2.667 , 1). The first-stage F F F averages 64. The multilag estimator Eq. (18) applied to the deconvolved series recovers R e f f R_{\mathrm{eff}} R eff accurately, with mean diag ( R ^ e f f ) = ( 1.20 , 0.79 , 0.39 ) × 10 − 3 \operatorname{diag}(\hat{R}_{\mathrm{eff}}) = (1.20, 0.79, 0.39) \times10^{-3} diag ( R ^ eff ) = ( 1.20 , 0.79 , 0.39 ) × 1 0 − 3 over the trajectories. The diffusion, estimated by the drift-corrected quadratic variation Eq. (17) with the estimated R ^ e f f \hat{R}_{\mathrm{eff}} R ^ eff , feasible here since ϱ ≈ 10 \varrho\approx10 ϱ ≈ 10 lies within the operating range, is recovered as σ ^ = ( 0.489 , 0.412 , 0.564 ) \hat{\sigma} = (0.489, 0.412, 0.564) σ ^ = ( 0.489 , 0.412 , 0.564 ) against ( 0.5 , 0.4 , 0.55 ) (0.5, 0.4, 0.55) ( 0.5 , 0.4 , 0.55 ) , with standard deviations (0.095, 0.072, 0.032); the slope form gives (0.457, 0.436, 0.544) with standard deviations (0.275, 0.312, 0.360), uninformative trajectory by trajectory on this short, deconvolved record. Every coefficient of a term absent from the true model is estimated without detectable bias, the average estimate across the 23 such coefficients lying within 1.9 Monte Carlo standard errors of zero. The accuracy matches that of the directly observed Lorenz-63 experiment. The noise amplification inherent in inverting C C C costs precision on the linear rates, with standard deviations of order 0.5 to 1 instead of a few tenths, but neither biases the estimator nor degrades the quadratic terms x z xz x z and x y xy x y , which remain the best-determined coefficients in the system. A non-identity observation map is, for this method, a change of variables and a recalibration of the noise covariance, not a change of estimator. At L = 5 L = 5 L = 5 the seven coefficients are recovered with about twice the spread of L = 20 L = 20 L = 20 , at L = 100 L = 100 L = 100 the deconvolved record is unstable (Table 2 ), and at the stress level the first stage collapses at every window and depth, on records of N = 2 × 10 5 N = 2 \times10^5 N = 2 × 1 0 5 points as well as N = 10 4 N = 10^4 N = 1 0 4 (Table 6 ).
Table 6. Drift recovery at the stress noise level of Table 5 (the per-component variance R s R_s R s , the square of 15% of the component’s range). Layout and conventions follow Table 2 ; the L = 100 L = 100 L = 100 and L = 5 L = 5 L = 5 columns report raw IV at depth one, while the operative window L = 20 L = 20 L = 20 shows all four depths: raw IV at depth one, and two-step GMM with two, three, and four instrument blocks at depths two to four. The depth rule (Remark 3) raises the depth to two when the depth-one F F F falls below 10 or the two depths disagree beyond their standard errors. The last row of each block gives the average first-stage F F F , with the percentage of replications whose own F F F falls below 10 in parentheses. Note that where the average F F F falls below 1 we regard the inference as unreliable and the cells read n.i. (see the caption of Table 2 and Appendix § 14 ).
For the remaining systems no comparably standard alternative exists, since exact likelihoods for noisily observed records are confined to linear Gaussian models such as OU, and particle or ensemble filters for nonlinear, chaotic, or high-dimensional dynamics require tuning and computation far beyond a single linear solve. By analogy with the two cases above, we expect a well-tuned likelihood method, where it can be run, to be more precise than WAIVES and substantially slower, and strong-form regression to be biased by the same errors-in-variables mechanism.
Drift and diffusion recovery
Table 2 reports the drift recovery for every parameter of the nine systems at the three window lengths. The quadratic-variation estimator Eq. (17) recovers the diffusion magnitude alongside the drift. Table 3 reports σ ^ \hat{\sigma} σ ^ at the operative window L = 20 L = 20 L = 20 , with all corrections of Eq. (17) enabled for every system, alongside the multilag slope form of § 4 . The noise level that the debiasing and the quadratic-variation forms use is each trajectory’s own multilag estimate Eq. (18) . Averaged over trajectories, it returns R ^ = 0.0200 \hat{R} = 0.0200 R ^ = 0.0200 for the OU and double-well records against R = 0.02 R = 0.02 R = 0.02 , ( 0.0500 , 0.0498 , 0.0498 ) (0.0500, 0.0498, 0.0498) ( 0.0500 , 0.0498 , 0.0498 ) for Lorenz-63 and ( 0.0500 , 0.0500 ) (0.0500, 0.0500) ( 0.0500 , 0.0500 ) for FitzHugh–Nagumo against 0.05. At the stress level of Table 6 it reproduces the fixed stress variances to within 0.03% for every component. The slope form recovers every dissipative-model component within Monte Carlo error. The lag-one form matches it except on the FitzHugh–Nagumo V V V component; there ϱ ≈ 22 \varrho\approx22 ϱ ≈ 22 approaches the edge of the operating regime and the lag-one estimate is biased upward by 3%. Lorenz-63 lies at ϱ ≈ 50 \varrho\approx50 ϱ ≈ 50 , where the variogram still recovers R ^ = 0.0499 \hat{R} = 0.0499 R ^ = 0.0499 against 0.05, but the y y y and z z z diffusions are biased upward by 7–8% in the lag-one form and 25–29% in the slope form (ten or more Monte Carlo standard errors; Table 3 ). The lag-one x x x is exact and the slope x x x is within 6%. The excess on the slope reflects drift contamination of the longer lags at this extreme ϱ \varrho ϱ , which the corrections of Eq. (17) remove only to leading order. On the dissipative systems the slope’s variance is comparable to the lag-one form.
Robustness to observation noise
Table 6 and Table 7 repeat every drift and diffusion estimate of all nine systems at a “stress” noise level, a fixed variance R s R_s R s per component equal to the square of 15% of the component’s range over a record of the length used in the experiments (Table 5 ; galleries in Figure 3 ). The OU and double-well estimates stay unbiased, with 1.9 and 3.7 times their spread at the main-analysis noise level. For FitzHugh–Nagumo and Lorenz-63 the depth-one first stage collapses (F < 1 F < 1 F < 1 ). The depth rule restores bounded FitzHugh–Nagumo intervals, while the Lorenz-63 estimates stay centered near the truth but with spreads too large to be informative (depth-two F ≈ 1 F \approx1 F ≈ 1 ). The diffusion estimators lose precision as § 4 anticipates, up to the point where the Lorenz-63 diffusion is no longer identifiable from the increments.
Table 7. Diffusion recovery at the stress noise level of Table 5 at L = 20 L = 20 L = 20 , with the conventions of Table 3 , at depths one to four. The column ϱ s = 2 R s / ( σ 2 Δ t ) \varrho_s = 2R_s/(\sigma^2\Delta t) ϱ s = 2 R s / ( σ 2 Δ t ) gives the noise-to-signal ratio of the increments at the stress level (§ 4 ). For the Lorenz-63 observed through C C C it is computed on the deconvolved record, and for Cox–Ingersoll–Ross the state-dependent σ 2 ( x ) \sigma^2(x) σ 2 ( x ) is evaluated at the mean μ \mu μ . Note that where ϱ s \varrho_s ϱ s exceeds 10 4 10^4 1 0 4 the diffusion signal lies four orders of magnitude below the noise floor and the increments carry essentially no information on σ \sigma σ . We label these cells as n.i. (not identified).
The resulting choices of span, gap, depth, and slope lags are collected as a design rule in Appendix § 12 .
Real-data illustrations
The example systems above validated WAIVES against parameters known by design. We now apply WAIVES to two real data sets to demonstrate the method’s ease of application and its usefulness in practice. The first is the unsmoothed Atlantic Multidecadal Oscillation (AMO) index, the area-averaged, detrended North Atlantic sea-surface-temperature anomaly [56 , 57 ] . It is an observational record with no ground truth and no tractable likelihood, where competing model classes must be fitted and compared, highlighting the need for a fast, assumption-light estimator such as WAIVES. The second is the calibration of an optical trap [58 ] , where the diffusion coefficient is fixed by first principles and the drift is measured independently by standard calibration, so WAIVES can be checked against known physics.
North Atlantic sea-surface temperature
Beyond being a canonical long climate series (N = 2005 N = 2005 N = 2005 monthly values, 1856–2023, Δ t = 1 / 12 y r \Delta t = 1/12\,\mathrm{yr} Δ t = 1/12 yr ), the AMO index is at the center of an active debate about the stability of the Atlantic meridional overturning circulation. In that debate, competing stochastic descriptions (monostable models with a single restoring equilibrium, and bistable or tipping-prone alternatives) are fitted to closely related sea-surface-temperature records and compared [59 , 60 , 61 ] . We use this data set to demonstrate WAIVES, not to enter that debate; the index is a temperature proxy whose relationship to the overturning circulation is itself contested [62 , 63 , 64 ] .
These data have several noteworthy features, including very low measurement noise, as expected for an index averaged over an entire ocean basin (the variogram estimator of Section 4 yields R ^ 1 / 2 = 0.006 ∘ C \hat{R}^{1/2} = 0.006\,^{\circ}\mathrm{C} R ^ 1/2 = 0.006 ∘ C ). With the observation error effectively absent, the dominant bias mechanism is expected to stem from the shared-driving-noise endogeneity, which makes the record a well-suited application for WAIVES. In addition, a residual annual cycle persists in these data, even after the standard anomaly processing. This cycle is also visible in the gap-sweep diagnostic plots in Figure 8 , where the drift intercept varies with the gap δ \delta δ in a systematic, calendar-dependent pattern. Removing the monthly climatology yields an intercept statistically indistinguishable from zero while leaving the restoring rate essentially unchanged (Section 8.1.2 and Figure 8 ). The relevance diagnostic detects very low measurement noise, as expected for an index averaged over an entire ocean basin (the variogram estimator of Section 4 yields R ^ 1 / 2 = 0.006 ∘ C \hat{R}^{1/2} = 0.006\,^{\circ}\mathrm{C} R ^ 1/2 = 0.006 ∘ C ). With the observation error effectively absent, the dominant bias mechanism is expected to stem from the shared-driving-noise endogeneity, which makes the record a well-suited application for WAIVES. In addition, a residual annual cycle persists in these data, even after the standard anomaly processing. This cycle is also visible in the gap-sweep diagnostic plots in Figure 8 , where the drift intercept varies with the gap δ \delta δ in a systematic, calendar-dependent pattern. Removing the monthly climatology yields an intercept statistically indistinguishable from zero while leaving the restoring rate essentially unchanged (Section 8.1.2 and Figure 8 ). The relevance diagnostic detects very low measurement noise, as well as all subsequent analyses use the deseasonalized series.
Figure 8. The gap sweep as a built-in misspecification check. Sweeping the instrument–target gap δ \delta δ at L = 6 L=6 L = 6 rotates the calendar phase of the window pairs (pair span 2 L + δ / Δ t 2L+\delta/\Delta t 2 L + δ /Δ t steps) through any residual annual cycle. A correctly specified, time-homogeneous drift does not depend on δ \delta δ . The shaded bands show the central 80% of the same estimates on matched synthetic records without seasonality, and the error bars are window-pair bootstrap intervals. Because all markers derive from the same record, they are strongly correlated across gaps and will tend to wander together. Thus, the diagnostic is a systematic dependence on δ \delta δ , for example a cluster of estimates pulled far from the null at particular gaps. The error bars, which describe each estimate’s own uncertainty, may likewise extend beyond the band without indicating misspecification. Top: on the raw series (open markers) the drift intercept c ^ 0 \hat{c}_0 c ^ 0 is displaced for outside the null at the calendar-aligned gaps (δ = 0 , 12 \delta=0,12 δ = 0 , 12 months): at these gaps the pair span is a whole number of years, so every pair occupies the same calendar months and the cycle’s contribution adds coherently across pairs instead of averaging out, displacing the intercept. Removing the monthly climatology (filled markers) reduces the δ \delta δ -dependence. Bottom: the slope κ ^ \hat{\kappa} κ ^ is essentially unchanged by deseasonalization and shows no systematic dependence on δ \delta δ , with its markers tracking the null band across the sweep.
We thus model the deseasonalized anomaly X t X_t X t as an SDE with unknown drift and constant unknown diffusion, beginning with the affine drift term candidates g ( x ) = [ 1 , x ] g(x) = [1,x] g ( x ) = [ 1 , x ] and diffusion term candidate h = [ 1 ] h = [1] h = [ 1 ] , that is, the OU model d X t = − κ ( X t − μ ) d t + σ d W t dX_t = -\kappa(X_t - \mu)\,dt + \sigma\,dW_t d X t = − κ ( X t − μ ) d t + σ d W t . Here 1 / κ 1/\kappa 1/ κ is the relaxation time, and the mean-reversion rate κ \kappa κ controls how strongly the anomaly is pulled toward its equilibrium. We choose the window span by sweeping over the spans of 0.5–3 yr, from well inside to just beyond the record’s ≈ 2.5 \approx2.5 ≈ 2.5 yr decorrelation time, the comparison that Proposition 3 ties instrument informativeness to. We select the operative span by the first-stage strength
F F F of § 7 , which is maximized at the shortest span of 0.5 yr (F = 197 F=197 F = 197 , satisfying F ≥ 10 F\ge10 F ≥ 10 ). Confidence intervals are Wald intervals from the plug-in covariance (Appendix § 14 ).
The resulting estimates are summarized in Figure 7 . At the operative window, the IV estimate of the mean-reversion rate is κ ^ = 0.76 \hat{\kappa}=0.76 κ ^ = 0.76 (95% CI [ 0.20 , 1.31 ] [0.20,1.31] [ 0.20 , 1.31 ] ) yr− 1 ^{-1} − 1 . The anomaly relaxes toward a single equilibrium, statistically indistinguishable from zero, on a relaxation time of 1 / κ ^ ≈ 1.3 1/\hat{\kappa}\approx1.3 1/ κ ^ ≈ 1.3 yr. The diffusion estimate is σ ^ = 0.30 ∘ C y r − 1 / 2 \hat{\sigma}=0.30\,^\circ\mathrm{C}\,\mathrm{yr}^{-1/2} σ ^ = 0.30 ∘ C yr − 1/2 , with two independent estimators (the multilag slope Eq. (21) and the quadratic-variation estimator Eq. (17) ) agreeing to within 7%.
Figure 7. WAIVES applied to the unsmoothed AMO index. (A) The deseasonalized monthly series, 1856–2023. (B) The estimated drift at the operative window: the affine fit (solid) with its 95% window-pair bootstrap band (shaded), the cubic-drift fit (dashed) with its band (lighter shading), and the empirical distribution of the observed anomalies (gray histogram). The affine fit’s slope is − κ ^ -\hat{\kappa} − κ ^ , where the restoring rate κ \kappa κ measures how strongly an anomaly is pulled back toward the long-term mean (κ ^ = 0.76 \hat{\kappa}=0.76 κ ^ = 0.76 with 95% Wald interval of [ 0.20 , 1.31 ] [0.20,1.31] [ 0.20 , 1.31 ] .) The bootstrap bands are shown as an additional uncertainty-quantification mode, serving as a diagnostic for unmodeled structure (in the gap and span sweeps of this section, an estimate that wanders outside of its bootstrap bands would flag a misspecified drift, as the residual seasonal cycle did for the intercept). Bistability, which is defined as two preferred AMO states, would appear as a cubic drift bending away from the affine fit. However, over the range of observed states the two fits coincide within their uncertainty bands, and the record cannot distinguish a single well from a shallow double well.
We check the fitted model in three ways. First, an OU process with parameters set to these estimates has stationary standard deviation σ ^ / 2 κ ^ = 0.25 ∘ C \hat{\sigma}/\sqrt{2\hat{\kappa}}=0.25\,^\circ\mathrm{C} σ ^ / 2 κ ^ = 0.25 ∘ C , close to the observed 0.21 °C, the record’s sample standard deviation. Second, the fitted model implies a correlation time near 1.3 yr, whereas the record’s measured 1 / e 1/e 1/ e time is 2.5 yr, so the index superposes several timescales and the operative-span estimate targets the fast, short-time relaxation. Third, repeating the estimation on simulated OU records of the same length, rate, and diffusion, with observation noise at the record’s scale, confirms that the Wald interval attains 93% coverage (§ 8.1.5 ). Weak-form least squares on the same moment system, by contrast, fails to identify the restoring rate. On the real record its estimates flip sign across the span sweep, and on matched synthetic records it is driven to zero at every span, as Proposition 1 implies (§ 8.1.5 ).
Finally, we examine the bistability hypothesis directly. Bistability here means that the system possesses two stable equilibria (two coexisting climate states), so that noise or slow forcing could tip it from one to the other. The minimal drift with this property is cubic: f ( x ) = θ 1 x + θ 3 x 3 f(x)=\theta_1x+\theta_3x^3 f ( x ) = θ 1 x + θ 3 x 3 with θ 1 > 0 \theta_1>0 θ 1 > 0 and θ 3 < 0 \theta_3<0 θ 3 < 0 has stable states at x ∗ = ± θ 1 / ∣ θ 3 ∣ x_*=\pm\sqrt{\theta_1/|\theta_3|} x ∗ = ± θ 1 /∣ θ 3 ∣ separated by an unstable equilibrium at zero (a “double well”). In contrast, the linear drift − κ x -\kappa x − κ x (with κ > 0 \kappa>0 κ > 0 ) always restores toward the single equilibrium state at zero. Enlarging the drift term candidates to [ x , x 3 ] [x,x^3] [ x , x 3 ] therefore allows the data to discrim- @page 18/38 starts_mid=0 ends_mid=1 inate between the two hypotheses, with bistability corresponding to a negative θ 3 \theta_3 θ 3 of sufficient magnitude. The statistical inference question is then whether the record can distinguish such a value from zero. The resulting 95% interval, θ 3 ∈ [ − 38 , + 21 ] \theta_3 \in[-38,+21] θ 3 ∈ [ − 38 , + 21 ] , admits both hypotheses at the same time. It contains zero, so a purely linear drift cannot be rejected. However, it also contains, for example, θ 3 = − 9 \theta_3=-9 θ 3 = − 9 , which corresponds to a double well calibrated to the AMO record itself, with stable states one sample standard deviation from zero (± 0.21 ∘ C \pm0.21\,^\circ\mathrm{C} ± 0.21 ∘ C ) and a near-well relaxation time corresponding to that of the fitted model, 1 / κ ^ 1/\hat{\kappa} 1/ κ ^ (Figure 7 B, with the derivation in § 8.1.7 ). In fact, stable states anywhere between 0.5 and 1.5 sample standard deviations imply ∣ θ 3 ∣ |\theta_3| ∣ θ 3 ∣ between roughly 34 and 4, all inside the interval. Furthermore, a synthetic power analysis (§ 8.1.7 ) indicates that no estimator built on these moment conditions could resolve a cubic component of that magnitude from a single 167-year record. This conclusion is also supported by a GMM model comparison. The linear model passes the Hansen J J J -test [36 ] (J = 0.73 J=0.73 J = 0.73 , p = 0.39 p=0.39 p = 0.39 ), with no evidence of misspecification in its over-identified moment system. Similarly, the moment-selection criteria of Andrews and Lu [65 ] , which can be viewed as the GMM counterparts of BIC and AIC, favor the linear model (§ 8.1.6 ). This preference may reflect parsimony rather than evidence against bistability; in synthetic double-well records the same criteria still select the linear model in 99% of replicates, so the comparison has very low power at this record length. The instrumental record, analyzed through this model class, therefore can neither support nor rule out bistability.
We emphasize that our analysis is complementary to the recent early-warning analyses of AMOC stability [59 , 60 ] . Those studies ask whether the stability of the currently occupied state has been declining over time as a bifurcation is approached. In contrast, we ask whether the time-averaged short-time drift of the basin-mean AMO index, over the range of states the record contains, supports one restoring equilibrium or two. These two questions are complementary, and a single well in the stationary short-time drift can be fully consistent with a slow approach to a fold. Our practical non-identifiability finding may, however, be directly relevant to that debate: projected tipping times are known to be sensitive to the assumed form of the drift [61 , 62 ] , and the power analysis above underlines that records of this length cannot distinguish the candidate drift structures.
Data and preprocessing
We use the unsmoothed, long AMO index from NOAA PSL (Kaplan SST V2 source [56 , 57 ] ), as monthly values January 1856 through January 2023 (N = 2005 N=2005 N = 2005 ; the series is no longer extended because its source data set was frozen). The published series is the area-weighted North Atlantic SST anomaly, linearly detrended by the provider. We do not smooth it, since smoothing would destroy the quadratic-variation structure on which the diffusion estimator and the noise-variance estimate rely. Time is measured in years, Δ t = 1 / 12 \Delta t=1/12 Δ t = 1/12 .
Residual seasonality and the gap-sweep diagnostic
Every choice of the gap δ \delta δ between the instrument and target windows defines a valid estimator of the same parameters. All such estimators share one probability limit, but not one finite-sample distribution, because instrument relevance decays with δ \delta δ and the sampling scatter grows accordingly. The specification comparison is therefore made against a null distribution that carries this gap dependence, the gray band of Figure 8 , which at each gap represents the central range of estimates obtained by repeating the entire analysis, including deseasonalization, on records simulated from the fitted model. On the raw series, the drift intercept departs from this null, varying with δ \delta δ in a systematic, calendar-dependent pattern. Calendar-month means range from − 0.035 -0.035 − 0.035 to + 0.035 ∘ C +0.035\,^\circ\mathrm{C} + 0.035 ∘ C , with the seasonal decline from September to December steepest. On the raw series this deterministic component contaminates the weak-form moments. The pair span is ( η l a g s + 1 ) L + δ / Δ t (\eta_{\mathrm{lags}}+1)L+\delta/\Delta t ( η lags + 1 ) L + δ /Δ t observations, so sweeping the gap δ \delta δ rotates the calendar phase of the window pairs through the annual cycle, and the drift intercept traces that cycle. At calendar-aligned spans, where every target window covers the same months, the bump-weighted autumn decline loads coherently onto the intercept, biasing it to − 0.29 ∘ C y r − 1 -0.29\,^\circ\mathrm{C}\,\mathrm{yr}^{-1} − 0.29 ∘ C yr − 1 , six standard errors from zero and far outside the matched synthetic null (Figure 8 , top). Removing the monthly pattern (subtracting each calendar month’s mean) restores an intercept statistically indistinguishable from zero at every window and at the operative configuration (c ^ 0 = 0.01 \hat{c}_0=0.01 c ^ 0 = 0.01 , s.e. 0.05), within the synthetic null across the gap sweep. The slope κ ^ \hat{\kappa} κ ^ is almost unaffected by deseasonalization (Figure 8 , bottom), because the annual cycle enters through the level moments, to which the intercept is sensitive, while the relaxation rate is identified by the state-dependent moments, which the cycle leaves nearly untouched. The remaining scatter of κ ^ \hat{\kappa} κ ^ across gaps, lag counts, and record halves is consistent with the null distribution obtained from matched synthetic records. The observed across-gap spread and the first/second-half difference lie at the 85th–90th percentile of that null, so we find no evidence of residual misspecification.
Estimation settings
All settings follow the analysis above, namely the affine drift term candidates [ 1 , x ] [1,x] [ 1 , x ] , enlarged to [ 1 , x , x 3 − 3 R ^ x ] [1,x,x^3-3\hat{R}x] [ 1 , x , x 3 − 3 R ^ x ] for the model comparison; diffusion term h = [ 1 ] h=[1] h = [ 1 ] ; bump banks p ∈ { 4 , 6 , 8 } p\in\{4,6,8\} p ∈ { 4 , 6 , 8 } ; depth η l a g s = 1 \eta_{\mathrm{lags}}=1 η lags = 1 ; the grid-sum quadrature of Eqs. (8) –(10) ; polynomial debiasing with the variogram R ^ \hat{R} R ^ ; Wald intervals from the plug-in covariance (Appendix D) for the operative-window estimates; and a window-pair bootstrap with 4000 resamples for the diagnostics at longer spans (span sweep, sliding windows, and model-comparison null bands). One setting departs from the benchmark conventions, the gap, which is δ = 0 \delta= 0 δ = 0 here instead of 4 Δ t 4\Delta t 4Δ t because the record is short and every observation spent on a buffer costs window pairs. At δ = 0 \delta= 0 δ = 0 the paired windows share only the boundary grid point, where every test function vanishes, so the discrete sums remain disjoint. Furthermore, the gap-sweep diagnostic of the preceding paragraph indicates that a larger buffer would add little robustness. Finally, the bootstrap resamples window pairs independently; the pairs are disjoint, and dependence between neighbouring pairs enters only through the state shared across their common boundary. On AMO-matched synthetic records this bootstrap is conservative at the longer spans, where the first stage is weak (coverage 96–98% at median F ≤ 9 F \le9 F ≤ 9 ), while the Wald interval stays calibrated down to F ≈ 3 F \approx3 F ≈ 3 (Appendix D). Windows of L ∈ { 6 , 9 , 12 , 18 , 24 , 36 } L \in\{6, 9, 12, 18, 24, 36\} L ∈ { 6 , 9 , 12 , 18 , 24 , 36 } observations give spans 0.5–3 yr, bracketing the record’s 1 / e 1/e 1/ e autocorrelation time of 2.5 yr. The operative window L = 6 L = 6 L = 6 has the strongest first stage, F = 197 F = 197 F = 197 versus F = 11 F = 11 F = 11 –65 at longer spans (equivalently, a first-stage R 2 R^2 R 2 , the share of regressor variance the instruments explain, of 0.55 versus 0.27–0.37), and the smallest standard error.
Robustness at the operative window
The baseline estimate is κ ^ = 0.76 \hat{\kappa} = 0.76 κ ^ = 0.76 (window-pair bootstrap s.e. 0.30; the standard errors in this subsection are bootstrap ones, while the interval quoted above is Wald, s.e. 0.28). Varying the test-function bank moves it well within sampling variation, to 0.75 (0.34) with bump powers { 6 , 8 , 10 } \{6, 8, 10\} { 6 , 8 , 10 } and 0.72 (0.23) with { 2 , 3 , 4 } \{2, 3, 4\} { 2 , 3 , 4 } . Deepening or displacing the instrument instead weakens the estimate substantially, to 0.07 (0.34) at n l a g s = 2 n_{\mathrm{lags}} = 2 n lags = 2 and 0.19 (0.60) at gap δ = 8 \delta= 8 δ = 8 months. Both changes are expected on a record this short, where the added block or buffer pushes the instrument outside the system’s memory and the first stage collapses (Proposition 3). Splitting the record gives 1.00 (0.46) on 1856–1939 and −0.14 (0.45) on 1940–2023. Each of these differences lies within the across-configuration variation of the matched synthetic null quantified in the preceding paragraph, at the 85th–90th percentile at the extremes, and the lower second half is consistent with the sliding-window view below.
Synthetic calibration and coverage
We simulate M = 800 M = 800 M = 800 OU records with the estimated parameters (κ = 0.77 \kappa= 0.77 κ = 0.77 , σ = 0.30 \sigma= 0.30 σ = 0.30 , N = 2005 N = 2005 N = 2005 , Δ t = 1 / 12 \Delta t = 1/12 Δ t = 1/12 ) and noise variance R = 3 × 10 − 4 R = 3 \times10^{-4} R = 3 × 1 0 − 4 , about seven times the record’s R ^ ≈ 4 × 10 − 5 \hat{R} \approx4 \times10^{-5} R ^ ≈ 4 × 1 0 − 5 (repeating the calibration at the record’s own R ^ \hat{R} R ^ changes none of the summaries), apply the identical procedure, including per-record climatology removal, and recover κ ^ \hat{\kappa} κ ^ with mean 0.81, median 0.80, and standard deviation 0.32. Two conclusions follow. First, the real-data Wald standard error (0.28) is close to the true sampling standard deviation, and the nominal-95% Wald interval attains 93% coverage. Second, the six-point window carries only a small (≈ 5 % \approx5\% ≈ 5% ) quadrature bias at this rate, negligible against the sampling spread. We report the uncorrected estimate. The weak-form OLS estimate on the same synthetic records is 0.00 (s.d. 0.19) at every window span from 0.5 to 3 yr, a complete cancellation, as Proposition 1 requires. On the real record, four of the six spans fall within this zero-centered null, while the shortest and the two-year spans exceed it (+0.61 and +0.67). Proposition 1 applies to any stationary diffusion, regardless of timescale structure, so these excursions are not a multiscale effect. They are consistent with sampling variability under the strong dependence of estimates across overlapping spans, or with mild nonstationarity of the record, not with a recoverable drift signal. The OLS values remain mutually inconsistent across spans, flipping sign, and identify no rate.
Weak-form model comparison, linear versus cubic drift
WAVIES never forms a likelihood, so classical information criteria are unavailable. The GMM framework supplies the standard substitutes. Taking the weak-form integrals of [ 1 , x , x 3 − 3 R ^ x ] [1, x, x^3 - 3\hat{R}x] [ 1 , x , x 3 − 3 R ^ x ] over the instrument window as instruments (m = 3 m = 3 m = 3 moment conditions per test function and window pair), the linear drift model (p = 2 p = 2 p = 2 ) is over-identified by one condition while the cubic model (p = 3 p = 3 p = 3 ) is exactly identified. Two-step efficient GMM with window-pair–clustered weighting yields the Hansen J J J -statistic [36 ] . Because the models are nested and share instruments and weighting, J l i n − J c u b = J l i n J_{\mathrm{lin}} - J_{\mathrm{cub}} = J_{\mathrm{lin}} J lin − J cub = J lin is the GMM distance test of θ 3 = 0 \theta_3 = 0 θ 3 = 0 [66 ] , and the Andrews–Lu moment-selection criteria [65 ] , MMSC-BIC = J − ( m − p ) ln N w = J - (m - p)\ln N_w = J − ( m − p ) ln N w and MMSC-AIC = J − 2 ( m − p ) = J - 2(m - p) = J − 2 ( m − p ) , evaluated per model, (smaller preferred), play the roles of BIC and AIC. On the AMO record at the operative window (N w = 166 N_w = 166 N w = 166 pairs), the cubic-model GMM estimate ( θ 1 , θ 3 ) = ( − 0.13 , − 5.9 ) (\theta_1,\theta_3) = (-0.13, -5.9) ( θ 1 , θ 3 ) = ( − 0.13 , − 5.9 ) agrees with the exactly identified IV estimate with the enlarged set of drift term candidates. The linear model passes the specification test (J = 0.73 J = 0.73 J = 0.73 , one degree of freedom, p = 0.39 p = 0.39 p = 0.39 ); and both criteria prefer the linear model (MMSC-BIC of −4.4 versus 0; MMSC-AIC of −1.3 versus 0). Validation on matched synthetics (300 replicates per scenario, full estimation procedure, including deseasonalization) shows the procedure is well calibrated but, at this record length, weakly powered. Under a true linear drift the nominal-5% J J J -test rejects in 5% of replicates and MMSC-BIC selects the cubic model in 1%; under true double wells with ∣ θ 3 ∣ = 9 |\theta_3| = 9 ∣ θ 3 ∣ = 9 (minima at ± 1 \pm1 ± 1 standard deviation) and ∣ θ 3 ∣ = 4 |\theta_3| = 4 ∣ θ 3 ∣ = 4 (± 1.5 \pm1.5 ± 1.5 standard deviations), the J J J -test rejects in only 3% and 4%, and MMSC-BIC selects the cubic model in 1% at either magnitude. The real-data preference for the linear drift therefore reflects parsimony at low power, not evidence against bistability.
Power analysis for the cubic component
On synthetic records matched to the AMO scales with a purely linear drift, the single-record IV estimate of the cubic coefficient is spread over several tens in magnitude at the operative window. A double well with minima at ± 1 \pm1 ± 1 standard deviation (± 0.21 ∘ C \pm0.21\,^\circ\mathrm{C} ± 0.21 ∘ C ) and near-well relaxation matching κ ^ \hat{\kappa} κ ^ requires ∣ θ 3 ∣ = κ ^ / ( 2 x ∗ 2 ) ≈ 9 \lvert\theta_3\rvert= \hat{\kappa}/(2x_*^2) \approx9 ∣ θ 3 ∣ = κ ^ / ( 2 x ∗ 2 ) ≈ 9 ; wells at ± 1.5 \pm1.5 ± 1.5 standard deviations require ∣ θ 3 ∣ ≈ 4 \lvert\theta_3\rvert\approx4 ∣ θ 3 ∣ ≈ 4 . Both magnitudes are inside the single-record sampling spread, and the real-data 95% interval [ − 38 , + 21 ] [-38,+21] [ − 38 , + 21 ] contains them together with zero. The cubic point estimate is itself monostable, and its bootstrap band contains the affine fit (Figure 7 B). In this case, the binding constraint is the record length, not the estimator. Distinguishing the record-calibrated θ 3 = − 9 \theta_3=-9 θ 3 = − 9 from zero at the 95% level would require the standard error reduced by a factor of three, and with the 1 / T 1/\sqrt{T} 1/ T scaling of Theorem 2 that means a record roughly ten times longer, on the order of two millennia.
A sliding-window view of the restoring rate
The analysis above focuses on the short-time drift. The same analysis nonetheless sheds light on the record’s stability history. The restoring rate κ \kappa κ is the quantity whose decline toward zero signals loss of resilience in recent analyses of closely related AMOC fingerprints [59 , 60 ] . Near its stable state, the ramped saddle-node model of ref. [60] linearizes to the OU form fitted here. Its restoring rate is α = 2 A ∣ λ ∣ \alpha= 2\sqrt{A\lvert\lambda\rvert} α = 2 A ∣ λ ∣ , the same quantity our κ \kappa κ estimates, from the drift, not from variance and autocorrelation. Their index also differs, the HadISST subpolar-gyre temperature with twice the global mean removed, and carries a faster rate (α ≈ 2 \alpha\approx2 α ≈ 2 –3 y r − 1 3\,\mathrm{yr}^{-1} 3 yr − 1 ) than the basin-wide AMO, so magnitudes are not directly comparable. We therefore estimate κ \kappa κ in sliding 60-yr windows stepped by 5 yr, each with the operative span and a window-pair bootstrap (Figure 9 ). The pointwise estimates fluctuate around the full-record value with wide intervals. They dip toward zero around the 1960s–70s window centers and recover thereafter. A variance-weighted trend across window centers is − 1.1 ± 1.2 y r − 1 -1.1 \pm1.2\,\mathrm{yr}^{-1} − 1.1 ± 1.2 yr − 1 per century, indistinguishable from zero at this record’s power. This is a demonstration of applicability, not an adjudication. A 60-yr window resolves rates of order 1 y r − 1 1\,\mathrm{yr}^{-1} 1 yr − 1 with roughly 50% relative uncertainty, so declines of the size debated for the overturning circulation remain below detectability here, consistent with the power analysis above. The value of the weak-form route is that it extracts the drift-based indicator directly, with likelihood-free uncertainty, rather than through variance and autocorrelation proxies.
Figure 9. Stability of the restoring rate over time from sliding-window estimates κ ^ ( t ) \hat{\kappa}(t) κ ^ ( t ) on the deseasonalized AMO record, 60-yr windows stepped by 5 yr, each estimated at the operative span with a window-pair bootstrap band (95%). The dashed line marks the full-record estimate κ ^ = 0.76 \hat{\kappa}=0.76 κ ^ = 0.76 . Consecutive windows share 55 of their 60 years, so the estimates are strongly correlated and the path wanders smoothly (its decadal wiggles are not independent evidence of change). No trend is resolvable at this record’s power (the variance-weighted trend across window centers is − 1.1 ± 1.2 y r − 1 -1.1 \pm1.2\,\mathrm{yr}^{-1} − 1.1 ± 1.2 yr − 1 per century). The record is thus consistent with a constant restoring rate, and slower drifts in the AMO’s persistence can be neither confirmed nor excluded.
Calibration of an optical trap
A dielectric bead held in an optical trap is, in the overdamped limit, the OU process of § 7 ,
d X t = − κ γ X t d t + 2 D d W t . (29) \mathrm{d}X_t=-\frac{\kappa}{\gamma}X_t\,\mathrm{d}t+\sqrt{2D}\,\mathrm{d}W_t. \tag*{(29)} d X t = − γ κ X t d t + 2 D d W t . (29)
It is one of the few measured systems in which both coefficients are known from physical first principles, with no time-series estimation. We use the published calibration data of [58 ] , a polystyrene bead of radius a = 1.030 μ m a=1.030\,\mu\mathrm{m} a = 1.030 μ m in water at T = 293.15 K T=293.15\,\mathrm{K} T = 293.15 K . Its transverse position is recorded by a quadrant photodiode at Δ t = 10 − 5 s \Delta t=10^{-5}\,\mathrm{s} Δ t = 1 0 − 5 s , with five trajectories of N = 10 6 N=10^{6} N = 1 0 6 samples at each of three laser powers.
The physical references follow from measured quantities alone. Stokes’ law determines the drag, γ = 6 π η a = 19.415 p N m s μ m − 1 \gamma=6\pi\eta a=19.415\,\mathrm{pN\,ms\,\mu m}^{-1} γ = 6 π η a = 19.415 pN ms μ m − 1 , where η \eta η is the dynamic viscosity of water. The Einstein–Smoluchowski relation then gives the diffusion, D = k B T / γ = 0.2085 μ m 2 s − 1 D=k_{\mathrm{B}}T/\gamma=0.2085\,\mathrm{\mu m}^{2}\mathrm{s}^{-1} D = k B T / γ = 0.2085 μ m 2 s − 1 . For the stiffness, the equipartition theorem gives κ = k B T / ⟨ x 2 ⟩ \kappa=k_{\mathrm{B}}T/\langle x^{2}\rangle κ = k B T / ⟨ x 2 ⟩ , where ⟨ x 2 ⟩ \langle x^{2}\rangle ⟨ x 2 ⟩ is the stationary variance of the measured bead position. No curve is fitted to obtain any of these, though all inherit the ∼ 2 % \sim2\% ∼ 2% per kelvin sensitivity of water’s viscosity through the Stokes drag. Because detector noise adds variance to the measured positions, the plain equipartition estimate divides by too large a ⟨ x 2 ⟩ \langle x^{2}\rangle ⟨ x 2 ⟩ and understates the stiffness. We therefore assess stiffness against the noise-corrected reference κ e q c o r r = k B T / ( ⟨ x 2 ⟩ − R ^ ) \kappa_{\mathrm{eq}}^{\mathrm{corr}}=k_{\mathrm{B}}T/(\langle x^{2}\rangle-\hat{R}) κ eq corr = k B T / (⟨ x 2 ⟩ − R ^ ) , with R ^ \hat{R} R ^ estimated from the fitted noise floor of the power spectrum (Table 4 ).
Table 4. Optical-trap calibration on the data of [58 ] at L = 20 L=20 L = 20 , n l a g s = 2 n_{\mathrm{lags}}=2 n lags = 2 , and δ = 8 Δ t \delta=8\Delta t δ = 8Δ t (Appendix § 12 ), showing mean (std) across the five records, each fitted separately. The gap exceeds the synthetic default δ = 4 Δ t \delta=4\Delta t δ = 4Δ t because the detector noise here is itself autocorrelated, and the instrument window must extend beyond its correlation length. Stiffness κ \kappa κ is reported in p N μ m − 1 \mathrm{pN}\,\mathrm{\mu m}^{-1} pN μ m − 1 , and diffusion D D D in μ m 2 s − 1 \mathrm{\mu m}^{2}\mathrm{s}^{-1} μ m 2 s − 1 (references are the Einstein–Smoluchowski D = k B T / γ D=k_{\mathrm{B}}T/\gamma D = k B T / γ and the noise-corrected equipartition κ e q c o r r = k B T / ( ⟨ x 2 ⟩ − R ^ P S D ) \kappa_{\mathrm{eq}}^{\mathrm{corr}}=k_{\mathrm{B}}T/(\langle x^{2}\rangle-\hat{R}_{\mathrm{PSD}}) κ eq corr = k B T / (⟨ x 2 ⟩ − R ^ PSD ) , while FORMA appears under both drag conventions). On real laboratory data, with no known ground truth, WAIVES recovers stiffness and diffusion in agreement with both the thermodynamic reference and the established FORMA estimator, and supplies a standard error for each, confirming that the method’s bias correction and uncertainty quantification carry over from simulation to experiment.
We fit WAIVES with drift term candidates g ( x ) = [ 1 , x ] g(x)=[1,x] g ( x ) = [ 1 , x ] and constant diffusion term candidate h = [ 1 ] h=[1] h = [ 1 ] . The released records are calibrated one-dimensional x x x positions, so no deconvolution is involved. We always report κ ^ = γ θ ^ \hat{\kappa}=\gamma\hat{\theta} κ ^ = γ θ ^ , with θ ^ \hat{\theta} θ ^ the recovered drift-rate coefficient of Eq. (29) , using the Stokes γ \gamma γ . For the diffusion we use the multilig slope form Eq. (21) of § 4 , a choice dictated by the detector’s filtering, which also places these records outside the white-noise observation model Eq. (2) of the theorems, so that this application tests robustness beyond their assumptions. Quadrant photodiodes low-pass the position signal at these bandwidths [67 ] , which depresses the shortest-lag increments and biases any estimator built on them. The lag-one variogram therefore fails to recover R R R on this detector, and FORMA’s diffusion estimate is low by roughly a third on these records for the same reason. The negative fitted multilig intercept Eq. (20) is what such filtering produces, whereas added white noise would push it positive, though we have not fit the detector’s transfer function directly. The slope is unaffected by either distortion, and it returns D ^ \hat{D} D ^ within − 0.2 % -0.2\% − 0.2% , + 2.3 % +2.3\% + 2.3% , and + 2.1 % +2.1\% + 2.1% of the Einstein value at the three powers (Table 4 ). Table 4 reports the full results at the operative window L = 20 L=20 L = 20 . WAIVES recovers the stiffness to within about 3 % 3\% 3% of the noise-corrected equipartition reference at all three powers, and agrees to within 1 1 1 –2 % 2\% 2% with the power-spectral Lorentzian fit, the accepted standard in this field.
The comparison with FORMA reveals that its drift coefficient is biased low by 31–37% and its diffusion by an almost identical factor. Its published stiffness is nonetheless accurate, because FORMA infers the drag self-consistently as k B T / D ^ k_{\mathrm{B}}T/\hat{D} k B T / D ^ instead of taking it from Stokes’ law, and the two biases cancel. That cancellation costs a drag coefficient inflated 45–59% above its measured value. Table 4 reports FORMA under both conventions, so the cancellation can be seen directly. The disjoint-window construction makes the cancellation unnecessary. The Lorentzian fit is equally accurate on this record, so the gain lies in scope. The weak-form estimator needs no spectral model, carries over directly to nonlinear drift term candidates, and returns drift and diffusion jointly from the same moment system.
Because raising the laser power shortens the relaxation time τ = γ / κ \tau= \gamma/\kappa τ = γ / κ from 1.43 to 0.35 ms at fixed Δ t \Delta t Δ t , this data set sweeps the ratio L Δ t / τ L\Delta t/\tau L Δ t / τ by experimental control, and so reproduces the window sweep of Figure 4 , on measured data. At L = 100 L = 100 L = 100 the span crosses τ \tau τ as the power is raised (L Δ t / τ = 0.70 L\Delta t/\tau= 0.70 L Δ t / τ = 0.70 , 1.80, 2.90) and the estimates degrade as that sweep predicts, to 13.93 (0.82), 32.71 (4.79), and 18.01 (21.65), the last no longer carrying any information. L = 20 L = 20 L = 20 is again the operative window.
Because this is the one record in the paper with an external ground truth, we also use it to check the F F F -triggered instrument depth rule of Remark 3. The first-stage F F F exceeds 10 4 10^4 1 0 4 at either depth and every power, so this is not a case of weak instruments in the sampling sense [46 ] . At the stiffest power the operative span lies at L Δ t / τ ≈ 0.57 L\Delta t/\tau\approx0.57 L Δ t / τ ≈ 0.57 , marginally beyond the nominal 0.5 τ 0.5\tau 0.5 τ band of § 7 , and the agreement below indicates that the band is conservative. Depth one is nonetheless attenuated by up to 10% at the highest power. The attenuation grows as the instrument moves further from the target, and matched synthetic records with white noise show no such effect. The signature therefore points to residual exogeneity contamination by slowly correlated detector noise. The depth-consistency check of Remark 3 detects this failure, which is invisible to the first-stage F F F , and escalates to depth two, with GMM weighting then suppressing the attenuation at every power. The slope-based diffusion estimate Eq. (21) , robust to short-lag distortion by construction, stays within 3% of the Einstein value at either depth.
Discussion
The weak form has gained popularity in deterministic equation learning by trading a noise-amplifying operation (differentiation) for a noise-suppressing one (integration). The same trade carries a hidden cost in the stochastic setting. The smoothing integral that makes the algorithm robust to measurement noise also entangles the driving Brownian increment in both the regressor and the residual, resulting in endogeneity that leaves ordinary least-squares estimators biased and inconsistent (§ 3 ). We restore consistency through instrumental variables built from nearby disjoint windows, whose disjointness ensures exogeneity, through independence of Brownian increments over non-overlapping intervals, while their proximity preserves relevance. The resulting estimator, WAIVES, is a consistent, asymptotically normal, and asymptotically unbiased GMM estimator. While WAIVES is presented here as the inference layer, it can also be used for simple model discovery. Given a library of term candidates, it provides unbiased coefficients with calibrated standard errors, so dropping the terms whose coefficients are statistically indistinguishable from zero, and refitting, provides a feasible and fast model selection, like what was done in § 8.1 for the linear against the cubic drift.
WAIVES needs no likelihood, which makes it an order of magnitude faster than the Kalman maximum-likelihood fit on OU (Appendix § 15 ) and applicable where likelihoods cannot be evaluated. We have presented several examples with joint drift and diffusion identification for diagonal diffusion (constant or entering through state-dependent diffusion term candidates). Every estimate reported here is based on a single trajectory. When several trajectories of the same system are available, the moment conditions, and the noise covariance, could be pooled across them, further reducing the standard errors (Appendix § 13 , examines the noise-covariance pooling).
In terms of limitations, diffusion recovery can be challenging when measurement noise is large relative to the increment, that is, when the noise-to-signal ratio ϱ i = 2 R i i / ( σ i 2 Δ t ) \varrho_i = 2R_{ii}/(\sigma_i^2\Delta t) ϱ i = 2 R ii / ( σ i 2 Δ t ) of § 4 is large. In such situations the accuracy of the diffusion estimators decreases (see § 7 ). Refining Δ t \Delta t Δ t improves the drift noise condition ∥ R ∥ o p Δ t / ( σ min 2 T w 2 ) → 0 \lVert R\rVert_{\mathrm{op}}\Delta t/(\sigma_{\min}^2T_w^2) \to0 ∥ R ∥ op Δ t / ( σ m i n 2 T w 2 ) → 0 but inflates ϱ \varrho ϱ , so finer sampling makes the diffusion harder to recover [68 ] ; with R R R set by the measurement device, Δ t \Delta t Δ t is a compromise, resolved by the operating-range diagnostics.
Several extensions are immediate, including a WSINDy-style extension of WAIVES [2 ] , for model discovery at scale, with a systematic study of sparsification and regularization. Similarly, we will extend WAIVES to accommodate partially observed states (d y < d d_y < d d y < d in Remark 2), where some state variable trajectories are completely unobserved and the instruments must be built from the observed components alone. Finally, we note that the methodology presented here is not restricted to SDEs. Any weak-form regression whose residual is a localized stochastic integral potentially faces the same endogeneity and can be addressed by the same principles. Specifically, space-time test functions for SPDEs are the natural next application, and our current research is developing the corresponding instrumental-variable theory and fast, likelihood-free algorithms for SPDEs as well.
Data and code availability
All data are publicly available. The unsmoothed AMO index (file amon.us.long.data.txt, January 1856–January 2023) is from NOAA PSL (https://psl.noaa.gov/data/timeseries/AMO/ ), Kaplan SST V2 source (DOI 10.1029/97JC01736). The optical-trap records are the files Data_x_positions_Exp_{I,II,III}.mat of Gieseler et al. tutorial data (DOI 10.1364/AOP.394888, Sec. 3 calibration data set). The code used to generate the tables and figures in the manuscript is archived on Zenodo (DOI 10.5281/zenodo.24369122).
Acknowledgments
The authors thank Daniel A. Messenger for valuable discussions during the development of this work, and Prof. Ian Grooms for his helpful feedback on the analysis of AMO index data. This work was funded in part by the NSF Division Of Environmental Biology (EEID grant DEB-2109774) Division of Mathematical Sciences (eMB grant DMS-2527532), NIFA Biological Sciences (grant 2019-67014-29919), and the NIH-NIGMS Division of Biophysics, Biomedical Technology, and Computational Bioscience (grant MIRA R35GM149335). This work utilized the Blanca condo computing resource at the University of Colorado Boulder. Blanca is jointly funded by computing users and the University of Colorado Boulder. Large language models (Claude, Anthropic; ChatGPT, OpenAI) were used under author direction for manuscript editing, for assistance with analysis and replication code, and for literature, reference, and internal-consistency checking. All scientific ideas, derivations, and conclusions are the authors’ own; the authors verified all results, proofs, and references and take full responsibility for the content of this article.
Proofs
Validity of the disjoint-window instrument
This appendix proves Proposition 1 and then establishes the two instrument conditions (IV1)–(IV2) for the construction of Definition 1.
Proof of Proposition 1. Center each g j ( X ) g_j(\mathbf{X}) g j ( X ) and each component X ( i ) X^{(i)} X ( i ) at its stationary mean; this changes neither b i b_i b i nor any covariance. Fix t t t and let s s s vary. For s ≠ t s \ne t s = t the product rule gives d s ( g j ( X t ) X s ( i ) ) = g j ( X t ) d X s ( i ) \mathrm{d}_s\left(g_j(X_t)X_s^{(i)}\right)=g_j(X_t)\mathrm{d}X_s^{(i)} d s ( g j ( X t ) X s ( i ) ) = g j ( X t ) d X s ( i ) , since g j ( X t ) g_j(X_t) g j ( X t ) does not vary with s s s ; taking expectations and using stationarity, E [ g j ( X t ) d X s ( i ) ] = ∂ s C j i ( t − s ) d s = − C j i ′ ( t − s ) d s \mathbb{E}[g_j(X_t)\,\mathrm{d}X_s^{(i)}]=\partial_s C_{ji}(t-s)\,\mathrm{d}s=-C'_{ji}(t-s)\,\mathrm{d}s E [ g j ( X t ) d X s ( i ) ] = ∂ s C j i ( t − s ) d s = − C j i ′ ( t − s ) d s . Reversibility and stationarity give C j i ( − u ) = Cov ( g j ( X 0 ) , X u ( i ) ) = Cov ( g j ( X u ) , X 0 ( i ) ) = C j i ( u ) C_{ji}(-u)=\operatorname{Cov}(g_j(X_0),X_u^{(i)})=\operatorname{Cov}(g_j(X_u),X_0^{(i)})=C_{ji}(u) C j i ( − u ) = Cov ( g j ( X 0 ) , X u ( i ) ) = Cov ( g j ( X u ) , X 0 ( i ) ) = C j i ( u ) , so C j i C_{ji} C j i is even and C j i ′ C'_{ji} C j i ′ is odd on u ≠ 0 u\ne0 u = 0 . By the stochastic Fubini theorem, justified by the stated moment bounds and the boundedness of ϕ \phi ϕ and C j i ′ C'_{ji} C j i ′ ,
E [ a j b i ] = − ∬ [ a , b ] 2 ϕ ( t ) ϕ ( s ) C j i ′ ( t − s ) d s d t = 0 , \mathbb{E}[a_jb_i]=-\iint_{[a,b]^2}\phi(t)\phi(s)C'_{ji}(t-s)\,\mathrm{d}s\,\mathrm{d}t=0, E [ a j b i ] = − ∬ [ a , b ] 2 ϕ ( t ) ϕ ( s ) C j i ′ ( t − s ) d s d t = 0 ,
the integrand being odd under the exchange of s s s and t t t on a symmetric domain. The diagonal s = t s=t s = t carries no contribution, since it has Lebesgue measure zero, d X \mathrm{d}X d X has no atoms in time, and E [ g j ( X t ) σ i ( X t ) d W t ] = 0 \mathbb{E}[g_j(X_t)\sigma_i(X_t)\,\mathrm{d}W_t]=0 E [ g j ( X t ) σ i ( X t ) d W t ] = 0 , where σ i \sigma_i σ i is the i i i th row of σ \sigma σ . At stationarity E [ b i ] = ∫ ϕ E [ d X ( i ) ] = 0 \mathbb{E}[b_i]=\int\phi\,\mathbb{E}[\mathrm{d}X^{(i)}]=0 E [ b i ] = ∫ ϕ E [ d X ( i ) ] = 0 , so Cov ( a j , b i ) = E [ a j b i ] = 0 \operatorname{Cov}(a_j,b_i)=\mathbb{E}[a_jb_i]=0 Cov ( a j , b i ) = E [ a j b i ] = 0 . The moment vector of the regression of b i b_i b i on the drift term candidates therefore vanishes, and so does its population coefficient vector whenever the Gram matrix is invertible.
As for the two special cases, a stationary ergodic scalar diffusion has zero stationary probability flux and is therefore reversible, so the conclusion holds for every choice of drift term candidates; and for g ( x ) = x g(x)=x g ( x ) = x in one dimension, C ( u ) = Cov ( X u , X 0 ) C(u)=\operatorname{Cov}(X_u,X_0) C ( u ) = Cov ( X u , X 0 ) is even for any stationary process, so the conclusion for the affine drift holds without reversibility.
The argument uses only the frozen-t t t product rule and the evenness of the cross-covariances, so the proposition extends verbatim to stationary, time-reversible, square-integrable semimartingales whose cross-covariances are continuously differentiable at all nonzero lags, including jump-diffusions. The identity it discards on the diagonal, in the scalar case E [ X t d X t ] = − 1 2 E [ σ 2 ( X 0 ) ] d t \mathbb{E}[X_t\,\mathrm{d}X_t]=-\frac{1}{2}\mathbb{E}[\sigma^2(X_0)]\,\mathrm{d}t E [ X t d X t ] = − 2 1 E [ σ 2 ( X 0 )] d t , follows from d E [ X t 2 ] = 0 \mathrm{d}\,\mathbb{E}[X_t^2]=0 d E [ X t 2 ] = 0 at stationarity, and is what makes the adapted one-step regression consistent.
For OU, E [ A 2 ] = σ 2 ∬ [ a , b ] 2 ϕ ( t ) ϕ ( s ) e − κ ∣ t − s ∣ d s d t > 0 \mathbb{E}[A^2]=\sigma^2\iint_{[a,b]^2}\phi(t)\phi(s)e^{-\kappa|t-s|}\,\mathrm{d}s\,\mathrm{d}t>0 E [ A 2 ] = σ 2 ∬ [ a , b ] 2 ϕ ( t ) ϕ ( s ) e − κ ∣ t − s ∣ d s d t > 0 for every nonzero ϕ \phi ϕ , since the kernel e − κ ∣ t − s ∣ e^{-\kappa|t-s|} e − κ ∣ t − s ∣ is strictly positive definite, being the Fourier transform of the everywhere-positive Cauchy density κ / [ π ( κ 2 + ω 2 ) ] \kappa/[\pi(\kappa^2+\omega^2)] κ / [ π ( κ 2 + ω 2 )] . With b = − κ A + ϵ b=-\kappa A+\epsilon b = − κ A + ϵ and E [ A b ] = 0 \mathbb{E}[Ab]=0 E [ A b ] = 0 , this gives E [ A ϵ ] = κ E [ A 2 ] > 0 \mathbb{E}[A\epsilon]=\kappa\mathbb{E}[A^2]>0 E [ A ϵ ] = κ E [ A 2 ] > 0 .
We now prove the two instrument conditions, Propositions 2 and 3, for the construction of Definition 1.
Recall the target residual ϵ k = ∫ a k a k + T w ϕ k ( t ) σ ( X t ) d W t \epsilon_k=\int_{a_k}^{a_k+T_w}\phi_k(t)\sigma(X_t)\,\mathrm{d}W_t ϵ k = ∫ a k a k + T w ϕ k ( t ) σ ( X t ) d W t and the instrument Z k j = ∫ a k ′ a k ′ + T w ψ k ( t ) g j ( X t ) d t Z_{kj}=\int_{a'_k}^{a'_k+T_w}\psi_k(t)g_j(X_t)\,\mathrm{d}t Z k j = ∫ a k ′ a k ′ + T w ψ k ( t ) g j ( X t ) d t with [ a k ′ , a k ′ + T w ) ∩ [ a k , a k + T w ] = ∅ [a'_k,a'_k+T_w)\cap[a_k,a_k+T_w]=\varnothing [ a k ′ , a k ′ + T w ) ∩ [ a k , a k + T w ] = ∅ , the instrument interval taken half-open so that the possibly shared endpoint a k ′ + T w = a k a'_k+T_w=a_k a k ′ + T w = a k is excluded, and a k ′ + T w ≤ a k a'_k+T_w\le a_k a k ′ + T w ≤ a k .
Proof of Proposition 2. Write the total residual including observation noise as ϵ k t o t = ϵ k + δ b k − ∑ j θ j δ A k j \epsilon_k^{\mathrm{tot}}=\epsilon_k+\delta b_k-\sum_j\theta_j\delta A_{kj} ϵ k tot = ϵ k + δ b k − ∑ j θ j δ A k j , where ϵ k = ∫ a k a k + T w ϕ k ( t ) σ ( X t ) d W t \epsilon_k=\int_{a_k}^{a_k+T_w}\phi_k(t)\sigma(X_t)\,\mathrm{d}W_t ϵ k = ∫ a k a k + T w ϕ k ( t ) σ ( X t ) d W t is the process-noise term and δ b k , δ A k j \delta b_k,\delta A_{kj} δ b k , δ A k j are the contributions of the measurement errors { η n } n ∈ I k \{\eta_n\}_{n\in\mathcal{I}_k} { η n } n ∈ I k to the target-window quadratures. We show each piece is uncorrelated with Z k j Z_{kj} Z k j .
As for the process noise, let F t = σ ( { W s : s ≤ t } ∪ { η m : t m ≤ t } ) \mathcal{F}_t=\sigma(\{W_s:s\le t\}\cup\{\eta_m:t_m\le t\}) F t = σ ({ W s : s ≤ t } ∪ { η m : t m ≤ t }) be the filtration generated by the Brownian path and the errors up to time t t t . The exact instrument Z k j = ∫ a k ′ a k ′ + T w ψ k ( t ) g j ( X t ) d t Z_{kj}=\int_{a'_k}^{a'_k+T_w}\psi_k(t)g_j(X_t)\,\mathrm{d}t Z k j = ∫ a k ′ a k ′ + T w ψ k ( t ) g j ( X t ) d t is a functional of the path { X t : a k ′ ≤ t ≤ a k ′ + T w } \{X_t:a'_k\le t\le a'_k+T_w\} { X t : a k ′ ≤ t ≤ a k ′ + T w } only, and its observable form of Section 4 is a function of the observations Y t m = X t m + η m Y_{t_m}=X_{t_m}+\eta_m Y t m = X t m + η m at the instants t m ∈ I k ′ ⊂ [ a k ′ , a k ′ + T w ] t_m\in\mathcal{I}'_k\subset[a'_k,a'_k+T_w] t m ∈ I k ′ ⊂ [ a k ′ , a k ′ + T w ] . Every time involved is at most a k ′ + T w a'_k+T_w a k ′ + T w , so Z k j Z_{kj} Z k j is F a k ′ + T w \mathcal{F}_{a'_k+T_w} F a k ′ + T w -measurable. Since a k ′ + T w ≤ a k a'_k+T_w\le a_k a k ′ + T w ≤ a k and the filtration grows with time, F a k ′ + T w ⊆ F a k \mathcal{F}_{a'_k+T_w}\subseteq\mathcal{F}_{a_k} F a k ′ + T w ⊆ F a k , so Z k j Z_{kj} Z k j is F a k \mathcal{F}_{a_k} F a k -measurable as well. The integrand ϕ k ( t ) σ ( X t ) \phi_k(t)\sigma(X_t) ϕ k ( t ) σ ( X t ) of ϵ k \epsilon_k ϵ k is adapted, so ϵ k \epsilon_k ϵ k is an Itô integral over [ a k , a k + T w ] [a_k,a_k+T_w] [ a k , a k + T w ] and hence a martingale increment beyond a k a_k a k , E [ ϵ k ∣ F a k ] = 0 \mathbb{E}[\epsilon_k\mid\mathcal{F}_{a_k}]=0 E [ ϵ k ∣ F a k ] = 0 . By the tower property,
E [ Z k j ϵ k ] = E [ Z k j E [ ϵ k ∣ F a k ] ] = E [ Z k j ⋅ 0 ] = 0. (30) \mathbb{E}[Z_{kj}\epsilon_k]=\mathbb{E}\left[Z_{kj}\mathbb{E}[\epsilon_k\mid\mathcal{F}_{a_k}]\right]=\mathbb{E}[Z_{kj}\cdot0]=0. \tag*{(30)} E [ Z k j ϵ k ] = E [ Z k j E [ ϵ k ∣ F a k ] ] = E [ Z k j ⋅ 0 ] = 0. (30)
Next we deal with the observation noise. The target perturbations δ b k , δ A k j \delta b_k,\delta A_{kj} δ b k , δ A k j are linear (to first order) in { η n } n ∈ I k \{\eta_n\}_{n\in\mathcal{I}_k} { η n } n ∈ I k , with coefficients that depend on the latent path through terms such as g j ′ ( X t n ) g'_j(X_{t_n}) g j ′ ( X t n ) , while Z k j Z_{kj} Z k j depends on { W t } \{W_t\} { W t } and on { η m } m ∈ I k ′ \{\eta_m\}_{m\in\mathcal{I}'_k} { η m } m ∈ I k ′ . Let G k \mathcal{G}_k G k be the σ \sigma σ -algebra generated by the latent path { X t } \{X_t\} { X t } and the instrument-window errors { η m } m ∈ I k ′ \{\eta_m\}_{m\in\mathcal{I}'_k} { η m } m ∈ I k ′ . Then Z k j Z_{kj} Z k j and the perturbation coefficients are G k \mathcal{G}_k G k -measurable, while each η n \eta_n η n with n ∈ I k n\in\mathcal{I}_k n ∈ I k is mean zero and independent of G k \mathcal{G}_k G k , because the index sets I k , I k ′ \mathcal{I}_k,\mathcal{I}'_k I k , I k ′ are disjoint and { η n } \{\eta_n\} { η n } is i.i.d. and independent of { W t } \{W_t\} { W t } . Conditioning therefore gives E [ δ b k ∣ G k ] = E [ δ A k j ∣ G k ] = 0 \mathbb{E}[\delta b_k\mid\mathcal{G}_k]=\mathbb{E}[\delta A_{kj}\mid\mathcal{G}_k]=0 E [ δ b k ∣ G k ] = E [ δ A k j ∣ G k ] = 0 (for δ A k j \delta A_{kj} δ A k j with the debiasing at the true R R R ; the plug-in R ^ \widehat R R depends on the target-window errors and is not G k \mathcal{G}_k G k -measurable, which Proposition 4 handles), and the tower property yields E [ Z k j δ b k ] = E [ Z k j E [ δ b k ∣ G k ] ] = 0 \mathbb{E}[Z_{kj}\delta b_k]=\mathbb{E}[Z_{kj}\mathbb{E}[\delta b_k\mid\mathcal{G}_k]]=0 E [ Z k j δ b k ] = E [ Z k j E [ δ b k ∣ G k ]] = 0 and likewise E [ Z k j δ A k j ] = 0 \mathbb{E}[Z_{kj}\delta A_{kj}]=0 E [ Z k j δ A k j ] = 0 . The suppressed second-order terms are quadratic in the errors with mean O ( ∥ R ∥ ) O(\|R\|) O ( ∥ R ∥ ) ; the debiased drift term candidates of § 6 remove that mean, so what survives in ρ k \rho_k ρ k has mean O ( ∥ R ∥ 2 ) O(\|R\|^2) O ( ∥ R ∥ 2 ) beside the O ( Δ t ) O(\Delta t) O ( Δ t ) quadrature part. Combining with Eq. (30) gives the displayed order.
Proof of Proposition 3. It suffices to show E [ Z T A ] \mathbb{E}[\mathbf{Z}^{\mathsf T}\mathbf{A}] E [ Z T A ] is nonsingular, since full column rank of a p × p p\times p p × p matrix is non-singularity. Consider a single window pair and a pair of term indices ( j , j ′ ) (j,j') ( j , j ′ ) . By stationarity and Fubini,
Q j j ′ = E [ Z k j A k j ′ ] = ∫ a k ′ a k ′ + T w ∫ a k a k + T w ψ k ( s ) ϕ k ( t ) E [ g j ( X s ) g j ′ ( X t ) ] d s d t . (31) Q_{jj'}=\mathbb{E}[Z_{kj}A_{kj'}]
=\int_{a'_k}^{a'_k+T_w}\int_{a_k}^{a_k+T_w}\psi_k(s)\phi_k(t)\mathbb{E}[g_j(X_s)g_{j'}(X_t)]\,\mathrm{d}s\,\mathrm{d}t. \tag*{(31)} Q j j ′ = E [ Z k j A k j ′ ] = ∫ a k ′ a k ′ + T w ∫ a k a k + T w ψ k ( s ) ϕ k ( t ) E [ g j ( X s ) g j ′ ( X t )] d s d t . (31)
Center the drift term candidates, g ˉ j = E [ g j ( X 0 ) ] \bar{g}_j=\mathbb{E}[g_j(X_0)] g ˉ j = E [ g j ( X 0 )] , and write E [ g j ( X s ) g j ′ ( X t ) ] = g ˉ j g ˉ j ′ + Γ j j ′ ( t − s ) \mathbb{E}[g_j(X_s)g_{j'}(X_t)]=\bar{g}_j\bar{g}_{j'}+\Gamma_{jj'}(t-s) E [ g j ( X s ) g j ′ ( X t )] = g ˉ j g ˉ j ′ + Γ j j ′ ( t − s ) . Collect the means in g ˉ = ( g ˉ 1 , … , g ˉ p ) ⊤ \bar{g}=(\bar{g}_1,\ldots,\bar{g}_p)^\top g ˉ = ( g ˉ 1 , … , g ˉ p ) ⊤ . Both test functions are nonnegative, and every separation ∣ t − s ∣ |t-s| ∣ t − s ∣ arising in Eq. (31) lies in [ δ , δ + 2 T w ] [\delta,\delta+2T_w] [ δ , δ + 2 T w ] . For any ζ ≠ 0 \zeta\ne0 ζ = 0 the quadratic form ζ ⊤ Q ζ = ∑ j , j ′ ζ j ζ j ′ Q j j ′ \zeta^\top Q\zeta=\sum_{j,j'}\zeta_j\zeta_{j'}Q_{jj'} ζ ⊤ Qζ = ∑ j , j ′ ζ j ζ j ′ Q j j ′ is therefore
ζ ⊤ Q ζ = ∫ a k ′ a k ′ + T w ∫ a k a k + T w ψ k ( s ) ϕ k ( t ) ( ( ζ ⊤ g ˉ ) 2 + ζ ⊤ Γ ( t − s ) ζ ) d s d t ≥ m ∥ ζ ∥ 2 μ ψ μ ϕ , \zeta^\top Q\zeta
=\int_{a'_k}^{a'_k+T_w}\int_{a_k}^{a_k+T_w}\psi_k(s)\phi_k(t)\left((\zeta^\top\bar{g})^2+\zeta^\top\Gamma(t-s)\zeta\right)\,\mathrm{d}s\,\mathrm{d}t
\ge m\|\zeta\|^2\mu_\psi\mu_\phi, ζ ⊤ Qζ = ∫ a k ′ a k ′ + T w ∫ a k a k + T w ψ k ( s ) ϕ k ( t ) ( ( ζ ⊤ g ˉ ) 2 + ζ ⊤ Γ ( t − s ) ζ ) d s d t ≥ m ∥ ζ ∥ 2 μ ψ μ ϕ ,
with μ ψ = ∫ ψ k > 0 \mu_\psi=\int\psi_k>0 μ ψ = ∫ ψ k > 0 and μ ϕ = ∫ ϕ k > 0 \mu_\phi=\int\phi_k>0 μ ϕ = ∫ ϕ k > 0 , so Q Q Q is positive definite.
Davydov’s covariance inequality, which is valid for β \beta β -mixing processes, bounds each entry by ∣ Γ j j ′ ( u ) ∣ ≤ 8 β ( u ) 1 − 2 / ν ∥ g j ( X 0 ) ∥ ν ∥ g j ′ ( X 0 ) ∥ ν |\Gamma_{jj'}(u)|\le8\beta(u)^{1-2/\nu}\|g_j(X_0)\|_\nu\|g_{j'}(X_0)\|_\nu ∣ Γ j j ′ ( u ) ∣ ≤ 8 β ( u ) 1 − 2/ ν ∥ g j ( X 0 ) ∥ ν ∥ g j ′ ( X 0 ) ∥ ν , so Γ ( u ) \Gamma(u) Γ ( u ) vanishes as u → ∞ u\to\infty u → ∞ , exponentially fast for the geometrically ergodic systems of Remark 5. The covariances of the drift term candidates are therefore appreciable for separations u ≲ τ u\lesssim\tau u ≲ τ , where τ \tau τ is the characteristic time, and negligible beyond. The positivity-within-memory hypothesis is therefore a short-separation condition. It holds whenever δ + T w ≲ τ \delta+T_w\lesssim\tau δ + T w ≲ τ for systems whose covariances of the drift term candidates decay without sign changes, and it can fail at particular lags for strongly oscillatory dynamics, which is where relevance degrades empirically.
Conversely, once δ + T w ≫ τ \delta+T_w\gg\tau δ + T w ≫ τ the fluctuating part is O ( β ) O(\beta) O ( β ) and only the rank-one constant part survives, so the matrix becomes ill-conditioned and (IV1) degrades. □
Remark 4. Proposition 3 is the theoretical basis of the empirical window-selection principle in the main text. Instruments are strong when T w ≲ τ T_w\lesssim\tau T w ≲ τ and weak otherwise. The sufficient condition can be checked directly when correlations decay monotonically within the window geometry; for oscillatory systems such as FitzHugh–Nagumo and Lorenz-63 it can fail at some separations, and nonsingularity of Q Q Q is then the operative hypothesis, verified empirically through the first-stage F F F , which is its sample test.
Consistency and asymptotic normality
Proof of Theorem 1. The IV estimator admits the exact error decomposition (from Eq. (11) )
θ ^ I V − θ = ( 1 K Z ⊤ A ) − 1 ( 1 K Z ⊤ ε t o t ) . (32) \hat{\theta}_{\mathrm{IV}}-\theta
=\left(\frac{1}{K}Z^\top A\right)^{-1}\left(\frac{1}{K}Z^\top\varepsilon^{\mathrm{tot}}\right). \tag*{(32)} θ ^ IV − θ = ( K 1 Z ⊤ A ) − 1 ( K 1 Z ⊤ ε tot ) . (32)
We treat the two factors in turn.
Consider first the denominator of Eq. (32) . The summands Z k j A k j ′ Z_{kj}A_{kj'} Z k j A k j ′ are functions of the path over disjoint window pairs of a stationary β \beta β -mixing process, so they form a stationary mixing array with the same mixing coefficients. Their common mean is E [ Z 1 j A 1 j ′ ] \mathbb{E}[Z_{1j}A_{1j'}] E [ Z 1 j A 1 j ′ ] , finite under the moment assumptions. Under ∑ u ≥ 1 β ( u ) 1 − 2 / ν < ∞ \sum_{u\ge1}\beta(u)^{1-2/\nu}<\infty ∑ u ≥ 1 β ( u ) 1 − 2/ ν < ∞ , Davydov’s inequality makes the autocovariances of this array summable, so Var ( 1 K ∑ k Z k j A k j ′ ) = O ( 1 / K ) \operatorname{Var}(\frac{1}{K}\sum_k Z_{kj}A_{kj'})=O(1/K) Var ( K 1 ∑ k Z k j A k j ′ ) = O ( 1/ K ) ; the array changes with Δ t \Delta t Δ t , its entries being quadratures of the path, but its mixing coefficients are those of the path at the same time separations and do not depend on Δ t \Delta t Δ t , so the bound holds uniformly along the joint limit, and a law of large numbers for β \beta β -mixing arrays gives 1 K Z ⊤ A → P Q \frac{1}{K}Z^\top A\xrightarrow{\mathbb{P}}Q K 1 Z ⊤ A P Q , the per-pair mean matrix Q = E [ Z 1 A 1 ⊤ ] Q=\mathbb{E}[Z_1A_1^\top] Q = E [ Z 1 A 1 ⊤ ] , which is invertible by Proposition 3.
On the other hand, for the numerator, by exogeneity (Proposition 2) each summand Z k ε k t o t Z_k\varepsilon_k^{\mathrm{tot}} Z k ε k tot has mean zero, so E [ 1 K Z ⊤ ε t o t ] = 0 \mathbb{E}[\frac{1}{K}Z^\top\varepsilon^{\mathrm{tot}}]=0 E [ K 1 Z ⊤ ε tot ] = 0 . Its variance has three sources, each vanishing under the stated rates, namely (i) the averaging of K K K weakly dependent mean-zero terms contributes O ( 1 / K ) O(1/K) O ( 1/ K ) ; (ii) replacing the continuous-time integrals by their data quadratures incurs an O ( Δ t ) O(\Delta t) O ( Δ t ) bias per window, whose average over the K K K windows is itself O ( Δ t ) O(\Delta t) O ( Δ t ) and vanishes as Δ t → 0 \Delta t\to0 Δ t → 0 with no condition on K K K (the rate K Δ t 2 → 0 K\Delta t^2\to0 K Δ t 2 → 0 is needed only once the sum is scaled by K \sqrt{K} K in the CLT) controlled in aggregate by K Δ t 2 → 0 K\Delta t^2\to0 K Δ t 2 → 0 ; and (iii) the observation-noise contribution scales as ∥ R ∥ o p Δ t / ( σ m 2 T w 2 ) \|R\|_{\mathrm{op}}\Delta t/(\sigma_m^2T_w^2) ∥ R ∥ op Δ t / ( σ m 2 T w 2 ) by the noise-ratio bound Eq. (16) , which holds entrywise for matrix R R R with these norms. Hence 1 K Z ⊤ ε t o t → P 0 \frac{1}{K}Z^\top\varepsilon^{\mathrm{tot}}\xrightarrow{\mathbb{P}}0 K 1 Z ⊤ ε tot P 0 .
Applying the continuous-mapping and Slutsky [69 ] theorems to Eq. (32) , a convergent invertible matrix times a vanishing vector, gives θ ^ I V − θ → P 0 \hat{\theta}_{\mathrm{IV}}-\theta\xrightarrow{\mathbb{P}}0 θ ^ IV − θ P 0 . □
Proof of Theorem 2. Scaling Eq. (32) by K \sqrt{K} K gives
K ( θ ^ I V − θ ) = ( 1 K Z ⊤ A ) − 1 1 K Z ⊤ ε t o t , \sqrt{K}(\hat{\theta}_{\mathrm{IV}}-\theta)
=\left(\frac{1}{K}Z^\top A\right)^{-1}\frac{1}{\sqrt{K}}Z^\top\varepsilon^{\mathrm{tot}}, K ( θ ^ IV − θ ) = ( K 1 Z ⊤ A ) − 1 K 1 Z ⊤ ε tot ,
and the first factor converges in probability to Q − 1 Q^{-1} Q − 1 as in the consistency proof. For the scaled sum S K = 1 K ∑ k ξ k S_K=\frac{1}{\sqrt{K}}\sum_k\xi_k S K = K 1 ∑ k ξ k with ξ k = Z k ε k t o t \xi_k=Z_k\varepsilon_k^{\mathrm{tot}} ξ k = Z k ε k tot , we use the martingale-difference property. Order the window pairs in time and let G k = F b k \mathcal{G}_k=\mathcal{F}_{b_k} G k = F b k be the σ \sigma σ -algebra generated by the Brownian path and the measurement errors up to the end of pair k k k . Then G k − 1 = F b k − 1 ⊆ F a k \mathcal{G}_{k-1}=\mathcal{F}_{b_{k-1}}\subseteq\mathcal{F}_{a_k} G k − 1 = F b k − 1 ⊆ F a k , because b k − 1 ≤ a k b_{k-1}\le a_k b k − 1 ≤ a k , and ξ k \xi_k ξ k is G k \mathcal{G}_k G k -measurable. Split the total residual as ε k t o t = ε ~ k + ρ k \varepsilon_k^{\mathrm{tot}}=\tilde{\varepsilon}_k+\rho_k ε k tot = ε ~ k + ρ k , where ε ~ k \tilde{\varepsilon}_k ε ~ k collects the process-noise integral ε k \varepsilon_k ε k and the observation-noise terms, and ρ k \rho_k ρ k is the quadrature remainder of Proposition 2, of mean O ( Δ t ) O(\Delta t) O ( Δ t ) . The proof of that proposition gives E [ ε ~ k ∣ F a k ] = 0 \mathbb{E}[\tilde{\varepsilon}_k\mid\mathcal{F}_{a_k}]=0 E [ ε ~ k ∣ F a k ] = 0 exactly. Indeed ε k \varepsilon_k ε k is a martingale increment beyond a k a_k a k , and the observation-noise terms are functions of the errors in the target window. Those errors are independent of F a k \mathcal{F}_{a_k} F a k and of the path, and they have mean zero given the path once the drift term candidates are debiased; this holds exactly for Gaussian errors and the polynomial terms of degree at most three and trigonometric terms used here. For other terms the debiasing leaves a remainder of mean O ( ∥ R ∥ 2 ) O(\lVert R\rVert^2) O (∥ R ∥ 2 ) in ρ k \rho_k ρ k , and the rate K ∥ R ∥ 2 → 0 \sqrt{K}\lVert R\rVert^2 \to0 K ∥ R ∥ 2 → 0 is then required as well. Write ξ k = ξ ~ k + Z k ρ k \xi_k = \tilde{\xi}_k + Z_k\rho_k ξ k = ξ ~ k + Z k ρ k with ξ ~ k = Z k ε ~ k \tilde{\xi}_k = Z_k\tilde{\varepsilon}_k ξ ~ k = Z k ε ~ k . The tower property, followed by pulling the F a k \mathcal{F}_{a_k} F a k -measurable factor Z k Z_k Z k out of the inner conditional expectation, gives
E [ ξ ~ k ∣ G k − 1 ] = E [ E [ Z k ε ~ k ∣ F a k ] ∣ G k − 1 ] = E [ Z k E [ ε ~ k ∣ F a k ] ∣ G k − 1 ] = 0. (33) \mathbb{E}[\tilde{\xi}_k \mid\mathcal{G}_{k-1}] = \mathbb{E}\left[\mathbb{E}[Z_k\tilde{\varepsilon}_k \mid\mathcal{F}_{a_k}] \mid\mathcal{G}_{k-1}\right]
= \mathbb{E}\left[Z_k\mathbb{E}[\tilde{\varepsilon}_k \mid\mathcal{F}_{a_k}] \mid\mathcal{G}_{k-1}\right] = 0. \tag*{(33)} E [ ξ ~ k ∣ G k − 1 ] = E [ E [ Z k ε ~ k ∣ F a k ] ∣ G k − 1 ] = E [ Z k E [ ε ~ k ∣ F a k ] ∣ G k − 1 ] = 0. (33)
Thus { ξ ~ k , G k } \{\tilde{\xi}_k,\mathcal{G}_k\} { ξ ~ k , G k } is a martingale-difference sequence. The remainder sum 1 K ∑ k Z k ρ k \frac{1}{\sqrt{K}}\sum_k Z_k\rho_k K 1 ∑ k Z k ρ k has mean O ( K Δ t ) O(\sqrt{K}\Delta t) O ( K Δ t ) and variance O ( Δ t 2 ) O(\Delta t^2) O ( Δ t 2 ) , so it vanishes in probability under the rate K Δ t 2 → 0 K\Delta t^2 \to0 K Δ t 2 → 0 of the consistency theorem. It remains to prove the central limit theorem for S ~ K = 1 K ∑ k ξ ~ k \tilde{S}_K = \frac{1}{\sqrt{K}}\sum_k \tilde{\xi}_k S ~ K = K 1 ∑ k ξ ~ k , whose covariance Σ = Var ( ξ ~ 1 ) \boldsymbol{\Sigma} = \operatorname{Var}(\tilde{\xi}_1) Σ = Var ( ξ ~ 1 ) agrees with Var ( Z 1 ε 1 t o t ) \operatorname{Var}(Z_1\varepsilon_1^{\mathrm{tot}}) Var ( Z 1 ε 1 tot ) up to O ( Δ t ) O(\Delta t) O ( Δ t ) . The sequence is strictly stationary, since each ξ ~ k \tilde{\xi}_k ξ ~ k is the same fixed functional of the path over pair k k k and the pairs are translates of one another by the stride 2 T w + δ 2T_w+\delta 2 T w + δ of a stationary process. It is ergodic, since β ( u ) → 0 \beta(u)\to0 β ( u ) → 0 makes the underlying shift mixing, hence ergodic. It is square integrable with Σ \boldsymbol{\Sigma} Σ finite by assumption. For such sequences the martingale central limit theorem applies [70 , 71 ] . The sequence is indexed by Δ t \Delta t Δ t as well as by k k k ; the theorem is stated for arrays, its conditions below hold for every Δ t \Delta t Δ t , and Σ ( Δ t ) → Σ \boldsymbol{\Sigma}(\Delta t)\to\boldsymbol{\Sigma} Σ ( Δ t ) → Σ as the window quadratures converge to their continuous-time integrals, so it applies along the joint limit. Its conditional-variance condition, 1 K ∑ k E [ ξ ~ k ξ ~ k ⊤ ∣ G k − 1 ] ⟶ P Σ \frac{1}{K}\sum_k\mathbb{E}[\tilde{\xi}_k\tilde{\xi}_k^\top\mid\mathcal{G}_{k-1}]\overset{P}{\longrightarrow}\boldsymbol{\Sigma} K 1 ∑ k E [ ξ ~ k ξ ~ k ⊤ ∣ G k − 1 ] ⟶ P Σ , follows from the ergodic theorem applied to the stationary sequence E [ ξ ~ k ξ ~ k ⊤ ∣ G k − 1 ] \mathbb{E}[\tilde{\xi}_k\tilde{\xi}_k^\top\mid\mathcal{G}_{k-1}] E [ ξ ~ k ξ ~ k ⊤ ∣ G k − 1 ] , whose mean is Σ \boldsymbol{\Sigma} Σ by the tower property. Its conditional Lindeberg condition [71 ] holds because the nonnegative averages 1 K ∑ k E [ ∥ ξ ~ k ∥ 2 1 { ∥ ξ ~ k ∥ > ϵ K } ∣ G k − 1 ] \frac{1}{K}\sum_k\mathbb{E}[\lVert\tilde{\xi}_k\rVert^2\mathbf{1}\{\lVert\tilde{\xi}_k\rVert>\epsilon\sqrt{K}\}\mid\mathcal{G}_{k-1}] K 1 ∑ k E [∥ ξ ~ k ∥ 2 1 {∥ ξ ~ k ∥ > ϵ K } ∣ G k − 1 ] have unconditional mean E [ ∥ ξ ~ 1 ∥ 2 1 { ∥ ξ ~ 1 ∥ > ϵ K } ] → 0 \mathbb{E}[\lVert\tilde{\xi}_1\rVert^2\mathbf{1}\{\lVert\tilde{\xi}_1\rVert>\epsilon\sqrt{K}\}]\to0 E [∥ ξ ~ 1 ∥ 2 1 {∥ ξ ~ 1 ∥ > ϵ K }] → 0 by dominated convergence, so they vanish in probability. Hence S ~ K ⟶ d N ( 0 , Σ ) \tilde{S}_K\overset{d}{\longrightarrow}\mathcal{N}(0,\boldsymbol{\Sigma}) S ~ K ⟶ d N ( 0 , Σ ) , so does S K S_K S K , and Slutsky’s theorem gives K ( θ ^ i v − θ ) ⟶ d N ( 0 , Q − 1 Σ Q − ⊤ ) \sqrt{K}(\hat{\theta}_{\mathrm{iv}}-\theta)\overset{d}{\longrightarrow}\mathcal{N}(0,Q^{-1}\boldsymbol{\Sigma}Q^{-\top}) K ( θ ^ iv − θ ) ⟶ d N ( 0 , Q − 1 Σ Q − ⊤ ) .
For the efficiency claim, the GMM estimator Eq. (25) with weight W ^ \widehat{W} W has asymptotic covariance ( Q ⊤ W ^ Q ) − 1 Q ⊤ W ^ Σ W ^ Q ( Q ⊤ W ^ Q ) − 1 (Q^\top\widehat{W}Q)^{-1}Q^\top\widehat{W}\boldsymbol{\Sigma}\widehat{W}Q(Q^\top\widehat{W}Q)^{-1} ( Q ⊤ W Q ) − 1 Q ⊤ W Σ W Q ( Q ⊤ W Q ) − 1 . Minimizing over W ^ \widehat{W} W (a standard generalized least-squares argument) yields the optimum at W ^ ∝ Σ − 1 \widehat{W}\propto\boldsymbol{\Sigma}^{-1} W ∝ Σ − 1 , with covariance ( Q ⊤ Σ − 1 Q ) − 1 (Q^\top\boldsymbol{\Sigma}^{-1}Q)^{-1} ( Q ⊤ Σ − 1 Q ) − 1 , the smallest asymptotic covariance attainable with the given moment conditions [36 ] . □
Proof of Proposition 4. The proof is a routine delta-method argument. The estimator is a continuously differentiable function of R ^ \widehat{R} R , ∥ R ^ − R ∥ = O p ( N − 1 / 2 ) \lVert\widehat{R}-R\rVert=O_p(N^{-1/2}) ∥ R − R ∥ = O p ( N − 1/2 ) by a law of large numbers for the squared increments of the stationary mixing path, and a weight W ^ ⟶ P Σ − 1 \widehat{W}\overset{P}{\longrightarrow}\boldsymbol{\Sigma}^{-1} W ⟶ P Σ − 1 moves the GMM estimator by only o p ( K − 1 / 2 ) o_p(K^{-1/2}) o p ( K − 1/2 ) . □
Remark 5. The stationarity and mixing hypothesis above is justified system by system. For the models in Section § 7 , the OU diffusion is geometrically ergodic. The double-well and FitzHugh–Nagumo drifts are dissipative at infinity with nondegenerate noise, so a Lyapunov and minorization argument gives geometric ergodicity as well. Stochastic Lorenz-63 with additive nondegenerate noise falls inside the framework of [72 ] , which was built for this class. Cox–Ingersoll–Ross is geometrically ergodic, the stochastic Kuramoto network evolves on a compact state space with elliptic noise and is uniformly ergodic, Lorenz-96 is similar to the Lorenz-63 model, and the hypoelliptic underdamped Langevin equation is also covered by the same work of [72 ] . Geometric ergodicity implies exponential β \beta β -mixing of the stationary law, and hence the summability condition assumed above. We make no claim about sharp mixing constants. The noise-ratio condition is stated with the infimum σ min \sigma_{\min} σ m i n and so presumes nondegenerate noise. It does not cover the Cox–Ingersoll–Ross process, whose diffusion vanishes at the boundary, or the position component of the hypoelliptic Langevin equation, for which Appendix Appendix § 13 reports the numerical evidence instead.
Practical instrument and window selection
Operative window
The main text selects L L L empirically. However, for the stationary OU model every quantity entering the estimator is Gaussian, so the window dependence can be computed in closed form, with no simulation. Consider the drift-slope coefficient with one bump φ ( u ) = ( u ( 1 − u ) ) p \varphi(u)=(u(1-u))^p φ ( u ) = ( u ( 1 − u ) ) p per window, span T w = L Δ t T_w=L\Delta t T w = L Δ t , gap δ \delta δ , rate κ \kappa κ , stationary variance σ 2 / 2 κ \sigma^2/2\kappa σ 2 /2 κ , and noise variance R R R at spacing Δ t \Delta t Δ t . Write μ = ∫ 0 1 φ \mu=\int_0^1\varphi μ = ∫ 0 1 φ , χ = ∫ 0 1 φ 2 \chi=\int_0^1\varphi^2 χ = ∫ 0 1 φ 2 and χ = ∫ 0 1 ( φ ′ ) 2 \chi=\int_0^1(\varphi')^2 χ = ∫ 0 1 ( φ ′ ) 2 for the bump’s shape constants, and
n ( λ ) = ∫ 0 1 ∫ 0 1 φ ( u ) φ ( w ) e − λ ∣ u − w ∣ d u d w n(\lambda)=\int_0^1\int_0^1\varphi(u)\varphi(w)e^{-\lambda\lvert u-w\rvert}\,du\,dw n ( λ ) = ∫ 0 1 ∫ 0 1 φ ( u ) φ ( w ) e − λ ∣ u − w ∣ d u d w
for its correlation function within one window, which satisfies n ( λ ) = μ 2 ( 1 − d 1 λ + O ( λ 2 ) ) n(\lambda)=\mu^2(1-d_1\lambda+O(\lambda^2)) n ( λ ) = μ 2 ( 1 − d 1 λ + O ( λ 2 )) where d 1 d_1 d 1 is defined below. Since φ / μ \varphi/\mu φ / μ is the Beta( p + 1 , p + 1 ) (p+1,p+1) ( p + 1 , p + 1 ) density, its variance v 2 v^2 v 2 and mean absolute difference d 1 = E ∣ u − w ∣ d_1=\mathbb{E}\lvert u-w\rvert d 1 = E ∣ u − w ∣ are v 2 = 1 / [ 4 ( 2 p + 3 ) ] v^2=1/[4(2p+3)] v 2 = 1/ [ 4 ( 2 p + 3 )] , and d 1 = 2 [ ( 2 p + 1 ) ! ] 4 / [ ( p + 1 ) ( 4 p + 3 ) ( p ! ) 4 ] d_1=2[(2p+1)!]^4/[(p+1)(4p+3)(p!)^4] d 1 = 2 [( 2 p + 1 )! ] 4 / [( p + 1 ) ( 4 p + 3 ) ( p ! ) 4 ] . Then, we have the following result for the OU model.
Proposition 5. As T → ∞ T\to\infty T → ∞ with K ∼ T / ( 2 T w + δ ) K\sim T/(2T_w+\delta) K ∼ T / ( 2 T w + δ ) window pairs, the exactly identified estimator obeys K ( θ ^ i v − θ ) ⟶ d N ( 0 , V ( T w , δ ) ) \sqrt{K}(\hat{\theta}_{\mathrm{iv}}-\theta)\overset{d}{\longrightarrow}\mathcal{N}(0,V(T_w,\delta)) K ( θ ^ iv − θ ) ⟶ d N ( 0 , V ( T w , δ )) with V = E [ Z 2 ] E [ ( ε t o t ) 2 ] / E [ Z A ] 2 V=\mathbb{E}[Z^2]\mathbb{E}[(\varepsilon^{\mathrm{tot}})^2]/\mathbb{E}[Z A]^2 V = E [ Z 2 ] E [( ε tot ) 2 ] / E [ Z A ] 2 , and
V ( T w , δ ) ≈ 4 κ 2 e 2 κ ( δ + a ∗ T w ) σ 2 σ 4 μ 4 T w 4 [ σ 2 2 κ T w 2 n ( κ T w ) + R Δ t T w ] × [ σ 2 v 2 T w + κ 2 R Δ t v T w + R Δ t χ q ( p , L ) / T w ] , V(T_w,\delta)\approx\frac{4\kappa^2e^{2\kappa(\delta+a_*T_w)}\sigma^2}{\sigma^4\mu^4T_w^4}\left[\frac{\sigma^2}{2\kappa}T_w^2n(\kappa T_w)+R\Delta t\,T_w\right]
\times\left[\sigma^2v^2T_w+\kappa^2R\Delta t\,vT_w+R\Delta t\,\chi q(p,L)/T_w\right], V ( T w , δ ) ≈ σ 4 μ 4 T w 4 4 κ 2 e 2 κ ( δ + a ∗ T w ) σ 2 [ 2 κ σ 2 T w 2 n ( κ T w ) + R Δ t T w ] × [ σ 2 v 2 T w + κ 2 R Δ t v T w + R Δ t χ q ( p , L ) / T w ] ,
where a ∗ ∈ ( 0 , 1 ] a_*\in(0,1] a ∗ ∈ ( 0 , 1 ] is shape dependent with a ∗ = 1 − v 2 κ T w + O ( ( κ T w ) 3 ) a_*=1-v^2\kappa T_w+O((\kappa T_w)^3) a ∗ = 1 − v 2 κ T w + O (( κ T w ) 3 ) , and q ( p , L ) ≥ 1 q(p,L)\ge1 q ( p , L ) ≥ 1 is a discretization factor with q → 1 q \to1 q → 1 as L / p → ∞ L/p \to\infty L / p → ∞ . With ρ = χ q R Δ t / ( ν σ 2 ) \rho= \chi q R\Delta t/(\nu\sigma^2) ρ = χ q R Δ t / ( ν σ 2 ) and a 0 = 1 − d 1 / 2 a_0 = 1-d_1/2 a 0 = 1 − d 1 /2 , the span that minimizes Var ( θ ^ ) = V / K \operatorname{Var}(\hat{\theta}) = V/K Var ( θ ^ ) = V / K solves, to leading order in κ T w \kappa T_w κ T w and δ / T w \delta/T_w δ / T w , κ a 0 T w ( T w 2 + ρ ) = ρ \kappa a_0 T_w(T_w^2+\rho)=\rho κ a 0 T w ( T w 2 + ρ ) = ρ , so that κ T w ∗ = ( κ 2 ρ / a 0 ) 1 / 3 \kappa T_w^* = (\kappa^2\rho/a_0)^{1/3} κ T w ∗ = ( κ 2 ρ / a 0 ) 1/3 when κ 2 ρ ≪ 1 \kappa^2\rho\ll1 κ 2 ρ ≪ 1 and κ T w ∗ → 1 / a 0 \kappa T_w^* \to1/a_0 κ T w ∗ → 1/ a 0 when κ 2 ρ ≫ 1 \kappa^2\rho\gg1 κ 2 ρ ≫ 1 .
Proof. All three moments, E [ Z 2 ] \mathbb{E}[Z^2] E [ Z 2 ] , E [ ( ε t o t ) 2 ] \mathbb{E}[(\varepsilon^{\mathrm{tot}})^2] E [( ε tot ) 2 ] and E [ Z A ] 2 \mathbb{E}[ZA]^2 E [ Z A ] 2 are double integrals of the stationary kernel σ 2 2 κ e − κ ∣ t − s ∣ \frac{\sigma^2}{2\kappa}e^{-\kappa|t-s|} 2 κ σ 2 e − κ ∣ t − s ∣ against the bump and its translate, and the substitution s = a k − δ − T w + T w u s=a_k-\delta-T_w+T_wu s = a k − δ − T w + T w u , t = a k + T w w t=a_k+T_ww t = a k + T w w maps each window to [ 0 , 1 ] [0,1] [ 0 , 1 ] . In the cross moment the two windows are separated by δ + T w + T w ( w − u ) \delta+T_w+T_w(w-u) δ + T w + T w ( w − u ) , so E [ Z A ] = σ 2 2 κ T w 2 e − κ ( δ + T w ) ∬ φ ( u ) φ ( w ) e κ T w ( u − w ) d u d w \mathbb{E}[ZA] = \frac{\sigma^2}{2\kappa}T_w^2e^{-\kappa(\delta+T_w)}\iint\varphi(u)\varphi(w)e^{\kappa T_w(u-w)}\,du\,dw E [ Z A ] = 2 κ σ 2 T w 2 e − κ ( δ + T w ) ∬ φ ( u ) φ ( w ) e κ T w ( u − w ) d u d w ; the double integral is μ 2 ( 1 + v 2 κ 2 T w 2 + O ( ( κ T w ) 4 ) ) \mu^2(1+v^2\kappa^2T_w^2+O((\kappa T_w)^4)) μ 2 ( 1 + v 2 κ 2 T w 2 + O (( κ T w ) 4 )) , the odd terms vanishing by the symmetry u ↔ w u\leftrightarrow w u ↔ w , which gives σ 2 2 κ μ 2 T w 2 e − κ ( δ + a ∗ T w ) \frac{\sigma^2}{2\kappa}\mu^2T_w^2e^{-\kappa(\delta+a_*T_w)} 2 κ σ 2 μ 2 T w 2 e − κ ( δ + a ∗ T w ) . For the instrument second moment the same kernel gives E [ Z 2 ] = σ 2 2 κ T w 2 m ( κ T w ) + R Δ t ν T w ( 1 + O ( κ T w ) ) \mathbb{E}[Z^2]=\frac{\sigma^2}{2\kappa}T_w^2m(\kappa T_w)+R\Delta t\nu T_w(1+O(\kappa T_w)) E [ Z 2 ] = 2 κ σ 2 T w 2 m ( κ T w ) + R Δ t ν T w ( 1 + O ( κ T w )) , where the term R Δ t ν T w R\Delta t\nu T_w R Δ t ν T w is added by the observation noise independence across samples. The total residual combines the Itô isometry with the noise in the response, E [ ( ε t o t ) 2 ] = σ 2 ν T w + R Δ t χ q / T w + κ 2 R Δ t ν T w \mathbb{E}[(\varepsilon^{\mathrm{tot}})^2]=\sigma^2\nu T_w+R\Delta t\chi q/T_w+\kappa^2R\Delta t\nu T_w E [( ε tot ) 2 ] = σ 2 ν T w + R Δ t χ q / T w + κ 2 R Δ t ν T w . The first term is just E [ ε k 2 ] \mathbb{E}[\varepsilon_k^2] E [ ε k 2 ] . The second term is Var ( δ b k ) \operatorname{Var}(\delta b_k) Var ( δ b k ) which carries ∥ φ ′ ∥ 2 ∝ χ / T w \|\varphi'\|^2\propto\chi/T_w ∥ φ ′ ∥ 2 ∝ χ / T w and, on L ≲ 2 p L\lesssim2p L ≲ 2 p grid points, an additional factor q ( p , L ) ≥ 1 q(p,L)\ge1 q ( p , L ) ≥ 1 from resolving the bump derivative discretely. The third term is θ 2 Var ( δ A k ) \theta^2\operatorname{Var}(\delta A_k) θ 2 Var ( δ A k ) . Because θ ^ − θ = ∑ k Z k ε k t o t / ∑ k Z k A k \hat{\theta}-\theta=\sum_k Z_k\varepsilon_k^{\mathrm{tot}}/\sum_k Z_kA_k θ ^ − θ = ∑ k Z k ε k tot / ∑ k Z k A k and the summands over window pairs are one-dependent, the martingale CLT of the main theorems applies and the per-pair variance is V = E [ Z 2 ] E [ ( ε t o t ) 2 ] / E [ Z A ] 2 V=\mathbb{E}[Z^2]\mathbb{E}[(\varepsilon^{\mathrm{tot}})^2]/\mathbb{E}[ZA]^2 V = E [ Z 2 ] E [( ε tot ) 2 ] / E [ Z A ] 2 , which gives the displayed equation in the proposition. The minimizer follows by differentiating log Var ( θ ^ ) = 2 a ∗ κ T w + log ( n ( κ T w ) / μ 2 ) + log ( 1 + ρ / T w 2 ) \log\operatorname{Var}(\hat{\theta})=2a_*\kappa T_w+\log(n(\kappa T_w)/\mu^2)+\log(1+\rho/T_w^2) log Var ( θ ^ ) = 2 a ∗ κ T w + log ( n ( κ T w ) / μ 2 ) + log ( 1 + ρ / T w 2 ) up to a constant, the first two terms having initial slope ( 2 − d 1 ) κ = 2 κ a 0 (2-d_1)\kappa=2\kappa a_0 ( 2 − d 1 ) κ = 2 κ a 0 . ▫
Evaluating the exact discrete Gaussian moments numerically, with no Monte Carlo and no asymptotics, reproduces the measured standard-deviation curve of Figure 4 across the sweep except at its longest span, beyond twice the relaxation time (Figure 10 ). The constants a ∗ a_* a ∗ , a 0 a_0 a 0 and q q q are shape dependent, so the rule the main text extracts, to take L Δ t L\Delta t L Δ t a modest fraction of τ \tau τ and refine by the first-stage F F F , is calibrated empirically, not from the proposition alone. For nonlinear systems κ \kappa κ is replaced by the empirical decorrelation rate of the relevant covariances of the drift term candidates, which is what the first-stage F F F measures from data.
Figure 10. Exact window dependence of the estimator standard deviation for the OU benchmark. The curve evaluates the discrete Gaussian moment formula of Proposition 5 with no simulation, with the markers indicating Monte Carlo standard deviations of the window sweep (M = 1000 M=1000 M = 1000 ) from Figure 4 . The dashed vertical line marks the operative window T w = 20 Δ t T_w=20\Delta t T w = 20Δ t , while the dotted line marks the relaxation time 1 / κ 1/\kappa 1/ κ . The formula matches the Monte Carlo points except at the longest span, more than twice 1 / κ 1/\kappa 1/ κ , where the sample standard deviation exceeds the asymptotic value, and explains the window dependence that motivates the span rule, including the sharp loss of precision as T w T_w T w approaches 1 / κ 1/\kappa 1/ κ .
Depth
The depth n l a g s n_{\mathrm{lags}} n lags is the number of past blocks that form the instrument. Each added block lies one span further back, so its correlation with the target window carries another factor e − κ T w e^{-\kappa T_w} e − κ T w , and a second block adds little when the first one already predicts the regressors well. That predictive power is what the first stage measures. It is the regression of the regressors A A A on the instruments Z Z Z of § 7 (summarized by its F F F statistic). Two distinct failure modes call for depth two. When the first
stage is weak in the sampling sense the exactly identified system inherits weak-instrument bias toward the least-squares limit (F < 10 F<10 F < 10 here). When a residual violation of exogeneity contaminates the adjacent-block moment, for example slowly correlated measurement noise on a real detector, F F F stays large while the depth-one estimate drifts. Comparing the two depths exposes it, and the over-identified GMM weighting empirically suppresses the contaminated direction (§ 8 ). The operational form of both is the dual role of Remark 3, with the threshold F ≥ 10 F\ge10 F ≥ 10 taken from the weak-instrument literature [46 , 51 ] . Where the average F F F falls below 1 the first stage carries less signal than noise, no depth repairs it, and we regard the inference as unreliable. The tables mark such columns n.i. (not identified). Where depth two leaves F F F between 1 and 10, depths three and four can be tried, and in the stress experiments of Table 6 they raise F F F by at most a factor of 1.6, each added block lying a span further back and predicting less.
The first-stage statistic
Writing the first-stage regression A = Z Π + V A=Z\Pi+V A = Z Π + V column-wise, with Π \Pi Π the first-stage coefficient matrix and V V V its residual matrix, the reported statistic is the Cragg–Donald F = λ min ( Σ ^ V V − 1 / 2 A T P Z A Σ ^ V V − 1 / 2 ) / m F=\lambda_{\min}(\hat{\Sigma}_{VV}^{-1/2}A^\mathsf{T}P_ZA\hat{\Sigma}_{VV}^{-1/2})/m F = λ m i n ( Σ ^ V V − 1/2 A T P Z A Σ ^ V V − 1/2 ) / m , where λ min \lambda_{\min} λ m i n is the smallest eigenvalue, P Z = Z ( Z T Z ) − 1 Z T P_Z=Z(Z^\mathsf{T}Z)^{-1}Z^\mathsf{T} P Z = Z ( Z T Z ) − 1 Z T is the projection of Eq. (27) , m m m is the number of instrument columns, and Σ ^ V V \hat{\Sigma}_{VV} Σ ^ V V is the first-stage residual covariance [50 ] . It is computed on one representative test function per pair, so the effective sample units are the N w N_w N w window pairs, and deterministic columns (such as the constant term) are excluded. For a single column it reduces to the familiar first-stage F F F .
Design rule
The theorems of Appendix § 11 .§ 2 assume the window, gap, depth and diffusion response are fixed in advance, whereas the rule below picks them from the data. We do not prove that this step is harmless; the coverage study of Appendix § 14 runs the full rule and shows that it costs little. The experiments of § 7 are designed according to the following choices, whose linear-case theory is given above.
Span: Take L Δ t ∈ [ 0.1 τ , 0.5 τ ] L\Delta t \in[0.1\tau, 0.5\tau] L Δ t ∈ [ 0.1 τ , 0.5 τ ] for the characteristic time τ \tau τ , refined by maximizing the first-stage F F F ; this reproduces L = 20 L = 20 L = 20 for every system of § 7 . In practice τ \tau τ is unknown and is replaced by a pilot estimate, the empirical autocorrelation time of the record, after which the F F F -maximization runs over a small grid of L L L ; the span sweeps of the two applications of § 8 each illustrate one of them. The window length plays the role of the test-function support radius in WSINDy and WENDy, where it is chosen adaptively from the data’s spectrum [6 ] ; an analogous automated rule for the stochastic setting, in which instrument relevance rather than smoothing governs the choice, is left to future work.
Gap: Keep δ \delta δ at a few multiples of Δ t \Delta t Δ t , longer than the correlation length of the measurement noise yet far below τ \tau τ . The default δ = 4 Δ t \delta= 4\Delta t δ = 4Δ t costs a variance factor e 2 δ / τ ≈ 1 e^{2\delta/\tau} \approx1 e 2 δ / τ ≈ 1 (1.07 to 1.17 on OU, double-well, Lorenz-63, and FitzHugh–Nagumo), while correlated measurement noise calls for a longer buffer, as on the filtered detector of § 8.2 (gap 8 Δ t 8\Delta t 8Δ t ).
Depth: Use nlags = 1 \text{nlags} = 1 nlags = 1 ; raise it to two, with GMM weighting, when F < 10 F < 10 F < 10 or the two depths disagree beyond their standard errors.
Slope lags: Take j max ≤ L / 2 j_{\max} \le L/2 j m a x ≤ L /2 with j max Δ t ≲ 0.1 τ j_{\max}\Delta t \lesssim0.1\tau j m a x Δ t ≲ 0.1 τ ; the default is j max = 8 j_{\max} = 8 j m a x = 8 .
Simulation protocol and additional diagnostics
Simulation protocol
Each trajectory’s observation-noise variance R ^ \hat{R} R ^ is estimated from that trajectory alone by the multilag variogram. When several trajectories of the same system are available, R ^ \hat{R} R ^ can instead be pooled across them, which changes the reported summaries only within Monte Carlo error. In paired comparisons over 100 seeds on OU, the
coherent Kuramoto network, and Lorenz-96, a single trajectory determines R ^ \hat{R} R ^ to within 0.6% (OU) and 0.4% (the other two); the drift estimates and the multilag slope form of the diffusion are identical under the two choices, and the lag-one weak-form diffusion estimate, the one form that subtracts 2 R ^ 2\hat{R} 2 R ^ within each window, moves individual trajectories by about half its standard deviation while its mean and standard deviation change by less than 0.002.
In the synthetic examples all trajectories are simulated by Euler–Maruyama on a grid n s u b n_{\mathrm{sub}} n sub times finer than the observation step and then subsampled (n s u b = 5 n_{\mathrm{sub}} = 5 n sub = 5 for OU, the double-well, Lorenz-63, FitzHugh–Nagumo, and the Kuramoto network; n s u b = 10 n_{\mathrm{sub}} = 10 n sub = 10 for Cox–Ingersoll–Ross, the underdamped Langevin, the observed Lorenz-63, and Lorenz-96), with M = 1000 M = 1000 M = 1000 independent trajectories per model, run in parallel. Measurement noise η η ∼ N ( 0 , R ) \eta_\eta\sim\mathcal{N}(0,R) η η ∼ N ( 0 , R ) is then added to the simulated states. Reported means and standard deviations are computed across the 1000 per-trajectory estimates.
Debiasing of the term candidates
With Y = X + η Y = X + \eta Y = X + η and η ∼ N ( 0 , R ) \eta\sim\mathcal{N}(0,R) η ∼ N ( 0 , R ) independent of the state, the odd moments of η \eta η vanish, so expanding the powers gives E [ Y 2 ∣ X ] = X 2 + R \mathbb{E}[Y^2 \mid X] = X^2 + R E [ Y 2 ∣ X ] = X 2 + R , E [ Y 3 ∣ X ] = X 3 + 3 X R \mathbb{E}[Y^3 \mid X] = X^3 + 3XR E [ Y 3 ∣ X ] = X 3 + 3 X R , and in general only the even moments of the noise survive, contributing powers of R R R . For drift term candidates x 2 x^2 x 2 and x 3 x^3 x 3 we therefore apply the corrections of § 6 , x 2 ^ ↦ Y 2 ^ − R ^ \widehat{x^2} \mapsto\widehat{Y^2} - \hat{R} x 2 ↦ Y 2 − R ^ (and x i x j ^ ↦ Y i Y j ^ − R ^ i j \widehat{x_i x_j} \mapsto\widehat{Y_iY_j} - \widehat{R}_{ij} x i x j ↦ Y i Y j − R ij for cross terms, which is why the multilag construction of § 4 returns the full matrix) and x 3 ^ ↦ Y 3 ^ − 3 R ^ Y ^ \widehat{x^3} \mapsto\widehat{Y^3} - 3\widehat{R}\widehat{Y} x 3 ↦ Y 3 − 3 R Y . Without these corrections the higher-order columns carry an O ( R ) O(R) O ( R ) bias that contaminates the drift estimate independently of the endogeneity correction. The same principle ex-
tends to any term candidate g g g . Evaluating g g g at noisy arguments blurs it, in that the expectation over the noise, E [ g ( Y ) ∣ X ] \mathbb{E}[g(Y)\mid X] E [ g ( Y ) ∣ X ] , is the convolution of g g g with the density of η \eta η , so g g g is smoothed in its state argument on the scale R ^ 1 / 2 \hat{R}^{1/2} R ^ 1/2 and the observed values g ( Y n ) g(Y_n) g ( Y n ) scatter around this blurred function instead of around g g g . This smoothing is distinct from the smoothing performed by the test functions, which averages over time and suppresses the fluctuations of the noise, whereas the blur is a mean shift, proportional to R R R and to the curvature of g g g and identical at every sample, which does not vanish under the temporal averaging of the weak form. The debiased entry g ~ \tilde{g} g ~ is the deconvolution, the function whose blur is g g g . Since E [ cos ( ω Y ) ∣ X ] = e − 1 2 ω 2 R cos ( ω X ) \mathbb{E}[\cos(\omega Y)\mid X]=e^{-\frac{1}{2}\omega^2R}\cos(\omega X) E [ cos ( ω Y ) ∣ X ] = e − 2 1 ω 2 R cos ( ω X ) and E [ sin ( ω Y ) ∣ X ] = e − 1 2 ω 2 R sin ( ω X ) \mathbb{E}[\sin(\omega Y)\mid X]=e^{-\frac{1}{2}\omega^2R}\sin(\omega X) E [ sin ( ω Y ) ∣ X ] = e − 2 1 ω 2 R sin ( ω X ) , trigonometric terms are eigenfunctions of the blur and are debiased by the multiplicative factor e 1 2 ω 2 R ^ e^{\frac{1}{2}\omega^2\hat{R}} e 2 1 ω 2 R ^ . More generally the debiased entry is g ~ ( Y ) = ( e − 1 2 R ^ ∂ 2 g ) ( Y ) \tilde{g}(Y)=(e^{-\frac{1}{2}\hat{R}\partial^2}g)(Y) g ~ ( Y ) = ( e − 2 1 R ^ ∂ 2 g ) ( Y ) , the inverse heat semigroup applied to g g g , computed via the Taylor expansion ∑ k ( − 1 2 R ^ ) k ∂ 2 k g / k ! \sum_k(-\frac{1}{2}\hat{R})^k\partial^{2k}g/k! ∑ k ( − 2 1 R ^ ) k ∂ 2 k g / k ! , which truncates exactly for polynomials and recovers the polynomial rules above. For term candidates that are neither polynomial nor trigonometric, rational or saturating drifts for instance, the deconvolution is ill posed, since the blur damps high frequencies by the factor e − 1 2 ω 2 R e^{-\frac{1}{2}\omega^2R} e − 2 1 ω 2 R that its inverse must amplify; the correction is then applied numerically by Gauss–Hermite quadrature or a low-order Taylor expansion truncated at the variance scale R ^ \hat{R} R ^ , which regularizes the inverse by discarding the amplified high-frequency content.
IV versus GMM
At the default depth n l a g s n_{\mathrm{lags}} n lags equals one, the moment system is exactly identified and GMM coincides with IV. When the depth rule selects two blocks, the reported estimates are two-step GMM. On systems with strong first stages the two variants differ by far less than one Monte Carlo standard deviation.
Ensemble size and computational cost
In our synthetic examples, the ensemble size M = 1000 M=1000 M = 1000 is a reporting choice, not a requirement. Setting M = 200 M=200 M = 200 , for example, reproduces every conclusion of the main text, with operative-window table entries agreeing to a few percent. An ensemble as small as M = 5 M=5 M = 5 already displays the estimator’s accuracy (as in the optical trap example). Once the weak-form integrals are formed, one drift recovery is a single linear solve. It takes tens of milliseconds per trajectory on one CPU core in our reference implementation; a measured comparison with the Kalman maximum-likelihood baseline appears in Appendix § 15 .
Off-diagonal noise entries
The scalar series S t n ± = Y t n i ± Y t n j S^{\pm}_{tn}=Y^i_{tn}\pm Y^j_{tn} S t n ± = Y t n i ± Y t n j carries noise with variance R i i + R j j ± 2 R i j R_{ii}+R_{jj}\pm2R_{ij} R ii + R j j ± 2 R ij , so fitting its scalar variogram ν k ± = 1 N − k ∑ n ( S t n + k ± − S t n ± ) 2 \nu_k^\pm=\frac{1}{N-k}\sum_n(S^\pm_{t_{n+k}}-S^\pm_{t_n})^2 ν k ± = N − k 1 ∑ n ( S t n + k ± − S t n ± ) 2 and halving the intercept yields R ^ ± \hat{R}_{\pm} R ^ ± estimating R i i + R j j ± 2 R i j R_{ii}+R_{jj}\pm2R_{ij} R ii + R j j ± 2 R ij , whence
R ^ i j = R ^ + − R ^ i i − R ^ j j 2 = R ^ i i + R ^ j j − R ^ − 2 = R ^ + − R ^ − 4 . (34) \hat{R}_{ij}=\frac{\hat{R}_{+}-\hat{R}_{ii}-\hat{R}_{jj}}{2}=\frac{\hat{R}_{ii}+\hat{R}_{jj}-\hat{R}_{-}}{2}=\frac{\hat{R}_{+}-\hat{R}_{-}}{4}. \tag*{(34)} R ^ ij = 2 R ^ + − R ^ ii − R ^ j j = 2 R ^ ii + R ^ j j − R ^ − = 4 R ^ + − R ^ − . (34)
The symmetric form is preferred for two reasons. It needs neither diagonal entry, and the fits behind R ^ + \hat{R}_{+} R ^ + and R ^ − \hat{R}_{-} R ^ − share the same drift and diffusion nuisance functionals, so their errors are positively correlated and partially cancel in the difference. Specializing the diagonal fit to n ν = 2 n_\nu=2 n ν = 2 (n ν n_\nu n ν counts the variogram lags of § 4 ) recovers the classical lag-1/lag-2 form R ^ i i = 1 2 ( 2 ν 1 − ν 2 ) \hat{R}_{ii}=\frac{1}{2}(2\nu_1-\nu_2) R ^ ii = 2 1 ( 2 ν 1 − ν 2 ) , which returns zero, or a clipped negative value, as soon as the drift bias dominates.
Derivation and defaults of the multilag slope estimator
By Eq. (19) , E [ S k , i ( j ) ] \mathbb{E}[S_{k,i}(j)] E [ S k , i ( j )] is quadratic in the lag j j j , and σ i 2 Δ t \sigma_i^2\Delta t σ i 2 Δ t is its linear coefficient. Extracting that coefficient is a fixed linear operation on the S k , i ( j ) S_{k,i}(j) S k , i ( j ) , whose weights ( w j ) (w_j) ( w j ) in Eq. (21) form the linear-coefficient row of the least-squares ( 1 , j , j 2 ) (1,j,j^2) ( 1 , j , j 2 ) design over j = 2 , … , j max j=2,\ldots,j_{\max} j = 2 , … , j m a x . The fit starts at j = 2 j=2 j = 2 , so the most distorted lag never enters. We use j max = 8 j_{\max}=8 j m a x = 8 , which requires L > j max L>j_{\max} L > j m a x ; the choice is examined with the other design choices in the main text and in § 4.6 .
Sensitivity to the number of variogram lags
On the OU benchmark the estimates are stable across variogram lag counts n ν ∈ { 3 , … , 10 } n_\nu\in\{3,\ldots,10\} n ν ∈ { 3 , … , 10 } . The recovered R ^ \hat{R} R ^ stays within 0.1 % 0.1\% 0.1% of the truth at every n ν n_\nu n ν , with spread shrinking from 2.3 × 10 − 4 2.3\times10^{-4} 2.3 × 1 0 − 4 at n ν = 3 n_\nu=3 n ν = 3 to 1.0 × 10 − 4 1.0\times10^{-4} 1.0 × 1 0 − 4 at n ν = 10 a g a i n s t n_\nu=10 against n ν = 10 a g ain s t R=0.02$, and σ ^ \hat{\sigma} σ ^ stays within 0.3 % o f 0.3\% of 0.3% o f 0.8$ throughout (M = 400 M=400 M = 400 ). The default n ν = 6 n_\nu=6 n ν = 6 lies in the flat region.
Stress noise level
Tables 6 and 7 report every system at the stress noise level of Table 5 , a fixed variance per component equal to the square of 15% of the component’s range over a record of the length used in the experiments. Each trajectory estimates its own R ^ \hat{R} R ^ , which recovers the stress variances to within 0.03% for every component of OU, the double-well, Lorenz-63 and FitzHugh–Nagumo. At this level the FitzHugh–Nagumo and Lorenz-63 first stages collapse at depth one (average F < 1 F<1 F < 1 ), and the depth rule selects two instrument blocks at the unchanged window and gap, whereas the OU and double-well first stages remain strong (F = 3005 F = 3005 F = 3005 and 66) and depth one is the reported estimator; the tables show depths one to four for every system. Table 7 lists the stress-level ratio ϱ s \varrho_s ϱ s next to the diffusion estimates; where it exceeds 10 4 10^4 1 0 4 (the chaotic systems) the increments carry no information on σ \sigma σ and the cells read n.i.
The F F F rows of Table 6 also separate what the first stage does and does not control. Where essentially every replication falls below the threshold, the estimates acquire the predicted attenuation toward the least-squares limit, with the FitzHugh–Nagumo current coming in 24% low at depth two, about ten Monte Carlo standard errors, and the per-trajectory distributions develop heavy tails, as in the Cox–Ingersoll–Ross column at L = 100 L = 100 L = 100 (F = 2.4 F = 2.4 F = 2.4 , with 97% of replications below 10). In the columns marked n.i. the failure is complete, with point estimates orders of magnitude from the truth (b ^ \hat b b ^ in the thousands against b = 28 b = 28 b = 28 ), as the ratio mechanism of Appendix § 14 predicts. Where no replication falls below the threshold, the estimates are centered with reliable standard errors, but a large F F F does not promise precision. The Cox–Ingersoll–Ross spreads at the operative window (F = 108 F = 108 F = 108 , none below 10) remain wide because at this noise level (R s \sqrt{R_s} R s above the component’s stationary SD, Table 5 ) the record itself carries little information, and the L = 5 L = 5 L = 5 column pairs a larger F F F (287) with a wider spread still. The threshold F ≥ 10 F \ge10 F ≥ 10 marks the regime in which per-trajectory inference is valid, with Wald coverage of 93–95% at F ≈ 3 F \approx3 F ≈ 3 and 90% at F ≈ 1 F \approx1 F ≈ 1 (Appendix § 14 ); precision is read from the standard errors it validates.
Standard errors, interval coverage, and Normality
Plug-in standard errors
At depth one the estimator Eq. (6) is exactly identified, and its error decomposes over window pairs, θ ^ − θ = G − 1 ∑ k = 1 K g k \hat{\theta} - \theta= G^{-1} \sum_{k=1}^{K} g_k θ ^ − θ = G − 1 ∑ k = 1 K g k , where G = ∑ k Z k T A k G = \sum_k Z_k^{\mathsf T} A_k G = ∑ k Z k T A k collects the instrument–regressor products of pair k k k and g k = Z k T ε k g_k = Z_k^{\mathsf T} \varepsilon_k g k = Z k T ε k is that pair’s score. By Theorem 2 the population scores form a martingale-difference sequence, so they are uncorrelated across pairs whatever the system’s memory. In finite samples, correlation can enter through the quadrature remainder ρ k \rho_k ρ k , through evaluating the scores at θ ^ \hat{\theta} θ ^ , and, on real data, through correlated measurement noise. The plug-in covariance therefore includes a long-run (Newey–West) term as a safeguard; on the benchmarks it changes the standard errors little (on the AMO operative window, 0.284 with it and 0.289 without),
Var ^ ( θ ^ ) = G − 1 [ ∑ k g ^ k g ^ k T + ∑ ℓ = 1 q ( 1 − ℓ q + 1 ) ∑ k ( g ^ k g ^ k + ℓ T + g ^ k + ℓ g ^ k T ) ] G − T . \widehat{\operatorname{Var}}(\hat{\theta}) = G^{-1}\left[\sum_k \hat{g}_k\hat{g}_k^{\mathsf T} + \sum_{\ell=1}^{q}\left(1-\frac{\ell}{q+1}\right)\sum_k\left(\hat{g}_k\hat{g}_{k+\ell}^{\mathsf T}+\hat{g}_{k+\ell}\hat{g}_k^{\mathsf T}\right)\right]G^{-\mathsf T}. Var ( θ ^ ) = G − 1 [ k ∑ g ^ k g ^ k T + ℓ = 1 ∑ q ( 1 − q + 1 ℓ ) k ∑ ( g ^ k g ^ k + ℓ T + g ^ k + ℓ g ^ k T ) ] G − T .
with residual scores g ^ k \hat{g}_k g ^ k , Bartlett weights [66 ] , and the automatic bandwidth q = ⌊ 4 ( K / 100 ) 2 / 9 ⌋ q = \lfloor4(K/100)^{2/9}\rfloor q = ⌊ 4 ( K /100 ) 2/9 ⌋ [73 ] (e.g. q = 9 q = 9 q = 9 for OU at T = 2000 T = 2000 T = 2000 ). Each equation is treated separately; derived quantities use the delta method, and ratios use Fieller intervals as in the main text. The computation needs only the per-pair moments already formed by the estimator.
Design
For OU (M = 500 M = 500 M = 500 ) and the double-well (M = 200 M = 200 M = 200 ) each replication is simulated once at T = 2000 T = 2000 T = 2000 , and its first T ∈ { 125 , 250 , 500 , 1000 , 2000 } T \in\{125, 250, 500, 1000, 2000\} T ∈ { 125 , 250 , 500 , 1000 , 2000 } time units give nested records of increasing length; estimates are independent across replications at every T T T . All runs use L = 20 L = 20 L = 20 , the default gap δ = 4 Δ t \delta= 4\Delta t δ = 4Δ t , depth one, and each record’s own R ^ \hat{R} R ^ . For Lorenz-63 and FitzHugh–Nagumo the check is made at the operative record length (M = 1000 M = 1000 M = 1000 ; T = 400 T = 400 T = 400 for Lorenz-63 and T = 5000 T = 5000 T = 5000 for FitzHugh–Nagumo, whose fast-equation and slow-equation model coefficients are assessed directly, as these, and not the derived ratios a , b a,b a , b , are the asymptotically Normal quantities). As a comparison, pair-bootstrap intervals (independent resampling of window pairs) and moving-block intervals (blocks of q q q pairs [74 ] ) are computed with B = 200 B = 200 B = 200 resamples on a subset of replications.
Results
Figure 11 shows that the mean plug-in standard error tracks the sampling standard deviation across replications at every record length from T = 250 T = 250 T = 250 to 2000, and that both decay at the T − 1 / 2 T^{-1/2} T − 1/2 rate. Wald coverage of nominal 95% intervals lies within or near the Monte Carlo band at every length. Table 8 summarizes the four systems at their operative lengths, where the ratio of mean plug-in standard error to sampling standard deviation lies between 0.95 and 1.02 for every parameter and Wald coverage between 93.2% and 96.1%, within or near the Monte Carlo band in every case. For the 23 drift term candidates of Lorenz-63 whose true value is zero, the Wald interval contains zero in 94.7% of replications on average, so a Wald test for including a term is also calibrated. The bootstrap intervals agree with the Wald intervals within Monte Carlo error throughout and do not improve on them; the moving-block bootstrap undercovers slightly at the longest records (91% for the OU rate at T = 1000 T = 1000 T = 1000 and 2000).
Figure 11. Plug-in (Newey–West) standard errors and Wald-interval coverage as the record length T T T grows, at L = 20 L = 20 L = 20 : OU in the left pair of panels (M = 500 M = 500 M = 500 ), the double-well in the right pair (M = 200 M = 200 M = 200 ). In the first panel of each pair, the sampling standard deviation across replications (solid) is compared with the mean plug-in standard error (dashed) and with the T − 1 / 2 T^{-1/2} T − 1/2 reference rate (dotted). For the double-well at T = 125 T = 125 T = 125 , where a few extreme records dominate the sample standard deviation, we use the robust spread (IQR/1.349) instead of the sample standard deviation, and indicate that by the open diamond. In the second panel of each pair, the coverage of nominal 95% Wald intervals (lines) is compared with that of the pair bootstrap (upward open triangles) and the moving-block bootstrap (downward open triangles). The shaded band is the binomial band 0.95 ± 1.96 0.95 ⋅ 0.05 / M 0.95 \pm1.96\sqrt{0.95\cdot0.05/M} 0.95 ± 1.96 0.95 ⋅ 0.05/ M for an empirical coverage at the nominal level of 95%, computed using the full M M M . For OU, where the bootstrap was run on 130 of the 500 replications, the triangles carry about twice the Monte Carlo error of the band. The plug-in standard error closely tracks the sampling variability at the T − 1 / 2 T^{-1/2} T − 1/2 rate, while the Wald intervals hold near-nominal coverage at every record length T T T , as corroborated by the pair bootstrap.
Table 8. Calibration of the plug-in (Newey–West) standard errors and of the Wald intervals at each benchmark’s operative record length T T T (second column, with the number of replications M M M ). The SE ratio is the mean plug-in standard error divided by the sampling standard deviation of the estimates across replications, so a value of one indicates a calibrated standard error. When a range is provided, that range is the range of the SE ratios for the listed parameters. Wald coverage is the fraction of replications whose nominal 95% interval contains the truth. (For calibration, note that a binomial Monte Carlo standard error at M = 1000 M = 1000 M = 1000 for p = 0.95 p = 0.95 p = 0.95 equals 0.7 percentage points.) Across all four systems the plug-in standard errors match the sampling variability to within a few percent, while the Wald intervals attain near-nominal coverage.
At T = 125 T = 125 T = 125 the double-well’s sampling standard deviation is dominated by a few of the 200 records (errors of 22, 5, and 4 in α ^ \hat{\alpha} α ^ ) whose trajectories visit both wells only a few times. These records report correspondingly large plug-in standard errors, so coverage is only mildly affected (92% for α \alpha α and 90% for β \beta β , with a Monte Carlo standard error of 1.5 points); a very large standard error on a short record is a signal of weak identification and should be read as such.
Probability limits, not finite-sample means
The analysis rests on probability limits and limiting distributions rather than on finite-sample means for a reason specific to instrumental variables. In the classical linear IV model, with fixed instruments and Gaussian errors, the number of finite moments of the IV estimator grows with the degree of over-identification [75 ] , and the exactly identified estimator is heavy-tailed, behaving as a ratio whose denominator has positive density at zero. That model does not describe WAIVES in full, since its instruments are random and its errors are stochastic integrals, but the same ratio mechanism is present in the depth-one estimator, which is exactly identified. With a strong instrument the tail mass is negligible, and the sample means and standard deviations reported in the tables are stable. In the deterministic output-error setting, Kuang and Lin [41 ] confront the same pathology by regularizing the estimator, through singular-value clipping and instrument truncation. We instead leave the estimator unregularized, report probability limits, and rely on over-identification with two-step GMM, which lightens the tails at depths beyond one and has standard asymptotic guarantees. As the instrument weakens they become erratic, as in the Lorenz-63 entries at L = 100 L = 100 L = 100 in Table 2 (first-stage F < 1 F < 1 F < 1 ), and medians and quantiles become the more reliable summaries [46 ] . Escalating to depth two over-identifies the system, which in the classical model lightens the tails. The ratio parameters of FitzHugh–Nagumo are summarized by Marsaglia modes and Fieller intervals for the same reason.
to Normal, with skewness and excess kurtosis within about ± 0.5 \pm0.5 ± 0.5 . The exception is the FitzHugh–Nagumo fast equation, whose model coefficients show mild right skew and excess kurtosis near 1, and whose ν \nu ν coefficient is biased upward by 2% (about three and a half Monte Carlo standard errors), consistent with its modest depth-one first stage (F ≈ 16 F \approx16 F ≈ 16 ); its Wald coverage is nevertheless nominal.
Normality
Figure 12 shows the sampling distribution of a representative parameter for each of the nine systems. All are close
Figure 12. Sampling distributions of single-trajectory estimates for each of the nine systems: histogram of the M M M estimates of a representative parameter, fitted Normal density (black), true value (red dashed), and Normal QQ plot (inset), with skewness and excess kurtosis. The Langevin panel shows the raw IV estimate, the FitzHugh–Nagumo panel shows the fast-equation coefficient with the heaviest tails, and the CIR panel shows the slope-form diffusion estimate. Although Theorem 2 is an asymptotic result, the sampling distributions are already close to Gaussian at these finite record lengths, with the printed skewness and excess kurtosis indicating minor remaining departures. This approximate normality is what justifies the Wald intervals of Table 8 .
Recommendation
For single-trajectory inference we recommend Wald intervals from the plug-in covariance above. They follow directly from the asymptotic normality of Section A.2, cost a single pass over the per-pair moments, and are calibrated on linear, bistable, slow–fast, and chaotic dynamics; the bootstrap adds computation without improving coverage. The condition is instrument strength, not record length. On synthetic records matched to the AMO application (§ 8.1 ; M = 1000 M = 1000 M = 1000 ), Wald coverage stays between 93% and 95% at spans leaving as few as 41 window pairs (median first-stage F ≈ 3 F \approx3 F ≈ 3 ), and falls to 90% only at F ≈ 1 F \approx1 F ≈ 1 , where the instrument identifies essentially nothing; the bootstrap over-covers there (97%). The strength rule F ≥ 10 F \ge10 F ≥ 10 of the main text therefore also secures the Wald intervals.
Comparison with likelihood baselines
Two baselines locate WAIVES against likelihood methods where those are feasible. For the noisy OU benchmark the exact likelihood is available through the Kalman filter, since the sampled process is a linear Gaussian state space model [52 ] . Maximizing it per trajectory gives the efficient benchmark, and the Cramér–Rao bound follows from the Fisher information of the same likelihood. Table 9 compares, over the same M = 1000 M = 1000 M = 1000 trajectories, WAIVES, the Kalman maximum-likelihood estimator, and the Cramér–Rao bound. The Kalman MLE standard deviation is the same as the Cramér–Rao bound. WAIVES, on the other hand, incurs a factor of about 3.1 in standard deviation on the rate κ \kappa κ at the operative window, the efficiency price of the likelihood-free route, while remaining as unbiased as the MLE. The computational comparison, by contrast, favors WAIVES, and Table 9 reports it alongside the statistical one. Both timings cover the same three parameters κ \kappa κ , μ \mu μ , and σ \sigma σ . For WAIVES these come from the drift system with its IV solve, from the multilag slope estimator of the diffusion, and the first-stage test F F F . It takes 13 ms per trajectory (the median). One evaluation of the Kalman likelihood takes 1.6 ms. The evaluation runs the steady-state filter, with the gain from iterating the scalar Riccati recursion to its fixed point. All N N N innovations then come from one call to SciPy’s compiled recursive filter, scipy.signal.lfilter. The maximum-likelihood fit runs Nelder–Mead (MLE-NM) over ( log κ , μ , log σ ) (\log\kappa, \mu, \log\sigma) ( log κ , μ , log σ ) , starting from κ = 1.2 \kappa= 1.2 κ = 1.2 , σ = 0.7 \sigma= 0.7 σ = 0.7 , and the sample mean of the record for μ \mu μ . Its tolerances are 10 − 6 10^{-6} 1 0 − 6 on the parameters and 10 − 8 10^{-8} 1 0 − 8 on the negative log-likelihood. The fit needs 180 evaluations, or 0.28 s, about 20 times the cost of WAIVES. A quasi-Newton search (in particular L-BFGS-B with finite-difference gradients) needs 52 evaluations, or 0.08 s, still six times the cost of WAIVES. It uses SciPy’s default tolerances, 2.2 × 10 − 9 2.2 \times10^{-9} 2.2 × 1 0 − 9 relative on the objective and 10 − 5 10^{-5} 1 0 − 5 on the projected gradient. Timings are medians over the M = 1000 M = 1000 M = 1000 trajectories on one core of an Intel Xeon Platinum 8592+ (a Blanca node of the University of Colorado Research Computing cluster), with Python 3.8.5, NumPy 1.24.3, SciPy 1.10.1, and single-threaded BLAS.
Table 9. Likelihood baselines for OU (true κ = 1.5 \kappa= 1.5 κ = 1.5 ) and the double-well (true α = β = 1 \alpha= \beta= 1 α = β = 1 ), on the same M = 1000 M = 1000 M = 1000 trajectories as Table 2 , with entries corresponding to the estimator average (std). For OU, the Kalman filter gives the exact likelihood: MLE-NM and MLE L-BFGS-B are its maximizers over ( κ , μ , σ ) (\kappa, \mu, \sigma) ( κ , μ , σ ) under two optimizers, and CRB is the Cramér–Rao bound of that likelihood, noted as a standard deviation. For the double-well, there is no exact likelihood available, and the comparison is done using least-squares optimization on the strong form of the equation. The last column is the median wall-clock time per trajectory on one core, covering drift, slope diffusion, R ^ \widehat{R} R , and first-stage F F F for WAIVES, the MLE-NM and L-BFGS-B fits of ( κ , μ , σ ) (\kappa, \mu, \sigma) ( κ , μ , σ ) for the Kalman MLE, and the least-squares solve for the strong form. While WAIVES is roughly ten times faster, it does pay a price of larger standard errors (up to a factor of three) compared to its exact-likelihood counterparts, when they exist. Where no exact likelihood exists, as is the case with the double-well example, the comparably fast alternative is the strong-form least squares. At nearly the same cost per trajectory, the strong form is biased by an order of magnitude (α ^ ≈ 11 \widehat{\alpha} \approx11 α ≈ 11 against α = 1 \alpha= 1 α = 1 ) while WAIVES remains unbiased.
For a nonlinear case with no exact likelihood we use the double-well benchmark and compare WAIVES against a strong-form least-squares baseline in the style of [1 , 25 ] , regressing Euler increments Δ Y n / Δ t \Delta Y_n / \Delta t Δ Y n /Δ t on the same drift term candidates evaluated at Y n Y_n Y n . Measurement noise enters that regression on both sides. The O ( R / Δ t ) O(R/\Delta t) O ( R /Δ t ) errors-in-variables terms of the differentiated noisy record then corrupt the recovered coefficients by an order of magnitude at the benchmark noise level. This is an endogeneity of the strong form, mirroring the weak-form endogeneity of § 3 . Smoothing mitigates but does not remove the effect, which is the documented noise sensitivity that weak-form methods were built to avoid. WAIVES removes both endogeneities at once (see Table 9 second block).