betaFeedback

WorksMathematics

Stochastic Kernel Mode Decomposition: Information, Uncertainty, and Conditional Regime Inference

Abstract

When do mode energies preserve regime information as measurement quality changes? Stochastic Kernel Mode Decomposition (sKMD) fits joint regime laws on signed KMD mode coordinates and transfers the reference mode posterior to regime probabilities and conditional reconstructions. For centered Gaussian regimes with independently varying known noise variances, fixed coordinate energies retain the full regime posterior precisely when all classes share their cross-covariances and class variances can differ only at isolated vertices of the covariance graph. We extend the characterization to nonzero means and show that freely varying correlated noise forces a fixed linear log-odds decoder to retain the full quadratic feature span even for two distinct centered regimes. At fixed quality, a three-regime example has no sufficient two-energy basis. Its smallest Bayes log-loss gap over bases of condition number at most κκ has sharp order κ−1κ^{-1}. Exact Bayes energy posteriors supply numerical gap estimates for specified bases. Additional bounds relate feature compression, independently learned mode spans and sensor-noise error to log loss. Two controlled experiments with 64 independent panels each show that signed information can improve prediction over energy summaries, while integration gives small gains over strong signed controls and depends on how the competing point rule is calibrated and how accurately observation noise is measured.

Introduction

Kernel Mode Decomposition (KMD) separates signals into components described by kernels and relates this recovery problem to Gaussian conditioning, optimal recovery and adaptive estimation of amplitude, phase and waveform [81]. Many uses of a decomposition require a second inference task. A vibration record may indicate a forcing regime; an oscillatory waveform may need reconstruction despite missing samples; and a change in observation quality may alter how much a small recovered mode should count as evidence. These tasks depend on the joint law of the modes, not only on their separate reconstruction energies.

Kernel representations also connect signal analysis with stochastic transitions. Maximum mean discrepancy detects changes in distributions in fast–slow systems [48] and has been studied for EEG seizure detection [47]. Transition-manifold embeddings reduce metastable dynamics [8], and kernel approximations of quasipotentials describe rare transitions in the weak-noise setting [72]. Microlocal, time–frequency–phase kernels give a direct KMD precedent by reconstructing signals around critical transitions in slow–fast models and EEG-related seizure examples [51]. This motivates retaining the observation posterior as well as the reconstructed modes. Related discrepancy constructions compare laws on countable spaces and prefix trees [39, 57] or drive artificial flows for matching prescribed measures [44]. The inferential task here is to assign probabilities to regimes of a measured process.

This paper develops stochastic KMD (sKMD) as a regime-inference extension of KMD. The extension fits joint class laws on signed mode coordinates, retains the KMD reference posterior mean and covariance, and uses a change of prior to obtain regime probabilities and regime-conditioned reconstructions. A regime can describe a class, an excitation law or a state in a specified temporal model. The same interface handles a known observation mask and noise covariance. Every method labeled sKMD in this paper is this construction: fitted regime laws on KMD mode coordinates, combined with the reference posterior (m,V)(m,V) through Proposition 2.2.

Building on KMD’s signed modes and probabilistic interpretation, sKMD introduces a fitted joint regime law. Comparing sKMD with separate mode energies tests information discarded by a particular summary of KMD output. Comparing its reconstructions with independent-mode or pooled full-covariance KMD tests the value of the regime law. Comparing it with a quality-aware signed classifier tests whether explicit integration adds predictive value beyond retaining the relevant information. These are different estimands and receive separate controls.

The research question and contributions

The central question is whether one representation of the modes remains adequate as observation quality changes. A coordinate system that makes energies sufficient at one noise level may fail at another. Conversely, changing posterior coordinates can sometimes restore sufficiency. This paper studies that distinction and uses it to specify the information that an sKMD implementation should preserve.

  1. Representation characterizations under changing quality (§ 2.12). Theorem 4.1 characterizes, through the covariance graph, when fixed coordinate energies remain sufficient for every diagonal noise covariance in an open set. Corollary 4.3 extends the characterization to nonzero means and Corollary 4.2 to correlated noise. Theorem 4.4 shows that correlated noise variation forces the complete quadratic span for any fixed linear-in-features decoder, even for two fixed class covariances. Proposition 4.5 and the stable approximation Proposition 4.6 bound the error of a restricted quadratic feature budget under the actual observed mixture, and Theorem 4.8 gives the local size of quality-induced energy loss.

  1. Fixed-quality geometry and a three-regime obstruction (§ 2.7). Theorem 3.1 gives the fixed-quality energy-sufficiency criterion. Theorem 3.2 and Corollary 3.3 show that a specified three-regime law has no sufficient nonredundant two-energy basis. Theorems 3.4 and 3.5 give positive information-loss bounds over orthogonal and conditioning-controlled bases. Corollary 3.6 establishes the sharp order of the best gap under a conditioning limit. § 6.6 estimates gap sizes using exact Bayes energy posteriors.

  1. Learning bounds (§ 2.20). Theorem 5.1 gives a local information expansion with a computable remainder. Lemma 5.2 and Theorem 5.3 bound learned-span leakage for an independently selected finite dictionary library, and Theorem 5.4 gives an explicit condition under which a fitted interaction rule beats an energy oracle. Proposition 5.5 bounds posterior log loss under covariance perturbation.

  1. A mode-posterior interface (§ 2.1). Definition 2.1 and Proposition 2.2 express regime inference and regime-conditioned reconstruction through the KMD posterior mean

and covariance, with an exact reduction to the reference KMD model. The Gaussian transfer formula is standard Gaussian conditioning expressed through the KMD posterior; matched latent-Gaussian implementations give the same output and serve as implementation checks.

  1. Controlled tests and applications (Part II). Two controlled experiments with 64 independent panels each separate designed binary calibration from finite-sample recovery of partial energy information in a three-regime family; a retrospective calibration analysis compares binary inference with quality-dependent point calibration. Studies with learned carriers, changing dynamics, ECG and bearings evaluate the same mechanisms on learned and recorded data. In those studies integration helped mainly under declared synthetic degradations; at native quality or under operating-condition shift, simpler controls were as good or better.

Relation to existing methods

Joint Gaussian class laws, QDA, common-factor discriminant models and heterogeneous-noise likelihoods are established tools [3, 7, 15]. Using the same dictionary, class parameters and observation model in extreme deconvolution or a latent-Gaussian classifier gives the same probabilities as the Gaussian sKMD calculation. The posterior transfer is therefore not a new probability calculation. The new content is the characterization of which mode features retain the regime posterior as the observation quality varies (§ 2.7 and § 2.12) and the bounds on learning them from finite data (§ 2.20).

At fixed quality, the energy-sufficiency criterion follows from the polynomial form of Gaussian log odds. Common principal components, common-orientation discriminant models and simultaneous congruence provide its coordinate geometry [5, 27, 65]. Conditional mutual information and proper-score deficiencies measure what a summary discards [6, 33]. Linear and quadratic sufficient features for Gaussian laws are classical, including their use in heteroscedastic Gaussian-process models [68]. Here one fixed class family is held constant while the observation covariance varies over an open set. This uniform constraint produces the covariance-graph characterization and the quadratic-span result. The approximation result then uses the actual quality-indexed class mixture to measure the cost of a limited feature budget.

Kernel models for dynamical systems target several different quantities. Balanced reduction uses controllability and observability to choose reduced coordinates for nonlinear systems. Early conference and workshop contributions preceded the journal treatment [10, 12, 13]. Related estimators recover these quantities and invariant measures from stochastically forced systems [11, 14]. Kernel approximations can estimate discrete-time update rules for linear autonomous and controlled systems [37, 38]. Center-manifold approximation and learned Gaussian-process transformations instead seek coordinates that simplify local dynamics [34, 35, 71]. A modal representation in sKMD serves another purpose. It must retain enough information from a noisy record to distinguish its candidate regimes.

That inference target also differs from operator estimation. Kernel spectral methods approximate the Koopman generator and the Schrödinger operator [66]. Approximating principal Koopman eigenfunctions yields coordinates for the evolution of observables [70]. Transport and Feynman–Kac representations yield kernels adapted to these eigenfunction problems in deterministic and stochastic systems, respectively [45, 60, 61]. Algorithmic kernels give another geometry for Koopman and transfer operators [55]. Stability analysis estimates Lyapunov functions directly from noisy observations [28] or constructs them from approximate Koopman eigenfunctions [29, 73]. Kernel methods can also enforce Hamiltonian structure in trajectory learning [63] or approximate the value function in nonlinear optimal control [46]. A distinct use of dynamical models treats the training process itself as a system, studying limits of trained weights or the policies needed to steer training [56, 87]. Here the criterion is preservation of the posterior over regime labels.

Organization

The paper has two parts. Part I (Theory) defines the method in § 2.1, which also collects the notation (Table 1). § 2.7 treats the information retained by mode coordinates and energies at fixed quality, § 2.12 the effect of changing observation quality, and § 2.20 learning, approximation and adaptive coordinates, ending with numerical checks of the bounds.

Part II (Applications) begins with the experimental design, the evidence overview and the statement of which analyses are confirmatory (§ 3.1). § 3.3 tests the mechanisms on known modal observations. § 3.7 adds learned modes and nonstationary signals, § 4.3 recorded ECG and bearing signals, and § 6.5 the exact energy-oracle and finite-sample coupled-mode benchmarks. § 7.3 discusses both parts and § 7.8 concludes. The appendices give the protocols of the two controlled experiments (Appendices § 8.1 and § 9), the computational record of all comparisons (Appendix § 9.1) and the computational reproduction and verification record (Appendix § 9.2).

I Theory

The sKMD extension of KMD

Reference decomposition and a joint regime law

Let a finite-dimensional representation of a KMD dictionary be H=[H1 ⋯ HJ]∈Rp×rH = [H_1 \ \cdots\ H_J] \in\mathbb{R}^{p \times r}, with coefficient blocks A=(A1T,…,AJT)TA = (A_1^{\mathsf{T}},\ldots,A_J^{\mathsf{T}})^{\mathsf{T}}. The synthesized signal is f=HA=∑jHjAjf = HA = \sum_j H_j A_j. A reference model with independent blocks has

A∼N(0,Γ),Γ=diag⁡(Γ1,…,ΓJ)≻0,Qj=HjΓjHjT.A \sim\mathcal{N}(0,\Gamma), \qquad\Gamma= \operatorname{diag}(\Gamma_1,\ldots,\Gamma_J) \succ0, \qquad Q_j = H_j\Gamma_j H_j^{\mathsf{T}}.

Here QjQ_j is the component kernel. If a known observation operator BoB_o acts on the signal, then Lo=BoHL_o = B_oH and y=LoA+ϵoy = L_oA + \epsilon_o, with independent Gaussian error of known positive-definite covariance RoR_o. Standard KMD conditioning gives a reference coefficient posterior N(m,V)\mathcal{N}(m,V) and mode estimates HjmjH_jm_j. The corresponding conditional mode covariances come from the blocks of VV; the observation can couple them even when the reference prior has independent blocks. A pooled reference may also use a full covariance Γ\Gamma.

Definition 2.1 (Stochastic KMD regime extension). Stochastic KMD (sKMD) is a supervised probabilistic extension of KMD. Fix a KMD coordinate model and its shared observation law (1), with reference coefficient posterior N(m,V)\mathcal{N}(m,V). Learn joint coefficient laws A∣Y=kA \mid Y = k from labeled training data for regimes Y∈{0,…,K−1}Y \in\{0,\ldots,K-1\} with priors πk>0\pi_k > 0. The Gaussian implementation uses A∣Y=k∼N(μk,Ck)A \mid Y = k \sim\mathcal{N}(\mu_k,C_k) with Ck≻0C_k \succ0. Transferring the reference posterior to these laws gives the regime posterior P(Y=k∣y,o)∝πkpk(y∣o)/pref(y∣o)\mathbb{P}(Y = k \mid y,o) \propto\pi_k p_k(y \mid o)/p_{\mathrm{ref}}(y \mid o), with the evidence ratio (2), the conditional mode laws (3), and their mixture over the regime posterior (Proposition 2.2).

Off-diagonal entries of CkC_k represent dependence between coordinates. They may encode relative modal polarity when a trigger or phase convention aligns records. The physical convention must be available to both fitting and prediction. An arbitrary sign choice that changes from record to record can erase this signal. Class-dependent means and variances can also distinguish regimes; coupling is one source of information, not a necessary feature of every useful regime model.

The fixed-model statements below condition on a supplied dictionary or on independent fitting data used to estimate it. Adaptation on the scored record needs a separate selection model; its limitations are addressed in § 2.20. Finite-rank notation describes the implemented examples and does not assert a finite-rank restriction on KMD as a general framework.

Coupled inference from a KMD posterior mean and covariance

A useful extension retains the full reference mode posterior when observation operators or noise vary. Let a prescribed KMD reference model have

A∼N(0,Γ),y=LoA+ϵo,ϵo∼N(0,Ro),Γ≻0, Ro≻0.(1)A \sim\mathcal{N}(0,\Gamma), \qquad y = L_o A + \epsilon_o, \qquad\epsilon_o \sim\mathcal{N}(0,R_o), \qquad\Gamma\succ0,\ R_o \succ0. \tag*{(1)}

The metadata oo specifies the observed mask/operator and noise covariance; Γ\Gamma may be diagonal. Its reference posterior is N(m,V)\mathcal{N}(m,V), where

V−1=Γ−1+LoTRo−1Lo,V−1m=LoTRo−1y.V^{-1} = \Gamma^{-1} + L_o^{\mathsf{T}} R_o^{-1} L_o, \qquad V^{-1}m = L_o^{\mathsf{T}} R_o^{-1}y.

Regime kk has a joint mode prior N(μk,Ck)\mathcal{N}(\mu_k,C_k), Ck≻0C_k \succ0. Assume that oo has the same distribution in every regime, or condition on a fixed oo. Otherwise its class likelihood must be included explicitly.

Proposition 2.2 (Exact posterior transfer through the KMD mode law). Define

Mk=V−1+Ck−1−Γ−1=Ck−1+LoTRo−1Lo≻0,bk=V−1m+Ck−1μk.M_k = V^{-1} + C_k^{-1} - \Gamma^{-1} = C_k^{-1} + L_o^{\mathsf{T}} R_o^{-1} L_o \succ0, \qquad b_k = V^{-1}m + C_k^{-1}\mu_k.

The class evidence relative to the reference is

pk(y∣o)pref(y∣o)=∣Γ∣1/2∣Ck∣1/2∣V∣1/2∣Mk∣1/2exp⁡{12[bkTMk−1bk−mTV−1m−μkTCk−1μk]}.(2)\frac{p_k(y\mid o)}{p_{\mathrm{ref}}(y\mid o)} = \frac{|\Gamma|^{1/2}}{|C_k|^{1/2}|V|^{1/2}|M_k|^{1/2}} \exp\left\{\frac{1}{2}\left[b_k^{\mathsf{T}}M_k^{-1}b_k-m^{\mathsf{T}}V^{-1}m-\mu_k^{\mathsf{T}}C_k^{-1}\mu_k\right]\right\}. \tag*{(2)}

Normalizing these ratios times πk\pi_k gives the regime posterior, and

A∣y,o,Y=k∼N(Mk−1bk,Mk−1).(3)A\mid y,o,Y=k \sim\mathcal{N}\left(M_k^{-1}b_k,M_k^{-1}\right). \tag*{(3)}

Thus (m,V)(m,V) is an exact interface for both regime inference and regime-conditioned reconstruction in this declared model family.

Proof. Bayes’ formula gives the change-of-prior identity

pk(y∣o)pref(y∣o)=EA∼N(m,V)pk(A)pref(A).\frac{p_k(y\mid o)}{p_{\mathrm{ref}}(y\mid o)} = \mathbb{E}_{A\sim\mathcal{N}(m,V)} \frac{p_k(A)}{p_{\mathrm{ref}}(A)}.

Collecting the quadratic and linear terms of its Gaussian integral gives Mk,bkM_k,b_k. Completing the square gives the determinant and exponential in (2), and the normalized integrand is the stated conditional Gaussian.

The reference mean alone is generally insufficient when oo varies. At the same m=0m=0, different posterior covariances VV give different determinant evidence for latent variance classes; continuity makes the distinction persist in neighborhoods. The posterior covariance records what a small coefficient estimate means under its actual observation quality. Conversely, with one fixed observation operator and fixed noise, VV is constant and mm determines V−1mV^{-1}m; the reference mean already preserves the class information in this model, even if the operator is rank deficient. The heterogeneous experiment can test the benefit of retaining this observation uncertainty; comparisons must also account for covariance estimation and regularization.

The final reconstructed mode law averages the conditional Gaussians over the regime posterior. Its covariance includes both conditional uncertainty and the covariance of the regime-specific means. The construction uses standard Gaussian conditioning, change of prior, and EM for latent Gaussian parameters. Common factor models and extreme deconvolution provide the corresponding generative families and heterogeneous-observation likelihoods [3, 15]. The sKMD extension expresses this inference through the KMD mode posterior, with matched dense likelihoods and simpler mode features as controls.

Reduction to KMD and the role of the controls

If every regime law equals the reference law, μk=0\mu_k = 0 and Ck=ΓC_k = \Gamma, then Mk=V−1M_k = V^{-1}, bk=V−1mb_k = V^{-1}m and every evidence ratio in (2) equals one. The regime posterior is πk\pi_k, and the coefficient posterior is exactly N(m,V)\mathcal{N}(m,V). Thus sKMD contains the reference KMD calculation as an exact special case.

Removing covariance off-diagonals gives an independent-coordinate regime ablation, but does not generally recover the reference model: the means or diagonal variances can still differ by regime. A single pooled full prior gives a stronger reconstruction control than separate independent mode priors. A signed point classifier retains coefficient signs while replacing the observation integration by a specified point-inference rule. Each experiment states that rule, its calibration and its fitting budget.

References

  1. [1]Othmane Aboussaad, Adam Miraoui, Boumediene Hamzi, and Houman Owhadi. Adaptive kernel selection for kernelized diffusion maps. Physica A: Statistical Mechanics and its Applications, 700:131853, 2026. doi: 10.1016/j.physa.2026.131853. URL https://doi.org/10.1016/j.physa.2026.131853.
  2. [2]Boris Alexeev, Afonso S. Bandeira, Matthew Fickus, and Dustin G. Mixon. Phase retrieval with polarization. SIAM Journal on Imaging Sciences, 7(1):35–66, 2014. doi: 10.1137/12089939X. URL https://arxiv.org/abs/1210.7752.
  3. [3]Jangsun Baek, Geoffrey J. McLachlan, and Lloyd K. Flack. Mixtures of factor analyzers with common factor loadings: Applications to the clustering and visualization of high-dimensional data. IEEE Transactions on Pattern Analysis and Machine Intelligence, 32(7):1298–1309, 2010. doi: 10.1109/TPAMI.2009.149.DOI
  4. [4]Paul Batlle, Matthieu Darcy, Matthew Levine, Boumediene Hamzi, Houman Owhadi, Mehrad Bastani, Yi Zhang, Khaled Sarsour, Francisco Talamas, and CPAD Database. Benchmarking Gaussian processes for prediction and data assimilation of Alzheimer’s disease progression. Journal of Alzheimer’s Disease, 109(3):1137–1141, 2026. doi: 10.1177/138728772514040482. URL https://doi.org/10.1177/138728772514040482.
  5. [5]Halima Bensmail and Gilles Celeux. Regularized Gaussian discriminant analysis through eigenvalue decomposition. Journal of the American Statistical Association, 91(436):1743–1748, 1996. doi: 10.1080/01621459.1996.10476746.DOI
  6. [6]Mario Beraha, Alberto Maria Metelli, Matteo Papini, Andrea Tirinzoni, and Marcello Restelli. Feature selection via mutual information: New theoretical insights. In International Joint Conference on Neural Networks, pages 1–9. IEEE, 2019. doi: 10.1109/IJCNN.2019.8852410. URL https://arxiv.org/abs/1907.07384.
  7. [7]Samuel Bilson, Miles McCrory, and Anna Pustogvar. An uncertainty-aware Bayesian framework for machine learning classification models: A case study in land cover classification, 2026. URL https://arxiv.org/abs/2503.21510v3. Version 3, revised 26 May 2026; first version posted 27 March 2025.
  8. [8]Andreas Bittracher, Stefan Klus, Boumediene Hamzi, Péter Koltai, and Christof Schütte. Dimensionality reduction of complex metastable systems via kernel embeddings of transition manifolds. Journal of Nonlinear Science, 31:3, 2021. doi: 10.1007/s00332-020-09668-z.DOI
  9. [9]Erik Lien Bolager, Boumediene Hamzi, Houman Owhadi, Ioannis G. Kevrekidis, and Felix Dietrich. Dictionary learning for kernel EDMD. Chaos: An Interdisciplinary Journal of Nonlinear Science, 36(10):103123, 2026. doi: 10.1063/5.0341144. URL https://doi.org/10.1063/5.0341144.
  10. [10]Jake Bouvrie and Boumediene Hamzi. Balanced reduction of nonlinear control systems in reproducing kernel Hilbert space. In Proceedings of the 48th Annual Allerton Conference on Communication, Control, and Computing, pages 294–301. IEEE, 2010. doi: 10.1109/ALLERTON.2010.5706920.DOI
  11. [11]Jake Bouvrie and Boumediene Hamzi. Empirical estimators for stochastically forced nonlinear systems: Observability, controllability and the invariant measure. In Proceedings of the 2012 American Control Conference, pages 4142–4148. IEEE, 2012. doi: 10.1109/ACC.2012.6315175.DOI
  12. [12]Jake Bouvrie and Boumediene Hamzi. Kernel methods for the model reduction of nonlinear control systems. Fourth International Workshop on Model Reduction in Reacting Flows, San Francisco, 2013. URL https://www.researchgate.net/publication/280038886_Kernel_Methods_for_the_Model_Reduction_of_Nonlinear_Control_Systems. Workshop contribution.
  13. [13]Jake Bouvrie and Boumediene Hamzi. Kernel methods for the approximation of nonlinear systems. SIAM Journal on Control and Optimization, 55(4):2460–2492, 2017. doi: 10.1137/14096815X.DOI
  14. [14]Jake Bouvrie and Boumediene Hamzi. Kernel methods for the approximation of some key quantities of nonlinear systems. Journal of Computational Dynamics, 4:1–19, 2017. doi: 10.3934/jcd.2017001.DOI
  15. [15]Jo Bovy, David W. Hogg, and Sam T. Roweis. Extreme deconvolution: Inferring complete distribution functions from noisy, heterogeneous and incomplete observations. The Annals of Applied Statistics, 5(2B):1657–1677, 2011. doi: 10.1214/10-AOAS439. URL https://arxiv.org/abs/0905.2979.
  16. [16]Michael Byrd, Linh Nghiem, and Monnie McGee. Bayesian regularization of Gaussian graphical models with measurement error. Computational Statistics & Data Analysis, 156:107085, 2021. doi: 10.1016/j.csda.2020.107085. URL https://arxiv.org/abs/1907.02241.
  17. [17]T. Tony Cai and Linjun Zhang. A convex optimization approach to high-dimensional sparse quadratic discriminant analysis. The Annals of Statistics, 49(3):1537–1568, 2021. doi: 10.1214/20-AOS2012. URL https://arxiv.org/abs/1912.02872.
  18. [18]Biao Chen and Joshua Kortje. Divergence maximizing linear projection for supervised dimension reduction. IEEE Transactions on Information Theory, 71(3):2104–2115, 2025. doi: 10.1109/TIT.2025.3528340. URL https://doi.org/10.1109/TIT.2025.3528340.
  19. [19]Marcelo A. Colominas and Hau-Tieng Wu. Time-varying spiky wave-shape functions for non-stationary signal decomposition, 2026. URL https://arxiv.org/abs/2608.26285v1. Version 1, 26 August 2026.
  20. [20]Patrick Danaher, Pei Wang, and Daniela M. Witten. The joint graphical lasso for inverse covariance estimation across multiple classes. Journal of the Royal Statistical Society: Series B (Statistical Methodology), 76(2):373–397, 2014. doi: 10.1111/rssb.12033. URL https://pmc.ncbi.nlm.nih.gov/articles/PMC4012833/.DOI
  21. [21]Matthieu Darcy, Boumediene Hamzi, Giulia Livieri, Houman Owhadi, and Peyman Tavallali. One-shot learning of stochastic differential equations with data adapted kernels. Physica D: Nonlinear Phenomena, 444:133583, 2023. doi: 10.1016/j.physd.2022.133583. URL https://doi.org/10.1016/j.physd.2022.133583.
  22. [22]Matthieu Darcy, Boumediene Hamzi, Jouni Susiluoto, Amy Braverman, and Houman Owhadi. Learning dynamical systems from data: A simple cross-validation perspective, part II: Nonparametric kernel flows. Physica D: Nonlinear Phenomena, 476:134641, 2025. doi: 10.1016/j.physd.2025.134641. URL https://doi.org/10.1016/j.physd.2025.134641.
  23. [23]Philip de Chazal, Maria O’Dwyer, and Richard B. Reilly. Automatic classification of heartbeats using ECG morphology and heartbeat interval features. IEEE Transactions on Biomedical Engineering, 51(7):1196–1206, 2004. doi: 10.1109/TBME.2004.827359.DOI
  24. [24]Kamal Dingle, Boumediene Hamzi, Marcus Hutter, and Houman Owhadi. Retrodicting chaotic systems: An algorithmic information theory approach. Entropy, 28(11):1208, 2026. doi: 10.3390/e28111208. URL https://doi.org/10.3390/e28111208.
  25. [25]Carl Eckart and Gale Young. The approximation of one matrix by another of lower rank. Psychometrika, 1(3):211–218, 1936. doi: 10.1007/BF02288367. URL https://doi.org/10.1007/BF02288367.
  26. [26]Yingying Fan, Yinfei Kong, Daoji Li, and Zemin Zheng. Innovated interaction screening for high-dimensional nonlinear classification. The Annals of Statistics, 43(3):1243–1272, 2015. doi: 10.1214/14-AOS1308. URL https://arxiv.org/abs/1501.01029.
  27. [27]Bernhard N. Flury. Common principal components in K groups. Journal of the American Statistical Association, 79(388):892–898, 1984. doi: 10.1080/01621459.1984.10477108.DOI
  28. [28]Peter Giesl, Boumediene Hamzi, Martin Rasmussen, and Kevin Webster. Approximation of Lyapunov functions from noisy data. Journal of Computational Dynamics, 7(1):57–81, 2020. doi: 10.3934/jcd.2020003.DOI
  29. [29]Peter Giesl, Sigurður F. Hafstein, Boumediene Hamzi, Jonghyeon Lee, Houman Owhadi, Gabriele Santin, and Umesh Vaidya. Kernel methods for the construction of certified Lyapunov functions via approximate Koopman eigenfunctions. Journal of Computational Physics, 568:115362, 2027. doi: 10.1016/j.jcp.2026.115362. URL https://doi.org/10.1016/j.j-cp.2026.115362. Published online in 2026.
  30. [30]Ary L. Goldberger, Luis A. N. Amaral, Leon Glass, Jeffrey M. Hausdorff, Plamen Ch. Ivanov, Roger G. Mark, Joseph E. Mietus, George B. Moody, Chung-Kang Peng, and H. Eugene Stanley. PhysioBank, PhysioToolkit, and PhysioNet: Components of a new research resource for complex physiologic signals. Circulation, 101(23):e215–e220, 2000. doi: 10.1161/01.CIR.101.23.e215.DOI
  31. [31]Lyudmila Grigoryeva, Boumediene Hamzi, Felix P. Kemeth, Yannis Kevrekidis, G. Manjunath, Juan-Pablo Ortega, and Matthys J. Steynberg. Data-driven cold starting of good reservoirs. Physica D: Nonlinear Phenomena, 469:134325, 2024. doi: 10.1016/j.physd.2024.134325. URL https://doi.org/10.1016/j.physd.2024.134325.
  32. [32]Yu Guo, Ting-Wei Liu, Jing Na, and Rong-Fong Fung. Envelope order tracking for fault detection in rolling element bearings. Journal of Sound and Vibration, 331(25):5644–5654, 2012. doi: 10.1016/j.jsv.2012.07.026. URL https://www.sciencedirect.com/science/article/abs/pii/S0022460X12005524.DOI
  33. [33]László Györfi, Tamás Linder, and Harro Walk. Lossless transformations and excess risk bounds in statistical inference. Entropy, 25(10):1394, 2023. doi: 10.3390/e25101394.DOI
  34. [34]Bernard Haasdonk, Boumediene Hamzi, Gabriele Santin, and Dominik Wittwar. Greedy kernel methods for center manifold approximation. In Spencer J. Sherwin, David Moxey, Joaquim Peiró, Peter E. Vincent, and Christoph Schwab, editors, Spectral and High Order Methods for Partial Differential Equations ICOSAHOM 2018, volume 134 of Lecture Notes in Computational Science and Engineering, pages 95–106. Springer, Cham, 2020. doi: 10.1007/978-3-030-39647-3_6.DOI
  35. [35]Bernard Haasdonk, Boumediene Hamzi, Gabriele Santin, and Dominik Wittwar. Kernel methods for center manifold approximation and a weak data-based version of the Center Manifold Theorem. Physica D: Nonlinear Phenomena, 427:133007, 2021. doi: 10.1016/j.physd.2021.133007.DOI
  36. [36]Till Habersetzer, Svea Steuer, Andreas Radeloff, and Bernd T. Meyer. MEG-SCANS: A comprehensive magnetoencephalography speech dataset with stories, chirps and noisy sentences. Scientific Data, 12:1938, 2025. doi: 10.1038/s41597-025-06397-4. URL https://www.nature.com/articles/s41597-025-06397-4.DOI
  37. [37]Boumediene Hamzi and Fritz Colonius. Kernel methods for discrete-time linear equations. In Computational Science – ICCS 2019, volume 11539 of Lecture Notes in Computer Science, pages 178–191. Springer, 2019. doi: 10.1007/978-3-030-22747-0_14. URL https://doi.org/10.1007/978-3-030-22747-0_14.
  38. [38]Boumediene Hamzi and Fritz Colonius. Kernel methods for the approximation of discrete-time linear autonomous and control systems. SN Applied Sciences, 1(7):674, 2019. doi: 10.1007/s42452-019-0701-3.DOI
  39. [39]Boum ediene Hamzi and Marcus Hutter. Algorithmic optimal transport: Predictive kernels, Gaussian laws, and description-length bounds, 2026. URL https://www.researchgate.net/publication/414897259. Author-posted preprint.
  40. [40]Boumediene Hamzi and Marcus Hutter. Category-native Solomonoff approximation: From algorithmic geometry to kernels, operators, and induction. Entropy, 28(9):1005, 2026. doi: 10.3390/e28091005. URL https://doi.org/10.3390/e28091005.
  41. [41]Boumediene Hamzi and Marcus Hutter. Computable approaches to Solomonoff induction: A survey from universal prediction to kernels, operators, and category-native projections, 2026. URL https://doi.org/10.13140/RG.2.2.25116.24960. Author-posted survey preprint.
  42. [42]Boumediene Hamzi and Houman Owhadi. Learning dynamical systems from data: A simple cross-validation perspective, part I: Parametric kernel flows. Physica D: Nonlinear Phenomena, 421:132817, 2021. doi: 10.1016/j.physd.2020.132817. URL https://doi.org/10.1016/j.physd.2020.132817.
  43. [43]Boumediene Hamzi and Houman Owhadi. A dynamics-informed Gaussian process framework for 2D stochastic Navier–Stokes via quasi-Gaussianity. arXiv:2511.21281, 2025. URL https://arxiv.org/abs/2511.21281. Preprint.
  44. [44]Boumediene Hamzi and Houman Owhadi. KSD–MMD transport with kernel learning for diffeomorphic measure matching. Communications in Nonlinear Science and Numerical Simulation, 163:110652, 2026. doi: 10.1016/j.cnsns.2026.110652. URL https://doi.org/10.1016/j.cnsns.2026.110652.
  45. [45]Boumediene Hamzi and Umesh Vaidya. Computation of Koopman eigenfunctions for stochastic systems via Feynman–Kac formula. IEEE Control Systems Letters, 10:2023–2028, 2026. doi: 10.1109/LCSYS.2026.3710305. URL https://doi.org/10.1109/LCSYS.2026.3710305.
  46. [46]Boumediene Hamzi and Umesh Vaidya. Kernel-based LMI approaches to solving the Hamilton–Jacobi–Bellman equation and nonlinear optimal control. Physica D: Nonlinear Phenomena, 497:135380, 2026. doi: 10.1016/j.physd.2026.135380. URL https://doi.org/10.1016/j.physd.2026.135380.
  47. [47]Boumediene Hamzi, Turky N. Alotaiby, Saleh Alshebeili, and Arwa Alangary. Kernel methods and the maximum mean discrepancy for seizure detection. In 2018 1st International Conference on Computer Applications & Information Security (ICCAIS), pages 1–6. IEEE, 2018. doi: 10.1109/CAIS.2018.8441977. URL https://doi.org/10.1109/CAIS.2018.8441977.
  48. [48]Boumediene Hamzi, Christian Kuehn, and Sameh Mohamed. A note on kernel methods for multiscale systems with critical transitions. Mathematical Methods in the Applied Sciences, 42(3):907–917, 2019. doi: 10.1002/mma.5394.DOI
  49. [49]Boumediene Hamzi, Romit Maulik, and Houman Owhadi. Simple, low-cost and accurate data-driven geophysical forecasting with learned kernels. Proceedings of the Royal Society A: Mathematical, Physical and Engineering Sciences, 477(2252):20210326, 2021. doi: 10.1098/rspa.2021.0326. URL https://doi.org/10.1098/rspa.2021.0326.
  50. [50]Boumediene Hamzi, Houman Owhadi, and Yannis Kevrekidis. Learning dynamical systems from data: A simple cross-validation perspective, part IV: Case with partial observations. Physica D: Nonlinear Phenomena, 454:133853, 2023. doi: 10.1016/j.physd.2023.133853. URL https://doi.org/10.1016/j.physd.2023.133853.
  51. [51]Boumédiene Hamzi, Houman Owhadi, and Léo Paillet. A note on microlocal kernel design for some slow–fast stochastic differential equations with critical transitions and application to EEG signals. Physica A: Statistical Mechanics and its Applications, 616:128583, 2023. doi: 10.1016/j.physa.2023.128583.DOI
  52. [52]Boumédiene Hamzi, Marcus Hutter, and Houman Owhadi. Bridging algorithmic information theory and machine learning: A new approach to kernel learning. Physica D: Nonlinear Phenomena, 464:134153, 2024. doi: 10.1016/j.physd.2024.134153. URL https://doi.org/10.1016/j.physd.2024.134153.
  53. [53]Boumédiene Hamzi, Marcus Hutter, and Houman Owhadi. Bridging algorithmic information theory and machine learning: Clustering, density estimation, Kolmogorov complexity-based kernels, and kernel learning in unsupervised learning. Physica D: Nonlinear Phenomena, 476:134669, 2025. doi: 10.1016/j.physd.2025.134669. URL https://doi.org/10.1016/j.physd.2025.134669.
  54. [54]Boumédiene Hamzi, Marcus Hutter, and Houman Owhadi. Learning theory from the viewpoint of algorithmic information theory: Kolmogorov complexity meets kernel methods. SSRN 5477437, 2025. URL https://doi.org/10.2139/ssrn.5477437. Preprint.
  55. [55]Boumédiene Hamzi, Kamal Dingle, Marcus Hutter, and Umesh Vaidya. Koopman and transfer operators on Kolmogorov RKHSs and Solomonoff Gaussian Hilbert spaces, 2026. URL https://www.researchgate.net/publication/414897201. Author-posted manuscript.
  56. [56]Boumédiene Hamzi, Marcus Hutter, Ioannis G. Kevrekidis, and Houman Owhadi. Learning stabilization entropy: A control-theoretic framework for quantifying training complexity. SSRN 6792409, 2026. URL https://doi.org/10.2139/ssrn.6792409. Preprint.
  57. [57]Boumédiene Hamzi, Marcus Hutter, and Houman Owhadi. Bridging machine learning and algorithmic information theory, part VII: Algorithmic information kernels and kernel discrepancies on countable spaces. Entropy, 28(10):1084, 2026. doi: 10.3390/e28101084. URL https://doi.org/10.3390/e28101084.
  58. [58]Boumédiene Hamzi, Marcus Hutter, Houman Owhadi, and Andrew Wilson. Bridging algorithmic information theory and machine learning, part IV: Solomonoff Gaussian Hilbert spaces, Solomonoff Gaussian processes, and Solomonoff Gaussian fields. Physica D: Nonlinear Phenomena, page 135371, 2026. doi: 10.1016/j.physd.2026.135371. URL https://doi.org/10.1016/j.physd.2026.135371. Published online 14 August 2026.
  59. [59]Boumédiene Hamzi, Marcus Hutter, Houman Owhadi, and Andrew Wilson. Bridging algorithmic information theory and machine learning, part V: Learning theory in Solomonoff Gaussian Hilbert spaces. SSRN 6792413, 2026. URL https://doi.org/10.2139/ssrn.6792413. Preprint.
  60. [60]Boumédiene Hamzi, Houman Owhadi, and Umesh Vaidya. Kernel methods for some transport equations with application to learning kernels for the approximation of Koopman eigenfunctions: A unified approach via variational methods, Green’s functions and the method of characteristics. Physica D: Nonlinear Phenomena, 496:135315, 2026. doi: 10.1016/j.physd.2026.135315. URL https://doi.org/10.1016/j.physd.2026.135315.
  61. [61]Boumédiene Hamzi, Houman Owhadi, and Umesh Vaidya. Kernel methods for stochastic dynamical systems with application to Koopman eigenfunctions: Feynman–Kac representations and RKHS approximation, 2026. URL https://arxiv.org/abs/2603.01077. arXiv:2603.01077.
  62. [62]Juee Jahagirdar, Boumediene Hamzi, Houman Owhadi, and Yannis Kevrekidis. Hausdorff metric-based kernel flows for learning invariant sets in dynamical systems. Physica D: Nonlinear Phenomena, 495:135287, 2026. doi: 10.1016/j.physd.2026.135287. URL https://doi.org/10.1016/j.physd.2026.135287.
  63. [63]Yasamin Jalalian, Mostafa Samir, Boumediene Hamzi, Peyman Tavallali, and Houman Owhadi. Data-efficient kernel methods for learning Hamiltonian systems, 2025. URL https://arxiv.org/abs/2509.17154. arXiv:2509.17154.
  64. [64]Binyan Jiang, Xiangyu Wang, and Chenlei Leng. A direct approach for sparse quadratic discriminant analysis. Journal of Machine Learning Research, 19(31):1–37, 2018. URL https://www.jmlr.org/papers/v19/17-285.html.
  65. [65]Rujun Jiang and Duan Li. Simultaneous diagonalization of matrices and its applications in quadratically constrained quadratic programming. SIAM Journal on Optimization, 26(3):1649–1668, 2016. doi: 10.1137/15M1023920. URL https://arxiv.org/abs/1507.05703.
  66. [66]Stefan Klus, Feliks Nü̈ske, and Boumediene Hamzi. Kernel-based approximation of the Koopman generator and Schrödinger operator. Entropy, 22(7):722, 2020. doi: 10.3390/e22070722.DOI
  67. [67]Béatrice Laurent and Pascal Massart. Adaptive estimation of a quadratic functional by model selection. The Annals of Statistics, 28(5):1302–1338, 2000. doi: 10.1214/aos/1015957395.
  68. [68]Quoc V. Le, Alex J. Smola, and Stéphane Canu. Heteroscedastic gaussian process regression. In Proceedings of the 22nd International Conference on Machine Learning, pages 489–496. ACM, 2005. doi: 10.1145/1102351.1102413. URL https://ai.stanford.edu/~quocle/LeSmoCan05.pdf.DOI
  69. [69]Jonghyeon Lee, Edward De Brouwer, Boumediene Hamzi, and Houman Owhadi. Learning dynamical systems from data: A simple cross-validation perspective, part III: Irregularly-sampled time series. Physica D: Nonlinear Phenomena, 443:133546, 2022. doi: 10.1016/j.physd.2022.133546. URL https://doi.org/10.1016/j.physd.2022.133546.
  70. [70]Jonghyeon Lee, Boumediene Hamzi, Boya Hou, Houman Owhadi, Gabriele Santin, and Umesh Vaidya. Kernel methods for the approximation of the eigenfunctions of the Koopman operator. Physica D: Nonlinear Phenomena, 476:134662, 2025. doi: 10.1016/j.physd.2025.134662. URL https://doi.org/10.1016/j.physd.2025.134662.
  71. [71]Jonghyeon Lee, Boumediene Hamzi, Yannis Kevrekidis, and Houman Owhadi. Gaussian processes simplify differential equations. Physica D: Nonlinear Phenomena, 483:134988, 2025. doi: 10.1016/j.physd.2025.134988. URL https://doi.org/10.1016/j.physd.2025.134988.
  72. [72]Jonghyeon Lee, Boumediene Hamzi, Yang Li, Houman Owhadi, Umesh Vaidya, and Naiming Xie. Kernel methods for the computation of quasi-potentials in stochastic dynamical systems. SSRN 5317127, 2025. URL https://doi.org/10.2139/ssrn.5317127. Preprint.
  73. [73]Jonghyeon Lee, Boumediene Hamzi, Houman Owhadi, and Umesh Vaidya. A note on kernel methods for the construction of Lyapunov functions using Koopman eigenfunctions. IFAC-PapersOnLine, 59(19):43–48, 2025. doi: 10.1016/j.ifacol.2025.11.008. URL https://doi.org/10.1016/j.ifacol.2025.11.008.
  74. [74]Leonidas Lefakis and François Fleuret. Jointly informative feature selection made tractable by Gaussian modeling. Journal of Machine Learning Research, 17(182):1–39, 2016. URL https://www.jmlr.org/papers/v17/15-026.html.
  75. [75]Daniel Lengyel, Boumediene Hamzi, Houman Owhadi, and Panos Parpas. Kernel sum of squares for data adapted kernel learning of dynamical systems from data: A global optimization approach. Physica D: Nonlinear Phenomena, 478:134693, 2025. doi: 10.1016/j.physd.2025.134693. URL https://doi.org/10.1016/j.physd.2025.134693.
  76. [76]Christian Lessmeier, James Kuria Kimotho, Detmar Zimmer, and Walter Sextro. Condition monitoring of bearing damage in electromechanical drive systems by using motor current signals of electric motors: A benchmark data set for data-driven classification. In PHM Society European Conference, volume 3, 2016. doi: 10.36001/phme.2016.v3i1.1577.DOI
  77. [77]Yu-Ting Lin, John Malik, and Hau-Tieng Wu. Wave-shape oscillatory model for non-stationary periodic time series analysis. Foundations of Data Science, 3(2):99–131, 2021. doi: 10.3934/fods.2021009. URL https://www.aimsciences.org/article/doi/10.3934/fods.2021009.DOI
  78. [78]George B. Moody and Roger G. Mark. The impact of the MIT-BIH arrhythmia database. IEEE Engineering in Medicine and Biology Magazine, 20(3):45–50, 2001. doi: 10.1109/51.932724. URL https://physionet.org/content/mitdb/1.0.0/.DOI
  79. [79]George B. Moody, W. E. Muldrow, and Roger G. Mark. A noise stress test for arrhythmia detectors. In Computers in Cardiology, pages 381–384, 1984. URL https://physionet.org/content/nstdb/1.0.0/.
  80. [80]Robb J. Muirhead. Aspects of Multivariate Statistical Theory. Wiley, 1982. doi: 10.1002/9780470316559.DOI
  81. [81]Houman Owhadi, Clint Scovel, and Gene Ryan Yoo. Kernel Mode Decomposition and the Programming of Kernels, volume 8 of Surveys and Tutorials in the Applied Mathematical Sciences. Springer, 2021. doi: 10.1007/978-3-030-82171-5. Preprint version: arXiv:1907.08592v2, 2020.
  82. [82]D. Peel and G. J. McLachlan. Robust mixture modelling using the t distribution. Statistics and Computing, 10(4):339–348, 2000. doi: 10.1023/A:1008981510081. URL https://people.smp.uq.edu.au/GeoffMcLachlan/pm_sc00.pdf.DOI
  83. [83]Tom Pollard, Benjamin E. Moody, Li-wei H. Lehman, Brian J. Gow, Chrystinne Fernandes, Chen Xie, Alistair Johnson, Roger G. Mark, and Thomas Heldt. PhysioNet as a global platform for biomedical research. Nature Health, 1(8):792–795, 2026. doi: 10.1038/s44360-026-00096-z.DOI
  84. [84]Viacheslav V. Prelov and Sergio Verdú. Second-order asymptotics of mutual information. IEEE Transactions on Information Theory, 50(8):1567–1580, 2004. doi: 10.1109/TIT.2004.831784.DOI
  85. [85]Lawrence R. Rabiner. A tutorial on hidden markov models and selected applications in speech recognition. Proceedings of the IEEE, 77(2):257–286, 1989. doi: 10.1109/5.18626.DOI
  86. [86]Joaquin Ruiz, Gastón Schlotthauer, Leandro Vignolo, and Marcelo A. Colominas. Fully adaptive time-varying wave-shape model: Applications in biomedical signal processing. Signal Processing, 214:109258, 2024. doi: 10.1016/j.sigpro.2023.109258. URL https://arxiv.org/abs/2309.15211.
  87. [87]Alexandre Smirnov, Boumediene Hamzi, and Houman Owhadi. Mean-field limits of trained weights in deep learning: A dynamical systems perspective. Dolomites Research Notes on Approximation, 15(3):125–145, 2022. doi: 10.14658/PUPJ-DRNA-2022-3-12. URL https://doi.org/10.14658/PUPJ-DRNA-2022-3-12.
  88. [88]Andrew H. Song, Demba Ba, and Emery N. Brown. PLSO: A generative framework for decomposing nonstationary time-series into piecewise stationary oscillatory components. In Proceedings of the Thirty-Seventh Conference on Uncertainty in Artificial Intelligence, volume 161 of Proceedings of Machine Learning Research, pages 1371–1381. PMLR, 2021. URL https://proceedings.mlr.press/v161/song21a.html.arxiv.org/abs/2010.11449
  89. [89]Roman Vershynin. High-Dimensional Probability: An Introduction with Applications in Data Science, volume 47 of Cambridge Series in Statistical and Probabilistic Mathematics. Cambridge University Press, 1 edition, 2018. doi: 10.1017/9781108231596. URL https://www.math.uci.edu/~rvershyn/papers/HDP-book/HDP-book.html.DOI
  90. [90]Baolei Wei, Lu Yang, Boumediene Hamzi, Houman Owhadi, and Naiming Xie. Ensemble kernel ridge regression for dynamical systems: Bagging, weighting, and uncertainty quantification, 2026. URL https://doi.org/10.13140/RG.2.2.18336.55041. Author-posted preprint.
  91. [91]William Wilkinson, Michael Andersen, Joshua D. Reiss, Dan Stowell, and Arno Solin. End-to-end probabilistic inference for nonstationary audio analysis. In Proceedings of the 36th International Conference on Machine Learning, volume 97 of Proceedings of Machine Learning Research, pages 6776–6785. PMLR, 2019. URL https://proceedings.mlr.press/v97/wilkinson19a.html.arxiv.org/abs/1901.11436
  92. [92]Evgeny Yakushenko. St petersburg INCART 12-lead arrhythmia database, 2008. URL https://physionet.org/content/incartdb/1.0.0/. Original release: 75 records grouped into 32 Holter sources; Open Data Commons Attribution License v1.0.
  93. [93]Lu Yang, Boumediene Hamzi, Yannis Kevrekidis, Houman Owhadi, Xiuwen Sun, and Naiming Xie. Hausdorff metric based training of kernels to learn attractors with application to 133 chaotic dynamical systems. Physica D: Nonlinear Phenomena, 466:134192, 2024. doi: 10.1016/j.physd.2024.134192. URL https://doi.org/10.1016/j.physd.2024.134192.
  94. [94]Lu Yang, Xiuwen Sun, Boumediene Hamzi, Houman Owhadi, and Naiming Xie. Learning dynamical systems from data: A simple cross-validation perspective, part V: Sparse Kernel Flows for 132 chaotic dynamical systems. Physica D: Nonlinear Phenomena, 460:134070, 2024. doi: 10.1016/j.physd.2024.134070. URL https://doi.org/10.1016/j.physd.2024.134070.

Paper details

vlnc:W_6eymsj5u
Version
v1 · latest
Published
Fresh — added to the Hub within the last 30 days
Added
License
CC BY 4.0
Usage

views · 0

downloads · 0

A view counts once per browser per day, a download each time the PDF is requested.

  • Mathematics

Not recorded for this paper

View statement
Looking for related papers…
Contents