betaFeedback

WorksComputer Science

The Clustering Occupation Kernel (CLOCK) Method for Time Series Arising from Dynamical Systems

Abstract

We introduce the Clustering Occupation Kernel (CLOCK) method, a novel representation-based framework for clustering time series arising from dynamical systems. CLOCK embeds observed sequences into a reproducing kernel Hilbert space (RKHS) via a time-averaged occupation kernel, enabling trajectory comparison through their RKHS norm. For systems governed by differential equations, this formulation rigorously approximates the kernel mean embedding of the invariant Sinai-Ruelle-Bowen (SRB) measure, meaning our dissimilarity metric approximates the maximum mean discrepancy (MMD) between attractors. To accommodate general time series, we introduce time lags via Takens' Embedding Theorem to reconstruct the phase space. Crucially, CLOCK natively handles trajectories of unequal lengths and irregularly-spaced observations. We validate the proposed metric using spectral clustering, demonstrating superior performance against benchmark methods on both synthetic datasets and a real-world intracranial large-scale electroencephalogram (EEG) dataset. Finally, ablation studies on the Lorenz-96 system are consistent with the theoretical convergence rates of the occupation kernels and highlight the method's strong robustness to zero-mean Gaussian noise.

Introduction

Time series, sequences of observations with accompanying timestamps [22], are ubiquitous objects in data science. The immense volume of available data makes annotating each time series infeasible, so unsupervised methods are needed to cluster data by shared characteristics. Time series clustering methods fall roughly into three categories: distance-based, model-based, and representation-based. Distance-based methods use a custom distance function defined on time series, so standard methods like K-means [19] can be adapted. The DTW barycenter-averaging method (DBA) [25] using dynamic time warping [30][4] is an example. Model-based methods cluster trajectories based on their similarity in the parameters of a model assumed to govern the data. For example, time-series-based Gaussian mixture models fall into this category [36]. Representation-based methods map the time series to latent feature vectors that capture the essential elements of each time series. Typical clustering methods in Euclidean space may be applied to these representations. Autoencoders [33][17] and contrastive-learning-based methods [40] are examples.

In this work, we adopt a representation-based perspective that views time series, after preprocessing, as solutions to ordinary differential equations (ODEs). Specifically, we embed each observed trajectory y:[0,T]→My:[0,T] \to M into a reproducing kernel Hilbert space (RKHS) by considering its time-averaged occupation embedding [28, 27]:

y↦1T∫0TK(⋅,y(t)) dt,(1)y \mapsto\frac{1}{T}\int_{0}^{T} K(\cdot,y(t))\,\mathrm{d}t, \tag*{(1)}

where KK is a positive-definite kernel defined on the state space MM.

This representation captures the time-averaged occupation measure of the trajectory in state space and provides a natural way to compare time series through Hilbert space distances. Its relevance becomes particularly clear when the observed trajectories are generated by dynamical systems. According to the Palis conjecture [23], a generic dynamical system defined as the flow of a CrC^r (r≥1r \ge1) map on a compact manifold admits (i) finitely many attractors, (ii) each of which is supported by a Sinai–Ruelle–Bowen (SRB) measure, and (iii) the union of the basins of these attractors has full Lebesgue measure. These SRB measures characterize the long-term statistical behavior of trajectories.

The main contribution of this work is to (1) show that the embedding defined by (1) approximates the kernel mean embedding (KME)[32] of these SRB measures, (2) leverage this connection to propose the Clustering Occupation Kernel method (CLOCK): a principled clustering method that groups trajectories according to their associated SRB measures, and (3) evaluate CLOCK against the state of the art.

We compute dissimilarities between time series by approximating the maximum mean discrepancy (MMD) between their estimated SRB measures. The authors in [5] took a related approach, embedding time series in high-dimensional feature space using sliding windows. Their goal was to provide a kernel-based distance measure that estimates distances by matching feature distributions that only partially overlap in their support. However, for time series generated by a dynamical system, their method clusters trajectories based on transient dynamics governed by the initial distribution. By contrast, our proposed method clusters trajectories according to their long-term, asymptotic statistics, making the resulting distances invariant to initial conditions as long as the trajectories are long enough. Ultimately, the choice between these methods depends on whether the application demands sensitivity to short-term transients or long-term systemic invariants.

Whether the time series is generated by an unknown discrete-time system or a continuous higher-order ODE, we can reconstruct its multi-dimensional phase space by preprocessing the observations using time-delay embedding before using CLOCK. Provided the underlying states evolve on a smooth, compact manifold, Takens’ theorem [35] guarantees that these delay-coordinate trajectories provide a diffeomorphism to the true orbits of the underlying system.

We demonstrate the proposed metric’s effectiveness through a series of clustering tasks. Specifically, we apply spectral clustering to similarity matrices derived from the MMD distances between trajectory embeddings. We present numerical experiments in which the time series are the solutions of ODEs corrupted by noise. Different clusters correspond to different vector-field parameters. In this context, we compare CLOCK with standard benchmark approaches. We also evaluate CLOCK on a large medical dataset of electroencephalogram measurements with a gold standard.

Method

Problem setup

We are given a collection of time series {yi}i=1n\{y_i\}_{i=1}^{n}, each composed of a finite collection of points {yij}j=0mi\{y_{ij}\}_{j=0}^{m_i} and a corresponding collection of timestamps {tij}j=0mi\{t_{ij}\}_{j=0}^{m_i}. In order to motivate the method, let the observations be noisy snapshots from the solutions to an initial value problem:

yij=xi(tij)+ϵij,dxidt=fc(xi),xi(ti0)=xi0,(2)y_{ij}=x_i(t_{ij})+\epsilon_{ij},\quad\frac{dx_i}{dt}=f_c(x_i),\quad x_i(t_{i0})=x_{i0}, \tag*{(2)}

where c∈{1,…,C}c \in\{1,\ldots,C\}, ϵij∼N(0,Σ)\epsilon_{ij} \sim N(0,\Sigma) are i.i.d. and where {fc}c∈{1,…,C}\{f_c\}_{c\in\{1,\ldots,C\}} is a finite collection of vector fields, typically within the same family but with different parameters. The goal is to find a cluster assignment {1,…,n}→{1,…,C}\{1,\ldots,n\}\to\{1,\ldots,C\} that maps each time series in the collection {(tij,yij)},1≤i≤n,0≤j≤mi\{(t_{ij},y_{ij})\},1\le i\le n,0\le j\le m_i to one of CC clusters without knowledge of the vector fields or the initial conditions. In this context, one could consider a model-based approach jointly estimating the cluster assignment, the vector fields, and the initial conditions using an Expectation-Maximization algorithm [6]. We present below a more direct approach: computing a similarity between trajectories that can be used as input to an existing clustering algorithm. Note that we do not assume that the time series are of equal length or that the observations are regularly spaced. In fact, the algorithm described below applies to any time-series clustering problem. However, it is justified for data generated by a dynamical system or data that can be preprocessed to exhibit the same statistical characteristics as if it were generated by a dynamical system.

Background

Let MM be a compact manifold, and let f:M→Mf:M\to M be a CrC^r map (r≥1r\ge1). A Sinai–Ruelle–Bowen (SRB) measure for ff is an ff-invariant Borel probability measure μ\mu on MM with several important properties. We describe the property of interest as follows: 1 Let ϕ:M→R\phi: M \to\mathbb{R} be a continuous function and let y:[0,∞)→My: [0, \infty) \to M be a solution to the differential equation defined by ff. There exists a positive-Lebesgue-measure set, denoted B(μ)B(\mu) and referred to as the basin of attraction for μ\mu, such that if y(0)∈B(μ)y(0) \in B(\mu), then the following holds:

lim⁡T→∞1T∫0Tϕ(y(t)) dt=∫Mϕ(p) dμ(p).(3)\lim_{T \to\infty} \frac{1}{T} \int_{0}^{T} \phi(y(t)) \, \mathrm{d}t = \int_{M} \phi(p) \, \mathrm{d}\mu(p). \tag*{(3)}

The Palis conjecture states that, for a dense set Φ\Phi in the space of all CrC^r (r≥1r \ge1) maps M→MM \to M with the CrC^r topology, each f∈Φf \in\Phi has a finite collection of SRB measures μ1,…,μn\mu_1, \ldots, \mu_n such that ⋃iB(μi)\bigcup_i B(\mu_i) has total Lebesgue measure. That is, λ(⋃iB(μi))=λ(M)\lambda(\bigcup_i B(\mu_i)) = \lambda(M) where λ(⋅)\lambda(\cdot) denotes the Lebesgue measure.

Let HH be a reproducing kernel Hilbert space (RKHS) of functions defined on a topological space X\mathcal{X}. Let ν\nu be a probability measure on X\mathcal{X}. Then the KME of ν\nu is given by the image of the mapping:

ν↦∫MK(⋅,p) dν(p).(4)\nu\mapsto\int_{M} K(\cdot,p) \, \mathrm{d}\nu(p). \tag*{(4)}

The MMD between two probability measures μ1,μ2\mu_1,\mu_2 defined on a topological space is the following quantity:

MMD⁡(μ1,μ2)=∥μ^1−μ^2∥H,(5)\operatorname{MMD}(\mu_1,\mu_2) = \lVert\hat{\mu}_1 - \hat{\mu}_2 \rVert_H, \tag*{(5)}

where μ^1,μ^2\hat{\mu}_1,\hat{\mu}_2 are the KME’s of μ1,μ2\mu_1,\mu_2, respectively. The MMD quantifies the dissimilarity of the measures μ1\mu_1 and μ2\mu_2. We say that KK is characteristic (see [32]) when the mapping (4) is one-to-one. If KK is characteristic, then the measures μ1\mu_1 and μ2\mu_2 are identical if and only if MMD⁡(μ1,μ2)=0\operatorname{MMD}(\mu_1,\mu_2)=0.

The clustering occupation kernel method (CLOCK)

Assume for now that each trajectory yiy_i is observed continuously over the time interval [ti0,timi][t_{i0},t_{im_i}]. Let us then embed yiy_i into an RKHS by computing its time-averaged occupation kernel. Thus, we select a continuous, positive-definite kernel KK with corresponding RKHS HH and compute the mapping:

yi↦1Ti∫ti0timiK(⋅,yi(t)) dt=:Li∗,(6)y_i \mapsto\frac{1}{T_i} \int_{t_{i0}}^{t_{im_i}} K(\cdot,y_i(t)) \, \mathrm{d}t =: L_i^*, \tag*{(6)}

where Ti:=timi−ti0T_i := t_{im_i} - t_{i0}. The motivation for this embedding comes from the following result, proved in Appendix § 7:

Theorem A.1. Let KK be a continuous, positive-definite kernel defined on a compact manifold MM. Let y:[0,∞)→My: [0,\infty) \to M be a solution to a first-order ODE satisfying the Palis conjecture. Then we have that the following holds almost surely:

lim⁡T→∞1T∫0TK(⋅,y(t)) dt=∫MK(⋅,p) dμ(p),(7)\lim_{T \to\infty} \frac{1}{T} \int_{0}^{T} K(\cdot,y(t)) \, \mathrm{d}t = \int_{M} K(\cdot,p) \, \mathrm{d}\mu(p), \tag*{(7)}

where μ\mu is the SRB measure of the attractor of the underlying dynamical system for which y(0)∈B(μ)y(0) \in B(\mu).

Proof sketch: We prove this theorem in two steps: first we prove weak convergence, and then we prove convergence of the norm, from which strong convergence follows. The convergence of the norm can be shown using the kernel’s Mercer property.

Now, according to Theorem A.1, we can approximate

Li∗≈∫MK(⋅,p) dμi(p)=:μ^i,(8)L_i^* \approx\int_M K(\cdot,p) \, \mathrm{d}\mu_i(p) =: \hat{\mu}_i, \tag*{(8)}

where μi\mu_i is the SRB measure of the attractor such that yi(0)∈B(μi)y_i(0) \in B(\mu_i). Note that μ^i\hat{\mu}_i is the KME of μi\mu_i.

We are now ready to propose the dissimilarity d2(⋅,⋅)d^2(\cdot,\cdot) between time series. It is defined by:

d2(yi,yj)=∥Li∗−Lj∗∥H2=1Ti2∫ti0timi∫ti0timiK(yi(t),yi(s)) dsdt+1Tj2∫tj0tjmj∫tj0tjmjK(yj(t),yj(s)) dsdt−2TiTj∫ti0timi∫tj0tjmjK(yi(t),yj(s)) dsdt.(9)\begin{aligned} d^2(y_i,y_j) &= \lVert L_i^* - L_j^* \rVert_H^2 = \\ &\frac{1}{T_i^2}\int_{t_{i0}}^{t_{im_i}}\int_{t_{i0}}^{t_{im_i}} K(y_i(t),y_i(s)) \, \mathrm{d}s\mathrm{d}t + \\ &\frac{1}{T_j^2}\int_{t_{j0}}^{t_{jm_j}}\int_{t_{j0}}^{t_{jm_j}} K(y_j(t),y_j(s)) \, \mathrm{d}s\mathrm{d}t - \\ &\frac{2}{T_iT_j}\int_{t_{i0}}^{t_{im_i}}\int_{t_{j0}}^{t_{jm_j}} K(y_i(t),y_j(s)) \, \mathrm{d}s\mathrm{d}t. \tag*{(9)} \end{aligned}

In practice, the time series are observed at a finite number of time points, thus we estimate the integrals with trapezoid-rule quadratures using the samples {yij}j=0mi\{y_{ij}\}_{j=0}^{m_i} and associated time points {tij}j=0mi\{t_{ij}\}_{j=0}^{m_i}, see Appendix § 8. This dissimilarity is an estimate of ∥μ^i−μ^j∥H2\lVert\hat{\mu}_i - \hat{\mu}_j \rVert_H^2, the squared MMD of μi\mu_i and μj\mu_j. Next, we convert dissimilarities to similarities in order to perform spectral clustering. We define the similarity S(⋅,⋅)S(\cdot,\cdot) according to:

S(yi,yj):=exp⁡(−12σ2∥Li∗−Lj∗∥H2),(10)S(y_i,y_j) := \exp\left(-\frac{1}{2\sigma^2}\lVert L_i^* - L_j^* \rVert_H^2\right), \tag*{(10)}

where σ:=median⁡{∥Li∗−Lj∗∥H}i,j\sigma:= \operatorname{median}\{\lVert L_i^* - L_j^* \rVert_H\}_{i,j} [10]. We obtain an (n,n)(n,n) matrix of similarities which we use as input for a standard spectral clustering algorithm. Note that other similarities may be derived from the given dissimilarity. All the experiments in this paper are presented with the similarity in (10).

Rate of convergence

We prove the following result in Appendix § 7:

Theorem A.2. Let M⊂RdM \subset\mathbb{R}^{d} be a compact forward-invariant region for the flow Φt\Phi_t of a C1C^{1} vector field, let μ\mu be an invariant probability measure on MM, and let KK be a continuous positive-definite kernel with Hölder feature map. Suppose that time-averaged occupation kernels started from X~∼μ\widetilde{X} \sim\mu converge to μ^\widehat{\mu} at mean-square rate O(T−1)O(T^{-1}) (e.g. under Lemma A.2), and that trajectories started from X∼νX \sim\nu are exponentially asymptotic to trajectories on the attractor, with asymptotic phases distributed absolutely continuously with respect to μ\mu (Assumption A.1). Then

∥1T∫0TK(⋅,Φt(X)) dt−∫K(⋅,p) dμ(p)∥H=OP(T−1/2).(11)\left\lVert\frac{1}{T}\int_{0}^{T} K(\cdot,\Phi_t(X))\,dt-\int K(\cdot,p)\,d\mu(p)\right\rVert_{H}=O_{\mathbb{P}}(T^{-1/2}). \tag*{(11)}

Remark 2.1. Among the systems of Section 3.1, the assumptions of Theorem A.2 hold for the FitzHugh–Nagumo system (hyperbolic sinks) and the Van der Pol system (hyperbolic limit cycles), with every kernel used in our experiments; see Appendix A.1. For the chaotic systems the theorem is not established. The theorem concerns noise-free trajectories as T→∞T \to\infty, so it motivates, rather than certifies, the finite-TT experiments.

We also provide a visualization of this rate in Section 3.5.

Application to general time series

We use Takens’ embedding theorem [35] to extend our setting to more general time series. We augment the time-series dimension by adding time lags. That is, we use trajectories of the form (y(t),y(t+τ),…,y(t+mτ))∈Rm+1(y(t), y(t+\tau), \ldots, y(t+m\tau)) \in\mathbb{R}^{m+1} where m∈Nm \in\mathbb{N} and τ>0\tau> 0 are fixed. If τ\tau and mm are chosen appropriately, the time-lagged trajectories will be the images of integral curves under a smooth embedding of a compact manifold into Rm+1\mathbb{R}^{m+1}. Thus, we may approximate the KMEs of their SRB measures.

The optimal time-lag τ\tau may be found by computing the first local minimum of the average mutual information of (y(t),y(t+τ))(y(t), y(t+\tau)) [9], while a large enough embedding dimension mm may be selected by minimizing the number of false nearest neighbors of the data points [12].

Scaling the method to large datasets

Like distance-based methods, this method requires computing dissimilarities for all possible pairs of time series in the dataset, so, by default, its complexity scales quadratically with the number of time series nn. However, by approximating an implicit kernel with an explicit one, e.g., pp random Fourier features [26] and using truncated (rank kk) singular value decomposition, the time and space complexities are reduced to, resp., O(npk)O(n p k) and O(k(n+p))O(k(n+p)).

Experiments

We compared the CLOCK method to a collection of standard methods on several synthetic datasets as well as a real, medical dataset.2 In addition, we performed ablation studies to determine the convergence rate of the time-averaged occupation kernels and the quality of the clustering as a function of trajectory length.

All experiments on synthetic data were conducted on a laptop with the Intel Core Ultra 9 Processor 185H, 64 GB RAM, and an NVIDIA GPU with 8GB of memory. The experiment on real data was conducted on a High-Performance Computing (HPC) cluster using a single NVIDIA A40 GPU with 45 GB RAM.

Synthetic datasets

For the synthetic datasets, we set the number of clusters to be 2 for simplifying the evaluation and generated 30 trajectories per cluster. Each cluster was generated by the same differential equation but with different parameters. Trajectories were integrated numerically using a standard fourth-order Runge-Kutta method with a fixed step size. White noise was then added. The datasets are visualized in Figure 1.

Synthetic datasets: (a) FHN, (b) Van der Pol, (c) Lorenz63, (d) Kuramoto, and (e) Lorenz96

Figure 1. Shown are the synthetic datasets in the (x1,x2)(x_1,x_2)-plane with coloring given by the true cluster labels. We present the true trajectories along with the noisy samples. For higher-dimensional datapoints, we plot only the first two coordinates.

  1. FitzHugh-Nagumo (FHN) oscillator: [8][21] This is a 2D system given by the equations:

dx1dt=x1−x133−x2+R,\frac{d x_{1}}{d t}=x_{1}-\frac{x_{1}^{3}}{3}-x_{2}+R,
dx2dt=1τ(x1+a−bx2),(12)\frac{d x_{2}}{d t}=\frac{1}{\tau}(x_{1}+a-bx_{2}), \tag*{(12)}

where aa, bb, RR, τ\tau are constants. For cluster 1, we used a=0.7a=0.7, b=0.8b=0.8, R=0.2R=0.2, τ=12.5\tau=12.5. For cluster 2, we used a=3.2a=3.2, b=1.7b=1.7, R=−0.4R=-0.4, τ=22.5\tau=22.5. We sampled initial conditions uniformly from the box [−2,2]2[-2,2]^{2} and integrated trajectories over the time interval [0,20][0,20]. We selected 101 regularly spaced points on each trajectory. We added zero-mean Gaussian noise with standard deviation 0.1.

  1. Van der Pol oscillator:[37] A 2D system with governing equations:

dx1dt=x2,dx2dt=μ(1−x12)x2−x1,(13)\frac{d x_{1}}{d t}=x_{2}, \qquad\frac{d x_{2}}{d t}=\mu(1-x_{1}^{2})x_{2}-x_{1}, \tag*{(13)}

where μ\mu is a constant. We selected μ=3\mu=3 for cluster 1 and μ=5\mu=5 for cluster 2. Initial conditions were sampled uniformly in the box [−2,2]2[-2,2]^2. We integrated trajectories over the time interval [0,80][0,80]. We then sampled 101 points at irregular intervals on each trajectory. We added Gaussian noise with standard deviation 0.5.

  1. Lorenz63:[14] This is a 3D dynamical system. The governing equations are given by:

dx1dt=σ(y−x),dx2dt=x(ρ−z)−y,\frac{dx_1}{dt} = \sigma(y-x), \quad\frac{dx_2}{dt} = x(\rho-z)-y,
dx3dt=xy−βz,(14)\frac{dx_3}{dt} = xy-\beta z, \tag*{(14)}

where σ,ρ,β\sigma,\rho,\beta are constants. For cluster 1, we used σ=10,ρ=28,β=8/3\sigma=10,\rho=28,\beta=8/3. For cluster 2, we used σ=8,ρ=33,β=2/3\sigma=8,\rho=33,\beta=2/3. Initial conditions were sampled uniformly from the box [−10,10]3[-10,10]^3. We integrated trajectories over the time interval [0,60][0,60] and selected 41 points at irregular intervals on each trajectory. We then added Gaussian noise with standard deviation 1.5

  1. Kuramoto:[13] We used a 10D Kuramoto model with governing equations:

dxidt=ωi+κ10∑j=1Nsin⁡(xj−xi)(15)\frac{dx_i}{dt}=\omega_i+\frac{\kappa}{10}\sum_{j=1}^{N}\sin(x_j-x_i) \tag*{(15)}

for i=1,…,10i=1,\ldots,10 where {ωi}i=110\{\omega_i\}_{i=1}^{10} and κ\kappa are constants. For both clusters, we set ω:=(1,1,…,1,2)\boldsymbol{\omega}:=(1,1,\ldots,1,\sqrt{2}). For cluster 1, we set κ=0.05\kappa=0.05, and for cluster 2, we set κ=1\kappa=1. Initial conditions were sampled from the box [−1,1]10[-1,1]^{10}. We integrated trajectories over the time interval [0,5][0,5] and selected 51 regularly-spaced points. We then added Gaussian noise with standard deviation 0.5.

  1. Lorenz96:[15] We used a 40D Lorenz96 system with governing equations:

dxidt=(xi+1−xi−2)xi−1−xi+F,(16)\frac{dx_i}{dt}=(x_{i+1}-x_{i-2})x_{i-1}-x_i+F, \tag*{(16)}

where we use the convention x−1:=x39x_{-1}:=x_{39}, x0:=x40x_0:=x_{40}, x41:=x1x_{41}:=x_1 and FF is a constant. For cluster 1, we used F=8F=8, and for cluster 2, we used F=10F=10. Initial conditions were sampled uniformly from the box [−10,10]40[-10,10]^{40}. We integrated trajectories over the time interval [0,60][0,60] and selected 41 irregularly-spaced points on each trajectory. We then added Gaussian noise with standard deviation 1.5.

Real dataset

  1. Bonn EEG time series dataset:[3] This dataset consists of 1D intracranial electroencephalogram (EEG) recordings of the human brain used for epileptic seizure detection research. There were two clusters in the dataset, each with 100 time series. Cluster 1 consisted of baseline, seizure-free readings in the epileptogenic zone, while Cluster 2 contained measurements of seizure activity taken from all regions of the brain where such activity was found. Automatically identifying seizure activity in EEG recordings has the potential to provide insight into brain function, and to aid in diagnosis and seizure prevention. Each time series was 23.6 seconds long with 4097 equally-spaced samples.

For the CLOCK method, we used Takens’ embedding theorem to augment the time series (see § 2.5). We set m=6m=6 and τ=13\tau=13 time-steps for this experiment.

Evaluation metrics

We used the class labels to define several clustering metrics. See Appendix § 10 for explicit formulas.

  1. Adjusted Rand Index (ARI): The adjusted Rand index [34] counts the proportion of data points that are clustered consistently with the true labels after adjusting for random chance. It takes values in the range of [−1,1][-1,1] with value 1 being a perfect clustering, 0 being entirely random, and negative values being worse than random.

  1. Normalized Mutual Information (NMI): The normalized mutual information [18] is an information-theoretic metric which quantifies how much the knowledge of the predicted labels reduces the information content of the true labels. It is normalized to take values in [0,1][0,1] with 1 being perfect correspondence and 0 indicating that the two sets of labels are independent.

  1. Purity: Purity [2] measures the homogeneity of the predicted clusters in terms of the true cluster assignments. In the setting of two predicted clusters and two true clusters, the purity is just the accuracy of the clustering.

Comparators

We tested the method against several benchmark methods. We implemented each method in a manner similar to that of CLOCK. Thus, each implementation consisted of the following general steps:

(a) Map the time series to representative feature vectors in latent space (using the identity mapping by default). (b) Compute pairwise distances between the feature vectors using a distance metric (using Euclidean distance by default).

(c) Convert distances to similarities using a Gaussian kernel, choosing the median σd:=median⁡{d(yi,yj)}i≠j\sigma_d := \operatorname{median}\{d(y_i,y_j)\}_{i\ne j} as the bandwidth.

(d) Obtain cluster assignments via spectral clustering on the matrix of similarities.

(e) By default, use linear interpolation to handle irregular spacing.

We tested against the following collection of methods:

  1. Euclidean distance: This baseline method uses ordinary Euclidean distance to compute distances between time series. If the time series yiy_i are given as matrices with shape T×dT \times d, with TT the number of timestamps and dd the dimension, we compute d2(yi,yj)=∥yi−yj∥RT×d2d^2(y_i,y_j) = \lVert y_i-y_j\rVert_{\mathbb{R}^{T\times d}}^2. Thus, the similarity metric S(⋅,⋅)S(\cdot,\cdot) is given by a standard Gaussian kernel.

  1. Dynamic Time Warping (DTW): We compute the distance between time series yiy_i and yjy_j with the DTW algorithm [30]. This algorithm aligns two time series by stretching and compressing segments with a dynamic one-to-many mapping before computing the distance between the aligned points. Time series with similar features that are out of phase along the time axis will be nearby in terms of DTW distance.

  1. Shape-Based Distance (SBD): SBD is a distance metric for time series used in the KShape algorithm [24]. Like DTW, it aligns the time series before computing distance. However, unlike DTW, it aligns time series by shifting; not stretching and compressing. The cross correlation statistic cc, which is a similarity measure in the range (−1,1)(-1,1), of the aligned, zz-normalized time series is computed, and the distance is given by 1−c1-c. This distance is invariant to scaling and shifting.

  1. Catch22: The Catch22 method [16] extracts 22 specially-chosen features from each time series to use as a low-dimensional representation. The 22 selected features were chosen from a pool of over 7000 candidate features and found to be optimal across a broad range of tasks (such as clustering and classification) using rigorous evaluation criteria.

  1. Latent ODE: This is a deep learning method [29] which models the time series as trajectories generated by a learned vector field and initial conditions in a latent, high-dimensional space. Since dynamical information from the trajectories is encoded in the latent initial conditions, we used them as feature vectors for the time series. This method takes irregular spacing into account, and thus we don’t have to interpolate.

  1. TS2Vec: TS2Vec [40] is another deep learning method using a contrastive framework to learn representations of time series. During training, positive pairs of context views are optimized via a contrastive loss. The context views are aligned across the time dimension, as well as across spatial dimensions. The contrastive loss is applied hierarchically at various scales to account for low- and high-resolution features. To handle irregular spacing, we formed a uniform grid of time-points [tmin⁡,tmax⁡][t_{\min},t_{\max}] and inserted samples at times corresponding to observations. We masked out the other points.

Latent ODE and TS2Vec have several hyperparameters to tune. We used an internal clustering metric, the density-based clustering validation (DBCV) score, for validating them. See Appendix § 11 for details. See Table 1 for the results of the numerical experiments.

DatasetMethodARINMIPurity
FHNCLOCK GM111
FHNCLOCK quad0.68930.66150.9167
FHNEuclidean0.06380.18920.6333
FHNDTW0.08210.21020.65
FHNSBD0.15080.27460.7
FHNCatch220.6340.6190.9
FHNLatent ODE0.12090.11440.6833
FHNTS2Vec111
Van der PolCLOCK GM0.68930.66150.9167
Van der PolCLOCK quad-0.017200.5
Van der PolEuclidean-0.017200.5
Van der PolDTW00.03010.5167
Van der PolSBD111
Van der PolCatch220.74690.70840.9333
Van der PolLatent ODE-0.01240.00330.5333
Van der PolTS2Vec111
Lorenz63CLOCK GM111
Lorenz63CLOCK quad0.86890.82140.9667
Lorenz63Euclidean0.09690.09110.6667
Lorenz63DTW00.03010.5167
Lorenz63SBD-0.014400.5
Lorenz63Catch22-0.01240.00330.5333
Lorenz63Latent ODE111
Lorenz63TS2Vec111
KuramotoCLOCK GM0.68930.66150.9167
KuramotoCLOCK quad0.11970.10080.6833
KuramotoEuclidean0.09580.08170.6667
KuramotoDTW0.15080.27460.7
KuramotoSBD-0.017200.5
KuramotoCatch22-0.01260.00320.5333
KuramotoLatent ODE-0.0070.00720.55
KuramotoTS2Vec0.00110.01320.5667
Lorenz96CLOCK GU111
Lorenz96CLOCK quad0.39220.4490.8167
Lorenz96Euclidean00.03010.5167
Lorenz96DTW00.03010.5167
Lorenz96SBD-0.01050.00130.5167
Lorenz96Catch22-0.002200.5
Lorenz96Latent ODE-0.00410.00370.5167
Lorenz96TS2Vec0.86890.82140.9667
EEGCLOCK GM0.68730.58210.915
EEGCLOCK Laplace0.73830.63410.93
EEGEuclidean0.00060.02730.515
EEGDTW0.11990.2420.675
EEGSBD0.06040.17870.625
EEGCatch220.01830.10810.57
EEGLatent ODE0.04210.12460.605
EEGTS2Vec0.68740.61590.915

Table 1. Results of Numerical Experiments. The CLOCK method is listed along with its accompanying kernel. Information on the kernels used can be found in Appendix C. The Gaussian Multiscale kernel (GM) with one bandwidth per dimension was the default choice. The Gaussian Uniscale kernel (GU) with a single bandwidth for all dimensions was chosen for high-dimensional data. The quadratic kernel (quad) and Laplace kernel were also used. Best results for each metric are in bold.

Ablation studies

We carried out two ablation studies to determine the the rate of convergence of time-averaged occupation kernels to the associated KMEs of SRB measures, and the quality of the clustering of the CLOCK method as a function of trajectory length. For both experiments, we used the Gaussian uniscale kernel (see section C for details).

The convergence rate of time-averaged integrals of observables over trajectories to the spatial integral of the observable with respect to the associated SRB measure is guaranteed to be O(T−1/2)O(T^{-1/2}) for systems with sufficient regularity (see [38] for details). We computed the empirical convergence rate for the 40D Lorenz-96 system with F=8F = 8. To do so, we integrated a single trajectory over the interval [0,300][0,300] with random initial condition in the box [−10,10]40[-10,10]^{40}. We computed a time-averaged occupation kernel for this trajectory, and used it as a proxy for the KME of the SRB measure of the Lorenz-96 attractor, denoted μ^\hat{\mu}. For each of a collection of equally-spaced increasing time-points {Tj}j=120\{T_j\}_{j=1}^{20}, Tj<300T_j < 300, we integrated 1000 trajectories over the interval [0,Tj][0,T_j] with random initial conditions drawn from the box [−10,10]40[-10,10]^{40} and computed ∥Li,j∗−μ^∥H\|L^*_{i,j}-\hat{\mu}\|_H for i=1,…,1000i=1,\ldots,1000, j=1,…,20j=1,\ldots,20, where Li,j∗L^*_{i,j} is the time-averaged occupation kernel of trajectory ii of length TjT_j. A log-log plot of the resulting mean error over all trajectories as a function of trajectory length is plotted in Figure 2 (a). Note that after an initial "burn-in" period, we observe a straight line with slope ≈−0.48\approx-0.48, as expected.

Figure 2. In (a), the log-mean-squared error between the time-averaged occupation kernels and the empirical KME of the SRB measure of the Lorenz-96 attractor is plotted as a function of the log of the trajectory length TT, along with a 95% confidence interval (CI). The line with slope −0.5-0.5 and intercept −0.96-0.96 is plotted for reference. In (b), the ARI of clusterings with the CLOCK method on the Lorenz-96 system is plotted with a 95% CI as a function of TT.

For the next experiment, we generated two sets of trajectories for the 40D Lorenz-96 system, each with 30 trajectories. The first set corresponded to the parameter choice of F=8F = 8, and the second to F=10F = 10. Each trajectory was integrated over the interval [0,15][0,15] with initial conditions drawn randomly from the box [−10,10]40[-10,10]^{40}. For each of an equally-spaced increasing sequence of time-points {Tj}j=1100\{T_j\}_{j=1}^{100}, 0≤Tj<60 \le T_j < 6, we clustered the trajectories truncated to the interval [0,Tj][0,T_j] with the CLOCK method. The mean ARI over 40 runs as a function of trajectory length is plotted in Figure 2 (b).

Discussion

As can be seen from Table 1, the CLOCK method achieved superior performance on all but one dataset. The quadratic kernel failed on the Van der Pol dataset most likely due to the degree mismatch between the underlying system and the kernel. However, no other method did as well as CLOCK overall, with the closest competitor being TS2Vec. TS2Vec experienced a significant degradation of performance on the Kuramoto dataset. This may be due to the trajectories in this dataset being locally similar but globally divergent (see Figure 1d), so that the hierarchical contrastive framework encoded dissimilar trajectories with similar feature vectors. We speculate that the overall good performance of the CLOCK method can be attributed to its capacity to accommodate the dynamical information contained in the trajectories. Although Latent ODE also incorporates dynamical information, it yields a single dynamical model for trajectories with multiple dynamics, which may be problematic. These results highlight the need for a method specifically-designed to cluster trajectories generated by differential equations.

In Appendix F, we demonstrate empirically how the CLOCK algorithm maintains good performance on extremely noisy datasets.

Limitations

While the proposed embedding and clustering framework can be applied to general time series without explicit assumptions, its theoretical motivation relies on the setting where trajectories are generated by dynamical systems. In practice, there are datasets for which this assumption is not satisfied. In such cases, it may be necessary to first transform the data into a representation that better reflects an underlying dynamical structure, for instance via delay-coordinate embeddings such as Takens’ embedding.

The method assumes that the time series are of sufficient length to capture the nature of the invariant measures of the attractors. Moreover, there exist trajectories generated by different dynamical systems that exhibit distinct short-term (transient) behaviors but share the same asymptotic behavior, i.e., are associated with the same SRB measure. In such cases, the proposed method may fail to distinguish between them, as it clusters trajectories based on their long-term statistical properties rather than their transient dynamics.

Conclusion

We have presented a clustering method using a novel dissimilarity metric for time series. The metric is given by the squared RKHS norm between time-averaged occupation kernels. The Palis conjecture and Theorem A.1 assert that this dissimilarity metric measures how distinct the associated SRB measures are, and thus is able to quantify the difference in the dynamics between two time series.

The method has displayed robust performance on a wide variety of synthetic datasets: low and high dimensions, regular and irregular spacing, and all with a significant amount of noise. This suggests that the RKHS distance between time-averaged occupation kernels is an effective metric whenever it is suspected that the data was generated by differential equations. In

addition, this method achieved the best performance on a real dataset consisting of EEG measurements.

Future work

There are a several directions for future research: Spectral clustering may be replaced with K-Means or community clustering. Additionally, different kernels may be selected to compute embeddings, and there are many ways to convert dissimilarities into similarities. A particularly promising direction is the extension of our framework to stochastic dynamical systems, including stochastic differential equations and continuous-time Markov chains where clustering could be performed based on invariant measures.

AI Use Statement

We were assisted by the Gemini 3.1 Pro LLM model in this research. Specifically, the LLM assisted in locating the foundational result that this work is based on: the Palis conjecture justifying the use of time-averaged occupation kernels as feature vectors for time series. Additionally, the LLM assisted with the design of the numerical experiments by recommending datasets and evaluation metrics to use. However, all theoretical proofs and numerical experiments were carried out by the authors themselves, and all theoretical results referenced by the LLM were verified in the original source material. We take responsibility for the final content of this work.

Supplementary Materials

Theoretical results

An SRB measure μ\mu for an attractor of an ODE defined on a compact manifold MM has the property that

lim⁡T→∞1T∫0Tϕ(y(t))dt=∫Mϕ(p)dμ(p)(17)\lim_{T\to\infty}\frac{1}{T}\int_0^T \phi(y(t))dt=\int_M \phi(p)d\mu(p) \tag*{(17)}

for every continuous function ϕ:M→R\phi:M\to\mathbb{R} and trajectory y:[0,∞)→My:[0,\infty)\to M such that y(0)∈B(μ)y(0)\in B(\mu), the basin of attraction for μ\mu. The Palis conjecture states that, for a dense set in the set of all CrC^r maps (r≥1r\geq1) on MM with the CrC^r topology, there exists a finite collection of SRB measures μ1,…,μn\mu_1,\ldots,\mu_n such that ∪iB(μi)\cup_iB(\mu_i) has total Lebesgue measure. That is, λ(∪iB(μi))=λ(M)\lambda(\cup_iB(\mu_i))=\lambda(M) where λ(⋅)\lambda(\cdot) denotes the Lebesgue measure.

We wish to show that (17) extends to feature vectors in an RKHS. More specifically, we want to show that

lim⁡T→∞1T∫0TK(⋅,y(t))dt=∫MK(⋅,p)dμ(p).(18)\lim_{T\to\infty}\frac{1}{T}\int_0^T K(\cdot,y(t))dt=\int_M K(\cdot,p)d\mu(p). \tag*{(18)}

This is the content of Theorem A.1 below. In Corollary ???, we show that the rate of convergence is O(T−1/2)\mathcal{O}(T^{-1/2}).

Proposition A.1. Let K:M×M→RK:M\times M\to\mathbb{R} be a continuous kernel defined on a compact manifold MM. Let {Ti}i=1∞\{T_i\}_{i=1}^{\infty} be an increasing sequence of positive numbers such that Ti→∞T_i\to\infty. Let y:[0,∞)→My:[0,\infty)\to M denote a trajectory generated by an ODE such that y(0)∈B(μ)y(0)\in B(\mu), the basin of attraction of an SRB measure μ\mu. Define the linear functional Li:H→RL_i:H\to\mathbb{R} according to

Liϕ=1Ti∫0Tiϕ(y(t))dt.(19)L_i\phi=\frac{1}{T_i}\int_0^{T_i}\phi(y(t))dt. \tag*{(19)}

Then each LiL_i is bounded, and their Riesz representers, denoted Li∗L_i^*, converge weakly to the function μ^\hat{\mu} defined by

μ^:=∫MK(⋅,p)dμ(p).(20)\hat{\mu}:=\int_M K(\cdot,p)d\mu(p). \tag*{(20)}

Proof. The boundedness of the LiL_i follows from the compactness of MM and the continuity of KK (see [27] for details). For the same reason, the operator

ϕ↦∫Mϕ(p)dμ(p)(21)\phi\mapsto\int_M\phi(p)d\mu(p) \tag*{(21)}

is bounded, and its Riesz representer is μ^\hat{\mu}. The functions Li∗L_i^* are given by

Li∗=1Ti∫0TiK(⋅,y(t))dt.(22)L_i^*=\frac{1}{T_i}\int_0^{T_i}K(\cdot,y(t))dt. \tag*{(22)}

Because KK is continuous, every function ϕ\phi in the corresponding RKHS is continuous. Thus, since μ\mu is an SRB measure, we get

⟨ϕ,Li∗⟩H=Liϕ=1Ti∫0Tiϕ(y(t))dt→∫Mϕ(p)dμ(p)=⟨ϕ,μ^⟩H.(23)\langle\phi,L_i^*\rangle_H=L_i\phi=\frac{1}{T_i}\int_0^{T_i}\phi(y(t))dt\to\int_M\phi(p)d\mu(p)=\langle\phi,\hat{\mu}\rangle_H. \tag*{(23)}

Lemma A.1. Let Li∗L_i^* and μ^\hat{\mu} be as in Proposition A.1. Then the following holds:

∥Li∗∥H→∥μ^∥H(24)\|L_i^*\|_H\to\|\hat{\mu}\|_H \tag*{(24)}

as i→∞i\to\infty.

Proof. We have

∥Li∗∥H2=⟨Li∗,Li∗⟩H=1Ti2∫0Ti∫0TiK(y(s),y(t)) dsdt.(25)\|L_i^*\|_H^2 = \langle L_i^*,L_i^*\rangle_H = \frac{1}{T_i^2}\int_0^{T_i}\int_0^{T_i} K(y(s),y(t))\,dsdt. \tag*{(25)}

Since KK is continuous and MM is compact, KK is a Mercer kernel. Thus,

1Ti2∫0Ti∫0TiK(y(s),y(t)) dsdt=1Ti2∫0Ti∫0Ti∑j=1∞λjϕj(y(s))ϕj(y(t)) dsdt(26)\frac{1}{T_i^2}\int_0^{T_i}\int_0^{T_i} K(y(s),y(t))\,dsdt = \frac{1}{T_i^2}\int_0^{T_i}\int_0^{T_i}\sum_{j=1}^{\infty}\lambda_j\phi_j(y(s))\phi_j(y(t))\,dsdt \tag*{(26)}

for some collection of non-negative eigenvalues {λj}\{\lambda_j\} and eigenfunctions {ϕj}\{\phi_j\}. Let

hi,N:=1Ti2∫0Ti∫0Ti∑j=1Nλjϕj(y(s))ϕj(y(t)) dsdt=∑j=1Nλj(1Ti∫0Tiϕj(y(t)) dt)2.(27)h_{i,N} := \frac{1}{T_i^2}\int_0^{T_i}\int_0^{T_i}\sum_{j=1}^{N}\lambda_j\phi_j(y(s))\phi_j(y(t))\,dsdt = \sum_{j=1}^{N}\lambda_j\left(\frac{1}{T_i}\int_0^{T_i}\phi_j(y(t))\,dt\right)^2. \tag*{(27)}

We have that for any fixed NN, the sequence hi,Nh_{i,N} converges as i→∞i \to\infty since μ\mu is an SRB measure. Likewise, for any fixed ii, the sequence has a limit as N→∞N \to\infty since the sequence of partial sums converges to the time-averaged double integral of the kernel over two copies of the trajectory (using the Dominated Convergence Theorem to pass the limit inside the integrals). We have that

lim⁡N→∞lim⁡i→∞hi,N=lim⁡N→∞∬∑j=1Nλjϕj(p)ϕj(q) dμ(p)dμ(q)=∬K(p,q) dμ(p)dμ(q)=∥μ∥H2,(28)\lim_{N\to\infty}\lim_{i\to\infty}h_{i,N} = \lim_{N\to\infty}\iint\sum_{j=1}^{N}\lambda_j\phi_j(p)\phi_j(q)\,d\mu(p)d\mu(q) = \iint K(p,q)\,d\mu(p)d\mu(q) = \|\mu\|_H^2, \tag*{(28)}

where in the second equality we used the Dominated Convergence Theorem with the bound

∣∑j=1Nλjϕj(p)ϕj(q)∣≤K(p,p)K(q,q).(29)\left|\sum_{j=1}^{N}\lambda_j\phi_j(p)\phi_j(q)\right| \le\sqrt{K(p,p)K(q,q)}. \tag*{(29)}

We also have

lim⁡i→∞lim⁡N→∞hi,N=lim⁡i→∞1Ti2∫0Ti∫0TiK(y(s),y(t)) dsdt=lim⁡i→∞∥Li∗∥H2,(30)\lim_{i\to\infty}\lim_{N\to\infty}h_{i,N} = \lim_{i\to\infty}\frac{1}{T_i^2}\int_0^{T_i}\int_0^{T_i}K(y(s),y(t))\,dsdt = \lim_{i\to\infty}\|L_i^*\|_H^2, \tag*{(30)}

using Dominated Convergence in the first equality. Thus, we need to show that the order of the limits is exchangeable. Let hih_i denote the limit lim⁡N→∞hi,N\lim_{N\to\infty}h_{i,N}. We have

∣hi,N−hi∣=∣1Ti2∫0Ti∫0Ti∑j=N+1∞λjϕj(y(s))ϕj(y(t)) dsdt∣≤1Ti2∫0Ti∫0Timax⁡(y1,y2)∈M×M∣∑j=N+1∞λjϕj(y1)ϕj(y2)∣ dsdt=max⁡(y1,y2)∈M×M∣∑j=N+1∞λjϕj(y1)ϕj(y2)∣→0,(31)\begin{aligned} |h_{i,N}-h_i| &= \left|\frac{1}{T_i^2}\int_0^{T_i}\int_0^{T_i}\sum_{j=N+1}^{\infty}\lambda_j\phi_j(y(s))\phi_j(y(t))\,dsdt\right| \\ &\le\frac{1}{T_i^2}\int_0^{T_i}\int_0^{T_i}\max_{(y_1,y_2)\in M\times M}\left|\sum_{j=N+1}^{\infty}\lambda_j\phi_j(y_1)\phi_j(y_2)\right|\,dsdt \\ &= \max_{(y_1,y_2)\in M\times M}\left|\sum_{j=N+1}^{\infty}\lambda_j\phi_j(y_1)\phi_j(y_2)\right| \to0, \tag*{(31)} \end{aligned}

where on the last line we used the uniform convergence of the Mercer kernel on M×MM\times M. The bound on ∣hi,N−hi∣|h_{i,N}-h_i| does not depend on ii and converges to zero. Thus, convergence of hi,Nh_{i,N} in NN is uniform, and we may apply the Moore-OsGood Theorem to show that the order of the limits is exchangeable. The result follows. □

Theorem A.1. Let the setting of Proposition A.1 hold. Then we have the following:

1Ti∫0TiK(⋅,y(t)) dt→∫MK(⋅,p) dμ(p)(32)\frac{1}{T_i}\int_0^{T_i}K(\cdot,y(t))\,dt \to\int_M K(\cdot,p)\,d\mu(p) \tag*{(32)}

as i→∞i \to\infty.

Proof. By Proposition A.1 and Lemma A.1, we have

∥Li∗−μ^∥H2=∥Li∗∥H2−2⟨Li∗,μ^⟩H+∥μ^∥H2→∥μ^∥H2−2∥μ^∥H2+∥μ^∥H2=0.(33)\left\|L_i^*-\hat{\mu}\right\|_H^2=\left\|L_i^*\right\|_H^2-2\langle L_i^*,\hat{\mu}\rangle_H+\left\|\hat{\mu}\right\|_H^2\to\left\|\hat{\mu}\right\|_H^2-2\left\|\hat{\mu}\right\|_H^2+\left\|\hat{\mu}\right\|_H^2=0. \tag*{(33)}

For systems with sufficient regularity (for example, hyperbolic systems; see [38] for details), the time-average of an observable over a trajectory converges to the spatial average with respect to the associated SRB measure at a rate of OP(T−1/2)O_{\mathbb{P}}(T^{-1/2}). We use this fact to show that the time-averaged occupation kernels with random initial conditions drawn from Lebesgue-absolutely-continuous distributions converge (in probability) to the KMEs of SRB measures at the same rate.

Definition A.1. Let f:M→Mf:M\to M be a CrC^r (r≥1r\ge1) map on MM. Let Φt:M→M\Phi_t:M\to M be the flow of ff at time tt. Define the centered-kernel function Kˉ:M×M→R\bar{K}:M\times M\to\mathbb{R} according to:

Kˉ(x,y):=K(x,y)−∫MK(x,p)dμ(p)−∫MK(q,y)dμ(q)+∫M∫MK(p,q)dμ(p)dμ(q).(34)\bar{K}(x,y):=K(x,y)-\int_M K(x,p)d\mu(p)-\int_M K(q,y)d\mu(q)+\int_M\int_M K(p,q)d\mu(p)d\mu(q). \tag*{(34)}

Define the auto-correlation function for Kˉ\bar{K} as:

CKˉ(τ):=∫MKˉ(p,Φτ(p))dμ(p)(35)C_{\bar{K}}(\tau):=\int_M \bar{K}(p,\Phi_\tau(p))d\mu(p) \tag*{(35)}

First, we prove a convergence result for the case that the initial condition is distributed according to μ\mu, an SRB measure for ff. We then extend the result to more general measures in the next result.

Lemma A.2. Assume we are in the setting of Definition A.1. Let X∼μX\sim\mu, where μ\mu is an SRB measure for ff. Suppose that the following regularity condition holds:

σKˉ2:=2∫0∞∣CKˉ(τ)∣dτ<∞(36)\sigma_{\bar{K}}^2:=2\int_0^\infty\left|C_{\bar{K}}(\tau)\right|d\tau<\infty \tag*{(36)}

Then we have the following convergence in probability:

∥1T∫0TK(⋅,Φt(X))dt−μ^∥H→P0(37)\left\|\frac{1}{T}\int_0^T K(\cdot,\Phi_t(X))dt-\hat{\mu}\right\|_H\xrightarrow{\mathbb{P}}0 \tag*{(37)}

at the rate OP(T−1/2)O_{\mathbb{P}}(T^{-1/2}).

Proof. Denote by L∗(μ,T)L^*(\mu,T) the following:

L∗(μ,T):=1T∫0TK(⋅,Φt(X))dt(38)L^*(\mu,T):=\frac{1}{T}\int_0^T K(\cdot,\Phi_t(X))dt \tag*{(38)}

where X∼μX\sim\mu. We have

∥L∗(μ,T)−μ^∥H2=1T2∫0T∫0TKˉ(Φt(X),Φs(X))dsdt.(39)\left\|L^*(\mu,T)-\hat{\mu}\right\|_H^2=\frac{1}{T^2}\int_0^T\int_0^T\bar{K}(\Phi_t(X),\Phi_s(X))dsdt. \tag*{(39)}

where Kˉ\bar{K} is the centered kernel in Definition A.1. We have

Eμ[∥L∗(μ,T)−μ^∥H2]=1T2∫0T∫0TCKˉ(∣t−s∣)dsdt.(40)\mathbb{E}_{\mu}\left[\left\|L^*(\mu,T)-\hat{\mu}\right\|_H^2\right]=\frac{1}{T^2}\int_0^T\int_0^T C_{\bar{K}}(|t-s|)dsdt. \tag*{(40)}

by integrating the right side of (39) with respect to μ\mu over MM and using the ff-invariance of μ\mu. Taking the change of variables τ=t−s\tau=t-s, we get

Eμ[∥L∗(μ,T)−μ^∥H2]=2T∫0T(1−τT)CKˉ(τ)dτ≤2T∫0T∣CKˉ(τ)∣dτ.(41)\mathbb{E}_{\mu}\left[\left\|L^*(\mu,T)-\hat{\mu}\right\|_H^2\right]=\frac{2}{T}\int_0^T\left(1-\frac{\tau}{T}\right)C_{\bar{K}}(\tau)d\tau\le\frac{2}{T}\int_0^T\left|C_{\bar{K}}(\tau)\right|d\tau. \tag*{(41)}

Using Markov’s inequality in conjunction with the regularity condition, we get that

P(∥L∗(μ,T)−μ^∥H≥δ)≤σK2δ2T.(42)\mathbb{P}\left(\left\|L^*(\mu,T)-\hat{\mu}\right\|_H\ge\delta\right)\le\frac{\sigma_K^2}{\delta^2T}. \tag*{(42)}

Set η:=σK2δ2T\eta:=\frac{\sigma_K^2}{\delta^2T}. Then, with probability at least 1−η1-\eta, we have

∥L∗(μ,T)−μ^∥H≤σKηT,(43)\left\|L^*(\mu,T)-\hat{\mu}\right\|_H\le\frac{\sigma_K}{\sqrt{\eta T}}, \tag*{(43)}

giving us a convergence rate of OP(T−1/2)O_{\mathbb{P}}(T^{-1/2}).

Suppose that the system ff is exponentially mixing, meaning that correlations decay exponentially [39]. Then the auto-correlation function of the centered kernel will be integrable for kernels which are α\alpha-Hölder continuous, for α∈(0,1]\alpha\in(0,1]. This holds since, in this case, ∣CKˉ(t)∣≤Ce−γt∥K∥Cα\left|C_{\bar{K}}(t)\right|\le Ce^{-\gamma t}\|K\|_{C^\alpha} [7], and so 2∫0∞∣CKˉ(τ)∣dτ<∞2\int_0^\infty\left|C_{\bar{K}}(\tau)\right|d\tau<\infty. Standard kernels such as Gaussian, Laplace, Matérn, and polynomial satisfy this condition.

Setting. Let ff be a C1C^1 vector field on Rd\mathbb{R}^d with flow Φt\Phi_t, and let M⊂RdM\subset\mathbb{R}^d be compact and forward invariant, i.e. Φt(M)⊂M\Phi_t(M)\subset M for all t≥0t\ge0 (e.g. a trapping region containing the attractor). Let μ\mu be a Φ\Phi-invariant Borel probability measure on MM, let KK be a continuous positive-definite kernel on MM with RKHS HH, and write Kx:=K(⋅,x)K_x:=K(\cdot,x) and μ^:=∫MKp dμ(p)\hat{\mu}:=\int_M K_p\,d\mu(p). For x∈Mx\in M and T>0T>0 define

LT(x):=1T∫0TKΦt(x) dt∈H.(44)L_T(x):=\frac{1}{T}\int_0^T K_{\Phi_t(x)}\,dt\in H. \tag*{(44)}

Since Φ\Phi and x↦Kxx\mapsto K_x are continuous, x↦LT(x)x\mapsto L_T(x) is continuous from MM to HH; in particular it is Borel measurable.

Hypotheses.

(K) Hölder feature map. There exist LK>0L_K>0 and κ∈(0,1]\kappa\in(0,1] such that ∥Kx−Ky∥H≤LK∣x−y∣κ\|K_x-K_y\|_H\le L_K|x-y|^\kappa for all x,y∈Mx,y\in M.

(L) Rate under the invariant measure. There exist c≥0c\ge0 and T0>0T_0>0 such that, for X~∼μ\widetilde{X}\sim\mu,

E[∥LT(X~)−μ^∥H2]≤cTfor all T≥T0.(45)\mathbb{E}\left[\left\|L_T(\widetilde{X})-\hat{\mu}\right\|_H^2\right]\le\frac{c}{T}\qquad\text{for all }T\ge T_0. \tag*{(45)}

Assumption A.1 (Asymptotic phase with absolutely continuous phase distribution). Let X∼νX\sim\nu, where ν\nu is a Borel probability measure on MM. There exist a random variable YY with values in MM, defined on a common probability space with XX, a constant λ>0\lambda>0, and a random variable C(X,Y)C(X,Y) that is finite almost surely, such that

(A1) ∣Φt(X)−Φt(Y)∣≤C(X,Y)e−λt|\Phi_t(X)-\Phi_t(Y)|\le C(X,Y)e^{-\lambda t} for all t≥0t\ge0, almost surely;

(A2) the law of YY is absolutely continuous with respect to μ\mu.

Assumption A.1 says that the orbit of a typical initial condition is exponentially asymptotic to the orbit of a point YY on the attractor (an asymptotic phase of XX), and that the law of YY is not concentrated on a μ\mu-null set. For hyperbolic attractors this is the mechanism by which SRB measures are physical.

Theorem A.2. Assume (K), (L), and Assumption A.1. Then

∥1T∫0TK(⋅,Φt(X)) dt−μ^∥H=OP(T−1/2).(46)\left\|\frac{1}{T}\int_0^T K(\cdot,\Phi_t(X))\,dt-\hat{\mu}\right\|_H=O_{\mathbb{P}}\left(T^{-1/2}\right). \tag*{(46)}

More precisely, for every ϵ>0\epsilon>0 there exists Dϵ<∞D_\epsilon<\infty such that

P(∥LT(X)−μ^∥H>DϵT−1/2)≤ϵfor all T≥max⁡(1,T0).(47)\mathbb{P}\left(\left\|L_T(X)-\hat{\mu}\right\|_H>D_\epsilon T^{-1/2}\right)\le\epsilon\qquad\text{for all }T\ge\max(1,T_0). \tag*{(47)}

Proof. By the triangle inequality,

∥LT(X)−μ^∥H≤∥LT(X)−LT(Y)∥H+∥LT(Y)−μ^∥H(48)\lVert L_T(X)-\hat{\mu}\rVert_H \le\lVert L_T(X)-L_T(Y)\rVert_H+\lVert L_T(Y)-\hat{\mu}\rVert_H \tag*{(48)}

Term (I). By (K) and (A1), almost surely,

(I)≤1T∫0T∥KΦt(X)−KΦt(Y)∥H dt≤LKC(X,Y)κT∫0Te−κλt dt≤LKC(X,Y)κκλT.(49)\begin{aligned} \mathrm{(I)} &\le\frac{1}{T}\int_0^T\lVert K_{\Phi_t(X)}-K_{\Phi_t(Y)}\rVert_H\,dt \le\frac{L_K C(X,Y)^\kappa}{T}\int_0^T e^{-\kappa\lambda t}\,dt\\ &\le\frac{L_K C(X,Y)^\kappa}{\kappa\lambda T}. \tag*{(49)} \end{aligned}

Term (II). Let ρ:=dLaw⁡(Y)/dμ\rho:= \mathrm{d}\operatorname{Law}(Y)/\mathrm{d}\mu, which exists by (A2) and is finite μ\mu-almost everywhere. For δ>0\delta>0 let Aδ:={y∈M:∥LT(y)−μ^∥H>δ}A_\delta:= \{y\in M:\lVert L_T(y)-\hat{\mu}\rVert_H>\delta\}, a Borel set. By Chebyshev’s inequality and (L), μ(Aδ)≤c/(δ2T)\mu(A_\delta)\le c/(\delta^2T) for T≥T0T\ge T_0. Hence, for any B>0B>0,

P((II)>δ)=∫Aδρ dμ≤∫{ρ>B}ρ dμ+Bμ(Aδ)≤P(ρ(Y)>B)+Bcδ2T.(50)\mathbb{P}(\mathrm{(II)}>\delta)=\int_{A_\delta}\rho\,\mathrm{d}\mu\le\int_{\{\rho>B\}}\rho\,\mathrm{d}\mu+B\mu(A_\delta)\le\mathbb{P}(\rho(Y)>B)+\frac{Bc}{\delta^2T}. \tag*{(50)}

Conclusion. Fix ε>0\varepsilon>0. Since ρ(Y)<∞\rho(Y)<\infty and C(X,Y)<∞C(X,Y)<\infty almost surely, choose BB with P(ρ(Y)>B)≤ε/3\mathbb{P}(\rho(Y)>B)\le\varepsilon/3 and bb with P(C(X,Y)>b)≤ε/3\mathbb{P}(C(X,Y)>b)\le\varepsilon/3. Set a:=3Bc/εa:=\sqrt{3Bc/\varepsilon}. Taking δ=aT−1/2\delta=aT^{-1/2} above gives P((II)>aT−1/2)≤2ε/3\mathbb{P}(\mathrm{(II)}>aT^{-1/2})\le2\varepsilon/3. On the event {C(X,Y)≤b}\{C(X,Y)\le b\} and for T≥1T\ge1, term (I) is at most LKbκ/(κλT)≤LKbκ/(κλT)L_Kb^\kappa/(\kappa\lambda T)\le L_Kb^\kappa/(\kappa\lambda\sqrt{T}). With Dε:=a+LKbκ/(κλ)D_\varepsilon:=a+L_Kb^\kappa/(\kappa\lambda), a union bound gives P(∥LT(X)−μ^∥H>DεT−1/2)≤ε\mathbb{P}(\lVert L_T(X)-\hat{\mu}\rVert_H>D_\varepsilon T^{-1/2})\le\varepsilon for all T≥max⁡(1,T0)T\ge\max(1,T_0).

Remark A.1 (Sufficient conditions for (L)). The proof of Lemma A.2 establishes Eμ∥LT(X~)−μ^∥H2≤σK2/T\mathbb{E}_\mu\lVert L_T(\widetilde{X})-\hat{\mu}\rVert_H^2\le\sigma_K^2/T, so (L) holds with c=σK2c=\sigma_K^2 whenever σK2<∞\sigma_K^2<\infty. A weaker sufficient condition is that the primitive S(τ):=∫0τCK(s) dsS(\tau):=\int_0^\tau C_K(s)\,\mathrm{d}s is bounded, sup⁡τ≥0∣S(τ)∣≤Sˉ\sup_{\tau\ge0}|S(\tau)|\le\bar{S}: integrating by parts in the proof of Lemma A.2,

Eμ∥LT(X~)−μ^∥H2=2T∫0T(1−τT)CK(τ) dτ=2T2∫0TS(τ) dτ≤2SˉT.(51)\mathbb{E}_\mu\lVert L_T(\widetilde{X})-\hat{\mu}\rVert_H^2=\frac{2}{T}\int_0^T\left(1-\frac{\tau}{T}\right)C_K(\tau)\,\mathrm{d}\tau=\frac{2}{T^2}\int_0^T S(\tau)\,\mathrm{d}\tau\le\frac{2\bar{S}}{T}. \tag*{(51)}

This weaker condition covers periodic attractors, for which CKC_K is periodic and σK2=∞\sigma_K^2=\infty.

Verification for the kernels and systems of Section 3.1

Kernels. All kernels of Appendix C satisfy (K). This follows from ∥Kx−Ky∥H2=K(x,x)+K(y,y)−2K(x,y)\lVert K_x-K_y\rVert_H^2=K(x,x)+K(y,y)-2K(x,y) together with 1−e−u≤u1-e^{-u}\le u for u≥0u\ge0, and, for the quadratic kernel, from its explicit feature map x↦(1,2x,xx⊤)x\mapsto(1,\sqrt{2}x,xx^\top). With R:=max⁡x∈M∣x∣R:=\max_{x\in M}|x|:

  • GU with bandwidth η\eta: LK=1/ηL_K=1/\eta, κ=1\kappa=1;

  • GM with bandwidths σi\sigma_i: LK=1/min⁡iσiL_K=1/\min_i\sigma_i, κ=1\kappa=1;

  • Laplace with bandwidths σi\sigma_i: LK=(2d/min⁡iσi)1/2L_K=(2\sqrt{d}/\min_i\sigma_i)^{1/2}, κ=1/2\kappa=1/2;

  • quadratic: LK=2+2RL_K=\sqrt{2}+2R, κ=1\kappa=1.

The bandwidths are computed once from the data and then held fixed; the results apply conditionally on them.

FitzHugh–Nagumo. For both parameter sets the unique equilibrium pp is a hyperbolic sink (Jacobian eigenvalues −0.104±0.28i-0.104\pm0.28i and {−3.46,−0.089}\{-3.46,-0.089\}), and numerically all initial conditions in [−2,2]2[-2,2]^2 converge to it. Hence μ=δp\mu=\delta_p and LT(p)=μ^L_T(p)=\hat{\mu}, so (L) holds with c=0c=0. Convergence to pp is exponential and uniform on compact subsets of the basin, so Assumption A.1 holds with Y:=pY:=p.

Van der Pol. For μ>0\mu>0 there is a unique hyperbolic attracting limit cycle Γ\Gamma, of period PP, attracting every point except the origin. For y∈Γy\in\Gamma, each full period contributes exactly μ^P\hat{\mu}P, so ∥LT(y)−μ^∥H≤2Pκˉ/T\lVert L_T(y)-\hat{\mu}\rVert_H\le2P\bar{\kappa}/T with κˉ:=sup⁡x∥Kx∥H\bar{\kappa}:=\sup_x\lVert K_x\rVert_H, and (L) holds with c=4P2κˉ2c=4P^2\bar{\kappa}^2. (Here CKC_K is periodic, so σK2=∞\sigma_K^2=\infty and Lemma A.2 does not apply as stated; cf. Remark A.1.) Take YY to be the asymptotic phase of XX on Γ\Gamma: (A1) holds by hyperbolicity of Γ\Gamma, and since the asymptotic-phase map is a C1C^1 submersion on the basin [11], (A2) holds when ν\nu has a density.

Approximating the dissimilarity measure

We use integral quadratures to approximate the integrals in the dissimilarity measure in 9. Assuming mi=mj=mm_i=m_j=m,

d^2(yi,yj)=1Ti2∑k=0m∑l=0mwikwilK(yi(tik),yi(til))+1Tj2∑k=0m∑l=0mwjkwjlK(yj(tjk),yj(tjl))−2TiTj∑k=0m∑l=0mwikwjlK(yi(tik),yj(tjl)),(52)\hat{d}^{2}(y_i,y_j)=\frac{1}{T_i^{2}}\sum_{k=0}^{m}\sum_{l=0}^{m}w_{ik}w_{il}K(y_i(t_{ik}),y_i(t_{il}))+\frac{1}{T_j^{2}}\sum_{k=0}^{m}\sum_{l=0}^{m}w_{jk}w_{jl}K(y_j(t_{jk}),y_j(t_{jl}))-\frac{2}{T_iT_j}\sum_{k=0}^{m}\sum_{l=0}^{m}w_{ik}w_{jl}K(y_i(t_{ik}),y_j(t_{jl})), \tag*{(52)}

The trapezoidal quadrature weights wikw_{ik} for the i-th time series (and similarly for the j-th one) are defined as:

wik={ti,1−ti,02if k=0,ti,k+1−ti,k−12if 0<k<m,ti,m−ti,m−12if k=m.(53)w_{ik}= \begin{cases} \frac{t_{i,1}-t_{i,0}}{2} & \text{if } k=0,\\ \frac{t_{i,k+1}-t_{i,k-1}}{2} & \text{if } 0<k<m,\\ \frac{t_{i,m}-t_{i,m-1}}{2} & \text{if } k=m. \end{cases} \tag*{(53)}

If the time series is sampled uniformly such that Δti=Tim\Delta t_i=\frac{T_i}{m}, the weights simplify to:

wik={Δti2if k=0 or k=m,Δtiif 0<k<m.(54)w_{ik}= \begin{cases} \frac{\Delta t_i}{2} & \text{if } k=0\text{ or }k=m,\\ \Delta t_i & \text{if } 0<k<m. \end{cases} \tag*{(54)}

Kernels used

We used several different kernels in the numerical experiments in § 3. We list their definitions here.

  1. Gaussian uniscale (GU):

K(x,y)=exp⁡(−12η2∥x−y∥2),(55)K(x,y)=\exp\left(-\frac{1}{2\eta^2}\lVert x-y\rVert^2\right), \tag*{(55)}

where we set η:=median⁡{∥xk−xl∥}k≠l\eta:=\operatorname{median}\{\lVert x_k-x_l\rVert\}_{k\ne l} where xkx_k ranged over each individual observation in the dataset. This kernel was used for high-dimensional data.

  1. Gaussian multiscale (GM):

K(x,y)=exp⁡(−∑i=1d(xi−yi)22σi2),(56)K(x,y)=\exp\left(-\sum_{i=1}^{d}\frac{(x_i-y_i)^2}{2\sigma_i^2}\right), \tag*{(56)}

where σi\sigma_i was set to be the standard deviation of the ithi^{\text{th}} coordinate of the dataset. This is the default kernel for CLOCK.

  1. Laplace:

K(x,y)=exp⁡(−∑i=1d∣xi−yi∣σi),(57)K(x,y)=\exp\left(-\sum_{i=1}^{d}\frac{\lvert x_i-y_i\rvert}{\sigma_i}\right), \tag*{(57)}

where σi\sigma_i was set to be the standard deviation of the ithi^{\text{th}} coordinate of the dataset.

  1. Quadratic (quad):

K(x,y)=(xTy+1)2.(58)K(x,y)=(x^{T}y+1)^2. \tag*{(58)}

Formulas for evaluation metrics

We provide explicit formulas for the evaluation metrics in § 3.3.

  1. Adjusted Rand Index (ARI): The formula is given by

ARI⁡=∑i,j(nij2)−∑i(ai2)∑j(bj2)(n2)12(∑i(ai2)+∑j(bj2))−∑i(ai2)∑j(bj2)(n2)(59)\operatorname{ARI} = \frac{\sum_{i,j} \binom{n_{ij}}{2} - \frac{\sum_i \binom{a_i}{2}\sum_j \binom{b_j}{2}}{\binom{n}{2}}} {\frac{1}{2}\left(\sum_i \binom{a_i}{2}+\sum_j \binom{b_j}{2}\right)-\frac{\sum_i \binom{a_i}{2}\sum_j \binom{b_j}{2}}{\binom{n}{2}}} \tag*{(59)}

where nn is the total number of time series in the dataset, nijn_{ij} is the number of trajectories that are in both the true cluster ii and the predicted cluster jj, aia_i is the total number of samples in true cluster ii, and bjb_j is the total number of samples in predicted cluster jj.

  1. Normalized Mutual Information (NMI): The formula is given by

NMI⁡=2∑i,jnijlog⁡(n⋅nijaibj)−∑iailog⁡(ain)−∑jbjlog⁡(bjn)(60)\operatorname{NMI} = \frac{2\sum_{i,j} n_{ij}\log\left(\frac{n\cdot n_{ij}}{a_i b_j}\right)} {-\sum_i a_i\log\left(\frac{a_i}{n}\right)-\sum_j b_j\log\left(\frac{b_j}{n}\right)} \tag*{(60)}

where nn, nijn_{ij}, aia_i, bjb_j are as above and by convention 0log⁡00\log0 is set to 0.

  1. Purity: If Ω\Omega is a predicted clustering and CC a true clustering, then the purity is defined as

purity⁡(Ω,C)=∑ω∈Ωmax⁡c∈C∣ω∩c∣.(61)\operatorname{purity}(\Omega, C)=\sum_{\omega\in\Omega}\max_{c\in C}|\omega\cap c|. \tag*{(61)}

In the setting of two predicted clusters and two true clusters, the purity is just the accuracy of the clustering.

Hyperparameter tuning

We validated the hyperparameters of the deep learning methods in § 3.4 using an internal clustering metric, the DBCV.

  1. Density-based clustering validation (DBCV): The DBCV [20] is an internal clustering metric, meaning it can be used in the absence of true cluster labels. This metric assigns a high value to partitions with highly-dense clusters with sparse between-cluster regions. It does so without assuming the clusters are globular, and so can measure the quality of clustering for highly-irregular collections of trajectories, which occurs frequently with chaotic systems, for example. The DBCV score is normalized to be in the range [−1,1][-1, 1] with 1 being a perfect clustering and -1 being exceptionally poor. We found it to be superior to the silhouette score [31] for the task of validating hyperparameters.

Noise robustness experiment

We speculate that the CLOCK method is highly resistant to zero-mean Gaussian noise since the feature vector is obtained by performing an integration along a trajectory. The integral effectively cancels out the perturbations caused by the noise. In order to gauge the impact of noise on the performance of CLOCK, we performed an experiment in which we gradually increased the noise level in the Lorenz96 dataset while leaving the true trajectories fixed. The CLOCK method with Gaussian uniscale and quadratic kernels (see Appendix § 9) was fit on each dataset, and we evaluated the quality of the clusterings. We ran this experiment with the baseline method TS2Vec to use as a comparison. The results are graphically represented in Figure 3.

Figure 3. Plots of evaluation metrics in the noise robustness test of Appendix § 12. The horizontal axis represents the standard deviation of the Gaussian noise added to each observation. The vertical axis represents one of the evaluation metrics: ARI, NMI or purity. In Figure 3a, we have ARI results, in Figure 3b, we have results for NMI, and in Figure 3c, we have purity results.

The results of the noise robustness experiment show that CLOCK is resistant to noise. It should be noted that the amount of noise added to the trajectories was extreme, even on the lower settings. In Figure 3, we see that CLOCK with Gaussian kernel fails as the noise level approaches 3.0, while CLOCK with quadratic kernel is affected at level 2.0. Thus, the choice of kernel is an important aspect of this clustering method. In practice, the kernel may be treated as a hyperparameter and validated using an internal clustering metric. TS2Vec generally performed worse than CLOCK with either kernel.

Assets used

The assets we used can be divided into methods, back-ends, and datasets.

Methods

  1. Dynamic Time Warping (DTW): We used the dtw function from the Python package tslearn.metrics. This has the BSD-2-Clause license. We used version 0.8.0.

  1. Shape-Based Distance (SBD): We used the package scipy.signal and the function correlate. This has the BSD 3-Clause license. We used version 1.17.1.

  1. Catch-22: We used the package pycatch22 and the function catch22_all. This has the GNU General Public License v3 (GPLv3) license. We used version 0.4.5.

  1. Latent ODE: We used the pytorch package and the torchdiffeq package. The pytorch package has the BSD 3-Clause license. We used version 2.10.0. The torchdiffeq package has the MIT license. We used version 0.2.5.

  1. TS2Vec: Code was downloaded from the GitHub repository at https://github.com/zhihanyue/ts2vec. This has the MIT license. We used the AAAI 2022 Official Release.

  1. Spectral clustering: We used sklearn.cluster package and the function SpectralClustering. This has the BSD 3-Clause license. We used version 1.8.0.

Back-ends

  1. JAX: We used the jax and jaxlib Python packages. These have the Apache 2.0 license. We used version 4.30.0 of jax and version 4.30.0 of jaxlib.

Datasets

  1. Bonn EEG dataset: Taken from the archive available at https://www.upf.edu/web/ntsa/downloads/-/asset_publisher/xvT6E4pczrBw/content/2001-indications-of-nonlinear-deterministic-and-finite-dimensional-structures-in-time-series-of-brain-electrical-activity-dependence-on-recording-regi. This data is in the public domain.

References

  1. [1]José F Alves. Nonuniformly hyperbolic attractors. Springer, 2020.DOI
  2. [2]Enrique Amigó et al. “A comparison of extrinsic clustering evaluation metrics based on formal constraints”. In: Information Retrieval 12.4 (2009), pp. 461–486.
  3. [3]Ralph G Andrzejak et al. “Indications of nonlinear deterministic and finite-dimensional structures in time series of brain electrical activity: Dependence on recording region and brain state”. In: Physical Review E 64.6 (2001), p. 061907.pubmed.ncbi.nlm.nih.gov/11736210
  4. [4]Donald J Berndt and James Clifford. “Using dynamic time warping to find patterns in time series”. In: Proceedings of the 3rd International Conference on Knowledge Discovery and Data Mining. 1994, pp. 359–370.
  5. [5]Srinjoy Das, Hrushikesh N Maskar, and Alexander Cloninger. “Kernel distance measures for time series, random fields and other structured data”. In: Frontiers in Applied Mathematics and Statistics 7 (2021), p. 787455.arxiv.org/abs/2109.14752
  6. [6]Arthur P Dempster, Nan M Laird, and Donald B Rubin. “Maximum likelihood from incomplete data via the EM algorithm”. In: Journal of the Royal Statistical Society: Series B (Methodological) 39.1 (1977), pp. 1–22.DOI
  7. [7]Dmitry Dolgopyat. “On decay of correlations in Anosov flows”. In: Annals of Mathematics 147.2 (1998), pp. 357–390.
  8. [9]Andrew M Fraser and Harry L Swinney. “Independent coordinates for strange attractors from mutual information”. In: Physical Review A 33.2 (1986), p. 1134.pubmed.ncbi.nlm.nih.gov/9896728
  9. [10]Damien Garreau, Wittawat Jitkrittum, and Motonobu Kanagawa. “Large sample analysis of the median heuristic”. In: arXiv preprint arXiv:1707.07269 (2017).arxiv.org/abs/1707.07269
  10. [11]John Guckenheimer. “Isochrons and phaseless sets”. In: Journal of Mathematical Biology 1.3 (1975), pp. 259–273. DOI: 10.1007/BF01273747.DOI
  11. [12]Matthew B Kennel, Reggie Brown, and Henry DI Abarbanel. “Determining embedding dimension for phase-space reconstruction using a geometrical construction”. In: Physical Review A 45.6 (1992), p. 3403.pubmed.ncbi.nlm.nih.gov/9907388
  12. [13]Yoshiki Kuramoto. “Self-entrainment of a population of coupled non-linear oscillators”. In: International Symposium on Mathematical Problems in Theoretical Physics: January 23–29, 1975, Kyoto University, Kyoto, Japan. Springer. 2005, pp. 420–422.
  13. [14]Edward N Lorenz. “Deterministic nonperiodic flow”. In: Journal of Atmospheric Sciences 20.2 (1963), pp. 130–141.
  14. [15]Edward N Lorenz. “Predictability: A problem partly solved”. In: Proc. Seminar on Predictability. Vol. 1. Reading. 1996, pp. 1–18.
  15. [16]Carl H Lubba et al. “catch22: CAnoniCal Time-series Characteristics: Selected through highly comparative time-series analysis”. In: Data Mining and Knowledge Discovery 33.6 (2019), pp. 1821–1852.
  16. [17]Pankaj Malhotra et al. “TimeNet: Pre-trained deep recurrent neural network for time series classification”. In: arXiv preprint arXiv:1706.08838 (2017).arxiv.org/abs/1706.08838
  17. [18]Aaron F McDaid, Derek Greene, and Neil Hurley. “Normalized mutual information to evaluate overlapping community finding algorithms”. In: arXiv preprint arXiv:1110.2515 (2011).arxiv.org/abs/1110.2515
  18. [19]James B McQueen. “Some methods of classification and analysis of multivariate observations”. In: Proc. of 5th Berkeley Symposium on Math. Stat. and Prob. 1967, pp. 281–297.
  19. [20]Davoud Moulavi et al. “Density-Based Clustering Validation”. In: Proceedings of the 2014 SIAM International Conference on Data Mining (SDM). 2014, pp. 839–847. DOI: 10.1137/1.9781611973440.96.DOI
  20. [21]Jinichi Nagumo, Suguru Arimoto, and Shuji Yoshizawa. “An active pulse transmission line simulating nerve axon”. In: Proceedings of the IRE 50.10 (2007), pp. 2061–2070.
  21. [22]Aileen Nielsen. Practical time series analysis: Prediction with statistics and machine learning. O’Reilly Media, 2019.
  22. [23]Jacob Palis. “A global view of dynamics and a conjecture on the denseness of finitude of attractors”. In: Astérisque 261.xxiiixiv (2000), pp. 335–347.
  23. [24]John Paparrizos and Luis Gravano. “k-Shape: Efficient and Accurate Clustering of Time Series”. In: SIGMOD Rec. 45.1 (June 2016), pp. 69–76. ISSN: 0163-5808. DOI: 10.1145/2949741.2949758. URL: https://doi-org.proxy.lib.pdx.edu/10.1145/2949741.2949758.DOI
  24. [25]François Petitjean, Alain Ketterlin, and Pierre Gançarski. “A global averaging method for dynamic time warping, with applications to clustering”. In: Pattern Recognition 44.3 (2011), pp. 678–693.DOI
  25. [26]Ali Rahimi and Benjamin Recht. “Random Features for Large-Scale Kernel Machines”. In: Advances in Neural Information Processing Systems. Vol. 20. Curran Associates, Inc., 2007, pp. 1177–1184.
  26. [27]Victor William Rielly et al. “MOCK: an Algorithm for Learning Nonparametric Differential Equations via Multivariate Occupation Kernel Functions”. In: Transactions on Machine Learning Research (2025). ISSN: 2835-8856. URL: https://openreview.net/forum?id=fjVIp2Z9RS.
  27. [29]Yulia Rubanova, Ricky TQ Chen, and David K Duvenaud. “Latent ordinary differential equations for irregularly-sampled time series”. In: Advances in Neural Information Processing Systems 32 (2019).
  28. [30]H. Sakoe and S. Chiba. “Dynamic programming algorithm optimization for spoken word recognition”. In: IEEE Transactions on Acoustics, Speech, and Signal Processing 26.1 (1978), pp. 43–49. DOI: 10.1109/TASSP.1978.1163055.DOI
  29. [31]Ketan Rajshekhar Shahapure and Charles Nicholas. “Cluster quality analysis using silhouette score”. In: 2020 IEEE 7th International Conference on Data Science and Advanced Analytics (DSAA). IEEE. 2020, pp. 747–748.
  30. [32]Bharath K Sriperumbudur, Kenji Fukumizu, and Gert RG Lanckriet. “Universality, Characteristic Kernels and RKHS Embedding of Measures”. In: Journal of Machine Learning Research 12.7 (2011).arxiv.org/abs/1003.0887
  31. [33]Nitish Srivastava, Elman Mansimov, and Ruslan Salakhudinov. “Unsupervised learning of video representations using lstms”. In: International Conference on Machine Learning. PMLR. 2015, pp. 843–852.arxiv.org/abs/1502.04681
  32. [34]Douglas Steinley. “Properties of the hubert-arable adjusted rand index”. In: Psychological Methods 9.3 (2004), p. 386.DOI
  33. [35]Floris Takens. “Detecting strange attractors in turbulence”. In: Dynamical Systems and Turbulence, Warwick 1980: Proceedings of a Symposium held at the University of Warwick 1979/80. Springer. 2006, pp. 366–381.
  34. [36]Ryohei Umatani et al. “Time series clustering with an EM algorithm for mixtures of linear Gaussian state space models”. In: Pattern Recognition 138 (2023), p. 109375. ISSN: 0031- 3203. DOI: https://doi.org/10.1016/j.patcog.2023.109375. URL: https://www.sciencedirect.com/science/article/pii/S0031320323000766.
  35. [37]Balth Van der Pol. “LXXXVIII. On “relaxation-oscillations””. In: The London, Edinburgh, and Dublin Philosophical Magazine and Journal of Science 2.11 (1926), pp. 978–992.DOI
  36. [38]Lai-Sang Young. “Recurrence times and rates of mixing”. In: Israel Journal of Mathematics 110.1 (1999), pp. 153–188.DOI
  37. [39]Lai-Sang Young. “Statistical properties of dynamical systems with some hyperbolicity”. In: Annals of Mathematics 147.3 (1998), pp. 585–650.DOI
  38. [40]Zhihan Yue et al. “Ts2vec: Towards universal representation of time series”. In: Proceedings of the AAAI Conference on Artificial Intelligence. Vol. 36. 8. 2022, pp. 8980–8987.arxiv.org/abs/2106.10466

Paper details

vlnc:W_wpz52h8r
Version
v1 · latest
Published
Fresh — added to the Hub within the last 30 days
Added
License
CC BY 4.0
Usage

views · 0

downloads · 0

A view counts once per browser per day, a download each time the PDF is requested.

  • Computer Science

Gemini 3.1 Pro (bare)

View statement
Looking for related papers…
Contents