Introduction
Hidden Markov models (HMMs) serve as a standard approach to modeling sequential data with underlying state changes and temporal dependence [9 , 52 , 21 , 12 , 65 ] . In this framework, the hidden states evolve according to a finite-state Markov chain, and each observation is generated from the emission distribution associated with its current state. Owing to these modeling capabilities, HMMs and closely related Markov-switching models have been employed extensively in speech recognition [52 , 24 ] , genomics [22 , 30 , 45 ] , single-molecule biophysics [47 , 11 , 59 ] , ecological modeling [36 , 46 ] , neuroscience [6 , 61 ] , and a broad range of time-series applications [23 , 65 ] . In many of these applications, the central goal is to interpret the underlying process through fitted states and transitions. However, a close fit to the observations alone is insufficient for reliable interpretation unless the fitted emission distributions, state probabilities, and transition flows also accurately reflect their true counterparts. For Gaussian emissions with a common, known covariance, this difficulty becomes especially pronounced when emission centers approach one another. Specifically, models with substantially different state probabilities or transition matrices can produce nearly identical observation densities.
This gap between observation fit and latent recovery calls for deriving convergence rates that explicitly account for emission separation. To obtain these rates, we establish inverse inequalities that bound latent recovery error in terms of observation-density error and emission separation. Formally, let θ \theta θ denote a candidate HMM parameter, θ ⋆ \theta^\star θ ⋆ the true parameter, and p θ , T p_{\theta,T} p θ , T the density of a length-T T T observation sequence under θ \theta θ . Our inverse inequalities take the following form
D r ( θ , θ ⋆ ) ≤ C r Δ s e p − α r h ( p θ , T , p θ ⋆ , T ) ν r , (1) \mathcal{D}_r(\theta,\theta^\star) \le C_r \Delta_{\mathrm{sep}}^{-\alpha_r} h(p_{\theta,T},p_{\theta^\star,T})^{\nu_r}, \tag*{(1)} D r ( θ , θ ⋆ ) ≤ C r Δ sep − α r h ( p θ , T , p θ ⋆ , T ) ν r , (1)
where h h h denotes the Hellinger distance defined in equation (14) , Δ s e p \Delta_{\mathrm{sep}} Δ sep is the truncated minimum separation between true emission centers, as defined in § 2.1 , and D r \mathcal{D}_r D r measures error in a latent target involving r r r time coordinates. The constant C r C_r C r is independent of Δ s e p \Delta_{\mathrm{sep}} Δ sep , while the exponents α r \alpha_r α r and ν r \nu_r ν r quantify the dependence of the recovery bound on emission separation and observation-density error, respectively. Once the Hellinger error of an estimator is controlled, these inequalities translate its density estimation rate into latent recovery rates. The recovery targets considered in this paper are model parameters and unconditional distributions of latent emission-center sequences, rather than the realized hidden path or its posterior distribution given the observations.
The separation exponent α r \alpha_r α r in (1) also depends on how many true emission centers approach one another, which is not captured by their minimum separation alone. We illustrate this dependence in Figure 1 , where four emission centers form either two separated pairs or a single cluster, with the same minimum separation in both configurations. In the first configuration, the four centers form two pairs of nearby centers that remain well separated from each other, whereas in the second, all four centers lie close together in a single cluster. This difference changes the separation factor in the exact-specification emission-center bound from Δ s e p − 2 \Delta_{\mathrm{sep}}^{-2} Δ sep − 2 to Δ s e p − 6 \Delta_{\mathrm{sep}}^{-6} Δ sep − 6 , as established in Theorem 3.1 . Driven by this example, we analyze how the number of nearby centers, in addition to their minimum separation, determines the sensitivity of latent recovery to observation error.
Vector (PDF) D μ ≤ C ( π min ∗ ) Δ s e p − ( 2 s max − 2 ) h ( p θ , T , p θ ∗ , T ) D_\mu\leq C(\pi_{\min}^{*})\Delta_{\mathrm{sep}}^{-(2s_{\max}-2)}h(p_\theta,T,p_{\theta^*},T) D μ ≤ C ( π m i n ∗ ) Δ sep − ( 2 s m a x − 2 ) h ( p θ , T , p θ ∗ , T ) (a) Two separated pairs: K = 4 K=4 K = 4 , s max = 2 s_{\max}=2 s m a x = 2 , D μ ≲ Δ s e p − 2 h D_\mu\lesssim\Delta_{\mathrm{sep}}^{-2}h D μ ≲ Δ sep − 2 h \hfill (b) One cluster of four: K = 4 K=4 K = 4 , s max = 4 s_{\max}=4 s m a x = 4 , D μ ≲ Δ s e p − 6 h D_\mu\lesssim\Delta_{\mathrm{sep}}^{-6}h D μ ≲ Δ sep − 6 h
Figure 1. Same state count and minimum separation, different separation exponents. Both configurations have K = 4 K=4 K = 4 Gaussian emission distributions and the same minimum separation Δ s e p \Delta_{\mathrm{sep}} Δ sep . Colored dots mark emission centers, and dashed outlines group centers in the same cluster. Configuration (a) consists of two well-separated pairs with s max = 2 s_{\max}=2 s m a x = 2 , whereas configuration (b) consists of one cluster of four with s max = 4 s_{\max}=4 s m a x = 4 . Theorem 3.1 gives separation factors Δ s e p − 2 \Delta_{\mathrm{sep}}^{-2} Δ sep − 2 and Δ s e p − 6 \Delta_{\mathrm{sep}}^{-6} Δ sep − 6 in the global emission-center recovery bounds under exact specification for configurations (a) and (b), respectively.
Latent Recovery in Hidden Markov Models
Latent recovery in HMMs has long been a subject of statistical research, with classical work on likelihood-based estimation and model identification [21 , 12 ] . Over time, this literature has expanded to include finite-sample recovery guarantees, with more recent work examining how emission separation affects estimation rates.
Estimation and identifiability. Likelihood-based inference provides a classical approach to estimating HMM parameters from observed sequences. Consistency and asymptotic normality describe the behavior of these estimators as the sample size increases under suitable assumptions [9 , 42 , 10 , 19 , 20 ] , while Lehéricy [41 ] studies nonasymptotic guarantees for maximum likelihood under model misspecification. In practical applications, likelihood-based estimation also requires algorithms for computing the estimators. The expectation-maximization (EM) algorithm provides an iterative approach to this computation with latent variables [17 ] , and Balakrishnan et al. [7 ] develop guarantees that connect its optimization and statistical errors under suitable initialization and local stability conditions. Apart from computation, the specification of the fitted model requires choosing the number of hidden states. When this number is unknown, state-count estimation is addressed through penalized likelihood and information-theoretic procedures ([25 ] ) and nonparametric order estimators ([40 ] ). From a Bayesian perspective, posterior analyses also account for an unknown state count or flexible emission distributions ([26 , 60 ] ).
Underlying these estimation approaches is the question of whether the latent parameters can be uniquely determined from the observation distribution. Since hidden-state labels are arbitrary, uniqueness is understood up to a common state permutation. General identifiability results for latent-variable models, including HMMs, are developed by Allman et al. [4 ] through algebraic arguments based on conditional independence. More specifically, Gassiat et al. [27 ] establish identifiability for finite-state nonparametric HMMs with a full-rank transition matrix and linearly independent emission distributions. The linear-independence requirement can be relaxed to distinct emissions when the transition matrix is full-rank and ergodic, as shown by Alexandrovich et al. [3 ] . These results clarify when the observation distribution distinguishes different latent models, but do not by themselves quantify how a small error in that distribution affects latent recovery. This quantitative relation is needed for recovery guarantees from finitely many observations, where the observation distribution must itself be estimated.
Finite-sample recovery. Spectral and method-of-moments approaches provide finite-sample guarantees by using observable moments to construct matrix or tensor decompositions of the latent model ([34 , 54 , 35 , 5 ] ). Building on these approaches, subsequent work improves the statistical efficiency of moment-based estimation ([57 ] ) and extends spectral inference to hidden semi-Markov models ([49 ] ). The recovery guarantees for such procedures depend on how accurately the latent quantities can be reconstructed from estimated moments. In the spectral analysis of Hsu et al. [34 ] , this reconstruction is controlled through lower bounds for singular values of the model matrices. These conditions ensure that errors in observable quantities do not become arbitrarily amplified during recovery. The propagation of estimation error can also be studied directly through inverse inequalities that bound latent recovery error in terms of observation-density error. In the nonparametric HMM setting, Castro et al. [13 ] use this approach to derive emission-estimation rates. Specifically, they first establish an oracle inequality for estimating the joint density of three consecutive observations, then show under an additional nondegeneracy condition that the squared emission-recovery error is bounded by the squared joint-density error. This relation converts their joint-density guarantee into minimax-adaptive rates for emission estimation, up to logarithmic factors. However, these rates depend on the lowest regularity among the emission densities, so they do not give faster rates for smoother densities within the same model. Lehéricy [39 ] addresses this limitation through state-specific rates that adapt to the regularity of each emission density. In addition to emission-density estimation, parameter-recovery guarantees also support inference on hidden states conditional on the observations. For conditional inference, De Castro et al. [16 ] control filtering and marginal smoothing errors in terms of parameter-estimation errors and provide bounds for nonparametric spectral estimators. Nevertheless, the finite-sample results discussed above do not explicitly quantify how latent recovery rates depend on emission separation.
Recovery as emissions approach one another. Abraham et al. [1 ] explicitly quantify how parameter-estimation rates depend on emission separation in two-state HMMs with multinomial emissions. More precisely, they establish matching nonasymptotic minimax upper and lower bounds over classes in which the separation between the two emission probability vectors may decrease with the sample size. Consequently, their bounds quantify the additional data needed for recovery as the emissions become harder to distinguish. With nonparametric emissions, recovery additionally requires estimating the emission densities themselves, so their smoothness also affects estimation accuracy. Moreover, temporal dependence can provide information for distinguishing the hidden states, so recovery becomes more difficult as the observations approach an i.i.d. sequence. In this setting, Abraham et al. [2 ] obtain minimax bounds for two-state HMMs that account jointly for emission smoothness and how close the observations are to independence. Despite these differences in emission assumptions, both studies consider two-state HMMs, where separation involves only a single pair of emission distributions. With more states, however, the same minimum separation can arise from one cluster of approaching Gaussian emission centers or several smaller, separated clusters, as illustrated in Figure 1 . Thus, the two-state results leave unresolved how the number of approaching centers within each cluster affects latent recovery rates.
Connections to Finite-Mixture Theory
For Gaussian HMMs with an arbitrary number of hidden states, finite-mixture theory provides a natural starting point for studying latent recovery. Indeed, over a fixed trajectory length, the joint observation density is a finite Gaussian mixture whose mixing measure describes the distribution of latent emission-center sequences. Therefore, recovering this measure amounts to recovering the latent location distributions of interest.
Mixing-distribution estimation. The mixture representation above connects latent recovery in Gaussian HMMs to the identification and estimation of mixing distributions. Both problems have been studied in finite and infinite mixture models [55 , 64 , 37 , 8 , 38 ] , with broader discussions of mixture inference in [56 , 43 , 48 ] . At the population level, identifiability establishes when the observation density uniquely determines the mixing distribution. With finitely many observations, estimation additionally requires quantifying how accurately that distribution can be recovered from data. Likelihood-based methods address this estimation problem, and their computation is reviewed by Redner and Walker [53 ] . However, their statistical behavior can depart from regular parametric theory near singular or nonidentifiable configurations, as studied by Chen [14 ] ; Dacunha-Castelle and Gassiat [15 ] ; Liu and Shao [44 ] . Near such configurations, differences in component locations and probabilities can produce much smaller differences in the observation density. As a result, accurate density estimation need not imply equally accurate recovery of the mixing distribution. On the density-estimation side, Ghosal and van der Vaart [28 ] derive Hellinger convergence rates for maximum-likelihood estimators of Gaussian mixtures through bracketing-entropy bounds and empirical process tools (van de Geer, [58 ] ). These guarantees control the observation-density error, but do not by themselves bound errors in component locations and probabilities. Accordingly, latent recovery requires an inverse bound that controls both types of error in terms of observation-density error.
Wasserstein recovery and separation geometry. Wasserstein distance measures errors in both component locations and probability masses, making it suitable for assessing mixing-distribution recovery (Villani, [62 ] ). As a link between density estimation and mixing-distribution recovery, Nguyen [51 ] establishes quantitative relations between Wasserstein distances on mixing measures and divergences between the corresponding mixture densities for independent observations. Within this framework, the recovery rates depend on the identifiability properties of the mixture family and on whether the fitted model contains excess components. In particular, Ho and Nguyen [32 ] show how strong and weak identifiability lead to different convergence behavior in exactly specified and overfitted finite mixtures. At the local minimax level, Heinrich and Kahn [31 ] further establish how the number of excess components determines the optimal recovery rate near a fixed mixing distribution.
Through moment comparisons, Wasserstein recovery bounds can also be obtained for Gaussian location mixtures. Specifically, Wu and Yang [63 ] derive polynomial interpolation inequalities that convert moment discrepancies into Wasserstein bounds and use them to obtain optimal estimation guarantees. In higher dimensions, Doss et al. [18 ] exploit moment tensors to establish minimax-optimal Wasserstein rates without separation assumptions. These minimax guarantees describe the worst-case difficulty over classes allowing arbitrarily close components, rather than the dependence of recovery accuracy on a particular configuration of component locations. From a complementary perspective, Nguyen et al. [50 ] develop polynomial constructions that make the dependence on component separation and minimum mixture weight explicit in recovery bounds.
However, directly applying generic mixture bounds to latent recovery in Gaussian HMMs poses two distinct difficulties. First, component-count-dependent bounds can make recovery guarantees depend on up to K T K^T K T hidden paths, even though these paths reuse the same K K K emission centers across time. Furthermore, even within this shared set, the total number of centers fails to reflect whether they all approach one another or instead concentrate into smaller, well-separated clusters. Second, unlike the freely varying weights inherent to generic mixtures, HMM path weights are constrained by the Markov factorization into initial-state and transition probabilities. In turn, generic bounds for unrestricted weights leave these structural dependencies unexploited.
Our Contributions
This paper addresses these difficulties through inverse inequalities that exploit the shared emission centers and Markov structure of Gaussian HMMs. These inequalities connect observation-density error to latent recovery error under both exact specification, where the fitted and true models have the same number of states, and over-specification, where the fitted model has more states than the true model. Combined with density-estimation guarantees, they yield statistical recovery rates that quantify how estimation accuracy depends on sample size, emission separation, and cluster size. Our main contributions are summarized as follows.
HMM-aware inverse bounds for latent recovery. We establish deterministic inverse inequalities that bound latent recovery errors in terms of the Hellinger distance between fitted and true observation densities. Our analysis exploits the shared emission centers across time and the Markov factorization of path probabilities, avoiding a treatment of the observation density as an arbitrary K T K^T K T -component mixture. This structure allows us to express the dependence on emission separation through explicit powers, with constants independent of that separation. When the true centers form clusters whose diameters are bounded by a fixed multiple of the minimum separation and whose intercluster distances remain bounded away from zero, these powers depend on the largest cluster size s max s_{\max} s m a x . Without cluster assumptions, the corresponding bounds hold with s max = K s_{\max}=K s m a x = K .
Sharp separation exponents under exact specification and over-specification. We derive separation exponents under both exact specification and over-specification and prove their sharpness through explicit examples, with the powers of Hellinger error held fixed. Under exact specification, our global bounds have separation exponents 2 s max − 1 2s_{\max}-1 2 s m a x − 1 for state-sequence probability recovery and 2 s max − 2 2s_{\max}-2 2 s m a x − 2 for 1-Wasserstein recovery of joint emission-center distributions, neither of which increases with the number of selected times. The constants in these global bounds may depend on the minimum true initial-state probability. When the observation-density error is sufficiently small, we prove that fitted and true emission centers admit a one-to-one matching and obtain inverse bounds with the same separation exponents but constants independent of that probability. Under over-specification, however, several fitted states may represent the same true state, so their individual transition rows need not be identifiable. Therefore, we control probabilities aggregated over these states, with separation exponents that now depend on the number of selected times. We further extend these aggregate probability bounds to grouping based on sufficiently accurate reference centers without changing their separation exponents.
Statistical recovery rates with explicit separation dependence. We combine a nearly parametric Hellinger bound for the global MLE with the inverse inequalities above to obtain latent recovery rates that depend explicitly on emission separation and cluster size. For instance, emission-center displacement, weighted by fitted state probabilities, has rate O P ( Δ s e p − ( 2 s max − 2 ) ( d / n ) 1 / 2 ) \mathcal{O}_{\mathbb{P}}\left(\Delta_{\mathrm{sep}}^{-(2s_{\max}-2)}(d/n)^{1/2}\right) O P ( Δ sep − ( 2 s m a x − 2 ) ( d / n ) 1/2 ) under exact specification and O P ( Δ s e p − ( s max − 1 ) ( d / n ) 1 / 4 ) \mathcal{O}_{\mathbb{P}}\left(\Delta_{\mathrm{sep}}^{-(s_{\max}-1)}(d/n)^{1/4}\right) O P ( Δ sep − ( s m a x − 1 ) ( d / n ) 1/4 ) under over-specification, up to logarithmic factors for fixed trajectory length. In both cases, the separation factors increase as the true emission centers approach one another, so a larger sample size is needed to obtain the same upper bound. At the same minimum emission separation, these recovery rates have smaller inverse powers of separation when fewer emission centers approach one another within each cluster. This results in different statistical recovery guarantees for HMMs with the same state count and minimum emission separation, according to how many emission centers approach one another within the largest cluster.
In this paper, we consider independent trajectories observed over a fixed time horizon. Such settings arise in longitudinal studies and repeated experiments, where each subject or experimental run is observed at a fixed number of time points. In both cases, the amount of data increases through the number of trajectories rather than their length. As more independent trajectories become available, our recovery rates describe how accurately individual-state and aggregate latent quantities can be estimated, providing a statistical basis for interpreting fitted HMMs in applications such as disease progression, human activity recognition, and economic modeling.
Paper Organization. The remainder of the paper is organized as follows. § 2 introduces the model, latent recovery losses, emission separation and clustering conditions, and establishes the density-estimation bound used to obtain statistical recovery rates. With these definitions, we develop the main recovery bounds under exact specification and over-specification in § 3 and § 4 , respectively. We empirically assess our recovery bounds and their corresponding statistical rates through synthetic experiments under exact specification and over-specification in § 5 . § 6 discusses what these guarantees imply for the interpretation and use of fitted HMMs, and § 7 concludes the paper. Proofs, supporting technical lemmas, and additional results are provided in the appendices.
Notation and Preliminaries
In this section, we first introduce the Gaussian HMM and define loss functions for comparing fitted and true emission centers, state probabilities, transition flows, and emission-center distributions. Subsequently, we specify conditions on the separation of true emission centers within and between clusters that determine how observation-density error translates into latent recovery bounds. Finally, we establish a finite-sample Hellinger bound for the global maximum likelihood estimator, which we use in later sections to derive latent recovery rates from the inverse inequalities.
Gaussian Hidden Markov Models
Model parameters. Let K ≥ 2 K \ge2 K ≥ 2 be the number of true hidden states and let k ≥ K k \ge K k ≥ K be the number of states in the fitted model. A Gaussian HMM is parameterized by θ = ( π , A , μ 1 , … , μ k ) \theta= (\pi, A, \mu_1, \ldots, \mu_k) θ = ( π , A , μ 1 , … , μ k ) , where π \pi π is the row vector of initial state probabilities, A A A is the transition matrix, and μ 1 , … , μ k ∈ R d \mu_1, \ldots, \mu_k \in\mathbb{R}^d μ 1 , … , μ k ∈ R d are the emission centers. We write P θ \mathbb{P}_{\theta} P θ for probabilities under the model with parameter θ \theta θ . The latent states form a Markov chain with initial probability π \pi π and transition matrix A A A
Z 1 ∼ π , P θ ( Z t + 1 = j ∣ Z t = i ) = A i j . Z_1 \sim\pi,\qquad\mathbb{P}_{\theta}(Z_{t+1}=j \mid Z_t=i) = A_{ij}. Z 1 ∼ π , P θ ( Z t + 1 = j ∣ Z t = i ) = A ij .
Conditioned on the latent states, the observations are independent and satisfy X t ∣ Z 1 : T ∼ N d ( μ Z t , Σ ) X_t \mid Z_{1:T} \sim\mathcal{N}_d(\mu_{Z_t}, \Sigma) X t ∣ Z 1 : T ∼ N d ( μ Z t , Σ ) , where the common covariance matrix Σ \Sigma Σ is known and positive definite.
Observed data. We observe n n n independent trajectories X 1 : T ( 1 ) , … , X 1 : T ( n ) X_{1:T}^{(1)}, \ldots, X_{1:T}^{(n)} X 1 : T ( 1 ) , … , X 1 : T ( n ) generated by the same Gaussian HMM with true parameter θ ∗ = ( π ∗ , A ∗ , μ 1 ∗ , … , μ K ∗ ) \theta^* = (\pi^*, A^*, \mu_1^*, \ldots, \mu_K^*) θ ∗ = ( π ∗ , A ∗ , μ 1 ∗ , … , μ K ∗ ) . Each trajectory consists of T T T observations in R d \mathbb{R}^d R d with its hidden-state sequence unobserved. Within each trajectory, however, the observations are dependent through the hidden Markov chain. Throughout our statistical analysis, T ≥ 2 T \ge2 T ≥ 2 remains fixed as n n n tends to infinity, so additional data come from more independent trajectories rather than longer sequences.
Normalization of the emission covariance. For simplicity, we reduce the emission covariance to the identity matrix by letting S = Σ − 1 / 2 S = \Sigma^{-1/2} S = Σ − 1/2 and transforming each observation X t X_t X t into S X t SX_t S X t . Accordingly, the transformed emission density is φ d ( x − S μ Z t ) \varphi_d(x - S\mu_{Z_t}) φ d ( x − S μ Z t ) , where φ d \varphi_d φ d denotes the standard Gaussian density on R d \mathbb{R}^d R d . The initial distribution π \pi π and transition matrix A A A remain unchanged. This transformation also preserves the Hellinger distance in equation (1) . For any u ∈ R d u \in\mathbb{R}^d u ∈ R d ,
λ min ( Σ ) ∥ S u ∥ 2 ≤ ∥ u ∥ 2 ≤ λ max ( Σ ) ∥ S u ∥ 2 , \sqrt{\lambda_{\min}(\Sigma)}\lVert Su\rVert_2 \le\lVert u\rVert_2 \le\sqrt{\lambda_{\max}(\Sigma)}\lVert Su\rVert_2, λ m i n ( Σ ) ∥ S u ∥ 2 ≤ ∥ u ∥ 2 ≤ λ m a x ( Σ ) ∥ S u ∥ 2 ,
where λ min ( Σ ) \lambda_{\min}(\Sigma) λ m i n ( Σ ) and λ max ( Σ ) \lambda_{\max}(\Sigma) λ m a x ( Σ ) denote the smallest and largest eigenvalues of Σ \Sigma Σ , respectively. For fixed Σ \Sigma Σ , these inequalities imply that distances between emission centers before and after the transformation differ by at most constant factors depending on Σ \Sigma Σ . Consequently, the recovery bounds established in subsequent sections for the transformed observations and emission centers can be expressed in the original coordinates without changing their orders. We provide the detailed justification of the reduction to identity emission covariance and its effect on the recovery bounds in Appendix § 9 .§ 1 .
Throughout the sequel, we retain the notation X t X_t X t and μ j \mu_j μ j for the transformed observations and centers. All radius constraints, separation and clustering conditions, and nearest-center assignments below are understood in these transformed coordinates.
Observation density. We write p θ , T p_{\theta,T} p θ , T for the density of a length-T T T observation sequence under parameter θ \theta θ . For every positive integer m m m , write [ m ] = { 1 , … , m } [m] = \{1,\ldots,m\} [ m ] = { 1 , … , m } . For any latent state sequence z ∈ [ k ] T z \in[k]^T z ∈ [ k ] T , its probability is
P θ ( Z 1 : T = z ) = π z 1 ∏ t = 1 T − 1 A z t z t + 1 , \mathbb{P}_{\theta}(Z_{1:T}=z) = \pi_{z_1}\prod_{t=1}^{T-1} A_{z_tz_{t+1}}, P θ ( Z 1 : T = z ) = π z 1 t = 1 ∏ T − 1 A z t z t + 1 ,
by the Markov property. Conditional on Z 1 : T = z Z_{1:T}=z Z 1 : T = z , the observations are independent, with joint density
p θ ( x 1 : T ∣ Z 1 : T = z ) = ∏ t = 1 T φ d ( x t − μ z t ) . p_{\theta}(x_{1:T}\mid Z_{1:T}=z) = \prod_{t=1}^{T}\varphi_d(x_t-\mu_{z_t}). p θ ( x 1 : T ∣ Z 1 : T = z ) = t = 1 ∏ T φ d ( x t − μ z t ) .
The unconditional density of the length-T T T observation sequence X 1 : T X_{1:T} X 1 : T is obtained by summing the conditional densities over all latent paths z ∈ [ k ] T z \in[k]^T z ∈ [ k ] T , weighted by their respective probabilities:
p θ , T ( x 1 : T ) = ∑ z ∈ [ k ] T π z 1 ∏ t = 1 T − 1 A z t z t + 1 ∏ t = 1 T φ d ( x t − μ z t ) . (2) p_{\theta,T}(x_{1:T}) = \sum_{z\in[k]^T} \pi_{z_1}\prod_{t=1}^{T-1} A_{z_tz_{t+1}}\prod_{t=1}^{T}\varphi_d(x_t-\mu_{z_t}). \tag*{(2)} p θ , T ( x 1 : T ) = z ∈ [ k ] T ∑ π z 1 t = 1 ∏ T − 1 A z t z t + 1 t = 1 ∏ T φ d ( x t − μ z t ) . (2)
Definition 2.1 (Parameter space and model assumptions). For R ≥ 1 R \ge1 R ≥ 1 , let Θ k ( R ) \Theta_k(R) Θ k ( R ) be the class of parameters θ \theta θ satisfying
π i ≥ 0 , ∑ i = 1 k π i = 1 , A i j ≥ 0 , ∑ j = 1 k A i j = 1 , μ j ∈ B d ( 0 , R ) , \pi_i \ge0,\qquad\sum_{i=1}^{k}\pi_i = 1,\qquad A_{ij} \ge0,\qquad\sum_{j=1}^{k}A_{ij} = 1,\qquad\mu_j \in B_d(0,R), π i ≥ 0 , i = 1 ∑ k π i = 1 , A ij ≥ 0 , j = 1 ∑ k A ij = 1 , μ j ∈ B d ( 0 , R ) , for all i , j ∈ [ k ] i,j \in[k] i , j ∈ [ k ] , where B d ( 0 , R ) B_d(0,R) B d ( 0 , R ) is the closed Euclidean ball of radius R R R centered at the origin. Zero initial and transition probabilities and repeated fitted emission centers are allowed.
The true parameter θ ⋆ ∈ Θ K ( R ) \theta^\star\in\Theta_K(R) θ ⋆ ∈ Θ K ( R ) has strictly positive initial state probabilities and pairwise distinct emission centers. Writing a j : = μ j ⋆ a_j := \mu_j^\star a j := μ j ⋆ , define
π min ⋆ : = min j ∈ [ K ] π j ⋆ > 0 , δ s e p : = min j , ℓ ∈ [ K ] j ≠ ℓ ∥ a j − a ℓ ∥ 2 > 0 , Δ s e p : = 1 ∧ δ s e p . \pi_{\min}^\star:= \min_{j\in[K]}\pi_j^\star> 0,\qquad
\delta_{\mathrm{sep}} := \min_{\substack{j,\ell\in[K]\\j\ne\ell}}\lVert a_j-a_\ell\rVert_2 > 0,\qquad
\Delta_{\mathrm{sep}} := 1\wedge\delta_{\mathrm{sep}}. π m i n ⋆ := j ∈ [ K ] min π j ⋆ > 0 , δ sep := j , ℓ ∈ [ K ] j = ℓ min ∥ a j − a ℓ ∥ 2 > 0 , Δ sep := 1 ∧ δ sep .
The probability and radius conditions make Θ k ( R ) \Theta_k(R) Θ k ( R ) a closed and bounded subset of a finite-dimensional Euclidean space, and hence compact. Since the log-likelihood is continuous on this space for any fixed observed sample, a global maximum exists. The radius bound also ensures that any two emission centers are at most 2 R 2R 2 R apart, which provides a uniform bound on transport costs in later recovery analysis. These properties hold without requiring the transition matrix to be invertible or the hidden chain to be stationary or irreducible. In particular, positive initial state probabilities do not ensure that every state remains occupied at later times. The fitted parameter space also permits states that are never visited, since initial and transition probabilities may be zero. Consequently, the true observation density can be represented in every class Θ k ( R ) \Theta_k(R) Θ k ( R ) with k ≥ K k \ge K k ≥ K by retaining the K K K true states and adding k − K k-K k − K unvisited states.2
Latent Recovery Loss
State labels are arbitrary, and under over-specification several fitted states can represent a single true state. To compare state probabilities between the fitted and the true models, we assign each fitted state to its nearest true emission center and aggregate probabilities according to this assignment. For a fitted parameter θ ∈ Θ k ( R ) \theta\in\Theta_k(R) θ ∈ Θ k ( R ) , define3
c ( i ) : = arg min j ∈ [ K ] ∥ μ i − a j ∥ 2 , V j : = { i ∈ [ k ] : c ( i ) = j } , r i : = ∥ μ i − a c ( i ) ∥ 2 , (3) c(i) := \mathop{\operatorname{arg\,min}}_{j \in[K]} \lVert\mu_i-a_j\rVert_2,\qquad V_j := \{i \in[k] : c(i)=j\},\qquad r_i := \lVert\mu_i-a_{c(i)}\rVert_2, \tag*{(3)} c ( i ) := arg min j ∈ [ K ] ∥ μ i − a j ∥ 2 , V j := { i ∈ [ k ] : c ( i ) = j } , r i := ∥ μ i − a c ( i ) ∥ 2 , (3)
The set V j V_j V j contains the fitted states assigned to the true center a j a_j a j ; and r i r_i r i measures the displacement of the fitted center μ i \mu_i μ i from its assigned true center. This construction allows multiple fitted states to belong to the same group without requiring a one-to-one matching.
State-sequence probability errors. We write q t = π A t − 1 q_t=\pi A^{t-1} q t = π A t − 1 and q t ⋆ = π ⋆ ( A ⋆ ) t − 1 q_t^\star=\pi^\star(A^\star)^{t-1} q t ⋆ = π ⋆ ( A ⋆ ) t − 1 for the row vectors of state probabilities at time t t t under the fitted and true models, respectively. Thus, q t ( i ) = P θ ( Z t = i ) q_t(i)=\mathbb{P}_{\theta}(Z_t=i) q t ( i ) = P θ ( Z t = i ) . To characterize temporal dependence beyond a single time point, consider indices 1 ≤ t 1 < ⋯ < t r ≤ T 1 \le t_1 < \cdots< t_r \le T 1 ≤ t 1 < ⋯ < t r ≤ T and write I = ( t 1 , … , t r ) I=(t_1,\ldots,t_r) I = ( t 1 , … , t r ) . We identify each nonempty subset of [ T ] [T] [ T ] with its increasing enumeration; the full set I = [ T ] I=[T] I = [ T ] is allowed. To compare state-sequence probabilities on the common index set [ K ] r [K]^r [ K ] r , define
P ‾ θ I ( j 1 , … , j r ) : = P θ ( c ( Z t 1 ) = j 1 , … , c ( Z t r ) = j r ) (4) \overline{P}_{\theta}^{I}(j_1,\ldots,j_r) := \mathbb{P}_{\theta}\bigl(c(Z_{t_1})=j_1,\ldots,c(Z_{t_r})=j_r\bigr) \tag*{(4)} P θ I ( j 1 , … , j r ) := P θ ( c ( Z t 1 ) = j 1 , … , c ( Z t r ) = j r ) (4)
and
P ⋆ I ( j 1 , … , j r ) : = P θ ⋆ ( Z t 1 = j 1 , … , Z t r = j r ) . (5) P_{\star}^{I}(j_1,\ldots,j_r) := \mathbb{P}_{\theta^\star}\bigl(Z_{t_1}=j_1,\ldots,Z_{t_r}=j_r\bigr). \tag*{(5)} P ⋆ I ( j 1 , … , j r ) := P θ ⋆ ( Z t 1 = j 1 , … , Z t r = j r ) . (5)
The distribution P ‾ θ I \overline{P}_{\theta}^{I} P θ I combines the probabilities of fitted state sequences assigned to the same sequence of true-state labels. It can therefore be compared with P ⋆ I P_{\star}^{I} P ⋆ I even when k > K k>K k > K . We measure their discrepancy by
D I : = ∥ P ‾ θ I − P ⋆ I ∥ 1 = ∑ ( j 1 , … , j r ) ∈ [ K ] r ∣ P ‾ θ I ( j 1 , … , j r ) − P ⋆ I ( j 1 , … , j r ) ∣ = 2 TV ( P ‾ θ I , P ⋆ I ) . (6) D_I := \lVert\overline{P}_{\theta}^{I}-P_{\star}^{I}\rVert_1 = \sum_{(j_1,\ldots,j_r)\in[K]^r}\left|\overline{P}_{\theta}^{I}(j_1,\ldots,j_r)-P_{\star}^{I}(j_1,\ldots,j_r)\right|=2\,\operatorname{TV}\left(\overline{P}_{\theta}^{I},P_{\star}^{I}\right). \tag*{(6)} D I := ∥ P θ I − P ⋆ I ∥ 1 = ( j 1 , … , j r ) ∈ [ K ] r ∑ P θ I ( j 1 , … , j r ) − P ⋆ I ( j 1 , … , j r ) = 2 TV ( P θ I , P ⋆ I ) . (6)
For a single time point, D { t } D_{\{t\}} D { t } compares the aggregate fitted state probabilities with the true state probabilities. In particular, D π : = D { 1 } D_{\pi}:=D_{\{1\}} D π := D { 1 } measures the discrepancy between the aggregate initial probabilities ∑ i ∈ V j π i \sum_{i\in V_j}\pi_i ∑ i ∈ V j π i and π j ⋆ \pi_j^\star π j ⋆ .
For consecutive times t t t and t + 1 t+1 t + 1 , the joint probability of Z t = i Z_t=i Z t = i and Z t + 1 = b Z_{t+1}=b Z t + 1 = b is q t ( i ) A i b q_t(i)A_{ib} q t ( i ) A ib . Summing over fitted states assigned to groups V j V_j V j and V ℓ V_\ell V ℓ therefore gives
P ˉ θ { t , t + 1 } ( j , ℓ ) = ∑ i ∈ V j ∑ b ∈ V ℓ q t ( i ) A i b , P ⋆ { t , t + 1 } ( j , ℓ ) = q t ⋆ ( j ) A j ℓ ⋆ . \bar{P}_{\theta}^{\{t,t+1\}}(j,\ell)=\sum_{i\in V_j}\sum_{b\in V_\ell}q_t(i)A_{ib},\qquad P_{\star}^{\{t,t+1\}}(j,\ell)=q_t^\star(j)A_{j\ell}^\star. P ˉ θ { t , t + 1 } ( j , ℓ ) = i ∈ V j ∑ b ∈ V ℓ ∑ q t ( i ) A ib , P ⋆ { t , t + 1 } ( j , ℓ ) = q t ⋆ ( j ) A j ℓ ⋆ .
We measure the largest discrepancy over consecutive time pairs by
D A : = max 1 ≤ t < T D { t , t + 1 } = max 1 ≤ t < T ∑ j , ℓ = 1 K ∣ ∑ i ∈ V j ∑ b ∈ V ℓ q t ( i ) A i b ⏟ fitted group-to-group flow − q t ⋆ ( j ) A j ℓ ⋆ ⏟ true state-to-state flow ∣ . (7) \begin{aligned}
D_A:=\max_{1\le t<T}D^{\{t,t+1\}} \\
&=\max_{1\le t<T}\sum_{j,\ell=1}^{K}\left|\underbrace{\sum_{i\in V_j}\sum_{b\in V_\ell}q_t(i)A_{ib}}_{\text{fitted group-to-group flow}}-\underbrace{q_t^\star(j)A_{j\ell}^\star}_{\text{true state-to-state flow}}\right|. \tag*{(7)}
\end{aligned} D A := 1 ≤ t < T max D { t , t + 1 } = 1 ≤ t < T max j , ℓ = 1 ∑ K fitted group-to-group flow i ∈ V j ∑ b ∈ V ℓ ∑ q t ( i ) A ib − true state-to-state flow q t ⋆ ( j ) A j ℓ ⋆ . (7)
Under state splitting, different allocations of flow among fitted states can leave the observation density unchanged. The fitted flows are therefore summed within each pair of groups before taking the absolute difference from the corresponding true flow. Thus, D A D_A D A measures discrepancies in occupancy-weighted joint probabilities . Although its notation refers to the transition matrix, it is not a norm of A − A ⋆ A-A^\star A − A ⋆ : a transition row contributes according to the probability of occupying its source state. Recovering an individual row therefore requires the additional normalization described in Corollary 3.8 . As with the other losses in this section, the probabilities entering D A D_A D A are unconditional under the fitted and true models.
Emission-center displacement. The aggregate probability losses do not measure how far fitted centers lie from their assigned true centers. To quantify these location errors, we further define
D μ : = max 1 ≤ t ≤ T ∑ i = 1 k q t ( i ) r i , V : = max 1 ≤ t ≤ T ∑ i = 1 k q t ( i ) r i 2 . (8) D_\mu:=\max_{1\le t\le T}\sum_{i=1}^{k}q_t(i)r_i,\qquad V:=\max_{1\le t\le T}\sum_{i=1}^{k}q_t(i)r_i^2. \tag*{(8)} D μ := 1 ≤ t ≤ T max i = 1 ∑ k q t ( i ) r i , V := 1 ≤ t ≤ T max i = 1 ∑ k q t ( i ) r i 2 . (8)
At each time, the sums are the expected distance and expected squared distance from a fitted emission center to its assigned true center. The weights q t ( i ) q_t(i) q t ( i ) account for the probability of occupying each fitted state, and the maximum gives the largest discrepancy over the observation horizon.
To measure the net displacement within each group, we also define
B : = max 1 ≤ t ≤ T ∑ j = 1 K ∥ ∑ i ∈ V j q t ( i ) ( μ i − a j ) ∥ 2 . (9) B:=\max_{1\le t\le T}\sum_{j=1}^{K}\left\|\sum_{i\in V_j}q_t(i)(\mu_i-a_j)\right\|_2. \tag*{(9)} B := 1 ≤ t ≤ T max j = 1 ∑ K i ∈ V j ∑ q t ( i ) ( μ i − a j ) 2 . (9)
The displacement vectors within a group are added before taking the norm, so deviations in opposite directions can cancel. For example, suppose two fitted states in V j V_j V j have emission centers a j − v a_j-v a j − v and a j + v a_j+v a j + v , where v ∈ R d v\in\mathbb{R}^d v ∈ R d is nonzero, and equal positive state probabilities at a given time. Although both centers differ from a j a_j a j , their net displacement is zero. However, their contributions to D μ D_\mu D μ and V V V remain positive. The quadratic quantity V V V measures this spread even when the net displacement vanishes. In § 4 , we bound V V V under over-specification and use this bound to control both D μ D_\mu D μ and squared Wasserstein distances. The three displacement losses are related by elementary inequalities. For every fitted parameter, the triangle inequality, Cauchy–Schwarz, and the bound r i ≤ 2 R r_i\le2R r i ≤ 2 R give
B ≤ D μ ≤ V , V ≤ 2 R D μ . (10) B\le D_\mu\le\sqrt{V},\qquad V\le2RD_\mu. \tag*{(10)} B ≤ D μ ≤ V , V ≤ 2 R D μ . (10)
These comparisons hold without matching or a density-error assumption, since they use only the definitions and bounded support. They show how a bound for V V V controls the first-order displacement D μ D_\mu D μ and the net displacement B B B . The converse implication from B B B to V V V is unavailable because opposite displacements within a group can cancel, as in the preceding example.
Wasserstein error for emission-center distributions. The grouped state-sequence probabilities can be correct even when the fitted emission centers differ from the true centers. To assess errors in both locations and probabilities, we define
G θ I : = L θ ( μ Z t 1 , … , μ Z t r ) , (11) G_\theta^I := \mathcal{L}_\theta(\mu_{Z_{t_1}},\ldots,\mu_{Z_{t_r}}), \tag*{(11)} G θ I := L θ ( μ Z t 1 , … , μ Z t r ) , (11)
where L θ \mathcal{L}_\theta L θ denotes distribution under θ \theta θ , and write G ⋆ I : = G θ ⋆ I G_\star^I := G_{\theta_\star}^I G ⋆ I := G θ ⋆ I for its true counterpart. This distribution assigns probabilities to sequences of actual emission-center locations, so it distinguishes fitted centers even when they belong to the same group. We compare G θ I G_\theta^I G θ I and G ⋆ I G_\star^I G ⋆ I through Wasserstein distances. For I = ( t 1 , … , t r ) I=(t_1,\ldots,t_r) I = ( t 1 , … , t r ) , both are supported on ( R d ) r (\mathbb{R}^d)^r ( R d ) r , whose elements represent sequences of emission centers at the selected times. For p ∈ { 1 , 2 } p\in\{1,2\} p ∈ { 1 , 2 } and u = ( u 1 , … , u r ) , v = ( v 1 , … , v r ) ∈ ( R d ) r u=(u_1,\ldots,u_r),v=(v_1,\ldots,v_r)\in(\mathbb{R}^d)^r u = ( u 1 , … , u r ) , v = ( v 1 , … , v r ) ∈ ( R d ) r , define
d p , r ( u , v ) : = ( 1 r ∑ ℓ = 1 r ∥ u ℓ − v ℓ ∥ 2 p ) 1 / p . (12) d_{p,r}(u,v):=\left(\frac{1}{r}\sum_{\ell=1}^{r}\lVert u_\ell-v_\ell\rVert_2^p\right)^{1/p}. \tag*{(12)} d p , r ( u , v ) := ( r 1 ℓ = 1 ∑ r ∥ u ℓ − v ℓ ∥ 2 p ) 1/ p . (12)
The associated Wasserstein distance is
W p , I ( G θ I , G ⋆ I ) : = ( inf γ ∈ Π ( G θ I , G ⋆ I ) ∫ d p , r ( u , v ) p d γ ( u , v ) ) 1 / p . W_{p,I}(G_\theta^I,G_\star^I):=\left(\inf_{\gamma\in\Pi(G_\theta^I,G_\star^I)}\int d_{p,r}(u,v)^p\,\mathrm{d}\gamma(u,v)\right)^{1/p}. W p , I ( G θ I , G ⋆ I ) := ( γ ∈ Π ( G θ I , G ⋆ I ) inf ∫ d p , r ( u , v ) p d γ ( u , v ) ) 1/ p .
where Π ( G θ I , G ⋆ I ) \Pi(G_\theta^I,G_\star^I) Π ( G θ I , G ⋆ I ) denotes the set of couplings with marginals G θ I G_\theta^I G θ I and G ⋆ I G_\star^I G ⋆ I . This definition allows different numbers of support points and does not require one-to-one matching between fitted and true states. The factor 1 / r 1/r 1/ r in equation (12) normalizes the transport cost by the number of selected time points. Table 1 summarizes the quantities measured by these losses and the scope of the corresponding recovery guarantees.
Table 1. Summary of latent recovery losses. Fitted states are grouped by their nearest true emission centers. Displacement weights are the fitted state probabilities q t ( i ) q_t(i) q t ( i ) . Bounds on D A D_A D A , D μ D_{\mu} D μ , B B B , and V V V control the corresponding errors uniformly over their indicated time ranges within the observed block.
Emission Separation and Clustering
The inverse bounds in § 3 and § 4 depend on both the separation of the true emission centers and the number of centers in each cluster. Nevertheless, the separation quantity Δ s e p \Delta_{\mathrm{sep}} Δ sep alone does not describe these cluster sizes. We capture the cluster structure through a partition in which centers within each cluster are close, while centers in different clusters are separated by a positive distance independent of Δ s e p \Delta_{\mathrm{sep}} Δ sep .
Structured regimes. In the structured regimes, assume that the true-state index set admits a partition [ K ] = C 1 ∪ ⋯ ∪ C M [K]=C_1\cup\cdots\cup C_M [ K ] = C 1 ∪ ⋯ ∪ C M into nonempty sets such that
∥ a j − a ℓ ∥ 2 ≤ C 0 Δ s e p , ∀ m ∈ [ M ] , ∀ j , ℓ ∈ C m , ∥ a j − a ℓ ∥ 2 ≥ D 0 , ∀ m , b ∈ [ M ] , m ≠ b , ∀ j ∈ C m , ℓ ∈ C b . (13) \begin{aligned}
\lVert a_j-a_\ell\rVert_2 &\le C_0\Delta_{\mathrm{sep}}, &&\forall m\in[M],\ \forall j,\ell\in C_m,\\
\lVert a_j-a_\ell\rVert_2 &\ge D_0, &&\forall m,b\in[M],\ m\ne b,\ \forall j\in C_m,\ \ell\in C_b. \tag*{(13)}
\end{aligned} ∥ a j − a ℓ ∥ 2 ∥ a j − a ℓ ∥ 2 ≤ C 0 Δ sep , ≥ D 0 , ∀ m ∈ [ M ] , ∀ j , ℓ ∈ C m , ∀ m , b ∈ [ M ] , m = b , ∀ j ∈ C m , ℓ ∈ C b . (13)
where C 0 , D 0 > 0 C_0,D_0>0 C 0 , D 0 > 0 are fixed constants independent of Δ s e p \Delta_{\mathrm{sep}} Δ sep . The diameter condition bounds every pairwise distance within a cluster by C 0 Δ s e p C_0\Delta_{\mathrm{sep}} C 0 Δ sep , whereas the second condition separates centers in different clusters by at least D 0 D_0 D 0 . Thus, as Δ s e p \Delta_{\mathrm{sep}} Δ sep decreases, centers within each cluster approach one another while distinct clusters remain separated. For M ≥ 2 M \ge2 M ≥ 2 , we additionally assume Δ s e p ≤ D 0 4 C 0 \Delta_{\mathrm{sep}} \le\frac{D_0}{4C_0} Δ sep ≤ 4 C 0 D 0 . This ensures that each cluster has diameter at most D 0 / 4 D_0/4 D 0 /4 , so its internal scale is smaller than the distance to any other cluster.
To express the dependence of the recovery bounds on cluster size, we define
s max : = max m ∈ [ M ] ∣ C m ∣ . s_{\max} := \max_{m \in[M]} |C_m|. s m a x := m ∈ [ M ] max ∣ C m ∣.
This quantity denotes the largest number of true emission centers in a single cluster. Our proofs use polynomials that equal one at a chosen true center and zero at the other true centers. The expectation of such a polynomial under the true emission-center distribution is therefore the probability assigned to the chosen center. Comparing this expectation between the fitted and true models requires a bound on the polynomial’s magnitude, which involves products of inverse distances between the centers. Within each cluster, these factors can contribute powers of Δ s e p − 1 \Delta_{\mathrm{sep}}^{-1} Δ sep − 1 . Between clusters, they are bounded by D 0 − 1 D_0^{-1} D 0 − 1 and affect only the constants in the bounds. This distinction allows the separation exponents to be expressed in terms of s max s_{\max} s m a x , although the constants can still depend on the total number of states K K K . When M = 1 M = 1 M = 1 , all K K K true centers belong to a single cluster of diameter at most C 0 Δ s e p C_0\Delta_{\mathrm{sep}} C 0 Δ sep , and s max = K s_{\max} = K s m a x = K . The condition on distances between distinct clusters is then vacuous.
Unstructured regime. We also consider an unstructured regime, in which no cluster conditions are imposed beyond the boundedness and distinctness assumptions of § 2.1 . In this regime, all the main inverse bounds use s max = K s_{\max} = K s m a x = K . Unlike the single-cluster regime, this convention does not require the full set of centers to have diameter O ( Δ s e p ) O(\Delta_{\mathrm{sep}}) O ( Δ sep ) .
Finite-Sample Density Estimation
To quantify the observation-density error, we use the Hellinger distance. For probability densities p , q p, q p , q on a common observation space, it is defined by
h 2 ( p , q ) : = 1 2 ∫ ( p ( x ) − q ( x ) ) 2 d x = 1 − ∫ p ( x ) q ( x ) d x . (14) h^2(p,q) := \frac{1}{2}\int\left(\sqrt{p(x)}-\sqrt{q(x)}\right)^2dx = 1-\int\sqrt{p(x)q(x)}dx. \tag*{(14)} h 2 ( p , q ) := 2 1 ∫ ( p ( x ) − q ( x ) ) 2 d x = 1 − ∫ p ( x ) q ( x ) d x . (14)
Under this convention, 0 ≤ h ( p , q ) ≤ 1 0 \le h(p,q) \le1 0 ≤ h ( p , q ) ≤ 1 .
The inverse inequalities in the following sections relate latent recovery errors to discrepancies between observation densities, independently of how the fitted model is obtained. They therefore yield statistical recovery guarantees for any estimator with a suitable observation-density error bound. Let θ ~ n \widetilde{\theta}_n θ n denote any data-dependent fitted parameter and suppose that, for some radius e n , T ( x ) e_{n,T}(x) e n , T ( x ) ,
P θ ⋆ { h ( p θ ~ n , T , p θ ⋆ , T ) ≤ e n , T ( x ) } ≥ 1 − e − x . (15) \mathbb{P}_{\theta^\star}\left\{h(p_{\widetilde{\theta}_n,T},p_{\theta^\star,T})\le e_{n,T}(x)\right\}\ge1-e^{-x}. \tag*{(15)} P θ ⋆ { h ( p θ n , T , p θ ⋆ , T ) ≤ e n , T ( x ) } ≥ 1 − e − x . (15)
Then every applicable deterministic result in § 3 and § 4 immediately becomes a statistical guarantee after replacing h ( p θ , T , p θ ⋆ , T ) h(p_{\theta,T},p_{\theta^\star,T}) h ( p θ , T , p θ ⋆ , T ) by e n , T ( x ) e_{n,T}(x) e n , T ( x ) .
Throughout the paper, we obtain explicit statistical recovery rates by applying this principle to the global MLE. We first bound its observation-density error over the full fitted model class and then use the inverse inequalities to recover latent quantities. This density-estimation step does not require separated emission centers; the dependence on emission geometry enters through the subsequent inverse bounds.
Maximum likelihood estimation (MLE). Given the n n n independent trajectories introduced in § 2.1 , we define the MLE θ ^ n \widehat{\theta}_n θ n over Θ k ( R ) \Theta_k(R) Θ k ( R ) by
θ ^ n ∈ arg max θ ∈ Θ k ( R ) ℓ n ( θ ) , ℓ n ( θ ) : = ∑ ℓ = 1 n log p θ , T ( X 1 : T ( ℓ ) ) . \widehat{\theta}_n\in\underset{\theta\in\Theta_k(R)}{\arg\max}\ \ell_n(\theta),\qquad\ell_n(\theta):=\sum_{\ell=1}^{n}\log p_{\theta,T}\left(X_{1:T}^{(\ell)}\right). θ n ∈ θ ∈ Θ k ( R ) arg max ℓ n ( θ ) , ℓ n ( θ ) := ℓ = 1 ∑ n log p θ , T ( X 1 : T ( ℓ ) ) .
We now establish a finite-sample Hellinger bound for this estimator in the following proposition.
Proposition 2.2 (Finite-sample Hellinger bound). There exists a constant C > 0 C>0 C > 0 , depending only on k k k and R R R , such that, for every n ≥ 2 n\ge2 n ≥ 2 , the radius ε n , T : = C d log ( n T d ) / n \varepsilon_{n,T}:=C\sqrt{d\log(nTd)/n} ε n , T := C d log ( n T d ) / n satisfies
P θ ⋆ { h ( p θ ^ n , T , p θ ⋆ , T ) > ε n , T } ≤ 1 n . \mathbb{P}_{\theta^\star}\left\{h(p_{\widehat{\theta}_n,T},p_{\theta^\star,T})>\varepsilon_{n,T}\right\}\le\frac{1}{n}. P θ ⋆ { h ( p θ n , T , p θ ⋆ , T ) > ε n , T } ≤ n 1 . The bound holds for any measurable choice of a global maximizer, including when the maximizer is not unique.
The proof is given in Appendix § 10 . For fixed k k k , d d d , T T T , R R R , Proposition 2.2 gives an n − 1 / 2 n^{-1/2} n − 1/2 rate up to a logarithmic factor, with probability at least 1 − 1 / n 1-1/n 1 − 1/ n . The density-estimation bound is independent of emission separation and remains valid at parameter configurations with singular Fisher information. Repeated fitted centers and zero fitted initial or transition probabilities are also allowed. The dependence on emission separation enters through the inverse inequalities used to recover latent quantities. We apply these inequalities on the common event E n : = { h ( p θ ^ n , T , p θ ⋆ , T ) ≤ ε n , T } \mathcal{E}_n:=\{h(p_{\widehat{\theta}_n,T},p_{\theta^\star,T})\le\varepsilon_{n,T}\} E n := { h ( p θ n , T , p θ ⋆ , T ) ≤ ε n , T } , which has probability at least 1 − 1 / n 1-1/n 1 − 1/ n . On this event, substituting ε n , T \varepsilon_{n,T} ε n , T for the observation-density error gives the corresponding MLE recovery bounds simultaneously for all applicable latent losses.
Throughout the following inverse inequalities and MLE rate propositions, factors depending only on K , k , R K,k,R K , k , R are absorbed into the constants. These constants may also depend on the cluster parameters in the structured regimes. Unless dependence on π min ⋆ \pi_{\min}^{\star} π m i n ⋆ is displayed or explicitly permitted, they do not depend on n , d , T , Δ s e p , π min ⋆ n,d,T,\Delta_{\mathrm{sep}},\pi_{\min}^{\star} n , d , T , Δ sep , π m i n ⋆ . A subscript r r r additionally permits dependence on the number of selected time coordinates. For a sequence of random variables Y n Y_n Y n and a positive deterministic sequence a n a_n a n , we write Y n = O P ( a n ) Y_n = O_{\mathbb{P}}(a_n) Y n = O P ( a n ) if Y n / a n Y_n/a_n Y n / a n is bounded in probability.
The recovery bounds below also apply when emission separation decreases as the sample size grows. Smaller separation increases the recovery bounds, so the number of independent trajectories must grow fast enough for these upper bounds to tend to zero. This guarantees consistent latent recovery provided that the constants remain uniformly bounded.
Exact Specification
In this section, we study latent recovery under exact specification, where the fitted and true models have the same number of states. However, equal state counts alone do not ensure that each true emission center has a fitted center nearby, since several fitted centers can be close to the same true center, while another true center has none nearby. To cover these configurations, we first establish inverse inequalities that apply to all fitted parameters in Θ K ( R ) \Theta_K(R) Θ K ( R ) , which we consider as global inverse bounds . Using these results, we show that sufficiently small observation-density error guarantees a one-to-one matching between fitted and true centers. This matching allows us to obtain recovery bounds with constants independent of the minimum true initial probability in § 3.2 . Combined with Proposition 2.2 , these inverse bounds provide convergence rates for the maximum likelihood estimator in exact-specified setting.
Separation-Dependent Bounds and Center Matching
To quantify latent recovery error, we consider both emission-center displacement and discrepancies in state probabilities and transition flows. Indeed, proximity of fitted centers to the true centers does not ensure that the probabilities assigned to these locations are accurate. As a result, location bounds must be accompanied by control of probability errors for individual states and state sequences. In addition to these separate location and probability losses, we consider Wasserstein distance to jointly assess errors in emission-center locations and their associated probabilities. The following theorem bounds these quantities in terms of the Hellinger distance between the fitted and true observation densities for a length-T T T sequence.
Theorem 3.1 (Global inverse bounds under exact specification). Let θ ∈ Θ K ( R ) \theta\in\Theta_K(R) θ ∈ Θ K ( R ) . In the structured regimes,
D μ ≤ C ( π min ⋆ ) Δ s e p − ( 2 s max − 2 ) h ( ρ θ , T , ρ θ ⋆ , T ) , (16) D_\mu\le C(\pi_{\min}^{\star})\Delta_{\mathrm{sep}}^{-(2s_{\max}-2)}h(\rho_{\theta,T},\rho_{\theta^{\star},T}), \tag*{(16)} D μ ≤ C ( π m i n ⋆ ) Δ sep − ( 2 s m a x − 2 ) h ( ρ θ , T , ρ θ ⋆ , T ) , (16) D π ∨ max t ≤ T D { t } ≤ C ( π min ⋆ ) Δ s e p − ( 2 s max − 1 ) h ( ρ θ , T , ρ θ ⋆ , T ) , (17) D_\pi\vee\max_{t\le T}D_{\{t\}} \le C(\pi_{\min}^{\star})\Delta_{\mathrm{sep}}^{-(2s_{\max}-1)}h(\rho_{\theta,T},\rho_{\theta^{\star},T}), \tag*{(17)} D π ∨ t ≤ T max D { t } ≤ C ( π m i n ⋆ ) Δ sep − ( 2 s m a x − 1 ) h ( ρ θ , T , ρ θ ⋆ , T ) , (17) D A ≤ C ( π min ⋆ ) Δ s e p − ( 2 s max − 1 ) h ( ρ θ , T , ρ θ ⋆ , T ) , (18) D_A \le C(\pi_{\min}^{\star})\Delta_{\mathrm{sep}}^{-(2s_{\max}-1)}h(\rho_{\theta,T},\rho_{\theta^{\star},T}), \tag*{(18)} D A ≤ C ( π m i n ⋆ ) Δ sep − ( 2 s m a x − 1 ) h ( ρ θ , T , ρ θ ⋆ , T ) , (18) max t ≤ T W 1 ( G θ { t } , G ⋆ { t } ) ≤ C ( π min ⋆ ) Δ s e p − ( 2 s max − 2 ) h ( ρ θ , T , ρ θ ⋆ , T ) . (19) \max_{t\le T} W_1\left(G_{\theta}^{\{t\}},G_{\star}^{\{t\}}\right) \le C(\pi_{\min}^{\star})\Delta_{\mathrm{sep}}^{-(2s_{\max}-2)}h(\rho_{\theta,T},\rho_{\theta^{\star},T}). \tag*{(19)} t ≤ T max W 1 ( G θ { t } , G ⋆ { t } ) ≤ C ( π m i n ⋆ ) Δ sep − ( 2 s m a x − 2 ) h ( ρ θ , T , ρ θ ⋆ , T ) . (19) More generally, for every nonempty I ⊆ [ T ] I \subseteq[T] I ⊆ [ T ] with ∣ I ∣ = r |I|=r ∣ I ∣ = r ,
D I ≤ C r ( π min ⋆ ) Δ s e p − ( 2 s max − 1 ) h ( ρ θ , T , ρ θ ⋆ , T ) . (20) D_I \le C_r(\pi_{\min}^{\star})\Delta_{\mathrm{sep}}^{-(2s_{\max}-1)}h(\rho_{\theta,T},\rho_{\theta^{\star},T}). \tag*{(20)} D I ≤ C r ( π m i n ⋆ ) Δ sep − ( 2 s m a x − 1 ) h ( ρ θ , T , ρ θ ⋆ , T ) . (20) The displacement and one-time probability bounds hold without cluster assumptions with s max = K s_{\max}=K s m a x = K , and
max t ≤ T W 1 ( G θ { t } , G ⋆ { t } ) ≤ C ( π min ⋆ ) Δ s e p − ( 2 K − 2 ) h ( ρ θ , T , ρ θ ⋆ , T ) . (21) \max_{t\le T} W_1\left(G_{\theta}^{\{t\}},G_{\star}^{\{t\}}\right) \le C(\pi_{\min}^{\star})\Delta_{\mathrm{sep}}^{-(2K-2)}h(\rho_{\theta,T},\rho_{\theta^{\star},T}). \tag*{(21)} t ≤ T max W 1 ( G θ { t } , G ⋆ { t } ) ≤ C ( π m i n ⋆ ) Δ sep − ( 2 K − 2 ) h ( ρ θ , T , ρ θ ⋆ , T ) . (21)
D I ≤ C r ( π min ⋆ ) Δ s e p − ( 2 K − 1 ) h ( p θ , T , p θ ⋆ , T ) , r ≥ 2. (22) D_I \le C_r(\pi^\star_{\min})\Delta_{\mathrm{sep}}^{-(2K-1)}h(p_{\theta,T},p_{\theta^\star,T}), \qquad r \ge2. \tag*{(22)} D I ≤ C r ( π m i n ⋆ ) Δ sep − ( 2 K − 1 ) h ( p θ , T , p θ ⋆ , T ) , r ≥ 2. (22)
The latter inequality also bounds D A D_A D A . All inequalities hold for every fitted parameter in Θ K ( R ) \Theta_K(R) Θ K ( R ) , with constants allowed to depend on the fixed positive quantity π min ⋆ \pi^\star_{\min} π m i n ⋆ .
The proof is given in Appendix D.1. Equation (16) controls the probability-weighted displacement of fitted centers, while equation (17) controls the probabilities assigned to the corresponding nearest-center groups. As B ≤ D μ B \le D_\mu B ≤ D μ , the displacement bound also controls the net displacement B B B with the same separation exponent. To account for dependence between consecutive states, equation (18) further bounds the aggregate transition flows. These comparisons use the aggregated quantities defined in Section 2.2, so they remain valid before any one-to-one matching of fitted and true centers. In particular, several fitted states may be assigned to the same true state, and fitted states with zero probability are allowed.
Separation exponents and dependence on the smallest initial-state probability. In the structured regimes, the state-probability errors D π D_\pi D π , D { t } D_{\{t\}} D { t } , and D I D_I D I and the transition-flow error D A D_A D A have separation exponent 2 s max − 1 2s_{\max}-1 2 s m a x − 1 , regardless of the number of selected times. The exponent for weighted emission-center displacement and one-time Wasserstein distance is 2 s max − 2 2s_{\max}-2 2 s m a x − 2 . Without cluster assumptions, the corresponding exponents are obtained by setting s max = K s_{\max}=K s m a x = K . Thus, the bounds distinguish the difficulty of recovering probability masses from that of recovering their locations, while accounting for the number of centers that approach one another.
The constants in Theorem 3.1 may depend on π min ⋆ \pi^\star_{\min} π m i n ⋆ . For emission-center displacement and one-time Wasserstein distance, we can make this dependence explicit by increasing the separation exponent. More concretely, the resulting bound in the structured regimes is
D μ ∨ max t ≤ T W 1 ( G θ { t } , G θ ⋆ { t } ) ≤ C π min ⋆ Δ s e p − ( 2 s max − 1 ) h ( p θ , T , p θ ⋆ , T ) . (23) D_\mu\vee\max_{t\le T} W_1(G_\theta^{\{t\}},G_{\theta^\star}^{\{t\}}) \le\frac{C}{\pi^\star_{\min}}\Delta_{\mathrm{sep}}^{-(2s_{\max}-1)}h(p_{\theta,T},p_{\theta^\star,T}). \tag*{(23)} D μ ∨ t ≤ T max W 1 ( G θ { t } , G θ ⋆ { t } ) ≤ π m i n ⋆ C Δ sep − ( 2 s m a x − 1 ) h ( p θ , T , p θ ⋆ , T ) . (23)
Compared with Theorem 3.1 , equation (23) shows explicitly how small initial-state probabilities affect the recovery bound, at the cost of one additional inverse power of separation. Without cluster assumptions, the displacement bound remains valid with s max = K s_{\max}=K s m a x = K , while the one-time Wasserstein bound becomes
max t ≤ T W 1 ( G θ { t } , G θ ⋆ { t } ) ≤ C { ( π min ⋆ ) − 1 Δ s e p − ( 2 K − 1 ) + Δ s e p − 2 K } h ( p θ , T , p θ ⋆ , T ) . \max_{t\le T} W_1(G_\theta^{\{t\}},G_{\theta^\star}^{\{t\}}) \le C\left\{(\pi^\star_{\min})^{-1}\Delta_{\mathrm{sep}}^{-(2K-1)}+\Delta_{\mathrm{sep}}^{-2K}\right\}h(p_{\theta,T},p_{\theta^\star,T}). t ≤ T max W 1 ( G θ { t } , G θ ⋆ { t } ) ≤ C { ( π m i n ⋆ ) − 1 Δ sep − ( 2 K − 1 ) + Δ sep − 2 K } h ( p θ , T , p θ ⋆ , T ) .
Both alternatives are established in the proof of Theorem 3.1 . In the unstructured Wasserstein bound, the additional term Δ s e p − 2 K \Delta_{\mathrm{sep}}^{-2K} Δ sep − 2 K produces a loss of two separation powers relative to the sharp bound in that theorem. For displacement and structured transport, the loss is one power. These explicit alternatives are useful when the initial probabilities vary, since they quantify the effect of small π min ⋆ \pi^\star_{\min} π m i n ⋆ . For fixed positive π min ⋆ \pi^\star_{\min} π m i n ⋆ , the bounds of Theorem 3.1 have the better separation exponents.
Applying the inverse bounds of Theorem 3.1 to the Hellinger error bound in Proposition 2.2 , we obtain the following finite-sample rates for latent recovery by the global maximum likelihood estimator.
Proposition 3.2 (Global MLE recovery rates). Let θ ^ n \widehat{\theta}_n θ n be a measurable global MLE over Θ K ( R ) \Theta_K(R) Θ K ( R ) . All losses below are evaluated at ( θ ^ n , θ ⋆ ) (\widehat{\theta}_n,\theta^\star) ( θ n , θ ⋆ ) . In the structured regimes,
D μ = O P ( Δ s e p − ( 2 s max − 2 ) d log ( n T d ) n ) , D_\mu= O_{\mathbb{P}}\left(\Delta_{\mathrm{sep}}^{-(2s_{\max}-2)}\sqrt{\frac{d\log(nTd)}{n}}\right), D μ = O P ( Δ sep − ( 2 s m a x − 2 ) n d log ( n T d ) ) ,
D π ∨ max t ≤ T D { t } = O P ( Δ s e p − ( 2 s max − 1 ) d log ( n T d ) n ) , D_{\pi} \vee\max_{t \le T} D_{\{t\}} = \mathcal{O}_{\mathbb{P}}\left(\Delta_{\mathrm{sep}}^{-(2s_{\max}-1)}\sqrt{\frac{d\log(nTd)}{n}}\right), D π ∨ t ≤ T max D { t } = O P ( Δ sep − ( 2 s m a x − 1 ) n d log ( n T d ) ) ,
D A = O P ( Δ s e p − ( 2 s max − 1 ) d log ( n T d ) n ) , D_A = \mathcal{O}_{\mathbb{P}}\left(\Delta_{\mathrm{sep}}^{-(2s_{\max}-1)}\sqrt{\frac{d\log(nTd)}{n}}\right), D A = O P ( Δ sep − ( 2 s m a x − 1 ) n d log ( n T d ) ) ,
max t ≤ T W 1 ( G θ ^ n { t } , G ⋆ { t } ) = O P ( Δ s e p − ( 2 s max − 2 ) d log ( n T d ) n ) . \max_{t \le T} W_1\left(G_{\widehat{\theta}_n}^{\{t\}},G_{\star}^{\{t\}}\right) = \mathcal{O}_{\mathbb{P}}\left(\Delta_{\mathrm{sep}}^{-(2s_{\max}-2)}\sqrt{\frac{d\log(nTd)}{n}}\right). t ≤ T max W 1 ( G θ n { t } , G ⋆ { t } ) = O P ( Δ sep − ( 2 s m a x − 2 ) n d log ( n T d ) ) .
The same probability rate holds for D I D_I D I , with a constant depending on r = ∣ I ∣ r=|I| r = ∣ I ∣ . Without cluster assumptions, all these rates hold with s max = K s_{\max}=K s m a x = K . All constants may depend on π min ⋆ \pi_{\min}^{\star} π m i n ⋆ . These global rates require no sample-size condition for matching.
The Wasserstein rates above describe recovery of the emission-center distribution at each individual time. However, these marginal guarantees alone do not quantify recovery of dependence across time. To address this limitation, the following corollary extends the Wasserstein bounds to the joint emission-center distributions G θ I G_{\theta}^{I} G θ I and G ⋆ I G_{\star}^{I} G ⋆ I , from which we derive the corresponding MLE rates.
Corollary 3.3 (Global Wasserstein bounds for emission-center distributions). Let θ ∈ Θ K ( R ) \theta\in\Theta_K(R) θ ∈ Θ K ( R ) . For nonempty I ⊆ [ T ] I \subseteq[T] I ⊆ [ T ] with ∣ I ∣ = r |I|=r ∣ I ∣ = r , the structured regimes satisfy
W 1 , I ( G θ I , G ⋆ I ) ≤ C r ( π min ⋆ ) Δ s e p − ( 2 s max − 2 ) h ( p θ , T , p θ ⋆ , T ) . (24) W_{1,I}\left(G_{\theta}^{I},G_{\star}^{I}\right) \le C_r(\pi_{\min}^{\star})\Delta_{\mathrm{sep}}^{-(2s_{\max}-2)}h(p_{\theta,T},p_{\theta^{\star},T}). \tag*{(24)} W 1 , I ( G θ I , G ⋆ I ) ≤ C r ( π m i n ⋆ ) Δ sep − ( 2 s m a x − 2 ) h ( p θ , T , p θ ⋆ , T ) . (24) In the unstructured regime, for every nonempty I I I ,
W 1 , I ( G θ I , G ⋆ I ) ≤ C r ( π min ⋆ ) Δ s e p − ( 2 K − 2 ) h ( p θ , T , p θ ⋆ , T ) . (25) W_{1,I}\left(G_{\theta}^{I},G_{\star}^{I}\right) \le C_r(\pi_{\min}^{\star})\Delta_{\mathrm{sep}}^{-(2K-2)}h(p_{\theta,T},p_{\theta^{\star},T}). \tag*{(25)} W 1 , I ( G θ I , G ⋆ I ) ≤ C r ( π m i n ⋆ ) Δ sep − ( 2 K − 2 ) h ( p θ , T , p θ ⋆ , T ) . (25) These inequalities do not require one-to-one matching between fitted and true emission centers.
The proof is given in Appendix § 12 .§ 2 . Compared with the one-time Wasserstein bounds in Theorem 3.1 , Corollary 3.3 controls the joint emission-center distributions with the same separation exponents. Consequently, the bounds account for dependence across several selected times without introducing additional inverse powers of separation. The constants may still depend on the number of selected times. The transport interpretation below explains why controlling the joint distribution of latent emission centers does not increase the separation exponent under exact specification.
Transport within and between cluster sequences. The distinction between the structured and unstructured bounds depends on both the transport distances and the probability mass moved over those distances. To describe this distinction, a cluster sequence records the cluster containing the center at each selected time. Two location sequences therefore share a cluster sequence when their corresponding centers belong to the same cluster at every selected time. For example, if a 1 , a 2 a_1,a_2 a 1 , a 2 belong to one cluster and a 3 , a 4 a_3,a_4 a 3 , a 4 to another, then ( a 1 , a 3 ) (a_1,a_3) ( a 1 , a 3 ) and ( a 2 , a 4 ) (a_2,a_4) ( a 2 , a 4 ) share a cluster sequence.
Within a cluster sequence. Transport between location sequences that share the same cluster sequence involves shorter distances than transport between different cluster sequences, so we bound these two contributions separately. After fitted locations are moved to their assigned true centers, redistributing probability within a cluster sequence costs at most C 0 Δ s e p C_0\Delta_{\mathrm{sep}} C 0 Δ sep per unit mass. This factor offsets one inverse power of separation in the probability bound (20) , giving Wasserstein exponent 2 s max − 2 2s_{\max}-2 2 s m a x − 2 .
(ii) Between cluster sequences. Transport between different cluster sequences can involve distances that do not shrink with Δ s e p \Delta_{\mathrm{sep}} Δ sep . The transport bound therefore relies on controlling how much probability must move between them. This comparison considers the total probability of each cluster sequence, without distinguishing individual centers within the same cluster. Since different clusters remain separated by a fixed gap, the proof can control these totals more accurately than individual state-sequence probabilities. More precisely, Lemma D.2 bounds the discrepancy between cluster-sequence probabilities by C r Δ s e p − ( 2 s max − 2 ) h ( p θ , T , p θ ⋆ , T ) C_r\Delta_{\mathrm{sep}}^{-(2s_{\max}-2)}h(p_{\theta,T},p_{\theta^\star,T}) C r Δ sep − ( 2 s m a x − 2 ) h ( p θ , T , p θ ⋆ , T ) . Since each unit of this discrepant mass can be transported over a distance of at most 2 R 2R 2 R , the resulting transport cost retains the same separation exponent.
(iii) Unstructured regime. Without cluster assumptions, distances between true centers need not shrink with Δ s e p \Delta_{\mathrm{sep}} Δ sep , so the preceding within-cluster argument no longer applies. The largest possible transport distance is instead the full support diameter D ⋆ : = max i , j ∥ a i − a j ∥ 2 D_\star:= \max_{i,j}\lVert a_i-a_j\rVert_2 D ⋆ := max i , j ∥ a i − a j ∥ 2 . A direct bound using this diameter and the probability inequality (22) would retain separation exponent 2 K − 1 2K-1 2 K − 1 , rather than the desired 2 K − 2 2K-2 2 K − 2 . The proof avoids this loss through a stronger probability bound that accounts for the actual distances between centers. More precisely, the proof of Corollary 3.3 gives
D I ≤ C r ( π min ⋆ ) D ⋆ − 2 Δ s e p − ( 2 K − 3 ) h ( p θ , T , p θ ⋆ , T ) . D_I \le C_r(\pi^\star_{\min})D_\star^{-2}\Delta_{\mathrm{sep}}^{-(2K-3)}h(p_{\theta,T},p_{\theta^\star,T}). D I ≤ C r ( π m i n ⋆ ) D ⋆ − 2 Δ sep − ( 2 K − 3 ) h ( p θ , T , p θ ⋆ , T ) .
After fitted locations are moved to their assigned true centers, the remaining probability mass can be redistributed at a cost of at most D ⋆ D_\star D ⋆ per unit mass. The resulting cost is therefore bounded by D ⋆ D I / 2 D_\star D_I/2 D ⋆ D I /2 , whose separation-dependent factor is D ⋆ − 1 Δ s e p − ( 2 K − 3 ) D_\star^{-1}\Delta_{\mathrm{sep}}^{-(2K-3)} D ⋆ − 1 Δ sep − ( 2 K − 3 ) . Because D ⋆ ≥ Δ s e p D_\star\ge\Delta_{\mathrm{sep}} D ⋆ ≥ Δ sep , this factor is at most Δ s e p − ( 2 K − 2 ) \Delta_{\mathrm{sep}}^{-(2K-2)} Δ sep − ( 2 K − 2 ) . The redistribution cost thus has the same separation exponent as the emission-center displacement, giving the joint Wasserstein bound without a cluster-diameter assumption.
Combining Corollary 3.3 with the density-estimation bound in Proposition 2.2 gives the following recovery rates for joint emission-center distributions.
Proposition 3.4 (Global MLE Wasserstein rates). Let θ ^ n \widehat{\theta}_n θ n be a measurable global MLE over Θ K ( R ) \Theta_K(R) Θ K ( R ) . For any nonempty I ⊆ [ T ] I \subseteq[T] I ⊆ [ T ] , let r = ∣ I ∣ r=|I| r = ∣ I ∣ . In the structured regimes,
W 1 , I ( G θ ^ n , I , G I ⋆ ) = O P ( Δ s e p − ( 2 s max − 2 ) d log ( n T d ) n ) . W_{1,I}(G_{\widehat{\theta}_n,I},G^\star_I)=O_{\mathbb{P}}\left(\Delta_{\mathrm{sep}}^{-(2s_{\max}-2)}\sqrt{\frac{d\log(nTd)}{n}}\right). W 1 , I ( G θ n , I , G I ⋆ ) = O P ( Δ sep − ( 2 s m a x − 2 ) n d log ( n T d ) ) . In the unstructured regime, the separation exponent is 2 K − 2 2K-2 2 K − 2 for every nonempty I I I . Constants may depend on π min ⋆ \pi^\star_{\min} π m i n ⋆ and r r r .
Because the same full-block Hellinger error controls every nonempty I ⊆ [ T ] I \subseteq[T] I ⊆ [ T ] , the recovery inequalities hold simultaneously for all these choices of I I I on the event in Proposition 2.2. These global guarantees allow several fitted centers to be associated with the same true center. They therefore establish recovery without first requiring a one-to-one correspondence between fitted and true states. To obtain such a correspondence, we next impose a sufficiently small observation-density error. The following corollary shows that this condition places exactly one fitted center within distance Δ s e p / 4 \Delta_{\mathrm{sep}}/4 Δ sep /4 of each true emission center. Whenever this correspondence holds, we write b j b_j b j for the fitted center relabeled to match a j a_j a j .
Corollary 3.5 (Density-error threshold for emission-center matching). For θ ∈ Θ K ( R ) \theta\in\Theta_K(R) θ ∈ Θ K ( R ) , there exists c ( π min ⋆ ) > 0 c(\pi^\star_{\min})>0 c ( π m i n ⋆ ) > 0 such that, if
h ( p θ , T , p θ ⋆ , T ) ≤ c ( π min ⋆ ) Δ s e p 2 s max − 1 , (26) h(p_{\theta,T},p_{\theta^\star,T}) \le c(\pi^\star_{\min})\Delta_{\mathrm{sep}}^{2s_{\max}-1}, \tag*{(26)} h ( p θ , T , p θ ⋆ , T ) ≤ c ( π m i n ⋆ ) Δ sep 2 s m a x − 1 , (26)
then a unique permutation pairs every fitted center b j b_j b j with a true center a j a_j a j such that ∥ b j − a j ∥ ≤ Δ s e p / 4 \lVert b_j-a_j\rVert\le\Delta_{\mathrm{sep}}/4 ∥ b j − a j ∥ ≤ Δ sep /4 . After relabeling, π j ≥ π j ⋆ / 2 \pi_j \ge\pi^\star_j/2 π j ≥ π j ⋆ /2 for every j j j . In the unstructured regime, take s max = K s_{\max}=K s m a x = K .
The proof of Corollary 3.5 is given in Appendix § 12 .§ 3 . The key step is to show that the density-error condition ensures a fitted center within distance Δ s e p / 4 \Delta_{\mathrm{sep}}/4 Δ sep /4 of every true center. Since these neighborhoods are disjoint and there are exactly K K K fitted centers for K K K true centers, each neighborhood contains exactly one fitted center, giving a unique matching. We use this matching to relabel the fitted states, their initial probabilities, and both indices of the transition matrix, so that fitted state j j j corresponds to true state j j j . After relabeling, the corollary also gives π j ≥ π j ⋆ / 2 \pi_j \ge\pi^\star_j/2 π j ≥ π j ⋆ /2 . Thus, the density-error condition in equation (26) provides both the center matching and the probability lower bound needed for the subsequent matched-center recovery bounds.
Sample size for guaranteed matching. For the global maximum likelihood estimator, Corollary 3.5 turns the Hellinger error bound of Proposition 2.2 into a sufficient sample-size condition for matching. Specifically, Proposition 2.2 gives an event E n : = { h ( p θ ^ n , T , p θ ⋆ , T ) ≤ ε n , T } \mathcal{E}_n := \{h(p_{\widehat{\theta}_n,T},p_{\theta^\star,T}) \le\varepsilon_{n,T}\} E n := { h ( p θ n , T , p θ ⋆ , T ) ≤ ε n , T } with probability at least 1 − 1 / n 1-1/n 1 − 1/ n . On this event, the density-error threshold in Corollary 3.5 is satisfied whenever
n ≥ C ( π min ⋆ ) Δ s e p − ( 4 s max − 2 ) d log ( n T d ) . (27) n \ge C(\pi^\star_{\min})\Delta_{\mathrm{sep}}^{-(4s_{\max}-2)}d\log(nTd). \tag*{(27)} n ≥ C ( π m i n ⋆ ) Δ sep − ( 4 s m a x − 2 ) d log ( n T d ) . (27)
For n ≥ 2 n \ge2 n ≥ 2 , this condition guarantees one-to-one emission-center matching with probability at least 1 − 1 / n 1-1/n 1 − 1/ n . In particular, closer emission centers require more independent trajectories to guarantee matching, with the separation exponent determined by the largest cluster size. Without cluster assumptions, the same conclusion holds with s max = K s_{\max}=K s m a x = K .
The constant in (27) may depend on π min ⋆ \pi^\star_{\min} π m i n ⋆ , so this condition does not specify how the sample size changes when some true initial-state probabilities become small. To quantify this dependence, we also give an alternative sufficient condition for the conclusions of Corollary 3.5 :
h ( p θ , T , p θ ⋆ , T ) ≤ c π min ⋆ Δ s e p 2 s max , (28) h(p_{\theta,T},p_{\theta^\star,T}) \le c\pi^\star_{\min}\Delta_{\mathrm{sep}}^{2s_{\max}}, \tag*{(28)} h ( p θ , T , p θ ⋆ , T ) ≤ c π m i n ⋆ Δ sep 2 s m a x , (28)
On the same event E n \mathcal{E}_n E n , condition (28) holds whenever
n ≥ C ( π min ⋆ ) 2 Δ s e p − 4 s max d log ( n T d ) . n \ge\frac{C}{(\pi^\star_{\min})^2}\Delta_{\mathrm{sep}}^{-4s_{\max}}d\log(nTd). n ≥ ( π m i n ⋆ ) 2 C Δ sep − 4 s m a x d log ( n T d ) .
This alternative condition shows that smaller minimum true initial-state probabilities require a larger sample size to guarantee matching, with an inverse-square dependence on π min ⋆ \pi^\star_{\min} π m i n ⋆ , but has two additional inverse powers of emission separation compared with (27) . For fixed positive π min ⋆ \pi^\star_{\min} π m i n ⋆ , condition (27) gives the better separation exponent. Without cluster assumptions, both alternative conditions hold with s max = K s_{\max}=K s m a x = K .
Inverse Bounds under Emission-Center Matching
Once fitted and true emission centers admit a one-to-one matching, we can compare the probabilities assigned to individual states and to sequences of states under a common labeling. Corollary 3.5 provides one sufficient condition for obtaining this correspondence. The following theorem gives recovery bounds under this matching, with ∥ b j − a j ∥ 2 ≤ Δ s e p / 4 \lVert b_j-a_j\rVert_2 \le\Delta_{\mathrm{sep}}/4 ∥ b j − a j ∥ 2 ≤ Δ sep /4 for every j j j after relabeling. Compared with the global bounds, the following result retains the same separation exponents for state and state-sequence probabilities, transition flows, probability-weighted emission-center displacement, and Wasserstein distances, while removing dependence on π min ⋆ \pi^\star_{\min} π m i n ⋆ from their constants. Thus, once the centers are matched, the same observation-density error controls these latent discrepancies without additional amplification from small initial-state probabilities. As in the global bounds, recovery across more selected times does not increase the separation exponents.
Theorem 3.6 (Paired-center inverse bounds). Suppose θ ∈ Θ K ( R ) \theta\in\Theta_K(R) θ ∈ Θ K ( R ) is labeled so that ∥ b j − a j ∥ ≤ Δ s e p / 4 \lVert b_j-a_j\rVert\le\Delta_{\mathrm{sep}}/4 ∥ b j − a j ∥ ≤ Δ sep /4 for every j j j . Then
D μ ≤ C Δ s e p − ( 2 s max − 2 ) h ( p θ , T , p θ ⋆ , T ) , (29) D_\mu\le C\Delta_{\mathrm{sep}}^{-(2s_{\max}-2)}h(p_{\theta,T},p_{\theta_\star,T}), \tag*{(29)} D μ ≤ C Δ sep − ( 2 s m a x − 2 ) h ( p θ , T , p θ ⋆ , T ) , (29) max t ≤ T D { t } ≤ C Δ s e p − ( 2 s max − 1 ) h ( p θ , T , p θ ⋆ , T ) . (30) \max_{t\le T}D_{\{t\}} \le C\Delta_{\mathrm{sep}}^{-(2s_{\max}-1)}h(p_{\theta,T},p_{\theta_\star,T}). \tag*{(30)} t ≤ T max D { t } ≤ C Δ sep − ( 2 s m a x − 1 ) h ( p θ , T , p θ ⋆ , T ) . (30) In the structured regimes, for every I ⊆ [ T ] I\subseteq[T] I ⊆ [ T ] with ∣ I ∣ = r ≥ 1 |I|=r\ge1 ∣ I ∣ = r ≥ 1 ,
D I ≤ C r Δ s e p − ( 2 s max − 1 ) h ( p θ , T , p θ ⋆ , T ) , (31) D_I \le C_r\Delta_{\mathrm{sep}}^{-(2s_{\max}-1)}h(p_{\theta,T},p_{\theta_\star,T}), \tag*{(31)} D I ≤ C r Δ sep − ( 2 s m a x − 1 ) h ( p θ , T , p θ ⋆ , T ) , (31) W 1 , I ( G θ I , G θ ⋆ I ) ≤ C r Δ s e p − ( 2 s max − 2 ) h ( p θ , T , p θ ⋆ , T ) . (32) W_{1,I}(G_\theta^I,G_{\theta_\star}^I) \le C_r\Delta_{\mathrm{sep}}^{-(2s_{\max}-2)}h(p_{\theta,T},p_{\theta_\star,T}). \tag*{(32)} W 1 , I ( G θ I , G θ ⋆ I ) ≤ C r Δ sep − ( 2 s m a x − 2 ) h ( p θ , T , p θ ⋆ , T ) . (32) In particular, D A ≤ C Δ s e p − ( 2 s max − 1 ) h ( p θ , T , p θ ⋆ , T ) D_A\le C\Delta_{\mathrm{sep}}^{-(2s_{\max}-1)}h(p_{\theta,T},p_{\theta_\star,T}) D A ≤ C Δ sep − ( 2 s m a x − 1 ) h ( p θ , T , p θ ⋆ , T ) . In the unstructured regime, the first two bounds hold with s max = K s_{\max}=K s m a x = K , and
D I ≤ C r Δ s e p − ( 2 K − 1 ) h ( p θ , T , p θ ⋆ , T ) , (33) D_I \le C_r\Delta_{\mathrm{sep}}^{-(2K-1)}h(p_{\theta,T},p_{\theta_\star,T}), \tag*{(33)} D I ≤ C r Δ sep − ( 2 K − 1 ) h ( p θ , T , p θ ⋆ , T ) , (33) max t ≤ T W 1 ( G θ { t } , G θ ⋆ { t } ) ≤ C Δ s e p − ( 2 K − 2 ) h ( p θ , T , p θ ⋆ , T ) , (34) \max_{t\le T}W_1(G_\theta^{\{t\}},G_{\theta_\star}^{\{t\}}) \le C\Delta_{\mathrm{sep}}^{-(2K-2)}h(p_{\theta,T},p_{\theta_\star,T}), \tag*{(34)} t ≤ T max W 1 ( G θ { t } , G θ ⋆ { t } ) ≤ C Δ sep − ( 2 K − 2 ) h ( p θ , T , p θ ⋆ , T ) , (34) W 1 , I ( G θ I , G θ ⋆ I ) ≤ C r Δ s e p − ( 2 K − 2 ) h ( p θ , T , p θ ⋆ , T ) , r ≥ 2. (35) W_{1,I}(G_\theta^I,G_{\theta_\star}^I) \le C_r\Delta_{\mathrm{sep}}^{-(2K-2)}h(p_{\theta,T},p_{\theta_\star,T}),\qquad r\ge2. \tag*{(35)} W 1 , I ( G θ I , G θ ⋆ I ) ≤ C r Δ sep − ( 2 K − 2 ) h ( p θ , T , p θ ⋆ , T ) , r ≥ 2. (35) The constants in these bounds are independent of π min ⋆ \pi_{\min}^\star π m i n ⋆ , and no lower bound on fitted state probabilities is required.
We defer the proof to Appendix § 12 .§ 4 . Under emission-center matching, we bound D μ D_\mu D μ directly with a constant independent of π min ⋆ \pi_{\min}^\star π m i n ⋆ . Since this loss already weights each center displacement by its fitted state probability, the bound does not require division by individual state probabilities. For joint state probabilities, the proof uses the Markov structure to reduce the comparison across all selected times to comparisons at successive selected times. These pairs need not be consecutive in the original trajectory. Under the common labeling supplied by matching, Lemma D.7 bounds the joint probability discrepancy by a sum of the corresponding two-time discrepancies. As each two-time discrepancy has separation exponent 2 s max − 1 2s_{\max}-1 2 s m a x − 1 , the joint bound retains this exponent, while the number of selected times affects only the multiplicative constant.
Sharpness of the structured separation exponents. Theorem 3.6 attains the sharp separation exponents for recovery under emission-center matching over structured classes containing collinear clusters of size s max ≥ 2 s_{\max}\ge2 s m a x ≥ 2 . Specifically, the exponents are 2 s max − 1 2s_{\max}-1 2 s m a x − 1 for state-probability and transition-flow recovery and 2 s max − 2 2s_{\max}-2 2 s m a x − 2 for weighted emission-center displacement and path Wasserstein recovery. Proposition D.8 establishes their sharpness through examples satisfying the matching condition with true initial-state probabilities bounded away from zero. The construction uses two mixing distributions with identical low-order moments but different mixing weights and center locations. These moment equalities cancel the corresponding low-order terms in the Gaussian observation densities, making the observation error small relative to the latent discrepancies. Each mixing distribution defines an HMM whose initial probabilities and transition rows all equal its mixture weights, so the same comparison holds for HMMs with independent hidden states. These examples also establish sharpness of the global structured bounds in Theorem 3.1 for fixed positive π min ⋆ \pi_{\min}^{\star} π m i n ⋆ . The separation exponents are therefore optimal in the worst case over these model classes.
The following proposition states the paired-center MLE rates. We use the sample-size condition in equation (27) to ensure that the fitted centers satisfy the matching hypothesis of Theorem 3.6 on the density event of Proposition 2.2 .
Proposition 3.7 (MLE recovery rates under emission-center matching). Let θ ^ n \widehat{\theta}_n θ n be a measurable global MLE over Θ K ( R ) \Theta_K(R) Θ K ( R ) , and suppose that the sample-size condition in (27) holds. After relabeling the fitted states according to the matching in Corollary 3.5 , the following rates hold as n → ∞ n \to\infty n → ∞ . All losses are evaluated at ( θ ^ n , θ ⋆ ) (\widehat{\theta}_n,\theta^\star) ( θ n , θ ⋆ ) :
D μ = O P ( Δ s e p − ( 2 s max − 2 ) d log ( n T d ) n ) D_{\mu}=\mathcal{O}_{\mathbb{P}}\left(\Delta_{\mathrm{sep}}^{-(2s_{\max}-2)}\sqrt{\frac{d\log(nTd)}{n}}\right) D μ = O P ( Δ sep − ( 2 s m a x − 2 ) n d log ( n T d ) )
D π ∨ max t ≤ T D { t } = O P ( Δ s e p − ( 2 s max − 1 ) d log ( n T d ) n ) D_{\pi}\vee\max_{t\leq T}D_{\{t\}}=\mathcal{O}_{\mathbb{P}}\left(\Delta_{\mathrm{sep}}^{-(2s_{\max}-1)}\sqrt{\frac{d\log(nTd)}{n}}\right) D π ∨ t ≤ T max D { t } = O P ( Δ sep − ( 2 s m a x − 1 ) n d log ( n T d ) )
For structured regimes,
D A = O P ( Δ s e p − ( 2 s max − 1 ) d log ( n T d ) n ) D_A=\mathcal{O}_{\mathbb{P}}\left(\Delta_{\mathrm{sep}}^{-(2s_{\max}-1)}\sqrt{\frac{d\log(nTd)}{n}}\right) D A = O P ( Δ sep − ( 2 s m a x − 1 ) n d log ( n T d ) )
D I = O P ( C r Δ s e p − ( 2 s max − 1 ) d log ( n T d ) n ) , ∣ I ∣ = r D_I=\mathcal{O}_{\mathbb{P}}\left(C_r\Delta_{\mathrm{sep}}^{-(2s_{\max}-1)}\sqrt{\frac{d\log(nTd)}{n}}\right),\qquad|I|=r D I = O P ( C r Δ sep − ( 2 s m a x − 1 ) n d log ( n T d ) ) , ∣ I ∣ = r
W 1 , I ( G θ ^ n I , G ⋆ I ) = O P ( C r Δ s e p − ( 2 s max − 2 ) d log ( n T d ) n ) . W_{1,I}\left(G_{\widehat{\theta}_n}^{I},G_{\star}^{I}\right)=\mathcal{O}_{\mathbb{P}}\left(C_r\Delta_{\mathrm{sep}}^{-(2s_{\max}-2)}\sqrt{\frac{d\log(nTd)}{n}}\right). W 1 , I ( G θ n I , G ⋆ I ) = O P ( C r Δ sep − ( 2 s m a x − 2 ) n d log ( n T d ) ) .
In the unstructured regime, the sample-size condition and the first two rates use s max = K s_{\max}=K s m a x = K , while
D A = O P ( Δ s e p − ( 2 K − 1 ) d log ( n T d ) n ) D_A=\mathcal{O}_{\mathbb{P}}\left(\Delta_{\mathrm{sep}}^{-(2K-1)}\sqrt{\frac{d\log(nTd)}{n}}\right) D A = O P ( Δ sep − ( 2 K − 1 ) n d log ( n T d ) )
D I = O P ( C r Δ s e p − ( 2 K − 1 ) d log ( n T d ) n ) D_I=\mathcal{O}_{\mathbb{P}}\left(C_r\Delta_{\mathrm{sep}}^{-(2K-1)}\sqrt{\frac{d\log(nTd)}{n}}\right) D I = O P ( C r Δ sep − ( 2 K − 1 ) n d log ( n T d ) )
max t ≤ T W 1 ( G θ ^ n { t } , G ⋆ { t } ) = O P ( Δ s e p − ( 2 K − 2 ) d log ( n T d ) n ) , \max_{t\leq T} W_1\left(G_{\hat{\theta}_n}^{\{t\}},G_\star^{\{t\}}\right)
= \mathcal{O}_{\mathbb{P}}\left(\Delta_{\mathrm{sep}}^{-(2K-2)}\sqrt{\frac{d\log(nTd)}{n}}\right), t ≤ T max W 1 ( G θ ^ n { t } , G ⋆ { t } ) = O P ( Δ sep − ( 2 K − 2 ) n d log ( n T d ) ) ,
W 1 , I ( G θ ^ n I , G ⋆ I ) = O P ( C r Δ s e p − ( 2 K − 2 ) d log ( n T d ) n ) , r ≥ 2. W_{1,I}\left(G_{\hat{\theta}_n}^{I},G_\star^{I}\right)
= \mathcal{O}_{\mathbb{P}}\left(C_r\Delta_{\mathrm{sep}}^{-(2K-2)}\sqrt{\frac{d\log(nTd)}{n}}\right),
\qquad r\geq2. W 1 , I ( G θ ^ n I , G ⋆ I ) = O P ( C r Δ sep − ( 2 K − 2 ) n d log ( n T d ) ) , r ≥ 2.
These rates hold simultaneously for all nonempty I ⊆ [ T ] I\subseteq[T] I ⊆ [ T ] .
We defer the proof to Appendix § 12 .§ 5 . Compared with the global rates in Proposition 3.2 , these rates retain the same dependence on sample size and emission separation. Once the centers are matched, the constants in the underlying probability-weighted inverse bounds are independent of π min ⋆ \pi_{\min}^\star π m i n ⋆ . Table 2 summarizes how these probability and location recovery rates depend on the emission geometry.
Table 2. Exact-fit MLE rates. Each entry gives an O P \mathcal{O}_{\mathbb{P}} O P upper rate. The second column covers D π D_{\pi} D π , D { t } D_{\{t\}} D { t } , D I D_I D I , and D A D_A D A ; the third covers D μ D_{\mu} D μ and W 1 , I W_{1,I} W 1 , I . Here, K K K is the number of true states and s max s_{\max} s m a x is the largest cluster size.
Emission-Center and Transition-Row Recovery
The bounds in § 3.2 measure emission-center displacement and transition-flow error through probability-weighted quantities. A state with small probability can therefore contribute little to these losses even when its emission center or transition probabilities are poorly estimated. To assess recovery of every state, we instead consider the largest emission-center error and the largest transition-row error. Using the labeling established in Corollary 3.5 , the following corollary bounds both errors in terms of the observation-density error.
Corollary 3.8 (Emission-center and transition-row bounds). Under the localization condition of Corollary 3.5 , use its aligned labeling. In the structured regimes,
max j ∥ b j − a j ∥ ≤ C π min ⋆ Δ s e p − ( 2 s max − 2 ) h ( p θ , T , p θ ⋆ , T ) , (36) \max_j \left\|b_j-a_j\right\|
\leq\frac{C}{\pi_{\min}^\star}\Delta_{\mathrm{sep}}^{-(2s_{\max}-2)}h(p_{\theta,T},p_{\theta^\star,T}), \tag*{(36)} j max ∥ b j − a j ∥ ≤ π m i n ⋆ C Δ sep − ( 2 s m a x − 2 ) h ( p θ , T , p θ ⋆ , T ) , (36) max j ∥ A j ⋅ − A j ⋅ ⋆ ∥ 1 ≤ C π min ⋆ Δ s e p − ( 2 s max − 1 ) h ( p θ , T , p θ ⋆ , T ) (37) \max_j \left\|A_{j\cdot}-A_{j\cdot}^\star\right\|_1
\leq\frac{C}{\pi_{\min}^\star}\Delta_{\mathrm{sep}}^{-(2s_{\max}-1)}h(p_{\theta,T},p_{\theta^\star,T}) \tag*{(37)} j max A j ⋅ − A j ⋅ ⋆ 1 ≤ π m i n ⋆ C Δ sep − ( 2 s m a x − 1 ) h ( p θ , T , p θ ⋆ , T ) (37) Without cluster assumptions, both bounds hold with s max = K s_{\max}=K s m a x = K .
The proof of Corollary 3.8 is deferred to Appendix § 12 .§ 5 . The corollary shows that controlling every center and transition row incurs a dependence on ( π min ⋆ ) − 1 (\pi_{\min}^\star)^{-1} ( π m i n ⋆ ) − 1 without increasing the separation exponents of Theorem 3.6 . This additional dependence arises because passing from probability-weighted errors to errors for each state requires division by the corresponding initial probability. For centers, this division removes the displacement weight. For transition rows, division by the initial probability requires control of both flow and initial-probability errors. In either case, smaller initial probabilities amplify the resulting error bounds, so the guarantee over all states depends on π min ⋆ \pi_{\min}^\star π m i n ⋆ . Since recovering transition rows requires only normalization by initial probabilities, neither invertibility of the transition matrix nor a positive lower bound on its individual entries is required. For the global maximum likelihood estimator, these bounds result in the following rates for recovering individual emission centers and transition rows.
Proposition 3.9 (MLE rates for emission centers and transition rows). Let θ ^ n \hat{\theta}_n θ ^ n be a measurable global MLE over Θ K ( R ) \Theta_K(\mathbb{R}) Θ K ( R ) , and suppose that the sample-size condition in (27) holds. After relabeling the fitted states according to the matching in Corollary 3.5 ,
max j ∈ [ K ] ∥ μ ^ j − μ j ⋆ ∥ 2 = O P ( ( π min ⋆ ) − 1 Δ s e p − ( 2 s max − 2 ) d log ( n T d ) n ) , \max_{j\in[K]}\left\|\hat{\mu}_j-\mu_j^\star\right\|_2
= \mathcal{O}_{\mathbb{P}}\left((\pi_{\min}^\star)^{-1}\Delta_{\mathrm{sep}}^{-(2s_{\max}-2)}\sqrt{\frac{d\log(nTd)}{n}}\right), j ∈ [ K ] max μ ^ j − μ j ⋆ 2 = O P ( ( π m i n ⋆ ) − 1 Δ sep − ( 2 s m a x − 2 ) n d log ( n T d ) ) ,
max j ∈ [ K ] ∥ A ^ j ⋅ − A j ⋅ ⋆ ∥ 1 = O P ( ( π min ⋆ ) − 1 Δ s e p − ( 2 s max − 1 ) d log ( n T d ) n ) . \max_{j\in[K]} \lVert\widehat{A}_{j\cdot}-A^\star_{j\cdot}\rVert_1 = O_{\mathbb{P}}\left((\pi^\star_{\min})^{-1}\Delta_{\mathrm{sep}}^{-(2s_{\max}-1)}\sqrt{\frac{d\log(nTd)}{n}}\right). j ∈ [ K ] max ∥ A j ⋅ − A j ⋅ ⋆ ∥ 1 = O P ( ( π m i n ⋆ ) − 1 Δ sep − ( 2 s m a x − 1 ) n d log ( n T d ) ) .
Without cluster assumptions, the sample-size condition and both rates hold with s max = K s_{\max}=K s m a x = K .
The result follows by applying Corollary 3.8 on the density event of Proposition 2.2 , with emission-center matching ensured by (27) .
Over-Specification
In this section, we study latent recovery when the fitted HMM has more states than the true model. Several fitted states may then represent the same true state, so a comparison of individual fitted and true parameters is no longer sufficient to describe what the observation distribution determines. We therefore consider recovery after grouping fitted states associated with the same true emission center. Our analysis focuses on the following objectives:
(i) Recover aggregate state probabilities and transition flows. Splitting a true state into several fitted states with the same emission center can preserve the observation distribution while changing their individual initial and transition probabilities. We therefore compare the total fitted probability assigned to each true center, or each sequence of true centers, with its true counterpart. Unlike the exactly specified case, the grouped process c ( Z t ) c(Z_t) c ( Z t ) need not be Markov, as illustrated in Example D.9 . Consequently, we bound aggregate state-sequence probabilities directly from the observation density rather than applying the Markov comparison through two-time probabilities used in § 3.2 .
(ii) Control emission-center displacement without losing opposing shifts. Fitted centers assigned to the same true center may move in opposite directions, so their contributions to the net displacement B B B can cancel. We also control the quadratic displacement V V V , which records squared distances without cancellation, and recover a bound for D μ D_\mu D μ through D μ ≤ V D_\mu\le\sqrt{V} D μ ≤ V . § 4.1 establishes bounds for B B B , V V V , and the aggregate probabilities considered in (i).
The following example illustrates the need for aggregation in (i) by showing that individual fitted transition rows can vary while the observation distribution remains unchanged.
Example 4.1 (Arbitrary internal transitions). Split each true state j j j into substates ( j , 1 ) , … , ( j , m j ) (j,1),\ldots,(j,m_j) ( j , 1 ) , … , ( j , m j ) sharing the same emission center a j a_j a j . For arbitrary probability vectors r j ℓ , u ( ⋅ ) r_{j\ell,u}(\cdot) r j ℓ , u ( ⋅ ) on the substates of state ℓ \ell ℓ , set
A ~ ( j , u ) , ( ℓ , v ) = A j ℓ ⋆ r j ℓ , u ( v ) . \widetilde{A}_{(j,u),(\ell,v)} = A^\star_{j\ell}r_{j\ell,u}(v). A ( j , u ) , ( ℓ , v ) = A j ℓ ⋆ r j ℓ , u ( v ) . If the initial substate masses sum to π j ⋆ \pi_j^\star π j ⋆ within each true state, then the fitted and true HMMs have identical joint observation densities for every sequence length. If m ℓ ≥ 2 m_\ell\ge2 m ℓ ≥ 2 and A j ℓ ⋆ > 0 A^\star_{j\ell}>0 A j ℓ ⋆ > 0 for some j , ℓ j,\ell j , ℓ , the individual substate transition rows can vary continuously without changing these observation densities. Thus, individual split-state transition rows are not meaningful estimands without an additional identifying convention, whereas aggregate probabilities and transition flows remain meaningful recovery targets.
We present the detailed verification of this example in Appendix § 13 .§ 5 . The construction shows that individual split-state transition rows cannot generally be recovered even if the observation distribution is known exactly. The relevant quantities are instead the total probabilities and transition flows associated with each true center, together with the distributions of latent emission-center sequences. Throughout this section, aggregation refers to summing fitted-state probabilities according to the assignment map.
Recovery after Aggregating Fitted States
We begin by extending the global inverse bounds of § 3.1 to over-specification. Two changes arise when several fitted states can correspond to one true state. First, the aggregate probability bounds have larger separation exponents than their exact-fit counterparts. Second, emission-center displacement can no longer be controlled linearly by the observation-density error in general. For instance, splitting a true center into two equally weighted fitted centers at distance ε \varepsilon ε on opposite sides gives a displacement proportional to ε \varepsilon ε . Relative to the true emission density, the two opposite shifts produce first-order changes of opposite signs. These changes cancel when the shifted densities are averaged, leaving a Hellinger error proportional to ε 2 \varepsilon^2 ε 2 . This first-order cancellation prevents a bound for D μ D_\mu D μ that is linear in h ( p θ , T , p θ ⋆ , T ) h(p_{\theta,T},p_{\theta^\star,T}) h ( p θ , T , p θ ⋆ , T ) from holding uniformly over all over-specified configurations. We instead bound V V V and B B B directly and control D μ D_\mu D μ through D μ ≤ V D_\mu\le\sqrt{V} D μ ≤ V . The following theorem bounds these displacement quantities and aggregate state-sequence probability errors by the full-block Hellinger error, uniformly over fitted parameters in Θ k ( R ) \Theta_k(\mathcal{R}) Θ k ( R ) .
Theorem 4.2 (Overfitted location and aggregate-flow bounds). For any k ≥ K k \ge K k ≥ K and θ ∈ Θ k ( R ) \theta\in\Theta_k(\mathcal{R}) θ ∈ Θ k ( R ) , in the structured regimes, put β r = max { 2 s max , r ( s max − 1 ) + s max } \beta_r=\max\{2s_{\max},r(s_{\max}-1)+s_{\max}\} β r = max { 2 s m a x , r ( s m a x − 1 ) + s m a x } . Then
V ≤ C Δ s e p − ( 2 s max − 2 ) h ( p θ , T , p θ ⋆ , T ) , (38) V \le C\Delta_{\mathrm{sep}}^{-(2s_{\max}-2)}h(p_{\theta,T},p_{\theta^\star,T}), \tag*{(38)} V ≤ C Δ sep − ( 2 s m a x − 2 ) h ( p θ , T , p θ ⋆ , T ) , (38) B ≤ C Δ s e p − ( 2 s max − 1 ) h ( p θ , T , p θ ⋆ , T ) , (39) B \le C\Delta_{\mathrm{sep}}^{-(2s_{\max}-1)}h(p_{\theta,T},p_{\theta^\star,T}), \tag*{(39)} B ≤ C Δ sep − ( 2 s m a x − 1 ) h ( p θ , T , p θ ⋆ , T ) , (39) D I ≤ C r Δ s e p − β r h ( p θ , T , p θ ⋆ , T ) , I ⊆ [ T ] , ∣ I ∣ = r ≥ 1. (40) D_I \le C_r\Delta_{\mathrm{sep}}^{-\beta_r}h(p_{\theta,T},p_{\theta^\star,T}), \qquad I\subseteq[T],\ |I|=r\ge1. \tag*{(40)} D I ≤ C r Δ sep − β r h ( p θ , T , p θ ⋆ , T ) , I ⊆ [ T ] , ∣ I ∣ = r ≥ 1. (40) In particular,
D π ≤ C Δ s e p − 2 s max h ( p θ , T , p θ ⋆ , T ) , D A ≤ C Δ s e p − β 2 h ( p θ , T , p θ ⋆ , T ) . D_\pi\le C\Delta_{\mathrm{sep}}^{-2s_{\max}}h(p_{\theta,T},p_{\theta^\star,T}), \qquad D_A \le C\Delta_{\mathrm{sep}}^{-\beta_2}h(p_{\theta,T},p_{\theta^\star,T}). D π ≤ C Δ sep − 2 s m a x h ( p θ , T , p θ ⋆ , T ) , D A ≤ C Δ sep − β 2 h ( p θ , T , p θ ⋆ , T ) . Without a cluster assumption, the V V V and B B B bounds hold with s max = K s_{\max}=K s m a x = K , while D I D_I D I has separation exponent
γ K , r : = max { 2 K , r ( K − 1 ) + K } . \gamma_{K,r}:=\max\{2K,r(K-1)+K\}. γ K , r := max { 2 K , r ( K − 1 ) + K } . In particular, γ K , 1 = 2 K \gamma_{K,1}=2K γ K , 1 = 2 K and γ K , 2 = 3 K − 2 \gamma_{K,2}=3K-2 γ K , 2 = 3 K − 2 . No lower bound on fitted state probabilities and no factor ( π min ⋆ ) − 1 (\pi^\star_{\min})^{-1} ( π m i n ⋆ ) − 1 is required.
The proof of Theorem 4.2 is given in Appendix § 13 .§ 1 . For aggregate state-sequence probabilities, the theorem gives bounds linear in the observation-density error even when the probabilities of individual split states cannot be identified. These bounds compare the total probability assigned to each true state sequence, regardless of how that probability is divided among fitted states. Unlike the exact-fit bounds, however, the separation exponents depend on the number of selected times r r r . For two selected times and s max ≥ 2 s_{\max} \ge2 s m a x ≥ 2 , the structured exponent is β 2 = 3 s max − 2 \beta_2 = 3s_{\max} - 2 β 2 = 3 s m a x − 2 , compared with 2 s max 2s_{\max} 2 s m a x for a single time. When s max > 2 s_{\max} > 2 s m a x > 2 , this larger exponent means that the guarantee for joint probabilities deteriorates faster as emission centers approach one another. The exponent β r \beta_r β r accounts for both the polynomial-moment comparison and the approximation error caused by displaced fitted centers. We first construct a polynomial that equals one at the target true-center sequence and zero at every other true-center sequence. Its expectation under the true model is therefore the probability of that sequence. Lemma A.3 then relates the difference between its fitted and true expectations to the observation Hellinger error, with a multiplier determined by the polynomial’s magnitude. The bound on this magnitude contributes the separation exponent ( 2 s max − 1 ) + ( r − 1 ) ( s max − 1 ) = r ( s max − 1 ) + s max (2s_{\max} - 1) + (r - 1)(s_{\max} - 1) = r(s_{\max} - 1) + s_{\max} ( 2 s m a x − 1 ) + ( r − 1 ) ( s m a x − 1 ) = r ( s m a x − 1 ) + s m a x for s max ≥ 2 s_{\max} \ge2 s m a x ≥ 2 . However, fitted centers may differ from their assigned true centers, so the fitted expectation only approximates the corresponding grouped probability. The polynomial is chosen to have zero first derivatives at the true centers, which makes this approximation error quadratic in the center displacements. The proof controls this second contribution with separation exponent β r = max { 2 s max , r ( s max − 1 ) + s max } \beta_r = \max\{2s_{\max}, r(s_{\max} - 1) + s_{\max}\} β r = max { 2 s m a x , r ( s m a x − 1 ) + s m a x } .
For emission-center recovery, however, the dependence on observation error changes. Since D μ ≤ V D_\mu\le\sqrt{V} D μ ≤ V , the quadratic displacement bound gives D μ ≤ C Δ s e p − ( s max − 1 ) h ( p θ , T , p θ ⋆ , T ) D_\mu\le C\Delta_{\mathrm{sep}}^{-(s_{\max}-1)}\sqrt{h(p_{\theta,T},p_{\theta^\star,T})} D μ ≤ C Δ sep − ( s m a x − 1 ) h ( p θ , T , p θ ⋆ , T ) , with C C C independent of π min ⋆ \pi^\star_{\min} π m i n ⋆ . In comparison, the aggregate probability bound is D I ≤ C r Δ s e p − β r h ( p θ , T , p θ ⋆ , T ) D_I \le C_r\Delta_{\mathrm{sep}}^{-\beta_r}h(p_{\theta,T},p_{\theta^\star,T}) D I ≤ C r Δ sep − β r h ( p θ , T , p θ ⋆ , T ) . For fixed emission geometry, the distinction is therefore between the powers h 1 / 2 h^{1/2} h 1/2 for probability-weighted emission-center displacement and h h h for aggregate state-sequence probabilities. If separation also varies, the corresponding factors Δ s e p − ( s max − 1 ) \Delta_{\mathrm{sep}}^{-(s_{\max}-1)} Δ sep − ( s m a x − 1 ) and Δ s e p − β r \Delta_{\mathrm{sep}}^{-\beta_r} Δ sep − β r must also be taken into account. This difference reflects the cancellation illustrated by symmetric splitting, where opposite center shifts can produce an observation error of quadratic order in their displacement.
We next relate these deterministic comparisons into statistical recovery rates. Combining Theorem 4.2 with the Hellinger bound in Proposition 2.2 , we obtain the following rates for the global maximum likelihood estimator.
Proposition 4.3 (MLE recovery rates under over-specification). Let θ ^ n \hat{\theta}_n θ ^ n be a measurable global MLE over Θ k ( R ) \Theta_k(\mathbb{R}) Θ k ( R ) , where k ≥ K k \ge K k ≥ K . All losses below are evaluated at ( θ ^ n , θ ⋆ ) (\hat{\theta}_n,\theta^\star) ( θ ^ n , θ ⋆ ) . In the structured regimes,
V = O P ( Δ s e p − ( 2 s max − 2 ) d log ( n T d ) n ) , V = \mathcal{O}_{\mathbb{P}}\left(\Delta_{\mathrm{sep}}^{-(2s_{\max}-2)}\sqrt{\frac{d\log(nTd)}{n}}\right), V = O P ( Δ sep − ( 2 s m a x − 2 ) n d log ( n T d ) ) , B = O P ( Δ s e p − ( 2 s max − 1 ) d log ( n T d ) n ) , B = \mathcal{O}_{\mathbb{P}}\left(\Delta_{\mathrm{sep}}^{-(2s_{\max}-1)}\sqrt{\frac{d\log(nTd)}{n}}\right), B = O P ( Δ sep − ( 2 s m a x − 1 ) n d log ( n T d ) ) , D π = O P ( Δ s e p − 2 s max d log ( n T d ) n ) , D_\pi= \mathcal{O}_{\mathbb{P}}\left(\Delta_{\mathrm{sep}}^{-2s_{\max}}\sqrt{\frac{d\log(nTd)}{n}}\right), D π = O P ( Δ sep − 2 s m a x n d log ( n T d ) ) , D A = O P ( Δ s e p − β 2 d log ( n T d ) n ) , D_A = \mathcal{O}_{\mathbb{P}}\left(\Delta_{\mathrm{sep}}^{-\beta_2}\sqrt{\frac{d\log(nTd)}{n}}\right), D A = O P ( Δ sep − β 2 n d log ( n T d ) ) , D I = O P ( C r Δ s e p − β r d log ( n T d ) n ) , ∣ I ∣ = r , D_I = \mathcal{O}_{\mathbb{P}}\left(C_r\Delta_{\mathrm{sep}}^{-\beta_r}\sqrt{\frac{d\log(nTd)}{n}}\right), \qquad|I| = r, D I = O P ( C r Δ sep − β r n d log ( n T d ) ) , ∣ I ∣ = r ,
D μ = O P ( Δ s e p − ( s max − 1 ) [ d log ( n T d ) n ] 1 / 4 ) . D_{\mu} = \mathcal{O}_{\mathbb{P}}\left(\Delta_{\mathrm{sep}}^{-(s_{\max}-1)}\left[\frac{d\log(nTd)}{n}\right]^{1/4}\right). D μ = O P ( Δ sep − ( s m a x − 1 ) [ n d log ( n T d ) ] 1/4 ) .
The rates hold simultaneously for all nonempty I ⊆ [ T ] I \subseteq[T] I ⊆ [ T ] . Without cluster assumptions, take s max = K s_{\max}=K s m a x = K in the V V V , B B B , D π D_{\pi} D π , D μ D_{\mu} D μ rates, and use exponents γ K , 2 \gamma_{K,2} γ K , 2 and γ K , r \gamma_{K,r} γ K , r for D A D_A D A and D I D_I D I , respectively. No lower bound on fitted state probabilities is required, and none of the rates contains a factor ( π min ⋆ ) − 1 (\pi_{\min}^{\star})^{-1} ( π m i n ⋆ ) − 1 .
The first five rates follow by applying Theorem 4.2 on the density event of Proposition 2.2 . For fixed model dimensions and emission geometry, these quantities, including the quadratic displacement V V V , have an n − 1 / 2 n^{-1/2} n − 1/2 upper rate, up to logarithmic factors. Since D μ ≤ V D_{\mu} \le\sqrt{V} D μ ≤ V , taking the square root of the bound for V V V gives the n − 1 / 4 n^{-1/4} n − 1/4 upper rate for probability-weighted emission-center displacement. This slower displacement rate reflects the symmetric splitting described above, where opposite center shifts can produce an observation-density error of quadratic order in their displacement.
Exact recovery of emission-center distributions. At zero density error, the emission-center distributions are recovered exactly within the observed block. This recovery requires both the emission-center locations and the probabilities of the corresponding location sequences to equal their true counterparts. Since V V V is a sum of nonnegative weighted squared distances at each time, V = 0 V=0 V = 0 forces every fitted center occupied with positive probability to equal its assigned true center, although several fitted states may share that center. The equality D I = 0 D_I=0 D I = 0 then ensures that, after these states are grouped, the joint probabilities of the corresponding location sequences equal those of the true model. It follows that the fitted and true latent location measures coincide within the observed block, even though their representations by individual states need not be unique.
Wasserstein Recovery of Emission-Center Sequence
We now use the bounds of § 4.1 to assess recovery of emission-center distributions. This comparison depends on both the displacement of fitted centers and the distances over which probability discrepancies must be transported. Because V V V controls squared displacement, the natural transport loss here is squared Wasserstein distance. The following theorem gives global bounds for squared Wasserstein error at any selection of times within the observed block.
Theorem 4.4 (Wasserstein Recovery of Emission-Center Sequence under over-specification). Let θ ∈ Θ k ( R ) \theta\in\Theta_k(\mathbb{R}) θ ∈ Θ k ( R ) with k ≥ K k \ge K k ≥ K . Let I ⊆ [ T ] I \subseteq[T] I ⊆ [ T ] be nonempty, with ∣ I ∣ = r |I|=r ∣ I ∣ = r , and let β r \beta_r β r be as in Theorem 4.2. In the structured regimes,
W 2 , I 2 ( G θ I , G ⋆ I ) ≤ C r Δ s e p − ( β r − 2 ) h ( p θ , T , p θ ⋆ , T ) . (41) W_{2,I}^{2}(G_{\theta}^{I},G_{\star}^{I}) \le C_r\Delta_{\mathrm{sep}}^{-(\beta_r-2)}h(p_{\theta,T},p_{\theta_{\star},T}). \tag*{(41)} W 2 , I 2 ( G θ I , G ⋆ I ) ≤ C r Δ sep − ( β r − 2 ) h ( p θ , T , p θ ⋆ , T ) . (41) Without cluster assumptions, the one-time bound is
max t ≤ T W 2 2 ( G θ { t } , G ⋆ { t } ) ≤ C Δ s e p − ( 2 K − 2 ) h ( p θ , T , p θ ⋆ , T ) . (42) \max_{t\le T} W_{2}^{2}(G_{\theta}^{\{t\}},G_{\star}^{\{t\}}) \le C\Delta_{\mathrm{sep}}^{-(2K-2)}h(p_{\theta,T},p_{\theta_{\star},T}). \tag*{(42)} t ≤ T max W 2 2 ( G θ { t } , G ⋆ { t } ) ≤ C Δ sep − ( 2 K − 2 ) h ( p θ , T , p θ ⋆ , T ) . (42) For r ≥ 2 r \ge2 r ≥ 2 , the unstructured bound is
W 2 , I 2 ( G θ I , G ⋆ I ) ≤ C r Δ s e p − ( γ K , r − 2 ) h ( p θ , T , p θ ⋆ , T ) . (43) W_{2,I}^{2}(G_{\theta}^{I},G_{\star}^{I}) \le C_r\Delta_{\mathrm{sep}}^{-(\gamma_{K,r}-2)}h(p_{\theta,T},p_{\theta_{\star},T}). \tag*{(43)} W 2 , I 2 ( G θ I , G ⋆ I ) ≤ C r Δ sep − ( γ K , r − 2 ) h ( p θ , T , p θ ⋆ , T ) . (43)
The proof of Theorem 4.4 is given in Appendix § 13 .§ 2 . The theorem combines location and probability errors to control the joint distribution of latent locations, without requiring a one-to-one correspondence between fitted and true states. Because this control is linear in h ( p θ , T , p θ ⋆ , T ) h(p_{\theta,T},p_{\theta_{\star},T}) h ( p θ , T , p θ ⋆ , T ) for squared Wasserstein distance, taking square roots gives the corresponding bound for W 2 , I W_{2,I} W 2 , I , which is proportional to h ( p θ , T , p θ ⋆ , T ) \sqrt{h(p_{\theta,T},p_{\theta_{\star},T})} h ( p θ , T , p θ ⋆ , T ) .
Proposition 4.5 (Sharpness of the transport separation exponents). Fix s ≥ 2 s \ge2 s ≥ 2 , r ≥ 1 r \ge1 r ≥ 1 , and T ≥ max { 2 , r } T \ge\max\{2,r\} T ≥ max { 2 , r } . There exist over-specified Gaussian HMMs with K = s K=s K = s true states in a single collinear cluster and true initial-state probabilities bounded away from zero such that, as Δ s e p → 0 \Delta_{\mathrm{sep}} \to0 Δ sep → 0 ,
W 2 , I 2 ( G θ I , G θ ⋆ I ) h ( p θ , T , p θ ⋆ , T ) ≍ Δ s e p − ( β r − 2 ) , I = { 1 , … , r } , \frac{W_{2,I}^{2}(G_{\theta}^{I},G_{\theta^\star}^{I})}{h(p_{\theta,T},p_{\theta^\star,T})} \asymp\Delta_{\mathrm{sep}}^{-(\beta_r-2)}, \qquad I=\{1,\ldots,r\}, h ( p θ , T , p θ ⋆ , T ) W 2 , I 2 ( G θ I , G θ ⋆ I ) ≍ Δ sep − ( β r − 2 ) , I = { 1 , … , r } , where β r = max { 2 s , r ( s − 1 ) + s } \beta_r=\max\{2s,r(s-1)+s\} β r = max { 2 s , r ( s − 1 ) + s } . The fitted-state count can be chosen independently of Δ s e p \Delta_{\mathrm{sep}} Δ sep . Hence, the structured exponent β r − 2 \beta_r-2 β r − 2 in Theorem 4.4 is sharp over model classes containing these collinear configurations. Taking s = K s=K s = K also establishes sharpness of the unstructured exponent γ K , r − 2 \gamma_{K,r}-2 γ K , r − 2 .
The result follows from Proposition E.1 , where we construct pairs of true and over-specified Gaussian HMMs with collinear emission centers that attain the separation exponents in Theorem 4.4 . These joint constructions establish sharpness over model classes containing the corresponding collinear configurations. For each choice of s s s and r r r , both the observation horizon and the number of fitted states remain fixed as emission separation tends to zero.
Cluster geometry and transport cost. The Wasserstein bounds account for both the displacement of fitted centers and the redistribution of probability between true centers. The first contribution is controlled by the bound for V V V in Theorem 4.2. To control the second, we move fitted locations to their assigned true centers and account for the distances over which probability mass is redistributed. The argument differs according to whether cluster assumptions are available.
(i) Under cluster assumptions. We separate the mass redistributed within each cluster sequence from the remaining mass transported between different cluster sequences. Within a cluster sequence, the two locations at each selected time belong to the same cluster and are at most C 0 Δ s e p C_0\Delta_{\mathrm{sep}} C 0 Δ sep apart. Each unit of redistributed mass therefore incurs a squared cost of at most C 0 2 Δ s e p 2 C_0^2\Delta_{\mathrm{sep}}^2 C 0 2 Δ sep 2 . Multiplying the aggregate probability-error bound in Theorem 4.2 by this cost reduces its separation exponent from β r \beta_r β r to β r − 2 \beta_r-2 β r − 2 .
For transport between different cluster sequences, however, the distances need not shrink with Δ s e p \Delta_{\mathrm{sep}} Δ sep . We instead control the amount of mass moved through the bound on cluster-sequence probability discrepancies in Lemma D.2, of order C r Δ s e p − ( 2 s max − 2 ) h ( p θ , T , p θ ⋆ , T ) C_r\Delta_{\mathrm{sep}}^{-(2s_{\max}-2)}h(p_{\theta,T},p_{\theta^\star,T}) C r Δ sep − ( 2 s m a x − 2 ) h ( p θ , T , p θ ⋆ , T ) . Since the squared transport distances are bounded, the resulting contribution has separation exponent 2 s max − 2 2s_{\max}-2 2 s m a x − 2 . This exponent is no larger than β r − 2 \beta_r-2 β r − 2 , so transport between cluster sequences does not worsen the separation dependence obtained within cluster sequences. The displacement contribution also has exponent 2 s max − 2 2s_{\max}-2 2 s m a x − 2 by the bound for V V V . Combining these contributions gives the squared Wasserstein exponent β r − 2 \beta_r-2 β r − 2 .
(ii) Without cluster assumptions. The true centers need not form clusters whose diameters shrink with Δ s e p \Delta_{\mathrm{sep}} Δ sep , so we cannot use the within-cluster cost reduction in (i). We instead retain the diameter of the full true support, D ⋆ : = max i , j ∥ a i − a j ∥ 2 D_\star:= \max_{i,j}\lVert a_i-a_j\rVert_2 D ⋆ := max i , j ∥ a i − a j ∥ 2 , in the probability comparison. The interpolation bounds in Lemma C.7 give D I ≤ C r D ⋆ − 2 Δ s e p − ( γ K , r − 2 ) h ( p θ , T , p θ ⋆ , T ) D_I \le C_rD_\star^{-2}\Delta_{\mathrm{sep}}^{-(\gamma_{K,r}-2)}h(p_{\theta,T},p_{\theta^\star,T}) D I ≤ C r D ⋆ − 2 Δ sep − ( γ K , r − 2 ) h ( p θ , T , p θ ⋆ , T ) . Redistributing probability between true centers incurs squared cost at most D ⋆ 2 D_\star^2 D ⋆ 2 per unit mass. Multiplying by this cost cancels the factor D ⋆ − 2 D_\star^{-2} D ⋆ − 2 , leaving separation exponent γ K , r − 2 \gamma_{K,r}-2 γ K , r − 2 . The displacement contribution has exponent 2 K − 2 2K-2 2 K − 2 , which is no larger because γ K , r ≥ 2 K \gamma_{K,r}\ge2K γ K , r ≥ 2 K . The combined bound therefore has squared Wasserstein exponent γ K , r − 2 \gamma_{K,r}-2 γ K , r − 2 . In particular, γ K , 1 = 2 K \gamma_{K,1}=2K γ K , 1 = 2 K gives the one-time exponent 2 K − 2 2K-2 2 K − 2 .
Having accounted for both displacement and probability redistribution, we can now obtain statistical rates for the latent emission-center distributions. Applying Theorem 4.4 on the density event of Proposition 2.2 gives the following result for the global maximum likelihood estimator.
Proposition 4.6 (MLE Wasserstein rates under over-specification). Let θ ^ n \hat{\theta}_n θ ^ n be a measurable global MLE over Θ k ( R ) \Theta_k(R) Θ k ( R ) , where k ≥ K k \ge K k ≥ K . For any nonempty I ⊆ [ T ] I \subseteq[T] I ⊆ [ T ] , let r = ∣ I ∣ r = |I| r = ∣ I ∣ . In the structured regimes,
W 2 , I ( G θ ^ n I , G ⋆ I ) = O P ( Δ s e p − ( β r − 2 ) / 2 [ d log ( n T d ) n ] 1 / 4 ) . W_{2,I}\left(G_{\hat{\theta}_n}^{I}, G_{\star}^{I}\right) = \mathcal{O}_{\mathbb{P}}\left(\Delta_{\mathrm{sep}}^{-(\beta_r-2)/2}\left[\frac{d\log(nTd)}{n}\right]^{1/4}\right). W 2 , I ( G θ ^ n I , G ⋆ I ) = O P ( Δ sep − ( β r − 2 ) /2 [ n d log ( n T d ) ] 1/4 ) . Without cluster assumptions,
max t ≤ T W 2 ( G θ ^ n { t } , G ⋆ { t } ) = O P ( Δ s e p − ( K − 1 ) [ d log ( n T d ) n ] 1 / 4 ) , \max_{t \le T} W_2\left(G_{\hat{\theta}_n}^{\{t\}}, G_{\star}^{\{t\}}\right) = \mathcal{O}_{\mathbb{P}}\left(\Delta_{\mathrm{sep}}^{-(K-1)}\left[\frac{d\log(nTd)}{n}\right]^{1/4}\right), t ≤ T max W 2 ( G θ ^ n { t } , G ⋆ { t } ) = O P ( Δ sep − ( K − 1 ) [ n d log ( n T d ) ] 1/4 ) , W 2 , I ( G θ ^ n I , G ⋆ I ) = O P ( Δ s e p − ( γ K , r − 2 ) / 2 [ d log ( n T d ) n ] 1 / 4 ) , r ≥ 2. W_{2,I}\left(G_{\hat{\theta}_n}^{I}, G_{\star}^{I}\right) = \mathcal{O}_{\mathbb{P}}\left(\Delta_{\mathrm{sep}}^{-(\gamma_{K,r}-2)/2}\left[\frac{d\log(nTd)}{n}\right]^{1/4}\right), \qquad r \ge2. W 2 , I ( G θ ^ n I , G ⋆ I ) = O P ( Δ sep − ( γ K , r − 2 ) /2 [ n d log ( n T d ) ] 1/4 ) , r ≥ 2. The rates hold simultaneously for all nonempty I ⊆ [ T ] I \subseteq[T] I ⊆ [ T ] . These bounds require neither emission-center matching nor a positive lower bound on fitted state probabilities.
The result follows by applying Theorem 4.4 on the density event of Proposition 2.2 and taking square roots. For fixed model dimensions and emission geometry, both single-time and joint emission-center distributions have an n − 1 / 4 n^{-1/4} n − 1/4 upper rate in W 2 W_2 W 2 , up to logarithmic factors. The corresponding squared Wasserstein errors have an n − 1 / 2 n^{-1/2} n − 1/2 upper rate.
Table 3 summarizes the MLE convergence rates and their separation dependence under over-specification. The common density-estimation scale is d log ( n T d ) / n \sqrt{d\log(nTd)/n} d log ( n T d ) / n . The upper bounds for aggregate probabilities, net displacement B B B , and quadratic displacement V V V are linear in this scale, whereas the bounds for probability-weighted emission-center displacement D μ D_\mu D μ and Wasserstein distance involve its square root. The table therefore distinguishes recovery of squared displacement from recovery in W 2 W_2 W 2 itself.
Table 3. Overfitted MLE rates for k ≥ K k\ge K k ≥ K . Each entry is an O P \mathcal{O}_{\mathbb{P}} O P upper rate. Here β r = max { 2 s max , r ( s max − 1 ) + s max } \beta_r=\max\{2s_{\max},r(s_{\max}-1)+s_{\max}\} β r = max { 2 s m a x , r ( s m a x − 1 ) + s m a x } and γ K , r = max { 2 K , r ( K − 1 ) + K } \gamma_{K,r}=\max\{2K,r(K-1)+K\} γ K , r = max { 2 K , r ( K − 1 ) + K } . No center matching or fitted-weight lower bound is required, and the constants are independent of π min ⋆ \pi_{\min}^{\star} π m i n ⋆ . Joint constants may depend on r = ∣ I ∣ r=|I| r = ∣ I ∣ . The notation max t W 2 \max_t W_2 max t W 2 abbreviates max t W 2 ( G θ n { t } , G ⋆ { t } ) \max_t W_2(G_{\theta_n}^{\{t\}},G_{\star}^{\{t\}}) max t W 2 ( G θ n { t } , G ⋆ { t } ) .
The following example compares the exact-fit and over-specified bounds for the same emission geometry, showing how both change when four centers form two separated pairs rather than a single cluster.
Example 4.7 (Four centers in two dimensions). For 0 < Δ s e p ≤ 1 / 4 0 < \Delta_{\mathrm{sep}} \le1/4 0 < Δ sep ≤ 1/4 , let
a 1 = ( − 2 , 0 ) , a 2 = ( − 2 , Δ s e p ) , a 3 = ( 2 , 0 ) , a 4 = ( 2 , Δ s e p ) . a_1 = (-2,0), \qquad a_2 = (-2,\Delta_{\mathrm{sep}}), \qquad a_3 = (2,0), \qquad a_4 = (2,\Delta_{\mathrm{sep}}). a 1 = ( − 2 , 0 ) , a 2 = ( − 2 , Δ sep ) , a 3 = ( 2 , 0 ) , a 4 = ( 2 , Δ sep ) . Under this configuration, the centers form two clusters of size two separated by a macroscopic gap, with K = 4 K = 4 K = 4 but s max = 2 s_{\max} = 2 s m a x = 2 . The matched one-time W 1 W_1 W 1 bound has separation factor Δ s e p − 2 \Delta_{\mathrm{sep}}^{-2} Δ sep − 2 , while the matched transition-flow bound has factor Δ s e p − 3 \Delta_{\mathrm{sep}}^{-3} Δ sep − 3 . The overfitted one-time W 2 W_2 W 2 bound has separation factor Δ s e p − 1 \Delta_{\mathrm{sep}}^{-1} Δ sep − 1 . The first two factors multiply the Hellinger error, whereas the W 2 W_2 W 2 factor multiplies its square root. If all four centers instead lie in one O ( Δ s e p ) O(\Delta_{\mathrm{sep}}) O ( Δ sep ) cluster, these multipliers become Δ s e p − 6 \Delta_{\mathrm{sep}}^{-6} Δ sep − 6 , Δ s e p − 7 \Delta_{\mathrm{sep}}^{-7} Δ sep − 7 , and Δ s e p − 3 \Delta_{\mathrm{sep}}^{-3} Δ sep − 3 .
This example shows why the largest cluster size, rather than the total number of states, determines the structured separation exponents. Although both configurations have four true states and the same minimum separation, two separated pairs give smaller recovery-error multipliers than a single cluster of four centers. The distances between clusters enter only through the constants in the bounds, while the within-cluster distances determine the inverse powers of Δ s e p \Delta_{\mathrm{sep}} Δ sep . For recovery at two selected times in the two-pair configuration, the overfitted joint probability bound has separation exponent β 2 = 4 \beta_2=4 β 2 = 4 , while the corresponding W 2 , I W_{2,I} W 2 , I bound has exponent ( β 2 − 2 ) / 2 = 1 (\beta_2-2)/2=1 ( β 2 − 2 ) /2 = 1 . The same comparison holds in higher dimensions after adding zero coordinates to the centers, although the density-estimation bound still depends on d d d .
Aggregate Recovery with Approximate Reference Centers
The recovery losses introduced above assign each fitted state to its nearest true emission center. This assignment provides a precise target for the inverse inequalities, but the true centers are unavailable in an application. A natural alternative is to group fitted states using estimated reference centers. To relate this construction to the theoretical losses, we must control how errors in the reference locations affect the assignment of fitted states and the resulting aggregate probabilities. The following result provides this comparison without imposing a Markov property on the grouped process or specifying the algorithm used to obtain the references.
Let ξ 1 , … , ξ K ∈ R d \xi_1,\ldots,\xi_K\in\mathbb{R}^d ξ 1 , … , ξ K ∈ R d be reference centers with labels aligned to the true centers, and define
c ^ ( i ) : = arg min j ∈ [ K ] ∥ μ i − ξ j ∥ 2 , P ^ θ I ( j ) : = P θ { c ^ ( Z t ℓ ) = j ℓ , 1 ≤ ℓ ≤ r } , \widehat{c}(i):=\arg\min_{j\in[K]}\|\mu_i-\xi_j\|_2,\qquad\widehat{P}_{\theta}^{I}(j):=P_{\theta}\{\widehat{c}(Z_{t_\ell})=j_\ell,\ 1\le\ell\le r\}, c ( i ) := arg j ∈ [ K ] min ∥ μ i − ξ j ∥ 2 , P θ I ( j ) := P θ { c ( Z t ℓ ) = j ℓ , 1 ≤ ℓ ≤ r } ,
with smallest-index tie breaking. Thus, P ^ θ I \widehat{P}_{\theta}^{I} P θ I is obtained by summing the probabilities of fitted state sequences assigned to the same reference-label sequence. Write D ^ I = ∥ P ^ θ I − P ⋆ I ∥ 1 \widehat{D}_I=\|\widehat{P}_{\theta}^{I}-P_{\star}^{I}\|_1 D I = ∥ P θ I − P ⋆ I ∥ 1 , and define D ^ π = D ^ { 1 } \widehat{D}_{\pi}=\widehat{D}_{\{1\}} D π = D { 1 } and D ^ A = max t < T D ^ { t , t + 1 } \widehat{D}_A=\max_{t<T}\widehat{D}_{\{t,t+1\}} D A = max t < T D { t , t + 1 } analogously.
Proposition 4.8 (Stability of grouped distributions under reference error). Suppose that, after a common relabeling of the reference centers,
max j ∈ [ K ] ∥ ξ j − a j ∥ 2 ≤ τ ≤ 1 4 δ s e p . \max_{j\in[K]} \lVert\xi_j-a_j\rVert_2\le\tau\le\frac{1}{4}\delta_{\mathrm{sep}}. j ∈ [ K ] max ∥ ξ j − a j ∥ 2 ≤ τ ≤ 4 1 δ sep . For each fitted state i i i , let
m i ( ξ ) : = min ℓ ≠ c ( i ) { ∥ μ i − ξ ℓ ∥ 2 − ∥ μ i − ξ c ( i ) ∥ 2 } , B τ ( ξ ) : = { i : m i ( ξ ) ≤ 2 τ } . m_i(\xi):=\min_{\ell\ne c(i)}\left\{\lVert\mu_i-\xi_\ell\rVert_2-\lVert\mu_i-\xi_{c(i)}\rVert_2\right\},\qquad B_\tau(\xi):=\{i:m_i(\xi)\le2\tau\}. m i ( ξ ) := ℓ = c ( i ) min { ∥ μ i − ξ ℓ ∥ 2 − ∥ μ i − ξ c ( i ) ∥ 2 } , B τ ( ξ ) := { i : m i ( ξ ) ≤ 2 τ } . The margin m i ( ξ ) m_i(\xi) m i ( ξ ) is the difference between the distances to the second-nearest and nearest reference centers and is computed without the true centers. Under the reference accuracy condition, each distance changes by at most τ \tau τ when a reference center is replaced by its corresponding true center, so the difference between two distances changes by at most 2 τ 2\tau 2 τ . A margin greater than 2 τ 2\tau 2 τ therefore guarantees an unchanged assignment, and only states in B τ ( ξ ) B_\tau(\xi) B τ ( ξ ) can contribute to the additional grouping error. For any fitted parameter θ ∈ Θ k ( R ) \theta\in\Theta_k(R) θ ∈ Θ k ( R ) with k ≥ K k\ge K k ≥ K and every nonempty I ⊆ [ T ] I\subseteq[T] I ⊆ [ T ] , r = ∣ I ∣ r=|I| r = ∣ I ∣ , one has
D ^ I ≤ D I + min { 2 ∑ t ∈ I ∑ i ∈ B τ ( ξ ) q t ( i ) , 8 r δ s e p D μ , 32 r δ s e p 2 V } . (44) \widehat D_I\le D_I+\min\left\{2\sum_{t\in I}\sum_{i\in B_\tau(\xi)}q_t(i),\frac{8r}{\delta_{\mathrm{sep}}}D_\mu,\frac{32r}{\delta_{\mathrm{sep}}^2}V\right\}. \tag*{(44)} D I ≤ D I + min ⎩ ⎨ ⎧ 2 t ∈ I ∑ i ∈ B τ ( ξ ) ∑ q t ( i ) , δ sep 8 r D μ , δ sep 2 32 r V ⎭ ⎬ ⎫ . (44) In particular, if no state with positive occupancy at a selected time lies in B τ ( ξ ) B_\tau(\xi) B τ ( ξ ) , then P ^ θ I = P ‾ θ I \widehat P^I_\theta=\overline P^I_\theta P θ I = P θ I exactly.
Under the reference accuracy condition, the margin term can be computed from the fitted model and reference centers to bound the additional error caused by reference-based grouping. Under exact specification, we control the additional grouping error through D μ / δ s e p D_\mu/\delta_{\mathrm{sep}} D μ / δ sep , and under over-specification through V / δ s e p 2 V/\delta_{\mathrm{sep}}^2 V / δ sep 2 . Substituting the preceding displacement bounds shows that these additional errors have no larger separation exponents than the corresponding bound for D I D_I D I . Consequently, the global exact-fit probability bounds of Theorem 3.1 and the overfitted probability bounds of Theorem 4.2 remain valid with D I D_I D I replaced by D ^ I \widehat D_I D I , with the same separation exponents. The same conclusion holds for D ^ π \widehat D_\pi D π and D ^ A \widehat D_A D A , and in the unstructured regimes with s max = K s_{\max}=K s m a x = K . In the matched exact-fit regime the corresponding constants remain independent of π min ∗ \pi_{\min}^* π m i n ∗ . In the overfitted regime no factor ( π min ∗ ) − 1 (\pi_{\min}^*)^{-1} ( π m i n ∗ ) − 1 is introduced.
Remark 4.9 (Why a reference accuracy condition is necessary). The following example shows why controlling reference-center error is necessary for aggregate recovery. Consider two exactly fitted states with true centers a 1 = 0 a_1=0 a 1 = 0 , a 2 = δ > 0 a_2=\delta>0 a 2 = δ > 0 and initial masses ( 1 / 2 , 1 / 2 ) (1/2,1/2) ( 1/2 , 1/2 ) . If the two supplied reference centers both equal δ / 2 \delta/2 δ /2 , their maximum true-center error is δ / 2 \delta/2 δ /2 . Smallest-index tie breaking then assigns both fitted states to the first reference group. Thus D ^ π = 1 \widehat D_\pi=1 D π = 1 even though D π = D μ = V = h ( p θ , T , p θ ∗ , T ) = 0 D_\pi=D_\mu=V=h(p_{\theta,T},p_{\theta^*,T})=0 D π = D μ = V = h ( p θ , T , p θ ∗ , T ) = 0 . Therefore no reference-based density-to-group-recovery implication can hold without some condition excluding such ambiguity.
The proof of Proposition 4.8 is given in Appendix § 9 .§ 5 . The bound separates two sources of error: the discrepancy under the true-center assignment and the additional discrepancy caused by changing that assignment. The margin term measures the fitted occupancy of states whose assignments may change, and it can be computed from the fitted model and supplied references once an accuracy bound τ \tau τ has been justified.
To obtain a statistical guarantee, suppose the reference accuracy event holds with probability at least 1 − γ n 1-\gamma_n 1 − γ n and the density event underlying a displayed recovery rate holds with probability at least 1 − α n 1-\alpha_n 1 − α n . On their intersection, the same recovery rate applies to the reference-grouped probabilities. This intersection has probability at least 1 − α n − γ n 1-\alpha_n-\gamma_n 1 − α n − γ n , without requiring independence between the reference estimates and the fitted model.
Obtaining reference centers from a pilot estimator. The exact-fit theory provides a way to construct sufficiently accurate reference centers for grouping. When K K K is known, we first fit a global K K K -state MLE to m m m independent trajectories and use its emission centers as references. If m m m satisfies the sample-size condition (27) , Corollary 3.5 guarantees that these reference centers can be matched to the true centers within distance τ = Δ s e p / 4 ≤ δ s e p / 4 \tau=\Delta_{\mathrm{sep}}/4\le\delta_{\mathrm{sep}}/4 τ = Δ sep /4 ≤ δ sep /4 with probability at least 1 − 1 / m 1-1/m 1 − 1/ m . This preliminary fit, which we call the pilot estimator, therefore supplies reference centers satisfying the accuracy condition required for aggregate recovery.
We then use these reference centers to group the states of a k k k -state estimator whose full-block Hellinger error is at most ε n \varepsilon_n ε n with probability at least 1 − α n 1-\alpha_n 1 − α n . On the intersection of the reference-accuracy and density-accuracy events, the reference-grouped probabilities satisfy the corresponding recovery bounds with h h h replaced by ε n \varepsilon_n ε n . This gives a probability guarantee of at least 1 − 1 / m − α n 1-1/m-\alpha_n 1 − 1/ m − α n , without requiring independence between the two estimators. In particular, Proposition 2.2 gives α n = 1 / n \alpha_n=1/n α n = 1/ n when the second estimator is the global MLE. Since the pilot fit determines reference centers only up to a permutation, comparing the grouped probabilities with their true counterparts also requires aligning the labels. The next corollary shows that one common relabeling suffices for all marginal and joint state-sequence distributions.
Corollary 4.10 (Reference recovery up to one common permutation). Suppose the K K K reference centers are unlabeled and some permutation σ \sigma σ of [ K ] [K] [ K ] satisfies
max ℓ ∈ [ K ] ∥ ξ ℓ − a σ ( ℓ ) ∥ 2 ≤ τ ≤ δ s e p / 4. (45) \max_{\ell\in[K]}\|\xi_\ell-a_{\sigma(\ell)}\|_2\le\tau\le\delta_{\mathrm{sep}}/4. \tag*{(45)} ℓ ∈ [ K ] max ∥ ξ ℓ − a σ ( ℓ ) ∥ 2 ≤ τ ≤ δ sep /4. (45) For a reference-label distribution Q Q Q on [ K ] r [K]^r [ K ] r , put ( σ # Q ) ( j 1 , … , j r ) = Q ( σ − 1 ( j 1 ) , … , σ − 1 ( j r ) ) (\sigma_{\#}Q)(j_1,\ldots,j_r)=Q(\sigma^{-1}(j_1),\ldots,\sigma^{-1}(j_r)) ( σ # Q ) ( j 1 , … , j r ) = Q ( σ − 1 ( j 1 ) , … , σ − 1 ( j r )) . There is a unique permutation σ \sigma σ satisfying (45) , and this same permutation satisfies, simultaneously for all nonempty I ⊆ [ T ] I\subseteq[T] I ⊆ [ T ] ,
∥ σ # P θ I − P ⋆ I ∥ 1 ≤ D I + min { 2 ∑ t ∈ I ∑ i ∈ B τ ( ξ ) q t ( i ) , 8 ∣ I ∣ D μ δ s e p , 32 ∣ I ∣ V δ s e p 2 } . \|\sigma_{\#}P_\theta^I-P^{\star I}\|_1\le D_I+\min\left\{2\sum_{t\in I}\sum_{i\in\mathcal{B}_\tau(\xi)}q_t(i),\frac{8|I|D_\mu}{\delta_{\mathrm{sep}}},\frac{32|I|V}{\delta_{\mathrm{sep}}^2}\right\}. ∥ σ # P θ I − P ⋆ I ∥ 1 ≤ D I + min ⎩ ⎨ ⎧ 2 t ∈ I ∑ i ∈ B τ ( ξ ) ∑ q t ( i ) , δ sep 8∣ I ∣ D μ , δ sep 2 32∣ I ∣ V ⎭ ⎬ ⎫ . Consequently, the reference-grouped probability bounds retain the separation exponents obtained using the true emission centers for grouping under one common relabeling of every time block.
If ξ \xi ξ consists of the centers from a measurable exactly specified global K K K -state MLE on m m m independent trajectories, with K K K known and m m m satisfying (27) , and a second (possibly dependent) estimator θ ~ n \widetilde{\theta}_n θ n satisfies P ⋆ { h ( P θ ~ n , T , P ⋆ , T ) ≤ ε n } ≥ 1 − α n \mathbb{P}_{\star}\{h(\mathbb{P}_{\widetilde{\theta}_n,T},\mathbb{P}_{\star,T})\le\varepsilon_n\}\ge1-\alpha_n P ⋆ { h ( P θ n , T , P ⋆ , T ) ≤ ε n } ≥ 1 − α n , then with probability at least 1 − m − 1 − α n 1-m^{-1}-\alpha_n 1 − m − 1 − α n one common permutation yields the corresponding D I D_I D I rates obtained using the true emission centers for grouping for all nonempty I ⊆ [ T ] I\subseteq[T] I ⊆ [ T ] .
The corollary uses the same permutation across all marginal and joint state-sequence distributions to preserve a consistent correspondence between reference groups and true states. Relabeling these distributions separately could conceal differences in the underlying dynamics. This common permutation is needed only to compare the grouped probabilities with their true counterparts, since constructing the reference groups requires only the reference centers.
Proposition 4.8 and Corollary 4.10 therefore extend the recovery bounds to sufficiently accurate reference centers while retaining the separation exponents obtained using the true emission centers for grouping. To examine how reference errors affect aggregate recovery, we perturb reference centers within the permitted radius and measure the resulting changes in assignments and grouped probability errors in Appendix § 14 .§ 5 . Beyond these controlled perturbations, we also examine grouping by complete-linkage clustering of the fitted centers in Appendix § 14 .§ 4 , where we compare the resulting aggregate-flow errors with those obtained using the true emission centers for grouping.
Empirical Analysis
In the preceding sections, we have established latent recovery bounds with explicit dependence on emission separation and derived the corresponding convergence rates under exact specification and over-specification. Guided by these results, we empirically investigate how emission geometry, sample size, and model specification affect observation fit and latent recovery through synthetic experiments with known generating parameters. More concretely, our experiments address three main questions: (i) how emission separation and local cluster size determine the extent to which observation error can translate into errors in state probabilities and latent location distributions; (ii) how splitting a true state into multiple fitted states affects center displacement and the recovery of aggregate transition flows versus individual transition rows; and (iii) how increasing the number of independent trajectories reduces recovery errors and promotes one-to-one matching between fitted and true emission centers in exact-specified setting. To this end, we implement two complementary experimental designs:
The first setup is intended to assess the separation powers in our inverse inequalities independently of estimation and optimization error. In particular, we construct pairs of HMMs, with one serving as the true model and the other as a candidate model. For each pair, we specify both sets of parameters directly to satisfy the moment-matching or joint cancellation conditions used in the sharpness constructions of Propositions D.8 and E.1 . For each fixed cluster size, we evaluate the ratios of latent recovery errors to Hellinger error over a range of emission separations and compare the resulting separation powers with the theoretical predictions. This comparison numerically illustrates the sharpness of the bounds, as ratios growing at the predicted powers show that the separation exponents cannot be reduced uniformly over the corresponding model classes. We repeat this comparison across cluster sizes to examine their effect on these powers. The model constructions and separation grids are detailed in § 5.1 and § 5.2 .
The second setup examines how latent recovery errors and emission-center matching behave when HMMs are estimated from data. Specifically, we generate independent observation trajectories from a known true HMM and fit models with either the true number of states or a larger number. Next, we compare the fitted models with the generating model to evaluate observation and latent recovery errors. At each fixed sample size, we repeat this procedure on independently generated data sets and summarize the errors and the frequency of successful one-to-one center matching under exact specification. The sample-size and matching experiments are presented in § 5.3 , which specifies the generating models, sample-size grids, and numbers of repetitions. The fitting and evaluation procedures are detailed in Appendix § 14 .§ 1 .
To evaluate aggregate recovery, we assign each fitted state to the group associated with its nearest true emission center, following § 2.2 . We sum state probabilities within each group and transition flows between each pair of groups, and compare these quantities with their true counterparts. This grouping is used only to calculate the recovery losses, leaving the fitted emission centers and transition parameters unchanged. Since the true emission centers are unavailable in practice, Appendix § 14 .§ 4 compares grouping based on fitted centers with the nearest-true-center assignment. Appendix § 14 .§ 5 then examines the effect of reference-center error under the stability result in § 4.3 .
Data generation and model fitting. Throughout the estimation experiments in the second setup described above, we generate n n n independent trajectories of fixed length T = 6 T = 6 T = 6 and hold the common emission covariance fixed at its known value during fitting. In these experiments, we generate and fit the models with identity emission covariance and constrain the emission centers to the Euclidean ball B d ( 0 , 6 ) B_d(0, 6) B d ( 0 , 6 ) . With T T T fixed, increasing n n n adds independent trajectories rather than extending individual trajectories, which follows the sampling scheme presented in § 2.1 . For model fitting, we use the Baum–Welch algorithm with four initializations, except for the globally optimized benchmark described in Appendix § 14 .§ 2 . The stopping criteria for each run are an absolute change in log-likelihood divided by n T nT n T of at most 10 − 7 10^{-7} 1 0 − 7 or a maximum of 400 iterations, whichever is reached first. Among the four runs, we select the fit with the highest final training likelihood and include it in the reported results regardless of which stopping condition is met. Table 4 summarizes the experiments and the quantities evaluated.
Table 4. Overview of the synthetic experiments in the main text. Here, h h h denotes the Hellinger distance between the compared observation distributions, and I I I indexes the selected observation times. The recovery losses and Wasserstein distances are defined in § 2.2 .
Evaluation protocol. We evaluate errors in both the observation distribution and the latent structure. The former is measured by Hellinger distance, and the latter by the probability and displacement losses D π D_\pi D π , max t D { t } \max_t D_{\{t\}} max t D { t } , D A D_A D A , D μ D_\mu D μ , V V V , and B B B . For T = 6 T = 6 T = 6 , we evaluate D A D_A D A as the maximum over all five consecutive time pairs. In addition, we consider the selected-time sets { 1 } \{1\} { 1 } , { 1 , 2 } \{1, 2\} { 1 , 2 } , { 1 , 6 } \{1, 6\} { 1 , 6 } , and { 1 , 3 , 6 } \{1, 3, 6\} { 1 , 3 , 6 } to examine marginal, consecutive, nonconsecutive, and three-time recovery. For each selected-time set I I I , we compute D I D_I D I and the corresponding Wasserstein distances between the fitted and true joint latent distributions. These calculations account for dependence across the selected times through the joint probabilities specified by each HMM. We use the normalized block costs d 1 , r d_{1,r} d 1 , r and d 2 , r d_{2,r} d 2 , r , with r = ∣ I ∣ r = |I| r = ∣ I ∣ , to average the transport cost over the selected times.
For the directly specified pairs of HMMs with one-dimensional observations, we compute Hellinger distance using Gauss–Hermite quadrature ([29 ] ) with 80-digit arithmetic. We repeat the calculation with more quadrature points to check that the computed values remain stable. Regarding the estimation experiments, we estimate Hellinger distance using 4096 new trajectories from the true model and another 4096 from the fitted model, independently of the training data. The Monte Carlo and quadrature calculations are detailed in Appendices § 14 .§ 1 and § 14 .§ 2 , respectively.
Separation Exponents under Exact Specification
We first examine how cluster size affects the separation powers in the exact-specification bounds and whether these powers change with the number of selected times. For this purpose, we compare pairs of HMMs with the same number of states, constructed from the moment-matched measures in Proposition D.8 .
Setup. For each s ∈ { 2 , 3 , 4 } s \in\{2, 3, 4\} s ∈ { 2 , 3 , 4 } , we construct a true HMM and a candidate HMM, each with s s s states and one-dimensional Gaussian emissions of variance one. Let a 1 , … , a s a_1, \ldots, a_s a 1 , … , a s and b 1 , … , b s b_1, \ldots, b_s b 1 , … , b s denote their respective unscaled emission centers. The true model has centers a j = j − ( s + 1 ) / 2 a_j = j - (s + 1)/2 a j = j − ( s + 1 ) /2 and state probabilities w j = 1 / s w_j = 1/s w j = 1/ s . The candidate model has positive state probabilities v 1 , … , v s v_1, \ldots, v_s v 1 , … , v s satisfying
∑ j = 1 s v j = 1. \sum_{j=1}^{s} v_j = 1. j = 1 ∑ s v j = 1.
Each probability vector serves as both the initial distribution and every row of the corresponding transition matrix. Thus, the hidden states are independent across time within each model, and state j j j has probability w j w_j w j in the true model and v j v_j v j in the candidate model at every time.
We choose different centers and probabilities for the two models but require their center moments to satisfy
∑ j = 1 s w j a j ℓ ⏟ true center moment = ∑ j = 1 s v j b j ℓ ⏟ candidate center moment , ℓ = 1 , … , 2 s − 2. \underbrace{\sum_{j=1}^{s} w_j a_j^\ell}_{\text{true center moment}
}
=
\underbrace{\sum_{j=1}^{s} v_j b_j^\ell}_{\text{candidate center moment}
},
\qquad\ell= 1,\ldots,2s-2. true center moment j = 1 ∑ s w j a j ℓ = candidate center moment j = 1 ∑ s v j b j ℓ , ℓ = 1 , … , 2 s − 2.
with unequal sums at ℓ = 2 s − 1 \ell= 2s-1 ℓ = 2 s − 1 . These moment equalities make the observation densities difficult to distinguish when the centers approach one another, despite the differences in their latent parameters.
Subsequently, we vary emission separation by setting the true and candidate centers to δ a j \delta a_j δ a j and δ b j \delta b_j δ b j , respectively, keeping the probabilities and emission variance fixed. Smaller values of δ \delta δ bring the centers closer together in both models. The moment equalities are preserved because scaling multiplies both moments of order ℓ \ell ℓ by δ ℓ \delta^\ell δ ℓ . As a result, the difference between the one-time observation densities first appears at order δ 2 s − 1 \delta^{2s-1} δ 2 s − 1 . Evaluating the latent-to-Hellinger error ratios across values of δ \delta δ allows us to compare their separation dependence with the powers predicted by the theory. We use δ ∈ { 0.80 , 0.55 , 0.38 , 0.26 , 0.18 , 0.12 , 0.08 , 0.05 } \delta\in\{0.80, 0.55, 0.38, 0.26, 0.18, 0.12, 0.08, 0.05\} δ ∈ { 0.80 , 0.55 , 0.38 , 0.26 , 0.18 , 0.12 , 0.08 , 0.05 } and fix the trajectory length at T = 6 T = 6 T = 6 . Because the unscaled true centers have minimum separation one and δ < 1 \delta< 1 δ < 1 throughout this grid, the scaled centers satisfy δ s e p = Δ s e p = δ \delta_{\mathrm{sep}} = \Delta_{\mathrm{sep}} = \delta δ sep = Δ sep = δ .
We also compare four-state models whose emission centers form two separated pairs. Starting from the two-state construction above, we define the four true center–probability pairs as
caption:
:::
The candidate model replaces a 1 , a 2 a_1,a_2 a 1 , a 2 by b 1 , b 2 b_1,b_2 b 1 , b 2 and w 1 , w 2 w_1,w_2 w 1 , w 2 by v 1 , v 2 v_1,v_2 v 1 , v 2 in this construction. In each model, these four probabilities form the initial distribution and every transition row. As δ \delta δ decreases, the centers within each pair approach one another, while the two pairs remain separated.
Evaluation. At each value of δ \delta δ , we compute the Hellinger distance h h h between the length-T T T observation distributions of the true and candidate models. We then evaluate the ratios D I / h D_I/h D I / h , D μ / h D_\mu/h D μ / h , and W 1 , I / h W_{1,I}/h W 1 , I / h , which measure the discrepancies in state probabilities and emission locations relative to the observation error. We consider sets I I I containing r = 1 , 2 , 3 r=1,2,3 r = 1 , 2 , 3 distinct times to examine whether the separation powers of D I / h D_I/h D I / h and W 1 , I / h W_{1,I}/h W 1 , I / h change when recovery is evaluated jointly across more times. Since the hidden states are independent and identically distributed in this construction, these losses depend only on the number of selected times, not on their positions. This independence also allows us to obtain the full-block Hellinger distance from the one-time distance h 1 h_1 h 1 through
h 2 = 1 − ( 1 − h 1 2 ) T . h^2 = 1 - (1-h_1^2)^T. h 2 = 1 − ( 1 − h 1 2 ) T .
We then evaluate h 1 h_1 h 1 numerically from the two one-time observation densities. Similarly, the joint state probabilities are products of the one-time probabilities, allowing us to compute D I D_I D I directly from the vectors w w w and v v v . Under the normalized block cost, W 1 , I W_{1,I} W 1 , I equals the one-time Wasserstein distance, which we compute by monotone matching between the two discrete distributions of emission centers.
From these ratios, we estimate the separation powers by fitting a straight line to the logarithm of each ratio against log δ \log\delta log δ over the four smallest separation values, δ ∈ { 0.18 , 0.12 , 0.08 , 0.05 } \delta\in\{0.18,0.12,0.08,0.05\} δ ∈ { 0.18 , 0.12 , 0.08 , 0.05 } . If a ratio behaves as C δ − α C\delta^{-\alpha} C δ − α , its logarithm behaves as log C − α log δ \log C-\alpha\log\delta log C − α log δ , so the negative fitted slope estimates the exponent α \alpha α . We compare the estimated exponents with the values 2 s − 1 2s-1 2 s − 1 for D I / h D_I/h D I / h and 2 s − 2 2s-2 2 s − 2 for both D μ / h D_\mu/h D μ / h and W 1 , I / h W_{1,I}/h W 1 , I / h , predicted by Proposition D.8.
Results. Figure 2 shows that the ratios D π / h D_\pi/h D π / h and W 1 / h W_1/h W 1 / h increase as the emission separation decreases, with a steeper increase for larger emission clusters. This increase illustrates that, for HMM pairs satisfying the assumptions of Theorem 3.1 , a small discrepancy between observation distributions can correspond to much larger discrepancies in latent probabilities and emission locations as the emission centers approach one another. For example, for a single cluster with s max = 3 s_{\max}=3 s m a x = 3 , reducing the separation from 0.8 0.8 0.8 to 0.05 0.05 0.05 increases D π / h D_\pi/h D π / h from approximately 10 2 10^2 1 0 2 to 6 × 10 7 6 \times10^7 6 × 1 0 7 , while W 1 / h W_1/h W 1 / h rises from approximately 1.5 × 10 2 1.5 \times10^2 1.5 × 1 0 2 to 6 × 10 6 6 \times10^6 6 × 1 0 6 . Appendix § 14 .§ 3 complements these comparisons with a separate estimation experiment using four-state true models whose centers form either one cluster or two separated pairs, with the same minimum separation. At the smallest separation considered, the two-pair configuration yields smaller aggregation transition-flow errors despite larger observation-density errors, under both exact specification and over-specification.
Figure 2. Exact specification with different emission-cluster sizes. Left: D π / h D_{\pi}/h D π / h for a single cluster of size s = 2 , 3 , 4 s = 2, 3, 4 s = 2 , 3 , 4 and for four states arranged in two separated pairs. Right: the corresponding W 1 / h W_{1}/h W 1 / h ratios. Each point is computed from a finite moment-matched pair of Gaussian HMMs. The separation slopes are estimated from the four smallest grid values after evaluating the densities.
For the directly specified pairs in Figure 2 , Table 5 quantifies how the growth of the error ratios depends on cluster size by comparing the estimated separation exponents with their theoretical values. For single clusters of size s max = 2 , 3 , 4 s_{\max}=2,3,4 s m a x = 2 , 3 , 4 , the estimated exponents for D π / h D_\pi/h D π / h are 2.992 2.992 2.992 , 4.965 4.965 4.965 , and 6.916 6.916 6.916 , respectively, close to 2 s max − 1 2s_{\max}-1 2 s m a x − 1 . The corresponding estimates for both D μ / h D_\mu/h D μ / h and W 1 / h W_1/h W 1 / h are 1.992 1.992 1.992 , 3.965 3.965 3.965 , and 5.916 5.916 5.916 , close to 2 s max − 2 2s_{\max}-2 2 s m a x − 2 . These estimates show that the probability ratio grows by one additional inverse power of separation compared with the displacement and Wasserstein ratios. In each case, the observed growth is close to the power in the corresponding upper bound, numerically illustrating the sharpness established by Proposition D.8 . The dependence of these powers on cluster size can also be examined while keeping the total state count fixed. In Figure 2 , the four-state model with two separated pairs follows the two-state single-cluster curves closely. Table 5 quantifies this similarity, with probability and Wasserstein exponents of 2.993 and 1.993 for the two-pair configuration, compared with 2.992 and 1.992 for the single two-state cluster. This agreement supports the role of the largest cluster size s max s_{\max} s m a x in Theorem 3.1 , whose structured bounds distinguish these configurations from a single four-state cluster despite their common total state count.
Table 5. Separation exponents under exact specification. Estimated exponents are the negative log–log slopes over the four smallest separation values. The theoretical exponents are 2 s max − 1 2s_{\max}-1 2 s m a x − 1 for D π / h D_{\pi}/h D π / h and 2 s max − 2 2s_{\max}-2 2 s m a x − 2 for D μ / h D_{\mu}/h D μ / h and W 1 / h W_1/h W 1 / h . The shaded row uses two separated pairs; the other rows use a single cluster. Bold estimates highlight the smaller exponents for two separated pairs than for a single four-center cluster at the same K = 4 K=4 K = 4 .
The dependence on cluster size raises a further question of whether joint recovery across more times incurs additional separation powers. Figure 3 shows that, for each fixed cluster size, the curves for r = 1 , 2 , 3 r = 1, 2, 3 r = 1 , 2 , 3 are nearly parallel on logarithmic axes, with some curves nearly overlapping. The similar slopes indicate that increasing the number of selected times does not introduce the stronger separation dependence observed when the cluster size increases. Instead, the differences appear in the multiplicative constants, as reflected in the joint probability bound of Theorem 3.1 , where r r r enters C r C_r C r while the separation exponent remains 2 s max − 1 2s_{\max} - 1 2 s m a x − 1 .
Figure 3. Exact specification across selected times. The ratios D I / h D_I/h D I / h are plotted against Δ s e p \Delta_{\mathrm{sep}} Δ sep for r = ∣ I ∣ ∈ { 1 , 2 , 3 } r = |I| \in\{1,2,3\} r = ∣ I ∣ ∈ { 1 , 2 , 3 } and a single emission cluster of size s max = 2 , 3 , 4 s_{\max} = 2,3,4 s m a x = 2 , 3 , 4 . Here, h h h denotes the Hellinger distance between the length-T T T observation distributions, with T = 6 T = 6 T = 6 . For each cluster size, the separation slopes remain unchanged across r r r , while the multiplicative constants differ. Some lines nearly overlap because these constants are close.
Separation Exponents under Over-Specification
We next examine how the separation dependence of latent recovery changes under over-specification, where several fitted states may represent the same true state. We first consider recovery at a single time to examine the effect of opposing center shifts within these groups, and then study whether recovering aggregate probabilities across several times introduces additional separation powers.
Setup. To distinguish the effect of state splitting from that of recovering probabilities across several times, we consider the following two experiments.
(a) Recovery at a single time. For each s ∈ { 2 , 3 , 4 } s \in\{2,3,4\} s ∈ { 2 , 3 , 4 } , we construct a true HMM with K = s K = s K = s states in one emission cluster and a candidate HMM with k = s + 1 k = s + 1 k = s + 1 states. As in § 5.1 , both models have one-dimensional Gaussian emissions of variance one, and each model uses its state-probability vector as both the initial distribution and every row of its transition matrix. With this choice, the hidden states are independent across time, so we can examine the effect of splitting without introducing temporal dependence.
To represent one true state by two candidate states, we place their centers at c 1 − ϵ c_1 - \epsilon c 1 − ϵ and c 1 + ϵ c_1 + \epsilon c 1 + ϵ , where c 1 c_1 c 1 is their group center and ϵ = 0.01 \epsilon= 0.01 ϵ = 0.01 , and assign half of the group probability to each. Since the two shifts have equal weights and opposite signs, their average location remains c 1 c_1 c 1 . Their first-order contributions to the emission density also cancel, so changes in the individual centers need not produce observation discrepancies of the same order.
To extend this cancellation across the cluster, we adjust the candidate group probabilities and centers to match the moments of the true emission-center distribution through degree 2 s − 1 2s - 1 2 s − 1 . In particular, for true centers and probabilities ( a j , w j ) j = 1 s (a_j,w_j)_{j=1}^{s} ( a j , w j ) j = 1 s and candidate counterparts ( b i , v i ) i = 1 s + 1 (b_i,v_i)_{i=1}^{s+1} ( b i , v i ) i = 1 s + 1 after splitting, we require
∑ j = 1 s w j a j ℓ ⏟ true center moment = ∑ i = 1 s + 1 v i b i ℓ ⏟ candidate center moment , ℓ = 1 , … , 2 s − 1. \underbrace{\sum_{j=1}^{s} w_j a_j^\ell}_{\text{true center moment}} =
\underbrace{\sum_{i=1}^{s+1} v_i b_i^\ell}_{\text{candidate center moment}},
\qquad\ell= 1,\ldots,2s-1. true center moment j = 1 ∑ s w j a j ℓ = candidate center moment i = 1 ∑ s + 1 v i b i ℓ , ℓ = 1 , … , 2 s − 1.
Compared with the exact-specification construction, this choice matches one additional moment. To examine its effect as the centers approach one another, we multiply all centers by δ \delta δ while keeping their probabilities fixed. Under this scaling, the moment equalities remain valid and the distance between the split centers becomes 2 δ ϵ 2\delta\epsilon 2 δ ϵ . We use δ ∈ { 0.80 , 0.55 , 0.38 , 0.26 , 0.18 , 0.12 , 0.08 , 0.05 } \delta\in\{0.80,0.55,0.38,0.26,0.18,0.12,0.08,0.05\} δ ∈ { 0.80 , 0.55 , 0.38 , 0.26 , 0.18 , 0.12 , 0.08 , 0.05 } and fix T = 6 T = 6 T = 6 .
(b) Recovery across several times. To examine joint recovery, we use a true three-state HMM with emission centers ( − δ , 0 , δ ) (-\delta, 0, \delta) ( − δ , 0 , δ ) and independent hidden states, each uniformly distributed over the three labels. We compare its latent emission-center distribution at I = { 1 , … , r } I = \{1, \ldots, r\} I = { 1 , … , r } , for r = 2 , 3 , 4 r = 2, 3, 4 r = 2 , 3 , 4 , with that of an over-specified candidate model. The candidate first selects a sequence of r r r labels, then follows that sequence to generate the first r r r observations. Each possible sequence has its own chain of r r r hidden states, one for each position. This realizes the joint construction in Proposition E.1 as a time-homogeneous HMM.
Each label sequence initially has probability 3 − r 3^{-r} 3 − r . We first change the probabilities of the label sequences, creating differences in the joint latent probabilities. To keep these differences difficult to detect from observations, we also shift the first emission center of each sequence so that the probability changes and center shifts partly compensate in the observation density. These changes follow the formulas in Proposition E.1 , which cancel the low-degree density terms at first order in the perturbation size. We then examine how much the joint latent probabilities and emission-center distributions can differ relative to the resulting observation error. After the first r r r observations, the candidate generates independent observations from the true model, so all observation discrepancy comes from the first r r r times. For each choice of r r r , we vary the emission separation over the same grid as in comparison (a). Throughout each scan, the candidate state count remains fixed at k = r 3 r + 3 k = r3^r + 3 k = r 3 r + 3 , although this count differs across the three choices of r r r .
Evaluation. For experiment (a), we first assign each candidate state to its nearest true center and sum the probabilities within each group. After this aggregation, we compute D π D_{\pi} D π to assess whether the total probability assigned to each true state is recovered. To relate this discrepancy to the observation error, we divide it by the Hellinger distance h h h between the length-T T T observation distributions. In addition to these probability discrepancies, we examine the locations of the candidate centers within each group. Since opposing shifts can cancel when averaged, we retain both the net displacement B B B and the quadratic displacement V V V . For example, if a true center at zero is represented by two equally weighted centers at − u -u − u and u u u , then
B = ∣ 1 2 ( − u ) + 1 2 u ∣ ⏟ magnitude of the average shift = 0 , B = \underbrace{\left|\frac{1}{2}(-u)+\frac{1}{2}u\right|}_{\text{magnitude of the average shift}} = 0, B = magnitude of the average shift 2 1 ( − u ) + 2 1 u = 0 ,
V = 1 2 ( − u ) 2 + 1 2 u 2 ⏟ average squared displacement = u 2 . V = \underbrace{\frac{1}{2}(-u)^2+\frac{1}{2}u^2}_{\text{average squared displacement}} = u^2. V = average squared displacement 2 1 ( − u ) 2 + 2 1 u 2 = u 2 .
Although the average location equals the true center, V V V remains positive because both fitted centers are displaced from it. To account also for differences in the mass assigned to emission locations, we compute the squared Wasserstein distance W 2 2 W_2^2 W 2 2 .
For each separation, we therefore evaluate D π / h D_\pi/h D π / h , B / h B/h B / h , V / h V/h V / h , and W 2 2 / h W_2^2/h W 2 2 / h from the recomputed observation densities and losses. We then estimate their separation exponents over the four smallest grid values, following the logarithmic regression in § 5.1 . According to Theorems 4.2 and 4.4 , the corresponding powers are 2 s 2s 2 s for D π / h D_\pi/h D π / h , 2 s − 1 2s-1 2 s − 1 for B / h B/h B / h , and 2 s − 2 2s-2 2 s − 2 for both V / h V/h V / h and W 2 2 / h W_2^2/h W 2 2 / h .
For experiment (b), let η \eta η control the size of the probability changes and center shifts. At η = 0 \eta=0 η = 0 , the candidate and true models have the same observation and latent emission-center distributions. We measure how rapidly the observation error, joint probability error, and squared Wasserstein error grow as η \eta η increases from zero. Comparing these initial rates of change isolates the effect of small perturbations before we examine their dependence on separation. For each fixed δ \delta δ , we compute
c h ( δ ) = lim η → 0 h η , for observation error, c_h(\delta)=\lim_{\eta\to0}\frac{h}{\eta},
\qquad\text{for observation error,} c h ( δ ) = η → 0 lim η h , for observation error,
c D ( δ ) = lim η → 0 D I η , for joint probability error, c_D(\delta)=\lim_{\eta\to0}\frac{D_I}{\eta},
\qquad\text{for joint probability error,} c D ( δ ) = η → 0 lim η D I , for joint probability error,
c W ( δ ) = lim η → 0 W 2 , I 2 η , for squared Wasserstein error. c_W(\delta)=\lim_{\eta\to0}\frac{W_{2,I}^2}{\eta},
\qquad\text{for squared Wasserstein error.} c W ( δ ) = η → 0 lim η W 2 , I 2 , for squared Wasserstein error.
For each fixed separation δ \delta δ with c h ( δ ) > 0 c_h(\delta)>0 c h ( δ ) > 0 , the coefficient ratios c D ( δ ) / c h ( δ ) c_D(\delta)/c_h(\delta) c D ( δ ) / c h ( δ ) and c W ( δ ) / c h ( δ ) c_W(\delta)/c_h(\delta) c W ( δ ) / c h ( δ ) are the limits of D I / h D_I/h D I / h and W 2 , I 2 / h W_{2,I}^2/h W 2 , I 2 / h , respectively, as η \eta η tends to zero. We therefore compare the same latent-to-observation error ratios as before, but in the limit of small perturbations.
To obtain these ratios, we evaluate c h c_h c h by numerical integration and c D c_D c D directly from the changes in joint probabilities. For c W c_W c W , we solve a finite linear program that redistributes probability mass among emission-location sequences under the normalized block cost, as detailed in Appendix § 14 .§ 2 . We then vary δ \delta δ and compare the resulting ratios with the theoretical powers β r \beta_r β r and β r − 2 \beta_r-2 β r − 2 , where β r = max { 2 s , r ( s − 1 ) + s } \beta_r=\max\{2s,r(s-1)+s\} β r = max { 2 s , r ( s − 1 ) + s } . In the displayed cases with s = 3 s=3 s = 3 and r = 2 , 3 , 4 r=2,3,4 r = 2 , 3 , 4 , these powers are 7 , 9 , 11 7,9,11 7 , 9 , 11 for joint probability error and 5 , 7 , 9 5,7,9 5 , 7 , 9 for squared Wasserstein error. Throughout this comparison, the limit in η \eta η is taken at each fixed separation before different separations are compared.
Results. As shown in Table 6 , permitting a state to split increases the separation exponent for probability recovery. In particular, for s max = 2 , 3 , 4 s_{\max}=2,3,4 s m a x = 2 , 3 , 4 , the estimated exponents for D π / h D_\pi/h D π / h are 3.989 3.989 3.989 , 5.960 5.960 5.960 , and 7.908 7.908 7.908 , respectively, close to the predicted values 4 4 4 , 6 6 6 , and 8 8 8 . Compared with Table 5 , each estimate is approximately one larger than its exact-specification counterpart. For s max = 3 s_{\max}=3 s m a x = 3 , for example, the exponent increases from 4.965 4.965 4.965 to 5.960 5.960 5.960 , so the probability discrepancy grows more rapidly relative to Hellinger error as separation decreases.
Table 6. Separation exponents under over-specification. Each true model has K = s max K = s_{\max} K = s m a x states in one emission cluster, and the fitted model has k = K + 1 k = K + 1 k = K + 1 states. Estimated exponents are the negative log–log slopes over the four smallest separation values. The theoretical exponents for D π / h D_{\pi}/h D π / h , B / h B/h B / h , V / h V/h V / h , and W 2 2 / h W_{2}^{2}/h W 2 2 / h are 2 s max 2s_{\max} 2 s m a x , 2 s max − 1 2s_{\max} - 1 2 s m a x − 1 , 2 s max − 2 2s_{\max} - 2 2 s m a x − 2 , and 2 s max − 2 2s_{\max} - 2 2 s m a x − 2 , respectively. The loss D π D_{\pi} D π compares initial probabilities after aggregating fitted states within each group.
Within the same configuration, however, the location ratios exhibit smaller separation powers. For s max = 3 s_{\max}=3 s m a x = 3 , the estimated exponent decreases from 5.960 5.960 5.960 for D π / h D_\pi/h D π / h to 4.960 4.960 4.960 for B / h B/h B / h and 3.960 3.960 3.960 for both V / h V/h V / h and W 2 2 / h W_2^2/h W 2 2 / h . Across the other cluster sizes, the estimates show the same ordering, close to 2 s max 2s_{\max} 2 s m a x , 2 s max − 1 2s_{\max} - 1 2 s m a x − 1 , and 2 s max − 2 2s_{\max} - 2 2 s m a x − 2 . This agreement with Theorems 4.2 and 4.4 numerically illustrates the sharpness of their one-time separation exponents, with the constructed pairs exhibiting the respective powers for probability, net displacement, and quadratic displacement. Although W 2 2 / h W_{2}^{2}/h W 2 2 / h has approximately the same exponent as W 1 / h W_{1}/h W 1 / h under exact specification, it controls a squared distance and therefore describes a different dependence on observation error.
When recovery is extended across several times, Figure 4 shows a further increase in sensitivity to separation. For both joint probability and squared Wasserstein error, the curves become steeper as r r r increases from 2 to 4. In particular, with s = 3 s = 3 s = 3 , the probability ratios exhibit powers close to 7, 9, and 11, while the squared Wasserstein ratios exhibit powers close to 5, 7, and 9. Each additional selected time therefore contributes approximately two inverse powers of separation in these configurations. Compared with the nearly parallel curves in Figure 3 , this increase shows that joint recovery under over-specification can change the separation exponent itself. The observed powers are close to β r \beta_r β r and β r − 2 \beta_r - 2 β r − 2 in Theorems 4.2 and 4.4 , respectively, which numerically illustrates the sharpness of their joint separation exponents. The additional separation cost with increasing r r r is thus visible in these constructions, whereas r r r enters only the multiplicative constant in the exact-specification bounds.
Figure 4. Joint recovery under over-specification. (a) Joint probability-error ratios c D / c h c_D/c_h c D / c h . (b) Squared Wasserstein-error ratios c W / c h c_W/c_h c W / c h . The true model has three states with emission centers in one cluster. Curves correspond to r = 2 , 3 , 4 r = 2, 3, 4 r = 2 , 3 , 4 selected times. These ratios are the limits of D I / h D_I/h D I / h and W 2 , I 2 / h W_{2,I}^2/h W 2 , I 2 / h , respectively, as the perturbation size tends to zero at each fixed separation.
Recovery Errors and Center Matching across Sample Sizes
The preceding comparisons show how latent discrepancies can grow relative to observation error as emission separation decreases. We now keep the true model fixed and increase the number of independent trajectories to examine whether improved observation fit is accompanied by improved latent recovery. We first evaluate models fitted by multistart EM, then examine location recovery in a simpler setting where global likelihood maximization can be verified. Finally, we investigate whether increasing the sample size makes it possible to pair every fitted emission center with a nearby true center.
Recovery as sample size increases. We generate data from a Gaussian HMM with three states, emission centers ( − 2 , 0 , 2 ) (-2, 0, 2) ( − 2 , 0 , 2 ) , variance one, and
π ∗ = ( 0.2 , 0.5 , 0.3 ) , A ∗ = ( 0.85 0.10 0.05 0.10 0.80 0.10 0.05 0.15 0.80 ) . \pi^{*} = (0.2, 0.5, 0.3), \qquad
A^{*} =
\begin{pmatrix}
0.85 & 0.10 & 0.05 \\
0.10 & 0.80 & 0.10 \\
0.05 & 0.15 & 0.80
\end{pmatrix}. π ∗ = ( 0.2 , 0.5 , 0.3 ) , A ∗ = 0.85 0.10 0.05 0.10 0.80 0.15 0.05 0.10 0.80 .
For each n ∈ { 128 , 512 , 2048 } n \in\{128, 512, 2048\} n ∈ { 128 , 512 , 2048 } , we generate 24 independent data sets, each containing n n n trajectories of length T = 6 T = 6 T = 6 . To compare exact specification and over-specification under the same observations, we fit models with k = 3 , 4 , 5 k = 3, 4, 5 k = 3 , 4 , 5 to every data set. For each model, we use four initializations based on the observations and retain the parameter with the largest final training likelihood.
For these fitted models, we evaluate the Hellinger error over the full trajectory and the aggregate transition-flow error D A D_A D A over all five consecutive time pairs. To assess emission-center recovery, we compute the Wasserstein distances between the fitted and true emission-center distributions at each of the six times and take the maximum within each fit. We also report R r o w R_{\mathrm{row}} R row , the largest ℓ 1 \ell_1 ℓ 1 discrepancy between a fitted state’s transition probabilities into the true-center groups and the transition row of its assigned true state. Different from D A D_A D A , this diagnostic retains each fitted source state separately. Figure 5 shows how the location errors decrease with sample size, using W 1 W_1 W 1 under exact specification and W 2 W_2 W 2 under over-specification to follow the corresponding theoretical bounds. Table 7 additionally reports both distances for every fitted state count, allowing comparisons across state counts using the same loss. All summaries are means over the 24 data sets, and the figure includes pointwise 95% bootstrap intervals for these means.
Table 7. Statistical recovery as the number of trajectories increases. The true HMM has three states, and each data set contains n n n independent trajectories of length T = 6 T = 6 T = 6 . Entries are means over 24 independent data sets, shared across k = 3 , 4 , 5 k = 3, 4, 5 k = 3 , 4 , 5 . The loss D A D_A D A measures aggregate transition-flow error, while R r o w R_{\mathrm{row}} R row measures the largest individual-row discrepancy after destination states are grouped. Each Wasserstein error is maximized over the six times within each fit before averaging over data sets. Bold entries in the shaded rows show that, at n = 2048 n = 2048 n = 2048 , individual transition-row errors remain much larger than aggregate transition-flow errors when fitting k = 4 , 5 k = 4, 5 k = 4 , 5 states to the true three-state model.
Figure 5. Observation and location errors as sample size increases. Data are generated from a three-state HMM with n n n independent trajectories of length T = 6 T=6 T = 6 . (a) Mean Hellinger error for k = 3 , 4 , 5 k=3,4,5 k = 3 , 4 , 5 . (b) Mean maximum Wasserstein error over the six times, using W 1 W_1 W 1 for k = 3 k=3 k = 3 and W 2 W_2 W 2 for k = 4 , 5 k=4,5 k = 4 , 5 . The maximum is computed within each fit before averaging over 24 independent data sets. Shaded bands show pointwise 95% bootstrap confidence intervals.
As shown in Figure 5 , both observation error and Wasserstein error decrease as more trajectories become available. Under exact specification, Table 7 shows that the mean Hellinger error decreases from 0.107 at n = 128 n = 128 n = 128 to 0.024 at n = 2048 n = 2048 n = 2048 , while the mean maximum W 1 W_1 W 1 error decreases from 0.268 to 0.067. Over the same sample sizes, the mean aggregate-flow error falls from 0.196 to 0.047. The improved observation fit is therefore accompanied by more accurate recovery of emission locations and transitions between the true states. Under over-specification, the errors also decrease, although the additional states incur a recovery cost in this experiment. For k = 5 k = 5 k = 5 , the mean Hellinger error falls from 0.155 to 0.040, while D A D_A D A falls from 0.290 to 0.074 and the mean maximum W 2 W_2 W 2 error from 0.661 to 0.349. At n = 2048 n = 2048 n = 2048 , the mean maximum W 1 W_1 W 1 error is 0.150, compared with 0.067 under exact specification, and the aggregate-flow error also remains larger. These comparisons show that allowing more fitted states can affect latent recovery even when all models achieve small observation errors. The remaining individual-row discrepancies in Table 7 motivate the constructions in § 5.4 , which show that these discrepancies can persist even when aggregate transition flows are recovered exactly.
Sample-size rates for emission-center recovery. To validate the n − 1 / 2 n^{-1/2} n − 1/2 and n − 1 / 4 n^{-1/4} n − 1/4 rates for emission-center recovery in Remark E.2 , we next compare the global maximum likelihood estimators in two simple Gaussian models. More concretely, we generate independent observations from N ( 0 , 1 ) \mathcal{N}(0, 1) N ( 0 , 1 ) and compare fitting a single Gaussian N ( m , 1 ) \mathcal{N}(m, 1) N ( m , 1 ) , with m ∈ [ − 6 , 6 ] m \in[-6, 6] m ∈ [ − 6 , 6 ] , against fitting the symmetric mixture 1 2 N ( − a , 1 ) + 1 2 N ( a , 1 ) \frac{1}{2}\mathcal{N}(-a, 1) + \frac{1}{2}\mathcal{N}(a, 1) 2 1 N ( − a , 1 ) + 2 1 N ( a , 1 ) , with a ∈ [ 0 , 6 ] a \in[0, 6] a ∈ [ 0 , 6 ] . The first model is exactly specified, whereas the second permits the true center at zero to split into two fitted centers. Both models can be represented as HMMs with independent hidden states. For each n ∈ { 128 , 512 , 2048 , 8192 } n \in\{128, 512, 2048, 8192\} n ∈ { 128 , 512 , 2048 , 8192 } , we generate 256 independent data sets, each containing n n n trajectories of length T = 6 T = 6 T = 6 , or equivalently 6 n 6n 6 n independent scalar observations. The global MLE in the single-Gaussian model is the sample mean projected onto [ − 6 , 6 ] [-6, 6] [ − 6 , 6 ] . In the symmetric-mixture model, we obtain the global maximizer by optimizing over the scalar a a a , as detailed in Appendix § 14 .§ 2 . Since the true emission-center distribution is concentrated at zero, the corresponding recovery errors are W 1 = ∣ m ^ ∣ W_1 = |\widehat{m}| W 1 = ∣ m ∣ under exact specification and W 2 = a ^ W_2 = \widehat{a} W 2 = a under over-specification. We compare their empirical means with reference curves proportional to n − 1 / 2 n^{-1/2} n − 1/2 and n − 1 / 4 n^{-1/4} n − 1/4 , respectively. The curves are scaled to pass through the first empirical mean in each case, allowing us to assess the predicted decay rates without specifying their multiplicative constants.
Figure 6 (a) shows that the single-Gaussian error closely follows the n − 1 / 2 n^{-1/2} n − 1/2 reference, while the symmetric-mixture error decreases more slowly, broadly following the n − 1 / 4 n^{-1/4} n − 1/4 reference. These results support the theoretical distinction between exact specification and over-specification: allowing a true center to split can slow emission-center recovery even when the fitted parameters globally maximize the likelihood.
Figure 6. Location recovery and emission-center matching. (a) Mean W 1 W_1 W 1 error for a single-Gaussian fit and mean W 2 W_2 W 2 error for a symmetric two-Gaussian fit to data from N ( 0 , 1 ) \mathcal{N}(0,1) N ( 0 , 1 ) , using global MLEs over 256 independent data sets at each n n n . Dashed curves show the n − 1 / 2 n^{-1/2} n − 1/2 and n − 1 / 4 n^{-1/4} n − 1/4 reference rates, respectively, scaled to the first empirical means. (b) Matching success rates over 16 independent data sets at each n n n and separation, with tolerance Δ s e p / 4 \Delta_{\mathrm{sep}}/4 Δ sep /4 . Shaded bands give 95% bootstrap intervals in (a) and 95% Wilson intervals in (b).
Emission-center matching. While the preceding comparison measures the size of the location error, we also examine whether every true center has a nearby fitted counterpart. We generate two-state Gaussian HMMs with variance one, initial probabilities ( 0.4 , 0.6 ) (0.4,0.6) ( 0.4 , 0.6 ) , transition matrix
A ∗ = ( 0.9 0.1 0.15 0.85 ) , A^*=\begin{pmatrix}0.9 & 0.1 \\ 0.15 & 0.85\end{pmatrix}, A ∗ = ( 0.9 0.15 0.1 0.85 ) ,
and emission centers ( − δ s e p / 2 , δ s e p / 2 ) (-\delta_{\mathrm{sep}}/2,\delta_{\mathrm{sep}}/2) ( − δ sep /2 , δ sep /2 ) , where δ s e p ∈ { 0.5 , 1 , 1.5 } \delta_{\mathrm{sep}}\in\{0.5,1,1.5\} δ sep ∈ { 0.5 , 1 , 1.5 } is their minimum separation. For each gap and n ∈ { 128 , 512 , 2048 , 8192 } n\in\{128,512,2048,8192\} n ∈ { 128 , 512 , 2048 , 8192 } , we fit two-state models to 16 independent data sets of trajectories with length T = 6 T=6 T = 6 . For each fit, we check whether a permutation pairs both fitted centers with the true centers within distance Δ s e p / 4 \Delta_{\mathrm{sep}}/4 Δ sep /4 , where Δ s e p = min { 1 , δ s e p } \Delta_{\mathrm{sep}}=\min\{1,\delta_{\mathrm{sep}}\} Δ sep = min { 1 , δ sep } .
We report the proportion satisfying this location condition on the right of Figure 6 , with Wilson intervals over the 16 repetitions. 4 At n = 128 n=128 n = 128 , only 2 of 16 fits satisfy the location condition for gap 0.5, compared with 13 of 16 for gap 1 and all 16 for gap 1.5. For the smallest gap, the frequency increases to 10/16 at n = 512 n=512 n = 512 , 11/16 at n = 2048 n=2048 n = 2048 , and 16/16 at n = 8192 n=8192 n = 8192 . The closely spaced centers therefore require more observations before matching becomes reliable in this experiment.
Aggregate-Flow and Individual-Row Recovery
The experiment presented in § 5.3 shows that aggregate transition-flow error decreases with sample size. Under over-specification, however, several fitted states may represent the same true state, and accurate transitions between their groups need not determine the transition probabilities of each fitted state. We examine this distinction using the same estimated models and then construct an exact example to explain why individual-row ambiguity can persist even when observation error vanishes.
Empirical comparison. We reuse the k = 4 , 5 k=4,5 k = 4 , 5 fits from § 5.3 and assign each fitted state to its nearest true emission center. To compare individual transition rows, we combine destination states within each group while retaining every fitted source state separately. With the nearest-center assignment c ( i ) c(i) c ( i ) defined in § 2.2 , we compute
R r o w = max i ∈ [ k ] ∑ j = 1 K ∣ ∑ ℓ : c ( ℓ ) = j A i ℓ ⏟ transition probability from fitted state i to group j − A c ( i ) , j ⋆ ⏟ corresponding true transition probability ∣ R_{\mathrm{row}}=\max_{i\in[k]}\sum_{j=1}^{K}\left|\underbrace{\sum_{\ell:c(\ell)=j}A_{i\ell}}_{\text{transition probability from fitted state }i\text{ to group }j}-\underbrace{A^\star_{c(i),j}}_{\text{corresponding true transition probability}}\right| R row = i ∈ [ k ] max j = 1 ∑ K transition probability from fitted state i to group j ℓ : c ( ℓ ) = j ∑ A i ℓ − corresponding true transition probability A c ( i ) , j ⋆
This diagnostic considers whether every fitted source state has the correct probabilities of moving to the true-center groups. To assess transitions at the group level, D A D_A D A instead combines both source and destination states, weighting each source state by its occupancy probability. It compares the resulting matrices of consecutive group probabilities, with the maximum taken over all five consecutive time pairs. Since states in the same source group can have opposing row discrepancies, these discrepancies may average out in the aggregate flow while remaining visible in R r o w R_{\mathrm{row}} R row . Both quantities are evaluated under the original fitted HMM.
Figure 7 shows that the aggregate-flow error decreases substantially with sample size, whereas the individual-row discrepancy remains appreciable. For k = 4 k=4 k = 4 , the mean D A D_A D A decreases from 0.255 to 0.061 as n n n increases from 128 to 2048, while the mean R r o w R_{\mathrm{row}} R row decreases from 0.441 to 0.319. For k = 5 k=5 k = 5 , the corresponding changes are from 0.290 to 0.074 and from 0.663 to 0.421. By comparison, the exact-fit row discrepancy in Table 7 falls from 0.139 to 0.031. More observations therefore improve aggregate recovery without producing a comparable improvement in the worst individual row of the over-specified models.
Figure 7. Aggregate-flow and individual-row recovery under over-specification. Results use the k = 4 , 5 k = 4, 5 k = 4 , 5 fits from the three-state experiment in § 5.3 . Each curve reports a mean over the same 24 independent data sets, with pointwise 95% bootstrap confidence intervals. The loss D A D_A D A combines source and destination states into groups and weights sources by their occupancies. In contrast, R r o w R_{\mathrm{row}} R row combines destinations but retains each fitted source state separately, taking the largest row discrepancy. All fitted source states enter this diagnostic.
Individual-row ambiguity. In Example 4.1 , summing transition probabilities over destination substates gives the correct group-transition probabilities for every fitted source state. The following construction shows that these probabilities can also differ from their true counterparts even when the fitted and true observation distributions are identical. Consider a true two-state HMM with distinct centers a , b a, b a , b , initial probabilities ( 1 / 2 , 1 / 2 ) (1/2, 1/2) ( 1/2 , 1/2 ) , and both transition rows equal to ( 1 / 2 , 1 / 2 ) (1/2, 1/2) ( 1/2 , 1/2 ) . We split the state at a a a into two candidate states and set
π ~ = ( 1 / 4 , 1 / 4 , 1 / 2 ) , A ~ = ( 1 / 2 1 / 2 0 0 0 1 1 / 4 1 / 4 1 / 2 ) , μ ~ = ( a , a , b ) . \widetilde{\pi} = (1/4,1/4,1/2), \qquad\widetilde{A} = \begin{pmatrix} 1/2 & 1/2 & 0 \\ 0 & 0 & 1 \\ 1/4 & 1/4 & 1/2 \end{pmatrix}, \qquad\widetilde{\mu} = (a,a,b). π = ( 1/4 , 1/4 , 1/2 ) , A = 1/2 0 1/4 1/2 0 1/4 0 1 1/2 , μ = ( a , a , b ) .
The candidate model replaces true state 1, whose emission center is a a a , by candidate states 1 and 2 with the same center. Candidate state 3 has center b b b and corresponds to true state 2. We therefore place candidate states 1 and 2 in group 1 and candidate state 3 in group 2. Although candidate states 1 and 2 have the same emission distribution, their transition probabilities differ. State 1 moves to group 1 with probability one, whereas state 2 moves to group 2 with probability one. Their group-transition rows are ( 1 , 0 ) (1, 0) ( 1 , 0 ) and ( 0 , 1 ) (0, 1) ( 0 , 1 ) , both different from the true row ( 1 / 2 , 1 / 2 ) (1/2, 1/2) ( 1/2 , 1/2 ) .
However, the transition probabilities between groups are still ( 1 / 2 , 1 / 2 ) (1/2, 1/2) ( 1/2 , 1/2 ) , exactly as in the true model. The two candidate states in group 1 have equal initial probabilities, and every row of A ~ \widetilde{A} A assigns equal probabilities to entering them. Conditional on any group history up to the current time that ends in group 1, the two states therefore have probabilities 1 / 2 1/2 1/2 each. The next-group probabilities are 1 2 ( 1 , 0 ) + 1 2 ( 0 , 1 ) = ( 1 / 2 , 1 / 2 ) \frac{1}{2}(1, 0) + \frac{1}{2}(0, 1) = (1/2, 1/2) 2 1 ( 1 , 0 ) + 2 1 ( 0 , 1 ) = ( 1/2 , 1/2 ) . Candidate state 3 also moves to the two groups with probabilities ( 1 / 2 , 1 / 2 ) (1/2, 1/2) ( 1/2 , 1/2 ) . Since the initial group probabilities are ( 1 / 2 , 1 / 2 ) (1/2, 1/2) ( 1/2 , 1/2 ) , the grouped process has exactly the same distribution as the true hidden-state process, despite the differences in the individual candidate transition rows.
Because the grouped process has the same distribution as the true hidden-state process and each group has the corresponding true emission distribution, the two models induce the same joint observation distribution for every sequence length. Hence h = 0 h = 0 h = 0 , D A = 0 D_A = 0 D A = 0 , and all aggregate path discrepancies vanish, but R r o w = 1 R_{\mathrm{row}} = 1 R row = 1 . The transition probabilities of individual split states therefore remain unidentifiable, even though both models have identical observation distributions and aggregate state-sequence probabilities. Moreover, every candidate state has positive occupancy at every time. The row discrepancy therefore reflects ambiguity in how a true state is represented by several fitted states, even when neither observation error nor vanishing occupancy is present.
The previous example shows that aggregate transition flows can be recovered even when individual fitted rows remain ambiguous. A separate question is whether these flows determine aggregate state-sequence probabilities at more than two times. In the eight-state construction of Example D.9, direct path enumeration gives zero discrepancy on every subset of at most three of the first four times, while D { 1 , 2 , 3 , 4 } = 1 D_{\{1,2,3,4\}} = 1 D { 1 , 2 , 3 , 4 } = 1 . The discrepancy is therefore visible only when all four times are considered jointly. This explains why the over-specification analysis controls D I D_I D I directly for the selected times, rather than reconstructing all joint probabilities from aggregate transition flows.
Practical Implications
Our recovery bounds distinguish the accuracy of the observation density from that of the latent quantities used to interpret a fitted Gaussian HMM. The relation between them depends on emission geometry and on whether the recovery targets are individual-state parameters or aggregate probabilities and transition flows. These results provide a theoretical basis for interpreting individual fitted states and transition probabilities in practical applications, and for assessing whether groups of states with similar emissions retain stable aggregate probabilities and transition flows across fits.
Interpreting individual states and transition probabilities. In practice, emission geometry, matching stability across fits, and state probabilities guide the interpretation of individual fitted states and the estimated transition probabilities between them.
Emission geometry. Since a close fit to the observations can coexist with substantial latent recovery error, practitioners should examine the geometry of the fitted emission centers before assigning separate interpretations to their states. As illustrated by Figure 1 and quantified in Theorems 3.1 and 3.6 , configurations with the same minimum separation can have different separation exponents when different numbers of centers approach one another. Accordingly, this examination should consider both the distances between centers and the sizes of groups of close centers. For an empirical summary of these groups, complete-linkage clustering provides control of the within-group diameter. By contrast, a distance-threshold graph can join a chain of close pairs whose overall diameter is large, so its connected components need not satisfy the theoretical diameter condition.
Matching across fits. Under exact specification, Corollary 3.5 gives a sufficient density-error condition for matching every fitted center to a true center. Since the true centers are unavailable, practitioners can examine whether the correspondence between fitted centers remains stable across bootstrap resamples, random initializations, or small changes in the training data, after accounting for label permutations. If this correspondence repeatedly changes, or centers repeatedly merge and split despite comparable observation fit, separate state interpretations require additional evidence.
Transition probabilities. In addition to matching emission centers across fits, practitioners should examine estimated state occupancies and transition flows before interpreting individual transition rows. This is especially important for rarely occupied states, since Corollary 3.8 shows that recovering individual rows from flows introduces an inverse initial-state probability factor. Consequently, accurate flow recovery can provide a weaker guarantee for the outgoing transition probabilities of such states. To assess whether these probabilities support a stable transition pattern, practitioners can compare them across the same fits used to examine center matching and report the corresponding occupancies and transition flows alongside the rows.
Interpreting groups of similar states. Under over-specification, several fitted states may represent the same true emission center, so their individual transition rows can differ even when the observation distributions are identical. As § 5.4 shows, this ambiguity persists even when every fitted state has positive occupancy, so interpreting each fitted state separately may obscure the behavior shared by the group. Instead, practitioners can examine aggregate probabilities, transition flows, and emission-center distributions, whose recovery is controlled by Theorems 4.2 and 4.4 . For interpretations involving several times, this examination should also include the joint probabilities of the corresponding group sequences: grouping can destroy the Markov property, so transition flows between groups alone need not determine these probabilities, as illustrated by Example D.9 .
To compare these aggregate quantities, our theoretical losses group fitted states by assigning each to its nearest true emission center. As these centers are unavailable in practice, Proposition 4.8 bounds the additional grouping error caused by replacing them with sufficiently accurate estimated reference centers. Under its reference accuracy condition, the distance margins identify fitted states whose group assignments may change under reference perturbations. Practitioners can therefore perturb the proposed reference centers and compare the resulting aggregate probabilities and transition flows to assess how sensitive their interpretation is to the chosen grouping.
These implications are not specific to a single fitting procedure. Because the inverse inequalities apply to any fitted estimator with suitable observation-density error control, the same reasoning can inform the interpretation of models obtained by maximum likelihood, Baum–Welch, spectral or moment-based methods, and Bayesian procedures.
Conclusion
In this paper, we establish latent recovery guarantees for multivariate Gaussian HMMs with a common, known emission covariance matrix. The guarantees relate observation-density error to latent recovery error through inverse powers of emission separation, making explicit how recovery becomes more demanding as emission centers approach one another. For mutually separated clusters, the shared emission centers across time allow us to obtain bounds whose separation exponents depend on the largest cluster of approaching centers rather than the total number of states or the exponentially many hidden paths. These bounds therefore distinguish configurations with the same minimum separation and quantify the advantage of smaller, mutually separated clusters. Under exact specification, direct transport interpolation and the Markov factorization yield global bounds with sharp separation exponents 2 s max − 1 2s_{\max}-1 2 s m a x − 1 for state-sequence probability errors and 2 s max − 2 2s_{\max}-2 2 s m a x − 2 for W 1 W_1 W 1 recovery of joint emission-center distributions, neither of which increases with the number of selected times. Without cluster assumptions, the same bounds hold with s max = K s_{\max}=K s m a x = K . When the observation-density error is sufficiently small, these bounds also guarantee emission-center matching, after which the probability and Wasserstein recovery bounds have constants independent of the minimum true initial probability. Under over-specification, several fitted states may represent the same true state, so we instead control aggregate probabilities, transition flows, and emission-center distributions that accommodate state splitting. In this case, the separation exponents for joint probability and W 2 W_2 W 2 recovery can increase with the number of selected times. Since these inverse bounds apply to any estimator with suitable observation-density error control, they also provide a basis for deriving statistical recovery rates. For the global MLE, the Hellinger estimation guarantee yields latent recovery rates of the form O P ( Δ s e p − α r [ d log ( n T d ) n ] ν r / 2 ) \mathcal{O}_{\mathrm{P}}\left(\Delta_{\mathrm{sep}}^{-\alpha_r}\left[\frac{d\log(nTd)}{n}\right]^{\nu_r/2}\right) O P ( Δ sep − α r [ n d l o g ( n T d ) ] ν r /2 ) , with explicit separation exponents determined by cluster size, the recovery target, and model specification. In these rates, the sample-size dependence is n − 1 / 2 n^{-1/2} n − 1/2 for errors in the probabilities of individual states and state sequences and in transition flows under exact specification, up to logarithmic factors. Under over-specification, the corresponding grouped probability and transition-flow bounds retain this dependence. By comparison, the bounds for probability-weighted emission-center displacement and Wasserstein error involve n − 1 / 2 n^{-1/2} n − 1/2 under exact specification using W 1 W_1 W 1 , and n − 1 / 4 n^{-1/4} n − 1/4 under over-specification using W 2 W_2 W 2 . Moreover, the underlying density guarantee remains valid with coincident fitted centers and zero fitted probabilities, so the statistical recovery analysis also covers these degenerate fitted models. To examine the separation dependence of the recovery guarantees, we first compare controlled model pairs with directly specified parameters. These comparisons illustrate how the predicted separation powers depend on cluster size and the number of selected times. We then examine recovery from data as the sample size increases, including emission-center matching under exact specification. Under over-specification, however, the fitted models show that decreasing aggregate-flow error can coexist with substantial individual-row discrepancies. Indeed, our constructions show that these discrepancies can persist even when the fitted and true observation distributions are identical. Beyond these statistical guarantees and empirical findings, our analysis offers a quantitative foundation for interpreting latent regimes in applications such as genomics, biophysics, and neuroscience. In such settings, the scientific value of a fitted HMM lies in what its states reveal about underlying observation patterns and their temporal organization. This perspective supports more informed interpretation of fitted models and lays the groundwork for future research on prediction and conditional inference with explicit latent recovery guarantees.
Supplement for “The Statistical Geometry of Latent Recovery in Hidden Markov Models”
Gaussian moment extraction and elementary transport
This appendix section establishes the analytic tools used in the inverse bounds. We first justify the reduction to identity emission covariance and compare transport distances before and after the transformation. We then show that the full-block Hellinger error controls polynomial moments of the latent locations, with constants independent of the ambient observation dimension and the number of possible latent paths. Finally, we relate errors in locations and aggregate probabilities to Wasserstein distances through a coupling on the true support.
Covariance normalization and comparison of transport metrics
We first verify that the covariance normalization in § 2.1 preserves Hellinger distance and likelihood maximizers. The following lemma also compares Wasserstein distances before and after the transformation.
Lemma A.1 (Whitening reduction for Gaussian HMMs). Let Σ ∈ R d × d \Sigma\in\mathbb{R}^{d \times d} Σ ∈ R d × d be a fixed symmetric positive-definite matrix. Consider Gaussian HMMs with parameter θ = ( π , A , μ 1 , … , μ k ) \theta= (\pi,A,\mu_1,\ldots,\mu_k) θ = ( π , A , μ 1 , … , μ k ) and common emission covariance Σ \Sigma Σ . Define S = Σ − 1 / 2 S = \Sigma^{-1/2} S = Σ − 1/2 , θ ~ = ( π , A , S μ 1 , … , S μ k ) \widetilde{\theta} = (\pi,A,S\mu_1,\ldots,S\mu_k) θ = ( π , A , S μ 1 , … , S μ k ) , and, for every r ≥ 1 r \ge1 r ≥ 1 , let S r ( x 1 , … , x r ) = ( S x 1 , … , S x r ) S_r(x_1,\ldots,x_r) = (Sx_1,\ldots,Sx_r) S r ( x 1 , … , x r ) = ( S x 1 , … , S x r ) . Write p θ , r Σ p_{\theta,r}^{\Sigma} p θ , r Σ and p θ ~ , r I d p_{\widetilde{\theta},r}^{I_d} p θ , r I d for the corresponding length-r r r observation densities. Then Y t = S X t Y_t = SX_t Y t = S X t is a Gaussian HMM with parameter θ ~ \widetilde{\theta} θ with common emission covariance I d I_d I d , and
p θ ~ , r I d ( y ) = ∣ det S ∣ − r p θ , r Σ ( S r − 1 y ) . p_{\widetilde{\theta},r}^{I_d}(y) = \lvert\det S\rvert^{-r}p_{\theta,r}^{\Sigma}(S_r^{-1}y). p θ , r I d ( y ) = ∣ det S ∣ − r p θ , r Σ ( S r − 1 y ) . Moreover, the following conclusions hold.
(Hellinger invariance) For any two parameters θ , θ ′ \theta,\theta' θ , θ ′ and every r ≥ 1 r \ge1 r ≥ 1 ,
h ( p θ , r Σ , p θ ′ , r Σ ) = h ( p θ ~ , r I d , p θ ~ ′ , r I d ) . h\left(p_{\theta,r}^{\Sigma},p_{\theta',r}^{\Sigma}\right)
=
h\left(p_{\widetilde{\theta},r}^{I_d},p_{\widetilde{\theta}',r}^{I_d}\right). h ( p θ , r Σ , p θ ′ , r Σ ) = h ( p θ , r I d , p θ ′ , r I d ) . (Correspondence of likelihood maximizers) For n n n independent length-T T T trajectories, let L X ( θ ) L_X(\theta) L X ( θ ) and L Y ( θ ~ ) L_Y(\widetilde{\theta}) L Y ( θ ) denote the likelihoods based on the original and transformed observations, respectively. Then
L Y ( θ ~ ) = ∣ det S ∣ − n T L X ( θ ) . L_Y(\widetilde{\theta}) = \lvert\det S\rvert^{-nT}L_X(\theta). L Y ( θ ) = ∣ det S ∣ − n T L X ( θ ) . Consequently, for any parameter class Θ \Theta Θ and its transformed image Θ ~ = { θ ~ : θ ∈ Θ } \widetilde{\Theta} = \{\widetilde{\theta}:\theta\in\Theta\} Θ = { θ : θ ∈ Θ } , the map θ ↦ θ ~ \theta\mapsto\widetilde{\theta} θ ↦ θ induces a bijection between the sets of global likelihood maximizers over Θ \Theta Θ and Θ ~ \widetilde{\Theta} Θ .
(Transport identity and comparison) Fix p ∈ { 1 , 2 } p \in\{1,2\} p ∈ { 1 , 2 } and r ≥ 1 r \ge1 r ≥ 1 . Throughout this lemma, W p , r W_{p,r} W p , r is the Wasserstein distance associated with d p , r ( x , y ) = { r − 1 ∑ ℓ = 1 r ∥ x ℓ − y ℓ ∥ 2 p } 1 / p d_{p,r}(x,y) = \left\{r^{-1}\sum_{\ell=1}^{r}\lVert x_\ell-y_\ell\rVert_2^p\right\}^{1/p} d p , r ( x , y ) = { r − 1 ∑ ℓ = 1 r ∥ x ℓ − y ℓ ∥ 2 p } 1/ p , exactly as in equation (12) . Let G , H G,H G , H be probability measures on ( R d ) r (\mathbb{R}^d)^r ( R d ) r with finite p p p th moments, and define
G ~ = ( S r ) # G , H ~ = ( S r ) # H . \widetilde{G} = (S_r)_{\#}G,\qquad\widetilde{H} = (S_r)_{\#}H. G = ( S r ) # G , H = ( S r ) # H . Then, by defining λ − = λ min ( Σ ) \lambda_- = \lambda_{\min}(\Sigma) λ − = λ m i n ( Σ ) and λ + = λ max ( Σ ) \lambda_+ = \lambda_{\max}(\Sigma) λ + = λ m a x ( Σ ) , we have
1 λ + W p , r ( G , H ) ≤ W p , r ( G ~ , H ~ ) ≤ 1 λ − W p , r ( G , H ) . (46) \frac{1}{\sqrt{\lambda_+}}W_{p,r}(G,H)
\le W_{p,r}(\widetilde{G},\widetilde{H})
\le\frac{1}{\sqrt{\lambda_-}}W_{p,r}(G,H). \tag*{(46)} λ + 1 W p , r ( G , H ) ≤ W p , r ( G , H ) ≤ λ − 1 W p , r ( G , H ) . (46)
Proof. The transformation leaves the hidden chain unchanged and takes each conditional emission N d ( μ i , Σ ) N_d(\mu_i,\Sigma) N d ( μ i , Σ ) to N d ( S μ i , I d ) N_d(S\mu_i,I_d) N d ( S μ i , I d ) . The block change of variables has Jacobian ∣ det S ∣ r \lvert\det S\rvert^r ∣ det S ∣ r , giving the asserted density identity. Applying this identity to both densities in the Hellinger affinity proves invariance; applying it to n n n independent length-T T T trajectories multiplies every likelihood by the same factor ∣ det S ∣ − n T |\det S|^{-nT} ∣ det S ∣ − n T . Hence maximizers correspond bijectively over a parameter class and its transformed image .
For the transport comparison, the eigenvalue bounds give, for any x , y x,y x , y ,
λ + − p / 2 d p , r ( x , y ) p ≤ d p , r ( S r x , S r y ) p ≤ λ − − p / 2 d p , r ( x , y ) p . \lambda_{+}^{-p/2}d_{p,r}(x,y)^p \le d_{p,r}(S_rx,S_ry)^p \le\lambda_{-}^{-p/2}d_{p,r}(x,y)^p. λ + − p /2 d p , r ( x , y ) p ≤ d p , r ( S r x , S r y ) p ≤ λ − − p /2 d p , r ( x , y ) p .
The map γ ↦ ( S r , S r ) # γ \gamma\mapsto(S_r,S_r)_{\#}\gamma γ ↦ ( S r , S r ) # γ is a bijection from Π ( G , H ) \Pi(G,H) Π ( G , H ) onto Π ( ( S r ) # G , ( S r ) # H ) \Pi((S_r)_{\#}G,(S_r)_{\#}H) Π (( S r ) # G , ( S r ) # H ) , because S r S_r S r is invertible. Integrate the pointwise inequality, take infima over couplings and then take p p p th roots. This proves (A.1), with the same normalized factor 1 / r 1/r 1/ r for p = 1 p=1 p = 1 and p = 2 p=2 p = 2 . □
For fixed Σ \Sigma Σ , the lemma transfers transport bounds between the original and whitened coordinates with only a change in constants. Throughout the subsequent proofs, separation, cluster geometry, radius constraints, and nearest-center assignments refer to the whitened centers. In the original coordinates, these correspond to the Mahalanobis geometry induced by Σ − 1 \Sigma^{-1} Σ − 1 . In particular, nearest-center assignments after whitening need not agree with those defined by the original Euclidean metric.
Hellinger distance decreases under marginalization
The inverse bounds use observation distributions at selected times within the observed block. To this end, the following lemma shows that the Hellinger error at any selection of times is bounded above by the Hellinger error of the full observation block.
Lemma A.2 (Marginalization). Let p p p and q q q be joint densities, and let p I p_I p I and q I q_I q I be their marginals on a subset I I I of coordinates. Then
h ( p I , q I ) ≤ h ( p , q ) . h(p_I,q_I) \le h(p,q). h ( p I , q I ) ≤ h ( p , q ) . Proof. Write the retained and discarded coordinates as ( x , y ) (x,y) ( x , y ) . Cauchy–Schwarz implies, for almost every x x x ,
∫ p ( x , y ) q ( x , y ) d y ≤ ∫ p ( x , y ) d y ∫ q ( x , y ) d y = p I ( x ) q I ( x ) . \int\sqrt{p(x,y)q(x,y)}\,\mathrm{d}y \le\sqrt{\int p(x,y)\,\mathrm{d}y\int q(x,y)\,\mathrm{d}y}=\sqrt{p_I(x)q_I(x)}. ∫ p ( x , y ) q ( x , y ) d y ≤ ∫ p ( x , y ) d y ∫ q ( x , y ) d y = p I ( x ) q I ( x ) . Integrating in x x x and using the affinity identity h 2 ( p , q ) = 1 − ∫ p q h^2(p,q)=1-\int\sqrt{pq} h 2 ( p , q ) = 1 − ∫ pq gives h 2 ( p I , q I ) ≤ h 2 ( p , q ) h^2(p_I,q_I)\le h^2(p,q) h 2 ( p I , q I ) ≤ h 2 ( p , q ) . □
An inverse heat identity with a uniform moment bound
The preceding lemma controls the Hellinger error of any retained observation marginal by the Hellinger error of full block. To recover information about the latent locations, we next relate their polynomial moments to these observation densities. For a given polynomial U U U of the latent locations, we construct a polynomial g g g of the observations whose conditional expectation equals U U U . Applying Cauchy–Schwarz to the difference between the expectations of g g g produces a Hellinger factor and a factor involving its second moments under the two observation densities. We therefore bound these second moments uniformly over the mixing distributions to obtain the desired inequality.
Lemma A.3 (Polynomial moments of Gaussian location mixtures). *Let S S S be a Euclidean space of dimension q ≤ K + k q\le K+k q ≤ K + k . Let H , H ′ H,H' H , H ′ be probability measures supported on B S ( 0 , R ) r B_S(0,R)^r B S ( 0 , R ) r , and let p H , p H ′ p_H,p_{H'} p H , p H ′
be the corresponding Gaussian location-mixture densities on S r S^r S r with identity covariance. If U U U is a polynomial of total degree at most D D D , then
∣ ∫ U d ( H − H ′ ) ∣ ≤ C ( q , r , R , D ) ∥ U ∥ ∞ , B S ( 0 , R ) r h ( p H , p H ′ ) . (47) \left|\int U\,\mathrm{d}(H-H')\right| \le C(q,r,R,D)\lVert U\rVert_{\infty,B_{S}(0,R)^r}h(p_H,p_{H'}). \tag*{(47)} ∫ U d ( H − H ′ ) ≤ C ( q , r , R , D ) ∥ U ∥ ∞ , B S ( 0 , R ) r h ( p H , p H ′ ) . (47)
The constant does not depend on the numbers of support points of H H H and H ′ H' H ′ . For a HMM block with r r r retained times, one may take
S = span { μ 1 , … , μ k , a 1 , … , a K } , S=\operatorname{span}\{\mu_1,\ldots,\mu_k,a_1,\ldots,a_K\}, S = span { μ 1 , … , μ k , a 1 , … , a K } ,
so the constant can be bounded by a quantity depending only on K , k , r , R , D K,k,r,R,D K , k , r , R , D .
Proof. Choose orthonormal coordinates on S r S^r S r , put m = q r m=qr m = q r , and write K = B S ( 0 , R ) r \mathcal{K}=B_S(0,R)^r K = B S ( 0 , R ) r . If q = 0 q=0 q = 0 or R = 0 R=0 R = 0 , the mixing measures coincide; if D = 0 D=0 D = 0 , their difference integrates the constant polynomial to zero. We may therefore assume q , D ≥ 1 q,D\ge1 q , D ≥ 1 and R > 0 R>0 R > 0 .
The inverse heat identity. Let Z ∼ N m ( 0 , I m ) Z\sim N_m(0,I_m) Z ∼ N m ( 0 , I m ) and Δ L = ∑ b = 1 m ∂ b 2 \Delta_L=\sum_{b=1}^{m}\partial_b^2 Δ L = ∑ b = 1 m ∂ b 2 . For a polynomial F F F of total degree at most D D D , its finite Taylor expansion and the Gaussian moments give
E F ( u + Z ) = ∑ 2 ∣ α ∣ ≤ D ∂ 2 α F ( u ) 2 ∣ α ∣ α ! = ∑ j = 0 J Δ L j F ( u ) 2 j j ! , J = ⌊ D / 2 ⌋ . (48) \mathbb{E}F(u+Z)=\sum_{2|\alpha|\le D}\frac{\partial^{2\alpha}F(u)}{2^{|\alpha|}\alpha!}=\sum_{j=0}^{J}\frac{\Delta_L^jF(u)}{2^j j!},\qquad J=\lfloor D/2\rfloor. \tag*{(48)} E F ( u + Z ) = 2∣ α ∣ ≤ D ∑ 2 ∣ α ∣ α ! ∂ 2 α F ( u ) = j = 0 ∑ J 2 j j ! Δ L j F ( u ) , J = ⌊ D /2 ⌋ . (48)
Here α \alpha α is a multi-index, and the second equality is the multinomial formula. On this finite-dimensional polynomial space, N = Δ L / 2 N=\Delta_L/2 N = Δ L /2 satisfies N J + 1 = 0 N^{J+1}=0 N J + 1 = 0 . Define
g = ∑ j = 0 J ( − 1 ) j j ! N j U . g=\sum_{j=0}^{J}\frac{(-1)^j}{j!}N^jU. g = j = 0 ∑ J j ! ( − 1 ) j N j U .
For 1 ≤ a ≤ J 1\le a\le J 1 ≤ a ≤ J , the coefficient of N a N^a N a in the product of the two truncated operator sums is
∑ j = 0 a ( − 1 ) j j ! ( a − j ) ! = ( 1 − 1 ) a a ! = 0. \sum_{j=0}^{a}\frac{(-1)^j}{j!(a-j)!}=\frac{(1-1)^a}{a!}=0. j = 0 ∑ a j ! ( a − j )! ( − 1 ) j = a ! ( 1 − 1 ) a = 0.
The constant coefficient is one, and all higher powers annihilate U U U . Thus equation (A.3) gives E g ( u + Z ) = U ( u ) \mathbb{E}g(u+Z)=U(u) E g ( u + Z ) = U ( u ) . No identity between unrestricted, truncated formal power series is being asserted.
A coefficient bound and uniform integrability. For F ( x ) = ∑ α f α x α F(x)=\sum_{\alpha}f_\alpha x^\alpha F ( x ) = ∑ α f α x α , write ∥ F ∥ c o e f , 1 = ∑ α ∣ f α ∣ \lVert F\rVert_{\mathrm{coef},1}=\sum_{\alpha}|f_\alpha| ∥ F ∥ coef , 1 = ∑ α ∣ f α ∣ . Set a = R / q a=R/\sqrt{q} a = R / q and ξ i = − a + 2 a i / D \xi_i=-a+2ai/D ξ i = − a + 2 ai / D for 0 ≤ i ≤ D 0\le i\le D 0 ≤ i ≤ D . The product grid { ξ 0 , … , ξ D } m \{\xi_0,\ldots,\xi_D\}^m { ξ 0 , … , ξ D } m lies in K \mathcal{K} K , since each of its r r r blocks has Euclidean norm at most R R R . Let
L i ( t ) = ∏ 0 ≤ b ≤ D b ≠ i t − ξ b ξ i − ξ b . L_i(t)=\prod_{\substack{0\le b\le D\\b\ne i}}\frac{t-\xi_b}{\xi_i-\xi_b}. L i ( t ) = 0 ≤ b ≤ D b = i ∏ ξ i − ξ b t − ξ b .
The numerator has coefficient norm at most ( 1 + a ) D (1+a)^D ( 1 + a ) D . The denominator has absolute value ( 2 a / D ) D i ! ( D − i ) ! (2a/D)^D i!(D-i)! ( 2 a / D ) D i ! ( D − i )! . Consequently,
A a , D : = ∑ i = 0 D ∥ L i ∥ c o e f , 1 ≤ ( 1 + a ) D D D a D D ! . A_{a,D}:=\sum_{i=0}^{D}\lVert L_i\rVert_{\mathrm{coef},1}\le\frac{(1+a)^D D^D}{a^D D!}. A a , D := i = 0 ∑ D ∥ L i ∥ coef , 1 ≤ a D D ! ( 1 + a ) D D D .
Since the degree of U U U in each coordinate is at most D D D , tensor-product interpolation is exact and implies
∥ U ∥ c o e f , 1 ≤ A a , D m ∥ U ∥ ∞ , K . \|U\|_{\mathrm{coef},1} \le A_{a,D}^{m}\|U\|_{\infty,\mathcal{K}}. ∥ U ∥ coef , 1 ≤ A a , D m ∥ U ∥ ∞ , K .
For every polynomial of total degree at most D D D , ∥ Δ L F ∥ c o e f , 1 ≤ m D 2 ∥ F ∥ c o e f , 1 \|\Delta_L F\|_{\mathrm{coef},1} \le mD^2\|F\|_{\mathrm{coef},1} ∥ Δ L F ∥ coef , 1 ≤ m D 2 ∥ F ∥ coef , 1 . Iteration and the finite definition of g g g therefore give
∥ g ∥ c o e f , 1 ≤ e m D 2 / 2 A a , D m ∥ U ∥ ∞ , K . \|g\|_{\mathrm{coef},1} \le e^{mD^2/2}A_{a,D}^{m}\|U\|_{\infty,\mathcal{K}}. ∥ g ∥ coef , 1 ≤ e m D 2 /2 A a , D m ∥ U ∥ ∞ , K .
It follows that ∣ g ( x ) ∣ ≤ C ∥ U ∥ ∞ , K ( 1 + ∥ x ∥ 2 D ) |g(x)| \le C\|U\|_{\infty,\mathcal{K}}(1+\|x\|_2^D) ∣ g ( x ) ∣ ≤ C ∥ U ∥ ∞ , K ( 1 + ∥ x ∥ 2 D ) , with C C C depending only on q , r , R , D q,r,R,D q , r , R , D . Since ∥ u ∥ 2 ≤ R r \|u\|_2 \le R\sqrt{r} ∥ u ∥ 2 ≤ R r on K \mathcal{K} K and E ∥ Z ∥ 2 2 D = ∏ j = 0 D − 1 ( m + 2 j ) < ∞ \mathbb{E}\|Z\|_2^{2D} = \prod_{j=0}^{D-1}(m+2j) < \infty E ∥ Z ∥ 2 2 D = ∏ j = 0 D − 1 ( m + 2 j ) < ∞ , we obtain
sup u ∈ K E ∣ g ( u + Z ) ∣ 2 ≤ B ( q , r , R , D ) ∥ U ∥ ∞ , K 2 . \sup_{u\in\mathcal{K}}\mathbb{E}|g(u+Z)|^2 \le B(q,r,R,D)\|U\|_{\infty,\mathcal{K}}^2. u ∈ K sup E ∣ g ( u + Z ) ∣ 2 ≤ B ( q , r , R , D ) ∥ U ∥ ∞ , K 2 .
This bound proves, before any interchange of integrals, absolute integrability of g g g under either observation mixture. Fubini’s theorem and the inverse heat identity now yield
∫ g ( x ) p H ( x ) d x = ∫ U d H , ∫ g ( x ) p H ′ ( x ) d x = ∫ U d H ′ . \int g(x)p_H(x)\,dx = \int U\,dH,\qquad\int g(x)p_{H'}(x)\,dx = \int U\,dH'. ∫ g ( x ) p H ( x ) d x = ∫ U d H , ∫ g ( x ) p H ′ ( x ) d x = ∫ U d H ′ .
Factoring the difference of the densities and applying Cauchy–Schwarz gives
∣ ∫ U d ( H − H ′ ) ∣ ≤ 2 h ( p H , p H ′ ) { 2 ∫ g 2 p H + 2 ∫ g 2 p H ′ } 1 / 2 \left|\int U\,d(H-H')\right| \le\sqrt{2}h(p_H,p_{H'})\left\{2\int g^2p_H+2\int g^2p_{H'}\right\}^{1/2} ∫ U d ( H − H ′ ) ≤ 2 h ( p H , p H ′ ) { 2 ∫ g 2 p H + 2 ∫ g 2 p H ′ } 1/2
≤ 2 2 B ∥ U ∥ ∞ , K h ( p H , p H ′ ) . (49) \le2\sqrt{2B}\,\|U\|_{\infty,\mathcal{K}}h(p_H,p_{H'}). \tag*{(49)} ≤ 2 2 B ∥ U ∥ ∞ , K h ( p H , p H ′ ) . (49)
This proves equation (A.2) for arbitrary mixing measures on K \mathcal{K} K ; their support cardinalities have not entered the argument.
Reduction of an HMM block to the center span. For the stated HMM application, choose S S S to contain every fitted and true center. After an orthogonal change of coordinates, the S ⊥ S^\perp S ⊥ coordinates of the observations have the same standard Gaussian law under both models. They are independent of the S S S coordinates and of the latent path. Thus, for the selected time set I I I ,
p θ I ( x S , x ⊥ ) = p H ( x S ) φ ( d − q ) r ( x ⊥ ) , p θ ⋆ I ( x S , x ⊥ ) = p H ′ ( x S ) φ ( d − q ) r ( x ⊥ ) . p_\theta^I(x_S,x_\perp)=p_H(x_S)\varphi_{(d-q)r}(x_\perp),\qquad p_{\theta^\star}^I(x_S,x_\perp)=p_{H'}(x_S)\varphi_{(d-q)r}(x_\perp). p θ I ( x S , x ⊥ ) = p H ( x S ) φ ( d − q ) r ( x ⊥ ) , p θ ⋆ I ( x S , x ⊥ ) = p H ′ ( x S ) φ ( d − q ) r ( x ⊥ ) .
The common factor integrates to one in the Hellinger affinity, so
h ( p H , p H ′ ) = h ( p θ I , p θ ⋆ I ) ≤ h ( p θ , T , p θ ⋆ , T ) . h(p_H,p_{H'})=h(p_\theta^I,p_{\theta^\star}^I)\le h(p_{\theta,T},p_{\theta^\star,T}). h ( p H , p H ′ ) = h ( p θ I , p θ ⋆ I ) ≤ h ( p θ , T , p θ ⋆ , T ) .
If q = d q=d q = d , the orthogonal factor is absent; if q = 0 q=0 q = 0 , the desired moment difference is zero. Finally, q ≤ K + k q\le K+k q ≤ K + k , so maximizing the constants over the finitely many admissible dimensions gives the asserted dependence on K , k , r , R , D K,k,r,R,D K , k , r , R , D , without dependence on d d d or on the number of latent paths. □
Transport through a common finite support
To compare the fitted and true latent location distributions, we first move each fitted center to its assigned true center. After this, both distributions are supported on the same finite set, and the remaining transport concerns differences in their probabilities. Under the cluster assumptions, some of this probability mismatch can be transported within clusters at a cost determined by their diameter. We therefore distinguish this part from the mass that must move between clusters.
Let I = { t 1 , … , t r } I=\{t_1,\ldots,t_r\} I = { t 1 , … , t r } with t 1 < ⋯ < t r t_1<\cdots<t_r t 1 < ⋯ < t r , and define the cluster map κ : [ K ] → [ M ] \kappa:[K]\to[M] κ : [ K ] → [ M ] by κ ( j ) = m \kappa(j)=m κ ( j ) = m for j ∈ C m j\in C_m j ∈ C m . For m ∈ [ M ] r m\in[M]^r m ∈ [ M ] r , set
P ‾ θ I , c l ( m ) = P θ { κ ( c ( Z t ℓ ) ) = m ℓ for every ℓ } , \overline{P}^{I,\mathrm{cl}}_{\theta}(m)=\mathbb{P}_{\theta}\{\kappa(c(Z_{t_\ell}))=m_\ell\text{ for every }\ell\}, P θ I , cl ( m ) = P θ { κ ( c ( Z t ℓ )) = m ℓ for every ℓ } ,
P ⋆ I , c l ( m ) = P θ ⋆ { κ ( Z t ℓ ) = m ℓ for every ℓ } , P^{I,\mathrm{cl}}_{\star}(m)=\mathbb{P}_{\theta_\star}\{\kappa(Z_{t_\ell})=m_\ell\text{ for every }\ell\}, P ⋆ I , cl ( m ) = P θ ⋆ { κ ( Z t ℓ ) = m ℓ for every ℓ } ,
J I = ∑ m ∈ [ M ] r ∣ P ‾ θ I , c l ( m ) − P ⋆ I , c l ( m ) ∣ . J_I=\sum_{m\in[M]^r}\left|\overline{P}^{I,\mathrm{cl}}_{\theta}(m)-P^{I,\mathrm{cl}}_{\star}(m)\right|. J I = m ∈ [ M ] r ∑ P θ I , cl ( m ) − P ⋆ I , cl ( m ) .
Each cluster-sequence difference is a sum of label-sequence differences. The triangle inequality therefore gives 0 ≤ J I ≤ D I 0\le J_I\le D_I 0 ≤ J I ≤ D I . For a single cluster both cluster-sequence laws are the same point mass, so J I = 0 J_I=0 J I = 0 .
Lemma A.4 (Hierarchical transport). For p ∈ { 1 , 2 } p\in\{1,2\} p ∈ { 1 , 2 } , let
V p , I = 1 r ∑ t ∈ I ∑ i = 1 k q t ( i ) r i p . V_{p,I}=\frac{1}{r}\sum_{t\in I}\sum_{i=1}^{k}q_t(i)r_i^p. V p , I = r 1 t ∈ I ∑ i = 1 ∑ k q t ( i ) r i p . where q t ( i ) = P θ ( Z t = i ) q_t(i)=\mathbb{P}_{\theta}(Z_t=i) q t ( i ) = P θ ( Z t = i ) and r i = ∥ μ i − a c ( i ) ∥ r_i=\|\mu_i-a_{c(i)}\| r i = ∥ μ i − a c ( i ) ∥ . In the structured regimes,
W p , I p ( G θ I , G ⋆ I ) ≤ 2 p − 1 { V p , I + ( C 0 Δ s e p ) p 2 D I + ( 2 R ) p 2 J I } . (50) W_{p,I}^p(G_\theta^I,G_\star^I)\le2^{p-1}\left\{V_{p,I}+\frac{(C_0\Delta_{\mathrm{sep}})^p}{2}D_I+\frac{(2R)^p}{2}J_I\right\}. \tag*{(50)} W p , I p ( G θ I , G ⋆ I ) ≤ 2 p − 1 { V p , I + 2 ( C 0 Δ sep ) p D I + 2 ( 2 R ) p J I } . (50) Otherwise, for unstructured regime, we have
W p , I p ( G θ I , G ⋆ I ) ≤ 2 p − 1 { V p , I + ( 2 R ) p 2 D I } . (51) W_{p,I}^p(G_\theta^I,G_\star^I)\le2^{p-1}\left\{V_{p,I}+\frac{(2R)^p}{2}D_I\right\}. \tag*{(51)} W p , I p ( G θ I , G ⋆ I ) ≤ 2 p − 1 { V p , I + 2 ( 2 R ) p D I } . (51)
Proof. Projection to the true support. Draw the fitted location tuple X = ( μ Z t 1 , … , μ Z t r ) X=(\mu_{Z_{t_1}},\ldots,\mu_{Z_{t_r}}) X = ( μ Z t 1 , … , μ Z t r ) and put Y = ( a c ( Z t 1 ) , … , a c ( Z t r ) ) Y=(a_{c(Z_{t_1})},\ldots,a_{c(Z_{t_r})}) Y = ( a c ( Z t 1 ) , … , a c ( Z t r ) ) . Its law on the true grid J = [ K ] r \mathcal{J}=[K]^r J = [ K ] r is u ( j ) = P ‾ θ I ( j ) u(j)=\overline{P}^{I}_{\theta}(j) u ( j ) = P θ I ( j ) ; write v ( j ) = P ⋆ I ( j ) v(j)=P^I_{\star}(j) v ( j ) = P ⋆ I ( j ) for the true grid law. This coupling has exact cost
E d p , r ( X , Y ) p = 1 r ∑ t ∈ I ∑ i q t ( i ) r i p = V p , I . \mathbb{E}d_{p,r}(X,Y)^p=\frac{1}{r}\sum_{t\in I}\sum_i q_t(i)r_i^p=V_{p,I}. E d p , r ( X , Y ) p = r 1 t ∈ I ∑ i ∑ q t ( i ) r i p = V p , I .
No independence among the selected hidden states is used.
Redistribution on the true grid. Write w ( j ) = min { u ( j ) , v ( j ) } w(j)=\min\{u(j),v(j)\} w ( j ) = min { u ( j ) , v ( j )} and let u ′ ( j ) = u ( j ) − w ( j ) u'(j)=u(j)-w(j) u ′ ( j ) = u ( j ) − w ( j ) , v ′ ( j ) = v ( j ) − w ( j ) v'(j)=v(j)-w(j) v ′ ( j ) = v ( j ) − w ( j ) . Their common total is t = D I / 2 t=D_I/2 t = D I /2 . Couple w ( j ) w(j) w ( j ) diagonally, at zero cost. Under cluster structure, partition J \mathcal{J} J into the cells J m \mathcal{J}_m J m of equal cluster sequences m ∈ [ M ] r m\in[M]^r m ∈ [ M ] r . Define
a m = ∑ j ∈ J m u ′ ( j ) , b m = ∑ j ∈ J m v ′ ( j ) , c m = min { a m , b m } . a_m=\sum_{j\in\mathcal{J}_m}u'(j),\qquad b_m=\sum_{j\in\mathcal{J}_m}v'(j),\qquad c_m=\min\{a_m,b_m\}. a m = j ∈ J m ∑ u ′ ( j ) , b m = j ∈ J m ∑ v ′ ( j ) , c m = min { a m , b m } .
For a m b m > 0 a_mb_m>0 a m b m > 0 , put
Γ m ( j , l ) = c m u ′ ( j ) a m v ′ ( l ) b m ( j , l ∈ J m ) , \Gamma_m(j,l)=c_m\frac{u'(j)}{a_m}\frac{v'(l)}{b_m}\qquad(j,l\in\mathcal{J}_m), Γ m ( j , l ) = c m a m u ′ ( j ) b m v ′ ( l ) ( j , l ∈ J m ) ,
and put Γ m = 0 \Gamma_m=0 Γ m = 0 if a m b m = 0 a_mb_m=0 a m b m = 0 . Its row sums are c m u ′ ( j ) / a m ≤ u ′ ( j ) c_mu'(j)/a_m\le u'(j) c m u ′ ( j ) / a m ≤ u ′ ( j ) and column sums are c m v ′ ( l ) / b m ≤ v ′ ( l ) c_mv'(l)/b_m\le v'(l) c m v ′ ( l ) / b m ≤ v ′ ( l ) . Thus it couples exactly c m c_m c m units within the cell, at cost per unit at most ( C 0 Δ s e p ) p (C_0\Delta_{\mathrm{sep}})^p ( C 0 Δ sep ) p . Let U m = ∑ j ∈ J m u ( j ) U_m=\sum_{j\in\mathcal{J}_m}u(j) U m = ∑ j ∈ J m u ( j ) and V m = ∑ j ∈ J m v ( j ) V_m=\sum_{j\in\mathcal{J}_m}v(j) V m = ∑ j ∈ J m v ( j ) . Because diagonal matching removes the same mass from both cell totals,
a m − c m = ( U m − V m ) + , b m − c m = ( V m − U m ) + . a_m-c_m=(U_m-V_m)_+,\qquad b_m-c_m=(V_m-U_m)_+. a m − c m = ( U m − V m ) + , b m − c m = ( V m − U m ) + .
The remaining supply and demand therefore each have total t 0 = 1 2 ∑ m ∣ U m − V m ∣ = J I / 2 t_0 = \frac{1}{2}\sum_m |U_m - V_m| = J_I/2 t 0 = 2 1 ∑ m ∣ U m − V m ∣ = J I /2 . Denote their pointwise masses by u ′ ′ ( j ) u^{\prime\prime}(j) u ′′ ( j ) and v ′ ′ ( l ) v^{\prime\prime}(l) v ′′ ( l ) . Couple them by Γ 0 ( j , l ) = u ′ ′ ( j ) v ′ ′ ( l ) / t 0 \Gamma_0(j,l) = u^{\prime\prime}(j)v^{\prime\prime}(l)/t_0 Γ 0 ( j , l ) = u ′′ ( j ) v ′′ ( l ) / t 0 when t 0 > 0 t_0 > 0 t 0 > 0 , and set Γ 0 = 0 \Gamma_0 = 0 Γ 0 = 0 otherwise. Its marginals are the two residual arrays. Consequently,
Γ ( j , l ) = w ( j ) 1 { j = l } + ∑ m Γ m ( j , l ) + Γ 0 ( j , l ) \Gamma(j,l) = w(j)\mathbf{1}\{j=l\} + \sum_m \Gamma_m(j,l) + \Gamma_0(j,l) Γ ( j , l ) = w ( j ) 1 { j = l } + m ∑ Γ m ( j , l ) + Γ 0 ( j , l )
has marginals u , v u,v u , v , with total cost at most
( C 0 Δ s e p ) p ( t − t 0 ) + ( 2 R ) p t 0 ≤ ( C 0 Δ s e p ) p 2 D I + ( 2 R ) p 2 J I . (C_0\Delta_{\mathrm{sep}})^p(t-t_0) + (2R)^p t_0 \le\frac{(C_0\Delta_{\mathrm{sep}})^p}{2}D_I + \frac{(2R)^p}{2}J_I. ( C 0 Δ sep ) p ( t − t 0 ) + ( 2 R ) p t 0 ≤ 2 ( C 0 Δ sep ) p D I + 2 ( 2 R ) p J I .
For unstructured centers, omit the cell couplings and couple the entire residual mass t = D I / 2 t = D_I/2 t = D I /2 by u ′ ( j ) v ′ ( l ) / t u^{\prime}(j)v^{\prime}(l)/t u ′ ( j ) v ′ ( l ) / t if t > 0 t > 0 t > 0 (and by zero otherwise). Its cost is at most ( 2 R ) p D I / 2 (2R)^pD_I/2 ( 2 R ) p D I /2 .
Gluing the couplings. Conditional on Y = a j Y = a_j Y = a j with u ( j ) > 0 u(j) > 0 u ( j ) > 0 , draw Y ′ = a l Y^{\prime} = a_l Y ′ = a l with probability Γ ( j , l ) / u ( j ) \Gamma(j,l)/u(j) Γ ( j , l ) / u ( j ) . For u ( j ) = 0 u(j) = 0 u ( j ) = 0 choose an arbitrary conditional probability: this event is null and the corresponding row of Γ \Gamma Γ vanishes. Then X ∼ G I ′ X \sim G_I^{\prime} X ∼ G I ′ , Y ′ ∼ G ⋆ I Y^{\prime} \sim G_{\star}^{I} Y ′ ∼ G ⋆ I and ( Y , Y ′ ) ∼ Γ (Y,Y^{\prime}) \sim\Gamma ( Y , Y ′ ) ∼ Γ . Applying d p , r ( X , Y ′ ) p ≤ 2 p − 1 { d p , r ( X , Y ) p + d p , r ( Y , Y ′ ) p } d_{p,r}(X,Y^{\prime})^p \le2^{p-1}\{d_{p,r}(X,Y)^p + d_{p,r}(Y,Y^{\prime})^p\} d p , r ( X , Y ′ ) p ≤ 2 p − 1 { d p , r ( X , Y ) p + d p , r ( Y , Y ′ ) p } and taking expectations gives exactly (A.5) and (A.6), respectively.
Lower transport bounds and matching. The preceding coupling gives upper transport bounds. We also record the reverse comparisons used to extract probability errors and matching from transport. They require no Markov property of the grouped fitted process.
Lemma A.5 (Transport, assigned labels, and positive-mass matching). Let 0 < δ ≤ δ s e p 0 < \delta\le\delta_{\mathrm{sep}} 0 < δ ≤ δ sep be any lower bound on the true-center separation. For p ∈ { 1 , 2 } p \in\{1,2\} p ∈ { 1 , 2 } and every nonempty I ⊆ [ T ] I \subseteq[T] I ⊆ [ T ] with ∣ I ∣ = r |I| = r ∣ I ∣ = r ,
V p , I ≤ W p , r ( G I I , G ⋆ I ) , D I ≤ 2 p + 1 r δ p W p , r ( G I I , G ⋆ I ) . (52) V_{p,I} \le W_{p,r}(G_I^I,G_{\star}^I), \qquad D_I \le\frac{2^{p+1}r}{\delta^p}W_{p,r}(G_I^I,G_{\star}^I). \tag*{(52)} V p , I ≤ W p , r ( G I I , G ⋆ I ) , D I ≤ δ p 2 p + 1 r W p , r ( G I I , G ⋆ I ) . (52) The same statements hold for arbitrary fitted and true location-block laws with the indicated supports. Zero atom masses and repeated fitted centers are allowed. For one-time measures H = ∑ i = 1 K v i δ b i H = \sum_{i=1}^{K}v_i\delta_{b_i} H = ∑ i = 1 K v i δ b i and H ⋆ = ∑ j = 1 K w j δ a j H^{\star} = \sum_{j=1}^{K}w_j\delta_{a_j} H ⋆ = ∑ j = 1 K w j δ a j , assume additionally that w j ≥ p 0 > 0 w_j \ge p_0 > 0 w j ≥ p 0 > 0 . If
W 1 ( H , H ⋆ ) < p 0 δ / 4 , W_1(H,H^{\star}) < p_0\delta/4, W 1 ( H , H ⋆ ) < p 0 δ /4 , then each open ball B ( a j , δ / 4 ) B(a_j,\delta/4) B ( a j , δ /4 ) contains exactly one fitted label and that label has positive mass. After this unique relabeling,
∥ v − w ∥ 1 ≤ 8 3 δ W 1 ( H , H ⋆ ) . (53) \|v-w\|_1 \le\frac{8}{3\delta}W_1(H,H^{\star}). \tag*{(53)} ∥ v − w ∥ 1 ≤ 3 δ 8 W 1 ( H , H ⋆ ) . (53) In particular, W 1 ( H , H ⋆ ) ≤ p 0 δ / 32 W_1(H,H^{\star}) \le p_0\delta/32 W 1 ( H , H ⋆ ) ≤ p 0 δ /32 implies v j ≥ w j / 2 v_j \ge w_j/2 v j ≥ w j /2 for every j j j . Every assertion is valid both for δ = δ s e p \delta= \delta_{\mathrm{sep}} δ = δ sep and for δ = Δ s e p \delta= \Delta_{\mathrm{sep}} δ = Δ sep .
Proof. We first compare the cost of an arbitrary coupling, then specialize to one-time exact specification.
Nearest-support cost. Let ( X , Y ) (X,Y) ( X , Y ) have any coupling of the fitted and true location-block laws. Let χ ( x ) \chi(x) χ ( x ) denote the smallest-index nearest true-center label, so that χ ( μ i ) = c ( i ) \chi(\mu_i) = c(i) χ ( μ i ) = c ( i ) , and put C ( X ) = ( χ ( X 1 ) , … , χ ( X r ) ) C(X) = (\chi(X_1),\ldots,\chi(X_r)) C ( X ) = ( χ ( X 1 ) , … , χ ( X r )) . For every coordinate, Y ℓ Y_\ell Y ℓ is a true center, so
∥ X ℓ − Y ℓ ∥ 2 p ≥ min j ∥ X ℓ − a j ∥ 2 p . \|X_\ell-Y_\ell\|_2^p \ge\min_j \|X_\ell-a_j\|_2^p. ∥ X ℓ − Y ℓ ∥ 2 p ≥ j min ∥ X ℓ − a j ∥ 2 p .
Averaging over coordinates and taking expectations gives a lower bound of V p , I V_{p,I} V p , I for the cost of every coupling. This proves the first inequality in equation (A.7). Mass assigned to different labels. Because the true centers are distinct, the labels of Y Y Y are uniquely determined by its locations. The two label marginals are P ˉ θ I \bar{P}_{\theta}^{I} P ˉ θ I and P ⋆ I P_{\star}^{I} P ⋆ I . The coupling can put at most ∑ j min { P ˉ θ I ( j ) , P ⋆ I ( j ) } = 1 − D I / 2 \sum_j \min\{\bar{P}_{\theta}^{I}(j),P_{\star}^{I}(j)\}=1-D_I/2 ∑ j min { P ˉ θ I ( j ) , P ⋆ I ( j )} = 1 − D I /2 on identical label sequences. On the complementary event, at least one coordinate has χ ( X ℓ ) = i \chi(X_\ell)=i χ ( X ℓ ) = i and Y ℓ = a j Y_\ell=a_j Y ℓ = a j with i ≠ j i\ne j i = j . The nearest-center property and the triangle inequality give
δ ≤ ∥ a i − a j ∥ 2 ≤ ∥ X ℓ − a i ∥ 2 + ∥ X ℓ − a j ∥ 2 ≤ 2 ∥ X ℓ − a j ∥ 2 . \delta\le\lVert a_i-a_j\rVert_2 \le\lVert X_\ell-a_i\rVert_2+\lVert X_\ell-a_j\rVert_2 \le2\lVert X_\ell-a_j\rVert_2. δ ≤ ∥ a i − a j ∥ 2 ≤ ∥ X ℓ − a i ∥ 2 + ∥ X ℓ − a j ∥ 2 ≤ 2 ∥ X ℓ − a j ∥ 2 .
The normalized cost on this event is therefore at least δ p / ( 2 p r ) \delta^p/(2^p r) δ p / ( 2 p r ) . Every coupling has expected cost at least δ p D I / ( 2 p + 1 r ) \delta^pD_I/(2^{p+1}r) δ p D I / ( 2 p + 1 r ) , proving the second inequality. This argument acts on locations, so repeated fitted labels at the same location have the same assigned label; no division by an atom mass is involved.
Matching and fitted masses. If an open ball B ( a j , δ / 4 ) B(a_j,\delta/4) B ( a j , δ /4 ) contained no positive-mass fitted atom, all mass w j w_j w j arriving at a j a_j a j would travel at least δ / 4 \delta/4 δ /4 , contradicting W 1 ( H , H ⋆ ) < p 0 δ / 4 W_1(H,H^\star)<p_0\delta/4 W 1 ( H , H ⋆ ) < p 0 δ /4 . The K K K balls are disjoint and there are only K K K fitted labels. Thus they contain exactly one label each, all with positive mass; no zero-weight or repeated extra label can remain. Within these balls, each matched true center is the unique nearest center, so the relabeling agrees with c c c . The fitted centers are now distinct, so an optimal location coupling is a matrix Γ \Gamma Γ on the aligned labels. Put O = ∑ i ≠ j Γ i j O=\sum_{i\ne j}\Gamma_{ij} O = ∑ i = j Γ ij . For i ≠ j i\ne j i = j , ∥ b i − a j ∥ 2 ≥ δ − ∥ b i − a i ∥ 2 > 3 δ / 4 \lVert b_i-a_j\rVert_2\ge\delta-\lVert b_i-a_i\rVert_2>3\delta/4 ∥ b i − a j ∥ 2 ≥ δ − ∥ b i − a i ∥ 2 > 3 δ /4 . Also,
∣ v i − w i ∣ = ∣ ∑ j ≠ i Γ i j − ∑ j ≠ i Γ j i ∣ ≤ ∑ j ≠ i ( Γ i j + Γ j i ) . |v_i-w_i|=\left|\sum_{j\ne i}\Gamma_{ij}-\sum_{j\ne i}\Gamma_{ji}\right|\le\sum_{j\ne i}(\Gamma_{ij}+\Gamma_{ji}). ∣ v i − w i ∣ = j = i ∑ Γ ij − j = i ∑ Γ j i ≤ j = i ∑ ( Γ ij + Γ j i ) .
Summation yields ∥ v − w ∥ 1 ≤ 2 O ≤ 8 W 1 ( H , H ⋆ ) / ( 3 δ ) \lVert v-w\rVert_1\le2O\le8W_1(H,H^\star)/(3\delta) ∥ v − w ∥ 1 ≤ 2 O ≤ 8 W 1 ( H , H ⋆ ) / ( 3 δ ) , which is equation (A.8). At W 1 ≤ p 0 δ / 32 W_1\le p_0\delta/32 W 1 ≤ p 0 δ /32 , this norm is at most p 0 / 12 p_0/12 p 0 /12 , and hence v j ≥ w j − p 0 / 12 ≥ w j / 2 v_j\ge w_j-p_0/12\ge w_j/2 v j ≥ w j − p 0 /12 ≥ w j /2 . All these comparisons are deterministic.
Stability of aggregation with estimated reference centers
Proof of Proposition 4.8 . Let E = { i ∈ [ k ] : c ^ ( i ) ≠ c ( i ) } \mathcal{E}=\{i\in[k]:\hat{c}(i)\ne c(i)\} E = { i ∈ [ k ] : c ^ ( i ) = c ( i )} be the set of labels whose assignment changes. Put δ = δ s e p \delta=\delta_{\mathrm{sep}} δ = δ sep and η = max j ∥ ξ j − a j ∥ 2 ≤ τ \eta=\max_j\lVert\xi_j-a_j\rVert_2\le\tau η = max j ∥ ξ j − a j ∥ 2 ≤ τ . For any i ∈ E i\in\mathcal{E} i ∈ E , write j = c ( i ) j=c(i) j = c ( i ) and ℓ = c ^ ( i ) ≠ j \ell=\hat{c}(i)\ne j ℓ = c ^ ( i ) = j . Since ℓ \ell ℓ is closest to μ i \mu_i μ i among the reference centers, including at ties,
δ ≤ ∥ a j − a ℓ ∥ 2 ≤ ∥ a j − μ i ∥ 2 + ∥ μ i − a ℓ ∥ 2 ≤ r i + ∥ μ i − ξ ℓ ∥ 2 + η ≤ r i + ∥ μ i − ξ j ∥ 2 + η ≤ 2 r i + 2 η . \delta\le\lVert a_j-a_\ell\rVert_2\le\lVert a_j-\mu_i\rVert_2+\lVert\mu_i-a_\ell\rVert_2
\le r_i+\lVert\mu_i-\xi_\ell\rVert_2+\eta\le r_i+\lVert\mu_i-\xi_j\rVert_2+\eta\le2r_i+2\eta. δ ≤ ∥ a j − a ℓ ∥ 2 ≤ ∥ a j − μ i ∥ 2 + ∥ μ i − a ℓ ∥ 2 ≤ r i + ∥ μ i − ξ ℓ ∥ 2 + η ≤ r i + ∥ μ i − ξ j ∥ 2 + η ≤ 2 r i + 2 η .
Thus r i ≥ δ / 2 − η ≥ δ / 4 r_i\ge\delta/2-\eta\ge\delta/4 r i ≥ δ /2 − η ≥ δ /4 . To prove the margin claim, fix any label i i i whose reference assignment is ℓ = c ^ ( i ) \ell=\hat{c}(i) ℓ = c ^ ( i ) , and suppose m i ( ξ ) > 2 τ m_i(\xi)>2\tau m i ( ξ ) > 2 τ . For every j ≠ ℓ j\ne\ell j = ℓ ,
∥ μ i − a j ∥ 2 − ∥ μ i − a ℓ ∥ 2 ≥ ∥ μ i − ξ j ∥ 2 − ∥ μ i − ξ ℓ ∥ 2 − 2 η ≥ m i ( ξ ) − 2 η > 0. \lVert\mu_i-a_j\rVert_2-\lVert\mu_i-a_\ell\rVert_2\ge\lVert\mu_i-\xi_j\rVert_2-\lVert\mu_i-\xi_\ell\rVert_2-2\eta\ge m_i(\xi)-2\eta>0. ∥ μ i − a j ∥ 2 − ∥ μ i − a ℓ ∥ 2 ≥ ∥ μ i − ξ j ∥ 2 − ∥ μ i − ξ ℓ ∥ 2 − 2 η ≥ m i ( ξ ) − 2 η > 0.
Hence c ( i ) = c ^ ( i ) = ℓ c(i)=\hat{c}(i)=\ell c ( i ) = c ^ ( i ) = ℓ , and by contraposition E ⊆ B τ ( ξ ) \mathcal{E}\subseteq B_\tau(\xi) E ⊆ B τ ( ξ ) . For every t ≤ T t\le T t ≤ T the probability of occupying a relabeled state satisfies both
P θ { Z t ∈ E } = ∑ i ∈ E q t ( i ) ≤ 4 δ ∑ i q t ( i ) r i ≤ 4 D μ δ , (54) \mathbb{P}_{\theta}\{Z_t\in\mathcal{E}\}=\sum_{i\in\mathcal{E}}q_t(i)\le\frac{4}{\delta}\sum_i q_t(i)r_i\le\frac{4D_\mu}{\delta}, \tag*{(54)} P θ { Z t ∈ E } = i ∈ E ∑ q t ( i ) ≤ δ 4 i ∑ q t ( i ) r i ≤ δ 4 D μ , (54)
P θ { Z t ∈ E } ≤ 16 δ 2 ∑ i q t ( i ) r i 2 ≤ 16 V δ 2 . (55) \mathbb{P}_{\theta}\{Z_t\in\mathcal{E}\}\le\frac{16}{\delta^2}\sum_i q_t(i)r_i^2\le\frac{16V}{\delta^2}. \tag*{(55)} P θ { Z t ∈ E } ≤ δ 2 16 i ∑ q t ( i ) r i 2 ≤ δ 2 16 V . (55)
On the same fitted hidden-state path, compare the two sequences of labels ( c ( Z t ℓ ) ) ℓ = 1 r (c(Z_{t_\ell}))_{\ell=1}^r ( c ( Z t ℓ ) ) ℓ = 1 r and ( c ˉ ( Z t ℓ ) ) ℓ = 1 r (\bar{c}(Z_{t_\ell}))_{\ell=1}^r ( c ˉ ( Z t ℓ ) ) ℓ = 1 r . They agree unless at least one of the selected states belongs to E \mathcal{E} E . The elementary coupling inequality for total variation and a union bound therefore give
∥ P ^ θ I − P ‾ θ I ∥ 1 ≤ 2 P θ { ∃ ℓ ≤ r : Z t ℓ ∈ E } \left\|\widehat{P}^{I}_{\theta}-\overline{P}^{I}_{\theta}\right\|_1 \le2\mathbb{P}_{\theta}\left\{\exists\ell\le r : Z_{t_\ell} \in\mathcal{E}\right\} P θ I − P θ I 1 ≤ 2 P θ { ∃ ℓ ≤ r : Z t ℓ ∈ E }
≤ 2 ∑ ℓ = 1 r P θ { Z t ℓ ∈ E } ≤ min { 2 ∑ t ∈ I ∑ i ∈ B τ ( ξ ) q t ( i ) , 8 r D μ δ , 32 r V δ 2 } . \le2\sum_{\ell=1}^{r}\mathbb{P}_{\theta}\{Z_{t_\ell}\in\mathcal{E}\}
\le\min\left\{2\sum_{t\in I}\sum_{i\in B_{\tau}(\xi)}q_t(i),\frac{8rD_{\mu}}{\delta},\frac{32rV}{\delta^2}\right\}. ≤ 2 ℓ = 1 ∑ r P θ { Z t ℓ ∈ E } ≤ min ⎩ ⎨ ⎧ 2 t ∈ I ∑ i ∈ B τ ( ξ ) ∑ q t ( i ) , δ 8 r D μ , δ 2 32 r V ⎭ ⎬ ⎫ .
The triangle inequality with the true label-sequence law P ⋆ I P_{\star}^{I} P ⋆ I proves equation (44) . This coupling works even if fitted states have zero masses or coincident emission centers, and it never uses a Markov assumption for the grouped process.
For the asserted exponents, write s = s max s=s_{\max} s = s m a x in a structured regime, and s = K s=K s = K otherwise. Under exact specification, Theorem 3.1 supplies D I ≤ C r ( π min ⋆ ) − 2 s + 1 Δ s e p − ( 2 s − 1 ) h D_I\le C r(\pi_{\min}^{\star})^{-2s+1}\Delta_{\mathrm{sep}}^{-(2s-1)}h D I ≤ C r ( π m i n ⋆ ) − 2 s + 1 Δ sep − ( 2 s − 1 ) h and D μ ≤ C ( π min ⋆ ) − 2 s + 2 Δ s e p − ( 2 s − 2 ) h D_{\mu}\le C(\pi_{\min}^{\star})^{-2s+2}\Delta_{\mathrm{sep}}^{-(2s-2)}h D μ ≤ C ( π m i n ⋆ ) − 2 s + 2 Δ sep − ( 2 s − 2 ) h . Since δ ≥ Δ s e p \delta\ge\Delta_{\mathrm{sep}} δ ≥ Δ sep , the linear alternative in equation (44) preserves exponent 2 s − 1 2s-1 2 s − 1 . If the paired-center hypothesis of Theorem 3.6 is available, the same argument uses its constants independent of π min ⋆ \pi_{\min}^{\star} π m i n ⋆ . Under over-specification, Theorem 4.2 gives D I ≤ C r Δ s e p − b r h D_I\le C r\Delta_{\mathrm{sep}}^{-b_r}h D I ≤ C r Δ sep − b r h , where b r = β r b_r=\beta_r b r = β r or γ K , r \gamma_{K,r} γ K , r , and V ≤ C Δ s e p − ( 2 s − 2 ) h V\le C\Delta_{\mathrm{sep}}^{-(2s-2)}h V ≤ C Δ sep − ( 2 s − 2 ) h . The quadratic alternative contributes at most C r Δ s e p − 2 s h C r\Delta_{\mathrm{sep}}^{-2s}h C r Δ sep − 2 s h , which is bounded by C r Δ s e p − b r h C r\Delta_{\mathrm{sep}}^{-b_r}h C r Δ sep − b r h because b r ≥ 2 s b_r\ge2s b r ≥ 2 s and Δ s e p ≤ 1 \Delta_{\mathrm{sep}}\le1 Δ sep ≤ 1 . For one time and consecutive pairs, the definitions of D ^ π \widehat{D}_{\pi} D π and D ^ A \widehat{D}_{A} D A give the same conclusions. All constants have the stated dependence, completing the proof.
Proof of Corollary 4.10. The balls B ( a j , τ ) B(a_j,\tau) B ( a j , τ ) are disjoint because 2 τ ≤ δ s e p / 2 < δ s e p 2\tau\le\delta_{\mathrm{sep}}/2<\delta_{\mathrm{sep}} 2 τ ≤ δ sep /2 < δ sep . Thus the K K K reference centers determine a unique bijection σ \sigma σ with the true centers. Apply σ \sigma σ to the labels after grouping with the originally indexed references. This transports the original smallest-index tie rule; it does not recompute a new tie order after permuting the centers. Every assigned reference remains a nearest reference under the aligned indices. Hence the coupling proof of Proposition 4.8 , which only uses this minimizing property, applies to σ # P ^ θ I \sigma_{\#}\widehat{P}^{I}_{\theta} σ # P θ I . The changed-label event is the same for all I I I , and the same permutation yields all the stated bounds. Since T T T is fixed, all 2 T − 1 2T-1 2 T − 1 inequalities hold on the same center-accuracy event without any additional union bound.
For the statistical assertion, on the global-MLE matching event Corollary 3.5 supplies (45) with τ = Δ s e p / 4 \tau=\Delta_{\mathrm{sep}}/4 τ = Δ sep /4 . Intersect it with the stated density event for θ ~ n \widetilde{\theta}_n θ n and apply the deterministic bounds. The union bound gives failure probability at most m − 1 + α n m^{-1}+\alpha_n m − 1 + α n , regardless of dependence between the estimators. For measurability, choose the first admissible permutation in a fixed ordering on the matching event, and the identity otherwise. The finitely many matching conditions are Borel in the reference centers, so this convention defines a measurable relabeling.
Finite-sample likelihood analysis
We prove Proposition 2.2 , which bounds the Hellinger error of the global maximum likelihood estimator. The likelihood maximization is over the entire parameter space, so the proof must control the likelihood comparison simultaneously for all fitted parameters. Lemma B.1 constructs finitely many upper functions, each of which dominates a group of observation densities and has integral close to one. Lemma B.2 uses the domination and integral bounds in (B.1), combined with the collection-size bound (B.2), to control the probability of a large Hellinger error. Appendix § 10 .§ 1 finally establishes the existence of a measurable global maximizer and chooses the radius in Proposition 2.2 .
We construct the finite collection by grouping parameters around finitely many representatives. For each representative, we need a function that dominates every observation density in its group. Its integral must remain close to one, since an upper function with large excess mass would give a poor likelihood comparison. We must also bound the number of functions, since the probability bound will sum over the collection. The following lemma provides these three properties.
Lemma B.1 (Upper likelihood cover). For every 0 < η ≤ 1 / 16 0 < \eta\le1/16 0 < η ≤ 1/16 , there is a finite collection of nonnegative integrable functions { u b : b ∈ B η } \{u_b : b \in\mathcal{B}_{\eta}\} { u b : b ∈ B η } such that every b ∈ B η b \in\mathcal{B}_{\eta} b ∈ B η satisfies the integral bounds below, and every θ ∈ Θ k ( R ) \theta\in\Theta_k(R) θ ∈ Θ k ( R ) has at least one b ∈ B η b \in\mathcal{B}_{\eta} b ∈ B η satisfying the pointwise domination:
p θ , T ( x 1 : T ) ≤ u b ( x 1 : T ) for all x 1 : T ∈ ( R d ) T , p_{\theta,T}(x_{1:T}) \le u_b(x_{1:T}) \qquad\text{for all } x_{1:T} \in(\mathbb{R}^d)^T, p θ , T ( x 1 : T ) ≤ u b ( x 1 : T ) for all x 1 : T ∈ ( R d ) T , 1 ≤ ∫ ( R d ) T u b ( x 1 : T ) d x 1 : T ≤ 1 + η . (56) 1 \le\int_{(\mathbb{R}^d)^T} u_b(x_{1:T})\,\mathrm{d}x_{1:T} \le1+\eta. \tag*{(56)} 1 ≤ ∫ ( R d ) T u b ( x 1 : T ) d x 1 : T ≤ 1 + η . (56) Moreover, with m k = k + k 2 + k d m_k = k + k^2 + kd m k = k + k 2 + k d ,
log ∣ B η ∣ ≤ m k log { C T ( k + 1 ) ( R + 1 ) ( d + 1 ) η } . (57) \log|\mathcal{B}_{\eta}| \le m_k \log\left\{\frac{CT(k+1)(R+1)(d+1)}{\eta}\right\}. \tag*{(57)} log ∣ B η ∣ ≤ m k log { η C T ( k + 1 ) ( R + 1 ) ( d + 1 ) } . (57)
Proof. Fix 0 < ρ ≤ 1 / 2 0 < \rho\le1/2 0 < ρ ≤ 1/2 . Partition each probability simplex into coordinate cells of side at most ρ \rho ρ , taking a simplex-valued representative from every intersecting cell, and take a ρ \rho ρ -net of B d ( 0 , R ) B_d(0,R) B d ( 0 , R ) for each emission center. This constructs a finite map θ ↦ θ 0 \theta\mapsto\theta^0 θ ↦ θ 0 satisfying
∥ π − π 0 ∥ ∞ ∨ ∥ A − A 0 ∥ ∞ ∨ max i ∥ μ i − μ i 0 ∥ 2 ≤ ρ , N ρ ≤ { 2 ( R + 1 ) ρ } m k , \|\pi-\pi^0\|_{\infty} \vee\|A-A^0\|_{\infty} \vee\max_i \|\mu_i-\mu_i^0\|_2 \le\rho,\qquad
N_\rho\le\left\{\frac{2(R+1)}{\rho}\right\}^{m_k}, ∥ π − π 0 ∥ ∞ ∨ ∥ A − A 0 ∥ ∞ ∨ i max ∥ μ i − μ i 0 ∥ 2 ≤ ρ , N ρ ≤ { ρ 2 ( R + 1 ) } m k ,
where m k = k + k 2 + k d m_k = k + k^2 + kd m k = k + k 2 + k d . The covering count follows from the coordinate-simplex grids and the volume bound for Euclidean ball nets.
For a representative center, put f μ 0 , ρ ( x ) = φ d ( x − μ 0 ) e ρ ∥ x − μ 0 ∥ 2 f_{\mu^0,\rho}(x) = \varphi_d(x-\mu^0)e^{\rho\|x-\mu^0\|_2} f μ 0 , ρ ( x ) = φ d ( x − μ 0 ) e ρ ∥ x − μ 0 ∥ 2 . The Gaussian likelihood ratio gives φ d ( x − μ ) ≤ f μ 0 , ρ ( x ) \varphi_d(x-\mu) \le f_{\mu^0,\rho}(x) φ d ( x − μ ) ≤ f μ 0 , ρ ( x ) whenever ∥ μ − μ 0 ∥ 2 ≤ ρ \|\mu-\mu^0\|_2 \le\rho ∥ μ − μ 0 ∥ 2 ≤ ρ . Its integral does not depend on μ 0 \mu^0 μ 0 :
M d ( ρ ) = E e ρ ∥ Z ∥ 2 ≤ e ρ d / 2 ( 1 − ρ / d ) − d / 2 ≤ e ρ d + ρ 2 / 2 , Z ∼ N d ( 0 , I d ) , M_d(\rho) = \mathbb{E}e^{\rho\|Z\|_2} \le e^{\rho\sqrt{d}/2}(1-\rho/\sqrt{d})^{-d/2} \le e^{\rho\sqrt{d}+\rho^2/2},\qquad Z\sim N_d(0,I_d), M d ( ρ ) = E e ρ ∥ Z ∥ 2 ≤ e ρ d /2 ( 1 − ρ / d ) − d /2 ≤ e ρ d + ρ 2 /2 , Z ∼ N d ( 0 , I d ) ,
provided ρ / d ≤ 1 / 2 \rho/\sqrt{d} \le1/2 ρ / d ≤ 1/2 . The first inequality follows from ∥ Z ∥ 2 ≤ d / 2 + ∥ Z ∥ 2 2 / ( 2 d ) \|Z\|_2 \le\sqrt{d}/2+\|Z\|_2^2/(2\sqrt{d}) ∥ Z ∥ 2 ≤ d /2 + ∥ Z ∥ 2 2 / ( 2 d ) and the chi-square moment-generating function; the second uses − log ( 1 − t ) ≤ t + t 2 -\log(1-t) \le t+t^2 − log ( 1 − t ) ≤ t + t 2 for 0 ≤ t ≤ 1 / 2 0 \le t \le1/2 0 ≤ t ≤ 1/2 .
For each representative θ 0 \theta^0 θ 0 , inflate the discrete path weights coordinatewise, setting π i + = π i 0 + ρ \pi_i^+ = \pi_i^0+\rho π i + = π i 0 + ρ and A i j + = A i j 0 + ρ A_{ij}^+ = A_{ij}^0+\rho A ij + = A ij 0 + ρ , and define
u θ 0 , ρ ( x 1 : T ) = ∑ z ∈ [ k ] T π z 1 + ∏ t = 1 T − 1 A z t z t + 1 + ∏ t = 1 T f μ z t 0 , ρ ( x t ) . u_{\theta^0,\rho}(x_{1:T}) = \sum_{z\in[k]^T} \pi_{z_1}^+ \prod_{t=1}^{T-1} A_{z_tz_{t+1}}^+ \prod_{t=1}^{T} f_{\mu_{z_t}^0,\rho}(x_t). u θ 0 , ρ ( x 1 : T ) = z ∈ [ k ] T ∑ π z 1 + t = 1 ∏ T − 1 A z t z t + 1 + t = 1 ∏ T f μ z t 0 , ρ ( x t ) .
It dominates p θ , T p_{\theta,T} p θ , T for every parameter mapped to θ 0 \theta^0 θ 0 . Each inflated initial or transition row sums to 1 + k ρ 1+k\rho 1 + k ρ , so integration and successive summation over the hidden path give
1 ≤ ∫ u θ 0 , ρ = M d ( ρ ) T ( 1 + k ρ ) T ≤ exp { T ρ ( k + d + 1 ) } . 1 \le\int u_{\theta^0,\rho} = M_d(\rho)^T(1+k\rho)^T \le\exp\{T\rho(k+\sqrt{d}+1)\}. 1 ≤ ∫ u θ 0 , ρ = M d ( ρ ) T ( 1 + k ρ ) T ≤ exp { T ρ ( k + d + 1 )} .
Choose ρ = η / [ 4 T ( k + d + 1 ) ] \rho= \eta/[4T(k+\sqrt{d}+1)] ρ = η / [ 4 T ( k + d + 1 )] ; it satisfies the restrictions above, and e η / 4 ≤ 1 + η e^{\eta/4} \le1+\eta e η /4 ≤ 1 + η for 0 < η ≤ 1 / 16 0 < \eta\le1/16 0 < η ≤ 1/16 . Thus every upper function has integral in [ 1 , 1 + η ] [1,1+\eta] [ 1 , 1 + η ] and the family dominates every fitted density. Finally,
log ∣ B η ∣ ≤ log N ρ ≤ m k log { 8 T ( R + 1 ) ( k + d + 1 ) η } ≤ m k log { C T ( k + 1 ) ( R + 1 ) ( d + 1 ) η } . \log|\mathcal{B}_{\eta}| \le\log N_\rho\le m_k\log\left\{\frac{8T(R+1)(k+\sqrt{d}+1)}{\eta}\right\}
\le m_k\log\left\{\frac{CT(k+1)(R+1)(d+1)}{\eta}\right\}. log ∣ B η ∣ ≤ log N ρ ≤ m k log { η 8 T ( R + 1 ) ( k + d + 1 ) } ≤ m k log { η C T ( k + 1 ) ( R + 1 ) ( d + 1 ) } .
No separation or positive fitted-weight assumption is used.
Lemma B.2 (Hellinger tail bound). Let p ^ = p θ ^ , T \widehat{p}=p_{\widehat{\theta},T} p = p θ , T be any measurable global maximum likelihood estimator over Θ k ( R ) \Theta_k(R) Θ k ( R ) , with k ≥ K k \ge K k ≥ K , and let p ⋆ = p θ ⋆ , T p^\star=p_{\theta^\star,T} p ⋆ = p θ ⋆ , T . For n n n independent length-T T T training trajectories and every 0 < r ≤ 1 0<r\le1 0 < r ≤ 1 ,
P ⋆ { h ( p ^ , p ⋆ ) > r } ≤ exp [ m k log { C T ( k + 1 ) ( R + 1 ) ( d + 1 ) r 4 } − 3 4 n r 2 ] , (58) \mathbb{P}_\star\{h(\widehat{p},p^\star)>r\}\le\exp\left[m_k\log\left\{\frac{CT(k+1)(R+1)(d+1)}{r^4}\right\}-\frac{3}{4}nr^2\right], \tag*{(58)} P ⋆ { h ( p , p ⋆ ) > r } ≤ exp [ m k log { r 4 C T ( k + 1 ) ( R + 1 ) ( d + 1 ) } − 4 3 n r 2 ] , (58) where m k = k + k 2 + k d m_k=k+k^2+kd m k = k + k 2 + k d and C > 0 C>0 C > 0 is a numerical constant.
Proof. Fix 0 < r ≤ 1 0<r\le1 0 < r ≤ 1 and apply Lemma B.1 with η = r 4 / 16 \eta=r^4/16 η = r 4 /16 . Retain precisely those upper functions u b u_b u b whose domination cells contain some density p p p with h ( p , p ⋆ ) > r h(p,p^\star)>r h ( p , p ⋆ ) > r . For each retained u b u_b u b , since u b ≥ p u_b\ge p u b ≥ p and ∫ ( u b − p ) ≤ η \int(u_b-p)\le\eta ∫ ( u b − p ) ≤ η , Cauchy–Schwarz yields
∫ u b p ⋆ ≤ ∫ p p ⋆ + { ∫ ( u b − p ) 2 } 1 / 2 ≤ 1 − r 2 2 + η = 1 − 3 r 2 4 . (59) \int\sqrt{u_bp^\star}\le\int\sqrt{pp^\star}+\left\{\int(\sqrt{u_b}-\sqrt{p})^2\right\}^{1/2}\le1-\frac{r^2}{2}+\sqrt{\eta}=1-\frac{3r^2}{4}. \tag*{(59)} ∫ u b p ⋆ ≤ ∫ p p ⋆ + { ∫ ( u b − p ) 2 } 1/2 ≤ 1 − 2 r 2 + η = 1 − 4 3 r 2 . (59)
The retained collection is deterministic (depends on p ⋆ p^\star p ⋆ and r r r , not on the sample). By the unused-state embedding in the model definition, p ⋆ p^\star p ⋆ belongs to the fitted density class for k ≥ K k\ge K k ≥ K . Hence a global MLE satisfies ∏ ℓ = 1 n p ^ ( X ( ℓ ) ) ≥ ∏ ℓ = 1 n p ⋆ ( X ( ℓ ) ) \prod_{\ell=1}^n\widehat{p}(X^{(\ell)})\ge\prod_{\ell=1}^np^\star(X^{(\ell)}) ∏ ℓ = 1 n p ( X ( ℓ ) ) ≥ ∏ ℓ = 1 n p ⋆ ( X ( ℓ ) ) . If h ( p ^ , p ⋆ ) > r h(\widehat{p},p^\star)>r h ( p , p ⋆ ) > r , a retained u b ≥ p ^ u_b\ge\widehat{p} u b ≥ p therefore satisfies ∏ ℓ u b ( X ( ℓ ) ) / p ⋆ ( X ( ℓ ) ) ≥ 1 \prod_{\ell}u_b(X^{(\ell)})/p^\star(X^{(\ell)})\ge1 ∏ ℓ u b ( X ( ℓ ) ) / p ⋆ ( X ( ℓ ) ) ≥ 1 . The denominators are strictly positive, because every nonempty Gaussian mixture density is positive everywhere. For each fixed u b u_b u b , independence, Markov’s inequality applied to the square-root likelihood ratio, and (B.4) give
P ⋆ { ∏ ℓ = 1 n u b ( X ( ℓ ) ) p ⋆ ( X ( ℓ ) ) ≥ 1 } ≤ ( ∫ u b p ⋆ ) n ≤ e − 3 n r 2 / 4 . \mathbb{P}_\star\left\{\prod_{\ell=1}^n\frac{u_b(X^{(\ell)})}{p^\star(X^{(\ell)})}\ge1\right\}\le\left(\int\sqrt{u_bp^\star}\right)^n\le e^{-3nr^2/4}. P ⋆ { ℓ = 1 ∏ n p ⋆ ( X ( ℓ ) ) u b ( X ( ℓ ) ) ≥ 1 } ≤ ( ∫ u b p ⋆ ) n ≤ e − 3 n r 2 /4 .
A union bound over at most ∣ B η ∣ |\mathcal{B}_\eta| ∣ B η ∣ functions and (B.2), with η = r 4 / 16 \eta=r^4/16 η = r 4 /16 , prove (B.3); absorb the factor 16 into C C C . If the retained family is empty, the bad event is empty. □
Proof of Proposition 2.2
Lemma B.2 bounds the probability of a large Hellinger error for a measurable global maximum likelihood estimator. We complete the proof of Proposition 2.2 by establishing the existence of such an estimator and choosing the radius that gives the stated probability bound.
Proof. We first prove that (i) the likelihood attains a global maximum for every sample and (ii) a global maximizer can be selected as a Borel measurable function of the sample. We then derive the Hellinger probability bound in Proposition 2.2 from Lemma B.2.
(i) Existence of a global maximizer. The parameter space Θ k ( R ) \Theta_k(R) Θ k ( R ) is a finite product of closed probability simplexes and closed Euclidean balls, and is therefore a nonempty compact subset of R m k \mathbb{R}^{m_k} R m k . The finite hidden-path expansion shows that ( θ , y 1 : T ) ↦ p θ , T ( y 1 : T ) (\theta,y_{1:T})\mapsto p_{\theta,T}(y_{1:T}) ( θ , y 1 : T ) ↦ p θ , T ( y 1 : T ) is jointly continuous. As verified in the proof of Lemma B.2, this density is strictly positive for every admissible parameter and every observation vector. Thus, for a sample y = ( y ( 1 ) , … , y ( n ) ) y=(y^{(1)},\ldots,y^{(n)}) y = ( y ( 1 ) , … , y ( n ) ) , the likelihood L ( θ ; y ) = ∏ ℓ = 1 n p θ , T ( y ( ℓ ) ) L(\theta;y)=\prod_{\ell=1}^n p_{\theta,T}(y^{(\ell)}) L ( θ ; y ) = ∏ ℓ = 1 n p θ , T ( y ( ℓ ) ) is positive and jointly continuous in ( θ , y ) (\theta,y) ( θ , y ) . Compactness ensures that it attains a global maximum over Θ k ( R ) \Theta_k(R) Θ k ( R ) . Its maximizers also maximize the log likelihood, since the logarithm is strictly increasing.
(ii) A measurable global maximizer. We now construct a global maximizer θ ^ n ( y ) \widehat{\theta}_n(y) θ n ( y ) that is a Borel measurable function of the sample y y y . Enclose Θ k ( R ) \Theta_k(R) Θ k ( R ) in a fixed closed coordinate cube and repeatedly subdivide each cube into its smaller closed cubes obtained by bisecting each side. Fix an ordering of the children of every cube. For each cube Q Q Q meeting Θ k ( R ) \Theta_k(R) Θ k ( R ) , define
M Q ( y ) = max θ ∈ Q ∩ Θ k ( R ) L ( θ ; y ) , M ( y ) = max θ ∈ Θ k ( R ) L ( θ ; y ) . M_Q(y)=\max_{\theta\in Q\cap\Theta_k(R)} L(\theta;y),\qquad M(y)=\max_{\theta\in\Theta_k(R)} L(\theta;y). M Q ( y ) = θ ∈ Q ∩ Θ k ( R ) max L ( θ ; y ) , M ( y ) = θ ∈ Θ k ( R ) max L ( θ ; y ) .
Both functions are continuous in y y y . Indeed, joint continuity makes L L L uniformly continuous on Θ k ( R ) \Theta_k(R) Θ k ( R ) times any compact neighborhood of a fixed sample. Hence, if y j → y y_j\to y y j → y , then
∣ M Q ( y j ) − M Q ( y ) ∣ ≤ sup θ ∈ Θ k ( R ) ∣ L ( θ ; y j ) − L ( θ ; y ) ∣ ⟶ 0 , \lvert M_Q(y_j)-M_Q(y)\rvert\le\sup_{\theta\in\Theta_k(R)}\lvert L(\theta;y_j)-L(\theta;y)\rvert\longrightarrow0, ∣ M Q ( y j ) − M Q ( y )∣ ≤ θ ∈ Θ k ( R ) sup ∣ L ( θ ; y j ) − L ( θ ; y )∣ ⟶ 0 ,
and the same bound holds for M M M .
Starting from the outer cube, select the first child Q Q Q meeting Θ k ( R ) \Theta_k(R) Θ k ( R ) for which M Q ( y ) = M ( y ) M_Q(y)=M(y) M Q ( y ) = M ( y ) , and repeat this selection inside the chosen cube. Such a child exists at every depth because the parent contains a global maximizer and its children cover it. Since M Q M_Q M Q and M M M are continuous, the set { y : M Q ( y ) = M ( y ) } \{y:M_Q(y)=M(y)\} { y : M Q ( y ) = M ( y )} is Borel. The fixed ordering therefore makes the selected cube at every finite depth a Borel measurable function of y y y .
For every cube meeting Θ k ( R ) \Theta_k(R) Θ k ( R ) , fix a representative in its intersection with Θ k ( R ) \Theta_k(R) Θ k ( R ) , independently of the sample. The representative of the selected cube at depth j j j is a finite-valued measurable function of y y y . The selected cubes are nested and their diameters tend to zero, so these representatives converge to a measurable limit θ ^ n ( y ) ∈ Θ k ( R ) \hat{\theta}_n(y)\in\Theta_k(R) θ ^ n ( y ) ∈ Θ k ( R ) . For each fixed y y y , a global maximizer in the selected cube lies within its diameter of the representative. These maximizing points therefore converge to θ ^ n ( y ) \hat{\theta}_n(y) θ ^ n ( y ) as well, and continuity gives L ( θ ^ n ( y ) ; y ) = M ( y ) L(\hat{\theta}_n(y);y)=M(y) L ( θ ^ n ( y ) ; y ) = M ( y ) . This constructs a measurable global maximizer even when the maximum is attained at more than one parameter.
(iii) Hellinger probability bound. We consider n ≥ 2 n\ge2 n ≥ 2 and set L n = log ( n T d ) L_n=\log(nTd) L n = log ( n T d ) and r = C d L n / n r=C\sqrt{dL_n/n} r = C d L n / n , where C ≥ ( log 2 ) − 1 / 2 C\ge(\log2)^{-1/2} C ≥ ( log 2 ) − 1/2 will be chosen below depending only on k k k and R R R . If r ≥ 1 r\ge1 r ≥ 1 , the claimed bound follows immediately from h ≤ 1 h\le1 h ≤ 1 . We therefore consider r < 1 r<1 r < 1 .
Let C t a i l C_{\mathrm{tail}} C tail denote the numerical constant in Lemma B.2 , enlarged if necessary so that C t a i l ≥ 1 C_{\mathrm{tail}}\ge1 C tail ≥ 1 , and write a T = C t a i l T ( k + 1 ) ( R + 1 ) ( d + 1 ) a_T=C_{\mathrm{tail}}T(k+1)(R+1)(d+1) a T = C tail T ( k + 1 ) ( R + 1 ) ( d + 1 ) . Since d ≥ 1 d\ge1 d ≥ 1 , we have a T ≤ B k , R T d a_T\le B_{k,R}Td a T ≤ B k , R T d , where B k , R = 2 C t a i l ( k + 1 ) ( R + 1 ) B_{k,R}=2C_{\mathrm{tail}}(k+1)(R+1) B k , R = 2 C tail ( k + 1 ) ( R + 1 ) . Moreover, L n ≥ log 2 L_n\ge\log2 L n ≥ log 2 and C ≥ ( log 2 ) − 1 / 2 C\ge(\log2)^{-1/2} C ≥ ( log 2 ) − 1/2 imply r ≥ n − 1 / 2 r\ge n^{-1/2} r ≥ n − 1/2 . Consequently,
log ( a T / r 4 ) ≤ log B k , R + log ( T d ) + 2 log n ≤ ( 2 + log B k , R log 2 ) L n . \log(a_T/r^4)\le\log B_{k,R}+\log(Td)+2\log n\le\left(2+\frac{\log B_{k,R}}{\log2}\right)L_n. log ( a T / r 4 ) ≤ log B k , R + log ( T d ) + 2 log n ≤ ( 2 + log 2 log B k , R ) L n .
Combining this inequality with m k = k d + k 2 + k ≤ ( k 2 + 2 k ) d m_k=kd+k^2+k\le(k^2+2k)d m k = k d + k 2 + k ≤ ( k 2 + 2 k ) d gives m k log ( a T / r 4 ) ≤ A k , R d L n m_k\log(a_T/r^4)\le A_{k,R}dL_n m k log ( a T / r 4 ) ≤ A k , R d L n for a constant A k , R > 0 A_{k,R}>0 A k , R > 0 depending only on k k k and R R R . Lemma B.2 now yields
P ⋆ { h ( p θ ^ n , T , p θ ⋆ , T ) > r } ≤ exp { ( A k , R − 3 4 C 2 ) d L n } . \mathbb{P}_{\star}\left\{h(p_{\hat{\theta}_n,T},p_{\theta^\star,T})>r\right\}\le\exp\left\{\left(A_{k,R}-\frac{3}{4}C^2\right)dL_n\right\}. P ⋆ { h ( p θ ^ n , T , p θ ⋆ , T ) > r } ≤ exp { ( A k , R − 4 3 C 2 ) d L n } .
Choose C C C sufficiently large that 3 4 C 2 ≥ A k , R + 1 \frac{3}{4}C^2\ge A_{k,R}+1 4 3 C 2 ≥ A k , R + 1 . The right-hand side is then at most e − d L n ≤ 1 / n e^{-dL_n}\le1/n e − d L n ≤ 1/ n . Since r = ε n , T r=\varepsilon_{n,T} r = ε n , T , this proves Proposition 2.2 . The argument applies to every measurable global maximizer, including when the maximizing parameter is not unique.
Measurability of the statistical losses. The measurable selection just constructed must also be compatible with the losses and the matching rule used later. We verify those properties for each fixed true parameter.
First, the map θ ↦ p θ , T ( x ) \theta\mapsto p_{\theta,T}(x) θ ↦ p θ , T ( x ) is continuous for every x x x , and its values are probability densities. If θ m → θ \theta_m \to\theta θ m → θ , dominated convergence applied to min { p θ m , T , p θ , T } ≤ p θ , T \min\{p_{\theta_m,T},p_{\theta,T}\} \le p_{\theta,T} min { p θ m , T , p θ , T } ≤ p θ , T gives
∫ ∣ p θ m , T − p θ , T ∣ = 2 − 2 ∫ min { p θ m , T , p θ , T } ⟶ 0. \int|p_{\theta_m,T}-p_{\theta,T}| = 2-2\int\min\{p_{\theta_m,T},p_{\theta,T}\} \longrightarrow0. ∫ ∣ p θ m , T − p θ , T ∣ = 2 − 2 ∫ min { p θ m , T , p θ , T } ⟶ 0.
Because h 2 ( p , q ) ≤ 1 2 ∥ p − q ∥ 1 h^2(p,q) \le\frac{1}{2}\lVert p-q\rVert_1 h 2 ( p , q ) ≤ 2 1 ∥ p − q ∥ 1 , the observation density is continuous in Hellinger distance. The triangle inequality then makes θ ↦ h ( p θ , T , p θ ⋆ , T ) \theta\mapsto h(p_{\theta,T},p_{\theta^\star,T}) θ ↦ h ( p θ , T , p θ ⋆ , T ) continuous. In particular, the density events used in the probability bounds are measurable.
For each fitted label i i i , the smallest-index nearest-center rule has the Borel cells
{ c ( i ) = j } = ⋂ ℓ < j { ∥ μ i − a j ∥ 2 < ∥ μ i − a ℓ ∥ 2 } ∩ ⋂ ℓ > j { ∥ μ i − a j ∥ 2 ≤ ∥ μ i − a ℓ ∥ 2 } . \{c(i)=j\}=\bigcap_{\ell<j}\{\lVert\mu_i-a_j\rVert_2<\lVert\mu_i-a_\ell\rVert_2\}\cap\bigcap_{\ell>j}\{\lVert\mu_i-a_j\rVert_2\le\lVert\mu_i-a_\ell\rVert_2\}. { c ( i ) = j } = ℓ < j ⋂ {∥ μ i − a j ∥ 2 < ∥ μ i − a ℓ ∥ 2 } ∩ ℓ > j ⋂ {∥ μ i − a j ∥ 2 ≤ ∥ μ i − a ℓ ∥ 2 } .
State and path probabilities are polynomial functions of π , A \pi,A π , A . Hence each grouped probability is a finite sum of continuous path probabilities multiplied by these Borel indicators. This proves that D I , D π , D A , D μ , V , B D_I,D_\pi,D_A,D_\mu,V,B D I , D π , D A , D μ , V , B , and J I J_I J I are Borel, including at nearest-center ties.
The Wasserstein losses are in fact continuous despite repeated centers and zero path probabilities. To see this, put λ z ( θ ) = π z 1 ∏ t < T A z t z t + 1 \lambda_z(\theta)=\pi_{z_1}\prod_{t<T}A_{z_tz_{t+1}} λ z ( θ ) = π z 1 ∏ t < T A z t z t + 1 for z ∈ [ k ] T z\in[k]^T z ∈ [ k ] T . Couple common mass min { λ z ( θ m ) , λ z ( θ ) } \min\{\lambda_z(\theta_m),\lambda_z(\theta)\} min { λ z ( θ m ) , λ z ( θ )} at identical hidden paths. The remaining mass is 1 2 ∑ z ∈ [ k ] T ∣ λ z ( θ m ) − λ z ( θ ) ∣ \frac{1}{2}\sum_{z\in[k]^T}|\lambda_z(\theta_m)-\lambda_z(\theta)| 2 1 ∑ z ∈ [ k ] T ∣ λ z ( θ m ) − λ z ( θ ) ∣ and can be coupled arbitrarily; its normalized cost is at most ( 2 R ) p (2R)^p ( 2 R ) p . Thus, for p ∈ { 1 , 2 } p\in\{1,2\} p ∈ { 1 , 2 } and any nonempty I I I ,
W p , I p ( G θ m I , G θ I ) ≤ max i ∈ [ k ] ∥ μ i ( θ m ) − μ i ( θ ) ∥ 2 p + ( 2 R ) p 2 ∑ z ∈ [ k ] T ∣ λ z ( θ m ) − λ z ( θ ) ∣ ⟶ 0. W_{p,I}^p(G_{\theta_m}^I,G_\theta^I)\le\max_{i\in[k]}\lVert\mu_i(\theta_m)-\mu_i(\theta)\rVert_2^p+\frac{(2R)^p}{2}\sum_{z\in[k]^T}|\lambda_z(\theta_m)-\lambda_z(\theta)|\longrightarrow0. W p , I p ( G θ m I , G θ I ) ≤ i ∈ [ k ] max ∥ μ i ( θ m ) − μ i ( θ ) ∥ 2 p + 2 ( 2 R ) p z ∈ [ k ] T ∑ ∣ λ z ( θ m ) − λ z ( θ ) ∣ ⟶ 0.
The triangle inequality proves continuity of the distance to the fixed true location law. The construction does not require distinct support points or strictly positive path weights.
Finally, when k = K k=K k = K , enumerate the finitely many permutations in a fixed order and select the first satisfying all matching inequalities ∥ b j − a j ∥ 2 ≤ Δ s e p / 4 \lVert b_j-a_j\rVert_2\le\Delta_{\mathrm{sep}}/4 ∥ b j − a j ∥ 2 ≤ Δ sep /4 , if one exists; select the identity permutation otherwise. Each feasibility set is closed, so the selected permutation is Borel. On the matching event it is the unique admissible permutation, because the true-center neighborhoods are disjoint. Off this event the convention still defines all relabeled center and transition-row losses. Consequently, the probability and O P \mathbb{O}_{\mathrm{P}} O P statements in the paper concern well-defined random variables on the entire sample space, rather than quantities defined only on a high-probability event.
Likelihood errors and approximate maximizers
The likelihood comparison in the preceding proof is the only step that uses global maximization. The following lemma allows a quantified likelihood shortfall and states the weaker comparison with the true likelihood that is actually sufficient. Write
ℓ n ( θ ) = ∑ i = 1 n log p θ , T ( X 1 : T ( i ) ) . \ell_n(\theta)=\sum_{i=1}^{n}\log p_{\theta,T}\left(X_{1:T}^{(i)}\right). ℓ n ( θ ) = i = 1 ∑ n log p θ , T ( X 1 : T ( i ) ) .
Lemma B.3 (Density control with a likelihood tolerance). Suppose that the assumptions of Proposition 2.2 hold, and let θ ~ n \widetilde{\theta}_n θ n be a measurable estimator taking values in Θ k ( R ) \Theta_k(R) Θ k ( R ) . Let τ n ≥ 0 \tau_n\ge0 τ n ≥ 0 and α n ∈ [ 0 , 1 ] \alpha_n\in[0,1] α n ∈ [ 0 , 1 ] be deterministic. If
P ⋆ ( G n ) ≥ 1 − α n , G n : = { ℓ n ( θ ~ n ) ≥ ℓ n ( θ ⋆ ) − n τ n } , (60) \mathbb{P}_\star(\mathcal{G}_n)\ge1-\alpha_n,\qquad\mathcal{G}_n:=\{\ell_n(\widetilde{\theta}_n)\ge\ell_n(\theta^\star)-n\tau_n\}, \tag*{(60)} P ⋆ ( G n ) ≥ 1 − α n , G n := { ℓ n ( θ n ) ≥ ℓ n ( θ ⋆ ) − n τ n } , (60)
then a constant C = C ( k , R ) C=C(k,R) C = C ( k , R ) satisfies, for every n ≥ 2 n\ge2 n ≥ 2 ,
P ⋆ { h ( p θ ~ n , T , p θ ⋆ , T ) > C d log ( n T d ) n + τ n } ≤ 1 n + α n . (61) \mathbb{P}_{\star}\left\{h(p_{\widetilde{\theta}_{n,T}},p_{\theta^\star,T})>C\sqrt{\frac{d\log(nTd)}{n}+\tau_n}\right\}\le\frac{1}{n}+\alpha_n. \tag*{(61)} P ⋆ { h ( p θ n , T , p θ ⋆ , T ) > C n d log ( n T d ) + τ n } ≤ n 1 + α n . (61)
A sufficient condition for equation (B.5) is that ℓ n ( θ ~ n ) ≥ sup θ ∈ Θ k ( R ) ℓ n ( θ ) − n τ n \ell_n(\widetilde{\theta}_n)\ge\sup_{\theta\in\Theta_k(R)}\ell_n(\theta)-n\tau_n ℓ n ( θ n ) ≥ sup θ ∈ Θ k ( R ) ℓ n ( θ ) − n τ n with probability at least 1 − α n 1-\alpha_n 1 − α n . The probability statements may also include independent auxiliary randomness used to construct the estimator.
Proof. Put p ⋆ = p θ ⋆ , T p^\star=p_{\theta^\star,T} p ⋆ = p θ ⋆ , T and m k = k d + k 2 + k m_k=kd+k^2+k m k = k d + k 2 + k . We first establish the explicit tail comparison
P ⋆ { h ( p θ ~ n , T , p ⋆ ) > r , G n } ≤ exp { m k log ( C c o v T ( k + 1 ) ( R + 1 ) ( d + 1 ) r 4 ) − 3 4 n r 2 + 1 2 n τ n } , 0 < r ≤ 1. (62) \mathbb{P}_{\star}\{h(p_{\widetilde{\theta}_{n,T}},p^\star)>r,\mathcal{G}_n\}
\le\exp\left\{m_k\log\left(\frac{C_{\mathrm{cov}}T(k+1)(R+1)(d+1)}{r^4}\right)-\frac{3}{4}nr^2+\frac{1}{2}n\tau_n\right\},\qquad0<r\le1. \tag*{(62)} P ⋆ { h ( p θ n , T , p ⋆ ) > r , G n } ≤ exp { m k log ( r 4 C cov T ( k + 1 ) ( R + 1 ) ( d + 1 ) ) − 4 3 n r 2 + 2 1 n τ n } , 0 < r ≤ 1. (62)
where C c o v ≥ 1 C_{\mathrm{cov}}\ge1 C cov ≥ 1 is numerical. To see this, use the upper cover in Lemma B.1 with η = r 4 / 16 \eta=r^4/16 η = r 4 /16 , and retain exactly the cells used in Lemma B.2 . These cells are fixed before the sample is observed. For every retained upper function u b u_b u b , equation (B.4) gives
∫ u b p ⋆ ≤ 1 − 3 4 r 2 . \int\sqrt{u_bp^\star}\le1-\frac{3}{4}r^2. ∫ u b p ⋆ ≤ 1 − 4 3 r 2 .
Define the nonnegative random variable
Z b : = ∏ i = 1 n u b ( X 1 : T ( i ) ) p ⋆ ( X 1 : T ( i ) ) . Z_b:=\prod_{i=1}^{n}\sqrt{\frac{u_b(X_{1:T}^{(i)})}{p^\star(X_{1:T}^{(i)})}}. Z b := i = 1 ∏ n p ⋆ ( X 1 : T ( i ) ) u b ( X 1 : T ( i ) ) .
Its expectation is at most ( 1 − 3 r 2 / 4 ) n ≤ e − 3 n r 2 / 4 (1-3r^2/4)^n\le e^{-3nr^2/4} ( 1 − 3 r 2 /4 ) n ≤ e − 3 n r 2 /4 by independence of the trajectories. On the event on the left of equation (B.7), some retained u b u_b u b dominates p θ ~ n , T p_{\widetilde{\theta}_{n,T}} p θ n , T , and the likelihood comparison gives Z b ≥ e − n τ n / 2 Z_b\ge e^{-n\tau_n/2} Z b ≥ e − n τ n /2 . Markov’s inequality and a union bound therefore yield the right-hand side of equation (B.7). If the estimator uses auxiliary randomness, this event inclusion still holds pointwise, while Z b Z_b Z b depends only on the observations; the same bound follows without any conditioning on a selected fit.
Set L n = log ( n T d ) L_n=\log(nTd) L n = log ( n T d ) and r = C d L n / n + τ n r=C\sqrt{dL_n/n+\tau_n} r = C d L n / n + τ n , with C ≥ ( log 2 ) − 1 / 2 C\ge(\log2)^{-1/2} C ≥ ( log 2 ) − 1/2 . When r ≥ 1 r\ge1 r ≥ 1 , equation (B.6) follows from h ≤ 1 h\le1 h ≤ 1 . Otherwise r ≥ n − 1 / 2 r\ge n^{-1/2} r ≥ n − 1/2 , and the covering logarithm is bounded exactly as in the proof of Proposition 2.2 :
m k log ( C c o v T ( k + 1 ) ( R + 1 ) ( d + 1 ) r 4 ) ≤ A k , R d L n . m_k\log\left(\frac{C_{\mathrm{cov}}T(k+1)(R+1)(d+1)}{r^4}\right)\le A_{k,R}dL_n. m k log ( r 4 C cov T ( k + 1 ) ( R + 1 ) ( d + 1 ) ) ≤ A k , R d L n .
Inserting the chosen r r r into equation (B.7) bounds its exponent by
( A k , R − 3 4 C 2 ) d L n + ( 1 2 − 3 4 C 2 ) n τ n . \left(A_{k,R}-\frac{3}{4}C^2\right)dL_n+\left(\frac{1}{2}-\frac{3}{4}C^2\right)n\tau_n. ( A k , R − 4 3 C 2 ) d L n + ( 2 1 − 4 3 C 2 ) n τ n .
Choose C C C so that 3 C 2 / 4 ≥ A k , R + 1 3C^2/4\ge A_{k,R}+1 3 C 2 /4 ≥ A k , R + 1 and 3 C 2 / 4 ≥ 1 / 2 3C^2/4\ge1/2 3 C 2 /4 ≥ 1/2 . The exponent is then at most − d L n -dL_n − d L n , whose exponential is at most 1 / n 1/n 1/ n . Adding P ⋆ ( G n c ) ≤ α n \mathbb{P}_{\star}(\mathcal{G}_n^c)\le\alpha_n P ⋆ ( G n c ) ≤ α n proves the claim. Finally, the true observation density belongs to the fitted class by the unused-state embedding already established, so a global likelihood-gap bound implies equation (B.5).
A tolerance τ n = O ( d log ( n T d ) / n ) \tau_n = O(d\log(nTd)/n) τ n = O ( d log ( n T d ) / n ) preserves the density order used in the recovery corollaries. More generally, outside the exceptional event bounded in equation (B.6), every deterministic inverse inequality applies with its density error replaced by the displayed radius. This observation requires a likelihood-gap or true-likelihood comparison with controlled probability. A small increase between successive EM iterates, or selection of the best among finitely many restarts, does not by itself supply that control.
Multivariate witness polynomials
This appendix section constructs polynomials that convert observation-density error into bounds on latent recovery. We begin with a nonnegative polynomial that vanishes at every true center and controls the squared distance to the true support. Throughout, x ∈ B d ( 0 , R ) x \in B_d(0,R) x ∈ B d ( 0 , R ) , and c ( x ) c(x) c ( x ) denotes the index of a nearest true center, with ties resolved by choosing the smallest index. When applying Lemma A.3 , we restrict the polynomials to the span of the true and fitted centers and take their supremum norms over the intersection of this span with B d ( 0 , R ) B_d(0,R) B d ( 0 , R ) .
The first subsections provide positive witnesses, flat selectors, and interpolation tools. Joint selectors then control multitime probabilities with explicit separation and diameter factors. The final subsection constructs polynomials that interpolate both true and fitted supports under an assumed pairing, for use in the matched bounds under exact specification.
A positive witness for all fitted centers
We first construct a polynomial that detects displacement from the true support without cancellation between fitted states. Define
P ( x ) = ∏ j = 1 K ∥ x − a j ∥ 2 . (63) P(x) = \prod_{j=1}^{K} \lVert x-a_j\rVert^2. \tag*{(63)} P ( x ) = j = 1 ∏ K ∥ x − a j ∥ 2 . (63)
Since P P P is nonnegative and vanishes at every true center, its expectation under the true latent distribution is zero, whereas its expectation under the fitted distribution is a sum of nonnegative terms. The following lemma shows that this sum controls the probability-weighted squared distances of fitted centers from their nearest true centers.
Lemma C.1 (Global positive coercivity). In a structured configuration, there is a constant c p o s > 0 c_{\mathrm{pos}}>0 c pos > 0 , independent of Δ s e p \Delta_{\mathrm{sep}} Δ sep , such that
P ( x ) ≥ c p o s Δ s e p 2 s max − 2 ∥ x − a c ( x ) ∥ 2 , ∥ P ∥ ∞ ≤ ( 2 R ) 2 K . (64) P(x) \ge c_{\mathrm{pos}}\Delta_{\mathrm{sep}}^{2s_{\max}-2}\lVert x-a_{c(x)}\rVert^2,\qquad\lVert P\rVert_\infty\le(2R)^{2K}. \tag*{(64)} P ( x ) ≥ c pos Δ sep 2 s m a x − 2 ∥ x − a c ( x ) ∥ 2 , ∥ P ∥ ∞ ≤ ( 2 R ) 2 K . (64)
Without a cluster assumption, the same statement holds with s max = K s_{\max}=K s m a x = K . Consequently, for every t ≤ T t\le T t ≤ T ,
0 ≤ ∑ i q t ( i ) P ( μ i ) ≤ C h ( p θ , T , p θ ⋆ , T ) , ∑ i q t ( i ) r i 2 ≤ C Δ s e p − ( 2 s max − 2 ) h ( p θ , T , p θ ⋆ , T ) . (65) \begin{aligned}
0 \le\sum_i q_t(i)P(\mu_i) &\le C h(p_{\theta,T},p_{\theta^\star,T}),\\
\sum_i q_t(i)r_i^2 &\le C\Delta_{\mathrm{sep}}^{-(2s_{\max}-2)}h(p_{\theta,T},p_{\theta^\star,T}). \tag*{(65)}
\end{aligned} 0 ≤ i ∑ q t ( i ) P ( μ i ) i ∑ q t ( i ) r i 2 ≤ C h ( p θ , T , p θ ⋆ , T ) , ≤ C Δ sep − ( 2 s m a x − 2 ) h ( p θ , T , p θ ⋆ , T ) . (65)
Proof. We first establish the pointwise lower bound for P P P and its supremum norm. We then apply Lemma A.3 to bound the fitted expectation of P P P and obtain the quadratic displacement bound in Equation (C.3).
Separating the nearest-center distance. Fix x ∈ B d ( 0 , R ) x \in B_d(0,R) x ∈ B d ( 0 , R ) , and write j 0 = c ( x ) j_0=c(x) j 0 = c ( x ) and ρ = ∥ x − a j 0 ∥ \rho=\lVert x-a_{j_0}\rVert ρ = ∥ x − a j 0 ∥ . We isolate the factor ρ 2 \rho^2 ρ 2 in P ( x ) P(x) P ( x ) and bound the remaining factors using distances between true centers. For each ℓ ≠ j 0 \ell\ne j_0 ℓ = j 0 , the nearest-center assignment gives ρ ≤ ∥ x − a ℓ ∥ \rho\le\lVert x-a_\ell\rVert ρ ≤ ∥ x − a ℓ ∥ , so
d j 0 ℓ : = ∥ a j 0 − a ℓ ∥ ≤ ∥ a j 0 − x ∥ + ∥ x − a ℓ ∥ (by triangle inequality) d_{j_0\ell}:=\lVert a_{j_0}-a_\ell\rVert\le\lVert a_{j_0}-x\rVert+\lVert x-a_\ell\rVert\qquad\text{(by triangle inequality)} d j 0 ℓ := ∥ a j 0 − a ℓ ∥ ≤ ∥ a j 0 − x ∥ + ∥ x − a ℓ ∥ (by triangle inequality)
≤ 2 ∥ x − a ℓ ∥ (nearest-center assignment) . \le2\lVert x-a_\ell\rVert\qquad\text{(nearest-center assignment)}. ≤ 2 ∥ x − a ℓ ∥ (nearest-center assignment) .
It follows that
P ( x ) = ρ 2 ∏ ℓ ≠ j 0 ∥ x − a ℓ ∥ 2 ≥ 4 − ( K − 1 ) ρ 2 ∏ ℓ ≠ j 0 d j 0 ℓ 2 , P(x)=\rho^2\prod_{\ell\ne j_0}\lVert x-a_\ell\rVert^2\ge4^{-(K-1)}\rho^2\prod_{\ell\ne j_0}d_{j_0\ell}^2, P ( x ) = ρ 2 ℓ = j 0 ∏ ∥ x − a ℓ ∥ 2 ≥ 4 − ( K − 1 ) ρ 2 ℓ = j 0 ∏ d j 0 ℓ 2 ,
where ∏ ℓ ≠ j 0 d j 0 ℓ 2 = : D j 0 \prod_{\ell\ne j_0}d_{j_0\ell}^2=:D_{j_0} ∏ ℓ = j 0 d j 0 ℓ 2 =: D j 0 .
This inequality holds throughout the ball, including points tied between nearest centers; it does not require x x x to be close to the true support.
Accounting for the cluster sizes. Suppose a j 0 a_{j_0} a j 0 belongs to a cluster C C C containing s j 0 ≤ s max s_{j_0}\le s_{\max} s j 0 ≤ s m a x true centers. The product D j 0 D_{j_0} D j 0 separates into distances to centers inside and outside this cluster:
D j 0 = ∏ ℓ ∈ C ℓ ≠ j 0 d j 0 ℓ 2 ⏟ s j 0 − 1 within-cluster factors ∏ ℓ ∉ C d j 0 ℓ 2 ⏟ K − s j 0 between-cluster factors ≥ Δ s e p 2 ( s j 0 − 1 ) D 0 2 ( K − s j 0 ) . D_{j_0}=\underbrace{\prod_{\substack{\ell\in C\\\ell\ne j_0}}d_{j_0\ell}^2}_{s_{j_0}-1\text{ within-cluster factors}}\underbrace{\prod_{\ell\notin C}d_{j_0\ell}^2}_{K-s_{j_0}\text{ between-cluster factors}}\ge\Delta_{\mathrm{sep}}^{2(s_{j_0}-1)}D_0^{2(K-s_{j_0})}. D j 0 = s j 0 − 1 within-cluster factors ℓ ∈ C ℓ = j 0 ∏ d j 0 ℓ 2 K − s j 0 between-cluster factors ℓ ∈ / C ∏ d j 0 ℓ 2 ≥ Δ sep 2 ( s j 0 − 1 ) D 0 2 ( K − s j 0 ) .
Only the within-cluster factors require the lower bound Δ s e p \Delta_{\mathrm{sep}} Δ sep ; the remaining distances are bounded below by the fixed constant D 0 D_0 D 0 . Let λ = min { 1 , D 0 } \lambda=\min\{1,D_0\} λ = min { 1 , D 0 } . Since 0 < Δ s e p ≤ 1 0<\Delta_{\mathrm{sep}}\le1 0 < Δ sep ≤ 1 and s j 0 ≤ s max s_{j_0}\le s_{\max} s j 0 ≤ s m a x , we have
Δ s e p 2 ( s j 0 − 1 ) ≥ Δ s e p 2 ( s max − 1 ) , D 0 2 ( K − s j 0 ) ≥ λ 2 ( K − s j 0 ) ≥ λ 2 K . \Delta_{\mathrm{sep}}^{2(s_{j_0}-1)}\ge\Delta_{\mathrm{sep}}^{2(s_{\max}-1)},\qquad D_0^{2(K-s_{j_0})}\ge\lambda^{2(K-s_{j_0})}\ge\lambda^{2K}. Δ sep 2 ( s j 0 − 1 ) ≥ Δ sep 2 ( s m a x − 1 ) , D 0 2 ( K − s j 0 ) ≥ λ 2 ( K − s j 0 ) ≥ λ 2 K .
Consequently,
P ( x ) ≥ 4 − ( K − 1 ) λ 2 K ⏟ = : c p o s Δ s e p 2 s max − 2 ∥ x − a c ( x ) ∥ 2 . P(x)\ge\underbrace{4^{-(K-1)}\lambda^{2K}}_{=:c_{\mathrm{pos}}}\Delta_{\mathrm{sep}}^{2s_{\max}-2}\lVert x-a_{c(x)}\rVert^2. P ( x ) ≥ =: c pos 4 − ( K − 1 ) λ 2 K Δ sep 2 s m a x − 2 ∥ x − a c ( x ) ∥ 2 .
Without a cluster assumption, every factor in D j 0 D_{j_0} D j 0 is at least Δ s e p 2 \Delta_{\mathrm{sep}}^2 Δ sep 2 , which gives
D j 0 ≥ Δ s e p 2 ( K − 1 ) . D_{j_0}\ge\Delta_{\mathrm{sep}}^{2(K-1)}. D j 0 ≥ Δ sep 2 ( K − 1 ) .
Thus the same argument applies with s max = K s_{\max}=K s m a x = K and c p o s = 4 − ( K − 1 ) c_{\mathrm{pos}}=4^{-(K-1)} c pos = 4 − ( K − 1 ) .
Bounding the polynomial norm. The polynomial P P P has degree 2 K 2K 2 K . Since x , a j ∈ B d ( 0 , R ) x,a_j\in B_d(0,R) x , a j ∈ B d ( 0 , R ) , the triangle inequality gives ∥ x − a j ∥ ≤ 2 R \lVert x-a_j\rVert\le2R ∥ x − a j ∥ ≤ 2 R for every j j j , and hence
0 ≤ P ( x ) = ∏ j = 1 K ∥ x − a j ∥ 2 ≤ ( 2 R ) 2 K . 0\le P(x)=\prod_{j=1}^{K}\lVert x-a_j\rVert^2\le(2R)^{2K}. 0 ≤ P ( x ) = j = 1 ∏ K ∥ x − a j ∥ 2 ≤ ( 2 R ) 2 K .
This proves the supremum bound in (C.2), which also holds after restricting P P P to the span of the true and fitted centers.
Controlling the fitted polynomial expectation. Fix t ≤ T t \le T t ≤ T . The true one-time latent location distribution is supported on { a 1 , … , a K } \{a_1,\ldots,a_K\} { a 1 , … , a K } , where P P P vanishes. Therefore,
∫ P d ( G θ { t } − G ⋆ { t } ) = ∑ i q t ( i ) P ( μ i ) − ∫ P d G ⋆ { t } ⏟ = 0 since P ( a j ) = 0 for all j = ∑ i q t ( i ) P ( μ i ) ≥ 0 since q t ( i ) ≥ 0 and P ( μ i ) ≥ 0 for all i . \int P\,\mathrm{d}\left(G_{\theta}^{\{t\}}-G_{\star}^{\{t\}}\right)=\sum_i q_t(i)P(\mu_i)-\underbrace{\int P\,\mathrm{d}G_{\star}^{\{t\}}}_{=0\ \text{since }P(a_j)=0\text{ for all }j}
=\sum_i q_t(i)P(\mu_i)\ge0\qquad\text{since }q_t(i)\ge0\text{ and }P(\mu_i)\ge0\text{ for all }i. ∫ P d ( G θ { t } − G ⋆ { t } ) = i ∑ q t ( i ) P ( μ i ) − = 0 since P ( a j ) = 0 for all j ∫ P d G ⋆ { t } = i ∑ q t ( i ) P ( μ i ) ≥ 0 since q t ( i ) ≥ 0 and P ( μ i ) ≥ 0 for all i .
Let p θ { t } p_{\theta}^{\{t\}} p θ { t } and p θ ⋆ { t } p_{\theta_\star}^{\{t\}} p θ ⋆ { t } denote the corresponding one-time observation densities. Applying Lemma A.3 on the span of the true and fitted centers gives
∑ i q t ( i ) P ( μ i ) = ∣ ∫ P d ( G θ { t } − G ⋆ { t } ) ∣ ≤ C ∥ P ∥ ∞ h ( p θ { t } , p θ ⋆ { t } ) (xref‘Lemma A.3‘) ≤ C ∥ P ∥ ∞ h ( p θ , T , p θ ⋆ , T ) (Hellinger contraction under marginalization) ≤ C h ( p θ , T , p θ ⋆ , T ) (the supremum bound in xref‘(C.2‘). \begin{aligned}
\sum_i q_t(i)P(\mu_i)=\left|\int P\,\mathrm{d}\left(G_{\theta}^{\{t\}}-G_{\star}^{\{t\}}\right)\right|
\le C\lVert P\rVert_{\infty}h\left(p_{\theta}^{\{t\}},p_{\theta_\star}^{\{t\}}\right)\qquad\text{({xref}`Lemma A.3`)} \\
&\le C\lVert P\rVert_{\infty}h\left(p_{\theta,T},p_{\theta_\star,T}\right)\qquad\text{(Hellinger contraction under marginalization)} \\
&\le Ch\left(p_{\theta,T},p_{\theta_\star,T}\right)\qquad\text{(the supremum bound in {xref}`(C.2`).}
\end{aligned} i ∑ q t ( i ) P ( μ i ) = ∫ P d ( G θ { t } − G ⋆ { t } ) ≤ C ∥ P ∥ ∞ h ( p θ { t } , p θ ⋆ { t } ) ( xref ‘Lemma A.3‘) ≤ C ∥ P ∥ ∞ h ( p θ , T , p θ ⋆ , T ) (Hellinger contraction under marginalization) ≤ C h ( p θ , T , p θ ⋆ , T ) (the supremum bound in xref ‘(C.2‘).
The span has dimension at most K + k K+k K + k , and the polynomial has degree 2 K 2K 2 K . Thus the constant supplied by the moment-extraction lemma is independent of emission separation and the ambient dimension. The fixed factor ( 2 R ) 2 K (2R)^{2K} ( 2 R ) 2 K is absorbed into C C C .
Recovering the quadratic displacement bound. For each fitted center, the pointwise bound in (C.2) gives
P ( μ i ) ≥ c p o s Δ s e p 2 s max − 2 r i 2 , r i = ∥ μ i − a c ( μ i ) ∥ . P(\mu_i)\ge c_{\mathrm{pos}}\Delta_{\mathrm{sep}}^{2s_{\max}-2}r_i^2,\qquad r_i=\lVert\mu_i-a_{c(\mu_i)}\rVert. P ( μ i ) ≥ c pos Δ sep 2 s m a x − 2 r i 2 , r i = ∥ μ i − a c ( μ i ) ∥ .
Multiplying by q t ( i ) ≥ 0 q_t(i)\ge0 q t ( i ) ≥ 0 and summing over fitted states yields
c p o s Δ s e p 2 s max − 2 ∑ i q t ( i ) r i 2 ≤ ∑ i q t ( i ) P ( μ i ) ≤ C h ( p θ , T , p θ ⋆ , T ) . c_{\mathrm{pos}}\Delta_{\mathrm{sep}}^{2s_{\max}-2}\sum_i q_t(i)r_i^2\le\sum_i q_t(i)P(\mu_i)\le Ch\left(p_{\theta,T},p_{\theta_\star,T}\right). c pos Δ sep 2 s m a x − 2 i ∑ q t ( i ) r i 2 ≤ i ∑ q t ( i ) P ( μ i ) ≤ C h ( p θ , T , p θ ⋆ , T ) .
Dividing by c p o s Δ s e p 2 s max − 2 c_{\mathrm{pos}}\Delta_{\mathrm{sep}}^{2s_{\max}-2} c pos Δ sep 2 s m a x − 2 proves the second inequality in (C.3). No fitted probability is divided out, so the argument also applies to zero fitted probabilities and repeated fitted centers.
Flat mass selectors and directional mean selectors
Lemma C.1 bounds the probability-weighted squared distances of fitted centers from the true support. We next construct the directional polynomial H j , u H_{j,u} H j , u to approximate 1 { c ( x ) = j } ⟨ u , x − a j ⟩ \mathbf{1}\{c(x)=j\}\langle u,x-a_j\rangle 1 { c ( x ) = j } ⟨ u , x − a j ⟩ . Its expectation will recover the displacement sum within each assigned group. We also construct the flat polynomial E j E_j E j , whose values and local second-order behavior will be used in Lemma C.7.
The mass polynomial should equal one at a j a_j a j and zero at the other true centers. We additionally require its gradient to vanish at every true center, so that small location displacements contribute only a second-order error. The directional polynomial should vanish at every true center, with gradient u u u at a j a_j a j and zero at the others. Its first-order term then agrees with the directional displacement we want to recover.
For j ∈ [ K ] j\in[K] j ∈ [ K ] , consider
Q j ( x ) = ∏ l ≠ j ∥ x − a l ∥ 2 2 ∥ a j − a l ∥ 2 2 . (66) Q_j(x)=\prod_{l\ne j}\frac{\lVert x-a_l\rVert_2^2}{\lVert a_j-a_l\rVert_2^2}. \tag*{(66)} Q j ( x ) = l = j ∏ ∥ a j − a l ∥ 2 2 ∥ x − a l ∥ 2 2 . (66)
Each factor equals one at a j a_j a j , so Q j ( a j ) = 1 Q_j(a_j)=1 Q j ( a j ) = 1 . At any other true center a l a_l a l , the squared factor ∥ x − a l ∥ 2 \lVert x-a_l\rVert^2 ∥ x − a l ∥ 2 makes both the value and gradient vanish. The remaining gradient at a j a_j a j is
v j : = ∇ Q j ( a j ) = 2 ∑ l ≠ j a j − a l ∥ a j − a l ∥ 2 . (67) v_j := \nabla Q_j(a_j) = 2\sum_{l\ne j}\frac{a_j-a_l}{\lVert a_j-a_l\rVert^2}. \tag*{(67)} v j := ∇ Q j ( a j ) = 2 l = j ∑ ∥ a j − a l ∥ 2 a j − a l . (67)
The polynomial Q j Q_j Q j already has the required values at the true centers, but its gradient at a j a_j a j is v j v_j v j , which need not be zero. Multiplication by 1 − ⟨ v j , x − a j ⟩ 1-\langle v_j,x-a_j\rangle 1 − ⟨ v j , x − a j ⟩ cancels this gradient without changing those values. Multiplication by ⟨ u , x − a j ⟩ \langle u,x-a_j\rangle ⟨ u , x − a j ⟩ instead gives a polynomial with value zero and gradient u u u at a j a_j a j . We therefore define
E j ( x ) = Q j ( x ) { 1 − ⟨ v j , x − a j ⟩ } , H j , u ( x ) = Q j ( x ) ⟨ u , x − a j ⟩ , ∥ u ∥ ≤ 1. \begin{aligned}
E_j(x)&=Q_j(x)\{1-\langle v_j,x-a_j\rangle\},\\
H_{j,u}(x)&=Q_j(x)\langle u,x-a_j\rangle,\qquad\lVert u\rVert\le1.
\end{aligned} E j ( x ) H j , u ( x ) = Q j ( x ) { 1 − ⟨ v j , x − a j ⟩} , = Q j ( x ) ⟨ u , x − a j ⟩ , ∥ u ∥ ≤ 1.
The following lemma verifies the interpolation values and gradients, bounds the supremum norms of Q j Q_j Q j and H j , u H_{j,u} H j , u , and controls the directional approximation error by P ( x ) P(x) P ( x ) . Its proof also establishes the local Hessian and Taylor bounds for E j E_j E j needed later.
Lemma C.2 (Values, norms, and pointwise leakage). We have
E j ( a l ) = 1 { j = l } , ∇ E j ( a l ) = 0 , (68) E_j(a_l)=1\{j=l\},\qquad\nabla E_j(a_l)=0, \tag*{(68)} E j ( a l ) = 1 { j = l } , ∇ E j ( a l ) = 0 , (68) H j , u ( a l ) = 0 , ∇ H j , u ( a l ) = 1 { j = l } u . (69) H_{j,u}(a_l)=0,\qquad\nabla H_{j,u}(a_l)=1\{j=l\}u. \tag*{(69)} H j , u ( a l ) = 0 , ∇ H j , u ( a l ) = 1 { j = l } u . (69) Their degrees are at most 2 K − 1 2K-1 2 K − 1 , and
∥ Q j ∥ ∞ + ∥ H j , u ∥ ∞ ≤ C Δ s e p − ( 2 s max − 2 ) . (70) \lVert Q_j\rVert_\infty+\lVert H_{j,u}\rVert_\infty\le C\Delta_{\mathrm{sep}}^{-(2s_{\max}-2)}. \tag*{(70)} ∥ Q j ∥ ∞ + ∥ H j , u ∥ ∞ ≤ C Δ sep − ( 2 s m a x − 2 ) . (70) For every x ∈ B d ( 0 , R ) x\in B_d(0,R) x ∈ B d ( 0 , R ) ,
∣ H j , u ( x ) − 1 { c ( x ) = j } ⟨ u , x − a j ⟩ ∣ ≤ C Δ s e p − ( 2 s max − 1 ) P ( x ) . (71) \left|H_{j,u}(x)-1\{c(x)=j\}\langle u,x-a_j\rangle\right|
\le C\Delta_{\mathrm{sep}}^{-(2s_{\max}-1)}P(x). \tag*{(71)} ∣ H j , u ( x ) − 1 { c ( x ) = j } ⟨ u , x − a j ⟩ ∣ ≤ C Δ sep − ( 2 s m a x − 1 ) P ( x ) . (71)
Proof. We first verify the interpolation values and gradients and bound the supremum norms of Q j Q_j Q j and H j , u H_{j,u} H j , u . We then prove the directional approximation bound by treating points away from a j a_j a j through direct product comparisons and points near a j a_j a j through Taylor expansion. The local derivative calculation also provides the second-order bound for E j E_j E j used in Lemma C.7 .
Values and gradients at the true centers. The definition gives Q j ( a j ) = 1 Q_j(a_j)=1 Q j ( a j ) = 1 . Differentiating the product at a j a_j a j yields
∇ Q j ( a j ) = ∑ l ≠ j 2 ( a j − a l ) ∥ a j − a l ∥ 2 ∏ m ≠ j m ≠ l ∥ a j − a m ∥ 2 ∥ a j − a m ∥ 2 = 2 ∑ l ≠ j a j − a l ∥ a j − a l ∥ 2 = v j . \begin{aligned}
\nabla Q_j(a_j)=\sum_{l\ne j}\frac{2(a_j-a_l)}{\lVert a_j-a_l\rVert^2}\prod_{\substack{m\ne j\\m\ne l}}\frac{\lVert a_j-a_m\rVert^2}{\lVert a_j-a_m\rVert^2} \\
&=2\sum_{l\ne j}\frac{a_j-a_l}{\lVert a_j-a_l\rVert^2}=v_j.
\end{aligned} ∇ Q j ( a j ) = l = j ∑ ∥ a j − a l ∥ 2 2 ( a j − a l ) m = j m = l ∏ ∥ a j − a m ∥ 2 ∥ a j − a m ∥ 2 = 2 l = j ∑ ∥ a j − a l ∥ 2 a j − a l = v j .
For l ≠ j l\ne j l = j , we can instead isolate the factor that vanishes at a l a_l a l :
Q j ( x ) = ∥ x − a l ∥ 2 ∥ a j − a l ∥ 2 ∏ m ≠ j m ≠ l ∥ x − a m ∥ 2 ∥ a j − a m ∥ 2 . Q_j(x)=\frac{\lVert x-a_l\rVert^2}{\lVert a_j-a_l\rVert^2}
\prod_{\substack{m\ne j\\m\ne l}}\frac{\lVert x-a_m\rVert^2}{\lVert a_j-a_m\rVert^2}. Q j ( x ) = ∥ a j − a l ∥ 2 ∥ x − a l ∥ 2 m = j m = l ∏ ∥ a j − a m ∥ 2 ∥ x − a m ∥ 2 .
Both ∥ x − a l ∥ 2 \lVert x-a_l\rVert^2 ∥ x − a l ∥ 2 and its gradient vanish at a l a_l a l . The product rule therefore gives Q j ( a l ) = 0 Q_j(a_l)=0 Q j ( a l ) = 0 and ∇ Q j ( a l ) = 0 \nabla Q_j(a_l)=0 ∇ Q j ( a l ) = 0 .
Differentiating the selectors, we obtain
∇ E j ( x ) = { 1 − ⟨ v j , x − a j ⟩ } ∇ Q j ( x ) − Q j ( x ) v j , ∇ H j , u ( x ) = ⟨ u , x − a j ⟩ ∇ Q j ( x ) + Q j ( x ) u . \nabla E_j(x)=\{1-\langle v_j,x-a_j\rangle\}\nabla Q_j(x)-Q_j(x)v_j,
\\
\nabla H_{j,u}(x)=\langle u,x-a_j\rangle\nabla Q_j(x)+Q_j(x)u. ∇ E j ( x ) = { 1 − ⟨ v j , x − a j ⟩} ∇ Q j ( x ) − Q j ( x ) v j , ∇ H j , u ( x ) = ⟨ u , x − a j ⟩ ∇ Q j ( x ) + Q j ( x ) u .
At a j a_j a j , these identities give
E j ( a j ) = 1 , ∇ E j ( a j ) = v j − v j = 0 , H j , u ( a j ) = 0 , ∇ H j , u ( a j ) = u . E_j(a_j)=1,\qquad\nabla E_j(a_j)=v_j-v_j=0,
\\
H_{j,u}(a_j)=0,\qquad\nabla H_{j,u}(a_j)=u. E j ( a j ) = 1 , ∇ E j ( a j ) = v j − v j = 0 , H j , u ( a j ) = 0 , ∇ H j , u ( a j ) = u .
At every other true center, both values and gradients vanish because Q j ( a l ) = 0 Q_j(a_l)=0 Q j ( a l ) = 0 and ∇ Q j ( a l ) = 0 \nabla Q_j(a_l)=0 ∇ Q j ( a l ) = 0 . This proves (C.6) and (C.7). The degree of Q j Q_j Q j is 2 K − 2 2K-2 2 K − 2 , so multiplication by either linear factor gives degree at most 2 K − 1 2K-1 2 K − 1 .
Supremum bounds for moment extraction. The product-distance estimate in the proof of Lemma C.1 gives
∏ l ≠ j ∥ a j − a l ∥ 2 ≥ c Δ s e p 2 s max − 2 . \prod_{l\ne j}\lVert a_j-a_l\rVert^2\ge c\Delta_{\mathrm{sep}}^{2s_{\max}-2}. l = j ∏ ∥ a j − a l ∥ 2 ≥ c Δ sep 2 s m a x − 2 .
Without cluster assumptions, the same estimate holds with s max = K s_{\max}=K s m a x = K .
For x ∈ B d ( 0 , R ) x\in B_d(0,R) x ∈ B d ( 0 , R ) , every numerator distance is at most 2 R 2R 2 R . Therefore,
∣ Q j ( x ) ∣ = ∏ l ≠ j ∥ x − a l ∥ 2 ∏ l ≠ j ∥ a j − a l ∥ 2 ≤ ( 2 R ) 2 K − 2 ∏ l ≠ j ∥ a j − a l ∥ 2 ≤ C Δ s e p − ( 2 s max − 2 ) . \lvert Q_j(x)\rvert
=
\frac{\prod_{l\ne j}\lVert x-a_l\rVert^2}
{\prod_{l\ne j}\lVert a_j-a_l\rVert^2}
\le
\frac{(2R)^{2K-2}}
{\prod_{l\ne j}\lVert a_j-a_l\rVert^2}
\le
C\Delta_{\mathrm{sep}}^{-(2s_{\max}-2)}. ∣ Q j ( x )∣ = ∏ l = j ∥ a j − a l ∥ 2 ∏ l = j ∥ x − a l ∥ 2 ≤ ∏ l = j ∥ a j − a l ∥ 2 ( 2 R ) 2 K − 2 ≤ C Δ sep − ( 2 s m a x − 2 ) .
The directional selector is obtained by multiplying Q j Q_j Q j by ⟨ u , x − a j ⟩ \langle u,x-a_j\rangle ⟨ u , x − a j ⟩ . Since ∥ u ∥ ≤ 1 \lVert u\rVert\le1 ∥ u ∥ ≤ 1 and both x x x and a j a_j a j lie in B d ( 0 , R ) B_d(0,R) B d ( 0 , R ) ,
∣ ⟨ u , x − a j ⟩ ∣ ≤ ∥ u ∥ ∥ x − a j ∥ ≤ 2 R . \lvert\langle u,x-a_j\rangle\rvert
\le\lVert u\rVert\lVert x-a_j\rVert
\le2R. ∣⟨ u , x − a j ⟩∣ ≤ ∥ u ∥ ∥ x − a j ∥ ≤ 2 R .
Consequently,
∣ H j , u ( x ) ∣ = ∣ Q j ( x ) ∣ ∣ ⟨ u , x − a j ⟩ ∣ ≤ 2 R ∣ Q j ( x ) ∣ ≤ C Δ s e p − ( 2 s max − 2 ) . \lvert H_{j,u}(x)\rvert
=
\lvert Q_j(x)\rvert\lvert\langle u,x-a_j\rangle\rvert
\le2R\lvert Q_j(x)\rvert
\le C\Delta_{\mathrm{sep}}^{-(2s_{\max}-2)}. ∣ H j , u ( x )∣ = ∣ Q j ( x )∣ ∣⟨ u , x − a j ⟩∣ ≤ 2 R ∣ Q j ( x )∣ ≤ C Δ sep − ( 2 s m a x − 2 ) .
Taking suprema and absorbing the fixed factors into C C C proves (C.8).
We also record a bound for v j v_j v j , which will enter the local Hessian calculation for E j E_j E j . Its definition and the separation of the true centers give
∥ v j ∥ = ∥ 2 ∑ l ≠ j a j − a l ∥ a j − a l ∥ 2 ∥ ≤ 2 ∑ l ≠ j 1 ∥ a j − a l ∥ ≤ 2 ( K − 1 ) Δ s e p − 1 . \lVert v_j\rVert
=
\left\lVert
2\sum_{l\ne j}\frac{a_j-a_l}{\lVert a_j-a_l\rVert^2}
\right\rVert
\le
2\sum_{l\ne j}\frac{1}{\lVert a_j-a_l\rVert}
\le
2(K-1)\Delta_{\mathrm{sep}}^{-1}. ∥ v j ∥ = 2 l = j ∑ ∥ a j − a l ∥ 2 a j − a l ≤ 2 l = j ∑ ∥ a j − a l ∥ 1 ≤ 2 ( K − 1 ) Δ sep − 1 .
Comparison with the positive witness away from the selected center. We next prove (C.9) away from a j a_j a j . If P ( x ) = 0 P(x)=0 P ( x ) = 0 , then x x x is a true center, and the interpolation values make the directional approximation error zero. We may therefore assume that P ( x ) > 0 P(x)>0 P ( x ) > 0 .
Write r = ∥ x − a j ∥ r=\lVert x-a_j\rVert r = ∥ x − a j ∥ . The product Q j Q_j Q j contains every numerator factor of P P P except ∥ x − a j ∥ 2 \lVert x-a_j\rVert^2 ∥ x − a j ∥ 2 , so
Q j ( x ) P ( x ) = 1 r 2 ∏ ℓ ≠ j ∥ a j − a ℓ ∥ 2 ≤ C Δ s e p − ( 2 s max − 2 ) r − 2 . \frac{Q_j(x)}{P(x)}=\frac{1}{r^2\prod_{\ell\ne j}\lVert a_j-a_\ell\rVert^2}
\le C\Delta_{\mathrm{sep}}^{-(2s_{\max}-2)}r^{-2}. P ( x ) Q j ( x ) = r 2 ∏ ℓ = j ∥ a j − a ℓ ∥ 2 1 ≤ C Δ sep − ( 2 s m a x − 2 ) r − 2 .
Whenever r > Δ s e p / 4 r>\Delta_{\mathrm{sep}}/4 r > Δ sep /4 , this gives
∣ H j , u ( x ) ∣ P ( x ) = Q j ( x ) ∣ ⟨ u , x − a j ⟩ ∣ P ( x ) ≤ r Q j ( x ) P ( x ) (because ∥ u ∥ ≤ 1 ) ≤ C Δ s e p − ( 2 s max − 2 ) r − 1 ≤ C Δ s e p − ( 2 s max − 1 ) (because r > Δ s e p / 4 ). (72) \begin{aligned}
\frac{\lvert H_{j,u}(x)\rvert}{P(x)}
&=\frac{Q_j(x)\lvert\langle u,x-a_j\rangle\rvert}{P(x)}\\
&\le r\frac{Q_j(x)}{P(x)} \qquad\text{(because \(\lVert u\rVert\le1\))}\\
&\le C\Delta_{\mathrm{sep}}^{-(2s_{\max}-2)}r^{-1}\\
&\le C\Delta_{\mathrm{sep}}^{-(2s_{\max}-1)}
\qquad\text{(because \(r>\Delta_{\mathrm{sep}}/4\)).} \tag*{(72)}
\end{aligned} P ( x ) ∣ H j , u ( x )∣ = P ( x ) Q j ( x ) ∣⟨ u , x − a j ⟩∣ ≤ r P ( x ) Q j ( x ) (because ∥ u ∥ ≤ 1) ≤ C Δ sep − ( 2 s m a x − 2 ) r − 1 ≤ C Δ sep − ( 2 s m a x − 1 ) (because r > Δ sep /4). (72)
Suppose first that c ( x ) ≠ j c(x)\ne j c ( x ) = j . The assigned center is distinct from a j a_j a j , and nearest-center assignment gives
Δ s e p ≤ ∥ a j − a c ( x ) ∥ ≤ ∥ a j − x ∥ + ∥ x − a c ( x ) ∥ (by the triangle inequality) ≤ 2 r (because ∥ x − a c ( x ) ∥ ≤ ∥ x − a j ∥ = r ). \begin{aligned}
\Delta_{\mathrm{sep}}&\le\lVert a_j-a_{c(x)}\rVert\\
&\le\lVert a_j-x\rVert+\lVert x-a_{c(x)}\rVert
\qquad\text{(by the triangle inequality)}\\
&\le2r
\qquad\text{(because \(\lVert x-a_{c(x)}\rVert\le\lVert x-a_j\rVert=r\)).}
\end{aligned} Δ sep ≤ ∥ a j − a c ( x ) ∥ ≤ ∥ a j − x ∥ + ∥ x − a c ( x ) ∥ (by the triangle inequality) ≤ 2 r (because ∥ x − a c ( x ) ∥ ≤ ∥ x − a j ∥ = r ).
Thus r ≥ Δ s e p / 2 r\ge\Delta_{\mathrm{sep}}/2 r ≥ Δ sep /2 , so (C.10) applies. The target 1 { c ( x ) = j } ⟨ u , x − a j ⟩ \mathbf{1}\{c(x)=j\}\langle u,x-a_j\rangle 1 { c ( x ) = j } ⟨ u , x − a j ⟩ is zero in this case. Hence
∣ H j , u ( x ) − 1 { c ( x ) = j } ⟨ u , x − a j ⟩ ∣ = ∣ H j , u ( x ) ∣ ≤ C Δ s e p − ( 2 s max − 1 ) P ( x ) . \lvert H_{j,u}(x)-\mathbf{1}\{c(x)=j\}\langle u,x-a_j\rangle\rvert
=\lvert H_{j,u}(x)\rvert
\le C\Delta_{\mathrm{sep}}^{-(2s_{\max}-1)}P(x). ∣ H j , u ( x ) − 1 { c ( x ) = j } ⟨ u , x − a j ⟩∣ = ∣ H j , u ( x )∣ ≤ C Δ sep − ( 2 s m a x − 1 ) P ( x ) .
Suppose instead that c ( x ) = j c(x)=j c ( x ) = j and r > Δ s e p / 4 r>\Delta_{\mathrm{sep}}/4 r > Δ sep /4 . We must now bound the target term as well as the polynomial. Since a j a_j a j is the assigned nearest center, Lemma C.1 gives
P ( x ) ≥ c Δ s e p 2 s max − 2 r 2 . P(x)\ge c\Delta_{\mathrm{sep}}^{2s_{\max}-2}r^2. P ( x ) ≥ c Δ sep 2 s m a x − 2 r 2 .
Therefore,
∣ ⟨ u , x − a j ⟩ ∣ P ( x ) ≤ r P ( x ) (because ∥ u ∥ ≤ 1 ) ≤ C Δ s e p − ( 2 s max − 2 ) r − 1 ≤ C Δ s e p − ( 2 s max − 1 ) (because r > Δ s e p / 4 ). \begin{aligned}
\frac{\lvert\langle u,x-a_j\rangle\rvert}{P(x)}
&\le\frac{r}{P(x)}
\qquad\text{(because \(\lVert u\rVert\le1\))}\\
&\le C\Delta_{\mathrm{sep}}^{-(2s_{\max}-2)}r^{-1}\\
&\le C\Delta_{\mathrm{sep}}^{-(2s_{\max}-1)}
\qquad\text{(because \(r>\Delta_{\mathrm{sep}}/4\)).}
\end{aligned} P ( x ) ∣⟨ u , x − a j ⟩∣ ≤ P ( x ) r (because ∥ u ∥ ≤ 1) ≤ C Δ sep − ( 2 s m a x − 2 ) r − 1 ≤ C Δ sep − ( 2 s m a x − 1 ) (because r > Δ sep /4).
We can now bound the approximation error by adding the polynomial and target contributions:
∣ H j , u ( x ) − ⟨ u , x − a j ⟩ ∣ P ( x ) ≤ ∣ H j , u ( x ) ∣ P ( x ) + ∣ ⟨ u , x − a j ⟩ ∣ P ( x ) ≤ C Δ s e p − ( 2 s max − 1 ) + C Δ s e p − ( 2 s max − 1 ) ≤ C Δ s e p − ( 2 s max − 1 ) . \begin{aligned}
\frac{\lvert H_{j,u}(x)-\langle u,x-a_j\rangle\rvert}{P(x)}
&\le\frac{\lvert H_{j,u}(x)\rvert}{P(x)}
+\frac{\lvert\langle u,x-a_j\rangle\rvert}{P(x)}\\
&\le C\Delta_{\mathrm{sep}}^{-(2s_{\max}-1)}
+C\Delta_{\mathrm{sep}}^{-(2s_{\max}-1)}\\
&\le C\Delta_{\mathrm{sep}}^{-(2s_{\max}-1)}.
\end{aligned} P ( x ) ∣ H j , u ( x ) − ⟨ u , x − a j ⟩∣ ≤ P ( x ) ∣ H j , u ( x )∣ + P ( x ) ∣⟨ u , x − a j ⟩∣ ≤ C Δ sep − ( 2 s m a x − 1 ) + C Δ sep − ( 2 s m a x − 1 ) ≤ C Δ sep − ( 2 s m a x − 1 ) .
The first contribution is bounded by (C.10), and the second by the preceding calculation. This proves (C.9) in the second case. It remains to treat c ( x ) = j c(x)=j c ( x ) = j and r ≤ Δ s e p / 4 r\le\Delta_{\mathrm{sep}}/4 r ≤ Δ sep /4 .
Derivative bounds near the selected center. Suppose c ( x ) = j c(x)=j c ( x ) = j and r = ∥ x − a j ∥ ≤ Δ s e p / 4 r=\lVert x-a_j\rVert\leq\Delta_{\mathrm{sep}}/4 r = ∥ x − a j ∥ ≤ Δ sep /4 . Directly bounding the selector and target terms separately would lose their cancellation at a j a_j a j . We therefore use their matching values and gradients and bound the second-order remainders. This requires Hessian bounds along the segment joining a j a_j a j to x x x .
Let y = a j + t ( x − a j ) y=a_j+t(x-a_j) y = a j + t ( x − a j ) , where 0 ≤ t ≤ 1 0\leq t\leq1 0 ≤ t ≤ 1 . For every l ≠ j l\neq j l = j ,
∥ y − a j ∥ ≤ Δ s e p 4 ≤ ∥ a j − a l ∥ 4 . \lVert y-a_j\rVert\leq\frac{\Delta_{\mathrm{sep}}}{4}\leq\frac{\lVert a_j-a_l\rVert}{4}. ∥ y − a j ∥ ≤ 4 Δ sep ≤ 4 ∥ a j − a l ∥ .
The triangle and reverse triangle inequalities imply
3 4 ∥ a j − a l ∥ ≤ ∥ y − a l ∥ ≤ 5 4 ∥ a j − a l ∥ . \frac{3}{4}\lVert a_j-a_l\rVert\leq\lVert y-a_l\rVert\leq\frac{5}{4}\lVert a_j-a_l\rVert. 4 3 ∥ a j − a l ∥ ≤ ∥ y − a l ∥ ≤ 4 5 ∥ a j − a l ∥ .
Substituting into the product defining Q j Q_j Q j gives
( 3 / 4 ) 2 ( K − 1 ) ≤ Q j ( y ) ≤ ( 5 / 4 ) 2 ( K − 1 ) . (3/4)^{2(K-1)}\leq Q_j(y)\leq(5/4)^{2(K-1)}. ( 3/4 ) 2 ( K − 1 ) ≤ Q j ( y ) ≤ ( 5/4 ) 2 ( K − 1 ) .
All factors are positive on this segment. Differentiating log Q j \log Q_j log Q j expresses the product derivatives through sums:
g j ( y ) : = ∇ log Q j ( y ) = 2 ∑ l ≠ j y − a l ∥ y − a l ∥ 2 g_j(y):=\nabla\log Q_j(y)=2\sum_{l\neq j}\frac{y-a_l}{\lVert y-a_l\rVert^2} g j ( y ) := ∇ log Q j ( y ) = 2 l = j ∑ ∥ y − a l ∥ 2 y − a l
B j ( y ) : = ∇ 2 log Q j ( y ) = ∑ l ≠ j { 2 I d ∥ y − a l ∥ 2 − 4 ( y − a l ) ( y − a l ) T ∥ y − a l ∥ 4 } . B_j(y):=\nabla^2\log Q_j(y)=\sum_{l\neq j}\left\{\frac{2I_d}{\lVert y-a_l\rVert^2}-\frac{4(y-a_l)(y-a_l)^{\mathsf T}}{\lVert y-a_l\rVert^4}\right\}. B j ( y ) := ∇ 2 log Q j ( y ) = l = j ∑ { ∥ y − a l ∥ 2 2 I d − ∥ y − a l ∥ 4 4 ( y − a l ) ( y − a l ) T } .
Since ∥ y − a l ∥ ≥ 3 Δ s e p / 4 \lVert y-a_l\rVert\geq3\Delta_{\mathrm{sep}}/4 ∥ y − a l ∥ ≥ 3 Δ sep /4 , we obtain
∥ g j ( y ) ∥ ≤ 2 ∑ l ≠ j 1 ∥ y − a l ∥ ≤ 8 ( K − 1 ) 3 Δ s e p . \lVert g_j(y)\rVert\leq2\sum_{l\neq j}\frac{1}{\lVert y-a_l\rVert}\leq\frac{8(K-1)}{3\Delta_{\mathrm{sep}}}. ∥ g j ( y )∥ ≤ 2 l = j ∑ ∥ y − a l ∥ 1 ≤ 3 Δ sep 8 ( K − 1 ) .
For a matrix M M M , we write ∥ M ∥ o p = sup ∥ v ∥ = 1 ∥ M v ∥ \lVert M\rVert_{\mathrm{op}}=\sup_{\lVert v\rVert=1}\lVert Mv\rVert ∥ M ∥ op = sup ∥ v ∥ = 1 ∥ M v ∥ for its operator norm, with Euclidean norms on the vectors. Using ∥ I d ∥ o p = 1 \lVert I_d\rVert_{\mathrm{op}}=1 ∥ I d ∥ op = 1 and ∥ z z T ∥ o p = ∥ z ∥ 2 \lVert zz^{\mathsf T}\rVert_{\mathrm{op}}=\lVert z\rVert^2 ∥ z z T ∥ op = ∥ z ∥ 2 also gives
∥ B j ( y ) ∥ o p ≤ 6 ∑ l ≠ j 1 ∥ y − a l ∥ 2 ≤ 96 ( K − 1 ) 9 Δ s e p 2 . \lVert B_j(y)\rVert_{\mathrm{op}}\leq6\sum_{l\neq j}\frac{1}{\lVert y-a_l\rVert^2}\leq\frac{96(K-1)}{9\Delta_{\mathrm{sep}}^2}. ∥ B j ( y ) ∥ op ≤ 6 l = j ∑ ∥ y − a l ∥ 2 1 ≤ 9 Δ sep 2 96 ( K − 1 ) .
The identities
∇ Q j = Q j g j , ∇ 2 Q j = Q j ( B j + g j g j T ) \nabla Q_j=Q_jg_j,\qquad\nabla^2Q_j=Q_j(B_j+g_jg_j^{\mathsf T}) ∇ Q j = Q j g j , ∇ 2 Q j = Q j ( B j + g j g j T )
therefore imply
∥ ∇ Q j ( y ) ∥ ≤ C Δ s e p − 1 , ∥ ∇ 2 Q j ( y ) ∥ o p ≤ C Δ s e p − 2 . \lVert\nabla Q_j(y)\rVert\leq C\Delta_{\mathrm{sep}}^{-1},\qquad\lVert\nabla^2Q_j(y)\rVert_{\mathrm{op}}\leq C\Delta_{\mathrm{sep}}^{-2}. ∥ ∇ Q j ( y )∥ ≤ C Δ sep − 1 , ∥ ∇ 2 Q j ( y ) ∥ op ≤ C Δ sep − 2 .
The operator-norm estimates introduce no factor depending on the ambient dimension.
We now apply these estimates to the selectors. On the same segment,
∣ 1 − ⟨ v j , y − a j ⟩ ∣ ≤ 1 + ∥ v j ∥ ∥ y − a j ∥ ≤ 1 + K − 1 2 . \left|1-\langle v_j,y-a_j\rangle\right|\leq1+\lVert v_j\rVert\lVert y-a_j\rVert\leq1+\frac{K-1}{2}. ∣ 1 − ⟨ v j , y − a j ⟩ ∣ ≤ 1 + ∥ v j ∥ ∥ y − a j ∥ ≤ 1 + 2 K − 1 .
Differentiating twice gives
∇ 2 E j ( y ) = { 1 − ⟨ v j , y − a j ⟩ } ∇ 2 Q j ( y ) − ∇ Q j ( y ) v j T − v j ∇ Q j ( y ) T , \nabla^2E_j(y)=\{1-\langle v_j,y-a_j\rangle\}\nabla^2Q_j(y)-\nabla Q_j(y)v_j^{\mathsf T}-v_j\nabla Q_j(y)^{\mathsf T}, ∇ 2 E j ( y ) = { 1 − ⟨ v j , y − a j ⟩} ∇ 2 Q j ( y ) − ∇ Q j ( y ) v j T − v j ∇ Q j ( y ) T ,
and
∇ 2 H j , u ( y ) = ⟨ u , y − a j ⟩ ∇ 2 Q j ( y ) + ∇ Q j ( y ) u ⊤ + u ∇ Q j ( y ) ⊤ . \nabla^2 H_{j,u}(y)=\langle u,y-a_j\rangle\nabla^2 Q_j(y)+\nabla Q_j(y)u^{\top}+u\nabla Q_j(y)^{\top}. ∇ 2 H j , u ( y ) = ⟨ u , y − a j ⟩ ∇ 2 Q j ( y ) + ∇ Q j ( y ) u ⊤ + u ∇ Q j ( y ) ⊤ .
Thus, for every y ∈ [ a j , x ] y\in[a_j,x] y ∈ [ a j , x ] ,
∥ ∇ 2 E j ( y ) ∥ o p ≤ ∣ 1 − ⟨ v j , y − a j ⟩ ∣ ∥ ∇ 2 Q j ( y ) ∥ o p + 2 ∥ ∇ Q j ( y ) ∥ ∥ v j ∥ ≤ C Δ s e p − 2 . (73) \left\|\nabla^2 E_j(y)\right\|_{\mathrm{op}}\leq\left|1-\langle v_j,y-a_j\rangle\right|\left\|\nabla^2 Q_j(y)\right\|_{\mathrm{op}}+2\left\|\nabla Q_j(y)\right\|\left\|v_j\right\|
\leq C\Delta_{\mathrm{sep}}^{-2}. \tag*{(73)} ∇ 2 E j ( y ) op ≤ ∣ 1 − ⟨ v j , y − a j ⟩ ∣ ∇ 2 Q j ( y ) op + 2 ∥ ∇ Q j ( y ) ∥ ∥ v j ∥ ≤ C Δ sep − 2 . (73)
For the directional selector, ∥ u ∥ ≤ 1 \left\|u\right\|\leq1 ∥ u ∥ ≤ 1 and ∥ y − a j ∥ ≤ Δ s e p / 4 \left\|y-a_j\right\|\leq\Delta_{\mathrm{sep}}/4 ∥ y − a j ∥ ≤ Δ sep /4 imply ∣ ⟨ u , y − a j ⟩ ∣ ≤ Δ s e p / 4 \left|\langle u,y-a_j\rangle\right|\leq\Delta_{\mathrm{sep}}/4 ∣ ⟨ u , y − a j ⟩ ∣ ≤ Δ sep /4 . Consequently,
∥ ∇ 2 H j , u ( y ) ∥ o p ≤ ∣ ⟨ u , y − a j ⟩ ∣ ∥ ∇ 2 Q j ( y ) ∥ o p + 2 ∥ ∇ Q j ( y ) ∥ ∥ u ∥ ≤ Δ s e p 4 C Δ s e p − 2 + 2 C Δ s e p − 1 ≤ C Δ s e p − 1 . (74) \left\|\nabla^2 H_{j,u}(y)\right\|_{\mathrm{op}}\leq\left|\langle u,y-a_j\rangle\right|\left\|\nabla^2 Q_j(y)\right\|_{\mathrm{op}}+2\left\|\nabla Q_j(y)\right\|\left\|u\right\|
\leq\frac{\Delta_{\mathrm{sep}}}{4}C\Delta_{\mathrm{sep}}^{-2}+2C\Delta_{\mathrm{sep}}^{-1}
\leq C\Delta_{\mathrm{sep}}^{-1}. \tag*{(74)} ∇ 2 H j , u ( y ) op ≤ ∣ ⟨ u , y − a j ⟩ ∣ ∇ 2 Q j ( y ) op + 2 ∥ ∇ Q j ( y ) ∥ ∥ u ∥ ≤ 4 Δ sep C Δ sep − 2 + 2 C Δ sep − 1 ≤ C Δ sep − 1 . (74)
These Hessian bounds hold uniformly along [ a j , x ] [a_j,x] [ a j , x ] . Taylor’s formula gives local quadratic error bounds for both selectors. We then use the lower bound for P ( x ) P(x) P ( x ) to convert the directional error into (C.9).
From Taylor remainders to the positive witness. The Hessian bounds control the approximation errors because the constant and linear terms already agree at a j a_j a j . Set h = x − a j h=x-a_j h = x − a j , so that ∥ h ∥ = r \left\|h\right\|=r ∥ h ∥ = r . For a twice continuously differentiable function F F F , put f ( t ) = F ( a j + t h ) f(t)=F(a_j+th) f ( t ) = F ( a j + t h ) . Applying the fundamental theorem of calculus twice gives
f ( 1 ) − f ( 0 ) − f ′ ( 0 ) = ∫ 0 1 { f ′ ( t ) − f ′ ( 0 ) } d t = ∫ 0 1 ∫ 0 t f ′ ′ ( z ) d z d t . f(1)-f(0)-f'(0)=\int_0^1\{f'(t)-f'(0)\}\,dt=\int_0^1\int_0^t f''(z)\,dz\,dt. f ( 1 ) − f ( 0 ) − f ′ ( 0 ) = ∫ 0 1 { f ′ ( t ) − f ′ ( 0 )} d t = ∫ 0 1 ∫ 0 t f ′′ ( z ) d z d t .
The integration region is 0 ≤ z ≤ t ≤ 1 0\leq z\leq t\leq1 0 ≤ z ≤ t ≤ 1 . For each fixed z ∈ [ 0 , 1 ] z\in[0,1] z ∈ [ 0 , 1 ] , the variable t t t ranges from z z z to 1 1 1 . Reversing the order of integration therefore gives
∫ 0 1 ∫ 0 t f ′ ′ ( z ) d z d t = ∫ 0 1 ∫ z 1 f ′ ′ ( z ) d t d z = ∫ 0 1 f ′ ′ ( z ) ( ∫ z 1 d t ) d z (because f ′ ′ ( z ) is independent of t ) = ∫ 0 1 ( 1 − z ) f ′ ′ ( z ) d z . \begin{aligned}
\int_0^1\int_0^t f''(z)\,dz\,dt=\int_0^1\int_z^1 f''(z)\,dt\,dz \\
&=\int_0^1 f''(z)\left(\int_z^1 dt\right) dz\quad\text{(because } f''(z) \text{ is independent of } t\text{)} \\
&=\int_0^1(1-z)f''(z)\,dz.
\end{aligned} ∫ 0 1 ∫ 0 t f ′′ ( z ) d z d t = ∫ 0 1 ∫ z 1 f ′′ ( z ) d t d z = ∫ 0 1 f ′′ ( z ) ( ∫ z 1 d t ) d z (because f ′′ ( z ) is independent of t ) = ∫ 0 1 ( 1 − z ) f ′′ ( z ) d z .
We have therefore proved
f ( 1 ) = f ( 0 ) + f ′ ( 0 ) + ∫ 0 1 ( 1 − t ) f ′ ′ ( t ) d t , f(1)=f(0)+f'(0)+\int_0^1(1-t)f''(t)\,dt, f ( 1 ) = f ( 0 ) + f ′ ( 0 ) + ∫ 0 1 ( 1 − t ) f ′′ ( t ) d t ,
where the integration variable has been renamed from z z z to t t t . By the definition f ( t ) = F ( a j + t h ) f(t)=F(a_j+th) f ( t ) = F ( a j + t h ) and the chain rule,
f ( 1 ) = F ( a j + h ) , f(1)=F(a_j+h), f ( 1 ) = F ( a j + h ) ,
f ( 0 ) = F ( a j ) , f(0)=F(a_j), f ( 0 ) = F ( a j ) ,
f ′ ( 0 ) = ⟨ ∇ F ( a j ) , h ⟩ , f'(0)=\langle\nabla F(a_j),h\rangle, f ′ ( 0 ) = ⟨ ∇ F ( a j ) , h ⟩ ,
f ′ ′ ( t ) = h ⊤ ∇ 2 F ( a j + t h ) h . f''(t)=h^{\top}\nabla^2F(a_j+th)h. f ′′ ( t ) = h ⊤ ∇ 2 F ( a j + t h ) h .
Substituting these expressions into the preceding identity gives
F ( a j + h ) = F ( a j ) + ⟨ ∇ F ( a j ) , h ⟩ + ∫ 0 1 ( 1 − t ) h T ∇ 2 F ( a j + t h ) h d t . (75) F(a_j+h)=F(a_j)+\langle\nabla F(a_j),h\rangle+\int_0^1(1-t)h^{\mathsf T}\nabla^2F(a_j+th)h\,\mathrm{d}t. \tag*{(75)} F ( a j + h ) = F ( a j ) + ⟨ ∇ F ( a j ) , h ⟩ + ∫ 0 1 ( 1 − t ) h T ∇ 2 F ( a j + t h ) h d t . (75)
For E j E_j E j , Equation (C.6) gives E j ( a j ) = 1 E_j(a_j)=1 E j ( a j ) = 1 and ∇ E j ( a j ) = 0 \nabla E_j(a_j)=0 ∇ E j ( a j ) = 0 . Substituting these values into Equation (C.13) yields
E j ( x ) − 1 = ∫ 0 1 ( 1 − t ) h T ∇ 2 E j ( a j + t h ) h d t . E_j(x)-1=\int_0^1(1-t)h^{\mathsf T}\nabla^2E_j(a_j+th)h\,\mathrm{d}t. E j ( x ) − 1 = ∫ 0 1 ( 1 − t ) h T ∇ 2 E j ( a j + t h ) h d t .
To bound this remainder, recall that, for any matrix M M M ,
∣ h T M h ∣ = ∣ ⟨ h , M h ⟩ ∣ ≤ ∥ h ∥ ∥ M h ∥ (by Cauchy–Schwarz) ≤ ∥ h ∥ 2 ∥ M ∥ o p (by the definition of the operator norm) = r 2 ∥ M ∥ o p . \begin{aligned}
\lvert h^{\mathsf T}Mh\rvert&=\lvert\langle h,Mh\rangle\rvert\\
&\le\lVert h\rVert\lVert Mh\rVert&&\text{(by Cauchy--Schwarz)}\\
&\le\lVert h\rVert^2\lVert M\rVert_{\mathrm{op}} &&\text{(by the definition of the operator norm)}\\
&=r^2\lVert M\rVert_{\mathrm{op}}.
\end{aligned} ∣ h T M h ∣ = ∣⟨ h , M h ⟩∣ ≤ ∥ h ∥ ∥ M h ∥ ≤ ∥ h ∥ 2 ∥ M ∥ op = r 2 ∥ M ∥ op . (by Cauchy–Schwarz) (by the definition of the operator norm)
Taking absolute values in the remainder identity and applying this inequality gives
∣ E j ( x ) − 1 ∣ ≤ ∫ 0 1 ( 1 − t ) ∣ h T ∇ 2 E j ( a j + t h ) h ∣ d t ≤ r 2 ∫ 0 1 ( 1 − t ) ∥ ∇ 2 E j ( a j + t h ) ∥ o p d t ≤ C Δ s e p − 2 r 2 ∫ 0 1 ( 1 − t ) d t (by xref‘(C.11)‘) = C 2 Δ s e p − 2 r 2 ≤ C Δ s e p − 2 r 2 . (76) \begin{aligned}
\lvert E_j(x)-1\rvert
&\le\int_0^1(1-t)\lvert h^{\mathsf T}\nabla^2E_j(a_j+th)h\rvert\,\mathrm{d}t\\
&\le r^2\int_0^1(1-t)\lVert\nabla^2E_j(a_j+th)\rVert_{\mathrm{op}}\,\mathrm{d}t\\
&\le C\Delta_{\mathrm{sep}}^{-2}r^2\int_0^1(1-t)\,\mathrm{d}t &&\text{(by {xref}`(C.11)`)}\\
&=\frac{C}{2}\Delta_{\mathrm{sep}}^{-2}r^2\\
&\le C\Delta_{\mathrm{sep}}^{-2}r^2. \tag*{(76)}
\end{aligned} ∣ E j ( x ) − 1 ∣ ≤ ∫ 0 1 ( 1 − t ) ∣ h T ∇ 2 E j ( a j + t h ) h ∣ d t ≤ r 2 ∫ 0 1 ( 1 − t ) ∥ ∇ 2 E j ( a j + t h ) ∥ op d t ≤ C Δ sep − 2 r 2 ∫ 0 1 ( 1 − t ) d t = 2 C Δ sep − 2 r 2 ≤ C Δ sep − 2 r 2 . (by xref ‘(C.11)‘) (76)
Here Equation (C.11) applies at every point a j + t h a_j+th a j + t h , since 0 ≤ t ≤ 1 0\le t\le1 0 ≤ t ≤ 1 .
For H j , u H_{j,u} H j , u , Equation (C.7) instead gives H j , u ( a j ) = 0 H_{j,u}(a_j)=0 H j , u ( a j ) = 0 and ∇ H j , u ( a j ) = u \nabla H_{j,u}(a_j)=u ∇ H j , u ( a j ) = u . Equation (C.13) therefore becomes
H j , u ( x ) − ⟨ u , h ⟩ = ∫ 0 1 ( 1 − t ) h T ∇ 2 H j , u ( a j + t h ) h d t . H_{j,u}(x)-\langle u,h\rangle=\int_0^1(1-t)h^{\mathsf T}\nabla^2H_{j,u}(a_j+th)h\,\mathrm{d}t. H j , u ( x ) − ⟨ u , h ⟩ = ∫ 0 1 ( 1 − t ) h T ∇ 2 H j , u ( a j + t h ) h d t .
Using the same operator-norm inequality and the Hessian bound in Equation (C.12), we obtain
∣ H j , u ( x ) − ⟨ u , x − a j ⟩ ∣ ≤ ∫ 0 1 ( 1 − t ) ∣ h T ∇ 2 H j , u ( a j + t h ) h ∣ d t ≤ r 2 ∫ 0 1 ( 1 − t ) ∥ ∇ 2 H j , u ( a j + t h ) ∥ o p d t ≤ C Δ s e p − 1 r 2 ∫ 0 1 ( 1 − t ) d t = C 2 Δ s e p − 1 r 2 ≤ C Δ s e p − 1 r 2 . (77) \begin{aligned}
\lvert H_{j,u}(x)-\langle u,x-a_j\rangle\rvert
&\le\int_0^1(1-t)\lvert h^{\mathsf T}\nabla^2H_{j,u}(a_j+th)h\rvert\,\mathrm{d}t\\
&\le r^2\int_0^1(1-t)\lVert\nabla^2H_{j,u}(a_j+th)\rVert_{\mathrm{op}}\,\mathrm{d}t\\
&\le C\Delta_{\mathrm{sep}}^{-1}r^2\int_0^1(1-t)\,\mathrm{d}t\\
&=\frac{C}{2}\Delta_{\mathrm{sep}}^{-1}r^2\\
&\le C\Delta_{\mathrm{sep}}^{-1}r^2. \tag*{(77)}
\end{aligned} ∣ H j , u ( x ) − ⟨ u , x − a j ⟩∣ ≤ ∫ 0 1 ( 1 − t ) ∣ h T ∇ 2 H j , u ( a j + t h ) h ∣ d t ≤ r 2 ∫ 0 1 ( 1 − t ) ∥ ∇ 2 H j , u ( a j + t h ) ∥ op d t ≤ C Δ sep − 1 r 2 ∫ 0 1 ( 1 − t ) d t = 2 C Δ sep − 1 r 2 ≤ C Δ sep − 1 r 2 . (77)
It remains to express the directional error in terms of P ( x ) P(x) P ( x ) . Since c ( x ) = j c(x)=j c ( x ) = j , the positive-coercivity bound in Equation (C.2) gives
P ( x ) ≥ c p o s Δ s e p 2 s max − 2 ∥ x − a j ∥ 2 = c p o s Δ s e p 2 s max − 2 r 2 . P(x)\ge c_{\mathrm{pos}}\Delta_{\mathrm{sep}}^{2s_{\max}-2}\lVert x-a_j\rVert^2
=c_{\mathrm{pos}}\Delta_{\mathrm{sep}}^{2s_{\max}-2}r^2. P ( x ) ≥ c pos Δ sep 2 s m a x − 2 ∥ x − a j ∥ 2 = c pos Δ sep 2 s m a x − 2 r 2 .
Dividing by the positive coefficient on the right yields
r 2 ≤ c p o s − 1 Δ s e p − ( 2 s max − 2 ) P ( x ) . (78) r^2 \le c_{\mathrm{pos}}^{-1}\Delta_{\mathrm{sep}}^{-(2s_{\max}-2)}P(x). \tag*{(78)} r 2 ≤ c pos − 1 Δ sep − ( 2 s m a x − 2 ) P ( x ) . (78)
Equation (C.15) gives
∣ H j , u ( x ) − ⟨ u , x − a j ⟩ ∣ ≤ C Δ s e p − 1 r 2 ≤ C Δ s e p − 1 Δ s e p − ( 2 s max − 2 ) P ( x ) = C Δ s e p − ( 2 s max − 1 ) P ( x ) , \begin{aligned}
|H_{j,u}(x)-\langle u,x-a_j\rangle| &\le C\Delta_{\mathrm{sep}}^{-1}r^2 \\
&\le C\Delta_{\mathrm{sep}}^{-1}\Delta_{\mathrm{sep}}^{-(2s_{\max}-2)}P(x) \\
&= C\Delta_{\mathrm{sep}}^{-(2s_{\max}-1)}P(x),
\end{aligned} ∣ H j , u ( x ) − ⟨ u , x − a j ⟩ ∣ ≤ C Δ sep − 1 r 2 ≤ C Δ sep − 1 Δ sep − ( 2 s m a x − 2 ) P ( x ) = C Δ sep − ( 2 s m a x − 1 ) P ( x ) ,
where c p o s − 1 c_{\mathrm{pos}}^{-1} c pos − 1 is absorbed into C C C . Because c ( x ) = j c(x)=j c ( x ) = j , the last inequality is precisely (C.9) in the remaining case r ≤ Δ s e p / 4 r\le\Delta_{\mathrm{sep}}/4 r ≤ Δ sep /4 . The cases c ( x ) ≠ j c(x)\ne j c ( x ) = j and c ( x ) = j c(x)=j c ( x ) = j with ∥ x − a j ∥ > Δ s e p / 4 \lVert x-a_j\rVert>\Delta_{\mathrm{sep}}/4 ∥ x − a j ∥ > Δ sep /4 were established above. Thus (C.9) holds for every x ∈ B d ( 0 , R ) x\in B_d(0,R) x ∈ B d ( 0 , R ) .
Polynomial approximation of cluster indicators
The polynomials in the preceding subsection distinguish individual true centers, and their bounds involve inverse powers of Δ s e p \Delta_{\mathrm{sep}} Δ sep . For cluster probabilities, we only need to distinguish centers belonging to different clusters. We therefore construct a polynomial that takes the same value at every center in a given cluster, with bounds that remain uniform as centers within that cluster approach one another. These polynomials will control the cluster-sequence probability error J I J_I J I appearing in the transport bounds of Appendix § 9 .
Our construction reduces interpolation to one dimension. We first choose a projection that preserves pairwise distances up to a factor depending only on K K K . We then establish an interpolation bound for smooth functions that does not involve inverse node spacings. This allows us to prescribe cluster values and zero gradients without introducing additional powers of Δ s e p − 1 \Delta_{\mathrm{sep}}^{-1} Δ sep − 1 .
Lemma C.3 (A projection preserving pairwise separation). Let a 1 , … , a K a_1,\ldots,a_K a 1 , … , a K be distinct points in a Euclidean space, with K ≥ 2 K\ge2 K ≥ 2 . There exists a unit vector v v v in the span of their differences such that
∣ ⟨ v , a j − a l ⟩ ∣ ≥ c K ∥ a j − a l ∥ , j ≠ l , (79) |\langle v,a_j-a_l\rangle|\ge c_K\lVert a_j-a_l\rVert,\qquad j\ne l, \tag*{(79)} ∣ ⟨ v , a j − a l ⟩ ∣ ≥ c K ∥ a j − a l ∥ , j = l , (79) where c K > 0 c_K>0 c K > 0 depends only on K K K .
Proof. Let S 0 = span { a j − a 1 : 2 ≤ j ≤ K } S_0=\operatorname{span}\{a_j-a_1:2\le j\le K\} S 0 = span { a j − a 1 : 2 ≤ j ≤ K } and q = dim S 0 ∈ [ 1 , K − 1 ] q=\dim S_0\in[1,K-1] q = dim S 0 ∈ [ 1 , K − 1 ] . If q = 1 q=1 q = 1 , either unit direction in S 0 S_0 S 0 works with c K = 1 c_K=1 c K = 1 . Otherwise, take a uniform random unit vector V ∈ S 0 V\in S_0 V ∈ S 0 . For every unit u ∈ S 0 u\in S_0 u ∈ S 0 , the density of ⟨ V , u ⟩ \langle V,u\rangle ⟨ V , u ⟩ is bounded on [ − 1 / 2 , 1 / 2 ] [-1/2,1/2] [ − 1/2 , 1/2 ] by a constant M q < ∞ M_q<\infty M q < ∞ , and thus P ( ∣ ⟨ V , u ⟩ ∣ < η ) ≤ 2 M q η \mathbb{P}(|\langle V,u\rangle|<\eta)\le2M_q\eta P ( ∣ ⟨ V , u ⟩ ∣ < η ) ≤ 2 M q η for η ≤ 1 / 2 \eta\le1/2 η ≤ 1/2 . There are N K = K ( K − 1 ) / 2 N_K=K(K-1)/2 N K = K ( K − 1 ) /2 normalized difference directions u j l = ( a j − a l ) / ∥ a j − a l ∥ u_{jl}=(a_j-a_l)/\lVert a_j-a_l\rVert u j l = ( a j − a l ) / ∥ a j − a l ∥ . Since q ≤ K − 1 q\le K-1 q ≤ K − 1 , choose η K = min { 1 / 2 , [ 4 N K max 2 ≤ q ′ ≤ K − 1 M q ′ ] − 1 } \eta_K=\min\{1/2,[4N_K\max_{2\le q'\le K-1}M_{q'}]^{-1}\} η K = min { 1/2 , [ 4 N K max 2 ≤ q ′ ≤ K − 1 M q ′ ] − 1 } . A union bound shows that with probability at least 1 / 2 1/2 1/2 , ∣ ⟨ V , u j l ⟩ ∣ ≥ η K |\langle V,u_{jl}\rangle|\ge\eta_K ∣ ⟨ V , u j l ⟩ ∣ ≥ η K for all pairs. Fix such a direction v v v and set c K = η K c_K=\eta_K c K = η K .
The projection preserves the gap between different clusters, but projected centers within a cluster may still be arbitrarily close. We therefore need interpolation bounds that remain valid as nodes approach one another. Repeated nodes will impose derivative conditions, allowing us to prescribe both the value and the first derivative at each projected center.
Lemma C.4 (Interpolation bounds independent of node spacings). Let z 1 , … , z N ∈ [ − R , R ] z_1,\ldots,z_N \in[-R,R] z 1 , … , z N ∈ [ − R , R ] be interpolation nodes, with repetitions allowed , and let f ∈ C N ( [ − R , R ] ) f \in C^N([-R,R]) f ∈ C N ([ − R , R ]) . A node occurring m m m times prescribes the value of f f f and its derivatives through order m − 1 m-1 m − 1 at that node. There is a unique polynomial H f H_f H f of degree at most N − 1 N-1 N − 1 satisfying these conditions. It has the Newton representation
H f ( t ) = ∑ b = 0 N − 1 f [ z 1 , … , z b + 1 ] ∏ a = 1 b ( t − z a ) , (80) H_f(t)=\sum_{b=0}^{N-1}f[z_1,\ldots,z_{b+1}]\prod_{a=1}^{b}(t-z_a), \tag*{(80)} H f ( t ) = b = 0 ∑ N − 1 f [ z 1 , … , z b + 1 ] a = 1 ∏ b ( t − z a ) , (80) where an empty product equals one, and
∣ f [ z 1 , … , z b + 1 ] ∣ ≤ ∥ f ( b ) ∥ ∞ , [ − R , R ] b ! , 0 ≤ b ≤ N − 1. (81) |f[z_1,\ldots,z_{b+1}]|\le\frac{\|f^{(b)}\|_{\infty,[-R,R]}}{b!},\qquad0\le b\le N-1. \tag*{(81)} ∣ f [ z 1 , … , z b + 1 ] ∣ ≤ b ! ∥ f ( b ) ∥ ∞ , [ − R , R ] , 0 ≤ b ≤ N − 1. (81) Consequently, bounds on f f f and its derivatives through order N − 1 N-1 N − 1 give bounds on ∥ H f ∥ ∞ , [ − R , R ] \|H_f\|_{\infty,[-R,R]} ∥ H f ∥ ∞ , [ − R , R ] and ∥ H f ′ ′ ∥ ∞ , [ − R , R ] \|H_f''\|_{\infty,[-R,R]} ∥ H f ′′ ∥ ∞ , [ − R , R ] that do not depend on the node spacings.
Proof. For 0 ≤ b ≤ N − 1 0\le b\le N-1 0 ≤ b ≤ N − 1 , the Hermite conditions at the first b + 1 b+1 b + 1 nodes (counted with multiplicity) determine a unique degree-b b b polynomial H b H_b H b : a polynomial in the kernel would be divisible by a degree-( b + 1 ) (b+1) ( b + 1 ) product of node factors, hence vanish identically. As H b − H b − 1 H_b-H_{b-1} H b − H b − 1 satisfies the first b b b homogeneous conditions, H b − H b − 1 = c b ω b H_b-H_{b-1}=c_b\omega_b H b − H b − 1 = c b ω b , where ω b ( t ) = ∏ a = 1 b ( t − z a ) \omega_b(t)=\prod_{a=1}^{b}(t-z_a) ω b ( t ) = ∏ a = 1 b ( t − z a ) and ω 0 = 1 \omega_0=1 ω 0 = 1 . Summing this recursion proves the Newton formula (C.18) with c b = f [ z 1 , … , z b + 1 ] c_b=f[z_1,\ldots,z_{b+1}] c b = f [ z 1 , … , z b + 1 ] .
The function f − H b f-H_b f − H b has b + 1 b+1 b + 1 zeros counted with multiplicity. Repeated application of Rolle’s theorem, including confluent nodes, produces a point ξ \xi ξ between the extreme nodes where ( f − H b ) ( b ) ( ξ ) = 0 (f-H_b)^{(b)}(\xi)=0 ( f − H b ) ( b ) ( ξ ) = 0 . Since H b ( b ) = b ! c b H_b^{(b)}=b!c_b H b ( b ) = b ! c b , this gives ∣ c b ∣ ≤ ∥ f ( b ) ∥ ∞ / b ! |c_b|\le\|f^{(b)}\|_\infty/b! ∣ c b ∣ ≤ ∥ f ( b ) ∥ ∞ / b ! , proving (C.19) without division by node gaps. Furthermore, for 0 ≤ ℓ ≤ b 0\le\ell\le b 0 ≤ ℓ ≤ b ,
∥ ω b ( ℓ ) ∥ ∞ ≤ b ! ( b − ℓ ) ! ( 2 R ) b − ℓ . \|\omega_b^{(\ell)}\|_\infty\le\frac{b!}{(b-\ell)!}(2R)^{b-\ell}. ∥ ω b ( ℓ ) ∥ ∞ ≤ ( b − ℓ )! b ! ( 2 R ) b − ℓ . Therefore the Newton formula yields
∥ H f ( ℓ ) ∥ ∞ ≤ ∑ b = ℓ N − 1 ∥ f ( b ) ∥ ∞ ( b − ℓ ) ! ( 2 R ) b − ℓ , 0 ≤ ℓ < N . \|H_f^{(\ell)}\|_\infty\le\sum_{b=\ell}^{N-1}\frac{\|f^{(b)}\|_\infty}{(b-\ell)!}(2R)^{b-\ell},\qquad0\le\ell<N. ∥ H f ( ℓ ) ∥ ∞ ≤ b = ℓ ∑ N − 1 ( b − ℓ )! ∥ f ( b ) ∥ ∞ ( 2 R ) b − ℓ , 0 ≤ ℓ < N . Take ℓ = 0 , 2 \ell=0,2 ℓ = 0 , 2 ; if N ≤ 2 N\le2 N ≤ 2 , then H f ′ ′ = 0 H_f''=0 H f ′′ = 0 . All constants are independent of the spacing and repetition of nodes.
We now construct the smooth function to be interpolated. Its value will be constant near every projected center, with value one for the selected cluster and zero for the others. The gap between different clusters allows these values to be joined smoothly with uniformly bounded derivatives. Interpolating both values and first derivatives then produces the required polynomial.
Lemma C.5 (Polynomial approximation of cluster indicators). Under the structured geometry, for each cluster C m C_m C m there exists a polynomial F m F_m F m of degree at most 2 K − 1 2K-1 2 K − 1 such that
F m ( a j ) = 1 { j ∈ C m } , ∇ F m ( a j ) = 0 , j ∈ [ K ] , (82) F_m(a_j)=\mathbf{1}\{j\in C_m\},\qquad\nabla F_m(a_j)=0,\qquad j\in[K], \tag*{(82)} F m ( a j ) = 1 { j ∈ C m } , ∇ F m ( a j ) = 0 , j ∈ [ K ] , (82) ∥ F m ∥ ∞ , B d ( 0 , R ) + sup x ∈ B d ( 0 , R ) ∥ ∇ 2 F m ( x ) ∥ o p ≤ C , (83) \|F_m\|_{\infty,B_d(0,R)}+\sup_{x\in B_d(0,R)}\|\nabla^2F_m(x)\|_{\mathrm{op}}\le C, \tag*{(83)} ∥ F m ∥ ∞ , B d ( 0 , R ) + x ∈ B d ( 0 , R ) sup ∥ ∇ 2 F m ( x ) ∥ op ≤ C , (83) and
∣ F m ( x ) − 1 { c ( x ) ∈ C m } ∣ ≤ C ∥ x − a c ( x ) ∥ 2 , x ∈ B d ( 0 , R ) . (84) |F_m(x)-\mathbf{1}\{c(x)\in C_m\}|\le C\|x-a_{c(x)}\|^2,\qquad x\in B_d(0,R). \tag*{(84)} ∣ F m ( x ) − 1 { c ( x ) ∈ C m } ∣ ≤ C ∥ x − a c ( x ) ∥ 2 , x ∈ B d ( 0 , R ) . (84) The constant depends only on K , R , D 0 K,R,D_0 K , R , D 0 , independently of Δ s e p \Delta_{\mathrm{sep}} Δ sep and the ambient dimension d d d . For a single cluster, one may take F 1 ≡ 1 F_1\equiv1 F 1 ≡ 1 .
Proof. If there is only one cluster, F 1 ≡ 1 F_1 \equiv1 F 1 ≡ 1 satisfies all three claims. Assume that there are at least two clusters, so K ≥ 2 K \ge2 K ≥ 2 .
Projected centers and the gap between clusters. Choose the unit vector v v v from Lemma C.3 and set
t j = ⟨ v , a j ⟩ , j ∈ [ K ] . t_j = \langle v, a_j \rangle, \qquad j \in[K]. t j = ⟨ v , a j ⟩ , j ∈ [ K ] .
The projected centers are distinct and satisfy ∣ t j ∣ ≤ R |t_j| \le R ∣ t j ∣ ≤ R . For centers in different clusters,
∣ t j − t l ∣ = ∣ ⟨ v , a j − a l ⟩ ∣ ≥ c K ∥ a j − a l ∥ (by xref‘(C.17) ) ≥ c K D 0 (by the between-cluster separation). \begin{aligned}
|t_j-t_l| &= |\langle v,a_j-a_l\rangle| \\
&\ge c_K\|a_j-a_l\| \qquad\text{(by {xref}`(C.17)}) \\
&\ge c_KD_0 \qquad\text{(by the between-cluster separation).}
\end{aligned} ∣ t j − t l ∣ = ∣ ⟨ v , a j − a l ⟩ ∣ ≥ c K ∥ a j − a l ∥ (by xref ‘(C.17) ) ≥ c K D 0 (by the between-cluster separation).
Write γ = c K D 0 \gamma= c_KD_0 γ = c K D 0 . Thus different clusters remain separated by at least γ \gamma γ on the line, even though projected centers within a cluster may be arbitrarily close.
A smooth function with the required cluster values. Fix a cluster C m C_m C m . We construct a smooth function that equals one near its projected centers and zero near all other projected centers. Begin with a fixed smooth cutoff ψ \psi ψ satisfying
0 ≤ ψ ≤ 1 , ψ ( t ) = 1 for ∣ t ∣ ≤ 1 2 , ψ ( t ) = 0 for ∣ t ∣ ≥ 1. 0 \le\psi\le1, \qquad\psi(t)=1\ \text{for } |t|\le\frac{1}{2}, \qquad\psi(t)=0\ \text{for } |t|\ge1. 0 ≤ ψ ≤ 1 , ψ ( t ) = 1 for ∣ t ∣ ≤ 2 1 , ψ ( t ) = 0 for ∣ t ∣ ≥ 1.
For an explicit choice, define
ζ ( t ) = { e − 1 / t , t > 0 , 0 , t ≤ 0 , χ ( t ) = ζ ( t ) ζ ( t ) + ζ ( 1 − t ) , ψ ( t ) = χ ( 4 ( 1 − t 2 ) 3 ) . \zeta(t)=
\begin{cases}
e^{-1/t}, & t>0,\\
0, & t\le0,
\end{cases}
\qquad
\chi(t)=\frac{\zeta(t)}{\zeta(t)+\zeta(1-t)},
\qquad
\psi(t)=\chi\left(\frac{4(1-t^2)}{3}\right). ζ ( t ) = { e − 1/ t , 0 , t > 0 , t ≤ 0 , χ ( t ) = ζ ( t ) + ζ ( 1 − t ) ζ ( t ) , ψ ( t ) = χ ( 3 4 ( 1 − t 2 ) ) .
The function χ \chi χ equals zero on ( − ∞ , 0 ] (-\infty,0] ( − ∞ , 0 ] and one on [ 1 , ∞ ) [1,\infty) [ 1 , ∞ ) , which gives the stated properties of ψ \psi ψ . Since ψ \psi ψ is smooth and compactly supported, each derivative has a finite bound
M b : = sup t ∈ R ∣ ψ ( b ) ( t ) ∣ . M_b := \sup_{t\in\mathbb{R}} |\psi^{(b)}(t)|. M b := t ∈ R sup ∣ ψ ( b ) ( t ) ∣.
For every j ∈ C m j \in C_m j ∈ C m , place a copy of this cutoff around t j t_j t j by setting
b j ( t ) = ψ ( 3 ( t − t j ) γ ) . b_j(t)=\psi\left(\frac{3(t-t_j)}{\gamma}\right). b j ( t ) = ψ ( γ 3 ( t − t j ) ) .
Then
b j ( t ) = 1 if ∣ t − t j ∣ ≤ γ 6 , b j ( t ) = 0 if ∣ t − t j ∣ ≥ γ 3 . b_j(t)=1\ \text{if } |t-t_j|\le\frac{\gamma}{6}, \qquad b_j(t)=0\ \text{if } |t-t_j|\ge\frac{\gamma}{3}. b j ( t ) = 1 if ∣ t − t j ∣ ≤ 6 γ , b j ( t ) = 0 if ∣ t − t j ∣ ≥ 3 γ .
To combine these cutoffs, define
f m ( t ) = 1 − ∏ j ∈ C m { 1 − b j ( t ) } . f_m(t)=1-\prod_{j\in C_m}\{1-b_j(t)\}. f m ( t ) = 1 − j ∈ C m ∏ { 1 − b j ( t )} .
If any one of the cutoffs equals one, the product vanishes and f m = 1 f_m=1 f m = 1 . If all cutoffs vanish, the product equals one and f m = 0 f_m=0 f m = 0 . In particular, f m f_m f m equals one in a neighborhood of every t j t_j t j with j ∈ C m j\in C_m j ∈ C m .
For l ∉ C m l \notin C_m l ∈ / C m and ∣ t − t l ∣ < γ / 3 |t-t_l|<\gamma/3 ∣ t − t l ∣ < γ /3 , every j ∈ C m j\in C_m j ∈ C m satisfies
∣ t − t j ∣ ≥ ∣ t l − t j ∣ − ∣ t − t l ∣ > γ − γ 3 = 2 γ 3 . |t-t_j|\ge|t_l-t_j|-|t-t_l|
>\gamma-\frac{\gamma}{3}=\frac{2\gamma}{3}. ∣ t − t j ∣ ≥ ∣ t l − t j ∣ − ∣ t − t l ∣ > γ − 3 γ = 3 2 γ .
All b j ( t ) b_j(t) b j ( t ) therefore vanish in this neighborhood of t l t_l t l , so f m = 0 f_m=0 f m = 0 there. We have obtained
f m ( t j ) = 1 { j ∈ C m } , f m ( b ) ( t j ) = 0 for every b ≥ 1. f_m(t_j)=\mathbf{1}\{j\in C_m\},\qquad f_m^{(b)}(t_j)=0\quad\text{for every }b\ge1. f m ( t j ) = 1 { j ∈ C m } , f m ( b ) ( t j ) = 0 for every b ≥ 1.
The derivative bounds of f m f_m f m depend on γ \gamma γ , not on distances between centers in the same cluster. To verify this, the chain rule gives
∥ b j ( b ) ∥ ∞ , R ≤ ( 3 γ ) b M b . \|b_j^{(b)}\|_{\infty,\mathbb{R}}\le\left(\frac{3}{\gamma}\right)^b M_b. ∥ b j ( b ) ∥ ∞ , R ≤ ( γ 3 ) b M b .
Put
B γ = max { 1 , max 1 ≤ b ≤ 2 K ( 3 γ ) b M b } , s m = ∣ C m ∣ . B_\gamma=\max\left\{1,\max_{1\le b\le2K}\left(\frac{3}{\gamma}\right)^b M_b\right\},\qquad s_m=|C_m|. B γ = max { 1 , 1 ≤ b ≤ 2 K max ( γ 3 ) b M b } , s m = ∣ C m ∣.
All derivatives through order 2 K 2K 2 K of each factor 1 − b j 1-b_j 1 − b j , including order zero, are bounded by B γ B_\gamma B γ . In each term of the product rule, the b b b derivatives are distributed among the s m s_m s m factors 1 − b j 1-b_j 1 − b j . If these factors receive ν 1 , … , ν s m \nu_1,\ldots,\nu_{s_m} ν 1 , … , ν s m derivatives, respectively, then ν 1 + ⋯ + ν s m = b \nu_1+\cdots+\nu_{s_m}=b ν 1 + ⋯ + ν s m = b , the coefficient is b ! / ( ν 1 ! ⋯ ν s m ! ) b!/(\nu_1!\cdots\nu_{s_m}!) b ! / ( ν 1 ! ⋯ ν s m !) , and the product of derivatives has absolute value at most B γ s m B_\gamma^{s_m} B γ s m . Thus, for 1 ≤ b ≤ 2 K 1\le b\le2K 1 ≤ b ≤ 2 K ,
∥ f m ( b ) ∥ ∞ , R ≤ ∑ ν 1 + ⋯ + ν s m = b ν i ≥ 0 b ! ν 1 ! ⋯ ν s m ! B γ s m = s m b B γ s m (by the multinomial formula) . \|f_m^{(b)}\|_{\infty,\mathbb{R}}
\le
\sum_{\substack{\nu_1+\cdots+\nu_{s_m}=b\\ \nu_i\ge0}}
\frac{b!}{\nu_1!\cdots\nu_{s_m}!}B_\gamma^{s_m}
=s_m^bB_\gamma^{s_m}
\quad\text{(by the multinomial formula)}. ∥ f m ( b ) ∥ ∞ , R ≤ ν 1 + ⋯ + ν s m = b ν i ≥ 0 ∑ ν 1 ! ⋯ ν s m ! b ! B γ s m = s m b B γ s m (by the multinomial formula) .
Also, 0 ≤ f m ≤ 1 0\le f_m\le1 0 ≤ f m ≤ 1 . Since s m ≤ K s_m\le K s m ≤ K and γ = c K D 0 \gamma=c_KD_0 γ = c K D 0 , these bounds depend only on K , D 0 K,D_0 K , D 0 .
Replacing the smooth function by a polynomial. Apply Lemma C.4 to f m f_m f m with each projected center repeated twice,
( t 1 , t 1 , … , t K , t K ) . (t_1,t_1,\ldots,t_K,t_K). ( t 1 , t 1 , … , t K , t K ) .
The two occurrences of t j t_j t j prescribe its function value and first derivative. The resulting polynomial H m H_m H m has degree at most 2 K − 1 2K-1 2 K − 1 and satisfies
H m ( t j ) = f m ( t j ) = 1 { j ∈ C m } ⏟ prescribed cluster value , H m ′ ( t j ) = f m ′ ( t j ) = 0 ⏟ prescribed zero derivative . \underbrace{H_m(t_j)=f_m(t_j)=\mathbf{1}\{j\in C_m\}}_{\text{prescribed cluster value}},
\qquad
\underbrace{H_m'(t_j)=f_m'(t_j)=0}_{\text{prescribed zero derivative}}. prescribed cluster value H m ( t j ) = f m ( t j ) = 1 { j ∈ C m } , prescribed zero derivative H m ′ ( t j ) = f m ′ ( t j ) = 0 .
Moreover, the derivative bounds just established give
∥ H m ∥ ∞ , [ − R , R ] ≤ ∑ b = 0 2 K − 1 ∥ f m ( b ) ∥ ∞ , [ − R , R ] b ! ( 2 R ) b , \|H_m\|_{\infty,[-R,R]}
\le
\sum_{b=0}^{2K-1}\frac{\|f_m^{(b)}\|_{\infty,[-R,R]}}{b!}(2R)^b, ∥ H m ∥ ∞ , [ − R , R ] ≤ b = 0 ∑ 2 K − 1 b ! ∥ f m ( b ) ∥ ∞ , [ − R , R ] ( 2 R ) b ,
∥ H m ′ ′ ∥ ∞ , [ − R , R ] ≤ ∑ b = 2 2 K − 1 ∥ f m ( b ) ∥ ∞ , [ − R , R ] ( b − 2 ) ! ( 2 R ) b − 2 . \|H_m''\|_{\infty,[-R,R]}
\le
\sum_{b=2}^{2K-1}\frac{\|f_m^{(b)}\|_{\infty,[-R,R]}}{(b-2)!}(2R)^{b-2}. ∥ H m ′′ ∥ ∞ , [ − R , R ] ≤ b = 2 ∑ 2 K − 1 ( b − 2 )! ∥ f m ( b ) ∥ ∞ , [ − R , R ] ( 2 R ) b − 2 .
Both sums are bounded by a constant depending only on K , R , D 0 K,R,D_0 K , R , D 0 . In particular, no inverse within-cluster spacing enters these estimates.
Return to the original space by defining
F m ( x ) = H m ( ⟨ v , x ⟩ ) . F_m(x) = H_m(\langle v,x\rangle). F m ( x ) = H m (⟨ v , x ⟩) .
The chain rule gives
∇ F m ( x ) = H m ′ ( ⟨ v , x ⟩ ) v , ∇ 2 F m ( x ) = H m ′ ′ ( ⟨ v , x ⟩ ) v v T . \nabla F_m(x) = H_m'(\langle v,x\rangle)v,\qquad\nabla^2 F_m(x) = H_m''(\langle v,x\rangle)vv^{\mathsf{T}}. ∇ F m ( x ) = H m ′ (⟨ v , x ⟩) v , ∇ 2 F m ( x ) = H m ′′ (⟨ v , x ⟩) v v T .
Evaluating at a j a_j a j proves (C.20). Furthermore, ∣ ⟨ v , x ⟩ ∣ ≤ R |\langle v,x\rangle| \le R ∣ ⟨ v , x ⟩ ∣ ≤ R for x ∈ B d ( 0 , R ) x \in B_d(0,R) x ∈ B d ( 0 , R ) and ∥ v v T ∥ o p = 1 \|vv^{\mathsf{T}}\|_{\mathrm{op}}=1 ∥ v v T ∥ op = 1 , so
∥ F m ∥ ∞ , B d ( 0 , R ) ≤ ∥ H m ∥ ∞ , [ − R , R ] , sup x ∈ B d ( 0 , R ) ∥ ∇ 2 F m ( x ) ∥ o p ≤ ∥ H m ′ ′ ∥ ∞ , [ − R , R ] . \begin{aligned}
\|F_m\|_{\infty,B_d(0,R)} &\le\|H_m\|_{\infty,[-R,R]},\\
\sup_{x\in B_d(0,R)}\|\nabla^2 F_m(x)\|_{\mathrm{op}} &\le\|H_m''\|_{\infty,[-R,R]}.
\end{aligned} ∥ F m ∥ ∞ , B d ( 0 , R ) x ∈ B d ( 0 , R ) sup ∥ ∇ 2 F m ( x ) ∥ op ≤ ∥ H m ∥ ∞ , [ − R , R ] , ≤ ∥ H m ′′ ∥ ∞ , [ − R , R ] .
This proves (C.21), with no factor depending on d d d .
Bounding the error away from the true centers. Fix x ∈ B d ( 0 , R ) x \in B_d(0,R) x ∈ B d ( 0 , R ) , let a = a c ( x ) a=a_{c(x)} a = a c ( x ) be its assigned true center, and write h = x − a h=x-a h = x − a . The segment a + t h a+th a + t h , 0 ≤ t ≤ 1 0\le t\le1 0 ≤ t ≤ 1 , lies in B d ( 0 , R ) B_d(0,R) B d ( 0 , R ) by convexity. Taylor’s formula with integral remainder gives
F m ( x ) = F m ( a ) ⏟ 1 { c ( x ) ∈ C m } + ⟨ ∇ F m ( a ) , h ⟩ ⏟ 0 + ∫ 0 1 ( 1 − t ) h T ∇ 2 F m ( a + t h ) h d t . \begin{aligned}
F_m(x)
&=\underbrace{F_m(a)}_{\mathbb{1}\{c(x)\in C_m\}}
+\underbrace{\langle\nabla F_m(a),h\rangle}_{0}\\
&\quad+\int_0^1(1-t)h^{\mathsf{T}}\nabla^2F_m(a+th)h\,dt.
\end{aligned} F m ( x ) = 1 { c ( x ) ∈ C m } F m ( a ) + 0 ⟨ ∇ F m ( a ) , h ⟩ + ∫ 0 1 ( 1 − t ) h T ∇ 2 F m ( a + t h ) h d t .
The zero gradient removes the linear term, leaving an error controlled by the squared displacement. Specifically,
∣ F m ( x ) − 1 { c ( x ) ∈ C m } ∣ ≤ ∫ 0 1 ( 1 − t ) ∣ h T ∇ 2 F m ( a + t h ) h ∣ d t ≤ ∥ h ∥ 2 ∫ 0 1 ( 1 − t ) ∥ ∇ 2 F m ( a + t h ) ∥ o p d t ≤ C ∥ h ∥ 2 ∫ 0 1 ( 1 − t ) d t (by xref‘(C.21)‘) = C 2 ∥ x − a c ( x ) ∥ 2 . \begin{aligned}
\left|F_m(x)-\mathbb{1}\{c(x)\in C_m\}\right|
&\le\int_0^1(1-t)\left|h^{\mathsf{T}}\nabla^2F_m(a+th)h\right|\,dt\\
&\le\|h\|^2\int_0^1(1-t)\|\nabla^2F_m(a+th)\|_{\mathrm{op}}\,dt\\
&\le C\|h\|^2\int_0^1(1-t)\,dt\qquad\text{(by {xref}`(C.21)`)}\\
&=\frac{C}{2}\|x-a_{c(x)}\|^2.
\end{aligned} ∣ F m ( x ) − 1 { c ( x ) ∈ C m } ∣ ≤ ∫ 0 1 ( 1 − t ) h T ∇ 2 F m ( a + t h ) h d t ≤ ∥ h ∥ 2 ∫ 0 1 ( 1 − t ) ∥ ∇ 2 F m ( a + t h ) ∥ op d t ≤ C ∥ h ∥ 2 ∫ 0 1 ( 1 − t ) d t (by xref ‘(C.21)‘) = 2 C ∥ x − a c ( x ) ∥ 2 .
After adjusting the constant, this proves (C.22). The argument applies to every x ∈ B d ( 0 , R ) x \in B_d(0,R) x ∈ B d ( 0 , R ) and requires no proximity assumption on the fitted centers.
Joint selectors with vanishing first derivatives
For a joint probability, it is enough to interpolate its values and first derivatives on the true support. Requiring a separate flat selector in every coordinate imposes additional conditions that are not needed for the quadratic approximation error. The following construction uses ordinary cardinal polynomials in the remaining coordinates.
Lemma C.6 (Joint probability selectors). Suppose that the structured geometry holds, put s = s max s=s_{\max} s = s m a x and Δ = Δ s e p \Delta=\Delta_{\mathrm{sep}} Δ = Δ sep , and define
d r = r ( s − 1 ) + s , β r = max { 2 s , d r } . d_r=r(s-1)+s,\qquad\beta_r=\max\{2s,d_r\}. d r = r ( s − 1 ) + s , β r = max { 2 s , d r } .
For every j = ( j 1 , … , j r ) ∈ [ K ] r j=(j_1,\ldots,j_r)\in[K]^r j = ( j 1 , … , j r ) ∈ [ K ] r , there is a polynomial U j U_j U j of degree bounded in terms of K , r K,r K , r such that
U j ( a i 1 , … , a i r ) = 1 { i = j } , ∇ U j ( a i 1 , … , a i r ) = 0 , (85) U_j(a_{i_1},\ldots,a_{i_r})=1\{i=j\},\qquad\nabla U_j(a_{i_1},\ldots,a_{i_r})=0, \tag*{(85)} U j ( a i 1 , … , a i r ) = 1 { i = j } , ∇ U j ( a i 1 , … , a i r ) = 0 , (85)
∥ U j ∥ ∞ , B d ( 0 , R ) r ≤ C r Δ − d r , (86) \|U_j\|_{\infty,B_d(0,R)^r}\le C_r\Delta^{-dr}, \tag*{(86)} ∥ U j ∥ ∞ , B d ( 0 , R ) r ≤ C r Δ − d r , (86)
∣ U j ( x 1 , … , x r ) − 1 { ( c ( x 1 ) , … , c ( x r ) ) = j } ∣ ≤ C r Δ − β r ∑ ℓ = 1 r P ( x ℓ ) . (87) \left|U_j(x_1,\ldots,x_r)-1\{(c(x_1),\ldots,c(x_r))=j\}\right|
\le C_r\Delta^{-\beta r}\sum_{\ell=1}^{r}P(x_\ell). \tag*{(87)} ∣ U j ( x 1 , … , x r ) − 1 {( c ( x 1 ) , … , c ( x r )) = j } ∣ ≤ C r Δ − β r ℓ = 1 ∑ r P ( x ℓ ) . (87)
Here P ( x ) = ∏ i = 1 K ∥ x − a i ∥ 2 2 P(x)=\prod_{i=1}^{K}\|x-a_i\|_2^2 P ( x ) = ∏ i = 1 K ∥ x − a i ∥ 2 2 is the positive polynomial in (C.1). Constants are independent of the ambient dimension and of Δ \Delta Δ .
Proof. We first construct localized one-coordinate polynomials, and then combine them so that their first derivatives cancel.
Cluster selectors with higher-order flatness. Choose N = 2 s + 2 N=2s+2 N = 2 s + 2 . Repeat the construction of Lemma C.5 , now repeating each projected true center N + 1 N+1 N + 1 times in Lemma C.4 . The smooth cluster indicator used there is constant near every projected center. Its derivatives of every fixed order are bounded independently of within-cluster spacing. The resulting polynomials F m F_m F m therefore satisfy
F m ( a i ) = 1 { i ∈ C m } , D v F m ( a i ) = 0 ( 1 ≤ v ≤ N ) , F_m(a_i)=1\{i\in C_m\},\qquad D^vF_m(a_i)=0\quad(1\le v\le N), F m ( a i ) = 1 { i ∈ C m } , D v F m ( a i ) = 0 ( 1 ≤ v ≤ N ) ,
and have uniformly bounded derivatives of every fixed order on B d ( 0 , R ) B_d(0,R) B d ( 0 , R ) . Their degree is at most ( N + 1 ) K − 1 (N+1)K-1 ( N + 1 ) K − 1 . These claims follow from the same Newton interpolation formula, with derivative orders up to ( N + 1 ) K − 1 (N+1)K-1 ( N + 1 ) K − 1 in place of 2 K − 1 2K-1 2 K − 1 . For a single cluster, take F 1 = 1 F_1=1 F 1 = 1 .
For j ∈ C m j\in C_m j ∈ C m , define the ordinary within-cluster cardinal polynomial
ℓ j ( x ) = ∏ i ∈ C m ∖ { j } ⟨ x − a i , a j − a i ⟩ ∥ a j − a i ∥ 2 2 , \ell_j(x)=\prod_{i\in C_m\setminus\{j\}}
\frac{\langle x-a_i,a_j-a_i\rangle}{\|a_j-a_i\|_2^2}, ℓ j ( x ) = i ∈ C m ∖ { j } ∏ ∥ a j − a i ∥ 2 2 ⟨ x − a i , a j − a i ⟩ ,
and its flat counterpart
e j ( x ) = { 1 − 2 ⟨ ∇ ℓ j ( a j ) , x − a j ⟩ } ℓ j ( x ) 2 . e_j(x)=\{1-2\langle\nabla\ell_j(a_j),x-a_j\rangle\}\ell_j(x)^2. e j ( x ) = { 1 − 2 ⟨ ∇ ℓ j ( a j ) , x − a j ⟩} ℓ j ( x ) 2 .
Their degrees are at most s m − 1 s_m-1 s m − 1 and 2 s m − 1 2s_m-1 2 s m − 1 , respectively, where s m = ∣ C m ∣ s_m=|C_m| s m = ∣ C m ∣ . Within C m C_m C m , both have the cardinal values, and ∇ e j \nabla e_j ∇ e j vanishes at every center in that cluster. No such assertion is needed for the unlocalized e j e_j e j at centers outside C m C_m C m . Set
L j = F m ℓ j , E j = F m e j . L_j=F_m\ell_j,\qquad E_j=F_me_j. L j = F m ℓ j , E j = F m e j .
Then L j ( a i ) = E j ( a i ) = 1 { i = j } L_j(a_i)=E_j(a_i)=1\{i=j\} L j ( a i ) = E j ( a i ) = 1 { i = j } for every true center, and ∇ E j ( a i ) = 0 \nabla E_j(a_i)=0 ∇ E j ( a i ) = 0 . The additional flatness of F m F_m F m also controls these polynomials near centers outside C m C_m C m .
Derivative bounds near the true support. Assume first s ≥ 2 s\ge2 s ≥ 2 , and put p = s − 1 p=s-1 p = s − 1 , q = 2 s − 1 q=2s-1 q = 2 s − 1 . For any true center a i a_i a i , any y ∈ B d ( 0 , R ) y\in B_d(0,R) y ∈ B d ( 0 , R ) , and v = 0 , 1 , 2 v=0,1,2 v = 0 , 1 , 2 , the preceding polynomials satisfy
∥ D v L j ( y ) ∥ ≤ C Δ − p ( Δ + ∥ y − a i ∥ ) p − v , ∥ D v E j ( y ) ∥ ≤ C Δ − q ( Δ + ∥ y − a i ∥ ) q − v . (88) \|D^vL_j(y)\|\le C\Delta^{-p}(\Delta+\|y-a_i\|)^{p-v},\qquad
\|D^vE_j(y)\|\le C\Delta^{-q}(\Delta+\|y-a_i\|)^{q-v}. \tag*{(88)} ∥ D v L j ( y ) ∥ ≤ C Δ − p ( Δ + ∥ y − a i ∥ ) p − v , ∥ D v E j ( y ) ∥ ≤ C Δ − q ( Δ + ∥ y − a i ∥ ) q − v . (88)
Here derivative norms are operator norms, with absolute value for v = 0 v=0 v = 0 . To verify the bounds when i ∈ C m i\in C_m i ∈ C m , every linear factor in ℓ j \ell_j ℓ j has numerator bounded by C ( Δ + ∥ y − a i ∥ ) C(\Delta+\|y-a_i\|) C ( Δ + ∥ y − a i ∥ ) and denominator at least Δ \Delta Δ . Differentiation removes at most one such numerator per derivative. Also ∥ ∇ ℓ j ( a j ) ∥ ≤ C Δ − 1 \|\nabla\ell_j(a_j)\|\le C\Delta^{-1} ∥∇ ℓ j ( a j ) ∥ ≤ C Δ − 1 , giving the stated scaling for e j e_j e j . Multiplication by the uniformly smooth F m F_m F m preserves these bounds. More explicitly, put t = Δ + ∥ y − a i ∥ t=\Delta+\lVert y-a_i\rVert t = Δ + ∥ y − a i ∥ and p m = s m − 1 p_m=s_m-1 p m = s m − 1 . A derivative of order b ≤ p m b\le p_m b ≤ p m of the product of linear factors is bounded by C Δ − p m t p m − b C\Delta^{-p_m}t^{p_m-b} C Δ − p m t p m − b , and a derivative of order b > p m b>p_m b > p m is identically zero. A term in D v ( F m ℓ j ) D^v(F_m\ell_j) D v ( F m ℓ j ) with v − b v-b v − b derivatives on F m F_m F m is therefore bounded by the desired expression times t v − b t^{v-b} t v − b , which is uniformly bounded on the ball. This also covers singleton clusters, where ℓ j = 1 \ell_j=1 ℓ j = 1 . The same product argument applies to e j e_j e j with degree bound 2 s m − 1 2s_m-1 2 s m − 1 ; a derivative beyond its actual degree is zero, not a term with a negative power. If s m < s s_m<s s m < s , increase the scaling degrees to p , q p,q p , q , using Δ / t ≤ 1 \Delta/t\le1 Δ/ t ≤ 1 .
When i ∉ C m i\notin C_m i ∈ / C m , Taylor’s formula and the vanishing derivatives of F m F_m F m give
∥ D v F m ( y ) ∥ ≤ C ∥ y − a i ∥ N + 1 − v , v = 0 , 1 , 2. \lVert D^vF_m(y)\rVert\le C\lVert y-a_i\rVert^{N+1-v},\qquad v=0,1,2. ∥ D v F m ( y )∥ ≤ C ∥ y − a i ∥ N + 1 − v , v = 0 , 1 , 2.
The derivatives through order two of ℓ j , e j \ell_j,e_j ℓ j , e j on the ball are bounded by C Δ − p , C Δ − q C\Delta^{-p},C\Delta^{-q} C Δ − p , C Δ − q . The product rule proves (C.26), since N + 1 ≥ q N+1\ge q N + 1 ≥ q and all distances are bounded. This argument does not require the projected interval hulls of different clusters to be disjoint.
Cancelling the first derivatives. Define
U j ( x 1 , … , x r ) = ∑ ℓ = 1 r E j ℓ ( x ℓ ) ∏ u ≠ ℓ L j u ( x u ) − ( r − 1 ) ∏ u = 1 r L j u ( x u ) . (89) U_j(x_1,\ldots,x_r)=\sum_{\ell=1}^{r}E_{j\ell}(x_\ell)\prod_{u\ne\ell}L_{ju}(x_u)-(r-1)\prod_{u=1}^{r}L_{ju}(x_u). \tag*{(89)} U j ( x 1 , … , x r ) = ℓ = 1 ∑ r E j ℓ ( x ℓ ) u = ℓ ∏ L j u ( x u ) − ( r − 1 ) u = 1 ∏ r L j u ( x u ) . (89)
At a true grid point, each of the first r r r terms has value 1 { i = j } \mathbf{1}\{i=j\} 1 { i = j } . For a derivative in coordinate v v v , the term with E j v E_{jv} E j v has zero derivative. Each of the other r − 1 r-1 r − 1 terms has exactly the same derivative as the last product, because E j u E_{ju} E j u and L j u L_{ju} L j u have the same values on the true support. These derivatives cancel, proving (C.23). The norm bounds for the factors give (C.24), since q + ( r − 1 ) p = d r q+(r-1)p=d_r q + ( r − 1 ) p = d r and r p ≤ d r rp\le d_r r p ≤ d r .
Let a i ℓ = a c ( x ℓ ) a_{i\ell}=a_c(x_\ell) a i ℓ = a c ( x ℓ ) and M = max ℓ ∥ x ℓ − a i ℓ ∥ M=\max_\ell\lVert x_\ell-a_{i\ell}\rVert M = max ℓ ∥ x ℓ − a i ℓ ∥ . Along the segment from this true grid point to ( x 1 , … , x r ) (x_1,\ldots,x_r) ( x 1 , … , x r ) , the product rule and (C.26) bound the Hessian of the first sum by C r Δ − d r ( Δ + M ) d r − 2 C_r\Delta^{-d_r}(\Delta+M)^{d_r-2} C r Δ − d r ( Δ + M ) d r − 2 . The last product has the bound C r Δ − r p ( Δ + M ) r p − 2 C_r\Delta^{-rp}(\Delta+M)^{rp-2} C r Δ − r p ( Δ + M ) r p − 2 , which is no larger, since Δ / ( Δ + M ) ≤ 1 \Delta/(\Delta+M)\le1 Δ/ ( Δ + M ) ≤ 1 . When p = 1 p=1 p = 1 and two derivatives fall on one L j L_j L j , one cannot simply replace its individual distance by M M M in a negative exponent. Instead, ℓ j \ell_j ℓ j has degree at most one, so D 2 ℓ j = 0 D^2\ell_j=0 D 2 ℓ j = 0 . The product rule gives
D 2 ( F m ℓ j ) = ℓ j D 2 F m + ( D F m ) ⊗ D ℓ j + D ℓ j ⊗ ( D F m ) . D^2(F_m\ell_j)=\ell_jD^2F_m+(DF_m)\otimes D\ell_j+D\ell_j\otimes(DF_m). D 2 ( F m ℓ j ) = ℓ j D 2 F m + ( D F m ) ⊗ D ℓ j + D ℓ j ⊗ ( D F m ) .
whose norm is at most C Δ − 1 C\Delta^{-1} C Δ − 1 on the ball. Since Δ + M ≤ 1 + 2 R \Delta+M\le1+2R Δ + M ≤ 1 + 2 R , this is at most C ( 1 + 2 R ) Δ − 1 ( Δ + M ) − 1 C(1+2R)\Delta^{-1}(\Delta+M)^{-1} C ( 1 + 2 R ) Δ − 1 ( Δ + M ) − 1 . All mixed-coordinate Hessian blocks differentiate each affected factor once and have nonnegative remaining powers. For r = 1 r=1 r = 1 , the final product in equation (C.27) has coefficient zero and U j = E j U_j=E_j U j = E j . Thus the displayed Hessian bound covers s = 2 s=2 s = 2 , r = 1 r=1 r = 1 , and singleton clusters without replacing an individual distance by a maximum in an unhandled negative power. Taylor’s formula therefore yields
∣ U j ( x 1 , … , x r ) − 1 { i = j } ∣ ≤ C r Δ − d r M 2 ( Δ + M ) d r − 2 . (90) \lvert U_j(x_1,\ldots,x_r)-\mathbf{1}\{i=j\}\rvert\le C_r\Delta^{-d_r}M^2(\Delta+M)^{d_r-2}. \tag*{(90)} ∣ U j ( x 1 , … , x r ) − 1 { i = j }∣ ≤ C r Δ − d r M 2 ( Δ + M ) d r − 2 . (90)
The factor r r r from the squared length of the segment is absorbed into C r C_r C r .
Comparison with the positive polynomial. Writing ρ = ∥ x − a c ( x ) ∥ \rho=\lVert x-a_c(x)\rVert ρ = ∥ x − a c ( x )∥ , we have the stronger pointwise bound
P ( x ) ≥ c ρ 2 ( Δ + ρ ) 2 s − 2 . (91) P(x)\ge c\rho^2(\Delta+\rho)^{2s-2}. \tag*{(91)} P ( x ) ≥ c ρ 2 ( Δ + ρ ) 2 s − 2 . (91)
Indeed, for any other true center a i a_i a i , nearest-center assignment and separation give ∥ x − a i ∥ ≥ ρ \lVert x-a_i\rVert\ge\rho ∥ x − a i ∥ ≥ ρ and ∥ x − a i ∥ ≥ Δ / 2 \lVert x-a_i\rVert\ge\Delta/2 ∥ x − a i ∥ ≥ Δ/2 . If ρ < D 0 / 4 \rho< D_0/4 ρ < D 0 /4 , the factors from other clusters are at least 3 D 0 / 4 3D_0/4 3 D 0 /4 , while the s m − 1 s_m-1 s m − 1 remaining factors in the assigned cluster are at least c ( Δ + ρ ) c(\Delta+\rho) c ( Δ + ρ ) . Increasing the exponent from 2 s m − 2 2s_m-2 2 s m − 2 to 2 s − 2 2s-2 2 s − 2 only changes the constant on the bounded ball. If ρ ≥ D 0 / 4 \rho\ge D_0/4 ρ ≥ D 0 /4 , all factors are bounded below by a fixed positive number, which proves the same bound. In the single-cluster case, the first two inequalities prove it directly, without D 0 D_0 D 0 .
Consequently, ∑ ℓ P ( x ℓ ) ≥ c M 2 ( Δ + M ) 2 s − 2 \sum_\ell P(x_\ell) \ge cM^2(\Delta+M)^{2s-2} ∑ ℓ P ( x ℓ ) ≥ c M 2 ( Δ + M ) 2 s − 2 . Dividing (C.28) by this lower bound leaves C r Δ − d r ( Δ + M ) d r − 2 s C_r\Delta^{-d_r}(\Delta+M)^{d_r-2s} C r Δ − d r ( Δ + M ) d r − 2 s . If d r ≥ 2 s d_r \ge2s d r ≥ 2 s , boundedness of M M M gives the multiplier C r Δ − d r C_r\Delta^{-d_r} C r Δ − d r ; otherwise Δ + M ≥ Δ \Delta+M \ge\Delta Δ + M ≥ Δ gives C r Δ − 2 s C_r\Delta^{-2s} C r Δ − 2 s . This proves (C.25).
Finally, if s = 1 s=1 s = 1 , distinct true centers are separated by at least D 0 D_0 D 0 , so Δ \Delta Δ is bounded below. Take U j = ∏ ℓ F κ ( j ℓ ) ( x ℓ ) U_j=\prod_\ell F_{\kappa(j_\ell)}(x_\ell) U j = ∏ ℓ F κ ( j ℓ ) ( x ℓ ) with the original flat cluster selectors, where κ ( j ℓ ) \kappa(j_\ell) κ ( j ℓ ) is the cluster index, not the state label. Their norms and Hessians are uniformly bounded, and P ( x ) ≥ c ρ 2 P(x)\ge c\rho^2 P ( x ) ≥ c ρ 2 . Taylor’s formula proves all the stated bounds, with d r = 1 d_r=1 d r = 1 and β r = 2 \beta_r=2 β r = 2 .
Joint selectors without cluster assumptions
The product of distances from a true center to the other true centers contains at least one distance comparable to the diameter of the true support. Retaining this factor in the interpolation bounds sharpens both joint probability and transport recovery without cluster assumptions.
Lemma C.7 (Diameter-dependent joint selectors). For K ≥ 2 K\ge2 K ≥ 2 arbitrary distinct true centers, put
Δ = Δ s e p , D ⋆ = max i , j ∥ a i − a j ∥ 2 , D j = ∏ i ≠ j ∥ a i − a j ∥ 2 . \Delta=\Delta_{\mathrm{sep}},\qquad D_\star=\max_{i,j}\lVert a_i-a_j\rVert_2,\qquad D_j=\prod_{i\ne j}\lVert a_i-a_j\rVert_2. Δ = Δ sep , D ⋆ = i , j max ∥ a i − a j ∥ 2 , D j = i = j ∏ ∥ a i − a j ∥ 2 . Then Δ ≤ D ⋆ ≤ 2 R \Delta\le D_\star\le2R Δ ≤ D ⋆ ≤ 2 R and D j ≥ ( D ⋆ / 2 ) Δ K − 2 D_j\ge(D_\star/2)\Delta^{K-2} D j ≥ ( D ⋆ /2 ) Δ K − 2 . Let P ( x ) = ∏ i ∥ x − a i ∥ 2 2 P(x)=\prod_i\lVert x-a_i\rVert_2^2 P ( x ) = ∏ i ∥ x − a i ∥ 2 2 , and define
L j ( x ) = ∏ i ≠ j ⟨ x − a i , a j − a i ⟩ ∥ a j − a i ∥ 2 2 , Q j ( x ) = ∏ i ≠ j ∥ x − a i ∥ 2 2 ∥ a j − a i ∥ 2 2 , E j ( x ) = { 1 − ⟨ ∇ Q j ( a j ) , x − a j ⟩ } Q j ( x ) . L_j(x)=\prod_{i\ne j}\frac{\langle x-a_i,a_j-a_i\rangle}{\lVert a_j-a_i\rVert_2^2},\qquad Q_j(x)=\prod_{i\ne j}\frac{\lVert x-a_i\rVert_2^2}{\lVert a_j-a_i\rVert_2^2},\qquad E_j(x)=\{1-\langle\nabla Q_j(a_j),x-a_j\rangle\}Q_j(x). L j ( x ) = i = j ∏ ∥ a j − a i ∥ 2 2 ⟨ x − a i , a j − a i ⟩ , Q j ( x ) = i = j ∏ ∥ a j − a i ∥ 2 2 ∥ x − a i ∥ 2 2 , E j ( x ) = { 1 − ⟨ ∇ Q j ( a j ) , x − a j ⟩} Q j ( x ) . For every x ∈ B d ( 0 , R ) x\in B_d(0,R) x ∈ B d ( 0 , R ) ,
∣ L j ( x ) − 1 { c ( x ) = j } ∣ ≤ C P ( x ) Δ D j , (92) \left|L_j(x)-\mathbf{1}\{c(x)=j\}\right|\le\frac{C\sqrt{P(x)}}{\Delta D_j}, \tag*{(92)} ∣ L j ( x ) − 1 { c ( x ) = j } ∣ ≤ Δ D j C P ( x ) , (92) ∣ E j ( x ) − 1 { c ( x ) = j } ∣ ≤ C P ( x ) Δ 2 D j 2 . (93) \left|E_j(x)-\mathbf{1}\{c(x)=j\}\right|\le\frac{CP(x)}{\Delta^2D_j^2}. \tag*{(93)} ∣ E j ( x ) − 1 { c ( x ) = j } ∣ ≤ Δ 2 D j 2 C P ( x ) . (93) For r ≥ 1 r\ge1 r ≥ 1 , set
d r = r ( K − 1 ) + K , γ K , r = max { 2 K , d r } . d_r=r(K-1)+K,\qquad\gamma_{K,r}=\max\{2K,d_r\}. d r = r ( K − 1 ) + K , γ K , r = max { 2 K , d r } . For each j = ( j 1 , … , j r ) ∈ [ K ] r j=(j_1,\ldots,j_r)\in[K]^r j = ( j 1 , … , j r ) ∈ [ K ] r , the construction in (C.27), with the one-coordinate polynomials above, gives a polynomial U j U_j U j of degree at most d r d_r d r , with the cardinal values and vanishing first derivatives on the true grid, such that
∥ U j ∥ ∞ ≤ C r D ⋆ − 2 Δ − ( γ K , r − 2 ) , (94) \lVert U_j\rVert_\infty\le C_rD_\star^{-2}\Delta^{-(\gamma_{K,r}-2)}, \tag*{(94)} ∥ U j ∥ ∞ ≤ C r D ⋆ − 2 Δ − ( γ K , r − 2 ) , (94) ∣ U j ( x ) − 1 { c ( x ) = j } ∣ ≤ C r D ⋆ − 2 Δ − ( γ K , r − 2 ) ∑ ℓ = 1 r P ( x ℓ ) . (95) \left|U_j(x)-\mathbf{1}\{c(x)=j\}\right|\le C_rD_\star^{-2}\Delta^{-(\gamma_{K,r}-2)}\sum_{\ell=1}^{r}P(x_\ell). \tag*{(95)} ∣ U j ( x ) − 1 { c ( x ) = j } ∣ ≤ C r D ⋆ − 2 Δ − ( γ K , r − 2 ) ℓ = 1 ∑ r P ( x ℓ ) . (95) Here c ( x ) = ( c ( x 1 ) , … , c ( x r ) ) c(x)=(c(x_1),\ldots,c(x_r)) c ( x ) = ( c ( x 1 ) , … , c ( x r )) , and the norm is over B d ( 0 , R ) r B_d(0,R)^r B d ( 0 , R ) r . Constants depend only on K , r , R K,r,R K , r , R .
Proof. We first retain the distance products in the one-coordinate estimates, then compare the largest nearest-center distance with the diameter of the true support.
One-coordinate estimates. For every j j j , at least one true center is at distance at least D ⋆ / 2 D_\star/2 D ⋆ /2 from a j a_j a j , by the triangle inequality applied to a pair attaining the diameter. Every other distance is at least Δ \Delta Δ , proving the bound on D j D_j D j .
Write ρ j = ∥ x − a j ∥ 2 \rho_j=\lVert x-a_j\rVert_2 ρ j = ∥ x − a j ∥ 2 . If c ( x ) ≠ j c(x)\ne j c ( x ) = j , then ρ j ≥ Δ / 2 \rho_j\ge\Delta/2 ρ j ≥ Δ/2 , and
∣ L j ( x ) ∣ ≤ P ( x ) ρ j D j , Q j ( x ) = P ( x ) ρ j 2 D j 2 . \lvert L_j(x)\rvert\le\frac{\sqrt{P(x)}}{\rho_jD_j},\qquad Q_j(x)=\frac{P(x)}{\rho_j^2D_j^2}. ∣ L j ( x )∣ ≤ ρ j D j P ( x ) , Q j ( x ) = ρ j 2 D j 2 P ( x ) .
Also ∥ ∇ Q j ( a j ) ∥ ≤ C / Δ \lVert\nabla Q_j(a_j)\rVert\le C/\Delta ∥ ∇ Q j ( a j )∥ ≤ C /Δ . These inequalities give (C.30)–(C.31) when c ( x ) ≠ j c(x)\ne j c ( x ) = j . If c ( x ) = j c(x)=j c ( x ) = j , nearest-center assignment gives P ( x ) ≥ c ρ j 2 D j 2 P(x)\ge c\rho_j^2D_j^2 P ( x ) ≥ c ρ j 2 D j 2 . For ρ j ≥ Δ / 4 \rho_j\ge\Delta/4 ρ j ≥ Δ/4 , the same formulas bound both the polynomial and indicator terms by the asserted right sides. For ρ j < Δ / 4 \rho_j<\Delta/4 ρ j < Δ/4 , the factors defining L j L_j L j give ∣ L j ( x ) − 1 ∣ ≤ C ρ j / Δ \lvert L_j(x)-1\rvert\le C\rho_j/\Delta ∣ L j ( x ) − 1 ∣ ≤ C ρ j /Δ . The polynomial E j E_j E j is the same flat selector considered in Lemma C.2. Its interpolation values give E j ( a j ) = 1 E_j(a_j)=1 E j ( a j ) = 1 and ∇ E j ( a j ) = 0 \nabla E_j(a_j)=0 ∇ E j ( a j ) = 0 . Thus the constant and linear terms cancel when Taylor’s formula is applied to E j ( x ) − 1 E_j(x)-1 E j ( x ) − 1 . The local Taylor estimate (C.14), proved using the Hessian bound (C.11), gives
∣ E j ( x ) − 1 ∣ ≤ C ρ j 2 / Δ 2 . \lvert E_j(x)-1\rvert\le C\rho_j^2/\Delta^2. ∣ E j ( x ) − 1 ∣ ≤ C ρ j 2 / Δ 2 .
To obtain the required comparison with P ( x ) P(x) P ( x ) , use the nearest-center bound P ( x ) ≥ c ρ j 2 D j 2 P(x)\ge c\rho_j^2D_j^2 P ( x ) ≥ c ρ j 2 D j 2 , which implies
ρ j 2 ≤ C P ( x ) D j 2 . \rho_j^2\le\frac{CP(x)}{D_j^2}. ρ j 2 ≤ D j 2 C P ( x ) .
Substitution yields
∣ E j ( x ) − 1 ∣ ≤ C P ( x ) Δ 2 D j 2 . \lvert E_j(x)-1\rvert\le\frac{CP(x)}{\Delta^2D_j^2}. ∣ E j ( x ) − 1 ∣ ≤ Δ 2 D j 2 C P ( x ) .
Since c ( x ) = j c(x)=j c ( x ) = j , this is exactly (C.31) in the remaining case. These arguments also cover a tie in nearest-center assignment.
The cardinal values and cancellation of first derivatives of U j U_j U j follow as in Lemma C.6; that cancellation uses only the equal cardinal values of L j L_j L j , E j E_j E j and the vanishing gradients of E j E_j E j . For r = 1 r=1 r = 1 , U j = E j U_j=E_j U j = E j , and (C.31) already proves (C.33). We henceforth take r ≥ 2 r\ge2 r ≥ 2 , so d r ≥ 2 K d_r\ge2K d r ≥ 2 K and γ K , r = d r \gamma_{K,r}=d_r γ K , r = d r .
Points within the true-support diameter. Put M = max ℓ ∥ x ℓ − a c ( x ℓ ) ∥ 2 M=\max_\ell\lVert x_\ell-a_{c(x_\ell)}\rVert_2 M = max ℓ ∥ x ℓ − a c ( x ℓ ) ∥ 2 and suppose M ≤ D ⋆ M\le D_\star M ≤ D ⋆ . Every distance from x ℓ x_\ell x ℓ to a true center is then at most 2 D ⋆ 2D_\star 2 D ⋆ . Cauchy–Schwarz and the lower bound on D j D_j D j give
∣ L j ( x ℓ ) ∣ ≤ C ( D ⋆ / Δ ) K − 2 = : B ⋆ , B ⋆ ≥ 1 , \lvert L_j(x_\ell)\rvert\le C(D_\star/\Delta)^{K-2}=:B_\star,\qquad B_\star\ge1, ∣ L j ( x ℓ )∣ ≤ C ( D ⋆ /Δ ) K − 2 =: B ⋆ , B ⋆ ≥ 1 ,
after increasing the constant. Write v ℓ = 1 { c ( x ℓ ) = j ℓ } v_\ell=\mathbf{1}\{c(x_\ell)=j_\ell\} v ℓ = 1 { c ( x ℓ ) = j ℓ } , ε ℓ = L j ℓ ( x ℓ ) − v ℓ \varepsilon_\ell=L_{j_\ell}(x_\ell)-v_\ell ε ℓ = L j ℓ ( x ℓ ) − v ℓ , and η ℓ = E j ℓ ( x ℓ ) − v ℓ \eta_\ell=E_{j_\ell}(x_\ell)-v_\ell η ℓ = E j ℓ ( x ℓ ) − v ℓ . Expanding (C.27), the constant term is ∏ ℓ v ℓ \prod_\ell v_\ell ∏ ℓ v ℓ and all terms containing exactly one ε ℓ \varepsilon_\ell ε ℓ cancel. Terms involving an η ℓ \eta_\ell η ℓ are bounded by ∣ η ℓ ∣ B ⋆ r − 1 \lvert\eta_\ell\rvert B_\star^{r-1} ∣ η ℓ ∣ B ⋆ r − 1 ; the remaining terms contain at least two ε \varepsilon ε factors and are bounded by C r B ⋆ r − 2 ∑ ℓ < u ∣ ε ℓ ε u ∣ C_rB_\star^{r-2}\sum_{\ell<u}\lvert\varepsilon_\ell\varepsilon_u\rvert C r B ⋆ r − 2 ∑ ℓ < u ∣ ε ℓ ε u ∣ . By (C.30)–(C.31) and 2 P ( x ℓ ) P ( x u ) ≤ P ( x ℓ ) + P ( x u ) 2\sqrt{P(x_\ell)P(x_u)}\le P(x_\ell)+P(x_u) 2 P ( x ℓ ) P ( x u ) ≤ P ( x ℓ ) + P ( x u ) ,
∣ U j ( x ) − 1 { c ( x ) = j } ∣ ≤ C r D ⋆ − 2 Δ − ( 2 K − 2 ) ( D ⋆ / Δ ) ( r − 1 ) ( K − 2 ) ∑ ℓ P ( x ℓ ) ≤ C r D ⋆ − 2 Δ − ( d r − 2 ) ∑ ℓ P ( x ℓ ) . \begin{aligned}
\lvert U_j(x)-\mathbf{1}\{c(x)=j\}\rvert\le C_rD_\star^{-2}\Delta^{-(2K-2)}(D_\star/\Delta)^{(r-1)(K-2)}\sum_\ell P(x_\ell) \\
&\le C_rD_\star^{-2}\Delta^{-(d_r-2)}\sum_\ell P(x_\ell).
\end{aligned} ∣ U j ( x ) − 1 { c ( x ) = j }∣ ≤ C r D ⋆ − 2 Δ − ( 2 K − 2 ) ( D ⋆ /Δ ) ( r − 1 ) ( K − 2 ) ℓ ∑ P ( x ℓ ) ≤ C r D ⋆ − 2 Δ − ( d r − 2 ) ℓ ∑ P ( x ℓ ) .
For the last inequality, the ratio of the first multiplier to the second is D ⋆ ( r − 1 ) ( K − 2 ) Δ r − 2 D_\star^{(r-1)(K-2)}\Delta^{r-2} D ⋆ ( r − 1 ) ( K − 2 ) Δ r − 2 , which is bounded by a constant depending only on K K K , r r r , R R R .
Points beyond the true-support diameter. Suppose M > D ⋆ M>D_\star M > D ⋆ . Every distance from a selected x ℓ x_\ell x ℓ to a true center is at most 2 M 2M 2 M . The product formulas and ∥ ∇ Q j ( a j ) ∥ ≤ C / Δ \lVert\nabla Q_j(a_j)\rVert\le C/\Delta ∥ ∇ Q j ( a j )∥ ≤ C /Δ therefore give
∣ L j ( x ℓ ) ∣ ≤ C M K − 1 D ⋆ Δ K − 2 , ∣ E j ( x ℓ ) ∣ ≤ C M 2 K − 1 D ⋆ 2 Δ 2 K − 3 . \lvert L_j(x_\ell)\rvert\le\frac{CM^{K-1}}{D_\star\Delta^{K-2}},\qquad\lvert E_j(x_\ell)\rvert\le\frac{CM^{2K-1}}{D_\star^2\Delta^{2K-3}}. ∣ L j ( x ℓ )∣ ≤ D ⋆ Δ K − 2 C M K − 1 , ∣ E j ( x ℓ )∣ ≤ D ⋆ 2 Δ 2 K − 3 C M 2 K − 1 .
Consequently,
∣ U j ( x ) ∣ ≤ C r M d r D ⋆ r + 1 Δ d r − r − 1 . \lvert U_j(x)\rvert\le\frac{C_rM^{d_r}}{D_\star^{r+1}\Delta^{d_r-r-1}}. ∣ U j ( x )∣ ≤ D ⋆ r + 1 Δ d r − r − 1 C r M d r .
The last product in (C.27) is absorbed in this bound because D ⋆ Δ K − 1 / M K ≤ 1 D_\star\Delta^{K-1}/M^K\le1 D ⋆ Δ K − 1 / M K ≤ 1 . For a coordinate attaining M M M , all its distances to true centers are at least M M M , so ∑ ℓ P ( x ℓ ) ≥ M 2 K \sum_\ell P(x_\ell)\ge M^{2K} ∑ ℓ P ( x ℓ ) ≥ M 2 K . Dividing the preceding bound by this sum and using M ≤ 2 R M\le2R M ≤ 2 R , d r ≥ 2 K d_r\ge2K d r ≥ 2 K , and Δ ≤ D ⋆ \Delta\le D_\star Δ ≤ D ⋆ yields
∣ U j ( x ) ∣ ∑ ℓ P ( x ℓ ) ≤ C r D ⋆ − 2 Δ − ( d r − 2 ) . \frac{\lvert U_j(x)\rvert}{\sum_\ell P(x_\ell)}\le C_rD_\star^{-2}\Delta^{-(d_r-2)}. ∑ ℓ P ( x ℓ ) ∣ U j ( x )∣ ≤ C r D ⋆ − 2 Δ − ( d r − 2 ) .
The indicator term satisfies the same bound, since D ⋆ − 2 Δ − ( d r − 2 ) M 2 K ≥ 1 D_\star^{-2}\Delta^{-(d_r-2)}M^{2K}\ge1 D ⋆ − 2 Δ − ( d r − 2 ) M 2 K ≥ 1 when M > D ⋆ ≥ Δ M>D_\star\ge\Delta M > D ⋆ ≥ Δ and Δ ≤ 1 \Delta\le1 Δ ≤ 1 . This proves (C.33). Finally, P P P is uniformly bounded on the ball and D ⋆ ≤ 2 R D_\star\le2R D ⋆ ≤ 2 R , so (C.33) also gives (C.32).
Exact interpolation at paired centers
The global selecting polynomials constructed above have the required values at true centers, but their values at fitted centers introduce approximation errors. For the matched bounds under exact specification, we assume a pairing satisfying the proximity condition below and impose the required values on both supports. We construct these polynomials by first making them vanish at every other true–fitted pair and then adjusting their values at the selected pair. This construction also covers pairs whose two centers coincide.
A polynomial that vanishes at every other pair
Assume that the fitted centers are labeled so that ∥ b j − a j ∥ 2 ≤ Δ s e p / 4 \lVert b_j-a_j\rVert_2\le\Delta_{\mathrm{sep}}/4 ∥ b j − a j ∥ 2 ≤ Δ sep /4 for every j ∈ [ K ] j\in[K] j ∈ [ K ] . Both a j a_j a j and b j b_j b j belong to B d ( 0 , R ) B_d(0,R) B d ( 0 , R ) . For j ≠ ℓ j\ne\ell j = ℓ , write d j ℓ = ∥ a j − a ℓ ∥ 2 d_{j\ell}=\lVert a_j-a_\ell\rVert_2 d j ℓ = ∥ a j − a ℓ ∥ 2 and define
u j ℓ = a j − a ℓ d j ℓ , R j ( x ) = ∏ ℓ ≠ j ⟨ x − a ℓ , u j ℓ ⟩ ⟨ x − b ℓ , u j ℓ ⟩ d j ℓ 2 . (96) u_{j\ell}=\frac{a_j-a_\ell}{d_{j\ell}},\qquad R_j(x)=\prod_{\ell\ne j}\frac{\langle x-a_\ell,u_{j\ell}\rangle\langle x-b_\ell,u_{j\ell}\rangle}{d_{j\ell}^2}. \tag*{(96)} u j ℓ = d j ℓ a j − a ℓ , R j ( x ) = ℓ = j ∏ d j ℓ 2 ⟨ x − a ℓ , u j ℓ ⟩ ⟨ x − b ℓ , u j ℓ ⟩ . (96)
Each factor indexed by ℓ \ell ℓ introduces a zero at a ℓ a_\ell a ℓ and a zero at b ℓ b_\ell b ℓ . We next verify that this product remains bounded away from zero on the selected segment [ a j , b j ] [a_j,b_j] [ a j , b j ] , so that its values there can be inverted.
Lemma C.8 (Conditioning of the paired product). The polynomial R j R_j R j has degree 2 K − 2 2K-2 2 K − 2 and satisfies R j ( a ℓ ) = R j ( b ℓ ) = 0 R_j(a_\ell)=R_j(b_\ell)=0 R j ( a ℓ ) = R j ( b ℓ ) = 0 for every ℓ ≠ j \ell\ne j ℓ = j . On the segment [ a j , b j ] [a_j,b_j] [ a j , b j ] ,
0 < c ≤ R j ( x ) ≤ C , ∥ ∇ ( 1 / R j ) ( x ) ∥ 2 ≤ C Δ s e p − 1 . (97) 0<c\le R_j(x)\le C,\qquad\lVert\nabla(1/R_j)(x)\rVert_2\le C\Delta_{\mathrm{sep}}^{-1}. \tag*{(97)} 0 < c ≤ R j ( x ) ≤ C , ∥ ∇ ( 1/ R j ) ( x ) ∥ 2 ≤ C Δ sep − 1 . (97) In the structured regimes,
∥ R j ∥ ∞ , B d ( 0 , R ) ≤ C Δ s e p − ( 2 s max − 2 ) . (98) \lVert R_j\rVert_{\infty,B_d(0,R)}\le C\Delta_{\mathrm{sep}}^{-(2s_{\max}-2)}. \tag*{(98)} ∥ R j ∥ ∞ , B d ( 0 , R ) ≤ C Δ sep − ( 2 s m a x − 2 ) . (98) Without cluster assumptions, the same supremum bound holds with s max = K s_{\max}=K s m a x = K .
Proof. Fix j ∈ [ K ] j \in[K] j ∈ [ K ] . Distinctness of the true centers gives d j ℓ > 0 d_{j\ell} > 0 d j ℓ > 0 , and the definition of u j ℓ u_{j\ell} u j ℓ gives ∥ u j ℓ ∥ 2 = 1 \lVert u_{j\ell}\rVert_2 = 1 ∥ u j ℓ ∥ 2 = 1 . Every factor in Equation (C.34) is therefore a well-defined polynomial of degree two. Their product has degree 2 K − 2 2K - 2 2 K − 2 . At the centers belonging to another pair,
⟨ a ℓ − a ℓ , u j ℓ ⟩ = 0 , ⟨ b ℓ − b ℓ , u j ℓ ⟩ = 0 , \langle a_\ell-a_\ell,u_{j\ell}\rangle=0,\qquad\langle b_\ell-b_\ell,u_{j\ell}\rangle=0, ⟨ a ℓ − a ℓ , u j ℓ ⟩ = 0 , ⟨ b ℓ − b ℓ , u j ℓ ⟩ = 0 ,
so R j ( a ℓ ) = R j ( b ℓ ) = 0 R_j(a_\ell)=R_j(b_\ell)=0 R j ( a ℓ ) = R j ( b ℓ ) = 0 . If a ℓ = b ℓ a_\ell=b_\ell a ℓ = b ℓ , the two linear factors coincide, and the resulting squared factor still vanishes at that common center.
Bounds on the selected segment. Fix x ∈ [ a j , b j ] x \in[a_j,b_j] x ∈ [ a j , b j ] , and write x = a j + t ( b j − a j ) x=a_j+t(b_j-a_j) x = a j + t ( b j − a j ) for some t ∈ [ 0 , 1 ] t \in[0,1] t ∈ [ 0 , 1 ] . Then
∥ x − a j ∥ 2 = t ∥ b j − a j ∥ 2 ≤ Δ s e p 4 (because 0 ≤ t ≤ 1 ). (99) \lVert x-a_j\rVert_2=t\lVert b_j-a_j\rVert_2\leq\frac{\Delta_{\mathrm{sep}}}{4}\qquad\text{(because }0\leq t\leq1\text{).} \tag*{(99)} ∥ x − a j ∥ 2 = t ∥ b j − a j ∥ 2 ≤ 4 Δ sep (because 0 ≤ t ≤ 1 ). (99)
For ℓ ≠ j \ell\ne j ℓ = j , denote the two linear factors by
L j ℓ ( x ) = ⟨ x − a ℓ , u j ℓ ⟩ , M j ℓ ( x ) = ⟨ x − b ℓ , u j ℓ ⟩ . L_{j\ell}(x)=\langle x-a_\ell,u_{j\ell}\rangle,\qquad M_{j\ell}(x)=\langle x-b_\ell,u_{j\ell}\rangle. L j ℓ ( x ) = ⟨ x − a ℓ , u j ℓ ⟩ , M j ℓ ( x ) = ⟨ x − b ℓ , u j ℓ ⟩ .
To bound the first factor, separate the displacement from a j a_j a j :
L j ℓ ( x ) = ⟨ a j − a ℓ , u j ℓ ⟩ + ⟨ x − a j , u j ℓ ⟩ = d j ℓ + ⟨ x − a j , u j ℓ ⟩ (by the definition of u j ℓ ). \begin{aligned}
L_{j\ell}(x)&=\langle a_j-a_\ell,u_{j\ell}\rangle+\langle x-a_j,u_{j\ell}\rangle\\
&=d_{j\ell}+\langle x-a_j,u_{j\ell}\rangle\qquad\text{(by the definition of }u_{j\ell}\text{).}
\end{aligned} L j ℓ ( x ) = ⟨ a j − a ℓ , u j ℓ ⟩ + ⟨ x − a j , u j ℓ ⟩ = d j ℓ + ⟨ x − a j , u j ℓ ⟩ (by the definition of u j ℓ ).
The second term satisfies
∣ ⟨ x − a j , u j ℓ ⟩ ∣ ≤ ∥ x − a j ∥ 2 ∥ u j ℓ ∥ 2 (by Cauchy–Schwarz) ≤ Δ s e p 4 (because ∥ x − a j ∥ 2 ≤ Δ s e p 4 by (C.37) and ∥ u j ℓ ∥ 2 = 1 by its definition) ≤ d j ℓ 4 (because d j ℓ ≥ Δ s e p ). \begin{aligned}
\lvert\langle x-a_j,u_{j\ell}\rangle\rvert
&\leq\lVert x-a_j\rVert_2\lVert u_{j\ell}\rVert_2\qquad\text{(by Cauchy--Schwarz)}\\
&\leq\frac{\Delta_{\mathrm{sep}}}{4}\qquad\text{(because }\lVert x-a_j\rVert_2\leq\frac{\Delta_{\mathrm{sep}}}{4}\text{ by (C.37) and }\lVert u_{j\ell}\rVert_2=1\text{ by its definition)}\\
&\leq\frac{d_{j\ell}}{4}\qquad\text{(because }d_{j\ell}\geq\Delta_{\mathrm{sep}}\text{).}
\end{aligned} ∣⟨ x − a j , u j ℓ ⟩∣ ≤ ∥ x − a j ∥ 2 ∥ u j ℓ ∥ 2 (by Cauchy–Schwarz) ≤ 4 Δ sep (because ∥ x − a j ∥ 2 ≤ 4 Δ sep by (C.37) and ∥ u j ℓ ∥ 2 = 1 by its definition) ≤ 4 d j ℓ (because d j ℓ ≥ Δ sep ).
Consequently,
3 4 d j ℓ ≤ L j ℓ ( x ) ≤ 5 4 d j ℓ . (100) \frac{3}{4}d_{j\ell}\leq L_{j\ell}(x)\leq\frac{5}{4}d_{j\ell}. \tag*{(100)} 4 3 d j ℓ ≤ L j ℓ ( x ) ≤ 4 5 d j ℓ . (100)
For the second factor, the displacement of b ℓ b_\ell b ℓ from its true center must also be included:
M j ℓ ( x ) = d j ℓ + ⟨ x − a j , u j ℓ ⟩ − ⟨ b ℓ − a ℓ , u j ℓ ⟩ . M_{j\ell}(x)=d_{j\ell}+\langle x-a_j,u_{j\ell}\rangle-\langle b_\ell-a_\ell,u_{j\ell}\rangle. M j ℓ ( x ) = d j ℓ + ⟨ x − a j , u j ℓ ⟩ − ⟨ b ℓ − a ℓ , u j ℓ ⟩ .
Its deviation from d j ℓ d_{j\ell} d j ℓ is bounded by
∣ M j ℓ ( x ) − d j ℓ ∣ ≤ ∣ ⟨ x − a j , u j ℓ ⟩ ∣ + ∣ ⟨ b ℓ − a ℓ , u j ℓ ⟩ ∣ (by the triangle inequality) ≤ ∥ x − a j ∥ 2 + ∥ b ℓ − a ℓ ∥ 2 (by Cauchy–Schwarz) ≤ Δ s e p 4 + Δ s e p 4 ≤ d j ℓ 2 . \begin{aligned}
\lvert M_{j\ell}(x)-d_{j\ell}\rvert
&\leq\lvert\langle x-a_j,u_{j\ell}\rangle\rvert+\lvert\langle b_\ell-a_\ell,u_{j\ell}\rangle\rvert\qquad\text{(by the triangle inequality)}\\
&\leq\lVert x-a_j\rVert_2+\lVert b_\ell-a_\ell\rVert_2\qquad\text{(by Cauchy--Schwarz)}\\
&\leq\frac{\Delta_{\mathrm{sep}}}{4}+\frac{\Delta_{\mathrm{sep}}}{4}\\
&\leq\frac{d_{j\ell}}{2}.
\end{aligned} ∣ M j ℓ ( x ) − d j ℓ ∣ ≤ ∣⟨ x − a j , u j ℓ ⟩∣ + ∣⟨ b ℓ − a ℓ , u j ℓ ⟩∣ (by the triangle inequality) ≤ ∥ x − a j ∥ 2 + ∥ b ℓ − a ℓ ∥ 2 (by Cauchy–Schwarz) ≤ 4 Δ sep + 4 Δ sep ≤ 2 d j ℓ .
It follows that
1 2 d j ℓ ≤ M j ℓ ( x ) ≤ 3 2 d j ℓ . (101) \frac{1}{2}d_{j\ell}\leq M_{j\ell}(x)\leq\frac{3}{2}d_{j\ell}. \tag*{(101)} 2 1 d j ℓ ≤ M j ℓ ( x ) ≤ 2 3 d j ℓ . (101)
Both linear factors are positive on the entire selected segment. We may therefore multiply their lower and upper bounds to obtain
3 8 = ( 3 d j ℓ / 4 ) ( d j ℓ / 2 ) d j ℓ 2 ≤ L j ℓ ( x ) M j ℓ ( x ) d j ℓ 2 ≤ ( 5 d j ℓ / 4 ) ( 3 d j ℓ / 2 ) d j ℓ 2 = 15 8 . \frac{3}{8}=\frac{(3d_{j\ell}/4)(d_{j\ell}/2)}{d_{j\ell}^{2}}\le\frac{L_{j\ell}(x)M_{j\ell}(x)}{d_{j\ell}^{2}}\le\frac{(5d_{j\ell}/4)(3d_{j\ell}/2)}{d_{j\ell}^{2}}=\frac{15}{8}. 8 3 = d j ℓ 2 ( 3 d j ℓ /4 ) ( d j ℓ /2 ) ≤ d j ℓ 2 L j ℓ ( x ) M j ℓ ( x ) ≤ d j ℓ 2 ( 5 d j ℓ /4 ) ( 3 d j ℓ /2 ) = 8 15 .
There are K − 1 K-1 K − 1 factors in R j R_j R j , so multiplying these inequalities over ℓ ≠ j \ell\ne j ℓ = j gives
( 3 8 ) K − 1 ≤ R j ( x ) ≤ ( 15 8 ) K − 1 , x ∈ [ a j , b j ] . (102) \left(\frac{3}{8}\right)^{K-1}\le R_j(x)\le\left(\frac{15}{8}\right)^{K-1},\qquad x\in[a_j,b_j]. \tag*{(102)} ( 8 3 ) K − 1 ≤ R j ( x ) ≤ ( 8 15 ) K − 1 , x ∈ [ a j , b j ] . (102)
In particular, R j ( a j ) R_j(a_j) R j ( a j ) and R j ( b j ) R_j(b_j) R j ( b j ) are strictly positive.
Derivative of the reciprocal. The positivity of all linear factors at a segment point persists in a neighborhood of that point. Thus 1 / R j 1/R_j 1/ R j is differentiable near the segment. Since ∇ L j ℓ = ∇ M j ℓ = u j ℓ \nabla L_{j\ell}=\nabla M_{j\ell}=u_{j\ell} ∇ L j ℓ = ∇ M j ℓ = u j ℓ , the product rule gives
∇ R j ( x ) = R j ( x ) ∑ ℓ ≠ j { u j ℓ L j ℓ ( x ) + u j ℓ M j ℓ ( x ) } . \nabla R_j(x)=R_j(x)\sum_{\ell\ne j}\left\{\frac{u_{j\ell}}{L_{j\ell}(x)}+\frac{u_{j\ell}}{M_{j\ell}(x)}\right\}. ∇ R j ( x ) = R j ( x ) ℓ = j ∑ { L j ℓ ( x ) u j ℓ + M j ℓ ( x ) u j ℓ } .
Applying the derivative formula for a reciprocal, we have
∇ ( 1 / R j ) ( x ) = − ∇ R j ( x ) R j ( x ) 2 = − 1 R j ( x ) ∑ ℓ ≠ j { u j ℓ L j ℓ ( x ) + u j ℓ M j ℓ ( x ) } . \nabla(1/R_j)(x)=-\frac{\nabla R_j(x)}{R_j(x)^2}=-\frac{1}{R_j(x)}\sum_{\ell\ne j}\left\{\frac{u_{j\ell}}{L_{j\ell}(x)}+\frac{u_{j\ell}}{M_{j\ell}(x)}\right\}. ∇ ( 1/ R j ) ( x ) = − R j ( x ) 2 ∇ R j ( x ) = − R j ( x ) 1 ℓ = j ∑ { L j ℓ ( x ) u j ℓ + M j ℓ ( x ) u j ℓ } .
Taking norms, we have:
∥ ∇ ( 1 / R j ) ( x ) ∥ 2 ≤ 1 R j ( x ) ∑ ℓ ≠ j { 1 L j ℓ ( x ) + 1 M j ℓ ( x ) } (because ∥ u j ℓ ∥ 2 = 1 ) ≤ 1 R j ( x ) ∑ ℓ ≠ j { 4 3 d j ℓ + 2 d j ℓ } (by xref‘(C.38)‘ and xref‘(C.39)‘ = 10 3 R j ( x ) ∑ ℓ ≠ j 1 d j ℓ ≤ 10 ( K − 1 ) 3 R j ( x ) Δ s e p (because d j ℓ ≥ Δ s e p ) ≤ ( 8 3 ) K − 1 10 ( K − 1 ) 3 Δ s e p (by Equation xref‘(C.40)‘). \begin{aligned}
\left\|\nabla(1/R_j)(x)\right\|_2\le\frac{1}{R_j(x)}\sum_{\ell\ne j}\left\{\frac{1}{L_{j\ell}(x)}+\frac{1}{M_{j\ell}(x)}\right\}\qquad\text{(because }\|u_{j\ell}\|_2=1\text{)} \\
&\le\frac{1}{R_j(x)}\sum_{\ell\ne j}\left\{\frac{4}{3d_{j\ell}}+\frac{2}{d_{j\ell}}\right\}\qquad\text{(by {xref}`(C.38)` and {xref}`(C.39)`} \\
&=\frac{10}{3R_j(x)}\sum_{\ell\ne j}\frac{1}{d_{j\ell}} \\
&\le\frac{10(K-1)}{3R_j(x)\Delta_{\mathrm{sep}}}\qquad\text{(because }d_{j\ell}\ge\Delta_{\mathrm{sep}}\text{)} \\
&\le\left(\frac{8}{3}\right)^{K-1}\frac{10(K-1)}{3\Delta_{\mathrm{sep}}}\qquad\text{(by Equation {xref}`(C.40)`).}
\end{aligned} ∥ ∇ ( 1/ R j ) ( x ) ∥ 2 ≤ R j ( x ) 1 ℓ = j ∑ { L j ℓ ( x ) 1 + M j ℓ ( x ) 1 } (because ∥ u j ℓ ∥ 2 = 1 ) ≤ R j ( x ) 1 ℓ = j ∑ { 3 d j ℓ 4 + d j ℓ 2 } (by xref ‘(C.38)‘ and xref ‘(C.39)‘ = 3 R j ( x ) 10 ℓ = j ∑ d j ℓ 1 ≤ 3 R j ( x ) Δ sep 10 ( K − 1 ) (because d j ℓ ≥ Δ sep ) ≤ ( 3 8 ) K − 1 3 Δ sep 10 ( K − 1 ) (by Equation xref ‘(C.40)‘).
This proves the reciprocal derivative bound. Its denominator involves distances between distinct true centers, and does not involve the distance ∥ b j − a j ∥ 2 \|b_j-a_j\|_2 ∥ b j − a j ∥ 2 within the selected pair.
Supremum bound on the full ball. For an arbitrary x ∈ B d ( 0 , R ) x\in B_d(0,R) x ∈ B d ( 0 , R ) , the signs of the linear factors need not be positive. Their absolute values nevertheless satisfy
∣ L j ℓ ( x ) ∣ ≤ ∥ x − a ℓ ∥ 2 (by Cauchy–Schwarz) ≤ ∥ x ∥ 2 + ∥ a ℓ ∥ 2 ≤ 2 R , \begin{aligned}
|L_{j\ell}(x)|\le\|x-a_\ell\|_2\qquad\text{(by Cauchy--Schwarz)} \\
&\le\|x\|_2+\|a_\ell\|_2\le2R,
\end{aligned} ∣ L j ℓ ( x ) ∣ ≤ ∥ x − a ℓ ∥ 2 (by Cauchy–Schwarz) ≤ ∥ x ∥ 2 + ∥ a ℓ ∥ 2 ≤ 2 R ,
and the same calculation gives ∣ M j ℓ ( x ) ∣ ≤ 2 R |M_{j\ell}(x)|\le2R ∣ M j ℓ ( x ) ∣ ≤ 2 R . Hence, writing D j = ∏ ℓ ≠ j d j ℓ 2 D_j=\prod_{\ell\ne j}d_{j\ell}^{2} D j = ∏ ℓ = j d j ℓ 2 , we obtain
∣ R j ( x ) ∣ = ∏ ℓ ≠ j ∣ L j ℓ ( x ) ∣ ∣ M j ℓ ( x ) ∣ d j ℓ 2 ≤ ( 2 R ) 2 K − 2 D j . |R_j(x)|=\prod_{\ell\ne j}\frac{|L_{j\ell}(x)||M_{j\ell}(x)|}{d_{j\ell}^{2}}\le\frac{(2R)^{2K-2}}{D_j}. ∣ R j ( x ) ∣ = ℓ = j ∏ d j ℓ 2 ∣ L j ℓ ( x ) ∣∣ M j ℓ ( x ) ∣ ≤ D j ( 2 R ) 2 K − 2 .
In the structured regimes, let C j C_j C j be the cluster containing j j j , and put s j = ∣ C j ∣ ≤ s max s_j = \lvert C_j\rvert\le s_{\max} s j = ∣ C j ∣ ≤ s m a x . Distances within this cluster are at least Δ s e p \Delta_{\mathrm{sep}} Δ sep , while distances to other clusters are at least D 0 D_0 D 0 . With λ 0 = min { 1 , D 0 } \lambda_0 = \min\{1,D_0\} λ 0 = min { 1 , D 0 } , the denominator satisfies
D j = ∏ ℓ ∈ C j ∖ { j } d j ℓ 2 ∏ ℓ ∉ C j d j ℓ 2 ≥ Δ s e p 2 ( s j − 1 ) D 0 2 ( K − s j ) (by the distance bounds) ≥ Δ s e p 2 ( s j − 1 ) λ 0 2 ( K − s j ) (because D 0 ≥ λ 0 ) ≥ Δ s e p 2 ( s j − 1 ) λ 0 2 ( K − 1 ) (because 0 < λ 0 ≤ 1 ) ≥ Δ s e p 2 ( s max − 1 ) λ 0 2 ( K − 1 ) (because s j ≤ s max and Δ s e p ≤ 1 ) . \begin{aligned}
D_j = \prod_{\ell\in C_j\setminus\{j\}} d_{j\ell}^2 \prod_{\ell\notin C_j} d_{j\ell}^2 \\
&\ge\Delta_{\mathrm{sep}}^{2(s_j-1)}D_0^{2(K-s_j)}
\qquad\text{(by the distance bounds)} \\
&\ge\Delta_{\mathrm{sep}}^{2(s_j-1)}\lambda_0^{2(K-s_j)}\qquad\text{(because }D_0\ge\lambda_0\text{)} \\
&\ge\Delta_{\mathrm{sep}}^{2(s_j-1)}\lambda_0^{2(K-1)}\qquad\text{(because }0<\lambda_0\le1\text{)} \\
&\ge\Delta_{\mathrm{sep}}^{2(s_{\max}-1)}\lambda_0^{2(K-1)}\qquad\text{(because }s_j\le s_{\max}\text{ and }\Delta_{\mathrm{sep}}\le1\text{)}.
\end{aligned} D j = ℓ ∈ C j ∖ { j } ∏ d j ℓ 2 ℓ ∈ / C j ∏ d j ℓ 2 ≥ Δ sep 2 ( s j − 1 ) D 0 2 ( K − s j ) (by the distance bounds) ≥ Δ sep 2 ( s j − 1 ) λ 0 2 ( K − s j ) (because D 0 ≥ λ 0 ) ≥ Δ sep 2 ( s j − 1 ) λ 0 2 ( K − 1 ) (because 0 < λ 0 ≤ 1 ) ≥ Δ sep 2 ( s m a x − 1 ) λ 0 2 ( K − 1 ) (because s j ≤ s m a x and Δ sep ≤ 1 ) .
Substitution into the bound for ∣ R j ( x ) ∣ \lvert R_j(x)\rvert ∣ R j ( x )∣ yields
∥ R j ∥ ∞ , B d ( 0 , R ) ≤ ( 2 R ) 2 K − 2 λ 0 − 2 ( K − 1 ) Δ s e p − ( 2 s max − 2 ) ≤ C Δ s e p − ( 2 s max − 2 ) . \lVert R_j\rVert_{\infty,B_d(0,R)} \le(2R)^{2K-2}\lambda_0^{-2(K-1)}\Delta_{\mathrm{sep}}^{-(2s_{\max}-2)} \le C\Delta_{\mathrm{sep}}^{-(2s_{\max}-2)}. ∥ R j ∥ ∞ , B d ( 0 , R ) ≤ ( 2 R ) 2 K − 2 λ 0 − 2 ( K − 1 ) Δ sep − ( 2 s m a x − 2 ) ≤ C Δ sep − ( 2 s m a x − 2 ) .
Without clustering, every factor in D j D_j D j is at least Δ s e p 2 \Delta_{\mathrm{sep}}^2 Δ sep 2 , so
D j ≥ Δ s e p 2 K − 2 . D_j \ge\Delta_{\mathrm{sep}}^{2K-2}. D j ≥ Δ sep 2 K − 2 .
The same numerator bound then proves the assertion with s max = K s_{\max}=K s m a x = K .
Exact mass and mean extraction
The product R j R_j R j already removes every pair other than the selected pair ( a j , b j ) (a_j,b_j) ( a j , b j ) . To extract its probability, we adjust the remaining factor so that the resulting polynomial equals one at both selected centers. For displacement, we instead require value zero at a j a_j a j and value ∥ b j − a j ∥ 2 \lVert b_j-a_j\rVert_2 ∥ b j − a j ∥ 2 at b j b_j b j . Write h j = ∥ b j − a j ∥ 2 h_j=\lVert b_j-a_j\rVert_2 h j = ∥ b j − a j ∥ 2 and α j = R j ( a j ) − 1 \alpha_j=R_j(a_j)^{-1} α j = R j ( a j ) − 1 . When h j > 0 h_j>0 h j > 0 , define
w j = b j − a j h j , γ j = R j ( b j ) − 1 − R j ( a j ) − 1 h j . w_j=\frac{b_j-a_j}{h_j},\qquad\gamma_j=\frac{R_j(b_j)^{-1}-R_j(a_j)^{-1}}{h_j}. w j = h j b j − a j , γ j = h j R j ( b j ) − 1 − R j ( a j ) − 1 .
When h j = 0 h_j=0 h j = 0 , choose any unit vector w j w_j w j and set
γ j = ⟨ ∇ ( 1 / R j ) ( a j ) , w j ⟩ . \gamma_j=\langle\nabla(1/R_j)(a_j),w_j\rangle. γ j = ⟨ ∇ ( 1/ R j ) ( a j ) , w j ⟩ .
In both cases, define
E ~ j ( x ) = R j ( x ) { α j + γ j ⟨ w j , x − a j ⟩ } . (103) \widetilde E_j(x)=R_j(x)\{\alpha_j+\gamma_j\langle w_j,x-a_j\rangle\}. \tag*{(103)} E j ( x ) = R j ( x ) { α j + γ j ⟨ w j , x − a j ⟩} . (103)
and
H ~ j ( x ) = { R j ( x ) R j ( b j ) ⟨ w j , x − a j ⟩ , h j > 0 , 0 , h j = 0. (104) \widetilde H_j(x)=
\begin{cases}
\dfrac{R_j(x)}{R_j(b_j)}\langle w_j,x-a_j\rangle, & h_j>0,\\
0, & h_j=0.
\end{cases} \tag*{(104)} H j ( x ) = ⎩ ⎨ ⎧ R j ( b j ) R j ( x ) ⟨ w j , x − a j ⟩ , 0 , h j > 0 , h j = 0. (104)
Lemma C.9 (Paired cardinal and displacement polynomials). The polynomials E ~ j \widetilde E_j E j and H ~ j \widetilde H_j H j have degree at most 2 K − 1 2K-1 2 K − 1 and satisfy
E ~ j ( a ℓ ) = E ~ j ( b ℓ ) = 1 { j = ℓ } , (105) \widetilde E_j(a_\ell)=\widetilde E_j(b_\ell)=1\{j=\ell\}, \tag*{(105)} E j ( a ℓ ) = E j ( b ℓ ) = 1 { j = ℓ } , (105) and
H ~ j ( a ℓ ) = 0 , H ~ j ( b ℓ ) = 1 { j = ℓ } h j . (106) \widetilde H_j(a_\ell)=0,\qquad\widetilde H_j(b_\ell)=1\{j=\ell\}h_j. \tag*{(106)} H j ( a ℓ ) = 0 , H j ( b ℓ ) = 1 { j = ℓ } h j . (106)
In the structured regimes, their supremum norms on B d ( 0 , R ) B_d(0,R) B d ( 0 , R ) satisfy
∥ E ~ j ∥ ∞ ≤ C Δ s e p − ( 2 s max − 1 ) , ∥ H ~ j ∥ ∞ ≤ C Δ s e p − ( 2 s max − 2 ) . (107) \lVert\widetilde{E}_j\rVert_\infty\le C\Delta_{\mathrm{sep}}^{-(2s_{\max}-1)}, \qquad\lVert\widetilde{H}_j\rVert_\infty\le C\Delta_{\mathrm{sep}}^{-(2s_{\max}-2)}. \tag*{(107)} ∥ E j ∥ ∞ ≤ C Δ sep − ( 2 s m a x − 1 ) , ∥ H j ∥ ∞ ≤ C Δ sep − ( 2 s m a x − 2 ) . (107)
Without cluster assumptions, these bounds hold with s max = K s_{\max}=K s m a x = K .
Proof. If h j > 0 h_j>0 h j > 0 , the fundamental theorem of calculus on [ a j , b j ] [a_j,b_j] [ a j , b j ] gives
γ j = ∫ 0 1 ⟨ ∇ ( 1 / R j ) ( a j + t ( b j − a j ) ) , w j ⟩ d t . \gamma_j=\int_0^1 \langle\nabla(1/R_j)(a_j+t(b_j-a_j)),w_j\rangle\,\mathrm{d}t. γ j = ∫ 0 1 ⟨ ∇ ( 1/ R j ) ( a j + t ( b j − a j )) , w j ⟩ d t .
Lemma C.8 therefore implies ∣ γ j ∣ ≤ C Δ s e p − 1 |\gamma_j|\le C\Delta_{\mathrm{sep}}^{-1} ∣ γ j ∣ ≤ C Δ sep − 1 , independently of h j h_j h j . When h j = 0 h_j=0 h j = 0 , the same estimate follows from the directional-derivative definition. The lemma also gives ∣ α j ∣ + R j ( b j ) − 1 ≤ C |\alpha_j|+R_j(b_j)^{-1}\le C ∣ α j ∣ + R j ( b j ) − 1 ≤ C and ∥ R j ∥ ∞ ≤ C Δ s e p − ( 2 s max − 2 ) \lVert R_j\rVert_\infty\le C\Delta_{\mathrm{sep}}^{-(2s_{\max}-2)} ∥ R j ∥ ∞ ≤ C Δ sep − ( 2 s m a x − 2 ) .
For ℓ ≠ j \ell\ne j ℓ = j , R j R_j R j vanishes at both a ℓ a_\ell a ℓ and b ℓ b_\ell b ℓ , so both selectors vanish there. At a j a_j a j , E ~ j ( a j ) = R j ( a j ) α j = 1 \widetilde{E}_j(a_j)=R_j(a_j)\alpha_j=1 E j ( a j ) = R j ( a j ) α j = 1 and H ~ j ( a j ) = 0 \widetilde{H}_j(a_j)=0 H j ( a j ) = 0 . If h j > 0 h_j>0 h j > 0 , the definition of γ j \gamma_j γ j gives
E ~ j ( b j ) = R j ( b j ) ( α j + γ j h j ) = 1 , H ~ j ( b j ) = h j . \widetilde{E}_j(b_j)=R_j(b_j)(\alpha_j+\gamma_jh_j)=1,\qquad\widetilde{H}_j(b_j)=h_j. E j ( b j ) = R j ( b j ) ( α j + γ j h j ) = 1 , H j ( b j ) = h j .
If h j = 0 h_j=0 h j = 0 , these assertions follow from a j = b j a_j=b_j a j = b j and H ~ j ≡ 0 \widetilde{H}_j\equiv0 H j ≡ 0 . This proves the interpolation identities even for collapsed fitted pairs.
The degrees are at most 2 K − 1 2K-1 2 K − 1 . Since ∣ ⟨ w j , x − a j ⟩ ∣ ≤ 2 R |\langle w_j,x-a_j\rangle|\le2R ∣ ⟨ w j , x − a j ⟩ ∣ ≤ 2 R on B d ( 0 , R ) B_d(0,R) B d ( 0 , R ) ,
∥ E ~ j ∥ ∞ ≤ ∥ R j ∥ ∞ ( ∣ α j ∣ + 2 R ∣ γ j ∣ ) ≤ C Δ s e p − ( 2 s max − 1 ) , \lVert\widetilde{E}_j\rVert_\infty\le\lVert R_j\rVert_\infty(|\alpha_j|+2R|\gamma_j|)\le C\Delta_{\mathrm{sep}}^{-(2s_{\max}-1)}, ∥ E j ∥ ∞ ≤ ∥ R j ∥ ∞ ( ∣ α j ∣ + 2 R ∣ γ j ∣ ) ≤ C Δ sep − ( 2 s m a x − 1 ) ,
and, for h j > 0 h_j>0 h j > 0 ,
∥ H ~ j ∥ ∞ ≤ 2 R ∥ R j ∥ ∞ / R j ( b j ) ≤ C Δ s e p − ( 2 s max − 2 ) . \lVert\widetilde{H}_j\rVert_\infty\le2R\lVert R_j\rVert_\infty/R_j(b_j)\le C\Delta_{\mathrm{sep}}^{-(2s_{\max}-2)}. ∥ H j ∥ ∞ ≤ 2 R ∥ R j ∥ ∞ / R j ( b j ) ≤ C Δ sep − ( 2 s m a x − 2 ) .
The second norm is zero when h j = 0 h_j=0 h j = 0 . Replacing s max s_{\max} s m a x by K K K proves the unstructured assertions. □
Proofs under exact specification
We first establish auxiliary displacement and probability bounds for any fitted state count k ≥ K k\ge K k ≥ K , which will be used in both the exact-specification and over-specification analyses. Under exact specification, direct transport interpolation and the positive initial weights give the global recovery and localization bounds. The counting argument also retains alternatives with explicit initial-probability dependence. The paired bounds are proved independently under an assumed matching. Their proof uses the polynomials in Appendix § 11 .§ 6 for one-time displacement and probability errors, followed by lower-degree selectors, full products of center distances, and a Markov path comparison for joint recovery. The global argument invokes these paired bounds only after establishing matching. Throughout this proof, for simplicity, we denote h T = h ( p θ , T , p θ ⋆ , T ) h_T=h(p_{\theta,T},p_{\theta^\star,T}) h T = h ( p θ , T , p θ ⋆ , T ) .
Dependency order. The exposition follows the main-text statements, so some ingredients are used before their proofs appear. The global theorem relies on the displacement, counting, moment-to-transport, and diameter-localization lemmas below, and then invokes the paired-center theorem. The paired-center proof uses the polynomial witnesses in Appendix § 11 , Gaussian moment extraction, Lemmas D.2 and D.4 , and the cardinal-polynomial and Markov-comparison lemmas in its own subsection. It does not use the global theorem or the center-matching corollary. Thus the forward reference is not circular. The overfitted proofs in Appendix § 13 use the global recovery lemma , which allows arbitrary fitted state counts, not the exact-fit theorem.
Proof of Theorem 3.1
Auxiliary recovery bounds
The following lemma bounds the quadratic displacement V V V , the within-group displacement sums defining B B B , and the aggregate state-sequence probability errors D I D_I D I . The positive polynomial P P P first controls V V V and provides a fitted expectation bound that will also control the approximation errors of the other polynomials. We then use H j , u H_{j,u} H j , u to recover displacement in a chosen direction. For state-sequence probabilities, we construct polynomials that equal one at the selected true-center tuple and zero at every other true-center tuple, with vanishing first derivatives at all such tuples. Their values extract the target probability, while the vanishing derivatives leave approximation errors that can be controlled by the quadratic displacement bound. In the unstructured regime, there is no fixed positive lower bound on distances between clusters; we therefore retain the full products of distances between true centers. This gives all three bounds without requiring a separate identification of fitted states assigned to the same true center.
Lemma D.1 (Global displacement and aggregate probability bounds). Under the standing model assumptions, for any k ≥ K k \ge K k ≥ K and θ ∈ Θ k ( R ) \theta\in\Theta_k(\mathbb{R}) θ ∈ Θ k ( R ) , in the structured regimes, with β r = max { 2 s max , r ( s max − 1 ) + s max } \beta_r = \max\{2s_{\max}, r(s_{\max}-1)+s_{\max}\} β r = max { 2 s m a x , r ( s m a x − 1 ) + s m a x } ,
V ≤ C Δ s e p − ( 2 s max − 2 ) h T , V \le C\Delta_{\mathrm{sep}}^{-(2s_{\max}-2)}h_T, V ≤ C Δ sep − ( 2 s m a x − 2 ) h T , B ≤ C Δ s e p − ( 2 s max − 1 ) h T , B \le C\Delta_{\mathrm{sep}}^{-(2s_{\max}-1)}h_T, B ≤ C Δ sep − ( 2 s m a x − 1 ) h T , D I ≤ C r Δ s e p − β r h T , I ⊆ [ T ] , ∣ I ∣ = r ≥ 1. D_I \le C_r\Delta_{\mathrm{sep}}^{-\beta_r}h_T,\qquad I \subseteq[T],\quad|I|=r\ge1. D I ≤ C r Δ sep − β r h T , I ⊆ [ T ] , ∣ I ∣ = r ≥ 1. Without a cluster assumption, the V V V and B B B bounds hold with s max = K s_{\max}=K s m a x = K , and D I D_I D I has exponent γ K , r = max { 2 K , r ( K − 1 ) + K } \gamma_{K,r}=\max\{2K,r(K-1)+K\} γ K , r = max { 2 K , r ( K − 1 ) + K } . More precisely, writing D ⋆ = max i , j ∥ a i − a j ∥ 2 D_\star=\max_{i,j}\lVert a_i-a_j\rVert_2 D ⋆ = max i , j ∥ a i − a j ∥ 2 , we have
D I ≤ C r D ⋆ − 2 Δ s e p − ( γ K , r − 2 ) h T . (108) D_I \le C_rD_\star^{-2}\Delta_{\mathrm{sep}}^{-(\gamma_{K,r}-2)}h_T. \tag*{(108)} D I ≤ C r D ⋆ − 2 Δ sep − ( γ K , r − 2 ) h T . (108) No lower bound on fitted state probabilities and no factor ( π min ⋆ ) − 1 (\pi^\star_{\min})^{-1} ( π m i n ⋆ ) − 1 is required.
Proof. In this proof, all test functions are polynomials on the span of the fitted and true centers, so Lemma A.3 and marginalization apply with constants depending only on the standing fixed parameters.
Step 1: Quadratic displacement. We show the bound of V V V based on Δ s e p \Delta_{\mathrm{sep}} Δ sep and h ( p θ , T , p θ ⋆ , T ) h(p_{\theta,T},p_{\theta^\star,T}) h ( p θ , T , p θ ⋆ , T ) . Consider the test polynomial P P P from Lemma C.1 which vanishes on the true support and satisfies
P ( μ i ) ≥ c Δ s e p 2 s max − 2 r i 2 . P(\mu_i)\ge c\Delta_{\mathrm{sep}}^{2s_{\max}-2}r_i^2. P ( μ i ) ≥ c Δ sep 2 s m a x − 2 r i 2 .
Therefore, by using Lemma C.1 and Lemma A.3 , we have simultaneously for all t ≤ T t\le T t ≤ T ,
c Δ s e p 2 s max − 2 ∑ i q t ( i ) r i 2 ≤ E θ P ( μ Z t ) = ∫ P d ( G θ { t } − G θ ⋆ { t } ) ≤ C h T . c\Delta_{\mathrm{sep}}^{2s_{\max}-2}\sum_i q_t(i)r_i^2
\le\mathbb{E}_\theta P(\mu_{Z_t})
= \int P\,\mathrm{d}(G_\theta^{\{t\}}-G_{\theta^\star}^{\{t\}})
\le Ch_T. c Δ sep 2 s m a x − 2 i ∑ q t ( i ) r i 2 ≤ E θ P ( μ Z t ) = ∫ P d ( G θ { t } − G θ ⋆ { t } ) ≤ C h T .
Thus V ≤ C δ − ( 2 s max − 2 ) h T V\le C\delta^{-(2s_{\max}-2)}h_T V ≤ C δ − ( 2 s m a x − 2 ) h T ; we retain the stronger bound max t E θ P ( μ Z t ) ≤ C h T \max_t\mathbb{E}_\theta P(\mu_{Z_t})\le Ch_T max t E θ P ( μ Z t ) ≤ C h T for the approximation errors below.
Step 2: Within-group first displacement. We prove the bound of B B B based on Δ s e p \Delta_{\mathrm{sep}} Δ sep and h ( p θ , T , p θ ⋆ , T ) h(p_{\theta,T},p_{\theta^\star,T}) h ( p θ , T , p θ ⋆ , T ) . Fix j , t j,t j , t and let b j , t = ∑ i ∈ V j q t ( i ) ( μ i − a j ) b_{j,t}=\sum_{i\in V_j}q_t(i)(\mu_i-a_j) b j , t = ∑ i ∈ V j q t ( i ) ( μ i − a j ) . Choose u = b j , t / ∥ b j , t ∥ 2 u=b_{j,t}/\lVert b_{j,t}\rVert_2 u = b j , t / ∥ b j , t ∥ 2 when nonzero and u = 0 u=0 u = 0 otherwise. According to Lemma C.2 there exists a polynomial H j , u H_{j,u} H j , u such that H j , u ( μ i ) = 0 H_{j,u}(\mu_i)=0 H j , u ( μ i ) = 0 and
∥ H j , u ∥ ∞ ≤ C δ − ( 2 s max − 2 ) , ∣ H j , u ( x ) − 1 { c ( x ) = j } ⟨ u , x − a j ⟩ ∣ ≤ C δ − ( 2 s max − 1 ) P ( x ) . \lVert H_{j,u}\rVert_\infty\le C\delta^{-(2s_{\max}-2)},\qquad
\left|H_{j,u}(x)-\mathbf{1}\{c(x)=j\}\langle u,x-a_j\rangle\right|
\le C\delta^{-(2s_{\max}-1)}P(x). ∥ H j , u ∥ ∞ ≤ C δ − ( 2 s m a x − 2 ) , ∣ H j , u ( x ) − 1 { c ( x ) = j } ⟨ u , x − a j ⟩ ∣ ≤ C δ − ( 2 s m a x − 1 ) P ( x ) .
Consequently, by moment extraction and the preceding bound for P P P ,
∥ b j , t ∥ 2 = ∣ ∫ 1 { c ( x ) = j } ⟨ w , x − a j ⟩ d ( G θ { t } − G ⋆ { t } ) ∣ ≤ ∣ ∫ H j , u ( x ) d ( G θ { t } − G ⋆ { t } ) ∣ + ∣ ∫ ( 1 { c ( x ) = j } ⟨ w , x − a j ⟩ − H j , u ( x ) ) d ( G θ { t } − G ⋆ { t } ) ∣ ≤ ∣ ∫ H j , u ( x ) d ( G θ { t } − G ⋆ { t } ) ∣ + C δ − ( 2 s max − 1 ) E θ P ( μ Z t ) ≤ C ( δ − ( 2 s max − 2 ) + δ − ( 2 s max − 1 ) ) h T ≤ C δ − ( 2 s max − 1 ) h T . \begin{aligned}
\lVert b_{j,t}\rVert_2 = \left|\int\mathbf{1}\{c(x)=j\}\langle w,x-a_j\rangle d(G_\theta^{\{t\}}-G_\star^{\{t\}})\right| \\
&\le\left|\int H_{j,u}(x)d(G_\theta^{\{t\}}-G_\star^{\{t\}})\right|+\left|\int\left(\mathbf{1}\{c(x)=j\}\langle w,x-a_j\rangle-H_{j,u}(x)\right)d(G_\theta^{\{t\}}-G_\star^{\{t\}})\right| \\
&\le\left|\int H_{j,u}(x)d(G_\theta^{\{t\}}-G_\star^{\{t\}})\right|+C\delta^{-(2s_{\max}-1)}\mathbb{E}_\theta P(\mu Z_t) \\
&\le C\left(\delta^{-(2s_{\max}-2)}+\delta^{-(2s_{\max}-1)}\right)h_T\le C\delta^{-(2s_{\max}-1)}h_T.
\end{aligned} ∥ b j , t ∥ 2 = ∫ 1 { c ( x ) = j } ⟨ w , x − a j ⟩ d ( G θ { t } − G ⋆ { t } ) ≤ ∫ H j , u ( x ) d ( G θ { t } − G ⋆ { t } ) + ∫ ( 1 { c ( x ) = j } ⟨ w , x − a j ⟩ − H j , u ( x ) ) d ( G θ { t } − G ⋆ { t } ) ≤ ∫ H j , u ( x ) d ( G θ { t } − G ⋆ { t } ) + C δ − ( 2 s m a x − 1 ) E θ P ( μ Z t ) ≤ C ( δ − ( 2 s m a x − 2 ) + δ − ( 2 s m a x − 1 ) ) h T ≤ C δ − ( 2 s m a x − 1 ) h T .
Summation over j j j and maximization over t t t prove the bound for B B B .
Step 3: Assigned path probabilities. We show the bound for D I D_I D I based on Δ s e p \Delta_{\mathrm{sep}} Δ sep and h ( p θ , T , p θ ⋆ , T ) h(p_{\theta,T},p_{\theta_\star,T}) h ( p θ , T , p θ ⋆ , T ) . For I = ( t 1 , … , t r ) I=(t_1,\ldots,t_r) I = ( t 1 , … , t r ) and j ∈ [ K ] r j\in[K]^r j ∈ [ K ] r , write 1 j ( x ) = 1 { ( c ( x 1 ) , … , c ( x r ) ) = j } \mathbf{1}_j(x)=\mathbf{1}\{(c(x_1),\ldots,c(x_r))=j\} 1 j ( x ) = 1 {( c ( x 1 ) , … , c ( x r )) = j } . In the structured case, the polynomial U j U_j U j of Lemma C.6 equals 1 j \mathbf{1}_j 1 j on the true grid and obeys, with d r = r ( s max − 1 ) + s max d_r=r(s_{\max}-1)+s_{\max} d r = r ( s m a x − 1 ) + s m a x and β r = max { 2 s max , d r } \beta_r=\max\{2s_{\max},d_r\} β r = max { 2 s m a x , d r }
∥ U j ∥ ∞ ≤ C r δ − d r , ∣ U j ( x ) − 1 j ( x ) ∣ ≤ C r δ − β r ∑ ℓ = 1 r P ( x ℓ ) . \lVert U_j\rVert_\infty\le C_r\delta^{-d_r},\qquad|U_j(x)-\mathbf{1}_j(x)|\le C_r\delta^{-\beta_r}\sum_{\ell=1}^{r}P(x_\ell). ∥ U j ∥ ∞ ≤ C r δ − d r , ∣ U j ( x ) − 1 j ( x ) ∣ ≤ C r δ − β r ℓ = 1 ∑ r P ( x ℓ ) .
Since ∫ U j d G ⋆ I = P ⋆ I ( j ) \int U_j\,dG_\star^I=P_\star^I(j) ∫ U j d G ⋆ I = P ⋆ I ( j ) and ∫ 1 j d G θ I = P ‾ θ I ( j ) \int\mathbf{1}_j\,dG_\theta^I=\overline{P}_\theta^I(j) ∫ 1 j d G θ I = P θ I ( j ) , we obtain
∣ P ‾ θ I ( j ) − P ⋆ I ( j ) ∣ ≤ ∣ ∫ U j d ( G θ I − G ⋆ I ) ∣ + ∣ ∫ ∣ U j − 1 j ∣ d G θ I ∣ ≤ C r δ − d r h T + C r δ − β r ∑ ℓ = 1 r E θ P ( μ Z t ℓ ) ≤ C r δ − β r h T . \begin{aligned}
|\overline{P}_\theta^I(j)-P_\star^I(j)|\le\left|\int U_j\,d(G_\theta^I-G_\star^I)\right|+\left|\int|U_j-\mathbf{1}_j|dG_\theta^I\right| \\
&\le C_r\delta^{-d_r}h_T+C_r\delta^{-\beta_r}\sum_{\ell=1}^{r}\mathbb{E}_\theta P(\mu Z_{t_\ell})\le C_r\delta^{-\beta_r}h_T.
\end{aligned} ∣ P θ I ( j ) − P ⋆ I ( j ) ∣ ≤ ∫ U j d ( G θ I − G ⋆ I ) + ∫ ∣ U j − 1 j ∣ d G θ I ≤ C r δ − d r h T + C r δ − β r ℓ = 1 ∑ r E θ P ( μ Z t ℓ ) ≤ C r δ − β r h T .
Summing over the K r K^r K r label tuples proves the structured bound. There is no independence assumption between selected hidden states.
Without cluster assumptions, Steps 1 and 2 apply with s max = K s_{\max}=K s m a x = K , giving the stated bounds for V V V and B B B . For D I D_I D I , we retain the diameter dependence provided by Lemma C.7 . Write
γ K , r = max { 2 K , r ( K − 1 ) + K } , F K , r = D ⋆ − 2 δ − ( γ K , r − 2 ) . \gamma_{K,r}=\max\{2K,r(K-1)+K\},\qquad F_{K,r}=D_\star^{-2}\delta^{-(\gamma_{K,r}-2)}. γ K , r = max { 2 K , r ( K − 1 ) + K } , F K , r = D ⋆ − 2 δ − ( γ K , r − 2 ) .
For each j ∈ [ K ] r j\in[K]^r j ∈ [ K ] r , that lemma supplies a polynomial U j U_j U j equal to 1 j \mathbf{1}_j 1 j on the true grid and satisfying
∥ U j ∥ ∞ ≤ C r F K , r , ∣ U j ( x ) − 1 j ( x ) ∣ ≤ C r F K , r ∑ ℓ = 1 r P ( x ℓ ) . \lVert U_j\rVert_\infty\le C_rF_{K,r},\qquad|U_j(x)-\mathbf{1}_j(x)|\le C_rF_{K,r}\sum_{\ell=1}^{r}P(x_\ell). ∥ U j ∥ ∞ ≤ C r F K , r , ∣ U j ( x ) − 1 j ( x ) ∣ ≤ C r F K , r ℓ = 1 ∑ r P ( x ℓ ) .
The cardinal values give ∫ U j d G ⋆ I = P ⋆ I ( j ) \int U_j\,dG_\star^I=P_\star^I(j) ∫ U j d G ⋆ I = P ⋆ I ( j ) . Hence moment extraction and the expectation bound from Step 1 yield
∣ P ‾ θ I ( j ) − P ⋆ I ( j ) ∣ ≤ ∣ ∫ U j d ( G θ I − G ⋆ I ) ∣ + ∣ ∫ ∣ 1 j − U j ∣ d G θ I ∣ ≤ C r F K , r h T + C r F K , r ∑ ℓ = 1 r E θ P ( μ Z t ℓ ) ≤ C r F K , r h T , \begin{aligned}
|\overline{P}_\theta^I(j)-P_\star^I(j)|\le\left|\int U_jd(G_\theta^I-G_\star^I)\right|+\left|\int|\mathbf{1}_j-U_j|dG_\theta^I\right| \\
&\le C_rF_{K,r}h_T+C_rF_{K,r}\sum_{\ell=1}^{r}\mathbb{E}_\theta P(\mu Z_{t_\ell}) \\
&\le C_rF_{K,r}h_T,
\end{aligned} ∣ P θ I ( j ) − P ⋆ I ( j ) ∣ ≤ ∫ U j d ( G θ I − G ⋆ I ) + ∫ ∣ 1 j − U j ∣ d G θ I ≤ C r F K , r h T + C r F K , r ℓ = 1 ∑ r E θ P ( μ Z t ℓ ) ≤ C r F K , r h T ,
where the fixed factor r r r is absorbed into C r C_r C r . Summing over j ∈ [ K ] r j \in[K]^r j ∈ [ K ] r and absorbing K r K^r K r into the constant proves
D I ≤ C r D ⋆ − 2 δ − ( γ K , r − 2 ) h T , D_I \le C_r D_\star^{-2}\delta^{-(\gamma_{K,r}-2)}h_T, D I ≤ C r D ⋆ − 2 δ − ( γ K , r − 2 ) h T ,
which is (D.1). Since D ⋆ ≥ δ D_\star\ge\delta D ⋆ ≥ δ , this also implies D I ≤ C r δ − γ K , r h T D_I \le C_r\delta^{-\gamma_{K,r}}h_T D I ≤ C r δ − γ K , r h T . All fitted probabilities enter only as nonnegative weights; thus zero fitted probabilities and coincident fitted centers are allowed.
We next compare probabilities of cluster sequences. For these probabilities, Lemma C.5 provides polynomials F m F_m F m whose supremum norms are bounded independently of emission separation. Their approximation errors are controlled by squared distances to the assigned true centers. Products of these polynomials therefore give an observation-density contribution with no inverse separation factor and an approximation contribution controlled by V V V .
Lemma D.2 (Cluster-sequence mass error). For every structured configuration and every I ⊆ [ T ] I \subseteq[T] I ⊆ [ T ] with ∣ I ∣ = r |I| = r ∣ I ∣ = r ,
J I ≤ C r ( h T + V ) ≤ C r Δ s e p − ( 2 s max − 2 ) h T . (109) J_I \le C_r(h_T+V) \le C_r\Delta_{\mathrm{sep}}^{-(2s_{\max}-2)}h_T. \tag*{(109)} J I ≤ C r ( h T + V ) ≤ C r Δ sep − ( 2 s m a x − 2 ) h T . (109) This holds for any fitted number of states k k k .
Proof. If there is only one true cluster, both cluster-sequence laws are point masses at the same label sequence, and J I = 0 J_I=0 J I = 0 , equation (D.2) automatically holds. Otherwise, for m = ( m 1 , … , m r ) m=(m_1,\ldots,m_r) m = ( m 1 , … , m r ) define
U m ( x ) = ∏ ℓ = 1 r F m ℓ ( x ℓ ) , B m ( x ) = ∏ ℓ = 1 r 1 { c ( x ℓ ) ∈ C m ℓ } . U_m(x)=\prod_{\ell=1}^{r}F_{m_\ell}(x_\ell), \qquad B_m(x)=\prod_{\ell=1}^{r}1\{c(x_\ell)\in C_{m_\ell}\}. U m ( x ) = ℓ = 1 ∏ r F m ℓ ( x ℓ ) , B m ( x ) = ℓ = 1 ∏ r 1 { c ( x ℓ ) ∈ C m ℓ } .
where the cluster polynomials F a F_a F a are those of Lemma C.5 . Their interpolation values on true centers show ∫ U m d G ⋆ I = P ⋆ I , c l ( m ) \int U_m\,\mathrm{d}G_\star^I=P_\star^{I,\mathrm{cl}}(m) ∫ U m d G ⋆ I = P ⋆ I , cl ( m ) , while ∫ B m d G θ I = P ˉ θ I , c l ( m ) \int B_m\,\mathrm{d}G_\theta^I=\bar P_\theta^{I,\mathrm{cl}}(m) ∫ B m d G θ I = P ˉ θ I , cl ( m ) . The uniform bounds ∥ F a ∥ ∞ ≤ C \|F_a\|_\infty\le C ∥ F a ∥ ∞ ≤ C yield ∥ U m ∥ ∞ ≤ C r \|U_m\|_\infty\le C_r ∥ U m ∥ ∞ ≤ C r , so moment extraction gives
∣ ∫ U m d ( G θ I − G ⋆ I ) ∣ ≤ C r h T . \left|\int U_m\,\mathrm{d}(G_\theta^I-G_\star^I)\right|\le C_rh_T. ∫ U m d ( G θ I − G ⋆ I ) ≤ C r h T .
The product telescoping identity and the cluster-polynomial error ∣ F a ( x ) − 1 { c ( x ) ∈ C a } ∣ ≤ C ∥ x − a c ( x ) ∥ 2 2 |F_a(x)-1\{c(x)\in C_a\}|\le C\|x-a_{c(x)}\|_2^2 ∣ F a ( x ) − 1 { c ( x ) ∈ C a } ∣ ≤ C ∥ x − a c ( x ) ∥ 2 2 imply
∣ U m ( x ) − B m ( x ) ∣ ≤ C r ∑ ℓ = 1 r ∥ x ℓ − a c ( x ℓ ) ∥ 2 2 . |U_m(x)-B_m(x)|\le C_r\sum_{\ell=1}^{r}\|x_\ell-a_{c(x_\ell)}\|_2^2. ∣ U m ( x ) − B m ( x ) ∣ ≤ C r ℓ = 1 ∑ r ∥ x ℓ − a c ( x ℓ ) ∥ 2 2 .
Integrating this bound against G θ I G_\theta^I G θ I gives
∫ ∣ U m − B m ∣ d G θ I ≤ C r ∑ ℓ = 1 r E θ ∥ μ Z t ℓ − a c ( μ Z t ℓ ) ∥ 2 2 = C r ∑ ℓ = 1 r V t ℓ ≤ C r V , \int|U_m-B_m|\,\mathrm{d}G_\theta^I\le C_r\sum_{\ell=1}^{r}\mathbb{E}_\theta\|\mu_{Z_{t_\ell}}-a_{c(\mu_{Z_{t_\ell}})}\|_2^2=C_r\sum_{\ell=1}^{r}V_{t_\ell}\le C_rV, ∫ ∣ U m − B m ∣ d G θ I ≤ C r ℓ = 1 ∑ r E θ ∥ μ Z t ℓ − a c ( μ Z t ℓ ) ∥ 2 2 = C r ℓ = 1 ∑ r V t ℓ ≤ C r V ,
where V = max t ≤ T V t V=\max_{t\le T}V_t V = max t ≤ T V t and the factor r r r is absorbed into C r C_r C r . Using the integral representations of the cluster-sequence probabilities, we obtain
∣ P ˉ θ I , c l ( m ) − P ⋆ I , c l ( m ) ∣ = ∣ ∫ B m d G θ I − ∫ U m d G ⋆ I ∣ ≤ ∣ ∫ U m d ( G θ I − G ⋆ I ) ∣ + ∫ ∣ B m − U m ∣ d G θ I ≤ C r ( h T + V ) . \left|\bar P_\theta^{I,\mathrm{cl}}(m)-P_\star^{I,\mathrm{cl}}(m)\right|
=\left|\int B_m\,\mathrm{d}G_\theta^I-\int U_m\,\mathrm{d}G_\star^I\right|
\le\left|\int U_m\,\mathrm{d}(G_\theta^I-G_\star^I)\right|+\int|B_m-U_m|\,\mathrm{d}G_\theta^I
\le C_r(h_T+V). P ˉ θ I , cl ( m ) − P ⋆ I , cl ( m ) = ∫ B m d G θ I − ∫ U m d G ⋆ I ≤ ∫ U m d ( G θ I − G ⋆ I ) + ∫ ∣ B m − U m ∣ d G θ I ≤ C r ( h T + V ) .
There are at most K r K^r K r cluster-label sequences. Summing over them and absorbing this factor into C r C_r C r yields
J I = ∑ m ∣ P ˉ θ I , c l ( m ) − P ⋆ I , c l ( m ) ∣ ≤ C r ( h T + V ) . J_I=\sum_m\left|\bar P_\theta^{I,\mathrm{cl}}(m)-P_\star^{I,\mathrm{cl}}(m)\right|\le C_r(h_T+V). J I = m ∑ P ˉ θ I , cl ( m ) − P ⋆ I , cl ( m ) ≤ C r ( h T + V ) .
Finally, Lemma D.1 gives
J I ≤ C r ( 1 + Δ s e p − ( 2 s max − 2 ) ) h T ≤ C r Δ s e p − ( 2 s max − 2 ) h T , \begin{aligned}
J_I \le C r\left(1+\Delta_{\mathrm{sep}}^{-(2s_{\max}-2)}\right)h_T \\
&\le C r\Delta_{\mathrm{sep}}^{-(2s_{\max}-2)}h_T,
\end{aligned} J I ≤ C r ( 1 + Δ sep − ( 2 s m a x − 2 ) ) h T ≤ C r Δ sep − ( 2 s m a x − 2 ) h T ,
where the last inequality uses 0 < Δ s e p ≤ R 0<\Delta_{\mathrm{sep}}\le R 0 < Δ sep ≤ R and s max ≥ 1 s_{\max}\ge1 s m a x ≥ 1 . This proves (D.2).
Controlling individual displacements under exact specification.
To obtain a bound for D μ D_\mu D μ that is linear in the observation-density error, we use the bounds for quadratic displacement and cell displacement vectors established in Lemma D.1. A cell displacement vector controls a single nearby fitted center, but several displacement vectors in the same cell may cancel. Under exact specification, a counting argument controls the contribution of cells containing several nearby fitted centers.
Lemma D.3 (Counting near centers). Suppose k = K k=K k = K , and define
N j = { i : c ( i ) = j , r i ≤ Δ s e p / 4 } , F = { i : r i > Δ s e p / 4 } . N_j=\{i:c(i)=j,\ r_i\le\Delta_{\mathrm{sep}}/4\},\qquad F=\{i:r_i>\Delta_{\mathrm{sep}}/4\}. N j = { i : c ( i ) = j , r i ≤ Δ sep /4 } , F = { i : r i > Δ sep /4 } . Then
max t ≤ T ∑ j = 1 K ∣ ∑ i ∈ N j q t ( i ) − q t ∗ ( j ) ∣ ≤ C Δ s e p − 2 s max h T . (110) \max_{t\le T}\sum_{j=1}^{K}\left|\sum_{i\in N_j}q_t(i)-q_t^*(j)\right|
\le C\Delta_{\mathrm{sep}}^{-2s_{\max}}h_T. \tag*{(110)} t ≤ T max j = 1 ∑ K i ∈ N j ∑ q t ( i ) − q t ∗ ( j ) ≤ C Δ sep − 2 s m a x h T . (110) If some N j N_j N j is empty, then
π min ∗ ≤ C Δ s e p − 2 s max h T . (111) \pi_{\min}^*\le C\Delta_{\mathrm{sep}}^{-2s_{\max}}h_T. \tag*{(111)} π m i n ∗ ≤ C Δ sep − 2 s m a x h T . (111) Moreover,
D μ ≤ C π min ∗ Δ s e p − ( 2 s max − 1 ) h T . (112) D_\mu\le\frac{C}{\pi_{\min}^*}\Delta_{\mathrm{sep}}^{-(2s_{\max}-1)}h_T. \tag*{(112)} D μ ≤ π m i n ∗ C Δ sep − ( 2 s m a x − 1 ) h T . (112)
Proof. Write δ = Δ s e p \delta=\Delta_{\mathrm{sep}} δ = Δ sep and s = s max s=s_{\max} s = s m a x . For each j ∈ [ K ] j\in[K] j ∈ [ K ] and t ≤ T t\le T t ≤ T , define
q ˉ t ( j ) = ∑ i ∈ V j q t ( i ) , b j , t = ∑ i ∈ V j q t ( i ) ( μ i − a j ) , V t = ∑ i = 1 K q t ( i ) r i 2 . \bar q_t(j)=\sum_{i\in V_j}q_t(i),\qquad b_{j,t}=\sum_{i\in V_j}q_t(i)(\mu_i-a_j),\qquad V_t=\sum_{i=1}^{K}q_t(i)r_i^2. q ˉ t ( j ) = i ∈ V j ∑ q t ( i ) , b j , t = i ∈ V j ∑ q t ( i ) ( μ i − a j ) , V t = i = 1 ∑ K q t ( i ) r i 2 .
Lemma D.1 gives, uniformly in t t t ,
V t ≤ C δ − ( 2 s − 2 ) h T , ∑ j = 1 K ∥ b j , t ∥ 2 ≤ C δ − ( 2 s − 1 ) h T , ∑ j = 1 K ∣ q ˉ t ( j ) − q t ∗ ( j ) ∣ ≤ C δ − 2 s h T . (113) V_t\le C\delta^{-(2s-2)}h_T,\qquad\sum_{j=1}^{K}\lVert b_{j,t}\rVert_2\le C\delta^{-(2s-1)}h_T,\qquad\sum_{j=1}^{K}\left|\bar q_t(j)-q_t^*(j)\right|\le C\delta^{-2s}h_T. \tag*{(113)} V t ≤ C δ − ( 2 s − 2 ) h T , j = 1 ∑ K ∥ b j , t ∥ 2 ≤ C δ − ( 2 s − 1 ) h T , j = 1 ∑ K ∣ q ˉ t ( j ) − q t ∗ ( j ) ∣ ≤ C δ − 2 s h T . (113)
Step 1: We bound the total probability and displacement of the far labels. Since r i > δ / 4 r_i>\delta/4 r i > δ /4 for i ∈ F i\in F i ∈ F , we have 1 ≤ 16 r i 2 / δ 2 1\le16r_i^2/\delta^2 1 ≤ 16 r i 2 / δ 2 , and r i ≤ 4 r i 2 / δ r_i\le4r_i^2/\delta r i ≤ 4 r i 2 / δ . Multiplying by q t ( i ) q_t(i) q t ( i ) and summing over F F F yields
∑ i ∈ F q t ( i ) ≤ 16 δ − 2 V t ≤ C δ − 2 s h T , ∑ i ∈ F q t ( i ) r i ≤ 4 δ − 1 V t ≤ C δ − ( 2 s − 1 ) h T . (114) \sum_{i\in F}q_t(i)\le16\delta^{-2}V_t\le C\delta^{-2s}h_T,\qquad\sum_{i\in F}q_t(i)r_i\le4\delta^{-1}V_t\le C\delta^{-(2s-1)}h_T. \tag*{(114)} i ∈ F ∑ q t ( i ) ≤ 16 δ − 2 V t ≤ C δ − 2 s h T , i ∈ F ∑ q t ( i ) r i ≤ 4 δ − 1 V t ≤ C δ − ( 2 s − 1 ) h T . (114)
Step 2: We compare the near-set masses with the true state probabilities and deduce the bound for an empty near set. Write
n j , t = ∑ i ∈ N j q t ( i ) , f j , t = ∑ i ∈ F ∩ V j q t ( i ) . n_{j,t}=\sum_{i\in N_j}q_t(i),\qquad f_{j,t}=\sum_{i\in F\cap V_j}q_t(i). n j , t = i ∈ N j ∑ q t ( i ) , f j , t = i ∈ F ∩ V j ∑ q t ( i ) .
Because V j = N j ∪ ˙ ( F ∩ V j ) V_j=N_j\mathbin{\dot\cup}(F\cap V_j) V j = N j ∪ ˙ ( F ∩ V j ) , we have q ˉ t ( j ) = n j , t + f j , t \bar q_t(j)=n_{j,t}+f_{j,t} q ˉ t ( j ) = n j , t + f j , t . Therefore,
∑ j = 1 K ∣ n j , t − q t ⋆ ( j ) ∣ = ∑ j = 1 K ∣ q ˉ t ( j ) − q t ⋆ ( j ) − f j , t ∣ ≤ ∑ j = 1 K ∣ q ˉ t ( j ) − q t ⋆ ( j ) ∣ + ∑ j = 1 K f j , t = D { t } + ∑ i ∈ F q t ( i ) ≤ C δ − 2 s h T . \sum_{j=1}^{K}\lvert n_{j,t}-q_t^\star(j)\rvert=\sum_{j=1}^{K}\lvert\bar q_t(j)-q_t^\star(j)-f_{j,t}\rvert\le\sum_{j=1}^{K}\lvert\bar q_t(j)-q_t^\star(j)\rvert+\sum_{j=1}^{K}f_{j,t}
=D_{\{t\}}+\sum_{i\in F}q_t(i)\le C\delta^{-2s}h_T. j = 1 ∑ K ∣ n j , t − q t ⋆ ( j )∣ = j = 1 ∑ K ∣ q ˉ t ( j ) − q t ⋆ ( j ) − f j , t ∣ ≤ j = 1 ∑ K ∣ q ˉ t ( j ) − q t ⋆ ( j )∣ + j = 1 ∑ K f j , t = D { t } + i ∈ F ∑ q t ( i ) ≤ C δ − 2 s h T .
Taking the maximum over t t t proves (D.3).
If N j 0 = ∅ N_{j_0}=\varnothing N j 0 = ∅ , then n j 0 , 1 = 0 n_{j_0,1}=0 n j 0 , 1 = 0 and q 1 ⋆ ( j 0 ) = π j 0 ⋆ q_1^\star(j_0)=\pi_{j_0}^\star q 1 ⋆ ( j 0 ) = π j 0 ⋆ . Consequently, π min ⋆ ≤ π j 0 ⋆ = ∣ n j 0 , 1 − q 1 ⋆ ( j 0 ) ∣ ≤ C δ − 2 s h T \pi_{\min}^\star\le\pi_{j_0}^\star=\lvert n_{j_0,1}-q_1^\star(j_0)\rvert\le C\delta^{-2s}h_T π m i n ⋆ ≤ π j 0 ⋆ = ∣ n j 0 , 1 − q 1 ⋆ ( j 0 )∣ ≤ C δ − 2 s h T , which proves (D.4).
Step 3: We control the displacement contributed by singleton near sets. Define
S = { j : ∣ N j ∣ = 1 } , M = { j : ∣ N j ∣ ≥ 2 } . \mathcal{S}=\{j:\lvert N_j\rvert=1\},\qquad\mathcal{M}=\{j:\lvert N_j\rvert\ge2\}. S = { j : ∣ N j ∣ = 1 } , M = { j : ∣ N j ∣ ≥ 2 } .
For j ∈ S j\in\mathcal{S} j ∈ S , write N j = { i j } N_j=\{i_j\} N j = { i j } . Then
b j , t = q t ( i j ) ( μ i j − a j ) + ∑ i ∈ F ∩ V j q t ( i ) ( μ i − a j ) . b_{j,t}=q_t(i_j)(\mu_{i_j}-a_j)+\sum_{i\in F\cap V_j}q_t(i)(\mu_i-a_j). b j , t = q t ( i j ) ( μ i j − a j ) + i ∈ F ∩ V j ∑ q t ( i ) ( μ i − a j ) .
Since c ( i j ) = j c(i_j)=j c ( i j ) = j , the triangle inequality gives q t ( i j ) r i j ≤ ∥ b j , t ∥ 2 + ∑ i ∈ F ∩ V j q t ( i ) r i q_t(i_j)r_{i_j}\le\lVert b_{j,t}\rVert_2+\sum_{i\in F\cap V_j}q_t(i)r_i q t ( i j ) r i j ≤ ∥ b j , t ∥ 2 + ∑ i ∈ F ∩ V j q t ( i ) r i . Summing over S \mathcal{S} S and using (D.6) and (D.7), we obtain
∑ j ∈ S ∑ i ∈ N j q t ( i ) r i ≤ ∑ j = 1 K ∥ b j , t ∥ 2 + ∑ i ∈ F q t ( i ) r i ≤ C δ − ( 2 s − 1 ) h T . \sum_{j\in\mathcal{S}}\sum_{i\in N_j}q_t(i)r_i\le\sum_{j=1}^{K}\lVert b_{j,t}\rVert_2+\sum_{i\in F}q_t(i)r_i\le C\delta^{-(2s-1)}h_T. j ∈ S ∑ i ∈ N j ∑ q t ( i ) r i ≤ j = 1 ∑ K ∥ b j , t ∥ 2 + i ∈ F ∑ q t ( i ) r i ≤ C δ − ( 2 s − 1 ) h T .
Step 4: We control the remaining near displacements using k = K k=K k = K and combine the contributions. If M ≠ ∅ \mathcal{M}\ne\varnothing M = ∅ , at least one near set must be empty. Indeed, if every N j N_j N j were nonempty, then their disjointness would imply K = k ≥ ∑ j = 1 K ∣ N j ∣ ≥ ( K − 1 ) + 2 = K + 1 K=k\ge\sum_{j=1}^{K}\lvert N_j\rvert\ge(K-1)+2=K+1 K = k ≥ ∑ j = 1 K ∣ N j ∣ ≥ ( K − 1 ) + 2 = K + 1 , a contradiction. Step 2 therefore gives
π min ⋆ ≤ C δ − 2 s h T . \pi_{\min}^\star\le C\delta^{-2s}h_T. π m i n ⋆ ≤ C δ − 2 s h T .
Since r i ≤ δ / 4 r_i\le\delta/4 r i ≤ δ /4 on every near set and ∑ i q t ( i ) = 1 \sum_iq_t(i)=1 ∑ i q t ( i ) = 1 ,
∑ j ∈ M ∑ i ∈ N j q t ( i ) r i ≤ δ 4 ∑ j ∈ M ∑ i ∈ N j q t ( i ) ≤ δ 4 ≤ C π min ⋆ δ − ( 2 s − 1 ) h T . \sum_{j\in\mathcal{M}}\sum_{i\in N_j}q_t(i)r_i\le\frac{\delta}{4}\sum_{j\in\mathcal{M}}\sum_{i\in N_j}q_t(i)\le\frac{\delta}{4}\le\frac{C}{\pi_{\min}^\star}\delta^{-(2s-1)}h_T. j ∈ M ∑ i ∈ N j ∑ q t ( i ) r i ≤ 4 δ j ∈ M ∑ i ∈ N j ∑ q t ( i ) ≤ 4 δ ≤ π m i n ⋆ C δ − ( 2 s − 1 ) h T .
If M = ∅ \mathcal{M}=\varnothing M = ∅ , this contribution is zero, so the same bound holds.
As the sets F , N 1 , … , N K F,N_1,\ldots,N_K F , N 1 , … , N K partition the fitted labels, we combine Steps 1, 3, and 4, and using 0 < π min ⋆ ≤ 1 0<\pi_{\min}^\star\le1 0 < π m i n ⋆ ≤ 1 , and yield
∑ i = 1 K q t ( i ) r i = ∑ i ∈ F q t ( i ) r i + ∑ j ∈ S ∑ i ∈ N j q t ( i ) r i + ∑ j ∈ M ∑ i ∈ N j q t ( i ) r i ≤ C π min ⋆ δ − ( 2 s − 1 ) h T . \sum_{i=1}^{K}q_t(i)r_i=\sum_{i\in F}q_t(i)r_i+\sum_{j\in\mathcal{S}}\sum_{i\in N_j}q_t(i)r_i+\sum_{j\in\mathcal{M}}\sum_{i\in N_j}q_t(i)r_i\le\frac{C}{\pi_{\min}^\star}\delta^{-(2s-1)}h_T. i = 1 ∑ K q t ( i ) r i = i ∈ F ∑ q t ( i ) r i + j ∈ S ∑ i ∈ N j ∑ q t ( i ) r i + j ∈ M ∑ i ∈ N j ∑ q t ( i ) r i ≤ π m i n ⋆ C δ − ( 2 s − 1 ) h T .
Taking the maximum over t ≤ T t\le T t ≤ T proves (D.5).
Direct transport control and global recovery
We next interpolate a Lipschitz Wasserstein test function directly. This gives the global bounds with sharper separation powers, and also the unstructured one-time bound under an assumed matching.
Lemma D.4 (Moment and density comparisons for finite-mixture transport). Let H = ∑ j = 1 K v j δ b j H = \sum_{j=1}^{K} v_j\delta_{b_j} H = ∑ j = 1 K v j δ b j and H ⋆ = ∑ j = 1 K w j δ a j H^\star= \sum_{j=1}^{K} w_j\delta_{a_j} H ⋆ = ∑ j = 1 K w j δ a j be probability measures on B d ( 0 , R ) B_d(0,R) B d ( 0 , R ) with nonnegative weights. Assume K ≥ 2 K \ge2 K ≥ 2 , the a j a_j a j are distinct, and put Δ = 1 ∧ min i ≠ j ∥ a i − a j ∥ 2 \Delta= 1 \wedge\min_{i \ne j}\lVert a_i-a_j\rVert_2 Δ = 1 ∧ min i = j ∥ a i − a j ∥ 2 . Define the polynomial-moment discrepancy
M D ( H , H ⋆ ) : = sup deg Q ≤ D ∥ Q ∥ ∞ , B d ( 0 , R ) ≤ 1 ∣ ∫ Q d ( H − H ⋆ ) ∣ . (115) \mathcal{M}_D(H,H^\star) := \sup_{\substack{\deg Q \le D\\ \lVert Q\rVert_{\infty,B_d(0,R)}\le1}} \left|\int Q\,\mathrm{d}(H-H^\star)\right|. \tag*{(115)} M D ( H , H ⋆ ) := d e g Q ≤ D ∥ Q ∥ ∞ , B d ( 0 , R ) ≤ 1 sup ∫ Q d ( H − H ⋆ ) . (115) If ∥ b j − a j ∥ 2 ≤ Δ / 4 \lVert b_j-a_j\rVert_2 \le\Delta/4 ∥ b j − a j ∥ 2 ≤ Δ/4 for every j j j , then
W 1 ( H , H ⋆ ) ≤ C ( K , R ) Δ − ( 2 K − 2 ) M 2 K − 1 ( H , H ⋆ ) . (116) W_1(H,H^\star) \le C(K,R)\Delta^{-(2K-2)}\mathcal{M}_{2K-1}(H,H^\star). \tag*{(116)} W 1 ( H , H ⋆ ) ≤ C ( K , R ) Δ − ( 2 K − 2 ) M 2 K − 1 ( H , H ⋆ ) . (116) Alternatively, if w j ≥ p 0 > 0 w_j \ge p_0 > 0 w j ≥ p 0 > 0 for every j j j , the same inequality holds without matching, with constant C ( K , R , p 0 ) C(K,R,p_0) C ( K , R , p 0 ) . The fitted weights may vanish and the fitted centers may coincide in this latter assertion. In particular, for the identity-covariance Gaussian mixture densities,
W 1 ( H , H ⋆ ) ≤ C ( K , R ) Δ − ( 2 K − 2 ) h ( p H , p H ⋆ ) under matching , (117) W_1(H,H^\star) \le C(K,R)\Delta^{-(2K-2)}h(p_H,p_{H^\star}) \qquad\text{under matching}, \tag*{(117)} W 1 ( H , H ⋆ ) ≤ C ( K , R ) Δ − ( 2 K − 2 ) h ( p H , p H ⋆ ) under matching , (117) W 1 ( H , H ⋆ ) ≤ C ( K , R , p 0 ) Δ − ( 2 K − 2 ) h ( p H , p H ⋆ ) if min j w j ≥ p 0 . (118) W_1(H,H^\star) \le C(K,R,p_0)\Delta^{-(2K-2)}h(p_H,p_{H^\star}) \qquad\text{if } \min_j w_j \ge p_0. \tag*{(118)} W 1 ( H , H ⋆ ) ≤ C ( K , R , p 0 ) Δ − ( 2 K − 2 ) h ( p H , p H ⋆ ) if j min w j ≥ p 0 . (118) For K = 1 K=1 K = 1 , no separation is needed: W 1 ( H , H ⋆ ) ≤ R M 1 ( H , H ⋆ ) W_1(H,H^\star) \le R\mathcal{M}_1(H,H^\star) W 1 ( H , H ⋆ ) ≤ R M 1 ( H , H ⋆ ) .
Proof. Write M = M 2 K − 1 ( H , H ⋆ ) M=\mathcal{M}_{2K-1}(H,H^\star) M = M 2 K − 1 ( H , H ⋆ ) . We divide our proof into following steps.
Step 1: We reduce the moment comparison to a one-dimensional interpolation problem. By Lemma C.3 , there exist a unit vector u u u and a constant c K ∈ ( 0 , 1 ] c_K \in(0,1] c K ∈ ( 0 , 1 ] such that
∣ ⟨ u , z − z ′ ⟩ ∣ ≥ c K ∥ z − z ′ ∥ 2 \left|\langle u,z-z'\rangle\right| \ge c_K\lVert z-z'\rVert_2 ∣ ⟨ u , z − z ′ ⟩ ∣ ≥ c K ∥ z − z ′ ∥ 2
for all distinct points in { a 1 , … , a K , b 1 , … , b K } \{a_1,\ldots,a_K,b_1,\ldots,b_K\} { a 1 , … , a K , b 1 , … , b K } . Define H ˉ = ( ⟨ u , ⋅ ⟩ ) # H \bar{H}=(\langle u,\cdot\rangle)_\#H H ˉ = (⟨ u , ⋅ ⟩ ) # H , H ˉ ⋆ = ( ⟨ u , ⋅ ⟩ ) # H ⋆ \bar{H}^\star=(\langle u,\cdot\rangle)_\#H^\star H ˉ ⋆ = (⟨ u , ⋅ ⟩ ) # H ⋆ , and ν = H ˉ − H ˉ ⋆ \nu=\bar{H}-\bar{H}^\star ν = H ˉ − H ˉ ⋆ . The projection z ↦ ⟨ u , z ⟩ z\mapsto\langle u,z\rangle z ↦ ⟨ u , z ⟩ onto the direction u u u is injective on supp ( H ) ∪ supp ( H ⋆ ) \operatorname{supp}(H)\cup\operatorname{supp}(H^\star) supp ( H ) ∪ supp ( H ⋆ ) as distinct points in this union have distinct projected values. Lifting any projected coupling therefore gives W 1 ( H , H ⋆ ) ≤ c K − 1 W 1 ( H ˉ , H ˉ ⋆ ) W_1(H,H^\star)\le c_K^{-1}W_1(\bar{H},\bar{H}^\star) W 1 ( H , H ⋆ ) ≤ c K − 1 W 1 ( H ˉ , H ˉ ⋆ ) .
Let W = W 1 ( H ˉ , H ˉ ⋆ ) W=W_1(\bar{H},\bar{H}^\star) W = W 1 ( H ˉ , H ˉ ⋆ ) . If W = 0 W=0 W = 0 , the assertion (D.9) is immediate. Otherwise, Kantorovich–Rubinstein duality provides a one-Lipschitz function ψ \psi ψ on [ − R , R ] [-R,R] [ − R , R ] such that
ψ ( 0 ) = 0 , ∥ ψ ∥ ∞ , [ − R , R ] ≤ R , ∫ ψ d ν = W . \psi(0)=0,\qquad\lVert\psi\rVert_{\infty,[-R,R]}\le R,\qquad\int\psi\,\mathrm{d}\nu=W. ψ ( 0 ) = 0 , ∥ ψ ∥ ∞ , [ − R , R ] ≤ R , ∫ ψ d ν = W .
As constant extension beyond the endpoints preserves the one-Lipschitz property, we can extend ψ \psi ψ to R \mathbb{R} R by setting ψ ( x ) = ψ ( − R ) \psi(x)=\psi(-R) ψ ( x ) = ψ ( − R ) for x < − R x<-R x < − R and ψ ( x ) = ψ ( R ) \psi(x)=\psi(R) ψ ( x ) = ψ ( R ) for x > R x>R x > R .
Having bounded the original Wasserstein distance by its one-dimensional counterpart, we now show that polynomial moments of the projected measures are controlled by the original discrepancy M = M 2 K − 1 ( H , H ⋆ ) M=\mathcal{M}_{2K-1}(H,H^\star) M = M 2 K − 1 ( H , H ⋆ ) . For every univariate polynomial P P P of degree at most 2 K − 1 2K-1 2 K − 1 , the polynomial Q ( x ) = P ( ⟨ u , x ⟩ ) Q(x)=P(\langle u,x\rangle) Q ( x ) = P (⟨ u , x ⟩) satisfies
∥ Q ∥ ∞ , B d ( 0 , R ) ≤ ∥ P ∥ ∞ , [ − R , R ] , ∫ Q d ( H − H ⋆ ) = ∫ P d ν . \lVert Q\rVert_{\infty,B_d(0,R)}\le\lVert P\rVert_{\infty,[-R,R]},\qquad\int Q\,\mathrm{d}(H-H^\star)=\int P\,\mathrm{d}\nu. ∥ Q ∥ ∞ , B d ( 0 , R ) ≤ ∥ P ∥ ∞ , [ − R , R ] , ∫ Q d ( H − H ⋆ ) = ∫ P d ν .
Consequently,
∣ ∫ P d ν ∣ ≤ ∥ P ∥ ∞ , [ − R , R ] M . (119) \left|\int P\,d\nu\right| \le\|P\|_{\infty,[-R,R]}M. \tag*{(119)} ∫ P d ν ≤ ∥ P ∥ ∞ , [ − R , R ] M . (119)
Step 2: We bound the interpolating polynomial when every three consecutive projected nodes have a fixed minimum span. Let z 1 < ⋯ < z N z_1 < \cdots< z_N z 1 < ⋯ < z N , N ≤ 2 K N \le2K N ≤ 2 K , be the distinct projected union nodes, and suppose
z i + 2 − z i ≥ γ > 0 , i = 1 , … , N − 2. z_{i+2}-z_i \ge\gamma> 0,\qquad i=1,\ldots,N-2. z i + 2 − z i ≥ γ > 0 , i = 1 , … , N − 2.
Put γ ˉ = 1 ∧ γ \bar{\gamma}=1\wedge\gamma γ ˉ = 1 ∧ γ . The Lipschitz property gives ∣ ψ [ z i , z i + 1 ] ∣ ≤ 1 |\psi[z_i,z_{i+1}]|\le1 ∣ ψ [ z i , z i + 1 ] ∣ ≤ 1 . For q ≥ 2 q\ge2 q ≥ 2 , the recursion
ψ [ z i , … , z i + q ] = ψ [ z i + 1 , … , z i + q ] − ψ [ z i , … , z i + q − 1 ] z i + q − z i \psi[z_i,\ldots,z_{i+q}]
=
\frac{\psi[z_{i+1},\ldots,z_{i+q}]-\psi[z_i,\ldots,z_{i+q-1}]}{z_{i+q}-z_i} ψ [ z i , … , z i + q ] = z i + q − z i ψ [ z i + 1 , … , z i + q ] − ψ [ z i , … , z i + q − 1 ]
therefore implies ∣ ψ [ z i , … , z i + q ] ∣ ≤ 2 q − 1 γ ˉ − ( q − 1 ) |\psi[z_i,\ldots,z_{i+q}]|\le2^{q-1}\bar{\gamma}^{-(q-1)} ∣ ψ [ z i , … , z i + q ] ∣ ≤ 2 q − 1 γ ˉ − ( q − 1 ) . The Newton interpolant
P ( z ) : = ψ ( z 1 ) + ∑ q = 1 N − 1 ψ [ z 1 , … , z q + 1 ] ∏ ℓ = 1 q ( z − z ℓ ) P(z):=\psi(z_1)+\sum_{q=1}^{N-1}\psi[z_1,\ldots,z_{q+1}]\prod_{\ell=1}^{q}(z-z_\ell) P ( z ) := ψ ( z 1 ) + q = 1 ∑ N − 1 ψ [ z 1 , … , z q + 1 ] ℓ = 1 ∏ q ( z − z ℓ )
has degree at most 2 K − 1 2K-1 2 K − 1 and satisfies
∥ P ∥ ∞ , [ − R , R ] ≤ R + ∑ q = 1 N − 1 2 q − 1 γ ˉ − ( q − 1 ) ( 2 R ) q ≤ C ( K , R ) γ ˉ − ( 2 K − 2 ) . \|P\|_{\infty,[-R,R]}
\le R+\sum_{q=1}^{N-1}2^{q-1}\bar{\gamma}^{-(q-1)}(2R)^q
\le C(K,R)\bar{\gamma}^{-(2K-2)}. ∥ P ∥ ∞ , [ − R , R ] ≤ R + q = 1 ∑ N − 1 2 q − 1 γ ˉ − ( q − 1 ) ( 2 R ) q ≤ C ( K , R ) γ ˉ − ( 2 K − 2 ) .
The same bound holds for N ≤ 2 N\le2 N ≤ 2 , without a three-node condition. Since P = ψ P=\psi P = ψ on the support of ν \nu ν , (D.12) yields
W ≤ C ( K , R ) ( 1 ∧ γ ) − ( 2 K − 2 ) M . (120) W\le C(K,R)(1\wedge\gamma)^{-(2K-2)}M. \tag*{(120)} W ≤ C ( K , R ) ( 1 ∧ γ ) − ( 2 K − 2 ) M . (120)
Step 3: We verify the three-node condition under the matching hypothesis. For i ≠ j i\ne j i = j , z ∈ { a i , b i } z\in\{a_i,b_i\} z ∈ { a i , b i } , and z ′ ∈ { a j , b j } z'\in\{a_j,b_j\} z ′ ∈ { a j , b j } ,
∥ z − z ′ ∥ 2 ≥ ∥ a i − a j ∥ 2 − ∥ z − a i ∥ 2 − ∥ z ′ − a j ∥ 2 ≥ Δ 2 . \|z-z'\|_2\ge\|a_i-a_j\|_2-\|z-a_i\|_2-\|z'-a_j\|_2\ge\frac{\Delta}{2}. ∥ z − z ′ ∥ 2 ≥ ∥ a i − a j ∥ 2 − ∥ z − a i ∥ 2 − ∥ z ′ − a j ∥ 2 ≥ 2 Δ .
Thus points from different pairs have projected distance at least c K Δ / 2 c_K\Delta/2 c K Δ/2 . Let z 1 < ⋯ < z N z_1<\cdots<z_N z 1 < ⋯ < z N be the distinct projected union nodes. Any three consecutive nodes include two from different pairs, since each pair contains at most two distinct points. Their distance is bounded above by the span of the triple, so
z ℓ + 2 − z ℓ ≥ c K Δ / 2 , ℓ = 1 , … , N − 2. z_{\ell+2}-z_\ell\ge c_K\Delta/2,\qquad\ell=1,\ldots,N-2. z ℓ + 2 − z ℓ ≥ c K Δ/2 , ℓ = 1 , … , N − 2.
Applying (D.13) with γ = c K Δ / 2 \gamma=c_K\Delta/2 γ = c K Δ/2 and then Step 1 gives
W 1 ( H , H ⋆ ) ≤ c K − 1 W ≤ C ( K , R ) Δ − ( 2 K − 2 ) M 2 K − 1 ( H , H ⋆ ) , W_1(H,H^\star)\le c_K^{-1}W\le C(K,R)\Delta^{-(2K-2)}\mathcal{M}_{2K-1}(H,H^\star), W 1 ( H , H ⋆ ) ≤ c K − 1 W ≤ C ( K , R ) Δ − ( 2 K − 2 ) M 2 K − 1 ( H , H ⋆ ) ,
where factors depending only on K K K are absorbed into C ( K , R ) C(K,R) C ( K , R ) . This proves (D.9).
Step 4: We establish the moment bound for arbitrary fitted centers when min j w j ≥ p 0 \min_j w_j\ge p_0 min j w j ≥ p 0 . Define the projected true and fitted centers by α j = ⟨ u , a j ⟩ \alpha_j=\langle u,a_j\rangle α j = ⟨ u , a j ⟩ and β i = ⟨ u , b i ⟩ \beta_i=\langle u,b_i\rangle β i = ⟨ u , b i ⟩ , so that
H ˉ = ∑ i = 1 K v i δ β i , H ˉ ⋆ = ∑ j = 1 K w j δ α j , δ u = min i ≠ j ∣ α i − α j ∣ ≥ c K Δ . \bar{H}=\sum_{i=1}^{K}v_i\delta_{\beta_i},\qquad
\bar{H}^{\star}=\sum_{j=1}^{K}w_j\delta_{\alpha_j},\qquad
\delta_u=\min_{i\ne j}|\alpha_i-\alpha_j|\ge c_K\Delta. H ˉ = i = 1 ∑ K v i δ β i , H ˉ ⋆ = j = 1 ∑ K w j δ α j , δ u = i = j min ∣ α i − α j ∣ ≥ c K Δ.
We distinguish two cases. If W < p 0 δ u / 8 W < p_0\delta_u/8 W < p 0 δ u /8 , every interval I j = ( α j − δ u / 4 , α j + δ u / 4 ) I_j = (\alpha_j - \delta_u/4, \alpha_j + \delta_u/4) I j = ( α j − δ u /4 , α j + δ u /4 ) contains a projected fitted center β i \beta_i β i with v i > 0 v_i > 0 v i > 0 . Otherwise, every coupling must transport the true mass w j w_j w j at α j \alpha_j α j by distance at least δ u / 4 \delta_u/4 δ u /4 , giving W ≥ w j δ u / 4 ≥ p 0 δ u / 4 W \ge w_j\delta_u/4 \ge p_0\delta_u/4 W ≥ w j δ u /4 ≥ p 0 δ u /4 , a contradiction. The K K K intervals are disjoint, and there are only K K K fitted labels. Thus each interval contains exactly one fitted label β j \beta_j β j . After relabeling, we can suppose ∣ β j − α j ∣ < δ u / 4 |\beta_j - \alpha_j| < \delta_u/4 ∣ β j − α j ∣ < δ u /4 for every j j j . For points ξ ∈ { α i , β i } \xi\in\{\alpha_i, \beta_i\} ξ ∈ { α i , β i } and ξ ′ ∈ { α j , β j } \xi' \in\{\alpha_j, \beta_j\} ξ ′ ∈ { α j , β j } with i ≠ j i \ne j i = j ,
∣ ξ − ξ ′ ∣ ≥ ∣ α i − α j ∣ − ∣ ξ − α i ∣ − ∣ ξ ′ − α j ∣ ≥ δ u / 2. |\xi- \xi'| \ge|\alpha_i - \alpha_j| - |\xi- \alpha_i| - |\xi' - \alpha_j| \ge\delta_u/2. ∣ ξ − ξ ′ ∣ ≥ ∣ α i − α j ∣ − ∣ ξ − α i ∣ − ∣ ξ ′ − α j ∣ ≥ δ u /2.
As in Step 3, every three consecutive distinct union nodes therefore span at least δ u / 2 \delta_u/2 δ u /2 . Step 2 and the fact that δ u ≥ c K Δ \delta_u \ge c_K\Delta δ u ≥ c K Δ yield
W ≤ C ( K , R ) Δ − ( 2 K − 2 ) M . W \le C(K,R)\Delta^{-(2K-2)}M. W ≤ C ( K , R ) Δ − ( 2 K − 2 ) M .
If W ≥ p 0 δ u / 8 W \ge p_0\delta_u/8 W ≥ p 0 δ u /8 , we smooth the Lipschitz dual function before interpolating, thereby avoiding any spacing requirement on the fitted centers. Choose a nonnegative C ∞ C^\infty C ∞ density ζ \zeta ζ supported on [ − 1 , 1 ] [-1,1] [ − 1 , 1 ] , and set
η = p 0 c K Δ 64 , ζ η ( x ) = η − 1 ζ ( x / η ) , ψ η = ψ ∗ ζ η . \eta= \frac{p_0c_K\Delta}{64}, \qquad\zeta_\eta(x) = \eta^{-1}\zeta(x/\eta), \qquad\psi_\eta= \psi* \zeta_\eta. η = 64 p 0 c K Δ , ζ η ( x ) = η − 1 ζ ( x / η ) , ψ η = ψ ∗ ζ η .
Since ψ \psi ψ is one-Lipschitz,
∣ ψ η ( x ) − ψ ( x ) ∣ ≤ ∫ ∣ ψ ( x − y ) − ψ ( x ) ∣ ζ η ( y ) d y ≤ η . |\psi_\eta(x) - \psi(x)| \le\int|\psi(x-y) - \psi(x)|\zeta_\eta(y)\,dy \le\eta. ∣ ψ η ( x ) − ψ ( x ) ∣ ≤ ∫ ∣ ψ ( x − y ) − ψ ( x ) ∣ ζ η ( y ) d y ≤ η .
Moreover, W ≥ p 0 c K Δ / 8 = 8 η W \ge p_0c_K\Delta/8 = 8\eta W ≥ p 0 c K Δ/8 = 8 η and ∥ ν ∥ T V ≤ 2 \|\nu\|_{\mathrm{TV}} \le2 ∥ ν ∥ TV ≤ 2 , so
∫ ψ η d ν ≥ ∫ ψ d ν − 2 η = W − 2 η ≥ 3 4 W . \int\psi_\eta\,d\nu\ge\int\psi\,d\nu- 2\eta= W - 2\eta\ge\frac{3}{4}W. ∫ ψ η d ν ≥ ∫ ψ d ν − 2 η = W − 2 η ≥ 4 3 W .
The weak derivative of ψ \psi ψ satisfies ∥ ψ ′ ∥ ∞ ≤ 1 \|\psi'\|_\infty\le1 ∥ ψ ′ ∥ ∞ ≤ 1 . Hence
∥ ψ η ∥ ∞ , [ − R , R ] ≤ R + 1 , ∥ ψ η ( q ) ∥ ∞ ≤ ∥ ψ ′ ∥ ∞ ∥ ζ η ( q − 1 ) ∥ 1 ≤ C q η − ( q − 1 ) , q ≥ 1. \|\psi_\eta\|_{\infty,[-R,R]} \le R+1, \qquad\|\psi_\eta^{(q)}\|_\infty\le\|\psi'\|_\infty\|\zeta_\eta^{(q-1)}\|_1 \le C_q\eta^{-(q-1)}, \qquad q \ge1. ∥ ψ η ∥ ∞ , [ − R , R ] ≤ R + 1 , ∥ ψ η ( q ) ∥ ∞ ≤ ∥ ψ ′ ∥ ∞ ∥ ζ η ( q − 1 ) ∥ 1 ≤ C q η − ( q − 1 ) , q ≥ 1.
Lemma C.4 therefore supplies a polynomial P η P_\eta P η interpolating ψ η \psi_\eta ψ η at the distinct projected union nodes, with
deg P η ≤ 2 K − 1 , ∥ P η ∥ ∞ , [ − R , R ] ≤ C ( K , R , p 0 ) Δ − ( 2 K − 2 ) . \deg P_\eta\le2K-1, \qquad\|P_\eta\|_{\infty,[-R,R]} \le C(K,R,p_0)\Delta^{-(2K-2)}. deg P η ≤ 2 K − 1 , ∥ P η ∥ ∞ , [ − R , R ] ≤ C ( K , R , p 0 ) Δ − ( 2 K − 2 ) .
Because ν \nu ν is supported on these nodes,
3 4 W ≤ ∫ ψ η d ν = ∫ P η d ν ≤ C ( K , R , p 0 ) Δ − ( 2 K − 2 ) M . \frac{3}{4}W \le\int\psi_\eta\,d\nu= \int P_\eta\,d\nu\le C(K,R,p_0)\Delta^{-(2K-2)}M. 4 3 W ≤ ∫ ψ η d ν = ∫ P η d ν ≤ C ( K , R , p 0 ) Δ − ( 2 K − 2 ) M .
Combining both cases with Step 1 proves
W 1 ( H , H ⋆ ) ≤ C ( K , R , p 0 ) Δ − ( 2 K − 2 ) M 2 K − 1 ( H , H ⋆ ) . W_1(H,H^\star) \le C(K,R,p_0)\Delta^{-(2K-2)}\mathcal{M}_{2K-1}(H,H^\star). W 1 ( H , H ⋆ ) ≤ C ( K , R , p 0 ) Δ − ( 2 K − 2 ) M 2 K − 1 ( H , H ⋆ ) .
Step 5: We derive the density comparisons before treating K = 1 K=1 K = 1 . By restricting each test polynomial to the span of the true and fitted centers, whose dimension is at most 2 K 2K 2 K , we use Lemma A.3 to achieve
∣ ∫ Q d ( H − H ⋆ ) ∣ ≤ C ( K , R ) h ( p H , p H ⋆ ) \left|\int Q\,d(H-H^\star)\right| \le C(K,R)h(p_H,p_{H^\star}) ∫ Q d ( H − H ⋆ ) ≤ C ( K , R ) h ( p H , p H ⋆ )
whenever deg Q ≤ 2 K − 1 \deg Q \le2K-1 deg Q ≤ 2 K − 1 and ∥ Q ∥ ∞ , B d ( 0 , R ) ≤ 1 \|Q\|_{\infty,B_d(0,R)} \le1 ∥ Q ∥ ∞ , B d ( 0 , R ) ≤ 1 . Taking the supremum over Q Q Q yields M ≤ C ( K , R ) h ( p H , p H ⋆ ) M \le C(K,R)h(p_H,p_{H^\star}) M ≤ C ( K , R ) h ( p H , p H ⋆ ) . Substitution into the moment bounds (D.9) proves (D.10) and (D.11).
If K = 1 K=1 K = 1 , both measures are point masses. For R > 0 R>0 R > 0 and a 1 ≠ b 1 a_1\ne b_1 a 1 = b 1 , take
Q ( x ) = ⟨ b 1 − a 1 , x ⟩ R ∥ b 1 − a 1 ∥ 2 . Q(x)=\frac{\langle b_1-a_1,x\rangle}{R\lVert b_1-a_1\rVert_2}. Q ( x ) = R ∥ b 1 − a 1 ∥ 2 ⟨ b 1 − a 1 , x ⟩ .
Then ∥ Q ∥ ∞ , B d ( 0 , R ) ≤ 1 \lVert Q\rVert_{\infty,B_d(0,R)}\le1 ∥ Q ∥ ∞ , B d ( 0 , R ) ≤ 1 and
M 1 ( H , H ⋆ ) ≥ ∣ ∫ Q d ( H − H ⋆ ) ∣ = ∥ b 1 − a 1 ∥ 2 R . \mathcal{M}_1(H,H^\star)\ge\left|\int Q\,\mathrm{d}(H-H^\star)\right|=\frac{\lVert b_1-a_1\rVert_2}{R}. M 1 ( H , H ⋆ ) ≥ ∫ Q d ( H − H ⋆ ) = R ∥ b 1 − a 1 ∥ 2 .
Thus W 1 ( H , H ⋆ ) ≤ R M 1 ( H , H ⋆ ) W_1(H,H^\star)\le R\mathcal{M}_1(H,H^\star) W 1 ( H , H ⋆ ) ≤ R M 1 ( H , H ⋆ ) . If the centers coincide or R = 0 R=0 R = 0 , both sides vanish.
Lemma D.5 (Cluster-adaptive global mixture transport). Under the assumptions of the global assertion of Lemma D.4 , suppose additionally that the true centers have the structured geometry. Then
W 1 ( H , H ⋆ ) ≤ C ( p 0 ) Δ s e p − ( 2 s max − 2 ) h ( p H , p H ⋆ ) . (121) W_1(H,H^\star)\le C(p_0)\Delta_{\mathrm{sep}}^{-(2s_{\max}-2)}h(p_H,p_{H^\star}). \tag*{(121)} W 1 ( H , H ⋆ ) ≤ C ( p 0 ) Δ sep − ( 2 s m a x − 2 ) h ( p H , p H ⋆ ) . (121)
The constant may depend on K K K , R R R and the cluster parameters, as well as on p 0 p_0 p 0 , but not on the fitted weights or the ambient dimension.
Proof. Write Δ = Δ s e p \Delta=\Delta_{\mathrm{sep}} Δ = Δ sep , s = s max s=s_{\max} s = s m a x , and h = h ( p H , p H ⋆ ) h=h(p_H,p_{H^\star}) h = h ( p H , p H ⋆ ) . All constants below may depend on K K K , R R R and the fixed cluster parameters.
Step 1: We construct disjoint groups containing the true clusters whose boundaries stay uniformly separated from both supports. If there is only one cluster, then s = K s=K s = K and Lemma D.4 proves the assertion. That lemma also proves the assertion when Δ \Delta Δ is bounded below by a fixed positive constant. We may therefore assume C 0 Δ < D 0 / 16 C_0\Delta<D_0/16 C 0 Δ < D 0 /16 .
Let C 1 , … , C M C_1,\ldots,C_M C 1 , … , C M be the true clusters, and choose a j m ∈ C m a_{jm}\in C_m a j m ∈ C m for each m m m . Partition [ D 0 / 8 , D 0 / 4 ] [D_0/8,D_0/4] [ D 0 /8 , D 0 /4 ] into K + 1 K+1 K + 1 equal subintervals. For fixed a j m a_{jm} a j m , since there are only K K K fitted labels, one subinterval contains no distance among { ∥ b i − a j m ∥ 2 , 1 ≤ i ≤ K } \{\lVert b_i-a_{jm}\rVert_2,1\le i\le K\} {∥ b i − a j m ∥ 2 , 1 ≤ i ≤ K } in its interior. Let ρ m \rho_m ρ m be its midpoint and define A m = B d ( a j m , ρ m ) A_m=B_d(a_{jm},\rho_m) A m = B d ( a j m , ρ m ) . The subinterval length is D 0 / [ 8 ( K + 1 ) ] D_0/[8(K+1)] D 0 / [ 8 ( K + 1 )] , so
∣ ∥ b i − a j m ∥ 2 − ρ m ∣ ≥ D 0 16 ( K + 1 ) for every i . \left|\lVert b_i-a_{jm}\rVert_2-\rho_m\right|\ge\frac{D_0}{16(K+1)}
\qquad\text{for every }i. ∣ ∥ b i − a j m ∥ 2 − ρ m ∣ ≥ 16 ( K + 1 ) D 0 for every i .
Recall from assumption in Equation (13) that the true clusters have diameter at most C 0 Δ < D 0 / 16 C_0\Delta<D_0/16 C 0 Δ < D 0 /16 and are separated by at least D 0 D_0 D 0 . Since ρ m ∈ [ D 0 / 8 , D 0 / 4 ] \rho_m\in[D_0/8,D_0/4] ρ m ∈ [ D 0 /8 , D 0 /4 ] , it follows that a j ∈ A m a_j\in A_m a j ∈ A m if and only if j ∈ C m j\in C_m j ∈ C m , and the balls A m A_m A m are pairwise disjoint. For the set of distinct union centers Z = { a 1 , … , a K , b 1 , … , b K } \mathcal{Z}=\{a_1,\ldots,a_K,b_1,\ldots,b_K\} Z = { a 1 , … , a K , b 1 , … , b K } , there is consequently a fixed g > 0 g>0 g > 0 such that ∥ z − z ′ ∥ 2 ≥ g \lVert z-z'\rVert_2\ge g ∥ z − z ′ ∥ 2 ≥ g whenever z ∈ Z ∩ A m z\in\mathcal{Z}\cap A_m z ∈ Z ∩ A m and z ′ ∈ Z ∖ A m z'\in\mathcal{Z}\setminus A_m z ′ ∈ Z ∖ A m .
Step 2: We construct bounded polynomial selectors and compare the fitted and true masses of each group. Choose a projection direction u u u from Lemma C.3 , and put E m = { ⟨ u , z ⟩ : z ∈ Z ∩ A m } E_m=\{\langle u,z\rangle:z\in\mathcal{Z}\cap A_m\} E m = {⟨ u , z ⟩ : z ∈ Z ∩ A m } . Projected nodes in E m E_m E m are separated from all other projected union nodes by at least g ′ = c K g g'=c_Kg g ′ = c K g . Let O m = { t ∈ R : dist ( t , E m ) < g ′ / 3 } O_m=\{t\in\mathbb{R}:\operatorname{dist}(t,E_m)<g'/3\} O m = { t ∈ R : dist ( t , E m ) < g ′ /3 } ,
f m = 1 O m ∗ ζ g ′ / 6 , f_m=1_{O_m}*\zeta_{g'/6}, f m = 1 O m ∗ ζ g ′ /6 ,
where ζ ε \zeta_\varepsilon ζ ε is a fixed smooth probability density supported on [ − ε , ε ] [-\varepsilon,\varepsilon] [ − ε , ε ] . Then f m f_m f m equals one on E m E_m E m and zero on every other projected union node, and ∥ f m ( q ) ∥ ∞ ≤ C q \lVert f_m^{(q)}\rVert_\infty\le C_q ∥ f m ( q ) ∥ ∞ ≤ C q , where q ≥ 0 q\ge0 q ≥ 0 . Interpolating f m f_m f m on the distinct projected union nodes using Lemma C.4 , and lifting the interpolant back to R d \mathbb{R}^d R d , gives a polynomial F m F_m F m with
deg F m ≤ 2 K − 1 , ∥ F m ∥ ∞ , B d ( 0 , R ) ≤ C , F m ( z ) = 1 { z ∈ A m } ( z ∈ Z ) . \deg F_m\le2K-1,\qquad\lVert F_m\rVert_{\infty,B_d(0,R)}\le C,\qquad F_m(z)=1_{\{z\in A_m\}}\quad(z\in\mathcal{Z}). deg F m ≤ 2 K − 1 , ∥ F m ∥ ∞ , B d ( 0 , R ) ≤ C , F m ( z ) = 1 { z ∈ A m } ( z ∈ Z ) .
Define v m = ∑ i : b i ∈ A m v i v_m=\sum_{i:b_i\in A_m}v_i v m = ∑ i : b i ∈ A m v i , and w m = ∑ j ∈ C m w j w_m=\sum_{j\in C_m}w_j w m = ∑ j ∈ C m w j . Moment extraction on the span of the union centers, whose dimension is at most 2 K 2K 2 K , yields
∣ v m − w m ∣ = ∣ ∫ F m d ( H − H ⋆ ) ∣ ≤ C h . (122) \left|v_m-w_m\right|=\left|\int F_m\,d(H-H^\star)\right|\le Ch. \tag*{(122)} ∣ v m − w m ∣ = ∫ F m d ( H − H ⋆ ) ≤ C h . (122)
Step 3: We show that either the desired bound follows immediately or each group contains exactly as many fitted labels as true labels. Write k m = # { i : b i ∈ A m } k_m=\#\{i:b_i\in A_m\} k m = # { i : b i ∈ A m } , counting repeated fitted centers and zero-weight labels, and s m = ∣ C m ∣ s_m=|C_m| s m = ∣ C m ∣ . If k m < s m k_m<s_m k m < s m for some m m m , we define
Q m ( x ) = F m ( x ) ∏ i : b i ∈ A m ∥ x − b i ∥ 2 2 . Q_m(x)=F_m(x)\prod_{i:b_i\in A_m}\|x-b_i\|_2^2. Q m ( x ) = F m ( x ) i : b i ∈ A m ∏ ∥ x − b i ∥ 2 2 .
This polynomial has uniformly bounded degree and norm, and Q m ( b i ) = 0 Q_m(b_i)=0 Q m ( b i ) = 0 for every fitted label. The s m s_m s m balls { B d ( a j , Δ / 3 ) , j ∈ C m } \{B_d(a_j,\Delta/3),\ j\in C_m\} { B d ( a j , Δ/3 ) , j ∈ C m } are disjoint. Since k m < s m k_m<s_m k m < s m , some a j 0 a_{j_0} a j 0 has distance at least Δ / 3 \Delta/3 Δ/3 from every fitted center in A m A_m A m . Therefore,
∫ Q m d H ⋆ = ∑ j ∈ C m w j ∏ i : b i ∈ A m ∥ a j − b i ∥ 2 2 ≥ p 0 ( Δ / 3 ) 2 k m ≥ c p 0 Δ 2 s − 2 . \int Q_m\,dH^\star=\sum_{j\in C_m}w_j\prod_{i:b_i\in A_m}\|a_j-b_i\|_2^2\ge p_0(\Delta/3)^{2k_m}\ge cp_0\Delta^{2s-2}. ∫ Q m d H ⋆ = j ∈ C m ∑ w j i : b i ∈ A m ∏ ∥ a j − b i ∥ 2 2 ≥ p 0 ( Δ/3 ) 2 k m ≥ c p 0 Δ 2 s − 2 .
Here the last inequality uses k m ≤ s m − 1 ≤ s − 1 k_m\le s_m-1\le s-1 k m ≤ s m − 1 ≤ s − 1 and 0 < Δ ≤ 1 0<\Delta\le1 0 < Δ ≤ 1 . Since ∫ Q m d H = 0 \int Q_m\,dH=0 ∫ Q m d H = 0 , moment extraction implies h ≥ c p 0 Δ 2 s − 2 h\ge cp_0\Delta^{2s-2} h ≥ c p 0 Δ 2 s − 2 . In that case, the trivial transport bound gives
W 1 ( H , H ⋆ ) ≤ 2 R ≤ C ( p 0 ) Δ − ( 2 s − 2 ) h . W_1(H,H^\star)\le2R\le C(p_0)\Delta^{-(2s-2)}h. W 1 ( H , H ⋆ ) ≤ 2 R ≤ C ( p 0 ) Δ − ( 2 s − 2 ) h .
Thus, we can assume that h < c 0 p 0 Δ 2 s − 2 h<c_0p_0\Delta^{2s-2} h < c 0 p 0 Δ 2 s − 2 for a sufficiently small fixed c 0 > 0 c_0>0 c 0 > 0 . In this case, k m ≥ s m k_m\ge s_m k m ≥ s m for every m m m . Since
K ≥ ∑ m k m ≥ ∑ m s m = K , K\ge\sum_m k_m\ge\sum_m s_m=K, K ≥ m ∑ k m ≥ m ∑ s m = K ,
we have k m = s m k_m=s_m k m = s m for all m m m , and no fitted labels lie outside the groups. By (D.15), choosing c 0 c_0 c 0 smaller if necessary also gives
v m ≥ w m − C h ≥ w m / 2 ≥ p 0 / 2. v_m\ge w_m-Ch\ge w_m/2\ge p_0/2. v m ≥ w m − C h ≥ w m /2 ≥ p 0 /2.
Step 4: We bound transport between the normalized restrictions within each group. Define
H m = H ∣ A m v m , H m ⋆ = H ⋆ ∣ A m w m . H_m=\frac{H|_{A_m}}{v_m},\qquad H_m^\star=\frac{H^\star|_{A_m}}{w_m}. H m = v m H ∣ A m , H m ⋆ = w m H ⋆ ∣ A m .
For every polynomial Q Q Q of degree at most 2 s m − 1 2s_m-1 2 s m − 1 , the exact selector values imply
∫ Q d ( H m − H m ⋆ ) = 1 v m ∫ F m Q d ( H − H ⋆ ) + w m − v m v m ∫ Q d H m ⋆ . \int Q\,d(H_m-H_m^\star)=\frac{1}{v_m}\int F_mQ\,d(H-H^\star)+\frac{w_m-v_m}{v_m}\int Q\,dH_m^\star. ∫ Q d ( H m − H m ⋆ ) = v m 1 ∫ F m Q d ( H − H ⋆ ) + v m w m − v m ∫ Q d H m ⋆ .
Since deg ( F m Q ) ≤ 2 K + 2 s m − 2 \deg(F_mQ)\le2K+2s_m-2 deg ( F m Q ) ≤ 2 K + 2 s m − 2 and ∥ F m Q ∥ ∞ ≤ C ∥ Q ∥ ∞ \|F_mQ\|_\infty\le C\|Q\|_\infty ∥ F m Q ∥ ∞ ≤ C ∥ Q ∥ ∞ , moment extraction and (D.15) yield
∣ ∫ Q d ( H m − H m ⋆ ) ∣ ≤ C ( p 0 ) ∥ Q ∥ ∞ h . \left|\int Q\,d(H_m-H_m^\star)\right|\le C(p_0)\|Q\|_\infty h. ∫ Q d ( H m − H m ⋆ ) ≤ C ( p 0 ) ∥ Q ∥ ∞ h .
Consequently,
M 2 s m − 1 ( H m , H m ⋆ ) ≤ C ( p 0 ) h . \mathcal{M}_{2s_m-1}(H_m,H_m^\star)\le C(p_0)h. M 2 s m − 1 ( H m , H m ⋆ ) ≤ C ( p 0 ) h .
Each normalized true weight is at least p 0 p_0 p 0 , since w j / w m ≥ w j ≥ p 0 w_j/w_m \ge w_j \ge p_0 w j / w m ≥ w j ≥ p 0 . For s m ≥ 2 s_m \ge2 s m ≥ 2 , the global moment bound in Lemma D.4 therefore gives
W 1 ( H m , H m ⋆ ) ≤ C ( p 0 ) Δ − ( 2 s m − 2 ) h ≤ C ( p 0 ) Δ − ( 2 s − 2 ) h . W_1(H_m,H_m^\star) \le C(p_0)\Delta^{-(2s_m-2)}h \le C(p_0)\Delta^{-(2s-2)}h. W 1 ( H m , H m ⋆ ) ≤ C ( p 0 ) Δ − ( 2 s m − 2 ) h ≤ C ( p 0 ) Δ − ( 2 s − 2 ) h .
For s m = 1 s_m=1 s m = 1 , the one-component assertion of that lemma gives the same conclusion.
Step 5: We combine the within-group transport bounds and account for the group-mass discrepancies. For each m m m , multiply an optimal coupling of H m , H m ⋆ H_m,H_m^\star H m , H m ⋆ by min { v m , w m } \min\{v_m,w_m\} min { v m , w m } . The residual marginals have equal total mass
τ = 1 2 ∑ m ∣ v m − w m ∣ \tau=\frac{1}{2}\sum_m |v_m-w_m| τ = 2 1 m ∑ ∣ v m − w m ∣
and can be coupled at cost at most 2 R τ 2R\tau 2 R τ . Thus
W 1 ( H , H ⋆ ) ≤ ∑ m min { v m , w m } W 1 ( H m , H m ⋆ ) + R ∑ m ∣ v m − w m ∣ ≤ C ( p 0 ) Δ − ( 2 s − 2 ) h + C h ≤ C ( p 0 ) Δ − ( 2 s − 2 ) h . \begin{aligned}
W_1(H,H^\star) &\le\sum_m \min\{v_m,w_m\}W_1(H_m,H_m^\star)+R\sum_m|v_m-w_m|\\
&\le C(p_0)\Delta^{-(2s-2)}h+Ch\le C(p_0)\Delta^{-(2s-2)}h.
\end{aligned} W 1 ( H , H ⋆ ) ≤ m ∑ min { v m , w m } W 1 ( H m , H m ⋆ ) + R m ∑ ∣ v m − w m ∣ ≤ C ( p 0 ) Δ − ( 2 s − 2 ) h + C h ≤ C ( p 0 ) Δ − ( 2 s − 2 ) h .
This proves (D.14). All moment comparisons take place on a subspace of dimension at most 2 K 2K 2 K , so the constants are independent of the ambient dimension and the fitted weights.
Proof of Theorem 3.1 . Put p 0 = π min ⋆ p_0=\pi_{\min}^\star p 0 = π m i n ⋆ , Δ = Δ s e p \Delta=\Delta_{\mathrm{sep}} Δ = Δ sep , and h = h ( p θ , T , p θ , T ⋆ ) h=h(p_{\theta,T},p_{\theta,T}^\star) h = h ( p θ , T , p θ , T ⋆ ) . Throughout the proof, set s = s max s=s_{\max} s = s m a x under the structured geometry and s = K s=K s = K without cluster assumptions.
Step 1: We bound the one-time transport errors W 1 ( G θ { t } , G ⋆ { t } ) W_1(G_\theta^{\{t\}},G_\star^{\{t\}}) W 1 ( G θ { t } , G ⋆ { t } ) for t ≤ T t\le T t ≤ T without imposing positivity of the later-time true state probabilities. For each t ≤ T t\le T t ≤ T , define
H t = 1 2 ( G θ { 1 } + G θ { t } ) , H t ⋆ = 1 2 ( G ⋆ { 1 } + G ⋆ { t } ) . H_t=\frac{1}{2}\left(G_\theta^{\{1\}}+G_\theta^{\{t\}}\right),\qquad H_t^\star=\frac{1}{2}\left(G_\star^{\{1\}}+G_\star^{\{t\}}\right). H t = 2 1 ( G θ { 1 } + G θ { t } ) , H t ⋆ = 2 1 ( G ⋆ { 1 } + G ⋆ { t } ) .
The weight of a j a_j a j in H t ⋆ H_t^\star H t ⋆ is 1 2 ( π j ⋆ + q t ⋆ ( j ) ) ≥ p 0 / 2 \frac{1}{2}(\pi_j^\star+q_t^\star(j))\ge p_0/2 2 1 ( π j ⋆ + q t ⋆ ( j )) ≥ p 0 /2 . By joint convexity of squared Hellinger distance and contraction under marginalization,
h 2 ( p H t , p H t ⋆ ) ≤ 1 2 h 2 ( p θ { 1 } , p ⋆ { 1 } ) + 1 2 h 2 ( p θ { t } , p ⋆ { t } ) ≤ h 2 . h^2(p_{H_t},p_{H_t^\star})\le\frac{1}{2}h^2(p_\theta^{\{1\}},p_\star^{\{1\}})+\frac{1}{2}h^2(p_\theta^{\{t\}},p_\star^{\{t\}})\le h^2. h 2 ( p H t , p H t ⋆ ) ≤ 2 1 h 2 ( p θ { 1 } , p ⋆ { 1 } ) + 2 1 h 2 ( p θ { t } , p ⋆ { t } ) ≤ h 2 .
Under structured geometry, applying Lemma D.5 separately to the averaged and initial mixtures, then adding the bounds, gives
W 1 ( H t , H t ⋆ ) + W 1 ( G θ { 1 } , G ⋆ { 1 } ) ≤ C ( p 0 ) Δ − ( 2 s − 2 ) h . W_1(H_t,H_t^\star)+W_1(G_\theta^{\{1\}},G_\star^{\{1\}})\le C(p_0)\Delta^{-(2s-2)}h. W 1 ( H t , H t ⋆ ) + W 1 ( G θ { 1 } , G ⋆ { 1 } ) ≤ C ( p 0 ) Δ − ( 2 s − 2 ) h .
Since G θ { t } − G ⋆ { t } = 2 ( H t − H t ⋆ ) − ( G θ { 1 } − G ⋆ { 1 } ) G_\theta^{\{t\}}-G_\star^{\{t\}}=2(H_t-H_t^\star)-(G_\theta^{\{1\}}-G_\star^{\{1\}}) G θ { t } − G ⋆ { t } = 2 ( H t − H t ⋆ ) − ( G θ { 1 } − G ⋆ { 1 } ) , Kantorovich–Rubinstein duality yields
W 1 ( G θ { t } , G ⋆ { t } ) ≤ 2 W 1 ( H t , H t ⋆ ) + W 1 ( G θ { 1 } , G ⋆ { 1 } ) ≤ C ( p 0 ) Δ − ( 2 s − 2 ) h . W_1(G_\theta^{\{t\}},G_\star^{\{t\}})\le2W_1(H_t,H_t^\star)+W_1(G_\theta^{\{1\}},G_\star^{\{1\}})\le C(p_0)\Delta^{-(2s-2)}h. W 1 ( G θ { t } , G ⋆ { t } ) ≤ 2 W 1 ( H t , H t ⋆ ) + W 1 ( G θ { 1 } , G ⋆ { 1 } ) ≤ C ( p 0 ) Δ − ( 2 s − 2 ) h .
Without clustering, the same argument uses Lemma D.4 with s = K s=K s = K . Thus, in both regimes,
max t ≤ T W 1 ( G θ { t } , G ⋆ { t } ) ≤ C ( p 0 ) Δ − ( 2 s − 2 ) h . (123) \max_{t\le T}W_1(G_\theta^{\{t\}},G_\star^{\{t\}})\le C(p_0)\Delta^{-(2s-2)}h. \tag*{(123)} t ≤ T max W 1 ( G θ { t } , G ⋆ { t } ) ≤ C ( p 0 ) Δ − ( 2 s − 2 ) h . (123)
Lemma A.5 , applied with p = r = 1 p=r=1 p = r = 1 , gives
D μ ≤ max t ≤ T W 1 ( G θ { t } , G ⋆ { t } ) , D { t } ≤ 4 Δ − 1 W 1 ( G θ { t } , G ⋆ { t } ) . D_\mu\le\max_{t\le T}W_1(G_\theta^{\{t\}},G_\star^{\{t\}}),\qquad D_{\{t\}}\le4\Delta^{-1}W_1(G_\theta^{\{t\}},G_\star^{\{t\}}). D μ ≤ t ≤ T max W 1 ( G θ { t } , G ⋆ { t } ) , D { t } ≤ 4 Δ − 1 W 1 ( G θ { t } , G ⋆ { t } ) .
Consequently,
D μ ≤ C ( p 0 ) Δ − ( 2 s − 2 ) h , max t ≤ T D { t } ≤ C ( p 0 ) Δ − ( 2 s − 1 ) h , D_\mu\le C(p_0)\Delta^{-(2s-2)}h,\qquad\max_{t\le T}D_{\{t\}} \le C(p_0)\Delta^{-(2s-1)}h, D μ ≤ C ( p 0 ) Δ − ( 2 s − 2 ) h , t ≤ T max D { t } ≤ C ( p 0 ) Δ − ( 2 s − 1 ) h ,
including the bound for D π = D { 1 } D_\pi=D_{\{1\}} D π = D { 1 } .
Step 2: We prove that, under the structured geometry, D I ≤ C r ( p 0 ) Δ − ( 2 s − 1 ) h D_I \le C_r(p_0)\Delta^{-(2s-1)}h D I ≤ C r ( p 0 ) Δ − ( 2 s − 1 ) h for every I ⊆ [ T ] I\subseteq[T] I ⊆ [ T ] with ∣ I ∣ = r |I|=r ∣ I ∣ = r . Suppose h ≤ c ( p 0 ) Δ 2 s − 1 h\le c(p_0)\Delta^{2s-1} h ≤ c ( p 0 ) Δ 2 s − 1 . By Step 1, choosing c ( p 0 ) c(p_0) c ( p 0 ) sufficiently small ensures
W 1 ( G θ { 1 } , G ⋆ { 1 } ) ≤ p 0 Δ / 32. W_1\left(G_\theta^{\{1\}},G_\star^{\{1\}}\right)\le p_0\Delta/32. W 1 ( G θ { 1 } , G ⋆ { 1 } ) ≤ p 0 Δ/32.
Each ball B d ( a j , Δ / 4 ) B_d(a_j,\Delta/4) B d ( a j , Δ/4 ) must contain a fitted center of positive initial weight; otherwise the true mass at a j a_j a j alone would incur transport cost at least p 0 Δ / 4 p_0\Delta/4 p 0 Δ/4 . The K K K balls are disjoint and there are exactly K K K fitted labels. Hence, after relabeling, ∥ μ j − a j ∥ 2 < Δ / 4 \|\mu_j-a_j\|_2<\Delta/4 ∥ μ j − a j ∥ 2 < Δ/4 for every j ∈ [ K ] j\in[K] j ∈ [ K ] .
For an optimal initial coupling, the off-diagonal transport distances are at least 3 Δ / 4 3\Delta/4 3Δ/4 . Therefore,
∑ j ∣ π j − π j ⋆ ∣ ≤ 8 3 Δ W 1 ( G θ { 1 } , G ⋆ { 1 } ) ≤ p 0 / 12 , \sum_j|\pi_j-\pi_j^\star|\le\frac{8}{3\Delta}W_1\left(G_\theta^{\{1\}},G_\star^{\{1\}}\right)\le p_0/12, j ∑ ∣ π j − π j ⋆ ∣ ≤ 3Δ 8 W 1 ( G θ { 1 } , G ⋆ { 1 } ) ≤ p 0 /12 ,
so π j ≥ π j ⋆ / 2 \pi_j\ge\pi_j^\star/2 π j ≥ π j ⋆ /2 . The independently proved paired bound in Theorem 3.6 now gives
D I ≤ C r ( p 0 ) Δ − ( 2 s − 1 ) h . D_I\le C_r(p_0)\Delta^{-(2s-1)}h. D I ≤ C r ( p 0 ) Δ − ( 2 s − 1 ) h .
If instead h > c ( p 0 ) Δ 2 s − 1 h>c(p_0)\Delta^{2s-1} h > c ( p 0 ) Δ 2 s − 1 , then D I ≤ 2 < ( 2 / c ( p 0 ) ) Δ − ( 2 s − 1 ) h D_I\le2<(2/c(p_0))\Delta^{-(2s-1)}h D I ≤ 2 < ( 2/ c ( p 0 )) Δ − ( 2 s − 1 ) h . Combining the two cases proves the structured joint bound globally.
Step 3: We prove that, without cluster assumptions, D I ≤ C r ( p 0 ) Δ − ( 2 s − 1 ) h D_I\le C_r(p_0)\Delta^{-(2s-1)}h D I ≤ C r ( p 0 ) Δ − ( 2 s − 1 ) h . If h ≤ c ( p 0 ) Δ 2 s − 1 h\le c(p_0)\Delta^{2s-1} h ≤ c ( p 0 ) Δ 2 s − 1 , Step 1 and the matching argument in Step 2 give, after relabeling,
∥ μ j − a j ∥ 2 < Δ / 4 , π j ≥ π j ⋆ / 2 , j ∈ [ K ] . \|\mu_j-a_j\|_2<\Delta/4,\qquad\pi_j\ge\pi_j^\star/2,\qquad j\in[K]. ∥ μ j − a j ∥ 2 < Δ/4 , π j ≥ π j ⋆ /2 , j ∈ [ K ] .
The paired bound (D.27) in Theorem 3.6 , together with D ⋆ ≥ Δ D_\star\ge\Delta D ⋆ ≥ Δ , yields
D I ≤ C r ( p 0 ) Δ − ( 2 s − 1 ) h . D_I\le C_r(p_0)\Delta^{-(2s-1)}h. D I ≤ C r ( p 0 ) Δ − ( 2 s − 1 ) h .
If instead h > c ( p 0 ) Δ 2 s − 1 h>c(p_0)\Delta^{2s-1} h > c ( p 0 ) Δ 2 s − 1 , the trivial bound D I ≤ 2 D_I\le2 D I ≤ 2 gives the same conclusion. Thus
D I ≤ C r ( p 0 ) Δ − ( 2 s − 1 ) h . (124) D_I\le C_r(p_0)\Delta^{-(2s-1)}h. \tag*{(124)} D I ≤ C r ( p 0 ) Δ − ( 2 s − 1 ) h . (124)
Step 4: We prove that D A ≤ C ( p 0 ) Δ − ( 2 s − 1 ) h D_A\le C(p_0)\Delta^{-(2s-1)}h D A ≤ C ( p 0 ) Δ − ( 2 s − 1 ) h in both geometric regimes. If h ≤ c ( p 0 ) Δ 2 s − 1 h\le c(p_0)\Delta^{2s-1} h ≤ c ( p 0 ) Δ 2 s − 1 , Steps 2 and 3 establish matching and π j ≥ π j ⋆ / 2 \pi_j\ge\pi_j^\star/2 π j ≥ π j ⋆ /2 . The local transition-flow bound in Theorem 3.6 therefore gives
D A ≤ C ( p 0 ) Δ − ( 2 s − 1 ) h . D_A\le C(p_0)\Delta^{-(2s-1)}h. D A ≤ C ( p 0 ) Δ − ( 2 s − 1 ) h .
If instead h > c ( p 0 ) Δ 2 s − 1 h>c(p_0)\Delta^{2s-1} h > c ( p 0 ) Δ 2 s − 1 , each fitted and true transition-flow array is a probability array, so
D A ≤ 2 < 2 c ( p 0 ) Δ − ( 2 s − 1 ) h . D_A\le2<\frac{2}{c(p_0)}\Delta^{-(2s-1)}h. D A ≤ 2 < c ( p 0 ) 2 Δ − ( 2 s − 1 ) h .
Combining the two cases proves the global transition-flow bound, with s = s max s=s_{\max} s = s m a x under the structured geometry and s = K s=K s = K without cluster assumptions.
Proof of Corollary 3.3
Proof. Fix I ⊆ [ T ] I \subseteq[T] I ⊆ [ T ] with ∣ I ∣ = r ≥ 1 |I| = r \ge1 ∣ I ∣ = r ≥ 1 . Put p 0 = π min ⋆ p_0 = \pi^\star_{\min} p 0 = π m i n ⋆ , Δ = Δ s e p \Delta= \Delta_{\mathrm{sep}} Δ = Δ sep , D ⋆ = max i , j ∥ a i − a j ∥ 2 D_\star= \max_{i,j} \lVert a_i-a_j\rVert_2 D ⋆ = max i , j ∥ a i − a j ∥ 2 , and h = h ( p θ , T , p θ ⋆ , T ) h = h(p_{\theta,T},p_{\theta^\star,T}) h = h ( p θ , T , p θ ⋆ , T ) .
Step 1: We prove that, under the structured geometry, W 1 , I ( G θ I , G ⋆ I ) ≤ C r ( p 0 ) Δ − ( 2 s max − 2 ) h W_{1,I}(G_\theta^I,G_\star^I) \le C_r(p_0)\Delta^{-(2s_{\max}-2)}h W 1 , I ( G θ I , G ⋆ I ) ≤ C r ( p 0 ) Δ − ( 2 s m a x − 2 ) h .
Lemma A.4 gives
W 1 , I ( G θ I , G ⋆ I ) ≤ V 1 , I + 1 2 C 0 Δ D I + R J I , V 1 , I ≤ D μ . W_{1,I}(G_\theta^I,G_\star^I) \le V_{1,I} + \frac{1}{2}C_0\Delta D_I + R J_I,\qquad V_{1,I} \le D_\mu. W 1 , I ( G θ I , G ⋆ I ) ≤ V 1 , I + 2 1 C 0 Δ D I + R J I , V 1 , I ≤ D μ .
Theorem 3.1 and Lemma D.2 imply
D μ ≤ C ( p 0 ) Δ − ( 2 s max − 2 ) h , D I ≤ C r ( p 0 ) Δ − ( 2 s max − 1 ) h , J I ≤ C r Δ − ( 2 s max − 2 ) h . D_\mu\le C(p_0)\Delta^{-(2s_{\max}-2)}h,\qquad D_I \le C_r(p_0)\Delta^{-(2s_{\max}-1)}h,\qquad J_I \le C_r\Delta^{-(2s_{\max}-2)}h. D μ ≤ C ( p 0 ) Δ − ( 2 s m a x − 2 ) h , D I ≤ C r ( p 0 ) Δ − ( 2 s m a x − 1 ) h , J I ≤ C r Δ − ( 2 s m a x − 2 ) h .
Consequently, we achieve the proof of Equation (24) as
W 1 , I ( G θ I , G ⋆ I ) ≤ C ( p 0 ) Δ − ( 2 s max − 2 ) h + C r ( p 0 ) Δ Δ − ( 2 s max − 1 ) h + C r Δ − ( 2 s max − 2 ) h ≤ C r ( p 0 ) Δ − ( 2 s max − 2 ) h . \begin{aligned}
W_{1,I}(G_\theta^I,G_\star^I) &\le C(p_0)\Delta^{-(2s_{\max}-2)}h + C_r(p_0)\Delta\Delta^{-(2s_{\max}-1)}h + C_r\Delta^{-(2s_{\max}-2)}h\\
&\le C_r(p_0)\Delta^{-(2s_{\max}-2)}h.
\end{aligned} W 1 , I ( G θ I , G ⋆ I ) ≤ C ( p 0 ) Δ − ( 2 s m a x − 2 ) h + C r ( p 0 ) Δ Δ − ( 2 s m a x − 1 ) h + C r Δ − ( 2 s m a x − 2 ) h ≤ C r ( p 0 ) Δ − ( 2 s m a x − 2 ) h .
Step 2: We prove that, without cluster assumptions, W 1 , I ( G θ I , G ⋆ I ) ≤ C r ( p 0 ) Δ − ( 2 K − 2 ) h W_{1,I}(G_\theta^I,G_\star^I) \le C_r(p_0)\Delta^{-(2K-2)}h W 1 , I ( G θ I , G ⋆ I ) ≤ C r ( p 0 ) Δ − ( 2 K − 2 ) h . For r = 1 r=1 r = 1 , this follows directly from (21) . For r ≥ 2 r \ge2 r ≥ 2 and K ≥ 2 K \ge2 K ≥ 2 , we use the same let G ˉ θ I \bar{G}_\theta^I G ˉ θ I be the measure obtained by replacing each fitted center by its assigned true center. The assignment coupling gives
W 1 , I ( G θ I , G ˉ θ I ) ≤ V 1 , I . W_{1,I}(G_\theta^I,\bar{G}_\theta^I) \le V_{1,I}. W 1 , I ( G θ I , G ˉ θ I ) ≤ V 1 , I .
Both G ˉ θ I \bar{G}_\theta^I G ˉ θ I and G ⋆ I G_\star^I G ⋆ I are supported on the true grid, whose diameter under d 1 , r d_{1,r} d 1 , r is at most D ⋆ D_\star D ⋆ . Matching their common masses and coupling the residual mass, which equals D I / 2 D_I/2 D I /2 , yields
W 1 , I ( G ˉ θ I , G ⋆ I ) ≤ 1 2 D ⋆ D I . W_{1,I}(\bar{G}_\theta^I,G_\star^I) \le\frac{1}{2}D_\star D_I. W 1 , I ( G ˉ θ I , G ⋆ I ) ≤ 2 1 D ⋆ D I .
Thus, by the triangle inequality,
W 1 , I ( G θ I , G ⋆ I ) ≤ V 1 , I + 1 2 D ⋆ D I . W_{1,I}(G_\theta^I,G_\star^I) \le V_{1,I} + \frac{1}{2}D_\star D_I. W 1 , I ( G θ I , G ⋆ I ) ≤ V 1 , I + 2 1 D ⋆ D I .
Using V 1 , I ≤ D μ V_{1,I} \le D_\mu V 1 , I ≤ D μ , Theorem 3.1 , and the diameter-dependent bound D I ≤ C r ( p 0 ) D ⋆ − 2 Δ − ( 2 K − 3 ) h D_I \le C_r(p_0)D_\star^{-2}\Delta^{-(2K-3)}h D I ≤ C r ( p 0 ) D ⋆ − 2 Δ − ( 2 K − 3 ) h , we obtain
W 1 , I ( G θ I , G ⋆ I ) ≤ C ( p 0 ) Δ − ( 2 K − 2 ) h + C r ( p 0 ) D ⋆ − 1 Δ − ( 2 K − 3 ) h ≤ C r ( p 0 ) Δ − ( 2 K − 2 ) h , \begin{aligned}
W_{1,I}(G_\theta^I,G_\star^I) &\le C(p_0)\Delta^{-(2K-2)}h + C_r(p_0)D_\star^{-1}\Delta^{-(2K-3)}h\\
&\le C_r(p_0)\Delta^{-(2K-2)}h,
\end{aligned} W 1 , I ( G θ I , G ⋆ I ) ≤ C ( p 0 ) Δ − ( 2 K − 2 ) h + C r ( p 0 ) D ⋆ − 1 Δ − ( 2 K − 3 ) h ≤ C r ( p 0 ) Δ − ( 2 K − 2 ) h ,
where the last inequality uses D ⋆ ≥ Δ D_\star\ge\Delta D ⋆ ≥ Δ . For K = 1 K=1 K = 1 , both location-sequence measures are point masses and their normalized path distance equals their one-time distance. This proves (25) . □
Proof of Corollary 3.5
Proof. The matching argument in the proof of Theorem 3.1 proves the corollary under (26) . Substituting the radius of Proposition 2.2 gives (27) . For the explicit alternative (28) , Lemma D.3 bounds the total discrepancy between fitted near-set masses and true initial masses by C Δ s e p − 2 s max h C\Delta_{\mathrm{sep}}^{-2s_{\max}}h C Δ sep − 2 s m a x h . Under the stated threshold this is at most π min ⋆ / 4 \pi^\star_{\min}/4 π m i n ⋆ /4 . Thus all K K K near sets are nonempty, each contains exactly one fitted label, and every fitted initial weight is at least 3 π j ⋆ / 4 3\pi^\star_j/4 3 π j ⋆ /4 . The same reasoning applies without clusters with s max = K s_{\max}=K s m a x = K . □
Proof of Theorem 3.6
We first construct lower-degree state selectors for the structured geometry. Their values at fitted centers need not be exact: their errors will be controlled by the weighted displacement bound. We then compare Markov path distributions through their two-time marginals. Without cluster assumptions, we retain full products of center distances in this comparison.
Lemma D.6 (Cluster-localized cardinal polynomials). Suppose that the structured geometry holds and ∥ b i − a i ∥ 2 ≤ Δ s e p / 4 \lVert b_i-a_i\rVert_2 \le\Delta_{\mathrm{sep}}/4 ∥ b i − a i ∥ 2 ≤ Δ sep /4 . For all sufficiently small Δ s e p \Delta_{\mathrm{sep}} Δ sep , with the threshold depending only on K K K , C 0 C_0 C 0 , D 0 D_0 D 0 , there are polynomials L j L_j L j of degree at most 3 K − 2 3K-2 3 K − 2 such that
L j ( a i ) = 1 { i = j } , ∥ L j ∥ ∞ , B d ( 0 , R ) ≤ C Δ s e p − ( s max − 1 ) , (125) L_j(a_i)=1\{i=j\}, \qquad\lVert L_j\rVert_{\infty,B_d(0,R)} \le C\Delta_{\mathrm{sep}}^{-(s_{\max}-1)}, \tag*{(125)} L j ( a i ) = 1 { i = j } , ∥ L j ∥ ∞ , B d ( 0 , R ) ≤ C Δ sep − ( s m a x − 1 ) , (125) ∑ j = 1 K ∣ L j ( b i ) ∣ ≤ C , ∑ j = 1 K ∣ L j ( b i ) − 1 { i = j } ∣ ≤ C Δ s e p − 1 ∥ b i − a i ∥ 2 . (126) \sum_{j=1}^{K}\lvert L_j(b_i)\rvert\le C, \qquad\sum_{j=1}^{K}\lvert L_j(b_i)-1\{i=j\}\rvert\le C\Delta_{\mathrm{sep}}^{-1}\lVert b_i-a_i\rVert_2. \tag*{(126)} j = 1 ∑ K ∣ L j ( b i )∣ ≤ C , j = 1 ∑ K ∣ L j ( b i ) − 1 { i = j }∣ ≤ C Δ sep − 1 ∥ b i − a i ∥ 2 . (126) The constants depend only on K K K , R R R , C 0 C_0 C 0 , D 0 D_0 D 0 . In the single-cluster case, the assertion holds for every 0 < Δ s e p ≤ 1 0<\Delta_{\mathrm{sep}}\le1 0 < Δ sep ≤ 1 .
Proof. We construct cardinal polynomials within each cluster and then separate the clusters.
Interpolation inside a cluster. For j ∈ C m j\in C_m j ∈ C m , put s m = ∣ C m ∣ s_m=\lvert C_m\rvert s m = ∣ C m ∣ and define
ℓ j ( x ) = ∏ ℓ ∈ C m ∖ { j } ⟨ x − a ℓ , a j − a ℓ ⟩ ∥ a j − a ℓ ∥ 2 2 . \ell_j(x)=\prod_{\ell\in C_m\setminus\{j\}}\frac{\langle x-a_\ell,a_j-a_\ell\rangle}{\lVert a_j-a_\ell\rVert_2^2}. ℓ j ( x ) = ℓ ∈ C m ∖ { j } ∏ ∥ a j − a ℓ ∥ 2 2 ⟨ x − a ℓ , a j − a ℓ ⟩ .
The empty product is one. On the true centers of C m C_m C m , this polynomial equals the cardinal indicator of j j j . Its degree is s m − 1 s_m-1 s m − 1 , and its norm on B d ( 0 , R ) B_d(0,R) B d ( 0 , R ) is at most C Δ s e p − ( s m − 1 ) C\Delta_{\mathrm{sep}}^{-(s_m-1)} C Δ sep − ( s m − 1 ) . If i ∈ C m i\in C_m i ∈ C m and x ∈ [ a i , b i ] x\in[a_i,b_i] x ∈ [ a i , b i ] , then
∥ x − a ℓ ∥ 2 ≤ ( C 0 + 1 / 4 ) Δ s e p , ℓ ∈ C m . \lVert x-a_\ell\rVert_2\le(C_0+1/4)\Delta_{\mathrm{sep}}, \qquad\ell\in C_m. ∥ x − a ℓ ∥ 2 ≤ ( C 0 + 1/4 ) Δ sep , ℓ ∈ C m .
Each factor in the product is therefore uniformly bounded on this segment, and the norm of its gradient is at most Δ s e p − 1 \Delta_{\mathrm{sep}}^{-1} Δ sep − 1 . The product rule and the mean-value formula give
∣ ℓ j ( b i ) ∣ ≤ C , ∣ ℓ j ( b i ) − ℓ j ( a i ) ∣ ≤ C Δ s e p − 1 ∥ b i − a i ∥ 2 . \lvert\ell_j(b_i)\rvert\le C, \qquad\lvert\ell_j(b_i)-\ell_j(a_i)\rvert\le C\Delta_{\mathrm{sep}}^{-1}\lVert b_i-a_i\rVert_2. ∣ ℓ j ( b i )∣ ≤ C , ∣ ℓ j ( b i ) − ℓ j ( a i )∣ ≤ C Δ sep − 1 ∥ b i − a i ∥ 2 .
For one cluster, take L j = ℓ j L_j=\ell_j L j = ℓ j , which proves both assertions.
Separating distinct clusters. For several clusters, apply Lemma C.3 to the distinct true and fitted centers, obtaining a projection u u u that preserves all their pairwise distances up to a constant depending only on K K K . The union of the true and fitted centers assigned to a cluster has diameter at most ( C 0 + 1 / 2 ) Δ s e p (C_0+1/2)\Delta_{\mathrm{sep}} ( C 0 + 1/2 ) Δ sep . Centers assigned to different clusters have distance at least D 0 − Δ s e p / 2 D_0-\Delta_{\mathrm{sep}}/2 D 0 − Δ sep /2 . Thus, for Δ s e p \Delta_{\mathrm{sep}} Δ sep below a fixed positive threshold, the projected cluster intervals are separated by a fixed positive gap. Indeed, their diameters are O ( Δ s e p ) O(\Delta_{\mathrm{sep}}) O ( Δ sep ) , whereas the projected distance between any points from different clusters is bounded below by a fixed multiple of D 0 D_0 D 0 ; overlapping intervals would contradict these two facts.
For each cluster m m m , choose a smooth scalar function f m f_m f m that equals one on its projected interval and zero on all other cluster intervals. The functions can be joined across the fixed gaps with derivatives through order 2 K 2K 2 K bounded independently of Δ s e p \Delta_{\mathrm{sep}} Δ sep . Interpolate f m f_m f m on the distinct projected union nodes. Lemma C.4 gives a polynomial Q m Q_m Q m of degree at most 2 K − 1 2K-1 2 K − 1 whose norm on [ − R , R ] [-R,R] [ − R , R ] is uniformly bounded. Define F m ( x ) = Q m ( ⟨ u , x ⟩ ) F_m(x)=Q_m(\langle u,x\rangle) F m ( x ) = Q m (⟨ u , x ⟩) . Then
F m ( a i ) = F m ( b i ) = 1 { i ∈ C m } , ∥ F m ∥ ∞ , B d ( 0 , R ) ≤ C . F_m(a_i)=F_m(b_i)=1\{i\in C_m\}, \qquad\lVert F_m\rVert_{\infty,B_d(0,R)}\le C. F m ( a i ) = F m ( b i ) = 1 { i ∈ C m } , ∥ F m ∥ ∞ , B d ( 0 , R ) ≤ C .
Combining the two constructions. For j ∈ C m j \in C_m j ∈ C m , set L j = F m ℓ j L_j = F_m\ell_j L j = F m ℓ j . Its degree is at most 2 K + s m − 2 ≤ 3 K − 2 2K + s_m - 2 \le3K - 2 2 K + s m − 2 ≤ 3 K − 2 , and its global norm satisfies (D.18). At every fitted or true center outside C m C_m C m , the factor F m F_m F m vanishes. At every such center inside C m C_m C m , that factor equals one. Thus the within-cluster value estimates above apply exactly, with no approximation from F m F_m F m . Summing them over j j j proves (D.19).
Lemma D.7 (Markov path comparison). Let P P P and P ⋆ P^\star P ⋆ be the joint distributions of two Markov chains on a common finite state space at times 1 , … , r 1,\ldots,r 1 , … , r , allowing time-inhomogeneous transition matrices. Write q ℓ , q ℓ ⋆ q_\ell,q^\star_\ell q ℓ , q ℓ ⋆ for their marginal state probabilities and F ℓ , F ℓ ⋆ F_\ell,F^\star_\ell F ℓ , F ℓ ⋆ for the joint probabilities at times ℓ , ℓ + 1 \ell,\ell+1 ℓ , ℓ + 1 . Then
∥ P − P ⋆ ∥ 1 ≤ ∥ q 1 − q 1 ⋆ ∥ 1 + 2 ∑ ℓ = 1 r − 1 ∥ F ℓ − F ℓ ⋆ ∥ e n t , 1 , (127) \lVert P-P^\star\rVert_1 \le\lVert q_1-q^\star_1\rVert_1 + 2\sum_{\ell=1}^{r-1}\lVert F_\ell-F^\star_\ell\rVert_{\mathrm{ent},1}, \tag*{(127)} ∥ P − P ⋆ ∥ 1 ≤ ∥ q 1 − q 1 ⋆ ∥ 1 + 2 ℓ = 1 ∑ r − 1 ∥ F ℓ − F ℓ ⋆ ∥ ent , 1 , (127) where ∥ M ∥ e n t , 1 = ∑ i , j ∣ M i j ∣ \lVert M\rVert_{\mathrm{ent},1}=\sum_{i,j}|M_{ij}| ∥ M ∥ ent , 1 = ∑ i , j ∣ M ij ∣ . No lower bound on state probabilities is required.
Proof. Let B ℓ , B ℓ ⋆ B_\ell,B^\star_\ell B ℓ , B ℓ ⋆ be the transition matrices. For a fixed path x = ( x 1 , … , x r ) x=(x_1,\ldots,x_r) x = ( x 1 , … , x r ) the product identity gives
P ( x ) − P ⋆ ( x ) = { q 1 ( x 1 ) − q 1 ⋆ ( x 1 ) } ∏ a = 1 r − 1 B a ( x a , x a + 1 ) + q 1 ⋆ ( x 1 ) ∑ ℓ = 1 r − 1 { ∏ a < ℓ B a ⋆ ( x a , x a + 1 ) } ( B ℓ − B ℓ ⋆ ) ( x ℓ , x ℓ + 1 ) { ∏ a > ℓ B a ( x a , x a + 1 ) } . \begin{aligned}
P(x)-P^\star(x)={}&\{q_1(x_1)-q^\star_1(x_1)\}\prod_{a=1}^{r-1}B_a(x_a,x_{a+1})\\
&+q^\star_1(x_1)\sum_{\ell=1}^{r-1}\left\{\prod_{a<\ell}B^\star_a(x_a,x_{a+1})\right\}(B_\ell-B^\star_\ell)(x_\ell,x_{\ell+1})\left\{\prod_{a>\ell}B_a(x_a,x_{a+1})\right\}.
\end{aligned} P ( x ) − P ⋆ ( x ) = { q 1 ( x 1 ) − q 1 ⋆ ( x 1 )} a = 1 ∏ r − 1 B a ( x a , x a + 1 ) + q 1 ⋆ ( x 1 ) ℓ = 1 ∑ r − 1 { a < ℓ ∏ B a ⋆ ( x a , x a + 1 ) } ( B ℓ − B ℓ ⋆ ) ( x ℓ , x ℓ + 1 ) { a > ℓ ∏ B a ( x a , x a + 1 ) } .
Empty products equal one. Taking absolute values and summing over paths leaves total future mass one and true past marginal q ℓ ⋆ q^\star_\ell q ℓ ⋆ , giving
∥ P − P ⋆ ∥ 1 ≤ ∥ q 1 − q 1 ⋆ ∥ 1 + ∑ ℓ = 1 r − 1 ∑ i q ℓ ⋆ ( i ) ∥ B ℓ ( i , ⋅ ) − B ℓ ⋆ ( i , ⋅ ) ∥ 1 . \lVert P-P^\star\rVert_1\le\lVert q_1-q^\star_1\rVert_1+\sum_{\ell=1}^{r-1}\sum_iq^\star_\ell(i)\lVert B_\ell(i,\cdot)-B^\star_\ell(i,\cdot)\rVert_1. ∥ P − P ⋆ ∥ 1 ≤ ∥ q 1 − q 1 ⋆ ∥ 1 + ℓ = 1 ∑ r − 1 i ∑ q ℓ ⋆ ( i ) ∥ B ℓ ( i , ⋅ ) − B ℓ ⋆ ( i , ⋅ ) ∥ 1 .
For each i , j i,j i , j ,
q ℓ ⋆ ( i ) { B ℓ ( i , j ) − B ℓ ⋆ ( i , j ) } = F ℓ ( i , j ) − F ℓ ⋆ ( i , j ) + { q ℓ ⋆ ( i ) − q ℓ ( i ) } B ℓ ( i , j ) . q^\star_\ell(i)\{B_\ell(i,j)-B^\star_\ell(i,j)\}=F_\ell(i,j)-F^\star_\ell(i,j)+\{q^\star_\ell(i)-q_\ell(i)\}B_\ell(i,j). q ℓ ⋆ ( i ) { B ℓ ( i , j ) − B ℓ ⋆ ( i , j )} = F ℓ ( i , j ) − F ℓ ⋆ ( i , j ) + { q ℓ ⋆ ( i ) − q ℓ ( i )} B ℓ ( i , j ) .
Summing absolute values and using the row sums of B ℓ B_\ell B ℓ yields
∑ i q ℓ ⋆ ( i ) ∥ B ℓ ( i , ⋅ ) − B ℓ ⋆ ( i , ⋅ ) ∥ 1 ≤ ∥ F ℓ − F ℓ ⋆ ∥ e n t , 1 + ∥ q ℓ − q ℓ ⋆ ∥ 1 \sum_iq^\star_\ell(i)\lVert B_\ell(i,\cdot)-B^\star_\ell(i,\cdot)\rVert_1\le\lVert F_\ell-F^\star_\ell\rVert_{\mathrm{ent},1}+\lVert q_\ell-q^\star_\ell\rVert_1 i ∑ q ℓ ⋆ ( i ) ∥ B ℓ ( i , ⋅ ) − B ℓ ⋆ ( i , ⋅ ) ∥ 1 ≤ ∥ F ℓ − F ℓ ⋆ ∥ ent , 1 + ∥ q ℓ − q ℓ ⋆ ∥ 1
≤ 2 ∥ F ℓ − F ℓ ⋆ ∥ e n t , 1 . \le2\lVert F_\ell-F^\star_\ell\rVert_{\mathrm{ent},1}. ≤ 2 ∥ F ℓ − F ℓ ⋆ ∥ ent , 1 .
The final inequality is contraction of ℓ 1 \ell_1 ℓ 1 distance on summing the second coordinate of the two-time distributions. Substitution proves equation (D.20). For r = 1 r=1 r = 1 the sum is empty and the assertion is equality. No step divides by a state or path probability. For nonconsecutive selected HMM times, the required matrices are A t ℓ + 1 − t ℓ A^{t_{\ell+1}-t_\ell} A t ℓ + 1 − t ℓ and ( A ⋆ ) t ℓ + 1 − t ℓ (A^\star)^{t_{\ell+1}-t_\ell} ( A ⋆ ) t ℓ + 1 − t ℓ ; they are stochastic even when some marginal probabilities vanish. The argument applies to aligned exact-fit chains, not to an arbitrary grouped overfitted process.
Proof of Theorem 3.6 . We use the assumed labeling, so that μ j = b j \mu_j=b_j μ j = b j and ∥ b j − a j ∥ 2 ≤ Δ s e p / 4 \lVert b_j-a_j\rVert_2\le\Delta_{\mathrm{sep}}/4 ∥ b j − a j ∥ 2 ≤ Δ sep /4 . This labeling also agrees with the nearest-center assignment. Indeed, for ℓ ≠ j \ell\ne j ℓ = j ,
∥ b j − a ℓ ∥ 2 ≥ ∥ a j − a ℓ ∥ 2 − ∥ b j − a j ∥ 2 (by the reverse triangle inequality) ≥ Δ s e p − Δ s e p 4 = 3 Δ s e p 4 > Δ s e p 4 ≥ ∥ b j − a j ∥ 2 . \begin{aligned}
\lVert b_j-a_\ell\rVert_2 &\ge\lVert a_j-a_\ell\rVert_2-\lVert b_j-a_j\rVert_2 && \text{(by the reverse triangle inequality)}\\
&\ge\Delta_{\mathrm{sep}}-\frac{\Delta_{\mathrm{sep}}}{4}\\
&=\frac{3\Delta_{\mathrm{sep}}}{4}\\
&>\frac{\Delta_{\mathrm{sep}}}{4}\ge\lVert b_j-a_j\rVert_2.
\end{aligned} ∥ b j − a ℓ ∥ 2 ≥ ∥ a j − a ℓ ∥ 2 − ∥ b j − a j ∥ 2 ≥ Δ sep − 4 Δ sep = 4 3 Δ sep > 4 Δ sep ≥ ∥ b j − a j ∥ 2 . (by the reverse triangle inequality)
Thus c ( j ) = j c(j)=j c ( j ) = j , every assignment set is a singleton, and r j = h j = ∥ b j − a j ∥ 2 r_j=h_j=\lVert b_j-a_j\rVert_2 r j = h j = ∥ b j − a j ∥ 2 . The aggregate losses can consequently be written using the aligned state labels.
Moment comparison for retained observations. Before extracting the different losses, we record the observation-error bound used for each polynomial. Let I ⊆ [ T ] I\subseteq[T] I ⊆ [ T ] contain r r r times, and let U U U be a polynomial on ( R d ) r (\mathbb{R}^d)^r ( R d ) r of degree at most r ( 3 K − 2 ) r(3K-2) r ( 3 K − 2 ) . Both location distributions are supported in the span
S = span { a 1 , … , a K , b 1 , … , b K } , dim S ≤ 2 K . S=\operatorname{span}\{a_1,\ldots,a_K,b_1,\ldots,b_K\},\qquad\dim S\le2K. S = span { a 1 , … , a K , b 1 , … , b K } , dim S ≤ 2 K .
The HMM application of Lemma A.3 therefore has a constant depending only on K K K , r r r , R R R and the degree. Applying it to the retained observations gives the first inequality below; marginalization then replaces their density error by that of the full observed block:
∣ ∫ U d ( G θ I − G ⋆ I ) ∣ ≤ C r ∥ U ∥ ∞ h ( p θ I , p ⋆ I ) (by xref‘Lemma A.3‘) ≤ C r ∥ U ∥ ∞ h ( p θ , T , p θ ⋆ , T ) (by xref‘Lemma A.2‘). (128) \begin{aligned}
\left|\int U\,\mathrm{d}(G_\theta^I-G_{\star}^I)\right|
\le C_r\lVert U\rVert_\infty h(p_\theta^I,p_\star^I)
\qquad\text{(by {xref}`Lemma A.3`)} \\
&\le C_r\lVert U\rVert_\infty h(p_{\theta,T},p_{\theta_\star,T})
\qquad\text{(by {xref}`Lemma A.2`).} \tag*{(128)}
\end{aligned} ∫ U d ( G θ I − G ⋆ I ) ≤ C r ∥ U ∥ ∞ h ( p θ I , p ⋆ I ) (by xref ‘Lemma A.3‘) ≤ C r ∥ U ∥ ∞ h ( p θ , T , p θ ⋆ , T ) (by xref ‘Lemma A.2‘). (128)
Here the supremum norm is taken on B d ( 0 , R ) r B_d(0,R)^r B d ( 0 , R ) r . The remaining task is to choose U U U whose moment difference equals the recovery quantity being bounded.
Weighted displacement. Fix j ∈ [ K ] j\in[K] j ∈ [ K ] and t ≤ T t\le T t ≤ T . At one time, the fitted and true location distributions are
G θ { t } = ∑ ℓ = 1 K q t ( ℓ ) δ b ℓ , G ⋆ { t } = ∑ ℓ = 1 K q t ⋆ ( ℓ ) δ a ℓ . G_\theta^{\{t\}}=\sum_{\ell=1}^{K}q_t(\ell)\delta_{b_\ell},\qquad
G_\star^{\{t\}}=\sum_{\ell=1}^{K}q_t^\star(\ell)\delta_{a_\ell}. G θ { t } = ℓ = 1 ∑ K q t ( ℓ ) δ b ℓ , G ⋆ { t } = ℓ = 1 ∑ K q t ⋆ ( ℓ ) δ a ℓ .
Expanding the moment of H ~ j \widetilde{H}_j H j over these supports gives
∫ H ~ j d ( G θ { t } − G ⋆ { t } ) = ∑ ℓ = 1 K q t ( ℓ ) H ~ j ( b ℓ ) − ∑ ℓ = 1 K q t ⋆ ( ℓ ) H ~ j ( a ℓ ) = ∑ ℓ = 1 K q t ( ℓ ) 1 { j = ℓ } h j − ∑ ℓ = 1 K q t ⋆ ( ℓ ) 0 (by xref‘Equation (C.44)‘) = q t ( j ) h j . \begin{aligned}
\int\widetilde{H}_j\,\mathrm{d}(G_\theta^{\{t\}}-G_\star^{\{t\}})
&=\sum_{\ell=1}^{K}q_t(\ell)\widetilde{H}_j(b_\ell)-\sum_{\ell=1}^{K}q_t^\star(\ell)\widetilde{H}_j(a_\ell)\\
&=\sum_{\ell=1}^{K}q_t(\ell)\mathbf{1}\{j=\ell\}h_j-\sum_{\ell=1}^{K}q_t^\star(\ell)0
\qquad\text{(by {xref}`Equation (C.44)`)}\\
&=q_t(j)h_j.
\end{aligned} ∫ H j d ( G θ { t } − G ⋆ { t } ) = ℓ = 1 ∑ K q t ( ℓ ) H j ( b ℓ ) − ℓ = 1 ∑ K q t ⋆ ( ℓ ) H j ( a ℓ ) = ℓ = 1 ∑ K q t ( ℓ ) 1 { j = ℓ } h j − ℓ = 1 ∑ K q t ⋆ ( ℓ ) 0 (by xref ‘Equation (C.44)‘) = q t ( j ) h j .
The extracted quantity is nonnegative, so taking an absolute value does not change it. Equation (D.21) now gives
q t ( j ) h j = ∣ ∫ H ~ j d ( G θ { t } − G ⋆ { t } ) ∣ ≤ C ∥ H ~ j ∥ ∞ h ( p θ , T , p θ ⋆ , T ) ≤ C Δ s e p − ( 2 s max − 2 ) h ( p θ , T , p θ ⋆ , T ) (by xref‘Equation (C.45)‘). \begin{aligned}
q_t(j)h_j
&=\left|\int\widetilde{H}_j\,\mathrm{d}(G_\theta^{\{t\}}-G_\star^{\{t\}})\right|\\
&\le C\lVert\widetilde{H}_j\rVert_\infty h(p_{\theta,T},p_{\theta_\star,T})\\
&\le C\Delta_{\mathrm{sep}}^{-(2s_{\max}-2)}h(p_{\theta,T},p_{\theta_\star,T})
\qquad\text{(by {xref}`Equation (C.45)`).}
\end{aligned} q t ( j ) h j = ∫ H j d ( G θ { t } − G ⋆ { t } ) ≤ C ∥ H j ∥ ∞ h ( p θ , T , p θ ⋆ , T ) ≤ C Δ sep − ( 2 s m a x − 2 ) h ( p θ , T , p θ ⋆ , T ) (by xref ‘Equation (C.45)‘).
Summing over j j j yields
∑ j = 1 K q t ( j ) h j ≤ K C Δ s e p − ( 2 s max − 2 ) h ( p θ , T , p θ ⋆ , T ) ≤ C Δ s e p − ( 2 s max − 2 ) h ( p θ , T , p θ ⋆ , T ) , \sum_{j=1}^{K} q_t(j)h_j \le KC\Delta_{\mathrm{sep}}^{-(2s_{\max}-2)}h(p_{\theta,T},p_{\theta_\star,T}) \le C\Delta_{\mathrm{sep}}^{-(2s_{\max}-2)}h(p_{\theta,T},p_{\theta_\star,T}), j = 1 ∑ K q t ( j ) h j ≤ K C Δ sep − ( 2 s m a x − 2 ) h ( p θ , T , p θ ⋆ , T ) ≤ C Δ sep − ( 2 s m a x − 2 ) h ( p θ , T , p θ ⋆ , T ) ,
where the fixed factor K K K is absorbed into C C C . Taking the maximum over t t t proves
D μ ≤ C Δ s e p − ( 2 s max − 2 ) h ( p θ , T , p θ ⋆ , T ) . (129) D_\mu\le C\Delta_{\mathrm{sep}}^{-(2s_{\max}-2)}h(p_{\theta,T},p_{\theta_\star,T}). \tag*{(129)} D μ ≤ C Δ sep − ( 2 s m a x − 2 ) h ( p θ , T , p θ ⋆ , T ) . (129)
No lower bound on q t ( j ) q_t(j) q t ( j ) is used; a state with zero probability contributes zero to the extracted displacement.
One-time probabilities. For the cardinal polynomial E ~ j \widetilde{E}_j E j , the same finite-support expansion gives
∫ E ~ j d ( G θ { t } − G ⋆ { t } ) = ∑ ℓ = 1 K q t ( ℓ ) E ~ j ( b ℓ ) − ∑ ℓ = 1 K q t ⋆ ( ℓ ) E ~ j ( a ℓ ) = ∑ ℓ = 1 K { q t ( ℓ ) − q t ⋆ ( ℓ ) } 1 { j = ℓ } (by Equation xref‘(C.43‘) = q t ( j ) − q t ⋆ ( j ) . \begin{aligned}
\int\widetilde{E}_j\,d\left(G_\theta^{\{t\}}-G_\star^{\{t\}}\right)
=\sum_{\ell=1}^{K}q_t(\ell)\widetilde{E}_j(b_\ell)-\sum_{\ell=1}^{K}q_t^\star(\ell)\widetilde{E}_j(a_\ell) \\
&=\sum_{\ell=1}^{K}\{q_t(\ell)-q_t^\star(\ell)\}1\{j=\ell\}
\quad\text{(by Equation {xref}`(C.43`)} \\
&=q_t(j)-q_t^\star(j).
\end{aligned} ∫ E j d ( G θ { t } − G ⋆ { t } ) = ℓ = 1 ∑ K q t ( ℓ ) E j ( b ℓ ) − ℓ = 1 ∑ K q t ⋆ ( ℓ ) E j ( a ℓ ) = ℓ = 1 ∑ K { q t ( ℓ ) − q t ⋆ ( ℓ )} 1 { j = ℓ } (by Equation xref ‘(C.43‘) = q t ( j ) − q t ⋆ ( j ) .
Applying Equation (D.21) to this identity, we obtain
∣ q t ( j ) − q t ⋆ ( j ) ∣ ≤ C ∥ E ~ j ∥ ∞ h ( p θ , T , p θ ⋆ , T ) ≤ C Δ s e p − ( 2 s max − 1 ) h ( p θ , T , p θ ⋆ , T ) (by Equation xref‘(C.45)‘) . \begin{aligned}
|q_t(j)-q_t^\star(j)|\le C\|\widetilde{E}_j\|_\infty h(p_{\theta,T},p_{\theta_\star,T}) \\
&\le C\Delta_{\mathrm{sep}}^{-(2s_{\max}-1)}h(p_{\theta,T},p_{\theta_\star,T})
\quad\text{(by Equation {xref}`(C.45)`)}.
\end{aligned} ∣ q t ( j ) − q t ⋆ ( j ) ∣ ≤ C ∥ E j ∥ ∞ h ( p θ , T , p θ ⋆ , T ) ≤ C Δ sep − ( 2 s m a x − 1 ) h ( p θ , T , p θ ⋆ , T ) (by Equation xref ‘(C.45)‘) .
We take absolute values before summing over states, as required by the definition of D { t } D_{\{t\}} D { t } :
D { t } = ∑ j = 1 K ∣ q t ( j ) − q t ⋆ ( j ) ∣ ≤ K C Δ s e p − ( 2 s max − 1 ) h ( p θ , T , p θ ⋆ , T ) ≤ C Δ s e p − ( 2 s max − 1 ) h ( p θ , T , p θ ⋆ , T ) . \begin{aligned}
D_{\{t\}}=\sum_{j=1}^{K}|q_t(j)-q_t^\star(j)| \\
&\le KC\Delta_{\mathrm{sep}}^{-(2s_{\max}-1)}h(p_{\theta,T},p_{\theta_\star,T}) \\
&\le C\Delta_{\mathrm{sep}}^{-(2s_{\max}-1)}h(p_{\theta,T},p_{\theta_\star,T}).
\end{aligned} D { t } = j = 1 ∑ K ∣ q t ( j ) − q t ⋆ ( j ) ∣ ≤ K C Δ sep − ( 2 s m a x − 1 ) h ( p θ , T , p θ ⋆ , T ) ≤ C Δ sep − ( 2 s m a x − 1 ) h ( p θ , T , p θ ⋆ , T ) .
This bound is uniform in t t t , proving the one-time probability assertion. In particular, D π = D { 1 } D_\pi=D_{\{1\}} D π = D { 1 } satisfies the same bound.
Two-time probabilities in structured configurations. We first treat Δ s e p \Delta_{\mathrm{sep}} Δ sep below the threshold in Lemma D.6. Fix arbitrary times t < u t<u t < u , and let
F i ℓ t , u = P θ ( Z t = i , Z u = ℓ ) , F ⋆ , i ℓ t , u = P θ ⋆ ( Z t = i , Z u = ℓ ) , E j i = L j ( b i ) . F_{i\ell}^{t,u}=\mathbb{P}_\theta(Z_t=i,Z_u=\ell),\qquad F_{\star,i\ell}^{t,u}=\mathbb{P}_{\theta_\star}(Z_t=i,Z_u=\ell),\qquad E_{ji}=L_j(b_i). F i ℓ t , u = P θ ( Z t = i , Z u = ℓ ) , F ⋆ , i ℓ t , u = P θ ⋆ ( Z t = i , Z u = ℓ ) , E j i = L j ( b i ) .
The row and column sums of F t , u F^{t,u} F t , u are q t q_t q t and q u q_u q u . Since L j ( a i ) = 1 { j = i } L_j(a_i)=1\{j=i\} L j ( a i ) = 1 { j = i } ,
∫ L j ( x ) L ℓ ( y ) d ( G θ { t , u } − G ⋆ { t , u } ) = { E F t , u E ⊤ − F ⋆ t , u } j ℓ . \int L_j(x)L_\ell(y)\,d\left(G_\theta^{\{t,u\}}-G_\star^{\{t,u\}}\right)
=\{EF^{t,u}E^\top-F_\star^{t,u}\}_{j\ell}. ∫ L j ( x ) L ℓ ( y ) d ( G θ { t , u } − G ⋆ { t , u } ) = { E F t , u E ⊤ − F ⋆ t , u } j ℓ .
The polynomial has degree at most 2 ( 3 K − 2 ) 2(3K-2) 2 ( 3 K − 2 ) and norm at most C Δ s e p − ( 2 s max − 2 ) C\Delta_{\mathrm{sep}}^{-(2s_{\max}-2)} C Δ sep − ( 2 s m a x − 2 ) . Applying (D.21) entry by entry and summing over the K 2 K^2 K 2 entries gives
∥ E F t , u E ⊤ − F ⋆ t , u ∥ e n t , 1 ≤ C Δ s e p − ( 2 s max − 2 ) h ( p θ , T , p θ ⋆ , T ) . (130) \|EF^{t,u}E^\top-F_\star^{t,u}\|_{\mathrm{ent},1}
\le C\Delta_{\mathrm{sep}}^{-(2s_{\max}-2)}h(p_{\theta,T},p_{\theta_\star,T}). \tag*{(130)} ∥ E F t , u E ⊤ − F ⋆ t , u ∥ ent , 1 ≤ C Δ sep − ( 2 s m a x − 2 ) h ( p θ , T , p θ ⋆ , T ) . (130)
It remains to account for the values of the selectors at fitted centers. Put
e i = ∑ j ∣ E j i − 1 { j = i } ∣ , m i = ∑ j ∣ E j i ∣ . e_i=\sum_j\lvert E_{ji}-1\{j=i\}\rvert,\qquad m_i=\sum_j\lvert E_{ji}\rvert. e i = j ∑ ∣ E j i − 1 { j = i }∣ , m i = j ∑ ∣ E j i ∣ .
Lemma D.6 gives e i ≤ C Δ s e p − 1 h i e_i \le C\Delta_{\mathrm{sep}}^{-1}h_i e i ≤ C Δ sep − 1 h i and m i ≤ C m_i \le C m i ≤ C . The identity
E F t , u E ⊤ − F t , u = ( E − I ) F t , u E ⊤ + F t , u ( E ⊤ − I ) EF^{t,u}E^{\top}-F^{t,u}=(E-I)F^{t,u}E^{\top}+F^{t,u}(E^{\top}-I) E F t , u E ⊤ − F t , u = ( E − I ) F t , u E ⊤ + F t , u ( E ⊤ − I )
and the nonnegativity of F t , u F^{t,u} F t , u imply
∥ E F t , u E ⊤ − F t , u ∥ e n t , 1 ≤ ∑ i , ℓ F i ℓ t , u ( e i m ℓ + e ℓ ) ≤ C Δ s e p − 1 { ∑ i q t ( i ) h i + ∑ ℓ q u ( ℓ ) h ℓ } ≤ C Δ s e p − 1 D μ . \begin{aligned}
\left\|EF^{t,u}E^{\top}-F^{t,u}\right\|_{\mathrm{ent},1}\le\sum_{i,\ell}F_{i\ell}^{t,u}(e_i m_\ell+e_\ell) \\
&\le C\Delta_{\mathrm{sep}}^{-1}\left\{\sum_i q_t(i)h_i+\sum_\ell q_u(\ell)h_\ell\right\} \\
&\le C\Delta_{\mathrm{sep}}^{-1}D_\mu.
\end{aligned} E F t , u E ⊤ − F t , u ent , 1 ≤ i , ℓ ∑ F i ℓ t , u ( e i m ℓ + e ℓ ) ≤ C Δ sep − 1 { i ∑ q t ( i ) h i + ℓ ∑ q u ( ℓ ) h ℓ } ≤ C Δ sep − 1 D μ .
Combining this inequality with (D.23) and (D.22) yields
D { t , u } = ∥ F t , u − F ⋆ t , u ∥ e n t , 1 ≤ C Δ s e p − ( 2 s max − 1 ) h ( p θ , T , p θ ⋆ , T ) . (131) D_{\{t,u\}}=\left\|F^{t,u}-F_\star^{t,u}\right\|_{\mathrm{ent},1}\le C\Delta_{\mathrm{sep}}^{-(2s_{\max}-1)}h(p_{\theta,T},p_{\theta^\star,T}). \tag*{(131)} D { t , u } = F t , u − F ⋆ t , u ent , 1 ≤ C Δ sep − ( 2 s m a x − 1 ) h ( p θ , T , p θ ⋆ , T ) . (131)
For completeness, when Δ s e p \Delta_{\mathrm{sep}} Δ sep is above the fixed threshold, use the products E ~ j ( x ) E ~ ℓ ( y ) \widetilde E_j(x)\widetilde E_\ell(y) E j ( x ) E ℓ ( y ) from Appendix § 11 .§ 6 . Their values on both supports are the exact pair indicators, and their norms are bounded by C Δ s e p − ( 4 s max − 2 ) C\Delta_{\mathrm{sep}}^{-(4s_{\max}-2)} C Δ sep − ( 4 s m a x − 2 ) . Since separation is now bounded below by a fixed constant, this is at most a constant times Δ s e p − ( 2 s max − 1 ) \Delta_{\mathrm{sep}}^{-(2s_{\max}-1)} Δ sep − ( 2 s m a x − 1 ) . Thus (D.24) holds throughout the structured regime. Taking u = t + 1 u=t+1 u = t + 1 and maximizing proves the asserted bound for D A D_A D A .
State-sequence probabilities. Let I = ( t 1 , … , t r ) I=(t_1,\ldots,t_r) I = ( t 1 , … , t r ) . Under either fitted or true parameters, the selected states form a Markov chain with transition matrices A t ℓ + 1 − t ℓ A^{t_{\ell+1}-t_\ell} A t ℓ + 1 − t ℓ or ( A ⋆ ) t ℓ + 1 − t ℓ (A^\star)^{t_{\ell+1}-t_\ell} ( A ⋆ ) t ℓ + 1 − t ℓ . Because the labels are aligned, Lemma D.7 applies directly to these two chains and gives
D I ≤ D { t 1 } + 2 ∑ ℓ = 1 r − 1 D { t ℓ , t ℓ + 1 } . D_I\le D_{\{t_1\}}+2\sum_{\ell=1}^{r-1}D_{\{t_\ell,t_{\ell+1}\}}. D I ≤ D { t 1 } + 2 ℓ = 1 ∑ r − 1 D { t ℓ , t ℓ + 1 } .
The one-time bound already proved and (D.24) establish (3.16). The number of selected times enters only the constant in this step.
Unstructured probability bounds with distance products. Write Δ = Δ s e p \Delta=\Delta_{\mathrm{sep}} Δ = Δ sep , D ⋆ = max i , j ∥ a i − a j ∥ 2 D_\star=\max_{i,j}\lVert a_i-a_j\rVert_2 D ⋆ = max i , j ∥ a i − a j ∥ 2 , and D i = ∏ j ≠ i ∥ a i − a j ∥ 2 D_i=\prod_{j\ne i}\lVert a_i-a_j\rVert_2 D i = ∏ j = i ∥ a i − a j ∥ 2 . The paired displacement polynomial retains the state-specific estimate
q t ( i ) h i ≤ C D i − 2 h ( p θ , T , p θ ⋆ , T ) . (132) q_t(i)h_i\le CD_i^{-2}h(p_{\theta,T},p_{\theta^\star,T}). \tag*{(132)} q t ( i ) h i ≤ C D i − 2 h ( p θ , T , p θ ⋆ , T ) . (132)
Indeed, the proof of Lemma C.8 gives ∥ R i ∥ ∞ ≤ C D i − 2 \lVert R_i\rVert_\infty\le CD_i^{-2} ∥ R i ∥ ∞ ≤ C D i − 2 before replacing its denominator by a separation power. The selected-segment lower bound for R i ( b i ) R_i(b_i) R i ( b i ) is uniform. The definition of H ~ i \widetilde H_i H i and moment extraction therefore prove (D.25), including when h i = 0 h_i=0 h i = 0 .
Use the ordinary cardinal polynomials L j L_j L j from Lemma C.7 , and put E j i = L j ( b i ) E_{ji}=L_j(b_i) E j i = L j ( b i ) as before. Since ∥ L j ∥ ∞ ≤ C / D j \lVert L_j\rVert_\infty\le C/D_j ∥ L j ∥ ∞ ≤ C / D j and D j ≥ ( D ⋆ / 2 ) Δ K − 2 D_j\ge(D_\star/2)\Delta^{K-2} D j ≥ ( D ⋆ /2 ) Δ K − 2 , the pair-moment comparison gives
∥ E F t , u E ⊤ − F ⋆ t , u ∥ e n t , 1 ≤ C D ⋆ − 2 Δ − ( 2 K − 4 ) h ( p θ , T , p θ ⋆ , T ) . \left\|EF^{t,u}E^{\top}-F_\star^{t,u}\right\|_{\mathrm{ent},1}\le CD_\star^{-2}\Delta^{-(2K-4)}h(p_{\theta,T},p_{\theta^\star,T}). E F t , u E ⊤ − F ⋆ t , u ent , 1 ≤ C D ⋆ − 2 Δ − ( 2 K − 4 ) h ( p θ , T , p θ ⋆ , T ) .
Also P ( b i ) ≤ C h i D i \sqrt{P(b_i)} \le C h_i \mathcal{D}_i P ( b i ) ≤ C h i D i under matching. Equation (C.30) gives
e i = ∑ j ∣ E j i − 1 { j = i } ∣ ≤ C a ⋆ h i D i , a ⋆ = ( D ⋆ Δ K − 1 ) − 1 . e_i = \sum_j \left|E_{ji}-1\{j=i\}\right| \le C a_\star h_i\mathcal{D}_i,\qquad a_\star= \left(\mathcal{D}_\star\Delta^{K-1}\right)^{-1}. e i = j ∑ ∣ E j i − 1 { j = i } ∣ ≤ C a ⋆ h i D i , a ⋆ = ( D ⋆ Δ K − 1 ) − 1 .
Expand E F t , u E ⊤ − F t , u EF^{t,u}E^\top-F^{t,u} E F t , u E ⊤ − F t , u into its two linear error terms and the product of the two errors. Nonnegativity of the pair probabilities gives
∥ E F t , u E ⊤ − F t , u ∥ e n t , 1 ≤ ∑ i q t ( i ) e i + ∑ ℓ q u ( ℓ ) e ℓ + ∑ i , ℓ F i ℓ t , u e i e ℓ . \left\|EF^{t,u}E^\top-F^{t,u}\right\|_{\mathrm{ent},1}\le\sum_i q_t(i)e_i+\sum_\ell q_u(\ell)e_\ell+\sum_{i,\ell}F_{i\ell}^{t,u}e_i e_\ell. E F t , u E ⊤ − F t , u ent , 1 ≤ i ∑ q t ( i ) e i + ℓ ∑ q u ( ℓ ) e ℓ + i , ℓ ∑ F i ℓ t , u e i e ℓ .
The first two sums are at most C D ⋆ − 2 Δ − ( 2 K − 3 ) h ( p θ , T , p θ ⋆ , T ) C\mathcal{D}_\star^{-2}\Delta^{-(2K-3)}h(p_{\theta,T},p_{\theta_\star,T}) C D ⋆ − 2 Δ − ( 2 K − 3 ) h ( p θ , T , p θ ⋆ , T ) by (D.25) and the lower bound on D i \mathcal{D}_i D i . For the last sum, use h i , h ℓ ≤ Δ / 4 h_i,h_\ell\le\Delta/4 h i , h ℓ ≤ Δ/4 and both marginals of F t , u F^{t,u} F t , u :
F i ℓ t , u h i h ℓ ≤ C Δ min { D i − 2 , D ℓ − 2 } h ( p θ , T , p θ ⋆ , T ) . F_{i\ell}^{t,u}h_i h_\ell\le C\Delta\min\{\mathcal{D}_i^{-2},\mathcal{D}_\ell^{-2}\}h(p_{\theta,T},p_{\theta_\star,T}). F i ℓ t , u h i h ℓ ≤ C Δ min { D i − 2 , D ℓ − 2 } h ( p θ , T , p θ ⋆ , T ) .
Multiplication by a ⋆ 2 D i D ℓ a_\star^2\mathcal{D}_i\mathcal{D}_\ell a ⋆ 2 D i D ℓ leaves the factor
a ⋆ 2 Δ min { D ℓ / D i , D i / D ℓ } ≤ D ⋆ − 2 Δ − ( 2 K − 3 ) . a_\star^2\Delta\min\{\mathcal{D}_\ell/\mathcal{D}_i,\mathcal{D}_i/\mathcal{D}_\ell\}\le\mathcal{D}_\star^{-2}\Delta^{-(2K-3)}. a ⋆ 2 Δ min { D ℓ / D i , D i / D ℓ } ≤ D ⋆ − 2 Δ − ( 2 K − 3 ) .
Thus the cross term has the same bound. Combining the moment and evaluation comparisons proves
D { t , u } ≤ C D ⋆ − 2 Δ − ( 2 K − 3 ) h ( p θ , T , p θ ⋆ , T ) . (133) D_{\{t,u\}}\le C\mathcal{D}_\star^{-2}\Delta^{-(2K-3)}h(p_{\theta,T},p_{\theta_\star,T}). \tag*{(133)} D { t , u } ≤ C D ⋆ − 2 Δ − ( 2 K − 3 ) h ( p θ , T , p θ ⋆ , T ) . (133)
Every one-time discrepancy is bounded by a pair discrepancy, since T ≥ 2 T\ge2 T ≥ 2 . Lemma D.7 consequently gives
D I ≤ C r D ⋆ − 2 Δ − ( 2 K − 3 ) h ( p θ , T , p θ ⋆ , T ) ≤ C r Δ − ( 2 K − 1 ) h ( p θ , T , p θ ⋆ , T ) . (134) D_I\le C_r\mathcal{D}_\star^{-2}\Delta^{-(2K-3)}h(p_{\theta,T},p_{\theta_\star,T})\le C_r\Delta^{-(2K-1)}h(p_{\theta,T},p_{\theta_\star,T}). \tag*{(134)} D I ≤ C r D ⋆ − 2 Δ − ( 2 K − 3 ) h ( p θ , T , p θ ⋆ , T ) ≤ C r Δ − ( 2 K − 1 ) h ( p θ , T , p θ ⋆ , T ) . (134)
This proves (33) for every nonempty I I I , with constants independent of state probabilities.
Structured transport. The transport decomposition in Lemma A.4, with p = 1 p=1 p = 1 , gives
W 1 , I ( G θ I , G θ ⋆ I ) ≤ V 1 , I + C 0 Δ s e p 2 D I + R J I . W_{1,I}(G_\theta^I,G_{\theta_\star}^I)\le V_{1,I}+\frac{C_0\Delta_{\mathrm{sep}}}{2}D_I+R J_I. W 1 , I ( G θ I , G θ ⋆ I ) ≤ V 1 , I + 2 C 0 Δ sep D I + R J I .
The displacement term is bounded by D μ D_\mu D μ , and Lemma D.2 bounds J I J_I J I by C r Δ s e p − ( 2 s max − 2 ) h ( p θ , T , p θ ⋆ , T ) C_r\Delta_{\mathrm{sep}}^{-(2s_{\max}-2)}h(p_{\theta,T},p_{\theta_\star,T}) C r Δ sep − ( 2 s m a x − 2 ) h ( p θ , T , p θ ⋆ , T ) . The state-sequence bound just proved gives
C 0 Δ s e p 2 D I ≤ C r Δ s e p − ( 2 s max − 2 ) h ( p θ , T , p θ ⋆ , T ) . \frac{C_0\Delta_{\mathrm{sep}}}{2}D_I\le C_r\Delta_{\mathrm{sep}}^{-(2s_{\max}-2)}h(p_{\theta,T},p_{\theta_\star,T}). 2 C 0 Δ sep D I ≤ C r Δ sep − ( 2 s m a x − 2 ) h ( p θ , T , p θ ⋆ , T ) .
All three contributions therefore have separation exponent 2 s max − 2 2s_{\max}-2 2 s m a x − 2 , proving (32) .
Unstructured transport. For a single selected time, apply (D.10) of Lemma D.4 to the corresponding location measures, and then use marginalization. This gives (34) , even when some true or fitted state probabilities at that time vanish. For every nonempty I I I , move fitted locations to their assigned true centers and then couple the two probability measures on the true grid. This gives
W 1 , I ( G θ I , G θ ⋆ I ) ≤ V 1 , I + 1 2 D ⋆ D I . W_{1,I}(G_\theta^I,G_{\theta_\star}^I)\le V_{1,I}+\frac{1}{2}\mathcal{D}_\star D_I. W 1 , I ( G θ I , G θ ⋆ I ) ≤ V 1 , I + 2 1 D ⋆ D I .
The first term is bounded by D μ ≤ C Δ − ( 2 K − 2 ) h ( p θ , T , p θ ⋆ , T ) D_\mu\le C\Delta^{-(2K-2)}h(p_{\theta,T},p_{\theta_\star,T}) D μ ≤ C Δ − ( 2 K − 2 ) h ( p θ , T , p θ ⋆ , T ) . Equation (D.27) bounds the second by C r D ⋆ − 1 Δ − ( 2 K − 3 ) h ( p θ , T , p θ ⋆ , T ) C_r\mathcal{D}_\star^{-1}\Delta^{-(2K-3)}h(p_{\theta,T},p_{\theta_\star,T}) C r D ⋆ − 1 Δ − ( 2 K − 3 ) h ( p θ , T , p θ ⋆ , T ) , which has separation exponent 2 K − 2 2K-2 2 K − 2 because D ⋆ ≥ Δ \mathcal{D}_\star\ge\Delta D ⋆ ≥ Δ . Their sum proves (35) . No state probability has been divided out in any of these comparisons.
Sharpness of the exact-fit exponents
The preceding bounds hold without restrictions on the rank of the transition matrix. This includes HMMs whose hidden states are independent. Moment matching in that submodel gives lower examples for the separation powers, even after the centers have been paired.
Proposition D.8 (Sharpness under exact specification). Fix s ≥ 2 s \ge2 s ≥ 2 , a dimension d ≥ 1 d \ge1 d ≥ 1 , and an observation horizon T ≥ 2 T \ge2 T ≥ 2 . There are exactly specified Gaussian HMMs with K = s K=s K = s , positive initial probabilities bounded away from zero, and a single cluster with Δ s e p ≍ δ ↓ 0 \Delta_{\mathrm{sep}} \asymp\delta\downarrow0 Δ sep ≍ δ ↓ 0 , such that their centers satisfy the matching condition of Theorem 3.6 and
h ( p θ δ , T , p θ δ ∗ , T ) ≍ δ 2 s − 1 , (135) h(p_{\theta_\delta,T},p_{\theta_\delta^*,T}) \asymp\delta^{2s-1}, \tag*{(135)} h ( p θ δ , T , p θ δ ∗ , T ) ≍ δ 2 s − 1 , (135) D π ≍ D A ≍ D I ≍ 1 , ∅ ≠ I ⊆ [ T ] , (136) D_\pi\asymp D_A \asymp D_I \asymp1,\qquad\varnothing\ne I \subseteq[T], \tag*{(136)} D π ≍ D A ≍ D I ≍ 1 , ∅ = I ⊆ [ T ] , (136) D μ ≍ W 1 , I ( G θ δ I , G θ δ ∗ I ) ≍ δ . (137) D_\mu\asymp W_{1,I}(G_{\theta_\delta}^I,G_{\theta_\delta^*}^I) \asymp\delta. \tag*{(137)} D μ ≍ W 1 , I ( G θ δ I , G θ δ ∗ I ) ≍ δ . (137) The same construction can be embedded in a largest cluster of size s s s with additional, fixed, separated clusters. Consequently, the structured probability exponent 2 s − 1 2s-1 2 s − 1 and transport exponent 2 s − 2 2s-2 2 s − 2 cannot be decreased uniformly over classes containing these collinear configurations. Taking s = K s=K s = K also proves sharpness of the unstructured probability and transport exponents 2 K − 1 2K-1 2 K − 1 and 2 K − 2 2K-2 2 K − 2 . This statement concerns the separation exponents of the deterministic inverse inequalities, rather than exact minimax constants or every individual spatial configuration.
Proof. We construct two fixed moment-matched mixing measures and then scale their supports.
Positive atomic measures with matching lower moments. Choose distinct real numbers t 1 < ⋯ < t s t_1 < \cdots< t_s t 1 < ⋯ < t s with minimum gap one and strictly positive weights w j w_j w j summing to one. Consider the moments
M m ( w , t ) = ∑ j = 1 s w j t j m , m = 0 , … , 2 s − 1. M_m(w,t)=\sum_{j=1}^{s} w_j t_j^m,\qquad m=0,\ldots,2s-1. M m ( w , t ) = j = 1 ∑ s w j t j m , m = 0 , … , 2 s − 1.
initially allowing the s s s weights to vary freely. The Jacobian of ( M 0 , … , M 2 s − 1 ) (M_0,\ldots,M_{2s-1}) ( M 0 , … , M 2 s − 1 ) in the 2 s 2s 2 s weight and location coordinates is invertible. Indeed, a linear dependence between its rows would give a polynomial P P P of degree at most 2 s − 1 2s-1 2 s − 1 satisfying
P ( t j ) = 0 , w j P ′ ( t j ) = 0 , j = 1 , … , s . P(t_j)=0,\qquad w_jP'(t_j)=0,\qquad j=1,\ldots,s. P ( t j ) = 0 , w j P ′ ( t j ) = 0 , j = 1 , … , s .
Since all w j > 0 w_j>0 w j > 0 , this polynomial has s s s double roots and must vanish identically. The inverse function theorem therefore supplies a smooth local inverse for this complete moment map. Apply that inverse to the moment vector whose first 2 s − 1 2s-1 2 s − 1 coordinates equal their original values and whose last coordinate is M 2 s − 1 ( w , t ) + u M_{2s-1}(w,t)+u M 2 s − 1 ( w , t ) + u . This defines a smooth curve ( w ( u ) , t ( u ) ) (w(u),t(u)) ( w ( u ) , t ( u )) such that
M j ( w ( u ) , t ( u ) ) = M j ( w , t ) ( 0 ≤ j ≤ 2 s − 2 ) , M 2 s − 1 ( w ( u ) , t ( u ) ) − M 2 s − 1 ( w , t ) = u . M_j(w(u),t(u))=M_j(w,t)\quad(0\le j\le2s-2),\qquad M_{2s-1}(w(u),t(u))-M_{2s-1}(w,t)=u. M j ( w ( u ) , t ( u )) = M j ( w , t ) ( 0 ≤ j ≤ 2 s − 2 ) , M 2 s − 1 ( w ( u ) , t ( u )) − M 2 s − 1 ( w , t ) = u .
In particular, its tangent is nonzero and its first unmatched moment is exactly u u u , not merely a quantity with a potentially vanishing higher-order remainder. The tangent can be written explicitly. Put g ( z ) = ∏ j = 1 s ( z − t j ) g(z)=\prod_{j=1}^{s}(z-t_j) g ( z ) = ∏ j = 1 s ( z − t j ) . The value and derivative Hermite cardinal polynomials have respective leading coefficients
b j = − g ′ ′ ( t j ) g ′ ( t j ) 3 , c j = 1 g ′ ( t j ) 2 . (138) b_j=-\frac{g''(t_j)}{g'(t_j)^3},\qquad c_j=\frac{1}{g'(t_j)^2}. \tag*{(138)} b j = − g ′ ( t j ) 3 g ′′ ( t j ) , c j = g ′ ( t j ) 2 1 . (138)
Indeed, with ℓ j ( z ) = g ( z ) / { g ′ ( t j ) ( z − t j ) } \ell_j(z) = g(z)/\{g'(t_j)(z-t_j)\} ℓ j ( z ) = g ( z ) / { g ′ ( t j ) ( z − t j )} , those polynomials are ( 1 − 2 ℓ j ′ ( t j ) ( z − t j ) ) ℓ j ( z ) 2 (1-2\ell_j'(t_j)(z-t_j))\ell_j(z)^2 ( 1 − 2 ℓ j ′ ( t j ) ( z − t j )) ℓ j ( z ) 2 and ( z − t j ) ℓ j ( z ) 2 (z-t_j)\ell_j(z)^2 ( z − t j ) ℓ j ( z ) 2 ; here 2 ℓ j ′ ( t j ) = g ′ ′ ( t j ) / g ′ ( t j ) 2\ell_j'(t_j)=g''(t_j)/g'(t_j) 2 ℓ j ′ ( t j ) = g ′′ ( t j ) / g ′ ( t j ) . Comparing the leading coefficients in the Hermite interpolation identity for polynomials of degree at most 2 s − 1 2s-1 2 s − 1 shows that the unique derivative of the moment-inverse curve is
w j ′ ( 0 ) = b j , t j ′ ( 0 ) = c j w j . (139) w_j'(0)=b_j,\qquad t_j'(0)=\frac{c_j}{w_j}. \tag*{(139)} w j ′ ( 0 ) = b j , t j ′ ( 0 ) = w j c j . (139)
Every location derivative is positive. The weight derivative vector is nonzero because g ′ ′ ( t s ) / g ′ ( t s ) = 2 ∑ j < s ( t s − t j ) − 1 > 0 g''(t_s)/g'(t_s)=2\sum_{j<s}(t_s-t_j)^{-1}>0 g ′′ ( t s ) / g ′ ( t s ) = 2 ∑ j < s ( t s − t j ) − 1 > 0 . Thus both kinds of parameters genuinely change along this curve. Choose a sufficiently small fixed u ≠ 0 u\ne0 u = 0 so that all weights remain positive, the weight vectors differ, the location vectors differ, and ∣ t j ( u ) − t j ∣ < 1 / 4 |t_j(u)-t_j|<1/4 ∣ t j ( u ) − t j ∣ < 1/4 . Set
ν ⋆ = ∑ j = 1 s w j δ t j , ν = ∑ j = 1 s w j ( u ) δ t j ( u ) . \nu^\star=\sum_{j=1}^{s}w_j\delta_{t_j},\qquad\nu=\sum_{j=1}^{s}w_j(u)\delta_{t_j(u)}. ν ⋆ = j = 1 ∑ s w j δ t j , ν = j = 1 ∑ s w j ( u ) δ t j ( u ) .
These measures match moments through order 2 s − 2 2s-2 2 s − 2 and have different moments of order m = 2 s − 1 m=2s-1 m = 2 s − 1 .
Scaling and Gaussian density comparison. Scale all locations by δ \delta δ and embed them along the first coordinate of R d \mathbb{R}^d R d . For sufficiently small δ \delta δ , both mixing measures lie in B d ( 0 , R ) B_d(0,R) B d ( 0 , R ) , the true minimum separation is δ \delta δ , and the paired displacements are below δ / 4 \delta/4 δ /4 . Their cluster diameter is a fixed multiple of δ \delta δ . Write p δ , p δ ⋆ p_\delta,p_\delta^\star p δ , p δ ⋆ for the scalar Gaussian mixture densities on the first coordinate; the other coordinates have a common independent standard Gaussian density. Taylor-expand φ ( x − δ t ) \varphi(x-\delta t) φ ( x − δ t ) in its shift through degree m − 1 m-1 m − 1 . Every term cancels in p δ − p δ ⋆ p_\delta-p_\delta^\star p δ − p δ ⋆ by the matched moments. The locations t j , t j ( u ) t_j,t_j(u) t j , t j ( u ) lie in a fixed bounded interval, so the integral remainder and the formula for derivatives of the Gaussian imply
∣ p δ ( x ) − p δ ⋆ ( x ) ∣ ≤ C δ m ( 1 + ∣ x ∣ m ) φ ( x ) e C δ ∣ x ∣ , 0 < δ ≤ δ 0 . |p_\delta(x)-p_\delta^\star(x)|\le C\delta^m(1+|x|^m)\varphi(x)e^{C\delta|x|},\qquad0<\delta\le\delta_0. ∣ p δ ( x ) − p δ ⋆ ( x ) ∣ ≤ C δ m ( 1 + ∣ x ∣ m ) φ ( x ) e C δ ∣ x ∣ , 0 < δ ≤ δ 0 .
Also, since the mixture weights are positive and the shifts are bounded by C δ C\delta C δ ,
p δ ⋆ ( x ) ≥ c φ ( x ) e − C δ ∣ x ∣ − C δ 2 . p_\delta^\star(x)\ge c\varphi(x)e^{-C\delta|x|-C\delta^2}. p δ ⋆ ( x ) ≥ c φ ( x ) e − C δ ∣ x ∣ − C δ 2 .
Consequently,
h 2 ( p δ , p δ ⋆ ) ≤ 1 2 ∫ ( p δ − p δ ⋆ ) 2 p δ ⋆ d x ≤ C δ 2 m ∫ ( 1 + ∣ x ∣ 2 m ) φ ( x ) e C δ ∣ x ∣ d x ≤ C δ 2 m . h^2(p_\delta,p_\delta^\star)\le\frac{1}{2}\int\frac{(p_\delta-p_\delta^\star)^2}{p_\delta^\star}\,dx
\le C\delta^{2m}\int(1+|x|^{2m})\varphi(x)e^{C\delta|x|}\,dx\le C\delta^{2m}. h 2 ( p δ , p δ ⋆ ) ≤ 2 1 ∫ p δ ⋆ ( p δ − p δ ⋆ ) 2 d x ≤ C δ 2 m ∫ ( 1 + ∣ x ∣ 2 m ) φ ( x ) e C δ ∣ x ∣ d x ≤ C δ 2 m .
The last integral is bounded uniformly for δ ≤ δ 0 \delta\le\delta_0 δ ≤ δ 0 . Conversely, the difference of the latent m m m th moments is a nonzero fixed constant times δ m \delta^m δ m . Applying Lemma A.3 to x m x^m x m on the fixed ball B d ( 0 , R ) B_d(0,R) B d ( 0 , R ) gives h ( p δ , p δ ⋆ ) ≥ c δ m h(p_\delta,p_\delta^\star)\ge c\delta^m h ( p δ , p δ ⋆ ) ≥ c δ m . Its norm is bounded by R m R^m R m , independently of δ \delta δ ; no norm on a shrinking support is used. Hence their Hellinger distance is of order δ 2 s − 1 \delta^{2s-1} δ 2 s − 1 .
Embedding into HMMs and comparing the latent losses. Take the true initial distribution and each row of its transition matrix to be w w w , and take the fitted initial distribution and each fitted transition row to be w ( u ) w(u) w ( u ) . Each observation sequence is then independent over time. The product affinity identity gives
h 2 ( p θ δ , T , p θ δ ⋆ , T ) = 1 − { 1 − h 2 ( p δ , p δ ⋆ ) } T , h^2(p_{\theta_\delta,T},p_{\theta_\delta^\star,T})=1-\{1-h^2(p_\delta,p_\delta^\star)\}^{T}, h 2 ( p θ δ , T , p θ δ ⋆ , T ) = 1 − { 1 − h 2 ( p δ , p δ ⋆ ) } T ,
which lies between the one-time squared Hellinger distance and T T T times that distance. This proves (D.28).
The weight vectors differ by a fixed nonzero amount, so D π D_\pi D π is bounded above and below by positive constants. Marginalization of any two-time or path distribution gives the same lower bound for D A D_A D A , D I D_I D I , and their universal upper bound is two. The weighted center displacement is a positive fixed constant times δ \delta δ . Finally,
W 1 ( G θ δ { 1 } , G θ δ ⋆ { 1 } ) = δ W 1 ( ν , ν ⋆ ) > 0. W_1\left(G_{\theta_\delta}^{\{1\}},G_{\theta_\delta^\star}^{\{1\}}\right)=\delta W_1(\nu,\nu^\star)>0. W 1 ( G θ δ { 1 } , G θ δ ⋆ { 1 } ) = δ W 1 ( ν , ν ⋆ ) > 0.
For the normalized additive path metric d 1 , r d_{1,r} d 1 , r , the Wasserstein distance between product measures equals this marginal distance. A product of optimal marginal couplings proves the upper bound; averaging an optimal marginal Lipschitz test function over the r r r coordinates proves the lower bound. This establishes (D.29)–(D.30).
To include further clusters, assign the moving cluster a fixed positive total mass and add identical fixed atoms outside it in both models, again using identical rows within each model. The density difference is multiplied by this fixed mass, while the preceding upper Hellinger argument remains valid using the moving cluster as a lower bound for the reference density. The m m m th moment difference still gives the matching lower bound. Probability and displacement discrepancies retain their orders. A Lipschitz transport potential on the moving cluster extends to the additional common atoms, whose signed contributions cancel, so the transport order also remains unchanged. This proves the claimed embedding.
Example D.9 (Aggregation can retain higher-order dependence). Consider a two-state true HMM with initial probabilities ( 1 / 2 , 1 / 2 ) (1/2,1/2) ( 1/2 , 1/2 ) , both transition rows ( 1 / 2 , 1 / 2 ) (1/2,1/2) ( 1/2 , 1/2 ) , and scalar emission centers − δ , δ -\delta,\delta − δ , δ . For a fitted eight-state HMM, label the states by ( u , v , w ) ∈ { 0 , 1 } 3 (u,v,w)\in\{0,1\}^3 ( u , v , w ) ∈ { 0 , 1 } 3 , use the uniform initial distribution, and take the deterministic transition
( u , v , w ) ↦ ( v , w , u ⊕ v ⊕ w ) , (u,v,w)\mapsto(v,w,u\oplus v\oplus w), ( u , v , w ) ↦ ( v , w , u ⊕ v ⊕ w ) , where ⊕ \oplus ⊕ denotes addition modulo two. The fitted emission center is − δ -\delta − δ or δ \delta δ according to its first bit. Its first four aggregate labels are
U , V , W , U ⊕ V ⊕ W , U,\quad V,\quad W,\quad U\oplus V\oplus W, U , V , W , U ⊕ V ⊕ W , where U , V , W U,V,W U , V , W are independent uniform bits. Every marginal on at most three of these four coordinates is uniform and agrees with the corresponding true marginal. However, the fitted joint distribution of the four coordinates is uniform on the eight even-parity sequences, whereas the true joint distribution is uniform on all sixteen sequences. Thus D { 1 , 2 , 3 , 4 } = 1 D_{\{1,2,3,4\}}=1 D { 1 , 2 , 3 , 4 } = 1 although all corresponding two-time discrepancies vanish.
Writing f 0 ( x ) = φ ( x + δ ) f_0(x)=\varphi(x+\delta) f 0 ( x ) = φ ( x + δ ) and f 1 ( x ) = φ ( x − δ ) f_1(x)=\varphi(x-\delta) f 1 ( x ) = φ ( x − δ ) , the four-observation density difference is exactly
p θ , 4 − p θ ⋆ , 4 = 1 16 ∏ ℓ = 1 4 { f 0 ( x ℓ ) − f 1 ( x ℓ ) } . p_{\theta,4}-p_{\theta^\star,4}=\frac{1}{16}\prod_{\ell=1}^{4}\{f_0(x_\ell)-f_1(x_\ell)\}. p θ , 4 − p θ ⋆ , 4 = 16 1 ℓ = 1 ∏ 4 { f 0 ( x ℓ ) − f 1 ( x ℓ )} . Its Hellinger distance is of order δ 4 \delta^4 δ 4 . For the upper bound, divide the squared density difference by the true density ∏ ℓ ( f 0 + f 1 ) / 2 \prod_\ell(f_0+f_1)/2 ∏ ℓ ( f 0 + f 1 ) /2 and integrate; each scalar factor is O ( δ 2 ) O(\delta^2) O ( δ 2 ) by the mean-value formula for the Gaussian. For the lower bound, apply Lemma A.3 to x 1 x 2 x 3 x 4 x_1x_2x_3x_4 x 1 x 2 x 3 x 4 : its latent expectation differs by δ 4 \delta^4 δ 4 . This example satisfies the over-specified parameter assumptions, including repeated fitted centers and zero transition entries. It shows why Lemma D.7 cannot be applied to an arbitrary aggregate fitted process, and why the structured exact-fit probability exponent 3 for two true states does not extend to all four-time joint distributions under over-specification.
Proofs of Proposition 3.7 and Corollary 3.8
Proof of Proposition 3.7 . Let θ ^ n \widehat{\theta}_n θ n be the exact-fit MLE over Θ K ( R ) \Theta_K(\mathbb{R}) Θ K ( R ) . Use the Borel relabeling convention in Appendix § 10 .§ 1 , including its identity-permutation extension outside the matching event. Define
E n = { h ( p θ ^ n , T , p θ ⋆ , T ) ≤ ε n , T } . \mathcal{E}_n = \left\{h(p_{\widehat{\theta}_n,T},p_{\theta^\star,T}) \le\varepsilon_{n,T}\right\}. E n = { h ( p θ n , T , p θ ⋆ , T ) ≤ ε n , T } .
By Proposition 2.2 , P ⋆ ( E n ) ≥ 1 − 1 n \mathbb{P}_\star(\mathcal{E}_n) \ge1-\frac{1}{n} P ⋆ ( E n ) ≥ 1 − n 1 . The sample-size condition in (27) ensures that ε n , T ≤ c ( π min ⋆ ) Δ s e p 2 s max − 1 \varepsilon_{n,T} \le c(\pi^\star_{\min})\Delta_{\mathrm{sep}}^{2s_{\max}-1} ε n , T ≤ c ( π m i n ⋆ ) Δ sep 2 s m a x − 1 . On E n \mathcal{E}_n E n , we have:
h ( p θ ^ n , T , p θ ⋆ , T ) ≤ ε n , T ≤ c ( π min ⋆ ) Δ s e p 2 s max − 1 . h(p_{\widehat{\theta}_n,T},p_{\theta^\star,T}) \le\varepsilon_{n,T} \le c(\pi^\star_{\min})\Delta_{\mathrm{sep}}^{2s_{\max}-1}. h ( p θ n , T , p θ ⋆ , T ) ≤ ε n , T ≤ c ( π m i n ⋆ ) Δ sep 2 s m a x − 1 .
Corollary 3.5 therefore supplies the matching required by Theorem 3.6 . Under this matching, its displacement bound gives
D μ ( θ ^ n , θ ⋆ ) ≤ C Δ s e p − ( 2 s max − 2 ) h ( p θ ^ n , T , p θ ⋆ , T ) ≤ C Δ s e p − ( 2 s max − 2 ) ε n , T ( because E n holds ) . \begin{aligned}
D_\mu(\widehat{\theta}_n,\theta^\star) \le C\Delta_{\mathrm{sep}}^{-(2s_{\max}-2)}h(p_{\widehat{\theta}_n,T},p_{\theta^\star,T}) \\
&\le C\Delta_{\mathrm{sep}}^{-(2s_{\max}-2)}\varepsilon_{n,T} \qquad(\text{because } \mathcal{E}_n\text{ holds}).
\end{aligned} D μ ( θ n , θ ⋆ ) ≤ C Δ sep − ( 2 s m a x − 2 ) h ( p θ n , T , p θ ⋆ , T ) ≤ C Δ sep − ( 2 s m a x − 2 ) ε n , T ( because E n holds ) .
Its one-time probability bound gives
D π ( θ ^ n , θ ⋆ ) ≤ C Δ s e p − ( 2 s max − 1 ) h ( p θ ^ n , T , p θ ⋆ , T ) ≤ C Δ s e p − ( 2 s max − 1 ) ε n , T , \begin{aligned}
D_\pi(\widehat{\theta}_n,\theta^\star) \le C\Delta_{\mathrm{sep}}^{-(2s_{\max}-1)}h(p_{\widehat{\theta}_n,T},p_{\theta^\star,T}) \\
&\le C\Delta_{\mathrm{sep}}^{-(2s_{\max}-1)}\varepsilon_{n,T},
\end{aligned} D π ( θ n , θ ⋆ ) ≤ C Δ sep − ( 2 s m a x − 1 ) h ( p θ n , T , p θ ⋆ , T ) ≤ C Δ sep − ( 2 s m a x − 1 ) ε n , T ,
and, in the structured regimes, its consecutive-pair bound gives
D A ( θ ^ n , θ ⋆ ) ≤ C Δ s e p − ( 2 s max − 1 ) h ( p θ ^ n , T , p θ ⋆ , T ) ≤ C Δ s e p − ( 2 s max − 1 ) ε n , T . \begin{aligned}
D_A(\widehat{\theta}_n,\theta^\star) \le C\Delta_{\mathrm{sep}}^{-(2s_{\max}-1)}h(p_{\widehat{\theta}_n,T},p_{\theta^\star,T}) \\
&\le C\Delta_{\mathrm{sep}}^{-(2s_{\max}-1)}\varepsilon_{n,T}.
\end{aligned} D A ( θ n , θ ⋆ ) ≤ C Δ sep − ( 2 s m a x − 1 ) h ( p θ n , T , p θ ⋆ , T ) ≤ C Δ sep − ( 2 s m a x − 1 ) ε n , T .
The structured joint probability and transport bounds follow in the same way from Theorem 3.6 . In the unstructured regime, take s max = K s_{\max}=K s m a x = K : the probability exponent is 2 K − 1 2K-1 2 K − 1 and the Wasserstein exponent is 2 K − 2 2K-2 2 K − 2 for every nonempty set of selected times. All these inequalities hold on E n \mathcal{E}_n E n , with probability at least 1 − 1 / n 1-1/n 1 − 1/ n . Substituting ε n , T = C d log ( n T d ) / n \varepsilon_{n,T}=C\sqrt{d\log(nTd)/n} ε n , T = C d log ( n T d ) / n gives the stated O P O_{\mathbb{P}} O P rates. □
Proof of Corollary 3.8 . Work under the localization condition and use the aligned labeling supplied by Corollary 3.5 . In particular,
π j ≥ π j ⋆ 2 ≥ π min ⋆ 2 , j ∈ [ K ] . \pi_j \ge\frac{\pi_j^\star}{2} \ge\frac{\pi_{\min}^\star}{2}, \qquad j\in[K]. π j ≥ 2 π j ⋆ ≥ 2 π m i n ⋆ , j ∈ [ K ] .
Maximum center displacement. Write h j = ∥ b j − a j ∥ 2 h_j=\lVert b_j-a_j\rVert_2 h j = ∥ b j − a j ∥ 2 , and choose j 0 j_0 j 0 such that h j 0 = max j h j h_{j_0}=\max_j h_j h j 0 = max j h j . The fitted initial probability lower bound allows us to remove the probability weight from this largest displacement:
π min ⋆ 2 max j h j = π min ⋆ 2 h j 0 ≤ π j 0 h j 0 ( because π j 0 ≥ π min ⋆ / 2 ) ≤ ∑ j = 1 K π j h j ( because every summand is nonnegative ) ≤ D μ ( because q 1 = π ) ≤ C Δ s e p − ( 2 s max − 2 ) h ( p θ , T , p θ ⋆ , T ) ( by x r e f ‘ T h e o r e m 3.6 ‘ ) . \begin{aligned}
\frac{\pi_{\min}^\star}{2}\max_j h_j = \frac{\pi_{\min}^\star}{2}h_{j_0} \\
&\le\pi_{j_0}h_{j_0} \qquad(\text{because } \pi_{j_0}\ge\pi_{\min}^\star/2) \\
&\le\sum_{j=1}^{K}\pi_jh_j \qquad(\text{because every summand is nonnegative}) \\
&\le D_\mu\qquad(\text{because } q_1=\pi) \\
&\le C\Delta_{\mathrm{sep}}^{-(2s_{\max}-2)}h(p_{\theta,T},p_{\theta^\star,T}) \qquad(\text{by } {xref}`Theorem 3.6`).
\end{aligned} 2 π m i n ⋆ j max h j = 2 π m i n ⋆ h j 0 ≤ π j 0 h j 0 ( because π j 0 ≥ π m i n ⋆ /2 ) ≤ j = 1 ∑ K π j h j ( because every summand is nonnegative ) ≤ D μ ( because q 1 = π ) ≤ C Δ sep − ( 2 s m a x − 2 ) h ( p θ , T , p θ ⋆ , T ) ( by x r e f ‘ T h eor e m 3.6‘ ) .
Dividing by π min ⋆ / 2 > 0 \pi^\star_{\min}/2 > 0 π m i n ⋆ /2 > 0 proves
max j ∥ b j − a j ∥ 2 ≤ C π min ⋆ Δ s e p − ( 2 s max − 2 ) h ( p θ , T , p θ ⋆ , T ) . \max_j \lVert b_j-a_j\rVert_2 \le\frac{C}{\pi^\star_{\min}}\Delta_{\mathrm{sep}}^{-(2s_{\max}-2)}h(p_{\theta,T},p_{\theta^\star,T}). j max ∥ b j − a j ∥ 2 ≤ π m i n ⋆ C Δ sep − ( 2 s m a x − 2 ) h ( p θ , T , p θ ⋆ , T ) .
Transition-row discrepancies. For the first consecutive pair, define
F j ℓ = π j A j ℓ , F j ℓ ⋆ = π j ⋆ A j ℓ ⋆ . F_{j\ell}=\pi_j A_{j\ell},\qquad F^\star_{j\ell}=\pi^\star_j A^\star_{j\ell}. F j ℓ = π j A j ℓ , F j ℓ ⋆ = π j ⋆ A j ℓ ⋆ .
These are the fitted and true joint probabilities of the states at times 1 and 2. Their row sums equal the initial probabilities:
∑ ℓ = 1 K F j ℓ = π j ∑ ℓ = 1 K A j ℓ = π j (because transition rows sum to one), \sum_{\ell=1}^{K}F_{j\ell}=\pi_j\sum_{\ell=1}^{K}A_{j\ell}=\pi_j\qquad\text{(because transition rows sum to one),} ℓ = 1 ∑ K F j ℓ = π j ℓ = 1 ∑ K A j ℓ = π j (because transition rows sum to one),
∑ ℓ = 1 K F j ℓ ⋆ = π j ⋆ ∑ ℓ = 1 K A j ℓ ⋆ = π j ⋆ . \sum_{\ell=1}^{K}F^\star_{j\ell}=\pi^\star_j\sum_{\ell=1}^{K}A^\star_{j\ell}=\pi^\star_j. ℓ = 1 ∑ K F j ℓ ⋆ = π j ⋆ ℓ = 1 ∑ K A j ℓ ⋆ = π j ⋆ .
It follows that the initial-probability discrepancy is bounded by the discrepancy in the corresponding flow row:
∣ π j − π j ⋆ ∣ = ∣ ∑ ℓ = 1 K ( F j ℓ − F j ℓ ⋆ ) ∣ ≤ ∑ ℓ = 1 K ∣ F j ℓ − F j ℓ ⋆ ∣ (by the triangle inequality). (140) \begin{aligned}
\left|\pi_j-\pi^\star_j\right|=\left|\sum_{\ell=1}^{K}(F_{j\ell}-F^\star_{j\ell})\right| \\
&\le\sum_{\ell=1}^{K}\left|F_{j\ell}-F^\star_{j\ell}\right|\qquad\text{(by the triangle inequality).} \tag*{(140)}
\end{aligned} π j − π j ⋆ = ℓ = 1 ∑ K ( F j ℓ − F j ℓ ⋆ ) ≤ ℓ = 1 ∑ K F j ℓ − F j ℓ ⋆ (by the triangle inequality). (140)
To pass from joint flows to conditional transition probabilities, isolate the transition-row difference entry by entry:
π j ⋆ ( A j ℓ − A j ℓ ⋆ ) = π j ⋆ A j ℓ − F j ℓ ⋆ = π j A j ℓ − F j ℓ ⋆ + ( π j ⋆ − π j ) A j ℓ = F j ℓ − F j ℓ ⋆ + ( π j ⋆ − π j ) A j ℓ . \begin{aligned}
\pi^\star_j(A_{j\ell}-A^\star_{j\ell})=\pi^\star_jA_{j\ell}-F^\star_{j\ell} \\
&=\pi_jA_{j\ell}-F^\star_{j\ell}+(\pi^\star_j-\pi_j)A_{j\ell} \\
&=F_{j\ell}-F^\star_{j\ell}+(\pi^\star_j-\pi_j)A_{j\ell}.
\end{aligned} π j ⋆ ( A j ℓ − A j ℓ ⋆ ) = π j ⋆ A j ℓ − F j ℓ ⋆ = π j A j ℓ − F j ℓ ⋆ + ( π j ⋆ − π j ) A j ℓ = F j ℓ − F j ℓ ⋆ + ( π j ⋆ − π j ) A j ℓ .
Taking absolute values gives
π j ⋆ ∣ A j ℓ − A j ℓ ⋆ ∣ ≤ ∣ F j ℓ − F j ℓ ⋆ ∣ + ∣ π j ⋆ − π j ∣ A j ℓ , \pi^\star_j\left|A_{j\ell}-A^\star_{j\ell}\right|\le\left|F_{j\ell}-F^\star_{j\ell}\right|+\left|\pi^\star_j-\pi_j\right|A_{j\ell}, π j ⋆ A j ℓ − A j ℓ ⋆ ≤ F j ℓ − F j ℓ ⋆ + π j ⋆ − π j A j ℓ ,
where we used π j ⋆ > 0 \pi^\star_j>0 π j ⋆ > 0 and A j ℓ ≥ 0 A_{j\ell}\ge0 A j ℓ ≥ 0 . Summing over ℓ \ell ℓ yields
π j ⋆ ∥ A j ⋅ − A j ⋅ ⋆ ∥ 1 ≤ ∑ ℓ = 1 K ∣ F j ℓ − F j ℓ ⋆ ∣ + ∣ π j ⋆ − π j ∣ ∑ ℓ = 1 K A j ℓ = ∑ ℓ = 1 K ∣ F j ℓ − F j ℓ ⋆ ∣ + ∣ π j ⋆ − π j ∣ (because ∑ ℓ A j ℓ = 1 ) \begin{aligned}
\pi^\star_j\lVert A_{j\cdot}-A^\star_{j\cdot}\rVert_1
\le\sum_{\ell=1}^{K}\left|F_{j\ell}-F^\star_{j\ell}\right|+\left|\pi^\star_j-\pi_j\right|\sum_{\ell=1}^{K}A_{j\ell} \\
&=\sum_{\ell=1}^{K}\left|F_{j\ell}-F^\star_{j\ell}\right|+\left|\pi^\star_j-\pi_j\right|\qquad\text{(because }\sum_{\ell}A_{j\ell}=1\text{)}
\end{aligned} π j ⋆ ∥ A j ⋅ − A j ⋅ ⋆ ∥ 1 ≤ ℓ = 1 ∑ K F j ℓ − F j ℓ ⋆ + π j ⋆ − π j ℓ = 1 ∑ K A j ℓ = ℓ = 1 ∑ K F j ℓ − F j ℓ ⋆ + π j ⋆ − π j (because ℓ ∑ A j ℓ = 1 )
≤ 2 ∑ ℓ = 1 K ∣ F j ℓ − F j ℓ ⋆ ∣ (by Equation xref‘(D.32‘). \le2\sum_{\ell=1}^{K}\left|F_{j\ell}-F^\star_{j\ell}\right|
\qquad\text{(by Equation {xref}`(D.32`).} ≤ 2 ℓ = 1 ∑ K F j ℓ − F j ℓ ⋆ (by Equation xref ‘(D.32‘).
We divide by the positive true initial probability to obtain
∥ A j ⋅ − A j ⋅ ⋆ ∥ 1 ≤ 2 π j ⋆ ∑ ℓ = 1 K ∣ F j ℓ − F j ℓ ⋆ ∣ ≤ 2 π min ⋆ ∑ ℓ = 1 K ∣ F j ℓ − F j ℓ ⋆ ∣ ≤ 2 π min ⋆ ∑ i = 1 K ∑ ℓ = 1 K ∣ F i ℓ − F i ℓ ⋆ ∣ = 2 π min ⋆ D { 1 , 2 } (because the labels are aligned). (141) \begin{aligned}
\lVert A_{j\cdot}-A^\star_{j\cdot}\rVert_1 \le\frac{2}{\pi^\star_j}\sum_{\ell=1}^{K}|F_{j\ell}-F^\star_{j\ell}| \\
&\le\frac{2}{\pi^\star_{\min}}\sum_{\ell=1}^{K}|F_{j\ell}-F^\star_{j\ell}| \\
&\le\frac{2}{\pi^\star_{\min}}\sum_{i=1}^{K}\sum_{\ell=1}^{K}|F_{i\ell}-F^\star_{i\ell}| \\
&= \frac{2}{\pi^\star_{\min}}D_{\{1,2\}}\qquad\text{(because the labels are aligned).} \tag*{(141)}
\end{aligned} ∥ A j ⋅ − A j ⋅ ⋆ ∥ 1 ≤ π j ⋆ 2 ℓ = 1 ∑ K ∣ F j ℓ − F j ℓ ⋆ ∣ ≤ π m i n ⋆ 2 ℓ = 1 ∑ K ∣ F j ℓ − F j ℓ ⋆ ∣ ≤ π m i n ⋆ 2 i = 1 ∑ K ℓ = 1 ∑ K ∣ F i ℓ − F i ℓ ⋆ ∣ = π m i n ⋆ 2 D { 1 , 2 } (because the labels are aligned). (141)
In the structured regimes, the two-time probability bound of Theorem 3.6 gives
D { 1 , 2 } ≤ C Δ s e p − ( 2 s max − 1 ) h ( p θ , T , p θ ⋆ , T ) . D_{\{1,2\}} \le C\Delta_{\mathrm{sep}}^{-(2s_{\max}-1)}h(p_{\theta,T},p_{\theta^\star,T}). D { 1 , 2 } ≤ C Δ sep − ( 2 s m a x − 1 ) h ( p θ , T , p θ ⋆ , T ) .
Substitution into Equation (D.33), followed by taking the maximum over j j j , proves
max j ∥ A j ⋅ − A j ⋅ ⋆ ∥ 1 ≤ C π min ⋆ Δ s e p − ( 2 s max − 1 ) h ( p θ , T , p θ ⋆ , T ) . \max_j\lVert A_{j\cdot}-A^\star_{j\cdot}\rVert_1 \le\frac{C}{\pi^\star_{\min}}\Delta_{\mathrm{sep}}^{-(2s_{\max}-1)}h(p_{\theta,T},p_{\theta^\star,T}). j max ∥ A j ⋅ − A j ⋅ ⋆ ∥ 1 ≤ π m i n ⋆ C Δ sep − ( 2 s m a x − 1 ) h ( p θ , T , p θ ⋆ , T ) .
Without cluster assumptions, the same calculation uses the pair exponent 2 K − 1 2K-1 2 K − 1 , giving the stated unstructured row bound. The row calculation divides only by a true initial probability. Fitted initial-probability positivity was needed for the maximum center bound, but is not needed for this row-division inequality.
For the MLE, assume the same sufficient condition ϵ n , T ≤ c ( π min ⋆ ) Δ s e p 2 s max − 1 \epsilon_{n,T}\le c(\pi^\star_{\min})\Delta_{\mathrm{sep}}^{2s_{\max}-1} ϵ n , T ≤ c ( π m i n ⋆ ) Δ sep 2 s m a x − 1 used in Proposition 3.7 . On E n \mathcal{E}_n E n , localization supplies the aligned labeling and the fitted initial-probability lower bounds. Applying the two deterministic inequalities to θ ^ n \widehat{\theta}_n θ n and using h ( p θ ^ n , T , p θ ⋆ , T ) ≤ ϵ n , T h(p_{\widehat{\theta}_n,T},p_{\theta^\star,T})\le\epsilon_{n,T} h ( p θ n , T , p θ ⋆ , T ) ≤ ϵ n , T proves their statistical versions on this same event.
Corollary D.10 (Identifiability from two observations). Under the standing assumptions, two exact-fit Gaussian HMMs with p θ , 2 = p θ ⋆ , 2 p_{\theta,2}=p_{\theta^\star,2} p θ , 2 = p θ ⋆ , 2 have the same emission centers, initial distribution, and transition matrix, up to one common permutation.
Proof. Apply Corollary 3.5 with T = 2 T=2 T = 2 and h ( p θ , 2 , p θ ⋆ , 2 ) = 0 h(p_{\theta,2},p_{\theta^\star,2})=0 h ( p θ , 2 , p θ ⋆ , 2 ) = 0 to obtain one common matching permutation and positive fitted initial probabilities. After relabeling, Theorem 3.6 yields D μ = D { 1 } = D { 1 , 2 } = 0 D_\mu=D_{\{1\}}=D_{\{1,2\}}=0 D μ = D { 1 } = D { 1 , 2 } = 0 . Since
0 ≤ ∑ j = 1 K π j ∥ b j − a j ∥ 2 ≤ D μ = 0 and π j > 0 , 0\le\sum_{j=1}^{K}\pi_j\lVert b_j-a_j\rVert_2\le D_\mu=0\quad\text{and}\quad\pi_j>0, 0 ≤ j = 1 ∑ K π j ∥ b j − a j ∥ 2 ≤ D μ = 0 and π j > 0 ,
every fitted center equals its matched true center. Also D { 1 } = ∑ j ∣ π j − π j ⋆ ∣ = 0 D_{\{1\}}=\sum_j|\pi_j-\pi^\star_j|=0 D { 1 } = ∑ j ∣ π j − π j ⋆ ∣ = 0 , so π = π ⋆ \pi=\pi^\star π = π ⋆ . Finally, D { 1 , 2 } = ∑ j , ℓ ∣ π j A j ℓ − π j ⋆ A j ℓ ⋆ ∣ = 0 D_{\{1,2\}}=\sum_{j,\ell}|\pi_jA_{j\ell}-\pi^\star_jA^\star_{j\ell}|=0 D { 1 , 2 } = ∑ j , ℓ ∣ π j A j ℓ − π j ⋆ A j ℓ ⋆ ∣ = 0 ; division by π j = π j ⋆ > 0 \pi_j=\pi^\star_j>0 π j = π j ⋆ > 0 gives A = A ⋆ A=A^\star A = A ⋆ . All equalities hold under the single permutation fixed at the start.
Proofs under over-specification
The displacement and aggregate probability bounds follow from Lemma D.1, established within the proof of Theorem 3.1 in Appendix D.1. The lemma applies without assuming exact specification. We then establish the one-time transport comparison, prove the path transport bounds, give lower examples for the separation exponents in both regimes, and treat the single true state case.
Proof of Theorem 4.2
Proof of Theorem 4.2. Lemma D.1 gives the asserted bounds for V V V , B B B , and D I D_I D I for every k ≥ K k \ge K k ≥ K . Taking I = { 1 } I = \{1\} I = { 1 } gives the bound for D π D_\pi D π , and taking the maximum over consecutive pairs I = { t , t + 1 } I = \{t,t+1\} I = { t , t + 1 } gives the bound for D A D_A D A . The structured separation exponents are 2 s max 2s_{\max} 2 s m a x and β 2 = max { 2 s max , 3 s max − 2 } \beta_2 = \max\{2s_{\max},3s_{\max}-2\} β 2 = max { 2 s m a x , 3 s m a x − 2 } , respectively. The same lemma gives the unstructured probability exponents 2 K 2K 2 K and γ K , 2 \gamma_{K,2} γ K , 2 , without a lower bound on fitted state probabilities or a factor ( π min ⋆ ) − 1 (\pi_{\min}^{\star})^{-1} ( π m i n ⋆ ) − 1 . □
Proof of Theorem 4.4
Proof of Theorem 4.4. Fix a nonempty set I ⊆ [ T ] I \subseteq[T] I ⊆ [ T ] with ∣ I ∣ = r |I| = r ∣ I ∣ = r . The bounds of Theorem 4.2 control the displacement of fitted centers and the discrepancy between aggregate label probabilities. To convert these bounds into a transport guarantee, we also account for the distances over which the probability discrepancies are redistributed. Lemma A.4 provides this comparison, while Lemma D.2 controls the mass that must be transported between cluster sequences.
Transport under the cluster assumptions. Recall that V = max t ≤ T ∑ i = 1 k q t ( i ) r i 2 V = \max_{t \le T} \sum_{i=1}^{k} q_t(i)r_i^2 V = max t ≤ T ∑ i = 1 k q t ( i ) r i 2 and V 2 , I = r − 1 ∑ t ∈ I ∑ i = 1 k q t ( i ) r i 2 V_{2,I} = r^{-1}\sum_{t \in I}\sum_{i=1}^{k}q_t(i)r_i^2 V 2 , I = r − 1 ∑ t ∈ I ∑ i = 1 k q t ( i ) r i 2 , where r = ∣ I ∣ r = |I| r = ∣ I ∣ . Since V V V is the maximum over all observed times and V 2 , I V_{2,I} V 2 , I is the average over the times in I I I , we have V 2 , I ≤ V V_{2,I} \le V V 2 , I ≤ V . Therefore:
V 2 , I = 1 r ∑ t ∈ I ∑ i = 1 k q t ( i ) r i 2 ≤ 1 r ∑ t ∈ I V = V . V_{2,I} = \frac{1}{r}\sum_{t\in I}\sum_{i=1}^{k}q_t(i)r_i^2 \le\frac{1}{r}\sum_{t\in I}V = V. V 2 , I = r 1 t ∈ I ∑ i = 1 ∑ k q t ( i ) r i 2 ≤ r 1 t ∈ I ∑ V = V .
Applying Lemma A.4 with p = 2 p = 2 p = 2 , we obtain
W 2 , I 2 ( G θ I , G ⋆ I ) ≤ 2 V 2 , I + C 0 2 Δ s e p 2 D I + 4 R 2 J I . W_{2,I}^2(G_\theta^I,G_\star^I) \le2V_{2,I} + C_0^2\Delta_{\mathrm{sep}}^2D_I + 4R^2J_I. W 2 , I 2 ( G θ I , G ⋆ I ) ≤ 2 V 2 , I + C 0 2 Δ sep 2 D I + 4 R 2 J I .
The factor Δ s e p 2 \Delta_{\mathrm{sep}}^2 Δ sep 2 in the second term comes from the squared distance between locations in the same cluster at each retained time. The last term allows transport between different cluster sequences, where distances are bounded by 2 R 2R 2 R .
We bound these contributions separately before combining them. Theorem 4.2 gives
V 2 , I ≤ V ≤ C Δ s e p − ( 2 s max − 2 ) h ( p θ , T , p θ ⋆ , T ) V_{2,I} \le V \le C\Delta_{\mathrm{sep}}^{-(2s_{\max}-2)}h(p_{\theta,T},p_{\theta^\star,T}) V 2 , I ≤ V ≤ C Δ sep − ( 2 s m a x − 2 ) h ( p θ , T , p θ ⋆ , T )
and
Δ s e p 2 D I ≤ C r Δ s e p − ( β r − 2 ) h ( p θ , T , p θ ⋆ , T ) . \Delta_{\mathrm{sep}}^2D_I \le C_r\Delta_{\mathrm{sep}}^{-(\beta_r-2)}h(p_{\theta,T},p_{\theta^\star,T}). Δ sep 2 D I ≤ C r Δ sep − ( β r − 2 ) h ( p θ , T , p θ ⋆ , T ) .
The squared within-cluster distance reduces the probability exponent by two. Lemma D.2 also gives
J I ≤ C r Δ s e p − ( 2 s max − 2 ) h ( p θ , T , p θ ⋆ , T ) . J_I \le C_r\Delta_{\mathrm{sep}}^{-(2s_{\max}-2)}h(p_{\theta,T},p_{\theta^\star,T}). J I ≤ C r Δ sep − ( 2 s m a x − 2 ) h ( p θ , T , p θ ⋆ , T ) .
Since β r ≥ 2 s max \beta_r \ge2s_{\max} β r ≥ 2 s m a x and 0 < Δ s e p ≤ 1 0 < \Delta_{\mathrm{sep}} \le1 0 < Δ sep ≤ 1 , both the displacement and between-cluster contributions are bounded by the multiplier Δ s e p − ( β r − 2 ) \Delta_{\mathrm{sep}}^{-(\beta_r-2)} Δ sep − ( β r − 2 ) . Adding the three contributions proves
W 2 , I 2 ( G θ I , G ⋆ I ) ≤ C r Δ s e p − ( β r − 2 ) h ( p θ , T , p θ ⋆ , T ) . (142) W_{2,I}^{2}(G_{\theta}^{I},G_{\star}^{I}) \le C_r\Delta_{\mathrm{sep}}^{-(\beta_r-2)}h(p_{\theta,T},p_{\theta^\star,T}). \tag*{(142)} W 2 , I 2 ( G θ I , G ⋆ I ) ≤ C r Δ sep − ( β r − 2 ) h ( p θ , T , p θ ⋆ , T ) . (142)
For r = 1 r=1 r = 1 , β 1 = 2 s max \beta_1=2s_{\max} β 1 = 2 s m a x , so the one-time squared Wasserstein exponent remains 2 s max − 2 2s_{\max}-2 2 s m a x − 2 . For s max ≥ 2 s_{\max}\ge2 s m a x ≥ 2 and r ≥ 2 r\ge2 r ≥ 2 , the exponent is r ( s max − 1 ) + s max − 2 r(s_{\max}-1)+s_{\max}-2 r ( s m a x − 1 ) + s m a x − 2 .
Transport without cluster assumptions. For any nonempty I I I , first move fitted locations to their assigned true centers, then couple the resulting distribution with G ⋆ I G_\star^I G ⋆ I on the true grid. The first step has squared cost V 2 , I V_{2,I} V 2 , I . For the second step, the two grid distributions have ℓ 1 \ell_1 ℓ 1 discrepancy D I D_I D I , so a coupling can leave their common mass fixed and move the remaining mass, which is D I / 2 D_I/2 D I /2 . The grid diameter under d 2 , r d_{2,r} d 2 , r is D ⋆ = max i , j ∥ a i − a j ∥ 2 D_\star=\max_{i,j}\lVert a_i-a_j\rVert_2 D ⋆ = max i , j ∥ a i − a j ∥ 2 . The squared triangle inequality therefore gives
W 2 , I 2 ( G θ I , G ⋆ I ) ≤ 2 V 2 , I + D ⋆ 2 D I . W_{2,I}^{2}(G_{\theta}^{I},G_{\star}^{I}) \le2V_{2,I}+D_\star^2D_I. W 2 , I 2 ( G θ I , G ⋆ I ) ≤ 2 V 2 , I + D ⋆ 2 D I .
Since V 2 , I ≤ V V_{2,I}\le V V 2 , I ≤ V , Lemma D.1 and (D.1) yield
W 2 , I 2 ( G θ I , G ⋆ I ) ≤ C r { Δ s e p − ( 2 K − 2 ) + Δ s e p − ( γ K , r − 2 ) } h ( p θ , T , p θ ⋆ , T ) . W_{2,I}^{2}(G_{\theta}^{I},G_{\star}^{I}) \le C_r\left\{\Delta_{\mathrm{sep}}^{-(2K-2)}+\Delta_{\mathrm{sep}}^{-(\gamma_{K,r}-2)}\right\}h(p_{\theta,T},p_{\theta^\star,T}). W 2 , I 2 ( G θ I , G ⋆ I ) ≤ C r { Δ sep − ( 2 K − 2 ) + Δ sep − ( γ K , r − 2 ) } h ( p θ , T , p θ ⋆ , T ) .
Here the factor D ⋆ 2 D_\star^2 D ⋆ 2 cancels the inverse diameter factor in the probability bound. Using γ K , r ≥ 2 K \gamma_{K,r}\ge2K γ K , r ≥ 2 K and 0 < Δ s e p ≤ 1 0<\Delta_{\mathrm{sep}}\le1 0 < Δ sep ≤ 1 , we obtain
W 2 , I 2 ( G θ I , G ⋆ I ) ≤ C r Δ s e p − ( γ K , r − 2 ) h ( p θ , T , p θ ⋆ , T ) . (143) W_{2,I}^{2}(G_{\theta}^{I},G_{\star}^{I}) \le C_r\Delta_{\mathrm{sep}}^{-(\gamma_{K,r}-2)}h(p_{\theta,T},p_{\theta^\star,T}). \tag*{(143)} W 2 , I 2 ( G θ I , G ⋆ I ) ≤ C r Δ sep − ( γ K , r − 2 ) h ( p θ , T , p θ ⋆ , T ) . (143)
For I = { t } I=\{t\} I = { t } , we have γ K , 1 = 2 K \gamma_{K,1}=2K γ K , 1 = 2 K . The constant is uniform in t t t , so taking the maximum gives
max t ≤ T W 2 2 ( G θ { t } , G ⋆ { t } ) ≤ C Δ s e p − ( 2 K − 2 ) h ( p θ , T , p θ ⋆ , T ) . (144) \max_{t\le T}W_{2}^{2}(G_{\theta}^{\{t\}},G_{\star}^{\{t\}}) \le C\Delta_{\mathrm{sep}}^{-(2K-2)}h(p_{\theta,T},p_{\theta^\star,T}). \tag*{(144)} t ≤ T max W 2 2 ( G θ { t } , G ⋆ { t } ) ≤ C Δ sep − ( 2 K − 2 ) h ( p θ , T , p θ ⋆ , T ) . (144)
Transport bounds for the MLE. The preceding inequalities hold for every fitted parameter satisfying the theorem assumptions. We may therefore apply them to θ ^ n \widehat{\theta}_n θ n on the density event
E n = { h ( p θ ^ n , T , p θ ⋆ , T ) ≤ ε n , T } , P θ ⋆ ( E n ) ≥ 1 − 1 n , \mathcal{E}_n=\left\{h(p_{\widehat{\theta}_n,T},p_{\theta^\star,T})\le\varepsilon_{n,T}\right\},\qquad P_{\theta^\star}(\mathcal{E}_n)\ge1-\frac{1}{n}, E n = { h ( p θ n , T , p θ ⋆ , T ) ≤ ε n , T } , P θ ⋆ ( E n ) ≥ 1 − n 1 ,
where the probability bound follows from Proposition 2.2.
For structured configurations, taking square roots in (E.1) gives, on E n \mathcal{E}_n E n ,
W 2 , I ( G θ ^ n I , G ⋆ I ) ≤ C r Δ s e p − ( β r − 2 ) / 2 h ( p θ ^ n , T , p θ ⋆ , T ) 1 / 2 ≤ C r Δ s e p − ( β r − 2 ) / 2 ε n , T 1 / 2 (by the definition of E n ). \begin{aligned}
W_{2,I}(G_{\widehat{\theta}_n}^{I},G_{\star}^{I})\le C_r\Delta_{\mathrm{sep}}^{-(\beta_r-2)/2}h(p_{\widehat{\theta}_n,T},p_{\theta^\star,T})^{1/2} \\
&\le C_r\Delta_{\mathrm{sep}}^{-(\beta_r-2)/2}\varepsilon_{n,T}^{1/2}\qquad\text{(by the definition of }\mathcal{E}_n\text{).}
\end{aligned} W 2 , I ( G θ n I , G ⋆ I ) ≤ C r Δ sep − ( β r − 2 ) /2 h ( p θ n , T , p θ ⋆ , T ) 1/2 ≤ C r Δ sep − ( β r − 2 ) /2 ε n , T 1/2 (by the definition of E n ).
In particular, for I = { t } I=\{t\} I = { t } ,
W 2 ( G θ ^ n { t } , G ⋆ { t } ) ≤ C Δ s e p − ( s max − 1 ) ε n , T 1 / 2 , t ≤ T . W_2(G_{\widehat{\theta}_n}^{\{t\}},G_{\star}^{\{t\}})\le C\Delta_{\mathrm{sep}}^{-(s_{\max}-1)}\varepsilon_{n,T}^{1/2},\qquad t\le T. W 2 ( G θ n { t } , G ⋆ { t } ) ≤ C Δ sep − ( s m a x − 1 ) ε n , T 1/2 , t ≤ T .
The constant is uniform in t t t , and each inequality holds on the same event E n \mathcal{E}_n E n . Taking the maximum therefore yields
max t ≤ T W 2 ( G θ ^ n { t } , G ⋆ { t } ) ≤ C Δ s e p − ( s max − 1 ) ε n , T 1 / 2 . \max_{t\le T}W_2(G_{\widehat{\theta}_n}^{\{t\}},G_{\star}^{\{t\}})\le C\Delta_{\mathrm{sep}}^{-(s_{\max}-1)}\varepsilon_{n,T}^{1/2}. t ≤ T max W 2 ( G θ n { t } , G ⋆ { t } ) ≤ C Δ sep − ( s m a x − 1 ) ε n , T 1/2 .
Without cluster assumptions, Equation (E.3) gives
max t ≤ T W 2 ( G θ ^ n { t } , G ⋆ { t } ) ≤ C Δ s e p − ( K − 1 ) ε n , T 1 / 2 . \max_{t\leq T} W_2\left(G_{\hat{\theta}_n}^{\{t\}},G_{\star}^{\{t\}}\right) \leq C\Delta_{\mathrm{sep}}^{-(K-1)}\varepsilon_{n,T}^{1/2}. t ≤ T max W 2 ( G θ ^ n { t } , G ⋆ { t } ) ≤ C Δ sep − ( K − 1 ) ε n , T 1/2 .
For r ≥ 2 r\geq2 r ≥ 2 , taking square roots in (E.2) gives
W 2 , I ( G θ ^ n I , G ⋆ I ) ≤ C r Δ s e p − ( γ K , r − 2 ) / 2 ε n , T 1 / 2 W_{2,I}\left(G_{\hat{\theta}_n}^{I},G_{\star}^{I}\right) \leq C r\Delta_{\mathrm{sep}}^{-(\gamma_{K,r}-2)/2}\varepsilon_{n,T}^{1/2} W 2 , I ( G θ ^ n I , G ⋆ I ) ≤ C r Δ sep − ( γ K , r − 2 ) /2 ε n , T 1/2
for the joint latent location distributions. Since the deterministic inequalities apply to every nonempty I ⊆ [ T ] I\subseteq[T] I ⊆ [ T ] , all these MLE bounds hold simultaneously on E n \mathcal{E}_n E n .
By Proposition 2.2 ,
ε n , T 1 / 2 = C ( d log ( n T d ) n ) 1 / 4 . \varepsilon_{n,T}^{1/2}=\sqrt{C}\left(\frac{d\log(nTd)}{n}\right)^{1/4}. ε n , T 1/2 = C ( n d log ( n T d ) ) 1/4 .
Substituting this expression into the preceding inequalities gives the stated Wasserstein rates. For fixed model dimensions, emission separation, and cluster parameters, these rates are of order ( log n / n ) 1 / 4 (\log n/n)^{1/4} ( log n / n ) 1/4 .
Sharpness of the structured overfitted exponents
The upper bounds under over-specification allow repeated fitted centers and zero fitted initial probabilities. The following lower examples use these possibilities. They establish sharpness over this full class; the required fitted state count for joint losses can exceed K + 1 K+1 K + 1 .
Proposition E.1 (Sharpness under over-specification). Fix s ≥ 2 s\geq2 s ≥ 2 and d ≥ 1 d\geq1 d ≥ 1 . There are true K = s K=s K = s state models with a single collinear cluster, Δ s e p = δ ↓ 0 \Delta_{\mathrm{sep}}=\delta\downarrow0 Δ sep = δ ↓ 0 , and true initial probabilities bounded away from zero, with the following properties.
For one-time recovery, a fitted model with k = s + 1 k=s+1 k = s + 1 states can satisfy, for a positive sequence η δ ↓ 0 \eta_\delta\downarrow0 η δ ↓ 0 and every fixed T ≥ 2 T\geq2 T ≥ 2 ,
h ( p θ δ , T , p θ ⋆ , T ) ≍ η δ δ 2 s , (145) h(p_{\theta_\delta,T},p_{\theta_\star,T})\asymp\eta_\delta\delta^{2s}, \tag*{(145)} h ( p θ δ , T , p θ ⋆ , T ) ≍ η δ δ 2 s , (145) D π ≲ max t ≤ T D { t } ≲ η δ , V ≲ max t ≤ T W 2 2 ( G θ δ { t } , G ⋆ { t } ) ≲ η δ δ 2 , (146) D_\pi\lesssim\max_{t\leq T}D_{\{t\}}\lesssim\eta_\delta,\qquad V\lesssim\max_{t\leq T}W_2^2\left(G_{\theta_\delta}^{\{t\}},G_\star^{\{t\}}\right)\lesssim\eta_\delta\delta^2, \tag*{(146)} D π ≲ t ≤ T max D { t } ≲ η δ , V ≲ t ≤ T max W 2 2 ( G θ δ { t } , G ⋆ { t } ) ≲ η δ δ 2 , (146) B ≍ η δ δ , D μ ≍ η δ δ . (147) B\asymp\eta_\delta\delta,\qquad D_\mu\asymp\sqrt{\eta_\delta}\,\delta. \tag*{(147)} B ≍ η δ δ , D μ ≍ η δ δ . (147) For every fixed r ≥ 2 r\geq2 r ≥ 2 and T ≥ r T\geq r T ≥ r , a fitted model with a fixed number of states depending only on s , r s,r s , r can satisfy, for I = { 1 , … , r } I=\{1,\ldots,r\} I = { 1 , … , r } ,
h ( p θ δ , T , p θ ⋆ , T ) ≍ η δ δ r ( s − 1 ) + s , (148) h(p_{\theta_\delta,T},p_{\theta_\star,T})\asymp\eta_\delta\delta^{r(s-1)+s}, \tag*{(148)} h ( p θ δ , T , p θ ⋆ , T ) ≍ η δ δ r ( s − 1 ) + s , (148) D I ≍ η δ , W 2 , I 2 ( G θ δ I , G ⋆ I ) ≍ η δ δ 2 . (149) D_I\asymp\eta_\delta,\qquad W_{2,I}^2\left(G_{\theta_\delta}^{I},G_\star^{I}\right)\asymp\eta_\delta\delta^2. \tag*{(149)} D I ≍ η δ , W 2 , I 2 ( G θ δ I , G ⋆ I ) ≍ η δ δ 2 . (149) Consequently, the structured exponents β r = max { 2 s , r ( s − 1 ) + s } \beta_r=\max\{2s,r(s-1)+s\} β r = max { 2 s , r ( s − 1 ) + s } for D I D_I D I and β r − 2 \beta_r-2 β r − 2 for W 2 , I 2 W_{2,I}^2 W 2 , I 2 cannot be reduced uniformly over the allowed over-specified models. The one-time displacement exponents are also sharp. Taking s = K s=K s = K proves the unstructured exponents γ K , r \gamma_{K,r} γ K , r and γ K , r − 2 \gamma_{K,r}-2 γ K , r − 2 as well. These conclusions concern deterministic separation powers with the displayed Hellinger powers fixed, not minimax rates in n n n or sharpness for each prescribed fitted state count.
Proof. Work first in dimension one with fixed nodes z 1 < ⋯ < z s z_1<\cdots<z_s z 1 < ⋯ < z s of minimum gap one, and put g ( z ) = ∏ j = 1 s ( z − z j ) g(z)=\prod_{j=1}^{s}(z-z_j) g ( z ) = ∏ j = 1 s ( z − z j ) . All locations below are multiplied by δ \delta δ . We use perturbations of size η = η δ \eta=\eta_\delta η = η δ , chosen sufficiently small relative to δ \delta δ .
A uniform Gaussian comparison for the perturbations. We first give the density comparison with both small parameters explicit. In each construction below, the unscaled locations remain in a fixed compact set for 0 ≤ η ≤ η 0 0 \le\eta\le\eta_0 0 ≤ η ≤ η 0 , and all coefficients are fixed independently of η , δ \eta,\delta η , δ . Let q η , δ q_{\eta,\delta} q η , δ be the resulting r r r -coordinate observation density, and write Λ \Lambda Λ for the first variation of its unscaled mixing distribution at η = 0 \eta=0 η = 0 . In the one-time construction take r = 1 r=1 r = 1 in this paragraph. Suppose Λ \Lambda Λ annihilates every polynomial of total degree less than m m m and is nonzero on a monomial z α z^\alpha z α of degree m m m .
For fixed x x x , apply Λ \Lambda Λ to z ↦ φ r ( x − δ z ) z \mapsto\varphi_r(x-\delta z) z ↦ φ r ( x − δ z ) . Taylor expansion in δ \delta δ through degree m − 1 m-1 m − 1 has polynomial coefficients in z z z of the same total degrees. All of them vanish under Λ \Lambda Λ . Gaussian derivative formulas and the integral remainder consequently give, for a fixed integer L L L ,
∣ q ˙ 0 , δ ( x ) ∣ ≤ C δ m ( 1 + ∥ x ∥ 2 L ) φ r ( x ) e C δ ∥ x ∥ 2 2 , 0 < δ ≤ δ 0 . |\dot q_{0,\delta}(x)| \le C\delta^m(1+\lVert x\rVert_2^L)\varphi_r(x)e^{C\delta\lVert x\rVert_2^2}, \qquad0<\delta\le\delta_0. ∣ q ˙ 0 , δ ( x ) ∣ ≤ C δ m ( 1 + ∥ x ∥ 2 L ) φ r ( x ) e C δ ∥ x ∥ 2 2 , 0 < δ ≤ δ 0 .
This bound also holds when Λ \Lambda Λ includes derivatives at grid points: the finite point-evaluation and derivative functionals commute with Taylor expansion, and their orders and coefficients are fixed. Ordinary shifts and weight perturbations have second-order remainders bounded uniformly in η , δ \eta,\delta η , δ . For the symmetric split, the same fact follows without differentiating η \sqrt{\eta} η at zero. For any smooth scalar f f f , fixed z , α z,\alpha z , α , and small η ≥ 0 \eta\ge0 η ≥ 0 , the central Taylor formula gives
f ( z + α η + η ) + f ( z + α η − η ) 2 = f ( z ) + η { α f ′ ( z ) + 1 2 f ′ ′ ( z ) } + O ( η 2 max 1 ≤ j ≤ 4 ∥ f ( j ) ∥ ∞ ) . \frac{f(z+\alpha\eta+\sqrt{\eta})+f(z+\alpha\eta-\sqrt{\eta})}{2}
=f(z)+\eta\left\{\alpha f'(z)+\frac{1}{2}f''(z)\right\}+O\left(\eta^2\max_{1\le j\le4}\lVert f^{(j)}\rVert_\infty\right). 2 f ( z + α η + η ) + f ( z + α η − η ) = f ( z ) + η { α f ′ ( z ) + 2 1 f ′′ ( z ) } + O ( η 2 1 ≤ j ≤ 4 max ∥ f ( j ) ∥ ∞ ) .
where the norms are over one fixed compact interval containing all the evaluation points. This follows by first expanding the central difference to order two with a fourth-derivative remainder, and then expanding about z z z . Applying it to the Gaussian emission and multiplying by the perturbed weight shows that, after increasing L L L if necessary,
∣ q η , δ ( x ) − q 0 , δ ( x ) − η q ˙ 0 , δ ( x ) ∣ ≤ C η 2 ( 1 + ∥ x ∥ 2 L ) φ r ( x ) e C δ ∥ x ∥ 2 2 . |q_{\eta,\delta}(x)-q_{0,\delta}(x)-\eta\dot q_{0,\delta}(x)|\le C\eta^2(1+\lVert x\rVert_2^L)\varphi_r(x)e^{C\delta\lVert x\rVert_2^2}. ∣ q η , δ ( x ) − q 0 , δ ( x ) − η q ˙ 0 , δ ( x ) ∣ ≤ C η 2 ( 1 + ∥ x ∥ 2 L ) φ r ( x ) e C δ ∥ x ∥ 2 2 .
All these constants are uniform for 0 ≤ η ≤ η 0 0\le\eta\le\eta_0 0 ≤ η ≤ η 0 and 0 < δ ≤ δ 0 0<\delta\le\delta_0 0 < δ ≤ δ 0 . The true grid density satisfies
q 0 , δ ( x ) ≥ c φ r ( x ) e − C δ ∥ x ∥ 2 − C δ 2 . q_{0,\delta}(x)\ge c\varphi_r(x)e^{-C\delta\lVert x\rVert_2-C\delta^2}. q 0 , δ ( x ) ≥ c φ r ( x ) e − C δ ∥ x ∥ 2 − C δ 2 .
Therefore the inequality ( q − p ) 2 ≤ ( q − p ) 2 / p (\sqrt q-\sqrt p)^2\le(q-p)^2/p ( q − p ) 2 ≤ ( q − p ) 2 / p and Gaussian integrability imply
h 2 ( q η , δ , q 0 , δ ) ≤ 1 2 ∫ ( q η , δ − q 0 , δ ) 2 q 0 , δ d x ≤ C ( η δ m + η 2 ) 2 , h^2(q_{\eta,\delta},q_{0,\delta})\le\frac{1}{2}\int\frac{(q_{\eta,\delta}-q_{0,\delta})^2}{q_{0,\delta}}\,dx\le C(\eta\delta^m+\eta^2)^2, h 2 ( q η , δ , q 0 , δ ) ≤ 2 1 ∫ q 0 , δ ( q η , δ − q 0 , δ ) 2 d x ≤ C ( η δ m + η 2 ) 2 ,
h ( q η , δ , q 0 , δ ) ≤ C ( η δ m + η 2 ) . (150) h(q_{\eta,\delta},q_{0,\delta})\le C(\eta\delta^m+\eta^2). \tag*{(150)} h ( q η , δ , q 0 , δ ) ≤ C ( η δ m + η 2 ) . (150)
For the chosen monomial, scaling the mixing locations gives the exact leading moment difference
∫ x α d ( H η , δ − H 0 , δ ) = δ m { η Λ ( z α ) + O ( η 2 ) } . \int x^\alpha\,d(H_{\eta,\delta}-H_{0,\delta})=\delta^m\{\eta\Lambda(z^\alpha)+O(\eta^2)\}. ∫ x α d ( H η , δ − H 0 , δ ) = δ m { η Λ ( z α ) + O ( η 2 )} .
Its coefficient is nonzero. More explicitly, write λ = Λ ( z α ) ≠ 0 \lambda=\Lambda(z^\alpha)\ne0 λ = Λ ( z α ) = 0 . The remainder in this moment expansion is bounded by C η 2 δ m C\eta^2\delta^m C η 2 δ m with C C C independent of η , δ \eta,\delta η , δ : the unscaled nodes and perturbation coefficients lie in a fixed compact set. Decrease η 0 \eta_0 η 0 so that C η 0 ≤ ∣ λ ∣ / 2 C\eta_0\le|\lambda|/2 C η 0 ≤ ∣ λ ∣/2 . Then the moment difference has absolute value at least ∣ λ ∣ η δ m / 2 |\lambda|\eta\delta^m/2 ∣ λ ∣ η δ m /2 throughout 0 < η ≤ η 0 0<\eta\le\eta_0 0 < η ≤ η 0 . Lemma A.3, applied to this monomial on a fixed ball rather than a shrinking support, gives a uniform lower bound h ( q η , δ , q 0 , δ ) ≥ c η δ m h(q_{\eta,\delta},q_{0,\delta})\ge c\eta\delta^m h ( q η , δ , q 0 , δ ) ≥ cη δ m . Combined with equation (E.9), this proves
c η δ m ≤ h ( q η , δ , q 0 , δ ) ≤ C η δ m , 0 < δ ≤ δ 0 , 0 < η ≤ min { η 0 , δ m } . (151) c\eta\delta^m\le h(q_{\eta,\delta},q_{0,\delta})\le C\eta\delta^m,\qquad0<\delta\le\delta_0,\quad0<\eta\le\min\{\eta_0,\delta^m\}. \tag*{(151)} cη δ m ≤ h ( q η , δ , q 0 , δ ) ≤ C η δ m , 0 < δ ≤ δ 0 , 0 < η ≤ min { η 0 , δ m } . (151)
The constants are uniform over this two-parameter region. Choose δ 0 ≤ 1 \delta_0 \le1 δ 0 ≤ 1 and η δ = δ m + 1 \eta_\delta= \delta^{m+1} η δ = δ m + 1 , decreasing δ 0 \delta_0 δ 0 further to ensure η δ ≤ η 0 \eta_\delta\le\eta_0 η δ ≤ η 0 . Then
η δ 2 η δ δ m = δ ⟶ 0 , \frac{\eta_\delta^2}{\eta_\delta\delta^m}=\delta\longrightarrow0, η δ δ m η δ 2 = δ ⟶ 0 ,
so the upper and lower bounds have the common order η δ δ m \eta_\delta\delta^m η δ δ m . This is a joint choice of the two parameters, not an exchange of iterated limits. In dimension d > 1 d > 1 d > 1 , putting the centers on the first coordinate axis appends the same independent Gaussian factor to both models and leaves their Hellinger distance unchanged.
One-time probabilities and displacement. Choose fixed positive weights w j w_j w j summing to one. There are unique coefficients u j , c j u_j,c_j u j , c j such that
Λ ( f ) = ∑ j = 1 s { u j f ( z j ) + c j f ′ ( z j ) } + w 1 2 f ′ ′ ( z 1 ) \Lambda(f)=\sum_{j=1}^{s}\{u_jf(z_j)+c_jf'(z_j)\}+\frac{w_1}{2}f''(z_1) Λ ( f ) = j = 1 ∑ s { u j f ( z j ) + c j f ′ ( z j )} + 2 w 1 f ′′ ( z 1 )
vanishes for every polynomial of degree at most 2 s − 1 2s-1 2 s − 1 . Existence and uniqueness follow from the invertibility of the confluent Vandermonde matrix, equivalently from the fact that a polynomial of degree at most 2 s − 1 2s-1 2 s − 1 with s s s double roots is zero. In particular, ∑ j u j = 0 \sum_j u_j=0 ∑ j u j = 0 .
Neither vector u u u nor c c c is zero. If u = 0 u=0 u = 0 , take an antiderivative f f f of g g g , whose degree is s + 1 ≤ 2 s − 1 s+1 \le2s-1 s + 1 ≤ 2 s − 1 . Then f ′ ( z j ) = 0 f'(z_j)=0 f ′ ( z j ) = 0 for all j j j but f ′ ′ ( z 1 ) = g ′ ( z 1 ) ≠ 0 f''(z_1)=g'(z_1)\ne0 f ′′ ( z 1 ) = g ′ ( z 1 ) = 0 , a contradiction. If c = 0 c=0 c = 0 , use f ( z ) = ( z − z 1 ) g ( z ) f(z)=(z-z_1)g(z) f ( z ) = ( z − z 1 ) g ( z ) , which vanishes on the grid but has f ′ ′ ( z 1 ) = 2 g ′ ( z 1 ) ≠ 0 f''(z_1)=2g'(z_1)\ne0 f ′′ ( z 1 ) = 2 g ′ ( z 1 ) = 0 . Also
Λ ( g 2 ) = w 1 g ′ ( z 1 ) 2 > 0. \Lambda(g^2)=w_1g'(z_1)^2>0. Λ ( g 2 ) = w 1 g ′ ( z 1 ) 2 > 0.
Since g 2 g^2 g 2 is monic of degree 2 s 2s 2 s , the first nonzero moment order of Λ \Lambda Λ is m = 2 s m=2s m = 2 s .
Give group j j j total weight w j + η u j w_j+\eta u_j w j + η u j and center z j + η c j / w j z_j+\eta c_j/w_j z j + η c j / w j . For j = 1 j=1 j = 1 , split this group into two equal weights at
z 1 + η c 1 / w 1 − η , z 1 + η c 1 / w 1 + η . z_1+\eta c_1/w_1-\sqrt{\eta},\qquad z_1+\eta c_1/w_1+\sqrt{\eta}. z 1 + η c 1 / w 1 − η , z 1 + η c 1 / w 1 + η .
The other groups each contain one atom. Choose η 0 > 0 \eta_0>0 η 0 > 0 so that w j + η u j ≥ w j / 2 w_j+\eta u_j\ge w_j/2 w j + η u j ≥ w j /2 for every j j j and η max j ∣ c j ∣ / w j + η < 1 / 4 \eta\max_j|c_j|/w_j+\sqrt{\eta}<1/4 η max j ∣ c j ∣/ w j + η < 1/4 for 0 < η ≤ η 0 0<\eta\le\eta_0 0 < η ≤ η 0 . The weights then sum to one, and every fitted atom is uniquely assigned to its indicated true center. The choice of η 0 \eta_0 η 0 depends only on the fixed nodes, weights, and coefficients, not on δ \delta δ . Expanding a smooth test function verifies that the first variation is exactly Λ \Lambda Λ . Thus (E.9), with m = 2 s m=2s m = 2 s and η δ = δ 2 s + 1 \eta_\delta=\delta^{2s+1} η δ = δ 2 s + 1 , gives the required density order for one observation. Take the initial distributions and every transition row to be the corresponding mixture weights. Both hidden processes are then independent over time, and the product-affinity identity preserves this Hellinger order at every fixed T T T .
The aggregate mass difference is exactly η u \eta u η u , so its ℓ 1 \ell_1 ℓ 1 norm is of order η \eta η . The split contributes order η δ 2 \eta\delta^2 η δ 2 to V V V and order η δ \sqrt{\eta}\,\delta η δ to D μ D_\mu D μ . The within-group signed displacements equal δ η c j + O ( δ η 2 ) \delta\eta c_j+O(\delta\eta^2) δ η c j + O ( δ η 2 ) , giving B ≍ η δ B\asymp\eta\delta B ≍ η δ because c ≠ 0 c\ne0 c = 0 . Every coupling to the true support has squared cost at least V V V . Conversely, move fitted atoms to their assigned centers and redistribute the O ( η ) O(\eta) O ( η ) mass discrepancy over distances O ( δ ) O(\delta) O ( δ ) . This gives W 2 2 ≍ η δ 2 W_2^2\asymp\eta\delta^2 W 2 2 ≍ η δ 2 and proves (E.4)–(E.6).
Joint probabilities and an HMM realization. Define two linear functionals on univariate polynomials by
A ( f ) = ∑ j f ( z j ) g ′ ( z j ) , B ( f ) = ∑ j { − g ′ ′ ( z j ) g ′ ( z j ) f ( z j ) + f ′ ( z j ) g ′ ( z j ) 2 } . \mathcal{A}(f)=\sum_j\frac{f(z_j)}{g'(z_j)},\qquad
\mathcal{B}(f)=\sum_j\left\{-\frac{g''(z_j)}{g'(z_j)}f(z_j)+\frac{f'(z_j)}{g'(z_j)^2}\right\}. A ( f ) = j ∑ g ′ ( z j ) f ( z j ) , B ( f ) = j ∑ { − g ′ ( z j ) g ′′ ( z j ) f ( z j ) + g ′ ( z j ) 2 f ′ ( z j ) } .
Ordinary Lagrange interpolation shows that A \mathcal{A} A extracts the leading coefficient on polynomials of degree at most s − 1 s-1 s − 1 . Hermite interpolation shows that B \mathcal{B} B extracts the leading coefficient on polynomials of degree at most 2 s − 1 2s-1 2 s − 1 : the leading coefficients of the value and derivative cardinal polynomials are respectively − g ′ ′ ( z j ) / g ′ ( z j ) 3 -g^{\prime\prime}(z_j)/g^\prime(z_j)^3 − g ′′ ( z j ) / g ′ ( z j ) 3 and 1 / g ′ ( z j ) 2 1/g^\prime(z_j)^2 1/ g ′ ( z j ) 2 . Consequently,
Λ r = B ⊗ A ⊗ ( r − 1 ) \Lambda_r=\mathcal{B}\otimes\mathcal{A}^{\otimes(r-1)} Λ r = B ⊗ A ⊗ ( r − 1 )
vanishes on every monomial of total degree below m = ( 2 s − 1 ) + ( r − 1 ) ( s − 1 ) = r ( s − 1 ) + s m=(2s-1)+(r-1)(s-1)=r(s-1)+s m = ( 2 s − 1 ) + ( r − 1 ) ( s − 1 ) = r ( s − 1 ) + s , and equals one on x 1 2 s − 1 ∏ ℓ = 2 r x ℓ s − 1 x_1^{2s-1}\prod_{\ell=2}^r x_\ell^{s-1} x 1 2 s − 1 ∏ ℓ = 2 r x ℓ s − 1 . This functional contains only values and derivatives in the first coordinate; it contains no mixed derivatives.
Write its coefficients at the true grid as
Λ r ( f ) = ∑ j ∈ [ s ] r { U j f ( z j 1 , … , z j r ) + C j ∂ 1 f ( z j 1 , … , z j r ) } . \Lambda_r(f)=\sum_{\mathbf{j}\in[s]^r}\{U_{\mathbf{j}}f(z_{j_1},\ldots,z_{j_r})+C_{\mathbf{j}}\partial_1f(z_{j_1},\ldots,z_{j_r})\}. Λ r ( f ) = j ∈ [ s ] r ∑ { U j f ( z j 1 , … , z j r ) + C j ∂ 1 f ( z j 1 , … , z j r )} .
The value vector U U U is nonzero. Indeed, g ′ ′ ( z s ) / g ′ ( z s ) = 2 ∑ j < s ( z s − z j ) − 1 > 0 g^{\prime\prime}(z_s)/g^\prime(z_s)=2\sum_{j<s}(z_s-z_j)^{-1}>0 g ′′ ( z s ) / g ′ ( z s ) = 2 ∑ j < s ( z s − z j ) − 1 > 0 , and every coefficient of A \mathcal{A} A is nonzero. Also ∑ j U j = 0 \sum_jU_j=0 ∑ j U j = 0 . Start with the product grid weights w j = s − r w_j=s^{-r} w j = s − r . Change them to w j + η U j w_j+\eta U_j w j + η U j , move the first coordinate of grid point j \mathbf{j} j by η C j / w j \eta C_j/w_j η C j / w j , and leave the other coordinates fixed. Choose η 0 \eta_0 η 0 so that η 0 max j ∣ U j ∣ < s − r / 2 \eta_0\max_j|U_j|<s^{-r}/2 η 0 max j ∣ U j ∣ < s − r /2 and η 0 max j ∣ C j ∣ / w j < 1 / 4 \eta_0\max_j|C_j|/w_j<1/4 η 0 max j ∣ C j ∣/ w j < 1/4 . For 0 < η ≤ η 0 0<\eta\leq\eta_0 0 < η ≤ η 0 , this gives positive normalized weights, unique indicated nearest-center assignments, and the prescribed first variation. Both inequalities involve only s s s , r r r and the fixed grid. Equation (E.9), with η δ = δ m + 1 \eta\delta=\delta^{m+1} η δ = δ m + 1 , gives (E.7) for the first r r r observations.
These joint mixing measures are realized by time-homogeneous HMMs. The true model has initial probabilities and transition rows all equal to ( 1 / s , … , 1 / s ) (1/s,\ldots,1/s) ( 1/ s , … , 1/ s ) . For the fitted model, introduce states ( j , ℓ ) (\mathbf{j},\ell) ( j , ℓ ) , with j ∈ [ s ] r \mathbf{j}\in[s]^r j ∈ [ s ] r and 1 ≤ ℓ ≤ r 1\leq\ell\leq r 1 ≤ ℓ ≤ r . Put initial mass w j + η U j w_j+\eta U_j w j + η U j on ( j , 1 ) (\mathbf{j},1) ( j , 1 ) and zero on the other states. Move deterministically from ( j , ℓ ) (\mathbf{j},\ell) ( j , ℓ ) to ( j , ℓ + 1 ) (\mathbf{j},\ell+1) ( j , ℓ + 1 ) for ℓ < r \ell<r ℓ < r , and use the perturbed grid coordinate as that state’s emission center. After time r r r , transition to an additional set of s s s states with the true emission centers. From every phase-r r r state, enter these additional states with probabilities ( 1 / s , … , 1 / s ) (1/s,\ldots,1/s) ( 1/ s , … , 1/ s ) , and give each additional state this same transition row. Thus k = r s r + s k=rs^r+s k = r s r + s suffices. The remaining observations are independent true draws, so they do not change the Hellinger distance for any T ≥ r T\geq r T ≥ r .
Nearest-center aggregation gives D I = η ∑ j ∣ U j ∣ ≍ η D_I=\eta\sum_j|U_j|\asymp\eta D I = η ∑ j ∣ U j ∣ ≍ η . In any coupling, at least D I / 2 D_I/2 D I /2 of the mass crosses different assigned grid labels. The assigned-label argument in Lemma A.5 applies without any Markov assumption on the grouped process: the mass on different labels is at least their total variation, D I / 2 D_I/2 D I /2 . On each such pair, at least one true coordinate differs from the fitted point’s assigned coordinate by at least δ \delta δ , whereas its perturbation is less than δ / 4 \delta/4 δ /4 . Thus the physical distance in that coordinate is at least 3 δ / 4 3\delta/4 3 δ /4 , and in particular its normalized squared cost is at least δ 2 / ( 4 r ) \delta^2/(4r) δ 2 / ( 4 r ) . This proves W 2 , I 2 ≥ c r η δ 2 W_{2,I}^2\geq c_r\eta\delta^2 W 2 , I 2 ≥ c r η δ 2 . Moving fitted grid points back to their assigned points costs O ( η 2 δ 2 ) O(\eta^2\delta^2) O ( η 2 δ 2 ) , and the remaining mass discrepancy costs O ( η δ 2 ) O(\eta\delta^2) O ( η δ 2 ) . This proves the reverse inequality and (E.8). Since m = β r m=\beta_r m = β r for s ≥ 2 s\geq2 s ≥ 2 , r ≥ 2 r\geq2 r ≥ 2 , the claimed joint powers are necessary. For r = 2 r=2 r = 2 , this also proves sharpness of the transition-flow exponent. □ \square □
To add separated clusters, give the moving cluster a fixed total initial mass α ∈ ( 0 , 1 ) \alpha\in(0,1) α ∈ ( 0 , 1 ) , keep both its true and fitted chains within that cluster, and give identical additional absorbing states the remaining positive initial masses in both models. Every true initial mass is then bounded away from zero along the separation sequence. The observation laws have the form α p η + ( 1 − α ) q \alpha p_\eta+(1-\alpha)q α p η + ( 1 − α ) q and α p 0 + ( 1 − α ) q \alpha p_0+(1-\alpha)q α p 0 + ( 1 − α ) q with the same q q q . Convexity of squared Hellinger distance gives the required upper order from the within-cluster comparison. The fixed polynomial moment used above has difference exactly α \alpha α times its within-cluster difference, giving the matching lower order by moment extraction. Probability discrepancies and displacement quantities acquire the same fixed factor. For transport, couple the additional identical paths to themselves for an upper bound; the preceding nearest-support or assigned-label arguments give the lower bound. Thus all displayed orders survive, with constants depending on α \alpha α , and establish the claimed structured worst-case powers when s = s max s=s_{\max} s = s m a x .
Emission-Center Recovery with One True State
The main text assumes K ≥ 2 K \ge2 K ≥ 2 because the paper studies separation among distinct true emission centers. When K = 1 K=1 K = 1 , there is no separation parameter. We can instead bound displacement directly from the sole true center, using a quadratic polynomial for an overfitted model and a linear polynomial under exact specification.
Proof. The true initial probability and transition probability are both one, so the true latent location equals a a a at every time. Thus G θ { t } = δ a G_{\theta}^{\{t\}}=\delta_a G θ { t } = δ a and G θ I = δ ( a , … , a ) G_{\theta}^{I}=\delta_{(a,\ldots,a)} G θ I = δ ( a , … , a ) . Every fitted state is assigned to this same center, meaning that c ( i ) = 1 c(i)=1 c ( i ) = 1 for all i ∈ [ k ] i \in[k] i ∈ [ k ] . After aggregation, the fitted and true label paths therefore both equal ( 1 , … , 1 ) (1,\ldots,1) ( 1 , … , 1 ) with probability one. All aggregate probability and flow discrepancies are zero; we only need to bound displacement from a a a .
Quadratic displacement. Fix t ≤ T t \le T t ≤ T . With q t ( i ) = P θ ( Z t = i ) q_t(i)=\mathbb{P}_{\theta}(Z_t=i) q t ( i ) = P θ ( Z t = i ) , the squared displacement at time t t t is V t = ∑ i = 1 k q t ( i ) ∥ μ i − a ∥ 2 2 V_t=\sum_{i=1}^{k}q_t(i)\|\mu_i-a\|_2^2 V t = ∑ i = 1 k q t ( i ) ∥ μ i − a ∥ 2 2 . We use P 1 ( x ) = ∥ x − a ∥ 2 2 P_1(x)=\|x-a\|_2^2 P 1 ( x ) = ∥ x − a ∥ 2 2 because its fitted expectation is exactly V t V_t V t and its true expectation is zero:
∫ P 1 d ( G θ { t } − δ a ) = ∑ i = 1 k q t ( i ) P 1 ( μ i ) − P 1 ( a ) = ∑ i = 1 k q t ( i ) ∥ μ i − a ∥ 2 2 = V t . \begin{aligned}
\int P_1\,\mathrm{d}\left(G_{\theta}^{\{t\}}-\delta_a\right)
=\sum_{i=1}^{k}q_t(i)P_1(\mu_i)-P_1(a) \\
&=\sum_{i=1}^{k}q_t(i)\|\mu_i-a\|_2^2=V_t.
\end{aligned} ∫ P 1 d ( G θ { t } − δ a ) = i = 1 ∑ k q t ( i ) P 1 ( μ i ) − P 1 ( a ) = i = 1 ∑ k q t ( i ) ∥ μ i − a ∥ 2 2 = V t .
Each summand is nonnegative, so displacements of different fitted centers cannot cancel in this expectation.
The polynomial has degree two, and ∥ P 1 ∥ ∞ , B d ( 0 , R ) ≤ 4 R 2 \|P_1\|_{\infty,B_d(0,R)} \le4R^2 ∥ P 1 ∥ ∞ , B d ( 0 , R ) ≤ 4 R 2 because ∥ x − a ∥ 2 ≤ 2 R \|x-a\|_2 \le2R ∥ x − a ∥ 2 ≤ 2 R on the ball. Let p θ { t } p_{\theta}^{\{t\}} p θ { t } and p θ ⋆ { t } p_{\theta^\star}^{\{t\}} p θ ⋆ { t } be the one-time observation densities. Lemma A.3 bounds the polynomial expectation by their Hellinger distance, and Lemma A.2 bounds this distance by the full-block error. Hence
V t ≤ C ∥ P 1 ∥ ∞ h ( p θ { t } , p θ ⋆ { t } ) ≤ C h ( p θ , T , p θ ⋆ , T ) , t ≤ T . (152) \begin{aligned}
V_t \le C\|P_1\|_{\infty}h\left(p_{\theta}^{\{t\}},p_{\theta^\star}^{\{t\}}\right) \\
&\le Ch(p_{\theta,T},p_{\theta^\star,T}), \qquad t \le T. \tag*{(152)}
\end{aligned} V t ≤ C ∥ P 1 ∥ ∞ h ( p θ { t } , p θ ⋆ { t } ) ≤ C h ( p θ , T , p θ ⋆ , T ) , t ≤ T . (152)
Here $4R^2$ is absorbed into C C C . As in the preceding proofs, we apply the moment bound on the span of the true and fitted centers. This span has dimension at most k + 1 k+1 k + 1 , so C C C may depend on k k k and $R. but not on the ambient dimension. The constant is uniform in t t t , since the polynomial and its norm bound are the same at every time. The argument also allows zero fitted probabilities and repeated fitted centers.
Transport of the location block. Fix a nonempty I ⊆ [ T ] I \subseteq[T] I ⊆ [ T ] with ∣ I ∣ = r |I| = r ∣ I ∣ = r . The true location block is the fixed vector ( a , … , a ) (a,\ldots,a) ( a , … , a ) . Any coupling with its point mass must therefore pair every fitted block with this vector. There is no choice of destination in the transport problem, and its cost is
W 2 , I 2 ( G θ I , δ ( a , … , a ) ) = E θ [ 1 r ∑ t ∈ I ∥ μ Z t − a ∥ 2 2 ] = 1 r ∑ t ∈ I ∑ i = 1 k q t ( i ) ∥ μ i − a ∥ 2 2 = 1 r ∑ t ∈ I V t ≤ C h ( p θ , T , p θ ⋆ , T ) , \begin{aligned}
W_{2,I}^{2}(G_{\theta}^{I},\delta_{(a,\ldots,a)}) &= \mathbb{E}_{\theta}\left[\frac{1}{r}\sum_{t\in I}\|\mu_{Z_t}-a\|_{2}^{2}\right]\\
&= \frac{1}{r}\sum_{t\in I}\sum_{i=1}^{k}q_t(i)\|\mu_i-a\|_{2}^{2}\\
&= \frac{1}{r}\sum_{t\in I}V_t\\
&\leq C h(p_{\theta,T},p_{\theta_\star,T}),
\end{aligned} W 2 , I 2 ( G θ I , δ ( a , … , a ) ) = E θ [ r 1 t ∈ I ∑ ∥ μ Z t − a ∥ 2 2 ] = r 1 t ∈ I ∑ i = 1 ∑ k q t ( i ) ∥ μ i − a ∥ 2 2 = r 1 t ∈ I ∑ V t ≤ C h ( p θ , T , p θ ⋆ , T ) ,
by (E.11). The second equality follows from linearity of expectation and the marginal distribution of Z t Z_t Z t ; no independence across times is needed. Since the normalized cost averages the one-time squared displacements, the constant does not grow with r r r . This proves the first bound.
Exact specification. If k = 1 k=1 k = 1 , the fitted location is also deterministic and equals μ 1 \mu_1 μ 1 at every time. When μ 1 = a \mu_1=a μ 1 = a , the second bound is immediate. Otherwise, take u = ( μ 1 − a ) / ∥ μ 1 − a ∥ 2 u=(\mu_1-a)/\|\mu_1-a\|_2 u = ( μ 1 − a ) /∥ μ 1 − a ∥ 2 and U ( x ) = ⟨ u , x − a ⟩ U(x)=\langle u,x-a\rangle U ( x ) = ⟨ u , x − a ⟩ . Choosing the direction of the displacement gives
U ( a ) = 0 , U ( μ 1 ) = ⟨ μ 1 − a ∥ μ 1 − a ∥ 2 , μ 1 − a ⟩ = ∥ μ 1 − a ∥ 2 . U(a)=0,\qquad U(\mu_1)=\left\langle\frac{\mu_1-a}{\|\mu_1-a\|_2},\mu_1-a\right\rangle=\|\mu_1-a\|_2. U ( a ) = 0 , U ( μ 1 ) = ⟨ ∥ μ 1 − a ∥ 2 μ 1 − a , μ 1 − a ⟩ = ∥ μ 1 − a ∥ 2 .
Thus, unlike the quadratic polynomial, this linear polynomial recovers the displacement itself:
∫ U d ( δ μ 1 − δ a ) = ∥ μ 1 − a ∥ 2 . \int U\,\mathrm{d}(\delta_{\mu_1}-\delta_a)=\|\mu_1-a\|_2. ∫ U d ( δ μ 1 − δ a ) = ∥ μ 1 − a ∥ 2 .
Its supremum norm is at most 2 R 2R 2 R , since ∣ U ( x ) ∣ ≤ ∥ u ∥ 2 ∥ x − a ∥ 2 ≤ 2 R |U(x)|\leq\|u\|_2\|x-a\|_2\leq2R ∣ U ( x ) ∣ ≤ ∥ u ∥ 2 ∥ x − a ∥ 2 ≤ 2 R . Lemmas A.3 and A.2 now give
∥ μ 1 − a ∥ 2 ≤ C ∥ U ∥ ∞ h ( p θ { t } , p θ ⋆ { t } ) ≤ C h ( p θ , T , p θ ⋆ , T ) , \begin{aligned}
\|\mu_1-a\|_2 &\leq C\|U\|_{\infty}h\left(p_{\theta}^{\{t\}},p_{\theta_\star}^{\{t\}}\right)\\
&\leq Ch(p_{\theta,T},p_{\theta_\star,T}),
\end{aligned} ∥ μ 1 − a ∥ 2 ≤ C ∥ U ∥ ∞ h ( p θ { t } , p θ ⋆ { t } ) ≤ C h ( p θ , T , p θ ⋆ , T ) ,
which proves the second bound. In this case, both location blocks are deterministic. Each of their r r r coordinates has squared displacement ∥ μ 1 − a ∥ 2 2 \|\mu_1-a\|_2^2 ∥ μ 1 − a ∥ 2 2 , so the normalized path cost gives
W 2 , I ( δ ( μ 1 , … , μ 1 ) , δ ( a , … , a ) ) = { r ∥ μ 1 − a ∥ 2 2 r } 1 / 2 = ∥ μ 1 − a ∥ 2 . W_{2,I}\left(\delta_{(\mu_1,\ldots,\mu_1)},\delta_{(a,\ldots,a)}\right)=\left\{\frac{r\|\mu_1-a\|_2^2}{r}\right\}^{1/2}=\|\mu_1-a\|_2. W 2 , I ( δ ( μ 1 , … , μ 1 ) , δ ( a , … , a ) ) = { r r ∥ μ 1 − a ∥ 2 2 } 1/2 = ∥ μ 1 − a ∥ 2 .
Thus the same linear bound also holds for the location-block Wasserstein distance.
The proof of Proposition 2.2 uses only compactness, Gaussian emissions, and inclusion of the true density in the fitted class; it does not use K ≥ 2 K\geq2 K ≥ 2 or distinctness of two true centers. It therefore applies unchanged when K = 1 K=1 K = 1 . For the MLE, it bounds the Hellinger error by ε n , T \varepsilon_{n,T} ε n , T on E n \mathcal{E}_n E n . Taking square roots in the first bound of Remark E.2 gives W 2 , I ≤ C ε n , T 1 / 2 W_{2,I}\leq C\varepsilon_{n,T}^{1/2} W 2 , I ≤ C ε n , T 1/2 . Under exact specification, the second bound instead gives ∥ μ 1 − a ∥ 2 ≤ C ε n , T \|\mu_1-a\|_2\leq C\varepsilon_{n,T} ∥ μ 1 − a ∥ 2 ≤ C ε n , T . For fixed model dimensions and radius, the resulting upper convergence rates are n − 1 / 4 n^{-1/4} n − 1/4 and n − 1 / 2 n^{-1/2} n − 1/2 , respectively, up to logarithmic factors.
State splitting and aggregate recovery under over-specification
This appendix verifies Example 4.1 in § 4 . The construction explains why the over-specification results are formulated for aggregate quantities. Individual fitted states can split a single true state while leaving the joint observation density unchanged at every sequence length, so their individual probabilities and transition rows are not identifiable from observations alone. Theorem 4.2 therefore controls aggregate state probabilities and transition flows, while Theorem 4.4 compares the induced emission-center distributions and finite-block latent location laws without requiring a one-to-one correspondence between fitted and true states.
Proof. For each true state j j j , choose m j ≥ 1 m_j \ge1 m j ≥ 1 substates ( j , 1 ) , … , ( j , m j ) (j,1),\ldots,(j,m_j) ( j , 1 ) , … , ( j , m j ) , all with emission center a j a_j a j . Write Z ~ t \widetilde{Z}_t Z t for the fitted substate at time t t t , and let Y t = j Y_t=j Y t = j whenever Z ~ t = ( j , u ) \widetilde{Z}_t=(j,u) Z t = ( j , u ) for some u u u . Thus Y t Y_t Y t records the true-state group containing the fitted substate. We first verify the fitted initial and transition probabilities, then show that summing over substates recovers the true path probabilities. Since all substates in a group have the same emission density, this will also give equality of the joint observation densities.
Initial and transition probabilities. Choose nonnegative fitted initial probabilities satisfying
∑ u = 1 m j π ~ j , u = π j ⋆ , j ∈ [ K ] . \sum_{u=1}^{m_j}\widetilde{\pi}_{j,u}=\pi_j^\star,\qquad j\in[K]. u = 1 ∑ m j π j , u = π j ⋆ , j ∈ [ K ] .
For each source substate ( j , u ) (j,u) ( j , u ) and destination group ℓ \ell ℓ , choose a probability vector r j ℓ , u r_{j\ell,u} r j ℓ , u on { 1 , … , m ℓ } \{1,\ldots,m_\ell\} { 1 , … , m ℓ } and set
A ~ ( j , u ) , ( ℓ , v ) = A j ℓ ⋆ r j ℓ , u ( v ) , r j ℓ , u ( v ) ≥ 0 , ∑ v = 1 m ℓ r j ℓ , u ( v ) = 1. \widetilde{A}_{(j,u),(\ell,v)}=A_{j\ell}^\star r_{j\ell,u}(v),\qquad r_{j\ell,u}(v)\ge0,\qquad\sum_{v=1}^{m_\ell}r_{j\ell,u}(v)=1. A ( j , u ) , ( ℓ , v ) = A j ℓ ⋆ r j ℓ , u ( v ) , r j ℓ , u ( v ) ≥ 0 , v = 1 ∑ m ℓ r j ℓ , u ( v ) = 1.
The initial probabilities sum to ∑ j π j ⋆ = 1 \sum_j\pi_j^\star=1 ∑ j π j ⋆ = 1 . Every transition entry is nonnegative, and each fitted row sums to one:
∑ ℓ = 1 K ∑ v = 1 m ℓ A ~ ( j , u ) , ( ℓ , v ) = ∑ ℓ = 1 K A j ℓ ⋆ ∑ v = 1 m ℓ r j ℓ , u ( v ) = ∑ ℓ = 1 K A j ℓ ⋆ = 1. \sum_{\ell=1}^{K}\sum_{v=1}^{m_\ell}\widetilde{A}_{(j,u),(\ell,v)}
=\sum_{\ell=1}^{K}A_{j\ell}^\star\sum_{v=1}^{m_\ell}r_{j\ell,u}(v)
=\sum_{\ell=1}^{K}A_{j\ell}^\star=1. ℓ = 1 ∑ K v = 1 ∑ m ℓ A ( j , u ) , ( ℓ , v ) = ℓ = 1 ∑ K A j ℓ ⋆ v = 1 ∑ m ℓ r j ℓ , u ( v ) = ℓ = 1 ∑ K A j ℓ ⋆ = 1.
Thus these parameters define a valid HMM. More specifically, the total transition probability from any substate ( j , u ) (j,u) ( j , u ) into group ℓ \ell ℓ is
∑ v = 1 m ℓ A ~ ( j , u ) , ( ℓ , v ) = A j ℓ ⋆ . (153) \sum_{v=1}^{m_\ell}\widetilde{A}_{(j,u),(\ell,v)}=A_{j\ell}^\star. \tag*{(153)} v = 1 ∑ m ℓ A ( j , u ) , ( ℓ , v ) = A j ℓ ⋆ . (153)
The allocation among destination substates may depend on u u u , but the total probability of entering their group does not.
Aggregate path probabilities. Fix a horizon L ≥ 1 L\ge1 L ≥ 1 and a group-label path z = ( z 1 , … , z L ) ∈ [ K ] L z=(z_1,\ldots,z_L)\in[K]^L z = ( z 1 , … , z L ) ∈ [ K ] L . At the first time,
P θ ( Y 1 = z 1 ) = ∑ u = 1 m z 1 π ~ z 1 , u = π z 1 ⋆ . \mathbb{P}_{\theta}(Y_1=z_1)=\sum_{u=1}^{m_{z_1}}\widetilde{\pi}_{z_1,u}=\pi_{z_1}^\star. P θ ( Y 1 = z 1 ) = u = 1 ∑ m z 1 π z 1 , u = π z 1 ⋆ .
For 1 ≤ t < L 1 \le t < L 1 ≤ t < L , partition the event { Y 1 : t = z 1 : t } \{Y_{1:t}=z_{1:t}\} { Y 1 : t = z 1 : t } according to the current substate ( z t , u ) (z_t,u) ( z t , u ) , and sum the next transition over all substates of group z t + 1 z_{t+1} z t + 1 . The Markov property of Z ~ t \widetilde{Z}_t Z t gives
P θ ( Y 1 : t + 1 = z 1 : t + 1 ) = ∑ u = 1 m z t P θ { Y 1 : t = z 1 : t , Z ~ t = ( z t , u ) } ∑ v = 1 m z t + 1 A ~ ( z t , u ) , ( z t + 1 , v ) = A z t z t + 1 ⋆ ∑ u = 1 m z t P θ { Y 1 : t = z 1 : t , Z ~ t = ( z t , u ) } = A z t z t + 1 ⋆ P θ ( Y 1 : t = z 1 : t ) . \begin{aligned}
\mathbb{P}_{\theta}(Y_{1:t+1}=z_{1:t+1})
&=\sum_{u=1}^{m_{z_t}}\mathbb{P}_{\theta}\{Y_{1:t}=z_{1:t},\widetilde{Z}_t=(z_t,u)\}\sum_{v=1}^{m_{z_{t+1}}}\widetilde{A}_{(z_t,u),(z_{t+1},v)}\\
&=A^\star_{z_tz_{t+1}}\sum_{u=1}^{m_{z_t}}\mathbb{P}_{\theta}\{Y_{1:t}=z_{1:t},\widetilde{Z}_t=(z_t,u)\}\\
&=A^\star_{z_tz_{t+1}}\mathbb{P}_{\theta}(Y_{1:t}=z_{1:t}).
\end{aligned} P θ ( Y 1 : t + 1 = z 1 : t + 1 ) = u = 1 ∑ m z t P θ { Y 1 : t = z 1 : t , Z t = ( z t , u )} v = 1 ∑ m z t + 1 A ( z t , u ) , ( z t + 1 , v ) = A z t z t + 1 ⋆ u = 1 ∑ m z t P θ { Y 1 : t = z 1 : t , Z t = ( z t , u )} = A z t z t + 1 ⋆ P θ ( Y 1 : t = z 1 : t ) .
The second equality uses (E.12); the third uses the fact that the current-substate events partition the specified group path. Starting from the initial probability and applying this recursion for t = 1 , … , L − 1 t=1,\ldots,L-1 t = 1 , … , L − 1 yields
P θ ( Y 1 : L = z ) = π z 1 ⋆ ∏ t = 1 L − 1 A z t z t + 1 ⋆ = P ⋆ ( Z 1 : L = z ) . (154) \mathbb{P}_{\theta}(Y_{1:L}=z)=\pi^\star_{z_1}\prod_{t=1}^{L-1}A^\star_{z_tz_{t+1}}=\mathbb{P}_{\star}(Z_{1:L}=z). \tag*{(154)} P θ ( Y 1 : L = z ) = π z 1 ⋆ t = 1 ∏ L − 1 A z t z t + 1 ⋆ = P ⋆ ( Z 1 : L = z ) . (154)
For L = 1 L=1 L = 1 , the product is empty and equals one. The calculation also covers zero-probability paths, since it never divides by a path probability or conditions on a specified group path.
Equation (E.13) proves that the fitted and true models assign the same probability to every group-label sequence. Summing over coordinates not retained in a given marginal therefore gives equality of all aggregate state probabilities and joint probabilities, including the transition flows.
Observation densities. Consider a fitted substate path ( ( z 1 , u 1 ) , … , ( z L , u L ) ) ((z_1,u_1),\ldots,(z_L,u_L)) (( z 1 , u 1 ) , … , ( z L , u L )) whose group labels are z z z . Every substate ( z t , u t ) (z_t,u_t) ( z t , u t ) has center a z t a_{z_t} a z t , so the observation density associated with this path is ∏ t = 1 L ϕ d ( x t − a z t ) \prod_{t=1}^{L}\phi_d(x_t-a_{z_t}) ∏ t = 1 L ϕ d ( x t − a z t ) . This product does not depend on u 1 , … , u L u_1,\ldots,u_L u 1 , … , u L .
In the finite-path expansion of the fitted observation density, we may therefore group together all substate paths with the same group labels and factor out their common emission density:
p θ , L ( x 1 : L ) = ∑ z ∈ [ K ] L { ∑ u 1 = 1 m z 1 ⋯ ∑ u L = 1 m z L π ~ z 1 , u 1 ∏ t = 1 L − 1 A ~ ( z t , u t ) , ( z t + 1 , u t + 1 ) } ∏ t = 1 L ϕ d ( x t − a z t ) = ∑ z ∈ [ K ] L P θ ( Y 1 : L = z ) ∏ t = 1 L ϕ d ( x t − a z t ) = ∑ z ∈ [ K ] L π z 1 ⋆ ∏ t = 1 L − 1 A z t z t + 1 ⋆ ∏ t = 1 L ϕ d ( x t − a z t ) = p θ ⋆ , L ( x 1 : L ) . \begin{aligned}
p_{\theta,L}(x_{1:L})
&=\sum_{z\in[K]^L}\left\{\sum_{u_1=1}^{m_{z_1}}\cdots\sum_{u_L=1}^{m_{z_L}}\widetilde{\pi}_{z_1,u_1}\prod_{t=1}^{L-1}\widetilde{A}_{(z_t,u_t),(z_{t+1},u_{t+1})}\right\}\prod_{t=1}^{L}\phi_d(x_t-a_{z_t})\\
&=\sum_{z\in[K]^L}\mathbb{P}_{\theta}(Y_{1:L}=z)\prod_{t=1}^{L}\phi_d(x_t-a_{z_t})\\
&=\sum_{z\in[K]^L}\pi^\star_{z_1}\prod_{t=1}^{L-1}A^\star_{z_tz_{t+1}}\prod_{t=1}^{L}\phi_d(x_t-a_{z_t})\\
&=p_{\theta^\star,L}(x_{1:L}).
\end{aligned} p θ , L ( x 1 : L ) = z ∈ [ K ] L ∑ { u 1 = 1 ∑ m z 1 ⋯ u L = 1 ∑ m z L π z 1 , u 1 t = 1 ∏ L − 1 A ( z t , u t ) , ( z t + 1 , u t + 1 ) } t = 1 ∏ L ϕ d ( x t − a z t ) = z ∈ [ K ] L ∑ P θ ( Y 1 : L = z ) t = 1 ∏ L ϕ d ( x t − a z t ) = z ∈ [ K ] L ∑ π z 1 ⋆ t = 1 ∏ L − 1 A z t z t + 1 ⋆ t = 1 ∏ L ϕ d ( x t − a z t ) = p θ ⋆ , L ( x 1 : L ) .
The expression in braces sums the probabilities of all fitted paths having group labels z z z , which explains the second equality. The third follows from (E.13), and the last is the true HMM path expansion. Since L L L and x 1 : L x_{1:L} x 1 : L were arbitrary, the joint observation densities agree for every sequence length. This holds for every admissible allocation of the initial probabilities and every choice of the vectors r j ℓ , u r_{j\ell,u} r j ℓ , u .
Different fitted parameters with identical observation densities. To see that the construction changes transition parameters, choose j , ℓ j,\ell j , ℓ with A j ℓ ⋆ > 0 A^\star_{j\ell}>0 A j ℓ ⋆ > 0 and give state ℓ \ell ℓ at least two substates. Fix a source substate ( j , u ) (j,u) ( j , u ) and vary only its allocation into group ℓ \ell ℓ :
r j ℓ , u ( 1 ) = λ , r j ℓ , u ( 2 ) = 1 − λ , r j ℓ , u ( v ) = 0 ( v ≥ 3 ) , 0 < λ < 1. r_{j\ell,u}(1)=\lambda,\qquad r_{j\ell,u}(2)=1-\lambda,\qquad r_{j\ell,u}(v)=0\quad(v\ge3),\qquad0<\lambda<1. r j ℓ , u ( 1 ) = λ , r j ℓ , u ( 2 ) = 1 − λ , r j ℓ , u ( v ) = 0 ( v ≥ 3 ) , 0 < λ < 1.
The corresponding transition entries are A j ℓ ⋆ λ A^\star_{j\ell}\lambda A j ℓ ⋆ λ and A j ℓ ⋆ ( 1 − λ ) A^\star_{j\ell}(1-\lambda) A j ℓ ⋆ ( 1 − λ ) . They change with λ \lambda λ , while their sum remains A j ℓ ⋆ A^\star_{j\ell} A j ℓ ⋆ . Keeping all other parameters fixed gives continuously many distinct fitted transition matrices, all yielding the same joint observation densities, as proved above.
This ambiguity cannot be accounted for by state permutations alone. For a fixed fitted state count, there are only finitely many permutations, so the continuously many parameter choices cannot all be relabelings of one parameter.
The initial probabilities can likewise vary whenever a state j j j with π j ⋆ > 0 \pi^\star_j>0 π j ⋆ > 0 has at least two substates. For example, setting π ~ j , 1 = λ π j ⋆ \widetilde{\pi}_{j,1}=\lambda\pi^\star_j π j , 1 = λ π j ⋆ and π ~ j , 2 = ( 1 − λ ) π j ⋆ \widetilde{\pi}_{j,2}=(1-\lambda)\pi^\star_j π j , 2 = ( 1 − λ ) π j ⋆ , with zero initial mass on any remaining substates of that group, preserves its total initial probability for every 0 < λ < 1 0<\lambda<1 0 < λ < 1 . The joint observation densities again remain unchanged.
Thus the observations cannot distinguish the individual substate allocations in this construction. Their aggregate probabilities and flows remain equal to the true quantities, and their latent location measures also coincide with those of the true model because each group retains its true emission center.
Additional Experimental Details
This appendix gives the fitting and evaluation details for § 5 , followed by additional results on emission geometry, fitted-center grouping, and reference-center accuracy.
Model fitting and evaluation
All estimation experiments use independent trajectories of length T = 6 T=6 T = 6 , known identity emission covariance, and centers constrained to B d ( 0 , 6 ) B_d(0,6) B d ( 0 , 6 ) . Models with different fitted state counts use the same data within each repetition. We implement Baum–Welch updates using scaled forward–backward recursions, following Rabiner [52 ] . Each fit uses four initializations and retains the one with the largest final training likelihood. Runs stop when the absolute likelihood increment divided by n T nT n T is at most 10 − 7 10^{-7} 1 0 − 7 or after 400 iterations. All selected fits enter the summaries. These stopping rules do not certify global likelihood maximization.
Initialization. In the scalar experiments, the k k k initial centers are empirical quantiles at equally spaced levels from 0.08 to 0.92, with independent Gaussian perturbations of standard deviation 0.25 + 0.08 j 0.25+0.08j 0.25 + 0.08 j at restart j ∈ { 0 , 1 , 2 , 3 } j\in\{0,1,2,3\} j ∈ { 0 , 1 , 2 , 3 } . Initial probabilities are drawn from Dirichlet ( 2 , … , 2 ) \operatorname{Dirichlet}(2,\ldots,2) Dirichlet ( 2 , … , 2 ) . Transition row i i i is drawn from Dirichlet ( 1 + λ e i ) \operatorname{Dirichlet}(1+\lambda e_i) Dirichlet ( 1 + λ e i ) , with an independent λ ∼ Unif [ 1 , 8 ] \lambda\sim\operatorname{Unif}[1,8] λ ∼ Unif [ 1 , 8 ] . Center updates are projected onto B d ( 0 , 6 ) B_d(0,6) B d ( 0 , 6 ) ; probabilities are updated from normalized expected counts without a positive floor. If an expected count is zero, the corresponding center or transition row is retained. The master statistical seed is 202610240, with separate random streams for data, initialization, and evaluation.
Latent losses. For I = ( t 1 , … , t r ) I=(t_1,\ldots,t_r) I = ( t 1 , … , t r ) , we compute the state-tuple probability as
q t 1 ( i 1 ) ∏ ℓ = 2 r ( A t ℓ − t ℓ − 1 ) i ℓ − 1 i ℓ . q_{t_1}(i_1)\prod_{\ell=2}^{r}\left(A^{t_\ell-t_{\ell-1}}\right)_{i_{\ell-1}i_\ell}. q t 1 ( i 1 ) ℓ = 2 ∏ r ( A t ℓ − t ℓ − 1 ) i ℓ − 1 i ℓ .
We sum these probabilities within the assigned groups before taking differences from the true law. This also applies to nonconsecutive times. The Wasserstein losses use the normalized block costs from § 2.2 . We solve the finite transport problem by linear programming with HiGHS; in the scalar independent-state comparisons, we use monotone matching of the one-time measures. No Markov approximation is imposed on a grouped process.
Observation error and uncertainty. We evaluate HMM densities by the forward algorithm. For true and fitted densities p p p , q q q , let m = ( p + q ) / 2 m = (p + q)/2 m = ( p + q ) /2 . We use
h 2 ( p , q ) = E X ∼ m [ ( p ( X ) − q ( X ) ) 2 p ( X ) + q ( X ) ] . (155) h^2(p,q) = \mathbb{E}_{X \sim m}\left[\frac{\left(\sqrt{p(X)}-\sqrt{q(X)}\right)^2}{p(X)+q(X)}\right]. \tag*{(155)} h 2 ( p , q ) = E X ∼ m p ( X ) + q ( X ) ( p ( X ) − q ( X ) ) 2 . (155)
The expectation is estimated using 4096 independent trajectories from each model, independently of training, with equal weight given to the two sample means. Reported Hellinger errors are the square roots of these estimates. Figure 5 and Figure 7 use means and pointwise percentile bootstrap intervals from 2000 resamples of whole data sets. The restricted benchmark in Figure 6 uses 4000 bootstrap resamples over its 256 data sets. Center-matching frequencies use 95% Wilson intervals. Restarts are not treated as independent data sets.
Separation Comparisons and the Global Likelihood Benchmark
Moment-matched pairs. For s = 2 , 3 , 4 s = 2, 3, 4 s = 2 , 3 , 4 , the true unscaled nodes are z j = j − ( s + 1 ) / 2 z_j = j - (s + 1)/2 z j = j − ( s + 1 ) /2 with weights 1 / s 1/s 1/ s . In the exact-fit comparison, we fix the last candidate node at z s + 0.01 z_s + 0.01 z s + 0.01 and solve for the remaining nodes and weights by matching moments of orders 0 , … , 2 s − 2 0, \ldots, 2s - 2 0 , … , 2 s − 2 . For the over-specified comparison, the first candidate group has two nodes at y 1 ± 0.01 y_1 \pm0.01 y 1 ± 0.01 , each with half the group weight; the group centers and weights are chosen to match moments of orders 0 , … , 2 s − 1 0,\ldots,2s-1 0 , … , 2 s − 1 . These are the constructions used in § 5.1 and § 5.2 . The four-state comparison uses the two-pair construction described in § 5.1 .
We scale the within-cluster nodes by each δ \delta δ in the main-text grid and evaluate the one-time Hellinger distance h 1 h_1 h 1 by Gauss–Hermite quadrature [29 ] . The full-block distance follows from h T 2 = 1 − ( 1 − h 1 2 ) T h_T^2=1-(1-h_1^2)^T h T 2 = 1 − ( 1 − h 1 2 ) T . Moment equations and density integrals use 80-digit arithmetic, with quadrature orders 128 and 192 compared for stability. The exponents in Table 5 and Table 6 are the negative least-squares slopes of the log error ratios against log δ \log\delta log δ over δ ∈ { 0.18 , 0.12 , 0.08 , 0.05 } \delta\in\{0.18,0.12,0.08,0.05\} δ ∈ { 0.18 , 0.12 , 0.08 , 0.05 } . The theoretical exponents are used only for comparison after the densities and losses have been evaluated.
Joint over-specified comparison. We use the perturbations in Proposition E.1 . With z j = j − ( s + 1 ) / 2 z_j=j-(s+1)/2 z j = j − ( s + 1 ) /2 and g ( z ) = ∏ j ( z − z j ) g(z)=\prod_j(z-z_j) g ( z ) = ∏ j ( z − z j ) , set a j = 1 / g ′ ( z j ) a_j=1/g'(z_j) a j = 1/ g ′ ( z j ) , b j = − g ′ ′ ( z j ) / g ′ ( z j ) 3 b_j=-g''(z_j)/g'(z_j)^3 b j = − g ′′ ( z j ) / g ′ ( z j ) 3 , and c j = 1 / g ′ ( z j ) 2 c_j=1/g'(z_j)^2 c j = 1/ g ′ ( z j ) 2 . Writing ϕ \phi ϕ for the standard Gaussian density, define
f δ ( x ) = 1 s ∑ j ϕ ( x − δ z j ) , A δ ( x ) = ∑ j a j ϕ ( x − δ z j ) , B δ ( x ) = ∑ j { b j + c j δ ( x − δ z j ) } ϕ ( x − δ z j ) . f_\delta(x)=\frac{1}{s}\sum_j\phi(x-\delta z_j),\qquad A_\delta(x)=\sum_j a_j\phi(x-\delta z_j),\qquad B_\delta(x)=\sum_j\{b_j+c_j\delta(x-\delta z_j)\}\phi(x-\delta z_j). f δ ( x ) = s 1 j ∑ ϕ ( x − δ z j ) , A δ ( x ) = j ∑ a j ϕ ( x − δ z j ) , B δ ( x ) = j ∑ { b j + c j δ ( x − δ z j )} ϕ ( x − δ z j ) .
The factorized first density variation gives the Hellinger coefficient
c h ( δ ) = 1 8 ( ∫ B δ ( x ) 2 f δ ( x ) d x ) 1 / 2 ( ∫ A δ ( x ) 2 f δ ( x ) d x ) ( r − 1 ) / 2 . (156) c_h(\delta)=\frac{1}{\sqrt{8}}\left(\int\frac{B_\delta(x)^2}{f_\delta(x)}\,dx\right)^{1/2}\left(\int\frac{A_\delta(x)^2}{f_\delta(x)}\,dx\right)^{(r-1)/2}. \tag*{(156)} c h ( δ ) = 8 1 ( ∫ f δ ( x ) B δ ( x ) 2 d x ) 1/2 ( ∫ f δ ( x ) A δ ( x ) 2 d x ) ( r − 1 ) /2 . (156)
We evaluate these scalar integrals by the same quadrature procedure. For a grid tuple j j j , the mass derivative is U j = b j 1 ∏ ℓ = 2 r a j ℓ U_j=b_{j_1}\prod_{\ell=2}^r a_{j_\ell} U j = b j 1 ∏ ℓ = 2 r a j ℓ , so c D = ∑ j ∣ U j ∣ c_D=\sum_j\lvert U_j\rvert c D = ∑ j ∣ U j ∣ . To compute c W c_W c W , we solve the grid-flow problem with divergence U U U , allowing mass to pass through occupied intermediate nodes. Directed nearest-neighbor edges have cost 1 / r 1/r 1/ r , and the optimal cost is multiplied by δ 2 \delta^2 δ 2 . The clock-state realization has k = r s r + s k=rs^r+s k = r s r + s states, as specified in Proposition E.1 . Figure 4 compares c D / c h c_D/c_h c D / c h and c W / c h c_W/c_h c W / c h at fixed s = 3 s=3 s = 3 and r = 2 , 3 , 4 r=2,3,4 r = 2 , 3 , 4 .
Restricted likelihood benchmark. Each data set contains N = 6 n N=6n N = 6 n independent standard Gaussian observations. The single-Gaussian MLE is the sample mean projected onto [ − 6 , 6 ] [-6,6] [ − 6 , 6 ] . For the equally weighted symmetric mixture with centers ± a \pm a ± a , the log likelihood, up to a constant, is
ℓ N ( a ) = − N a 2 2 + ∑ i = 1 N log cosh ( a Y i ) , a ∈ [ 0 , 6 ] . \ell_N(a)=-\frac{Na^2}{2}+\sum_{i=1}^{N}\log\cosh(aY_i),\qquad a\in[0,6]. ℓ N ( a ) = − 2 N a 2 + i = 1 ∑ N log cosh ( a Y i ) , a ∈ [ 0 , 6 ] .
For a > 0 a>0 a > 0 , ℓ N ′ ( a ) / ( N a ) = N − 1 ∑ i Y i 2 tanh ( a Y i ) / ( a Y i ) − 1 \ell_N'(a)/(Na)=N^{-1}\sum_iY_i^2\tanh(aY_i)/(aY_i)-1 ℓ N ′ ( a ) / ( N a ) = N − 1 ∑ i Y i 2 tanh ( a Y i ) / ( a Y i ) − 1 is decreasing in a a a , since tanh ( x ) / x \tanh(x)/x tanh ( x ) / x decreases with ∣ x ∣ \lvert x\rvert ∣ x ∣ . The maximizer is zero if N − 1 ∑ i Y i 2 ≤ 1 N^{-1}\sum_iY_i^2\leq1 N − 1 ∑ i Y i 2 ≤ 1 . Otherwise it is six if the derivative remains nonnegative there, or the unique positive root of the derivative, found by a bracketed scalar solver. This gives a global maximizer within the stated symmetric submodel. Zero estimates are retained in the means and bootstrap intervals.
Estimation under different cluster geometries
We supplement the controlled comparisons in § 5.1 by fitting models under two emission geometries with the same K = 4 K=4 K = 4 and minimum separation δ \delta δ . The single-cluster centers are δ ( − 1.5 , − 0.5 , 0.5 , 1.5 ) \delta(-1.5,-0.5,0.5,1.5) δ ( − 1.5 , − 0.5 , 0.5 , 1.5 ) , while the two-pair centers are
( − 2.5 − δ / 2 , − 2.5 + δ / 2 , 2.5 − δ / 2 , 2.5 + δ / 2 ) . (-2.5-\delta/2,-2.5+\delta/2,2.5-\delta/2,2.5+\delta/2). ( − 2.5 − δ /2 , − 2.5 + δ /2 , 2.5 − δ /2 , 2.5 + δ /2 ) .
The initial distribution is uniform and both geometries use A ⋆ = 0.76 I 4 + 0.2411 T / 4 A^\star= 0.76I_4 + 0.2411^\mathsf{T}/4 A ⋆ = 0.76 I 4 + 0.241 1 T /4 . For each δ ∈ { 1 , 0.65 , 0.4 , 0.25 } \delta\in\{1, 0.65, 0.4, 0.25\} δ ∈ { 1 , 0.65 , 0.4 , 0.25 } , we generate 12 independent data sets with n = 512 n = 512 n = 512 and fit k = 4 , 5 k = 4, 5 k = 4 , 5 models using the procedure above.
Figure 8 shows that smaller observation error need not imply better aggregate recovery. For the single cluster with k = 4 k = 4 k = 4 , reducing δ \delta δ from 1 to 0.25 decreases mean h h h from 0.065 to 0.041, while mean D A D_A D A increases from 0.404 to 1.008. At δ = 0.25 \delta= 0.25 δ = 0.25 , the two-pair model has larger mean h h h but smaller mean D A D_A D A . The same pattern occurs for k = 5 k = 5 k = 5 . These fitted-model results complement the controlled separation scans, where the loss-to-Hellinger ratios isolate the dependence on cluster size.
Figure 8. Observation fit and aggregate recovery under different cluster geometries. Each point reports mean h h h and D A D_A D A over 12 independent data sets with K = 4 K = 4 K = 4 , n = 512 n = 512 n = 512 , and T = 6 T = 6 T = 6 . Labels give the minimum separation δ \delta δ ; panels use k = 4 k = 4 k = 4 and k = 5 k = 5 k = 5 .
Grouping based on fitted centers
We reuse the k = 4 , 5 k = 4, 5 k = 4 , 5 fits from § 5.3 . We apply complete-linkage clustering to the fitted centers and cut the hierarchy at the known true group count K = 3 K = 3 K = 3 . Each group is represented by its mean center, weighted by the fitted states’ average occupancies over the six times. To compare the resulting groups with the true states, we match their representatives one-to-one with the true centers by minimizing the total distance between matched centers. The true centers are not used to form the partition. We then compute D A D_A D A using this assignment and compare it with the nearest-true-center assignment.
As shown in Table 8 , the two assignments agree for all 24 data sets at n = 2048 n = 2048 n = 2048 . At smaller sample sizes, they differ in up to two data sets per configuration, while the mean flow errors remain close. Fitted-center grouping therefore gives similar aggregate recovery in this experiment. This comparison assumes K K K is known and does not establish the reference-accuracy condition in § 4.3 .
Table 8. Aggregate recovery using fitted-center groups. Entries are means over the same 24 independent data sets as Table 7 . “Same groups” counts partitions agreeing with the nearest-true-center assignment after label alignment. The fitted-center partition uses the known group count K = 3 K = 3 K = 3 . Aggregate-flow error.
Sensitivity to reference-center error
To examine Proposition 4.8 , we keep each fitted HMM fixed and perturb its true centers to obtain reference centers ξ j = μ j ⋆ + τ u j \xi_j = \mu_j^\star+ \tau u_j ξ j = μ j ⋆ + τ u j , where τ / δ s e p ∈ { 0 , 0.10 , 0.20 , 0.25 } \tau/\delta_{\mathrm{sep}} \in\{0, 0.10, 0.20, 0.25\} τ / δ sep ∈ { 0 , 0.10 , 0.20 , 0.25 } . Each u j u_j u j is an independently sampled unit direction, a random sign in one dimension. At every level, we draw eight sets of references and reassign fitted states to the nearest reference. We compare the resulting grouped distributions at I = { 1 , 2 } I = \{1,2\} I = { 1 , 2 } and I = { 1 , 3 , 6 } I = \{1,3,6\} I = { 1 , 3 , 6 } with the true distributions.
This calculation uses 216 fits from the sample-size experiment, 192 from the emission-geometry experiment, and 144 from the matching experiment with n ≤ 2048 n \le2048 n ≤ 2048 , together with 80 fits from two additional settings described below. The 48 matching fits at n = 8192 n = 8192 n = 8192 are excluded. It contains 632 × 4 × 8 × 2 = 40,448 632 \times4 \times8 \times2 = 40{,}448 632 × 4 × 8 × 2 = 40 , 448 comparisons. Reference perturbations use the same master seed 202610240, task identifiers, and a separate random stream. Table 9 reports three-time recovery; the assignment-change counts are the same for the two-time calculation.
Table 9. Grouped recovery under reference-center perturbations. Oracle and Reference report mean three-time probability errors D { 1 , 3 , 6 } D_{\{1,3,6\}} D { 1 , 3 , 6 } under grouping by true and perturbed centers, respectively, over 5056 reference configurations at each perturbation level. “Changed assignments” counts reference configurations in which at least one fitted state changes groups relative to true-center grouping. The last column gives the largest increase in error caused by changing the grouping, with each fitted HMM held fixed.
The first additional setting varies the initial-state probabilities in a two-state Gaussian HMM with emission centers ( − 1 , 1 ) (-1,1) ( − 1 , 1 ) and transition matrix
A ⋆ = ( 0.8 0.2 0.01 0.99 ) . A^\star=
\begin{pmatrix}
0.8 & 0.2 \\
0.01 & 0.99
\end{pmatrix}. A ⋆ = ( 0.8 0.01 0.2 0.99 ) .
For each ρ ∈ { 0.3 , 0.03 } \rho\in\{0.3,0.03\} ρ ∈ { 0.3 , 0.03 } , we set π ⋆ = ( ρ , 1 − ρ ) \pi^\star= (\rho,1-\rho) π ⋆ = ( ρ , 1 − ρ ) and generate 16 independent data sets, each containing n = 1024 n = 1024 n = 1024 trajectories of length T = 6 T = 6 T = 6 . Fitting a two-state model to each data set gives 32 fits.
The second setting uses three noncollinear emission centers δ ( − 1 , 0 ) \delta(-1,0) δ ( − 1 , 0 ) , δ ( 0 , 0.6 ) \delta(0,0.6) δ ( 0 , 0.6 ) , and δ ( 1 , 0 ) \delta(1,0) δ ( 1 , 0 ) in two dimensions, with identity emission covariance, initial probabilities ( 0.3 , 0.4 , 0.3 ) (0.3,0.4,0.3) ( 0.3 , 0.4 , 0.3 ) , and transition matrix A ⋆ = 0.75 I 3 + 0.2511 T / 3 A^\star= 0.75I_3 + 0.25\mathbf{1}\mathbf{1}^{\mathsf{T}}/3 A ⋆ = 0.75 I 3 + 0.25 1 1 T /3 . For each δ ∈ { 0.4 , 0.8 } \delta\in\{0.4,0.8\} δ ∈ { 0.4 , 0.8 } , we generate 12 independent data sets of n = 512 n = 512 n = 512 trajectories of length T = 6 T = 6 T = 6 and fit models with k = 3 , 4 k = 3,4 k = 3 , 4 to each data set, giving another 48 fits.
All 40,448 comparisons satisfy equation (44) , including configurations where an assignment changes. At τ = 0.10 δ s e p \tau= 0.10\delta_{\mathrm{sep}} τ = 0.10 δ sep , assignments change in 225 of 5056 configurations, increasing to 546 at 0.25 δ s e p 0.25\delta_{\mathrm{sep}} 0.25 δ sep . Mean probability errors change little, although individual increases can be much larger. The result therefore allows assignments to change within the permitted radius while controlling their effect on grouped recovery. Since the references are constructed from known true centers, this experiment evaluates the conditional stability bound rather than the accuracy of an estimated reference-center procedure.